2024-05-28T13:42:27.172 INFO:root:teuthology version: 0.0.1.dev291+ge7ae758 2024-05-28T13:42:27.172 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} --verbose --name rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074 --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074/orig.config.yaml 2024-05-28T13:42:27.217 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-28T13:42:27.306 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074 branch: rishabh-clone-stats description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} email: null first_in_suite: false job_id: '7730074' kernel: flavor: default kdb: 1 sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: rishabh-clone-stats ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 9ed025fbf4406fb54980a0647506b15895b6d863 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 9ed025fbf4406fb54980a0647506b15895b6d863 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: mgr-vol-clone-stats sha1: 9ed025fbf4406fb54980a0647506b15895b6d863 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3033 sha1: 9ed025fbf4406fb54980a0647506b15895b6d863 sleep_before_teardown: 0 suite: fs:volumes suite_branch: mgr-vol-clone-stats suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_9ed025fbf4406fb54980a0647506b15895b6d863/qa suite_relpath: qa suite_repo: https://github.com/rishabh-d-dave/ceph suite_sha1: 9ed025fbf4406fb54980a0647506b15895b6d863 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNWkvlu8UwMHXr28BoHMNn0gWNh333ViF+9vdxWkqBx4QEJHSKz4S8H2bTfeaMYce0qEcJYhRIsKBLt78AcuiC0= smithi063.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPshCWmES4Eoeq1NfxCIk7iNQ+4aMfQ2UavFJQlmGvYH2NXjFeCCU2ESV93e0xy4x+uctvGY/EcCMMppwmATAVU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgressReporter teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e7ae7580c417d6ab27fda98fecadca5d9f85d63e timestamp: 2024-05-28_13:36:18 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-28T13:42:27.307 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_rishabh-d-dave_ceph_9ed025fbf4406fb54980a0647506b15895b6d863/qa; will attempt to use it 2024-05-28T13:42:27.307 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_rishabh-d-dave_ceph_9ed025fbf4406fb54980a0647506b15895b6d863/qa/tasks 2024-05-28T13:42:27.307 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-28T13:42:27.309 INFO:teuthology.task.internal:Checking packages... 2024-05-28T13:42:27.329 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '9ed025fbf4406fb54980a0647506b15895b6d863' 2024-05-28T13:42:27.329 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-28T13:42:27.329 INFO:teuthology.packaging:ref: None 2024-05-28T13:42:27.329 INFO:teuthology.packaging:tag: None 2024-05-28T13:42:27.329 INFO:teuthology.packaging:branch: rishabh-clone-stats 2024-05-28T13:42:27.329 INFO:teuthology.packaging:sha1: 9ed025fbf4406fb54980a0647506b15895b6d863 2024-05-28T13:42:27.329 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=rishabh-clone-stats 2024-05-28T13:42:27.549 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3154-g9ed025fb-1jammy 2024-05-28T13:42:27.551 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-28T13:42:27.557 INFO:teuthology.task.internal:no buildpackages task found 2024-05-28T13:42:27.557 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-28T13:42:27.565 INFO:teuthology.task.internal:Saving configuration 2024-05-28T13:42:27.583 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-28T13:42:27.589 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-28T13:42:27.609 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-28 13:36:55.676084', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBUtq7KEMEWyP8TQSCc+X+sWeLi77nX7wMZjCYXmraLv'} 2024-05-28T13:42:27.629 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-28 13:36:55.674769', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAIXcBx7cF7GBg4dWV9LD5PkWxrSYzKbjSBI7ZRFZ34W'} 2024-05-28T13:42:27.630 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-28T13:42:27.635 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-28T13:42:27.635 INFO:teuthology.task.internal:roles: ubuntu@smithi063.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-28T13:42:27.635 INFO:teuthology.run_tasks:Running task console_log... 2024-05-28T13:42:27.695 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faabdc5fbe0>, signals=[15]) 2024-05-28T13:42:27.695 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-28T13:42:27.708 INFO:teuthology.task.internal:Opening connections... 2024-05-28T13:42:27.708 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T13:42:27.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-28T13:42:27.802 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T13:42:27.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-28T13:42:27.886 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-28T13:42:27.892 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-05-28T13:42:27.897 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-05-28T13:42:27.898 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-05-28T13:42:27.944 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-28T13:42:27.944 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="Ubuntu" 2024-05-28T13:42:27.944 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="22.04" 2024-05-28T13:42:27.944 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-28T13:42:27.944 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_CODENAME=jammy 2024-05-28T13:42:27.944 INFO:teuthology.orchestra.run.smithi042.stdout:ID=ubuntu 2024-05-28T13:42:27.944 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE=debian 2024-05-28T13:42:27.945 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-28T13:42:27.945 INFO:teuthology.orchestra.run.smithi042.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-28T13:42:27.945 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-28T13:42:27.945 INFO:teuthology.orchestra.run.smithi042.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-28T13:42:27.945 INFO:teuthology.orchestra.run.smithi042.stdout:UBUNTU_CODENAME=jammy 2024-05-28T13:42:27.945 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-05-28T13:42:27.969 DEBUG:teuthology.orchestra.run.smithi063:> uname -m 2024-05-28T13:42:27.974 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64 2024-05-28T13:42:27.975 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release 2024-05-28T13:42:28.022 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-28T13:42:28.022 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="Ubuntu" 2024-05-28T13:42:28.022 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="22.04" 2024-05-28T13:42:28.022 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-28T13:42:28.022 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_CODENAME=jammy 2024-05-28T13:42:28.022 INFO:teuthology.orchestra.run.smithi063.stdout:ID=ubuntu 2024-05-28T13:42:28.022 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE=debian 2024-05-28T13:42:28.022 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-28T13:42:28.022 INFO:teuthology.orchestra.run.smithi063.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-28T13:42:28.023 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-28T13:42:28.023 INFO:teuthology.orchestra.run.smithi063.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-28T13:42:28.023 INFO:teuthology.orchestra.run.smithi063.stdout:UBUNTU_CODENAME=jammy 2024-05-28T13:42:28.023 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server 2024-05-28T13:42:28.045 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-28T13:42:28.053 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-28T13:42:28.058 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-28T13:42:28.058 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-05-28T13:42:28.061 DEBUG:teuthology.orchestra.run.smithi063:> test '!' -e /home/ubuntu/cephtest 2024-05-28T13:42:28.069 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-28T13:42:28.075 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-28T13:42:28.075 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-05-28T13:42:28.109 DEBUG:teuthology.orchestra.run.smithi063:> test -z $(ls -A /var/lib/ceph) 2024-05-28T13:42:28.131 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-28T13:42:28.188 INFO:teuthology.run_tasks:Running task kernel... 2024-05-28T13:42:28.203 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'} 2024-05-28T13:42:28.203 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'mon.b': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}}, timeout 300 2024-05-28T13:42:28.203 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-28T13:42:28.204 DEBUG:teuthology.orchestra.run.smithi063:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-28T13:42:28.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T13:42:28.209 DEBUG:teuthology.task.kernel:sha1 for mon.a is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-28T13:42:28.209 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-28T13:42:28.209 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-05-28T13:42:28.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T13:42:28.211 DEBUG:teuthology.task.kernel:sha1 for mon.b is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-28T13:42:28.211 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-28T13:42:28.211 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-05-28T13:42:28.257 INFO:teuthology.orchestra.run.smithi042.stdout:5.15.0-56-generic 2024-05-28T13:42:28.257 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-28T13:42:28.257 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-28T13:42:28.257 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-28T13:42:28.262 INFO:teuthology.orchestra.run.smithi063.stdout:5.15.0-56-generic 2024-05-28T13:42:28.262 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-28T13:42:28.262 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-28T13:42:28.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-28T13:42:28.421 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2024-05-28T13:42:28.421 DEBUG:teuthology.orchestra.run.smithi042:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-05-28T13:42:28.429 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2024-05-28T13:42:28.429 DEBUG:teuthology.orchestra.run.smithi063:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-05-28T13:42:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-05-28T13:42:28.442 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-05-28T13:42:28.565 INFO:teuthology.orchestra.run.smithi063.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-05-28T13:42:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-05-28T13:42:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-05-28T13:42:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:hint: 2024-05-28T13:42:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:hint: git config --global init.defaultBranch 2024-05-28T13:42:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:hint: 2024-05-28T13:42:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-05-28T13:42:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-05-28T13:42:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:hint: 2024-05-28T13:42:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:hint: git branch -m 2024-05-28T13:42:28.567 INFO:teuthology.orchestra.run.smithi063.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-05-28T13:42:28.569 DEBUG:teuthology.orchestra.run.smithi063:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-05-28T13:42:28.574 INFO:teuthology.orchestra.run.smithi042.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-05-28T13:42:28.574 INFO:teuthology.orchestra.run.smithi042.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-05-28T13:42:28.574 INFO:teuthology.orchestra.run.smithi042.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-05-28T13:42:28.574 INFO:teuthology.orchestra.run.smithi042.stderr:hint: 2024-05-28T13:42:28.574 INFO:teuthology.orchestra.run.smithi042.stderr:hint: git config --global init.defaultBranch 2024-05-28T13:42:28.574 INFO:teuthology.orchestra.run.smithi042.stderr:hint: 2024-05-28T13:42:28.575 INFO:teuthology.orchestra.run.smithi042.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-05-28T13:42:28.575 INFO:teuthology.orchestra.run.smithi042.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-05-28T13:42:28.575 INFO:teuthology.orchestra.run.smithi042.stderr:hint: 2024-05-28T13:42:28.575 INFO:teuthology.orchestra.run.smithi042.stderr:hint: git branch -m 2024-05-28T13:42:28.575 INFO:teuthology.orchestra.run.smithi042.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-05-28T13:42:28.578 DEBUG:teuthology.orchestra.run.smithi042:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-05-28T13:42:28.640 DEBUG:teuthology.orchestra.run.smithi063:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-05-28T13:42:28.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-05-28T13:42:28.696 DEBUG:teuthology.orchestra.run.smithi063:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-05-28T13:42:28.702 DEBUG:teuthology.orchestra.run.smithi042:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-05-28T13:43:27.226 INFO:teuthology.orchestra.run.smithi042.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-05-28T13:43:27.226 INFO:teuthology.orchestra.run.smithi042.stderr: * [new branch] main -> origin/main 2024-05-28T13:43:27.261 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20190312 -> 20190312 2024-05-28T13:43:27.261 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20190416 -> 20190416 2024-05-28T13:43:27.261 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20190514 -> 20190514 2024-05-28T13:43:27.262 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20190618 -> 20190618 2024-05-28T13:43:27.262 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20190717 -> 20190717 2024-05-28T13:43:27.262 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20190815 -> 20190815 2024-05-28T13:43:27.263 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20190923 -> 20190923 2024-05-28T13:43:27.263 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20191022 -> 20191022 2024-05-28T13:43:27.263 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20191215 -> 20191215 2024-05-28T13:43:27.263 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20200122 -> 20200122 2024-05-28T13:43:27.263 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20200316 -> 20200316 2024-05-28T13:43:27.263 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20200421 -> 20200421 2024-05-28T13:43:27.263 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20200519 -> 20200519 2024-05-28T13:43:27.263 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20200619 -> 20200619 2024-05-28T13:43:27.263 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20200721 -> 20200721 2024-05-28T13:43:27.265 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20200817 -> 20200817 2024-05-28T13:43:27.265 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20200918 -> 20200918 2024-05-28T13:43:27.265 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20201022 -> 20201022 2024-05-28T13:43:27.265 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20201118 -> 20201118 2024-05-28T13:43:27.265 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20201218 -> 20201218 2024-05-28T13:43:27.265 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20210208 -> 20210208 2024-05-28T13:43:27.265 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20210315 -> 20210315 2024-05-28T13:43:27.266 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20210511 -> 20210511 2024-05-28T13:43:27.266 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20210716 -> 20210716 2024-05-28T13:43:27.266 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20210818 -> 20210818 2024-05-28T13:43:27.266 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20210919 -> 20210919 2024-05-28T13:43:27.266 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20211027 -> 20211027 2024-05-28T13:43:27.266 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20211216 -> 20211216 2024-05-28T13:43:27.266 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20220209 -> 20220209 2024-05-28T13:43:27.266 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20220310 -> 20220310 2024-05-28T13:43:27.266 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20220411 -> 20220411 2024-05-28T13:43:27.268 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20220509 -> 20220509 2024-05-28T13:43:27.268 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20220610 -> 20220610 2024-05-28T13:43:27.269 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20220708 -> 20220708 2024-05-28T13:43:27.269 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20220815 -> 20220815 2024-05-28T13:43:27.269 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20220913 -> 20220913 2024-05-28T13:43:27.269 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20221012 -> 20221012 2024-05-28T13:43:27.269 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20221109 -> 20221109 2024-05-28T13:43:27.269 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20221214 -> 20221214 2024-05-28T13:43:27.269 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20230117 -> 20230117 2024-05-28T13:43:27.269 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20230210 -> 20230210 2024-05-28T13:43:27.269 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20230310 -> 20230310 2024-05-28T13:43:27.269 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20230404 -> 20230404 2024-05-28T13:43:27.269 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20230515 -> 20230515 2024-05-28T13:43:27.269 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20230625 -> 20230625 2024-05-28T13:43:27.269 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20230804 -> 20230804 2024-05-28T13:43:27.269 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20230919 -> 20230919 2024-05-28T13:43:27.270 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20231030 -> 20231030 2024-05-28T13:43:27.270 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20231111 -> 20231111 2024-05-28T13:43:27.270 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20231211 -> 20231211 2024-05-28T13:43:27.270 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20240115 -> 20240115 2024-05-28T13:43:27.270 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20240220 -> 20240220 2024-05-28T13:43:27.270 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20240312 -> 20240312 2024-05-28T13:43:27.270 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20240410 -> 20240410 2024-05-28T13:43:27.270 INFO:teuthology.orchestra.run.smithi042.stderr: * [new tag] 20240513 -> 20240513 2024-05-28T13:43:28.319 INFO:teuthology.orchestra.run.smithi063.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-05-28T13:43:28.319 INFO:teuthology.orchestra.run.smithi063.stderr: * [new branch] main -> origin/main 2024-05-28T13:43:28.355 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20190312 -> 20190312 2024-05-28T13:43:28.355 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20190416 -> 20190416 2024-05-28T13:43:28.355 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20190514 -> 20190514 2024-05-28T13:43:28.355 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20190618 -> 20190618 2024-05-28T13:43:28.355 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20190717 -> 20190717 2024-05-28T13:43:28.356 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20190815 -> 20190815 2024-05-28T13:43:28.357 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20190923 -> 20190923 2024-05-28T13:43:28.357 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20191022 -> 20191022 2024-05-28T13:43:28.357 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20191215 -> 20191215 2024-05-28T13:43:28.357 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20200122 -> 20200122 2024-05-28T13:43:28.357 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20200316 -> 20200316 2024-05-28T13:43:28.357 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20200421 -> 20200421 2024-05-28T13:43:28.357 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20200519 -> 20200519 2024-05-28T13:43:28.358 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20200619 -> 20200619 2024-05-28T13:43:28.358 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20200721 -> 20200721 2024-05-28T13:43:28.358 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20200817 -> 20200817 2024-05-28T13:43:28.363 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20200918 -> 20200918 2024-05-28T13:43:28.363 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20201022 -> 20201022 2024-05-28T13:43:28.363 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20201118 -> 20201118 2024-05-28T13:43:28.364 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20201218 -> 20201218 2024-05-28T13:43:28.364 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20210208 -> 20210208 2024-05-28T13:43:28.364 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20210315 -> 20210315 2024-05-28T13:43:28.364 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20210511 -> 20210511 2024-05-28T13:43:28.364 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20210716 -> 20210716 2024-05-28T13:43:28.364 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20210818 -> 20210818 2024-05-28T13:43:28.364 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20210919 -> 20210919 2024-05-28T13:43:28.364 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20211027 -> 20211027 2024-05-28T13:43:28.364 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20211216 -> 20211216 2024-05-28T13:43:28.364 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20220209 -> 20220209 2024-05-28T13:43:28.364 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20220310 -> 20220310 2024-05-28T13:43:28.364 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20220411 -> 20220411 2024-05-28T13:43:28.364 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20220509 -> 20220509 2024-05-28T13:43:28.364 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20220610 -> 20220610 2024-05-28T13:43:28.365 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20220708 -> 20220708 2024-05-28T13:43:28.365 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20220815 -> 20220815 2024-05-28T13:43:28.365 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20220913 -> 20220913 2024-05-28T13:43:28.365 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20221012 -> 20221012 2024-05-28T13:43:28.365 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20221109 -> 20221109 2024-05-28T13:43:28.365 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20221214 -> 20221214 2024-05-28T13:43:28.365 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20230117 -> 20230117 2024-05-28T13:43:28.365 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20230210 -> 20230210 2024-05-28T13:43:28.365 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20230310 -> 20230310 2024-05-28T13:43:28.365 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20230404 -> 20230404 2024-05-28T13:43:28.365 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20230515 -> 20230515 2024-05-28T13:43:28.365 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20230625 -> 20230625 2024-05-28T13:43:28.365 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20230804 -> 20230804 2024-05-28T13:43:28.365 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20230919 -> 20230919 2024-05-28T13:43:28.366 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20231030 -> 20231030 2024-05-28T13:43:28.366 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20231111 -> 20231111 2024-05-28T13:43:28.366 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20231211 -> 20231211 2024-05-28T13:43:28.366 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20240115 -> 20240115 2024-05-28T13:43:28.366 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20240220 -> 20240220 2024-05-28T13:43:28.366 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20240312 -> 20240312 2024-05-28T13:43:28.366 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20240410 -> 20240410 2024-05-28T13:43:28.366 INFO:teuthology.orchestra.run.smithi063.stderr: * [new tag] 20240513 -> 20240513 2024-05-28T13:43:31.824 INFO:teuthology.orchestra.run.smithi042.stderr:Updating files: 43% (1481/3422) Updating files: 44% (1506/3422) Updating files: 45% (1540/3422) Updating files: 46% (1575/3422) Updating files: 47% (1609/3422) Updating files: 48% (1643/3422) Updating files: 49% (1677/3422) Updating files: 50% (1711/3422) Updating files: 51% (1746/3422) Updating files: 52% (1780/3422) Updating files: 53% (1814/3422) Updating files: 54% (1848/3422) Updating files: 55% (1883/3422) Updating files: 56% (1917/3422) Updating files: 57% (1951/3422) Updating files: 58% (1985/3422) Updating files: 58% (2009/3422) Updating files: 59% (2019/3422) Updating files: 60% (2054/3422) Updating files: 61% (2088/3422) Updating files: 62% (2122/3422) Updating files: 63% (2156/3422) Updating files: 64% (2191/3422) Updating files: 65% (2225/3422) Updating files: 66% (2259/3422) Updating files: 67% (2293/3422) Updating files: 68% (2327/3422) Updating files: 69% (2362/3422) Updating files: 69% (2364/3422) Updating files: 70% (2396/3422) Updating files: 71% (2430/3422) Updating files: 72% (2464/3422) Updating files: 73% (2499/3422) Updating files: 74% (2533/3422) Updating files: 75% (2567/3422) Updating files: 76% (2601/3422) Updating files: 77% (2635/3422) Updating files: 78% (2670/3422) Updating files: 79% (2704/3422) Updating files: 80% (2738/3422) Updating files: 81% (2772/3422) Updating files: 82% (2807/3422) Updating files: 82% (2818/3422) Updating files: 83% (2841/3422) Updating files: 84% (2875/3422) Updating files: 85% (2909/3422) Updating files: 86% (2943/3422) Updating files: 87% (2978/3422) Updating files: 88% (3012/3422) Updating files: 89% (3046/3422) Updating files: 90% (3080/3422) Updating files: 91% (3115/3422) Updating files: 92% (3149/3422) Updating files: 93% (3183/3422) Updating files: 94% (3217/3422) Updating files: 95% (3251/3422) Updating files: 96% (3286/3422) Updating files: 97% (3320/3422) Updating files: 98% (3354/3422) Updating files: 99% (3388/3422) Updating files: 100% (3422/3422) Updating files: 100% (3422/3422), done. 2024-05-28T13:43:31.829 INFO:teuthology.orchestra.run.smithi042.stdout:HEAD is now at 99c0ff00 Merge branch 'robot/patch-0-1715993176' into 'main' 2024-05-28T13:43:31.852 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.a... 2024-05-28T13:43:31.853 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-28T13:43:32.079 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-gb9459a26b088/ 2024-05-28T13:43:32.080 DEBUG:teuthology.orchestra.run.smithi042:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-gb9459a26b088_6.9.0-gb9459a26b088-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-gb9459a26b088/ --input-file=- 2024-05-28T13:43:32.083 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi042.front.sepia.ceph.com to complete... 2024-05-28T13:43:32.906 INFO:teuthology.orchestra.run.smithi063.stderr:Updating files: 44% (1524/3422) Updating files: 45% (1540/3422) Updating files: 46% (1575/3422) Updating files: 47% (1609/3422) Updating files: 48% (1643/3422) Updating files: 49% (1677/3422) Updating files: 50% (1711/3422) Updating files: 51% (1746/3422) Updating files: 52% (1780/3422) Updating files: 53% (1814/3422) Updating files: 54% (1848/3422) Updating files: 55% (1883/3422) Updating files: 56% (1917/3422) Updating files: 57% (1951/3422) Updating files: 58% (1985/3422) Updating files: 58% (2013/3422) Updating files: 59% (2019/3422) Updating files: 60% (2054/3422) Updating files: 61% (2088/3422) Updating files: 62% (2122/3422) Updating files: 63% (2156/3422) Updating files: 64% (2191/3422) Updating files: 65% (2225/3422) Updating files: 66% (2259/3422) Updating files: 67% (2293/3422) Updating files: 68% (2327/3422) Updating files: 69% (2362/3422) Updating files: 69% (2367/3422) Updating files: 70% (2396/3422) Updating files: 71% (2430/3422) Updating files: 72% (2464/3422) Updating files: 73% (2499/3422) Updating files: 74% (2533/3422) Updating files: 75% (2567/3422) Updating files: 76% (2601/3422) Updating files: 77% (2635/3422) Updating files: 78% (2670/3422) Updating files: 79% (2704/3422) Updating files: 80% (2738/3422) Updating files: 81% (2772/3422) Updating files: 82% (2807/3422) Updating files: 82% (2820/3422) Updating files: 83% (2841/3422) Updating files: 84% (2875/3422) Updating files: 85% (2909/3422) Updating files: 86% (2943/3422) Updating files: 87% (2978/3422) Updating files: 88% (3012/3422) Updating files: 89% (3046/3422) Updating files: 90% (3080/3422) Updating files: 91% (3115/3422) Updating files: 92% (3149/3422) Updating files: 93% (3183/3422) Updating files: 94% (3217/3422) Updating files: 95% (3251/3422) Updating files: 96% (3286/3422) Updating files: 97% (3320/3422) Updating files: 98% (3354/3422) Updating files: 99% (3388/3422) Updating files: 100% (3422/3422) Updating files: 100% (3422/3422), done. 2024-05-28T13:43:32.911 INFO:teuthology.orchestra.run.smithi063.stdout:HEAD is now at 99c0ff00 Merge branch 'robot/patch-0-1715993176' into 'main' 2024-05-28T13:43:32.934 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.b... 2024-05-28T13:43:32.934 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-28T13:43:33.153 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-gb9459a26b088/ 2024-05-28T13:43:33.153 DEBUG:teuthology.orchestra.run.smithi063:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-gb9459a26b088_6.9.0-gb9459a26b088-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-gb9459a26b088/ --input-file=- 2024-05-28T13:43:33.156 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi063.front.sepia.ceph.com to complete... 2024-05-28T13:43:33.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28 13:43:34 URL:https://2.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-gb9459a26b088/linux-image-6.9.0-gb9459a26b088_6.9.0-gb9459a26b088-1_amd64.deb [44988888/44988888] -> "/tmp/linux-image.deb" [1] 2024-05-28T13:43:33.842 INFO:teuthology.orchestra.run.smithi042.stderr:FINISHED --2024-05-28 13:43:34-- 2024-05-28T13:43:33.842 INFO:teuthology.orchestra.run.smithi042.stderr:Total wall clock time: 1.7s 2024-05-28T13:43:33.842 INFO:teuthology.orchestra.run.smithi042.stderr:Downloaded: 1 files, 43M in 1.6s (27.3 MB/s) 2024-05-28T13:43:33.846 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.a... 2024-05-28T13:43:33.846 DEBUG:teuthology.orchestra.run.smithi042:> sudo dpkg -i /tmp/linux-image.deb 2024-05-28T13:43:34.105 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package linux-image-6.9.0-gb9459a26b088. 2024-05-28T13:43:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-05-28T13:43:36.257 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-05-28T13:43:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-image-6.9.0-gb9459a26b088 (6.9.0-gb9459a26b088-1) ... 2024-05-28T13:43:36.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28 13:43:36 URL:https://2.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-gb9459a26b088/linux-image-6.9.0-gb9459a26b088_6.9.0-gb9459a26b088-1_amd64.deb [44988888/44988888] -> "/tmp/linux-image.deb" [1] 2024-05-28T13:43:36.631 INFO:teuthology.orchestra.run.smithi063.stderr:FINISHED --2024-05-28 13:43:36-- 2024-05-28T13:43:36.631 INFO:teuthology.orchestra.run.smithi063.stderr:Total wall clock time: 3.4s 2024-05-28T13:43:36.632 INFO:teuthology.orchestra.run.smithi063.stderr:Downloaded: 1 files, 43M in 3.1s (13.8 MB/s) 2024-05-28T13:43:36.635 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.b... 2024-05-28T13:43:36.636 DEBUG:teuthology.orchestra.run.smithi063:> sudo dpkg -i /tmp/linux-image.deb 2024-05-28T13:43:36.876 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package linux-image-6.9.0-gb9459a26b088. 2024-05-28T13:43:38.809 INFO:teuthology.orchestra.run.smithi063.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-05-28T13:43:38.814 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-05-28T13:43:38.977 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-image-6.9.0-gb9459a26b088 (6.9.0-gb9459a26b088-1) ... 2024-05-28T13:43:40.597 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-image-6.9.0-gb9459a26b088 (6.9.0-gb9459a26b088-1) ... 2024-05-28T13:43:40.717 INFO:teuthology.orchestra.run.smithi042.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-gb9459a26b088 2024-05-28T13:43:42.580 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-image-6.9.0-gb9459a26b088 (6.9.0-gb9459a26b088-1) ... 2024-05-28T13:43:42.683 INFO:teuthology.orchestra.run.smithi063.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-gb9459a26b088 2024-05-28T13:44:03.902 INFO:teuthology.orchestra.run.smithi042.stderr:Sourcing file `/etc/default/grub' 2024-05-28T13:44:03.922 INFO:teuthology.orchestra.run.smithi042.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-28T13:44:03.948 INFO:teuthology.orchestra.run.smithi042.stderr:Generating grub configuration file ... 2024-05-28T13:44:04.302 INFO:teuthology.orchestra.run.smithi042.stderr:Found linux image: /boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-28T13:44:04.319 INFO:teuthology.orchestra.run.smithi042.stderr:Found initrd image: /boot/initrd.img-6.9.0-gb9459a26b088 2024-05-28T13:44:04.633 INFO:teuthology.orchestra.run.smithi042.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-28T13:44:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-28T13:44:04.707 INFO:teuthology.orchestra.run.smithi042.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-28T13:44:04.713 INFO:teuthology.orchestra.run.smithi042.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-28T13:44:04.870 INFO:teuthology.orchestra.run.smithi042.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-28T13:44:04.870 INFO:teuthology.orchestra.run.smithi042.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-28T13:44:04.870 INFO:teuthology.orchestra.run.smithi042.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-28T13:44:04.910 INFO:teuthology.orchestra.run.smithi042.stderr:done 2024-05-28T13:44:05.104 DEBUG:teuthology.orchestra.run.smithi042:> dpkg-deb -c /tmp/linux-image.deb 2024-05-28T13:44:05.112 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./ 2024-05-28T13:44:05.112 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./boot/ 2024-05-28T13:44:05.112 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7004139 2024-05-23 11:07 ./boot/System.map-6.9.0-gb9459a26b088 2024-05-28T13:44:05.127 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 172251 2024-05-23 11:07 ./boot/config-6.9.0-gb9459a26b088 2024-05-28T13:44:05.127 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13136896 2024-05-23 11:07 ./boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-28T13:44:05.147 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./etc/ 2024-05-28T13:44:05.147 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./etc/kernel/ 2024-05-28T13:44:05.147 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./etc/kernel/postinst.d/ 2024-05-28T13:44:05.147 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./etc/kernel/postrm.d/ 2024-05-28T13:44:05.147 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./etc/kernel/preinst.d/ 2024-05-28T13:44:05.148 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./etc/kernel/prerm.d/ 2024-05-28T13:44:05.148 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/ 2024-05-28T13:44:05.148 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/ 2024-05-28T13:44:05.148 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/ 2024-05-28T13:44:05.148 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/ 2024-05-28T13:44:05.148 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/ 2024-05-28T13:44:05.148 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/ 2024-05-28T13:44:05.148 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ 2024-05-28T13:44:05.148 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 507497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/aesni-intel.ko 2024-05-28T13:44:05.148 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-28T13:44:05.148 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-28T13:44:05.148 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-28T13:44:05.148 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 97945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-28T13:44:05.149 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 109897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-28T13:44:05.149 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 155241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-28T13:44:05.151 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-28T13:44:05.151 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-28T13:44:05.151 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 87001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-28T13:44:05.151 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-28T13:44:05.151 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 50297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-28T13:44:05.151 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 50369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-28T13:44:05.152 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 54505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-28T13:44:05.152 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-28T13:44:05.152 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 48041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-28T13:44:05.152 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 62865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-28T13:44:05.152 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 53561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-28T13:44:05.152 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-28T13:44:05.152 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-28T13:44:05.152 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/ 2024-05-28T13:44:05.152 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/ 2024-05-28T13:44:05.152 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/ 2024-05-28T13:44:05.152 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 49073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-28T13:44:05.152 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpuid.ko 2024-05-28T13:44:05.152 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/msr.ko 2024-05-28T13:44:05.153 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/ 2024-05-28T13:44:05.153 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 443289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-28T13:44:05.153 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1037081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-28T13:44:05.153 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 2850153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-28T13:44:05.229 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ 2024-05-28T13:44:05.229 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 65017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/af_alg.ko 2024-05-28T13:44:05.229 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/algif_hash.ko 2024-05-28T13:44:05.229 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/algif_skcipher.ko 2024-05-28T13:44:05.229 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-28T13:44:05.229 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-28T13:44:05.229 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-28T13:44:05.229 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/asymmetric_keys/ 2024-05-28T13:44:05.229 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-05-28T13:44:05.229 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/ 2024-05-28T13:44:05.229 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-28T13:44:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-28T13:44:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-28T13:44:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-28T13:44:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-28T13:44:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-28T13:44:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-28T13:44:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-28T13:44:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-28T13:44:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-28T13:44:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-28T13:44:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-28T13:44:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-28T13:44:05.230 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-28T13:44:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-28T13:44:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-28T13:44:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-28T13:44:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-28T13:44:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 59121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cryptd.ko 2024-05-28T13:44:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 48105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-28T13:44:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_simd.ko 2024-05-28T13:44:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-28T13:44:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-28T13:44:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-28T13:44:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-28T13:44:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-28T13:44:05.231 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-28T13:44:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-28T13:44:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-28T13:44:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lrw.ko 2024-05-28T13:44:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-28T13:44:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-28T13:44:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-28T13:44:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-28T13:44:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-28T13:44:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-28T13:44:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-28T13:44:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-28T13:44:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-28T13:44:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 169057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-28T13:44:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-28T13:44:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-28T13:44:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-28T13:44:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-28T13:44:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-28T13:44:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-28T13:44:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-28T13:44:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xts.ko 2024-05-28T13:44:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-28T13:44:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ 2024-05-28T13:44:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ 2024-05-28T13:44:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_extlog.ko 2024-05-28T13:44:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-28T13:44:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-28T13:44:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/apei/ 2024-05-28T13:44:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/apei/einj.ko 2024-05-28T13:44:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-28T13:44:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-28T13:44:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-28T13:44:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-28T13:44:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 119017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-28T13:44:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ 2024-05-28T13:44:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-28T13:44:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 99177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci.ko 2024-05-28T13:44:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-28T13:44:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 107729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci.ko 2024-05-28T13:44:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-28T13:44:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-28T13:44:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-28T13:44:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-28T13:44:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-28T13:44:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-28T13:44:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-28T13:44:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-28T13:44:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-28T13:44:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-28T13:44:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-28T13:44:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-28T13:44:05.235 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 60249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_legacy.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-28T13:44:05.236 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-28T13:44:05.237 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-28T13:44:05.237 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-28T13:44:05.237 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-28T13:44:05.237 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-28T13:44:05.237 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_radisys.ko 2024-05-28T13:44:05.237 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-28T13:44:05.237 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rz1000.ko 2024-05-28T13:44:05.237 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-28T13:44:05.237 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-28T13:44:05.237 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-28T13:44:05.237 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-28T13:44:05.237 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-28T13:44:05.237 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-28T13:44:05.238 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-28T13:44:05.238 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-28T13:44:05.238 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 102113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-28T13:44:05.238 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 85401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-28T13:44:05.238 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-28T13:44:05.238 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-28T13:44:05.238 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-28T13:44:05.238 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 45897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-28T13:44:05.238 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-28T13:44:05.238 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-28T13:44:05.239 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 53345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-28T13:44:05.239 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-28T13:44:05.239 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-28T13:44:05.239 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-28T13:44:05.239 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ 2024-05-28T13:44:05.239 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-28T13:44:05.239 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/ 2024-05-28T13:44:05.239 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 163833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-28T13:44:05.239 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/ 2024-05-28T13:44:05.239 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/ 2024-05-28T13:44:05.239 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 140833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-28T13:44:05.239 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/ 2024-05-28T13:44:05.240 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1241585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-28T13:44:05.240 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 203281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-28T13:44:05.240 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/ 2024-05-28T13:44:05.240 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 115257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-28T13:44:05.240 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 143425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-28T13:44:05.240 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/ 2024-05-28T13:44:05.240 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 126609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-28T13:44:05.240 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 119865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-28T13:44:05.240 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 313209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-28T13:44:05.240 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/ 2024-05-28T13:44:05.241 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 117217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-28T13:44:05.241 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/ 2024-05-28T13:44:05.241 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 70369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-28T13:44:05.241 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ 2024-05-28T13:44:05.241 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/applicom.ko 2024-05-28T13:44:05.241 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-28T13:44:05.241 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/ 2024-05-28T13:44:05.241 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-28T13:44:05.241 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-28T13:44:05.241 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-28T13:44:05.241 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-28T13:44:05.242 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-28T13:44:05.242 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ 2024-05-28T13:44:05.242 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 37185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-28T13:44:05.242 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 186913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-28T13:44:05.242 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-28T13:44:05.242 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 179441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-28T13:44:05.242 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 85433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-28T13:44:05.242 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 65305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-28T13:44:05.242 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-28T13:44:05.242 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/ 2024-05-28T13:44:05.242 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 103385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-28T13:44:05.242 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/nvram.ko 2024-05-28T13:44:05.243 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-28T13:44:05.243 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 59233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-28T13:44:05.243 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/ 2024-05-28T13:44:05.243 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-28T13:44:05.243 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-05-28T13:44:05.243 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-05-28T13:44:05.243 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 39185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-05-28T13:44:05.243 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-28T13:44:05.243 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-28T13:44:05.243 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-05-28T13:44:05.243 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/xillybus/ 2024-05-28T13:44:05.243 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/xillybus/xillybus_class.ko 2024-05-28T13:44:05.244 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 62537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/xillybus/xillybus_core.ko 2024-05-28T13:44:05.244 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-05-28T13:44:05.244 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/clk/ 2024-05-28T13:44:05.244 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/clk/clk-palmas.ko 2024-05-28T13:44:05.244 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 57281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/clk/clk-si5351.ko 2024-05-28T13:44:05.244 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/clk/clk-twl6040.ko 2024-05-28T13:44:05.244 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/clk/clk-wm831x.ko 2024-05-28T13:44:05.244 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/ 2024-05-28T13:44:05.244 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-28T13:44:05.244 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-28T13:44:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-28T13:44:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ 2024-05-28T13:44:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ 2024-05-28T13:44:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 89209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-28T13:44:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 333593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-28T13:44:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/ 2024-05-28T13:44:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/ 2024-05-28T13:44:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/ 2024-05-28T13:44:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 751433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-28T13:44:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-05-28T13:44:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-28T13:44:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-28T13:44:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-28T13:44:05.246 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/ 2024-05-28T13:44:05.246 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 90465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-28T13:44:05.246 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/ 2024-05-28T13:44:05.246 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-28T13:44:05.246 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ 2024-05-28T13:44:05.246 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/ 2024-05-28T13:44:05.246 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-28T13:44:05.246 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ 2024-05-28T13:44:05.246 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 146745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-28T13:44:05.246 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ 2024-05-28T13:44:05.246 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 96137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-28T13:44:05.246 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-28T13:44:05.246 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 51097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-28T13:44:05.247 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-28T13:44:05.247 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-28T13:44:05.247 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-28T13:44:05.247 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-28T13:44:05.247 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-28T13:44:05.247 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 64153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-28T13:44:05.247 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-28T13:44:05.247 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-28T13:44:05.247 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 68689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-28T13:44:05.247 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-28T13:44:05.247 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/ 2024-05-28T13:44:05.247 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/extcon-gpio.ko 2024-05-28T13:44:05.248 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/extcon-max14577.ko 2024-05-28T13:44:05.248 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/extcon-max77693.ko 2024-05-28T13:44:05.248 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/extcon-max8997.ko 2024-05-28T13:44:05.248 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/extcon-palmas.ko 2024-05-28T13:44:05.248 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/extcon-rt8973a.ko 2024-05-28T13:44:05.248 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/extcon-sm5502.ko 2024-05-28T13:44:05.248 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/ 2024-05-28T13:44:05.248 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/dmi-sysfs.ko 2024-05-28T13:44:05.248 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/efi/ 2024-05-28T13:44:05.248 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/efi/efi-pstore.ko 2024-05-28T13:44:05.248 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-28T13:44:05.248 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/ 2024-05-28T13:44:05.248 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-adp5520.ko 2024-05-28T13:44:05.249 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-amd8111.ko 2024-05-28T13:44:05.249 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-arizona.ko 2024-05-28T13:44:05.249 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-da9052.ko 2024-05-28T13:44:05.249 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-da9055.ko 2024-05-28T13:44:05.249 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-dln2.ko 2024-05-28T13:44:05.249 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 81065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-f7188x.ko 2024-05-28T13:44:05.249 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-generic.ko 2024-05-28T13:44:05.249 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-28T13:44:05.249 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-05-28T13:44:05.250 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-kempld.ko 2024-05-28T13:44:05.250 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-lp3943.ko 2024-05-28T13:44:05.250 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-max7300.ko 2024-05-28T13:44:05.250 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-max730x.ko 2024-05-28T13:44:05.250 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-max732x.ko 2024-05-28T13:44:05.250 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-05-28T13:44:05.250 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 55497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-pca953x.ko 2024-05-28T13:44:05.250 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-pcf857x.ko 2024-05-28T13:44:05.250 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-rdc321x.ko 2024-05-28T13:44:05.250 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-sch.ko 2024-05-28T13:44:05.250 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-sch311x.ko 2024-05-28T13:44:05.250 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-tps65912.ko 2024-05-28T13:44:05.251 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-twl4030.ko 2024-05-28T13:44:05.251 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-twl6040.ko 2024-05-28T13:44:05.251 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-28T13:44:05.251 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-vx855.ko 2024-05-28T13:44:05.251 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-wm831x.ko 2024-05-28T13:44:05.251 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-wm8350.ko 2024-05-28T13:44:05.251 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-wm8994.ko 2024-05-28T13:44:05.251 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/ 2024-05-28T13:44:05.251 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-a4tech.ko 2024-05-28T13:44:05.251 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-apple.ko 2024-05-28T13:44:05.251 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-28T13:44:05.251 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-28T13:44:05.251 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-28T13:44:05.251 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-belkin.ko 2024-05-28T13:44:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-cherry.ko 2024-05-28T13:44:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-chicony.ko 2024-05-28T13:44:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 74385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-cp2112.ko 2024-05-28T13:44:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-cypress.ko 2024-05-28T13:44:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-28T13:44:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-28T13:44:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-28T13:44:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-28T13:44:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ezkey.ko 2024-05-28T13:44:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-28T13:44:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-generic.ko 2024-05-28T13:44:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-28T13:44:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-28T13:44:05.253 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-28T13:44:05.253 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-28T13:44:05.253 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-28T13:44:05.253 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-28T13:44:05.253 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-28T13:44:05.253 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kensington.ko 2024-05-28T13:44:05.253 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-28T13:44:05.253 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-28T13:44:05.253 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-28T13:44:05.253 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-28T13:44:05.253 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 54209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-28T13:44:05.253 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 37545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lg-g15.ko 2024-05-28T13:44:05.253 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 67961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-28T13:44:05.254 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 121361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-28T13:44:05.254 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 97929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech.ko 2024-05-28T13:44:05.254 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-magicmouse.ko 2024-05-28T13:44:05.254 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-microsoft.ko 2024-05-28T13:44:05.254 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-monterey.ko 2024-05-28T13:44:05.254 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 58993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-28T13:44:05.254 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 39689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ntrig.ko 2024-05-28T13:44:05.254 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-28T13:44:05.254 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-28T13:44:05.254 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-28T13:44:05.254 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 91217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-28T13:44:05.254 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-28T13:44:05.255 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-28T13:44:05.255 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-28T13:44:05.255 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-28T13:44:05.255 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-28T13:44:05.255 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-28T13:44:05.255 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-28T13:44:05.255 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-28T13:44:05.255 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-28T13:44:05.255 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-28T13:44:05.255 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-28T13:44:05.255 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-28T13:44:05.255 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-28T13:44:05.255 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-28T13:44:05.256 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-28T13:44:05.256 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-28T13:44:05.256 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-28T13:44:05.256 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-28T13:44:05.256 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-28T13:44:05.256 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-28T13:44:05.256 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 62121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-28T13:44:05.256 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-28T13:44:05.256 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-28T13:44:05.256 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-28T13:44:05.257 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-28T13:44:05.257 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-28T13:44:05.257 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-28T13:44:05.257 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-28T13:44:05.257 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-28T13:44:05.257 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 81097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-28T13:44:05.257 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-28T13:44:05.257 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 184729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-28T13:44:05.257 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-28T13:44:05.257 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-28T13:44:05.257 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-28T13:44:05.258 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 319897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid.ko 2024-05-28T13:44:05.258 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 39521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-28T13:44:05.258 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/usbhid/ 2024-05-28T13:44:05.258 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 153617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/usbhid/usbhid.ko 2024-05-28T13:44:05.258 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/usbhid/usbkbd.ko 2024-05-28T13:44:05.258 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/usbhid/usbmouse.ko 2024-05-28T13:44:05.258 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 261929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-28T13:44:05.258 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/ 2024-05-28T13:44:05.258 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 75057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-28T13:44:05.258 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 110657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-28T13:44:05.258 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 381625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-28T13:44:05.258 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ 2024-05-28T13:44:05.259 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 77145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-28T13:44:05.259 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 67001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-28T13:44:05.259 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-28T13:44:05.259 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-28T13:44:05.259 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-28T13:44:05.259 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-28T13:44:05.259 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-28T13:44:05.259 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 115049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-28T13:44:05.259 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-28T13:44:05.259 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 55217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-28T13:44:05.259 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-28T13:44:05.259 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-28T13:44:05.260 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-28T13:44:05.260 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-28T13:44:05.260 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 89017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-28T13:44:05.260 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 49145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-28T13:44:05.260 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 81465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-28T13:44:05.260 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-28T13:44:05.260 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 48169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-28T13:44:05.260 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 68857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-28T13:44:05.260 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 60185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-28T13:44:05.260 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 69001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-28T13:44:05.260 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 47969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-28T13:44:05.261 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-28T13:44:05.261 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-28T13:44:05.261 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/da9052-hwmon.ko 2024-05-28T13:44:05.261 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/da9055-hwmon.ko 2024-05-28T13:44:05.261 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 55705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-28T13:44:05.261 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 113129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-28T13:44:05.261 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-28T13:44:05.261 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-28T13:44:05.261 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-28T13:44:05.261 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc2103.ko 2024-05-28T13:44:05.261 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-28T13:44:05.261 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 78913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-28T13:44:05.261 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 108953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-28T13:44:05.262 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 50689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-28T13:44:05.262 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-28T13:44:05.262 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 65761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-28T13:44:05.262 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-28T13:44:05.262 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-28T13:44:05.262 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 59753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-28T13:44:05.262 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 56305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-28T13:44:05.262 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hih6130.ko 2024-05-28T13:44:05.262 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-28T13:44:05.262 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-28T13:44:05.262 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-28T13:44:05.262 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 156457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/jc42.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 57761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 37041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-28T13:44:05.263 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-28T13:44:05.264 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 86489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-28T13:44:05.264 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 55633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-28T13:44:05.264 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 76481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-28T13:44:05.264 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-28T13:44:05.264 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 131497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-28T13:44:05.264 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-28T13:44:05.264 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-28T13:44:05.264 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-28T13:44:05.264 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-28T13:44:05.264 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-28T13:44:05.264 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-28T13:44:05.264 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-28T13:44:05.264 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-28T13:44:05.264 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-28T13:44:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-28T13:44:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-28T13:44:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-28T13:44:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-28T13:44:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-28T13:44:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-28T13:44:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-28T13:44:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 39721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-28T13:44:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mc13783-adc.ko 2024-05-28T13:44:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-28T13:44:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-05-28T13:44:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 56497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-28T13:44:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 141345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-28T13:44:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 72729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-28T13:44:05.266 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 67769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-28T13:44:05.266 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 97769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-28T13:44:05.266 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 61913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-28T13:44:05.266 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-28T13:44:05.266 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ 2024-05-28T13:44:05.266 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-28T13:44:05.266 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-28T13:44:05.266 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-28T13:44:05.266 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-28T13:44:05.266 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-28T13:44:05.267 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-28T13:44:05.267 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-28T13:44:05.267 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 101225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-28T13:44:05.267 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-28T13:44:05.267 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-28T13:44:05.267 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-28T13:44:05.267 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-28T13:44:05.268 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-28T13:44:05.268 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-28T13:44:05.268 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-28T13:44:05.268 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-28T13:44:05.268 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-28T13:44:05.268 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-28T13:44:05.269 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-28T13:44:05.269 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 48705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-28T13:44:05.269 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-28T13:44:05.269 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 46465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-28T13:44:05.269 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-28T13:44:05.269 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-28T13:44:05.269 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-28T13:44:05.270 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-28T13:44:05.270 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-28T13:44:05.270 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-28T13:44:05.270 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-28T13:44:05.270 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 50209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-28T13:44:05.270 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 59657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-28T13:44:05.271 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 62729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-28T13:44:05.271 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 71161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-28T13:44:05.271 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 96569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-28T13:44:05.271 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 85905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-28T13:44:05.271 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 75121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-28T13:44:05.271 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 83417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-28T13:44:05.272 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 116769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-28T13:44:05.272 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 134433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-28T13:44:05.272 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-28T13:44:05.272 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-28T13:44:05.272 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-05-28T13:44:05.272 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-05-28T13:44:05.273 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/ 2024-05-28T13:44:05.273 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/ 2024-05-28T13:44:05.273 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-28T13:44:05.273 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-28T13:44:05.273 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/ 2024-05-28T13:44:05.273 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-05-28T13:44:05.274 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-05-28T13:44:05.274 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-05-28T13:44:05.274 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-28T13:44:05.274 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-28T13:44:05.274 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-28T13:44:05.275 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-05-28T13:44:05.275 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-28T13:44:05.275 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-05-28T13:44:05.275 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 64217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-28T13:44:05.275 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-28T13:44:05.275 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-28T13:44:05.276 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-28T13:44:05.276 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-05-28T13:44:05.276 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-05-28T13:44:05.276 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 76961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-28T13:44:05.276 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-28T13:44:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 52929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-28T13:44:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-05-28T13:44:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-28T13:44:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-28T13:44:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-05-28T13:44:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-28T13:44:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-28T13:44:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 57209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-28T13:44:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-05-28T13:44:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-28T13:44:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-28T13:44:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-05-28T13:44:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-05-28T13:44:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-28T13:44:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-05-28T13:44:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-28T13:44:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-28T13:44:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-28T13:44:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-28T13:44:05.280 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 59121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-05-28T13:44:05.280 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-28T13:44:05.280 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-28T13:44:05.280 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-28T13:44:05.280 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/muxes/ 2024-05-28T13:44:05.281 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-05-28T13:44:05.281 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-05-28T13:44:05.281 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-05-28T13:44:05.281 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ 2024-05-28T13:44:05.281 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ 2024-05-28T13:44:05.281 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 329681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-28T13:44:05.282 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1119785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-28T13:44:05.282 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 83897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-28T13:44:05.282 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 408329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-28T13:44:05.282 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 120905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-28T13:44:05.282 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 347201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-28T13:44:05.282 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 84937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-28T13:44:05.283 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/ 2024-05-28T13:44:05.283 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/ 2024-05-28T13:44:05.283 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 590873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-28T13:44:05.283 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/ 2024-05-28T13:44:05.283 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 514577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-28T13:44:05.284 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/ 2024-05-28T13:44:05.284 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 911649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-28T13:44:05.284 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ 2024-05-28T13:44:05.284 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 375449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-28T13:44:05.284 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/ocrdma/ 2024-05-28T13:44:05.284 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 197097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-05-28T13:44:05.285 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/usnic/ 2024-05-28T13:44:05.285 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 212497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-05-28T13:44:05.285 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ 2024-05-28T13:44:05.285 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ 2024-05-28T13:44:05.285 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 334353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-28T13:44:05.286 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ 2024-05-28T13:44:05.286 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 120649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-28T13:44:05.286 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ 2024-05-28T13:44:05.286 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 132577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-28T13:44:05.286 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ 2024-05-28T13:44:05.286 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 157929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-28T13:44:05.287 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ 2024-05-28T13:44:05.287 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 156665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-28T13:44:05.287 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/ 2024-05-28T13:44:05.287 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/evbug.ko 2024-05-28T13:44:05.287 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/ff-memless.ko 2024-05-28T13:44:05.288 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/ 2024-05-28T13:44:05.288 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-28T13:44:05.288 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-28T13:44:05.288 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 50705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-28T13:44:05.288 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 49433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-28T13:44:05.289 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/ 2024-05-28T13:44:05.289 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-05-28T13:44:05.289 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-05-28T13:44:05.289 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-05-28T13:44:05.289 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-05-28T13:44:05.289 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 45465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-28T13:44:05.290 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-05-28T13:44:05.290 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/lkkbd.ko 2024-05-28T13:44:05.290 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/lm8323.ko 2024-05-28T13:44:05.290 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/lm8333.ko 2024-05-28T13:44:05.290 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-05-28T13:44:05.290 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-05-28T13:44:05.291 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-05-28T13:44:05.291 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-05-28T13:44:05.291 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/newtonkbd.ko 2024-05-28T13:44:05.291 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-05-28T13:44:05.291 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/qt1070.ko 2024-05-28T13:44:05.291 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/qt2160.ko 2024-05-28T13:44:05.292 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-05-28T13:44:05.292 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/stowaway.ko 2024-05-28T13:44:05.292 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/sunkbd.ko 2024-05-28T13:44:05.292 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-05-28T13:44:05.292 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-05-28T13:44:05.292 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-05-28T13:44:05.293 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/xtkbd.ko 2024-05-28T13:44:05.293 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/matrix-keymap.ko 2024-05-28T13:44:05.293 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/ 2024-05-28T13:44:05.293 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-28T13:44:05.293 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 39545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-28T13:44:05.293 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 159993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-28T13:44:05.294 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 114193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-28T13:44:05.294 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/gpio_mouse.ko 2024-05-28T13:44:05.294 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 428121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/psmouse.ko 2024-05-28T13:44:05.294 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-28T13:44:05.294 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-28T13:44:05.294 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-28T13:44:05.295 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-28T13:44:05.295 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/ 2024-05-28T13:44:05.295 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 140921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-28T13:44:05.295 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/ 2024-05-28T13:44:05.295 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-28T13:44:05.296 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-28T13:44:05.296 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/ct82c710.ko 2024-05-28T13:44:05.296 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-28T13:44:05.296 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/parkbd.ko 2024-05-28T13:44:05.296 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/pcips2.ko 2024-05-28T13:44:05.296 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/ps2mult.ko 2024-05-28T13:44:05.297 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-28T13:44:05.297 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serport.ko 2024-05-28T13:44:05.297 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-28T13:44:05.297 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/ 2024-05-28T13:44:05.297 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-88pm860x.ko 2024-05-28T13:44:05.298 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-adp5520.ko 2024-05-28T13:44:05.298 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 77193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-bd2802.ko 2024-05-28T13:44:05.298 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-28T13:44:05.298 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-da903x.ko 2024-05-28T13:44:05.298 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-da9052.ko 2024-05-28T13:44:05.299 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-gpio.ko 2024-05-28T13:44:05.299 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-28T13:44:05.299 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3533.ko 2024-05-28T13:44:05.299 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm355x.ko 2024-05-28T13:44:05.299 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3642.ko 2024-05-28T13:44:05.299 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-28T13:44:05.300 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp8788.ko 2024-05-28T13:44:05.300 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-max8997.ko 2024-05-28T13:44:05.300 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-mc13783.ko 2024-05-28T13:44:05.300 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-menf21bmc.ko 2024-05-28T13:44:05.300 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-pca9532.ko 2024-05-28T13:44:05.300 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-pca955x.ko 2024-05-28T13:44:05.301 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-pca963x.ko 2024-05-28T13:44:05.301 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-28T13:44:05.301 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-tca6507.ko 2024-05-28T13:44:05.301 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-wm831x-status.ko 2024-05-28T13:44:05.301 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-wm8350.ko 2024-05-28T13:44:05.302 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ 2024-05-28T13:44:05.302 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-28T13:44:05.302 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-28T13:44:05.302 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-28T13:44:05.302 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-28T13:44:05.302 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-28T13:44:05.303 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-28T13:44:05.303 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-28T13:44:05.303 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-28T13:44:05.303 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-28T13:44:05.303 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/ 2024-05-28T13:44:05.304 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/ 2024-05-28T13:44:05.304 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 790665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-28T13:44:05.304 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 49065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-28T13:44:05.304 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 116817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bufio.ko 2024-05-28T13:44:05.304 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 58273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-28T13:44:05.304 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 199017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-28T13:44:05.305 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 134937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-28T13:44:05.305 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-28T13:44:05.305 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 67489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-era.ko 2024-05-28T13:44:05.305 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-28T13:44:05.305 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 50441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-28T13:44:05.306 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log.ko 2024-05-28T13:44:05.306 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 60969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-mirror.ko 2024-05-28T13:44:05.306 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 97193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-28T13:44:05.306 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-28T13:44:05.306 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 85681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-28T13:44:05.306 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-region-hash.ko 2024-05-28T13:44:05.307 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-28T13:44:05.307 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-28T13:44:05.307 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 139793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-snapshot.ko 2024-05-28T13:44:05.307 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-28T13:44:05.307 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 234441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-28T13:44:05.308 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 66417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-28T13:44:05.308 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-zero.ko 2024-05-28T13:44:05.308 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/ 2024-05-28T13:44:05.308 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 264273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-28T13:44:05.308 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-28T13:44:05.308 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 124705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-28T13:44:05.309 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 157673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-28T13:44:05.309 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 444649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-28T13:44:05.309 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/ 2024-05-28T13:44:05.309 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/ 2024-05-28T13:44:05.309 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 214833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-28T13:44:05.309 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 77977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-28T13:44:05.310 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 54777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-28T13:44:05.310 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-28T13:44:05.310 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 150753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-28T13:44:05.310 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 97209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-28T13:44:05.310 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 63865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-28T13:44:05.311 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/ 2024-05-28T13:44:05.311 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/88pm800.ko 2024-05-28T13:44:05.311 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/88pm805.ko 2024-05-28T13:44:05.311 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/88pm80x.ko 2024-05-28T13:44:05.311 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/arizona-i2c.ko 2024-05-28T13:44:05.312 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 148545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/arizona.ko 2024-05-28T13:44:05.312 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/bcm590xx.ko 2024-05-28T13:44:05.312 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/cros_ec_dev.ko 2024-05-28T13:44:05.312 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/dln2.ko 2024-05-28T13:44:05.312 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/janz-cmodio.ko 2024-05-28T13:44:05.312 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 59081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/kempld-core.ko 2024-05-28T13:44:05.313 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lm3533-core.ko 2024-05-28T13:44:05.313 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-05-28T13:44:05.313 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lp3943.ko 2024-05-28T13:44:05.313 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 60489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-28T13:44:05.313 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-28T13:44:05.313 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/max8907.ko 2024-05-28T13:44:05.314 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mc13xxx-core.ko 2024-05-28T13:44:05.314 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-05-28T13:44:05.314 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/menf21bmc.ko 2024-05-28T13:44:05.314 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/pcf50633-adc.ko 2024-05-28T13:44:05.314 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/pcf50633-gpio.ko 2024-05-28T13:44:05.315 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/pcf50633.ko 2024-05-28T13:44:05.315 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-05-28T13:44:05.315 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/retu-mfd.ko 2024-05-28T13:44:05.315 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 99409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/si476x-core.ko 2024-05-28T13:44:05.315 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 58049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-28T13:44:05.315 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 60673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/tps65010.ko 2024-05-28T13:44:05.316 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/tps6507x.ko 2024-05-28T13:44:05.316 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-28T13:44:05.316 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-28T13:44:05.316 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-28T13:44:05.316 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 52345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wm8994.ko 2024-05-28T13:44:05.316 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ 2024-05-28T13:44:05.317 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-05-28T13:44:05.317 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 64257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ad525x_dpot.ko 2024-05-28T13:44:05.317 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/ 2024-05-28T13:44:05.317 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 57497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-28T13:44:05.317 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-28T13:44:05.317 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 47681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-28T13:44:05.318 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 58625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-28T13:44:05.318 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/c2port/ 2024-05-28T13:44:05.318 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-05-28T13:44:05.318 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 47441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/c2port/core.ko 2024-05-28T13:44:05.318 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/ 2024-05-28T13:44:05.318 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-28T13:44:05.319 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ds1682.ko 2024-05-28T13:44:05.319 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/dummy-irq.ko 2024-05-28T13:44:05.319 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/echo/ 2024-05-28T13:44:05.319 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/echo/echo.ko 2024-05-28T13:44:05.319 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/ 2024-05-28T13:44:05.319 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-28T13:44:05.320 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-28T13:44:05.320 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-28T13:44:05.320 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-28T13:44:05.320 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/genwqe/ 2024-05-28T13:44:05.320 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 191617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-05-28T13:44:05.320 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hmc6352.ko 2024-05-28T13:44:05.320 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 37753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-28T13:44:05.321 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ibmasm/ 2024-05-28T13:44:05.321 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 92289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-05-28T13:44:05.321 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ics932s401.ko 2024-05-28T13:44:05.321 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-28T13:44:05.321 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-28T13:44:05.321 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/ 2024-05-28T13:44:05.321 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 54857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-28T13:44:05.322 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-28T13:44:05.322 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/ 2024-05-28T13:44:05.322 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 154257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-28T13:44:05.322 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 69793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-28T13:44:05.322 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 388977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-28T13:44:05.323 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 37953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/phantom.ko 2024-05-28T13:44:05.323 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ti-st/ 2024-05-28T13:44:05.323 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 91129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ti-st/st_drv.ko 2024-05-28T13:44:05.323 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-28T13:44:05.323 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-28T13:44:05.323 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-28T13:44:05.324 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 55785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-28T13:44:05.324 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/ 2024-05-28T13:44:05.324 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 234849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-28T13:44:05.324 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ 2024-05-28T13:44:05.324 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/ 2024-05-28T13:44:05.325 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 620849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-28T13:44:05.325 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/ 2024-05-28T13:44:05.325 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/ 2024-05-28T13:44:05.325 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 139745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-28T13:44:05.325 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 49529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-28T13:44:05.325 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-28T13:44:05.325 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/ 2024-05-28T13:44:05.326 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 589537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-28T13:44:05.326 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-28T13:44:05.326 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-28T13:44:05.326 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ 2024-05-28T13:44:05.326 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/ 2024-05-28T13:44:05.326 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 53745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-28T13:44:05.326 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 49193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-28T13:44:05.326 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 158641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-28T13:44:05.326 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 85945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-28T13:44:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ 2024-05-28T13:44:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-28T13:44:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 73857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-28T13:44:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-28T13:44:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 109513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-28T13:44:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/ 2024-05-28T13:44:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 64697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-28T13:44:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/ 2024-05-28T13:44:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 84321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-28T13:44:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/ 2024-05-28T13:44:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 70729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-28T13:44:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/ 2024-05-28T13:44:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 52681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-28T13:44:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/ 2024-05-28T13:44:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 65785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-28T13:44:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 56049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-28T13:44:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 115873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-28T13:44:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/ 2024-05-28T13:44:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/ 2024-05-28T13:44:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 124481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-28T13:44:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-05-28T13:44:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 128393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-28T13:44:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-05-28T13:44:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 110321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-28T13:44:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/ 2024-05-28T13:44:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 101849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-28T13:44:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 88945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-28T13:44:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/ 2024-05-28T13:44:05.329 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 105625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-28T13:44:05.329 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 228977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-28T13:44:05.329 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-05-28T13:44:05.329 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1733297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-28T13:44:05.330 INFO:teuthology.orchestra.run.smithi063.stderr:Sourcing file `/etc/default/grub' 2024-05-28T13:44:05.330 INFO:teuthology.orchestra.run.smithi063.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-28T13:44:05.330 INFO:teuthology.orchestra.run.smithi063.stderr:Generating grub configuration file ... 2024-05-28T13:44:05.413 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 167177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-28T13:44:05.413 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/ 2024-05-28T13:44:05.413 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 164249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-28T13:44:05.413 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 440553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-28T13:44:05.413 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/ 2024-05-28T13:44:05.414 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/ 2024-05-28T13:44:05.414 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 432145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-28T13:44:05.414 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/ 2024-05-28T13:44:05.414 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-05-28T13:44:05.414 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 257145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-28T13:44:05.414 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-05-28T13:44:05.414 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 446057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-28T13:44:05.414 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-05-28T13:44:05.414 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1311193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-28T13:44:05.414 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-05-28T13:44:05.414 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 206673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-28T13:44:05.415 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-05-28T13:44:05.415 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-28T13:44:05.415 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/ 2024-05-28T13:44:05.415 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/ 2024-05-28T13:44:05.415 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 269249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-28T13:44:05.415 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/ 2024-05-28T13:44:05.415 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/ 2024-05-28T13:44:05.415 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 85377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-28T13:44:05.415 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 77449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-28T13:44:05.415 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 169041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-28T13:44:05.415 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 65905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-28T13:44:05.415 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 76201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-28T13:44:05.415 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-28T13:44:05.416 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/ 2024-05-28T13:44:05.416 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 75937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-28T13:44:05.416 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 78697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-28T13:44:05.416 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 52305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-28T13:44:05.416 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ec_bhf.ko 2024-05-28T13:44:05.416 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/ 2024-05-28T13:44:05.416 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/ 2024-05-28T13:44:05.416 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 356353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-28T13:44:05.416 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 61729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-28T13:44:05.416 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 63177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-28T13:44:05.416 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fujitsu/ 2024-05-28T13:44:05.416 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 51505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-05-28T13:44:05.416 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ 2024-05-28T13:44:05.417 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 115561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-28T13:44:05.417 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/ 2024-05-28T13:44:05.417 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 366193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-28T13:44:05.417 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/ 2024-05-28T13:44:05.417 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 755673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-28T13:44:05.417 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/ 2024-05-28T13:44:05.417 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 324513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-28T13:44:05.417 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/ 2024-05-28T13:44:05.417 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1196201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-28T13:44:05.417 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/ 2024-05-28T13:44:05.417 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 362609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-28T13:44:05.417 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/ 2024-05-28T13:44:05.417 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 672313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-28T13:44:05.418 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/ 2024-05-28T13:44:05.418 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 121513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-28T13:44:05.418 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-05-28T13:44:05.418 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 966601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-28T13:44:05.418 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-05-28T13:44:05.418 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 193465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-28T13:44:05.418 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 108625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-28T13:44:05.418 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/ 2024-05-28T13:44:05.418 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-28T13:44:05.418 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 126873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-28T13:44:05.419 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 150769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-28T13:44:05.419 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/ 2024-05-28T13:44:05.419 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-05-28T13:44:05.419 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 904929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-28T13:44:05.419 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 344089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-28T13:44:05.419 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-05-28T13:44:05.419 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-05-28T13:44:05.419 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1413529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-28T13:44:05.419 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ 2024-05-28T13:44:05.419 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 65905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-05-28T13:44:05.419 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 54217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-05-28T13:44:05.419 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-05-28T13:44:05.420 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 117345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-28T13:44:05.420 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/ 2024-05-28T13:44:05.420 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-05-28T13:44:05.420 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 131353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-28T13:44:05.420 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ 2024-05-28T13:44:05.420 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 91457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-28T13:44:05.420 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 56681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-28T13:44:05.420 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/ 2024-05-28T13:44:05.420 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 181201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-28T13:44:05.420 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/ 2024-05-28T13:44:05.420 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 171409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-28T13:44:05.420 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/ 2024-05-28T13:44:05.421 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 58241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-28T13:44:05.421 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 50113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-28T13:44:05.421 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/ 2024-05-28T13:44:05.421 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-05-28T13:44:05.421 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 300793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-28T13:44:05.421 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 105681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-28T13:44:05.421 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-05-28T13:44:05.421 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 885057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-28T13:44:05.421 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/ 2024-05-28T13:44:05.421 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 52297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-28T13:44:05.421 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/ 2024-05-28T13:44:05.421 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 81633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-28T13:44:05.421 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 100681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-28T13:44:05.421 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-28T13:44:05.422 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 264145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-28T13:44:05.422 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/samsung/ 2024-05-28T13:44:05.422 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-05-28T13:44:05.422 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 142401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-05-28T13:44:05.422 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/ 2024-05-28T13:44:05.422 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1350201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-28T13:44:05.422 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/ 2024-05-28T13:44:05.422 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 57945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-28T13:44:05.422 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/ 2024-05-28T13:44:05.422 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 66497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-28T13:44:05.422 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 80057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-28T13:44:05.422 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/ 2024-05-28T13:44:05.422 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 74281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-28T13:44:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 79449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-28T13:44:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 100777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-28T13:44:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 83433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-28T13:44:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/ 2024-05-28T13:44:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-05-28T13:44:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-05-28T13:44:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 46977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-28T13:44:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-05-28T13:44:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 644329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-28T13:44:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/ 2024-05-28T13:44:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 128489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-28T13:44:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 207985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-28T13:44:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 82625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-28T13:44:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 91345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-28T13:44:05.424 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/ 2024-05-28T13:44:05.424 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 59793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-28T13:44:05.424 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/ 2024-05-28T13:44:05.424 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 110721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-28T13:44:05.424 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/ 2024-05-28T13:44:05.424 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 98681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-28T13:44:05.424 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 102505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-28T13:44:05.424 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/ 2024-05-28T13:44:05.424 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 61489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-28T13:44:05.424 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-28T13:44:05.424 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/ 2024-05-28T13:44:05.424 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 69905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-28T13:44:05.424 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/fddi/ 2024-05-28T13:44:05.424 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 49441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/fddi/defxx.ko 2024-05-28T13:44:05.425 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/fddi/skfp/ 2024-05-28T13:44:05.425 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 275713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/fddi/skfp/skfp.ko 2024-05-28T13:44:05.425 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 91665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-28T13:44:05.425 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/ 2024-05-28T13:44:05.425 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 290441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-28T13:44:05.425 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/ 2024-05-28T13:44:05.425 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-28T13:44:05.425 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-28T13:44:05.425 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ 2024-05-28T13:44:05.425 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 108313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-28T13:44:05.425 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 78057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-28T13:44:05.425 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-28T13:44:05.425 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/ 2024-05-28T13:44:05.425 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-28T13:44:05.426 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-28T13:44:05.426 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-gpio.ko 2024-05-28T13:44:05.426 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-28T13:44:05.426 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-28T13:44:05.426 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-28T13:44:05.426 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 62673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-28T13:44:05.426 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-28T13:44:05.426 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/ 2024-05-28T13:44:05.426 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-28T13:44:05.426 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 48233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-28T13:44:05.426 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ 2024-05-28T13:44:05.426 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-28T13:44:05.427 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 61977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-28T13:44:05.427 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 52857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-28T13:44:05.427 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-28T13:44:05.427 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 53401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-28T13:44:05.427 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-28T13:44:05.427 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-28T13:44:05.427 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-28T13:44:05.427 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-28T13:44:05.427 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-28T13:44:05.427 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 103369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-28T13:44:05.427 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 96017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-28T13:44:05.427 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-28T13:44:05.427 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 121097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-28T13:44:05.427 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-28T13:44:05.428 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 62105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-28T13:44:05.428 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-28T13:44:05.428 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-28T13:44:05.428 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-28T13:44:05.428 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/plip/ 2024-05-28T13:44:05.428 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 55345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/plip/plip.ko 2024-05-28T13:44:05.428 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ 2024-05-28T13:44:05.428 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-28T13:44:05.428 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-28T13:44:05.428 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-28T13:44:05.428 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-28T13:44:05.428 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-28T13:44:05.429 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 53225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-28T13:44:05.429 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-28T13:44:05.429 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-28T13:44:05.429 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sb1000.ko 2024-05-28T13:44:05.429 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/ 2024-05-28T13:44:05.429 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 52065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-28T13:44:05.429 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-28T13:44:05.429 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 80153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-28T13:44:05.429 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/ 2024-05-28T13:44:05.429 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 137313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-28T13:44:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-28T13:44:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-28T13:44:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-28T13:44:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-28T13:44:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-28T13:44:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 108081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-28T13:44:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/ 2024-05-28T13:44:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 176481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-28T13:44:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/ 2024-05-28T13:44:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 384329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-28T13:44:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/ 2024-05-28T13:44:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 175513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-28T13:44:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ 2024-05-28T13:44:05.431 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-28T13:44:05.431 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/ 2024-05-28T13:44:05.431 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/ 2024-05-28T13:44:05.431 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 335489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-28T13:44:05.431 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 127497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-28T13:44:05.431 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/ 2024-05-28T13:44:05.431 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 168113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-28T13:44:05.431 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-28T13:44:05.431 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 103881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-28T13:44:05.431 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-28T13:44:05.431 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/ 2024-05-28T13:44:05.431 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/ 2024-05-28T13:44:05.431 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-28T13:44:05.432 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-05-28T13:44:05.432 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-05-28T13:44:05.432 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pci-stub.ko 2024-05-28T13:44:05.432 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 55321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-28T13:44:05.432 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/ 2024-05-28T13:44:05.432 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-28T13:44:05.432 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 179937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pcmcia.ko 2024-05-28T13:44:05.432 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 78145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pcmcia_core.ko 2024-05-28T13:44:05.432 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-05-28T13:44:05.432 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-28T13:44:05.432 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 133745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-28T13:44:05.432 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/phy/ 2024-05-28T13:44:05.432 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/phy/broadcom/ 2024-05-28T13:44:05.432 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-05-28T13:44:05.433 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/phy/samsung/ 2024-05-28T13:44:05.433 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-05-28T13:44:05.433 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/ 2024-05-28T13:44:05.433 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/ 2024-05-28T13:44:05.433 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 78185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-28T13:44:05.433 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/ 2024-05-28T13:44:05.433 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/ 2024-05-28T13:44:05.433 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 61025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-05-28T13:44:05.433 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/cros_ec.ko 2024-05-28T13:44:05.433 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-05-28T13:44:05.433 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-05-28T13:44:05.433 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-05-28T13:44:05.433 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-05-28T13:44:05.433 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-05-28T13:44:05.434 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/ 2024-05-28T13:44:05.434 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 87929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-28T13:44:05.434 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-28T13:44:05.434 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 48249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-28T13:44:05.434 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 73857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-28T13:44:05.434 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-28T13:44:05.434 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 164761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-28T13:44:05.434 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 46177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-28T13:44:05.434 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 62065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-28T13:44:05.434 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-28T13:44:05.434 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 51873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-28T13:44:05.435 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-28T13:44:05.435 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-28T13:44:05.435 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/ibm_rtl.ko 2024-05-28T13:44:05.435 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/ 2024-05-28T13:44:05.435 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-28T13:44:05.435 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-05-28T13:44:05.435 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 53993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-28T13:44:05.435 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-28T13:44:05.435 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-28T13:44:05.435 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-28T13:44:05.436 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 51889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-28T13:44:05.436 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-28T13:44:05.436 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-28T13:44:05.436 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-28T13:44:05.436 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-28T13:44:05.436 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-28T13:44:05.436 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 61409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-28T13:44:05.436 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/ 2024-05-28T13:44:05.436 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/ 2024-05-28T13:44:05.436 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/88pm860x_battery.ko 2024-05-28T13:44:05.436 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/88pm860x_charger.ko 2024-05-28T13:44:05.436 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 68825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/bq2415x_charger.ko 2024-05-28T13:44:05.436 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 73065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/bq24190_charger.ko 2024-05-28T13:44:05.436 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/bq24735-charger.ko 2024-05-28T13:44:05.437 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-05-28T13:44:05.437 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/da9030_battery.ko 2024-05-28T13:44:05.437 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/da9052-battery.ko 2024-05-28T13:44:05.437 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/ds2760_battery.ko 2024-05-28T13:44:05.437 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/ds2780_battery.ko 2024-05-28T13:44:05.437 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/ds2781_battery.ko 2024-05-28T13:44:05.437 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/ds2782_battery.ko 2024-05-28T13:44:05.437 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/gpio-charger.ko 2024-05-28T13:44:05.437 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/isp1704_charger.ko 2024-05-28T13:44:05.437 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/lp8727_charger.ko 2024-05-28T13:44:05.437 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/max14577_charger.ko 2024-05-28T13:44:05.437 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 37825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/max17042_battery.ko 2024-05-28T13:44:05.437 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/max8903_charger.ko 2024-05-28T13:44:05.438 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/max8925_power.ko 2024-05-28T13:44:05.438 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/pcf50633-charger.ko 2024-05-28T13:44:05.438 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/sbs-battery.ko 2024-05-28T13:44:05.438 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/test_power.ko 2024-05-28T13:44:05.438 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/tps65090-charger.ko 2024-05-28T13:44:05.438 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/wm831x_backup.ko 2024-05-28T13:44:05.438 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/wm831x_power.ko 2024-05-28T13:44:05.438 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/wm8350_power.ko 2024-05-28T13:44:05.438 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/ 2024-05-28T13:44:05.438 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 73289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-28T13:44:05.438 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-28T13:44:05.438 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/ 2024-05-28T13:44:05.438 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/ 2024-05-28T13:44:05.439 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-28T13:44:05.439 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-28T13:44:05.439 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-28T13:44:05.439 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 49801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-28T13:44:05.439 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ 2024-05-28T13:44:05.439 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 91369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-28T13:44:05.439 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-28T13:44:05.439 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/ 2024-05-28T13:44:05.439 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/ 2024-05-28T13:44:05.439 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/ 2024-05-28T13:44:05.440 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 67473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-28T13:44:05.440 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 47129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-28T13:44:05.440 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/ 2024-05-28T13:44:05.440 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-88pm80x.ko 2024-05-28T13:44:05.440 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-88pm860x.ko 2024-05-28T13:44:05.440 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-28T13:44:05.440 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-da9052.ko 2024-05-28T13:44:05.440 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-da9055.ko 2024-05-28T13:44:05.440 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-da9063.ko 2024-05-28T13:44:05.440 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-28T13:44:05.440 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 82217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-28T13:44:05.440 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-28T13:44:05.440 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-28T13:44:05.441 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-28T13:44:05.441 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-28T13:44:05.441 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-28T13:44:05.441 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-28T13:44:05.441 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-28T13:44:05.441 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-28T13:44:05.441 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-28T13:44:05.441 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-28T13:44:05.441 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-28T13:44:05.441 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-lp8788.ko 2024-05-28T13:44:05.442 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 46905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-28T13:44:05.442 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-28T13:44:05.442 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-28T13:44:05.442 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t86.ko 2024-05-28T13:44:05.442 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-28T13:44:05.442 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max8907.ko 2024-05-28T13:44:05.442 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max8925.ko 2024-05-28T13:44:05.442 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max8997.ko 2024-05-28T13:44:05.442 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max8998.ko 2024-05-28T13:44:05.442 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-05-28T13:44:05.442 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-28T13:44:05.443 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-palmas.ko 2024-05-28T13:44:05.443 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 56769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-28T13:44:05.443 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf50633.ko 2024-05-28T13:44:05.443 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-28T13:44:05.443 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-28T13:44:05.443 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-28T13:44:05.443 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-28T13:44:05.443 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rc5t583.ko 2024-05-28T13:44:05.443 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-28T13:44:05.444 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-28T13:44:05.444 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-28T13:44:05.444 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-28T13:44:05.444 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-28T13:44:05.444 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-s35390a.ko 2024-05-28T13:44:05.444 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-28T13:44:05.444 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-tps6586x.ko 2024-05-28T13:44:05.444 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-tps65910.ko 2024-05-28T13:44:05.445 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-wm831x.ko 2024-05-28T13:44:05.445 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-wm8350.ko 2024-05-28T13:44:05.445 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-28T13:44:05.445 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ 2024-05-28T13:44:05.445 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 90369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-28T13:44:05.445 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 69585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-28T13:44:05.445 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 65937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-28T13:44:05.445 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 164473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-28T13:44:05.446 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-28T13:44:05.446 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/ 2024-05-28T13:44:05.446 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 340689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-28T13:44:05.446 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 120393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-28T13:44:05.446 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/ 2024-05-28T13:44:05.446 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 468049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-28T13:44:05.446 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 380633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-28T13:44:05.446 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic94xx/ 2024-05-28T13:44:05.447 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 243649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-05-28T13:44:05.447 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-28T13:44:05.447 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/ 2024-05-28T13:44:05.447 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 135601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-28T13:44:05.447 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-28T13:44:05.447 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/ 2024-05-28T13:44:05.447 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 304969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-28T13:44:05.447 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/ 2024-05-28T13:44:05.448 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1527017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-28T13:44:05.448 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/ 2024-05-28T13:44:05.448 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 297769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-28T13:44:05.448 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/ 2024-05-28T13:44:05.448 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 144377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-28T13:44:05.448 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 47449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-28T13:44:05.448 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/ 2024-05-28T13:44:05.449 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 390657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-28T13:44:05.449 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/ 2024-05-28T13:44:05.449 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-05-28T13:44:05.449 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 88873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-28T13:44:05.449 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-05-28T13:44:05.449 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 117473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-28T13:44:05.449 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 173009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-28T13:44:05.449 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 91497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-28T13:44:05.450 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 45393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-28T13:44:05.450 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/ 2024-05-28T13:44:05.450 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 267521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-28T13:44:05.450 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 78961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-28T13:44:05.450 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/ 2024-05-28T13:44:05.450 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 95321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-28T13:44:05.450 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 194001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-28T13:44:05.450 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fdomain.ko 2024-05-28T13:44:05.451 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/ 2024-05-28T13:44:05.451 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 366457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-28T13:44:05.451 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 251497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-28T13:44:05.451 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 65689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-28T13:44:05.451 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 57089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-28T13:44:05.451 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/imm.ko 2024-05-28T13:44:05.451 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 55145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-28T13:44:05.451 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 327833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-28T13:44:05.451 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 129777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-28T13:44:05.452 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/ 2024-05-28T13:44:05.452 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 397657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-28T13:44:05.452 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-28T13:44:05.452 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 63113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-28T13:44:05.452 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/ 2024-05-28T13:44:05.452 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 386801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-28T13:44:05.452 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 178169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-28T13:44:05.452 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 64801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-28T13:44:05.453 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/ 2024-05-28T13:44:05.453 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 273953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-28T13:44:05.453 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/ 2024-05-28T13:44:05.453 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 2735913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-28T13:44:05.453 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/ 2024-05-28T13:44:05.453 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 88521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-28T13:44:05.453 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-28T13:44:05.453 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 389089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-28T13:44:05.454 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 104761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-28T13:44:05.454 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/ 2024-05-28T13:44:05.454 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 836897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-28T13:44:05.454 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/ 2024-05-28T13:44:05.454 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 158705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-28T13:44:05.454 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 96529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-28T13:44:05.454 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pcmcia/ 2024-05-28T13:44:05.454 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 101609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-05-28T13:44:05.454 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-05-28T13:44:05.455 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-05-28T13:44:05.455 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-05-28T13:44:05.455 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/ 2024-05-28T13:44:05.455 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 443713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-28T13:44:05.455 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 152025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-28T13:44:05.455 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ppa.ko 2024-05-28T13:44:05.455 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 66969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-28T13:44:05.455 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/ 2024-05-28T13:44:05.456 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 2324713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-28T13:44:05.456 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 103593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-28T13:44:05.456 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/ 2024-05-28T13:44:05.456 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 697281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-28T13:44:05.456 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qlogicfas408.ko 2024-05-28T13:44:05.456 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-28T13:44:05.456 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 322857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-28T13:44:05.456 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 219849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-28T13:44:05.456 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 357145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-28T13:44:05.457 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 121177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-28T13:44:05.457 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 93473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-28T13:44:05.457 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 55393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-28T13:44:05.457 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-28T13:44:05.457 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 138113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-28T13:44:05.457 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 57393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-28T13:44:05.457 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/ 2024-05-28T13:44:05.457 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 205001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-28T13:44:05.457 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 55465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-28T13:44:05.457 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 59113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-28T13:44:05.457 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 48769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-28T13:44:05.457 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 59409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/xen-scsifront.ko 2024-05-28T13:44:05.457 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ 2024-05-28T13:44:05.458 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 174865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-28T13:44:05.458 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/ 2024-05-28T13:44:05.458 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/ 2024-05-28T13:44:05.458 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 920073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-28T13:44:05.458 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/ 2024-05-28T13:44:05.458 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 60313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-28T13:44:05.458 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 48121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-28T13:44:05.458 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 54225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-28T13:44:05.458 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1170873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-28T13:44:05.458 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 51889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-28T13:44:05.458 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 149777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-28T13:44:05.458 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/ 2024-05-28T13:44:05.459 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 90033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-28T13:44:05.459 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/ 2024-05-28T13:44:05.459 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/ 2024-05-28T13:44:05.459 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/ 2024-05-28T13:44:05.459 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-28T13:44:05.459 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-28T13:44:05.459 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-28T13:44:05.459 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-28T13:44:05.459 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-28T13:44:05.459 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-28T13:44:05.460 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-28T13:44:05.460 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-28T13:44:05.460 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-28T13:44:05.460 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-28T13:44:05.460 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-28T13:44:05.460 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-28T13:44:05.460 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 45993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-28T13:44:05.460 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-28T13:44:05.460 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-28T13:44:05.460 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-05-28T13:44:05.460 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-28T13:44:05.460 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-28T13:44:05.460 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-28T13:44:05.460 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ 2024-05-28T13:44:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ 2024-05-28T13:44:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 99289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-28T13:44:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 69721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/moxa.ko 2024-05-28T13:44:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 61641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/mxser.ko 2024-05-28T13:44:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 134329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-28T13:44:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 37897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-28T13:44:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 78961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-28T13:44:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/ 2024-05-28T13:44:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/ 2024-05-28T13:44:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-05-28T13:44:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 65113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-28T13:44:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-05-28T13:44:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/altera_uart.ko 2024-05-28T13:44:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-28T13:44:05.462 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 91593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-05-28T13:44:05.462 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/ 2024-05-28T13:44:05.462 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 120553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-28T13:44:05.462 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/rp2.ko 2024-05-28T13:44:05.462 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 65633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/sc16is7xx.ko 2024-05-28T13:44:05.462 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/ 2024-05-28T13:44:05.462 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ 2024-05-28T13:44:05.462 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 693057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-28T13:44:05.462 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ 2024-05-28T13:44:05.462 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-28T13:44:05.462 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-05-28T13:44:05.462 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/ 2024-05-28T13:44:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 55609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-28T13:44:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-28T13:44:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-28T13:44:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_dmem_genirq.ko 2024-05-28T13:44:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_mf624.ko 2024-05-28T13:44:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_netx.ko 2024-05-28T13:44:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-28T13:44:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-05-28T13:44:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-28T13:44:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/ 2024-05-28T13:44:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/c67x00/ 2024-05-28T13:44:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 90297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/c67x00/c67x00.ko 2024-05-28T13:44:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/chipidea/ 2024-05-28T13:44:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 189169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-05-28T13:44:05.464 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-05-28T13:44:05.464 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-05-28T13:44:05.464 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-05-28T13:44:05.464 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-05-28T13:44:05.464 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/ 2024-05-28T13:44:05.464 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 101537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-28T13:44:05.464 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 63017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-28T13:44:05.464 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 56713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-28T13:44:05.464 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 107409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-28T13:44:05.464 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/common/ 2024-05-28T13:44:05.464 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/common/ulpi.ko 2024-05-28T13:44:05.464 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/dwc2/ 2024-05-28T13:44:05.464 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-05-28T13:44:05.465 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/dwc3/ 2024-05-28T13:44:05.465 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-05-28T13:44:05.465 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-05-28T13:44:05.465 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 575217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/dwc3/dwc3.ko 2024-05-28T13:44:05.465 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/ 2024-05-28T13:44:05.465 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/ 2024-05-28T13:44:05.465 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 72241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/u_ether.ko 2024-05-28T13:44:05.465 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 61945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/u_serial.ko 2024-05-28T13:44:05.465 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-05-28T13:44:05.465 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 49729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-05-28T13:44:05.465 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-05-28T13:44:05.465 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-05-28T13:44:05.465 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 131513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-05-28T13:44:05.466 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 62313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-05-28T13:44:05.466 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 146601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-05-28T13:44:05.466 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 70569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-05-28T13:44:05.466 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-05-28T13:44:05.466 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 37641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-05-28T13:44:05.466 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 70009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-05-28T13:44:05.466 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 101233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-05-28T13:44:05.466 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-05-28T13:44:05.466 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 71241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-05-28T13:44:05.466 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 84929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-05-28T13:44:05.466 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/ 2024-05-28T13:44:05.466 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-05-28T13:44:05.466 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-05-28T13:44:05.467 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-05-28T13:44:05.467 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-05-28T13:44:05.467 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-05-28T13:44:05.467 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-05-28T13:44:05.467 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-05-28T13:44:05.467 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-05-28T13:44:05.467 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-05-28T13:44:05.467 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-05-28T13:44:05.467 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-05-28T13:44:05.468 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-05-28T13:44:05.468 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 78065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-05-28T13:44:05.468 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-05-28T13:44:05.468 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 204009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/libcomposite.ko 2024-05-28T13:44:05.468 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/ 2024-05-28T13:44:05.468 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-05-28T13:44:05.468 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/bdc/ 2024-05-28T13:44:05.468 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 182537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-05-28T13:44:05.468 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 45537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-05-28T13:44:05.468 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 66505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-05-28T13:44:05.468 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 85209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-05-28T13:44:05.468 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 69305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-05-28T13:44:05.468 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 90305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/net2272.ko 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 90353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/net2280.ko 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 94833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 80513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 87457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 95433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 195953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/ 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/bcma-hcd.ko 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 108313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/isp116x-hcd.ko 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 104433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 102265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 60097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811_cs.ko 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/ssb-hcd.ko 2024-05-28T13:44:05.469 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-05-28T13:44:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/ 2024-05-28T13:44:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-28T13:44:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-28T13:44:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ 2024-05-28T13:44:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 47345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-28T13:44:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-28T13:44:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-05-28T13:44:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/cytherm.ko 2024-05-28T13:44:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ehset.ko 2024-05-28T13:44:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-28T13:44:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-28T13:44:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-28T13:44:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-28T13:44:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-28T13:44:05.471 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-28T13:44:05.471 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-28T13:44:05.471 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-28T13:44:05.471 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/lvstest.ko 2024-05-28T13:44:05.471 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/ 2024-05-28T13:44:05.471 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 85305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-28T13:44:05.471 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-28T13:44:05.471 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-28T13:44:05.471 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-28T13:44:05.471 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-28T13:44:05.471 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 84257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbtest.ko 2024-05-28T13:44:05.471 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 52057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-28T13:44:05.471 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-28T13:44:05.471 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/mon/ 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 91505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/mon/usbmon.ko 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/musb/ 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 396169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/musb/musb_hdrc.ko 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/ 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-isp1301.ko 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-tahvo.ko 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 79673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-28T13:44:05.472 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-28T13:44:05.473 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 66865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-28T13:44:05.473 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 58321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-28T13:44:05.473 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-28T13:44:05.473 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 53001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/f81232.ko 2024-05-28T13:44:05.473 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 188601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-28T13:44:05.473 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 61481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-28T13:44:05.473 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 152297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-28T13:44:05.473 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 142825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-28T13:44:05.473 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 69209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-28T13:44:05.473 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-28T13:44:05.473 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-28T13:44:05.473 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 66353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-28T13:44:05.473 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 122321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-28T13:44:05.473 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-28T13:44:05.474 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-28T13:44:05.474 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-28T13:44:05.474 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-28T13:44:05.474 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/metro-usb.ko 2024-05-28T13:44:05.474 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 79673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-28T13:44:05.474 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 89961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-28T13:44:05.474 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 65065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mxuport.ko 2024-05-28T13:44:05.474 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-28T13:44:05.474 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-28T13:44:05.474 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-28T13:44:05.474 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 197201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-28T13:44:05.474 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 45329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-28T13:44:05.475 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 61753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-28T13:44:05.475 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-28T13:44:05.475 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 39977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-28T13:44:05.475 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-28T13:44:05.475 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-28T13:44:05.475 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 59337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-28T13:44:05.475 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-28T13:44:05.475 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-28T13:44:05.475 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-28T13:44:05.475 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 70489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-28T13:44:05.475 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-28T13:44:05.475 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-28T13:44:05.475 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 39313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-28T13:44:05.476 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 151121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usbserial.ko 2024-05-28T13:44:05.476 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-28T13:44:05.476 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-28T13:44:05.476 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/wishbone-serial.ko 2024-05-28T13:44:05.476 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-28T13:44:05.476 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ 2024-05-28T13:44:05.476 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 64041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-28T13:44:05.476 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-28T13:44:05.476 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-28T13:44:05.476 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-28T13:44:05.476 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-28T13:44:05.476 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-28T13:44:05.476 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-28T13:44:05.477 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-28T13:44:05.477 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-28T13:44:05.477 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-28T13:44:05.477 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-28T13:44:05.477 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-28T13:44:05.477 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-28T13:44:05.477 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-28T13:44:05.477 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 201145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-28T13:44:05.477 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/ 2024-05-28T13:44:05.477 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 76025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-28T13:44:05.477 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 85233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-28T13:44:05.477 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 122233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-28T13:44:05.477 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/ 2024-05-28T13:44:05.478 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/ 2024-05-28T13:44:05.478 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 206897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-28T13:44:05.478 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-28T13:44:05.478 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 134241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-28T13:44:05.478 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 93665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-28T13:44:05.479 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/ 2024-05-28T13:44:05.479 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 138641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-28T13:44:05.479 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-28T13:44:05.479 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 71273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-28T13:44:05.479 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 107897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-28T13:44:05.479 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/ 2024-05-28T13:44:05.480 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/ 2024-05-28T13:44:05.480 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-05-28T13:44:05.480 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/aat2870_bl.ko 2024-05-28T13:44:05.480 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/adp5520_bl.ko 2024-05-28T13:44:05.480 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/adp8860_bl.ko 2024-05-28T13:44:05.480 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 51737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/adp8870_bl.ko 2024-05-28T13:44:05.481 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-28T13:44:05.481 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/as3711_bl.ko 2024-05-28T13:44:05.481 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/bd6107.ko 2024-05-28T13:44:05.481 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/da903x_bl.ko 2024-05-28T13:44:05.481 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/da9052_bl.ko 2024-05-28T13:44:05.481 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/gpio_backlight.ko 2024-05-28T13:44:05.481 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/kb3886_bl.ko 2024-05-28T13:44:05.482 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-28T13:44:05.482 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lm3533_bl.ko 2024-05-28T13:44:05.482 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lm3639_bl.ko 2024-05-28T13:44:05.482 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lv5207lp.ko 2024-05-28T13:44:05.482 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/max8925_bl.ko 2024-05-28T13:44:05.482 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/pandora_bl.ko 2024-05-28T13:44:05.482 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-05-28T13:44:05.483 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-28T13:44:05.483 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/wm831x_bl.ko 2024-05-28T13:44:05.483 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/ 2024-05-28T13:44:05.483 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ 2024-05-28T13:44:05.483 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-28T13:44:05.483 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 46793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-28T13:44:05.483 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/matrox_w1.ko 2024-05-28T13:44:05.483 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/w1-gpio.ko 2024-05-28T13:44:05.484 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/ 2024-05-28T13:44:05.484 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-28T13:44:05.484 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-28T13:44:05.484 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-28T13:44:05.484 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-28T13:44:05.484 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-28T13:44:05.484 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-28T13:44:05.484 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-28T13:44:05.484 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-28T13:44:05.485 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-28T13:44:05.485 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-28T13:44:05.485 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 70361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-28T13:44:05.485 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 129409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-28T13:44:05.485 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ 2024-05-28T13:44:05.485 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/acquirewdt.ko 2024-05-28T13:44:05.485 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/advantechwdt.ko 2024-05-28T13:44:05.486 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-28T13:44:05.486 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-28T13:44:05.486 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/cpu5wdt.ko 2024-05-28T13:44:05.486 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/da9052_wdt.ko 2024-05-28T13:44:05.486 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/da9055_wdt.ko 2024-05-28T13:44:05.486 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/da9063_wdt.ko 2024-05-28T13:44:05.486 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/dw_wdt.ko 2024-05-28T13:44:05.487 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/eurotechwdt.ko 2024-05-28T13:44:05.487 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-28T13:44:05.487 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-28T13:44:05.487 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-28T13:44:05.487 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-28T13:44:05.487 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-28T13:44:05.487 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-28T13:44:05.487 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-28T13:44:05.487 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-28T13:44:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-28T13:44:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-28T13:44:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/kempld_wdt.ko 2024-05-28T13:44:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-28T13:44:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/mena21_wdt.ko 2024-05-28T13:44:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-05-28T13:44:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-28T13:44:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-05-28T13:44:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pc87413_wdt.ko 2024-05-28T13:44:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 46945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-28T13:44:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 37185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-28T13:44:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/retu_wdt.ko 2024-05-28T13:44:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-05-28T13:44:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-05-28T13:44:05.489 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-28T13:44:05.489 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sc1200wdt.ko 2024-05-28T13:44:05.489 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-28T13:44:05.489 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-05-28T13:44:05.489 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-28T13:44:05.489 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-28T13:44:05.489 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/twl4030_wdt.ko 2024-05-28T13:44:05.489 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-28T13:44:05.489 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-28T13:44:05.489 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-28T13:44:05.489 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-28T13:44:05.489 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wafer5823wdt.ko 2024-05-28T13:44:05.489 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-28T13:44:05.490 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wm831x_wdt.ko 2024-05-28T13:44:05.490 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wm8350_wdt.ko 2024-05-28T13:44:05.490 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-28T13:44:05.490 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/ 2024-05-28T13:44:05.490 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-28T13:44:05.490 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-28T13:44:05.490 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 55705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-28T13:44:05.490 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/ 2024-05-28T13:44:05.490 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 213865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-28T13:44:05.490 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 47673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-28T13:44:05.490 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 85705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-scsiback.ko 2024-05-28T13:44:05.490 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/ 2024-05-28T13:44:05.490 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-28T13:44:05.491 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ 2024-05-28T13:44:05.491 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/ 2024-05-28T13:44:05.491 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 194449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-28T13:44:05.491 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/adfs/ 2024-05-28T13:44:05.491 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 78049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/adfs/adfs.ko 2024-05-28T13:44:05.491 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/ 2024-05-28T13:44:05.491 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 197801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-28T13:44:05.491 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/afs/ 2024-05-28T13:44:05.491 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1260937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/afs/kafs.ko 2024-05-28T13:44:05.491 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/autofs/ 2024-05-28T13:44:05.491 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 127337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/autofs/autofs4.ko 2024-05-28T13:44:05.491 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/ 2024-05-28T13:44:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 80481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-28T13:44:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/bfs/ 2024-05-28T13:44:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 50457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/bfs/bfs.ko 2024-05-28T13:44:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 45553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-28T13:44:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/ 2024-05-28T13:44:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 4491713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-28T13:44:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/ 2024-05-28T13:44:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 376801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-28T13:44:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ 2024-05-28T13:44:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1586617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-28T13:44:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/ 2024-05-28T13:44:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 130017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-28T13:44:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/configfs/ 2024-05-28T13:44:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 139049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/configfs/configfs.ko 2024-05-28T13:44:05.493 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/ 2024-05-28T13:44:05.493 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-28T13:44:05.493 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/ 2024-05-28T13:44:05.493 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 813137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-28T13:44:05.493 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/efs/ 2024-05-28T13:44:05.493 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 46377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/efs/efs.ko 2024-05-28T13:44:05.493 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/ 2024-05-28T13:44:05.493 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1906713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-28T13:44:05.493 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/ 2024-05-28T13:44:05.493 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-28T13:44:05.493 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/freevxfs/ 2024-05-28T13:44:05.494 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 52265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/freevxfs/freevxfs.ko 2024-05-28T13:44:05.494 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/ 2024-05-28T13:44:05.494 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-28T13:44:05.494 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/ 2024-05-28T13:44:05.494 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1486393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-28T13:44:05.494 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/ 2024-05-28T13:44:05.494 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 183217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-28T13:44:05.494 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/ 2024-05-28T13:44:05.494 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 281993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-28T13:44:05.494 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/hpfs/ 2024-05-28T13:44:05.494 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 248945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/hpfs/hpfs.ko 2024-05-28T13:44:05.494 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/ 2024-05-28T13:44:05.495 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 92985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-28T13:44:05.495 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/ 2024-05-28T13:44:05.495 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 532953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-28T13:44:05.495 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/ 2024-05-28T13:44:05.495 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 361281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-28T13:44:05.495 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/ 2024-05-28T13:44:05.495 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 112569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-28T13:44:05.495 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/ 2024-05-28T13:44:05.495 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 591625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-28T13:44:05.495 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/ 2024-05-28T13:44:05.495 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/ 2024-05-28T13:44:05.495 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 102537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-28T13:44:05.495 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/ 2024-05-28T13:44:05.495 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 74337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-28T13:44:05.496 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/ 2024-05-28T13:44:05.496 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 147673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-28T13:44:05.496 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1320377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-28T13:44:05.496 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 82089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv2.ko 2024-05-28T13:44:05.496 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 151617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-28T13:44:05.496 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 2518385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-28T13:44:05.496 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/ 2024-05-28T13:44:05.496 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-28T13:44:05.496 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-28T13:44:05.496 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/ 2024-05-28T13:44:05.496 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 2021969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-28T13:44:05.496 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/ 2024-05-28T13:44:05.496 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 733969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-28T13:44:05.497 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/ 2024-05-28T13:44:05.497 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-28T13:44:05.497 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-28T13:44:05.497 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-28T13:44:05.497 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-28T13:44:05.497 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-28T13:44:05.497 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-28T13:44:05.497 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-28T13:44:05.497 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-28T13:44:05.497 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-28T13:44:05.497 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-28T13:44:05.498 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-28T13:44:05.498 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ascii.ko 2024-05-28T13:44:05.498 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-28T13:44:05.498 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-28T13:44:05.498 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-28T13:44:05.498 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-28T13:44:05.498 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-28T13:44:05.498 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-28T13:44:05.499 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-28T13:44:05.499 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-28T13:44:05.499 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-28T13:44:05.499 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-28T13:44:05.499 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-28T13:44:05.499 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-28T13:44:05.499 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-28T13:44:05.500 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-28T13:44:05.500 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-28T13:44:05.500 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-28T13:44:05.500 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-28T13:44:05.500 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-28T13:44:05.500 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 95617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-28T13:44:05.500 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 144201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-28T13:44:05.501 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 167129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-28T13:44:05.501 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 119505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-28T13:44:05.501 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-28T13:44:05.501 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-28T13:44:05.501 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-28T13:44:05.501 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-28T13:44:05.501 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-28T13:44:05.502 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-28T13:44:05.502 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-28T13:44:05.502 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-28T13:44:05.502 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-28T13:44:05.502 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-28T13:44:05.502 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-28T13:44:05.502 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-28T13:44:05.502 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-28T13:44:05.503 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-28T13:44:05.503 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-28T13:44:05.503 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-28T13:44:05.503 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-28T13:44:05.503 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ntfs3/ 2024-05-28T13:44:05.503 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 634457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ntfs3/ntfs3.ko 2024-05-28T13:44:05.504 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ 2024-05-28T13:44:05.504 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ 2024-05-28T13:44:05.504 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 302033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-28T13:44:05.504 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ 2024-05-28T13:44:05.504 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 548057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-28T13:44:05.504 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ 2024-05-28T13:44:05.504 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 66345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-28T13:44:05.504 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 3245449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-28T13:44:05.505 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 22105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-28T13:44:05.505 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 39305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-28T13:44:05.505 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-28T13:44:05.505 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/omfs/ 2024-05-28T13:44:05.505 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 67465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/omfs/omfs.ko 2024-05-28T13:44:05.505 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/ 2024-05-28T13:44:05.506 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 462817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-28T13:44:05.506 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ 2024-05-28T13:44:05.506 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 67961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-28T13:44:05.506 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/qnx4/ 2024-05-28T13:44:05.506 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/qnx4/qnx4.ko 2024-05-28T13:44:05.506 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/qnx6/ 2024-05-28T13:44:05.506 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 56073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/qnx6/qnx6.ko 2024-05-28T13:44:05.506 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/quota/ 2024-05-28T13:44:05.506 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/quota/quota_tree.ko 2024-05-28T13:44:05.506 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/quota/quota_v1.ko 2024-05-28T13:44:05.507 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/quota/quota_v2.ko 2024-05-28T13:44:05.507 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/ 2024-05-28T13:44:05.507 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 684713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-28T13:44:05.507 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/ 2024-05-28T13:44:05.507 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-28T13:44:05.507 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/ 2024-05-28T13:44:05.507 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/ 2024-05-28T13:44:05.507 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 3350841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-28T13:44:05.507 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/ 2024-05-28T13:44:05.507 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-28T13:44:05.507 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-28T13:44:05.507 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/ 2024-05-28T13:44:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 164145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-28T13:44:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/ 2024-05-28T13:44:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 115865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-28T13:44:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/ 2024-05-28T13:44:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 309937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-28T13:44:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ 2024-05-28T13:44:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 198185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-28T13:44:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/ 2024-05-28T13:44:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 5192193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-28T13:44:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/kernel/ 2024-05-28T13:44:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/kernel/locking/ 2024-05-28T13:44:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 97617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/kernel/locking/locktorture.ko 2024-05-28T13:44:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/kernel/time/ 2024-05-28T13:44:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/kernel/time/test_udelay.ko 2024-05-28T13:44:05.509 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 72657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-28T13:44:05.509 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/ 2024-05-28T13:44:05.509 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-28T13:44:05.509 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc7.ko 2024-05-28T13:44:05.509 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-28T13:44:05.509 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/ 2024-05-28T13:44:05.509 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-28T13:44:05.509 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-28T13:44:05.509 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/interval_tree_test.ko 2024-05-28T13:44:05.509 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-28T13:44:05.509 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-28T13:44:05.509 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/ 2024-05-28T13:44:05.509 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-28T13:44:05.509 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-28T13:44:05.510 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/ 2024-05-28T13:44:05.510 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-28T13:44:05.510 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/memory-notifier-error-inject.ko 2024-05-28T13:44:05.510 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/notifier-error-inject.ko 2024-05-28T13:44:05.510 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/percpu_test.ko 2024-05-28T13:44:05.510 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/pm-notifier-error-inject.ko 2024-05-28T13:44:05.510 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/ 2024-05-28T13:44:05.510 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 151945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-28T13:44:05.510 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/rbtree_test.ko 2024-05-28T13:44:05.510 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/ 2024-05-28T13:44:05.510 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-28T13:44:05.510 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/test-kstrtox.ko 2024-05-28T13:44:05.510 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 4912433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/test_bpf.ko 2024-05-28T13:44:05.547 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 61161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/test_firmware.ko 2024-05-28T13:44:05.547 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/test_module.ko 2024-05-28T13:44:05.547 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/test_user_copy.ko 2024-05-28T13:44:05.547 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-28T13:44:05.548 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-28T13:44:05.548 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-28T13:44:05.548 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/xz/ 2024-05-28T13:44:05.548 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/xz/xz_dec_test.ko 2024-05-28T13:44:05.548 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/mm/ 2024-05-28T13:44:05.548 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-28T13:44:05.548 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ 2024-05-28T13:44:05.548 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/ 2024-05-28T13:44:05.548 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 66945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-28T13:44:05.548 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-28T13:44:05.548 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-28T13:44:05.548 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-28T13:44:05.548 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/802/ 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/ 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 106697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/ 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 236769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-28T13:44:05.549 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-28T13:44:05.550 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 39553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-28T13:44:05.550 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/ 2024-05-28T13:44:05.550 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 879593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-28T13:44:05.550 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/ 2024-05-28T13:44:05.550 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 85225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-28T13:44:05.550 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1031177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-28T13:44:05.550 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ 2024-05-28T13:44:05.550 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-28T13:44:05.550 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-28T13:44:05.550 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-28T13:44:05.550 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-28T13:44:05.550 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-28T13:44:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-28T13:44:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-28T13:44:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-28T13:44:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-28T13:44:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-28T13:44:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-28T13:44:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-28T13:44:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-28T13:44:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-28T13:44:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-28T13:44:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-28T13:44:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-28T13:44:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 70521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/ 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1243369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/core/ 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 153793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/ 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 244825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 71025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 74985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-28T13:44:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/ 2024-05-28T13:44:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 507649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-28T13:44:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-28T13:44:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-28T13:44:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-28T13:44:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-28T13:44:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/hsr/ 2024-05-28T13:44:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 128577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/hsr/hsr.ko 2024-05-28T13:44:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ 2024-05-28T13:44:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ 2024-05-28T13:44:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 55481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-28T13:44:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 357369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-28T13:44:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 59385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-28T13:44:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ 2024-05-28T13:44:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-28T13:44:05.554 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 46409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-28T13:44:05.554 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 60217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-28T13:44:05.554 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-28T13:44:05.554 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 65657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-28T13:44:05.554 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 75481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-28T13:44:05.554 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 77713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-28T13:44:05.554 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-28T13:44:05.554 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-28T13:44:05.554 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-28T13:44:05.554 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ 2024-05-28T13:44:05.554 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 45513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-28T13:44:05.554 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-28T13:44:05.554 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-28T13:44:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 50161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ip_tables.ko 2024-05-28T13:44:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-28T13:44:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-05-28T13:44:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-28T13:44:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-28T13:44:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-28T13:44:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-05-28T13:44:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-28T13:44:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-28T13:44:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-28T13:44:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-28T13:44:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-28T13:44:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-28T13:44:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-28T13:44:05.556 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-28T13:44:05.556 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-28T13:44:05.556 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-05-28T13:44:05.556 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-28T13:44:05.556 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-28T13:44:05.556 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-28T13:44:05.556 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-28T13:44:05.556 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-28T13:44:05.556 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-28T13:44:05.556 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-28T13:44:05.556 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-28T13:44:05.556 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-28T13:44:05.557 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-28T13:44:05.557 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-28T13:44:05.557 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-28T13:44:05.557 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-28T13:44:05.557 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-28T13:44:05.557 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-28T13:44:05.557 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-28T13:44:05.557 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-28T13:44:05.557 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-28T13:44:05.557 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-28T13:44:05.557 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-28T13:44:05.557 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 54425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-28T13:44:05.557 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-28T13:44:05.557 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ 2024-05-28T13:44:05.558 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-28T13:44:05.558 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 47425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-28T13:44:05.558 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-28T13:44:05.558 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 103169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_gre.ko 2024-05-28T13:44:05.558 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 118873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-28T13:44:05.558 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-28T13:44:05.558 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 71081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-28T13:44:05.558 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-28T13:44:05.558 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/mip6.ko 2024-05-28T13:44:05.558 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ 2024-05-28T13:44:05.558 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 50729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-28T13:44:05.559 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-05-28T13:44:05.559 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-28T13:44:05.559 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-28T13:44:05.559 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-28T13:44:05.559 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-28T13:44:05.559 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-28T13:44:05.559 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-28T13:44:05.559 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-28T13:44:05.559 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-28T13:44:05.559 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-28T13:44:05.559 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-28T13:44:05.560 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-28T13:44:05.560 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-28T13:44:05.560 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-28T13:44:05.560 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-28T13:44:05.560 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-28T13:44:05.560 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 46297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-28T13:44:05.560 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-28T13:44:05.560 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 29809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-28T13:44:05.560 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-28T13:44:05.560 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-28T13:44:05.560 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-28T13:44:05.560 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-28T13:44:05.560 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-28T13:44:05.561 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 94081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-28T13:44:05.561 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 30289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-28T13:44:05.561 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-28T13:44:05.561 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/key/ 2024-05-28T13:44:05.561 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 106361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-28T13:44:05.561 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/ 2024-05-28T13:44:05.561 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 153209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-28T13:44:05.561 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-28T13:44:05.561 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-28T13:44:05.561 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-28T13:44:05.561 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-28T13:44:05.561 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-28T13:44:05.562 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 66233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-28T13:44:05.562 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/lapb/ 2024-05-28T13:44:05.562 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 63321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/lapb/lapb.ko 2024-05-28T13:44:05.562 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/ 2024-05-28T13:44:05.562 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-28T13:44:05.562 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 332817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc2.ko 2024-05-28T13:44:05.563 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/ 2024-05-28T13:44:05.563 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 388817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-28T13:44:05.563 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/ 2024-05-28T13:44:05.563 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-28T13:44:05.563 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ 2024-05-28T13:44:05.563 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ 2024-05-28T13:44:05.564 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 147729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-28T13:44:05.564 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-28T13:44:05.564 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-28T13:44:05.564 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-28T13:44:05.564 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 124033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-28T13:44:05.564 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 124337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-28T13:44:05.565 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 126673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-28T13:44:05.565 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 126465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-28T13:44:05.565 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 137825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-28T13:44:05.565 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 72345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-28T13:44:05.565 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 133673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-28T13:44:05.565 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 139065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-28T13:44:05.566 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 138257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-28T13:44:05.566 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 136169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-28T13:44:05.566 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 139889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-28T13:44:05.566 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 37121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-28T13:44:05.566 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ 2024-05-28T13:44:05.566 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 534033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-28T13:44:05.567 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-28T13:44:05.567 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-28T13:44:05.567 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-28T13:44:05.567 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-28T13:44:05.567 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-28T13:44:05.568 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-28T13:44:05.568 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-28T13:44:05.568 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-28T13:44:05.568 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-28T13:44:05.568 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-28T13:44:05.568 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-28T13:44:05.569 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-28T13:44:05.569 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-28T13:44:05.569 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 55209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-28T13:44:05.569 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 496713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-28T13:44:05.569 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-28T13:44:05.569 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-28T13:44:05.570 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 37857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-28T13:44:05.570 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 134361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-28T13:44:05.570 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-28T13:44:05.570 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-28T13:44:05.570 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 140793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-28T13:44:05.571 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-28T13:44:05.571 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-28T13:44:05.571 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 79865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-28T13:44:05.571 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-28T13:44:05.571 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-28T13:44:05.571 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-28T13:44:05.572 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 117385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-28T13:44:05.572 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-28T13:44:05.572 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 46097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-28T13:44:05.572 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 118337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-28T13:44:05.572 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-28T13:44:05.572 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-28T13:44:05.573 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-28T13:44:05.573 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-28T13:44:05.573 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-28T13:44:05.573 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 53385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-28T13:44:05.573 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 811705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-28T13:44:05.573 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 53041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-28T13:44:05.574 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-28T13:44:05.574 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 37201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-05-28T13:44:05.574 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 49289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-28T13:44:05.574 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-28T13:44:05.574 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 65137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-28T13:44:05.575 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-28T13:44:05.575 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-28T13:44:05.575 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 50649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-28T13:44:05.575 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-28T13:44:05.575 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-28T13:44:05.575 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-28T13:44:05.575 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-28T13:44:05.576 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-28T13:44:05.576 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-28T13:44:05.576 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-28T13:44:05.576 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-28T13:44:05.576 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-28T13:44:05.576 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_osf.ko 2024-05-28T13:44:05.577 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-28T13:44:05.577 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-28T13:44:05.577 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-28T13:44:05.577 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-28T13:44:05.577 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-28T13:44:05.577 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 24985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_synproxy.ko 2024-05-28T13:44:05.578 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tproxy.ko 2024-05-28T13:44:05.578 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 108385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/x_tables.ko 2024-05-28T13:44:05.578 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-28T13:44:05.578 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-28T13:44:05.578 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-28T13:44:05.579 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-28T13:44:05.579 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 21857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-28T13:44:05.579 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-28T13:44:05.579 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-28T13:44:05.579 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-28T13:44:05.579 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 45209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-28T13:44:05.580 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-28T13:44:05.580 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-28T13:44:05.580 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-28T13:44:05.580 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-28T13:44:05.580 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-28T13:44:05.580 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-28T13:44:05.581 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-28T13:44:05.581 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-28T13:44:05.581 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-28T13:44:05.581 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-28T13:44:05.581 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-28T13:44:05.582 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-28T13:44:05.582 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-28T13:44:05.582 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-28T13:44:05.582 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-28T13:44:05.582 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-28T13:44:05.582 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-28T13:44:05.583 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-28T13:44:05.583 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-28T13:44:05.583 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-28T13:44:05.583 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-28T13:44:05.583 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-28T13:44:05.583 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-28T13:44:05.584 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-28T13:44:05.584 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-28T13:44:05.584 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-28T13:44:05.584 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-28T13:44:05.584 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-28T13:44:05.584 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-28T13:44:05.585 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-28T13:44:05.585 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 52809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-28T13:44:05.585 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-28T13:44:05.585 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-28T13:44:05.585 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-28T13:44:05.586 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-28T13:44:05.586 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-28T13:44:05.586 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-28T13:44:05.586 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-28T13:44:05.586 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-28T13:44:05.586 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-28T13:44:05.587 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-28T13:44:05.587 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-28T13:44:05.587 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 16153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-28T13:44:05.587 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-28T13:44:05.587 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-28T13:44:05.587 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-28T13:44:05.588 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-28T13:44:05.588 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-28T13:44:05.588 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-28T13:44:05.588 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-28T13:44:05.588 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 11873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-28T13:44:05.588 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-28T13:44:05.588 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-28T13:44:05.589 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-28T13:44:05.589 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-28T13:44:05.589 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 19769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-28T13:44:05.589 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-28T13:44:05.589 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-28T13:44:05.589 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-28T13:44:05.590 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-28T13:44:05.590 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpudp.ko 2024-05-28T13:44:05.590 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-28T13:44:05.590 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-28T13:44:05.590 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/ 2024-05-28T13:44:05.590 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 17225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-28T13:44:05.590 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/ 2024-05-28T13:44:05.590 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 14241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-28T13:44:05.590 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/ 2024-05-28T13:44:05.590 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 488457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-28T13:44:05.590 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-28T13:44:05.591 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 10865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-28T13:44:05.591 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-28T13:44:05.591 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/ 2024-05-28T13:44:05.591 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-28T13:44:05.591 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/phonet/ 2024-05-28T13:44:05.591 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 141993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/phonet/phonet.ko 2024-05-28T13:44:05.591 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 68801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/phonet/pn_pep.ko 2024-05-28T13:44:05.591 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/ 2024-05-28T13:44:05.591 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 318537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-28T13:44:05.591 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 202249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-28T13:44:05.591 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 85073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-28T13:44:05.591 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/rxrpc/ 2024-05-28T13:44:05.591 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1028129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/rxrpc/rxrpc.ko 2024-05-28T13:44:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/ 2024-05-28T13:44:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-28T13:44:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 28713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-28T13:44:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 45401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-28T13:44:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 27585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-28T13:44:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 53457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-28T13:44:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-28T13:44:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-28T13:44:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-28T13:44:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 45665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-28T13:44:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-28T13:44:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 41449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-28T13:44:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_cgroup.ko 2024-05-28T13:44:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 37297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-28T13:44:05.593 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 39593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-28T13:44:05.593 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 49313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-28T13:44:05.593 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 79137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-28T13:44:05.593 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-28T13:44:05.593 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 15377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-28T13:44:05.593 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 48777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-28T13:44:05.593 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-28T13:44:05.593 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-28T13:44:05.593 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 8305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-28T13:44:05.593 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-28T13:44:05.593 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 25089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-28T13:44:05.593 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-28T13:44:05.594 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 43145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-28T13:44:05.594 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 39169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq_codel.ko 2024-05-28T13:44:05.594 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 42881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-28T13:44:05.594 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 62809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-28T13:44:05.594 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-28T13:44:05.594 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 81825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-28T13:44:05.594 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-28T13:44:05.594 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 36617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-28T13:44:05.594 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-28T13:44:05.594 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-28T13:44:05.594 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 44785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-28T13:44:05.594 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 32825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-28T13:44:05.594 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 13497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-28T13:44:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 31793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-28T13:44:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 60241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-28T13:44:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 40881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-28T13:44:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 33425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-28T13:44:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 38321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-28T13:44:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 35769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-28T13:44:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 34153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-28T13:44:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/ 2024-05-28T13:44:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1187153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-28T13:44:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-28T13:44:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/ 2024-05-28T13:44:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/ 2024-05-28T13:44:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 432225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-28T13:44:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 70081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-28T13:44:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1940753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-28T13:44:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/ 2024-05-28T13:44:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 889729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-28T13:44:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/ 2024-05-28T13:44:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 1122273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-28T13:44:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-28T13:44:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/ 2024-05-28T13:44:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 18793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-28T13:44:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/ 2024-05-28T13:44:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 105601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-28T13:44:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 90249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-28T13:44:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 123745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-28T13:44:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 12897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-28T13:44:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 20393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-28T13:44:05.597 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/x25/ 2024-05-28T13:44:05.597 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 165777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/x25/x25.ko 2024-05-28T13:44:05.597 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/ 2024-05-28T13:44:05.597 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 26721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_algo.ko 2024-05-28T13:44:05.597 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-28T13:44:05.597 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 131857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_user.ko 2024-05-28T13:44:05.597 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 646387 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-28T13:44:05.597 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 630543 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-28T13:44:05.597 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 7706 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-28T13:44:05.597 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 23609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-28T13:44:05.597 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 9941 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-28T13:44:05.597 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 70990 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-28T13:44:05.598 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 146985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-28T13:44:05.598 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 232892 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-28T13:44:05.598 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 210 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-28T13:44:05.598 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 72627 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-28T13:44:05.598 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 879 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-28T13:44:05.598 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 258450 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-28T13:44:05.598 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 313019 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-28T13:44:05.598 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./usr/ 2024-05-28T13:44:05.598 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./usr/share/ 2024-05-28T13:44:05.598 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./usr/share/doc/ 2024-05-28T13:44:05.598 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./usr/share/doc/linux-image-6.9.0-gb9459a26b088/ 2024-05-28T13:44:05.598 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 171 2024-05-23 11:07 ./usr/share/doc/linux-image-6.9.0-gb9459a26b088/changelog.Debian.gz 2024-05-28T13:44:05.599 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r-- root/root 691 2024-05-23 11:07 ./usr/share/doc/linux-image-6.9.0-gb9459a26b088/copyright 2024-05-28T13:44:05.604 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-28T13:44:05.604 INFO:teuthology.task.kernel:searching for kernel 6.9.0-gb9459a26b088 2024-05-28T13:44:05.604 DEBUG:teuthology.orchestra.run.smithi042:> egrep '(submenu|menuentry.*6.9.0-gb9459a26b088).*{' /boot/grub/grub.cfg 2024-05-28T13:44:05.605 INFO:teuthology.orchestra.run.smithi063.stderr:Found linux image: /boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-28T13:44:05.611 INFO:teuthology.orchestra.run.smithi042.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-28T13:44:05.611 INFO:teuthology.orchestra.run.smithi042.stdout: menuentry 'Ubuntu, with Linux 6.9.0-gb9459a26b088' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-gb9459a26b088-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-28T13:44:05.611 INFO:teuthology.orchestra.run.smithi042.stdout: menuentry 'Ubuntu, with Linux 6.9.0-gb9459a26b088 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-gb9459a26b088-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-28T13:44:05.612 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-05-28T13:44:05.612 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-gb9459a26b088 2024-05-28T13:44:05.612 DEBUG:teuthology.orchestra.run.smithi042:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-05-28T13:44:05.616 INFO:teuthology.orchestra.run.smithi063.stderr:Found initrd image: /boot/initrd.img-6.9.0-gb9459a26b088 2024-05-28T13:44:05.657 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi042.front.sepia.ceph.com to complete... 2024-05-28T13:44:06.010 INFO:teuthology.orchestra.run.smithi063.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-28T13:44:06.020 INFO:teuthology.orchestra.run.smithi063.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-28T13:44:06.089 INFO:teuthology.orchestra.run.smithi063.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-28T13:44:06.094 INFO:teuthology.orchestra.run.smithi063.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-28T13:44:06.110 INFO:teuthology.orchestra.run.smithi042.stderr:Sourcing file `/etc/default/grub' 2024-05-28T13:44:06.111 INFO:teuthology.orchestra.run.smithi042.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-28T13:44:06.119 INFO:teuthology.orchestra.run.smithi042.stderr:Generating grub configuration file ... 2024-05-28T13:44:06.226 INFO:teuthology.orchestra.run.smithi063.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-28T13:44:06.226 INFO:teuthology.orchestra.run.smithi063.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-28T13:44:06.226 INFO:teuthology.orchestra.run.smithi063.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-28T13:44:06.256 INFO:teuthology.orchestra.run.smithi063.stderr:done 2024-05-28T13:44:06.328 INFO:teuthology.orchestra.run.smithi042.stderr:Found linux image: /boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-28T13:44:06.335 INFO:teuthology.orchestra.run.smithi042.stderr:Found initrd image: /boot/initrd.img-6.9.0-gb9459a26b088 2024-05-28T13:44:06.445 DEBUG:teuthology.orchestra.run.smithi063:> dpkg-deb -c /tmp/linux-image.deb 2024-05-28T13:44:06.497 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./ 2024-05-28T13:44:06.497 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./boot/ 2024-05-28T13:44:06.497 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7004139 2024-05-23 11:07 ./boot/System.map-6.9.0-gb9459a26b088 2024-05-28T13:44:06.516 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 172251 2024-05-23 11:07 ./boot/config-6.9.0-gb9459a26b088 2024-05-28T13:44:06.516 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13136896 2024-05-23 11:07 ./boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-28T13:44:06.540 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./etc/ 2024-05-28T13:44:06.540 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./etc/kernel/ 2024-05-28T13:44:06.540 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./etc/kernel/postinst.d/ 2024-05-28T13:44:06.541 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./etc/kernel/postrm.d/ 2024-05-28T13:44:06.541 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./etc/kernel/preinst.d/ 2024-05-28T13:44:06.541 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./etc/kernel/prerm.d/ 2024-05-28T13:44:06.541 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/ 2024-05-28T13:44:06.541 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/ 2024-05-28T13:44:06.541 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/ 2024-05-28T13:44:06.541 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/ 2024-05-28T13:44:06.541 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/ 2024-05-28T13:44:06.541 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/ 2024-05-28T13:44:06.541 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ 2024-05-28T13:44:06.541 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 507497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/aesni-intel.ko 2024-05-28T13:44:06.543 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-28T13:44:06.543 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-28T13:44:06.543 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-28T13:44:06.543 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 97945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-28T13:44:06.543 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 109897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-28T13:44:06.543 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 155241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-28T13:44:06.543 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-28T13:44:06.543 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-28T13:44:06.543 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 87001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-28T13:44:06.544 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-28T13:44:06.544 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 50297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-28T13:44:06.544 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 50369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-28T13:44:06.544 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 54505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-28T13:44:06.544 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-28T13:44:06.546 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 48041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-28T13:44:06.546 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 62865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-28T13:44:06.546 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 53561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-28T13:44:06.546 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-28T13:44:06.546 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-28T13:44:06.546 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/ 2024-05-28T13:44:06.546 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/ 2024-05-28T13:44:06.546 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/ 2024-05-28T13:44:06.546 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 49073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-28T13:44:06.546 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpuid.ko 2024-05-28T13:44:06.546 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/msr.ko 2024-05-28T13:44:06.546 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/ 2024-05-28T13:44:06.547 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 443289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-28T13:44:06.547 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1037081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-28T13:44:06.547 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 2850153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-28T13:44:06.564 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ 2024-05-28T13:44:06.564 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 65017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/af_alg.ko 2024-05-28T13:44:06.564 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/algif_hash.ko 2024-05-28T13:44:06.564 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/algif_skcipher.ko 2024-05-28T13:44:06.565 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-28T13:44:06.565 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-28T13:44:06.565 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-28T13:44:06.565 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/asymmetric_keys/ 2024-05-28T13:44:06.565 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-05-28T13:44:06.565 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/ 2024-05-28T13:44:06.565 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-28T13:44:06.565 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-28T13:44:06.565 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-28T13:44:06.565 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-28T13:44:06.565 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-28T13:44:06.565 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-28T13:44:06.565 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-28T13:44:06.566 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-28T13:44:06.566 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-28T13:44:06.566 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-28T13:44:06.566 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-28T13:44:06.566 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-28T13:44:06.566 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-28T13:44:06.566 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-28T13:44:06.566 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-28T13:44:06.566 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-28T13:44:06.566 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-28T13:44:06.566 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-28T13:44:06.566 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 59121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cryptd.ko 2024-05-28T13:44:06.566 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 48105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-28T13:44:06.567 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_simd.ko 2024-05-28T13:44:06.567 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-28T13:44:06.567 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-28T13:44:06.567 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-28T13:44:06.567 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-28T13:44:06.567 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-28T13:44:06.567 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-28T13:44:06.567 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-28T13:44:06.567 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-28T13:44:06.567 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lrw.ko 2024-05-28T13:44:06.567 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-28T13:44:06.567 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-28T13:44:06.567 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-28T13:44:06.567 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-28T13:44:06.568 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-28T13:44:06.568 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-28T13:44:06.568 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-28T13:44:06.568 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-28T13:44:06.568 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-28T13:44:06.568 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 169057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-28T13:44:06.568 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-28T13:44:06.568 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-28T13:44:06.568 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-28T13:44:06.568 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-28T13:44:06.568 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-28T13:44:06.568 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-28T13:44:06.568 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-28T13:44:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xts.ko 2024-05-28T13:44:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-28T13:44:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ 2024-05-28T13:44:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ 2024-05-28T13:44:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_extlog.ko 2024-05-28T13:44:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-28T13:44:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-28T13:44:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/apei/ 2024-05-28T13:44:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/apei/einj.ko 2024-05-28T13:44:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-28T13:44:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-28T13:44:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-28T13:44:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-28T13:44:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 119017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-28T13:44:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ 2024-05-28T13:44:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-28T13:44:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 99177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci.ko 2024-05-28T13:44:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-28T13:44:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 107729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci.ko 2024-05-28T13:44:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-28T13:44:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-28T13:44:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-28T13:44:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-28T13:44:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-28T13:44:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-28T13:44:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-28T13:44:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-28T13:44:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-28T13:44:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-28T13:44:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-28T13:44:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-28T13:44:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-28T13:44:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-28T13:44:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-28T13:44:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-28T13:44:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-28T13:44:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-28T13:44:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 60249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_legacy.ko 2024-05-28T13:44:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-28T13:44:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-28T13:44:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-28T13:44:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-28T13:44:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-28T13:44:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-28T13:44:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-28T13:44:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-28T13:44:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-28T13:44:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-28T13:44:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-28T13:44:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-28T13:44:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-28T13:44:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_radisys.ko 2024-05-28T13:44:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-28T13:44:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rz1000.ko 2024-05-28T13:44:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-28T13:44:06.573 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-28T13:44:06.573 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-28T13:44:06.573 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-28T13:44:06.573 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-28T13:44:06.573 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-28T13:44:06.573 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-28T13:44:06.573 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-28T13:44:06.573 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 102113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-28T13:44:06.573 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 85401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-28T13:44:06.573 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-28T13:44:06.573 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-28T13:44:06.573 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-28T13:44:06.573 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 45897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-28T13:44:06.573 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-28T13:44:06.574 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-28T13:44:06.574 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 53345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-28T13:44:06.574 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-28T13:44:06.574 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-28T13:44:06.574 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-28T13:44:06.574 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ 2024-05-28T13:44:06.574 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-28T13:44:06.574 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/ 2024-05-28T13:44:06.574 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 163833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-28T13:44:06.574 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/ 2024-05-28T13:44:06.574 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/ 2024-05-28T13:44:06.574 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 140833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-28T13:44:06.574 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/ 2024-05-28T13:44:06.574 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1241585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-28T13:44:06.575 INFO:teuthology.orchestra.run.smithi042.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-28T13:44:06.575 INFO:teuthology.orchestra.run.smithi042.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-28T13:44:06.596 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 203281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-28T13:44:06.597 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/ 2024-05-28T13:44:06.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 115257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-28T13:44:06.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 143425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-28T13:44:06.597 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/ 2024-05-28T13:44:06.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 126609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-28T13:44:06.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 119865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-28T13:44:06.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 313209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-28T13:44:06.597 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/ 2024-05-28T13:44:06.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 117217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-28T13:44:06.597 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/ 2024-05-28T13:44:06.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 70369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-28T13:44:06.597 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ 2024-05-28T13:44:06.597 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/applicom.ko 2024-05-28T13:44:06.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-28T13:44:06.598 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/ 2024-05-28T13:44:06.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-28T13:44:06.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-28T13:44:06.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-28T13:44:06.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-28T13:44:06.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-28T13:44:06.598 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ 2024-05-28T13:44:06.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 37185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-28T13:44:06.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 186913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-28T13:44:06.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-28T13:44:06.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 179441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-28T13:44:06.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 85433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-28T13:44:06.598 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 65305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-28T13:44:06.599 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-28T13:44:06.599 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/ 2024-05-28T13:44:06.599 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 103385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-28T13:44:06.599 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/nvram.ko 2024-05-28T13:44:06.599 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-28T13:44:06.599 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 59233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-28T13:44:06.599 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/ 2024-05-28T13:44:06.599 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-28T13:44:06.599 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-05-28T13:44:06.599 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-05-28T13:44:06.599 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 39185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-05-28T13:44:06.599 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-28T13:44:06.599 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/xillybus/ 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/xillybus/xillybus_class.ko 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 62537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/xillybus/xillybus_core.ko 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/clk/ 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/clk/clk-palmas.ko 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 57281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/clk/clk-si5351.ko 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/clk/clk-twl6040.ko 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/clk/clk-wm831x.ko 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/ 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-28T13:44:06.600 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ 2024-05-28T13:44:06.601 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ 2024-05-28T13:44:06.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 89209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-28T13:44:06.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 333593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-28T13:44:06.601 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/ 2024-05-28T13:44:06.601 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/ 2024-05-28T13:44:06.601 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/ 2024-05-28T13:44:06.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 751433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-28T13:44:06.601 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-05-28T13:44:06.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-28T13:44:06.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-28T13:44:06.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-28T13:44:06.601 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/ 2024-05-28T13:44:06.601 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 90465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-28T13:44:06.602 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/ 2024-05-28T13:44:06.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-28T13:44:06.602 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ 2024-05-28T13:44:06.602 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/ 2024-05-28T13:44:06.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-28T13:44:06.602 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ 2024-05-28T13:44:06.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 146745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-28T13:44:06.602 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ 2024-05-28T13:44:06.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 96137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-28T13:44:06.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-28T13:44:06.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 51097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-28T13:44:06.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-28T13:44:06.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-28T13:44:06.602 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-28T13:44:06.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-28T13:44:06.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-28T13:44:06.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 64153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-28T13:44:06.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-28T13:44:06.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-28T13:44:06.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 68689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-28T13:44:06.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-28T13:44:06.603 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/ 2024-05-28T13:44:06.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/extcon-gpio.ko 2024-05-28T13:44:06.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/extcon-max14577.ko 2024-05-28T13:44:06.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/extcon-max77693.ko 2024-05-28T13:44:06.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/extcon-max8997.ko 2024-05-28T13:44:06.603 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/extcon-palmas.ko 2024-05-28T13:44:06.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/extcon-rt8973a.ko 2024-05-28T13:44:06.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/extcon/extcon-sm5502.ko 2024-05-28T13:44:06.604 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/ 2024-05-28T13:44:06.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/dmi-sysfs.ko 2024-05-28T13:44:06.604 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/efi/ 2024-05-28T13:44:06.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/efi/efi-pstore.ko 2024-05-28T13:44:06.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-28T13:44:06.604 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/ 2024-05-28T13:44:06.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-adp5520.ko 2024-05-28T13:44:06.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-amd8111.ko 2024-05-28T13:44:06.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-arizona.ko 2024-05-28T13:44:06.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-da9052.ko 2024-05-28T13:44:06.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-da9055.ko 2024-05-28T13:44:06.604 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-dln2.ko 2024-05-28T13:44:06.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 81065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-f7188x.ko 2024-05-28T13:44:06.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-generic.ko 2024-05-28T13:44:06.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-28T13:44:06.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-05-28T13:44:06.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-kempld.ko 2024-05-28T13:44:06.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-lp3943.ko 2024-05-28T13:44:06.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-max7300.ko 2024-05-28T13:44:06.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-max730x.ko 2024-05-28T13:44:06.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-max732x.ko 2024-05-28T13:44:06.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-05-28T13:44:06.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 55497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-pca953x.ko 2024-05-28T13:44:06.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-pcf857x.ko 2024-05-28T13:44:06.605 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-rdc321x.ko 2024-05-28T13:44:06.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-sch.ko 2024-05-28T13:44:06.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-sch311x.ko 2024-05-28T13:44:06.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-tps65912.ko 2024-05-28T13:44:06.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-twl4030.ko 2024-05-28T13:44:06.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-twl6040.ko 2024-05-28T13:44:06.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-28T13:44:06.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-vx855.ko 2024-05-28T13:44:06.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-wm831x.ko 2024-05-28T13:44:06.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-wm8350.ko 2024-05-28T13:44:06.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-wm8994.ko 2024-05-28T13:44:06.606 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/ 2024-05-28T13:44:06.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-a4tech.ko 2024-05-28T13:44:06.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-apple.ko 2024-05-28T13:44:06.606 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-28T13:44:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-28T13:44:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-28T13:44:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-belkin.ko 2024-05-28T13:44:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-cherry.ko 2024-05-28T13:44:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-chicony.ko 2024-05-28T13:44:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 74385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-cp2112.ko 2024-05-28T13:44:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-cypress.ko 2024-05-28T13:44:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-28T13:44:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-28T13:44:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-28T13:44:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-28T13:44:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ezkey.ko 2024-05-28T13:44:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-28T13:44:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-generic.ko 2024-05-28T13:44:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-28T13:44:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-28T13:44:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-28T13:44:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-28T13:44:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-28T13:44:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-28T13:44:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-28T13:44:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kensington.ko 2024-05-28T13:44:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-28T13:44:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-28T13:44:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-28T13:44:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-28T13:44:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 54209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-28T13:44:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 37545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lg-g15.ko 2024-05-28T13:44:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 67961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-28T13:44:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 121361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-28T13:44:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 97929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech.ko 2024-05-28T13:44:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-magicmouse.ko 2024-05-28T13:44:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-microsoft.ko 2024-05-28T13:44:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-monterey.ko 2024-05-28T13:44:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 58993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-28T13:44:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 39689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ntrig.ko 2024-05-28T13:44:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-28T13:44:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-28T13:44:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-28T13:44:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 91217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-28T13:44:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-28T13:44:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-28T13:44:06.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-28T13:44:06.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-28T13:44:06.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-28T13:44:06.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-28T13:44:06.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-28T13:44:06.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-28T13:44:06.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-28T13:44:06.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-28T13:44:06.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-28T13:44:06.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-28T13:44:06.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-28T13:44:06.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-28T13:44:06.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-28T13:44:06.610 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-28T13:44:06.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-28T13:44:06.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-28T13:44:06.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-28T13:44:06.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-28T13:44:06.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 62121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-28T13:44:06.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-28T13:44:06.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-28T13:44:06.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-28T13:44:06.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-28T13:44:06.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-28T13:44:06.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-28T13:44:06.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-28T13:44:06.611 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-28T13:44:06.612 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 81097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-28T13:44:06.612 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-28T13:44:06.612 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 184729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-28T13:44:06.612 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-28T13:44:06.612 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-28T13:44:06.612 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-28T13:44:06.612 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 319897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid.ko 2024-05-28T13:44:06.612 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 39521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-28T13:44:06.612 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/usbhid/ 2024-05-28T13:44:06.612 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 153617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/usbhid/usbhid.ko 2024-05-28T13:44:06.612 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/usbhid/usbkbd.ko 2024-05-28T13:44:06.612 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/usbhid/usbmouse.ko 2024-05-28T13:44:06.612 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 261929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-28T13:44:06.612 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/ 2024-05-28T13:44:06.613 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 75057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-28T13:44:06.613 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 110657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-28T13:44:06.613 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 381625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-28T13:44:06.613 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ 2024-05-28T13:44:06.613 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 77145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-28T13:44:06.613 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 67001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-28T13:44:06.613 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-28T13:44:06.613 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-28T13:44:06.613 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-28T13:44:06.613 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-28T13:44:06.613 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-28T13:44:06.613 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 115049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-28T13:44:06.613 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-28T13:44:06.613 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 55217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-28T13:44:06.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-28T13:44:06.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-28T13:44:06.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-28T13:44:06.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-28T13:44:06.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 89017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-28T13:44:06.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 49145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-28T13:44:06.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 81465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-28T13:44:06.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-28T13:44:06.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 48169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-28T13:44:06.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 68857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-28T13:44:06.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 60185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-28T13:44:06.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 69001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-28T13:44:06.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 47969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-28T13:44:06.614 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-28T13:44:06.615 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-28T13:44:06.615 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/da9052-hwmon.ko 2024-05-28T13:44:06.615 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/da9055-hwmon.ko 2024-05-28T13:44:06.615 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 55705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-28T13:44:06.615 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 113129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-28T13:44:06.615 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-28T13:44:06.615 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-28T13:44:06.615 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-28T13:44:06.615 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc2103.ko 2024-05-28T13:44:06.615 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-28T13:44:06.615 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 78913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-28T13:44:06.615 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 108953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-28T13:44:06.615 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 50689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-28T13:44:06.615 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-28T13:44:06.616 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 65761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-28T13:44:06.616 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-28T13:44:06.616 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-28T13:44:06.616 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 59753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-28T13:44:06.616 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 56305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-28T13:44:06.616 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hih6130.ko 2024-05-28T13:44:06.616 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-28T13:44:06.616 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-28T13:44:06.616 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-28T13:44:06.616 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-28T13:44:06.616 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-28T13:44:06.616 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-28T13:44:06.616 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-28T13:44:06.616 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 156457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-28T13:44:06.623 INFO:teuthology.orchestra.run.smithi042.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-28T13:44:06.630 INFO:teuthology.orchestra.run.smithi042.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-28T13:44:06.646 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/jc42.ko 2024-05-28T13:44:06.646 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-28T13:44:06.646 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-28T13:44:06.646 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-28T13:44:06.646 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 57761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-28T13:44:06.646 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-28T13:44:06.647 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-28T13:44:06.647 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-28T13:44:06.647 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-28T13:44:06.647 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 37041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-28T13:44:06.647 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-28T13:44:06.647 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 86489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-28T13:44:06.647 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 55633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-28T13:44:06.647 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 76481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-28T13:44:06.647 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-28T13:44:06.647 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 131497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-28T13:44:06.647 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-28T13:44:06.647 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-28T13:44:06.647 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-28T13:44:06.647 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-28T13:44:06.648 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-28T13:44:06.648 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-28T13:44:06.648 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-28T13:44:06.648 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-28T13:44:06.648 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-28T13:44:06.648 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-28T13:44:06.648 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-28T13:44:06.648 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-28T13:44:06.648 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-28T13:44:06.648 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-28T13:44:06.648 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-28T13:44:06.648 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-28T13:44:06.648 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 39721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-28T13:44:06.649 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mc13783-adc.ko 2024-05-28T13:44:06.649 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-28T13:44:06.649 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-05-28T13:44:06.649 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 56497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-28T13:44:06.649 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 141345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-28T13:44:06.649 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 72729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-28T13:44:06.649 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 67769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-28T13:44:06.649 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 97769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-28T13:44:06.649 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 61913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-28T13:44:06.649 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-28T13:44:06.649 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ 2024-05-28T13:44:06.649 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-28T13:44:06.649 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-28T13:44:06.649 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-28T13:44:06.650 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-28T13:44:06.650 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-28T13:44:06.650 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-28T13:44:06.650 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-28T13:44:06.650 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 101225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-28T13:44:06.650 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-28T13:44:06.650 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-28T13:44:06.650 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-28T13:44:06.650 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-28T13:44:06.650 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-28T13:44:06.650 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-28T13:44:06.650 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-28T13:44:06.650 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-28T13:44:06.651 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-28T13:44:06.651 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-28T13:44:06.651 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-28T13:44:06.651 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 48705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-28T13:44:06.651 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-28T13:44:06.651 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 46465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-28T13:44:06.651 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-28T13:44:06.651 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-28T13:44:06.651 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-28T13:44:06.651 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-28T13:44:06.651 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-28T13:44:06.651 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-28T13:44:06.651 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-28T13:44:06.651 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 50209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-28T13:44:06.652 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 59657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-28T13:44:06.652 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 62729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-28T13:44:06.652 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 71161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-28T13:44:06.652 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 96569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-28T13:44:06.652 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 85905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-28T13:44:06.652 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 75121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-28T13:44:06.652 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 83417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-28T13:44:06.652 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 116769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-28T13:44:06.652 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 134433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-28T13:44:06.652 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-28T13:44:06.652 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-28T13:44:06.652 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-05-28T13:44:06.652 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-05-28T13:44:06.652 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/ 2024-05-28T13:44:06.653 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/ 2024-05-28T13:44:06.653 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-28T13:44:06.653 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-28T13:44:06.653 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/ 2024-05-28T13:44:06.653 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-05-28T13:44:06.653 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-05-28T13:44:06.653 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-05-28T13:44:06.653 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-28T13:44:06.653 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-28T13:44:06.653 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-28T13:44:06.653 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-05-28T13:44:06.653 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-28T13:44:06.653 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-05-28T13:44:06.653 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 64217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-28T13:44:06.654 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-28T13:44:06.654 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-28T13:44:06.654 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-28T13:44:06.654 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-05-28T13:44:06.654 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-05-28T13:44:06.654 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 76961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-28T13:44:06.654 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-28T13:44:06.654 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 52929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-28T13:44:06.654 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-05-28T13:44:06.654 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-28T13:44:06.654 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-28T13:44:06.654 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-05-28T13:44:06.654 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 57209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 59121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-05-28T13:44:06.655 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-28T13:44:06.656 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-28T13:44:06.656 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-28T13:44:06.656 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/muxes/ 2024-05-28T13:44:06.656 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-05-28T13:44:06.656 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-05-28T13:44:06.656 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-05-28T13:44:06.656 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ 2024-05-28T13:44:06.656 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ 2024-05-28T13:44:06.656 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 329681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-28T13:44:06.656 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1119785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-28T13:44:06.656 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 83897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-28T13:44:06.656 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 408329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-28T13:44:06.656 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 120905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-28T13:44:06.657 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 347201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-28T13:44:06.657 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 84937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-28T13:44:06.657 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/ 2024-05-28T13:44:06.657 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/ 2024-05-28T13:44:06.657 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 590873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-28T13:44:06.657 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/ 2024-05-28T13:44:06.657 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 514577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-28T13:44:06.657 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/ 2024-05-28T13:44:06.657 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 911649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-28T13:44:06.657 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ 2024-05-28T13:44:06.657 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 375449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-28T13:44:06.657 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/ocrdma/ 2024-05-28T13:44:06.657 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 197097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-05-28T13:44:06.657 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/usnic/ 2024-05-28T13:44:06.658 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 212497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-05-28T13:44:06.658 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ 2024-05-28T13:44:06.658 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ 2024-05-28T13:44:06.658 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 334353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-28T13:44:06.658 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ 2024-05-28T13:44:06.658 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 120649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-28T13:44:06.658 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ 2024-05-28T13:44:06.658 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 132577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-28T13:44:06.658 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ 2024-05-28T13:44:06.658 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 157929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-28T13:44:06.658 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ 2024-05-28T13:44:06.658 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 156665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-28T13:44:06.658 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/ 2024-05-28T13:44:06.659 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/evbug.ko 2024-05-28T13:44:06.659 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/ff-memless.ko 2024-05-28T13:44:06.659 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/ 2024-05-28T13:44:06.659 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-28T13:44:06.659 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-28T13:44:06.659 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 50705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-28T13:44:06.659 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 49433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-28T13:44:06.659 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/ 2024-05-28T13:44:06.659 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-05-28T13:44:06.659 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-05-28T13:44:06.659 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-05-28T13:44:06.659 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-05-28T13:44:06.659 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 45465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-28T13:44:06.659 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-05-28T13:44:06.660 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/lkkbd.ko 2024-05-28T13:44:06.660 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/lm8323.ko 2024-05-28T13:44:06.660 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/lm8333.ko 2024-05-28T13:44:06.660 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-05-28T13:44:06.660 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-05-28T13:44:06.660 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-05-28T13:44:06.660 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-05-28T13:44:06.660 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/newtonkbd.ko 2024-05-28T13:44:06.660 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-05-28T13:44:06.660 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/qt1070.ko 2024-05-28T13:44:06.660 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/qt2160.ko 2024-05-28T13:44:06.660 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-05-28T13:44:06.660 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/stowaway.ko 2024-05-28T13:44:06.660 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/sunkbd.ko 2024-05-28T13:44:06.661 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-05-28T13:44:06.661 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-05-28T13:44:06.661 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-05-28T13:44:06.661 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/xtkbd.ko 2024-05-28T13:44:06.661 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/matrix-keymap.ko 2024-05-28T13:44:06.661 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/ 2024-05-28T13:44:06.661 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-28T13:44:06.661 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 39545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-28T13:44:06.661 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 159993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-28T13:44:06.661 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 114193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-28T13:44:06.661 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/gpio_mouse.ko 2024-05-28T13:44:06.661 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 428121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/psmouse.ko 2024-05-28T13:44:06.661 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-28T13:44:06.662 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-28T13:44:06.662 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-28T13:44:06.662 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-28T13:44:06.662 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/ 2024-05-28T13:44:06.662 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 140921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-28T13:44:06.662 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/ 2024-05-28T13:44:06.662 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-28T13:44:06.662 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-28T13:44:06.662 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/ct82c710.ko 2024-05-28T13:44:06.662 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-28T13:44:06.662 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/parkbd.ko 2024-05-28T13:44:06.662 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/pcips2.ko 2024-05-28T13:44:06.662 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/ps2mult.ko 2024-05-28T13:44:06.663 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-28T13:44:06.663 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serport.ko 2024-05-28T13:44:06.663 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-28T13:44:06.663 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/ 2024-05-28T13:44:06.663 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-88pm860x.ko 2024-05-28T13:44:06.663 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-adp5520.ko 2024-05-28T13:44:06.663 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 77193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-bd2802.ko 2024-05-28T13:44:06.663 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-28T13:44:06.663 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-da903x.ko 2024-05-28T13:44:06.663 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-da9052.ko 2024-05-28T13:44:06.663 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-gpio.ko 2024-05-28T13:44:06.663 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-28T13:44:06.663 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3533.ko 2024-05-28T13:44:06.663 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm355x.ko 2024-05-28T13:44:06.664 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3642.ko 2024-05-28T13:44:06.664 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-28T13:44:06.664 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp8788.ko 2024-05-28T13:44:06.664 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-max8997.ko 2024-05-28T13:44:06.664 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-mc13783.ko 2024-05-28T13:44:06.664 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-menf21bmc.ko 2024-05-28T13:44:06.664 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-pca9532.ko 2024-05-28T13:44:06.664 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-pca955x.ko 2024-05-28T13:44:06.664 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-pca963x.ko 2024-05-28T13:44:06.664 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-28T13:44:06.664 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-tca6507.ko 2024-05-28T13:44:06.664 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-wm831x-status.ko 2024-05-28T13:44:06.664 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-wm8350.ko 2024-05-28T13:44:06.664 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ 2024-05-28T13:44:06.665 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-28T13:44:06.665 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-28T13:44:06.665 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-28T13:44:06.665 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-28T13:44:06.665 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-28T13:44:06.665 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-28T13:44:06.665 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-28T13:44:06.665 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-28T13:44:06.665 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-28T13:44:06.665 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/ 2024-05-28T13:44:06.665 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/ 2024-05-28T13:44:06.665 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 790665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-28T13:44:06.665 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 49065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-28T13:44:06.665 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 116817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bufio.ko 2024-05-28T13:44:06.666 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 58273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-28T13:44:06.666 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 199017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-28T13:44:06.666 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 134937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-28T13:44:06.666 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-28T13:44:06.666 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 67489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-era.ko 2024-05-28T13:44:06.666 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-28T13:44:06.666 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 50441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-28T13:44:06.666 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log.ko 2024-05-28T13:44:06.666 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 60969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-mirror.ko 2024-05-28T13:44:06.666 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 97193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-28T13:44:06.666 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-28T13:44:06.666 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 85681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-28T13:44:06.666 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-region-hash.ko 2024-05-28T13:44:06.667 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-28T13:44:06.667 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-28T13:44:06.667 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 139793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-snapshot.ko 2024-05-28T13:44:06.667 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-28T13:44:06.667 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 234441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-28T13:44:06.667 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 66417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-28T13:44:06.667 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-zero.ko 2024-05-28T13:44:06.667 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/ 2024-05-28T13:44:06.667 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 264273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-28T13:44:06.667 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-28T13:44:06.667 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 124705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-28T13:44:06.667 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 157673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-28T13:44:06.667 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 444649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-28T13:44:06.667 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/ 2024-05-28T13:44:06.668 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/ 2024-05-28T13:44:06.668 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 214833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-28T13:44:06.668 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 77977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-28T13:44:06.668 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 54777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-28T13:44:06.668 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-28T13:44:06.668 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 150753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-28T13:44:06.668 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 97209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-28T13:44:06.668 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 63865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-28T13:44:06.668 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/ 2024-05-28T13:44:06.668 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/88pm800.ko 2024-05-28T13:44:06.668 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/88pm805.ko 2024-05-28T13:44:06.668 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/88pm80x.ko 2024-05-28T13:44:06.668 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/arizona-i2c.ko 2024-05-28T13:44:06.668 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 148545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/arizona.ko 2024-05-28T13:44:06.669 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/bcm590xx.ko 2024-05-28T13:44:06.669 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/cros_ec_dev.ko 2024-05-28T13:44:06.669 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/dln2.ko 2024-05-28T13:44:06.669 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/janz-cmodio.ko 2024-05-28T13:44:06.669 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 59081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/kempld-core.ko 2024-05-28T13:44:06.669 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lm3533-core.ko 2024-05-28T13:44:06.669 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-05-28T13:44:06.669 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lp3943.ko 2024-05-28T13:44:06.669 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 60489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-28T13:44:06.669 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-28T13:44:06.669 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/max8907.ko 2024-05-28T13:44:06.669 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mc13xxx-core.ko 2024-05-28T13:44:06.669 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-05-28T13:44:06.669 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/menf21bmc.ko 2024-05-28T13:44:06.670 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/pcf50633-adc.ko 2024-05-28T13:44:06.670 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/pcf50633-gpio.ko 2024-05-28T13:44:06.670 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/pcf50633.ko 2024-05-28T13:44:06.670 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-05-28T13:44:06.670 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/retu-mfd.ko 2024-05-28T13:44:06.670 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 99409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/si476x-core.ko 2024-05-28T13:44:06.670 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 58049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-28T13:44:06.670 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 60673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/tps65010.ko 2024-05-28T13:44:06.670 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/tps6507x.ko 2024-05-28T13:44:06.670 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-28T13:44:06.670 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-28T13:44:06.670 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-28T13:44:06.670 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 52345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wm8994.ko 2024-05-28T13:44:06.670 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ 2024-05-28T13:44:06.671 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-05-28T13:44:06.671 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 64257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ad525x_dpot.ko 2024-05-28T13:44:06.671 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/ 2024-05-28T13:44:06.671 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 57497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-28T13:44:06.671 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-28T13:44:06.671 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 47681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-28T13:44:06.671 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 58625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-28T13:44:06.671 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/c2port/ 2024-05-28T13:44:06.671 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-05-28T13:44:06.671 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 47441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/c2port/core.ko 2024-05-28T13:44:06.671 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/ 2024-05-28T13:44:06.671 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-28T13:44:06.671 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ds1682.ko 2024-05-28T13:44:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/dummy-irq.ko 2024-05-28T13:44:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/echo/ 2024-05-28T13:44:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/echo/echo.ko 2024-05-28T13:44:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/ 2024-05-28T13:44:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-28T13:44:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-28T13:44:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-28T13:44:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-28T13:44:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/genwqe/ 2024-05-28T13:44:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 191617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-05-28T13:44:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hmc6352.ko 2024-05-28T13:44:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 37753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-28T13:44:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ibmasm/ 2024-05-28T13:44:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 92289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-05-28T13:44:06.673 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ics932s401.ko 2024-05-28T13:44:06.673 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-28T13:44:06.673 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-28T13:44:06.673 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/ 2024-05-28T13:44:06.673 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 54857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-28T13:44:06.673 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-28T13:44:06.673 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/ 2024-05-28T13:44:06.673 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 154257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-28T13:44:06.673 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 69793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-28T13:44:06.673 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 388977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-28T13:44:06.682 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 37953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/phantom.ko 2024-05-28T13:44:06.682 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ti-st/ 2024-05-28T13:44:06.682 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 91129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/ti-st/st_drv.ko 2024-05-28T13:44:06.682 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-28T13:44:06.682 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-28T13:44:06.682 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-28T13:44:06.682 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 55785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-28T13:44:06.682 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/ 2024-05-28T13:44:06.682 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 234849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-28T13:44:06.682 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ 2024-05-28T13:44:06.683 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/ 2024-05-28T13:44:06.683 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 620849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-28T13:44:06.683 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/ 2024-05-28T13:44:06.683 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/ 2024-05-28T13:44:06.683 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 139745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-28T13:44:06.683 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 49529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-28T13:44:06.683 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-28T13:44:06.683 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/ 2024-05-28T13:44:06.683 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 589537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-28T13:44:06.683 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-28T13:44:06.683 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-28T13:44:06.683 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ 2024-05-28T13:44:06.683 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/ 2024-05-28T13:44:06.684 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 53745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-28T13:44:06.684 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 49193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-28T13:44:06.684 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 158641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-28T13:44:06.684 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 85945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-28T13:44:06.684 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ 2024-05-28T13:44:06.684 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-28T13:44:06.684 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 73857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-28T13:44:06.684 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-28T13:44:06.684 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 109513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-28T13:44:06.684 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/ 2024-05-28T13:44:06.684 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 64697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-28T13:44:06.684 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/ 2024-05-28T13:44:06.684 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 84321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-28T13:44:06.684 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/ 2024-05-28T13:44:06.685 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 70729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-28T13:44:06.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/ 2024-05-28T13:44:06.685 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 52681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-28T13:44:06.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/ 2024-05-28T13:44:06.685 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 65785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-28T13:44:06.685 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 56049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-28T13:44:06.685 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 115873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-28T13:44:06.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/ 2024-05-28T13:44:06.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/ 2024-05-28T13:44:06.685 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 124481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-28T13:44:06.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-05-28T13:44:06.685 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 128393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-28T13:44:06.685 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-05-28T13:44:06.685 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 110321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-28T13:44:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/ 2024-05-28T13:44:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 101849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-28T13:44:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 88945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-28T13:44:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/ 2024-05-28T13:44:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 105625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-28T13:44:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 228977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-28T13:44:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-05-28T13:44:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1733297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-28T13:44:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 167177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-28T13:44:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/ 2024-05-28T13:44:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 164249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-28T13:44:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 440553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-28T13:44:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/ 2024-05-28T13:44:06.687 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/ 2024-05-28T13:44:06.687 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 432145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-28T13:44:06.687 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/ 2024-05-28T13:44:06.687 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-05-28T13:44:06.687 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 257145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-28T13:44:06.687 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-05-28T13:44:06.687 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 446057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-28T13:44:06.687 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-05-28T13:44:06.687 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1311193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-28T13:44:06.687 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-05-28T13:44:06.687 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 206673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-28T13:44:06.687 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-05-28T13:44:06.687 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-28T13:44:06.688 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/ 2024-05-28T13:44:06.688 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/ 2024-05-28T13:44:06.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 269249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-28T13:44:06.688 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/ 2024-05-28T13:44:06.688 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/ 2024-05-28T13:44:06.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 85377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-28T13:44:06.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 77449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-28T13:44:06.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 169041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-28T13:44:06.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 65905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-28T13:44:06.688 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 76201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-28T13:44:06.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-28T13:44:06.689 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/ 2024-05-28T13:44:06.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 75937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-28T13:44:06.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 78697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-28T13:44:06.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 52305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-28T13:44:06.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ec_bhf.ko 2024-05-28T13:44:06.689 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/ 2024-05-28T13:44:06.689 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/ 2024-05-28T13:44:06.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 356353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-28T13:44:06.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 61729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-28T13:44:06.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 63177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-28T13:44:06.689 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fujitsu/ 2024-05-28T13:44:06.689 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 51505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-05-28T13:44:06.690 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ 2024-05-28T13:44:06.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 115561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-28T13:44:06.690 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/ 2024-05-28T13:44:06.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 366193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-28T13:44:06.690 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/ 2024-05-28T13:44:06.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 755673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-28T13:44:06.690 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/ 2024-05-28T13:44:06.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 324513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-28T13:44:06.690 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/ 2024-05-28T13:44:06.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1196201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-28T13:44:06.690 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/ 2024-05-28T13:44:06.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 362609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-28T13:44:06.690 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/ 2024-05-28T13:44:06.690 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 672313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-28T13:44:06.691 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/ 2024-05-28T13:44:06.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 121513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-28T13:44:06.691 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-05-28T13:44:06.691 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 966601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-28T13:44:06.693 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-05-28T13:44:06.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 193465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-28T13:44:06.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 108625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-28T13:44:06.694 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/ 2024-05-28T13:44:06.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-28T13:44:06.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 126873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-28T13:44:06.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 150769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-28T13:44:06.694 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/ 2024-05-28T13:44:06.694 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-05-28T13:44:06.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 904929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-28T13:44:06.694 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 344089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-28T13:44:06.694 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-05-28T13:44:06.694 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-05-28T13:44:06.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1413529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-28T13:44:06.695 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ 2024-05-28T13:44:06.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 65905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-05-28T13:44:06.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 54217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-05-28T13:44:06.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-05-28T13:44:06.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 117345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-28T13:44:06.695 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/ 2024-05-28T13:44:06.695 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-05-28T13:44:06.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 131353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-28T13:44:06.695 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ 2024-05-28T13:44:06.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 91457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-28T13:44:06.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 56681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-28T13:44:06.695 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/ 2024-05-28T13:44:06.695 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 181201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-28T13:44:06.696 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/ 2024-05-28T13:44:06.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 171409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-28T13:44:06.696 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/ 2024-05-28T13:44:06.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 58241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-28T13:44:06.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 50113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-28T13:44:06.696 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/ 2024-05-28T13:44:06.696 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-05-28T13:44:06.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 300793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-28T13:44:06.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 105681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-28T13:44:06.696 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-05-28T13:44:06.696 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 885057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-28T13:44:06.697 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/ 2024-05-28T13:44:06.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 52297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-28T13:44:06.698 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/ 2024-05-28T13:44:06.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 81633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-28T13:44:06.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 100681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-28T13:44:06.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-28T13:44:06.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 264145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-28T13:44:06.698 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/samsung/ 2024-05-28T13:44:06.698 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-05-28T13:44:06.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 142401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-05-28T13:44:06.698 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/ 2024-05-28T13:44:06.698 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1350201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-28T13:44:06.700 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/ 2024-05-28T13:44:06.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 57945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-28T13:44:06.700 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/ 2024-05-28T13:44:06.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 66497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-28T13:44:06.700 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 80057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-28T13:44:06.701 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/ 2024-05-28T13:44:06.701 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 74281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-28T13:44:06.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 79449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-28T13:44:06.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 100777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-28T13:44:06.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 83433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-28T13:44:06.702 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/ 2024-05-28T13:44:06.702 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-05-28T13:44:06.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-05-28T13:44:06.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 46977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-28T13:44:06.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-05-28T13:44:06.702 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 644329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-28T13:44:06.702 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/ 2024-05-28T13:44:06.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 128489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-28T13:44:06.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 207985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-28T13:44:06.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 82625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-28T13:44:06.703 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 91345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-28T13:44:06.711 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/ 2024-05-28T13:44:06.712 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 59793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-28T13:44:06.712 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/ 2024-05-28T13:44:06.712 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 110721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-28T13:44:06.712 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/ 2024-05-28T13:44:06.712 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 98681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-28T13:44:06.712 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 102505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-28T13:44:06.712 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/ 2024-05-28T13:44:06.712 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 61489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-28T13:44:06.712 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-28T13:44:06.712 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/ 2024-05-28T13:44:06.712 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 69905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-28T13:44:06.712 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/fddi/ 2024-05-28T13:44:06.712 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 49441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/fddi/defxx.ko 2024-05-28T13:44:06.713 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/fddi/skfp/ 2024-05-28T13:44:06.713 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 275713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/fddi/skfp/skfp.ko 2024-05-28T13:44:06.713 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 91665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-28T13:44:06.713 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/ 2024-05-28T13:44:06.713 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 290441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-28T13:44:06.713 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/ 2024-05-28T13:44:06.713 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-28T13:44:06.713 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-28T13:44:06.713 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ 2024-05-28T13:44:06.713 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 108313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-28T13:44:06.713 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 78057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-28T13:44:06.713 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-28T13:44:06.713 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/ 2024-05-28T13:44:06.714 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-28T13:44:06.714 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-28T13:44:06.714 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-gpio.ko 2024-05-28T13:44:06.714 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-28T13:44:06.714 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-28T13:44:06.714 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-28T13:44:06.714 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 62673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-28T13:44:06.714 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-28T13:44:06.714 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/ 2024-05-28T13:44:06.714 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-28T13:44:06.714 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 48233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-28T13:44:06.714 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ 2024-05-28T13:44:06.714 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-28T13:44:06.714 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 61977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-28T13:44:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 52857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-28T13:44:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-28T13:44:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 53401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-28T13:44:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-28T13:44:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-28T13:44:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-28T13:44:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-28T13:44:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-28T13:44:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 103369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-28T13:44:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 96017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-28T13:44:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-28T13:44:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 121097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-28T13:44:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-28T13:44:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 62105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-28T13:44:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-28T13:44:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-28T13:44:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-28T13:44:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/plip/ 2024-05-28T13:44:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 55345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/plip/plip.ko 2024-05-28T13:44:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ 2024-05-28T13:44:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-28T13:44:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-28T13:44:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-28T13:44:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-28T13:44:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-28T13:44:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 53225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-28T13:44:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-28T13:44:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-28T13:44:06.717 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sb1000.ko 2024-05-28T13:44:06.717 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/ 2024-05-28T13:44:06.717 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 52065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-28T13:44:06.717 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-28T13:44:06.717 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 80153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-28T13:44:06.717 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/ 2024-05-28T13:44:06.717 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 137313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-28T13:44:06.717 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-28T13:44:06.717 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-28T13:44:06.717 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-28T13:44:06.717 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-28T13:44:06.717 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-28T13:44:06.717 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 108081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-28T13:44:06.717 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/ 2024-05-28T13:44:06.718 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 176481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-28T13:44:06.732 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/ 2024-05-28T13:44:06.732 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 384329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-28T13:44:06.732 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/ 2024-05-28T13:44:06.732 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 175513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-28T13:44:06.732 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ 2024-05-28T13:44:06.732 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-28T13:44:06.732 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/ 2024-05-28T13:44:06.732 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/ 2024-05-28T13:44:06.732 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 335489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-28T13:44:06.732 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 127497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-28T13:44:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/ 2024-05-28T13:44:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 168113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-28T13:44:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-28T13:44:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 103881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-28T13:44:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-28T13:44:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/ 2024-05-28T13:44:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/ 2024-05-28T13:44:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-28T13:44:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-05-28T13:44:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-05-28T13:44:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pci-stub.ko 2024-05-28T13:44:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 55321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-28T13:44:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/ 2024-05-28T13:44:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-28T13:44:06.734 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 179937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pcmcia.ko 2024-05-28T13:44:06.734 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 78145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pcmcia_core.ko 2024-05-28T13:44:06.734 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-05-28T13:44:06.734 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-28T13:44:06.734 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 133745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-28T13:44:06.734 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/phy/ 2024-05-28T13:44:06.734 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/phy/broadcom/ 2024-05-28T13:44:06.734 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-05-28T13:44:06.734 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/phy/samsung/ 2024-05-28T13:44:06.734 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-05-28T13:44:06.734 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/ 2024-05-28T13:44:06.734 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/ 2024-05-28T13:44:06.734 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 78185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-28T13:44:06.735 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/ 2024-05-28T13:44:06.735 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/ 2024-05-28T13:44:06.735 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 61025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-05-28T13:44:06.735 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/cros_ec.ko 2024-05-28T13:44:06.735 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-05-28T13:44:06.735 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-05-28T13:44:06.735 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-05-28T13:44:06.735 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-05-28T13:44:06.735 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-05-28T13:44:06.735 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/ 2024-05-28T13:44:06.735 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 87929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-28T13:44:06.735 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-28T13:44:06.735 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 48249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-28T13:44:06.735 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 73857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-28T13:44:06.736 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-28T13:44:06.736 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 164761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-28T13:44:06.736 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 46177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-28T13:44:06.736 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 62065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-28T13:44:06.736 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-28T13:44:06.736 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 51873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-28T13:44:06.736 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-28T13:44:06.736 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-28T13:44:06.736 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/ibm_rtl.ko 2024-05-28T13:44:06.736 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/ 2024-05-28T13:44:06.736 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-28T13:44:06.736 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-05-28T13:44:06.736 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 53993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-28T13:44:06.737 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-28T13:44:06.737 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-28T13:44:06.737 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-28T13:44:06.737 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 51889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-28T13:44:06.737 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-28T13:44:06.737 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-28T13:44:06.737 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-28T13:44:06.737 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-28T13:44:06.737 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-28T13:44:06.737 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 61409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-28T13:44:06.737 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/ 2024-05-28T13:44:06.737 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/ 2024-05-28T13:44:06.737 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/88pm860x_battery.ko 2024-05-28T13:44:06.737 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/88pm860x_charger.ko 2024-05-28T13:44:06.738 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 68825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/bq2415x_charger.ko 2024-05-28T13:44:06.738 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 73065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/bq24190_charger.ko 2024-05-28T13:44:06.738 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/bq24735-charger.ko 2024-05-28T13:44:06.738 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-05-28T13:44:06.738 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/da9030_battery.ko 2024-05-28T13:44:06.738 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/da9052-battery.ko 2024-05-28T13:44:06.738 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/ds2760_battery.ko 2024-05-28T13:44:06.738 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/ds2780_battery.ko 2024-05-28T13:44:06.738 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/ds2781_battery.ko 2024-05-28T13:44:06.738 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/ds2782_battery.ko 2024-05-28T13:44:06.738 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/gpio-charger.ko 2024-05-28T13:44:06.738 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/isp1704_charger.ko 2024-05-28T13:44:06.738 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/lp8727_charger.ko 2024-05-28T13:44:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/max14577_charger.ko 2024-05-28T13:44:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 37825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/max17042_battery.ko 2024-05-28T13:44:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/max8903_charger.ko 2024-05-28T13:44:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/max8925_power.ko 2024-05-28T13:44:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/pcf50633-charger.ko 2024-05-28T13:44:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/sbs-battery.ko 2024-05-28T13:44:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/test_power.ko 2024-05-28T13:44:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/tps65090-charger.ko 2024-05-28T13:44:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/wm831x_backup.ko 2024-05-28T13:44:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/wm831x_power.ko 2024-05-28T13:44:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/power/supply/wm8350_power.ko 2024-05-28T13:44:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/ 2024-05-28T13:44:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 73289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-28T13:44:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-28T13:44:06.740 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/ 2024-05-28T13:44:06.740 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/ 2024-05-28T13:44:06.740 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-28T13:44:06.740 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-28T13:44:06.740 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-28T13:44:06.740 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 49801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-28T13:44:06.740 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ 2024-05-28T13:44:06.740 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 91369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-28T13:44:06.740 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-28T13:44:06.740 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/ 2024-05-28T13:44:06.740 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/ 2024-05-28T13:44:06.740 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/ 2024-05-28T13:44:06.740 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 67473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-28T13:44:06.740 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 47129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-28T13:44:06.741 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/ 2024-05-28T13:44:06.741 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-88pm80x.ko 2024-05-28T13:44:06.741 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-88pm860x.ko 2024-05-28T13:44:06.741 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-28T13:44:06.741 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-da9052.ko 2024-05-28T13:44:06.741 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-da9055.ko 2024-05-28T13:44:06.741 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-da9063.ko 2024-05-28T13:44:06.741 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-28T13:44:06.741 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 82217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-28T13:44:06.741 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-28T13:44:06.741 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-28T13:44:06.741 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-28T13:44:06.741 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-28T13:44:06.741 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-28T13:44:06.742 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-28T13:44:06.742 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-28T13:44:06.742 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-28T13:44:06.742 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-28T13:44:06.742 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-28T13:44:06.742 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-28T13:44:06.742 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-lp8788.ko 2024-05-28T13:44:06.742 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 46905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-28T13:44:06.742 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-28T13:44:06.742 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-28T13:44:06.742 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t86.ko 2024-05-28T13:44:06.742 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-28T13:44:06.742 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max8907.ko 2024-05-28T13:44:06.743 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max8925.ko 2024-05-28T13:44:06.743 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max8997.ko 2024-05-28T13:44:06.743 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max8998.ko 2024-05-28T13:44:06.743 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-05-28T13:44:06.743 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-28T13:44:06.743 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-palmas.ko 2024-05-28T13:44:06.743 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 56769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-28T13:44:06.743 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf50633.ko 2024-05-28T13:44:06.743 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-28T13:44:06.743 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-28T13:44:06.743 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-28T13:44:06.743 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-28T13:44:06.743 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rc5t583.ko 2024-05-28T13:44:06.744 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-28T13:44:06.744 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-28T13:44:06.744 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-28T13:44:06.744 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-28T13:44:06.744 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-28T13:44:06.744 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-s35390a.ko 2024-05-28T13:44:06.744 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-28T13:44:06.744 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-tps6586x.ko 2024-05-28T13:44:06.744 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-tps65910.ko 2024-05-28T13:44:06.744 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-wm831x.ko 2024-05-28T13:44:06.744 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-wm8350.ko 2024-05-28T13:44:06.744 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-28T13:44:06.744 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ 2024-05-28T13:44:06.744 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 90369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-28T13:44:06.745 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 69585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-28T13:44:06.745 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 65937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-28T13:44:06.745 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 164473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-28T13:44:06.745 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-28T13:44:06.745 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/ 2024-05-28T13:44:06.745 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 340689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-28T13:44:06.745 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 120393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-28T13:44:06.745 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/ 2024-05-28T13:44:06.745 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 468049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-28T13:44:06.745 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 380633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-28T13:44:06.745 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic94xx/ 2024-05-28T13:44:06.745 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 243649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-05-28T13:44:06.745 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-28T13:44:06.745 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/ 2024-05-28T13:44:06.746 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 135601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-28T13:44:06.746 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-28T13:44:06.746 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/ 2024-05-28T13:44:06.746 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 304969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-28T13:44:06.746 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/ 2024-05-28T13:44:06.746 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1527017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-28T13:44:06.749 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/ 2024-05-28T13:44:06.749 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 297769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-28T13:44:06.750 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/ 2024-05-28T13:44:06.750 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 144377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-28T13:44:06.750 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 47449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-28T13:44:06.750 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/ 2024-05-28T13:44:06.750 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 390657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-28T13:44:06.750 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/ 2024-05-28T13:44:06.750 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-05-28T13:44:06.750 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 88873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-28T13:44:06.750 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-05-28T13:44:06.750 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 117473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-28T13:44:06.750 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 173009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-28T13:44:06.750 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 91497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-28T13:44:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 45393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-28T13:44:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/ 2024-05-28T13:44:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 267521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-28T13:44:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 78961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-28T13:44:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/ 2024-05-28T13:44:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 95321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-28T13:44:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 194001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-28T13:44:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fdomain.ko 2024-05-28T13:44:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/ 2024-05-28T13:44:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 366457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-28T13:44:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 251497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-28T13:44:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 65689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-28T13:44:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 57089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-28T13:44:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/imm.ko 2024-05-28T13:44:06.752 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 55145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-28T13:44:06.752 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 327833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-28T13:44:06.752 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 129777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-28T13:44:06.752 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/ 2024-05-28T13:44:06.752 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 397657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-28T13:44:06.752 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-28T13:44:06.752 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 63113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-28T13:44:06.752 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/ 2024-05-28T13:44:06.752 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 386801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-28T13:44:06.752 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 178169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-28T13:44:06.752 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 64801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-28T13:44:06.752 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/ 2024-05-28T13:44:06.752 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 273953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-28T13:44:06.752 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/ 2024-05-28T13:44:06.753 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 2735913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-28T13:44:06.754 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/ 2024-05-28T13:44:06.754 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 88521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-28T13:44:06.754 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-28T13:44:06.754 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 389089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-28T13:44:06.754 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 104761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-28T13:44:06.754 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/ 2024-05-28T13:44:06.754 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 836897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-28T13:44:06.754 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/ 2024-05-28T13:44:06.754 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 158705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-28T13:44:06.755 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 96529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-28T13:44:06.755 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pcmcia/ 2024-05-28T13:44:06.755 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 101609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-05-28T13:44:06.755 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-05-28T13:44:06.756 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-05-28T13:44:06.756 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-05-28T13:44:06.756 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/ 2024-05-28T13:44:06.756 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 443713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-28T13:44:06.756 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 152025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-28T13:44:06.757 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ppa.ko 2024-05-28T13:44:06.757 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 66969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-28T13:44:06.757 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/ 2024-05-28T13:44:06.757 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 2324713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-28T13:44:06.761 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 103593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-28T13:44:06.761 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/ 2024-05-28T13:44:06.762 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 697281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-28T13:44:06.763 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qlogicfas408.ko 2024-05-28T13:44:06.763 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-28T13:44:06.763 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 322857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-28T13:44:06.764 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 219849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-28T13:44:06.764 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 357145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-28T13:44:06.765 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 121177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-28T13:44:06.765 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 93473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-28T13:44:06.765 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 55393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-28T13:44:06.765 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-28T13:44:06.766 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 138113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-28T13:44:06.766 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 57393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-28T13:44:06.767 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/ 2024-05-28T13:44:06.767 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 205001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-28T13:44:06.767 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 55465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-28T13:44:06.767 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 59113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-28T13:44:06.767 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 48769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-28T13:44:06.767 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 59409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/xen-scsifront.ko 2024-05-28T13:44:06.767 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ 2024-05-28T13:44:06.767 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 174865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-28T13:44:06.768 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/ 2024-05-28T13:44:06.768 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/ 2024-05-28T13:44:06.768 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 920073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-28T13:44:06.769 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/ 2024-05-28T13:44:06.769 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 60313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-28T13:44:06.769 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 48121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-28T13:44:06.769 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 54225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-28T13:44:06.769 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1170873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-28T13:44:06.771 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 51889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-28T13:44:06.771 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 149777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-28T13:44:06.775 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/ 2024-05-28T13:44:06.775 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 90033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-28T13:44:06.775 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/ 2024-05-28T13:44:06.775 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/ 2024-05-28T13:44:06.775 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/ 2024-05-28T13:44:06.775 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-28T13:44:06.776 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-28T13:44:06.776 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-28T13:44:06.776 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-28T13:44:06.776 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-28T13:44:06.776 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-28T13:44:06.776 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-28T13:44:06.776 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-28T13:44:06.776 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-28T13:44:06.776 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-28T13:44:06.776 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-28T13:44:06.776 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-28T13:44:06.776 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 45993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-28T13:44:06.776 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-28T13:44:06.776 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-28T13:44:06.777 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-05-28T13:44:06.777 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-28T13:44:06.777 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-28T13:44:06.777 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-28T13:44:06.777 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ 2024-05-28T13:44:06.777 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ 2024-05-28T13:44:06.777 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 99289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-28T13:44:06.777 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 69721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/moxa.ko 2024-05-28T13:44:06.777 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 61641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/mxser.ko 2024-05-28T13:44:06.777 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 134329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-28T13:44:06.777 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 37897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-28T13:44:06.777 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 78961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-28T13:44:06.778 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/ 2024-05-28T13:44:06.778 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/ 2024-05-28T13:44:06.778 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-05-28T13:44:06.778 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 65113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-28T13:44:06.778 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-05-28T13:44:06.778 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/altera_uart.ko 2024-05-28T13:44:06.778 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-28T13:44:06.778 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 91593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-05-28T13:44:06.778 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/ 2024-05-28T13:44:06.778 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 120553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-28T13:44:06.778 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/rp2.ko 2024-05-28T13:44:06.778 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 65633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/sc16is7xx.ko 2024-05-28T13:44:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-28T13:44:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-28T13:44:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-28T13:44:06.789 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/ 2024-05-28T13:44:06.789 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ 2024-05-28T13:44:06.789 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 693057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-28T13:44:06.789 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ 2024-05-28T13:44:06.789 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-28T13:44:06.789 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-05-28T13:44:06.789 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/ 2024-05-28T13:44:06.789 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 55609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-28T13:44:06.790 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-28T13:44:06.790 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-28T13:44:06.790 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_dmem_genirq.ko 2024-05-28T13:44:06.790 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_mf624.ko 2024-05-28T13:44:06.790 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_netx.ko 2024-05-28T13:44:06.790 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-28T13:44:06.790 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-05-28T13:44:06.790 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-28T13:44:06.790 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/ 2024-05-28T13:44:06.790 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/c67x00/ 2024-05-28T13:44:06.790 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 90297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/c67x00/c67x00.ko 2024-05-28T13:44:06.790 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/chipidea/ 2024-05-28T13:44:06.790 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 189169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-05-28T13:44:06.791 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-05-28T13:44:06.791 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-05-28T13:44:06.791 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-05-28T13:44:06.791 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-05-28T13:44:06.791 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/ 2024-05-28T13:44:06.791 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 101537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-28T13:44:06.791 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 63017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-28T13:44:06.791 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 56713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-28T13:44:06.791 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 107409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-28T13:44:06.791 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/common/ 2024-05-28T13:44:06.791 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/common/ulpi.ko 2024-05-28T13:44:06.791 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/dwc2/ 2024-05-28T13:44:06.791 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-05-28T13:44:06.791 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/dwc3/ 2024-05-28T13:44:06.792 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-05-28T13:44:06.792 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-05-28T13:44:06.792 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 575217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/dwc3/dwc3.ko 2024-05-28T13:44:06.792 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/ 2024-05-28T13:44:06.792 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/ 2024-05-28T13:44:06.792 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 72241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/u_ether.ko 2024-05-28T13:44:06.792 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 61945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/u_serial.ko 2024-05-28T13:44:06.792 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-05-28T13:44:06.792 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 49729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-05-28T13:44:06.792 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-05-28T13:44:06.792 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-05-28T13:44:06.792 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 131513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-05-28T13:44:06.792 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 62313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-05-28T13:44:06.793 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 146601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-05-28T13:44:06.793 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 70569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-05-28T13:44:06.793 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-05-28T13:44:06.793 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 37641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-05-28T13:44:06.793 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 70009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-05-28T13:44:06.793 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 101233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-05-28T13:44:06.793 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-05-28T13:44:06.793 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 71241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-05-28T13:44:06.793 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 84929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-05-28T13:44:06.793 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/ 2024-05-28T13:44:06.793 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-05-28T13:44:06.793 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-05-28T13:44:06.793 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-05-28T13:44:06.793 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-05-28T13:44:06.794 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-05-28T13:44:06.794 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-05-28T13:44:06.794 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-05-28T13:44:06.794 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-05-28T13:44:06.794 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-05-28T13:44:06.794 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-05-28T13:44:06.794 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-05-28T13:44:06.794 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-05-28T13:44:06.794 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 78065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-05-28T13:44:06.794 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-05-28T13:44:06.794 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 204009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/libcomposite.ko 2024-05-28T13:44:06.794 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/ 2024-05-28T13:44:06.795 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-05-28T13:44:06.795 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/bdc/ 2024-05-28T13:44:06.795 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 182537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-05-28T13:44:06.795 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 45537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-05-28T13:44:06.795 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 66505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-05-28T13:44:06.795 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 85209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-05-28T13:44:06.795 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 69305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-05-28T13:44:06.795 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 90305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/net2272.ko 2024-05-28T13:44:06.795 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 90353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/net2280.ko 2024-05-28T13:44:06.795 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 94833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-05-28T13:44:06.795 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 80513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-05-28T13:44:06.795 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 87457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-05-28T13:44:06.795 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 95433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 195953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/ 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/bcma-hcd.ko 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 108313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/isp116x-hcd.ko 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 104433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 102265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 60097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811_cs.ko 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/ssb-hcd.ko 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/ 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ 2024-05-28T13:44:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 47345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-28T13:44:06.797 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-28T13:44:06.797 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-05-28T13:44:06.797 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/cytherm.ko 2024-05-28T13:44:06.797 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ehset.ko 2024-05-28T13:44:06.797 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-28T13:44:06.797 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-28T13:44:06.797 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-28T13:44:06.797 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-28T13:44:06.797 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-28T13:44:06.797 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-28T13:44:06.797 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-28T13:44:06.797 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-28T13:44:06.797 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/lvstest.ko 2024-05-28T13:44:06.797 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/ 2024-05-28T13:44:06.798 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 85305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-28T13:44:06.798 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-28T13:44:06.798 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-28T13:44:06.798 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-28T13:44:06.798 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-28T13:44:06.798 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 84257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbtest.ko 2024-05-28T13:44:06.798 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 52057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-28T13:44:06.798 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-28T13:44:06.798 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/mon/ 2024-05-28T13:44:06.798 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 91505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/mon/usbmon.ko 2024-05-28T13:44:06.798 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/musb/ 2024-05-28T13:44:06.798 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 396169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/musb/musb_hdrc.ko 2024-05-28T13:44:06.799 INFO:teuthology.orchestra.run.smithi042.stderr:done 2024-05-28T13:44:06.814 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/ 2024-05-28T13:44:06.815 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-28T13:44:06.815 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-05-28T13:44:06.815 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-isp1301.ko 2024-05-28T13:44:06.815 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-tahvo.ko 2024-05-28T13:44:06.815 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ 2024-05-28T13:44:06.815 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-28T13:44:06.815 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-28T13:44:06.815 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-28T13:44:06.815 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-28T13:44:06.815 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 79673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-28T13:44:06.815 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-28T13:44:06.816 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 66865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-28T13:44:06.816 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 58321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-28T13:44:06.816 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-28T13:44:06.816 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 53001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/f81232.ko 2024-05-28T13:44:06.816 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 188601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-28T13:44:06.816 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 61481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-28T13:44:06.816 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 152297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-28T13:44:06.816 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 142825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-28T13:44:06.816 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 69209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-28T13:44:06.816 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-28T13:44:06.816 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-28T13:44:06.816 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 66353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-28T13:44:06.816 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 122321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-28T13:44:06.816 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-28T13:44:06.817 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-28T13:44:06.817 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-28T13:44:06.817 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-28T13:44:06.817 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/metro-usb.ko 2024-05-28T13:44:06.817 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 79673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-28T13:44:06.817 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 89961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-28T13:44:06.817 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 65065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mxuport.ko 2024-05-28T13:44:06.817 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-28T13:44:06.817 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-28T13:44:06.817 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-28T13:44:06.817 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 197201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-28T13:44:06.817 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 45329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-28T13:44:06.817 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 61753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-28T13:44:06.817 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-28T13:44:06.818 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 39977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-28T13:44:06.818 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-28T13:44:06.818 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-28T13:44:06.818 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 59337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-28T13:44:06.818 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-28T13:44:06.818 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-28T13:44:06.818 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-28T13:44:06.818 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 70489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-28T13:44:06.818 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-28T13:44:06.818 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-28T13:44:06.818 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 39313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-28T13:44:06.818 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 151121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usbserial.ko 2024-05-28T13:44:06.818 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/wishbone-serial.ko 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 64041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-28T13:44:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-28T13:44:06.820 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-28T13:44:06.820 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-28T13:44:06.820 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-28T13:44:06.820 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 201145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-28T13:44:06.820 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/ 2024-05-28T13:44:06.820 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 76025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-28T13:44:06.820 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 85233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-28T13:44:06.820 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 122233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-28T13:44:06.820 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/ 2024-05-28T13:44:06.820 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/ 2024-05-28T13:44:06.820 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 206897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-28T13:44:06.820 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-28T13:44:06.820 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 134241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-28T13:44:06.821 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 93665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-28T13:44:06.821 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/ 2024-05-28T13:44:06.821 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 138641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-28T13:44:06.821 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-28T13:44:06.821 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 71273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-28T13:44:06.821 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 107897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-28T13:44:06.821 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/ 2024-05-28T13:44:06.821 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/ 2024-05-28T13:44:06.821 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-05-28T13:44:06.821 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/aat2870_bl.ko 2024-05-28T13:44:06.821 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/adp5520_bl.ko 2024-05-28T13:44:06.821 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/adp8860_bl.ko 2024-05-28T13:44:06.821 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 51737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/adp8870_bl.ko 2024-05-28T13:44:06.821 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-28T13:44:06.822 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/as3711_bl.ko 2024-05-28T13:44:06.822 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/bd6107.ko 2024-05-28T13:44:06.822 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/da903x_bl.ko 2024-05-28T13:44:06.822 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/da9052_bl.ko 2024-05-28T13:44:06.822 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/gpio_backlight.ko 2024-05-28T13:44:06.822 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/kb3886_bl.ko 2024-05-28T13:44:06.822 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-28T13:44:06.822 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lm3533_bl.ko 2024-05-28T13:44:06.822 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lm3639_bl.ko 2024-05-28T13:44:06.822 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lv5207lp.ko 2024-05-28T13:44:06.822 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/max8925_bl.ko 2024-05-28T13:44:06.822 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/pandora_bl.ko 2024-05-28T13:44:06.822 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-05-28T13:44:06.822 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-28T13:44:06.823 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/wm831x_bl.ko 2024-05-28T13:44:06.823 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/ 2024-05-28T13:44:06.823 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ 2024-05-28T13:44:06.823 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-28T13:44:06.823 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 46793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-28T13:44:06.823 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/matrox_w1.ko 2024-05-28T13:44:06.823 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/w1-gpio.ko 2024-05-28T13:44:06.823 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/ 2024-05-28T13:44:06.823 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-28T13:44:06.823 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-28T13:44:06.823 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-28T13:44:06.823 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-28T13:44:06.823 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-28T13:44:06.823 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-28T13:44:06.824 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-28T13:44:06.824 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-28T13:44:06.824 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-28T13:44:06.824 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-28T13:44:06.824 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 70361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-28T13:44:06.824 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 129409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-28T13:44:06.824 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ 2024-05-28T13:44:06.824 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/acquirewdt.ko 2024-05-28T13:44:06.824 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/advantechwdt.ko 2024-05-28T13:44:06.824 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-28T13:44:06.824 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-28T13:44:06.824 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/cpu5wdt.ko 2024-05-28T13:44:06.824 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/da9052_wdt.ko 2024-05-28T13:44:06.824 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/da9055_wdt.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/da9063_wdt.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/dw_wdt.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/eurotechwdt.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/kempld_wdt.ko 2024-05-28T13:44:06.825 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-28T13:44:06.826 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/mena21_wdt.ko 2024-05-28T13:44:06.826 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-05-28T13:44:06.826 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-28T13:44:06.826 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-05-28T13:44:06.826 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pc87413_wdt.ko 2024-05-28T13:44:06.826 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 46945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-28T13:44:06.826 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 37185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-28T13:44:06.826 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/retu_wdt.ko 2024-05-28T13:44:06.826 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-05-28T13:44:06.826 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-05-28T13:44:06.826 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-28T13:44:06.826 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sc1200wdt.ko 2024-05-28T13:44:06.826 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-28T13:44:06.826 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-05-28T13:44:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-28T13:44:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-28T13:44:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/twl4030_wdt.ko 2024-05-28T13:44:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-28T13:44:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-28T13:44:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-28T13:44:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-28T13:44:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wafer5823wdt.ko 2024-05-28T13:44:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-28T13:44:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wm831x_wdt.ko 2024-05-28T13:44:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wm8350_wdt.ko 2024-05-28T13:44:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-28T13:44:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/ 2024-05-28T13:44:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-28T13:44:06.828 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-28T13:44:06.828 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 55705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-28T13:44:06.828 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/ 2024-05-28T13:44:06.828 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 213865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-28T13:44:06.828 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 47673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-28T13:44:06.828 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 85705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-scsiback.ko 2024-05-28T13:44:06.828 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/ 2024-05-28T13:44:06.828 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23417 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-28T13:44:06.828 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ 2024-05-28T13:44:06.828 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/ 2024-05-28T13:44:06.828 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 194449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-28T13:44:06.828 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/adfs/ 2024-05-28T13:44:06.828 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 78049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/adfs/adfs.ko 2024-05-28T13:44:06.829 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/ 2024-05-28T13:44:06.829 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 197801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-28T13:44:06.829 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/afs/ 2024-05-28T13:44:06.829 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1260937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/afs/kafs.ko 2024-05-28T13:44:06.829 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/autofs/ 2024-05-28T13:44:06.829 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 127337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/autofs/autofs4.ko 2024-05-28T13:44:06.829 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/ 2024-05-28T13:44:06.829 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 80481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-28T13:44:06.829 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/bfs/ 2024-05-28T13:44:06.829 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 50457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/bfs/bfs.ko 2024-05-28T13:44:06.829 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 45553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-28T13:44:06.829 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/ 2024-05-28T13:44:06.829 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 4491713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-28T13:44:06.832 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/ 2024-05-28T13:44:06.832 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 376801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-28T13:44:06.832 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ 2024-05-28T13:44:06.832 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1586617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-28T13:44:06.832 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/ 2024-05-28T13:44:06.832 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 130017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-28T13:44:06.832 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/configfs/ 2024-05-28T13:44:06.833 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 139049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/configfs/configfs.ko 2024-05-28T13:44:06.833 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/ 2024-05-28T13:44:06.833 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-28T13:44:06.833 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/ 2024-05-28T13:44:06.833 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 813137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-28T13:44:06.833 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/efs/ 2024-05-28T13:44:06.833 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 46377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/efs/efs.ko 2024-05-28T13:44:06.833 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/ 2024-05-28T13:44:06.833 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1906713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-28T13:44:06.833 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/ 2024-05-28T13:44:06.833 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-28T13:44:06.833 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/freevxfs/ 2024-05-28T13:44:06.833 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 52265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/freevxfs/freevxfs.ko 2024-05-28T13:44:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/ 2024-05-28T13:44:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-28T13:44:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/ 2024-05-28T13:44:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1486393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-28T13:44:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/ 2024-05-28T13:44:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 183217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-28T13:44:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/ 2024-05-28T13:44:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 281993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-28T13:44:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/hpfs/ 2024-05-28T13:44:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 248945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/hpfs/hpfs.ko 2024-05-28T13:44:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/ 2024-05-28T13:44:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 92985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-28T13:44:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/ 2024-05-28T13:44:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 532953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-28T13:44:06.836 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/ 2024-05-28T13:44:06.836 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 361281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-28T13:44:06.836 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/ 2024-05-28T13:44:06.836 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 112569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-28T13:44:06.836 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/ 2024-05-28T13:44:06.836 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 591625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-28T13:44:06.836 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/ 2024-05-28T13:44:06.836 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/ 2024-05-28T13:44:06.836 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 102537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-28T13:44:06.836 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/ 2024-05-28T13:44:06.836 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 74337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-28T13:44:06.837 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/ 2024-05-28T13:44:06.837 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 147673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-28T13:44:06.837 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1320377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-28T13:44:06.837 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 82089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv2.ko 2024-05-28T13:44:06.837 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 151617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-28T13:44:06.838 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 2518385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-28T13:44:06.842 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/ 2024-05-28T13:44:06.842 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-28T13:44:06.842 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-28T13:44:06.842 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/ 2024-05-28T13:44:06.842 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 2021969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-28T13:44:06.845 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/ 2024-05-28T13:44:06.845 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 733969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-28T13:44:06.850 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/ 2024-05-28T13:44:06.850 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-28T13:44:06.850 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-28T13:44:06.850 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-28T13:44:06.851 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-28T13:44:06.851 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-28T13:44:06.851 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-28T13:44:06.851 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-28T13:44:06.851 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-28T13:44:06.851 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-28T13:44:06.851 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-28T13:44:06.851 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-28T13:44:06.851 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ascii.ko 2024-05-28T13:44:06.851 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-28T13:44:06.851 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-28T13:44:06.851 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-28T13:44:06.851 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-28T13:44:06.852 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-28T13:44:06.852 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-28T13:44:06.852 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-28T13:44:06.852 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-28T13:44:06.852 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-28T13:44:06.852 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-28T13:44:06.852 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-28T13:44:06.852 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-28T13:44:06.852 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-28T13:44:06.852 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-28T13:44:06.852 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-28T13:44:06.852 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-28T13:44:06.852 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-28T13:44:06.852 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-28T13:44:06.853 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 95617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-28T13:44:06.853 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 144201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-28T13:44:06.853 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 167129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-28T13:44:06.853 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 119505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-28T13:44:06.853 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-28T13:44:06.853 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-28T13:44:06.853 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-28T13:44:06.853 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-28T13:44:06.853 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-28T13:44:06.853 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-28T13:44:06.853 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-28T13:44:06.853 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-28T13:44:06.853 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-28T13:44:06.853 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-28T13:44:06.854 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-28T13:44:06.854 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-28T13:44:06.854 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-28T13:44:06.854 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-28T13:44:06.854 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-28T13:44:06.854 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-28T13:44:06.854 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-28T13:44:06.854 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ntfs3/ 2024-05-28T13:44:06.854 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 634457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ntfs3/ntfs3.ko 2024-05-28T13:44:06.854 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ 2024-05-28T13:44:06.854 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ 2024-05-28T13:44:06.854 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 302033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-28T13:44:06.855 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ 2024-05-28T13:44:06.855 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 548057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-28T13:44:06.855 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ 2024-05-28T13:44:06.855 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 66345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-28T13:44:06.855 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 3245449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-28T13:44:06.858 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 22105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-28T13:44:06.858 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 39305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-28T13:44:06.859 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-28T13:44:06.859 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/omfs/ 2024-05-28T13:44:06.859 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 67465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/omfs/omfs.ko 2024-05-28T13:44:06.859 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/ 2024-05-28T13:44:06.859 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 462817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-28T13:44:06.860 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ 2024-05-28T13:44:06.860 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 67961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-28T13:44:06.860 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/qnx4/ 2024-05-28T13:44:06.860 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/qnx4/qnx4.ko 2024-05-28T13:44:06.860 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/qnx6/ 2024-05-28T13:44:06.860 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 56073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/qnx6/qnx6.ko 2024-05-28T13:44:06.860 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/quota/ 2024-05-28T13:44:06.861 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/quota/quota_tree.ko 2024-05-28T13:44:06.861 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/quota/quota_v1.ko 2024-05-28T13:44:06.861 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/quota/quota_v2.ko 2024-05-28T13:44:06.861 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/ 2024-05-28T13:44:06.861 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 684713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-28T13:44:06.861 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/ 2024-05-28T13:44:06.861 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-28T13:44:06.862 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/ 2024-05-28T13:44:06.862 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/ 2024-05-28T13:44:06.862 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 3350841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-28T13:44:06.867 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/ 2024-05-28T13:44:06.868 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-28T13:44:06.868 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-28T13:44:06.868 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/ 2024-05-28T13:44:06.868 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 164145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-28T13:44:06.868 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/ 2024-05-28T13:44:06.868 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 115865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-28T13:44:06.868 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/ 2024-05-28T13:44:06.868 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 309937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-28T13:44:06.869 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ 2024-05-28T13:44:06.869 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 198185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-28T13:44:06.869 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/ 2024-05-28T13:44:06.869 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 5192193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-28T13:44:06.879 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/kernel/ 2024-05-28T13:44:06.879 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/kernel/locking/ 2024-05-28T13:44:06.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 97617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/kernel/locking/locktorture.ko 2024-05-28T13:44:06.879 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/kernel/time/ 2024-05-28T13:44:06.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/kernel/time/test_udelay.ko 2024-05-28T13:44:06.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 72657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-28T13:44:06.879 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/ 2024-05-28T13:44:06.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-28T13:44:06.879 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc7.ko 2024-05-28T13:44:06.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-28T13:44:06.880 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/ 2024-05-28T13:44:06.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-28T13:44:06.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-28T13:44:06.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/interval_tree_test.ko 2024-05-28T13:44:06.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-28T13:44:06.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-28T13:44:06.880 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/ 2024-05-28T13:44:06.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-28T13:44:06.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-28T13:44:06.880 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/ 2024-05-28T13:44:06.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-28T13:44:06.880 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/memory-notifier-error-inject.ko 2024-05-28T13:44:06.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/notifier-error-inject.ko 2024-05-28T13:44:06.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/percpu_test.ko 2024-05-28T13:44:06.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/pm-notifier-error-inject.ko 2024-05-28T13:44:06.881 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/ 2024-05-28T13:44:06.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 151945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-28T13:44:06.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/rbtree_test.ko 2024-05-28T13:44:06.881 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/ 2024-05-28T13:44:06.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-28T13:44:06.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/test-kstrtox.ko 2024-05-28T13:44:06.881 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 4912433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/test_bpf.ko 2024-05-28T13:44:06.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 61161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/test_firmware.ko 2024-05-28T13:44:06.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7113 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/test_module.ko 2024-05-28T13:44:06.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/test_user_copy.ko 2024-05-28T13:44:06.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-28T13:44:06.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-28T13:44:06.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-28T13:44:06.885 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/xz/ 2024-05-28T13:44:06.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/lib/xz/xz_dec_test.ko 2024-05-28T13:44:06.885 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/mm/ 2024-05-28T13:44:06.885 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-28T13:44:06.886 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ 2024-05-28T13:44:06.886 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/ 2024-05-28T13:44:06.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 66945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-28T13:44:06.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-28T13:44:06.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-28T13:44:06.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-28T13:44:06.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-28T13:44:06.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-28T13:44:06.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-28T13:44:06.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-28T13:44:06.886 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/802/ 2024-05-28T13:44:06.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-28T13:44:06.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-28T13:44:06.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-28T13:44:06.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-28T13:44:06.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13529 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-28T13:44:06.887 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/ 2024-05-28T13:44:06.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 106697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-28T13:44:06.887 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/ 2024-05-28T13:44:06.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 236769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-28T13:44:06.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-28T13:44:06.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-28T13:44:06.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 39553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-28T13:44:06.887 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/ 2024-05-28T13:44:06.887 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 879593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-28T13:44:06.889 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/ 2024-05-28T13:44:06.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 85225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-28T13:44:06.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1031177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-28T13:44:06.890 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ 2024-05-28T13:44:06.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-28T13:44:06.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-28T13:44:06.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-28T13:44:06.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-28T13:44:06.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-28T13:44:06.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-28T13:44:06.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10329 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-28T13:44:06.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-28T13:44:06.890 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-28T13:44:06.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-28T13:44:06.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-28T13:44:06.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9561 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-28T13:44:06.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-28T13:44:06.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-28T13:44:06.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-28T13:44:06.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-28T13:44:06.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-28T13:44:06.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-28T13:44:06.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-28T13:44:06.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-28T13:44:06.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 70521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-28T13:44:06.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-28T13:44:06.891 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-28T13:44:06.892 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/ 2024-05-28T13:44:06.892 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1243369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-28T13:44:06.903 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/core/ 2024-05-28T13:44:06.903 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 153793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-28T13:44:06.903 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/ 2024-05-28T13:44:06.904 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 244825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-28T13:44:06.904 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-28T13:44:06.904 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 71025 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-28T13:44:06.904 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 74985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-28T13:44:06.904 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/ 2024-05-28T13:44:06.904 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 507649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-28T13:44:06.904 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-28T13:44:06.904 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-28T13:44:06.904 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-28T13:44:06.904 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-28T13:44:06.904 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/hsr/ 2024-05-28T13:44:06.904 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 128577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/hsr/hsr.ko 2024-05-28T13:44:06.904 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ 2024-05-28T13:44:06.905 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ 2024-05-28T13:44:06.905 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 55481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-28T13:44:06.905 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 357369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-28T13:44:06.905 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 59385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-28T13:44:06.905 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ 2024-05-28T13:44:06.905 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-28T13:44:06.905 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 46409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-28T13:44:06.905 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 60217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-28T13:44:06.905 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-28T13:44:06.905 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 65657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-28T13:44:06.905 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 75481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-28T13:44:06.905 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 77713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-28T13:44:06.905 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-28T13:44:06.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-28T13:44:06.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-28T13:44:06.906 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ 2024-05-28T13:44:06.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 45513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-28T13:44:06.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-28T13:44:06.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-28T13:44:06.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 50161 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ip_tables.ko 2024-05-28T13:44:06.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-28T13:44:06.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-05-28T13:44:06.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-28T13:44:06.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-28T13:44:06.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-28T13:44:06.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-05-28T13:44:06.906 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-28T13:44:06.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-28T13:44:06.907 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-28T13:44:06.909 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-28T13:44:06.909 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-28T13:44:06.909 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9697 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-28T13:44:06.909 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-28T13:44:06.909 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-28T13:44:06.909 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-28T13:44:06.909 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-05-28T13:44:06.909 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-28T13:44:06.909 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-28T13:44:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-28T13:44:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-28T13:44:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-28T13:44:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-28T13:44:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-28T13:44:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-28T13:44:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-28T13:44:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-28T13:44:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11657 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-28T13:44:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-28T13:44:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-28T13:44:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-28T13:44:06.910 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-28T13:44:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-28T13:44:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-28T13:44:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-28T13:44:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-28T13:44:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-28T13:44:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 54425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-28T13:44:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-28T13:44:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ 2024-05-28T13:44:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-28T13:44:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 47425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-28T13:44:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17105 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-28T13:44:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 103169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_gre.ko 2024-05-28T13:44:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 118873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-28T13:44:06.911 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-28T13:44:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 71081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-28T13:44:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-28T13:44:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/mip6.ko 2024-05-28T13:44:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ 2024-05-28T13:44:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 50729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-28T13:44:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-05-28T13:44:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11521 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-28T13:44:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-28T13:44:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-28T13:44:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-28T13:44:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-28T13:44:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-28T13:44:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-28T13:44:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-28T13:44:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-28T13:44:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-28T13:44:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-28T13:44:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-28T13:44:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-28T13:44:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-28T13:44:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-28T13:44:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 46297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-28T13:44:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-28T13:44:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 29809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-28T13:44:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-28T13:44:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-28T13:44:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-28T13:44:06.913 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-28T13:44:06.914 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-28T13:44:06.914 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 94081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-28T13:44:06.914 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 30289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-28T13:44:06.914 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-28T13:44:06.914 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/key/ 2024-05-28T13:44:06.914 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 106361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-28T13:44:06.914 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/ 2024-05-28T13:44:06.914 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 153209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-28T13:44:06.914 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-28T13:44:06.914 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-28T13:44:06.914 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-28T13:44:06.914 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-28T13:44:06.914 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-28T13:44:06.915 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 66233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-28T13:44:06.915 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/lapb/ 2024-05-28T13:44:06.915 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 63321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/lapb/lapb.ko 2024-05-28T13:44:06.915 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/ 2024-05-28T13:44:06.915 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-28T13:44:06.915 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 332817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc2.ko 2024-05-28T13:44:06.915 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/ 2024-05-28T13:44:06.915 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 388817 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-28T13:44:06.915 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/ 2024-05-28T13:44:06.915 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-28T13:44:06.915 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ 2024-05-28T13:44:06.915 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ 2024-05-28T13:44:06.915 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 147729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-28T13:44:06.915 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-28T13:44:06.916 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-28T13:44:06.916 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-28T13:44:06.916 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 124033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-28T13:44:06.916 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 124337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-28T13:44:06.916 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 126673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-28T13:44:06.916 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 126465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-28T13:44:06.916 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 137825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-28T13:44:06.916 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 72345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-28T13:44:06.916 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 133673 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-28T13:44:06.916 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 139065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-28T13:44:06.916 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 138257 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-28T13:44:06.916 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 136169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-28T13:44:06.916 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 139889 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-28T13:44:06.917 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 37121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-28T13:44:06.917 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ 2024-05-28T13:44:06.917 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 534033 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-28T13:44:06.935 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14193 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-28T13:44:06.935 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-28T13:44:06.935 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-28T13:44:06.935 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20945 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-28T13:44:06.935 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24569 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-28T13:44:06.935 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-28T13:44:06.935 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-28T13:44:06.935 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12641 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-28T13:44:06.935 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-28T13:44:06.935 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-28T13:44:06.936 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15969 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-28T13:44:06.936 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-28T13:44:06.936 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-28T13:44:06.936 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 55209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-28T13:44:06.936 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 496713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-28T13:44:06.936 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-28T13:44:06.936 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-28T13:44:06.936 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 37857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-28T13:44:06.936 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 134361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-28T13:44:06.936 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24505 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-28T13:44:06.936 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-28T13:44:06.936 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 140793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-28T13:44:06.936 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-28T13:44:06.937 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-28T13:44:06.937 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 79865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-28T13:44:06.937 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12553 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-28T13:44:06.937 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-28T13:44:06.937 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-28T13:44:06.937 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 117385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-28T13:44:06.937 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-28T13:44:06.937 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 46097 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-28T13:44:06.937 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 118337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-28T13:44:06.937 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-28T13:44:06.937 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14177 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-28T13:44:06.937 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13337 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-28T13:44:06.937 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-28T13:44:06.938 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-28T13:44:06.938 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 53385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-28T13:44:06.938 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 811705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-28T13:44:06.938 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 53041 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-28T13:44:06.938 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-28T13:44:06.938 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 37201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-05-28T13:44:06.938 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 49289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-28T13:44:06.938 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-28T13:44:06.938 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 65137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-28T13:44:06.938 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-28T13:44:06.938 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-28T13:44:06.938 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 50649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-28T13:44:06.938 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-28T13:44:06.939 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-28T13:44:06.939 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-28T13:44:06.939 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-28T13:44:06.939 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-28T13:44:06.939 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17913 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-28T13:44:06.939 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-28T13:44:06.939 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-28T13:44:06.939 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16465 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-28T13:44:06.939 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13937 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_osf.ko 2024-05-28T13:44:06.939 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-28T13:44:06.939 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-28T13:44:06.939 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-28T13:44:06.939 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-28T13:44:06.939 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15049 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-28T13:44:06.940 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 24985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_synproxy.ko 2024-05-28T13:44:06.940 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tproxy.ko 2024-05-28T13:44:06.940 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 108385 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/x_tables.ko 2024-05-28T13:44:06.940 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-28T13:44:06.940 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-28T13:44:06.940 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-28T13:44:06.940 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-28T13:44:06.940 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 21857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-28T13:44:06.940 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12281 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-28T13:44:06.940 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11065 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-28T13:44:06.940 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15737 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-28T13:44:06.940 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 45209 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-28T13:44:06.940 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15929 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-28T13:44:06.941 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13009 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-28T13:44:06.941 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13369 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-28T13:44:06.941 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-28T13:44:06.941 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10849 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-28T13:44:06.941 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15353 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-28T13:44:06.941 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-28T13:44:06.941 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-28T13:44:06.941 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16681 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-28T13:44:06.941 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18977 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-28T13:44:06.941 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11761 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-28T13:44:06.941 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20689 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-28T13:44:06.941 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-28T13:44:06.941 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-28T13:44:06.942 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-28T13:44:06.942 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16625 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-28T13:44:06.942 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-28T13:44:06.942 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-28T13:44:06.942 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-28T13:44:06.942 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11953 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-28T13:44:06.942 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-28T13:44:06.942 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-28T13:44:06.942 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15833 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-28T13:44:06.942 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14201 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-28T13:44:06.942 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-28T13:44:06.942 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-28T13:44:06.942 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-28T13:44:06.943 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-28T13:44:06.943 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12649 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-28T13:44:06.943 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-28T13:44:06.943 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 52809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-28T13:44:06.943 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-28T13:44:06.943 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-28T13:44:06.943 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-28T13:44:06.943 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-28T13:44:06.943 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-28T13:44:06.943 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19185 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-28T13:44:06.943 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-28T13:44:06.943 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11513 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-28T13:44:06.943 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-28T13:44:06.944 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-28T13:44:06.944 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-28T13:44:06.944 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 16153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-28T13:44:06.944 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-28T13:44:06.944 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-28T13:44:06.944 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11489 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-28T13:44:06.944 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-28T13:44:06.944 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-28T13:44:06.944 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13545 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-28T13:44:06.944 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-28T13:44:06.944 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 11873 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-28T13:44:06.944 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9265 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-28T13:44:06.944 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-28T13:44:06.945 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-28T13:44:06.945 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36481 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-28T13:44:06.945 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 19769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-28T13:44:06.945 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-28T13:44:06.945 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10841 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-28T13:44:06.945 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10017 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-28T13:44:06.945 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-28T13:44:06.945 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18441 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpudp.ko 2024-05-28T13:44:06.945 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12433 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-28T13:44:06.945 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-28T13:44:06.945 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/ 2024-05-28T13:44:06.945 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 17225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-28T13:44:06.945 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/ 2024-05-28T13:44:06.946 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 14241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-28T13:44:06.946 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/ 2024-05-28T13:44:06.946 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 488457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-28T13:44:06.946 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12001 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-28T13:44:06.946 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 10865 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-28T13:44:06.946 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-28T13:44:06.946 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/ 2024-05-28T13:44:06.946 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15921 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-28T13:44:06.946 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/phonet/ 2024-05-28T13:44:06.946 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 141993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/phonet/phonet.ko 2024-05-28T13:44:06.946 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 68801 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/phonet/pn_pep.ko 2024-05-28T13:44:06.946 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/ 2024-05-28T13:44:06.946 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 318537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-28T13:44:06.946 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 202249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-28T13:44:06.947 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 85073 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-28T13:44:06.947 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/rxrpc/ 2024-05-28T13:44:06.947 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1028129 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/rxrpc/rxrpc.ko 2024-05-28T13:44:06.947 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/ 2024-05-28T13:44:06.947 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38993 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-28T13:44:06.947 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 28713 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-28T13:44:06.947 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 45401 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-28T13:44:06.947 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 27585 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-28T13:44:06.947 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 53457 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-28T13:44:06.947 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38633 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-28T13:44:06.947 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23961 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-28T13:44:06.947 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43409 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-28T13:44:06.947 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 45665 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-28T13:44:06.947 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32121 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-28T13:44:06.948 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 41449 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-28T13:44:06.948 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23217 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_cgroup.ko 2024-05-28T13:44:06.948 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 37297 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-28T13:44:06.948 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 39593 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-28T13:44:06.948 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 49313 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-28T13:44:06.948 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 79137 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-28T13:44:06.948 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8705 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-28T13:44:06.948 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 15377 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-28T13:44:06.948 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 48777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-28T13:44:06.948 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9233 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-28T13:44:06.948 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13361 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-28T13:44:06.948 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 8305 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-28T13:44:06.948 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-28T13:44:06.949 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 25089 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-28T13:44:06.949 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43473 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-28T13:44:06.949 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 43145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-28T13:44:06.949 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 39169 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq_codel.ko 2024-05-28T13:44:06.949 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 42881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-28T13:44:06.949 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 62809 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-28T13:44:06.949 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26577 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-28T13:44:06.949 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 81825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-28T13:44:06.949 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33537 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-28T13:44:06.949 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 36617 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-28T13:44:06.949 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12289 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-28T13:44:06.949 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33057 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-28T13:44:06.949 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 44785 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-28T13:44:06.950 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 32825 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-28T13:44:06.950 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 13497 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-28T13:44:06.950 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 31793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-28T13:44:06.950 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 60241 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-28T13:44:06.950 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 40881 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-28T13:44:06.950 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 33425 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-28T13:44:06.950 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 38321 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-28T13:44:06.950 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 35769 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-28T13:44:06.950 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 34153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-28T13:44:06.950 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/ 2024-05-28T13:44:06.950 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1187153 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-28T13:44:06.950 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23905 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-28T13:44:06.950 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/ 2024-05-28T13:44:06.951 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/ 2024-05-28T13:44:06.951 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 432225 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-28T13:44:06.951 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 70081 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-28T13:44:06.951 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1940753 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-28T13:44:06.951 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/ 2024-05-28T13:44:06.951 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 889729 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-28T13:44:06.951 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/ 2024-05-28T13:44:06.951 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 1122273 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-28T13:44:06.951 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12145 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-28T13:44:06.951 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/ 2024-05-28T13:44:06.951 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 18793 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-28T13:44:06.951 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/ 2024-05-28T13:44:06.951 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 105601 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-28T13:44:06.951 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 90249 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-28T13:44:06.952 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 123745 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-28T13:44:06.952 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 12897 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-28T13:44:06.952 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 20393 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-28T13:44:06.952 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/x25/ 2024-05-28T13:44:06.952 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 165777 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/x25/x25.ko 2024-05-28T13:44:06.952 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/ 2024-05-28T13:44:06.952 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 26721 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_algo.ko 2024-05-28T13:44:06.952 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23345 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-28T13:44:06.952 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 131857 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_user.ko 2024-05-28T13:44:06.952 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 646387 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-28T13:44:06.952 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 630543 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-28T13:44:06.952 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 7706 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-28T13:44:06.952 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 23609 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-28T13:44:06.953 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 9941 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-28T13:44:06.953 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 70990 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-28T13:44:06.953 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 146985 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-28T13:44:06.953 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 232892 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-28T13:44:06.954 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 210 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-28T13:44:06.955 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 72627 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-28T13:44:06.955 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 879 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-28T13:44:06.955 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 258450 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-28T13:44:06.955 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 313019 2024-05-23 11:07 ./lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-28T13:44:06.955 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./usr/ 2024-05-28T13:44:06.955 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./usr/share/ 2024-05-28T13:44:06.955 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./usr/share/doc/ 2024-05-28T13:44:06.955 INFO:teuthology.orchestra.run.smithi063.stdout:drwxr-xr-x root/root 0 2024-05-23 11:07 ./usr/share/doc/linux-image-6.9.0-gb9459a26b088/ 2024-05-28T13:44:06.955 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 171 2024-05-23 11:07 ./usr/share/doc/linux-image-6.9.0-gb9459a26b088/changelog.Debian.gz 2024-05-28T13:44:06.955 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r-- root/root 691 2024-05-23 11:07 ./usr/share/doc/linux-image-6.9.0-gb9459a26b088/copyright 2024-05-28T13:44:06.959 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-28T13:44:06.960 INFO:teuthology.task.kernel:searching for kernel 6.9.0-gb9459a26b088 2024-05-28T13:44:06.960 DEBUG:teuthology.orchestra.run.smithi063:> egrep '(submenu|menuentry.*6.9.0-gb9459a26b088).*{' /boot/grub/grub.cfg 2024-05-28T13:44:07.004 INFO:teuthology.orchestra.run.smithi063.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-28T13:44:07.004 INFO:teuthology.orchestra.run.smithi063.stdout: menuentry 'Ubuntu, with Linux 6.9.0-gb9459a26b088' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-gb9459a26b088-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-28T13:44:07.004 INFO:teuthology.orchestra.run.smithi063.stdout: menuentry 'Ubuntu, with Linux 6.9.0-gb9459a26b088 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-gb9459a26b088-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-28T13:44:07.005 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-05-28T13:44:07.005 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-gb9459a26b088 2024-05-28T13:44:07.006 DEBUG:teuthology.orchestra.run.smithi063:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-05-28T13:44:07.047 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi063.front.sepia.ceph.com to complete... 2024-05-28T13:44:07.449 INFO:teuthology.orchestra.run.smithi063.stderr:Sourcing file `/etc/default/grub' 2024-05-28T13:44:07.450 INFO:teuthology.orchestra.run.smithi063.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-28T13:44:07.480 INFO:teuthology.orchestra.run.smithi063.stderr:Generating grub configuration file ... 2024-05-28T13:44:07.740 INFO:teuthology.orchestra.run.smithi063.stderr:Found linux image: /boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-28T13:44:07.747 INFO:teuthology.orchestra.run.smithi063.stderr:Found initrd image: /boot/initrd.img-6.9.0-gb9459a26b088 2024-05-28T13:44:08.056 INFO:teuthology.orchestra.run.smithi063.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-28T13:44:08.062 INFO:teuthology.orchestra.run.smithi063.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-28T13:44:08.104 INFO:teuthology.orchestra.run.smithi063.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-28T13:44:08.109 INFO:teuthology.orchestra.run.smithi063.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-28T13:44:08.331 INFO:teuthology.orchestra.run.smithi063.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-28T13:44:08.331 INFO:teuthology.orchestra.run.smithi063.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-28T13:44:08.331 INFO:teuthology.orchestra.run.smithi063.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-28T13:44:08.352 INFO:teuthology.orchestra.run.smithi063.stderr:done 2024-05-28T13:44:37.973 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-28T13:44:37.974 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-05-28T13:44:37.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-28T13:44:39.532 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-28T13:44:39.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-28T13:44:39.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-28T13:44:56.355 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-05-28T13:44:57.886 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-28T13:45:05.362 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-05-28T13:45:05.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-28T13:45:06.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-28T13:45:06.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-28T13:45:09.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-28T13:45:21.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-28T13:45:21.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-28T13:45:40.383 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-28T13:45:55.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-28T13:45:55.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-28T13:45:55.837 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-28T13:45:56.814 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-28T13:45:56.814 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-gb9459a26b088"... 2024-05-28T13:45:56.815 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-05-28T13:45:56.869 INFO:teuthology.orchestra.run.smithi063.stdout:6.9.0-gb9459a26b088 2024-05-28T13:45:56.869 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-28T13:45:56.869 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-28T13:45:56.869 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-28T13:45:57.870 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-28T13:45:57.870 DEBUG:teuthology.orchestra.run.smithi063:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-28T13:45:58.088 INFO:teuthology.orchestra.run.smithi063.stdout:ttyS1 2024-05-28T13:45:58.181 DEBUG:teuthology.parallel:result is None 2024-05-28T13:46:05.365 DEBUG:teuthology.orchestra.remote:timed out 2024-05-28T13:46:17.366 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-05-28T13:46:17.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-28T13:46:17.656 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-05-28T13:46:18.281 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-05-28T13:46:18.281 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-gb9459a26b088"... 2024-05-28T13:46:18.281 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-05-28T13:46:18.329 INFO:teuthology.orchestra.run.smithi042.stdout:6.9.0-gb9459a26b088 2024-05-28T13:46:18.329 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-28T13:46:18.329 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-28T13:46:18.330 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-28T13:46:19.330 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-28T13:46:19.330 DEBUG:teuthology.orchestra.run.smithi042:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-28T13:46:19.431 INFO:teuthology.orchestra.run.smithi042.stdout:ttyS1 2024-05-28T13:46:19.491 DEBUG:teuthology.parallel:result is None 2024-05-28T13:46:19.491 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-28T13:46:19.498 INFO:teuthology.task.internal:Creating test directory... 2024-05-28T13:46:19.498 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-28T13:46:19.502 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-28T13:46:19.511 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-28T13:46:19.520 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-28T13:46:19.527 INFO:teuthology.task.internal:Creating archive directory... 2024-05-28T13:46:19.527 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-28T13:46:19.552 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-28T13:46:19.575 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-28T13:46:19.581 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-28T13:46:19.582 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-28T13:46:19.608 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-28T13:46:19.633 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-28T13:46:19.642 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-28T13:46:19.645 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-28T13:46:19.656 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-28T13:46:19.659 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-28T13:46:19.665 INFO:teuthology.task.internal:Configuring sudo... 2024-05-28T13:46:19.665 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-28T13:46:19.692 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-28T13:46:19.723 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-28T13:46:19.733 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-28T13:46:19.733 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-28T13:46:19.756 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-28T13:46:19.778 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-28T13:46:19.814 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-28T13:46:19.866 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:46:19.866 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-28T13:46:19.939 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-28T13:46:19.948 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-28T13:46:19.998 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:46:19.998 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-28T13:46:20.070 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-05-28T13:46:20.073 DEBUG:teuthology.orchestra.run.smithi063:> sudo service rsyslog restart 2024-05-28T13:46:20.208 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-28T13:46:20.216 INFO:teuthology.task.internal:Starting timer... 2024-05-28T13:46:20.216 INFO:teuthology.run_tasks:Running task pcp... 2024-05-28T13:46:20.261 INFO:teuthology.run_tasks:Running task selinux... 2024-05-28T13:46:20.269 DEBUG:teuthology.task.selinux:Excluding smithi042: OS 'ubuntu' does not support SELinux 2024-05-28T13:46:20.269 DEBUG:teuthology.task.selinux:Excluding smithi063: OS 'ubuntu' does not support SELinux 2024-05-28T13:46:20.269 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-28T13:46:20.269 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-05-28T13:46:20.269 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-28T13:46:20.270 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-28T13:46:20.280 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-28T13:46:20.292 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-28T13:46:20.293 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi042.front.sepia.ceph.com,smithi063.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-28T13:54:13.446 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi042.front.sepia.ceph.com'), Remote(name='ubuntu@smithi063.front.sepia.ceph.com')] 2024-05-28T13:54:13.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-05-28T13:54:13.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-28T13:54:13.529 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-05-28T13:54:13.673 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-05-28T13:54:13.673 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-28T13:54:13.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-28T13:54:13.757 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-28T13:54:13.873 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-28T13:54:13.873 INFO:teuthology.run_tasks:Running task clock... 2024-05-28T13:54:13.885 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-28T13:54:13.885 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-28T13:54:13.886 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-28T13:54:13.889 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-28T13:54:13.889 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-28T13:54:13.925 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-28T13:54:13.925 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: Command line: ntpd -gq 2024-05-28T13:54:13.925 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: ---------------------------------------------------- 2024-05-28T13:54:13.925 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: ntp-4 is maintained by Network Time Foundation, 2024-05-28T13:54:13.925 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-28T13:54:13.925 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: corporation. Support and training for ntp-4 are 2024-05-28T13:54:13.925 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: available at https://www.nwtime.org/support 2024-05-28T13:54:13.926 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: ---------------------------------------------------- 2024-05-28T13:54:13.926 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: proto: precision = 0.045 usec (-24) 2024-05-28T13:54:13.926 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: basedate set to 2022-02-04 2024-05-28T13:54:13.926 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: gps base set to 2022-02-06 (week 2196) 2024-05-28T13:54:13.927 INFO:teuthology.orchestra.run.smithi042.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-28T13:54:13.927 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-28T13:54:13.927 INFO:teuthology.orchestra.run.smithi042.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-28T13:54:13.927 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: restrict ::: KOD does nothing without LIMITED. 2024-05-28T13:54:13.928 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: Listen and drop on 0 v6wildcard [::]:123 2024-05-28T13:54:13.928 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-28T13:54:13.928 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: Listen normally on 2 lo 127.0.0.1:123 2024-05-28T13:54:13.929 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: Listen normally on 3 ens1f0 172.21.15.42:123 2024-05-28T13:54:13.929 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: Listen normally on 4 lo [::1]:123 2024-05-28T13:54:13.929 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1440%4]:123 2024-05-28T13:54:13.929 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:13 ntpd[17475]: Listening on routing socket on fd #22 for interface updates 2024-05-28T13:54:13.954 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-28T13:54:13.954 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: Command line: ntpd -gq 2024-05-28T13:54:13.954 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: ---------------------------------------------------- 2024-05-28T13:54:13.954 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: ntp-4 is maintained by Network Time Foundation, 2024-05-28T13:54:13.954 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-28T13:54:13.954 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: corporation. Support and training for ntp-4 are 2024-05-28T13:54:13.955 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: available at https://www.nwtime.org/support 2024-05-28T13:54:13.955 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: ---------------------------------------------------- 2024-05-28T13:54:13.955 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: proto: precision = 0.043 usec (-24) 2024-05-28T13:54:13.955 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: basedate set to 2022-02-04 2024-05-28T13:54:13.955 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: gps base set to 2022-02-06 (week 2196) 2024-05-28T13:54:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-28T13:54:13.956 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-28T13:54:13.956 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: restrict ::: KOD does nothing without LIMITED. 2024-05-28T13:54:13.956 INFO:teuthology.orchestra.run.smithi063.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-28T13:54:13.957 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: Listen and drop on 0 v6wildcard [::]:123 2024-05-28T13:54:13.957 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-28T13:54:13.957 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: Listen normally on 2 lo 127.0.0.1:123 2024-05-28T13:54:13.957 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: Listen normally on 3 enp3s0f1 172.21.15.63:123 2024-05-28T13:54:13.957 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: Listen normally on 4 lo [::1]:123 2024-05-28T13:54:13.957 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:feb6:f8d7%5]:123 2024-05-28T13:54:13.957 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:13 ntpd[17578]: Listening on routing socket on fd #22 for interface updates 2024-05-28T13:54:14.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:14 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:14.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:14 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:14.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:14 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:14.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:14 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:14.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:14 ntpd[17475]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:14.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:14 ntpd[17475]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:14.954 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:14 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:14.954 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:14 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:14.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:14 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:14.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:14 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:14.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:14 ntpd[17578]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:14.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:14 ntpd[17578]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:15.925 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:15 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:15.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:15 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:15.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:15 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:15.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:15 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:15.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:15 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:15.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:15 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:15.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:15 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:15.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:15 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:15.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:15 ntpd[17578]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:15.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:15 ntpd[17578]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:15.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:15 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:15.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:15 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:15.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:15 ntpd[17578]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:15.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:15 ntpd[17578]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:16.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:16 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:16.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:16 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:16 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:16 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:16 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:16 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:16 ntpd[17475]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:16 ntpd[17475]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:16 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:16 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:16 ntpd[17475]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:16 ntpd[17475]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:16.954 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:16 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:16.954 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:16 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:16.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:16 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:16.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:16 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:16.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:16 ntpd[17578]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:16.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:16 ntpd[17578]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:17.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:17 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:17.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:17 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:17.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:17 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:17.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:17 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:17.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:17 ntpd[17475]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:17.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:17 ntpd[17475]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:17.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:17 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:17.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:17 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:17.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:17 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:17 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:17 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:17 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:17 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:17 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:17 ntpd[17578]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:17 ntpd[17578]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:17 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:17 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:17 ntpd[17578]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:17 ntpd[17578]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:18.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:18 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:18.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:18 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:18.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:18 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:18.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:18 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:18.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:18 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:18.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:18 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:18.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:18 ntpd[17475]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:18.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:18 ntpd[17475]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:18.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:18 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:18.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:18 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:18.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:18 ntpd[17475]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:18.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:18 ntpd[17475]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:18.954 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:18 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:18.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:18 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:18.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:18 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:18.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:18 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:18.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:18 ntpd[17578]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:18.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:18 ntpd[17578]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:19.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:19 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:19.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:19 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:19.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:19 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:19.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:19 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:19.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:19 ntpd[17475]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:19.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:19 ntpd[17475]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:19.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:19 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:19.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:19 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:19.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:19 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:19.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:19 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:19.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:19 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:19.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:19 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:19.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:19 ntpd[17578]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:19.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:19 ntpd[17578]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:19.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:19 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:19.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:19 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:19.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:19 ntpd[17578]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:19.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:19 ntpd[17578]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:20.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:20 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:20.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:20 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:20.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:20 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:20.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:20 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:20.926 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:20 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:20.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:20 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:20.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:20 ntpd[17475]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:20.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:20 ntpd[17475]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:20.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:20 ntpd[17475]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:20.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:20 ntpd[17475]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:20.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:20 ntpd[17475]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:20.927 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:20 ntpd[17475]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:20.927 INFO:teuthology.orchestra.run.smithi042.stdout:28 May 13:54:20 ntpd[17475]: ntpd: time slew -0.001291 s 2024-05-28T13:54:20.927 INFO:teuthology.orchestra.run.smithi042.stdout:ntpd: time slew -0.001291s 2024-05-28T13:54:20.928 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:20 ntpd[17475]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-28T13:54:20.928 INFO:teuthology.orchestra.run.smithi042.stderr:28 May 13:54:20 ntpd[17475]: can't open /var/log/ntpstats/loopstats.20240528: Permission denied 2024-05-28T13:54:20.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:20 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:20.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:20 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:20.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:20 ntpd[17578]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-28T13:54:20.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:20 ntpd[17578]: can't open /var/log/ntpstats/rawstats.20240528: Permission denied 2024-05-28T13:54:20.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:20 ntpd[17578]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-28T13:54:20.955 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:20 ntpd[17578]: can't open /var/log/ntpstats/peerstats.20240528: Permission denied 2024-05-28T13:54:20.956 INFO:teuthology.orchestra.run.smithi063.stdout:28 May 13:54:20 ntpd[17578]: ntpd: time slew -0.001003 s 2024-05-28T13:54:20.956 INFO:teuthology.orchestra.run.smithi063.stdout:ntpd: time slew -0.001003s 2024-05-28T13:54:20.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:20 ntpd[17578]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-28T13:54:20.956 INFO:teuthology.orchestra.run.smithi063.stderr:28 May 13:54:20 ntpd[17578]: can't open /var/log/ntpstats/loopstats.20240528: Permission denied 2024-05-28T13:54:21.021 INFO:teuthology.orchestra.run.smithi042.stdout: remote refid st t when poll reach delay offset jitter 2024-05-28T13:54:21.021 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================== 2024-05-28T13:54:21.021 INFO:teuthology.orchestra.run.smithi042.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-28T13:54:21.022 INFO:teuthology.orchestra.run.smithi042.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-28T13:54:21.022 INFO:teuthology.orchestra.run.smithi042.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-28T13:54:21.022 INFO:teuthology.orchestra.run.smithi042.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-28T13:54:21.053 INFO:teuthology.orchestra.run.smithi063.stdout: remote refid st t when poll reach delay offset jitter 2024-05-28T13:54:21.053 INFO:teuthology.orchestra.run.smithi063.stdout:============================================================================== 2024-05-28T13:54:21.053 INFO:teuthology.orchestra.run.smithi063.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-28T13:54:21.053 INFO:teuthology.orchestra.run.smithi063.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-28T13:54:21.053 INFO:teuthology.orchestra.run.smithi063.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-28T13:54:21.054 INFO:teuthology.orchestra.run.smithi063.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-28T13:54:21.055 INFO:teuthology.run_tasks:Running task install... 2024-05-28T13:54:21.063 DEBUG:teuthology.task.install:project ceph 2024-05-28T13:54:21.063 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863'}} 2024-05-28T13:54:21.064 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863'} 2024-05-28T13:54:21.064 INFO:teuthology.task.install:Using flavor: default 2024-05-28T13:54:21.069 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-28T13:54:21.069 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-28T13:54:21.070 DEBUG:teuthology.orchestra.run.smithi042:> sudo apt-key list | grep Ceph 2024-05-28T13:54:21.070 DEBUG:teuthology.orchestra.run.smithi063:> sudo apt-key list | grep Ceph 2024-05-28T13:54:21.174 INFO:teuthology.orchestra.run.smithi042.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-28T13:54:21.200 INFO:teuthology.orchestra.run.smithi063.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-28T13:54:21.225 INFO:teuthology.orchestra.run.smithi042.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-28T13:54:21.226 INFO:teuthology.orchestra.run.smithi042.stdout:uid [ unknown] Ceph.com (release key) 2024-05-28T13:54:21.227 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-28T13:54:21.227 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-28T13:54:21.227 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=9ed025fbf4406fb54980a0647506b15895b6d863 2024-05-28T13:54:21.252 INFO:teuthology.orchestra.run.smithi063.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-28T13:54:21.253 INFO:teuthology.orchestra.run.smithi063.stdout:uid [ unknown] Ceph.com (release key) 2024-05-28T13:54:21.254 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-28T13:54:21.254 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-28T13:54:21.254 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=9ed025fbf4406fb54980a0647506b15895b6d863 2024-05-28T13:54:21.447 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default/ 2024-05-28T13:54:21.447 INFO:teuthology.task.install.deb:Package version is 19.0.0-3154-g9ed025fb-1jammy 2024-05-28T13:54:21.479 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default/ 2024-05-28T13:54:21.479 INFO:teuthology.task.install.deb:Package version is 19.0.0-3154-g9ed025fb-1jammy 2024-05-28T13:54:21.558 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:54:21.558 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-28T13:54:21.578 DEBUG:teuthology.orchestra.run.smithi042:> sudo apt-get update 2024-05-28T13:54:21.590 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:54:21.590 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-28T13:54:21.610 DEBUG:teuthology.orchestra.run.smithi063:> sudo apt-get update 2024-05-28T13:54:21.783 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-28T13:54:21.796 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-28T13:54:21.817 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-28T13:54:21.850 INFO:teuthology.orchestra.run.smithi063.stdout:Ign:1 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy InRelease 2024-05-28T13:54:21.871 INFO:teuthology.orchestra.run.smithi063.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-05-28T13:54:21.880 INFO:teuthology.orchestra.run.smithi042.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy InRelease 2024-05-28T13:54:21.885 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-28T13:54:21.894 INFO:teuthology.orchestra.run.smithi063.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-28T13:54:21.915 INFO:teuthology.orchestra.run.smithi063.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2024-05-28T13:54:21.917 INFO:teuthology.orchestra.run.smithi042.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-05-28T13:54:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-28T13:54:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-28T13:54:21.955 INFO:teuthology.orchestra.run.smithi042.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-28T13:54:21.958 INFO:teuthology.orchestra.run.smithi063.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-05-28T13:54:21.992 INFO:teuthology.orchestra.run.smithi042.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2024-05-28T13:54:21.999 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-28T13:54:22.045 INFO:teuthology.orchestra.run.smithi042.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-05-28T13:54:22.081 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-28T13:54:22.394 INFO:teuthology.orchestra.run.smithi042.stdout:Fetched 29.2 kB in 1s (49.9 kB/s) 2024-05-28T13:54:22.547 INFO:teuthology.orchestra.run.smithi063.stdout:Fetched 29.2 kB in 1s (41.3 kB/s) 2024-05-28T13:54:23.438 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-05-28T13:54:23.453 DEBUG:teuthology.orchestra.run.smithi042:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3154-g9ed025fb-1jammy cephadm=19.0.0-3154-g9ed025fb-1jammy ceph-mds=19.0.0-3154-g9ed025fb-1jammy ceph-mgr=19.0.0-3154-g9ed025fb-1jammy ceph-common=19.0.0-3154-g9ed025fb-1jammy ceph-fuse=19.0.0-3154-g9ed025fb-1jammy ceph-test=19.0.0-3154-g9ed025fb-1jammy ceph-volume=19.0.0-3154-g9ed025fb-1jammy radosgw=19.0.0-3154-g9ed025fb-1jammy python3-rados=19.0.0-3154-g9ed025fb-1jammy python3-rgw=19.0.0-3154-g9ed025fb-1jammy python3-cephfs=19.0.0-3154-g9ed025fb-1jammy python3-rbd=19.0.0-3154-g9ed025fb-1jammy libcephfs2=19.0.0-3154-g9ed025fb-1jammy libcephfs-dev=19.0.0-3154-g9ed025fb-1jammy librados2=19.0.0-3154-g9ed025fb-1jammy librbd1=19.0.0-3154-g9ed025fb-1jammy rbd-fuse=19.0.0-3154-g9ed025fb-1jammy python3-cephfs=19.0.0-3154-g9ed025fb-1jammy cephfs-shell=19.0.0-3154-g9ed025fb-1jammy cephfs-top=19.0.0-3154-g9ed025fb-1jammy cephfs-mirror=19.0.0-3154-g9ed025fb-1jammy 2024-05-28T13:54:23.529 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-05-28T13:54:23.589 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-05-28T13:54:23.604 DEBUG:teuthology.orchestra.run.smithi063:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3154-g9ed025fb-1jammy cephadm=19.0.0-3154-g9ed025fb-1jammy ceph-mds=19.0.0-3154-g9ed025fb-1jammy ceph-mgr=19.0.0-3154-g9ed025fb-1jammy ceph-common=19.0.0-3154-g9ed025fb-1jammy ceph-fuse=19.0.0-3154-g9ed025fb-1jammy ceph-test=19.0.0-3154-g9ed025fb-1jammy ceph-volume=19.0.0-3154-g9ed025fb-1jammy radosgw=19.0.0-3154-g9ed025fb-1jammy python3-rados=19.0.0-3154-g9ed025fb-1jammy python3-rgw=19.0.0-3154-g9ed025fb-1jammy python3-cephfs=19.0.0-3154-g9ed025fb-1jammy python3-rbd=19.0.0-3154-g9ed025fb-1jammy libcephfs2=19.0.0-3154-g9ed025fb-1jammy libcephfs-dev=19.0.0-3154-g9ed025fb-1jammy librados2=19.0.0-3154-g9ed025fb-1jammy librbd1=19.0.0-3154-g9ed025fb-1jammy rbd-fuse=19.0.0-3154-g9ed025fb-1jammy python3-cephfs=19.0.0-3154-g9ed025fb-1jammy cephfs-shell=19.0.0-3154-g9ed025fb-1jammy cephfs-top=19.0.0-3154-g9ed025fb-1jammy cephfs-mirror=19.0.0-3154-g9ed025fb-1jammy 2024-05-28T13:54:23.681 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-05-28T13:54:23.711 INFO:teuthology.orchestra.run.smithi042.stdout:Building dependency tree... 2024-05-28T13:54:23.712 INFO:teuthology.orchestra.run.smithi042.stdout:Reading state information... 2024-05-28T13:54:23.861 INFO:teuthology.orchestra.run.smithi063.stdout:Building dependency tree... 2024-05-28T13:54:23.861 INFO:teuthology.orchestra.run.smithi063.stdout:Reading state information... 2024-05-28T13:54:23.898 INFO:teuthology.orchestra.run.smithi042.stdout:The following packages were automatically installed and are no longer required: 2024-05-28T13:54:23.898 INFO:teuthology.orchestra.run.smithi042.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-28T13:54:23.898 INFO:teuthology.orchestra.run.smithi042.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-28T13:54:23.899 INFO:teuthology.orchestra.run.smithi042.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-28T13:54:23.899 INFO:teuthology.orchestra.run.smithi042.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-28T13:54:23.899 INFO:teuthology.orchestra.run.smithi042.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-28T13:54:23.899 INFO:teuthology.orchestra.run.smithi042.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-28T13:54:23.899 INFO:teuthology.orchestra.run.smithi042.stdout:The following additional packages will be installed: 2024-05-28T13:54:23.899 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-28T13:54:23.899 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-28T13:54:23.899 INFO:teuthology.orchestra.run.smithi042.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-28T13:54:23.900 INFO:teuthology.orchestra.run.smithi042.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-28T13:54:23.900 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-05-28T13:54:23.900 INFO:teuthology.orchestra.run.smithi042.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-05-28T13:54:23.900 INFO:teuthology.orchestra.run.smithi042.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-28T13:54:23.900 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-28T13:54:23.900 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-28T13:54:23.900 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-05-28T13:54:23.900 INFO:teuthology.orchestra.run.smithi042.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-05-28T13:54:23.901 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-05-28T13:54:23.901 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-05-28T13:54:23.901 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-05-28T13:54:23.901 INFO:teuthology.orchestra.run.smithi042.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-28T13:54:23.901 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-05-28T13:54:23.901 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-05-28T13:54:23.901 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-05-28T13:54:23.901 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-05-28T13:54:23.901 INFO:teuthology.orchestra.run.smithi042.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-05-28T13:54:23.901 INFO:teuthology.orchestra.run.smithi042.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-05-28T13:54:23.901 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-05-28T13:54:23.901 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-05-28T13:54:23.901 INFO:teuthology.orchestra.run.smithi042.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-05-28T13:54:23.902 INFO:teuthology.orchestra.run.smithi042.stdout:Suggested packages: 2024-05-28T13:54:23.902 INFO:teuthology.orchestra.run.smithi042.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-05-28T13:54:23.902 INFO:teuthology.orchestra.run.smithi042.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-05-28T13:54:23.902 INFO:teuthology.orchestra.run.smithi042.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-05-28T13:54:23.902 INFO:teuthology.orchestra.run.smithi042.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-05-28T13:54:23.902 INFO:teuthology.orchestra.run.smithi042.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-05-28T13:54:23.902 INFO:teuthology.orchestra.run.smithi042.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-28T13:54:23.902 INFO:teuthology.orchestra.run.smithi042.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-05-28T13:54:23.902 INFO:teuthology.orchestra.run.smithi042.stdout:Recommended packages: 2024-05-28T13:54:23.902 INFO:teuthology.orchestra.run.smithi042.stdout: btrfs-tools 2024-05-28T13:54:24.042 INFO:teuthology.orchestra.run.smithi063.stdout:The following packages were automatically installed and are no longer required: 2024-05-28T13:54:24.043 INFO:teuthology.orchestra.run.smithi063.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-28T13:54:24.043 INFO:teuthology.orchestra.run.smithi063.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-28T13:54:24.043 INFO:teuthology.orchestra.run.smithi063.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-28T13:54:24.043 INFO:teuthology.orchestra.run.smithi063.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-28T13:54:24.043 INFO:teuthology.orchestra.run.smithi063.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-28T13:54:24.044 INFO:teuthology.orchestra.run.smithi063.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-28T13:54:24.044 INFO:teuthology.orchestra.run.smithi063.stdout:The following additional packages will be installed: 2024-05-28T13:54:24.044 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-28T13:54:24.045 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-28T13:54:24.045 INFO:teuthology.orchestra.run.smithi063.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-28T13:54:24.045 INFO:teuthology.orchestra.run.smithi063.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-28T13:54:24.046 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-05-28T13:54:24.046 INFO:teuthology.orchestra.run.smithi063.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-05-28T13:54:24.046 INFO:teuthology.orchestra.run.smithi063.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-28T13:54:24.046 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-28T13:54:24.046 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-28T13:54:24.046 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-05-28T13:54:24.046 INFO:teuthology.orchestra.run.smithi063.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-05-28T13:54:24.046 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-05-28T13:54:24.046 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-05-28T13:54:24.046 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-05-28T13:54:24.046 INFO:teuthology.orchestra.run.smithi063.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-28T13:54:24.046 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-05-28T13:54:24.046 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-05-28T13:54:24.047 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-05-28T13:54:24.047 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-05-28T13:54:24.047 INFO:teuthology.orchestra.run.smithi063.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-05-28T13:54:24.047 INFO:teuthology.orchestra.run.smithi063.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-05-28T13:54:24.047 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-05-28T13:54:24.047 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-05-28T13:54:24.047 INFO:teuthology.orchestra.run.smithi063.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-05-28T13:54:24.047 INFO:teuthology.orchestra.run.smithi063.stdout:Suggested packages: 2024-05-28T13:54:24.047 INFO:teuthology.orchestra.run.smithi063.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-05-28T13:54:24.047 INFO:teuthology.orchestra.run.smithi063.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-05-28T13:54:24.048 INFO:teuthology.orchestra.run.smithi063.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-05-28T13:54:24.048 INFO:teuthology.orchestra.run.smithi063.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-05-28T13:54:24.048 INFO:teuthology.orchestra.run.smithi063.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-05-28T13:54:24.048 INFO:teuthology.orchestra.run.smithi063.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-28T13:54:24.048 INFO:teuthology.orchestra.run.smithi063.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-05-28T13:54:24.048 INFO:teuthology.orchestra.run.smithi063.stdout:Recommended packages: 2024-05-28T13:54:24.048 INFO:teuthology.orchestra.run.smithi063.stdout: btrfs-tools 2024-05-28T13:54:24.061 INFO:teuthology.orchestra.run.smithi042.stdout:The following NEW packages will be installed: 2024-05-28T13:54:24.061 INFO:teuthology.orchestra.run.smithi042.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-28T13:54:24.061 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-28T13:54:24.062 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-28T13:54:24.062 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-05-28T13:54:24.062 INFO:teuthology.orchestra.run.smithi042.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-28T13:54:24.062 INFO:teuthology.orchestra.run.smithi042.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-05-28T13:54:24.062 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-05-28T13:54:24.062 INFO:teuthology.orchestra.run.smithi042.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-05-28T13:54:24.063 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-05-28T13:54:24.064 INFO:teuthology.orchestra.run.smithi042.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-05-28T13:54:24.064 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-05-28T13:54:24.064 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-05-28T13:54:24.064 INFO:teuthology.orchestra.run.smithi042.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-05-28T13:54:24.064 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-28T13:54:24.064 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-28T13:54:24.064 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-28T13:54:24.064 INFO:teuthology.orchestra.run.smithi042.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-28T13:54:24.064 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-28T13:54:24.064 INFO:teuthology.orchestra.run.smithi042.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-28T13:54:24.064 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-05-28T13:54:24.065 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-05-28T13:54:24.065 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-28T13:54:24.065 INFO:teuthology.orchestra.run.smithi042.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-28T13:54:24.065 INFO:teuthology.orchestra.run.smithi042.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-28T13:54:24.065 INFO:teuthology.orchestra.run.smithi042.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-28T13:54:24.065 INFO:teuthology.orchestra.run.smithi042.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-28T13:54:24.065 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-05-28T13:54:24.065 INFO:teuthology.orchestra.run.smithi042.stdout: socat unzip xmlstarlet zip 2024-05-28T13:54:24.137 INFO:teuthology.orchestra.run.smithi042.stdout:0 upgraded, 124 newly installed, 0 to remove and 310 not upgraded. 2024-05-28T13:54:24.137 INFO:teuthology.orchestra.run.smithi042.stdout:Need to get 182 MB of archives. 2024-05-28T13:54:24.137 INFO:teuthology.orchestra.run.smithi042.stdout:After this operation, 831 MB of additional disk space will be used. 2024-05-28T13:54:24.137 INFO:teuthology.orchestra.run.smithi042.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-05-28T13:54:24.157 INFO:teuthology.orchestra.run.smithi042.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-28T13:54:24.169 INFO:teuthology.orchestra.run.smithi042.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-05-28T13:54:24.209 INFO:teuthology.orchestra.run.smithi063.stdout:The following NEW packages will be installed: 2024-05-28T13:54:24.209 INFO:teuthology.orchestra.run.smithi063.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-28T13:54:24.209 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-28T13:54:24.209 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-28T13:54:24.209 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-05-28T13:54:24.209 INFO:teuthology.orchestra.run.smithi063.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-28T13:54:24.209 INFO:teuthology.orchestra.run.smithi063.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-05-28T13:54:24.209 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-05-28T13:54:24.210 INFO:teuthology.orchestra.run.smithi063.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-05-28T13:54:24.211 INFO:teuthology.orchestra.run.smithi063.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-05-28T13:54:24.211 INFO:teuthology.orchestra.run.smithi063.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-05-28T13:54:24.211 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-05-28T13:54:24.211 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-05-28T13:54:24.211 INFO:teuthology.orchestra.run.smithi063.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-05-28T13:54:24.211 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-28T13:54:24.211 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-28T13:54:24.211 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-28T13:54:24.211 INFO:teuthology.orchestra.run.smithi063.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-28T13:54:24.211 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-28T13:54:24.211 INFO:teuthology.orchestra.run.smithi063.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-28T13:54:24.212 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-05-28T13:54:24.212 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-05-28T13:54:24.212 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-28T13:54:24.212 INFO:teuthology.orchestra.run.smithi063.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-28T13:54:24.212 INFO:teuthology.orchestra.run.smithi063.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-28T13:54:24.212 INFO:teuthology.orchestra.run.smithi063.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-28T13:54:24.212 INFO:teuthology.orchestra.run.smithi063.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-28T13:54:24.212 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-05-28T13:54:24.212 INFO:teuthology.orchestra.run.smithi063.stdout: socat unzip xmlstarlet zip 2024-05-28T13:54:24.216 INFO:teuthology.orchestra.run.smithi042.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-28T13:54:24.240 INFO:teuthology.orchestra.run.smithi042.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-28T13:54:24.242 INFO:teuthology.orchestra.run.smithi042.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-28T13:54:24.256 INFO:teuthology.orchestra.run.smithi042.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-28T13:54:24.287 INFO:teuthology.orchestra.run.smithi063.stdout:0 upgraded, 124 newly installed, 0 to remove and 310 not upgraded. 2024-05-28T13:54:24.288 INFO:teuthology.orchestra.run.smithi063.stdout:Need to get 182 MB of archives. 2024-05-28T13:54:24.288 INFO:teuthology.orchestra.run.smithi063.stdout:After this operation, 831 MB of additional disk space will be used. 2024-05-28T13:54:24.288 INFO:teuthology.orchestra.run.smithi063.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-05-28T13:54:24.294 INFO:teuthology.orchestra.run.smithi042.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3154-g9ed025fb-1jammy [3,555 kB] 2024-05-28T13:54:24.309 INFO:teuthology.orchestra.run.smithi042.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-28T13:54:24.311 INFO:teuthology.orchestra.run.smithi042.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-28T13:54:24.314 INFO:teuthology.orchestra.run.smithi063.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-28T13:54:24.320 INFO:teuthology.orchestra.run.smithi042.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-28T13:54:24.324 INFO:teuthology.orchestra.run.smithi042.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-28T13:54:24.325 INFO:teuthology.orchestra.run.smithi042.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-28T13:54:24.331 INFO:teuthology.orchestra.run.smithi063.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-05-28T13:54:24.334 INFO:teuthology.orchestra.run.smithi042.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-05-28T13:54:24.337 INFO:teuthology.orchestra.run.smithi042.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-05-28T13:54:24.341 INFO:teuthology.orchestra.run.smithi042.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-05-28T13:54:24.343 INFO:teuthology.orchestra.run.smithi042.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-05-28T13:54:24.345 INFO:teuthology.orchestra.run.smithi042.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-05-28T13:54:24.360 INFO:teuthology.orchestra.run.smithi063.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3154-g9ed025fb-1jammy [3,555 kB] 2024-05-28T13:54:24.369 INFO:teuthology.orchestra.run.smithi042.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-05-28T13:54:24.371 INFO:teuthology.orchestra.run.smithi042.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-05-28T13:54:24.373 INFO:teuthology.orchestra.run.smithi042.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-28T13:54:24.373 INFO:teuthology.orchestra.run.smithi042.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-28T13:54:24.374 INFO:teuthology.orchestra.run.smithi042.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-28T13:54:24.375 INFO:teuthology.orchestra.run.smithi042.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-28T13:54:24.376 INFO:teuthology.orchestra.run.smithi042.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-28T13:54:24.376 INFO:teuthology.orchestra.run.smithi042.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-28T13:54:24.377 INFO:teuthology.orchestra.run.smithi042.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-28T13:54:24.393 INFO:teuthology.orchestra.run.smithi042.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-28T13:54:24.393 INFO:teuthology.orchestra.run.smithi042.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-28T13:54:24.402 INFO:teuthology.orchestra.run.smithi063.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-28T13:54:24.412 INFO:teuthology.orchestra.run.smithi042.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-28T13:54:24.415 INFO:teuthology.orchestra.run.smithi042.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-28T13:54:24.416 INFO:teuthology.orchestra.run.smithi042.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-28T13:54:24.416 INFO:teuthology.orchestra.run.smithi042.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-28T13:54:24.417 INFO:teuthology.orchestra.run.smithi042.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-28T13:54:24.418 INFO:teuthology.orchestra.run.smithi042.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-28T13:54:24.418 INFO:teuthology.orchestra.run.smithi042.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-28T13:54:24.419 INFO:teuthology.orchestra.run.smithi042.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-28T13:54:24.437 INFO:teuthology.orchestra.run.smithi042.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-28T13:54:24.438 INFO:teuthology.orchestra.run.smithi042.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-28T13:54:24.440 INFO:teuthology.orchestra.run.smithi063.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-28T13:54:24.445 INFO:teuthology.orchestra.run.smithi063.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-28T13:54:24.458 INFO:teuthology.orchestra.run.smithi042.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-28T13:54:24.458 INFO:teuthology.orchestra.run.smithi063.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-28T13:54:24.464 INFO:teuthology.orchestra.run.smithi042.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-28T13:54:24.465 INFO:teuthology.orchestra.run.smithi042.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-28T13:54:24.465 INFO:teuthology.orchestra.run.smithi042.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-28T13:54:24.466 INFO:teuthology.orchestra.run.smithi042.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-28T13:54:24.467 INFO:teuthology.orchestra.run.smithi042.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-28T13:54:24.469 INFO:teuthology.orchestra.run.smithi042.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-05-28T13:54:24.470 INFO:teuthology.orchestra.run.smithi042.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-28T13:54:24.479 INFO:teuthology.orchestra.run.smithi042.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-28T13:54:24.480 INFO:teuthology.orchestra.run.smithi042.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-05-28T13:54:24.499 INFO:teuthology.orchestra.run.smithi042.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-28T13:54:24.500 INFO:teuthology.orchestra.run.smithi042.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-28T13:54:24.500 INFO:teuthology.orchestra.run.smithi042.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-28T13:54:24.503 INFO:teuthology.orchestra.run.smithi042.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-28T13:54:24.541 INFO:teuthology.orchestra.run.smithi063.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-28T13:54:24.550 INFO:teuthology.orchestra.run.smithi063.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-28T13:54:24.565 INFO:teuthology.orchestra.run.smithi063.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-28T13:54:24.570 INFO:teuthology.orchestra.run.smithi063.stdout:Get:12 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3154-g9ed025fb-1jammy [3,224 kB] 2024-05-28T13:54:24.571 INFO:teuthology.orchestra.run.smithi063.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-28T13:54:24.572 INFO:teuthology.orchestra.run.smithi063.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-28T13:54:24.629 INFO:teuthology.orchestra.run.smithi063.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-05-28T13:54:24.637 INFO:teuthology.orchestra.run.smithi042.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-28T13:54:24.638 INFO:teuthology.orchestra.run.smithi042.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-05-28T13:54:24.639 INFO:teuthology.orchestra.run.smithi042.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-28T13:54:24.640 INFO:teuthology.orchestra.run.smithi063.stdout:Get:16 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3154-g9ed025fb-1jammy [712 kB] 2024-05-28T13:54:24.640 INFO:teuthology.orchestra.run.smithi042.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-28T13:54:24.641 INFO:teuthology.orchestra.run.smithi042.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-28T13:54:24.644 INFO:teuthology.orchestra.run.smithi063.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-05-28T13:54:24.645 INFO:teuthology.orchestra.run.smithi063.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-05-28T13:54:24.647 INFO:teuthology.orchestra.run.smithi063.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-05-28T13:54:24.647 INFO:teuthology.orchestra.run.smithi063.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-05-28T13:54:24.649 INFO:teuthology.orchestra.run.smithi063.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-05-28T13:54:24.651 INFO:teuthology.orchestra.run.smithi063.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-05-28T13:54:24.653 INFO:teuthology.orchestra.run.smithi063.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-28T13:54:24.653 INFO:teuthology.orchestra.run.smithi063.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-28T13:54:24.657 INFO:teuthology.orchestra.run.smithi063.stdout:Get:25 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3154-g9ed025fb-1jammy [356 kB] 2024-05-28T13:54:24.660 INFO:teuthology.orchestra.run.smithi063.stdout:Get:26 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3154-g9ed025fb-1jammy [32.8 kB] 2024-05-28T13:54:24.661 INFO:teuthology.orchestra.run.smithi063.stdout:Get:27 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3154-g9ed025fb-1jammy [180 kB] 2024-05-28T13:54:24.661 INFO:teuthology.orchestra.run.smithi042.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-28T13:54:24.664 INFO:teuthology.orchestra.run.smithi042.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-28T13:54:24.665 INFO:teuthology.orchestra.run.smithi042.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-28T13:54:24.671 INFO:teuthology.orchestra.run.smithi063.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-28T13:54:24.675 INFO:teuthology.orchestra.run.smithi063.stdout:Get:29 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3154-g9ed025fb-1jammy [66.9 kB] 2024-05-28T13:54:24.675 INFO:teuthology.orchestra.run.smithi063.stdout:Get:30 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3154-g9ed025fb-1jammy [330 kB] 2024-05-28T13:54:24.678 INFO:teuthology.orchestra.run.smithi063.stdout:Get:31 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3154-g9ed025fb-1jammy [6,895 kB] 2024-05-28T13:54:24.688 INFO:teuthology.orchestra.run.smithi063.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-28T13:54:24.689 INFO:teuthology.orchestra.run.smithi063.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-28T13:54:24.689 INFO:teuthology.orchestra.run.smithi063.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-28T13:54:24.690 INFO:teuthology.orchestra.run.smithi042.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-28T13:54:24.690 INFO:teuthology.orchestra.run.smithi042.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-28T13:54:24.690 INFO:teuthology.orchestra.run.smithi063.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-28T13:54:24.690 INFO:teuthology.orchestra.run.smithi063.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-28T13:54:24.691 INFO:teuthology.orchestra.run.smithi063.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-28T13:54:24.691 INFO:teuthology.orchestra.run.smithi063.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-28T13:54:24.693 INFO:teuthology.orchestra.run.smithi063.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-28T13:54:24.696 INFO:teuthology.orchestra.run.smithi063.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-28T13:54:24.698 INFO:teuthology.orchestra.run.smithi042.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-28T13:54:24.699 INFO:teuthology.orchestra.run.smithi042.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-28T13:54:24.699 INFO:teuthology.orchestra.run.smithi042.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-28T13:54:24.700 INFO:teuthology.orchestra.run.smithi042.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-28T13:54:24.701 INFO:teuthology.orchestra.run.smithi042.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-28T13:54:24.702 INFO:teuthology.orchestra.run.smithi042.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-28T13:54:24.705 INFO:teuthology.orchestra.run.smithi042.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-28T13:54:24.713 INFO:teuthology.orchestra.run.smithi063.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-28T13:54:24.714 INFO:teuthology.orchestra.run.smithi063.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-28T13:54:24.715 INFO:teuthology.orchestra.run.smithi063.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-28T13:54:24.715 INFO:teuthology.orchestra.run.smithi063.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-28T13:54:24.716 INFO:teuthology.orchestra.run.smithi063.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-28T13:54:24.717 INFO:teuthology.orchestra.run.smithi063.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-28T13:54:24.718 INFO:teuthology.orchestra.run.smithi063.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-28T13:54:24.718 INFO:teuthology.orchestra.run.smithi063.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-28T13:54:24.719 INFO:teuthology.orchestra.run.smithi042.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-28T13:54:24.721 INFO:teuthology.orchestra.run.smithi042.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-28T13:54:24.722 INFO:teuthology.orchestra.run.smithi042.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-28T13:54:24.728 INFO:teuthology.orchestra.run.smithi063.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-28T13:54:24.739 INFO:teuthology.orchestra.run.smithi063.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-28T13:54:24.739 INFO:teuthology.orchestra.run.smithi042.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-28T13:54:24.740 INFO:teuthology.orchestra.run.smithi063.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-28T13:54:24.741 INFO:teuthology.orchestra.run.smithi063.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-28T13:54:24.744 INFO:teuthology.orchestra.run.smithi042.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-05-28T13:54:24.744 INFO:teuthology.orchestra.run.smithi063.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-28T13:54:24.745 INFO:teuthology.orchestra.run.smithi042.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-05-28T13:54:24.746 INFO:teuthology.orchestra.run.smithi042.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-05-28T13:54:24.747 INFO:teuthology.orchestra.run.smithi042.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-28T13:54:24.749 INFO:teuthology.orchestra.run.smithi063.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-05-28T13:54:24.751 INFO:teuthology.orchestra.run.smithi063.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-28T13:54:24.753 INFO:teuthology.orchestra.run.smithi042.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-05-28T13:54:24.754 INFO:teuthology.orchestra.run.smithi042.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-28T13:54:24.761 INFO:teuthology.orchestra.run.smithi063.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-28T13:54:24.762 INFO:teuthology.orchestra.run.smithi042.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-28T13:54:24.762 INFO:teuthology.orchestra.run.smithi063.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-05-28T13:54:24.763 INFO:teuthology.orchestra.run.smithi042.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-28T13:54:24.763 INFO:teuthology.orchestra.run.smithi042.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-28T13:54:24.765 INFO:teuthology.orchestra.run.smithi063.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-28T13:54:24.766 INFO:teuthology.orchestra.run.smithi063.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-28T13:54:24.781 INFO:teuthology.orchestra.run.smithi063.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-28T13:54:24.782 INFO:teuthology.orchestra.run.smithi042.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-28T13:54:24.785 INFO:teuthology.orchestra.run.smithi042.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-28T13:54:24.785 INFO:teuthology.orchestra.run.smithi063.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-28T13:54:24.786 INFO:teuthology.orchestra.run.smithi042.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-28T13:54:24.796 INFO:teuthology.orchestra.run.smithi042.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-28T13:54:24.797 INFO:teuthology.orchestra.run.smithi042.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-28T13:54:24.797 INFO:teuthology.orchestra.run.smithi042.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-28T13:54:24.800 INFO:teuthology.orchestra.run.smithi042.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-28T13:54:24.803 INFO:teuthology.orchestra.run.smithi063.stdout:Get:62 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3154-g9ed025fb-1jammy [112 kB] 2024-05-28T13:54:24.804 INFO:teuthology.orchestra.run.smithi063.stdout:Get:63 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3154-g9ed025fb-1jammy [466 kB] 2024-05-28T13:54:24.808 INFO:teuthology.orchestra.run.smithi063.stdout:Get:64 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3154-g9ed025fb-1jammy [25.8 MB] 2024-05-28T13:54:24.819 INFO:teuthology.orchestra.run.smithi042.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-28T13:54:24.892 INFO:teuthology.orchestra.run.smithi063.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-28T13:54:24.893 INFO:teuthology.orchestra.run.smithi063.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-05-28T13:54:24.894 INFO:teuthology.orchestra.run.smithi063.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-28T13:54:24.894 INFO:teuthology.orchestra.run.smithi063.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-28T13:54:24.896 INFO:teuthology.orchestra.run.smithi063.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-28T13:54:24.917 INFO:teuthology.orchestra.run.smithi063.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-28T13:54:24.920 INFO:teuthology.orchestra.run.smithi063.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-28T13:54:24.920 INFO:teuthology.orchestra.run.smithi063.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-28T13:54:24.945 INFO:teuthology.orchestra.run.smithi063.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-28T13:54:24.946 INFO:teuthology.orchestra.run.smithi063.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-28T13:54:24.952 INFO:teuthology.orchestra.run.smithi063.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-28T13:54:24.953 INFO:teuthology.orchestra.run.smithi063.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-28T13:54:24.953 INFO:teuthology.orchestra.run.smithi063.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-28T13:54:24.954 INFO:teuthology.orchestra.run.smithi063.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-28T13:54:24.954 INFO:teuthology.orchestra.run.smithi063.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-28T13:54:24.955 INFO:teuthology.orchestra.run.smithi063.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-28T13:54:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-28T13:54:24.971 INFO:teuthology.orchestra.run.smithi063.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-28T13:54:24.973 INFO:teuthology.orchestra.run.smithi063.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-28T13:54:24.974 INFO:teuthology.orchestra.run.smithi063.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-28T13:54:24.990 INFO:teuthology.orchestra.run.smithi063.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-28T13:54:24.994 INFO:teuthology.orchestra.run.smithi063.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-05-28T13:54:24.995 INFO:teuthology.orchestra.run.smithi063.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-05-28T13:54:24.996 INFO:teuthology.orchestra.run.smithi063.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-05-28T13:54:24.996 INFO:teuthology.orchestra.run.smithi063.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-28T13:54:25.001 INFO:teuthology.orchestra.run.smithi063.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-05-28T13:54:25.002 INFO:teuthology.orchestra.run.smithi063.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-28T13:54:25.013 INFO:teuthology.orchestra.run.smithi063.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-28T13:54:25.013 INFO:teuthology.orchestra.run.smithi063.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-28T13:54:25.014 INFO:teuthology.orchestra.run.smithi063.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-28T13:54:25.032 INFO:teuthology.orchestra.run.smithi063.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-28T13:54:25.035 INFO:teuthology.orchestra.run.smithi063.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-28T13:54:25.036 INFO:teuthology.orchestra.run.smithi063.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-28T13:54:25.046 INFO:teuthology.orchestra.run.smithi063.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-28T13:54:25.046 INFO:teuthology.orchestra.run.smithi063.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-28T13:54:25.046 INFO:teuthology.orchestra.run.smithi063.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-28T13:54:25.047 INFO:teuthology.orchestra.run.smithi063.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-28T13:54:25.052 INFO:teuthology.orchestra.run.smithi063.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-28T13:54:25.260 INFO:teuthology.orchestra.run.smithi063.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3154-g9ed025fb-1jammy [5,092 kB] 2024-05-28T13:54:25.340 INFO:teuthology.orchestra.run.smithi063.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3154-g9ed025fb-1jammy [247 kB] 2024-05-28T13:54:25.342 INFO:teuthology.orchestra.run.smithi063.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3154-g9ed025fb-1jammy [124 kB] 2024-05-28T13:54:25.345 INFO:teuthology.orchestra.run.smithi063.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3154-g9ed025fb-1jammy [1,527 kB] 2024-05-28T13:54:25.372 INFO:teuthology.orchestra.run.smithi063.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3154-g9ed025fb-1jammy [6,257 kB] 2024-05-28T13:54:25.482 INFO:teuthology.orchestra.run.smithi063.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3154-g9ed025fb-1jammy [22.8 MB] 2024-05-28T13:54:25.640 INFO:teuthology.orchestra.run.smithi042.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3154-g9ed025fb-1jammy [3,224 kB] 2024-05-28T13:54:25.886 INFO:teuthology.orchestra.run.smithi063.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3154-g9ed025fb-1jammy [14.1 kB] 2024-05-28T13:54:25.886 INFO:teuthology.orchestra.run.smithi063.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3154-g9ed025fb-1jammy [870 kB] 2024-05-28T13:54:25.894 INFO:teuthology.orchestra.run.smithi063.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3154-g9ed025fb-1jammy [2,436 kB] 2024-05-28T13:54:25.938 INFO:teuthology.orchestra.run.smithi063.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3154-g9ed025fb-1jammy [791 kB] 2024-05-28T13:54:25.950 INFO:teuthology.orchestra.run.smithi063.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3154-g9ed025fb-1jammy [148 kB] 2024-05-28T13:54:25.953 INFO:teuthology.orchestra.run.smithi063.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3154-g9ed025fb-1jammy [2,297 kB] 2024-05-28T13:54:25.991 INFO:teuthology.orchestra.run.smithi063.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3154-g9ed025fb-1jammy [8,625 kB] 2024-05-28T13:54:26.144 INFO:teuthology.orchestra.run.smithi063.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3154-g9ed025fb-1jammy [14.2 kB] 2024-05-28T13:54:26.144 INFO:teuthology.orchestra.run.smithi063.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3154-g9ed025fb-1jammy [51.6 MB] 2024-05-28T13:54:26.630 INFO:teuthology.orchestra.run.smithi042.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3154-g9ed025fb-1jammy [712 kB] 2024-05-28T13:54:26.847 INFO:teuthology.orchestra.run.smithi042.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3154-g9ed025fb-1jammy [356 kB] 2024-05-28T13:54:26.993 INFO:teuthology.orchestra.run.smithi042.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3154-g9ed025fb-1jammy [32.8 kB] 2024-05-28T13:54:27.001 INFO:teuthology.orchestra.run.smithi042.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3154-g9ed025fb-1jammy [180 kB] 2024-05-28T13:54:27.049 INFO:teuthology.orchestra.run.smithi063.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3154-g9ed025fb-1jammy [124 kB] 2024-05-28T13:54:27.050 INFO:teuthology.orchestra.run.smithi063.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3154-g9ed025fb-1jammy [232 kB] 2024-05-28T13:54:27.052 INFO:teuthology.orchestra.run.smithi063.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3154-g9ed025fb-1jammy [33.9 kB] 2024-05-28T13:54:27.052 INFO:teuthology.orchestra.run.smithi063.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3154-g9ed025fb-1jammy [23.4 kB] 2024-05-28T13:54:27.053 INFO:teuthology.orchestra.run.smithi063.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3154-g9ed025fb-1jammy [38.9 kB] 2024-05-28T13:54:27.053 INFO:teuthology.orchestra.run.smithi063.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3154-g9ed025fb-1jammy [13.6 MB] 2024-05-28T13:54:27.067 INFO:teuthology.orchestra.run.smithi042.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3154-g9ed025fb-1jammy [66.9 kB] 2024-05-28T13:54:27.083 INFO:teuthology.orchestra.run.smithi042.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3154-g9ed025fb-1jammy [330 kB] 2024-05-28T13:54:27.186 INFO:teuthology.orchestra.run.smithi042.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3154-g9ed025fb-1jammy [6,895 kB] 2024-05-28T13:54:27.279 INFO:teuthology.orchestra.run.smithi063.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3154-g9ed025fb-1jammy [91.2 kB] 2024-05-28T13:54:27.942 INFO:teuthology.orchestra.run.smithi063.stdout:Fetched 182 MB in 3s (59.4 MB/s) 2024-05-28T13:54:28.147 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-28T13:54:28.216 INFO:teuthology.orchestra.run.smithi063.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132924 files and directories currently installed.) 2024-05-28T13:54:28.220 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-05-28T13:54:28.279 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-05-28T13:54:28.424 INFO:teuthology.orchestra.run.smithi042.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3154-g9ed025fb-1jammy [112 kB] 2024-05-28T13:54:28.433 INFO:teuthology.orchestra.run.smithi042.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3154-g9ed025fb-1jammy [466 kB] 2024-05-28T13:54:28.503 INFO:teuthology.orchestra.run.smithi042.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3154-g9ed025fb-1jammy [25.8 MB] 2024-05-28T13:54:28.631 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-28T13:54:28.645 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-28T13:54:28.690 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-28T13:54:29.076 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-cmd2. 2024-05-28T13:54:29.091 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-05-28T13:54:29.143 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-05-28T13:54:29.446 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-28T13:54:29.461 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-28T13:54:29.546 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-28T13:54:29.925 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-28T13:54:29.940 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-28T13:54:29.983 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-28T13:54:30.361 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-28T13:54:30.377 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-28T13:54:30.419 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-28T13:54:30.831 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-28T13:54:30.847 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-28T13:54:30.889 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-28T13:54:31.335 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-28T13:54:31.350 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-28T13:54:31.393 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-28T13:54:31.771 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-28T13:54:31.787 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-28T13:54:31.830 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-28T13:54:32.132 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-28T13:54:32.148 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-28T13:54:32.216 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-28T13:54:32.544 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package librados2. 2024-05-28T13:54:32.559 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../010-librados2_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:32.560 INFO:teuthology.orchestra.run.smithi042.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3154-g9ed025fb-1jammy [5,092 kB] 2024-05-28T13:54:32.602 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking librados2 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:33.047 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package librbd1. 2024-05-28T13:54:33.062 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../011-librbd1_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:33.104 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking librbd1 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:33.426 INFO:teuthology.orchestra.run.smithi042.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3154-g9ed025fb-1jammy [247 kB] 2024-05-28T13:54:33.462 INFO:teuthology.orchestra.run.smithi042.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3154-g9ed025fb-1jammy [124 kB] 2024-05-28T13:54:33.483 INFO:teuthology.orchestra.run.smithi042.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3154-g9ed025fb-1jammy [1,527 kB] 2024-05-28T13:54:33.516 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libcephfs2. 2024-05-28T13:54:33.531 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:33.566 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libcephfs2 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:33.672 INFO:teuthology.orchestra.run.smithi042.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3154-g9ed025fb-1jammy [6,257 kB] 2024-05-28T13:54:33.854 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-rados. 2024-05-28T13:54:33.869 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:33.912 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-rados (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:34.215 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-28T13:54:34.229 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:54:34.298 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-ceph-argparse (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:34.424 INFO:teuthology.orchestra.run.smithi042.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3154-g9ed025fb-1jammy [22.8 MB] 2024-05-28T13:54:34.550 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-cephfs. 2024-05-28T13:54:34.566 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:34.600 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-cephfs (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:34.903 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-28T13:54:34.918 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:54:34.961 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-ceph-common (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:35.314 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-prettytable. 2024-05-28T13:54:35.330 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-05-28T13:54:35.372 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-28T13:54:35.641 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-rbd. 2024-05-28T13:54:35.657 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:35.700 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-rbd (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:36.077 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-28T13:54:36.093 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-28T13:54:36.136 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-28T13:54:36.447 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-05-28T13:54:36.462 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-05-28T13:54:36.496 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-05-28T13:54:37.008 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-05-28T13:54:37.024 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-05-28T13:54:37.066 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-28T13:54:37.394 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package lua5.1. 2024-05-28T13:54:37.410 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-05-28T13:54:37.469 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-05-28T13:54:37.671 INFO:teuthology.orchestra.run.smithi042.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3154-g9ed025fb-1jammy [14.1 kB] 2024-05-28T13:54:37.672 INFO:teuthology.orchestra.run.smithi042.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3154-g9ed025fb-1jammy [870 kB] 2024-05-28T13:54:37.797 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package lua-any. 2024-05-28T13:54:37.813 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-05-28T13:54:37.843 INFO:teuthology.orchestra.run.smithi042.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3154-g9ed025fb-1jammy [2,436 kB] 2024-05-28T13:54:37.855 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking lua-any (27ubuntu1) ... 2024-05-28T13:54:38.099 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package zip. 2024-05-28T13:54:38.117 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-05-28T13:54:38.149 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking zip (3.0-12build2) ... 2024-05-28T13:54:38.361 INFO:teuthology.orchestra.run.smithi042.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3154-g9ed025fb-1jammy [791 kB] 2024-05-28T13:54:38.473 INFO:teuthology.orchestra.run.smithi042.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3154-g9ed025fb-1jammy [148 kB] 2024-05-28T13:54:38.489 INFO:teuthology.orchestra.run.smithi042.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3154-g9ed025fb-1jammy [2,297 kB] 2024-05-28T13:54:38.494 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package unzip. 2024-05-28T13:54:38.510 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-05-28T13:54:38.552 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-05-28T13:54:38.769 INFO:teuthology.orchestra.run.smithi042.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3154-g9ed025fb-1jammy [8,625 kB] 2024-05-28T13:54:38.931 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package luarocks. 2024-05-28T13:54:38.946 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-05-28T13:54:39.005 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-05-28T13:54:39.396 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package librgw2. 2024-05-28T13:54:39.411 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../027-librgw2_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:39.446 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking librgw2 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:39.753 INFO:teuthology.orchestra.run.smithi042.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3154-g9ed025fb-1jammy [14.2 kB] 2024-05-28T13:54:39.754 INFO:teuthology.orchestra.run.smithi042.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3154-g9ed025fb-1jammy [51.6 MB] 2024-05-28T13:54:39.890 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-rgw. 2024-05-28T13:54:39.905 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:39.940 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-rgw (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:40.310 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-28T13:54:40.325 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-28T13:54:40.367 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-28T13:54:40.712 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libradosstriper1. 2024-05-28T13:54:40.728 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:40.779 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libradosstriper1 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:41.182 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-common. 2024-05-28T13:54:41.198 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:41.231 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-common (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:42.612 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-base. 2024-05-28T13:54:42.628 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:42.671 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-base (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:43.166 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-28T13:54:43.180 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-28T13:54:43.258 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-28T13:54:43.560 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-cheroot. 2024-05-28T13:54:43.575 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-28T13:54:43.618 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-28T13:54:43.972 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-28T13:54:43.987 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-28T13:54:44.030 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-28T13:54:44.358 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-28T13:54:44.373 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-28T13:54:44.415 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-28T13:54:44.693 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-28T13:54:44.709 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-28T13:54:44.768 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-28T13:54:45.046 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-tempora. 2024-05-28T13:54:45.062 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-05-28T13:54:45.104 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-28T13:54:45.136 INFO:teuthology.orchestra.run.smithi042.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3154-g9ed025fb-1jammy [124 kB] 2024-05-28T13:54:45.137 INFO:teuthology.orchestra.run.smithi042.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3154-g9ed025fb-1jammy [232 kB] 2024-05-28T13:54:45.153 INFO:teuthology.orchestra.run.smithi042.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3154-g9ed025fb-1jammy [33.9 kB] 2024-05-28T13:54:45.174 INFO:teuthology.orchestra.run.smithi042.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3154-g9ed025fb-1jammy [23.4 kB] 2024-05-28T13:54:45.174 INFO:teuthology.orchestra.run.smithi042.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3154-g9ed025fb-1jammy [38.9 kB] 2024-05-28T13:54:45.174 INFO:teuthology.orchestra.run.smithi042.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3154-g9ed025fb-1jammy [13.6 MB] 2024-05-28T13:54:45.432 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-portend. 2024-05-28T13:54:45.448 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-05-28T13:54:45.490 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-28T13:54:45.826 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-28T13:54:45.842 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-28T13:54:45.884 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-28T13:54:46.162 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-28T13:54:46.178 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-28T13:54:46.212 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-28T13:54:46.397 INFO:teuthology.orchestra.run.smithi042.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3154-g9ed025fb-1jammy [91.2 kB] 2024-05-28T13:54:46.566 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-natsort. 2024-05-28T13:54:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-05-28T13:54:46.624 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-28T13:54:46.927 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-logutils. 2024-05-28T13:54:46.942 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-05-28T13:54:46.984 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-28T13:54:47.068 INFO:teuthology.orchestra.run.smithi042.stdout:Fetched 182 MB in 22s (8,133 kB/s) 2024-05-28T13:54:47.222 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-28T13:54:47.271 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-28T13:54:47.286 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-28T13:54:47.290 INFO:teuthology.orchestra.run.smithi042.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132924 files and directories currently installed.) 2024-05-28T13:54:47.294 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-05-28T13:54:47.337 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-05-28T13:54:47.354 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-28T13:54:47.657 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-mako. 2024-05-28T13:54:47.672 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-28T13:54:47.723 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-28T13:54:47.773 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-28T13:54:47.787 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-28T13:54:47.840 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-28T13:54:48.085 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-28T13:54:48.101 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-28T13:54:48.109 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-cmd2. 2024-05-28T13:54:48.123 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-05-28T13:54:48.144 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-28T13:54:48.168 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-05-28T13:54:48.471 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-28T13:54:48.486 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-28T13:54:48.487 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-28T13:54:48.502 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-28T13:54:48.521 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-28T13:54:48.588 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-28T13:54:48.782 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-webob. 2024-05-28T13:54:48.797 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-28T13:54:48.865 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-28T13:54:48.967 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-28T13:54:48.982 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-28T13:54:49.025 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-28T13:54:49.218 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-waitress. 2024-05-28T13:54:49.233 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-28T13:54:49.279 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-28T13:54:49.361 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-28T13:54:49.377 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-28T13:54:49.420 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-28T13:54:49.621 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-tempita. 2024-05-28T13:54:49.636 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-28T13:54:49.679 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-28T13:54:49.823 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-28T13:54:49.838 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-28T13:54:49.898 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-28T13:54:50.023 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-paste. 2024-05-28T13:54:50.039 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-28T13:54:50.081 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-28T13:54:50.310 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-28T13:54:50.325 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-28T13:54:50.368 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-28T13:54:50.418 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-28T13:54:50.433 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-28T13:54:50.476 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-28T13:54:50.663 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-28T13:54:50.678 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-28T13:54:50.713 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-28T13:54:50.762 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-28T13:54:50.778 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-28T13:54:50.820 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-28T13:54:51.041 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-28T13:54:51.057 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-28T13:54:51.100 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-28T13:54:51.157 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-webtest. 2024-05-28T13:54:51.172 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-05-28T13:54:51.215 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-28T13:54:51.445 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package librados2. 2024-05-28T13:54:51.460 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../010-librados2_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:51.503 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking librados2 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:51.534 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pecan. 2024-05-28T13:54:51.550 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-28T13:54:51.584 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-28T13:54:51.947 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-certifi. 2024-05-28T13:54:51.963 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-05-28T13:54:51.997 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-28T13:54:52.097 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package librbd1. 2024-05-28T13:54:52.113 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../011-librbd1_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:52.147 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking librbd1 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:52.308 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-idna. 2024-05-28T13:54:52.323 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../057-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-05-28T13:54:52.366 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-05-28T13:54:52.567 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libcephfs2. 2024-05-28T13:54:52.582 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:52.625 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libcephfs2 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:52.744 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-urllib3. 2024-05-28T13:54:52.759 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-28T13:54:52.802 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-28T13:54:52.911 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-rados. 2024-05-28T13:54:52.927 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:52.970 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-rados (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:53.197 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-requests. 2024-05-28T13:54:53.212 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-28T13:54:53.264 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-28T13:54:53.289 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-28T13:54:53.305 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:54:53.347 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-ceph-argparse (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:53.609 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-cephfs. 2024-05-28T13:54:53.624 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:53.658 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-28T13:54:53.659 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-cephfs (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:53.674 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-05-28T13:54:53.724 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-28T13:54:53.953 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-28T13:54:53.968 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:54:54.011 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-ceph-common (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:54.077 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-28T13:54:54.093 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:54:54.144 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:54.340 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-prettytable. 2024-05-28T13:54:54.355 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-05-28T13:54:54.398 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-28T13:54:54.565 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-28T13:54:54.581 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:54.615 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:54.684 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-rbd. 2024-05-28T13:54:54.699 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:54.742 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-rbd (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:54.918 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-28T13:54:54.934 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-28T13:54:54.993 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-28T13:54:55.037 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-28T13:54:55.053 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-28T13:54:55.087 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-28T13:54:55.321 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-openssl. 2024-05-28T13:54:55.336 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-05-28T13:54:55.348 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-05-28T13:54:55.363 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-05-28T13:54:55.378 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-28T13:54:55.406 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-05-28T13:54:55.693 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-05-28T13:54:55.708 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-05-28T13:54:55.751 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-28T13:54:55.765 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr. 2024-05-28T13:54:55.781 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:55.823 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:56.105 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package lua5.1. 2024-05-28T13:54:56.120 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-05-28T13:54:56.155 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-05-28T13:54:56.201 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mon. 2024-05-28T13:54:56.217 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:56.251 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mon (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:56.425 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package lua-any. 2024-05-28T13:54:56.440 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-05-28T13:54:56.483 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking lua-any (27ubuntu1) ... 2024-05-28T13:54:56.744 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package zip. 2024-05-28T13:54:56.759 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-05-28T13:54:56.802 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking zip (3.0-12build2) ... 2024-05-28T13:54:56.854 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-osd. 2024-05-28T13:54:56.870 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:56.912 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-osd (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:58.048 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package unzip. 2024-05-28T13:54:58.063 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-05-28T13:54:58.114 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-05-28T13:54:58.299 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph. 2024-05-28T13:54:58.315 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../068-ceph_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:58.357 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:58.594 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package luarocks. 2024-05-28T13:54:58.609 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-05-28T13:54:58.660 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-05-28T13:54:58.676 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-fuse. 2024-05-28T13:54:58.692 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:58.726 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-fuse (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:59.010 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package librgw2. 2024-05-28T13:54:59.024 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../027-librgw2_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:59.068 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking librgw2 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:59.079 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mds. 2024-05-28T13:54:59.095 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:59.137 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mds (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:59.557 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package cephadm. 2024-05-28T13:54:59.573 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../071-cephadm_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:59.604 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-rgw. 2024-05-28T13:54:59.615 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking cephadm (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:59.619 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:54:59.663 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-rgw (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:54:59.943 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-28T13:54:59.959 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-28T13:54:59.999 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-28T13:55:00.001 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-28T13:55:00.014 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-28T13:55:00.057 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-28T13:55:00.328 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-28T13:55:00.343 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-28T13:55:00.369 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libradosstriper1. 2024-05-28T13:55:00.384 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:00.387 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-28T13:55:00.427 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libradosstriper1 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:00.889 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-common. 2024-05-28T13:55:00.904 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:00.939 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-common (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:01.433 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-babel. 2024-05-28T13:55:01.448 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-28T13:55:01.491 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-28T13:55:01.819 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-jinja2. 2024-05-28T13:55:01.833 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-05-28T13:55:01.877 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-05-28T13:55:02.020 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-base. 2024-05-28T13:55:02.035 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:02.087 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-base (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:02.256 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-28T13:55:02.270 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:02.347 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:02.641 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-28T13:55:02.656 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-28T13:55:02.699 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-28T13:55:02.717 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-28T13:55:02.732 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-05-28T13:55:02.775 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-28T13:55:02.985 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-cheroot. 2024-05-28T13:55:03.001 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-28T13:55:03.035 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-28T13:55:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-routes. 2024-05-28T13:55:03.127 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-28T13:55:03.161 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-28T13:55:03.330 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-28T13:55:03.346 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-28T13:55:03.388 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-28T13:55:03.506 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-28T13:55:03.522 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:03.589 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:03.675 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-28T13:55:03.690 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-28T13:55:03.742 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-28T13:55:04.045 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-28T13:55:04.060 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-28T13:55:04.103 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-28T13:55:04.406 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-tempora. 2024-05-28T13:55:04.421 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-05-28T13:55:04.464 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-28T13:55:04.751 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-portend. 2024-05-28T13:55:04.766 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-05-28T13:55:04.809 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-28T13:55:04.844 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-28T13:55:04.862 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-28T13:55:04.902 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-28T13:55:05.109 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-28T13:55:05.124 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-28T13:55:05.162 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-28T13:55:05.406 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-joblib. 2024-05-28T13:55:05.420 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-28T13:55:05.455 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-28T13:55:05.465 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-28T13:55:05.480 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-28T13:55:05.523 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-28T13:55:05.834 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-28T13:55:05.849 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-28T13:55:05.885 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-natsort. 2024-05-28T13:55:05.901 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-05-28T13:55:05.935 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-28T13:55:05.977 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-28T13:55:06.222 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-logutils. 2024-05-28T13:55:06.237 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-05-28T13:55:06.288 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-28T13:55:06.330 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-sklearn. 2024-05-28T13:55:06.345 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-28T13:55:06.388 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-28T13:55:06.575 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-28T13:55:06.590 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-28T13:55:06.716 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-28T13:55:07.005 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-mako. 2024-05-28T13:55:07.021 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-28T13:55:07.063 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-28T13:55:07.088 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-28T13:55:07.103 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:07.147 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:07.641 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-28T13:55:07.657 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-28T13:55:07.700 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-28T13:55:08.002 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-28T13:55:08.018 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-28T13:55:08.061 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-28T13:55:08.322 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-webob. 2024-05-28T13:55:08.337 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-28T13:55:08.371 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-28T13:55:08.400 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-cachetools. 2024-05-28T13:55:08.416 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-05-28T13:55:08.449 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-28T13:55:08.766 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-waitress. 2024-05-28T13:55:08.778 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-28T13:55:08.781 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-28T13:55:08.794 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-28T13:55:08.819 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-28T13:55:08.835 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-28T13:55:09.086 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-tempita. 2024-05-28T13:55:09.102 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-28T13:55:09.144 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-28T13:55:09.197 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-28T13:55:09.213 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-28T13:55:09.255 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-28T13:55:09.430 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-paste. 2024-05-28T13:55:09.446 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-28T13:55:09.488 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-28T13:55:09.550 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-rsa. 2024-05-28T13:55:09.566 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-05-28T13:55:09.608 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-28T13:55:09.808 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-28T13:55:09.824 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-28T13:55:09.858 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-28T13:55:09.961 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-google-auth. 2024-05-28T13:55:09.978 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-05-28T13:55:10.020 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-28T13:55:10.161 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-28T13:55:10.177 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-28T13:55:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-28T13:55:10.364 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-28T13:55:10.380 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-28T13:55:10.414 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-28T13:55:10.522 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-webtest. 2024-05-28T13:55:10.538 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-05-28T13:55:10.581 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-28T13:55:10.676 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-websocket. 2024-05-28T13:55:10.692 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-05-28T13:55:10.750 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-28T13:55:10.901 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pecan. 2024-05-28T13:55:10.916 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-28T13:55:10.958 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-28T13:55:11.062 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-28T13:55:11.078 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-28T13:55:11.120 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-28T13:55:11.322 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-certifi. 2024-05-28T13:55:11.337 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-05-28T13:55:11.380 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-28T13:55:11.649 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-idna. 2024-05-28T13:55:11.665 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../057-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-05-28T13:55:11.699 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-05-28T13:55:11.819 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-28T13:55:11.834 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:11.868 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:11.985 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-urllib3. 2024-05-28T13:55:12.001 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-28T13:55:12.052 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-28T13:55:12.163 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-28T13:55:12.177 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-28T13:55:12.213 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-28T13:55:12.305 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-requests. 2024-05-28T13:55:12.321 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-28T13:55:12.355 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-28T13:55:12.582 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-28T13:55:12.597 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-28T13:55:12.640 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-28T13:55:12.641 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-28T13:55:12.657 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-05-28T13:55:12.699 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-28T13:55:12.952 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package jq. 2024-05-28T13:55:12.966 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-28T13:55:12.986 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-28T13:55:13.001 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:13.009 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-28T13:55:13.044 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:13.237 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package socat. 2024-05-28T13:55:13.252 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-28T13:55:13.304 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-28T13:55:13.466 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-28T13:55:13.481 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:13.524 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:13.582 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package xmlstarlet. 2024-05-28T13:55:13.597 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-28T13:55:13.640 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-28T13:55:13.819 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-28T13:55:13.834 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-28T13:55:13.876 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-28T13:55:14.028 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-test. 2024-05-28T13:55:14.042 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:14.086 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-test (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:14.188 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-openssl. 2024-05-28T13:55:14.204 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-05-28T13:55:14.246 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-28T13:55:14.566 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr. 2024-05-28T13:55:14.582 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:14.616 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:14.952 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mon. 2024-05-28T13:55:14.963 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:15.002 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mon (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:15.614 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-osd. 2024-05-28T13:55:15.624 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:15.664 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-osd (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:16.140 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-volume. 2024-05-28T13:55:16.156 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:16.190 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-volume (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:16.409 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph. 2024-05-28T13:55:16.419 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../068-ceph_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:16.458 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:16.619 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-28T13:55:16.635 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:16.677 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking cephfs-mirror (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:16.770 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-fuse. 2024-05-28T13:55:16.780 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:16.811 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-fuse (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:16.971 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-colorama. 2024-05-28T13:55:16.987 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-05-28T13:55:17.021 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-05-28T13:55:17.156 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mds. 2024-05-28T13:55:17.166 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:17.206 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mds (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:17.274 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package cephfs-shell. 2024-05-28T13:55:17.290 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:17.332 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking cephfs-shell (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:17.559 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package cephadm. 2024-05-28T13:55:17.569 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../071-cephadm_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:17.609 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking cephadm (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:17.643 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package cephfs-top. 2024-05-28T13:55:17.659 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:17.701 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking cephfs-top (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:17.987 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-28T13:55:18.003 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:18.045 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libcephfs-dev (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:18.062 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-28T13:55:18.072 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-28T13:55:18.120 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-28T13:55:18.373 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package lua-socket:amd64. 2024-05-28T13:55:18.389 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-05-28T13:55:18.430 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-28T13:55:18.523 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-28T13:55:18.533 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-28T13:55:18.572 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-28T13:55:18.760 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package lua-sec:amd64. 2024-05-28T13:55:18.776 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-05-28T13:55:18.810 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-05-28T13:55:19.222 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package nvme-cli. 2024-05-28T13:55:19.238 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-28T13:55:19.280 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-28T13:55:19.544 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-babel. 2024-05-28T13:55:19.553 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-28T13:55:19.594 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-28T13:55:19.752 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package pkg-config. 2024-05-28T13:55:19.768 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-05-28T13:55:19.810 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-05-28T13:55:19.914 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-jinja2. 2024-05-28T13:55:19.922 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-05-28T13:55:19.963 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-05-28T13:55:20.088 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-28T13:55:20.104 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-28T13:55:20.145 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-28T13:55:20.342 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-28T13:55:20.351 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:20.391 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:20.599 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-28T13:55:20.615 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-28T13:55:20.656 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-28T13:55:20.703 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-28T13:55:20.713 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-05-28T13:55:20.753 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-28T13:55:20.943 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pastescript. 2024-05-28T13:55:20.959 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-05-28T13:55:21.016 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-28T13:55:21.039 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-routes. 2024-05-28T13:55:21.049 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-28T13:55:21.089 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-28T13:55:21.279 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pluggy. 2024-05-28T13:55:21.296 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-28T13:55:21.329 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-28T13:55:21.451 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-28T13:55:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:21.500 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:21.674 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-psutil. 2024-05-28T13:55:21.690 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-28T13:55:21.732 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-28T13:55:22.101 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-py. 2024-05-28T13:55:22.114 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-05-28T13:55:22.151 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-28T13:55:22.488 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pygments. 2024-05-28T13:55:22.502 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-28T13:55:22.579 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-28T13:55:22.714 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-28T13:55:22.723 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-28T13:55:22.764 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-28T13:55:23.033 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-28T13:55:23.045 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-28T13:55:23.107 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-28T13:55:23.225 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-joblib. 2024-05-28T13:55:23.237 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-28T13:55:23.267 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-28T13:55:23.394 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-toml. 2024-05-28T13:55:23.407 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-05-28T13:55:23.435 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-28T13:55:23.646 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-28T13:55:23.655 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-28T13:55:23.738 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pytest. 2024-05-28T13:55:23.751 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-28T13:55:23.773 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-28T13:55:23.796 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-28T13:55:24.075 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-sklearn. 2024-05-28T13:55:24.086 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-28T13:55:24.125 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-28T13:55:24.158 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-simplejson. 2024-05-28T13:55:24.171 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-28T13:55:24.207 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-28T13:55:24.460 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-28T13:55:24.474 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-28T13:55:24.510 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-28T13:55:24.810 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-28T13:55:24.819 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:24.859 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:25.272 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package radosgw. 2024-05-28T13:55:25.284 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../122-radosgw_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:25.330 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking radosgw (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:25.830 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-cachetools. 2024-05-28T13:55:25.837 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-05-28T13:55:25.916 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-28T13:55:25.992 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package rbd-fuse. 2024-05-28T13:55:26.005 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:26.049 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking rbd-fuse (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:26.189 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-28T13:55:26.190 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-28T13:55:26.224 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-28T13:55:26.393 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-28T13:55:26.519 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-28T13:55:26.526 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-28T13:55:26.560 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-28T13:55:26.680 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-28T13:55:26.822 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-rsa. 2024-05-28T13:55:26.830 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-28T13:55:26.832 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-05-28T13:55:26.871 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-28T13:55:27.105 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-28T13:55:27.209 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-google-auth. 2024-05-28T13:55:27.217 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-05-28T13:55:27.258 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-28T13:55:27.538 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-28T13:55:27.578 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-28T13:55:27.586 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-28T13:55:27.628 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-28T13:55:27.915 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-websocket. 2024-05-28T13:55:27.926 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-05-28T13:55:27.973 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-28T13:55:28.194 INFO:teuthology.orchestra.run.smithi063.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-28T13:55:28.258 INFO:teuthology.orchestra.run.smithi063.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-05-28T13:55:28.333 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up cephadm (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:28.510 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-28T13:55:28.519 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-28T13:55:28.559 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-28T13:55:28.851 INFO:teuthology.orchestra.run.smithi063.stdout:Adding system user cephadm....done 2024-05-28T13:55:29.022 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-28T13:55:29.148 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-28T13:55:29.157 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:29.190 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:29.348 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-28T13:55:29.526 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-28T13:55:29.534 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-28T13:55:29.576 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-28T13:55:29.624 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-28T13:55:29.775 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-28T13:55:29.912 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-28T13:55:29.921 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-28T13:55:29.962 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-28T13:55:30.093 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-28T13:55:30.215 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package jq. 2024-05-28T13:55:30.224 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-28T13:55:30.265 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-28T13:55:30.369 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-28T13:55:30.470 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-28T13:55:30.484 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package socat. 2024-05-28T13:55:30.494 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-28T13:55:30.559 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-28T13:55:30.780 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-28T13:55:30.837 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package xmlstarlet. 2024-05-28T13:55:30.847 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-28T13:55:30.887 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-28T13:55:31.182 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-28T13:55:31.192 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-test. 2024-05-28T13:55:31.201 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:31.241 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-test (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:31.458 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-05-28T13:55:31.734 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-28T13:55:32.027 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-05-28T13:55:32.253 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-28T13:55:32.513 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-28T13:55:32.933 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-28T13:55:33.223 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-ceph-argparse (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:33.354 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-volume. 2024-05-28T13:55:33.364 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:33.404 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-volume (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:33.775 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-28T13:55:33.785 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:33.825 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking cephfs-mirror (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:33.866 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-28T13:55:34.001 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-28T13:55:34.119 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-colorama. 2024-05-28T13:55:34.128 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-05-28T13:55:34.151 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-05-28T13:55:34.169 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-05-28T13:55:34.445 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-05-28T13:55:34.472 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package cephfs-shell. 2024-05-28T13:55:34.481 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:34.521 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking cephfs-shell (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:34.629 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-28T13:55:34.741 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package cephfs-top. 2024-05-28T13:55:34.750 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3154-g9ed025fb-1jammy_all.deb ... 2024-05-28T13:55:34.782 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking cephfs-top (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:34.913 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-05-28T13:55:35.001 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-28T13:55:35.011 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:35.051 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libcephfs-dev (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:35.303 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-05-28T13:55:35.368 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-05-28T13:55:35.387 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package lua-socket:amd64. 2024-05-28T13:55:35.397 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-05-28T13:55:35.433 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-28T13:55:35.437 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-28T13:55:35.559 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-28T13:55:35.767 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package lua-sec:amd64. 2024-05-28T13:55:35.776 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-05-28T13:55:35.816 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-05-28T13:55:35.976 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-28T13:55:36.129 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package nvme-cli. 2024-05-28T13:55:36.141 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-28T13:55:36.164 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-28T13:55:36.170 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-28T13:55:36.455 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-28T13:55:36.551 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package pkg-config. 2024-05-28T13:55:36.564 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-05-28T13:55:36.600 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-05-28T13:55:36.738 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-28T13:55:36.845 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-28T13:55:36.859 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-28T13:55:36.903 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-28T13:55:37.072 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up zip (3.0-12build2) ... 2024-05-28T13:55:37.223 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-28T13:55:37.349 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-28T13:55:37.361 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-28T13:55:37.398 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-28T13:55:37.692 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pastescript. 2024-05-28T13:55:37.705 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-05-28T13:55:37.742 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-28T13:55:37.950 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-28T13:55:38.062 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pluggy. 2024-05-28T13:55:38.074 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-28T13:55:38.112 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-28T13:55:38.235 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-28T13:55:38.378 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-28T13:55:38.382 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-psutil. 2024-05-28T13:55:38.394 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-28T13:55:38.423 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-28T13:55:38.528 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-28T13:55:38.760 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-py. 2024-05-28T13:55:38.772 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-05-28T13:55:38.846 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-28T13:55:38.880 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-28T13:55:39.137 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pygments. 2024-05-28T13:55:39.150 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-28T13:55:39.187 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-28T13:55:39.297 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-28T13:55:39.565 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-28T13:55:39.599 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-28T13:55:39.611 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-28T13:55:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-28T13:55:39.936 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-toml. 2024-05-28T13:55:39.948 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-05-28T13:55:39.958 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-28T13:55:40.019 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-28T13:55:40.285 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-05-28T13:55:40.322 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pytest. 2024-05-28T13:55:40.334 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-28T13:55:40.371 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-28T13:55:40.594 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-28T13:55:40.717 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-simplejson. 2024-05-28T13:55:40.729 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-28T13:55:40.766 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-28T13:55:40.945 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-28T13:55:41.036 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-28T13:55:41.048 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-28T13:55:41.086 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-28T13:55:41.213 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-28T13:55:41.363 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-ceph-common (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:41.681 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-05-28T13:55:41.759 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package radosgw. 2024-05-28T13:55:41.768 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../122-radosgw_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:41.809 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking radosgw (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:42.069 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-28T13:55:42.484 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package rbd-fuse. 2024-05-28T13:55:42.497 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3154-g9ed025fb-1jammy_amd64.deb ... 2024-05-28T13:55:42.534 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking rbd-fuse (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:42.802 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-28T13:55:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-28T13:55:43.248 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-28T13:55:43.371 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-05-28T13:55:43.518 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-28T13:55:43.614 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-28T13:55:43.749 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-28T13:55:43.911 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-28T13:55:44.075 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-28T13:55:44.201 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-28T13:55:44.302 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-28T13:55:44.518 INFO:teuthology.orchestra.run.smithi042.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-28T13:55:44.563 INFO:teuthology.orchestra.run.smithi042.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-05-28T13:55:44.625 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up cephadm (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:44.636 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-28T13:55:44.913 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-28T13:55:45.180 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-28T13:55:45.228 INFO:teuthology.orchestra.run.smithi042.stdout:Adding system user cephadm....done 2024-05-28T13:55:45.306 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-28T13:55:45.383 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-28T13:55:45.666 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-28T13:55:45.676 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-28T13:55:45.942 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-28T13:55:45.961 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-28T13:55:46.112 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-28T13:55:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-28T13:55:46.396 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-28T13:55:46.477 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-28T13:55:46.681 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-28T13:55:46.728 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-28T13:55:46.832 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-28T13:55:47.004 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-28T13:55:47.130 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-28T13:55:47.200 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-28T13:55:47.456 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-28T13:55:47.582 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-28T13:55:47.585 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-28T13:55:47.870 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-05-28T13:55:47.892 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-28T13:55:48.146 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-28T13:55:48.235 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-28T13:55:48.448 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-05-28T13:55:48.486 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-28T13:55:48.587 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-05-28T13:55:48.707 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-28T13:55:48.712 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-28T13:55:48.801 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-28T13:55:48.983 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-28T13:55:49.056 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-28T13:55:49.157 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-28T13:55:49.260 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-28T13:55:49.550 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-28T13:55:49.552 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-ceph-argparse (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:49.843 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up lua-any (27ubuntu1) ... 2024-05-28T13:55:49.853 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-28T13:55:50.002 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-28T13:55:50.004 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-28T13:55:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-05-28T13:55:50.270 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-28T13:55:50.387 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-05-28T13:55:50.457 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-05-28T13:55:50.616 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-28T13:55:50.755 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-28T13:55:50.892 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-05-28T13:55:51.072 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-28T13:55:51.301 INFO:teuthology.orchestra.run.smithi042.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-05-28T13:55:51.331 INFO:teuthology.orchestra.run.smithi042.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-05-28T13:55:51.348 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-28T13:55:51.395 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-28T13:55:51.529 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-28T13:55:51.586 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-28T13:55:51.844 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-28T13:55:51.922 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-28T13:55:52.073 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-28T13:55:52.121 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-28T13:55:52.375 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-28T13:55:52.406 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-28T13:55:52.685 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-28T13:55:52.799 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-28T13:55:53.002 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up zip (3.0-12build2) ... 2024-05-28T13:55:53.092 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-28T13:55:53.153 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-28T13:55:53.453 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-28T13:55:53.595 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up librados2 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:53.721 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:53.872 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-05-28T13:55:53.897 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-28T13:55:54.183 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libcephfs2 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:54.189 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-28T13:55:54.318 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libradosstriper1 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:54.324 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-28T13:55:54.446 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-28T13:55:54.450 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-28T13:55:54.737 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up librbd1 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:54.785 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-28T13:55:54.863 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-modules-core (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:54.988 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-fuse (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:55.197 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-28T13:55:55.197 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-28T13:55:55.219 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-28T13:55:55.487 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-28T13:55:55.913 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-28T13:55:55.923 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libcephfs-dev (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:56.024 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-rados (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:56.158 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up librgw2 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:56.240 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-05-28T13:55:56.293 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up cephfs-top (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:56.419 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-28T13:55:56.566 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-28T13:55:56.934 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-28T13:55:57.202 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-28T13:55:57.370 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-ceph-common (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:57.679 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-05-28T13:55:57.764 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-rbd (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:57.889 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up rbd-fuse (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:58.015 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-rgw (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:58.022 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-28T13:55:58.175 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-cephfs (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:58.301 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up cephfs-shell (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:58.444 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-common (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:55:59.030 INFO:teuthology.orchestra.run.smithi063.stdout:Adding group ceph....done 2024-05-28T13:55:59.328 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-05-28T13:55:59.414 INFO:teuthology.orchestra.run.smithi063.stdout:Adding system user ceph....done 2024-05-28T13:55:59.583 INFO:teuthology.orchestra.run.smithi063.stdout:Setting system user ceph properties....done 2024-05-28T13:55:59.607 INFO:teuthology.orchestra.run.smithi063.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-28T13:55:59.638 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-28T13:55:59.760 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-28T13:55:59.797 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-28T13:56:00.115 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-28T13:56:00.178 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-28T13:56:00.258 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-28T13:56:00.384 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-28T13:56:00.735 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-28T13:56:00.902 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up cephfs-mirror (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:01.012 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-28T13:56:01.118 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-28T13:56:01.118 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-28T13:56:01.262 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-28T13:56:01.380 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-28T13:56:01.698 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-28T13:56:01.787 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-test (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:01.913 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up radosgw (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:01.974 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-28T13:56:02.251 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-28T13:56:02.478 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-28T13:56:02.478 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-28T13:56:02.527 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-28T13:56:02.794 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-28T13:56:03.071 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-28T13:56:03.156 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-base (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:03.180 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-28T13:56:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-28T13:56:03.518 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-28T13:56:03.650 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-28T13:56:03.975 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-28T13:56:04.224 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mds (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:04.293 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-28T13:56:04.577 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-28T13:56:04.686 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-05-28T13:56:04.787 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-28T13:56:04.816 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-28T13:56:04.816 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-28T13:56:04.868 INFO:teuthology.orchestra.run.smithi042.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-28T13:56:05.123 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-28T13:56:05.257 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-28T13:56:05.497 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:05.700 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-28T13:56:05.701 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-28T13:56:05.702 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-28T13:56:05.976 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up lua-any (27ubuntu1) ... 2024-05-28T13:56:06.102 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-28T13:56:06.357 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-osd (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:06.370 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-28T13:56:06.505 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-05-28T13:56:06.724 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-28T13:56:06.724 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-28T13:56:06.831 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-28T13:56:07.157 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-28T13:56:07.317 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:07.443 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:07.450 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-28T13:56:07.612 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mon (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:07.694 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-28T13:56:07.826 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-28T13:56:07.827 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-28T13:56:07.963 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-28T13:56:08.230 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-28T13:56:08.453 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-cephadm (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:08.506 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-28T13:56:08.621 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:08.747 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-dashboard (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:08.890 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-volume (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:08.900 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-28T13:56:09.202 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-28T13:56:09.259 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-28T13:56:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-28T13:56:09.520 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-28T13:56:09.554 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-28T13:56:09.663 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up librados2 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:09.789 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:09.939 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-05-28T13:56:10.277 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libcephfs2 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:10.403 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libradosstriper1 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:10.520 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-28T13:56:10.775 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-28T13:56:10.797 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up librbd1 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:10.923 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-modules-core (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:11.049 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-fuse (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:11.303 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-28T13:56:11.303 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-28T13:56:11.496 INFO:teuthology.orchestra.run.smithi063.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-28T13:56:11.502 DEBUG:teuthology.orchestra.run.smithi063:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-28T13:56:11.578 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-05-28T13:56:11.760 INFO:teuthology.orchestra.run.smithi063.stdout:Building dependency tree... 2024-05-28T13:56:11.761 INFO:teuthology.orchestra.run.smithi063.stdout:Reading state information... 2024-05-28T13:56:11.945 INFO:teuthology.orchestra.run.smithi063.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-05-28T13:56:11.945 INFO:teuthology.orchestra.run.smithi063.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-05-28T13:56:11.946 INFO:teuthology.orchestra.run.smithi063.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-05-28T13:56:11.946 INFO:teuthology.orchestra.run.smithi063.stdout:The following packages were automatically installed and are no longer required: 2024-05-28T13:56:11.946 INFO:teuthology.orchestra.run.smithi063.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-28T13:56:11.946 INFO:teuthology.orchestra.run.smithi063.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-28T13:56:11.946 INFO:teuthology.orchestra.run.smithi063.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-28T13:56:11.947 INFO:teuthology.orchestra.run.smithi063.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-28T13:56:11.947 INFO:teuthology.orchestra.run.smithi063.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-28T13:56:11.947 INFO:teuthology.orchestra.run.smithi063.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-28T13:56:11.947 INFO:teuthology.orchestra.run.smithi063.stdout:The following additional packages will be installed: 2024-05-28T13:56:11.947 INFO:teuthology.orchestra.run.smithi063.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-05-28T13:56:11.948 INFO:teuthology.orchestra.run.smithi063.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-05-28T13:56:11.948 INFO:teuthology.orchestra.run.smithi063.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-05-28T13:56:11.948 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-05-28T13:56:11.948 INFO:teuthology.orchestra.run.smithi063.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-28T13:56:11.949 INFO:teuthology.orchestra.run.smithi063.stdout:Suggested packages: 2024-05-28T13:56:11.949 INFO:teuthology.orchestra.run.smithi063.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-05-28T13:56:11.949 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-05-28T13:56:11.949 INFO:teuthology.orchestra.run.smithi063.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-05-28T13:56:11.959 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libcephfs-dev (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:12.005 INFO:teuthology.orchestra.run.smithi063.stdout:The following NEW packages will be installed: 2024-05-28T13:56:12.006 INFO:teuthology.orchestra.run.smithi063.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-05-28T13:56:12.006 INFO:teuthology.orchestra.run.smithi063.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-05-28T13:56:12.006 INFO:teuthology.orchestra.run.smithi063.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-05-28T13:56:12.006 INFO:teuthology.orchestra.run.smithi063.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-05-28T13:56:12.006 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-05-28T13:56:12.006 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-05-28T13:56:12.006 INFO:teuthology.orchestra.run.smithi063.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-28T13:56:12.007 INFO:teuthology.orchestra.run.smithi063.stdout:The following packages will be upgraded: 2024-05-28T13:56:12.007 INFO:teuthology.orchestra.run.smithi063.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-05-28T13:56:12.084 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-rados (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:12.093 INFO:teuthology.orchestra.run.smithi063.stdout:4 upgraded, 39 newly installed, 0 to remove and 306 not upgraded. 2024-05-28T13:56:12.093 INFO:teuthology.orchestra.run.smithi063.stdout:Need to get 55.7 MB of archives. 2024-05-28T13:56:12.093 INFO:teuthology.orchestra.run.smithi063.stdout:After this operation, 202 MB of additional disk space will be used. 2024-05-28T13:56:12.093 INFO:teuthology.orchestra.run.smithi063.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-05-28T13:56:12.210 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up librgw2 (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:12.261 INFO:teuthology.orchestra.run.smithi063.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-05-28T13:56:12.266 INFO:teuthology.orchestra.run.smithi063.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-05-28T13:56:12.299 INFO:teuthology.orchestra.run.smithi063.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-05-28T13:56:12.300 INFO:teuthology.orchestra.run.smithi063.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-05-28T13:56:12.300 INFO:teuthology.orchestra.run.smithi063.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-05-28T13:56:12.308 INFO:teuthology.orchestra.run.smithi063.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-05-28T13:56:12.308 INFO:teuthology.orchestra.run.smithi063.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-05-28T13:56:12.310 INFO:teuthology.orchestra.run.smithi063.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-05-28T13:56:12.312 INFO:teuthology.orchestra.run.smithi063.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-05-28T13:56:12.312 INFO:teuthology.orchestra.run.smithi063.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-05-28T13:56:12.312 INFO:teuthology.orchestra.run.smithi063.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-05-28T13:56:12.324 INFO:teuthology.orchestra.run.smithi063.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-05-28T13:56:12.324 INFO:teuthology.orchestra.run.smithi063.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-05-28T13:56:12.325 INFO:teuthology.orchestra.run.smithi063.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-05-28T13:56:12.345 INFO:teuthology.orchestra.run.smithi063.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-05-28T13:56:12.345 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up cephfs-top (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:12.446 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-28T13:56:12.612 INFO:teuthology.orchestra.run.smithi063.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-05-28T13:56:12.614 INFO:teuthology.orchestra.run.smithi063.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-05-28T13:56:12.614 INFO:teuthology.orchestra.run.smithi063.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-05-28T13:56:12.617 INFO:teuthology.orchestra.run.smithi063.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-05-28T13:56:12.618 INFO:teuthology.orchestra.run.smithi063.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-05-28T13:56:12.622 INFO:teuthology.orchestra.run.smithi063.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-05-28T13:56:12.624 INFO:teuthology.orchestra.run.smithi063.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-05-28T13:56:12.631 INFO:teuthology.orchestra.run.smithi063.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-05-28T13:56:12.632 INFO:teuthology.orchestra.run.smithi063.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-05-28T13:56:12.654 INFO:teuthology.orchestra.run.smithi063.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-05-28T13:56:12.658 INFO:teuthology.orchestra.run.smithi063.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-05-28T13:56:12.666 INFO:teuthology.orchestra.run.smithi063.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-05-28T13:56:12.689 INFO:teuthology.orchestra.run.smithi063.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-05-28T13:56:12.692 INFO:teuthology.orchestra.run.smithi063.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-05-28T13:56:12.693 INFO:teuthology.orchestra.run.smithi063.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-05-28T13:56:12.693 INFO:teuthology.orchestra.run.smithi063.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-05-28T13:56:12.694 INFO:teuthology.orchestra.run.smithi063.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-05-28T13:56:12.706 INFO:teuthology.orchestra.run.smithi063.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-05-28T13:56:12.707 INFO:teuthology.orchestra.run.smithi063.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-05-28T13:56:12.876 INFO:teuthology.orchestra.run.smithi063.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-05-28T13:56:12.876 INFO:teuthology.orchestra.run.smithi063.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-05-28T13:56:12.876 INFO:teuthology.orchestra.run.smithi063.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-05-28T13:56:12.877 INFO:teuthology.orchestra.run.smithi063.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-05-28T13:56:12.877 INFO:teuthology.orchestra.run.smithi063.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-05-28T13:56:12.877 INFO:teuthology.orchestra.run.smithi063.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-05-28T13:56:12.891 INFO:teuthology.orchestra.run.smithi063.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-05-28T13:56:12.892 INFO:teuthology.orchestra.run.smithi063.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-05-28T13:56:13.287 INFO:teuthology.orchestra.run.smithi063.stdout:Preconfiguring packages ... 2024-05-28T13:56:13.450 INFO:teuthology.orchestra.run.smithi063.stdout:Fetched 55.7 MB in 1s (63.6 MB/s) 2024-05-28T13:56:13.749 INFO:teuthology.orchestra.run.smithi063.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140925 files and directories currently installed.) 2024-05-28T13:56:13.752 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-28T13:56:13.791 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-rbd (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:13.898 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-28T13:56:13.917 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up rbd-fuse (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:14.043 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-rgw (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:14.161 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-cephfs (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:14.287 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up cephfs-shell (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:14.388 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-common (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:14.566 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-05-28T13:56:15.258 INFO:teuthology.orchestra.run.smithi042.stdout:Adding group ceph....done 2024-05-28T13:56:15.626 INFO:teuthology.orchestra.run.smithi042.stdout:Adding system user ceph....done 2024-05-28T13:56:15.698 INFO:teuthology.orchestra.run.smithi063.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-05-28T13:56:15.828 INFO:teuthology.orchestra.run.smithi042.stdout:Setting system user ceph properties....done 2024-05-28T13:56:15.851 INFO:teuthology.orchestra.run.smithi042.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-28T13:56:15.979 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package flex. 2024-05-28T13:56:16.004 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-28T13:56:16.053 INFO:teuthology.orchestra.run.smithi063.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140925 files and directories currently installed.) 2024-05-28T13:56:16.057 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-05-28T13:56:16.103 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking flex (2.6.4-8build2) ... 2024-05-28T13:56:16.427 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-28T13:56:16.648 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-28T13:56:16.792 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-05-28T13:56:17.122 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up cephfs-mirror (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:17.216 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-28T13:56:17.339 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-28T13:56:17.339 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-28T13:56:17.774 INFO:teuthology.orchestra.run.smithi063.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140998 files and directories currently installed.) 2024-05-28T13:56:17.778 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-28T13:56:17.934 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-28T13:56:18.015 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-test (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:18.150 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up radosgw (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:18.330 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-28T13:56:18.490 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-28T13:56:18.695 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-28T13:56:18.695 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-28T13:56:18.807 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-28T13:56:19.258 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package bison. 2024-05-28T13:56:19.338 INFO:teuthology.orchestra.run.smithi063.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140998 files and directories currently installed.) 2024-05-28T13:56:19.342 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-05-28T13:56:19.401 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-base (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:19.418 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-05-28T13:56:19.756 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-28T13:56:19.928 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-28T13:56:19.943 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-05-28T13:56:19.977 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-05-28T13:56:20.306 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package dump. 2024-05-28T13:56:20.322 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-05-28T13:56:20.364 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking dump (0.4b47-3) ... 2024-05-28T13:56:20.479 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mds (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:20.660 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package indent. 2024-05-28T13:56:20.676 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-05-28T13:56:20.718 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-28T13:56:20.725 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-28T13:56:20.725 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-28T13:56:20.988 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-28T13:56:21.004 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-05-28T13:56:21.037 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-28T13:56:21.348 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:21.358 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-28T13:56:21.374 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-05-28T13:56:21.449 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-28T13:56:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-28T13:56:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-28T13:56:21.719 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-28T13:56:21.735 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-05-28T13:56:21.777 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-05-28T13:56:22.081 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libfl2:amd64. 2024-05-28T13:56:22.097 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-05-28T13:56:22.131 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-05-28T13:56:22.207 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-osd (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:22.467 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-05-28T13:56:22.484 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-05-28T13:56:22.533 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-28T13:56:22.533 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-28T13:56:22.533 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-28T13:56:22.836 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libjson-perl. 2024-05-28T13:56:22.853 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-05-28T13:56:22.903 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-05-28T13:56:23.176 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:23.265 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libllvm14:amd64. 2024-05-28T13:56:23.281 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-05-28T13:56:23.302 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:23.323 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-28T13:56:23.453 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mon (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:23.652 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-28T13:56:23.652 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-28T13:56:24.287 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-cephadm (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:24.413 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:24.547 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-dashboard (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:24.690 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-volume (19.0.0-3154-g9ed025fb-1jammy) ... 2024-05-28T13:56:24.718 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-28T13:56:24.734 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-28T13:56:24.801 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-28T13:56:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-28T13:56:25.087 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-28T13:56:25.104 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-28T13:56:25.138 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-28T13:56:25.177 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-28T13:56:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-28T13:56:25.432 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-28T13:56:25.448 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-28T13:56:25.490 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-28T13:56:25.826 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-28T13:56:25.843 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-05-28T13:56:25.918 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-28T13:56:26.238 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-28T13:56:26.254 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-05-28T13:56:26.288 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-28T13:56:26.431 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-28T13:56:26.624 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-28T13:56:26.640 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-28T13:56:26.682 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-28T13:56:27.061 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-28T13:56:27.077 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-28T13:56:27.119 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-28T13:56:27.165 INFO:teuthology.orchestra.run.smithi042.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-28T13:56:27.170 DEBUG:teuthology.orchestra.run.smithi042:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-28T13:56:27.247 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-05-28T13:56:27.428 INFO:teuthology.orchestra.run.smithi042.stdout:Building dependency tree... 2024-05-28T13:56:27.429 INFO:teuthology.orchestra.run.smithi042.stdout:Reading state information... 2024-05-28T13:56:27.464 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-28T13:56:27.480 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-28T13:56:27.513 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-28T13:56:27.613 INFO:teuthology.orchestra.run.smithi042.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-05-28T13:56:27.613 INFO:teuthology.orchestra.run.smithi042.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-05-28T13:56:27.614 INFO:teuthology.orchestra.run.smithi042.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-05-28T13:56:27.614 INFO:teuthology.orchestra.run.smithi042.stdout:The following packages were automatically installed and are no longer required: 2024-05-28T13:56:27.614 INFO:teuthology.orchestra.run.smithi042.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-28T13:56:27.614 INFO:teuthology.orchestra.run.smithi042.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-28T13:56:27.614 INFO:teuthology.orchestra.run.smithi042.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-28T13:56:27.615 INFO:teuthology.orchestra.run.smithi042.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-28T13:56:27.615 INFO:teuthology.orchestra.run.smithi042.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-28T13:56:27.615 INFO:teuthology.orchestra.run.smithi042.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-28T13:56:27.615 INFO:teuthology.orchestra.run.smithi042.stdout:The following additional packages will be installed: 2024-05-28T13:56:27.615 INFO:teuthology.orchestra.run.smithi042.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-05-28T13:56:27.615 INFO:teuthology.orchestra.run.smithi042.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-05-28T13:56:27.616 INFO:teuthology.orchestra.run.smithi042.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-05-28T13:56:27.616 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-05-28T13:56:27.616 INFO:teuthology.orchestra.run.smithi042.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-28T13:56:27.617 INFO:teuthology.orchestra.run.smithi042.stdout:Suggested packages: 2024-05-28T13:56:27.617 INFO:teuthology.orchestra.run.smithi042.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-05-28T13:56:27.617 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-05-28T13:56:27.617 INFO:teuthology.orchestra.run.smithi042.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-05-28T13:56:27.674 INFO:teuthology.orchestra.run.smithi042.stdout:The following NEW packages will be installed: 2024-05-28T13:56:27.675 INFO:teuthology.orchestra.run.smithi042.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-05-28T13:56:27.675 INFO:teuthology.orchestra.run.smithi042.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-05-28T13:56:27.675 INFO:teuthology.orchestra.run.smithi042.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-05-28T13:56:27.675 INFO:teuthology.orchestra.run.smithi042.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-05-28T13:56:27.676 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-05-28T13:56:27.676 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-05-28T13:56:27.676 INFO:teuthology.orchestra.run.smithi042.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-28T13:56:27.676 INFO:teuthology.orchestra.run.smithi042.stdout:The following packages will be upgraded: 2024-05-28T13:56:27.677 INFO:teuthology.orchestra.run.smithi042.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-05-28T13:56:27.699 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-28T13:56:27.715 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-28T13:56:27.766 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-28T13:56:27.779 INFO:teuthology.orchestra.run.smithi042.stdout:4 upgraded, 39 newly installed, 0 to remove and 306 not upgraded. 2024-05-28T13:56:27.779 INFO:teuthology.orchestra.run.smithi042.stdout:Need to get 55.7 MB of archives. 2024-05-28T13:56:27.779 INFO:teuthology.orchestra.run.smithi042.stdout:After this operation, 202 MB of additional disk space will be used. 2024-05-28T13:56:27.779 INFO:teuthology.orchestra.run.smithi042.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-05-28T13:56:28.214 INFO:teuthology.orchestra.run.smithi042.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-05-28T13:56:28.240 INFO:teuthology.orchestra.run.smithi042.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-05-28T13:56:28.286 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-28T13:56:28.302 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-05-28T13:56:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-05-28T13:56:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-05-28T13:56:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-05-28T13:56:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-05-28T13:56:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-05-28T13:56:28.336 INFO:teuthology.orchestra.run.smithi042.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-05-28T13:56:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-05-28T13:56:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-05-28T13:56:28.339 INFO:teuthology.orchestra.run.smithi042.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-05-28T13:56:28.344 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-05-28T13:56:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-05-28T13:56:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-05-28T13:56:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-05-28T13:56:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-05-28T13:56:28.681 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package modemmanager. 2024-05-28T13:56:28.697 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-28T13:56:28.739 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-28T13:56:28.942 INFO:teuthology.orchestra.run.smithi042.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-05-28T13:56:28.944 INFO:teuthology.orchestra.run.smithi042.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-05-28T13:56:28.944 INFO:teuthology.orchestra.run.smithi042.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-05-28T13:56:28.947 INFO:teuthology.orchestra.run.smithi042.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-05-28T13:56:28.947 INFO:teuthology.orchestra.run.smithi042.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-05-28T13:56:28.952 INFO:teuthology.orchestra.run.smithi042.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-05-28T13:56:28.954 INFO:teuthology.orchestra.run.smithi042.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-05-28T13:56:28.971 INFO:teuthology.orchestra.run.smithi042.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-05-28T13:56:28.971 INFO:teuthology.orchestra.run.smithi042.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-05-28T13:56:29.092 INFO:teuthology.orchestra.run.smithi042.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-05-28T13:56:29.092 INFO:teuthology.orchestra.run.smithi042.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-05-28T13:56:29.106 INFO:teuthology.orchestra.run.smithi042.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-05-28T13:56:29.126 INFO:teuthology.orchestra.run.smithi042.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-05-28T13:56:29.139 INFO:teuthology.orchestra.run.smithi042.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-05-28T13:56:29.140 INFO:teuthology.orchestra.run.smithi042.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-05-28T13:56:29.141 INFO:teuthology.orchestra.run.smithi042.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-05-28T13:56:29.141 INFO:teuthology.orchestra.run.smithi042.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-05-28T13:56:29.171 INFO:teuthology.orchestra.run.smithi042.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-05-28T13:56:29.173 INFO:teuthology.orchestra.run.smithi042.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-05-28T13:56:29.413 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package network-manager. 2024-05-28T13:56:29.430 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-05-28T13:56:29.471 INFO:teuthology.orchestra.run.smithi042.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-05-28T13:56:29.471 INFO:teuthology.orchestra.run.smithi042.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-05-28T13:56:29.471 INFO:teuthology.orchestra.run.smithi042.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-05-28T13:56:29.471 INFO:teuthology.orchestra.run.smithi042.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-05-28T13:56:29.471 INFO:teuthology.orchestra.run.smithi042.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-05-28T13:56:29.472 INFO:teuthology.orchestra.run.smithi042.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-05-28T13:56:29.490 INFO:teuthology.orchestra.run.smithi042.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-05-28T13:56:29.490 INFO:teuthology.orchestra.run.smithi042.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-05-28T13:56:29.511 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-05-28T13:56:29.886 INFO:teuthology.orchestra.run.smithi042.stdout:Preconfiguring packages ... 2024-05-28T13:56:30.039 INFO:teuthology.orchestra.run.smithi042.stdout:Fetched 55.7 MB in 2s (30.9 MB/s) 2024-05-28T13:56:30.196 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ppp. 2024-05-28T13:56:30.212 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-05-28T13:56:30.453 INFO:teuthology.orchestra.run.smithi042.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140925 files and directories currently installed.) 2024-05-28T13:56:30.457 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-28T13:56:30.553 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-05-28T13:56:30.605 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-28T13:56:30.951 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package pptp-linux. 2024-05-28T13:56:30.966 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-05-28T13:56:31.042 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-05-28T13:56:31.239 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-05-28T13:56:31.337 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-28T13:56:31.352 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-05-28T13:56:31.395 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-05-28T13:56:31.740 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-28T13:56:31.756 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-05-28T13:56:31.798 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking postgresql-client-common (238) ... 2024-05-28T13:56:32.086 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package postgresql-client-14. 2024-05-28T13:56:32.102 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-28T13:56:32.136 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-28T13:56:32.347 INFO:teuthology.orchestra.run.smithi042.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-05-28T13:56:32.610 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package flex. 2024-05-28T13:56:32.683 INFO:teuthology.orchestra.run.smithi042.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140925 files and directories currently installed.) 2024-05-28T13:56:32.687 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-05-28T13:56:32.734 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking flex (2.6.4-8build2) ... 2024-05-28T13:56:32.740 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package postgresql-common. 2024-05-28T13:56:32.757 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-05-28T13:56:32.819 INFO:teuthology.orchestra.run.smithi063.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-28T13:56:32.848 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking postgresql-common (238) ... 2024-05-28T13:56:33.270 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-28T13:56:33.357 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package postgresql-14. 2024-05-28T13:56:33.373 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-28T13:56:33.417 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-28T13:56:33.431 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-05-28T13:56:33.815 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-28T13:56:34.414 INFO:teuthology.orchestra.run.smithi042.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140998 files and directories currently installed.) 2024-05-28T13:56:34.417 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-28T13:56:34.551 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package postgresql. 2024-05-28T13:56:34.563 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-28T13:56:34.567 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-05-28T13:56:34.643 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking postgresql (14+238) ... 2024-05-28T13:56:34.937 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package postgresql-client. 2024-05-28T13:56:34.953 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-05-28T13:56:35.036 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-28T13:56:35.232 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-28T13:56:35.412 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking postgresql-client (14+238) ... 2024-05-28T13:56:35.580 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-28T13:56:35.623 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-28T13:56:35.639 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-05-28T13:56:35.673 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking postgresql-contrib (14+238) ... 2024-05-28T13:56:35.971 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package bison. 2024-05-28T13:56:35.983 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package tcl8.6. 2024-05-28T13:56:36.000 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-05-28T13:56:36.042 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-28T13:56:36.052 INFO:teuthology.orchestra.run.smithi042.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140998 files and directories currently installed.) 2024-05-28T13:56:36.056 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-05-28T13:56:36.099 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-05-28T13:56:36.328 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package tcl. 2024-05-28T13:56:36.344 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-05-28T13:56:36.418 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-05-28T13:56:36.516 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-28T13:56:36.531 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-05-28T13:56:36.574 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-05-28T13:56:36.739 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package wpasupplicant. 2024-05-28T13:56:36.756 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-05-28T13:56:36.797 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-28T13:56:36.869 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package dump. 2024-05-28T13:56:36.886 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-05-28T13:56:36.919 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking dump (0.4b47-3) ... 2024-05-28T13:56:37.173 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package indent. 2024-05-28T13:56:37.189 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-05-28T13:56:37.210 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-28T13:56:37.223 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-28T13:56:37.226 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-05-28T13:56:37.276 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-05-28T13:56:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-28T13:56:37.509 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-05-28T13:56:37.550 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-28T13:56:37.688 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-28T13:56:37.704 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-05-28T13:56:37.755 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-28T13:56:37.921 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-28T13:56:37.938 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-05-28T13:56:37.979 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-28T13:56:38.075 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up postgresql-client-common (238) ... 2024-05-28T13:56:38.207 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-28T13:56:38.223 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-05-28T13:56:38.258 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-05-28T13:56:38.294 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up flex (2.6.4-8build2) ... 2024-05-28T13:56:38.454 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-28T13:56:38.579 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-28T13:56:38.587 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libfl2:amd64. 2024-05-28T13:56:38.604 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-05-28T13:56:38.645 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-05-28T13:56:38.714 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-05-28T13:56:38.857 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-28T13:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-05-28T13:56:38.956 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-05-28T13:56:38.998 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-28T13:56:39.033 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-28T13:56:39.217 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libjson-perl. 2024-05-28T13:56:39.233 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-05-28T13:56:39.278 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-05-28T13:56:39.587 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libllvm14:amd64. 2024-05-28T13:56:39.604 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-05-28T13:56:39.645 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-28T13:56:40.032 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-28T13:56:40.147 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-28T13:56:40.557 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-28T13:56:40.558 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-28T13:56:41.050 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-28T13:56:41.066 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-28T13:56:41.133 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-28T13:56:41.259 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-28T13:56:41.335 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-28T13:56:41.352 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-28T13:56:41.385 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-28T13:56:41.410 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-05-28T13:56:41.586 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-28T13:56:41.722 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-28T13:56:41.737 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-05-28T13:56:41.738 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-28T13:56:41.780 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-28T13:56:41.888 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-05-28T13:56:42.064 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-05-28T13:56:42.158 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-28T13:56:42.174 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-05-28T13:56:42.215 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-28T13:56:42.216 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-28T13:56:42.341 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-05-28T13:56:42.519 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-28T13:56:42.535 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-05-28T13:56:42.568 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-28T13:56:42.922 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-28T13:56:42.938 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-28T13:56:42.980 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-28T13:56:43.141 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-28T13:56:43.267 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up dump (0.4b47-3) ... 2024-05-28T13:56:43.316 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-28T13:56:43.333 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-28T13:56:43.362 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-28T13:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-28T13:56:43.435 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-28T13:56:43.611 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-28T13:56:43.678 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-28T13:56:43.694 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-28T13:56:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-28T13:56:43.762 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up tcl (8.6.11+1build2) ... 2024-05-28T13:56:43.913 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libjson-perl (4.04000-1) ... 2024-05-28T13:56:43.972 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-28T13:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-28T13:56:44.030 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-28T13:56:44.089 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-05-28T13:56:44.195 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-05-28T13:56:44.282 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-28T13:56:44.425 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-28T13:56:44.592 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-28T13:56:44.608 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-05-28T13:56:44.626 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-28T13:56:44.650 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-05-28T13:56:44.777 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-28T13:56:44.928 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-05-28T13:56:45.020 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package modemmanager. 2024-05-28T13:56:45.037 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-28T13:56:45.078 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-28T13:56:45.146 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up postgresql-client (14+238) ... 2024-05-28T13:56:45.272 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-05-28T13:56:46.151 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:56:46.151 INFO:teuthology.orchestra.run.smithi063.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-28T13:56:46.152 INFO:teuthology.orchestra.run.smithi063.stdout:which means they are currently configured by ifupdown: 2024-05-28T13:56:46.152 INFO:teuthology.orchestra.run.smithi063.stdout:- enp3s0f1 2024-05-28T13:56:46.152 INFO:teuthology.orchestra.run.smithi063.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-28T13:56:46.152 INFO:teuthology.orchestra.run.smithi063.stdout:remove their configuration from /etc/network/interfaces. 2024-05-28T13:56:46.152 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:56:46.290 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-28T13:56:46.345 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package network-manager. 2024-05-28T13:56:46.362 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-05-28T13:56:46.412 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-05-28T13:56:46.699 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-28T13:56:47.052 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ppp. 2024-05-28T13:56:47.069 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-05-28T13:56:47.106 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-28T13:56:47.428 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-05-28T13:56:47.741 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package pptp-linux. 2024-05-28T13:56:47.755 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-05-28T13:56:47.790 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-05-28T13:56:47.825 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-28T13:56:47.976 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-28T13:56:48.026 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-28T13:56:48.042 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-05-28T13:56:48.076 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-05-28T13:56:48.118 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-05-28T13:56:48.286 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up postgresql-common (238) ... 2024-05-28T13:56:48.413 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-28T13:56:48.428 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-05-28T13:56:48.471 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking postgresql-client-common (238) ... 2024-05-28T13:56:48.758 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package postgresql-client-14. 2024-05-28T13:56:48.774 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-28T13:56:48.816 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-28T13:56:49.309 INFO:teuthology.orchestra.run.smithi063.stdout:Adding user postgres to group ssl-cert 2024-05-28T13:56:49.396 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package postgresql-common. 2024-05-28T13:56:49.413 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-05-28T13:56:49.474 INFO:teuthology.orchestra.run.smithi042.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-28T13:56:49.512 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking postgresql-common (238) ... 2024-05-28T13:56:49.525 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:56:49.526 INFO:teuthology.orchestra.run.smithi063.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-28T13:56:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-28T13:56:49.789 INFO:teuthology.orchestra.run.smithi063.stdout:Removing obsolete dictionary files: 2024-05-28T13:56:49.953 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package postgresql-14. 2024-05-28T13:56:49.970 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-28T13:56:50.023 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-28T13:56:50.519 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-28T13:56:51.012 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-28T13:56:51.156 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package postgresql. 2024-05-28T13:56:51.172 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-05-28T13:56:51.206 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking postgresql (14+238) ... 2024-05-28T13:56:51.365 INFO:teuthology.orchestra.run.smithi063.stdout:Creating new PostgreSQL cluster 14/main ... 2024-05-28T13:56:51.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-05-28T13:56:51.433 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package postgresql-client. 2024-05-28T13:56:51.435 INFO:teuthology.orchestra.run.smithi063.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-28T13:56:51.436 INFO:teuthology.orchestra.run.smithi063.stdout:This user must also own the server process. 2024-05-28T13:56:51.436 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:56:51.436 INFO:teuthology.orchestra.run.smithi063.stdout:The database cluster will be initialized with locale "en_US". 2024-05-28T13:56:51.436 INFO:teuthology.orchestra.run.smithi063.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-28T13:56:51.436 INFO:teuthology.orchestra.run.smithi063.stdout:The default text search configuration will be set to "english". 2024-05-28T13:56:51.436 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:56:51.436 INFO:teuthology.orchestra.run.smithi063.stdout:Data page checksums are disabled. 2024-05-28T13:56:51.436 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:56:51.436 INFO:teuthology.orchestra.run.smithi063.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-05-28T13:56:51.437 INFO:teuthology.orchestra.run.smithi063.stdout:creating subdirectories ... ok 2024-05-28T13:56:51.437 INFO:teuthology.orchestra.run.smithi063.stdout:selecting dynamic shared memory implementation ... posix 2024-05-28T13:56:51.449 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-05-28T13:56:51.461 INFO:teuthology.orchestra.run.smithi063.stdout:selecting default max_connections ... 100 2024-05-28T13:56:51.492 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking postgresql-client (14+238) ... 2024-05-28T13:56:51.497 INFO:teuthology.orchestra.run.smithi063.stdout:selecting default shared_buffers ... 128MB 2024-05-28T13:56:51.511 INFO:teuthology.orchestra.run.smithi063.stdout:selecting default time zone ... Etc/UTC 2024-05-28T13:56:51.512 INFO:teuthology.orchestra.run.smithi063.stdout:creating configuration files ... ok 2024-05-28T13:56:51.688 INFO:teuthology.orchestra.run.smithi063.stdout:running bootstrap script ... ok 2024-05-28T13:56:51.761 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-28T13:56:51.777 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-05-28T13:56:51.819 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking postgresql-contrib (14+238) ... 2024-05-28T13:56:52.147 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package tcl8.6. 2024-05-28T13:56:52.166 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-05-28T13:56:52.205 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-28T13:56:52.332 INFO:teuthology.orchestra.run.smithi063.stdout:performing post-bootstrap initialization ... ok 2024-05-28T13:56:52.457 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package tcl. 2024-05-28T13:56:52.475 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-05-28T13:56:52.512 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-05-28T13:56:52.900 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package wpasupplicant. 2024-05-28T13:56:52.917 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-05-28T13:56:52.943 INFO:teuthology.orchestra.run.smithi063.stdout:syncing data to disk ... ok 2024-05-28T13:56:53.010 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-28T13:56:53.373 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-28T13:56:53.390 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-05-28T13:56:53.431 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-05-28T13:56:53.500 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-28T13:56:53.793 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-28T13:56:53.809 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-05-28T13:56:53.851 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-28T13:56:54.163 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up postgresql-client-common (238) ... 2024-05-28T13:56:54.373 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up flex (2.6.4-8build2) ... 2024-05-28T13:56:54.499 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-28T13:56:54.617 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-28T13:56:54.718 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-05-28T13:56:54.836 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-28T13:56:54.970 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-28T13:56:55.970 INFO:teuthology.orchestra.run.smithi042.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-28T13:56:56.076 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-28T13:56:56.494 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-28T13:56:56.503 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-28T13:56:56.503 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-28T13:56:56.620 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up postgresql-contrib (14+238) ... 2024-05-28T13:56:56.746 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up postgresql (14+238) ... 2024-05-28T13:56:56.914 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-28T13:56:57.048 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-28T13:56:57.246 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-28T13:56:57.309 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-28T13:56:57.310 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-28T13:56:57.388 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-05-28T13:56:57.498 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-28T13:56:57.599 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-05-28T13:56:57.717 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-05-28T13:56:57.842 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-05-28T13:56:57.968 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-28T13:56:58.100 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-28T13:56:58.153 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-05-28T13:56:58.192 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-28T13:56:58.537 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-28T13:56:58.713 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-28T13:56:58.800 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-28T13:56:58.926 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up dump (0.4b47-3) ... 2024-05-28T13:56:59.021 INFO:teuthology.orchestra.run.smithi042.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-28T13:56:59.086 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-28T13:56:59.212 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-28T13:56:59.346 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up tcl (8.6.11+1build2) ... 2024-05-28T13:56:59.472 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libjson-perl (4.04000-1) ... 2024-05-28T13:56:59.598 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-05-28T13:56:59.705 INFO:teuthology.orchestra.run.smithi042.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-05-28T13:56:59.766 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-28T13:56:59.876 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-28T13:57:00.068 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-28T13:57:00.194 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-28T13:57:00.323 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-05-28T13:57:00.337 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-05-28T13:57:00.407 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-28T13:57:00.505 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up postgresql-client (14+238) ... 2024-05-28T13:57:00.615 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-05-28T13:57:01.001 INFO:teuthology.orchestra.run.smithi063.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-28T13:57:01.011 DEBUG:teuthology.parallel:result is None 2024-05-28T13:57:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:01.244 INFO:teuthology.orchestra.run.smithi042.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-28T13:57:01.244 INFO:teuthology.orchestra.run.smithi042.stdout:which means they are currently configured by ifupdown: 2024-05-28T13:57:01.244 INFO:teuthology.orchestra.run.smithi042.stdout:- ens1f0 2024-05-28T13:57:01.244 INFO:teuthology.orchestra.run.smithi042.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-28T13:57:01.244 INFO:teuthology.orchestra.run.smithi042.stdout:remove their configuration from /etc/network/interfaces. 2024-05-28T13:57:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:01.383 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-28T13:57:01.800 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-28T13:57:02.248 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-28T13:57:02.959 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-28T13:57:03.102 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-28T13:57:03.253 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-05-28T13:57:03.388 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up postgresql-common (238) ... 2024-05-28T13:57:04.339 INFO:teuthology.orchestra.run.smithi042.stdout:Adding user postgres to group ssl-cert 2024-05-28T13:57:04.588 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:04.588 INFO:teuthology.orchestra.run.smithi042.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-28T13:57:04.843 INFO:teuthology.orchestra.run.smithi042.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-28T13:57:04.843 INFO:teuthology.orchestra.run.smithi042.stdout:Removing obsolete dictionary files: 2024-05-28T13:57:05.605 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-28T13:57:06.073 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-28T13:57:06.411 INFO:teuthology.orchestra.run.smithi042.stdout:Creating new PostgreSQL cluster 14/main ... 2024-05-28T13:57:06.430 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-05-28T13:57:06.484 INFO:teuthology.orchestra.run.smithi042.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-28T13:57:06.484 INFO:teuthology.orchestra.run.smithi042.stdout:This user must also own the server process. 2024-05-28T13:57:06.484 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:06.484 INFO:teuthology.orchestra.run.smithi042.stdout:The database cluster will be initialized with locale "en_US". 2024-05-28T13:57:06.485 INFO:teuthology.orchestra.run.smithi042.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-28T13:57:06.485 INFO:teuthology.orchestra.run.smithi042.stdout:The default text search configuration will be set to "english". 2024-05-28T13:57:06.485 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:06.485 INFO:teuthology.orchestra.run.smithi042.stdout:Data page checksums are disabled. 2024-05-28T13:57:06.485 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:06.485 INFO:teuthology.orchestra.run.smithi042.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-05-28T13:57:06.485 INFO:teuthology.orchestra.run.smithi042.stdout:creating subdirectories ... ok 2024-05-28T13:57:06.486 INFO:teuthology.orchestra.run.smithi042.stdout:selecting dynamic shared memory implementation ... posix 2024-05-28T13:57:06.510 INFO:teuthology.orchestra.run.smithi042.stdout:selecting default max_connections ... 100 2024-05-28T13:57:06.544 INFO:teuthology.orchestra.run.smithi042.stdout:selecting default shared_buffers ... 128MB 2024-05-28T13:57:06.559 INFO:teuthology.orchestra.run.smithi042.stdout:selecting default time zone ... Etc/UTC 2024-05-28T13:57:06.560 INFO:teuthology.orchestra.run.smithi042.stdout:creating configuration files ... ok 2024-05-28T13:57:06.737 INFO:teuthology.orchestra.run.smithi042.stdout:running bootstrap script ... ok 2024-05-28T13:57:07.356 INFO:teuthology.orchestra.run.smithi042.stdout:performing post-bootstrap initialization ... ok 2024-05-28T13:57:07.911 INFO:teuthology.orchestra.run.smithi042.stdout:syncing data to disk ... ok 2024-05-28T13:57:08.445 INFO:teuthology.orchestra.run.smithi042.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-28T13:57:11.439 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-28T13:57:11.564 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up postgresql-contrib (14+238) ... 2024-05-28T13:57:11.690 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up postgresql (14+238) ... 2024-05-28T13:57:11.800 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-28T13:57:11.901 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-28T13:57:12.161 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-28T13:57:12.161 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-28T13:57:12.953 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-28T13:57:13.045 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-28T13:57:13.406 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-28T13:57:13.566 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-28T13:57:15.069 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-05-28T13:57:15.169 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-28T13:57:15.714 INFO:teuthology.orchestra.run.smithi042.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-28T13:57:15.722 DEBUG:teuthology.parallel:result is None 2024-05-28T13:57:15.723 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=9ed025fbf4406fb54980a0647506b15895b6d863 2024-05-28T13:57:15.939 DEBUG:teuthology.orchestra.run.smithi042:> dpkg-query -W -f '${Version}' ceph 2024-05-28T13:57:15.962 INFO:teuthology.orchestra.run.smithi042.stdout:19.0.0-3154-g9ed025fb-1jammy 2024-05-28T13:57:15.963 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154-g9ed025fb-1jammy 2024-05-28T13:57:15.963 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154-g9ed025fb-1jammy is installed. 2024-05-28T13:57:15.965 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=9ed025fbf4406fb54980a0647506b15895b6d863 2024-05-28T13:57:16.104 DEBUG:teuthology.orchestra.run.smithi063:> dpkg-query -W -f '${Version}' ceph 2024-05-28T13:57:16.129 INFO:teuthology.orchestra.run.smithi063.stdout:19.0.0-3154-g9ed025fb-1jammy 2024-05-28T13:57:16.130 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154-g9ed025fb-1jammy 2024-05-28T13:57:16.130 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154-g9ed025fb-1jammy is installed. 2024-05-28T13:57:16.132 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-28T13:57:16.132 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:16.132 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-28T13:57:16.152 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:16.152 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-28T13:57:16.193 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-28T13:57:16.193 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:16.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-05-28T13:57:16.215 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-28T13:57:16.276 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:16.276 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/daemon-helper 2024-05-28T13:57:16.298 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-28T13:57:16.360 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-28T13:57:16.360 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:16.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-28T13:57:16.380 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-28T13:57:16.447 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:16.447 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-28T13:57:16.469 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-28T13:57:16.532 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-28T13:57:16.532 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:16.533 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-05-28T13:57:16.553 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-28T13:57:16.616 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:16.616 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/stdin-killer 2024-05-28T13:57:16.637 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-28T13:57:16.700 INFO:teuthology.run_tasks:Running task ceph... 2024-05-28T13:57:16.780 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-28T13:57:16.780 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /var/log/ceph 2024-05-28T13:57:16.784 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 777 /var/log/ceph 2024-05-28T13:57:16.804 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-28T13:57:16.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-28T13:57:16.847 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-28T13:57:16.868 INFO:tasks.ceph:Creating extra log directories... 2024-05-28T13:57:16.868 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-28T13:57:16.911 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-28T13:57:16.938 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-28T13:57:16.938 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-28T13:57:16.938 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-28T13:57:16.938 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:16.938 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-28T13:57:16.980 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-28T13:57:17.073 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:17.073 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-28T13:57:17.083 DEBUG:teuthology.orchestra.run.smithi063:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-28T13:57:17.179 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-28T13:57:17.179 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-28T13:57:17.179 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074', 'branch': 'rishabh-clone-stats', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}}', 'email': None, 'first_in_suite': False, 'job_id': '7730074', 'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'rishabh-clone-stats'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'mgr-vol-clone-stats', 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3033, 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'mgr-vol-clone-stats', 'suite_path': '/home/teuthworker/src/github.com_rishabh-d-dave_ceph_9ed025fbf4406fb54980a0647506b15895b6d863/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/rishabh-d-dave/ceph', 'suite_sha1': '9ed025fbf4406fb54980a0647506b15895b6d863', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi042.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNWkvlu8UwMHXr28BoHMNn0gWNh333ViF+9vdxWkqBx4QEJHSKz4S8H2bTfeaMYce0qEcJYhRIsKBLt78AcuiC0=', 'smithi063.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPshCWmES4Eoeq1NfxCIk7iNQ+4aMfQ2UavFJQlmGvYH2NXjFeCCU2ESV93e0xy4x+uctvGY/EcCMMppwmATAVU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgressReporter']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e7ae7580c417d6ab27fda98fecadca5d9f85d63e', 'timestamp': '2024-05-28_13:36:18', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841'} 2024-05-28T13:57:17.180 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-28T13:57:17.184 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-28T13:57:17.192 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/run/ceph 2024-05-28T13:57:17.235 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m0777 -- /var/run/ceph 2024-05-28T13:57:17.256 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:17.256 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-05-28T13:57:17.305 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-28T13:57:17.305 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-05-28T13:57:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-28T13:57:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-28T13:57:17.357 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 896 Links: 1 2024-05-28T13:57:17.357 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-28T13:57:17.357 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-05-28 13:50:38.045726434 +0000 2024-05-28T13:57:17.357 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-05-28 13:50:38.041726469 +0000 2024-05-28T13:57:17.357 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-05-28 13:50:38.041726469 +0000 2024-05-28T13:57:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-05-28 13:50:38.041726469 +0000 2024-05-28T13:57:17.358 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-28T13:57:17.419 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-05-28T13:57:17.419 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-05-28T13:57:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000268185 s, 1.9 MB/s 2024-05-28T13:57:17.421 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-28T13:57:17.475 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-05-28T13:57:17.529 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-28T13:57:17.529 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-28T13:57:17.529 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 910 Links: 1 2024-05-28T13:57:17.529 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-28T13:57:17.529 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-05-28 13:50:38.845719288 +0000 2024-05-28T13:57:17.530 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-05-28 13:50:38.557721860 +0000 2024-05-28T13:57:17.530 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-05-28 13:50:38.557721860 +0000 2024-05-28T13:57:17.530 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-05-28 13:50:38.557721860 +0000 2024-05-28T13:57:17.530 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-28T13:57:17.596 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-05-28T13:57:17.596 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-05-28T13:57:17.596 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000264484 s, 1.9 MB/s 2024-05-28T13:57:17.598 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-28T13:57:17.651 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-05-28T13:57:17.701 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-28T13:57:17.701 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-28T13:57:17.701 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 927 Links: 1 2024-05-28T13:57:17.701 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-28T13:57:17.701 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-05-28 13:50:39.397714357 +0000 2024-05-28T13:57:17.701 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-05-28 13:50:39.125716786 +0000 2024-05-28T13:57:17.701 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-05-28 13:50:39.125716786 +0000 2024-05-28T13:57:17.702 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-05-28 13:50:39.125716786 +0000 2024-05-28T13:57:17.702 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-28T13:57:17.768 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-05-28T13:57:17.768 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-05-28T13:57:17.768 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000271724 s, 1.9 MB/s 2024-05-28T13:57:17.770 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-28T13:57:17.827 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-05-28T13:57:17.881 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-28T13:57:17.881 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-28T13:57:17.881 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 944 Links: 1 2024-05-28T13:57:17.881 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-28T13:57:17.881 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-05-28 13:50:39.973709212 +0000 2024-05-28T13:57:17.881 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-05-28 13:50:39.717711499 +0000 2024-05-28T13:57:17.881 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-05-28 13:50:39.717711499 +0000 2024-05-28T13:57:17.881 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-05-28 13:50:39.717711499 +0000 2024-05-28T13:57:17.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-28T13:57:17.950 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-05-28T13:57:17.950 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-05-28T13:57:17.950 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000262072 s, 2.0 MB/s 2024-05-28T13:57:17.950 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-28T13:57:18.003 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-28T13:57:18.003 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:18.003 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/scratch_devs of=/dev/stdout 2024-05-28T13:57:18.012 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-28T13:57:18.012 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_1 2024-05-28T13:57:18.064 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-28T13:57:18.064 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-28T13:57:18.064 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 891 Links: 1 2024-05-28T13:57:18.064 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-28T13:57:18.064 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-28 13:50:36.843667125 +0000 2024-05-28T13:57:18.064 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-28 13:50:36.571669553 +0000 2024-05-28T13:57:18.064 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-28 13:50:36.571669553 +0000 2024-05-28T13:57:18.064 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-28 13:50:36.571669553 +0000 2024-05-28T13:57:18.065 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-28T13:57:18.126 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-28T13:57:18.126 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-28T13:57:18.126 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.00063339 s, 808 kB/s 2024-05-28T13:57:18.128 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-28T13:57:18.185 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_2 2024-05-28T13:57:18.240 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-28T13:57:18.240 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-28T13:57:18.240 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 910 Links: 1 2024-05-28T13:57:18.240 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-28T13:57:18.240 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-28 13:50:37.455661667 +0000 2024-05-28T13:57:18.240 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-28 13:50:37.155664343 +0000 2024-05-28T13:57:18.240 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-28 13:50:37.155664343 +0000 2024-05-28T13:57:18.240 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-28 13:50:37.155664343 +0000 2024-05-28T13:57:18.240 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-28T13:57:18.301 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-28T13:57:18.302 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-28T13:57:18.302 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000353079 s, 1.5 MB/s 2024-05-28T13:57:18.303 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-28T13:57:18.362 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_3 2024-05-28T13:57:18.416 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-28T13:57:18.416 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-28T13:57:18.416 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 927 Links: 1 2024-05-28T13:57:18.416 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-28T13:57:18.416 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-28 13:50:37.983656956 +0000 2024-05-28T13:57:18.416 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-28 13:50:37.739659133 +0000 2024-05-28T13:57:18.416 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-28 13:50:37.739659133 +0000 2024-05-28T13:57:18.416 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-28 13:50:37.739659133 +0000 2024-05-28T13:57:18.416 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-28T13:57:18.478 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-28T13:57:18.479 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-28T13:57:18.479 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000374058 s, 1.4 MB/s 2024-05-28T13:57:18.481 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-28T13:57:18.538 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_4 2024-05-28T13:57:18.588 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-28T13:57:18.588 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-28T13:57:18.588 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 945 Links: 1 2024-05-28T13:57:18.588 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-28T13:57:18.588 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-28 13:50:38.515652209 +0000 2024-05-28T13:57:18.588 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-28 13:50:38.263654458 +0000 2024-05-28T13:57:18.588 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-28 13:50:38.263654458 +0000 2024-05-28T13:57:18.588 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-28 13:50:38.263654458 +0000 2024-05-28T13:57:18.589 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-28T13:57:18.655 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-28T13:57:18.655 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-28T13:57:18.655 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000660243 s, 775 kB/s 2024-05-28T13:57:18.657 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-28T13:57:18.710 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-28T13:57:18.710 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi063.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-28T13:57:18.710 INFO:tasks.ceph:Generating config... 2024-05-28T13:57:18.711 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-28T13:57:18.712 INFO:tasks.ceph:[client] debug client = 20 2024-05-28T13:57:18.712 INFO:tasks.ceph:[client] debug ms = 1 2024-05-28T13:57:18.712 INFO:tasks.ceph:[client] pid file = 2024-05-28T13:57:18.712 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-28T13:57:18.712 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-28T13:57:18.712 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-28T13:57:18.712 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-28T13:57:18.712 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-28T13:57:18.712 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-28T13:57:18.712 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-28T13:57:18.712 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-28T13:57:18.712 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-28T13:57:18.712 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-28T13:57:18.713 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-28T13:57:18.713 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-28T13:57:18.713 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-28T13:57:18.713 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-05-28T13:57:18.713 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-28T13:57:18.713 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-28T13:57:18.713 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-28T13:57:18.713 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-28T13:57:18.713 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-28T13:57:18.713 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-28T13:57:18.713 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-28T13:57:18.713 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-28T13:57:18.713 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-28T13:57:18.713 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-28T13:57:18.714 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-28T13:57:18.714 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-28T13:57:18.714 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-28T13:57:18.714 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-28T13:57:18.714 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-28T13:57:18.714 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-28T13:57:18.714 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-28T13:57:18.714 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-28T13:57:18.714 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-28T13:57:18.714 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-28T13:57:18.714 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-28T13:57:18.714 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-28T13:57:18.714 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-28T13:57:18.715 INFO:tasks.ceph:Setting up mon.a... 2024-05-28T13:57:18.715 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-28T13:57:18.762 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.keyring 2024-05-28T13:57:18.843 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-28T13:57:18.891 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-28T13:57:18.911 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.42'), ('mon.b', '172.21.15.63'), ('mon.c', '[v2:172.21.15.63:3301,v1:172.21.15.63:6790]')] 2024-05-28T13:57:18.912 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.42,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-28T13:57:18.913 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:18.914 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-28T13:57:18.960 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.42 --add b 172.21.15.63 --addv c '[v2:172.21.15.63:3301,v1:172.21.15.63:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-28T13:57:19.052 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-28T13:57:19.052 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: generated fsid 7380b1c2-773d-4bd7-abac-066c3536cd24 2024-05-28T13:57:19.052 INFO:teuthology.orchestra.run.smithi042.stdout:setting min_mon_release = quincy 2024-05-28T13:57:19.052 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 0 2024-05-28T13:57:19.052 INFO:teuthology.orchestra.run.smithi042.stdout:fsid 7380b1c2-773d-4bd7-abac-066c3536cd24 2024-05-28T13:57:19.052 INFO:teuthology.orchestra.run.smithi042.stdout:last_changed 2024-05-28T13:57:19.069072+0000 2024-05-28T13:57:19.052 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-05-28T13:57:19.069072+0000 2024-05-28T13:57:19.052 INFO:teuthology.orchestra.run.smithi042.stdout:min_mon_release 17 (quincy) 2024-05-28T13:57:19.052 INFO:teuthology.orchestra.run.smithi042.stdout:election_strategy: 1 2024-05-28T13:57:19.053 INFO:teuthology.orchestra.run.smithi042.stdout:0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-05-28T13:57:19.053 INFO:teuthology.orchestra.run.smithi042.stdout:1: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.b 2024-05-28T13:57:19.053 INFO:teuthology.orchestra.run.smithi042.stdout:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-28T13:57:19.053 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-28T13:57:19.055 DEBUG:teuthology.orchestra.run.smithi042:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-28T13:57:19.063 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7380b1c2-773d-4bd7-abac-066c3536cd24... 2024-05-28T13:57:19.064 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-28T13:57:19.110 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-28T13:57:19.151 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-05-28T13:57:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: chdir = "" 2024-05-28T13:57:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-28T13:57:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: auth supported = cephx 2024-05-28T13:57:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: filestore xattr use omap = true 2024-05-28T13:57:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: mon clock drift allowed = 1.000 2024-05-28T13:57:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: osd crush chooseleaf type = 0 2024-05-28T13:57:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: auth debug = true 2024-05-28T13:57:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on old message = true 2024-05-28T13:57:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on bug = true 2024-05-28T13:57:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-28T13:57:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: mon pg warn max object skew = 0 2024-05-28T13:57:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: # disable pg_autoscaler by default for new pools 2024-05-28T13:57:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-28T13:57:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default size = 2 2024-05-28T13:57:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow primary affinity = true 2024-05-28T13:57:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow pg remap = true 2024-05-28T13:57:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on legacy crush tunables = false 2024-05-28T13:57:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on crush straw calc version zero = false 2024-05-28T13:57:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on no sortbitwise = false 2024-05-28T13:57:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on osd down out interval zero = false 2024-05-28T13:57:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on too few osds = false 2024-05-28T13:57:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-28T13:57:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-28T13:57:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: mon_allow_pool_size_one = true 2024-05-28T13:57:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-28T13:57:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: osd default data pool replay window = 5 2024-05-28T13:57:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: mon allow pool delete = true 2024-05-28T13:57:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: mon cluster log file level = debug 2024-05-28T13:57:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: debug asserts on shutdown = true 2024-05-28T13:57:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: mon health detail to clog = false 2024-05-28T13:57:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: mon host = "172.21.15.42,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-28T13:57:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = 7380b1c2-773d-4bd7-abac-066c3536cd24 2024-05-28T13:57:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.154 INFO:teuthology.orchestra.run.smithi042.stdout:[osd] 2024-05-28T13:57:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: osd journal size = 100 2024-05-28T13:57:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub load threshold = 5.0 2024-05-28T13:57:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub max interval = 600 2024-05-28T13:57:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: osd mclock profile = high_recovery_ops 2024-05-28T13:57:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: osd recover clone overlap = true 2024-05-28T13:57:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: osd recovery max chunk = 1048576 2024-05-28T13:57:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug shutdown = true 2024-05-28T13:57:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug op order = true 2024-05-28T13:57:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug verify stray on activate = true 2024-05-28T13:57:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug trim objects = true 2024-05-28T13:57:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: osd open classes on start = true 2024-05-28T13:57:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug pg log writeout = true 2024-05-28T13:57:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: osd deep scrub update digest min age = 30 2024-05-28T13:57:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: osd map max advance = 10 2024-05-28T13:57:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: journal zero on create = true 2024-05-28T13:57:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: filestore ondisk finisher threads = 3 2024-05-28T13:57:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: filestore apply finisher threads = 3 2024-05-28T13:57:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: bdev debug aio = true 2024-05-28T13:57:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug misdirected ops = true 2024-05-28T13:57:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: bdev async discard = True 2024-05-28T13:57:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: bdev enable discard = True 2024-05-28T13:57:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore allocator = bitmap 2024-05-28T13:57:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore block size = 96636764160 2024-05-28T13:57:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore fsck on mount = True 2024-05-28T13:57:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: debug bluefs = 1/20 2024-05-28T13:57:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: debug bluestore = 1/20 2024-05-28T13:57:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-05-28T13:57:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: debug osd = 20 2024-05-28T13:57:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: debug rocksdb = 4/10 2024-05-28T13:57:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd backfillfull_ratio = 0.85 2024-05-28T13:57:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd full ratio = 0.9 2024-05-28T13:57:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd nearfull ratio = 0.8 2024-05-28T13:57:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: osd failsafe full ratio = 0.95 2024-05-28T13:57:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: osd objectstore = bluestore 2024-05-28T13:57:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: osd op complaint time = 180 2024-05-28T13:57:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.158 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr] 2024-05-28T13:57:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-05-28T13:57:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: debug mgr = 20 2024-05-28T13:57:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-05-28T13:57:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-05-28T13:57:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-05-28T13:57:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-05-28T13:57:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: mgr/telemetry/nag = false 2024-05-28T13:57:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: debug client = 20 2024-05-28T13:57:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: debug finisher = 20 2024-05-28T13:57:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.159 INFO:teuthology.orchestra.run.smithi042.stdout:[mon] 2024-05-28T13:57:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-05-28T13:57:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-05-28T13:57:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: debug paxos = 20 2024-05-28T13:57:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-05-28T13:57:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: mon data avail warn = 5 2024-05-28T13:57:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: mon mgr mkfs grace = 240 2024-05-28T13:57:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-05-28T13:57:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd reporter subtree level = osd 2024-05-28T13:57:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd prime pg temp = true 2024-05-28T13:57:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-05-28T13:57:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-28T13:57:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: auth mon ticket ttl = 660 # 11m 2024-05-28T13:57:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: auth service ticket ttl = 240 # 4m 2024-05-28T13:57:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: # don't complain about insecure global_id in the test suite 2024-05-28T13:57:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-28T13:57:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-28T13:57:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: # 1m isn't quite enough 2024-05-28T13:57:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: mon_down_mkfs_grace = 2m 2024-05-28T13:57:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_filestore_osds = false 2024-05-28T13:57:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: mon down mkfs grace = 300 2024-05-28T13:57:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: mon op complaint time = 120 2024-05-28T13:57:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:57:19.161 INFO:teuthology.orchestra.run.smithi042.stdout:[client] 2024-05-28T13:57:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: rgw cache enabled = true 2024-05-28T13:57:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable ops log = true 2024-05-28T13:57:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable usage log = true 2024-05-28T13:57:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-28T13:57:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-28T13:57:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: client mount timeout = 600 2024-05-28T13:57:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: debug client = 20 2024-05-28T13:57:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-05-28T13:57:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: pid file = "" 2024-05-28T13:57:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: rados mon op timeout = 900 2024-05-28T13:57:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: rados osd op timeout = 900 2024-05-28T13:57:19.162 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.a] 2024-05-28T13:57:19.162 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.b] 2024-05-28T13:57:19.162 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.c] 2024-05-28T13:57:19.162 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.a] 2024-05-28T13:57:19.162 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.c] 2024-05-28T13:57:19.162 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.b] 2024-05-28T13:57:19.163 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.d] 2024-05-28T13:57:19.163 INFO:teuthology.orchestra.run.smithi042.stdout:[mds] 2024-05-28T13:57:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: debug mds = 20 2024-05-28T13:57:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: debug mds balancer = 20 2024-05-28T13:57:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-05-28T13:57:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: mds debug frag = True 2024-05-28T13:57:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: mds debug scatterstat = True 2024-05-28T13:57:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: mds op complaint time = 180 2024-05-28T13:57:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: mds verify scatter = True 2024-05-28T13:57:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: osd op complaint time = 180 2024-05-28T13:57:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: rados mon op timeout = 900 2024-05-28T13:57:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: rados osd op timeout = 900 2024-05-28T13:57:19.165 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-28T13:57:19.166 INFO:teuthology.orchestra.run.smithi063.stdout: chdir = "" 2024-05-28T13:57:19.166 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-28T13:57:19.166 INFO:teuthology.orchestra.run.smithi063.stdout: auth supported = cephx 2024-05-28T13:57:19.166 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.166 INFO:teuthology.orchestra.run.smithi063.stdout: filestore xattr use omap = true 2024-05-28T13:57:19.166 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.166 INFO:teuthology.orchestra.run.smithi063.stdout: mon clock drift allowed = 1.000 2024-05-28T13:57:19.166 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.166 INFO:teuthology.orchestra.run.smithi063.stdout: osd crush chooseleaf type = 0 2024-05-28T13:57:19.166 INFO:teuthology.orchestra.run.smithi063.stdout: auth debug = true 2024-05-28T13:57:19.167 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.167 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on old message = true 2024-05-28T13:57:19.167 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on bug = true 2024-05-28T13:57:19.167 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.167 INFO:teuthology.orchestra.run.smithi063.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-28T13:57:19.167 INFO:teuthology.orchestra.run.smithi063.stdout: mon pg warn max object skew = 0 2024-05-28T13:57:19.168 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.168 INFO:teuthology.orchestra.run.smithi063.stdout: # disable pg_autoscaler by default for new pools 2024-05-28T13:57:19.168 INFO:teuthology.orchestra.run.smithi063.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-28T13:57:19.168 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.168 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default size = 2 2024-05-28T13:57:19.168 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.168 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow primary affinity = true 2024-05-28T13:57:19.168 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow pg remap = true 2024-05-28T13:57:19.168 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on legacy crush tunables = false 2024-05-28T13:57:19.169 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on crush straw calc version zero = false 2024-05-28T13:57:19.169 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on no sortbitwise = false 2024-05-28T13:57:19.169 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on osd down out interval zero = false 2024-05-28T13:57:19.169 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on too few osds = false 2024-05-28T13:57:19.169 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-28T13:57:19.169 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-28T13:57:19.169 INFO:teuthology.orchestra.run.smithi063.stdout: mon_allow_pool_size_one = true 2024-05-28T13:57:19.169 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.169 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-28T13:57:19.169 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.169 INFO:teuthology.orchestra.run.smithi063.stdout: osd default data pool replay window = 5 2024-05-28T13:57:19.169 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.169 INFO:teuthology.orchestra.run.smithi063.stdout: mon allow pool delete = true 2024-05-28T13:57:19.170 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.170 INFO:teuthology.orchestra.run.smithi063.stdout: mon cluster log file level = debug 2024-05-28T13:57:19.170 INFO:teuthology.orchestra.run.smithi063.stdout: debug asserts on shutdown = true 2024-05-28T13:57:19.170 INFO:teuthology.orchestra.run.smithi063.stdout: mon health detail to clog = false 2024-05-28T13:57:19.170 INFO:teuthology.orchestra.run.smithi063.stdout: mon host = "172.21.15.42,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-28T13:57:19.170 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = 7380b1c2-773d-4bd7-abac-066c3536cd24 2024-05-28T13:57:19.170 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.170 INFO:teuthology.orchestra.run.smithi063.stdout:[osd] 2024-05-28T13:57:19.170 INFO:teuthology.orchestra.run.smithi063.stdout: osd journal size = 100 2024-05-28T13:57:19.170 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.170 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub load threshold = 5.0 2024-05-28T13:57:19.170 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub max interval = 600 2024-05-28T13:57:19.170 INFO:teuthology.orchestra.run.smithi063.stdout: osd mclock profile = high_recovery_ops 2024-05-28T13:57:19.170 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.171 INFO:teuthology.orchestra.run.smithi063.stdout: osd recover clone overlap = true 2024-05-28T13:57:19.171 INFO:teuthology.orchestra.run.smithi063.stdout: osd recovery max chunk = 1048576 2024-05-28T13:57:19.171 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.171 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug shutdown = true 2024-05-28T13:57:19.171 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug op order = true 2024-05-28T13:57:19.171 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug verify stray on activate = true 2024-05-28T13:57:19.171 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug trim objects = true 2024-05-28T13:57:19.171 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.171 INFO:teuthology.orchestra.run.smithi063.stdout: osd open classes on start = true 2024-05-28T13:57:19.171 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug pg log writeout = true 2024-05-28T13:57:19.171 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.171 INFO:teuthology.orchestra.run.smithi063.stdout: osd deep scrub update digest min age = 30 2024-05-28T13:57:19.171 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.171 INFO:teuthology.orchestra.run.smithi063.stdout: osd map max advance = 10 2024-05-28T13:57:19.172 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.172 INFO:teuthology.orchestra.run.smithi063.stdout: journal zero on create = true 2024-05-28T13:57:19.172 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.172 INFO:teuthology.orchestra.run.smithi063.stdout: filestore ondisk finisher threads = 3 2024-05-28T13:57:19.172 INFO:teuthology.orchestra.run.smithi063.stdout: filestore apply finisher threads = 3 2024-05-28T13:57:19.172 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.172 INFO:teuthology.orchestra.run.smithi063.stdout: bdev debug aio = true 2024-05-28T13:57:19.172 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug misdirected ops = true 2024-05-28T13:57:19.172 INFO:teuthology.orchestra.run.smithi063.stdout: bdev async discard = True 2024-05-28T13:57:19.172 INFO:teuthology.orchestra.run.smithi063.stdout: bdev enable discard = True 2024-05-28T13:57:19.172 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore allocator = bitmap 2024-05-28T13:57:19.172 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore block size = 96636764160 2024-05-28T13:57:19.172 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore fsck on mount = True 2024-05-28T13:57:19.172 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluefs = 1/20 2024-05-28T13:57:19.173 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluestore = 1/20 2024-05-28T13:57:19.173 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-28T13:57:19.173 INFO:teuthology.orchestra.run.smithi063.stdout: debug osd = 20 2024-05-28T13:57:19.173 INFO:teuthology.orchestra.run.smithi063.stdout: debug rocksdb = 4/10 2024-05-28T13:57:19.173 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd backfillfull_ratio = 0.85 2024-05-28T13:57:19.173 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd full ratio = 0.9 2024-05-28T13:57:19.173 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd nearfull ratio = 0.8 2024-05-28T13:57:19.173 INFO:teuthology.orchestra.run.smithi063.stdout: osd failsafe full ratio = 0.95 2024-05-28T13:57:19.173 INFO:teuthology.orchestra.run.smithi063.stdout: osd objectstore = bluestore 2024-05-28T13:57:19.173 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-28T13:57:19.173 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.173 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr] 2024-05-28T13:57:19.173 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-28T13:57:19.173 INFO:teuthology.orchestra.run.smithi063.stdout: debug mgr = 20 2024-05-28T13:57:19.174 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-28T13:57:19.174 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-28T13:57:19.174 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-28T13:57:19.174 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-28T13:57:19.174 INFO:teuthology.orchestra.run.smithi063.stdout: mgr/telemetry/nag = false 2024-05-28T13:57:19.174 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-28T13:57:19.174 INFO:teuthology.orchestra.run.smithi063.stdout: debug finisher = 20 2024-05-28T13:57:19.174 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.174 INFO:teuthology.orchestra.run.smithi063.stdout:[mon] 2024-05-28T13:57:19.174 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-28T13:57:19.174 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-28T13:57:19.174 INFO:teuthology.orchestra.run.smithi063.stdout: debug paxos = 20 2024-05-28T13:57:19.174 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-28T13:57:19.175 INFO:teuthology.orchestra.run.smithi063.stdout: mon data avail warn = 5 2024-05-28T13:57:19.175 INFO:teuthology.orchestra.run.smithi063.stdout: mon mgr mkfs grace = 240 2024-05-28T13:57:19.175 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-28T13:57:19.175 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd reporter subtree level = osd 2024-05-28T13:57:19.175 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd prime pg temp = true 2024-05-28T13:57:19.175 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-28T13:57:19.175 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.175 INFO:teuthology.orchestra.run.smithi063.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-28T13:57:19.175 INFO:teuthology.orchestra.run.smithi063.stdout: auth mon ticket ttl = 660 # 11m 2024-05-28T13:57:19.175 INFO:teuthology.orchestra.run.smithi063.stdout: auth service ticket ttl = 240 # 4m 2024-05-28T13:57:19.175 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout: # don't complain about insecure global_id in the test suite 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout: # 1m isn't quite enough 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout: mon_down_mkfs_grace = 2m 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_filestore_osds = false 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout: mon down mkfs grace = 300 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout: mon op complaint time = 120 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout:[client] 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout: rgw cache enabled = true 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable ops log = true 2024-05-28T13:57:19.176 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable usage log = true 2024-05-28T13:57:19.177 INFO:teuthology.orchestra.run.smithi063.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-28T13:57:19.177 INFO:teuthology.orchestra.run.smithi063.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-28T13:57:19.177 INFO:teuthology.orchestra.run.smithi063.stdout: client mount timeout = 600 2024-05-28T13:57:19.177 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-28T13:57:19.177 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-28T13:57:19.177 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = "" 2024-05-28T13:57:19.177 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-28T13:57:19.177 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-28T13:57:19.177 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.a] 2024-05-28T13:57:19.177 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.b] 2024-05-28T13:57:19.177 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.c] 2024-05-28T13:57:19.177 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a] 2024-05-28T13:57:19.177 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c] 2024-05-28T13:57:19.178 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b] 2024-05-28T13:57:19.178 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d] 2024-05-28T13:57:19.178 INFO:teuthology.orchestra.run.smithi063.stdout:[mds] 2024-05-28T13:57:19.178 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds = 20 2024-05-28T13:57:19.178 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds balancer = 20 2024-05-28T13:57:19.178 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-28T13:57:19.178 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug frag = True 2024-05-28T13:57:19.178 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug scatterstat = True 2024-05-28T13:57:19.178 INFO:teuthology.orchestra.run.smithi063.stdout: mds op complaint time = 180 2024-05-28T13:57:19.178 INFO:teuthology.orchestra.run.smithi063.stdout: mds verify scatter = True 2024-05-28T13:57:19.178 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-28T13:57:19.178 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-28T13:57:19.178 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-28T13:57:19.180 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-28T13:57:19.180 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-28T13:57:19.277 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-28T13:57:19.277 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:19.278 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-28T13:57:19.286 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:19.287 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-28T13:57:19.337 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T13:57:19.337 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:19.337 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-28T13:57:19.337 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-28T13:57:19.412 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:19.412 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-28T13:57:19.468 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T13:57:19.469 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:19.469 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-28T13:57:19.469 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-28T13:57:19.505 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:19.506 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-28T13:57:19.556 INFO:tasks.ceph:Setting up mon nodes... 2024-05-28T13:57:19.556 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-28T13:57:19.556 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-28T13:57:19.611 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-28T13:57:19.616 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-28T13:57:19.677 INFO:teuthology.orchestra.run.smithi063.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-28T13:57:19.695 INFO:tasks.ceph:Setting up mds nodes... 2024-05-28T13:57:19.695 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-28T13:57:19.751 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-28T13:57:19.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-28T13:57:19.776 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-28T13:57:19.881 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-28T13:57:19.886 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-28T13:57:19.906 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-28T13:57:19.965 INFO:teuthology.orchestra.run.smithi063.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-28T13:57:19.970 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-28T13:57:19.990 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-28T13:57:20.087 INFO:teuthology.orchestra.run.smithi063.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-28T13:57:20.091 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-28T13:57:20.111 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-28T13:57:20.112 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-28T13:57:20.154 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-28T13:57:20.171 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-28T13:57:20.259 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-28T13:57:20.276 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-28T13:57:20.318 INFO:teuthology.orchestra.run.smithi063.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-28T13:57:20.336 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-28T13:57:20.421 INFO:teuthology.orchestra.run.smithi063.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-28T13:57:20.439 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-28T13:57:20.439 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi063.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-28T13:57:20.439 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-28T13:57:20.459 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-28T13:57:20.459 INFO:tasks.ceph:role: osd.0 2024-05-28T13:57:20.460 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T13:57:20.460 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-28T13:57:20.519 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-28T13:57:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-28T13:57:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-28T13:57:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-28T13:57:20.520 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-28T13:57:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-05-28T13:57:20.520 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-28T13:57:20.520 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-28T13:57:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-28T13:57:20.520 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-28T13:57:20.544 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-05-28T13:57:20.547 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-05-28T13:57:20.547 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-28T13:57:20.613 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-28T13:57:20.672 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: /sbin/restorecon: command not found 2024-05-28T13:57:20.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T13:57:20.674 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-28T13:57:20.736 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-28T13:57:20.736 INFO:tasks.ceph:role: osd.1 2024-05-28T13:57:20.736 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T13:57:20.736 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-28T13:57:20.795 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-28T13:57:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-28T13:57:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-28T13:57:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-28T13:57:20.795 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-28T13:57:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-05-28T13:57:20.795 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-28T13:57:20.795 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-28T13:57:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-28T13:57:20.796 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-28T13:57:20.820 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-05-28T13:57:20.822 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-05-28T13:57:20.823 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-28T13:57:20.890 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-28T13:57:20.948 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: /sbin/restorecon: command not found 2024-05-28T13:57:20.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T13:57:20.949 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-28T13:57:21.016 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-28T13:57:21.016 INFO:tasks.ceph:role: osd.2 2024-05-28T13:57:21.016 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T13:57:21.016 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-28T13:57:21.076 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-28T13:57:21.076 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-28T13:57:21.076 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-28T13:57:21.076 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-28T13:57:21.076 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-28T13:57:21.076 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-05-28T13:57:21.076 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-28T13:57:21.076 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-28T13:57:21.076 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-28T13:57:21.076 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-28T13:57:21.100 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-05-28T13:57:21.102 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-05-28T13:57:21.103 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-28T13:57:21.169 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-28T13:57:21.227 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: /sbin/restorecon: command not found 2024-05-28T13:57:21.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T13:57:21.229 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-28T13:57:21.291 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-28T13:57:21.291 INFO:tasks.ceph:role: osd.3 2024-05-28T13:57:21.291 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T13:57:21.291 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-28T13:57:21.351 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-28T13:57:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-28T13:57:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-28T13:57:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-28T13:57:21.352 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-28T13:57:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-05-28T13:57:21.353 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-28T13:57:21.353 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-28T13:57:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-28T13:57:21.354 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-28T13:57:21.376 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-05-28T13:57:21.378 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-05-28T13:57:21.378 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-28T13:57:21.446 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-28T13:57:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: /sbin/restorecon: command not found 2024-05-28T13:57:21.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T13:57:21.505 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-28T13:57:21.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:21.646+0000 7f89cc6467c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-28T13:57:21.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:21.646+0000 7f89cc6467c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-28T13:57:21.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:21.646+0000 7f89cc6467c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-28T13:57:21.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:21.646+0000 7f89cc6467c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-28T13:57:21.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:21.646+0000 7f89cc6467c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-28T13:57:21.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:21.646+0000 7f89cc6467c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-28T13:57:24.664 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-28T13:57:24.685 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-28T13:57:24.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:24.794+0000 7f0a3422f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-28T13:57:24.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:24.794+0000 7f0a3422f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-28T13:57:24.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:24.794+0000 7f0a3422f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-28T13:57:24.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:24.794+0000 7f0a3422f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-28T13:57:24.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:24.794+0000 7f0a3422f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-28T13:57:24.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:24.794+0000 7f0a3422f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-28T13:57:27.839 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-28T13:57:27.860 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-28T13:57:27.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:27.970+0000 7f232e2577c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-28T13:57:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:27.970+0000 7f232e2577c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-28T13:57:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:27.970+0000 7f232e2577c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-28T13:57:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:27.970+0000 7f232e2577c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-28T13:57:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:27.970+0000 7f232e2577c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-28T13:57:27.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:27.974+0000 7f232e2577c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-28T13:57:30.978 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-28T13:57:30.999 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-28T13:57:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:31.110+0000 7fb5cb7887c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-28T13:57:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:31.110+0000 7fb5cb7887c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-28T13:57:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:31.110+0000 7fb5cb7887c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-28T13:57:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:31.110+0000 7fb5cb7887c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-28T13:57:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:31.110+0000 7fb5cb7887c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-28T13:57:31.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:31.110+0000 7fb5cb7887c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-28T13:57:34.154 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-28T13:57:34.176 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-28T13:57:34.198 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-28T13:57:34.198 INFO:tasks.ceph:role: osd.4 2024-05-28T13:57:34.198 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T13:57:34.198 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-28T13:57:34.268 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-28T13:57:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-28T13:57:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-28T13:57:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-28T13:57:34.268 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-28T13:57:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-28T13:57:34.268 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-28T13:57:34.269 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-28T13:57:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-28T13:57:34.269 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-28T13:57:34.658 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-28T13:57:34.661 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-28T13:57:34.661 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-28T13:57:34.752 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-28T13:57:34.769 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: /sbin/restorecon: command not found 2024-05-28T13:57:34.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T13:57:34.771 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-28T13:57:34.833 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-28T13:57:34.833 INFO:tasks.ceph:role: osd.5 2024-05-28T13:57:34.833 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T13:57:34.833 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-28T13:57:34.904 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-28T13:57:34.904 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-28T13:57:34.904 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-28T13:57:34.904 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-28T13:57:34.904 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-28T13:57:34.904 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-28T13:57:34.904 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-28T13:57:34.904 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-28T13:57:34.904 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-28T13:57:34.904 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-28T13:57:35.296 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-28T13:57:35.299 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-28T13:57:35.299 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-28T13:57:35.389 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-28T13:57:35.448 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: /sbin/restorecon: command not found 2024-05-28T13:57:35.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T13:57:35.450 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-28T13:57:35.512 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-28T13:57:35.512 INFO:tasks.ceph:role: osd.6 2024-05-28T13:57:35.512 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T13:57:35.512 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-28T13:57:35.584 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-28T13:57:35.584 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-28T13:57:35.584 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-28T13:57:35.584 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-28T13:57:35.584 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-28T13:57:35.584 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-28T13:57:35.584 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-28T13:57:35.584 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-28T13:57:35.584 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-28T13:57:35.584 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-28T13:57:35.995 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-28T13:57:35.999 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-28T13:57:35.999 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-28T13:57:36.141 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-28T13:57:36.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T13:57:36.162 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: /sbin/restorecon: command not found 2024-05-28T13:57:36.162 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-28T13:57:36.224 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-28T13:57:36.224 INFO:tasks.ceph:role: osd.7 2024-05-28T13:57:36.224 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T13:57:36.225 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-28T13:57:36.296 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-28T13:57:36.296 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-28T13:57:36.296 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-28T13:57:36.296 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-28T13:57:36.296 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-28T13:57:36.296 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-28T13:57:36.296 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-28T13:57:36.296 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-28T13:57:36.296 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-28T13:57:36.297 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-28T13:57:36.652 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-28T13:57:36.655 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-28T13:57:36.655 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-28T13:57:36.732 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-28T13:57:36.794 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: /sbin/restorecon: command not found 2024-05-28T13:57:36.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T13:57:36.795 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-28T13:57:36.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:36.983+0000 7f75d082b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-28T13:57:36.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:36.987+0000 7f75d082b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-28T13:57:36.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:36.987+0000 7f75d082b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-28T13:57:36.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:36.987+0000 7f75d082b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-28T13:57:36.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:36.987+0000 7f75d082b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-28T13:57:36.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:36.987+0000 7f75d082b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-28T13:57:40.047 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-28T13:57:40.068 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-28T13:57:40.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:40.183+0000 7f2b69fa17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-28T13:57:40.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:40.183+0000 7f2b69fa17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-28T13:57:40.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:40.183+0000 7f2b69fa17c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-28T13:57:40.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:40.183+0000 7f2b69fa17c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-28T13:57:40.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:40.183+0000 7f2b69fa17c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-28T13:57:40.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:40.183+0000 7f2b69fa17c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-28T13:57:43.251 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-28T13:57:43.273 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-28T13:57:43.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:43.391+0000 7f398d1677c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-28T13:57:43.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:43.391+0000 7f398d1677c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-28T13:57:43.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:43.391+0000 7f398d1677c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-28T13:57:43.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:43.391+0000 7f398d1677c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-28T13:57:43.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:43.391+0000 7f398d1677c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-28T13:57:43.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:43.391+0000 7f398d1677c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-28T13:57:46.425 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-28T13:57:46.447 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-28T13:57:46.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:46.555+0000 7f16c6cba7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-28T13:57:46.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:46.559+0000 7f16c6cba7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-28T13:57:46.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:46.559+0000 7f16c6cba7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-28T13:57:46.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:46.559+0000 7f16c6cba7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-28T13:57:46.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:46.559+0000 7f16c6cba7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-28T13:57:46.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:46.559+0000 7f16c6cba7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-28T13:57:47.181 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T13:57:47.185 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T13:57:49.598 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-28T13:57:49.620 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-28T13:57:49.620 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:49.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-28T13:57:49.644 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:49.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-28T13:57:49.710 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:49.710 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-28T13:57:49.778 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:49.778 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-28T13:57:49.842 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:49.842 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-28T13:57:49.909 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:49.909 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-28T13:57:49.974 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:49.974 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-28T13:57:50.037 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:50.038 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-28T13:57:50.060 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:50.060 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-28T13:57:50.128 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:50.128 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-28T13:57:50.196 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:50.196 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-28T13:57:50.264 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:50.264 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-28T13:57:50.332 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:50.333 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-28T13:57:50.400 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:50.400 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-28T13:57:50.468 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:50.468 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-28T13:57:50.477 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:50.478 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-28T13:57:50.529 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:50.529 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-28T13:57:50.538 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:50.538 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-28T13:57:50.587 INFO:tasks.ceph:Adding keys to all mons... 2024-05-28T13:57:50.588 DEBUG:teuthology.orchestra.run.smithi042:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-28T13:57:50.591 DEBUG:teuthology.orchestra.run.smithi063:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-28T13:57:50.635 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.y] 2024-05-28T13:57:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQC/4lVmxTVyJRAATMLAAqKne20YvV537uXZ6g== 2024-05-28T13:57:50.635 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.a] 2024-05-28T13:57:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQC/4lVmE+3RLRAADuTKGfKeVafGeLpgr9WKUg== 2024-05-28T13:57:50.635 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.c] 2024-05-28T13:57:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQC/4lVme4aYNRAAVxJTlugnxtFTexUclF/IwA== 2024-05-28T13:57:50.635 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.0] 2024-05-28T13:57:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDB4lVmGMDBJhAA3jIyR7XKLGXEgimIGNsw6A== 2024-05-28T13:57:50.635 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.1] 2024-05-28T13:57:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDE4lVmxMisLxAAcIz7GPlkZwvW5ig+DbAVGg== 2024-05-28T13:57:50.636 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.2] 2024-05-28T13:57:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDH4lVmSSk4OhAAmLlLysM4Im/Kew4Xbq2/OA== 2024-05-28T13:57:50.636 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.3] 2024-05-28T13:57:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDL4lVmvzrYBhAAiAAOQXZBuTsMVYa1vE5rkg== 2024-05-28T13:57:50.636 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.x] 2024-05-28T13:57:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQC/4lVmNO5tKRAA70hPLwoE83keovlq2rolxQ== 2024-05-28T13:57:50.636 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.b] 2024-05-28T13:57:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQC/4lVm1rmVOhAAV46KzDJnRfDZYIZanS0g2g== 2024-05-28T13:57:50.636 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.d] 2024-05-28T13:57:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDA4lVmeAdBBhAAr4IVFQpjQXFVlnFTHnPV9w== 2024-05-28T13:57:50.636 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.4] 2024-05-28T13:57:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDQ4lVmrqnTOhAA2pIfgBT9LJDIhp+GgFy8Gg== 2024-05-28T13:57:50.636 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.5] 2024-05-28T13:57:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDU4lVmYpsQCxAAoTK2aMBPrHKvsUnNOTj/Dw== 2024-05-28T13:57:50.637 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.6] 2024-05-28T13:57:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDX4lVm6TCAFxAAs3rkDyKcLhWrn/EkIo0bVA== 2024-05-28T13:57:50.637 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.7] 2024-05-28T13:57:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDa4lVmr5NwIRAA7Lb70VYV82dr0Frn8CCUXg== 2024-05-28T13:57:50.637 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-05-28T13:57:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw== 2024-05-28T13:57:50.637 INFO:teuthology.orchestra.run.smithi042.stdout:[client.2] 2024-05-28T13:57:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDA4lVmoxx+EBAA8mLNak5AwRmWQIgkYtMOrg== 2024-05-28T13:57:50.637 INFO:teuthology.orchestra.run.smithi042.stdout:[client.1] 2024-05-28T13:57:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDA4lVmcfkLFBAADNyODKTtYAlFWquZ/LeeqQ== 2024-05-28T13:57:50.637 INFO:teuthology.orchestra.run.smithi042.stdout:[client.3] 2024-05-28T13:57:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDA4lVmxSUsGhAAbgr1eOh/sMfrEONKKHFo0g== 2024-05-28T13:57:50.650 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr.y] 2024-05-28T13:57:50.650 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQC/4lVmxTVyJRAATMLAAqKne20YvV537uXZ6g== 2024-05-28T13:57:50.650 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a] 2024-05-28T13:57:50.650 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQC/4lVmE+3RLRAADuTKGfKeVafGeLpgr9WKUg== 2024-05-28T13:57:50.650 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c] 2024-05-28T13:57:50.650 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQC/4lVme4aYNRAAVxJTlugnxtFTexUclF/IwA== 2024-05-28T13:57:50.650 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.0] 2024-05-28T13:57:50.651 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDB4lVmGMDBJhAA3jIyR7XKLGXEgimIGNsw6A== 2024-05-28T13:57:50.651 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.1] 2024-05-28T13:57:50.651 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDE4lVmxMisLxAAcIz7GPlkZwvW5ig+DbAVGg== 2024-05-28T13:57:50.651 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.2] 2024-05-28T13:57:50.651 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDH4lVmSSk4OhAAmLlLysM4Im/Kew4Xbq2/OA== 2024-05-28T13:57:50.651 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.3] 2024-05-28T13:57:50.651 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDL4lVmvzrYBhAAiAAOQXZBuTsMVYa1vE5rkg== 2024-05-28T13:57:50.651 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr.x] 2024-05-28T13:57:50.651 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQC/4lVmNO5tKRAA70hPLwoE83keovlq2rolxQ== 2024-05-28T13:57:50.651 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b] 2024-05-28T13:57:50.651 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQC/4lVm1rmVOhAAV46KzDJnRfDZYIZanS0g2g== 2024-05-28T13:57:50.651 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d] 2024-05-28T13:57:50.651 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDA4lVmeAdBBhAAr4IVFQpjQXFVlnFTHnPV9w== 2024-05-28T13:57:50.651 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.4] 2024-05-28T13:57:50.652 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDQ4lVmrqnTOhAA2pIfgBT9LJDIhp+GgFy8Gg== 2024-05-28T13:57:50.652 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.5] 2024-05-28T13:57:50.652 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDU4lVmYpsQCxAAoTK2aMBPrHKvsUnNOTj/Dw== 2024-05-28T13:57:50.652 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.6] 2024-05-28T13:57:50.652 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDX4lVm6TCAFxAAs3rkDyKcLhWrn/EkIo0bVA== 2024-05-28T13:57:50.652 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.7] 2024-05-28T13:57:50.652 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDa4lVmr5NwIRAA7Lb70VYV82dr0Frn8CCUXg== 2024-05-28T13:57:50.652 INFO:teuthology.orchestra.run.smithi063.stdout:[client.0] 2024-05-28T13:57:50.652 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw== 2024-05-28T13:57:50.652 INFO:teuthology.orchestra.run.smithi063.stdout:[client.2] 2024-05-28T13:57:50.652 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDA4lVmoxx+EBAA8mLNak5AwRmWQIgkYtMOrg== 2024-05-28T13:57:50.652 INFO:teuthology.orchestra.run.smithi063.stdout:[client.1] 2024-05-28T13:57:50.653 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDA4lVmcfkLFBAADNyODKTtYAlFWquZ/LeeqQ== 2024-05-28T13:57:50.653 INFO:teuthology.orchestra.run.smithi063.stdout:[client.3] 2024-05-28T13:57:50.653 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDA4lVmxSUsGhAAbgr1eOh/sMfrEONKKHFo0g== 2024-05-28T13:57:50.654 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-28T13:57:50.687 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-28T13:57:50.759 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-28T13:57:50.762 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-28T13:57:50.822 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-28T13:57:50.826 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-28T13:57:50.884 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:50.888 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:50.943 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:50.946 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:51.000 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:51.003 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:51.058 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:51.062 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:51.116 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-28T13:57:51.119 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-28T13:57:51.174 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-28T13:57:51.178 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-28T13:57:51.236 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-28T13:57:51.239 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-28T13:57:51.297 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:51.300 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:51.355 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:51.358 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:51.414 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:51.418 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:51.473 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:51.476 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-28T13:57:51.531 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-28T13:57:51.535 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-28T13:57:51.593 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-28T13:57:51.596 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-28T13:57:51.655 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-28T13:57:51.658 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-28T13:57:51.715 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-28T13:57:51.718 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-28T13:57:51.775 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-28T13:57:51.775 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-28T13:57:51.795 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-28T13:57:51.904 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-28T13:57:51.923 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-28T13:57:51.944 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-28T13:57:52.060 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-28T13:57:52.080 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-28T13:57:52.146 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-28T13:57:52.258 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-28T13:57:52.278 DEBUG:teuthology.orchestra.run.smithi042:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-28T13:57:52.281 DEBUG:teuthology.orchestra.run.smithi063:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-28T13:57:52.327 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-28T13:57:52.327 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-28T13:57:52.327 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-28T13:57:52.334 INFO:tasks.ceph.mon.a:Started 2024-05-28T13:57:52.335 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-28T13:57:52.335 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-28T13:57:52.370 INFO:tasks.ceph.mon.b:Started 2024-05-28T13:57:52.370 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-28T13:57:52.370 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-28T13:57:52.373 INFO:tasks.ceph.mon.c:Started 2024-05-28T13:57:52.373 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-28T13:57:52.373 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-28T13:57:52.373 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-28T13:57:52.377 INFO:tasks.ceph.mgr.y:Started 2024-05-28T13:57:52.377 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-28T13:57:52.377 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-28T13:57:52.380 INFO:tasks.ceph.mgr.x:Started 2024-05-28T13:57:52.380 INFO:tasks.ceph:Setting crush tunables to default 2024-05-28T13:57:52.380 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd crush tunables default 2024-05-28T13:57:52.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f6753000640 1 Processor -- start 2024-05-28T13:57:52.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f6753000640 1 -- start start 2024-05-28T13:57:52.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f6753000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 0x7f674c0d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:52.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f6753000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 0x7f674c0b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:52.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f6753000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f674c0d1a80 con 0x7f674c0b7ba0 2024-05-28T13:57:52.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f6753000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f674c0cd500 con 0x7f674c15ec50 2024-05-28T13:57:52.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f6753000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f674c0ce9a0 con 0x7f674c15e870 2024-05-28T13:57:52.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f6743400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 0x7f674c0b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:52.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f6751000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f674c0b7ba0 0x7f674c06cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60448/0 (socket says 172.21.15.42:60448) 2024-05-28T13:57:52.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f674be00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 0x7f674c0d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:52.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f6751000640 1 -- 172.21.15.42:0/1518113643 learned_addr learned my addr 172.21.15.42:0/1518113643 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:57:52.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f674be00640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 msgr2=0x7f674c0d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-28T13:57:52.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f674be00640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 msgr2=0x7f674c0d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:52.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f674be00640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 0x7f674c0d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:52.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 msgr2=0x7f674c0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-28T13:57:52.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 msgr2=0x7f674c0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:52.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 0x7f674c0b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:52.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f674be00640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 0x7f674c0d1230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:52.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.534+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 0x7f674c0b7660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:52.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.738+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 0x7f674c0b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:52.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.738+0000 7f674be00640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 0x7f674c0d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:52.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.738+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 msgr2=0x7f674c0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-28T13:57:52.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.738+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 msgr2=0x7f674c0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:52.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.738+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 0x7f674c0b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:52.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.738+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 0x7f674c0b7660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-28T13:57:52.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.738+0000 7f674be00640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 msgr2=0x7f674c0d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-28T13:57:52.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.738+0000 7f674be00640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 msgr2=0x7f674c0d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:52.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.738+0000 7f674be00640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 0x7f674c0d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:52.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:52.738+0000 7f674be00640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 0x7f674c0d1230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-28T13:57:53.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.142+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 0x7f674c0b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.142+0000 7f674be00640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 0x7f674c0d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.142+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 msgr2=0x7f674c0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-28T13:57:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.142+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 msgr2=0x7f674c0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.142+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 0x7f674c0b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.142+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 0x7f674c0b7660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-28T13:57:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.142+0000 7f674be00640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 msgr2=0x7f674c0d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-28T13:57:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.142+0000 7f674be00640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 msgr2=0x7f674c0d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.142+0000 7f674be00640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 0x7f674c0d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:53.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.142+0000 7f674be00640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 0x7f674c0d1230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-28T13:57:53.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.946+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 0x7f674c0b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:53.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.946+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 msgr2=0x7f674c0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-28T13:57:53.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.946+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 msgr2=0x7f674c0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:53.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.946+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 0x7f674c0b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:53.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.946+0000 7f674be00640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 0x7f674c0d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:53.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.946+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 0x7f674c0b7660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-28T13:57:53.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.946+0000 7f674be00640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 msgr2=0x7f674c0d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-28T13:57:53.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.946+0000 7f674be00640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 msgr2=0x7f674c0d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:53.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.946+0000 7f674be00640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 0x7f674c0d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:53.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:53.946+0000 7f674be00640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 0x7f674c0d1230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-28T13:57:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.538+0000 7f674a000640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 msgr2=0x7f674c0d1230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:57:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.538+0000 7f674a000640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.63:3300/0 conn(0x7f674c15e870 0x7f674c0d1230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.538+0000 7f674a000640 1 -- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 msgr2=0x7f674c0b7660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:57:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.538+0000 7f674a000640 1 --2- 172.21.15.42:0/1518113643 >> v2:172.21.15.42:3300/0 conn(0x7f674c15ec50 0x7f674c0b7660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.538+0000 7f674a000640 1 -- 172.21.15.42:0/1518113643 >> v1:172.21.15.42:6789/0 conn(0x7f674c0b7ba0 legacy=0x7f674c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.538+0000 7f674a000640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 0x7f6728009820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.538+0000 7f674a000640 1 -- 172.21.15.42:0/1518113643 --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f674c0ccbd0 con 0x7f67280036c0 2024-05-28T13:57:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.538+0000 7f674a000640 1 -- 172.21.15.42:0/1518113643 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f674c0cd500 con 0x7f6728007380 2024-05-28T13:57:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.538+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 0x7f6728009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.538+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 msgr2=0x7f6728009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-28T13:57:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.538+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 msgr2=0x7f6728009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.538+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 0x7f6728009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.538+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 0x7f6728009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:55.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.742+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 0x7f6728009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:55.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.742+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 msgr2=0x7f6728009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-28T13:57:55.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.742+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 msgr2=0x7f6728009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:55.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.742+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 0x7f6728009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:55.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:55.742+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 0x7f6728009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-28T13:57:56.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:56.146+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 0x7f6728009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:56.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:56.146+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 msgr2=0x7f6728009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-28T13:57:56.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:56.146+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 msgr2=0x7f6728009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:56.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:56.146+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 0x7f6728009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:56.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:56.146+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 0x7f6728009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-28T13:57:56.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:56.950+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 0x7f6728009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:56.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:56.950+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 msgr2=0x7f6728009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-28T13:57:56.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:56.950+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 msgr2=0x7f6728009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:56.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:56.950+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 0x7f6728009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:56.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:56.950+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 0x7f6728009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-28T13:57:57.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.682+0000 7f674b400640 1 -- 172.21.15.42:0/1518113643 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 553091374 0 0) 0x7f674c0ccbd0 con 0x7f67280036c0 2024-05-28T13:57:57.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.682+0000 7f674b400640 1 -- 172.21.15.42:0/1518113643 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6730003530 con 0x7f67280036c0 2024-05-28T13:57:57.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674b400640 1 -- 172.21.15.42:0/1518113643 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2993085519 0 0) 0x7f6730003530 con 0x7f67280036c0 2024-05-28T13:57:57.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674b400640 1 -- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 msgr2=0x7f6728009820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:57:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674b400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6728007380 0x7f6728009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674b400640 1 -- 172.21.15.42:0/1518113643 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672800afe0 con 0x7f67280036c0 2024-05-28T13:57:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674b400640 1 -- 172.21.15.42:0/1518113643 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f673c00f040 con 0x7f67280036c0 2024-05-28T13:57:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674b400640 1 -- 172.21.15.42:0/1518113643 >> v1:172.21.15.63:6789/0 conn(0x7f67280036c0 legacy=0x7f6728005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f6751000640 1 -- 172.21.15.42:0/1518113643 reap_dead start 2024-05-28T13:57:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674b400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 0x7f6730003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674b400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730004810 0x7f6730004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674b400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67300052f0 0x7f673000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674b400640 1 -- 172.21.15.42:0/1518113643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f674c0cd500 con 0x7f6730004810 2024-05-28T13:57:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674b400640 1 -- 172.21.15.42:0/1518113643 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f673000c240 con 0x7f67300052f0 2024-05-28T13:57:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674b400640 1 -- 172.21.15.42:0/1518113643 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f673000c440 con 0x7f67300039c0 2024-05-28T13:57:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730004810 0x7f6730004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674be00640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 0x7f6730003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:57.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f6751000640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67300052f0 0x7f673000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:57.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674be00640 1 -- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 msgr2=0x7f6730003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-28T13:57:57.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674be00640 1 -- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 msgr2=0x7f6730003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:57.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674be00640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 0x7f6730003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:57.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f674be00640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 0x7f6730003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:57.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 msgr2=0x7f6730003dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:57:57.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 0x7f6730003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:57.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67300052f0 msgr2=0x7f673000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:57:57.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67300052f0 0x7f673000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:57.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.690+0000 7f6743400640 1 -- 172.21.15.42:0/1518113643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6730003530 con 0x7f6730004810 2024-05-28T13:57:57.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.694+0000 7f6743400640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730004810 0x7f6730004c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6734000ee0 tx=0x7f673401dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8469789442dcb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:57.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.694+0000 7f674b400640 1 -- 172.21.15.42:0/1518113643 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673400f040 con 0x7f6730004810 2024-05-28T13:57:57.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.694+0000 7f674b400640 1 -- 172.21.15.42:0/1518113643 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67340a6d00 con 0x7f6730004810 2024-05-28T13:57:57.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.694+0000 7f674b400640 1 -- 172.21.15.42:0/1518113643 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734004260 con 0x7f6730004810 2024-05-28T13:57:57.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.694+0000 7f6753000640 1 -- 172.21.15.42:0/1518113643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730004810 msgr2=0x7f6730004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:57.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.694+0000 7f6753000640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730004810 0x7f6730004c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6734000ee0 tx=0x7f673401dd50 comp rx=0 tx=0).stop 2024-05-28T13:57:57.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.694+0000 7f6753000640 1 -- 172.21.15.42:0/1518113643 shutdown_connections 2024-05-28T13:57:57.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.694+0000 7f6753000640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67300052f0 0x7f673000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:57.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.694+0000 7f6753000640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730004810 0x7f6730004c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:57.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.694+0000 7f6753000640 1 --2- 172.21.15.42:0/1518113643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 0x7f6730003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:57.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.694+0000 7f6753000640 1 -- 172.21.15.42:0/1518113643 >> 172.21.15.42:0/1518113643 conn(0x7f674c0b3fd0 msgr2=0x7f674c0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:57.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.694+0000 7f6753000640 1 -- 172.21.15.42:0/1518113643 shutdown_connections 2024-05-28T13:57:57.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.694+0000 7f6753000640 1 -- 172.21.15.42:0/1518113643 wait complete. 2024-05-28T13:57:57.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.698+0000 7f6753000640 1 Processor -- start 2024-05-28T13:57:57.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.698+0000 7f6753000640 1 -- start start 2024-05-28T13:57:57.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.698+0000 7f6753000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 0x7f674c15a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:57.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.698+0000 7f6753000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730004810 0x7f674c1e8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:57.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.698+0000 7f6753000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67300052f0 0x7f674c1e87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:57.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.698+0000 7f6753000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f674c0b5330 con 0x7f6730004810 2024-05-28T13:57:57.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.698+0000 7f6753000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f674c0cd190 con 0x7f67300052f0 2024-05-28T13:57:57.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.698+0000 7f6753000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f674c160660 con 0x7f67300039c0 2024-05-28T13:57:57.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.698+0000 7f6743400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730004810 0x7f674c1e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:57.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.698+0000 7f674be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 0x7f674c15a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:57.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6743400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730004810 0x7f674c1e8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39984/0 (socket says 172.21.15.42:39984) 2024-05-28T13:57:57.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6743400640 1 -- 172.21.15.42:0/1748177790 learned_addr learned my addr 172.21.15.42:0/1748177790 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:57:57.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f674be00640 1 -- 172.21.15.42:0/1748177790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 msgr2=0x7f674c15a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-28T13:57:57.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f674be00640 1 -- 172.21.15.42:0/1748177790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 msgr2=0x7f674c15a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-28T13:57:57.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f674be00640 1 --2- 172.21.15.42:0/1748177790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 0x7f674c15a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:57.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f674be00640 1 --2- 172.21.15.42:0/1748177790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 0x7f674c15a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6743400640 1 -- 172.21.15.42:0/1748177790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 msgr2=0x7f674c15a7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-28T13:57:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6743400640 1 --2- 172.21.15.42:0/1748177790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 0x7f674c15a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6743400640 1 -- 172.21.15.42:0/1748177790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67300052f0 msgr2=0x7f674c1e87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6743400640 1 --2- 172.21.15.42:0/1748177790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67300052f0 0x7f674c1e87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6743400640 1 -- 172.21.15.42:0/1748177790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c1f3f30 con 0x7f6730004810 2024-05-28T13:57:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6743400640 1 --2- 172.21.15.42:0/1748177790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730004810 0x7f674c1e8290 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f673401db80 tx=0x7f6730005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6748c00640 1 -- 172.21.15.42:0/1748177790 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67340be020 con 0x7f6730004810 2024-05-28T13:57:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6748c00640 1 -- 172.21.15.42:0/1748177790 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6734003830 con 0x7f6730004810 2024-05-28T13:57:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6753000640 1 -- 172.21.15.42:0/1748177790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f674c15b9e0 con 0x7f6730004810 2024-05-28T13:57:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6748c00640 1 -- 172.21.15.42:0/1748177790 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673400f040 con 0x7f6730004810 2024-05-28T13:57:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6753000640 1 -- 172.21.15.42:0/1748177790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f674c15bfa0 con 0x7f6730004810 2024-05-28T13:57:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6748c00640 1 -- 172.21.15.42:0/1748177790 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f67340c1020 con 0x7f6730004810 2024-05-28T13:57:57.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6748c00640 1 -- 172.21.15.42:0/1748177790 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f67340ae440 con 0x7f6730004810 2024-05-28T13:57:57.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.702+0000 7f6753000640 1 -- 172.21.15.42:0/1748177790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f674c04fdd0 con 0x7f6730004810 2024-05-28T13:57:57.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.706+0000 7f6748c00640 1 -- 172.21.15.42:0/1748177790 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f67340c8030 con 0x7f6730004810 2024-05-28T13:57:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.770+0000 7f6753000640 1 -- 172.21.15.42:0/1748177790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f674c0719d0 con 0x7f6730004810 2024-05-28T13:57:57.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.774+0000 7f6748c00640 1 -- 172.21.15.42:0/1748177790 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f6734008890 con 0x7f6730004810 2024-05-28T13:57:57.761 INFO:teuthology.orchestra.run.smithi042.stderr:adjusted tunables profile to default 2024-05-28T13:57:57.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.774+0000 7f6753000640 1 -- 172.21.15.42:0/1748177790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730004810 msgr2=0x7f674c1e8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:57.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.774+0000 7f6753000640 1 --2- 172.21.15.42:0/1748177790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730004810 0x7f674c1e8290 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f673401db80 tx=0x7f6730005840 comp rx=0 tx=0).stop 2024-05-28T13:57:57.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.774+0000 7f6753000640 1 -- 172.21.15.42:0/1748177790 shutdown_connections 2024-05-28T13:57:57.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.774+0000 7f6753000640 1 --2- 172.21.15.42:0/1748177790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67300052f0 0x7f674c1e87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:57.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.774+0000 7f6753000640 1 --2- 172.21.15.42:0/1748177790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730004810 0x7f674c1e8290 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:57.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.774+0000 7f6753000640 1 --2- 172.21.15.42:0/1748177790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67300039c0 0x7f674c15a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:57.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.774+0000 7f6753000640 1 -- 172.21.15.42:0/1748177790 >> 172.21.15.42:0/1748177790 conn(0x7f674c0b3fd0 msgr2=0x7f674c15dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:57.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.774+0000 7f6753000640 1 -- 172.21.15.42:0/1748177790 shutdown_connections 2024-05-28T13:57:57.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:57:57.774+0000 7f6753000640 1 -- 172.21.15.42:0/1748177790 wait complete. 2024-05-28T13:57:57.783 INFO:tasks.ceph:check_enable_crimson: False 2024-05-28T13:57:57.783 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-28T13:57:57.784 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:57.784 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-28T13:57:57.804 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:57.804 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-28T13:57:57.870 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:57.870 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-28T13:57:57.875 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:57:57.886+0000 7f07f8d0b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-28T13:57:57.894 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:57:57.894 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-28T13:57:57.956 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:57.957 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-28T13:57:57.979 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:57.979 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-28T13:57:58.048 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:58.048 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-28T13:57:58.116 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:57:58.116 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-28T13:57:58.184 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 7837a445-7a48-4c2e-afb3-bf233faa91e7 0 2024-05-28T13:57:58.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f065d400640 1 Processor -- start 2024-05-28T13:57:58.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f065d400640 1 -- start start 2024-05-28T13:57:58.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f065d400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06581602c0 0x7f06580cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f065d400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f06580d0ac0 0x7f065809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f065d400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06580cd170 con 0x7f0658160e60 2024-05-28T13:57:58.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f065d400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f06580ce400 con 0x7f06580d0ac0 2024-05-28T13:57:58.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f065d400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f06580cd3e0 con 0x7f06581602c0 2024-05-28T13:57:58.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f0656800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06581602c0 0x7f06580cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f064de00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f0658160e60 0x7f06581606a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.63:38368/0 (socket says 172.21.15.63:38368) 2024-05-28T13:57:58.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f064de00640 1 -- 172.21.15.63:0/1471719782 learned_addr learned my addr 172.21.15.63:0/1471719782 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-05-28T13:57:58.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f0656800640 1 -- 172.21.15.63:0/1471719782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06581602c0 msgr2=0x7f06580cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-28T13:57:58.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f0657200640 1 --2- 172.21.15.63:0/1471719782 >> v2:172.21.15.42:3300/0 conn(0x7f06580d0ac0 0x7f065809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f0656800640 1 -- 172.21.15.63:0/1471719782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06581602c0 msgr2=0x7f06580cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:58.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f0656800640 1 --2- 172.21.15.63:0/1471719782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06581602c0 0x7f06580cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:58.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f0656800640 1 --2- 172.21.15.63:0/1471719782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06581602c0 0x7f06580cdec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:58.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f0655e00640 1 -- 172.21.15.63:0/1471719782 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2189125024 0 0) 0x7f06580cd170 con 0x7f0658160e60 2024-05-28T13:57:58.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f0655e00640 1 -- 172.21.15.63:0/1471719782 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f063c003530 con 0x7f0658160e60 2024-05-28T13:57:58.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f0657200640 1 -- 172.21.15.63:0/1471719782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06581602c0 msgr2=0x7f06580cdec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:57:58.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f0657200640 1 --2- 172.21.15.63:0/1471719782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06581602c0 0x7f06580cdec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f0657200640 1 -- 172.21.15.63:0/1471719782 >> v1:172.21.15.63:6789/0 conn(0x7f0658160e60 legacy=0x7f06581606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:58.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.395+0000 7f0657200640 1 -- 172.21.15.63:0/1471719782 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06580cd6c0 con 0x7f06580d0ac0 2024-05-28T13:57:58.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.399+0000 7f0657200640 1 --2- 172.21.15.63:0/1471719782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06580d0ac0 0x7f065809aa50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f06500097c0 tx=0x7f065001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8005f37e0d8feb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:58.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.399+0000 7f0655e00640 1 -- 172.21.15.63:0/1471719782 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06500ab610 con 0x7f06580d0ac0 2024-05-28T13:57:58.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.399+0000 7f0655e00640 1 -- 172.21.15.63:0/1471719782 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06500b0070 con 0x7f06580d0ac0 2024-05-28T13:57:58.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.399+0000 7f0655e00640 1 -- 172.21.15.63:0/1471719782 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06500ab9d0 con 0x7f06580d0ac0 2024-05-28T13:57:58.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.399+0000 7f065d400640 1 -- 172.21.15.63:0/1471719782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06580d0ac0 msgr2=0x7f065809aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:58.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.399+0000 7f065d400640 1 --2- 172.21.15.63:0/1471719782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06580d0ac0 0x7f065809aa50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f06500097c0 tx=0x7f065001c470 comp rx=0 tx=0).stop 2024-05-28T13:57:58.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.399+0000 7f065d400640 1 -- 172.21.15.63:0/1471719782 shutdown_connections 2024-05-28T13:57:58.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.399+0000 7f065d400640 1 --2- 172.21.15.63:0/1471719782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06580d0ac0 0x7f065809aa50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.399+0000 7f065d400640 1 --2- 172.21.15.63:0/1471719782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06581602c0 0x7f06580cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.399+0000 7f065d400640 1 -- 172.21.15.63:0/1471719782 >> 172.21.15.63:0/1471719782 conn(0x7f06580b3fd0 msgr2=0x7f06580b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:58.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.399+0000 7f065d400640 1 -- 172.21.15.63:0/1471719782 shutdown_connections 2024-05-28T13:57:58.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.399+0000 7f065d400640 1 -- 172.21.15.63:0/1471719782 wait complete. 2024-05-28T13:57:58.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f065d400640 1 Processor -- start 2024-05-28T13:57:58.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f065d400640 1 -- start start 2024-05-28T13:57:58.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f065d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06580d0ac0 0x7f065814e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f065d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06581602c0 0x7f0658157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f065d400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0658160e60 0x7f0658157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f065d400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f065815dba0 con 0x7f06581602c0 2024-05-28T13:57:58.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f065d400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f065815da20 con 0x7f06580d0ac0 2024-05-28T13:57:58.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f065d400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0658169640 con 0x7f0658160e60 2024-05-28T13:57:58.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f0656800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06580d0ac0 0x7f065814e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f0657200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0658160e60 0x7f0658157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f0656800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06580d0ac0 0x7f065814e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42736/0 (socket says 172.21.15.63:42736) 2024-05-28T13:57:58.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f0656800640 1 -- 172.21.15.63:0/3227369085 learned_addr learned my addr 172.21.15.63:0/3227369085 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-28T13:57:58.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f064de00640 1 --2- 172.21.15.63:0/3227369085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06581602c0 0x7f0658157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f0657200640 1 -- 172.21.15.63:0/3227369085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0658160e60 msgr2=0x7f0658157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-28T13:57:58.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f0657200640 1 -- 172.21.15.63:0/3227369085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0658160e60 msgr2=0x7f0658157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-28T13:57:58.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f0657200640 1 --2- 172.21.15.63:0/3227369085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0658160e60 0x7f0658157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:58.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f0657200640 1 --2- 172.21.15.63:0/3227369085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0658160e60 0x7f0658157660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:58.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f0656800640 1 -- 172.21.15.63:0/3227369085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0658160e60 msgr2=0x7f0658157660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-28T13:57:58.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f0656800640 1 --2- 172.21.15.63:0/3227369085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0658160e60 0x7f0658157660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f0656800640 1 -- 172.21.15.63:0/3227369085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06581602c0 msgr2=0x7f0658157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:58.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f0656800640 1 --2- 172.21.15.63:0/3227369085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06581602c0 0x7f0658157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f0656800640 1 -- 172.21.15.63:0/3227369085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06580b0000 con 0x7f06580d0ac0 2024-05-28T13:57:58.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f0656800640 1 --2- 172.21.15.63:0/3227369085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06580d0ac0 0x7f065814e120 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f064800e7e0 tx=0x7f064800eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:58.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f064f400640 1 -- 172.21.15.63:0/3227369085 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648017020 con 0x7f06580d0ac0 2024-05-28T13:57:58.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f064f400640 1 -- 172.21.15.63:0/3227369085 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f064800c6d0 con 0x7f06580d0ac0 2024-05-28T13:57:58.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f065d400640 1 -- 172.21.15.63:0/3227369085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06580b02e0 con 0x7f06580d0ac0 2024-05-28T13:57:58.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f064f400640 1 -- 172.21.15.63:0/3227369085 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064800f040 con 0x7f06580d0ac0 2024-05-28T13:57:58.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f065d400640 1 -- 172.21.15.63:0/3227369085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06580b08a0 con 0x7f06580d0ac0 2024-05-28T13:57:58.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f064f400640 1 -- 172.21.15.63:0/3227369085 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f064801a020 con 0x7f06580d0ac0 2024-05-28T13:57:58.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f064f400640 1 -- 172.21.15.63:0/3227369085 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f064801c570 con 0x7f06580d0ac0 2024-05-28T13:57:58.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.403+0000 7f065d400640 1 -- 172.21.15.63:0/3227369085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06580a0570 con 0x7f06580d0ac0 2024-05-28T13:57:58.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.407+0000 7f064f400640 1 -- 172.21.15.63:0/3227369085 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0648021020 con 0x7f06580d0ac0 2024-05-28T13:57:58.398 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:57:58.410+0000 7f07f8d0b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-28T13:57:58.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.475+0000 7f065d400640 1 -- 172.21.15.63:0/3227369085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7837a445-7a48-4c2e-afb3-bf233faa91e7", "id": 0} v 0) -- 0x7f06580b3d40 con 0x7f06580d0ac0 2024-05-28T13:57:58.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.483+0000 7f064f400640 1 -- 172.21.15.63:0/3227369085 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7837a445-7a48-4c2e-afb3-bf233faa91e7", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f064801c770 con 0x7f06580d0ac0 2024-05-28T13:57:58.469 INFO:teuthology.orchestra.run.smithi063.stdout:0 2024-05-28T13:57:58.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.483+0000 7f065d400640 1 -- 172.21.15.63:0/3227369085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06580d0ac0 msgr2=0x7f065814e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:58.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.483+0000 7f065d400640 1 --2- 172.21.15.63:0/3227369085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06580d0ac0 0x7f065814e120 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f064800e7e0 tx=0x7f064800eca0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.483+0000 7f065d400640 1 -- 172.21.15.63:0/3227369085 shutdown_connections 2024-05-28T13:57:58.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.483+0000 7f065d400640 1 --2- 172.21.15.63:0/3227369085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0658160e60 0x7f0658157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.483+0000 7f065d400640 1 --2- 172.21.15.63:0/3227369085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06581602c0 0x7f0658157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.483+0000 7f065d400640 1 --2- 172.21.15.63:0/3227369085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06580d0ac0 0x7f065814e120 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.483+0000 7f065d400640 1 -- 172.21.15.63:0/3227369085 >> 172.21.15.63:0/3227369085 conn(0x7f06580b3fd0 msgr2=0x7f0658070a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:58.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.483+0000 7f065d400640 1 -- 172.21.15.63:0/3227369085 shutdown_connections 2024-05-28T13:57:58.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.483+0000 7f065d400640 1 -- 172.21.15.63:0/3227369085 wait complete. 2024-05-28T13:57:58.487 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 1b3b7367-edb9-4b74-80e1-c244b4eaf23a 1 2024-05-28T13:57:58.628 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:57:58.638+0000 7f07f8d0b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-28T13:57:58.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.643+0000 7fc22e600640 1 Processor -- start 2024-05-28T13:57:58.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.643+0000 7fc22e600640 1 -- start start 2024-05-28T13:57:58.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc22e600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc22815e870 0x7fc2280d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc22e600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc22815ec50 0x7fc2280b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc22e600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2280d1a80 con 0x7fc2280b7ba0 2024-05-28T13:57:58.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc22e600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc2280cd500 con 0x7fc22815e870 2024-05-28T13:57:58.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc22e600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc2280ce9a0 con 0x7fc22815ec50 2024-05-28T13:57:58.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc21ea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc22815ec50 0x7fc2280b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc227e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fc2280b7ba0 0x7fc22806cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.63:38396/0 (socket says 172.21.15.63:38396) 2024-05-28T13:57:58.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc227e00640 1 -- 172.21.15.63:0/1001304639 learned_addr learned my addr 172.21.15.63:0/1001304639 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-05-28T13:57:58.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc227400640 1 --2- 172.21.15.63:0/1001304639 >> v2:172.21.15.42:3300/0 conn(0x7fc22815e870 0x7fc2280d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc21ea00640 1 -- 172.21.15.63:0/1001304639 >> v2:172.21.15.42:3300/0 conn(0x7fc22815e870 msgr2=0x7fc2280d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:57:58.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc21ea00640 1 --2- 172.21.15.63:0/1001304639 >> v2:172.21.15.42:3300/0 conn(0x7fc22815e870 0x7fc2280d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc21ea00640 1 -- 172.21.15.63:0/1001304639 >> v1:172.21.15.63:6789/0 conn(0x7fc2280b7ba0 legacy=0x7fc22806cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:58.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc21ea00640 1 -- 172.21.15.63:0/1001304639 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2280ccbd0 con 0x7fc22815ec50 2024-05-28T13:57:58.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc226a00640 1 -- 172.21.15.63:0/1001304639 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 467205620 0 0) 0x7fc2280d1a80 con 0x7fc2280b7ba0 2024-05-28T13:57:58.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.647+0000 7fc21ea00640 1 --2- 172.21.15.63:0/1001304639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22815ec50 0x7fc2280b7660 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc228162500 tx=0x7fc21001ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36c0280e280bbb27 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:58.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc226a00640 1 -- 172.21.15.63:0/1001304639 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2100b5070 con 0x7fc22815ec50 2024-05-28T13:57:58.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc226a00640 1 -- 172.21.15.63:0/1001304639 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2100a8970 con 0x7fc22815ec50 2024-05-28T13:57:58.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc226a00640 1 -- 172.21.15.63:0/1001304639 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2100a8d40 con 0x7fc22815ec50 2024-05-28T13:57:58.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 -- 172.21.15.63:0/1001304639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22815ec50 msgr2=0x7fc2280b7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:58.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 --2- 172.21.15.63:0/1001304639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22815ec50 0x7fc2280b7660 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc228162500 tx=0x7fc21001ec70 comp rx=0 tx=0).stop 2024-05-28T13:57:58.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 -- 172.21.15.63:0/1001304639 shutdown_connections 2024-05-28T13:57:58.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 --2- 172.21.15.63:0/1001304639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22815ec50 0x7fc2280b7660 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 --2- 172.21.15.63:0/1001304639 >> v2:172.21.15.42:3300/0 conn(0x7fc22815e870 0x7fc2280d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 -- 172.21.15.63:0/1001304639 >> 172.21.15.63:0/1001304639 conn(0x7fc2280b3fd0 msgr2=0x7fc2280b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:58.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 -- 172.21.15.63:0/1001304639 shutdown_connections 2024-05-28T13:57:58.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 -- 172.21.15.63:0/1001304639 wait complete. 2024-05-28T13:57:58.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 Processor -- start 2024-05-28T13:57:58.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 -- start start 2024-05-28T13:57:58.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2280b7ba0 0x7fc2281f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22815e870 0x7fc2280cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc22815ec50 0x7fc2280c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc2280d1770 con 0x7fc22815ec50 2024-05-28T13:57:58.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2280cd190 con 0x7fc22815e870 2024-05-28T13:57:58.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.651+0000 7fc22e600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc2280b5330 con 0x7fc2280b7ba0 2024-05-28T13:57:58.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc21ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22815e870 0x7fc2280cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc21ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22815e870 0x7fc2280cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42762/0 (socket says 172.21.15.63:42762) 2024-05-28T13:57:58.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc21ea00640 1 -- 172.21.15.63:0/1168461423 learned_addr learned my addr 172.21.15.63:0/1168461423 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-28T13:57:58.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc227400640 1 --2- 172.21.15.63:0/1168461423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2280b7ba0 0x7fc2281f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc227e00640 1 --2- 172.21.15.63:0/1168461423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc22815ec50 0x7fc2280c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc21ea00640 1 -- 172.21.15.63:0/1168461423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2280b7ba0 msgr2=0x7fc2281f5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:58.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc21ea00640 1 --2- 172.21.15.63:0/1168461423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2280b7ba0 0x7fc2281f5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc21ea00640 1 -- 172.21.15.63:0/1168461423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc22815ec50 msgr2=0x7fc2280c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc21ea00640 1 --2- 172.21.15.63:0/1168461423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc22815ec50 0x7fc2280c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc21ea00640 1 -- 172.21.15.63:0/1168461423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2280c6a20 con 0x7fc22815e870 2024-05-28T13:57:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc227e00640 1 --2- 172.21.15.63:0/1168461423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc22815ec50 0x7fc2280c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:57:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc21ea00640 1 --2- 172.21.15.63:0/1168461423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22815e870 0x7fc2280cb820 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc21001c8a0 tx=0x7fc21001ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc21fe00640 1 -- 172.21.15.63:0/1168461423 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2100bf070 con 0x7fc22815e870 2024-05-28T13:57:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc21fe00640 1 -- 172.21.15.63:0/1168461423 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2100b0640 con 0x7fc22815e870 2024-05-28T13:57:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc21fe00640 1 -- 172.21.15.63:0/1168461423 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2100b5040 con 0x7fc22815e870 2024-05-28T13:57:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc22e600640 1 -- 172.21.15.63:0/1168461423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2280c6d00 con 0x7fc22815e870 2024-05-28T13:57:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc22e600640 1 -- 172.21.15.63:0/1168461423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc228160300 con 0x7fc22815e870 2024-05-28T13:57:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc21fe00640 1 -- 172.21.15.63:0/1168461423 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc2100c2070 con 0x7fc22815e870 2024-05-28T13:57:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc21fe00640 1 -- 172.21.15.63:0/1168461423 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fc2100c4ce0 con 0x7fc22815e870 2024-05-28T13:57:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.655+0000 7fc22e600640 1 -- 172.21.15.63:0/1168461423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc22804fdd0 con 0x7fc22815e870 2024-05-28T13:57:58.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.659+0000 7fc21fe00640 1 -- 172.21.15.63:0/1168461423 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc2100ca020 con 0x7fc22815e870 2024-05-28T13:57:58.696 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:57:58.711+0000 7fc74f862180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-28T13:57:58.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.731+0000 7fc22e600640 1 -- 172.21.15.63:0/1168461423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1b3b7367-edb9-4b74-80e1-c244b4eaf23a", "id": 1} v 0) -- 0x7fc2281600b0 con 0x7fc22815e870 2024-05-28T13:57:58.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.735+0000 7fc21fe00640 1 -- 172.21.15.63:0/1168461423 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1b3b7367-edb9-4b74-80e1-c244b4eaf23a", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fc2100187d0 con 0x7fc22815e870 2024-05-28T13:57:58.719 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-05-28T13:57:58.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.735+0000 7fc22e600640 1 -- 172.21.15.63:0/1168461423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22815e870 msgr2=0x7fc2280cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:58.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.735+0000 7fc22e600640 1 --2- 172.21.15.63:0/1168461423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22815e870 0x7fc2280cb820 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc21001c8a0 tx=0x7fc21001ba10 comp rx=0 tx=0).stop 2024-05-28T13:57:58.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.735+0000 7fc22e600640 1 -- 172.21.15.63:0/1168461423 shutdown_connections 2024-05-28T13:57:58.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.735+0000 7fc22e600640 1 --2- 172.21.15.63:0/1168461423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc22815ec50 0x7fc2280c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.735+0000 7fc22e600640 1 --2- 172.21.15.63:0/1168461423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc22815e870 0x7fc2280cb820 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.735+0000 7fc22e600640 1 --2- 172.21.15.63:0/1168461423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2280b7ba0 0x7fc2281f5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.735+0000 7fc22e600640 1 -- 172.21.15.63:0/1168461423 >> 172.21.15.63:0/1168461423 conn(0x7fc2280b3fd0 msgr2=0x7fc22809aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:58.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.735+0000 7fc22e600640 1 -- 172.21.15.63:0/1168461423 shutdown_connections 2024-05-28T13:57:58.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.735+0000 7fc22e600640 1 -- 172.21.15.63:0/1168461423 wait complete. 2024-05-28T13:57:58.740 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7 2 2024-05-28T13:57:58.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34b9e00640 1 Processor -- start 2024-05-28T13:57:58.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34b9e00640 1 -- start start 2024-05-28T13:57:58.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34b9e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f34b415ec50 0x7f34b40b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34b9e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f34b40b7ba0 0x7f34b40b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34b9e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34b40d1a80 con 0x7f34b415e870 2024-05-28T13:57:58.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34b9e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f34b40cd500 con 0x7f34b40b7ba0 2024-05-28T13:57:58.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34b9e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f34b40ccbd0 con 0x7f34b415ec50 2024-05-28T13:57:58.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34abe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f34b415ec50 0x7f34b40b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34b2a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f34b415e870 0x7f34b40d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.63:41058/0 (socket says 172.21.15.63:41058) 2024-05-28T13:57:58.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34b2a00640 1 -- 172.21.15.63:0/1999357406 learned_addr learned my addr 172.21.15.63:0/1999357406 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-05-28T13:57:58.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34b3400640 1 --2- 172.21.15.63:0/1999357406 >> v2:172.21.15.42:3300/0 conn(0x7f34b40b7ba0 0x7f34b40b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34b2000640 1 -- 172.21.15.63:0/1999357406 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 257743563 0 0) 0x7f34b40d1a80 con 0x7f34b415e870 2024-05-28T13:57:58.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34b2000640 1 -- 172.21.15.63:0/1999357406 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3498003530 con 0x7f34b415e870 2024-05-28T13:57:58.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34abe00640 1 -- 172.21.15.63:0/1999357406 >> v2:172.21.15.42:3300/0 conn(0x7f34b40b7ba0 msgr2=0x7f34b40b7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:57:58.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34abe00640 1 --2- 172.21.15.63:0/1999357406 >> v2:172.21.15.42:3300/0 conn(0x7f34b40b7ba0 0x7f34b40b7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34abe00640 1 -- 172.21.15.63:0/1999357406 >> v1:172.21.15.42:6789/0 conn(0x7f34b415e870 legacy=0x7f34b40d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:58.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.895+0000 7f34abe00640 1 -- 172.21.15.63:0/1999357406 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b40cceb0 con 0x7f34b415ec50 2024-05-28T13:57:58.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.899+0000 7f34abe00640 1 --2- 172.21.15.63:0/1999357406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34b415ec50 0x7f34b40b7660 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f34b4162500 tx=0x7f349c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b8394c44929ec2d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:58.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.899+0000 7f34b2000640 1 -- 172.21.15.63:0/1999357406 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c0b0070 con 0x7f34b415ec50 2024-05-28T13:57:58.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.899+0000 7f34b2000640 1 -- 172.21.15.63:0/1999357406 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349c0ab600 con 0x7f34b415ec50 2024-05-28T13:57:58.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.899+0000 7f34b2000640 1 -- 172.21.15.63:0/1999357406 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c0ab9d0 con 0x7f34b415ec50 2024-05-28T13:57:58.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.899+0000 7f34b9e00640 1 -- 172.21.15.63:0/1999357406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34b415ec50 msgr2=0x7f34b40b7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:58.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.899+0000 7f34b9e00640 1 --2- 172.21.15.63:0/1999357406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34b415ec50 0x7f34b40b7660 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f34b4162500 tx=0x7f349c01c450 comp rx=0 tx=0).stop 2024-05-28T13:57:58.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.899+0000 7f34b9e00640 1 -- 172.21.15.63:0/1999357406 shutdown_connections 2024-05-28T13:57:58.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.899+0000 7f34b9e00640 1 --2- 172.21.15.63:0/1999357406 >> v2:172.21.15.42:3300/0 conn(0x7f34b40b7ba0 0x7f34b40b7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.899+0000 7f34b9e00640 1 --2- 172.21.15.63:0/1999357406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34b415ec50 0x7f34b40b7660 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.899+0000 7f34b9e00640 1 -- 172.21.15.63:0/1999357406 >> 172.21.15.63:0/1999357406 conn(0x7f34b40b3fd0 msgr2=0x7f34b40b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:58.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.899+0000 7f34b9e00640 1 -- 172.21.15.63:0/1999357406 shutdown_connections 2024-05-28T13:57:58.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.899+0000 7f34b9e00640 1 -- 172.21.15.63:0/1999357406 wait complete. 2024-05-28T13:57:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b9e00640 1 Processor -- start 2024-05-28T13:57:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b9e00640 1 -- start start 2024-05-28T13:57:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34b40b7ba0 0x7f34b4161470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b9e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b415e870 0x7f34b40b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34b415ec50 0x7f34b40cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b9e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f34b409b5f0 con 0x7f34b40b7ba0 2024-05-28T13:57:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b9e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34b409b470 con 0x7f34b415ec50 2024-05-28T13:57:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b9e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f34b409b770 con 0x7f34b415e870 2024-05-28T13:57:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34b40b7ba0 0x7f34b4161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34abe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b415e870 0x7f34b40b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34b415ec50 0x7f34b40cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:58.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34b40b7ba0 0x7f34b4161470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.63:57934/0 (socket says 172.21.15.63:57934) 2024-05-28T13:57:58.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b2a00640 1 -- 172.21.15.63:0/2286662378 learned_addr learned my addr 172.21.15.63:0/2286662378 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-28T13:57:58.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34abe00640 1 -- 172.21.15.63:0/2286662378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b415e870 msgr2=0x7f34b40b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-28T13:57:58.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34abe00640 1 -- 172.21.15.63:0/2286662378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b415e870 msgr2=0x7f34b40b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-28T13:57:58.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34abe00640 1 --2- 172.21.15.63:0/2286662378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b415e870 0x7f34b40b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34abe00640 1 --2- 172.21.15.63:0/2286662378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b415e870 0x7f34b40b3600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b2a00640 1 -- 172.21.15.63:0/2286662378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b415e870 msgr2=0x7f34b40b3600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-28T13:57:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b2a00640 1 --2- 172.21.15.63:0/2286662378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b415e870 0x7f34b40b3600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b2a00640 1 -- 172.21.15.63:0/2286662378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34b415ec50 msgr2=0x7f34b40cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b2a00640 1 --2- 172.21.15.63:0/2286662378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34b415ec50 0x7f34b40cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b2a00640 1 -- 172.21.15.63:0/2286662378 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b41606b0 con 0x7f34b40b7ba0 2024-05-28T13:57:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b2a00640 1 --2- 172.21.15.63:0/2286662378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34b40b7ba0 0x7f34b4161470 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f34a400c450 tx=0x7f34a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34ab400640 1 -- 172.21.15.63:0/2286662378 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4017070 con 0x7f34b40b7ba0 2024-05-28T13:57:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34ab400640 1 -- 172.21.15.63:0/2286662378 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a4005cf0 con 0x7f34b40b7ba0 2024-05-28T13:57:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b9e00640 1 -- 172.21.15.63:0/2286662378 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b4160990 con 0x7f34b40b7ba0 2024-05-28T13:57:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34ab400640 1 -- 172.21.15.63:0/2286662378 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4007290 con 0x7f34b40b7ba0 2024-05-28T13:57:58.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34b9e00640 1 -- 172.21.15.63:0/2286662378 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b40cfac0 con 0x7f34b40b7ba0 2024-05-28T13:57:58.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34ab400640 1 -- 172.21.15.63:0/2286662378 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f34a401a070 con 0x7f34b40b7ba0 2024-05-28T13:57:58.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.903+0000 7f34ab400640 1 -- 172.21.15.63:0/2286662378 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f34a401c440 con 0x7f34b40b7ba0 2024-05-28T13:57:58.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.907+0000 7f34b9e00640 1 -- 172.21.15.63:0/2286662378 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b404fdd0 con 0x7f34b40b7ba0 2024-05-28T13:57:58.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.907+0000 7f34ab400640 1 -- 172.21.15.63:0/2286662378 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f34a4021020 con 0x7f34b40b7ba0 2024-05-28T13:57:58.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.987+0000 7f34b9e00640 1 -- 172.21.15.63:0/2286662378 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7", "id": 2} v 0) -- 0x7f34b4155e70 con 0x7f34b40b7ba0 2024-05-28T13:57:58.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.991+0000 7f34ab400640 1 -- 172.21.15.63:0/2286662378 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f34a401c760 con 0x7f34b40b7ba0 2024-05-28T13:57:58.975 INFO:teuthology.orchestra.run.smithi063.stdout:2 2024-05-28T13:57:58.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.991+0000 7f34b9e00640 1 -- 172.21.15.63:0/2286662378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34b40b7ba0 msgr2=0x7f34b4161470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:58.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.991+0000 7f34b9e00640 1 --2- 172.21.15.63:0/2286662378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34b40b7ba0 0x7f34b4161470 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f34a400c450 tx=0x7f34a400c910 comp rx=0 tx=0).stop 2024-05-28T13:57:58.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.991+0000 7f34b9e00640 1 -- 172.21.15.63:0/2286662378 shutdown_connections 2024-05-28T13:57:58.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.991+0000 7f34b9e00640 1 --2- 172.21.15.63:0/2286662378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34b415ec50 0x7f34b40cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.991+0000 7f34b9e00640 1 --2- 172.21.15.63:0/2286662378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b415e870 0x7f34b40b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.991+0000 7f34b9e00640 1 --2- 172.21.15.63:0/2286662378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34b40b7ba0 0x7f34b4161470 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:58.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.991+0000 7f34b9e00640 1 -- 172.21.15.63:0/2286662378 >> 172.21.15.63:0/2286662378 conn(0x7f34b40b3fd0 msgr2=0x7f34b40b7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:58.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.991+0000 7f34b9e00640 1 -- 172.21.15.63:0/2286662378 shutdown_connections 2024-05-28T13:57:58.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:58.991+0000 7f34b9e00640 1 -- 172.21.15.63:0/2286662378 wait complete. 2024-05-28T13:57:58.995 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 54bda6c4-a5ca-4391-8299-b1b05d859110 3 2024-05-28T13:57:59.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.147+0000 7fde69400640 1 Processor -- start 2024-05-28T13:57:59.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.147+0000 7fde69400640 1 -- start start 2024-05-28T13:57:59.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.147+0000 7fde69400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406f820 0x7fde6406ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.147+0000 7fde69400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fde6406cf40 0x7fde64166380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.147+0000 7fde69400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde640737a0 con 0x7fde6415c7b0 2024-05-28T13:57:59.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.147+0000 7fde69400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fde6406d3b0 con 0x7fde6406cf40 2024-05-28T13:57:59.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.147+0000 7fde69400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fde640cfa00 con 0x7fde6406f820 2024-05-28T13:57:59.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde59e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406f820 0x7fde6406ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde62800640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fde6415c7b0 0x7fde64073090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.63:38404/0 (socket says 172.21.15.63:38404) 2024-05-28T13:57:59.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde62800640 1 -- 172.21.15.63:0/3731936405 learned_addr learned my addr 172.21.15.63:0/3731936405 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-05-28T13:57:59.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde59e00640 1 -- 172.21.15.63:0/3731936405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406f820 msgr2=0x7fde6406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-28T13:57:59.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde59e00640 1 -- 172.21.15.63:0/3731936405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406f820 msgr2=0x7fde6406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:59.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde59e00640 1 --2- 172.21.15.63:0/3731936405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406f820 0x7fde6406ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:59.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde59e00640 1 --2- 172.21.15.63:0/3731936405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406f820 0x7fde6406ca00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:59.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde63200640 1 --2- 172.21.15.63:0/3731936405 >> v2:172.21.15.63:3300/0 conn(0x7fde6406cf40 0x7fde64166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde61e00640 1 -- 172.21.15.63:0/3731936405 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4224704217 0 0) 0x7fde640737a0 con 0x7fde6415c7b0 2024-05-28T13:57:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde61e00640 1 -- 172.21.15.63:0/3731936405 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde48003530 con 0x7fde6415c7b0 2024-05-28T13:57:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde63200640 1 -- 172.21.15.63:0/3731936405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406f820 msgr2=0x7fde6406ca00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:57:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde63200640 1 --2- 172.21.15.63:0/3731936405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406f820 0x7fde6406ca00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde63200640 1 -- 172.21.15.63:0/3731936405 >> v1:172.21.15.63:6789/0 conn(0x7fde6415c7b0 legacy=0x7fde64073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde63200640 1 -- 172.21.15.63:0/3731936405 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde640cfc30 con 0x7fde6406cf40 2024-05-28T13:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.151+0000 7fde63200640 1 --2- 172.21.15.63:0/3731936405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde6406cf40 0x7fde64166380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009d90 tx=0x7fde5c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b64136680db3ba33 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde61e00640 1 -- 172.21.15.63:0/3731936405 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c0b2070 con 0x7fde6406cf40 2024-05-28T13:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde61e00640 1 -- 172.21.15.63:0/3731936405 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde5c0a5920 con 0x7fde6406cf40 2024-05-28T13:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde61e00640 1 -- 172.21.15.63:0/3731936405 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c0a5cf0 con 0x7fde6406cf40 2024-05-28T13:57:59.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 -- 172.21.15.63:0/3731936405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde6406cf40 msgr2=0x7fde64166380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 --2- 172.21.15.63:0/3731936405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde6406cf40 0x7fde64166380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009d90 tx=0x7fde5c01c970 comp rx=0 tx=0).stop 2024-05-28T13:57:59.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 -- 172.21.15.63:0/3731936405 shutdown_connections 2024-05-28T13:57:59.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 --2- 172.21.15.63:0/3731936405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde6406cf40 0x7fde64166380 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 --2- 172.21.15.63:0/3731936405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406f820 0x7fde6406ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 -- 172.21.15.63:0/3731936405 >> 172.21.15.63:0/3731936405 conn(0x7fde64072570 msgr2=0x7fde64072980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:59.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 -- 172.21.15.63:0/3731936405 shutdown_connections 2024-05-28T13:57:59.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 -- 172.21.15.63:0/3731936405 wait complete. 2024-05-28T13:57:59.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 Processor -- start 2024-05-28T13:57:59.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 -- start start 2024-05-28T13:57:59.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406cf40 0x7fde6415ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde6406f820 0x7fde641434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde6415c7b0 0x7fde6415e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fde640b0530 con 0x7fde6406f820 2024-05-28T13:57:59.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde6415cb90 con 0x7fde6415c7b0 2024-05-28T13:57:59.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde69400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fde640b06b0 con 0x7fde6406cf40 2024-05-28T13:57:59.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde63200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde6415c7b0 0x7fde6415e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde62800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406cf40 0x7fde6415ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.155+0000 7fde63200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde6415c7b0 0x7fde6415e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42786/0 (socket says 172.21.15.63:42786) 2024-05-28T13:57:59.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde63200640 1 -- 172.21.15.63:0/954734827 learned_addr learned my addr 172.21.15.63:0/954734827 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-28T13:57:59.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde59e00640 1 --2- 172.21.15.63:0/954734827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde6406f820 0x7fde641434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde62800640 1 -- 172.21.15.63:0/954734827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406cf40 msgr2=0x7fde6415ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-28T13:57:59.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde62800640 1 -- 172.21.15.63:0/954734827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406cf40 msgr2=0x7fde6415ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-28T13:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde62800640 1 --2- 172.21.15.63:0/954734827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406cf40 0x7fde6415ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde62800640 1 --2- 172.21.15.63:0/954734827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406cf40 0x7fde6415ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde63200640 1 -- 172.21.15.63:0/954734827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406cf40 msgr2=0x7fde6415ba40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-28T13:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde63200640 1 --2- 172.21.15.63:0/954734827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406cf40 0x7fde6415ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde63200640 1 -- 172.21.15.63:0/954734827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde6406f820 msgr2=0x7fde641434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde63200640 1 --2- 172.21.15.63:0/954734827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde6406f820 0x7fde641434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde63200640 1 -- 172.21.15.63:0/954734827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde640a9c70 con 0x7fde6415c7b0 2024-05-28T13:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde63200640 1 --2- 172.21.15.63:0/954734827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde6415c7b0 0x7fde6415e160 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0b0040 tx=0x7fde5c019710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde5b400640 1 -- 172.21.15.63:0/954734827 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c0b2050 con 0x7fde6415c7b0 2024-05-28T13:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde5b400640 1 -- 172.21.15.63:0/954734827 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde5c0be070 con 0x7fde6415c7b0 2024-05-28T13:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde5b400640 1 -- 172.21.15.63:0/954734827 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c0ada90 con 0x7fde6415c7b0 2024-05-28T13:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde69400640 1 -- 172.21.15.63:0/954734827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde640a9990 con 0x7fde6415c7b0 2024-05-28T13:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde69400640 1 -- 172.21.15.63:0/954734827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6415a9f0 con 0x7fde6415c7b0 2024-05-28T13:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde5b400640 1 -- 172.21.15.63:0/954734827 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fde5c0adc30 con 0x7fde6415c7b0 2024-05-28T13:57:59.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde5b400640 1 -- 172.21.15.63:0/954734827 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fde5c0a58a0 con 0x7fde6415c7b0 2024-05-28T13:57:59.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.159+0000 7fde69400640 1 -- 172.21.15.63:0/954734827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde6415a520 con 0x7fde6415c7b0 2024-05-28T13:57:59.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.163+0000 7fde5b400640 1 -- 172.21.15.63:0/954734827 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fde5c016560 con 0x7fde6415c7b0 2024-05-28T13:57:59.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.231+0000 7fde69400640 1 -- 172.21.15.63:0/954734827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd new", "uuid": "54bda6c4-a5ca-4391-8299-b1b05d859110", "id": 3} v 0) -- 0x7fde6404fdd0 con 0x7fde6415c7b0 2024-05-28T13:57:59.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.235+0000 7fde5b400640 1 -- 172.21.15.63:0/954734827 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "54bda6c4-a5ca-4391-8299-b1b05d859110", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fde5c0c3460 con 0x7fde6415c7b0 2024-05-28T13:57:59.221 INFO:teuthology.orchestra.run.smithi063.stdout:3 2024-05-28T13:57:59.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.235+0000 7fde69400640 1 -- 172.21.15.63:0/954734827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde6415c7b0 msgr2=0x7fde6415e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.235+0000 7fde69400640 1 --2- 172.21.15.63:0/954734827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde6415c7b0 0x7fde6415e160 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0b0040 tx=0x7fde5c019710 comp rx=0 tx=0).stop 2024-05-28T13:57:59.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.235+0000 7fde69400640 1 -- 172.21.15.63:0/954734827 shutdown_connections 2024-05-28T13:57:59.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.235+0000 7fde69400640 1 --2- 172.21.15.63:0/954734827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde6415c7b0 0x7fde6415e160 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.235+0000 7fde69400640 1 --2- 172.21.15.63:0/954734827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde6406f820 0x7fde641434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.235+0000 7fde69400640 1 --2- 172.21.15.63:0/954734827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde6406cf40 0x7fde6415ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.235+0000 7fde69400640 1 -- 172.21.15.63:0/954734827 >> 172.21.15.63:0/954734827 conn(0x7fde64072570 msgr2=0x7fde64155c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:59.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.235+0000 7fde69400640 1 -- 172.21.15.63:0/954734827 shutdown_connections 2024-05-28T13:57:59.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.235+0000 7fde69400640 1 -- 172.21.15.63:0/954734827 wait complete. 2024-05-28T13:57:59.240 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62 4 2024-05-28T13:57:59.260 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:57:59.275+0000 7fc74f862180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-28T13:57:59.271 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:57:59.282+0000 7f07f8d0b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-28T13:57:59.337 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:57:59.350+0000 7f07f8d0b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-28T13:57:59.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.391+0000 7f236b200640 1 Processor -- start 2024-05-28T13:57:59.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.391+0000 7f236b200640 1 -- start start 2024-05-28T13:57:59.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f236b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415ec50 0x7f23640b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f236b200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f23640b7ba0 0x7f23640b7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f236b200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23640d1a80 con 0x7f236415e870 2024-05-28T13:57:59.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f236b200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f23640cd500 con 0x7f23640b7ba0 2024-05-28T13:57:59.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f236b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f23640ccbd0 con 0x7f236415ec50 2024-05-28T13:57:59.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f235be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415ec50 0x7f23640b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f2363e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f236415e870 0x7f23640d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.63:38412/0 (socket says 172.21.15.63:38412) 2024-05-28T13:57:59.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f2363e00640 1 -- 172.21.15.63:0/3012823994 learned_addr learned my addr 172.21.15.63:0/3012823994 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-05-28T13:57:59.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f2369200640 1 --2- 172.21.15.63:0/3012823994 >> v2:172.21.15.63:3300/0 conn(0x7f23640b7ba0 0x7f23640b7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f235be00640 1 -- 172.21.15.63:0/3012823994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415ec50 msgr2=0x7f23640b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-28T13:57:59.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f235be00640 1 -- 172.21.15.63:0/3012823994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415ec50 msgr2=0x7f23640b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:59.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f235be00640 1 --2- 172.21.15.63:0/3012823994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415ec50 0x7f23640b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:59.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f235be00640 1 --2- 172.21.15.63:0/3012823994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415ec50 0x7f23640b7660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:59.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f2369200640 1 -- 172.21.15.63:0/3012823994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415ec50 msgr2=0x7f23640b7660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:57:59.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f2369200640 1 --2- 172.21.15.63:0/3012823994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415ec50 0x7f23640b7660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f2369200640 1 -- 172.21.15.63:0/3012823994 >> v1:172.21.15.63:6789/0 conn(0x7f236415e870 legacy=0x7f23640d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.395+0000 7f2369200640 1 -- 172.21.15.63:0/3012823994 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23640cceb0 con 0x7f23640b7ba0 2024-05-28T13:57:59.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.399+0000 7f2369200640 1 --2- 172.21.15.63:0/3012823994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23640b7ba0 0x7f23640b7ff0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f235c009d90 tx=0x7f235c01f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3ef0cbed6a33890 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:59.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.399+0000 7f2363400640 1 -- 172.21.15.63:0/3012823994 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c0b4070 con 0x7f23640b7ba0 2024-05-28T13:57:59.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.399+0000 7f2363400640 1 -- 172.21.15.63:0/3012823994 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f235c0af600 con 0x7f23640b7ba0 2024-05-28T13:57:59.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.399+0000 7f2363400640 1 -- 172.21.15.63:0/3012823994 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c0af9d0 con 0x7f23640b7ba0 2024-05-28T13:57:59.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.399+0000 7f236b200640 1 -- 172.21.15.63:0/3012823994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23640b7ba0 msgr2=0x7f23640b7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.399+0000 7f236b200640 1 --2- 172.21.15.63:0/3012823994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23640b7ba0 0x7f23640b7ff0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f235c009d90 tx=0x7f235c01f150 comp rx=0 tx=0).stop 2024-05-28T13:57:59.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.399+0000 7f236b200640 1 -- 172.21.15.63:0/3012823994 shutdown_connections 2024-05-28T13:57:59.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.399+0000 7f236b200640 1 --2- 172.21.15.63:0/3012823994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23640b7ba0 0x7f23640b7ff0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.399+0000 7f236b200640 1 --2- 172.21.15.63:0/3012823994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415ec50 0x7f23640b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.399+0000 7f236b200640 1 -- 172.21.15.63:0/3012823994 >> 172.21.15.63:0/3012823994 conn(0x7f23640b3fd0 msgr2=0x7f23640b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:59.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.399+0000 7f236b200640 1 -- 172.21.15.63:0/3012823994 shutdown_connections 2024-05-28T13:57:59.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.399+0000 7f236b200640 1 -- 172.21.15.63:0/3012823994 wait complete. 2024-05-28T13:57:59.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.399+0000 7f236b200640 1 Processor -- start 2024-05-28T13:57:59.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.399+0000 7f236b200640 1 -- start start 2024-05-28T13:57:59.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f236b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23640b7ba0 0x7f236415ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f236b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415e870 0x7f23640c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f236b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f236415ec50 0x7f2364161280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f236b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f23640b5330 con 0x7f23640b7ba0 2024-05-28T13:57:59.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f236b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f236409aa50 con 0x7f236415ec50 2024-05-28T13:57:59.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f236b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2364158af0 con 0x7f236415e870 2024-05-28T13:57:59.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f235be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415e870 0x7f23640c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f2369200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f236415ec50 0x7f2364161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f235be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415e870 0x7f23640c9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:41894/0 (socket says 172.21.15.63:41894) 2024-05-28T13:57:59.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f235be00640 1 -- 172.21.15.63:0/889186220 learned_addr learned my addr 172.21.15.63:0/889186220 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-28T13:57:59.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f2363e00640 1 --2- 172.21.15.63:0/889186220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23640b7ba0 0x7f236415ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f235be00640 1 -- 172.21.15.63:0/889186220 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415e870 msgr2=0x7f23640c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-28T13:57:59.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f235be00640 1 -- 172.21.15.63:0/889186220 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415e870 msgr2=0x7f23640c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-28T13:57:59.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f235be00640 1 --2- 172.21.15.63:0/889186220 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415e870 0x7f23640c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:59.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f235be00640 1 --2- 172.21.15.63:0/889186220 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415e870 0x7f23640c9fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:59.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f2369200640 1 -- 172.21.15.63:0/889186220 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415e870 msgr2=0x7f23640c9fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-28T13:57:59.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f2369200640 1 --2- 172.21.15.63:0/889186220 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415e870 0x7f23640c9fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f2369200640 1 -- 172.21.15.63:0/889186220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23640b7ba0 msgr2=0x7f236415ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f2369200640 1 --2- 172.21.15.63:0/889186220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23640b7ba0 0x7f236415ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f2369200640 1 -- 172.21.15.63:0/889186220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23640ca650 con 0x7f236415ec50 2024-05-28T13:57:59.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f2369200640 1 --2- 172.21.15.63:0/889186220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f236415ec50 0x7f2364161280 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f235c0af000 tx=0x7f235c01bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:59.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f2360c00640 1 -- 172.21.15.63:0/889186220 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c0c0020 con 0x7f236415ec50 2024-05-28T13:57:59.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f2360c00640 1 -- 172.21.15.63:0/889186220 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f235c0ba470 con 0x7f236415ec50 2024-05-28T13:57:59.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f2360c00640 1 -- 172.21.15.63:0/889186220 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c0b4040 con 0x7f236415ec50 2024-05-28T13:57:59.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f236b200640 1 -- 172.21.15.63:0/889186220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23640ca930 con 0x7f236415ec50 2024-05-28T13:57:59.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f236b200640 1 -- 172.21.15.63:0/889186220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23640caef0 con 0x7f236415ec50 2024-05-28T13:57:59.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f2360c00640 1 -- 172.21.15.63:0/889186220 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f235c0c3070 con 0x7f236415ec50 2024-05-28T13:57:59.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f2360c00640 1 -- 172.21.15.63:0/889186220 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f235c0bb420 con 0x7f236415ec50 2024-05-28T13:57:59.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.403+0000 7f236b200640 1 -- 172.21.15.63:0/889186220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f236404fdd0 con 0x7f236415ec50 2024-05-28T13:57:59.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.407+0000 7f2360c00640 1 -- 172.21.15.63:0/889186220 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f235c0ca070 con 0x7f236415ec50 2024-05-28T13:57:59.402 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:57:59.414+0000 7f07f8d0b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-28T13:57:59.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.475+0000 7f236b200640 1 -- 172.21.15.63:0/889186220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62", "id": 4} v 0) -- 0x7f236406e2c0 con 0x7f236415ec50 2024-05-28T13:57:59.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.479+0000 7f2360c00640 1 -- 172.21.15.63:0/889186220 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f235c018cb0 con 0x7f236415ec50 2024-05-28T13:57:59.466 INFO:teuthology.orchestra.run.smithi063.stdout:4 2024-05-28T13:57:59.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.479+0000 7f236b200640 1 -- 172.21.15.63:0/889186220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f236415ec50 msgr2=0x7f2364161280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.479+0000 7f236b200640 1 --2- 172.21.15.63:0/889186220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f236415ec50 0x7f2364161280 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f235c0af000 tx=0x7f235c01bef0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.479+0000 7f236b200640 1 -- 172.21.15.63:0/889186220 shutdown_connections 2024-05-28T13:57:59.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.479+0000 7f236b200640 1 --2- 172.21.15.63:0/889186220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f236415ec50 0x7f2364161280 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.479+0000 7f236b200640 1 --2- 172.21.15.63:0/889186220 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f236415e870 0x7f23640c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.479+0000 7f236b200640 1 --2- 172.21.15.63:0/889186220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23640b7ba0 0x7f236415ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.479+0000 7f236b200640 1 -- 172.21.15.63:0/889186220 >> 172.21.15.63:0/889186220 conn(0x7f23640b3fd0 msgr2=0x7f23640b6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:59.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.479+0000 7f236b200640 1 -- 172.21.15.63:0/889186220 shutdown_connections 2024-05-28T13:57:59.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.479+0000 7f236b200640 1 -- 172.21.15.63:0/889186220 wait complete. 2024-05-28T13:57:59.485 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 3e53bcb8-bda8-4013-90db-730b694cf40f 5 2024-05-28T13:57:59.496 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:57:59.506+0000 7f07f8d0b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-28T13:57:59.497 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:57:59.511+0000 7fc74f862180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-28T13:57:59.560 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:57:59.570+0000 7f07f8d0b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-28T13:57:59.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.651+0000 7ff2ada00640 1 Processor -- start 2024-05-28T13:57:59.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.651+0000 7ff2ada00640 1 -- start start 2024-05-28T13:57:59.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.651+0000 7ff2ada00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a806b9f0 0x7ff2a806bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.651+0000 7ff2ada00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff2a8070000 0x7ff2a80cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.651+0000 7ff2ada00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2a806c310 con 0x7ff2a80b08e0 2024-05-28T13:57:59.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.651+0000 7ff2ada00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff2a80cbf40 con 0x7ff2a8070000 2024-05-28T13:57:59.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.651+0000 7ff2ada00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a80cdb40 con 0x7ff2a806b9f0 2024-05-28T13:57:59.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.651+0000 7ff2a7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a806b9f0 0x7ff2a806bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.651+0000 7ff29fe00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7ff2a80b08e0 0x7ff2a80cb830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.63:38424/0 (socket says 172.21.15.63:38424) 2024-05-28T13:57:59.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.651+0000 7ff29fe00640 1 -- 172.21.15.63:0/1709394075 learned_addr learned my addr 172.21.15.63:0/1709394075 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-05-28T13:57:59.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.655+0000 7ff2ad000640 1 --2- 172.21.15.63:0/1709394075 >> v2:172.21.15.42:3300/0 conn(0x7ff2a8070000 0x7ff2a80cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.655+0000 7ff2a7e00640 1 -- 172.21.15.63:0/1709394075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a806b9f0 msgr2=0x7ff2a806bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-28T13:57:59.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.655+0000 7ff2a7e00640 1 -- 172.21.15.63:0/1709394075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a806b9f0 msgr2=0x7ff2a806bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:59.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.655+0000 7ff2a7e00640 1 --2- 172.21.15.63:0/1709394075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a806b9f0 0x7ff2a806bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:59.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.655+0000 7ff2a7e00640 1 --2- 172.21.15.63:0/1709394075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a806b9f0 0x7ff2a806bdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:59.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.655+0000 7ff2ad000640 1 -- 172.21.15.63:0/1709394075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a806b9f0 msgr2=0x7ff2a806bdd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:57:59.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.655+0000 7ff2ad000640 1 --2- 172.21.15.63:0/1709394075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a806b9f0 0x7ff2a806bdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.655+0000 7ff2ad000640 1 -- 172.21.15.63:0/1709394075 >> v1:172.21.15.63:6789/0 conn(0x7ff2a80b08e0 legacy=0x7ff2a80cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.655+0000 7ff2ad000640 1 -- 172.21.15.63:0/1709394075 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a80704e0 con 0x7ff2a8070000 2024-05-28T13:57:59.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.655+0000 7ff2a7400640 1 -- 172.21.15.63:0/1709394075 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2039776396 0 0) 0x7ff2a806c310 con 0x7ff2a80b08e0 2024-05-28T13:57:59.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.655+0000 7ff2ad000640 1 --2- 172.21.15.63:0/1709394075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a8070000 0x7ff2a80cdfd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009d90 tx=0x7ff2a001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afac12ea556e38ac server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:59.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.655+0000 7ff2a7400640 1 -- 172.21.15.63:0/1709394075 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00b2070 con 0x7ff2a8070000 2024-05-28T13:57:59.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.655+0000 7ff2a7400640 1 -- 172.21.15.63:0/1709394075 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a00a5920 con 0x7ff2a8070000 2024-05-28T13:57:59.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.655+0000 7ff2a7400640 1 -- 172.21.15.63:0/1709394075 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00a5cf0 con 0x7ff2a8070000 2024-05-28T13:57:59.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 -- 172.21.15.63:0/1709394075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a8070000 msgr2=0x7ff2a80cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 --2- 172.21.15.63:0/1709394075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a8070000 0x7ff2a80cdfd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009d90 tx=0x7ff2a001c970 comp rx=0 tx=0).stop 2024-05-28T13:57:59.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 -- 172.21.15.63:0/1709394075 shutdown_connections 2024-05-28T13:57:59.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 --2- 172.21.15.63:0/1709394075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a8070000 0x7ff2a80cdfd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 --2- 172.21.15.63:0/1709394075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a806b9f0 0x7ff2a806bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 -- 172.21.15.63:0/1709394075 >> 172.21.15.63:0/1709394075 conn(0x7ff2a80b2d70 msgr2=0x7ff2a80b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:59.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 -- 172.21.15.63:0/1709394075 shutdown_connections 2024-05-28T13:57:59.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 -- 172.21.15.63:0/1709394075 wait complete. 2024-05-28T13:57:59.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 Processor -- start 2024-05-28T13:57:59.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 -- start start 2024-05-28T13:57:59.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2a806b9f0 0x7ff2a8151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a8070000 0x7ff2a80af450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a80b08e0 0x7ff2a8156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a80ceaa0 con 0x7ff2a8070000 2024-05-28T13:57:59.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a80ce920 con 0x7ff2a806b9f0 2024-05-28T13:57:59.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ada00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a816bb50 con 0x7ff2a80b08e0 2024-05-28T13:57:59.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ad000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a80b08e0 0x7ff2a8156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2a7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2a806b9f0 0x7ff2a8151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ad000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a80b08e0 0x7ff2a8156810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:41902/0 (socket says 172.21.15.63:41902) 2024-05-28T13:57:59.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ad000640 1 -- 172.21.15.63:0/1696247899 learned_addr learned my addr 172.21.15.63:0/1696247899 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-28T13:57:59.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff29fe00640 1 --2- 172.21.15.63:0/1696247899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a8070000 0x7ff2a80af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ad000640 1 -- 172.21.15.63:0/1696247899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a80b08e0 msgr2=0x7ff2a8156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-28T13:57:59.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ad000640 1 -- 172.21.15.63:0/1696247899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a80b08e0 msgr2=0x7ff2a8156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-28T13:57:59.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ad000640 1 --2- 172.21.15.63:0/1696247899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a80b08e0 0x7ff2a8156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:59.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff2ad000640 1 --2- 172.21.15.63:0/1696247899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a80b08e0 0x7ff2a8156810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:59.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff29fe00640 1 -- 172.21.15.63:0/1696247899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a80b08e0 msgr2=0x7ff2a8156810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-28T13:57:59.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff29fe00640 1 --2- 172.21.15.63:0/1696247899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a80b08e0 0x7ff2a8156810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff29fe00640 1 -- 172.21.15.63:0/1696247899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2a806b9f0 msgr2=0x7ff2a8151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff29fe00640 1 --2- 172.21.15.63:0/1696247899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2a806b9f0 0x7ff2a8151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.659+0000 7ff29fe00640 1 -- 172.21.15.63:0/1696247899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a80af990 con 0x7ff2a8070000 2024-05-28T13:57:59.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.663+0000 7ff29fe00640 1 --2- 172.21.15.63:0/1696247899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a8070000 0x7ff2a80af450 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff29000c410 tx=0x7ff29000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:59.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.663+0000 7ff2a4c00640 1 -- 172.21.15.63:0/1696247899 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900074e0 con 0x7ff2a8070000 2024-05-28T13:57:59.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.663+0000 7ff2a4c00640 1 -- 172.21.15.63:0/1696247899 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2900047f0 con 0x7ff2a8070000 2024-05-28T13:57:59.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.663+0000 7ff2a4c00640 1 -- 172.21.15.63:0/1696247899 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290007680 con 0x7ff2a8070000 2024-05-28T13:57:59.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.663+0000 7ff2ada00640 1 -- 172.21.15.63:0/1696247899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a8152580 con 0x7ff2a8070000 2024-05-28T13:57:59.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.663+0000 7ff2ada00640 1 -- 172.21.15.63:0/1696247899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a8152b40 con 0x7ff2a8070000 2024-05-28T13:57:59.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.663+0000 7ff2a4c00640 1 -- 172.21.15.63:0/1696247899 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff290011070 con 0x7ff2a8070000 2024-05-28T13:57:59.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.663+0000 7ff2a4c00640 1 -- 172.21.15.63:0/1696247899 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7ff290013940 con 0x7ff2a8070000 2024-05-28T13:57:59.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.663+0000 7ff2ada00640 1 -- 172.21.15.63:0/1696247899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a804fcf0 con 0x7ff2a8070000 2024-05-28T13:57:59.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.663+0000 7ff2a4c00640 1 -- 172.21.15.63:0/1696247899 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff290004be0 con 0x7ff2a8070000 2024-05-28T13:57:59.654 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:57:59.666+0000 7f07f8d0b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-28T13:57:59.721 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:57:59.734+0000 7f07f8d0b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-28T13:57:59.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.735+0000 7ff2ada00640 1 -- 172.21.15.63:0/1696247899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3e53bcb8-bda8-4013-90db-730b694cf40f", "id": 5} v 0) -- 0x7ff2a815d0f0 con 0x7ff2a8070000 2024-05-28T13:57:59.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.739+0000 7ff2a4c00640 1 -- 172.21.15.63:0/1696247899 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3e53bcb8-bda8-4013-90db-730b694cf40f", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7ff290004dc0 con 0x7ff2a8070000 2024-05-28T13:57:59.723 INFO:teuthology.orchestra.run.smithi063.stdout:5 2024-05-28T13:57:59.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.739+0000 7ff2ada00640 1 -- 172.21.15.63:0/1696247899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a8070000 msgr2=0x7ff2a80af450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.739+0000 7ff2ada00640 1 --2- 172.21.15.63:0/1696247899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a8070000 0x7ff2a80af450 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff29000c410 tx=0x7ff29000c8d0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.739+0000 7ff2ada00640 1 -- 172.21.15.63:0/1696247899 shutdown_connections 2024-05-28T13:57:59.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.739+0000 7ff2ada00640 1 --2- 172.21.15.63:0/1696247899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2a80b08e0 0x7ff2a8156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.739+0000 7ff2ada00640 1 --2- 172.21.15.63:0/1696247899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a8070000 0x7ff2a80af450 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.739+0000 7ff2ada00640 1 --2- 172.21.15.63:0/1696247899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2a806b9f0 0x7ff2a8151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.739+0000 7ff2ada00640 1 -- 172.21.15.63:0/1696247899 >> 172.21.15.63:0/1696247899 conn(0x7ff2a80b2d70 msgr2=0x7ff2a815c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:59.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.739+0000 7ff2ada00640 1 -- 172.21.15.63:0/1696247899 shutdown_connections 2024-05-28T13:57:59.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.739+0000 7ff2ada00640 1 -- 172.21.15.63:0/1696247899 wait complete. 2024-05-28T13:57:59.743 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 9ba8ecb4-2658-4657-b6fb-569565983fb7 6 2024-05-28T13:57:59.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1520400640 1 Processor -- start 2024-05-28T13:57:59.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1520400640 1 -- start start 2024-05-28T13:57:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1520400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f151815cb50 0x7f151809b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1520400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f151809ba40 con 0x7f15180d11d0 2024-05-28T13:57:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1520400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15180c60d0 con 0x7f151815c770 2024-05-28T13:57:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1520400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f15180c6720 con 0x7f151815cb50 2024-05-28T13:57:59.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f151da00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f151815c770 0x7f15180d0ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.63:38440/0 (socket says 172.21.15.63:38440) 2024-05-28T13:57:59.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f151da00640 1 -- 172.21.15.63:0/36163864 learned_addr learned my addr 172.21.15.63:0/36163864 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-05-28T13:57:59.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f151d000640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f151815cb50 0x7f151809b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f151d000640 1 -- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f151815cb50 msgr2=0x7f151809b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-28T13:57:59.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f151d000640 1 -- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f151815cb50 msgr2=0x7f151809b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:57:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f151d000640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f151815cb50 0x7f151809b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f151d000640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f151815cb50 0x7f151809b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2222739009 0 0) 0x7f15180c60d0 con 0x7f151815c770 2024-05-28T13:57:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1500003530 con 0x7f151815c770 2024-05-28T13:57:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3420137532 0 0) 0x7f151809ba40 con 0x7f15180d11d0 2024-05-28T13:57:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15180c60d0 con 0x7f15180d11d0 2024-05-28T13:57:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1674394454 0 0) 0x7f1500003530 con 0x7f151815c770 2024-05-28T13:57:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f151815cb50 msgr2=0x7f151809b460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:57:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f151815cb50 0x7f151809b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 >> v1:172.21.15.42:6789/0 conn(0x7f15180d11d0 legacy=0x7f1518166470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15180c63b0 con 0x7f151815c770 2024-05-28T13:57:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f150c001680 con 0x7f151815c770 2024-05-28T13:57:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 >> v1:172.21.15.63:6789/0 conn(0x7f151815c770 legacy=0x7f15180d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1500003b80 0x7f1500003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1500004810 0x7f1500004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f151815cb50 0x7f15000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f15180c6720 con 0x7f1500003b80 2024-05-28T13:57:59.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f150000ba80 con 0x7f1500004810 2024-05-28T13:57:59.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f150000bc80 con 0x7f151815cb50 2024-05-28T13:57:59.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f151d000640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1500004810 0x7f1500004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f151da00640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1500003b80 0x7f1500003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.899+0000 7f151e400640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f151815cb50 0x7f15000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f151d000640 1 -- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f151815cb50 msgr2=0x7f15000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:57:59.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f151d000640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f151815cb50 0x7f15000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f151d000640 1 -- 172.21.15.63:0/36163864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1500003b80 msgr2=0x7f1500003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:57:59.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f151d000640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1500003b80 0x7f1500003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f151d000640 1 -- 172.21.15.63:0/36163864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1500003530 con 0x7f1500004810 2024-05-28T13:57:59.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f151da00640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1500003b80 0x7f1500003f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:57:59.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f151d000640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1500004810 0x7f1500004c90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1504000ee0 tx=0x7f150401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=617175e2ba20a72b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:59.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150400f040 con 0x7f1500004810 2024-05-28T13:57:59.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15040b0070 con 0x7f1500004810 2024-05-28T13:57:59.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f1517e00640 1 -- 172.21.15.63:0/36163864 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15040ab850 con 0x7f1500004810 2024-05-28T13:57:59.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f1520400640 1 -- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1500004810 msgr2=0x7f1500004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f1520400640 1 --2- 172.21.15.63:0/36163864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1500004810 0x7f1500004c90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1504000ee0 tx=0x7f150401c4d0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f151e400640 1 -- 172.21.15.63:0/36163864 reap_dead start 2024-05-28T13:57:59.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f1520400640 1 -- 172.21.15.63:0/36163864 shutdown_connections 2024-05-28T13:57:59.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.903+0000 7f1520400640 1 -- 172.21.15.63:0/36163864 >> 172.21.15.63:0/36163864 conn(0x7f15180b3fd0 msgr2=0x7f15180b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:59.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f1520400640 1 -- 172.21.15.63:0/36163864 shutdown_connections 2024-05-28T13:57:59.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f1520400640 1 -- 172.21.15.63:0/36163864 wait complete. 2024-05-28T13:57:59.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f1520400640 1 Processor -- start 2024-05-28T13:57:59.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f1520400640 1 -- start start 2024-05-28T13:57:59.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f1520400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1500004810 0x7f15180b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f1520400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15181f28e0 0x7f15180b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f1520400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151809c580 0x7f151809ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:57:59.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f1520400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f15180c8bc0 con 0x7f151809c580 2024-05-28T13:57:59.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f1520400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f15180ce2c0 con 0x7f1500004810 2024-05-28T13:57:59.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f1520400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f15180c8d40 con 0x7f15181f28e0 2024-05-28T13:57:59.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1500004810 0x7f15180b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151d000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15181f28e0 0x7f15180b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151d000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15181f28e0 0x7f15180b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:41924/0 (socket says 172.21.15.63:41924) 2024-05-28T13:57:59.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151d000640 1 -- 172.21.15.63:0/2267030316 learned_addr learned my addr 172.21.15.63:0/2267030316 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-28T13:57:59.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1500004810 0x7f15180b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42812/0 (socket says 172.21.15.63:42812) 2024-05-28T13:57:59.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151e400640 1 --2- 172.21.15.63:0/2267030316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151809c580 0x7f151809ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:57:59.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151d000640 1 -- 172.21.15.63:0/2267030316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15181f28e0 msgr2=0x7f15180b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-28T13:57:59.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151d000640 1 -- 172.21.15.63:0/2267030316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15181f28e0 msgr2=0x7f15180b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-28T13:57:59.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151d000640 1 --2- 172.21.15.63:0/2267030316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15181f28e0 0x7f15180b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:57:59.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151d000640 1 --2- 172.21.15.63:0/2267030316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15181f28e0 0x7f15180b5d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:57:59.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151da00640 1 -- 172.21.15.63:0/2267030316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15181f28e0 msgr2=0x7f15180b5d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-28T13:57:59.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151da00640 1 --2- 172.21.15.63:0/2267030316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15181f28e0 0x7f15180b5d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151da00640 1 -- 172.21.15.63:0/2267030316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151809c580 msgr2=0x7f151809ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151da00640 1 --2- 172.21.15.63:0/2267030316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151809c580 0x7f151809ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151da00640 1 -- 172.21.15.63:0/2267030316 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15181f2d60 con 0x7f1500004810 2024-05-28T13:57:59.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151e400640 1 --2- 172.21.15.63:0/2267030316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151809c580 0x7f151809ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:57:59.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.907+0000 7f151da00640 1 --2- 172.21.15.63:0/2267030316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1500004810 0x7f15180b50e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f150c007290 tx=0x7f150c007690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:57:59.897 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:57:59.910+0000 7f07f8d0b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-28T13:57:59.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.911+0000 7f1515600640 1 -- 172.21.15.63:0/2267030316 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c000ee0 con 0x7f1500004810 2024-05-28T13:57:59.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.911+0000 7f1515600640 1 -- 172.21.15.63:0/2267030316 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c011c50 con 0x7f1500004810 2024-05-28T13:57:59.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.911+0000 7f1515600640 1 -- 172.21.15.63:0/2267030316 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c01b420 con 0x7f1500004810 2024-05-28T13:57:59.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.911+0000 7f1520400640 1 -- 172.21.15.63:0/2267030316 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15181f3040 con 0x7f1500004810 2024-05-28T13:57:59.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.911+0000 7f1520400640 1 -- 172.21.15.63:0/2267030316 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15181f3600 con 0x7f1500004810 2024-05-28T13:57:59.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.911+0000 7f1515600640 1 -- 172.21.15.63:0/2267030316 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f150c0257d0 con 0x7f1500004810 2024-05-28T13:57:59.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.911+0000 7f1515600640 1 -- 172.21.15.63:0/2267030316 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f150c024ca0 con 0x7f1500004810 2024-05-28T13:57:59.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.911+0000 7f1520400640 1 -- 172.21.15.63:0/2267030316 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15180a04d0 con 0x7f1500004810 2024-05-28T13:57:59.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.911+0000 7f1515600640 1 -- 172.21.15.63:0/2267030316 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f150c020030 con 0x7f1500004810 2024-05-28T13:57:59.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.983+0000 7f1520400640 1 -- 172.21.15.63:0/2267030316 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9ba8ecb4-2658-4657-b6fb-569565983fb7", "id": 6} v 0) -- 0x7f15180b3d30 con 0x7f1500004810 2024-05-28T13:57:59.972 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:57:59.982+0000 7f07f8d0b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-28T13:57:59.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.987+0000 7f1515600640 1 -- 172.21.15.63:0/2267030316 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9ba8ecb4-2658-4657-b6fb-569565983fb7", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f150c026b30 con 0x7f1500004810 2024-05-28T13:57:59.972 INFO:teuthology.orchestra.run.smithi063.stdout:6 2024-05-28T13:57:59.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.987+0000 7f1520400640 1 -- 172.21.15.63:0/2267030316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1500004810 msgr2=0x7f15180b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:57:59.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.987+0000 7f1520400640 1 --2- 172.21.15.63:0/2267030316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1500004810 0x7f15180b50e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f150c007290 tx=0x7f150c007690 comp rx=0 tx=0).stop 2024-05-28T13:57:59.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.987+0000 7f1520400640 1 -- 172.21.15.63:0/2267030316 shutdown_connections 2024-05-28T13:57:59.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.987+0000 7f1520400640 1 --2- 172.21.15.63:0/2267030316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151809c580 0x7f151809ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.987+0000 7f1520400640 1 --2- 172.21.15.63:0/2267030316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15181f28e0 0x7f15180b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.987+0000 7f1520400640 1 --2- 172.21.15.63:0/2267030316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1500004810 0x7f15180b50e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:57:59.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.987+0000 7f1520400640 1 -- 172.21.15.63:0/2267030316 >> 172.21.15.63:0/2267030316 conn(0x7f15180b3fd0 msgr2=0x7f151815ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:57:59.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.987+0000 7f1520400640 1 -- 172.21.15.63:0/2267030316 shutdown_connections 2024-05-28T13:57:59.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:57:59.987+0000 7f1520400640 1 -- 172.21.15.63:0/2267030316 wait complete. 2024-05-28T13:57:59.998 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 91586b96-af01-4370-b64c-71ff4d46a7cb 7 2024-05-28T13:58:00.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.151+0000 7fc1fe000640 1 Processor -- start 2024-05-28T13:58:00.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.151+0000 7fc1fe000640 1 -- start start 2024-05-28T13:58:00.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1fe000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f815c770 0x7fc1f809b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:00.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1fe000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc1f80d11d0 0x7fc1f8166470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:00.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1fe000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1f809ba40 con 0x7fc1f815cb50 2024-05-28T13:58:00.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1fe000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc1f80c6720 con 0x7fc1f80d11d0 2024-05-28T13:58:00.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1fe000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f80c60d0 con 0x7fc1f815c770 2024-05-28T13:58:00.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1efe00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fc1f815cb50 0x7fc1f80d0ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.63:38450/0 (socket says 172.21.15.63:38450) 2024-05-28T13:58:00.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1efe00640 1 -- 172.21.15.63:0/200134094 learned_addr learned my addr 172.21.15.63:0/200134094 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-05-28T13:58:00.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1f7400640 1 --2- 172.21.15.63:0/200134094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f815c770 0x7fc1f809b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:00.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1f7e00640 1 --2- 172.21.15.63:0/200134094 >> v2:172.21.15.42:3300/0 conn(0x7fc1f80d11d0 0x7fc1f8166470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:00.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1f7400640 1 -- 172.21.15.63:0/200134094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f815c770 msgr2=0x7fc1f809b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-28T13:58:00.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1f7400640 1 -- 172.21.15.63:0/200134094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f815c770 msgr2=0x7fc1f809b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-28T13:58:00.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1f7400640 1 --2- 172.21.15.63:0/200134094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f815c770 0x7fc1f809b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:58:00.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1f7400640 1 --2- 172.21.15.63:0/200134094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f815c770 0x7fc1f809b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:58:00.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1f6a00640 1 -- 172.21.15.63:0/200134094 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3410563097 0 0) 0x7fc1f809ba40 con 0x7fc1f815cb50 2024-05-28T13:58:00.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1f6a00640 1 -- 172.21.15.63:0/200134094 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1dc003530 con 0x7fc1f815cb50 2024-05-28T13:58:00.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1f7e00640 1 -- 172.21.15.63:0/200134094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f815c770 msgr2=0x7fc1f809b460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:58:00.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1f7e00640 1 --2- 172.21.15.63:0/200134094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f815c770 0x7fc1f809b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:00.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1f7e00640 1 -- 172.21.15.63:0/200134094 >> v1:172.21.15.63:6789/0 conn(0x7fc1f815cb50 legacy=0x7fc1f80d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:00.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.155+0000 7fc1f7e00640 1 -- 172.21.15.63:0/200134094 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f80c63b0 con 0x7fc1f80d11d0 2024-05-28T13:58:00.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.159+0000 7fc1f7e00640 1 --2- 172.21.15.63:0/200134094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f80d11d0 0x7fc1f8166470 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009d90 tx=0x7fc1f001f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65ab428806b8509 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:00.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.159+0000 7fc1f6a00640 1 -- 172.21.15.63:0/200134094 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00b4070 con 0x7fc1f80d11d0 2024-05-28T13:58:00.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.159+0000 7fc1f6a00640 1 -- 172.21.15.63:0/200134094 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f00af600 con 0x7fc1f80d11d0 2024-05-28T13:58:00.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.159+0000 7fc1f6a00640 1 -- 172.21.15.63:0/200134094 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00af9d0 con 0x7fc1f80d11d0 2024-05-28T13:58:00.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.159+0000 7fc1fe000640 1 -- 172.21.15.63:0/200134094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f80d11d0 msgr2=0x7fc1f8166470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:00.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.159+0000 7fc1fe000640 1 --2- 172.21.15.63:0/200134094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f80d11d0 0x7fc1f8166470 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009d90 tx=0x7fc1f001f150 comp rx=0 tx=0).stop 2024-05-28T13:58:00.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.159+0000 7fc1fe000640 1 -- 172.21.15.63:0/200134094 shutdown_connections 2024-05-28T13:58:00.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.159+0000 7fc1fe000640 1 --2- 172.21.15.63:0/200134094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f80d11d0 0x7fc1f8166470 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:00.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.159+0000 7fc1fe000640 1 --2- 172.21.15.63:0/200134094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f815c770 0x7fc1f809b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:00.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.159+0000 7fc1fe000640 1 -- 172.21.15.63:0/200134094 >> 172.21.15.63:0/200134094 conn(0x7fc1f80b3fd0 msgr2=0x7fc1f80b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:00.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.159+0000 7fc1fe000640 1 -- 172.21.15.63:0/200134094 shutdown_connections 2024-05-28T13:58:00.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.159+0000 7fc1fe000640 1 -- 172.21.15.63:0/200134094 wait complete. 2024-05-28T13:58:00.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.159+0000 7fc1fe000640 1 Processor -- start 2024-05-28T13:58:00.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.159+0000 7fc1fe000640 1 -- start start 2024-05-28T13:58:00.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1fe000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f80d11d0 0x7fc1f8159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:00.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1fe000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f815c770 0x7fc1f81e50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:00.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1fe000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1f815cb50 0x7fc1f81f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:00.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1fe000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f80c8bc0 con 0x7fc1f815c770 2024-05-28T13:58:00.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1fe000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f80ce2c0 con 0x7fc1f815cb50 2024-05-28T13:58:00.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1fe000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f80c8d40 con 0x7fc1f80d11d0 2024-05-28T13:58:00.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1f7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1f815cb50 0x7fc1f81f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:00.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1f7400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f80d11d0 0x7fc1f8159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:00.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1f7400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f80d11d0 0x7fc1f8159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:41952/0 (socket says 172.21.15.63:41952) 2024-05-28T13:58:00.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1f7400640 1 -- 172.21.15.63:0/2818067853 learned_addr learned my addr 172.21.15.63:0/2818067853 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-28T13:58:00.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1f7e00640 1 --2- 172.21.15.63:0/2818067853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1f815cb50 0x7fc1f81f2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42828/0 (socket says 172.21.15.63:42828) 2024-05-28T13:58:00.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1efe00640 1 --2- 172.21.15.63:0/2818067853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f815c770 0x7fc1f81e50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:00.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1f7400640 1 -- 172.21.15.63:0/2818067853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f80d11d0 msgr2=0x7fc1f8159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-28T13:58:00.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1f7400640 1 -- 172.21.15.63:0/2818067853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f80d11d0 msgr2=0x7fc1f8159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-28T13:58:00.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1f7400640 1 --2- 172.21.15.63:0/2818067853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f80d11d0 0x7fc1f8159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:58:00.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1f7400640 1 --2- 172.21.15.63:0/2818067853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f80d11d0 0x7fc1f8159fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:58:00.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1efe00640 1 -- 172.21.15.63:0/2818067853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f80d11d0 msgr2=0x7fc1f8159fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-28T13:58:00.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1efe00640 1 --2- 172.21.15.63:0/2818067853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f80d11d0 0x7fc1f8159fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:00.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1efe00640 1 -- 172.21.15.63:0/2818067853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1f815cb50 msgr2=0x7fc1f81f2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:00.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1efe00640 1 --2- 172.21.15.63:0/2818067853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1f815cb50 0x7fc1f81f2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:00.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1efe00640 1 -- 172.21.15.63:0/2818067853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f80ca3c0 con 0x7fc1f815c770 2024-05-28T13:58:00.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1f7e00640 1 --2- 172.21.15.63:0/2818067853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1f815cb50 0x7fc1f81f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:00.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1efe00640 1 --2- 172.21.15.63:0/2818067853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f815c770 0x7fc1f81e50d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0007fd0 tx=0x7fc1e0001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:00.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1ef400640 1 -- 172.21.15.63:0/2818067853 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0017070 con 0x7fc1f815c770 2024-05-28T13:58:00.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1ef400640 1 -- 172.21.15.63:0/2818067853 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e0005cf0 con 0x7fc1f815c770 2024-05-28T13:58:00.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1ef400640 1 -- 172.21.15.63:0/2818067853 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0007310 con 0x7fc1f815c770 2024-05-28T13:58:00.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1fe000640 1 -- 172.21.15.63:0/2818067853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f80ca6a0 con 0x7fc1f815c770 2024-05-28T13:58:00.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1fe000640 1 -- 172.21.15.63:0/2818067853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f80cac60 con 0x7fc1f815c770 2024-05-28T13:58:00.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1ef400640 1 -- 172.21.15.63:0/2818067853 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc1e001a070 con 0x7fc1f815c770 2024-05-28T13:58:00.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1ef400640 1 -- 172.21.15.63:0/2818067853 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fc1e00074b0 con 0x7fc1f815c770 2024-05-28T13:58:00.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.163+0000 7fc1fe000640 1 -- 172.21.15.63:0/2818067853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f80a04d0 con 0x7fc1f815c770 2024-05-28T13:58:00.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.167+0000 7fc1ef400640 1 -- 172.21.15.63:0/2818067853 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc1e0021020 con 0x7fc1f815c770 2024-05-28T13:58:00.169 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:00.183+0000 7fc74f862180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-28T13:58:00.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.235+0000 7fc1fe000640 1 -- 172.21.15.63:0/2818067853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd new", "uuid": "91586b96-af01-4370-b64c-71ff4d46a7cb", "id": 7} v 0) -- 0x7fc1f804fdd0 con 0x7fc1f815c770 2024-05-28T13:58:00.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.239+0000 7fc1ef400640 1 -- 172.21.15.63:0/2818067853 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "91586b96-af01-4370-b64c-71ff4d46a7cb", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fc1e001c440 con 0x7fc1f815c770 2024-05-28T13:58:00.224 INFO:teuthology.orchestra.run.smithi063.stdout:7 2024-05-28T13:58:00.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.239+0000 7fc1fe000640 1 -- 172.21.15.63:0/2818067853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f815c770 msgr2=0x7fc1f81e50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:00.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.239+0000 7fc1fe000640 1 --2- 172.21.15.63:0/2818067853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f815c770 0x7fc1f81e50d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0007fd0 tx=0x7fc1e0001e80 comp rx=0 tx=0).stop 2024-05-28T13:58:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.239+0000 7fc1fe000640 1 -- 172.21.15.63:0/2818067853 shutdown_connections 2024-05-28T13:58:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.239+0000 7fc1fe000640 1 --2- 172.21.15.63:0/2818067853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1f815cb50 0x7fc1f81f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.239+0000 7fc1fe000640 1 --2- 172.21.15.63:0/2818067853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1f815c770 0x7fc1f81e50d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.239+0000 7fc1fe000640 1 --2- 172.21.15.63:0/2818067853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1f80d11d0 0x7fc1f8159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.239+0000 7fc1fe000640 1 -- 172.21.15.63:0/2818067853 >> 172.21.15.63:0/2818067853 conn(0x7fc1f80b3fd0 msgr2=0x7fc1f80c6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.239+0000 7fc1fe000640 1 -- 172.21.15.63:0/2818067853 shutdown_connections 2024-05-28T13:58:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:00.239+0000 7fc1fe000640 1 -- 172.21.15.63:0/2818067853 wait complete. 2024-05-28T13:58:00.244 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-28T13:58:00.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-28T13:58:00.245 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:00.259+0000 7fc74f862180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-28T13:58:00.248 INFO:tasks.ceph.osd.0:Started 2024-05-28T13:58:00.248 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-28T13:58:00.248 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-28T13:58:00.251 INFO:tasks.ceph.osd.1:Started 2024-05-28T13:58:00.252 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-28T13:58:00.252 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-28T13:58:00.255 INFO:tasks.ceph.osd.2:Started 2024-05-28T13:58:00.255 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-28T13:58:00.255 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-28T13:58:00.259 INFO:tasks.ceph.osd.3:Started 2024-05-28T13:58:00.259 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-28T13:58:00.259 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-28T13:58:00.262 INFO:tasks.ceph.osd.4:Started 2024-05-28T13:58:00.262 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-28T13:58:00.262 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-28T13:58:00.267 INFO:tasks.ceph.osd.5:Started 2024-05-28T13:58:00.267 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-28T13:58:00.267 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-28T13:58:00.270 INFO:tasks.ceph.osd.6:Started 2024-05-28T13:58:00.270 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-28T13:58:00.270 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-28T13:58:00.274 INFO:tasks.ceph.osd.7:Started 2024-05-28T13:58:00.275 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:00.329 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:00.343+0000 7fc74f862180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-28T13:58:00.440 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:00.455+0000 7fc74f862180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-28T13:58:00.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.490+0000 7f1a10e00640 1 Processor -- start 2024-05-28T13:58:00.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.490+0000 7f1a10e00640 1 -- start start 2024-05-28T13:58:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.490+0000 7f1a10e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1a0c0afa50 0x7f1a0c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.490+0000 7f1a10e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a0c15cb90 con 0x7f1a0c156a70 2024-05-28T13:58:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.490+0000 7f1a10e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a0c15ce50 con 0x7f1a0c156e50 2024-05-28T13:58:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.490+0000 7f1a10e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1a0c15d130 con 0x7f1a0c0afa50 2024-05-28T13:58:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.490+0000 7f1a01400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1a0c0afa50 0x7f1a0c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.494+0000 7f1a09e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1a0c156a70 0x7f1a0c0af340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50624/0 (socket says 172.21.15.42:50624) 2024-05-28T13:58:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.494+0000 7f1a09e00640 1 -- 172.21.15.42:0/2066565958 learned_addr learned my addr 172.21.15.42:0/2066565958 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.494+0000 7f1a01400640 1 -- 172.21.15.42:0/2066565958 >> v1:172.21.15.63:6789/0 conn(0x7f1a0c156e50 legacy=0x7f1a0c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.494+0000 7f1a01400640 1 -- 172.21.15.42:0/2066565958 >> v1:172.21.15.42:6789/0 conn(0x7f1a0c156a70 legacy=0x7f1a0c0af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.494+0000 7f1a01400640 1 -- 172.21.15.42:0/2066565958 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c15d410 con 0x7f1a0c0afa50 2024-05-28T13:58:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.494+0000 7f1a01400640 1 --2- 172.21.15.42:0/2066565958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0c0afa50 0x7f1a0c06dd60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009d90 tx=0x7f19f401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c6e739c9d3e8eb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:00.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.494+0000 7f1a09400640 1 -- 172.21.15.42:0/2066565958 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f40b2070 con 0x7f1a0c0afa50 2024-05-28T13:58:00.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.494+0000 7f1a09400640 1 -- 172.21.15.42:0/2066565958 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19f40a5a50 con 0x7f1a0c0afa50 2024-05-28T13:58:00.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.494+0000 7f1a09400640 1 -- 172.21.15.42:0/2066565958 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f40a5e20 con 0x7f1a0c0afa50 2024-05-28T13:58:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.498+0000 7f1a10e00640 1 -- 172.21.15.42:0/2066565958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0c0afa50 msgr2=0x7f1a0c06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.498+0000 7f1a10e00640 1 --2- 172.21.15.42:0/2066565958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0c0afa50 0x7f1a0c06dd60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009d90 tx=0x7f19f401c970 comp rx=0 tx=0).stop 2024-05-28T13:58:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.498+0000 7f1a10e00640 1 -- 172.21.15.42:0/2066565958 shutdown_connections 2024-05-28T13:58:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.498+0000 7f1a10e00640 1 --2- 172.21.15.42:0/2066565958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0c0afa50 0x7f1a0c06dd60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.498+0000 7f1a10e00640 1 -- 172.21.15.42:0/2066565958 >> 172.21.15.42:0/2066565958 conn(0x7f1a0c06bdf0 msgr2=0x7f1a0c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.498+0000 7f1a10e00640 1 -- 172.21.15.42:0/2066565958 shutdown_connections 2024-05-28T13:58:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.498+0000 7f1a10e00640 1 -- 172.21.15.42:0/2066565958 wait complete. 2024-05-28T13:58:00.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.498+0000 7f1a10e00640 1 Processor -- start 2024-05-28T13:58:00.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.498+0000 7f1a10e00640 1 -- start start 2024-05-28T13:58:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a10e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a0c0afa50 0x7f1a0c15e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a10e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0c156a70 0x7f1a0c147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a10e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a0c156e50 0x7f1a0c147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a10e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c160f90 con 0x7f1a0c156a70 2024-05-28T13:58:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a10e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c160e10 con 0x7f1a0c0afa50 2024-05-28T13:58:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a10e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0c16a530 con 0x7f1a0c156e50 2024-05-28T13:58:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a01400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0c156a70 0x7f1a0c147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a09e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a0c0afa50 0x7f1a0c15e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a01400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0c156a70 0x7f1a0c147400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40040/0 (socket says 172.21.15.42:40040) 2024-05-28T13:58:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a01400640 1 -- 172.21.15.42:0/2191865400 learned_addr learned my addr 172.21.15.42:0/2191865400 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a0a800640 1 --2- 172.21.15.42:0/2191865400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a0c156e50 0x7f1a0c147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a0a800640 1 -- 172.21.15.42:0/2191865400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a0c156e50 msgr2=0x7f1a0c147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-28T13:58:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a0a800640 1 -- 172.21.15.42:0/2191865400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a0c156e50 msgr2=0x7f1a0c147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-28T13:58:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a0a800640 1 --2- 172.21.15.42:0/2191865400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a0c156e50 0x7f1a0c147940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:58:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a0a800640 1 --2- 172.21.15.42:0/2191865400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a0c156e50 0x7f1a0c147940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:58:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a01400640 1 -- 172.21.15.42:0/2191865400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a0c156e50 msgr2=0x7f1a0c147940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-28T13:58:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a01400640 1 --2- 172.21.15.42:0/2191865400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a0c156e50 0x7f1a0c147940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a01400640 1 -- 172.21.15.42:0/2191865400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a0c0afa50 msgr2=0x7f1a0c15e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a01400640 1 --2- 172.21.15.42:0/2191865400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a0c0afa50 0x7f1a0c15e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a01400640 1 -- 172.21.15.42:0/2191865400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c14a9e0 con 0x7f1a0c156a70 2024-05-28T13:58:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a09e00640 1 --2- 172.21.15.42:0/2191865400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a0c0afa50 0x7f1a0c15e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a01400640 1 --2- 172.21.15.42:0/2191865400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0c156a70 0x7f1a0c147400 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f19f4019580 tx=0x7f19f4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a02a00640 1 -- 172.21.15.42:0/2191865400 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f40b2050 con 0x7f1a0c156a70 2024-05-28T13:58:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a02a00640 1 -- 172.21.15.42:0/2191865400 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19f40be070 con 0x7f1a0c156a70 2024-05-28T13:58:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a02a00640 1 -- 172.21.15.42:0/2191865400 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f40b9450 con 0x7f1a0c156a70 2024-05-28T13:58:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a10e00640 1 -- 172.21.15.42:0/2191865400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0c154440 con 0x7f1a0c156a70 2024-05-28T13:58:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a10e00640 1 -- 172.21.15.42:0/2191865400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a0c154a00 con 0x7f1a0c156a70 2024-05-28T13:58:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a02a00640 1 -- 172.21.15.42:0/2191865400 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f19f40b95f0 con 0x7f1a0c156a70 2024-05-28T13:58:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a02a00640 1 -- 172.21.15.42:0/2191865400 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f19f40b7400 con 0x7f1a0c156a70 2024-05-28T13:58:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.502+0000 7f1a10e00640 1 -- 172.21.15.42:0/2191865400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a0c068a90 con 0x7f1a0c156a70 2024-05-28T13:58:00.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.506+0000 7f1a02a00640 1 -- 172.21.15.42:0/2191865400 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f19f4016560 con 0x7f1a0c156a70 2024-05-28T13:58:00.504 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:00.519+0000 7fc74f862180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-28T13:58:00.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.562+0000 7f1a10e00640 1 -- 172.21.15.42:0/2191865400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a0c0b5160 con 0x7f1a0c156a70 2024-05-28T13:58:00.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.566+0000 7f1a02a00640 1 -- 172.21.15.42:0/2191865400 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f19f40b7600 con 0x7f1a0c156a70 2024-05-28T13:58:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:00.553 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":10,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:00.239616+0000","last_up_change":"0.000000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:00.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.566+0000 7f1a10e00640 1 -- 172.21.15.42:0/2191865400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0c156a70 msgr2=0x7f1a0c147400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:00.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.566+0000 7f1a10e00640 1 --2- 172.21.15.42:0/2191865400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0c156a70 0x7f1a0c147400 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f19f4019580 tx=0x7f19f4019710 comp rx=0 tx=0).stop 2024-05-28T13:58:00.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.566+0000 7f1a10e00640 1 -- 172.21.15.42:0/2191865400 shutdown_connections 2024-05-28T13:58:00.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.566+0000 7f1a10e00640 1 --2- 172.21.15.42:0/2191865400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a0c156e50 0x7f1a0c147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:00.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.566+0000 7f1a10e00640 1 --2- 172.21.15.42:0/2191865400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0c156a70 0x7f1a0c147400 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:00.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.566+0000 7f1a10e00640 1 --2- 172.21.15.42:0/2191865400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a0c0afa50 0x7f1a0c15e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:00.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.566+0000 7f1a10e00640 1 -- 172.21.15.42:0/2191865400 >> 172.21.15.42:0/2191865400 conn(0x7f1a0c06bdf0 msgr2=0x7f1a0c06f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:00.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.566+0000 7f1a10e00640 1 -- 172.21.15.42:0/2191865400 shutdown_connections 2024-05-28T13:58:00.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:00.566+0000 7f1a10e00640 1 -- 172.21.15.42:0/2191865400 wait complete. 2024-05-28T13:58:00.572 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-28T13:58:00.572 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-28T13:58:00.597 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:00.611+0000 7fc74f862180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-28T13:58:00.658 INFO:tasks.ceph.mgr.y.smithi042.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-28T13:58:00.658 INFO:tasks.ceph.mgr.y.smithi042.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-28T13:58:00.658 INFO:tasks.ceph.mgr.y.smithi042.stderr: from numpy import show_config as show_numpy_config 2024-05-28T13:58:00.663 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:00.675+0000 7fc74f862180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-28T13:58:00.708 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:58:00.718+0000 7f07f8d0b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-28T13:58:00.838 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:00.851+0000 7fc74f862180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-28T13:58:00.864 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:58:00.874+0000 7f07f8d0b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-28T13:58:00.904 INFO:tasks.ceph.osd.4.smithi063.stderr:2024-05-28T13:58:00.919+0000 7fcc1848c7c0 -1 Falling back to public interface 2024-05-28T13:58:00.913 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:00.927+0000 7fc74f862180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-28T13:58:00.918 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-05-28T13:58:00.930+0000 7f10b8f157c0 -1 Falling back to public interface 2024-05-28T13:58:00.933 INFO:tasks.ceph.osd.5.smithi063.stderr:2024-05-28T13:58:00.947+0000 7f1bf43757c0 -1 Falling back to public interface 2024-05-28T13:58:00.933 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-05-28T13:58:00.946+0000 7fca881e17c0 -1 Falling back to public interface 2024-05-28T13:58:00.940 INFO:tasks.ceph.osd.7.smithi063.stderr:2024-05-28T13:58:00.955+0000 7f36d70b87c0 -1 Falling back to public interface 2024-05-28T13:58:00.946 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-05-28T13:58:00.958+0000 7fa780e2b7c0 -1 Falling back to public interface 2024-05-28T13:58:00.964 INFO:tasks.ceph.osd.6.smithi063.stderr:2024-05-28T13:58:00.979+0000 7f3911ab87c0 -1 Falling back to public interface 2024-05-28T13:58:00.973 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-05-28T13:58:00.986+0000 7fa6db88e7c0 -1 Falling back to public interface 2024-05-28T13:58:01.055 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:58:01.066+0000 7f07f8d0b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-28T13:58:01.132 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:58:01.142+0000 7f07f8d0b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-28T13:58:01.291 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:58:01.302+0000 7f07f8d0b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-28T13:58:01.379 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:01.510 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91ec200640 1 Processor -- start 2024-05-28T13:58:01.510 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91ec200640 1 -- start start 2024-05-28T13:58:01.511 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91ec200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f91e409b120 0x7f91e406dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:01.512 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91ec200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f91e40b14c0 0x7f91e40cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:01.512 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91ec200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91e415efd0 con 0x7f91e415ebf0 2024-05-28T13:58:01.512 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91ec200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f91e406e810 con 0x7f91e40b14c0 2024-05-28T13:58:01.512 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91ec200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f91e40ccf70 con 0x7f91e409b120 2024-05-28T13:58:01.512 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91e9800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f91e415ebf0 0x7f91e409aa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50634/0 (socket says 172.21.15.42:50634) 2024-05-28T13:58:01.512 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91e9800640 1 -- 172.21.15.42:0/2566851493 learned_addr learned my addr 172.21.15.42:0/2566851493 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:01.512 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91ea200640 1 --2- 172.21.15.42:0/2566851493 >> v2:172.21.15.42:3300/0 conn(0x7f91e40b14c0 0x7f91e40cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:01.512 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91e8e00640 1 --2- 172.21.15.42:0/2566851493 >> v2:172.21.15.63:3300/0 conn(0x7f91e409b120 0x7f91e406dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:01.513 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91ea200640 1 -- 172.21.15.42:0/2566851493 >> v2:172.21.15.63:3300/0 conn(0x7f91e409b120 msgr2=0x7f91e406dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:01.513 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91ea200640 1 --2- 172.21.15.42:0/2566851493 >> v2:172.21.15.63:3300/0 conn(0x7f91e409b120 0x7f91e406dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:01.513 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91ea200640 1 -- 172.21.15.42:0/2566851493 >> v1:172.21.15.42:6789/0 conn(0x7f91e415ebf0 legacy=0x7f91e409aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:01.513 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.522+0000 7f91ea200640 1 -- 172.21.15.42:0/2566851493 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e406e250 con 0x7f91e40b14c0 2024-05-28T13:58:01.515 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.526+0000 7f91ea200640 1 --2- 172.21.15.42:0/2566851493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e40b14c0 0x7f91e40cd400 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f91e40b3f40 tx=0x7f91dc01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faeee550486a4e4d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:01.515 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.526+0000 7f91e3e00640 1 -- 172.21.15.42:0/2566851493 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0b2070 con 0x7f91e40b14c0 2024-05-28T13:58:01.515 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.526+0000 7f91e3e00640 1 -- 172.21.15.42:0/2566851493 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91dc0a57c0 con 0x7f91e40b14c0 2024-05-28T13:58:01.515 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.526+0000 7f91e3e00640 1 -- 172.21.15.42:0/2566851493 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0a5b90 con 0x7f91e40b14c0 2024-05-28T13:58:01.516 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.526+0000 7f91ec200640 1 -- 172.21.15.42:0/2566851493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e40b14c0 msgr2=0x7f91e40cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:01.516 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.526+0000 7f91ec200640 1 --2- 172.21.15.42:0/2566851493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e40b14c0 0x7f91e40cd400 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f91e40b3f40 tx=0x7f91dc01c810 comp rx=0 tx=0).stop 2024-05-28T13:58:01.516 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.526+0000 7f91ec200640 1 -- 172.21.15.42:0/2566851493 shutdown_connections 2024-05-28T13:58:01.516 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.526+0000 7f91ec200640 1 --2- 172.21.15.42:0/2566851493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e40b14c0 0x7f91e40cd400 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:01.516 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.526+0000 7f91ec200640 1 --2- 172.21.15.42:0/2566851493 >> v2:172.21.15.63:3300/0 conn(0x7f91e409b120 0x7f91e406dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:01.516 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.526+0000 7f91ec200640 1 -- 172.21.15.42:0/2566851493 >> 172.21.15.42:0/2566851493 conn(0x7f91e40b3f70 msgr2=0x7f91e40b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:01.516 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.526+0000 7f91ec200640 1 -- 172.21.15.42:0/2566851493 shutdown_connections 2024-05-28T13:58:01.516 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.526+0000 7f91ec200640 1 -- 172.21.15.42:0/2566851493 wait complete. 2024-05-28T13:58:01.517 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ec200640 1 Processor -- start 2024-05-28T13:58:01.517 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ec200640 1 -- start start 2024-05-28T13:58:01.520 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ec200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e409b120 0x7f91e4152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:01.520 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ec200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91e40b14c0 0x7f91e41534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:01.520 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ec200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91e415ebf0 0x7f91e4153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:01.520 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ec200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f91e40ce2d0 con 0x7f91e409b120 2024-05-28T13:58:01.520 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ec200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f91e40ce150 con 0x7f91e40b14c0 2024-05-28T13:58:01.521 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ec200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f91e40ce450 con 0x7f91e415ebf0 2024-05-28T13:58:01.521 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e409b120 0x7f91e4152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:01.521 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e409b120 0x7f91e4152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40058/0 (socket says 172.21.15.42:40058) 2024-05-28T13:58:01.521 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e9800640 1 -- 172.21.15.42:0/130969114 learned_addr learned my addr 172.21.15.42:0/130969114 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:01.521 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e8e00640 1 --2- 172.21.15.42:0/130969114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91e40b14c0 0x7f91e41534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:01.521 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ea200640 1 --2- 172.21.15.42:0/130969114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91e415ebf0 0x7f91e4153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:01.521 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ea200640 1 -- 172.21.15.42:0/130969114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91e415ebf0 msgr2=0x7f91e4153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-28T13:58:01.521 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ea200640 1 -- 172.21.15.42:0/130969114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91e415ebf0 msgr2=0x7f91e4153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-28T13:58:01.521 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ea200640 1 --2- 172.21.15.42:0/130969114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91e415ebf0 0x7f91e4153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-28T13:58:01.521 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ea200640 1 --2- 172.21.15.42:0/130969114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91e415ebf0 0x7f91e4153a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-28T13:58:01.521 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e9800640 1 -- 172.21.15.42:0/130969114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91e415ebf0 msgr2=0x7f91e4153a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-28T13:58:01.521 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e9800640 1 --2- 172.21.15.42:0/130969114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91e415ebf0 0x7f91e4153a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:01.521 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e9800640 1 -- 172.21.15.42:0/130969114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91e40b14c0 msgr2=0x7f91e41534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:01.521 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e9800640 1 --2- 172.21.15.42:0/130969114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91e40b14c0 0x7f91e41534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:01.522 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e9800640 1 -- 172.21.15.42:0/130969114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e40b7950 con 0x7f91e409b120 2024-05-28T13:58:01.522 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e9800640 1 --2- 172.21.15.42:0/130969114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e409b120 0x7f91e4152cc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f91d800c370 tx=0x7f91d800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:01.522 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e1600640 1 -- 172.21.15.42:0/130969114 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8017070 con 0x7f91e409b120 2024-05-28T13:58:01.522 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e1600640 1 -- 172.21.15.42:0/130969114 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d8005cf0 con 0x7f91e409b120 2024-05-28T13:58:01.522 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ec200640 1 -- 172.21.15.42:0/130969114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e40b7c30 con 0x7f91e409b120 2024-05-28T13:58:01.522 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e1600640 1 -- 172.21.15.42:0/130969114 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d80072f0 con 0x7f91e409b120 2024-05-28T13:58:01.522 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91ec200640 1 -- 172.21.15.42:0/130969114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e415bf40 con 0x7f91e409b120 2024-05-28T13:58:01.522 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e1600640 1 -- 172.21.15.42:0/130969114 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f91d801a070 con 0x7f91e409b120 2024-05-28T13:58:01.522 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.530+0000 7f91e1600640 1 -- 172.21.15.42:0/130969114 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f91d8007490 con 0x7f91e409b120 2024-05-28T13:58:01.522 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.534+0000 7f91ec200640 1 -- 172.21.15.42:0/130969114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e404fcf0 con 0x7f91e409b120 2024-05-28T13:58:01.523 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.534+0000 7f91e1600640 1 -- 172.21.15.42:0/130969114 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f91d8021070 con 0x7f91e409b120 2024-05-28T13:58:01.546 INFO:tasks.ceph.mgr.x.smithi063.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-28T13:58:01.546 INFO:tasks.ceph.mgr.x.smithi063.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-28T13:58:01.546 INFO:tasks.ceph.mgr.x.smithi063.stderr: from numpy import show_config as show_numpy_config 2024-05-28T13:58:01.583 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.594+0000 7f91ec200640 1 -- 172.21.15.42:0/130969114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91e40cf090 con 0x7f91e409b120 2024-05-28T13:58:01.584 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.594+0000 7f91e1600640 1 -- 172.21.15.42:0/130969114 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f91d801c440 con 0x7f91e409b120 2024-05-28T13:58:01.584 INFO:teuthology.misc.health.smithi042.stdout: 2024-05-28T13:58:01.584 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":10,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:00.239616+0000","last_up_change":"0.000000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:01.585 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.598+0000 7f91ec200640 1 -- 172.21.15.42:0/130969114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e409b120 msgr2=0x7f91e4152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:01.585 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.598+0000 7f91ec200640 1 --2- 172.21.15.42:0/130969114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e409b120 0x7f91e4152cc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f91d800c370 tx=0x7f91d800c830 comp rx=0 tx=0).stop 2024-05-28T13:58:01.585 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.598+0000 7f91ec200640 1 -- 172.21.15.42:0/130969114 shutdown_connections 2024-05-28T13:58:01.585 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.598+0000 7f91ec200640 1 --2- 172.21.15.42:0/130969114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91e415ebf0 0x7f91e4153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:01.585 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.598+0000 7f91ec200640 1 --2- 172.21.15.42:0/130969114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91e40b14c0 0x7f91e41534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:01.585 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.598+0000 7f91ec200640 1 --2- 172.21.15.42:0/130969114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e409b120 0x7f91e4152cc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:01.585 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.598+0000 7f91ec200640 1 -- 172.21.15.42:0/130969114 >> 172.21.15.42:0/130969114 conn(0x7f91e40b3f70 msgr2=0x7f91e40af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:01.586 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.598+0000 7f91ec200640 1 -- 172.21.15.42:0/130969114 shutdown_connections 2024-05-28T13:58:01.586 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:01.598+0000 7f91ec200640 1 -- 172.21.15.42:0/130969114 wait complete. 2024-05-28T13:58:01.599 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:01.615+0000 7fc74f862180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-28T13:58:01.601 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-28T13:58:01.757 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:01.771+0000 7fc74f862180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-28T13:58:01.947 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:01.963+0000 7fc74f862180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-28T13:58:02.001 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:58:02.014+0000 7f07f8d0b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-28T13:58:02.035 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:02.051+0000 7fc74f862180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-28T13:58:02.093 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:58:02.106+0000 7f07f8d0b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-28T13:58:02.194 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:02.207+0000 7fc74f862180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-28T13:58:02.310 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:58:02.322+0000 7f07f8d0b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-28T13:58:02.399 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:58:02.410+0000 7f07f8d0b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-28T13:58:02.750 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:58:02.762+0000 7f07f8d0b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-28T13:58:02.825 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:58:02.838+0000 7f07f8d0b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-28T13:58:02.915 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:02.931+0000 7fc74f862180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-28T13:58:03.006 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:03.019+0000 7fc74f862180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-28T13:58:03.222 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:03.235+0000 7fc74f862180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-28T13:58:03.310 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:03.323+0000 7fc74f862180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-28T13:58:03.661 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:03.675+0000 7fc74f862180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-28T13:58:03.737 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-28T13:58:03.751+0000 7fc74f862180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-28T13:58:03.844 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:58:03.854+0000 7f079ee00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-28T13:58:03.844 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:58:03.854+0000 7f079ee00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-28T13:58:04.630 INFO:tasks.ceph.osd.5.smithi063.stderr:2024-05-28T13:58:04.643+0000 7f1bf43757c0 -1 osd.5 0 log_to_monitors true 2024-05-28T13:58:04.719 INFO:tasks.ceph.osd.7.smithi063.stderr:2024-05-28T13:58:04.731+0000 7f36d70b87c0 -1 osd.7 0 log_to_monitors true 2024-05-28T13:58:04.812 INFO:tasks.ceph.osd.4.smithi063.stderr:2024-05-28T13:58:04.827+0000 7fcc1848c7c0 -1 osd.4 0 log_to_monitors true 2024-05-28T13:58:04.921 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-05-28T13:58:04.934+0000 7fa6db88e7c0 -1 osd.3 0 log_to_monitors true 2024-05-28T13:58:04.959 INFO:tasks.ceph.osd.6.smithi063.stderr:2024-05-28T13:58:04.971+0000 7f3911ab87c0 -1 osd.6 0 log_to_monitors true 2024-05-28T13:58:04.969 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-05-28T13:58:04.982+0000 7fca881e17c0 -1 osd.0 0 log_to_monitors true 2024-05-28T13:58:04.989 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-05-28T13:58:05.002+0000 7fa780e2b7c0 -1 osd.2 0 log_to_monitors true 2024-05-28T13:58:05.128 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-05-28T13:58:05.142+0000 7f10b8f157c0 -1 osd.1 0 log_to_monitors true 2024-05-28T13:58:08.166 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-05-28T13:58:08.178+0000 7fa77b200640 -1 osd.2 0 waiting for initial osdmap 2024-05-28T13:58:08.167 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-05-28T13:58:08.178+0000 7fa6d6800640 -1 osd.3 0 waiting for initial osdmap 2024-05-28T13:58:08.188 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-05-28T13:58:08.198+0000 7fca83000640 -1 osd.0 0 waiting for initial osdmap 2024-05-28T13:58:08.194 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-05-28T13:58:08.206+0000 7f10b3e00640 -1 osd.1 0 waiting for initial osdmap 2024-05-28T13:58:08.198 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-05-28T13:58:08.210+0000 7fa771200640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-28T13:58:08.209 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-05-28T13:58:08.226+0000 7fa6cc800640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-28T13:58:08.221 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-05-28T13:58:08.238+0000 7fca79000640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-28T13:58:08.233 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-05-28T13:58:08.250+0000 7f10a9e00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-28T13:58:08.410 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:08.543 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.558+0000 7f186fe00640 1 Processor -- start 2024-05-28T13:58:08.543 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.558+0000 7f186fe00640 1 -- start start 2024-05-28T13:58:08.544 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.558+0000 7f186fe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f187015c750 0x7f18700d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:08.544 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.558+0000 7f186fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18700cf440 0x7f18700ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:08.544 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.558+0000 7f186fe00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18700d1d10 con 0x7f18700cc970 2024-05-28T13:58:08.544 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.558+0000 7f186fe00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f18700d1290 con 0x7f187015c750 2024-05-28T13:58:08.544 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.558+0000 7f186fe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f18700d1ff0 con 0x7f18700cf440 2024-05-28T13:58:08.545 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.558+0000 7f186f400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18700cf440 0x7f18700ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:08.545 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.558+0000 7f186f400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18700cf440 0x7f18700ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:37346/0 (socket says 172.21.15.42:37346) 2024-05-28T13:58:08.545 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.558+0000 7f186f400640 1 -- 172.21.15.42:0/468680594 learned_addr learned my addr 172.21.15.42:0/468680594 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:08.545 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.562+0000 7f186ea00640 1 --2- 172.21.15.42:0/468680594 >> v2:172.21.15.63:3300/0 conn(0x7f187015c750 0x7f18700d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:08.545 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.562+0000 7f186ea00640 1 -- 172.21.15.42:0/468680594 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18700cf440 msgr2=0x7f18700ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:08.545 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.562+0000 7f186ea00640 1 --2- 172.21.15.42:0/468680594 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18700cf440 0x7f18700ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:08.545 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.562+0000 7f186ea00640 1 -- 172.21.15.42:0/468680594 >> v1:172.21.15.63:6789/0 conn(0x7f18700cc970 legacy=0x7f18700ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:08.545 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.562+0000 7f186ea00640 1 -- 172.21.15.42:0/468680594 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18700d22d0 con 0x7f187015c750 2024-05-28T13:58:08.548 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.562+0000 7f186ea00640 1 --2- 172.21.15.42:0/468680594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f187015c750 0x7f18700d2ad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f18600097c0 tx=0x7f186001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa38ee58767001e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:08.548 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.562+0000 7f186e000640 1 -- 172.21.15.42:0/468680594 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18600b0070 con 0x7f187015c750 2024-05-28T13:58:08.548 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.562+0000 7f186e000640 1 -- 172.21.15.42:0/468680594 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1860004230 con 0x7f187015c750 2024-05-28T13:58:08.548 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.562+0000 7f186e000640 1 -- 172.21.15.42:0/468680594 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860003620 con 0x7f187015c750 2024-05-28T13:58:08.549 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 -- 172.21.15.42:0/468680594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f187015c750 msgr2=0x7f18700d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:08.549 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 --2- 172.21.15.42:0/468680594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f187015c750 0x7f18700d2ad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f18600097c0 tx=0x7f186001c4a0 comp rx=0 tx=0).stop 2024-05-28T13:58:08.549 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 -- 172.21.15.42:0/468680594 shutdown_connections 2024-05-28T13:58:08.549 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 --2- 172.21.15.42:0/468680594 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18700cf440 0x7f18700ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:08.550 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 --2- 172.21.15.42:0/468680594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f187015c750 0x7f18700d2ad0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:08.550 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 -- 172.21.15.42:0/468680594 >> 172.21.15.42:0/468680594 conn(0x7f187006b9e0 msgr2=0x7f187006bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:08.550 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 -- 172.21.15.42:0/468680594 shutdown_connections 2024-05-28T13:58:08.550 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 -- 172.21.15.42:0/468680594 wait complete. 2024-05-28T13:58:08.550 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 Processor -- start 2024-05-28T13:58:08.551 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 -- start start 2024-05-28T13:58:08.552 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18700cc970 0x7f187015a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:08.552 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18700cf440 0x7f18700cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:08.553 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f187015c750 0x7f18700b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:08.553 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f18700d3190 con 0x7f18700cc970 2024-05-28T13:58:08.553 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18700d3010 con 0x7f18700cf440 2024-05-28T13:58:08.553 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186fe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f18700b5980 con 0x7f187015c750 2024-05-28T13:58:08.553 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18700cc970 0x7f187015a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:08.553 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18700cc970 0x7f187015a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54244/0 (socket says 172.21.15.42:54244) 2024-05-28T13:58:08.553 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186ea00640 1 -- 172.21.15.42:0/935189739 learned_addr learned my addr 172.21.15.42:0/935189739 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:08.553 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f1866000640 1 --2- 172.21.15.42:0/935189739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18700cf440 0x7f18700cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:08.553 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186f400640 1 --2- 172.21.15.42:0/935189739 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f187015c750 0x7f18700b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:08.553 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186ea00640 1 -- 172.21.15.42:0/935189739 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f187015c750 msgr2=0x7f18700b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:08.553 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186ea00640 1 --2- 172.21.15.42:0/935189739 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f187015c750 0x7f18700b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:08.553 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186ea00640 1 -- 172.21.15.42:0/935189739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18700cf440 msgr2=0x7f18700cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:08.554 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186ea00640 1 --2- 172.21.15.42:0/935189739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18700cf440 0x7f18700cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:08.554 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.566+0000 7f186ea00640 1 -- 172.21.15.42:0/935189739 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187015ade0 con 0x7f18700cc970 2024-05-28T13:58:08.554 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.570+0000 7f186ea00640 1 --2- 172.21.15.42:0/935189739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18700cc970 0x7f187015a8a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f18600ae040 tx=0x7f1860019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:08.555 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.570+0000 7f1867400640 1 -- 172.21.15.42:0/935189739 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18600b9020 con 0x7f18700cc970 2024-05-28T13:58:08.555 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.570+0000 7f1867400640 1 -- 172.21.15.42:0/935189739 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1860003df0 con 0x7f18700cc970 2024-05-28T13:58:08.555 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.570+0000 7f1867400640 1 -- 172.21.15.42:0/935189739 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18600b0040 con 0x7f18700cc970 2024-05-28T13:58:08.555 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.570+0000 7f186fe00640 1 -- 172.21.15.42:0/935189739 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18700c7650 con 0x7f18700cc970 2024-05-28T13:58:08.555 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.570+0000 7f186fe00640 1 -- 172.21.15.42:0/935189739 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18700c7c10 con 0x7f18700cc970 2024-05-28T13:58:08.555 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.570+0000 7f186fe00640 1 -- 172.21.15.42:0/935189739 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1870075df0 con 0x7f18700cc970 2024-05-28T13:58:08.555 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.570+0000 7f1867400640 1 -- 172.21.15.42:0/935189739 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7f18600bc020 con 0x7f18700cc970 2024-05-28T13:58:08.556 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.570+0000 7f1867400640 1 --2- 172.21.15.42:0/935189739 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f184c068320 0x7f184c06a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:08.556 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.570+0000 7f1867400640 1 -- 172.21.15.42:0/935189739 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f186011f810 con 0x7f18700cc970 2024-05-28T13:58:08.556 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.570+0000 7f1866000640 1 --2- 172.21.15.42:0/935189739 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f184c068320 0x7f184c06a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:08.556 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.574+0000 7f1866000640 1 --2- 172.21.15.42:0/935189739 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f184c068320 0x7f184c06a7e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f18700d2a60 tx=0x7f1858009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:08.561 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.578+0000 7f1867400640 1 -- 172.21.15.42:0/935189739 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f18600c3030 con 0x7f18700cc970 2024-05-28T13:58:08.725 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.742+0000 7f186fe00640 1 -- 172.21.15.42:0/935189739 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1870166af0 con 0x7f18700cc970 2024-05-28T13:58:08.726 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.742+0000 7f1867400640 1 -- 172.21.15.42:0/935189739 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f1860016090 con 0x7f18700cc970 2024-05-28T13:58:08.726 INFO:teuthology.misc.health.smithi042.stdout: 2024-05-28T13:58:08.726 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":12,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:06.503749+0000","last_up_change":"0.000000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:08.728 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.742+0000 7f186fe00640 1 -- 172.21.15.42:0/935189739 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f184c068320 msgr2=0x7f184c06a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:08.729 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.742+0000 7f186fe00640 1 --2- 172.21.15.42:0/935189739 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f184c068320 0x7f184c06a7e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f18700d2a60 tx=0x7f1858009040 comp rx=0 tx=0).stop 2024-05-28T13:58:08.729 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.742+0000 7f186fe00640 1 -- 172.21.15.42:0/935189739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18700cc970 msgr2=0x7f187015a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:08.729 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.742+0000 7f186fe00640 1 --2- 172.21.15.42:0/935189739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18700cc970 0x7f187015a8a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f18600ae040 tx=0x7f1860019240 comp rx=0 tx=0).stop 2024-05-28T13:58:08.729 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.742+0000 7f186fe00640 1 -- 172.21.15.42:0/935189739 shutdown_connections 2024-05-28T13:58:08.729 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.742+0000 7f186fe00640 1 --2- 172.21.15.42:0/935189739 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f184c068320 0x7f184c06a7e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:08.729 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.742+0000 7f186fe00640 1 --2- 172.21.15.42:0/935189739 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f187015c750 0x7f18700b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:08.729 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.742+0000 7f186fe00640 1 --2- 172.21.15.42:0/935189739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18700cf440 0x7f18700cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:08.729 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.742+0000 7f186fe00640 1 --2- 172.21.15.42:0/935189739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18700cc970 0x7f187015a8a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:08.729 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.742+0000 7f186fe00640 1 -- 172.21.15.42:0/935189739 >> 172.21.15.42:0/935189739 conn(0x7f187006b9e0 msgr2=0x7f187006bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:08.729 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.742+0000 7f186fe00640 1 -- 172.21.15.42:0/935189739 shutdown_connections 2024-05-28T13:58:08.729 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:08.742+0000 7f186fe00640 1 -- 172.21.15.42:0/935189739 wait complete. 2024-05-28T13:58:08.745 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-28T13:58:08.784 INFO:tasks.ceph.osd.5.smithi063.stderr:2024-05-28T13:58:08.800+0000 7f1befc00640 -1 osd.5 0 waiting for initial osdmap 2024-05-28T13:58:08.796 INFO:tasks.ceph.osd.7.smithi063.stderr:2024-05-28T13:58:08.812+0000 7f36d2000640 -1 osd.7 0 waiting for initial osdmap 2024-05-28T13:58:08.812 INFO:tasks.ceph.osd.5.smithi063.stderr:2024-05-28T13:58:08.828+0000 7f1be5200640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-28T13:58:08.824 INFO:tasks.ceph.osd.7.smithi063.stderr:2024-05-28T13:58:08.840+0000 7f36c8000640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-28T13:58:08.844 INFO:tasks.ceph.osd.4.smithi063.stderr:2024-05-28T13:58:08.860+0000 7fcc13400640 -1 osd.4 0 waiting for initial osdmap 2024-05-28T13:58:08.844 INFO:tasks.ceph.osd.6.smithi063.stderr:2024-05-28T13:58:08.860+0000 7f390ca00640 -1 osd.6 0 waiting for initial osdmap 2024-05-28T13:58:08.872 INFO:tasks.ceph.osd.4.smithi063.stderr:2024-05-28T13:58:08.888+0000 7fcc09400640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-28T13:58:08.872 INFO:tasks.ceph.osd.6.smithi063.stderr:2024-05-28T13:58:08.888+0000 7f3902a00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-28T13:58:15.552 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:15.697 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe88d600640 1 Processor -- start 2024-05-28T13:58:15.698 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe88d600640 1 -- start start 2024-05-28T13:58:15.699 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe88d600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe88815ebf0 0x7fe88806dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:15.699 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe88d600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe88815efd0 con 0x7fe8880b14c0 2024-05-28T13:58:15.699 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe88d600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe88806e250 con 0x7fe88809b120 2024-05-28T13:58:15.699 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe88d600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe88806e810 con 0x7fe88815ebf0 2024-05-28T13:58:15.699 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe87e000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe8880b14c0 0x7fe88809aa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42756/0 (socket says 172.21.15.42:42756) 2024-05-28T13:58:15.699 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe87e000640 1 -- 172.21.15.42:0/2771740142 learned_addr learned my addr 172.21.15.42:0/2771740142 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:15.701 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886a00640 1 --2- 172.21.15.42:0/2771740142 >> v2:172.21.15.63:3300/0 conn(0x7fe88815ebf0 0x7fe88806dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:15.701 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 -- 172.21.15.42:0/2771740142 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 807789529 0 0) 0x7fe88815efd0 con 0x7fe8880b14c0 2024-05-28T13:58:15.701 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 -- 172.21.15.42:0/2771740142 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe86c003530 con 0x7fe8880b14c0 2024-05-28T13:58:15.701 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 -- 172.21.15.42:0/2771740142 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 675062831 0 0) 0x7fe86c003530 con 0x7fe8880b14c0 2024-05-28T13:58:15.701 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 -- 172.21.15.42:0/2771740142 >> v2:172.21.15.63:3300/0 conn(0x7fe88815ebf0 msgr2=0x7fe88806dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:15.701 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 --2- 172.21.15.42:0/2771740142 >> v2:172.21.15.63:3300/0 conn(0x7fe88815ebf0 0x7fe88806dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:15.701 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 -- 172.21.15.42:0/2771740142 >> v1:172.21.15.63:6789/0 conn(0x7fe88809b120 legacy=0x7fe8880cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:15.701 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 -- 172.21.15.42:0/2771740142 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88806e530 con 0x7fe8880b14c0 2024-05-28T13:58:15.701 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 -- 172.21.15.42:0/2771740142 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fe870001680 con 0x7fe8880b14c0 2024-05-28T13:58:15.701 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886a00640 1 --2- 172.21.15.42:0/2771740142 >> v2:172.21.15.63:3300/0 conn(0x7fe88815ebf0 0x7fe88806dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:15.702 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 -- 172.21.15.42:0/2771740142 >> v1:172.21.15.42:6789/0 conn(0x7fe8880b14c0 legacy=0x7fe88809aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:15.702 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 --2- 172.21.15.42:0/2771740142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe86c0039c0 0x7fe86c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:15.702 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 --2- 172.21.15.42:0/2771740142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe86c004810 0x7fe86c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:15.702 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 --2- 172.21.15.42:0/2771740142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86c0052f0 0x7fe86c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:15.702 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 -- 172.21.15.42:0/2771740142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe88806e810 con 0x7fe86c0039c0 2024-05-28T13:58:15.702 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 -- 172.21.15.42:0/2771740142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c00c1c0 con 0x7fe86c004810 2024-05-28T13:58:15.702 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886000640 1 -- 172.21.15.42:0/2771740142 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe86c00c3c0 con 0x7fe86c0052f0 2024-05-28T13:58:15.702 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe886a00640 1 --2- 172.21.15.42:0/2771740142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe86c0039c0 0x7fe86c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:15.702 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.713+0000 7fe87e000640 1 --2- 172.21.15.42:0/2771740142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe86c004810 0x7fe86c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:15.702 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe887400640 1 --2- 172.21.15.42:0/2771740142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86c0052f0 0x7fe86c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:15.702 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe886a00640 1 -- 172.21.15.42:0/2771740142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86c0052f0 msgr2=0x7fe86c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:15.702 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe886a00640 1 --2- 172.21.15.42:0/2771740142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86c0052f0 0x7fe86c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:15.702 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe886a00640 1 -- 172.21.15.42:0/2771740142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe86c004810 msgr2=0x7fe86c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:15.703 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe886a00640 1 --2- 172.21.15.42:0/2771740142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe86c004810 0x7fe86c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:15.703 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe886a00640 1 -- 172.21.15.42:0/2771740142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c003530 con 0x7fe86c0039c0 2024-05-28T13:58:15.703 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe87e000640 1 --2- 172.21.15.42:0/2771740142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe86c004810 0x7fe86c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:15.703 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe887400640 1 -- 172.21.15.42:0/2771740142 reap_dead start 2024-05-28T13:58:15.703 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe886a00640 1 --2- 172.21.15.42:0/2771740142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe86c0039c0 0x7fe86c003de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe878009c80 tx=0x7fe87801cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cbeb33844eb911 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:15.703 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe886000640 1 -- 172.21.15.42:0/2771740142 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780b2070 con 0x7fe86c0039c0 2024-05-28T13:58:15.703 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe886000640 1 -- 172.21.15.42:0/2771740142 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8780a5650 con 0x7fe86c0039c0 2024-05-28T13:58:15.704 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe886000640 1 -- 172.21.15.42:0/2771740142 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878009880 con 0x7fe86c0039c0 2024-05-28T13:58:15.704 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe88d600640 1 -- 172.21.15.42:0/2771740142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe86c0039c0 msgr2=0x7fe86c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:15.704 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe88d600640 1 --2- 172.21.15.42:0/2771740142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe86c0039c0 0x7fe86c003de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe878009c80 tx=0x7fe87801cbb0 comp rx=0 tx=0).stop 2024-05-28T13:58:15.704 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe88d600640 1 -- 172.21.15.42:0/2771740142 shutdown_connections 2024-05-28T13:58:15.705 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe88d600640 1 --2- 172.21.15.42:0/2771740142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe86c0039c0 0x7fe86c003de0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:15.705 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe88d600640 1 -- 172.21.15.42:0/2771740142 >> 172.21.15.42:0/2771740142 conn(0x7fe8880b3f70 msgr2=0x7fe8880b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:15.705 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe88d600640 1 -- 172.21.15.42:0/2771740142 shutdown_connections 2024-05-28T13:58:15.705 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.717+0000 7fe88d600640 1 -- 172.21.15.42:0/2771740142 wait complete. 2024-05-28T13:58:15.706 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe88d600640 1 Processor -- start 2024-05-28T13:58:15.706 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe88d600640 1 -- start start 2024-05-28T13:58:15.707 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe88d600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86c0039c0 0x7fe888161b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:15.708 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe88d600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe88815acc0 0x7fe8880b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:15.708 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe88d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8880b54b0 0x7fe8880b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:15.708 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe88d600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe8880d0d10 con 0x7fe8880b54b0 2024-05-28T13:58:15.708 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe88d600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8880d0b90 con 0x7fe88815acc0 2024-05-28T13:58:15.708 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe88d600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe8880d0e90 con 0x7fe86c0039c0 2024-05-28T13:58:15.708 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe887400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8880b54b0 0x7fe8880b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:15.708 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe887400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8880b54b0 0x7fe8880b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54286/0 (socket says 172.21.15.42:54286) 2024-05-28T13:58:15.708 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe87e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe88815acc0 0x7fe8880b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:15.708 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe887400640 1 -- 172.21.15.42:0/3228065683 learned_addr learned my addr 172.21.15.42:0/3228065683 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:15.708 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe886a00640 1 --2- 172.21.15.42:0/3228065683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86c0039c0 0x7fe888161b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:15.708 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe887400640 1 -- 172.21.15.42:0/3228065683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86c0039c0 msgr2=0x7fe888161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:15.708 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe887400640 1 --2- 172.21.15.42:0/3228065683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86c0039c0 0x7fe888161b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:15.709 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe887400640 1 -- 172.21.15.42:0/3228065683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe88815acc0 msgr2=0x7fe8880b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:15.709 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe887400640 1 --2- 172.21.15.42:0/3228065683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe88815acc0 0x7fe8880b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:15.709 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe887400640 1 -- 172.21.15.42:0/3228065683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88815b530 con 0x7fe8880b54b0 2024-05-28T13:58:15.709 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe887400640 1 --2- 172.21.15.42:0/3228065683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8880b54b0 0x7fe8880b5930 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe8880685b0 tx=0x7fe88000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:15.710 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe87f400640 1 -- 172.21.15.42:0/3228065683 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880003980 con 0x7fe8880b54b0 2024-05-28T13:58:15.710 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.721+0000 7fe87f400640 1 -- 172.21.15.42:0/3228065683 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe880018070 con 0x7fe8880b54b0 2024-05-28T13:58:15.710 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.725+0000 7fe87f400640 1 -- 172.21.15.42:0/3228065683 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880003d80 con 0x7fe8880b54b0 2024-05-28T13:58:15.710 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.725+0000 7fe88d600640 1 -- 172.21.15.42:0/3228065683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88815b810 con 0x7fe8880b54b0 2024-05-28T13:58:15.710 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.725+0000 7fe88d600640 1 -- 172.21.15.42:0/3228065683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88815bdd0 con 0x7fe8880b54b0 2024-05-28T13:58:15.710 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.725+0000 7fe88d600640 1 -- 172.21.15.42:0/3228065683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe88804fcf0 con 0x7fe8880b54b0 2024-05-28T13:58:15.710 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.725+0000 7fe87f400640 1 -- 172.21.15.42:0/3228065683 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe8800050d0 con 0x7fe8880b54b0 2024-05-28T13:58:15.711 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.725+0000 7fe87f400640 1 --2- 172.21.15.42:0/3228065683 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe8640683b0 0x7fe86406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:15.711 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.725+0000 7fe87f400640 1 -- 172.21.15.42:0/3228065683 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fe880086160 con 0x7fe8880b54b0 2024-05-28T13:58:15.711 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.725+0000 7fe886a00640 1 --2- 172.21.15.42:0/3228065683 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe8640683b0 0x7fe86406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:15.712 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.725+0000 7fe886a00640 1 --2- 172.21.15.42:0/3228065683 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe8640683b0 0x7fe86406a870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe8780af000 tx=0x7fe8780a7040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:15.716 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.729+0000 7fe87f400640 1 -- 172.21.15.42:0/3228065683 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe880056380 con 0x7fe8880b54b0 2024-05-28T13:58:15.882 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.897+0000 7fe88d600640 1 -- 172.21.15.42:0/3228065683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8880ce8d0 con 0x7fe8880b54b0 2024-05-28T13:58:15.883 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.897+0000 7fe87f400640 1 -- 172.21.15.42:0/3228065683 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12824 (secure 0 0 0) 0x7fe88005a680 con 0x7fe8880b54b0 2024-05-28T13:58:15.883 INFO:teuthology.misc.health.smithi042.stdout: 2024-05-28T13:58:15.883 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":17,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:13.603613+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:15.885 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.901+0000 7fe88d600640 1 -- 172.21.15.42:0/3228065683 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe8640683b0 msgr2=0x7fe86406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:15.885 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.901+0000 7fe88d600640 1 --2- 172.21.15.42:0/3228065683 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe8640683b0 0x7fe86406a870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe8780af000 tx=0x7fe8780a7040 comp rx=0 tx=0).stop 2024-05-28T13:58:15.885 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.901+0000 7fe88d600640 1 -- 172.21.15.42:0/3228065683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8880b54b0 msgr2=0x7fe8880b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:15.885 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.901+0000 7fe88d600640 1 --2- 172.21.15.42:0/3228065683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8880b54b0 0x7fe8880b5930 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe8880685b0 tx=0x7fe88000eb90 comp rx=0 tx=0).stop 2024-05-28T13:58:15.886 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.901+0000 7fe88d600640 1 -- 172.21.15.42:0/3228065683 shutdown_connections 2024-05-28T13:58:15.886 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.901+0000 7fe88d600640 1 --2- 172.21.15.42:0/3228065683 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe8640683b0 0x7fe86406a870 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:15.886 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.901+0000 7fe88d600640 1 --2- 172.21.15.42:0/3228065683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8880b54b0 0x7fe8880b5930 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:15.886 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.901+0000 7fe88d600640 1 --2- 172.21.15.42:0/3228065683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe88815acc0 0x7fe8880b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:15.886 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.901+0000 7fe88d600640 1 --2- 172.21.15.42:0/3228065683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86c0039c0 0x7fe888161b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:15.886 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.901+0000 7fe88d600640 1 -- 172.21.15.42:0/3228065683 >> 172.21.15.42:0/3228065683 conn(0x7fe8880b3f70 msgr2=0x7fe88815ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:15.886 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.901+0000 7fe88d600640 1 -- 172.21.15.42:0/3228065683 shutdown_connections 2024-05-28T13:58:15.886 INFO:teuthology.misc.health.smithi042.stderr:2024-05-28T13:58:15.901+0000 7fe88d600640 1 -- 172.21.15.42:0/3228065683 wait complete. 2024-05-28T13:58:15.902 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-28T13:58:15.902 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-28T13:58:15.902 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-28T13:58:15.903 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-28T13:58:15.906 INFO:tasks.ceph.mds.a:Started 2024-05-28T13:58:15.906 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-28T13:58:15.907 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-28T13:58:15.909 INFO:tasks.ceph.mds.c:Started 2024-05-28T13:58:15.910 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-28T13:58:15.910 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-28T13:58:15.913 INFO:tasks.ceph.mds.b:Started 2024-05-28T13:58:15.913 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-28T13:58:15.913 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-28T13:58:15.916 INFO:tasks.ceph.mds.d:Started 2024-05-28T13:58:15.917 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-28T13:58:15.917 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:16.008 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-05-28T13:58:16.009 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-05-28T13:58:16.022 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-28T13:58:16.023 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T13:58:16.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.077+0000 7fe383200640 1 Processor -- start 2024-05-28T13:58:16.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.077+0000 7fe383200640 1 -- start start 2024-05-28T13:58:16.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.077+0000 7fe383200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe3840d0450 0x7fe38409b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.077+0000 7fe383200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe38415d680 0x7fe3840b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.077+0000 7fe383200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3840d0160 con 0x7fe38415c880 2024-05-28T13:58:16.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.077+0000 7fe383200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fe38409b520 con 0x7fe38415d680 2024-05-28T13:58:16.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.077+0000 7fe383200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe38409abf0 con 0x7fe3840d0450 2024-05-28T13:58:16.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.077+0000 7fe381400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe38415c880 0x7fe3840d0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42774/0 (socket says 172.21.15.42:42774) 2024-05-28T13:58:16.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.077+0000 7fe382800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe38415d680 0x7fe3840b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.077+0000 7fe381400640 1 -- 172.21.15.42:0/3500155314 learned_addr learned my addr 172.21.15.42:0/3500155314 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:16.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.077+0000 7fe381e00640 1 --2- 172.21.15.42:0/3500155314 >> v2:172.21.15.63:3300/0 conn(0x7fe3840d0450 0x7fe38409b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe382800640 1 -- 172.21.15.42:0/3500155314 >> v2:172.21.15.63:3300/0 conn(0x7fe3840d0450 msgr2=0x7fe38409b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:16.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe382800640 1 --2- 172.21.15.42:0/3500155314 >> v2:172.21.15.63:3300/0 conn(0x7fe3840d0450 0x7fe38409b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe382800640 1 -- 172.21.15.42:0/3500155314 >> v1:172.21.15.42:6789/0 conn(0x7fe38415c880 legacy=0x7fe3840d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe380a00640 1 -- 172.21.15.42:0/3500155314 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2692963749 0 0) 0x7fe3840d0160 con 0x7fe38415c880 2024-05-28T13:58:16.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe382800640 1 -- 172.21.15.42:0/3500155314 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38409aed0 con 0x7fe38415d680 2024-05-28T13:58:16.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe382800640 1 --2- 172.21.15.42:0/3500155314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe38415d680 0x7fe3840b16e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe384073020 tx=0x7fe37c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6b2eafafe104af server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:16.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe380a00640 1 -- 172.21.15.42:0/3500155314 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c0b2070 con 0x7fe38415d680 2024-05-28T13:58:16.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe380a00640 1 -- 172.21.15.42:0/3500155314 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe37c0a57f0 con 0x7fe38415d680 2024-05-28T13:58:16.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe380a00640 1 -- 172.21.15.42:0/3500155314 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c0a5bc0 con 0x7fe38415d680 2024-05-28T13:58:16.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe383200640 1 -- 172.21.15.42:0/3500155314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe38415d680 msgr2=0x7fe3840b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe383200640 1 --2- 172.21.15.42:0/3500155314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe38415d680 0x7fe3840b16e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe384073020 tx=0x7fe37c01c8a0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe383200640 1 -- 172.21.15.42:0/3500155314 shutdown_connections 2024-05-28T13:58:16.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe383200640 1 --2- 172.21.15.42:0/3500155314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe38415d680 0x7fe3840b16e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe383200640 1 --2- 172.21.15.42:0/3500155314 >> v2:172.21.15.63:3300/0 conn(0x7fe3840d0450 0x7fe38409b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe383200640 1 -- 172.21.15.42:0/3500155314 >> 172.21.15.42:0/3500155314 conn(0x7fe3840b3dd0 msgr2=0x7fe3840b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:16.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe383200640 1 -- 172.21.15.42:0/3500155314 shutdown_connections 2024-05-28T13:58:16.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.081+0000 7fe383200640 1 -- 172.21.15.42:0/3500155314 wait complete. 2024-05-28T13:58:16.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe383200640 1 Processor -- start 2024-05-28T13:58:16.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe383200640 1 -- start start 2024-05-28T13:58:16.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe383200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3840d0450 0x7fe384157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe383200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe38415c880 0x7fe3840ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe383200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe38415d680 0x7fe38406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe383200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe3840cd990 con 0x7fe38415d680 2024-05-28T13:58:16.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe383200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe3840cd810 con 0x7fe3840d0450 2024-05-28T13:58:16.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe383200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe3840cdb10 con 0x7fe38415c880 2024-05-28T13:58:16.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe382800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe38415d680 0x7fe38406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe382800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe38415d680 0x7fe38406d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54316/0 (socket says 172.21.15.42:54316) 2024-05-28T13:58:16.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe382800640 1 -- 172.21.15.42:0/3174819307 learned_addr learned my addr 172.21.15.42:0/3174819307 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:16.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe381e00640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3840d0450 0x7fe384157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe381400640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe38415c880 0x7fe3840ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe382800640 1 -- 172.21.15.42:0/3174819307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe38415c880 msgr2=0x7fe3840ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe382800640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe38415c880 0x7fe3840ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe382800640 1 -- 172.21.15.42:0/3174819307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3840d0450 msgr2=0x7fe384157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe382800640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3840d0450 0x7fe384157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe382800640 1 -- 172.21.15.42:0/3174819307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3840c9870 con 0x7fe38415d680 2024-05-28T13:58:16.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe381e00640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3840d0450 0x7fe384157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:16.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe382800640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe38415d680 0x7fe38406d4f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0194b0 tx=0x7fe37c019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:16.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe37a000640 1 -- 172.21.15.42:0/3174819307 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c0b2050 con 0x7fe38415d680 2024-05-28T13:58:16.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe37a000640 1 -- 172.21.15.42:0/3174819307 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe37c0be070 con 0x7fe38415d680 2024-05-28T13:58:16.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe37a000640 1 -- 172.21.15.42:0/3174819307 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c009450 con 0x7fe38415d680 2024-05-28T13:58:16.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe383200640 1 -- 172.21.15.42:0/3174819307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3840c9b50 con 0x7fe38415d680 2024-05-28T13:58:16.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.085+0000 7fe383200640 1 -- 172.21.15.42:0/3174819307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3840ca110 con 0x7fe38415d680 2024-05-28T13:58:16.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.089+0000 7fe383200640 1 -- 172.21.15.42:0/3174819307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe38404fcf0 con 0x7fe38415d680 2024-05-28T13:58:16.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.089+0000 7fe37a000640 1 -- 172.21.15.42:0/3174819307 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe37c0adc40 con 0x7fe38415d680 2024-05-28T13:58:16.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.089+0000 7fe37a000640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe364068380 0x7fe36406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.089+0000 7fe37a000640 1 -- 172.21.15.42:0/3174819307 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fe37c121800 con 0x7fe38415d680 2024-05-28T13:58:16.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.089+0000 7fe381e00640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe364068380 0x7fe36406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.089+0000 7fe381e00640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe364068380 0x7fe36406a840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe384070d20 tx=0x7fe374008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:16.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.093+0000 7fe37a000640 1 -- 172.21.15.42:0/3174819307 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe37c016520 con 0x7fe38415d680 2024-05-28T13:58:16.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.257+0000 7fe383200640 1 -- 172.21.15.42:0/3174819307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3840b4ce0 con 0x7fe38415d680 2024-05-28T13:58:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.261+0000 7fe37a000640 1 -- 172.21.15.42:0/3174819307 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12824 (secure 0 0 0) 0x7fe37c0f2070 con 0x7fe38415d680 2024-05-28T13:58:16.245 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:16.245 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:13.603613+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:16.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.261+0000 7fe383200640 1 -- 172.21.15.42:0/3174819307 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe364068380 msgr2=0x7fe36406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.261+0000 7fe383200640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe364068380 0x7fe36406a840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe384070d20 tx=0x7fe374008040 comp rx=0 tx=0).stop 2024-05-28T13:58:16.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.261+0000 7fe383200640 1 -- 172.21.15.42:0/3174819307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe38415d680 msgr2=0x7fe38406d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.261+0000 7fe383200640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe38415d680 0x7fe38406d4f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0194b0 tx=0x7fe37c019640 comp rx=0 tx=0).stop 2024-05-28T13:58:16.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.261+0000 7fe383200640 1 -- 172.21.15.42:0/3174819307 shutdown_connections 2024-05-28T13:58:16.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.261+0000 7fe383200640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe364068380 0x7fe36406a840 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.261+0000 7fe383200640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe38415d680 0x7fe38406d4f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.261+0000 7fe383200640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe38415c880 0x7fe3840ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.261+0000 7fe383200640 1 --2- 172.21.15.42:0/3174819307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3840d0450 0x7fe384157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.261+0000 7fe383200640 1 -- 172.21.15.42:0/3174819307 >> 172.21.15.42:0/3174819307 conn(0x7fe3840b3dd0 msgr2=0x7fe3840749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:16.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.261+0000 7fe383200640 1 -- 172.21.15.42:0/3174819307 shutdown_connections 2024-05-28T13:58:16.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.261+0000 7fe383200640 1 -- 172.21.15.42:0/3174819307 wait complete. 2024-05-28T13:58:16.266 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:58:16.267 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:58:16.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c47800640 1 Processor -- start 2024-05-28T13:58:16.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c47800640 1 -- start start 2024-05-28T13:58:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c47800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4c4015c770 0x7f4c4009b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c47800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4c4015cb50 0x7f4c400d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c47800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c4009ba40 con 0x7f4c400c8bc0 2024-05-28T13:58:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c47800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f4c400c6720 con 0x7f4c4015cb50 2024-05-28T13:58:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c47800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4c400c60d0 con 0x7f4c4015c770 2024-05-28T13:58:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c37e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4c4015cb50 0x7f4c400d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c37e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4c4015cb50 0x7f4c400d0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54320/0 (socket says 172.21.15.42:54320) 2024-05-28T13:58:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c37e00640 1 -- 172.21.15.42:0/3554255731 learned_addr learned my addr 172.21.15.42:0/3554255731 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c44e00640 1 --2- 172.21.15.42:0/3554255731 >> v2:172.21.15.63:3300/0 conn(0x7f4c4015c770 0x7f4c4009b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c37e00640 1 -- 172.21.15.42:0/3554255731 >> v2:172.21.15.63:3300/0 conn(0x7f4c4015c770 msgr2=0x7f4c4009b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c37e00640 1 --2- 172.21.15.42:0/3554255731 >> v2:172.21.15.63:3300/0 conn(0x7f4c4015c770 0x7f4c4009b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c37e00640 1 -- 172.21.15.42:0/3554255731 >> v1:172.21.15.63:6789/0 conn(0x7f4c400c8bc0 legacy=0x7f4c400ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.425+0000 7f4c37e00640 1 -- 172.21.15.42:0/3554255731 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c400c63b0 con 0x7f4c4015cb50 2024-05-28T13:58:16.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.429+0000 7f4c37e00640 1 --2- 172.21.15.42:0/3554255731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c4015cb50 0x7f4c400d0ac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c400730d0 tx=0x7f4c2801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e479379e15cc05 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:16.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.429+0000 7f4c3fe00640 1 -- 172.21.15.42:0/3554255731 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c280b5070 con 0x7f4c4015cb50 2024-05-28T13:58:16.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.429+0000 7f4c3fe00640 1 -- 172.21.15.42:0/3554255731 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c280a8a10 con 0x7f4c4015cb50 2024-05-28T13:58:16.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.429+0000 7f4c3fe00640 1 -- 172.21.15.42:0/3554255731 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c280a8de0 con 0x7f4c4015cb50 2024-05-28T13:58:16.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.429+0000 7f4c47800640 1 -- 172.21.15.42:0/3554255731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c4015cb50 msgr2=0x7f4c400d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.429+0000 7f4c47800640 1 --2- 172.21.15.42:0/3554255731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c4015cb50 0x7f4c400d0ac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c400730d0 tx=0x7f4c2801ed10 comp rx=0 tx=0).stop 2024-05-28T13:58:16.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.429+0000 7f4c47800640 1 -- 172.21.15.42:0/3554255731 shutdown_connections 2024-05-28T13:58:16.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.429+0000 7f4c47800640 1 --2- 172.21.15.42:0/3554255731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c4015cb50 0x7f4c400d0ac0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.429+0000 7f4c47800640 1 --2- 172.21.15.42:0/3554255731 >> v2:172.21.15.63:3300/0 conn(0x7f4c4015c770 0x7f4c4009b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.429+0000 7f4c47800640 1 -- 172.21.15.42:0/3554255731 >> 172.21.15.42:0/3554255731 conn(0x7f4c400b3fd0 msgr2=0x7f4c400b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:16.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.429+0000 7f4c47800640 1 -- 172.21.15.42:0/3554255731 shutdown_connections 2024-05-28T13:58:16.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.429+0000 7f4c47800640 1 -- 172.21.15.42:0/3554255731 wait complete. 2024-05-28T13:58:16.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c47800640 1 Processor -- start 2024-05-28T13:58:16.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c47800640 1 -- start start 2024-05-28T13:58:16.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c47800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c400c8bc0 0x7f4c401e9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c47800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c4015c770 0x7f4c400c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c47800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c4015cb50 0x7f4c4006cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c47800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c400d02c0 con 0x7f4c4015cb50 2024-05-28T13:58:16.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c47800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4c400ce2c0 con 0x7f4c4015c770 2024-05-28T13:58:16.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c47800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4c400d0440 con 0x7f4c400c8bc0 2024-05-28T13:58:16.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c45800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c4015cb50 0x7f4c4006cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c45800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c4015cb50 0x7f4c4006cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54322/0 (socket says 172.21.15.42:54322) 2024-05-28T13:58:16.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c45800640 1 -- 172.21.15.42:0/2234068001 learned_addr learned my addr 172.21.15.42:0/2234068001 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:16.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c37e00640 1 --2- 172.21.15.42:0/2234068001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c4015c770 0x7f4c400c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c44e00640 1 --2- 172.21.15.42:0/2234068001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c400c8bc0 0x7f4c401e9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c45800640 1 -- 172.21.15.42:0/2234068001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c400c8bc0 msgr2=0x7f4c401e9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c45800640 1 --2- 172.21.15.42:0/2234068001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c400c8bc0 0x7f4c401e9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c45800640 1 -- 172.21.15.42:0/2234068001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c4015c770 msgr2=0x7f4c400c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c45800640 1 --2- 172.21.15.42:0/2234068001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c4015c770 0x7f4c400c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c45800640 1 -- 172.21.15.42:0/2234068001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c401575d0 con 0x7f4c4015cb50 2024-05-28T13:58:16.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c45800640 1 --2- 172.21.15.42:0/2234068001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c4015cb50 0x7f4c4006cb00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800c770 tx=0x7f4c3800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:16.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c3d600640 1 -- 172.21.15.42:0/2234068001 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38003b80 con 0x7f4c4015cb50 2024-05-28T13:58:16.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c3d600640 1 -- 172.21.15.42:0/2234068001 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c38003d20 con 0x7f4c4015cb50 2024-05-28T13:58:16.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c3d600640 1 -- 172.21.15.42:0/2234068001 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800aa90 con 0x7f4c4015cb50 2024-05-28T13:58:16.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c47800640 1 -- 172.21.15.42:0/2234068001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c400c55e0 con 0x7f4c4015cb50 2024-05-28T13:58:16.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.433+0000 7f4c47800640 1 -- 172.21.15.42:0/2234068001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c40157b50 con 0x7f4c4015cb50 2024-05-28T13:58:16.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.437+0000 7f4c47800640 1 -- 172.21.15.42:0/2234068001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c400a04d0 con 0x7f4c4015cb50 2024-05-28T13:58:16.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.437+0000 7f4c3d600640 1 -- 172.21.15.42:0/2234068001 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4c3800ac30 con 0x7f4c4015cb50 2024-05-28T13:58:16.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.437+0000 7f4c3d600640 1 --2- 172.21.15.42:0/2234068001 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4c1c068380 0x7f4c1c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.437+0000 7f4c3d600640 1 -- 172.21.15.42:0/2234068001 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f4c38086430 con 0x7f4c4015cb50 2024-05-28T13:58:16.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.437+0000 7f4c44e00640 1 --2- 172.21.15.42:0/2234068001 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4c1c068380 0x7f4c1c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.437+0000 7f4c44e00640 1 --2- 172.21.15.42:0/2234068001 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4c1c068380 0x7f4c1c06a840 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009b60 tx=0x7f4c30009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:16.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.441+0000 7f4c3d600640 1 -- 172.21.15.42:0/2234068001 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4c380566e0 con 0x7f4c4015cb50 2024-05-28T13:58:16.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.609+0000 7f4c47800640 1 -- 172.21.15.42:0/2234068001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c40160170 con 0x7f4c4015cb50 2024-05-28T13:58:16.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.609+0000 7f4c3d600640 1 -- 172.21.15.42:0/2234068001 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f4c3805a9e0 con 0x7f4c4015cb50 2024-05-28T13:58:16.595 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:58:16.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.613+0000 7f4c47800640 1 -- 172.21.15.42:0/2234068001 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4c1c068380 msgr2=0x7f4c1c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.613+0000 7f4c47800640 1 --2- 172.21.15.42:0/2234068001 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4c1c068380 0x7f4c1c06a840 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009b60 tx=0x7f4c30009190 comp rx=0 tx=0).stop 2024-05-28T13:58:16.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.613+0000 7f4c47800640 1 -- 172.21.15.42:0/2234068001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c4015cb50 msgr2=0x7f4c4006cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.613+0000 7f4c47800640 1 --2- 172.21.15.42:0/2234068001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c4015cb50 0x7f4c4006cb00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800c770 tx=0x7f4c3800cc30 comp rx=0 tx=0).stop 2024-05-28T13:58:16.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.613+0000 7f4c47800640 1 -- 172.21.15.42:0/2234068001 shutdown_connections 2024-05-28T13:58:16.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.613+0000 7f4c47800640 1 --2- 172.21.15.42:0/2234068001 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4c1c068380 0x7f4c1c06a840 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.613+0000 7f4c47800640 1 --2- 172.21.15.42:0/2234068001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c4015cb50 0x7f4c4006cb00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.613+0000 7f4c47800640 1 --2- 172.21.15.42:0/2234068001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c4015c770 0x7f4c400c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.613+0000 7f4c47800640 1 --2- 172.21.15.42:0/2234068001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c400c8bc0 0x7f4c401e9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.613+0000 7f4c47800640 1 -- 172.21.15.42:0/2234068001 >> 172.21.15.42:0/2234068001 conn(0x7f4c400b3fd0 msgr2=0x7f4c400c7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:16.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.613+0000 7f4c47800640 1 -- 172.21.15.42:0/2234068001 shutdown_connections 2024-05-28T13:58:16.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.613+0000 7f4c47800640 1 -- 172.21.15.42:0/2234068001 wait complete. 2024-05-28T13:58:16.616 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T13:58:16.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb16da00640 1 Processor -- start 2024-05-28T13:58:16.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb16da00640 1 -- start start 2024-05-28T13:58:16.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb16da00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb1680d0900 0x7fb168162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb16da00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1680d0de0 con 0x7fb168162ab0 2024-05-28T13:58:16.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb16da00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1680d10c0 con 0x7fb1680ce860 2024-05-28T13:58:16.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb16da00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb1680d19f0 con 0x7fb1680d0900 2024-05-28T13:58:16.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb167400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb168162ab0 0x7fb16815e200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42778/0 (socket says 172.21.15.42:42778) 2024-05-28T13:58:16.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb167400640 1 -- 172.21.15.42:0/3397153506 learned_addr learned my addr 172.21.15.42:0/3397153506 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:16.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb15fe00640 1 --2- 172.21.15.42:0/3397153506 >> v2:172.21.15.63:3300/0 conn(0x7fb1680d0900 0x7fb168162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb166000640 1 -- 172.21.15.42:0/3397153506 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860056577 0 0) 0x7fb1680d0de0 con 0x7fb168162ab0 2024-05-28T13:58:16.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb166000640 1 -- 172.21.15.42:0/3397153506 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb14c003530 con 0x7fb168162ab0 2024-05-28T13:58:16.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb166000640 1 -- 172.21.15.42:0/3397153506 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4015283078 0 0) 0x7fb1680d10c0 con 0x7fb1680ce860 2024-05-28T13:58:16.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb166000640 1 -- 172.21.15.42:0/3397153506 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1680d0de0 con 0x7fb1680ce860 2024-05-28T13:58:16.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb166000640 1 -- 172.21.15.42:0/3397153506 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3425629376 0 0) 0x7fb14c003530 con 0x7fb168162ab0 2024-05-28T13:58:16.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb15fe00640 1 -- 172.21.15.42:0/3397153506 >> v1:172.21.15.63:6789/0 conn(0x7fb1680ce860 legacy=0x7fb16806d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb15fe00640 1 -- 172.21.15.42:0/3397153506 >> v1:172.21.15.42:6789/0 conn(0x7fb168162ab0 legacy=0x7fb16815e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.773+0000 7fb15fe00640 1 -- 172.21.15.42:0/3397153506 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1680d13a0 con 0x7fb1680d0900 2024-05-28T13:58:16.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.777+0000 7fb15fe00640 1 --2- 172.21.15.42:0/3397153506 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1680d0900 0x7fb168162570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb1500097c0 tx=0x7fb15001c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cbd1b97b3c9317b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:16.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.777+0000 7fb166000640 1 -- 172.21.15.42:0/3397153506 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1500b0070 con 0x7fb1680d0900 2024-05-28T13:58:16.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.777+0000 7fb166000640 1 -- 172.21.15.42:0/3397153506 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1500ab400 con 0x7fb1680d0900 2024-05-28T13:58:16.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.777+0000 7fb166000640 1 -- 172.21.15.42:0/3397153506 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160001a00 con 0x7fb1680d0900 2024-05-28T13:58:16.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.777+0000 7fb16da00640 1 -- 172.21.15.42:0/3397153506 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1680d0900 msgr2=0x7fb168162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.777+0000 7fb16da00640 1 --2- 172.21.15.42:0/3397153506 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1680d0900 0x7fb168162570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb1500097c0 tx=0x7fb15001c330 comp rx=0 tx=0).stop 2024-05-28T13:58:16.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.777+0000 7fb16da00640 1 -- 172.21.15.42:0/3397153506 shutdown_connections 2024-05-28T13:58:16.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.777+0000 7fb16da00640 1 --2- 172.21.15.42:0/3397153506 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1680d0900 0x7fb168162570 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.777+0000 7fb16da00640 1 -- 172.21.15.42:0/3397153506 >> 172.21.15.42:0/3397153506 conn(0x7fb1680b4080 msgr2=0x7fb1680b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:16.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.777+0000 7fb16da00640 1 -- 172.21.15.42:0/3397153506 shutdown_connections 2024-05-28T13:58:16.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.777+0000 7fb16da00640 1 -- 172.21.15.42:0/3397153506 wait complete. 2024-05-28T13:58:16.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb16da00640 1 Processor -- start 2024-05-28T13:58:16.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb16da00640 1 -- start start 2024-05-28T13:58:16.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb16da00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1680ce860 0x7fb168160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb16da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1680d0900 0x7fb16815ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb16da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb168162ab0 0x7fb168161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb16da00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb16806e040 con 0x7fb1680d0900 2024-05-28T13:58:16.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb16da00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb16806dec0 con 0x7fb168162ab0 2024-05-28T13:58:16.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb16da00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb16816d0d0 con 0x7fb1680ce860 2024-05-28T13:58:16.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb15fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1680d0900 0x7fb16815ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb15fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1680d0900 0x7fb16815ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54338/0 (socket says 172.21.15.42:54338) 2024-05-28T13:58:16.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb15fe00640 1 -- 172.21.15.42:0/863877207 learned_addr learned my addr 172.21.15.42:0/863877207 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:16.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb15fe00640 1 -- 172.21.15.42:0/863877207 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1680ce860 msgr2=0x7fb168160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb166a00640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1680ce860 0x7fb168160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb167400640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb168162ab0 0x7fb168161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb15fe00640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1680ce860 0x7fb168160160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb15fe00640 1 -- 172.21.15.42:0/863877207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb168162ab0 msgr2=0x7fb168161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb15fe00640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb168162ab0 0x7fb168161100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb15fe00640 1 -- 172.21.15.42:0/863877207 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb168160740 con 0x7fb1680d0900 2024-05-28T13:58:16.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb166a00640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1680ce860 0x7fb168160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:16.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb15fe00640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1680d0900 0x7fb16815ba20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb1500ae040 tx=0x7fb15001ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:16.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb15ea00640 1 -- 172.21.15.42:0/863877207 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1500bb020 con 0x7fb1680d0900 2024-05-28T13:58:16.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb15ea00640 1 -- 172.21.15.42:0/863877207 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1500b1040 con 0x7fb1680d0900 2024-05-28T13:58:16.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb16da00640 1 -- 172.21.15.42:0/863877207 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16815af40 con 0x7fb1680d0900 2024-05-28T13:58:16.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb15ea00640 1 -- 172.21.15.42:0/863877207 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1500b0050 con 0x7fb1680d0900 2024-05-28T13:58:16.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb16da00640 1 -- 172.21.15.42:0/863877207 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb168150c20 con 0x7fb1680d0900 2024-05-28T13:58:16.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.781+0000 7fb16da00640 1 -- 172.21.15.42:0/863877207 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb168068a90 con 0x7fb1680d0900 2024-05-28T13:58:16.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.785+0000 7fb15ea00640 1 -- 172.21.15.42:0/863877207 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb1500be070 con 0x7fb1680d0900 2024-05-28T13:58:16.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.785+0000 7fb15ea00640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb144068380 0x7fb14406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:16.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.785+0000 7fb15ea00640 1 -- 172.21.15.42:0/863877207 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fb15011fd40 con 0x7fb1680d0900 2024-05-28T13:58:16.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.785+0000 7fb166a00640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb144068380 0x7fb14406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:16.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.785+0000 7fb166a00640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb144068380 0x7fb14406a840 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb1580011f0 tx=0x7fb158009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:16.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.789+0000 7fb15ea00640 1 -- 172.21.15.42:0/863877207 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb150015fb0 con 0x7fb1680d0900 2024-05-28T13:58:16.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.989+0000 7fb16da00640 1 -- 172.21.15.42:0/863877207 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1680b4ce0 con 0x7fb1680d0900 2024-05-28T13:58:16.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.989+0000 7fb15ea00640 1 -- 172.21.15.42:0/863877207 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7fb1500b3300 con 0x7fb1680d0900 2024-05-28T13:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/3753517701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3753517701},{"type":"v1","addr":"172.21.15.42:6836","nonce":3753517701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-28T13:58:16.974 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 2 2024-05-28T13:58:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.989+0000 7fb16da00640 1 -- 172.21.15.42:0/863877207 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb144068380 msgr2=0x7fb14406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.989+0000 7fb16da00640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb144068380 0x7fb14406a840 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb1580011f0 tx=0x7fb158009040 comp rx=0 tx=0).stop 2024-05-28T13:58:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.989+0000 7fb16da00640 1 -- 172.21.15.42:0/863877207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1680d0900 msgr2=0x7fb16815ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.989+0000 7fb16da00640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1680d0900 0x7fb16815ba20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb1500ae040 tx=0x7fb15001ab70 comp rx=0 tx=0).stop 2024-05-28T13:58:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.989+0000 7fb16da00640 1 -- 172.21.15.42:0/863877207 shutdown_connections 2024-05-28T13:58:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.989+0000 7fb16da00640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb144068380 0x7fb14406a840 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.989+0000 7fb16da00640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb168162ab0 0x7fb168161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.989+0000 7fb16da00640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1680d0900 0x7fb16815ba20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.989+0000 7fb16da00640 1 --2- 172.21.15.42:0/863877207 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1680ce860 0x7fb168160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:16.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.989+0000 7fb16da00640 1 -- 172.21.15.42:0/863877207 >> 172.21.15.42:0/863877207 conn(0x7fb1680b4080 msgr2=0x7fb16815e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:16.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.989+0000 7fb16da00640 1 -- 172.21.15.42:0/863877207 shutdown_connections 2024-05-28T13:58:16.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:16.989+0000 7fb16da00640 1 -- 172.21.15.42:0/863877207 wait complete. 2024-05-28T13:58:17.276 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T13:58:17.280 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T13:58:18.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T13:58:19.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.161+0000 7fd29a600640 1 Processor -- start 2024-05-28T13:58:19.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.161+0000 7fd29a600640 1 -- start start 2024-05-28T13:58:19.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.165+0000 7fd29a600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd29415ba40 0x7fd2940d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.165+0000 7fd29a600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd294072dd0 0x7fd2940d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.165+0000 7fd29a600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd29415f280 con 0x7fd29406f660 2024-05-28T13:58:19.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.165+0000 7fd29a600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd29415f030 con 0x7fd29415ba40 2024-05-28T13:58:19.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.165+0000 7fd29a600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd29415f560 con 0x7fd294072dd0 2024-05-28T13:58:19.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.165+0000 7fd28be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd29415ba40 0x7fd2940d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.165+0000 7fd28be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd29415ba40 0x7fd2940d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49048/0 (socket says 172.21.15.42:49048) 2024-05-28T13:58:19.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.165+0000 7fd28be00640 1 -- 172.21.15.42:0/583069143 learned_addr learned my addr 172.21.15.42:0/583069143 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:19.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.165+0000 7fd293e00640 1 --2- 172.21.15.42:0/583069143 >> v2:172.21.15.63:3300/0 conn(0x7fd294072dd0 0x7fd2940d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.165+0000 7fd28be00640 1 -- 172.21.15.42:0/583069143 >> v2:172.21.15.63:3300/0 conn(0x7fd294072dd0 msgr2=0x7fd2940d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:19.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.165+0000 7fd28be00640 1 --2- 172.21.15.42:0/583069143 >> v2:172.21.15.63:3300/0 conn(0x7fd294072dd0 0x7fd2940d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.165+0000 7fd28be00640 1 -- 172.21.15.42:0/583069143 >> v1:172.21.15.63:6789/0 conn(0x7fd29406f660 legacy=0x7fd294073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:19.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.165+0000 7fd28be00640 1 -- 172.21.15.42:0/583069143 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29415f840 con 0x7fd29415ba40 2024-05-28T13:58:19.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.169+0000 7fd28be00640 1 --2- 172.21.15.42:0/583069143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd29415ba40 0x7fd2940d06f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd29415cb20 tx=0x7fd27c01c0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805a8da170a8a2bd server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:19.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.169+0000 7fd292a00640 1 -- 172.21.15.42:0/583069143 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0b1070 con 0x7fd29415ba40 2024-05-28T13:58:19.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.169+0000 7fd292a00640 1 -- 172.21.15.42:0/583069143 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd27c0a48f0 con 0x7fd29415ba40 2024-05-28T13:58:19.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.169+0000 7fd292a00640 1 -- 172.21.15.42:0/583069143 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0a4cc0 con 0x7fd29415ba40 2024-05-28T13:58:19.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.169+0000 7fd29a600640 1 -- 172.21.15.42:0/583069143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd29415ba40 msgr2=0x7fd2940d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.169+0000 7fd29a600640 1 --2- 172.21.15.42:0/583069143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd29415ba40 0x7fd2940d06f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd29415cb20 tx=0x7fd27c01c0b0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.169+0000 7fd29a600640 1 -- 172.21.15.42:0/583069143 shutdown_connections 2024-05-28T13:58:19.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.169+0000 7fd29a600640 1 --2- 172.21.15.42:0/583069143 >> v2:172.21.15.63:3300/0 conn(0x7fd294072dd0 0x7fd2940d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.169+0000 7fd29a600640 1 --2- 172.21.15.42:0/583069143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd29415ba40 0x7fd2940d06f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.169+0000 7fd29a600640 1 -- 172.21.15.42:0/583069143 >> 172.21.15.42:0/583069143 conn(0x7fd29406bdf0 msgr2=0x7fd29406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:19.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.169+0000 7fd29a600640 1 -- 172.21.15.42:0/583069143 shutdown_connections 2024-05-28T13:58:19.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.169+0000 7fd29a600640 1 -- 172.21.15.42:0/583069143 wait complete. 2024-05-28T13:58:19.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.169+0000 7fd29a600640 1 Processor -- start 2024-05-28T13:58:19.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.169+0000 7fd29a600640 1 -- start start 2024-05-28T13:58:19.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd29a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd29406f660 0x7fd294150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd29a600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd294072dd0 0x7fd2940b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd29a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29414eff0 0x7fd2940b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd29a600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd294168f70 con 0x7fd29406f660 2024-05-28T13:58:19.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd29a600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd294168df0 con 0x7fd29414eff0 2024-05-28T13:58:19.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd29a600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2941690f0 con 0x7fd294072dd0 2024-05-28T13:58:19.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd293400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd29406f660 0x7fd294150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd293400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd29406f660 0x7fd294150520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49054/0 (socket says 172.21.15.42:49054) 2024-05-28T13:58:19.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd293400640 1 -- 172.21.15.42:0/744829709 learned_addr learned my addr 172.21.15.42:0/744829709 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd293e00640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29414eff0 0x7fd2940b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd28be00640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd294072dd0 0x7fd2940b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd293400640 1 -- 172.21.15.42:0/744829709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd294072dd0 msgr2=0x7fd2940b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd293400640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd294072dd0 0x7fd2940b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd293400640 1 -- 172.21.15.42:0/744829709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29414eff0 msgr2=0x7fd2940b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd293400640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29414eff0 0x7fd2940b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd293400640 1 -- 172.21.15.42:0/744829709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29414f480 con 0x7fd29406f660 2024-05-28T13:58:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd293e00640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29414eff0 0x7fd2940b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd293400640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd29406f660 0x7fd294150520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd28400c3a0 tx=0x7fd28400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd28b400640 1 -- 172.21.15.42:0/744829709 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2840052d0 con 0x7fd29406f660 2024-05-28T13:58:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd28b400640 1 -- 172.21.15.42:0/744829709 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd284005470 con 0x7fd29406f660 2024-05-28T13:58:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd28b400640 1 -- 172.21.15.42:0/744829709 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284007260 con 0x7fd29406f660 2024-05-28T13:58:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd29a600640 1 -- 172.21.15.42:0/744829709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29414f760 con 0x7fd29406f660 2024-05-28T13:58:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd29a600640 1 -- 172.21.15.42:0/744829709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd29414fd20 con 0x7fd29406f660 2024-05-28T13:58:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd29a600640 1 -- 172.21.15.42:0/744829709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd294068a90 con 0x7fd29406f660 2024-05-28T13:58:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.173+0000 7fd28b400640 1 -- 172.21.15.42:0/744829709 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd284007400 con 0x7fd29406f660 2024-05-28T13:58:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.177+0000 7fd28b400640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd270068380 0x7fd27006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.177+0000 7fd28b400640 1 -- 172.21.15.42:0/744829709 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd284085ac0 con 0x7fd29406f660 2024-05-28T13:58:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.177+0000 7fd28be00640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd270068380 0x7fd27006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.177+0000 7fd28be00640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd270068380 0x7fd27006a840 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0ae000 tx=0x7fd27c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:19.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.181+0000 7fd28b400640 1 -- 172.21.15.42:0/744829709 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd284055ce0 con 0x7fd29406f660 2024-05-28T13:58:19.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.381+0000 7fd29a600640 1 -- 172.21.15.42:0/744829709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd294072bb0 con 0x7fd29406f660 2024-05-28T13:58:19.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.381+0000 7fd28b400640 1 -- 172.21.15.42:0/744829709 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3861 (secure 0 0 0) 0x7fd284059fe0 con 0x7fd29406f660 2024-05-28T13:58:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:19.366 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1409117123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1409117123},{"type":"v1","addr":"172.21.15.42:6837","nonce":1409117123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/3753517701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3753517701},{"type":"v1","addr":"172.21.15.42:6836","nonce":3753517701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/882866492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":882866492},{"type":"v1","addr":"172.21.15.63:6834","nonce":882866492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1552779700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1552779700},{"type":"v1","addr":"172.21.15.63:6835","nonce":1552779700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-28T13:58:19.367 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 3 2024-05-28T13:58:19.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.381+0000 7fd29a600640 1 -- 172.21.15.42:0/744829709 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd270068380 msgr2=0x7fd27006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.381+0000 7fd29a600640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd270068380 0x7fd27006a840 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0ae000 tx=0x7fd27c009040 comp rx=0 tx=0).stop 2024-05-28T13:58:19.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.381+0000 7fd29a600640 1 -- 172.21.15.42:0/744829709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd29406f660 msgr2=0x7fd294150520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.381+0000 7fd29a600640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd29406f660 0x7fd294150520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd28400c3a0 tx=0x7fd28400c860 comp rx=0 tx=0).stop 2024-05-28T13:58:19.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.381+0000 7fd29a600640 1 -- 172.21.15.42:0/744829709 shutdown_connections 2024-05-28T13:58:19.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.381+0000 7fd29a600640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd270068380 0x7fd27006a840 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.381+0000 7fd29a600640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd29414eff0 0x7fd2940b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.381+0000 7fd29a600640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd294072dd0 0x7fd2940b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.381+0000 7fd29a600640 1 --2- 172.21.15.42:0/744829709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd29406f660 0x7fd294150520 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.381+0000 7fd29a600640 1 -- 172.21.15.42:0/744829709 >> 172.21.15.42:0/744829709 conn(0x7fd29406bdf0 msgr2=0x7fd29406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:19.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.385+0000 7fd29a600640 1 -- 172.21.15.42:0/744829709 shutdown_connections 2024-05-28T13:58:19.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.385+0000 7fd29a600640 1 -- 172.21.15.42:0/744829709 wait complete. 2024-05-28T13:58:19.389 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:19.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b95600640 1 Processor -- start 2024-05-28T13:58:19.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b95600640 1 -- start start 2024-05-28T13:58:19.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b95600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3b90156ac0 0x7f3b9015d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b95600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9006e4e0 0x7f3b900b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b95600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b9006e960 con 0x7f3b900b6440 2024-05-28T13:58:19.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b95600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3b9015dc00 con 0x7f3b90156ac0 2024-05-28T13:58:19.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b95600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9006ebe0 con 0x7f3b9006e4e0 2024-05-28T13:58:19.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b94c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f3b900b6440 0x7f3b90160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:58062/0 (socket says 172.21.15.42:58062) 2024-05-28T13:58:19.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b94c00640 1 -- 172.21.15.42:0/128410762 learned_addr learned my addr 172.21.15.42:0/128410762 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:19.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b8fe00640 1 --2- 172.21.15.42:0/128410762 >> v2:172.21.15.63:3300/0 conn(0x7f3b90156ac0 0x7f3b9015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b87400640 1 --2- 172.21.15.42:0/128410762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9006e4e0 0x7f3b900b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b8f400640 1 -- 172.21.15.42:0/128410762 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3368508018 0 0) 0x7f3b88001120 con 0x7f3b900b6440 2024-05-28T13:58:19.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b8f400640 1 -- 172.21.15.42:0/128410762 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b74003530 con 0x7f3b900b6440 2024-05-28T13:58:19.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b8fe00640 1 -- 172.21.15.42:0/128410762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9006e4e0 msgr2=0x7f3b900b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:19.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b8fe00640 1 --2- 172.21.15.42:0/128410762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9006e4e0 0x7f3b900b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b8fe00640 1 -- 172.21.15.42:0/128410762 >> v1:172.21.15.42:6789/0 conn(0x7f3b900b6440 legacy=0x7f3b90160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b8fe00640 1 -- 172.21.15.42:0/128410762 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9006eec0 con 0x7f3b90156ac0 2024-05-28T13:58:19.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.545+0000 7f3b87400640 1 --2- 172.21.15.42:0/128410762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9006e4e0 0x7f3b900b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:19.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.549+0000 7f3b8fe00640 1 --2- 172.21.15.42:0/128410762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b90156ac0 0x7f3b9015d550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b800097c0 tx=0x7f3b8001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d5fcd7a83d981e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:19.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.549+0000 7f3b8f400640 1 -- 172.21.15.42:0/128410762 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b800b0070 con 0x7f3b90156ac0 2024-05-28T13:58:19.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.549+0000 7f3b8f400640 1 -- 172.21.15.42:0/128410762 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b800ab600 con 0x7f3b90156ac0 2024-05-28T13:58:19.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.549+0000 7f3b8f400640 1 -- 172.21.15.42:0/128410762 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b800ab9d0 con 0x7f3b90156ac0 2024-05-28T13:58:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.549+0000 7f3b95600640 1 -- 172.21.15.42:0/128410762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b90156ac0 msgr2=0x7f3b9015d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.549+0000 7f3b95600640 1 --2- 172.21.15.42:0/128410762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b90156ac0 0x7f3b9015d550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b800097c0 tx=0x7f3b8001c410 comp rx=0 tx=0).stop 2024-05-28T13:58:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.549+0000 7f3b95600640 1 -- 172.21.15.42:0/128410762 shutdown_connections 2024-05-28T13:58:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.549+0000 7f3b95600640 1 --2- 172.21.15.42:0/128410762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9006e4e0 0x7f3b900b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.549+0000 7f3b95600640 1 --2- 172.21.15.42:0/128410762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b90156ac0 0x7f3b9015d550 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.549+0000 7f3b95600640 1 -- 172.21.15.42:0/128410762 >> 172.21.15.42:0/128410762 conn(0x7f3b9006b9f0 msgr2=0x7f3b9006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.549+0000 7f3b95600640 1 -- 172.21.15.42:0/128410762 shutdown_connections 2024-05-28T13:58:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.549+0000 7f3b95600640 1 -- 172.21.15.42:0/128410762 wait complete. 2024-05-28T13:58:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b95600640 1 Processor -- start 2024-05-28T13:58:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b95600640 1 -- start start 2024-05-28T13:58:19.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b95600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9006e4e0 0x7f3b90152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b95600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b900b6440 0x7f3b901531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b95600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b90156ac0 0x7f3b9014b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b95600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90169550 con 0x7f3b90156ac0 2024-05-28T13:58:19.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b95600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901693d0 con 0x7f3b900b6440 2024-05-28T13:58:19.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b95600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3b901696d0 con 0x7f3b9006e4e0 2024-05-28T13:58:19.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b94c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b90156ac0 0x7f3b9014b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b8fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9006e4e0 0x7f3b90152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b87400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b900b6440 0x7f3b901531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b8fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9006e4e0 0x7f3b90152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:50606/0 (socket says 172.21.15.42:50606) 2024-05-28T13:58:19.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b8fe00640 1 -- 172.21.15.42:0/797683963 learned_addr learned my addr 172.21.15.42:0/797683963 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:19.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b94c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b90156ac0 0x7f3b9014b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49056/0 (socket says 172.21.15.42:49056) 2024-05-28T13:58:19.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b94c00640 1 -- 172.21.15.42:0/797683963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9006e4e0 msgr2=0x7f3b90152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b94c00640 1 --2- 172.21.15.42:0/797683963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9006e4e0 0x7f3b90152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b94c00640 1 -- 172.21.15.42:0/797683963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b900b6440 msgr2=0x7f3b901531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b94c00640 1 --2- 172.21.15.42:0/797683963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b900b6440 0x7f3b901531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b94c00640 1 -- 172.21.15.42:0/797683963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b90148a10 con 0x7f3b90156ac0 2024-05-28T13:58:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b94c00640 1 --2- 172.21.15.42:0/797683963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b90156ac0 0x7f3b9014b030 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800c3c0 tx=0x7f3b8800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b8cc00640 1 -- 172.21.15.42:0/797683963 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b88017070 con 0x7f3b90156ac0 2024-05-28T13:58:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b8cc00640 1 -- 172.21.15.42:0/797683963 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b88005db0 con 0x7f3b90156ac0 2024-05-28T13:58:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b95600640 1 -- 172.21.15.42:0/797683963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b90148cf0 con 0x7f3b90156ac0 2024-05-28T13:58:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b8cc00640 1 -- 172.21.15.42:0/797683963 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b880072f0 con 0x7f3b90156ac0 2024-05-28T13:58:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.553+0000 7f3b95600640 1 -- 172.21.15.42:0/797683963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b901492b0 con 0x7f3b90156ac0 2024-05-28T13:58:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.557+0000 7f3b95600640 1 -- 172.21.15.42:0/797683963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b9004fcf0 con 0x7f3b90156ac0 2024-05-28T13:58:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.557+0000 7f3b8cc00640 1 -- 172.21.15.42:0/797683963 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3b8801a070 con 0x7f3b90156ac0 2024-05-28T13:58:19.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.557+0000 7f3b8cc00640 1 --2- 172.21.15.42:0/797683963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b6c0683b0 0x7f3b6c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.557+0000 7f3b8cc00640 1 -- 172.21.15.42:0/797683963 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f3b880858e0 con 0x7f3b90156ac0 2024-05-28T13:58:19.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.557+0000 7f3b8fe00640 1 --2- 172.21.15.42:0/797683963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b6c0683b0 0x7f3b6c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.557+0000 7f3b8fe00640 1 --2- 172.21.15.42:0/797683963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b6c0683b0 0x7f3b6c06a870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3b8001aac0 tx=0x7f3b800a4f90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:19.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.561+0000 7f3b8cc00640 1 -- 172.21.15.42:0/797683963 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3b9006e960 con 0x7f3b90156ac0 2024-05-28T13:58:19.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.729+0000 7f3b95600640 1 -- 172.21.15.42:0/797683963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b9006fa90 con 0x7f3b90156ac0 2024-05-28T13:58:19.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.729+0000 7f3b8cc00640 1 -- 172.21.15.42:0/797683963 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12824 (secure 0 0 0) 0x7f3b88055b00 con 0x7f3b90156ac0 2024-05-28T13:58:19.716 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:19.716 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:13.603613+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.733+0000 7f3b95600640 1 -- 172.21.15.42:0/797683963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b6c0683b0 msgr2=0x7f3b6c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.733+0000 7f3b95600640 1 --2- 172.21.15.42:0/797683963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b6c0683b0 0x7f3b6c06a870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3b8001aac0 tx=0x7f3b800a4f90 comp rx=0 tx=0).stop 2024-05-28T13:58:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.733+0000 7f3b95600640 1 -- 172.21.15.42:0/797683963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b90156ac0 msgr2=0x7f3b9014b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.733+0000 7f3b95600640 1 --2- 172.21.15.42:0/797683963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b90156ac0 0x7f3b9014b030 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800c3c0 tx=0x7f3b8800c880 comp rx=0 tx=0).stop 2024-05-28T13:58:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.733+0000 7f3b95600640 1 -- 172.21.15.42:0/797683963 shutdown_connections 2024-05-28T13:58:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.733+0000 7f3b95600640 1 --2- 172.21.15.42:0/797683963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b6c0683b0 0x7f3b6c06a870 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.733+0000 7f3b95600640 1 --2- 172.21.15.42:0/797683963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b90156ac0 0x7f3b9014b030 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.733+0000 7f3b95600640 1 --2- 172.21.15.42:0/797683963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b900b6440 0x7f3b901531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.733+0000 7f3b95600640 1 --2- 172.21.15.42:0/797683963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9006e4e0 0x7f3b90152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.733+0000 7f3b95600640 1 -- 172.21.15.42:0/797683963 >> 172.21.15.42:0/797683963 conn(0x7f3b9006b9f0 msgr2=0x7f3b90154e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.733+0000 7f3b95600640 1 -- 172.21.15.42:0/797683963 shutdown_connections 2024-05-28T13:58:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.733+0000 7f3b95600640 1 -- 172.21.15.42:0/797683963 wait complete. 2024-05-28T13:58:19.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:58:19.738 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:58:19.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.897+0000 7fc4c0200640 1 Processor -- start 2024-05-28T13:58:19.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.897+0000 7fc4c0200640 1 -- start start 2024-05-28T13:58:19.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.897+0000 7fc4c0200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b809b160 0x7fc4b806cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.897+0000 7fc4c0200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b80d0770 con 0x7fc4b809b8e0 2024-05-28T13:58:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.897+0000 7fc4c0200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b80d0a10 con 0x7fc4b80b0e80 2024-05-28T13:58:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.897+0000 7fc4c0200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b80d0cf0 con 0x7fc4b809b160 2024-05-28T13:58:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.897+0000 7fc4be200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc4b809b8e0 0x7fc4b815e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:58066/0 (socket says 172.21.15.42:58066) 2024-05-28T13:58:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.897+0000 7fc4be200640 1 -- 172.21.15.42:0/117180983 learned_addr learned my addr 172.21.15.42:0/117180983 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:19.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.897+0000 7fc4bce00640 1 --2- 172.21.15.42:0/117180983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b809b160 0x7fc4b806cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.897+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 584145471 0 0) 0x7fc4b80d0770 con 0x7fc4b809b8e0 2024-05-28T13:58:19.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.897+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4a0003530 con 0x7fc4b809b8e0 2024-05-28T13:58:19.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.897+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1420615977 0 0) 0x7fc4b80d0a10 con 0x7fc4b80b0e80 2024-05-28T13:58:19.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.897+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4b80d0770 con 0x7fc4b80b0e80 2024-05-28T13:58:19.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.897+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1108787797 0 0) 0x7fc4a0003530 con 0x7fc4b809b8e0 2024-05-28T13:58:19.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b809b160 msgr2=0x7fc4b806cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:19.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4b7e00640 1 --2- 172.21.15.42:0/117180983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b809b160 0x7fc4b806cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 >> v1:172.21.15.63:6789/0 conn(0x7fc4b80b0e80 legacy=0x7fc4b809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b80d0fd0 con 0x7fc4b809b8e0 2024-05-28T13:58:19.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fc4b0001a00 con 0x7fc4b809b8e0 2024-05-28T13:58:19.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 >> v1:172.21.15.42:6789/0 conn(0x7fc4b809b8e0 legacy=0x7fc4b815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4b7e00640 1 --2- 172.21.15.42:0/117180983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b809b160 0x7fc4a0003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4b7e00640 1 --2- 172.21.15.42:0/117180983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4a00040e0 0x7fc4a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4b7e00640 1 --2- 172.21.15.42:0/117180983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4a0004d50 0x7fc4a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b80d0cf0 con 0x7fc4a0004d50 2024-05-28T13:58:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a000bab0 con 0x7fc4a00040e0 2024-05-28T13:58:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a000bcb0 con 0x7fc4b809b160 2024-05-28T13:58:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4be200640 1 --2- 172.21.15.42:0/117180983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4a0004d50 0x7fc4a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4bd800640 1 --2- 172.21.15.42:0/117180983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b809b160 0x7fc4a0003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4bce00640 1 --2- 172.21.15.42:0/117180983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4a00040e0 0x7fc4a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4be200640 1 -- 172.21.15.42:0/117180983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b809b160 msgr2=0x7fc4a0003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4be200640 1 --2- 172.21.15.42:0/117180983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b809b160 0x7fc4a0003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4be200640 1 -- 172.21.15.42:0/117180983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4a00040e0 msgr2=0x7fc4a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4be200640 1 --2- 172.21.15.42:0/117180983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4a00040e0 0x7fc4a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4be200640 1 -- 172.21.15.42:0/117180983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a0003530 con 0x7fc4a0004d50 2024-05-28T13:58:19.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4be200640 1 --2- 172.21.15.42:0/117180983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4a0004d50 0x7fc4a00051d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000a800 tx=0x7fc4b001fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d625083f59f04d01 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:19.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.901+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b000a430 con 0x7fc4a0004d50 2024-05-28T13:58:19.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b00aa690 con 0x7fc4a0004d50 2024-05-28T13:58:19.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4b7e00640 1 -- 172.21.15.42:0/117180983 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00aaac0 con 0x7fc4a0004d50 2024-05-28T13:58:19.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4c0200640 1 -- 172.21.15.42:0/117180983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4a0004d50 msgr2=0x7fc4a00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4c0200640 1 --2- 172.21.15.42:0/117180983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4a0004d50 0x7fc4a00051d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000a800 tx=0x7fc4b001fa60 comp rx=0 tx=0).stop 2024-05-28T13:58:19.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4be200640 1 -- 172.21.15.42:0/117180983 reap_dead start 2024-05-28T13:58:19.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4c0200640 1 -- 172.21.15.42:0/117180983 shutdown_connections 2024-05-28T13:58:19.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4c0200640 1 -- 172.21.15.42:0/117180983 >> 172.21.15.42:0/117180983 conn(0x7fc4b80b3f90 msgr2=0x7fc4b80b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:19.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4c0200640 1 -- 172.21.15.42:0/117180983 shutdown_connections 2024-05-28T13:58:19.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4c0200640 1 -- 172.21.15.42:0/117180983 wait complete. 2024-05-28T13:58:19.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4c0200640 1 Processor -- start 2024-05-28T13:58:19.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4c0200640 1 -- start start 2024-05-28T13:58:19.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4c0200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4a0004d50 0x7fc4b8157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4c0200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4b80c5f00 0x7fc4b80c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4c0200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b81f5ed0 0x7fc4b80b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4c0200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b80b01e0 con 0x7fc4b80c5f00 2024-05-28T13:58:19.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4c0200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b80d1590 con 0x7fc4b81f5ed0 2024-05-28T13:58:19.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4c0200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b80d1a90 con 0x7fc4a0004d50 2024-05-28T13:58:19.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4bce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4b80c5f00 0x7fc4b80c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4bce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4b80c5f00 0x7fc4b80c7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49070/0 (socket says 172.21.15.42:49070) 2024-05-28T13:58:19.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4bce00640 1 -- 172.21.15.42:0/1796093478 learned_addr learned my addr 172.21.15.42:0/1796093478 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:19.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4be200640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b81f5ed0 0x7fc4b80b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4bd800640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4a0004d50 0x7fc4b8157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4bce00640 1 -- 172.21.15.42:0/1796093478 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4a0004d50 msgr2=0x7fc4b8157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.905+0000 7fc4bce00640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4a0004d50 0x7fc4b8157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.909+0000 7fc4bce00640 1 -- 172.21.15.42:0/1796093478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b81f5ed0 msgr2=0x7fc4b80b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:19.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.909+0000 7fc4bce00640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b81f5ed0 0x7fc4b80b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:19.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.909+0000 7fc4bce00640 1 -- 172.21.15.42:0/1796093478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b80c7a30 con 0x7fc4b80c5f00 2024-05-28T13:58:19.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.909+0000 7fc4be200640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b81f5ed0 0x7fc4b80b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:19.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.909+0000 7fc4bce00640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4b80c5f00 0x7fc4b80c7da0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4011a00 tx=0x7fc4a4011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:19.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.909+0000 7fc4b5600640 1 -- 172.21.15.42:0/1796093478 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a400fb50 con 0x7fc4b80c5f00 2024-05-28T13:58:19.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.909+0000 7fc4b5600640 1 -- 172.21.15.42:0/1796093478 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a400fcf0 con 0x7fc4b80c5f00 2024-05-28T13:58:19.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.909+0000 7fc4b5600640 1 -- 172.21.15.42:0/1796093478 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a400fea0 con 0x7fc4b80c5f00 2024-05-28T13:58:19.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.909+0000 7fc4c0200640 1 -- 172.21.15.42:0/1796093478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b80c6390 con 0x7fc4b80c5f00 2024-05-28T13:58:19.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.909+0000 7fc4c0200640 1 -- 172.21.15.42:0/1796093478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b80c6950 con 0x7fc4b80c5f00 2024-05-28T13:58:19.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.909+0000 7fc4c0200640 1 -- 172.21.15.42:0/1796093478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b80a0550 con 0x7fc4b80c5f00 2024-05-28T13:58:19.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.909+0000 7fc4b5600640 1 -- 172.21.15.42:0/1796093478 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc4a4012040 con 0x7fc4b80c5f00 2024-05-28T13:58:19.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.909+0000 7fc4b5600640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc498068380 0x7fc49806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:19.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.913+0000 7fc4b5600640 1 -- 172.21.15.42:0/1796093478 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fc4a4086d00 con 0x7fc4b80c5f00 2024-05-28T13:58:19.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.913+0000 7fc4bd800640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc498068380 0x7fc49806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:19.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.913+0000 7fc4bd800640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc498068380 0x7fc49806a840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac007fd0 tx=0x7fc4ac009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:19.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:19.917+0000 7fc4b5600640 1 -- 172.21.15.42:0/1796093478 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc4b80c7a30 con 0x7fc4b80c5f00 2024-05-28T13:58:20.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.085+0000 7fc4c0200640 1 -- 172.21.15.42:0/1796093478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc4b80b80c0 con 0x7fc4b80c5f00 2024-05-28T13:58:20.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.085+0000 7fc4b5600640 1 -- 172.21.15.42:0/1796093478 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7fc4a4019030 con 0x7fc4b80c5f00 2024-05-28T13:58:20.070 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:58:20.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.085+0000 7fc4c0200640 1 -- 172.21.15.42:0/1796093478 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc498068380 msgr2=0x7fc49806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:20.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.085+0000 7fc4c0200640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc498068380 0x7fc49806a840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac007fd0 tx=0x7fc4ac009040 comp rx=0 tx=0).stop 2024-05-28T13:58:20.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.085+0000 7fc4c0200640 1 -- 172.21.15.42:0/1796093478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4b80c5f00 msgr2=0x7fc4b80c7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:20.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.085+0000 7fc4c0200640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4b80c5f00 0x7fc4b80c7da0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4011a00 tx=0x7fc4a4011ec0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.085+0000 7fc4c0200640 1 -- 172.21.15.42:0/1796093478 shutdown_connections 2024-05-28T13:58:20.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.085+0000 7fc4c0200640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc498068380 0x7fc49806a840 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.085+0000 7fc4c0200640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b81f5ed0 0x7fc4b80b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.085+0000 7fc4c0200640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4b80c5f00 0x7fc4b80c7da0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.085+0000 7fc4c0200640 1 --2- 172.21.15.42:0/1796093478 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4a0004d50 0x7fc4b8157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.085+0000 7fc4c0200640 1 -- 172.21.15.42:0/1796093478 >> 172.21.15.42:0/1796093478 conn(0x7fc4b80b3f90 msgr2=0x7fc4b8160b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:20.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.085+0000 7fc4c0200640 1 -- 172.21.15.42:0/1796093478 shutdown_connections 2024-05-28T13:58:20.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.085+0000 7fc4c0200640 1 -- 172.21.15.42:0/1796093478 wait complete. 2024-05-28T13:58:20.091 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-28T13:58:20.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.245+0000 7f920f000640 1 Processor -- start 2024-05-28T13:58:20.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.245+0000 7f920f000640 1 -- start start 2024-05-28T13:58:20.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.245+0000 7f920f000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f920806f660 0x7f9208073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:20.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f920f000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92080d0e00 0x7f920806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:20.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f920f000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f920815f280 con 0x7f920815ba40 2024-05-28T13:58:20.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f920f000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f920815f030 con 0x7f92080d0e00 2024-05-28T13:58:20.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f920f000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f920815f560 con 0x7f920806f660 2024-05-28T13:58:20.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f920d000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92080d0e00 0x7f920806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:20.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f920d000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92080d0e00 0x7f920806dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49072/0 (socket says 172.21.15.42:49072) 2024-05-28T13:58:20.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f920d000640 1 -- 172.21.15.42:0/554190890 learned_addr learned my addr 172.21.15.42:0/554190890 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:20.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f9207400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f920815ba40 0x7f92080d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:58082/0 (socket says 172.21.15.42:58082) 2024-05-28T13:58:20.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f9207e00640 1 --2- 172.21.15.42:0/554190890 >> v2:172.21.15.63:3300/0 conn(0x7f920806f660 0x7f9208073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:20.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f920d000640 1 -- 172.21.15.42:0/554190890 >> v2:172.21.15.63:3300/0 conn(0x7f920806f660 msgr2=0x7f9208073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:20.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f920d000640 1 --2- 172.21.15.42:0/554190890 >> v2:172.21.15.63:3300/0 conn(0x7f920806f660 0x7f9208073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f920d000640 1 -- 172.21.15.42:0/554190890 >> v1:172.21.15.42:6789/0 conn(0x7f920815ba40 legacy=0x7f92080d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:20.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f920d000640 1 -- 172.21.15.42:0/554190890 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920815f840 con 0x7f92080d0e00 2024-05-28T13:58:20.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f9206a00640 1 -- 172.21.15.42:0/554190890 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2489081623 0 0) 0x7f920815f280 con 0x7f920815ba40 2024-05-28T13:58:20.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f9207e00640 1 --2- 172.21.15.42:0/554190890 >> v2:172.21.15.63:3300/0 conn(0x7f920806f660 0x7f9208073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:20.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f920d000640 1 --2- 172.21.15.42:0/554190890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92080d0e00 0x7f920806dd60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f92080d13a0 tx=0x7f920001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15f73ab9dcaf534 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:20.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f9206a00640 1 -- 172.21.15.42:0/554190890 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92000a5740 con 0x7f92080d0e00 2024-05-28T13:58:20.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f9206a00640 1 -- 172.21.15.42:0/554190890 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92000a58e0 con 0x7f92080d0e00 2024-05-28T13:58:20.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.249+0000 7f9206a00640 1 -- 172.21.15.42:0/554190890 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92000a5cb0 con 0x7f92080d0e00 2024-05-28T13:58:20.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 -- 172.21.15.42:0/554190890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92080d0e00 msgr2=0x7f920806dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:20.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 --2- 172.21.15.42:0/554190890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92080d0e00 0x7f920806dd60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f92080d13a0 tx=0x7f920001c8a0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 -- 172.21.15.42:0/554190890 shutdown_connections 2024-05-28T13:58:20.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 --2- 172.21.15.42:0/554190890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92080d0e00 0x7f920806dd60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 --2- 172.21.15.42:0/554190890 >> v2:172.21.15.63:3300/0 conn(0x7f920806f660 0x7f9208073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 -- 172.21.15.42:0/554190890 >> 172.21.15.42:0/554190890 conn(0x7f920806bdf0 msgr2=0x7f920806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 -- 172.21.15.42:0/554190890 shutdown_connections 2024-05-28T13:58:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 -- 172.21.15.42:0/554190890 wait complete. 2024-05-28T13:58:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 Processor -- start 2024-05-28T13:58:20.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 -- start start 2024-05-28T13:58:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f920806f660 0x7f92080b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92080d0e00 0x7f920815df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f920809dc30 0x7f92081507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9208166540 con 0x7f92080d0e00 2024-05-28T13:58:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92081663c0 con 0x7f920809dc30 2024-05-28T13:58:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920f000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f92081666c0 con 0x7f920806f660 2024-05-28T13:58:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f9207400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92080d0e00 0x7f920815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f9207400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92080d0e00 0x7f920815df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49076/0 (socket says 172.21.15.42:49076) 2024-05-28T13:58:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f9207400640 1 -- 172.21.15.42:0/516248417 learned_addr learned my addr 172.21.15.42:0/516248417 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:20.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f9207e00640 1 --2- 172.21.15.42:0/516248417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f920806f660 0x7f92080b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:20.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f920d000640 1 --2- 172.21.15.42:0/516248417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f920809dc30 0x7f92081507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:20.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f9207400640 1 -- 172.21.15.42:0/516248417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f920806f660 msgr2=0x7f92080b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:20.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f9207400640 1 --2- 172.21.15.42:0/516248417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f920806f660 0x7f92080b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f9207400640 1 -- 172.21.15.42:0/516248417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f920809dc30 msgr2=0x7f92081507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:20.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f9207400640 1 --2- 172.21.15.42:0/516248417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f920809dc30 0x7f92081507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.253+0000 7f9207400640 1 -- 172.21.15.42:0/516248417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92080b5200 con 0x7f92080d0e00 2024-05-28T13:58:20.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.257+0000 7f9207400640 1 --2- 172.21.15.42:0/516248417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92080d0e00 0x7f920815df80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f91f400c480 tx=0x7f91f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:20.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.257+0000 7f91ffe00640 1 -- 172.21.15.42:0/516248417 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4018070 con 0x7f92080d0e00 2024-05-28T13:58:20.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.257+0000 7f91ffe00640 1 -- 172.21.15.42:0/516248417 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f4007500 con 0x7f92080d0e00 2024-05-28T13:58:20.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.257+0000 7f91ffe00640 1 -- 172.21.15.42:0/516248417 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4004a30 con 0x7f92080d0e00 2024-05-28T13:58:20.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.257+0000 7f920f000640 1 -- 172.21.15.42:0/516248417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920809e150 con 0x7f92080d0e00 2024-05-28T13:58:20.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.257+0000 7f920f000640 1 -- 172.21.15.42:0/516248417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92080cf340 con 0x7f92080d0e00 2024-05-28T13:58:20.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.257+0000 7f920f000640 1 -- 172.21.15.42:0/516248417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9208068a90 con 0x7f92080d0e00 2024-05-28T13:58:20.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.257+0000 7f91ffe00640 1 -- 172.21.15.42:0/516248417 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f91f4004bd0 con 0x7f92080d0e00 2024-05-28T13:58:20.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.257+0000 7f91ffe00640 1 --2- 172.21.15.42:0/516248417 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f91e8068380 0x7f91e806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:20.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.257+0000 7f91ffe00640 1 -- 172.21.15.42:0/516248417 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f91f4085940 con 0x7f92080d0e00 2024-05-28T13:58:20.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.257+0000 7f9207e00640 1 --2- 172.21.15.42:0/516248417 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f91e8068380 0x7f91e806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:20.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.261+0000 7f9207e00640 1 --2- 172.21.15.42:0/516248417 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f91e8068380 0x7f91e806a840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f920815e960 tx=0x7f91f8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:20.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.261+0000 7f91ffe00640 1 -- 172.21.15.42:0/516248417 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f91f4056240 con 0x7f92080d0e00 2024-05-28T13:58:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.429+0000 7f920f000640 1 -- 172.21.15.42:0/516248417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9208151610 con 0x7f92080d0e00 2024-05-28T13:58:20.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.429+0000 7f91ffe00640 1 -- 172.21.15.42:0/516248417 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f91f4055b40 con 0x7f92080d0e00 2024-05-28T13:58:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:20.416 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T13:58:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:58:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-05-28T13:58:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-05-28T13:58:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:58:20.416 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T13:58:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.433+0000 7f920f000640 1 -- 172.21.15.42:0/516248417 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f91e8068380 msgr2=0x7f91e806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.433+0000 7f920f000640 1 --2- 172.21.15.42:0/516248417 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f91e8068380 0x7f91e806a840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f920815e960 tx=0x7f91f8008040 comp rx=0 tx=0).stop 2024-05-28T13:58:20.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.433+0000 7f920f000640 1 -- 172.21.15.42:0/516248417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92080d0e00 msgr2=0x7f920815df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.433+0000 7f920f000640 1 --2- 172.21.15.42:0/516248417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92080d0e00 0x7f920815df80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f91f400c480 tx=0x7f91f400c940 comp rx=0 tx=0).stop 2024-05-28T13:58:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.433+0000 7f920f000640 1 -- 172.21.15.42:0/516248417 shutdown_connections 2024-05-28T13:58:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.433+0000 7f920f000640 1 --2- 172.21.15.42:0/516248417 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f91e8068380 0x7f91e806a840 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.433+0000 7f920f000640 1 --2- 172.21.15.42:0/516248417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f920809dc30 0x7f92081507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.433+0000 7f920f000640 1 --2- 172.21.15.42:0/516248417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92080d0e00 0x7f920815df80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.433+0000 7f920f000640 1 --2- 172.21.15.42:0/516248417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f920806f660 0x7f92080b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.433+0000 7f920f000640 1 -- 172.21.15.42:0/516248417 >> 172.21.15.42:0/516248417 conn(0x7f920806bdf0 msgr2=0x7f920806c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.433+0000 7f920f000640 1 -- 172.21.15.42:0/516248417 shutdown_connections 2024-05-28T13:58:20.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.433+0000 7f920f000640 1 -- 172.21.15.42:0/516248417 wait complete. 2024-05-28T13:58:20.437 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-28T13:58:20.437 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-28T13:58:20.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7368200640 1 Processor -- start 2024-05-28T13:58:20.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7368200640 1 -- start start 2024-05-28T13:58:20.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7368200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f736006e740 0x7f73600cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:20.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7368200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f73600ce110 0x7f736009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:20.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7368200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f736006ff10 con 0x7f736006c940 2024-05-28T13:58:20.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7368200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f736006eb20 con 0x7f736006e740 2024-05-28T13:58:20.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7368200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f73600701b0 con 0x7f73600ce110 2024-05-28T13:58:20.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7365800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f736006e740 0x7f73600cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:20.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7366200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f736006c940 0x7f7360070f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:58084/0 (socket says 172.21.15.42:58084) 2024-05-28T13:58:20.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7366200640 1 -- 172.21.15.42:0/2471509053 learned_addr learned my addr 172.21.15.42:0/2471509053 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:20.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7364e00640 1 --2- 172.21.15.42:0/2471509053 >> v2:172.21.15.63:3300/0 conn(0x7f73600ce110 0x7f736009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:20.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7365800640 1 -- 172.21.15.42:0/2471509053 >> v2:172.21.15.63:3300/0 conn(0x7f73600ce110 msgr2=0x7f736009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:20.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7365800640 1 --2- 172.21.15.42:0/2471509053 >> v2:172.21.15.63:3300/0 conn(0x7f73600ce110 0x7f736009aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7365800640 1 -- 172.21.15.42:0/2471509053 >> v1:172.21.15.42:6789/0 conn(0x7f736006c940 legacy=0x7f7360070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:20.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f7365800640 1 -- 172.21.15.42:0/2471509053 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7360070490 con 0x7f736006e740 2024-05-28T13:58:20.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.597+0000 7f735fe00640 1 -- 172.21.15.42:0/2471509053 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1240948380 0 0) 0x7f736006ff10 con 0x7f736006c940 2024-05-28T13:58:20.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.601+0000 7f7365800640 1 --2- 172.21.15.42:0/2471509053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f736006e740 0x7f73600cdbd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f73600734f0 tx=0x7f735401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6477b77e4ed680 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:20.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.601+0000 7f735fe00640 1 -- 172.21.15.42:0/2471509053 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73540b2070 con 0x7f736006e740 2024-05-28T13:58:20.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.601+0000 7f735fe00640 1 -- 172.21.15.42:0/2471509053 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73540a9070 con 0x7f736006e740 2024-05-28T13:58:20.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.601+0000 7f735fe00640 1 -- 172.21.15.42:0/2471509053 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73540ad880 con 0x7f736006e740 2024-05-28T13:58:20.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.601+0000 7f7368200640 1 -- 172.21.15.42:0/2471509053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f736006e740 msgr2=0x7f73600cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:20.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.601+0000 7f7368200640 1 --2- 172.21.15.42:0/2471509053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f736006e740 0x7f73600cdbd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f73600734f0 tx=0x7f735401c450 comp rx=0 tx=0).stop 2024-05-28T13:58:20.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.601+0000 7f7368200640 1 -- 172.21.15.42:0/2471509053 shutdown_connections 2024-05-28T13:58:20.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.601+0000 7f7368200640 1 --2- 172.21.15.42:0/2471509053 >> v2:172.21.15.63:3300/0 conn(0x7f73600ce110 0x7f736009aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.601+0000 7f7368200640 1 --2- 172.21.15.42:0/2471509053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f736006e740 0x7f73600cdbd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.601+0000 7f7368200640 1 -- 172.21.15.42:0/2471509053 >> 172.21.15.42:0/2471509053 conn(0x7f73600b3fa0 msgr2=0x7f73600b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:20.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.601+0000 7f7368200640 1 -- 172.21.15.42:0/2471509053 shutdown_connections 2024-05-28T13:58:20.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.601+0000 7f7368200640 1 -- 172.21.15.42:0/2471509053 wait complete. 2024-05-28T13:58:20.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.601+0000 7f7368200640 1 Processor -- start 2024-05-28T13:58:20.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.601+0000 7f7368200640 1 -- start start 2024-05-28T13:58:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7368200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f736006c940 0x7f73600c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7368200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f736006e740 0x7f736014fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7368200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73600ce110 0x7f73600b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7368200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f736015e2a0 con 0x7f736006c940 2024-05-28T13:58:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7368200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f736015e120 con 0x7f73600ce110 2024-05-28T13:58:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7368200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f736015e420 con 0x7f736006e740 2024-05-28T13:58:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7365800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f736006c940 0x7f73600c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7365800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f736006c940 0x7f73600c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49104/0 (socket says 172.21.15.42:49104) 2024-05-28T13:58:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7365800640 1 -- 172.21.15.42:0/1693926113 learned_addr learned my addr 172.21.15.42:0/1693926113 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7366200640 1 --2- 172.21.15.42:0/1693926113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73600ce110 0x7f73600b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7364e00640 1 --2- 172.21.15.42:0/1693926113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f736006e740 0x7f736014fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7365800640 1 -- 172.21.15.42:0/1693926113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f736006e740 msgr2=0x7f736014fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7365800640 1 --2- 172.21.15.42:0/1693926113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f736006e740 0x7f736014fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7365800640 1 -- 172.21.15.42:0/1693926113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73600ce110 msgr2=0x7f73600b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7365800640 1 --2- 172.21.15.42:0/1693926113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73600ce110 0x7f73600b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7365800640 1 -- 172.21.15.42:0/1693926113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73600b0830 con 0x7f736006c940 2024-05-28T13:58:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7365800640 1 --2- 172.21.15.42:0/1693926113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f736006c940 0x7f73600c9580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f735401a080 tx=0x7f73540191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f735d600640 1 -- 172.21.15.42:0/1693926113 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73540b2040 con 0x7f736006c940 2024-05-28T13:58:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f735d600640 1 -- 172.21.15.42:0/1693926113 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73540bb070 con 0x7f736006c940 2024-05-28T13:58:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f735d600640 1 -- 172.21.15.42:0/1693926113 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73540b6890 con 0x7f736006c940 2024-05-28T13:58:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7368200640 1 -- 172.21.15.42:0/1693926113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73600b0270 con 0x7f736006c940 2024-05-28T13:58:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7368200640 1 -- 172.21.15.42:0/1693926113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73600cbe20 con 0x7f736006c940 2024-05-28T13:58:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.605+0000 7f7368200640 1 -- 172.21.15.42:0/1693926113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7360068a90 con 0x7f736006c940 2024-05-28T13:58:20.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.609+0000 7f735d600640 1 -- 172.21.15.42:0/1693926113 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f73540ab070 con 0x7f736006c940 2024-05-28T13:58:20.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.609+0000 7f735d600640 1 --2- 172.21.15.42:0/1693926113 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7340068380 0x7f734006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:20.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.609+0000 7f735d600640 1 -- 172.21.15.42:0/1693926113 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f7354120470 con 0x7f736006c940 2024-05-28T13:58:20.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.609+0000 7f7364e00640 1 --2- 172.21.15.42:0/1693926113 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7340068380 0x7f734006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:20.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.609+0000 7f7364e00640 1 --2- 172.21.15.42:0/1693926113 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7340068380 0x7f734006a840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7360071090 tx=0x7f734c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:20.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.613+0000 7f735d600640 1 -- 172.21.15.42:0/1693926113 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7354016070 con 0x7f736006c940 2024-05-28T13:58:20.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:20.777+0000 7f7368200640 1 -- 172.21.15.42:0/1693926113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7360157030 con 0x7f736006c940 2024-05-28T13:58:21.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.641+0000 7f735d600640 1 -- 172.21.15.42:0/1693926113 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f73540f0e10 con 0x7f736006c940 2024-05-28T13:58:21.628 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-05-28T13:58:21.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.645+0000 7f7368200640 1 -- 172.21.15.42:0/1693926113 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7340068380 msgr2=0x7f734006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:21.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.645+0000 7f7368200640 1 --2- 172.21.15.42:0/1693926113 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7340068380 0x7f734006a840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7360071090 tx=0x7f734c008040 comp rx=0 tx=0).stop 2024-05-28T13:58:21.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.645+0000 7f7368200640 1 -- 172.21.15.42:0/1693926113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f736006c940 msgr2=0x7f73600c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:21.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.645+0000 7f7368200640 1 --2- 172.21.15.42:0/1693926113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f736006c940 0x7f73600c9580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f735401a080 tx=0x7f73540191f0 comp rx=0 tx=0).stop 2024-05-28T13:58:21.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.645+0000 7f7368200640 1 -- 172.21.15.42:0/1693926113 shutdown_connections 2024-05-28T13:58:21.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.645+0000 7f7368200640 1 --2- 172.21.15.42:0/1693926113 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7340068380 0x7f734006a840 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:21.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.645+0000 7f7368200640 1 --2- 172.21.15.42:0/1693926113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73600ce110 0x7f73600b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:21.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.645+0000 7f7368200640 1 --2- 172.21.15.42:0/1693926113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f736006e740 0x7f736014fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:21.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.645+0000 7f7368200640 1 --2- 172.21.15.42:0/1693926113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f736006c940 0x7f73600c9580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:21.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.645+0000 7f7368200640 1 -- 172.21.15.42:0/1693926113 >> 172.21.15.42:0/1693926113 conn(0x7f73600b3fa0 msgr2=0x7f73600d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:21.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.645+0000 7f7368200640 1 -- 172.21.15.42:0/1693926113 shutdown_connections 2024-05-28T13:58:21.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.645+0000 7f7368200640 1 -- 172.21.15.42:0/1693926113 wait complete. 2024-05-28T13:58:21.661 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-28T13:58:21.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36f1c00640 1 Processor -- start 2024-05-28T13:58:21.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36f1c00640 1 -- start start 2024-05-28T13:58:21.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36f1c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f36ec071de0 0x7f36ec0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:21.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36f1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ec0ce3b0 0x7f36ec0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:21.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36f1c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36ec072700 con 0x7f36ec141d10 2024-05-28T13:58:21.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36f1c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f36ec160230 con 0x7f36ec071de0 2024-05-28T13:58:21.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36f1c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f36ec160510 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36eaa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f36ec071de0 0x7f36ec0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:21.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36e2000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ec0ce3b0 0x7f36ec0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:21.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36e2000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ec0ce3b0 0x7f36ec0ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:50690/0 (socket says 172.21.15.42:50690) 2024-05-28T13:58:21.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36e2000640 1 -- 172.21.15.42:0/4291065381 learned_addr learned my addr 172.21.15.42:0/4291065381 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:21.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36eb400640 1 --1- 172.21.15.42:0/4291065381 >> v1:172.21.15.63:6789/0 conn(0x7f36ec141d10 0x7f36ec1421a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:58566/0 (socket says 172.21.15.42:58566) 2024-05-28T13:58:21.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36e2000640 1 -- 172.21.15.42:0/4291065381 >> v2:172.21.15.63:3300/0 conn(0x7f36ec071de0 msgr2=0x7f36ec0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:21.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36e2000640 1 --2- 172.21.15.42:0/4291065381 >> v2:172.21.15.63:3300/0 conn(0x7f36ec071de0 0x7f36ec0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:21.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36e2000640 1 -- 172.21.15.42:0/4291065381 >> v1:172.21.15.63:6789/0 conn(0x7f36ec141d10 legacy=0x7f36ec1421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:21.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.829+0000 7f36e2000640 1 -- 172.21.15.42:0/4291065381 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec1607f0 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.833+0000 7f36e2000640 1 --2- 172.21.15.42:0/4291065381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ec0ce3b0 0x7f36ec0ce810 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009870 tx=0x7f36d401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14269d8295c3a58b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:21.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.833+0000 7f36ea000640 1 -- 172.21.15.42:0/4291065381 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40b0070 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.833+0000 7f36ea000640 1 -- 172.21.15.42:0/4291065381 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36d40ab600 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.833+0000 7f36ea000640 1 -- 172.21.15.42:0/4291065381 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40ab9d0 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.833+0000 7f36f1c00640 1 -- 172.21.15.42:0/4291065381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ec0ce3b0 msgr2=0x7f36ec0ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:21.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.833+0000 7f36f1c00640 1 --2- 172.21.15.42:0/4291065381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ec0ce3b0 0x7f36ec0ce810 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009870 tx=0x7f36d401c440 comp rx=0 tx=0).stop 2024-05-28T13:58:21.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.833+0000 7f36f1c00640 1 -- 172.21.15.42:0/4291065381 shutdown_connections 2024-05-28T13:58:21.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.833+0000 7f36f1c00640 1 --2- 172.21.15.42:0/4291065381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ec0ce3b0 0x7f36ec0ce810 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:21.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.833+0000 7f36f1c00640 1 --2- 172.21.15.42:0/4291065381 >> v2:172.21.15.63:3300/0 conn(0x7f36ec071de0 0x7f36ec0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:21.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.833+0000 7f36f1c00640 1 -- 172.21.15.42:0/4291065381 >> 172.21.15.42:0/4291065381 conn(0x7f36ec0b21f0 msgr2=0x7f36ec0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:21.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.833+0000 7f36f1c00640 1 -- 172.21.15.42:0/4291065381 shutdown_connections 2024-05-28T13:58:21.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.833+0000 7f36f1c00640 1 -- 172.21.15.42:0/4291065381 wait complete. 2024-05-28T13:58:21.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36f1c00640 1 Processor -- start 2024-05-28T13:58:21.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36f1c00640 1 -- start start 2024-05-28T13:58:21.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36f1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36ec071de0 0x7f36ec154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36f1c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ec0ce3b0 0x7f36ec145660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36f1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ec141d10 0x7f36ec152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36f1c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec169e30 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36f1c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec169cb0 con 0x7f36ec071de0 2024-05-28T13:58:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36f1c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f36ec169fb0 con 0x7f36ec141d10 2024-05-28T13:58:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36e2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ec0ce3b0 0x7f36ec145660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36e2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ec0ce3b0 0x7f36ec145660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49116/0 (socket says 172.21.15.42:49116) 2024-05-28T13:58:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36e2000640 1 -- 172.21.15.42:0/1558247292 learned_addr learned my addr 172.21.15.42:0/1558247292 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36eaa00640 1 --2- 172.21.15.42:0/1558247292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36ec071de0 0x7f36ec154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36eb400640 1 --2- 172.21.15.42:0/1558247292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ec141d10 0x7f36ec152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36e2000640 1 -- 172.21.15.42:0/1558247292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ec141d10 msgr2=0x7f36ec152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36e2000640 1 --2- 172.21.15.42:0/1558247292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ec141d10 0x7f36ec152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36e2000640 1 -- 172.21.15.42:0/1558247292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36ec071de0 msgr2=0x7f36ec154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36e2000640 1 --2- 172.21.15.42:0/1558247292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36ec071de0 0x7f36ec154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:21.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36e2000640 1 -- 172.21.15.42:0/1558247292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec155440 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36e2000640 1 --2- 172.21.15.42:0/1558247292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ec0ce3b0 0x7f36ec145660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36d401a070 tx=0x7f36d401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:21.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36e3400640 1 -- 172.21.15.42:0/1558247292 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40ba020 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36e3400640 1 -- 172.21.15.42:0/1558247292 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36d40b6420 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36e3400640 1 -- 172.21.15.42:0/1558247292 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40b0040 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36f1c00640 1 -- 172.21.15.42:0/1558247292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ec155710 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.837+0000 7f36f1c00640 1 -- 172.21.15.42:0/1558247292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ec155cd0 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.841+0000 7f36f1c00640 1 -- 172.21.15.42:0/1558247292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36ec0738b0 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.841+0000 7f36e3400640 1 -- 172.21.15.42:0/1558247292 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f36d40bd020 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.841+0000 7f36e3400640 1 --2- 172.21.15.42:0/1558247292 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36c8068380 0x7f36c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.841+0000 7f36e3400640 1 -- 172.21.15.42:0/1558247292 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f36d4120880 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.841+0000 7f36eaa00640 1 --2- 172.21.15.42:0/1558247292 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36c8068380 0x7f36c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.841+0000 7f36eaa00640 1 --2- 172.21.15.42:0/1558247292 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36c8068380 0x7f36c806a840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f36ec15d160 tx=0x7f36dc009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:21.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:21.845+0000 7f36e3400640 1 -- 172.21.15.42:0/1558247292 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f36d40c4030 con 0x7f36ec0ce3b0 2024-05-28T13:58:21.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.009+0000 7f36f1c00640 1 -- 172.21.15.42:0/1558247292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f36ec04fdd0 con 0x7f36ec0ce3b0 2024-05-28T13:58:22.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.645+0000 7f36e3400640 1 -- 172.21.15.42:0/1558247292 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7f36d4016150 con 0x7f36ec0ce3b0 2024-05-28T13:58:22.631 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-05-28T13:58:22.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.649+0000 7f36f1c00640 1 -- 172.21.15.42:0/1558247292 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36c8068380 msgr2=0x7f36c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:22.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.649+0000 7f36f1c00640 1 --2- 172.21.15.42:0/1558247292 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36c8068380 0x7f36c806a840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f36ec15d160 tx=0x7f36dc009190 comp rx=0 tx=0).stop 2024-05-28T13:58:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.649+0000 7f36f1c00640 1 -- 172.21.15.42:0/1558247292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ec0ce3b0 msgr2=0x7f36ec145660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.649+0000 7f36f1c00640 1 --2- 172.21.15.42:0/1558247292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ec0ce3b0 0x7f36ec145660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36d401a070 tx=0x7f36d401ac80 comp rx=0 tx=0).stop 2024-05-28T13:58:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.649+0000 7f36f1c00640 1 -- 172.21.15.42:0/1558247292 shutdown_connections 2024-05-28T13:58:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.649+0000 7f36f1c00640 1 --2- 172.21.15.42:0/1558247292 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36c8068380 0x7f36c806a840 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.649+0000 7f36f1c00640 1 --2- 172.21.15.42:0/1558247292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ec141d10 0x7f36ec152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.649+0000 7f36f1c00640 1 --2- 172.21.15.42:0/1558247292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ec0ce3b0 0x7f36ec145660 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.649+0000 7f36f1c00640 1 --2- 172.21.15.42:0/1558247292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36ec071de0 0x7f36ec154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.649+0000 7f36f1c00640 1 -- 172.21.15.42:0/1558247292 >> 172.21.15.42:0/1558247292 conn(0x7f36ec0b21f0 msgr2=0x7f36ec15b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.649+0000 7f36f1c00640 1 -- 172.21.15.42:0/1558247292 shutdown_connections 2024-05-28T13:58:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.649+0000 7f36f1c00640 1 -- 172.21.15.42:0/1558247292 wait complete. 2024-05-28T13:58:22.675 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-28T13:58:22.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.837+0000 7fa9e6e00640 1 Processor -- start 2024-05-28T13:58:22.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.837+0000 7fa9e6e00640 1 -- start start 2024-05-28T13:58:22.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.837+0000 7fa9e6e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa9e009b160 0x7fa9e006cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:22.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.837+0000 7fa9e6e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9e009b8e0 0x7fa9e015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:22.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.837+0000 7fa9e6e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e00d0770 con 0x7fa9e00b0e80 2024-05-28T13:58:22.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.837+0000 7fa9e6e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e00d0a10 con 0x7fa9e009b160 2024-05-28T13:58:22.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.837+0000 7fa9e6e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e00d0cf0 con 0x7fa9e009b8e0 2024-05-28T13:58:22.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.837+0000 7fa9d7400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa9e009b160 0x7fa9e006cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:22.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.837+0000 7fa9d7400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa9e009b160 0x7fa9e006cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49124/0 (socket says 172.21.15.42:49124) 2024-05-28T13:58:22.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.837+0000 7fa9d7400640 1 -- 172.21.15.42:0/1863112156 learned_addr learned my addr 172.21.15.42:0/1863112156 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:22.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.837+0000 7fa9e4e00640 1 --2- 172.21.15.42:0/1863112156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9e009b8e0 0x7fa9e015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:22.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.841+0000 7fa9d7400640 1 -- 172.21.15.42:0/1863112156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9e009b8e0 msgr2=0x7fa9e015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:22.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.841+0000 7fa9d7400640 1 --2- 172.21.15.42:0/1863112156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9e009b8e0 0x7fa9e015e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:22.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.841+0000 7fa9d7400640 1 -- 172.21.15.42:0/1863112156 >> v1:172.21.15.63:6789/0 conn(0x7fa9e00b0e80 legacy=0x7fa9e009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:22.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.841+0000 7fa9d7400640 1 -- 172.21.15.42:0/1863112156 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e00d0fd0 con 0x7fa9e009b160 2024-05-28T13:58:22.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.841+0000 7fa9d7400640 1 --2- 172.21.15.42:0/1863112156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9e009b160 0x7fa9e006cb00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009870 tx=0x7fa9c801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246b745b039adbf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:22.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.841+0000 7fa9df400640 1 -- 172.21.15.42:0/1863112156 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80b2070 con 0x7fa9e009b160 2024-05-28T13:58:22.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.841+0000 7fa9df400640 1 -- 172.21.15.42:0/1863112156 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c80a9070 con 0x7fa9e009b160 2024-05-28T13:58:22.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.841+0000 7fa9df400640 1 -- 172.21.15.42:0/1863112156 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80ad880 con 0x7fa9e009b160 2024-05-28T13:58:22.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.841+0000 7fa9e6e00640 1 -- 172.21.15.42:0/1863112156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9e009b160 msgr2=0x7fa9e006cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:22.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.841+0000 7fa9e6e00640 1 --2- 172.21.15.42:0/1863112156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9e009b160 0x7fa9e006cb00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009870 tx=0x7fa9c801c560 comp rx=0 tx=0).stop 2024-05-28T13:58:22.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e6e00640 1 -- 172.21.15.42:0/1863112156 shutdown_connections 2024-05-28T13:58:22.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e6e00640 1 --2- 172.21.15.42:0/1863112156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9e009b8e0 0x7fa9e015e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:22.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e6e00640 1 --2- 172.21.15.42:0/1863112156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9e009b160 0x7fa9e006cb00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:22.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e6e00640 1 -- 172.21.15.42:0/1863112156 >> 172.21.15.42:0/1863112156 conn(0x7fa9e00b3f90 msgr2=0x7fa9e00b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:22.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e6e00640 1 -- 172.21.15.42:0/1863112156 shutdown_connections 2024-05-28T13:58:22.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e6e00640 1 -- 172.21.15.42:0/1863112156 wait complete. 2024-05-28T13:58:22.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e6e00640 1 Processor -- start 2024-05-28T13:58:22.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e6e00640 1 -- start start 2024-05-28T13:58:22.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e6e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9e009b160 0x7fa9e0147fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:22.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e6e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9e009b8e0 0x7fa9e01523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:22.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e6e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9e00b0e80 0x7fa9e01584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:22.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e6e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e00af350 con 0x7fa9e00b0e80 2024-05-28T13:58:22.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e6e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e00af1d0 con 0x7fa9e009b160 2024-05-28T13:58:22.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e6e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e00af4d0 con 0x7fa9e009b8e0 2024-05-28T13:58:22.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9e00b0e80 0x7fa9e01584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:22.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9e00b0e80 0x7fa9e01584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49126/0 (socket says 172.21.15.42:49126) 2024-05-28T13:58:22.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9e4e00640 1 -- 172.21.15.42:0/877574282 learned_addr learned my addr 172.21.15.42:0/877574282 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:22.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9dfe00640 1 --2- 172.21.15.42:0/877574282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9e009b160 0x7fa9e0147fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:22.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9d7400640 1 --2- 172.21.15.42:0/877574282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9e009b8e0 0x7fa9e01523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:22.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9d7400640 1 -- 172.21.15.42:0/877574282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9e009b160 msgr2=0x7fa9e0147fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:22.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9d7400640 1 --2- 172.21.15.42:0/877574282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9e009b160 0x7fa9e0147fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:22.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9d7400640 1 -- 172.21.15.42:0/877574282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9e00b0e80 msgr2=0x7fa9e01584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:22.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9d7400640 1 --2- 172.21.15.42:0/877574282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9e00b0e80 0x7fa9e01584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:22.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.845+0000 7fa9d7400640 1 -- 172.21.15.42:0/877574282 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e0153760 con 0x7fa9e009b8e0 2024-05-28T13:58:22.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.849+0000 7fa9d7400640 1 --2- 172.21.15.42:0/877574282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9e009b8e0 0x7fa9e01523f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80ac000 tx=0x7fa9c8019300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:22.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.849+0000 7fa9dcc00640 1 -- 172.21.15.42:0/877574282 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80b2040 con 0x7fa9e009b8e0 2024-05-28T13:58:22.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.849+0000 7fa9dcc00640 1 -- 172.21.15.42:0/877574282 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c80adcd0 con 0x7fa9e009b8e0 2024-05-28T13:58:22.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.849+0000 7fa9e6e00640 1 -- 172.21.15.42:0/877574282 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e0153a40 con 0x7fa9e009b8e0 2024-05-28T13:58:22.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.849+0000 7fa9dcc00640 1 -- 172.21.15.42:0/877574282 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80b6660 con 0x7fa9e009b8e0 2024-05-28T13:58:22.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.849+0000 7fa9e6e00640 1 -- 172.21.15.42:0/877574282 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e0154000 con 0x7fa9e009b8e0 2024-05-28T13:58:22.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.849+0000 7fa9e6e00640 1 -- 172.21.15.42:0/877574282 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e00a0550 con 0x7fa9e009b8e0 2024-05-28T13:58:22.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.849+0000 7fa9dcc00640 1 -- 172.21.15.42:0/877574282 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa9c80ab070 con 0x7fa9e009b8e0 2024-05-28T13:58:22.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.849+0000 7fa9dcc00640 1 --2- 172.21.15.42:0/877574282 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa9bc068380 0x7fa9bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:22.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.849+0000 7fa9dcc00640 1 -- 172.21.15.42:0/877574282 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7fa9c811f510 con 0x7fa9e009b8e0 2024-05-28T13:58:22.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.849+0000 7fa9dfe00640 1 --2- 172.21.15.42:0/877574282 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa9bc068380 0x7fa9bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:22.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.853+0000 7fa9dfe00640 1 --2- 172.21.15.42:0/877574282 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa9bc068380 0x7fa9bc06a840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0001770 tx=0x7fa9d0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:22.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:22.873+0000 7fa9dcc00640 1 -- 172.21.15.42:0/877574282 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa9c80bc060 con 0x7fa9e009b8e0 2024-05-28T13:58:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.073+0000 7fa9e6e00640 1 -- 172.21.15.42:0/877574282 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa9e00b16a0 con 0x7fa9e009b8e0 2024-05-28T13:58:23.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.685+0000 7fa9dcc00640 1 -- 172.21.15.42:0/877574282 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fa9c8016150 con 0x7fa9e009b8e0 2024-05-28T13:58:23.671 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 2 and data pool 3 2024-05-28T13:58:23.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.689+0000 7fa9e6e00640 1 -- 172.21.15.42:0/877574282 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa9bc068380 msgr2=0x7fa9bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:23.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.689+0000 7fa9e6e00640 1 --2- 172.21.15.42:0/877574282 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa9bc068380 0x7fa9bc06a840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0001770 tx=0x7fa9d0009040 comp rx=0 tx=0).stop 2024-05-28T13:58:23.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.689+0000 7fa9e6e00640 1 -- 172.21.15.42:0/877574282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9e009b8e0 msgr2=0x7fa9e01523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:23.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.689+0000 7fa9e6e00640 1 --2- 172.21.15.42:0/877574282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9e009b8e0 0x7fa9e01523f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80ac000 tx=0x7fa9c8019300 comp rx=0 tx=0).stop 2024-05-28T13:58:23.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.689+0000 7fa9e6e00640 1 -- 172.21.15.42:0/877574282 shutdown_connections 2024-05-28T13:58:23.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.689+0000 7fa9e6e00640 1 --2- 172.21.15.42:0/877574282 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa9bc068380 0x7fa9bc06a840 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:23.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.689+0000 7fa9e6e00640 1 --2- 172.21.15.42:0/877574282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9e00b0e80 0x7fa9e01584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:23.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.689+0000 7fa9e6e00640 1 --2- 172.21.15.42:0/877574282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9e009b8e0 0x7fa9e01523f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:23.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.689+0000 7fa9e6e00640 1 --2- 172.21.15.42:0/877574282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9e009b160 0x7fa9e0147fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:23.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.689+0000 7fa9e6e00640 1 -- 172.21.15.42:0/877574282 >> 172.21.15.42:0/877574282 conn(0x7fa9e00b3f90 msgr2=0x7fa9e006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:23.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.689+0000 7fa9e6e00640 1 -- 172.21.15.42:0/877574282 shutdown_connections 2024-05-28T13:58:23.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.689+0000 7fa9e6e00640 1 -- 172.21.15.42:0/877574282 wait complete. 2024-05-28T13:58:23.706 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:23.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.865+0000 7f4828400640 1 Processor -- start 2024-05-28T13:58:23.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.865+0000 7f4828400640 1 -- start start 2024-05-28T13:58:23.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.865+0000 7f4828400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f482015c990 0x7f48200d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:23.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.865+0000 7f4828400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48200cd300 con 0x7f48200d06a0 2024-05-28T13:58:23.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.865+0000 7f4828400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48200cd5e0 con 0x7f482009bbd0 2024-05-28T13:58:23.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.865+0000 7f4828400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f482015cd70 con 0x7f482015c990 2024-05-28T13:58:23.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.865+0000 7f4826400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f48200d06a0 0x7f48200b53e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:58094/0 (socket says 172.21.15.42:58094) 2024-05-28T13:58:23.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.865+0000 7f4826400640 1 -- 172.21.15.42:0/2693209719 learned_addr learned my addr 172.21.15.42:0/2693209719 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:23.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.865+0000 7f4825a00640 1 --2- 172.21.15.42:0/2693209719 >> v2:172.21.15.63:3300/0 conn(0x7f482015c990 0x7f48200d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:23.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.865+0000 7f481fe00640 1 -- 172.21.15.42:0/2693209719 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3060226022 0 0) 0x7f48200cd300 con 0x7f48200d06a0 2024-05-28T13:58:23.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.865+0000 7f481fe00640 1 -- 172.21.15.42:0/2693209719 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4808003530 con 0x7f48200d06a0 2024-05-28T13:58:23.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.865+0000 7f4825a00640 1 -- 172.21.15.42:0/2693209719 >> v1:172.21.15.63:6789/0 conn(0x7f482009bbd0 legacy=0x7f482009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:23.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.865+0000 7f4825a00640 1 -- 172.21.15.42:0/2693209719 >> v1:172.21.15.42:6789/0 conn(0x7f48200d06a0 legacy=0x7f48200b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:23.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.865+0000 7f4825a00640 1 -- 172.21.15.42:0/2693209719 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48200cd8c0 con 0x7f482015c990 2024-05-28T13:58:23.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.869+0000 7f4825a00640 1 --2- 172.21.15.42:0/2693209719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f482015c990 0x7f48200d0160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48140097c0 tx=0x7f481401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=546ed8d8a102d549 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:23.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.869+0000 7f481fe00640 1 -- 172.21.15.42:0/2693209719 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48140a7070 con 0x7f482015c990 2024-05-28T13:58:23.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.869+0000 7f481fe00640 1 -- 172.21.15.42:0/2693209719 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48140b0070 con 0x7f482015c990 2024-05-28T13:58:23.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.869+0000 7f481fe00640 1 -- 172.21.15.42:0/2693209719 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48140ab890 con 0x7f482015c990 2024-05-28T13:58:23.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.869+0000 7f4828400640 1 -- 172.21.15.42:0/2693209719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f482015c990 msgr2=0x7f48200d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:23.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.869+0000 7f4828400640 1 --2- 172.21.15.42:0/2693209719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f482015c990 0x7f48200d0160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48140097c0 tx=0x7f481401c470 comp rx=0 tx=0).stop 2024-05-28T13:58:23.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.869+0000 7f4828400640 1 -- 172.21.15.42:0/2693209719 shutdown_connections 2024-05-28T13:58:23.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.869+0000 7f4828400640 1 --2- 172.21.15.42:0/2693209719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f482015c990 0x7f48200d0160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:23.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.869+0000 7f4828400640 1 -- 172.21.15.42:0/2693209719 >> 172.21.15.42:0/2693209719 conn(0x7f48200b4080 msgr2=0x7f48200b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:23.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.869+0000 7f4828400640 1 -- 172.21.15.42:0/2693209719 shutdown_connections 2024-05-28T13:58:23.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.869+0000 7f4828400640 1 -- 172.21.15.42:0/2693209719 wait complete. 2024-05-28T13:58:23.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4828400640 1 Processor -- start 2024-05-28T13:58:23.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4828400640 1 -- start start 2024-05-28T13:58:23.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4828400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48200d06a0 0x7f4820159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4828400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f482015c990 0x7f48200af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4828400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f482015b790 0x7f48200cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4828400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f48200ce3a0 con 0x7f48200d06a0 2024-05-28T13:58:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4828400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f482009b160 con 0x7f482015b790 2024-05-28T13:58:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4828400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f48200c7720 con 0x7f482015c990 2024-05-28T13:58:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4825a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48200d06a0 0x7f4820159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4825a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48200d06a0 0x7f4820159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49136/0 (socket says 172.21.15.42:49136) 2024-05-28T13:58:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4825a00640 1 -- 172.21.15.42:0/326699607 learned_addr learned my addr 172.21.15.42:0/326699607 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4825000640 1 --2- 172.21.15.42:0/326699607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f482015c990 0x7f48200af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4826400640 1 --2- 172.21.15.42:0/326699607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f482015b790 0x7f48200cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4825a00640 1 -- 172.21.15.42:0/326699607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f482015c990 msgr2=0x7f48200af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4825a00640 1 --2- 172.21.15.42:0/326699607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f482015c990 0x7f48200af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4825a00640 1 -- 172.21.15.42:0/326699607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f482015b790 msgr2=0x7f48200cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4825a00640 1 --2- 172.21.15.42:0/326699607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f482015b790 0x7f48200cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4825a00640 1 -- 172.21.15.42:0/326699607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48200c7a00 con 0x7f48200d06a0 2024-05-28T13:58:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4825a00640 1 --2- 172.21.15.42:0/326699607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48200d06a0 0x7f4820159780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f48140ad040 tx=0x7f481401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f481d600640 1 -- 172.21.15.42:0/326699607 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48140bf610 con 0x7f48200d06a0 2024-05-28T13:58:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f481d600640 1 -- 172.21.15.42:0/326699607 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48140bf7d0 con 0x7f48200d06a0 2024-05-28T13:58:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f481d600640 1 -- 172.21.15.42:0/326699607 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48140bfbc0 con 0x7f48200d06a0 2024-05-28T13:58:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4828400640 1 -- 172.21.15.42:0/326699607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48200c7ce0 con 0x7f48200d06a0 2024-05-28T13:58:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.873+0000 7f4828400640 1 -- 172.21.15.42:0/326699607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4820159fd0 con 0x7f48200d06a0 2024-05-28T13:58:23.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.877+0000 7f4828400640 1 -- 172.21.15.42:0/326699607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4820068a90 con 0x7f48200d06a0 2024-05-28T13:58:23.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.877+0000 7f481d600640 1 -- 172.21.15.42:0/326699607 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f48140bd020 con 0x7f48200d06a0 2024-05-28T13:58:23.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.877+0000 7f481d600640 1 --2- 172.21.15.42:0/326699607 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f48000683b0 0x7f480006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:23.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.877+0000 7f481d600640 1 -- 172.21.15.42:0/326699607 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f48141210a0 con 0x7f48200d06a0 2024-05-28T13:58:23.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.877+0000 7f4825000640 1 --2- 172.21.15.42:0/326699607 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f48000683b0 0x7f480006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:23.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.877+0000 7f4825000640 1 --2- 172.21.15.42:0/326699607 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f48000683b0 0x7f480006a870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f480c001650 tx=0x7f480c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:23.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:23.881+0000 7f481d600640 1 -- 172.21.15.42:0/326699607 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4814016090 con 0x7f48200d06a0 2024-05-28T13:58:24.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.049+0000 7f4828400640 1 -- 172.21.15.42:0/326699607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48200b4ce0 con 0x7f48200d06a0 2024-05-28T13:58:24.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.049+0000 7f481d600640 1 -- 172.21.15.42:0/326699607 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16558 (secure 0 0 0) 0x7f48140f1770 con 0x7f48200d06a0 2024-05-28T13:58:24.035 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:24.035 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":20,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:23.075389+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:24.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.049+0000 7f4828400640 1 -- 172.21.15.42:0/326699607 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f48000683b0 msgr2=0x7f480006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:24.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.049+0000 7f4828400640 1 --2- 172.21.15.42:0/326699607 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f48000683b0 0x7f480006a870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f480c001650 tx=0x7f480c009040 comp rx=0 tx=0).stop 2024-05-28T13:58:24.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.049+0000 7f4828400640 1 -- 172.21.15.42:0/326699607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48200d06a0 msgr2=0x7f4820159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.049+0000 7f4828400640 1 --2- 172.21.15.42:0/326699607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48200d06a0 0x7f4820159780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f48140ad040 tx=0x7f481401acb0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.049+0000 7f4828400640 1 -- 172.21.15.42:0/326699607 shutdown_connections 2024-05-28T13:58:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.049+0000 7f4828400640 1 --2- 172.21.15.42:0/326699607 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f48000683b0 0x7f480006a870 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.049+0000 7f4828400640 1 --2- 172.21.15.42:0/326699607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f482015b790 0x7f48200cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.049+0000 7f4828400640 1 --2- 172.21.15.42:0/326699607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f482015c990 0x7f48200af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.049+0000 7f4828400640 1 --2- 172.21.15.42:0/326699607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48200d06a0 0x7f4820159780 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.049+0000 7f4828400640 1 -- 172.21.15.42:0/326699607 >> 172.21.15.42:0/326699607 conn(0x7f48200b4080 msgr2=0x7f48200b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.049+0000 7f4828400640 1 -- 172.21.15.42:0/326699607 shutdown_connections 2024-05-28T13:58:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.053+0000 7f4828400640 1 -- 172.21.15.42:0/326699607 wait complete. 2024-05-28T13:58:24.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.209+0000 7f3586a00640 1 Processor -- start 2024-05-28T13:58:24.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.213+0000 7f3586a00640 1 -- start start 2024-05-28T13:58:24.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.213+0000 7f3586a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f358015c990 0x7f35800d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:24.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.213+0000 7f3586a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f358009bbd0 0x7f358009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:24.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.213+0000 7f3586a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35800cd300 con 0x7f35800d06a0 2024-05-28T13:58:24.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.213+0000 7f3586a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f358015cd70 con 0x7f358015c990 2024-05-28T13:58:24.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.213+0000 7f3586a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f35800b63f0 con 0x7f358009bbd0 2024-05-28T13:58:24.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.213+0000 7f357fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f358015c990 0x7f35800d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:24.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.213+0000 7f357fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f358015c990 0x7f35800d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49142/0 (socket says 172.21.15.42:49142) 2024-05-28T13:58:24.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.213+0000 7f357fe00640 1 -- 172.21.15.42:0/4172049233 learned_addr learned my addr 172.21.15.42:0/4172049233 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:24.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.213+0000 7f3577400640 1 --2- 172.21.15.42:0/4172049233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f358009bbd0 0x7f358009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:24.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.213+0000 7f357fe00640 1 -- 172.21.15.42:0/4172049233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f358009bbd0 msgr2=0x7f358009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:24.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.213+0000 7f357fe00640 1 --2- 172.21.15.42:0/4172049233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f358009bbd0 0x7f358009aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.213+0000 7f357fe00640 1 -- 172.21.15.42:0/4172049233 >> v1:172.21.15.63:6789/0 conn(0x7f35800d06a0 legacy=0x7f3580160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:24.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.213+0000 7f357fe00640 1 -- 172.21.15.42:0/4172049233 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35800cd5e0 con 0x7f358015c990 2024-05-28T13:58:24.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.217+0000 7f357fe00640 1 --2- 172.21.15.42:0/4172049233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f358015c990 0x7f35800d0160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3570009e50 tx=0x7f357001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce88f1c9c790af1 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:24.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.217+0000 7f357f400640 1 -- 172.21.15.42:0/4172049233 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35700a57e0 con 0x7f358015c990 2024-05-28T13:58:24.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.217+0000 7f357f400640 1 -- 172.21.15.42:0/4172049233 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35700a5980 con 0x7f358015c990 2024-05-28T13:58:24.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.217+0000 7f357f400640 1 -- 172.21.15.42:0/4172049233 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35700a5d50 con 0x7f358015c990 2024-05-28T13:58:24.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.217+0000 7f3586a00640 1 -- 172.21.15.42:0/4172049233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f358015c990 msgr2=0x7f35800d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:24.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.217+0000 7f3586a00640 1 --2- 172.21.15.42:0/4172049233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f358015c990 0x7f35800d0160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3570009e50 tx=0x7f357001ca00 comp rx=0 tx=0).stop 2024-05-28T13:58:24.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.217+0000 7f3586a00640 1 -- 172.21.15.42:0/4172049233 shutdown_connections 2024-05-28T13:58:24.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.217+0000 7f3586a00640 1 --2- 172.21.15.42:0/4172049233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f358009bbd0 0x7f358009aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.217+0000 7f3586a00640 1 --2- 172.21.15.42:0/4172049233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f358015c990 0x7f35800d0160 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.217+0000 7f3586a00640 1 -- 172.21.15.42:0/4172049233 >> 172.21.15.42:0/4172049233 conn(0x7f35800b4080 msgr2=0x7f35800b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:24.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.217+0000 7f3586a00640 1 -- 172.21.15.42:0/4172049233 shutdown_connections 2024-05-28T13:58:24.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.217+0000 7f3586a00640 1 -- 172.21.15.42:0/4172049233 wait complete. 2024-05-28T13:58:24.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.217+0000 7f3586a00640 1 Processor -- start 2024-05-28T13:58:24.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.217+0000 7f3586a00640 1 -- start start 2024-05-28T13:58:24.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f3586a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35800d06a0 0x7f3580152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:24.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f3586a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f358015c990 0x7f35801548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:24.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f3586a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3580154e20 0x7f35800705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:24.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f3586a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f35800ce3a0 con 0x7f35800d06a0 2024-05-28T13:58:24.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f3586a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3580169740 con 0x7f3580154e20 2024-05-28T13:58:24.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f3586a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3580165950 con 0x7f358015c990 2024-05-28T13:58:24.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f357fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35800d06a0 0x7f3580152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:24.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f357fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35800d06a0 0x7f3580152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49144/0 (socket says 172.21.15.42:49144) 2024-05-28T13:58:24.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f357fe00640 1 -- 172.21.15.42:0/1390904299 learned_addr learned my addr 172.21.15.42:0/1390904299 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:24.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f3584a00640 1 --2- 172.21.15.42:0/1390904299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3580154e20 0x7f35800705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:24.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f3577400640 1 --2- 172.21.15.42:0/1390904299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f358015c990 0x7f35801548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:24.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f357fe00640 1 -- 172.21.15.42:0/1390904299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f358015c990 msgr2=0x7f35801548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:24.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f357fe00640 1 --2- 172.21.15.42:0/1390904299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f358015c990 0x7f35801548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f357fe00640 1 -- 172.21.15.42:0/1390904299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3580154e20 msgr2=0x7f35800705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:24.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f357fe00640 1 --2- 172.21.15.42:0/1390904299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3580154e20 0x7f35800705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f357fe00640 1 -- 172.21.15.42:0/1390904299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3580165c30 con 0x7f35800d06a0 2024-05-28T13:58:24.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f357fe00640 1 --2- 172.21.15.42:0/1390904299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35800d06a0 0x7f3580152180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f35700af040 tx=0x7f35700197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:24.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f357cc00640 1 -- 172.21.15.42:0/1390904299 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35700bc020 con 0x7f35800d06a0 2024-05-28T13:58:24.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f357cc00640 1 -- 172.21.15.42:0/1390904299 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35700b8670 con 0x7f35800d06a0 2024-05-28T13:58:24.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f3586a00640 1 -- 172.21.15.42:0/1390904299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3580165f10 con 0x7f35800d06a0 2024-05-28T13:58:24.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f357cc00640 1 -- 172.21.15.42:0/1390904299 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35700b8a60 con 0x7f35800d06a0 2024-05-28T13:58:24.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f3586a00640 1 -- 172.21.15.42:0/1390904299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35801664d0 con 0x7f35800d06a0 2024-05-28T13:58:24.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f3586a00640 1 -- 172.21.15.42:0/1390904299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3580068a90 con 0x7f35800d06a0 2024-05-28T13:58:24.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.221+0000 7f357cc00640 1 -- 172.21.15.42:0/1390904299 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f35700bf020 con 0x7f35800d06a0 2024-05-28T13:58:24.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.225+0000 7f357cc00640 1 --2- 172.21.15.42:0/1390904299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f355c068380 0x7f355c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:24.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.225+0000 7f357cc00640 1 -- 172.21.15.42:0/1390904299 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3570121930 con 0x7f35800d06a0 2024-05-28T13:58:24.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.225+0000 7f3577400640 1 --2- 172.21.15.42:0/1390904299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f355c068380 0x7f355c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:24.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.225+0000 7f3577400640 1 --2- 172.21.15.42:0/1390904299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f355c068380 0x7f355c06a840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3580160120 tx=0x7f3568009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:24.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.229+0000 7f357cc00640 1 -- 172.21.15.42:0/1390904299 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f35700c6030 con 0x7f35800d06a0 2024-05-28T13:58:24.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.393+0000 7f3586a00640 1 -- 172.21.15.42:0/1390904299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35800b4ce0 con 0x7f35800d06a0 2024-05-28T13:58:24.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.397+0000 7f357cc00640 1 -- 172.21.15.42:0/1390904299 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16558 (secure 0 0 0) 0x7f3570016620 con 0x7f35800d06a0 2024-05-28T13:58:24.381 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:24.381 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":20,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:23.075389+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:24.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.397+0000 7f3586a00640 1 -- 172.21.15.42:0/1390904299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f355c068380 msgr2=0x7f355c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:24.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.397+0000 7f3586a00640 1 --2- 172.21.15.42:0/1390904299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f355c068380 0x7f355c06a840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3580160120 tx=0x7f3568009190 comp rx=0 tx=0).stop 2024-05-28T13:58:24.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.397+0000 7f3586a00640 1 -- 172.21.15.42:0/1390904299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35800d06a0 msgr2=0x7f3580152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:24.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.397+0000 7f3586a00640 1 --2- 172.21.15.42:0/1390904299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35800d06a0 0x7f3580152180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f35700af040 tx=0x7f35700197a0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.397+0000 7f3586a00640 1 -- 172.21.15.42:0/1390904299 shutdown_connections 2024-05-28T13:58:24.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.397+0000 7f3586a00640 1 --2- 172.21.15.42:0/1390904299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f355c068380 0x7f355c06a840 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.397+0000 7f3586a00640 1 --2- 172.21.15.42:0/1390904299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3580154e20 0x7f35800705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.397+0000 7f3586a00640 1 --2- 172.21.15.42:0/1390904299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f358015c990 0x7f35801548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.397+0000 7f3586a00640 1 --2- 172.21.15.42:0/1390904299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35800d06a0 0x7f3580152180 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.397+0000 7f3586a00640 1 -- 172.21.15.42:0/1390904299 >> 172.21.15.42:0/1390904299 conn(0x7f35800b4080 msgr2=0x7f358015b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:24.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.397+0000 7f3586a00640 1 -- 172.21.15.42:0/1390904299 shutdown_connections 2024-05-28T13:58:24.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.397+0000 7f3586a00640 1 -- 172.21.15.42:0/1390904299 wait complete. 2024-05-28T13:58:24.403 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-28T13:58:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.561+0000 7fde1fe00640 1 Processor -- start 2024-05-28T13:58:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.561+0000 7fde1fe00640 1 -- start start 2024-05-28T13:58:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde1fe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fde20147a80 0x7fde2006d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde1fe00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde20144540 con 0x7fde20143f90 2024-05-28T13:58:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde1fe00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde20144820 con 0x7fde200cfec0 2024-05-28T13:58:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde1fe00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fde2006cb00 con 0x7fde20147a80 2024-05-28T13:58:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde17e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fde20143f90 0x7fde2006cf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:58104/0 (socket says 172.21.15.42:58104) 2024-05-28T13:58:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde17e00640 1 -- 172.21.15.42:0/70888162 learned_addr learned my addr 172.21.15.42:0/70888162 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde1f400640 1 --2- 172.21.15.42:0/70888162 >> v2:172.21.15.63:3300/0 conn(0x7fde20147a80 0x7fde2006d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde1e000640 1 -- 172.21.15.42:0/70888162 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409891954 0 0) 0x7fde20144540 con 0x7fde20143f90 2024-05-28T13:58:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde1e000640 1 -- 172.21.15.42:0/70888162 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde04003530 con 0x7fde20143f90 2024-05-28T13:58:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde1e000640 1 -- 172.21.15.42:0/70888162 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1764349923 0 0) 0x7fde20144820 con 0x7fde200cfec0 2024-05-28T13:58:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde1e000640 1 -- 172.21.15.42:0/70888162 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde20144540 con 0x7fde200cfec0 2024-05-28T13:58:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde1f400640 1 -- 172.21.15.42:0/70888162 >> v1:172.21.15.63:6789/0 conn(0x7fde200cfec0 legacy=0x7fde200d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde1e000640 1 -- 172.21.15.42:0/70888162 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2944781735 0 0) 0x7fde04003530 con 0x7fde20143f90 2024-05-28T13:58:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde1f400640 1 -- 172.21.15.42:0/70888162 >> v1:172.21.15.42:6789/0 conn(0x7fde20143f90 legacy=0x7fde2006cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.565+0000 7fde1f400640 1 -- 172.21.15.42:0/70888162 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde20144b00 con 0x7fde20147a80 2024-05-28T13:58:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.569+0000 7fde1f400640 1 --2- 172.21.15.42:0/70888162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde20147a80 0x7fde2006d6a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde18009d90 tx=0x7fde1801c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=980a3b13af32b748 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.569+0000 7fde1e000640 1 -- 172.21.15.42:0/70888162 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180b2070 con 0x7fde20147a80 2024-05-28T13:58:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.569+0000 7fde1e000640 1 -- 172.21.15.42:0/70888162 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde180a55a0 con 0x7fde20147a80 2024-05-28T13:58:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.569+0000 7fde1e000640 1 -- 172.21.15.42:0/70888162 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde08001680 con 0x7fde20147a80 2024-05-28T13:58:24.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.569+0000 7fde1fe00640 1 -- 172.21.15.42:0/70888162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde20147a80 msgr2=0x7fde2006d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:24.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.569+0000 7fde1fe00640 1 --2- 172.21.15.42:0/70888162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde20147a80 0x7fde2006d6a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde18009d90 tx=0x7fde1801c740 comp rx=0 tx=0).stop 2024-05-28T13:58:24.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.569+0000 7fde1fe00640 1 -- 172.21.15.42:0/70888162 shutdown_connections 2024-05-28T13:58:24.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.569+0000 7fde1fe00640 1 --2- 172.21.15.42:0/70888162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde20147a80 0x7fde2006d6a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.569+0000 7fde1fe00640 1 -- 172.21.15.42:0/70888162 >> 172.21.15.42:0/70888162 conn(0x7fde2006fcc0 msgr2=0x7fde200700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:24.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.569+0000 7fde1fe00640 1 -- 172.21.15.42:0/70888162 shutdown_connections 2024-05-28T13:58:24.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.569+0000 7fde1fe00640 1 -- 172.21.15.42:0/70888162 wait complete. 2024-05-28T13:58:24.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.569+0000 7fde1fe00640 1 Processor -- start 2024-05-28T13:58:24.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.569+0000 7fde1fe00640 1 -- start start 2024-05-28T13:58:24.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.569+0000 7fde1fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde200cfec0 0x7fde2015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:24.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde20143f90 0x7fde2015b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:24.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde20147a80 0x7fde200b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:24.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fde2016c470 con 0x7fde20147a80 2024-05-28T13:58:24.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde2016c2f0 con 0x7fde20143f90 2024-05-28T13:58:24.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1fe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fde2016c5f0 con 0x7fde200cfec0 2024-05-28T13:58:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde20147a80 0x7fde200b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde20147a80 0x7fde200b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49156/0 (socket says 172.21.15.42:49156) 2024-05-28T13:58:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1f400640 1 -- 172.21.15.42:0/1952812552 learned_addr learned my addr 172.21.15.42:0/1952812552 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1f400640 1 -- 172.21.15.42:0/1952812552 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde200cfec0 msgr2=0x7fde2015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1f400640 1 --2- 172.21.15.42:0/1952812552 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde200cfec0 0x7fde2015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1f400640 1 -- 172.21.15.42:0/1952812552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde20143f90 msgr2=0x7fde2015b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1f400640 1 --2- 172.21.15.42:0/1952812552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde20143f90 0x7fde2015b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1f400640 1 -- 172.21.15.42:0/1952812552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2015daa0 con 0x7fde20147a80 2024-05-28T13:58:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1f400640 1 --2- 172.21.15.42:0/1952812552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde20147a80 0x7fde200b3000 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde18019350 tx=0x7fde2006e3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde16a00640 1 -- 172.21.15.42:0/1952812552 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180b2050 con 0x7fde20147a80 2024-05-28T13:58:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde16a00640 1 -- 172.21.15.42:0/1952812552 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde180be070 con 0x7fde20147a80 2024-05-28T13:58:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde16a00640 1 -- 172.21.15.42:0/1952812552 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180b9890 con 0x7fde20147a80 2024-05-28T13:58:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1fe00640 1 -- 172.21.15.42:0/1952812552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2015ae00 con 0x7fde20147a80 2024-05-28T13:58:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1fe00640 1 -- 172.21.15.42:0/1952812552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde2015b3c0 con 0x7fde20147a80 2024-05-28T13:58:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde1fe00640 1 -- 172.21.15.42:0/1952812552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde2004fcf0 con 0x7fde20147a80 2024-05-28T13:58:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.573+0000 7fde16a00640 1 -- 172.21.15.42:0/1952812552 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fde180b9a30 con 0x7fde20147a80 2024-05-28T13:58:24.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.577+0000 7fde16a00640 1 --2- 172.21.15.42:0/1952812552 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fddfc068380 0x7fddfc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:24.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.577+0000 7fde16a00640 1 -- 172.21.15.42:0/1952812552 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fde18002020 con 0x7fde20147a80 2024-05-28T13:58:24.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.577+0000 7fde1ea00640 1 --2- 172.21.15.42:0/1952812552 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fddfc068380 0x7fddfc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:24.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.577+0000 7fde1ea00640 1 --2- 172.21.15.42:0/1952812552 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fddfc068380 0x7fddfc06a840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde10007fd0 tx=0x7fde10007900 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:24.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.581+0000 7fde16a00640 1 -- 172.21.15.42:0/1952812552 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fde18016480 con 0x7fde20147a80 2024-05-28T13:58:24.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:24.781+0000 7fde1fe00640 1 -- 172.21.15.42:0/1952812552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fde200d2030 con 0x7fde20147a80 2024-05-28T13:58:25.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.689+0000 7fde16a00640 1 -- 172.21.15.42:0/1952812552 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fde180f1f00 con 0x7fde20147a80 2024-05-28T13:58:25.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.689+0000 7fde1fe00640 1 -- 172.21.15.42:0/1952812552 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fddfc068380 msgr2=0x7fddfc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:25.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.689+0000 7fde1fe00640 1 --2- 172.21.15.42:0/1952812552 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fddfc068380 0x7fddfc06a840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde10007fd0 tx=0x7fde10007900 comp rx=0 tx=0).stop 2024-05-28T13:58:25.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.689+0000 7fde1fe00640 1 -- 172.21.15.42:0/1952812552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde20147a80 msgr2=0x7fde200b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:25.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.689+0000 7fde1fe00640 1 --2- 172.21.15.42:0/1952812552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde20147a80 0x7fde200b3000 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde18019350 tx=0x7fde2006e3f0 comp rx=0 tx=0).stop 2024-05-28T13:58:25.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.693+0000 7fde1fe00640 1 -- 172.21.15.42:0/1952812552 shutdown_connections 2024-05-28T13:58:25.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.693+0000 7fde1fe00640 1 --2- 172.21.15.42:0/1952812552 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fddfc068380 0x7fddfc06a840 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:25.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.693+0000 7fde1fe00640 1 --2- 172.21.15.42:0/1952812552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde20147a80 0x7fde200b3000 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:25.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.693+0000 7fde1fe00640 1 --2- 172.21.15.42:0/1952812552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde20143f90 0x7fde2015b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:25.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.693+0000 7fde1fe00640 1 --2- 172.21.15.42:0/1952812552 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde200cfec0 0x7fde2015d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:25.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.693+0000 7fde1fe00640 1 -- 172.21.15.42:0/1952812552 >> 172.21.15.42:0/1952812552 conn(0x7fde2006fcc0 msgr2=0x7fde200b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:25.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.693+0000 7fde1fe00640 1 -- 172.21.15.42:0/1952812552 shutdown_connections 2024-05-28T13:58:25.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.693+0000 7fde1fe00640 1 -- 172.21.15.42:0/1952812552 wait complete. 2024-05-28T13:58:25.701 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-28T13:58:25.701 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-28T13:58:25.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.865+0000 7f51a4200640 1 Processor -- start 2024-05-28T13:58:25.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.865+0000 7f51a4200640 1 -- start start 2024-05-28T13:58:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.865+0000 7f51a4200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f519c06cfb0 0x7f519c15f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.865+0000 7f51a4200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f519c15fe30 con 0x7f519c0b32f0 2024-05-28T13:58:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.865+0000 7f51a4200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f519c09bb00 con 0x7f519c09aa50 2024-05-28T13:58:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.865+0000 7f51a4200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f519c0d0800 con 0x7f519c06cfb0 2024-05-28T13:58:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.865+0000 7f51a1800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f519c06cfb0 0x7f519c15f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.865+0000 7f51a2200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f519c0b32f0 0x7f519c0b3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:58116/0 (socket says 172.21.15.42:58116) 2024-05-28T13:58:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.865+0000 7f51a2200640 1 -- 172.21.15.42:0/3760070204 learned_addr learned my addr 172.21.15.42:0/3760070204 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.865+0000 7f51a1800640 1 -- 172.21.15.42:0/3760070204 >> v1:172.21.15.63:6789/0 conn(0x7f519c09aa50 legacy=0x7f519c06d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.865+0000 7f51a1800640 1 -- 172.21.15.42:0/3760070204 >> v1:172.21.15.42:6789/0 conn(0x7f519c0b32f0 legacy=0x7f519c0b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.865+0000 7f51a1800640 1 -- 172.21.15.42:0/3760070204 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c15c8c0 con 0x7f519c06cfb0 2024-05-28T13:58:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.869+0000 7f51a1800640 1 --2- 172.21.15.42:0/3760070204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f519c06cfb0 0x7f519c15f8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f51900097c0 tx=0x7f519001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208c7dcc4cfc65a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.869+0000 7f519be00640 1 -- 172.21.15.42:0/3760070204 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51900b1070 con 0x7f519c06cfb0 2024-05-28T13:58:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.869+0000 7f519be00640 1 -- 172.21.15.42:0/3760070204 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51900ac410 con 0x7f519c06cfb0 2024-05-28T13:58:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.869+0000 7f519be00640 1 -- 172.21.15.42:0/3760070204 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51900ac7e0 con 0x7f519c06cfb0 2024-05-28T13:58:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.869+0000 7f51a4200640 1 -- 172.21.15.42:0/3760070204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f519c06cfb0 msgr2=0x7f519c15f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.869+0000 7f51a4200640 1 --2- 172.21.15.42:0/3760070204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f519c06cfb0 0x7f519c15f8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f51900097c0 tx=0x7f519001c4a0 comp rx=0 tx=0).stop 2024-05-28T13:58:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.869+0000 7f51a4200640 1 -- 172.21.15.42:0/3760070204 shutdown_connections 2024-05-28T13:58:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.869+0000 7f51a4200640 1 --2- 172.21.15.42:0/3760070204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f519c06cfb0 0x7f519c15f8f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.869+0000 7f51a4200640 1 -- 172.21.15.42:0/3760070204 >> 172.21.15.42:0/3760070204 conn(0x7f519c0b40d0 msgr2=0x7f519c0b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.869+0000 7f51a4200640 1 -- 172.21.15.42:0/3760070204 shutdown_connections 2024-05-28T13:58:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.869+0000 7f51a4200640 1 -- 172.21.15.42:0/3760070204 wait complete. 2024-05-28T13:58:25.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a4200640 1 Processor -- start 2024-05-28T13:58:25.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a4200640 1 -- start start 2024-05-28T13:58:25.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a4200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f519c06cfb0 0x7f519c152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:25.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a4200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f519c09aa50 0x7f519c161330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:25.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a4200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f519c0b32f0 0x7f519c161870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:25.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a4200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f519c0b3e10 con 0x7f519c0b32f0 2024-05-28T13:58:25.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a4200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f519c09b820 con 0x7f519c06cfb0 2024-05-28T13:58:25.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a4200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f519c0b1f40 con 0x7f519c09aa50 2024-05-28T13:58:25.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a2200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f519c0b32f0 0x7f519c161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:25.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a2200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f519c0b32f0 0x7f519c161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49178/0 (socket says 172.21.15.42:49178) 2024-05-28T13:58:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a2200640 1 -- 172.21.15.42:0/3083211820 learned_addr learned my addr 172.21.15.42:0/3083211820 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a1800640 1 --2- 172.21.15.42:0/3083211820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f519c06cfb0 0x7f519c152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a0e00640 1 --2- 172.21.15.42:0/3083211820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f519c09aa50 0x7f519c161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a2200640 1 -- 172.21.15.42:0/3083211820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f519c09aa50 msgr2=0x7f519c161330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a2200640 1 --2- 172.21.15.42:0/3083211820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f519c09aa50 0x7f519c161330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a2200640 1 -- 172.21.15.42:0/3083211820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f519c06cfb0 msgr2=0x7f519c152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a2200640 1 --2- 172.21.15.42:0/3083211820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f519c06cfb0 0x7f519c152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a2200640 1 -- 172.21.15.42:0/3083211820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c15b360 con 0x7f519c0b32f0 2024-05-28T13:58:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a2200640 1 --2- 172.21.15.42:0/3083211820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f519c0b32f0 0x7f519c161870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f519400ca80 tx=0x7f519400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f5199600640 1 -- 172.21.15.42:0/3083211820 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194017070 con 0x7f519c0b32f0 2024-05-28T13:58:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f5199600640 1 -- 172.21.15.42:0/3083211820 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f519400ac30 con 0x7f519c0b32f0 2024-05-28T13:58:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a4200640 1 -- 172.21.15.42:0/3083211820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519c15b640 con 0x7f519c0b32f0 2024-05-28T13:58:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f5199600640 1 -- 172.21.15.42:0/3083211820 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194012600 con 0x7f519c0b32f0 2024-05-28T13:58:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.873+0000 7f51a4200640 1 -- 172.21.15.42:0/3083211820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519c15bc00 con 0x7f519c0b32f0 2024-05-28T13:58:25.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.877+0000 7f51a4200640 1 -- 172.21.15.42:0/3083211820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519c0a06c0 con 0x7f519c0b32f0 2024-05-28T13:58:25.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.877+0000 7f5199600640 1 -- 172.21.15.42:0/3083211820 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f519401a070 con 0x7f519c0b32f0 2024-05-28T13:58:25.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.877+0000 7f5199600640 1 --2- 172.21.15.42:0/3083211820 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f517c0683b0 0x7f517c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:25.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.877+0000 7f5199600640 1 -- 172.21.15.42:0/3083211820 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5194086610 con 0x7f519c0b32f0 2024-05-28T13:58:25.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.877+0000 7f51a1800640 1 --2- 172.21.15.42:0/3083211820 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f517c0683b0 0x7f517c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:25.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.877+0000 7f51a1800640 1 --2- 172.21.15.42:0/3083211820 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f517c0683b0 0x7f517c06a870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f51900ae000 tx=0x7f5190008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:25.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:25.881+0000 7f5199600640 1 -- 172.21.15.42:0/3083211820 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f519c15fe30 con 0x7f519c0b32f0 2024-05-28T13:58:26.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.081+0000 7f51a4200640 1 -- 172.21.15.42:0/3083211820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f519c0b4d60 con 0x7f519c0b32f0 2024-05-28T13:58:26.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.689+0000 7f5199600640 1 -- 172.21.15.42:0/3083211820 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f519400ae20 con 0x7f519c0b32f0 2024-05-28T13:58:26.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.693+0000 7f51a4200640 1 -- 172.21.15.42:0/3083211820 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f517c0683b0 msgr2=0x7f517c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:26.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.693+0000 7f51a4200640 1 --2- 172.21.15.42:0/3083211820 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f517c0683b0 0x7f517c06a870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f51900ae000 tx=0x7f5190008040 comp rx=0 tx=0).stop 2024-05-28T13:58:26.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.693+0000 7f51a4200640 1 -- 172.21.15.42:0/3083211820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f519c0b32f0 msgr2=0x7f519c161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:26.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.693+0000 7f51a4200640 1 --2- 172.21.15.42:0/3083211820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f519c0b32f0 0x7f519c161870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f519400ca80 tx=0x7f519400cf40 comp rx=0 tx=0).stop 2024-05-28T13:58:26.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.693+0000 7f51a4200640 1 -- 172.21.15.42:0/3083211820 shutdown_connections 2024-05-28T13:58:26.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.693+0000 7f51a4200640 1 --2- 172.21.15.42:0/3083211820 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f517c0683b0 0x7f517c06a870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:26.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.693+0000 7f51a4200640 1 --2- 172.21.15.42:0/3083211820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f519c0b32f0 0x7f519c161870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:26.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.693+0000 7f51a4200640 1 --2- 172.21.15.42:0/3083211820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f519c09aa50 0x7f519c161330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:26.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.693+0000 7f51a4200640 1 --2- 172.21.15.42:0/3083211820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f519c06cfb0 0x7f519c152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:26.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.693+0000 7f51a4200640 1 -- 172.21.15.42:0/3083211820 >> 172.21.15.42:0/3083211820 conn(0x7f519c0b40d0 msgr2=0x7f519c15ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:26.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.693+0000 7f51a4200640 1 -- 172.21.15.42:0/3083211820 shutdown_connections 2024-05-28T13:58:26.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.693+0000 7f51a4200640 1 -- 172.21.15.42:0/3083211820 wait complete. 2024-05-28T13:58:26.700 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-28T13:58:26.701 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T13:58:26.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.861+0000 7f3778600640 1 Processor -- start 2024-05-28T13:58:26.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.861+0000 7f3778600640 1 -- start start 2024-05-28T13:58:26.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.861+0000 7f3778600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37700d0900 0x7f3770162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:26.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.861+0000 7f3778600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37700d0de0 con 0x7f3770162ab0 2024-05-28T13:58:26.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.861+0000 7f3778600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37700d10c0 con 0x7f37700ce860 2024-05-28T13:58:26.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.861+0000 7f3778600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f37700d19f0 con 0x7f37700d0900 2024-05-28T13:58:26.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.861+0000 7f3776600640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f3770162ab0 0x7f377015e200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:58118/0 (socket says 172.21.15.42:58118) 2024-05-28T13:58:26.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.861+0000 7f3776600640 1 -- 172.21.15.42:0/2620073011 learned_addr learned my addr 172.21.15.42:0/2620073011 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:26.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.861+0000 7f3775200640 1 --2- 172.21.15.42:0/2620073011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37700d0900 0x7f3770162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:26.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.861+0000 7f376fe00640 1 -- 172.21.15.42:0/2620073011 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4192518062 0 0) 0x7f37700d0de0 con 0x7f3770162ab0 2024-05-28T13:58:26.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.861+0000 7f376fe00640 1 -- 172.21.15.42:0/2620073011 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3758003530 con 0x7f3770162ab0 2024-05-28T13:58:26.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.861+0000 7f376fe00640 1 -- 172.21.15.42:0/2620073011 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2620948376 0 0) 0x7f37700d10c0 con 0x7f37700ce860 2024-05-28T13:58:26.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.861+0000 7f376fe00640 1 -- 172.21.15.42:0/2620073011 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37700d0de0 con 0x7f37700ce860 2024-05-28T13:58:26.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.861+0000 7f376fe00640 1 -- 172.21.15.42:0/2620073011 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2596563891 0 0) 0x7f3758003530 con 0x7f3770162ab0 2024-05-28T13:58:26.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.865+0000 7f3775200640 1 -- 172.21.15.42:0/2620073011 >> v1:172.21.15.63:6789/0 conn(0x7f37700ce860 legacy=0x7f377006d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:26.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.865+0000 7f3775200640 1 -- 172.21.15.42:0/2620073011 >> v1:172.21.15.42:6789/0 conn(0x7f3770162ab0 legacy=0x7f377015e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:26.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.865+0000 7f3775200640 1 -- 172.21.15.42:0/2620073011 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37700d13a0 con 0x7f37700d0900 2024-05-28T13:58:26.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.865+0000 7f3775200640 1 --2- 172.21.15.42:0/2620073011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37700d0900 0x7f3770162570 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f375c0097f0 tx=0x7f375c01c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52c9995517be2d16 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:26.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.865+0000 7f376fe00640 1 -- 172.21.15.42:0/2620073011 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0b0070 con 0x7f37700d0900 2024-05-28T13:58:26.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.865+0000 7f376fe00640 1 -- 172.21.15.42:0/2620073011 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375c0ab400 con 0x7f37700d0900 2024-05-28T13:58:26.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.865+0000 7f376fe00640 1 -- 172.21.15.42:0/2620073011 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768001a00 con 0x7f37700d0900 2024-05-28T13:58:26.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.865+0000 7f3778600640 1 -- 172.21.15.42:0/2620073011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37700d0900 msgr2=0x7f3770162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:26.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.865+0000 7f3778600640 1 --2- 172.21.15.42:0/2620073011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37700d0900 0x7f3770162570 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f375c0097f0 tx=0x7f375c01c370 comp rx=0 tx=0).stop 2024-05-28T13:58:26.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.865+0000 7f3778600640 1 -- 172.21.15.42:0/2620073011 shutdown_connections 2024-05-28T13:58:26.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.865+0000 7f3778600640 1 --2- 172.21.15.42:0/2620073011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37700d0900 0x7f3770162570 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:26.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.865+0000 7f3778600640 1 -- 172.21.15.42:0/2620073011 >> 172.21.15.42:0/2620073011 conn(0x7f37700b4080 msgr2=0x7f37700b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:26.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.865+0000 7f3778600640 1 -- 172.21.15.42:0/2620073011 shutdown_connections 2024-05-28T13:58:26.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.865+0000 7f3778600640 1 -- 172.21.15.42:0/2620073011 wait complete. 2024-05-28T13:58:26.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3778600640 1 Processor -- start 2024-05-28T13:58:26.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3778600640 1 -- start start 2024-05-28T13:58:26.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3778600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37700ce860 0x7f3770158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:26.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3778600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37700d0900 0x7f37701528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:26.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3778600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3770162ab0 0x7f3770152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:26.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3778600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f377006e040 con 0x7f3770162ab0 2024-05-28T13:58:26.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3778600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f377006dec0 con 0x7f37700d0900 2024-05-28T13:58:26.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3778600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f377016d0d0 con 0x7f37700ce860 2024-05-28T13:58:26.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3776600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3770162ab0 0x7f3770152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:26.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3776600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3770162ab0 0x7f3770152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49182/0 (socket says 172.21.15.42:49182) 2024-05-28T13:58:26.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3776600640 1 -- 172.21.15.42:0/2399885790 learned_addr learned my addr 172.21.15.42:0/2399885790 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:26.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3775c00640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37700ce860 0x7f3770158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:26.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3775200640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37700d0900 0x7f37701528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:26.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3776600640 1 -- 172.21.15.42:0/2399885790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37700ce860 msgr2=0x7f3770158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:26.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3776600640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37700ce860 0x7f3770158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:26.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3776600640 1 -- 172.21.15.42:0/2399885790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37700d0900 msgr2=0x7f37701528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:26.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3776600640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37700d0900 0x7f37701528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:26.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3776600640 1 -- 172.21.15.42:0/2399885790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37700b6200 con 0x7f3770162ab0 2024-05-28T13:58:26.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3775200640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37700d0900 0x7f37701528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:26.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3776600640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3770162ab0 0x7f3770152e00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f376800cc10 tx=0x7f376800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:26.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f376d600640 1 -- 172.21.15.42:0/2399885790 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768017070 con 0x7f3770162ab0 2024-05-28T13:58:26.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f376d600640 1 -- 172.21.15.42:0/2399885790 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f376800ce20 con 0x7f3770162ab0 2024-05-28T13:58:26.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f376d600640 1 -- 172.21.15.42:0/2399885790 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768012600 con 0x7f3770162ab0 2024-05-28T13:58:26.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3778600640 1 -- 172.21.15.42:0/2399885790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37700b5520 con 0x7f3770162ab0 2024-05-28T13:58:26.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.869+0000 7f3778600640 1 -- 172.21.15.42:0/2399885790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37700b5f20 con 0x7f3770162ab0 2024-05-28T13:58:26.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.873+0000 7f3778600640 1 -- 172.21.15.42:0/2399885790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3770068a90 con 0x7f3770162ab0 2024-05-28T13:58:26.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.873+0000 7f376d600640 1 -- 172.21.15.42:0/2399885790 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f376801a070 con 0x7f3770162ab0 2024-05-28T13:58:26.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.873+0000 7f376d600640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3750068380 0x7f375006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:26.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.873+0000 7f376d600640 1 -- 172.21.15.42:0/2399885790 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3768086860 con 0x7f3770162ab0 2024-05-28T13:58:26.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.873+0000 7f3775c00640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3750068380 0x7f375006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:26.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.873+0000 7f3775c00640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3750068380 0x7f375006a840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f37640011f0 tx=0x7f3764009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:26.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:26.877+0000 7f376d600640 1 -- 172.21.15.42:0/2399885790 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3768056730 con 0x7f3770162ab0 2024-05-28T13:58:27.027 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-28T13:58:27.044+0000 7f88d8e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-28T13:58:27.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.081+0000 7f3778600640 1 -- 172.21.15.42:0/2399885790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37700b4ce0 con 0x7f3770162ab0 2024-05-28T13:58:27.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.081+0000 7f376d600640 1 -- 172.21.15.42:0/2399885790 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f376805aa30 con 0x7f3770162ab0 2024-05-28T13:58:27.067 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:27.067 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1409117123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1409117123},{"type":"v1","addr":"172.21.15.42:6837","nonce":1409117123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/3753517701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3753517701},{"type":"v1","addr":"172.21.15.42:6836","nonce":3753517701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/882866492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":882866492},{"type":"v1","addr":"172.21.15.63:6834","nonce":882866492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T13:58:23.075354+0000","modified":"2024-05-28T13:58:26.084895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/1552779700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1552779700},{"type":"v1","addr":"172.21.15.63:6835","nonce":1552779700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-05-28T13:58:27.067 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-05-28T13:58:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.085+0000 7f3778600640 1 -- 172.21.15.42:0/2399885790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3750068380 msgr2=0x7f375006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.085+0000 7f3778600640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3750068380 0x7f375006a840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f37640011f0 tx=0x7f3764009040 comp rx=0 tx=0).stop 2024-05-28T13:58:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.085+0000 7f3778600640 1 -- 172.21.15.42:0/2399885790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3770162ab0 msgr2=0x7f3770152e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.085+0000 7f3778600640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3770162ab0 0x7f3770152e00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f376800cc10 tx=0x7f376800a430 comp rx=0 tx=0).stop 2024-05-28T13:58:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.085+0000 7f3778600640 1 -- 172.21.15.42:0/2399885790 shutdown_connections 2024-05-28T13:58:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.085+0000 7f3778600640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3750068380 0x7f375006a840 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.085+0000 7f3778600640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3770162ab0 0x7f3770152e00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.085+0000 7f3778600640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37700d0900 0x7f37701528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.085+0000 7f3778600640 1 --2- 172.21.15.42:0/2399885790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37700ce860 0x7f3770158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.085+0000 7f3778600640 1 -- 172.21.15.42:0/2399885790 >> 172.21.15.42:0/2399885790 conn(0x7f37700b4080 msgr2=0x7f3770156f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.085+0000 7f3778600640 1 -- 172.21.15.42:0/2399885790 shutdown_connections 2024-05-28T13:58:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.085+0000 7f3778600640 1 -- 172.21.15.42:0/2399885790 wait complete. 2024-05-28T13:58:27.089 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:27.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f5504a00640 1 Processor -- start 2024-05-28T13:58:27.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f5504a00640 1 -- start start 2024-05-28T13:58:27.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f5504a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f550015c990 0x7f55000d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f5504a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f550009bbd0 0x7f550009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f5504a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55000cd300 con 0x7f55000d06a0 2024-05-28T13:58:27.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f5504a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f550015cd70 con 0x7f550009bbd0 2024-05-28T13:58:27.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f5504a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f55000b63f0 con 0x7f550015c990 2024-05-28T13:58:27.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f54f5400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f550009bbd0 0x7f550009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f54fde00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f550015c990 0x7f55000d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f54f5400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f550009bbd0 0x7f550009aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:35958/0 (socket says 172.21.15.42:35958) 2024-05-28T13:58:27.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f54fde00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f550015c990 0x7f55000d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:50790/0 (socket says 172.21.15.42:50790) 2024-05-28T13:58:27.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f54f5400640 1 -- 172.21.15.42:0/2358080556 learned_addr learned my addr 172.21.15.42:0/2358080556 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:27.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f54f5400640 1 -- 172.21.15.42:0/2358080556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f550015c990 msgr2=0x7f55000d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:27.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f54f5400640 1 --2- 172.21.15.42:0/2358080556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f550015c990 0x7f55000d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f54f5400640 1 -- 172.21.15.42:0/2358080556 >> v1:172.21.15.63:6789/0 conn(0x7f55000d06a0 legacy=0x7f5500160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.245+0000 7f54f5400640 1 -- 172.21.15.42:0/2358080556 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55000cd5e0 con 0x7f550009bbd0 2024-05-28T13:58:27.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.249+0000 7f54f5400640 1 --2- 172.21.15.42:0/2358080556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f550009bbd0 0x7f550009aa50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009870 tx=0x7f54e801c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1c39b607b100eed server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:27.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.249+0000 7f54fd400640 1 -- 172.21.15.42:0/2358080556 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e80ab640 con 0x7f550009bbd0 2024-05-28T13:58:27.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.249+0000 7f54fd400640 1 -- 172.21.15.42:0/2358080556 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e80b0070 con 0x7f550009bbd0 2024-05-28T13:58:27.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.249+0000 7f54fd400640 1 -- 172.21.15.42:0/2358080556 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e80aba00 con 0x7f550009bbd0 2024-05-28T13:58:27.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.249+0000 7f5504a00640 1 -- 172.21.15.42:0/2358080556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f550009bbd0 msgr2=0x7f550009aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.249+0000 7f5504a00640 1 --2- 172.21.15.42:0/2358080556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f550009bbd0 0x7f550009aa50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009870 tx=0x7f54e801c530 comp rx=0 tx=0).stop 2024-05-28T13:58:27.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.249+0000 7f5504a00640 1 -- 172.21.15.42:0/2358080556 shutdown_connections 2024-05-28T13:58:27.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.249+0000 7f5504a00640 1 --2- 172.21.15.42:0/2358080556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f550009bbd0 0x7f550009aa50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.249+0000 7f5504a00640 1 --2- 172.21.15.42:0/2358080556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f550015c990 0x7f55000d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.249+0000 7f5504a00640 1 -- 172.21.15.42:0/2358080556 >> 172.21.15.42:0/2358080556 conn(0x7f55000b4080 msgr2=0x7f55000b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:27.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.249+0000 7f5504a00640 1 -- 172.21.15.42:0/2358080556 shutdown_connections 2024-05-28T13:58:27.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.249+0000 7f5504a00640 1 -- 172.21.15.42:0/2358080556 wait complete. 2024-05-28T13:58:27.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f5504a00640 1 Processor -- start 2024-05-28T13:58:27.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f5504a00640 1 -- start start 2024-05-28T13:58:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f5504a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55000d06a0 0x7f55000c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f5504a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f550015c990 0x7f55000ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f5504a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55000ca740 0x7f5500155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f5504a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55000ce3a0 con 0x7f550015c990 2024-05-28T13:58:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f5504a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5500169740 con 0x7f55000d06a0 2024-05-28T13:58:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f5504a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5500156700 con 0x7f55000ca740 2024-05-28T13:58:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54f5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f550015c990 0x7f55000ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54f5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f550015c990 0x7f55000ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49196/0 (socket says 172.21.15.42:49196) 2024-05-28T13:58:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54f5400640 1 -- 172.21.15.42:0/4202047134 learned_addr learned my addr 172.21.15.42:0/4202047134 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54fde00640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55000d06a0 0x7f55000c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54fe800640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55000ca740 0x7f5500155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54f5400640 1 -- 172.21.15.42:0/4202047134 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55000ca740 msgr2=0x7f5500155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54f5400640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55000ca740 0x7f5500155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54f5400640 1 -- 172.21.15.42:0/4202047134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55000d06a0 msgr2=0x7f55000c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54f5400640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55000d06a0 0x7f55000c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54f5400640 1 -- 172.21.15.42:0/4202047134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55001569e0 con 0x7f550015c990 2024-05-28T13:58:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54fe800640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55000ca740 0x7f5500155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54f5400640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f550015c990 0x7f55000ca200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f54e801abe0 tx=0x7f54e801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54f6a00640 1 -- 172.21.15.42:0/4202047134 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e80be3f0 con 0x7f550015c990 2024-05-28T13:58:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54f6a00640 1 -- 172.21.15.42:0/4202047134 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e80be5b0 con 0x7f550015c990 2024-05-28T13:58:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f54f6a00640 1 -- 172.21.15.42:0/4202047134 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e80be9a0 con 0x7f550015c990 2024-05-28T13:58:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f5504a00640 1 -- 172.21.15.42:0/4202047134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5500156cc0 con 0x7f550015c990 2024-05-28T13:58:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.253+0000 7f5504a00640 1 -- 172.21.15.42:0/4202047134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55000c9150 con 0x7f550015c990 2024-05-28T13:58:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.257+0000 7f5504a00640 1 -- 172.21.15.42:0/4202047134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55000cab50 con 0x7f550015c990 2024-05-28T13:58:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.257+0000 7f54f6a00640 1 -- 172.21.15.42:0/4202047134 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f54e80b43f0 con 0x7f550015c990 2024-05-28T13:58:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.257+0000 7f54f6a00640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f54dc068380 0x7f54dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.257+0000 7f54f6a00640 1 -- 172.21.15.42:0/4202047134 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f54e8120950 con 0x7f550015c990 2024-05-28T13:58:27.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.257+0000 7f54fde00640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f54dc068380 0x7f54dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.257+0000 7f54fde00640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f54dc068380 0x7f54dc06a840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f54f0011340 tx=0x7f54f000b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:27.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.261+0000 7f54f6a00640 1 -- 172.21.15.42:0/4202047134 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f55000cab50 con 0x7f550015c990 2024-05-28T13:58:27.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.425+0000 7f5504a00640 1 -- 172.21.15.42:0/4202047134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5500068a90 con 0x7f550015c990 2024-05-28T13:58:27.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.425+0000 7f54f6a00640 1 -- 172.21.15.42:0/4202047134 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16545 (secure 0 0 0) 0x7f54e8016150 con 0x7f550015c990 2024-05-28T13:58:27.412 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:27.413 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:24.659233+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.429+0000 7f5504a00640 1 -- 172.21.15.42:0/4202047134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f54dc068380 msgr2=0x7f54dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.429+0000 7f5504a00640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f54dc068380 0x7f54dc06a840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f54f0011340 tx=0x7f54f000b040 comp rx=0 tx=0).stop 2024-05-28T13:58:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.429+0000 7f5504a00640 1 -- 172.21.15.42:0/4202047134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f550015c990 msgr2=0x7f55000ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.429+0000 7f5504a00640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f550015c990 0x7f55000ca200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f54e801abe0 tx=0x7f54e801ad70 comp rx=0 tx=0).stop 2024-05-28T13:58:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.429+0000 7f5504a00640 1 -- 172.21.15.42:0/4202047134 shutdown_connections 2024-05-28T13:58:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.429+0000 7f5504a00640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f54dc068380 0x7f54dc06a840 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.429+0000 7f5504a00640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55000ca740 0x7f5500155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.429+0000 7f5504a00640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f550015c990 0x7f55000ca200 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.429+0000 7f5504a00640 1 --2- 172.21.15.42:0/4202047134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55000d06a0 0x7f55000c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.429+0000 7f5504a00640 1 -- 172.21.15.42:0/4202047134 >> 172.21.15.42:0/4202047134 conn(0x7f55000b4080 msgr2=0x7f55000b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.429+0000 7f5504a00640 1 -- 172.21.15.42:0/4202047134 shutdown_connections 2024-05-28T13:58:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.429+0000 7f5504a00640 1 -- 172.21.15.42:0/4202047134 wait complete. 2024-05-28T13:58:27.434 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-28T13:58:27.434 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T13:58:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.589+0000 7fc689400640 1 Processor -- start 2024-05-28T13:58:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.589+0000 7fc689400640 1 -- start start 2024-05-28T13:58:27.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.589+0000 7fc689400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc684156a70 0x7fc68406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.589+0000 7fc689400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6840afa50 0x7fc6840d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.589+0000 7fc689400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc68415cb90 con 0x7fc684156e50 2024-05-28T13:58:27.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.589+0000 7fc689400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc68415ce50 con 0x7fc684156a70 2024-05-28T13:58:27.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.589+0000 7fc689400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc68415d130 con 0x7fc6840afa50 2024-05-28T13:58:27.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.589+0000 7fc67be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc684156e50 0x7fc6840af340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60450/0 (socket says 172.21.15.42:60450) 2024-05-28T13:58:27.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.589+0000 7fc67be00640 1 -- 172.21.15.42:0/1063340626 learned_addr learned my addr 172.21.15.42:0/1063340626 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:27.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.589+0000 7fc683200640 1 --2- 172.21.15.42:0/1063340626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6840afa50 0x7fc6840d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.589+0000 7fc682800640 1 --2- 172.21.15.42:0/1063340626 >> v2:172.21.15.63:3300/0 conn(0x7fc684156a70 0x7fc68406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.593+0000 7fc681e00640 1 -- 172.21.15.42:0/1063340626 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3560093500 0 0) 0x7fc68415cb90 con 0x7fc684156e50 2024-05-28T13:58:27.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.593+0000 7fc681e00640 1 -- 172.21.15.42:0/1063340626 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc668003530 con 0x7fc684156e50 2024-05-28T13:58:27.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.593+0000 7fc682800640 1 -- 172.21.15.42:0/1063340626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6840afa50 msgr2=0x7fc6840d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.593+0000 7fc682800640 1 --2- 172.21.15.42:0/1063340626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6840afa50 0x7fc6840d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.593+0000 7fc682800640 1 -- 172.21.15.42:0/1063340626 >> v1:172.21.15.42:6789/0 conn(0x7fc684156e50 legacy=0x7fc6840af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.593+0000 7fc681e00640 1 -- 172.21.15.42:0/1063340626 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1216062356 0 0) 0x7fc668003530 con 0x7fc684156e50 2024-05-28T13:58:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.593+0000 7fc682800640 1 -- 172.21.15.42:0/1063340626 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68415d410 con 0x7fc684156a70 2024-05-28T13:58:27.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.593+0000 7fc682800640 1 --2- 172.21.15.42:0/1063340626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc684156a70 0x7fc68406dd60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc6740097c0 tx=0x7fc67401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=298ae1f813c7e799 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:27.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.593+0000 7fc681e00640 1 -- 172.21.15.42:0/1063340626 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6740b0070 con 0x7fc684156a70 2024-05-28T13:58:27.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.593+0000 7fc681e00640 1 -- 172.21.15.42:0/1063340626 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6740ab400 con 0x7fc684156a70 2024-05-28T13:58:27.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.593+0000 7fc681e00640 1 -- 172.21.15.42:0/1063340626 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c001680 con 0x7fc684156a70 2024-05-28T13:58:27.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 -- 172.21.15.42:0/1063340626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc684156a70 msgr2=0x7fc68406dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 --2- 172.21.15.42:0/1063340626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc684156a70 0x7fc68406dd60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc6740097c0 tx=0x7fc67401c3c0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 -- 172.21.15.42:0/1063340626 shutdown_connections 2024-05-28T13:58:27.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 --2- 172.21.15.42:0/1063340626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6840afa50 0x7fc6840d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 --2- 172.21.15.42:0/1063340626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc684156a70 0x7fc68406dd60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 -- 172.21.15.42:0/1063340626 >> 172.21.15.42:0/1063340626 conn(0x7fc68406bdf0 msgr2=0x7fc68406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:27.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 -- 172.21.15.42:0/1063340626 shutdown_connections 2024-05-28T13:58:27.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 -- 172.21.15.42:0/1063340626 wait complete. 2024-05-28T13:58:27.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 Processor -- start 2024-05-28T13:58:27.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 -- start start 2024-05-28T13:58:27.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6840afa50 0x7fc68415f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc684156a70 0x7fc6840cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc684156e50 0x7fc684152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc684160f50 con 0x7fc684156a70 2024-05-28T13:58:27.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc684160dd0 con 0x7fc6840afa50 2024-05-28T13:58:27.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc689400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc6841610d0 con 0x7fc684156e50 2024-05-28T13:58:27.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc67be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc684156a70 0x7fc6840cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc67be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc684156a70 0x7fc6840cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38344/0 (socket says 172.21.15.42:38344) 2024-05-28T13:58:27.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc67be00640 1 -- 172.21.15.42:0/308904629 learned_addr learned my addr 172.21.15.42:0/308904629 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:27.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc682800640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6840afa50 0x7fc68415f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc683200640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc684156e50 0x7fc684152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc67be00640 1 -- 172.21.15.42:0/308904629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc684156e50 msgr2=0x7fc684152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc67be00640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc684156e50 0x7fc684152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc67be00640 1 -- 172.21.15.42:0/308904629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6840afa50 msgr2=0x7fc68415f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc67be00640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6840afa50 0x7fc68415f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc67be00640 1 -- 172.21.15.42:0/308904629 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6840cd230 con 0x7fc684156a70 2024-05-28T13:58:27.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.597+0000 7fc682800640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6840afa50 0x7fc68415f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:27.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.601+0000 7fc67be00640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc684156a70 0x7fc6840cf720 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00c750 tx=0x7fc66c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:27.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.601+0000 7fc67aa00640 1 -- 172.21.15.42:0/308904629 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c005ae0 con 0x7fc684156a70 2024-05-28T13:58:27.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.601+0000 7fc67aa00640 1 -- 172.21.15.42:0/308904629 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc66c005ca0 con 0x7fc684156a70 2024-05-28T13:58:27.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.601+0000 7fc67aa00640 1 -- 172.21.15.42:0/308904629 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c0072b0 con 0x7fc684156a70 2024-05-28T13:58:27.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.601+0000 7fc689400640 1 -- 172.21.15.42:0/308904629 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6840cd510 con 0x7fc684156a70 2024-05-28T13:58:27.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.601+0000 7fc689400640 1 -- 172.21.15.42:0/308904629 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6840cf200 con 0x7fc684156a70 2024-05-28T13:58:27.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.601+0000 7fc689400640 1 -- 172.21.15.42:0/308904629 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6840cf400 con 0x7fc684156a70 2024-05-28T13:58:27.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.601+0000 7fc67aa00640 1 -- 172.21.15.42:0/308904629 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc66c007450 con 0x7fc684156a70 2024-05-28T13:58:27.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.601+0000 7fc67aa00640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc6600683b0 0x7fc66006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.601+0000 7fc67aa00640 1 -- 172.21.15.42:0/308904629 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc66c086800 con 0x7fc684156a70 2024-05-28T13:58:27.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.601+0000 7fc682800640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc6600683b0 0x7fc66006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.605+0000 7fc682800640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc6600683b0 0x7fc66006a870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc6740ad000 tx=0x7fc674008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:27.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.609+0000 7fc67aa00640 1 -- 172.21.15.42:0/308904629 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc6840cf400 con 0x7fc684156a70 2024-05-28T13:58:27.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.769+0000 7fc689400640 1 -- 172.21.15.42:0/308904629 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc684068a90 con 0x7fc6600683b0 2024-05-28T13:58:27.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.777+0000 7fc67aa00640 1 -- 172.21.15.42:0/308904629 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251883 (secure 0 0 0) 0x7fc684068a90 con 0x7fc6600683b0 2024-05-28T13:58:27.764 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:27.766 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T13:58:27.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.781+0000 7fc689400640 1 -- 172.21.15.42:0/308904629 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc6600683b0 msgr2=0x7fc66006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.781+0000 7fc689400640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc6600683b0 0x7fc66006a870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc6740ad000 tx=0x7fc674008040 comp rx=0 tx=0).stop 2024-05-28T13:58:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.781+0000 7fc689400640 1 -- 172.21.15.42:0/308904629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc684156a70 msgr2=0x7fc6840cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.781+0000 7fc689400640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc684156a70 0x7fc6840cf720 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00c750 tx=0x7fc66c00cc10 comp rx=0 tx=0).stop 2024-05-28T13:58:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.781+0000 7fc689400640 1 -- 172.21.15.42:0/308904629 shutdown_connections 2024-05-28T13:58:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.781+0000 7fc689400640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc6600683b0 0x7fc66006a870 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.781+0000 7fc689400640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc684156e50 0x7fc684152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.781+0000 7fc689400640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc684156a70 0x7fc6840cf720 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.781+0000 7fc689400640 1 --2- 172.21.15.42:0/308904629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6840afa50 0x7fc68415f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.781+0000 7fc689400640 1 -- 172.21.15.42:0/308904629 >> 172.21.15.42:0/308904629 conn(0x7fc68406bdf0 msgr2=0x7fc6841661c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.781+0000 7fc689400640 1 -- 172.21.15.42:0/308904629 shutdown_connections 2024-05-28T13:58:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.781+0000 7fc689400640 1 -- 172.21.15.42:0/308904629 wait complete. 2024-05-28T13:58:27.784 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-28T13:58:26.856642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235644,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739076,"statfs":{"total":773094113280,"available":772852813824,"internally_reserved":0,"allocated":2891776,"data_stored":1739996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.210998"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.698853+0000","last_change":"2024-05-28T13:58:23.690106+0000","last_active":"2024-05-28T13:58:23.698853+0000","last_peered":"2024-05-28T13:58:23.698853+0000","last_clean":"2024-05-28T13:58:23.698853+0000","last_became_active":"2024-05-28T13:58:23.689597+0000","last_became_peered":"2024-05-28T13:58:23.689597+0000","last_unstale":"2024-05-28T13:58:23.698853+0000","last_undegraded":"2024-05-28T13:58:23.698853+0000","last_fullsized":"2024-05-28T13:58:23.698853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:46.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.676326+0000","last_change":"2024-05-28T13:58:23.674151+0000","last_active":"2024-05-28T13:58:23.676326+0000","last_peered":"2024-05-28T13:58:23.676326+0000","last_clean":"2024-05-28T13:58:23.676326+0000","last_became_active":"2024-05-28T13:58:23.673848+0000","last_became_peered":"2024-05-28T13:58:23.673848+0000","last_unstale":"2024-05-28T13:58:23.676326+0000","last_undegraded":"2024-05-28T13:58:23.676326+0000","last_fullsized":"2024-05-28T13:58:23.676326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:13.470877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.688474+0000","last_change":"2024-05-28T13:58:23.685573+0000","last_active":"2024-05-28T13:58:23.688474+0000","last_peered":"2024-05-28T13:58:23.688474+0000","last_clean":"2024-05-28T13:58:23.688474+0000","last_became_active":"2024-05-28T13:58:23.685243+0000","last_became_peered":"2024-05-28T13:58:23.685243+0000","last_unstale":"2024-05-28T13:58:23.688474+0000","last_undegraded":"2024-05-28T13:58:23.688474+0000","last_fullsized":"2024-05-28T13:58:23.688474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:03.122796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.710763+0000","last_change":"2024-05-28T13:58:23.706150+0000","last_active":"2024-05-28T13:58:23.710763+0000","last_peered":"2024-05-28T13:58:23.710763+0000","last_clean":"2024-05-28T13:58:23.710763+0000","last_became_active":"2024-05-28T13:58:23.689938+0000","last_became_peered":"2024-05-28T13:58:23.689938+0000","last_unstale":"2024-05-28T13:58:23.710763+0000","last_undegraded":"2024-05-28T13:58:23.710763+0000","last_fullsized":"2024-05-28T13:58:23.710763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:10.036541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.693679+0000","last_change":"2024-05-28T13:58:23.683882+0000","last_active":"2024-05-28T13:58:23.693679+0000","last_peered":"2024-05-28T13:58:23.693679+0000","last_clean":"2024-05-28T13:58:23.693679+0000","last_became_active":"2024-05-28T13:58:23.683368+0000","last_became_peered":"2024-05-28T13:58:23.683368+0000","last_unstale":"2024-05-28T13:58:23.693679+0000","last_undegraded":"2024-05-28T13:58:23.693679+0000","last_fullsized":"2024-05-28T13:58:23.693679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:12:43.447237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.709862+0000","last_change":"2024-05-28T13:58:23.695111+0000","last_active":"2024-05-28T13:58:23.709862+0000","last_peered":"2024-05-28T13:58:23.709862+0000","last_clean":"2024-05-28T13:58:23.709862+0000","last_became_active":"2024-05-28T13:58:23.688505+0000","last_became_peered":"2024-05-28T13:58:23.688505+0000","last_unstale":"2024-05-28T13:58:23.709862+0000","last_undegraded":"2024-05-28T13:58:23.709862+0000","last_fullsized":"2024-05-28T13:58:23.709862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:07:03.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.697549+0000","last_change":"2024-05-28T13:58:23.684793+0000","last_active":"2024-05-28T13:58:23.697549+0000","last_peered":"2024-05-28T13:58:23.697549+0000","last_clean":"2024-05-28T13:58:23.697549+0000","last_became_active":"2024-05-28T13:58:23.684500+0000","last_became_peered":"2024-05-28T13:58:23.684500+0000","last_unstale":"2024-05-28T13:58:23.697549+0000","last_undegraded":"2024-05-28T13:58:23.697549+0000","last_fullsized":"2024-05-28T13:58:23.697549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:07:03.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.689728+0000","last_change":"2024-05-28T13:58:23.688670+0000","last_active":"2024-05-28T13:58:23.689728+0000","last_peered":"2024-05-28T13:58:23.689728+0000","last_clean":"2024-05-28T13:58:23.689728+0000","last_became_active":"2024-05-28T13:58:23.688243+0000","last_became_peered":"2024-05-28T13:58:23.688243+0000","last_unstale":"2024-05-28T13:58:23.689728+0000","last_undegraded":"2024-05-28T13:58:23.689728+0000","last_fullsized":"2024-05-28T13:58:23.689728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:56.904188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.709234+0000","last_change":"2024-05-28T13:58:23.702434+0000","last_active":"2024-05-28T13:58:23.709234+0000","last_peered":"2024-05-28T13:58:23.709234+0000","last_clean":"2024-05-28T13:58:23.709234+0000","last_became_active":"2024-05-28T13:58:23.701857+0000","last_became_peered":"2024-05-28T13:58:23.701857+0000","last_unstale":"2024-05-28T13:58:23.709234+0000","last_undegraded":"2024-05-28T13:58:23.709234+0000","last_fullsized":"2024-05-28T13:58:23.709234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:03:59.917115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.711329+0000","last_change":"2024-05-28T13:58:23.709688+0000","last_active":"2024-05-28T13:58:23.711329+0000","last_peered":"2024-05-28T13:58:23.711329+0000","last_clean":"2024-05-28T13:58:23.711329+0000","last_became_active":"2024-05-28T13:58:23.709431+0000","last_became_peered":"2024-05-28T13:58:23.709431+0000","last_unstale":"2024-05-28T13:58:23.711329+0000","last_undegraded":"2024-05-28T13:58:23.711329+0000","last_fullsized":"2024-05-28T13:58:23.711329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:12:43.447237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.708252+0000","last_change":"2024-05-28T13:58:23.701527+0000","last_active":"2024-05-28T13:58:23.708252+0000","last_peered":"2024-05-28T13:58:23.708252+0000","last_clean":"2024-05-28T13:58:23.708252+0000","last_became_active":"2024-05-28T13:58:23.701160+0000","last_became_peered":"2024-05-28T13:58:23.701160+0000","last_unstale":"2024-05-28T13:58:23.708252+0000","last_undegraded":"2024-05-28T13:58:23.708252+0000","last_fullsized":"2024-05-28T13:58:23.708252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:13.470877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715282+0000","last_change":"2024-05-28T13:58:23.707766+0000","last_active":"2024-05-28T13:58:23.715282+0000","last_peered":"2024-05-28T13:58:23.715282+0000","last_clean":"2024-05-28T13:58:23.715282+0000","last_became_active":"2024-05-28T13:58:23.707542+0000","last_became_peered":"2024-05-28T13:58:23.707542+0000","last_unstale":"2024-05-28T13:58:23.715282+0000","last_undegraded":"2024-05-28T13:58:23.715282+0000","last_fullsized":"2024-05-28T13:58:23.715282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:39.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.688953+0000","last_change":"2024-05-28T13:58:23.687526+0000","last_active":"2024-05-28T13:58:23.688953+0000","last_peered":"2024-05-28T13:58:23.688953+0000","last_clean":"2024-05-28T13:58:23.688953+0000","last_became_active":"2024-05-28T13:58:23.687138+0000","last_became_peered":"2024-05-28T13:58:23.687138+0000","last_unstale":"2024-05-28T13:58:23.688953+0000","last_undegraded":"2024-05-28T13:58:23.688953+0000","last_fullsized":"2024-05-28T13:58:23.688953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:36.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.687615+0000","last_change":"2024-05-28T13:58:23.681967+0000","last_active":"2024-05-28T13:58:23.687615+0000","last_peered":"2024-05-28T13:58:23.687615+0000","last_clean":"2024-05-28T13:58:23.687615+0000","last_became_active":"2024-05-28T13:58:23.680491+0000","last_became_peered":"2024-05-28T13:58:23.680491+0000","last_unstale":"2024-05-28T13:58:23.687615+0000","last_undegraded":"2024-05-28T13:58:23.687615+0000","last_fullsized":"2024-05-28T13:58:23.687615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:08.011020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.710950+0000","last_change":"2024-05-28T13:58:23.708051+0000","last_active":"2024-05-28T13:58:23.710950+0000","last_peered":"2024-05-28T13:58:23.710950+0000","last_clean":"2024-05-28T13:58:23.710950+0000","last_became_active":"2024-05-28T13:58:23.707749+0000","last_became_peered":"2024-05-28T13:58:23.707749+0000","last_unstale":"2024-05-28T13:58:23.710950+0000","last_undegraded":"2024-05-28T13:58:23.710950+0000","last_fullsized":"2024-05-28T13:58:23.710950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:07:03.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.693440+0000","last_change":"2024-05-28T13:58:23.684368+0000","last_active":"2024-05-28T13:58:23.693440+0000","last_peered":"2024-05-28T13:58:23.693440+0000","last_clean":"2024-05-28T13:58:23.693440+0000","last_became_active":"2024-05-28T13:58:23.683905+0000","last_became_peered":"2024-05-28T13:58:23.683905+0000","last_unstale":"2024-05-28T13:58:23.693440+0000","last_undegraded":"2024-05-28T13:58:23.693440+0000","last_fullsized":"2024-05-28T13:58:23.693440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:36.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.710968+0000","last_change":"2024-05-28T13:58:23.706633+0000","last_active":"2024-05-28T13:58:23.710968+0000","last_peered":"2024-05-28T13:58:23.710968+0000","last_clean":"2024-05-28T13:58:23.710968+0000","last_became_active":"2024-05-28T13:58:23.693217+0000","last_became_peered":"2024-05-28T13:58:23.693217+0000","last_unstale":"2024-05-28T13:58:23.710968+0000","last_undegraded":"2024-05-28T13:58:23.710968+0000","last_fullsized":"2024-05-28T13:58:23.710968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:22.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.691229+0000","last_change":"2024-05-28T13:58:23.690320+0000","last_active":"2024-05-28T13:58:23.691229+0000","last_peered":"2024-05-28T13:58:23.691229+0000","last_clean":"2024-05-28T13:58:23.691229+0000","last_became_active":"2024-05-28T13:58:23.690028+0000","last_became_peered":"2024-05-28T13:58:23.690028+0000","last_unstale":"2024-05-28T13:58:23.691229+0000","last_undegraded":"2024-05-28T13:58:23.691229+0000","last_fullsized":"2024-05-28T13:58:23.691229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:08.011020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.691324+0000","last_change":"2024-05-28T13:58:23.683310+0000","last_active":"2024-05-28T13:58:23.691324+0000","last_peered":"2024-05-28T13:58:23.691324+0000","last_clean":"2024-05-28T13:58:23.691324+0000","last_became_active":"2024-05-28T13:58:23.682882+0000","last_became_peered":"2024-05-28T13:58:23.682882+0000","last_unstale":"2024-05-28T13:58:23.691324+0000","last_undegraded":"2024-05-28T13:58:23.691324+0000","last_fullsized":"2024-05-28T13:58:23.691324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:45.207023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.708646+0000","last_change":"2024-05-28T13:58:23.706004+0000","last_active":"2024-05-28T13:58:23.708646+0000","last_peered":"2024-05-28T13:58:23.708646+0000","last_clean":"2024-05-28T13:58:23.708646+0000","last_became_active":"2024-05-28T13:58:23.705676+0000","last_became_peered":"2024-05-28T13:58:23.705676+0000","last_unstale":"2024-05-28T13:58:23.708646+0000","last_undegraded":"2024-05-28T13:58:23.708646+0000","last_fullsized":"2024-05-28T13:58:23.708646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:45.207023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.684136+0000","last_change":"2024-05-28T13:58:23.678693+0000","last_active":"2024-05-28T13:58:23.684136+0000","last_peered":"2024-05-28T13:58:23.684136+0000","last_clean":"2024-05-28T13:58:23.684136+0000","last_became_active":"2024-05-28T13:58:23.678258+0000","last_became_peered":"2024-05-28T13:58:23.678258+0000","last_unstale":"2024-05-28T13:58:23.684136+0000","last_undegraded":"2024-05-28T13:58:23.684136+0000","last_fullsized":"2024-05-28T13:58:23.684136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:35.740742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.710779+0000","last_change":"2024-05-28T13:58:23.706461+0000","last_active":"2024-05-28T13:58:23.710779+0000","last_peered":"2024-05-28T13:58:23.710779+0000","last_clean":"2024-05-28T13:58:23.710779+0000","last_became_active":"2024-05-28T13:58:23.702598+0000","last_became_peered":"2024-05-28T13:58:23.702598+0000","last_unstale":"2024-05-28T13:58:23.710779+0000","last_undegraded":"2024-05-28T13:58:23.710779+0000","last_fullsized":"2024-05-28T13:58:23.710779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:46.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715591+0000","last_change":"2024-05-28T13:58:23.714967+0000","last_active":"2024-05-28T13:58:23.715591+0000","last_peered":"2024-05-28T13:58:23.715591+0000","last_clean":"2024-05-28T13:58:23.715591+0000","last_became_active":"2024-05-28T13:58:23.714653+0000","last_became_peered":"2024-05-28T13:58:23.714653+0000","last_unstale":"2024-05-28T13:58:23.715591+0000","last_undegraded":"2024-05-28T13:58:23.715591+0000","last_fullsized":"2024-05-28T13:58:23.715591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:07:03.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.703866+0000","last_change":"2024-05-28T13:58:23.686847+0000","last_active":"2024-05-28T13:58:23.703866+0000","last_peered":"2024-05-28T13:58:23.703866+0000","last_clean":"2024-05-28T13:58:23.703866+0000","last_became_active":"2024-05-28T13:58:23.686518+0000","last_became_peered":"2024-05-28T13:58:23.686518+0000","last_unstale":"2024-05-28T13:58:23.703866+0000","last_undegraded":"2024-05-28T13:58:23.703866+0000","last_fullsized":"2024-05-28T13:58:23.703866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:59.716378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.709684+0000","last_change":"2024-05-28T13:58:23.707351+0000","last_active":"2024-05-28T13:58:23.709684+0000","last_peered":"2024-05-28T13:58:23.709684+0000","last_clean":"2024-05-28T13:58:23.709684+0000","last_became_active":"2024-05-28T13:58:23.707077+0000","last_became_peered":"2024-05-28T13:58:23.707077+0000","last_unstale":"2024-05-28T13:58:23.709684+0000","last_undegraded":"2024-05-28T13:58:23.709684+0000","last_fullsized":"2024-05-28T13:58:23.709684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:12:43.447237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.694717+0000","last_change":"2024-05-28T13:58:23.688056+0000","last_active":"2024-05-28T13:58:23.694717+0000","last_peered":"2024-05-28T13:58:23.694717+0000","last_clean":"2024-05-28T13:58:23.694717+0000","last_became_active":"2024-05-28T13:58:23.687623+0000","last_became_peered":"2024-05-28T13:58:23.687623+0000","last_unstale":"2024-05-28T13:58:23.694717+0000","last_undegraded":"2024-05-28T13:58:23.694717+0000","last_fullsized":"2024-05-28T13:58:23.694717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:59.716378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.677334+0000","last_change":"2024-05-28T13:58:23.672661+0000","last_active":"2024-05-28T13:58:23.677334+0000","last_peered":"2024-05-28T13:58:23.677334+0000","last_clean":"2024-05-28T13:58:23.677334+0000","last_became_active":"2024-05-28T13:58:23.672378+0000","last_became_peered":"2024-05-28T13:58:23.672378+0000","last_unstale":"2024-05-28T13:58:23.677334+0000","last_undegraded":"2024-05-28T13:58:23.677334+0000","last_fullsized":"2024-05-28T13:58:23.677334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:56.904188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.709610+0000","last_change":"2024-05-28T13:58:23.707282+0000","last_active":"2024-05-28T13:58:23.709610+0000","last_peered":"2024-05-28T13:58:23.709610+0000","last_clean":"2024-05-28T13:58:23.709610+0000","last_became_active":"2024-05-28T13:58:23.706977+0000","last_became_peered":"2024-05-28T13:58:23.706977+0000","last_unstale":"2024-05-28T13:58:23.709610+0000","last_undegraded":"2024-05-28T13:58:23.709610+0000","last_fullsized":"2024-05-28T13:58:23.709610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:45.207023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.687807+0000","last_change":"2024-05-28T13:58:23.684735+0000","last_active":"2024-05-28T13:58:23.687807+0000","last_peered":"2024-05-28T13:58:23.687807+0000","last_clean":"2024-05-28T13:58:23.687807+0000","last_became_active":"2024-05-28T13:58:23.684389+0000","last_became_peered":"2024-05-28T13:58:23.684389+0000","last_unstale":"2024-05-28T13:58:23.687807+0000","last_undegraded":"2024-05-28T13:58:23.687807+0000","last_fullsized":"2024-05-28T13:58:23.687807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:30.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.703762+0000","last_change":"2024-05-28T13:58:23.686822+0000","last_active":"2024-05-28T13:58:23.703762+0000","last_peered":"2024-05-28T13:58:23.703762+0000","last_clean":"2024-05-28T13:58:23.703762+0000","last_became_active":"2024-05-28T13:58:23.686454+0000","last_became_peered":"2024-05-28T13:58:23.686454+0000","last_unstale":"2024-05-28T13:58:23.703762+0000","last_undegraded":"2024-05-28T13:58:23.703762+0000","last_fullsized":"2024-05-28T13:58:23.703762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:56.904188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.703117+0000","last_change":"2024-05-28T13:58:23.702203+0000","last_active":"2024-05-28T13:58:23.703117+0000","last_peered":"2024-05-28T13:58:23.703117+0000","last_clean":"2024-05-28T13:58:23.703117+0000","last_became_active":"2024-05-28T13:58:23.701830+0000","last_became_peered":"2024-05-28T13:58:23.701830+0000","last_unstale":"2024-05-28T13:58:23.703117+0000","last_undegraded":"2024-05-28T13:58:23.703117+0000","last_fullsized":"2024-05-28T13:58:23.703117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:30.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.694193+0000","last_change":"2024-05-28T13:58:23.684058+0000","last_active":"2024-05-28T13:58:23.694193+0000","last_peered":"2024-05-28T13:58:23.694193+0000","last_clean":"2024-05-28T13:58:23.694193+0000","last_became_active":"2024-05-28T13:58:23.683555+0000","last_became_peered":"2024-05-28T13:58:23.683555+0000","last_unstale":"2024-05-28T13:58:23.694193+0000","last_undegraded":"2024-05-28T13:58:23.694193+0000","last_fullsized":"2024-05-28T13:58:23.694193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:36.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.713509+0000","last_change":"2024-05-28T13:58:23.704842+0000","last_active":"2024-05-28T13:58:23.713509+0000","last_peered":"2024-05-28T13:58:23.713509+0000","last_clean":"2024-05-28T13:58:23.713509+0000","last_became_active":"2024-05-28T13:58:23.704465+0000","last_became_peered":"2024-05-28T13:58:23.704465+0000","last_unstale":"2024-05-28T13:58:23.713509+0000","last_undegraded":"2024-05-28T13:58:23.713509+0000","last_fullsized":"2024-05-28T13:58:23.713509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:53.495402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.712747+0000","last_change":"2024-05-28T13:58:22.678741+0000","last_active":"2024-05-28T13:58:23.712747+0000","last_peered":"2024-05-28T13:58:23.712747+0000","last_clean":"2024-05-28T13:58:23.712747+0000","last_became_active":"2024-05-28T13:58:22.678532+0000","last_became_peered":"2024-05-28T13:58:22.678532+0000","last_unstale":"2024-05-28T13:58:23.712747+0000","last_undegraded":"2024-05-28T13:58:23.712747+0000","last_fullsized":"2024-05-28T13:58:23.712747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.710717+0000","last_change":"2024-05-28T13:58:23.708224+0000","last_active":"2024-05-28T13:58:23.710717+0000","last_peered":"2024-05-28T13:58:23.710717+0000","last_clean":"2024-05-28T13:58:23.710717+0000","last_became_active":"2024-05-28T13:58:23.707930+0000","last_became_peered":"2024-05-28T13:58:23.707930+0000","last_unstale":"2024-05-28T13:58:23.710717+0000","last_undegraded":"2024-05-28T13:58:23.710717+0000","last_fullsized":"2024-05-28T13:58:23.710717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:59.716378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715475+0000","last_change":"2024-05-28T13:58:22.673310+0000","last_active":"2024-05-28T13:58:23.715475+0000","last_peered":"2024-05-28T13:58:23.715475+0000","last_clean":"2024-05-28T13:58:23.715475+0000","last_became_active":"2024-05-28T13:58:22.673003+0000","last_became_peered":"2024-05-28T13:58:22.673003+0000","last_unstale":"2024-05-28T13:58:23.715475+0000","last_undegraded":"2024-05-28T13:58:23.715475+0000","last_fullsized":"2024-05-28T13:58:23.715475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.668422+0000","last_change":"2024-05-28T13:58:22.677344+0000","last_active":"2024-05-28T13:58:23.668422+0000","last_peered":"2024-05-28T13:58:23.668422+0000","last_clean":"2024-05-28T13:58:23.668422+0000","last_became_active":"2024-05-28T13:58:22.677024+0000","last_became_peered":"2024-05-28T13:58:22.677024+0000","last_unstale":"2024-05-28T13:58:23.668422+0000","last_undegraded":"2024-05-28T13:58:23.668422+0000","last_fullsized":"2024-05-28T13:58:23.668422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:52.890121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.688012+0000","last_change":"2024-05-28T13:58:23.686005+0000","last_active":"2024-05-28T13:58:23.688012+0000","last_peered":"2024-05-28T13:58:23.688012+0000","last_clean":"2024-05-28T13:58:23.688012+0000","last_became_active":"2024-05-28T13:58:23.682423+0000","last_became_peered":"2024-05-28T13:58:23.682423+0000","last_unstale":"2024-05-28T13:58:23.688012+0000","last_undegraded":"2024-05-28T13:58:23.688012+0000","last_fullsized":"2024-05-28T13:58:23.688012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:03.122796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.669976+0000","last_change":"2024-05-28T13:58:22.673902+0000","last_active":"2024-05-28T13:58:23.669976+0000","last_peered":"2024-05-28T13:58:23.669976+0000","last_clean":"2024-05-28T13:58:23.669976+0000","last_became_active":"2024-05-28T13:58:22.673588+0000","last_became_peered":"2024-05-28T13:58:22.673588+0000","last_unstale":"2024-05-28T13:58:23.669976+0000","last_undegraded":"2024-05-28T13:58:23.669976+0000","last_fullsized":"2024-05-28T13:58:23.669976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.676281+0000","last_change":"2024-05-28T13:58:23.672761+0000","last_active":"2024-05-28T13:58:23.676281+0000","last_peered":"2024-05-28T13:58:23.676281+0000","last_clean":"2024-05-28T13:58:23.676281+0000","last_became_active":"2024-05-28T13:58:23.672276+0000","last_became_peered":"2024-05-28T13:58:23.672276+0000","last_unstale":"2024-05-28T13:58:23.676281+0000","last_undegraded":"2024-05-28T13:58:23.676281+0000","last_fullsized":"2024-05-28T13:58:23.676281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:59.716378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.673183+0000","last_change":"2024-05-28T13:58:22.676336+0000","last_active":"2024-05-28T13:58:23.673183+0000","last_peered":"2024-05-28T13:58:23.673183+0000","last_clean":"2024-05-28T13:58:23.673183+0000","last_became_active":"2024-05-28T13:58:22.676013+0000","last_became_peered":"2024-05-28T13:58:22.676013+0000","last_unstale":"2024-05-28T13:58:23.673183+0000","last_undegraded":"2024-05-28T13:58:23.673183+0000","last_fullsized":"2024-05-28T13:58:23.673183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.707875+0000","last_change":"2024-05-28T13:58:23.691740+0000","last_active":"2024-05-28T13:58:23.707875+0000","last_peered":"2024-05-28T13:58:23.707875+0000","last_clean":"2024-05-28T13:58:23.707875+0000","last_became_active":"2024-05-28T13:58:23.691056+0000","last_became_peered":"2024-05-28T13:58:23.691056+0000","last_unstale":"2024-05-28T13:58:23.707875+0000","last_undegraded":"2024-05-28T13:58:23.707875+0000","last_fullsized":"2024-05-28T13:58:23.707875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:45.207023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.677240+0000","last_change":"2024-05-28T13:58:22.678043+0000","last_active":"2024-05-28T13:58:23.677240+0000","last_peered":"2024-05-28T13:58:23.677240+0000","last_clean":"2024-05-28T13:58:23.677240+0000","last_became_active":"2024-05-28T13:58:22.677684+0000","last_became_peered":"2024-05-28T13:58:22.677684+0000","last_unstale":"2024-05-28T13:58:23.677240+0000","last_undegraded":"2024-05-28T13:58:23.677240+0000","last_fullsized":"2024-05-28T13:58:23.677240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:02.116873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.709891+0000","last_change":"2024-05-28T13:58:23.706540+0000","last_active":"2024-05-28T13:58:23.709891+0000","last_peered":"2024-05-28T13:58:23.709891+0000","last_clean":"2024-05-28T13:58:23.709891+0000","last_became_active":"2024-05-28T13:58:23.706209+0000","last_became_peered":"2024-05-28T13:58:23.706209+0000","last_unstale":"2024-05-28T13:58:23.709891+0000","last_undegraded":"2024-05-28T13:58:23.709891+0000","last_fullsized":"2024-05-28T13:58:23.709891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:56.904188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.718389+0000","last_change":"2024-05-28T13:58:22.684218+0000","last_active":"2024-05-28T13:58:23.718389+0000","last_peered":"2024-05-28T13:58:23.718389+0000","last_clean":"2024-05-28T13:58:23.718389+0000","last_became_active":"2024-05-28T13:58:22.683984+0000","last_became_peered":"2024-05-28T13:58:22.683984+0000","last_unstale":"2024-05-28T13:58:23.718389+0000","last_undegraded":"2024-05-28T13:58:23.718389+0000","last_fullsized":"2024-05-28T13:58:23.718389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715256+0000","last_change":"2024-05-28T13:58:23.705226+0000","last_active":"2024-05-28T13:58:23.715256+0000","last_peered":"2024-05-28T13:58:23.715256+0000","last_clean":"2024-05-28T13:58:23.715256+0000","last_became_active":"2024-05-28T13:58:23.704265+0000","last_became_peered":"2024-05-28T13:58:23.704265+0000","last_unstale":"2024-05-28T13:58:23.715256+0000","last_undegraded":"2024-05-28T13:58:23.715256+0000","last_fullsized":"2024-05-28T13:58:23.715256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:53.896043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.718907+0000","last_change":"2024-05-28T13:58:22.674273+0000","last_active":"2024-05-28T13:58:23.718907+0000","last_peered":"2024-05-28T13:58:23.718907+0000","last_clean":"2024-05-28T13:58:23.718907+0000","last_became_active":"2024-05-28T13:58:22.673984+0000","last_became_peered":"2024-05-28T13:58:22.673984+0000","last_unstale":"2024-05-28T13:58:23.718907+0000","last_undegraded":"2024-05-28T13:58:23.718907+0000","last_fullsized":"2024-05-28T13:58:23.718907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.698781+0000","last_change":"2024-05-28T13:58:23.685763+0000","last_active":"2024-05-28T13:58:23.698781+0000","last_peered":"2024-05-28T13:58:23.698781+0000","last_clean":"2024-05-28T13:58:23.698781+0000","last_became_active":"2024-05-28T13:58:23.685512+0000","last_became_peered":"2024-05-28T13:58:23.685512+0000","last_unstale":"2024-05-28T13:58:23.698781+0000","last_undegraded":"2024-05-28T13:58:23.698781+0000","last_fullsized":"2024-05-28T13:58:23.698781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:10.036541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.714209+0000","last_change":"2024-05-28T13:58:22.668047+0000","last_active":"2024-05-28T13:58:23.714209+0000","last_peered":"2024-05-28T13:58:23.714209+0000","last_clean":"2024-05-28T13:58:23.714209+0000","last_became_active":"2024-05-28T13:58:22.667747+0000","last_became_peered":"2024-05-28T13:58:22.667747+0000","last_unstale":"2024-05-28T13:58:23.714209+0000","last_undegraded":"2024-05-28T13:58:23.714209+0000","last_fullsized":"2024-05-28T13:58:23.714209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:52.489479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.680108+0000","last_change":"2024-05-28T13:58:23.675653+0000","last_active":"2024-05-28T13:58:23.680108+0000","last_peered":"2024-05-28T13:58:23.680108+0000","last_clean":"2024-05-28T13:58:23.680108+0000","last_became_active":"2024-05-28T13:58:23.675371+0000","last_became_peered":"2024-05-28T13:58:23.675371+0000","last_unstale":"2024-05-28T13:58:23.680108+0000","last_undegraded":"2024-05-28T13:58:23.680108+0000","last_fullsized":"2024-05-28T13:58:23.680108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:53.896043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.675637+0000","last_change":"2024-05-28T13:58:22.682942+0000","last_active":"2024-05-28T13:58:23.675637+0000","last_peered":"2024-05-28T13:58:23.675637+0000","last_clean":"2024-05-28T13:58:23.675637+0000","last_became_active":"2024-05-28T13:58:22.681946+0000","last_became_peered":"2024-05-28T13:58:22.681946+0000","last_unstale":"2024-05-28T13:58:23.675637+0000","last_undegraded":"2024-05-28T13:58:23.675637+0000","last_fullsized":"2024-05-28T13:58:23.675637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:07.005097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.721508+0000","last_change":"2024-05-28T13:58:23.716761+0000","last_active":"2024-05-28T13:58:23.721508+0000","last_peered":"2024-05-28T13:58:23.721508+0000","last_clean":"2024-05-28T13:58:23.721508+0000","last_became_active":"2024-05-28T13:58:23.716488+0000","last_became_peered":"2024-05-28T13:58:23.716488+0000","last_unstale":"2024-05-28T13:58:23.721508+0000","last_undegraded":"2024-05-28T13:58:23.721508+0000","last_fullsized":"2024-05-28T13:58:23.721508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:49.556769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.673110+0000","last_change":"2024-05-28T13:58:22.678451+0000","last_active":"2024-05-28T13:58:23.673110+0000","last_peered":"2024-05-28T13:58:23.673110+0000","last_clean":"2024-05-28T13:58:23.673110+0000","last_became_active":"2024-05-28T13:58:22.675695+0000","last_became_peered":"2024-05-28T13:58:22.675695+0000","last_unstale":"2024-05-28T13:58:23.673110+0000","last_undegraded":"2024-05-28T13:58:23.673110+0000","last_fullsized":"2024-05-28T13:58:23.673110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.694164+0000","last_change":"2024-05-28T13:58:23.683800+0000","last_active":"2024-05-28T13:58:23.694164+0000","last_peered":"2024-05-28T13:58:23.694164+0000","last_clean":"2024-05-28T13:58:23.694164+0000","last_became_active":"2024-05-28T13:58:23.683208+0000","last_became_peered":"2024-05-28T13:58:23.683208+0000","last_unstale":"2024-05-28T13:58:23.694164+0000","last_undegraded":"2024-05-28T13:58:23.694164+0000","last_fullsized":"2024-05-28T13:58:23.694164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:03.122796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.700600+0000","last_change":"2024-05-28T13:58:22.676468+0000","last_active":"2024-05-28T13:58:23.700600+0000","last_peered":"2024-05-28T13:58:23.700600+0000","last_clean":"2024-05-28T13:58:23.700600+0000","last_became_active":"2024-05-28T13:58:22.676202+0000","last_became_peered":"2024-05-28T13:58:22.676202+0000","last_unstale":"2024-05-28T13:58:23.700600+0000","last_undegraded":"2024-05-28T13:58:23.700600+0000","last_fullsized":"2024-05-28T13:58:23.700600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:52.890121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.721528+0000","last_change":"2024-05-28T13:58:23.717324+0000","last_active":"2024-05-28T13:58:23.721528+0000","last_peered":"2024-05-28T13:58:23.721528+0000","last_clean":"2024-05-28T13:58:23.721528+0000","last_became_active":"2024-05-28T13:58:23.716047+0000","last_became_peered":"2024-05-28T13:58:23.716047+0000","last_unstale":"2024-05-28T13:58:23.721528+0000","last_undegraded":"2024-05-28T13:58:23.721528+0000","last_fullsized":"2024-05-28T13:58:23.721528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:31.530501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.721934+0000","last_change":"2024-05-28T13:58:22.671015+0000","last_active":"2024-05-28T13:58:23.721934+0000","last_peered":"2024-05-28T13:58:23.721934+0000","last_clean":"2024-05-28T13:58:23.721934+0000","last_became_active":"2024-05-28T13:58:22.663746+0000","last_became_peered":"2024-05-28T13:58:22.663746+0000","last_unstale":"2024-05-28T13:58:23.721934+0000","last_undegraded":"2024-05-28T13:58:23.721934+0000","last_fullsized":"2024-05-28T13:58:23.721934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:22:51.238198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.709248+0000","last_change":"2024-05-28T13:58:23.705595+0000","last_active":"2024-05-28T13:58:23.709248+0000","last_peered":"2024-05-28T13:58:23.709248+0000","last_clean":"2024-05-28T13:58:23.709248+0000","last_became_active":"2024-05-28T13:58:23.705326+0000","last_became_peered":"2024-05-28T13:58:23.705326+0000","last_unstale":"2024-05-28T13:58:23.709248+0000","last_undegraded":"2024-05-28T13:58:23.709248+0000","last_fullsized":"2024-05-28T13:58:23.709248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:23.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.699199+0000","last_change":"2024-05-28T13:58:22.673480+0000","last_active":"2024-05-28T13:58:23.699199+0000","last_peered":"2024-05-28T13:58:23.699199+0000","last_clean":"2024-05-28T13:58:23.699199+0000","last_became_active":"2024-05-28T13:58:22.671634+0000","last_became_peered":"2024-05-28T13:58:22.671634+0000","last_unstale":"2024-05-28T13:58:23.699199+0000","last_undegraded":"2024-05-28T13:58:23.699199+0000","last_fullsized":"2024-05-28T13:58:23.699199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.711118+0000","last_change":"2024-05-28T13:58:23.708965+0000","last_active":"2024-05-28T13:58:23.711118+0000","last_peered":"2024-05-28T13:58:23.711118+0000","last_clean":"2024-05-28T13:58:23.711118+0000","last_became_active":"2024-05-28T13:58:23.708298+0000","last_became_peered":"2024-05-28T13:58:23.708298+0000","last_unstale":"2024-05-28T13:58:23.711118+0000","last_undegraded":"2024-05-28T13:58:23.711118+0000","last_fullsized":"2024-05-28T13:58:23.711118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:02.981053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715446+0000","last_change":"2024-05-28T13:58:22.688102+0000","last_active":"2024-05-28T13:58:23.715446+0000","last_peered":"2024-05-28T13:58:23.715446+0000","last_clean":"2024-05-28T13:58:23.715446+0000","last_became_active":"2024-05-28T13:58:22.686031+0000","last_became_peered":"2024-05-28T13:58:22.686031+0000","last_unstale":"2024-05-28T13:58:23.715446+0000","last_undegraded":"2024-05-28T13:58:23.715446+0000","last_fullsized":"2024-05-28T13:58:23.715446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.707795+0000","last_change":"2024-05-28T13:58:23.690696+0000","last_active":"2024-05-28T13:58:23.707795+0000","last_peered":"2024-05-28T13:58:23.707795+0000","last_clean":"2024-05-28T13:58:23.707795+0000","last_became_active":"2024-05-28T13:58:23.689615+0000","last_became_peered":"2024-05-28T13:58:23.689615+0000","last_unstale":"2024-05-28T13:58:23.707795+0000","last_undegraded":"2024-05-28T13:58:23.707795+0000","last_fullsized":"2024-05-28T13:58:23.707795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:13.470877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.678441+0000","last_change":"2024-05-28T13:58:22.674653+0000","last_active":"2024-05-28T13:58:23.678441+0000","last_peered":"2024-05-28T13:58:23.678441+0000","last_clean":"2024-05-28T13:58:23.678441+0000","last_became_active":"2024-05-28T13:58:22.674308+0000","last_became_peered":"2024-05-28T13:58:22.674308+0000","last_unstale":"2024-05-28T13:58:23.678441+0000","last_undegraded":"2024-05-28T13:58:23.678441+0000","last_fullsized":"2024-05-28T13:58:23.678441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.709185+0000","last_change":"2024-05-28T13:58:23.702244+0000","last_active":"2024-05-28T13:58:23.709185+0000","last_peered":"2024-05-28T13:58:23.709185+0000","last_clean":"2024-05-28T13:58:23.709185+0000","last_became_active":"2024-05-28T13:58:23.701858+0000","last_became_peered":"2024-05-28T13:58:23.701858+0000","last_unstale":"2024-05-28T13:58:23.709185+0000","last_undegraded":"2024-05-28T13:58:23.709185+0000","last_fullsized":"2024-05-28T13:58:23.709185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:22.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.670305+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T13:58:23.670305+0000","last_peered":"2024-05-28T13:58:23.670305+0000","last_clean":"2024-05-28T13:58:23.670305+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T13:58:23.670305+0000","last_undegraded":"2024-05-28T13:58:23.670305+0000","last_fullsized":"2024-05-28T13:58:23.670305+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.672682+0000","last_change":"2024-05-28T13:58:22.668236+0000","last_active":"2024-05-28T13:58:23.672682+0000","last_peered":"2024-05-28T13:58:23.672682+0000","last_clean":"2024-05-28T13:58:23.672682+0000","last_became_active":"2024-05-28T13:58:22.667928+0000","last_became_peered":"2024-05-28T13:58:22.667928+0000","last_unstale":"2024-05-28T13:58:23.672682+0000","last_undegraded":"2024-05-28T13:58:23.672682+0000","last_fullsized":"2024-05-28T13:58:23.672682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.702261+0000","last_change":"2024-05-28T13:58:23.701173+0000","last_active":"2024-05-28T13:58:23.702261+0000","last_peered":"2024-05-28T13:58:23.702261+0000","last_clean":"2024-05-28T13:58:23.702261+0000","last_became_active":"2024-05-28T13:58:23.700830+0000","last_became_peered":"2024-05-28T13:58:23.700830+0000","last_unstale":"2024-05-28T13:58:23.702261+0000","last_undegraded":"2024-05-28T13:58:23.702261+0000","last_fullsized":"2024-05-28T13:58:23.702261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:13.470877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.670941+0000","last_change":"2024-05-28T13:58:22.687098+0000","last_active":"2024-05-28T13:58:23.670941+0000","last_peered":"2024-05-28T13:58:23.670941+0000","last_clean":"2024-05-28T13:58:23.670941+0000","last_became_active":"2024-05-28T13:58:22.686751+0000","last_became_peered":"2024-05-28T13:58:22.686751+0000","last_unstale":"2024-05-28T13:58:23.670941+0000","last_undegraded":"2024-05-28T13:58:23.670941+0000","last_fullsized":"2024-05-28T13:58:23.670941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:35.326847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.710891+0000","last_change":"2024-05-28T13:58:23.709225+0000","last_active":"2024-05-28T13:58:23.710891+0000","last_peered":"2024-05-28T13:58:23.710891+0000","last_clean":"2024-05-28T13:58:23.710891+0000","last_became_active":"2024-05-28T13:58:23.708785+0000","last_became_peered":"2024-05-28T13:58:23.708785+0000","last_unstale":"2024-05-28T13:58:23.710891+0000","last_undegraded":"2024-05-28T13:58:23.710891+0000","last_fullsized":"2024-05-28T13:58:23.710891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:13.470877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.678384+0000","last_change":"2024-05-28T13:58:22.667443+0000","last_active":"2024-05-28T13:58:23.678384+0000","last_peered":"2024-05-28T13:58:23.678384+0000","last_clean":"2024-05-28T13:58:23.678384+0000","last_became_active":"2024-05-28T13:58:22.666967+0000","last_became_peered":"2024-05-28T13:58:22.666967+0000","last_unstale":"2024-05-28T13:58:23.678384+0000","last_undegraded":"2024-05-28T13:58:23.678384+0000","last_fullsized":"2024-05-28T13:58:23.678384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.681505+0000","last_change":"2024-05-28T13:58:23.680134+0000","last_active":"2024-05-28T13:58:23.681505+0000","last_peered":"2024-05-28T13:58:23.681505+0000","last_clean":"2024-05-28T13:58:23.681505+0000","last_became_active":"2024-05-28T13:58:23.679912+0000","last_became_peered":"2024-05-28T13:58:23.679912+0000","last_unstale":"2024-05-28T13:58:23.681505+0000","last_undegraded":"2024-05-28T13:58:23.681505+0000","last_fullsized":"2024-05-28T13:58:23.681505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:45.207023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.675838+0000","last_change":"2024-05-28T13:58:22.683368+0000","last_active":"2024-05-28T13:58:23.675838+0000","last_peered":"2024-05-28T13:58:23.675838+0000","last_clean":"2024-05-28T13:58:23.675838+0000","last_became_active":"2024-05-28T13:58:22.683102+0000","last_became_peered":"2024-05-28T13:58:22.683102+0000","last_unstale":"2024-05-28T13:58:23.675838+0000","last_undegraded":"2024-05-28T13:58:23.675838+0000","last_fullsized":"2024-05-28T13:58:23.675838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715466+0000","last_change":"2024-05-28T13:58:23.707964+0000","last_active":"2024-05-28T13:58:23.715466+0000","last_peered":"2024-05-28T13:58:23.715466+0000","last_clean":"2024-05-28T13:58:23.715466+0000","last_became_active":"2024-05-28T13:58:23.707430+0000","last_became_peered":"2024-05-28T13:58:23.707430+0000","last_unstale":"2024-05-28T13:58:23.715466+0000","last_undegraded":"2024-05-28T13:58:23.715466+0000","last_fullsized":"2024-05-28T13:58:23.715466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:35.740742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.662937+0000","last_change":"2024-05-28T13:58:22.676423+0000","last_active":"2024-05-28T13:58:23.662937+0000","last_peered":"2024-05-28T13:58:23.662937+0000","last_clean":"2024-05-28T13:58:23.662937+0000","last_became_active":"2024-05-28T13:58:22.675960+0000","last_became_peered":"2024-05-28T13:58:22.675960+0000","last_unstale":"2024-05-28T13:58:23.662937+0000","last_undegraded":"2024-05-28T13:58:23.662937+0000","last_fullsized":"2024-05-28T13:58:23.662937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:02.116873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.707879+0000","last_change":"2024-05-28T13:58:23.692003+0000","last_active":"2024-05-28T13:58:23.707879+0000","last_peered":"2024-05-28T13:58:23.707879+0000","last_clean":"2024-05-28T13:58:23.707879+0000","last_became_active":"2024-05-28T13:58:23.689087+0000","last_became_peered":"2024-05-28T13:58:23.689087+0000","last_unstale":"2024-05-28T13:58:23.707879+0000","last_undegraded":"2024-05-28T13:58:23.707879+0000","last_fullsized":"2024-05-28T13:58:23.707879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:12:43.447237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.672020+0000","last_change":"2024-05-28T13:58:22.686829+0000","last_active":"2024-05-28T13:58:23.672020+0000","last_peered":"2024-05-28T13:58:23.672020+0000","last_clean":"2024-05-28T13:58:23.672020+0000","last_became_active":"2024-05-28T13:58:22.686497+0000","last_became_peered":"2024-05-28T13:58:22.686497+0000","last_unstale":"2024-05-28T13:58:23.672020+0000","last_undegraded":"2024-05-28T13:58:23.672020+0000","last_fullsized":"2024-05-28T13:58:23.672020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:58.710455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.686334+0000","last_change":"2024-05-28T13:58:23.684658+0000","last_active":"2024-05-28T13:58:23.686334+0000","last_peered":"2024-05-28T13:58:23.686334+0000","last_clean":"2024-05-28T13:58:23.686334+0000","last_became_active":"2024-05-28T13:58:23.682872+0000","last_became_peered":"2024-05-28T13:58:23.682872+0000","last_unstale":"2024-05-28T13:58:23.686334+0000","last_undegraded":"2024-05-28T13:58:23.686334+0000","last_fullsized":"2024-05-28T13:58:23.686334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:12:43.447237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.662441+0000","last_change":"2024-05-28T13:58:22.682664+0000","last_active":"2024-05-28T13:58:23.662441+0000","last_peered":"2024-05-28T13:58:23.662441+0000","last_clean":"2024-05-28T13:58:23.662441+0000","last_became_active":"2024-05-28T13:58:22.682422+0000","last_became_peered":"2024-05-28T13:58:22.682422+0000","last_unstale":"2024-05-28T13:58:23.662441+0000","last_undegraded":"2024-05-28T13:58:23.662441+0000","last_fullsized":"2024-05-28T13:58:23.662441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:07.005097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.682997+0000","last_change":"2024-05-28T13:58:23.678389+0000","last_active":"2024-05-28T13:58:23.682997+0000","last_peered":"2024-05-28T13:58:23.682997+0000","last_clean":"2024-05-28T13:58:23.682997+0000","last_became_active":"2024-05-28T13:58:23.678081+0000","last_became_peered":"2024-05-28T13:58:23.678081+0000","last_unstale":"2024-05-28T13:58:23.682997+0000","last_undegraded":"2024-05-28T13:58:23.682997+0000","last_fullsized":"2024-05-28T13:58:23.682997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:08.011020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.703434+0000","last_change":"2024-05-28T13:58:22.680245+0000","last_active":"2024-05-28T13:58:23.703434+0000","last_peered":"2024-05-28T13:58:23.703434+0000","last_clean":"2024-05-28T13:58:23.703434+0000","last_became_active":"2024-05-28T13:58:22.679323+0000","last_became_peered":"2024-05-28T13:58:22.679323+0000","last_unstale":"2024-05-28T13:58:23.703434+0000","last_undegraded":"2024-05-28T13:58:23.703434+0000","last_fullsized":"2024-05-28T13:58:23.703434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:07.005097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.681944+0000","last_change":"2024-05-28T13:58:23.677850+0000","last_active":"2024-05-28T13:58:23.681944+0000","last_peered":"2024-05-28T13:58:23.681944+0000","last_clean":"2024-05-28T13:58:23.681944+0000","last_became_active":"2024-05-28T13:58:23.677654+0000","last_became_peered":"2024-05-28T13:58:23.677654+0000","last_unstale":"2024-05-28T13:58:23.681944+0000","last_undegraded":"2024-05-28T13:58:23.681944+0000","last_fullsized":"2024-05-28T13:58:23.681944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:39.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.669983+0000","last_change":"2024-05-28T13:58:22.682426+0000","last_active":"2024-05-28T13:58:23.669983+0000","last_peered":"2024-05-28T13:58:23.669983+0000","last_clean":"2024-05-28T13:58:23.669983+0000","last_became_active":"2024-05-28T13:58:22.682099+0000","last_became_peered":"2024-05-28T13:58:22.682099+0000","last_unstale":"2024-05-28T13:58:23.669983+0000","last_undegraded":"2024-05-28T13:58:23.669983+0000","last_fullsized":"2024-05-28T13:58:23.669983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715195+0000","last_change":"2024-05-28T13:58:23.706882+0000","last_active":"2024-05-28T13:58:23.715195+0000","last_peered":"2024-05-28T13:58:23.715195+0000","last_clean":"2024-05-28T13:58:23.715195+0000","last_became_active":"2024-05-28T13:58:23.706588+0000","last_became_peered":"2024-05-28T13:58:23.706588+0000","last_unstale":"2024-05-28T13:58:23.715195+0000","last_undegraded":"2024-05-28T13:58:23.715195+0000","last_fullsized":"2024-05-28T13:58:23.715195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:34.131954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.663046+0000","last_change":"2024-05-28T13:58:22.678682+0000","last_active":"2024-05-28T13:58:23.663046+0000","last_peered":"2024-05-28T13:58:23.663046+0000","last_clean":"2024-05-28T13:58:23.663046+0000","last_became_active":"2024-05-28T13:58:22.678233+0000","last_became_peered":"2024-05-28T13:58:22.678233+0000","last_unstale":"2024-05-28T13:58:23.663046+0000","last_undegraded":"2024-05-28T13:58:23.663046+0000","last_fullsized":"2024-05-28T13:58:23.663046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:35.326847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.678134+0000","last_change":"2024-05-28T13:58:23.669871+0000","last_active":"2024-05-28T13:58:23.678134+0000","last_peered":"2024-05-28T13:58:23.678134+0000","last_clean":"2024-05-28T13:58:23.678134+0000","last_became_active":"2024-05-28T13:58:23.669643+0000","last_became_peered":"2024-05-28T13:58:23.669643+0000","last_unstale":"2024-05-28T13:58:23.678134+0000","last_undegraded":"2024-05-28T13:58:23.678134+0000","last_fullsized":"2024-05-28T13:58:23.678134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:39.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.661860+0000","last_change":"2024-05-28T13:58:22.680769+0000","last_active":"2024-05-28T13:58:23.661860+0000","last_peered":"2024-05-28T13:58:23.661860+0000","last_clean":"2024-05-28T13:58:23.661860+0000","last_became_active":"2024-05-28T13:58:22.679781+0000","last_became_peered":"2024-05-28T13:58:22.679781+0000","last_unstale":"2024-05-28T13:58:23.661860+0000","last_undegraded":"2024-05-28T13:58:23.661860+0000","last_fullsized":"2024-05-28T13:58:23.661860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.687680+0000","last_change":"2024-05-28T13:58:23.683534+0000","last_active":"2024-05-28T13:58:23.687680+0000","last_peered":"2024-05-28T13:58:23.687680+0000","last_clean":"2024-05-28T13:58:23.687680+0000","last_became_active":"2024-05-28T13:58:23.683185+0000","last_became_peered":"2024-05-28T13:58:23.683185+0000","last_unstale":"2024-05-28T13:58:23.687680+0000","last_undegraded":"2024-05-28T13:58:23.687680+0000","last_fullsized":"2024-05-28T13:58:23.687680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:30.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.716058+0000","last_change":"2024-05-28T13:58:22.683550+0000","last_active":"2024-05-28T13:58:23.716058+0000","last_peered":"2024-05-28T13:58:23.716058+0000","last_clean":"2024-05-28T13:58:23.716058+0000","last_became_active":"2024-05-28T13:58:22.683218+0000","last_became_peered":"2024-05-28T13:58:22.683218+0000","last_unstale":"2024-05-28T13:58:23.716058+0000","last_undegraded":"2024-05-28T13:58:23.716058+0000","last_fullsized":"2024-05-28T13:58:23.716058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:12.464954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.716911+0000","last_change":"2024-05-28T13:58:23.709888+0000","last_active":"2024-05-28T13:58:23.716911+0000","last_peered":"2024-05-28T13:58:23.716911+0000","last_clean":"2024-05-28T13:58:23.716911+0000","last_became_active":"2024-05-28T13:58:23.709658+0000","last_became_peered":"2024-05-28T13:58:23.709658+0000","last_unstale":"2024-05-28T13:58:23.716911+0000","last_undegraded":"2024-05-28T13:58:23.716911+0000","last_fullsized":"2024-05-28T13:58:23.716911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:07:03.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.714405+0000","last_change":"2024-05-28T13:58:23.710526+0000","last_active":"2024-05-28T13:58:23.714405+0000","last_peered":"2024-05-28T13:58:23.714405+0000","last_clean":"2024-05-28T13:58:23.714405+0000","last_became_active":"2024-05-28T13:58:23.710118+0000","last_became_peered":"2024-05-28T13:58:23.710118+0000","last_unstale":"2024-05-28T13:58:23.714405+0000","last_undegraded":"2024-05-28T13:58:23.714405+0000","last_fullsized":"2024-05-28T13:58:23.714405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:23.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715806+0000","last_change":"2024-05-28T13:58:22.681209+0000","last_active":"2024-05-28T13:58:23.715806+0000","last_peered":"2024-05-28T13:58:23.715806+0000","last_clean":"2024-05-28T13:58:23.715806+0000","last_became_active":"2024-05-28T13:58:22.680889+0000","last_became_peered":"2024-05-28T13:58:22.680889+0000","last_unstale":"2024-05-28T13:58:23.715806+0000","last_undegraded":"2024-05-28T13:58:23.715806+0000","last_fullsized":"2024-05-28T13:58:23.715806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:02.116873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.714445+0000","last_change":"2024-05-28T13:58:23.710507+0000","last_active":"2024-05-28T13:58:23.714445+0000","last_peered":"2024-05-28T13:58:23.714445+0000","last_clean":"2024-05-28T13:58:23.714445+0000","last_became_active":"2024-05-28T13:58:23.710104+0000","last_became_peered":"2024-05-28T13:58:23.710104+0000","last_unstale":"2024-05-28T13:58:23.714445+0000","last_undegraded":"2024-05-28T13:58:23.714445+0000","last_fullsized":"2024-05-28T13:58:23.714445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:03:59.917115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.705096+0000","last_change":"2024-05-28T13:58:22.688976+0000","last_active":"2024-05-28T13:58:23.705096+0000","last_peered":"2024-05-28T13:58:23.705096+0000","last_clean":"2024-05-28T13:58:23.705096+0000","last_became_active":"2024-05-28T13:58:22.688607+0000","last_became_peered":"2024-05-28T13:58:22.688607+0000","last_unstale":"2024-05-28T13:58:23.705096+0000","last_undegraded":"2024-05-28T13:58:23.705096+0000","last_fullsized":"2024-05-28T13:58:23.705096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:55.898265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.702358+0000","last_change":"2024-05-28T13:58:23.695036+0000","last_active":"2024-05-28T13:58:23.702358+0000","last_peered":"2024-05-28T13:58:23.702358+0000","last_clean":"2024-05-28T13:58:23.702358+0000","last_became_active":"2024-05-28T13:58:23.692922+0000","last_became_peered":"2024-05-28T13:58:23.692922+0000","last_unstale":"2024-05-28T13:58:23.702358+0000","last_undegraded":"2024-05-28T13:58:23.702358+0000","last_fullsized":"2024-05-28T13:58:23.702358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:36.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.711165+0000","last_change":"2024-05-28T13:58:22.684711+0000","last_active":"2024-05-28T13:58:23.711165+0000","last_peered":"2024-05-28T13:58:23.711165+0000","last_clean":"2024-05-28T13:58:23.711165+0000","last_became_active":"2024-05-28T13:58:22.680359+0000","last_became_peered":"2024-05-28T13:58:22.680359+0000","last_unstale":"2024-05-28T13:58:23.711165+0000","last_undegraded":"2024-05-28T13:58:23.711165+0000","last_fullsized":"2024-05-28T13:58:23.711165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.691389+0000","last_change":"2024-05-28T13:58:23.690282+0000","last_active":"2024-05-28T13:58:23.691389+0000","last_peered":"2024-05-28T13:58:23.691389+0000","last_clean":"2024-05-28T13:58:23.691389+0000","last_became_active":"2024-05-28T13:58:23.689975+0000","last_became_peered":"2024-05-28T13:58:23.689975+0000","last_unstale":"2024-05-28T13:58:23.691389+0000","last_undegraded":"2024-05-28T13:58:23.691389+0000","last_fullsized":"2024-05-28T13:58:23.691389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:59.716378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.708882+0000","last_change":"2024-05-28T13:58:22.679511+0000","last_active":"2024-05-28T13:58:23.708882+0000","last_peered":"2024-05-28T13:58:23.708882+0000","last_clean":"2024-05-28T13:58:23.708882+0000","last_became_active":"2024-05-28T13:58:22.679207+0000","last_became_peered":"2024-05-28T13:58:22.679207+0000","last_unstale":"2024-05-28T13:58:23.708882+0000","last_undegraded":"2024-05-28T13:58:23.708882+0000","last_fullsized":"2024-05-28T13:58:23.708882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":660255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":69395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":69431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":70917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":204800,"data_stored":69305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":69465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":811008,"data_stored":660311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T13:58:27.789 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T13:58:27.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd135e00640 1 Processor -- start 2024-05-28T13:58:27.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd135e00640 1 -- start start 2024-05-28T13:58:27.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd135e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd130070800 0x7fd130070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd135e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd13006ff10 0x7fd13009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd135e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd130070370 0x7fd13009af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd135e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd1300d0660 con 0x7fd13006ff10 2024-05-28T13:58:27.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd135e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd1300d0940 con 0x7fd130070370 2024-05-28T13:58:27.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd135e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd1300d0c20 con 0x7fd130070800 2024-05-28T13:58:27.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd12fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd13006ff10 0x7fd13009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd12fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd13006ff10 0x7fd13009aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38358/0 (socket says 172.21.15.42:38358) 2024-05-28T13:58:27.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd12fe00640 1 -- 172.21.15.42:0/1442401915 learned_addr learned my addr 172.21.15.42:0/1442401915 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:27.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd12fe00640 1 -- 172.21.15.42:0/1442401915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd130070800 msgr2=0x7fd130070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:27.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd12fe00640 1 --2- 172.21.15.42:0/1442401915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd130070800 0x7fd130070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd12fe00640 1 -- 172.21.15.42:0/1442401915 >> v2:172.21.15.63:3300/0 conn(0x7fd130070370 msgr2=0x7fd13009af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:27.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd12fe00640 1 --2- 172.21.15.42:0/1442401915 >> v2:172.21.15.63:3300/0 conn(0x7fd130070370 0x7fd13009af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.985+0000 7fd12fe00640 1 -- 172.21.15.42:0/1442401915 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1300d0f00 con 0x7fd13006ff10 2024-05-28T13:58:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd12fe00640 1 --2- 172.21.15.42:0/1442401915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd13006ff10 0x7fd13009aa50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd1300cc8c0 tx=0x7fd11c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b15817f957caee8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd12f400640 1 -- 172.21.15.42:0/1442401915 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0b2070 con 0x7fd13006ff10 2024-05-28T13:58:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd12f400640 1 -- 172.21.15.42:0/1442401915 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd11c0a9070 con 0x7fd13006ff10 2024-05-28T13:58:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd12f400640 1 -- 172.21.15.42:0/1442401915 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0ad880 con 0x7fd13006ff10 2024-05-28T13:58:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd135e00640 1 -- 172.21.15.42:0/1442401915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd13006ff10 msgr2=0x7fd13009aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd135e00640 1 --2- 172.21.15.42:0/1442401915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd13006ff10 0x7fd13009aa50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd1300cc8c0 tx=0x7fd11c01c520 comp rx=0 tx=0).stop 2024-05-28T13:58:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd135e00640 1 -- 172.21.15.42:0/1442401915 shutdown_connections 2024-05-28T13:58:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd135e00640 1 --2- 172.21.15.42:0/1442401915 >> v2:172.21.15.63:3300/0 conn(0x7fd130070370 0x7fd13009af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd135e00640 1 --2- 172.21.15.42:0/1442401915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd13006ff10 0x7fd13009aa50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd135e00640 1 --2- 172.21.15.42:0/1442401915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd130070800 0x7fd130070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd135e00640 1 -- 172.21.15.42:0/1442401915 >> 172.21.15.42:0/1442401915 conn(0x7fd1300b3dd0 msgr2=0x7fd1300b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd135e00640 1 -- 172.21.15.42:0/1442401915 shutdown_connections 2024-05-28T13:58:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd135e00640 1 -- 172.21.15.42:0/1442401915 wait complete. 2024-05-28T13:58:27.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd135e00640 1 Processor -- start 2024-05-28T13:58:27.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.989+0000 7fd135e00640 1 -- start start 2024-05-28T13:58:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd13006ff10 0x7fd1301f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd130070370 0x7fd1300cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd130070800 0x7fd1300b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd13009ba60 con 0x7fd130070800 2024-05-28T13:58:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd13009b8e0 con 0x7fd130070370 2024-05-28T13:58:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd1300b1890 con 0x7fd13006ff10 2024-05-28T13:58:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd130070800 0x7fd1300b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd130070800 0x7fd1300b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38372/0 (socket says 172.21.15.42:38372) 2024-05-28T13:58:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135400640 1 -- 172.21.15.42:0/1076026681 learned_addr learned my addr 172.21.15.42:0/1076026681 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd12fe00640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd130070370 0x7fd1300cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd134a00640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd13006ff10 0x7fd1301f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135400640 1 -- 172.21.15.42:0/1076026681 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd13006ff10 msgr2=0x7fd1301f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135400640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd13006ff10 0x7fd1301f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135400640 1 -- 172.21.15.42:0/1076026681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd130070370 msgr2=0x7fd1300cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:27.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135400640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd130070370 0x7fd1300cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:27.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135400640 1 -- 172.21.15.42:0/1076026681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1301f83f0 con 0x7fd130070800 2024-05-28T13:58:27.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd134a00640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd13006ff10 0x7fd1301f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:27.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135400640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd130070800 0x7fd1300b00e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd12800b430 tx=0x7fd12800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:27.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd12cc00640 1 -- 172.21.15.42:0/1076026681 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128015020 con 0x7fd130070800 2024-05-28T13:58:27.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd12cc00640 1 -- 172.21.15.42:0/1076026681 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd128011830 con 0x7fd130070800 2024-05-28T13:58:27.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd12cc00640 1 -- 172.21.15.42:0/1076026681 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128011c20 con 0x7fd130070800 2024-05-28T13:58:27.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135e00640 1 -- 172.21.15.42:0/1076026681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1301f86d0 con 0x7fd130070800 2024-05-28T13:58:27.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135e00640 1 -- 172.21.15.42:0/1076026681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1301fb3f0 con 0x7fd130070800 2024-05-28T13:58:27.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.993+0000 7fd135e00640 1 -- 172.21.15.42:0/1076026681 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd13004fcf0 con 0x7fd130070800 2024-05-28T13:58:27.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.997+0000 7fd12cc00640 1 -- 172.21.15.42:0/1076026681 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd128018020 con 0x7fd130070800 2024-05-28T13:58:27.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.997+0000 7fd12cc00640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd1100683b0 0x7fd11006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:27.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.997+0000 7fd12cc00640 1 -- 172.21.15.42:0/1076026681 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd128086840 con 0x7fd130070800 2024-05-28T13:58:27.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.997+0000 7fd134a00640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd1100683b0 0x7fd11006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:27.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:27.997+0000 7fd134a00640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd1100683b0 0x7fd11006a870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd124007eb0 tx=0x7fd1240073c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:27.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.001+0000 7fd12cc00640 1 -- 172.21.15.42:0/1076026681 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd12801f020 con 0x7fd130070800 2024-05-28T13:58:28.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.161+0000 7fd135e00640 1 -- 172.21.15.42:0/1076026681 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd1300b0620 con 0x7fd1100683b0 2024-05-28T13:58:28.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.169+0000 7fd12cc00640 1 -- 172.21.15.42:0/1076026681 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251883 (secure 0 0 0) 0x7fd1300b0620 con 0x7fd1100683b0 2024-05-28T13:58:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:28.157 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T13:58:28.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.173+0000 7fd135e00640 1 -- 172.21.15.42:0/1076026681 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd1100683b0 msgr2=0x7fd11006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.173+0000 7fd135e00640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd1100683b0 0x7fd11006a870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd124007eb0 tx=0x7fd1240073c0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.173+0000 7fd135e00640 1 -- 172.21.15.42:0/1076026681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd130070800 msgr2=0x7fd1300b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.173+0000 7fd135e00640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd130070800 0x7fd1300b00e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd12800b430 tx=0x7fd12800b8f0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.173+0000 7fd135e00640 1 -- 172.21.15.42:0/1076026681 shutdown_connections 2024-05-28T13:58:28.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.173+0000 7fd135e00640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd1100683b0 0x7fd11006a870 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.173+0000 7fd135e00640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd130070800 0x7fd1300b00e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.173+0000 7fd135e00640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd130070370 0x7fd1300cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.173+0000 7fd135e00640 1 --2- 172.21.15.42:0/1076026681 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd13006ff10 0x7fd1301f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.173+0000 7fd135e00640 1 -- 172.21.15.42:0/1076026681 >> 172.21.15.42:0/1076026681 conn(0x7fd1300b3dd0 msgr2=0x7fd13006cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:28.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.173+0000 7fd135e00640 1 -- 172.21.15.42:0/1076026681 shutdown_connections 2024-05-28T13:58:28.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.173+0000 7fd135e00640 1 -- 172.21.15.42:0/1076026681 wait complete. 2024-05-28T13:58:28.176 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-28T13:58:26.856642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235644,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739076,"statfs":{"total":773094113280,"available":772852813824,"internally_reserved":0,"allocated":2891776,"data_stored":1739996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.210998"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.698853+0000","last_change":"2024-05-28T13:58:23.690106+0000","last_active":"2024-05-28T13:58:23.698853+0000","last_peered":"2024-05-28T13:58:23.698853+0000","last_clean":"2024-05-28T13:58:23.698853+0000","last_became_active":"2024-05-28T13:58:23.689597+0000","last_became_peered":"2024-05-28T13:58:23.689597+0000","last_unstale":"2024-05-28T13:58:23.698853+0000","last_undegraded":"2024-05-28T13:58:23.698853+0000","last_fullsized":"2024-05-28T13:58:23.698853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:46.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.676326+0000","last_change":"2024-05-28T13:58:23.674151+0000","last_active":"2024-05-28T13:58:23.676326+0000","last_peered":"2024-05-28T13:58:23.676326+0000","last_clean":"2024-05-28T13:58:23.676326+0000","last_became_active":"2024-05-28T13:58:23.673848+0000","last_became_peered":"2024-05-28T13:58:23.673848+0000","last_unstale":"2024-05-28T13:58:23.676326+0000","last_undegraded":"2024-05-28T13:58:23.676326+0000","last_fullsized":"2024-05-28T13:58:23.676326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:13.470877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.688474+0000","last_change":"2024-05-28T13:58:23.685573+0000","last_active":"2024-05-28T13:58:23.688474+0000","last_peered":"2024-05-28T13:58:23.688474+0000","last_clean":"2024-05-28T13:58:23.688474+0000","last_became_active":"2024-05-28T13:58:23.685243+0000","last_became_peered":"2024-05-28T13:58:23.685243+0000","last_unstale":"2024-05-28T13:58:23.688474+0000","last_undegraded":"2024-05-28T13:58:23.688474+0000","last_fullsized":"2024-05-28T13:58:23.688474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:03.122796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.710763+0000","last_change":"2024-05-28T13:58:23.706150+0000","last_active":"2024-05-28T13:58:23.710763+0000","last_peered":"2024-05-28T13:58:23.710763+0000","last_clean":"2024-05-28T13:58:23.710763+0000","last_became_active":"2024-05-28T13:58:23.689938+0000","last_became_peered":"2024-05-28T13:58:23.689938+0000","last_unstale":"2024-05-28T13:58:23.710763+0000","last_undegraded":"2024-05-28T13:58:23.710763+0000","last_fullsized":"2024-05-28T13:58:23.710763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:10.036541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.693679+0000","last_change":"2024-05-28T13:58:23.683882+0000","last_active":"2024-05-28T13:58:23.693679+0000","last_peered":"2024-05-28T13:58:23.693679+0000","last_clean":"2024-05-28T13:58:23.693679+0000","last_became_active":"2024-05-28T13:58:23.683368+0000","last_became_peered":"2024-05-28T13:58:23.683368+0000","last_unstale":"2024-05-28T13:58:23.693679+0000","last_undegraded":"2024-05-28T13:58:23.693679+0000","last_fullsized":"2024-05-28T13:58:23.693679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:12:43.447237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.709862+0000","last_change":"2024-05-28T13:58:23.695111+0000","last_active":"2024-05-28T13:58:23.709862+0000","last_peered":"2024-05-28T13:58:23.709862+0000","last_clean":"2024-05-28T13:58:23.709862+0000","last_became_active":"2024-05-28T13:58:23.688505+0000","last_became_peered":"2024-05-28T13:58:23.688505+0000","last_unstale":"2024-05-28T13:58:23.709862+0000","last_undegraded":"2024-05-28T13:58:23.709862+0000","last_fullsized":"2024-05-28T13:58:23.709862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:07:03.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.697549+0000","last_change":"2024-05-28T13:58:23.684793+0000","last_active":"2024-05-28T13:58:23.697549+0000","last_peered":"2024-05-28T13:58:23.697549+0000","last_clean":"2024-05-28T13:58:23.697549+0000","last_became_active":"2024-05-28T13:58:23.684500+0000","last_became_peered":"2024-05-28T13:58:23.684500+0000","last_unstale":"2024-05-28T13:58:23.697549+0000","last_undegraded":"2024-05-28T13:58:23.697549+0000","last_fullsized":"2024-05-28T13:58:23.697549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:07:03.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.689728+0000","last_change":"2024-05-28T13:58:23.688670+0000","last_active":"2024-05-28T13:58:23.689728+0000","last_peered":"2024-05-28T13:58:23.689728+0000","last_clean":"2024-05-28T13:58:23.689728+0000","last_became_active":"2024-05-28T13:58:23.688243+0000","last_became_peered":"2024-05-28T13:58:23.688243+0000","last_unstale":"2024-05-28T13:58:23.689728+0000","last_undegraded":"2024-05-28T13:58:23.689728+0000","last_fullsized":"2024-05-28T13:58:23.689728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:56.904188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.709234+0000","last_change":"2024-05-28T13:58:23.702434+0000","last_active":"2024-05-28T13:58:23.709234+0000","last_peered":"2024-05-28T13:58:23.709234+0000","last_clean":"2024-05-28T13:58:23.709234+0000","last_became_active":"2024-05-28T13:58:23.701857+0000","last_became_peered":"2024-05-28T13:58:23.701857+0000","last_unstale":"2024-05-28T13:58:23.709234+0000","last_undegraded":"2024-05-28T13:58:23.709234+0000","last_fullsized":"2024-05-28T13:58:23.709234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:03:59.917115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.711329+0000","last_change":"2024-05-28T13:58:23.709688+0000","last_active":"2024-05-28T13:58:23.711329+0000","last_peered":"2024-05-28T13:58:23.711329+0000","last_clean":"2024-05-28T13:58:23.711329+0000","last_became_active":"2024-05-28T13:58:23.709431+0000","last_became_peered":"2024-05-28T13:58:23.709431+0000","last_unstale":"2024-05-28T13:58:23.711329+0000","last_undegraded":"2024-05-28T13:58:23.711329+0000","last_fullsized":"2024-05-28T13:58:23.711329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:12:43.447237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.708252+0000","last_change":"2024-05-28T13:58:23.701527+0000","last_active":"2024-05-28T13:58:23.708252+0000","last_peered":"2024-05-28T13:58:23.708252+0000","last_clean":"2024-05-28T13:58:23.708252+0000","last_became_active":"2024-05-28T13:58:23.701160+0000","last_became_peered":"2024-05-28T13:58:23.701160+0000","last_unstale":"2024-05-28T13:58:23.708252+0000","last_undegraded":"2024-05-28T13:58:23.708252+0000","last_fullsized":"2024-05-28T13:58:23.708252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:13.470877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715282+0000","last_change":"2024-05-28T13:58:23.707766+0000","last_active":"2024-05-28T13:58:23.715282+0000","last_peered":"2024-05-28T13:58:23.715282+0000","last_clean":"2024-05-28T13:58:23.715282+0000","last_became_active":"2024-05-28T13:58:23.707542+0000","last_became_peered":"2024-05-28T13:58:23.707542+0000","last_unstale":"2024-05-28T13:58:23.715282+0000","last_undegraded":"2024-05-28T13:58:23.715282+0000","last_fullsized":"2024-05-28T13:58:23.715282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:39.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.688953+0000","last_change":"2024-05-28T13:58:23.687526+0000","last_active":"2024-05-28T13:58:23.688953+0000","last_peered":"2024-05-28T13:58:23.688953+0000","last_clean":"2024-05-28T13:58:23.688953+0000","last_became_active":"2024-05-28T13:58:23.687138+0000","last_became_peered":"2024-05-28T13:58:23.687138+0000","last_unstale":"2024-05-28T13:58:23.688953+0000","last_undegraded":"2024-05-28T13:58:23.688953+0000","last_fullsized":"2024-05-28T13:58:23.688953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:36.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.687615+0000","last_change":"2024-05-28T13:58:23.681967+0000","last_active":"2024-05-28T13:58:23.687615+0000","last_peered":"2024-05-28T13:58:23.687615+0000","last_clean":"2024-05-28T13:58:23.687615+0000","last_became_active":"2024-05-28T13:58:23.680491+0000","last_became_peered":"2024-05-28T13:58:23.680491+0000","last_unstale":"2024-05-28T13:58:23.687615+0000","last_undegraded":"2024-05-28T13:58:23.687615+0000","last_fullsized":"2024-05-28T13:58:23.687615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:08.011020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.710950+0000","last_change":"2024-05-28T13:58:23.708051+0000","last_active":"2024-05-28T13:58:23.710950+0000","last_peered":"2024-05-28T13:58:23.710950+0000","last_clean":"2024-05-28T13:58:23.710950+0000","last_became_active":"2024-05-28T13:58:23.707749+0000","last_became_peered":"2024-05-28T13:58:23.707749+0000","last_unstale":"2024-05-28T13:58:23.710950+0000","last_undegraded":"2024-05-28T13:58:23.710950+0000","last_fullsized":"2024-05-28T13:58:23.710950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:07:03.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.693440+0000","last_change":"2024-05-28T13:58:23.684368+0000","last_active":"2024-05-28T13:58:23.693440+0000","last_peered":"2024-05-28T13:58:23.693440+0000","last_clean":"2024-05-28T13:58:23.693440+0000","last_became_active":"2024-05-28T13:58:23.683905+0000","last_became_peered":"2024-05-28T13:58:23.683905+0000","last_unstale":"2024-05-28T13:58:23.693440+0000","last_undegraded":"2024-05-28T13:58:23.693440+0000","last_fullsized":"2024-05-28T13:58:23.693440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:36.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.710968+0000","last_change":"2024-05-28T13:58:23.706633+0000","last_active":"2024-05-28T13:58:23.710968+0000","last_peered":"2024-05-28T13:58:23.710968+0000","last_clean":"2024-05-28T13:58:23.710968+0000","last_became_active":"2024-05-28T13:58:23.693217+0000","last_became_peered":"2024-05-28T13:58:23.693217+0000","last_unstale":"2024-05-28T13:58:23.710968+0000","last_undegraded":"2024-05-28T13:58:23.710968+0000","last_fullsized":"2024-05-28T13:58:23.710968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:22.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.691229+0000","last_change":"2024-05-28T13:58:23.690320+0000","last_active":"2024-05-28T13:58:23.691229+0000","last_peered":"2024-05-28T13:58:23.691229+0000","last_clean":"2024-05-28T13:58:23.691229+0000","last_became_active":"2024-05-28T13:58:23.690028+0000","last_became_peered":"2024-05-28T13:58:23.690028+0000","last_unstale":"2024-05-28T13:58:23.691229+0000","last_undegraded":"2024-05-28T13:58:23.691229+0000","last_fullsized":"2024-05-28T13:58:23.691229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:08.011020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.691324+0000","last_change":"2024-05-28T13:58:23.683310+0000","last_active":"2024-05-28T13:58:23.691324+0000","last_peered":"2024-05-28T13:58:23.691324+0000","last_clean":"2024-05-28T13:58:23.691324+0000","last_became_active":"2024-05-28T13:58:23.682882+0000","last_became_peered":"2024-05-28T13:58:23.682882+0000","last_unstale":"2024-05-28T13:58:23.691324+0000","last_undegraded":"2024-05-28T13:58:23.691324+0000","last_fullsized":"2024-05-28T13:58:23.691324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:45.207023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.708646+0000","last_change":"2024-05-28T13:58:23.706004+0000","last_active":"2024-05-28T13:58:23.708646+0000","last_peered":"2024-05-28T13:58:23.708646+0000","last_clean":"2024-05-28T13:58:23.708646+0000","last_became_active":"2024-05-28T13:58:23.705676+0000","last_became_peered":"2024-05-28T13:58:23.705676+0000","last_unstale":"2024-05-28T13:58:23.708646+0000","last_undegraded":"2024-05-28T13:58:23.708646+0000","last_fullsized":"2024-05-28T13:58:23.708646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:45.207023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.684136+0000","last_change":"2024-05-28T13:58:23.678693+0000","last_active":"2024-05-28T13:58:23.684136+0000","last_peered":"2024-05-28T13:58:23.684136+0000","last_clean":"2024-05-28T13:58:23.684136+0000","last_became_active":"2024-05-28T13:58:23.678258+0000","last_became_peered":"2024-05-28T13:58:23.678258+0000","last_unstale":"2024-05-28T13:58:23.684136+0000","last_undegraded":"2024-05-28T13:58:23.684136+0000","last_fullsized":"2024-05-28T13:58:23.684136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:35.740742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.710779+0000","last_change":"2024-05-28T13:58:23.706461+0000","last_active":"2024-05-28T13:58:23.710779+0000","last_peered":"2024-05-28T13:58:23.710779+0000","last_clean":"2024-05-28T13:58:23.710779+0000","last_became_active":"2024-05-28T13:58:23.702598+0000","last_became_peered":"2024-05-28T13:58:23.702598+0000","last_unstale":"2024-05-28T13:58:23.710779+0000","last_undegraded":"2024-05-28T13:58:23.710779+0000","last_fullsized":"2024-05-28T13:58:23.710779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:46.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715591+0000","last_change":"2024-05-28T13:58:23.714967+0000","last_active":"2024-05-28T13:58:23.715591+0000","last_peered":"2024-05-28T13:58:23.715591+0000","last_clean":"2024-05-28T13:58:23.715591+0000","last_became_active":"2024-05-28T13:58:23.714653+0000","last_became_peered":"2024-05-28T13:58:23.714653+0000","last_unstale":"2024-05-28T13:58:23.715591+0000","last_undegraded":"2024-05-28T13:58:23.715591+0000","last_fullsized":"2024-05-28T13:58:23.715591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:07:03.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.703866+0000","last_change":"2024-05-28T13:58:23.686847+0000","last_active":"2024-05-28T13:58:23.703866+0000","last_peered":"2024-05-28T13:58:23.703866+0000","last_clean":"2024-05-28T13:58:23.703866+0000","last_became_active":"2024-05-28T13:58:23.686518+0000","last_became_peered":"2024-05-28T13:58:23.686518+0000","last_unstale":"2024-05-28T13:58:23.703866+0000","last_undegraded":"2024-05-28T13:58:23.703866+0000","last_fullsized":"2024-05-28T13:58:23.703866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:59.716378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.709684+0000","last_change":"2024-05-28T13:58:23.707351+0000","last_active":"2024-05-28T13:58:23.709684+0000","last_peered":"2024-05-28T13:58:23.709684+0000","last_clean":"2024-05-28T13:58:23.709684+0000","last_became_active":"2024-05-28T13:58:23.707077+0000","last_became_peered":"2024-05-28T13:58:23.707077+0000","last_unstale":"2024-05-28T13:58:23.709684+0000","last_undegraded":"2024-05-28T13:58:23.709684+0000","last_fullsized":"2024-05-28T13:58:23.709684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:12:43.447237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.694717+0000","last_change":"2024-05-28T13:58:23.688056+0000","last_active":"2024-05-28T13:58:23.694717+0000","last_peered":"2024-05-28T13:58:23.694717+0000","last_clean":"2024-05-28T13:58:23.694717+0000","last_became_active":"2024-05-28T13:58:23.687623+0000","last_became_peered":"2024-05-28T13:58:23.687623+0000","last_unstale":"2024-05-28T13:58:23.694717+0000","last_undegraded":"2024-05-28T13:58:23.694717+0000","last_fullsized":"2024-05-28T13:58:23.694717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:59.716378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.677334+0000","last_change":"2024-05-28T13:58:23.672661+0000","last_active":"2024-05-28T13:58:23.677334+0000","last_peered":"2024-05-28T13:58:23.677334+0000","last_clean":"2024-05-28T13:58:23.677334+0000","last_became_active":"2024-05-28T13:58:23.672378+0000","last_became_peered":"2024-05-28T13:58:23.672378+0000","last_unstale":"2024-05-28T13:58:23.677334+0000","last_undegraded":"2024-05-28T13:58:23.677334+0000","last_fullsized":"2024-05-28T13:58:23.677334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:56.904188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.709610+0000","last_change":"2024-05-28T13:58:23.707282+0000","last_active":"2024-05-28T13:58:23.709610+0000","last_peered":"2024-05-28T13:58:23.709610+0000","last_clean":"2024-05-28T13:58:23.709610+0000","last_became_active":"2024-05-28T13:58:23.706977+0000","last_became_peered":"2024-05-28T13:58:23.706977+0000","last_unstale":"2024-05-28T13:58:23.709610+0000","last_undegraded":"2024-05-28T13:58:23.709610+0000","last_fullsized":"2024-05-28T13:58:23.709610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:45.207023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.687807+0000","last_change":"2024-05-28T13:58:23.684735+0000","last_active":"2024-05-28T13:58:23.687807+0000","last_peered":"2024-05-28T13:58:23.687807+0000","last_clean":"2024-05-28T13:58:23.687807+0000","last_became_active":"2024-05-28T13:58:23.684389+0000","last_became_peered":"2024-05-28T13:58:23.684389+0000","last_unstale":"2024-05-28T13:58:23.687807+0000","last_undegraded":"2024-05-28T13:58:23.687807+0000","last_fullsized":"2024-05-28T13:58:23.687807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:30.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.703762+0000","last_change":"2024-05-28T13:58:23.686822+0000","last_active":"2024-05-28T13:58:23.703762+0000","last_peered":"2024-05-28T13:58:23.703762+0000","last_clean":"2024-05-28T13:58:23.703762+0000","last_became_active":"2024-05-28T13:58:23.686454+0000","last_became_peered":"2024-05-28T13:58:23.686454+0000","last_unstale":"2024-05-28T13:58:23.703762+0000","last_undegraded":"2024-05-28T13:58:23.703762+0000","last_fullsized":"2024-05-28T13:58:23.703762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:56.904188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.703117+0000","last_change":"2024-05-28T13:58:23.702203+0000","last_active":"2024-05-28T13:58:23.703117+0000","last_peered":"2024-05-28T13:58:23.703117+0000","last_clean":"2024-05-28T13:58:23.703117+0000","last_became_active":"2024-05-28T13:58:23.701830+0000","last_became_peered":"2024-05-28T13:58:23.701830+0000","last_unstale":"2024-05-28T13:58:23.703117+0000","last_undegraded":"2024-05-28T13:58:23.703117+0000","last_fullsized":"2024-05-28T13:58:23.703117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:30.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.694193+0000","last_change":"2024-05-28T13:58:23.684058+0000","last_active":"2024-05-28T13:58:23.694193+0000","last_peered":"2024-05-28T13:58:23.694193+0000","last_clean":"2024-05-28T13:58:23.694193+0000","last_became_active":"2024-05-28T13:58:23.683555+0000","last_became_peered":"2024-05-28T13:58:23.683555+0000","last_unstale":"2024-05-28T13:58:23.694193+0000","last_undegraded":"2024-05-28T13:58:23.694193+0000","last_fullsized":"2024-05-28T13:58:23.694193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:36.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.713509+0000","last_change":"2024-05-28T13:58:23.704842+0000","last_active":"2024-05-28T13:58:23.713509+0000","last_peered":"2024-05-28T13:58:23.713509+0000","last_clean":"2024-05-28T13:58:23.713509+0000","last_became_active":"2024-05-28T13:58:23.704465+0000","last_became_peered":"2024-05-28T13:58:23.704465+0000","last_unstale":"2024-05-28T13:58:23.713509+0000","last_undegraded":"2024-05-28T13:58:23.713509+0000","last_fullsized":"2024-05-28T13:58:23.713509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:53.495402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.712747+0000","last_change":"2024-05-28T13:58:22.678741+0000","last_active":"2024-05-28T13:58:23.712747+0000","last_peered":"2024-05-28T13:58:23.712747+0000","last_clean":"2024-05-28T13:58:23.712747+0000","last_became_active":"2024-05-28T13:58:22.678532+0000","last_became_peered":"2024-05-28T13:58:22.678532+0000","last_unstale":"2024-05-28T13:58:23.712747+0000","last_undegraded":"2024-05-28T13:58:23.712747+0000","last_fullsized":"2024-05-28T13:58:23.712747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.710717+0000","last_change":"2024-05-28T13:58:23.708224+0000","last_active":"2024-05-28T13:58:23.710717+0000","last_peered":"2024-05-28T13:58:23.710717+0000","last_clean":"2024-05-28T13:58:23.710717+0000","last_became_active":"2024-05-28T13:58:23.707930+0000","last_became_peered":"2024-05-28T13:58:23.707930+0000","last_unstale":"2024-05-28T13:58:23.710717+0000","last_undegraded":"2024-05-28T13:58:23.710717+0000","last_fullsized":"2024-05-28T13:58:23.710717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:59.716378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715475+0000","last_change":"2024-05-28T13:58:22.673310+0000","last_active":"2024-05-28T13:58:23.715475+0000","last_peered":"2024-05-28T13:58:23.715475+0000","last_clean":"2024-05-28T13:58:23.715475+0000","last_became_active":"2024-05-28T13:58:22.673003+0000","last_became_peered":"2024-05-28T13:58:22.673003+0000","last_unstale":"2024-05-28T13:58:23.715475+0000","last_undegraded":"2024-05-28T13:58:23.715475+0000","last_fullsized":"2024-05-28T13:58:23.715475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.668422+0000","last_change":"2024-05-28T13:58:22.677344+0000","last_active":"2024-05-28T13:58:23.668422+0000","last_peered":"2024-05-28T13:58:23.668422+0000","last_clean":"2024-05-28T13:58:23.668422+0000","last_became_active":"2024-05-28T13:58:22.677024+0000","last_became_peered":"2024-05-28T13:58:22.677024+0000","last_unstale":"2024-05-28T13:58:23.668422+0000","last_undegraded":"2024-05-28T13:58:23.668422+0000","last_fullsized":"2024-05-28T13:58:23.668422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:52.890121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.688012+0000","last_change":"2024-05-28T13:58:23.686005+0000","last_active":"2024-05-28T13:58:23.688012+0000","last_peered":"2024-05-28T13:58:23.688012+0000","last_clean":"2024-05-28T13:58:23.688012+0000","last_became_active":"2024-05-28T13:58:23.682423+0000","last_became_peered":"2024-05-28T13:58:23.682423+0000","last_unstale":"2024-05-28T13:58:23.688012+0000","last_undegraded":"2024-05-28T13:58:23.688012+0000","last_fullsized":"2024-05-28T13:58:23.688012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:03.122796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.669976+0000","last_change":"2024-05-28T13:58:22.673902+0000","last_active":"2024-05-28T13:58:23.669976+0000","last_peered":"2024-05-28T13:58:23.669976+0000","last_clean":"2024-05-28T13:58:23.669976+0000","last_became_active":"2024-05-28T13:58:22.673588+0000","last_became_peered":"2024-05-28T13:58:22.673588+0000","last_unstale":"2024-05-28T13:58:23.669976+0000","last_undegraded":"2024-05-28T13:58:23.669976+0000","last_fullsized":"2024-05-28T13:58:23.669976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.676281+0000","last_change":"2024-05-28T13:58:23.672761+0000","last_active":"2024-05-28T13:58:23.676281+0000","last_peered":"2024-05-28T13:58:23.676281+0000","last_clean":"2024-05-28T13:58:23.676281+0000","last_became_active":"2024-05-28T13:58:23.672276+0000","last_became_peered":"2024-05-28T13:58:23.672276+0000","last_unstale":"2024-05-28T13:58:23.676281+0000","last_undegraded":"2024-05-28T13:58:23.676281+0000","last_fullsized":"2024-05-28T13:58:23.676281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:59.716378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.673183+0000","last_change":"2024-05-28T13:58:22.676336+0000","last_active":"2024-05-28T13:58:23.673183+0000","last_peered":"2024-05-28T13:58:23.673183+0000","last_clean":"2024-05-28T13:58:23.673183+0000","last_became_active":"2024-05-28T13:58:22.676013+0000","last_became_peered":"2024-05-28T13:58:22.676013+0000","last_unstale":"2024-05-28T13:58:23.673183+0000","last_undegraded":"2024-05-28T13:58:23.673183+0000","last_fullsized":"2024-05-28T13:58:23.673183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.707875+0000","last_change":"2024-05-28T13:58:23.691740+0000","last_active":"2024-05-28T13:58:23.707875+0000","last_peered":"2024-05-28T13:58:23.707875+0000","last_clean":"2024-05-28T13:58:23.707875+0000","last_became_active":"2024-05-28T13:58:23.691056+0000","last_became_peered":"2024-05-28T13:58:23.691056+0000","last_unstale":"2024-05-28T13:58:23.707875+0000","last_undegraded":"2024-05-28T13:58:23.707875+0000","last_fullsized":"2024-05-28T13:58:23.707875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:45.207023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.677240+0000","last_change":"2024-05-28T13:58:22.678043+0000","last_active":"2024-05-28T13:58:23.677240+0000","last_peered":"2024-05-28T13:58:23.677240+0000","last_clean":"2024-05-28T13:58:23.677240+0000","last_became_active":"2024-05-28T13:58:22.677684+0000","last_became_peered":"2024-05-28T13:58:22.677684+0000","last_unstale":"2024-05-28T13:58:23.677240+0000","last_undegraded":"2024-05-28T13:58:23.677240+0000","last_fullsized":"2024-05-28T13:58:23.677240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:02.116873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.709891+0000","last_change":"2024-05-28T13:58:23.706540+0000","last_active":"2024-05-28T13:58:23.709891+0000","last_peered":"2024-05-28T13:58:23.709891+0000","last_clean":"2024-05-28T13:58:23.709891+0000","last_became_active":"2024-05-28T13:58:23.706209+0000","last_became_peered":"2024-05-28T13:58:23.706209+0000","last_unstale":"2024-05-28T13:58:23.709891+0000","last_undegraded":"2024-05-28T13:58:23.709891+0000","last_fullsized":"2024-05-28T13:58:23.709891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:56.904188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.718389+0000","last_change":"2024-05-28T13:58:22.684218+0000","last_active":"2024-05-28T13:58:23.718389+0000","last_peered":"2024-05-28T13:58:23.718389+0000","last_clean":"2024-05-28T13:58:23.718389+0000","last_became_active":"2024-05-28T13:58:22.683984+0000","last_became_peered":"2024-05-28T13:58:22.683984+0000","last_unstale":"2024-05-28T13:58:23.718389+0000","last_undegraded":"2024-05-28T13:58:23.718389+0000","last_fullsized":"2024-05-28T13:58:23.718389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715256+0000","last_change":"2024-05-28T13:58:23.705226+0000","last_active":"2024-05-28T13:58:23.715256+0000","last_peered":"2024-05-28T13:58:23.715256+0000","last_clean":"2024-05-28T13:58:23.715256+0000","last_became_active":"2024-05-28T13:58:23.704265+0000","last_became_peered":"2024-05-28T13:58:23.704265+0000","last_unstale":"2024-05-28T13:58:23.715256+0000","last_undegraded":"2024-05-28T13:58:23.715256+0000","last_fullsized":"2024-05-28T13:58:23.715256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:53.896043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.718907+0000","last_change":"2024-05-28T13:58:22.674273+0000","last_active":"2024-05-28T13:58:23.718907+0000","last_peered":"2024-05-28T13:58:23.718907+0000","last_clean":"2024-05-28T13:58:23.718907+0000","last_became_active":"2024-05-28T13:58:22.673984+0000","last_became_peered":"2024-05-28T13:58:22.673984+0000","last_unstale":"2024-05-28T13:58:23.718907+0000","last_undegraded":"2024-05-28T13:58:23.718907+0000","last_fullsized":"2024-05-28T13:58:23.718907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.698781+0000","last_change":"2024-05-28T13:58:23.685763+0000","last_active":"2024-05-28T13:58:23.698781+0000","last_peered":"2024-05-28T13:58:23.698781+0000","last_clean":"2024-05-28T13:58:23.698781+0000","last_became_active":"2024-05-28T13:58:23.685512+0000","last_became_peered":"2024-05-28T13:58:23.685512+0000","last_unstale":"2024-05-28T13:58:23.698781+0000","last_undegraded":"2024-05-28T13:58:23.698781+0000","last_fullsized":"2024-05-28T13:58:23.698781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:10.036541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.714209+0000","last_change":"2024-05-28T13:58:22.668047+0000","last_active":"2024-05-28T13:58:23.714209+0000","last_peered":"2024-05-28T13:58:23.714209+0000","last_clean":"2024-05-28T13:58:23.714209+0000","last_became_active":"2024-05-28T13:58:22.667747+0000","last_became_peered":"2024-05-28T13:58:22.667747+0000","last_unstale":"2024-05-28T13:58:23.714209+0000","last_undegraded":"2024-05-28T13:58:23.714209+0000","last_fullsized":"2024-05-28T13:58:23.714209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:52.489479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.680108+0000","last_change":"2024-05-28T13:58:23.675653+0000","last_active":"2024-05-28T13:58:23.680108+0000","last_peered":"2024-05-28T13:58:23.680108+0000","last_clean":"2024-05-28T13:58:23.680108+0000","last_became_active":"2024-05-28T13:58:23.675371+0000","last_became_peered":"2024-05-28T13:58:23.675371+0000","last_unstale":"2024-05-28T13:58:23.680108+0000","last_undegraded":"2024-05-28T13:58:23.680108+0000","last_fullsized":"2024-05-28T13:58:23.680108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:53.896043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.675637+0000","last_change":"2024-05-28T13:58:22.682942+0000","last_active":"2024-05-28T13:58:23.675637+0000","last_peered":"2024-05-28T13:58:23.675637+0000","last_clean":"2024-05-28T13:58:23.675637+0000","last_became_active":"2024-05-28T13:58:22.681946+0000","last_became_peered":"2024-05-28T13:58:22.681946+0000","last_unstale":"2024-05-28T13:58:23.675637+0000","last_undegraded":"2024-05-28T13:58:23.675637+0000","last_fullsized":"2024-05-28T13:58:23.675637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:07.005097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.721508+0000","last_change":"2024-05-28T13:58:23.716761+0000","last_active":"2024-05-28T13:58:23.721508+0000","last_peered":"2024-05-28T13:58:23.721508+0000","last_clean":"2024-05-28T13:58:23.721508+0000","last_became_active":"2024-05-28T13:58:23.716488+0000","last_became_peered":"2024-05-28T13:58:23.716488+0000","last_unstale":"2024-05-28T13:58:23.721508+0000","last_undegraded":"2024-05-28T13:58:23.721508+0000","last_fullsized":"2024-05-28T13:58:23.721508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:49.556769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.673110+0000","last_change":"2024-05-28T13:58:22.678451+0000","last_active":"2024-05-28T13:58:23.673110+0000","last_peered":"2024-05-28T13:58:23.673110+0000","last_clean":"2024-05-28T13:58:23.673110+0000","last_became_active":"2024-05-28T13:58:22.675695+0000","last_became_peered":"2024-05-28T13:58:22.675695+0000","last_unstale":"2024-05-28T13:58:23.673110+0000","last_undegraded":"2024-05-28T13:58:23.673110+0000","last_fullsized":"2024-05-28T13:58:23.673110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.694164+0000","last_change":"2024-05-28T13:58:23.683800+0000","last_active":"2024-05-28T13:58:23.694164+0000","last_peered":"2024-05-28T13:58:23.694164+0000","last_clean":"2024-05-28T13:58:23.694164+0000","last_became_active":"2024-05-28T13:58:23.683208+0000","last_became_peered":"2024-05-28T13:58:23.683208+0000","last_unstale":"2024-05-28T13:58:23.694164+0000","last_undegraded":"2024-05-28T13:58:23.694164+0000","last_fullsized":"2024-05-28T13:58:23.694164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:03.122796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.700600+0000","last_change":"2024-05-28T13:58:22.676468+0000","last_active":"2024-05-28T13:58:23.700600+0000","last_peered":"2024-05-28T13:58:23.700600+0000","last_clean":"2024-05-28T13:58:23.700600+0000","last_became_active":"2024-05-28T13:58:22.676202+0000","last_became_peered":"2024-05-28T13:58:22.676202+0000","last_unstale":"2024-05-28T13:58:23.700600+0000","last_undegraded":"2024-05-28T13:58:23.700600+0000","last_fullsized":"2024-05-28T13:58:23.700600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:52.890121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.721528+0000","last_change":"2024-05-28T13:58:23.717324+0000","last_active":"2024-05-28T13:58:23.721528+0000","last_peered":"2024-05-28T13:58:23.721528+0000","last_clean":"2024-05-28T13:58:23.721528+0000","last_became_active":"2024-05-28T13:58:23.716047+0000","last_became_peered":"2024-05-28T13:58:23.716047+0000","last_unstale":"2024-05-28T13:58:23.721528+0000","last_undegraded":"2024-05-28T13:58:23.721528+0000","last_fullsized":"2024-05-28T13:58:23.721528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:31.530501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.721934+0000","last_change":"2024-05-28T13:58:22.671015+0000","last_active":"2024-05-28T13:58:23.721934+0000","last_peered":"2024-05-28T13:58:23.721934+0000","last_clean":"2024-05-28T13:58:23.721934+0000","last_became_active":"2024-05-28T13:58:22.663746+0000","last_became_peered":"2024-05-28T13:58:22.663746+0000","last_unstale":"2024-05-28T13:58:23.721934+0000","last_undegraded":"2024-05-28T13:58:23.721934+0000","last_fullsized":"2024-05-28T13:58:23.721934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:22:51.238198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.709248+0000","last_change":"2024-05-28T13:58:23.705595+0000","last_active":"2024-05-28T13:58:23.709248+0000","last_peered":"2024-05-28T13:58:23.709248+0000","last_clean":"2024-05-28T13:58:23.709248+0000","last_became_active":"2024-05-28T13:58:23.705326+0000","last_became_peered":"2024-05-28T13:58:23.705326+0000","last_unstale":"2024-05-28T13:58:23.709248+0000","last_undegraded":"2024-05-28T13:58:23.709248+0000","last_fullsized":"2024-05-28T13:58:23.709248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:23.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.699199+0000","last_change":"2024-05-28T13:58:22.673480+0000","last_active":"2024-05-28T13:58:23.699199+0000","last_peered":"2024-05-28T13:58:23.699199+0000","last_clean":"2024-05-28T13:58:23.699199+0000","last_became_active":"2024-05-28T13:58:22.671634+0000","last_became_peered":"2024-05-28T13:58:22.671634+0000","last_unstale":"2024-05-28T13:58:23.699199+0000","last_undegraded":"2024-05-28T13:58:23.699199+0000","last_fullsized":"2024-05-28T13:58:23.699199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.711118+0000","last_change":"2024-05-28T13:58:23.708965+0000","last_active":"2024-05-28T13:58:23.711118+0000","last_peered":"2024-05-28T13:58:23.711118+0000","last_clean":"2024-05-28T13:58:23.711118+0000","last_became_active":"2024-05-28T13:58:23.708298+0000","last_became_peered":"2024-05-28T13:58:23.708298+0000","last_unstale":"2024-05-28T13:58:23.711118+0000","last_undegraded":"2024-05-28T13:58:23.711118+0000","last_fullsized":"2024-05-28T13:58:23.711118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:02.981053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715446+0000","last_change":"2024-05-28T13:58:22.688102+0000","last_active":"2024-05-28T13:58:23.715446+0000","last_peered":"2024-05-28T13:58:23.715446+0000","last_clean":"2024-05-28T13:58:23.715446+0000","last_became_active":"2024-05-28T13:58:22.686031+0000","last_became_peered":"2024-05-28T13:58:22.686031+0000","last_unstale":"2024-05-28T13:58:23.715446+0000","last_undegraded":"2024-05-28T13:58:23.715446+0000","last_fullsized":"2024-05-28T13:58:23.715446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.707795+0000","last_change":"2024-05-28T13:58:23.690696+0000","last_active":"2024-05-28T13:58:23.707795+0000","last_peered":"2024-05-28T13:58:23.707795+0000","last_clean":"2024-05-28T13:58:23.707795+0000","last_became_active":"2024-05-28T13:58:23.689615+0000","last_became_peered":"2024-05-28T13:58:23.689615+0000","last_unstale":"2024-05-28T13:58:23.707795+0000","last_undegraded":"2024-05-28T13:58:23.707795+0000","last_fullsized":"2024-05-28T13:58:23.707795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:13.470877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.678441+0000","last_change":"2024-05-28T13:58:22.674653+0000","last_active":"2024-05-28T13:58:23.678441+0000","last_peered":"2024-05-28T13:58:23.678441+0000","last_clean":"2024-05-28T13:58:23.678441+0000","last_became_active":"2024-05-28T13:58:22.674308+0000","last_became_peered":"2024-05-28T13:58:22.674308+0000","last_unstale":"2024-05-28T13:58:23.678441+0000","last_undegraded":"2024-05-28T13:58:23.678441+0000","last_fullsized":"2024-05-28T13:58:23.678441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.709185+0000","last_change":"2024-05-28T13:58:23.702244+0000","last_active":"2024-05-28T13:58:23.709185+0000","last_peered":"2024-05-28T13:58:23.709185+0000","last_clean":"2024-05-28T13:58:23.709185+0000","last_became_active":"2024-05-28T13:58:23.701858+0000","last_became_peered":"2024-05-28T13:58:23.701858+0000","last_unstale":"2024-05-28T13:58:23.709185+0000","last_undegraded":"2024-05-28T13:58:23.709185+0000","last_fullsized":"2024-05-28T13:58:23.709185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:22.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.670305+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T13:58:23.670305+0000","last_peered":"2024-05-28T13:58:23.670305+0000","last_clean":"2024-05-28T13:58:23.670305+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T13:58:23.670305+0000","last_undegraded":"2024-05-28T13:58:23.670305+0000","last_fullsized":"2024-05-28T13:58:23.670305+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.672682+0000","last_change":"2024-05-28T13:58:22.668236+0000","last_active":"2024-05-28T13:58:23.672682+0000","last_peered":"2024-05-28T13:58:23.672682+0000","last_clean":"2024-05-28T13:58:23.672682+0000","last_became_active":"2024-05-28T13:58:22.667928+0000","last_became_peered":"2024-05-28T13:58:22.667928+0000","last_unstale":"2024-05-28T13:58:23.672682+0000","last_undegraded":"2024-05-28T13:58:23.672682+0000","last_fullsized":"2024-05-28T13:58:23.672682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.702261+0000","last_change":"2024-05-28T13:58:23.701173+0000","last_active":"2024-05-28T13:58:23.702261+0000","last_peered":"2024-05-28T13:58:23.702261+0000","last_clean":"2024-05-28T13:58:23.702261+0000","last_became_active":"2024-05-28T13:58:23.700830+0000","last_became_peered":"2024-05-28T13:58:23.700830+0000","last_unstale":"2024-05-28T13:58:23.702261+0000","last_undegraded":"2024-05-28T13:58:23.702261+0000","last_fullsized":"2024-05-28T13:58:23.702261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:13.470877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.670941+0000","last_change":"2024-05-28T13:58:22.687098+0000","last_active":"2024-05-28T13:58:23.670941+0000","last_peered":"2024-05-28T13:58:23.670941+0000","last_clean":"2024-05-28T13:58:23.670941+0000","last_became_active":"2024-05-28T13:58:22.686751+0000","last_became_peered":"2024-05-28T13:58:22.686751+0000","last_unstale":"2024-05-28T13:58:23.670941+0000","last_undegraded":"2024-05-28T13:58:23.670941+0000","last_fullsized":"2024-05-28T13:58:23.670941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:35.326847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.710891+0000","last_change":"2024-05-28T13:58:23.709225+0000","last_active":"2024-05-28T13:58:23.710891+0000","last_peered":"2024-05-28T13:58:23.710891+0000","last_clean":"2024-05-28T13:58:23.710891+0000","last_became_active":"2024-05-28T13:58:23.708785+0000","last_became_peered":"2024-05-28T13:58:23.708785+0000","last_unstale":"2024-05-28T13:58:23.710891+0000","last_undegraded":"2024-05-28T13:58:23.710891+0000","last_fullsized":"2024-05-28T13:58:23.710891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:13.470877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.678384+0000","last_change":"2024-05-28T13:58:22.667443+0000","last_active":"2024-05-28T13:58:23.678384+0000","last_peered":"2024-05-28T13:58:23.678384+0000","last_clean":"2024-05-28T13:58:23.678384+0000","last_became_active":"2024-05-28T13:58:22.666967+0000","last_became_peered":"2024-05-28T13:58:22.666967+0000","last_unstale":"2024-05-28T13:58:23.678384+0000","last_undegraded":"2024-05-28T13:58:23.678384+0000","last_fullsized":"2024-05-28T13:58:23.678384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.681505+0000","last_change":"2024-05-28T13:58:23.680134+0000","last_active":"2024-05-28T13:58:23.681505+0000","last_peered":"2024-05-28T13:58:23.681505+0000","last_clean":"2024-05-28T13:58:23.681505+0000","last_became_active":"2024-05-28T13:58:23.679912+0000","last_became_peered":"2024-05-28T13:58:23.679912+0000","last_unstale":"2024-05-28T13:58:23.681505+0000","last_undegraded":"2024-05-28T13:58:23.681505+0000","last_fullsized":"2024-05-28T13:58:23.681505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:45.207023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.675838+0000","last_change":"2024-05-28T13:58:22.683368+0000","last_active":"2024-05-28T13:58:23.675838+0000","last_peered":"2024-05-28T13:58:23.675838+0000","last_clean":"2024-05-28T13:58:23.675838+0000","last_became_active":"2024-05-28T13:58:22.683102+0000","last_became_peered":"2024-05-28T13:58:22.683102+0000","last_unstale":"2024-05-28T13:58:23.675838+0000","last_undegraded":"2024-05-28T13:58:23.675838+0000","last_fullsized":"2024-05-28T13:58:23.675838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715466+0000","last_change":"2024-05-28T13:58:23.707964+0000","last_active":"2024-05-28T13:58:23.715466+0000","last_peered":"2024-05-28T13:58:23.715466+0000","last_clean":"2024-05-28T13:58:23.715466+0000","last_became_active":"2024-05-28T13:58:23.707430+0000","last_became_peered":"2024-05-28T13:58:23.707430+0000","last_unstale":"2024-05-28T13:58:23.715466+0000","last_undegraded":"2024-05-28T13:58:23.715466+0000","last_fullsized":"2024-05-28T13:58:23.715466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:35.740742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.662937+0000","last_change":"2024-05-28T13:58:22.676423+0000","last_active":"2024-05-28T13:58:23.662937+0000","last_peered":"2024-05-28T13:58:23.662937+0000","last_clean":"2024-05-28T13:58:23.662937+0000","last_became_active":"2024-05-28T13:58:22.675960+0000","last_became_peered":"2024-05-28T13:58:22.675960+0000","last_unstale":"2024-05-28T13:58:23.662937+0000","last_undegraded":"2024-05-28T13:58:23.662937+0000","last_fullsized":"2024-05-28T13:58:23.662937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:02.116873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.707879+0000","last_change":"2024-05-28T13:58:23.692003+0000","last_active":"2024-05-28T13:58:23.707879+0000","last_peered":"2024-05-28T13:58:23.707879+0000","last_clean":"2024-05-28T13:58:23.707879+0000","last_became_active":"2024-05-28T13:58:23.689087+0000","last_became_peered":"2024-05-28T13:58:23.689087+0000","last_unstale":"2024-05-28T13:58:23.707879+0000","last_undegraded":"2024-05-28T13:58:23.707879+0000","last_fullsized":"2024-05-28T13:58:23.707879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:12:43.447237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.672020+0000","last_change":"2024-05-28T13:58:22.686829+0000","last_active":"2024-05-28T13:58:23.672020+0000","last_peered":"2024-05-28T13:58:23.672020+0000","last_clean":"2024-05-28T13:58:23.672020+0000","last_became_active":"2024-05-28T13:58:22.686497+0000","last_became_peered":"2024-05-28T13:58:22.686497+0000","last_unstale":"2024-05-28T13:58:23.672020+0000","last_undegraded":"2024-05-28T13:58:23.672020+0000","last_fullsized":"2024-05-28T13:58:23.672020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:58.710455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.686334+0000","last_change":"2024-05-28T13:58:23.684658+0000","last_active":"2024-05-28T13:58:23.686334+0000","last_peered":"2024-05-28T13:58:23.686334+0000","last_clean":"2024-05-28T13:58:23.686334+0000","last_became_active":"2024-05-28T13:58:23.682872+0000","last_became_peered":"2024-05-28T13:58:23.682872+0000","last_unstale":"2024-05-28T13:58:23.686334+0000","last_undegraded":"2024-05-28T13:58:23.686334+0000","last_fullsized":"2024-05-28T13:58:23.686334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:12:43.447237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.662441+0000","last_change":"2024-05-28T13:58:22.682664+0000","last_active":"2024-05-28T13:58:23.662441+0000","last_peered":"2024-05-28T13:58:23.662441+0000","last_clean":"2024-05-28T13:58:23.662441+0000","last_became_active":"2024-05-28T13:58:22.682422+0000","last_became_peered":"2024-05-28T13:58:22.682422+0000","last_unstale":"2024-05-28T13:58:23.662441+0000","last_undegraded":"2024-05-28T13:58:23.662441+0000","last_fullsized":"2024-05-28T13:58:23.662441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:07.005097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.682997+0000","last_change":"2024-05-28T13:58:23.678389+0000","last_active":"2024-05-28T13:58:23.682997+0000","last_peered":"2024-05-28T13:58:23.682997+0000","last_clean":"2024-05-28T13:58:23.682997+0000","last_became_active":"2024-05-28T13:58:23.678081+0000","last_became_peered":"2024-05-28T13:58:23.678081+0000","last_unstale":"2024-05-28T13:58:23.682997+0000","last_undegraded":"2024-05-28T13:58:23.682997+0000","last_fullsized":"2024-05-28T13:58:23.682997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:08.011020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.703434+0000","last_change":"2024-05-28T13:58:22.680245+0000","last_active":"2024-05-28T13:58:23.703434+0000","last_peered":"2024-05-28T13:58:23.703434+0000","last_clean":"2024-05-28T13:58:23.703434+0000","last_became_active":"2024-05-28T13:58:22.679323+0000","last_became_peered":"2024-05-28T13:58:22.679323+0000","last_unstale":"2024-05-28T13:58:23.703434+0000","last_undegraded":"2024-05-28T13:58:23.703434+0000","last_fullsized":"2024-05-28T13:58:23.703434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:07.005097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.681944+0000","last_change":"2024-05-28T13:58:23.677850+0000","last_active":"2024-05-28T13:58:23.681944+0000","last_peered":"2024-05-28T13:58:23.681944+0000","last_clean":"2024-05-28T13:58:23.681944+0000","last_became_active":"2024-05-28T13:58:23.677654+0000","last_became_peered":"2024-05-28T13:58:23.677654+0000","last_unstale":"2024-05-28T13:58:23.681944+0000","last_undegraded":"2024-05-28T13:58:23.681944+0000","last_fullsized":"2024-05-28T13:58:23.681944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:39.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.669983+0000","last_change":"2024-05-28T13:58:22.682426+0000","last_active":"2024-05-28T13:58:23.669983+0000","last_peered":"2024-05-28T13:58:23.669983+0000","last_clean":"2024-05-28T13:58:23.669983+0000","last_became_active":"2024-05-28T13:58:22.682099+0000","last_became_peered":"2024-05-28T13:58:22.682099+0000","last_unstale":"2024-05-28T13:58:23.669983+0000","last_undegraded":"2024-05-28T13:58:23.669983+0000","last_fullsized":"2024-05-28T13:58:23.669983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715195+0000","last_change":"2024-05-28T13:58:23.706882+0000","last_active":"2024-05-28T13:58:23.715195+0000","last_peered":"2024-05-28T13:58:23.715195+0000","last_clean":"2024-05-28T13:58:23.715195+0000","last_became_active":"2024-05-28T13:58:23.706588+0000","last_became_peered":"2024-05-28T13:58:23.706588+0000","last_unstale":"2024-05-28T13:58:23.715195+0000","last_undegraded":"2024-05-28T13:58:23.715195+0000","last_fullsized":"2024-05-28T13:58:23.715195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:34.131954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.663046+0000","last_change":"2024-05-28T13:58:22.678682+0000","last_active":"2024-05-28T13:58:23.663046+0000","last_peered":"2024-05-28T13:58:23.663046+0000","last_clean":"2024-05-28T13:58:23.663046+0000","last_became_active":"2024-05-28T13:58:22.678233+0000","last_became_peered":"2024-05-28T13:58:22.678233+0000","last_unstale":"2024-05-28T13:58:23.663046+0000","last_undegraded":"2024-05-28T13:58:23.663046+0000","last_fullsized":"2024-05-28T13:58:23.663046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:35.326847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.678134+0000","last_change":"2024-05-28T13:58:23.669871+0000","last_active":"2024-05-28T13:58:23.678134+0000","last_peered":"2024-05-28T13:58:23.678134+0000","last_clean":"2024-05-28T13:58:23.678134+0000","last_became_active":"2024-05-28T13:58:23.669643+0000","last_became_peered":"2024-05-28T13:58:23.669643+0000","last_unstale":"2024-05-28T13:58:23.678134+0000","last_undegraded":"2024-05-28T13:58:23.678134+0000","last_fullsized":"2024-05-28T13:58:23.678134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:39.727353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.661860+0000","last_change":"2024-05-28T13:58:22.680769+0000","last_active":"2024-05-28T13:58:23.661860+0000","last_peered":"2024-05-28T13:58:23.661860+0000","last_clean":"2024-05-28T13:58:23.661860+0000","last_became_active":"2024-05-28T13:58:22.679781+0000","last_became_peered":"2024-05-28T13:58:22.679781+0000","last_unstale":"2024-05-28T13:58:23.661860+0000","last_undegraded":"2024-05-28T13:58:23.661860+0000","last_fullsized":"2024-05-28T13:58:23.661860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.687680+0000","last_change":"2024-05-28T13:58:23.683534+0000","last_active":"2024-05-28T13:58:23.687680+0000","last_peered":"2024-05-28T13:58:23.687680+0000","last_clean":"2024-05-28T13:58:23.687680+0000","last_became_active":"2024-05-28T13:58:23.683185+0000","last_became_peered":"2024-05-28T13:58:23.683185+0000","last_unstale":"2024-05-28T13:58:23.687680+0000","last_undegraded":"2024-05-28T13:58:23.687680+0000","last_fullsized":"2024-05-28T13:58:23.687680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:30.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.716058+0000","last_change":"2024-05-28T13:58:22.683550+0000","last_active":"2024-05-28T13:58:23.716058+0000","last_peered":"2024-05-28T13:58:23.716058+0000","last_clean":"2024-05-28T13:58:23.716058+0000","last_became_active":"2024-05-28T13:58:22.683218+0000","last_became_peered":"2024-05-28T13:58:22.683218+0000","last_unstale":"2024-05-28T13:58:23.716058+0000","last_undegraded":"2024-05-28T13:58:23.716058+0000","last_fullsized":"2024-05-28T13:58:23.716058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:12.464954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.716911+0000","last_change":"2024-05-28T13:58:23.709888+0000","last_active":"2024-05-28T13:58:23.716911+0000","last_peered":"2024-05-28T13:58:23.716911+0000","last_clean":"2024-05-28T13:58:23.716911+0000","last_became_active":"2024-05-28T13:58:23.709658+0000","last_became_peered":"2024-05-28T13:58:23.709658+0000","last_unstale":"2024-05-28T13:58:23.716911+0000","last_undegraded":"2024-05-28T13:58:23.716911+0000","last_fullsized":"2024-05-28T13:58:23.716911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:07:03.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.714405+0000","last_change":"2024-05-28T13:58:23.710526+0000","last_active":"2024-05-28T13:58:23.714405+0000","last_peered":"2024-05-28T13:58:23.714405+0000","last_clean":"2024-05-28T13:58:23.714405+0000","last_became_active":"2024-05-28T13:58:23.710118+0000","last_became_peered":"2024-05-28T13:58:23.710118+0000","last_unstale":"2024-05-28T13:58:23.714405+0000","last_undegraded":"2024-05-28T13:58:23.714405+0000","last_fullsized":"2024-05-28T13:58:23.714405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:23.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.715806+0000","last_change":"2024-05-28T13:58:22.681209+0000","last_active":"2024-05-28T13:58:23.715806+0000","last_peered":"2024-05-28T13:58:23.715806+0000","last_clean":"2024-05-28T13:58:23.715806+0000","last_became_active":"2024-05-28T13:58:22.680889+0000","last_became_peered":"2024-05-28T13:58:22.680889+0000","last_unstale":"2024-05-28T13:58:23.715806+0000","last_undegraded":"2024-05-28T13:58:23.715806+0000","last_fullsized":"2024-05-28T13:58:23.715806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:02.116873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.714445+0000","last_change":"2024-05-28T13:58:23.710507+0000","last_active":"2024-05-28T13:58:23.714445+0000","last_peered":"2024-05-28T13:58:23.714445+0000","last_clean":"2024-05-28T13:58:23.714445+0000","last_became_active":"2024-05-28T13:58:23.710104+0000","last_became_peered":"2024-05-28T13:58:23.710104+0000","last_unstale":"2024-05-28T13:58:23.714445+0000","last_undegraded":"2024-05-28T13:58:23.714445+0000","last_fullsized":"2024-05-28T13:58:23.714445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:03:59.917115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.705096+0000","last_change":"2024-05-28T13:58:22.688976+0000","last_active":"2024-05-28T13:58:23.705096+0000","last_peered":"2024-05-28T13:58:23.705096+0000","last_clean":"2024-05-28T13:58:23.705096+0000","last_became_active":"2024-05-28T13:58:22.688607+0000","last_became_peered":"2024-05-28T13:58:22.688607+0000","last_unstale":"2024-05-28T13:58:23.705096+0000","last_undegraded":"2024-05-28T13:58:23.705096+0000","last_fullsized":"2024-05-28T13:58:23.705096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:55.898265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.702358+0000","last_change":"2024-05-28T13:58:23.695036+0000","last_active":"2024-05-28T13:58:23.702358+0000","last_peered":"2024-05-28T13:58:23.702358+0000","last_clean":"2024-05-28T13:58:23.702358+0000","last_became_active":"2024-05-28T13:58:23.692922+0000","last_became_peered":"2024-05-28T13:58:23.692922+0000","last_unstale":"2024-05-28T13:58:23.702358+0000","last_undegraded":"2024-05-28T13:58:23.702358+0000","last_fullsized":"2024-05-28T13:58:23.702358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:36.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.711165+0000","last_change":"2024-05-28T13:58:22.684711+0000","last_active":"2024-05-28T13:58:23.711165+0000","last_peered":"2024-05-28T13:58:23.711165+0000","last_clean":"2024-05-28T13:58:23.711165+0000","last_became_active":"2024-05-28T13:58:22.680359+0000","last_became_peered":"2024-05-28T13:58:22.680359+0000","last_unstale":"2024-05-28T13:58:23.711165+0000","last_undegraded":"2024-05-28T13:58:23.711165+0000","last_fullsized":"2024-05-28T13:58:23.711165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.691389+0000","last_change":"2024-05-28T13:58:23.690282+0000","last_active":"2024-05-28T13:58:23.691389+0000","last_peered":"2024-05-28T13:58:23.691389+0000","last_clean":"2024-05-28T13:58:23.691389+0000","last_became_active":"2024-05-28T13:58:23.689975+0000","last_became_peered":"2024-05-28T13:58:23.689975+0000","last_unstale":"2024-05-28T13:58:23.691389+0000","last_undegraded":"2024-05-28T13:58:23.691389+0000","last_fullsized":"2024-05-28T13:58:23.691389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:59.716378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-28T13:58:23.708882+0000","last_change":"2024-05-28T13:58:22.679511+0000","last_active":"2024-05-28T13:58:23.708882+0000","last_peered":"2024-05-28T13:58:23.708882+0000","last_clean":"2024-05-28T13:58:23.708882+0000","last_became_active":"2024-05-28T13:58:22.679207+0000","last_became_peered":"2024-05-28T13:58:22.679207+0000","last_unstale":"2024-05-28T13:58:23.708882+0000","last_undegraded":"2024-05-28T13:58:23.708882+0000","last_fullsized":"2024-05-28T13:58:23.708882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":660255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":69395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":69431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":70917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":204800,"data_stored":69305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":69465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":811008,"data_stored":660311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T13:58:28.180 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-28T13:58:28.181 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-28T13:58:28.181 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-28T13:58:28.181 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-28T13:58:28.181 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-28T13:58:28.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.377+0000 7f3d57400640 1 Processor -- start 2024-05-28T13:58:28.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.377+0000 7f3d57400640 1 -- start start 2024-05-28T13:58:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.377+0000 7f3d57400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3d5015e5c0 0x7f3d500b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.377+0000 7f3d57400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500b51b0 0x7f3d500d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.377+0000 7f3d57400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d500b3720 con 0x7f3d500d0c30 2024-05-28T13:58:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.377+0000 7f3d57400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f3d500b39f0 con 0x7f3d5015e5c0 2024-05-28T13:58:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.377+0000 7f3d57400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d500b3cd0 con 0x7f3d500b51b0 2024-05-28T13:58:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.377+0000 7f3d54a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3d5015e5c0 0x7f3d500b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.377+0000 7f3d55400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f3d500d0c30 0x7f3d50072250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60452/0 (socket says 172.21.15.42:60452) 2024-05-28T13:58:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.377+0000 7f3d55400640 1 -- 172.21.15.42:0/1509624432 learned_addr learned my addr 172.21.15.42:0/1509624432 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.377+0000 7f3d47e00640 1 --2- 172.21.15.42:0/1509624432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500b51b0 0x7f3d500d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d54a00640 1 -- 172.21.15.42:0/1509624432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500b51b0 msgr2=0x7f3d500d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d54a00640 1 --2- 172.21.15.42:0/1509624432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500b51b0 0x7f3d500d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d54a00640 1 -- 172.21.15.42:0/1509624432 >> v1:172.21.15.42:6789/0 conn(0x7f3d500d0c30 legacy=0x7f3d50072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d4fe00640 1 -- 172.21.15.42:0/1509624432 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 17055644 0 0) 0x7f3d500b3720 con 0x7f3d500d0c30 2024-05-28T13:58:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d54a00640 1 -- 172.21.15.42:0/1509624432 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d500b3fb0 con 0x7f3d5015e5c0 2024-05-28T13:58:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d54a00640 1 --2- 172.21.15.42:0/1509624432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d5015e5c0 0x7f3d500b31e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3d400097c0 tx=0x7f3d4001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca63d86018990bd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d4fe00640 1 -- 172.21.15.42:0/1509624432 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d400b1070 con 0x7f3d5015e5c0 2024-05-28T13:58:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d4fe00640 1 -- 172.21.15.42:0/1509624432 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d400ac410 con 0x7f3d5015e5c0 2024-05-28T13:58:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d4fe00640 1 -- 172.21.15.42:0/1509624432 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d400ac7e0 con 0x7f3d5015e5c0 2024-05-28T13:58:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d57400640 1 -- 172.21.15.42:0/1509624432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d5015e5c0 msgr2=0x7f3d500b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d57400640 1 --2- 172.21.15.42:0/1509624432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d5015e5c0 0x7f3d500b31e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3d400097c0 tx=0x7f3d4001c4a0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d57400640 1 -- 172.21.15.42:0/1509624432 shutdown_connections 2024-05-28T13:58:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d57400640 1 --2- 172.21.15.42:0/1509624432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500b51b0 0x7f3d500d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d57400640 1 --2- 172.21.15.42:0/1509624432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d5015e5c0 0x7f3d500b31e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d57400640 1 -- 172.21.15.42:0/1509624432 >> 172.21.15.42:0/1509624432 conn(0x7f3d5006bdf0 msgr2=0x7f3d5006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d57400640 1 -- 172.21.15.42:0/1509624432 shutdown_connections 2024-05-28T13:58:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.381+0000 7f3d57400640 1 -- 172.21.15.42:0/1509624432 wait complete. 2024-05-28T13:58:28.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d57400640 1 Processor -- start 2024-05-28T13:58:28.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d57400640 1 -- start start 2024-05-28T13:58:28.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d57400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500b51b0 0x7f3d5015a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:28.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d57400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d500d0c30 0x7f3d501475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:28.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d57400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d5015e5c0 0x7f3d5009d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:28.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d57400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5016b650 con 0x7f3d500d0c30 2024-05-28T13:58:28.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d57400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5016b4d0 con 0x7f3d5015e5c0 2024-05-28T13:58:28.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d57400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d5016b7d0 con 0x7f3d500b51b0 2024-05-28T13:58:28.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d500d0c30 0x7f3d501475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:28.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d500d0c30 0x7f3d501475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38396/0 (socket says 172.21.15.42:38396) 2024-05-28T13:58:28.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d47e00640 1 -- 172.21.15.42:0/1672462000 learned_addr learned my addr 172.21.15.42:0/1672462000 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:28.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d54a00640 1 --2- 172.21.15.42:0/1672462000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500b51b0 0x7f3d5015a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:28.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d55400640 1 --2- 172.21.15.42:0/1672462000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d5015e5c0 0x7f3d5009d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:28.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d47e00640 1 -- 172.21.15.42:0/1672462000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500b51b0 msgr2=0x7f3d5015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d47e00640 1 --2- 172.21.15.42:0/1672462000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500b51b0 0x7f3d5015a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d47e00640 1 -- 172.21.15.42:0/1672462000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d5015e5c0 msgr2=0x7f3d5009d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d47e00640 1 --2- 172.21.15.42:0/1672462000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d5015e5c0 0x7f3d5009d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d47e00640 1 -- 172.21.15.42:0/1672462000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5009b250 con 0x7f3d500d0c30 2024-05-28T13:58:28.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d47e00640 1 --2- 172.21.15.42:0/1672462000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d500d0c30 0x7f3d501475c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800e930 tx=0x7f3d3800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:28.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d4d600640 1 -- 172.21.15.42:0/1672462000 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38017070 con 0x7f3d500d0c30 2024-05-28T13:58:28.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d4d600640 1 -- 172.21.15.42:0/1672462000 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d3800ca00 con 0x7f3d500d0c30 2024-05-28T13:58:28.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d4d600640 1 -- 172.21.15.42:0/1672462000 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3800cdb0 con 0x7f3d500d0c30 2024-05-28T13:58:28.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d57400640 1 -- 172.21.15.42:0/1672462000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5009b530 con 0x7f3d500d0c30 2024-05-28T13:58:28.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.385+0000 7f3d57400640 1 -- 172.21.15.42:0/1672462000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d5009baf0 con 0x7f3d500d0c30 2024-05-28T13:58:28.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.389+0000 7f3d57400640 1 -- 172.21.15.42:0/1672462000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d50068a90 con 0x7f3d500d0c30 2024-05-28T13:58:28.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.389+0000 7f3d4d600640 1 -- 172.21.15.42:0/1672462000 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3d3801a070 con 0x7f3d500d0c30 2024-05-28T13:58:28.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.389+0000 7f3d4d600640 1 --2- 172.21.15.42:0/1672462000 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d2c0683b0 0x7f3d2c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:28.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.389+0000 7f3d4d600640 1 -- 172.21.15.42:0/1672462000 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3d38086d30 con 0x7f3d500d0c30 2024-05-28T13:58:28.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.389+0000 7f3d54a00640 1 --2- 172.21.15.42:0/1672462000 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d2c0683b0 0x7f3d2c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:28.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.389+0000 7f3d54a00640 1 --2- 172.21.15.42:0/1672462000 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d2c0683b0 0x7f3d2c06a870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3d400ae000 tx=0x7f3d400a6040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:28.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.393+0000 7f3d4d600640 1 -- 172.21.15.42:0/1672462000 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3d38056cc0 con 0x7f3d500d0c30 2024-05-28T13:58:28.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.601+0000 7f3d57400640 1 -- 172.21.15.42:0/1672462000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f3d5015b060 con 0x7f3d500d0c30 2024-05-28T13:58:28.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.605+0000 7f3d4d600640 1 -- 172.21.15.42:0/1672462000 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+168083 (secure 0 0 0) 0x7f3d3805afc0 con 0x7f3d500d0c30 2024-05-28T13:58:28.592 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.609+0000 7f3d57400640 1 -- 172.21.15.42:0/1672462000 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d2c0683b0 msgr2=0x7f3d2c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.609+0000 7f3d57400640 1 --2- 172.21.15.42:0/1672462000 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d2c0683b0 0x7f3d2c06a870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3d400ae000 tx=0x7f3d400a6040 comp rx=0 tx=0).stop 2024-05-28T13:58:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.609+0000 7f3d57400640 1 -- 172.21.15.42:0/1672462000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d500d0c30 msgr2=0x7f3d501475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.609+0000 7f3d57400640 1 --2- 172.21.15.42:0/1672462000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d500d0c30 0x7f3d501475c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800e930 tx=0x7f3d3800edf0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.609+0000 7f3d57400640 1 -- 172.21.15.42:0/1672462000 shutdown_connections 2024-05-28T13:58:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.609+0000 7f3d57400640 1 --2- 172.21.15.42:0/1672462000 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d2c0683b0 0x7f3d2c06a870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.609+0000 7f3d57400640 1 --2- 172.21.15.42:0/1672462000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d5015e5c0 0x7f3d5009d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.609+0000 7f3d57400640 1 --2- 172.21.15.42:0/1672462000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d500d0c30 0x7f3d501475c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.609+0000 7f3d57400640 1 --2- 172.21.15.42:0/1672462000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500b51b0 0x7f3d5015a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.609+0000 7f3d57400640 1 -- 172.21.15.42:0/1672462000 >> 172.21.15.42:0/1672462000 conn(0x7f3d5006bdf0 msgr2=0x7f3d500b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.609+0000 7f3d57400640 1 -- 172.21.15.42:0/1672462000 shutdown_connections 2024-05-28T13:58:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.609+0000 7f3d57400640 1 -- 172.21.15.42:0/1672462000 wait complete. 2024-05-28T13:58:28.613 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"flags":0,"active_gid":4098,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1976130233},{"type":"v1","addr":"172.21.15.42:6833","nonce":1976130233}]},"active_addr":"172.21.15.42:6833/1976130233","active_change":"2024-05-28T13:58:02.843722+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4120,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":4092647486}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":2927675820}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":981882910}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":724263915}]}]} 2024-05-28T13:58:28.616 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-28T13:58:28.616 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-28T13:58:28.617 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:28.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.773+0000 7f0d01e00640 1 Processor -- start 2024-05-28T13:58:28.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.773+0000 7f0d01e00640 1 -- start start 2024-05-28T13:58:28.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.773+0000 7f0d01e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0cfc156e50 0x7f0cfc06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:28.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.773+0000 7f0d01e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cfc15cb90 con 0x7f0cfc156a70 2024-05-28T13:58:28.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.773+0000 7f0d01e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cfc15ce50 con 0x7f0cfc0afa50 2024-05-28T13:58:28.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.773+0000 7f0d01e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0cfc15d130 con 0x7f0cfc156e50 2024-05-28T13:58:28.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.773+0000 7f0cfb400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0cfc156e50 0x7f0cfc06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:28.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.773+0000 7f0cfaa00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0cfc156a70 0x7f0cfc0af340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60468/0 (socket says 172.21.15.42:60468) 2024-05-28T13:58:28.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.773+0000 7f0cfaa00640 1 -- 172.21.15.42:0/2429150135 learned_addr learned my addr 172.21.15.42:0/2429150135 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:28.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.773+0000 7f0cfa000640 1 -- 172.21.15.42:0/2429150135 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 598713573 0 0) 0x7f0cfc15cb90 con 0x7f0cfc156a70 2024-05-28T13:58:28.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.773+0000 7f0cfa000640 1 -- 172.21.15.42:0/2429150135 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ce0003530 con 0x7f0cfc156a70 2024-05-28T13:58:28.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.773+0000 7f0cfb400640 1 -- 172.21.15.42:0/2429150135 >> v1:172.21.15.63:6789/0 conn(0x7f0cfc0afa50 legacy=0x7f0cfc0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.773+0000 7f0cfb400640 1 -- 172.21.15.42:0/2429150135 >> v1:172.21.15.42:6789/0 conn(0x7f0cfc156a70 legacy=0x7f0cfc0af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.773+0000 7f0cfb400640 1 -- 172.21.15.42:0/2429150135 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc15d410 con 0x7f0cfc156e50 2024-05-28T13:58:28.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.777+0000 7f0cfb400640 1 --2- 172.21.15.42:0/2429150135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cfc156e50 0x7f0cfc06dd60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009d90 tx=0x7f0cf401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ac7f3b7efc06cf server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:28.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.777+0000 7f0cfa000640 1 -- 172.21.15.42:0/2429150135 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf40b2070 con 0x7f0cfc156e50 2024-05-28T13:58:28.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.777+0000 7f0cfa000640 1 -- 172.21.15.42:0/2429150135 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf40a5a50 con 0x7f0cfc156e50 2024-05-28T13:58:28.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.777+0000 7f0cfa000640 1 -- 172.21.15.42:0/2429150135 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf40a5e20 con 0x7f0cfc156e50 2024-05-28T13:58:28.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.777+0000 7f0d01e00640 1 -- 172.21.15.42:0/2429150135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cfc156e50 msgr2=0x7f0cfc06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.777+0000 7f0d01e00640 1 --2- 172.21.15.42:0/2429150135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cfc156e50 0x7f0cfc06dd60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009d90 tx=0x7f0cf401c970 comp rx=0 tx=0).stop 2024-05-28T13:58:28.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.777+0000 7f0d01e00640 1 -- 172.21.15.42:0/2429150135 shutdown_connections 2024-05-28T13:58:28.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.777+0000 7f0d01e00640 1 --2- 172.21.15.42:0/2429150135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cfc156e50 0x7f0cfc06dd60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.777+0000 7f0d01e00640 1 -- 172.21.15.42:0/2429150135 >> 172.21.15.42:0/2429150135 conn(0x7f0cfc06bdf0 msgr2=0x7f0cfc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:28.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.777+0000 7f0d01e00640 1 -- 172.21.15.42:0/2429150135 shutdown_connections 2024-05-28T13:58:28.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.777+0000 7f0d01e00640 1 -- 172.21.15.42:0/2429150135 wait complete. 2024-05-28T13:58:28.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0d01e00640 1 Processor -- start 2024-05-28T13:58:28.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0d01e00640 1 -- start start 2024-05-28T13:58:28.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0d01e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cfc0afa50 0x7f0cfc15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:28.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0d01e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cfc156a70 0x7f0cfc155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:28.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0d01e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cfc156e50 0x7f0cfc14c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:28.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0d01e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc160f10 con 0x7f0cfc156a70 2024-05-28T13:58:28.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0d01e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc160d90 con 0x7f0cfc156e50 2024-05-28T13:58:28.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0d01e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0cfc161090 con 0x7f0cfc0afa50 2024-05-28T13:58:28.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0cf2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cfc156a70 0x7f0cfc155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:28.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0cf2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cfc156a70 0x7f0cfc155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38410/0 (socket says 172.21.15.42:38410) 2024-05-28T13:58:28.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0cf2000640 1 -- 172.21.15.42:0/1160217596 learned_addr learned my addr 172.21.15.42:0/1160217596 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:28.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0cfaa00640 1 --2- 172.21.15.42:0/1160217596 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cfc0afa50 0x7f0cfc15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:28.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0cfb400640 1 --2- 172.21.15.42:0/1160217596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cfc156e50 0x7f0cfc14c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:28.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0cf2000640 1 -- 172.21.15.42:0/1160217596 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cfc0afa50 msgr2=0x7f0cfc15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0cf2000640 1 --2- 172.21.15.42:0/1160217596 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cfc0afa50 0x7f0cfc15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0cf2000640 1 -- 172.21.15.42:0/1160217596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cfc156e50 msgr2=0x7f0cfc14c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0cf2000640 1 --2- 172.21.15.42:0/1160217596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cfc156e50 0x7f0cfc14c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0cf2000640 1 -- 172.21.15.42:0/1160217596 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc073050 con 0x7f0cfc156a70 2024-05-28T13:58:28.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0cf2000640 1 --2- 172.21.15.42:0/1160217596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cfc156a70 0x7f0cfc155390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400c3a0 tx=0x7f0ce400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:28.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0cf3e00640 1 -- 172.21.15.42:0/1160217596 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40052d0 con 0x7f0cfc156a70 2024-05-28T13:58:28.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0cf3e00640 1 -- 172.21.15.42:0/1160217596 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce4005470 con 0x7f0cfc156a70 2024-05-28T13:58:28.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0cf3e00640 1 -- 172.21.15.42:0/1160217596 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4007260 con 0x7f0cfc156a70 2024-05-28T13:58:28.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0d01e00640 1 -- 172.21.15.42:0/1160217596 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cfc152640 con 0x7f0cfc156a70 2024-05-28T13:58:28.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.781+0000 7f0d01e00640 1 -- 172.21.15.42:0/1160217596 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cfc1507b0 con 0x7f0cfc156a70 2024-05-28T13:58:28.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.785+0000 7f0d01e00640 1 -- 172.21.15.42:0/1160217596 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cfc068a90 con 0x7f0cfc156a70 2024-05-28T13:58:28.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.785+0000 7f0cf3e00640 1 -- 172.21.15.42:0/1160217596 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0ce4007400 con 0x7f0cfc156a70 2024-05-28T13:58:28.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.785+0000 7f0cf3e00640 1 --2- 172.21.15.42:0/1160217596 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0cd8068380 0x7f0cd806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:28.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.785+0000 7f0cf3e00640 1 -- 172.21.15.42:0/1160217596 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0ce4085e80 con 0x7f0cfc156a70 2024-05-28T13:58:28.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.785+0000 7f0cfaa00640 1 --2- 172.21.15.42:0/1160217596 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0cd8068380 0x7f0cd806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:28.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.785+0000 7f0cfaa00640 1 --2- 172.21.15.42:0/1160217596 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0cd8068380 0x7f0cd806a840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0011f0 tx=0x7f0cec007d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:28.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.789+0000 7f0cf3e00640 1 -- 172.21.15.42:0/1160217596 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ce4055d50 con 0x7f0cfc156a70 2024-05-28T13:58:28.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.953+0000 7f0d01e00640 1 -- 172.21.15.42:0/1160217596 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cfc153d30 con 0x7f0cfc156a70 2024-05-28T13:58:28.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.957+0000 7f0cf3e00640 1 -- 172.21.15.42:0/1160217596 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16545 (secure 0 0 0) 0x7f0ce405a050 con 0x7f0cfc156a70 2024-05-28T13:58:28.942 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:28.942 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:24.659233+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.957+0000 7f0d01e00640 1 -- 172.21.15.42:0/1160217596 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0cd8068380 msgr2=0x7f0cd806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.957+0000 7f0d01e00640 1 --2- 172.21.15.42:0/1160217596 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0cd8068380 0x7f0cd806a840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0011f0 tx=0x7f0cec007d40 comp rx=0 tx=0).stop 2024-05-28T13:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.957+0000 7f0d01e00640 1 -- 172.21.15.42:0/1160217596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cfc156a70 msgr2=0x7f0cfc155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.957+0000 7f0d01e00640 1 --2- 172.21.15.42:0/1160217596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cfc156a70 0x7f0cfc155390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400c3a0 tx=0x7f0ce400c860 comp rx=0 tx=0).stop 2024-05-28T13:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.957+0000 7f0d01e00640 1 -- 172.21.15.42:0/1160217596 shutdown_connections 2024-05-28T13:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.957+0000 7f0d01e00640 1 --2- 172.21.15.42:0/1160217596 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0cd8068380 0x7f0cd806a840 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.957+0000 7f0d01e00640 1 --2- 172.21.15.42:0/1160217596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cfc156e50 0x7f0cfc14c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.957+0000 7f0d01e00640 1 --2- 172.21.15.42:0/1160217596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cfc156a70 0x7f0cfc155390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.957+0000 7f0d01e00640 1 --2- 172.21.15.42:0/1160217596 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cfc0afa50 0x7f0cfc15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.957+0000 7f0d01e00640 1 -- 172.21.15.42:0/1160217596 >> 172.21.15.42:0/1160217596 conn(0x7f0cfc06bdf0 msgr2=0x7f0cfc167e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.957+0000 7f0d01e00640 1 -- 172.21.15.42:0/1160217596 shutdown_connections 2024-05-28T13:58:28.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:28.957+0000 7f0d01e00640 1 -- 172.21.15.42:0/1160217596 wait complete. 2024-05-28T13:58:28.963 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-28T13:58:28.963 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:29.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.117+0000 7f7687a00640 1 Processor -- start 2024-05-28T13:58:29.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.117+0000 7f7687a00640 1 -- start start 2024-05-28T13:58:29.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f7687a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f768015ba40 0x7f76800d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f7687a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7680072dd0 0x7f76800d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f7687a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f768015f280 con 0x7f768006f660 2024-05-28T13:58:29.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f7687a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f768015f030 con 0x7f768015ba40 2024-05-28T13:58:29.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f7687a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f768015f560 con 0x7f7680072dd0 2024-05-28T13:58:29.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f767fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f768015ba40 0x7f76800d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f767fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f768015ba40 0x7f76800d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38424/0 (socket says 172.21.15.42:38424) 2024-05-28T13:58:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f767fe00640 1 -- 172.21.15.42:0/439071877 learned_addr learned my addr 172.21.15.42:0/439071877 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f7685a00640 1 --2- 172.21.15.42:0/439071877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7680072dd0 0x7f76800d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f767fe00640 1 -- 172.21.15.42:0/439071877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7680072dd0 msgr2=0x7f76800d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f767fe00640 1 --2- 172.21.15.42:0/439071877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7680072dd0 0x7f76800d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f767fe00640 1 -- 172.21.15.42:0/439071877 >> v1:172.21.15.63:6789/0 conn(0x7f768006f660 legacy=0x7f7680073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f767fe00640 1 -- 172.21.15.42:0/439071877 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768015f840 con 0x7f768015ba40 2024-05-28T13:58:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f767fe00640 1 --2- 172.21.15.42:0/439071877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f768015ba40 0x7f76800d06f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f766c009870 tx=0x7f766c01c0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8c43517aff813a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f767f400640 1 -- 172.21.15.42:0/439071877 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c0b1070 con 0x7f768015ba40 2024-05-28T13:58:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f767f400640 1 -- 172.21.15.42:0/439071877 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766c0a48b0 con 0x7f768015ba40 2024-05-28T13:58:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.121+0000 7f767f400640 1 -- 172.21.15.42:0/439071877 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c0a4c80 con 0x7f768015ba40 2024-05-28T13:58:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 -- 172.21.15.42:0/439071877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f768015ba40 msgr2=0x7f76800d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 --2- 172.21.15.42:0/439071877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f768015ba40 0x7f76800d06f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f766c009870 tx=0x7f766c01c0b0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 -- 172.21.15.42:0/439071877 shutdown_connections 2024-05-28T13:58:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 --2- 172.21.15.42:0/439071877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7680072dd0 0x7f76800d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 --2- 172.21.15.42:0/439071877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f768015ba40 0x7f76800d06f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 -- 172.21.15.42:0/439071877 >> 172.21.15.42:0/439071877 conn(0x7f768006bdf0 msgr2=0x7f768006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 -- 172.21.15.42:0/439071877 shutdown_connections 2024-05-28T13:58:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 -- 172.21.15.42:0/439071877 wait complete. 2024-05-28T13:58:29.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 Processor -- start 2024-05-28T13:58:29.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 -- start start 2024-05-28T13:58:29.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768006f660 0x7f768015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7680072dd0 0x7f768014db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768014e340 0x7f76801555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f76800b3ec0 con 0x7f7680072dd0 2024-05-28T13:58:29.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76800b3d40 con 0x7f768006f660 2024-05-28T13:58:29.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7687a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76800b4040 con 0x7f768014e340 2024-05-28T13:58:29.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f767fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7680072dd0 0x7f768014db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f767fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7680072dd0 0x7f768014db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38426/0 (socket says 172.21.15.42:38426) 2024-05-28T13:58:29.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f767fe00640 1 -- 172.21.15.42:0/2859927836 learned_addr learned my addr 172.21.15.42:0/2859927836 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7685000640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768006f660 0x7f768015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7685a00640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768014e340 0x7f76801555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f767fe00640 1 -- 172.21.15.42:0/2859927836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768014e340 msgr2=0x7f76801555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f767fe00640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768014e340 0x7f76801555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f767fe00640 1 -- 172.21.15.42:0/2859927836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768006f660 msgr2=0x7f768015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f767fe00640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768006f660 0x7f768015aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f767fe00640 1 -- 172.21.15.42:0/2859927836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76800b10d0 con 0x7f7680072dd0 2024-05-28T13:58:29.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.125+0000 7f7685000640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768006f660 0x7f768015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:29.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.129+0000 7f767fe00640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7680072dd0 0x7f768014db70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f766c0af040 tx=0x7f766c019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.129+0000 7f767cc00640 1 -- 172.21.15.42:0/2859927836 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c0bc070 con 0x7f7680072dd0 2024-05-28T13:58:29.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.129+0000 7f767cc00640 1 -- 172.21.15.42:0/2859927836 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766c0acab0 con 0x7f7680072dd0 2024-05-28T13:58:29.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.129+0000 7f7687a00640 1 -- 172.21.15.42:0/2859927836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76801547f0 con 0x7f7680072dd0 2024-05-28T13:58:29.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.129+0000 7f767cc00640 1 -- 172.21.15.42:0/2859927836 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c0b1040 con 0x7f7680072dd0 2024-05-28T13:58:29.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.129+0000 7f7687a00640 1 -- 172.21.15.42:0/2859927836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7680154db0 con 0x7f7680072dd0 2024-05-28T13:58:29.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.129+0000 7f7687a00640 1 -- 172.21.15.42:0/2859927836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7680068a90 con 0x7f7680072dd0 2024-05-28T13:58:29.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.129+0000 7f767cc00640 1 -- 172.21.15.42:0/2859927836 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f766c0bf070 con 0x7f7680072dd0 2024-05-28T13:58:29.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.129+0000 7f767cc00640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7660068380 0x7f766006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.129+0000 7f767cc00640 1 -- 172.21.15.42:0/2859927836 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f766c11fc80 con 0x7f7680072dd0 2024-05-28T13:58:29.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.129+0000 7f7685000640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7660068380 0x7f766006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.133+0000 7f7685000640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7660068380 0x7f766006a840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f76740098b0 tx=0x7f7674008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.133+0000 7f767cc00640 1 -- 172.21.15.42:0/2859927836 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f766c016110 con 0x7f7680072dd0 2024-05-28T13:58:29.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.301+0000 7f7687a00640 1 -- 172.21.15.42:0/2859927836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7680072bb0 con 0x7f7680072dd0 2024-05-28T13:58:29.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.301+0000 7f767cc00640 1 -- 172.21.15.42:0/2859927836 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16545 (secure 0 0 0) 0x7f766c0f0110 con 0x7f7680072dd0 2024-05-28T13:58:29.286 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:29.286 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:24.659233+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.301+0000 7f7687a00640 1 -- 172.21.15.42:0/2859927836 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7660068380 msgr2=0x7f766006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.301+0000 7f7687a00640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7660068380 0x7f766006a840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f76740098b0 tx=0x7f7674008040 comp rx=0 tx=0).stop 2024-05-28T13:58:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.301+0000 7f7687a00640 1 -- 172.21.15.42:0/2859927836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7680072dd0 msgr2=0x7f768014db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.301+0000 7f7687a00640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7680072dd0 0x7f768014db70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f766c0af040 tx=0x7f766c019170 comp rx=0 tx=0).stop 2024-05-28T13:58:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.301+0000 7f7687a00640 1 -- 172.21.15.42:0/2859927836 shutdown_connections 2024-05-28T13:58:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.301+0000 7f7687a00640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7660068380 0x7f766006a840 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.301+0000 7f7687a00640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768014e340 0x7f76801555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.301+0000 7f7687a00640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7680072dd0 0x7f768014db70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.301+0000 7f7687a00640 1 --2- 172.21.15.42:0/2859927836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768006f660 0x7f768015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.301+0000 7f7687a00640 1 -- 172.21.15.42:0/2859927836 >> 172.21.15.42:0/2859927836 conn(0x7f768006bdf0 msgr2=0x7f76800adee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.301+0000 7f7687a00640 1 -- 172.21.15.42:0/2859927836 shutdown_connections 2024-05-28T13:58:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.301+0000 7f7687a00640 1 -- 172.21.15.42:0/2859927836 wait complete. 2024-05-28T13:58:29.307 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-28T13:58:29.308 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-28T13:58:29.308 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-28T13:58:29.309 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-28T13:58:29.309 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-28T13:58:29.310 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-28T13:58:29.310 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-28T13:58:29.310 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-28T13:58:29.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.569+0000 7ff16ec00640 1 Processor -- start 2024-05-28T13:58:29.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.569+0000 7ff16ec00640 1 -- start start 2024-05-28T13:58:29.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff16ec00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff168162ca0 0x7ff16806f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff16ec00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff16806f6c0 0x7ff168071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff16ec00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff16806dd50 con 0x7ff168072500 2024-05-28T13:58:29.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff16ec00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff16806fb20 con 0x7ff168162ca0 2024-05-28T13:58:29.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff16ec00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff168072990 con 0x7ff16806f6c0 2024-05-28T13:58:29.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff16cc00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7ff168072500 0x7ff1680b34f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:50552/0 (socket says 172.21.15.42:50552) 2024-05-28T13:58:29.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff16cc00640 1 -- 172.21.15.42:0/2496555075 learned_addr learned my addr 172.21.15.42:0/2496555075 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:29.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff167e00640 1 --2- 172.21.15.42:0/2496555075 >> v2:172.21.15.63:3300/0 conn(0x7ff168162ca0 0x7ff16806f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff166a00640 1 -- 172.21.15.42:0/2496555075 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3633928216 0 0) 0x7ff16806dd50 con 0x7ff168072500 2024-05-28T13:58:29.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff166a00640 1 -- 172.21.15.42:0/2496555075 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff14c003530 con 0x7ff168072500 2024-05-28T13:58:29.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff167e00640 1 -- 172.21.15.42:0/2496555075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff16806f6c0 msgr2=0x7ff168071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff167400640 1 --2- 172.21.15.42:0/2496555075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff16806f6c0 0x7ff168071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff167e00640 1 --2- 172.21.15.42:0/2496555075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff16806f6c0 0x7ff168071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff167e00640 1 -- 172.21.15.42:0/2496555075 >> v1:172.21.15.63:6789/0 conn(0x7ff168072500 legacy=0x7ff1680b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff167e00640 1 -- 172.21.15.42:0/2496555075 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16806e030 con 0x7ff168162ca0 2024-05-28T13:58:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7ff167400640 1 --2- 172.21.15.42:0/2496555075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff16806f6c0 0x7ff168071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7f83b4600640 1 Processor -- start 2024-05-28T13:58:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.573+0000 7f83b4600640 1 -- start start 2024-05-28T13:58:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f83b4600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f83ac145920 0x7f83ac06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f83b4600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f83ac0724d0 0x7f83ac15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f83b4600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83ac06e2b0 con 0x7f83ac06db40 2024-05-28T13:58:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f83b4600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f83ac06e590 con 0x7f83ac145920 2024-05-28T13:58:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f83b4600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f83ac06e870 con 0x7f83ac0724d0 2024-05-28T13:58:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f83b1200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f83ac06db40 0x7f83ac071dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60482/0 (socket says 172.21.15.42:60482) 2024-05-28T13:58:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f83b1200640 1 -- 172.21.15.42:0/485849550 learned_addr learned my addr 172.21.15.42:0/485849550 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f83b2600640 1 --2- 172.21.15.42:0/485849550 >> v2:172.21.15.63:3300/0 conn(0x7f83ac0724d0 0x7f83ac15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f83b1c00640 1 --2- 172.21.15.42:0/485849550 >> v2:172.21.15.42:3300/0 conn(0x7f83ac145920 0x7f83ac06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f16c5400640 1 Processor -- start 2024-05-28T13:58:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f16c5400640 1 -- start start 2024-05-28T13:58:29.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f83b1c00640 1 -- 172.21.15.42:0/485849550 >> v2:172.21.15.63:3300/0 conn(0x7f83ac0724d0 msgr2=0x7f83ac15d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f83b1c00640 1 --2- 172.21.15.42:0/485849550 >> v2:172.21.15.63:3300/0 conn(0x7f83ac0724d0 0x7f83ac15d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f83b1c00640 1 -- 172.21.15.42:0/485849550 >> v1:172.21.15.42:6789/0 conn(0x7f83ac06db40 legacy=0x7f83ac071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7fe619800640 1 Processor -- start 2024-05-28T13:58:29.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7ff167e00640 1 --2- 172.21.15.42:0/2496555075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff168162ca0 0x7ff16806f180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff160009d90 tx=0x7ff16001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b7540862fea7c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7fe619800640 1 -- start start 2024-05-28T13:58:29.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7ff166a00640 1 -- 172.21.15.42:0/2496555075 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1600b2070 con 0x7ff168162ca0 2024-05-28T13:58:29.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7ff166a00640 1 -- 172.21.15.42:0/2496555075 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1600a57a0 con 0x7ff168162ca0 2024-05-28T13:58:29.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7ff166a00640 1 -- 172.21.15.42:0/2496555075 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1600a5b50 con 0x7ff168162ca0 2024-05-28T13:58:29.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16c5400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f16c0158ff0 0x7f16c0070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7ff16ec00640 1 -- 172.21.15.42:0/2496555075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff168162ca0 msgr2=0x7ff16806f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7ff16ec00640 1 --2- 172.21.15.42:0/2496555075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff168162ca0 0x7ff16806f180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff160009d90 tx=0x7ff16001c8e0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16c5400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16c00b4b70 0x7f16c00b5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16c5400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16c0070c50 con 0x7f16c015bad0 2024-05-28T13:58:29.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16c5400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f16c006f670 con 0x7f16c0158ff0 2024-05-28T13:58:29.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16c5400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f16c006f950 con 0x7f16c00b4b70 2024-05-28T13:58:29.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16bfe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f16c0158ff0 0x7f16c0070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16c00b4b70 0x7f16c00b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16c00b4b70 0x7f16c00b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42710/0 (socket says 172.21.15.42:42710) 2024-05-28T13:58:29.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16c4a00640 1 -- 172.21.15.42:0/270743870 learned_addr learned my addr 172.21.15.42:0/270743870 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16bf400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f16c015bad0 0x7f16c006ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:50568/0 (socket says 172.21.15.42:50568) 2024-05-28T13:58:29.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7ff16ec00640 1 -- 172.21.15.42:0/2496555075 shutdown_connections 2024-05-28T13:58:29.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7ff16ec00640 1 --2- 172.21.15.42:0/2496555075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff16806f6c0 0x7ff168071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7ff16ec00640 1 --2- 172.21.15.42:0/2496555075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff168162ca0 0x7ff16806f180 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7ff16ec00640 1 -- 172.21.15.42:0/2496555075 >> 172.21.15.42:0/2496555075 conn(0x7ff168067b60 msgr2=0x7ff16806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.577+0000 7f83b1c00640 1 -- 172.21.15.42:0/485849550 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ac06eb50 con 0x7f83ac145920 2024-05-28T13:58:29.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7ff16ec00640 1 -- 172.21.15.42:0/2496555075 shutdown_connections 2024-05-28T13:58:29.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe619800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe614163d30 0x7fe614072210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16bea00640 1 -- 172.21.15.42:0/270743870 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2054824150 0 0) 0x7f16c0070c50 con 0x7f16c015bad0 2024-05-28T13:58:29.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16bea00640 1 -- 172.21.15.42:0/270743870 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16a4003530 con 0x7f16c015bad0 2024-05-28T13:58:29.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16c4a00640 1 -- 172.21.15.42:0/270743870 >> v2:172.21.15.63:3300/0 conn(0x7f16c0158ff0 msgr2=0x7f16c0070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16c4a00640 1 --2- 172.21.15.42:0/270743870 >> v2:172.21.15.63:3300/0 conn(0x7f16c0158ff0 0x7f16c0070710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fc80d200640 1 Processor -- start 2024-05-28T13:58:29.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe619800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6140b4c00 con 0x7fe614072750 2024-05-28T13:58:29.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe619800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6140b3af0 con 0x7fe61415d030 2024-05-28T13:58:29.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe619800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe614072ba0 con 0x7fe614163d30 2024-05-28T13:58:29.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16c4a00640 1 -- 172.21.15.42:0/270743870 >> v1:172.21.15.63:6789/0 conn(0x7f16c015bad0 legacy=0x7f16c006ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f16c4a00640 1 -- 172.21.15.42:0/270743870 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c006fc30 con 0x7f16c00b4b70 2024-05-28T13:58:29.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7ff16ec00640 1 -- 172.21.15.42:0/2496555075 wait complete. 2024-05-28T13:58:29.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fc80d200640 1 -- start start 2024-05-28T13:58:29.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe612000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe614163d30 0x7fe614072210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe612a00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fe61415d030 0x7fe61406f920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:50584/0 (socket says 172.21.15.42:50584) 2024-05-28T13:58:29.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe612a00640 1 -- 172.21.15.42:0/3817733003 learned_addr learned my addr 172.21.15.42:0/3817733003 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:29.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe611600640 1 -- 172.21.15.42:0/3817733003 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 755139935 0 0) 0x7fe6140b3af0 con 0x7fe61415d030 2024-05-28T13:58:29.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe611600640 1 -- 172.21.15.42:0/3817733003 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5f8003530 con 0x7fe61415d030 2024-05-28T13:58:29.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe611600640 1 -- 172.21.15.42:0/3817733003 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 69705387 0 0) 0x7fe5f8003530 con 0x7fe61415d030 2024-05-28T13:58:29.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe611600640 1 -- 172.21.15.42:0/3817733003 >> v2:172.21.15.63:3300/0 conn(0x7fe614163d30 msgr2=0x7fe614072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe611600640 1 --2- 172.21.15.42:0/3817733003 >> v2:172.21.15.63:3300/0 conn(0x7fe614163d30 0x7fe614072210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe611600640 1 -- 172.21.15.42:0/3817733003 >> v1:172.21.15.42:6789/0 conn(0x7fe614072750 legacy=0x7fe61406e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe611600640 1 -- 172.21.15.42:0/3817733003 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6140b3dd0 con 0x7fe61415d030 2024-05-28T13:58:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7f83b1c00640 1 --2- 172.21.15.42:0/485849550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83ac145920 0x7f83ac06f0e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83ac06c4d0 tx=0x7f83a001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90004b9946c7c620 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7f83abe00640 1 -- 172.21.15.42:0/485849550 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00b0070 con 0x7f83ac145920 2024-05-28T13:58:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7f83abe00640 1 -- 172.21.15.42:0/485849550 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a00ab400 con 0x7f83ac145920 2024-05-28T13:58:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7f83b4600640 1 -- 172.21.15.42:0/485849550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83ac145920 msgr2=0x7f83ac06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7f83b4600640 1 --2- 172.21.15.42:0/485849550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83ac145920 0x7f83ac06f0e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83ac06c4d0 tx=0x7f83a001c3f0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16ec00640 1 Processor -- start 2024-05-28T13:58:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16ec00640 1 -- start start 2024-05-28T13:58:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16ec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff16806f6c0 0x7ff16815da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16ec00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff168072500 0x7ff168151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16ec00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff168162ca0 0x7ff1681527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16ec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff16816f900 con 0x7ff16806f6c0 2024-05-28T13:58:29.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16ec00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff16816f780 con 0x7ff168162ca0 2024-05-28T13:58:29.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16ec00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff16816fa80 con 0x7ff168072500 2024-05-28T13:58:29.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc80d200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc808162c80 0x7fc8080ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc80d200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc80806f180 0x7fc80806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc805e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc808162c80 0x7fc8080ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc805e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc808162c80 0x7fc8080ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42714/0 (socket says 172.21.15.42:42714) 2024-05-28T13:58:29.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc805e00640 1 -- 172.21.15.42:0/2242957631 learned_addr learned my addr 172.21.15.42:0/2242957631 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc80d200640 1 --2- 172.21.15.42:0/2242957631 >> v2:172.21.15.63:3300/0 conn(0x7fc808071fe0 0x7fc808072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc80d200640 1 -- 172.21.15.42:0/2242957631 --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc808166730 con 0x7fc80806f180 2024-05-28T13:58:29.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc80d200640 1 -- 172.21.15.42:0/2242957631 --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc80806fb20 con 0x7fc808071fe0 2024-05-28T13:58:29.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc80d200640 1 -- 172.21.15.42:0/2242957631 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc808167570 con 0x7fc808162c80 2024-05-28T13:58:29.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc806800640 1 --2- 172.21.15.42:0/2242957631 >> v2:172.21.15.63:3300/0 conn(0x7fc808071fe0 0x7fc808072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc805e00640 1 -- 172.21.15.42:0/2242957631 >> v2:172.21.15.63:3300/0 conn(0x7fc808071fe0 msgr2=0x7fc808072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc805e00640 1 --2- 172.21.15.42:0/2242957631 >> v2:172.21.15.63:3300/0 conn(0x7fc808071fe0 0x7fc808072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc805e00640 1 -- 172.21.15.42:0/2242957631 >> v2:172.21.15.42:3300/0 conn(0x7fc80806f180 msgr2=0x7fc80806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc805e00640 1 --2- 172.21.15.42:0/2242957631 >> v2:172.21.15.42:3300/0 conn(0x7fc80806f180 0x7fc80806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fc805e00640 1 -- 172.21.15.42:0/2242957631 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc808166960 con 0x7fc808162c80 2024-05-28T13:58:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.581+0000 7fe611600640 1 -- 172.21.15.42:0/3817733003 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fe600001680 con 0x7fe61415d030 2024-05-28T13:58:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fe611600640 1 -- 172.21.15.42:0/3817733003 >> v1:172.21.15.63:6789/0 conn(0x7fe61415d030 legacy=0x7fe61406f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fe611600640 1 --2- 172.21.15.42:0/3817733003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5f80039c0 0x7fe5f8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fe611600640 1 --2- 172.21.15.42:0/3817733003 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f8004810 0x7fe5f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fe611600640 1 --2- 172.21.15.42:0/3817733003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5f80051d0 0x7fe5f800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fe611600640 1 -- 172.21.15.42:0/3817733003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe614072ba0 con 0x7fe5f80039c0 2024-05-28T13:58:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fe611600640 1 -- 172.21.15.42:0/3817733003 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f800c100 con 0x7fe5f80051d0 2024-05-28T13:58:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fe611600640 1 -- 172.21.15.42:0/3817733003 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f800c300 con 0x7fe5f8004810 2024-05-28T13:58:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fe612a00640 1 --2- 172.21.15.42:0/3817733003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5f80039c0 0x7fe5f8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fe612a00640 1 -- 172.21.15.42:0/3817733003 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f8004810 msgr2=0x7fe5f8004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:58:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fe612a00640 1 --2- 172.21.15.42:0/3817733003 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f8004810 0x7fe5f8004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fe612a00640 1 -- 172.21.15.42:0/3817733003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5f80051d0 msgr2=0x7fe5f800b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:58:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fe612a00640 1 --2- 172.21.15.42:0/3817733003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5f80051d0 0x7fe5f800b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fe612a00640 1 -- 172.21.15.42:0/3817733003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6140b3dd0 con 0x7fe5f80039c0 2024-05-28T13:58:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff168162ca0 0x7ff1681527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff168162ca0 0x7ff1681527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:37886/0 (socket says 172.21.15.42:37886) 2024-05-28T13:58:29.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16cc00640 1 -- 172.21.15.42:0/525158182 learned_addr learned my addr 172.21.15.42:0/525158182 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff167400640 1 --2- 172.21.15.42:0/525158182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff168072500 0x7ff168151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16cc00640 1 -- 172.21.15.42:0/525158182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff168072500 msgr2=0x7ff168151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16cc00640 1 --2- 172.21.15.42:0/525158182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff168072500 0x7ff168151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16cc00640 1 -- 172.21.15.42:0/525158182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff16806f6c0 msgr2=0x7ff16815da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16cc00640 1 --2- 172.21.15.42:0/525158182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff16806f6c0 0x7ff16815da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7ff16cc00640 1 -- 172.21.15.42:0/525158182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16815be20 con 0x7ff168162ca0 2024-05-28T13:58:29.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7fe613400640 1 -- 172.21.15.42:0/3817733003 reap_dead start 2024-05-28T13:58:29.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.585+0000 7f16c4a00640 1 --2- 172.21.15.42:0/270743870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16c00b4b70 0x7f16c00b5000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009e50 tx=0x7f16b801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=644857d501dd11ae server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7ff16cc00640 1 --2- 172.21.15.42:0/525158182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff168162ca0 0x7ff1681527d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff15400c400 tx=0x7ff15400c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f83b4600640 1 -- 172.21.15.42:0/485849550 shutdown_connections 2024-05-28T13:58:29.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f83b4600640 1 --2- 172.21.15.42:0/485849550 >> v2:172.21.15.63:3300/0 conn(0x7f83ac0724d0 0x7f83ac15d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f83b4600640 1 --2- 172.21.15.42:0/485849550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83ac145920 0x7f83ac06f0e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f83b4600640 1 -- 172.21.15.42:0/485849550 >> 172.21.15.42:0/485849550 conn(0x7f83ac067b60 msgr2=0x7f83ac06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f83b4600640 1 -- 172.21.15.42:0/485849550 shutdown_connections 2024-05-28T13:58:29.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f16bea00640 1 -- 172.21.15.42:0/270743870 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b80a5700 con 0x7f16c00b4b70 2024-05-28T13:58:29.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f16bea00640 1 -- 172.21.15.42:0/270743870 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b80a58a0 con 0x7f16c00b4b70 2024-05-28T13:58:29.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f16bea00640 1 -- 172.21.15.42:0/270743870 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b80a5c70 con 0x7f16c00b4b70 2024-05-28T13:58:29.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f83b4600640 1 -- 172.21.15.42:0/485849550 wait complete. 2024-05-28T13:58:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f6c6a400640 1 Processor -- start 2024-05-28T13:58:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f16c5400640 1 -- 172.21.15.42:0/270743870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16c00b4b70 msgr2=0x7f16c00b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f16c5400640 1 --2- 172.21.15.42:0/270743870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16c00b4b70 0x7f16c00b5000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009e50 tx=0x7f16b801ca30 comp rx=0 tx=0).stop 2024-05-28T13:58:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f6c6a400640 1 -- start start 2024-05-28T13:58:29.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7ff15fe00640 1 -- 172.21.15.42:0/525158182 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154017070 con 0x7ff168162ca0 2024-05-28T13:58:29.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f5953600640 1 Processor -- start 2024-05-28T13:58:29.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f5953600640 1 -- start start 2024-05-28T13:58:29.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f16c5400640 1 -- 172.21.15.42:0/270743870 shutdown_connections 2024-05-28T13:58:29.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f16c5400640 1 --2- 172.21.15.42:0/270743870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16c00b4b70 0x7f16c00b5000 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f16c5400640 1 --2- 172.21.15.42:0/270743870 >> v2:172.21.15.63:3300/0 conn(0x7f16c0158ff0 0x7f16c0070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f16c5400640 1 -- 172.21.15.42:0/270743870 >> 172.21.15.42:0/270743870 conn(0x7f16c00b6780 msgr2=0x7f16c00b6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7fe612a00640 1 --2- 172.21.15.42:0/3817733003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5f80039c0 0x7fe5f8003dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe60000c260 tx=0x7fe60001cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474d9ec954b4a16 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f16c5400640 1 -- 172.21.15.42:0/270743870 shutdown_connections 2024-05-28T13:58:29.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f16c5400640 1 -- 172.21.15.42:0/270743870 wait complete. 2024-05-28T13:58:29.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7fc805e00640 1 --2- 172.21.15.42:0/2242957631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc808162c80 0x7fc8080ce0c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc80806ae50 tx=0x7fc7f801ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a0a98f3935685e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7f6c6a400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6c6415dc30 0x7f6c640ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7ff15fe00640 1 -- 172.21.15.42:0/525158182 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff154005cf0 con 0x7ff168162ca0 2024-05-28T13:58:29.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7fc805400640 1 -- 172.21.15.42:0/2242957631 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80a5670 con 0x7fc808162c80 2024-05-28T13:58:29.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7fc805400640 1 -- 172.21.15.42:0/2242957631 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f80a5810 con 0x7fc808162c80 2024-05-28T13:58:29.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7fc805400640 1 -- 172.21.15.42:0/2242957631 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80a5be0 con 0x7fc808162c80 2024-05-28T13:58:29.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7ff15fe00640 1 -- 172.21.15.42:0/525158182 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154017070 con 0x7ff168162ca0 2024-05-28T13:58:29.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7ff16ec00640 1 -- 172.21.15.42:0/525158182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16815c100 con 0x7ff168162ca0 2024-05-28T13:58:29.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.589+0000 7ff16ec00640 1 -- 172.21.15.42:0/525158182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16815c6c0 con 0x7ff168162ca0 2024-05-28T13:58:29.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f83b4600640 1 Processor -- start 2024-05-28T13:58:29.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fe611600640 1 -- 172.21.15.42:0/3817733003 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600007d80 con 0x7fe5f80039c0 2024-05-28T13:58:29.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fe611600640 1 -- 172.21.15.42:0/3817733003 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6000a9070 con 0x7fe5f80039c0 2024-05-28T13:58:29.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fe611600640 1 -- 172.21.15.42:0/3817733003 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600007d80 con 0x7fe5f80039c0 2024-05-28T13:58:29.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f83b4600640 1 -- start start 2024-05-28T13:58:29.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f16c5400640 1 Processor -- start 2024-05-28T13:58:29.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f5953600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f594c0cd810 0x7f594c15b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f5953600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f594c15bf20 0x7f594c15c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f5953600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f594c0d0230 con 0x7f594c144d40 2024-05-28T13:58:29.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f5953600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f594c0b3d60 con 0x7f594c0cd810 2024-05-28T13:58:29.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f5953600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f594c0cf230 con 0x7f594c15bf20 2024-05-28T13:58:29.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f5950c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f594c144d40 0x7f594c0cd100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60500/0 (socket says 172.21.15.42:60500) 2024-05-28T13:58:29.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f5950c00640 1 -- 172.21.15.42:0/4139444264 learned_addr learned my addr 172.21.15.42:0/4139444264 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:29.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f594be00640 1 --2- 172.21.15.42:0/4139444264 >> v2:172.21.15.63:3300/0 conn(0x7f594c0cd810 0x7f594c15b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fddc1800640 1 Processor -- start 2024-05-28T13:58:29.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7ff15fe00640 1 -- 172.21.15.42:0/525158182 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff15401a070 con 0x7ff168162ca0 2024-05-28T13:58:29.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c69000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6c6415dc30 0x7f6c640ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c6a400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c6406ef00 0x7f6c6406f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c6a400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c6406de60 con 0x7f6c640b2a00 2024-05-28T13:58:29.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c6a400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6c6406e140 con 0x7f6c6415dc30 2024-05-28T13:58:29.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f6c640b2a00 0x7f6c64073a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:50586/0 (socket says 172.21.15.42:50586) 2024-05-28T13:58:29.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63e00640 1 -- 172.21.15.42:0/2202430146 learned_addr learned my addr 172.21.15.42:0/2202430146 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:29.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c69a00640 1 --2- 172.21.15.42:0/2202430146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c6406ef00 0x7f6c6406f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 -- 172.21.15.42:0/2202430146 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3137004546 0 0) 0x7f6c6406de60 con 0x7f6c640b2a00 2024-05-28T13:58:29.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c6a400640 1 -- 172.21.15.42:0/2202430146 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c6406e420 con 0x7f6c6406ef00 2024-05-28T13:58:29.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f5951600640 1 --2- 172.21.15.42:0/4139444264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f594c15bf20 0x7f594c15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 -- 172.21.15.42:0/2202430146 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c48003530 con 0x7f6c640b2a00 2024-05-28T13:58:29.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fe619800640 1 -- 172.21.15.42:0/3817733003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5f80039c0 msgr2=0x7fe5f8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fe619800640 1 --2- 172.21.15.42:0/3817733003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5f80039c0 0x7fe5f8003dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe60000c260 tx=0x7fe60001cd20 comp rx=0 tx=0).stop 2024-05-28T13:58:29.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fc80d200640 1 -- 172.21.15.42:0/2242957631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc808162c80 msgr2=0x7fc8080ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fc80d200640 1 --2- 172.21.15.42:0/2242957631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc808162c80 0x7fc8080ce0c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc80806ae50 tx=0x7fc7f801ca10 comp rx=0 tx=0).stop 2024-05-28T13:58:29.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f594be00640 1 -- 172.21.15.42:0/4139444264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f594c15bf20 msgr2=0x7f594c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f594be00640 1 --2- 172.21.15.42:0/4139444264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f594c15bf20 0x7f594c15c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 -- 172.21.15.42:0/2202430146 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2643663051 0 0) 0x7f6c48003530 con 0x7f6c640b2a00 2024-05-28T13:58:29.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 -- 172.21.15.42:0/2202430146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c6406ef00 msgr2=0x7f6c6406f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 --2- 172.21.15.42:0/2202430146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c6406ef00 0x7f6c6406f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 -- 172.21.15.42:0/2202430146 >> v2:172.21.15.42:3300/0 conn(0x7f6c6415dc30 msgr2=0x7f6c640ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fc80d200640 1 -- 172.21.15.42:0/2242957631 shutdown_connections 2024-05-28T13:58:29.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fc80d200640 1 --2- 172.21.15.42:0/2242957631 >> v2:172.21.15.63:3300/0 conn(0x7fc808071fe0 0x7fc808072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fddc1800640 1 -- start start 2024-05-28T13:58:29.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fc80d200640 1 --2- 172.21.15.42:0/2242957631 >> v2:172.21.15.42:3300/0 conn(0x7fc80806f180 0x7fc80806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fc80d200640 1 --2- 172.21.15.42:0/2242957631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc808162c80 0x7fc8080ce0c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fc80d200640 1 -- 172.21.15.42:0/2242957631 >> 172.21.15.42:0/2242957631 conn(0x7fc80806b9f0 msgr2=0x7fc80806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f16c5400640 1 -- start start 2024-05-28T13:58:29.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7ff16ec00640 1 -- 172.21.15.42:0/525158182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff168073890 con 0x7ff168162ca0 2024-05-28T13:58:29.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fe619800640 1 -- 172.21.15.42:0/3817733003 shutdown_connections 2024-05-28T13:58:29.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fe619800640 1 --2- 172.21.15.42:0/3817733003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5f80039c0 0x7fe5f8003dc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fe619800640 1 -- 172.21.15.42:0/3817733003 >> 172.21.15.42:0/3817733003 conn(0x7fe6140b25e0 msgr2=0x7fe6140b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 --2- 172.21.15.42:0/2202430146 >> v2:172.21.15.42:3300/0 conn(0x7f6c6415dc30 0x7f6c640ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 -- 172.21.15.42:0/2202430146 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6406e700 con 0x7f6c640b2a00 2024-05-28T13:58:29.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f594be00640 1 -- 172.21.15.42:0/4139444264 >> v1:172.21.15.42:6789/0 conn(0x7f594c144d40 legacy=0x7f594c0cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f594be00640 1 -- 172.21.15.42:0/4139444264 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594c0cf490 con 0x7f594c0cd810 2024-05-28T13:58:29.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fe619800640 1 -- 172.21.15.42:0/3817733003 shutdown_connections 2024-05-28T13:58:29.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7ff15fe00640 1 --2- 172.21.15.42:0/525158182 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff148068580 0x7ff14806aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7ff15fe00640 1 -- 172.21.15.42:0/525158182 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff154085860 con 0x7ff168162ca0 2024-05-28T13:58:29.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fe619800640 1 -- 172.21.15.42:0/3817733003 wait complete. 2024-05-28T13:58:29.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 -- 172.21.15.42:0/2202430146 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f6c54001680 con 0x7f6c640b2a00 2024-05-28T13:58:29.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 -- 172.21.15.42:0/2202430146 >> v1:172.21.15.63:6789/0 conn(0x7f6c640b2a00 legacy=0x7f6c64073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 --2- 172.21.15.42:0/2202430146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c480039c0 0x7f6c48003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f83b4600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83ac06db40 0x7f83ac142a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 --2- 172.21.15.42:0/2202430146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c6406ef00 0x7f6c48004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fc80d200640 1 -- 172.21.15.42:0/2242957631 shutdown_connections 2024-05-28T13:58:29.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7ff15fe00640 1 --2- 172.21.15.42:0/525158182 >> [v2:172.21.15.42:6800/437002159,v1:172.21.15.42:6801/437002159] conn(0x7ff148070720 0x7ff148072b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7ff15fe00640 1 -- 172.21.15.42:0/525158182 --> [v2:172.21.15.42:6800/437002159,v1:172.21.15.42:6801/437002159] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1480731f0 con 0x7ff148070720 2024-05-28T13:58:29.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7ff15fe00640 1 -- 172.21.15.42:0/525158182 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff154051a70 con 0x7ff168162ca0 2024-05-28T13:58:29.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7ff167e00640 1 --2- 172.21.15.42:0/525158182 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff148068580 0x7ff14806aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f83b4600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83ac0724d0 0x7f83ac164d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f83b4600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83ac145920 0x7f83ac12fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f83b4600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac15e2a0 con 0x7f83ac145920 2024-05-28T13:58:29.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f83b4600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac15e120 con 0x7f83ac0724d0 2024-05-28T13:58:29.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f83b4600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f83ac15e420 con 0x7f83ac06db40 2024-05-28T13:58:29.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f83b1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83ac06db40 0x7f83ac142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f83b1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83ac06db40 0x7f83ac142a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42760/0 (socket says 172.21.15.42:42760) 2024-05-28T13:58:29.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f83b1c00640 1 -- 172.21.15.42:0/2598473864 learned_addr learned my addr 172.21.15.42:0/2598473864 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7ff167400640 1 --2- 172.21.15.42:0/525158182 >> [v2:172.21.15.42:6800/437002159,v1:172.21.15.42:6801/437002159] conn(0x7ff148070720 0x7ff148072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 --2- 172.21.15.42:0/2202430146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c48004d50 0x7f6c480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7ff167400640 1 --2- 172.21.15.42:0/525158182 >> [v2:172.21.15.42:6800/437002159,v1:172.21.15.42:6801/437002159] conn(0x7ff148070720 0x7ff148072b40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7fc80d200640 1 -- 172.21.15.42:0/2242957631 wait complete. 2024-05-28T13:58:29.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 -- 172.21.15.42:0/2202430146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6406e140 con 0x7f6c480039c0 2024-05-28T13:58:29.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 -- 172.21.15.42:0/2202430146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6406e420 con 0x7f6c48004d50 2024-05-28T13:58:29.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.593+0000 7f6c63400640 1 -- 172.21.15.42:0/2202430146 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c4800bb50 con 0x7f6c6406ef00 2024-05-28T13:58:29.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f6c63e00640 1 --2- 172.21.15.42:0/2202430146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c6406ef00 0x7f6c48004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f6c69a00640 1 --2- 172.21.15.42:0/2202430146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c48004d50 0x7f6c480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f6c69000640 1 --2- 172.21.15.42:0/2202430146 >> v2:172.21.15.42:3300/0 conn(0x7f6c6415dc30 0x7f6c640ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:29.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7ff167e00640 1 --2- 172.21.15.42:0/525158182 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff148068580 0x7ff14806aa40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff1600af000 tx=0x7ff160004ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f6c69a00640 1 -- 172.21.15.42:0/2202430146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c6406ef00 msgr2=0x7f6c48004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f6c69a00640 1 --2- 172.21.15.42:0/2202430146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c6406ef00 0x7f6c48004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f6c69a00640 1 -- 172.21.15.42:0/2202430146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c480039c0 msgr2=0x7f6c48003de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:58:29.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f6c69a00640 1 --2- 172.21.15.42:0/2202430146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c480039c0 0x7f6c48003de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f6c69a00640 1 -- 172.21.15.42:0/2202430146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c48003530 con 0x7f6c48004d50 2024-05-28T13:58:29.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fddc1800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fddbc15e8a0 0x7fddbc0d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fddc1800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddbc0d5390 0x7fddbc0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fddc1800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddbc0d3540 con 0x7fddbc15ec80 2024-05-28T13:58:29.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fddc1800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fddbc0d0810 con 0x7fddbc15e8a0 2024-05-28T13:58:29.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f83b2600640 1 --2- 172.21.15.42:0/2598473864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83ac145920 0x7f83ac12fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f83b1c00640 1 -- 172.21.15.42:0/2598473864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83ac0724d0 msgr2=0x7f83ac164d90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T13:58:29.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f83b1c00640 1 --2- 172.21.15.42:0/2598473864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83ac0724d0 0x7f83ac164d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f83b1c00640 1 -- 172.21.15.42:0/2598473864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83ac145920 msgr2=0x7f83ac12fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fddc1800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fddbc0cf620 con 0x7fddbc0d5390 2024-05-28T13:58:29.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fe619800640 1 Processor -- start 2024-05-28T13:58:29.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f83b1c00640 1 --2- 172.21.15.42:0/2598473864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83ac145920 0x7f83ac12fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f83b1c00640 1 -- 172.21.15.42:0/2598473864 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ac165630 con 0x7f83ac06db40 2024-05-28T13:58:29.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f83b2600640 1 --2- 172.21.15.42:0/2598473864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83ac145920 0x7f83ac12fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:29.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f83b1c00640 1 --2- 172.21.15.42:0/2598473864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83ac06db40 0x7f83ac142a70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f83a001a020 tx=0x7f83a001ac30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fddbaa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fddbc15e8a0 0x7fddbc0d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fddbaa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fddbc15e8a0 0x7fddbc0d2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38498/0 (socket says 172.21.15.42:38498) 2024-05-28T13:58:29.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fddbaa00640 1 -- 172.21.15.42:0/2575272265 learned_addr learned my addr 172.21.15.42:0/2575272265 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f83a9600640 1 -- 172.21.15.42:0/2598473864 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00c0410 con 0x7f83ac06db40 2024-05-28T13:58:29.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f83a9600640 1 -- 172.21.15.42:0/2598473864 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a00c05b0 con 0x7f83ac06db40 2024-05-28T13:58:29.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f83a9600640 1 -- 172.21.15.42:0/2598473864 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00c0960 con 0x7f83ac06db40 2024-05-28T13:58:29.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7ff15fe00640 1 -- 172.21.15.42:0/525158182 <== osd.1 v2:172.21.15.42:6800/437002159 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1480731f0 con 0x7ff148070720 2024-05-28T13:58:29.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fe619800640 1 -- start start 2024-05-28T13:58:29.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c5400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16c00b4b70 0x7f16c014e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16c0158ff0 0x7f16c014b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16c015bad0 0x7f16c009b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c5400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f16c015d7a0 con 0x7f16c0158ff0 2024-05-28T13:58:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c5400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f16c015d620 con 0x7f16c015bad0 2024-05-28T13:58:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c5400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f16c006b9f0 con 0x7f16c00b4b70 2024-05-28T13:58:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16bfe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16c00b4b70 0x7f16c014e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16c015bad0 0x7f16c009b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16c015bad0 0x7f16c009b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:37922/0 (socket says 172.21.15.42:37922) 2024-05-28T13:58:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c4a00640 1 -- 172.21.15.42:0/286500610 learned_addr learned my addr 172.21.15.42:0/286500610 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c4a00640 1 -- 172.21.15.42:0/286500610 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16c00b4b70 msgr2=0x7f16c014e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c4a00640 1 --2- 172.21.15.42:0/286500610 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16c00b4b70 0x7f16c014e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c4a00640 1 -- 172.21.15.42:0/286500610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16c0158ff0 msgr2=0x7f16c014b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c4a00640 1 --2- 172.21.15.42:0/286500610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16c0158ff0 0x7f16c014b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c4a00640 1 -- 172.21.15.42:0/286500610 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c009b890 con 0x7f16c015bad0 2024-05-28T13:58:29.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fddbb400640 1 --2- 172.21.15.42:0/2575272265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddbc0d5390 0x7fddbc0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fc80d200640 1 Processor -- start 2024-05-28T13:58:29.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7f16c4a00640 1 --2- 172.21.15.42:0/286500610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16c015bad0 0x7f16c009b350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f16b80af040 tx=0x7f16b80197d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fc80d200640 1 -- start start 2024-05-28T13:58:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe619800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f80039c0 0x7fe6141fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe619800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe614165e00 0x7fe61415bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe619800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6140ce540 0x7fe61415c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe619800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe614072390 con 0x7fe614165e00 2024-05-28T13:58:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe619800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe614072210 con 0x7fe6140ce540 2024-05-28T13:58:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe619800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe614072510 con 0x7fe5f80039c0 2024-05-28T13:58:29.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe612000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe614165e00 0x7fe61415bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe612000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe614165e00 0x7fe61415bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38514/0 (socket says 172.21.15.42:38514) 2024-05-28T13:58:29.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe612000640 1 -- 172.21.15.42:0/1496241203 learned_addr learned my addr 172.21.15.42:0/1496241203 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe612a00640 1 --2- 172.21.15.42:0/1496241203 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f80039c0 0x7fe6141fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7f83b4600640 1 -- 172.21.15.42:0/2598473864 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ac165910 con 0x7f83ac06db40 2024-05-28T13:58:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe612000640 1 -- 172.21.15.42:0/1496241203 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f80039c0 msgr2=0x7fe6141fb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe612000640 1 --2- 172.21.15.42:0/1496241203 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f80039c0 0x7fe6141fb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe612000640 1 -- 172.21.15.42:0/1496241203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6140ce540 msgr2=0x7fe61415c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe612000640 1 --2- 172.21.15.42:0/1496241203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6140ce540 0x7fe61415c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe612000640 1 -- 172.21.15.42:0/1496241203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61415b040 con 0x7fe614165e00 2024-05-28T13:58:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe612a00640 1 --2- 172.21.15.42:0/1496241203 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f80039c0 0x7fe6141fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fc80d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc80806f180 0x7fc808150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fc80d200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc808071fe0 0x7fc808154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fc80d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc808162c80 0x7fc808163710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fc80d200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc80806e680 con 0x7fc80806f180 2024-05-28T13:58:29.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fc80d200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc80806e500 con 0x7fc808162c80 2024-05-28T13:58:29.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fc80d200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc80806e800 con 0x7fc808071fe0 2024-05-28T13:58:29.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fddbb400640 1 -- 172.21.15.42:0/2575272265 >> v2:172.21.15.42:3300/0 conn(0x7fddbc15e8a0 msgr2=0x7fddbc0d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fddbb400640 1 --2- 172.21.15.42:0/2575272265 >> v2:172.21.15.42:3300/0 conn(0x7fddbc15e8a0 0x7fddbc0d2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fddbb400640 1 -- 172.21.15.42:0/2575272265 >> v1:172.21.15.42:6789/0 conn(0x7fddbc15ec80 legacy=0x7fddbc0d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.597+0000 7fddbb400640 1 -- 172.21.15.42:0/2575272265 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc0cf900 con 0x7fddbc0d5390 2024-05-28T13:58:29.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe612000640 1 --2- 172.21.15.42:0/1496241203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe614165e00 0x7fe61415bae0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe60400d490 tx=0x7fe60400d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7f16b7e00640 1 -- 172.21.15.42:0/286500610 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b80cd020 con 0x7f16c015bad0 2024-05-28T13:58:29.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7f16b7e00640 1 -- 172.21.15.42:0/286500610 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b80c9680 con 0x7f16c015bad0 2024-05-28T13:58:29.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7f16b7e00640 1 -- 172.21.15.42:0/286500610 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b80a5700 con 0x7f16c015bad0 2024-05-28T13:58:29.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe60aa00640 1 -- 172.21.15.42:0/1496241203 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe604004020 con 0x7fe614165e00 2024-05-28T13:58:29.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fc805e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc80806f180 0x7fc808150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fc7fd400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc808071fe0 0x7fc808154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fc7fd400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc808071fe0 0x7fc808154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42804/0 (socket says 172.21.15.42:42804) 2024-05-28T13:58:29.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fc7fd400640 1 -- 172.21.15.42:0/1901618735 learned_addr learned my addr 172.21.15.42:0/1901618735 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7f83b4600640 1 -- 172.21.15.42:0/2598473864 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ac12dc50 con 0x7f83ac06db40 2024-05-28T13:58:29.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7f83b4600640 1 -- 172.21.15.42:0/2598473864 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83ac073890 con 0x7f83ac06db40 2024-05-28T13:58:29.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe60aa00640 1 -- 172.21.15.42:0/1496241203 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6040041e0 con 0x7fe614165e00 2024-05-28T13:58:29.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe60aa00640 1 -- 172.21.15.42:0/1496241203 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6040043d0 con 0x7fe614165e00 2024-05-28T13:58:29.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7f16c5400640 1 -- 172.21.15.42:0/286500610 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c014a6b0 con 0x7f16c015bad0 2024-05-28T13:58:29.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7f16c5400640 1 -- 172.21.15.42:0/286500610 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c014ac70 con 0x7f16c015bad0 2024-05-28T13:58:29.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe619800640 1 -- 172.21.15.42:0/1496241203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6141591c0 con 0x7fe614165e00 2024-05-28T13:58:29.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7fe619800640 1 -- 172.21.15.42:0/1496241203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61415b2e0 con 0x7fe614165e00 2024-05-28T13:58:29.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.601+0000 7f6c69a00640 1 --2- 172.21.15.42:0/2202430146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c48004d50 0x7f6c480051d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c014990 tx=0x7f6c5c0257d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4520d9a0cadd5766 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f16c5400640 1 -- 172.21.15.42:0/286500610 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f16c009bb30 con 0x7f16c015bad0 2024-05-28T13:58:29.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fc7fd400640 1 -- 172.21.15.42:0/1901618735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc808162c80 msgr2=0x7fc808163710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T13:58:29.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fc7fd400640 1 --2- 172.21.15.42:0/1901618735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc808162c80 0x7fc808163710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fc7fd400640 1 -- 172.21.15.42:0/1901618735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc80806f180 msgr2=0x7fc808150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fc7fd400640 1 --2- 172.21.15.42:0/1901618735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc80806f180 0x7fc808150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fc7fd400640 1 -- 172.21.15.42:0/1901618735 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80815c7e0 con 0x7fc808071fe0 2024-05-28T13:58:29.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f594be00640 1 --2- 172.21.15.42:0/4139444264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f594c0cd810 0x7f594c15b9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f59380097f0 tx=0x7f593801c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=860d1216700b130d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fc7fd400640 1 --2- 172.21.15.42:0/1901618735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc808071fe0 0x7fc808154140 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000b550 tx=0x7fc7f000ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fe619800640 1 -- 172.21.15.42:0/1496241203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe614073c50 con 0x7fe614165e00 2024-05-28T13:58:29.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f83a9600640 1 -- 172.21.15.42:0/2598473864 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f83a00be070 con 0x7f83ac06db40 2024-05-28T13:58:29.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f594b400640 1 -- 172.21.15.42:0/4139444264 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59380b0070 con 0x7f594c0cd810 2024-05-28T13:58:29.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f594b400640 1 -- 172.21.15.42:0/4139444264 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59380ab600 con 0x7f594c0cd810 2024-05-28T13:58:29.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f594b400640 1 -- 172.21.15.42:0/4139444264 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59380ab9d0 con 0x7f594c0cd810 2024-05-28T13:58:29.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fc7fea00640 1 -- 172.21.15.42:0/1901618735 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0014070 con 0x7fc808071fe0 2024-05-28T13:58:29.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fc7fea00640 1 -- 172.21.15.42:0/1901618735 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f0010890 con 0x7fc808071fe0 2024-05-28T13:58:29.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fc7fea00640 1 -- 172.21.15.42:0/1901618735 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0010c80 con 0x7fc808071fe0 2024-05-28T13:58:29.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f6c63400640 1 -- 172.21.15.42:0/2202430146 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c00fd90 con 0x7f6c48004d50 2024-05-28T13:58:29.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f6c63400640 1 -- 172.21.15.42:0/2202430146 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c5c0af7d0 con 0x7f6c48004d50 2024-05-28T13:58:29.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f83a9600640 1 --2- 172.21.15.42:0/2598473864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f838c068380 0x7f838c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fc80d200640 1 -- 172.21.15.42:0/1901618735 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc808152250 con 0x7fc808071fe0 2024-05-28T13:58:29.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fc80d200640 1 -- 172.21.15.42:0/1901618735 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc808151490 con 0x7fc808071fe0 2024-05-28T13:58:29.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f6c63400640 1 -- 172.21.15.42:0/2202430146 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0afbe0 con 0x7f6c48004d50 2024-05-28T13:58:29.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f6c6a400640 1 -- 172.21.15.42:0/2202430146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c48004d50 msgr2=0x7f6c480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f6c6a400640 1 --2- 172.21.15.42:0/2202430146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c48004d50 0x7f6c480051d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c014990 tx=0x7f6c5c0257d0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f83a9600640 1 -- 172.21.15.42:0/2598473864 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f83a0128710 con 0x7f83ac06db40 2024-05-28T13:58:29.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f83a9600640 1 --2- 172.21.15.42:0/2598473864 >> [v2:172.21.15.42:6808/2018240931,v1:172.21.15.42:6809/2018240931] conn(0x7f838c0705a0 0x7f838c0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fddbb400640 1 --2- 172.21.15.42:0/2575272265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddbc0d5390 0x7fddbc0d1120 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fddb4009d90 tx=0x7fddb401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90f98b641a78d728 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f83a9600640 1 -- 172.21.15.42:0/2598473864 --> [v2:172.21.15.42:6808/2018240931,v1:172.21.15.42:6809/2018240931] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83a00c5e50 con 0x7f838c0705a0 2024-05-28T13:58:29.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f83a9600640 1 -- 172.21.15.42:0/2598473864 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f83a012f050 con 0x7f83ac06db40 2024-05-28T13:58:29.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f83b1200640 1 --2- 172.21.15.42:0/2598473864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f838c068380 0x7f838c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fddb9600640 1 -- 172.21.15.42:0/2575272265 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb40b2070 con 0x7fddbc0d5390 2024-05-28T13:58:29.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fddb9600640 1 -- 172.21.15.42:0/2575272265 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb40a5890 con 0x7fddbc0d5390 2024-05-28T13:58:29.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fddb9600640 1 -- 172.21.15.42:0/2575272265 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb40a5c60 con 0x7fddbc0d5390 2024-05-28T13:58:29.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fc80d200640 1 -- 172.21.15.42:0/1901618735 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc808150d90 con 0x7fc808071fe0 2024-05-28T13:58:29.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f83b2600640 1 --2- 172.21.15.42:0/2598473864 >> [v2:172.21.15.42:6808/2018240931,v1:172.21.15.42:6809/2018240931] conn(0x7f838c0705a0 0x7f838c0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f5953600640 1 -- 172.21.15.42:0/4139444264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f594c0cd810 msgr2=0x7f594c15b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f5953600640 1 --2- 172.21.15.42:0/4139444264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f594c0cd810 0x7f594c15b9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f59380097f0 tx=0x7f593801c380 comp rx=0 tx=0).stop 2024-05-28T13:58:29.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f16b7e00640 1 -- 172.21.15.42:0/286500610 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f16b80d0020 con 0x7f16c015bad0 2024-05-28T13:58:29.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f16b7e00640 1 --2- 172.21.15.42:0/286500610 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16a0068450 0x7f16a006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f16b7e00640 1 -- 172.21.15.42:0/286500610 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f16b8132c50 con 0x7f16c015bad0 2024-05-28T13:58:29.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f16b7e00640 1 --2- 172.21.15.42:0/286500610 >> [v2:172.21.15.42:6824/2252246131,v1:172.21.15.42:6825/2252246131] conn(0x7f16a0070670 0x7f16a0072a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f16b7e00640 1 -- 172.21.15.42:0/286500610 --> [v2:172.21.15.42:6824/2252246131,v1:172.21.15.42:6825/2252246131] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16a0073140 con 0x7f16a0070670 2024-05-28T13:58:29.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f16b7e00640 1 -- 172.21.15.42:0/286500610 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f16b8132fe0 con 0x7f16c015bad0 2024-05-28T13:58:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7f6c69a00640 1 -- 172.21.15.42:0/2202430146 reap_dead start 2024-05-28T13:58:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f5953600640 1 -- 172.21.15.42:0/4139444264 shutdown_connections 2024-05-28T13:58:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f5953600640 1 --2- 172.21.15.42:0/4139444264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f594c15bf20 0x7f594c15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f5953600640 1 --2- 172.21.15.42:0/4139444264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f594c0cd810 0x7f594c15b9e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f83b2600640 1 --2- 172.21.15.42:0/2598473864 >> [v2:172.21.15.42:6808/2018240931,v1:172.21.15.42:6809/2018240931] conn(0x7f838c0705a0 0x7f838c0729c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f5953600640 1 -- 172.21.15.42:0/4139444264 >> 172.21.15.42:0/4139444264 conn(0x7f594c067b60 msgr2=0x7f594c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fddc1800640 1 -- 172.21.15.42:0/2575272265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddbc0d5390 msgr2=0x7fddbc0d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fddc1800640 1 --2- 172.21.15.42:0/2575272265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddbc0d5390 0x7fddbc0d1120 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fddb4009d90 tx=0x7fddb401c900 comp rx=0 tx=0).stop 2024-05-28T13:58:29.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fddc1800640 1 -- 172.21.15.42:0/2575272265 shutdown_connections 2024-05-28T13:58:29.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fddc1800640 1 --2- 172.21.15.42:0/2575272265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddbc0d5390 0x7fddbc0d1120 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fddc1800640 1 --2- 172.21.15.42:0/2575272265 >> v2:172.21.15.42:3300/0 conn(0x7fddbc15e8a0 0x7fddbc0d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fddc1800640 1 -- 172.21.15.42:0/2575272265 >> 172.21.15.42:0/2575272265 conn(0x7fddbc08bfe0 msgr2=0x7fddbc0d4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.605+0000 7fe60aa00640 1 -- 172.21.15.42:0/1496241203 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe60401b020 con 0x7fe614165e00 2024-05-28T13:58:29.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fe60aa00640 1 --2- 172.21.15.42:0/1496241203 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe5f40683b0 0x7fe5f406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fe60aa00640 1 -- 172.21.15.42:0/1496241203 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe604086d70 con 0x7fe614165e00 2024-05-28T13:58:29.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fe60aa00640 1 --2- 172.21.15.42:0/1496241203 >> [v2:172.21.15.42:6816/254756381,v1:172.21.15.42:6817/254756381] conn(0x7fe5f40705b0 0x7fe5f40729d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fe60aa00640 1 -- 172.21.15.42:0/1496241203 --> [v2:172.21.15.42:6816/254756381,v1:172.21.15.42:6817/254756381] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe5f4073080 con 0x7fe5f40705b0 2024-05-28T13:58:29.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fe60aa00640 1 -- 172.21.15.42:0/1496241203 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe604087200 con 0x7fe614165e00 2024-05-28T13:58:29.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f5953600640 1 -- 172.21.15.42:0/4139444264 shutdown_connections 2024-05-28T13:58:29.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f16bfe00640 1 --2- 172.21.15.42:0/286500610 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16a0068450 0x7f16a006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f6c6a400640 1 -- 172.21.15.42:0/2202430146 shutdown_connections 2024-05-28T13:58:29.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f6c6a400640 1 -- 172.21.15.42:0/2202430146 >> 172.21.15.42:0/2202430146 conn(0x7f6c640b1ee0 msgr2=0x7f6c640b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fddc1800640 1 -- 172.21.15.42:0/2575272265 shutdown_connections 2024-05-28T13:58:29.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fddc1800640 1 -- 172.21.15.42:0/2575272265 wait complete. 2024-05-28T13:58:29.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f6c6a400640 1 -- 172.21.15.42:0/2202430146 shutdown_connections 2024-05-28T13:58:29.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f6c6a400640 1 -- 172.21.15.42:0/2202430146 wait complete. 2024-05-28T13:58:29.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f16bf400640 1 --2- 172.21.15.42:0/286500610 >> [v2:172.21.15.42:6824/2252246131,v1:172.21.15.42:6825/2252246131] conn(0x7f16a0070670 0x7f16a0072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f16bfe00640 1 --2- 172.21.15.42:0/286500610 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16a0068450 0x7f16a006a910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f16c01fa9f0 tx=0x7f16ac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fe613400640 1 --2- 172.21.15.42:0/1496241203 >> [v2:172.21.15.42:6816/254756381,v1:172.21.15.42:6817/254756381] conn(0x7fe5f40705b0 0x7fe5f40729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fe612a00640 1 --2- 172.21.15.42:0/1496241203 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe5f40683b0 0x7fe5f406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fddc1800640 1 Processor -- start 2024-05-28T13:58:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7f5953600640 1 -- 172.21.15.42:0/4139444264 wait complete. 2024-05-28T13:58:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f5953600640 1 Processor -- start 2024-05-28T13:58:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f5953600640 1 -- start start 2024-05-28T13:58:29.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fddc1800640 1 -- start start 2024-05-28T13:58:29.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fe612a00640 1 --2- 172.21.15.42:0/1496241203 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe5f40683b0 0x7fe5f406a870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6000ab000 tx=0x7fe600006040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f6c6a400640 1 Processor -- start 2024-05-28T13:58:29.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f83b1200640 1 --2- 172.21.15.42:0/2598473864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f838c068380 0x7f838c06a840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0af570 tx=0x7f8398007a50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fc7fea00640 1 -- 172.21.15.42:0/1901618735 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc7f0017070 con 0x7fc808071fe0 2024-05-28T13:58:29.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f6c6a400640 1 -- start start 2024-05-28T13:58:29.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fc7fea00640 1 --2- 172.21.15.42:0/1901618735 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7e4068380 0x7fc7e406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fc7fea00640 1 -- 172.21.15.42:0/1901618735 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc7f0086780 con 0x7fc808071fe0 2024-05-28T13:58:29.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fc7fea00640 1 --2- 172.21.15.42:0/1901618735 >> [v2:172.21.15.63:6800/4268829337,v1:172.21.15.63:6801/4268829337] conn(0x7fc7e4070520 0x7fc7e4072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fc7fea00640 1 -- 172.21.15.42:0/1901618735 --> [v2:172.21.15.63:6800/4268829337,v1:172.21.15.63:6801/4268829337] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7e4072ff0 con 0x7fc7e4070520 2024-05-28T13:58:29.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fc7fea00640 1 -- 172.21.15.42:0/1901618735 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc7f008d050 con 0x7fc808071fe0 2024-05-28T13:58:29.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fc806800640 1 --2- 172.21.15.42:0/1901618735 >> [v2:172.21.15.63:6800/4268829337,v1:172.21.15.63:6801/4268829337] conn(0x7fc7e4070520 0x7fc7e4072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.609+0000 7fc806800640 1 --2- 172.21.15.42:0/1901618735 >> [v2:172.21.15.63:6800/4268829337,v1:172.21.15.63:6801/4268829337] conn(0x7fc7e4070520 0x7fc7e4072940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fc805e00640 1 --2- 172.21.15.42:0/1901618735 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7e4068380 0x7fc7e406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fc7fea00640 1 -- 172.21.15.42:0/1901618735 <== osd.4 v2:172.21.15.63:6800/4268829337 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc7e4072ff0 con 0x7fc7e4070520 2024-05-28T13:58:29.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f16bf400640 1 --2- 172.21.15.42:0/286500610 >> [v2:172.21.15.42:6824/2252246131,v1:172.21.15.42:6825/2252246131] conn(0x7f16a0070670 0x7f16a0072a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fddc1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddbc0d5390 0x7fddbc14e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fddc1800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddbc15e8a0 0x7fddbc0b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fddbaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddbc0d5390 0x7fddbc14e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f5953600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f594c0cd810 0x7f594c151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fddbaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddbc0d5390 0x7fddbc14e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38530/0 (socket says 172.21.15.42:38530) 2024-05-28T13:58:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fddc1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddbc15ec80 0x7fddbc0b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f5953600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f594c144d40 0x7f594c156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fe613400640 1 --2- 172.21.15.42:0/1496241203 >> [v2:172.21.15.42:6816/254756381,v1:172.21.15.42:6817/254756381] conn(0x7fe5f40705b0 0x7fe5f40729d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fddba000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddbc15e8a0 0x7fddbc0b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fddba000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddbc15e8a0 0x7fddbc0b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42812/0 (socket says 172.21.15.42:42812) 2024-05-28T13:58:29.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f5950c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f594c0cd810 0x7f594c151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f5950c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f594c0cd810 0x7f594c151780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42820/0 (socket says 172.21.15.42:42820) 2024-05-28T13:58:29.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f83a9600640 1 -- 172.21.15.42:0/2598473864 <== osd.0 v2:172.21.15.42:6808/2018240931 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83a00c5e50 con 0x7f838c0705a0 2024-05-28T13:58:29.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f594be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f594c144d40 0x7f594c156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f594be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f594c144d40 0x7f594c156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38544/0 (socket says 172.21.15.42:38544) 2024-05-28T13:58:29.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f5953600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f594c15bf20 0x7f594c130ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fddc1800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc16ba60 con 0x7fddbc0d5390 2024-05-28T13:58:29.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fddc1800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc16b8e0 con 0x7fddbc15ec80 2024-05-28T13:58:29.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fddc1800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fddbc16bbe0 con 0x7fddbc15e8a0 2024-05-28T13:58:29.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fddbaa00640 1 -- 172.21.15.42:0/2894357326 learned_addr learned my addr 172.21.15.42:0/2894357326 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fddbb400640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddbc15ec80 0x7fddbc0b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7fc805e00640 1 --2- 172.21.15.42:0/1901618735 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7e4068380 0x7fc7e406a840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80ae000 tx=0x7fc7f80a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.613+0000 7f5950c00640 1 -- 172.21.15.42:0/1639087528 learned_addr learned my addr 172.21.15.42:0/1639087528 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c6a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c48004d50 0x7f6c64099990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c6a400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c640b3160 0x7f6c640b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c6a400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c640ad310 0x7f6c640ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c6a400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6415e740 con 0x7f6c48004d50 2024-05-28T13:58:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c6a400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6415e5c0 con 0x7f6c640ad310 2024-05-28T13:58:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c6a400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c6415e8c0 con 0x7f6c640b3160 2024-05-28T13:58:29.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fddbb400640 1 -- 172.21.15.42:0/2894357326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddbc15e8a0 msgr2=0x7fddbc0b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fddbb400640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddbc15e8a0 0x7fddbc0b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fddbb400640 1 -- 172.21.15.42:0/2894357326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddbc0d5390 msgr2=0x7fddbc14e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fddbb400640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddbc0d5390 0x7fddbc14e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fddbb400640 1 -- 172.21.15.42:0/2894357326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc0b9c20 con 0x7fddbc15ec80 2024-05-28T13:58:29.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fddbaa00640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddbc0d5390 0x7fddbc14e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:29.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fddba000640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddbc15e8a0 0x7fddbc0b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:29.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fddbb400640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddbc15ec80 0x7fddbc0b64b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fddb40ac000 tx=0x7fddb40196a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c69a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c640ad310 0x7f6c640ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c69a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c640ad310 0x7f6c640ad790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:37980/0 (socket says 172.21.15.42:37980) 2024-05-28T13:58:29.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c69a00640 1 -- 172.21.15.42:0/603651415 learned_addr learned my addr 172.21.15.42:0/603651415 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c63e00640 1 --2- 172.21.15.42:0/603651415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c640b3160 0x7f6c640b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f16b7e00640 1 -- 172.21.15.42:0/286500610 <== osd.3 v2:172.21.15.42:6824/2252246131 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f16a0073140 con 0x7f16a0070670 2024-05-28T13:58:29.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fddb2a00640 1 -- 172.21.15.42:0/2894357326 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb40b2050 con 0x7fddbc15ec80 2024-05-28T13:58:29.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c69000640 1 --2- 172.21.15.42:0/603651415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c48004d50 0x7f6c64099990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fddb2a00640 1 -- 172.21.15.42:0/2894357326 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb40b3040 con 0x7fddbc15ec80 2024-05-28T13:58:29.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fddc1800640 1 -- 172.21.15.42:0/2894357326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddbc0b8b80 con 0x7fddbc15ec80 2024-05-28T13:58:29.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fddc1800640 1 -- 172.21.15.42:0/2894357326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddbc0b9140 con 0x7fddbc15ec80 2024-05-28T13:58:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c63e00640 1 -- 172.21.15.42:0/603651415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c640ad310 msgr2=0x7f6c640ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c63e00640 1 --2- 172.21.15.42:0/603651415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c640ad310 0x7f6c640ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c63e00640 1 -- 172.21.15.42:0/603651415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c48004d50 msgr2=0x7f6c64099990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c63e00640 1 --2- 172.21.15.42:0/603651415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c48004d50 0x7f6c64099990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c63e00640 1 -- 172.21.15.42:0/603651415 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c64099ed0 con 0x7f6c640b3160 2024-05-28T13:58:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fddb2a00640 1 -- 172.21.15.42:0/2894357326 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb40ad850 con 0x7fddbc15ec80 2024-05-28T13:58:29.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fe60aa00640 1 -- 172.21.15.42:0/1496241203 <== osd.2 v2:172.21.15.42:6816/254756381 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe5f4073080 con 0x7fe5f40705b0 2024-05-28T13:58:29.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c63e00640 1 --2- 172.21.15.42:0/603651415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c640b3160 0x7f6c640b35e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6c540085f0 tx=0x7f6c54008ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c60c00640 1 -- 172.21.15.42:0/603651415 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5400ab00 con 0x7f6c640b3160 2024-05-28T13:58:29.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c60c00640 1 -- 172.21.15.42:0/603651415 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c540060d0 con 0x7f6c640b3160 2024-05-28T13:58:29.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f6c60c00640 1 -- 172.21.15.42:0/603651415 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c540064c0 con 0x7f6c640b3160 2024-05-28T13:58:29.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7fddc1800640 1 -- 172.21.15.42:0/2894357326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdd94001040 con 0x7fddbc15ec80 2024-05-28T13:58:29.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f6c6a400640 1 -- 172.21.15.42:0/603651415 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c640ae040 con 0x7f6c640b3160 2024-05-28T13:58:29.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f6c6a400640 1 -- 172.21.15.42:0/603651415 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c64160dd0 con 0x7f6c640b3160 2024-05-28T13:58:29.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.617+0000 7f5951600640 1 --2- 172.21.15.42:0/1639087528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f594c15bf20 0x7f594c130ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f5953600640 1 -- 172.21.15.42:0/1639087528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f594c15c9e0 con 0x7f594c144d40 2024-05-28T13:58:29.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f5953600640 1 -- 172.21.15.42:0/1639087528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f594c15c860 con 0x7f594c15bf20 2024-05-28T13:58:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f5953600640 1 -- 172.21.15.42:0/1639087528 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f594c145120 con 0x7f594c0cd810 2024-05-28T13:58:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f5951600640 1 -- 172.21.15.42:0/1639087528 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f594c0cd810 msgr2=0x7f594c151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f5951600640 1 --2- 172.21.15.42:0/1639087528 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f594c0cd810 0x7f594c151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f5951600640 1 -- 172.21.15.42:0/1639087528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f594c144d40 msgr2=0x7f594c156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f5951600640 1 --2- 172.21.15.42:0/1639087528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f594c144d40 0x7f594c156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f5951600640 1 -- 172.21.15.42:0/1639087528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594c1572d0 con 0x7f594c15bf20 2024-05-28T13:58:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7fddb2a00640 1 -- 172.21.15.42:0/2894357326 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fddb40c0020 con 0x7fddbc15ec80 2024-05-28T13:58:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7fddb2a00640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd9c068380 0x7fdd9c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7fddb2a00640 1 -- 172.21.15.42:0/2894357326 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fddb4121470 con 0x7fddbc15ec80 2024-05-28T13:58:29.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7fddb2a00640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.63:6824/1958395718,v1:172.21.15.63:6825/1958395718] conn(0x7fdd9c070520 0x7fdd9c072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7fddb2a00640 1 -- 172.21.15.42:0/2894357326 --> [v2:172.21.15.63:6824/1958395718,v1:172.21.15.63:6825/1958395718] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fddb40a7ea0 con 0x7fdd9c070520 2024-05-28T13:58:29.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7fddb2a00640 1 -- 172.21.15.42:0/2894357326 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fddb4128050 con 0x7fddbc15ec80 2024-05-28T13:58:29.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f5951600640 1 --2- 172.21.15.42:0/1639087528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f594c15bf20 0x7f594c130ad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5944009d90 tx=0x7f594400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f5948c00640 1 -- 172.21.15.42:0/1639087528 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944018020 con 0x7f594c15bf20 2024-05-28T13:58:29.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7fddbaa00640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd9c068380 0x7fdd9c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f5948c00640 1 -- 172.21.15.42:0/1639087528 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5944014480 con 0x7f594c15bf20 2024-05-28T13:58:29.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f6c5a000640 1 -- 172.21.15.42:0/603651415 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c6404fcf0 con 0x7f6c640b3160 2024-05-28T13:58:29.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7fddbaa00640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd9c068380 0x7fdd9c06a840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fddac006aa0 tx=0x7fddac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f5953600640 1 -- 172.21.15.42:0/1639087528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594c0b03b0 con 0x7f594c15bf20 2024-05-28T13:58:29.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f5953600640 1 -- 172.21.15.42:0/1639087528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f594c0b0970 con 0x7f594c15bf20 2024-05-28T13:58:29.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.621+0000 7f5948c00640 1 -- 172.21.15.42:0/1639087528 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944014870 con 0x7f594c15bf20 2024-05-28T13:58:29.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.625+0000 7fddba000640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.63:6824/1958395718,v1:172.21.15.63:6825/1958395718] conn(0x7fdd9c070520 0x7fdd9c072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.625+0000 7fddba000640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.63:6824/1958395718,v1:172.21.15.63:6825/1958395718] conn(0x7fdd9c070520 0x7fdd9c072940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.625+0000 7f5953600640 1 -- 172.21.15.42:0/1639087528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5928001040 con 0x7f594c15bf20 2024-05-28T13:58:29.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.625+0000 7f6c60c00640 1 -- 172.21.15.42:0/603651415 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6c540047d0 con 0x7f6c640b3160 2024-05-28T13:58:29.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.625+0000 7f6c60c00640 1 --2- 172.21.15.42:0/603651415 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c44068400 0x7f6c4406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.625+0000 7f6c60c00640 1 -- 172.21.15.42:0/603651415 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6c540869f0 con 0x7f6c640b3160 2024-05-28T13:58:29.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.625+0000 7f6c60c00640 1 --2- 172.21.15.42:0/603651415 >> [v2:172.21.15.63:6808/1802936259,v1:172.21.15.63:6809/1802936259] conn(0x7f6c440705a0 0x7f6c440729c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.625+0000 7f6c60c00640 1 -- 172.21.15.42:0/603651415 --> [v2:172.21.15.63:6808/1802936259,v1:172.21.15.63:6809/1802936259] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c44073070 con 0x7f6c440705a0 2024-05-28T13:58:29.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.625+0000 7fddb2a00640 1 -- 172.21.15.42:0/2894357326 <== osd.6 v2:172.21.15.63:6824/1958395718 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fddb40a7ea0 con 0x7fdd9c070520 2024-05-28T13:58:29.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.625+0000 7f5948c00640 1 -- 172.21.15.42:0/1639087528 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f594401b020 con 0x7f594c15bf20 2024-05-28T13:58:29.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.625+0000 7f5948c00640 1 --2- 172.21.15.42:0/1639087528 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f592c068450 0x7f592c06a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.625+0000 7f5948c00640 1 -- 172.21.15.42:0/1639087528 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5944087720 con 0x7f594c15bf20 2024-05-28T13:58:29.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.629+0000 7f5948c00640 1 --2- 172.21.15.42:0/1639087528 >> [v2:172.21.15.63:6816/3211394017,v1:172.21.15.63:6817/3211394017] conn(0x7f592c070650 0x7f592c072a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.629+0000 7f5948c00640 1 -- 172.21.15.42:0/1639087528 --> [v2:172.21.15.63:6816/3211394017,v1:172.21.15.63:6817/3211394017] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f592c073120 con 0x7f592c070650 2024-05-28T13:58:29.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.629+0000 7f6c69a00640 1 --2- 172.21.15.42:0/603651415 >> [v2:172.21.15.63:6808/1802936259,v1:172.21.15.63:6809/1802936259] conn(0x7f6c440705a0 0x7f6c440729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.629+0000 7f6c69000640 1 --2- 172.21.15.42:0/603651415 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c44068400 0x7f6c4406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.629+0000 7f5948c00640 1 -- 172.21.15.42:0/1639087528 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f594408e050 con 0x7f594c15bf20 2024-05-28T13:58:29.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.629+0000 7f594be00640 1 --2- 172.21.15.42:0/1639087528 >> [v2:172.21.15.63:6816/3211394017,v1:172.21.15.63:6817/3211394017] conn(0x7f592c070650 0x7f592c072a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.629+0000 7f5950c00640 1 --2- 172.21.15.42:0/1639087528 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f592c068450 0x7f592c06a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.629+0000 7f594be00640 1 --2- 172.21.15.42:0/1639087528 >> [v2:172.21.15.63:6816/3211394017,v1:172.21.15.63:6817/3211394017] conn(0x7f592c070650 0x7f592c072a70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.629+0000 7f6c69a00640 1 --2- 172.21.15.42:0/603651415 >> [v2:172.21.15.63:6808/1802936259,v1:172.21.15.63:6809/1802936259] conn(0x7f6c440705a0 0x7f6c440729c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.629+0000 7f5948c00640 1 -- 172.21.15.42:0/1639087528 <== osd.7 v2:172.21.15.63:6816/3211394017 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f592c073120 con 0x7f592c070650 2024-05-28T13:58:29.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.629+0000 7f6c60c00640 1 -- 172.21.15.42:0/603651415 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6c54052ce0 con 0x7f6c640b3160 2024-05-28T13:58:29.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.629+0000 7f6c60c00640 1 -- 172.21.15.42:0/603651415 <== osd.5 v2:172.21.15.63:6808/1802936259 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6c44073070 con 0x7f6c440705a0 2024-05-28T13:58:29.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.629+0000 7f6c69000640 1 --2- 172.21.15.42:0/603651415 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c44068400 0x7f6c4406a8c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000c880 tx=0x7f6c50008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.633+0000 7ff16ec00640 1 -- 172.21.15.42:0/525158182 --> [v2:172.21.15.42:6800/437002159,v1:172.21.15.42:6801/437002159] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff16804fdd0 con 0x7ff148070720 2024-05-28T13:58:29.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.633+0000 7f5950c00640 1 --2- 172.21.15.42:0/1639087528 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f592c068450 0x7f592c06a910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5940000f70 tx=0x7f5940009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.633+0000 7ff15fe00640 1 -- 172.21.15.42:0/525158182 <== osd.1 v2:172.21.15.42:6800/437002159 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff16804fdd0 con 0x7ff148070720 2024-05-28T13:58:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.633+0000 7ff16ec00640 1 -- 172.21.15.42:0/525158182 >> [v2:172.21.15.42:6800/437002159,v1:172.21.15.42:6801/437002159] conn(0x7ff148070720 msgr2=0x7ff148072b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.633+0000 7ff16ec00640 1 --2- 172.21.15.42:0/525158182 >> [v2:172.21.15.42:6800/437002159,v1:172.21.15.42:6801/437002159] conn(0x7ff148070720 0x7ff148072b40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.633+0000 7ff16ec00640 1 -- 172.21.15.42:0/525158182 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff148068580 msgr2=0x7ff14806aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.633+0000 7ff16ec00640 1 --2- 172.21.15.42:0/525158182 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff148068580 0x7ff14806aa40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff1600af000 tx=0x7ff160004ac0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.633+0000 7ff16ec00640 1 -- 172.21.15.42:0/525158182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff168162ca0 msgr2=0x7ff1681527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.633+0000 7ff16ec00640 1 --2- 172.21.15.42:0/525158182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff168162ca0 0x7ff1681527d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff15400c400 tx=0x7ff15400c8c0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.633+0000 7ff16cc00640 1 -- 172.21.15.42:0/525158182 reap_dead start 2024-05-28T13:58:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.633+0000 7ff16ec00640 1 -- 172.21.15.42:0/525158182 shutdown_connections 2024-05-28T13:58:29.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.633+0000 7ff16ec00640 1 -- 172.21.15.42:0/525158182 >> 172.21.15.42:0/525158182 conn(0x7ff168067b60 msgr2=0x7ff1680b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.637+0000 7ff16ec00640 1 -- 172.21.15.42:0/525158182 shutdown_connections 2024-05-28T13:58:29.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.637+0000 7ff16ec00640 1 -- 172.21.15.42:0/525158182 wait complete. 2024-05-28T13:58:29.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.641+0000 7fc80d200640 1 -- 172.21.15.42:0/1901618735 --> [v2:172.21.15.63:6800/4268829337,v1:172.21.15.63:6801/4268829337] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc808073890 con 0x7fc7e4070520 2024-05-28T13:58:29.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.641+0000 7fc7fea00640 1 -- 172.21.15.42:0/1901618735 <== osd.4 v2:172.21.15.63:6800/4268829337 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc808073890 con 0x7fc7e4070520 2024-05-28T13:58:29.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.641+0000 7fc80d200640 1 -- 172.21.15.42:0/1901618735 >> [v2:172.21.15.63:6800/4268829337,v1:172.21.15.63:6801/4268829337] conn(0x7fc7e4070520 msgr2=0x7fc7e4072940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.641+0000 7fc80d200640 1 --2- 172.21.15.42:0/1901618735 >> [v2:172.21.15.63:6800/4268829337,v1:172.21.15.63:6801/4268829337] conn(0x7fc7e4070520 0x7fc7e4072940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.641+0000 7fc80d200640 1 -- 172.21.15.42:0/1901618735 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7e4068380 msgr2=0x7fc7e406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.641+0000 7fc80d200640 1 --2- 172.21.15.42:0/1901618735 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7e4068380 0x7fc7e406a840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80ae000 tx=0x7fc7f80a7070 comp rx=0 tx=0).stop 2024-05-28T13:58:29.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.641+0000 7fc80d200640 1 -- 172.21.15.42:0/1901618735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc808071fe0 msgr2=0x7fc808154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.641+0000 7fc80d200640 1 --2- 172.21.15.42:0/1901618735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc808071fe0 0x7fc808154140 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000b550 tx=0x7fc7f000ba10 comp rx=0 tx=0).stop 2024-05-28T13:58:29.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.641+0000 7fc806800640 1 -- 172.21.15.42:0/1901618735 reap_dead start 2024-05-28T13:58:29.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.641+0000 7fc80d200640 1 -- 172.21.15.42:0/1901618735 shutdown_connections 2024-05-28T13:58:29.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.641+0000 7fc80d200640 1 -- 172.21.15.42:0/1901618735 >> 172.21.15.42:0/1901618735 conn(0x7fc80806b9f0 msgr2=0x7fc8080b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.641+0000 7f83b4600640 1 -- 172.21.15.42:0/2598473864 --> [v2:172.21.15.42:6808/2018240931,v1:172.21.15.42:6809/2018240931] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83ac04fdd0 con 0x7f838c0705a0 2024-05-28T13:58:29.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.641+0000 7f83a9600640 1 -- 172.21.15.42:0/2598473864 <== osd.0 v2:172.21.15.42:6808/2018240931 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f83ac04fdd0 con 0x7f838c0705a0 2024-05-28T13:58:29.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7f83b4600640 1 -- 172.21.15.42:0/2598473864 >> [v2:172.21.15.42:6808/2018240931,v1:172.21.15.42:6809/2018240931] conn(0x7f838c0705a0 msgr2=0x7f838c0729c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7f83b4600640 1 --2- 172.21.15.42:0/2598473864 >> [v2:172.21.15.42:6808/2018240931,v1:172.21.15.42:6809/2018240931] conn(0x7f838c0705a0 0x7f838c0729c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7f83b4600640 1 -- 172.21.15.42:0/2598473864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f838c068380 msgr2=0x7f838c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7f83b4600640 1 --2- 172.21.15.42:0/2598473864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f838c068380 0x7f838c06a840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0af570 tx=0x7f8398007a50 comp rx=0 tx=0).stop 2024-05-28T13:58:29.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7f83b4600640 1 -- 172.21.15.42:0/2598473864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83ac06db40 msgr2=0x7f83ac142a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7f83b4600640 1 --2- 172.21.15.42:0/2598473864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83ac06db40 0x7f83ac142a70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f83a001a020 tx=0x7f83a001ac30 comp rx=0 tx=0).stop 2024-05-28T13:58:29.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7fc80d200640 1 -- 172.21.15.42:0/1901618735 shutdown_connections 2024-05-28T13:58:29.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7f83b2600640 1 -- 172.21.15.42:0/2598473864 reap_dead start 2024-05-28T13:58:29.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7fc80d200640 1 -- 172.21.15.42:0/1901618735 wait complete. 2024-05-28T13:58:29.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7f83b4600640 1 -- 172.21.15.42:0/2598473864 shutdown_connections 2024-05-28T13:58:29.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7f83b4600640 1 -- 172.21.15.42:0/2598473864 >> 172.21.15.42:0/2598473864 conn(0x7f83ac067b60 msgr2=0x7f83ac1609c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7f83b4600640 1 -- 172.21.15.42:0/2598473864 shutdown_connections 2024-05-28T13:58:29.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7f83b4600640 1 -- 172.21.15.42:0/2598473864 wait complete. 2024-05-28T13:58:29.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7f16c5400640 1 -- 172.21.15.42:0/286500610 --> [v2:172.21.15.42:6824/2252246131,v1:172.21.15.42:6825/2252246131] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f16c004fcf0 con 0x7f16a0070670 2024-05-28T13:58:29.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.645+0000 7f16b7e00640 1 -- 172.21.15.42:0/286500610 <== osd.3 v2:172.21.15.42:6824/2252246131 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f16c004fcf0 con 0x7f16a0070670 2024-05-28T13:58:29.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.649+0000 7fe619800640 1 -- 172.21.15.42:0/1496241203 --> [v2:172.21.15.42:6816/254756381,v1:172.21.15.42:6817/254756381] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe61404fdd0 con 0x7fe5f40705b0 2024-05-28T13:58:29.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7f16c5400640 1 -- 172.21.15.42:0/286500610 >> [v2:172.21.15.42:6824/2252246131,v1:172.21.15.42:6825/2252246131] conn(0x7f16a0070670 msgr2=0x7f16a0072a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7f16c5400640 1 --2- 172.21.15.42:0/286500610 >> [v2:172.21.15.42:6824/2252246131,v1:172.21.15.42:6825/2252246131] conn(0x7f16a0070670 0x7f16a0072a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7f16c5400640 1 -- 172.21.15.42:0/286500610 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16a0068450 msgr2=0x7f16a006a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7f16c5400640 1 --2- 172.21.15.42:0/286500610 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16a0068450 0x7f16a006a910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f16c01fa9f0 tx=0x7f16ac008040 comp rx=0 tx=0).stop 2024-05-28T13:58:29.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7f16c5400640 1 -- 172.21.15.42:0/286500610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16c015bad0 msgr2=0x7f16c009b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7f16c5400640 1 --2- 172.21.15.42:0/286500610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16c015bad0 0x7f16c009b350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f16b80af040 tx=0x7f16b80197d0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7fe60aa00640 1 -- 172.21.15.42:0/1496241203 <== osd.2 v2:172.21.15.42:6816/254756381 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe61404fdd0 con 0x7fe5f40705b0 2024-05-28T13:58:29.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7f16c4a00640 1 -- 172.21.15.42:0/286500610 reap_dead start 2024-05-28T13:58:29.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7f16c5400640 1 -- 172.21.15.42:0/286500610 shutdown_connections 2024-05-28T13:58:29.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7f16c5400640 1 -- 172.21.15.42:0/286500610 >> 172.21.15.42:0/286500610 conn(0x7f16c00b6780 msgr2=0x7f16c0074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7f16c5400640 1 -- 172.21.15.42:0/286500610 shutdown_connections 2024-05-28T13:58:29.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7f16c5400640 1 -- 172.21.15.42:0/286500610 wait complete. 2024-05-28T13:58:29.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7fe619800640 1 -- 172.21.15.42:0/1496241203 >> [v2:172.21.15.42:6816/254756381,v1:172.21.15.42:6817/254756381] conn(0x7fe5f40705b0 msgr2=0x7fe5f40729d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7fe619800640 1 --2- 172.21.15.42:0/1496241203 >> [v2:172.21.15.42:6816/254756381,v1:172.21.15.42:6817/254756381] conn(0x7fe5f40705b0 0x7fe5f40729d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7fe619800640 1 -- 172.21.15.42:0/1496241203 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe5f40683b0 msgr2=0x7fe5f406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7fe619800640 1 --2- 172.21.15.42:0/1496241203 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe5f40683b0 0x7fe5f406a870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6000ab000 tx=0x7fe600006040 comp rx=0 tx=0).stop 2024-05-28T13:58:29.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7fe619800640 1 -- 172.21.15.42:0/1496241203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe614165e00 msgr2=0x7fe61415bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7fe619800640 1 --2- 172.21.15.42:0/1496241203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe614165e00 0x7fe61415bae0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe60400d490 tx=0x7fe60400d950 comp rx=0 tx=0).stop 2024-05-28T13:58:29.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7f6c5a000640 1 -- 172.21.15.42:0/603651415 --> [v2:172.21.15.63:6808/1802936259,v1:172.21.15.63:6809/1802936259] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6c64160ba0 con 0x7f6c440705a0 2024-05-28T13:58:29.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7fe613400640 1 -- 172.21.15.42:0/1496241203 reap_dead start 2024-05-28T13:58:29.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7fe619800640 1 -- 172.21.15.42:0/1496241203 shutdown_connections 2024-05-28T13:58:29.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.653+0000 7fe619800640 1 -- 172.21.15.42:0/1496241203 >> 172.21.15.42:0/1496241203 conn(0x7fe6140b25e0 msgr2=0x7fe614163170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7f6c60c00640 1 -- 172.21.15.42:0/603651415 <== osd.5 v2:172.21.15.63:6808/1802936259 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6c64160ba0 con 0x7f6c440705a0 2024-05-28T13:58:29.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7fe619800640 1 -- 172.21.15.42:0/1496241203 shutdown_connections 2024-05-28T13:58:29.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7fe619800640 1 -- 172.21.15.42:0/1496241203 wait complete. 2024-05-28T13:58:29.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7f6c5a000640 1 -- 172.21.15.42:0/603651415 >> [v2:172.21.15.63:6808/1802936259,v1:172.21.15.63:6809/1802936259] conn(0x7f6c440705a0 msgr2=0x7f6c440729c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7f6c5a000640 1 --2- 172.21.15.42:0/603651415 >> [v2:172.21.15.63:6808/1802936259,v1:172.21.15.63:6809/1802936259] conn(0x7f6c440705a0 0x7f6c440729c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7fddc1800640 1 -- 172.21.15.42:0/2894357326 --> [v2:172.21.15.63:6824/1958395718,v1:172.21.15.63:6825/1958395718] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdd94002de0 con 0x7fdd9c070520 2024-05-28T13:58:29.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7f6c5a000640 1 -- 172.21.15.42:0/603651415 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c44068400 msgr2=0x7f6c4406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7f6c5a000640 1 --2- 172.21.15.42:0/603651415 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c44068400 0x7f6c4406a8c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000c880 tx=0x7f6c50008040 comp rx=0 tx=0).stop 2024-05-28T13:58:29.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7f6c5a000640 1 -- 172.21.15.42:0/603651415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c640b3160 msgr2=0x7f6c640b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7fddb2a00640 1 -- 172.21.15.42:0/2894357326 <== osd.6 v2:172.21.15.63:6824/1958395718 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdd94002de0 con 0x7fdd9c070520 2024-05-28T13:58:29.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7f6c5a000640 1 --2- 172.21.15.42:0/603651415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c640b3160 0x7f6c640b35e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6c540085f0 tx=0x7f6c54008ab0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7f6c69a00640 1 -- 172.21.15.42:0/603651415 reap_dead start 2024-05-28T13:58:29.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7f6c5a000640 1 -- 172.21.15.42:0/603651415 shutdown_connections 2024-05-28T13:58:29.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7f6c5a000640 1 -- 172.21.15.42:0/603651415 >> 172.21.15.42:0/603651415 conn(0x7f6c640b1ee0 msgr2=0x7f6c641503c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7f6c5a000640 1 -- 172.21.15.42:0/603651415 shutdown_connections 2024-05-28T13:58:29.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7f6c5a000640 1 -- 172.21.15.42:0/603651415 wait complete. 2024-05-28T13:58:29.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7fddc1800640 1 -- 172.21.15.42:0/2894357326 >> [v2:172.21.15.63:6824/1958395718,v1:172.21.15.63:6825/1958395718] conn(0x7fdd9c070520 msgr2=0x7fdd9c072940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.657+0000 7fddc1800640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.63:6824/1958395718,v1:172.21.15.63:6825/1958395718] conn(0x7fdd9c070520 0x7fdd9c072940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.661+0000 7fddc1800640 1 -- 172.21.15.42:0/2894357326 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd9c068380 msgr2=0x7fdd9c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.661+0000 7fddc1800640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd9c068380 0x7fdd9c06a840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fddac006aa0 tx=0x7fddac008040 comp rx=0 tx=0).stop 2024-05-28T13:58:29.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.661+0000 7fddc1800640 1 -- 172.21.15.42:0/2894357326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddbc15ec80 msgr2=0x7fddbc0b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.661+0000 7fddc1800640 1 --2- 172.21.15.42:0/2894357326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddbc15ec80 0x7fddbc0b64b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fddb40ac000 tx=0x7fddb40196a0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.661+0000 7fddbb400640 1 -- 172.21.15.42:0/2894357326 reap_dead start 2024-05-28T13:58:29.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.661+0000 7fddc1800640 1 -- 172.21.15.42:0/2894357326 shutdown_connections 2024-05-28T13:58:29.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.661+0000 7fddc1800640 1 -- 172.21.15.42:0/2894357326 >> 172.21.15.42:0/2894357326 conn(0x7fddbc08bfe0 msgr2=0x7fddbc15fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.661+0000 7fddc1800640 1 -- 172.21.15.42:0/2894357326 shutdown_connections 2024-05-28T13:58:29.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.661+0000 7fddc1800640 1 -- 172.21.15.42:0/2894357326 wait complete. 2024-05-28T13:58:29.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.661+0000 7f5953600640 1 -- 172.21.15.42:0/1639087528 --> [v2:172.21.15.63:6816/3211394017,v1:172.21.15.63:6817/3211394017] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5928002de0 con 0x7f592c070650 2024-05-28T13:58:29.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.665+0000 7f5948c00640 1 -- 172.21.15.42:0/1639087528 <== osd.7 v2:172.21.15.63:6816/3211394017 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5928002de0 con 0x7f592c070650 2024-05-28T13:58:29.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.669+0000 7f593e000640 1 -- 172.21.15.42:0/1639087528 >> [v2:172.21.15.63:6816/3211394017,v1:172.21.15.63:6817/3211394017] conn(0x7f592c070650 msgr2=0x7f592c072a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.669+0000 7f593e000640 1 --2- 172.21.15.42:0/1639087528 >> [v2:172.21.15.63:6816/3211394017,v1:172.21.15.63:6817/3211394017] conn(0x7f592c070650 0x7f592c072a70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.669+0000 7f593e000640 1 -- 172.21.15.42:0/1639087528 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f592c068450 msgr2=0x7f592c06a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.669+0000 7f593e000640 1 --2- 172.21.15.42:0/1639087528 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f592c068450 0x7f592c06a910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5940000f70 tx=0x7f5940009040 comp rx=0 tx=0).stop 2024-05-28T13:58:29.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.669+0000 7f593e000640 1 -- 172.21.15.42:0/1639087528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f594c15bf20 msgr2=0x7f594c130ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.669+0000 7f593e000640 1 --2- 172.21.15.42:0/1639087528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f594c15bf20 0x7f594c130ad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5944009d90 tx=0x7f594400c490 comp rx=0 tx=0).stop 2024-05-28T13:58:29.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.669+0000 7f5951600640 1 -- 172.21.15.42:0/1639087528 reap_dead start 2024-05-28T13:58:29.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.669+0000 7f593e000640 1 -- 172.21.15.42:0/1639087528 shutdown_connections 2024-05-28T13:58:29.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.669+0000 7f593e000640 1 -- 172.21.15.42:0/1639087528 >> 172.21.15.42:0/1639087528 conn(0x7f594c067b60 msgr2=0x7f594c159690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.669+0000 7f593e000640 1 -- 172.21.15.42:0/1639087528 shutdown_connections 2024-05-28T13:58:29.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.669+0000 7f593e000640 1 -- 172.21.15.42:0/1639087528 wait complete. 2024-05-28T13:58:29.671 INFO:teuthology.orchestra.run.smithi042.stdout:55834574854 2024-05-28T13:58:29.672 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-28T13:58:29.672 INFO:teuthology.orchestra.run.smithi042.stdout:55834574855 2024-05-28T13:58:29.673 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-28T13:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout:55834574855 2024-05-28T13:58:29.677 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-28T13:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout:55834574854 2024-05-28T13:58:29.681 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-28T13:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout:55834574855 2024-05-28T13:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout:55834574854 2024-05-28T13:58:29.690 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-28T13:58:29.691 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-28T13:58:29.693 INFO:teuthology.orchestra.run.smithi042.stdout:55834574856 2024-05-28T13:58:29.694 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-28T13:58:29.695 INFO:teuthology.orchestra.run.smithi042.stdout:55834574854 2024-05-28T13:58:29.695 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-28T13:58:29.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.929+0000 7f42e9800640 1 Processor -- start 2024-05-28T13:58:29.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.929+0000 7f42e9800640 1 -- start start 2024-05-28T13:58:29.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.929+0000 7f42e9800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42e40cc170 0x7f42e40ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.929+0000 7f42e9800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f42e4162490 0x7f42e41628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.929+0000 7f42e9800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f42e4162e30 0x7f42e40ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.929+0000 7f42e9800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f42e40cfa90 con 0x7f42e4162490 2024-05-28T13:58:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.929+0000 7f42e9800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f42e41632c0 con 0x7f42e4162e30 2024-05-28T13:58:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.929+0000 7f42e9800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f42e40ce700 con 0x7f42e40cc170 2024-05-28T13:58:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.929+0000 7f42e3400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f42e4162e30 0x7f42e40ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.929+0000 7f42e3400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f42e4162e30 0x7f42e40ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:37994/0 (socket says 172.21.15.42:37994) 2024-05-28T13:58:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.933+0000 7f42e2a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42e40cc170 0x7f42e40ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.933+0000 7f42e3400640 1 -- 172.21.15.42:0/3503695295 learned_addr learned my addr 172.21.15.42:0/3503695295 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.933+0000 7f42e2000640 1 --2- 172.21.15.42:0/3503695295 >> v2:172.21.15.42:3300/0 conn(0x7f42e4162490 0x7f42e41628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.933+0000 7f42e3400640 1 -- 172.21.15.42:0/3503695295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42e40cc170 msgr2=0x7f42e40ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.933+0000 7f42e3400640 1 --2- 172.21.15.42:0/3503695295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42e40cc170 0x7f42e40ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.933+0000 7f42e3400640 1 -- 172.21.15.42:0/3503695295 >> v2:172.21.15.42:3300/0 conn(0x7f42e4162490 msgr2=0x7f42e41628f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.933+0000 7f42e3400640 1 --2- 172.21.15.42:0/3503695295 >> v2:172.21.15.42:3300/0 conn(0x7f42e4162490 0x7f42e41628f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.933+0000 7f42e3400640 1 -- 172.21.15.42:0/3503695295 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e40ce9e0 con 0x7f42e4162e30 2024-05-28T13:58:29.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.933+0000 7f42e2000640 1 --2- 172.21.15.42:0/3503695295 >> v2:172.21.15.42:3300/0 conn(0x7f42e4162490 0x7f42e41628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:29.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.937+0000 7f42e3400640 1 --2- 172.21.15.42:0/3503695295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e4162e30 0x7f42e40ccae0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f42e4070640 tx=0x7f42d001c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4669c084f8170445 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.937+0000 7f9421a00640 1 Processor -- start 2024-05-28T13:58:29.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.937+0000 7f42e1600640 1 -- 172.21.15.42:0/3503695295 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d00b0070 con 0x7f42e4162e30 2024-05-28T13:58:29.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.937+0000 7f42e1600640 1 -- 172.21.15.42:0/3503695295 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d00ab600 con 0x7f42e4162e30 2024-05-28T13:58:29.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.937+0000 7f42e1600640 1 -- 172.21.15.42:0/3503695295 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d00ab9d0 con 0x7f42e4162e30 2024-05-28T13:58:29.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.937+0000 7f9421a00640 1 -- start start 2024-05-28T13:58:29.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f1716400640 1 Processor -- start 2024-05-28T13:58:29.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f9421a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9414053d00 0x7f9414006f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f941a000640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f9414055790 0x7f94140df120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:50602/0 (socket says 172.21.15.42:50602) 2024-05-28T13:58:29.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f9419600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9414053d00 0x7f9414006f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f9419600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9414053d00 0x7f9414006f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42834/0 (socket says 172.21.15.42:42834) 2024-05-28T13:58:29.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f941aa00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f9414007460 0x7f941404f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60510/0 (socket says 172.21.15.42:60510) 2024-05-28T13:58:29.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f941a000640 1 -- 172.21.15.42:0/2839464117 learned_addr learned my addr 172.21.15.42:0/2839464117 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:29.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f42e9800640 1 -- 172.21.15.42:0/3503695295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e4162e30 msgr2=0x7f42e40ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f42e9800640 1 --2- 172.21.15.42:0/3503695295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e4162e30 0x7f42e40ccae0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f42e4070640 tx=0x7f42d001c490 comp rx=0 tx=0).stop 2024-05-28T13:58:29.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f9421a00640 1 -- 172.21.15.42:0/2839464117 --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9414052be0 con 0x7f9414007460 2024-05-28T13:58:29.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f42e9800640 1 -- 172.21.15.42:0/3503695295 shutdown_connections 2024-05-28T13:58:29.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f9421a00640 1 -- 172.21.15.42:0/2839464117 --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9414054180 con 0x7f9414055790 2024-05-28T13:58:29.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f9421a00640 1 -- 172.21.15.42:0/2839464117 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9414054400 con 0x7f9414053d00 2024-05-28T13:58:29.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f42e9800640 1 --2- 172.21.15.42:0/3503695295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e4162e30 0x7f42e40ccae0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f42e9800640 1 --2- 172.21.15.42:0/3503695295 >> v2:172.21.15.42:3300/0 conn(0x7f42e4162490 0x7f42e41628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f42e9800640 1 --2- 172.21.15.42:0/3503695295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42e40cc170 0x7f42e40ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f42e9800640 1 -- 172.21.15.42:0/3503695295 >> 172.21.15.42:0/3503695295 conn(0x7f42e4073010 msgr2=0x7f42e4073420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f42e9800640 1 -- 172.21.15.42:0/3503695295 shutdown_connections 2024-05-28T13:58:29.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f42e9800640 1 -- 172.21.15.42:0/3503695295 wait complete. 2024-05-28T13:58:29.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f1716400640 1 -- start start 2024-05-28T13:58:29.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f1716400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f17100ae390 0x7f171006f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f1716400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17100af5f0 con 0x7f17100aee80 2024-05-28T13:58:29.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f1716400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17100af8d0 con 0x7f171015a920 2024-05-28T13:58:29.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f1716400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f17100b31e0 con 0x7f17100ae390 2024-05-28T13:58:29.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1391819027 0 0) 0x7f941c050c80 con 0x7f9414055790 2024-05-28T13:58:29.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9400003530 con 0x7f9414055790 2024-05-28T13:58:29.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.941+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3968160508 0 0) 0x7f9408001120 con 0x7f9414007460 2024-05-28T13:58:29.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f1715a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f17100aee80 0x7f17100b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60526/0 (socket says 172.21.15.42:60526) 2024-05-28T13:58:29.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f1715a00640 1 -- 172.21.15.42:0/935611267 learned_addr learned my addr 172.21.15.42:0/935611267 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:29.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f941c050c80 con 0x7f9414007460 2024-05-28T13:58:29.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2422442455 0 0) 0x7f9400003530 con 0x7f9414055790 2024-05-28T13:58:29.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9414053d00 msgr2=0x7f9414006f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f1715000640 1 --2- 172.21.15.42:0/935611267 >> v2:172.21.15.63:3300/0 conn(0x7f17100ae390 0x7f171006f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 -- 172.21.15.42:0/935611267 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 996711043 0 0) 0x7f17100af5f0 con 0x7f17100aee80 2024-05-28T13:58:29.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941be00640 1 --2- 172.21.15.42:0/2839464117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9414053d00 0x7f9414006f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 -- 172.21.15.42:0/935611267 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16f4003530 con 0x7f17100aee80 2024-05-28T13:58:29.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f02a0200640 1 Processor -- start 2024-05-28T13:58:29.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f472b800640 1 Processor -- start 2024-05-28T13:58:29.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 >> v1:172.21.15.42:6789/0 conn(0x7f9414007460 legacy=0x7f941404f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9414006160 con 0x7f9414055790 2024-05-28T13:58:29.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f941c075df0 con 0x7f9414055790 2024-05-28T13:58:29.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 >> v1:172.21.15.63:6789/0 conn(0x7f9414055790 legacy=0x7f94140df120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f02a0200640 1 -- start start 2024-05-28T13:58:29.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f42e9800640 1 Processor -- start 2024-05-28T13:58:29.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 -- 172.21.15.42:0/935611267 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3255506034 0 0) 0x7f16f4003530 con 0x7f17100aee80 2024-05-28T13:58:29.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 -- 172.21.15.42:0/935611267 >> v2:172.21.15.63:3300/0 conn(0x7f17100ae390 msgr2=0x7f171006f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941be00640 1 --2- 172.21.15.42:0/2839464117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9400003b50 0x7f9400003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f42e9800640 1 -- start start 2024-05-28T13:58:29.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 --2- 172.21.15.42:0/935611267 >> v2:172.21.15.63:3300/0 conn(0x7f17100ae390 0x7f171006f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f472b800640 1 -- start start 2024-05-28T13:58:29.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941be00640 1 --2- 172.21.15.42:0/2839464117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9400004810 0x7f9400004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941be00640 1 --2- 172.21.15.42:0/2839464117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94000051d0 0x7f9400010060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9414054400 con 0x7f9400003b50 2024-05-28T13:58:29.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f94000045c0 con 0x7f94000051d0 2024-05-28T13:58:29.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f94000105a0 con 0x7f9400004810 2024-05-28T13:58:29.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f9419600640 1 --2- 172.21.15.42:0/2839464117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9414053d00 0x7f9414006f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:29.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941a000640 1 --2- 172.21.15.42:0/2839464117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9400003b50 0x7f9400003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 -- 172.21.15.42:0/935611267 >> v1:172.21.15.63:6789/0 conn(0x7f171015a920 legacy=0x7f17100ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 -- 172.21.15.42:0/935611267 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17100b3430 con 0x7f17100aee80 2024-05-28T13:58:29.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f9419600640 1 --2- 172.21.15.42:0/2839464117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9400004810 0x7f9400004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941aa00640 1 --2- 172.21.15.42:0/2839464117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94000051d0 0x7f9400010060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f82ac000640 1 Processor -- start 2024-05-28T13:58:29.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f02a0200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0298145920 0x7f029806f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f02a0200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f029806db40 0x7f0298071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f02a0200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f029806dfa0 con 0x7f0298072300 2024-05-28T13:58:29.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f02a0200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0298072750 con 0x7f029806db40 2024-05-28T13:58:29.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f02a0200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f029806e240 con 0x7f0298145920 2024-05-28T13:58:29.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 -- 172.21.15.42:0/935611267 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f1708001a00 con 0x7f17100aee80 2024-05-28T13:58:29.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 -- 172.21.15.42:0/935611267 >> v1:172.21.15.42:6789/0 conn(0x7f17100aee80 legacy=0x7f17100b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 --2- 172.21.15.42:0/935611267 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16f40039c0 0x7f16f4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 --2- 172.21.15.42:0/935611267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4004810 0x7f16f4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 --2- 172.21.15.42:0/935611267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16f4005240 0x7f16f400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 -- 172.21.15.42:0/935611267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f17100b31e0 con 0x7f16f4005240 2024-05-28T13:58:29.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 -- 172.21.15.42:0/935611267 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f16f400c1a0 con 0x7f16f4004810 2024-05-28T13:58:29.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f170f400640 1 -- 172.21.15.42:0/935611267 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f16f400c3a0 con 0x7f16f40039c0 2024-05-28T13:58:29.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f1715000640 1 --2- 172.21.15.42:0/935611267 >> v2:172.21.15.63:3300/0 conn(0x7f17100ae390 0x7f171006f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:29.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f9419600640 1 -- 172.21.15.42:0/2839464117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94000051d0 msgr2=0x7f9400010060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f9419600640 1 --2- 172.21.15.42:0/2839464117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94000051d0 0x7f9400010060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f9419600640 1 -- 172.21.15.42:0/2839464117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9400003b50 msgr2=0x7f9400003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f9419600640 1 --2- 172.21.15.42:0/2839464117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9400003b50 0x7f9400003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f9419600640 1 -- 172.21.15.42:0/2839464117 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9400003530 con 0x7f9400004810 2024-05-28T13:58:29.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f941a000640 1 --2- 172.21.15.42:0/2839464117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9400003b50 0x7f9400003f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:29.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.945+0000 7f42e9800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42e40cc170 0x7f42e4155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f170fe00640 1 --2- 172.21.15.42:0/935611267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4004810 0x7f16f4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f1715000640 1 --2- 172.21.15.42:0/935611267 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16f40039c0 0x7f16f4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42e4162490 0x7f42e4159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e4162e30 0x7f42e40c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e9800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f42e40cc550 con 0x7f42e4162490 2024-05-28T13:58:29.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e9800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f42e40ca7e0 con 0x7f42e4162e30 2024-05-28T13:58:29.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e9800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f42e40cc6d0 con 0x7f42e40cc170 2024-05-28T13:58:29.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f1715a00640 1 --2- 172.21.15.42:0/935611267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16f4005240 0x7f16f400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e2a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42e40cc170 0x7f42e4155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e2a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42e40cc170 0x7f42e4155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42878/0 (socket says 172.21.15.42:42878) 2024-05-28T13:58:29.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f029d800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0298145920 0x7f029806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f029d800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0298145920 0x7f029806f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42854/0 (socket says 172.21.15.42:42854) 2024-05-28T13:58:29.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f029d800640 1 -- 172.21.15.42:0/2010445818 learned_addr learned my addr 172.21.15.42:0/2010445818 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f029ce00640 1 --2- 172.21.15.42:0/2010445818 >> v2:172.21.15.63:3300/0 conn(0x7f029806db40 0x7f0298071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f82ac000640 1 -- start start 2024-05-28T13:58:29.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e2a00640 1 -- 172.21.15.42:0/2838532329 learned_addr learned my addr 172.21.15.42:0/2838532329 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e2000640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42e4162490 0x7f42e4159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f029d800640 1 -- 172.21.15.42:0/2010445818 >> v2:172.21.15.63:3300/0 conn(0x7f029806db40 msgr2=0x7f0298071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f029d800640 1 --2- 172.21.15.42:0/2010445818 >> v2:172.21.15.63:3300/0 conn(0x7f029806db40 0x7f0298071dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f472b800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f472406f890 0x7f47240ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f170fe00640 1 -- 172.21.15.42:0/935611267 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16f40039c0 msgr2=0x7f16f4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f170fe00640 1 --2- 172.21.15.42:0/935611267 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16f40039c0 0x7f16f4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f170fe00640 1 -- 172.21.15.42:0/935611267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16f4005240 msgr2=0x7f16f400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f170fe00640 1 --2- 172.21.15.42:0/935611267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16f4005240 0x7f16f400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f170fe00640 1 -- 172.21.15.42:0/935611267 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f4003530 con 0x7f16f4004810 2024-05-28T13:58:29.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e2a00640 1 -- 172.21.15.42:0/2838532329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e4162e30 msgr2=0x7f42e40c89d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T13:58:29.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f1715a00640 1 --2- 172.21.15.42:0/935611267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16f4005240 0x7f16f400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:29.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f1715a00640 1 -- 172.21.15.42:0/935611267 reap_dead start 2024-05-28T13:58:29.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f472b800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4724166960 con 0x7f4724162c80 2024-05-28T13:58:29.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f472b800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4724166c40 con 0x7f4724071fe0 2024-05-28T13:58:29.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f472b800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4724166730 con 0x7f472406f890 2024-05-28T13:58:29.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e3400640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e4162e30 0x7f42e40c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f4728e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f4724162c80 0x7f472406f180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60538/0 (socket says 172.21.15.42:60538) 2024-05-28T13:58:29.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f4723e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f472406f890 0x7f47240ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e2a00640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e4162e30 0x7f42e40c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e2a00640 1 -- 172.21.15.42:0/2838532329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42e4162490 msgr2=0x7f42e4159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e2a00640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42e4162490 0x7f42e4159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e2a00640 1 -- 172.21.15.42:0/2838532329 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e40c9340 con 0x7f42e40cc170 2024-05-28T13:58:29.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e3400640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e4162e30 0x7f42e40c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:29.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f4723e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f472406f890 0x7f47240ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38038/0 (socket says 172.21.15.42:38038) 2024-05-28T13:58:29.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f4723e00640 1 -- 172.21.15.42:0/4286087822 learned_addr learned my addr 172.21.15.42:0/4286087822 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f42e2a00640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42e40cc170 0x7f42e4155bb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f42d400d830 tx=0x7f42d400dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f4723e00640 1 -- 172.21.15.42:0/4286087822 >> v1:172.21.15.63:6789/0 conn(0x7f4724071fe0 legacy=0x7f472406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f4723e00640 1 -- 172.21.15.42:0/4286087822 >> v1:172.21.15.42:6789/0 conn(0x7f4724162c80 legacy=0x7f472406f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f4723e00640 1 -- 172.21.15.42:0/4286087822 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4724166f20 con 0x7f472406f890 2024-05-28T13:58:29.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f029d800640 1 -- 172.21.15.42:0/2010445818 >> v1:172.21.15.42:6789/0 conn(0x7f0298072300 legacy=0x7f0298160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f029d800640 1 -- 172.21.15.42:0/2010445818 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029806e520 con 0x7f0298145920 2024-05-28T13:58:29.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f82ac000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f82a4162c80 0x7f82a40ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f82ac000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40726f0 0x7f82a406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f82ac000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82a4166960 con 0x7f82a406f180 2024-05-28T13:58:29.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f82ac000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f82a4166730 con 0x7f82a4162c80 2024-05-28T13:58:29.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f82ac000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82a406fb20 con 0x7f82a40726f0 2024-05-28T13:58:29.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f82aa000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40726f0 0x7f82a406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f82aa000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40726f0 0x7f82a406f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42882/0 (socket says 172.21.15.42:42882) 2024-05-28T13:58:29.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.949+0000 7f82aa000640 1 -- 172.21.15.42:0/1916290795 learned_addr learned my addr 172.21.15.42:0/1916290795 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f82a9600640 1 --2- 172.21.15.42:0/1916290795 >> v2:172.21.15.63:3300/0 conn(0x7f82a4162c80 0x7f82a40ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f82aa000640 1 -- 172.21.15.42:0/1916290795 >> v2:172.21.15.63:3300/0 conn(0x7f82a4162c80 msgr2=0x7f82a40ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:29.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f82aa000640 1 --2- 172.21.15.42:0/1916290795 >> v2:172.21.15.63:3300/0 conn(0x7f82a4162c80 0x7f82a40ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f82aa000640 1 -- 172.21.15.42:0/1916290795 >> v1:172.21.15.42:6789/0 conn(0x7f82a406f180 legacy=0x7f82a4071fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:58:29.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f82aa000640 1 -- 172.21.15.42:0/1916290795 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a4166c40 con 0x7f82a40726f0 2024-05-28T13:58:29.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f9419600640 1 --2- 172.21.15.42:0/2839464117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9400004810 0x7f9400004c90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f941404f4c0 tx=0x7f940c01cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ab8fd264305f8d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f42daa00640 1 -- 172.21.15.42:0/2838532329 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4018030 con 0x7f42e40cc170 2024-05-28T13:58:29.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f42daa00640 1 -- 172.21.15.42:0/2838532329 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d400be10 con 0x7f42e40cc170 2024-05-28T13:58:29.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f42daa00640 1 -- 172.21.15.42:0/2838532329 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d40098b0 con 0x7f42e40cc170 2024-05-28T13:58:29.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f42e9800640 1 -- 172.21.15.42:0/2838532329 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e40c9620 con 0x7f42e40cc170 2024-05-28T13:58:29.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f42e9800640 1 -- 172.21.15.42:0/2838532329 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e415a130 con 0x7f42e40cc170 2024-05-28T13:58:29.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c00b040 con 0x7f9400004810 2024-05-28T13:58:29.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f940c0a5bd0 con 0x7f9400004810 2024-05-28T13:58:29.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f941be00640 1 -- 172.21.15.42:0/2839464117 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c004030 con 0x7f9400004810 2024-05-28T13:58:29.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f029d800640 1 --2- 172.21.15.42:0/2010445818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0298145920 0x7f029806f030 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f028c009870 tx=0x7f028c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d72d8f1683802d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f42e9800640 1 -- 172.21.15.42:0/2838532329 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e404fcf0 con 0x7f42e40cc170 2024-05-28T13:58:29.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f170fe00640 1 --2- 172.21.15.42:0/935611267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4004810 0x7f16f4004c90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f16fc001860 tx=0x7f16fc01e6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6878cae802c11278 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f0297e00640 1 -- 172.21.15.42:0/2010445818 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c0b0070 con 0x7f0298145920 2024-05-28T13:58:29.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f170f400640 1 -- 172.21.15.42:0/935611267 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc001460 con 0x7f16f4004810 2024-05-28T13:58:29.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f170f400640 1 -- 172.21.15.42:0/935611267 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16fc0aa640 con 0x7f16f4004810 2024-05-28T13:58:29.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.953+0000 7f170f400640 1 -- 172.21.15.42:0/935611267 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0aaa70 con 0x7f16f4004810 2024-05-28T13:58:29.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f4723e00640 1 --2- 172.21.15.42:0/4286087822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f472406f890 0x7f47240ce0c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4714009870 tx=0x7f471401c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bf00460059f141e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f1716400640 1 -- 172.21.15.42:0/935611267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4004810 msgr2=0x7f16f4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f1716400640 1 --2- 172.21.15.42:0/935611267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4004810 0x7f16f4004c90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f16fc001860 tx=0x7f16fc01e6f0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f9421a00640 1 -- 172.21.15.42:0/2839464117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9400004810 msgr2=0x7f9400004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f9421a00640 1 --2- 172.21.15.42:0/2839464117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9400004810 0x7f9400004c90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f941404f4c0 tx=0x7f940c01cc50 comp rx=0 tx=0).stop 2024-05-28T13:58:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f4723400640 1 -- 172.21.15.42:0/4286087822 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47140b0070 con 0x7f472406f890 2024-05-28T13:58:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f4723400640 1 -- 172.21.15.42:0/4286087822 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47140ab600 con 0x7f472406f890 2024-05-28T13:58:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f4723400640 1 -- 172.21.15.42:0/4286087822 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47140ab9d0 con 0x7f472406f890 2024-05-28T13:58:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f42daa00640 1 -- 172.21.15.42:0/2838532329 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f42d401b070 con 0x7f42e40cc170 2024-05-28T13:58:29.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f1716400640 1 -- 172.21.15.42:0/935611267 shutdown_connections 2024-05-28T13:58:29.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f1716400640 1 --2- 172.21.15.42:0/935611267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16f4004810 0x7f16f4004c90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f1716400640 1 -- 172.21.15.42:0/935611267 >> 172.21.15.42:0/935611267 conn(0x7f17100b1ee0 msgr2=0x7f17100b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f1716400640 1 -- 172.21.15.42:0/935611267 shutdown_connections 2024-05-28T13:58:29.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f82aa000640 1 --2- 172.21.15.42:0/1916290795 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40726f0 0x7f82a406f5e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8290009870 tx=0x7f829001c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c798abc25209494d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f472b800640 1 -- 172.21.15.42:0/4286087822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f472406f890 msgr2=0x7f47240ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f472b800640 1 --2- 172.21.15.42:0/4286087822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f472406f890 0x7f47240ce0c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4714009870 tx=0x7f471401c520 comp rx=0 tx=0).stop 2024-05-28T13:58:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f472b800640 1 -- 172.21.15.42:0/4286087822 shutdown_connections 2024-05-28T13:58:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f472b800640 1 --2- 172.21.15.42:0/4286087822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f472406f890 0x7f47240ce0c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f472b800640 1 -- 172.21.15.42:0/4286087822 >> 172.21.15.42:0/4286087822 conn(0x7f472406b9f0 msgr2=0x7f472406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f1716400640 1 -- 172.21.15.42:0/935611267 wait complete. 2024-05-28T13:58:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f42daa00640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f42c40683d0 0x7f42c406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f42daa00640 1 -- 172.21.15.42:0/2838532329 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f42d40873a0 con 0x7f42e40cc170 2024-05-28T13:58:29.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f472b800640 1 -- 172.21.15.42:0/4286087822 shutdown_connections 2024-05-28T13:58:29.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f941aa00640 1 -- 172.21.15.42:0/2839464117 reap_dead start 2024-05-28T13:58:29.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f472b800640 1 -- 172.21.15.42:0/4286087822 wait complete. 2024-05-28T13:58:29.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f9421a00640 1 -- 172.21.15.42:0/2839464117 shutdown_connections 2024-05-28T13:58:29.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f9421a00640 1 -- 172.21.15.42:0/2839464117 >> 172.21.15.42:0/2839464117 conn(0x7f9414054ca0 msgr2=0x7f9414055080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f42e2000640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f42c40683d0 0x7f42c406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f9421a00640 1 -- 172.21.15.42:0/2839464117 shutdown_connections 2024-05-28T13:58:29.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f9421a00640 1 -- 172.21.15.42:0/2839464117 wait complete. 2024-05-28T13:58:29.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.957+0000 7f472b800640 1 Processor -- start 2024-05-28T13:58:29.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f472b800640 1 -- start start 2024-05-28T13:58:29.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f1716400640 1 Processor -- start 2024-05-28T13:58:29.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f0297e00640 1 -- 172.21.15.42:0/2010445818 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028c0a4e50 con 0x7f0298145920 2024-05-28T13:58:29.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f0297e00640 1 -- 172.21.15.42:0/2010445818 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c0ab620 con 0x7f0298145920 2024-05-28T13:58:29.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f42e2000640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f42c40683d0 0x7f42c406a890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f42dc000bb0 tx=0x7f42dc00b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f1716400640 1 -- start start 2024-05-28T13:58:29.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f472b800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f472406f890 0x7f4724145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f472b800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4724071fe0 0x7f47241565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f472b800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4724162c80 0x7f4724156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f472b800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4724170c60 con 0x7f4724071fe0 2024-05-28T13:58:29.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f472b800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4724170ae0 con 0x7f472406f890 2024-05-28T13:58:29.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f472b800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4724170de0 con 0x7f4724162c80 2024-05-28T13:58:29.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f82a3e00640 1 -- 172.21.15.42:0/1916290795 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82900b0070 con 0x7f82a40726f0 2024-05-28T13:58:29.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f82a3e00640 1 -- 172.21.15.42:0/1916290795 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82900ab600 con 0x7f82a40726f0 2024-05-28T13:58:29.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f82a3e00640 1 -- 172.21.15.42:0/1916290795 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82900ab9d0 con 0x7f82a40726f0 2024-05-28T13:58:29.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f02a0200640 1 -- 172.21.15.42:0/2010445818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0298145920 msgr2=0x7f029806f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f02a0200640 1 --2- 172.21.15.42:0/2010445818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0298145920 0x7f029806f030 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f028c009870 tx=0x7f028c01c4d0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f9421a00640 1 Processor -- start 2024-05-28T13:58:29.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f82ac000640 1 -- 172.21.15.42:0/1916290795 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40726f0 msgr2=0x7f82a406f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f82ac000640 1 --2- 172.21.15.42:0/1916290795 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40726f0 0x7f82a406f5e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8290009870 tx=0x7f829001c520 comp rx=0 tx=0).stop 2024-05-28T13:58:29.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f82ac000640 1 -- 172.21.15.42:0/1916290795 shutdown_connections 2024-05-28T13:58:29.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f82ac000640 1 --2- 172.21.15.42:0/1916290795 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40726f0 0x7f82a406f5e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f82ac000640 1 --2- 172.21.15.42:0/1916290795 >> v2:172.21.15.63:3300/0 conn(0x7f82a4162c80 0x7f82a40ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f82ac000640 1 -- 172.21.15.42:0/1916290795 >> 172.21.15.42:0/1916290795 conn(0x7f82a406b9f0 msgr2=0x7f82a406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f82ac000640 1 -- 172.21.15.42:0/1916290795 shutdown_connections 2024-05-28T13:58:29.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f82ac000640 1 -- 172.21.15.42:0/1916290795 wait complete. 2024-05-28T13:58:29.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f02a0200640 1 -- 172.21.15.42:0/2010445818 shutdown_connections 2024-05-28T13:58:29.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f02a0200640 1 --2- 172.21.15.42:0/2010445818 >> v2:172.21.15.63:3300/0 conn(0x7f029806db40 0x7f0298071dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f02a0200640 1 --2- 172.21.15.42:0/2010445818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0298145920 0x7f029806f030 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f02a0200640 1 -- 172.21.15.42:0/2010445818 >> 172.21.15.42:0/2010445818 conn(0x7f029806b9f0 msgr2=0x7f029806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:29.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f42daa00640 1 -- 172.21.15.42:0/2838532329 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f42d4026e90 con 0x7f42e40cc170 2024-05-28T13:58:29.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f02a0200640 1 -- 172.21.15.42:0/2010445818 shutdown_connections 2024-05-28T13:58:29.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f1716400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16f4004810 0x7f17101fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f1716400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17101506d0 0x7f17101f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f1716400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1710152ff0 0x7f1710153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f9421a00640 1 -- start start 2024-05-28T13:58:29.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f1715a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1710152ff0 0x7f1710153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f1715a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1710152ff0 0x7f1710153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38592/0 (socket says 172.21.15.42:38592) 2024-05-28T13:58:29.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f170fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17101506d0 0x7f17101f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f170fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17101506d0 0x7f17101f75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38054/0 (socket says 172.21.15.42:38054) 2024-05-28T13:58:29.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f1715a00640 1 -- 172.21.15.42:0/729344758 learned_addr learned my addr 172.21.15.42:0/729344758 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f1716400640 1 -- 172.21.15.42:0/729344758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f17100aeb00 con 0x7f1710152ff0 2024-05-28T13:58:29.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f1715000640 1 --2- 172.21.15.42:0/729344758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16f4004810 0x7f17101fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f1716400640 1 -- 172.21.15.42:0/729344758 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17100ae980 con 0x7f17101506d0 2024-05-28T13:58:29.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f1716400640 1 -- 172.21.15.42:0/729344758 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f17100aec80 con 0x7f16f4004810 2024-05-28T13:58:29.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f82ac000640 1 Processor -- start 2024-05-28T13:58:29.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f02a0200640 1 -- 172.21.15.42:0/2010445818 wait complete. 2024-05-28T13:58:29.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f4729800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4724162c80 0x7f4724156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f4728e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f472406f890 0x7f4724145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f4723e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4724071fe0 0x7f47241565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f02a0200640 1 Processor -- start 2024-05-28T13:58:29.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f4723e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4724071fe0 0x7f47241565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38586/0 (socket says 172.21.15.42:38586) 2024-05-28T13:58:29.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f4723e00640 1 -- 172.21.15.42:0/2930109139 learned_addr learned my addr 172.21.15.42:0/2930109139 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f82ac000640 1 -- start start 2024-05-28T13:58:29.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f4728e00640 1 -- 172.21.15.42:0/2930109139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4724162c80 msgr2=0x7f4724156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f4728e00640 1 --2- 172.21.15.42:0/2930109139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4724162c80 0x7f4724156b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f4728e00640 1 -- 172.21.15.42:0/2930109139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4724071fe0 msgr2=0x7f47241565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f4728e00640 1 --2- 172.21.15.42:0/2930109139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4724071fe0 0x7f47241565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f4728e00640 1 -- 172.21.15.42:0/2930109139 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47241530d0 con 0x7f472406f890 2024-05-28T13:58:29.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.961+0000 7f4728e00640 1 --2- 172.21.15.42:0/2930109139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f472406f890 0x7f4724145c20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f471c00c6c0 tx=0x7f471c00cb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f4720c00640 1 -- 172.21.15.42:0/2930109139 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c018070 con 0x7f472406f890 2024-05-28T13:58:29.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f4720c00640 1 -- 172.21.15.42:0/2930109139 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f471c004d70 con 0x7f472406f890 2024-05-28T13:58:29.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f4720c00640 1 -- 172.21.15.42:0/2930109139 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c00a840 con 0x7f472406f890 2024-05-28T13:58:29.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f472b800640 1 -- 172.21.15.42:0/2930109139 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47241533b0 con 0x7f472406f890 2024-05-28T13:58:29.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f472b800640 1 -- 172.21.15.42:0/2930109139 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4724153970 con 0x7f472406f890 2024-05-28T13:58:29.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f1715000640 1 -- 172.21.15.42:0/729344758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17101506d0 msgr2=0x7f17101f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f1715000640 1 --2- 172.21.15.42:0/729344758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17101506d0 0x7f17101f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f1715000640 1 -- 172.21.15.42:0/729344758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1710152ff0 msgr2=0x7f1710153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f1715000640 1 --2- 172.21.15.42:0/729344758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1710152ff0 0x7f1710153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f1715000640 1 -- 172.21.15.42:0/729344758 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171015b0e0 con 0x7f16f4004810 2024-05-28T13:58:29.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f1715000640 1 --2- 172.21.15.42:0/729344758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16f4004810 0x7f17101fbcb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1700011680 tx=0x7f1700011b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f9421a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94141859a0 0x7f941404dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f170cc00640 1 -- 172.21.15.42:0/729344758 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700019020 con 0x7f16f4004810 2024-05-28T13:58:29.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f170cc00640 1 -- 172.21.15.42:0/729344758 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1700004440 con 0x7f16f4004810 2024-05-28T13:58:29.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f170cc00640 1 -- 172.21.15.42:0/729344758 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170000c070 con 0x7f16f4004810 2024-05-28T13:58:29.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f4720c00640 1 -- 172.21.15.42:0/2930109139 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f471c00a9e0 con 0x7f472406f890 2024-05-28T13:58:29.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.965+0000 7f9421a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f941404e000 0x7f941404be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f9421a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f941404c360 0x7f94140d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f9421a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9414006a00 con 0x7f941404e000 2024-05-28T13:58:29.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f9421a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9414006720 con 0x7f94141859a0 2024-05-28T13:58:29.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f9421a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f94140078f0 con 0x7f941404c360 2024-05-28T13:58:29.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82ac000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82a406f180 0x7f82a415e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82ac000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40726f0 0x7f82a41588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82ac000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a4162c80 0x7f82a41597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82ac000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f82a406e870 con 0x7f82a406f180 2024-05-28T13:58:29.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82ac000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82a406e6f0 con 0x7f82a4162c80 2024-05-28T13:58:29.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82ac000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82a406e9f0 con 0x7f82a40726f0 2024-05-28T13:58:29.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f472b800640 1 -- 172.21.15.42:0/2930109139 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4724073890 con 0x7f472406f890 2024-05-28T13:58:29.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f941a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94141859a0 0x7f941404dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82aa000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a4162c80 0x7f82a41597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82a8c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40726f0 0x7f82a41588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82a8c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40726f0 0x7f82a41588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42910/0 (socket says 172.21.15.42:42910) 2024-05-28T13:58:29.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82a8c00640 1 -- 172.21.15.42:0/2810058391 learned_addr learned my addr 172.21.15.42:0/2810058391 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f941a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94141859a0 0x7f941404dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38068/0 (socket says 172.21.15.42:38068) 2024-05-28T13:58:29.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f941a000640 1 -- 172.21.15.42:0/973644588 learned_addr learned my addr 172.21.15.42:0/973644588 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f02a0200640 1 -- start start 2024-05-28T13:58:29.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82aa000640 1 -- 172.21.15.42:0/2810058391 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40726f0 msgr2=0x7f82a41588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82aa000640 1 --2- 172.21.15.42:0/2810058391 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40726f0 0x7f82a41588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82aa000640 1 -- 172.21.15.42:0/2810058391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82a406f180 msgr2=0x7f82a415e4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-28T13:58:29.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82aa000640 1 --2- 172.21.15.42:0/2810058391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82a406f180 0x7f82a415e4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82aa000640 1 -- 172.21.15.42:0/2810058391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a4153cf0 con 0x7f82a4162c80 2024-05-28T13:58:29.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f4720c00640 1 --2- 172.21.15.42:0/2930109139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4704068380 0x7f470406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f4720c00640 1 -- 172.21.15.42:0/2930109139 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f471c085e20 con 0x7f472406f890 2024-05-28T13:58:29.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f941a000640 1 -- 172.21.15.42:0/973644588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f941404c360 msgr2=0x7f94140d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f941aa00640 1 --2- 172.21.15.42:0/973644588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f941404c360 0x7f94140d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f941a000640 1 --2- 172.21.15.42:0/973644588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f941404c360 0x7f94140d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f941a000640 1 -- 172.21.15.42:0/973644588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f941404e000 msgr2=0x7f941404be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f941a000640 1 --2- 172.21.15.42:0/973644588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f941404e000 0x7f941404be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f82aa000640 1 --2- 172.21.15.42:0/2810058391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a4162c80 0x7f82a41597f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82a40cdeb0 tx=0x7f82900192c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f941a000640 1 -- 172.21.15.42:0/973644588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941404cb80 con 0x7f94141859a0 2024-05-28T13:58:29.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f941aa00640 1 --2- 172.21.15.42:0/973644588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f941404c360 0x7f94140d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:29.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f4723e00640 1 --2- 172.21.15.42:0/2930109139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4704068380 0x7f470406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f941a000640 1 --2- 172.21.15.42:0/973644588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94141859a0 0x7f941404dac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f941c076bd0 tx=0x7f941c076010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f02a0200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f029806db40 0x7f029813aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f1716400640 1 -- 172.21.15.42:0/729344758 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171015b3c0 con 0x7f16f4004810 2024-05-28T13:58:29.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f1716400640 1 -- 172.21.15.42:0/729344758 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f171015b980 con 0x7f16f4004810 2024-05-28T13:58:29.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f02a0200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298072300 0x7f029813d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f02a0200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0298145920 0x7f0298165680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f4723e00640 1 --2- 172.21.15.42:0/2930109139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4704068380 0x7f470406a840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4714000c10 tx=0x7f47140a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f02a0200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f02981673c0 con 0x7f0298145920 2024-05-28T13:58:29.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f02a0200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0298167240 con 0x7f0298072300 2024-05-28T13:58:29.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.969+0000 7f02a0200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0298167540 con 0x7f029806db40 2024-05-28T13:58:29.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f029ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298072300 0x7f029813d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f029ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298072300 0x7f029813d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38088/0 (socket says 172.21.15.42:38088) 2024-05-28T13:58:29.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f029ce00640 1 -- 172.21.15.42:0/3289762854 learned_addr learned my addr 172.21.15.42:0/3289762854 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:29.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f029ce00640 1 -- 172.21.15.42:0/3289762854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f029806db40 msgr2=0x7f029813aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f029d800640 1 --2- 172.21.15.42:0/3289762854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f029806db40 0x7f029813aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f029ce00640 1 --2- 172.21.15.42:0/3289762854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f029806db40 0x7f029813aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f029ce00640 1 -- 172.21.15.42:0/3289762854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0298145920 msgr2=0x7f0298165680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f029ce00640 1 --2- 172.21.15.42:0/3289762854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0298145920 0x7f0298165680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:29.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f029ce00640 1 -- 172.21.15.42:0/3289762854 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029813e070 con 0x7f0298072300 2024-05-28T13:58:29.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f1716400640 1 -- 172.21.15.42:0/729344758 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171004fcf0 con 0x7f16f4004810 2024-05-28T13:58:29.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f029ce00640 1 --2- 172.21.15.42:0/3289762854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298072300 0x7f029813d420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f028400ba70 tx=0x7f028400bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f170cc00640 1 -- 172.21.15.42:0/729344758 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f170001c020 con 0x7f16f4004810 2024-05-28T13:58:29.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f170cc00640 1 --2- 172.21.15.42:0/729344758 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16f00683d0 0x7f16f006a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f82a1600640 1 -- 172.21.15.42:0/2810058391 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82900b0050 con 0x7f82a4162c80 2024-05-28T13:58:29.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f82a1600640 1 -- 172.21.15.42:0/2810058391 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82900ba070 con 0x7f82a4162c80 2024-05-28T13:58:29.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f82a1600640 1 -- 172.21.15.42:0/2810058391 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82900b47b0 con 0x7f82a4162c80 2024-05-28T13:58:29.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f170cc00640 1 -- 172.21.15.42:0/729344758 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f17000871b0 con 0x7f16f4004810 2024-05-28T13:58:29.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f9412a00640 1 -- 172.21.15.42:0/973644588 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c098070 con 0x7f94141859a0 2024-05-28T13:58:29.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.973+0000 7f9421a00640 1 -- 172.21.15.42:0/973644588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941404ce60 con 0x7f94141859a0 2024-05-28T13:58:29.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f9412a00640 1 -- 172.21.15.42:0/973644588 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f941c071d80 con 0x7f94141859a0 2024-05-28T13:58:29.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f170fe00640 1 --2- 172.21.15.42:0/729344758 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16f00683d0 0x7f16f006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f0295600640 1 -- 172.21.15.42:0/3289762854 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284018ce0 con 0x7f0298072300 2024-05-28T13:58:29.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f0295600640 1 -- 172.21.15.42:0/3289762854 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028400d280 con 0x7f0298072300 2024-05-28T13:58:29.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f0295600640 1 -- 172.21.15.42:0/3289762854 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028400f8f0 con 0x7f0298072300 2024-05-28T13:58:29.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f82ac000640 1 -- 172.21.15.42:0/2810058391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a41631b0 con 0x7f82a4162c80 2024-05-28T13:58:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f82ac000640 1 -- 172.21.15.42:0/2810058391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a41533f0 con 0x7f82a4162c80 2024-05-28T13:58:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f02a0200640 1 -- 172.21.15.42:0/3289762854 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029813e350 con 0x7f0298072300 2024-05-28T13:58:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f02a0200640 1 -- 172.21.15.42:0/3289762854 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f029813e910 con 0x7f0298072300 2024-05-28T13:58:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f4720c00640 1 -- 172.21.15.42:0/2930109139 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f471c055d80 con 0x7f472406f890 2024-05-28T13:58:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f9412a00640 1 -- 172.21.15.42:0/973644588 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c093650 con 0x7f94141859a0 2024-05-28T13:58:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f9421a00640 1 -- 172.21.15.42:0/973644588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f941404d420 con 0x7f94141859a0 2024-05-28T13:58:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f170fe00640 1 --2- 172.21.15.42:0/729344758 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16f00683d0 0x7f16f006a890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0b1000 tx=0x7f16fc0a9000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f02a0200640 1 -- 172.21.15.42:0/3289762854 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0298073890 con 0x7f0298072300 2024-05-28T13:58:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f9421a00640 1 -- 172.21.15.42:0/973644588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9414008bb0 con 0x7f94141859a0 2024-05-28T13:58:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f82ac000640 1 -- 172.21.15.42:0/2810058391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a4073890 con 0x7f82a4162c80 2024-05-28T13:58:29.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f9412a00640 1 -- 172.21.15.42:0/973644588 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f941c0937f0 con 0x7f94141859a0 2024-05-28T13:58:29.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.977+0000 7f170cc00640 1 -- 172.21.15.42:0/729344758 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1700023020 con 0x7f16f4004810 2024-05-28T13:58:29.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.981+0000 7f0295600640 1 -- 172.21.15.42:0/3289762854 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f028400fa90 con 0x7f0298072300 2024-05-28T13:58:29.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.981+0000 7f82a1600640 1 -- 172.21.15.42:0/2810058391 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f82900b4950 con 0x7f82a4162c80 2024-05-28T13:58:29.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.981+0000 7f9412a00640 1 --2- 172.21.15.42:0/973644588 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f93fc068380 0x7f93fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.981+0000 7f9412a00640 1 -- 172.21.15.42:0/973644588 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f941c101e60 con 0x7f94141859a0 2024-05-28T13:58:29.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.981+0000 7f9419600640 1 --2- 172.21.15.42:0/973644588 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f93fc068380 0x7f93fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.981+0000 7f0295600640 1 --2- 172.21.15.42:0/3289762854 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0278068480 0x7f027806a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.981+0000 7f0295600640 1 -- 172.21.15.42:0/3289762854 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f028408ef00 con 0x7f0298072300 2024-05-28T13:58:29.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.981+0000 7f82a1600640 1 --2- 172.21.15.42:0/2810058391 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8284068450 0x7f828406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.981+0000 7f82a1600640 1 -- 172.21.15.42:0/2810058391 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f82901207e0 con 0x7f82a4162c80 2024-05-28T13:58:29.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.981+0000 7f82a9600640 1 --2- 172.21.15.42:0/2810058391 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8284068450 0x7f828406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.981+0000 7f029d800640 1 --2- 172.21.15.42:0/3289762854 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0278068480 0x7f027806a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.981+0000 7f82a9600640 1 --2- 172.21.15.42:0/2810058391 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8284068450 0x7f828406a910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8294007de0 tx=0x7f8294007310 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.981+0000 7f9419600640 1 --2- 172.21.15.42:0/973644588 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f93fc068380 0x7f93fc06a840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f941404f4c0 tx=0x7f940c004740 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.981+0000 7f029d800640 1 --2- 172.21.15.42:0/3289762854 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0278068480 0x7f027806a940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f028c0ad000 tx=0x7f028c0a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:29.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.985+0000 7f9412a00640 1 -- 172.21.15.42:0/973644588 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9414054180 con 0x7f94141859a0 2024-05-28T13:58:29.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.989+0000 7f0295600640 1 -- 172.21.15.42:0/3289762854 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f028405f4a0 con 0x7f0298072300 2024-05-28T13:58:29.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:29.997+0000 7f82a1600640 1 -- 172.21.15.42:0/2810058391 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8290016110 con 0x7f82a4162c80 2024-05-28T13:58:29.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.009+0000 7f4085600640 1 Processor -- start 2024-05-28T13:58:29.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.009+0000 7f4085600640 1 -- start start 2024-05-28T13:58:29.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.009+0000 7f4085600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40800ad0b0 0x7f40800b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:29.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.009+0000 7f4085600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40800b2ba0 con 0x7f408015dfd0 2024-05-28T13:58:29.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.009+0000 7f4085600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40800cf580 con 0x7f40800b1360 2024-05-28T13:58:29.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.009+0000 7f4085600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f40800b0110 con 0x7f40800ad0b0 2024-05-28T13:58:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.009+0000 7f407ea00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f408015dfd0 0x7f40800b0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60548/0 (socket says 172.21.15.42:60548) 2024-05-28T13:58:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.013+0000 7f407ea00640 1 -- 172.21.15.42:0/1957617966 learned_addr learned my addr 172.21.15.42:0/1957617966 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.013+0000 7f407e000640 1 --2- 172.21.15.42:0/1957617966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40800ad0b0 0x7f40800b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.013+0000 7f407d600640 1 -- 172.21.15.42:0/1957617966 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2103003130 0 0) 0x7f40800cf580 con 0x7f40800b1360 2024-05-28T13:58:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.013+0000 7f407d600640 1 -- 172.21.15.42:0/1957617966 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4064003530 con 0x7f40800b1360 2024-05-28T13:58:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.013+0000 7f407e000640 1 -- 172.21.15.42:0/1957617966 >> v1:172.21.15.63:6789/0 conn(0x7f40800b1360 legacy=0x7f40800730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.013+0000 7f407e000640 1 -- 172.21.15.42:0/1957617966 >> v1:172.21.15.42:6789/0 conn(0x7f408015dfd0 legacy=0x7f40800b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.013+0000 7f407e000640 1 -- 172.21.15.42:0/1957617966 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40800cf860 con 0x7f40800ad0b0 2024-05-28T13:58:30.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.017+0000 7f407e000640 1 --2- 172.21.15.42:0/1957617966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40800ad0b0 0x7f40800b24e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4078009dc0 tx=0x7f407801c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12b189a740c2d5bf server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:30.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.017+0000 7f407d600640 1 -- 172.21.15.42:0/1957617966 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40780b2070 con 0x7f40800ad0b0 2024-05-28T13:58:30.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.017+0000 7f407d600640 1 -- 172.21.15.42:0/1957617966 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40780a58f0 con 0x7f40800ad0b0 2024-05-28T13:58:30.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.017+0000 7f407d600640 1 -- 172.21.15.42:0/1957617966 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40780a5cc0 con 0x7f40800ad0b0 2024-05-28T13:58:30.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.017+0000 7f4085600640 1 -- 172.21.15.42:0/1957617966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40800ad0b0 msgr2=0x7f40800b24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.017+0000 7f4085600640 1 --2- 172.21.15.42:0/1957617966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40800ad0b0 0x7f40800b24e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4078009dc0 tx=0x7f407801c920 comp rx=0 tx=0).stop 2024-05-28T13:58:30.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.017+0000 7f4085600640 1 -- 172.21.15.42:0/1957617966 shutdown_connections 2024-05-28T13:58:30.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.017+0000 7f4085600640 1 --2- 172.21.15.42:0/1957617966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40800ad0b0 0x7f40800b24e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.017+0000 7f4085600640 1 -- 172.21.15.42:0/1957617966 >> 172.21.15.42:0/1957617966 conn(0x7f40800721f0 msgr2=0x7f40800b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:30.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.017+0000 7f4085600640 1 -- 172.21.15.42:0/1957617966 shutdown_connections 2024-05-28T13:58:30.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.021+0000 7f4085600640 1 -- 172.21.15.42:0/1957617966 wait complete. 2024-05-28T13:58:30.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.021+0000 7f4085600640 1 Processor -- start 2024-05-28T13:58:30.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.021+0000 7f4085600640 1 -- start start 2024-05-28T13:58:30.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f4085600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40800ad0b0 0x7f40800ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:30.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f3014a00640 1 Processor -- start 2024-05-28T13:58:30.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f407ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40800ad0b0 0x7f40800ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:30.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f407ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40800ad0b0 0x7f40800ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38610/0 (socket says 172.21.15.42:38610) 2024-05-28T13:58:30.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f407ea00640 1 -- 172.21.15.42:0/419851297 learned_addr learned my addr 172.21.15.42:0/419851297 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:30.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f4085600640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40800b1360 0x7f4080157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:30.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f4085600640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f408015dfd0 0x7f408015b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:30.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f4085600640 1 -- 172.21.15.42:0/419851297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4080168870 con 0x7f40800ad0b0 2024-05-28T13:58:30.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f4085600640 1 -- 172.21.15.42:0/419851297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f40801686f0 con 0x7f40800b1360 2024-05-28T13:58:30.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f4085600640 1 -- 172.21.15.42:0/419851297 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f40801689f0 con 0x7f408015dfd0 2024-05-28T13:58:30.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f407f400640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f408015dfd0 0x7f408015b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:30.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f407f400640 1 -- 172.21.15.42:0/419851297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40800b1360 msgr2=0x7f4080157200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T13:58:30.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f407f400640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40800b1360 0x7f4080157200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f407f400640 1 -- 172.21.15.42:0/419851297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40800ad0b0 msgr2=0x7f40800ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f407f400640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40800ad0b0 0x7f40800ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f407f400640 1 -- 172.21.15.42:0/419851297 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40800cd4e0 con 0x7f408015dfd0 2024-05-28T13:58:30.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f407f400640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f408015dfd0 0x7f408015b830 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f407000c3c0 tx=0x7f407000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:30.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f4076a00640 1 -- 172.21.15.42:0/419851297 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070017070 con 0x7f408015dfd0 2024-05-28T13:58:30.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f3014a00640 1 -- start start 2024-05-28T13:58:30.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f3014a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f30100b6670 0x7f3010165670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:30.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.029+0000 7f300f400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f3010162be0 0x7f30100d2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:50634/0 (socket says 172.21.15.42:50634) 2024-05-28T13:58:30.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.029+0000 7f300f400640 1 -- 172.21.15.42:0/2146944133 learned_addr learned my addr 172.21.15.42:0/2146944133 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:30.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.029+0000 7f300fe00640 1 --2- 172.21.15.42:0/2146944133 >> v2:172.21.15.63:3300/0 conn(0x7f30100b6670 0x7f3010165670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:30.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.029+0000 7f3014a00640 1 -- 172.21.15.42:0/2146944133 --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30101645d0 con 0x7f30100ceb30 2024-05-28T13:58:30.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.029+0000 7f3014a00640 1 -- 172.21.15.42:0/2146944133 --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30101648b0 con 0x7f3010162be0 2024-05-28T13:58:30.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.029+0000 7f3014a00640 1 -- 172.21.15.42:0/2146944133 --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f30100cefb0 con 0x7f30100b6670 2024-05-28T13:58:30.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f4085600640 1 -- 172.21.15.42:0/419851297 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40801557e0 con 0x7f408015dfd0 2024-05-28T13:58:30.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.029+0000 7f300fe00640 1 -- 172.21.15.42:0/2146944133 >> v1:172.21.15.63:6789/0 conn(0x7f3010162be0 legacy=0x7f30100d2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.029+0000 7f300fe00640 1 -- 172.21.15.42:0/2146944133 >> v1:172.21.15.42:6789/0 conn(0x7f30100ceb30 legacy=0x7f30100b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f4085600640 1 -- 172.21.15.42:0/419851297 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4080155da0 con 0x7f408015dfd0 2024-05-28T13:58:30.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.029+0000 7f300fe00640 1 -- 172.21.15.42:0/2146944133 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3010164b90 con 0x7f30100b6670 2024-05-28T13:58:30.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.025+0000 7f4076a00640 1 -- 172.21.15.42:0/419851297 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40700052d0 con 0x7f408015dfd0 2024-05-28T13:58:30.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.029+0000 7f4076a00640 1 -- 172.21.15.42:0/419851297 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070017070 con 0x7f408015dfd0 2024-05-28T13:58:30.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.033+0000 7f4085600640 1 -- 172.21.15.42:0/419851297 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40580052a0 con 0x7f408015dfd0 2024-05-28T13:58:30.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.033+0000 7f4076a00640 1 -- 172.21.15.42:0/419851297 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f407001a070 con 0x7f408015dfd0 2024-05-28T13:58:30.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.033+0000 7f4076a00640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f40600687b0 0x7f406006ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:30.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.033+0000 7f4076a00640 1 -- 172.21.15.42:0/419851297 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f40700050d0 con 0x7f408015dfd0 2024-05-28T13:58:30.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.033+0000 7f300fe00640 1 --2- 172.21.15.42:0/2146944133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30100b6670 0x7f3010165670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0097c0 tx=0x7f2ffc01c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=449ab532ff72c6b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:30.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.033+0000 7f300e000640 1 -- 172.21.15.42:0/2146944133 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0b0070 con 0x7f30100b6670 2024-05-28T13:58:30.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.033+0000 7f407ea00640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f40600687b0 0x7f406006ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:30.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.033+0000 7f300e000640 1 -- 172.21.15.42:0/2146944133 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ffc0ab600 con 0x7f30100b6670 2024-05-28T13:58:30.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.033+0000 7f300e000640 1 -- 172.21.15.42:0/2146944133 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0b0070 con 0x7f30100b6670 2024-05-28T13:58:30.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.037+0000 7f3014a00640 1 -- 172.21.15.42:0/2146944133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30100b6670 msgr2=0x7f3010165670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.037+0000 7f3014a00640 1 --2- 172.21.15.42:0/2146944133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30100b6670 0x7f3010165670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0097c0 tx=0x7f2ffc01c340 comp rx=0 tx=0).stop 2024-05-28T13:58:30.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.037+0000 7f3014a00640 1 -- 172.21.15.42:0/2146944133 shutdown_connections 2024-05-28T13:58:30.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.037+0000 7f3014a00640 1 --2- 172.21.15.42:0/2146944133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30100b6670 0x7f3010165670 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.037+0000 7f3014a00640 1 -- 172.21.15.42:0/2146944133 >> 172.21.15.42:0/2146944133 conn(0x7f301008bfe0 msgr2=0x7f30100d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:30.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.037+0000 7f407ea00640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f40600687b0 0x7f406006ac70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f406c005890 tx=0x7f406c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:30.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.037+0000 7f3014a00640 1 -- 172.21.15.42:0/2146944133 shutdown_connections 2024-05-28T13:58:30.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.037+0000 7f3014a00640 1 -- 172.21.15.42:0/2146944133 wait complete. 2024-05-28T13:58:30.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.037+0000 7f4076a00640 1 -- 172.21.15.42:0/419851297 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4070005480 con 0x7f408015dfd0 2024-05-28T13:58:30.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.041+0000 7f3014a00640 1 Processor -- start 2024-05-28T13:58:30.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.041+0000 7f3014a00640 1 -- start start 2024-05-28T13:58:30.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.045+0000 7f3014a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30100b6670 0x7f30101514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:30.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.045+0000 7f3014a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30100ceb30 0x7f30100ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:30.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.045+0000 7f300f400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30100b6670 0x7f30101514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:30.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.045+0000 7f3014a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3010162be0 0x7f30100cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:30.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.045+0000 7f3014a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f30100c74f0 con 0x7f30100ceb30 2024-05-28T13:58:30.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.045+0000 7f3014a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f30100c7370 con 0x7f3010162be0 2024-05-28T13:58:30.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.045+0000 7f3014a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f30100c7670 con 0x7f30100b6670 2024-05-28T13:58:30.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.045+0000 7f300fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3010162be0 0x7f30100cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:30.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.045+0000 7f300fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3010162be0 0x7f30100cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38112/0 (socket says 172.21.15.42:38112) 2024-05-28T13:58:30.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.045+0000 7f300fe00640 1 -- 172.21.15.42:0/3852903606 learned_addr learned my addr 172.21.15.42:0/3852903606 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:30.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.045+0000 7f300fe00640 1 -- 172.21.15.42:0/3852903606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30100b6670 msgr2=0x7f30101514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.049+0000 7f300ea00640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30100ceb30 0x7f30100ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:30.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.049+0000 7f300fe00640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30100b6670 0x7f30101514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.049+0000 7f300fe00640 1 -- 172.21.15.42:0/3852903606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30100ceb30 msgr2=0x7f30100ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.049+0000 7f300fe00640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30100ceb30 0x7f30100ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.049+0000 7f300fe00640 1 -- 172.21.15.42:0/3852903606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30101569c0 con 0x7f3010162be0 2024-05-28T13:58:30.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.049+0000 7f300f400640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30100b6670 0x7f30101514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:30.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.049+0000 7f300ea00640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30100ceb30 0x7f30100ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:30.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.053+0000 7f300fe00640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3010162be0 0x7f30100cd2b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc019f70 tx=0x7f2ffc0190e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:30.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.053+0000 7f3007400640 1 -- 172.21.15.42:0/3852903606 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0b9020 con 0x7f3010162be0 2024-05-28T13:58:30.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.053+0000 7f3007400640 1 -- 172.21.15.42:0/3852903606 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ffc0ab9b0 con 0x7f3010162be0 2024-05-28T13:58:30.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.053+0000 7f3007400640 1 -- 172.21.15.42:0/3852903606 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0b0040 con 0x7f3010162be0 2024-05-28T13:58:30.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.053+0000 7f3014a00640 1 -- 172.21.15.42:0/3852903606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30100ce070 con 0x7f3010162be0 2024-05-28T13:58:30.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.053+0000 7f3014a00640 1 -- 172.21.15.42:0/3852903606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3010156c80 con 0x7f3010162be0 2024-05-28T13:58:30.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.057+0000 7f3007400640 1 -- 172.21.15.42:0/3852903606 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2ffc0bc020 con 0x7f3010162be0 2024-05-28T13:58:30.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.057+0000 7f3014a00640 1 -- 172.21.15.42:0/3852903606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe80052a0 con 0x7f3010162be0 2024-05-28T13:58:30.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.057+0000 7f3007400640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2ff0068450 0x7f2ff006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:30.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.057+0000 7f3007400640 1 -- 172.21.15.42:0/3852903606 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2ffc120ab0 con 0x7f3010162be0 2024-05-28T13:58:30.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.061+0000 7f300f400640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2ff0068450 0x7f2ff006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:30.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.061+0000 7f300f400640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2ff0068450 0x7f2ff006a910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3008005c10 tx=0x7f3008009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:30.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.065+0000 7f3007400640 1 -- 172.21.15.42:0/3852903606 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2ffc0c3030 con 0x7f3010162be0 2024-05-28T13:58:30.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.317+0000 7f42e9800640 1 -- 172.21.15.42:0/2838532329 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f42e406fd20 con 0x7f42e40cc170 2024-05-28T13:58:30.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.317+0000 7f42daa00640 1 -- 172.21.15.42:0/2838532329 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f42d40579d0 con 0x7f42e40cc170 2024-05-28T13:58:30.307 INFO:teuthology.orchestra.run.smithi042.stdout:55834574852 2024-05-28T13:58:30.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.325+0000 7f42e9800640 1 -- 172.21.15.42:0/2838532329 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f42c40683d0 msgr2=0x7f42c406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.325+0000 7f42e9800640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f42c40683d0 0x7f42c406a890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f42dc000bb0 tx=0x7f42dc00b040 comp rx=0 tx=0).stop 2024-05-28T13:58:30.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.325+0000 7f42e9800640 1 -- 172.21.15.42:0/2838532329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42e40cc170 msgr2=0x7f42e4155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.325+0000 7f42e9800640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42e40cc170 0x7f42e4155bb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f42d400d830 tx=0x7f42d400dcf0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.325+0000 7f42e9800640 1 -- 172.21.15.42:0/2838532329 shutdown_connections 2024-05-28T13:58:30.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.325+0000 7f42e9800640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f42c40683d0 0x7f42c406a890 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.325+0000 7f42e9800640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e4162e30 0x7f42e40c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.325+0000 7f42e9800640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42e4162490 0x7f42e4159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.325+0000 7f42e9800640 1 --2- 172.21.15.42:0/2838532329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42e40cc170 0x7f42e4155bb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.325+0000 7f42e9800640 1 -- 172.21.15.42:0/2838532329 >> 172.21.15.42:0/2838532329 conn(0x7f42e4073010 msgr2=0x7f42e40ad480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:30.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.325+0000 7f42e9800640 1 -- 172.21.15.42:0/2838532329 shutdown_connections 2024-05-28T13:58:30.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.325+0000 7f42e9800640 1 -- 172.21.15.42:0/2838532329 wait complete. 2024-05-28T13:58:30.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.333+0000 7f9421a00640 1 -- 172.21.15.42:0/973644588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f94140054a0 con 0x7f94141859a0 2024-05-28T13:58:30.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.337+0000 7f1716400640 1 -- 172.21.15.42:0/729344758 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1710166030 con 0x7f16f4004810 2024-05-28T13:58:30.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.337+0000 7f9412a00640 1 -- 172.21.15.42:0/973644588 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f941c0d23a0 con 0x7f94141859a0 2024-05-28T13:58:30.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.337+0000 7f170cc00640 1 -- 172.21.15.42:0/729344758 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1700057200 con 0x7f16f4004810 2024-05-28T13:58:30.323 INFO:teuthology.orchestra.run.smithi042.stdout:55834574852 2024-05-28T13:58:30.323 INFO:teuthology.orchestra.run.smithi042.stdout:55834574853 2024-05-28T13:58:30.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f9421a00640 1 -- 172.21.15.42:0/973644588 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f93fc068380 msgr2=0x7f93fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f9421a00640 1 --2- 172.21.15.42:0/973644588 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f93fc068380 0x7f93fc06a840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f941404f4c0 tx=0x7f940c004740 comp rx=0 tx=0).stop 2024-05-28T13:58:30.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f9421a00640 1 -- 172.21.15.42:0/973644588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94141859a0 msgr2=0x7f941404dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f9421a00640 1 --2- 172.21.15.42:0/973644588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94141859a0 0x7f941404dac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f941c076bd0 tx=0x7f941c076010 comp rx=0 tx=0).stop 2024-05-28T13:58:30.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f9421a00640 1 -- 172.21.15.42:0/973644588 shutdown_connections 2024-05-28T13:58:30.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f9421a00640 1 --2- 172.21.15.42:0/973644588 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f93fc068380 0x7f93fc06a840 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f941404f4c0 tx=0x7f940c004740 comp rx=0 tx=0).stop 2024-05-28T13:58:30.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f9421a00640 1 --2- 172.21.15.42:0/973644588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f941404c360 0x7f94140d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f9421a00640 1 --2- 172.21.15.42:0/973644588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f941404e000 0x7f941404be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f9421a00640 1 --2- 172.21.15.42:0/973644588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94141859a0 0x7f941404dac0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f9421a00640 1 -- 172.21.15.42:0/973644588 >> 172.21.15.42:0/973644588 conn(0x7f9414054ca0 msgr2=0x7f94140d9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:30.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f9421a00640 1 -- 172.21.15.42:0/973644588 shutdown_connections 2024-05-28T13:58:30.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f9421a00640 1 -- 172.21.15.42:0/973644588 wait complete. 2024-05-28T13:58:30.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f1716400640 1 -- 172.21.15.42:0/729344758 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16f00683d0 msgr2=0x7f16f006a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f1716400640 1 --2- 172.21.15.42:0/729344758 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16f00683d0 0x7f16f006a890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0b1000 tx=0x7f16fc0a9000 comp rx=0 tx=0).stop 2024-05-28T13:58:30.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f1716400640 1 -- 172.21.15.42:0/729344758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16f4004810 msgr2=0x7f17101fbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f1716400640 1 --2- 172.21.15.42:0/729344758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16f4004810 0x7f17101fbcb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1700011680 tx=0x7f1700011b40 comp rx=0 tx=0).stop 2024-05-28T13:58:30.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f1716400640 1 -- 172.21.15.42:0/729344758 shutdown_connections 2024-05-28T13:58:30.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f1716400640 1 --2- 172.21.15.42:0/729344758 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16f00683d0 0x7f16f006a890 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f1716400640 1 --2- 172.21.15.42:0/729344758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1710152ff0 0x7f1710153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f1716400640 1 --2- 172.21.15.42:0/729344758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17101506d0 0x7f17101f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f1716400640 1 --2- 172.21.15.42:0/729344758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16f4004810 0x7f17101fbcb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.341+0000 7f1716400640 1 -- 172.21.15.42:0/729344758 >> 172.21.15.42:0/729344758 conn(0x7f17100b1ee0 msgr2=0x7f171015d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:30.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.345+0000 7f472b800640 1 -- 172.21.15.42:0/2930109139 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f472404fdd0 con 0x7f472406f890 2024-05-28T13:58:30.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.345+0000 7f1716400640 1 -- 172.21.15.42:0/729344758 shutdown_connections 2024-05-28T13:58:30.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.345+0000 7f1716400640 1 -- 172.21.15.42:0/729344758 wait complete. 2024-05-28T13:58:30.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.345+0000 7f82ac000640 1 -- 172.21.15.42:0/2810058391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f82a404fdd0 con 0x7f82a4162c80 2024-05-28T13:58:30.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.345+0000 7f4720c00640 1 -- 172.21.15.42:0/2930109139 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f471c05a080 con 0x7f472406f890 2024-05-28T13:58:30.334 INFO:teuthology.orchestra.run.smithi042.stdout:55834574852 2024-05-28T13:58:30.335 INFO:teuthology.orchestra.run.smithi042.stdout:55834574853 2024-05-28T13:58:30.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.345+0000 7f82a1600640 1 -- 172.21.15.42:0/2810058391 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f82900f0cf0 con 0x7f82a4162c80 2024-05-28T13:58:30.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f82ac000640 1 -- 172.21.15.42:0/2810058391 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8284068450 msgr2=0x7f828406a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f82ac000640 1 --2- 172.21.15.42:0/2810058391 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8284068450 0x7f828406a910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8294007de0 tx=0x7f8294007310 comp rx=0 tx=0).stop 2024-05-28T13:58:30.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f82ac000640 1 -- 172.21.15.42:0/2810058391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a4162c80 msgr2=0x7f82a41597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f82ac000640 1 --2- 172.21.15.42:0/2810058391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a4162c80 0x7f82a41597f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82a40cdeb0 tx=0x7f82900192c0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f02a0200640 1 -- 172.21.15.42:0/3289762854 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f029804fdd0 con 0x7f0298072300 2024-05-28T13:58:30.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f472b800640 1 -- 172.21.15.42:0/2930109139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4704068380 msgr2=0x7f470406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f472b800640 1 --2- 172.21.15.42:0/2930109139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4704068380 0x7f470406a840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4714000c10 tx=0x7f47140a6000 comp rx=0 tx=0).stop 2024-05-28T13:58:30.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f472b800640 1 -- 172.21.15.42:0/2930109139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f472406f890 msgr2=0x7f4724145c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f472b800640 1 --2- 172.21.15.42:0/2930109139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f472406f890 0x7f4724145c20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f471c00c6c0 tx=0x7f471c00cb80 comp rx=0 tx=0).stop 2024-05-28T13:58:30.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f82ac000640 1 -- 172.21.15.42:0/2810058391 shutdown_connections 2024-05-28T13:58:30.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f82ac000640 1 --2- 172.21.15.42:0/2810058391 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8284068450 0x7f828406a910 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f82ac000640 1 --2- 172.21.15.42:0/2810058391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a4162c80 0x7f82a41597f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f82ac000640 1 --2- 172.21.15.42:0/2810058391 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40726f0 0x7f82a41588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f82ac000640 1 --2- 172.21.15.42:0/2810058391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82a406f180 0x7f82a415e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f82ac000640 1 -- 172.21.15.42:0/2810058391 >> 172.21.15.42:0/2810058391 conn(0x7f82a406b9f0 msgr2=0x7f82a40ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:30.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f472b800640 1 -- 172.21.15.42:0/2930109139 shutdown_connections 2024-05-28T13:58:30.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f82ac000640 1 -- 172.21.15.42:0/2810058391 shutdown_connections 2024-05-28T13:58:30.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f82ac000640 1 -- 172.21.15.42:0/2810058391 wait complete. 2024-05-28T13:58:30.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f472b800640 1 --2- 172.21.15.42:0/2930109139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4704068380 0x7f470406a840 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f472b800640 1 --2- 172.21.15.42:0/2930109139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4724162c80 0x7f4724156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f472b800640 1 --2- 172.21.15.42:0/2930109139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4724071fe0 0x7f47241565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f472b800640 1 --2- 172.21.15.42:0/2930109139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f472406f890 0x7f4724145c20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f472b800640 1 -- 172.21.15.42:0/2930109139 >> 172.21.15.42:0/2930109139 conn(0x7f472406b9f0 msgr2=0x7f472406dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:30.339 INFO:teuthology.orchestra.run.smithi042.stdout:55834574853 2024-05-28T13:58:30.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f0295600640 1 -- 172.21.15.42:0/3289762854 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f028405eda0 con 0x7f0298072300 2024-05-28T13:58:30.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f472b800640 1 -- 172.21.15.42:0/2930109139 shutdown_connections 2024-05-28T13:58:30.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.349+0000 7f472b800640 1 -- 172.21.15.42:0/2930109139 wait complete. 2024-05-28T13:58:30.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.357+0000 7f02a0200640 1 -- 172.21.15.42:0/3289762854 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0278068480 msgr2=0x7f027806a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.357+0000 7f02a0200640 1 --2- 172.21.15.42:0/3289762854 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0278068480 0x7f027806a940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f028c0ad000 tx=0x7f028c0a6000 comp rx=0 tx=0).stop 2024-05-28T13:58:30.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.357+0000 7f02a0200640 1 -- 172.21.15.42:0/3289762854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298072300 msgr2=0x7f029813d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.357+0000 7f02a0200640 1 --2- 172.21.15.42:0/3289762854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298072300 0x7f029813d420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f028400ba70 tx=0x7f028400bf30 comp rx=0 tx=0).stop 2024-05-28T13:58:30.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.357+0000 7f02a0200640 1 -- 172.21.15.42:0/3289762854 shutdown_connections 2024-05-28T13:58:30.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.357+0000 7f02a0200640 1 --2- 172.21.15.42:0/3289762854 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0278068480 0x7f027806a940 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f028c0ad000 tx=0x7f028c0a6000 comp rx=0 tx=0).stop 2024-05-28T13:58:30.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.357+0000 7f02a0200640 1 --2- 172.21.15.42:0/3289762854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0298145920 0x7f0298165680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.357+0000 7f02a0200640 1 --2- 172.21.15.42:0/3289762854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298072300 0x7f029813d420 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f028400ba70 tx=0x7f028400bf30 comp rx=0 tx=0).stop 2024-05-28T13:58:30.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.357+0000 7f02a0200640 1 --2- 172.21.15.42:0/3289762854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f029806db40 0x7f029813aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.357+0000 7f02a0200640 1 -- 172.21.15.42:0/3289762854 >> 172.21.15.42:0/3289762854 conn(0x7f029806b9f0 msgr2=0x7f02980b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:30.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.1 2024-05-28T13:58:30.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.357+0000 7f02a0200640 1 -- 172.21.15.42:0/3289762854 shutdown_connections 2024-05-28T13:58:30.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.357+0000 7f02a0200640 1 -- 172.21.15.42:0/3289762854 wait complete. 2024-05-28T13:58:30.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.6 2024-05-28T13:58:30.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.389+0000 7f4085600640 1 -- 172.21.15.42:0/419851297 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4058005590 con 0x7f408015dfd0 2024-05-28T13:58:30.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.4 2024-05-28T13:58:30.374 INFO:teuthology.orchestra.run.smithi042.stdout:55834574854 2024-05-28T13:58:30.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.389+0000 7f4076a00640 1 -- 172.21.15.42:0/419851297 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4070055400 con 0x7f408015dfd0 2024-05-28T13:58:30.376 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.3 2024-05-28T13:58:30.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.389+0000 7f4085600640 1 -- 172.21.15.42:0/419851297 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f40600687b0 msgr2=0x7f406006ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.389+0000 7f4085600640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f40600687b0 0x7f406006ac70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f406c005890 tx=0x7f406c009040 comp rx=0 tx=0).stop 2024-05-28T13:58:30.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.389+0000 7f4085600640 1 -- 172.21.15.42:0/419851297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f408015dfd0 msgr2=0x7f408015b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.389+0000 7f4085600640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f408015dfd0 0x7f408015b830 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f407000c3c0 tx=0x7f407000c880 comp rx=0 tx=0).stop 2024-05-28T13:58:30.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.389+0000 7f4085600640 1 -- 172.21.15.42:0/419851297 shutdown_connections 2024-05-28T13:58:30.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.389+0000 7f4085600640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f40600687b0 0x7f406006ac70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.389+0000 7f4085600640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f408015dfd0 0x7f408015b830 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.389+0000 7f4085600640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40800b1360 0x7f4080157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.389+0000 7f4085600640 1 --2- 172.21.15.42:0/419851297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40800ad0b0 0x7f40800ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.389+0000 7f4085600640 1 -- 172.21.15.42:0/419851297 >> 172.21.15.42:0/419851297 conn(0x7f40800721f0 msgr2=0x7f40800ad490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:30.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.393+0000 7f4085600640 1 -- 172.21.15.42:0/419851297 shutdown_connections 2024-05-28T13:58:30.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.393+0000 7f4085600640 1 -- 172.21.15.42:0/419851297 wait complete. 2024-05-28T13:58:30.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.0 2024-05-28T13:58:30.381 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.2 2024-05-28T13:58:30.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.397+0000 7f3014a00640 1 -- 172.21.15.42:0/3852903606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2fe8005590 con 0x7f3010162be0 2024-05-28T13:58:30.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.397+0000 7f3007400640 1 -- 172.21.15.42:0/3852903606 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2ffc016050 con 0x7f3010162be0 2024-05-28T13:58:30.382 INFO:teuthology.orchestra.run.smithi042.stdout:55834574852 2024-05-28T13:58:30.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.397+0000 7f3014a00640 1 -- 172.21.15.42:0/3852903606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2ff0068450 msgr2=0x7f2ff006a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.397+0000 7f3014a00640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2ff0068450 0x7f2ff006a910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3008005c10 tx=0x7f3008009040 comp rx=0 tx=0).stop 2024-05-28T13:58:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.397+0000 7f3014a00640 1 -- 172.21.15.42:0/3852903606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3010162be0 msgr2=0x7f30100cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.397+0000 7f3014a00640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3010162be0 0x7f30100cd2b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc019f70 tx=0x7f2ffc0190e0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.397+0000 7f3014a00640 1 -- 172.21.15.42:0/3852903606 shutdown_connections 2024-05-28T13:58:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.397+0000 7f3014a00640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2ff0068450 0x7f2ff006a910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.397+0000 7f3014a00640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3010162be0 0x7f30100cd2b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.397+0000 7f3014a00640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30100ceb30 0x7f30100ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.397+0000 7f3014a00640 1 --2- 172.21.15.42:0/3852903606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30100b6670 0x7f30101514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.397+0000 7f3014a00640 1 -- 172.21.15.42:0/3852903606 >> 172.21.15.42:0/3852903606 conn(0x7f301008bfe0 msgr2=0x7f301015a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.401+0000 7f3014a00640 1 -- 172.21.15.42:0/3852903606 shutdown_connections 2024-05-28T13:58:30.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:30.401+0000 7f3014a00640 1 -- 172.21.15.42:0/3852903606 wait complete. 2024-05-28T13:58:30.397 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.5 2024-05-28T13:58:30.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.7 2024-05-28T13:58:31.345 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-28T13:58:31.362 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-28T13:58:31.374 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-28T13:58:31.377 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-28T13:58:31.379 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-28T13:58:31.382 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-28T13:58:31.399 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-28T13:58:31.406 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-28T13:58:31.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.577+0000 7fbf96a00640 1 Processor -- start 2024-05-28T13:58:31.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.577+0000 7fbf96a00640 1 -- start start 2024-05-28T13:58:31.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.581+0000 7fbf96a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9015dc30 0x7fbf900ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.581+0000 7fbf96a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbf9006f610 0x7fbf9015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.581+0000 7fbf96a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf9006de60 con 0x7fbf90073a90 2024-05-28T13:58:31.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.581+0000 7fbf96a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fbf9006e140 con 0x7fbf9006f610 2024-05-28T13:58:31.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.581+0000 7fbf96a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf9006e420 con 0x7fbf9015dc30 2024-05-28T13:58:31.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.581+0000 7fbf95600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9015dc30 0x7fbf900ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.581+0000 7fbf95600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9015dc30 0x7fbf900ad1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42958/0 (socket says 172.21.15.42:42958) 2024-05-28T13:58:31.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.581+0000 7fbf95600640 1 -- 172.21.15.42:0/2995685291 learned_addr learned my addr 172.21.15.42:0/2995685291 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:31.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.581+0000 7fbf96000640 1 --2- 172.21.15.42:0/2995685291 >> v2:172.21.15.42:3300/0 conn(0x7fbf9006f610 0x7fbf9015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.581+0000 7fbf95600640 1 -- 172.21.15.42:0/2995685291 >> v2:172.21.15.42:3300/0 conn(0x7fbf9006f610 msgr2=0x7fbf9015e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:31.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.581+0000 7fbf95600640 1 --2- 172.21.15.42:0/2995685291 >> v2:172.21.15.42:3300/0 conn(0x7fbf9006f610 0x7fbf9015e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.581+0000 7fbf95600640 1 -- 172.21.15.42:0/2995685291 >> v1:172.21.15.42:6789/0 conn(0x7fbf90073a90 legacy=0x7fbf9006ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:31.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.581+0000 7fbf95600640 1 -- 172.21.15.42:0/2995685291 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9006e700 con 0x7fbf9015dc30 2024-05-28T13:58:31.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.597+0000 7fbf95600640 1 --2- 172.21.15.42:0/2995685291 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9015dc30 0x7fbf900ad1c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009870 tx=0x7fbf7c01c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13f6c1020a1826b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.597+0000 7fbf8fe00640 1 -- 172.21.15.42:0/2995685291 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0b0070 con 0x7fbf9015dc30 2024-05-28T13:58:31.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.597+0000 7fbf8fe00640 1 -- 172.21.15.42:0/2995685291 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf7c0ab600 con 0x7fbf9015dc30 2024-05-28T13:58:31.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.597+0000 7fbf8fe00640 1 -- 172.21.15.42:0/2995685291 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0ab9d0 con 0x7fbf9015dc30 2024-05-28T13:58:31.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.597+0000 7fbf96a00640 1 -- 172.21.15.42:0/2995685291 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9015dc30 msgr2=0x7fbf900ad1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.597+0000 7fbf96a00640 1 --2- 172.21.15.42:0/2995685291 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9015dc30 0x7fbf900ad1c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009870 tx=0x7fbf7c01c3d0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.597+0000 7fbf96a00640 1 -- 172.21.15.42:0/2995685291 shutdown_connections 2024-05-28T13:58:31.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.597+0000 7fbf96a00640 1 --2- 172.21.15.42:0/2995685291 >> v2:172.21.15.42:3300/0 conn(0x7fbf9006f610 0x7fbf9015e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.597+0000 7fbf96a00640 1 --2- 172.21.15.42:0/2995685291 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9015dc30 0x7fbf900ad1c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.597+0000 7fbf96a00640 1 -- 172.21.15.42:0/2995685291 >> 172.21.15.42:0/2995685291 conn(0x7fbf900b1ee0 msgr2=0x7fbf900b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:31.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.597+0000 7fbf96a00640 1 -- 172.21.15.42:0/2995685291 shutdown_connections 2024-05-28T13:58:31.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.597+0000 7fbf96a00640 1 -- 172.21.15.42:0/2995685291 wait complete. 2024-05-28T13:58:31.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.601+0000 7fbf96a00640 1 Processor -- start 2024-05-28T13:58:31.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.601+0000 7fbf96a00640 1 -- start start 2024-05-28T13:58:31.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf96a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9006f610 0x7fbf90201620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf96a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf90073a90 0x7fbf902051d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf95600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9006f610 0x7fbf90201620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf95600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9006f610 0x7fbf90201620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42968/0 (socket says 172.21.15.42:42968) 2024-05-28T13:58:31.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf95600640 1 -- 172.21.15.42:0/1981058750 learned_addr learned my addr 172.21.15.42:0/1981058750 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:31.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf96a00640 1 --2- 172.21.15.42:0/1981058750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf9015dc30 0x7fbf901583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf96a00640 1 -- 172.21.15.42:0/1981058750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9006fba0 con 0x7fbf90073a90 2024-05-28T13:58:31.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf96a00640 1 -- 172.21.15.42:0/1981058750 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9006fa20 con 0x7fbf9015dc30 2024-05-28T13:58:31.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf96a00640 1 -- 172.21.15.42:0/1981058750 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf90169840 con 0x7fbf9006f610 2024-05-28T13:58:31.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf95600640 1 -- 172.21.15.42:0/1981058750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf9015dc30 msgr2=0x7fbf901583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf95600640 1 --2- 172.21.15.42:0/1981058750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf9015dc30 0x7fbf901583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf95600640 1 -- 172.21.15.42:0/1981058750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf90073a90 msgr2=0x7fbf902051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf95600640 1 --2- 172.21.15.42:0/1981058750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf90073a90 0x7fbf902051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf95600640 1 -- 172.21.15.42:0/1981058750 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf90157690 con 0x7fbf9006f610 2024-05-28T13:58:31.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf95600640 1 --2- 172.21.15.42:0/1981058750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9006f610 0x7fbf90201620 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0ae040 tx=0x7fbf7c019170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf8d600640 1 -- 172.21.15.42:0/1981058750 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0ba020 con 0x7fbf9006f610 2024-05-28T13:58:31.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf8d600640 1 -- 172.21.15.42:0/1981058750 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf7c0b6660 con 0x7fbf9006f610 2024-05-28T13:58:31.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf8d600640 1 -- 172.21.15.42:0/1981058750 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0b0040 con 0x7fbf9006f610 2024-05-28T13:58:31.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf96a00640 1 -- 172.21.15.42:0/1981058750 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf90204c90 con 0x7fbf9006f610 2024-05-28T13:58:31.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.605+0000 7fbf96a00640 1 -- 172.21.15.42:0/1981058750 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf90157900 con 0x7fbf9006f610 2024-05-28T13:58:31.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.609+0000 7fbf8d600640 1 -- 172.21.15.42:0/1981058750 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbf7c0bd020 con 0x7fbf9006f610 2024-05-28T13:58:31.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.609+0000 7fbf96a00640 1 -- 172.21.15.42:0/1981058750 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf9004fcf0 con 0x7fbf9006f610 2024-05-28T13:58:31.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.613+0000 7fbf8d600640 1 --2- 172.21.15.42:0/1981058750 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf700683b0 0x7fbf7006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.613+0000 7fbf8d600640 1 -- 172.21.15.42:0/1981058750 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbf7c120df0 con 0x7fbf9006f610 2024-05-28T13:58:31.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.613+0000 7fbf94c00640 1 --2- 172.21.15.42:0/1981058750 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf700683b0 0x7fbf7006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.613+0000 7fbf94c00640 1 --2- 172.21.15.42:0/1981058750 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf700683b0 0x7fbf7006a870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbf9009e060 tx=0x7fbf88007d20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.617+0000 7fbf8d600640 1 -- 172.21.15.42:0/1981058750 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbf7c0c4030 con 0x7fbf9006f610 2024-05-28T13:58:31.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.649+0000 7fbf35000640 1 Processor -- start 2024-05-28T13:58:31.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.649+0000 7fbf35000640 1 -- start start 2024-05-28T13:58:31.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fbf35000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf300ad1c0 0x7fbf3015d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fbf35000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbf3015df10 0x7fbf3006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fbf35000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf300ad620 con 0x7fbf300b2a00 2024-05-28T13:58:31.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fbf35000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fbf3006e140 con 0x7fbf3015df10 2024-05-28T13:58:31.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fbf35000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3006e420 con 0x7fbf300ad1c0 2024-05-28T13:58:31.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fbf2ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf300ad1c0 0x7fbf3015d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fbf2ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf300ad1c0 0x7fbf3015d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42974/0 (socket says 172.21.15.42:42974) 2024-05-28T13:58:31.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fbf2ea00640 1 -- 172.21.15.42:0/95743571 learned_addr learned my addr 172.21.15.42:0/95743571 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:31.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fbf2fe00640 1 --2- 172.21.15.42:0/95743571 >> v2:172.21.15.42:3300/0 conn(0x7fbf3015df10 0x7fbf3006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fbf2ea00640 1 -- 172.21.15.42:0/95743571 >> v2:172.21.15.42:3300/0 conn(0x7fbf3015df10 msgr2=0x7fbf3006ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:31.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fbf2ea00640 1 --2- 172.21.15.42:0/95743571 >> v2:172.21.15.42:3300/0 conn(0x7fbf3015df10 0x7fbf3006ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fbf2ea00640 1 -- 172.21.15.42:0/95743571 >> v1:172.21.15.42:6789/0 conn(0x7fbf300b2a00 legacy=0x7fbf30073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:31.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fbf2ea00640 1 -- 172.21.15.42:0/95743571 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3006e700 con 0x7fbf300ad1c0 2024-05-28T13:58:31.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fef03200640 1 Processor -- start 2024-05-28T13:58:31.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fef03200640 1 -- start start 2024-05-28T13:58:31.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.653+0000 7fef03200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feefc0e33d0 0x7feefc03a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fef03200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feefc0062e0 0x7feefc052610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fef03200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feefc04fdf0 con 0x7feefc050700 2024-05-28T13:58:31.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fef03200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7feefc052b50 con 0x7feefc0062e0 2024-05-28T13:58:31.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fef03200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feefc0500d0 con 0x7feefc0e33d0 2024-05-28T13:58:31.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fef02800640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7feefc050700 0x7feefc0396c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:50650/0 (socket says 172.21.15.42:50650) 2024-05-28T13:58:31.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7f946f200640 1 Processor -- start 2024-05-28T13:58:31.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7f49a6800640 1 Processor -- start 2024-05-28T13:58:31.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fef02800640 1 -- 172.21.15.42:0/3292467730 learned_addr learned my addr 172.21.15.42:0/3292467730 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:31.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fef01e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feefc0e33d0 0x7feefc03a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7f49a6800640 1 -- start start 2024-05-28T13:58:31.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fef00a00640 1 -- 172.21.15.42:0/3292467730 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2523006436 0 0) 0x7feefc04fdf0 con 0x7feefc050700 2024-05-28T13:58:31.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fef00a00640 1 -- 172.21.15.42:0/3292467730 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feee8003530 con 0x7feefc050700 2024-05-28T13:58:31.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fef01e00640 1 -- 172.21.15.42:0/3292467730 >> v2:172.21.15.63:3300/0 conn(0x7feefc0062e0 msgr2=0x7feefc052610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T13:58:31.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fef01e00640 1 --2- 172.21.15.42:0/3292467730 >> v2:172.21.15.63:3300/0 conn(0x7feefc0062e0 0x7feefc052610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fef01e00640 1 -- 172.21.15.42:0/3292467730 >> v1:172.21.15.63:6789/0 conn(0x7feefc050700 legacy=0x7feefc0396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fef01e00640 1 -- 172.21.15.42:0/3292467730 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc038ec0 con 0x7feefc0e33d0 2024-05-28T13:58:31.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7f946f200640 1 -- start start 2024-05-28T13:58:31.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fbf2ea00640 1 --2- 172.21.15.42:0/95743571 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf300ad1c0 0x7fbf3015d9d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009870 tx=0x7fbf2001c2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fee22062e3f3dd33 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fbf2e000640 1 -- 172.21.15.42:0/95743571 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf200b0070 con 0x7fbf300ad1c0 2024-05-28T13:58:31.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7f946f200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f947006df60 0x7f94700b1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7f946f200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f94700cf4d0 0x7f94700cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7f946f200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94700cf1d0 con 0x7f947015e9a0 2024-05-28T13:58:31.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7f946f200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f94700d2040 con 0x7f947006df60 2024-05-28T13:58:31.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7f946f200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f94700d2320 con 0x7f94700cf4d0 2024-05-28T13:58:31.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7fbf2e000640 1 -- 172.21.15.42:0/95743571 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf200ab600 con 0x7fbf300ad1c0 2024-05-28T13:58:31.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f946e800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f94700cf4d0 0x7f94700cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f946de00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f947015e9a0 0x7f94700ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:50660/0 (socket says 172.21.15.42:50660) 2024-05-28T13:58:31.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f946de00640 1 -- 172.21.15.42:0/3662112973 learned_addr learned my addr 172.21.15.42:0/3662112973 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:31.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7f49a6800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f49a00ce3c0 0x7f49a00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7f49a6800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f49a015b7e0 0x7f49a006b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.657+0000 7f49a6800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a00708b0 0x7f49a006bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f49a6800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f49a00cdff0 con 0x7f49a00ce3c0 2024-05-28T13:58:31.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f49a6800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f49a00b4760 con 0x7f49a015b7e0 2024-05-28T13:58:31.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f49a6800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f49a00cece0 con 0x7f49a00708b0 2024-05-28T13:58:31.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7fbf35000640 1 -- 172.21.15.42:0/95743571 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf300ad1c0 msgr2=0x7fbf3015d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f946d400640 1 --2- 172.21.15.42:0/3662112973 >> v2:172.21.15.42:3300/0 conn(0x7f947006df60 0x7f94700b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7fbf2e000640 1 -- 172.21.15.42:0/95743571 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf200b0070 con 0x7fbf300ad1c0 2024-05-28T13:58:31.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f499fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a00708b0 0x7f49a006bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f499f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f49a00ce3c0 0x7f49a00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7fbf35000640 1 --2- 172.21.15.42:0/95743571 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf300ad1c0 0x7fbf3015d9d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009870 tx=0x7fbf2001c2a0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f499f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f49a00ce3c0 0x7f49a00ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38654/0 (socket says 172.21.15.42:38654) 2024-05-28T13:58:31.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f499f400640 1 -- 172.21.15.42:0/1181859669 learned_addr learned my addr 172.21.15.42:0/1181859669 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:31.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f499fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a00708b0 0x7f49a006bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42984/0 (socket says 172.21.15.42:42984) 2024-05-28T13:58:31.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7fbf35000640 1 -- 172.21.15.42:0/95743571 shutdown_connections 2024-05-28T13:58:31.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7fbf35000640 1 --2- 172.21.15.42:0/95743571 >> v2:172.21.15.42:3300/0 conn(0x7fbf3015df10 0x7fbf3006ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f4996a00640 1 --2- 172.21.15.42:0/1181859669 >> v2:172.21.15.63:3300/0 conn(0x7f49a015b7e0 0x7f49a006b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7fbf35000640 1 --2- 172.21.15.42:0/95743571 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf300ad1c0 0x7fbf3015d9d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7fbf35000640 1 -- 172.21.15.42:0/95743571 >> 172.21.15.42:0/95743571 conn(0x7fbf300b1ee0 msgr2=0x7fbf300b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f499f400640 1 -- 172.21.15.42:0/1181859669 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a00708b0 msgr2=0x7f49a006bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:31.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7fbf35000640 1 -- 172.21.15.42:0/95743571 shutdown_connections 2024-05-28T13:58:31.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f499f400640 1 --2- 172.21.15.42:0/1181859669 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a00708b0 0x7f49a006bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7fbf35000640 1 -- 172.21.15.42:0/95743571 wait complete. 2024-05-28T13:58:31.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f499f400640 1 -- 172.21.15.42:0/1181859669 >> v2:172.21.15.63:3300/0 conn(0x7f49a015b7e0 msgr2=0x7f49a006b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:31.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f499f400640 1 --2- 172.21.15.42:0/1181859669 >> v2:172.21.15.63:3300/0 conn(0x7f49a015b7e0 0x7f49a006b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f499f400640 1 -- 172.21.15.42:0/1181859669 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a00b49b0 con 0x7f49a00ce3c0 2024-05-28T13:58:31.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f4996a00640 1 --2- 172.21.15.42:0/1181859669 >> v2:172.21.15.63:3300/0 conn(0x7f49a015b7e0 0x7f49a006b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:31.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f946d400640 1 -- 172.21.15.42:0/3662112973 >> v2:172.21.15.63:3300/0 conn(0x7f94700cf4d0 msgr2=0x7f94700cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:31.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f499fe00640 1 --2- 172.21.15.42:0/1181859669 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a00708b0 0x7f49a006bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:31.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f946d400640 1 --2- 172.21.15.42:0/3662112973 >> v2:172.21.15.63:3300/0 conn(0x7f94700cf4d0 0x7f94700cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7fef01e00640 1 --2- 172.21.15.42:0/3292467730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feefc0e33d0 0x7feefc03a640 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feef400ba60 tx=0x7feef401eb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=463460071848a5a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7fef00a00640 1 -- 172.21.15.42:0/3292467730 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef40b4070 con 0x7feefc0e33d0 2024-05-28T13:58:31.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7fef00a00640 1 -- 172.21.15.42:0/3292467730 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef40a7b10 con 0x7feefc0e33d0 2024-05-28T13:58:31.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7fef00a00640 1 -- 172.21.15.42:0/3292467730 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef40af440 con 0x7feefc0e33d0 2024-05-28T13:58:31.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f946d400640 1 -- 172.21.15.42:0/3662112973 >> v1:172.21.15.63:6789/0 conn(0x7f947015e9a0 legacy=0x7f94700ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fef03200640 1 -- 172.21.15.42:0/3292467730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feefc0e33d0 msgr2=0x7feefc03a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fef03200640 1 --2- 172.21.15.42:0/3292467730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feefc0e33d0 0x7feefc03a640 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feef400ba60 tx=0x7feef401eb60 comp rx=0 tx=0).stop 2024-05-28T13:58:31.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.661+0000 7f946d400640 1 -- 172.21.15.42:0/3662112973 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94700d2600 con 0x7f947006df60 2024-05-28T13:58:31.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fef03200640 1 -- 172.21.15.42:0/3292467730 shutdown_connections 2024-05-28T13:58:31.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fef03200640 1 --2- 172.21.15.42:0/3292467730 >> v2:172.21.15.63:3300/0 conn(0x7feefc0062e0 0x7feefc052610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fef03200640 1 --2- 172.21.15.42:0/3292467730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feefc0e33d0 0x7feefc03a640 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fef03200640 1 -- 172.21.15.42:0/3292467730 >> 172.21.15.42:0/3292467730 conn(0x7feefc009fc0 msgr2=0x7feefc00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:31.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fef03200640 1 -- 172.21.15.42:0/3292467730 shutdown_connections 2024-05-28T13:58:31.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fa9ad200640 1 Processor -- start 2024-05-28T13:58:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fbf35000640 1 Processor -- start 2024-05-28T13:58:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fef03200640 1 -- 172.21.15.42:0/3292467730 wait complete. 2024-05-28T13:58:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fa9ad200640 1 -- start start 2024-05-28T13:58:31.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fbf35000640 1 -- start start 2024-05-28T13:58:31.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fbf35000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf300ad1c0 0x7fbf3015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fbf35000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf300b2a00 0x7fbf300b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fbf35000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf3015df10 0x7fbf30099560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fbf2f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf300ad1c0 0x7fbf3015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fbf2f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf300ad1c0 0x7fbf3015c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38138/0 (socket says 172.21.15.42:38138) 2024-05-28T13:58:31.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fbf2f400640 1 -- 172.21.15.42:0/2175637460 learned_addr learned my addr 172.21.15.42:0/2175637460 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:31.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fbf35000640 1 -- 172.21.15.42:0/2175637460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3015ef50 con 0x7fbf3015df10 2024-05-28T13:58:31.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fbf35000640 1 -- 172.21.15.42:0/2175637460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3015edd0 con 0x7fbf300ad1c0 2024-05-28T13:58:31.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fa9ad200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a815e680 0x7fa9a80b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fbf35000640 1 -- 172.21.15.42:0/2175637460 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3015f0d0 con 0x7fbf300b2a00 2024-05-28T13:58:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fa9ad200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa9a80b2bb0 0x7fa9a80b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fa9ad200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a8073080 con 0x7fa9a80b51d0 2024-05-28T13:58:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fa9ad200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa9a8073c90 con 0x7fa9a80b2bb0 2024-05-28T13:58:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fa9ad200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a8073360 con 0x7fa9a815e680 2024-05-28T13:58:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7f499f400640 1 --2- 172.21.15.42:0/1181859669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a00ce3c0 0x7f49a00ce7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49a00b4090 tx=0x7f499001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b446c7fed4d012d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.665+0000 7fbf2ea00640 1 --2- 172.21.15.42:0/2175637460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf300b2a00 0x7fbf300b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fbf2f400640 1 -- 172.21.15.42:0/2175637460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf300b2a00 msgr2=0x7fbf300b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a7400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a815e680 0x7fa9a80b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fbf2f400640 1 --2- 172.21.15.42:0/2175637460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf300b2a00 0x7fbf300b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fbf2f400640 1 -- 172.21.15.42:0/2175637460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf3015df10 msgr2=0x7fbf30099560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fbf2f400640 1 --2- 172.21.15.42:0/2175637460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf3015df10 0x7fbf30099560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fbf2f400640 1 -- 172.21.15.42:0/2175637460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf301562f0 con 0x7fbf300ad1c0 2024-05-28T13:58:31.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a7400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a815e680 0x7fa9a80b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:43006/0 (socket says 172.21.15.42:43006) 2024-05-28T13:58:31.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a7400640 1 -- 172.21.15.42:0/232308823 learned_addr learned my addr 172.21.15.42:0/232308823 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:31.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a6a00640 1 --2- 172.21.15.42:0/232308823 >> v2:172.21.15.42:3300/0 conn(0x7fa9a80b2bb0 0x7fa9a80b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fef03200640 1 Processor -- start 2024-05-28T13:58:31.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fbf2f400640 1 --2- 172.21.15.42:0/2175637460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf300ad1c0 0x7fbf3015c1b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800c7a0 tx=0x7fbf2800cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fbf27400640 1 -- 172.21.15.42:0/2175637460 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf28017070 con 0x7fbf300ad1c0 2024-05-28T13:58:31.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fbf27400640 1 -- 172.21.15.42:0/2175637460 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf28003db0 con 0x7fbf300ad1c0 2024-05-28T13:58:31.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fbf27400640 1 -- 172.21.15.42:0/2175637460 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf280052b0 con 0x7fbf300ad1c0 2024-05-28T13:58:31.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fbf35000640 1 -- 172.21.15.42:0/2175637460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf300b17a0 con 0x7fbf300ad1c0 2024-05-28T13:58:31.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fbf35000640 1 -- 172.21.15.42:0/2175637460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf30156590 con 0x7fbf300ad1c0 2024-05-28T13:58:31.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7f946d400640 1 --2- 172.21.15.42:0/3662112973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f947006df60 0x7f94700b1150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f947008c680 tx=0x7f945c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edf50b452db6ba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7f946ca00640 1 -- 172.21.15.42:0/3662112973 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c0b0070 con 0x7f947006df60 2024-05-28T13:58:31.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fef03200640 1 -- start start 2024-05-28T13:58:31.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7f499ea00640 1 -- 172.21.15.42:0/1181859669 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49900b0070 con 0x7f49a00ce3c0 2024-05-28T13:58:31.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7f499ea00640 1 -- 172.21.15.42:0/1181859669 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49900ab600 con 0x7f49a00ce3c0 2024-05-28T13:58:31.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7f499ea00640 1 -- 172.21.15.42:0/1181859669 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49900ab9d0 con 0x7f49a00ce3c0 2024-05-28T13:58:31.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fbf35000640 1 -- 172.21.15.42:0/2175637460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf3004fcf0 con 0x7fbf300ad1c0 2024-05-28T13:58:31.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7f946ca00640 1 -- 172.21.15.42:0/3662112973 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945c0ab600 con 0x7f947006df60 2024-05-28T13:58:31.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7f946ca00640 1 -- 172.21.15.42:0/3662112973 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c0ab9d0 con 0x7f947006df60 2024-05-28T13:58:31.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7f946f200640 1 -- 172.21.15.42:0/3662112973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f947006df60 msgr2=0x7f94700b1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7f946f200640 1 --2- 172.21.15.42:0/3662112973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f947006df60 0x7f94700b1150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f947008c680 tx=0x7f945c01c480 comp rx=0 tx=0).stop 2024-05-28T13:58:31.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fef03200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feefc0062e0 0x7feefc17f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fef03200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feefc050700 0x7feefc0df830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fef01e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feefc0062e0 0x7feefc17f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fef01e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feefc0062e0 0x7feefc17f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:43020/0 (socket says 172.21.15.42:43020) 2024-05-28T13:58:31.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fef03200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feefc0e33d0 0x7feefc0dfd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fef01400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feefc050700 0x7feefc0df830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a6000640 1 -- 172.21.15.42:0/232308823 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3082150270 0 0) 0x7fa9a8073080 con 0x7fa9a80b51d0 2024-05-28T13:58:31.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a6000640 1 -- 172.21.15.42:0/232308823 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa98c003530 con 0x7fa9a80b51d0 2024-05-28T13:58:31.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a6000640 1 -- 172.21.15.42:0/232308823 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 961159481 0 0) 0x7fa98c003530 con 0x7fa9a80b51d0 2024-05-28T13:58:31.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a6000640 1 -- 172.21.15.42:0/232308823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a815e680 msgr2=0x7fa9a80b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:31.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a6000640 1 --2- 172.21.15.42:0/232308823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a815e680 0x7fa9a80b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a6000640 1 -- 172.21.15.42:0/232308823 >> v2:172.21.15.42:3300/0 conn(0x7fa9a80b2bb0 msgr2=0x7fa9a80b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:31.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a6000640 1 --2- 172.21.15.42:0/232308823 >> v2:172.21.15.42:3300/0 conn(0x7fa9a80b2bb0 0x7fa9a80b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a6000640 1 -- 172.21.15.42:0/232308823 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a8073640 con 0x7fa9a80b51d0 2024-05-28T13:58:31.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a6000640 1 -- 172.21.15.42:0/232308823 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fa994001680 con 0x7fa9a80b51d0 2024-05-28T13:58:31.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a6000640 1 -- 172.21.15.42:0/232308823 >> v1:172.21.15.63:6789/0 conn(0x7fa9a80b51d0 legacy=0x7fa9a806e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.669+0000 7fa9a6000640 1 --2- 172.21.15.42:0/232308823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98c0039c0 0x7fa98c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fef01400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feefc050700 0x7feefc0df830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38148/0 (socket says 172.21.15.42:38148) 2024-05-28T13:58:31.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fef01e00640 1 -- 172.21.15.42:0/3455988038 learned_addr learned my addr 172.21.15.42:0/3455988038 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:31.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fef03200640 1 -- 172.21.15.42:0/3455988038 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feefc0e37b0 con 0x7feefc0e33d0 2024-05-28T13:58:31.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fef03200640 1 -- 172.21.15.42:0/3455988038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feefc0503b0 con 0x7feefc050700 2024-05-28T13:58:31.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fef03200640 1 -- 172.21.15.42:0/3455988038 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feefc050f20 con 0x7feefc0062e0 2024-05-28T13:58:31.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fa9a6000640 1 --2- 172.21.15.42:0/232308823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a815e680 0x7fa98c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fa9a6000640 1 --2- 172.21.15.42:0/232308823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa98c004d50 0x7fa98c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fa9a6000640 1 -- 172.21.15.42:0/232308823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a8073c90 con 0x7fa98c004d50 2024-05-28T13:58:31.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fa9a6000640 1 -- 172.21.15.42:0/232308823 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a8073360 con 0x7fa98c0039c0 2024-05-28T13:58:31.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fa9a6000640 1 -- 172.21.15.42:0/232308823 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa98c0141e0 con 0x7fa9a815e680 2024-05-28T13:58:31.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7f946f200640 1 -- 172.21.15.42:0/3662112973 shutdown_connections 2024-05-28T13:58:31.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7f946f200640 1 --2- 172.21.15.42:0/3662112973 >> v2:172.21.15.63:3300/0 conn(0x7f94700cf4d0 0x7f94700cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7f946f200640 1 --2- 172.21.15.42:0/3662112973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f947006df60 0x7f94700b1150 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7f946f200640 1 -- 172.21.15.42:0/3662112973 >> 172.21.15.42:0/3662112973 conn(0x7f947008bfe0 msgr2=0x7f94700d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:31.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7f946f200640 1 -- 172.21.15.42:0/3662112973 shutdown_connections 2024-05-28T13:58:31.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7f946f200640 1 -- 172.21.15.42:0/3662112973 wait complete. 2024-05-28T13:58:31.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fef02800640 1 --2- 172.21.15.42:0/3455988038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feefc0e33d0 0x7feefc0dfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fef01e00640 1 -- 172.21.15.42:0/3455988038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feefc050700 msgr2=0x7feefc0df830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fef01e00640 1 --2- 172.21.15.42:0/3455988038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feefc050700 0x7feefc0df830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fef01e00640 1 -- 172.21.15.42:0/3455988038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feefc0e33d0 msgr2=0x7feefc0dfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fef01e00640 1 --2- 172.21.15.42:0/3455988038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feefc0e33d0 0x7feefc0dfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fef01e00640 1 -- 172.21.15.42:0/3455988038 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc04bed0 con 0x7feefc0062e0 2024-05-28T13:58:31.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fa9a6a00640 1 --2- 172.21.15.42:0/232308823 >> v2:172.21.15.42:3300/0 conn(0x7fa9a80b2bb0 0x7fa9a80b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:31.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fa9a7e00640 1 --2- 172.21.15.42:0/232308823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa98c004d50 0x7fa98c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fa9a6a00640 1 --2- 172.21.15.42:0/232308823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a815e680 0x7fa98c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fa9a7400640 1 --2- 172.21.15.42:0/232308823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98c0039c0 0x7fa98c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fa9a6a00640 1 -- 172.21.15.42:0/232308823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98c0039c0 msgr2=0x7fa98c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:31.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fa9a6a00640 1 --2- 172.21.15.42:0/232308823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98c0039c0 0x7fa98c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fa9a6a00640 1 -- 172.21.15.42:0/232308823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa98c004d50 msgr2=0x7fa98c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:31.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fa9a6a00640 1 --2- 172.21.15.42:0/232308823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa98c004d50 0x7fa98c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fa9a6a00640 1 -- 172.21.15.42:0/232308823 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a8073640 con 0x7fa9a815e680 2024-05-28T13:58:31.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7f49a6800640 1 -- 172.21.15.42:0/1181859669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a00ce3c0 msgr2=0x7f49a00ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7f49a6800640 1 --2- 172.21.15.42:0/1181859669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a00ce3c0 0x7f49a00ce7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49a00b4090 tx=0x7f499001c490 comp rx=0 tx=0).stop 2024-05-28T13:58:31.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.677+0000 7fbf27400640 1 -- 172.21.15.42:0/2175637460 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbf2801a070 con 0x7fbf300ad1c0 2024-05-28T13:58:31.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7f49a6800640 1 -- 172.21.15.42:0/1181859669 shutdown_connections 2024-05-28T13:58:31.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7f49a6800640 1 --2- 172.21.15.42:0/1181859669 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a00708b0 0x7f49a006bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7f49a6800640 1 --2- 172.21.15.42:0/1181859669 >> v2:172.21.15.63:3300/0 conn(0x7f49a015b7e0 0x7f49a006b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7f49a6800640 1 --2- 172.21.15.42:0/1181859669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a00ce3c0 0x7f49a00ce7a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7f49a6800640 1 -- 172.21.15.42:0/1181859669 >> 172.21.15.42:0/1181859669 conn(0x7f49a00b3460 msgr2=0x7f49a00b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:31.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.677+0000 7f49a6800640 1 -- 172.21.15.42:0/1181859669 shutdown_connections 2024-05-28T13:58:31.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.673+0000 7fef01e00640 1 --2- 172.21.15.42:0/3455988038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feefc0062e0 0x7feefc17f3f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feef40b2040 tx=0x7feef401b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.677+0000 7f946f200640 1 Processor -- start 2024-05-28T13:58:31.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.677+0000 7feefa000640 1 -- 172.21.15.42:0/3455988038 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef40bf070 con 0x7feefc0062e0 2024-05-28T13:58:31.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.677+0000 7fef03200640 1 -- 172.21.15.42:0/3455988038 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc049750 con 0x7feefc0062e0 2024-05-28T13:58:31.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.677+0000 7fbf27400640 1 --2- 172.21.15.42:0/2175637460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf10068450 0x7fbf1006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.677+0000 7fbf27400640 1 -- 172.21.15.42:0/2175637460 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbf280860d0 con 0x7fbf300ad1c0 2024-05-28T13:58:31.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.677+0000 7fef03200640 1 -- 172.21.15.42:0/3455988038 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc04c630 con 0x7feefc0062e0 2024-05-28T13:58:31.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.677+0000 7feefa000640 1 -- 172.21.15.42:0/3455988038 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef40afb90 con 0x7feefc0062e0 2024-05-28T13:58:31.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.677+0000 7f49a6800640 1 -- 172.21.15.42:0/1181859669 wait complete. 2024-05-28T13:58:31.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.677+0000 7feefa000640 1 -- 172.21.15.42:0/3455988038 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef40bf070 con 0x7feefc0062e0 2024-05-28T13:58:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.677+0000 7f946f200640 1 -- start start 2024-05-28T13:58:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.677+0000 7f946f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f947006df60 0x7f9470151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.677+0000 7f946f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94700cf4d0 0x7f9470158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7f946de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f947006df60 0x7f9470151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7f946f200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f947015e9a0 0x7f947015c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7f946de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f947006df60 0x7f9470151080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38174/0 (socket says 172.21.15.42:38174) 2024-05-28T13:58:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7f946de00640 1 -- 172.21.15.42:0/1264951119 learned_addr learned my addr 172.21.15.42:0/1264951119 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7f946f200640 1 -- 172.21.15.42:0/1264951119 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f94700d3c80 con 0x7f94700cf4d0 2024-05-28T13:58:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7f946f200640 1 -- 172.21.15.42:0/1264951119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f94700d3b00 con 0x7f947006df60 2024-05-28T13:58:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7f946f200640 1 -- 172.21.15.42:0/1264951119 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f94700d3e00 con 0x7f947015e9a0 2024-05-28T13:58:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7f946d400640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94700cf4d0 0x7f9470158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7f946e800640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f947015e9a0 0x7f947015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7f49a6800640 1 Processor -- start 2024-05-28T13:58:31.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fa9a6a00640 1 --2- 172.21.15.42:0/232308823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a815e680 0x7fa98c004810 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000e350 tx=0x7fa9a001cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28609aaf9b1531f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fa9a6000640 1 -- 172.21.15.42:0/232308823 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00b2070 con 0x7fa9a815e680 2024-05-28T13:58:31.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fbf2ea00640 1 --2- 172.21.15.42:0/2175637460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf10068450 0x7fbf1006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fa9a6000640 1 -- 172.21.15.42:0/232308823 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a00ad440 con 0x7fa9a815e680 2024-05-28T13:58:31.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fa9a6000640 1 -- 172.21.15.42:0/232308823 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0009de0 con 0x7fa9a815e680 2024-05-28T13:58:31.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fbf27400640 1 -- 172.21.15.42:0/2175637460 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbf28056020 con 0x7fbf300ad1c0 2024-05-28T13:58:31.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fbf2ea00640 1 --2- 172.21.15.42:0/2175637460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf10068450 0x7fbf1006a910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf200ad000 tx=0x7fbf200a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7f49a6800640 1 -- start start 2024-05-28T13:58:31.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7feefa000640 1 -- 172.21.15.42:0/3455988038 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7feef40c2070 con 0x7feefc0062e0 2024-05-28T13:58:31.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fef03200640 1 -- 172.21.15.42:0/3455988038 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feedc0052a0 con 0x7feefc0062e0 2024-05-28T13:58:31.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7feefa000640 1 --2- 172.21.15.42:0/3455988038 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feee4068580 0x7feee406aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7feefa000640 1 -- 172.21.15.42:0/3455988038 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feef4120a40 con 0x7feefc0062e0 2024-05-28T13:58:31.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fef01400640 1 --2- 172.21.15.42:0/3455988038 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feee4068580 0x7feee406aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fa9ad200640 1 -- 172.21.15.42:0/232308823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a815e680 msgr2=0x7fa98c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fa9ad200640 1 --2- 172.21.15.42:0/232308823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a815e680 0x7fa98c004810 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000e350 tx=0x7fa9a001cea0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fa9a7e00640 1 -- 172.21.15.42:0/232308823 reap_dead start 2024-05-28T13:58:31.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fa9ad200640 1 -- 172.21.15.42:0/232308823 shutdown_connections 2024-05-28T13:58:31.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fa9ad200640 1 -- 172.21.15.42:0/232308823 >> 172.21.15.42:0/232308823 conn(0x7fa9a80b1ee0 msgr2=0x7fa9a80b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:31.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fa9ad200640 1 -- 172.21.15.42:0/232308823 shutdown_connections 2024-05-28T13:58:31.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.681+0000 7fa9ad200640 1 -- 172.21.15.42:0/232308823 wait complete. 2024-05-28T13:58:31.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f946de00640 1 -- 172.21.15.42:0/1264951119 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f947015e9a0 msgr2=0x7f947015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f946de00640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f947015e9a0 0x7f947015c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f946de00640 1 -- 172.21.15.42:0/1264951119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94700cf4d0 msgr2=0x7f9470158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f946de00640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94700cf4d0 0x7f9470158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f946de00640 1 -- 172.21.15.42:0/1264951119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947015b530 con 0x7f947006df60 2024-05-28T13:58:31.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f946d400640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94700cf4d0 0x7f9470158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:31.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f946de00640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f947006df60 0x7f9470151080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f946000c500 tx=0x7f946000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f49a6800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a00708b0 0x7f49a01450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f49a6800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a00ce3c0 0x7f49a015cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f49a6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a015b7e0 0x7f49a014ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f4996a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a00ce3c0 0x7f49a015cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f4996a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a00ce3c0 0x7f49a015cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:43044/0 (socket says 172.21.15.42:43044) 2024-05-28T13:58:31.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f499f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a00708b0 0x7f49a01450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f499f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a00708b0 0x7f49a01450a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38714/0 (socket says 172.21.15.42:38714) 2024-05-28T13:58:31.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f4996a00640 1 -- 172.21.15.42:0/442454034 learned_addr learned my addr 172.21.15.42:0/442454034 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:31.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7fef01400640 1 --2- 172.21.15.42:0/3455988038 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feee4068580 0x7feee406aa40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feefc0eae50 tx=0x7feef0009060 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f49a6800640 1 -- 172.21.15.42:0/442454034 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f49a006c880 con 0x7f49a00708b0 2024-05-28T13:58:31.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f49a6800640 1 -- 172.21.15.42:0/442454034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f49a0164150 con 0x7f49a015b7e0 2024-05-28T13:58:31.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f49a6800640 1 -- 172.21.15.42:0/442454034 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f49a00d0d30 con 0x7f49a00ce3c0 2024-05-28T13:58:31.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f9466000640 1 -- 172.21.15.42:0/1264951119 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460017070 con 0x7f947006df60 2024-05-28T13:58:31.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f9466000640 1 -- 172.21.15.42:0/1264951119 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9460005cf0 con 0x7f947006df60 2024-05-28T13:58:31.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f9466000640 1 -- 172.21.15.42:0/1264951119 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94600072f0 con 0x7f947006df60 2024-05-28T13:58:31.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f946f200640 1 -- 172.21.15.42:0/1264951119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947015ac90 con 0x7f947006df60 2024-05-28T13:58:31.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f946f200640 1 -- 172.21.15.42:0/1264951119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947015b250 con 0x7f947006df60 2024-05-28T13:58:31.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f499fe00640 1 --2- 172.21.15.42:0/442454034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a015b7e0 0x7f49a014ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f499fe00640 1 -- 172.21.15.42:0/442454034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a00ce3c0 msgr2=0x7f49a015cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f499fe00640 1 --2- 172.21.15.42:0/442454034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a00ce3c0 0x7f49a015cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f499fe00640 1 -- 172.21.15.42:0/442454034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a00708b0 msgr2=0x7f49a01450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f499fe00640 1 --2- 172.21.15.42:0/442454034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a00708b0 0x7f49a01450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.685+0000 7f499fe00640 1 -- 172.21.15.42:0/442454034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a01455e0 con 0x7f49a015b7e0 2024-05-28T13:58:31.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.689+0000 7f499fe00640 1 --2- 172.21.15.42:0/442454034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a015b7e0 0x7f49a014ce40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4998007fd0 tx=0x7f499800a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.689+0000 7f4997e00640 1 -- 172.21.15.42:0/442454034 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499801a070 con 0x7f49a015b7e0 2024-05-28T13:58:31.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.689+0000 7f946f200640 1 -- 172.21.15.42:0/1264951119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f947004fcf0 con 0x7f947006df60 2024-05-28T13:58:31.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.689+0000 7f4997e00640 1 -- 172.21.15.42:0/442454034 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f499800d840 con 0x7f49a015b7e0 2024-05-28T13:58:31.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.689+0000 7f4997e00640 1 -- 172.21.15.42:0/442454034 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499800dc30 con 0x7f49a015b7e0 2024-05-28T13:58:31.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.689+0000 7f49a6800640 1 -- 172.21.15.42:0/442454034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a015d9f0 con 0x7f49a015b7e0 2024-05-28T13:58:31.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.689+0000 7f49a6800640 1 -- 172.21.15.42:0/442454034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a015dfb0 con 0x7f49a015b7e0 2024-05-28T13:58:31.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.689+0000 7fa9ad200640 1 Processor -- start 2024-05-28T13:58:31.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.689+0000 7fa9ad200640 1 -- start start 2024-05-28T13:58:31.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.689+0000 7feefa000640 1 -- 172.21.15.42:0/3455988038 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feefc04bed0 con 0x7feefc0062e0 2024-05-28T13:58:31.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.689+0000 7f9466000640 1 -- 172.21.15.42:0/1264951119 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f946001a070 con 0x7f947006df60 2024-05-28T13:58:31.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.693+0000 7f9466000640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9450068480 0x7f945006a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.693+0000 7f9466000640 1 -- 172.21.15.42:0/1264951119 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9460085c30 con 0x7f947006df60 2024-05-28T13:58:31.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.693+0000 7f946d400640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9450068480 0x7f945006a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.689+0000 7f49a6800640 1 -- 172.21.15.42:0/442454034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a00706a0 con 0x7f49a015b7e0 2024-05-28T13:58:31.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.693+0000 7f4997e00640 1 -- 172.21.15.42:0/442454034 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f499801d070 con 0x7f49a015b7e0 2024-05-28T13:58:31.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.693+0000 7f946d400640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9450068480 0x7f945006a940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f945c0ad000 tx=0x7f945c0a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.693+0000 7f4997e00640 1 --2- 172.21.15.42:0/442454034 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f497c068400 0x7f497c06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.693+0000 7f4997e00640 1 -- 172.21.15.42:0/442454034 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4998086fe0 con 0x7f49a015b7e0 2024-05-28T13:58:31.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.693+0000 7f499f400640 1 --2- 172.21.15.42:0/442454034 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f497c068400 0x7f497c06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.693+0000 7fa9ad200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9a815e680 0x7fa9a8099900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.693+0000 7fa9ad200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9a81602e0 0x7fa9a8160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.693+0000 7fa9ad200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a8150980 0x7fa9a8150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.693+0000 7fa9ad200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a80af9e0 con 0x7fa9a815e680 2024-05-28T13:58:31.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.693+0000 7fa9ad200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a815ea60 con 0x7fa9a81602e0 2024-05-28T13:58:31.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.693+0000 7fa9ad200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a80afb60 con 0x7fa9a8150980 2024-05-28T13:58:31.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7f499f400640 1 --2- 172.21.15.42:0/442454034 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f497c068400 0x7f497c06a8c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f49900ad000 tx=0x7f4990008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7fa9a7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9a815e680 0x7fa9a8099900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7fa9a7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9a815e680 0x7fa9a8099900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38722/0 (socket says 172.21.15.42:38722) 2024-05-28T13:58:31.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7fa9a7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a8150980 0x7fa9a8150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7fa9a7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a8150980 0x7fa9a8150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:43058/0 (socket says 172.21.15.42:43058) 2024-05-28T13:58:31.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7fa9a6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9a81602e0 0x7fa9a8160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7fa9a6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9a81602e0 0x7fa9a8160760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38190/0 (socket says 172.21.15.42:38190) 2024-05-28T13:58:31.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7fa9a7400640 1 -- 172.21.15.42:0/3163650222 learned_addr learned my addr 172.21.15.42:0/3163650222 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:31.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7fa9a6a00640 1 -- 172.21.15.42:0/3163650222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a8150980 msgr2=0x7fa9a8150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7fa9a6a00640 1 --2- 172.21.15.42:0/3163650222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a8150980 0x7fa9a8150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7fa9a6a00640 1 -- 172.21.15.42:0/3163650222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9a815e680 msgr2=0x7fa9a8099900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7fa9a6a00640 1 --2- 172.21.15.42:0/3163650222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9a815e680 0x7fa9a8099900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7fa9a6a00640 1 -- 172.21.15.42:0/3163650222 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a815df00 con 0x7fa9a81602e0 2024-05-28T13:58:31.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7fa9a6a00640 1 --2- 172.21.15.42:0/3163650222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9a81602e0 0x7fa9a8160760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa98c004750 tx=0x7fa9a0009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.697+0000 7f9466000640 1 -- 172.21.15.42:0/1264951119 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9460055c10 con 0x7f947006df60 2024-05-28T13:58:31.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.701+0000 7f4997e00640 1 -- 172.21.15.42:0/442454034 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4998057000 con 0x7f49a015b7e0 2024-05-28T13:58:31.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.701+0000 7fa99f400640 1 -- 172.21.15.42:0/3163650222 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00b2040 con 0x7fa9a81602e0 2024-05-28T13:58:31.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.701+0000 7fa99f400640 1 -- 172.21.15.42:0/3163650222 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a00bb070 con 0x7fa9a81602e0 2024-05-28T13:58:31.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.701+0000 7fa99f400640 1 -- 172.21.15.42:0/3163650222 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0009de0 con 0x7fa9a81602e0 2024-05-28T13:58:31.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.701+0000 7fa9ad200640 1 -- 172.21.15.42:0/3163650222 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a8160d60 con 0x7fa9a81602e0 2024-05-28T13:58:31.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.701+0000 7fa9ad200640 1 -- 172.21.15.42:0/3163650222 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a8150180 con 0x7fa9a81602e0 2024-05-28T13:58:31.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.701+0000 7fa9ad200640 1 -- 172.21.15.42:0/3163650222 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a804fcf0 con 0x7fa9a81602e0 2024-05-28T13:58:31.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.705+0000 7fa99f400640 1 -- 172.21.15.42:0/3163650222 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa9a00b6890 con 0x7fa9a81602e0 2024-05-28T13:58:31.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.705+0000 7fa99f400640 1 --2- 172.21.15.42:0/3163650222 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa988068380 0x7fa98806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.709+0000 7fa9a7400640 1 --2- 172.21.15.42:0/3163650222 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa988068380 0x7fa98806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.709+0000 7fa99f400640 1 -- 172.21.15.42:0/3163650222 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa9a00ef8f0 con 0x7fa9a81602e0 2024-05-28T13:58:31.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.709+0000 7fa9a7400640 1 --2- 172.21.15.42:0/3163650222 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa988068380 0x7fa98806a840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa998006e10 tx=0x7fa998008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.713+0000 7fa99f400640 1 -- 172.21.15.42:0/3163650222 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa9a0016bb0 con 0x7fa9a81602e0 2024-05-28T13:58:31.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.721+0000 7fbbbba00640 1 Processor -- start 2024-05-28T13:58:31.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.721+0000 7fbbbba00640 1 -- start start 2024-05-28T13:58:31.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.729+0000 7fbbbba00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbbb40b3c30 0x7fbbb406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.729+0000 7fbbbba00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbb4073380 0x7fbbb40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.729+0000 7fbbbba00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbb40b76a0 con 0x7fbbb40b3850 2024-05-28T13:58:31.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.729+0000 7fbbbba00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbbb409c840 con 0x7fbbb40b3c30 2024-05-28T13:58:31.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.729+0000 7fbbbba00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb409cb20 con 0x7fbbb4073380 2024-05-28T13:58:31.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.729+0000 7fbbb9a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbb4073380 0x7fbbb40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.729+0000 7fbbb9a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbb4073380 0x7fbbb40b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:43060/0 (socket says 172.21.15.42:43060) 2024-05-28T13:58:31.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.729+0000 7fbbb9a00640 1 -- 172.21.15.42:0/1450234767 learned_addr learned my addr 172.21.15.42:0/1450234767 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:31.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.729+0000 7fbbb9a00640 1 -- 172.21.15.42:0/1450234767 >> v2:172.21.15.63:3300/0 conn(0x7fbbb40b3c30 msgr2=0x7fbbb406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:31.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.729+0000 7fbbb9a00640 1 --2- 172.21.15.42:0/1450234767 >> v2:172.21.15.63:3300/0 conn(0x7fbbb40b3c30 0x7fbbb406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.729+0000 7fbbb9a00640 1 -- 172.21.15.42:0/1450234767 >> v1:172.21.15.63:6789/0 conn(0x7fbbb40b3850 legacy=0x7fbbb409d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.729+0000 7fbbb9a00640 1 -- 172.21.15.42:0/1450234767 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb40d3bb0 con 0x7fbbb4073380 2024-05-28T13:58:31.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.733+0000 7f19eb000640 1 Processor -- start 2024-05-28T13:58:31.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.733+0000 7f19eb000640 1 -- start start 2024-05-28T13:58:31.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.733+0000 7f19eb000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19e415b760 0x7f19e40cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.733+0000 7f19eb000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f19e40cf970 0x7f19e40c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.733+0000 7f19eb000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19e40c5020 con 0x7f19e40ce390 2024-05-28T13:58:31.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.733+0000 7f19eb000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f19e40cff90 con 0x7f19e40cf970 2024-05-28T13:58:31.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.733+0000 7f19eb000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f19e40d0270 con 0x7f19e415b760 2024-05-28T13:58:31.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.733+0000 7f19e3400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f19e40ce390 0x7f19e40cf260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:50686/0 (socket says 172.21.15.42:50686) 2024-05-28T13:58:31.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.733+0000 7f19e3400640 1 -- 172.21.15.42:0/3382339002 learned_addr learned my addr 172.21.15.42:0/3382339002 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:31.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.733+0000 7f19e3e00640 1 --2- 172.21.15.42:0/3382339002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19e415b760 0x7f19e40cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.733+0000 7f19e9000640 1 --2- 172.21.15.42:0/3382339002 >> v2:172.21.15.42:3300/0 conn(0x7f19e40cf970 0x7f19e40c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.733+0000 7f19e2a00640 1 -- 172.21.15.42:0/3382339002 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 831015111 0 0) 0x7f19e40c5020 con 0x7f19e40ce390 2024-05-28T13:58:31.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.733+0000 7f19e2a00640 1 -- 172.21.15.42:0/3382339002 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19c8003530 con 0x7f19e40ce390 2024-05-28T13:58:31.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.733+0000 7f19e3e00640 1 -- 172.21.15.42:0/3382339002 >> v2:172.21.15.42:3300/0 conn(0x7f19e40cf970 msgr2=0x7f19e40c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:31.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7f19e2a00640 1 -- 172.21.15.42:0/3382339002 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 977040624 0 0) 0x7f19c8003530 con 0x7f19e40ce390 2024-05-28T13:58:31.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7f19e3e00640 1 --2- 172.21.15.42:0/3382339002 >> v2:172.21.15.42:3300/0 conn(0x7f19e40cf970 0x7f19e40c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7f19e3e00640 1 -- 172.21.15.42:0/3382339002 >> v1:172.21.15.63:6789/0 conn(0x7f19e40ce390 legacy=0x7f19e40cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7f19e3e00640 1 -- 172.21.15.42:0/3382339002 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e40c52c0 con 0x7f19e415b760 2024-05-28T13:58:31.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7f19e9000640 1 --2- 172.21.15.42:0/3382339002 >> v2:172.21.15.42:3300/0 conn(0x7f19e40cf970 0x7f19e40c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:31.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7fbbb9a00640 1 --2- 172.21.15.42:0/1450234767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbb4073380 0x7fbbb40b6130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009e50 tx=0x7fbbac01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fefd65b2604d94ca server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7fbbb3400640 1 -- 172.21.15.42:0/1450234767 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0a5620 con 0x7fbbb4073380 2024-05-28T13:58:31.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7fbbb3400640 1 -- 172.21.15.42:0/1450234767 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbac0a57c0 con 0x7fbbb4073380 2024-05-28T13:58:31.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7fbbb3400640 1 -- 172.21.15.42:0/1450234767 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0a5620 con 0x7fbbb4073380 2024-05-28T13:58:31.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7fbbbba00640 1 -- 172.21.15.42:0/1450234767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbb4073380 msgr2=0x7fbbb40b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7fbbbba00640 1 --2- 172.21.15.42:0/1450234767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbb4073380 0x7fbbb40b6130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009e50 tx=0x7fbbac01c890 comp rx=0 tx=0).stop 2024-05-28T13:58:31.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7fbbbba00640 1 -- 172.21.15.42:0/1450234767 shutdown_connections 2024-05-28T13:58:31.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7fbbbba00640 1 --2- 172.21.15.42:0/1450234767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbb4073380 0x7fbbb40b6130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7fbbbba00640 1 --2- 172.21.15.42:0/1450234767 >> v2:172.21.15.63:3300/0 conn(0x7fbbb40b3c30 0x7fbbb406d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7fbbbba00640 1 -- 172.21.15.42:0/1450234767 >> 172.21.15.42:0/1450234767 conn(0x7fbbb40d1130 msgr2=0x7fbbb40d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:31.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7fbbbba00640 1 -- 172.21.15.42:0/1450234767 shutdown_connections 2024-05-28T13:58:31.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.737+0000 7fbbbba00640 1 -- 172.21.15.42:0/1450234767 wait complete. 2024-05-28T13:58:31.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.741+0000 7f19e3e00640 1 --2- 172.21.15.42:0/3382339002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19e415b760 0x7f19e40cde50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f19dc014a00 tx=0x7f19dc025470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5146d83a3f21ca61 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.741+0000 7fbbbba00640 1 Processor -- start 2024-05-28T13:58:31.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.741+0000 7fbbbba00640 1 -- start start 2024-05-28T13:58:31.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.741+0000 7f19e2a00640 1 -- 172.21.15.42:0/3382339002 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc0ba070 con 0x7f19e415b760 2024-05-28T13:58:31.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.741+0000 7f19e2a00640 1 -- 172.21.15.42:0/3382339002 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19dc00fb40 con 0x7f19e415b760 2024-05-28T13:58:31.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.741+0000 7f19e2a00640 1 -- 172.21.15.42:0/3382339002 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc0ba070 con 0x7f19e415b760 2024-05-28T13:58:31.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.741+0000 7f19eb000640 1 -- 172.21.15.42:0/3382339002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19e415b760 msgr2=0x7f19e40cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.741+0000 7f19eb000640 1 --2- 172.21.15.42:0/3382339002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19e415b760 0x7f19e40cde50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f19dc014a00 tx=0x7f19dc025470 comp rx=0 tx=0).stop 2024-05-28T13:58:31.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7f19eb000640 1 -- 172.21.15.42:0/3382339002 shutdown_connections 2024-05-28T13:58:31.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7f19eb000640 1 --2- 172.21.15.42:0/3382339002 >> v2:172.21.15.42:3300/0 conn(0x7f19e40cf970 0x7f19e40c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7f19eb000640 1 --2- 172.21.15.42:0/3382339002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19e415b760 0x7f19e40cde50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7f19eb000640 1 -- 172.21.15.42:0/3382339002 >> 172.21.15.42:0/3382339002 conn(0x7f19e409aa50 msgr2=0x7f19e40b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:31.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7f19eb000640 1 -- 172.21.15.42:0/3382339002 shutdown_connections 2024-05-28T13:58:31.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7f19eb000640 1 -- 172.21.15.42:0/3382339002 wait complete. 2024-05-28T13:58:31.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7fbbbba00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbb4073380 0x7fbbb406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7fbbbba00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb40b3850 0x7fbbb4166780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7fbbb9000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbb4073380 0x7fbbb406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7fbbb9000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbb4073380 0x7fbbb406fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:43074/0 (socket says 172.21.15.42:43074) 2024-05-28T13:58:31.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7fbbb9000640 1 -- 172.21.15.42:0/3863321868 learned_addr learned my addr 172.21.15.42:0/3863321868 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:31.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7fbbb3e00640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb40b3850 0x7fbbb4166780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7fbbbba00640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbb40b3c30 0x7fbbb4071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7f19eb000640 1 Processor -- start 2024-05-28T13:58:31.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7fbbbba00640 1 -- 172.21.15.42:0/3863321868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb40d2fe0 con 0x7fbbb40b3c30 2024-05-28T13:58:31.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7f19eb000640 1 -- start start 2024-05-28T13:58:31.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.745+0000 7fbbb9a00640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbb40b3c30 0x7fbbb4071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7fbbbba00640 1 -- 172.21.15.42:0/3863321868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb40d2e60 con 0x7fbbb40b3850 2024-05-28T13:58:31.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7fbbbba00640 1 -- 172.21.15.42:0/3863321868 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb416ffa0 con 0x7fbbb4073380 2024-05-28T13:58:31.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7fbbb3e00640 1 -- 172.21.15.42:0/3863321868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbb4073380 msgr2=0x7fbbb406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7fbbb3e00640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbb4073380 0x7fbbb406fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7fbbb3e00640 1 -- 172.21.15.42:0/3863321868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbb40b3c30 msgr2=0x7fbbb4071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7fbbb3e00640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbb40b3c30 0x7fbbb4071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7fbbb3e00640 1 -- 172.21.15.42:0/3863321868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb4072990 con 0x7fbbb40b3850 2024-05-28T13:58:31.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19eb000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19e40ce390 0x7f19e414caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7fbbb9a00640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbb40b3c30 0x7fbbb4071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:31.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19eb000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e40cf970 0x7f19e40cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7fbbb3e00640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb40b3850 0x7fbbb4166780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbba400b3f0 tx=0x7fbba400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7fbbb0c00640 1 -- 172.21.15.42:0/3863321868 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba4015020 con 0x7fbbb40b3850 2024-05-28T13:58:31.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19e3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19e40ce390 0x7f19e414caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19e3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19e40ce390 0x7f19e414caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:43082/0 (socket says 172.21.15.42:43082) 2024-05-28T13:58:31.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19e3e00640 1 -- 172.21.15.42:0/2768862299 learned_addr learned my addr 172.21.15.42:0/2768862299 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:31.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19eb000640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e415b760 0x7f19e406be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19eb000640 1 -- 172.21.15.42:0/2768862299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f19e40c6be0 con 0x7f19e40cf970 2024-05-28T13:58:31.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19e3400640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e40cf970 0x7f19e40cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7fbbbba00640 1 -- 172.21.15.42:0/3863321868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb40726b0 con 0x7fbbb40b3850 2024-05-28T13:58:31.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19eb000640 1 -- 172.21.15.42:0/2768862299 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19e40c6a60 con 0x7f19e415b760 2024-05-28T13:58:31.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19eb000640 1 -- 172.21.15.42:0/2768862299 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f19e40c6d60 con 0x7f19e40ce390 2024-05-28T13:58:31.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19e9000640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e415b760 0x7f19e406be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7fbbb0c00640 1 -- 172.21.15.42:0/3863321868 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba4011830 con 0x7fbbb40b3850 2024-05-28T13:58:31.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19e9000640 1 -- 172.21.15.42:0/2768862299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19e40ce390 msgr2=0x7f19e414caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19e9000640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19e40ce390 0x7f19e414caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19e9000640 1 -- 172.21.15.42:0/2768862299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e40cf970 msgr2=0x7f19e40cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19e9000640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e40cf970 0x7f19e40cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7f19e9000640 1 -- 172.21.15.42:0/2768862299 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e406e930 con 0x7f19e415b760 2024-05-28T13:58:31.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.749+0000 7fbbbba00640 1 -- 172.21.15.42:0/3863321868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb40c96c0 con 0x7fbbb40b3850 2024-05-28T13:58:31.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.753+0000 7f19e9000640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e415b760 0x7f19e406be20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f19d400e630 tx=0x7f19d400eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.753+0000 7f19dbe00640 1 -- 172.21.15.42:0/2768862299 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4017070 con 0x7f19e415b760 2024-05-28T13:58:31.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.753+0000 7f19dbe00640 1 -- 172.21.15.42:0/2768862299 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d400c620 con 0x7f19e415b760 2024-05-28T13:58:31.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.753+0000 7f19eb000640 1 -- 172.21.15.42:0/2768862299 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e40b6050 con 0x7f19e415b760 2024-05-28T13:58:31.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.753+0000 7f19eb000640 1 -- 172.21.15.42:0/2768862299 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e40b6610 con 0x7f19e415b760 2024-05-28T13:58:31.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.753+0000 7f19dbe00640 1 -- 172.21.15.42:0/2768862299 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d400f040 con 0x7f19e415b760 2024-05-28T13:58:31.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.753+0000 7f19eb000640 1 -- 172.21.15.42:0/2768862299 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19bc0052a0 con 0x7f19e415b760 2024-05-28T13:58:31.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.753+0000 7fbbb0c00640 1 -- 172.21.15.42:0/3863321868 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba4011c20 con 0x7fbbb40b3850 2024-05-28T13:58:31.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.753+0000 7fbbb0c00640 1 -- 172.21.15.42:0/3863321868 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbba4018020 con 0x7fbbb40b3850 2024-05-28T13:58:31.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.757+0000 7fbbb0c00640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb94068380 0x7fbb9406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.757+0000 7fbbb9000640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb94068380 0x7fbb9406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.757+0000 7f19dbe00640 1 -- 172.21.15.42:0/2768862299 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f19d401a070 con 0x7f19e415b760 2024-05-28T13:58:31.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.757+0000 7f19dbe00640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19c40683d0 0x7f19c406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:31.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.757+0000 7fbbbba00640 1 -- 172.21.15.42:0/3863321868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb8c0052a0 con 0x7fbbb40b3850 2024-05-28T13:58:31.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.757+0000 7fbbb0c00640 1 -- 172.21.15.42:0/3863321868 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbba4011dc0 con 0x7fbbb40b3850 2024-05-28T13:58:31.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.757+0000 7f19dbe00640 1 -- 172.21.15.42:0/2768862299 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f19d40514e0 con 0x7f19e415b760 2024-05-28T13:58:31.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.757+0000 7f19e3e00640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19c40683d0 0x7f19c406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:31.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.761+0000 7fbbb9000640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb94068380 0x7fbb9406a840 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbba0007fd0 tx=0x7fbba0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.761+0000 7f19e3e00640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19c40683d0 0x7f19c406a890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f19dc025d10 tx=0x7f19dc0b0000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:31.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.761+0000 7f19dbe00640 1 -- 172.21.15.42:0/2768862299 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f19d4055790 con 0x7f19e415b760 2024-05-28T13:58:31.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.769+0000 7fbbb0c00640 1 -- 172.21.15.42:0/3863321868 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbba4056110 con 0x7fbbb40b3850 2024-05-28T13:58:31.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.977+0000 7fbf96a00640 1 -- 172.21.15.42:0/1981058750 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fbf900b3e30 con 0x7fbf9006f610 2024-05-28T13:58:31.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.977+0000 7fbf8d600640 1 -- 172.21.15.42:0/1981058750 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbf7c016110 con 0x7fbf9006f610 2024-05-28T13:58:31.964 INFO:teuthology.orchestra.run.smithi042.stdout:55834574854 2024-05-28T13:58:31.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.981+0000 7fbf96a00640 1 -- 172.21.15.42:0/1981058750 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf700683b0 msgr2=0x7fbf7006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.981+0000 7fbf96a00640 1 --2- 172.21.15.42:0/1981058750 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf700683b0 0x7fbf7006a870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbf9009e060 tx=0x7fbf88007d20 comp rx=0 tx=0).stop 2024-05-28T13:58:31.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.981+0000 7fbf96a00640 1 -- 172.21.15.42:0/1981058750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9006f610 msgr2=0x7fbf90201620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:31.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.981+0000 7fbf96a00640 1 --2- 172.21.15.42:0/1981058750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9006f610 0x7fbf90201620 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0ae040 tx=0x7fbf7c019170 comp rx=0 tx=0).stop 2024-05-28T13:58:31.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.985+0000 7fbf96a00640 1 -- 172.21.15.42:0/1981058750 shutdown_connections 2024-05-28T13:58:31.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.985+0000 7fbf96a00640 1 --2- 172.21.15.42:0/1981058750 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf700683b0 0x7fbf7006a870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.985+0000 7fbf96a00640 1 --2- 172.21.15.42:0/1981058750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf9015dc30 0x7fbf901583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.985+0000 7fbf96a00640 1 --2- 172.21.15.42:0/1981058750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf90073a90 0x7fbf902051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.985+0000 7fbf96a00640 1 --2- 172.21.15.42:0/1981058750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9006f610 0x7fbf90201620 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:31.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.985+0000 7fbf96a00640 1 -- 172.21.15.42:0/1981058750 >> 172.21.15.42:0/1981058750 conn(0x7fbf900b1ee0 msgr2=0x7fbf901689d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:31.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.985+0000 7fbf96a00640 1 -- 172.21.15.42:0/1981058750 shutdown_connections 2024-05-28T13:58:31.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:31.985+0000 7fbf96a00640 1 -- 172.21.15.42:0/1981058750 wait complete. 2024-05-28T13:58:31.999 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-05-28T13:58:32.000 DEBUG:teuthology.parallel:result is None 2024-05-28T13:58:32.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.017+0000 7fbf35000640 1 -- 172.21.15.42:0/2175637460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbf301540c0 con 0x7fbf300ad1c0 2024-05-28T13:58:32.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.017+0000 7fbf27400640 1 -- 172.21.15.42:0/2175637460 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbf2805a320 con 0x7fbf300ad1c0 2024-05-28T13:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout:55834574854 2024-05-28T13:58:32.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.021+0000 7fbf35000640 1 -- 172.21.15.42:0/2175637460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf10068450 msgr2=0x7fbf1006a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.021+0000 7fbf35000640 1 --2- 172.21.15.42:0/2175637460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf10068450 0x7fbf1006a910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf200ad000 tx=0x7fbf200a6000 comp rx=0 tx=0).stop 2024-05-28T13:58:32.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.021+0000 7fbf35000640 1 -- 172.21.15.42:0/2175637460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf300ad1c0 msgr2=0x7fbf3015c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.021+0000 7fbf35000640 1 --2- 172.21.15.42:0/2175637460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf300ad1c0 0x7fbf3015c1b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800c7a0 tx=0x7fbf2800cc60 comp rx=0 tx=0).stop 2024-05-28T13:58:32.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.021+0000 7fbf35000640 1 -- 172.21.15.42:0/2175637460 shutdown_connections 2024-05-28T13:58:32.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.021+0000 7fbf35000640 1 --2- 172.21.15.42:0/2175637460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf10068450 0x7fbf1006a910 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.021+0000 7fbf35000640 1 --2- 172.21.15.42:0/2175637460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf3015df10 0x7fbf30099560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.021+0000 7fbf35000640 1 --2- 172.21.15.42:0/2175637460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf300b2a00 0x7fbf300b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.021+0000 7fbf35000640 1 --2- 172.21.15.42:0/2175637460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf300ad1c0 0x7fbf3015c1b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.021+0000 7fbf35000640 1 -- 172.21.15.42:0/2175637460 >> 172.21.15.42:0/2175637460 conn(0x7fbf300b1ee0 msgr2=0x7fbf300b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:32.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.021+0000 7fbf35000640 1 -- 172.21.15.42:0/2175637460 shutdown_connections 2024-05-28T13:58:32.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.021+0000 7fbf35000640 1 -- 172.21.15.42:0/2175637460 wait complete. 2024-05-28T13:58:32.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.037+0000 7f946f200640 1 -- 172.21.15.42:0/1264951119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f947006e6c0 con 0x7f947006df60 2024-05-28T13:58:32.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.037+0000 7f9466000640 1 -- 172.21.15.42:0/1264951119 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9460059f10 con 0x7f947006df60 2024-05-28T13:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout:55834574855 2024-05-28T13:58:32.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.041+0000 7f946f200640 1 -- 172.21.15.42:0/1264951119 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9450068480 msgr2=0x7f945006a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.041+0000 7f946f200640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9450068480 0x7f945006a940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f945c0ad000 tx=0x7f945c0a6000 comp rx=0 tx=0).stop 2024-05-28T13:58:32.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.041+0000 7f946f200640 1 -- 172.21.15.42:0/1264951119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f947006df60 msgr2=0x7f9470151080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.041+0000 7f946f200640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f947006df60 0x7f9470151080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f946000c500 tx=0x7f946000c9c0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.041+0000 7f946f200640 1 -- 172.21.15.42:0/1264951119 shutdown_connections 2024-05-28T13:58:32.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.041+0000 7f946f200640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9450068480 0x7f945006a940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.041+0000 7f946f200640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f947015e9a0 0x7f947015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.041+0000 7f946f200640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94700cf4d0 0x7f9470158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.041+0000 7f946f200640 1 --2- 172.21.15.42:0/1264951119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f947006df60 0x7f9470151080 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.041+0000 7f946f200640 1 -- 172.21.15.42:0/1264951119 >> 172.21.15.42:0/1264951119 conn(0x7f947008bfe0 msgr2=0x7f94700b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:32.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.041+0000 7f946f200640 1 -- 172.21.15.42:0/1264951119 shutdown_connections 2024-05-28T13:58:32.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.041+0000 7f946f200640 1 -- 172.21.15.42:0/1264951119 wait complete. 2024-05-28T13:58:32.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.045+0000 7fa9ad200640 1 -- 172.21.15.42:0/3163650222 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa9a806d990 con 0x7fa9a81602e0 2024-05-28T13:58:32.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.045+0000 7fa99f400640 1 -- 172.21.15.42:0/3163650222 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa9a00f1850 con 0x7fa9a81602e0 2024-05-28T13:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout:55834574855 2024-05-28T13:58:32.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.049+0000 7fa9ad200640 1 -- 172.21.15.42:0/3163650222 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa988068380 msgr2=0x7fa98806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.049+0000 7fa9ad200640 1 --2- 172.21.15.42:0/3163650222 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa988068380 0x7fa98806a840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa998006e10 tx=0x7fa998008040 comp rx=0 tx=0).stop 2024-05-28T13:58:32.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.049+0000 7fa9ad200640 1 -- 172.21.15.42:0/3163650222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9a81602e0 msgr2=0x7fa9a8160760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.049+0000 7fa9ad200640 1 --2- 172.21.15.42:0/3163650222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9a81602e0 0x7fa9a8160760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa98c004750 tx=0x7fa9a0009750 comp rx=0 tx=0).stop 2024-05-28T13:58:32.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.049+0000 7fa9ad200640 1 -- 172.21.15.42:0/3163650222 shutdown_connections 2024-05-28T13:58:32.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.049+0000 7fa9ad200640 1 --2- 172.21.15.42:0/3163650222 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa988068380 0x7fa98806a840 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.049+0000 7fa9ad200640 1 --2- 172.21.15.42:0/3163650222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9a8150980 0x7fa9a8150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.049+0000 7fa9ad200640 1 --2- 172.21.15.42:0/3163650222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9a81602e0 0x7fa9a8160760 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.049+0000 7fa9ad200640 1 --2- 172.21.15.42:0/3163650222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9a815e680 0x7fa9a8099900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.049+0000 7fa9ad200640 1 -- 172.21.15.42:0/3163650222 >> 172.21.15.42:0/3163650222 conn(0x7fa9a80b1ee0 msgr2=0x7fa9a80ad1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:32.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.053+0000 7fa9ad200640 1 -- 172.21.15.42:0/3163650222 shutdown_connections 2024-05-28T13:58:32.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.053+0000 7fa9ad200640 1 -- 172.21.15.42:0/3163650222 wait complete. 2024-05-28T13:58:32.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.053+0000 7f49a6800640 1 -- 172.21.15.42:0/442454034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f49a00b0d10 con 0x7f49a015b7e0 2024-05-28T13:58:32.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.053+0000 7fef03200640 1 -- 172.21.15.42:0/3455988038 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7feedc005590 con 0x7feefc0062e0 2024-05-28T13:58:32.039 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-05-28T13:58:32.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.053+0000 7feefa000640 1 -- 172.21.15.42:0/3455988038 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feef4018780 con 0x7feefc0062e0 2024-05-28T13:58:32.039 DEBUG:teuthology.parallel:result is None 2024-05-28T13:58:32.040 INFO:teuthology.orchestra.run.smithi042.stdout:55834574854 2024-05-28T13:58:32.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.053+0000 7f4997e00640 1 -- 172.21.15.42:0/442454034 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f499805b300 con 0x7f49a015b7e0 2024-05-28T13:58:32.041 INFO:teuthology.orchestra.run.smithi042.stdout:55834574855 2024-05-28T13:58:32.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.057+0000 7fef03200640 1 -- 172.21.15.42:0/3455988038 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feee4068580 msgr2=0x7feee406aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.057+0000 7fef03200640 1 --2- 172.21.15.42:0/3455988038 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feee4068580 0x7feee406aa40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feefc0eae50 tx=0x7feef0009060 comp rx=0 tx=0).stop 2024-05-28T13:58:32.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.057+0000 7fef03200640 1 -- 172.21.15.42:0/3455988038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feefc0062e0 msgr2=0x7feefc17f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.057+0000 7fef03200640 1 --2- 172.21.15.42:0/3455988038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feefc0062e0 0x7feefc17f3f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feef40b2040 tx=0x7feef401b900 comp rx=0 tx=0).stop 2024-05-28T13:58:32.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.057+0000 7fef03200640 1 -- 172.21.15.42:0/3455988038 shutdown_connections 2024-05-28T13:58:32.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.057+0000 7fef03200640 1 --2- 172.21.15.42:0/3455988038 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feee4068580 0x7feee406aa40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.057+0000 7fef03200640 1 --2- 172.21.15.42:0/3455988038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feefc0e33d0 0x7feefc0dfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.057+0000 7fef03200640 1 --2- 172.21.15.42:0/3455988038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feefc050700 0x7feefc0df830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.057+0000 7fef03200640 1 --2- 172.21.15.42:0/3455988038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feefc0062e0 0x7feefc17f3f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.057+0000 7fef03200640 1 -- 172.21.15.42:0/3455988038 >> 172.21.15.42:0/3455988038 conn(0x7feefc009fc0 msgr2=0x7feefc0d7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.057+0000 7f49a6800640 1 -- 172.21.15.42:0/442454034 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f497c068400 msgr2=0x7f497c06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.057+0000 7f49a6800640 1 --2- 172.21.15.42:0/442454034 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f497c068400 0x7f497c06a8c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f49900ad000 tx=0x7f4990008040 comp rx=0 tx=0).stop 2024-05-28T13:58:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.057+0000 7f49a6800640 1 -- 172.21.15.42:0/442454034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a015b7e0 msgr2=0x7f49a014ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.057+0000 7f49a6800640 1 --2- 172.21.15.42:0/442454034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a015b7e0 0x7f49a014ce40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4998007fd0 tx=0x7f499800a710 comp rx=0 tx=0).stop 2024-05-28T13:58:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.061+0000 7f49a6800640 1 -- 172.21.15.42:0/442454034 shutdown_connections 2024-05-28T13:58:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.061+0000 7f49a6800640 1 --2- 172.21.15.42:0/442454034 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f497c068400 0x7f497c06a8c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.061+0000 7f49a6800640 1 --2- 172.21.15.42:0/442454034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a015b7e0 0x7f49a014ce40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.061+0000 7f49a6800640 1 --2- 172.21.15.42:0/442454034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a00ce3c0 0x7f49a015cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.061+0000 7f49a6800640 1 --2- 172.21.15.42:0/442454034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a00708b0 0x7f49a01450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.061+0000 7f49a6800640 1 -- 172.21.15.42:0/442454034 >> 172.21.15.42:0/442454034 conn(0x7f49a00b3460 msgr2=0x7f49a01598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:32.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.061+0000 7fef03200640 1 -- 172.21.15.42:0/3455988038 shutdown_connections 2024-05-28T13:58:32.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.061+0000 7fef03200640 1 -- 172.21.15.42:0/3455988038 wait complete. 2024-05-28T13:58:32.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.061+0000 7f49a6800640 1 -- 172.21.15.42:0/442454034 shutdown_connections 2024-05-28T13:58:32.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.061+0000 7f49a6800640 1 -- 172.21.15.42:0/442454034 wait complete. 2024-05-28T13:58:32.059 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-05-28T13:58:32.059 DEBUG:teuthology.parallel:result is None 2024-05-28T13:58:32.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.085+0000 7fbbbba00640 1 -- 172.21.15.42:0/3863321868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbb8c005860 con 0x7fbbb40b3850 2024-05-28T13:58:32.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-05-28T13:58:32.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.085+0000 7fbbb0c00640 1 -- 172.21.15.42:0/3863321868 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbba4055a10 con 0x7fbbb40b3850 2024-05-28T13:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout:55834574854 2024-05-28T13:58:32.071 DEBUG:teuthology.parallel:result is None 2024-05-28T13:58:32.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.085+0000 7f19eb000640 1 -- 172.21.15.42:0/2768862299 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f19bc005860 con 0x7f19e415b760 2024-05-28T13:58:32.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7f19dbe00640 1 -- 172.21.15.42:0/2768862299 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f19d4059a90 con 0x7f19e415b760 2024-05-28T13:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout:55834574856 2024-05-28T13:58:32.074 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-05-28T13:58:32.075 DEBUG:teuthology.parallel:result is None 2024-05-28T13:58:32.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7fbbbba00640 1 -- 172.21.15.42:0/3863321868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb94068380 msgr2=0x7fbb9406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7fbbbba00640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb94068380 0x7fbb9406a840 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbba0007fd0 tx=0x7fbba0009040 comp rx=0 tx=0).stop 2024-05-28T13:58:32.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7fbbbba00640 1 -- 172.21.15.42:0/3863321868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb40b3850 msgr2=0x7fbbb4166780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7fbbbba00640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb40b3850 0x7fbbb4166780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbba400b3f0 tx=0x7fbba400b8b0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7fbbbba00640 1 -- 172.21.15.42:0/3863321868 shutdown_connections 2024-05-28T13:58:32.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7fbbbba00640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb94068380 0x7fbb9406a840 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7fbbbba00640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbb40b3c30 0x7fbbb4071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7fbbbba00640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb40b3850 0x7fbbb4166780 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7fbbbba00640 1 --2- 172.21.15.42:0/3863321868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbb4073380 0x7fbbb406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7fbbbba00640 1 -- 172.21.15.42:0/3863321868 >> 172.21.15.42:0/3863321868 conn(0x7fbbb40d1130 msgr2=0x7fbbb40ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:32.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7fbbbba00640 1 -- 172.21.15.42:0/3863321868 shutdown_connections 2024-05-28T13:58:32.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7fbbbba00640 1 -- 172.21.15.42:0/3863321868 wait complete. 2024-05-28T13:58:32.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7f19eb000640 1 -- 172.21.15.42:0/2768862299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19c40683d0 msgr2=0x7f19c406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7f19eb000640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19c40683d0 0x7f19c406a890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f19dc025d10 tx=0x7f19dc0b0000 comp rx=0 tx=0).stop 2024-05-28T13:58:32.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7f19eb000640 1 -- 172.21.15.42:0/2768862299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e415b760 msgr2=0x7f19e406be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7f19eb000640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e415b760 0x7f19e406be20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f19d400e630 tx=0x7f19d400eaf0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7f19eb000640 1 -- 172.21.15.42:0/2768862299 shutdown_connections 2024-05-28T13:58:32.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7f19eb000640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19c40683d0 0x7f19c406a890 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7f19eb000640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e415b760 0x7f19e406be20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7f19eb000640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e40cf970 0x7f19e40cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7f19eb000640 1 --2- 172.21.15.42:0/2768862299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19e40ce390 0x7f19e414caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7f19eb000640 1 -- 172.21.15.42:0/2768862299 >> 172.21.15.42:0/2768862299 conn(0x7f19e409aa50 msgr2=0x7f19e4152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:32.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7f19eb000640 1 -- 172.21.15.42:0/2768862299 shutdown_connections 2024-05-28T13:58:32.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.089+0000 7f19eb000640 1 -- 172.21.15.42:0/2768862299 wait complete. 2024-05-28T13:58:32.078 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-05-28T13:58:32.078 DEBUG:teuthology.parallel:result is None 2024-05-28T13:58:32.096 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-05-28T13:58:32.096 DEBUG:teuthology.parallel:result is None 2024-05-28T13:58:32.097 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-05-28T13:58:32.098 DEBUG:teuthology.parallel:result is None 2024-05-28T13:58:32.098 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-28T13:58:32.098 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T13:58:32.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.293+0000 7f987e200640 1 Processor -- start 2024-05-28T13:58:32.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.293+0000 7f987e200640 1 -- start start 2024-05-28T13:58:32.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.293+0000 7f987e200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9878156a70 0x7f987806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.293+0000 7f987e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9878156e50 0x7f98780af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f987e200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f987815cb90 con 0x7f98780af880 2024-05-28T13:58:32.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f987e200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f987815ce50 con 0x7f9878156a70 2024-05-28T13:58:32.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f987e200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f987815d130 con 0x7f9878156e50 2024-05-28T13:58:32.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9877e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f98780af880 0x7f98780d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60600/0 (socket says 172.21.15.42:60600) 2024-05-28T13:58:32.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9877e00640 1 -- 172.21.15.42:0/207270539 learned_addr learned my addr 172.21.15.42:0/207270539 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:32.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9877400640 1 --2- 172.21.15.42:0/207270539 >> v2:172.21.15.63:3300/0 conn(0x7f9878156a70 0x7f987806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:32.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 -- 172.21.15.42:0/207270539 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1122305601 0 0) 0x7f987815cb90 con 0x7f98780af880 2024-05-28T13:58:32.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f986fe00640 1 --2- 172.21.15.42:0/207270539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9878156e50 0x7f98780af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:32.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 -- 172.21.15.42:0/207270539 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f985c003530 con 0x7f98780af880 2024-05-28T13:58:32.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 -- 172.21.15.42:0/207270539 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1004678153 0 0) 0x7f985c003530 con 0x7f98780af880 2024-05-28T13:58:32.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 -- 172.21.15.42:0/207270539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9878156e50 msgr2=0x7f98780af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:32.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 --2- 172.21.15.42:0/207270539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9878156e50 0x7f98780af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 -- 172.21.15.42:0/207270539 >> v2:172.21.15.63:3300/0 conn(0x7f9878156a70 msgr2=0x7f987806dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:32.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 --2- 172.21.15.42:0/207270539 >> v2:172.21.15.63:3300/0 conn(0x7f9878156a70 0x7f987806dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 -- 172.21.15.42:0/207270539 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987815d410 con 0x7f98780af880 2024-05-28T13:58:32.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 -- 172.21.15.42:0/207270539 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f9870001a00 con 0x7f98780af880 2024-05-28T13:58:32.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 -- 172.21.15.42:0/207270539 >> v1:172.21.15.42:6789/0 conn(0x7f98780af880 legacy=0x7f98780d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 --2- 172.21.15.42:0/207270539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f985c0039c0 0x7f985c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 --2- 172.21.15.42:0/207270539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f985c004810 0x7f985c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 --2- 172.21.15.42:0/207270539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9878156e50 0x7f985c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 -- 172.21.15.42:0/207270539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f987815ce50 con 0x7f985c0039c0 2024-05-28T13:58:32.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 -- 172.21.15.42:0/207270539 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f987815d130 con 0x7f985c004810 2024-05-28T13:58:32.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9876a00640 1 -- 172.21.15.42:0/207270539 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f985c00b9d0 con 0x7f9878156e50 2024-05-28T13:58:32.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9877400640 1 --2- 172.21.15.42:0/207270539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f985c0039c0 0x7f985c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:32.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f986fe00640 1 --2- 172.21.15.42:0/207270539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f985c004810 0x7f985c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:32.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9877e00640 1 --2- 172.21.15.42:0/207270539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9878156e50 0x7f985c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:32.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9877400640 1 -- 172.21.15.42:0/207270539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9878156e50 msgr2=0x7f985c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:32.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9877400640 1 --2- 172.21.15.42:0/207270539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9878156e50 0x7f985c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9877400640 1 -- 172.21.15.42:0/207270539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f985c004810 msgr2=0x7f985c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:32.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9877400640 1 --2- 172.21.15.42:0/207270539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f985c004810 0x7f985c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.297+0000 7f9877400640 1 -- 172.21.15.42:0/207270539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985c003530 con 0x7f985c0039c0 2024-05-28T13:58:32.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.301+0000 7f9877400640 1 --2- 172.21.15.42:0/207270539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f985c0039c0 0x7f985c003de0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9868009ae0 tx=0x7f986801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69596ba735f14794 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:32.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.301+0000 7f9876a00640 1 -- 172.21.15.42:0/207270539 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98680096b0 con 0x7f985c0039c0 2024-05-28T13:58:32.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.301+0000 7f9876a00640 1 -- 172.21.15.42:0/207270539 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98680a64c0 con 0x7f985c0039c0 2024-05-28T13:58:32.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.301+0000 7f9876a00640 1 -- 172.21.15.42:0/207270539 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98680a68b0 con 0x7f985c0039c0 2024-05-28T13:58:32.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.301+0000 7f987e200640 1 -- 172.21.15.42:0/207270539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f985c0039c0 msgr2=0x7f985c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.301+0000 7f987e200640 1 --2- 172.21.15.42:0/207270539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f985c0039c0 0x7f985c003de0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9868009ae0 tx=0x7f986801cab0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.301+0000 7f9877e00640 1 -- 172.21.15.42:0/207270539 reap_dead start 2024-05-28T13:58:32.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.301+0000 7f987e200640 1 -- 172.21.15.42:0/207270539 shutdown_connections 2024-05-28T13:58:32.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.301+0000 7f987e200640 1 -- 172.21.15.42:0/207270539 >> 172.21.15.42:0/207270539 conn(0x7f987806bdf0 msgr2=0x7f987806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:32.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.301+0000 7f987e200640 1 -- 172.21.15.42:0/207270539 shutdown_connections 2024-05-28T13:58:32.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.301+0000 7f987e200640 1 -- 172.21.15.42:0/207270539 wait complete. 2024-05-28T13:58:32.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.301+0000 7f987e200640 1 Processor -- start 2024-05-28T13:58:32.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.301+0000 7f987e200640 1 -- start start 2024-05-28T13:58:32.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f987e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f985c0039c0 0x7f98780ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f987e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98781f6760 0x7f9878154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f987e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9878155350 0x7f987815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f987e200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f98780af340 con 0x7f9878155350 2024-05-28T13:58:32.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f987e200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f98780d1010 con 0x7f98781f6760 2024-05-28T13:58:32.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f987e200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f98780af4c0 con 0x7f985c0039c0 2024-05-28T13:58:32.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f9877e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9878155350 0x7f987815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:32.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f9877e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9878155350 0x7f987815b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38752/0 (socket says 172.21.15.42:38752) 2024-05-28T13:58:32.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f9877e00640 1 -- 172.21.15.42:0/1769867318 learned_addr learned my addr 172.21.15.42:0/1769867318 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:32.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f986fe00640 1 --2- 172.21.15.42:0/1769867318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98781f6760 0x7f9878154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:32.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f9877400640 1 --2- 172.21.15.42:0/1769867318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f985c0039c0 0x7f98780ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:32.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f9877e00640 1 -- 172.21.15.42:0/1769867318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f985c0039c0 msgr2=0x7f98780ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f9877e00640 1 --2- 172.21.15.42:0/1769867318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f985c0039c0 0x7f98780ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f9877e00640 1 -- 172.21.15.42:0/1769867318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98781f6760 msgr2=0x7f9878154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f9877e00640 1 --2- 172.21.15.42:0/1769867318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98781f6760 0x7f9878154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f9877e00640 1 -- 172.21.15.42:0/1769867318 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9878071d40 con 0x7f9878155350 2024-05-28T13:58:32.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f9877e00640 1 --2- 172.21.15.42:0/1769867318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9878155350 0x7f987815b640 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9878074200 tx=0x7f9870007ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:32.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f986f400640 1 -- 172.21.15.42:0/1769867318 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987000eaa0 con 0x7f9878155350 2024-05-28T13:58:32.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f986f400640 1 -- 172.21.15.42:0/1769867318 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f987000ec40 con 0x7f9878155350 2024-05-28T13:58:32.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f987e200640 1 -- 172.21.15.42:0/1769867318 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9878072020 con 0x7f9878155350 2024-05-28T13:58:32.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f986f400640 1 -- 172.21.15.42:0/1769867318 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987000ee30 con 0x7f9878155350 2024-05-28T13:58:32.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f987e200640 1 -- 172.21.15.42:0/1769867318 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98780725e0 con 0x7f9878155350 2024-05-28T13:58:32.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f987e200640 1 -- 172.21.15.42:0/1769867318 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9878068a90 con 0x7f9878155350 2024-05-28T13:58:32.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.305+0000 7f986f400640 1 -- 172.21.15.42:0/1769867318 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f987001a020 con 0x7f9878155350 2024-05-28T13:58:32.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.309+0000 7f986f400640 1 --2- 172.21.15.42:0/1769867318 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9854068380 0x7f985406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.309+0000 7f986f400640 1 -- 172.21.15.42:0/1769867318 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f98700862e0 con 0x7f9878155350 2024-05-28T13:58:32.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.309+0000 7f9877400640 1 --2- 172.21.15.42:0/1769867318 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9854068380 0x7f985406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:32.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.309+0000 7f9877400640 1 --2- 172.21.15.42:0/1769867318 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9854068380 0x7f985406a840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f98680af000 tx=0x7f98680046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:32.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.313+0000 7f986f400640 1 -- 172.21.15.42:0/1769867318 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9870021020 con 0x7f9878155350 2024-05-28T13:58:32.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.473+0000 7f987e200640 1 -- 172.21.15.42:0/1769867318 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f98780b5160 con 0x7f9854068380 2024-05-28T13:58:32.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.481+0000 7f986f400640 1 -- 172.21.15.42:0/1769867318 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f98780b5160 con 0x7f9854068380 2024-05-28T13:58:32.470 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:32.472 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T13:58:32.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.485+0000 7f987e200640 1 -- 172.21.15.42:0/1769867318 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9854068380 msgr2=0x7f985406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.485+0000 7f987e200640 1 --2- 172.21.15.42:0/1769867318 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9854068380 0x7f985406a840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f98680af000 tx=0x7f98680046d0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.485+0000 7f987e200640 1 -- 172.21.15.42:0/1769867318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9878155350 msgr2=0x7f987815b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.485+0000 7f987e200640 1 --2- 172.21.15.42:0/1769867318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9878155350 0x7f987815b640 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9878074200 tx=0x7f9870007ae0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.485+0000 7f987e200640 1 -- 172.21.15.42:0/1769867318 shutdown_connections 2024-05-28T13:58:32.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.485+0000 7f987e200640 1 --2- 172.21.15.42:0/1769867318 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9854068380 0x7f985406a840 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.485+0000 7f987e200640 1 --2- 172.21.15.42:0/1769867318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9878155350 0x7f987815b640 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.485+0000 7f987e200640 1 --2- 172.21.15.42:0/1769867318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98781f6760 0x7f9878154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.485+0000 7f987e200640 1 --2- 172.21.15.42:0/1769867318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f985c0039c0 0x7f98780ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.485+0000 7f987e200640 1 -- 172.21.15.42:0/1769867318 >> 172.21.15.42:0/1769867318 conn(0x7f987806bdf0 msgr2=0x7f9878155d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:32.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.485+0000 7f987e200640 1 -- 172.21.15.42:0/1769867318 shutdown_connections 2024-05-28T13:58:32.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.485+0000 7f987e200640 1 -- 172.21.15.42:0/1769867318 wait complete. 2024-05-28T13:58:32.488 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-05-28T13:58:30.857328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235740,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738980,"statfs":{"total":773094113280,"available":772852715520,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001354"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440826+0000","last_change":"2024-05-28T13:58:23.690106+0000","last_active":"2024-05-28T13:58:26.440826+0000","last_peered":"2024-05-28T13:58:26.440826+0000","last_clean":"2024-05-28T13:58:26.440826+0000","last_became_active":"2024-05-28T13:58:23.689597+0000","last_became_peered":"2024-05-28T13:58:23.689597+0000","last_unstale":"2024-05-28T13:58:26.440826+0000","last_undegraded":"2024-05-28T13:58:26.440826+0000","last_fullsized":"2024-05-28T13:58:26.440826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:32:51.191091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663757+0000","last_change":"2024-05-28T13:58:23.674151+0000","last_active":"2024-05-28T13:58:24.663757+0000","last_peered":"2024-05-28T13:58:24.663757+0000","last_clean":"2024-05-28T13:58:24.663757+0000","last_became_active":"2024-05-28T13:58:23.673848+0000","last_became_peered":"2024-05-28T13:58:23.673848+0000","last_unstale":"2024-05-28T13:58:24.663757+0000","last_undegraded":"2024-05-28T13:58:24.663757+0000","last_fullsized":"2024-05-28T13:58:24.663757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:23.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439412+0000","last_change":"2024-05-28T13:58:23.685573+0000","last_active":"2024-05-28T13:58:26.439412+0000","last_peered":"2024-05-28T13:58:26.439412+0000","last_clean":"2024-05-28T13:58:26.439412+0000","last_became_active":"2024-05-28T13:58:23.685243+0000","last_became_peered":"2024-05-28T13:58:23.685243+0000","last_unstale":"2024-05-28T13:58:26.439412+0000","last_undegraded":"2024-05-28T13:58:26.439412+0000","last_fullsized":"2024-05-28T13:58:26.439412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:46.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668781+0000","last_change":"2024-05-28T13:58:23.706150+0000","last_active":"2024-05-28T13:58:24.668781+0000","last_peered":"2024-05-28T13:58:24.668781+0000","last_clean":"2024-05-28T13:58:24.668781+0000","last_became_active":"2024-05-28T13:58:23.689938+0000","last_became_peered":"2024-05-28T13:58:23.689938+0000","last_unstale":"2024-05-28T13:58:24.668781+0000","last_undegraded":"2024-05-28T13:58:24.668781+0000","last_fullsized":"2024-05-28T13:58:24.668781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:16:14.092125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440405+0000","last_change":"2024-05-28T13:58:23.683882+0000","last_active":"2024-05-28T13:58:26.440405+0000","last_peered":"2024-05-28T13:58:26.440405+0000","last_clean":"2024-05-28T13:58:26.440405+0000","last_became_active":"2024-05-28T13:58:23.683368+0000","last_became_peered":"2024-05-28T13:58:23.683368+0000","last_unstale":"2024-05-28T13:58:26.440405+0000","last_undegraded":"2024-05-28T13:58:26.440405+0000","last_fullsized":"2024-05-28T13:58:26.440405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:23:14.222046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668075+0000","last_change":"2024-05-28T13:58:23.695111+0000","last_active":"2024-05-28T13:58:24.668075+0000","last_peered":"2024-05-28T13:58:24.668075+0000","last_clean":"2024-05-28T13:58:24.668075+0000","last_became_active":"2024-05-28T13:58:23.688505+0000","last_became_peered":"2024-05-28T13:58:23.688505+0000","last_unstale":"2024-05-28T13:58:24.668075+0000","last_undegraded":"2024-05-28T13:58:24.668075+0000","last_fullsized":"2024-05-28T13:58:24.668075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:49.556769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440203+0000","last_change":"2024-05-28T13:58:23.684793+0000","last_active":"2024-05-28T13:58:26.440203+0000","last_peered":"2024-05-28T13:58:26.440203+0000","last_clean":"2024-05-28T13:58:26.440203+0000","last_became_active":"2024-05-28T13:58:23.684500+0000","last_became_peered":"2024-05-28T13:58:23.684500+0000","last_unstale":"2024-05-28T13:58:26.440203+0000","last_undegraded":"2024-05-28T13:58:26.440203+0000","last_fullsized":"2024-05-28T13:58:26.440203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:27:53.809481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439015+0000","last_change":"2024-05-28T13:58:23.688670+0000","last_active":"2024-05-28T13:58:26.439015+0000","last_peered":"2024-05-28T13:58:26.439015+0000","last_clean":"2024-05-28T13:58:26.439015+0000","last_became_active":"2024-05-28T13:58:23.688243+0000","last_became_peered":"2024-05-28T13:58:23.688243+0000","last_unstale":"2024-05-28T13:58:26.439015+0000","last_undegraded":"2024-05-28T13:58:26.439015+0000","last_fullsized":"2024-05-28T13:58:26.439015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:10.036541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440836+0000","last_change":"2024-05-28T13:58:23.702434+0000","last_active":"2024-05-28T13:58:26.440836+0000","last_peered":"2024-05-28T13:58:26.440836+0000","last_clean":"2024-05-28T13:58:26.440836+0000","last_became_active":"2024-05-28T13:58:23.701857+0000","last_became_peered":"2024-05-28T13:58:23.701857+0000","last_unstale":"2024-05-28T13:58:26.440836+0000","last_undegraded":"2024-05-28T13:58:26.440836+0000","last_fullsized":"2024-05-28T13:58:26.440836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:41:47.900159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694909+0000","last_change":"2024-05-28T13:58:23.709688+0000","last_active":"2024-05-28T13:58:24.694909+0000","last_peered":"2024-05-28T13:58:24.694909+0000","last_clean":"2024-05-28T13:58:24.694909+0000","last_became_active":"2024-05-28T13:58:23.709431+0000","last_became_peered":"2024-05-28T13:58:23.709431+0000","last_unstale":"2024-05-28T13:58:24.694909+0000","last_undegraded":"2024-05-28T13:58:24.694909+0000","last_fullsized":"2024-05-28T13:58:24.694909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:49.556769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694612+0000","last_change":"2024-05-28T13:58:23.701527+0000","last_active":"2024-05-28T13:58:24.694612+0000","last_peered":"2024-05-28T13:58:24.694612+0000","last_clean":"2024-05-28T13:58:24.694612+0000","last_became_active":"2024-05-28T13:58:23.701160+0000","last_became_peered":"2024-05-28T13:58:23.701160+0000","last_unstale":"2024-05-28T13:58:24.694612+0000","last_undegraded":"2024-05-28T13:58:24.694612+0000","last_fullsized":"2024-05-28T13:58:24.694612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:03:59.917115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668501+0000","last_change":"2024-05-28T13:58:23.707766+0000","last_active":"2024-05-28T13:58:24.668501+0000","last_peered":"2024-05-28T13:58:24.668501+0000","last_clean":"2024-05-28T13:58:24.668501+0000","last_became_active":"2024-05-28T13:58:23.707542+0000","last_became_peered":"2024-05-28T13:58:23.707542+0000","last_unstale":"2024-05-28T13:58:24.668501+0000","last_undegraded":"2024-05-28T13:58:24.668501+0000","last_fullsized":"2024-05-28T13:58:24.668501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:03.122796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439237+0000","last_change":"2024-05-28T13:58:23.687526+0000","last_active":"2024-05-28T13:58:26.439237+0000","last_peered":"2024-05-28T13:58:26.439237+0000","last_clean":"2024-05-28T13:58:26.439237+0000","last_became_active":"2024-05-28T13:58:23.687138+0000","last_became_peered":"2024-05-28T13:58:23.687138+0000","last_unstale":"2024-05-28T13:58:26.439237+0000","last_undegraded":"2024-05-28T13:58:26.439237+0000","last_fullsized":"2024-05-28T13:58:26.439237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:27:53.809481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.438877+0000","last_change":"2024-05-28T13:58:23.681967+0000","last_active":"2024-05-28T13:58:26.438877+0000","last_peered":"2024-05-28T13:58:26.438877+0000","last_clean":"2024-05-28T13:58:26.438877+0000","last_became_active":"2024-05-28T13:58:23.680491+0000","last_became_peered":"2024-05-28T13:58:23.680491+0000","last_unstale":"2024-05-28T13:58:26.438877+0000","last_undegraded":"2024-05-28T13:58:26.438877+0000","last_fullsized":"2024-05-28T13:58:26.438877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:07:03.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663864+0000","last_change":"2024-05-28T13:58:23.708051+0000","last_active":"2024-05-28T13:58:24.663864+0000","last_peered":"2024-05-28T13:58:24.663864+0000","last_clean":"2024-05-28T13:58:24.663864+0000","last_became_active":"2024-05-28T13:58:23.707749+0000","last_became_peered":"2024-05-28T13:58:23.707749+0000","last_unstale":"2024-05-28T13:58:24.663864+0000","last_undegraded":"2024-05-28T13:58:24.663864+0000","last_fullsized":"2024-05-28T13:58:24.663864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:02.981053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.667924+0000","last_change":"2024-05-28T13:58:23.684368+0000","last_active":"2024-05-28T13:58:24.667924+0000","last_peered":"2024-05-28T13:58:24.667924+0000","last_clean":"2024-05-28T13:58:24.667924+0000","last_became_active":"2024-05-28T13:58:23.683905+0000","last_became_peered":"2024-05-28T13:58:23.683905+0000","last_unstale":"2024-05-28T13:58:24.667924+0000","last_undegraded":"2024-05-28T13:58:24.667924+0000","last_fullsized":"2024-05-28T13:58:24.667924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:02.981053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668773+0000","last_change":"2024-05-28T13:58:23.706633+0000","last_active":"2024-05-28T13:58:24.668773+0000","last_peered":"2024-05-28T13:58:24.668773+0000","last_clean":"2024-05-28T13:58:24.668773+0000","last_became_active":"2024-05-28T13:58:23.693217+0000","last_became_peered":"2024-05-28T13:58:23.693217+0000","last_unstale":"2024-05-28T13:58:24.668773+0000","last_undegraded":"2024-05-28T13:58:24.668773+0000","last_fullsized":"2024-05-28T13:58:24.668773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:23:14.222046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:25.415731+0000","last_change":"2024-05-28T13:58:23.690320+0000","last_active":"2024-05-28T13:58:25.415731+0000","last_peered":"2024-05-28T13:58:25.415731+0000","last_clean":"2024-05-28T13:58:25.415731+0000","last_became_active":"2024-05-28T13:58:23.690028+0000","last_became_peered":"2024-05-28T13:58:23.690028+0000","last_unstale":"2024-05-28T13:58:25.415731+0000","last_undegraded":"2024-05-28T13:58:25.415731+0000","last_fullsized":"2024-05-28T13:58:25.415731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:59.716378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440863+0000","last_change":"2024-05-28T13:58:23.683310+0000","last_active":"2024-05-28T13:58:26.440863+0000","last_peered":"2024-05-28T13:58:26.440863+0000","last_clean":"2024-05-28T13:58:26.440863+0000","last_became_active":"2024-05-28T13:58:23.682882+0000","last_became_peered":"2024-05-28T13:58:23.682882+0000","last_unstale":"2024-05-28T13:58:26.440863+0000","last_undegraded":"2024-05-28T13:58:26.440863+0000","last_fullsized":"2024-05-28T13:58:26.440863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:28:55.178232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694723+0000","last_change":"2024-05-28T13:58:23.706004+0000","last_active":"2024-05-28T13:58:24.694723+0000","last_peered":"2024-05-28T13:58:24.694723+0000","last_clean":"2024-05-28T13:58:24.694723+0000","last_became_active":"2024-05-28T13:58:23.705676+0000","last_became_peered":"2024-05-28T13:58:23.705676+0000","last_unstale":"2024-05-28T13:58:24.694723+0000","last_undegraded":"2024-05-28T13:58:24.694723+0000","last_fullsized":"2024-05-28T13:58:24.694723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:23.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439738+0000","last_change":"2024-05-28T13:58:23.678693+0000","last_active":"2024-05-28T13:58:26.439738+0000","last_peered":"2024-05-28T13:58:26.439738+0000","last_clean":"2024-05-28T13:58:26.439738+0000","last_became_active":"2024-05-28T13:58:23.678258+0000","last_became_peered":"2024-05-28T13:58:23.678258+0000","last_unstale":"2024-05-28T13:58:26.439738+0000","last_undegraded":"2024-05-28T13:58:26.439738+0000","last_fullsized":"2024-05-28T13:58:26.439738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:23.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668261+0000","last_change":"2024-05-28T13:58:23.706461+0000","last_active":"2024-05-28T13:58:24.668261+0000","last_peered":"2024-05-28T13:58:24.668261+0000","last_clean":"2024-05-28T13:58:24.668261+0000","last_became_active":"2024-05-28T13:58:23.702598+0000","last_became_peered":"2024-05-28T13:58:23.702598+0000","last_unstale":"2024-05-28T13:58:24.668261+0000","last_undegraded":"2024-05-28T13:58:24.668261+0000","last_fullsized":"2024-05-28T13:58:24.668261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:31.530501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439024+0000","last_change":"2024-05-28T13:58:23.714967+0000","last_active":"2024-05-28T13:58:26.439024+0000","last_peered":"2024-05-28T13:58:26.439024+0000","last_clean":"2024-05-28T13:58:26.439024+0000","last_became_active":"2024-05-28T13:58:23.714653+0000","last_became_peered":"2024-05-28T13:58:23.714653+0000","last_unstale":"2024-05-28T13:58:26.439024+0000","last_undegraded":"2024-05-28T13:58:26.439024+0000","last_fullsized":"2024-05-28T13:58:26.439024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:46.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668849+0000","last_change":"2024-05-28T13:58:23.686847+0000","last_active":"2024-05-28T13:58:24.668849+0000","last_peered":"2024-05-28T13:58:24.668849+0000","last_clean":"2024-05-28T13:58:24.668849+0000","last_became_active":"2024-05-28T13:58:23.686518+0000","last_became_peered":"2024-05-28T13:58:23.686518+0000","last_unstale":"2024-05-28T13:58:24.668849+0000","last_undegraded":"2024-05-28T13:58:24.668849+0000","last_fullsized":"2024-05-28T13:58:24.668849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:43:20.340652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439484+0000","last_change":"2024-05-28T13:58:23.707351+0000","last_active":"2024-05-28T13:58:26.439484+0000","last_peered":"2024-05-28T13:58:26.439484+0000","last_clean":"2024-05-28T13:58:26.439484+0000","last_became_active":"2024-05-28T13:58:23.707077+0000","last_became_peered":"2024-05-28T13:58:23.707077+0000","last_unstale":"2024-05-28T13:58:26.439484+0000","last_undegraded":"2024-05-28T13:58:26.439484+0000","last_fullsized":"2024-05-28T13:58:26.439484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:49.556769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694896+0000","last_change":"2024-05-28T13:58:23.688056+0000","last_active":"2024-05-28T13:58:24.694896+0000","last_peered":"2024-05-28T13:58:24.694896+0000","last_clean":"2024-05-28T13:58:24.694896+0000","last_became_active":"2024-05-28T13:58:23.687623+0000","last_became_peered":"2024-05-28T13:58:23.687623+0000","last_unstale":"2024-05-28T13:58:24.694896+0000","last_undegraded":"2024-05-28T13:58:24.694896+0000","last_fullsized":"2024-05-28T13:58:24.694896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:02.981053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663895+0000","last_change":"2024-05-28T13:58:23.672661+0000","last_active":"2024-05-28T13:58:24.663895+0000","last_peered":"2024-05-28T13:58:24.663895+0000","last_clean":"2024-05-28T13:58:24.663895+0000","last_became_active":"2024-05-28T13:58:23.672378+0000","last_became_peered":"2024-05-28T13:58:23.672378+0000","last_unstale":"2024-05-28T13:58:24.663895+0000","last_undegraded":"2024-05-28T13:58:24.663895+0000","last_fullsized":"2024-05-28T13:58:24.663895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:49.556769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439309+0000","last_change":"2024-05-28T13:58:23.707282+0000","last_active":"2024-05-28T13:58:26.439309+0000","last_peered":"2024-05-28T13:58:26.439309+0000","last_clean":"2024-05-28T13:58:26.439309+0000","last_became_active":"2024-05-28T13:58:23.706977+0000","last_became_peered":"2024-05-28T13:58:23.706977+0000","last_unstale":"2024-05-28T13:58:26.439309+0000","last_undegraded":"2024-05-28T13:58:26.439309+0000","last_fullsized":"2024-05-28T13:58:26.439309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:03:59.917115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439837+0000","last_change":"2024-05-28T13:58:23.684735+0000","last_active":"2024-05-28T13:58:26.439837+0000","last_peered":"2024-05-28T13:58:26.439837+0000","last_clean":"2024-05-28T13:58:26.439837+0000","last_became_active":"2024-05-28T13:58:23.684389+0000","last_became_peered":"2024-05-28T13:58:23.684389+0000","last_unstale":"2024-05-28T13:58:26.439837+0000","last_undegraded":"2024-05-28T13:58:26.439837+0000","last_fullsized":"2024-05-28T13:58:26.439837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:02.981053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668321+0000","last_change":"2024-05-28T13:58:23.686822+0000","last_active":"2024-05-28T13:58:24.668321+0000","last_peered":"2024-05-28T13:58:24.668321+0000","last_clean":"2024-05-28T13:58:24.668321+0000","last_became_active":"2024-05-28T13:58:23.686454+0000","last_became_peered":"2024-05-28T13:58:23.686454+0000","last_unstale":"2024-05-28T13:58:24.668321+0000","last_undegraded":"2024-05-28T13:58:24.668321+0000","last_fullsized":"2024-05-28T13:58:24.668321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:22:38.773556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:25.415579+0000","last_change":"2024-05-28T13:58:23.702203+0000","last_active":"2024-05-28T13:58:25.415579+0000","last_peered":"2024-05-28T13:58:25.415579+0000","last_clean":"2024-05-28T13:58:25.415579+0000","last_became_active":"2024-05-28T13:58:23.701830+0000","last_became_peered":"2024-05-28T13:58:23.701830+0000","last_unstale":"2024-05-28T13:58:25.415579+0000","last_undegraded":"2024-05-28T13:58:25.415579+0000","last_fullsized":"2024-05-28T13:58:25.415579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:56.904188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694422+0000","last_change":"2024-05-28T13:58:23.684058+0000","last_active":"2024-05-28T13:58:24.694422+0000","last_peered":"2024-05-28T13:58:24.694422+0000","last_clean":"2024-05-28T13:58:24.694422+0000","last_became_active":"2024-05-28T13:58:23.683555+0000","last_became_peered":"2024-05-28T13:58:23.683555+0000","last_unstale":"2024-05-28T13:58:24.694422+0000","last_undegraded":"2024-05-28T13:58:24.694422+0000","last_fullsized":"2024-05-28T13:58:24.694422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:10.036541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668066+0000","last_change":"2024-05-28T13:58:23.704842+0000","last_active":"2024-05-28T13:58:24.668066+0000","last_peered":"2024-05-28T13:58:24.668066+0000","last_clean":"2024-05-28T13:58:24.668066+0000","last_became_active":"2024-05-28T13:58:23.704465+0000","last_became_peered":"2024-05-28T13:58:23.704465+0000","last_unstale":"2024-05-28T13:58:24.668066+0000","last_undegraded":"2024-05-28T13:58:24.668066+0000","last_fullsized":"2024-05-28T13:58:24.668066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:30.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439668+0000","last_change":"2024-05-28T13:58:22.678741+0000","last_active":"2024-05-28T13:58:26.439668+0000","last_peered":"2024-05-28T13:58:26.439668+0000","last_clean":"2024-05-28T13:58:26.439668+0000","last_became_active":"2024-05-28T13:58:22.678532+0000","last_became_peered":"2024-05-28T13:58:22.678532+0000","last_unstale":"2024-05-28T13:58:26.439668+0000","last_undegraded":"2024-05-28T13:58:26.439668+0000","last_fullsized":"2024-05-28T13:58:26.439668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439908+0000","last_change":"2024-05-28T13:58:23.708224+0000","last_active":"2024-05-28T13:58:26.439908+0000","last_peered":"2024-05-28T13:58:26.439908+0000","last_clean":"2024-05-28T13:58:26.439908+0000","last_became_active":"2024-05-28T13:58:23.707930+0000","last_became_peered":"2024-05-28T13:58:23.707930+0000","last_unstale":"2024-05-28T13:58:26.439908+0000","last_undegraded":"2024-05-28T13:58:26.439908+0000","last_fullsized":"2024-05-28T13:58:26.439908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:31.530501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694550+0000","last_change":"2024-05-28T13:58:22.673310+0000","last_active":"2024-05-28T13:58:24.694550+0000","last_peered":"2024-05-28T13:58:24.694550+0000","last_clean":"2024-05-28T13:58:24.694550+0000","last_became_active":"2024-05-28T13:58:22.673003+0000","last_became_peered":"2024-05-28T13:58:22.673003+0000","last_unstale":"2024-05-28T13:58:24.694550+0000","last_undegraded":"2024-05-28T13:58:24.694550+0000","last_fullsized":"2024-05-28T13:58:24.694550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439193+0000","last_change":"2024-05-28T13:58:22.677344+0000","last_active":"2024-05-28T13:58:26.439193+0000","last_peered":"2024-05-28T13:58:26.439193+0000","last_clean":"2024-05-28T13:58:26.439193+0000","last_became_active":"2024-05-28T13:58:22.677024+0000","last_became_peered":"2024-05-28T13:58:22.677024+0000","last_unstale":"2024-05-28T13:58:26.439193+0000","last_undegraded":"2024-05-28T13:58:26.439193+0000","last_fullsized":"2024-05-28T13:58:26.439193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:52.890121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439186+0000","last_change":"2024-05-28T13:58:23.686005+0000","last_active":"2024-05-28T13:58:26.439186+0000","last_peered":"2024-05-28T13:58:26.439186+0000","last_clean":"2024-05-28T13:58:26.439186+0000","last_became_active":"2024-05-28T13:58:23.682423+0000","last_became_peered":"2024-05-28T13:58:23.682423+0000","last_unstale":"2024-05-28T13:58:26.439186+0000","last_undegraded":"2024-05-28T13:58:26.439186+0000","last_fullsized":"2024-05-28T13:58:26.439186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:22.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668085+0000","last_change":"2024-05-28T13:58:22.673902+0000","last_active":"2024-05-28T13:58:24.668085+0000","last_peered":"2024-05-28T13:58:24.668085+0000","last_clean":"2024-05-28T13:58:24.668085+0000","last_became_active":"2024-05-28T13:58:22.673588+0000","last_became_peered":"2024-05-28T13:58:22.673588+0000","last_unstale":"2024-05-28T13:58:24.668085+0000","last_undegraded":"2024-05-28T13:58:24.668085+0000","last_fullsized":"2024-05-28T13:58:24.668085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663575+0000","last_change":"2024-05-28T13:58:23.672761+0000","last_active":"2024-05-28T13:58:24.663575+0000","last_peered":"2024-05-28T13:58:24.663575+0000","last_clean":"2024-05-28T13:58:24.663575+0000","last_became_active":"2024-05-28T13:58:23.672276+0000","last_became_peered":"2024-05-28T13:58:23.672276+0000","last_unstale":"2024-05-28T13:58:24.663575+0000","last_undegraded":"2024-05-28T13:58:24.663575+0000","last_fullsized":"2024-05-28T13:58:24.663575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:46.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694720+0000","last_change":"2024-05-28T13:58:22.676336+0000","last_active":"2024-05-28T13:58:24.694720+0000","last_peered":"2024-05-28T13:58:24.694720+0000","last_clean":"2024-05-28T13:58:24.694720+0000","last_became_active":"2024-05-28T13:58:22.676013+0000","last_became_peered":"2024-05-28T13:58:22.676013+0000","last_unstale":"2024-05-28T13:58:24.694720+0000","last_undegraded":"2024-05-28T13:58:24.694720+0000","last_fullsized":"2024-05-28T13:58:24.694720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668373+0000","last_change":"2024-05-28T13:58:23.691740+0000","last_active":"2024-05-28T13:58:24.668373+0000","last_peered":"2024-05-28T13:58:24.668373+0000","last_clean":"2024-05-28T13:58:24.668373+0000","last_became_active":"2024-05-28T13:58:23.691056+0000","last_became_peered":"2024-05-28T13:58:23.691056+0000","last_unstale":"2024-05-28T13:58:24.668373+0000","last_undegraded":"2024-05-28T13:58:24.668373+0000","last_fullsized":"2024-05-28T13:58:24.668373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:27:53.809481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668300+0000","last_change":"2024-05-28T13:58:22.678043+0000","last_active":"2024-05-28T13:58:24.668300+0000","last_peered":"2024-05-28T13:58:24.668300+0000","last_clean":"2024-05-28T13:58:24.668300+0000","last_became_active":"2024-05-28T13:58:22.677684+0000","last_became_peered":"2024-05-28T13:58:22.677684+0000","last_unstale":"2024-05-28T13:58:24.668300+0000","last_undegraded":"2024-05-28T13:58:24.668300+0000","last_fullsized":"2024-05-28T13:58:24.668300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:02.116873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439582+0000","last_change":"2024-05-28T13:58:23.706540+0000","last_active":"2024-05-28T13:58:26.439582+0000","last_peered":"2024-05-28T13:58:26.439582+0000","last_clean":"2024-05-28T13:58:26.439582+0000","last_became_active":"2024-05-28T13:58:23.706209+0000","last_became_peered":"2024-05-28T13:58:23.706209+0000","last_unstale":"2024-05-28T13:58:26.439582+0000","last_undegraded":"2024-05-28T13:58:26.439582+0000","last_fullsized":"2024-05-28T13:58:26.439582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:03:59.917115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663850+0000","last_change":"2024-05-28T13:58:22.684218+0000","last_active":"2024-05-28T13:58:24.663850+0000","last_peered":"2024-05-28T13:58:24.663850+0000","last_clean":"2024-05-28T13:58:24.663850+0000","last_became_active":"2024-05-28T13:58:22.683984+0000","last_became_peered":"2024-05-28T13:58:22.683984+0000","last_unstale":"2024-05-28T13:58:24.663850+0000","last_undegraded":"2024-05-28T13:58:24.663850+0000","last_fullsized":"2024-05-28T13:58:24.663850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668661+0000","last_change":"2024-05-28T13:58:23.705226+0000","last_active":"2024-05-28T13:58:24.668661+0000","last_peered":"2024-05-28T13:58:24.668661+0000","last_clean":"2024-05-28T13:58:24.668661+0000","last_became_active":"2024-05-28T13:58:23.704265+0000","last_became_peered":"2024-05-28T13:58:23.704265+0000","last_unstale":"2024-05-28T13:58:24.668661+0000","last_undegraded":"2024-05-28T13:58:24.668661+0000","last_fullsized":"2024-05-28T13:58:24.668661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:36.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694803+0000","last_change":"2024-05-28T13:58:22.674273+0000","last_active":"2024-05-28T13:58:24.694803+0000","last_peered":"2024-05-28T13:58:24.694803+0000","last_clean":"2024-05-28T13:58:24.694803+0000","last_became_active":"2024-05-28T13:58:22.673984+0000","last_became_peered":"2024-05-28T13:58:22.673984+0000","last_unstale":"2024-05-28T13:58:24.694803+0000","last_undegraded":"2024-05-28T13:58:24.694803+0000","last_fullsized":"2024-05-28T13:58:24.694803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440523+0000","last_change":"2024-05-28T13:58:23.685763+0000","last_active":"2024-05-28T13:58:26.440523+0000","last_peered":"2024-05-28T13:58:26.440523+0000","last_clean":"2024-05-28T13:58:26.440523+0000","last_became_active":"2024-05-28T13:58:23.685512+0000","last_became_peered":"2024-05-28T13:58:23.685512+0000","last_unstale":"2024-05-28T13:58:26.440523+0000","last_undegraded":"2024-05-28T13:58:26.440523+0000","last_fullsized":"2024-05-28T13:58:26.440523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:43:20.340652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439949+0000","last_change":"2024-05-28T13:58:22.668047+0000","last_active":"2024-05-28T13:58:26.439949+0000","last_peered":"2024-05-28T13:58:26.439949+0000","last_clean":"2024-05-28T13:58:26.439949+0000","last_became_active":"2024-05-28T13:58:22.667747+0000","last_became_peered":"2024-05-28T13:58:22.667747+0000","last_unstale":"2024-05-28T13:58:26.439949+0000","last_undegraded":"2024-05-28T13:58:26.439949+0000","last_fullsized":"2024-05-28T13:58:26.439949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:52.489479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439199+0000","last_change":"2024-05-28T13:58:23.675653+0000","last_active":"2024-05-28T13:58:26.439199+0000","last_peered":"2024-05-28T13:58:26.439199+0000","last_clean":"2024-05-28T13:58:26.439199+0000","last_became_active":"2024-05-28T13:58:23.675371+0000","last_became_peered":"2024-05-28T13:58:23.675371+0000","last_unstale":"2024-05-28T13:58:26.439199+0000","last_undegraded":"2024-05-28T13:58:26.439199+0000","last_fullsized":"2024-05-28T13:58:26.439199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:10.036541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668378+0000","last_change":"2024-05-28T13:58:22.682942+0000","last_active":"2024-05-28T13:58:24.668378+0000","last_peered":"2024-05-28T13:58:24.668378+0000","last_clean":"2024-05-28T13:58:24.668378+0000","last_became_active":"2024-05-28T13:58:22.681946+0000","last_became_peered":"2024-05-28T13:58:22.681946+0000","last_unstale":"2024-05-28T13:58:24.668378+0000","last_undegraded":"2024-05-28T13:58:24.668378+0000","last_fullsized":"2024-05-28T13:58:24.668378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:07.005097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440496+0000","last_change":"2024-05-28T13:58:23.716761+0000","last_active":"2024-05-28T13:58:26.440496+0000","last_peered":"2024-05-28T13:58:26.440496+0000","last_clean":"2024-05-28T13:58:26.440496+0000","last_became_active":"2024-05-28T13:58:23.716488+0000","last_became_peered":"2024-05-28T13:58:23.716488+0000","last_unstale":"2024-05-28T13:58:26.440496+0000","last_undegraded":"2024-05-28T13:58:26.440496+0000","last_fullsized":"2024-05-28T13:58:26.440496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:16:14.092125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694386+0000","last_change":"2024-05-28T13:58:22.678451+0000","last_active":"2024-05-28T13:58:24.694386+0000","last_peered":"2024-05-28T13:58:24.694386+0000","last_clean":"2024-05-28T13:58:24.694386+0000","last_became_active":"2024-05-28T13:58:22.675695+0000","last_became_peered":"2024-05-28T13:58:22.675695+0000","last_unstale":"2024-05-28T13:58:24.694386+0000","last_undegraded":"2024-05-28T13:58:24.694386+0000","last_fullsized":"2024-05-28T13:58:24.694386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694443+0000","last_change":"2024-05-28T13:58:23.683800+0000","last_active":"2024-05-28T13:58:24.694443+0000","last_peered":"2024-05-28T13:58:24.694443+0000","last_clean":"2024-05-28T13:58:24.694443+0000","last_became_active":"2024-05-28T13:58:23.683208+0000","last_became_peered":"2024-05-28T13:58:23.683208+0000","last_unstale":"2024-05-28T13:58:24.694443+0000","last_undegraded":"2024-05-28T13:58:24.694443+0000","last_fullsized":"2024-05-28T13:58:24.694443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:46.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:25.415502+0000","last_change":"2024-05-28T13:58:22.676468+0000","last_active":"2024-05-28T13:58:25.415502+0000","last_peered":"2024-05-28T13:58:25.415502+0000","last_clean":"2024-05-28T13:58:25.415502+0000","last_became_active":"2024-05-28T13:58:22.676202+0000","last_became_peered":"2024-05-28T13:58:22.676202+0000","last_unstale":"2024-05-28T13:58:25.415502+0000","last_undegraded":"2024-05-28T13:58:25.415502+0000","last_fullsized":"2024-05-28T13:58:25.415502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:52.890121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440228+0000","last_change":"2024-05-28T13:58:23.717324+0000","last_active":"2024-05-28T13:58:26.440228+0000","last_peered":"2024-05-28T13:58:26.440228+0000","last_clean":"2024-05-28T13:58:26.440228+0000","last_became_active":"2024-05-28T13:58:23.716047+0000","last_became_peered":"2024-05-28T13:58:23.716047+0000","last_unstale":"2024-05-28T13:58:26.440228+0000","last_undegraded":"2024-05-28T13:58:26.440228+0000","last_fullsized":"2024-05-28T13:58:26.440228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:01:32.723660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668416+0000","last_change":"2024-05-28T13:58:22.671015+0000","last_active":"2024-05-28T13:58:24.668416+0000","last_peered":"2024-05-28T13:58:24.668416+0000","last_clean":"2024-05-28T13:58:24.668416+0000","last_became_active":"2024-05-28T13:58:22.663746+0000","last_became_peered":"2024-05-28T13:58:22.663746+0000","last_unstale":"2024-05-28T13:58:24.668416+0000","last_undegraded":"2024-05-28T13:58:24.668416+0000","last_fullsized":"2024-05-28T13:58:24.668416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:22:51.238198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440629+0000","last_change":"2024-05-28T13:58:23.705595+0000","last_active":"2024-05-28T13:58:26.440629+0000","last_peered":"2024-05-28T13:58:26.440629+0000","last_clean":"2024-05-28T13:58:26.440629+0000","last_became_active":"2024-05-28T13:58:23.705326+0000","last_became_peered":"2024-05-28T13:58:23.705326+0000","last_unstale":"2024-05-28T13:58:26.440629+0000","last_undegraded":"2024-05-28T13:58:26.440629+0000","last_fullsized":"2024-05-28T13:58:26.440629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:16.649099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:25.415746+0000","last_change":"2024-05-28T13:58:22.673480+0000","last_active":"2024-05-28T13:58:25.415746+0000","last_peered":"2024-05-28T13:58:25.415746+0000","last_clean":"2024-05-28T13:58:25.415746+0000","last_became_active":"2024-05-28T13:58:22.671634+0000","last_became_peered":"2024-05-28T13:58:22.671634+0000","last_unstale":"2024-05-28T13:58:25.415746+0000","last_undegraded":"2024-05-28T13:58:25.415746+0000","last_fullsized":"2024-05-28T13:58:25.415746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440947+0000","last_change":"2024-05-28T13:58:23.708965+0000","last_active":"2024-05-28T13:58:26.440947+0000","last_peered":"2024-05-28T13:58:26.440947+0000","last_clean":"2024-05-28T13:58:26.440947+0000","last_became_active":"2024-05-28T13:58:23.708298+0000","last_became_peered":"2024-05-28T13:58:23.708298+0000","last_unstale":"2024-05-28T13:58:26.440947+0000","last_undegraded":"2024-05-28T13:58:26.440947+0000","last_fullsized":"2024-05-28T13:58:26.440947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:29:08.273460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663963+0000","last_change":"2024-05-28T13:58:22.688102+0000","last_active":"2024-05-28T13:58:24.663963+0000","last_peered":"2024-05-28T13:58:24.663963+0000","last_clean":"2024-05-28T13:58:24.663963+0000","last_became_active":"2024-05-28T13:58:22.686031+0000","last_became_peered":"2024-05-28T13:58:22.686031+0000","last_unstale":"2024-05-28T13:58:24.663963+0000","last_undegraded":"2024-05-28T13:58:24.663963+0000","last_fullsized":"2024-05-28T13:58:24.663963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668988+0000","last_change":"2024-05-28T13:58:23.690696+0000","last_active":"2024-05-28T13:58:24.668988+0000","last_peered":"2024-05-28T13:58:24.668988+0000","last_clean":"2024-05-28T13:58:24.668988+0000","last_became_active":"2024-05-28T13:58:23.689615+0000","last_became_peered":"2024-05-28T13:58:23.689615+0000","last_unstale":"2024-05-28T13:58:24.668988+0000","last_undegraded":"2024-05-28T13:58:24.668988+0000","last_fullsized":"2024-05-28T13:58:24.668988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:40:43.691282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:25.415621+0000","last_change":"2024-05-28T13:58:22.674653+0000","last_active":"2024-05-28T13:58:25.415621+0000","last_peered":"2024-05-28T13:58:25.415621+0000","last_clean":"2024-05-28T13:58:25.415621+0000","last_became_active":"2024-05-28T13:58:22.674308+0000","last_became_peered":"2024-05-28T13:58:22.674308+0000","last_unstale":"2024-05-28T13:58:25.415621+0000","last_undegraded":"2024-05-28T13:58:25.415621+0000","last_fullsized":"2024-05-28T13:58:25.415621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440733+0000","last_change":"2024-05-28T13:58:23.702244+0000","last_active":"2024-05-28T13:58:26.440733+0000","last_peered":"2024-05-28T13:58:26.440733+0000","last_clean":"2024-05-28T13:58:26.440733+0000","last_became_active":"2024-05-28T13:58:23.701858+0000","last_became_peered":"2024-05-28T13:58:23.701858+0000","last_unstale":"2024-05-28T13:58:26.440733+0000","last_undegraded":"2024-05-28T13:58:26.440733+0000","last_fullsized":"2024-05-28T13:58:26.440733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:40:43.691282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440128+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T13:58:26.440128+0000","last_peered":"2024-05-28T13:58:26.440128+0000","last_clean":"2024-05-28T13:58:26.440128+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T13:58:26.440128+0000","last_undegraded":"2024-05-28T13:58:26.440128+0000","last_fullsized":"2024-05-28T13:58:26.440128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668007+0000","last_change":"2024-05-28T13:58:22.668236+0000","last_active":"2024-05-28T13:58:24.668007+0000","last_peered":"2024-05-28T13:58:24.668007+0000","last_clean":"2024-05-28T13:58:24.668007+0000","last_became_active":"2024-05-28T13:58:22.667928+0000","last_became_peered":"2024-05-28T13:58:22.667928+0000","last_unstale":"2024-05-28T13:58:24.668007+0000","last_undegraded":"2024-05-28T13:58:24.668007+0000","last_fullsized":"2024-05-28T13:58:24.668007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439204+0000","last_change":"2024-05-28T13:58:23.701173+0000","last_active":"2024-05-28T13:58:26.439204+0000","last_peered":"2024-05-28T13:58:26.439204+0000","last_clean":"2024-05-28T13:58:26.439204+0000","last_became_active":"2024-05-28T13:58:23.700830+0000","last_became_peered":"2024-05-28T13:58:23.700830+0000","last_unstale":"2024-05-28T13:58:26.439204+0000","last_undegraded":"2024-05-28T13:58:26.439204+0000","last_fullsized":"2024-05-28T13:58:26.439204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:01:32.723660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440029+0000","last_change":"2024-05-28T13:58:22.687098+0000","last_active":"2024-05-28T13:58:26.440029+0000","last_peered":"2024-05-28T13:58:26.440029+0000","last_clean":"2024-05-28T13:58:26.440029+0000","last_became_active":"2024-05-28T13:58:22.686751+0000","last_became_peered":"2024-05-28T13:58:22.686751+0000","last_unstale":"2024-05-28T13:58:26.440029+0000","last_undegraded":"2024-05-28T13:58:26.440029+0000","last_fullsized":"2024-05-28T13:58:26.440029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:35.326847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.438801+0000","last_change":"2024-05-28T13:58:23.709225+0000","last_active":"2024-05-28T13:58:26.438801+0000","last_peered":"2024-05-28T13:58:26.438801+0000","last_clean":"2024-05-28T13:58:26.438801+0000","last_became_active":"2024-05-28T13:58:23.708785+0000","last_became_peered":"2024-05-28T13:58:23.708785+0000","last_unstale":"2024-05-28T13:58:26.438801+0000","last_undegraded":"2024-05-28T13:58:26.438801+0000","last_fullsized":"2024-05-28T13:58:26.438801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:12:43.447237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:25.415595+0000","last_change":"2024-05-28T13:58:22.667443+0000","last_active":"2024-05-28T13:58:25.415595+0000","last_peered":"2024-05-28T13:58:25.415595+0000","last_clean":"2024-05-28T13:58:25.415595+0000","last_became_active":"2024-05-28T13:58:22.666967+0000","last_became_peered":"2024-05-28T13:58:22.666967+0000","last_unstale":"2024-05-28T13:58:25.415595+0000","last_undegraded":"2024-05-28T13:58:25.415595+0000","last_fullsized":"2024-05-28T13:58:25.415595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663665+0000","last_change":"2024-05-28T13:58:23.680134+0000","last_active":"2024-05-28T13:58:24.663665+0000","last_peered":"2024-05-28T13:58:24.663665+0000","last_clean":"2024-05-28T13:58:24.663665+0000","last_became_active":"2024-05-28T13:58:23.679912+0000","last_became_peered":"2024-05-28T13:58:23.679912+0000","last_unstale":"2024-05-28T13:58:24.663665+0000","last_undegraded":"2024-05-28T13:58:24.663665+0000","last_fullsized":"2024-05-28T13:58:24.663665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:03:59.917115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668429+0000","last_change":"2024-05-28T13:58:22.683368+0000","last_active":"2024-05-28T13:58:24.668429+0000","last_peered":"2024-05-28T13:58:24.668429+0000","last_clean":"2024-05-28T13:58:24.668429+0000","last_became_active":"2024-05-28T13:58:22.683102+0000","last_became_peered":"2024-05-28T13:58:22.683102+0000","last_unstale":"2024-05-28T13:58:24.668429+0000","last_undegraded":"2024-05-28T13:58:24.668429+0000","last_fullsized":"2024-05-28T13:58:24.668429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.669031+0000","last_change":"2024-05-28T13:58:23.707964+0000","last_active":"2024-05-28T13:58:24.669031+0000","last_peered":"2024-05-28T13:58:24.669031+0000","last_clean":"2024-05-28T13:58:24.669031+0000","last_became_active":"2024-05-28T13:58:23.707430+0000","last_became_peered":"2024-05-28T13:58:23.707430+0000","last_unstale":"2024-05-28T13:58:24.669031+0000","last_undegraded":"2024-05-28T13:58:24.669031+0000","last_fullsized":"2024-05-28T13:58:24.669031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:56.904188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663715+0000","last_change":"2024-05-28T13:58:22.676423+0000","last_active":"2024-05-28T13:58:24.663715+0000","last_peered":"2024-05-28T13:58:24.663715+0000","last_clean":"2024-05-28T13:58:24.663715+0000","last_became_active":"2024-05-28T13:58:22.675960+0000","last_became_peered":"2024-05-28T13:58:22.675960+0000","last_unstale":"2024-05-28T13:58:24.663715+0000","last_undegraded":"2024-05-28T13:58:24.663715+0000","last_fullsized":"2024-05-28T13:58:24.663715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:02.116873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668838+0000","last_change":"2024-05-28T13:58:23.692003+0000","last_active":"2024-05-28T13:58:24.668838+0000","last_peered":"2024-05-28T13:58:24.668838+0000","last_clean":"2024-05-28T13:58:24.668838+0000","last_became_active":"2024-05-28T13:58:23.689087+0000","last_became_peered":"2024-05-28T13:58:23.689087+0000","last_unstale":"2024-05-28T13:58:24.668838+0000","last_undegraded":"2024-05-28T13:58:24.668838+0000","last_fullsized":"2024-05-28T13:58:24.668838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:32:51.191091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440859+0000","last_change":"2024-05-28T13:58:22.686829+0000","last_active":"2024-05-28T13:58:26.440859+0000","last_peered":"2024-05-28T13:58:26.440859+0000","last_clean":"2024-05-28T13:58:26.440859+0000","last_became_active":"2024-05-28T13:58:22.686497+0000","last_became_peered":"2024-05-28T13:58:22.686497+0000","last_unstale":"2024-05-28T13:58:26.440859+0000","last_undegraded":"2024-05-28T13:58:26.440859+0000","last_fullsized":"2024-05-28T13:58:26.440859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:58.710455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663623+0000","last_change":"2024-05-28T13:58:23.684658+0000","last_active":"2024-05-28T13:58:24.663623+0000","last_peered":"2024-05-28T13:58:24.663623+0000","last_clean":"2024-05-28T13:58:24.663623+0000","last_became_active":"2024-05-28T13:58:23.682872+0000","last_became_peered":"2024-05-28T13:58:23.682872+0000","last_unstale":"2024-05-28T13:58:24.663623+0000","last_undegraded":"2024-05-28T13:58:24.663623+0000","last_fullsized":"2024-05-28T13:58:24.663623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:22.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663835+0000","last_change":"2024-05-28T13:58:22.682664+0000","last_active":"2024-05-28T13:58:24.663835+0000","last_peered":"2024-05-28T13:58:24.663835+0000","last_clean":"2024-05-28T13:58:24.663835+0000","last_became_active":"2024-05-28T13:58:22.682422+0000","last_became_peered":"2024-05-28T13:58:22.682422+0000","last_unstale":"2024-05-28T13:58:24.663835+0000","last_undegraded":"2024-05-28T13:58:24.663835+0000","last_fullsized":"2024-05-28T13:58:24.663835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:07.005097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439642+0000","last_change":"2024-05-28T13:58:23.678389+0000","last_active":"2024-05-28T13:58:26.439642+0000","last_peered":"2024-05-28T13:58:26.439642+0000","last_clean":"2024-05-28T13:58:26.439642+0000","last_became_active":"2024-05-28T13:58:23.678081+0000","last_became_peered":"2024-05-28T13:58:23.678081+0000","last_unstale":"2024-05-28T13:58:26.439642+0000","last_undegraded":"2024-05-28T13:58:26.439642+0000","last_fullsized":"2024-05-28T13:58:26.439642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:22:38.773556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694827+0000","last_change":"2024-05-28T13:58:22.680245+0000","last_active":"2024-05-28T13:58:24.694827+0000","last_peered":"2024-05-28T13:58:24.694827+0000","last_clean":"2024-05-28T13:58:24.694827+0000","last_became_active":"2024-05-28T13:58:22.679323+0000","last_became_peered":"2024-05-28T13:58:22.679323+0000","last_unstale":"2024-05-28T13:58:24.694827+0000","last_undegraded":"2024-05-28T13:58:24.694827+0000","last_fullsized":"2024-05-28T13:58:24.694827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:07.005097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439847+0000","last_change":"2024-05-28T13:58:23.677850+0000","last_active":"2024-05-28T13:58:26.439847+0000","last_peered":"2024-05-28T13:58:26.439847+0000","last_clean":"2024-05-28T13:58:26.439847+0000","last_became_active":"2024-05-28T13:58:23.677654+0000","last_became_peered":"2024-05-28T13:58:23.677654+0000","last_unstale":"2024-05-28T13:58:26.439847+0000","last_undegraded":"2024-05-28T13:58:26.439847+0000","last_fullsized":"2024-05-28T13:58:26.439847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:49.556769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440004+0000","last_change":"2024-05-28T13:58:22.682426+0000","last_active":"2024-05-28T13:58:26.440004+0000","last_peered":"2024-05-28T13:58:26.440004+0000","last_clean":"2024-05-28T13:58:26.440004+0000","last_became_active":"2024-05-28T13:58:22.682099+0000","last_became_peered":"2024-05-28T13:58:22.682099+0000","last_unstale":"2024-05-28T13:58:26.440004+0000","last_undegraded":"2024-05-28T13:58:26.440004+0000","last_fullsized":"2024-05-28T13:58:26.440004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.667930+0000","last_change":"2024-05-28T13:58:23.706882+0000","last_active":"2024-05-28T13:58:24.667930+0000","last_peered":"2024-05-28T13:58:24.667930+0000","last_clean":"2024-05-28T13:58:24.667930+0000","last_became_active":"2024-05-28T13:58:23.706588+0000","last_became_peered":"2024-05-28T13:58:23.706588+0000","last_unstale":"2024-05-28T13:58:24.667930+0000","last_undegraded":"2024-05-28T13:58:24.667930+0000","last_fullsized":"2024-05-28T13:58:24.667930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:08.011020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663707+0000","last_change":"2024-05-28T13:58:22.678682+0000","last_active":"2024-05-28T13:58:24.663707+0000","last_peered":"2024-05-28T13:58:24.663707+0000","last_clean":"2024-05-28T13:58:24.663707+0000","last_became_active":"2024-05-28T13:58:22.678233+0000","last_became_peered":"2024-05-28T13:58:22.678233+0000","last_unstale":"2024-05-28T13:58:24.663707+0000","last_undegraded":"2024-05-28T13:58:24.663707+0000","last_fullsized":"2024-05-28T13:58:24.663707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:35.326847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439379+0000","last_change":"2024-05-28T13:58:23.669871+0000","last_active":"2024-05-28T13:58:26.439379+0000","last_peered":"2024-05-28T13:58:26.439379+0000","last_clean":"2024-05-28T13:58:26.439379+0000","last_became_active":"2024-05-28T13:58:23.669643+0000","last_became_peered":"2024-05-28T13:58:23.669643+0000","last_unstale":"2024-05-28T13:58:26.439379+0000","last_undegraded":"2024-05-28T13:58:26.439379+0000","last_fullsized":"2024-05-28T13:58:26.439379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:02.981053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663683+0000","last_change":"2024-05-28T13:58:22.680769+0000","last_active":"2024-05-28T13:58:24.663683+0000","last_peered":"2024-05-28T13:58:24.663683+0000","last_clean":"2024-05-28T13:58:24.663683+0000","last_became_active":"2024-05-28T13:58:22.679781+0000","last_became_peered":"2024-05-28T13:58:22.679781+0000","last_unstale":"2024-05-28T13:58:24.663683+0000","last_undegraded":"2024-05-28T13:58:24.663683+0000","last_fullsized":"2024-05-28T13:58:24.663683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439570+0000","last_change":"2024-05-28T13:58:23.683534+0000","last_active":"2024-05-28T13:58:26.439570+0000","last_peered":"2024-05-28T13:58:26.439570+0000","last_clean":"2024-05-28T13:58:26.439570+0000","last_became_active":"2024-05-28T13:58:23.683185+0000","last_became_peered":"2024-05-28T13:58:23.683185+0000","last_unstale":"2024-05-28T13:58:26.439570+0000","last_undegraded":"2024-05-28T13:58:26.439570+0000","last_fullsized":"2024-05-28T13:58:26.439570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:31.530501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440541+0000","last_change":"2024-05-28T13:58:22.683550+0000","last_active":"2024-05-28T13:58:26.440541+0000","last_peered":"2024-05-28T13:58:26.440541+0000","last_clean":"2024-05-28T13:58:26.440541+0000","last_became_active":"2024-05-28T13:58:22.683218+0000","last_became_peered":"2024-05-28T13:58:22.683218+0000","last_unstale":"2024-05-28T13:58:26.440541+0000","last_undegraded":"2024-05-28T13:58:26.440541+0000","last_fullsized":"2024-05-28T13:58:26.440541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:12.464954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694678+0000","last_change":"2024-05-28T13:58:23.709888+0000","last_active":"2024-05-28T13:58:24.694678+0000","last_peered":"2024-05-28T13:58:24.694678+0000","last_clean":"2024-05-28T13:58:24.694678+0000","last_became_active":"2024-05-28T13:58:23.709658+0000","last_became_peered":"2024-05-28T13:58:23.709658+0000","last_unstale":"2024-05-28T13:58:24.694678+0000","last_undegraded":"2024-05-28T13:58:24.694678+0000","last_fullsized":"2024-05-28T13:58:24.694678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:22.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668669+0000","last_change":"2024-05-28T13:58:23.710526+0000","last_active":"2024-05-28T13:58:24.668669+0000","last_peered":"2024-05-28T13:58:24.668669+0000","last_clean":"2024-05-28T13:58:24.668669+0000","last_became_active":"2024-05-28T13:58:23.710118+0000","last_became_peered":"2024-05-28T13:58:23.710118+0000","last_unstale":"2024-05-28T13:58:24.668669+0000","last_undegraded":"2024-05-28T13:58:24.668669+0000","last_fullsized":"2024-05-28T13:58:24.668669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:01:32.723660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440228+0000","last_change":"2024-05-28T13:58:22.681209+0000","last_active":"2024-05-28T13:58:26.440228+0000","last_peered":"2024-05-28T13:58:26.440228+0000","last_clean":"2024-05-28T13:58:26.440228+0000","last_became_active":"2024-05-28T13:58:22.680889+0000","last_became_peered":"2024-05-28T13:58:22.680889+0000","last_unstale":"2024-05-28T13:58:26.440228+0000","last_undegraded":"2024-05-28T13:58:26.440228+0000","last_fullsized":"2024-05-28T13:58:26.440228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:02.116873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668891+0000","last_change":"2024-05-28T13:58:23.710507+0000","last_active":"2024-05-28T13:58:24.668891+0000","last_peered":"2024-05-28T13:58:24.668891+0000","last_clean":"2024-05-28T13:58:24.668891+0000","last_became_active":"2024-05-28T13:58:23.710104+0000","last_became_peered":"2024-05-28T13:58:23.710104+0000","last_unstale":"2024-05-28T13:58:24.668891+0000","last_undegraded":"2024-05-28T13:58:24.668891+0000","last_fullsized":"2024-05-28T13:58:24.668891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:16.649099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440682+0000","last_change":"2024-05-28T13:58:22.688976+0000","last_active":"2024-05-28T13:58:26.440682+0000","last_peered":"2024-05-28T13:58:26.440682+0000","last_clean":"2024-05-28T13:58:26.440682+0000","last_became_active":"2024-05-28T13:58:22.688607+0000","last_became_peered":"2024-05-28T13:58:22.688607+0000","last_unstale":"2024-05-28T13:58:26.440682+0000","last_undegraded":"2024-05-28T13:58:26.440682+0000","last_fullsized":"2024-05-28T13:58:26.440682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:55.898265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439430+0000","last_change":"2024-05-28T13:58:23.695036+0000","last_active":"2024-05-28T13:58:26.439430+0000","last_peered":"2024-05-28T13:58:26.439430+0000","last_clean":"2024-05-28T13:58:26.439430+0000","last_became_active":"2024-05-28T13:58:23.692922+0000","last_became_peered":"2024-05-28T13:58:23.692922+0000","last_unstale":"2024-05-28T13:58:26.439430+0000","last_undegraded":"2024-05-28T13:58:26.439430+0000","last_fullsized":"2024-05-28T13:58:26.439430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:22.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439495+0000","last_change":"2024-05-28T13:58:22.684711+0000","last_active":"2024-05-28T13:58:26.439495+0000","last_peered":"2024-05-28T13:58:26.439495+0000","last_clean":"2024-05-28T13:58:26.439495+0000","last_became_active":"2024-05-28T13:58:22.680359+0000","last_became_peered":"2024-05-28T13:58:22.680359+0000","last_unstale":"2024-05-28T13:58:26.439495+0000","last_undegraded":"2024-05-28T13:58:26.439495+0000","last_fullsized":"2024-05-28T13:58:26.439495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439311+0000","last_change":"2024-05-28T13:58:23.690282+0000","last_active":"2024-05-28T13:58:26.439311+0000","last_peered":"2024-05-28T13:58:26.439311+0000","last_clean":"2024-05-28T13:58:26.439311+0000","last_became_active":"2024-05-28T13:58:23.689975+0000","last_became_peered":"2024-05-28T13:58:23.689975+0000","last_unstale":"2024-05-28T13:58:26.439311+0000","last_undegraded":"2024-05-28T13:58:26.439311+0000","last_fullsized":"2024-05-28T13:58:26.439311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:23.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439751+0000","last_change":"2024-05-28T13:58:22.679511+0000","last_active":"2024-05-28T13:58:26.439751+0000","last_peered":"2024-05-28T13:58:26.439751+0000","last_clean":"2024-05-28T13:58:26.439751+0000","last_became_active":"2024-05-28T13:58:22.679207+0000","last_became_peered":"2024-05-28T13:58:22.679207+0000","last_unstale":"2024-05-28T13:58:26.439751+0000","last_undegraded":"2024-05-28T13:58:26.439751+0000","last_fullsized":"2024-05-28T13:58:26.439751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T13:58:32.492 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T13:58:32.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.685+0000 7f95f9000640 1 Processor -- start 2024-05-28T13:58:32.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95f9000640 1 -- start start 2024-05-28T13:58:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95f9000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f95f406ff10 0x7f95f415e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95f9000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95f40703a0 0x7f95f415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95f9000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95f40d0660 con 0x7f95f4070800 2024-05-28T13:58:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95f9000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f95f409ba60 con 0x7f95f406ff10 2024-05-28T13:58:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95f9000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f95f40d0940 con 0x7f95f40703a0 2024-05-28T13:58:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95f3400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f95f4070800 0x7f95f4070be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60608/0 (socket says 172.21.15.42:60608) 2024-05-28T13:58:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95f3400640 1 -- 172.21.15.42:0/2657493280 learned_addr learned my addr 172.21.15.42:0/2657493280 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:32.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95eaa00640 1 --2- 172.21.15.42:0/2657493280 >> v2:172.21.15.63:3300/0 conn(0x7f95f406ff10 0x7f95f415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:32.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95f3e00640 1 --2- 172.21.15.42:0/2657493280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95f40703a0 0x7f95f415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:32.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95f2a00640 1 -- 172.21.15.42:0/2657493280 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2505334059 0 0) 0x7f95f40d0660 con 0x7f95f4070800 2024-05-28T13:58:32.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95f2a00640 1 -- 172.21.15.42:0/2657493280 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95d8003530 con 0x7f95f4070800 2024-05-28T13:58:32.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95eaa00640 1 -- 172.21.15.42:0/2657493280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95f40703a0 msgr2=0x7f95f415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:32.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95eaa00640 1 --2- 172.21.15.42:0/2657493280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95f40703a0 0x7f95f415ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95eaa00640 1 -- 172.21.15.42:0/2657493280 >> v1:172.21.15.42:6789/0 conn(0x7f95f4070800 legacy=0x7f95f4070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.689+0000 7f95eaa00640 1 -- 172.21.15.42:0/2657493280 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f40d0c20 con 0x7f95f406ff10 2024-05-28T13:58:32.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.693+0000 7f95eaa00640 1 --2- 172.21.15.42:0/2657493280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95f406ff10 0x7f95f415e910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0097f0 tx=0x7f95dc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c3c835e60b815e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:32.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.693+0000 7f95f2a00640 1 -- 172.21.15.42:0/2657493280 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0b2070 con 0x7f95f406ff10 2024-05-28T13:58:32.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.693+0000 7f95f2a00640 1 -- 172.21.15.42:0/2657493280 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95dc0ad640 con 0x7f95f406ff10 2024-05-28T13:58:32.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.693+0000 7f95f2a00640 1 -- 172.21.15.42:0/2657493280 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0ada10 con 0x7f95f406ff10 2024-05-28T13:58:32.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.693+0000 7f95f9000640 1 -- 172.21.15.42:0/2657493280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95f406ff10 msgr2=0x7f95f415e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.693+0000 7f95f9000640 1 --2- 172.21.15.42:0/2657493280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95f406ff10 0x7f95f415e910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0097f0 tx=0x7f95dc01c4a0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.693+0000 7f95f9000640 1 -- 172.21.15.42:0/2657493280 shutdown_connections 2024-05-28T13:58:32.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.693+0000 7f95f9000640 1 --2- 172.21.15.42:0/2657493280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95f40703a0 0x7f95f415ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.693+0000 7f95f9000640 1 --2- 172.21.15.42:0/2657493280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95f406ff10 0x7f95f415e910 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.693+0000 7f95f9000640 1 -- 172.21.15.42:0/2657493280 >> 172.21.15.42:0/2657493280 conn(0x7f95f40b3dd0 msgr2=0x7f95f40b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:32.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.693+0000 7f95f9000640 1 -- 172.21.15.42:0/2657493280 shutdown_connections 2024-05-28T13:58:32.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.693+0000 7f95f9000640 1 -- 172.21.15.42:0/2657493280 wait complete. 2024-05-28T13:58:32.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.693+0000 7f95f9000640 1 Processor -- start 2024-05-28T13:58:32.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f9000640 1 -- start start 2024-05-28T13:58:32.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f9000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95f406ff10 0x7f95f4152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f9000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95f40703a0 0x7f95f415d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f9000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95f4070800 0x7f95f4159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f9000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f95f40b1a10 con 0x7f95f406ff10 2024-05-28T13:58:32.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f9000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f95f40b1890 con 0x7f95f40703a0 2024-05-28T13:58:32.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f9000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f95f416b5d0 con 0x7f95f4070800 2024-05-28T13:58:32.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95f406ff10 0x7f95f4152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:32.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95f406ff10 0x7f95f4152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38760/0 (socket says 172.21.15.42:38760) 2024-05-28T13:58:32.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f3400640 1 -- 172.21.15.42:0/3497612175 learned_addr learned my addr 172.21.15.42:0/3497612175 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:32.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f3400640 1 -- 172.21.15.42:0/3497612175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95f4070800 msgr2=0x7f95f4159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f3400640 1 --2- 172.21.15.42:0/3497612175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95f4070800 0x7f95f4159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f3400640 1 -- 172.21.15.42:0/3497612175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95f40703a0 msgr2=0x7f95f415d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f3400640 1 --2- 172.21.15.42:0/3497612175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95f40703a0 0x7f95f415d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f3400640 1 -- 172.21.15.42:0/3497612175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f40cb390 con 0x7f95f406ff10 2024-05-28T13:58:32.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f3400640 1 --2- 172.21.15.42:0/3497612175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95f406ff10 0x7f95f4152640 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f95e400caa0 tx=0x7f95e400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:32.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95ebe00640 1 -- 172.21.15.42:0/3497612175 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4017070 con 0x7f95f406ff10 2024-05-28T13:58:32.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95ebe00640 1 -- 172.21.15.42:0/3497612175 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e400ac00 con 0x7f95f406ff10 2024-05-28T13:58:32.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95ebe00640 1 -- 172.21.15.42:0/3497612175 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4012400 con 0x7f95f406ff10 2024-05-28T13:58:32.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f9000640 1 -- 172.21.15.42:0/3497612175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f40cb670 con 0x7f95f406ff10 2024-05-28T13:58:32.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f9000640 1 -- 172.21.15.42:0/3497612175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f40cbc30 con 0x7f95f406ff10 2024-05-28T13:58:32.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.697+0000 7f95f9000640 1 -- 172.21.15.42:0/3497612175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f404fcf0 con 0x7f95f406ff10 2024-05-28T13:58:32.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.701+0000 7f95ebe00640 1 -- 172.21.15.42:0/3497612175 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f95e401a070 con 0x7f95f406ff10 2024-05-28T13:58:32.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.701+0000 7f95ebe00640 1 --2- 172.21.15.42:0/3497612175 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f95d00683b0 0x7f95d006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:32.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.701+0000 7f95ebe00640 1 -- 172.21.15.42:0/3497612175 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f95e40867b0 con 0x7f95f406ff10 2024-05-28T13:58:32.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.701+0000 7f95eaa00640 1 --2- 172.21.15.42:0/3497612175 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f95d00683b0 0x7f95d006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:32.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.701+0000 7f95eaa00640 1 --2- 172.21.15.42:0/3497612175 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f95d00683b0 0x7f95d006a870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0af000 tx=0x7f95dc004740 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:32.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.705+0000 7f95ebe00640 1 -- 172.21.15.42:0/3497612175 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f95e4056d70 con 0x7f95f406ff10 2024-05-28T13:58:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.869+0000 7f95f9000640 1 -- 172.21.15.42:0/3497612175 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95f40d1d50 con 0x7f95d00683b0 2024-05-28T13:58:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.873+0000 7f95ebe00640 1 -- 172.21.15.42:0/3497612175 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f95f40d1d50 con 0x7f95d00683b0 2024-05-28T13:58:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:32.865 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T13:58:32.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.877+0000 7f95f9000640 1 -- 172.21.15.42:0/3497612175 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f95d00683b0 msgr2=0x7f95d006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.877+0000 7f95f9000640 1 --2- 172.21.15.42:0/3497612175 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f95d00683b0 0x7f95d006a870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0af000 tx=0x7f95dc004740 comp rx=0 tx=0).stop 2024-05-28T13:58:32.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.877+0000 7f95f9000640 1 -- 172.21.15.42:0/3497612175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95f406ff10 msgr2=0x7f95f4152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:32.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.877+0000 7f95f9000640 1 --2- 172.21.15.42:0/3497612175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95f406ff10 0x7f95f4152640 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f95e400caa0 tx=0x7f95e400cf60 comp rx=0 tx=0).stop 2024-05-28T13:58:32.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.877+0000 7f95f9000640 1 -- 172.21.15.42:0/3497612175 shutdown_connections 2024-05-28T13:58:32.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.877+0000 7f95f9000640 1 --2- 172.21.15.42:0/3497612175 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f95d00683b0 0x7f95d006a870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.877+0000 7f95f9000640 1 --2- 172.21.15.42:0/3497612175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95f4070800 0x7f95f4159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.877+0000 7f95f9000640 1 --2- 172.21.15.42:0/3497612175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95f40703a0 0x7f95f415d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.877+0000 7f95f9000640 1 --2- 172.21.15.42:0/3497612175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95f406ff10 0x7f95f4152640 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.877+0000 7f95f9000640 1 -- 172.21.15.42:0/3497612175 >> 172.21.15.42:0/3497612175 conn(0x7f95f40b3dd0 msgr2=0x7f95f406d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.877+0000 7f95f9000640 1 -- 172.21.15.42:0/3497612175 shutdown_connections 2024-05-28T13:58:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:32.877+0000 7f95f9000640 1 -- 172.21.15.42:0/3497612175 wait complete. 2024-05-28T13:58:32.882 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-05-28T13:58:32.857780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235740,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738980,"statfs":{"total":773094113280,"available":772852715520,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.198003"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440826+0000","last_change":"2024-05-28T13:58:23.690106+0000","last_active":"2024-05-28T13:58:26.440826+0000","last_peered":"2024-05-28T13:58:26.440826+0000","last_clean":"2024-05-28T13:58:26.440826+0000","last_became_active":"2024-05-28T13:58:23.689597+0000","last_became_peered":"2024-05-28T13:58:23.689597+0000","last_unstale":"2024-05-28T13:58:26.440826+0000","last_undegraded":"2024-05-28T13:58:26.440826+0000","last_fullsized":"2024-05-28T13:58:26.440826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:32:51.191091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663757+0000","last_change":"2024-05-28T13:58:23.674151+0000","last_active":"2024-05-28T13:58:24.663757+0000","last_peered":"2024-05-28T13:58:24.663757+0000","last_clean":"2024-05-28T13:58:24.663757+0000","last_became_active":"2024-05-28T13:58:23.673848+0000","last_became_peered":"2024-05-28T13:58:23.673848+0000","last_unstale":"2024-05-28T13:58:24.663757+0000","last_undegraded":"2024-05-28T13:58:24.663757+0000","last_fullsized":"2024-05-28T13:58:24.663757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:23.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439412+0000","last_change":"2024-05-28T13:58:23.685573+0000","last_active":"2024-05-28T13:58:26.439412+0000","last_peered":"2024-05-28T13:58:26.439412+0000","last_clean":"2024-05-28T13:58:26.439412+0000","last_became_active":"2024-05-28T13:58:23.685243+0000","last_became_peered":"2024-05-28T13:58:23.685243+0000","last_unstale":"2024-05-28T13:58:26.439412+0000","last_undegraded":"2024-05-28T13:58:26.439412+0000","last_fullsized":"2024-05-28T13:58:26.439412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:46.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668781+0000","last_change":"2024-05-28T13:58:23.706150+0000","last_active":"2024-05-28T13:58:24.668781+0000","last_peered":"2024-05-28T13:58:24.668781+0000","last_clean":"2024-05-28T13:58:24.668781+0000","last_became_active":"2024-05-28T13:58:23.689938+0000","last_became_peered":"2024-05-28T13:58:23.689938+0000","last_unstale":"2024-05-28T13:58:24.668781+0000","last_undegraded":"2024-05-28T13:58:24.668781+0000","last_fullsized":"2024-05-28T13:58:24.668781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:16:14.092125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440405+0000","last_change":"2024-05-28T13:58:23.683882+0000","last_active":"2024-05-28T13:58:26.440405+0000","last_peered":"2024-05-28T13:58:26.440405+0000","last_clean":"2024-05-28T13:58:26.440405+0000","last_became_active":"2024-05-28T13:58:23.683368+0000","last_became_peered":"2024-05-28T13:58:23.683368+0000","last_unstale":"2024-05-28T13:58:26.440405+0000","last_undegraded":"2024-05-28T13:58:26.440405+0000","last_fullsized":"2024-05-28T13:58:26.440405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:23:14.222046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668075+0000","last_change":"2024-05-28T13:58:23.695111+0000","last_active":"2024-05-28T13:58:24.668075+0000","last_peered":"2024-05-28T13:58:24.668075+0000","last_clean":"2024-05-28T13:58:24.668075+0000","last_became_active":"2024-05-28T13:58:23.688505+0000","last_became_peered":"2024-05-28T13:58:23.688505+0000","last_unstale":"2024-05-28T13:58:24.668075+0000","last_undegraded":"2024-05-28T13:58:24.668075+0000","last_fullsized":"2024-05-28T13:58:24.668075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:49.556769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440203+0000","last_change":"2024-05-28T13:58:23.684793+0000","last_active":"2024-05-28T13:58:26.440203+0000","last_peered":"2024-05-28T13:58:26.440203+0000","last_clean":"2024-05-28T13:58:26.440203+0000","last_became_active":"2024-05-28T13:58:23.684500+0000","last_became_peered":"2024-05-28T13:58:23.684500+0000","last_unstale":"2024-05-28T13:58:26.440203+0000","last_undegraded":"2024-05-28T13:58:26.440203+0000","last_fullsized":"2024-05-28T13:58:26.440203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:27:53.809481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439015+0000","last_change":"2024-05-28T13:58:23.688670+0000","last_active":"2024-05-28T13:58:26.439015+0000","last_peered":"2024-05-28T13:58:26.439015+0000","last_clean":"2024-05-28T13:58:26.439015+0000","last_became_active":"2024-05-28T13:58:23.688243+0000","last_became_peered":"2024-05-28T13:58:23.688243+0000","last_unstale":"2024-05-28T13:58:26.439015+0000","last_undegraded":"2024-05-28T13:58:26.439015+0000","last_fullsized":"2024-05-28T13:58:26.439015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:10.036541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440836+0000","last_change":"2024-05-28T13:58:23.702434+0000","last_active":"2024-05-28T13:58:26.440836+0000","last_peered":"2024-05-28T13:58:26.440836+0000","last_clean":"2024-05-28T13:58:26.440836+0000","last_became_active":"2024-05-28T13:58:23.701857+0000","last_became_peered":"2024-05-28T13:58:23.701857+0000","last_unstale":"2024-05-28T13:58:26.440836+0000","last_undegraded":"2024-05-28T13:58:26.440836+0000","last_fullsized":"2024-05-28T13:58:26.440836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:41:47.900159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694909+0000","last_change":"2024-05-28T13:58:23.709688+0000","last_active":"2024-05-28T13:58:24.694909+0000","last_peered":"2024-05-28T13:58:24.694909+0000","last_clean":"2024-05-28T13:58:24.694909+0000","last_became_active":"2024-05-28T13:58:23.709431+0000","last_became_peered":"2024-05-28T13:58:23.709431+0000","last_unstale":"2024-05-28T13:58:24.694909+0000","last_undegraded":"2024-05-28T13:58:24.694909+0000","last_fullsized":"2024-05-28T13:58:24.694909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:49.556769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694612+0000","last_change":"2024-05-28T13:58:23.701527+0000","last_active":"2024-05-28T13:58:24.694612+0000","last_peered":"2024-05-28T13:58:24.694612+0000","last_clean":"2024-05-28T13:58:24.694612+0000","last_became_active":"2024-05-28T13:58:23.701160+0000","last_became_peered":"2024-05-28T13:58:23.701160+0000","last_unstale":"2024-05-28T13:58:24.694612+0000","last_undegraded":"2024-05-28T13:58:24.694612+0000","last_fullsized":"2024-05-28T13:58:24.694612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:03:59.917115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668501+0000","last_change":"2024-05-28T13:58:23.707766+0000","last_active":"2024-05-28T13:58:24.668501+0000","last_peered":"2024-05-28T13:58:24.668501+0000","last_clean":"2024-05-28T13:58:24.668501+0000","last_became_active":"2024-05-28T13:58:23.707542+0000","last_became_peered":"2024-05-28T13:58:23.707542+0000","last_unstale":"2024-05-28T13:58:24.668501+0000","last_undegraded":"2024-05-28T13:58:24.668501+0000","last_fullsized":"2024-05-28T13:58:24.668501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:03.122796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439237+0000","last_change":"2024-05-28T13:58:23.687526+0000","last_active":"2024-05-28T13:58:26.439237+0000","last_peered":"2024-05-28T13:58:26.439237+0000","last_clean":"2024-05-28T13:58:26.439237+0000","last_became_active":"2024-05-28T13:58:23.687138+0000","last_became_peered":"2024-05-28T13:58:23.687138+0000","last_unstale":"2024-05-28T13:58:26.439237+0000","last_undegraded":"2024-05-28T13:58:26.439237+0000","last_fullsized":"2024-05-28T13:58:26.439237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:27:53.809481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.438877+0000","last_change":"2024-05-28T13:58:23.681967+0000","last_active":"2024-05-28T13:58:26.438877+0000","last_peered":"2024-05-28T13:58:26.438877+0000","last_clean":"2024-05-28T13:58:26.438877+0000","last_became_active":"2024-05-28T13:58:23.680491+0000","last_became_peered":"2024-05-28T13:58:23.680491+0000","last_unstale":"2024-05-28T13:58:26.438877+0000","last_undegraded":"2024-05-28T13:58:26.438877+0000","last_fullsized":"2024-05-28T13:58:26.438877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:07:03.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663864+0000","last_change":"2024-05-28T13:58:23.708051+0000","last_active":"2024-05-28T13:58:24.663864+0000","last_peered":"2024-05-28T13:58:24.663864+0000","last_clean":"2024-05-28T13:58:24.663864+0000","last_became_active":"2024-05-28T13:58:23.707749+0000","last_became_peered":"2024-05-28T13:58:23.707749+0000","last_unstale":"2024-05-28T13:58:24.663864+0000","last_undegraded":"2024-05-28T13:58:24.663864+0000","last_fullsized":"2024-05-28T13:58:24.663864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:02.981053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.667924+0000","last_change":"2024-05-28T13:58:23.684368+0000","last_active":"2024-05-28T13:58:24.667924+0000","last_peered":"2024-05-28T13:58:24.667924+0000","last_clean":"2024-05-28T13:58:24.667924+0000","last_became_active":"2024-05-28T13:58:23.683905+0000","last_became_peered":"2024-05-28T13:58:23.683905+0000","last_unstale":"2024-05-28T13:58:24.667924+0000","last_undegraded":"2024-05-28T13:58:24.667924+0000","last_fullsized":"2024-05-28T13:58:24.667924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:02.981053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668773+0000","last_change":"2024-05-28T13:58:23.706633+0000","last_active":"2024-05-28T13:58:24.668773+0000","last_peered":"2024-05-28T13:58:24.668773+0000","last_clean":"2024-05-28T13:58:24.668773+0000","last_became_active":"2024-05-28T13:58:23.693217+0000","last_became_peered":"2024-05-28T13:58:23.693217+0000","last_unstale":"2024-05-28T13:58:24.668773+0000","last_undegraded":"2024-05-28T13:58:24.668773+0000","last_fullsized":"2024-05-28T13:58:24.668773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:23:14.222046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:25.415731+0000","last_change":"2024-05-28T13:58:23.690320+0000","last_active":"2024-05-28T13:58:25.415731+0000","last_peered":"2024-05-28T13:58:25.415731+0000","last_clean":"2024-05-28T13:58:25.415731+0000","last_became_active":"2024-05-28T13:58:23.690028+0000","last_became_peered":"2024-05-28T13:58:23.690028+0000","last_unstale":"2024-05-28T13:58:25.415731+0000","last_undegraded":"2024-05-28T13:58:25.415731+0000","last_fullsized":"2024-05-28T13:58:25.415731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:59.716378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440863+0000","last_change":"2024-05-28T13:58:23.683310+0000","last_active":"2024-05-28T13:58:26.440863+0000","last_peered":"2024-05-28T13:58:26.440863+0000","last_clean":"2024-05-28T13:58:26.440863+0000","last_became_active":"2024-05-28T13:58:23.682882+0000","last_became_peered":"2024-05-28T13:58:23.682882+0000","last_unstale":"2024-05-28T13:58:26.440863+0000","last_undegraded":"2024-05-28T13:58:26.440863+0000","last_fullsized":"2024-05-28T13:58:26.440863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:28:55.178232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694723+0000","last_change":"2024-05-28T13:58:23.706004+0000","last_active":"2024-05-28T13:58:24.694723+0000","last_peered":"2024-05-28T13:58:24.694723+0000","last_clean":"2024-05-28T13:58:24.694723+0000","last_became_active":"2024-05-28T13:58:23.705676+0000","last_became_peered":"2024-05-28T13:58:23.705676+0000","last_unstale":"2024-05-28T13:58:24.694723+0000","last_undegraded":"2024-05-28T13:58:24.694723+0000","last_fullsized":"2024-05-28T13:58:24.694723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:23.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439738+0000","last_change":"2024-05-28T13:58:23.678693+0000","last_active":"2024-05-28T13:58:26.439738+0000","last_peered":"2024-05-28T13:58:26.439738+0000","last_clean":"2024-05-28T13:58:26.439738+0000","last_became_active":"2024-05-28T13:58:23.678258+0000","last_became_peered":"2024-05-28T13:58:23.678258+0000","last_unstale":"2024-05-28T13:58:26.439738+0000","last_undegraded":"2024-05-28T13:58:26.439738+0000","last_fullsized":"2024-05-28T13:58:26.439738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:23.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668261+0000","last_change":"2024-05-28T13:58:23.706461+0000","last_active":"2024-05-28T13:58:24.668261+0000","last_peered":"2024-05-28T13:58:24.668261+0000","last_clean":"2024-05-28T13:58:24.668261+0000","last_became_active":"2024-05-28T13:58:23.702598+0000","last_became_peered":"2024-05-28T13:58:23.702598+0000","last_unstale":"2024-05-28T13:58:24.668261+0000","last_undegraded":"2024-05-28T13:58:24.668261+0000","last_fullsized":"2024-05-28T13:58:24.668261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:31.530501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439024+0000","last_change":"2024-05-28T13:58:23.714967+0000","last_active":"2024-05-28T13:58:26.439024+0000","last_peered":"2024-05-28T13:58:26.439024+0000","last_clean":"2024-05-28T13:58:26.439024+0000","last_became_active":"2024-05-28T13:58:23.714653+0000","last_became_peered":"2024-05-28T13:58:23.714653+0000","last_unstale":"2024-05-28T13:58:26.439024+0000","last_undegraded":"2024-05-28T13:58:26.439024+0000","last_fullsized":"2024-05-28T13:58:26.439024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:46.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668849+0000","last_change":"2024-05-28T13:58:23.686847+0000","last_active":"2024-05-28T13:58:24.668849+0000","last_peered":"2024-05-28T13:58:24.668849+0000","last_clean":"2024-05-28T13:58:24.668849+0000","last_became_active":"2024-05-28T13:58:23.686518+0000","last_became_peered":"2024-05-28T13:58:23.686518+0000","last_unstale":"2024-05-28T13:58:24.668849+0000","last_undegraded":"2024-05-28T13:58:24.668849+0000","last_fullsized":"2024-05-28T13:58:24.668849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:43:20.340652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439484+0000","last_change":"2024-05-28T13:58:23.707351+0000","last_active":"2024-05-28T13:58:26.439484+0000","last_peered":"2024-05-28T13:58:26.439484+0000","last_clean":"2024-05-28T13:58:26.439484+0000","last_became_active":"2024-05-28T13:58:23.707077+0000","last_became_peered":"2024-05-28T13:58:23.707077+0000","last_unstale":"2024-05-28T13:58:26.439484+0000","last_undegraded":"2024-05-28T13:58:26.439484+0000","last_fullsized":"2024-05-28T13:58:26.439484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:49.556769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694896+0000","last_change":"2024-05-28T13:58:23.688056+0000","last_active":"2024-05-28T13:58:24.694896+0000","last_peered":"2024-05-28T13:58:24.694896+0000","last_clean":"2024-05-28T13:58:24.694896+0000","last_became_active":"2024-05-28T13:58:23.687623+0000","last_became_peered":"2024-05-28T13:58:23.687623+0000","last_unstale":"2024-05-28T13:58:24.694896+0000","last_undegraded":"2024-05-28T13:58:24.694896+0000","last_fullsized":"2024-05-28T13:58:24.694896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:02.981053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663895+0000","last_change":"2024-05-28T13:58:23.672661+0000","last_active":"2024-05-28T13:58:24.663895+0000","last_peered":"2024-05-28T13:58:24.663895+0000","last_clean":"2024-05-28T13:58:24.663895+0000","last_became_active":"2024-05-28T13:58:23.672378+0000","last_became_peered":"2024-05-28T13:58:23.672378+0000","last_unstale":"2024-05-28T13:58:24.663895+0000","last_undegraded":"2024-05-28T13:58:24.663895+0000","last_fullsized":"2024-05-28T13:58:24.663895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:49.556769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439309+0000","last_change":"2024-05-28T13:58:23.707282+0000","last_active":"2024-05-28T13:58:26.439309+0000","last_peered":"2024-05-28T13:58:26.439309+0000","last_clean":"2024-05-28T13:58:26.439309+0000","last_became_active":"2024-05-28T13:58:23.706977+0000","last_became_peered":"2024-05-28T13:58:23.706977+0000","last_unstale":"2024-05-28T13:58:26.439309+0000","last_undegraded":"2024-05-28T13:58:26.439309+0000","last_fullsized":"2024-05-28T13:58:26.439309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:03:59.917115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439837+0000","last_change":"2024-05-28T13:58:23.684735+0000","last_active":"2024-05-28T13:58:26.439837+0000","last_peered":"2024-05-28T13:58:26.439837+0000","last_clean":"2024-05-28T13:58:26.439837+0000","last_became_active":"2024-05-28T13:58:23.684389+0000","last_became_peered":"2024-05-28T13:58:23.684389+0000","last_unstale":"2024-05-28T13:58:26.439837+0000","last_undegraded":"2024-05-28T13:58:26.439837+0000","last_fullsized":"2024-05-28T13:58:26.439837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:02.981053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668321+0000","last_change":"2024-05-28T13:58:23.686822+0000","last_active":"2024-05-28T13:58:24.668321+0000","last_peered":"2024-05-28T13:58:24.668321+0000","last_clean":"2024-05-28T13:58:24.668321+0000","last_became_active":"2024-05-28T13:58:23.686454+0000","last_became_peered":"2024-05-28T13:58:23.686454+0000","last_unstale":"2024-05-28T13:58:24.668321+0000","last_undegraded":"2024-05-28T13:58:24.668321+0000","last_fullsized":"2024-05-28T13:58:24.668321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:22:38.773556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:25.415579+0000","last_change":"2024-05-28T13:58:23.702203+0000","last_active":"2024-05-28T13:58:25.415579+0000","last_peered":"2024-05-28T13:58:25.415579+0000","last_clean":"2024-05-28T13:58:25.415579+0000","last_became_active":"2024-05-28T13:58:23.701830+0000","last_became_peered":"2024-05-28T13:58:23.701830+0000","last_unstale":"2024-05-28T13:58:25.415579+0000","last_undegraded":"2024-05-28T13:58:25.415579+0000","last_fullsized":"2024-05-28T13:58:25.415579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:56.904188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694422+0000","last_change":"2024-05-28T13:58:23.684058+0000","last_active":"2024-05-28T13:58:24.694422+0000","last_peered":"2024-05-28T13:58:24.694422+0000","last_clean":"2024-05-28T13:58:24.694422+0000","last_became_active":"2024-05-28T13:58:23.683555+0000","last_became_peered":"2024-05-28T13:58:23.683555+0000","last_unstale":"2024-05-28T13:58:24.694422+0000","last_undegraded":"2024-05-28T13:58:24.694422+0000","last_fullsized":"2024-05-28T13:58:24.694422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:10.036541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668066+0000","last_change":"2024-05-28T13:58:23.704842+0000","last_active":"2024-05-28T13:58:24.668066+0000","last_peered":"2024-05-28T13:58:24.668066+0000","last_clean":"2024-05-28T13:58:24.668066+0000","last_became_active":"2024-05-28T13:58:23.704465+0000","last_became_peered":"2024-05-28T13:58:23.704465+0000","last_unstale":"2024-05-28T13:58:24.668066+0000","last_undegraded":"2024-05-28T13:58:24.668066+0000","last_fullsized":"2024-05-28T13:58:24.668066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:30.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439668+0000","last_change":"2024-05-28T13:58:22.678741+0000","last_active":"2024-05-28T13:58:26.439668+0000","last_peered":"2024-05-28T13:58:26.439668+0000","last_clean":"2024-05-28T13:58:26.439668+0000","last_became_active":"2024-05-28T13:58:22.678532+0000","last_became_peered":"2024-05-28T13:58:22.678532+0000","last_unstale":"2024-05-28T13:58:26.439668+0000","last_undegraded":"2024-05-28T13:58:26.439668+0000","last_fullsized":"2024-05-28T13:58:26.439668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439908+0000","last_change":"2024-05-28T13:58:23.708224+0000","last_active":"2024-05-28T13:58:26.439908+0000","last_peered":"2024-05-28T13:58:26.439908+0000","last_clean":"2024-05-28T13:58:26.439908+0000","last_became_active":"2024-05-28T13:58:23.707930+0000","last_became_peered":"2024-05-28T13:58:23.707930+0000","last_unstale":"2024-05-28T13:58:26.439908+0000","last_undegraded":"2024-05-28T13:58:26.439908+0000","last_fullsized":"2024-05-28T13:58:26.439908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:31.530501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694550+0000","last_change":"2024-05-28T13:58:22.673310+0000","last_active":"2024-05-28T13:58:24.694550+0000","last_peered":"2024-05-28T13:58:24.694550+0000","last_clean":"2024-05-28T13:58:24.694550+0000","last_became_active":"2024-05-28T13:58:22.673003+0000","last_became_peered":"2024-05-28T13:58:22.673003+0000","last_unstale":"2024-05-28T13:58:24.694550+0000","last_undegraded":"2024-05-28T13:58:24.694550+0000","last_fullsized":"2024-05-28T13:58:24.694550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439193+0000","last_change":"2024-05-28T13:58:22.677344+0000","last_active":"2024-05-28T13:58:26.439193+0000","last_peered":"2024-05-28T13:58:26.439193+0000","last_clean":"2024-05-28T13:58:26.439193+0000","last_became_active":"2024-05-28T13:58:22.677024+0000","last_became_peered":"2024-05-28T13:58:22.677024+0000","last_unstale":"2024-05-28T13:58:26.439193+0000","last_undegraded":"2024-05-28T13:58:26.439193+0000","last_fullsized":"2024-05-28T13:58:26.439193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:52.890121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439186+0000","last_change":"2024-05-28T13:58:23.686005+0000","last_active":"2024-05-28T13:58:26.439186+0000","last_peered":"2024-05-28T13:58:26.439186+0000","last_clean":"2024-05-28T13:58:26.439186+0000","last_became_active":"2024-05-28T13:58:23.682423+0000","last_became_peered":"2024-05-28T13:58:23.682423+0000","last_unstale":"2024-05-28T13:58:26.439186+0000","last_undegraded":"2024-05-28T13:58:26.439186+0000","last_fullsized":"2024-05-28T13:58:26.439186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:22.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668085+0000","last_change":"2024-05-28T13:58:22.673902+0000","last_active":"2024-05-28T13:58:24.668085+0000","last_peered":"2024-05-28T13:58:24.668085+0000","last_clean":"2024-05-28T13:58:24.668085+0000","last_became_active":"2024-05-28T13:58:22.673588+0000","last_became_peered":"2024-05-28T13:58:22.673588+0000","last_unstale":"2024-05-28T13:58:24.668085+0000","last_undegraded":"2024-05-28T13:58:24.668085+0000","last_fullsized":"2024-05-28T13:58:24.668085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663575+0000","last_change":"2024-05-28T13:58:23.672761+0000","last_active":"2024-05-28T13:58:24.663575+0000","last_peered":"2024-05-28T13:58:24.663575+0000","last_clean":"2024-05-28T13:58:24.663575+0000","last_became_active":"2024-05-28T13:58:23.672276+0000","last_became_peered":"2024-05-28T13:58:23.672276+0000","last_unstale":"2024-05-28T13:58:24.663575+0000","last_undegraded":"2024-05-28T13:58:24.663575+0000","last_fullsized":"2024-05-28T13:58:24.663575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:46.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694720+0000","last_change":"2024-05-28T13:58:22.676336+0000","last_active":"2024-05-28T13:58:24.694720+0000","last_peered":"2024-05-28T13:58:24.694720+0000","last_clean":"2024-05-28T13:58:24.694720+0000","last_became_active":"2024-05-28T13:58:22.676013+0000","last_became_peered":"2024-05-28T13:58:22.676013+0000","last_unstale":"2024-05-28T13:58:24.694720+0000","last_undegraded":"2024-05-28T13:58:24.694720+0000","last_fullsized":"2024-05-28T13:58:24.694720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668373+0000","last_change":"2024-05-28T13:58:23.691740+0000","last_active":"2024-05-28T13:58:24.668373+0000","last_peered":"2024-05-28T13:58:24.668373+0000","last_clean":"2024-05-28T13:58:24.668373+0000","last_became_active":"2024-05-28T13:58:23.691056+0000","last_became_peered":"2024-05-28T13:58:23.691056+0000","last_unstale":"2024-05-28T13:58:24.668373+0000","last_undegraded":"2024-05-28T13:58:24.668373+0000","last_fullsized":"2024-05-28T13:58:24.668373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:27:53.809481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668300+0000","last_change":"2024-05-28T13:58:22.678043+0000","last_active":"2024-05-28T13:58:24.668300+0000","last_peered":"2024-05-28T13:58:24.668300+0000","last_clean":"2024-05-28T13:58:24.668300+0000","last_became_active":"2024-05-28T13:58:22.677684+0000","last_became_peered":"2024-05-28T13:58:22.677684+0000","last_unstale":"2024-05-28T13:58:24.668300+0000","last_undegraded":"2024-05-28T13:58:24.668300+0000","last_fullsized":"2024-05-28T13:58:24.668300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:02.116873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439582+0000","last_change":"2024-05-28T13:58:23.706540+0000","last_active":"2024-05-28T13:58:26.439582+0000","last_peered":"2024-05-28T13:58:26.439582+0000","last_clean":"2024-05-28T13:58:26.439582+0000","last_became_active":"2024-05-28T13:58:23.706209+0000","last_became_peered":"2024-05-28T13:58:23.706209+0000","last_unstale":"2024-05-28T13:58:26.439582+0000","last_undegraded":"2024-05-28T13:58:26.439582+0000","last_fullsized":"2024-05-28T13:58:26.439582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:03:59.917115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663850+0000","last_change":"2024-05-28T13:58:22.684218+0000","last_active":"2024-05-28T13:58:24.663850+0000","last_peered":"2024-05-28T13:58:24.663850+0000","last_clean":"2024-05-28T13:58:24.663850+0000","last_became_active":"2024-05-28T13:58:22.683984+0000","last_became_peered":"2024-05-28T13:58:22.683984+0000","last_unstale":"2024-05-28T13:58:24.663850+0000","last_undegraded":"2024-05-28T13:58:24.663850+0000","last_fullsized":"2024-05-28T13:58:24.663850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668661+0000","last_change":"2024-05-28T13:58:23.705226+0000","last_active":"2024-05-28T13:58:24.668661+0000","last_peered":"2024-05-28T13:58:24.668661+0000","last_clean":"2024-05-28T13:58:24.668661+0000","last_became_active":"2024-05-28T13:58:23.704265+0000","last_became_peered":"2024-05-28T13:58:23.704265+0000","last_unstale":"2024-05-28T13:58:24.668661+0000","last_undegraded":"2024-05-28T13:58:24.668661+0000","last_fullsized":"2024-05-28T13:58:24.668661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:36.332770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694803+0000","last_change":"2024-05-28T13:58:22.674273+0000","last_active":"2024-05-28T13:58:24.694803+0000","last_peered":"2024-05-28T13:58:24.694803+0000","last_clean":"2024-05-28T13:58:24.694803+0000","last_became_active":"2024-05-28T13:58:22.673984+0000","last_became_peered":"2024-05-28T13:58:22.673984+0000","last_unstale":"2024-05-28T13:58:24.694803+0000","last_undegraded":"2024-05-28T13:58:24.694803+0000","last_fullsized":"2024-05-28T13:58:24.694803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440523+0000","last_change":"2024-05-28T13:58:23.685763+0000","last_active":"2024-05-28T13:58:26.440523+0000","last_peered":"2024-05-28T13:58:26.440523+0000","last_clean":"2024-05-28T13:58:26.440523+0000","last_became_active":"2024-05-28T13:58:23.685512+0000","last_became_peered":"2024-05-28T13:58:23.685512+0000","last_unstale":"2024-05-28T13:58:26.440523+0000","last_undegraded":"2024-05-28T13:58:26.440523+0000","last_fullsized":"2024-05-28T13:58:26.440523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:43:20.340652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439949+0000","last_change":"2024-05-28T13:58:22.668047+0000","last_active":"2024-05-28T13:58:26.439949+0000","last_peered":"2024-05-28T13:58:26.439949+0000","last_clean":"2024-05-28T13:58:26.439949+0000","last_became_active":"2024-05-28T13:58:22.667747+0000","last_became_peered":"2024-05-28T13:58:22.667747+0000","last_unstale":"2024-05-28T13:58:26.439949+0000","last_undegraded":"2024-05-28T13:58:26.439949+0000","last_fullsized":"2024-05-28T13:58:26.439949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:52.489479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439199+0000","last_change":"2024-05-28T13:58:23.675653+0000","last_active":"2024-05-28T13:58:26.439199+0000","last_peered":"2024-05-28T13:58:26.439199+0000","last_clean":"2024-05-28T13:58:26.439199+0000","last_became_active":"2024-05-28T13:58:23.675371+0000","last_became_peered":"2024-05-28T13:58:23.675371+0000","last_unstale":"2024-05-28T13:58:26.439199+0000","last_undegraded":"2024-05-28T13:58:26.439199+0000","last_fullsized":"2024-05-28T13:58:26.439199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:10.036541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668378+0000","last_change":"2024-05-28T13:58:22.682942+0000","last_active":"2024-05-28T13:58:24.668378+0000","last_peered":"2024-05-28T13:58:24.668378+0000","last_clean":"2024-05-28T13:58:24.668378+0000","last_became_active":"2024-05-28T13:58:22.681946+0000","last_became_peered":"2024-05-28T13:58:22.681946+0000","last_unstale":"2024-05-28T13:58:24.668378+0000","last_undegraded":"2024-05-28T13:58:24.668378+0000","last_fullsized":"2024-05-28T13:58:24.668378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:07.005097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440496+0000","last_change":"2024-05-28T13:58:23.716761+0000","last_active":"2024-05-28T13:58:26.440496+0000","last_peered":"2024-05-28T13:58:26.440496+0000","last_clean":"2024-05-28T13:58:26.440496+0000","last_became_active":"2024-05-28T13:58:23.716488+0000","last_became_peered":"2024-05-28T13:58:23.716488+0000","last_unstale":"2024-05-28T13:58:26.440496+0000","last_undegraded":"2024-05-28T13:58:26.440496+0000","last_fullsized":"2024-05-28T13:58:26.440496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:16:14.092125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694386+0000","last_change":"2024-05-28T13:58:22.678451+0000","last_active":"2024-05-28T13:58:24.694386+0000","last_peered":"2024-05-28T13:58:24.694386+0000","last_clean":"2024-05-28T13:58:24.694386+0000","last_became_active":"2024-05-28T13:58:22.675695+0000","last_became_peered":"2024-05-28T13:58:22.675695+0000","last_unstale":"2024-05-28T13:58:24.694386+0000","last_undegraded":"2024-05-28T13:58:24.694386+0000","last_fullsized":"2024-05-28T13:58:24.694386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694443+0000","last_change":"2024-05-28T13:58:23.683800+0000","last_active":"2024-05-28T13:58:24.694443+0000","last_peered":"2024-05-28T13:58:24.694443+0000","last_clean":"2024-05-28T13:58:24.694443+0000","last_became_active":"2024-05-28T13:58:23.683208+0000","last_became_peered":"2024-05-28T13:58:23.683208+0000","last_unstale":"2024-05-28T13:58:24.694443+0000","last_undegraded":"2024-05-28T13:58:24.694443+0000","last_fullsized":"2024-05-28T13:58:24.694443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:46.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:25.415502+0000","last_change":"2024-05-28T13:58:22.676468+0000","last_active":"2024-05-28T13:58:25.415502+0000","last_peered":"2024-05-28T13:58:25.415502+0000","last_clean":"2024-05-28T13:58:25.415502+0000","last_became_active":"2024-05-28T13:58:22.676202+0000","last_became_peered":"2024-05-28T13:58:22.676202+0000","last_unstale":"2024-05-28T13:58:25.415502+0000","last_undegraded":"2024-05-28T13:58:25.415502+0000","last_fullsized":"2024-05-28T13:58:25.415502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:47:52.890121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440228+0000","last_change":"2024-05-28T13:58:23.717324+0000","last_active":"2024-05-28T13:58:26.440228+0000","last_peered":"2024-05-28T13:58:26.440228+0000","last_clean":"2024-05-28T13:58:26.440228+0000","last_became_active":"2024-05-28T13:58:23.716047+0000","last_became_peered":"2024-05-28T13:58:23.716047+0000","last_unstale":"2024-05-28T13:58:26.440228+0000","last_undegraded":"2024-05-28T13:58:26.440228+0000","last_fullsized":"2024-05-28T13:58:26.440228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:01:32.723660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668416+0000","last_change":"2024-05-28T13:58:22.671015+0000","last_active":"2024-05-28T13:58:24.668416+0000","last_peered":"2024-05-28T13:58:24.668416+0000","last_clean":"2024-05-28T13:58:24.668416+0000","last_became_active":"2024-05-28T13:58:22.663746+0000","last_became_peered":"2024-05-28T13:58:22.663746+0000","last_unstale":"2024-05-28T13:58:24.668416+0000","last_undegraded":"2024-05-28T13:58:24.668416+0000","last_fullsized":"2024-05-28T13:58:24.668416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:22:51.238198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440629+0000","last_change":"2024-05-28T13:58:23.705595+0000","last_active":"2024-05-28T13:58:26.440629+0000","last_peered":"2024-05-28T13:58:26.440629+0000","last_clean":"2024-05-28T13:58:26.440629+0000","last_became_active":"2024-05-28T13:58:23.705326+0000","last_became_peered":"2024-05-28T13:58:23.705326+0000","last_unstale":"2024-05-28T13:58:26.440629+0000","last_undegraded":"2024-05-28T13:58:26.440629+0000","last_fullsized":"2024-05-28T13:58:26.440629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:16.649099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:25.415746+0000","last_change":"2024-05-28T13:58:22.673480+0000","last_active":"2024-05-28T13:58:25.415746+0000","last_peered":"2024-05-28T13:58:25.415746+0000","last_clean":"2024-05-28T13:58:25.415746+0000","last_became_active":"2024-05-28T13:58:22.671634+0000","last_became_peered":"2024-05-28T13:58:22.671634+0000","last_unstale":"2024-05-28T13:58:25.415746+0000","last_undegraded":"2024-05-28T13:58:25.415746+0000","last_fullsized":"2024-05-28T13:58:25.415746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440947+0000","last_change":"2024-05-28T13:58:23.708965+0000","last_active":"2024-05-28T13:58:26.440947+0000","last_peered":"2024-05-28T13:58:26.440947+0000","last_clean":"2024-05-28T13:58:26.440947+0000","last_became_active":"2024-05-28T13:58:23.708298+0000","last_became_peered":"2024-05-28T13:58:23.708298+0000","last_unstale":"2024-05-28T13:58:26.440947+0000","last_undegraded":"2024-05-28T13:58:26.440947+0000","last_fullsized":"2024-05-28T13:58:26.440947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:29:08.273460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663963+0000","last_change":"2024-05-28T13:58:22.688102+0000","last_active":"2024-05-28T13:58:24.663963+0000","last_peered":"2024-05-28T13:58:24.663963+0000","last_clean":"2024-05-28T13:58:24.663963+0000","last_became_active":"2024-05-28T13:58:22.686031+0000","last_became_peered":"2024-05-28T13:58:22.686031+0000","last_unstale":"2024-05-28T13:58:24.663963+0000","last_undegraded":"2024-05-28T13:58:24.663963+0000","last_fullsized":"2024-05-28T13:58:24.663963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668988+0000","last_change":"2024-05-28T13:58:23.690696+0000","last_active":"2024-05-28T13:58:24.668988+0000","last_peered":"2024-05-28T13:58:24.668988+0000","last_clean":"2024-05-28T13:58:24.668988+0000","last_became_active":"2024-05-28T13:58:23.689615+0000","last_became_peered":"2024-05-28T13:58:23.689615+0000","last_unstale":"2024-05-28T13:58:24.668988+0000","last_undegraded":"2024-05-28T13:58:24.668988+0000","last_fullsized":"2024-05-28T13:58:24.668988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:40:43.691282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:25.415621+0000","last_change":"2024-05-28T13:58:22.674653+0000","last_active":"2024-05-28T13:58:25.415621+0000","last_peered":"2024-05-28T13:58:25.415621+0000","last_clean":"2024-05-28T13:58:25.415621+0000","last_became_active":"2024-05-28T13:58:22.674308+0000","last_became_peered":"2024-05-28T13:58:22.674308+0000","last_unstale":"2024-05-28T13:58:25.415621+0000","last_undegraded":"2024-05-28T13:58:25.415621+0000","last_fullsized":"2024-05-28T13:58:25.415621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440733+0000","last_change":"2024-05-28T13:58:23.702244+0000","last_active":"2024-05-28T13:58:26.440733+0000","last_peered":"2024-05-28T13:58:26.440733+0000","last_clean":"2024-05-28T13:58:26.440733+0000","last_became_active":"2024-05-28T13:58:23.701858+0000","last_became_peered":"2024-05-28T13:58:23.701858+0000","last_unstale":"2024-05-28T13:58:26.440733+0000","last_undegraded":"2024-05-28T13:58:26.440733+0000","last_fullsized":"2024-05-28T13:58:26.440733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:40:43.691282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440128+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T13:58:26.440128+0000","last_peered":"2024-05-28T13:58:26.440128+0000","last_clean":"2024-05-28T13:58:26.440128+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T13:58:26.440128+0000","last_undegraded":"2024-05-28T13:58:26.440128+0000","last_fullsized":"2024-05-28T13:58:26.440128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668007+0000","last_change":"2024-05-28T13:58:22.668236+0000","last_active":"2024-05-28T13:58:24.668007+0000","last_peered":"2024-05-28T13:58:24.668007+0000","last_clean":"2024-05-28T13:58:24.668007+0000","last_became_active":"2024-05-28T13:58:22.667928+0000","last_became_peered":"2024-05-28T13:58:22.667928+0000","last_unstale":"2024-05-28T13:58:24.668007+0000","last_undegraded":"2024-05-28T13:58:24.668007+0000","last_fullsized":"2024-05-28T13:58:24.668007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439204+0000","last_change":"2024-05-28T13:58:23.701173+0000","last_active":"2024-05-28T13:58:26.439204+0000","last_peered":"2024-05-28T13:58:26.439204+0000","last_clean":"2024-05-28T13:58:26.439204+0000","last_became_active":"2024-05-28T13:58:23.700830+0000","last_became_peered":"2024-05-28T13:58:23.700830+0000","last_unstale":"2024-05-28T13:58:26.439204+0000","last_undegraded":"2024-05-28T13:58:26.439204+0000","last_fullsized":"2024-05-28T13:58:26.439204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:01:32.723660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440029+0000","last_change":"2024-05-28T13:58:22.687098+0000","last_active":"2024-05-28T13:58:26.440029+0000","last_peered":"2024-05-28T13:58:26.440029+0000","last_clean":"2024-05-28T13:58:26.440029+0000","last_became_active":"2024-05-28T13:58:22.686751+0000","last_became_peered":"2024-05-28T13:58:22.686751+0000","last_unstale":"2024-05-28T13:58:26.440029+0000","last_undegraded":"2024-05-28T13:58:26.440029+0000","last_fullsized":"2024-05-28T13:58:26.440029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:35.326847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.438801+0000","last_change":"2024-05-28T13:58:23.709225+0000","last_active":"2024-05-28T13:58:26.438801+0000","last_peered":"2024-05-28T13:58:26.438801+0000","last_clean":"2024-05-28T13:58:26.438801+0000","last_became_active":"2024-05-28T13:58:23.708785+0000","last_became_peered":"2024-05-28T13:58:23.708785+0000","last_unstale":"2024-05-28T13:58:26.438801+0000","last_undegraded":"2024-05-28T13:58:26.438801+0000","last_fullsized":"2024-05-28T13:58:26.438801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:12:43.447237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:25.415595+0000","last_change":"2024-05-28T13:58:22.667443+0000","last_active":"2024-05-28T13:58:25.415595+0000","last_peered":"2024-05-28T13:58:25.415595+0000","last_clean":"2024-05-28T13:58:25.415595+0000","last_became_active":"2024-05-28T13:58:22.666967+0000","last_became_peered":"2024-05-28T13:58:22.666967+0000","last_unstale":"2024-05-28T13:58:25.415595+0000","last_undegraded":"2024-05-28T13:58:25.415595+0000","last_fullsized":"2024-05-28T13:58:25.415595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663665+0000","last_change":"2024-05-28T13:58:23.680134+0000","last_active":"2024-05-28T13:58:24.663665+0000","last_peered":"2024-05-28T13:58:24.663665+0000","last_clean":"2024-05-28T13:58:24.663665+0000","last_became_active":"2024-05-28T13:58:23.679912+0000","last_became_peered":"2024-05-28T13:58:23.679912+0000","last_unstale":"2024-05-28T13:58:24.663665+0000","last_undegraded":"2024-05-28T13:58:24.663665+0000","last_fullsized":"2024-05-28T13:58:24.663665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:03:59.917115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668429+0000","last_change":"2024-05-28T13:58:22.683368+0000","last_active":"2024-05-28T13:58:24.668429+0000","last_peered":"2024-05-28T13:58:24.668429+0000","last_clean":"2024-05-28T13:58:24.668429+0000","last_became_active":"2024-05-28T13:58:22.683102+0000","last_became_peered":"2024-05-28T13:58:22.683102+0000","last_unstale":"2024-05-28T13:58:24.668429+0000","last_undegraded":"2024-05-28T13:58:24.668429+0000","last_fullsized":"2024-05-28T13:58:24.668429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.669031+0000","last_change":"2024-05-28T13:58:23.707964+0000","last_active":"2024-05-28T13:58:24.669031+0000","last_peered":"2024-05-28T13:58:24.669031+0000","last_clean":"2024-05-28T13:58:24.669031+0000","last_became_active":"2024-05-28T13:58:23.707430+0000","last_became_peered":"2024-05-28T13:58:23.707430+0000","last_unstale":"2024-05-28T13:58:24.669031+0000","last_undegraded":"2024-05-28T13:58:24.669031+0000","last_fullsized":"2024-05-28T13:58:24.669031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:56.904188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663715+0000","last_change":"2024-05-28T13:58:22.676423+0000","last_active":"2024-05-28T13:58:24.663715+0000","last_peered":"2024-05-28T13:58:24.663715+0000","last_clean":"2024-05-28T13:58:24.663715+0000","last_became_active":"2024-05-28T13:58:22.675960+0000","last_became_peered":"2024-05-28T13:58:22.675960+0000","last_unstale":"2024-05-28T13:58:24.663715+0000","last_undegraded":"2024-05-28T13:58:24.663715+0000","last_fullsized":"2024-05-28T13:58:24.663715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:02.116873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668838+0000","last_change":"2024-05-28T13:58:23.692003+0000","last_active":"2024-05-28T13:58:24.668838+0000","last_peered":"2024-05-28T13:58:24.668838+0000","last_clean":"2024-05-28T13:58:24.668838+0000","last_became_active":"2024-05-28T13:58:23.689087+0000","last_became_peered":"2024-05-28T13:58:23.689087+0000","last_unstale":"2024-05-28T13:58:24.668838+0000","last_undegraded":"2024-05-28T13:58:24.668838+0000","last_fullsized":"2024-05-28T13:58:24.668838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:32:51.191091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440859+0000","last_change":"2024-05-28T13:58:22.686829+0000","last_active":"2024-05-28T13:58:26.440859+0000","last_peered":"2024-05-28T13:58:26.440859+0000","last_clean":"2024-05-28T13:58:26.440859+0000","last_became_active":"2024-05-28T13:58:22.686497+0000","last_became_peered":"2024-05-28T13:58:22.686497+0000","last_unstale":"2024-05-28T13:58:26.440859+0000","last_undegraded":"2024-05-28T13:58:26.440859+0000","last_fullsized":"2024-05-28T13:58:26.440859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T13:58:58.710455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663623+0000","last_change":"2024-05-28T13:58:23.684658+0000","last_active":"2024-05-28T13:58:24.663623+0000","last_peered":"2024-05-28T13:58:24.663623+0000","last_clean":"2024-05-28T13:58:24.663623+0000","last_became_active":"2024-05-28T13:58:23.682872+0000","last_became_peered":"2024-05-28T13:58:23.682872+0000","last_unstale":"2024-05-28T13:58:24.663623+0000","last_undegraded":"2024-05-28T13:58:24.663623+0000","last_fullsized":"2024-05-28T13:58:24.663623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:22.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663835+0000","last_change":"2024-05-28T13:58:22.682664+0000","last_active":"2024-05-28T13:58:24.663835+0000","last_peered":"2024-05-28T13:58:24.663835+0000","last_clean":"2024-05-28T13:58:24.663835+0000","last_became_active":"2024-05-28T13:58:22.682422+0000","last_became_peered":"2024-05-28T13:58:22.682422+0000","last_unstale":"2024-05-28T13:58:24.663835+0000","last_undegraded":"2024-05-28T13:58:24.663835+0000","last_fullsized":"2024-05-28T13:58:24.663835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:07.005097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439642+0000","last_change":"2024-05-28T13:58:23.678389+0000","last_active":"2024-05-28T13:58:26.439642+0000","last_peered":"2024-05-28T13:58:26.439642+0000","last_clean":"2024-05-28T13:58:26.439642+0000","last_became_active":"2024-05-28T13:58:23.678081+0000","last_became_peered":"2024-05-28T13:58:23.678081+0000","last_unstale":"2024-05-28T13:58:26.439642+0000","last_undegraded":"2024-05-28T13:58:26.439642+0000","last_fullsized":"2024-05-28T13:58:26.439642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:22:38.773556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694827+0000","last_change":"2024-05-28T13:58:22.680245+0000","last_active":"2024-05-28T13:58:24.694827+0000","last_peered":"2024-05-28T13:58:24.694827+0000","last_clean":"2024-05-28T13:58:24.694827+0000","last_became_active":"2024-05-28T13:58:22.679323+0000","last_became_peered":"2024-05-28T13:58:22.679323+0000","last_unstale":"2024-05-28T13:58:24.694827+0000","last_undegraded":"2024-05-28T13:58:24.694827+0000","last_fullsized":"2024-05-28T13:58:24.694827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:07.005097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439847+0000","last_change":"2024-05-28T13:58:23.677850+0000","last_active":"2024-05-28T13:58:26.439847+0000","last_peered":"2024-05-28T13:58:26.439847+0000","last_clean":"2024-05-28T13:58:26.439847+0000","last_became_active":"2024-05-28T13:58:23.677654+0000","last_became_peered":"2024-05-28T13:58:23.677654+0000","last_unstale":"2024-05-28T13:58:26.439847+0000","last_undegraded":"2024-05-28T13:58:26.439847+0000","last_fullsized":"2024-05-28T13:58:26.439847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:49.556769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440004+0000","last_change":"2024-05-28T13:58:22.682426+0000","last_active":"2024-05-28T13:58:26.440004+0000","last_peered":"2024-05-28T13:58:26.440004+0000","last_clean":"2024-05-28T13:58:26.440004+0000","last_became_active":"2024-05-28T13:58:22.682099+0000","last_became_peered":"2024-05-28T13:58:22.682099+0000","last_unstale":"2024-05-28T13:58:26.440004+0000","last_undegraded":"2024-05-28T13:58:26.440004+0000","last_fullsized":"2024-05-28T13:58:26.440004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:29.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.667930+0000","last_change":"2024-05-28T13:58:23.706882+0000","last_active":"2024-05-28T13:58:24.667930+0000","last_peered":"2024-05-28T13:58:24.667930+0000","last_clean":"2024-05-28T13:58:24.667930+0000","last_became_active":"2024-05-28T13:58:23.706588+0000","last_became_peered":"2024-05-28T13:58:23.706588+0000","last_unstale":"2024-05-28T13:58:24.667930+0000","last_undegraded":"2024-05-28T13:58:24.667930+0000","last_fullsized":"2024-05-28T13:58:24.667930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:43:08.011020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663707+0000","last_change":"2024-05-28T13:58:22.678682+0000","last_active":"2024-05-28T13:58:24.663707+0000","last_peered":"2024-05-28T13:58:24.663707+0000","last_clean":"2024-05-28T13:58:24.663707+0000","last_became_active":"2024-05-28T13:58:22.678233+0000","last_became_peered":"2024-05-28T13:58:22.678233+0000","last_unstale":"2024-05-28T13:58:24.663707+0000","last_undegraded":"2024-05-28T13:58:24.663707+0000","last_fullsized":"2024-05-28T13:58:24.663707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:35.326847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439379+0000","last_change":"2024-05-28T13:58:23.669871+0000","last_active":"2024-05-28T13:58:26.439379+0000","last_peered":"2024-05-28T13:58:26.439379+0000","last_clean":"2024-05-28T13:58:26.439379+0000","last_became_active":"2024-05-28T13:58:23.669643+0000","last_became_peered":"2024-05-28T13:58:23.669643+0000","last_unstale":"2024-05-28T13:58:26.439379+0000","last_undegraded":"2024-05-28T13:58:26.439379+0000","last_fullsized":"2024-05-28T13:58:26.439379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:02.981053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.663683+0000","last_change":"2024-05-28T13:58:22.680769+0000","last_active":"2024-05-28T13:58:24.663683+0000","last_peered":"2024-05-28T13:58:24.663683+0000","last_clean":"2024-05-28T13:58:24.663683+0000","last_became_active":"2024-05-28T13:58:22.679781+0000","last_became_peered":"2024-05-28T13:58:22.679781+0000","last_unstale":"2024-05-28T13:58:24.663683+0000","last_undegraded":"2024-05-28T13:58:24.663683+0000","last_fullsized":"2024-05-28T13:58:24.663683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:54:38.721431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439570+0000","last_change":"2024-05-28T13:58:23.683534+0000","last_active":"2024-05-28T13:58:26.439570+0000","last_peered":"2024-05-28T13:58:26.439570+0000","last_clean":"2024-05-28T13:58:26.439570+0000","last_became_active":"2024-05-28T13:58:23.683185+0000","last_became_peered":"2024-05-28T13:58:23.683185+0000","last_unstale":"2024-05-28T13:58:26.439570+0000","last_undegraded":"2024-05-28T13:58:26.439570+0000","last_fullsized":"2024-05-28T13:58:26.439570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:31.530501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440541+0000","last_change":"2024-05-28T13:58:22.683550+0000","last_active":"2024-05-28T13:58:26.440541+0000","last_peered":"2024-05-28T13:58:26.440541+0000","last_clean":"2024-05-28T13:58:26.440541+0000","last_became_active":"2024-05-28T13:58:22.683218+0000","last_became_peered":"2024-05-28T13:58:22.683218+0000","last_unstale":"2024-05-28T13:58:26.440541+0000","last_undegraded":"2024-05-28T13:58:26.440541+0000","last_fullsized":"2024-05-28T13:58:26.440541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:12.464954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.694678+0000","last_change":"2024-05-28T13:58:23.709888+0000","last_active":"2024-05-28T13:58:24.694678+0000","last_peered":"2024-05-28T13:58:24.694678+0000","last_clean":"2024-05-28T13:58:24.694678+0000","last_became_active":"2024-05-28T13:58:23.709658+0000","last_became_peered":"2024-05-28T13:58:23.709658+0000","last_unstale":"2024-05-28T13:58:24.694678+0000","last_undegraded":"2024-05-28T13:58:24.694678+0000","last_fullsized":"2024-05-28T13:58:24.694678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:22.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668669+0000","last_change":"2024-05-28T13:58:23.710526+0000","last_active":"2024-05-28T13:58:24.668669+0000","last_peered":"2024-05-28T13:58:24.668669+0000","last_clean":"2024-05-28T13:58:24.668669+0000","last_became_active":"2024-05-28T13:58:23.710118+0000","last_became_peered":"2024-05-28T13:58:23.710118+0000","last_unstale":"2024-05-28T13:58:24.668669+0000","last_undegraded":"2024-05-28T13:58:24.668669+0000","last_fullsized":"2024-05-28T13:58:24.668669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:01:32.723660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440228+0000","last_change":"2024-05-28T13:58:22.681209+0000","last_active":"2024-05-28T13:58:26.440228+0000","last_peered":"2024-05-28T13:58:26.440228+0000","last_clean":"2024-05-28T13:58:26.440228+0000","last_became_active":"2024-05-28T13:58:22.680889+0000","last_became_peered":"2024-05-28T13:58:22.680889+0000","last_unstale":"2024-05-28T13:58:26.440228+0000","last_undegraded":"2024-05-28T13:58:26.440228+0000","last_fullsized":"2024-05-28T13:58:26.440228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:48:02.116873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:24.668891+0000","last_change":"2024-05-28T13:58:23.710507+0000","last_active":"2024-05-28T13:58:24.668891+0000","last_peered":"2024-05-28T13:58:24.668891+0000","last_clean":"2024-05-28T13:58:24.668891+0000","last_became_active":"2024-05-28T13:58:23.710104+0000","last_became_peered":"2024-05-28T13:58:23.710104+0000","last_unstale":"2024-05-28T13:58:24.668891+0000","last_undegraded":"2024-05-28T13:58:24.668891+0000","last_fullsized":"2024-05-28T13:58:24.668891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:16.649099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.440682+0000","last_change":"2024-05-28T13:58:22.688976+0000","last_active":"2024-05-28T13:58:26.440682+0000","last_peered":"2024-05-28T13:58:26.440682+0000","last_clean":"2024-05-28T13:58:26.440682+0000","last_became_active":"2024-05-28T13:58:22.688607+0000","last_became_peered":"2024-05-28T13:58:22.688607+0000","last_unstale":"2024-05-28T13:58:26.440682+0000","last_undegraded":"2024-05-28T13:58:26.440682+0000","last_fullsized":"2024-05-28T13:58:26.440682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:38:55.898265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439430+0000","last_change":"2024-05-28T13:58:23.695036+0000","last_active":"2024-05-28T13:58:26.439430+0000","last_peered":"2024-05-28T13:58:26.439430+0000","last_clean":"2024-05-28T13:58:26.439430+0000","last_became_active":"2024-05-28T13:58:23.692922+0000","last_became_peered":"2024-05-28T13:58:23.692922+0000","last_unstale":"2024-05-28T13:58:26.439430+0000","last_undegraded":"2024-05-28T13:58:26.439430+0000","last_fullsized":"2024-05-28T13:58:26.439430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:22.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439495+0000","last_change":"2024-05-28T13:58:22.684711+0000","last_active":"2024-05-28T13:58:26.439495+0000","last_peered":"2024-05-28T13:58:26.439495+0000","last_clean":"2024-05-28T13:58:26.439495+0000","last_became_active":"2024-05-28T13:58:22.680359+0000","last_became_peered":"2024-05-28T13:58:22.680359+0000","last_unstale":"2024-05-28T13:58:26.439495+0000","last_undegraded":"2024-05-28T13:58:26.439495+0000","last_fullsized":"2024-05-28T13:58:26.439495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:21:34.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439311+0000","last_change":"2024-05-28T13:58:23.690282+0000","last_active":"2024-05-28T13:58:26.439311+0000","last_peered":"2024-05-28T13:58:26.439311+0000","last_clean":"2024-05-28T13:58:26.439311+0000","last_became_active":"2024-05-28T13:58:23.689975+0000","last_became_peered":"2024-05-28T13:58:23.689975+0000","last_unstale":"2024-05-28T13:58:26.439311+0000","last_undegraded":"2024-05-28T13:58:26.439311+0000","last_fullsized":"2024-05-28T13:58:26.439311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:22.644984+0000","last_clean_scrub_stamp":"2024-05-28T13:58:22.644984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:23.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-28T13:58:26.439751+0000","last_change":"2024-05-28T13:58:22.679511+0000","last_active":"2024-05-28T13:58:26.439751+0000","last_peered":"2024-05-28T13:58:26.439751+0000","last_clean":"2024-05-28T13:58:26.439751+0000","last_became_active":"2024-05-28T13:58:22.679207+0000","last_became_peered":"2024-05-28T13:58:22.679207+0000","last_unstale":"2024-05-28T13:58:26.439751+0000","last_undegraded":"2024-05-28T13:58:26.439751+0000","last_fullsized":"2024-05-28T13:58:26.439751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:21.639061+0000","last_clean_scrub_stamp":"2024-05-28T13:58:21.639061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:46:33.126031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T13:58:32.886 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-28T13:58:32.886 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-28T13:58:32.886 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-28T13:58:32.886 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-28T13:58:33.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.077+0000 7f49ad600640 1 Processor -- start 2024-05-28T13:58:33.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.077+0000 7f49ad600640 1 -- start start 2024-05-28T13:58:33.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f49ad600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f49a806ee10 0x7f49a806f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f49ad600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49a815c120 con 0x7f49a80b4f00 2024-05-28T13:58:33.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f49ad600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49a815c3f0 con 0x7f49a8158f30 2024-05-28T13:58:33.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f49ad600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f49a806f7a0 con 0x7f49a806ee10 2024-05-28T13:58:33.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f49a7400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f49a806ee10 0x7f49a806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f499e000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f49a80b4f00 0x7f49a815ba10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60618/0 (socket says 172.21.15.42:60618) 2024-05-28T13:58:33.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f499e000640 1 -- 172.21.15.42:0/3617645068 learned_addr learned my addr 172.21.15.42:0/3617645068 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:33.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f49a7400640 1 -- 172.21.15.42:0/3617645068 >> v1:172.21.15.63:6789/0 conn(0x7f49a8158f30 legacy=0x7f49a8072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f49a7400640 1 -- 172.21.15.42:0/3617645068 >> v1:172.21.15.42:6789/0 conn(0x7f49a80b4f00 legacy=0x7f49a815ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f49a7400640 1 -- 172.21.15.42:0/3617645068 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a815c6d0 con 0x7f49a806ee10 2024-05-28T13:58:33.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f49a6000640 1 -- 172.21.15.42:0/3617645068 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3763446972 0 0) 0x7f49a815c120 con 0x7f49a80b4f00 2024-05-28T13:58:33.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f49a7400640 1 --2- 172.21.15.42:0/3617645068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a806ee10 0x7f49a806f260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009d90 tx=0x7f49a001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec072739e35427 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:33.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f49a6000640 1 -- 172.21.15.42:0/3617645068 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a00b2070 con 0x7f49a806ee10 2024-05-28T13:58:33.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f49a6000640 1 -- 172.21.15.42:0/3617645068 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a00a57a0 con 0x7f49a806ee10 2024-05-28T13:58:33.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.081+0000 7f49a6000640 1 -- 172.21.15.42:0/3617645068 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a00a5b70 con 0x7f49a806ee10 2024-05-28T13:58:33.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 -- 172.21.15.42:0/3617645068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a806ee10 msgr2=0x7f49a806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 --2- 172.21.15.42:0/3617645068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a806ee10 0x7f49a806f260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009d90 tx=0x7f49a001c8e0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 -- 172.21.15.42:0/3617645068 shutdown_connections 2024-05-28T13:58:33.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 --2- 172.21.15.42:0/3617645068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a806ee10 0x7f49a806f260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 -- 172.21.15.42:0/3617645068 >> 172.21.15.42:0/3617645068 conn(0x7f49a806bff0 msgr2=0x7f49a80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:33.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 -- 172.21.15.42:0/3617645068 shutdown_connections 2024-05-28T13:58:33.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 -- 172.21.15.42:0/3617645068 wait complete. 2024-05-28T13:58:33.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 Processor -- start 2024-05-28T13:58:33.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 -- start start 2024-05-28T13:58:33.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a806ee10 0x7f49a814ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a80b4f00 0x7f49a8153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a8158f30 0x7f49a806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f49a80b3b70 con 0x7f49a806ee10 2024-05-28T13:58:33.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f49a80b39f0 con 0x7f49a80b4f00 2024-05-28T13:58:33.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49ad600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f49a8169180 con 0x7f49a8158f30 2024-05-28T13:58:33.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49a6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a806ee10 0x7f49a814ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49a6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a806ee10 0x7f49a814ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38774/0 (socket says 172.21.15.42:38774) 2024-05-28T13:58:33.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49a6a00640 1 -- 172.21.15.42:0/3516203118 learned_addr learned my addr 172.21.15.42:0/3516203118 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:33.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f499e000640 1 --2- 172.21.15.42:0/3516203118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a80b4f00 0x7f49a8153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49a7400640 1 --2- 172.21.15.42:0/3516203118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a8158f30 0x7f49a806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49a6a00640 1 -- 172.21.15.42:0/3516203118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a8158f30 msgr2=0x7f49a806dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49a6a00640 1 --2- 172.21.15.42:0/3516203118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a8158f30 0x7f49a806dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49a6a00640 1 -- 172.21.15.42:0/3516203118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a80b4f00 msgr2=0x7f49a8153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49a6a00640 1 --2- 172.21.15.42:0/3516203118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a80b4f00 0x7f49a8153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49a6a00640 1 -- 172.21.15.42:0/3516203118 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a814f560 con 0x7f49a806ee10 2024-05-28T13:58:33.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.085+0000 7f49a6a00640 1 --2- 172.21.15.42:0/3516203118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a806ee10 0x7f49a814ee40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f499800c3c0 tx=0x7f499800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:33.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.089+0000 7f499f400640 1 -- 172.21.15.42:0/3516203118 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998017070 con 0x7f49a806ee10 2024-05-28T13:58:33.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.089+0000 7f499f400640 1 -- 172.21.15.42:0/3516203118 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49980052d0 con 0x7f49a806ee10 2024-05-28T13:58:33.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.089+0000 7f499f400640 1 -- 172.21.15.42:0/3516203118 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998003840 con 0x7f49a806ee10 2024-05-28T13:58:33.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.089+0000 7f49ad600640 1 -- 172.21.15.42:0/3516203118 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a81f89a0 con 0x7f49a806ee10 2024-05-28T13:58:33.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.089+0000 7f49ad600640 1 -- 172.21.15.42:0/3516203118 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a81f8f20 con 0x7f49a806ee10 2024-05-28T13:58:33.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.089+0000 7f49ad600640 1 -- 172.21.15.42:0/3516203118 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a8068a90 con 0x7f49a806ee10 2024-05-28T13:58:33.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.089+0000 7f499f400640 1 -- 172.21.15.42:0/3516203118 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f499801a070 con 0x7f49a806ee10 2024-05-28T13:58:33.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.089+0000 7f499f400640 1 --2- 172.21.15.42:0/3516203118 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4984068380 0x7f498406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.089+0000 7f499f400640 1 -- 172.21.15.42:0/3516203118 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4998085d10 con 0x7f49a806ee10 2024-05-28T13:58:33.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.089+0000 7f499e000640 1 --2- 172.21.15.42:0/3516203118 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4984068380 0x7f498406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.093+0000 7f499e000640 1 --2- 172.21.15.42:0/3516203118 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4984068380 0x7f498406a840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4990007fd0 tx=0x7f4990007ba0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:33.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.093+0000 7f499f400640 1 -- 172.21.15.42:0/3516203118 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f499808b020 con 0x7f49a806ee10 2024-05-28T13:58:33.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.293+0000 7f49ad600640 1 -- 172.21.15.42:0/3516203118 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f49a8075150 con 0x7f49a806ee10 2024-05-28T13:58:33.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.293+0000 7f499f400640 1 -- 172.21.15.42:0/3516203118 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f49a814f560 con 0x7f49a806ee10 2024-05-28T13:58:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:33.279 INFO:teuthology.orchestra.run.smithi042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-28T13:58:33.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.293+0000 7f49ad600640 1 -- 172.21.15.42:0/3516203118 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4984068380 msgr2=0x7f498406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.293+0000 7f49ad600640 1 --2- 172.21.15.42:0/3516203118 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4984068380 0x7f498406a840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4990007fd0 tx=0x7f4990007ba0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.293+0000 7f49ad600640 1 -- 172.21.15.42:0/3516203118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a806ee10 msgr2=0x7f49a814ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.293+0000 7f49ad600640 1 --2- 172.21.15.42:0/3516203118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a806ee10 0x7f49a814ee40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f499800c3c0 tx=0x7f499800c880 comp rx=0 tx=0).stop 2024-05-28T13:58:33.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.297+0000 7f49ad600640 1 -- 172.21.15.42:0/3516203118 shutdown_connections 2024-05-28T13:58:33.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.297+0000 7f49ad600640 1 --2- 172.21.15.42:0/3516203118 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4984068380 0x7f498406a840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.297+0000 7f49ad600640 1 --2- 172.21.15.42:0/3516203118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49a8158f30 0x7f49a806dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.297+0000 7f49ad600640 1 --2- 172.21.15.42:0/3516203118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a80b4f00 0x7f49a8153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.297+0000 7f49ad600640 1 --2- 172.21.15.42:0/3516203118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49a806ee10 0x7f49a814ee40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.297+0000 7f49ad600640 1 -- 172.21.15.42:0/3516203118 >> 172.21.15.42:0/3516203118 conn(0x7f49a806bff0 msgr2=0x7f49a815f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:33.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.297+0000 7f49ad600640 1 -- 172.21.15.42:0/3516203118 shutdown_connections 2024-05-28T13:58:33.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.297+0000 7f49ad600640 1 -- 172.21.15.42:0/3516203118 wait complete. 2024-05-28T13:58:33.299 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-28T13:58:33.300 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:33.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.453+0000 7f8fcf600640 1 Processor -- start 2024-05-28T13:58:33.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.453+0000 7f8fcf600640 1 -- start start 2024-05-28T13:58:33.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fcf600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8fc815e970 0x7f8fc80d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fcf600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fc80cd210 0x7f8fc809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fcf600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fc8070740 con 0x7f8fc806ff60 2024-05-28T13:58:33.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fcf600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8fc80d0640 con 0x7f8fc815e970 2024-05-28T13:58:33.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fcf600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc80d1160 con 0x7f8fc80cd210 2024-05-28T13:58:33.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fccc00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8fc815e970 0x7f8fc80d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fcd600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fc80cd210 0x7f8fc809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fccc00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8fc815e970 0x7f8fc80d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38272/0 (socket says 172.21.15.42:38272) 2024-05-28T13:58:33.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fcd600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fc80cd210 0x7f8fc809aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:43150/0 (socket says 172.21.15.42:43150) 2024-05-28T13:58:33.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fbfe00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f8fc806ff60 0x7f8fc80ccb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:50700/0 (socket says 172.21.15.42:50700) 2024-05-28T13:58:33.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fcd600640 1 -- 172.21.15.42:0/390114736 learned_addr learned my addr 172.21.15.42:0/390114736 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:33.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fccc00640 1 -- 172.21.15.42:0/390114736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fc80cd210 msgr2=0x7f8fc809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:33.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fccc00640 1 --2- 172.21.15.42:0/390114736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fc80cd210 0x7f8fc809aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fccc00640 1 -- 172.21.15.42:0/390114736 >> v1:172.21.15.63:6789/0 conn(0x7f8fc806ff60 legacy=0x7f8fc80ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fccc00640 1 -- 172.21.15.42:0/390114736 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc80d0870 con 0x7f8fc815e970 2024-05-28T13:58:33.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.457+0000 7f8fcd600640 1 --2- 172.21.15.42:0/390114736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fc80cd210 0x7f8fc809aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:33.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.461+0000 7f8fccc00640 1 --2- 172.21.15.42:0/390114736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fc815e970 0x7f8fc80d15f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80097c0 tx=0x7f8fb801c120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93d1e9bca54b6928 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:33.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.461+0000 7f8fc7e00640 1 -- 172.21.15.42:0/390114736 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80b0070 con 0x7f8fc815e970 2024-05-28T13:58:33.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.461+0000 7f8fc7e00640 1 -- 172.21.15.42:0/390114736 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb80ab600 con 0x7f8fc815e970 2024-05-28T13:58:33.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.461+0000 7f8fc7e00640 1 -- 172.21.15.42:0/390114736 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80ab9d0 con 0x7f8fc815e970 2024-05-28T13:58:33.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.461+0000 7f8fcf600640 1 -- 172.21.15.42:0/390114736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fc815e970 msgr2=0x7f8fc80d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.461+0000 7f8fcf600640 1 --2- 172.21.15.42:0/390114736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fc815e970 0x7f8fc80d15f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80097c0 tx=0x7f8fb801c120 comp rx=0 tx=0).stop 2024-05-28T13:58:33.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.461+0000 7f8fcf600640 1 -- 172.21.15.42:0/390114736 shutdown_connections 2024-05-28T13:58:33.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.461+0000 7f8fcf600640 1 --2- 172.21.15.42:0/390114736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fc80cd210 0x7f8fc809aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.461+0000 7f8fcf600640 1 --2- 172.21.15.42:0/390114736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fc815e970 0x7f8fc80d15f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.461+0000 7f8fcf600640 1 -- 172.21.15.42:0/390114736 >> 172.21.15.42:0/390114736 conn(0x7f8fc80b3fa0 msgr2=0x7f8fc80b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:33.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.461+0000 7f8fcf600640 1 -- 172.21.15.42:0/390114736 shutdown_connections 2024-05-28T13:58:33.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.461+0000 7f8fcf600640 1 -- 172.21.15.42:0/390114736 wait complete. 2024-05-28T13:58:33.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.461+0000 7f8fcf600640 1 Processor -- start 2024-05-28T13:58:33.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.461+0000 7f8fcf600640 1 -- start start 2024-05-28T13:58:33.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fcf600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fc806ff60 0x7f8fc814e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fcf600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fc80cd210 0x7f8fc80c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fcf600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fc815e970 0x7f8fc80b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fcf600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc80704c0 con 0x7f8fc80cd210 2024-05-28T13:58:33.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fcf600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc8070340 con 0x7f8fc815e970 2024-05-28T13:58:33.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fcf600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc80cd620 con 0x7f8fc806ff60 2024-05-28T13:58:33.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fbfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fc80cd210 0x7f8fc80c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fbfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fc80cd210 0x7f8fc80c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38782/0 (socket says 172.21.15.42:38782) 2024-05-28T13:58:33.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fbfe00640 1 -- 172.21.15.42:0/2598102639 learned_addr learned my addr 172.21.15.42:0/2598102639 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:33.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fccc00640 1 --2- 172.21.15.42:0/2598102639 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fc806ff60 0x7f8fc814e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fcd600640 1 --2- 172.21.15.42:0/2598102639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fc815e970 0x7f8fc80b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fbfe00640 1 -- 172.21.15.42:0/2598102639 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fc806ff60 msgr2=0x7f8fc814e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fbfe00640 1 --2- 172.21.15.42:0/2598102639 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fc806ff60 0x7f8fc814e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fbfe00640 1 -- 172.21.15.42:0/2598102639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fc815e970 msgr2=0x7f8fc80b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fbfe00640 1 --2- 172.21.15.42:0/2598102639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fc815e970 0x7f8fc80b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fbfe00640 1 -- 172.21.15.42:0/2598102639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc80ca4d0 con 0x7f8fc80cd210 2024-05-28T13:58:33.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fbfe00640 1 --2- 172.21.15.42:0/2598102639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fc80cd210 0x7f8fc80c9f90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000c420 tx=0x7f8fb000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:33.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fc5600640 1 -- 172.21.15.42:0/2598102639 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00074e0 con 0x7f8fc80cd210 2024-05-28T13:58:33.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fc5600640 1 -- 172.21.15.42:0/2598102639 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb0007680 con 0x7f8fc80cd210 2024-05-28T13:58:33.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fcf600640 1 -- 172.21.15.42:0/2598102639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc80af740 con 0x7f8fc80cd210 2024-05-28T13:58:33.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fc5600640 1 -- 172.21.15.42:0/2598102639 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0004a30 con 0x7f8fc80cd210 2024-05-28T13:58:33.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fcf600640 1 -- 172.21.15.42:0/2598102639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc80afd00 con 0x7f8fc80cd210 2024-05-28T13:58:33.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fcf600640 1 -- 172.21.15.42:0/2598102639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc8068a90 con 0x7f8fc80cd210 2024-05-28T13:58:33.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.465+0000 7f8fc5600640 1 -- 172.21.15.42:0/2598102639 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8fb001a020 con 0x7f8fc80cd210 2024-05-28T13:58:33.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.469+0000 7f8fc5600640 1 --2- 172.21.15.42:0/2598102639 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fa4068380 0x7f8fa406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.469+0000 7f8fc5600640 1 -- 172.21.15.42:0/2598102639 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8fb00860b0 con 0x7f8fc80cd210 2024-05-28T13:58:33.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.469+0000 7f8fccc00640 1 --2- 172.21.15.42:0/2598102639 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fa4068380 0x7f8fa406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.469+0000 7f8fccc00640 1 --2- 172.21.15.42:0/2598102639 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fa4068380 0x7f8fa406a840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80ad000 tx=0x7f8fb80a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:33.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.473+0000 7f8fc5600640 1 -- 172.21.15.42:0/2598102639 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8fb0056010 con 0x7f8fc80cd210 2024-05-28T13:58:33.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.637+0000 7f8fcf600640 1 -- 172.21.15.42:0/2598102639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fc80c9d70 con 0x7f8fc80cd210 2024-05-28T13:58:33.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.637+0000 7f8fc5600640 1 -- 172.21.15.42:0/2598102639 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16545 (secure 0 0 0) 0x7f8fb005a310 con 0x7f8fc80cd210 2024-05-28T13:58:33.623 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:33.623 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:24.659233+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:33.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.641+0000 7f8fcf600640 1 -- 172.21.15.42:0/2598102639 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fa4068380 msgr2=0x7f8fa406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.641+0000 7f8fcf600640 1 --2- 172.21.15.42:0/2598102639 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fa4068380 0x7f8fa406a840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80ad000 tx=0x7f8fb80a6000 comp rx=0 tx=0).stop 2024-05-28T13:58:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.641+0000 7f8fcf600640 1 -- 172.21.15.42:0/2598102639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fc80cd210 msgr2=0x7f8fc80c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.641+0000 7f8fcf600640 1 --2- 172.21.15.42:0/2598102639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fc80cd210 0x7f8fc80c9f90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000c420 tx=0x7f8fb000c8e0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.641+0000 7f8fcf600640 1 -- 172.21.15.42:0/2598102639 shutdown_connections 2024-05-28T13:58:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.641+0000 7f8fcf600640 1 --2- 172.21.15.42:0/2598102639 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fa4068380 0x7f8fa406a840 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.641+0000 7f8fcf600640 1 --2- 172.21.15.42:0/2598102639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fc815e970 0x7f8fc80b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.641+0000 7f8fcf600640 1 --2- 172.21.15.42:0/2598102639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fc80cd210 0x7f8fc80c9f90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.641+0000 7f8fcf600640 1 --2- 172.21.15.42:0/2598102639 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fc806ff60 0x7f8fc814e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.641+0000 7f8fcf600640 1 -- 172.21.15.42:0/2598102639 >> 172.21.15.42:0/2598102639 conn(0x7f8fc80b3fa0 msgr2=0x7f8fc806cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.641+0000 7f8fcf600640 1 -- 172.21.15.42:0/2598102639 shutdown_connections 2024-05-28T13:58:33.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.641+0000 7f8fcf600640 1 -- 172.21.15.42:0/2598102639 wait complete. 2024-05-28T13:58:33.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:58:33.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:58:33.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa598a00640 1 Processor -- start 2024-05-28T13:58:33.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa598a00640 1 -- start start 2024-05-28T13:58:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa598a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa59409aa50 0x7fa5940d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa598a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5940d0f50 con 0x7fa59409ae30 2024-05-28T13:58:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa598a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5940d1230 con 0x7fa59406dd90 2024-05-28T13:58:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa598a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa5940d1510 con 0x7fa59409aa50 2024-05-28T13:58:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa591400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa59409ae30 0x7fa59406d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60630/0 (socket says 172.21.15.42:60630) 2024-05-28T13:58:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa591400640 1 -- 172.21.15.42:0/2000985811 learned_addr learned my addr 172.21.15.42:0/2000985811 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa591e00640 1 --2- 172.21.15.42:0/2000985811 >> v2:172.21.15.63:3300/0 conn(0x7fa59409aa50 0x7fa5940d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa590a00640 1 -- 172.21.15.42:0/2000985811 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 300712976 0 0) 0x7fa5940d0f50 con 0x7fa59409ae30 2024-05-28T13:58:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa590a00640 1 -- 172.21.15.42:0/2000985811 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa578003530 con 0x7fa59409ae30 2024-05-28T13:58:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa591e00640 1 -- 172.21.15.42:0/2000985811 >> v1:172.21.15.63:6789/0 conn(0x7fa59406dd90 legacy=0x7fa5940b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa590a00640 1 -- 172.21.15.42:0/2000985811 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3960871751 0 0) 0x7fa578003530 con 0x7fa59409ae30 2024-05-28T13:58:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa591e00640 1 -- 172.21.15.42:0/2000985811 >> v1:172.21.15.42:6789/0 conn(0x7fa59409ae30 legacy=0x7fa59406d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.809+0000 7fa591e00640 1 -- 172.21.15.42:0/2000985811 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5940d17f0 con 0x7fa59409aa50 2024-05-28T13:58:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.813+0000 7fa591e00640 1 --2- 172.21.15.42:0/2000985811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa59409aa50 0x7fa5940d0820 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa5840097c0 tx=0x7fa58401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4c30ae9eef99bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.813+0000 7fa590a00640 1 -- 172.21.15.42:0/2000985811 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5840b0070 con 0x7fa59409aa50 2024-05-28T13:58:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.813+0000 7fa590a00640 1 -- 172.21.15.42:0/2000985811 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5840ab400 con 0x7fa59409aa50 2024-05-28T13:58:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.813+0000 7fa590a00640 1 -- 172.21.15.42:0/2000985811 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580001680 con 0x7fa59409aa50 2024-05-28T13:58:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.813+0000 7fa598a00640 1 -- 172.21.15.42:0/2000985811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa59409aa50 msgr2=0x7fa5940d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.813+0000 7fa598a00640 1 --2- 172.21.15.42:0/2000985811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa59409aa50 0x7fa5940d0820 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa5840097c0 tx=0x7fa58401c3c0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.813+0000 7fa598a00640 1 -- 172.21.15.42:0/2000985811 shutdown_connections 2024-05-28T13:58:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.813+0000 7fa598a00640 1 --2- 172.21.15.42:0/2000985811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa59409aa50 0x7fa5940d0820 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.813+0000 7fa598a00640 1 -- 172.21.15.42:0/2000985811 >> 172.21.15.42:0/2000985811 conn(0x7fa5940b3fe0 msgr2=0x7fa5940b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.813+0000 7fa598a00640 1 -- 172.21.15.42:0/2000985811 shutdown_connections 2024-05-28T13:58:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.813+0000 7fa598a00640 1 -- 172.21.15.42:0/2000985811 wait complete. 2024-05-28T13:58:33.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.813+0000 7fa598a00640 1 Processor -- start 2024-05-28T13:58:33.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa598a00640 1 -- start start 2024-05-28T13:58:33.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa598a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa59406dd90 0x7fa594156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa598a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa59409aa50 0x7fa5940cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa598a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa59409ae30 0x7fa59415d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa598a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa5940b3360 con 0x7fa59409aa50 2024-05-28T13:58:33.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa598a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa5940b31e0 con 0x7fa59406dd90 2024-05-28T13:58:33.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa598a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa59416c820 con 0x7fa59409ae30 2024-05-28T13:58:33.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa591400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa59409aa50 0x7fa5940cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa591400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa59409aa50 0x7fa5940cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38798/0 (socket says 172.21.15.42:38798) 2024-05-28T13:58:33.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa591400640 1 -- 172.21.15.42:0/2349916259 learned_addr learned my addr 172.21.15.42:0/2349916259 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:33.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa591e00640 1 --2- 172.21.15.42:0/2349916259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa59406dd90 0x7fa594156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa592800640 1 --2- 172.21.15.42:0/2349916259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa59409ae30 0x7fa59415d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa591400640 1 -- 172.21.15.42:0/2349916259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa59409ae30 msgr2=0x7fa59415d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa591400640 1 --2- 172.21.15.42:0/2349916259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa59409ae30 0x7fa59415d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa591400640 1 -- 172.21.15.42:0/2349916259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa59406dd90 msgr2=0x7fa594156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa591400640 1 --2- 172.21.15.42:0/2349916259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa59406dd90 0x7fa594156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa591400640 1 -- 172.21.15.42:0/2349916259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa594156dd0 con 0x7fa59409aa50 2024-05-28T13:58:33.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa591400640 1 --2- 172.21.15.42:0/2349916259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa59409aa50 0x7fa5940cded0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa58000c760 tx=0x7fa58000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:33.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa58a000640 1 -- 172.21.15.42:0/2349916259 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580017070 con 0x7fa59409aa50 2024-05-28T13:58:33.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa58a000640 1 -- 172.21.15.42:0/2349916259 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa580005b00 con 0x7fa59409aa50 2024-05-28T13:58:33.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa58a000640 1 -- 172.21.15.42:0/2349916259 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580007070 con 0x7fa59409aa50 2024-05-28T13:58:33.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa598a00640 1 -- 172.21.15.42:0/2349916259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5940ce920 con 0x7fa59409aa50 2024-05-28T13:58:33.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa598a00640 1 -- 172.21.15.42:0/2349916259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5940cbc50 con 0x7fa59409aa50 2024-05-28T13:58:33.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.817+0000 7fa598a00640 1 -- 172.21.15.42:0/2349916259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5940a0540 con 0x7fa59409aa50 2024-05-28T13:58:33.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.821+0000 7fa58a000640 1 -- 172.21.15.42:0/2349916259 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa58001a070 con 0x7fa59409aa50 2024-05-28T13:58:33.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.821+0000 7fa58a000640 1 --2- 172.21.15.42:0/2349916259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5740683b0 0x7fa57406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:33.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.821+0000 7fa58a000640 1 -- 172.21.15.42:0/2349916259 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa580085e00 con 0x7fa59409aa50 2024-05-28T13:58:33.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.821+0000 7fa591e00640 1 --2- 172.21.15.42:0/2349916259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5740683b0 0x7fa57406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:33.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.821+0000 7fa591e00640 1 --2- 172.21.15.42:0/2349916259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5740683b0 0x7fa57406a870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa5840ad000 tx=0x7fa5840a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:33.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.825+0000 7fa58a000640 1 -- 172.21.15.42:0/2349916259 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa580055d60 con 0x7fa59409aa50 2024-05-28T13:58:33.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.989+0000 7fa598a00640 1 -- 172.21.15.42:0/2349916259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa59404fdd0 con 0x7fa59409aa50 2024-05-28T13:58:33.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.989+0000 7fa58a000640 1 -- 172.21.15.42:0/2349916259 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fa58005a060 con 0x7fa59409aa50 2024-05-28T13:58:33.974 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:58:33.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.989+0000 7fa598a00640 1 -- 172.21.15.42:0/2349916259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5740683b0 msgr2=0x7fa57406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.989+0000 7fa598a00640 1 --2- 172.21.15.42:0/2349916259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5740683b0 0x7fa57406a870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa5840ad000 tx=0x7fa5840a6000 comp rx=0 tx=0).stop 2024-05-28T13:58:33.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.989+0000 7fa598a00640 1 -- 172.21.15.42:0/2349916259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa59409aa50 msgr2=0x7fa5940cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:33.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.989+0000 7fa598a00640 1 --2- 172.21.15.42:0/2349916259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa59409aa50 0x7fa5940cded0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa58000c760 tx=0x7fa58000cc20 comp rx=0 tx=0).stop 2024-05-28T13:58:33.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.989+0000 7fa598a00640 1 -- 172.21.15.42:0/2349916259 shutdown_connections 2024-05-28T13:58:33.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.989+0000 7fa598a00640 1 --2- 172.21.15.42:0/2349916259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5740683b0 0x7fa57406a870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.989+0000 7fa598a00640 1 --2- 172.21.15.42:0/2349916259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa59409ae30 0x7fa59415d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.989+0000 7fa598a00640 1 --2- 172.21.15.42:0/2349916259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa59409aa50 0x7fa5940cded0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.989+0000 7fa598a00640 1 --2- 172.21.15.42:0/2349916259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa59406dd90 0x7fa594156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:33.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.989+0000 7fa598a00640 1 -- 172.21.15.42:0/2349916259 >> 172.21.15.42:0/2349916259 conn(0x7fa5940b3fe0 msgr2=0x7fa59406cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:33.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.989+0000 7fa598a00640 1 -- 172.21.15.42:0/2349916259 shutdown_connections 2024-05-28T13:58:33.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:33.989+0000 7fa598a00640 1 -- 172.21.15.42:0/2349916259 wait complete. 2024-05-28T13:58:33.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:58:34.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bbc800640 1 Processor -- start 2024-05-28T13:58:34.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bbc800640 1 -- start start 2024-05-28T13:58:34.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bbc800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bb4160e60 0x7f1bb40cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bbc800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1bb40d0ac0 0x7f1bb409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bbc800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bb40cd170 con 0x7f1bb41602c0 2024-05-28T13:58:34.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bbc800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1bb40ce400 con 0x7f1bb40d0ac0 2024-05-28T13:58:34.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bbc800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb40cd3e0 con 0x7f1bb4160e60 2024-05-28T13:58:34.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bba800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1bb40d0ac0 0x7f1bb409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bb9400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bb4160e60 0x7f1bb40cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bb9e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f1bb41602c0 0x7f1bb41606a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:50708/0 (socket says 172.21.15.42:50708) 2024-05-28T13:58:34.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bba800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1bb40d0ac0 0x7f1bb409aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38318/0 (socket says 172.21.15.42:38318) 2024-05-28T13:58:34.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bb9e00640 1 -- 172.21.15.42:0/3212184250 learned_addr learned my addr 172.21.15.42:0/3212184250 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:34.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bba800640 1 -- 172.21.15.42:0/3212184250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bb4160e60 msgr2=0x7f1bb40cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:34.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bba800640 1 --2- 172.21.15.42:0/3212184250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bb4160e60 0x7f1bb40cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bba800640 1 -- 172.21.15.42:0/3212184250 >> v1:172.21.15.63:6789/0 conn(0x7f1bb41602c0 legacy=0x7f1bb41606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.153+0000 7f1bba800640 1 -- 172.21.15.42:0/3212184250 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb40cd6c0 con 0x7f1bb40d0ac0 2024-05-28T13:58:34.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.157+0000 7f1bba800640 1 --2- 172.21.15.42:0/3212184250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb40d0ac0 0x7f1bb409aa50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009d90 tx=0x7f1bb001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e0f6fc136a7c9ed server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:34.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.157+0000 7f1bb8a00640 1 -- 172.21.15.42:0/3212184250 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00b2070 con 0x7f1bb40d0ac0 2024-05-28T13:58:34.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.157+0000 7f1bb8a00640 1 -- 172.21.15.42:0/3212184250 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb00a59a0 con 0x7f1bb40d0ac0 2024-05-28T13:58:34.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.157+0000 7f1bb8a00640 1 -- 172.21.15.42:0/3212184250 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00a5d70 con 0x7f1bb40d0ac0 2024-05-28T13:58:34.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.157+0000 7f1bbc800640 1 -- 172.21.15.42:0/3212184250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb40d0ac0 msgr2=0x7f1bb409aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.157+0000 7f1bbc800640 1 --2- 172.21.15.42:0/3212184250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb40d0ac0 0x7f1bb409aa50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009d90 tx=0x7f1bb001c940 comp rx=0 tx=0).stop 2024-05-28T13:58:34.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.157+0000 7f1bbc800640 1 -- 172.21.15.42:0/3212184250 shutdown_connections 2024-05-28T13:58:34.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.157+0000 7f1bbc800640 1 --2- 172.21.15.42:0/3212184250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb40d0ac0 0x7f1bb409aa50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.157+0000 7f1bbc800640 1 --2- 172.21.15.42:0/3212184250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bb4160e60 0x7f1bb40cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.157+0000 7f1bbc800640 1 -- 172.21.15.42:0/3212184250 >> 172.21.15.42:0/3212184250 conn(0x7f1bb40b3fd0 msgr2=0x7f1bb40b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:34.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.157+0000 7f1bbc800640 1 -- 172.21.15.42:0/3212184250 shutdown_connections 2024-05-28T13:58:34.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.157+0000 7f1bbc800640 1 -- 172.21.15.42:0/3212184250 wait complete. 2024-05-28T13:58:34.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bbc800640 1 Processor -- start 2024-05-28T13:58:34.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bbc800640 1 -- start start 2024-05-28T13:58:34.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bbc800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bb40d0ac0 0x7f1bb4071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bbc800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb41602c0 0x7f1bb4166dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bbc800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bb4160e60 0x7f1bb414e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bbc800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb415dba0 con 0x7f1bb4160e60 2024-05-28T13:58:34.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bbc800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb415da20 con 0x7f1bb41602c0 2024-05-28T13:58:34.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bbc800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb4169640 con 0x7f1bb40d0ac0 2024-05-28T13:58:34.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bba800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bb4160e60 0x7f1bb414e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bba800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bb4160e60 0x7f1bb414e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38808/0 (socket says 172.21.15.42:38808) 2024-05-28T13:58:34.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bba800640 1 -- 172.21.15.42:0/174753642 learned_addr learned my addr 172.21.15.42:0/174753642 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:34.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bb9e00640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bb40d0ac0 0x7f1bb4071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bb9400640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb41602c0 0x7f1bb4166dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bba800640 1 -- 172.21.15.42:0/174753642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bb40d0ac0 msgr2=0x7f1bb4071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bba800640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bb40d0ac0 0x7f1bb4071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bba800640 1 -- 172.21.15.42:0/174753642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb41602c0 msgr2=0x7f1bb4166dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bba800640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb41602c0 0x7f1bb4166dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bba800640 1 -- 172.21.15.42:0/174753642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb40ce580 con 0x7f1bb4160e60 2024-05-28T13:58:34.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bb9e00640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bb40d0ac0 0x7f1bb4071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:34.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bba800640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bb4160e60 0x7f1bb414e050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009d90 tx=0x7f1bb00196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:34.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bae000640 1 -- 172.21.15.42:0/174753642 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00b2040 con 0x7f1bb4160e60 2024-05-28T13:58:34.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bae000640 1 -- 172.21.15.42:0/174753642 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb00bf070 con 0x7f1bb4160e60 2024-05-28T13:58:34.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bbc800640 1 -- 172.21.15.42:0/174753642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb40ce820 con 0x7f1bb4160e60 2024-05-28T13:58:34.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bae000640 1 -- 172.21.15.42:0/174753642 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00b88b0 con 0x7f1bb4160e60 2024-05-28T13:58:34.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.161+0000 7f1bbc800640 1 -- 172.21.15.42:0/174753642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb40cede0 con 0x7f1bb4160e60 2024-05-28T13:58:34.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.165+0000 7f1bbc800640 1 -- 172.21.15.42:0/174753642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb40a0570 con 0x7f1bb4160e60 2024-05-28T13:58:34.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.165+0000 7f1bae000640 1 -- 172.21.15.42:0/174753642 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1bb00b8a50 con 0x7f1bb4160e60 2024-05-28T13:58:34.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.165+0000 7f1bae000640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b98068380 0x7f1b9806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.165+0000 7f1bae000640 1 -- 172.21.15.42:0/174753642 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1bb0121ac0 con 0x7f1bb4160e60 2024-05-28T13:58:34.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.165+0000 7f1bb9e00640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b98068380 0x7f1b9806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.165+0000 7f1bb9e00640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b98068380 0x7f1b9806a840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8005890 tx=0x7f1ba8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:34.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.169+0000 7f1bae000640 1 -- 172.21.15.42:0/174753642 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1bb0016560 con 0x7f1bb4160e60 2024-05-28T13:58:34.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.333+0000 7f1bbc800640 1 -- 172.21.15.42:0/174753642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1bb404fdd0 con 0x7f1bb4160e60 2024-05-28T13:58:34.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.333+0000 7f1bae000640 1 -- 172.21.15.42:0/174753642 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f1bb00f1f50 con 0x7f1bb4160e60 2024-05-28T13:58:34.319 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:58:34.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.333+0000 7f1bbc800640 1 -- 172.21.15.42:0/174753642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b98068380 msgr2=0x7f1b9806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.333+0000 7f1bbc800640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b98068380 0x7f1b9806a840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8005890 tx=0x7f1ba8009040 comp rx=0 tx=0).stop 2024-05-28T13:58:34.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.333+0000 7f1bbc800640 1 -- 172.21.15.42:0/174753642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bb4160e60 msgr2=0x7f1bb414e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.333+0000 7f1bbc800640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bb4160e60 0x7f1bb414e050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009d90 tx=0x7f1bb00196e0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.333+0000 7f1bbc800640 1 -- 172.21.15.42:0/174753642 shutdown_connections 2024-05-28T13:58:34.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.333+0000 7f1bbc800640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b98068380 0x7f1b9806a840 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.333+0000 7f1bbc800640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bb4160e60 0x7f1bb414e050 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.333+0000 7f1bbc800640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb41602c0 0x7f1bb4166dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.333+0000 7f1bbc800640 1 --2- 172.21.15.42:0/174753642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bb40d0ac0 0x7f1bb4071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.333+0000 7f1bbc800640 1 -- 172.21.15.42:0/174753642 >> 172.21.15.42:0/174753642 conn(0x7f1bb40b3fd0 msgr2=0x7f1bb40b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:34.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.333+0000 7f1bbc800640 1 -- 172.21.15.42:0/174753642 shutdown_connections 2024-05-28T13:58:34.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.333+0000 7f1bbc800640 1 -- 172.21.15.42:0/174753642 wait complete. 2024-05-28T13:58:34.339 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:58:34.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1dbc00640 1 Processor -- start 2024-05-28T13:58:34.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1dbc00640 1 -- start start 2024-05-28T13:58:34.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1dbc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1d4144f80 0x7ff1d4072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1dbc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff1d406ee20 0x7ff1d406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1dbc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1d40afe40 con 0x7ff1d415b9b0 2024-05-28T13:58:34.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1dbc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff1d406f7c0 con 0x7ff1d406ee20 2024-05-28T13:58:34.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1dbc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d40724d0 con 0x7ff1d4144f80 2024-05-28T13:58:34.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1cbe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff1d406ee20 0x7ff1d406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1cbe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff1d406ee20 0x7ff1d406f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38822/0 (socket says 172.21.15.42:38822) 2024-05-28T13:58:34.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1cbe00640 1 -- 172.21.15.42:0/1887714374 learned_addr learned my addr 172.21.15.42:0/1887714374 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:34.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1cbe00640 1 -- 172.21.15.42:0/1887714374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1d4144f80 msgr2=0x7ff1d4072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:34.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1d9200640 1 --2- 172.21.15.42:0/1887714374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1d4144f80 0x7ff1d4072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1cbe00640 1 --2- 172.21.15.42:0/1887714374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1d4144f80 0x7ff1d4072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1cbe00640 1 -- 172.21.15.42:0/1887714374 >> v1:172.21.15.63:6789/0 conn(0x7ff1d415b9b0 legacy=0x7ff1d415be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:34.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1cbe00640 1 -- 172.21.15.42:0/1887714374 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d415c600 con 0x7ff1d406ee20 2024-05-28T13:58:34.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.497+0000 7ff1d9200640 1 --2- 172.21.15.42:0/1887714374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1d4144f80 0x7ff1d4072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:34.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.501+0000 7ff1cbe00640 1 --2- 172.21.15.42:0/1887714374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1d406ee20 0x7ff1d406f280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009870 tx=0x7ff1bc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df51758f3cca14a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:34.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.501+0000 7ff1d3e00640 1 -- 172.21.15.42:0/1887714374 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc0b1070 con 0x7ff1d406ee20 2024-05-28T13:58:34.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.501+0000 7ff1d3e00640 1 -- 172.21.15.42:0/1887714374 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1bc0ac630 con 0x7ff1d406ee20 2024-05-28T13:58:34.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.501+0000 7ff1d3e00640 1 -- 172.21.15.42:0/1887714374 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc0a8050 con 0x7ff1d406ee20 2024-05-28T13:58:34.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.501+0000 7ff1dbc00640 1 -- 172.21.15.42:0/1887714374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1d406ee20 msgr2=0x7ff1d406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.501+0000 7ff1dbc00640 1 --2- 172.21.15.42:0/1887714374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1d406ee20 0x7ff1d406f280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009870 tx=0x7ff1bc01c520 comp rx=0 tx=0).stop 2024-05-28T13:58:34.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.501+0000 7ff1dbc00640 1 -- 172.21.15.42:0/1887714374 shutdown_connections 2024-05-28T13:58:34.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.501+0000 7ff1dbc00640 1 --2- 172.21.15.42:0/1887714374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1d406ee20 0x7ff1d406f280 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.501+0000 7ff1dbc00640 1 --2- 172.21.15.42:0/1887714374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1d4144f80 0x7ff1d4072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.501+0000 7ff1dbc00640 1 -- 172.21.15.42:0/1887714374 >> 172.21.15.42:0/1887714374 conn(0x7ff1d40b21f0 msgr2=0x7ff1d40b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:34.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.501+0000 7ff1dbc00640 1 -- 172.21.15.42:0/1887714374 shutdown_connections 2024-05-28T13:58:34.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.501+0000 7ff1dbc00640 1 -- 172.21.15.42:0/1887714374 wait complete. 2024-05-28T13:58:34.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1dbc00640 1 Processor -- start 2024-05-28T13:58:34.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1dbc00640 1 -- start start 2024-05-28T13:58:34.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1dbc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1d406ee20 0x7ff1d406df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1dbc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1d4144f80 0x7ff1d4154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1dbc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1d415b9b0 0x7ff1d4154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1dbc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d4072ea0 con 0x7ff1d415b9b0 2024-05-28T13:58:34.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1dbc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d40b4010 con 0x7ff1d4144f80 2024-05-28T13:58:34.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1dbc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d413f680 con 0x7ff1d406ee20 2024-05-28T13:58:34.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1d9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1d415b9b0 0x7ff1d4154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1d9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1d415b9b0 0x7ff1d4154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38832/0 (socket says 172.21.15.42:38832) 2024-05-28T13:58:34.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1d9c00640 1 -- 172.21.15.42:0/849085429 learned_addr learned my addr 172.21.15.42:0/849085429 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:34.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1d9200640 1 --2- 172.21.15.42:0/849085429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1d406ee20 0x7ff1d406df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1cbe00640 1 --2- 172.21.15.42:0/849085429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1d4144f80 0x7ff1d4154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1d9c00640 1 -- 172.21.15.42:0/849085429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1d406ee20 msgr2=0x7ff1d406df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1d9c00640 1 --2- 172.21.15.42:0/849085429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1d406ee20 0x7ff1d406df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1d9c00640 1 -- 172.21.15.42:0/849085429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1d4144f80 msgr2=0x7ff1d4154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1d9c00640 1 --2- 172.21.15.42:0/849085429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1d4144f80 0x7ff1d4154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1d9c00640 1 -- 172.21.15.42:0/849085429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d4143c20 con 0x7ff1d415b9b0 2024-05-28T13:58:34.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1d9c00640 1 --2- 172.21.15.42:0/849085429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1d415b9b0 0x7ff1d4154ff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00b970 tx=0x7ff1cc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:34.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1d1600640 1 -- 172.21.15.42:0/849085429 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc019070 con 0x7ff1d415b9b0 2024-05-28T13:58:34.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1d1600640 1 -- 172.21.15.42:0/849085429 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1cc00cb40 con 0x7ff1d415b9b0 2024-05-28T13:58:34.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1d1600640 1 -- 172.21.15.42:0/849085429 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc014440 con 0x7ff1d415b9b0 2024-05-28T13:58:34.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1dbc00640 1 -- 172.21.15.42:0/849085429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d4143f00 con 0x7ff1d415b9b0 2024-05-28T13:58:34.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.505+0000 7ff1dbc00640 1 -- 172.21.15.42:0/849085429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d41444c0 con 0x7ff1d415b9b0 2024-05-28T13:58:34.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.509+0000 7ff1dbc00640 1 -- 172.21.15.42:0/849085429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d40738b0 con 0x7ff1d415b9b0 2024-05-28T13:58:34.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.509+0000 7ff1d1600640 1 -- 172.21.15.42:0/849085429 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff1cc01c070 con 0x7ff1d415b9b0 2024-05-28T13:58:34.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.509+0000 7ff1d1600640 1 --2- 172.21.15.42:0/849085429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1b0068380 0x7ff1b006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.509+0000 7ff1d1600640 1 -- 172.21.15.42:0/849085429 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff1cc086680 con 0x7ff1d415b9b0 2024-05-28T13:58:34.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.509+0000 7ff1d9200640 1 --2- 172.21.15.42:0/849085429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1b0068380 0x7ff1b006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.509+0000 7ff1d9200640 1 --2- 172.21.15.42:0/849085429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1b0068380 0x7ff1b006a840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400a9a0 tx=0x7ff1c4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:34.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.513+0000 7ff1d1600640 1 -- 172.21.15.42:0/849085429 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff1cc056550 con 0x7ff1d415b9b0 2024-05-28T13:58:34.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.677+0000 7ff1dbc00640 1 -- 172.21.15.42:0/849085429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff1d404fdd0 con 0x7ff1d415b9b0 2024-05-28T13:58:34.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.677+0000 7ff1d1600640 1 -- 172.21.15.42:0/849085429 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7ff1cc05a850 con 0x7ff1d415b9b0 2024-05-28T13:58:34.662 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:58:34.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.677+0000 7ff1dbc00640 1 -- 172.21.15.42:0/849085429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1b0068380 msgr2=0x7ff1b006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.677+0000 7ff1dbc00640 1 --2- 172.21.15.42:0/849085429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1b0068380 0x7ff1b006a840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400a9a0 tx=0x7ff1c4008040 comp rx=0 tx=0).stop 2024-05-28T13:58:34.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.677+0000 7ff1dbc00640 1 -- 172.21.15.42:0/849085429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1d415b9b0 msgr2=0x7ff1d4154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.677+0000 7ff1dbc00640 1 --2- 172.21.15.42:0/849085429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1d415b9b0 0x7ff1d4154ff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00b970 tx=0x7ff1cc00be30 comp rx=0 tx=0).stop 2024-05-28T13:58:34.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.677+0000 7ff1dbc00640 1 -- 172.21.15.42:0/849085429 shutdown_connections 2024-05-28T13:58:34.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.677+0000 7ff1dbc00640 1 --2- 172.21.15.42:0/849085429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1b0068380 0x7ff1b006a840 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.677+0000 7ff1dbc00640 1 --2- 172.21.15.42:0/849085429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1d415b9b0 0x7ff1d4154ff0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.677+0000 7ff1dbc00640 1 --2- 172.21.15.42:0/849085429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1d4144f80 0x7ff1d4154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.677+0000 7ff1dbc00640 1 --2- 172.21.15.42:0/849085429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1d406ee20 0x7ff1d406df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.677+0000 7ff1dbc00640 1 -- 172.21.15.42:0/849085429 >> 172.21.15.42:0/849085429 conn(0x7ff1d40b21f0 msgr2=0x7ff1d40b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:34.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.677+0000 7ff1dbc00640 1 -- 172.21.15.42:0/849085429 shutdown_connections 2024-05-28T13:58:34.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.677+0000 7ff1dbc00640 1 -- 172.21.15.42:0/849085429 wait complete. 2024-05-28T13:58:34.683 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T13:58:34.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.837+0000 7f6100000640 1 Processor -- start 2024-05-28T13:58:34.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.837+0000 7f6100000640 1 -- start start 2024-05-28T13:58:34.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f6100000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f60f8156a70 0x7f60f806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f6100000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f60f80afa50 0x7f60f80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f6100000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60f815cb90 con 0x7f60f8156e50 2024-05-28T13:58:34.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f6100000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f60f815ce50 con 0x7f60f80afa50 2024-05-28T13:58:34.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f6100000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f60f815d130 con 0x7f60f8156a70 2024-05-28T13:58:34.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f60fe000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f60f80afa50 0x7f60f80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f60fe000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f60f80afa50 0x7f60f80d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38844/0 (socket says 172.21.15.42:38844) 2024-05-28T13:58:34.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f60fe000640 1 -- 172.21.15.42:0/3344832248 learned_addr learned my addr 172.21.15.42:0/3344832248 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:34.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f60fcc00640 1 --1- 172.21.15.42:0/3344832248 >> v1:172.21.15.42:6789/0 conn(0x7f60f8156e50 0x7f60f80af340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60646/0 (socket says 172.21.15.42:60646) 2024-05-28T13:58:34.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f60fd600640 1 --2- 172.21.15.42:0/3344832248 >> v2:172.21.15.63:3300/0 conn(0x7f60f8156a70 0x7f60f806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f60fe000640 1 -- 172.21.15.42:0/3344832248 >> v2:172.21.15.63:3300/0 conn(0x7f60f8156a70 msgr2=0x7f60f806dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:34.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f60fe000640 1 --2- 172.21.15.42:0/3344832248 >> v2:172.21.15.63:3300/0 conn(0x7f60f8156a70 0x7f60f806dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f60fe000640 1 -- 172.21.15.42:0/3344832248 >> v1:172.21.15.42:6789/0 conn(0x7f60f8156e50 legacy=0x7f60f80af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f60fe000640 1 -- 172.21.15.42:0/3344832248 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f815d410 con 0x7f60f80afa50 2024-05-28T13:58:34.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.841+0000 7f60fd600640 1 --2- 172.21.15.42:0/3344832248 >> v2:172.21.15.63:3300/0 conn(0x7f60f8156a70 0x7f60f806dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:34.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f60fe000640 1 --2- 172.21.15.42:0/3344832248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f80afa50 0x7f60f80d06f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f60f80d13a0 tx=0x7f60f001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ddcfb5276bbb5a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:34.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f60f7e00640 1 -- 172.21.15.42:0/3344832248 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f00a57d0 con 0x7f60f80afa50 2024-05-28T13:58:34.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f60f7e00640 1 -- 172.21.15.42:0/3344832248 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f00a5970 con 0x7f60f80afa50 2024-05-28T13:58:34.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f60f7e00640 1 -- 172.21.15.42:0/3344832248 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f00a5d40 con 0x7f60f80afa50 2024-05-28T13:58:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 -- 172.21.15.42:0/3344832248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f80afa50 msgr2=0x7f60f80d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 --2- 172.21.15.42:0/3344832248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f80afa50 0x7f60f80d06f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f60f80d13a0 tx=0x7f60f001c9f0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 -- 172.21.15.42:0/3344832248 shutdown_connections 2024-05-28T13:58:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 --2- 172.21.15.42:0/3344832248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f80afa50 0x7f60f80d06f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 --2- 172.21.15.42:0/3344832248 >> v2:172.21.15.63:3300/0 conn(0x7f60f8156a70 0x7f60f806dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 -- 172.21.15.42:0/3344832248 >> 172.21.15.42:0/3344832248 conn(0x7f60f806bdf0 msgr2=0x7f60f806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 -- 172.21.15.42:0/3344832248 shutdown_connections 2024-05-28T13:58:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 -- 172.21.15.42:0/3344832248 wait complete. 2024-05-28T13:58:34.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 Processor -- start 2024-05-28T13:58:34.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 -- start start 2024-05-28T13:58:34.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f80afa50 0x7f60f8099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f8156a70 0x7f60f8073730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8156e50 0x7f60f8071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8160eb0 con 0x7f60f8156e50 2024-05-28T13:58:34.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8160d30 con 0x7f60f8156a70 2024-05-28T13:58:34.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f6100000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60f8161030 con 0x7f60f80afa50 2024-05-28T13:58:34.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f60fe000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8156e50 0x7f60f8071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f60fe000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8156e50 0x7f60f8071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38850/0 (socket says 172.21.15.42:38850) 2024-05-28T13:58:34.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.845+0000 7f60fe000640 1 -- 172.21.15.42:0/2467759422 learned_addr learned my addr 172.21.15.42:0/2467759422 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:34.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f60fcc00640 1 --2- 172.21.15.42:0/2467759422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f8156a70 0x7f60f8073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f60fd600640 1 --2- 172.21.15.42:0/2467759422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f80afa50 0x7f60f8099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f60fe000640 1 -- 172.21.15.42:0/2467759422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f80afa50 msgr2=0x7f60f8099920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f60fe000640 1 --2- 172.21.15.42:0/2467759422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f80afa50 0x7f60f8099920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f60fe000640 1 -- 172.21.15.42:0/2467759422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f8156a70 msgr2=0x7f60f8073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:34.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f60fe000640 1 --2- 172.21.15.42:0/2467759422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f8156a70 0x7f60f8073730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:34.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f60fe000640 1 -- 172.21.15.42:0/2467759422 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f8072690 con 0x7f60f8156e50 2024-05-28T13:58:34.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f60fe000640 1 --2- 172.21.15.42:0/2467759422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8156e50 0x7f60f8071d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f60f00af040 tx=0x7f60f0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:34.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f60f5600640 1 -- 172.21.15.42:0/2467759422 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f00b8880 con 0x7f60f8156e50 2024-05-28T13:58:34.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f60f5600640 1 -- 172.21.15.42:0/2467759422 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f00be070 con 0x7f60f8156e50 2024-05-28T13:58:34.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f6100000640 1 -- 172.21.15.42:0/2467759422 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f8072970 con 0x7f60f8156e50 2024-05-28T13:58:34.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f60f5600640 1 -- 172.21.15.42:0/2467759422 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f00b8c60 con 0x7f60f8156e50 2024-05-28T13:58:34.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f6100000640 1 -- 172.21.15.42:0/2467759422 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f8072f30 con 0x7f60f8156e50 2024-05-28T13:58:34.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f6100000640 1 -- 172.21.15.42:0/2467759422 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f8068a90 con 0x7f60f8156e50 2024-05-28T13:58:34.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.849+0000 7f60f5600640 1 -- 172.21.15.42:0/2467759422 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f60f00c2410 con 0x7f60f8156e50 2024-05-28T13:58:34.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.853+0000 7f60f5600640 1 --2- 172.21.15.42:0/2467759422 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60d80683b0 0x7f60d806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:34.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.853+0000 7f60f5600640 1 -- 172.21.15.42:0/2467759422 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f60f01216d0 con 0x7f60f8156e50 2024-05-28T13:58:34.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.853+0000 7f60fd600640 1 --2- 172.21.15.42:0/2467759422 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60d80683b0 0x7f60d806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:34.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.853+0000 7f60fd600640 1 --2- 172.21.15.42:0/2467759422 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60d80683b0 0x7f60d806a870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00c900 tx=0x7f60ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:34.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:34.857+0000 7f60f5600640 1 -- 172.21.15.42:0/2467759422 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f60f00c7020 con 0x7f60f8156e50 2024-05-28T13:58:35.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.053+0000 7f6100000640 1 -- 172.21.15.42:0/2467759422 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60f80af770 con 0x7f60f8156e50 2024-05-28T13:58:35.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.053+0000 7f60f5600640 1 -- 172.21.15.42:0/2467759422 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f60f00165e0 con 0x7f60f8156e50 2024-05-28T13:58:35.038 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:35.038 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1409117123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1409117123},{"type":"v1","addr":"172.21.15.42:6837","nonce":1409117123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/3753517701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3753517701},{"type":"v1","addr":"172.21.15.42:6836","nonce":3753517701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/882866492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":882866492},{"type":"v1","addr":"172.21.15.63:6834","nonce":882866492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T13:58:23.075354+0000","modified":"2024-05-28T13:58:26.084895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/1552779700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1552779700},{"type":"v1","addr":"172.21.15.63:6835","nonce":1552779700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-05-28T13:58:35.038 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-05-28T13:58:35.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.053+0000 7f6100000640 1 -- 172.21.15.42:0/2467759422 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60d80683b0 msgr2=0x7f60d806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.053+0000 7f6100000640 1 --2- 172.21.15.42:0/2467759422 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60d80683b0 0x7f60d806a870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00c900 tx=0x7f60ec008040 comp rx=0 tx=0).stop 2024-05-28T13:58:35.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.053+0000 7f6100000640 1 -- 172.21.15.42:0/2467759422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8156e50 msgr2=0x7f60f8071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.053+0000 7f6100000640 1 --2- 172.21.15.42:0/2467759422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8156e50 0x7f60f8071d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f60f00af040 tx=0x7f60f0019790 comp rx=0 tx=0).stop 2024-05-28T13:58:35.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.053+0000 7f6100000640 1 -- 172.21.15.42:0/2467759422 shutdown_connections 2024-05-28T13:58:35.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.053+0000 7f6100000640 1 --2- 172.21.15.42:0/2467759422 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60d80683b0 0x7f60d806a870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.053+0000 7f6100000640 1 --2- 172.21.15.42:0/2467759422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8156e50 0x7f60f8071d40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.053+0000 7f6100000640 1 --2- 172.21.15.42:0/2467759422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f8156a70 0x7f60f8073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.053+0000 7f6100000640 1 --2- 172.21.15.42:0/2467759422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f80afa50 0x7f60f8099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.053+0000 7f6100000640 1 -- 172.21.15.42:0/2467759422 >> 172.21.15.42:0/2467759422 conn(0x7f60f806bdf0 msgr2=0x7f60f81543a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:35.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.053+0000 7f6100000640 1 -- 172.21.15.42:0/2467759422 shutdown_connections 2024-05-28T13:58:35.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.053+0000 7f6100000640 1 -- 172.21.15.42:0/2467759422 wait complete. 2024-05-28T13:58:35.060 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:35.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.213+0000 7f1c4c600640 1 Processor -- start 2024-05-28T13:58:35.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.213+0000 7f1c4c600640 1 -- start start 2024-05-28T13:58:35.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.213+0000 7f1c4c600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c440b1860 0x7f1c4409b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.217+0000 7f1c4c600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1c4409b160 0x7f1c440ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.217+0000 7f1c4c600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c440710e0 con 0x7f1c440712e0 2024-05-28T13:58:35.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.217+0000 7f1c4c600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1c44071f90 con 0x7f1c4409b160 2024-05-28T13:58:35.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.217+0000 7f1c4c600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c44070c50 con 0x7f1c440b1860 2024-05-28T13:58:35.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.217+0000 7f1c49c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c440b1860 0x7f1c4409b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.217+0000 7f1c4a600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1c4409b160 0x7f1c440ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.217+0000 7f1c49c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c440b1860 0x7f1c4409b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:43226/0 (socket says 172.21.15.42:43226) 2024-05-28T13:58:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.217+0000 7f1c4a600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1c4409b160 0x7f1c440ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38360/0 (socket says 172.21.15.42:38360) 2024-05-28T13:58:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.217+0000 7f1c49c00640 1 -- 172.21.15.42:0/4119826799 learned_addr learned my addr 172.21.15.42:0/4119826799 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.217+0000 7f1c49c00640 1 -- 172.21.15.42:0/4119826799 >> v2:172.21.15.63:3300/0 conn(0x7f1c4409b160 msgr2=0x7f1c440ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.217+0000 7f1c49c00640 1 --2- 172.21.15.42:0/4119826799 >> v2:172.21.15.63:3300/0 conn(0x7f1c4409b160 0x7f1c440ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.217+0000 7f1c49c00640 1 -- 172.21.15.42:0/4119826799 >> v1:172.21.15.63:6789/0 conn(0x7f1c440712e0 legacy=0x7f1c4409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.217+0000 7f1c49c00640 1 -- 172.21.15.42:0/4119826799 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c44070040 con 0x7f1c440b1860 2024-05-28T13:58:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.217+0000 7f1c4a600640 1 --2- 172.21.15.42:0/4119826799 >> v2:172.21.15.63:3300/0 conn(0x7f1c4409b160 0x7f1c440ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:35.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c49c00640 1 --2- 172.21.15.42:0/4119826799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c440b1860 0x7f1c4409b9c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009870 tx=0x7f1c3801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=329fe7ecf5e808bc server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c43e00640 1 -- 172.21.15.42:0/4119826799 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c380b1070 con 0x7f1c440b1860 2024-05-28T13:58:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c43e00640 1 -- 172.21.15.42:0/4119826799 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c380ac410 con 0x7f1c440b1860 2024-05-28T13:58:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c43e00640 1 -- 172.21.15.42:0/4119826799 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c380ac7e0 con 0x7f1c440b1860 2024-05-28T13:58:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 -- 172.21.15.42:0/4119826799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c440b1860 msgr2=0x7f1c4409b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 --2- 172.21.15.42:0/4119826799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c440b1860 0x7f1c4409b9c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009870 tx=0x7f1c3801c4a0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 -- 172.21.15.42:0/4119826799 shutdown_connections 2024-05-28T13:58:35.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 --2- 172.21.15.42:0/4119826799 >> v2:172.21.15.63:3300/0 conn(0x7f1c4409b160 0x7f1c440ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 --2- 172.21.15.42:0/4119826799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c440b1860 0x7f1c4409b9c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 -- 172.21.15.42:0/4119826799 >> 172.21.15.42:0/4119826799 conn(0x7f1c440b3fa0 msgr2=0x7f1c440b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:35.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 -- 172.21.15.42:0/4119826799 shutdown_connections 2024-05-28T13:58:35.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 -- 172.21.15.42:0/4119826799 wait complete. 2024-05-28T13:58:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 Processor -- start 2024-05-28T13:58:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 -- start start 2024-05-28T13:58:35.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c440712e0 0x7f1c440b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c4409b160 0x7f1c441575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c440b1860 0x7f1c441faf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1c440cdc30 con 0x7f1c440b1860 2024-05-28T13:58:35.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c440b1c40 con 0x7f1c440712e0 2024-05-28T13:58:35.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4c600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c440cddb0 con 0x7f1c4409b160 2024-05-28T13:58:35.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c440b1860 0x7f1c441faf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.221+0000 7f1c4a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c440b1860 0x7f1c441faf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38866/0 (socket says 172.21.15.42:38866) 2024-05-28T13:58:35.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c4a600640 1 -- 172.21.15.42:0/1672596698 learned_addr learned my addr 172.21.15.42:0/1672596698 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:35.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c49c00640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c440712e0 0x7f1c440b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c49200640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c4409b160 0x7f1c441575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c4a600640 1 -- 172.21.15.42:0/1672596698 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c4409b160 msgr2=0x7f1c441575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c4a600640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c4409b160 0x7f1c441575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c4a600640 1 -- 172.21.15.42:0/1672596698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c440712e0 msgr2=0x7f1c440b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c4a600640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c440712e0 0x7f1c440b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c4a600640 1 -- 172.21.15.42:0/1672596698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c440b3950 con 0x7f1c440b1860 2024-05-28T13:58:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c49c00640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c440712e0 0x7f1c440b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c4a600640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c440b1860 0x7f1c441faf00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00efc0 tx=0x7f1c3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c41600640 1 -- 172.21.15.42:0/1672596698 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c018020 con 0x7f1c440b1860 2024-05-28T13:58:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c41600640 1 -- 172.21.15.42:0/1672596698 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c3c014480 con 0x7f1c440b1860 2024-05-28T13:58:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c41600640 1 -- 172.21.15.42:0/1672596698 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c014850 con 0x7f1c440b1860 2024-05-28T13:58:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c4c600640 1 -- 172.21.15.42:0/1672596698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c440b0540 con 0x7f1c440b1860 2024-05-28T13:58:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c4c600640 1 -- 172.21.15.42:0/1672596698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c440b0de0 con 0x7f1c440b1860 2024-05-28T13:58:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c4c600640 1 -- 172.21.15.42:0/1672596698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c44068a90 con 0x7f1c440b1860 2024-05-28T13:58:35.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c41600640 1 -- 172.21.15.42:0/1672596698 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1c3c01b020 con 0x7f1c440b1860 2024-05-28T13:58:35.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.225+0000 7f1c41600640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c24068380 0x7f1c2406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.229+0000 7f1c41600640 1 -- 172.21.15.42:0/1672596698 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1c3c0876d0 con 0x7f1c440b1860 2024-05-28T13:58:35.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.229+0000 7f1c49c00640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c24068380 0x7f1c2406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.229+0000 7f1c49c00640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c24068380 0x7f1c2406a840 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c380ae000 tx=0x7f1c380a6040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.233+0000 7f1c41600640 1 -- 172.21.15.42:0/1672596698 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1c3c022030 con 0x7f1c440b1860 2024-05-28T13:58:35.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.397+0000 7f1c4c600640 1 -- 172.21.15.42:0/1672596698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c440cfa50 con 0x7f1c440b1860 2024-05-28T13:58:35.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.397+0000 7f1c41600640 1 -- 172.21.15.42:0/1672596698 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16545 (secure 0 0 0) 0x7f1c3c010040 con 0x7f1c440b1860 2024-05-28T13:58:35.383 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:35.383 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:24.659233+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:35.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.401+0000 7f1c4c600640 1 -- 172.21.15.42:0/1672596698 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c24068380 msgr2=0x7f1c2406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.401+0000 7f1c4c600640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c24068380 0x7f1c2406a840 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c380ae000 tx=0x7f1c380a6040 comp rx=0 tx=0).stop 2024-05-28T13:58:35.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.401+0000 7f1c4c600640 1 -- 172.21.15.42:0/1672596698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c440b1860 msgr2=0x7f1c441faf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.401+0000 7f1c4c600640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c440b1860 0x7f1c441faf00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00efc0 tx=0x7f1c3c00c490 comp rx=0 tx=0).stop 2024-05-28T13:58:35.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.401+0000 7f1c4c600640 1 -- 172.21.15.42:0/1672596698 shutdown_connections 2024-05-28T13:58:35.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.401+0000 7f1c4c600640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c24068380 0x7f1c2406a840 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.401+0000 7f1c4c600640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c440b1860 0x7f1c441faf00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.401+0000 7f1c4c600640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c4409b160 0x7f1c441575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.401+0000 7f1c4c600640 1 --2- 172.21.15.42:0/1672596698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c440712e0 0x7f1c440b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.401+0000 7f1c4c600640 1 -- 172.21.15.42:0/1672596698 >> 172.21.15.42:0/1672596698 conn(0x7f1c440b3fa0 msgr2=0x7f1c440ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:35.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.401+0000 7f1c4c600640 1 -- 172.21.15.42:0/1672596698 shutdown_connections 2024-05-28T13:58:35.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.401+0000 7f1c4c600640 1 -- 172.21.15.42:0/1672596698 wait complete. 2024-05-28T13:58:35.404 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:58:35.404 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:58:35.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c3e00640 1 Processor -- start 2024-05-28T13:58:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c3e00640 1 -- start start 2024-05-28T13:58:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd2bc15e8c0 0x7fd2bc0b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2bc06c100 0x7fd2bc0af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c3e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2bc0b4440 con 0x7fd2bc0b0650 2024-05-28T13:58:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c3e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd2bc0ae420 con 0x7fd2bc15e8c0 2024-05-28T13:58:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c3e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2bc0b0f10 con 0x7fd2bc06c100 2024-05-28T13:58:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c1400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd2bc15e8c0 0x7fd2bc0b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c1400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd2bc15e8c0 0x7fd2bc0b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38882/0 (socket says 172.21.15.42:38882) 2024-05-28T13:58:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c1400640 1 -- 172.21.15.42:0/1024699116 learned_addr learned my addr 172.21.15.42:0/1024699116 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c1e00640 1 --2- 172.21.15.42:0/1024699116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2bc06c100 0x7fd2bc0af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c1400640 1 -- 172.21.15.42:0/1024699116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2bc06c100 msgr2=0x7fd2bc0af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c1400640 1 --2- 172.21.15.42:0/1024699116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2bc06c100 0x7fd2bc0af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c1400640 1 -- 172.21.15.42:0/1024699116 >> v1:172.21.15.63:6789/0 conn(0x7fd2bc0b0650 legacy=0x7fd2bc06b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.565+0000 7fd2c1400640 1 -- 172.21.15.42:0/1024699116 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2bc0ae670 con 0x7fd2bc15e8c0 2024-05-28T13:58:35.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.569+0000 7fd2c1400640 1 --2- 172.21.15.42:0/1024699116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2bc15e8c0 0x7fd2bc0b2240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009870 tx=0x7fd2b001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f713203ab50951a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:35.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.569+0000 7fd2bbe00640 1 -- 172.21.15.42:0/1024699116 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b00aa600 con 0x7fd2bc15e8c0 2024-05-28T13:58:35.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.569+0000 7fd2bbe00640 1 -- 172.21.15.42:0/1024699116 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b00af070 con 0x7fd2bc15e8c0 2024-05-28T13:58:35.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.569+0000 7fd2bbe00640 1 -- 172.21.15.42:0/1024699116 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b00aa9c0 con 0x7fd2bc15e8c0 2024-05-28T13:58:35.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.569+0000 7fd2c3e00640 1 -- 172.21.15.42:0/1024699116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2bc15e8c0 msgr2=0x7fd2bc0b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.569+0000 7fd2c3e00640 1 --2- 172.21.15.42:0/1024699116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2bc15e8c0 0x7fd2bc0b2240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009870 tx=0x7fd2b001c4a0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.569+0000 7fd2c3e00640 1 -- 172.21.15.42:0/1024699116 shutdown_connections 2024-05-28T13:58:35.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.569+0000 7fd2c3e00640 1 --2- 172.21.15.42:0/1024699116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2bc06c100 0x7fd2bc0af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.569+0000 7fd2c3e00640 1 --2- 172.21.15.42:0/1024699116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2bc15e8c0 0x7fd2bc0b2240 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.569+0000 7fd2c3e00640 1 -- 172.21.15.42:0/1024699116 >> 172.21.15.42:0/1024699116 conn(0x7fd2bc0b6960 msgr2=0x7fd2bc0b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:35.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.569+0000 7fd2c3e00640 1 -- 172.21.15.42:0/1024699116 shutdown_connections 2024-05-28T13:58:35.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.569+0000 7fd2c3e00640 1 -- 172.21.15.42:0/1024699116 wait complete. 2024-05-28T13:58:35.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c3e00640 1 Processor -- start 2024-05-28T13:58:35.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c3e00640 1 -- start start 2024-05-28T13:58:35.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2bc06c100 0x7fd2bc1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2bc0b0650 0x7fd2bc1544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2bc15e8c0 0x7fd2bc1f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c3e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc0b0020 con 0x7fd2bc06c100 2024-05-28T13:58:35.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c3e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc0afea0 con 0x7fd2bc0b0650 2024-05-28T13:58:35.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c3e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2bc0b01a0 con 0x7fd2bc15e8c0 2024-05-28T13:58:35.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2bc06c100 0x7fd2bc1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2bc06c100 0x7fd2bc1578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38896/0 (socket says 172.21.15.42:38896) 2024-05-28T13:58:35.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c1400640 1 -- 172.21.15.42:0/878124271 learned_addr learned my addr 172.21.15.42:0/878124271 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:35.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c1e00640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2bc15e8c0 0x7fd2bc1f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c0a00640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2bc0b0650 0x7fd2bc1544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c1400640 1 -- 172.21.15.42:0/878124271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2bc15e8c0 msgr2=0x7fd2bc1f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c1400640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2bc15e8c0 0x7fd2bc1f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c1400640 1 -- 172.21.15.42:0/878124271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2bc0b0650 msgr2=0x7fd2bc1544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c1400640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2bc0b0650 0x7fd2bc1544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c1400640 1 -- 172.21.15.42:0/878124271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2bc0d1360 con 0x7fd2bc06c100 2024-05-28T13:58:35.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c0a00640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2bc0b0650 0x7fd2bc1544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:35.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c1400640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2bc06c100 0x7fd2bc1578a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00ac040 tx=0x7fd2b001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:35.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2b9600640 1 -- 172.21.15.42:0/878124271 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b00bd5f0 con 0x7fd2bc06c100 2024-05-28T13:58:35.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2b9600640 1 -- 172.21.15.42:0/878124271 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b00bd7b0 con 0x7fd2bc06c100 2024-05-28T13:58:35.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2b9600640 1 -- 172.21.15.42:0/878124271 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b00bdba0 con 0x7fd2bc06c100 2024-05-28T13:58:35.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c3e00640 1 -- 172.21.15.42:0/878124271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2bc0d02c0 con 0x7fd2bc06c100 2024-05-28T13:58:35.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c3e00640 1 -- 172.21.15.42:0/878124271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2bc0d0880 con 0x7fd2bc06c100 2024-05-28T13:58:35.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.573+0000 7fd2c3e00640 1 -- 172.21.15.42:0/878124271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2bc09f7a0 con 0x7fd2bc06c100 2024-05-28T13:58:35.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.577+0000 7fd2b9600640 1 -- 172.21.15.42:0/878124271 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd2b00a9070 con 0x7fd2bc06c100 2024-05-28T13:58:35.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.577+0000 7fd2b9600640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd29c068380 0x7fd29c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.577+0000 7fd2b9600640 1 -- 172.21.15.42:0/878124271 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd2b01213b0 con 0x7fd2bc06c100 2024-05-28T13:58:35.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.577+0000 7fd2c0a00640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd29c068380 0x7fd29c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.577+0000 7fd2c0a00640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd29c068380 0x7fd29c06a840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc15a4d0 tx=0x7fd2a8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:35.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.581+0000 7fd2b9600640 1 -- 172.21.15.42:0/878124271 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd2b00c3060 con 0x7fd2bc06c100 2024-05-28T13:58:35.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.745+0000 7fd2c3e00640 1 -- 172.21.15.42:0/878124271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd2bc15e080 con 0x7fd2bc06c100 2024-05-28T13:58:35.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.745+0000 7fd2b9600640 1 -- 172.21.15.42:0/878124271 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fd2b0016150 con 0x7fd2bc06c100 2024-05-28T13:58:35.733 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:58:35.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.749+0000 7fd2c3e00640 1 -- 172.21.15.42:0/878124271 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd29c068380 msgr2=0x7fd29c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.749+0000 7fd2c3e00640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd29c068380 0x7fd29c06a840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc15a4d0 tx=0x7fd2a8009040 comp rx=0 tx=0).stop 2024-05-28T13:58:35.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.749+0000 7fd2c3e00640 1 -- 172.21.15.42:0/878124271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2bc06c100 msgr2=0x7fd2bc1578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.749+0000 7fd2c3e00640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2bc06c100 0x7fd2bc1578a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00ac040 tx=0x7fd2b001ace0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.749+0000 7fd2c3e00640 1 -- 172.21.15.42:0/878124271 shutdown_connections 2024-05-28T13:58:35.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.749+0000 7fd2c3e00640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd29c068380 0x7fd29c06a840 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.749+0000 7fd2c3e00640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2bc15e8c0 0x7fd2bc1f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.749+0000 7fd2c3e00640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2bc0b0650 0x7fd2bc1544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.749+0000 7fd2c3e00640 1 --2- 172.21.15.42:0/878124271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2bc06c100 0x7fd2bc1578a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.749+0000 7fd2c3e00640 1 -- 172.21.15.42:0/878124271 >> 172.21.15.42:0/878124271 conn(0x7fd2bc0b6960 msgr2=0x7fd2bc0b5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:35.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.749+0000 7fd2c3e00640 1 -- 172.21.15.42:0/878124271 shutdown_connections 2024-05-28T13:58:35.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.749+0000 7fd2c3e00640 1 -- 172.21.15.42:0/878124271 wait complete. 2024-05-28T13:58:35.753 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:58:35.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b9ea00640 1 Processor -- start 2024-05-28T13:58:35.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b9ea00640 1 -- start start 2024-05-28T13:58:35.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b9ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4b9806cfb0 0x7f4b9815f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b9ea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4b9809aa50 0x7f4b9806d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b9ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b980b32f0 0x7f4b980b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b9ea00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f4b980d0800 con 0x7f4b9806cfb0 2024-05-28T13:58:35.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b9ea00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4b9806df20 con 0x7f4b9809aa50 2024-05-28T13:58:35.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b9ea00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9815fe30 con 0x7f4b980b32f0 2024-05-28T13:58:35.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b97e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4b9806cfb0 0x7f4b9815f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b97e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4b9806cfb0 0x7f4b9815f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38898/0 (socket says 172.21.15.42:38898) 2024-05-28T13:58:35.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b97e00640 1 -- 172.21.15.42:0/4029551772 learned_addr learned my addr 172.21.15.42:0/4029551772 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:35.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b8fe00640 1 --2- 172.21.15.42:0/4029551772 >> v2:172.21.15.63:3300/0 conn(0x7f4b9809aa50 0x7f4b9806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b9ca00640 1 --2- 172.21.15.42:0/4029551772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b980b32f0 0x7f4b980b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b97e00640 1 -- 172.21.15.42:0/4029551772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b980b32f0 msgr2=0x7f4b980b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:35.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b97e00640 1 --2- 172.21.15.42:0/4029551772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b980b32f0 0x7f4b980b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b97e00640 1 -- 172.21.15.42:0/4029551772 >> v2:172.21.15.63:3300/0 conn(0x7f4b9809aa50 msgr2=0x7f4b9806d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:35.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b97e00640 1 --2- 172.21.15.42:0/4029551772 >> v2:172.21.15.63:3300/0 conn(0x7f4b9809aa50 0x7f4b9806d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b97e00640 1 -- 172.21.15.42:0/4029551772 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9809bb00 con 0x7f4b9806cfb0 2024-05-28T13:58:35.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b9ca00640 1 --2- 172.21.15.42:0/4029551772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b980b32f0 0x7f4b980b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:35.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.913+0000 7f4b8fe00640 1 --2- 172.21.15.42:0/4029551772 >> v2:172.21.15.63:3300/0 conn(0x7f4b9809aa50 0x7f4b9806d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:35.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.917+0000 7f4b97e00640 1 --2- 172.21.15.42:0/4029551772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b9806cfb0 0x7f4b9815f8f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4b980d3520 tx=0x7f4b8801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e41eeaefc378a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:35.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.917+0000 7f4b97400640 1 -- 172.21.15.42:0/4029551772 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b880aa660 con 0x7f4b9806cfb0 2024-05-28T13:58:35.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.917+0000 7f4b97400640 1 -- 172.21.15.42:0/4029551772 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b880af070 con 0x7f4b9806cfb0 2024-05-28T13:58:35.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.917+0000 7f4b97400640 1 -- 172.21.15.42:0/4029551772 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b880aaa20 con 0x7f4b9806cfb0 2024-05-28T13:58:35.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.917+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4029551772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b9806cfb0 msgr2=0x7f4b9815f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.917+0000 7f4b9ea00640 1 --2- 172.21.15.42:0/4029551772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b9806cfb0 0x7f4b9815f8f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4b980d3520 tx=0x7f4b8801c520 comp rx=0 tx=0).stop 2024-05-28T13:58:35.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.917+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4029551772 shutdown_connections 2024-05-28T13:58:35.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.917+0000 7f4b9ea00640 1 --2- 172.21.15.42:0/4029551772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b980b32f0 0x7f4b980b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.917+0000 7f4b9ea00640 1 --2- 172.21.15.42:0/4029551772 >> v2:172.21.15.63:3300/0 conn(0x7f4b9809aa50 0x7f4b9806d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.917+0000 7f4b9ea00640 1 --2- 172.21.15.42:0/4029551772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b9806cfb0 0x7f4b9815f8f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.917+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4029551772 >> 172.21.15.42:0/4029551772 conn(0x7f4b980b40d0 msgr2=0x7f4b980b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:35.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.917+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4029551772 shutdown_connections 2024-05-28T13:58:35.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.917+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4029551772 wait complete. 2024-05-28T13:58:35.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ea00640 1 Processor -- start 2024-05-28T13:58:35.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ea00640 1 -- start start 2024-05-28T13:58:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b9806cfb0 0x7f4b98158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b9809aa50 0x7f4b980b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b980b32f0 0x7f4b98159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9809b540 con 0x7f4b980b32f0 2024-05-28T13:58:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9809b260 con 0x7f4b9806cfb0 2024-05-28T13:58:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ea00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9809b820 con 0x7f4b9809aa50 2024-05-28T13:58:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b97e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b9806cfb0 0x7f4b98158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b8fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b9809aa50 0x7f4b980b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b980b32f0 0x7f4b98159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b97e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b9806cfb0 0x7f4b98158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38398/0 (socket says 172.21.15.42:38398) 2024-05-28T13:58:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b97e00640 1 -- 172.21.15.42:0/4163508001 learned_addr learned my addr 172.21.15.42:0/4163508001 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:35.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ca00640 1 -- 172.21.15.42:0/4163508001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b9809aa50 msgr2=0x7f4b980b6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ca00640 1 --2- 172.21.15.42:0/4163508001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b9809aa50 0x7f4b980b6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ca00640 1 -- 172.21.15.42:0/4163508001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b9806cfb0 msgr2=0x7f4b98158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:35.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ca00640 1 --2- 172.21.15.42:0/4163508001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b9806cfb0 0x7f4b98158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:35.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ca00640 1 -- 172.21.15.42:0/4163508001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b980b7a00 con 0x7f4b980b32f0 2024-05-28T13:58:35.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b8fe00640 1 --2- 172.21.15.42:0/4163508001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b9809aa50 0x7f4b980b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:35.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b97e00640 1 --2- 172.21.15.42:0/4163508001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b9806cfb0 0x7f4b98158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:35.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ca00640 1 --2- 172.21.15.42:0/4163508001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b980b32f0 0x7f4b98159ac0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b90007db0 tx=0x7f4b9000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:35.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b94c00640 1 -- 172.21.15.42:0/4163508001 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9001a070 con 0x7f4b980b32f0 2024-05-28T13:58:35.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b94c00640 1 -- 172.21.15.42:0/4163508001 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b900039a0 con 0x7f4b980b32f0 2024-05-28T13:58:35.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b94c00640 1 -- 172.21.15.42:0/4163508001 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90003d90 con 0x7f4b980b32f0 2024-05-28T13:58:35.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4163508001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b980b7ce0 con 0x7f4b980b32f0 2024-05-28T13:58:35.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4163508001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b980b82a0 con 0x7f4b980b32f0 2024-05-28T13:58:35.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.921+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4163508001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b980a06c0 con 0x7f4b980b32f0 2024-05-28T13:58:35.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.925+0000 7f4b94c00640 1 -- 172.21.15.42:0/4163508001 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4b9001d070 con 0x7f4b980b32f0 2024-05-28T13:58:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.925+0000 7f4b94c00640 1 --2- 172.21.15.42:0/4163508001 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b74068380 0x7f4b7406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.925+0000 7f4b94c00640 1 -- 172.21.15.42:0/4163508001 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4b90086a20 con 0x7f4b980b32f0 2024-05-28T13:58:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.925+0000 7f4b97e00640 1 --2- 172.21.15.42:0/4163508001 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b74068380 0x7f4b7406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.925+0000 7f4b97e00640 1 --2- 172.21.15.42:0/4163508001 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b74068380 0x7f4b7406a840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4b8801abd0 tx=0x7f4b880a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:35.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:35.929+0000 7f4b94c00640 1 -- 172.21.15.42:0/4163508001 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4b90010070 con 0x7f4b980b32f0 2024-05-28T13:58:36.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.093+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4163508001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4b980d1ed0 con 0x7f4b980b32f0 2024-05-28T13:58:36.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.093+0000 7f4b94c00640 1 -- 172.21.15.42:0/4163508001 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f4b90056a20 con 0x7f4b980b32f0 2024-05-28T13:58:36.080 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:58:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.097+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4163508001 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b74068380 msgr2=0x7f4b7406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.097+0000 7f4b9ea00640 1 --2- 172.21.15.42:0/4163508001 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b74068380 0x7f4b7406a840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4b8801abd0 tx=0x7f4b880a7000 comp rx=0 tx=0).stop 2024-05-28T13:58:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.097+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4163508001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b980b32f0 msgr2=0x7f4b98159ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.097+0000 7f4b9ea00640 1 --2- 172.21.15.42:0/4163508001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b980b32f0 0x7f4b98159ac0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b90007db0 tx=0x7f4b9000d4a0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.097+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4163508001 shutdown_connections 2024-05-28T13:58:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.097+0000 7f4b9ea00640 1 --2- 172.21.15.42:0/4163508001 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b74068380 0x7f4b7406a840 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.097+0000 7f4b9ea00640 1 --2- 172.21.15.42:0/4163508001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b980b32f0 0x7f4b98159ac0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.097+0000 7f4b9ea00640 1 --2- 172.21.15.42:0/4163508001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b9809aa50 0x7f4b980b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.097+0000 7f4b9ea00640 1 --2- 172.21.15.42:0/4163508001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b9806cfb0 0x7f4b98158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.097+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4163508001 >> 172.21.15.42:0/4163508001 conn(0x7f4b980b40d0 msgr2=0x7f4b980cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.097+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4163508001 shutdown_connections 2024-05-28T13:58:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.097+0000 7f4b9ea00640 1 -- 172.21.15.42:0/4163508001 wait complete. 2024-05-28T13:58:36.102 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:58:36.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e0c400640 1 Processor -- start 2024-05-28T13:58:36.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e0c400640 1 -- start start 2024-05-28T13:58:36.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e0c400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e04148090 0x7f3e040b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e0c400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3e040724f0 0x7f3e040aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e0c400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e040b3690 con 0x7f3e040b4c10 2024-05-28T13:58:36.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e0c400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3e04148470 con 0x7f3e040724f0 2024-05-28T13:58:36.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e0c400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3e040b4730 con 0x7f3e04148090 2024-05-28T13:58:36.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e09000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f3e040b4c10 0x7f3e04071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60650/0 (socket says 172.21.15.42:60650) 2024-05-28T13:58:36.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e09000640 1 -- 172.21.15.42:0/486691318 learned_addr learned my addr 172.21.15.42:0/486691318 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:36.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e09a00640 1 --2- 172.21.15.42:0/486691318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e04148090 0x7f3e040b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e0a400640 1 --2- 172.21.15.42:0/486691318 >> v2:172.21.15.63:3300/0 conn(0x7f3e040724f0 0x7f3e040aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e03e00640 1 -- 172.21.15.42:0/486691318 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2761264856 0 0) 0x7f3e040b3690 con 0x7f3e040b4c10 2024-05-28T13:58:36.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e03e00640 1 -- 172.21.15.42:0/486691318 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3dec003530 con 0x7f3e040b4c10 2024-05-28T13:58:36.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e0a400640 1 -- 172.21.15.42:0/486691318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e04148090 msgr2=0x7f3e040b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:36.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e0a400640 1 --2- 172.21.15.42:0/486691318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e04148090 0x7f3e040b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e0a400640 1 -- 172.21.15.42:0/486691318 >> v1:172.21.15.42:6789/0 conn(0x7f3e040b4c10 legacy=0x7f3e04071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.261+0000 7f3e0a400640 1 -- 172.21.15.42:0/486691318 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e040b3970 con 0x7f3e040724f0 2024-05-28T13:58:36.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.265+0000 7f3e0a400640 1 --2- 172.21.15.42:0/486691318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e040724f0 0x7f3e040aeb40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009d90 tx=0x7f3dfc01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3c8d81a3ea6ccc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:36.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.265+0000 7f3e03e00640 1 -- 172.21.15.42:0/486691318 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0b2070 con 0x7f3e040724f0 2024-05-28T13:58:36.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.265+0000 7f3e03e00640 1 -- 172.21.15.42:0/486691318 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dfc0a5970 con 0x7f3e040724f0 2024-05-28T13:58:36.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.265+0000 7f3e03e00640 1 -- 172.21.15.42:0/486691318 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0a5d20 con 0x7f3e040724f0 2024-05-28T13:58:36.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.265+0000 7f3e0c400640 1 -- 172.21.15.42:0/486691318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e040724f0 msgr2=0x7f3e040aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.265+0000 7f3e0c400640 1 --2- 172.21.15.42:0/486691318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e040724f0 0x7f3e040aeb40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009d90 tx=0x7f3dfc01c8b0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.265+0000 7f3e0c400640 1 -- 172.21.15.42:0/486691318 shutdown_connections 2024-05-28T13:58:36.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.265+0000 7f3e0c400640 1 --2- 172.21.15.42:0/486691318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e040724f0 0x7f3e040aeb40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.265+0000 7f3e0c400640 1 --2- 172.21.15.42:0/486691318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e04148090 0x7f3e040b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.265+0000 7f3e0c400640 1 -- 172.21.15.42:0/486691318 >> 172.21.15.42:0/486691318 conn(0x7f3e040b21f0 msgr2=0x7f3e040b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:36.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.265+0000 7f3e0c400640 1 -- 172.21.15.42:0/486691318 shutdown_connections 2024-05-28T13:58:36.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.265+0000 7f3e0c400640 1 -- 172.21.15.42:0/486691318 wait complete. 2024-05-28T13:58:36.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e0c400640 1 Processor -- start 2024-05-28T13:58:36.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e0c400640 1 -- start start 2024-05-28T13:58:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e0c400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040724f0 0x7f3e0413ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e0c400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e04148090 0x7f3e0415b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e0c400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e041424c0 0x7f3e0413fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e0c400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3e040b57e0 con 0x7f3e040724f0 2024-05-28T13:58:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e0c400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3e040b5660 con 0x7f3e04148090 2024-05-28T13:58:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e0c400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3e040af890 con 0x7f3e041424c0 2024-05-28T13:58:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e09a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040724f0 0x7f3e0413ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e09a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040724f0 0x7f3e0413ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38920/0 (socket says 172.21.15.42:38920) 2024-05-28T13:58:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e09a00640 1 -- 172.21.15.42:0/3464499822 learned_addr learned my addr 172.21.15.42:0/3464499822 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e0a400640 1 --2- 172.21.15.42:0/3464499822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e041424c0 0x7f3e0413fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e09000640 1 --2- 172.21.15.42:0/3464499822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e04148090 0x7f3e0415b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e09a00640 1 -- 172.21.15.42:0/3464499822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e041424c0 msgr2=0x7f3e0413fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e09a00640 1 --2- 172.21.15.42:0/3464499822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e041424c0 0x7f3e0413fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e09a00640 1 -- 172.21.15.42:0/3464499822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e04148090 msgr2=0x7f3e0415b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e09a00640 1 --2- 172.21.15.42:0/3464499822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e04148090 0x7f3e0415b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e09a00640 1 -- 172.21.15.42:0/3464499822 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0413cf80 con 0x7f3e040724f0 2024-05-28T13:58:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e09a00640 1 --2- 172.21.15.42:0/3464499822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040724f0 0x7f3e0413ca40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3df800e870 tx=0x7f3df800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:36.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e01600640 1 -- 172.21.15.42:0/3464499822 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8017070 con 0x7f3e040724f0 2024-05-28T13:58:36.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e01600640 1 -- 172.21.15.42:0/3464499822 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df800c940 con 0x7f3e040724f0 2024-05-28T13:58:36.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e0c400640 1 -- 172.21.15.42:0/3464499822 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0415bbb0 con 0x7f3e040724f0 2024-05-28T13:58:36.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e01600640 1 -- 172.21.15.42:0/3464499822 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df800ccf0 con 0x7f3e040724f0 2024-05-28T13:58:36.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.269+0000 7f3e0c400640 1 -- 172.21.15.42:0/3464499822 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0415c170 con 0x7f3e040724f0 2024-05-28T13:58:36.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.273+0000 7f3e0c400640 1 -- 172.21.15.42:0/3464499822 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e040738b0 con 0x7f3e040724f0 2024-05-28T13:58:36.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.273+0000 7f3e01600640 1 -- 172.21.15.42:0/3464499822 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3df801a070 con 0x7f3e040724f0 2024-05-28T13:58:36.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.273+0000 7f3e01600640 1 --2- 172.21.15.42:0/3464499822 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de40683b0 0x7f3de406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.273+0000 7f3e01600640 1 -- 172.21.15.42:0/3464499822 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3df8086800 con 0x7f3e040724f0 2024-05-28T13:58:36.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.273+0000 7f3e09000640 1 --2- 172.21.15.42:0/3464499822 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de40683b0 0x7f3de406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.273+0000 7f3e09000640 1 --2- 172.21.15.42:0/3464499822 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de40683b0 0x7f3de406a870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3e04147190 tx=0x7f3df0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:36.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.277+0000 7f3e01600640 1 -- 172.21.15.42:0/3464499822 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3df8056db0 con 0x7f3e040724f0 2024-05-28T13:58:36.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.441+0000 7f3e0c400640 1 -- 172.21.15.42:0/3464499822 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3e0404fdd0 con 0x7f3e040724f0 2024-05-28T13:58:36.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.441+0000 7f3e01600640 1 -- 172.21.15.42:0/3464499822 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f3df80566b0 con 0x7f3e040724f0 2024-05-28T13:58:36.427 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:58:36.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.441+0000 7f3e0c400640 1 -- 172.21.15.42:0/3464499822 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de40683b0 msgr2=0x7f3de406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.441+0000 7f3e0c400640 1 --2- 172.21.15.42:0/3464499822 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de40683b0 0x7f3de406a870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3e04147190 tx=0x7f3df0009040 comp rx=0 tx=0).stop 2024-05-28T13:58:36.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.441+0000 7f3e0c400640 1 -- 172.21.15.42:0/3464499822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040724f0 msgr2=0x7f3e0413ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.441+0000 7f3e0c400640 1 --2- 172.21.15.42:0/3464499822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040724f0 0x7f3e0413ca40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3df800e870 tx=0x7f3df800ec70 comp rx=0 tx=0).stop 2024-05-28T13:58:36.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.445+0000 7f3e0c400640 1 -- 172.21.15.42:0/3464499822 shutdown_connections 2024-05-28T13:58:36.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.445+0000 7f3e0c400640 1 --2- 172.21.15.42:0/3464499822 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de40683b0 0x7f3de406a870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.445+0000 7f3e0c400640 1 --2- 172.21.15.42:0/3464499822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e041424c0 0x7f3e0413fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.445+0000 7f3e0c400640 1 --2- 172.21.15.42:0/3464499822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e04148090 0x7f3e0415b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.445+0000 7f3e0c400640 1 --2- 172.21.15.42:0/3464499822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040724f0 0x7f3e0413ca40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.445+0000 7f3e0c400640 1 -- 172.21.15.42:0/3464499822 >> 172.21.15.42:0/3464499822 conn(0x7f3e040b21f0 msgr2=0x7f3e0406e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:36.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.445+0000 7f3e0c400640 1 -- 172.21.15.42:0/3464499822 shutdown_connections 2024-05-28T13:58:36.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.445+0000 7f3e0c400640 1 -- 172.21.15.42:0/3464499822 wait complete. 2024-05-28T13:58:36.448 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T13:58:36.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832c000640 1 Processor -- start 2024-05-28T13:58:36.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832c000640 1 -- start start 2024-05-28T13:58:36.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832c000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8324156a70 0x7f832406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832c000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8324156e50 0x7f83240af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832c000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f83240af880 0x7f83240d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832c000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f832415cb90 con 0x7f83240af880 2024-05-28T13:58:36.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832c000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f832415ce50 con 0x7f8324156a70 2024-05-28T13:58:36.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832c000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f832415d130 con 0x7f8324156e50 2024-05-28T13:58:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832a000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f83240af880 0x7f83240d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832a000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f83240af880 0x7f83240d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38922/0 (socket says 172.21.15.42:38922) 2024-05-28T13:58:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832a000640 1 -- 172.21.15.42:0/876862453 learned_addr learned my addr 172.21.15.42:0/876862453 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f8329600640 1 --2- 172.21.15.42:0/876862453 >> v2:172.21.15.63:3300/0 conn(0x7f8324156a70 0x7f832406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f8328c00640 1 --2- 172.21.15.42:0/876862453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8324156e50 0x7f83240af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832a000640 1 -- 172.21.15.42:0/876862453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8324156e50 msgr2=0x7f83240af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832a000640 1 --2- 172.21.15.42:0/876862453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8324156e50 0x7f83240af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832a000640 1 -- 172.21.15.42:0/876862453 >> v2:172.21.15.63:3300/0 conn(0x7f8324156a70 msgr2=0x7f832406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832a000640 1 --2- 172.21.15.42:0/876862453 >> v2:172.21.15.63:3300/0 conn(0x7f8324156a70 0x7f832406dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.605+0000 7f832a000640 1 -- 172.21.15.42:0/876862453 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832415d410 con 0x7f83240af880 2024-05-28T13:58:36.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.609+0000 7f832a000640 1 --2- 172.21.15.42:0/876862453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83240af880 0x7f83240d06f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f83240d13a0 tx=0x7f831c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc6c7b4f3ab1f5b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.609+0000 7f8323e00640 1 -- 172.21.15.42:0/876862453 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c0a5780 con 0x7f83240af880 2024-05-28T13:58:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.609+0000 7f8323e00640 1 -- 172.21.15.42:0/876862453 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f831c0a5920 con 0x7f83240af880 2024-05-28T13:58:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.609+0000 7f8323e00640 1 -- 172.21.15.42:0/876862453 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c0a5cf0 con 0x7f83240af880 2024-05-28T13:58:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.609+0000 7f832c000640 1 -- 172.21.15.42:0/876862453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83240af880 msgr2=0x7f83240d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.609+0000 7f832c000640 1 --2- 172.21.15.42:0/876862453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83240af880 0x7f83240d06f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f83240d13a0 tx=0x7f831c01c9f0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.609+0000 7f832c000640 1 -- 172.21.15.42:0/876862453 shutdown_connections 2024-05-28T13:58:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.609+0000 7f832c000640 1 --2- 172.21.15.42:0/876862453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83240af880 0x7f83240d06f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.609+0000 7f832c000640 1 --2- 172.21.15.42:0/876862453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8324156e50 0x7f83240af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.609+0000 7f832c000640 1 --2- 172.21.15.42:0/876862453 >> v2:172.21.15.63:3300/0 conn(0x7f8324156a70 0x7f832406dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.609+0000 7f832c000640 1 -- 172.21.15.42:0/876862453 >> 172.21.15.42:0/876862453 conn(0x7f832406bdf0 msgr2=0x7f832406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:36.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.609+0000 7f832c000640 1 -- 172.21.15.42:0/876862453 shutdown_connections 2024-05-28T13:58:36.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.609+0000 7f832c000640 1 -- 172.21.15.42:0/876862453 wait complete. 2024-05-28T13:58:36.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f832c000640 1 Processor -- start 2024-05-28T13:58:36.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f832c000640 1 -- start start 2024-05-28T13:58:36.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f832c000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83240af880 0x7f832415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f832c000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8324156a70 0x7f8324099a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f832c000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8324156e50 0x7f83241667d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f832c000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8324160eb0 con 0x7f8324156a70 2024-05-28T13:58:36.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f832c000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8324160d30 con 0x7f8324156e50 2024-05-28T13:58:36.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f832c000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8324161030 con 0x7f83240af880 2024-05-28T13:58:36.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f8328c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8324156a70 0x7f8324099a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f8328c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8324156a70 0x7f8324099a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38938/0 (socket says 172.21.15.42:38938) 2024-05-28T13:58:36.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f8328c00640 1 -- 172.21.15.42:0/3897314451 learned_addr learned my addr 172.21.15.42:0/3897314451 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:36.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f8329600640 1 --2- 172.21.15.42:0/3897314451 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83240af880 0x7f832415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f832a000640 1 --2- 172.21.15.42:0/3897314451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8324156e50 0x7f83241667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f8328c00640 1 -- 172.21.15.42:0/3897314451 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83240af880 msgr2=0x7f832415c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f8328c00640 1 --2- 172.21.15.42:0/3897314451 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83240af880 0x7f832415c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f8328c00640 1 -- 172.21.15.42:0/3897314451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8324156e50 msgr2=0x7f83241667d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f8328c00640 1 --2- 172.21.15.42:0/3897314451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8324156e50 0x7f83241667d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f8328c00640 1 -- 172.21.15.42:0/3897314451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832415b800 con 0x7f8324156a70 2024-05-28T13:58:36.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f8328c00640 1 --2- 172.21.15.42:0/3897314451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8324156a70 0x7f8324099a90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f831000d620 tx=0x7f831000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:36.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f8321600640 1 -- 172.21.15.42:0/3897314451 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831000b620 con 0x7f8324156a70 2024-05-28T13:58:36.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f8321600640 1 -- 172.21.15.42:0/3897314451 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8310019070 con 0x7f8324156a70 2024-05-28T13:58:36.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f8321600640 1 -- 172.21.15.42:0/3897314451 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831000b9e0 con 0x7f8324156a70 2024-05-28T13:58:36.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f832c000640 1 -- 172.21.15.42:0/3897314451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832415bae0 con 0x7f8324156a70 2024-05-28T13:58:36.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f832c000640 1 -- 172.21.15.42:0/3897314451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832414e2d0 con 0x7f8324156a70 2024-05-28T13:58:36.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.613+0000 7f832c000640 1 -- 172.21.15.42:0/3897314451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8324068a90 con 0x7f8324156a70 2024-05-28T13:58:36.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.617+0000 7f8321600640 1 -- 172.21.15.42:0/3897314451 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f831000bb80 con 0x7f8324156a70 2024-05-28T13:58:36.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.617+0000 7f8321600640 1 --2- 172.21.15.42:0/3897314451 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8304068380 0x7f830406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.617+0000 7f8321600640 1 -- 172.21.15.42:0/3897314451 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8310086c80 con 0x7f8324156a70 2024-05-28T13:58:36.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.617+0000 7f8329600640 1 --2- 172.21.15.42:0/3897314451 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8304068380 0x7f830406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.617+0000 7f8329600640 1 --2- 172.21.15.42:0/3897314451 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8304068380 0x7f830406a840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f832406eb90 tx=0x7f8318008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:36.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.621+0000 7f8321600640 1 -- 172.21.15.42:0/3897314451 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8310056be0 con 0x7f8324156a70 2024-05-28T13:58:36.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.821+0000 7f832c000640 1 -- 172.21.15.42:0/3897314451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f832414ca80 con 0x7f8324156a70 2024-05-28T13:58:36.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.821+0000 7f8321600640 1 -- 172.21.15.42:0/3897314451 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f831005aee0 con 0x7f8324156a70 2024-05-28T13:58:36.806 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:36.806 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1409117123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1409117123},{"type":"v1","addr":"172.21.15.42:6837","nonce":1409117123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/3753517701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3753517701},{"type":"v1","addr":"172.21.15.42:6836","nonce":3753517701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/882866492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":882866492},{"type":"v1","addr":"172.21.15.63:6834","nonce":882866492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T13:58:23.075354+0000","modified":"2024-05-28T13:58:26.084895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/1552779700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1552779700},{"type":"v1","addr":"172.21.15.63:6835","nonce":1552779700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-05-28T13:58:36.807 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-05-28T13:58:36.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.821+0000 7f832c000640 1 -- 172.21.15.42:0/3897314451 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8304068380 msgr2=0x7f830406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.821+0000 7f832c000640 1 --2- 172.21.15.42:0/3897314451 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8304068380 0x7f830406a840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f832406eb90 tx=0x7f8318008040 comp rx=0 tx=0).stop 2024-05-28T13:58:36.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.821+0000 7f832c000640 1 -- 172.21.15.42:0/3897314451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8324156a70 msgr2=0x7f8324099a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.821+0000 7f832c000640 1 --2- 172.21.15.42:0/3897314451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8324156a70 0x7f8324099a90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f831000d620 tx=0x7f831000dae0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.821+0000 7f832c000640 1 -- 172.21.15.42:0/3897314451 shutdown_connections 2024-05-28T13:58:36.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.821+0000 7f832c000640 1 --2- 172.21.15.42:0/3897314451 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8304068380 0x7f830406a840 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.821+0000 7f832c000640 1 --2- 172.21.15.42:0/3897314451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8324156e50 0x7f83241667d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.821+0000 7f832c000640 1 --2- 172.21.15.42:0/3897314451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8324156a70 0x7f8324099a90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.821+0000 7f832c000640 1 --2- 172.21.15.42:0/3897314451 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83240af880 0x7f832415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.821+0000 7f832c000640 1 -- 172.21.15.42:0/3897314451 >> 172.21.15.42:0/3897314451 conn(0x7f832406bdf0 msgr2=0x7f832414fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:36.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.825+0000 7f832c000640 1 -- 172.21.15.42:0/3897314451 shutdown_connections 2024-05-28T13:58:36.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.825+0000 7f832c000640 1 -- 172.21.15.42:0/3897314451 wait complete. 2024-05-28T13:58:36.827 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:36.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.981+0000 7f2415800640 1 Processor -- start 2024-05-28T13:58:36.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.981+0000 7f2415800640 1 -- start start 2024-05-28T13:58:36.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f2415800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f241015eb00 0x7f24100cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f2415800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f24100cfe00 0x7f241006b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f2415800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241006bf30 0x7f24100b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f2415800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f241006c3a0 con 0x7f241015eb00 2024-05-28T13:58:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f2415800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f241015eee0 con 0x7f24100cfe00 2024-05-28T13:58:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f2415800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f24100ceb20 con 0x7f241006bf30 2024-05-28T13:58:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f240ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f241015eb00 0x7f24100cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f240ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f241015eb00 0x7f24100cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38942/0 (socket says 172.21.15.42:38942) 2024-05-28T13:58:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f240ea00640 1 -- 172.21.15.42:0/1496123768 learned_addr learned my addr 172.21.15.42:0/1496123768 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f240f400640 1 --2- 172.21.15.42:0/1496123768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241006bf30 0x7f24100b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f2407e00640 1 --2- 172.21.15.42:0/1496123768 >> v2:172.21.15.63:3300/0 conn(0x7f24100cfe00 0x7f241006b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f240ea00640 1 -- 172.21.15.42:0/1496123768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241006bf30 msgr2=0x7f24100b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f240ea00640 1 --2- 172.21.15.42:0/1496123768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241006bf30 0x7f24100b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f240ea00640 1 -- 172.21.15.42:0/1496123768 >> v2:172.21.15.63:3300/0 conn(0x7f24100cfe00 msgr2=0x7f241006b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f240ea00640 1 --2- 172.21.15.42:0/1496123768 >> v2:172.21.15.63:3300/0 conn(0x7f24100cfe00 0x7f241006b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f240ea00640 1 -- 172.21.15.42:0/1496123768 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24100cee00 con 0x7f241015eb00 2024-05-28T13:58:36.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f2407e00640 1 --2- 172.21.15.42:0/1496123768 >> v2:172.21.15.63:3300/0 conn(0x7f24100cfe00 0x7f241006b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:36.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.985+0000 7f240f400640 1 --2- 172.21.15.42:0/1496123768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241006bf30 0x7f24100b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:36.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f240ea00640 1 --2- 172.21.15.42:0/1496123768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f241015eb00 0x7f24100cf8c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f24100b7590 tx=0x7f240001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec23d60b98e7862e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:36.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f240e000640 1 -- 172.21.15.42:0/1496123768 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000b0070 con 0x7f241015eb00 2024-05-28T13:58:36.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f240e000640 1 -- 172.21.15.42:0/1496123768 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24000ab600 con 0x7f241015eb00 2024-05-28T13:58:36.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f240e000640 1 -- 172.21.15.42:0/1496123768 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000ab9d0 con 0x7f241015eb00 2024-05-28T13:58:36.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f2415800640 1 -- 172.21.15.42:0/1496123768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f241015eb00 msgr2=0x7f24100cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f2415800640 1 --2- 172.21.15.42:0/1496123768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f241015eb00 0x7f24100cf8c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f24100b7590 tx=0x7f240001c520 comp rx=0 tx=0).stop 2024-05-28T13:58:36.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f2415800640 1 -- 172.21.15.42:0/1496123768 shutdown_connections 2024-05-28T13:58:36.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f2415800640 1 --2- 172.21.15.42:0/1496123768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241006bf30 0x7f24100b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f2415800640 1 --2- 172.21.15.42:0/1496123768 >> v2:172.21.15.63:3300/0 conn(0x7f24100cfe00 0x7f241006b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f2415800640 1 --2- 172.21.15.42:0/1496123768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f241015eb00 0x7f24100cf8c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f2415800640 1 -- 172.21.15.42:0/1496123768 >> 172.21.15.42:0/1496123768 conn(0x7f24100b68f0 msgr2=0x7f24100b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:36.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f2415800640 1 -- 172.21.15.42:0/1496123768 shutdown_connections 2024-05-28T13:58:36.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f2415800640 1 -- 172.21.15.42:0/1496123768 wait complete. 2024-05-28T13:58:36.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f2415800640 1 Processor -- start 2024-05-28T13:58:36.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.989+0000 7f2415800640 1 -- start start 2024-05-28T13:58:36.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f2415800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f241006bf30 0x7f241014b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f2415800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24100cfe00 0x7f241014c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f2415800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241015eb00 0x7f241014ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f2415800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f24100b66d0 con 0x7f241006bf30 2024-05-28T13:58:36.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f2415800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24100ce890 con 0x7f24100cfe00 2024-05-28T13:58:36.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f2415800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f24100b43d0 con 0x7f241015eb00 2024-05-28T13:58:36.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f240ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f241006bf30 0x7f241014b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f240ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f241006bf30 0x7f241014b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38950/0 (socket says 172.21.15.42:38950) 2024-05-28T13:58:36.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f240ea00640 1 -- 172.21.15.42:0/1066163738 learned_addr learned my addr 172.21.15.42:0/1066163738 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:36.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f2407e00640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24100cfe00 0x7f241014c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f240f400640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241015eb00 0x7f241014ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f240ea00640 1 -- 172.21.15.42:0/1066163738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241015eb00 msgr2=0x7f241014ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f240ea00640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241015eb00 0x7f241014ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f240ea00640 1 -- 172.21.15.42:0/1066163738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24100cfe00 msgr2=0x7f241014c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:36.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f240ea00640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24100cfe00 0x7f241014c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:36.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f240ea00640 1 -- 172.21.15.42:0/1066163738 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241014d480 con 0x7f241006bf30 2024-05-28T13:58:36.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f240f400640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241015eb00 0x7f241014ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:36.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f240ea00640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f241006bf30 0x7f241014b5a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f240001cdc0 tx=0x7f240001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:36.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f2406a00640 1 -- 172.21.15.42:0/1066163738 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000ba020 con 0x7f241006bf30 2024-05-28T13:58:36.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f2406a00640 1 -- 172.21.15.42:0/1066163738 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24000b6620 con 0x7f241006bf30 2024-05-28T13:58:36.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f2415800640 1 -- 172.21.15.42:0/1066163738 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24100ccbe0 con 0x7f241006bf30 2024-05-28T13:58:36.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f2406a00640 1 -- 172.21.15.42:0/1066163738 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000b0040 con 0x7f241006bf30 2024-05-28T13:58:36.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f2415800640 1 -- 172.21.15.42:0/1066163738 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24100cd1a0 con 0x7f241006bf30 2024-05-28T13:58:36.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f2415800640 1 -- 172.21.15.42:0/1066163738 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2410068a90 con 0x7f241006bf30 2024-05-28T13:58:36.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.993+0000 7f2406a00640 1 -- 172.21.15.42:0/1066163738 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f24000bd020 con 0x7f241006bf30 2024-05-28T13:58:36.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.997+0000 7f2406a00640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23ec068380 0x7f23ec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:36.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.997+0000 7f2406a00640 1 -- 172.21.15.42:0/1066163738 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2400120dc0 con 0x7f241006bf30 2024-05-28T13:58:36.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.997+0000 7f2407e00640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23ec068380 0x7f23ec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:36.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:36.997+0000 7f2407e00640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23ec068380 0x7f23ec06a840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f241015e4d0 tx=0x7f23f8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:36.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.001+0000 7f2406a00640 1 -- 172.21.15.42:0/1066163738 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f24000c4030 con 0x7f241006bf30 2024-05-28T13:58:37.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.169+0000 7f2415800640 1 -- 172.21.15.42:0/1066163738 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2410157a30 con 0x7f241006bf30 2024-05-28T13:58:37.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.173+0000 7f2406a00640 1 -- 172.21.15.42:0/1066163738 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16545 (secure 0 0 0) 0x7f24000f7030 con 0x7f241006bf30 2024-05-28T13:58:37.158 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:37.158 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:24.659233+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:37.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.173+0000 7f2415800640 1 -- 172.21.15.42:0/1066163738 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23ec068380 msgr2=0x7f23ec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:37.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.173+0000 7f2415800640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23ec068380 0x7f23ec06a840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f241015e4d0 tx=0x7f23f8008040 comp rx=0 tx=0).stop 2024-05-28T13:58:37.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.173+0000 7f2415800640 1 -- 172.21.15.42:0/1066163738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f241006bf30 msgr2=0x7f241014b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:37.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.173+0000 7f2415800640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f241006bf30 0x7f241014b5a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f240001cdc0 tx=0x7f240001ad60 comp rx=0 tx=0).stop 2024-05-28T13:58:37.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.173+0000 7f2415800640 1 -- 172.21.15.42:0/1066163738 shutdown_connections 2024-05-28T13:58:37.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.173+0000 7f2415800640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23ec068380 0x7f23ec06a840 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:37.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.173+0000 7f2415800640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241015eb00 0x7f241014ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:37.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.173+0000 7f2415800640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24100cfe00 0x7f241014c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:37.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.173+0000 7f2415800640 1 --2- 172.21.15.42:0/1066163738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f241006bf30 0x7f241014b5a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:37.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.173+0000 7f2415800640 1 -- 172.21.15.42:0/1066163738 >> 172.21.15.42:0/1066163738 conn(0x7f24100b68f0 msgr2=0x7f2410157f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:37.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.173+0000 7f2415800640 1 -- 172.21.15.42:0/1066163738 shutdown_connections 2024-05-28T13:58:37.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.173+0000 7f2415800640 1 -- 172.21.15.42:0/1066163738 wait complete. 2024-05-28T13:58:37.179 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T13:58:37.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.333+0000 7fd3a8200640 1 Processor -- start 2024-05-28T13:58:37.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.333+0000 7fd3a8200640 1 -- start start 2024-05-28T13:58:37.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.337+0000 7fd3a8200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3a00d0900 0x7fd3a0162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:37.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.337+0000 7fd3a8200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd3a0162ab0 0x7fd3a015e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:37.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.337+0000 7fd3a8200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3a00d0de0 con 0x7fd3a00ce860 2024-05-28T13:58:37.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.337+0000 7fd3a8200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd3a00d19f0 con 0x7fd3a0162ab0 2024-05-28T13:58:37.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.337+0000 7fd3a8200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a00d10c0 con 0x7fd3a00d0900 2024-05-28T13:58:37.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.337+0000 7fd3a6200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd3a0162ab0 0x7fd3a015e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:37.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.337+0000 7fd3a6200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd3a0162ab0 0x7fd3a015e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38962/0 (socket says 172.21.15.42:38962) 2024-05-28T13:58:37.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.337+0000 7fd3a6200640 1 -- 172.21.15.42:0/2125998485 learned_addr learned my addr 172.21.15.42:0/2125998485 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:37.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.337+0000 7fd3a4e00640 1 --2- 172.21.15.42:0/2125998485 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3a00d0900 0x7fd3a0162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:37.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.337+0000 7fd3a6200640 1 -- 172.21.15.42:0/2125998485 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3a00d0900 msgr2=0x7fd3a0162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:37.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.337+0000 7fd3a6200640 1 --2- 172.21.15.42:0/2125998485 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3a00d0900 0x7fd3a0162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:37.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.337+0000 7fd3a6200640 1 -- 172.21.15.42:0/2125998485 >> v1:172.21.15.63:6789/0 conn(0x7fd3a00ce860 legacy=0x7fd3a006d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:37.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.337+0000 7fd3a6200640 1 -- 172.21.15.42:0/2125998485 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a00d13a0 con 0x7fd3a0162ab0 2024-05-28T13:58:37.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.341+0000 7fd3a6200640 1 --2- 172.21.15.42:0/2125998485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3a0162ab0 0x7fd3a015e200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd398009e50 tx=0x7fd39801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da99ef55f60fa23 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:37.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.341+0000 7fd39fe00640 1 -- 172.21.15.42:0/2125998485 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3980a5670 con 0x7fd3a0162ab0 2024-05-28T13:58:37.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.341+0000 7fd39fe00640 1 -- 172.21.15.42:0/2125998485 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3980a5810 con 0x7fd3a0162ab0 2024-05-28T13:58:37.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.341+0000 7fd39fe00640 1 -- 172.21.15.42:0/2125998485 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3980a5be0 con 0x7fd3a0162ab0 2024-05-28T13:58:37.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.341+0000 7fd3a8200640 1 -- 172.21.15.42:0/2125998485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3a0162ab0 msgr2=0x7fd3a015e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:37.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.341+0000 7fd3a8200640 1 --2- 172.21.15.42:0/2125998485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3a0162ab0 0x7fd3a015e200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd398009e50 tx=0x7fd39801c9a0 comp rx=0 tx=0).stop 2024-05-28T13:58:37.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.341+0000 7fd3a8200640 1 -- 172.21.15.42:0/2125998485 shutdown_connections 2024-05-28T13:58:37.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.341+0000 7fd3a8200640 1 --2- 172.21.15.42:0/2125998485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3a0162ab0 0x7fd3a015e200 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:37.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.341+0000 7fd3a8200640 1 --2- 172.21.15.42:0/2125998485 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3a00d0900 0x7fd3a0162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:37.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.341+0000 7fd3a8200640 1 -- 172.21.15.42:0/2125998485 >> 172.21.15.42:0/2125998485 conn(0x7fd3a00b4080 msgr2=0x7fd3a00b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:37.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.341+0000 7fd3a8200640 1 -- 172.21.15.42:0/2125998485 shutdown_connections 2024-05-28T13:58:37.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.341+0000 7fd3a8200640 1 -- 172.21.15.42:0/2125998485 wait complete. 2024-05-28T13:58:37.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.341+0000 7fd3a8200640 1 Processor -- start 2024-05-28T13:58:37.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.341+0000 7fd3a8200640 1 -- start start 2024-05-28T13:58:37.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a8200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3a00ce860 0x7fd3a015ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:37.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a8200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a00d0900 0x7fd3a0151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:37.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a8200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3a0162ab0 0x7fd3a00c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:37.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a8200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a00ccd60 con 0x7fd3a00ce860 2024-05-28T13:58:37.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a8200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a00ccbe0 con 0x7fd3a00d0900 2024-05-28T13:58:37.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a8200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a00ccee0 con 0x7fd3a0162ab0 2024-05-28T13:58:37.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3a00ce860 0x7fd3a015ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:37.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3a00ce860 0x7fd3a015ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38970/0 (socket says 172.21.15.42:38970) 2024-05-28T13:58:37.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a5800640 1 -- 172.21.15.42:0/1110541623 learned_addr learned my addr 172.21.15.42:0/1110541623 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:37.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a4e00640 1 --2- 172.21.15.42:0/1110541623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a00d0900 0x7fd3a0151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:37.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a6200640 1 --2- 172.21.15.42:0/1110541623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3a0162ab0 0x7fd3a00c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:37.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a5800640 1 -- 172.21.15.42:0/1110541623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3a0162ab0 msgr2=0x7fd3a00c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:37.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a5800640 1 --2- 172.21.15.42:0/1110541623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3a0162ab0 0x7fd3a00c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:37.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a5800640 1 -- 172.21.15.42:0/1110541623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a00d0900 msgr2=0x7fd3a0151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:37.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a5800640 1 --2- 172.21.15.42:0/1110541623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a00d0900 0x7fd3a0151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:37.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a5800640 1 -- 172.21.15.42:0/1110541623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a0151670 con 0x7fd3a00ce860 2024-05-28T13:58:37.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a5800640 1 --2- 172.21.15.42:0/1110541623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3a00ce860 0x7fd3a015ccb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd39400c450 tx=0x7fd39400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:37.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd39d600640 1 -- 172.21.15.42:0/1110541623 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394016020 con 0x7fd3a00ce860 2024-05-28T13:58:37.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd39d600640 1 -- 172.21.15.42:0/1110541623 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3940052d0 con 0x7fd3a00ce860 2024-05-28T13:58:37.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd39d600640 1 -- 172.21.15.42:0/1110541623 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394007260 con 0x7fd3a00ce860 2024-05-28T13:58:37.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a8200640 1 -- 172.21.15.42:0/1110541623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a00c7aa0 con 0x7fd3a00ce860 2024-05-28T13:58:37.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a8200640 1 -- 172.21.15.42:0/1110541623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a015c4b0 con 0x7fd3a00ce860 2024-05-28T13:58:37.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd3a8200640 1 -- 172.21.15.42:0/1110541623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a0068a90 con 0x7fd3a00ce860 2024-05-28T13:58:37.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.345+0000 7fd39d600640 1 -- 172.21.15.42:0/1110541623 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd394007400 con 0x7fd3a00ce860 2024-05-28T13:58:37.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.349+0000 7fd39d600640 1 --2- 172.21.15.42:0/1110541623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd380068380 0x7fd38006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:37.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.349+0000 7fd39d600640 1 -- 172.21.15.42:0/1110541623 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd3940861a0 con 0x7fd3a00ce860 2024-05-28T13:58:37.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.349+0000 7fd3a4e00640 1 --2- 172.21.15.42:0/1110541623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd380068380 0x7fd38006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:37.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.349+0000 7fd3a4e00640 1 --2- 172.21.15.42:0/1110541623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd380068380 0x7fd38006a840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd3a01fb2e0 tx=0x7fd38c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:37.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.353+0000 7fd39d600640 1 -- 172.21.15.42:0/1110541623 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd394020020 con 0x7fd3a00ce860 2024-05-28T13:58:37.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.557+0000 7fd3a8200640 1 -- 172.21.15.42:0/1110541623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3a00cfe00 con 0x7fd3a00ce860 2024-05-28T13:58:37.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.557+0000 7fd39d600640 1 -- 172.21.15.42:0/1110541623 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7fd394005480 con 0x7fd3a00ce860 2024-05-28T13:58:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:37.542 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1409117123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1409117123},{"type":"v1","addr":"172.21.15.42:6837","nonce":1409117123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/3753517701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3753517701},{"type":"v1","addr":"172.21.15.42:6836","nonce":3753517701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/882866492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":882866492},{"type":"v1","addr":"172.21.15.63:6834","nonce":882866492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T13:58:23.075354+0000","modified":"2024-05-28T13:58:26.084895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/1552779700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1552779700},{"type":"v1","addr":"172.21.15.63:6835","nonce":1552779700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-05-28T13:58:37.542 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-05-28T13:58:37.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.557+0000 7fd3a8200640 1 -- 172.21.15.42:0/1110541623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd380068380 msgr2=0x7fd38006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:37.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.557+0000 7fd3a8200640 1 --2- 172.21.15.42:0/1110541623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd380068380 0x7fd38006a840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd3a01fb2e0 tx=0x7fd38c008040 comp rx=0 tx=0).stop 2024-05-28T13:58:37.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.557+0000 7fd3a8200640 1 -- 172.21.15.42:0/1110541623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3a00ce860 msgr2=0x7fd3a015ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:37.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.557+0000 7fd3a8200640 1 --2- 172.21.15.42:0/1110541623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3a00ce860 0x7fd3a015ccb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd39400c450 tx=0x7fd39400c910 comp rx=0 tx=0).stop 2024-05-28T13:58:37.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.557+0000 7fd3a8200640 1 -- 172.21.15.42:0/1110541623 shutdown_connections 2024-05-28T13:58:37.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.557+0000 7fd3a8200640 1 --2- 172.21.15.42:0/1110541623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd380068380 0x7fd38006a840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:37.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.557+0000 7fd3a8200640 1 --2- 172.21.15.42:0/1110541623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3a0162ab0 0x7fd3a00c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:37.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.557+0000 7fd3a8200640 1 --2- 172.21.15.42:0/1110541623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a00d0900 0x7fd3a0151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:37.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.557+0000 7fd3a8200640 1 --2- 172.21.15.42:0/1110541623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3a00ce860 0x7fd3a015ccb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:37.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.557+0000 7fd3a8200640 1 -- 172.21.15.42:0/1110541623 >> 172.21.15.42:0/1110541623 conn(0x7fd3a00b4080 msgr2=0x7fd3a0161080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:37.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.557+0000 7fd3a8200640 1 -- 172.21.15.42:0/1110541623 shutdown_connections 2024-05-28T13:58:37.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:37.557+0000 7fd3a8200640 1 -- 172.21.15.42:0/1110541623 wait complete. 2024-05-28T13:58:37.968 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-28T13:58:23.075354+0000', 'modified': '2024-05-28T13:58:26.084895+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4306': {'gid': 4306, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6835/1552779700', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6833', 'nonce': 1552779700}, {'type': 'v1', 'addr': '172.21.15.63:6835', 'nonce': 1552779700}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4306, 'qdb_cluster': [4306]} 2024-05-28T13:58:37.968 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-28T13:58:37.969 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-28T13:58:38.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e45c00640 1 Processor -- start 2024-05-28T13:58:38.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e45c00640 1 -- start start 2024-05-28T13:58:38.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e45c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3e400b33f0 0x7f3e400b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e45c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e400b1ac0 con 0x7f3e4009b160 2024-05-28T13:58:38.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e45c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e400d0c80 con 0x7f3e4015f9c0 2024-05-28T13:58:38.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e45c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3e400d3260 con 0x7f3e400b33f0 2024-05-28T13:58:38.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e37e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f3e4009b160 0x7f3e400b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:35148/0 (socket says 172.21.15.42:35148) 2024-05-28T13:58:38.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e37e00640 1 -- 172.21.15.42:0/370578720 learned_addr learned my addr 172.21.15.42:0/370578720 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:38.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e45200640 1 --2- 172.21.15.42:0/370578720 >> v2:172.21.15.63:3300/0 conn(0x7f3e400b33f0 0x7f3e400b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 -- 172.21.15.42:0/370578720 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2524237408 0 0) 0x7f3e400b1ac0 con 0x7f3e4009b160 2024-05-28T13:58:38.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 -- 172.21.15.42:0/370578720 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e24003530 con 0x7f3e4009b160 2024-05-28T13:58:38.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 -- 172.21.15.42:0/370578720 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1587665310 0 0) 0x7f3e24003530 con 0x7f3e4009b160 2024-05-28T13:58:38.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 -- 172.21.15.42:0/370578720 >> v2:172.21.15.63:3300/0 conn(0x7f3e400b33f0 msgr2=0x7f3e400b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:38.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 --2- 172.21.15.42:0/370578720 >> v2:172.21.15.63:3300/0 conn(0x7f3e400b33f0 0x7f3e400b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 -- 172.21.15.42:0/370578720 >> v1:172.21.15.63:6789/0 conn(0x7f3e4015f9c0 legacy=0x7f3e4009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 -- 172.21.15.42:0/370578720 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e400d0f60 con 0x7f3e4009b160 2024-05-28T13:58:38.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 -- 172.21.15.42:0/370578720 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f3e28001680 con 0x7f3e4009b160 2024-05-28T13:58:38.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 -- 172.21.15.42:0/370578720 >> v1:172.21.15.42:6789/0 conn(0x7f3e4009b160 legacy=0x7f3e400b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 --2- 172.21.15.42:0/370578720 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e240039c0 0x7f3e24003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 --2- 172.21.15.42:0/370578720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e24004810 0x7f3e24004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 --2- 172.21.15.42:0/370578720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e240051d0 0x7f3e2400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 -- 172.21.15.42:0/370578720 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3e400d3260 con 0x7f3e240051d0 2024-05-28T13:58:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 -- 172.21.15.42:0/370578720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2400c070 con 0x7f3e24004810 2024-05-28T13:58:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3f400640 1 -- 172.21.15.42:0/370578720 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3e2400c270 con 0x7f3e240039c0 2024-05-28T13:58:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e45200640 1 --2- 172.21.15.42:0/370578720 >> v2:172.21.15.63:3300/0 conn(0x7f3e400b33f0 0x7f3e400b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e45200640 1 --2- 172.21.15.42:0/370578720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e240051d0 0x7f3e2400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e37e00640 1 --2- 172.21.15.42:0/370578720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e24004810 0x7f3e24004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.129+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/370578720 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e240039c0 0x7f3e24003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e45200640 1 -- 172.21.15.42:0/370578720 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e240039c0 msgr2=0x7f3e24003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:38.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e45200640 1 --2- 172.21.15.42:0/370578720 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e240039c0 0x7f3e24003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e45200640 1 -- 172.21.15.42:0/370578720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e24004810 msgr2=0x7f3e24004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:38.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e45200640 1 --2- 172.21.15.42:0/370578720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e24004810 0x7f3e24004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e45200640 1 -- 172.21.15.42:0/370578720 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e24003530 con 0x7f3e240051d0 2024-05-28T13:58:38.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e45200640 1 -- 172.21.15.42:0/370578720 reap_dead start 2024-05-28T13:58:38.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e45200640 1 --2- 172.21.15.42:0/370578720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e240051d0 0x7f3e2400b9d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800e350 tx=0x7f3e3801ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6543df98010b1a0e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:38.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e3f400640 1 -- 172.21.15.42:0/370578720 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38009dc0 con 0x7f3e240051d0 2024-05-28T13:58:38.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e3f400640 1 -- 172.21.15.42:0/370578720 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e380039a0 con 0x7f3e240051d0 2024-05-28T13:58:38.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e3f400640 1 -- 172.21.15.42:0/370578720 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38003d90 con 0x7f3e240051d0 2024-05-28T13:58:38.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e45c00640 1 -- 172.21.15.42:0/370578720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e240051d0 msgr2=0x7f3e2400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e45c00640 1 --2- 172.21.15.42:0/370578720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e240051d0 0x7f3e2400b9d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800e350 tx=0x7f3e3801ce60 comp rx=0 tx=0).stop 2024-05-28T13:58:38.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e45c00640 1 -- 172.21.15.42:0/370578720 shutdown_connections 2024-05-28T13:58:38.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e45c00640 1 --2- 172.21.15.42:0/370578720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e240051d0 0x7f3e2400b9d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e45c00640 1 -- 172.21.15.42:0/370578720 >> 172.21.15.42:0/370578720 conn(0x7f3e400d2360 msgr2=0x7f3e400d2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:38.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e45c00640 1 -- 172.21.15.42:0/370578720 shutdown_connections 2024-05-28T13:58:38.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.133+0000 7f3e45c00640 1 -- 172.21.15.42:0/370578720 wait complete. 2024-05-28T13:58:38.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45c00640 1 Processor -- start 2024-05-28T13:58:38.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45c00640 1 -- start start 2024-05-28T13:58:38.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e240051d0 0x7f3e4015c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e400ca4b0 0x7f3e400ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e4015d420 0x7f3e4015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3e400b1df0 con 0x7f3e4015d420 2024-05-28T13:58:38.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3e400d3090 con 0x7f3e240051d0 2024-05-28T13:58:38.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3e4015fda0 con 0x7f3e400ca4b0 2024-05-28T13:58:38.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e4015d420 0x7f3e4015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e4015d420 0x7f3e4015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56980/0 (socket says 172.21.15.42:56980) 2024-05-28T13:58:38.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45200640 1 -- 172.21.15.42:0/647152392 learned_addr learned my addr 172.21.15.42:0/647152392 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:38.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/647152392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e240051d0 0x7f3e4015c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e37e00640 1 --2- 172.21.15.42:0/647152392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e400ca4b0 0x7f3e400ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45200640 1 -- 172.21.15.42:0/647152392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e400ca4b0 msgr2=0x7f3e400ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45200640 1 --2- 172.21.15.42:0/647152392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e400ca4b0 0x7f3e400ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45200640 1 -- 172.21.15.42:0/647152392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e240051d0 msgr2=0x7f3e4015c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45200640 1 --2- 172.21.15.42:0/647152392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e240051d0 0x7f3e4015c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45200640 1 -- 172.21.15.42:0/647152392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e400b5170 con 0x7f3e4015d420 2024-05-28T13:58:38.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45200640 1 --2- 172.21.15.42:0/647152392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e4015d420 0x7f3e4015d860 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3e3801cea0 tx=0x7f3e38009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:38.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e3cc00640 1 -- 172.21.15.42:0/647152392 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380bb020 con 0x7f3e4015d420 2024-05-28T13:58:38.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e3cc00640 1 -- 172.21.15.42:0/647152392 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e38004760 con 0x7f3e4015d420 2024-05-28T13:58:38.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e3cc00640 1 -- 172.21.15.42:0/647152392 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38009dc0 con 0x7f3e4015d420 2024-05-28T13:58:38.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.137+0000 7f3e45c00640 1 -- 172.21.15.42:0/647152392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e400b5450 con 0x7f3e4015d420 2024-05-28T13:58:38.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.141+0000 7f3e45c00640 1 -- 172.21.15.42:0/647152392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e400b5a10 con 0x7f3e4015d420 2024-05-28T13:58:38.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.141+0000 7f3e3cc00640 1 -- 172.21.15.42:0/647152392 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3e380be070 con 0x7f3e4015d420 2024-05-28T13:58:38.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.141+0000 7f3e3cc00640 1 --2- 172.21.15.42:0/647152392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3e1c068380 0x7f3e1c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.141+0000 7f3e3cc00640 1 -- 172.21.15.42:0/647152392 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3e38121390 con 0x7f3e4015d420 2024-05-28T13:58:38.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.141+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/647152392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3e1c068380 0x7f3e1c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 Processor -- start 2024-05-28T13:58:38.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 -- start start 2024-05-28T13:58:38.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/647152392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3e1c068380 0x7f3e1c06a840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3e30000c10 tx=0x7f3e30009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:38.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161120 0x55caa4161500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4161a40 0x55caa4114110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4114650 0x55caa4114a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55caa3f65420 con 0x55caa4161120 2024-05-28T13:58:38.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55caa3effbf0 con 0x55caa4114650 2024-05-28T13:58:38.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55caa3f613e0 con 0x55caa4161a40 2024-05-28T13:58:38.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e3fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161120 0x55caa4161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e3fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161120 0x55caa4161500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56988/0 (socket says 172.21.15.42:56988) 2024-05-28T13:58:38.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e3fe00640 1 -- 172.21.15.42:0/1503960727 learned_addr learned my addr 172.21.15.42:0/1503960727 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:38.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e3fe00640 1 -- 172.21.15.42:0/1503960727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4161a40 msgr2=0x55caa4114110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:38.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/1503960727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4161a40 0x55caa4114110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e3fe00640 1 -- 172.21.15.42:0/1503960727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4114650 msgr2=0x55caa4114a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:38.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e45200640 1 --2- 172.21.15.42:0/1503960727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4114650 0x55caa4114a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/1503960727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4114650 0x55caa4114a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e3fe00640 1 -- 172.21.15.42:0/1503960727 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caa400d450 con 0x55caa4161120 2024-05-28T13:58:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/1503960727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161120 0x55caa4161500 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e30012c30 tx=0x7f3e30015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb89bbd1911ce1a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e34c00640 1 -- 172.21.15.42:0/1503960727 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30028070 con 0x55caa4161120 2024-05-28T13:58:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e34c00640 1 -- 172.21.15.42:0/1503960727 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e30007530 con 0x55caa4161120 2024-05-28T13:58:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e34c00640 1 -- 172.21.15.42:0/1503960727 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300069e0 con 0x55caa4161120 2024-05-28T13:58:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/1503960727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161120 msgr2=0x55caa4161500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/1503960727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161120 0x55caa4161500 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e30012c30 tx=0x7f3e30015000 comp rx=0 tx=0).stop 2024-05-28T13:58:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/1503960727 shutdown_connections 2024-05-28T13:58:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/1503960727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4114650 0x55caa4114a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/1503960727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4161a40 0x55caa4114110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/1503960727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161120 0x55caa4161500 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/1503960727 >> 172.21.15.42:0/1503960727 conn(0x55caa4141ec0 msgr2=0x55caa41442b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/1503960727 shutdown_connections 2024-05-28T13:58:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/1503960727 wait complete. 2024-05-28T13:58:38.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.145+0000 7f3e4a6f4000 1 -- start start 2024-05-28T13:58:38.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 1 Processor -- start 2024-05-28T13:58:38.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 0x55caa4119b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161120 0x55caa411a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161a40 0x55caa411a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55caa3f05ec0 con 0x55caa4161a40 2024-05-28T13:58:38.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55caa3fe6120 con 0x55caa4161120 2024-05-28T13:58:38.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55caa3f70350 con 0x55caa4114650 2024-05-28T13:58:38.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 10 client.0 _ll_register_callbacks cb 0x55caa3f424d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T13:58:38.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e45200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161a40 0x55caa411a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e45200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161a40 0x55caa411a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56992/0 (socket says 172.21.15.42:56992) 2024-05-28T13:58:38.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e45200640 1 -- 172.21.15.42:0/3063805133 learned_addr learned my addr 172.21.15.42:0/3063805133 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:38.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 0x55caa4119b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e37e00640 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161120 0x55caa411a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e45200640 1 -- 172.21.15.42:0/3063805133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 msgr2=0x55caa4119b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:38.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e45200640 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 0x55caa4119b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e45200640 1 -- 172.21.15.42:0/3063805133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161120 msgr2=0x55caa411a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:38.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e45200640 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161120 0x55caa411a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e45200640 1 -- 172.21.15.42:0/3063805133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caa3faaf80 con 0x55caa4161a40 2024-05-28T13:58:38.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 0x55caa4119b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:38.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e37e00640 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161120 0x55caa411a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:38.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e45200640 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161a40 0x55caa411a580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e380f11f0 tx=0x7f3e380f0c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1d7ca2354fada4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:38.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e2d600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T13:58:38.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e2d600640 1 -- 172.21.15.42:0/3063805133 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380f50d0 con 0x55caa4161a40 2024-05-28T13:58:38.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e2d600640 1 -- 172.21.15.42:0/3063805133 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3e380f5270 con 0x55caa4161a40 2024-05-28T13:58:38.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/3063805133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e30007530 con 0x55caa4161a40 2024-05-28T13:58:38.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e2d600640 1 -- 172.21.15.42:0/3063805133 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f3e3811f450 con 0x55caa4161a40 2024-05-28T13:58:38.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 10 client.4608 fetch_fsmap learned FSMap version 8 2024-05-28T13:58:38.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/3063805133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55caa3f3ef00 con 0x55caa4161a40 2024-05-28T13:58:38.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e2d600640 1 -- 172.21.15.42:0/3063805133 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e30007530 con 0x55caa4161a40 2024-05-28T13:58:38.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e2d600640 1 -- 172.21.15.42:0/3063805133 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7f3e380f9900 con 0x55caa4161a40 2024-05-28T13:58:38.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 10 client.4608 fetch_fsmap finished waiting for FSMap version 8 2024-05-28T13:58:38.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 10 client.4608 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-28T13:58:38.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 20 client.4608 populate_metadata read hostname 'smithi042' 2024-05-28T13:58:38.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.63:6833/1552779700,v1:172.21.15.63:6835/1552779700] conn(0x55caa41703c0 0x55caa41727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 4 client.4608 mds_command: new command op to 4306 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-28T13:58:38.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/3063805133 --> [v2:172.21.15.63:6833/1552779700,v1:172.21.15.63:6835/1552779700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caa3efebc0 con 0x55caa41703c0 2024-05-28T13:58:38.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.63:6833/1552779700,v1:172.21.15.63:6835/1552779700] conn(0x55caa41703c0 0x55caa41727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.63:6833/1552779700,v1:172.21.15.63:6835/1552779700] conn(0x55caa41703c0 0x55caa41727b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a90ab50d57c65ba server_cookie=1b48571d677da4aa in_seq=0 out_seq=0 2024-05-28T13:58:38.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.149+0000 7f3e2d600640 10 client.4608 ms_handle_connect on v2:172.21.15.63:6833/1552779700 2024-05-28T13:58:38.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e2d600640 1 -- 172.21.15.42:0/3063805133 <== mds.0 v2:172.21.15.63:6833/1552779700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55caa3efebc0 con 0x55caa41703c0 2024-05-28T13:58:38.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e2d600640 10 client.4608 handle_command_reply: tid=0 2024-05-28T13:58:38.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e4a6f4000 1 client.4608 shutdown 2024-05-28T13:58:38.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/3063805133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161a40 msgr2=0x55caa411a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161a40 0x55caa411a580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e380f11f0 tx=0x7f3e380f0c80 comp rx=0 tx=0).stop 2024-05-28T13:58:38.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/3063805133 shutdown_connections 2024-05-28T13:58:38.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161a40 0x55caa411a580 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161120 0x55caa411a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.63:6833/1552779700,v1:172.21.15.63:6835/1552779700] conn(0x55caa41703c0 0x55caa41727b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/3063805133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 0x55caa4119b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/3063805133 >> 172.21.15.42:0/3063805133 conn(0x55caa4141ec0 msgr2=0x55caa41442b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:38.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e2d600640 0 client.4608 ms_handle_reset on v2:172.21.15.63:6833/1552779700 2024-05-28T13:58:38.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/3063805133 shutdown_connections 2024-05-28T13:58:38.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/3063805133 wait complete. 2024-05-28T13:58:38.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.153+0000 7f3e4a6f4000 20 client.4608 trim_cache size 0 max 16384 2024-05-28T13:58:38.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.169+0000 7f3e4a6f4000 1 Processor -- start 2024-05-28T13:58:38.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.169+0000 7f3e4a6f4000 1 -- start start 2024-05-28T13:58:38.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 0x55caa4113810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161120 0x55caa4113d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161a40 0x55caa41727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3e400ca1a0 con 0x55caa4161a40 2024-05-28T13:58:38.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3e400c9500 con 0x55caa4161120 2024-05-28T13:58:38.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3e400c9b50 con 0x55caa4114650 2024-05-28T13:58:38.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e45200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161a40 0x55caa41727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e45200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161a40 0x55caa41727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57004/0 (socket says 172.21.15.42:57004) 2024-05-28T13:58:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e45200640 1 -- 172.21.15.42:0/851360585 learned_addr learned my addr 172.21.15.42:0/851360585 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e37e00640 1 --2- 172.21.15.42:0/851360585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161120 0x55caa4113d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/851360585 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 0x55caa4113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e45200640 1 -- 172.21.15.42:0/851360585 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 msgr2=0x55caa4113810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e45200640 1 --2- 172.21.15.42:0/851360585 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 0x55caa4113810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e45200640 1 -- 172.21.15.42:0/851360585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161120 msgr2=0x55caa4113d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e45200640 1 --2- 172.21.15.42:0/851360585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161120 0x55caa4113d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e45200640 1 -- 172.21.15.42:0/851360585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caa3eebe80 con 0x55caa4161a40 2024-05-28T13:58:38.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e45200640 1 --2- 172.21.15.42:0/851360585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161a40 0x55caa41727b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3e380f0f60 tx=0x7f3e38107000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656a91a359563263 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:38.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e2fe00640 1 -- 172.21.15.42:0/851360585 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380f9900 con 0x55caa4161a40 2024-05-28T13:58:38.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e2fe00640 1 -- 172.21.15.42:0/851360585 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e3811a070 con 0x55caa4161a40 2024-05-28T13:58:38.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e2fe00640 1 -- 172.21.15.42:0/851360585 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38105bc0 con 0x55caa4161a40 2024-05-28T13:58:38.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/851360585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161a40 msgr2=0x55caa41727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/851360585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161a40 0x55caa41727b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3e380f0f60 tx=0x7f3e38107000 comp rx=0 tx=0).stop 2024-05-28T13:58:38.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/851360585 shutdown_connections 2024-05-28T13:58:38.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/851360585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161a40 0x55caa41727b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/851360585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161120 0x55caa4113d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/851360585 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 0x55caa4113810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/851360585 >> 172.21.15.42:0/851360585 conn(0x55caa4141ec0 msgr2=0x55caa4113100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:38.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/851360585 shutdown_connections 2024-05-28T13:58:38.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/851360585 wait complete. 2024-05-28T13:58:38.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 -- start start 2024-05-28T13:58:38.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 Processor -- start 2024-05-28T13:58:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 0x55caa4175650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161120 0x55caa418c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161a40 0x55caa418c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55caa3f64730 con 0x55caa4161120 2024-05-28T13:58:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55caa3f319e0 con 0x55caa4161a40 2024-05-28T13:58:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55caa3f69a50 con 0x55caa4114650 2024-05-28T13:58:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e4a6f4000 10 client.0 _ll_register_callbacks cb 0x7f3e38004910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T13:58:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e37e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161120 0x55caa418c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e37e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161120 0x55caa418c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57008/0 (socket says 172.21.15.42:57008) 2024-05-28T13:58:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.173+0000 7f3e37e00640 1 -- 172.21.15.42:0/714315505 learned_addr learned my addr 172.21.15.42:0/714315505 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/714315505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 0x55caa4175650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e45200640 1 --2- 172.21.15.42:0/714315505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161a40 0x55caa418c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e37e00640 1 -- 172.21.15.42:0/714315505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 msgr2=0x55caa4175650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:38.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e37e00640 1 --2- 172.21.15.42:0/714315505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 0x55caa4175650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e37e00640 1 -- 172.21.15.42:0/714315505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161a40 msgr2=0x55caa418c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:38.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e37e00640 1 --2- 172.21.15.42:0/714315505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161a40 0x55caa418c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e37e00640 1 -- 172.21.15.42:0/714315505 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caa3f2db30 con 0x55caa4161120 2024-05-28T13:58:38.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e37e00640 1 --2- 172.21.15.42:0/714315505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161120 0x55caa418c340 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3e28004d40 tx=0x7f3e28007490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a241e712a3ae6c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:38.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e2d600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T13:58:38.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e2d600640 1 -- 172.21.15.42:0/714315505 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e280177c0 con 0x55caa4161120 2024-05-28T13:58:38.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/714315505 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3811a070 con 0x55caa4161120 2024-05-28T13:58:38.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e2d600640 1 -- 172.21.15.42:0/714315505 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3e28010040 con 0x55caa4161120 2024-05-28T13:58:38.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e2d600640 1 -- 172.21.15.42:0/714315505 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f3e28017960 con 0x55caa4161120 2024-05-28T13:58:38.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 10 client.4614 fetch_fsmap learned FSMap version 8 2024-05-28T13:58:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/714315505 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55caa3fe65b0 con 0x55caa4161120 2024-05-28T13:58:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e2d600640 1 -- 172.21.15.42:0/714315505 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e3811a070 con 0x55caa4161120 2024-05-28T13:58:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e2d600640 1 -- 172.21.15.42:0/714315505 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7f3e28017b00 con 0x55caa4161120 2024-05-28T13:58:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 10 client.4614 fetch_fsmap finished waiting for FSMap version 8 2024-05-28T13:58:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 10 client.4614 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-28T13:58:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 20 client.4614 populate_metadata read hostname 'smithi042' 2024-05-28T13:58:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/714315505 >> [v2:172.21.15.63:6833/1552779700,v1:172.21.15.63:6835/1552779700] conn(0x55caa418cdc0 0x55caa41cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 4 client.4614 mds_command: new command op to 4306 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-28T13:58:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/714315505 --> [v2:172.21.15.63:6833/1552779700,v1:172.21.15.63:6835/1552779700] -- command(tid 0: {"prefix": "status"}) -- 0x55caa3f72e40 con 0x55caa418cdc0 2024-05-28T13:58:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/714315505 >> [v2:172.21.15.63:6833/1552779700,v1:172.21.15.63:6835/1552779700] conn(0x55caa418cdc0 0x55caa41cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e3fe00640 1 --2- 172.21.15.42:0/714315505 >> [v2:172.21.15.63:6833/1552779700,v1:172.21.15.63:6835/1552779700] conn(0x55caa418cdc0 0x55caa41cffb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f975e49770235fc server_cookie=15e96807c5b48271 in_seq=0 out_seq=0 2024-05-28T13:58:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e2d600640 10 client.4614 ms_handle_connect on v2:172.21.15.63:6833/1552779700 2024-05-28T13:58:38.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e2d600640 1 -- 172.21.15.42:0/714315505 <== mds.0 v2:172.21.15.63:6833/1552779700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55caa3f72e40 con 0x55caa418cdc0 2024-05-28T13:58:38.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e2d600640 10 client.4614 handle_command_reply: tid=0 2024-05-28T13:58:38.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 client.4614 shutdown 2024-05-28T13:58:38.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/714315505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161120 msgr2=0x55caa418c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/714315505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161120 0x55caa418c340 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3e28004d40 tx=0x7f3e28007490 comp rx=0 tx=0).stop 2024-05-28T13:58:38.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/714315505 shutdown_connections 2024-05-28T13:58:38.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/714315505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55caa4161a40 0x55caa418c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/714315505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caa4161120 0x55caa418c340 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/714315505 >> [v2:172.21.15.63:6833/1552779700,v1:172.21.15.63:6835/1552779700] conn(0x55caa418cdc0 0x55caa41cffb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 --2- 172.21.15.42:0/714315505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55caa4114650 0x55caa4175650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/714315505 >> 172.21.15.42:0/714315505 conn(0x55caa4141ec0 msgr2=0x55caa4113810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:38.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e2d600640 0 client.4614 ms_handle_reset on v2:172.21.15.63:6833/1552779700 2024-05-28T13:58:38.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/714315505 shutdown_connections 2024-05-28T13:58:38.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 1 -- 172.21.15.42:0/714315505 wait complete. 2024-05-28T13:58:38.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.177+0000 7f3e4a6f4000 20 client.4614 trim_cache size 0 max 16384 2024-05-28T13:58:38.168 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T13:58:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T13:58:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-05-28T13:58:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4306, 2024-05-28T13:58:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-28T13:58:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-28T13:58:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-28T13:58:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 14.502225018000001, 2024-05-28T13:58:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 8, 2024-05-28T13:58:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 21, 2024-05-28T13:58:38.169 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 21, 2024-05-28T13:58:38.169 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 22.137466661000001 2024-05-28T13:58:38.169 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T13:58:38.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.181+0000 7f3e45c00640 1 -- 172.21.15.42:0/647152392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3e1c068380 msgr2=0x7f3e1c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.181+0000 7f3e45c00640 1 --2- 172.21.15.42:0/647152392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3e1c068380 0x7f3e1c06a840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3e30000c10 tx=0x7f3e30009040 comp rx=0 tx=0).stop 2024-05-28T13:58:38.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.181+0000 7f3e45c00640 1 -- 172.21.15.42:0/647152392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e4015d420 msgr2=0x7f3e4015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.181+0000 7f3e45c00640 1 --2- 172.21.15.42:0/647152392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e4015d420 0x7f3e4015d860 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3e3801cea0 tx=0x7f3e38009750 comp rx=0 tx=0).stop 2024-05-28T13:58:38.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.181+0000 7f3e45c00640 1 -- 172.21.15.42:0/647152392 shutdown_connections 2024-05-28T13:58:38.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.181+0000 7f3e45c00640 1 --2- 172.21.15.42:0/647152392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3e1c068380 0x7f3e1c06a840 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.181+0000 7f3e45c00640 1 --2- 172.21.15.42:0/647152392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e4015d420 0x7f3e4015d860 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.181+0000 7f3e45c00640 1 --2- 172.21.15.42:0/647152392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e400ca4b0 0x7f3e400ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.181+0000 7f3e45c00640 1 --2- 172.21.15.42:0/647152392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e240051d0 0x7f3e4015c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.181+0000 7f3e45c00640 1 -- 172.21.15.42:0/647152392 >> 172.21.15.42:0/647152392 conn(0x7f3e400d2360 msgr2=0x7f3e4009aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:38.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.181+0000 7f3e45c00640 1 -- 172.21.15.42:0/647152392 shutdown_connections 2024-05-28T13:58:38.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.181+0000 7f3e45c00640 1 -- 172.21.15.42:0/647152392 wait complete. 2024-05-28T13:58:38.183 INFO:teuthology.run_tasks:Running task sequential... 2024-05-28T13:58:38.194 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-28T13:58:38.194 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-28T13:58:38.195 INFO:teuthology.task.print:Enabling mgr modules 2024-05-28T13:58:38.195 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-28T13:58:38.257 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-28T13:58:38.257 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-28T13:58:38.258 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-28T13:58:38.258 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-28T13:58:38.258 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-28T13:58:38.258 INFO:tasks.ceph_fuse:client.2 config is {} 2024-05-28T13:58:38.258 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-28T13:58:38.258 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-28T13:58:38.258 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-28T13:58:38.258 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-28T13:58:38.258 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-28T13:58:38.258 INFO:tasks.ceph_fuse:client.3 config is {} 2024-05-28T13:58:38.259 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-28T13:58:38.259 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-28T13:58:38.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:38.259 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-28T13:58:38.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:38.270 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-28T13:58:38.336 INFO:teuthology.orchestra.run.smithi063.stderr:Cannot find device "ceph-brx" 2024-05-28T13:58:38.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T13:58:38.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:38.338 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-28T13:58:38.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:38.348 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-05-28T13:58:38.416 INFO:teuthology.orchestra.run.smithi042.stderr:Cannot find device "ceph-brx" 2024-05-28T13:58:38.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T13:58:38.418 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-28T13:58:38.418 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-28T13:58:38.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.613+0000 7f437c600640 1 Processor -- start 2024-05-28T13:58:38.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.613+0000 7f437c600640 1 -- start start 2024-05-28T13:58:38.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.617+0000 7f437c600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f43740d0640 0x7f43740d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.617+0000 7f437c600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437415cc90 con 0x7f43740d0080 2024-05-28T13:58:38.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.617+0000 7f437c600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437415d850 con 0x7f43740b18d0 2024-05-28T13:58:38.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.617+0000 7f437c600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f43740ccf20 con 0x7f43740d0640 2024-05-28T13:58:38.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.617+0000 7f4379c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f43740d0640 0x7f43740d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.617+0000 7f4379c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f43740d0640 0x7f43740d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57010/0 (socket says 172.21.15.42:57010) 2024-05-28T13:58:38.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.617+0000 7f437a600640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f43740d0080 0x7f437409aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:35154/0 (socket says 172.21.15.42:35154) 2024-05-28T13:58:38.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.617+0000 7f4379c00640 1 -- 172.21.15.42:0/3476372550 learned_addr learned my addr 172.21.15.42:0/3476372550 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:38.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.617+0000 7f4379c00640 1 -- 172.21.15.42:0/3476372550 >> v1:172.21.15.63:6789/0 conn(0x7f43740b18d0 legacy=0x7f43740cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.617+0000 7f4379c00640 1 -- 172.21.15.42:0/3476372550 >> v1:172.21.15.42:6789/0 conn(0x7f43740d0080 legacy=0x7f437409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.617+0000 7f4379c00640 1 -- 172.21.15.42:0/3476372550 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43740d0f60 con 0x7f43740d0640 2024-05-28T13:58:38.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.617+0000 7f4373e00640 1 -- 172.21.15.42:0/3476372550 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1816643114 0 0) 0x7f437415cc90 con 0x7f43740d0080 2024-05-28T13:58:38.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f4379c00640 1 --2- 172.21.15.42:0/3476372550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43740d0640 0x7f43740d0a20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4368009870 tx=0x7f436801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424561da29222197 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:38.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f4373e00640 1 -- 172.21.15.42:0/3476372550 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680b1070 con 0x7f43740d0640 2024-05-28T13:58:38.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f4373e00640 1 -- 172.21.15.42:0/3476372550 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43680ac620 con 0x7f43740d0640 2024-05-28T13:58:38.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f4373e00640 1 -- 172.21.15.42:0/3476372550 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680ac9f0 con 0x7f43740d0640 2024-05-28T13:58:38.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 -- 172.21.15.42:0/3476372550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43740d0640 msgr2=0x7f43740d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 --2- 172.21.15.42:0/3476372550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43740d0640 0x7f43740d0a20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4368009870 tx=0x7f436801c4d0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 -- 172.21.15.42:0/3476372550 shutdown_connections 2024-05-28T13:58:38.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 --2- 172.21.15.42:0/3476372550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43740d0640 0x7f43740d0a20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 -- 172.21.15.42:0/3476372550 >> 172.21.15.42:0/3476372550 conn(0x7f43740b3fa0 msgr2=0x7f43740b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:38.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 -- 172.21.15.42:0/3476372550 shutdown_connections 2024-05-28T13:58:38.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 -- 172.21.15.42:0/3476372550 wait complete. 2024-05-28T13:58:38.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 Processor -- start 2024-05-28T13:58:38.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 -- start start 2024-05-28T13:58:38.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43740b18d0 0x7f43740cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43740d0080 0x7f43740cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43740d0640 0x7f43740c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f43740cd710 con 0x7f43740b18d0 2024-05-28T13:58:38.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f43740cd430 con 0x7f43740d0640 2024-05-28T13:58:38.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f437c600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f43740d1330 con 0x7f43740d0080 2024-05-28T13:58:38.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f4379c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43740b18d0 0x7f43740cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f4379c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43740b18d0 0x7f43740cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57020/0 (socket says 172.21.15.42:57020) 2024-05-28T13:58:38.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.621+0000 7f4379c00640 1 -- 172.21.15.42:0/3629887262 learned_addr learned my addr 172.21.15.42:0/3629887262 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:38.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f437a600640 1 --2- 172.21.15.42:0/3629887262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43740d0640 0x7f43740c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f4379200640 1 --2- 172.21.15.42:0/3629887262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43740d0080 0x7f43740cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f4379c00640 1 -- 172.21.15.42:0/3629887262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43740d0080 msgr2=0x7f43740cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f4379c00640 1 --2- 172.21.15.42:0/3629887262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43740d0080 0x7f43740cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f4379c00640 1 -- 172.21.15.42:0/3629887262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43740d0640 msgr2=0x7f43740c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f4379c00640 1 --2- 172.21.15.42:0/3629887262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43740d0640 0x7f43740c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f4379c00640 1 -- 172.21.15.42:0/3629887262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43740cbd70 con 0x7f43740b18d0 2024-05-28T13:58:38.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f4379c00640 1 --2- 172.21.15.42:0/3629887262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43740b18d0 0x7f43740cb790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f43680af040 tx=0x7f436801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:38.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f4371600640 1 -- 172.21.15.42:0/3629887262 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680b1040 con 0x7f43740b18d0 2024-05-28T13:58:38.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f4371600640 1 -- 172.21.15.42:0/3629887262 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4368004030 con 0x7f43740b18d0 2024-05-28T13:58:38.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f4371600640 1 -- 172.21.15.42:0/3629887262 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368004400 con 0x7f43740b18d0 2024-05-28T13:58:38.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f437c600640 1 -- 172.21.15.42:0/3629887262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43740ca1e0 con 0x7f43740b18d0 2024-05-28T13:58:38.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f437c600640 1 -- 172.21.15.42:0/3629887262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43740b2530 con 0x7f43740b18d0 2024-05-28T13:58:38.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f437c600640 1 -- 172.21.15.42:0/3629887262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4374068a90 con 0x7f43740b18d0 2024-05-28T13:58:38.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f4371600640 1 -- 172.21.15.42:0/3629887262 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f43680b4070 con 0x7f43740b18d0 2024-05-28T13:58:38.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f4371600640 1 --2- 172.21.15.42:0/3629887262 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f43540683b0 0x7f435406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:38.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f4371600640 1 -- 172.21.15.42:0/3629887262 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4368120db0 con 0x7f43740b18d0 2024-05-28T13:58:38.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.625+0000 7f4379200640 1 --2- 172.21.15.42:0/3629887262 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f43540683b0 0x7f435406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.629+0000 7f4379200640 1 --2- 172.21.15.42:0/3629887262 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f43540683b0 0x7f435406a870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f43740b4ff0 tx=0x7f4360009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:38.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.633+0000 7f4371600640 1 -- 172.21.15.42:0/3629887262 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4368125070 con 0x7f43740b18d0 2024-05-28T13:58:38.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.833+0000 7f437c600640 1 -- 172.21.15.42:0/3629887262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f43740ce940 con 0x7f43740b18d0 2024-05-28T13:58:38.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.833+0000 7f4371600640 1 -- 172.21.15.42:0/3629887262 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f43680bb020 con 0x7f43740b18d0 2024-05-28T13:58:38.820 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-28T13:58:38.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.837+0000 7f437c600640 1 -- 172.21.15.42:0/3629887262 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f43540683b0 msgr2=0x7f435406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.837+0000 7f437c600640 1 --2- 172.21.15.42:0/3629887262 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f43540683b0 0x7f435406a870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f43740b4ff0 tx=0x7f4360009040 comp rx=0 tx=0).stop 2024-05-28T13:58:38.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.837+0000 7f437c600640 1 -- 172.21.15.42:0/3629887262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43740b18d0 msgr2=0x7f43740cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:38.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.837+0000 7f437c600640 1 --2- 172.21.15.42:0/3629887262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43740b18d0 0x7f43740cb790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f43680af040 tx=0x7f436801ad10 comp rx=0 tx=0).stop 2024-05-28T13:58:38.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.837+0000 7f437c600640 1 -- 172.21.15.42:0/3629887262 shutdown_connections 2024-05-28T13:58:38.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.837+0000 7f437c600640 1 --2- 172.21.15.42:0/3629887262 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f43540683b0 0x7f435406a870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.837+0000 7f437c600640 1 --2- 172.21.15.42:0/3629887262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43740d0640 0x7f43740c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.837+0000 7f437c600640 1 --2- 172.21.15.42:0/3629887262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43740d0080 0x7f43740cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.837+0000 7f437c600640 1 --2- 172.21.15.42:0/3629887262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43740b18d0 0x7f43740cb790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:38.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.837+0000 7f437c600640 1 -- 172.21.15.42:0/3629887262 >> 172.21.15.42:0/3629887262 conn(0x7f43740b3fa0 msgr2=0x7f43740b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:38.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.837+0000 7f437c600640 1 -- 172.21.15.42:0/3629887262 shutdown_connections 2024-05-28T13:58:38.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38.837+0000 7f437c600640 1 -- 172.21.15.42:0/3629887262 wait complete. 2024-05-28T13:58:38.841 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-28T13:58:38.841 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-28T13:58:38.841 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-05-28T13:58:38.841 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-28T13:58:38.841 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-28T13:58:38.841 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-28T13:58:38.842 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-28T13:58:38.842 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-28T13:58:38.842 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T13:58:38.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:38.842 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-05-28T13:58:38.851 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T13:58:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T13:58:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T13:58:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-05-28T13:58:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:38.851 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:58:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-05-28T13:58:38.851 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42428sec preferred_lft 42428sec 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 02:42:65:2b:63:73 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-28T13:58:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:38.853 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-28T13:58:38.854 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:38.854 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T13:58:38.854 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add name ceph-brx type bridge 2024-05-28T13:58:38.854 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr flush dev ceph-brx 2024-05-28T13:58:38.854 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set ceph-brx up 2024-05-28T13:58:38.854 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-28T13:58:38.854 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T13:58:38.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:39.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:39.013 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-28T13:58:39.078 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-05-28T13:58:39.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:39.081 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-05-28T13:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-05-28T13:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-28T13:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-05-28T13:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-28T13:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-05-28T13:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-28T13:58:39.156 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:39.156 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T13:58:39.156 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-28T13:58:39.156 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-28T13:58:39.156 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-28T13:58:39.156 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T13:58:39.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:39.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:39.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:39.292 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-28T13:58:39.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:39.302 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list-id 2024-05-28T13:58:39.357 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:39.357 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T13:58:39.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:58:39.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-28T13:58:39.357 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T13:58:39.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:39.486 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-28T13:58:39.486 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:39.486 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T13:58:39.486 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-28T13:58:39.486 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-28T13:58:39.486 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-28T13:58:39.486 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-28T13:58:39.486 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-28T13:58:39.486 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T13:58:39.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:39.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:39.657 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:39.657 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T13:58:39.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-05-28T13:58:39.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set dev brx.0 master ceph-brx 2024-05-28T13:58:39.657 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T13:58:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:39.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:39.746 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-28T13:58:39.747 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T13:58:39.747 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-28T13:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-28T13:58:39.756 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T13:58:39.756 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-28T13:58:39.807 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T13:58:39.868 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T13:58:39.925 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T13:58:39.925 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T13:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T13:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T13:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T13:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T13:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T13:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T13:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T13:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T13:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T13:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T13:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T13:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T13:58:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T13:58:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T13:58:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T13:58:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T13:58:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T13:58:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T13:58:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T13:58:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T13:58:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T13:58:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T13:58:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T13:58:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T13:58:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T13:58:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T13:58:39.928 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T13:58:39.928 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:58:39.928 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:58:39.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:39.929 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T13:58:39.980 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-28T13:58:39.980 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-28T13:58:40.027 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T13:58:40.047 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T13:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T13:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T13:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T13:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T13:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T13:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T13:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T13:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T13:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T13:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T13:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T13:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T13:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T13:58:40.105 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T13:58:40.105 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T13:58:40.105 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:58:40.105 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:58:40.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:40.106 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T13:58:40.125 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-05-28T13:58:40.137+0000 7fdf7e138580 -1 init, newargv = 0x564436523770 newargc=15 2024-05-28T13:58:40.125 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[33891]: starting ceph client 2024-05-28T13:58:40.134 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[33891]: starting fuse 2024-05-28T13:58:41.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T13:58:41.259 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T13:58:41.315 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T13:58:41.315 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T13:58:41.315 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T13:58:41.315 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T13:58:41.315 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T13:58:41.315 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T13:58:41.315 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T13:58:41.315 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T13:58:41.315 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T13:58:41.315 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T13:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T13:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T13:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T13:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T13:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T13:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T13:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T13:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T13:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T13:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T13:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T13:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T13:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T13:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T13:58:41.317 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T13:58:41.317 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T13:58:41.317 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T13:58:41.317 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T13:58:41.317 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T13:58:41.317 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:58:41.317 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:58:41.317 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-28T13:58:41.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:41.318 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T13:58:41.369 INFO:teuthology.orchestra.run.smithi042.stdout:44 2024-05-28T13:58:41.369 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-28T13:58:41.370 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-05-28T13:58:41.370 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-05-28T13:58:41.370 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-05-28T13:58:41.370 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-05-28T13:58:41.370 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-05-28T13:58:41.370 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T13:58:41.370 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-05-28T13:58:41.370 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-28T13:58:41.370 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-05-28T13:58:41.370 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-28T13:58:41.370 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T13:58:41.370 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-05-28T13:58:41.370 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-05-28T13:58:41.370 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-05-28T13:58:41.371 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-05-28T13:58:41.371 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T13:58:41.371 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-05-28T13:58:41.371 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-28T13:58:41.371 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-28T13:58:41.371 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-28T13:58:41.372 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-05-28T13:58:41.372 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-05-28T13:58:41.372 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-05-28T13:58:41.372 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-28T13:58:41.372 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T13:58:41.372 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-05-28T13:58:41.373 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-05-28T13:58:41.509 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.33891.asok 2024-05-28T13:58:41.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:41.522 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33891.asok 2024-05-28T13:58:41.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:41.522 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33891.asok status 2024-05-28T13:58:41.698 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T13:58:41.699 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-05-28T13:58:41.699 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-28T13:58:41.699 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-28T13:58:41.699 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-05-28T13:58:41.699 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-05-28T13:58:41.700 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-28T13:58:41.700 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "33891", 2024-05-28T13:58:41.700 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-05-28T13:58:41.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:58:41.700 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-05-28T13:58:41.700 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-05-28T13:58:41.701 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 54449, 2024-05-28T13:58:41.701 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-05-28T13:58:41.701 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-05-28T13:58:41.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-05-28T13:58:41.701 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 54449 2024-05-28T13:58:41.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:58:41.702 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T13:58:41.702 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T13:58:41.702 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T13:58:41.702 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3181381289 2024-05-28T13:58:41.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:58:41.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:58:41.702 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T13:58:41.702 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T13:58:41.702 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T13:58:41.702 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3181381289 2024-05-28T13:58:41.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:58:41.702 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.54449 192.168.0.1:0/3181381289", 2024-05-28T13:58:41.703 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/3181381289", 2024-05-28T13:58:41.703 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-05-28T13:58:41.703 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 8, 2024-05-28T13:58:41.703 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 21, 2024-05-28T13:58:41.703 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-05-28T13:58:41.703 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-05-28T13:58:41.703 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-05-28T13:58:41.703 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T13:58:41.714 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-28T13:58:41.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.869+0000 7f6eed600640 1 Processor -- start 2024-05-28T13:58:41.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.869+0000 7f6eed600640 1 -- start start 2024-05-28T13:58:41.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.873+0000 7f6eed600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6ee806f660 0x7f6ee8073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:41.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.873+0000 7f6eed600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee80d0e00 0x7f6ee806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:41.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.873+0000 7f6eed600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee815f280 con 0x7f6ee815ba40 2024-05-28T13:58:41.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.873+0000 7f6eed600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee815f030 con 0x7f6ee806f660 2024-05-28T13:58:41.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.873+0000 7f6eed600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee815f560 con 0x7f6ee80d0e00 2024-05-28T13:58:41.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.873+0000 7f6ee6a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6ee806f660 0x7f6ee8073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:41.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.873+0000 7f6ee7400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee80d0e00 0x7f6ee806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:41.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.873+0000 7f6ee6a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6ee806f660 0x7f6ee8073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:34522/0 (socket says 172.21.15.42:34522) 2024-05-28T13:58:41.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.873+0000 7f6ee6a00640 1 -- 172.21.15.42:0/1009912642 learned_addr learned my addr 172.21.15.42:0/1009912642 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:41.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.873+0000 7f6ee7400640 1 -- 172.21.15.42:0/1009912642 >> v2:172.21.15.63:3300/0 conn(0x7f6ee806f660 msgr2=0x7f6ee8073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:41.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.873+0000 7f6ee7400640 1 --2- 172.21.15.42:0/1009912642 >> v2:172.21.15.63:3300/0 conn(0x7f6ee806f660 0x7f6ee8073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:41.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.873+0000 7f6ee7400640 1 -- 172.21.15.42:0/1009912642 >> v1:172.21.15.63:6789/0 conn(0x7f6ee815ba40 legacy=0x7f6ee80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:41.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.873+0000 7f6ee7400640 1 -- 172.21.15.42:0/1009912642 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee815f840 con 0x7f6ee80d0e00 2024-05-28T13:58:41.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.873+0000 7f6ee6a00640 1 --2- 172.21.15.42:0/1009912642 >> v2:172.21.15.63:3300/0 conn(0x7f6ee806f660 0x7f6ee8073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:41.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.877+0000 7f6ee7400640 1 --2- 172.21.15.42:0/1009912642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee80d0e00 0x7f6ee806dd60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009d90 tx=0x7f6ee001c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f0e2929f929281e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:41.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.877+0000 7f6ee6000640 1 -- 172.21.15.42:0/1009912642 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00b2070 con 0x7f6ee80d0e00 2024-05-28T13:58:41.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.877+0000 7f6ee6000640 1 -- 172.21.15.42:0/1009912642 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee00a58c0 con 0x7f6ee80d0e00 2024-05-28T13:58:41.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.877+0000 7f6ee6000640 1 -- 172.21.15.42:0/1009912642 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00a5c90 con 0x7f6ee80d0e00 2024-05-28T13:58:41.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.877+0000 7f6eed600640 1 -- 172.21.15.42:0/1009912642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee80d0e00 msgr2=0x7f6ee806dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:41.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.877+0000 7f6eed600640 1 --2- 172.21.15.42:0/1009912642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee80d0e00 0x7f6ee806dd60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009d90 tx=0x7f6ee001c820 comp rx=0 tx=0).stop 2024-05-28T13:58:41.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.877+0000 7f6eed600640 1 -- 172.21.15.42:0/1009912642 shutdown_connections 2024-05-28T13:58:41.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.877+0000 7f6eed600640 1 --2- 172.21.15.42:0/1009912642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee80d0e00 0x7f6ee806dd60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:41.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.877+0000 7f6eed600640 1 --2- 172.21.15.42:0/1009912642 >> v2:172.21.15.63:3300/0 conn(0x7f6ee806f660 0x7f6ee8073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:41.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.877+0000 7f6eed600640 1 -- 172.21.15.42:0/1009912642 >> 172.21.15.42:0/1009912642 conn(0x7f6ee806bdf0 msgr2=0x7f6ee806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:41.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.877+0000 7f6eed600640 1 -- 172.21.15.42:0/1009912642 shutdown_connections 2024-05-28T13:58:41.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.877+0000 7f6eed600640 1 -- 172.21.15.42:0/1009912642 wait complete. 2024-05-28T13:58:41.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.877+0000 7f6eed600640 1 Processor -- start 2024-05-28T13:58:41.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.877+0000 7f6eed600640 1 -- start start 2024-05-28T13:58:41.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6eed600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ee806f660 0x7f6ee815e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:41.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6eed600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee80d0e00 0x7f6ee8150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:41.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6eed600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ee8151b10 0x7f6ee815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:41.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6eed600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee8168550 con 0x7f6ee806f660 2024-05-28T13:58:41.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6eed600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee81683d0 con 0x7f6ee8151b10 2024-05-28T13:58:41.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6eed600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee81686d0 con 0x7f6ee80d0e00 2024-05-28T13:58:41.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6ee6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ee806f660 0x7f6ee815e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:41.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6ee6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ee806f660 0x7f6ee815e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57036/0 (socket says 172.21.15.42:57036) 2024-05-28T13:58:41.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6ee6a00640 1 -- 172.21.15.42:0/959494290 learned_addr learned my addr 172.21.15.42:0/959494290 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:41.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6edfe00640 1 --2- 172.21.15.42:0/959494290 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee80d0e00 0x7f6ee8150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:41.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6ee7400640 1 --2- 172.21.15.42:0/959494290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ee8151b10 0x7f6ee815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:41.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6ee6a00640 1 -- 172.21.15.42:0/959494290 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee80d0e00 msgr2=0x7f6ee8150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:41.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6ee6a00640 1 --2- 172.21.15.42:0/959494290 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee80d0e00 0x7f6ee8150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:41.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6ee6a00640 1 -- 172.21.15.42:0/959494290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ee8151b10 msgr2=0x7f6ee815a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:41.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6ee6a00640 1 --2- 172.21.15.42:0/959494290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ee8151b10 0x7f6ee815a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:41.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6ee6a00640 1 -- 172.21.15.42:0/959494290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee81507c0 con 0x7f6ee806f660 2024-05-28T13:58:41.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6ee6a00640 1 --2- 172.21.15.42:0/959494290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ee806f660 0x7f6ee815e800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800e9d0 tx=0x7f6ed800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:41.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6edea00640 1 -- 172.21.15.42:0/959494290 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8017070 con 0x7f6ee806f660 2024-05-28T13:58:41.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6edea00640 1 -- 172.21.15.42:0/959494290 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed800ca60 con 0x7f6ee806f660 2024-05-28T13:58:41.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6edea00640 1 -- 172.21.15.42:0/959494290 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed800ce10 con 0x7f6ee806f660 2024-05-28T13:58:41.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6eed600640 1 -- 172.21.15.42:0/959494290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee8150a70 con 0x7f6ee806f660 2024-05-28T13:58:41.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6eed600640 1 -- 172.21.15.42:0/959494290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee8151030 con 0x7f6ee806f660 2024-05-28T13:58:41.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6eed600640 1 -- 172.21.15.42:0/959494290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee8151740 con 0x7f6ee806f660 2024-05-28T13:58:41.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.881+0000 7f6edea00640 1 -- 172.21.15.42:0/959494290 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6ed801a070 con 0x7f6ee806f660 2024-05-28T13:58:41.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.885+0000 7f6edea00640 1 --2- 172.21.15.42:0/959494290 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ec4068380 0x7f6ec406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:41.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.885+0000 7f6edea00640 1 -- 172.21.15.42:0/959494290 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6ed80869d0 con 0x7f6ee806f660 2024-05-28T13:58:41.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.885+0000 7f6edfe00640 1 --2- 172.21.15.42:0/959494290 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ec4068380 0x7f6ec406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:41.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.885+0000 7f6edfe00640 1 --2- 172.21.15.42:0/959494290 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ec4068380 0x7f6ec406a840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6ee81fb220 tx=0x7f6ed0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:41.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:41.889+0000 7f6edea00640 1 -- 172.21.15.42:0/959494290 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6ed8056ef0 con 0x7f6ee806f660 2024-05-28T13:58:42.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42.093+0000 7f6eed600640 1 -- 172.21.15.42:0/959494290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6ee8068a90 con 0x7f6ee806f660 2024-05-28T13:58:42.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42.093+0000 7f6edea00640 1 -- 172.21.15.42:0/959494290 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f6ed80567f0 con 0x7f6ee806f660 2024-05-28T13:58:42.078 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-28T13:58:42.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42.093+0000 7f6eed600640 1 -- 172.21.15.42:0/959494290 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ec4068380 msgr2=0x7f6ec406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:42.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42.093+0000 7f6eed600640 1 --2- 172.21.15.42:0/959494290 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ec4068380 0x7f6ec406a840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6ee81fb220 tx=0x7f6ed0009040 comp rx=0 tx=0).stop 2024-05-28T13:58:42.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42.093+0000 7f6eed600640 1 -- 172.21.15.42:0/959494290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ee806f660 msgr2=0x7f6ee815e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:42.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42.093+0000 7f6eed600640 1 --2- 172.21.15.42:0/959494290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ee806f660 0x7f6ee815e800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800e9d0 tx=0x7f6ed800ee90 comp rx=0 tx=0).stop 2024-05-28T13:58:42.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42.093+0000 7f6eed600640 1 -- 172.21.15.42:0/959494290 shutdown_connections 2024-05-28T13:58:42.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42.093+0000 7f6eed600640 1 --2- 172.21.15.42:0/959494290 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ec4068380 0x7f6ec406a840 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:42.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42.093+0000 7f6eed600640 1 --2- 172.21.15.42:0/959494290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ee8151b10 0x7f6ee815a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:42.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42.093+0000 7f6eed600640 1 --2- 172.21.15.42:0/959494290 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee80d0e00 0x7f6ee8150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:42.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42.093+0000 7f6eed600640 1 --2- 172.21.15.42:0/959494290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ee806f660 0x7f6ee815e800 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:42.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42.093+0000 7f6eed600640 1 -- 172.21.15.42:0/959494290 >> 172.21.15.42:0/959494290 conn(0x7f6ee806bdf0 msgr2=0x7f6ee80d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:42.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42.093+0000 7f6eed600640 1 -- 172.21.15.42:0/959494290 shutdown_connections 2024-05-28T13:58:42.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42.093+0000 7f6eed600640 1 -- 172.21.15.42:0/959494290 wait complete. 2024-05-28T13:58:42.099 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-28T13:58:42.099 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-28T13:58:42.099 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-05-28T13:58:42.099 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-28T13:58:42.099 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-28T13:58:42.100 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-28T13:58:42.100 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-28T13:58:42.100 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-28T13:58:42.100 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-28T13:58:42.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:42.100 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-05-28T13:58:42.109 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T13:58:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T13:58:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T13:58:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-05-28T13:58:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.109 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:58:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-05-28T13:58:42.109 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:58:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-05-28T13:58:42.110 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-28T13:58:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-05-28T13:58:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-28T13:58:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42425sec preferred_lft 42425sec 2024-05-28T13:58:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2024-05-28T13:58:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.110 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:58:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 02:42:65:2b:63:73 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-28T13:58:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:42.113 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-28T13:58:42.174 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-05-28T13:58:42.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:42.176 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-05-28T13:58:42.233 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-05-28T13:58:42.233 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-28T13:58:42.234 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-05-28T13:58:42.234 INFO:teuthology.orchestra.run.smithi042.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-28T13:58:42.234 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-05-28T13:58:42.234 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-28T13:58:42.234 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:42.234 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T13:58:42.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-28T13:58:42.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-28T13:58:42.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-28T13:58:42.234 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T13:58:42.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:42.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:42.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:42.373 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-28T13:58:42.428 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-28T13:58:42.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:42.429 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list-id 2024-05-28T13:58:42.484 INFO:teuthology.orchestra.run.smithi042.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-28T13:58:42.485 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:42.485 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T13:58:42.485 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T13:58:42.485 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-28T13:58:42.485 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T13:58:42.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:42.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:42.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:42.612 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-28T13:58:42.677 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T13:58:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T13:58:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T13:58:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-05-28T13:58:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.677 INFO:teuthology.orchestra.run.smithi042.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-28T13:58:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 22:df:3a:59:6a:8d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-28T13:58:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-28T13:58:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::20df:3aff:fe59:6a8d/64 scope link 2024-05-28T13:58:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:42.679 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-28T13:58:42.740 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T13:58:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T13:58:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T13:58:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-05-28T13:58:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.741 INFO:teuthology.orchestra.run.smithi042.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-28T13:58:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 22:df:3a:59:6a:8d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-28T13:58:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-28T13:58:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::20df:3aff:fe59:6a8d/64 scope link 2024-05-28T13:58:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:42.742 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-28T13:58:42.743 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:42.743 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T13:58:42.743 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-28T13:58:42.743 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-28T13:58:42.743 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-28T13:58:42.743 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-28T13:58:42.743 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-28T13:58:42.743 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T13:58:42.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:42.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:42.927 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:42.927 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T13:58:42.927 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.1 up 2024-05-28T13:58:42.928 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set dev brx.1 master ceph-brx 2024-05-28T13:58:42.928 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T13:58:42.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:43.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:43.013 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-05-28T13:58:43.013 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T13:58:43.013 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-28T13:58:43.022 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-28T13:58:43.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T13:58:43.022 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-28T13:58:43.071 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T13:58:43.132 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T13:58:43.185 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T13:58:43.185 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T13:58:43.186 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T13:58:43.186 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T13:58:43.186 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T13:58:43.186 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T13:58:43.186 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T13:58:43.186 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T13:58:43.186 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T13:58:43.186 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T13:58:43.186 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T13:58:43.186 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T13:58:43.186 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T13:58:43.186 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T13:58:43.186 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T13:58:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:58:43.188 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:58:43.188 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-28T13:58:43.188 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T13:58:43.188 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T13:58:43.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:43.189 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T13:58:43.240 INFO:teuthology.orchestra.run.smithi042.stdout:44 2024-05-28T13:58:43.240 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2024-05-28T13:58:43.240 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-05-28T13:58:43.291 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T13:58:43.312 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T13:58:43.369 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T13:58:43.370 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T13:58:43.370 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T13:58:43.370 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T13:58:43.370 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T13:58:43.370 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T13:58:43.370 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T13:58:43.370 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T13:58:43.370 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T13:58:43.370 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T13:58:43.370 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T13:58:43.371 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T13:58:43.371 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T13:58:43.371 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T13:58:43.371 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T13:58:43.371 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T13:58:43.371 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T13:58:43.371 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T13:58:43.371 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T13:58:43.371 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T13:58:43.371 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T13:58:43.371 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T13:58:43.371 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T13:58:43.372 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T13:58:43.372 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T13:58:43.372 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T13:58:43.372 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T13:58:43.372 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T13:58:43.372 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T13:58:43.372 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:58:43.372 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:58:43.372 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-28T13:58:43.372 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T13:58:43.372 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T13:58:43.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:43.373 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T13:58:43.387 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi042.stderr:2024-05-28T13:58:43.401+0000 7f2a22e14580 -1 init, newargv = 0x5574a077f7d0 newargc=15 2024-05-28T13:58:43.387 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi042.stderr:ceph-fuse[34036]: starting ceph client 2024-05-28T13:58:43.394 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi042.stderr:ceph-fuse[34036]: starting fuse 2024-05-28T13:58:43.395 INFO:teuthology.orchestra.run.smithi042.stdout:44 2024-05-28T13:58:43.395 INFO:teuthology.orchestra.run.smithi042.stdout:45 2024-05-28T13:58:43.396 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 45] 2024-05-28T13:58:43.396 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-05-28T13:58:43.396 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-05-28T13:58:43.396 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-05-28T13:58:43.396 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-05-28T13:58:43.396 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-05-28T13:58:43.396 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T13:58:43.396 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-05-28T13:58:43.396 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-05-28T13:58:43.396 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-05-28T13:58:43.396 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-05-28T13:58:43.396 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T13:58:43.397 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.2")) 2024-05-28T13:58:43.398 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-05-28T13:58:43.522 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.2.34036.asok 2024-05-28T13:58:43.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:43.534 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.34036.asok 2024-05-28T13:58:43.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:43.535 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.34036.asok status 2024-05-28T13:58:43.708 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T13:58:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-05-28T13:58:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-28T13:58:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-28T13:58:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "2", 2024-05-28T13:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-05-28T13:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-05-28T13:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "34036", 2024-05-28T13:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-05-28T13:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-05-28T13:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-05-28T13:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4638, 2024-05-28T13:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-05-28T13:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-05-28T13:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-05-28T13:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 4638 2024-05-28T13:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3624038313 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3624038313 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.4638 192.168.0.2:0/3624038313", 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.2:0/3624038313", 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 8, 2024-05-28T13:58:43.710 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 21, 2024-05-28T13:58:43.711 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-05-28T13:58:43.711 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-05-28T13:58:43.711 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-05-28T13:58:43.711 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T13:58:43.723 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-28T13:58:43.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.877+0000 7f8ce8000640 1 Processor -- start 2024-05-28T13:58:43.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.877+0000 7f8ce8000640 1 -- start start 2024-05-28T13:58:43.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce8000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8ce006f660 0x7f8ce0073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:43.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce8000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ce015ba40 0x7f8ce00d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:43.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce8000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ce015f280 con 0x7f8ce0072dd0 2024-05-28T13:58:43.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce8000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8ce015f030 con 0x7f8ce006f660 2024-05-28T13:58:43.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce8000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce015f560 con 0x7f8ce015ba40 2024-05-28T13:58:43.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce4c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ce015ba40 0x7f8ce00d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:43.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce5600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8ce006f660 0x7f8ce0073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:43.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce5600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8ce006f660 0x7f8ce0073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:34542/0 (socket says 172.21.15.42:34542) 2024-05-28T13:58:43.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce4c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ce015ba40 0x7f8ce00d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:59616/0 (socket says 172.21.15.42:59616) 2024-05-28T13:58:43.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce5600640 1 -- 172.21.15.42:0/2611535562 learned_addr learned my addr 172.21.15.42:0/2611535562 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:43.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce5600640 1 -- 172.21.15.42:0/2611535562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ce015ba40 msgr2=0x7f8ce00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:43.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce5600640 1 --2- 172.21.15.42:0/2611535562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ce015ba40 0x7f8ce00d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:43.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce5600640 1 -- 172.21.15.42:0/2611535562 >> v1:172.21.15.63:6789/0 conn(0x7f8ce0072dd0 legacy=0x7f8ce00b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:43.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce5600640 1 -- 172.21.15.42:0/2611535562 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce015f840 con 0x7f8ce006f660 2024-05-28T13:58:43.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.881+0000 7f8ce4c00640 1 --2- 172.21.15.42:0/2611535562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ce015ba40 0x7f8ce00d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:43.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.885+0000 7f8ce5600640 1 --2- 172.21.15.42:0/2611535562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce006f660 0x7f8ce0073980 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009870 tx=0x7f8cd401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2547791859a0f279 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.885+0000 7f8cdfe00640 1 -- 172.21.15.42:0/2611535562 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd40b0070 con 0x7f8ce006f660 2024-05-28T13:58:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.885+0000 7f8cdfe00640 1 -- 172.21.15.42:0/2611535562 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd40ab600 con 0x7f8ce006f660 2024-05-28T13:58:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.885+0000 7f8cdfe00640 1 -- 172.21.15.42:0/2611535562 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd40ab9d0 con 0x7f8ce006f660 2024-05-28T13:58:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.885+0000 7f8ce8000640 1 -- 172.21.15.42:0/2611535562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce006f660 msgr2=0x7f8ce0073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.885+0000 7f8ce8000640 1 --2- 172.21.15.42:0/2611535562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce006f660 0x7f8ce0073980 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009870 tx=0x7f8cd401c410 comp rx=0 tx=0).stop 2024-05-28T13:58:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.885+0000 7f8ce8000640 1 -- 172.21.15.42:0/2611535562 shutdown_connections 2024-05-28T13:58:43.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.885+0000 7f8ce8000640 1 --2- 172.21.15.42:0/2611535562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ce015ba40 0x7f8ce00d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:43.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.885+0000 7f8ce8000640 1 --2- 172.21.15.42:0/2611535562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce006f660 0x7f8ce0073980 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:43.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.885+0000 7f8ce8000640 1 -- 172.21.15.42:0/2611535562 >> 172.21.15.42:0/2611535562 conn(0x7f8ce006bdf0 msgr2=0x7f8ce006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:43.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.885+0000 7f8ce8000640 1 -- 172.21.15.42:0/2611535562 shutdown_connections 2024-05-28T13:58:43.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.885+0000 7f8ce8000640 1 -- 172.21.15.42:0/2611535562 wait complete. 2024-05-28T13:58:43.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.885+0000 7f8ce8000640 1 Processor -- start 2024-05-28T13:58:43.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.885+0000 7f8ce8000640 1 -- start start 2024-05-28T13:58:43.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce8000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ce006f660 0x7f8ce0153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:43.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce8000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ce0072dd0 0x7f8ce01f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:43.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce8000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce01f92a0 0x7f8ce01f9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:43.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce8000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce0168550 con 0x7f8ce0072dd0 2024-05-28T13:58:43.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce8000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce01683d0 con 0x7f8ce01f92a0 2024-05-28T13:58:43.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce8000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce01686d0 con 0x7f8ce006f660 2024-05-28T13:58:43.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce4c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ce0072dd0 0x7f8ce01f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:43.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce4c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ce0072dd0 0x7f8ce01f8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57052/0 (socket says 172.21.15.42:57052) 2024-05-28T13:58:43.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce4c00640 1 -- 172.21.15.42:0/3038082356 learned_addr learned my addr 172.21.15.42:0/3038082356 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:43.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce5600640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ce006f660 0x7f8ce0153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:43.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce6000640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce01f92a0 0x7f8ce01f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:43.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce4c00640 1 -- 172.21.15.42:0/3038082356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ce006f660 msgr2=0x7f8ce0153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:43.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce4c00640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ce006f660 0x7f8ce0153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:43.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce4c00640 1 -- 172.21.15.42:0/3038082356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce01f92a0 msgr2=0x7f8ce01f9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:43.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce4c00640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce01f92a0 0x7f8ce01f9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:43.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce4c00640 1 -- 172.21.15.42:0/3038082356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce01f9c70 con 0x7f8ce0072dd0 2024-05-28T13:58:43.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce5600640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ce006f660 0x7f8ce0153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:43.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce6000640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce01f92a0 0x7f8ce01f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:43.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce4c00640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ce0072dd0 0x7f8ce01f8d60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00e980 tx=0x7f8ccc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:43.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8cdd600640 1 -- 172.21.15.42:0/3038082356 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc017070 con 0x7f8ce0072dd0 2024-05-28T13:58:43.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8cdd600640 1 -- 172.21.15.42:0/3038082356 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ccc00ca00 con 0x7f8ce0072dd0 2024-05-28T13:58:43.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8cdd600640 1 -- 172.21.15.42:0/3038082356 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc00cdd0 con 0x7f8ce0072dd0 2024-05-28T13:58:43.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce8000640 1 -- 172.21.15.42:0/3038082356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce0153ba0 con 0x7f8ce0072dd0 2024-05-28T13:58:43.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce8000640 1 -- 172.21.15.42:0/3038082356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce00b3a20 con 0x7f8ce0072dd0 2024-05-28T13:58:43.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8ce8000640 1 -- 172.21.15.42:0/3038082356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce0068a90 con 0x7f8ce0072dd0 2024-05-28T13:58:43.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.889+0000 7f8cdd600640 1 -- 172.21.15.42:0/3038082356 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8ccc01a070 con 0x7f8ce0072dd0 2024-05-28T13:58:43.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.893+0000 7f8cdd600640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8cc00683b0 0x7f8cc006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:43.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.893+0000 7f8cdd600640 1 -- 172.21.15.42:0/3038082356 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8ccc0868c0 con 0x7f8ce0072dd0 2024-05-28T13:58:43.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.893+0000 7f8ce5600640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8cc00683b0 0x7f8cc006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:43.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.893+0000 7f8ce5600640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8cc00683b0 0x7f8cc006a870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40ad000 tx=0x7f8cd4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:43.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:43.897+0000 7f8cdd600640 1 -- 172.21.15.42:0/3038082356 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ce01f9c70 con 0x7f8ce0072dd0 2024-05-28T13:58:44.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:44.097+0000 7f8ce8000640 1 -- 172.21.15.42:0/3038082356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8ce0072bb0 con 0x7f8ce0072dd0 2024-05-28T13:58:44.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:44.097+0000 7f8cdd600640 1 -- 172.21.15.42:0/3038082356 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f8ccc056de0 con 0x7f8ce0072dd0 2024-05-28T13:58:44.083 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-28T13:58:44.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:44.101+0000 7f8ce8000640 1 -- 172.21.15.42:0/3038082356 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8cc00683b0 msgr2=0x7f8cc006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:44.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:44.101+0000 7f8ce8000640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8cc00683b0 0x7f8cc006a870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40ad000 tx=0x7f8cd4008040 comp rx=0 tx=0).stop 2024-05-28T13:58:44.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:44.101+0000 7f8ce8000640 1 -- 172.21.15.42:0/3038082356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ce0072dd0 msgr2=0x7f8ce01f8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:44.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:44.101+0000 7f8ce8000640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ce0072dd0 0x7f8ce01f8d60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00e980 tx=0x7f8ccc00ee40 comp rx=0 tx=0).stop 2024-05-28T13:58:44.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:44.101+0000 7f8ce8000640 1 -- 172.21.15.42:0/3038082356 shutdown_connections 2024-05-28T13:58:44.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:44.101+0000 7f8ce8000640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8cc00683b0 0x7f8cc006a870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:44.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:44.101+0000 7f8ce8000640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ce01f92a0 0x7f8ce01f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:44.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:44.101+0000 7f8ce8000640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ce0072dd0 0x7f8ce01f8d60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:44.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:44.101+0000 7f8ce8000640 1 --2- 172.21.15.42:0/3038082356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ce006f660 0x7f8ce0153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:44.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:44.101+0000 7f8ce8000640 1 -- 172.21.15.42:0/3038082356 >> 172.21.15.42:0/3038082356 conn(0x7f8ce006bdf0 msgr2=0x7f8ce015dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:44.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:44.101+0000 7f8ce8000640 1 -- 172.21.15.42:0/3038082356 shutdown_connections 2024-05-28T13:58:44.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:44.101+0000 7f8ce8000640 1 -- 172.21.15.42:0/3038082356 wait complete. 2024-05-28T13:58:44.104 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-28T13:58:44.104 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-28T13:58:44.104 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi063.front.sepia.ceph.com 2024-05-28T13:58:44.105 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-28T13:58:44.105 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-28T13:58:44.105 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-28T13:58:44.105 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-28T13:58:44.105 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-28T13:58:44.105 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-28T13:58:44.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:44.105 DEBUG:teuthology.orchestra.run.smithi063:> ip addr 2024-05-28T13:58:44.115 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T13:58:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T13:58:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T13:58:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-28T13:58:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:44.116 INFO:teuthology.orchestra.run.smithi063.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:58:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:76 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f0 2024-05-28T13:58:44.116 INFO:teuthology.orchestra.run.smithi063.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:58:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:77 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f1 2024-05-28T13:58:44.116 INFO:teuthology.orchestra.run.smithi063.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:58:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d6 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:44.117 INFO:teuthology.orchestra.run.smithi063.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-28T13:58:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d7 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.21.15.63/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-28T13:58:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft 42427sec preferred_lft 42427sec 2024-05-28T13:58:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-05-28T13:58:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:44.117 INFO:teuthology.orchestra.run.smithi063.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-28T13:58:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 02:42:0d:27:e7:79 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-28T13:58:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:44.118 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-28T13:58:44.118 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:44.118 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-28T13:58:44.118 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add name ceph-brx type bridge 2024-05-28T13:58:44.118 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr flush dev ceph-brx 2024-05-28T13:58:44.119 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set ceph-brx up 2024-05-28T13:58:44.119 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-28T13:58:44.119 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-28T13:58:44.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:44.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:44.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:44.284 DEBUG:teuthology.orchestra.run.smithi063:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-28T13:58:44.303 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-05-28T13:58:44.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:44.305 DEBUG:teuthology.orchestra.run.smithi063:> route 2024-05-28T13:58:44.387 INFO:teuthology.orchestra.run.smithi063.stdout:Kernel IP routing table 2024-05-28T13:58:44.387 INFO:teuthology.orchestra.run.smithi063.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-28T13:58:44.387 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-28T13:58:44.388 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-28T13:58:44.388 INFO:teuthology.orchestra.run.smithi063.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-28T13:58:44.388 INFO:teuthology.orchestra.run.smithi063.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-28T13:58:44.388 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-28T13:58:44.388 INFO:teuthology.orchestra.run.smithi063.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-28T13:58:44.388 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-28T13:58:44.388 INFO:teuthology.orchestra.run.smithi063.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-28T13:58:44.388 INFO:teuthology.orchestra.run.smithi063.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-28T13:58:44.388 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:44.388 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-28T13:58:44.388 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-28T13:58:44.389 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-28T13:58:44.389 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-28T13:58:44.389 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-28T13:58:44.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:44.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:44.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:44.527 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-28T13:58:44.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:44.537 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list-id 2024-05-28T13:58:44.587 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:44.587 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-28T13:58:44.587 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-28T13:58:44.587 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-28T13:58:44.587 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-28T13:58:44.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:44.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:44.715 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-28T13:58:44.716 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:44.716 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-28T13:58:44.716 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-28T13:58:44.716 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-28T13:58:44.716 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-28T13:58:44.716 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-28T13:58:44.716 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-28T13:58:44.716 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-28T13:58:44.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:44.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:45.000 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:45.001 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-28T13:58:45.001 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set brx.0 up 2024-05-28T13:58:45.001 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set dev brx.0 master ceph-brx 2024-05-28T13:58:45.001 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-28T13:58:45.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:45.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:45.127 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-28T13:58:45.128 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T13:58:45.128 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-28T13:58:45.136 INFO:teuthology.orchestra.run.smithi063.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-28T13:58:45.136 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T13:58:45.137 DEBUG:teuthology.orchestra.run.smithi063:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-28T13:58:45.186 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-28T13:58:45.246 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T13:58:45.300 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-28T13:58:45.300 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-28T13:58:45.300 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-28T13:58:45.300 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-28T13:58:45.300 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-28T13:58:45.300 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-28T13:58:45.300 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-28T13:58:45.300 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-28T13:58:45.300 INFO:teuthology.orchestra.run.smithi063.stdout:/run/lock 2024-05-28T13:58:45.300 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-28T13:58:45.300 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-28T13:58:45.300 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-28T13:58:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T13:58:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-28T13:58:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-28T13:58:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-28T13:58:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-28T13:58:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/fs/nfsd 2024-05-28T13:58:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-28T13:58:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-28T13:58:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T13:58:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:/run/rpc_pipefs 2024-05-28T13:58:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-28T13:58:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-28T13:58:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-4 2024-05-28T13:58:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-5 2024-05-28T13:58:45.302 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-6 2024-05-28T13:58:45.302 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-7 2024-05-28T13:58:45.302 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-28T13:58:45.302 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-28T13:58:45.302 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-28T13:58:45.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:45.303 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-28T13:58:45.358 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-28T13:58:45.359 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-28T13:58:45.409 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-28T13:58:45.430 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T13:58:45.488 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-28T13:58:45.488 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-28T13:58:45.488 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-28T13:58:45.488 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-28T13:58:45.488 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-28T13:58:45.488 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-28T13:58:45.488 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-28T13:58:45.488 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-28T13:58:45.488 INFO:teuthology.orchestra.run.smithi063.stdout:/run/lock 2024-05-28T13:58:45.488 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-28T13:58:45.488 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/fs/nfsd 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/run/rpc_pipefs 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-4 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-5 2024-05-28T13:58:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-6 2024-05-28T13:58:45.490 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-7 2024-05-28T13:58:45.490 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-28T13:58:45.490 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-28T13:58:45.490 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-28T13:58:45.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:45.491 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-28T13:58:45.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi063.stderr:2024-05-28T13:58:45.519+0000 7f88e8611580 -1 init, newargv = 0x55bfcdb85b70 newargc=15 2024-05-28T13:58:45.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi063.stderr:ceph-fuse[33718]: starting ceph client 2024-05-28T13:58:45.514 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi063.stderr:ceph-fuse[33718]: starting fuse 2024-05-28T13:58:46.615 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-28T13:58:46.640 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T13:58:46.697 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-28T13:58:46.697 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-28T13:58:46.697 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-28T13:58:46.697 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-28T13:58:46.697 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-28T13:58:46.697 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-28T13:58:46.697 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-28T13:58:46.697 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-28T13:58:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:/run/lock 2024-05-28T13:58:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-28T13:58:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-28T13:58:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-28T13:58:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T13:58:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-28T13:58:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-28T13:58:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-28T13:58:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-28T13:58:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/fs/nfsd 2024-05-28T13:58:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-28T13:58:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-28T13:58:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T13:58:46.699 INFO:teuthology.orchestra.run.smithi063.stdout:/run/rpc_pipefs 2024-05-28T13:58:46.699 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-28T13:58:46.699 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-28T13:58:46.699 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-4 2024-05-28T13:58:46.699 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-5 2024-05-28T13:58:46.699 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-6 2024-05-28T13:58:46.699 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-7 2024-05-28T13:58:46.699 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-28T13:58:46.699 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-28T13:58:46.699 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-28T13:58:46.699 INFO:teuthology.orchestra.run.smithi063.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-28T13:58:46.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:46.700 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-28T13:58:46.755 INFO:teuthology.orchestra.run.smithi063.stdout:44 2024-05-28T13:58:46.756 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-28T13:58:46.756 DEBUG:teuthology.orchestra.run.smithi063:> sudo stdin-killer -- python3 -c ' 2024-05-28T13:58:46.756 DEBUG:teuthology.orchestra.run.smithi063:> import glob 2024-05-28T13:58:46.756 DEBUG:teuthology.orchestra.run.smithi063:> import re 2024-05-28T13:58:46.756 DEBUG:teuthology.orchestra.run.smithi063:> import os 2024-05-28T13:58:46.756 DEBUG:teuthology.orchestra.run.smithi063:> import subprocess 2024-05-28T13:58:46.756 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-28T13:58:46.756 DEBUG:teuthology.orchestra.run.smithi063:> def _find_admin_socket(client_name): 2024-05-28T13:58:46.756 DEBUG:teuthology.orchestra.run.smithi063:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-28T13:58:46.756 DEBUG:teuthology.orchestra.run.smithi063:> files = glob.glob(asok_path) 2024-05-28T13:58:46.756 DEBUG:teuthology.orchestra.run.smithi063:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-28T13:58:46.757 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-28T13:58:46.757 DEBUG:teuthology.orchestra.run.smithi063:> # Given a non-glob path, it better be there 2024-05-28T13:58:46.757 DEBUG:teuthology.orchestra.run.smithi063:> if "*" not in asok_path: 2024-05-28T13:58:46.757 DEBUG:teuthology.orchestra.run.smithi063:> assert(len(files) == 1) 2024-05-28T13:58:46.757 DEBUG:teuthology.orchestra.run.smithi063:> return files[0] 2024-05-28T13:58:46.757 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-28T13:58:46.757 DEBUG:teuthology.orchestra.run.smithi063:> for f in files: 2024-05-28T13:58:46.757 DEBUG:teuthology.orchestra.run.smithi063:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-28T13:58:46.757 DEBUG:teuthology.orchestra.run.smithi063:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-28T13:58:46.757 DEBUG:teuthology.orchestra.run.smithi063:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-28T13:58:46.757 DEBUG:teuthology.orchestra.run.smithi063:> contents = proc_f.read() 2024-05-28T13:58:46.757 DEBUG:teuthology.orchestra.run.smithi063:> if mountpoint in contents: 2024-05-28T13:58:46.757 DEBUG:teuthology.orchestra.run.smithi063:> return f 2024-05-28T13:58:46.757 DEBUG:teuthology.orchestra.run.smithi063:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-28T13:58:46.758 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-28T13:58:46.758 DEBUG:teuthology.orchestra.run.smithi063:> print(_find_admin_socket("client.1")) 2024-05-28T13:58:46.758 DEBUG:teuthology.orchestra.run.smithi063:> ' 2024-05-28T13:58:46.891 INFO:teuthology.orchestra.run.smithi063.stdout:/var/run/ceph/ceph-client.1.33718.asok 2024-05-28T13:58:46.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:46.904 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33718.asok 2024-05-28T13:58:46.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:46.904 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33718.asok status 2024-05-28T13:58:47.080 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-28T13:58:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "metadata": { 2024-05-28T13:58:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-28T13:58:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-28T13:58:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "entity_id": "1", 2024-05-28T13:58:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "hostname": "smithi063", 2024-05-28T13:58:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-28T13:58:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "pid": "33718", 2024-05-28T13:58:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "root": "/" 2024-05-28T13:58:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_count": 0, 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_pinned_count": 0, 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 54467, 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "inst": { 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "name": { 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "client", 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "num": 54467 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.1:0", 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 72814863 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-28T13:58:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-05-28T13:58:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-28T13:58:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.1:0", 2024-05-28T13:58:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 72814863 2024-05-28T13:58:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-28T13:58:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "inst_str": "client.54467 192.168.0.1:0/72814863", 2024-05-28T13:58:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "addr_str": "192.168.0.1:0/72814863", 2024-05-28T13:58:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "inode_count": 1, 2024-05-28T13:58:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_epoch": 8, 2024-05-28T13:58:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch": 21, 2024-05-28T13:58:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch_barrier": 0, 2024-05-28T13:58:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "blocklisted": false, 2024-05-28T13:58:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs" 2024-05-28T13:58:47.083 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-28T13:58:47.096 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-28T13:58:47.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.261+0000 7f50e9800640 1 Processor -- start 2024-05-28T13:58:47.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.265+0000 7f50e9800640 1 -- start start 2024-05-28T13:58:47.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.265+0000 7f50e9800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f50e40afa50 0x7f50e406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:47.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.265+0000 7f50e9800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f50e4156e50 0x7f50e40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:47.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.265+0000 7f50e9800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50e415cb90 con 0x7f50e4156a70 2024-05-28T13:58:47.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.265+0000 7f50e9800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f50e415ce50 con 0x7f50e4156e50 2024-05-28T13:58:47.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.265+0000 7f50e9800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f50e415d130 con 0x7f50e40afa50 2024-05-28T13:58:47.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.265+0000 7f50e3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f50e4156e50 0x7f50e40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:47.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.265+0000 7f50e3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f50e4156e50 0x7f50e40d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57058/0 (socket says 172.21.15.42:57058) 2024-05-28T13:58:47.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.265+0000 7f50e3400640 1 -- 172.21.15.42:0/2344468615 learned_addr learned my addr 172.21.15.42:0/2344468615 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:47.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.265+0000 7f50da000640 1 --2- 172.21.15.42:0/2344468615 >> v2:172.21.15.63:3300/0 conn(0x7f50e40afa50 0x7f50e406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:47.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.265+0000 7f50e3400640 1 -- 172.21.15.42:0/2344468615 >> v2:172.21.15.63:3300/0 conn(0x7f50e40afa50 msgr2=0x7f50e406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:47.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.265+0000 7f50e3400640 1 --2- 172.21.15.42:0/2344468615 >> v2:172.21.15.63:3300/0 conn(0x7f50e40afa50 0x7f50e406dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:47.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.265+0000 7f50e3400640 1 -- 172.21.15.42:0/2344468615 >> v1:172.21.15.63:6789/0 conn(0x7f50e4156a70 legacy=0x7f50e40af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:47.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.265+0000 7f50e3400640 1 -- 172.21.15.42:0/2344468615 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e415d410 con 0x7f50e4156e50 2024-05-28T13:58:47.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.269+0000 7f50e3400640 1 --2- 172.21.15.42:0/2344468615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50e4156e50 0x7f50e40d06f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f50e40701f0 tx=0x7f50dc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6cc4ba201e1549 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:47.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.269+0000 7f50e2000640 1 -- 172.21.15.42:0/2344468615 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc0a57d0 con 0x7f50e4156e50 2024-05-28T13:58:47.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.269+0000 7f50e2000640 1 -- 172.21.15.42:0/2344468615 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50dc0a5970 con 0x7f50e4156e50 2024-05-28T13:58:47.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.269+0000 7f50e2000640 1 -- 172.21.15.42:0/2344468615 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc0a5d40 con 0x7f50e4156e50 2024-05-28T13:58:47.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.269+0000 7f50e9800640 1 -- 172.21.15.42:0/2344468615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50e4156e50 msgr2=0x7f50e40d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:47.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.269+0000 7f50e9800640 1 --2- 172.21.15.42:0/2344468615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50e4156e50 0x7f50e40d06f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f50e40701f0 tx=0x7f50dc01c9f0 comp rx=0 tx=0).stop 2024-05-28T13:58:47.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.269+0000 7f50e9800640 1 -- 172.21.15.42:0/2344468615 shutdown_connections 2024-05-28T13:58:47.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.269+0000 7f50e9800640 1 --2- 172.21.15.42:0/2344468615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50e4156e50 0x7f50e40d06f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:47.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.269+0000 7f50e9800640 1 --2- 172.21.15.42:0/2344468615 >> v2:172.21.15.63:3300/0 conn(0x7f50e40afa50 0x7f50e406dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:47.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.269+0000 7f50e9800640 1 -- 172.21.15.42:0/2344468615 >> 172.21.15.42:0/2344468615 conn(0x7f50e406bdf0 msgr2=0x7f50e406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:47.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.269+0000 7f50e9800640 1 -- 172.21.15.42:0/2344468615 shutdown_connections 2024-05-28T13:58:47.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.269+0000 7f50e9800640 1 -- 172.21.15.42:0/2344468615 wait complete. 2024-05-28T13:58:47.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.269+0000 7f50e9800640 1 Processor -- start 2024-05-28T13:58:47.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.269+0000 7f50e9800640 1 -- start start 2024-05-28T13:58:47.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50e40afa50 0x7f50e414f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:47.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e4156a70 0x7f50e40ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:47.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e9800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50e4156e50 0x7f50e4152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:47.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e9800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f50e4160ed0 con 0x7f50e40afa50 2024-05-28T13:58:47.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e9800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50e4160d50 con 0x7f50e4156a70 2024-05-28T13:58:47.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e9800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f50e4161050 con 0x7f50e4156e50 2024-05-28T13:58:47.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50e40afa50 0x7f50e414f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:47.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50e40afa50 0x7f50e414f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57060/0 (socket says 172.21.15.42:57060) 2024-05-28T13:58:47.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e2a00640 1 -- 172.21.15.42:0/4179867215 learned_addr learned my addr 172.21.15.42:0/4179867215 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:47.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e3400640 1 --2- 172.21.15.42:0/4179867215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50e4156e50 0x7f50e4152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:47.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50da000640 1 --2- 172.21.15.42:0/4179867215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e4156a70 0x7f50e40ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:47.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e2a00640 1 -- 172.21.15.42:0/4179867215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50e4156e50 msgr2=0x7f50e4152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:47.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e2a00640 1 --2- 172.21.15.42:0/4179867215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50e4156e50 0x7f50e4152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:47.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e2a00640 1 -- 172.21.15.42:0/4179867215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e4156a70 msgr2=0x7f50e40ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:47.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e2a00640 1 --2- 172.21.15.42:0/4179867215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e4156a70 0x7f50e40ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:47.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e2a00640 1 -- 172.21.15.42:0/4179867215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e415dd30 con 0x7f50e40afa50 2024-05-28T13:58:47.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e2a00640 1 --2- 172.21.15.42:0/4179867215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50e40afa50 0x7f50e414f990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f50d400c420 tx=0x7f50d400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:47.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50db400640 1 -- 172.21.15.42:0/4179867215 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d40052d0 con 0x7f50e40afa50 2024-05-28T13:58:47.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50db400640 1 -- 172.21.15.42:0/4179867215 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d4005470 con 0x7f50e40afa50 2024-05-28T13:58:47.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e9800640 1 -- 172.21.15.42:0/4179867215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e414fed0 con 0x7f50e40afa50 2024-05-28T13:58:47.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50db400640 1 -- 172.21.15.42:0/4179867215 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4007260 con 0x7f50e40afa50 2024-05-28T13:58:47.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e9800640 1 -- 172.21.15.42:0/4179867215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e40738d0 con 0x7f50e40afa50 2024-05-28T13:58:47.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50e9800640 1 -- 172.21.15.42:0/4179867215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50e4068a90 con 0x7f50e40afa50 2024-05-28T13:58:47.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.273+0000 7f50db400640 1 -- 172.21.15.42:0/4179867215 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f50d4007400 con 0x7f50e40afa50 2024-05-28T13:58:47.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.277+0000 7f50db400640 1 --2- 172.21.15.42:0/4179867215 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f50c0068380 0x7f50c006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:47.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.277+0000 7f50db400640 1 -- 172.21.15.42:0/4179867215 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f50d4085ed0 con 0x7f50e40afa50 2024-05-28T13:58:47.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.277+0000 7f50da000640 1 --2- 172.21.15.42:0/4179867215 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f50c0068380 0x7f50c006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:47.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.277+0000 7f50da000640 1 --2- 172.21.15.42:0/4179867215 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f50c0068380 0x7f50c006a840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f50e40cfd00 tx=0x7f50cc009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:47.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.281+0000 7f50db400640 1 -- 172.21.15.42:0/4179867215 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f50d4055da0 con 0x7f50e40afa50 2024-05-28T13:58:47.333 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T13:58:47.336 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T13:58:47.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.489+0000 7f50e9800640 1 -- 172.21.15.42:0/4179867215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f50e406dd90 con 0x7f50e40afa50 2024-05-28T13:58:47.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.489+0000 7f50db400640 1 -- 172.21.15.42:0/4179867215 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f50d405a0a0 con 0x7f50e40afa50 2024-05-28T13:58:47.475 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-28T13:58:47.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.493+0000 7f50e9800640 1 -- 172.21.15.42:0/4179867215 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f50c0068380 msgr2=0x7f50c006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:47.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.493+0000 7f50e9800640 1 --2- 172.21.15.42:0/4179867215 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f50c0068380 0x7f50c006a840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f50e40cfd00 tx=0x7f50cc009190 comp rx=0 tx=0).stop 2024-05-28T13:58:47.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.493+0000 7f50e9800640 1 -- 172.21.15.42:0/4179867215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50e40afa50 msgr2=0x7f50e414f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:47.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.493+0000 7f50e9800640 1 --2- 172.21.15.42:0/4179867215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50e40afa50 0x7f50e414f990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f50d400c420 tx=0x7f50d400c8e0 comp rx=0 tx=0).stop 2024-05-28T13:58:47.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.493+0000 7f50e9800640 1 -- 172.21.15.42:0/4179867215 shutdown_connections 2024-05-28T13:58:47.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.493+0000 7f50e9800640 1 --2- 172.21.15.42:0/4179867215 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f50c0068380 0x7f50c006a840 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:47.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.493+0000 7f50e9800640 1 --2- 172.21.15.42:0/4179867215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50e4156e50 0x7f50e4152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:47.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.493+0000 7f50e9800640 1 --2- 172.21.15.42:0/4179867215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e4156a70 0x7f50e40ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:47.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.493+0000 7f50e9800640 1 --2- 172.21.15.42:0/4179867215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50e40afa50 0x7f50e414f990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:47.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.493+0000 7f50e9800640 1 -- 172.21.15.42:0/4179867215 >> 172.21.15.42:0/4179867215 conn(0x7f50e406bdf0 msgr2=0x7f50e4166b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:47.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.493+0000 7f50e9800640 1 -- 172.21.15.42:0/4179867215 shutdown_connections 2024-05-28T13:58:47.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:47.493+0000 7f50e9800640 1 -- 172.21.15.42:0/4179867215 wait complete. 2024-05-28T13:58:47.496 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-28T13:58:47.497 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-28T13:58:47.497 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi063.front.sepia.ceph.com 2024-05-28T13:58:47.497 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-28T13:58:47.497 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-28T13:58:47.497 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-28T13:58:47.497 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-28T13:58:47.497 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-28T13:58:47.497 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-28T13:58:47.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:47.497 DEBUG:teuthology.orchestra.run.smithi063:> ip addr 2024-05-28T13:58:47.506 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T13:58:47.506 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T13:58:47.506 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T13:58:47.506 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:47.506 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-28T13:58:47.506 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:47.506 INFO:teuthology.orchestra.run.smithi063.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:76 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f0 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:77 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f1 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d6 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d7 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.21.15.63/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft 42423sec preferred_lft 42423sec 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-28T13:58:47.507 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 02:42:0d:27:e7:79 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:47.508 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-28T13:58:47.508 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:47.508 INFO:teuthology.orchestra.run.smithi063.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-28T13:58:47.508 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-28T13:58:47.508 INFO:teuthology.orchestra.run.smithi063.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-28T13:58:47.508 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:47.508 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-28T13:58:47.508 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:47.508 INFO:teuthology.orchestra.run.smithi063.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T13:58:47.508 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-28T13:58:47.508 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-28T13:58:47.508 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:47.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:47.509 DEBUG:teuthology.orchestra.run.smithi063:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-28T13:58:47.569 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-05-28T13:58:47.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:47.571 DEBUG:teuthology.orchestra.run.smithi063:> route 2024-05-28T13:58:47.626 INFO:teuthology.orchestra.run.smithi063.stdout:Kernel IP routing table 2024-05-28T13:58:47.627 INFO:teuthology.orchestra.run.smithi063.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-28T13:58:47.627 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-28T13:58:47.627 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-28T13:58:47.627 INFO:teuthology.orchestra.run.smithi063.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-28T13:58:47.627 INFO:teuthology.orchestra.run.smithi063.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-28T13:58:47.627 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-28T13:58:47.627 INFO:teuthology.orchestra.run.smithi063.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-28T13:58:47.627 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-28T13:58:47.627 INFO:teuthology.orchestra.run.smithi063.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-28T13:58:47.627 INFO:teuthology.orchestra.run.smithi063.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-28T13:58:47.628 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:47.628 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-28T13:58:47.628 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-28T13:58:47.628 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-28T13:58:47.628 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-28T13:58:47.628 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-28T13:58:47.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:47.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:47.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:47.772 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-28T13:58:47.780 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-28T13:58:47.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:47.781 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list-id 2024-05-28T13:58:47.835 INFO:teuthology.orchestra.run.smithi063.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-28T13:58:47.835 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:47.835 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-28T13:58:47.835 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-28T13:58:47.835 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-28T13:58:47.835 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-28T13:58:47.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:47.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:47.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:47.968 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-28T13:58:48.033 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T13:58:48.033 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T13:58:48.033 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T13:58:48.033 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:48.033 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-28T13:58:48.033 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:48.033 INFO:teuthology.orchestra.run.smithi063.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-28T13:58:48.033 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 1e:55:e8:7d:97:8c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-28T13:58:48.033 INFO:teuthology.orchestra.run.smithi063.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-28T13:58:48.034 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:48.034 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::1c55:e8ff:fe7d:978c/64 scope link 2024-05-28T13:58:48.034 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:48.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:48.035 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-28T13:58:48.098 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T13:58:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T13:58:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T13:58:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-28T13:58:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:48.099 INFO:teuthology.orchestra.run.smithi063.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-28T13:58:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 1e:55:e8:7d:97:8c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-28T13:58:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-28T13:58:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::1c55:e8ff:fe7d:978c/64 scope link 2024-05-28T13:58:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:58:48.100 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-28T13:58:48.100 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:48.100 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-28T13:58:48.100 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-28T13:58:48.101 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-28T13:58:48.101 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-28T13:58:48.101 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-28T13:58:48.101 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-28T13:58:48.101 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-28T13:58:48.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:48.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:48.285 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:58:48.286 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-28T13:58:48.286 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set brx.1 up 2024-05-28T13:58:48.286 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set dev brx.1 master ceph-brx 2024-05-28T13:58:48.286 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-28T13:58:48.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:58:48.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:48.371 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-05-28T13:58:48.371 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T13:58:48.371 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-28T13:58:48.380 INFO:teuthology.orchestra.run.smithi063.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-28T13:58:48.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T13:58:48.380 DEBUG:teuthology.orchestra.run.smithi063:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-28T13:58:48.430 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-28T13:58:48.491 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T13:58:48.543 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-28T13:58:48.543 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-28T13:58:48.543 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-28T13:58:48.543 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-28T13:58:48.544 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-28T13:58:48.544 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-28T13:58:48.544 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-28T13:58:48.544 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-28T13:58:48.544 INFO:teuthology.orchestra.run.smithi063.stdout:/run/lock 2024-05-28T13:58:48.544 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-28T13:58:48.544 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-28T13:58:48.544 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-28T13:58:48.544 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T13:58:48.544 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-28T13:58:48.544 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-28T13:58:48.544 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-28T13:58:48.545 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-28T13:58:48.545 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/fs/nfsd 2024-05-28T13:58:48.545 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-28T13:58:48.545 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-28T13:58:48.545 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T13:58:48.545 INFO:teuthology.orchestra.run.smithi063.stdout:/run/rpc_pipefs 2024-05-28T13:58:48.545 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-28T13:58:48.545 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-28T13:58:48.545 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-4 2024-05-28T13:58:48.545 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-5 2024-05-28T13:58:48.545 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-6 2024-05-28T13:58:48.545 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-7 2024-05-28T13:58:48.545 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-28T13:58:48.546 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-28T13:58:48.546 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-28T13:58:48.546 INFO:teuthology.orchestra.run.smithi063.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-28T13:58:48.546 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-28T13:58:48.546 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-28T13:58:48.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:48.547 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-28T13:58:48.602 INFO:teuthology.orchestra.run.smithi063.stdout:44 2024-05-28T13:58:48.602 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2024-05-28T13:58:48.602 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-05-28T13:58:48.645 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-28T13:58:48.666 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T13:58:48.719 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-28T13:58:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-28T13:58:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-28T13:58:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-28T13:58:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-28T13:58:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-28T13:58:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-28T13:58:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-28T13:58:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:/run/lock 2024-05-28T13:58:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-28T13:58:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-28T13:58:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-28T13:58:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T13:58:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-28T13:58:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-28T13:58:48.721 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-28T13:58:48.721 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-28T13:58:48.721 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/fs/nfsd 2024-05-28T13:58:48.721 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-28T13:58:48.721 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-28T13:58:48.721 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T13:58:48.721 INFO:teuthology.orchestra.run.smithi063.stdout:/run/rpc_pipefs 2024-05-28T13:58:48.721 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-28T13:58:48.721 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-28T13:58:48.721 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-4 2024-05-28T13:58:48.721 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-5 2024-05-28T13:58:48.721 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-6 2024-05-28T13:58:48.721 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph/osd/ceph-7 2024-05-28T13:58:48.721 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-28T13:58:48.722 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-28T13:58:48.722 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-28T13:58:48.722 INFO:teuthology.orchestra.run.smithi063.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-28T13:58:48.722 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-28T13:58:48.722 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-28T13:58:48.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:48.723 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-28T13:58:48.740 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi063.stderr:2024-05-28T13:58:48.755+0000 7fb2c31c9580 -1 init, newargv = 0x5642bb3daa40 newargc=15 2024-05-28T13:58:48.741 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi063.stderr:ceph-fuse[33839]: starting ceph client 2024-05-28T13:58:48.748 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi063.stderr:ceph-fuse[33839]: starting fuse 2024-05-28T13:58:48.749 INFO:teuthology.orchestra.run.smithi063.stdout:44 2024-05-28T13:58:48.749 INFO:teuthology.orchestra.run.smithi063.stdout:45 2024-05-28T13:58:48.749 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 45] 2024-05-28T13:58:48.749 DEBUG:teuthology.orchestra.run.smithi063:> sudo stdin-killer -- python3 -c ' 2024-05-28T13:58:48.749 DEBUG:teuthology.orchestra.run.smithi063:> import glob 2024-05-28T13:58:48.750 DEBUG:teuthology.orchestra.run.smithi063:> import re 2024-05-28T13:58:48.750 DEBUG:teuthology.orchestra.run.smithi063:> import os 2024-05-28T13:58:48.750 DEBUG:teuthology.orchestra.run.smithi063:> import subprocess 2024-05-28T13:58:48.750 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-28T13:58:48.750 DEBUG:teuthology.orchestra.run.smithi063:> def _find_admin_socket(client_name): 2024-05-28T13:58:48.750 DEBUG:teuthology.orchestra.run.smithi063:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-05-28T13:58:48.750 DEBUG:teuthology.orchestra.run.smithi063:> files = glob.glob(asok_path) 2024-05-28T13:58:48.750 DEBUG:teuthology.orchestra.run.smithi063:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-05-28T13:58:48.750 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-28T13:58:48.750 DEBUG:teuthology.orchestra.run.smithi063:> # Given a non-glob path, it better be there 2024-05-28T13:58:48.750 DEBUG:teuthology.orchestra.run.smithi063:> if "*" not in asok_path: 2024-05-28T13:58:48.750 DEBUG:teuthology.orchestra.run.smithi063:> assert(len(files) == 1) 2024-05-28T13:58:48.750 DEBUG:teuthology.orchestra.run.smithi063:> return files[0] 2024-05-28T13:58:48.750 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-28T13:58:48.751 DEBUG:teuthology.orchestra.run.smithi063:> for f in files: 2024-05-28T13:58:48.751 DEBUG:teuthology.orchestra.run.smithi063:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-28T13:58:48.751 DEBUG:teuthology.orchestra.run.smithi063:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-28T13:58:48.751 DEBUG:teuthology.orchestra.run.smithi063:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-28T13:58:48.751 DEBUG:teuthology.orchestra.run.smithi063:> contents = proc_f.read() 2024-05-28T13:58:48.751 DEBUG:teuthology.orchestra.run.smithi063:> if mountpoint in contents: 2024-05-28T13:58:48.751 DEBUG:teuthology.orchestra.run.smithi063:> return f 2024-05-28T13:58:48.751 DEBUG:teuthology.orchestra.run.smithi063:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-28T13:58:48.751 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-28T13:58:48.751 DEBUG:teuthology.orchestra.run.smithi063:> print(_find_admin_socket("client.3")) 2024-05-28T13:58:48.751 DEBUG:teuthology.orchestra.run.smithi063:> ' 2024-05-28T13:58:48.873 INFO:teuthology.orchestra.run.smithi063.stdout:/var/run/ceph/ceph-client.3.33839.asok 2024-05-28T13:58:48.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-28T13:58:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:58:48.884 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33839.asok 2024-05-28T13:58:48.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:48.884 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33839.asok status 2024-05-28T13:58:49.063 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-28T13:58:49.063 INFO:teuthology.orchestra.run.smithi063.stdout: "metadata": { 2024-05-28T13:58:49.063 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-28T13:58:49.063 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-28T13:58:49.063 INFO:teuthology.orchestra.run.smithi063.stdout: "entity_id": "3", 2024-05-28T13:58:49.063 INFO:teuthology.orchestra.run.smithi063.stdout: "hostname": "smithi063", 2024-05-28T13:58:49.063 INFO:teuthology.orchestra.run.smithi063.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-05-28T13:58:49.063 INFO:teuthology.orchestra.run.smithi063.stdout: "pid": "33839", 2024-05-28T13:58:49.063 INFO:teuthology.orchestra.run.smithi063.stdout: "root": "/" 2024-05-28T13:58:49.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_count": 0, 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_pinned_count": 0, 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 54473, 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: "inst": { 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: "name": { 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "client", 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: "num": 54473 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.2:0", 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3657647403 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-28T13:58:49.064 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-05-28T13:58:49.065 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-28T13:58:49.065 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.2:0", 2024-05-28T13:58:49.065 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3657647403 2024-05-28T13:58:49.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-28T13:58:49.065 INFO:teuthology.orchestra.run.smithi063.stdout: "inst_str": "client.54473 192.168.0.2:0/3657647403", 2024-05-28T13:58:49.065 INFO:teuthology.orchestra.run.smithi063.stdout: "addr_str": "192.168.0.2:0/3657647403", 2024-05-28T13:58:49.065 INFO:teuthology.orchestra.run.smithi063.stdout: "inode_count": 1, 2024-05-28T13:58:49.065 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_epoch": 8, 2024-05-28T13:58:49.065 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch": 21, 2024-05-28T13:58:49.065 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch_barrier": 0, 2024-05-28T13:58:49.065 INFO:teuthology.orchestra.run.smithi063.stdout: "blocklisted": false, 2024-05-28T13:58:49.065 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs" 2024-05-28T13:58:49.065 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-28T13:58:49.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:49.078 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-05-28T13:58:49.078 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-28T13:58:49.089 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-05-28T13:58:49.089 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-28T13:58:49.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:49.089 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-28T13:58:49.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:49.157 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-05-28T13:58:49.157 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.2 2024-05-28T13:58:49.214 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-05-28T13:58:49.214 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-05-28T13:58:49.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:49.214 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-28T13:58:49.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:49.282 DEBUG:teuthology.orchestra.run.smithi063:> stat --file-system '--printf=%T 2024-05-28T13:58:49.282 DEBUG:teuthology.orchestra.run.smithi063:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-28T13:58:49.291 INFO:teuthology.orchestra.run.smithi063.stdout:fuseblk 2024-05-28T13:58:49.292 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-28T13:58:49.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:49.292 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-28T13:58:49.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:49.354 DEBUG:teuthology.orchestra.run.smithi063:> stat --file-system '--printf=%T 2024-05-28T13:58:49.354 DEBUG:teuthology.orchestra.run.smithi063:> ' -- /home/ubuntu/cephtest/mnt.3 2024-05-28T13:58:49.407 INFO:teuthology.orchestra.run.smithi063.stdout:fuseblk 2024-05-28T13:58:49.407 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-05-28T13:58:49.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:58:49.407 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-28T13:58:49.470 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-28T13:58:49.487 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:49.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.653+0000 7f109b200640 1 Processor -- start 2024-05-28T13:58:49.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f109b200640 1 -- start start 2024-05-28T13:58:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f109b200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1094156a70 0x7f109406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f109b200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f10940afa50 0x7f10940d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f109b200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f109415cb90 con 0x7f1094156e50 2024-05-28T13:58:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f109b200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f109415ce50 con 0x7f1094156a70 2024-05-28T13:58:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f109b200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f109415d130 con 0x7f10940afa50 2024-05-28T13:58:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f1093e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1094156a70 0x7f109406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f1093e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1094156a70 0x7f109406dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38630/0 (socket says 172.21.15.42:38630) 2024-05-28T13:58:49.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f108b400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1094156e50 0x7f10940af340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50502/0 (socket says 172.21.15.42:50502) 2024-05-28T13:58:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f1093e00640 1 -- 172.21.15.42:0/4158644915 learned_addr learned my addr 172.21.15.42:0/4158644915 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f1099200640 1 --2- 172.21.15.42:0/4158644915 >> v2:172.21.15.63:3300/0 conn(0x7f10940afa50 0x7f10940d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f1093e00640 1 -- 172.21.15.42:0/4158644915 >> v2:172.21.15.63:3300/0 conn(0x7f10940afa50 msgr2=0x7f10940d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f1093e00640 1 --2- 172.21.15.42:0/4158644915 >> v2:172.21.15.63:3300/0 conn(0x7f10940afa50 0x7f10940d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f1093e00640 1 -- 172.21.15.42:0/4158644915 >> v1:172.21.15.42:6789/0 conn(0x7f1094156e50 legacy=0x7f10940af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f1093400640 1 -- 172.21.15.42:0/4158644915 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3766458625 0 0) 0x7f109415cb90 con 0x7f1094156e50 2024-05-28T13:58:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.657+0000 7f1093e00640 1 -- 172.21.15.42:0/4158644915 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109415d410 con 0x7f1094156a70 2024-05-28T13:58:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.661+0000 7f1093e00640 1 --2- 172.21.15.42:0/4158644915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1094156a70 0x7f109406dd60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f109406fea0 tx=0x7f108401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff08edbf4a6a141 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.661+0000 7f1093400640 1 -- 172.21.15.42:0/4158644915 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10840ab430 con 0x7f1094156a70 2024-05-28T13:58:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.661+0000 7f1093400640 1 -- 172.21.15.42:0/4158644915 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10840b0070 con 0x7f1094156a70 2024-05-28T13:58:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.661+0000 7f1093400640 1 -- 172.21.15.42:0/4158644915 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10840ab7f0 con 0x7f1094156a70 2024-05-28T13:58:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.661+0000 7f109b200640 1 -- 172.21.15.42:0/4158644915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1094156a70 msgr2=0x7f109406dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.661+0000 7f109b200640 1 --2- 172.21.15.42:0/4158644915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1094156a70 0x7f109406dd60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f109406fea0 tx=0x7f108401c520 comp rx=0 tx=0).stop 2024-05-28T13:58:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.661+0000 7f109b200640 1 -- 172.21.15.42:0/4158644915 shutdown_connections 2024-05-28T13:58:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.661+0000 7f109b200640 1 --2- 172.21.15.42:0/4158644915 >> v2:172.21.15.63:3300/0 conn(0x7f10940afa50 0x7f10940d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.661+0000 7f109b200640 1 --2- 172.21.15.42:0/4158644915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1094156a70 0x7f109406dd60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.661+0000 7f109b200640 1 -- 172.21.15.42:0/4158644915 >> 172.21.15.42:0/4158644915 conn(0x7f109406bdf0 msgr2=0x7f109406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.661+0000 7f109b200640 1 -- 172.21.15.42:0/4158644915 shutdown_connections 2024-05-28T13:58:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.661+0000 7f109b200640 1 -- 172.21.15.42:0/4158644915 wait complete. 2024-05-28T13:58:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.661+0000 7f109b200640 1 Processor -- start 2024-05-28T13:58:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.661+0000 7f109b200640 1 -- start start 2024-05-28T13:58:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f109b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10940afa50 0x7f10940cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f109b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1094156a70 0x7f1094155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f109b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094156e50 0x7f10940726e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f109b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1094160ed0 con 0x7f1094156a70 2024-05-28T13:58:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f109b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1094160d50 con 0x7f1094156e50 2024-05-28T13:58:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f109b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1094161050 con 0x7f10940afa50 2024-05-28T13:58:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f108b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1094156a70 0x7f1094155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f108b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1094156a70 0x7f1094155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38636/0 (socket says 172.21.15.42:38636) 2024-05-28T13:58:49.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f108b400640 1 -- 172.21.15.42:0/3969556946 learned_addr learned my addr 172.21.15.42:0/3969556946 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f1099200640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094156e50 0x7f10940726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f108b400640 1 -- 172.21.15.42:0/3969556946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10940afa50 msgr2=0x7f10940cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f1093e00640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10940afa50 0x7f10940cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f108b400640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10940afa50 0x7f10940cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f108b400640 1 -- 172.21.15.42:0/3969556946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094156e50 msgr2=0x7f10940726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f108b400640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094156e50 0x7f10940726e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f108b400640 1 -- 172.21.15.42:0/3969556946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1094073790 con 0x7f1094156a70 2024-05-28T13:58:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f1099200640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094156e50 0x7f10940726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f1093e00640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10940afa50 0x7f10940cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f108b400640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1094156a70 0x7f1094155090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f107c00c4b0 tx=0x7f107c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f1090c00640 1 -- 172.21.15.42:0/3969556946 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c018070 con 0x7f1094156a70 2024-05-28T13:58:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f1090c00640 1 -- 172.21.15.42:0/3969556946 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f107c007500 con 0x7f1094156a70 2024-05-28T13:58:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f1090c00640 1 -- 172.21.15.42:0/3969556946 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c004a30 con 0x7f1094156a70 2024-05-28T13:58:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f109b200640 1 -- 172.21.15.42:0/3969556946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1094073a70 con 0x7f1094156a70 2024-05-28T13:58:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f109b200640 1 -- 172.21.15.42:0/3969556946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10941555d0 con 0x7f1094156a70 2024-05-28T13:58:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.665+0000 7f109b200640 1 -- 172.21.15.42:0/3969556946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1094073c70 con 0x7f1094156a70 2024-05-28T13:58:49.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.669+0000 7f1090c00640 1 -- 172.21.15.42:0/3969556946 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f107c004bd0 con 0x7f1094156a70 2024-05-28T13:58:49.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.669+0000 7f1090c00640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1070068380 0x7f107006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:49.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.669+0000 7f1090c00640 1 -- 172.21.15.42:0/3969556946 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f107c085c70 con 0x7f1094156a70 2024-05-28T13:58:49.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.669+0000 7f1093e00640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1070068380 0x7f107006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:49.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.669+0000 7f1093e00640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1070068380 0x7f107006a840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10840af000 tx=0x7f1084008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:49.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.673+0000 7f1090c00640 1 -- 172.21.15.42:0/3969556946 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1094073c70 con 0x7f1094156a70 2024-05-28T13:58:49.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.841+0000 7f109b200640 1 -- 172.21.15.42:0/3969556946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1094068a90 con 0x7f1094156a70 2024-05-28T13:58:49.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.841+0000 7f1090c00640 1 -- 172.21.15.42:0/3969556946 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16545 (secure 0 0 0) 0x7f107c056190 con 0x7f1094156a70 2024-05-28T13:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:24.659233+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:49.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.845+0000 7f109b200640 1 -- 172.21.15.42:0/3969556946 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1070068380 msgr2=0x7f107006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:49.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.845+0000 7f109b200640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1070068380 0x7f107006a840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10840af000 tx=0x7f1084008040 comp rx=0 tx=0).stop 2024-05-28T13:58:49.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.845+0000 7f109b200640 1 -- 172.21.15.42:0/3969556946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1094156a70 msgr2=0x7f1094155090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:49.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.845+0000 7f109b200640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1094156a70 0x7f1094155090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f107c00c4b0 tx=0x7f107c00c970 comp rx=0 tx=0).stop 2024-05-28T13:58:49.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.845+0000 7f109b200640 1 -- 172.21.15.42:0/3969556946 shutdown_connections 2024-05-28T13:58:49.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.845+0000 7f109b200640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1070068380 0x7f107006a840 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:49.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.845+0000 7f109b200640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1094156e50 0x7f10940726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:49.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.845+0000 7f109b200640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1094156a70 0x7f1094155090 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:49.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.845+0000 7f109b200640 1 --2- 172.21.15.42:0/3969556946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10940afa50 0x7f10940cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:49.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.845+0000 7f109b200640 1 -- 172.21.15.42:0/3969556946 >> 172.21.15.42:0/3969556946 conn(0x7f109406bdf0 msgr2=0x7f10940b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:49.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.845+0000 7f109b200640 1 -- 172.21.15.42:0/3969556946 shutdown_connections 2024-05-28T13:58:49.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:49.845+0000 7f109b200640 1 -- 172.21.15.42:0/3969556946 wait complete. 2024-05-28T13:58:49.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:58:49.854 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:58:49.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a39400640 1 Processor -- start 2024-05-28T13:58:49.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a39400640 1 -- start start 2024-05-28T13:58:49.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a39400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6a3409b090 0x7f6a3409b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a39400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a34161850 0x7f6a34161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a39400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a3406c970 con 0x7f6a3409acb0 2024-05-28T13:58:50.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a39400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6a34162200 con 0x7f6a3409b090 2024-05-28T13:58:50.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a39400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6a340b51f0 con 0x7f6a34161850 2024-05-28T13:58:50.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a2b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6a3409b090 0x7f6a3409b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a33e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f6a3409acb0 0x7f6a340d0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50514/0 (socket says 172.21.15.42:50514) 2024-05-28T13:58:50.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a2b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6a3409b090 0x7f6a3409b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38640/0 (socket says 172.21.15.42:38640) 2024-05-28T13:58:50.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a33e00640 1 -- 172.21.15.42:0/1821489034 learned_addr learned my addr 172.21.15.42:0/1821489034 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:50.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a38a00640 1 --2- 172.21.15.42:0/1821489034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a34161850 0x7f6a34161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a2b400640 1 -- 172.21.15.42:0/1821489034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a34161850 msgr2=0x7f6a34161cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:50.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a2b400640 1 --2- 172.21.15.42:0/1821489034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a34161850 0x7f6a34161cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a2b400640 1 -- 172.21.15.42:0/1821489034 >> v1:172.21.15.42:6789/0 conn(0x7f6a3409acb0 legacy=0x7f6a340d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.013+0000 7f6a2b400640 1 -- 172.21.15.42:0/1821489034 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3406cc10 con 0x7f6a3409b090 2024-05-28T13:58:50.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.017+0000 7f6a2b400640 1 --2- 172.21.15.42:0/1821489034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3409b090 0x7f6a3409b960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0097c0 tx=0x7f6a1c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497a1e6dbae0261e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:50.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.017+0000 7f6a33400640 1 -- 172.21.15.42:0/1821489034 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0b0070 con 0x7f6a3409b090 2024-05-28T13:58:50.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.017+0000 7f6a33400640 1 -- 172.21.15.42:0/1821489034 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a1c0ab600 con 0x7f6a3409b090 2024-05-28T13:58:50.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.017+0000 7f6a33400640 1 -- 172.21.15.42:0/1821489034 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0ab9d0 con 0x7f6a3409b090 2024-05-28T13:58:50.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.017+0000 7f6a39400640 1 -- 172.21.15.42:0/1821489034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3409b090 msgr2=0x7f6a3409b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.017+0000 7f6a39400640 1 --2- 172.21.15.42:0/1821489034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3409b090 0x7f6a3409b960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0097c0 tx=0x7f6a1c01c380 comp rx=0 tx=0).stop 2024-05-28T13:58:50.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.017+0000 7f6a39400640 1 -- 172.21.15.42:0/1821489034 shutdown_connections 2024-05-28T13:58:50.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.017+0000 7f6a39400640 1 --2- 172.21.15.42:0/1821489034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a34161850 0x7f6a34161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.017+0000 7f6a39400640 1 --2- 172.21.15.42:0/1821489034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3409b090 0x7f6a3409b960 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.017+0000 7f6a39400640 1 -- 172.21.15.42:0/1821489034 >> 172.21.15.42:0/1821489034 conn(0x7f6a340b3e90 msgr2=0x7f6a340b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:50.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.017+0000 7f6a39400640 1 -- 172.21.15.42:0/1821489034 shutdown_connections 2024-05-28T13:58:50.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.017+0000 7f6a39400640 1 -- 172.21.15.42:0/1821489034 wait complete. 2024-05-28T13:58:50.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a39400640 1 Processor -- start 2024-05-28T13:58:50.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a39400640 1 -- start start 2024-05-28T13:58:50.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a39400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a3409acb0 0x7f6a340b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a39400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a3409b090 0x7f6a34150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a39400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a34161850 0x7f6a3415da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a39400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3406e590 con 0x7f6a34161850 2024-05-28T13:58:50.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a39400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3406e410 con 0x7f6a3409acb0 2024-05-28T13:58:50.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a39400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3406e710 con 0x7f6a3409b090 2024-05-28T13:58:50.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a38a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a34161850 0x7f6a3415da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a33e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a3409acb0 0x7f6a340b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a38a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a34161850 0x7f6a3415da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38652/0 (socket says 172.21.15.42:38652) 2024-05-28T13:58:50.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a38a00640 1 -- 172.21.15.42:0/3498749537 learned_addr learned my addr 172.21.15.42:0/3498749537 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:50.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a2b400640 1 --2- 172.21.15.42:0/3498749537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a3409b090 0x7f6a34150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a38a00640 1 -- 172.21.15.42:0/3498749537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a3409b090 msgr2=0x7f6a34150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a38a00640 1 --2- 172.21.15.42:0/3498749537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a3409b090 0x7f6a34150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a38a00640 1 -- 172.21.15.42:0/3498749537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a3409acb0 msgr2=0x7f6a340b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a38a00640 1 --2- 172.21.15.42:0/3498749537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a3409acb0 0x7f6a340b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a38a00640 1 -- 172.21.15.42:0/3498749537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a34150a90 con 0x7f6a34161850 2024-05-28T13:58:50.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a33e00640 1 --2- 172.21.15.42:0/3498749537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a3409acb0 0x7f6a340b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:50.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a38a00640 1 --2- 172.21.15.42:0/3498749537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a34161850 0x7f6a3415da40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00efc0 tx=0x7f6a2c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:50.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a30c00640 1 -- 172.21.15.42:0/3498749537 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c018020 con 0x7f6a34161850 2024-05-28T13:58:50.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a30c00640 1 -- 172.21.15.42:0/3498749537 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a2c014480 con 0x7f6a34161850 2024-05-28T13:58:50.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a39400640 1 -- 172.21.15.42:0/3498749537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a340b6b30 con 0x7f6a34161850 2024-05-28T13:58:50.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a30c00640 1 -- 172.21.15.42:0/3498749537 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c014870 con 0x7f6a34161850 2024-05-28T13:58:50.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.021+0000 7f6a39400640 1 -- 172.21.15.42:0/3498749537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a340b70f0 con 0x7f6a34161850 2024-05-28T13:58:50.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.025+0000 7f6a39400640 1 -- 172.21.15.42:0/3498749537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a3404fcf0 con 0x7f6a34161850 2024-05-28T13:58:50.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.025+0000 7f6a30c00640 1 -- 172.21.15.42:0/3498749537 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6a2c01b020 con 0x7f6a34161850 2024-05-28T13:58:50.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.025+0000 7f6a30c00640 1 --2- 172.21.15.42:0/3498749537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a10068380 0x7f6a1006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.025+0000 7f6a30c00640 1 -- 172.21.15.42:0/3498749537 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6a2c087670 con 0x7f6a34161850 2024-05-28T13:58:50.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.025+0000 7f6a33e00640 1 --2- 172.21.15.42:0/3498749537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a10068380 0x7f6a1006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.025+0000 7f6a33e00640 1 --2- 172.21.15.42:0/3498749537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a10068380 0x7f6a1006a840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a24000f70 tx=0x7f6a24007e90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:50.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.029+0000 7f6a30c00640 1 -- 172.21.15.42:0/3498749537 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6a2c022030 con 0x7f6a34161850 2024-05-28T13:58:50.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.197+0000 7f6a39400640 1 -- 172.21.15.42:0/3498749537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6a340d0dc0 con 0x7f6a34161850 2024-05-28T13:58:50.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.197+0000 7f6a30c00640 1 -- 172.21.15.42:0/3498749537 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f6a2c010040 con 0x7f6a34161850 2024-05-28T13:58:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:58:50.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.197+0000 7f6a39400640 1 -- 172.21.15.42:0/3498749537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a10068380 msgr2=0x7f6a1006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.197+0000 7f6a39400640 1 --2- 172.21.15.42:0/3498749537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a10068380 0x7f6a1006a840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a24000f70 tx=0x7f6a24007e90 comp rx=0 tx=0).stop 2024-05-28T13:58:50.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.197+0000 7f6a39400640 1 -- 172.21.15.42:0/3498749537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a34161850 msgr2=0x7f6a3415da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.197+0000 7f6a39400640 1 --2- 172.21.15.42:0/3498749537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a34161850 0x7f6a3415da40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00efc0 tx=0x7f6a2c00c490 comp rx=0 tx=0).stop 2024-05-28T13:58:50.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.201+0000 7f6a39400640 1 -- 172.21.15.42:0/3498749537 shutdown_connections 2024-05-28T13:58:50.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.201+0000 7f6a39400640 1 --2- 172.21.15.42:0/3498749537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a10068380 0x7f6a1006a840 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.201+0000 7f6a39400640 1 --2- 172.21.15.42:0/3498749537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a34161850 0x7f6a3415da40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.201+0000 7f6a39400640 1 --2- 172.21.15.42:0/3498749537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a3409b090 0x7f6a34150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.201+0000 7f6a39400640 1 --2- 172.21.15.42:0/3498749537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a3409acb0 0x7f6a340b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.201+0000 7f6a39400640 1 -- 172.21.15.42:0/3498749537 >> 172.21.15.42:0/3498749537 conn(0x7f6a340b3e90 msgr2=0x7f6a3415e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:50.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.201+0000 7f6a39400640 1 -- 172.21.15.42:0/3498749537 shutdown_connections 2024-05-28T13:58:50.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.201+0000 7f6a39400640 1 -- 172.21.15.42:0/3498749537 wait complete. 2024-05-28T13:58:50.205 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:58:50.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.361+0000 7fec5a600640 1 Processor -- start 2024-05-28T13:58:50.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.361+0000 7fec5a600640 1 -- start start 2024-05-28T13:58:50.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.365+0000 7fec5a600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fec5406ee20 0x7fec5406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.365+0000 7fec5a600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec540afe40 con 0x7fec54144f80 2024-05-28T13:58:50.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.365+0000 7fec5a600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec5415c600 con 0x7fec5415b9b0 2024-05-28T13:58:50.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.365+0000 7fec5a600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fec5406f7c0 con 0x7fec5406ee20 2024-05-28T13:58:50.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.365+0000 7fec4be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fec5406ee20 0x7fec5406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.365+0000 7fec53400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fec54144f80 0x7fec54072960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50520/0 (socket says 172.21.15.42:50520) 2024-05-28T13:58:50.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.365+0000 7fec53400640 1 -- 172.21.15.42:0/814447828 learned_addr learned my addr 172.21.15.42:0/814447828 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:50.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.365+0000 7fec4be00640 1 -- 172.21.15.42:0/814447828 >> v1:172.21.15.63:6789/0 conn(0x7fec5415b9b0 legacy=0x7fec5415be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.365+0000 7fec4be00640 1 -- 172.21.15.42:0/814447828 >> v1:172.21.15.42:6789/0 conn(0x7fec54144f80 legacy=0x7fec54072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.365+0000 7fec4be00640 1 -- 172.21.15.42:0/814447828 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5415c8e0 con 0x7fec5406ee20 2024-05-28T13:58:50.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.365+0000 7fec52a00640 1 -- 172.21.15.42:0/814447828 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2563225495 0 0) 0x7fec540afe40 con 0x7fec54144f80 2024-05-28T13:58:50.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec4be00640 1 --2- 172.21.15.42:0/814447828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec5406ee20 0x7fec5406f280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0097c0 tx=0x7fec3c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e996c69cad3b407a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:50.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec52a00640 1 -- 172.21.15.42:0/814447828 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0b0070 con 0x7fec5406ee20 2024-05-28T13:58:50.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec52a00640 1 -- 172.21.15.42:0/814447828 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3c0ab600 con 0x7fec5406ee20 2024-05-28T13:58:50.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec52a00640 1 -- 172.21.15.42:0/814447828 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0ab9d0 con 0x7fec5406ee20 2024-05-28T13:58:50.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 -- 172.21.15.42:0/814447828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec5406ee20 msgr2=0x7fec5406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 --2- 172.21.15.42:0/814447828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec5406ee20 0x7fec5406f280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0097c0 tx=0x7fec3c01c4a0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 -- 172.21.15.42:0/814447828 shutdown_connections 2024-05-28T13:58:50.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 --2- 172.21.15.42:0/814447828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec5406ee20 0x7fec5406f280 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 -- 172.21.15.42:0/814447828 >> 172.21.15.42:0/814447828 conn(0x7fec540b21f0 msgr2=0x7fec540b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:50.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 -- 172.21.15.42:0/814447828 shutdown_connections 2024-05-28T13:58:50.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 -- 172.21.15.42:0/814447828 wait complete. 2024-05-28T13:58:50.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 Processor -- start 2024-05-28T13:58:50.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 -- start start 2024-05-28T13:58:50.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec5406ee20 0x7fec5413daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec54144f80 0x7fec5413c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec5415b9b0 0x7fec5413f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fec540724d0 con 0x7fec5415b9b0 2024-05-28T13:58:50.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fec540b4260 con 0x7fec5406ee20 2024-05-28T13:58:50.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec5a600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fec54169140 con 0x7fec54144f80 2024-05-28T13:58:50.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec53e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec5415b9b0 0x7fec5413f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.369+0000 7fec53e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec5415b9b0 0x7fec5413f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38682/0 (socket says 172.21.15.42:38682) 2024-05-28T13:58:50.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec53e00640 1 -- 172.21.15.42:0/4146644430 learned_addr learned my addr 172.21.15.42:0/4146644430 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:50.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec53400640 1 --2- 172.21.15.42:0/4146644430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec5406ee20 0x7fec5413daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec4be00640 1 --2- 172.21.15.42:0/4146644430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec54144f80 0x7fec5413c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec53e00640 1 -- 172.21.15.42:0/4146644430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec54144f80 msgr2=0x7fec5413c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec53e00640 1 --2- 172.21.15.42:0/4146644430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec54144f80 0x7fec5413c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec53e00640 1 -- 172.21.15.42:0/4146644430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec5406ee20 msgr2=0x7fec5413daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec53e00640 1 --2- 172.21.15.42:0/4146644430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec5406ee20 0x7fec5413daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec53e00640 1 -- 172.21.15.42:0/4146644430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5413fd00 con 0x7fec5415b9b0 2024-05-28T13:58:50.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec53e00640 1 --2- 172.21.15.42:0/4146644430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec5415b9b0 0x7fec5413f490 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00c730 tx=0x7fec4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:50.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec4b400640 1 -- 172.21.15.42:0/4146644430 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c003b80 con 0x7fec5415b9b0 2024-05-28T13:58:50.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec4b400640 1 -- 172.21.15.42:0/4146644430 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec4c003d20 con 0x7fec5415b9b0 2024-05-28T13:58:50.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec4b400640 1 -- 172.21.15.42:0/4146644430 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c00aa30 con 0x7fec5415b9b0 2024-05-28T13:58:50.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec5a600640 1 -- 172.21.15.42:0/4146644430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5413ffe0 con 0x7fec5415b9b0 2024-05-28T13:58:50.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec5a600640 1 -- 172.21.15.42:0/4146644430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec541405a0 con 0x7fec5415b9b0 2024-05-28T13:58:50.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec5a600640 1 -- 172.21.15.42:0/4146644430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec540738b0 con 0x7fec5415b9b0 2024-05-28T13:58:50.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec4b400640 1 -- 172.21.15.42:0/4146644430 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fec4c00abd0 con 0x7fec5415b9b0 2024-05-28T13:58:50.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec4b400640 1 --2- 172.21.15.42:0/4146644430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fec300683b0 0x7fec3006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.373+0000 7fec4b400640 1 -- 172.21.15.42:0/4146644430 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fec4c086780 con 0x7fec5415b9b0 2024-05-28T13:58:50.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.377+0000 7fec53400640 1 --2- 172.21.15.42:0/4146644430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fec300683b0 0x7fec3006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.377+0000 7fec53400640 1 --2- 172.21.15.42:0/4146644430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fec300683b0 0x7fec3006a870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec44000f70 tx=0x7fec44007240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:50.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.377+0000 7fec4b400640 1 -- 172.21.15.42:0/4146644430 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fec4c056650 con 0x7fec5415b9b0 2024-05-28T13:58:50.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.545+0000 7fec5a600640 1 -- 172.21.15.42:0/4146644430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fec5404fdd0 con 0x7fec5415b9b0 2024-05-28T13:58:50.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.545+0000 7fec4b400640 1 -- 172.21.15.42:0/4146644430 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fec4c05a950 con 0x7fec5415b9b0 2024-05-28T13:58:50.529 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:58:50.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.545+0000 7fec5a600640 1 -- 172.21.15.42:0/4146644430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fec300683b0 msgr2=0x7fec3006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.545+0000 7fec5a600640 1 --2- 172.21.15.42:0/4146644430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fec300683b0 0x7fec3006a870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec44000f70 tx=0x7fec44007240 comp rx=0 tx=0).stop 2024-05-28T13:58:50.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.545+0000 7fec5a600640 1 -- 172.21.15.42:0/4146644430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec5415b9b0 msgr2=0x7fec5413f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.545+0000 7fec5a600640 1 --2- 172.21.15.42:0/4146644430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec5415b9b0 0x7fec5413f490 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00c730 tx=0x7fec4c00cbf0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.545+0000 7fec5a600640 1 -- 172.21.15.42:0/4146644430 shutdown_connections 2024-05-28T13:58:50.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.545+0000 7fec5a600640 1 --2- 172.21.15.42:0/4146644430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fec300683b0 0x7fec3006a870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.545+0000 7fec5a600640 1 --2- 172.21.15.42:0/4146644430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec5415b9b0 0x7fec5413f490 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.545+0000 7fec5a600640 1 --2- 172.21.15.42:0/4146644430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec54144f80 0x7fec5413c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.545+0000 7fec5a600640 1 --2- 172.21.15.42:0/4146644430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec5406ee20 0x7fec5413daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.545+0000 7fec5a600640 1 -- 172.21.15.42:0/4146644430 >> 172.21.15.42:0/4146644430 conn(0x7fec540b21f0 msgr2=0x7fec540b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:50.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.545+0000 7fec5a600640 1 -- 172.21.15.42:0/4146644430 shutdown_connections 2024-05-28T13:58:50.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.545+0000 7fec5a600640 1 -- 172.21.15.42:0/4146644430 wait complete. 2024-05-28T13:58:50.550 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:58:50.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3dbe00640 1 Processor -- start 2024-05-28T13:58:50.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3dbe00640 1 -- start start 2024-05-28T13:58:50.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3dbe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa3d40b2240 0x7fa3d406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3dbe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa3d406bf30 0x7fa3d40af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3dbe00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3d40b4440 con 0x7fa3d415e8c0 2024-05-28T13:58:50.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3dbe00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d40ae420 con 0x7fa3d40b2240 2024-05-28T13:58:50.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3dbe00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d406c370 con 0x7fa3d406bf30 2024-05-28T13:58:50.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3d8a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa3d40b2240 0x7fa3d406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3d8a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa3d40b2240 0x7fa3d406b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38690/0 (socket says 172.21.15.42:38690) 2024-05-28T13:58:50.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3d9400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa3d415e8c0 0x7fa3d40b0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50526/0 (socket says 172.21.15.42:50526) 2024-05-28T13:58:50.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3d8a00640 1 -- 172.21.15.42:0/2228798387 learned_addr learned my addr 172.21.15.42:0/2228798387 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:50.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3d9e00640 1 --2- 172.21.15.42:0/2228798387 >> v2:172.21.15.63:3300/0 conn(0x7fa3d406bf30 0x7fa3d40af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3d8a00640 1 -- 172.21.15.42:0/2228798387 >> v2:172.21.15.63:3300/0 conn(0x7fa3d406bf30 msgr2=0x7fa3d40af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:50.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3d8a00640 1 --2- 172.21.15.42:0/2228798387 >> v2:172.21.15.63:3300/0 conn(0x7fa3d406bf30 0x7fa3d40af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3d8a00640 1 -- 172.21.15.42:0/2228798387 >> v1:172.21.15.42:6789/0 conn(0x7fa3d415e8c0 legacy=0x7fa3d40b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.713+0000 7fa3d8a00640 1 -- 172.21.15.42:0/2228798387 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d40b0f10 con 0x7fa3d40b2240 2024-05-28T13:58:50.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.717+0000 7fa3d8a00640 1 --2- 172.21.15.42:0/2228798387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3d40b2240 0x7fa3d406b9f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa3d406eea0 tx=0x7fa3c001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593a26f28cc0875 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:50.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.717+0000 7fa3d3e00640 1 -- 172.21.15.42:0/2228798387 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00b2070 con 0x7fa3d40b2240 2024-05-28T13:58:50.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.717+0000 7fa3d3e00640 1 -- 172.21.15.42:0/2228798387 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c00ad680 con 0x7fa3d40b2240 2024-05-28T13:58:50.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.717+0000 7fa3d3e00640 1 -- 172.21.15.42:0/2228798387 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00ada50 con 0x7fa3d40b2240 2024-05-28T13:58:50.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.717+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2228798387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3d40b2240 msgr2=0x7fa3d406b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.717+0000 7fa3dbe00640 1 --2- 172.21.15.42:0/2228798387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3d40b2240 0x7fa3d406b9f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa3d406eea0 tx=0x7fa3c001c490 comp rx=0 tx=0).stop 2024-05-28T13:58:50.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.717+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2228798387 shutdown_connections 2024-05-28T13:58:50.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.717+0000 7fa3dbe00640 1 --2- 172.21.15.42:0/2228798387 >> v2:172.21.15.63:3300/0 conn(0x7fa3d406bf30 0x7fa3d40af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.717+0000 7fa3dbe00640 1 --2- 172.21.15.42:0/2228798387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3d40b2240 0x7fa3d406b9f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.717+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2228798387 >> 172.21.15.42:0/2228798387 conn(0x7fa3d40b6960 msgr2=0x7fa3d40b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:50.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.717+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2228798387 shutdown_connections 2024-05-28T13:58:50.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.717+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2228798387 wait complete. 2024-05-28T13:58:50.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3dbe00640 1 Processor -- start 2024-05-28T13:58:50.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3dbe00640 1 -- start start 2024-05-28T13:58:50.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3dbe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa3d406bf30 0x7fa3d4156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3dbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3d40b2240 0x7fa3d414bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3dbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3d415e8c0 0x7fa3d414d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3dbe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d415ee20 con 0x7fa3d40b2240 2024-05-28T13:58:50.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3dbe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d415eca0 con 0x7fa3d415e8c0 2024-05-28T13:58:50.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3dbe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d40afea0 con 0x7fa3d406bf30 2024-05-28T13:58:50.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d8a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3d40b2240 0x7fa3d414bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d8a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3d40b2240 0x7fa3d414bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38704/0 (socket says 172.21.15.42:38704) 2024-05-28T13:58:50.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d8a00640 1 -- 172.21.15.42:0/2206900455 learned_addr learned my addr 172.21.15.42:0/2206900455 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:50.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d9400640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa3d406bf30 0x7fa3d4156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d9e00640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3d415e8c0 0x7fa3d414d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d8a00640 1 -- 172.21.15.42:0/2206900455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa3d406bf30 msgr2=0x7fa3d4156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d8a00640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa3d406bf30 0x7fa3d4156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d8a00640 1 -- 172.21.15.42:0/2206900455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3d415e8c0 msgr2=0x7fa3d414d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d8a00640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3d415e8c0 0x7fa3d414d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d8a00640 1 -- 172.21.15.42:0/2206900455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d414c860 con 0x7fa3d40b2240 2024-05-28T13:58:50.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d9400640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa3d406bf30 0x7fa3d4156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:50.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d9e00640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3d415e8c0 0x7fa3d414d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:50.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d8a00640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3d40b2240 0x7fa3d414bfc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa3c001a0c0 tx=0x7fa3c001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:50.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d1600640 1 -- 172.21.15.42:0/2206900455 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00b2040 con 0x7fa3d40b2240 2024-05-28T13:58:50.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d1600640 1 -- 172.21.15.42:0/2206900455 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c00bd070 con 0x7fa3d40b2240 2024-05-28T13:58:50.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3d1600640 1 -- 172.21.15.42:0/2206900455 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00ade30 con 0x7fa3d40b2240 2024-05-28T13:58:50.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2206900455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d414cb40 con 0x7fa3d40b2240 2024-05-28T13:58:50.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2206900455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d414d100 con 0x7fa3d40b2240 2024-05-28T13:58:50.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.721+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2206900455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d409f7a0 con 0x7fa3d40b2240 2024-05-28T13:58:50.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.725+0000 7fa3d1600640 1 -- 172.21.15.42:0/2206900455 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa3c00c2460 con 0x7fa3d40b2240 2024-05-28T13:58:50.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.725+0000 7fa3d1600640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa3b4068380 0x7fa3b406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:50.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.725+0000 7fa3d1600640 1 -- 172.21.15.42:0/2206900455 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa3c0120ce0 con 0x7fa3d40b2240 2024-05-28T13:58:50.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.725+0000 7fa3d9400640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa3b4068380 0x7fa3b406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:50.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.725+0000 7fa3d9400640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa3b4068380 0x7fa3b406a840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4159530 tx=0x7fa3c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:50.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.729+0000 7fa3d1600640 1 -- 172.21.15.42:0/2206900455 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa3c0016110 con 0x7fa3d40b2240 2024-05-28T13:58:50.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.897+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2206900455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa3d404fdd0 con 0x7fa3d40b2240 2024-05-28T13:58:50.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.897+0000 7fa3d1600640 1 -- 172.21.15.42:0/2206900455 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fa3c00f1200 con 0x7fa3d40b2240 2024-05-28T13:58:50.881 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:58:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.897+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2206900455 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa3b4068380 msgr2=0x7fa3b406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.897+0000 7fa3dbe00640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa3b4068380 0x7fa3b406a840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4159530 tx=0x7fa3c8009040 comp rx=0 tx=0).stop 2024-05-28T13:58:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.897+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2206900455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3d40b2240 msgr2=0x7fa3d414bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.897+0000 7fa3dbe00640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3d40b2240 0x7fa3d414bfc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa3c001a0c0 tx=0x7fa3c001acd0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.897+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2206900455 shutdown_connections 2024-05-28T13:58:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.897+0000 7fa3dbe00640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa3b4068380 0x7fa3b406a840 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.897+0000 7fa3dbe00640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3d415e8c0 0x7fa3d414d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.897+0000 7fa3dbe00640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3d40b2240 0x7fa3d414bfc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.897+0000 7fa3dbe00640 1 --2- 172.21.15.42:0/2206900455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa3d406bf30 0x7fa3d4156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.897+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2206900455 >> 172.21.15.42:0/2206900455 conn(0x7fa3d40b6960 msgr2=0x7fa3d415d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.897+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2206900455 shutdown_connections 2024-05-28T13:58:50.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:50.897+0000 7fa3dbe00640 1 -- 172.21.15.42:0/2206900455 wait complete. 2024-05-28T13:58:50.903 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:51.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.057+0000 7f5e5f200640 1 Processor -- start 2024-05-28T13:58:51.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.061+0000 7f5e5f200640 1 -- start start 2024-05-28T13:58:51.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.061+0000 7f5e5f200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e600d0450 0x7f5e6009b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.061+0000 7f5e5f200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e600d0160 con 0x7f5e6015d680 2024-05-28T13:58:51.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.061+0000 7f5e5f200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e6009abf0 con 0x7f5e6015c880 2024-05-28T13:58:51.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.061+0000 7f5e5f200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5e6009b520 con 0x7f5e600d0450 2024-05-28T13:58:51.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.061+0000 7f5e5de00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e600d0450 0x7f5e6009b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.061+0000 7f5e5e800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f5e6015d680 0x7f5e600ccdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50532/0 (socket says 172.21.15.42:50532) 2024-05-28T13:58:51.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.061+0000 7f5e5e800640 1 -- 172.21.15.42:0/1541538476 learned_addr learned my addr 172.21.15.42:0/1541538476 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.061+0000 7f5e5de00640 1 -- 172.21.15.42:0/1541538476 >> v1:172.21.15.63:6789/0 conn(0x7f5e6015c880 legacy=0x7f5e600d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.061+0000 7f5e5de00640 1 -- 172.21.15.42:0/1541538476 >> v1:172.21.15.42:6789/0 conn(0x7f5e6015d680 legacy=0x7f5e600ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.061+0000 7f5e5d400640 1 -- 172.21.15.42:0/1541538476 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2483777128 0 0) 0x7f5e600d0160 con 0x7f5e6015d680 2024-05-28T13:58:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.061+0000 7f5e5de00640 1 -- 172.21.15.42:0/1541538476 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6009aed0 con 0x7f5e600d0450 2024-05-28T13:58:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.065+0000 7f5e5de00640 1 --2- 172.21.15.42:0/1541538476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e600d0450 0x7f5e6009b9b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5e500097c0 tx=0x7f5e5001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194cdfe0a6c935a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:51.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.065+0000 7f5e5d400640 1 -- 172.21.15.42:0/1541538476 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e500b0070 con 0x7f5e600d0450 2024-05-28T13:58:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.065+0000 7f5e5d400640 1 -- 172.21.15.42:0/1541538476 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e500ab600 con 0x7f5e600d0450 2024-05-28T13:58:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.065+0000 7f5e5d400640 1 -- 172.21.15.42:0/1541538476 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e500ab9d0 con 0x7f5e600d0450 2024-05-28T13:58:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.065+0000 7f5e5f200640 1 -- 172.21.15.42:0/1541538476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e600d0450 msgr2=0x7f5e6009b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.065+0000 7f5e5f200640 1 --2- 172.21.15.42:0/1541538476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e600d0450 0x7f5e6009b9b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5e500097c0 tx=0x7f5e5001c410 comp rx=0 tx=0).stop 2024-05-28T13:58:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.065+0000 7f5e5f200640 1 -- 172.21.15.42:0/1541538476 shutdown_connections 2024-05-28T13:58:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.065+0000 7f5e5f200640 1 --2- 172.21.15.42:0/1541538476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e600d0450 0x7f5e6009b9b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.065+0000 7f5e5f200640 1 -- 172.21.15.42:0/1541538476 >> 172.21.15.42:0/1541538476 conn(0x7f5e600b3dd0 msgr2=0x7f5e600b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.065+0000 7f5e5f200640 1 -- 172.21.15.42:0/1541538476 shutdown_connections 2024-05-28T13:58:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.065+0000 7f5e5f200640 1 -- 172.21.15.42:0/1541538476 wait complete. 2024-05-28T13:58:51.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.065+0000 7f5e5f200640 1 Processor -- start 2024-05-28T13:58:51.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.065+0000 7f5e5f200640 1 -- start start 2024-05-28T13:58:51.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e5f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e600d0450 0x7f5e600c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e5f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e6015c880 0x7f5e6015acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e5f200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e6015d680 0x7f5e6006d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e5f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6016a8a0 con 0x7f5e6015c880 2024-05-28T13:58:51.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e5f200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6016a720 con 0x7f5e600d0450 2024-05-28T13:58:51.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e5f200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5e6016aa20 con 0x7f5e6015d680 2024-05-28T13:58:51.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e57e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e6015c880 0x7f5e6015acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e57e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e6015c880 0x7f5e6015acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38732/0 (socket says 172.21.15.42:38732) 2024-05-28T13:58:51.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e57e00640 1 -- 172.21.15.42:0/4070368751 learned_addr learned my addr 172.21.15.42:0/4070368751 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:51.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e5e800640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e6015d680 0x7f5e6006d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e5de00640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e600d0450 0x7f5e600c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e57e00640 1 -- 172.21.15.42:0/4070368751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e6015d680 msgr2=0x7f5e6006d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e57e00640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e6015d680 0x7f5e6006d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e57e00640 1 -- 172.21.15.42:0/4070368751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e600d0450 msgr2=0x7f5e600c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e57e00640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e600d0450 0x7f5e600c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e57e00640 1 -- 172.21.15.42:0/4070368751 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e600ca180 con 0x7f5e6015c880 2024-05-28T13:58:51.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e5de00640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e600d0450 0x7f5e600c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:51.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e5e800640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e6015d680 0x7f5e6006d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:51.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e57e00640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e6015c880 0x7f5e6015acb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800c3c0 tx=0x7f5e4800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:51.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e56000640 1 -- 172.21.15.42:0/4070368751 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48017070 con 0x7f5e6015c880 2024-05-28T13:58:51.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e56000640 1 -- 172.21.15.42:0/4070368751 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e480052d0 con 0x7f5e6015c880 2024-05-28T13:58:51.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e56000640 1 -- 172.21.15.42:0/4070368751 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48003840 con 0x7f5e6015c880 2024-05-28T13:58:51.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e5f200640 1 -- 172.21.15.42:0/4070368751 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e600af050 con 0x7f5e6015c880 2024-05-28T13:58:51.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e5f200640 1 -- 172.21.15.42:0/4070368751 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e600af610 con 0x7f5e6015c880 2024-05-28T13:58:51.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e5f200640 1 -- 172.21.15.42:0/4070368751 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e6004fcf0 con 0x7f5e6015c880 2024-05-28T13:58:51.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.069+0000 7f5e56000640 1 -- 172.21.15.42:0/4070368751 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5e4801a070 con 0x7f5e6015c880 2024-05-28T13:58:51.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.073+0000 7f5e56000640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e3c0683b0 0x7f5e3c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.073+0000 7f5e56000640 1 -- 172.21.15.42:0/4070368751 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5e48085d40 con 0x7f5e6015c880 2024-05-28T13:58:51.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.073+0000 7f5e5de00640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e3c0683b0 0x7f5e3c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.073+0000 7f5e5de00640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e3c0683b0 0x7f5e3c06a870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5e500ad000 tx=0x7f5e500a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.077+0000 7f5e56000640 1 -- 172.21.15.42:0/4070368751 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5e48005480 con 0x7f5e6015c880 2024-05-28T13:58:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.241+0000 7f5e5f200640 1 -- 172.21.15.42:0/4070368751 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e600ce780 con 0x7f5e6015c880 2024-05-28T13:58:51.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.241+0000 7f5e56000640 1 -- 172.21.15.42:0/4070368751 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16545 (secure 0 0 0) 0x7f5e48056260 con 0x7f5e6015c880 2024-05-28T13:58:51.228 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:51.228 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:24.659233+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:51.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.245+0000 7f5e5f200640 1 -- 172.21.15.42:0/4070368751 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e3c0683b0 msgr2=0x7f5e3c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.245+0000 7f5e5f200640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e3c0683b0 0x7f5e3c06a870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5e500ad000 tx=0x7f5e500a6000 comp rx=0 tx=0).stop 2024-05-28T13:58:51.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.245+0000 7f5e5f200640 1 -- 172.21.15.42:0/4070368751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e6015c880 msgr2=0x7f5e6015acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.245+0000 7f5e5f200640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e6015c880 0x7f5e6015acb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800c3c0 tx=0x7f5e4800c880 comp rx=0 tx=0).stop 2024-05-28T13:58:51.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.245+0000 7f5e5f200640 1 -- 172.21.15.42:0/4070368751 shutdown_connections 2024-05-28T13:58:51.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.245+0000 7f5e5f200640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e3c0683b0 0x7f5e3c06a870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.245+0000 7f5e5f200640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e6015d680 0x7f5e6006d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.245+0000 7f5e5f200640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e6015c880 0x7f5e6015acb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.245+0000 7f5e5f200640 1 --2- 172.21.15.42:0/4070368751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e600d0450 0x7f5e600c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.245+0000 7f5e5f200640 1 -- 172.21.15.42:0/4070368751 >> 172.21.15.42:0/4070368751 conn(0x7f5e600b3dd0 msgr2=0x7f5e600716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:51.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.245+0000 7f5e5f200640 1 -- 172.21.15.42:0/4070368751 shutdown_connections 2024-05-28T13:58:51.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.245+0000 7f5e5f200640 1 -- 172.21.15.42:0/4070368751 wait complete. 2024-05-28T13:58:51.251 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:58:51.251 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:58:51.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.409+0000 7fe798200640 1 Processor -- start 2024-05-28T13:58:51.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.409+0000 7fe798200640 1 -- start start 2024-05-28T13:58:51.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.409+0000 7fe798200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe7900b02e0 0x7fe79006eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.409+0000 7fe798200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe79015fb60 con 0x7fe79006d110 2024-05-28T13:58:51.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.409+0000 7fe798200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe79006f190 con 0x7fe79015f710 2024-05-28T13:58:51.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.409+0000 7fe798200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe79006f470 con 0x7fe7900b02e0 2024-05-28T13:58:51.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.409+0000 7fe796200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe79006d110 0x7fe790154da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50548/0 (socket says 172.21.15.42:50548) 2024-05-28T13:58:51.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.409+0000 7fe796200640 1 -- 172.21.15.42:0/1781275925 learned_addr learned my addr 172.21.15.42:0/1781275925 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:51.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.409+0000 7fe795800640 1 --2- 172.21.15.42:0/1781275925 >> v2:172.21.15.63:3300/0 conn(0x7fe7900b02e0 0x7fe79006eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.409+0000 7fe78fe00640 1 -- 172.21.15.42:0/1781275925 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 148000198 0 0) 0x7fe7880014a0 con 0x7fe79006d110 2024-05-28T13:58:51.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.409+0000 7fe78fe00640 1 -- 172.21.15.42:0/1781275925 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe778003530 con 0x7fe79006d110 2024-05-28T13:58:51.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.413+0000 7fe795800640 1 -- 172.21.15.42:0/1781275925 >> v1:172.21.15.63:6789/0 conn(0x7fe79015f710 legacy=0x7fe79006ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.413+0000 7fe795800640 1 -- 172.21.15.42:0/1781275925 >> v1:172.21.15.42:6789/0 conn(0x7fe79006d110 legacy=0x7fe790154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.413+0000 7fe78fe00640 1 -- 172.21.15.42:0/1781275925 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2792451934 0 0) 0x7fe778003530 con 0x7fe79006d110 2024-05-28T13:58:51.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.413+0000 7fe795800640 1 -- 172.21.15.42:0/1781275925 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79006f750 con 0x7fe7900b02e0 2024-05-28T13:58:51.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.413+0000 7fe795800640 1 --2- 172.21.15.42:0/1781275925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7900b02e0 0x7fe79006eb00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe7840097c0 tx=0x7fe78401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c12121c1c7dd1d06 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:51.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.413+0000 7fe78fe00640 1 -- 172.21.15.42:0/1781275925 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7840b0070 con 0x7fe7900b02e0 2024-05-28T13:58:51.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.413+0000 7fe78fe00640 1 -- 172.21.15.42:0/1781275925 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7840ab400 con 0x7fe7900b02e0 2024-05-28T13:58:51.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.413+0000 7fe78fe00640 1 -- 172.21.15.42:0/1781275925 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788001bd0 con 0x7fe7900b02e0 2024-05-28T13:58:51.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.413+0000 7fe798200640 1 -- 172.21.15.42:0/1781275925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7900b02e0 msgr2=0x7fe79006eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.413+0000 7fe798200640 1 --2- 172.21.15.42:0/1781275925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7900b02e0 0x7fe79006eb00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe7840097c0 tx=0x7fe78401c390 comp rx=0 tx=0).stop 2024-05-28T13:58:51.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.413+0000 7fe798200640 1 -- 172.21.15.42:0/1781275925 shutdown_connections 2024-05-28T13:58:51.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.413+0000 7fe798200640 1 --2- 172.21.15.42:0/1781275925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7900b02e0 0x7fe79006eb00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.413+0000 7fe798200640 1 -- 172.21.15.42:0/1781275925 >> 172.21.15.42:0/1781275925 conn(0x7fe790072600 msgr2=0x7fe790072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:51.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.413+0000 7fe798200640 1 -- 172.21.15.42:0/1781275925 shutdown_connections 2024-05-28T13:58:51.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe798200640 1 -- 172.21.15.42:0/1781275925 wait complete. 2024-05-28T13:58:51.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe798200640 1 Processor -- start 2024-05-28T13:58:51.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe798200640 1 -- start start 2024-05-28T13:58:51.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe798200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe79006d110 0x7fe79015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe798200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7900b02e0 0x7fe7900aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe798200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe79015f710 0x7fe790150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe798200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe79016c9c0 con 0x7fe7900b02e0 2024-05-28T13:58:51.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe798200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe79016c840 con 0x7fe79006d110 2024-05-28T13:58:51.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe798200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe79016cb40 con 0x7fe79015f710 2024-05-28T13:58:51.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe794e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7900b02e0 0x7fe7900aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe794e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7900b02e0 0x7fe7900aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38746/0 (socket says 172.21.15.42:38746) 2024-05-28T13:58:51.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe794e00640 1 -- 172.21.15.42:0/1442738842 learned_addr learned my addr 172.21.15.42:0/1442738842 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:51.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe795800640 1 --2- 172.21.15.42:0/1442738842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe79006d110 0x7fe79015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe796200640 1 --2- 172.21.15.42:0/1442738842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe79015f710 0x7fe790150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe794e00640 1 -- 172.21.15.42:0/1442738842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe79015f710 msgr2=0x7fe790150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe794e00640 1 --2- 172.21.15.42:0/1442738842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe79015f710 0x7fe790150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe794e00640 1 -- 172.21.15.42:0/1442738842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe79006d110 msgr2=0x7fe79015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe794e00640 1 --2- 172.21.15.42:0/1442738842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe79006d110 0x7fe79015b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe794e00640 1 -- 172.21.15.42:0/1442738842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79015bc20 con 0x7fe7900b02e0 2024-05-28T13:58:51.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.417+0000 7fe794e00640 1 --2- 172.21.15.42:0/1442738842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7900b02e0 0x7fe7900aee80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00c3c0 tx=0x7fe77c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:51.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.421+0000 7fe78d600640 1 -- 172.21.15.42:0/1442738842 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c017070 con 0x7fe7900b02e0 2024-05-28T13:58:51.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.421+0000 7fe78d600640 1 -- 172.21.15.42:0/1442738842 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe77c0052d0 con 0x7fe7900b02e0 2024-05-28T13:58:51.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.421+0000 7fe78d600640 1 -- 172.21.15.42:0/1442738842 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c003840 con 0x7fe7900b02e0 2024-05-28T13:58:51.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.421+0000 7fe798200640 1 -- 172.21.15.42:0/1442738842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7900ce790 con 0x7fe7900b02e0 2024-05-28T13:58:51.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.421+0000 7fe798200640 1 -- 172.21.15.42:0/1442738842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7900ced50 con 0x7fe7900b02e0 2024-05-28T13:58:51.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.421+0000 7fe798200640 1 -- 172.21.15.42:0/1442738842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe790073f40 con 0x7fe7900b02e0 2024-05-28T13:58:51.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.421+0000 7fe78d600640 1 -- 172.21.15.42:0/1442738842 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe77c01a070 con 0x7fe7900b02e0 2024-05-28T13:58:51.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.421+0000 7fe78d600640 1 --2- 172.21.15.42:0/1442738842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe7700683b0 0x7fe77006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.421+0000 7fe78d600640 1 -- 172.21.15.42:0/1442738842 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe77c085d60 con 0x7fe7900b02e0 2024-05-28T13:58:51.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.421+0000 7fe795800640 1 --2- 172.21.15.42:0/1442738842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe7700683b0 0x7fe77006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.425+0000 7fe795800640 1 --2- 172.21.15.42:0/1442738842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe7700683b0 0x7fe77006a870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe7840ad000 tx=0x7fe7840a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:51.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.425+0000 7fe78d600640 1 -- 172.21.15.42:0/1442738842 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe77c005480 con 0x7fe7900b02e0 2024-05-28T13:58:51.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.589+0000 7fe798200640 1 -- 172.21.15.42:0/1442738842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe7900d0900 con 0x7fe7900b02e0 2024-05-28T13:58:51.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.589+0000 7fe78d600640 1 -- 172.21.15.42:0/1442738842 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fe77c055ba0 con 0x7fe7900b02e0 2024-05-28T13:58:51.576 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:58:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.593+0000 7fe798200640 1 -- 172.21.15.42:0/1442738842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe7700683b0 msgr2=0x7fe77006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.593+0000 7fe798200640 1 --2- 172.21.15.42:0/1442738842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe7700683b0 0x7fe77006a870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe7840ad000 tx=0x7fe7840a6000 comp rx=0 tx=0).stop 2024-05-28T13:58:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.593+0000 7fe798200640 1 -- 172.21.15.42:0/1442738842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7900b02e0 msgr2=0x7fe7900aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.593+0000 7fe798200640 1 --2- 172.21.15.42:0/1442738842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7900b02e0 0x7fe7900aee80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00c3c0 tx=0x7fe77c00c880 comp rx=0 tx=0).stop 2024-05-28T13:58:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.593+0000 7fe798200640 1 -- 172.21.15.42:0/1442738842 shutdown_connections 2024-05-28T13:58:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.593+0000 7fe798200640 1 --2- 172.21.15.42:0/1442738842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe7700683b0 0x7fe77006a870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.593+0000 7fe798200640 1 --2- 172.21.15.42:0/1442738842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe79015f710 0x7fe790150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.593+0000 7fe798200640 1 --2- 172.21.15.42:0/1442738842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7900b02e0 0x7fe7900aee80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.593+0000 7fe798200640 1 --2- 172.21.15.42:0/1442738842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe79006d110 0x7fe79015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.593+0000 7fe798200640 1 -- 172.21.15.42:0/1442738842 >> 172.21.15.42:0/1442738842 conn(0x7fe790072600 msgr2=0x7fe790161ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:51.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.593+0000 7fe798200640 1 -- 172.21.15.42:0/1442738842 shutdown_connections 2024-05-28T13:58:51.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.593+0000 7fe798200640 1 -- 172.21.15.42:0/1442738842 wait complete. 2024-05-28T13:58:51.597 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:58:51.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.757+0000 7fd28ae00640 1 Processor -- start 2024-05-28T13:58:51.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.757+0000 7fd28ae00640 1 -- start start 2024-05-28T13:58:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd28ae00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28409b160 0x7fd28406cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd28ae00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd28409b8e0 0x7fd28415e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd28ae00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2840d0770 con 0x7fd2840b0e80 2024-05-28T13:58:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd28ae00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd2840d0a10 con 0x7fd28409b8e0 2024-05-28T13:58:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd28ae00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2840d0cf0 con 0x7fd28409b160 2024-05-28T13:58:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd288e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd28409b8e0 0x7fd28415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd27b400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28409b160 0x7fd28406cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd288e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd28409b8e0 0x7fd28415e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38754/0 (socket says 172.21.15.42:38754) 2024-05-28T13:58:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd288e00640 1 -- 172.21.15.42:0/821139979 learned_addr learned my addr 172.21.15.42:0/821139979 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd283e00640 1 --1- 172.21.15.42:0/821139979 >> v1:172.21.15.63:6789/0 conn(0x7fd2840b0e80 0x7fd28409aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:54122/0 (socket says 172.21.15.42:54122) 2024-05-28T13:58:51.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd288e00640 1 -- 172.21.15.42:0/821139979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28409b160 msgr2=0x7fd28406cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:51.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd283400640 1 -- 172.21.15.42:0/821139979 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800223898 0 0) 0x7fd2840d0770 con 0x7fd2840b0e80 2024-05-28T13:58:51.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd288e00640 1 --2- 172.21.15.42:0/821139979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28409b160 0x7fd28406cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd288e00640 1 -- 172.21.15.42:0/821139979 >> v1:172.21.15.63:6789/0 conn(0x7fd2840b0e80 legacy=0x7fd28409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd288e00640 1 -- 172.21.15.42:0/821139979 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2840d0fd0 con 0x7fd28409b8e0 2024-05-28T13:58:51.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd288e00640 1 --2- 172.21.15.42:0/821139979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28409b8e0 0x7fd28415e110 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009870 tx=0x7fd27c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64d0079f9844fa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:51.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.761+0000 7fd283400640 1 -- 172.21.15.42:0/821139979 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0b2070 con 0x7fd28409b8e0 2024-05-28T13:58:51.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd283400640 1 -- 172.21.15.42:0/821139979 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd27c0a9070 con 0x7fd28409b8e0 2024-05-28T13:58:51.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd283400640 1 -- 172.21.15.42:0/821139979 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0ad880 con 0x7fd28409b8e0 2024-05-28T13:58:51.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 -- 172.21.15.42:0/821139979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28409b8e0 msgr2=0x7fd28415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 --2- 172.21.15.42:0/821139979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28409b8e0 0x7fd28415e110 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009870 tx=0x7fd27c01c560 comp rx=0 tx=0).stop 2024-05-28T13:58:51.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 -- 172.21.15.42:0/821139979 shutdown_connections 2024-05-28T13:58:51.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 --2- 172.21.15.42:0/821139979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28409b8e0 0x7fd28415e110 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 --2- 172.21.15.42:0/821139979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28409b160 0x7fd28406cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 -- 172.21.15.42:0/821139979 >> 172.21.15.42:0/821139979 conn(0x7fd2840b3f90 msgr2=0x7fd2840b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 -- 172.21.15.42:0/821139979 shutdown_connections 2024-05-28T13:58:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 -- 172.21.15.42:0/821139979 wait complete. 2024-05-28T13:58:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 Processor -- start 2024-05-28T13:58:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 -- start start 2024-05-28T13:58:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd28409b160 0x7fd284151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28409b8e0 0x7fd28415f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2840b0e80 0x7fd28415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd2840af350 con 0x7fd28409b8e0 2024-05-28T13:58:51.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2840af1d0 con 0x7fd28409b160 2024-05-28T13:58:51.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd28ae00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2840af4d0 con 0x7fd2840b0e80 2024-05-28T13:58:51.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd27b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28409b8e0 0x7fd28415f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd27b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28409b8e0 0x7fd28415f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38762/0 (socket says 172.21.15.42:38762) 2024-05-28T13:58:51.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd27b400640 1 -- 172.21.15.42:0/1811326442 learned_addr learned my addr 172.21.15.42:0/1811326442 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:51.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd288e00640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2840b0e80 0x7fd28415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.765+0000 7fd283e00640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd28409b160 0x7fd284151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd288e00640 1 -- 172.21.15.42:0/1811326442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd28409b160 msgr2=0x7fd284151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd288e00640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd28409b160 0x7fd284151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd288e00640 1 -- 172.21.15.42:0/1811326442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28409b8e0 msgr2=0x7fd28415f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd288e00640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28409b8e0 0x7fd28415f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd288e00640 1 -- 172.21.15.42:0/1811326442 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2840c7290 con 0x7fd2840b0e80 2024-05-28T13:58:51.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd27b400640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28409b8e0 0x7fd28415f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:51.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd288e00640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2840b0e80 0x7fd28415a070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0ac000 tx=0x7fd27c01ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd280c00640 1 -- 172.21.15.42:0/1811326442 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0b2040 con 0x7fd2840b0e80 2024-05-28T13:58:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd280c00640 1 -- 172.21.15.42:0/1811326442 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd27c0be070 con 0x7fd2840b0e80 2024-05-28T13:58:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd28ae00640 1 -- 172.21.15.42:0/1811326442 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd284160290 con 0x7fd2840b0e80 2024-05-28T13:58:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd280c00640 1 -- 172.21.15.42:0/1811326442 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0b8420 con 0x7fd2840b0e80 2024-05-28T13:58:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd28ae00640 1 -- 172.21.15.42:0/1811326442 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2840c7530 con 0x7fd2840b0e80 2024-05-28T13:58:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd28ae00640 1 -- 172.21.15.42:0/1811326442 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2840a0550 con 0x7fd2840b0e80 2024-05-28T13:58:51.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd280c00640 1 -- 172.21.15.42:0/1811326442 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd27c0ab070 con 0x7fd2840b0e80 2024-05-28T13:58:51.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.769+0000 7fd280c00640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd260068380 0x7fd26006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:51.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.773+0000 7fd280c00640 1 -- 172.21.15.42:0/1811326442 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd27c121220 con 0x7fd2840b0e80 2024-05-28T13:58:51.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.773+0000 7fd283e00640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd260068380 0x7fd26006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:51.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.773+0000 7fd283e00640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd260068380 0x7fd26006a840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd2740012f0 tx=0x7fd274001690 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:51.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.777+0000 7fd280c00640 1 -- 172.21.15.42:0/1811326442 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd27c016150 con 0x7fd2840b0e80 2024-05-28T13:58:51.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.945+0000 7fd28ae00640 1 -- 172.21.15.42:0/1811326442 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd28404fdd0 con 0x7fd2840b0e80 2024-05-28T13:58:51.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.949+0000 7fd280c00640 1 -- 172.21.15.42:0/1811326442 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fd27c0f1170 con 0x7fd2840b0e80 2024-05-28T13:58:51.933 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:58:51.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.949+0000 7fd28ae00640 1 -- 172.21.15.42:0/1811326442 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd260068380 msgr2=0x7fd26006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.949+0000 7fd28ae00640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd260068380 0x7fd26006a840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd2740012f0 tx=0x7fd274001690 comp rx=0 tx=0).stop 2024-05-28T13:58:51.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.949+0000 7fd28ae00640 1 -- 172.21.15.42:0/1811326442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2840b0e80 msgr2=0x7fd28415a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:51.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.949+0000 7fd28ae00640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2840b0e80 0x7fd28415a070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0ac000 tx=0x7fd27c01ada0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.949+0000 7fd28ae00640 1 -- 172.21.15.42:0/1811326442 shutdown_connections 2024-05-28T13:58:51.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.949+0000 7fd28ae00640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd260068380 0x7fd26006a840 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.949+0000 7fd28ae00640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2840b0e80 0x7fd28415a070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.949+0000 7fd28ae00640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28409b8e0 0x7fd28415f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.949+0000 7fd28ae00640 1 --2- 172.21.15.42:0/1811326442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd28409b160 0x7fd284151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:51.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.949+0000 7fd28ae00640 1 -- 172.21.15.42:0/1811326442 >> 172.21.15.42:0/1811326442 conn(0x7fd2840b3f90 msgr2=0x7fd28409aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:51.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.949+0000 7fd28ae00640 1 -- 172.21.15.42:0/1811326442 shutdown_connections 2024-05-28T13:58:51.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:51.949+0000 7fd28ae00640 1 -- 172.21.15.42:0/1811326442 wait complete. 2024-05-28T13:58:51.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:58:52.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.113+0000 7f2da1800640 1 Processor -- start 2024-05-28T13:58:52.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.113+0000 7f2da1800640 1 -- start start 2024-05-28T13:58:52.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.113+0000 7f2da1800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c06cfb0 0x7f2d9c15f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.113+0000 7f2da1800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2d9c0b32f0 0x7f2d9c0b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.113+0000 7f2da1800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d9c15fe30 con 0x7f2d9c09aa50 2024-05-28T13:58:52.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2da1800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2d9c0d0800 con 0x7f2d9c0b32f0 2024-05-28T13:58:52.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2da1800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2d9c06df20 con 0x7f2d9c06cfb0 2024-05-28T13:58:52.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d92000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f2d9c09aa50 0x7f2d9c06d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50550/0 (socket says 172.21.15.42:50550) 2024-05-28T13:58:52.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d92000640 1 -- 172.21.15.42:0/4293332842 learned_addr learned my addr 172.21.15.42:0/4293332842 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:52.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 -- 172.21.15.42:0/4293332842 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 85125067 0 0) 0x7f2d84001120 con 0x7f2d9c09aa50 2024-05-28T13:58:52.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 -- 172.21.15.42:0/4293332842 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d80003530 con 0x7f2d9c09aa50 2024-05-28T13:58:52.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9aa00640 1 --2- 172.21.15.42:0/4293332842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c06cfb0 0x7f2d9c15f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9b400640 1 --2- 172.21.15.42:0/4293332842 >> v2:172.21.15.63:3300/0 conn(0x7f2d9c0b32f0 0x7f2d9c0b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 -- 172.21.15.42:0/4293332842 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3729284864 0 0) 0x7f2d80003530 con 0x7f2d9c09aa50 2024-05-28T13:58:52.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 -- 172.21.15.42:0/4293332842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c06cfb0 msgr2=0x7f2d9c15f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:52.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 --2- 172.21.15.42:0/4293332842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c06cfb0 0x7f2d9c15f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 -- 172.21.15.42:0/4293332842 >> v2:172.21.15.63:3300/0 conn(0x7f2d9c0b32f0 msgr2=0x7f2d9c0b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:52.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 --2- 172.21.15.42:0/4293332842 >> v2:172.21.15.63:3300/0 conn(0x7f2d9c0b32f0 0x7f2d9c0b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 -- 172.21.15.42:0/4293332842 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9c09bb00 con 0x7f2d9c09aa50 2024-05-28T13:58:52.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 -- 172.21.15.42:0/4293332842 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f2d84001850 con 0x7f2d9c09aa50 2024-05-28T13:58:52.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9aa00640 1 --2- 172.21.15.42:0/4293332842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c06cfb0 0x7f2d9c15f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:52.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 -- 172.21.15.42:0/4293332842 >> v1:172.21.15.42:6789/0 conn(0x7f2d9c09aa50 legacy=0x7f2d9c06d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 --2- 172.21.15.42:0/4293332842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d800039f0 0x7f2d80003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 --2- 172.21.15.42:0/4293332842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c06cfb0 0x7f2d80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 --2- 172.21.15.42:0/4293332842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d80004d50 0x7f2d800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 -- 172.21.15.42:0/4293332842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c06df20 con 0x7f2d80004d50 2024-05-28T13:58:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 -- 172.21.15.42:0/4293332842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8000ba80 con 0x7f2d800039f0 2024-05-28T13:58:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9a000640 1 -- 172.21.15.42:0/4293332842 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8000bc00 con 0x7f2d9c06cfb0 2024-05-28T13:58:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9b400640 1 --2- 172.21.15.42:0/4293332842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d80004d50 0x7f2d800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d92000640 1 --2- 172.21.15.42:0/4293332842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c06cfb0 0x7f2d80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9aa00640 1 --2- 172.21.15.42:0/4293332842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d800039f0 0x7f2d80003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9b400640 1 -- 172.21.15.42:0/4293332842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c06cfb0 msgr2=0x7f2d80004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9b400640 1 --2- 172.21.15.42:0/4293332842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c06cfb0 0x7f2d80004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9b400640 1 -- 172.21.15.42:0/4293332842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d800039f0 msgr2=0x7f2d80003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9b400640 1 --2- 172.21.15.42:0/4293332842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d800039f0 0x7f2d80003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.117+0000 7f2d9b400640 1 -- 172.21.15.42:0/4293332842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d80003530 con 0x7f2d80004d50 2024-05-28T13:58:52.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.121+0000 7f2d9b400640 1 --2- 172.21.15.42:0/4293332842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d80004d50 0x7f2d800051d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400ba30 tx=0x7f2d9401c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1424d2dfaa8da96b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.121+0000 7f2d9a000640 1 -- 172.21.15.42:0/4293332842 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400b600 con 0x7f2d80004d50 2024-05-28T13:58:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.121+0000 7f2d9a000640 1 -- 172.21.15.42:0/4293332842 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d940aa650 con 0x7f2d80004d50 2024-05-28T13:58:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.121+0000 7f2d9a000640 1 -- 172.21.15.42:0/4293332842 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d940aaa40 con 0x7f2d80004d50 2024-05-28T13:58:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.121+0000 7f2da1800640 1 -- 172.21.15.42:0/4293332842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d80004d50 msgr2=0x7f2d800051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.121+0000 7f2da1800640 1 --2- 172.21.15.42:0/4293332842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d80004d50 0x7f2d800051d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400ba30 tx=0x7f2d9401c6e0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.121+0000 7f2d9b400640 1 -- 172.21.15.42:0/4293332842 reap_dead start 2024-05-28T13:58:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.121+0000 7f2da1800640 1 -- 172.21.15.42:0/4293332842 shutdown_connections 2024-05-28T13:58:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.121+0000 7f2da1800640 1 -- 172.21.15.42:0/4293332842 >> 172.21.15.42:0/4293332842 conn(0x7f2d9c0b40d0 msgr2=0x7f2d9c0b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.121+0000 7f2da1800640 1 -- 172.21.15.42:0/4293332842 shutdown_connections 2024-05-28T13:58:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.121+0000 7f2da1800640 1 -- 172.21.15.42:0/4293332842 wait complete. 2024-05-28T13:58:52.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.121+0000 7f2da1800640 1 Processor -- start 2024-05-28T13:58:52.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.121+0000 7f2da1800640 1 -- start start 2024-05-28T13:58:52.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2da1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d80004d50 0x7f2d9c157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2da1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d9c1f5c50 0x7f2d9c1617d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2da1800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c0b7650 0x7f2d9c0d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2da1800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c09b260 con 0x7f2d9c1f5c50 2024-05-28T13:58:52.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2da1800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c09af80 con 0x7f2d80004d50 2024-05-28T13:58:52.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2da1800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2d9c09b540 con 0x7f2d9c0b7650 2024-05-28T13:58:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d92000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d9c1f5c50 0x7f2d9c1617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d92000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d9c1f5c50 0x7f2d9c1617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38786/0 (socket says 172.21.15.42:38786) 2024-05-28T13:58:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d92000640 1 -- 172.21.15.42:0/2797449253 learned_addr learned my addr 172.21.15.42:0/2797449253 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d9b400640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c0b7650 0x7f2d9c0d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d9aa00640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d80004d50 0x7f2d9c157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d92000640 1 -- 172.21.15.42:0/2797449253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c0b7650 msgr2=0x7f2d9c0d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d92000640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c0b7650 0x7f2d9c0d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d92000640 1 -- 172.21.15.42:0/2797449253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d80004d50 msgr2=0x7f2d9c157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d92000640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d80004d50 0x7f2d9c157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d92000640 1 -- 172.21.15.42:0/2797449253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9c161d10 con 0x7f2d9c1f5c50 2024-05-28T13:58:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d9b400640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c0b7650 0x7f2d9c0d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d9aa00640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d80004d50 0x7f2d9c157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d92000640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d9c1f5c50 0x7f2d9c1617d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2d840019c0 tx=0x7f2d84007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:52.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d93400640 1 -- 172.21.15.42:0/2797449253 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400f940 con 0x7f2d9c1f5c50 2024-05-28T13:58:52.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d93400640 1 -- 172.21.15.42:0/2797449253 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d84018070 con 0x7f2d9c1f5c50 2024-05-28T13:58:52.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d93400640 1 -- 172.21.15.42:0/2797449253 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400fd20 con 0x7f2d9c1f5c50 2024-05-28T13:58:52.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2da1800640 1 -- 172.21.15.42:0/2797449253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9c0b7fa0 con 0x7f2d9c1f5c50 2024-05-28T13:58:52.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2da1800640 1 -- 172.21.15.42:0/2797449253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d9c0b5430 con 0x7f2d9c1f5c50 2024-05-28T13:58:52.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2da1800640 1 -- 172.21.15.42:0/2797449253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d9c0a06c0 con 0x7f2d9c1f5c50 2024-05-28T13:58:52.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.125+0000 7f2d93400640 1 -- 172.21.15.42:0/2797449253 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2d8401d460 con 0x7f2d9c1f5c50 2024-05-28T13:58:52.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.129+0000 7f2d93400640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2d780683b0 0x7f2d7806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.129+0000 7f2d93400640 1 -- 172.21.15.42:0/2797449253 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2d84086ae0 con 0x7f2d9c1f5c50 2024-05-28T13:58:52.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.129+0000 7f2d9aa00640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2d780683b0 0x7f2d7806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.129+0000 7f2d9aa00640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2d780683b0 0x7f2d7806a870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c007380 tx=0x7f2d8c00b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:52.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.133+0000 7f2d93400640 1 -- 172.21.15.42:0/2797449253 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2d9c161d10 con 0x7f2d9c1f5c50 2024-05-28T13:58:52.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.297+0000 7f2da1800640 1 -- 172.21.15.42:0/2797449253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2d9c163310 con 0x7f2d9c1f5c50 2024-05-28T13:58:52.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.297+0000 7f2d93400640 1 -- 172.21.15.42:0/2797449253 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f2d9c15fe30 con 0x7f2d9c1f5c50 2024-05-28T13:58:52.282 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:58:52.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.297+0000 7f2da1800640 1 -- 172.21.15.42:0/2797449253 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2d780683b0 msgr2=0x7f2d7806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.297+0000 7f2da1800640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2d780683b0 0x7f2d7806a870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c007380 tx=0x7f2d8c00b040 comp rx=0 tx=0).stop 2024-05-28T13:58:52.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.297+0000 7f2da1800640 1 -- 172.21.15.42:0/2797449253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d9c1f5c50 msgr2=0x7f2d9c1617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.297+0000 7f2da1800640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d9c1f5c50 0x7f2d9c1617d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2d840019c0 tx=0x7f2d84007720 comp rx=0 tx=0).stop 2024-05-28T13:58:52.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.297+0000 7f2da1800640 1 -- 172.21.15.42:0/2797449253 shutdown_connections 2024-05-28T13:58:52.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.297+0000 7f2da1800640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2d780683b0 0x7f2d7806a870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.297+0000 7f2da1800640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d9c0b7650 0x7f2d9c0d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.297+0000 7f2da1800640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d9c1f5c50 0x7f2d9c1617d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.297+0000 7f2da1800640 1 --2- 172.21.15.42:0/2797449253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d80004d50 0x7f2d9c157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.297+0000 7f2da1800640 1 -- 172.21.15.42:0/2797449253 >> 172.21.15.42:0/2797449253 conn(0x7f2d9c0b40d0 msgr2=0x7f2d9c15dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:52.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.297+0000 7f2da1800640 1 -- 172.21.15.42:0/2797449253 shutdown_connections 2024-05-28T13:58:52.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.297+0000 7f2da1800640 1 -- 172.21.15.42:0/2797449253 wait complete. 2024-05-28T13:58:52.304 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:52.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.465+0000 7f3d3dc00640 1 Processor -- start 2024-05-28T13:58:52.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.465+0000 7f3d3dc00640 1 -- start start 2024-05-28T13:58:52.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d3dc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3806f660 0x7f3d38073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d3dc00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3d380d0e00 0x7f3d3806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d3dc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d3815f280 con 0x7f3d3815ba40 2024-05-28T13:58:52.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d3dc00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3d3815f030 con 0x7f3d380d0e00 2024-05-28T13:58:52.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d3dc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3815f560 con 0x7f3d3806f660 2024-05-28T13:58:52.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d36a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3806f660 0x7f3d38073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d37400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3d380d0e00 0x7f3d3806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d36a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3806f660 0x7f3d38073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:45840/0 (socket says 172.21.15.42:45840) 2024-05-28T13:58:52.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d36a00640 1 -- 172.21.15.42:0/2608444062 learned_addr learned my addr 172.21.15.42:0/2608444062 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:52.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d36a00640 1 -- 172.21.15.42:0/2608444062 >> v2:172.21.15.63:3300/0 conn(0x7f3d380d0e00 msgr2=0x7f3d3806dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:52.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d36000640 1 -- 172.21.15.42:0/2608444062 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2038238502 0 0) 0x7f3d3815f280 con 0x7f3d3815ba40 2024-05-28T13:58:52.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d36a00640 1 --2- 172.21.15.42:0/2608444062 >> v2:172.21.15.63:3300/0 conn(0x7f3d380d0e00 0x7f3d3806dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d36a00640 1 -- 172.21.15.42:0/2608444062 >> v1:172.21.15.63:6789/0 conn(0x7f3d3815ba40 legacy=0x7f3d380d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d36a00640 1 -- 172.21.15.42:0/2608444062 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3815f840 con 0x7f3d3806f660 2024-05-28T13:58:52.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.469+0000 7f3d37400640 1 --2- 172.21.15.42:0/2608444062 >> v2:172.21.15.63:3300/0 conn(0x7f3d380d0e00 0x7f3d3806dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:52.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.473+0000 7f3d36a00640 1 --2- 172.21.15.42:0/2608444062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3806f660 0x7f3d38073980 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800bb10 tx=0x7f3d2801eae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=524a53f4d273fed4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:52.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.473+0000 7f3d36000640 1 -- 172.21.15.42:0/2608444062 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d280b4070 con 0x7f3d3806f660 2024-05-28T13:58:52.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.473+0000 7f3d36000640 1 -- 172.21.15.42:0/2608444062 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d280a7b90 con 0x7f3d3806f660 2024-05-28T13:58:52.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.473+0000 7f3d36000640 1 -- 172.21.15.42:0/2608444062 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d280af440 con 0x7f3d3806f660 2024-05-28T13:58:52.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.473+0000 7f3d3dc00640 1 -- 172.21.15.42:0/2608444062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3806f660 msgr2=0x7f3d38073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.473+0000 7f3d3dc00640 1 --2- 172.21.15.42:0/2608444062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3806f660 0x7f3d38073980 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800bb10 tx=0x7f3d2801eae0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.473+0000 7f3d3dc00640 1 -- 172.21.15.42:0/2608444062 shutdown_connections 2024-05-28T13:58:52.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.473+0000 7f3d3dc00640 1 --2- 172.21.15.42:0/2608444062 >> v2:172.21.15.63:3300/0 conn(0x7f3d380d0e00 0x7f3d3806dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.473+0000 7f3d3dc00640 1 --2- 172.21.15.42:0/2608444062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3806f660 0x7f3d38073980 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.473+0000 7f3d3dc00640 1 -- 172.21.15.42:0/2608444062 >> 172.21.15.42:0/2608444062 conn(0x7f3d3806bdf0 msgr2=0x7f3d3806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:52.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.473+0000 7f3d3dc00640 1 -- 172.21.15.42:0/2608444062 shutdown_connections 2024-05-28T13:58:52.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.473+0000 7f3d3dc00640 1 -- 172.21.15.42:0/2608444062 wait complete. 2024-05-28T13:58:52.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.473+0000 7f3d3dc00640 1 Processor -- start 2024-05-28T13:58:52.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.473+0000 7f3d3dc00640 1 -- start start 2024-05-28T13:58:52.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d3dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3806f660 0x7f3d380cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d3dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d380d0e00 0x7f3d380995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d3dc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3814dd40 0x7f3d380b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d3dc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3d380b3460 con 0x7f3d3806f660 2024-05-28T13:58:52.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d3dc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d380b32e0 con 0x7f3d380d0e00 2024-05-28T13:58:52.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d3dc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d380b35e0 con 0x7f3d3814dd40 2024-05-28T13:58:52.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d36a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3806f660 0x7f3d380cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d36a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3806f660 0x7f3d380cf8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38794/0 (socket says 172.21.15.42:38794) 2024-05-28T13:58:52.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d36a00640 1 -- 172.21.15.42:0/728379231 learned_addr learned my addr 172.21.15.42:0/728379231 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:52.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d36a00640 1 -- 172.21.15.42:0/728379231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3814dd40 msgr2=0x7f3d380b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d36a00640 1 --2- 172.21.15.42:0/728379231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3814dd40 0x7f3d380b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d36a00640 1 -- 172.21.15.42:0/728379231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d380d0e00 msgr2=0x7f3d380995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d2fe00640 1 --2- 172.21.15.42:0/728379231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d380d0e00 0x7f3d380995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d36a00640 1 --2- 172.21.15.42:0/728379231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d380d0e00 0x7f3d380995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d36a00640 1 -- 172.21.15.42:0/728379231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d381f7ed0 con 0x7f3d3806f660 2024-05-28T13:58:52.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d2fe00640 1 --2- 172.21.15.42:0/728379231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d380d0e00 0x7f3d380995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:52.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d36a00640 1 --2- 172.21.15.42:0/728379231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3806f660 0x7f3d380cf8e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800b2c0 tx=0x7f3d2801d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:52.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d2ea00640 1 -- 172.21.15.42:0/728379231 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d280bf070 con 0x7f3d3806f660 2024-05-28T13:58:52.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d2ea00640 1 -- 172.21.15.42:0/728379231 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d280afb50 con 0x7f3d3806f660 2024-05-28T13:58:52.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d2ea00640 1 -- 172.21.15.42:0/728379231 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d280b4040 con 0x7f3d3806f660 2024-05-28T13:58:52.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d3dc00640 1 -- 172.21.15.42:0/728379231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d381f81b0 con 0x7f3d3806f660 2024-05-28T13:58:52.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d3dc00640 1 -- 172.21.15.42:0/728379231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d381f8770 con 0x7f3d3806f660 2024-05-28T13:58:52.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d3dc00640 1 -- 172.21.15.42:0/728379231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d381f8f90 con 0x7f3d3806f660 2024-05-28T13:58:52.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.477+0000 7f3d2ea00640 1 -- 172.21.15.42:0/728379231 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3d28004030 con 0x7f3d3806f660 2024-05-28T13:58:52.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.481+0000 7f3d2ea00640 1 --2- 172.21.15.42:0/728379231 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d140683b0 0x7f3d1406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.481+0000 7f3d2ea00640 1 -- 172.21.15.42:0/728379231 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3d28120d70 con 0x7f3d3806f660 2024-05-28T13:58:52.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.481+0000 7f3d2fe00640 1 --2- 172.21.15.42:0/728379231 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d140683b0 0x7f3d1406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.481+0000 7f3d2fe00640 1 --2- 172.21.15.42:0/728379231 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d140683b0 0x7f3d1406a870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3d38148850 tx=0x7f3d20009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:52.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.485+0000 7f3d2ea00640 1 -- 172.21.15.42:0/728379231 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3d28018820 con 0x7f3d3806f660 2024-05-28T13:58:52.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.653+0000 7f3d3dc00640 1 -- 172.21.15.42:0/728379231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d38072bb0 con 0x7f3d3806f660 2024-05-28T13:58:52.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.653+0000 7f3d2ea00640 1 -- 172.21.15.42:0/728379231 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16545 (secure 0 0 0) 0x7f3d280f1330 con 0x7f3d3806f660 2024-05-28T13:58:52.638 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:52.639 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:24.659233+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:52.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.653+0000 7f3d3dc00640 1 -- 172.21.15.42:0/728379231 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d140683b0 msgr2=0x7f3d1406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.653+0000 7f3d3dc00640 1 --2- 172.21.15.42:0/728379231 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d140683b0 0x7f3d1406a870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3d38148850 tx=0x7f3d20009040 comp rx=0 tx=0).stop 2024-05-28T13:58:52.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.653+0000 7f3d3dc00640 1 -- 172.21.15.42:0/728379231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3806f660 msgr2=0x7f3d380cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.653+0000 7f3d3dc00640 1 --2- 172.21.15.42:0/728379231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3806f660 0x7f3d380cf8e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800b2c0 tx=0x7f3d2801d320 comp rx=0 tx=0).stop 2024-05-28T13:58:52.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.653+0000 7f3d3dc00640 1 -- 172.21.15.42:0/728379231 shutdown_connections 2024-05-28T13:58:52.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.653+0000 7f3d3dc00640 1 --2- 172.21.15.42:0/728379231 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d140683b0 0x7f3d1406a870 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.653+0000 7f3d3dc00640 1 --2- 172.21.15.42:0/728379231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3814dd40 0x7f3d380b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.653+0000 7f3d3dc00640 1 --2- 172.21.15.42:0/728379231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d380d0e00 0x7f3d380995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.653+0000 7f3d3dc00640 1 --2- 172.21.15.42:0/728379231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3806f660 0x7f3d380cf8e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.653+0000 7f3d3dc00640 1 -- 172.21.15.42:0/728379231 >> 172.21.15.42:0/728379231 conn(0x7f3d3806bdf0 msgr2=0x7f3d380b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:52.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.657+0000 7f3d3dc00640 1 -- 172.21.15.42:0/728379231 shutdown_connections 2024-05-28T13:58:52.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.657+0000 7f3d3dc00640 1 -- 172.21.15.42:0/728379231 wait complete. 2024-05-28T13:58:52.660 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:58:52.660 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:58:52.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.817+0000 7fe56ec00640 1 Processor -- start 2024-05-28T13:58:52.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.817+0000 7fe56ec00640 1 -- start start 2024-05-28T13:58:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.821+0000 7fe56ec00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe56806d110 0x7fe56806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.821+0000 7fe56ec00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe56815fb20 con 0x7fe56815f710 2024-05-28T13:58:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.821+0000 7fe56ec00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe56806f190 con 0x7fe5680b02e0 2024-05-28T13:58:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.821+0000 7fe56ec00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe56806f470 con 0x7fe56806d110 2024-05-28T13:58:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.821+0000 7fe56cc00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe56815f710 0x7fe568154da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50558/0 (socket says 172.21.15.42:50558) 2024-05-28T13:58:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.821+0000 7fe56cc00640 1 -- 172.21.15.42:0/2608595138 learned_addr learned my addr 172.21.15.42:0/2608595138 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:52.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.821+0000 7fe55f400640 1 --2- 172.21.15.42:0/2608595138 >> v2:172.21.15.63:3300/0 conn(0x7fe56806d110 0x7fe56806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.821+0000 7fe567400640 1 -- 172.21.15.42:0/2608595138 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 267683754 0 0) 0x7fe56815fb20 con 0x7fe56815f710 2024-05-28T13:58:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.821+0000 7fe567400640 1 -- 172.21.15.42:0/2608595138 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe54c003530 con 0x7fe56815f710 2024-05-28T13:58:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.821+0000 7fe567400640 1 -- 172.21.15.42:0/2608595138 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 572881675 0 0) 0x7fe56806f190 con 0x7fe5680b02e0 2024-05-28T13:58:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.821+0000 7fe567400640 1 -- 172.21.15.42:0/2608595138 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe56815fb20 con 0x7fe5680b02e0 2024-05-28T13:58:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.821+0000 7fe55f400640 1 -- 172.21.15.42:0/2608595138 >> v1:172.21.15.63:6789/0 conn(0x7fe5680b02e0 legacy=0x7fe56806ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.821+0000 7fe55f400640 1 -- 172.21.15.42:0/2608595138 >> v1:172.21.15.42:6789/0 conn(0x7fe56815f710 legacy=0x7fe568154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.821+0000 7fe55f400640 1 -- 172.21.15.42:0/2608595138 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56806f750 con 0x7fe56806d110 2024-05-28T13:58:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.825+0000 7fe55f400640 1 --2- 172.21.15.42:0/2608595138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe56806d110 0x7fe56806eb00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe5500097f0 tx=0x7fe55001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5efdfb836d20668c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.825+0000 7fe567400640 1 -- 172.21.15.42:0/2608595138 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5500b1070 con 0x7fe56806d110 2024-05-28T13:58:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.825+0000 7fe567400640 1 -- 172.21.15.42:0/2608595138 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5500ac600 con 0x7fe56806d110 2024-05-28T13:58:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.825+0000 7fe567400640 1 -- 172.21.15.42:0/2608595138 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5500ac9d0 con 0x7fe56806d110 2024-05-28T13:58:52.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.825+0000 7fe56ec00640 1 -- 172.21.15.42:0/2608595138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe56806d110 msgr2=0x7fe56806eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.825+0000 7fe56ec00640 1 --2- 172.21.15.42:0/2608595138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe56806d110 0x7fe56806eb00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe5500097f0 tx=0x7fe55001c470 comp rx=0 tx=0).stop 2024-05-28T13:58:52.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.825+0000 7fe56ec00640 1 -- 172.21.15.42:0/2608595138 shutdown_connections 2024-05-28T13:58:52.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.825+0000 7fe56ec00640 1 --2- 172.21.15.42:0/2608595138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe56806d110 0x7fe56806eb00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.825+0000 7fe56ec00640 1 -- 172.21.15.42:0/2608595138 >> 172.21.15.42:0/2608595138 conn(0x7fe568072600 msgr2=0x7fe568072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:52.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.825+0000 7fe56ec00640 1 -- 172.21.15.42:0/2608595138 shutdown_connections 2024-05-28T13:58:52.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.825+0000 7fe56ec00640 1 -- 172.21.15.42:0/2608595138 wait complete. 2024-05-28T13:58:52.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.825+0000 7fe56ec00640 1 Processor -- start 2024-05-28T13:58:52.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.825+0000 7fe56ec00640 1 -- start start 2024-05-28T13:58:52.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe56ec00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe56806d110 0x7fe56814e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe56ec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b02e0 0x7fe568153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe56ec00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe56815f710 0x7fe568153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe56ec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe56816c9c0 con 0x7fe5680b02e0 2024-05-28T13:58:52.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe56ec00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe56816c840 con 0x7fe56815f710 2024-05-28T13:58:52.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe56ec00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe56816cb40 con 0x7fe56806d110 2024-05-28T13:58:52.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe55f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b02e0 0x7fe568153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe55f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b02e0 0x7fe568153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38810/0 (socket says 172.21.15.42:38810) 2024-05-28T13:58:52.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe55f400640 1 -- 172.21.15.42:0/2461902433 learned_addr learned my addr 172.21.15.42:0/2461902433 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:52.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe567e00640 1 --2- 172.21.15.42:0/2461902433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe56806d110 0x7fe56814e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe56cc00640 1 --2- 172.21.15.42:0/2461902433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe56815f710 0x7fe568153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe55f400640 1 -- 172.21.15.42:0/2461902433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe56806d110 msgr2=0x7fe56814e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe55f400640 1 --2- 172.21.15.42:0/2461902433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe56806d110 0x7fe56814e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe55f400640 1 -- 172.21.15.42:0/2461902433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe56815f710 msgr2=0x7fe568153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe55f400640 1 --2- 172.21.15.42:0/2461902433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe56815f710 0x7fe568153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe55f400640 1 -- 172.21.15.42:0/2461902433 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5681543f0 con 0x7fe5680b02e0 2024-05-28T13:58:52.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe55f400640 1 --2- 172.21.15.42:0/2461902433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b02e0 0x7fe568153770 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe55001a0a0 tx=0x7fe55001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:52.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe564c00640 1 -- 172.21.15.42:0/2461902433 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5500b1040 con 0x7fe5680b02e0 2024-05-28T13:58:52.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe564c00640 1 -- 172.21.15.42:0/2461902433 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5500bc070 con 0x7fe5680b02e0 2024-05-28T13:58:52.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe56ec00640 1 -- 172.21.15.42:0/2461902433 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5681546d0 con 0x7fe5680b02e0 2024-05-28T13:58:52.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe564c00640 1 -- 172.21.15.42:0/2461902433 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550006e20 con 0x7fe5680b02e0 2024-05-28T13:58:52.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe56ec00640 1 -- 172.21.15.42:0/2461902433 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe568154c90 con 0x7fe5680b02e0 2024-05-28T13:58:52.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.829+0000 7fe56ec00640 1 -- 172.21.15.42:0/2461902433 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56816a400 con 0x7fe5680b02e0 2024-05-28T13:58:52.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.833+0000 7fe564c00640 1 -- 172.21.15.42:0/2461902433 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe5500b76b0 con 0x7fe5680b02e0 2024-05-28T13:58:52.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.833+0000 7fe564c00640 1 --2- 172.21.15.42:0/2461902433 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe5440683b0 0x7fe54406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:52.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.833+0000 7fe564c00640 1 -- 172.21.15.42:0/2461902433 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe550120cf0 con 0x7fe5680b02e0 2024-05-28T13:58:52.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.833+0000 7fe567e00640 1 --2- 172.21.15.42:0/2461902433 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe5440683b0 0x7fe54406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:52.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.833+0000 7fe567e00640 1 --2- 172.21.15.42:0/2461902433 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe5440683b0 0x7fe54406a870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe5580011f0 tx=0x7fe558009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:52.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:52.837+0000 7fe564c00640 1 -- 172.21.15.42:0/2461902433 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe550016090 con 0x7fe5680b02e0 2024-05-28T13:58:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.005+0000 7fe56ec00640 1 -- 172.21.15.42:0/2461902433 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe56815e640 con 0x7fe5680b02e0 2024-05-28T13:58:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.005+0000 7fe564c00640 1 -- 172.21.15.42:0/2461902433 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fe5500f1210 con 0x7fe5680b02e0 2024-05-28T13:58:52.991 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:58:52.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.009+0000 7fe56ec00640 1 -- 172.21.15.42:0/2461902433 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe5440683b0 msgr2=0x7fe54406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.009+0000 7fe56ec00640 1 --2- 172.21.15.42:0/2461902433 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe5440683b0 0x7fe54406a870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe5580011f0 tx=0x7fe558009040 comp rx=0 tx=0).stop 2024-05-28T13:58:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.009+0000 7fe56ec00640 1 -- 172.21.15.42:0/2461902433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b02e0 msgr2=0x7fe568153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.009+0000 7fe56ec00640 1 --2- 172.21.15.42:0/2461902433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b02e0 0x7fe568153770 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe55001a0a0 tx=0x7fe55001acb0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.009+0000 7fe56ec00640 1 -- 172.21.15.42:0/2461902433 shutdown_connections 2024-05-28T13:58:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.009+0000 7fe56ec00640 1 --2- 172.21.15.42:0/2461902433 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe5440683b0 0x7fe54406a870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.009+0000 7fe56ec00640 1 --2- 172.21.15.42:0/2461902433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe56815f710 0x7fe568153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.009+0000 7fe56ec00640 1 --2- 172.21.15.42:0/2461902433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b02e0 0x7fe568153770 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.009+0000 7fe56ec00640 1 --2- 172.21.15.42:0/2461902433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe56806d110 0x7fe56814e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.009+0000 7fe56ec00640 1 -- 172.21.15.42:0/2461902433 >> 172.21.15.42:0/2461902433 conn(0x7fe568072600 msgr2=0x7fe56806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.009+0000 7fe56ec00640 1 -- 172.21.15.42:0/2461902433 shutdown_connections 2024-05-28T13:58:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.009+0000 7fe56ec00640 1 -- 172.21.15.42:0/2461902433 wait complete. 2024-05-28T13:58:53.013 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:58:53.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26ff000640 1 Processor -- start 2024-05-28T13:58:53.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26ff000640 1 -- start start 2024-05-28T13:58:53.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26ff000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f26f806ee20 0x7f26f806f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26ff000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f26f815b9b0 0x7f26f815be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26ff000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26f80afe40 con 0x7f26f8144f80 2024-05-28T13:58:53.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26ff000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f26f806f7c0 con 0x7f26f815b9b0 2024-05-28T13:58:53.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26ff000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f26f80724d0 con 0x7f26f806ee20 2024-05-28T13:58:53.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26fd000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f26f815b9b0 0x7f26f815be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26fd000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f26f815b9b0 0x7f26f815be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38812/0 (socket says 172.21.15.42:38812) 2024-05-28T13:58:53.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26fd000640 1 -- 172.21.15.42:0/752384 learned_addr learned my addr 172.21.15.42:0/752384 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:53.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26ef400640 1 --2- 172.21.15.42:0/752384 >> v2:172.21.15.63:3300/0 conn(0x7f26f806ee20 0x7f26f806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26fd000640 1 -- 172.21.15.42:0/752384 >> v2:172.21.15.63:3300/0 conn(0x7f26f806ee20 msgr2=0x7f26f806f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:53.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26fd000640 1 --2- 172.21.15.42:0/752384 >> v2:172.21.15.63:3300/0 conn(0x7f26f806ee20 0x7f26f806f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26fd000640 1 -- 172.21.15.42:0/752384 >> v1:172.21.15.63:6789/0 conn(0x7f26f8144f80 legacy=0x7f26f8072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.173+0000 7f26fd000640 1 -- 172.21.15.42:0/752384 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f815c340 con 0x7f26f815b9b0 2024-05-28T13:58:53.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.177+0000 7f26fd000640 1 --2- 172.21.15.42:0/752384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f815b9b0 0x7f26f815be00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f26f8075500 tx=0x7f26f001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265a91e2eed5ff42 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:53.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.177+0000 7f26f7400640 1 -- 172.21.15.42:0/752384 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f00b4070 con 0x7f26f815b9b0 2024-05-28T13:58:53.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.177+0000 7f26f7400640 1 -- 172.21.15.42:0/752384 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f00a7ac0 con 0x7f26f815b9b0 2024-05-28T13:58:53.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.177+0000 7f26f7400640 1 -- 172.21.15.42:0/752384 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f00af440 con 0x7f26f815b9b0 2024-05-28T13:58:53.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.177+0000 7f26ff000640 1 -- 172.21.15.42:0/752384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f815b9b0 msgr2=0x7f26f815be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.177+0000 7f26ff000640 1 --2- 172.21.15.42:0/752384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f815b9b0 0x7f26f815be00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f26f8075500 tx=0x7f26f001ebf0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.177+0000 7f26ff000640 1 -- 172.21.15.42:0/752384 shutdown_connections 2024-05-28T13:58:53.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.177+0000 7f26ff000640 1 --2- 172.21.15.42:0/752384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f815b9b0 0x7f26f815be00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.177+0000 7f26ff000640 1 --2- 172.21.15.42:0/752384 >> v2:172.21.15.63:3300/0 conn(0x7f26f806ee20 0x7f26f806f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.177+0000 7f26ff000640 1 -- 172.21.15.42:0/752384 >> 172.21.15.42:0/752384 conn(0x7f26f80b21f0 msgr2=0x7f26f80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:53.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.177+0000 7f26ff000640 1 -- 172.21.15.42:0/752384 shutdown_connections 2024-05-28T13:58:53.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.177+0000 7f26ff000640 1 -- 172.21.15.42:0/752384 wait complete. 2024-05-28T13:58:53.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.177+0000 7f26ff000640 1 Processor -- start 2024-05-28T13:58:53.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.177+0000 7f26ff000640 1 -- start start 2024-05-28T13:58:53.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26ff000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f26f806ee20 0x7f26f81371f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26ff000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f8144f80 0x7f26f8139270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26ff000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f815b9b0 0x7f26f815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26ff000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f26f80b4010 con 0x7f26f815b9b0 2024-05-28T13:58:53.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26ff000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26f8072160 con 0x7f26f8144f80 2024-05-28T13:58:53.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26ff000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f26f80b4190 con 0x7f26f806ee20 2024-05-28T13:58:53.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26fd000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f815b9b0 0x7f26f815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26fd000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f815b9b0 0x7f26f815aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38824/0 (socket says 172.21.15.42:38824) 2024-05-28T13:58:53.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26fd000640 1 -- 172.21.15.42:0/3862638405 learned_addr learned my addr 172.21.15.42:0/3862638405 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:53.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26ef400640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f8144f80 0x7f26f8139270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26f7e00640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f26f806ee20 0x7f26f81371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26fd000640 1 -- 172.21.15.42:0/3862638405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f26f806ee20 msgr2=0x7f26f81371f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26fd000640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f26f806ee20 0x7f26f81371f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26fd000640 1 -- 172.21.15.42:0/3862638405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f8144f80 msgr2=0x7f26f8139270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26fd000640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f8144f80 0x7f26f8139270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26fd000640 1 -- 172.21.15.42:0/3862638405 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f8138c00 con 0x7f26f815b9b0 2024-05-28T13:58:53.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26ef400640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f8144f80 0x7f26f8139270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:53.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26fd000640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f815b9b0 0x7f26f815aac0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f26f000b2c0 tx=0x7f26f001b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:53.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26f4c00640 1 -- 172.21.15.42:0/3862638405 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f00b4040 con 0x7f26f815b9b0 2024-05-28T13:58:53.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26f4c00640 1 -- 172.21.15.42:0/3862638405 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f00afb30 con 0x7f26f815b9b0 2024-05-28T13:58:53.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26f4c00640 1 -- 172.21.15.42:0/3862638405 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f00b8430 con 0x7f26f815b9b0 2024-05-28T13:58:53.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26ff000640 1 -- 172.21.15.42:0/3862638405 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f8137e40 con 0x7f26f815b9b0 2024-05-28T13:58:53.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26ff000640 1 -- 172.21.15.42:0/3862638405 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f8138400 con 0x7f26f815b9b0 2024-05-28T13:58:53.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26ff000640 1 -- 172.21.15.42:0/3862638405 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f8137bc0 con 0x7f26f815b9b0 2024-05-28T13:58:53.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.181+0000 7f26f4c00640 1 -- 172.21.15.42:0/3862638405 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f26f00b85d0 con 0x7f26f815b9b0 2024-05-28T13:58:53.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.185+0000 7f26f4c00640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f26d4068380 0x7f26d406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.185+0000 7f26f4c00640 1 -- 172.21.15.42:0/3862638405 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f26f0123950 con 0x7f26f815b9b0 2024-05-28T13:58:53.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.185+0000 7f26f7e00640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f26d4068380 0x7f26d406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.185+0000 7f26f7e00640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f26d4068380 0x7f26d406a840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f26f813a090 tx=0x7f26e8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:53.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.189+0000 7f26f4c00640 1 -- 172.21.15.42:0/3862638405 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f26f8137bc0 con 0x7f26f815b9b0 2024-05-28T13:58:53.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.353+0000 7f26ff000640 1 -- 172.21.15.42:0/3862638405 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f26f804fdd0 con 0x7f26f815b9b0 2024-05-28T13:58:53.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.353+0000 7f26f4c00640 1 -- 172.21.15.42:0/3862638405 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f26f00187e0 con 0x7f26f815b9b0 2024-05-28T13:58:53.338 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:58:53.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.353+0000 7f26ff000640 1 -- 172.21.15.42:0/3862638405 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f26d4068380 msgr2=0x7f26d406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.353+0000 7f26ff000640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f26d4068380 0x7f26d406a840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f26f813a090 tx=0x7f26e8009040 comp rx=0 tx=0).stop 2024-05-28T13:58:53.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.353+0000 7f26ff000640 1 -- 172.21.15.42:0/3862638405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f815b9b0 msgr2=0x7f26f815aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.353+0000 7f26ff000640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f815b9b0 0x7f26f815aac0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f26f000b2c0 tx=0x7f26f001b990 comp rx=0 tx=0).stop 2024-05-28T13:58:53.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.353+0000 7f26ff000640 1 -- 172.21.15.42:0/3862638405 shutdown_connections 2024-05-28T13:58:53.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.353+0000 7f26ff000640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f26d4068380 0x7f26d406a840 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.353+0000 7f26ff000640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f815b9b0 0x7f26f815aac0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.353+0000 7f26ff000640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f8144f80 0x7f26f8139270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.353+0000 7f26ff000640 1 --2- 172.21.15.42:0/3862638405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f26f806ee20 0x7f26f81371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.353+0000 7f26ff000640 1 -- 172.21.15.42:0/3862638405 >> 172.21.15.42:0/3862638405 conn(0x7f26f80b21f0 msgr2=0x7f26f80b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:53.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.353+0000 7f26ff000640 1 -- 172.21.15.42:0/3862638405 shutdown_connections 2024-05-28T13:58:53.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.353+0000 7f26ff000640 1 -- 172.21.15.42:0/3862638405 wait complete. 2024-05-28T13:58:53.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:58:53.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.517+0000 7f80a1400640 1 Processor -- start 2024-05-28T13:58:53.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.517+0000 7f80a1400640 1 -- start start 2024-05-28T13:58:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.517+0000 7f80a1400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f809c071de0 0x7f809c0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.517+0000 7f80a1400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f809c13fd10 0x7f809c1401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f80a1400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f809c072700 con 0x7f809c0ce3b0 2024-05-28T13:58:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f80a1400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f809c160230 con 0x7f809c071de0 2024-05-28T13:58:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f80a1400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f809c160510 con 0x7f809c13fd10 2024-05-28T13:58:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f809a800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f809c071de0 0x7f809c0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f8091e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f809c0ce3b0 0x7f809c0ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50564/0 (socket says 172.21.15.42:50564) 2024-05-28T13:58:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f8091e00640 1 -- 172.21.15.42:0/2369940293 learned_addr learned my addr 172.21.15.42:0/2369940293 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f809b200640 1 --2- 172.21.15.42:0/2369940293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f809c13fd10 0x7f809c1401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f809a800640 1 -- 172.21.15.42:0/2369940293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f809c13fd10 msgr2=0x7f809c1401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:53.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f809a800640 1 --2- 172.21.15.42:0/2369940293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f809c13fd10 0x7f809c1401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f809a800640 1 -- 172.21.15.42:0/2369940293 >> v1:172.21.15.42:6789/0 conn(0x7f809c0ce3b0 legacy=0x7f809c0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f809a800640 1 -- 172.21.15.42:0/2369940293 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809c1607f0 con 0x7f809c071de0 2024-05-28T13:58:53.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f8099e00640 1 -- 172.21.15.42:0/2369940293 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 372613311 0 0) 0x7f809c072700 con 0x7f809c0ce3b0 2024-05-28T13:58:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f809a800640 1 --2- 172.21.15.42:0/2369940293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f809c071de0 0x7f809c0721c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f808c009d90 tx=0x7f808c01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e923ba4aaf2cd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f8099e00640 1 -- 172.21.15.42:0/2369940293 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c0b2070 con 0x7f809c071de0 2024-05-28T13:58:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f8099e00640 1 -- 172.21.15.42:0/2369940293 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f808c0a5710 con 0x7f809c071de0 2024-05-28T13:58:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.521+0000 7f8099e00640 1 -- 172.21.15.42:0/2369940293 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c0a5ae0 con 0x7f809c071de0 2024-05-28T13:58:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 -- 172.21.15.42:0/2369940293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f809c071de0 msgr2=0x7f809c0721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 --2- 172.21.15.42:0/2369940293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f809c071de0 0x7f809c0721c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f808c009d90 tx=0x7f808c01c850 comp rx=0 tx=0).stop 2024-05-28T13:58:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 -- 172.21.15.42:0/2369940293 shutdown_connections 2024-05-28T13:58:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 --2- 172.21.15.42:0/2369940293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f809c13fd10 0x7f809c1401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 --2- 172.21.15.42:0/2369940293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f809c071de0 0x7f809c0721c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 -- 172.21.15.42:0/2369940293 >> 172.21.15.42:0/2369940293 conn(0x7f809c0b21f0 msgr2=0x7f809c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 -- 172.21.15.42:0/2369940293 shutdown_connections 2024-05-28T13:58:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 -- 172.21.15.42:0/2369940293 wait complete. 2024-05-28T13:58:53.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 Processor -- start 2024-05-28T13:58:53.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 -- start start 2024-05-28T13:58:53.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f809c071de0 0x7f809c145bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f809c0ce3b0 0x7f809c13de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f809c13fd10 0x7f809c1371e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f809c169830 con 0x7f809c0ce3b0 2024-05-28T13:58:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f809c1696b0 con 0x7f809c071de0 2024-05-28T13:58:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f80a1400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f809c1699b0 con 0x7f809c13fd10 2024-05-28T13:58:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f8091e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f809c0ce3b0 0x7f809c13de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f8091e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f809c0ce3b0 0x7f809c13de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38836/0 (socket says 172.21.15.42:38836) 2024-05-28T13:58:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f8091e00640 1 -- 172.21.15.42:0/255986995 learned_addr learned my addr 172.21.15.42:0/255986995 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f809a800640 1 --2- 172.21.15.42:0/255986995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f809c071de0 0x7f809c145bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f809b200640 1 --2- 172.21.15.42:0/255986995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f809c13fd10 0x7f809c1371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f8091e00640 1 -- 172.21.15.42:0/255986995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f809c13fd10 msgr2=0x7f809c1371e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f8091e00640 1 --2- 172.21.15.42:0/255986995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f809c13fd10 0x7f809c1371e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f8091e00640 1 -- 172.21.15.42:0/255986995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f809c071de0 msgr2=0x7f809c145bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f8091e00640 1 --2- 172.21.15.42:0/255986995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f809c071de0 0x7f809c145bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.525+0000 7f8091e00640 1 -- 172.21.15.42:0/255986995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809c13cae0 con 0x7f809c0ce3b0 2024-05-28T13:58:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.529+0000 7f8091e00640 1 --2- 172.21.15.42:0/255986995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f809c0ce3b0 0x7f809c13de60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f80840011f0 tx=0x7f808400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:53.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.529+0000 7f8093400640 1 -- 172.21.15.42:0/255986995 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084018070 con 0x7f809c0ce3b0 2024-05-28T13:58:53.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.529+0000 7f8093400640 1 -- 172.21.15.42:0/255986995 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80840074e0 con 0x7f809c0ce3b0 2024-05-28T13:58:53.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.529+0000 7f8093400640 1 -- 172.21.15.42:0/255986995 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084004a10 con 0x7f809c0ce3b0 2024-05-28T13:58:53.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.529+0000 7f80a1400640 1 -- 172.21.15.42:0/255986995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809c13cdc0 con 0x7f809c0ce3b0 2024-05-28T13:58:53.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.529+0000 7f80a1400640 1 -- 172.21.15.42:0/255986995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809c13d380 con 0x7f809c0ce3b0 2024-05-28T13:58:53.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.529+0000 7f80a1400640 1 -- 172.21.15.42:0/255986995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f809c0738b0 con 0x7f809c0ce3b0 2024-05-28T13:58:53.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.529+0000 7f8093400640 1 -- 172.21.15.42:0/255986995 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f808401b070 con 0x7f809c0ce3b0 2024-05-28T13:58:53.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.529+0000 7f8093400640 1 --2- 172.21.15.42:0/255986995 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8078068380 0x7f807806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.529+0000 7f8093400640 1 -- 172.21.15.42:0/255986995 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8084085c90 con 0x7f809c0ce3b0 2024-05-28T13:58:53.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.529+0000 7f809a800640 1 --2- 172.21.15.42:0/255986995 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8078068380 0x7f807806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.533+0000 7f809a800640 1 --2- 172.21.15.42:0/255986995 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8078068380 0x7f807806a840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f808c0af000 tx=0x7f808c0a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:53.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.533+0000 7f8093400640 1 -- 172.21.15.42:0/255986995 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8084055c00 con 0x7f809c0ce3b0 2024-05-28T13:58:53.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.697+0000 7f80a1400640 1 -- 172.21.15.42:0/255986995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f809c04fdd0 con 0x7f809c0ce3b0 2024-05-28T13:58:53.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.697+0000 7f8093400640 1 -- 172.21.15.42:0/255986995 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f8084059f00 con 0x7f809c0ce3b0 2024-05-28T13:58:53.684 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:58:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.701+0000 7f80a1400640 1 -- 172.21.15.42:0/255986995 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8078068380 msgr2=0x7f807806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.701+0000 7f80a1400640 1 --2- 172.21.15.42:0/255986995 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8078068380 0x7f807806a840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f808c0af000 tx=0x7f808c0a7070 comp rx=0 tx=0).stop 2024-05-28T13:58:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.701+0000 7f80a1400640 1 -- 172.21.15.42:0/255986995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f809c0ce3b0 msgr2=0x7f809c13de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.701+0000 7f80a1400640 1 --2- 172.21.15.42:0/255986995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f809c0ce3b0 0x7f809c13de60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f80840011f0 tx=0x7f808400c9b0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.701+0000 7f80a1400640 1 -- 172.21.15.42:0/255986995 shutdown_connections 2024-05-28T13:58:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.701+0000 7f80a1400640 1 --2- 172.21.15.42:0/255986995 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8078068380 0x7f807806a840 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.701+0000 7f80a1400640 1 --2- 172.21.15.42:0/255986995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f809c13fd10 0x7f809c1371e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.701+0000 7f80a1400640 1 --2- 172.21.15.42:0/255986995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f809c0ce3b0 0x7f809c13de60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.701+0000 7f80a1400640 1 --2- 172.21.15.42:0/255986995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f809c071de0 0x7f809c145bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.701+0000 7f80a1400640 1 -- 172.21.15.42:0/255986995 >> 172.21.15.42:0/255986995 conn(0x7f809c0b21f0 msgr2=0x7f809c06f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:53.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.701+0000 7f80a1400640 1 -- 172.21.15.42:0/255986995 shutdown_connections 2024-05-28T13:58:53.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.701+0000 7f80a1400640 1 -- 172.21.15.42:0/255986995 wait complete. 2024-05-28T13:58:53.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:53.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bfc00640 1 Processor -- start 2024-05-28T13:58:53.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bfc00640 1 -- start start 2024-05-28T13:58:53.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bfc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4b80ce860 0x7fa4b806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bfc00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa4b80d0900 0x7fa4b8162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bfc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa4b806dcf0 0x7fa4b8162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bfc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b80d19f0 con 0x7fa4b806dcf0 2024-05-28T13:58:53.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bfc00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b80d0de0 con 0x7fa4b80d0900 2024-05-28T13:58:53.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bfc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b80d10c0 con 0x7fa4b80ce860 2024-05-28T13:58:53.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bdc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa4b806dcf0 0x7fa4b8162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bdc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa4b806dcf0 0x7fa4b8162ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38846/0 (socket says 172.21.15.42:38846) 2024-05-28T13:58:53.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bdc00640 1 -- 172.21.15.42:0/1381330128 learned_addr learned my addr 172.21.15.42:0/1381330128 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:53.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/1381330128 >> v2:172.21.15.63:3300/0 conn(0x7fa4b80d0900 0x7fa4b8162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bd200640 1 --2- 172.21.15.42:0/1381330128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4b80ce860 0x7fa4b806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bdc00640 1 -- 172.21.15.42:0/1381330128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4b80ce860 msgr2=0x7fa4b806d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:53.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bdc00640 1 --2- 172.21.15.42:0/1381330128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4b80ce860 0x7fa4b806d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bdc00640 1 -- 172.21.15.42:0/1381330128 >> v2:172.21.15.63:3300/0 conn(0x7fa4b80d0900 msgr2=0x7fa4b8162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:53.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bdc00640 1 --2- 172.21.15.42:0/1381330128 >> v2:172.21.15.63:3300/0 conn(0x7fa4b80d0900 0x7fa4b8162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bdc00640 1 -- 172.21.15.42:0/1381330128 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b80d13a0 con 0x7fa4b806dcf0 2024-05-28T13:58:53.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.861+0000 7fa4bd200640 1 --2- 172.21.15.42:0/1381330128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4b80ce860 0x7fa4b806d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:53.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.865+0000 7fa4bdc00640 1 --2- 172.21.15.42:0/1381330128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b806dcf0 0x7fa4b8162ab0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80735e0 tx=0x7fa4b001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607c685f0d6073ad server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:53.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.865+0000 7fa4b7400640 1 -- 172.21.15.42:0/1381330128 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00a56d0 con 0x7fa4b806dcf0 2024-05-28T13:58:53.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.865+0000 7fa4b7400640 1 -- 172.21.15.42:0/1381330128 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b00a5870 con 0x7fa4b806dcf0 2024-05-28T13:58:53.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.865+0000 7fa4b7400640 1 -- 172.21.15.42:0/1381330128 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00a5c40 con 0x7fa4b806dcf0 2024-05-28T13:58:53.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.865+0000 7fa4bfc00640 1 -- 172.21.15.42:0/1381330128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b806dcf0 msgr2=0x7fa4b8162ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.865+0000 7fa4bfc00640 1 --2- 172.21.15.42:0/1381330128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b806dcf0 0x7fa4b8162ab0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80735e0 tx=0x7fa4b001c930 comp rx=0 tx=0).stop 2024-05-28T13:58:53.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.865+0000 7fa4bfc00640 1 -- 172.21.15.42:0/1381330128 shutdown_connections 2024-05-28T13:58:53.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.865+0000 7fa4bfc00640 1 --2- 172.21.15.42:0/1381330128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b806dcf0 0x7fa4b8162ab0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.865+0000 7fa4bfc00640 1 --2- 172.21.15.42:0/1381330128 >> v2:172.21.15.63:3300/0 conn(0x7fa4b80d0900 0x7fa4b8162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.865+0000 7fa4bfc00640 1 --2- 172.21.15.42:0/1381330128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4b80ce860 0x7fa4b806d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.865+0000 7fa4bfc00640 1 -- 172.21.15.42:0/1381330128 >> 172.21.15.42:0/1381330128 conn(0x7fa4b80b4080 msgr2=0x7fa4b80b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:53.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.865+0000 7fa4bfc00640 1 -- 172.21.15.42:0/1381330128 shutdown_connections 2024-05-28T13:58:53.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.865+0000 7fa4bfc00640 1 -- 172.21.15.42:0/1381330128 wait complete. 2024-05-28T13:58:53.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4bfc00640 1 Processor -- start 2024-05-28T13:58:53.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4bfc00640 1 -- start start 2024-05-28T13:58:53.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4bfc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b806dcf0 0x7fa4b80c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4bfc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b80ce860 0x7fa4b814f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4bfc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4b80d0900 0x7fa4b8161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4bfc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b815eda0 con 0x7fa4b80ce860 2024-05-28T13:58:53.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4bfc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b815ec20 con 0x7fa4b806dcf0 2024-05-28T13:58:53.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4bfc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b809aa50 con 0x7fa4b80d0900 2024-05-28T13:58:53.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b80ce860 0x7fa4b814f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b80ce860 0x7fa4b814f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38858/0 (socket says 172.21.15.42:38858) 2024-05-28T13:58:53.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4b7e00640 1 -- 172.21.15.42:0/3965697889 learned_addr learned my addr 172.21.15.42:0/3965697889 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:53.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4bd200640 1 --2- 172.21.15.42:0/3965697889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b806dcf0 0x7fa4b80c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4bdc00640 1 --2- 172.21.15.42:0/3965697889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4b80d0900 0x7fa4b8161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4b7e00640 1 -- 172.21.15.42:0/3965697889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4b80d0900 msgr2=0x7fa4b8161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/3965697889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4b80d0900 0x7fa4b8161610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4b7e00640 1 -- 172.21.15.42:0/3965697889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b806dcf0 msgr2=0x7fa4b80c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:53.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/3965697889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b806dcf0 0x7fa4b80c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:53.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4b7e00640 1 -- 172.21.15.42:0/3965697889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b80c6bb0 con 0x7fa4b80ce860 2024-05-28T13:58:53.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4b7e00640 1 --2- 172.21.15.42:0/3965697889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b80ce860 0x7fa4b814f5a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400d620 tx=0x7fa4a400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:53.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4b4c00640 1 -- 172.21.15.42:0/3965697889 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a400b620 con 0x7fa4b80ce860 2024-05-28T13:58:53.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4b4c00640 1 -- 172.21.15.42:0/3965697889 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a4019070 con 0x7fa4b80ce860 2024-05-28T13:58:53.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4b4c00640 1 -- 172.21.15.42:0/3965697889 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a400b9e0 con 0x7fa4b80ce860 2024-05-28T13:58:53.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4bfc00640 1 -- 172.21.15.42:0/3965697889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b80c6e90 con 0x7fa4b80ce860 2024-05-28T13:58:53.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.869+0000 7fa4bfc00640 1 -- 172.21.15.42:0/3965697889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b814fae0 con 0x7fa4b80ce860 2024-05-28T13:58:53.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.873+0000 7fa4bfc00640 1 -- 172.21.15.42:0/3965697889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b8068a90 con 0x7fa4b80ce860 2024-05-28T13:58:53.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.873+0000 7fa4b4c00640 1 -- 172.21.15.42:0/3965697889 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa4a400bb80 con 0x7fa4b80ce860 2024-05-28T13:58:53.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.873+0000 7fa4b4c00640 1 --2- 172.21.15.42:0/3965697889 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa4980683b0 0x7fa49806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:53.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.873+0000 7fa4b4c00640 1 -- 172.21.15.42:0/3965697889 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa4a4086c60 con 0x7fa4b80ce860 2024-05-28T13:58:53.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.873+0000 7fa4bd200640 1 --2- 172.21.15.42:0/3965697889 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa4980683b0 0x7fa49806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:53.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.873+0000 7fa4bd200640 1 --2- 172.21.15.42:0/3965697889 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa4980683b0 0x7fa49806a870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8161400 tx=0x7fa4ac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:53.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:53.877+0000 7fa4b4c00640 1 -- 172.21.15.42:0/3965697889 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa4a4056bc0 con 0x7fa4b80ce860 2024-05-28T13:58:54.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.045+0000 7fa4bfc00640 1 -- 172.21.15.42:0/3965697889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4b80b4ce0 con 0x7fa4b80ce860 2024-05-28T13:58:54.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.045+0000 7fa4b4c00640 1 -- 172.21.15.42:0/3965697889 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16545 (secure 0 0 0) 0x7fa4a405aec0 con 0x7fa4b80ce860 2024-05-28T13:58:54.032 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:54.032 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:24.659233+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.049+0000 7fa4bfc00640 1 -- 172.21.15.42:0/3965697889 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa4980683b0 msgr2=0x7fa49806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.049+0000 7fa4bfc00640 1 --2- 172.21.15.42:0/3965697889 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa4980683b0 0x7fa49806a870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8161400 tx=0x7fa4ac008040 comp rx=0 tx=0).stop 2024-05-28T13:58:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.049+0000 7fa4bfc00640 1 -- 172.21.15.42:0/3965697889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b80ce860 msgr2=0x7fa4b814f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.049+0000 7fa4bfc00640 1 --2- 172.21.15.42:0/3965697889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b80ce860 0x7fa4b814f5a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400d620 tx=0x7fa4a400dae0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.049+0000 7fa4bfc00640 1 -- 172.21.15.42:0/3965697889 shutdown_connections 2024-05-28T13:58:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.049+0000 7fa4bfc00640 1 --2- 172.21.15.42:0/3965697889 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa4980683b0 0x7fa49806a870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.049+0000 7fa4bfc00640 1 --2- 172.21.15.42:0/3965697889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4b80d0900 0x7fa4b8161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.049+0000 7fa4bfc00640 1 --2- 172.21.15.42:0/3965697889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b80ce860 0x7fa4b814f5a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.049+0000 7fa4bfc00640 1 --2- 172.21.15.42:0/3965697889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b806dcf0 0x7fa4b80c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.049+0000 7fa4bfc00640 1 -- 172.21.15.42:0/3965697889 >> 172.21.15.42:0/3965697889 conn(0x7fa4b80b4080 msgr2=0x7fa4b8070720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.049+0000 7fa4bfc00640 1 -- 172.21.15.42:0/3965697889 shutdown_connections 2024-05-28T13:58:54.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.049+0000 7fa4bfc00640 1 -- 172.21.15.42:0/3965697889 wait complete. 2024-05-28T13:58:54.053 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:58:54.053 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:58:54.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a47400640 1 Processor -- start 2024-05-28T13:58:54.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a47400640 1 -- start start 2024-05-28T13:58:54.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a47400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a40148090 0x7f5a400b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a47400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5a400b4c10 0x7f5a40071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a47400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a400b3690 con 0x7f5a40072320 2024-05-28T13:58:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a47400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5a40148470 con 0x7f5a400b4c10 2024-05-28T13:58:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a47400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5a400b4730 con 0x7f5a40148090 2024-05-28T13:58:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a3fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5a400b4c10 0x7f5a40071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a3fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5a400b4c10 0x7f5a40071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38860/0 (socket says 172.21.15.42:38860) 2024-05-28T13:58:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a3fe00640 1 -- 172.21.15.42:0/2090590135 learned_addr learned my addr 172.21.15.42:0/2090590135 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a44a00640 1 --2- 172.21.15.42:0/2090590135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a40148090 0x7f5a400b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a3fe00640 1 -- 172.21.15.42:0/2090590135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a40148090 msgr2=0x7f5a400b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a3fe00640 1 --2- 172.21.15.42:0/2090590135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a40148090 0x7f5a400b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a3fe00640 1 -- 172.21.15.42:0/2090590135 >> v1:172.21.15.63:6789/0 conn(0x7f5a40072320 legacy=0x7f5a401623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.213+0000 7f5a3fe00640 1 -- 172.21.15.42:0/2090590135 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a400b3970 con 0x7f5a400b4c10 2024-05-28T13:58:54.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.217+0000 7f5a3fe00640 1 --2- 172.21.15.42:0/2090590135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a400b4c10 0x7f5a40071de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009870 tx=0x7f5a2c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a437256db025136 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:54.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.217+0000 7f5a3f400640 1 -- 172.21.15.42:0/2090590135 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0b1070 con 0x7f5a400b4c10 2024-05-28T13:58:54.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.217+0000 7f5a3f400640 1 -- 172.21.15.42:0/2090590135 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a2c0ac620 con 0x7f5a400b4c10 2024-05-28T13:58:54.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.217+0000 7f5a3f400640 1 -- 172.21.15.42:0/2090590135 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0ac9f0 con 0x7f5a400b4c10 2024-05-28T13:58:54.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.217+0000 7f5a47400640 1 -- 172.21.15.42:0/2090590135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a400b4c10 msgr2=0x7f5a40071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.217+0000 7f5a47400640 1 --2- 172.21.15.42:0/2090590135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a400b4c10 0x7f5a40071de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009870 tx=0x7f5a2c01c4d0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.217+0000 7f5a47400640 1 -- 172.21.15.42:0/2090590135 shutdown_connections 2024-05-28T13:58:54.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.217+0000 7f5a47400640 1 --2- 172.21.15.42:0/2090590135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a400b4c10 0x7f5a40071de0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.217+0000 7f5a47400640 1 --2- 172.21.15.42:0/2090590135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a40148090 0x7f5a400b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.217+0000 7f5a47400640 1 -- 172.21.15.42:0/2090590135 >> 172.21.15.42:0/2090590135 conn(0x7f5a400b21f0 msgr2=0x7f5a400b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:54.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.217+0000 7f5a47400640 1 -- 172.21.15.42:0/2090590135 shutdown_connections 2024-05-28T13:58:54.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.217+0000 7f5a47400640 1 -- 172.21.15.42:0/2090590135 wait complete. 2024-05-28T13:58:54.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.217+0000 7f5a47400640 1 Processor -- start 2024-05-28T13:58:54.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a47400640 1 -- start start 2024-05-28T13:58:54.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a47400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a40072320 0x7f5a4013d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a47400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a40148090 0x7f5a4013a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a47400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a4013ad70 0x7f5a40160970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a47400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5a400b57e0 con 0x7f5a40072320 2024-05-28T13:58:54.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a47400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a400b5660 con 0x7f5a4013ad70 2024-05-28T13:58:54.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a47400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4016b540 con 0x7f5a40148090 2024-05-28T13:58:54.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a44a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a40072320 0x7f5a4013d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a44a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a40072320 0x7f5a4013d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38870/0 (socket says 172.21.15.42:38870) 2024-05-28T13:58:54.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a44a00640 1 -- 172.21.15.42:0/2101054603 learned_addr learned my addr 172.21.15.42:0/2101054603 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:54.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a3fe00640 1 --2- 172.21.15.42:0/2101054603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a40148090 0x7f5a4013a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a45400640 1 --2- 172.21.15.42:0/2101054603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a4013ad70 0x7f5a40160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a44a00640 1 -- 172.21.15.42:0/2101054603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a40148090 msgr2=0x7f5a4013a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a44a00640 1 --2- 172.21.15.42:0/2101054603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a40148090 0x7f5a4013a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a44a00640 1 -- 172.21.15.42:0/2101054603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a4013ad70 msgr2=0x7f5a40160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a44a00640 1 --2- 172.21.15.42:0/2101054603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a4013ad70 0x7f5a40160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a44a00640 1 -- 172.21.15.42:0/2101054603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a40161680 con 0x7f5a40072320 2024-05-28T13:58:54.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a44a00640 1 --2- 172.21.15.42:0/2101054603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a40072320 0x7f5a4013d0f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400d5f0 tx=0x7f5a3400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:54.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a3cc00640 1 -- 172.21.15.42:0/2101054603 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a340125f0 con 0x7f5a40072320 2024-05-28T13:58:54.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a3cc00640 1 -- 172.21.15.42:0/2101054603 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a34019070 con 0x7f5a40072320 2024-05-28T13:58:54.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a47400640 1 -- 172.21.15.42:0/2101054603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a40128440 con 0x7f5a40072320 2024-05-28T13:58:54.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a3cc00640 1 -- 172.21.15.42:0/2101054603 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a340129b0 con 0x7f5a40072320 2024-05-28T13:58:54.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a47400640 1 -- 172.21.15.42:0/2101054603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4013d630 con 0x7f5a40072320 2024-05-28T13:58:54.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.221+0000 7f5a47400640 1 -- 172.21.15.42:0/2101054603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a4013b180 con 0x7f5a40072320 2024-05-28T13:58:54.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.225+0000 7f5a3cc00640 1 -- 172.21.15.42:0/2101054603 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5a34012b50 con 0x7f5a40072320 2024-05-28T13:58:54.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.225+0000 7f5a3cc00640 1 --2- 172.21.15.42:0/2101054603 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a20068380 0x7f5a2006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.225+0000 7f5a3cc00640 1 -- 172.21.15.42:0/2101054603 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5a34086d00 con 0x7f5a40072320 2024-05-28T13:58:54.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.225+0000 7f5a3fe00640 1 --2- 172.21.15.42:0/2101054603 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a20068380 0x7f5a2006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.225+0000 7f5a3fe00640 1 --2- 172.21.15.42:0/2101054603 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a20068380 0x7f5a2006a840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0ae000 tx=0x7f5a2c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:54.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.229+0000 7f5a3cc00640 1 -- 172.21.15.42:0/2101054603 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5a4013b180 con 0x7f5a40072320 2024-05-28T13:58:54.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.393+0000 7f5a47400640 1 -- 172.21.15.42:0/2101054603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5a400738b0 con 0x7f5a40072320 2024-05-28T13:58:54.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.393+0000 7f5a3cc00640 1 -- 172.21.15.42:0/2101054603 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f5a34056be0 con 0x7f5a40072320 2024-05-28T13:58:54.380 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:58:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.397+0000 7f5a47400640 1 -- 172.21.15.42:0/2101054603 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a20068380 msgr2=0x7f5a2006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.397+0000 7f5a47400640 1 --2- 172.21.15.42:0/2101054603 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a20068380 0x7f5a2006a840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0ae000 tx=0x7f5a2c008040 comp rx=0 tx=0).stop 2024-05-28T13:58:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.397+0000 7f5a47400640 1 -- 172.21.15.42:0/2101054603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a40072320 msgr2=0x7f5a4013d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.397+0000 7f5a47400640 1 --2- 172.21.15.42:0/2101054603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a40072320 0x7f5a4013d0f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400d5f0 tx=0x7f5a3400dab0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.397+0000 7f5a47400640 1 -- 172.21.15.42:0/2101054603 shutdown_connections 2024-05-28T13:58:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.397+0000 7f5a47400640 1 --2- 172.21.15.42:0/2101054603 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a20068380 0x7f5a2006a840 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.397+0000 7f5a47400640 1 --2- 172.21.15.42:0/2101054603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a4013ad70 0x7f5a40160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.397+0000 7f5a47400640 1 --2- 172.21.15.42:0/2101054603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a40148090 0x7f5a4013a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.397+0000 7f5a47400640 1 --2- 172.21.15.42:0/2101054603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a40072320 0x7f5a4013d0f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.397+0000 7f5a47400640 1 -- 172.21.15.42:0/2101054603 >> 172.21.15.42:0/2101054603 conn(0x7f5a400b21f0 msgr2=0x7f5a400aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:54.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.397+0000 7f5a47400640 1 -- 172.21.15.42:0/2101054603 shutdown_connections 2024-05-28T13:58:54.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.397+0000 7f5a47400640 1 -- 172.21.15.42:0/2101054603 wait complete. 2024-05-28T13:58:54.402 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:58:54.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ef7a00640 1 Processor -- start 2024-05-28T13:58:54.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ef7a00640 1 -- start start 2024-05-28T13:58:54.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ef7a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef00ce3b0 0x7f8ef00ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ef7a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8ef013fd10 0x7f8ef01401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ef7a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef0160230 con 0x7f8ef0071de0 2024-05-28T13:58:54.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ef7a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8ef0160510 con 0x7f8ef013fd10 2024-05-28T13:58:54.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ef7a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef01607f0 con 0x7f8ef00ce3b0 2024-05-28T13:58:54.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ef5000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8ef0071de0 0x7f8ef00721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50576/0 (socket says 172.21.15.42:50576) 2024-05-28T13:58:54.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ef5000640 1 -- 172.21.15.42:0/51540688 learned_addr learned my addr 172.21.15.42:0/51540688 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:54.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ee7e00640 1 --2- 172.21.15.42:0/51540688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef00ce3b0 0x7f8ef00ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ef5a00640 1 --2- 172.21.15.42:0/51540688 >> v2:172.21.15.63:3300/0 conn(0x7f8ef013fd10 0x7f8ef01401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8eefe00640 1 -- 172.21.15.42:0/51540688 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2813246029 0 0) 0x7f8ef0160230 con 0x7f8ef0071de0 2024-05-28T13:58:54.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8eefe00640 1 -- 172.21.15.42:0/51540688 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ed4003530 con 0x7f8ef0071de0 2024-05-28T13:58:54.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8eefe00640 1 -- 172.21.15.42:0/51540688 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 249915046 0 0) 0x7f8ed4003530 con 0x7f8ef0071de0 2024-05-28T13:58:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ee7e00640 1 -- 172.21.15.42:0/51540688 >> v2:172.21.15.63:3300/0 conn(0x7f8ef013fd10 msgr2=0x7f8ef01401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ee7e00640 1 --2- 172.21.15.42:0/51540688 >> v2:172.21.15.63:3300/0 conn(0x7f8ef013fd10 0x7f8ef01401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ee7e00640 1 -- 172.21.15.42:0/51540688 >> v1:172.21.15.42:6789/0 conn(0x7f8ef0071de0 legacy=0x7f8ef00721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.561+0000 7f8ee7e00640 1 -- 172.21.15.42:0/51540688 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef0160ad0 con 0x7f8ef00ce3b0 2024-05-28T13:58:54.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.565+0000 7f8ee7e00640 1 --2- 172.21.15.42:0/51540688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef00ce3b0 0x7f8ef00ce810 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80097c0 tx=0x7f8ed801c2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=944e57bef591501 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:54.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.565+0000 7f8eefe00640 1 -- 172.21.15.42:0/51540688 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80b0070 con 0x7f8ef00ce3b0 2024-05-28T13:58:54.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.565+0000 7f8eefe00640 1 -- 172.21.15.42:0/51540688 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed80ab400 con 0x7f8ef00ce3b0 2024-05-28T13:58:54.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.565+0000 7f8eefe00640 1 -- 172.21.15.42:0/51540688 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0001680 con 0x7f8ef00ce3b0 2024-05-28T13:58:54.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.565+0000 7f8ef7a00640 1 -- 172.21.15.42:0/51540688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef00ce3b0 msgr2=0x7f8ef00ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.565+0000 7f8ef7a00640 1 --2- 172.21.15.42:0/51540688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef00ce3b0 0x7f8ef00ce810 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80097c0 tx=0x7f8ed801c2a0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.565+0000 7f8ef7a00640 1 -- 172.21.15.42:0/51540688 shutdown_connections 2024-05-28T13:58:54.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.565+0000 7f8ef7a00640 1 --2- 172.21.15.42:0/51540688 >> v2:172.21.15.63:3300/0 conn(0x7f8ef013fd10 0x7f8ef01401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.565+0000 7f8ef7a00640 1 --2- 172.21.15.42:0/51540688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef00ce3b0 0x7f8ef00ce810 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.565+0000 7f8ef7a00640 1 -- 172.21.15.42:0/51540688 >> 172.21.15.42:0/51540688 conn(0x7f8ef00b21f0 msgr2=0x7f8ef00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:54.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.565+0000 7f8ef7a00640 1 -- 172.21.15.42:0/51540688 shutdown_connections 2024-05-28T13:58:54.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.565+0000 7f8ef7a00640 1 -- 172.21.15.42:0/51540688 wait complete. 2024-05-28T13:58:54.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef7a00640 1 Processor -- start 2024-05-28T13:58:54.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef7a00640 1 -- start start 2024-05-28T13:58:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef7a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ef0071de0 0x7f8ef013afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef7a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef00ce3b0 0x7f8ef015f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef7a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef013fd10 0x7f8ef0155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef7a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0169830 con 0x7f8ef013fd10 2024-05-28T13:58:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef7a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01696b0 con 0x7f8ef0071de0 2024-05-28T13:58:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef7a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef01699b0 con 0x7f8ef00ce3b0 2024-05-28T13:58:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef013fd10 0x7f8ef0155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef013fd10 0x7f8ef0155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38874/0 (socket says 172.21.15.42:38874) 2024-05-28T13:58:54.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef5a00640 1 -- 172.21.15.42:0/2876606949 learned_addr learned my addr 172.21.15.42:0/2876606949 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:54.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ee7e00640 1 --2- 172.21.15.42:0/2876606949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef00ce3b0 0x7f8ef015f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef5000640 1 --2- 172.21.15.42:0/2876606949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ef0071de0 0x7f8ef013afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef5a00640 1 -- 172.21.15.42:0/2876606949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef00ce3b0 msgr2=0x7f8ef015f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef5a00640 1 --2- 172.21.15.42:0/2876606949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef00ce3b0 0x7f8ef015f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef5a00640 1 -- 172.21.15.42:0/2876606949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ef0071de0 msgr2=0x7f8ef013afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef5a00640 1 --2- 172.21.15.42:0/2876606949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ef0071de0 0x7f8ef013afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef5a00640 1 -- 172.21.15.42:0/2876606949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef013b830 con 0x7f8ef013fd10 2024-05-28T13:58:54.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef5a00640 1 --2- 172.21.15.42:0/2876606949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef013fd10 0x7f8ef0155c70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800efc0 tx=0x7f8ee800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:54.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8eed600640 1 -- 172.21.15.42:0/2876606949 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8018020 con 0x7f8ef013fd10 2024-05-28T13:58:54.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8eed600640 1 -- 172.21.15.42:0/2876606949 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee8014480 con 0x7f8ef013fd10 2024-05-28T13:58:54.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef7a00640 1 -- 172.21.15.42:0/2876606949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef013bb10 con 0x7f8ef013fd10 2024-05-28T13:58:54.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8eed600640 1 -- 172.21.15.42:0/2876606949 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8014870 con 0x7f8ef013fd10 2024-05-28T13:58:54.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.569+0000 7f8ef7a00640 1 -- 172.21.15.42:0/2876606949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef013c0d0 con 0x7f8ef013fd10 2024-05-28T13:58:54.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.573+0000 7f8ef7a00640 1 -- 172.21.15.42:0/2876606949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef00738b0 con 0x7f8ef013fd10 2024-05-28T13:58:54.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.573+0000 7f8eed600640 1 -- 172.21.15.42:0/2876606949 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8ee801b020 con 0x7f8ef013fd10 2024-05-28T13:58:54.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.573+0000 7f8eed600640 1 --2- 172.21.15.42:0/2876606949 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ecc068380 0x7f8ecc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.573+0000 7f8eed600640 1 -- 172.21.15.42:0/2876606949 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8ee8087330 con 0x7f8ef013fd10 2024-05-28T13:58:54.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.573+0000 7f8ef5000640 1 --2- 172.21.15.42:0/2876606949 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ecc068380 0x7f8ecc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.573+0000 7f8ef5000640 1 --2- 172.21.15.42:0/2876606949 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ecc068380 0x7f8ecc06a840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00017f0 tx=0x7f8ee0001580 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:54.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.577+0000 7f8eed600640 1 -- 172.21.15.42:0/2876606949 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ee8022070 con 0x7f8ef013fd10 2024-05-28T13:58:54.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.745+0000 7f8ef7a00640 1 -- 172.21.15.42:0/2876606949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8ef004fdd0 con 0x7f8ef013fd10 2024-05-28T13:58:54.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.745+0000 7f8eed600640 1 -- 172.21.15.42:0/2876606949 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f8ee8010040 con 0x7f8ef013fd10 2024-05-28T13:58:54.729 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:58:54.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.745+0000 7f8ef7a00640 1 -- 172.21.15.42:0/2876606949 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ecc068380 msgr2=0x7f8ecc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.745+0000 7f8ef7a00640 1 --2- 172.21.15.42:0/2876606949 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ecc068380 0x7f8ecc06a840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00017f0 tx=0x7f8ee0001580 comp rx=0 tx=0).stop 2024-05-28T13:58:54.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.745+0000 7f8ef7a00640 1 -- 172.21.15.42:0/2876606949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef013fd10 msgr2=0x7f8ef0155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.745+0000 7f8ef7a00640 1 --2- 172.21.15.42:0/2876606949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef013fd10 0x7f8ef0155c70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800efc0 tx=0x7f8ee800c490 comp rx=0 tx=0).stop 2024-05-28T13:58:54.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.745+0000 7f8ef7a00640 1 -- 172.21.15.42:0/2876606949 shutdown_connections 2024-05-28T13:58:54.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.745+0000 7f8ef7a00640 1 --2- 172.21.15.42:0/2876606949 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ecc068380 0x7f8ecc06a840 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.745+0000 7f8ef7a00640 1 --2- 172.21.15.42:0/2876606949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef013fd10 0x7f8ef0155c70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.745+0000 7f8ef7a00640 1 --2- 172.21.15.42:0/2876606949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef00ce3b0 0x7f8ef015f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.745+0000 7f8ef7a00640 1 --2- 172.21.15.42:0/2876606949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ef0071de0 0x7f8ef013afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.745+0000 7f8ef7a00640 1 -- 172.21.15.42:0/2876606949 >> 172.21.15.42:0/2876606949 conn(0x7f8ef00b21f0 msgr2=0x7f8ef015b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:54.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.745+0000 7f8ef7a00640 1 -- 172.21.15.42:0/2876606949 shutdown_connections 2024-05-28T13:58:54.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.745+0000 7f8ef7a00640 1 -- 172.21.15.42:0/2876606949 wait complete. 2024-05-28T13:58:54.751 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:58:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f06a4e00640 1 Processor -- start 2024-05-28T13:58:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f06a4e00640 1 -- start start 2024-05-28T13:58:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f06a4e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f06a015c770 0x7f06a009b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f06a4e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f06a00d11d0 0x7f06a0166470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f06a4e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a009ba40 con 0x7f06a015cb50 2024-05-28T13:58:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f06a4e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f06a00c6720 con 0x7f06a015c770 2024-05-28T13:58:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f06a4e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f06a00c60d0 con 0x7f06a00d11d0 2024-05-28T13:58:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f069de00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f06a015c770 0x7f06a009b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f0695400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f06a015cb50 0x7f06a00d0ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50588/0 (socket says 172.21.15.42:50588) 2024-05-28T13:58:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f0695400640 1 -- 172.21.15.42:0/3392157602 learned_addr learned my addr 172.21.15.42:0/3392157602 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f069e800640 1 --2- 172.21.15.42:0/3392157602 >> v2:172.21.15.63:3300/0 conn(0x7f06a00d11d0 0x7f06a0166470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f069de00640 1 -- 172.21.15.42:0/3392157602 >> v2:172.21.15.63:3300/0 conn(0x7f06a00d11d0 msgr2=0x7f06a0166470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:54.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f069de00640 1 --2- 172.21.15.42:0/3392157602 >> v2:172.21.15.63:3300/0 conn(0x7f06a00d11d0 0x7f06a0166470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f069de00640 1 -- 172.21.15.42:0/3392157602 >> v1:172.21.15.42:6789/0 conn(0x7f06a015cb50 legacy=0x7f06a00d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f069de00640 1 -- 172.21.15.42:0/3392157602 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a00c63b0 con 0x7f06a015c770 2024-05-28T13:58:54.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.909+0000 7f069d400640 1 -- 172.21.15.42:0/3392157602 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4243789829 0 0) 0x7f06a009ba40 con 0x7f06a015cb50 2024-05-28T13:58:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.913+0000 7f069de00640 1 --2- 172.21.15.42:0/3392157602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a015c770 0x7f06a009b460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f06a0160390 tx=0x7f069001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42f2b56f8d37649 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:54.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.913+0000 7f069d400640 1 -- 172.21.15.42:0/3392157602 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06900b5070 con 0x7f06a015c770 2024-05-28T13:58:54.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.913+0000 7f069d400640 1 -- 172.21.15.42:0/3392157602 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06900a8970 con 0x7f06a015c770 2024-05-28T13:58:54.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.913+0000 7f069d400640 1 -- 172.21.15.42:0/3392157602 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06900a8d40 con 0x7f06a015c770 2024-05-28T13:58:54.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.913+0000 7f06a4e00640 1 -- 172.21.15.42:0/3392157602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a015c770 msgr2=0x7f06a009b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.913+0000 7f06a4e00640 1 --2- 172.21.15.42:0/3392157602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a015c770 0x7f06a009b460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f06a0160390 tx=0x7f069001ec70 comp rx=0 tx=0).stop 2024-05-28T13:58:54.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.913+0000 7f06a4e00640 1 -- 172.21.15.42:0/3392157602 shutdown_connections 2024-05-28T13:58:54.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.913+0000 7f06a4e00640 1 --2- 172.21.15.42:0/3392157602 >> v2:172.21.15.63:3300/0 conn(0x7f06a00d11d0 0x7f06a0166470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.913+0000 7f06a4e00640 1 --2- 172.21.15.42:0/3392157602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a015c770 0x7f06a009b460 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.913+0000 7f06a4e00640 1 -- 172.21.15.42:0/3392157602 >> 172.21.15.42:0/3392157602 conn(0x7f06a00b3fd0 msgr2=0x7f06a00b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:54.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.913+0000 7f06a4e00640 1 -- 172.21.15.42:0/3392157602 shutdown_connections 2024-05-28T13:58:54.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.913+0000 7f06a4e00640 1 -- 172.21.15.42:0/3392157602 wait complete. 2024-05-28T13:58:54.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f06a4e00640 1 Processor -- start 2024-05-28T13:58:54.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f06a4e00640 1 -- start start 2024-05-28T13:58:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f06a4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a00d11d0 0x7f06a0155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f06a4e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06a015c770 0x7f06a00cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f06a4e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a015cb50 0x7f06a00cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f06a4e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f06a00c8bc0 con 0x7f06a00d11d0 2024-05-28T13:58:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f06a4e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f06a00ce2c0 con 0x7f06a015cb50 2024-05-28T13:58:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f06a4e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f06a00c8d40 con 0x7f06a015c770 2024-05-28T13:58:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f069de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a00d11d0 0x7f06a0155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f069de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a00d11d0 0x7f06a0155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38900/0 (socket says 172.21.15.42:38900) 2024-05-28T13:58:54.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f069de00640 1 -- 172.21.15.42:0/1660262566 learned_addr learned my addr 172.21.15.42:0/1660262566 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f0695400640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06a015c770 0x7f06a00cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f069e800640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a015cb50 0x7f06a00cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f069de00640 1 -- 172.21.15.42:0/1660262566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06a015c770 msgr2=0x7f06a00cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f069de00640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06a015c770 0x7f06a00cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f069de00640 1 -- 172.21.15.42:0/1660262566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a015cb50 msgr2=0x7f06a00cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f069de00640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a015cb50 0x7f06a00cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f069de00640 1 -- 172.21.15.42:0/1660262566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a00d1cb0 con 0x7f06a00d11d0 2024-05-28T13:58:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f0695400640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06a015c770 0x7f06a00cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f069e800640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a015cb50 0x7f06a00cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f069de00640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a00d11d0 0x7f06a0155b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f06900b3040 tx=0x7f069001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f0696a00640 1 -- 172.21.15.42:0/1660262566 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06900bf070 con 0x7f06a00d11d0 2024-05-28T13:58:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f0696a00640 1 -- 172.21.15.42:0/1660262566 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06900b0640 con 0x7f06a00d11d0 2024-05-28T13:58:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f0696a00640 1 -- 172.21.15.42:0/1660262566 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06900b5040 con 0x7f06a00d11d0 2024-05-28T13:58:54.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f06a4e00640 1 -- 172.21.15.42:0/1660262566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a015aa50 con 0x7f06a00d11d0 2024-05-28T13:58:54.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.917+0000 7f06a4e00640 1 -- 172.21.15.42:0/1660262566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a0156890 con 0x7f06a00d11d0 2024-05-28T13:58:54.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.921+0000 7f06a4e00640 1 -- 172.21.15.42:0/1660262566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a00d0f80 con 0x7f06a00d11d0 2024-05-28T13:58:54.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.921+0000 7f0696a00640 1 -- 172.21.15.42:0/1660262566 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f06900b0a30 con 0x7f06a00d11d0 2024-05-28T13:58:54.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.921+0000 7f0696a00640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f067c068380 0x7f067c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:54.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.921+0000 7f0696a00640 1 -- 172.21.15.42:0/1660262566 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f06901232a0 con 0x7f06a00d11d0 2024-05-28T13:58:54.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.921+0000 7f0695400640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f067c068380 0x7f067c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:54.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.921+0000 7f0695400640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f067c068380 0x7f067c06a840 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0688007fd0 tx=0x7f0688009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:54.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:54.925+0000 7f0696a00640 1 -- 172.21.15.42:0/1660262566 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f06900187d0 con 0x7f06a00d11d0 2024-05-28T13:58:55.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.089+0000 7f06a4e00640 1 -- 172.21.15.42:0/1660262566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f06a00ced40 con 0x7f06a00d11d0 2024-05-28T13:58:55.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.089+0000 7f0696a00640 1 -- 172.21.15.42:0/1660262566 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f06900f3860 con 0x7f06a00d11d0 2024-05-28T13:58:55.076 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:58:55.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.093+0000 7f06a4e00640 1 -- 172.21.15.42:0/1660262566 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f067c068380 msgr2=0x7f067c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:55.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.093+0000 7f06a4e00640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f067c068380 0x7f067c06a840 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0688007fd0 tx=0x7f0688009040 comp rx=0 tx=0).stop 2024-05-28T13:58:55.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.093+0000 7f06a4e00640 1 -- 172.21.15.42:0/1660262566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a00d11d0 msgr2=0x7f06a0155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:55.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.093+0000 7f06a4e00640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a00d11d0 0x7f06a0155b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f06900b3040 tx=0x7f069001ba10 comp rx=0 tx=0).stop 2024-05-28T13:58:55.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.093+0000 7f06a4e00640 1 -- 172.21.15.42:0/1660262566 shutdown_connections 2024-05-28T13:58:55.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.093+0000 7f06a4e00640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f067c068380 0x7f067c06a840 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.093+0000 7f06a4e00640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06a015cb50 0x7f06a00cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.093+0000 7f06a4e00640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06a015c770 0x7f06a00cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.093+0000 7f06a4e00640 1 --2- 172.21.15.42:0/1660262566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06a00d11d0 0x7f06a0155b30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.093+0000 7f06a4e00640 1 -- 172.21.15.42:0/1660262566 >> 172.21.15.42:0/1660262566 conn(0x7f06a00b3fd0 msgr2=0x7f06a00700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:55.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.093+0000 7f06a4e00640 1 -- 172.21.15.42:0/1660262566 shutdown_connections 2024-05-28T13:58:55.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.093+0000 7f06a4e00640 1 -- 172.21.15.42:0/1660262566 wait complete. 2024-05-28T13:58:55.167 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that when one clone is in progress, one progress bar is printed 2024-05-28T13:58:55.168 INFO:tasks.cephfs_test_runner:test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-28T13:58:55.168 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:55.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0c1000640 1 Processor -- start 2024-05-28T13:58:55.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0c1000640 1 -- start start 2024-05-28T13:58:55.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0c1000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb0bc161d80 0x7fb0bc0b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0c1000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0bc0cf390 0x7fb0bc0d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0c1000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0bc1631f0 con 0x7fb0bc0d0240 2024-05-28T13:58:55.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0c1000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb0bc162f10 con 0x7fb0bc161d80 2024-05-28T13:58:55.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0c1000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0bc0b3770 con 0x7fb0bc0cf390 2024-05-28T13:58:55.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0b3e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb0bc0d0240 0x7fb0bc0cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50604/0 (socket says 172.21.15.42:50604) 2024-05-28T13:58:55.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0b3e00640 1 -- 172.21.15.42:0/819105574 learned_addr learned my addr 172.21.15.42:0/819105574 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:55.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0bb400640 1 --2- 172.21.15.42:0/819105574 >> v2:172.21.15.63:3300/0 conn(0x7fb0bc161d80 0x7fb0bc0b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0bbe00640 1 --2- 172.21.15.42:0/819105574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0bc0cf390 0x7fb0bc0d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 -- 172.21.15.42:0/819105574 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 316801777 0 0) 0x7fb0bc1631f0 con 0x7fb0bc0d0240 2024-05-28T13:58:55.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 -- 172.21.15.42:0/819105574 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0a0003530 con 0x7fb0bc0d0240 2024-05-28T13:58:55.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 -- 172.21.15.42:0/819105574 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4229674899 0 0) 0x7fb0a0003530 con 0x7fb0bc0d0240 2024-05-28T13:58:55.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 -- 172.21.15.42:0/819105574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0bc0cf390 msgr2=0x7fb0bc0d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:55.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 --2- 172.21.15.42:0/819105574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0bc0cf390 0x7fb0bc0d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 -- 172.21.15.42:0/819105574 >> v2:172.21.15.63:3300/0 conn(0x7fb0bc161d80 msgr2=0x7fb0bc0b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:55.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 --2- 172.21.15.42:0/819105574 >> v2:172.21.15.63:3300/0 conn(0x7fb0bc161d80 0x7fb0bc0b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 -- 172.21.15.42:0/819105574 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc0b3a50 con 0x7fb0bc0d0240 2024-05-28T13:58:55.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 -- 172.21.15.42:0/819105574 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fb0a4001680 con 0x7fb0bc0d0240 2024-05-28T13:58:55.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 -- 172.21.15.42:0/819105574 >> v1:172.21.15.42:6789/0 conn(0x7fb0bc0d0240 legacy=0x7fb0bc0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:55.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 --2- 172.21.15.42:0/819105574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a00039c0 0x7fb0a0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 --2- 172.21.15.42:0/819105574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0a0004810 0x7fb0a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 --2- 172.21.15.42:0/819105574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0bc0cf390 0x7fb0a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 -- 172.21.15.42:0/819105574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc162f10 con 0x7fb0a00039c0 2024-05-28T13:58:55.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 -- 172.21.15.42:0/819105574 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc0b3770 con 0x7fb0a0004810 2024-05-28T13:58:55.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.325+0000 7fb0baa00640 1 -- 172.21.15.42:0/819105574 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a000b9d0 con 0x7fb0bc0cf390 2024-05-28T13:58:55.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.329+0000 7fb0bb400640 1 --2- 172.21.15.42:0/819105574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a00039c0 0x7fb0a0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.329+0000 7fb0b3e00640 1 --2- 172.21.15.42:0/819105574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0a0004810 0x7fb0a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.329+0000 7fb0bbe00640 1 --2- 172.21.15.42:0/819105574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0bc0cf390 0x7fb0a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.329+0000 7fb0bb400640 1 -- 172.21.15.42:0/819105574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0bc0cf390 msgr2=0x7fb0a00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:55.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.329+0000 7fb0bb400640 1 --2- 172.21.15.42:0/819105574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0bc0cf390 0x7fb0a00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.329+0000 7fb0bb400640 1 -- 172.21.15.42:0/819105574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0a0004810 msgr2=0x7fb0a0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:55.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.329+0000 7fb0bb400640 1 --2- 172.21.15.42:0/819105574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0a0004810 0x7fb0a0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.329+0000 7fb0bb400640 1 -- 172.21.15.42:0/819105574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a0003530 con 0x7fb0a00039c0 2024-05-28T13:58:55.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.329+0000 7fb0bb400640 1 --2- 172.21.15.42:0/819105574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a00039c0 0x7fb0a0003de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009ae0 tx=0x7fb0ac01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7b311ec3bfc3f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:55.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.329+0000 7fb0baa00640 1 -- 172.21.15.42:0/819105574 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac0096b0 con 0x7fb0a00039c0 2024-05-28T13:58:55.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.329+0000 7fb0baa00640 1 -- 172.21.15.42:0/819105574 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ac0a64c0 con 0x7fb0a00039c0 2024-05-28T13:58:55.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.329+0000 7fb0baa00640 1 -- 172.21.15.42:0/819105574 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac0a68b0 con 0x7fb0a00039c0 2024-05-28T13:58:55.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0c1000640 1 -- 172.21.15.42:0/819105574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a00039c0 msgr2=0x7fb0a0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:55.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0c1000640 1 --2- 172.21.15.42:0/819105574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a00039c0 0x7fb0a0003de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009ae0 tx=0x7fb0ac01cab0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0bbe00640 1 -- 172.21.15.42:0/819105574 reap_dead start 2024-05-28T13:58:55.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0c1000640 1 -- 172.21.15.42:0/819105574 shutdown_connections 2024-05-28T13:58:55.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0c1000640 1 -- 172.21.15.42:0/819105574 >> 172.21.15.42:0/819105574 conn(0x7fb0bc0b7a30 msgr2=0x7fb0bc0b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:55.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0c1000640 1 -- 172.21.15.42:0/819105574 shutdown_connections 2024-05-28T13:58:55.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0c1000640 1 -- 172.21.15.42:0/819105574 wait complete. 2024-05-28T13:58:55.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0c1000640 1 Processor -- start 2024-05-28T13:58:55.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0c1000640 1 -- start start 2024-05-28T13:58:55.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0c1000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0a00039c0 0x7fb0bc15e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0c1000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0bc152450 0x7fb0bc1528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0c1000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0bc0b63e0 0x7fb0bc0b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0c1000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc09b630 con 0x7fb0bc152450 2024-05-28T13:58:55.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0c1000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc09b4b0 con 0x7fb0a00039c0 2024-05-28T13:58:55.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0c1000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0bc169680 con 0x7fb0bc0b63e0 2024-05-28T13:58:55.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0b3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0bc152450 0x7fb0bc1528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0b3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0bc152450 0x7fb0bc1528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38904/0 (socket says 172.21.15.42:38904) 2024-05-28T13:58:55.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0b3e00640 1 -- 172.21.15.42:0/2899420375 learned_addr learned my addr 172.21.15.42:0/2899420375 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:55.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0bb400640 1 --2- 172.21.15.42:0/2899420375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0a00039c0 0x7fb0bc15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0bbe00640 1 --2- 172.21.15.42:0/2899420375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0bc0b63e0 0x7fb0bc0b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0b3e00640 1 -- 172.21.15.42:0/2899420375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0bc0b63e0 msgr2=0x7fb0bc0b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:55.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0b3e00640 1 --2- 172.21.15.42:0/2899420375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0bc0b63e0 0x7fb0bc0b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0b3e00640 1 -- 172.21.15.42:0/2899420375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0a00039c0 msgr2=0x7fb0bc15e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:55.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0b3e00640 1 --2- 172.21.15.42:0/2899420375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0a00039c0 0x7fb0bc15e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0b3e00640 1 -- 172.21.15.42:0/2899420375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc0af370 con 0x7fb0bc152450 2024-05-28T13:58:55.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.333+0000 7fb0b3e00640 1 --2- 172.21.15.42:0/2899420375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0bc152450 0x7fb0bc1528d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4001060 tx=0x7fb0a400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:55.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.337+0000 7fb0b3400640 1 -- 172.21.15.42:0/2899420375 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400ee80 con 0x7fb0bc152450 2024-05-28T13:58:55.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.337+0000 7fb0b3400640 1 -- 172.21.15.42:0/2899420375 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a4018070 con 0x7fb0bc152450 2024-05-28T13:58:55.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.337+0000 7fb0b3400640 1 -- 172.21.15.42:0/2899420375 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400caf0 con 0x7fb0bc152450 2024-05-28T13:58:55.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.337+0000 7fb0c1000640 1 -- 172.21.15.42:0/2899420375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0bc0af650 con 0x7fb0bc152450 2024-05-28T13:58:55.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.337+0000 7fb0c1000640 1 -- 172.21.15.42:0/2899420375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0bc0afc10 con 0x7fb0bc152450 2024-05-28T13:58:55.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.337+0000 7fb0c1000640 1 -- 172.21.15.42:0/2899420375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0bc04fcf0 con 0x7fb0bc152450 2024-05-28T13:58:55.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.337+0000 7fb0b3400640 1 -- 172.21.15.42:0/2899420375 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb0a400cc90 con 0x7fb0bc152450 2024-05-28T13:58:55.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.337+0000 7fb0b3400640 1 --2- 172.21.15.42:0/2899420375 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb0980683b0 0x7fb09806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.337+0000 7fb0b3400640 1 -- 172.21.15.42:0/2899420375 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb0a40866c0 con 0x7fb0bc152450 2024-05-28T13:58:55.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.337+0000 7fb0bb400640 1 --2- 172.21.15.42:0/2899420375 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb0980683b0 0x7fb09806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.341+0000 7fb0bb400640 1 --2- 172.21.15.42:0/2899420375 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb0980683b0 0x7fb09806a870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0af000 tx=0x7fb0ac0046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:55.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.341+0000 7fb0b3400640 1 -- 172.21.15.42:0/2899420375 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb0a40566c0 con 0x7fb0bc152450 2024-05-28T13:58:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.505+0000 7fb0c1000640 1 -- 172.21.15.42:0/2899420375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0bc06d3a0 con 0x7fb0bc152450 2024-05-28T13:58:55.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.509+0000 7fb0b3400640 1 -- 172.21.15.42:0/2899420375 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16545 (secure 0 0 0) 0x7fb0a405a9c0 con 0x7fb0bc152450 2024-05-28T13:58:55.494 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:55.494 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:24.659233+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.509+0000 7fb0c1000640 1 -- 172.21.15.42:0/2899420375 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb0980683b0 msgr2=0x7fb09806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.509+0000 7fb0c1000640 1 --2- 172.21.15.42:0/2899420375 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb0980683b0 0x7fb09806a870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0af000 tx=0x7fb0ac0046d0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.509+0000 7fb0c1000640 1 -- 172.21.15.42:0/2899420375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0bc152450 msgr2=0x7fb0bc1528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.509+0000 7fb0c1000640 1 --2- 172.21.15.42:0/2899420375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0bc152450 0x7fb0bc1528d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4001060 tx=0x7fb0a400ec30 comp rx=0 tx=0).stop 2024-05-28T13:58:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.509+0000 7fb0c1000640 1 -- 172.21.15.42:0/2899420375 shutdown_connections 2024-05-28T13:58:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.509+0000 7fb0c1000640 1 --2- 172.21.15.42:0/2899420375 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb0980683b0 0x7fb09806a870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.509+0000 7fb0c1000640 1 --2- 172.21.15.42:0/2899420375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0bc0b63e0 0x7fb0bc0b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.509+0000 7fb0c1000640 1 --2- 172.21.15.42:0/2899420375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0bc152450 0x7fb0bc1528d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.509+0000 7fb0c1000640 1 --2- 172.21.15.42:0/2899420375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0a00039c0 0x7fb0bc15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.509+0000 7fb0c1000640 1 -- 172.21.15.42:0/2899420375 >> 172.21.15.42:0/2899420375 conn(0x7fb0bc0b7a30 msgr2=0x7fb0bc06ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.509+0000 7fb0c1000640 1 -- 172.21.15.42:0/2899420375 shutdown_connections 2024-05-28T13:58:55.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.509+0000 7fb0c1000640 1 -- 172.21.15.42:0/2899420375 wait complete. 2024-05-28T13:58:55.515 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:58:55.516 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:58:55.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.673+0000 7f36b1600640 1 Processor -- start 2024-05-28T13:58:55.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.673+0000 7f36b1600640 1 -- start start 2024-05-28T13:58:55.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.677+0000 7f36b1600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f36ac0cdcf0 0x7f36ac0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.677+0000 7f36b1600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f36ac0d0d40 0x7f36ac06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.677+0000 7f36b1600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36ac0cd100 con 0x7f36ac070a10 2024-05-28T13:58:55.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.677+0000 7f36b1600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f36ac0ccc70 con 0x7f36ac0d0d40 2024-05-28T13:58:55.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.677+0000 7f36b1600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f36ac0cc060 con 0x7f36ac0cdcf0 2024-05-28T13:58:55.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.677+0000 7f36ab400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f36ac0d0d40 0x7f36ac06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.677+0000 7f36ab400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f36ac0d0d40 0x7f36ac06eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38916/0 (socket says 172.21.15.42:38916) 2024-05-28T13:58:55.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.677+0000 7f36ab400640 1 -- 172.21.15.42:0/1289917366 learned_addr learned my addr 172.21.15.42:0/1289917366 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:55.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.677+0000 7f36aa000640 1 --2- 172.21.15.42:0/1289917366 >> v2:172.21.15.63:3300/0 conn(0x7f36ac0cdcf0 0x7f36ac0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.677+0000 7f36ab400640 1 -- 172.21.15.42:0/1289917366 >> v2:172.21.15.63:3300/0 conn(0x7f36ac0cdcf0 msgr2=0x7f36ac0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:55.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.677+0000 7f36ab400640 1 --2- 172.21.15.42:0/1289917366 >> v2:172.21.15.63:3300/0 conn(0x7f36ac0cdcf0 0x7f36ac0cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.677+0000 7f36ab400640 1 -- 172.21.15.42:0/1289917366 >> v1:172.21.15.63:6789/0 conn(0x7f36ac070a10 legacy=0x7f36ac0d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:55.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.677+0000 7f36ab400640 1 -- 172.21.15.42:0/1289917366 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ac0cc340 con 0x7f36ac0d0d40 2024-05-28T13:58:55.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.677+0000 7f36a9600640 1 -- 172.21.15.42:0/1289917366 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2369722551 0 0) 0x7f36ac0cd100 con 0x7f36ac070a10 2024-05-28T13:58:55.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.681+0000 7f36ab400640 1 --2- 172.21.15.42:0/1289917366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ac0d0d40 0x7f36ac06eef0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0d2440 tx=0x7f36a401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c8f0996d34ce36 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:55.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.681+0000 7f36a9600640 1 -- 172.21.15.42:0/1289917366 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a40a5710 con 0x7f36ac0d0d40 2024-05-28T13:58:55.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.681+0000 7f36a9600640 1 -- 172.21.15.42:0/1289917366 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a40a58b0 con 0x7f36ac0d0d40 2024-05-28T13:58:55.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.681+0000 7f36a9600640 1 -- 172.21.15.42:0/1289917366 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a40a5c80 con 0x7f36ac0d0d40 2024-05-28T13:58:55.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.681+0000 7f36b1600640 1 -- 172.21.15.42:0/1289917366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ac0d0d40 msgr2=0x7f36ac06eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:55.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.681+0000 7f36b1600640 1 --2- 172.21.15.42:0/1289917366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ac0d0d40 0x7f36ac06eef0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0d2440 tx=0x7f36a401c960 comp rx=0 tx=0).stop 2024-05-28T13:58:55.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.681+0000 7f36b1600640 1 -- 172.21.15.42:0/1289917366 shutdown_connections 2024-05-28T13:58:55.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.681+0000 7f36b1600640 1 --2- 172.21.15.42:0/1289917366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ac0d0d40 0x7f36ac06eef0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.681+0000 7f36b1600640 1 --2- 172.21.15.42:0/1289917366 >> v2:172.21.15.63:3300/0 conn(0x7f36ac0cdcf0 0x7f36ac0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.681+0000 7f36b1600640 1 -- 172.21.15.42:0/1289917366 >> 172.21.15.42:0/1289917366 conn(0x7f36ac0b2fd0 msgr2=0x7f36ac0b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:55.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.681+0000 7f36b1600640 1 -- 172.21.15.42:0/1289917366 shutdown_connections 2024-05-28T13:58:55.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.681+0000 7f36b1600640 1 -- 172.21.15.42:0/1289917366 wait complete. 2024-05-28T13:58:55.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36b1600640 1 Processor -- start 2024-05-28T13:58:55.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36b1600640 1 -- start start 2024-05-28T13:58:55.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36b1600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ac070a10 0x7f36ac0b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36b1600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36ac0cdcf0 0x7f36ac15a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36b1600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ac0d0d40 0x7f36ac0b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36b1600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac070f70 con 0x7f36ac070a10 2024-05-28T13:58:55.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36b1600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac070df0 con 0x7f36ac0cdcf0 2024-05-28T13:58:55.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36b1600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f36ac06fc40 con 0x7f36ac0d0d40 2024-05-28T13:58:55.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ac070a10 0x7f36ac0b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ac070a10 0x7f36ac0b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38918/0 (socket says 172.21.15.42:38918) 2024-05-28T13:58:55.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36aaa00640 1 -- 172.21.15.42:0/2367726400 learned_addr learned my addr 172.21.15.42:0/2367726400 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:55.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36aa000640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36ac0cdcf0 0x7f36ac15a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36ab400640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ac0d0d40 0x7f36ac0b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36aa000640 1 -- 172.21.15.42:0/2367726400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ac0d0d40 msgr2=0x7f36ac0b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:55.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36aa000640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ac0d0d40 0x7f36ac0b5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36aa000640 1 -- 172.21.15.42:0/2367726400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ac070a10 msgr2=0x7f36ac0b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:55.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36aa000640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ac070a10 0x7f36ac0b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36aa000640 1 -- 172.21.15.42:0/2367726400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ac0b5c80 con 0x7f36ac0cdcf0 2024-05-28T13:58:55.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36aaa00640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ac070a10 0x7f36ac0b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:55.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36aa000640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36ac0cdcf0 0x7f36ac15a160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f369800e870 tx=0x7f369800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:55.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36a2a00640 1 -- 172.21.15.42:0/2367726400 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698017070 con 0x7f36ac0cdcf0 2024-05-28T13:58:55.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36a2a00640 1 -- 172.21.15.42:0/2367726400 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f369800c940 con 0x7f36ac0cdcf0 2024-05-28T13:58:55.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36b1600640 1 -- 172.21.15.42:0/2367726400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ac0b46a0 con 0x7f36ac0cdcf0 2024-05-28T13:58:55.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36a2a00640 1 -- 172.21.15.42:0/2367726400 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369800ccf0 con 0x7f36ac0cdcf0 2024-05-28T13:58:55.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.685+0000 7f36b1600640 1 -- 172.21.15.42:0/2367726400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ac0b4c60 con 0x7f36ac0cdcf0 2024-05-28T13:58:55.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.689+0000 7f36b1600640 1 -- 172.21.15.42:0/2367726400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36ac09f630 con 0x7f36ac0cdcf0 2024-05-28T13:58:55.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.689+0000 7f36a2a00640 1 -- 172.21.15.42:0/2367726400 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f369801a070 con 0x7f36ac0cdcf0 2024-05-28T13:58:55.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.689+0000 7f36a2a00640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f368c068380 0x7f368c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:55.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.689+0000 7f36a2a00640 1 -- 172.21.15.42:0/2367726400 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3698087110 con 0x7f36ac0cdcf0 2024-05-28T13:58:55.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.689+0000 7f36aaa00640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f368c068380 0x7f368c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:55.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.689+0000 7f36aaa00640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f368c068380 0x7f368c06a840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f369c001ca0 tx=0x7f369c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:55.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.693+0000 7f36a2a00640 1 -- 172.21.15.42:0/2367726400 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3698057060 con 0x7f36ac0cdcf0 2024-05-28T13:58:55.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.861+0000 7f36b1600640 1 -- 172.21.15.42:0/2367726400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f36ac04fdd0 con 0x7f36ac0cdcf0 2024-05-28T13:58:55.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.865+0000 7f36a2a00640 1 -- 172.21.15.42:0/2367726400 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f369805b360 con 0x7f36ac0cdcf0 2024-05-28T13:58:55.849 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:58:55.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.865+0000 7f36b1600640 1 -- 172.21.15.42:0/2367726400 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f368c068380 msgr2=0x7f368c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:55.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.865+0000 7f36b1600640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f368c068380 0x7f368c06a840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f369c001ca0 tx=0x7f369c009040 comp rx=0 tx=0).stop 2024-05-28T13:58:55.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.865+0000 7f36b1600640 1 -- 172.21.15.42:0/2367726400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36ac0cdcf0 msgr2=0x7f36ac15a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:55.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.865+0000 7f36b1600640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36ac0cdcf0 0x7f36ac15a160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f369800e870 tx=0x7f369800ed30 comp rx=0 tx=0).stop 2024-05-28T13:58:55.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.865+0000 7f36b1600640 1 -- 172.21.15.42:0/2367726400 shutdown_connections 2024-05-28T13:58:55.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.865+0000 7f36b1600640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f368c068380 0x7f368c06a840 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.865+0000 7f36b1600640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36ac0d0d40 0x7f36ac0b5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.865+0000 7f36b1600640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36ac0cdcf0 0x7f36ac15a160 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.865+0000 7f36b1600640 1 --2- 172.21.15.42:0/2367726400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36ac070a10 0x7f36ac0b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:55.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.865+0000 7f36b1600640 1 -- 172.21.15.42:0/2367726400 >> 172.21.15.42:0/2367726400 conn(0x7f36ac0b2fd0 msgr2=0x7f36ac06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:55.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.865+0000 7f36b1600640 1 -- 172.21.15.42:0/2367726400 shutdown_connections 2024-05-28T13:58:55.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:55.865+0000 7f36b1600640 1 -- 172.21.15.42:0/2367726400 wait complete. 2024-05-28T13:58:55.870 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:58:56.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80bb400640 1 Processor -- start 2024-05-28T13:58:56.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80bb400640 1 -- start start 2024-05-28T13:58:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80bb400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80b4071de0 0x7f80b40721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80bb400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f80b40ce3b0 0x7f80b40ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80bb400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80b4072700 con 0x7f80b4141d10 2024-05-28T13:58:56.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80bb400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f80b4160230 con 0x7f80b40ce3b0 2024-05-28T13:58:56.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80bb400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80b4160510 con 0x7f80b4071de0 2024-05-28T13:58:56.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80b8a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80b4071de0 0x7f80b40721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80abe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f80b40ce3b0 0x7f80b40ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80b8a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80b4071de0 0x7f80b40721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:45996/0 (socket says 172.21.15.42:45996) 2024-05-28T13:58:56.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80b8a00640 1 -- 172.21.15.42:0/2144804220 learned_addr learned my addr 172.21.15.42:0/2144804220 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:56.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80b8a00640 1 -- 172.21.15.42:0/2144804220 >> v2:172.21.15.63:3300/0 conn(0x7f80b40ce3b0 msgr2=0x7f80b40ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:56.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80b8a00640 1 --2- 172.21.15.42:0/2144804220 >> v2:172.21.15.63:3300/0 conn(0x7f80b40ce3b0 0x7f80b40ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80b8a00640 1 -- 172.21.15.42:0/2144804220 >> v1:172.21.15.63:6789/0 conn(0x7f80b4141d10 legacy=0x7f80b41421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.029+0000 7f80b8a00640 1 -- 172.21.15.42:0/2144804220 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b41607f0 con 0x7f80b4071de0 2024-05-28T13:58:56.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.033+0000 7f80b8a00640 1 --2- 172.21.15.42:0/2144804220 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80b4071de0 0x7f80b40721c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009870 tx=0x7f80a401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f81dd3527bc61dfd server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:56.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.033+0000 7f80b3e00640 1 -- 172.21.15.42:0/2144804220 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a40b0070 con 0x7f80b4071de0 2024-05-28T13:58:56.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.033+0000 7f80b3e00640 1 -- 172.21.15.42:0/2144804220 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a40ab600 con 0x7f80b4071de0 2024-05-28T13:58:56.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.033+0000 7f80b3e00640 1 -- 172.21.15.42:0/2144804220 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a40ab9d0 con 0x7f80b4071de0 2024-05-28T13:58:56.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.033+0000 7f80bb400640 1 -- 172.21.15.42:0/2144804220 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80b4071de0 msgr2=0x7f80b40721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.033+0000 7f80bb400640 1 --2- 172.21.15.42:0/2144804220 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80b4071de0 0x7f80b40721c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009870 tx=0x7f80a401c440 comp rx=0 tx=0).stop 2024-05-28T13:58:56.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.033+0000 7f80bb400640 1 -- 172.21.15.42:0/2144804220 shutdown_connections 2024-05-28T13:58:56.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.033+0000 7f80bb400640 1 --2- 172.21.15.42:0/2144804220 >> v2:172.21.15.63:3300/0 conn(0x7f80b40ce3b0 0x7f80b40ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.033+0000 7f80bb400640 1 --2- 172.21.15.42:0/2144804220 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80b4071de0 0x7f80b40721c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.033+0000 7f80bb400640 1 -- 172.21.15.42:0/2144804220 >> 172.21.15.42:0/2144804220 conn(0x7f80b40b21f0 msgr2=0x7f80b40b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:56.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.033+0000 7f80bb400640 1 -- 172.21.15.42:0/2144804220 shutdown_connections 2024-05-28T13:58:56.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.033+0000 7f80bb400640 1 -- 172.21.15.42:0/2144804220 wait complete. 2024-05-28T13:58:56.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80bb400640 1 Processor -- start 2024-05-28T13:58:56.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80bb400640 1 -- start start 2024-05-28T13:58:56.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80bb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80b4071de0 0x7f80b413e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80bb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80b40ce3b0 0x7f80b4155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80bb400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80b4141d10 0x7f80b413d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80bb400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4169e30 con 0x7f80b40ce3b0 2024-05-28T13:58:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80bb400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4169cb0 con 0x7f80b4071de0 2024-05-28T13:58:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80bb400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80b4169fb0 con 0x7f80b4141d10 2024-05-28T13:58:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80b40ce3b0 0x7f80b4155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80b40ce3b0 0x7f80b4155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38924/0 (socket says 172.21.15.42:38924) 2024-05-28T13:58:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80abe00640 1 -- 172.21.15.42:0/2453100011 learned_addr learned my addr 172.21.15.42:0/2453100011 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80b9400640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80b4141d10 0x7f80b413d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80b8a00640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80b4071de0 0x7f80b413e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80abe00640 1 -- 172.21.15.42:0/2453100011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80b4141d10 msgr2=0x7f80b413d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80abe00640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80b4141d10 0x7f80b413d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80abe00640 1 -- 172.21.15.42:0/2453100011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80b4071de0 msgr2=0x7f80b413e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80abe00640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80b4071de0 0x7f80b413e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80abe00640 1 -- 172.21.15.42:0/2453100011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b413eb10 con 0x7f80b40ce3b0 2024-05-28T13:58:56.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80b9400640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80b4141d10 0x7f80b413d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:56.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80abe00640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80b40ce3b0 0x7f80b4155560 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f809c00e870 tx=0x7f809c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:56.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80b1600640 1 -- 172.21.15.42:0/2453100011 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c017070 con 0x7f80b40ce3b0 2024-05-28T13:58:56.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80b1600640 1 -- 172.21.15.42:0/2453100011 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f809c00c940 con 0x7f80b40ce3b0 2024-05-28T13:58:56.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80bb400640 1 -- 172.21.15.42:0/2453100011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b413af30 con 0x7f80b40ce3b0 2024-05-28T13:58:56.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80b1600640 1 -- 172.21.15.42:0/2453100011 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c00ccf0 con 0x7f80b40ce3b0 2024-05-28T13:58:56.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.037+0000 7f80bb400640 1 -- 172.21.15.42:0/2453100011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b413b4f0 con 0x7f80b40ce3b0 2024-05-28T13:58:56.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.041+0000 7f80bb400640 1 -- 172.21.15.42:0/2453100011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b40738b0 con 0x7f80b40ce3b0 2024-05-28T13:58:56.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.041+0000 7f80b1600640 1 -- 172.21.15.42:0/2453100011 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f809c01a070 con 0x7f80b40ce3b0 2024-05-28T13:58:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.041+0000 7f80b1600640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80900683b0 0x7f809006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.041+0000 7f80b1600640 1 -- 172.21.15.42:0/2453100011 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f809c086ba0 con 0x7f80b40ce3b0 2024-05-28T13:58:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.041+0000 7f80b8a00640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80900683b0 0x7f809006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.041+0000 7f80b8a00640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80900683b0 0x7f809006a870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f80a40ad000 tx=0x7f80a40a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:56.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.045+0000 7f80b1600640 1 -- 172.21.15.42:0/2453100011 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f809c056b00 con 0x7f80b40ce3b0 2024-05-28T13:58:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.209+0000 7f80bb400640 1 -- 172.21.15.42:0/2453100011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f80b404fdd0 con 0x7f80b40ce3b0 2024-05-28T13:58:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.213+0000 7f80b1600640 1 -- 172.21.15.42:0/2453100011 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f809c05ae00 con 0x7f80b40ce3b0 2024-05-28T13:58:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:58:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.213+0000 7f80bb400640 1 -- 172.21.15.42:0/2453100011 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80900683b0 msgr2=0x7f809006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.213+0000 7f80bb400640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80900683b0 0x7f809006a870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f80a40ad000 tx=0x7f80a40a6000 comp rx=0 tx=0).stop 2024-05-28T13:58:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.213+0000 7f80bb400640 1 -- 172.21.15.42:0/2453100011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80b40ce3b0 msgr2=0x7f80b4155560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.213+0000 7f80bb400640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80b40ce3b0 0x7f80b4155560 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f809c00e870 tx=0x7f809c00ec70 comp rx=0 tx=0).stop 2024-05-28T13:58:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.213+0000 7f80bb400640 1 -- 172.21.15.42:0/2453100011 shutdown_connections 2024-05-28T13:58:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.213+0000 7f80bb400640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80900683b0 0x7f809006a870 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.213+0000 7f80bb400640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80b4141d10 0x7f80b413d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.213+0000 7f80bb400640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80b40ce3b0 0x7f80b4155560 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.213+0000 7f80bb400640 1 --2- 172.21.15.42:0/2453100011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80b4071de0 0x7f80b413e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.213+0000 7f80bb400640 1 -- 172.21.15.42:0/2453100011 >> 172.21.15.42:0/2453100011 conn(0x7f80b40b21f0 msgr2=0x7f80b415bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.213+0000 7f80bb400640 1 -- 172.21.15.42:0/2453100011 shutdown_connections 2024-05-28T13:58:56.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.213+0000 7f80bb400640 1 -- 172.21.15.42:0/2453100011 wait complete. 2024-05-28T13:58:56.218 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:58:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f7571c00640 1 Processor -- start 2024-05-28T13:58:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f7571c00640 1 -- start start 2024-05-28T13:58:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f7571c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f756c0d11d0 0x7f756c09b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f7571c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f756c09ba40 con 0x7f756c15c770 2024-05-28T13:58:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f7571c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f756c0c60d0 con 0x7f756c15cb50 2024-05-28T13:58:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f7571c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f756c0c6720 con 0x7f756c0d11d0 2024-05-28T13:58:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f756aa00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f756c15c770 0x7f756c0d0ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50618/0 (socket says 172.21.15.42:50618) 2024-05-28T13:58:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f756aa00640 1 -- 172.21.15.42:0/2509112465 learned_addr learned my addr 172.21.15.42:0/2509112465 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:56.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f756b400640 1 --2- 172.21.15.42:0/2509112465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f756c0d11d0 0x7f756c09b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f756a000640 1 -- 172.21.15.42:0/2509112465 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3622116769 0 0) 0x7f756c09ba40 con 0x7f756c15c770 2024-05-28T13:58:56.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f756a000640 1 -- 172.21.15.42:0/2509112465 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7550003530 con 0x7f756c15c770 2024-05-28T13:58:56.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f756a000640 1 -- 172.21.15.42:0/2509112465 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1037681163 0 0) 0x7f756c0c60d0 con 0x7f756c15cb50 2024-05-28T13:58:56.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f756b400640 1 -- 172.21.15.42:0/2509112465 >> v1:172.21.15.63:6789/0 conn(0x7f756c15cb50 legacy=0x7f756c164460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f756b400640 1 -- 172.21.15.42:0/2509112465 >> v1:172.21.15.42:6789/0 conn(0x7f756c15c770 legacy=0x7f756c0d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.377+0000 7f756b400640 1 -- 172.21.15.42:0/2509112465 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756c0c63b0 con 0x7f756c0d11d0 2024-05-28T13:58:56.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.381+0000 7f756b400640 1 --2- 172.21.15.42:0/2509112465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f756c0d11d0 0x7f756c09b460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f75640097f0 tx=0x7f756401ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb34952ea58d4bde server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:56.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.381+0000 7f756a000640 1 -- 172.21.15.42:0/2509112465 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75640b5070 con 0x7f756c0d11d0 2024-05-28T13:58:56.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.381+0000 7f756a000640 1 -- 172.21.15.42:0/2509112465 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75640a89d0 con 0x7f756c0d11d0 2024-05-28T13:58:56.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.381+0000 7f756a000640 1 -- 172.21.15.42:0/2509112465 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75640a8da0 con 0x7f756c0d11d0 2024-05-28T13:58:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.381+0000 7f7571c00640 1 -- 172.21.15.42:0/2509112465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f756c0d11d0 msgr2=0x7f756c09b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.381+0000 7f7571c00640 1 --2- 172.21.15.42:0/2509112465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f756c0d11d0 0x7f756c09b460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f75640097f0 tx=0x7f756401ecd0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.381+0000 7f7571c00640 1 -- 172.21.15.42:0/2509112465 shutdown_connections 2024-05-28T13:58:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.381+0000 7f7571c00640 1 --2- 172.21.15.42:0/2509112465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f756c0d11d0 0x7f756c09b460 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.381+0000 7f7571c00640 1 -- 172.21.15.42:0/2509112465 >> 172.21.15.42:0/2509112465 conn(0x7f756c0b3fd0 msgr2=0x7f756c0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.381+0000 7f7571c00640 1 -- 172.21.15.42:0/2509112465 shutdown_connections 2024-05-28T13:58:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.381+0000 7f7571c00640 1 -- 172.21.15.42:0/2509112465 wait complete. 2024-05-28T13:58:56.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f7571c00640 1 Processor -- start 2024-05-28T13:58:56.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f7571c00640 1 -- start start 2024-05-28T13:58:56.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f7571c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f756c0d11d0 0x7f756c14f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f7571c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f756c15c770 0x7f756c0b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f7571c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f756c15cb50 0x7f756c1503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f7571c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f756c0c8bc0 con 0x7f756c15cb50 2024-05-28T13:58:56.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f7571c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f756c0ce2c0 con 0x7f756c0d11d0 2024-05-28T13:58:56.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f7571c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f756c0c8d40 con 0x7f756c15c770 2024-05-28T13:58:56.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f756b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f756c15cb50 0x7f756c1503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f756b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f756c15cb50 0x7f756c1503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38932/0 (socket says 172.21.15.42:38932) 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f756b400640 1 -- 172.21.15.42:0/2112792797 learned_addr learned my addr 172.21.15.42:0/2112792797 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f7562000640 1 --2- 172.21.15.42:0/2112792797 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f756c15c770 0x7f756c0b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f756aa00640 1 --2- 172.21.15.42:0/2112792797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f756c0d11d0 0x7f756c14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f756b400640 1 -- 172.21.15.42:0/2112792797 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f756c15c770 msgr2=0x7f756c0b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f756b400640 1 --2- 172.21.15.42:0/2112792797 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f756c15c770 0x7f756c0b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f756b400640 1 -- 172.21.15.42:0/2112792797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f756c0d11d0 msgr2=0x7f756c14f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f756b400640 1 --2- 172.21.15.42:0/2112792797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f756c0d11d0 0x7f756c14f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f756b400640 1 -- 172.21.15.42:0/2112792797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756c06ff40 con 0x7f756c15cb50 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f756b400640 1 --2- 172.21.15.42:0/2112792797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f756c15cb50 0x7f756c1503a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f75640097f0 tx=0x7f756401ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f7563400640 1 -- 172.21.15.42:0/2112792797 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75640be070 con 0x7f756c15cb50 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f7563400640 1 -- 172.21.15.42:0/2112792797 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7564004230 con 0x7f756c15cb50 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f7571c00640 1 -- 172.21.15.42:0/2112792797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756c14fe80 con 0x7f756c15cb50 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f7563400640 1 -- 172.21.15.42:0/2112792797 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75640b5040 con 0x7f756c15cb50 2024-05-28T13:58:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.385+0000 7f7571c00640 1 -- 172.21.15.42:0/2112792797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756c0b6910 con 0x7f756c15cb50 2024-05-28T13:58:56.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.389+0000 7f7571c00640 1 -- 172.21.15.42:0/2112792797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f756c0a04d0 con 0x7f756c15cb50 2024-05-28T13:58:56.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.389+0000 7f7563400640 1 -- 172.21.15.42:0/2112792797 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7564003620 con 0x7f756c15cb50 2024-05-28T13:58:56.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.389+0000 7f7563400640 1 --2- 172.21.15.42:0/2112792797 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7548068380 0x7f754806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.389+0000 7f7563400640 1 -- 172.21.15.42:0/2112792797 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7564123260 con 0x7f756c15cb50 2024-05-28T13:58:56.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.389+0000 7f756aa00640 1 --2- 172.21.15.42:0/2112792797 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7548068380 0x7f754806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.389+0000 7f756aa00640 1 --2- 172.21.15.42:0/2112792797 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7548068380 0x7f754806a840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f755c001570 tx=0x7f755c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:56.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.393+0000 7f7563400640 1 -- 172.21.15.42:0/2112792797 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7564018830 con 0x7f756c15cb50 2024-05-28T13:58:56.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.557+0000 7f7571c00640 1 -- 172.21.15.42:0/2112792797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f756c0c89a0 con 0x7f756c15cb50 2024-05-28T13:58:56.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.557+0000 7f7563400640 1 -- 172.21.15.42:0/2112792797 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f75640a89a0 con 0x7f756c15cb50 2024-05-28T13:58:56.543 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:58:56.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.561+0000 7f7571c00640 1 -- 172.21.15.42:0/2112792797 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7548068380 msgr2=0x7f754806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.561+0000 7f7571c00640 1 --2- 172.21.15.42:0/2112792797 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7548068380 0x7f754806a840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f755c001570 tx=0x7f755c009040 comp rx=0 tx=0).stop 2024-05-28T13:58:56.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.561+0000 7f7571c00640 1 -- 172.21.15.42:0/2112792797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f756c15cb50 msgr2=0x7f756c1503a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.561+0000 7f7571c00640 1 --2- 172.21.15.42:0/2112792797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f756c15cb50 0x7f756c1503a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f75640097f0 tx=0x7f756401ba70 comp rx=0 tx=0).stop 2024-05-28T13:58:56.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.561+0000 7f7571c00640 1 -- 172.21.15.42:0/2112792797 shutdown_connections 2024-05-28T13:58:56.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.561+0000 7f7571c00640 1 --2- 172.21.15.42:0/2112792797 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7548068380 0x7f754806a840 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.561+0000 7f7571c00640 1 --2- 172.21.15.42:0/2112792797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f756c15cb50 0x7f756c1503a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.561+0000 7f7571c00640 1 --2- 172.21.15.42:0/2112792797 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f756c15c770 0x7f756c0b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.561+0000 7f7571c00640 1 --2- 172.21.15.42:0/2112792797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f756c0d11d0 0x7f756c14f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.561+0000 7f7571c00640 1 -- 172.21.15.42:0/2112792797 >> 172.21.15.42:0/2112792797 conn(0x7f756c0b3fd0 msgr2=0x7f756c0c6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:56.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.561+0000 7f7571c00640 1 -- 172.21.15.42:0/2112792797 shutdown_connections 2024-05-28T13:58:56.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.561+0000 7f7571c00640 1 -- 172.21.15.42:0/2112792797 wait complete. 2024-05-28T13:58:56.566 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone' 2024-05-28T13:58:56.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.721+0000 7f191ea00640 1 Processor -- start 2024-05-28T13:58:56.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.721+0000 7f191ea00640 1 -- start start 2024-05-28T13:58:56.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.721+0000 7f191ea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f191815e990 0x7f191809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.721+0000 7f191ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f19180b1a80 0x7f191809af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.721+0000 7f191ea00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19180d0330 con 0x7f191806d780 2024-05-28T13:58:56.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.721+0000 7f191ea00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f191806d2f0 con 0x7f19180b1a80 2024-05-28T13:58:56.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.721+0000 7f191ea00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f191806cca0 con 0x7f191815e990 2024-05-28T13:58:56.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.721+0000 7f190fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f19180b1a80 0x7f191809af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.721+0000 7f191ca00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f191806d780 0x7f191806dc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50628/0 (socket says 172.21.15.42:50628) 2024-05-28T13:58:56.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.721+0000 7f191ca00640 1 -- 172.21.15.42:0/3613665854 learned_addr learned my addr 172.21.15.42:0/3613665854 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:56.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.725+0000 7f1917e00640 1 --2- 172.21.15.42:0/3613665854 >> v2:172.21.15.63:3300/0 conn(0x7f191815e990 0x7f191809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.725+0000 7f190fe00640 1 -- 172.21.15.42:0/3613665854 >> v2:172.21.15.63:3300/0 conn(0x7f191815e990 msgr2=0x7f191809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:56.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.725+0000 7f190fe00640 1 --2- 172.21.15.42:0/3613665854 >> v2:172.21.15.63:3300/0 conn(0x7f191815e990 0x7f191809aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.725+0000 7f190fe00640 1 -- 172.21.15.42:0/3613665854 >> v1:172.21.15.42:6789/0 conn(0x7f191806d780 legacy=0x7f191806dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.725+0000 7f190fe00640 1 -- 172.21.15.42:0/3613665854 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191806cf80 con 0x7f19180b1a80 2024-05-28T13:58:56.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.725+0000 7f1917400640 1 -- 172.21.15.42:0/3613665854 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2900964147 0 0) 0x7f19180d0330 con 0x7f191806d780 2024-05-28T13:58:56.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.725+0000 7f190fe00640 1 --2- 172.21.15.42:0/3613665854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19180b1a80 0x7f191809af90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1918073bd0 tx=0x7f190001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb7177093ecac48 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:56.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.725+0000 7f1917400640 1 -- 172.21.15.42:0/3613665854 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19000b0070 con 0x7f19180b1a80 2024-05-28T13:58:56.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.725+0000 7f1917400640 1 -- 172.21.15.42:0/3613665854 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19000ab600 con 0x7f19180b1a80 2024-05-28T13:58:56.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.725+0000 7f1917400640 1 -- 172.21.15.42:0/3613665854 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19000ab9d0 con 0x7f19180b1a80 2024-05-28T13:58:56.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.725+0000 7f191ea00640 1 -- 172.21.15.42:0/3613665854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19180b1a80 msgr2=0x7f191809af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.725+0000 7f191ea00640 1 --2- 172.21.15.42:0/3613665854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19180b1a80 0x7f191809af90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1918073bd0 tx=0x7f190001c3f0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ea00640 1 -- 172.21.15.42:0/3613665854 shutdown_connections 2024-05-28T13:58:56.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ea00640 1 --2- 172.21.15.42:0/3613665854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19180b1a80 0x7f191809af90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ea00640 1 --2- 172.21.15.42:0/3613665854 >> v2:172.21.15.63:3300/0 conn(0x7f191815e990 0x7f191809aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ea00640 1 -- 172.21.15.42:0/3613665854 >> 172.21.15.42:0/3613665854 conn(0x7f19180b3fa0 msgr2=0x7f19180b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:56.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ea00640 1 -- 172.21.15.42:0/3613665854 shutdown_connections 2024-05-28T13:58:56.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ea00640 1 -- 172.21.15.42:0/3613665854 wait complete. 2024-05-28T13:58:56.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ea00640 1 Processor -- start 2024-05-28T13:58:56.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ea00640 1 -- start start 2024-05-28T13:58:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f191806d780 0x7f191814b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19180b1a80 0x7f191815a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f191815e990 0x7f19180b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f191816b950 con 0x7f191815e990 2024-05-28T13:58:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f191816b7d0 con 0x7f191806d780 2024-05-28T13:58:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ea00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f191816bad0 con 0x7f19180b1a80 2024-05-28T13:58:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f191815e990 0x7f19180b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f191815e990 0x7f19180b71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38952/0 (socket says 172.21.15.42:38952) 2024-05-28T13:58:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ca00640 1 -- 172.21.15.42:0/3923827904 learned_addr learned my addr 172.21.15.42:0/3923827904 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f190fe00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19180b1a80 0x7f191815a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ca00640 1 -- 172.21.15.42:0/3923827904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19180b1a80 msgr2=0x7f191815a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ca00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19180b1a80 0x7f191815a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ca00640 1 -- 172.21.15.42:0/3923827904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f191806d780 msgr2=0x7f191814b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f1917e00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f191806d780 0x7f191814b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ca00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f191806d780 0x7f191814b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ca00640 1 -- 172.21.15.42:0/3923827904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191815aa70 con 0x7f191815e990 2024-05-28T13:58:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f190fe00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19180b1a80 0x7f191815a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f1917e00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f191806d780 0x7f191814b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f191ca00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f191815e990 0x7f19180b71b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f191000cb20 tx=0x7f191000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:56.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f1914c00640 1 -- 172.21.15.42:0/3923827904 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191000aca0 con 0x7f191815e990 2024-05-28T13:58:56.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.729+0000 7f1914c00640 1 -- 172.21.15.42:0/3923827904 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1910012400 con 0x7f191815e990 2024-05-28T13:58:56.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.733+0000 7f1914c00640 1 -- 172.21.15.42:0/3923827904 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191000ae60 con 0x7f191815e990 2024-05-28T13:58:56.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.733+0000 7f191ea00640 1 -- 172.21.15.42:0/3923827904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19180c60b0 con 0x7f191815e990 2024-05-28T13:58:56.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.733+0000 7f191ea00640 1 -- 172.21.15.42:0/3923827904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19180b0960 con 0x7f191815e990 2024-05-28T13:58:56.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.733+0000 7f191ea00640 1 -- 172.21.15.42:0/3923827904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1918068a90 con 0x7f191815e990 2024-05-28T13:58:56.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.733+0000 7f1914c00640 1 -- 172.21.15.42:0/3923827904 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f19100127f0 con 0x7f191815e990 2024-05-28T13:58:56.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.733+0000 7f1914c00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f18f4068380 0x7f18f406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:56.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.733+0000 7f1914c00640 1 -- 172.21.15.42:0/3923827904 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1910087170 con 0x7f191815e990 2024-05-28T13:58:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.733+0000 7f1917e00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f18f4068380 0x7f18f406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.733+0000 7f1917e00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f18f4068380 0x7f18f406a840 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1908009a40 tx=0x7f1908008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:56.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.737+0000 7f1914c00640 1 -- 172.21.15.42:0/3923827904 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1910057600 con 0x7f191815e990 2024-05-28T13:58:56.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.961+0000 7f191ea00640 1 -- 172.21.15.42:0/3923827904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone"]} v 0) -- 0x7f19180b1860 con 0x7f191815e990 2024-05-28T13:58:56.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.969+0000 7f1914c00640 1 -- 172.21.15.42:0/3923827904 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone"]}]=0 v56) ==== 163+0+0 (secure 0 0 0) 0x7f1910056f00 con 0x7f191815e990 2024-05-28T13:58:56.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.973+0000 7f191ea00640 1 -- 172.21.15.42:0/3923827904 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f18f4068380 msgr2=0x7f18f406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.973+0000 7f191ea00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f18f4068380 0x7f18f406a840 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1908009a40 tx=0x7f1908008040 comp rx=0 tx=0).stop 2024-05-28T13:58:56.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.973+0000 7f191ea00640 1 -- 172.21.15.42:0/3923827904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f191815e990 msgr2=0x7f19180b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:56.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.973+0000 7f191ea00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f191815e990 0x7f19180b71b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f191000cb20 tx=0x7f191000a060 comp rx=0 tx=0).stop 2024-05-28T13:58:56.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.973+0000 7f191ea00640 1 -- 172.21.15.42:0/3923827904 shutdown_connections 2024-05-28T13:58:56.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.973+0000 7f191ea00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f18f4068380 0x7f18f406a840 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.973+0000 7f191ea00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f191815e990 0x7f19180b71b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.973+0000 7f191ea00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19180b1a80 0x7f191815a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.973+0000 7f191ea00640 1 --2- 172.21.15.42:0/3923827904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f191806d780 0x7f191814b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:56.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.973+0000 7f191ea00640 1 -- 172.21.15.42:0/3923827904 >> 172.21.15.42:0/3923827904 conn(0x7f19180b3fa0 msgr2=0x7f191806dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:56.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.973+0000 7f191ea00640 1 -- 172.21.15.42:0/3923827904 shutdown_connections 2024-05-28T13:58:56.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:56.973+0000 7f191ea00640 1 -- 172.21.15.42:0/3923827904 wait complete. 2024-05-28T13:58:56.978 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T13:58:57.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fefa8c00640 1 Processor -- start 2024-05-28T13:58:57.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fefa8c00640 1 -- start start 2024-05-28T13:58:57.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fefa8c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefa4071de0 0x7fefa40721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:57.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fefa8c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fefa40ce3b0 0x7fefa40ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:57.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fefa8c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefa4072700 con 0x7fefa4141d10 2024-05-28T13:58:57.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fefa8c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fefa4160230 con 0x7fefa40ce3b0 2024-05-28T13:58:57.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fefa8c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fefa4160510 con 0x7fefa4071de0 2024-05-28T13:58:57.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fefa2800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fefa4141d10 0x7fefa41421a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50640/0 (socket says 172.21.15.42:50640) 2024-05-28T13:58:57.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fefa2800640 1 -- 172.21.15.42:0/2701656704 learned_addr learned my addr 172.21.15.42:0/2701656704 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:57.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fefa1e00640 1 --2- 172.21.15.42:0/2701656704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefa4071de0 0x7fefa40721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:57.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fef99400640 1 --2- 172.21.15.42:0/2701656704 >> v2:172.21.15.63:3300/0 conn(0x7fefa40ce3b0 0x7fefa40ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:57.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fefa1400640 1 -- 172.21.15.42:0/2701656704 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3770753477 0 0) 0x7fefa4072700 con 0x7fefa4141d10 2024-05-28T13:58:57.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fefa1400640 1 -- 172.21.15.42:0/2701656704 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef88003530 con 0x7fefa4141d10 2024-05-28T13:58:57.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fef99400640 1 -- 172.21.15.42:0/2701656704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefa4071de0 msgr2=0x7fefa40721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:57.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fef99400640 1 --2- 172.21.15.42:0/2701656704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefa4071de0 0x7fefa40721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fef99400640 1 -- 172.21.15.42:0/2701656704 >> v1:172.21.15.42:6789/0 conn(0x7fefa4141d10 legacy=0x7fefa41421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:57.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fef99400640 1 -- 172.21.15.42:0/2701656704 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa41607f0 con 0x7fefa40ce3b0 2024-05-28T13:58:57.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.137+0000 7fefa1e00640 1 --2- 172.21.15.42:0/2701656704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefa4071de0 0x7fefa40721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:57.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.141+0000 7fef99400640 1 --2- 172.21.15.42:0/2701656704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefa40ce3b0 0x7fefa40ce810 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0097c0 tx=0x7fef8c01c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61057921d894e13 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:57.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.141+0000 7fefa1400640 1 -- 172.21.15.42:0/2701656704 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0b0070 con 0x7fefa40ce3b0 2024-05-28T13:58:57.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.141+0000 7fefa1400640 1 -- 172.21.15.42:0/2701656704 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8c0ab600 con 0x7fefa40ce3b0 2024-05-28T13:58:57.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.141+0000 7fefa1400640 1 -- 172.21.15.42:0/2701656704 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0ab9d0 con 0x7fefa40ce3b0 2024-05-28T13:58:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.141+0000 7fefa8c00640 1 -- 172.21.15.42:0/2701656704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefa40ce3b0 msgr2=0x7fefa40ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.141+0000 7fefa8c00640 1 --2- 172.21.15.42:0/2701656704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefa40ce3b0 0x7fefa40ce810 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0097c0 tx=0x7fef8c01c380 comp rx=0 tx=0).stop 2024-05-28T13:58:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.141+0000 7fefa8c00640 1 -- 172.21.15.42:0/2701656704 shutdown_connections 2024-05-28T13:58:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.141+0000 7fefa8c00640 1 --2- 172.21.15.42:0/2701656704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefa40ce3b0 0x7fefa40ce810 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.141+0000 7fefa8c00640 1 --2- 172.21.15.42:0/2701656704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefa4071de0 0x7fefa40721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.141+0000 7fefa8c00640 1 -- 172.21.15.42:0/2701656704 >> 172.21.15.42:0/2701656704 conn(0x7fefa40b21f0 msgr2=0x7fefa40b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.141+0000 7fefa8c00640 1 -- 172.21.15.42:0/2701656704 shutdown_connections 2024-05-28T13:58:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.141+0000 7fefa8c00640 1 -- 172.21.15.42:0/2701656704 wait complete. 2024-05-28T13:58:57.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fefa8c00640 1 Processor -- start 2024-05-28T13:58:57.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fefa8c00640 1 -- start start 2024-05-28T13:58:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fefa8c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefa4071de0 0x7fefa41547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fefa8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefa40ce3b0 0x7fefa406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fefa8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefa4141d10 0x7fefa406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fefa8c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fefa4169e30 con 0x7fefa40ce3b0 2024-05-28T13:58:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fefa8c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fefa4169cb0 con 0x7fefa4141d10 2024-05-28T13:58:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fefa8c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fefa4169fb0 con 0x7fefa4071de0 2024-05-28T13:58:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fef99400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefa40ce3b0 0x7fefa406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fef99400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefa40ce3b0 0x7fefa406ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38954/0 (socket says 172.21.15.42:38954) 2024-05-28T13:58:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fef99400640 1 -- 172.21.15.42:0/1520025453 learned_addr learned my addr 172.21.15.42:0/1520025453 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fefa1e00640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefa4071de0 0x7fefa41547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fefa2800640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefa4141d10 0x7fefa406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fef99400640 1 -- 172.21.15.42:0/1520025453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefa4071de0 msgr2=0x7fefa41547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fef99400640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefa4071de0 0x7fefa41547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fef99400640 1 -- 172.21.15.42:0/1520025453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefa4141d10 msgr2=0x7fefa406f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fef99400640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefa4141d10 0x7fefa406f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fef99400640 1 -- 172.21.15.42:0/1520025453 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa4157370 con 0x7fefa40ce3b0 2024-05-28T13:58:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fefa1e00640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefa4071de0 0x7fefa41547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fef99400640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefa40ce3b0 0x7fefa406ed20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fef8c019fb0 tx=0x7fef8c01abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fef9aa00640 1 -- 172.21.15.42:0/1520025453 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0b0050 con 0x7fefa40ce3b0 2024-05-28T13:58:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fef9aa00640 1 -- 172.21.15.42:0/1520025453 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8c0bc070 con 0x7fefa40ce3b0 2024-05-28T13:58:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fef9aa00640 1 -- 172.21.15.42:0/1520025453 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c006e20 con 0x7fefa40ce3b0 2024-05-28T13:58:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fefa8c00640 1 -- 172.21.15.42:0/1520025453 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa41578f0 con 0x7fefa40ce3b0 2024-05-28T13:58:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.145+0000 7fefa8c00640 1 -- 172.21.15.42:0/1520025453 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa4154d30 con 0x7fefa40ce3b0 2024-05-28T13:58:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.149+0000 7fefa8c00640 1 -- 172.21.15.42:0/1520025453 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa40738b0 con 0x7fefa40ce3b0 2024-05-28T13:58:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.149+0000 7fef9aa00640 1 -- 172.21.15.42:0/1520025453 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fef8c0b5830 con 0x7fefa40ce3b0 2024-05-28T13:58:57.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.149+0000 7fef9aa00640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef80068380 0x7fef8006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:57.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.149+0000 7fef9aa00640 1 -- 172.21.15.42:0/1520025453 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fef8c120fe0 con 0x7fefa40ce3b0 2024-05-28T13:58:57.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.149+0000 7fefa1e00640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef80068380 0x7fef8006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:57.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.149+0000 7fefa1e00640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef80068380 0x7fef8006a840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fefa415a1d0 tx=0x7fef94008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:57.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.153+0000 7fef9aa00640 1 -- 172.21.15.42:0/1520025453 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fef8c016090 con 0x7fefa40ce3b0 2024-05-28T13:58:57.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.317+0000 7fefa8c00640 1 -- 172.21.15.42:0/1520025453 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fefa404fdd0 con 0x7fefa40ce3b0 2024-05-28T13:58:57.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.321+0000 7fef9aa00640 1 -- 172.21.15.42:0/1520025453 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fef8c0f1710 con 0x7fefa40ce3b0 2024-05-28T13:58:57.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.325+0000 7fefa8c00640 1 -- 172.21.15.42:0/1520025453 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef80068380 msgr2=0x7fef8006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:57.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.325+0000 7fefa8c00640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef80068380 0x7fef8006a840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fefa415a1d0 tx=0x7fef94008040 comp rx=0 tx=0).stop 2024-05-28T13:58:57.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.325+0000 7fefa8c00640 1 -- 172.21.15.42:0/1520025453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefa40ce3b0 msgr2=0x7fefa406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:57.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.325+0000 7fefa8c00640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefa40ce3b0 0x7fefa406ed20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fef8c019fb0 tx=0x7fef8c01abc0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.325+0000 7fefa8c00640 1 -- 172.21.15.42:0/1520025453 shutdown_connections 2024-05-28T13:58:57.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.325+0000 7fefa8c00640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef80068380 0x7fef8006a840 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.325+0000 7fefa8c00640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefa4141d10 0x7fefa406f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.325+0000 7fefa8c00640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefa40ce3b0 0x7fefa406ed20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.325+0000 7fefa8c00640 1 --2- 172.21.15.42:0/1520025453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefa4071de0 0x7fefa41547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.325+0000 7fefa8c00640 1 -- 172.21.15.42:0/1520025453 >> 172.21.15.42:0/1520025453 conn(0x7fefa40b21f0 msgr2=0x7fefa4141130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:57.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.325+0000 7fefa8c00640 1 -- 172.21.15.42:0/1520025453 shutdown_connections 2024-05-28T13:58:57.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.325+0000 7fefa8c00640 1 -- 172.21.15.42:0/1520025453 wait complete. 2024-05-28T13:58:57.330 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T13:58:57.330 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T13:58:57.431 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T13:58:57.506 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T13:58:57.506 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:58:57.506 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:58:57.516 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:58:57.516 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:58:57.565 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:57.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80e9c00640 1 Processor -- start 2024-05-28T13:58:57.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80e9c00640 1 -- start start 2024-05-28T13:58:57.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80e9c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80e40b2800 0x7f80e40d0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:57.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80e9c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f80e40ce990 0x7f80e40d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:57.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80e9c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80e40cdbd0 con 0x7f80e415e9e0 2024-05-28T13:58:57.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80e9c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f80e415edc0 con 0x7f80e40ce990 2024-05-28T13:58:57.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80e9c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80e40b2db0 con 0x7f80e40b2800 2024-05-28T13:58:57.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80e3400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f80e40ce990 0x7f80e40d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:57.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80dbe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80e40b2800 0x7f80e40d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:57.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80e3400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f80e40ce990 0x7f80e40d0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:58032/0 (socket says 172.21.15.42:58032) 2024-05-28T13:58:57.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80e2a00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f80e415e9e0 0x7f80e40b20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:55432/0 (socket says 172.21.15.42:55432) 2024-05-28T13:58:57.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80e3400640 1 -- 172.21.15.42:0/359884629 learned_addr learned my addr 172.21.15.42:0/359884629 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:57.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80e2000640 1 -- 172.21.15.42:0/359884629 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2859979190 0 0) 0x7f80e40cdbd0 con 0x7f80e415e9e0 2024-05-28T13:58:57.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80e2000640 1 -- 172.21.15.42:0/359884629 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80c8003530 con 0x7f80e415e9e0 2024-05-28T13:58:57.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80dbe00640 1 -- 172.21.15.42:0/359884629 >> v2:172.21.15.63:3300/0 conn(0x7f80e40ce990 msgr2=0x7f80e40d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:57.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80dbe00640 1 --2- 172.21.15.42:0/359884629 >> v2:172.21.15.63:3300/0 conn(0x7f80e40ce990 0x7f80e40d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80dbe00640 1 -- 172.21.15.42:0/359884629 >> v1:172.21.15.63:6789/0 conn(0x7f80e415e9e0 legacy=0x7f80e40b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:57.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80dbe00640 1 -- 172.21.15.42:0/359884629 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e40cdeb0 con 0x7f80e40b2800 2024-05-28T13:58:57.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.765+0000 7f80e3400640 1 --2- 172.21.15.42:0/359884629 >> v2:172.21.15.63:3300/0 conn(0x7f80e40ce990 0x7f80e40d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:58:57.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.769+0000 7f80dbe00640 1 --2- 172.21.15.42:0/359884629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80e40b2800 0x7f80e40d0730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0097c0 tx=0x7f80cc01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=434bf4ef524c65b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:57.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.769+0000 7f80e2000640 1 -- 172.21.15.42:0/359884629 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc0b0070 con 0x7f80e40b2800 2024-05-28T13:58:57.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.769+0000 7f80e2000640 1 -- 172.21.15.42:0/359884629 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80cc0ab600 con 0x7f80e40b2800 2024-05-28T13:58:57.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.769+0000 7f80e2000640 1 -- 172.21.15.42:0/359884629 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc0ab9b0 con 0x7f80e40b2800 2024-05-28T13:58:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.769+0000 7f80e9c00640 1 -- 172.21.15.42:0/359884629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80e40b2800 msgr2=0x7f80e40d0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.769+0000 7f80e9c00640 1 --2- 172.21.15.42:0/359884629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80e40b2800 0x7f80e40d0730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0097c0 tx=0x7f80cc01c410 comp rx=0 tx=0).stop 2024-05-28T13:58:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.769+0000 7f80e9c00640 1 -- 172.21.15.42:0/359884629 shutdown_connections 2024-05-28T13:58:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.769+0000 7f80e9c00640 1 --2- 172.21.15.42:0/359884629 >> v2:172.21.15.63:3300/0 conn(0x7f80e40ce990 0x7f80e40d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.769+0000 7f80e9c00640 1 --2- 172.21.15.42:0/359884629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80e40b2800 0x7f80e40d0730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.769+0000 7f80e9c00640 1 -- 172.21.15.42:0/359884629 >> 172.21.15.42:0/359884629 conn(0x7f80e409aa50 msgr2=0x7f80e40b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.769+0000 7f80e9c00640 1 -- 172.21.15.42:0/359884629 shutdown_connections 2024-05-28T13:58:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.769+0000 7f80e9c00640 1 -- 172.21.15.42:0/359884629 wait complete. 2024-05-28T13:58:57.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80e9c00640 1 Processor -- start 2024-05-28T13:58:57.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80e9c00640 1 -- start start 2024-05-28T13:58:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80e9c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e40b2800 0x7f80e4152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80e9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80e40ce990 0x7f80e4152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80e9c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80e415e9e0 0x7f80e4153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80e9c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f80e40b15b0 con 0x7f80e40ce990 2024-05-28T13:58:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80e9c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80e40b1430 con 0x7f80e40b2800 2024-05-28T13:58:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80e9c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80e40b1730 con 0x7f80e415e9e0 2024-05-28T13:58:57.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80dbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80e40ce990 0x7f80e4152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:57.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80dbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80e40ce990 0x7f80e4152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52544/0 (socket says 172.21.15.42:52544) 2024-05-28T13:58:57.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80dbe00640 1 -- 172.21.15.42:0/484721013 learned_addr learned my addr 172.21.15.42:0/484721013 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:57.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80e2a00640 1 --2- 172.21.15.42:0/484721013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e40b2800 0x7f80e4152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:57.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80e3400640 1 --2- 172.21.15.42:0/484721013 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80e415e9e0 0x7f80e4153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:57.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80dbe00640 1 -- 172.21.15.42:0/484721013 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80e415e9e0 msgr2=0x7f80e4153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:57.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80dbe00640 1 --2- 172.21.15.42:0/484721013 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80e415e9e0 0x7f80e4153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80dbe00640 1 -- 172.21.15.42:0/484721013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e40b2800 msgr2=0x7f80e4152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:57.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80dbe00640 1 --2- 172.21.15.42:0/484721013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e40b2800 0x7f80e4152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80dbe00640 1 -- 172.21.15.42:0/484721013 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e4155df0 con 0x7f80e40ce990 2024-05-28T13:58:57.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80dbe00640 1 --2- 172.21.15.42:0/484721013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80e40ce990 0x7f80e4152fc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80cc000c10 tx=0x7f80cc0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:57.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80daa00640 1 -- 172.21.15.42:0/484721013 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc0ba070 con 0x7f80e40ce990 2024-05-28T13:58:57.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80daa00640 1 -- 172.21.15.42:0/484721013 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80cc0ab9b0 con 0x7f80e40ce990 2024-05-28T13:58:57.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80daa00640 1 -- 172.21.15.42:0/484721013 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc0b0040 con 0x7f80e40ce990 2024-05-28T13:58:57.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80e9c00640 1 -- 172.21.15.42:0/484721013 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e41560d0 con 0x7f80e40ce990 2024-05-28T13:58:57.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.773+0000 7f80e9c00640 1 -- 172.21.15.42:0/484721013 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e4156690 con 0x7f80e40ce990 2024-05-28T13:58:57.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.777+0000 7f80e9c00640 1 -- 172.21.15.42:0/484721013 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e4068a90 con 0x7f80e40ce990 2024-05-28T13:58:57.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.777+0000 7f80daa00640 1 -- 172.21.15.42:0/484721013 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f80cc0ab4c0 con 0x7f80e40ce990 2024-05-28T13:58:57.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.777+0000 7f80daa00640 1 --2- 172.21.15.42:0/484721013 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80c00683b0 0x7f80c006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:57.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.777+0000 7f80daa00640 1 -- 172.21.15.42:0/484721013 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f80cc11f9d0 con 0x7f80e40ce990 2024-05-28T13:58:57.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.777+0000 7f80e2a00640 1 --2- 172.21.15.42:0/484721013 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80c00683b0 0x7f80c006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:57.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.777+0000 7f80e2a00640 1 --2- 172.21.15.42:0/484721013 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80c00683b0 0x7f80c006a870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f80d4000f70 tx=0x7f80d4007d00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:57.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.781+0000 7f80daa00640 1 -- 172.21.15.42:0/484721013 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f80cc016090 con 0x7f80e40ce990 2024-05-28T13:58:57.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.949+0000 7f80e9c00640 1 -- 172.21.15.42:0/484721013 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80e40b7390 con 0x7f80e40ce990 2024-05-28T13:58:57.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.949+0000 7f80daa00640 1 -- 172.21.15.42:0/484721013 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16545 (secure 0 0 0) 0x7f80cc0ef8a0 con 0x7f80e40ce990 2024-05-28T13:58:57.935 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:57.935 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:24.659233+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:57.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.953+0000 7f80e9c00640 1 -- 172.21.15.42:0/484721013 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80c00683b0 msgr2=0x7f80c006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:57.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.953+0000 7f80e9c00640 1 --2- 172.21.15.42:0/484721013 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80c00683b0 0x7f80c006a870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f80d4000f70 tx=0x7f80d4007d00 comp rx=0 tx=0).stop 2024-05-28T13:58:57.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.953+0000 7f80e9c00640 1 -- 172.21.15.42:0/484721013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80e40ce990 msgr2=0x7f80e4152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:57.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.953+0000 7f80e9c00640 1 --2- 172.21.15.42:0/484721013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80e40ce990 0x7f80e4152fc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80cc000c10 tx=0x7f80cc0191b0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.953+0000 7f80e9c00640 1 -- 172.21.15.42:0/484721013 shutdown_connections 2024-05-28T13:58:57.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.953+0000 7f80e9c00640 1 --2- 172.21.15.42:0/484721013 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80c00683b0 0x7f80c006a870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.953+0000 7f80e9c00640 1 --2- 172.21.15.42:0/484721013 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80e415e9e0 0x7f80e4153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.953+0000 7f80e9c00640 1 --2- 172.21.15.42:0/484721013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80e40ce990 0x7f80e4152fc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.953+0000 7f80e9c00640 1 --2- 172.21.15.42:0/484721013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e40b2800 0x7f80e4152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:57.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.953+0000 7f80e9c00640 1 -- 172.21.15.42:0/484721013 >> 172.21.15.42:0/484721013 conn(0x7f80e409aa50 msgr2=0x7f80e40cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:57.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.953+0000 7f80e9c00640 1 -- 172.21.15.42:0/484721013 shutdown_connections 2024-05-28T13:58:57.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:57.953+0000 7f80e9c00640 1 -- 172.21.15.42:0/484721013 wait complete. 2024-05-28T13:58:57.956 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:58:57.957 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:58:58.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.117+0000 7f2013200640 1 Processor -- start 2024-05-28T13:58:58.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.117+0000 7f2013200640 1 -- start start 2024-05-28T13:58:58.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.117+0000 7f2013200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f20140cd050 0x7f20140d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.117+0000 7f2013200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2014160170 con 0x7f20140cf540 2024-05-28T13:58:58.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.117+0000 7f2013200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2014160450 con 0x7f2014145d50 2024-05-28T13:58:58.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.117+0000 7f2013200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f2014160730 con 0x7f20140cd050 2024-05-28T13:58:58.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.117+0000 7f2012800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f20140cd050 0x7f20140d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.117+0000 7f2011e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f20140cf540 0x7f20140cc940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34754/0 (socket says 172.21.15.42:34754) 2024-05-28T13:58:58.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.117+0000 7f2011e00640 1 -- 172.21.15.42:0/3841325347 learned_addr learned my addr 172.21.15.42:0/3841325347 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:58.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.117+0000 7f2012800640 1 -- 172.21.15.42:0/3841325347 >> v1:172.21.15.63:6789/0 conn(0x7f2014145d50 legacy=0x7f20140cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.117+0000 7f2012800640 1 -- 172.21.15.42:0/3841325347 >> v1:172.21.15.42:6789/0 conn(0x7f20140cf540 legacy=0x7f20140cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.117+0000 7f2012800640 1 -- 172.21.15.42:0/3841325347 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20140707e0 con 0x7f20140cd050 2024-05-28T13:58:58.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.121+0000 7f2012800640 1 --2- 172.21.15.42:0/3841325347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20140cd050 0x7f20140d05d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f200c009d90 tx=0x7f200c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99c97e148ff1744 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:58.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.121+0000 7f2011400640 1 -- 172.21.15.42:0/3841325347 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0a8620 con 0x7f20140cd050 2024-05-28T13:58:58.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.121+0000 7f2011400640 1 -- 172.21.15.42:0/3841325347 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f200c0a87c0 con 0x7f20140cd050 2024-05-28T13:58:58.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.121+0000 7f2011400640 1 -- 172.21.15.42:0/3841325347 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0a8b90 con 0x7f20140cd050 2024-05-28T13:58:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.121+0000 7f2013200640 1 -- 172.21.15.42:0/3841325347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20140cd050 msgr2=0x7f20140d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.121+0000 7f2013200640 1 --2- 172.21.15.42:0/3841325347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20140cd050 0x7f20140d05d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f200c009d90 tx=0x7f200c01c8e0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.121+0000 7f2013200640 1 -- 172.21.15.42:0/3841325347 shutdown_connections 2024-05-28T13:58:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.121+0000 7f2013200640 1 --2- 172.21.15.42:0/3841325347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20140cd050 0x7f20140d05d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.121+0000 7f2013200640 1 -- 172.21.15.42:0/3841325347 >> 172.21.15.42:0/3841325347 conn(0x7f201406fcc0 msgr2=0x7f20140700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.121+0000 7f2013200640 1 -- 172.21.15.42:0/3841325347 shutdown_connections 2024-05-28T13:58:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.121+0000 7f2013200640 1 -- 172.21.15.42:0/3841325347 wait complete. 2024-05-28T13:58:58.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.121+0000 7f2013200640 1 Processor -- start 2024-05-28T13:58:58.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2013200640 1 -- start start 2024-05-28T13:58:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2013200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20140cd050 0x7f2014153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2013200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20140cf540 0x7f20140caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2013200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2014145d50 0x7f20140cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2013200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f201416ab70 con 0x7f20140cd050 2024-05-28T13:58:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2013200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2014070a70 con 0x7f20140cf540 2024-05-28T13:58:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2013200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f201416acf0 con 0x7f2014145d50 2024-05-28T13:58:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2011e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20140cd050 0x7f2014153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2011e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20140cd050 0x7f2014153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52550/0 (socket says 172.21.15.42:52550) 2024-05-28T13:58:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2011e00640 1 -- 172.21.15.42:0/925651870 learned_addr learned my addr 172.21.15.42:0/925651870 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2009400640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20140cf540 0x7f20140caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2012800640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2014145d50 0x7f20140cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2011e00640 1 -- 172.21.15.42:0/925651870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2014145d50 msgr2=0x7f20140cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2011e00640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2014145d50 0x7f20140cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2011e00640 1 -- 172.21.15.42:0/925651870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20140cf540 msgr2=0x7f20140caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2011e00640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20140cf540 0x7f20140caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2011e00640 1 -- 172.21.15.42:0/925651870 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20140b3320 con 0x7f20140cd050 2024-05-28T13:58:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2009400640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20140cf540 0x7f20140caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:58:58.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2011e00640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20140cd050 0x7f2014153130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f200400c530 tx=0x7f200400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:58.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f200aa00640 1 -- 172.21.15.42:0/925651870 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004017070 con 0x7f20140cd050 2024-05-28T13:58:58.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f200aa00640 1 -- 172.21.15.42:0/925651870 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2004005cf0 con 0x7f20140cd050 2024-05-28T13:58:58.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f200aa00640 1 -- 172.21.15.42:0/925651870 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004007270 con 0x7f20140cd050 2024-05-28T13:58:58.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2013200640 1 -- 172.21.15.42:0/925651870 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20140b3600 con 0x7f20140cd050 2024-05-28T13:58:58.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2013200640 1 -- 172.21.15.42:0/925651870 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20141556c0 con 0x7f20140cd050 2024-05-28T13:58:58.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.125+0000 7f2013200640 1 -- 172.21.15.42:0/925651870 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201404fcf0 con 0x7f20140cd050 2024-05-28T13:58:58.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.129+0000 7f200aa00640 1 -- 172.21.15.42:0/925651870 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f200401a070 con 0x7f20140cd050 2024-05-28T13:58:58.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.129+0000 7f200aa00640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ff0068380 0x7f1ff006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.129+0000 7f200aa00640 1 -- 172.21.15.42:0/925651870 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2004085c40 con 0x7f20140cd050 2024-05-28T13:58:58.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.129+0000 7f2009400640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ff0068380 0x7f1ff006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.129+0000 7f2009400640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ff0068380 0x7f1ff006a840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f20140cc4a0 tx=0x7f1ffc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:58.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.133+0000 7f200aa00640 1 -- 172.21.15.42:0/925651870 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2004055ba0 con 0x7f20140cd050 2024-05-28T13:58:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.297+0000 7f2013200640 1 -- 172.21.15.42:0/925651870 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f201415a930 con 0x7f20140cd050 2024-05-28T13:58:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.297+0000 7f200aa00640 1 -- 172.21.15.42:0/925651870 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f2004059ea0 con 0x7f20140cd050 2024-05-28T13:58:58.283 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:58:58.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.301+0000 7f2013200640 1 -- 172.21.15.42:0/925651870 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ff0068380 msgr2=0x7f1ff006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.301+0000 7f2013200640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ff0068380 0x7f1ff006a840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f20140cc4a0 tx=0x7f1ffc009040 comp rx=0 tx=0).stop 2024-05-28T13:58:58.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.301+0000 7f2013200640 1 -- 172.21.15.42:0/925651870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20140cd050 msgr2=0x7f2014153130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.301+0000 7f2013200640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20140cd050 0x7f2014153130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f200400c530 tx=0x7f200400c9f0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.301+0000 7f2013200640 1 -- 172.21.15.42:0/925651870 shutdown_connections 2024-05-28T13:58:58.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.301+0000 7f2013200640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ff0068380 0x7f1ff006a840 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.301+0000 7f2013200640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2014145d50 0x7f20140cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.301+0000 7f2013200640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20140cf540 0x7f20140caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.301+0000 7f2013200640 1 --2- 172.21.15.42:0/925651870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20140cd050 0x7f2014153130 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.301+0000 7f2013200640 1 -- 172.21.15.42:0/925651870 >> 172.21.15.42:0/925651870 conn(0x7f201406fcc0 msgr2=0x7f2014159ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:58.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.301+0000 7f2013200640 1 -- 172.21.15.42:0/925651870 shutdown_connections 2024-05-28T13:58:58.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.301+0000 7f2013200640 1 -- 172.21.15.42:0/925651870 wait complete. 2024-05-28T13:58:58.304 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:58:58.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.461+0000 7fc9e6a00640 1 Processor -- start 2024-05-28T13:58:58.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.461+0000 7fc9e6a00640 1 -- start start 2024-05-28T13:58:58.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.465+0000 7fc9e6a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc9e015cb50 0x7fc9e009b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.465+0000 7fc9e6a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e009ba40 con 0x7fc9e00d11d0 2024-05-28T13:58:58.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.465+0000 7fc9e6a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e00c60d0 con 0x7fc9e015c770 2024-05-28T13:58:58.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.465+0000 7fc9e6a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc9e00c6720 con 0x7fc9e015cb50 2024-05-28T13:58:58.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.465+0000 7fc9d7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc9e015cb50 0x7fc9e009b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.465+0000 7fc9e4a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc9e00d11d0 0x7fc9e0166470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34770/0 (socket says 172.21.15.42:34770) 2024-05-28T13:58:58.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.465+0000 7fc9e4a00640 1 -- 172.21.15.42:0/1450388338 learned_addr learned my addr 172.21.15.42:0/1450388338 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:58.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.465+0000 7fc9d7e00640 1 -- 172.21.15.42:0/1450388338 >> v1:172.21.15.63:6789/0 conn(0x7fc9e015c770 legacy=0x7fc9e00d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.465+0000 7fc9d7e00640 1 -- 172.21.15.42:0/1450388338 >> v1:172.21.15.42:6789/0 conn(0x7fc9e00d11d0 legacy=0x7fc9e0166470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.465+0000 7fc9d7e00640 1 -- 172.21.15.42:0/1450388338 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e00c63b0 con 0x7fc9e015cb50 2024-05-28T13:58:58.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.469+0000 7fc9d7e00640 1 --2- 172.21.15.42:0/1450388338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e015cb50 0x7fc9e009b460 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80097c0 tx=0x7fc9c801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fa35d8f7e40329 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.469+0000 7fc9df400640 1 -- 172.21.15.42:0/1450388338 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80b5070 con 0x7fc9e015cb50 2024-05-28T13:58:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.469+0000 7fc9df400640 1 -- 172.21.15.42:0/1450388338 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c80a8990 con 0x7fc9e015cb50 2024-05-28T13:58:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.469+0000 7fc9df400640 1 -- 172.21.15.42:0/1450388338 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80a8d60 con 0x7fc9e015cb50 2024-05-28T13:58:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.469+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1450388338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e015cb50 msgr2=0x7fc9e009b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.469+0000 7fc9e6a00640 1 --2- 172.21.15.42:0/1450388338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e015cb50 0x7fc9e009b460 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80097c0 tx=0x7fc9c801ec90 comp rx=0 tx=0).stop 2024-05-28T13:58:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.469+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1450388338 shutdown_connections 2024-05-28T13:58:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.469+0000 7fc9e6a00640 1 --2- 172.21.15.42:0/1450388338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e015cb50 0x7fc9e009b460 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.469+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1450388338 >> 172.21.15.42:0/1450388338 conn(0x7fc9e00b3fd0 msgr2=0x7fc9e00b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.469+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1450388338 shutdown_connections 2024-05-28T13:58:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.469+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1450388338 wait complete. 2024-05-28T13:58:58.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.469+0000 7fc9e6a00640 1 Processor -- start 2024-05-28T13:58:58.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.469+0000 7fc9e6a00640 1 -- start start 2024-05-28T13:58:58.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9e6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e00d11d0 0x7fc9e00c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9e6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9e015c770 0x7fc9e00c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9e6a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9e015cb50 0x7fc9e00cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9e6a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e00c8bc0 con 0x7fc9e00d11d0 2024-05-28T13:58:58.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9e6a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e00ce2c0 con 0x7fc9e015c770 2024-05-28T13:58:58.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9e6a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e00c8d40 con 0x7fc9e015cb50 2024-05-28T13:58:58.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9dfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e00d11d0 0x7fc9e00c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9dfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e00d11d0 0x7fc9e00c4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52570/0 (socket says 172.21.15.42:52570) 2024-05-28T13:58:58.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9dfe00640 1 -- 172.21.15.42:0/1426749790 learned_addr learned my addr 172.21.15.42:0/1426749790 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:58.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9dfe00640 1 -- 172.21.15.42:0/1426749790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9e015cb50 msgr2=0x7fc9e00cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9d7e00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9e015c770 0x7fc9e00c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9e4a00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9e015cb50 0x7fc9e00cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9dfe00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9e015cb50 0x7fc9e00cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9dfe00640 1 -- 172.21.15.42:0/1426749790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9e015c770 msgr2=0x7fc9e00c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9dfe00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9e015c770 0x7fc9e00c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9dfe00640 1 -- 172.21.15.42:0/1426749790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e00b5280 con 0x7fc9e00d11d0 2024-05-28T13:58:58.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9d7e00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9e015c770 0x7fc9e00c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:58.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9e4a00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9e015cb50 0x7fc9e00cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:58.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9dfe00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e00d11d0 0x7fc9e00c4d20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000c500 tx=0x7fc9d000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:58.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9dcc00640 1 -- 172.21.15.42:0/1426749790 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0016020 con 0x7fc9e00d11d0 2024-05-28T13:58:58.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9dcc00640 1 -- 172.21.15.42:0/1426749790 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d0005220 con 0x7fc9e00d11d0 2024-05-28T13:58:58.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1426749790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e00b5560 con 0x7fc9e00d11d0 2024-05-28T13:58:58.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9dcc00640 1 -- 172.21.15.42:0/1426749790 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0007030 con 0x7fc9e00d11d0 2024-05-28T13:58:58.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1426749790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e00b5b20 con 0x7fc9e00d11d0 2024-05-28T13:58:58.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1426749790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e00a04d0 con 0x7fc9e00d11d0 2024-05-28T13:58:58.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.473+0000 7fc9dcc00640 1 -- 172.21.15.42:0/1426749790 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc9d00071d0 con 0x7fc9e00d11d0 2024-05-28T13:58:58.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.477+0000 7fc9dcc00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9bc068380 0x7fc9bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.477+0000 7fc9dcc00640 1 -- 172.21.15.42:0/1426749790 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc9d00863b0 con 0x7fc9e00d11d0 2024-05-28T13:58:58.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.477+0000 7fc9d7e00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9bc068380 0x7fc9bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.477+0000 7fc9d7e00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9bc068380 0x7fc9bc06a840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80b2000 tx=0x7fc9c80aa070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:58.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.481+0000 7fc9dcc00640 1 -- 172.21.15.42:0/1426749790 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc9d0020020 con 0x7fc9e00d11d0 2024-05-28T13:58:58.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.641+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1426749790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc9e00ced40 con 0x7fc9e00d11d0 2024-05-28T13:58:58.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.641+0000 7fc9dcc00640 1 -- 172.21.15.42:0/1426749790 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fc9d00074c0 con 0x7fc9e00d11d0 2024-05-28T13:58:58.628 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:58:58.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.645+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1426749790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9bc068380 msgr2=0x7fc9bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.645+0000 7fc9e6a00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9bc068380 0x7fc9bc06a840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80b2000 tx=0x7fc9c80aa070 comp rx=0 tx=0).stop 2024-05-28T13:58:58.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.645+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1426749790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e00d11d0 msgr2=0x7fc9e00c4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.645+0000 7fc9e6a00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e00d11d0 0x7fc9e00c4d20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000c500 tx=0x7fc9d000c9c0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.645+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1426749790 shutdown_connections 2024-05-28T13:58:58.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.645+0000 7fc9e6a00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9bc068380 0x7fc9bc06a840 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.645+0000 7fc9e6a00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9e015cb50 0x7fc9e00cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.645+0000 7fc9e6a00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9e015c770 0x7fc9e00c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.645+0000 7fc9e6a00640 1 --2- 172.21.15.42:0/1426749790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e00d11d0 0x7fc9e00c4d20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.645+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1426749790 >> 172.21.15.42:0/1426749790 conn(0x7fc9e00b3fd0 msgr2=0x7fc9e00cfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:58.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.645+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1426749790 shutdown_connections 2024-05-28T13:58:58.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.645+0000 7fc9e6a00640 1 -- 172.21.15.42:0/1426749790 wait complete. 2024-05-28T13:58:58.650 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:58:58.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.809+0000 7f8264c00640 1 Processor -- start 2024-05-28T13:58:58.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.809+0000 7f8264c00640 1 -- start start 2024-05-28T13:58:58.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.809+0000 7f8264c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f826009b160 0x7f826006cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.809+0000 7f8264c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f826009b8e0 0x7f826015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.809+0000 7f8264c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82600d0770 con 0x7f82600b0e80 2024-05-28T13:58:58.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.809+0000 7f8264c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f82600d0a10 con 0x7f826009b8e0 2024-05-28T13:58:58.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.809+0000 7f8264c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f82600d0cf0 con 0x7f826009b160 2024-05-28T13:58:58.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.809+0000 7f825e800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f826009b8e0 0x7f826015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.809+0000 7f825e800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f826009b8e0 0x7f826015e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52586/0 (socket says 172.21.15.42:52586) 2024-05-28T13:58:58.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.809+0000 7f825e800640 1 -- 172.21.15.42:0/3442688290 learned_addr learned my addr 172.21.15.42:0/3442688290 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:58.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.809+0000 7f825e800640 1 -- 172.21.15.42:0/3442688290 >> v2:172.21.15.63:3300/0 conn(0x7f826009b160 msgr2=0x7f826006cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:58.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.809+0000 7f8255400640 1 --2- 172.21.15.42:0/3442688290 >> v2:172.21.15.63:3300/0 conn(0x7f826009b160 0x7f826006cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.809+0000 7f825e800640 1 --2- 172.21.15.42:0/3442688290 >> v2:172.21.15.63:3300/0 conn(0x7f826009b160 0x7f826006cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.809+0000 7f825e800640 1 -- 172.21.15.42:0/3442688290 >> v1:172.21.15.63:6789/0 conn(0x7f82600b0e80 legacy=0x7f826009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:58.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.813+0000 7f825e800640 1 -- 172.21.15.42:0/3442688290 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82600d0fd0 con 0x7f826009b8e0 2024-05-28T13:58:58.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.813+0000 7f8255400640 1 --2- 172.21.15.42:0/3442688290 >> v2:172.21.15.63:3300/0 conn(0x7f826009b160 0x7f826006cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:58:58.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.813+0000 7f825e800640 1 --2- 172.21.15.42:0/3442688290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f826009b8e0 0x7f826015e110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8260073ec0 tx=0x7f825801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed6137ef5dc00fe server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:58.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.813+0000 7f825d400640 1 -- 172.21.15.42:0/3442688290 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82580a5670 con 0x7f826009b8e0 2024-05-28T13:58:58.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.813+0000 7f825d400640 1 -- 172.21.15.42:0/3442688290 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82580a5810 con 0x7f826009b8e0 2024-05-28T13:58:58.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.813+0000 7f825d400640 1 -- 172.21.15.42:0/3442688290 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82580a5be0 con 0x7f826009b8e0 2024-05-28T13:58:58.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.813+0000 7f8264c00640 1 -- 172.21.15.42:0/3442688290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f826009b8e0 msgr2=0x7f826015e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.813+0000 7f8264c00640 1 --2- 172.21.15.42:0/3442688290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f826009b8e0 0x7f826015e110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8260073ec0 tx=0x7f825801c9b0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.813+0000 7f8264c00640 1 -- 172.21.15.42:0/3442688290 shutdown_connections 2024-05-28T13:58:58.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.813+0000 7f8264c00640 1 --2- 172.21.15.42:0/3442688290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f826009b8e0 0x7f826015e110 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.813+0000 7f8264c00640 1 --2- 172.21.15.42:0/3442688290 >> v2:172.21.15.63:3300/0 conn(0x7f826009b160 0x7f826006cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.813+0000 7f8264c00640 1 -- 172.21.15.42:0/3442688290 >> 172.21.15.42:0/3442688290 conn(0x7f82600b3f90 msgr2=0x7f82600b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:58.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.813+0000 7f8264c00640 1 -- 172.21.15.42:0/3442688290 shutdown_connections 2024-05-28T13:58:58.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.813+0000 7f8264c00640 1 -- 172.21.15.42:0/3442688290 wait complete. 2024-05-28T13:58:58.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f8264c00640 1 Processor -- start 2024-05-28T13:58:58.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f8264c00640 1 -- start start 2024-05-28T13:58:58.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f8264c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f826009b160 0x7f82600c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f8264c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f826009b8e0 0x7f82600b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f8264c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82600b0e80 0x7f82600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f8264c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f82600af350 con 0x7f826009b160 2024-05-28T13:58:58.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f8264c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82600af1d0 con 0x7f82600b0e80 2024-05-28T13:58:58.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f8264c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82600af4d0 con 0x7f826009b8e0 2024-05-28T13:58:58.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f825de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f826009b160 0x7f82600c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f825de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f826009b160 0x7f82600c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52588/0 (socket says 172.21.15.42:52588) 2024-05-28T13:58:58.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f825de00640 1 -- 172.21.15.42:0/533516883 learned_addr learned my addr 172.21.15.42:0/533516883 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:58.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f825e800640 1 --2- 172.21.15.42:0/533516883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82600b0e80 0x7f82600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f8255400640 1 --2- 172.21.15.42:0/533516883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f826009b8e0 0x7f82600b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f825de00640 1 -- 172.21.15.42:0/533516883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f826009b8e0 msgr2=0x7f82600b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f825de00640 1 --2- 172.21.15.42:0/533516883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f826009b8e0 0x7f82600b7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f825de00640 1 -- 172.21.15.42:0/533516883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82600b0e80 msgr2=0x7f82600b77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f825de00640 1 --2- 172.21.15.42:0/533516883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82600b0e80 0x7f82600b77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f825de00640 1 -- 172.21.15.42:0/533516883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82600b27d0 con 0x7f826009b160 2024-05-28T13:58:58.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f825de00640 1 --2- 172.21.15.42:0/533516883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f826009b160 0x7f82600c5fd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f825000b410 tx=0x7f825000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:58.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f8256a00640 1 -- 172.21.15.42:0/533516883 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250015030 con 0x7f826009b160 2024-05-28T13:58:58.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f8256a00640 1 -- 172.21.15.42:0/533516883 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8250004020 con 0x7f826009b160 2024-05-28T13:58:58.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f8256a00640 1 -- 172.21.15.42:0/533516883 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250004450 con 0x7f826009b160 2024-05-28T13:58:58.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f8264c00640 1 -- 172.21.15.42:0/533516883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82600c65d0 con 0x7f826009b160 2024-05-28T13:58:58.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.817+0000 7f8264c00640 1 -- 172.21.15.42:0/533516883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82600c6b90 con 0x7f826009b160 2024-05-28T13:58:58.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.821+0000 7f8264c00640 1 -- 172.21.15.42:0/533516883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82600a0550 con 0x7f826009b160 2024-05-28T13:58:58.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.821+0000 7f8256a00640 1 -- 172.21.15.42:0/533516883 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8250018070 con 0x7f826009b160 2024-05-28T13:58:58.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.821+0000 7f8256a00640 1 --2- 172.21.15.42:0/533516883 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f823c068380 0x7f823c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:58.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.821+0000 7f8256a00640 1 -- 172.21.15.42:0/533516883 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8250086410 con 0x7f826009b160 2024-05-28T13:58:58.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.821+0000 7f8255400640 1 --2- 172.21.15.42:0/533516883 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f823c068380 0x7f823c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:58.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.821+0000 7f8255400640 1 --2- 172.21.15.42:0/533516883 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f823c068380 0x7f823c06a840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f82600b2030 tx=0x7f8248008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:58.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.825+0000 7f8256a00640 1 -- 172.21.15.42:0/533516883 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f82500562e0 con 0x7f826009b160 2024-05-28T13:58:58.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.993+0000 7f8264c00640 1 -- 172.21.15.42:0/533516883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f82600b16a0 con 0x7f826009b160 2024-05-28T13:58:58.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.993+0000 7f8256a00640 1 -- 172.21.15.42:0/533516883 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f825005a5e0 con 0x7f826009b160 2024-05-28T13:58:58.977 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:58:58.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.993+0000 7f8264c00640 1 -- 172.21.15.42:0/533516883 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f823c068380 msgr2=0x7f823c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.993+0000 7f8264c00640 1 --2- 172.21.15.42:0/533516883 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f823c068380 0x7f823c06a840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f82600b2030 tx=0x7f8248008040 comp rx=0 tx=0).stop 2024-05-28T13:58:58.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.993+0000 7f8264c00640 1 -- 172.21.15.42:0/533516883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f826009b160 msgr2=0x7f82600c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:58.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.993+0000 7f8264c00640 1 --2- 172.21.15.42:0/533516883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f826009b160 0x7f82600c5fd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f825000b410 tx=0x7f825000b8d0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.993+0000 7f8264c00640 1 -- 172.21.15.42:0/533516883 shutdown_connections 2024-05-28T13:58:58.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.993+0000 7f8264c00640 1 --2- 172.21.15.42:0/533516883 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f823c068380 0x7f823c06a840 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.993+0000 7f8264c00640 1 --2- 172.21.15.42:0/533516883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82600b0e80 0x7f82600b77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.993+0000 7f8264c00640 1 --2- 172.21.15.42:0/533516883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f826009b8e0 0x7f82600b7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.993+0000 7f8264c00640 1 --2- 172.21.15.42:0/533516883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f826009b160 0x7f82600c5fd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:58.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.993+0000 7f8264c00640 1 -- 172.21.15.42:0/533516883 >> 172.21.15.42:0/533516883 conn(0x7f82600b3f90 msgr2=0x7f82600b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:58.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.993+0000 7f8264c00640 1 -- 172.21.15.42:0/533516883 shutdown_connections 2024-05-28T13:58:58.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:58.993+0000 7f8264c00640 1 -- 172.21.15.42:0/533516883 wait complete. 2024-05-28T13:58:58.998 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T13:58:59.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e88a00640 1 Processor -- start 2024-05-28T13:58:59.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e88a00640 1 -- start start 2024-05-28T13:58:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e88a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e840d2570 0x7f1e8406da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e88a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e8406c980 con 0x7f1e8415e7c0 2024-05-28T13:58:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e88a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e8406cc60 con 0x7f1e840b5f60 2024-05-28T13:58:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e88a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1e840d29c0 con 0x7f1e840d2570 2024-05-28T13:58:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e79e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1e8415e7c0 0x7f1e840ceb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34782/0 (socket says 172.21.15.42:34782) 2024-05-28T13:58:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e79e00640 1 -- 172.21.15.42:0/1474960154 learned_addr learned my addr 172.21.15.42:0/1474960154 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e83200640 1 --2- 172.21.15.42:0/1474960154 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e840d2570 0x7f1e8406da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e81e00640 1 -- 172.21.15.42:0/1474960154 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3647441054 0 0) 0x7f1e8406c980 con 0x7f1e8415e7c0 2024-05-28T13:58:59.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e81e00640 1 -- 172.21.15.42:0/1474960154 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e68003530 con 0x7f1e8415e7c0 2024-05-28T13:58:59.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e81e00640 1 -- 172.21.15.42:0/1474960154 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2652232177 0 0) 0x7f1e8406cc60 con 0x7f1e840b5f60 2024-05-28T13:58:59.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e81e00640 1 -- 172.21.15.42:0/1474960154 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e8406c980 con 0x7f1e840b5f60 2024-05-28T13:58:59.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e83200640 1 -- 172.21.15.42:0/1474960154 >> v1:172.21.15.63:6789/0 conn(0x7f1e840b5f60 legacy=0x7f1e840b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e83200640 1 -- 172.21.15.42:0/1474960154 >> v1:172.21.15.42:6789/0 conn(0x7f1e8415e7c0 legacy=0x7f1e840ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.157+0000 7f1e83200640 1 -- 172.21.15.42:0/1474960154 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8406cf40 con 0x7f1e840d2570 2024-05-28T13:58:59.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.161+0000 7f1e83200640 1 --2- 172.21.15.42:0/1474960154 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e840d2570 0x7f1e8406da20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0097f0 tx=0x7f1e7c01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27bf11f1436a2ebb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:59.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.161+0000 7f1e81e00640 1 -- 172.21.15.42:0/1474960154 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0b1070 con 0x7f1e840d2570 2024-05-28T13:58:59.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.161+0000 7f1e81e00640 1 -- 172.21.15.42:0/1474960154 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e7c0ac620 con 0x7f1e840d2570 2024-05-28T13:58:59.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.161+0000 7f1e81e00640 1 -- 172.21.15.42:0/1474960154 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0ac9d0 con 0x7f1e840d2570 2024-05-28T13:58:59.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.161+0000 7f1e88a00640 1 -- 172.21.15.42:0/1474960154 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e840d2570 msgr2=0x7f1e8406da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.161+0000 7f1e88a00640 1 --2- 172.21.15.42:0/1474960154 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e840d2570 0x7f1e8406da20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0097f0 tx=0x7f1e7c01c4b0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.161+0000 7f1e88a00640 1 -- 172.21.15.42:0/1474960154 shutdown_connections 2024-05-28T13:58:59.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.161+0000 7f1e88a00640 1 --2- 172.21.15.42:0/1474960154 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e840d2570 0x7f1e8406da20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.161+0000 7f1e88a00640 1 -- 172.21.15.42:0/1474960154 >> 172.21.15.42:0/1474960154 conn(0x7f1e8408bfe0 msgr2=0x7f1e840d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:59.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.161+0000 7f1e88a00640 1 -- 172.21.15.42:0/1474960154 shutdown_connections 2024-05-28T13:58:59.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.161+0000 7f1e88a00640 1 -- 172.21.15.42:0/1474960154 wait complete. 2024-05-28T13:58:59.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e88a00640 1 Processor -- start 2024-05-28T13:58:59.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e88a00640 1 -- start start 2024-05-28T13:58:59.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e88a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e840b5f60 0x7f1e840cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e88a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e840d2570 0x7f1e840b9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e88a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e8415e7c0 0x7f1e840b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e88a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e840cf7d0 con 0x7f1e8415e7c0 2024-05-28T13:58:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e88a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1e840cf650 con 0x7f1e840d2570 2024-05-28T13:58:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e88a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8416cb00 con 0x7f1e840b5f60 2024-05-28T13:58:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e83200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e8415e7c0 0x7f1e840b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e83200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e8415e7c0 0x7f1e840b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52598/0 (socket says 172.21.15.42:52598) 2024-05-28T13:58:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e83200640 1 -- 172.21.15.42:0/3689301167 learned_addr learned my addr 172.21.15.42:0/3689301167 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e83200640 1 -- 172.21.15.42:0/3689301167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e840b5f60 msgr2=0x7f1e840cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e83200640 1 --2- 172.21.15.42:0/3689301167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e840b5f60 0x7f1e840cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e83200640 1 -- 172.21.15.42:0/3689301167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e840d2570 msgr2=0x7f1e840b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e83200640 1 --2- 172.21.15.42:0/3689301167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e840d2570 0x7f1e840b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e83200640 1 -- 172.21.15.42:0/3689301167 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e840ce470 con 0x7f1e8415e7c0 2024-05-28T13:58:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e83200640 1 --2- 172.21.15.42:0/3689301167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e8415e7c0 0x7f1e840b1720 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0af040 tx=0x7f1e7c019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e7b400640 1 -- 172.21.15.42:0/3689301167 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0b9070 con 0x7f1e8415e7c0 2024-05-28T13:58:59.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e7b400640 1 -- 172.21.15.42:0/3689301167 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e7c0b5410 con 0x7f1e8415e7c0 2024-05-28T13:58:59.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e7b400640 1 -- 172.21.15.42:0/3689301167 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0b1040 con 0x7f1e8415e7c0 2024-05-28T13:58:59.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e88a00640 1 -- 172.21.15.42:0/3689301167 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e840b9ef0 con 0x7f1e8415e7c0 2024-05-28T13:58:59.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e88a00640 1 -- 172.21.15.42:0/3689301167 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e84158980 con 0x7f1e8415e7c0 2024-05-28T13:58:59.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.165+0000 7f1e88a00640 1 -- 172.21.15.42:0/3689301167 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e8404fcf0 con 0x7f1e8415e7c0 2024-05-28T13:58:59.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.169+0000 7f1e7b400640 1 -- 172.21.15.42:0/3689301167 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1e7c0bc070 con 0x7f1e8415e7c0 2024-05-28T13:58:59.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.169+0000 7f1e7b400640 1 --2- 172.21.15.42:0/3689301167 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1e600683b0 0x7f1e6006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.169+0000 7f1e7b400640 1 -- 172.21.15.42:0/3689301167 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1e7c1203a0 con 0x7f1e8415e7c0 2024-05-28T13:58:59.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.169+0000 7f1e82800640 1 --2- 172.21.15.42:0/3689301167 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1e600683b0 0x7f1e6006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.169+0000 7f1e82800640 1 --2- 172.21.15.42:0/3689301167 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1e600683b0 0x7f1e6006a870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1e74001570 tx=0x7f1e74007c80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:59.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.173+0000 7f1e7b400640 1 -- 172.21.15.42:0/3689301167 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e7c0c3070 con 0x7f1e8415e7c0 2024-05-28T13:58:59.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.349+0000 7f1e88a00640 1 -- 172.21.15.42:0/3689301167 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1e8416a0c0 con 0x7f1e8415e7c0 2024-05-28T13:58:59.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.349+0000 7f1e7b400640 1 -- 172.21.15.42:0/3689301167 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f1e840ce470 con 0x7f1e8415e7c0 2024-05-28T13:58:59.335 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T13:58:59.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.353+0000 7f1e88a00640 1 -- 172.21.15.42:0/3689301167 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1e600683b0 msgr2=0x7f1e6006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.353+0000 7f1e88a00640 1 --2- 172.21.15.42:0/3689301167 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1e600683b0 0x7f1e6006a870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1e74001570 tx=0x7f1e74007c80 comp rx=0 tx=0).stop 2024-05-28T13:58:59.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.353+0000 7f1e88a00640 1 -- 172.21.15.42:0/3689301167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e8415e7c0 msgr2=0x7f1e840b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.353+0000 7f1e88a00640 1 --2- 172.21.15.42:0/3689301167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e8415e7c0 0x7f1e840b1720 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0af040 tx=0x7f1e7c019250 comp rx=0 tx=0).stop 2024-05-28T13:58:59.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.353+0000 7f1e88a00640 1 -- 172.21.15.42:0/3689301167 shutdown_connections 2024-05-28T13:58:59.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.353+0000 7f1e88a00640 1 --2- 172.21.15.42:0/3689301167 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1e600683b0 0x7f1e6006a870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.353+0000 7f1e88a00640 1 --2- 172.21.15.42:0/3689301167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e8415e7c0 0x7f1e840b1720 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.353+0000 7f1e88a00640 1 --2- 172.21.15.42:0/3689301167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e840d2570 0x7f1e840b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.353+0000 7f1e88a00640 1 --2- 172.21.15.42:0/3689301167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e840b5f60 0x7f1e840cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.353+0000 7f1e88a00640 1 -- 172.21.15.42:0/3689301167 >> 172.21.15.42:0/3689301167 conn(0x7f1e8408bfe0 msgr2=0x7f1e840b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:59.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.353+0000 7f1e88a00640 1 -- 172.21.15.42:0/3689301167 shutdown_connections 2024-05-28T13:58:59.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.353+0000 7f1e88a00640 1 -- 172.21.15.42:0/3689301167 wait complete. 2024-05-28T13:58:59.356 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:58:59.356 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:58:59.366 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:58:59.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.565+0000 7fa1af600640 1 Processor -- start 2024-05-28T13:58:59.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.565+0000 7fa1af600640 1 -- start start 2024-05-28T13:58:59.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.569+0000 7fa1af600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa1a815fdb0 0x7fa1a809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.569+0000 7fa1af600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa1a80b64b0 0x7fa1a809af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.569+0000 7fa1af600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a809bd00 con 0x7fa1a809b790 2024-05-28T13:58:59.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.569+0000 7fa1af600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a80d0b20 con 0x7fa1a815fdb0 2024-05-28T13:58:59.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.569+0000 7fa1af600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a80d0e00 con 0x7fa1a80b64b0 2024-05-28T13:58:59.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.569+0000 7fa1acc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa1a815fdb0 0x7fa1a809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.569+0000 7fa1a7e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa1a809b790 0x7fa1a80cd0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34790/0 (socket says 172.21.15.42:34790) 2024-05-28T13:58:59.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.569+0000 7fa1a7e00640 1 -- 172.21.15.42:0/4257392886 learned_addr learned my addr 172.21.15.42:0/4257392886 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:58:59.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.569+0000 7fa1ad600640 1 --2- 172.21.15.42:0/4257392886 >> v2:172.21.15.63:3300/0 conn(0x7fa1a80b64b0 0x7fa1a809af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.569+0000 7fa1a7400640 1 -- 172.21.15.42:0/4257392886 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 613345785 0 0) 0x7fa1a809bd00 con 0x7fa1a809b790 2024-05-28T13:58:59.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.569+0000 7fa1acc00640 1 -- 172.21.15.42:0/4257392886 >> v2:172.21.15.63:3300/0 conn(0x7fa1a80b64b0 msgr2=0x7fa1a809af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:59.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.569+0000 7fa1acc00640 1 --2- 172.21.15.42:0/4257392886 >> v2:172.21.15.63:3300/0 conn(0x7fa1a80b64b0 0x7fa1a809af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.569+0000 7fa1acc00640 1 -- 172.21.15.42:0/4257392886 >> v1:172.21.15.42:6789/0 conn(0x7fa1a809b790 legacy=0x7fa1a80cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.569+0000 7fa1acc00640 1 -- 172.21.15.42:0/4257392886 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a80d10e0 con 0x7fa1a815fdb0 2024-05-28T13:58:59.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1acc00640 1 --2- 172.21.15.42:0/4257392886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1a815fdb0 0x7fa1a809aa50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8073510 tx=0x7fa19c01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74c47e9e4dd4708 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:59.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1a7400640 1 -- 172.21.15.42:0/4257392886 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0b0070 con 0x7fa1a815fdb0 2024-05-28T13:58:59.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1a7400640 1 -- 172.21.15.42:0/4257392886 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa19c0ab600 con 0x7fa1a815fdb0 2024-05-28T13:58:59.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1a7400640 1 -- 172.21.15.42:0/4257392886 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0ab9d0 con 0x7fa1a815fdb0 2024-05-28T13:58:59.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 -- 172.21.15.42:0/4257392886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1a815fdb0 msgr2=0x7fa1a809aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 --2- 172.21.15.42:0/4257392886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1a815fdb0 0x7fa1a809aa50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8073510 tx=0x7fa19c01c360 comp rx=0 tx=0).stop 2024-05-28T13:58:59.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 -- 172.21.15.42:0/4257392886 shutdown_connections 2024-05-28T13:58:59.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 --2- 172.21.15.42:0/4257392886 >> v2:172.21.15.63:3300/0 conn(0x7fa1a80b64b0 0x7fa1a809af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 --2- 172.21.15.42:0/4257392886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1a815fdb0 0x7fa1a809aa50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 -- 172.21.15.42:0/4257392886 >> 172.21.15.42:0/4257392886 conn(0x7fa1a80b3f90 msgr2=0x7fa1a80b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:59.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 -- 172.21.15.42:0/4257392886 shutdown_connections 2024-05-28T13:58:59.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 -- 172.21.15.42:0/4257392886 wait complete. 2024-05-28T13:58:59.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 Processor -- start 2024-05-28T13:58:59.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 -- start start 2024-05-28T13:58:59.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1a809b790 0x7fa1a814b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1a80b64b0 0x7fa1a8165d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1a815fdb0 0x7fa1a806db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8169900 con 0x7fa1a809b790 2024-05-28T13:58:59.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8169780 con 0x7fa1a80b64b0 2024-05-28T13:58:59.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.573+0000 7fa1af600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a8169a80 con 0x7fa1a815fdb0 2024-05-28T13:58:59.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1acc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1a809b790 0x7fa1a814b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1acc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1a809b790 0x7fa1a814b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52620/0 (socket says 172.21.15.42:52620) 2024-05-28T13:58:59.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1acc00640 1 -- 172.21.15.42:0/2153480577 learned_addr learned my addr 172.21.15.42:0/2153480577 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:59.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1ad600640 1 --2- 172.21.15.42:0/2153480577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1a815fdb0 0x7fa1a806db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1a7e00640 1 --2- 172.21.15.42:0/2153480577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1a80b64b0 0x7fa1a8165d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1acc00640 1 -- 172.21.15.42:0/2153480577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1a815fdb0 msgr2=0x7fa1a806db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1acc00640 1 --2- 172.21.15.42:0/2153480577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1a815fdb0 0x7fa1a806db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1acc00640 1 -- 172.21.15.42:0/2153480577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1a80b64b0 msgr2=0x7fa1a8165d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1acc00640 1 --2- 172.21.15.42:0/2153480577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1a80b64b0 0x7fa1a8165d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1acc00640 1 -- 172.21.15.42:0/2153480577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a81662c0 con 0x7fa1a809b790 2024-05-28T13:58:59.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1acc00640 1 --2- 172.21.15.42:0/2153480577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1a809b790 0x7fa1a814b850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa19c019f90 tx=0x7fa19c01aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:59.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1a4c00640 1 -- 172.21.15.42:0/2153480577 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0b0050 con 0x7fa1a809b790 2024-05-28T13:58:59.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1a4c00640 1 -- 172.21.15.42:0/2153480577 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa19c0b1040 con 0x7fa1a809b790 2024-05-28T13:58:59.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1af600640 1 -- 172.21.15.42:0/2153480577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a80cb6d0 con 0x7fa1a809b790 2024-05-28T13:58:59.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1a4c00640 1 -- 172.21.15.42:0/2153480577 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c004220 con 0x7fa1a809b790 2024-05-28T13:58:59.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1af600640 1 -- 172.21.15.42:0/2153480577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a80cbc90 con 0x7fa1a809b790 2024-05-28T13:58:59.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1af600640 1 -- 172.21.15.42:0/2153480577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a8068a90 con 0x7fa1a809b790 2024-05-28T13:58:59.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.577+0000 7fa1a4c00640 1 -- 172.21.15.42:0/2153480577 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa19c0be060 con 0x7fa1a809b790 2024-05-28T13:58:59.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.581+0000 7fa1a4c00640 1 --2- 172.21.15.42:0/2153480577 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa188068380 0x7fa18806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.581+0000 7fa1a4c00640 1 -- 172.21.15.42:0/2153480577 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa19c120000 con 0x7fa1a809b790 2024-05-28T13:58:59.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.581+0000 7fa1a7e00640 1 --2- 172.21.15.42:0/2153480577 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa188068380 0x7fa18806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.581+0000 7fa1a7e00640 1 --2- 172.21.15.42:0/2153480577 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa188068380 0x7fa18806a840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa1a815bb10 tx=0x7fa194007df0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:59.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.585+0000 7fa1a4c00640 1 -- 172.21.15.42:0/2153480577 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa19c016070 con 0x7fa1a809b790 2024-05-28T13:58:59.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.749+0000 7fa1af600640 1 -- 172.21.15.42:0/2153480577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1a80b3190 con 0x7fa1a809b790 2024-05-28T13:58:59.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.749+0000 7fa1a4c00640 1 -- 172.21.15.42:0/2153480577 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16545 (secure 0 0 0) 0x7fa19c0f0000 con 0x7fa1a809b790 2024-05-28T13:58:59.735 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:58:59.735 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:24.659233+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:58:59.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.753+0000 7fa1af600640 1 -- 172.21.15.42:0/2153480577 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa188068380 msgr2=0x7fa18806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.753+0000 7fa1af600640 1 --2- 172.21.15.42:0/2153480577 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa188068380 0x7fa18806a840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa1a815bb10 tx=0x7fa194007df0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.753+0000 7fa1af600640 1 -- 172.21.15.42:0/2153480577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1a809b790 msgr2=0x7fa1a814b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.753+0000 7fa1af600640 1 --2- 172.21.15.42:0/2153480577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1a809b790 0x7fa1a814b850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa19c019f90 tx=0x7fa19c01aba0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.753+0000 7fa1af600640 1 -- 172.21.15.42:0/2153480577 shutdown_connections 2024-05-28T13:58:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.753+0000 7fa1af600640 1 --2- 172.21.15.42:0/2153480577 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa188068380 0x7fa18806a840 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.753+0000 7fa1af600640 1 --2- 172.21.15.42:0/2153480577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1a815fdb0 0x7fa1a806db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.753+0000 7fa1af600640 1 --2- 172.21.15.42:0/2153480577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1a80b64b0 0x7fa1a8165d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.753+0000 7fa1af600640 1 --2- 172.21.15.42:0/2153480577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1a809b790 0x7fa1a814b850 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.753+0000 7fa1af600640 1 -- 172.21.15.42:0/2153480577 >> 172.21.15.42:0/2153480577 conn(0x7fa1a80b3f90 msgr2=0x7fa1a8070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.753+0000 7fa1af600640 1 -- 172.21.15.42:0/2153480577 shutdown_connections 2024-05-28T13:58:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.753+0000 7fa1af600640 1 -- 172.21.15.42:0/2153480577 wait complete. 2024-05-28T13:58:59.757 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:58:59.758 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:58:59.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cbe600640 1 Processor -- start 2024-05-28T13:58:59.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cbe600640 1 -- start start 2024-05-28T13:58:59.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cbe600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7cb80b0e80 0x7f7cb806cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cbe600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7cb809b160 0x7f7cb815e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cbe600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cb80d0770 con 0x7f7cb809b8e0 2024-05-28T13:58:59.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cbe600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f7cb80d0a10 con 0x7f7cb80b0e80 2024-05-28T13:58:59.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cbe600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb80d0cf0 con 0x7f7cb809b160 2024-05-28T13:58:59.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cb7400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7cb80b0e80 0x7f7cb806cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cb7400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7cb80b0e80 0x7f7cb806cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52626/0 (socket says 172.21.15.42:52626) 2024-05-28T13:58:59.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cb7400640 1 -- 172.21.15.42:0/673353371 learned_addr learned my addr 172.21.15.42:0/673353371 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:59.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7caea00640 1 --1- 172.21.15.42:0/673353371 >> v1:172.21.15.63:6789/0 conn(0x7f7cb809b8e0 0x7f7cb809aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:55490/0 (socket says 172.21.15.42:55490) 2024-05-28T13:58:59.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cb7e00640 1 --2- 172.21.15.42:0/673353371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7cb809b160 0x7f7cb815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cb7400640 1 -- 172.21.15.42:0/673353371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7cb809b160 msgr2=0x7f7cb815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:58:59.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cb7400640 1 --2- 172.21.15.42:0/673353371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7cb809b160 0x7f7cb815e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cb7400640 1 -- 172.21.15.42:0/673353371 >> v1:172.21.15.63:6789/0 conn(0x7f7cb809b8e0 legacy=0x7f7cb809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.917+0000 7f7cb7400640 1 -- 172.21.15.42:0/673353371 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb80d0fd0 con 0x7f7cb80b0e80 2024-05-28T13:58:59.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.921+0000 7f7cb7400640 1 --2- 172.21.15.42:0/673353371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cb80b0e80 0x7f7cb806cb00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009870 tx=0x7f7ca801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced27bfa7e4c6a8c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:59.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.921+0000 7f7cb6a00640 1 -- 172.21.15.42:0/673353371 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80b2070 con 0x7f7cb80b0e80 2024-05-28T13:58:59.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.921+0000 7f7cb6a00640 1 -- 172.21.15.42:0/673353371 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca80a9070 con 0x7f7cb80b0e80 2024-05-28T13:58:59.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.921+0000 7f7cb6a00640 1 -- 172.21.15.42:0/673353371 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80ad880 con 0x7f7cb80b0e80 2024-05-28T13:58:59.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.921+0000 7f7cbe600640 1 -- 172.21.15.42:0/673353371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cb80b0e80 msgr2=0x7f7cb806cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.921+0000 7f7cbe600640 1 --2- 172.21.15.42:0/673353371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cb80b0e80 0x7f7cb806cb00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009870 tx=0x7f7ca801c560 comp rx=0 tx=0).stop 2024-05-28T13:58:59.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.921+0000 7f7cbe600640 1 -- 172.21.15.42:0/673353371 shutdown_connections 2024-05-28T13:58:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.921+0000 7f7cbe600640 1 --2- 172.21.15.42:0/673353371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7cb809b160 0x7f7cb815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.921+0000 7f7cbe600640 1 --2- 172.21.15.42:0/673353371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cb80b0e80 0x7f7cb806cb00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.921+0000 7f7cbe600640 1 -- 172.21.15.42:0/673353371 >> 172.21.15.42:0/673353371 conn(0x7f7cb80b3f90 msgr2=0x7f7cb80b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:58:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.921+0000 7f7cbe600640 1 -- 172.21.15.42:0/673353371 shutdown_connections 2024-05-28T13:58:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.921+0000 7f7cbe600640 1 -- 172.21.15.42:0/673353371 wait complete. 2024-05-28T13:58:59.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cbe600640 1 Processor -- start 2024-05-28T13:58:59.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cbe600640 1 -- start start 2024-05-28T13:58:59.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cbe600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7cb809b160 0x7f7cb8156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cbe600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cb809b8e0 0x7f7cb815ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cbe600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cb80b0e80 0x7f7cb815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cbe600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb80af350 con 0x7f7cb80b0e80 2024-05-28T13:58:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cbe600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb80af1d0 con 0x7f7cb809b8e0 2024-05-28T13:58:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cbe600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb80af4d0 con 0x7f7cb809b160 2024-05-28T13:58:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cb80b0e80 0x7f7cb815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cb80b0e80 0x7f7cb815d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52632/0 (socket says 172.21.15.42:52632) 2024-05-28T13:58:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cb7e00640 1 -- 172.21.15.42:0/2340674341 learned_addr learned my addr 172.21.15.42:0/2340674341 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:58:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cb7400640 1 --2- 172.21.15.42:0/2340674341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7cb809b160 0x7f7cb8156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7caea00640 1 --2- 172.21.15.42:0/2340674341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cb809b8e0 0x7f7cb815ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cb7e00640 1 -- 172.21.15.42:0/2340674341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7cb809b160 msgr2=0x7f7cb8156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cb7e00640 1 --2- 172.21.15.42:0/2340674341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7cb809b160 0x7f7cb8156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cb7e00640 1 -- 172.21.15.42:0/2340674341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cb809b8e0 msgr2=0x7f7cb815ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:58:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cb7e00640 1 --2- 172.21.15.42:0/2340674341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cb809b8e0 0x7f7cb815ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:58:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cb7e00640 1 -- 172.21.15.42:0/2340674341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb815f990 con 0x7f7cb80b0e80 2024-05-28T13:58:59.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cb7e00640 1 --2- 172.21.15.42:0/2340674341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cb80b0e80 0x7f7cb815d8e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000efc0 tx=0x7f7cb000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:59.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cafe00640 1 -- 172.21.15.42:0/2340674341 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0018020 con 0x7f7cb80b0e80 2024-05-28T13:58:59.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cafe00640 1 -- 172.21.15.42:0/2340674341 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb0014480 con 0x7f7cb80b0e80 2024-05-28T13:58:59.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cafe00640 1 -- 172.21.15.42:0/2340674341 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0014870 con 0x7f7cb80b0e80 2024-05-28T13:58:59.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cbe600640 1 -- 172.21.15.42:0/2340674341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb815fc70 con 0x7f7cb80b0e80 2024-05-28T13:58:59.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cbe600640 1 -- 172.21.15.42:0/2340674341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb8160230 con 0x7f7cb80b0e80 2024-05-28T13:58:59.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.925+0000 7f7cbe600640 1 -- 172.21.15.42:0/2340674341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb80a0550 con 0x7f7cb80b0e80 2024-05-28T13:58:59.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.929+0000 7f7cafe00640 1 -- 172.21.15.42:0/2340674341 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7cb001b020 con 0x7f7cb80b0e80 2024-05-28T13:58:59.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.929+0000 7f7cafe00640 1 --2- 172.21.15.42:0/2340674341 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c94068380 0x7f7c9406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:58:59.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.929+0000 7f7cafe00640 1 -- 172.21.15.42:0/2340674341 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7cb0087860 con 0x7f7cb80b0e80 2024-05-28T13:58:59.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.929+0000 7f7cb7400640 1 --2- 172.21.15.42:0/2340674341 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c94068380 0x7f7c9406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:58:59.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.929+0000 7f7cb7400640 1 --2- 172.21.15.42:0/2340674341 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c94068380 0x7f7c9406a840 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80af000 tx=0x7f7ca8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:58:59.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:58:59.933+0000 7f7cafe00640 1 -- 172.21.15.42:0/2340674341 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7cb0022030 con 0x7f7cb80b0e80 2024-05-28T13:59:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.097+0000 7f7cbe600640 1 -- 172.21.15.42:0/2340674341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7cb80c7900 con 0x7f7cb80b0e80 2024-05-28T13:59:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.101+0000 7f7cafe00640 1 -- 172.21.15.42:0/2340674341 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f7cb0010040 con 0x7f7cb80b0e80 2024-05-28T13:59:00.085 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:59:00.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.101+0000 7f7cbe600640 1 -- 172.21.15.42:0/2340674341 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c94068380 msgr2=0x7f7c9406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.101+0000 7f7cbe600640 1 --2- 172.21.15.42:0/2340674341 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c94068380 0x7f7c9406a840 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80af000 tx=0x7f7ca8008040 comp rx=0 tx=0).stop 2024-05-28T13:59:00.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.101+0000 7f7cbe600640 1 -- 172.21.15.42:0/2340674341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cb80b0e80 msgr2=0x7f7cb815d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.101+0000 7f7cbe600640 1 --2- 172.21.15.42:0/2340674341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cb80b0e80 0x7f7cb815d8e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000efc0 tx=0x7f7cb000c490 comp rx=0 tx=0).stop 2024-05-28T13:59:00.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.101+0000 7f7cbe600640 1 -- 172.21.15.42:0/2340674341 shutdown_connections 2024-05-28T13:59:00.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.101+0000 7f7cbe600640 1 --2- 172.21.15.42:0/2340674341 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c94068380 0x7f7c9406a840 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.101+0000 7f7cbe600640 1 --2- 172.21.15.42:0/2340674341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cb80b0e80 0x7f7cb815d8e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.101+0000 7f7cbe600640 1 --2- 172.21.15.42:0/2340674341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cb809b8e0 0x7f7cb815ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.101+0000 7f7cbe600640 1 --2- 172.21.15.42:0/2340674341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7cb809b160 0x7f7cb8156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.101+0000 7f7cbe600640 1 -- 172.21.15.42:0/2340674341 >> 172.21.15.42:0/2340674341 conn(0x7f7cb80b3f90 msgr2=0x7f7cb80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:00.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.101+0000 7f7cbe600640 1 -- 172.21.15.42:0/2340674341 shutdown_connections 2024-05-28T13:59:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.101+0000 7f7cbe600640 1 -- 172.21.15.42:0/2340674341 wait complete. 2024-05-28T13:59:00.106 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:59:00.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.265+0000 7f537f400640 1 Processor -- start 2024-05-28T13:59:00.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.265+0000 7f537f400640 1 -- start start 2024-05-28T13:59:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.265+0000 7f537f400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537815e870 0x7f53780d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f537f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f53780b7d70 0x7f537809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f537f400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53780d1a80 con 0x7f537815ec50 2024-05-28T13:59:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f537f400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f53780cd500 con 0x7f53780b7d70 2024-05-28T13:59:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f537f400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f53780ce9a0 con 0x7f537815e870 2024-05-28T13:59:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f537d400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f53780b7d70 0x7f537809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f537d400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f53780b7d70 0x7f537809aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52638/0 (socket says 172.21.15.42:52638) 2024-05-28T13:59:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f537d400640 1 -- 172.21.15.42:0/2571392796 learned_addr learned my addr 172.21.15.42:0/2571392796 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:00.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f537ca00640 1 --2- 172.21.15.42:0/2571392796 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537815e870 0x7f53780d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f537d400640 1 -- 172.21.15.42:0/2571392796 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537815e870 msgr2=0x7f53780d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:00.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f537d400640 1 --2- 172.21.15.42:0/2571392796 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537815e870 0x7f53780d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f537d400640 1 -- 172.21.15.42:0/2571392796 >> v1:172.21.15.63:6789/0 conn(0x7f537815ec50 legacy=0x7f53780b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f537d400640 1 -- 172.21.15.42:0/2571392796 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53780ccbd0 con 0x7f53780b7d70 2024-05-28T13:59:00.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f537d400640 1 --2- 172.21.15.42:0/2571392796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53780b7d70 0x7f537809aa50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5370009e50 tx=0x7f537001f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4988d26248b8d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:00.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f5377e00640 1 -- 172.21.15.42:0/2571392796 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53700b4070 con 0x7f53780b7d70 2024-05-28T13:59:00.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f5377e00640 1 -- 172.21.15.42:0/2571392796 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53700af400 con 0x7f53780b7d70 2024-05-28T13:59:00.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.269+0000 7f5377e00640 1 -- 172.21.15.42:0/2571392796 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53700af7d0 con 0x7f53780b7d70 2024-05-28T13:59:00.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 -- 172.21.15.42:0/2571392796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53780b7d70 msgr2=0x7f537809aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 --2- 172.21.15.42:0/2571392796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53780b7d70 0x7f537809aa50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5370009e50 tx=0x7f537001f210 comp rx=0 tx=0).stop 2024-05-28T13:59:00.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 -- 172.21.15.42:0/2571392796 shutdown_connections 2024-05-28T13:59:00.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 --2- 172.21.15.42:0/2571392796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53780b7d70 0x7f537809aa50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 --2- 172.21.15.42:0/2571392796 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537815e870 0x7f53780d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 -- 172.21.15.42:0/2571392796 >> 172.21.15.42:0/2571392796 conn(0x7f53780b3fd0 msgr2=0x7f53780b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:00.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 -- 172.21.15.42:0/2571392796 shutdown_connections 2024-05-28T13:59:00.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 -- 172.21.15.42:0/2571392796 wait complete. 2024-05-28T13:59:00.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 Processor -- start 2024-05-28T13:59:00.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 -- start start 2024-05-28T13:59:00.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53780b7d70 0x7f5378158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f537815e870 0x7f53780c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537815ec50 0x7f53780c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f53780d1770 con 0x7f53780b7d70 2024-05-28T13:59:00.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53780cd190 con 0x7f537815e870 2024-05-28T13:59:00.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537f400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f53780b5330 con 0x7f537815ec50 2024-05-28T13:59:00.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53780b7d70 0x7f5378158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53780b7d70 0x7f5378158ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52648/0 (socket says 172.21.15.42:52648) 2024-05-28T13:59:00.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537ca00640 1 -- 172.21.15.42:0/2493097837 learned_addr learned my addr 172.21.15.42:0/2493097837 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:00.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f537d400640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537815ec50 0x7f53780c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.273+0000 7f536fe00640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f537815e870 0x7f53780c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.277+0000 7f537ca00640 1 -- 172.21.15.42:0/2493097837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537815ec50 msgr2=0x7f53780c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.277+0000 7f537ca00640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537815ec50 0x7f53780c9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.277+0000 7f537ca00640 1 -- 172.21.15.42:0/2493097837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f537815e870 msgr2=0x7f53780c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.277+0000 7f537ca00640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f537815e870 0x7f53780c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.277+0000 7f537ca00640 1 -- 172.21.15.42:0/2493097837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53780d1f10 con 0x7f53780b7d70 2024-05-28T13:59:00.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.277+0000 7f537d400640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537815ec50 0x7f53780c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:00.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.277+0000 7f537ca00640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53780b7d70 0x7f5378158ab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f536800e930 tx=0x7f536800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:00.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.277+0000 7f5375600640 1 -- 172.21.15.42:0/2493097837 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368017070 con 0x7f53780b7d70 2024-05-28T13:59:00.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.277+0000 7f5375600640 1 -- 172.21.15.42:0/2493097837 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f536800ca00 con 0x7f53780b7d70 2024-05-28T13:59:00.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.277+0000 7f5375600640 1 -- 172.21.15.42:0/2493097837 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536800cdb0 con 0x7f53780b7d70 2024-05-28T13:59:00.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.277+0000 7f537f400640 1 -- 172.21.15.42:0/2493097837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537815bef0 con 0x7f53780b7d70 2024-05-28T13:59:00.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.277+0000 7f537f400640 1 -- 172.21.15.42:0/2493097837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537815c4b0 con 0x7f53780b7d70 2024-05-28T13:59:00.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.277+0000 7f537f400640 1 -- 172.21.15.42:0/2493097837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537815e0a0 con 0x7f53780b7d70 2024-05-28T13:59:00.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.277+0000 7f5375600640 1 -- 172.21.15.42:0/2493097837 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f536801a070 con 0x7f53780b7d70 2024-05-28T13:59:00.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.281+0000 7f5375600640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5354068380 0x7f535406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.281+0000 7f5375600640 1 -- 172.21.15.42:0/2493097837 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5368086dc0 con 0x7f53780b7d70 2024-05-28T13:59:00.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.281+0000 7f536fe00640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5354068380 0x7f535406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.281+0000 7f536fe00640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5354068380 0x7f535406a840 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f537806d5d0 tx=0x7f5360009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:00.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.285+0000 7f5375600640 1 -- 172.21.15.42:0/2493097837 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f53680573a0 con 0x7f53780b7d70 2024-05-28T13:59:00.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.445+0000 7f537f400640 1 -- 172.21.15.42:0/2493097837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f53780b3760 con 0x7f53780b7d70 2024-05-28T13:59:00.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.449+0000 7f5375600640 1 -- 172.21.15.42:0/2493097837 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f5368056ca0 con 0x7f53780b7d70 2024-05-28T13:59:00.433 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:59:00.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.449+0000 7f537f400640 1 -- 172.21.15.42:0/2493097837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5354068380 msgr2=0x7f535406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.449+0000 7f537f400640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5354068380 0x7f535406a840 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f537806d5d0 tx=0x7f5360009040 comp rx=0 tx=0).stop 2024-05-28T13:59:00.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.449+0000 7f537f400640 1 -- 172.21.15.42:0/2493097837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53780b7d70 msgr2=0x7f5378158ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.449+0000 7f537f400640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53780b7d70 0x7f5378158ab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f536800e930 tx=0x7f536800edf0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.449+0000 7f537f400640 1 -- 172.21.15.42:0/2493097837 shutdown_connections 2024-05-28T13:59:00.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.449+0000 7f537f400640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5354068380 0x7f535406a840 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.449+0000 7f537f400640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537815ec50 0x7f53780c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.449+0000 7f537f400640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f537815e870 0x7f53780c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.449+0000 7f537f400640 1 --2- 172.21.15.42:0/2493097837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53780b7d70 0x7f5378158ab0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.449+0000 7f537f400640 1 -- 172.21.15.42:0/2493097837 >> 172.21.15.42:0/2493097837 conn(0x7f53780b3fd0 msgr2=0x7f5378070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:00.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.449+0000 7f537f400640 1 -- 172.21.15.42:0/2493097837 shutdown_connections 2024-05-28T13:59:00.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.449+0000 7f537f400640 1 -- 172.21.15.42:0/2493097837 wait complete. 2024-05-28T13:59:00.454 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:59:00.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92d3800640 1 Processor -- start 2024-05-28T13:59:00.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92d3800640 1 -- start start 2024-05-28T13:59:00.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92d3800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f92cc148090 0x7f92cc0b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92d3800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92cc0b4c10 0x7f92cc071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92d3800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92cc0b3690 con 0x7f92cc072320 2024-05-28T13:59:00.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92d3800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f92cc148470 con 0x7f92cc0b4c10 2024-05-28T13:59:00.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92d3800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f92cc072770 con 0x7f92cc148090 2024-05-28T13:59:00.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92cbe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92cc0b4c10 0x7f92cc071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92cbe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92cc0b4c10 0x7f92cc071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52656/0 (socket says 172.21.15.42:52656) 2024-05-28T13:59:00.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92d0e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f92cc148090 0x7f92cc0b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92cbe00640 1 -- 172.21.15.42:0/3569851053 learned_addr learned my addr 172.21.15.42:0/3569851053 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:00.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92cbe00640 1 -- 172.21.15.42:0/3569851053 >> v2:172.21.15.63:3300/0 conn(0x7f92cc148090 msgr2=0x7f92cc0b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:00.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92cbe00640 1 --2- 172.21.15.42:0/3569851053 >> v2:172.21.15.63:3300/0 conn(0x7f92cc148090 0x7f92cc0b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92cbe00640 1 -- 172.21.15.42:0/3569851053 >> v1:172.21.15.63:6789/0 conn(0x7f92cc072320 legacy=0x7f92cc1623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.613+0000 7f92cbe00640 1 -- 172.21.15.42:0/3569851053 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc0b3970 con 0x7f92cc0b4c10 2024-05-28T13:59:00.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.617+0000 7f92cbe00640 1 --2- 172.21.15.42:0/3569851053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc0b4c10 0x7f92cc071de0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f92cc071730 tx=0x7f92b801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e835b1c69c6650 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:00.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.617+0000 7f92cb400640 1 -- 172.21.15.42:0/3569851053 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b80b2070 con 0x7f92cc0b4c10 2024-05-28T13:59:00.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.617+0000 7f92cb400640 1 -- 172.21.15.42:0/3569851053 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b80ad680 con 0x7f92cc0b4c10 2024-05-28T13:59:00.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.617+0000 7f92cb400640 1 -- 172.21.15.42:0/3569851053 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b80ada50 con 0x7f92cc0b4c10 2024-05-28T13:59:00.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.617+0000 7f92d3800640 1 -- 172.21.15.42:0/3569851053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc0b4c10 msgr2=0x7f92cc071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.617+0000 7f92d3800640 1 --2- 172.21.15.42:0/3569851053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc0b4c10 0x7f92cc071de0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f92cc071730 tx=0x7f92b801c490 comp rx=0 tx=0).stop 2024-05-28T13:59:00.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.617+0000 7f92d3800640 1 -- 172.21.15.42:0/3569851053 shutdown_connections 2024-05-28T13:59:00.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.617+0000 7f92d3800640 1 --2- 172.21.15.42:0/3569851053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc0b4c10 0x7f92cc071de0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.617+0000 7f92d3800640 1 --2- 172.21.15.42:0/3569851053 >> v2:172.21.15.63:3300/0 conn(0x7f92cc148090 0x7f92cc0b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.617+0000 7f92d3800640 1 -- 172.21.15.42:0/3569851053 >> 172.21.15.42:0/3569851053 conn(0x7f92cc0b21f0 msgr2=0x7f92cc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:00.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.617+0000 7f92d3800640 1 -- 172.21.15.42:0/3569851053 shutdown_connections 2024-05-28T13:59:00.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.617+0000 7f92d3800640 1 -- 172.21.15.42:0/3569851053 wait complete. 2024-05-28T13:59:00.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d3800640 1 Processor -- start 2024-05-28T13:59:00.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d3800640 1 -- start start 2024-05-28T13:59:00.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d3800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc072320 0x7f92cc15ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d3800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92cc148090 0x7f92cc0b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d3800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92cc158150 0x7f92cc161700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d3800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc0b57e0 con 0x7f92cc072320 2024-05-28T13:59:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d3800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc0b5660 con 0x7f92cc158150 2024-05-28T13:59:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d3800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f92cc16b540 con 0x7f92cc148090 2024-05-28T13:59:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc072320 0x7f92cc15ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc072320 0x7f92cc15ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52666/0 (socket says 172.21.15.42:52666) 2024-05-28T13:59:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d0e00640 1 -- 172.21.15.42:0/3771547803 learned_addr learned my addr 172.21.15.42:0/3771547803 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92cbe00640 1 --2- 172.21.15.42:0/3771547803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92cc148090 0x7f92cc0b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d1800640 1 --2- 172.21.15.42:0/3771547803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92cc158150 0x7f92cc161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d0e00640 1 -- 172.21.15.42:0/3771547803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92cc148090 msgr2=0x7f92cc0b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d0e00640 1 --2- 172.21.15.42:0/3771547803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92cc148090 0x7f92cc0b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d0e00640 1 -- 172.21.15.42:0/3771547803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92cc158150 msgr2=0x7f92cc161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d0e00640 1 --2- 172.21.15.42:0/3771547803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92cc158150 0x7f92cc161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d0e00640 1 -- 172.21.15.42:0/3771547803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc160660 con 0x7f92cc072320 2024-05-28T13:59:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d0e00640 1 --2- 172.21.15.42:0/3771547803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc072320 0x7f92cc15ff60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f92c000e8a0 tx=0x7f92c000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92c8c00640 1 -- 172.21.15.42:0/3771547803 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0017070 con 0x7f92cc072320 2024-05-28T13:59:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92c8c00640 1 -- 172.21.15.42:0/3771547803 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c000c940 con 0x7f92cc072320 2024-05-28T13:59:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d3800640 1 -- 172.21.15.42:0/3771547803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92cc160940 con 0x7f92cc072320 2024-05-28T13:59:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92c8c00640 1 -- 172.21.15.42:0/3771547803 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c000ccf0 con 0x7f92cc072320 2024-05-28T13:59:00.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.621+0000 7f92d3800640 1 -- 172.21.15.42:0/3771547803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92cc160f00 con 0x7f92cc072320 2024-05-28T13:59:00.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.625+0000 7f92d3800640 1 -- 172.21.15.42:0/3771547803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92cc0738b0 con 0x7f92cc072320 2024-05-28T13:59:00.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.625+0000 7f92c8c00640 1 -- 172.21.15.42:0/3771547803 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f92c001a070 con 0x7f92cc072320 2024-05-28T13:59:00.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.625+0000 7f92c8c00640 1 --2- 172.21.15.42:0/3771547803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92ac0683b0 0x7f92ac06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.625+0000 7f92c8c00640 1 -- 172.21.15.42:0/3771547803 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f92c0086b80 con 0x7f92cc072320 2024-05-28T13:59:00.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.625+0000 7f92cbe00640 1 --2- 172.21.15.42:0/3771547803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92ac0683b0 0x7f92ac06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.625+0000 7f92cbe00640 1 --2- 172.21.15.42:0/3771547803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92ac0683b0 0x7f92ac06a870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f92b80af000 tx=0x7f92b8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:00.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.629+0000 7f92c8c00640 1 -- 172.21.15.42:0/3771547803 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f92c0056ae0 con 0x7f92cc072320 2024-05-28T13:59:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.797+0000 7f92d3800640 1 -- 172.21.15.42:0/3771547803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f92cc04fdd0 con 0x7f92cc072320 2024-05-28T13:59:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.797+0000 7f92c8c00640 1 -- 172.21.15.42:0/3771547803 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f92c005ade0 con 0x7f92cc072320 2024-05-28T13:59:00.782 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:59:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.797+0000 7f92d3800640 1 -- 172.21.15.42:0/3771547803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92ac0683b0 msgr2=0x7f92ac06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.797+0000 7f92d3800640 1 --2- 172.21.15.42:0/3771547803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92ac0683b0 0x7f92ac06a870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f92b80af000 tx=0x7f92b8008040 comp rx=0 tx=0).stop 2024-05-28T13:59:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.797+0000 7f92d3800640 1 -- 172.21.15.42:0/3771547803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc072320 msgr2=0x7f92cc15ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.797+0000 7f92d3800640 1 --2- 172.21.15.42:0/3771547803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc072320 0x7f92cc15ff60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f92c000e8a0 tx=0x7f92c000eca0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.797+0000 7f92d3800640 1 -- 172.21.15.42:0/3771547803 shutdown_connections 2024-05-28T13:59:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.797+0000 7f92d3800640 1 --2- 172.21.15.42:0/3771547803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92ac0683b0 0x7f92ac06a870 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.797+0000 7f92d3800640 1 --2- 172.21.15.42:0/3771547803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92cc158150 0x7f92cc161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.797+0000 7f92d3800640 1 --2- 172.21.15.42:0/3771547803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92cc148090 0x7f92cc0b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.797+0000 7f92d3800640 1 --2- 172.21.15.42:0/3771547803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc072320 0x7f92cc15ff60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.797+0000 7f92d3800640 1 -- 172.21.15.42:0/3771547803 >> 172.21.15.42:0/3771547803 conn(0x7f92cc0b21f0 msgr2=0x7f92cc1415a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.797+0000 7f92d3800640 1 -- 172.21.15.42:0/3771547803 shutdown_connections 2024-05-28T13:59:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.797+0000 7f92d3800640 1 -- 172.21.15.42:0/3771547803 wait complete. 2024-05-28T13:59:00.803 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T13:59:00.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe75c400640 1 Processor -- start 2024-05-28T13:59:00.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe75c400640 1 -- start start 2024-05-28T13:59:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe75c400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75409aa50 0x7fe75409ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe75c400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe7540b4bc0 0x7fe7540cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe75c400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe75409b370 con 0x7fe7540cd8b0 2024-05-28T13:59:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe75c400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fe7540ccee0 con 0x7fe7540b4bc0 2024-05-28T13:59:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe75c400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7541622a0 con 0x7fe75409aa50 2024-05-28T13:59:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe759000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe7540b4bc0 0x7fe7540cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe759000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe7540b4bc0 0x7fe7540cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52682/0 (socket says 172.21.15.42:52682) 2024-05-28T13:59:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe759000640 1 -- 172.21.15.42:0/619302673 learned_addr learned my addr 172.21.15.42:0/619302673 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe759a00640 1 --2- 172.21.15.42:0/619302673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75409aa50 0x7fe75409ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe759000640 1 -- 172.21.15.42:0/619302673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75409aa50 msgr2=0x7fe75409ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe759000640 1 --2- 172.21.15.42:0/619302673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75409aa50 0x7fe75409ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe759000640 1 -- 172.21.15.42:0/619302673 >> v1:172.21.15.63:6789/0 conn(0x7fe7540cd8b0 legacy=0x7fe7540cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.961+0000 7fe759000640 1 -- 172.21.15.42:0/619302673 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75409b620 con 0x7fe7540b4bc0 2024-05-28T13:59:00.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.965+0000 7fe759000640 1 --2- 172.21.15.42:0/619302673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7540b4bc0 0x7fe7540cd370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe740009870 tx=0x7fe74001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba22c0eb16c6d10c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:00.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.965+0000 7fe753e00640 1 -- 172.21.15.42:0/619302673 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7400a95f0 con 0x7fe7540b4bc0 2024-05-28T13:59:00.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.965+0000 7fe753e00640 1 -- 172.21.15.42:0/619302673 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7400a9790 con 0x7fe7540b4bc0 2024-05-28T13:59:00.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.965+0000 7fe753e00640 1 -- 172.21.15.42:0/619302673 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7400a9b60 con 0x7fe7540b4bc0 2024-05-28T13:59:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.965+0000 7fe75c400640 1 -- 172.21.15.42:0/619302673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7540b4bc0 msgr2=0x7fe7540cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.965+0000 7fe75c400640 1 --2- 172.21.15.42:0/619302673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7540b4bc0 0x7fe7540cd370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe740009870 tx=0x7fe74001c530 comp rx=0 tx=0).stop 2024-05-28T13:59:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.965+0000 7fe75c400640 1 -- 172.21.15.42:0/619302673 shutdown_connections 2024-05-28T13:59:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.965+0000 7fe75c400640 1 --2- 172.21.15.42:0/619302673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7540b4bc0 0x7fe7540cd370 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.965+0000 7fe75c400640 1 --2- 172.21.15.42:0/619302673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75409aa50 0x7fe75409ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.965+0000 7fe75c400640 1 -- 172.21.15.42:0/619302673 >> 172.21.15.42:0/619302673 conn(0x7fe7540b4010 msgr2=0x7fe7540b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.965+0000 7fe75c400640 1 -- 172.21.15.42:0/619302673 shutdown_connections 2024-05-28T13:59:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.965+0000 7fe75c400640 1 -- 172.21.15.42:0/619302673 wait complete. 2024-05-28T13:59:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe75c400640 1 Processor -- start 2024-05-28T13:59:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe75c400640 1 -- start start 2024-05-28T13:59:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe75c400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75409aa50 0x7fe754151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe75c400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7540b4bc0 0x7fe75415e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe75c400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7540cd8b0 0x7fe75415eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe75c400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe7540ce5d0 con 0x7fe7540cd8b0 2024-05-28T13:59:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe75c400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7540ce450 con 0x7fe7540b4bc0 2024-05-28T13:59:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe75c400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe75416b5a0 con 0x7fe75409aa50 2024-05-28T13:59:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe75a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7540cd8b0 0x7fe75415eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe759a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75409aa50 0x7fe754151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe759a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75409aa50 0x7fe754151300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:49980/0 (socket says 172.21.15.42:49980) 2024-05-28T13:59:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe759a00640 1 -- 172.21.15.42:0/504182727 learned_addr learned my addr 172.21.15.42:0/504182727 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe759000640 1 --2- 172.21.15.42:0/504182727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7540b4bc0 0x7fe75415e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe759a00640 1 -- 172.21.15.42:0/504182727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7540b4bc0 msgr2=0x7fe75415e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe759a00640 1 --2- 172.21.15.42:0/504182727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7540b4bc0 0x7fe75415e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe759a00640 1 -- 172.21.15.42:0/504182727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7540cd8b0 msgr2=0x7fe75415eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe759a00640 1 --2- 172.21.15.42:0/504182727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7540cd8b0 0x7fe75415eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe759a00640 1 -- 172.21.15.42:0/504182727 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7540ca460 con 0x7fe75409aa50 2024-05-28T13:59:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe759a00640 1 --2- 172.21.15.42:0/504182727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75409aa50 0x7fe754151300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe74800e870 tx=0x7fe74800ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe751600640 1 -- 172.21.15.42:0/504182727 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748017070 con 0x7fe75409aa50 2024-05-28T13:59:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe751600640 1 -- 172.21.15.42:0/504182727 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74800c940 con 0x7fe75409aa50 2024-05-28T13:59:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe751600640 1 -- 172.21.15.42:0/504182727 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74800ccf0 con 0x7fe75409aa50 2024-05-28T13:59:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe75c400640 1 -- 172.21.15.42:0/504182727 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7540ca740 con 0x7fe75409aa50 2024-05-28T13:59:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.969+0000 7fe75c400640 1 -- 172.21.15.42:0/504182727 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7540cad00 con 0x7fe75409aa50 2024-05-28T13:59:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.973+0000 7fe75c400640 1 -- 172.21.15.42:0/504182727 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7540a0630 con 0x7fe75409aa50 2024-05-28T13:59:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.973+0000 7fe751600640 1 -- 172.21.15.42:0/504182727 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe74801a070 con 0x7fe75409aa50 2024-05-28T13:59:00.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.973+0000 7fe751600640 1 --2- 172.21.15.42:0/504182727 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe734068380 0x7fe73406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:00.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.973+0000 7fe751600640 1 -- 172.21.15.42:0/504182727 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe748086b80 con 0x7fe75409aa50 2024-05-28T13:59:00.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.973+0000 7fe759000640 1 --2- 172.21.15.42:0/504182727 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe734068380 0x7fe73406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:00.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.973+0000 7fe759000640 1 --2- 172.21.15.42:0/504182727 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe734068380 0x7fe73406a840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe7400ae000 tx=0x7fe7400b3040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:00.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:00.977+0000 7fe751600640 1 -- 172.21.15.42:0/504182727 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe748056ae0 con 0x7fe75409aa50 2024-05-28T13:59:01.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.153+0000 7fe75c400640 1 -- 172.21.15.42:0/504182727 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe75406cc00 con 0x7fe75409aa50 2024-05-28T13:59:01.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.157+0000 7fe751600640 1 -- 172.21.15.42:0/504182727 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fe74805ade0 con 0x7fe75409aa50 2024-05-28T13:59:01.141 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T13:59:01.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.157+0000 7fe75c400640 1 -- 172.21.15.42:0/504182727 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe734068380 msgr2=0x7fe73406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:01.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.157+0000 7fe75c400640 1 --2- 172.21.15.42:0/504182727 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe734068380 0x7fe73406a840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe7400ae000 tx=0x7fe7400b3040 comp rx=0 tx=0).stop 2024-05-28T13:59:01.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.157+0000 7fe75c400640 1 -- 172.21.15.42:0/504182727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75409aa50 msgr2=0x7fe754151300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:01.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.157+0000 7fe75c400640 1 --2- 172.21.15.42:0/504182727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75409aa50 0x7fe754151300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe74800e870 tx=0x7fe74800ec70 comp rx=0 tx=0).stop 2024-05-28T13:59:01.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.157+0000 7fe75c400640 1 -- 172.21.15.42:0/504182727 shutdown_connections 2024-05-28T13:59:01.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.157+0000 7fe75c400640 1 --2- 172.21.15.42:0/504182727 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe734068380 0x7fe73406a840 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.157+0000 7fe75c400640 1 --2- 172.21.15.42:0/504182727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7540cd8b0 0x7fe75415eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.157+0000 7fe75c400640 1 --2- 172.21.15.42:0/504182727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7540b4bc0 0x7fe75415e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.157+0000 7fe75c400640 1 --2- 172.21.15.42:0/504182727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75409aa50 0x7fe754151300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.157+0000 7fe75c400640 1 -- 172.21.15.42:0/504182727 >> 172.21.15.42:0/504182727 conn(0x7fe7540b4010 msgr2=0x7fe75406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:01.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.157+0000 7fe75c400640 1 -- 172.21.15.42:0/504182727 shutdown_connections 2024-05-28T13:59:01.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.157+0000 7fe75c400640 1 -- 172.21.15.42:0/504182727 wait complete. 2024-05-28T13:59:01.163 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-05-28T13:59:01.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:01.163 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-28T13:59:01.197 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T13:59:01.197 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T13:59:01.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:01.197 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-28T13:59:01.254 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:59:01.254 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:59:01.262 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:59:01.263 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:59:01.314 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:59:01.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22e0000640 1 Processor -- start 2024-05-28T13:59:01.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22e0000640 1 -- start start 2024-05-28T13:59:01.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22e0000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f22d806f660 0x7f22d8073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:01.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22e0000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22d815ba40 0x7f22d80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:01.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22e0000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f22d8072dd0 0x7f22d80d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:01.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22e0000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f22d815f030 con 0x7f22d8072dd0 2024-05-28T13:59:01.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22e0000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f22d815f280 con 0x7f22d806f660 2024-05-28T13:59:01.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22e0000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f22d815f560 con 0x7f22d815ba40 2024-05-28T13:59:01.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22de000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f22d8072dd0 0x7f22d80d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:01.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22de000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f22d8072dd0 0x7f22d80d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52692/0 (socket says 172.21.15.42:52692) 2024-05-28T13:59:01.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22de000640 1 -- 172.21.15.42:0/2747341300 learned_addr learned my addr 172.21.15.42:0/2747341300 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:01.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22dcc00640 1 --2- 172.21.15.42:0/2747341300 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22d815ba40 0x7f22d80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:01.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22dd600640 1 --2- 172.21.15.42:0/2747341300 >> v2:172.21.15.63:3300/0 conn(0x7f22d806f660 0x7f22d8073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:01.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22de000640 1 -- 172.21.15.42:0/2747341300 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22d815ba40 msgr2=0x7f22d80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:01.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22de000640 1 --2- 172.21.15.42:0/2747341300 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22d815ba40 0x7f22d80d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22de000640 1 -- 172.21.15.42:0/2747341300 >> v2:172.21.15.63:3300/0 conn(0x7f22d806f660 msgr2=0x7f22d8073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:01.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22de000640 1 --2- 172.21.15.42:0/2747341300 >> v2:172.21.15.63:3300/0 conn(0x7f22d806f660 0x7f22d8073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.469+0000 7f22de000640 1 -- 172.21.15.42:0/2747341300 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d815f840 con 0x7f22d8072dd0 2024-05-28T13:59:01.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.473+0000 7f22de000640 1 --2- 172.21.15.42:0/2747341300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22d8072dd0 0x7f22d80d0c30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f22d80d13a0 tx=0x7f22d001e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa275f9475735e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:01.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.473+0000 7f22d7e00640 1 -- 172.21.15.42:0/2747341300 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d00b2070 con 0x7f22d8072dd0 2024-05-28T13:59:01.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.473+0000 7f22d7e00640 1 -- 172.21.15.42:0/2747341300 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d000b2d0 con 0x7f22d8072dd0 2024-05-28T13:59:01.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.473+0000 7f22d7e00640 1 -- 172.21.15.42:0/2747341300 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d000b480 con 0x7f22d8072dd0 2024-05-28T13:59:01.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.473+0000 7f22e0000640 1 -- 172.21.15.42:0/2747341300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22d8072dd0 msgr2=0x7f22d80d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:01.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.473+0000 7f22e0000640 1 --2- 172.21.15.42:0/2747341300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22d8072dd0 0x7f22d80d0c30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f22d80d13a0 tx=0x7f22d001e780 comp rx=0 tx=0).stop 2024-05-28T13:59:01.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.473+0000 7f22e0000640 1 -- 172.21.15.42:0/2747341300 shutdown_connections 2024-05-28T13:59:01.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.473+0000 7f22e0000640 1 --2- 172.21.15.42:0/2747341300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22d8072dd0 0x7f22d80d0c30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.473+0000 7f22e0000640 1 --2- 172.21.15.42:0/2747341300 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22d815ba40 0x7f22d80d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.473+0000 7f22e0000640 1 --2- 172.21.15.42:0/2747341300 >> v2:172.21.15.63:3300/0 conn(0x7f22d806f660 0x7f22d8073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.473+0000 7f22e0000640 1 -- 172.21.15.42:0/2747341300 >> 172.21.15.42:0/2747341300 conn(0x7f22d806bdf0 msgr2=0x7f22d806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.473+0000 7f22e0000640 1 -- 172.21.15.42:0/2747341300 shutdown_connections 2024-05-28T13:59:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.473+0000 7f22e0000640 1 -- 172.21.15.42:0/2747341300 wait complete. 2024-05-28T13:59:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22e0000640 1 Processor -- start 2024-05-28T13:59:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22e0000640 1 -- start start 2024-05-28T13:59:01.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22e0000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d806f660 0x7f22d80cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:01.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22e0000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22d8072dd0 0x7f22d815e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:01.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22e0000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22d8146290 0x7f22d80b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:01.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22e0000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8166f60 con 0x7f22d8072dd0 2024-05-28T13:59:01.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22e0000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8166de0 con 0x7f22d806f660 2024-05-28T13:59:01.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22e0000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f22d81670e0 con 0x7f22d8146290 2024-05-28T13:59:01.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22dcc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22d8072dd0 0x7f22d815e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:01.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22dcc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22d8072dd0 0x7f22d815e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52706/0 (socket says 172.21.15.42:52706) 2024-05-28T13:59:01.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22dcc00640 1 -- 172.21.15.42:0/1883309204 learned_addr learned my addr 172.21.15.42:0/1883309204 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:01.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22dd600640 1 --2- 172.21.15.42:0/1883309204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d806f660 0x7f22d80cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:01.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22de000640 1 --2- 172.21.15.42:0/1883309204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22d8146290 0x7f22d80b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:01.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22dcc00640 1 -- 172.21.15.42:0/1883309204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22d8146290 msgr2=0x7f22d80b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:01.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22dcc00640 1 --2- 172.21.15.42:0/1883309204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22d8146290 0x7f22d80b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22dcc00640 1 -- 172.21.15.42:0/1883309204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d806f660 msgr2=0x7f22d80cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:01.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22dcc00640 1 --2- 172.21.15.42:0/1883309204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d806f660 0x7f22d80cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22dcc00640 1 -- 172.21.15.42:0/1883309204 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d814f010 con 0x7f22d8072dd0 2024-05-28T13:59:01.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22dcc00640 1 --2- 172.21.15.42:0/1883309204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22d8072dd0 0x7f22d815e9e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f22c400d5f0 tx=0x7f22c400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:01.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22d5600640 1 -- 172.21.15.42:0/1883309204 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c40125f0 con 0x7f22d8072dd0 2024-05-28T13:59:01.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22d5600640 1 -- 172.21.15.42:0/1883309204 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c4019070 con 0x7f22d8072dd0 2024-05-28T13:59:01.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22e0000640 1 -- 172.21.15.42:0/1883309204 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d814f2b0 con 0x7f22d8072dd0 2024-05-28T13:59:01.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22d5600640 1 -- 172.21.15.42:0/1883309204 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c40129b0 con 0x7f22d8072dd0 2024-05-28T13:59:01.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.477+0000 7f22e0000640 1 -- 172.21.15.42:0/1883309204 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d814f870 con 0x7f22d8072dd0 2024-05-28T13:59:01.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.481+0000 7f22e0000640 1 -- 172.21.15.42:0/1883309204 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d8068a90 con 0x7f22d8072dd0 2024-05-28T13:59:01.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.481+0000 7f22d5600640 1 -- 172.21.15.42:0/1883309204 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f22c4012b50 con 0x7f22d8072dd0 2024-05-28T13:59:01.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.481+0000 7f22d5600640 1 --2- 172.21.15.42:0/1883309204 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22b8068380 0x7f22b806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:01.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.481+0000 7f22d5600640 1 -- 172.21.15.42:0/1883309204 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f22c4086bf0 con 0x7f22d8072dd0 2024-05-28T13:59:01.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.481+0000 7f22dd600640 1 --2- 172.21.15.42:0/1883309204 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22b8068380 0x7f22b806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:01.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.481+0000 7f22dd600640 1 --2- 172.21.15.42:0/1883309204 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22b8068380 0x7f22b806a840 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0045f0 tx=0x7f22cc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:01.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.485+0000 7f22d5600640 1 -- 172.21.15.42:0/1883309204 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f22c4056bf0 con 0x7f22d8072dd0 2024-05-28T13:59:01.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.649+0000 7f22e0000640 1 -- 172.21.15.42:0/1883309204 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22d8072bb0 con 0x7f22d8072dd0 2024-05-28T13:59:01.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.653+0000 7f22d5600640 1 -- 172.21.15.42:0/1883309204 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16545 (secure 0 0 0) 0x7f22c405aef0 con 0x7f22d8072dd0 2024-05-28T13:59:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:01.637 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:58:24.659233+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:59:01.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.653+0000 7f22e0000640 1 -- 172.21.15.42:0/1883309204 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22b8068380 msgr2=0x7f22b806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:01.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.653+0000 7f22e0000640 1 --2- 172.21.15.42:0/1883309204 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22b8068380 0x7f22b806a840 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0045f0 tx=0x7f22cc008040 comp rx=0 tx=0).stop 2024-05-28T13:59:01.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.653+0000 7f22e0000640 1 -- 172.21.15.42:0/1883309204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22d8072dd0 msgr2=0x7f22d815e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:01.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.653+0000 7f22e0000640 1 --2- 172.21.15.42:0/1883309204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22d8072dd0 0x7f22d815e9e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f22c400d5f0 tx=0x7f22c400dab0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.653+0000 7f22e0000640 1 -- 172.21.15.42:0/1883309204 shutdown_connections 2024-05-28T13:59:01.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.653+0000 7f22e0000640 1 --2- 172.21.15.42:0/1883309204 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22b8068380 0x7f22b806a840 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.653+0000 7f22e0000640 1 --2- 172.21.15.42:0/1883309204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22d8146290 0x7f22d80b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.653+0000 7f22e0000640 1 --2- 172.21.15.42:0/1883309204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22d8072dd0 0x7f22d815e9e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.653+0000 7f22e0000640 1 --2- 172.21.15.42:0/1883309204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d806f660 0x7f22d80cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.653+0000 7f22e0000640 1 -- 172.21.15.42:0/1883309204 >> 172.21.15.42:0/1883309204 conn(0x7f22d806bdf0 msgr2=0x7f22d8153680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:01.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.653+0000 7f22e0000640 1 -- 172.21.15.42:0/1883309204 shutdown_connections 2024-05-28T13:59:01.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.653+0000 7f22e0000640 1 -- 172.21.15.42:0/1883309204 wait complete. 2024-05-28T13:59:01.660 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:59:01.660 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:59:01.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.817+0000 7f8453000640 1 Processor -- start 2024-05-28T13:59:01.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.817+0000 7f8453000640 1 -- start start 2024-05-28T13:59:01.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.821+0000 7f8453000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f844c15c0c0 0x7f844c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:01.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.821+0000 7f8453000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f844c0afe40 con 0x7f844c144f80 2024-05-28T13:59:01.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.821+0000 7f8453000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f844c15c600 con 0x7f844c06ee20 2024-05-28T13:59:01.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.821+0000 7f8453000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f844c06f7c0 con 0x7f844c15c0c0 2024-05-28T13:59:01.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.821+0000 7f844be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f844c144f80 0x7f844c072960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34794/0 (socket says 172.21.15.42:34794) 2024-05-28T13:59:01.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.821+0000 7f844be00640 1 -- 172.21.15.42:0/651430517 learned_addr learned my addr 172.21.15.42:0/651430517 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:01.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.821+0000 7f8451000640 1 --2- 172.21.15.42:0/651430517 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f844c15c0c0 0x7f844c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:01.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.821+0000 7f844b400640 1 -- 172.21.15.42:0/651430517 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1152266982 0 0) 0x7f844c0afe40 con 0x7f844c144f80 2024-05-28T13:59:01.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.821+0000 7f844b400640 1 -- 172.21.15.42:0/651430517 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8430003530 con 0x7f844c144f80 2024-05-28T13:59:01.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.821+0000 7f844b400640 1 -- 172.21.15.42:0/651430517 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4293285857 0 0) 0x7f844c15c600 con 0x7f844c06ee20 2024-05-28T13:59:01.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.821+0000 7f844b400640 1 -- 172.21.15.42:0/651430517 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f844c0afe40 con 0x7f844c06ee20 2024-05-28T13:59:01.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.821+0000 7f8451000640 1 -- 172.21.15.42:0/651430517 >> v1:172.21.15.63:6789/0 conn(0x7f844c06ee20 legacy=0x7f844c15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:01.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.821+0000 7f8451000640 1 -- 172.21.15.42:0/651430517 >> v1:172.21.15.42:6789/0 conn(0x7f844c144f80 legacy=0x7f844c072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:01.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.821+0000 7f8451000640 1 -- 172.21.15.42:0/651430517 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c15c8e0 con 0x7f844c15c0c0 2024-05-28T13:59:01.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.825+0000 7f8451000640 1 --2- 172.21.15.42:0/651430517 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f844c15c0c0 0x7f844c06f280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8444009dc0 tx=0x7f844401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83286cf21971a608 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:01.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.825+0000 7f844b400640 1 -- 172.21.15.42:0/651430517 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84440b2070 con 0x7f844c15c0c0 2024-05-28T13:59:01.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.825+0000 7f844b400640 1 -- 172.21.15.42:0/651430517 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84440a5850 con 0x7f844c15c0c0 2024-05-28T13:59:01.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.825+0000 7f844b400640 1 -- 172.21.15.42:0/651430517 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84440a5c20 con 0x7f844c15c0c0 2024-05-28T13:59:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.825+0000 7f8453000640 1 -- 172.21.15.42:0/651430517 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f844c15c0c0 msgr2=0x7f844c06f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.825+0000 7f8453000640 1 --2- 172.21.15.42:0/651430517 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f844c15c0c0 0x7f844c06f280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8444009dc0 tx=0x7f844401c990 comp rx=0 tx=0).stop 2024-05-28T13:59:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.825+0000 7f8453000640 1 -- 172.21.15.42:0/651430517 shutdown_connections 2024-05-28T13:59:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.825+0000 7f8453000640 1 --2- 172.21.15.42:0/651430517 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f844c15c0c0 0x7f844c06f280 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.825+0000 7f8453000640 1 -- 172.21.15.42:0/651430517 >> 172.21.15.42:0/651430517 conn(0x7f844c0b21f0 msgr2=0x7f844c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.825+0000 7f8453000640 1 -- 172.21.15.42:0/651430517 shutdown_connections 2024-05-28T13:59:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.825+0000 7f8453000640 1 -- 172.21.15.42:0/651430517 wait complete. 2024-05-28T13:59:01.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.825+0000 7f8453000640 1 Processor -- start 2024-05-28T13:59:01.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.825+0000 7f8453000640 1 -- start start 2024-05-28T13:59:01.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8453000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f844c06ee20 0x7f844c13d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:01.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8453000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f844c144f80 0x7f844c1383c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:01.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8453000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f844c15c0c0 0x7f844c13b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:01.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8453000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f844c0724d0 con 0x7f844c15c0c0 2024-05-28T13:59:01.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8453000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f844c0b42e0 con 0x7f844c144f80 2024-05-28T13:59:01.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8453000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f844c06dd10 con 0x7f844c06ee20 2024-05-28T13:59:01.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8451000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f844c15c0c0 0x7f844c13b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:01.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8451000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f844c15c0c0 0x7f844c13b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52712/0 (socket says 172.21.15.42:52712) 2024-05-28T13:59:01.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8451000640 1 -- 172.21.15.42:0/326052746 learned_addr learned my addr 172.21.15.42:0/326052746 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:01.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8443e00640 1 --2- 172.21.15.42:0/326052746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f844c144f80 0x7f844c1383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:01.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f844be00640 1 --2- 172.21.15.42:0/326052746 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f844c06ee20 0x7f844c13d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8451000640 1 -- 172.21.15.42:0/326052746 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f844c06ee20 msgr2=0x7f844c13d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8451000640 1 --2- 172.21.15.42:0/326052746 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f844c06ee20 0x7f844c13d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8451000640 1 -- 172.21.15.42:0/326052746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f844c144f80 msgr2=0x7f844c1383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8451000640 1 --2- 172.21.15.42:0/326052746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f844c144f80 0x7f844c1383c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8451000640 1 -- 172.21.15.42:0/326052746 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c13c090 con 0x7f844c15c0c0 2024-05-28T13:59:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8451000640 1 --2- 172.21.15.42:0/326052746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f844c15c0c0 0x7f844c13b6b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f84440195a0 tx=0x7f8444019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8448c00640 1 -- 172.21.15.42:0/326052746 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84440b2050 con 0x7f844c15c0c0 2024-05-28T13:59:01.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8448c00640 1 -- 172.21.15.42:0/326052746 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84440be070 con 0x7f844c15c0c0 2024-05-28T13:59:01.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8453000640 1 -- 172.21.15.42:0/326052746 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844c13c370 con 0x7f844c15c0c0 2024-05-28T13:59:01.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8448c00640 1 -- 172.21.15.42:0/326052746 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444009450 con 0x7f844c15c0c0 2024-05-28T13:59:01.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8453000640 1 -- 172.21.15.42:0/326052746 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844c13c930 con 0x7f844c15c0c0 2024-05-28T13:59:01.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8453000640 1 -- 172.21.15.42:0/326052746 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f844c0738b0 con 0x7f844c15c0c0 2024-05-28T13:59:01.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.829+0000 7f8448c00640 1 -- 172.21.15.42:0/326052746 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f84440ad870 con 0x7f844c15c0c0 2024-05-28T13:59:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.833+0000 7f8448c00640 1 --2- 172.21.15.42:0/326052746 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8428068380 0x7f842806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.833+0000 7f8448c00640 1 -- 172.21.15.42:0/326052746 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8444121940 con 0x7f844c15c0c0 2024-05-28T13:59:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.833+0000 7f844be00640 1 --2- 172.21.15.42:0/326052746 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8428068380 0x7f842806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:01.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.833+0000 7f844be00640 1 --2- 172.21.15.42:0/326052746 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8428068380 0x7f842806a840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f843c007fd0 tx=0x7f843c007d80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:01.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:01.837+0000 7f8448c00640 1 -- 172.21.15.42:0/326052746 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8444016580 con 0x7f844c15c0c0 2024-05-28T13:59:01.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.001+0000 7f8453000640 1 -- 172.21.15.42:0/326052746 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f844c04fdd0 con 0x7f844c15c0c0 2024-05-28T13:59:01.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.001+0000 7f8448c00640 1 -- 172.21.15.42:0/326052746 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f84440f19c0 con 0x7f844c15c0c0 2024-05-28T13:59:01.988 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:59:01.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.005+0000 7f8453000640 1 -- 172.21.15.42:0/326052746 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8428068380 msgr2=0x7f842806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:01.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.005+0000 7f8453000640 1 --2- 172.21.15.42:0/326052746 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8428068380 0x7f842806a840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f843c007fd0 tx=0x7f843c007d80 comp rx=0 tx=0).stop 2024-05-28T13:59:01.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.005+0000 7f8453000640 1 -- 172.21.15.42:0/326052746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f844c15c0c0 msgr2=0x7f844c13b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:01.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.005+0000 7f8453000640 1 --2- 172.21.15.42:0/326052746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f844c15c0c0 0x7f844c13b6b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f84440195a0 tx=0x7f8444019730 comp rx=0 tx=0).stop 2024-05-28T13:59:01.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.005+0000 7f8453000640 1 -- 172.21.15.42:0/326052746 shutdown_connections 2024-05-28T13:59:01.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.005+0000 7f8453000640 1 --2- 172.21.15.42:0/326052746 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8428068380 0x7f842806a840 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.005+0000 7f8453000640 1 --2- 172.21.15.42:0/326052746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f844c15c0c0 0x7f844c13b6b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.005+0000 7f8453000640 1 --2- 172.21.15.42:0/326052746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f844c144f80 0x7f844c1383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.005+0000 7f8453000640 1 --2- 172.21.15.42:0/326052746 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f844c06ee20 0x7f844c13d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:01.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.005+0000 7f8453000640 1 -- 172.21.15.42:0/326052746 >> 172.21.15.42:0/326052746 conn(0x7f844c0b21f0 msgr2=0x7f844c0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:01.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.005+0000 7f8453000640 1 -- 172.21.15.42:0/326052746 shutdown_connections 2024-05-28T13:59:01.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.005+0000 7f8453000640 1 -- 172.21.15.42:0/326052746 wait complete. 2024-05-28T13:59:02.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:59:02.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.165+0000 7ff4fe000640 1 Processor -- start 2024-05-28T13:59:02.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.169+0000 7ff4fe000640 1 -- start start 2024-05-28T13:59:02.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.169+0000 7ff4fe000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff4f806ee20 0x7ff4f806f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.169+0000 7ff4fe000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4f80afe40 con 0x7ff4f815b9b0 2024-05-28T13:59:02.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.169+0000 7ff4fe000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4f815c600 con 0x7ff4f8144f80 2024-05-28T13:59:02.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.169+0000 7ff4fe000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff4f806f7c0 con 0x7ff4f806ee20 2024-05-28T13:59:02.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.169+0000 7ff4efe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff4f806ee20 0x7ff4f806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.169+0000 7ff4efe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff4f806ee20 0x7ff4f806f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52728/0 (socket says 172.21.15.42:52728) 2024-05-28T13:59:02.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.169+0000 7ff4f7e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff4f815b9b0 0x7ff4f815be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34810/0 (socket says 172.21.15.42:34810) 2024-05-28T13:59:02.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.169+0000 7ff4efe00640 1 -- 172.21.15.42:0/1154815057 learned_addr learned my addr 172.21.15.42:0/1154815057 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:02.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.169+0000 7ff4efe00640 1 -- 172.21.15.42:0/1154815057 >> v1:172.21.15.63:6789/0 conn(0x7ff4f8144f80 legacy=0x7ff4f8072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:02.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.169+0000 7ff4efe00640 1 -- 172.21.15.42:0/1154815057 >> v1:172.21.15.42:6789/0 conn(0x7ff4f815b9b0 legacy=0x7ff4f815be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.169+0000 7ff4efe00640 1 -- 172.21.15.42:0/1154815057 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f815c8e0 con 0x7ff4f806ee20 2024-05-28T13:59:02.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.173+0000 7ff4efe00640 1 --2- 172.21.15.42:0/1154815057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4f806ee20 0x7ff4f806f280 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000bb10 tx=0x7ff4e001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a073c5079f55f97 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:02.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.173+0000 7ff4f6a00640 1 -- 172.21.15.42:0/1154815057 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00b4070 con 0x7ff4f806ee20 2024-05-28T13:59:02.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.173+0000 7ff4f6a00640 1 -- 172.21.15.42:0/1154815057 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e00a7ac0 con 0x7ff4f806ee20 2024-05-28T13:59:02.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.173+0000 7ff4f6a00640 1 -- 172.21.15.42:0/1154815057 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00af440 con 0x7ff4f806ee20 2024-05-28T13:59:02.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.173+0000 7ff4fe000640 1 -- 172.21.15.42:0/1154815057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4f806ee20 msgr2=0x7ff4f806f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.173+0000 7ff4fe000640 1 --2- 172.21.15.42:0/1154815057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4f806ee20 0x7ff4f806f280 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000bb10 tx=0x7ff4e001ebf0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.173+0000 7ff4fe000640 1 -- 172.21.15.42:0/1154815057 shutdown_connections 2024-05-28T13:59:02.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.173+0000 7ff4fe000640 1 --2- 172.21.15.42:0/1154815057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4f806ee20 0x7ff4f806f280 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.173+0000 7ff4fe000640 1 -- 172.21.15.42:0/1154815057 >> 172.21.15.42:0/1154815057 conn(0x7ff4f80b21f0 msgr2=0x7ff4f80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:02.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.173+0000 7ff4fe000640 1 -- 172.21.15.42:0/1154815057 shutdown_connections 2024-05-28T13:59:02.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.173+0000 7ff4fe000640 1 -- 172.21.15.42:0/1154815057 wait complete. 2024-05-28T13:59:02.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.173+0000 7ff4fe000640 1 Processor -- start 2024-05-28T13:59:02.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.173+0000 7ff4fe000640 1 -- start start 2024-05-28T13:59:02.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4fe000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4f806ee20 0x7ff4f80b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4fe000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4f8144f80 0x7ff4f80b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4fe000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4f815b9b0 0x7ff4f815acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4fe000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f80724d0 con 0x7ff4f8144f80 2024-05-28T13:59:02.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4fe000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f80b4260 con 0x7ff4f815b9b0 2024-05-28T13:59:02.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4fe000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f806dd10 con 0x7ff4f806ee20 2024-05-28T13:59:02.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4efe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4f8144f80 0x7ff4f80b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4efe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4f8144f80 0x7ff4f80b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52744/0 (socket says 172.21.15.42:52744) 2024-05-28T13:59:02.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4efe00640 1 -- 172.21.15.42:0/3962024865 learned_addr learned my addr 172.21.15.42:0/3962024865 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4efe00640 1 -- 172.21.15.42:0/3962024865 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4f806ee20 msgr2=0x7ff4f80b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4f7e00640 1 --2- 172.21.15.42:0/3962024865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4f815b9b0 0x7ff4f815acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4efe00640 1 --2- 172.21.15.42:0/3962024865 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4f806ee20 0x7ff4f80b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4efe00640 1 -- 172.21.15.42:0/3962024865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4f815b9b0 msgr2=0x7ff4f815acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4efe00640 1 --2- 172.21.15.42:0/3962024865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4f815b9b0 0x7ff4f815acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4efe00640 1 -- 172.21.15.42:0/3962024865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f813dee0 con 0x7ff4f8144f80 2024-05-28T13:59:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4f7e00640 1 --2- 172.21.15.42:0/3962024865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4f815b9b0 0x7ff4f815acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4efe00640 1 --2- 172.21.15.42:0/3962024865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4f8144f80 0x7ff4f80b0320 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000b2c0 tx=0x7ff4e001d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4ef400640 1 -- 172.21.15.42:0/3962024865 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00bf070 con 0x7ff4f8144f80 2024-05-28T13:59:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4ef400640 1 -- 172.21.15.42:0/3962024865 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e00ba660 con 0x7ff4f8144f80 2024-05-28T13:59:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4ef400640 1 -- 172.21.15.42:0/3962024865 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00b4070 con 0x7ff4f8144f80 2024-05-28T13:59:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4fe000640 1 -- 172.21.15.42:0/3962024865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f80b1a30 con 0x7ff4f8144f80 2024-05-28T13:59:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4fe000640 1 -- 172.21.15.42:0/3962024865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f8158a30 con 0x7ff4f8144f80 2024-05-28T13:59:02.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4fe000640 1 -- 172.21.15.42:0/3962024865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f80738b0 con 0x7ff4f8144f80 2024-05-28T13:59:02.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.177+0000 7ff4ef400640 1 -- 172.21.15.42:0/3962024865 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff4e00c2070 con 0x7ff4f8144f80 2024-05-28T13:59:02.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.181+0000 7ff4ef400640 1 --2- 172.21.15.42:0/3962024865 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff4d40683d0 0x7ff4d406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.181+0000 7ff4ef400640 1 -- 172.21.15.42:0/3962024865 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff4e01212d0 con 0x7ff4f8144f80 2024-05-28T13:59:02.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.181+0000 7ff4f7400640 1 --2- 172.21.15.42:0/3962024865 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff4d40683d0 0x7ff4d406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.181+0000 7ff4f7400640 1 --2- 172.21.15.42:0/3962024865 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff4d40683d0 0x7ff4d406a890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8003cf0 tx=0x7ff4e80079c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:02.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.185+0000 7ff4ef400640 1 -- 172.21.15.42:0/3962024865 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff4e00187e0 con 0x7ff4f8144f80 2024-05-28T13:59:02.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.349+0000 7ff4fe000640 1 -- 172.21.15.42:0/3962024865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff4f813d8d0 con 0x7ff4f8144f80 2024-05-28T13:59:02.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.349+0000 7ff4ef400640 1 -- 172.21.15.42:0/3962024865 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7ff4e00f1870 con 0x7ff4f8144f80 2024-05-28T13:59:02.333 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:59:02.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.349+0000 7ff4fe000640 1 -- 172.21.15.42:0/3962024865 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff4d40683d0 msgr2=0x7ff4d406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.349+0000 7ff4fe000640 1 --2- 172.21.15.42:0/3962024865 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff4d40683d0 0x7ff4d406a890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8003cf0 tx=0x7ff4e80079c0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.349+0000 7ff4fe000640 1 -- 172.21.15.42:0/3962024865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4f8144f80 msgr2=0x7ff4f80b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.349+0000 7ff4fe000640 1 --2- 172.21.15.42:0/3962024865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4f8144f80 0x7ff4f80b0320 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000b2c0 tx=0x7ff4e001d430 comp rx=0 tx=0).stop 2024-05-28T13:59:02.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.349+0000 7ff4fe000640 1 -- 172.21.15.42:0/3962024865 shutdown_connections 2024-05-28T13:59:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.349+0000 7ff4fe000640 1 --2- 172.21.15.42:0/3962024865 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff4d40683d0 0x7ff4d406a890 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.349+0000 7ff4fe000640 1 --2- 172.21.15.42:0/3962024865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4f815b9b0 0x7ff4f815acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.349+0000 7ff4fe000640 1 --2- 172.21.15.42:0/3962024865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4f8144f80 0x7ff4f80b0320 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.349+0000 7ff4fe000640 1 --2- 172.21.15.42:0/3962024865 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4f806ee20 0x7ff4f80b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.349+0000 7ff4fe000640 1 -- 172.21.15.42:0/3962024865 >> 172.21.15.42:0/3962024865 conn(0x7ff4f80b21f0 msgr2=0x7ff4f80adea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.349+0000 7ff4fe000640 1 -- 172.21.15.42:0/3962024865 shutdown_connections 2024-05-28T13:59:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.349+0000 7ff4fe000640 1 -- 172.21.15.42:0/3962024865 wait complete. 2024-05-28T13:59:02.355 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:59:02.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.513+0000 7f0fff600640 1 Processor -- start 2024-05-28T13:59:02.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.513+0000 7f0fff600640 1 -- start start 2024-05-28T13:59:02.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0fff600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff80ce3b0 0x7f0ff80ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0fff600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ff8160230 con 0x7f0ff8071de0 2024-05-28T13:59:02.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0fff600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ff8160510 con 0x7f0ff813fd10 2024-05-28T13:59:02.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0fff600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff81607f0 con 0x7f0ff80ce3b0 2024-05-28T13:59:02.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ffcc00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0ff8071de0 0x7f0ff80721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34814/0 (socket says 172.21.15.42:34814) 2024-05-28T13:59:02.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ffcc00640 1 -- 172.21.15.42:0/290777088 learned_addr learned my addr 172.21.15.42:0/290777088 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:02.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0fefe00640 1 --2- 172.21.15.42:0/290777088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff80ce3b0 0x7f0ff80ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3313832899 0 0) 0x7f0ff8160230 con 0x7f0ff8071de0 2024-05-28T13:59:02.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fdc003530 con 0x7f0ff8071de0 2024-05-28T13:59:02.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 31866238 0 0) 0x7f0ff8160510 con 0x7f0ff813fd10 2024-05-28T13:59:02.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ff8160230 con 0x7f0ff813fd10 2024-05-28T13:59:02.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2233116636 0 0) 0x7f0fdc003530 con 0x7f0ff8071de0 2024-05-28T13:59:02.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff80ce3b0 msgr2=0x7f0ff80ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:02.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 --2- 172.21.15.42:0/290777088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff80ce3b0 0x7f0ff80ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 >> v1:172.21.15.63:6789/0 conn(0x7f0ff813fd10 legacy=0x7f0ff81401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff8160ad0 con 0x7f0ff8071de0 2024-05-28T13:59:02.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f0fe8001680 con 0x7f0ff8071de0 2024-05-28T13:59:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 >> v1:172.21.15.42:6789/0 conn(0x7f0ff8071de0 legacy=0x7f0ff80721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 --2- 172.21.15.42:0/290777088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fdc003b80 0x7f0fdc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 --2- 172.21.15.42:0/290777088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc004810 0x7f0fdc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 --2- 172.21.15.42:0/290777088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff80ce3b0 0x7f0fdc005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff81607f0 con 0x7f0fdc003b80 2024-05-28T13:59:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc00ffa0 con 0x7f0fdc004810 2024-05-28T13:59:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0fdc0101a0 con 0x7f0ff80ce3b0 2024-05-28T13:59:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ffcc00640 1 --2- 172.21.15.42:0/290777088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fdc003b80 0x7f0fdc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0fefe00640 1 --2- 172.21.15.42:0/290777088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc004810 0x7f0fdc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ffd600640 1 --2- 172.21.15.42:0/290777088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff80ce3b0 0x7f0fdc005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ffcc00640 1 -- 172.21.15.42:0/290777088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff80ce3b0 msgr2=0x7f0fdc005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ffcc00640 1 --2- 172.21.15.42:0/290777088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff80ce3b0 0x7f0fdc005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ffcc00640 1 -- 172.21.15.42:0/290777088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc004810 msgr2=0x7f0fdc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ffcc00640 1 --2- 172.21.15.42:0/290777088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc004810 0x7f0fdc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ffcc00640 1 -- 172.21.15.42:0/290777088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fdc003530 con 0x7f0fdc003b80 2024-05-28T13:59:02.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.517+0000 7f0ffd600640 1 --2- 172.21.15.42:0/290777088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff80ce3b0 0x7f0fdc005280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:02.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.521+0000 7f0ffcc00640 1 --2- 172.21.15.42:0/290777088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fdc003b80 0x7f0fdc003f80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800c190 tx=0x7f0fe801cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bb0eeb6ab89162 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:02.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.521+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8007cd0 con 0x7f0fdc003b80 2024-05-28T13:59:02.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.521+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe80a9020 con 0x7f0fdc003b80 2024-05-28T13:59:02.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.521+0000 7f0ff7e00640 1 -- 172.21.15.42:0/290777088 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8005ae0 con 0x7f0fdc003b80 2024-05-28T13:59:02.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.521+0000 7f0fff600640 1 -- 172.21.15.42:0/290777088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fdc003b80 msgr2=0x7f0fdc003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.521+0000 7f0fff600640 1 --2- 172.21.15.42:0/290777088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fdc003b80 0x7f0fdc003f80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800c190 tx=0x7f0fe801cbd0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.521+0000 7f0ffd600640 1 -- 172.21.15.42:0/290777088 reap_dead start 2024-05-28T13:59:02.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.521+0000 7f0fff600640 1 -- 172.21.15.42:0/290777088 shutdown_connections 2024-05-28T13:59:02.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.521+0000 7f0fff600640 1 -- 172.21.15.42:0/290777088 >> 172.21.15.42:0/290777088 conn(0x7f0ff80b21f0 msgr2=0x7f0ff80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:02.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.521+0000 7f0fff600640 1 -- 172.21.15.42:0/290777088 shutdown_connections 2024-05-28T13:59:02.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.521+0000 7f0fff600640 1 -- 172.21.15.42:0/290777088 wait complete. 2024-05-28T13:59:02.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.521+0000 7f0fff600640 1 Processor -- start 2024-05-28T13:59:02.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.521+0000 7f0fff600640 1 -- start start 2024-05-28T13:59:02.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0fff600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc003b80 0x7f0ff8157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0fff600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff8162070 0x7f0ff8162910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0fff600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ff806ef80 0x7f0ff81533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0fff600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff80b3980 con 0x7f0ff806ef80 2024-05-28T13:59:02.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0fff600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff80b3800 con 0x7f0fdc003b80 2024-05-28T13:59:02.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0fff600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff80b3b00 con 0x7f0ff8162070 2024-05-28T13:59:02.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0ffd600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ff806ef80 0x7f0ff81533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0ffd600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ff806ef80 0x7f0ff81533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52760/0 (socket says 172.21.15.42:52760) 2024-05-28T13:59:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0ffd600640 1 -- 172.21.15.42:0/1619122981 learned_addr learned my addr 172.21.15.42:0/1619122981 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0ffcc00640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc003b80 0x7f0ff8157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0fefe00640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff8162070 0x7f0ff8162910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0ffd600640 1 -- 172.21.15.42:0/1619122981 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff8162070 msgr2=0x7f0ff8162910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0ffd600640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff8162070 0x7f0ff8162910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0ffd600640 1 -- 172.21.15.42:0/1619122981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc003b80 msgr2=0x7f0ff8157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0ffd600640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc003b80 0x7f0ff8157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0ffd600640 1 -- 172.21.15.42:0/1619122981 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff81612f0 con 0x7f0ff806ef80 2024-05-28T13:59:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0fefe00640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff8162070 0x7f0ff8162910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0ffd600640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ff806ef80 0x7f0ff81533d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0001340 tx=0x7f0ff000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0ff5600640 1 -- 172.21.15.42:0/1619122981 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0006b60 con 0x7f0ff806ef80 2024-05-28T13:59:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0ff5600640 1 -- 172.21.15.42:0/1619122981 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff0006d00 con 0x7f0ff806ef80 2024-05-28T13:59:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0ff5600640 1 -- 172.21.15.42:0/1619122981 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0003bc0 con 0x7f0ff806ef80 2024-05-28T13:59:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0fff600640 1 -- 172.21.15.42:0/1619122981 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff8161590 con 0x7f0ff806ef80 2024-05-28T13:59:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0fff600640 1 -- 172.21.15.42:0/1619122981 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff8162e50 con 0x7f0ff806ef80 2024-05-28T13:59:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0fff600640 1 -- 172.21.15.42:0/1619122981 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff80738b0 con 0x7f0ff806ef80 2024-05-28T13:59:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.525+0000 7f0ff5600640 1 -- 172.21.15.42:0/1619122981 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0ff0005070 con 0x7f0ff806ef80 2024-05-28T13:59:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.529+0000 7f0ff5600640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0fd4068380 0x7f0fd406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.529+0000 7f0ff5600640 1 -- 172.21.15.42:0/1619122981 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0ff00864e0 con 0x7f0ff806ef80 2024-05-28T13:59:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.529+0000 7f0ffcc00640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0fd4068380 0x7f0fd406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.529+0000 7f0ffcc00640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0fd4068380 0x7f0fd406a840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80aa000 tx=0x7f0fe80b0040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:02.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.533+0000 7f0ff5600640 1 -- 172.21.15.42:0/1619122981 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ff0056a90 con 0x7f0ff806ef80 2024-05-28T13:59:02.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.697+0000 7f0fff600640 1 -- 172.21.15.42:0/1619122981 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0ff8072ed0 con 0x7f0ff806ef80 2024-05-28T13:59:02.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.697+0000 7f0ff5600640 1 -- 172.21.15.42:0/1619122981 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f0ff0056390 con 0x7f0ff806ef80 2024-05-28T13:59:02.683 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:59:02.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.701+0000 7f0fff600640 1 -- 172.21.15.42:0/1619122981 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0fd4068380 msgr2=0x7f0fd406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.701+0000 7f0fff600640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0fd4068380 0x7f0fd406a840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80aa000 tx=0x7f0fe80b0040 comp rx=0 tx=0).stop 2024-05-28T13:59:02.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.701+0000 7f0fff600640 1 -- 172.21.15.42:0/1619122981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ff806ef80 msgr2=0x7f0ff81533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.701+0000 7f0fff600640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ff806ef80 0x7f0ff81533d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0001340 tx=0x7f0ff000eb90 comp rx=0 tx=0).stop 2024-05-28T13:59:02.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.701+0000 7f0fff600640 1 -- 172.21.15.42:0/1619122981 shutdown_connections 2024-05-28T13:59:02.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.701+0000 7f0fff600640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0fd4068380 0x7f0fd406a840 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.701+0000 7f0fff600640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ff806ef80 0x7f0ff81533d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.701+0000 7f0fff600640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ff8162070 0x7f0ff8162910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.701+0000 7f0fff600640 1 --2- 172.21.15.42:0/1619122981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fdc003b80 0x7f0ff8157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.701+0000 7f0fff600640 1 -- 172.21.15.42:0/1619122981 >> 172.21.15.42:0/1619122981 conn(0x7f0ff80b21f0 msgr2=0x7f0ff815a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:02.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.701+0000 7f0fff600640 1 -- 172.21.15.42:0/1619122981 shutdown_connections 2024-05-28T13:59:02.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.701+0000 7f0fff600640 1 -- 172.21.15.42:0/1619122981 wait complete. 2024-05-28T13:59:02.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T13:59:02.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.861+0000 7ff28b600640 1 Processor -- start 2024-05-28T13:59:02.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.861+0000 7ff28b600640 1 -- start start 2024-05-28T13:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.861+0000 7ff28b600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff28415bf30 0x7ff28415c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.861+0000 7ff28b600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff2840cf230 0x7ff284072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.861+0000 7ff28b600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2840cfc10 con 0x7ff284072ba0 2024-05-28T13:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.861+0000 7ff28b600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff2840721d0 con 0x7ff2840cf230 2024-05-28T13:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.861+0000 7ff28b600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff2840cfef0 con 0x7ff28415bf30 2024-05-28T13:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.865+0000 7ff283e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff2840cf230 0x7ff284072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.865+0000 7ff288c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff28415bf30 0x7ff28415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.865+0000 7ff288c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff28415bf30 0x7ff28415c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:50074/0 (socket says 172.21.15.42:50074) 2024-05-28T13:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.865+0000 7ff288c00640 1 -- 172.21.15.42:0/3161081911 learned_addr learned my addr 172.21.15.42:0/3161081911 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.865+0000 7ff288c00640 1 -- 172.21.15.42:0/3161081911 >> v2:172.21.15.63:3300/0 conn(0x7ff2840cf230 msgr2=0x7ff284072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.865+0000 7ff288c00640 1 --2- 172.21.15.42:0/3161081911 >> v2:172.21.15.63:3300/0 conn(0x7ff2840cf230 0x7ff284072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.865+0000 7ff288c00640 1 -- 172.21.15.42:0/3161081911 >> v1:172.21.15.63:6789/0 conn(0x7ff284072ba0 legacy=0x7ff28406ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.865+0000 7ff288c00640 1 -- 172.21.15.42:0/3161081911 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2840d01d0 con 0x7ff28415bf30 2024-05-28T13:59:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.865+0000 7ff283e00640 1 --2- 172.21.15.42:0/3161081911 >> v2:172.21.15.63:3300/0 conn(0x7ff2840cf230 0x7ff284072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:59:02.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.865+0000 7ff288c00640 1 --2- 172.21.15.42:0/3161081911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff28415bf30 0x7ff28415c310 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff278009870 tx=0x7ff27801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d89c9add40878b35 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:02.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff283400640 1 -- 172.21.15.42:0/3161081911 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780ab650 con 0x7ff28415bf30 2024-05-28T13:59:02.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff283400640 1 -- 172.21.15.42:0/3161081911 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2780b0070 con 0x7ff28415bf30 2024-05-28T13:59:02.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff283400640 1 -- 172.21.15.42:0/3161081911 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780aba10 con 0x7ff28415bf30 2024-05-28T13:59:02.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 -- 172.21.15.42:0/3161081911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff28415bf30 msgr2=0x7ff28415c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 --2- 172.21.15.42:0/3161081911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff28415bf30 0x7ff28415c310 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff278009870 tx=0x7ff27801c530 comp rx=0 tx=0).stop 2024-05-28T13:59:02.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 -- 172.21.15.42:0/3161081911 shutdown_connections 2024-05-28T13:59:02.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 --2- 172.21.15.42:0/3161081911 >> v2:172.21.15.63:3300/0 conn(0x7ff2840cf230 0x7ff284072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 --2- 172.21.15.42:0/3161081911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff28415bf30 0x7ff28415c310 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 -- 172.21.15.42:0/3161081911 >> 172.21.15.42:0/3161081911 conn(0x7ff28406b9f0 msgr2=0x7ff28406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:02.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 -- 172.21.15.42:0/3161081911 shutdown_connections 2024-05-28T13:59:02.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 -- 172.21.15.42:0/3161081911 wait complete. 2024-05-28T13:59:02.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 Processor -- start 2024-05-28T13:59:02.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 -- start start 2024-05-28T13:59:02.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff284072ba0 0x7ff28414b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2840cf230 0x7ff284151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff28415bf30 0x7ff2840ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff28406f4e0 con 0x7ff284072ba0 2024-05-28T13:59:02.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff28406f360 con 0x7ff2840cf230 2024-05-28T13:59:02.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff28b600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff2840cf610 con 0x7ff28415bf30 2024-05-28T13:59:02.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.869+0000 7ff288c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff284072ba0 0x7ff28414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff288c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff284072ba0 0x7ff28414b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52774/0 (socket says 172.21.15.42:52774) 2024-05-28T13:59:02.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff288c00640 1 -- 172.21.15.42:0/3868055650 learned_addr learned my addr 172.21.15.42:0/3868055650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:02.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff283e00640 1 --2- 172.21.15.42:0/3868055650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2840cf230 0x7ff284151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff289600640 1 --2- 172.21.15.42:0/3868055650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff28415bf30 0x7ff2840ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff288c00640 1 -- 172.21.15.42:0/3868055650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff28415bf30 msgr2=0x7ff2840ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff288c00640 1 --2- 172.21.15.42:0/3868055650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff28415bf30 0x7ff2840ac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff288c00640 1 -- 172.21.15.42:0/3868055650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2840cf230 msgr2=0x7ff284151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:02.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff288c00640 1 --2- 172.21.15.42:0/3868055650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2840cf230 0x7ff284151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:02.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff288c00640 1 -- 172.21.15.42:0/3868055650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff284151d50 con 0x7ff284072ba0 2024-05-28T13:59:02.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff288c00640 1 --2- 172.21.15.42:0/3868055650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff284072ba0 0x7ff28414b530 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff2780ad040 tx=0x7ff27801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:02.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff280c00640 1 -- 172.21.15.42:0/3868055650 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780be5f0 con 0x7ff284072ba0 2024-05-28T13:59:02.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff280c00640 1 -- 172.21.15.42:0/3868055650 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2780be7b0 con 0x7ff284072ba0 2024-05-28T13:59:02.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff28b600640 1 -- 172.21.15.42:0/3868055650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2840a9910 con 0x7ff284072ba0 2024-05-28T13:59:02.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff280c00640 1 -- 172.21.15.42:0/3868055650 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780beba0 con 0x7ff284072ba0 2024-05-28T13:59:02.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff28b600640 1 -- 172.21.15.42:0/3868055650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2840a9ed0 con 0x7ff284072ba0 2024-05-28T13:59:02.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff28b600640 1 -- 172.21.15.42:0/3868055650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff284073890 con 0x7ff284072ba0 2024-05-28T13:59:02.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff280c00640 1 -- 172.21.15.42:0/3868055650 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff2780b43f0 con 0x7ff284072ba0 2024-05-28T13:59:02.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.873+0000 7ff280c00640 1 --2- 172.21.15.42:0/3868055650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff264068380 0x7ff26406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:02.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.877+0000 7ff280c00640 1 -- 172.21.15.42:0/3868055650 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff278121520 con 0x7ff284072ba0 2024-05-28T13:59:02.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.877+0000 7ff283e00640 1 --2- 172.21.15.42:0/3868055650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff264068380 0x7ff26406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:02.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.877+0000 7ff283e00640 1 --2- 172.21.15.42:0/3868055650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff264068380 0x7ff26406a840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff270009580 tx=0x7ff270009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:02.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:02.881+0000 7ff280c00640 1 -- 172.21.15.42:0/3868055650 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff278016150 con 0x7ff284072ba0 2024-05-28T13:59:03.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.061+0000 7ff28b600640 1 -- 172.21.15.42:0/3868055650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff28404fdd0 con 0x7ff284072ba0 2024-05-28T13:59:03.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.061+0000 7ff280c00640 1 -- 172.21.15.42:0/3868055650 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7ff2780f1400 con 0x7ff284072ba0 2024-05-28T13:59:03.048 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T13:59:03.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.065+0000 7ff28b600640 1 -- 172.21.15.42:0/3868055650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff264068380 msgr2=0x7ff26406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.065+0000 7ff28b600640 1 --2- 172.21.15.42:0/3868055650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff264068380 0x7ff26406a840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff270009580 tx=0x7ff270009190 comp rx=0 tx=0).stop 2024-05-28T13:59:03.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.065+0000 7ff28b600640 1 -- 172.21.15.42:0/3868055650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff284072ba0 msgr2=0x7ff28414b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.065+0000 7ff28b600640 1 --2- 172.21.15.42:0/3868055650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff284072ba0 0x7ff28414b530 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff2780ad040 tx=0x7ff27801ad70 comp rx=0 tx=0).stop 2024-05-28T13:59:03.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.065+0000 7ff28b600640 1 -- 172.21.15.42:0/3868055650 shutdown_connections 2024-05-28T13:59:03.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.065+0000 7ff28b600640 1 --2- 172.21.15.42:0/3868055650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff264068380 0x7ff26406a840 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.065+0000 7ff28b600640 1 --2- 172.21.15.42:0/3868055650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff28415bf30 0x7ff2840ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.065+0000 7ff28b600640 1 --2- 172.21.15.42:0/3868055650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2840cf230 0x7ff284151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.065+0000 7ff28b600640 1 --2- 172.21.15.42:0/3868055650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff284072ba0 0x7ff28414b530 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.065+0000 7ff28b600640 1 -- 172.21.15.42:0/3868055650 >> 172.21.15.42:0/3868055650 conn(0x7ff28406b9f0 msgr2=0x7ff28415a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:03.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.065+0000 7ff28b600640 1 -- 172.21.15.42:0/3868055650 shutdown_connections 2024-05-28T13:59:03.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.065+0000 7ff28b600640 1 -- 172.21.15.42:0/3868055650 wait complete. 2024-05-28T13:59:03.070 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:59:03.070 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:59:03.079 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:59:03.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.233+0000 7fbf16600640 1 Processor -- start 2024-05-28T13:59:03.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.233+0000 7fbf16600640 1 -- start start 2024-05-28T13:59:03.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf16600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbf1006f660 0x7fbf10073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf16600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf1015f280 con 0x7fbf100d0e00 2024-05-28T13:59:03.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf16600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf1015f560 con 0x7fbf1015ba40 2024-05-28T13:59:03.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf16600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbf1015f030 con 0x7fbf1006f660 2024-05-28T13:59:03.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fbf100d0e00 0x7fbf1006dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34816/0 (socket says 172.21.15.42:34816) 2024-05-28T13:59:03.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0fe00640 1 -- 172.21.15.42:0/1155382330 learned_addr learned my addr 172.21.15.42:0/1155382330 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:03.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 -- 172.21.15.42:0/1155382330 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3947541837 0 0) 0x7fbf1015f280 con 0x7fbf100d0e00 2024-05-28T13:59:03.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0f400640 1 --2- 172.21.15.42:0/1155382330 >> v2:172.21.15.63:3300/0 conn(0x7fbf1006f660 0x7fbf10073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 -- 172.21.15.42:0/1155382330 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbef4003530 con 0x7fbf100d0e00 2024-05-28T13:59:03.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 -- 172.21.15.42:0/1155382330 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4044067301 0 0) 0x7fbef4003530 con 0x7fbf100d0e00 2024-05-28T13:59:03.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 -- 172.21.15.42:0/1155382330 >> v2:172.21.15.63:3300/0 conn(0x7fbf1006f660 msgr2=0x7fbf10073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:03.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 --2- 172.21.15.42:0/1155382330 >> v2:172.21.15.63:3300/0 conn(0x7fbf1006f660 0x7fbf10073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 -- 172.21.15.42:0/1155382330 >> v1:172.21.15.63:6789/0 conn(0x7fbf1015ba40 legacy=0x7fbf100d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 -- 172.21.15.42:0/1155382330 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1015f840 con 0x7fbf100d0e00 2024-05-28T13:59:03.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 -- 172.21.15.42:0/1155382330 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fbf08001a00 con 0x7fbf100d0e00 2024-05-28T13:59:03.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 -- 172.21.15.42:0/1155382330 >> v1:172.21.15.42:6789/0 conn(0x7fbf100d0e00 legacy=0x7fbf1006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 --2- 172.21.15.42:0/1155382330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbef40039c0 0x7fbef4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 --2- 172.21.15.42:0/1155382330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbef4004810 0x7fbef4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 --2- 172.21.15.42:0/1155382330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef40052f0 0x7fbef400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 -- 172.21.15.42:0/1155382330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1015f030 con 0x7fbef40052f0 2024-05-28T13:59:03.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 -- 172.21.15.42:0/1155382330 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbef400e3d0 con 0x7fbef40039c0 2024-05-28T13:59:03.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0ea00640 1 -- 172.21.15.42:0/1155382330 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbef400e5d0 con 0x7fbef4004810 2024-05-28T13:59:03.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0fe00640 1 --2- 172.21.15.42:0/1155382330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef40052f0 0x7fbef400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0f400640 1 --2- 172.21.15.42:0/1155382330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbef40039c0 0x7fbef4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf07e00640 1 --2- 172.21.15.42:0/1155382330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbef4004810 0x7fbef4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0fe00640 1 -- 172.21.15.42:0/1155382330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbef4004810 msgr2=0x7fbef4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:03.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0fe00640 1 --2- 172.21.15.42:0/1155382330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbef4004810 0x7fbef4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0fe00640 1 -- 172.21.15.42:0/1155382330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbef40039c0 msgr2=0x7fbef4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:03.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0fe00640 1 --2- 172.21.15.42:0/1155382330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbef40039c0 0x7fbef4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.237+0000 7fbf0fe00640 1 -- 172.21.15.42:0/1155382330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef4003530 con 0x7fbef40052f0 2024-05-28T13:59:03.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.241+0000 7fbf0fe00640 1 -- 172.21.15.42:0/1155382330 reap_dead start 2024-05-28T13:59:03.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.241+0000 7fbf0fe00640 1 --2- 172.21.15.42:0/1155382330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef40052f0 0x7fbef400b9d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800a810 tx=0x7fbf0801d310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf76a29badc03c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:03.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.241+0000 7fbf0ea00640 1 -- 172.21.15.42:0/1155382330 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0800a460 con 0x7fbef40052f0 2024-05-28T13:59:03.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.241+0000 7fbf0ea00640 1 -- 172.21.15.42:0/1155382330 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf080ac640 con 0x7fbef40052f0 2024-05-28T13:59:03.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.241+0000 7fbf0ea00640 1 -- 172.21.15.42:0/1155382330 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf080aca30 con 0x7fbef40052f0 2024-05-28T13:59:03.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.241+0000 7fbf16600640 1 -- 172.21.15.42:0/1155382330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef40052f0 msgr2=0x7fbef400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.241+0000 7fbf16600640 1 --2- 172.21.15.42:0/1155382330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef40052f0 0x7fbef400b9d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800a810 tx=0x7fbf0801d310 comp rx=0 tx=0).stop 2024-05-28T13:59:03.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.241+0000 7fbf16600640 1 -- 172.21.15.42:0/1155382330 shutdown_connections 2024-05-28T13:59:03.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.241+0000 7fbf16600640 1 --2- 172.21.15.42:0/1155382330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef40052f0 0x7fbef400b9d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.241+0000 7fbf16600640 1 -- 172.21.15.42:0/1155382330 >> 172.21.15.42:0/1155382330 conn(0x7fbf1006bdf0 msgr2=0x7fbf1006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:03.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.241+0000 7fbf16600640 1 -- 172.21.15.42:0/1155382330 shutdown_connections 2024-05-28T13:59:03.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.241+0000 7fbf16600640 1 -- 172.21.15.42:0/1155382330 wait complete. 2024-05-28T13:59:03.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf16600640 1 Processor -- start 2024-05-28T13:59:03.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf16600640 1 -- start start 2024-05-28T13:59:03.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf16600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef40052f0 0x7fbf1014dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf16600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf100b4770 0x7fbf10147980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf16600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf100ceed0 0x7fbf100cf360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf16600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbf10073b00 con 0x7fbef40052f0 2024-05-28T13:59:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf16600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf10073980 con 0x7fbf100ceed0 2024-05-28T13:59:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf16600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf10073c80 con 0x7fbf100b4770 2024-05-28T13:59:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf0f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef40052f0 0x7fbf1014dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf0f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef40052f0 0x7fbf1014dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52796/0 (socket says 172.21.15.42:52796) 2024-05-28T13:59:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf0f400640 1 -- 172.21.15.42:0/2543621951 learned_addr learned my addr 172.21.15.42:0/2543621951 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf0fe00640 1 --2- 172.21.15.42:0/2543621951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf100ceed0 0x7fbf100cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf07e00640 1 --2- 172.21.15.42:0/2543621951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf100b4770 0x7fbf10147980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf0f400640 1 -- 172.21.15.42:0/2543621951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf100b4770 msgr2=0x7fbf10147980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf0f400640 1 --2- 172.21.15.42:0/2543621951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf100b4770 0x7fbf10147980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf0f400640 1 -- 172.21.15.42:0/2543621951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf100ceed0 msgr2=0x7fbf100cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf0f400640 1 --2- 172.21.15.42:0/2543621951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf100ceed0 0x7fbf100cf360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf0f400640 1 -- 172.21.15.42:0/2543621951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1014e350 con 0x7fbef40052f0 2024-05-28T13:59:03.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf0f400640 1 --2- 172.21.15.42:0/2543621951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef40052f0 0x7fbf1014dce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbf00011960 tx=0x7fbf00011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:03.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf07400640 1 -- 172.21.15.42:0/2543621951 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0000faf0 con 0x7fbef40052f0 2024-05-28T13:59:03.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf07400640 1 -- 172.21.15.42:0/2543621951 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0000fc90 con 0x7fbef40052f0 2024-05-28T13:59:03.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf07400640 1 -- 172.21.15.42:0/2543621951 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00014440 con 0x7fbef40052f0 2024-05-28T13:59:03.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf16600640 1 -- 172.21.15.42:0/2543621951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1014e630 con 0x7fbef40052f0 2024-05-28T13:59:03.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf16600640 1 -- 172.21.15.42:0/2543621951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1014ebf0 con 0x7fbef40052f0 2024-05-28T13:59:03.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.245+0000 7fbf16600640 1 -- 172.21.15.42:0/2543621951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf10145c80 con 0x7fbef40052f0 2024-05-28T13:59:03.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.249+0000 7fbf07400640 1 -- 172.21.15.42:0/2543621951 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbf00012040 con 0x7fbef40052f0 2024-05-28T13:59:03.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.249+0000 7fbf07400640 1 --2- 172.21.15.42:0/2543621951 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbeec068380 0x7fbeec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.249+0000 7fbf07400640 1 -- 172.21.15.42:0/2543621951 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fbf00087170 con 0x7fbef40052f0 2024-05-28T13:59:03.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.249+0000 7fbf07e00640 1 --2- 172.21.15.42:0/2543621951 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbeec068380 0x7fbeec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.249+0000 7fbf07e00640 1 --2- 172.21.15.42:0/2543621951 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbeec068380 0x7fbeec06a840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbef8007fd0 tx=0x7fbef8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:03.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.253+0000 7fbf07400640 1 -- 172.21.15.42:0/2543621951 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbf00019030 con 0x7fbef40052f0 2024-05-28T13:59:03.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.417+0000 7fbf16600640 1 -- 172.21.15.42:0/2543621951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf10068a90 con 0x7fbef40052f0 2024-05-28T13:59:03.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.421+0000 7fbf07400640 1 -- 172.21.15.42:0/2543621951 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16737 (secure 0 0 0) 0x7fbf000576a0 con 0x7fbef40052f0 2024-05-28T13:59:03.405 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:03.405 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":22,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:59:02.987231+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:59:03.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.421+0000 7fbf16600640 1 -- 172.21.15.42:0/2543621951 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbeec068380 msgr2=0x7fbeec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.421+0000 7fbf16600640 1 --2- 172.21.15.42:0/2543621951 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbeec068380 0x7fbeec06a840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbef8007fd0 tx=0x7fbef8009040 comp rx=0 tx=0).stop 2024-05-28T13:59:03.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.421+0000 7fbf16600640 1 -- 172.21.15.42:0/2543621951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef40052f0 msgr2=0x7fbf1014dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.421+0000 7fbf16600640 1 --2- 172.21.15.42:0/2543621951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef40052f0 0x7fbf1014dce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbf00011960 tx=0x7fbf00011e20 comp rx=0 tx=0).stop 2024-05-28T13:59:03.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.421+0000 7fbf16600640 1 -- 172.21.15.42:0/2543621951 shutdown_connections 2024-05-28T13:59:03.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.421+0000 7fbf16600640 1 --2- 172.21.15.42:0/2543621951 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbeec068380 0x7fbeec06a840 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.421+0000 7fbf16600640 1 --2- 172.21.15.42:0/2543621951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf100ceed0 0x7fbf100cf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.421+0000 7fbf16600640 1 --2- 172.21.15.42:0/2543621951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf100b4770 0x7fbf10147980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.421+0000 7fbf16600640 1 --2- 172.21.15.42:0/2543621951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbef40052f0 0x7fbf1014dce0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.421+0000 7fbf16600640 1 -- 172.21.15.42:0/2543621951 >> 172.21.15.42:0/2543621951 conn(0x7fbf1006bdf0 msgr2=0x7fbf100b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:03.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.421+0000 7fbf16600640 1 -- 172.21.15.42:0/2543621951 shutdown_connections 2024-05-28T13:59:03.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.421+0000 7fbf16600640 1 -- 172.21.15.42:0/2543621951 wait complete. 2024-05-28T13:59:03.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:59:03.427 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:59:03.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.585+0000 7f59c4800640 1 Processor -- start 2024-05-28T13:59:03.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.585+0000 7f59c4800640 1 -- start start 2024-05-28T13:59:03.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.589+0000 7f59c4800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f59bc06ee20 0x7f59bc06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.589+0000 7f59c4800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59bc0afe40 con 0x7f59bc15b9b0 2024-05-28T13:59:03.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.589+0000 7f59c4800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59bc15c600 con 0x7f59bc144f80 2024-05-28T13:59:03.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.589+0000 7f59c4800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f59bc06f7c0 con 0x7f59bc06ee20 2024-05-28T13:59:03.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.589+0000 7f59c1400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f59bc06ee20 0x7f59bc06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.589+0000 7f59c2800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f59bc15b9b0 0x7f59bc15be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34828/0 (socket says 172.21.15.42:34828) 2024-05-28T13:59:03.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.589+0000 7f59c1400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f59bc06ee20 0x7f59bc06f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52806/0 (socket says 172.21.15.42:52806) 2024-05-28T13:59:03.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.589+0000 7f59c2800640 1 -- 172.21.15.42:0/3204930051 learned_addr learned my addr 172.21.15.42:0/3204930051 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:03.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.589+0000 7f59c1400640 1 -- 172.21.15.42:0/3204930051 >> v1:172.21.15.63:6789/0 conn(0x7f59bc144f80 legacy=0x7f59bc072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.589+0000 7f59c1400640 1 -- 172.21.15.42:0/3204930051 >> v1:172.21.15.42:6789/0 conn(0x7f59bc15b9b0 legacy=0x7f59bc15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.589+0000 7f59c1400640 1 -- 172.21.15.42:0/3204930051 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc15c8e0 con 0x7f59bc06ee20 2024-05-28T13:59:03.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.589+0000 7f59c0a00640 1 -- 172.21.15.42:0/3204930051 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2129721176 0 0) 0x7f59bc0afe40 con 0x7f59bc15b9b0 2024-05-28T13:59:03.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c1400640 1 --2- 172.21.15.42:0/3204930051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59bc06ee20 0x7f59bc06f280 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0097c0 tx=0x7f59ac01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fb64b7a735a17e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:03.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c0a00640 1 -- 172.21.15.42:0/3204930051 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac0b0070 con 0x7f59bc06ee20 2024-05-28T13:59:03.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c0a00640 1 -- 172.21.15.42:0/3204930051 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ac0ab600 con 0x7f59bc06ee20 2024-05-28T13:59:03.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c0a00640 1 -- 172.21.15.42:0/3204930051 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac0ab9d0 con 0x7f59bc06ee20 2024-05-28T13:59:03.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 -- 172.21.15.42:0/3204930051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59bc06ee20 msgr2=0x7f59bc06f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 --2- 172.21.15.42:0/3204930051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59bc06ee20 0x7f59bc06f280 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0097c0 tx=0x7f59ac01c4a0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 -- 172.21.15.42:0/3204930051 shutdown_connections 2024-05-28T13:59:03.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 --2- 172.21.15.42:0/3204930051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59bc06ee20 0x7f59bc06f280 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 -- 172.21.15.42:0/3204930051 >> 172.21.15.42:0/3204930051 conn(0x7f59bc0b21f0 msgr2=0x7f59bc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:03.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 -- 172.21.15.42:0/3204930051 shutdown_connections 2024-05-28T13:59:03.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 -- 172.21.15.42:0/3204930051 wait complete. 2024-05-28T13:59:03.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 Processor -- start 2024-05-28T13:59:03.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 -- start start 2024-05-28T13:59:03.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59bc06ee20 0x7f59bc13bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59bc144f80 0x7f59bc13a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59bc15b9b0 0x7f59bc155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc0724d0 con 0x7f59bc06ee20 2024-05-28T13:59:03.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc0b4260 con 0x7f59bc144f80 2024-05-28T13:59:03.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.593+0000 7f59c4800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f59bc169140 con 0x7f59bc15b9b0 2024-05-28T13:59:03.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59c1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59bc06ee20 0x7f59bc13bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59c1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59bc06ee20 0x7f59bc13bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52818/0 (socket says 172.21.15.42:52818) 2024-05-28T13:59:03.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59c1e00640 1 -- 172.21.15.42:0/3240900763 learned_addr learned my addr 172.21.15.42:0/3240900763 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:03.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59c1400640 1 --2- 172.21.15.42:0/3240900763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59bc144f80 0x7f59bc13a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59c2800640 1 --2- 172.21.15.42:0/3240900763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59bc15b9b0 0x7f59bc155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59c1e00640 1 -- 172.21.15.42:0/3240900763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59bc15b9b0 msgr2=0x7f59bc155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59c1e00640 1 --2- 172.21.15.42:0/3240900763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59bc15b9b0 0x7f59bc155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59c1e00640 1 -- 172.21.15.42:0/3240900763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59bc144f80 msgr2=0x7f59bc13a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59c1e00640 1 --2- 172.21.15.42:0/3240900763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59bc144f80 0x7f59bc13a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59c1e00640 1 -- 172.21.15.42:0/3240900763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc154b40 con 0x7f59bc06ee20 2024-05-28T13:59:03.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59c1e00640 1 --2- 172.21.15.42:0/3240900763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59bc06ee20 0x7f59bc13bfa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f59b000c530 tx=0x7f59b000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:03.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59b6000640 1 -- 172.21.15.42:0/3240900763 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0016020 con 0x7f59bc06ee20 2024-05-28T13:59:03.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59b6000640 1 -- 172.21.15.42:0/3240900763 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b0005220 con 0x7f59bc06ee20 2024-05-28T13:59:03.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59b6000640 1 -- 172.21.15.42:0/3240900763 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0007030 con 0x7f59bc06ee20 2024-05-28T13:59:03.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59c4800640 1 -- 172.21.15.42:0/3240900763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59bc154e20 con 0x7f59bc06ee20 2024-05-28T13:59:03.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59c4800640 1 -- 172.21.15.42:0/3240900763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59bc1553e0 con 0x7f59bc06ee20 2024-05-28T13:59:03.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59c4800640 1 -- 172.21.15.42:0/3240900763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59bc0738b0 con 0x7f59bc06ee20 2024-05-28T13:59:03.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.597+0000 7f59b6000640 1 -- 172.21.15.42:0/3240900763 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f59b00071d0 con 0x7f59bc06ee20 2024-05-28T13:59:03.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.601+0000 7f59b6000640 1 --2- 172.21.15.42:0/3240900763 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59a0068380 0x7f59a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.601+0000 7f59b6000640 1 -- 172.21.15.42:0/3240900763 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f59b00863d0 con 0x7f59bc06ee20 2024-05-28T13:59:03.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.601+0000 7f59c1400640 1 --2- 172.21.15.42:0/3240900763 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59a0068380 0x7f59a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.601+0000 7f59c1400640 1 --2- 172.21.15.42:0/3240900763 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59a0068380 0x7f59a006a840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0ad000 tx=0x7f59ac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:03.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.605+0000 7f59b6000640 1 -- 172.21.15.42:0/3240900763 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f59b0020020 con 0x7f59bc06ee20 2024-05-28T13:59:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.773+0000 7f59c4800640 1 -- 172.21.15.42:0/3240900763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f59bc04fdd0 con 0x7f59bc06ee20 2024-05-28T13:59:03.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.773+0000 7f59b6000640 1 -- 172.21.15.42:0/3240900763 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f59b00074c0 con 0x7f59bc06ee20 2024-05-28T13:59:03.758 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:59:03.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.773+0000 7f59c4800640 1 -- 172.21.15.42:0/3240900763 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59a0068380 msgr2=0x7f59a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.773+0000 7f59c4800640 1 --2- 172.21.15.42:0/3240900763 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59a0068380 0x7f59a006a840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0ad000 tx=0x7f59ac008040 comp rx=0 tx=0).stop 2024-05-28T13:59:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.773+0000 7f59c4800640 1 -- 172.21.15.42:0/3240900763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59bc06ee20 msgr2=0x7f59bc13bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.773+0000 7f59c4800640 1 --2- 172.21.15.42:0/3240900763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59bc06ee20 0x7f59bc13bfa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f59b000c530 tx=0x7f59b000c9f0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.773+0000 7f59c4800640 1 -- 172.21.15.42:0/3240900763 shutdown_connections 2024-05-28T13:59:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.773+0000 7f59c4800640 1 --2- 172.21.15.42:0/3240900763 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59a0068380 0x7f59a006a840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.773+0000 7f59c4800640 1 --2- 172.21.15.42:0/3240900763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59bc15b9b0 0x7f59bc155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.773+0000 7f59c4800640 1 --2- 172.21.15.42:0/3240900763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59bc144f80 0x7f59bc13a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.773+0000 7f59c4800640 1 --2- 172.21.15.42:0/3240900763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59bc06ee20 0x7f59bc13bfa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.773+0000 7f59c4800640 1 -- 172.21.15.42:0/3240900763 >> 172.21.15.42:0/3240900763 conn(0x7f59bc0b21f0 msgr2=0x7f59bc0cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.773+0000 7f59c4800640 1 -- 172.21.15.42:0/3240900763 shutdown_connections 2024-05-28T13:59:03.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.773+0000 7f59c4800640 1 -- 172.21.15.42:0/3240900763 wait complete. 2024-05-28T13:59:03.779 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:59:03.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.937+0000 7f34dd800640 1 Processor -- start 2024-05-28T13:59:03.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.937+0000 7f34dd800640 1 -- start start 2024-05-28T13:59:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.941+0000 7f34dd800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d80b5cf0 0x7f34d80b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.941+0000 7f34dd800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f34d806c320 0x7f34d806c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.941+0000 7f34dd800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34d80d19c0 con 0x7f34d80b3af0 2024-05-28T13:59:03.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.941+0000 7f34dd800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f34d80cd330 con 0x7f34d806c320 2024-05-28T13:59:03.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.941+0000 7f34dd800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f34d80b4410 con 0x7f34d80b5cf0 2024-05-28T13:59:03.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.941+0000 7f34dce00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f34d806c320 0x7f34d806c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.941+0000 7f34dce00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f34d806c320 0x7f34d806c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52834/0 (socket says 172.21.15.42:52834) 2024-05-28T13:59:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.941+0000 7f34dce00640 1 -- 172.21.15.42:0/659108700 learned_addr learned my addr 172.21.15.42:0/659108700 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.941+0000 7f34cf400640 1 --2- 172.21.15.42:0/659108700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d80b5cf0 0x7f34d80b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.941+0000 7f34dce00640 1 -- 172.21.15.42:0/659108700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d80b5cf0 msgr2=0x7f34d80b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.941+0000 7f34dce00640 1 --2- 172.21.15.42:0/659108700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d80b5cf0 0x7f34d80b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.941+0000 7f34dce00640 1 -- 172.21.15.42:0/659108700 >> v1:172.21.15.63:6789/0 conn(0x7f34d80b3af0 legacy=0x7f34d815f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.941+0000 7f34dce00640 1 -- 172.21.15.42:0/659108700 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d80ccd70 con 0x7f34d806c320 2024-05-28T13:59:03.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dce00640 1 --2- 172.21.15.42:0/659108700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d806c320 0x7f34d806c7b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009870 tx=0x7f34d001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422952fb4569b804 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:03.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34d7400640 1 -- 172.21.15.42:0/659108700 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d00b0070 con 0x7f34d806c320 2024-05-28T13:59:03.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34d7400640 1 -- 172.21.15.42:0/659108700 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d00ab600 con 0x7f34d806c320 2024-05-28T13:59:03.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34d7400640 1 -- 172.21.15.42:0/659108700 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d00ab9d0 con 0x7f34d806c320 2024-05-28T13:59:03.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 -- 172.21.15.42:0/659108700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d806c320 msgr2=0x7f34d806c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 --2- 172.21.15.42:0/659108700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d806c320 0x7f34d806c7b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009870 tx=0x7f34d001c470 comp rx=0 tx=0).stop 2024-05-28T13:59:03.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 -- 172.21.15.42:0/659108700 shutdown_connections 2024-05-28T13:59:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 --2- 172.21.15.42:0/659108700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d806c320 0x7f34d806c7b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 --2- 172.21.15.42:0/659108700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d80b5cf0 0x7f34d80b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 -- 172.21.15.42:0/659108700 >> 172.21.15.42:0/659108700 conn(0x7f34d80cd540 msgr2=0x7f34d80cd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 -- 172.21.15.42:0/659108700 shutdown_connections 2024-05-28T13:59:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 -- 172.21.15.42:0/659108700 wait complete. 2024-05-28T13:59:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 Processor -- start 2024-05-28T13:59:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 -- start start 2024-05-28T13:59:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d806c320 0x7f34d81534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34d80b3af0 0x7f34d8070830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d80b5cf0 0x7f34d8152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f34d815ec10 con 0x7f34d806c320 2024-05-28T13:59:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34d80ce960 con 0x7f34d80b3af0 2024-05-28T13:59:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34dd800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f34d816b800 con 0x7f34d80b5cf0 2024-05-28T13:59:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34d7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d806c320 0x7f34d81534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34d7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d806c320 0x7f34d81534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52838/0 (socket says 172.21.15.42:52838) 2024-05-28T13:59:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.945+0000 7f34d7e00640 1 -- 172.21.15.42:0/2295652033 learned_addr learned my addr 172.21.15.42:0/2295652033 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34dce00640 1 --2- 172.21.15.42:0/2295652033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d80b5cf0 0x7f34d8152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34cf400640 1 --2- 172.21.15.42:0/2295652033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34d80b3af0 0x7f34d8070830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34d7e00640 1 -- 172.21.15.42:0/2295652033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d80b5cf0 msgr2=0x7f34d8152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34d7e00640 1 --2- 172.21.15.42:0/2295652033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d80b5cf0 0x7f34d8152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34d7e00640 1 -- 172.21.15.42:0/2295652033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34d80b3af0 msgr2=0x7f34d8070830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34d7e00640 1 --2- 172.21.15.42:0/2295652033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34d80b3af0 0x7f34d8070830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34d7e00640 1 -- 172.21.15.42:0/2295652033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d8153a80 con 0x7f34d806c320 2024-05-28T13:59:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34d7e00640 1 --2- 172.21.15.42:0/2295652033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d806c320 0x7f34d81534a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f34c800c730 tx=0x7f34c800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34d4c00640 1 -- 172.21.15.42:0/2295652033 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8016020 con 0x7f34d806c320 2024-05-28T13:59:03.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34d4c00640 1 -- 172.21.15.42:0/2295652033 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c8006d60 con 0x7f34d806c320 2024-05-28T13:59:03.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34dd800640 1 -- 172.21.15.42:0/2295652033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d8161810 con 0x7f34d806c320 2024-05-28T13:59:03.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34d4c00640 1 -- 172.21.15.42:0/2295652033 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8003bc0 con 0x7f34d806c320 2024-05-28T13:59:03.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34dd800640 1 -- 172.21.15.42:0/2295652033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d8161dd0 con 0x7f34d806c320 2024-05-28T13:59:03.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34dd800640 1 -- 172.21.15.42:0/2295652033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d80ce5b0 con 0x7f34d806c320 2024-05-28T13:59:03.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.949+0000 7f34d4c00640 1 -- 172.21.15.42:0/2295652033 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f34c800a850 con 0x7f34d806c320 2024-05-28T13:59:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.953+0000 7f34d4c00640 1 --2- 172.21.15.42:0/2295652033 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f34b4068380 0x7f34b406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.953+0000 7f34d4c00640 1 -- 172.21.15.42:0/2295652033 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f34c8086680 con 0x7f34d806c320 2024-05-28T13:59:03.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.953+0000 7f34cf400640 1 --2- 172.21.15.42:0/2295652033 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f34b4068380 0x7f34b406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:03.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.953+0000 7f34cf400640 1 --2- 172.21.15.42:0/2295652033 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f34b4068380 0x7f34b406a840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34d80ceb90 tx=0x7f34c0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:03.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:03.957+0000 7f34d4c00640 1 -- 172.21.15.42:0/2295652033 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f34c8020020 con 0x7f34d806c320 2024-05-28T13:59:04.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.125+0000 7f34dd800640 1 -- 172.21.15.42:0/2295652033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f34d815a7b0 con 0x7f34d806c320 2024-05-28T13:59:04.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.125+0000 7f34d4c00640 1 -- 172.21.15.42:0/2295652033 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f34c8056c50 con 0x7f34d806c320 2024-05-28T13:59:04.111 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:59:04.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.125+0000 7f34dd800640 1 -- 172.21.15.42:0/2295652033 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f34b4068380 msgr2=0x7f34b406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:04.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.125+0000 7f34dd800640 1 --2- 172.21.15.42:0/2295652033 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f34b4068380 0x7f34b406a840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34d80ceb90 tx=0x7f34c0008040 comp rx=0 tx=0).stop 2024-05-28T13:59:04.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.125+0000 7f34dd800640 1 -- 172.21.15.42:0/2295652033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d806c320 msgr2=0x7f34d81534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:04.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.125+0000 7f34dd800640 1 --2- 172.21.15.42:0/2295652033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d806c320 0x7f34d81534a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f34c800c730 tx=0x7f34c800cbf0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.125+0000 7f34dd800640 1 -- 172.21.15.42:0/2295652033 shutdown_connections 2024-05-28T13:59:04.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.125+0000 7f34dd800640 1 --2- 172.21.15.42:0/2295652033 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f34b4068380 0x7f34b406a840 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.125+0000 7f34dd800640 1 --2- 172.21.15.42:0/2295652033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d80b5cf0 0x7f34d8152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.125+0000 7f34dd800640 1 --2- 172.21.15.42:0/2295652033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34d80b3af0 0x7f34d8070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.125+0000 7f34dd800640 1 --2- 172.21.15.42:0/2295652033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d806c320 0x7f34d81534a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.125+0000 7f34dd800640 1 -- 172.21.15.42:0/2295652033 >> 172.21.15.42:0/2295652033 conn(0x7f34d80cd540 msgr2=0x7f34d8157f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:04.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.129+0000 7f34dd800640 1 -- 172.21.15.42:0/2295652033 shutdown_connections 2024-05-28T13:59:04.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.129+0000 7f34dd800640 1 -- 172.21.15.42:0/2295652033 wait complete. 2024-05-28T13:59:04.131 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:59:04.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.289+0000 7f9b7b200640 1 Processor -- start 2024-05-28T13:59:04.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.289+0000 7f9b7b200640 1 -- start start 2024-05-28T13:59:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.293+0000 7f9b7b200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9b740cf540 0x7f9b740cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.293+0000 7f9b7b200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9b74145d50 0x7f9b740cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.293+0000 7f9b7b200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b74160170 con 0x7f9b740707e0 2024-05-28T13:59:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.293+0000 7f9b7b200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9b74160450 con 0x7f9b74145d50 2024-05-28T13:59:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.293+0000 7f9b7b200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9b74160730 con 0x7f9b740cf540 2024-05-28T13:59:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.293+0000 7f9b79400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9b74145d50 0x7f9b740cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.293+0000 7f9b79400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9b74145d50 0x7f9b740cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52848/0 (socket says 172.21.15.42:52848) 2024-05-28T13:59:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.293+0000 7f9b79400640 1 -- 172.21.15.42:0/671399656 learned_addr learned my addr 172.21.15.42:0/671399656 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:04.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.293+0000 7f9b79e00640 1 --2- 172.21.15.42:0/671399656 >> v2:172.21.15.63:3300/0 conn(0x7f9b740cf540 0x7f9b740cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:04.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.293+0000 7f9b79400640 1 -- 172.21.15.42:0/671399656 >> v2:172.21.15.63:3300/0 conn(0x7f9b740cf540 msgr2=0x7f9b740cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:04.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.293+0000 7f9b79400640 1 --2- 172.21.15.42:0/671399656 >> v2:172.21.15.63:3300/0 conn(0x7f9b740cf540 0x7f9b740cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.293+0000 7f9b79400640 1 -- 172.21.15.42:0/671399656 >> v1:172.21.15.63:6789/0 conn(0x7f9b740707e0 legacy=0x7f9b74071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:04.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.293+0000 7f9b79400640 1 -- 172.21.15.42:0/671399656 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b740d0ed0 con 0x7f9b74145d50 2024-05-28T13:59:04.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.293+0000 7f9b79400640 1 --2- 172.21.15.42:0/671399656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b74145d50 0x7f9b740cce80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9b740cecd0 tx=0x7f9b6401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1285113b7248c259 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:04.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b78a00640 1 -- 172.21.15.42:0/671399656 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b640b0070 con 0x7f9b74145d50 2024-05-28T13:59:04.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b78a00640 1 -- 172.21.15.42:0/671399656 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b640ab400 con 0x7f9b74145d50 2024-05-28T13:59:04.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b78a00640 1 -- 172.21.15.42:0/671399656 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b640ab7d0 con 0x7f9b74145d50 2024-05-28T13:59:04.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 -- 172.21.15.42:0/671399656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b74145d50 msgr2=0x7f9b740cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:04.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 --2- 172.21.15.42:0/671399656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b74145d50 0x7f9b740cce80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9b740cecd0 tx=0x7f9b6401c450 comp rx=0 tx=0).stop 2024-05-28T13:59:04.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 -- 172.21.15.42:0/671399656 shutdown_connections 2024-05-28T13:59:04.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 --2- 172.21.15.42:0/671399656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b74145d50 0x7f9b740cce80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 --2- 172.21.15.42:0/671399656 >> v2:172.21.15.63:3300/0 conn(0x7f9b740cf540 0x7f9b740cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 -- 172.21.15.42:0/671399656 >> 172.21.15.42:0/671399656 conn(0x7f9b7406fcc0 msgr2=0x7f9b740700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:04.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 -- 172.21.15.42:0/671399656 shutdown_connections 2024-05-28T13:59:04.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 -- 172.21.15.42:0/671399656 wait complete. 2024-05-28T13:59:04.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 Processor -- start 2024-05-28T13:59:04.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 -- start start 2024-05-28T13:59:04.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b740707e0 0x7f9b740c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:04.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b740cf540 0x7f9b7415b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:04.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b74145d50 0x7f9b74155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:04.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9b741694d0 con 0x7f9b740cf540 2024-05-28T13:59:04.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9b74169350 con 0x7f9b74145d50 2024-05-28T13:59:04.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9b74169650 con 0x7f9b740707e0 2024-05-28T13:59:04.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b79400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b740cf540 0x7f9b7415b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:04.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b79400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b740cf540 0x7f9b7415b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52862/0 (socket says 172.21.15.42:52862) 2024-05-28T13:59:04.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b79400640 1 -- 172.21.15.42:0/1894733048 learned_addr learned my addr 172.21.15.42:0/1894733048 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:04.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.297+0000 7f9b7a800640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b74145d50 0x7f9b74155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:04.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b79400640 1 -- 172.21.15.42:0/1894733048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b740707e0 msgr2=0x7f9b740c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:04.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b79e00640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b740707e0 0x7f9b740c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:04.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b79400640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b740707e0 0x7f9b740c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b79400640 1 -- 172.21.15.42:0/1894733048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b74145d50 msgr2=0x7f9b74155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:04.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b79400640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b74145d50 0x7f9b74155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b79400640 1 -- 172.21.15.42:0/1894733048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7415bce0 con 0x7f9b740cf540 2024-05-28T13:59:04.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b7a800640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b74145d50 0x7f9b74155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:04.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b79e00640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b740707e0 0x7f9b740c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:04.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b79400640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b740cf540 0x7f9b7415b7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9b640ae040 tx=0x7f9b640191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:04.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b6e000640 1 -- 172.21.15.42:0/1894733048 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b640bb070 con 0x7f9b740cf540 2024-05-28T13:59:04.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b6e000640 1 -- 172.21.15.42:0/1894733048 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b640b6650 con 0x7f9b740cf540 2024-05-28T13:59:04.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b6e000640 1 -- 172.21.15.42:0/1894733048 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b640b0040 con 0x7f9b740cf540 2024-05-28T13:59:04.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b7b200640 1 -- 172.21.15.42:0/1894733048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b740c6ae0 con 0x7f9b740cf540 2024-05-28T13:59:04.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b7b200640 1 -- 172.21.15.42:0/1894733048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b740c70a0 con 0x7f9b740cf540 2024-05-28T13:59:04.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b7b200640 1 -- 172.21.15.42:0/1894733048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b7404fcf0 con 0x7f9b740cf540 2024-05-28T13:59:04.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b6e000640 1 -- 172.21.15.42:0/1894733048 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9b640b53d0 con 0x7f9b740cf540 2024-05-28T13:59:04.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b6e000640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b58068380 0x7f9b5806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:04.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b6e000640 1 -- 172.21.15.42:0/1894733048 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f9b641203a0 con 0x7f9b740cf540 2024-05-28T13:59:04.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.301+0000 7f9b79e00640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b58068380 0x7f9b5806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:04.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.305+0000 7f9b79e00640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b58068380 0x7f9b5806a840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009840 tx=0x7f9b68008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:04.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.305+0000 7f9b6e000640 1 -- 172.21.15.42:0/1894733048 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9b7415bce0 con 0x7f9b740cf540 2024-05-28T13:59:04.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.481+0000 7f9b7b200640 1 -- 172.21.15.42:0/1894733048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9b74071920 con 0x7f9b740cf540 2024-05-28T13:59:04.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.481+0000 7f9b6e000640 1 -- 172.21.15.42:0/1894733048 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f9b640160d0 con 0x7f9b740cf540 2024-05-28T13:59:04.466 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:59:04.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.481+0000 7f9b7b200640 1 -- 172.21.15.42:0/1894733048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b58068380 msgr2=0x7f9b5806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:04.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.481+0000 7f9b7b200640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b58068380 0x7f9b5806a840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009840 tx=0x7f9b68008040 comp rx=0 tx=0).stop 2024-05-28T13:59:04.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.481+0000 7f9b7b200640 1 -- 172.21.15.42:0/1894733048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b740cf540 msgr2=0x7f9b7415b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:04.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.481+0000 7f9b7b200640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b740cf540 0x7f9b7415b7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9b640ae040 tx=0x7f9b640191f0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.481+0000 7f9b7b200640 1 -- 172.21.15.42:0/1894733048 shutdown_connections 2024-05-28T13:59:04.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.481+0000 7f9b7b200640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b58068380 0x7f9b5806a840 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.481+0000 7f9b7b200640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b74145d50 0x7f9b74155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.481+0000 7f9b7b200640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b740cf540 0x7f9b7415b7a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.481+0000 7f9b7b200640 1 --2- 172.21.15.42:0/1894733048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b740707e0 0x7f9b740c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.481+0000 7f9b7b200640 1 -- 172.21.15.42:0/1894733048 >> 172.21.15.42:0/1894733048 conn(0x7f9b7406fcc0 msgr2=0x7f9b7415bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:04.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.481+0000 7f9b7b200640 1 -- 172.21.15.42:0/1894733048 shutdown_connections 2024-05-28T13:59:04.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.481+0000 7f9b7b200640 1 -- 172.21.15.42:0/1894733048 wait complete. 2024-05-28T13:59:04.488 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T13:59:04.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1011000640 1 Processor -- start 2024-05-28T13:59:04.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1011000640 1 -- start start 2024-05-28T13:59:04.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1011000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f100c162130 0x7f100c0b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:04.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1011000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f100c0d0870 0x7f100c0d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1011000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f100c0ccdb0 0x7f100c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1011000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f100c0d11d0 con 0x7f100c0d0870 2024-05-28T13:59:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1011000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f100c06dc50 con 0x7f100c0ccdb0 2024-05-28T13:59:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1011000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f100c06df30 con 0x7f100c162130 2024-05-28T13:59:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1001400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f100c0d0870 0x7f100c0d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1001400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f100c0d0870 0x7f100c0d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52878/0 (socket says 172.21.15.42:52878) 2024-05-28T13:59:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1001400640 1 -- 172.21.15.42:0/184478731 learned_addr learned my addr 172.21.15.42:0/184478731 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f100a800640 1 --2- 172.21.15.42:0/184478731 >> v2:172.21.15.63:3300/0 conn(0x7f100c0ccdb0 0x7f100c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1009e00640 1 --2- 172.21.15.42:0/184478731 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f100c162130 0x7f100c0b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1001400640 1 -- 172.21.15.42:0/184478731 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f100c162130 msgr2=0x7f100c0b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1001400640 1 --2- 172.21.15.42:0/184478731 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f100c162130 0x7f100c0b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1001400640 1 -- 172.21.15.42:0/184478731 >> v2:172.21.15.63:3300/0 conn(0x7f100c0ccdb0 msgr2=0x7f100c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1001400640 1 --2- 172.21.15.42:0/184478731 >> v2:172.21.15.63:3300/0 conn(0x7f100c0ccdb0 0x7f100c0cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.645+0000 7f1001400640 1 -- 172.21.15.42:0/184478731 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100c06e210 con 0x7f100c0d0870 2024-05-28T13:59:04.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.649+0000 7f1001400640 1 --2- 172.21.15.42:0/184478731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f100c0d0870 0x7f100c0d0c90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f100c0c6040 tx=0x7f0ff401cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608ae1a5f99e959c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.649+0000 7f1009400640 1 -- 172.21.15.42:0/184478731 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40aa660 con 0x7f100c0d0870 2024-05-28T13:59:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.649+0000 7f1009400640 1 -- 172.21.15.42:0/184478731 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff40af070 con 0x7f100c0d0870 2024-05-28T13:59:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.649+0000 7f1009400640 1 -- 172.21.15.42:0/184478731 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40aaa20 con 0x7f100c0d0870 2024-05-28T13:59:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.649+0000 7f1011000640 1 -- 172.21.15.42:0/184478731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f100c0d0870 msgr2=0x7f100c0d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.649+0000 7f1011000640 1 --2- 172.21.15.42:0/184478731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f100c0d0870 0x7f100c0d0c90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f100c0c6040 tx=0x7f0ff401cf80 comp rx=0 tx=0).stop 2024-05-28T13:59:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.649+0000 7f1011000640 1 -- 172.21.15.42:0/184478731 shutdown_connections 2024-05-28T13:59:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.649+0000 7f1011000640 1 --2- 172.21.15.42:0/184478731 >> v2:172.21.15.63:3300/0 conn(0x7f100c0ccdb0 0x7f100c0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.649+0000 7f1011000640 1 --2- 172.21.15.42:0/184478731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f100c0d0870 0x7f100c0d0c90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.649+0000 7f1011000640 1 --2- 172.21.15.42:0/184478731 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f100c162130 0x7f100c0b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.649+0000 7f1011000640 1 -- 172.21.15.42:0/184478731 >> 172.21.15.42:0/184478731 conn(0x7f100c0b4050 msgr2=0x7f100c0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.649+0000 7f1011000640 1 -- 172.21.15.42:0/184478731 shutdown_connections 2024-05-28T13:59:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.649+0000 7f1011000640 1 -- 172.21.15.42:0/184478731 wait complete. 2024-05-28T13:59:04.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1011000640 1 Processor -- start 2024-05-28T13:59:04.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1011000640 1 -- start start 2024-05-28T13:59:04.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1011000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f100c0ccdb0 0x7f100c1524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1011000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f100c0d0870 0x7f100c157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1011000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f100c162130 0x7f100c0b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1011000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f100c09ade0 con 0x7f100c162130 2024-05-28T13:59:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1011000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f100c09ac60 con 0x7f100c0ccdb0 2024-05-28T13:59:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1011000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f100c09af60 con 0x7f100c0d0870 2024-05-28T13:59:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1009e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f100c0ccdb0 0x7f100c1524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f100a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f100c162130 0x7f100c0b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1009e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f100c0ccdb0 0x7f100c1524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:58338/0 (socket says 172.21.15.42:58338) 2024-05-28T13:59:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1009e00640 1 -- 172.21.15.42:0/327527072 learned_addr learned my addr 172.21.15.42:0/327527072 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1001400640 1 --2- 172.21.15.42:0/327527072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f100c0d0870 0x7f100c157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f100a800640 1 -- 172.21.15.42:0/327527072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f100c0d0870 msgr2=0x7f100c157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f100a800640 1 --2- 172.21.15.42:0/327527072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f100c0d0870 0x7f100c157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f100a800640 1 -- 172.21.15.42:0/327527072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f100c0ccdb0 msgr2=0x7f100c1524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:04.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f100a800640 1 --2- 172.21.15.42:0/327527072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f100c0ccdb0 0x7f100c1524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f100a800640 1 -- 172.21.15.42:0/327527072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100c0cb160 con 0x7f100c162130 2024-05-28T13:59:04.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1009e00640 1 --2- 172.21.15.42:0/327527072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f100c0ccdb0 0x7f100c1524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:59:04.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f100a800640 1 --2- 172.21.15.42:0/327527072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f100c162130 0x7f100c0b62f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1004007dc0 tx=0x7f100400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:04.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1002a00640 1 -- 172.21.15.42:0/327527072 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100400edd0 con 0x7f100c162130 2024-05-28T13:59:04.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1002a00640 1 -- 172.21.15.42:0/327527072 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10040039a0 con 0x7f100c162130 2024-05-28T13:59:04.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1002a00640 1 -- 172.21.15.42:0/327527072 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004003d90 con 0x7f100c162130 2024-05-28T13:59:04.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1011000640 1 -- 172.21.15.42:0/327527072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f100c0cb440 con 0x7f100c162130 2024-05-28T13:59:04.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.653+0000 7f1011000640 1 -- 172.21.15.42:0/327527072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f100c0cba00 con 0x7f100c162130 2024-05-28T13:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.657+0000 7f1011000640 1 -- 172.21.15.42:0/327527072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f100c0a0650 con 0x7f100c162130 2024-05-28T13:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.657+0000 7f1002a00640 1 -- 172.21.15.42:0/327527072 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f100401b020 con 0x7f100c162130 2024-05-28T13:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.657+0000 7f1002a00640 1 --2- 172.21.15.42:0/327527072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0fe8068380 0x7f0fe806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.657+0000 7f1002a00640 1 -- 172.21.15.42:0/327527072 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f1004086b50 con 0x7f100c162130 2024-05-28T13:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.657+0000 7f1009e00640 1 --2- 172.21.15.42:0/327527072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0fe8068380 0x7f0fe806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.657+0000 7f1009e00640 1 --2- 172.21.15.42:0/327527072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0fe8068380 0x7f0fe806a840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f100c158a50 tx=0x7f0ffc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:04.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.661+0000 7f1002a00640 1 -- 172.21.15.42:0/327527072 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f10040569a0 con 0x7f100c162130 2024-05-28T13:59:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.841+0000 7f1011000640 1 -- 172.21.15.42:0/327527072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f100c0b3240 con 0x7f100c162130 2024-05-28T13:59:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.841+0000 7f1002a00640 1 -- 172.21.15.42:0/327527072 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f100405aca0 con 0x7f100c162130 2024-05-28T13:59:04.826 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T13:59:04.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.841+0000 7f1011000640 1 -- 172.21.15.42:0/327527072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0fe8068380 msgr2=0x7f0fe806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:04.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.841+0000 7f1011000640 1 --2- 172.21.15.42:0/327527072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0fe8068380 0x7f0fe806a840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f100c158a50 tx=0x7f0ffc008040 comp rx=0 tx=0).stop 2024-05-28T13:59:04.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.841+0000 7f1011000640 1 -- 172.21.15.42:0/327527072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f100c162130 msgr2=0x7f100c0b62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:04.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.841+0000 7f1011000640 1 --2- 172.21.15.42:0/327527072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f100c162130 0x7f100c0b62f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1004007dc0 tx=0x7f100400e4b0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.841+0000 7f1011000640 1 -- 172.21.15.42:0/327527072 shutdown_connections 2024-05-28T13:59:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.841+0000 7f1011000640 1 --2- 172.21.15.42:0/327527072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0fe8068380 0x7f0fe806a840 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.841+0000 7f1011000640 1 --2- 172.21.15.42:0/327527072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f100c162130 0x7f100c0b62f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.841+0000 7f1011000640 1 --2- 172.21.15.42:0/327527072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f100c0d0870 0x7f100c157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.841+0000 7f1011000640 1 --2- 172.21.15.42:0/327527072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f100c0ccdb0 0x7f100c1524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.841+0000 7f1011000640 1 -- 172.21.15.42:0/327527072 >> 172.21.15.42:0/327527072 conn(0x7f100c0b4050 msgr2=0x7f100c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.841+0000 7f1011000640 1 -- 172.21.15.42:0/327527072 shutdown_connections 2024-05-28T13:59:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:04.845+0000 7f1011000640 1 -- 172.21.15.42:0/327527072 wait complete. 2024-05-28T13:59:04.848 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi063.front.sepia.ceph.com... 2024-05-28T13:59:04.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:04.848 DEBUG:teuthology.orchestra.run.smithi063:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-05-28T13:59:04.885 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T13:59:04.885 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T13:59:04.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:04.886 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-28T13:59:04.943 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:59:04.943 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:59:04.952 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:59:04.952 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:59:05.004 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:59:05.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71de000640 1 Processor -- start 2024-05-28T13:59:05.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71de000640 1 -- start start 2024-05-28T13:59:05.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71de000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d806f660 0x7f71d8073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71de000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f71d80d0e00 0x7f71d806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71de000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71d815f280 con 0x7f71d815ba40 2024-05-28T13:59:05.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71de000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f71d815f030 con 0x7f71d80d0e00 2024-05-28T13:59:05.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71de000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f71d815f560 con 0x7f71d806f660 2024-05-28T13:59:05.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71d7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f71d80d0e00 0x7f71d806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71cfe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f71d815ba40 0x7f71d80d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34834/0 (socket says 172.21.15.42:34834) 2024-05-28T13:59:05.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71d7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f71d80d0e00 0x7f71d806dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52904/0 (socket says 172.21.15.42:52904) 2024-05-28T13:59:05.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71cfe00640 1 -- 172.21.15.42:0/12685741 learned_addr learned my addr 172.21.15.42:0/12685741 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:05.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71d7400640 1 --2- 172.21.15.42:0/12685741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d806f660 0x7f71d8073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71d7e00640 1 -- 172.21.15.42:0/12685741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d806f660 msgr2=0x7f71d8073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:05.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71d7e00640 1 --2- 172.21.15.42:0/12685741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d806f660 0x7f71d8073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71d7e00640 1 -- 172.21.15.42:0/12685741 >> v1:172.21.15.42:6789/0 conn(0x7f71d815ba40 legacy=0x7f71d80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71d7e00640 1 -- 172.21.15.42:0/12685741 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d815f840 con 0x7f71d80d0e00 2024-05-28T13:59:05.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.209+0000 7f71d7400640 1 --2- 172.21.15.42:0/12685741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d806f660 0x7f71d8073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:05.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.213+0000 7f71d7e00640 1 --2- 172.21.15.42:0/12685741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d80d0e00 0x7f71d806dd60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009d90 tx=0x7f71d001c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9b3f9dea70fdf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:05.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.213+0000 7f71d6a00640 1 -- 172.21.15.42:0/12685741 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d00b2070 con 0x7f71d80d0e00 2024-05-28T13:59:05.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.213+0000 7f71d6a00640 1 -- 172.21.15.42:0/12685741 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d00a58c0 con 0x7f71d80d0e00 2024-05-28T13:59:05.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.213+0000 7f71d6a00640 1 -- 172.21.15.42:0/12685741 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d00a5c90 con 0x7f71d80d0e00 2024-05-28T13:59:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.213+0000 7f71de000640 1 -- 172.21.15.42:0/12685741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d80d0e00 msgr2=0x7f71d806dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.213+0000 7f71de000640 1 --2- 172.21.15.42:0/12685741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d80d0e00 0x7f71d806dd60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009d90 tx=0x7f71d001c820 comp rx=0 tx=0).stop 2024-05-28T13:59:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.213+0000 7f71de000640 1 -- 172.21.15.42:0/12685741 shutdown_connections 2024-05-28T13:59:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.213+0000 7f71de000640 1 --2- 172.21.15.42:0/12685741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d80d0e00 0x7f71d806dd60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.213+0000 7f71de000640 1 --2- 172.21.15.42:0/12685741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d806f660 0x7f71d8073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.213+0000 7f71de000640 1 -- 172.21.15.42:0/12685741 >> 172.21.15.42:0/12685741 conn(0x7f71d806bdf0 msgr2=0x7f71d806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.213+0000 7f71de000640 1 -- 172.21.15.42:0/12685741 shutdown_connections 2024-05-28T13:59:05.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.213+0000 7f71de000640 1 -- 172.21.15.42:0/12685741 wait complete. 2024-05-28T13:59:05.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71de000640 1 Processor -- start 2024-05-28T13:59:05.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71de000640 1 -- start start 2024-05-28T13:59:05.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71de000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d806f660 0x7f71d80ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71de000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71d80d0e00 0x7f71d8099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71de000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d814fa20 0x7f71d809ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71de000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8168550 con 0x7f71d806f660 2024-05-28T13:59:05.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71de000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f71d81683d0 con 0x7f71d80d0e00 2024-05-28T13:59:05.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71de000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f71d81686d0 con 0x7f71d814fa20 2024-05-28T13:59:05.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71d7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d806f660 0x7f71d80ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71d7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d806f660 0x7f71d80ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52906/0 (socket says 172.21.15.42:52906) 2024-05-28T13:59:05.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71d7400640 1 -- 172.21.15.42:0/4274550884 learned_addr learned my addr 172.21.15.42:0/4274550884 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:05.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71d7400640 1 -- 172.21.15.42:0/4274550884 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d814fa20 msgr2=0x7f71d809ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71d7400640 1 --2- 172.21.15.42:0/4274550884 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d814fa20 0x7f71d809ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71d7400640 1 -- 172.21.15.42:0/4274550884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71d80d0e00 msgr2=0x7f71d8099920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71d7400640 1 --2- 172.21.15.42:0/4274550884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71d80d0e00 0x7f71d8099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71d7400640 1 -- 172.21.15.42:0/4274550884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d80ce5d0 con 0x7f71d806f660 2024-05-28T13:59:05.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71d7400640 1 --2- 172.21.15.42:0/4274550884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d806f660 0x7f71d80ce090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e810 tx=0x7f71c800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:05.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71cf400640 1 -- 172.21.15.42:0/4274550884 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8017070 con 0x7f71d806f660 2024-05-28T13:59:05.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71cf400640 1 -- 172.21.15.42:0/4274550884 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c800c8d0 con 0x7f71d806f660 2024-05-28T13:59:05.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71de000640 1 -- 172.21.15.42:0/4274550884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d80ce8a0 con 0x7f71d806f660 2024-05-28T13:59:05.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71cf400640 1 -- 172.21.15.42:0/4274550884 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c800f040 con 0x7f71d806f660 2024-05-28T13:59:05.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.217+0000 7f71de000640 1 -- 172.21.15.42:0/4274550884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d809a4e0 con 0x7f71d806f660 2024-05-28T13:59:05.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.221+0000 7f71de000640 1 -- 172.21.15.42:0/4274550884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d806ee00 con 0x7f71d806f660 2024-05-28T13:59:05.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.221+0000 7f71cf400640 1 -- 172.21.15.42:0/4274550884 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f71c801a070 con 0x7f71d806f660 2024-05-28T13:59:05.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.221+0000 7f71cf400640 1 --2- 172.21.15.42:0/4274550884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f71b4068380 0x7f71b406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.221+0000 7f71cf400640 1 -- 172.21.15.42:0/4274550884 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f71c8086680 con 0x7f71d806f660 2024-05-28T13:59:05.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.221+0000 7f71cfe00640 1 --2- 172.21.15.42:0/4274550884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f71b4068380 0x7f71b406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.221+0000 7f71cfe00640 1 --2- 172.21.15.42:0/4274550884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f71b4068380 0x7f71b406a840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f71c0000f70 tx=0x7f71c0007a90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:05.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.225+0000 7f71cf400640 1 -- 172.21.15.42:0/4274550884 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f71c8056b20 con 0x7f71d806f660 2024-05-28T13:59:05.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.393+0000 7f71de000640 1 -- 172.21.15.42:0/4274550884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71d8068a90 con 0x7f71d806f660 2024-05-28T13:59:05.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.393+0000 7f71cf400640 1 -- 172.21.15.42:0/4274550884 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16737 (secure 0 0 0) 0x7f71c8056420 con 0x7f71d806f660 2024-05-28T13:59:05.378 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:05.378 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:59:03.992606+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:59:05.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.393+0000 7f71de000640 1 -- 172.21.15.42:0/4274550884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f71b4068380 msgr2=0x7f71b406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.393+0000 7f71de000640 1 --2- 172.21.15.42:0/4274550884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f71b4068380 0x7f71b406a840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f71c0000f70 tx=0x7f71c0007a90 comp rx=0 tx=0).stop 2024-05-28T13:59:05.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.393+0000 7f71de000640 1 -- 172.21.15.42:0/4274550884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d806f660 msgr2=0x7f71d80ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.393+0000 7f71de000640 1 --2- 172.21.15.42:0/4274550884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d806f660 0x7f71d80ce090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e810 tx=0x7f71c800ecd0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.393+0000 7f71de000640 1 -- 172.21.15.42:0/4274550884 shutdown_connections 2024-05-28T13:59:05.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.393+0000 7f71de000640 1 --2- 172.21.15.42:0/4274550884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f71b4068380 0x7f71b406a840 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.393+0000 7f71de000640 1 --2- 172.21.15.42:0/4274550884 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d814fa20 0x7f71d809ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.393+0000 7f71de000640 1 --2- 172.21.15.42:0/4274550884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71d80d0e00 0x7f71d8099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.393+0000 7f71de000640 1 --2- 172.21.15.42:0/4274550884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d806f660 0x7f71d80ce090 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.393+0000 7f71de000640 1 -- 172.21.15.42:0/4274550884 >> 172.21.15.42:0/4274550884 conn(0x7f71d806bdf0 msgr2=0x7f71d80d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.393+0000 7f71de000640 1 -- 172.21.15.42:0/4274550884 shutdown_connections 2024-05-28T13:59:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.393+0000 7f71de000640 1 -- 172.21.15.42:0/4274550884 wait complete. 2024-05-28T13:59:05.400 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:59:05.400 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:59:05.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.557+0000 7fa3a2600640 1 Processor -- start 2024-05-28T13:59:05.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.557+0000 7fa3a2600640 1 -- start start 2024-05-28T13:59:05.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa3a2600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa39c071de0 0x7fa39c0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa3a2600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa39c13fd10 0x7fa39c1401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa3a2600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa39c072700 con 0x7fa39c0ce3b0 2024-05-28T13:59:05.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa3a2600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa39c160230 con 0x7fa39c071de0 2024-05-28T13:59:05.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa3a2600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa39c160510 con 0x7fa39c13fd10 2024-05-28T13:59:05.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa393e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa39c0ce3b0 0x7fa39c0ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34850/0 (socket says 172.21.15.42:34850) 2024-05-28T13:59:05.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa393e00640 1 -- 172.21.15.42:0/3678569440 learned_addr learned my addr 172.21.15.42:0/3678569440 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:05.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39be00640 1 --2- 172.21.15.42:0/3678569440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa39c13fd10 0x7fa39c1401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39b400640 1 --2- 172.21.15.42:0/3678569440 >> v2:172.21.15.63:3300/0 conn(0x7fa39c071de0 0x7fa39c0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 -- 172.21.15.42:0/3678569440 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1532815126 0 0) 0x7fa39c072700 con 0x7fa39c0ce3b0 2024-05-28T13:59:05.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 -- 172.21.15.42:0/3678569440 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa380003530 con 0x7fa39c0ce3b0 2024-05-28T13:59:05.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 -- 172.21.15.42:0/3678569440 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 660700414 0 0) 0x7fa380003530 con 0x7fa39c0ce3b0 2024-05-28T13:59:05.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 -- 172.21.15.42:0/3678569440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa39c13fd10 msgr2=0x7fa39c1401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:05.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 --2- 172.21.15.42:0/3678569440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa39c13fd10 0x7fa39c1401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 -- 172.21.15.42:0/3678569440 >> v2:172.21.15.63:3300/0 conn(0x7fa39c071de0 msgr2=0x7fa39c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:05.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 --2- 172.21.15.42:0/3678569440 >> v2:172.21.15.63:3300/0 conn(0x7fa39c071de0 0x7fa39c0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 -- 172.21.15.42:0/3678569440 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39c1607f0 con 0x7fa39c0ce3b0 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 -- 172.21.15.42:0/3678569440 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fa384001680 con 0x7fa39c0ce3b0 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 -- 172.21.15.42:0/3678569440 >> v1:172.21.15.42:6789/0 conn(0x7fa39c0ce3b0 legacy=0x7fa39c0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 --2- 172.21.15.42:0/3678569440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa39c13fd10 0x7fa3800039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 --2- 172.21.15.42:0/3678569440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa380003f40 0x7fa380004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 --2- 172.21.15.42:0/3678569440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa380004d50 0x7fa3800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 -- 172.21.15.42:0/3678569440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c160230 con 0x7fa380004d50 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 -- 172.21.15.42:0/3678569440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c160510 con 0x7fa380003f40 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39aa00640 1 -- 172.21.15.42:0/3678569440 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa38000bab0 con 0x7fa39c13fd10 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39be00640 1 --2- 172.21.15.42:0/3678569440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa380004d50 0x7fa3800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39b400640 1 --2- 172.21.15.42:0/3678569440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa39c13fd10 0x7fa3800039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa393e00640 1 --2- 172.21.15.42:0/3678569440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa380003f40 0x7fa380004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39be00640 1 -- 172.21.15.42:0/3678569440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa39c13fd10 msgr2=0x7fa3800039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39be00640 1 --2- 172.21.15.42:0/3678569440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa39c13fd10 0x7fa3800039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39be00640 1 -- 172.21.15.42:0/3678569440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa380003f40 msgr2=0x7fa380004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:05.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39be00640 1 --2- 172.21.15.42:0/3678569440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa380003f40 0x7fa380004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa39be00640 1 -- 172.21.15.42:0/3678569440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa380003530 con 0x7fa380004d50 2024-05-28T13:59:05.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.561+0000 7fa393e00640 1 --2- 172.21.15.42:0/3678569440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa380003f40 0x7fa380004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:05.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.565+0000 7fa39be00640 1 --2- 172.21.15.42:0/3678569440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa380004d50 0x7fa3800051d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa394009fc0 tx=0x7fa39401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c38aca408bc2ec8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:05.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.565+0000 7fa39aa00640 1 -- 172.21.15.42:0/3678569440 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3940b2070 con 0x7fa380004d50 2024-05-28T13:59:05.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.565+0000 7fa39aa00640 1 -- 172.21.15.42:0/3678569440 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3940a5a70 con 0x7fa380004d50 2024-05-28T13:59:05.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.565+0000 7fa39aa00640 1 -- 172.21.15.42:0/3678569440 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394009c10 con 0x7fa380004d50 2024-05-28T13:59:05.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.565+0000 7fa3a2600640 1 -- 172.21.15.42:0/3678569440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa380004d50 msgr2=0x7fa3800051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.565+0000 7fa3a2600640 1 --2- 172.21.15.42:0/3678569440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa380004d50 0x7fa3800051d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa394009fc0 tx=0x7fa39401cc80 comp rx=0 tx=0).stop 2024-05-28T13:59:05.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.565+0000 7fa39be00640 1 -- 172.21.15.42:0/3678569440 reap_dead start 2024-05-28T13:59:05.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.565+0000 7fa3a2600640 1 -- 172.21.15.42:0/3678569440 shutdown_connections 2024-05-28T13:59:05.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.565+0000 7fa3a2600640 1 -- 172.21.15.42:0/3678569440 >> 172.21.15.42:0/3678569440 conn(0x7fa39c0b21f0 msgr2=0x7fa39c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:05.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.565+0000 7fa3a2600640 1 -- 172.21.15.42:0/3678569440 shutdown_connections 2024-05-28T13:59:05.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.565+0000 7fa3a2600640 1 -- 172.21.15.42:0/3678569440 wait complete. 2024-05-28T13:59:05.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.565+0000 7fa3a2600640 1 Processor -- start 2024-05-28T13:59:05.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.565+0000 7fa3a2600640 1 -- start start 2024-05-28T13:59:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa3a2600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa380004d50 0x7fa39c137870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa3a2600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa39c155dd0 0x7fa39c1f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa3a2600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa39c1f76d0 0x7fa39c139440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa3a2600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c140c90 con 0x7fa39c155dd0 2024-05-28T13:59:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa3a2600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c140b10 con 0x7fa380004d50 2024-05-28T13:59:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa3a2600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa39c140e10 con 0x7fa39c1f76d0 2024-05-28T13:59:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa393e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa39c155dd0 0x7fa39c1f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa393e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa39c155dd0 0x7fa39c1f7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52912/0 (socket says 172.21.15.42:52912) 2024-05-28T13:59:05.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa393e00640 1 -- 172.21.15.42:0/1419888238 learned_addr learned my addr 172.21.15.42:0/1419888238 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa39b400640 1 --2- 172.21.15.42:0/1419888238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa380004d50 0x7fa39c137870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa39be00640 1 --2- 172.21.15.42:0/1419888238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa39c1f76d0 0x7fa39c139440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa393e00640 1 -- 172.21.15.42:0/1419888238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa39c1f76d0 msgr2=0x7fa39c139440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa393e00640 1 --2- 172.21.15.42:0/1419888238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa39c1f76d0 0x7fa39c139440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa393e00640 1 -- 172.21.15.42:0/1419888238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa380004d50 msgr2=0x7fa39c137870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa393e00640 1 --2- 172.21.15.42:0/1419888238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa380004d50 0x7fa39c137870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa393e00640 1 -- 172.21.15.42:0/1419888238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39c134a80 con 0x7fa39c155dd0 2024-05-28T13:59:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa393e00640 1 --2- 172.21.15.42:0/1419888238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa39c155dd0 0x7fa39c1f7190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa38400e7f0 tx=0x7fa38400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa393400640 1 -- 172.21.15.42:0/1419888238 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38400c870 con 0x7fa39c155dd0 2024-05-28T13:59:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa393400640 1 -- 172.21.15.42:0/1419888238 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa384018070 con 0x7fa39c155dd0 2024-05-28T13:59:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa3a2600640 1 -- 172.21.15.42:0/1419888238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39c134d60 con 0x7fa39c155dd0 2024-05-28T13:59:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa393400640 1 -- 172.21.15.42:0/1419888238 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38400cc10 con 0x7fa39c155dd0 2024-05-28T13:59:05.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa3a2600640 1 -- 172.21.15.42:0/1419888238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39c136b10 con 0x7fa39c155dd0 2024-05-28T13:59:05.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa3a2600640 1 -- 172.21.15.42:0/1419888238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa39c1f7ab0 con 0x7fa39c155dd0 2024-05-28T13:59:05.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.569+0000 7fa393400640 1 -- 172.21.15.42:0/1419888238 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa38401d460 con 0x7fa39c155dd0 2024-05-28T13:59:05.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.573+0000 7fa393400640 1 --2- 172.21.15.42:0/1419888238 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa378068380 0x7fa37806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.573+0000 7fa393400640 1 -- 172.21.15.42:0/1419888238 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa384086950 con 0x7fa39c155dd0 2024-05-28T13:59:05.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.573+0000 7fa39b400640 1 --2- 172.21.15.42:0/1419888238 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa378068380 0x7fa37806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.573+0000 7fa39b400640 1 --2- 172.21.15.42:0/1419888238 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa378068380 0x7fa37806a840 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009ba0 tx=0x7fa38c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:05.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.577+0000 7fa393400640 1 -- 172.21.15.42:0/1419888238 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa384056df0 con 0x7fa39c155dd0 2024-05-28T13:59:05.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.737+0000 7fa3a2600640 1 -- 172.21.15.42:0/1419888238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa39c04fdd0 con 0x7fa39c155dd0 2024-05-28T13:59:05.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.741+0000 7fa393400640 1 -- 172.21.15.42:0/1419888238 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fa3840566f0 con 0x7fa39c155dd0 2024-05-28T13:59:05.725 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:59:05.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.741+0000 7fa3a2600640 1 -- 172.21.15.42:0/1419888238 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa378068380 msgr2=0x7fa37806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.741+0000 7fa3a2600640 1 --2- 172.21.15.42:0/1419888238 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa378068380 0x7fa37806a840 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009ba0 tx=0x7fa38c008040 comp rx=0 tx=0).stop 2024-05-28T13:59:05.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.741+0000 7fa3a2600640 1 -- 172.21.15.42:0/1419888238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa39c155dd0 msgr2=0x7fa39c1f7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.741+0000 7fa3a2600640 1 --2- 172.21.15.42:0/1419888238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa39c155dd0 0x7fa39c1f7190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa38400e7f0 tx=0x7fa38400ecb0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.741+0000 7fa3a2600640 1 -- 172.21.15.42:0/1419888238 shutdown_connections 2024-05-28T13:59:05.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.741+0000 7fa3a2600640 1 --2- 172.21.15.42:0/1419888238 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa378068380 0x7fa37806a840 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.741+0000 7fa3a2600640 1 --2- 172.21.15.42:0/1419888238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa39c1f76d0 0x7fa39c139440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.741+0000 7fa3a2600640 1 --2- 172.21.15.42:0/1419888238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa39c155dd0 0x7fa39c1f7190 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.741+0000 7fa3a2600640 1 --2- 172.21.15.42:0/1419888238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa380004d50 0x7fa39c137870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.741+0000 7fa3a2600640 1 -- 172.21.15.42:0/1419888238 >> 172.21.15.42:0/1419888238 conn(0x7fa39c0b21f0 msgr2=0x7fa39c13ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:05.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.741+0000 7fa3a2600640 1 -- 172.21.15.42:0/1419888238 shutdown_connections 2024-05-28T13:59:05.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.741+0000 7fa3a2600640 1 -- 172.21.15.42:0/1419888238 wait complete. 2024-05-28T13:59:05.746 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:59:05.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.905+0000 7f8d50600640 1 Processor -- start 2024-05-28T13:59:05.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.905+0000 7f8d50600640 1 -- start start 2024-05-28T13:59:05.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d50600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8d480d1770 0x7f8d480cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d50600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d4815eb50 0x7f8d480d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d50600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d480d2400 con 0x7f8d4815e770 2024-05-28T13:59:05.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d50600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8d480d26e0 con 0x7f8d480d1770 2024-05-28T13:59:05.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d50600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8d480d29c0 con 0x7f8d4815eb50 2024-05-28T13:59:05.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d4d200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8d480d1770 0x7f8d480cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d4d200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8d480d1770 0x7f8d480cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52924/0 (socket says 172.21.15.42:52924) 2024-05-28T13:59:05.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d4d200640 1 -- 172.21.15.42:0/2789220361 learned_addr learned my addr 172.21.15.42:0/2789220361 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:05.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d4dc00640 1 --1- 172.21.15.42:0/2789220361 >> v1:172.21.15.42:6789/0 conn(0x7f8d4815e770 0x7f8d480d1060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34860/0 (socket says 172.21.15.42:34860) 2024-05-28T13:59:05.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d4e600640 1 --2- 172.21.15.42:0/2789220361 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d4815eb50 0x7f8d480d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d4d200640 1 -- 172.21.15.42:0/2789220361 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d4815eb50 msgr2=0x7f8d480d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:05.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d4d200640 1 --2- 172.21.15.42:0/2789220361 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d4815eb50 0x7f8d480d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d4d200640 1 -- 172.21.15.42:0/2789220361 >> v1:172.21.15.42:6789/0 conn(0x7f8d4815e770 legacy=0x7f8d480d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d4d200640 1 -- 172.21.15.42:0/2789220361 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d480d2ca0 con 0x7f8d480d1770 2024-05-28T13:59:05.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.909+0000 7f8d47e00640 1 -- 172.21.15.42:0/2789220361 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1488452102 0 0) 0x7f8d480d2400 con 0x7f8d4815e770 2024-05-28T13:59:05.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.913+0000 7f8d4d200640 1 --2- 172.21.15.42:0/2789220361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480d1770 0x7f8d480cfdb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009870 tx=0x7f8d3401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa70d8c2a38e4465 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:05.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.913+0000 7f8d47e00640 1 -- 172.21.15.42:0/2789220361 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d340a7070 con 0x7f8d480d1770 2024-05-28T13:59:05.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.913+0000 7f8d47e00640 1 -- 172.21.15.42:0/2789220361 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d340b0070 con 0x7f8d480d1770 2024-05-28T13:59:05.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.913+0000 7f8d47e00640 1 -- 172.21.15.42:0/2789220361 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d340ab890 con 0x7f8d480d1770 2024-05-28T13:59:05.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.913+0000 7f8d50600640 1 -- 172.21.15.42:0/2789220361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480d1770 msgr2=0x7f8d480cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.913+0000 7f8d50600640 1 --2- 172.21.15.42:0/2789220361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480d1770 0x7f8d480cfdb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009870 tx=0x7f8d3401c560 comp rx=0 tx=0).stop 2024-05-28T13:59:05.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.913+0000 7f8d50600640 1 -- 172.21.15.42:0/2789220361 shutdown_connections 2024-05-28T13:59:05.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.913+0000 7f8d50600640 1 --2- 172.21.15.42:0/2789220361 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d4815eb50 0x7f8d480d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.913+0000 7f8d50600640 1 --2- 172.21.15.42:0/2789220361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480d1770 0x7f8d480cfdb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.913+0000 7f8d50600640 1 -- 172.21.15.42:0/2789220361 >> 172.21.15.42:0/2789220361 conn(0x7f8d4808bfe0 msgr2=0x7f8d480d4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:05.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.913+0000 7f8d50600640 1 -- 172.21.15.42:0/2789220361 shutdown_connections 2024-05-28T13:59:05.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.913+0000 7f8d50600640 1 -- 172.21.15.42:0/2789220361 wait complete. 2024-05-28T13:59:05.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.913+0000 7f8d50600640 1 Processor -- start 2024-05-28T13:59:05.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.913+0000 7f8d50600640 1 -- start start 2024-05-28T13:59:05.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d50600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d480d1770 0x7f8d48157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d50600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d4815e770 0x7f8d480b9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d50600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d4815eb50 0x7f8d480b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d50600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4816b350 con 0x7f8d4815e770 2024-05-28T13:59:05.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d50600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4816b1d0 con 0x7f8d4815eb50 2024-05-28T13:59:05.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d50600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8d4816b4d0 con 0x7f8d480d1770 2024-05-28T13:59:05.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d4d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d4815e770 0x7f8d480b9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d4d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d4815e770 0x7f8d480b9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52928/0 (socket says 172.21.15.42:52928) 2024-05-28T13:59:05.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d4d200640 1 -- 172.21.15.42:0/75465265 learned_addr learned my addr 172.21.15.42:0/75465265 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:05.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d4d200640 1 -- 172.21.15.42:0/75465265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d480d1770 msgr2=0x7f8d48157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d4d200640 1 --2- 172.21.15.42:0/75465265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d480d1770 0x7f8d48157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d4d200640 1 -- 172.21.15.42:0/75465265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d4815eb50 msgr2=0x7f8d480b7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:05.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d4d200640 1 --2- 172.21.15.42:0/75465265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d4815eb50 0x7f8d480b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:05.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d4d200640 1 -- 172.21.15.42:0/75465265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d48159960 con 0x7f8d4815e770 2024-05-28T13:59:05.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d4d200640 1 --2- 172.21.15.42:0/75465265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d4815e770 0x7f8d480b9b00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8d3401ac10 tx=0x7f8d3401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:05.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d45600640 1 -- 172.21.15.42:0/75465265 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d340b4800 con 0x7f8d4815e770 2024-05-28T13:59:05.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d45600640 1 -- 172.21.15.42:0/75465265 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d340b49a0 con 0x7f8d4815e770 2024-05-28T13:59:05.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d45600640 1 -- 172.21.15.42:0/75465265 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d340b4d90 con 0x7f8d4815e770 2024-05-28T13:59:05.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d50600640 1 -- 172.21.15.42:0/75465265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d481593a0 con 0x7f8d4815e770 2024-05-28T13:59:05.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d50600640 1 -- 172.21.15.42:0/75465265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4815fcf0 con 0x7f8d4815e770 2024-05-28T13:59:05.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d50600640 1 -- 172.21.15.42:0/75465265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d480a2580 con 0x7f8d4815e770 2024-05-28T13:59:05.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.917+0000 7f8d45600640 1 -- 172.21.15.42:0/75465265 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8d340b3380 con 0x7f8d4815e770 2024-05-28T13:59:05.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.921+0000 7f8d45600640 1 --2- 172.21.15.42:0/75465265 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d280683b0 0x7f8d2806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:05.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.921+0000 7f8d45600640 1 -- 172.21.15.42:0/75465265 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f8d34120e80 con 0x7f8d4815e770 2024-05-28T13:59:05.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.921+0000 7f8d4dc00640 1 --2- 172.21.15.42:0/75465265 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d280683b0 0x7f8d2806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:05.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.921+0000 7f8d4dc00640 1 --2- 172.21.15.42:0/75465265 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d280683b0 0x7f8d2806a870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0011f0 tx=0x7f8d3c007930 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:05.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:05.925+0000 7f8d45600640 1 -- 172.21.15.42:0/75465265 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8d34016150 con 0x7f8d4815e770 2024-05-28T13:59:06.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.097+0000 7f8d50600640 1 -- 172.21.15.42:0/75465265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8d480ba610 con 0x7f8d4815e770 2024-05-28T13:59:06.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.097+0000 7f8d45600640 1 -- 172.21.15.42:0/75465265 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f8d340f13c0 con 0x7f8d4815e770 2024-05-28T13:59:06.084 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:59:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.101+0000 7f8d50600640 1 -- 172.21.15.42:0/75465265 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d280683b0 msgr2=0x7f8d2806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.101+0000 7f8d50600640 1 --2- 172.21.15.42:0/75465265 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d280683b0 0x7f8d2806a870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0011f0 tx=0x7f8d3c007930 comp rx=0 tx=0).stop 2024-05-28T13:59:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.101+0000 7f8d50600640 1 -- 172.21.15.42:0/75465265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d4815e770 msgr2=0x7f8d480b9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.101+0000 7f8d50600640 1 --2- 172.21.15.42:0/75465265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d4815e770 0x7f8d480b9b00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8d3401ac10 tx=0x7f8d3401ada0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.101+0000 7f8d50600640 1 -- 172.21.15.42:0/75465265 shutdown_connections 2024-05-28T13:59:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.101+0000 7f8d50600640 1 --2- 172.21.15.42:0/75465265 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d280683b0 0x7f8d2806a870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.101+0000 7f8d50600640 1 --2- 172.21.15.42:0/75465265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d4815eb50 0x7f8d480b7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.101+0000 7f8d50600640 1 --2- 172.21.15.42:0/75465265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d4815e770 0x7f8d480b9b00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.101+0000 7f8d50600640 1 --2- 172.21.15.42:0/75465265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d480d1770 0x7f8d48157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.101+0000 7f8d50600640 1 -- 172.21.15.42:0/75465265 >> 172.21.15.42:0/75465265 conn(0x7f8d4808bfe0 msgr2=0x7f8d480b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:06.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.101+0000 7f8d50600640 1 -- 172.21.15.42:0/75465265 shutdown_connections 2024-05-28T13:59:06.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.101+0000 7f8d50600640 1 -- 172.21.15.42:0/75465265 wait complete. 2024-05-28T13:59:06.105 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:59:06.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.261+0000 7f859ae00640 1 Processor -- start 2024-05-28T13:59:06.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.265+0000 7f859ae00640 1 -- start start 2024-05-28T13:59:06.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.265+0000 7f859ae00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8594071de0 0x7f85940721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:06.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.265+0000 7f859ae00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8594072700 con 0x7f85940ce3b0 2024-05-28T13:59:06.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.265+0000 7f859ae00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8594160230 con 0x7f859413fd10 2024-05-28T13:59:06.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.265+0000 7f859ae00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8594160510 con 0x7f8594071de0 2024-05-28T13:59:06.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.265+0000 7f8593e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8594071de0 0x7f85940721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:06.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.265+0000 7f8593e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8594071de0 0x7f85940721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52942/0 (socket says 172.21.15.42:52942) 2024-05-28T13:59:06.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.265+0000 7f8593e00640 1 -- 172.21.15.42:0/3166257192 learned_addr learned my addr 172.21.15.42:0/3166257192 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:06.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.265+0000 7f858b400640 1 --1- 172.21.15.42:0/3166257192 >> v1:172.21.15.42:6789/0 conn(0x7f85940ce3b0 0x7f85940ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34870/0 (socket says 172.21.15.42:34870) 2024-05-28T13:59:06.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.265+0000 7f8593e00640 1 -- 172.21.15.42:0/3166257192 >> v1:172.21.15.63:6789/0 conn(0x7f859413fd10 legacy=0x7f8594140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.265+0000 7f8593e00640 1 -- 172.21.15.42:0/3166257192 >> v1:172.21.15.42:6789/0 conn(0x7f85940ce3b0 legacy=0x7f85940ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.265+0000 7f8593e00640 1 -- 172.21.15.42:0/3166257192 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85941607f0 con 0x7f8594071de0 2024-05-28T13:59:06.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.269+0000 7f8593e00640 1 --2- 172.21.15.42:0/3166257192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8594071de0 0x7f85940721c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8584009870 tx=0x7f858401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d663f0521ce0da1a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:06.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.269+0000 7f8593400640 1 -- 172.21.15.42:0/3166257192 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85840b0070 con 0x7f8594071de0 2024-05-28T13:59:06.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.269+0000 7f8593400640 1 -- 172.21.15.42:0/3166257192 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85840ab600 con 0x7f8594071de0 2024-05-28T13:59:06.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.269+0000 7f8593400640 1 -- 172.21.15.42:0/3166257192 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85840ab9d0 con 0x7f8594071de0 2024-05-28T13:59:06.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.269+0000 7f859ae00640 1 -- 172.21.15.42:0/3166257192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8594071de0 msgr2=0x7f85940721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.269+0000 7f859ae00640 1 --2- 172.21.15.42:0/3166257192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8594071de0 0x7f85940721c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8584009870 tx=0x7f858401c440 comp rx=0 tx=0).stop 2024-05-28T13:59:06.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.269+0000 7f859ae00640 1 -- 172.21.15.42:0/3166257192 shutdown_connections 2024-05-28T13:59:06.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.269+0000 7f859ae00640 1 --2- 172.21.15.42:0/3166257192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8594071de0 0x7f85940721c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.269+0000 7f859ae00640 1 -- 172.21.15.42:0/3166257192 >> 172.21.15.42:0/3166257192 conn(0x7f85940b21f0 msgr2=0x7f85940b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:06.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.269+0000 7f859ae00640 1 -- 172.21.15.42:0/3166257192 shutdown_connections 2024-05-28T13:59:06.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.269+0000 7f859ae00640 1 -- 172.21.15.42:0/3166257192 wait complete. 2024-05-28T13:59:06.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.269+0000 7f859ae00640 1 Processor -- start 2024-05-28T13:59:06.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.269+0000 7f859ae00640 1 -- start start 2024-05-28T13:59:06.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f859ae00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8594071de0 0x7f859413c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:06.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f859ae00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85940ce3b0 0x7f8594154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:06.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f859ae00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f859413fd10 0x7f8594155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:06.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f859ae00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f859416a840 con 0x7f85940ce3b0 2024-05-28T13:59:06.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f859ae00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f859416a6c0 con 0x7f8594071de0 2024-05-28T13:59:06.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f859ae00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f859416a9c0 con 0x7f859413fd10 2024-05-28T13:59:06.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f858b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85940ce3b0 0x7f8594154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f858b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85940ce3b0 0x7f8594154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52944/0 (socket says 172.21.15.42:52944) 2024-05-28T13:59:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f858b400640 1 -- 172.21.15.42:0/911099851 learned_addr learned my addr 172.21.15.42:0/911099851 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f8593e00640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8594071de0 0x7f859413c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f8598e00640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f859413fd10 0x7f8594155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f858b400640 1 -- 172.21.15.42:0/911099851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f859413fd10 msgr2=0x7f8594155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f858b400640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f859413fd10 0x7f8594155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f858b400640 1 -- 172.21.15.42:0/911099851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8594071de0 msgr2=0x7f859413c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f858b400640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8594071de0 0x7f859413c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f858b400640 1 -- 172.21.15.42:0/911099851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859415f650 con 0x7f85940ce3b0 2024-05-28T13:59:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f8598e00640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f859413fd10 0x7f8594155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f858b400640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85940ce3b0 0x7f8594154f00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f857c00c490 tx=0x7f857c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f8590c00640 1 -- 172.21.15.42:0/911099851 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c018070 con 0x7f85940ce3b0 2024-05-28T13:59:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f8590c00640 1 -- 172.21.15.42:0/911099851 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857c007480 con 0x7f85940ce3b0 2024-05-28T13:59:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f8590c00640 1 -- 172.21.15.42:0/911099851 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c004a10 con 0x7f85940ce3b0 2024-05-28T13:59:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f859ae00640 1 -- 172.21.15.42:0/911099851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859415f370 con 0x7f85940ce3b0 2024-05-28T13:59:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f859ae00640 1 -- 172.21.15.42:0/911099851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85940b3970 con 0x7f85940ce3b0 2024-05-28T13:59:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f859ae00640 1 -- 172.21.15.42:0/911099851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85940738b0 con 0x7f85940ce3b0 2024-05-28T13:59:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.273+0000 7f8590c00640 1 -- 172.21.15.42:0/911099851 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f857c004bb0 con 0x7f85940ce3b0 2024-05-28T13:59:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.277+0000 7f8590c00640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8570068380 0x7f857006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.277+0000 7f8590c00640 1 -- 172.21.15.42:0/911099851 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f857c085bb0 con 0x7f85940ce3b0 2024-05-28T13:59:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.277+0000 7f8593e00640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8570068380 0x7f857006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:06.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.277+0000 7f8593e00640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8570068380 0x7f857006a840 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f85840ad000 tx=0x7f85840a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:06.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.281+0000 7f8590c00640 1 -- 172.21.15.42:0/911099851 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f857c055b30 con 0x7f85940ce3b0 2024-05-28T13:59:06.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.445+0000 7f859ae00640 1 -- 172.21.15.42:0/911099851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8594072ed0 con 0x7f85940ce3b0 2024-05-28T13:59:06.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.445+0000 7f8590c00640 1 -- 172.21.15.42:0/911099851 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f857c059e30 con 0x7f85940ce3b0 2024-05-28T13:59:06.430 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:59:06.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.445+0000 7f859ae00640 1 -- 172.21.15.42:0/911099851 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8570068380 msgr2=0x7f857006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.445+0000 7f859ae00640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8570068380 0x7f857006a840 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f85840ad000 tx=0x7f85840a6000 comp rx=0 tx=0).stop 2024-05-28T13:59:06.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.445+0000 7f859ae00640 1 -- 172.21.15.42:0/911099851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85940ce3b0 msgr2=0x7f8594154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.445+0000 7f859ae00640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85940ce3b0 0x7f8594154f00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f857c00c490 tx=0x7f857c00c950 comp rx=0 tx=0).stop 2024-05-28T13:59:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.445+0000 7f859ae00640 1 -- 172.21.15.42:0/911099851 shutdown_connections 2024-05-28T13:59:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.445+0000 7f859ae00640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8570068380 0x7f857006a840 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.445+0000 7f859ae00640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f859413fd10 0x7f8594155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.445+0000 7f859ae00640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85940ce3b0 0x7f8594154f00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.445+0000 7f859ae00640 1 --2- 172.21.15.42:0/911099851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8594071de0 0x7f859413c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.445+0000 7f859ae00640 1 -- 172.21.15.42:0/911099851 >> 172.21.15.42:0/911099851 conn(0x7f85940b21f0 msgr2=0x7f85941565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.445+0000 7f859ae00640 1 -- 172.21.15.42:0/911099851 shutdown_connections 2024-05-28T13:59:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.445+0000 7f859ae00640 1 -- 172.21.15.42:0/911099851 wait complete. 2024-05-28T13:59:06.452 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T13:59:06.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.609+0000 7f19c3e00640 1 Processor -- start 2024-05-28T13:59:06.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.609+0000 7f19c3e00640 1 -- start start 2024-05-28T13:59:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.609+0000 7f19c3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f19bc0cf690 0x7f19bc06ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.609+0000 7f19c3e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19bc0cfc10 con 0x7f19bc0cf230 2024-05-28T13:59:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.609+0000 7f19c3e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19bc0cfef0 con 0x7f19bc15bf30 2024-05-28T13:59:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.609+0000 7f19c3e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f19bc0721d0 con 0x7f19bc0cf690 2024-05-28T13:59:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.609+0000 7f19c1e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f19bc0cf690 0x7f19bc06ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.609+0000 7f19c1e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f19bc0cf690 0x7f19bc06ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52948/0 (socket says 172.21.15.42:52948) 2024-05-28T13:59:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.609+0000 7f19c1e00640 1 -- 172.21.15.42:0/1722279975 learned_addr learned my addr 172.21.15.42:0/1722279975 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.609+0000 7f19c0a00640 1 --1- 172.21.15.42:0/1722279975 >> v1:172.21.15.42:6789/0 conn(0x7f19bc0cf230 0x7f19bc072660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34882/0 (socket says 172.21.15.42:34882) 2024-05-28T13:59:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.609+0000 7f19c1e00640 1 -- 172.21.15.42:0/1722279975 >> v1:172.21.15.63:6789/0 conn(0x7f19bc15bf30 legacy=0x7f19bc15c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.609+0000 7f19c1e00640 1 -- 172.21.15.42:0/1722279975 >> v1:172.21.15.42:6789/0 conn(0x7f19bc0cf230 legacy=0x7f19bc072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.609+0000 7f19c1e00640 1 -- 172.21.15.42:0/1722279975 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19bc0d01d0 con 0x7f19bc0cf690 2024-05-28T13:59:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.609+0000 7f19bbe00640 1 -- 172.21.15.42:0/1722279975 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3406711971 0 0) 0x7f19bc0cfc10 con 0x7f19bc0cf230 2024-05-28T13:59:06.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.613+0000 7f19c1e00640 1 --2- 172.21.15.42:0/1722279975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19bc0cf690 0x7f19bc06ec50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009e50 tx=0x7f19b401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acece2a9348bb87 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:06.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.613+0000 7f19bbe00640 1 -- 172.21.15.42:0/1722279975 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b40a5660 con 0x7f19bc0cf690 2024-05-28T13:59:06.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.613+0000 7f19bbe00640 1 -- 172.21.15.42:0/1722279975 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19b40a5800 con 0x7f19bc0cf690 2024-05-28T13:59:06.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.613+0000 7f19bbe00640 1 -- 172.21.15.42:0/1722279975 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b40a5bd0 con 0x7f19bc0cf690 2024-05-28T13:59:06.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.613+0000 7f19c3e00640 1 -- 172.21.15.42:0/1722279975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19bc0cf690 msgr2=0x7f19bc06ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.613+0000 7f19c3e00640 1 --2- 172.21.15.42:0/1722279975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19bc0cf690 0x7f19bc06ec50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009e50 tx=0x7f19b401c9a0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.613+0000 7f19c3e00640 1 -- 172.21.15.42:0/1722279975 shutdown_connections 2024-05-28T13:59:06.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.613+0000 7f19c3e00640 1 --2- 172.21.15.42:0/1722279975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19bc0cf690 0x7f19bc06ec50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.613+0000 7f19c3e00640 1 -- 172.21.15.42:0/1722279975 >> 172.21.15.42:0/1722279975 conn(0x7f19bc06b9f0 msgr2=0x7f19bc06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:06.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.613+0000 7f19c3e00640 1 -- 172.21.15.42:0/1722279975 shutdown_connections 2024-05-28T13:59:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.613+0000 7f19c3e00640 1 -- 172.21.15.42:0/1722279975 wait complete. 2024-05-28T13:59:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c3e00640 1 Processor -- start 2024-05-28T13:59:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c3e00640 1 -- start start 2024-05-28T13:59:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19bc0cf230 0x7f19bc153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19bc0cf690 0x7f19bc13bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19bc15bf30 0x7f19bc151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c3e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc169550 con 0x7f19bc0cf690 2024-05-28T13:59:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c3e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc1693d0 con 0x7f19bc15bf30 2024-05-28T13:59:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c3e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f19bc1696d0 con 0x7f19bc0cf230 2024-05-28T13:59:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19bc0cf690 0x7f19bc13bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19bc0cf690 0x7f19bc13bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52952/0 (socket says 172.21.15.42:52952) 2024-05-28T13:59:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c0a00640 1 -- 172.21.15.42:0/2379992200 learned_addr learned my addr 172.21.15.42:0/2379992200 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c1e00640 1 --2- 172.21.15.42:0/2379992200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19bc15bf30 0x7f19bc151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c1400640 1 --2- 172.21.15.42:0/2379992200 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19bc0cf230 0x7f19bc153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c0a00640 1 -- 172.21.15.42:0/2379992200 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19bc0cf230 msgr2=0x7f19bc153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c0a00640 1 --2- 172.21.15.42:0/2379992200 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19bc0cf230 0x7f19bc153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c0a00640 1 -- 172.21.15.42:0/2379992200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19bc15bf30 msgr2=0x7f19bc151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c0a00640 1 --2- 172.21.15.42:0/2379992200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19bc15bf30 0x7f19bc151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c0a00640 1 -- 172.21.15.42:0/2379992200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19bc153b90 con 0x7f19bc0cf690 2024-05-28T13:59:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c0a00640 1 --2- 172.21.15.42:0/2379992200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19bc0cf690 0x7f19bc13bb10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f19a800c410 tx=0x7f19a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19b9600640 1 -- 172.21.15.42:0/2379992200 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a80074e0 con 0x7f19bc0cf690 2024-05-28T13:59:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19b9600640 1 -- 172.21.15.42:0/2379992200 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a80047f0 con 0x7f19bc0cf690 2024-05-28T13:59:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c3e00640 1 -- 172.21.15.42:0/2379992200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19bc153e70 con 0x7f19bc0cf690 2024-05-28T13:59:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19b9600640 1 -- 172.21.15.42:0/2379992200 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8007680 con 0x7f19bc0cf690 2024-05-28T13:59:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c3e00640 1 -- 172.21.15.42:0/2379992200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19bc154430 con 0x7f19bc0cf690 2024-05-28T13:59:06.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.617+0000 7f19c3e00640 1 -- 172.21.15.42:0/2379992200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19bc073890 con 0x7f19bc0cf690 2024-05-28T13:59:06.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.621+0000 7f19b9600640 1 -- 172.21.15.42:0/2379992200 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f19a8011070 con 0x7f19bc0cf690 2024-05-28T13:59:06.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.621+0000 7f19b9600640 1 --2- 172.21.15.42:0/2379992200 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f199c068380 0x7f199c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:06.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.621+0000 7f19b9600640 1 -- 172.21.15.42:0/2379992200 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f19a8086540 con 0x7f19bc0cf690 2024-05-28T13:59:06.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.621+0000 7f19c1400640 1 --2- 172.21.15.42:0/2379992200 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f199c068380 0x7f199c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:06.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.621+0000 7f19c1400640 1 --2- 172.21.15.42:0/2379992200 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f199c068380 0x7f199c06a840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f19b0007fd0 tx=0x7f19b0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:06.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.625+0000 7f19b9600640 1 -- 172.21.15.42:0/2379992200 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f19a8056420 con 0x7f19bc0cf690 2024-05-28T13:59:06.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.801+0000 7f19c3e00640 1 -- 172.21.15.42:0/2379992200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f19bc04fdd0 con 0x7f19bc0cf690 2024-05-28T13:59:06.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.801+0000 7f19b9600640 1 -- 172.21.15.42:0/2379992200 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f19a805a720 con 0x7f19bc0cf690 2024-05-28T13:59:06.786 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T13:59:06.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.801+0000 7f19c3e00640 1 -- 172.21.15.42:0/2379992200 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f199c068380 msgr2=0x7f199c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.801+0000 7f19c3e00640 1 --2- 172.21.15.42:0/2379992200 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f199c068380 0x7f199c06a840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f19b0007fd0 tx=0x7f19b0009040 comp rx=0 tx=0).stop 2024-05-28T13:59:06.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.801+0000 7f19c3e00640 1 -- 172.21.15.42:0/2379992200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19bc0cf690 msgr2=0x7f19bc13bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:06.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.801+0000 7f19c3e00640 1 --2- 172.21.15.42:0/2379992200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19bc0cf690 0x7f19bc13bb10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f19a800c410 tx=0x7f19a800c8d0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.801+0000 7f19c3e00640 1 -- 172.21.15.42:0/2379992200 shutdown_connections 2024-05-28T13:59:06.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.801+0000 7f19c3e00640 1 --2- 172.21.15.42:0/2379992200 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f199c068380 0x7f199c06a840 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.801+0000 7f19c3e00640 1 --2- 172.21.15.42:0/2379992200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19bc15bf30 0x7f19bc151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.801+0000 7f19c3e00640 1 --2- 172.21.15.42:0/2379992200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19bc0cf690 0x7f19bc13bb10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.801+0000 7f19c3e00640 1 --2- 172.21.15.42:0/2379992200 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19bc0cf230 0x7f19bc153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:06.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.801+0000 7f19c3e00640 1 -- 172.21.15.42:0/2379992200 >> 172.21.15.42:0/2379992200 conn(0x7f19bc06b9f0 msgr2=0x7f19bc0b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:06.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.801+0000 7f19c3e00640 1 -- 172.21.15.42:0/2379992200 shutdown_connections 2024-05-28T13:59:06.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:06.801+0000 7f19c3e00640 1 -- 172.21.15.42:0/2379992200 wait complete. 2024-05-28T13:59:06.810 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:59:06.811 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:59:06.819 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:59:07.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.017+0000 7f0996e00640 1 Processor -- start 2024-05-28T13:59:07.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.021+0000 7f0996e00640 1 -- start start 2024-05-28T13:59:07.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.021+0000 7f0996e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f099015c990 0x7f09900d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.021+0000 7f0996e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09900cd300 con 0x7f09900d06a0 2024-05-28T13:59:07.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.021+0000 7f0996e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09900cd5e0 con 0x7f099009bbd0 2024-05-28T13:59:07.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.021+0000 7f0996e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f099015cd70 con 0x7f099015c990 2024-05-28T13:59:07.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.021+0000 7f098fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f099015c990 0x7f09900d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.021+0000 7f098fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f099015c990 0x7f09900d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52966/0 (socket says 172.21.15.42:52966) 2024-05-28T13:59:07.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.021+0000 7f098fe00640 1 -- 172.21.15.42:0/2581811949 learned_addr learned my addr 172.21.15.42:0/2581811949 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:07.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.021+0000 7f0994e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f09900d06a0 0x7f09900b53e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34896/0 (socket says 172.21.15.42:34896) 2024-05-28T13:59:07.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.021+0000 7f098fe00640 1 -- 172.21.15.42:0/2581811949 >> v1:172.21.15.63:6789/0 conn(0x7f099009bbd0 legacy=0x7f099009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.021+0000 7f098fe00640 1 -- 172.21.15.42:0/2581811949 >> v1:172.21.15.42:6789/0 conn(0x7f09900d06a0 legacy=0x7f09900b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.021+0000 7f098fe00640 1 -- 172.21.15.42:0/2581811949 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09900cd8c0 con 0x7f099015c990 2024-05-28T13:59:07.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.021+0000 7f098f400640 1 -- 172.21.15.42:0/2581811949 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4272094329 0 0) 0x7f09900cd300 con 0x7f09900d06a0 2024-05-28T13:59:07.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.025+0000 7f098fe00640 1 --2- 172.21.15.42:0/2581811949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f099015c990 0x7f09900d0160 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0980009870 tx=0x7f098001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7025e3b9ce7dbd0c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:07.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.025+0000 7f098f400640 1 -- 172.21.15.42:0/2581811949 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09800a95f0 con 0x7f099015c990 2024-05-28T13:59:07.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.025+0000 7f098f400640 1 -- 172.21.15.42:0/2581811949 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09800a9790 con 0x7f099015c990 2024-05-28T13:59:07.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.025+0000 7f098f400640 1 -- 172.21.15.42:0/2581811949 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09800a9b60 con 0x7f099015c990 2024-05-28T13:59:07.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.025+0000 7f0996e00640 1 -- 172.21.15.42:0/2581811949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f099015c990 msgr2=0x7f09900d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.025+0000 7f0996e00640 1 --2- 172.21.15.42:0/2581811949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f099015c990 0x7f09900d0160 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0980009870 tx=0x7f098001c530 comp rx=0 tx=0).stop 2024-05-28T13:59:07.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.025+0000 7f0996e00640 1 -- 172.21.15.42:0/2581811949 shutdown_connections 2024-05-28T13:59:07.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.025+0000 7f0996e00640 1 --2- 172.21.15.42:0/2581811949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f099015c990 0x7f09900d0160 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.025+0000 7f0996e00640 1 -- 172.21.15.42:0/2581811949 >> 172.21.15.42:0/2581811949 conn(0x7f09900b4080 msgr2=0x7f09900b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:07.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.025+0000 7f0996e00640 1 -- 172.21.15.42:0/2581811949 shutdown_connections 2024-05-28T13:59:07.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.025+0000 7f0996e00640 1 -- 172.21.15.42:0/2581811949 wait complete. 2024-05-28T13:59:07.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.025+0000 7f0996e00640 1 Processor -- start 2024-05-28T13:59:07.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.025+0000 7f0996e00640 1 -- start start 2024-05-28T13:59:07.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0996e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09900d06a0 0x7f09900cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0996e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f099015c990 0x7f099015e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0996e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f099006d2b0 0x7f0990156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0996e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f09900ce3a0 con 0x7f099015c990 2024-05-28T13:59:07.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0996e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f099009b160 con 0x7f09900d06a0 2024-05-28T13:59:07.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0996e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f09900cb060 con 0x7f099006d2b0 2024-05-28T13:59:07.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0987e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f099015c990 0x7f099015e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0987e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f099015c990 0x7f099015e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52968/0 (socket says 172.21.15.42:52968) 2024-05-28T13:59:07.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0987e00640 1 -- 172.21.15.42:0/653513054 learned_addr learned my addr 172.21.15.42:0/653513054 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:07.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f098fe00640 1 --2- 172.21.15.42:0/653513054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09900d06a0 0x7f09900cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0994e00640 1 --2- 172.21.15.42:0/653513054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f099006d2b0 0x7f0990156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0987e00640 1 -- 172.21.15.42:0/653513054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f099006d2b0 msgr2=0x7f0990156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0987e00640 1 --2- 172.21.15.42:0/653513054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f099006d2b0 0x7f0990156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0987e00640 1 -- 172.21.15.42:0/653513054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09900d06a0 msgr2=0x7f09900cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0987e00640 1 --2- 172.21.15.42:0/653513054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09900d06a0 0x7f09900cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0987e00640 1 -- 172.21.15.42:0/653513054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09900cb340 con 0x7f099015c990 2024-05-28T13:59:07.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0987e00640 1 --2- 172.21.15.42:0/653513054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f099015c990 0x7f099015e200 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f097800c500 tx=0x7f097800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:07.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f098cc00640 1 -- 172.21.15.42:0/653513054 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0978016020 con 0x7f099015c990 2024-05-28T13:59:07.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f098cc00640 1 -- 172.21.15.42:0/653513054 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0978005220 con 0x7f099015c990 2024-05-28T13:59:07.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f098cc00640 1 -- 172.21.15.42:0/653513054 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0978007030 con 0x7f099015c990 2024-05-28T13:59:07.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0996e00640 1 -- 172.21.15.42:0/653513054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09900cb620 con 0x7f099015c990 2024-05-28T13:59:07.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0996e00640 1 -- 172.21.15.42:0/653513054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09900cbbe0 con 0x7f099015c990 2024-05-28T13:59:07.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f0996e00640 1 -- 172.21.15.42:0/653513054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0990068a90 con 0x7f099015c990 2024-05-28T13:59:07.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.029+0000 7f098cc00640 1 -- 172.21.15.42:0/653513054 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f09780071f0 con 0x7f099015c990 2024-05-28T13:59:07.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.033+0000 7f098cc00640 1 --2- 172.21.15.42:0/653513054 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f096c068380 0x7f096c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.033+0000 7f098cc00640 1 -- 172.21.15.42:0/653513054 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f09780863f0 con 0x7f099015c990 2024-05-28T13:59:07.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.033+0000 7f098fe00640 1 --2- 172.21.15.42:0/653513054 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f096c068380 0x7f096c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.033+0000 7f098fe00640 1 --2- 172.21.15.42:0/653513054 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f096c068380 0x7f096c06a840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f09800ae000 tx=0x7f09800b3040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:07.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.037+0000 7f098cc00640 1 -- 172.21.15.42:0/653513054 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0978020020 con 0x7f099015c990 2024-05-28T13:59:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.201+0000 7f0996e00640 1 -- 172.21.15.42:0/653513054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09900ce520 con 0x7f099015c990 2024-05-28T13:59:07.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.201+0000 7f098cc00640 1 -- 172.21.15.42:0/653513054 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16737 (secure 0 0 0) 0x7f0978056250 con 0x7f099015c990 2024-05-28T13:59:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:07.188 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:59:03.992606+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:59:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.205+0000 7f0996e00640 1 -- 172.21.15.42:0/653513054 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f096c068380 msgr2=0x7f096c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.205+0000 7f0996e00640 1 --2- 172.21.15.42:0/653513054 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f096c068380 0x7f096c06a840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f09800ae000 tx=0x7f09800b3040 comp rx=0 tx=0).stop 2024-05-28T13:59:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.205+0000 7f0996e00640 1 -- 172.21.15.42:0/653513054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f099015c990 msgr2=0x7f099015e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.205+0000 7f0996e00640 1 --2- 172.21.15.42:0/653513054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f099015c990 0x7f099015e200 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f097800c500 tx=0x7f097800c9c0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.205+0000 7f0996e00640 1 -- 172.21.15.42:0/653513054 shutdown_connections 2024-05-28T13:59:07.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.205+0000 7f0996e00640 1 --2- 172.21.15.42:0/653513054 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f096c068380 0x7f096c06a840 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.205+0000 7f0996e00640 1 --2- 172.21.15.42:0/653513054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f099006d2b0 0x7f0990156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.205+0000 7f0996e00640 1 --2- 172.21.15.42:0/653513054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f099015c990 0x7f099015e200 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.205+0000 7f0996e00640 1 --2- 172.21.15.42:0/653513054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09900d06a0 0x7f09900cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.205+0000 7f0996e00640 1 -- 172.21.15.42:0/653513054 >> 172.21.15.42:0/653513054 conn(0x7f09900b4080 msgr2=0x7f09900b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:07.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.205+0000 7f0996e00640 1 -- 172.21.15.42:0/653513054 shutdown_connections 2024-05-28T13:59:07.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.205+0000 7f0996e00640 1 -- 172.21.15.42:0/653513054 wait complete. 2024-05-28T13:59:07.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:59:07.210 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:59:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.369+0000 7ff76b200640 1 Processor -- start 2024-05-28T13:59:07.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.369+0000 7ff76b200640 1 -- start start 2024-05-28T13:59:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.373+0000 7ff76b200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff76c0cfec0 0x7ff76c0d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.373+0000 7ff76b200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff76c143f90 0x7ff76c071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.373+0000 7ff76b200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff76c0d07e0 con 0x7ff76c147a80 2024-05-28T13:59:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.373+0000 7ff76b200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff76c06cb00 con 0x7ff76c0cfec0 2024-05-28T13:59:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.373+0000 7ff76b200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff76c144540 con 0x7ff76c143f90 2024-05-28T13:59:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.373+0000 7ff769e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff76c0cfec0 0x7ff76c0d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.373+0000 7ff76a800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff76c147a80 0x7ff76c071c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46002/0 (socket says 172.21.15.42:46002) 2024-05-28T13:59:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.373+0000 7ff76a800640 1 -- 172.21.15.42:0/2402762132 learned_addr learned my addr 172.21.15.42:0/2402762132 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.373+0000 7ff763e00640 1 --2- 172.21.15.42:0/2402762132 >> v2:172.21.15.63:3300/0 conn(0x7ff76c143f90 0x7ff76c071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.373+0000 7ff769e00640 1 -- 172.21.15.42:0/2402762132 >> v2:172.21.15.63:3300/0 conn(0x7ff76c143f90 msgr2=0x7ff76c071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:07.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.373+0000 7ff769e00640 1 --2- 172.21.15.42:0/2402762132 >> v2:172.21.15.63:3300/0 conn(0x7ff76c143f90 0x7ff76c071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.373+0000 7ff769e00640 1 -- 172.21.15.42:0/2402762132 >> v1:172.21.15.42:6789/0 conn(0x7ff76c147a80 legacy=0x7ff76c071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.373+0000 7ff769e00640 1 -- 172.21.15.42:0/2402762132 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76c144820 con 0x7ff76c0cfec0 2024-05-28T13:59:07.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.373+0000 7ff769e00640 1 --2- 172.21.15.42:0/2402762132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff76c0cfec0 0x7ff76c0d02a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff76c06cc80 tx=0x7ff75c01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7c4f8d6ea5bb6e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:07.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff769400640 1 -- 172.21.15.42:0/2402762132 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c0b0070 con 0x7ff76c0cfec0 2024-05-28T13:59:07.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff769400640 1 -- 172.21.15.42:0/2402762132 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff75c0ab600 con 0x7ff76c0cfec0 2024-05-28T13:59:07.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff769400640 1 -- 172.21.15.42:0/2402762132 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c0ab9d0 con 0x7ff76c0cfec0 2024-05-28T13:59:07.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 -- 172.21.15.42:0/2402762132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff76c0cfec0 msgr2=0x7ff76c0d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 --2- 172.21.15.42:0/2402762132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff76c0cfec0 0x7ff76c0d02a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff76c06cc80 tx=0x7ff75c01c310 comp rx=0 tx=0).stop 2024-05-28T13:59:07.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 -- 172.21.15.42:0/2402762132 shutdown_connections 2024-05-28T13:59:07.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 --2- 172.21.15.42:0/2402762132 >> v2:172.21.15.63:3300/0 conn(0x7ff76c143f90 0x7ff76c071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 --2- 172.21.15.42:0/2402762132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff76c0cfec0 0x7ff76c0d02a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 -- 172.21.15.42:0/2402762132 >> 172.21.15.42:0/2402762132 conn(0x7ff76c06fcc0 msgr2=0x7ff76c0700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:07.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 -- 172.21.15.42:0/2402762132 shutdown_connections 2024-05-28T13:59:07.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 -- 172.21.15.42:0/2402762132 wait complete. 2024-05-28T13:59:07.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 Processor -- start 2024-05-28T13:59:07.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 -- start start 2024-05-28T13:59:07.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff76c0cfec0 0x7ff76c0c3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff76c143f90 0x7ff76c15fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff76c147a80 0x7ff76c157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c161fe0 con 0x7ff76c147a80 2024-05-28T13:59:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c161e60 con 0x7ff76c143f90 2024-05-28T13:59:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff76c072360 con 0x7ff76c0cfec0 2024-05-28T13:59:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff76c147a80 0x7ff76c157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff76c147a80 0x7ff76c157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47842/0 (socket says 172.21.15.42:47842) 2024-05-28T13:59:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff76a800640 1 -- 172.21.15.42:0/2951185911 learned_addr learned my addr 172.21.15.42:0/2951185911 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff769e00640 1 --2- 172.21.15.42:0/2951185911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff76c0cfec0 0x7ff76c0c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.377+0000 7ff763e00640 1 --2- 172.21.15.42:0/2951185911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff76c143f90 0x7ff76c15fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff76a800640 1 -- 172.21.15.42:0/2951185911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff76c0cfec0 msgr2=0x7ff76c0c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff76a800640 1 --2- 172.21.15.42:0/2951185911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff76c0cfec0 0x7ff76c0c3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff76a800640 1 -- 172.21.15.42:0/2951185911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff76c143f90 msgr2=0x7ff76c15fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff76a800640 1 --2- 172.21.15.42:0/2951185911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff76c143f90 0x7ff76c15fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff76a800640 1 -- 172.21.15.42:0/2951185911 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76c0c5570 con 0x7ff76c147a80 2024-05-28T13:59:07.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff76a800640 1 --2- 172.21.15.42:0/2951185911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff76c147a80 0x7ff76c157990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff76400cbc0 tx=0x7ff76400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:07.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff762000640 1 -- 172.21.15.42:0/2951185911 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764017070 con 0x7ff76c147a80 2024-05-28T13:59:07.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff762000640 1 -- 172.21.15.42:0/2951185911 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76400ad40 con 0x7ff76c147a80 2024-05-28T13:59:07.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff762000640 1 -- 172.21.15.42:0/2951185911 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764012600 con 0x7ff76c147a80 2024-05-28T13:59:07.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff76b200640 1 -- 172.21.15.42:0/2951185911 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76c154a20 con 0x7ff76c147a80 2024-05-28T13:59:07.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff76b200640 1 -- 172.21.15.42:0/2951185911 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff76c152840 con 0x7ff76c147a80 2024-05-28T13:59:07.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff76b200640 1 -- 172.21.15.42:0/2951185911 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff76c04fcf0 con 0x7ff76c147a80 2024-05-28T13:59:07.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff762000640 1 -- 172.21.15.42:0/2951185911 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff76401a070 con 0x7ff76c147a80 2024-05-28T13:59:07.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff762000640 1 --2- 172.21.15.42:0/2951185911 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff7480683b0 0x7ff74806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff762000640 1 -- 172.21.15.42:0/2951185911 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff764086700 con 0x7ff76c147a80 2024-05-28T13:59:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.381+0000 7ff769e00640 1 --2- 172.21.15.42:0/2951185911 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff7480683b0 0x7ff74806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.385+0000 7ff769e00640 1 --2- 172.21.15.42:0/2951185911 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff7480683b0 0x7ff74806a870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0ad000 tx=0x7ff75c0a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:07.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.385+0000 7ff762000640 1 -- 172.21.15.42:0/2951185911 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff76c0c5570 con 0x7ff76c147a80 2024-05-28T13:59:07.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.553+0000 7ff76b200640 1 -- 172.21.15.42:0/2951185911 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff76c146730 con 0x7ff76c147a80 2024-05-28T13:59:07.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.553+0000 7ff762000640 1 -- 172.21.15.42:0/2951185911 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7ff764056550 con 0x7ff76c147a80 2024-05-28T13:59:07.540 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:59:07.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.557+0000 7ff76b200640 1 -- 172.21.15.42:0/2951185911 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff7480683b0 msgr2=0x7ff74806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.557+0000 7ff76b200640 1 --2- 172.21.15.42:0/2951185911 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff7480683b0 0x7ff74806a870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0ad000 tx=0x7ff75c0a6000 comp rx=0 tx=0).stop 2024-05-28T13:59:07.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.557+0000 7ff76b200640 1 -- 172.21.15.42:0/2951185911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff76c147a80 msgr2=0x7ff76c157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.557+0000 7ff76b200640 1 --2- 172.21.15.42:0/2951185911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff76c147a80 0x7ff76c157990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff76400cbc0 tx=0x7ff76400a430 comp rx=0 tx=0).stop 2024-05-28T13:59:07.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.557+0000 7ff76b200640 1 -- 172.21.15.42:0/2951185911 shutdown_connections 2024-05-28T13:59:07.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.557+0000 7ff76b200640 1 --2- 172.21.15.42:0/2951185911 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff7480683b0 0x7ff74806a870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.557+0000 7ff76b200640 1 --2- 172.21.15.42:0/2951185911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff76c147a80 0x7ff76c157990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.557+0000 7ff76b200640 1 --2- 172.21.15.42:0/2951185911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff76c143f90 0x7ff76c15fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.557+0000 7ff76b200640 1 --2- 172.21.15.42:0/2951185911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff76c0cfec0 0x7ff76c0c3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.557+0000 7ff76b200640 1 -- 172.21.15.42:0/2951185911 >> 172.21.15.42:0/2951185911 conn(0x7ff76c06fcc0 msgr2=0x7ff76c0cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:07.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.557+0000 7ff76b200640 1 -- 172.21.15.42:0/2951185911 shutdown_connections 2024-05-28T13:59:07.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.557+0000 7ff76b200640 1 -- 172.21.15.42:0/2951185911 wait complete. 2024-05-28T13:59:07.561 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:59:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.721+0000 7fd316a00640 1 Processor -- start 2024-05-28T13:59:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.721+0000 7fd316a00640 1 -- start start 2024-05-28T13:59:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.721+0000 7fd316a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd310071de0 0x7fd3100721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.721+0000 7fd316a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd310072700 con 0x7fd3100ce3b0 2024-05-28T13:59:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.721+0000 7fd316a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd310160230 con 0x7fd31013fd10 2024-05-28T13:59:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.721+0000 7fd316a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd310160510 con 0x7fd310071de0 2024-05-28T13:59:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.721+0000 7fd307400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd3100ce3b0 0x7fd3100ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46008/0 (socket says 172.21.15.42:46008) 2024-05-28T13:59:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.721+0000 7fd307400640 1 -- 172.21.15.42:0/1909036848 learned_addr learned my addr 172.21.15.42:0/1909036848 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.721+0000 7fd30fe00640 1 --2- 172.21.15.42:0/1909036848 >> v2:172.21.15.63:3300/0 conn(0x7fd310071de0 0x7fd3100721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.721+0000 7fd30f400640 1 -- 172.21.15.42:0/1909036848 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4246443711 0 0) 0x7fd310072700 con 0x7fd3100ce3b0 2024-05-28T13:59:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.721+0000 7fd30f400640 1 -- 172.21.15.42:0/1909036848 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2f4003530 con 0x7fd3100ce3b0 2024-05-28T13:59:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.721+0000 7fd30fe00640 1 -- 172.21.15.42:0/1909036848 >> v1:172.21.15.63:6789/0 conn(0x7fd31013fd10 legacy=0x7fd310140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.721+0000 7fd30fe00640 1 -- 172.21.15.42:0/1909036848 >> v1:172.21.15.42:6789/0 conn(0x7fd3100ce3b0 legacy=0x7fd3100ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.721+0000 7fd30fe00640 1 -- 172.21.15.42:0/1909036848 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3101607f0 con 0x7fd310071de0 2024-05-28T13:59:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.725+0000 7fd30fe00640 1 --2- 172.21.15.42:0/1909036848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd310071de0 0x7fd3100721c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd300009d90 tx=0x7fd30001c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcb8bc9495e990d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:07.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.725+0000 7fd30f400640 1 -- 172.21.15.42:0/1909036848 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3000b2070 con 0x7fd310071de0 2024-05-28T13:59:07.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.725+0000 7fd30f400640 1 -- 172.21.15.42:0/1909036848 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3000a5710 con 0x7fd310071de0 2024-05-28T13:59:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.725+0000 7fd30f400640 1 -- 172.21.15.42:0/1909036848 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3000a5ae0 con 0x7fd310071de0 2024-05-28T13:59:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.725+0000 7fd316a00640 1 -- 172.21.15.42:0/1909036848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd310071de0 msgr2=0x7fd3100721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.725+0000 7fd316a00640 1 --2- 172.21.15.42:0/1909036848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd310071de0 0x7fd3100721c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd300009d90 tx=0x7fd30001c850 comp rx=0 tx=0).stop 2024-05-28T13:59:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.725+0000 7fd316a00640 1 -- 172.21.15.42:0/1909036848 shutdown_connections 2024-05-28T13:59:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.725+0000 7fd316a00640 1 --2- 172.21.15.42:0/1909036848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd310071de0 0x7fd3100721c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.725+0000 7fd316a00640 1 -- 172.21.15.42:0/1909036848 >> 172.21.15.42:0/1909036848 conn(0x7fd3100b21f0 msgr2=0x7fd3100b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.725+0000 7fd316a00640 1 -- 172.21.15.42:0/1909036848 shutdown_connections 2024-05-28T13:59:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.725+0000 7fd316a00640 1 -- 172.21.15.42:0/1909036848 wait complete. 2024-05-28T13:59:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd316a00640 1 Processor -- start 2024-05-28T13:59:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd316a00640 1 -- start start 2024-05-28T13:59:07.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd316a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd310071de0 0x7fd3100b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd316a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3100ce3b0 0x7fd310155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd316a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd31013fd10 0x7fd310138400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd316a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd31016a840 con 0x7fd310071de0 2024-05-28T13:59:07.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd316a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd31016a6c0 con 0x7fd31013fd10 2024-05-28T13:59:07.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd316a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd31016a9c0 con 0x7fd3100ce3b0 2024-05-28T13:59:07.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd30fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd310071de0 0x7fd3100b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd30fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd310071de0 0x7fd3100b19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47858/0 (socket says 172.21.15.42:47858) 2024-05-28T13:59:07.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd30fe00640 1 -- 172.21.15.42:0/127082623 learned_addr learned my addr 172.21.15.42:0/127082623 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:07.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd307400640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3100ce3b0 0x7fd310155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd314a00640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd31013fd10 0x7fd310138400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd30fe00640 1 -- 172.21.15.42:0/127082623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3100ce3b0 msgr2=0x7fd310155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd30fe00640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3100ce3b0 0x7fd310155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd30fe00640 1 -- 172.21.15.42:0/127082623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd31013fd10 msgr2=0x7fd310138400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd30fe00640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd31013fd10 0x7fd310138400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd30fe00640 1 -- 172.21.15.42:0/127082623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd310155dc0 con 0x7fd310071de0 2024-05-28T13:59:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd307400640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3100ce3b0 0x7fd310155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd30fe00640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd310071de0 0x7fd3100b19a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd3000b0040 tx=0x7fd3000195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd30cc00640 1 -- 172.21.15.42:0/127082623 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3000bd020 con 0x7fd310071de0 2024-05-28T13:59:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd30cc00640 1 -- 172.21.15.42:0/127082623 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3000b8470 con 0x7fd310071de0 2024-05-28T13:59:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd30cc00640 1 -- 172.21.15.42:0/127082623 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3000b2040 con 0x7fd310071de0 2024-05-28T13:59:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd316a00640 1 -- 172.21.15.42:0/127082623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd310131410 con 0x7fd310071de0 2024-05-28T13:59:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.729+0000 7fd316a00640 1 -- 172.21.15.42:0/127082623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3100b01a0 con 0x7fd310071de0 2024-05-28T13:59:07.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.733+0000 7fd316a00640 1 -- 172.21.15.42:0/127082623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3100738b0 con 0x7fd310071de0 2024-05-28T13:59:07.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.733+0000 7fd30cc00640 1 -- 172.21.15.42:0/127082623 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd3000c0020 con 0x7fd310071de0 2024-05-28T13:59:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.733+0000 7fd30cc00640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd2ec068380 0x7fd2ec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.733+0000 7fd30cc00640 1 -- 172.21.15.42:0/127082623 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fd300121780 con 0x7fd310071de0 2024-05-28T13:59:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.733+0000 7fd307400640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd2ec068380 0x7fd2ec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.733+0000 7fd307400640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd2ec068380 0x7fd2ec06a840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd2f80011f0 tx=0x7fd2f8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:07.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.737+0000 7fd30cc00640 1 -- 172.21.15.42:0/127082623 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd3000c7030 con 0x7fd310071de0 2024-05-28T13:59:07.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.909+0000 7fd316a00640 1 -- 172.21.15.42:0/127082623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd31004fdd0 con 0x7fd310071de0 2024-05-28T13:59:07.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.909+0000 7fd30cc00640 1 -- 172.21.15.42:0/127082623 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fd300016560 con 0x7fd310071de0 2024-05-28T13:59:07.894 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:59:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.909+0000 7fd316a00640 1 -- 172.21.15.42:0/127082623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd2ec068380 msgr2=0x7fd2ec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.909+0000 7fd316a00640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd2ec068380 0x7fd2ec06a840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd2f80011f0 tx=0x7fd2f8009040 comp rx=0 tx=0).stop 2024-05-28T13:59:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.909+0000 7fd316a00640 1 -- 172.21.15.42:0/127082623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd310071de0 msgr2=0x7fd3100b19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.909+0000 7fd316a00640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd310071de0 0x7fd3100b19a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd3000b0040 tx=0x7fd3000195f0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.909+0000 7fd316a00640 1 -- 172.21.15.42:0/127082623 shutdown_connections 2024-05-28T13:59:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.909+0000 7fd316a00640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd2ec068380 0x7fd2ec06a840 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.909+0000 7fd316a00640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd31013fd10 0x7fd310138400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.909+0000 7fd316a00640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3100ce3b0 0x7fd310155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.909+0000 7fd316a00640 1 --2- 172.21.15.42:0/127082623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd310071de0 0x7fd3100b19a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:07.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.909+0000 7fd316a00640 1 -- 172.21.15.42:0/127082623 >> 172.21.15.42:0/127082623 conn(0x7fd3100b21f0 msgr2=0x7fd3101565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:07.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.909+0000 7fd316a00640 1 -- 172.21.15.42:0/127082623 shutdown_connections 2024-05-28T13:59:07.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:07.909+0000 7fd316a00640 1 -- 172.21.15.42:0/127082623 wait complete. 2024-05-28T13:59:07.916 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:59:08.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f741ea00640 1 Processor -- start 2024-05-28T13:59:08.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f741ea00640 1 -- start start 2024-05-28T13:59:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f741ea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f741815e8c0 0x7f74180b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f741ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f741806c100 0x7f74180af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f741ea00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74180b4440 con 0x7f74180b0650 2024-05-28T13:59:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f741ea00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f74180ae420 con 0x7f741806c100 2024-05-28T13:59:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f741ea00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f74180b0f10 con 0x7f741815e8c0 2024-05-28T13:59:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f741ca00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f741806c100 0x7f74180af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f7417400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f74180b0650 0x7f741806b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46018/0 (socket says 172.21.15.42:46018) 2024-05-28T13:59:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f7417400640 1 -- 172.21.15.42:0/1417351234 learned_addr learned my addr 172.21.15.42:0/1417351234 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f7417e00640 1 --2- 172.21.15.42:0/1417351234 >> v2:172.21.15.63:3300/0 conn(0x7f741815e8c0 0x7f74180b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f741ca00640 1 -- 172.21.15.42:0/1417351234 >> v2:172.21.15.63:3300/0 conn(0x7f741815e8c0 msgr2=0x7f74180b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f741ca00640 1 --2- 172.21.15.42:0/1417351234 >> v2:172.21.15.63:3300/0 conn(0x7f741815e8c0 0x7f74180b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f741ca00640 1 -- 172.21.15.42:0/1417351234 >> v1:172.21.15.42:6789/0 conn(0x7f74180b0650 legacy=0x7f741806b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f741ca00640 1 -- 172.21.15.42:0/1417351234 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74180ae670 con 0x7f741806c100 2024-05-28T13:59:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.077+0000 7f7417e00640 1 --2- 172.21.15.42:0/1417351234 >> v2:172.21.15.63:3300/0 conn(0x7f741815e8c0 0x7f74180b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.081+0000 7f741ca00640 1 --2- 172.21.15.42:0/1417351234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741806c100 0x7f74180af150 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f74180b41b0 tx=0x7f741001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa371549e3d3c12e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.081+0000 7f7416a00640 1 -- 172.21.15.42:0/1417351234 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74100b2070 con 0x7f741806c100 2024-05-28T13:59:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.081+0000 7f7416a00640 1 -- 172.21.15.42:0/1417351234 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74100a5930 con 0x7f741806c100 2024-05-28T13:59:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.081+0000 7f7416a00640 1 -- 172.21.15.42:0/1417351234 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74100a5d00 con 0x7f741806c100 2024-05-28T13:59:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.081+0000 7f741ea00640 1 -- 172.21.15.42:0/1417351234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741806c100 msgr2=0x7f74180af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.081+0000 7f741ea00640 1 --2- 172.21.15.42:0/1417351234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741806c100 0x7f74180af150 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f74180b41b0 tx=0x7f741001c870 comp rx=0 tx=0).stop 2024-05-28T13:59:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.081+0000 7f741ea00640 1 -- 172.21.15.42:0/1417351234 shutdown_connections 2024-05-28T13:59:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.081+0000 7f741ea00640 1 --2- 172.21.15.42:0/1417351234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741806c100 0x7f74180af150 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.081+0000 7f741ea00640 1 --2- 172.21.15.42:0/1417351234 >> v2:172.21.15.63:3300/0 conn(0x7f741815e8c0 0x7f74180b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.081+0000 7f741ea00640 1 -- 172.21.15.42:0/1417351234 >> 172.21.15.42:0/1417351234 conn(0x7f74180b6960 msgr2=0x7f74180b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.081+0000 7f741ea00640 1 -- 172.21.15.42:0/1417351234 shutdown_connections 2024-05-28T13:59:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.081+0000 7f741ea00640 1 -- 172.21.15.42:0/1417351234 wait complete. 2024-05-28T13:59:08.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ea00640 1 Processor -- start 2024-05-28T13:59:08.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ea00640 1 -- start start 2024-05-28T13:59:08.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741806c100 0x7f741814e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74180b0650 0x7f741814fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741815e8c0 0x7f7418150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f74180b0020 con 0x7f741815e8c0 2024-05-28T13:59:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f74180afea0 con 0x7f74180b0650 2024-05-28T13:59:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ea00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f74180b01a0 con 0x7f741806c100 2024-05-28T13:59:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741815e8c0 0x7f7418150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741815e8c0 0x7f7418150090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47882/0 (socket says 172.21.15.42:47882) 2024-05-28T13:59:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ca00640 1 -- 172.21.15.42:0/2735990190 learned_addr learned my addr 172.21.15.42:0/2735990190 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f7417e00640 1 --2- 172.21.15.42:0/2735990190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741806c100 0x7f741814e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f7417400640 1 --2- 172.21.15.42:0/2735990190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74180b0650 0x7f741814fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ca00640 1 -- 172.21.15.42:0/2735990190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741806c100 msgr2=0x7f741814e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ca00640 1 --2- 172.21.15.42:0/2735990190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741806c100 0x7f741814e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ca00640 1 -- 172.21.15.42:0/2735990190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74180b0650 msgr2=0x7f741814fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ca00640 1 --2- 172.21.15.42:0/2735990190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74180b0650 0x7f741814fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ca00640 1 -- 172.21.15.42:0/2735990190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7418159b60 con 0x7f741815e8c0 2024-05-28T13:59:08.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ca00640 1 --2- 172.21.15.42:0/2735990190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741815e8c0 0x7f7418150090 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f74100b0040 tx=0x7f7410019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:08.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f740fe00640 1 -- 172.21.15.42:0/2735990190 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74100b2050 con 0x7f741815e8c0 2024-05-28T13:59:08.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f740fe00640 1 -- 172.21.15.42:0/2735990190 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74100be070 con 0x7f741815e8c0 2024-05-28T13:59:08.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ea00640 1 -- 172.21.15.42:0/2735990190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7418151470 con 0x7f741815e8c0 2024-05-28T13:59:08.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f740fe00640 1 -- 172.21.15.42:0/2735990190 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74100ada70 con 0x7f741815e8c0 2024-05-28T13:59:08.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ea00640 1 -- 172.21.15.42:0/2735990190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7418151a30 con 0x7f741815e8c0 2024-05-28T13:59:08.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.085+0000 7f741ea00640 1 -- 172.21.15.42:0/2735990190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f741809f7a0 con 0x7f741815e8c0 2024-05-28T13:59:08.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.089+0000 7f740fe00640 1 -- 172.21.15.42:0/2735990190 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f74100adc10 con 0x7f741815e8c0 2024-05-28T13:59:08.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.089+0000 7f740fe00640 1 --2- 172.21.15.42:0/2735990190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73f8068380 0x7f73f806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.089+0000 7f740fe00640 1 -- 172.21.15.42:0/2735990190 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f7410121b70 con 0x7f741815e8c0 2024-05-28T13:59:08.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.089+0000 7f7417e00640 1 --2- 172.21.15.42:0/2735990190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73f8068380 0x7f73f806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.089+0000 7f7417e00640 1 --2- 172.21.15.42:0/2735990190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73f8068380 0x7f73f806a840 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7418150d10 tx=0x7f7408008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:08.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.093+0000 7f740fe00640 1 -- 172.21.15.42:0/2735990190 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7410016520 con 0x7f741815e8c0 2024-05-28T13:59:08.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.257+0000 7f741ea00640 1 -- 172.21.15.42:0/2735990190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f741804fdd0 con 0x7f741815e8c0 2024-05-28T13:59:08.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.257+0000 7f740fe00640 1 -- 172.21.15.42:0/2735990190 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f74100f1a70 con 0x7f741815e8c0 2024-05-28T13:59:08.243 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:59:08.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.261+0000 7f741ea00640 1 -- 172.21.15.42:0/2735990190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73f8068380 msgr2=0x7f73f806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.261+0000 7f741ea00640 1 --2- 172.21.15.42:0/2735990190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73f8068380 0x7f73f806a840 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7418150d10 tx=0x7f7408008040 comp rx=0 tx=0).stop 2024-05-28T13:59:08.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.261+0000 7f741ea00640 1 -- 172.21.15.42:0/2735990190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741815e8c0 msgr2=0x7f7418150090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.261+0000 7f741ea00640 1 --2- 172.21.15.42:0/2735990190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741815e8c0 0x7f7418150090 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f74100b0040 tx=0x7f7410019610 comp rx=0 tx=0).stop 2024-05-28T13:59:08.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.261+0000 7f741ea00640 1 -- 172.21.15.42:0/2735990190 shutdown_connections 2024-05-28T13:59:08.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.261+0000 7f741ea00640 1 --2- 172.21.15.42:0/2735990190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73f8068380 0x7f73f806a840 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.261+0000 7f741ea00640 1 --2- 172.21.15.42:0/2735990190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741815e8c0 0x7f7418150090 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.261+0000 7f741ea00640 1 --2- 172.21.15.42:0/2735990190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74180b0650 0x7f741814fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.261+0000 7f741ea00640 1 --2- 172.21.15.42:0/2735990190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741806c100 0x7f741814e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.261+0000 7f741ea00640 1 -- 172.21.15.42:0/2735990190 >> 172.21.15.42:0/2735990190 conn(0x7f74180b6960 msgr2=0x7f741815ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:08.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.261+0000 7f741ea00640 1 -- 172.21.15.42:0/2735990190 shutdown_connections 2024-05-28T13:59:08.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.261+0000 7f741ea00640 1 -- 172.21.15.42:0/2735990190 wait complete. 2024-05-28T13:59:08.265 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T13:59:08.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.421+0000 7f7d91e00640 1 Processor -- start 2024-05-28T13:59:08.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.421+0000 7f7d91e00640 1 -- start start 2024-05-28T13:59:08.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.421+0000 7f7d91e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7d8c15c7a0 0x7f7d8c0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.421+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d8c0d0070 0x7f7d8c09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.421+0000 7f7d91e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7d8c09b740 0x7f7d8c09af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.421+0000 7f7d91e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f7d8c15cb80 con 0x7f7d8c09b740 2024-05-28T13:59:08.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.421+0000 7f7d91e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7d8c160fa0 con 0x7f7d8c15c7a0 2024-05-28T13:59:08.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.421+0000 7f7d91e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8c0cd300 con 0x7f7d8c0d0070 2024-05-28T13:59:08.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.421+0000 7f7d8b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7d8c09b740 0x7f7d8c09af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.421+0000 7f7d8b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7d8c09b740 0x7f7d8c09af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47886/0 (socket says 172.21.15.42:47886) 2024-05-28T13:59:08.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.421+0000 7f7d8b400640 1 -- 172.21.15.42:0/1220837535 learned_addr learned my addr 172.21.15.42:0/1220837535 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:08.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.421+0000 7f7d8aa00640 1 --2- 172.21.15.42:0/1220837535 >> v2:172.21.15.63:3300/0 conn(0x7f7d8c15c7a0 0x7f7d8c0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.421+0000 7f7d82000640 1 --2- 172.21.15.42:0/1220837535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d8c0d0070 0x7f7d8c09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.425+0000 7f7d8b400640 1 -- 172.21.15.42:0/1220837535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d8c0d0070 msgr2=0x7f7d8c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:08.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.425+0000 7f7d8b400640 1 --2- 172.21.15.42:0/1220837535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d8c0d0070 0x7f7d8c09aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.425+0000 7f7d8b400640 1 -- 172.21.15.42:0/1220837535 >> v2:172.21.15.63:3300/0 conn(0x7f7d8c15c7a0 msgr2=0x7f7d8c0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:08.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.425+0000 7f7d8b400640 1 --2- 172.21.15.42:0/1220837535 >> v2:172.21.15.63:3300/0 conn(0x7f7d8c15c7a0 0x7f7d8c0cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.425+0000 7f7d8b400640 1 -- 172.21.15.42:0/1220837535 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c0cd5e0 con 0x7f7d8c09b740 2024-05-28T13:59:08.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.425+0000 7f7d82000640 1 --2- 172.21.15.42:0/1220837535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d8c0d0070 0x7f7d8c09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:08.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.425+0000 7f7d8b400640 1 --2- 172.21.15.42:0/1220837535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d8c09b740 0x7f7d8c09af90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c09b710 tx=0x7f7d8401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1463e15b015b5be server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:08.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.425+0000 7f7d8a000640 1 -- 172.21.15.42:0/1220837535 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840aa430 con 0x7f7d8c09b740 2024-05-28T13:59:08.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.425+0000 7f7d8a000640 1 -- 172.21.15.42:0/1220837535 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d840af070 con 0x7f7d8c09b740 2024-05-28T13:59:08.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.425+0000 7f7d8a000640 1 -- 172.21.15.42:0/1220837535 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840aa7f0 con 0x7f7d8c09b740 2024-05-28T13:59:08.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.425+0000 7f7d91e00640 1 -- 172.21.15.42:0/1220837535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d8c09b740 msgr2=0x7f7d8c09af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.425+0000 7f7d91e00640 1 --2- 172.21.15.42:0/1220837535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d8c09b740 0x7f7d8c09af90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c09b710 tx=0x7f7d8401c4f0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 -- 172.21.15.42:0/1220837535 shutdown_connections 2024-05-28T13:59:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 --2- 172.21.15.42:0/1220837535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d8c09b740 0x7f7d8c09af90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 --2- 172.21.15.42:0/1220837535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d8c0d0070 0x7f7d8c09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 --2- 172.21.15.42:0/1220837535 >> v2:172.21.15.63:3300/0 conn(0x7f7d8c15c7a0 0x7f7d8c0cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 -- 172.21.15.42:0/1220837535 >> 172.21.15.42:0/1220837535 conn(0x7f7d8c0b3f90 msgr2=0x7f7d8c0b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 -- 172.21.15.42:0/1220837535 shutdown_connections 2024-05-28T13:59:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 -- 172.21.15.42:0/1220837535 wait complete. 2024-05-28T13:59:08.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 Processor -- start 2024-05-28T13:59:08.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 -- start start 2024-05-28T13:59:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d8c09b740 0x7f7d8c154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d8c0d0070 0x7f7d8c159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d8c15c7a0 0x7f7d8c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c160d50 con 0x7f7d8c09b740 2024-05-28T13:59:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c160bd0 con 0x7f7d8c15c7a0 2024-05-28T13:59:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d91e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8c0c6800 con 0x7f7d8c0d0070 2024-05-28T13:59:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d8aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d8c09b740 0x7f7d8c154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d8aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d8c09b740 0x7f7d8c154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47900/0 (socket says 172.21.15.42:47900) 2024-05-28T13:59:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d8aa00640 1 -- 172.21.15.42:0/3682181098 learned_addr learned my addr 172.21.15.42:0/3682181098 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d82000640 1 --2- 172.21.15.42:0/3682181098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d8c0d0070 0x7f7d8c159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d8b400640 1 --2- 172.21.15.42:0/3682181098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d8c15c7a0 0x7f7d8c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d8aa00640 1 -- 172.21.15.42:0/3682181098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d8c0d0070 msgr2=0x7f7d8c159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d8aa00640 1 --2- 172.21.15.42:0/3682181098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d8c0d0070 0x7f7d8c159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d8aa00640 1 -- 172.21.15.42:0/3682181098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d8c15c7a0 msgr2=0x7f7d8c071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d8aa00640 1 --2- 172.21.15.42:0/3682181098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d8c15c7a0 0x7f7d8c071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d8aa00640 1 -- 172.21.15.42:0/3682181098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c0c9910 con 0x7f7d8c09b740 2024-05-28T13:59:08.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d8aa00640 1 --2- 172.21.15.42:0/3682181098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d8c09b740 0x7f7d8c154b30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009d10 tx=0x7f7d7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.429+0000 7f7d83e00640 1 -- 172.21.15.42:0/3682181098 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c018020 con 0x7f7d8c09b740 2024-05-28T13:59:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.433+0000 7f7d83e00640 1 -- 172.21.15.42:0/3682181098 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7c014480 con 0x7f7d8c09b740 2024-05-28T13:59:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.433+0000 7f7d83e00640 1 -- 172.21.15.42:0/3682181098 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c014830 con 0x7f7d8c09b740 2024-05-28T13:59:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.433+0000 7f7d91e00640 1 -- 172.21.15.42:0/3682181098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8c159e50 con 0x7f7d8c09b740 2024-05-28T13:59:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.433+0000 7f7d91e00640 1 -- 172.21.15.42:0/3682181098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8c15a3d0 con 0x7f7d8c09b740 2024-05-28T13:59:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.433+0000 7f7d91e00640 1 -- 172.21.15.42:0/3682181098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d8c0a0550 con 0x7f7d8c09b740 2024-05-28T13:59:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.433+0000 7f7d83e00640 1 -- 172.21.15.42:0/3682181098 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7d7c01b020 con 0x7f7d8c09b740 2024-05-28T13:59:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.433+0000 7f7d83e00640 1 --2- 172.21.15.42:0/3682181098 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7d68068380 0x7f7d6806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.433+0000 7f7d83e00640 1 -- 172.21.15.42:0/3682181098 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f7d7c0872d0 con 0x7f7d8c09b740 2024-05-28T13:59:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.433+0000 7f7d82000640 1 --2- 172.21.15.42:0/3682181098 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7d68068380 0x7f7d6806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.437+0000 7f7d82000640 1 --2- 172.21.15.42:0/3682181098 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7d68068380 0x7f7d6806a840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0a04d0 tx=0x7f7d74008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:08.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.437+0000 7f7d83e00640 1 -- 172.21.15.42:0/3682181098 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7d7c022070 con 0x7f7d8c09b740 2024-05-28T13:59:08.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.613+0000 7f7d91e00640 1 -- 172.21.15.42:0/3682181098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7d8c04fdd0 con 0x7f7d8c09b740 2024-05-28T13:59:08.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.613+0000 7f7d83e00640 1 -- 172.21.15.42:0/3682181098 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f7d7c010040 con 0x7f7d8c09b740 2024-05-28T13:59:08.599 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T13:59:08.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.617+0000 7f7d91e00640 1 -- 172.21.15.42:0/3682181098 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7d68068380 msgr2=0x7f7d6806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.617+0000 7f7d91e00640 1 --2- 172.21.15.42:0/3682181098 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7d68068380 0x7f7d6806a840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0a04d0 tx=0x7f7d74008040 comp rx=0 tx=0).stop 2024-05-28T13:59:08.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.617+0000 7f7d91e00640 1 -- 172.21.15.42:0/3682181098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d8c09b740 msgr2=0x7f7d8c154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.617+0000 7f7d91e00640 1 --2- 172.21.15.42:0/3682181098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d8c09b740 0x7f7d8c154b30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009d10 tx=0x7f7d7c00c490 comp rx=0 tx=0).stop 2024-05-28T13:59:08.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.617+0000 7f7d91e00640 1 -- 172.21.15.42:0/3682181098 shutdown_connections 2024-05-28T13:59:08.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.617+0000 7f7d91e00640 1 --2- 172.21.15.42:0/3682181098 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7d68068380 0x7f7d6806a840 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.617+0000 7f7d91e00640 1 --2- 172.21.15.42:0/3682181098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d8c15c7a0 0x7f7d8c071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.617+0000 7f7d91e00640 1 --2- 172.21.15.42:0/3682181098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d8c0d0070 0x7f7d8c159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.617+0000 7f7d91e00640 1 --2- 172.21.15.42:0/3682181098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d8c09b740 0x7f7d8c154b30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.617+0000 7f7d91e00640 1 -- 172.21.15.42:0/3682181098 >> 172.21.15.42:0/3682181098 conn(0x7f7d8c0b3f90 msgr2=0x7f7d8c0c6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:08.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.617+0000 7f7d91e00640 1 -- 172.21.15.42:0/3682181098 shutdown_connections 2024-05-28T13:59:08.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.617+0000 7f7d91e00640 1 -- 172.21.15.42:0/3682181098 wait complete. 2024-05-28T13:59:08.621 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-05-28T13:59:08.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:08.621 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-05-28T13:59:08.642 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T13:59:08.643 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T13:59:08.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:08.643 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-28T13:59:08.697 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:59:08.697 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:59:08.706 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:59:08.706 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:59:08.763 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:59:08.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.917+0000 7f679ba00640 1 Processor -- start 2024-05-28T13:59:08.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.917+0000 7f679ba00640 1 -- start start 2024-05-28T13:59:08.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.921+0000 7f679ba00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f67940d0e00 0x7f67941663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.921+0000 7f679ba00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f679415f280 con 0x7f679415ba40 2024-05-28T13:59:08.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.921+0000 7f679ba00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f679415f560 con 0x7f679406f660 2024-05-28T13:59:08.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.921+0000 7f679ba00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f679415f030 con 0x7f67940d0e00 2024-05-28T13:59:08.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.921+0000 7f6799a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f67940d0e00 0x7f67941663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.921+0000 7f6793e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f679415ba40 0x7f67940d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46022/0 (socket says 172.21.15.42:46022) 2024-05-28T13:59:08.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.921+0000 7f6793e00640 1 -- 172.21.15.42:0/1674839005 learned_addr learned my addr 172.21.15.42:0/1674839005 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:08.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.921+0000 7f6799a00640 1 --2- 172.21.15.42:0/1674839005 >> v2:172.21.15.42:3300/0 conn(0x7f67940d0e00 0x7f67941663c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47904/0 (socket says 172.21.15.42:47904) 2024-05-28T13:59:08.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.921+0000 7f6799a00640 1 -- 172.21.15.42:0/1674839005 >> v1:172.21.15.63:6789/0 conn(0x7f679406f660 legacy=0x7f6794073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.921+0000 7f6799a00640 1 -- 172.21.15.42:0/1674839005 >> v1:172.21.15.42:6789/0 conn(0x7f679415ba40 legacy=0x7f67940d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.921+0000 7f6799a00640 1 -- 172.21.15.42:0/1674839005 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679415f840 con 0x7f67940d0e00 2024-05-28T13:59:08.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.921+0000 7f6793400640 1 -- 172.21.15.42:0/1674839005 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1130755013 0 0) 0x7f679415f280 con 0x7f679415ba40 2024-05-28T13:59:08.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.925+0000 7f6799a00640 1 --2- 172.21.15.42:0/1674839005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67940d0e00 0x7f67941663c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f678c00bfa0 tx=0x7f678c01ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf282fac123c1b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:08.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.925+0000 7f6793400640 1 -- 172.21.15.42:0/1674839005 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c00b2d0 con 0x7f67940d0e00 2024-05-28T13:59:08.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.925+0000 7f6793400640 1 -- 172.21.15.42:0/1674839005 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f678c0a7810 con 0x7f67940d0e00 2024-05-28T13:59:08.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.925+0000 7f6793400640 1 -- 172.21.15.42:0/1674839005 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c0a7be0 con 0x7f67940d0e00 2024-05-28T13:59:08.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.925+0000 7f679ba00640 1 -- 172.21.15.42:0/1674839005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67940d0e00 msgr2=0x7f67941663c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.925+0000 7f679ba00640 1 --2- 172.21.15.42:0/1674839005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67940d0e00 0x7f67941663c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f678c00bfa0 tx=0x7f678c01ea20 comp rx=0 tx=0).stop 2024-05-28T13:59:08.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.925+0000 7f679ba00640 1 -- 172.21.15.42:0/1674839005 shutdown_connections 2024-05-28T13:59:08.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.925+0000 7f679ba00640 1 --2- 172.21.15.42:0/1674839005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67940d0e00 0x7f67941663c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.925+0000 7f679ba00640 1 -- 172.21.15.42:0/1674839005 >> 172.21.15.42:0/1674839005 conn(0x7f679406bdf0 msgr2=0x7f679406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:08.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.925+0000 7f679ba00640 1 -- 172.21.15.42:0/1674839005 shutdown_connections 2024-05-28T13:59:08.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.925+0000 7f679ba00640 1 -- 172.21.15.42:0/1674839005 wait complete. 2024-05-28T13:59:08.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.925+0000 7f679ba00640 1 Processor -- start 2024-05-28T13:59:08.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.925+0000 7f679ba00640 1 -- start start 2024-05-28T13:59:08.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.925+0000 7f679ba00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f679406f660 0x7f67941f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f679ba00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67940d0e00 0x7f67941f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f679ba00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67940cf9d0 0x7f679415a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f679ba00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f67940b3ec0 con 0x7f67940d0e00 2024-05-28T13:59:08.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f679ba00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f67940b3d40 con 0x7f679406f660 2024-05-28T13:59:08.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f679ba00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f67940b4040 con 0x7f67940cf9d0 2024-05-28T13:59:08.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6793e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67940d0e00 0x7f67941f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6793e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67940d0e00 0x7f67941f7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47910/0 (socket says 172.21.15.42:47910) 2024-05-28T13:59:08.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6793e00640 1 -- 172.21.15.42:0/1569848987 learned_addr learned my addr 172.21.15.42:0/1569848987 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:08.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6799000640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f679406f660 0x7f67941f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6799a00640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67940cf9d0 0x7f679415a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6793e00640 1 -- 172.21.15.42:0/1569848987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67940cf9d0 msgr2=0x7f679415a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6793e00640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67940cf9d0 0x7f679415a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6793e00640 1 -- 172.21.15.42:0/1569848987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f679406f660 msgr2=0x7f67941f6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:08.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6793e00640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f679406f660 0x7f67941f6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:08.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6793e00640 1 -- 172.21.15.42:0/1569848987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67941f84e0 con 0x7f67940d0e00 2024-05-28T13:59:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6799000640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f679406f660 0x7f67941f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6793e00640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67940d0e00 0x7f67941f7fa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f678000c4a0 tx=0x7f678000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6790c00640 1 -- 172.21.15.42:0/1569848987 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780018070 con 0x7f67940d0e00 2024-05-28T13:59:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6790c00640 1 -- 172.21.15.42:0/1569848987 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67800074e0 con 0x7f67940d0e00 2024-05-28T13:59:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f679ba00640 1 -- 172.21.15.42:0/1569848987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6794149340 con 0x7f67940d0e00 2024-05-28T13:59:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6790c00640 1 -- 172.21.15.42:0/1569848987 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780004a30 con 0x7f67940d0e00 2024-05-28T13:59:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f679ba00640 1 -- 172.21.15.42:0/1569848987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6794149900 con 0x7f67940d0e00 2024-05-28T13:59:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f679ba00640 1 -- 172.21.15.42:0/1569848987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6794068a90 con 0x7f67940d0e00 2024-05-28T13:59:08.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.929+0000 7f6790c00640 1 -- 172.21.15.42:0/1569848987 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f678001b070 con 0x7f67940d0e00 2024-05-28T13:59:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.933+0000 7f6790c00640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6774068380 0x7f677406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.933+0000 7f6790c00640 1 -- 172.21.15.42:0/1569848987 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f6780085ca0 con 0x7f67940d0e00 2024-05-28T13:59:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.933+0000 7f6799000640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6774068380 0x7f677406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.933+0000 7f6799000640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6774068380 0x7f677406a840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f67940af560 tx=0x7f6788009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:08.937+0000 7f6790c00640 1 -- 172.21.15.42:0/1569848987 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f67941f84e0 con 0x7f67940d0e00 2024-05-28T13:59:09.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.101+0000 7f679ba00640 1 -- 172.21.15.42:0/1569848987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6794073960 con 0x7f67940d0e00 2024-05-28T13:59:09.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.101+0000 7f6790c00640 1 -- 172.21.15.42:0/1569848987 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16737 (secure 0 0 0) 0x7f6780055b80 con 0x7f67940d0e00 2024-05-28T13:59:09.087 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:09.087 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:59:03.992606+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:59:09.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.105+0000 7f679ba00640 1 -- 172.21.15.42:0/1569848987 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6774068380 msgr2=0x7f677406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.105+0000 7f679ba00640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6774068380 0x7f677406a840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f67940af560 tx=0x7f6788009040 comp rx=0 tx=0).stop 2024-05-28T13:59:09.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.105+0000 7f679ba00640 1 -- 172.21.15.42:0/1569848987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67940d0e00 msgr2=0x7f67941f7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.105+0000 7f679ba00640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67940d0e00 0x7f67941f7fa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f678000c4a0 tx=0x7f678000c960 comp rx=0 tx=0).stop 2024-05-28T13:59:09.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.105+0000 7f679ba00640 1 -- 172.21.15.42:0/1569848987 shutdown_connections 2024-05-28T13:59:09.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.105+0000 7f679ba00640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6774068380 0x7f677406a840 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.105+0000 7f679ba00640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67940cf9d0 0x7f679415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.105+0000 7f679ba00640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67940d0e00 0x7f67941f7fa0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.105+0000 7f679ba00640 1 --2- 172.21.15.42:0/1569848987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f679406f660 0x7f67941f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.105+0000 7f679ba00640 1 -- 172.21.15.42:0/1569848987 >> 172.21.15.42:0/1569848987 conn(0x7f679406bdf0 msgr2=0x7f679406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:09.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.105+0000 7f679ba00640 1 -- 172.21.15.42:0/1569848987 shutdown_connections 2024-05-28T13:59:09.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.105+0000 7f679ba00640 1 -- 172.21.15.42:0/1569848987 wait complete. 2024-05-28T13:59:09.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:59:09.109 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:59:09.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.265+0000 7fcd5c600640 1 Processor -- start 2024-05-28T13:59:09.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.269+0000 7fcd5c600640 1 -- start start 2024-05-28T13:59:09.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.269+0000 7fcd5c600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcd540b0e80 0x7fcd5406cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.269+0000 7fcd5c600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5409b160 0x7fcd5415e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.269+0000 7fcd5c600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd540d0770 con 0x7fcd5409b8e0 2024-05-28T13:59:09.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.269+0000 7fcd5c600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcd540d0a10 con 0x7fcd540b0e80 2024-05-28T13:59:09.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.269+0000 7fcd5c600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcd540d0cf0 con 0x7fcd5409b160 2024-05-28T13:59:09.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.269+0000 7fcd59c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcd540b0e80 0x7fcd5406cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.269+0000 7fcd5a600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5409b160 0x7fcd5415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.269+0000 7fcd59c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcd540b0e80 0x7fcd5406cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:60334/0 (socket says 172.21.15.42:60334) 2024-05-28T13:59:09.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.269+0000 7fcd59c00640 1 -- 172.21.15.42:0/3154869377 learned_addr learned my addr 172.21.15.42:0/3154869377 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:09.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.269+0000 7fcd59c00640 1 -- 172.21.15.42:0/3154869377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5409b160 msgr2=0x7fcd5415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:09.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.269+0000 7fcd59c00640 1 --2- 172.21.15.42:0/3154869377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5409b160 0x7fcd5415e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.269+0000 7fcd59c00640 1 -- 172.21.15.42:0/3154869377 >> v1:172.21.15.63:6789/0 conn(0x7fcd5409b8e0 legacy=0x7fcd5409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.269+0000 7fcd59c00640 1 -- 172.21.15.42:0/3154869377 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd540d0fd0 con 0x7fcd540b0e80 2024-05-28T13:59:09.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.273+0000 7fcd59c00640 1 --2- 172.21.15.42:0/3154869377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd540b0e80 0x7fcd5406cb00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009870 tx=0x7fcd4801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96b1e40e9721bdcf server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:09.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.273+0000 7fcd53e00640 1 -- 172.21.15.42:0/3154869377 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd480b2070 con 0x7fcd540b0e80 2024-05-28T13:59:09.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.273+0000 7fcd53e00640 1 -- 172.21.15.42:0/3154869377 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd480a9070 con 0x7fcd540b0e80 2024-05-28T13:59:09.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.273+0000 7fcd53e00640 1 -- 172.21.15.42:0/3154869377 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd480ad880 con 0x7fcd540b0e80 2024-05-28T13:59:09.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.273+0000 7fcd5c600640 1 -- 172.21.15.42:0/3154869377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd540b0e80 msgr2=0x7fcd5406cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.273+0000 7fcd5c600640 1 --2- 172.21.15.42:0/3154869377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd540b0e80 0x7fcd5406cb00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009870 tx=0x7fcd4801c560 comp rx=0 tx=0).stop 2024-05-28T13:59:09.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.273+0000 7fcd5c600640 1 -- 172.21.15.42:0/3154869377 shutdown_connections 2024-05-28T13:59:09.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.273+0000 7fcd5c600640 1 --2- 172.21.15.42:0/3154869377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5409b160 0x7fcd5415e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.273+0000 7fcd5c600640 1 --2- 172.21.15.42:0/3154869377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd540b0e80 0x7fcd5406cb00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.273+0000 7fcd5c600640 1 -- 172.21.15.42:0/3154869377 >> 172.21.15.42:0/3154869377 conn(0x7fcd540b3f90 msgr2=0x7fcd540b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:09.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.273+0000 7fcd5c600640 1 -- 172.21.15.42:0/3154869377 shutdown_connections 2024-05-28T13:59:09.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.273+0000 7fcd5c600640 1 -- 172.21.15.42:0/3154869377 wait complete. 2024-05-28T13:59:09.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.273+0000 7fcd5c600640 1 Processor -- start 2024-05-28T13:59:09.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.273+0000 7fcd5c600640 1 -- start start 2024-05-28T13:59:09.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5c600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5409b160 0x7fcd54156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5c600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5409b8e0 0x7fcd5415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5c600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd540b0e80 0x7fcd5415d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5c600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcd540af350 con 0x7fcd540b0e80 2024-05-28T13:59:09.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5c600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcd540af1d0 con 0x7fcd5409b160 2024-05-28T13:59:09.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5c600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcd540af4d0 con 0x7fcd5409b8e0 2024-05-28T13:59:09.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd540b0e80 0x7fcd5415d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd540b0e80 0x7fcd5415d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47922/0 (socket says 172.21.15.42:47922) 2024-05-28T13:59:09.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd59200640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5409b8e0 0x7fcd5415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd59c00640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5409b160 0x7fcd54156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5a600640 1 -- 172.21.15.42:0/3821812672 learned_addr learned my addr 172.21.15.42:0/3821812672 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:09.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5a600640 1 -- 172.21.15.42:0/3821812672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5409b8e0 msgr2=0x7fcd5415ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5a600640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5409b8e0 0x7fcd5415ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5a600640 1 -- 172.21.15.42:0/3821812672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5409b160 msgr2=0x7fcd54156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5a600640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5409b160 0x7fcd54156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5a600640 1 -- 172.21.15.42:0/3821812672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5415f990 con 0x7fcd540b0e80 2024-05-28T13:59:09.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd59200640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5409b8e0 0x7fcd5415ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:59:09.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd59c00640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5409b160 0x7fcd54156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:59:09.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5a600640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd540b0e80 0x7fcd5415d8e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00efc0 tx=0x7fcd4c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:09.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd51600640 1 -- 172.21.15.42:0/3821812672 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c018020 con 0x7fcd540b0e80 2024-05-28T13:59:09.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd51600640 1 -- 172.21.15.42:0/3821812672 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4c014480 con 0x7fcd540b0e80 2024-05-28T13:59:09.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd51600640 1 -- 172.21.15.42:0/3821812672 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c014870 con 0x7fcd540b0e80 2024-05-28T13:59:09.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5c600640 1 -- 172.21.15.42:0/3821812672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5415fc70 con 0x7fcd540b0e80 2024-05-28T13:59:09.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5c600640 1 -- 172.21.15.42:0/3821812672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd54160230 con 0x7fcd540b0e80 2024-05-28T13:59:09.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.277+0000 7fcd5c600640 1 -- 172.21.15.42:0/3821812672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd540a0550 con 0x7fcd540b0e80 2024-05-28T13:59:09.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.281+0000 7fcd51600640 1 -- 172.21.15.42:0/3821812672 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcd4c01b020 con 0x7fcd540b0e80 2024-05-28T13:59:09.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.281+0000 7fcd51600640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcd34068380 0x7fcd3406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.281+0000 7fcd51600640 1 -- 172.21.15.42:0/3821812672 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcd4c0878e0 con 0x7fcd540b0e80 2024-05-28T13:59:09.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.281+0000 7fcd59c00640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcd34068380 0x7fcd3406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.281+0000 7fcd59c00640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcd34068380 0x7fcd3406a840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcd480af000 tx=0x7fcd48008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:09.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.285+0000 7fcd51600640 1 -- 172.21.15.42:0/3821812672 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcd4c022030 con 0x7fcd540b0e80 2024-05-28T13:59:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.453+0000 7fcd5c600640 1 -- 172.21.15.42:0/3821812672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd540c7900 con 0x7fcd540b0e80 2024-05-28T13:59:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.453+0000 7fcd51600640 1 -- 172.21.15.42:0/3821812672 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fcd4c010040 con 0x7fcd540b0e80 2024-05-28T13:59:09.438 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:59:09.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.453+0000 7fcd5c600640 1 -- 172.21.15.42:0/3821812672 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcd34068380 msgr2=0x7fcd3406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.453+0000 7fcd5c600640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcd34068380 0x7fcd3406a840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcd480af000 tx=0x7fcd48008040 comp rx=0 tx=0).stop 2024-05-28T13:59:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.453+0000 7fcd5c600640 1 -- 172.21.15.42:0/3821812672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd540b0e80 msgr2=0x7fcd5415d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.453+0000 7fcd5c600640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd540b0e80 0x7fcd5415d8e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00efc0 tx=0x7fcd4c00c490 comp rx=0 tx=0).stop 2024-05-28T13:59:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.453+0000 7fcd5c600640 1 -- 172.21.15.42:0/3821812672 shutdown_connections 2024-05-28T13:59:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.453+0000 7fcd5c600640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcd34068380 0x7fcd3406a840 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.453+0000 7fcd5c600640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd540b0e80 0x7fcd5415d8e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.453+0000 7fcd5c600640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5409b8e0 0x7fcd5415ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.453+0000 7fcd5c600640 1 --2- 172.21.15.42:0/3821812672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5409b160 0x7fcd54156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.453+0000 7fcd5c600640 1 -- 172.21.15.42:0/3821812672 >> 172.21.15.42:0/3821812672 conn(0x7fcd540b3f90 msgr2=0x7fcd540b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.453+0000 7fcd5c600640 1 -- 172.21.15.42:0/3821812672 shutdown_connections 2024-05-28T13:59:09.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.453+0000 7fcd5c600640 1 -- 172.21.15.42:0/3821812672 wait complete. 2024-05-28T13:59:09.462 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:59:09.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f046b200640 1 Processor -- start 2024-05-28T13:59:09.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f046b200640 1 -- start start 2024-05-28T13:59:09.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f046b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04640cf540 0x7f04640cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f046b200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f04640707e0 0x7f04640cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f046b200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0464160170 con 0x7f0464145d50 2024-05-28T13:59:09.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f046b200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0464160450 con 0x7f04640707e0 2024-05-28T13:59:09.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f046b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0464160730 con 0x7f04640cf540 2024-05-28T13:59:09.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f046a800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f04640707e0 0x7f04640cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f0469400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0464145d50 0x7f0464071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46036/0 (socket says 172.21.15.42:46036) 2024-05-28T13:59:09.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f0469400640 1 -- 172.21.15.42:0/2325964181 learned_addr learned my addr 172.21.15.42:0/2325964181 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:09.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f0469e00640 1 --2- 172.21.15.42:0/2325964181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04640cf540 0x7f04640cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f046a800640 1 -- 172.21.15.42:0/2325964181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04640cf540 msgr2=0x7f04640cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:09.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f046a800640 1 --2- 172.21.15.42:0/2325964181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04640cf540 0x7f04640cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f046a800640 1 -- 172.21.15.42:0/2325964181 >> v1:172.21.15.42:6789/0 conn(0x7f0464145d50 legacy=0x7f0464071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.621+0000 7f046a800640 1 -- 172.21.15.42:0/2325964181 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04640d0ed0 con 0x7f04640707e0 2024-05-28T13:59:09.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.625+0000 7f046a800640 1 --2- 172.21.15.42:0/2325964181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04640707e0 0x7f04640cce80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0460009d90 tx=0x7f046001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62e424ad54f5956 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:09.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.625+0000 7f0468a00640 1 -- 172.21.15.42:0/2325964181 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04600a8620 con 0x7f04640707e0 2024-05-28T13:59:09.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.625+0000 7f0468a00640 1 -- 172.21.15.42:0/2325964181 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04600a87c0 con 0x7f04640707e0 2024-05-28T13:59:09.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.625+0000 7f0468a00640 1 -- 172.21.15.42:0/2325964181 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04600a8b90 con 0x7f04640707e0 2024-05-28T13:59:09.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.625+0000 7f046b200640 1 -- 172.21.15.42:0/2325964181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04640707e0 msgr2=0x7f04640cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.625+0000 7f046b200640 1 --2- 172.21.15.42:0/2325964181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04640707e0 0x7f04640cce80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0460009d90 tx=0x7f046001c8e0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.625+0000 7f046b200640 1 -- 172.21.15.42:0/2325964181 shutdown_connections 2024-05-28T13:59:09.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.625+0000 7f046b200640 1 --2- 172.21.15.42:0/2325964181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04640707e0 0x7f04640cce80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.625+0000 7f046b200640 1 --2- 172.21.15.42:0/2325964181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04640cf540 0x7f04640cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.625+0000 7f046b200640 1 -- 172.21.15.42:0/2325964181 >> 172.21.15.42:0/2325964181 conn(0x7f046406fcc0 msgr2=0x7f04640700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:09.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.625+0000 7f046b200640 1 -- 172.21.15.42:0/2325964181 shutdown_connections 2024-05-28T13:59:09.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.625+0000 7f046b200640 1 -- 172.21.15.42:0/2325964181 wait complete. 2024-05-28T13:59:09.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f046b200640 1 Processor -- start 2024-05-28T13:59:09.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f046b200640 1 -- start start 2024-05-28T13:59:09.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f046b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04640707e0 0x7f046406d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f046b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04640cf540 0x7f04641fa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f046b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0464145d50 0x7f04641f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f046b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0464169350 con 0x7f04640cf540 2024-05-28T13:59:09.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f046b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04640d1100 con 0x7f04640707e0 2024-05-28T13:59:09.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f046b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f04641694d0 con 0x7f0464145d50 2024-05-28T13:59:09.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f0469400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04640cf540 0x7f04641fa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f0469400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04640cf540 0x7f04641fa630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47940/0 (socket says 172.21.15.42:47940) 2024-05-28T13:59:09.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f0469400640 1 -- 172.21.15.42:0/186730237 learned_addr learned my addr 172.21.15.42:0/186730237 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:09.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f0469e00640 1 --2- 172.21.15.42:0/186730237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04640707e0 0x7f046406d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f046a800640 1 --2- 172.21.15.42:0/186730237 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0464145d50 0x7f04641f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f0469400640 1 -- 172.21.15.42:0/186730237 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0464145d50 msgr2=0x7f04641f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f0469400640 1 --2- 172.21.15.42:0/186730237 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0464145d50 0x7f04641f78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f0469400640 1 -- 172.21.15.42:0/186730237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04640707e0 msgr2=0x7f046406d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f0469400640 1 --2- 172.21.15.42:0/186730237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04640707e0 0x7f046406d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f0469400640 1 -- 172.21.15.42:0/186730237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0464159790 con 0x7f04640cf540 2024-05-28T13:59:09.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f0469400640 1 --2- 172.21.15.42:0/186730237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04640cf540 0x7f04641fa630 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f045400c370 tx=0x7f045400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:09.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f045e000640 1 -- 172.21.15.42:0/186730237 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454017070 con 0x7f04640cf540 2024-05-28T13:59:09.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f045e000640 1 -- 172.21.15.42:0/186730237 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0454005cf0 con 0x7f04640cf540 2024-05-28T13:59:09.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f045e000640 1 -- 172.21.15.42:0/186730237 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04540072f0 con 0x7f04640cf540 2024-05-28T13:59:09.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f046b200640 1 -- 172.21.15.42:0/186730237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04641f8500 con 0x7f04640cf540 2024-05-28T13:59:09.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f046b200640 1 -- 172.21.15.42:0/186730237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04641655f0 con 0x7f04640cf540 2024-05-28T13:59:09.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.629+0000 7f046b200640 1 -- 172.21.15.42:0/186730237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04641f8a20 con 0x7f04640cf540 2024-05-28T13:59:09.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.633+0000 7f045e000640 1 -- 172.21.15.42:0/186730237 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f045401a070 con 0x7f04640cf540 2024-05-28T13:59:09.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.633+0000 7f045e000640 1 --2- 172.21.15.42:0/186730237 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0448068380 0x7f044806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.633+0000 7f045e000640 1 -- 172.21.15.42:0/186730237 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f0454085d80 con 0x7f04640cf540 2024-05-28T13:59:09.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.633+0000 7f0469e00640 1 --2- 172.21.15.42:0/186730237 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0448068380 0x7f044806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.633+0000 7f0469e00640 1 --2- 172.21.15.42:0/186730237 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0448068380 0x7f044806a840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0458009af0 tx=0x7f0458008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:09.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.637+0000 7f045e000640 1 -- 172.21.15.42:0/186730237 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f04641f8a20 con 0x7f04640cf540 2024-05-28T13:59:09.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.801+0000 7f046b200640 1 -- 172.21.15.42:0/186730237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f046404fcf0 con 0x7f04640cf540 2024-05-28T13:59:09.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.801+0000 7f045e000640 1 -- 172.21.15.42:0/186730237 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f04540562b0 con 0x7f04640cf540 2024-05-28T13:59:09.787 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:59:09.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.805+0000 7f046b200640 1 -- 172.21.15.42:0/186730237 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0448068380 msgr2=0x7f044806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.805+0000 7f046b200640 1 --2- 172.21.15.42:0/186730237 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0448068380 0x7f044806a840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0458009af0 tx=0x7f0458008040 comp rx=0 tx=0).stop 2024-05-28T13:59:09.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.805+0000 7f046b200640 1 -- 172.21.15.42:0/186730237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04640cf540 msgr2=0x7f04641fa630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.805+0000 7f046b200640 1 --2- 172.21.15.42:0/186730237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04640cf540 0x7f04641fa630 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f045400c370 tx=0x7f045400c830 comp rx=0 tx=0).stop 2024-05-28T13:59:09.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.805+0000 7f046b200640 1 -- 172.21.15.42:0/186730237 shutdown_connections 2024-05-28T13:59:09.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.805+0000 7f046b200640 1 --2- 172.21.15.42:0/186730237 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0448068380 0x7f044806a840 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.805+0000 7f046b200640 1 --2- 172.21.15.42:0/186730237 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0464145d50 0x7f04641f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.805+0000 7f046b200640 1 --2- 172.21.15.42:0/186730237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04640cf540 0x7f04641fa630 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.805+0000 7f046b200640 1 --2- 172.21.15.42:0/186730237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04640707e0 0x7f046406d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.805+0000 7f046b200640 1 -- 172.21.15.42:0/186730237 >> 172.21.15.42:0/186730237 conn(0x7f046406fcc0 msgr2=0x7f04640c6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:09.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.805+0000 7f046b200640 1 -- 172.21.15.42:0/186730237 shutdown_connections 2024-05-28T13:59:09.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.805+0000 7f046b200640 1 -- 172.21.15.42:0/186730237 wait complete. 2024-05-28T13:59:09.809 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:59:09.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.965+0000 7f70c3600640 1 Processor -- start 2024-05-28T13:59:09.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.965+0000 7f70c3600640 1 -- start start 2024-05-28T13:59:09.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.969+0000 7f70c3600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f70bc0b0e80 0x7f70bc06cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.969+0000 7f70c3600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f70bc09b160 0x7f70bc15e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.969+0000 7f70c3600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70bc0d0770 con 0x7f70bc09b8e0 2024-05-28T13:59:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.969+0000 7f70c3600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f70bc0d0a10 con 0x7f70bc09b160 2024-05-28T13:59:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.969+0000 7f70c3600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f70bc0d0cf0 con 0x7f70bc0b0e80 2024-05-28T13:59:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.969+0000 7f70c1600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f70bc09b160 0x7f70bc15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.969+0000 7f70c1600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f70bc09b160 0x7f70bc15e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47944/0 (socket says 172.21.15.42:47944) 2024-05-28T13:59:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.969+0000 7f70c1600640 1 -- 172.21.15.42:0/3337850029 learned_addr learned my addr 172.21.15.42:0/3337850029 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.969+0000 7f70c0c00640 1 --2- 172.21.15.42:0/3337850029 >> v2:172.21.15.63:3300/0 conn(0x7f70bc0b0e80 0x7f70bc06cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.969+0000 7f70c1600640 1 -- 172.21.15.42:0/3337850029 >> v2:172.21.15.63:3300/0 conn(0x7f70bc0b0e80 msgr2=0x7f70bc06cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.969+0000 7f70c1600640 1 --2- 172.21.15.42:0/3337850029 >> v2:172.21.15.63:3300/0 conn(0x7f70bc0b0e80 0x7f70bc06cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.969+0000 7f70c1600640 1 -- 172.21.15.42:0/3337850029 >> v1:172.21.15.63:6789/0 conn(0x7f70bc09b8e0 legacy=0x7f70bc09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.969+0000 7f70c1600640 1 -- 172.21.15.42:0/3337850029 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70bc0d0fd0 con 0x7f70bc09b160 2024-05-28T13:59:09.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.973+0000 7f70c1600640 1 --2- 172.21.15.42:0/3337850029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70bc09b160 0x7f70bc15e110 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f70bc072fd0 tx=0x7f70b401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436ed047329dd54a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:09.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.973+0000 7f70bbe00640 1 -- 172.21.15.42:0/3337850029 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b40a56c0 con 0x7f70bc09b160 2024-05-28T13:59:09.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.973+0000 7f70bbe00640 1 -- 172.21.15.42:0/3337850029 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b40a5860 con 0x7f70bc09b160 2024-05-28T13:59:09.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.973+0000 7f70bbe00640 1 -- 172.21.15.42:0/3337850029 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b40a5c30 con 0x7f70bc09b160 2024-05-28T13:59:09.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.973+0000 7f70c3600640 1 -- 172.21.15.42:0/3337850029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70bc09b160 msgr2=0x7f70bc15e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.973+0000 7f70c3600640 1 --2- 172.21.15.42:0/3337850029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70bc09b160 0x7f70bc15e110 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f70bc072fd0 tx=0x7f70b401c9f0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.973+0000 7f70c3600640 1 -- 172.21.15.42:0/3337850029 shutdown_connections 2024-05-28T13:59:09.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.973+0000 7f70c3600640 1 --2- 172.21.15.42:0/3337850029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70bc09b160 0x7f70bc15e110 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.973+0000 7f70c3600640 1 --2- 172.21.15.42:0/3337850029 >> v2:172.21.15.63:3300/0 conn(0x7f70bc0b0e80 0x7f70bc06cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.973+0000 7f70c3600640 1 -- 172.21.15.42:0/3337850029 >> 172.21.15.42:0/3337850029 conn(0x7f70bc0b3f90 msgr2=0x7f70bc0b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:09.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.973+0000 7f70c3600640 1 -- 172.21.15.42:0/3337850029 shutdown_connections 2024-05-28T13:59:09.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.973+0000 7f70c3600640 1 -- 172.21.15.42:0/3337850029 wait complete. 2024-05-28T13:59:09.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.973+0000 7f70c3600640 1 Processor -- start 2024-05-28T13:59:09.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.973+0000 7f70c3600640 1 -- start start 2024-05-28T13:59:09.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c3600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70bc09b160 0x7f70bc155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c3600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70bc09b8e0 0x7f70bc158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c3600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70bc0b0e80 0x7f70bc158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c3600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc0af350 con 0x7f70bc0b0e80 2024-05-28T13:59:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c3600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc0af1d0 con 0x7f70bc09b8e0 2024-05-28T13:59:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c3600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f70bc0af4d0 con 0x7f70bc09b160 2024-05-28T13:59:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c1600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70bc0b0e80 0x7f70bc158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c0c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70bc09b160 0x7f70bc155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c1600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70bc0b0e80 0x7f70bc158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47958/0 (socket says 172.21.15.42:47958) 2024-05-28T13:59:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70bc09b8e0 0x7f70bc158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c1600640 1 -- 172.21.15.42:0/2769367986 learned_addr learned my addr 172.21.15.42:0/2769367986 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c1600640 1 -- 172.21.15.42:0/2769367986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70bc09b160 msgr2=0x7f70bc155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c1600640 1 --2- 172.21.15.42:0/2769367986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70bc09b160 0x7f70bc155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c1600640 1 -- 172.21.15.42:0/2769367986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70bc09b8e0 msgr2=0x7f70bc158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c1600640 1 --2- 172.21.15.42:0/2769367986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70bc09b8e0 0x7f70bc158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c1600640 1 -- 172.21.15.42:0/2769367986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70bc0c6480 con 0x7f70bc0b0e80 2024-05-28T13:59:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c1600640 1 --2- 172.21.15.42:0/2769367986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70bc0b0e80 0x7f70bc158560 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f70b40af040 tx=0x7f70b4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70b9600640 1 -- 172.21.15.42:0/2769367986 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b40bc020 con 0x7f70bc0b0e80 2024-05-28T13:59:09.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70b9600640 1 -- 172.21.15.42:0/2769367986 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b40b8670 con 0x7f70bc0b0e80 2024-05-28T13:59:09.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c3600640 1 -- 172.21.15.42:0/2769367986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70bc0c6760 con 0x7f70bc0b0e80 2024-05-28T13:59:09.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70b9600640 1 -- 172.21.15.42:0/2769367986 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4009510 con 0x7f70bc0b0e80 2024-05-28T13:59:09.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c3600640 1 -- 172.21.15.42:0/2769367986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70bc0c8ec0 con 0x7f70bc0b0e80 2024-05-28T13:59:09.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70c3600640 1 -- 172.21.15.42:0/2769367986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70bc0a0550 con 0x7f70bc0b0e80 2024-05-28T13:59:09.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.977+0000 7f70b9600640 1 -- 172.21.15.42:0/2769367986 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f70b40bf020 con 0x7f70bc0b0e80 2024-05-28T13:59:09.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.981+0000 7f70b9600640 1 --2- 172.21.15.42:0/2769367986 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70980683b0 0x7f709806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:09.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.981+0000 7f70b9600640 1 -- 172.21.15.42:0/2769367986 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f70b41217c0 con 0x7f70bc0b0e80 2024-05-28T13:59:09.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.981+0000 7f70c0c00640 1 --2- 172.21.15.42:0/2769367986 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70980683b0 0x7f709806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:09.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.981+0000 7f70c0c00640 1 --2- 172.21.15.42:0/2769367986 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70980683b0 0x7f709806a870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f70ac009ac0 tx=0x7f70ac009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:09.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:09.985+0000 7f70b9600640 1 -- 172.21.15.42:0/2769367986 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f70b40c6030 con 0x7f70bc0b0e80 2024-05-28T13:59:10.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.145+0000 7f70c3600640 1 -- 172.21.15.42:0/2769367986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f70bc06cdd0 con 0x7f70bc0b0e80 2024-05-28T13:59:10.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.149+0000 7f70b9600640 1 -- 172.21.15.42:0/2769367986 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f70b40165e0 con 0x7f70bc0b0e80 2024-05-28T13:59:10.132 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:59:10.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.149+0000 7f70c3600640 1 -- 172.21.15.42:0/2769367986 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70980683b0 msgr2=0x7f709806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:10.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.149+0000 7f70c3600640 1 --2- 172.21.15.42:0/2769367986 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70980683b0 0x7f709806a870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f70ac009ac0 tx=0x7f70ac009190 comp rx=0 tx=0).stop 2024-05-28T13:59:10.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.149+0000 7f70c3600640 1 -- 172.21.15.42:0/2769367986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70bc0b0e80 msgr2=0x7f70bc158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:10.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.149+0000 7f70c3600640 1 --2- 172.21.15.42:0/2769367986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70bc0b0e80 0x7f70bc158560 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f70b40af040 tx=0x7f70b4019790 comp rx=0 tx=0).stop 2024-05-28T13:59:10.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.149+0000 7f70c3600640 1 -- 172.21.15.42:0/2769367986 shutdown_connections 2024-05-28T13:59:10.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.149+0000 7f70c3600640 1 --2- 172.21.15.42:0/2769367986 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70980683b0 0x7f709806a870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.149+0000 7f70c3600640 1 --2- 172.21.15.42:0/2769367986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70bc0b0e80 0x7f70bc158560 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.149+0000 7f70c3600640 1 --2- 172.21.15.42:0/2769367986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70bc09b8e0 0x7f70bc158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.149+0000 7f70c3600640 1 --2- 172.21.15.42:0/2769367986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70bc09b160 0x7f70bc155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.149+0000 7f70c3600640 1 -- 172.21.15.42:0/2769367986 >> 172.21.15.42:0/2769367986 conn(0x7f70bc0b3f90 msgr2=0x7f70bc09aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:10.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.149+0000 7f70c3600640 1 -- 172.21.15.42:0/2769367986 shutdown_connections 2024-05-28T13:59:10.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.149+0000 7f70c3600640 1 -- 172.21.15.42:0/2769367986 wait complete. 2024-05-28T13:59:10.155 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T13:59:10.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b4400640 1 Processor -- start 2024-05-28T13:59:10.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b4400640 1 -- start start 2024-05-28T13:59:10.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b4400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc5ac15ccb0 0x7fc5ac15d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:10.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b4400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc5ac0cd0d0 0x7fc5ac09b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:10.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b4400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5ac15d5d0 con 0x7fc5ac160210 2024-05-28T13:59:10.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b4400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc5ac0ce210 con 0x7fc5ac15ccb0 2024-05-28T13:59:10.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b4400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc5ac0cd600 con 0x7fc5ac0cd0d0 2024-05-28T13:59:10.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b1a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc5ac15ccb0 0x7fc5ac15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:10.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b1a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc5ac15ccb0 0x7fc5ac15d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47966/0 (socket says 172.21.15.42:47966) 2024-05-28T13:59:10.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b1a00640 1 -- 172.21.15.42:0/3594577138 learned_addr learned my addr 172.21.15.42:0/3594577138 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:10.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b1000640 1 --2- 172.21.15.42:0/3594577138 >> v2:172.21.15.63:3300/0 conn(0x7fc5ac0cd0d0 0x7fc5ac09b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:10.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b1a00640 1 -- 172.21.15.42:0/3594577138 >> v2:172.21.15.63:3300/0 conn(0x7fc5ac0cd0d0 msgr2=0x7fc5ac09b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b1a00640 1 --2- 172.21.15.42:0/3594577138 >> v2:172.21.15.63:3300/0 conn(0x7fc5ac0cd0d0 0x7fc5ac09b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b1a00640 1 -- 172.21.15.42:0/3594577138 >> v1:172.21.15.63:6789/0 conn(0x7fc5ac160210 legacy=0x7fc5ac160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b1a00640 1 -- 172.21.15.42:0/3594577138 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac0cd8e0 con 0x7fc5ac15ccb0 2024-05-28T13:59:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.313+0000 7fc5b1000640 1 --2- 172.21.15.42:0/3594577138 >> v2:172.21.15.63:3300/0 conn(0x7fc5ac0cd0d0 0x7fc5ac09b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:10.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.317+0000 7fc5b1a00640 1 --2- 172.21.15.42:0/3594577138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5ac15ccb0 0x7fc5ac15d090 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0733a0 tx=0x7fc5a001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da6bf91a63bd8be server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:10.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.317+0000 7fc5abe00640 1 -- 172.21.15.42:0/3594577138 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00b1070 con 0x7fc5ac15ccb0 2024-05-28T13:59:10.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.317+0000 7fc5abe00640 1 -- 172.21.15.42:0/3594577138 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a00ac630 con 0x7fc5ac15ccb0 2024-05-28T13:59:10.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.317+0000 7fc5abe00640 1 -- 172.21.15.42:0/3594577138 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00a8050 con 0x7fc5ac15ccb0 2024-05-28T13:59:10.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.317+0000 7fc5b4400640 1 -- 172.21.15.42:0/3594577138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5ac15ccb0 msgr2=0x7fc5ac15d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:10.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.317+0000 7fc5b4400640 1 --2- 172.21.15.42:0/3594577138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5ac15ccb0 0x7fc5ac15d090 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0733a0 tx=0x7fc5a001c4e0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.317+0000 7fc5b4400640 1 -- 172.21.15.42:0/3594577138 shutdown_connections 2024-05-28T13:59:10.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.317+0000 7fc5b4400640 1 --2- 172.21.15.42:0/3594577138 >> v2:172.21.15.63:3300/0 conn(0x7fc5ac0cd0d0 0x7fc5ac09b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.317+0000 7fc5b4400640 1 --2- 172.21.15.42:0/3594577138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5ac15ccb0 0x7fc5ac15d090 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.317+0000 7fc5b4400640 1 -- 172.21.15.42:0/3594577138 >> 172.21.15.42:0/3594577138 conn(0x7fc5ac0b40d0 msgr2=0x7fc5ac0b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:10.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.317+0000 7fc5b4400640 1 -- 172.21.15.42:0/3594577138 shutdown_connections 2024-05-28T13:59:10.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.317+0000 7fc5b4400640 1 -- 172.21.15.42:0/3594577138 wait complete. 2024-05-28T13:59:10.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b4400640 1 Processor -- start 2024-05-28T13:59:10.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b4400640 1 -- start start 2024-05-28T13:59:10.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b4400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5ac0cd0d0 0x7fc5ac0b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:10.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b4400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5ac15ccb0 0x7fc5ac0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:10.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b4400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5ac160210 0x7fc5ac1523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:10.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b4400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac15ca60 con 0x7fc5ac0cd0d0 2024-05-28T13:59:10.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b4400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac15c8e0 con 0x7fc5ac15ccb0 2024-05-28T13:59:10.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b4400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc5ac169dc0 con 0x7fc5ac160210 2024-05-28T13:59:10.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b1a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5ac0cd0d0 0x7fc5ac0b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:10.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b1a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5ac0cd0d0 0x7fc5ac0b7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47976/0 (socket says 172.21.15.42:47976) 2024-05-28T13:59:10.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b1a00640 1 -- 172.21.15.42:0/481365255 learned_addr learned my addr 172.21.15.42:0/481365255 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:10.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b1a00640 1 -- 172.21.15.42:0/481365255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5ac160210 msgr2=0x7fc5ac1523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:10.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b1a00640 1 --2- 172.21.15.42:0/481365255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5ac160210 0x7fc5ac1523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b1a00640 1 -- 172.21.15.42:0/481365255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5ac15ccb0 msgr2=0x7fc5ac0b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:10.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b1a00640 1 --2- 172.21.15.42:0/481365255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5ac15ccb0 0x7fc5ac0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b1a00640 1 -- 172.21.15.42:0/481365255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac0b7140 con 0x7fc5ac0cd0d0 2024-05-28T13:59:10.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b1a00640 1 --2- 172.21.15.42:0/481365255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5ac0cd0d0 0x7fc5ac0b7f00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00af040 tx=0x7fc5a0019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:10.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5a9600640 1 -- 172.21.15.42:0/481365255 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00bc070 con 0x7fc5ac0cd0d0 2024-05-28T13:59:10.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5a9600640 1 -- 172.21.15.42:0/481365255 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a00ab800 con 0x7fc5ac0cd0d0 2024-05-28T13:59:10.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b4400640 1 -- 172.21.15.42:0/481365255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac0b7420 con 0x7fc5ac0cd0d0 2024-05-28T13:59:10.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5a9600640 1 -- 172.21.15.42:0/481365255 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00b1070 con 0x7fc5ac0cd0d0 2024-05-28T13:59:10.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.321+0000 7fc5b4400640 1 -- 172.21.15.42:0/481365255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac153000 con 0x7fc5ac0cd0d0 2024-05-28T13:59:10.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.325+0000 7fc5b4400640 1 -- 172.21.15.42:0/481365255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5ac0a06c0 con 0x7fc5ac0cd0d0 2024-05-28T13:59:10.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.325+0000 7fc5a9600640 1 -- 172.21.15.42:0/481365255 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc5a00bf070 con 0x7fc5ac0cd0d0 2024-05-28T13:59:10.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.325+0000 7fc5a9600640 1 --2- 172.21.15.42:0/481365255 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc58c0683b0 0x7fc58c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:10.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.325+0000 7fc5a9600640 1 -- 172.21.15.42:0/481365255 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc5a0120440 con 0x7fc5ac0cd0d0 2024-05-28T13:59:10.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.325+0000 7fc5b1000640 1 --2- 172.21.15.42:0/481365255 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc58c0683b0 0x7fc58c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:10.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.325+0000 7fc5b1000640 1 --2- 172.21.15.42:0/481365255 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc58c0683b0 0x7fc58c06a870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac153530 tx=0x7fc598008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:10.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.329+0000 7fc5a9600640 1 -- 172.21.15.42:0/481365255 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc5a00160d0 con 0x7fc5ac0cd0d0 2024-05-28T13:59:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.505+0000 7fc5b4400640 1 -- 172.21.15.42:0/481365255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc5ac04fdd0 con 0x7fc5ac0cd0d0 2024-05-28T13:59:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.505+0000 7fc5a9600640 1 -- 172.21.15.42:0/481365255 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fc5a00f08e0 con 0x7fc5ac0cd0d0 2024-05-28T13:59:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T13:59:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.509+0000 7fc5b4400640 1 -- 172.21.15.42:0/481365255 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc58c0683b0 msgr2=0x7fc58c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.509+0000 7fc5b4400640 1 --2- 172.21.15.42:0/481365255 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc58c0683b0 0x7fc58c06a870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac153530 tx=0x7fc598008040 comp rx=0 tx=0).stop 2024-05-28T13:59:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.509+0000 7fc5b4400640 1 -- 172.21.15.42:0/481365255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5ac0cd0d0 msgr2=0x7fc5ac0b7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.509+0000 7fc5b4400640 1 --2- 172.21.15.42:0/481365255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5ac0cd0d0 0x7fc5ac0b7f00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00af040 tx=0x7fc5a0019280 comp rx=0 tx=0).stop 2024-05-28T13:59:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.509+0000 7fc5b4400640 1 -- 172.21.15.42:0/481365255 shutdown_connections 2024-05-28T13:59:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.509+0000 7fc5b4400640 1 --2- 172.21.15.42:0/481365255 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc58c0683b0 0x7fc58c06a870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.509+0000 7fc5b4400640 1 --2- 172.21.15.42:0/481365255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5ac160210 0x7fc5ac1523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.509+0000 7fc5b4400640 1 --2- 172.21.15.42:0/481365255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5ac15ccb0 0x7fc5ac0b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.509+0000 7fc5b4400640 1 --2- 172.21.15.42:0/481365255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5ac0cd0d0 0x7fc5ac0b7f00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.509+0000 7fc5b4400640 1 -- 172.21.15.42:0/481365255 >> 172.21.15.42:0/481365255 conn(0x7fc5ac0b40d0 msgr2=0x7fc5ac06e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.509+0000 7fc5b4400640 1 -- 172.21.15.42:0/481365255 shutdown_connections 2024-05-28T13:59:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.509+0000 7fc5b4400640 1 -- 172.21.15.42:0/481365255 wait complete. 2024-05-28T13:59:10.513 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T13:59:10.514 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:59:10.523 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:59:10.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a37200640 1 Processor -- start 2024-05-28T13:59:10.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a37200640 1 -- start start 2024-05-28T13:59:10.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a37200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a380ce100 0x7f5a3809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a37200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5a3806cd10 0x7f5a3809af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a37200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a380d0690 con 0x7f5a380ce4e0 2024-05-28T13:59:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a37200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5a380cd910 con 0x7f5a3806cd10 2024-05-28T13:59:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a37200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5a380d0970 con 0x7f5a380ce100 2024-05-28T13:59:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a35400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f5a380ce4e0 0x7f5a3815df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46050/0 (socket says 172.21.15.42:46050) 2024-05-28T13:59:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a35400640 1 -- 172.21.15.42:0/897086624 learned_addr learned my addr 172.21.15.42:0/897086624 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a36800640 1 --2- 172.21.15.42:0/897086624 >> v2:172.21.15.63:3300/0 conn(0x7f5a3806cd10 0x7f5a3809af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a35e00640 1 --2- 172.21.15.42:0/897086624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a380ce100 0x7f5a3809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a34a00640 1 -- 172.21.15.42:0/897086624 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 91774409 0 0) 0x7f5a380d0690 con 0x7f5a380ce4e0 2024-05-28T13:59:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a34a00640 1 -- 172.21.15.42:0/897086624 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a1c003530 con 0x7f5a380ce4e0 2024-05-28T13:59:10.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a35e00640 1 -- 172.21.15.42:0/897086624 >> v2:172.21.15.63:3300/0 conn(0x7f5a3806cd10 msgr2=0x7f5a3809af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:10.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a35e00640 1 --2- 172.21.15.42:0/897086624 >> v2:172.21.15.63:3300/0 conn(0x7f5a3806cd10 0x7f5a3809af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a35e00640 1 -- 172.21.15.42:0/897086624 >> v1:172.21.15.42:6789/0 conn(0x7f5a380ce4e0 legacy=0x7f5a3815df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:10.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a35e00640 1 -- 172.21.15.42:0/897086624 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a380d0c50 con 0x7f5a380ce100 2024-05-28T13:59:10.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.681+0000 7f5a36800640 1 --2- 172.21.15.42:0/897086624 >> v2:172.21.15.63:3300/0 conn(0x7f5a3806cd10 0x7f5a3809af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:59:10.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.685+0000 7f5a35e00640 1 --2- 172.21.15.42:0/897086624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a380ce100 0x7f5a3809aa50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5a280097c0 tx=0x7f5a2801ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64394974c9278839 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:10.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.685+0000 7f5a34a00640 1 -- 172.21.15.42:0/897086624 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a280b5070 con 0x7f5a380ce100 2024-05-28T13:59:10.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.685+0000 7f5a34a00640 1 -- 172.21.15.42:0/897086624 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a280a8990 con 0x7f5a380ce100 2024-05-28T13:59:10.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.685+0000 7f5a34a00640 1 -- 172.21.15.42:0/897086624 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a280a8d40 con 0x7f5a380ce100 2024-05-28T13:59:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.685+0000 7f5a37200640 1 -- 172.21.15.42:0/897086624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a380ce100 msgr2=0x7f5a3809aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.685+0000 7f5a37200640 1 --2- 172.21.15.42:0/897086624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a380ce100 0x7f5a3809aa50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5a280097c0 tx=0x7f5a2801ec90 comp rx=0 tx=0).stop 2024-05-28T13:59:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.685+0000 7f5a37200640 1 -- 172.21.15.42:0/897086624 shutdown_connections 2024-05-28T13:59:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.685+0000 7f5a37200640 1 --2- 172.21.15.42:0/897086624 >> v2:172.21.15.63:3300/0 conn(0x7f5a3806cd10 0x7f5a3809af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.685+0000 7f5a37200640 1 --2- 172.21.15.42:0/897086624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a380ce100 0x7f5a3809aa50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.685+0000 7f5a37200640 1 -- 172.21.15.42:0/897086624 >> 172.21.15.42:0/897086624 conn(0x7f5a380b3da0 msgr2=0x7f5a380b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:10.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.685+0000 7f5a37200640 1 -- 172.21.15.42:0/897086624 shutdown_connections 2024-05-28T13:59:10.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.685+0000 7f5a37200640 1 -- 172.21.15.42:0/897086624 wait complete. 2024-05-28T13:59:10.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a37200640 1 Processor -- start 2024-05-28T13:59:10.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a37200640 1 -- start start 2024-05-28T13:59:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a37200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a3806cd10 0x7f5a380c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a37200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a380ce100 0x7f5a3815a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a37200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a380ce4e0 0x7f5a3815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a37200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3806dd30 con 0x7f5a380ce100 2024-05-28T13:59:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a37200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a380d1210 con 0x7f5a380ce4e0 2024-05-28T13:59:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a37200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5a3806deb0 con 0x7f5a3806cd10 2024-05-28T13:59:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a35400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a380ce100 0x7f5a3815a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a35400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a380ce100 0x7f5a3815a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47988/0 (socket says 172.21.15.42:47988) 2024-05-28T13:59:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a35e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a3806cd10 0x7f5a380c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a35400640 1 -- 172.21.15.42:0/2738336665 learned_addr learned my addr 172.21.15.42:0/2738336665 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a36800640 1 --2- 172.21.15.42:0/2738336665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a380ce4e0 0x7f5a3815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a35400640 1 -- 172.21.15.42:0/2738336665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a3806cd10 msgr2=0x7f5a380c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a35400640 1 --2- 172.21.15.42:0/2738336665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a3806cd10 0x7f5a380c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a35400640 1 -- 172.21.15.42:0/2738336665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a380ce4e0 msgr2=0x7f5a3815adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a35400640 1 --2- 172.21.15.42:0/2738336665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a380ce4e0 0x7f5a3815adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a35400640 1 -- 172.21.15.42:0/2738336665 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a380c6860 con 0x7f5a380ce100 2024-05-28T13:59:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a35400640 1 --2- 172.21.15.42:0/2738336665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a380ce100 0x7f5a3815a880 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400c450 tx=0x7f5a2400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a2e000640 1 -- 172.21.15.42:0/2738336665 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24017070 con 0x7f5a380ce100 2024-05-28T13:59:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a2e000640 1 -- 172.21.15.42:0/2738336665 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a24005cf0 con 0x7f5a380ce100 2024-05-28T13:59:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a2e000640 1 -- 172.21.15.42:0/2738336665 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a240072f0 con 0x7f5a380ce100 2024-05-28T13:59:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a37200640 1 -- 172.21.15.42:0/2738336665 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a380c6b40 con 0x7f5a380ce100 2024-05-28T13:59:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.689+0000 7f5a37200640 1 -- 172.21.15.42:0/2738336665 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a380c7100 con 0x7f5a380ce100 2024-05-28T13:59:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.693+0000 7f5a37200640 1 -- 172.21.15.42:0/2738336665 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a3804fcf0 con 0x7f5a380ce100 2024-05-28T13:59:10.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.693+0000 7f5a2e000640 1 -- 172.21.15.42:0/2738336665 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5a2401a070 con 0x7f5a380ce100 2024-05-28T13:59:10.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.693+0000 7f5a2e000640 1 --2- 172.21.15.42:0/2738336665 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a18068380 0x7f5a1806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.693+0000 7f5a2e000640 1 -- 172.21.15.42:0/2738336665 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5a24085e40 con 0x7f5a380ce100 2024-05-28T13:59:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.693+0000 7f5a35e00640 1 --2- 172.21.15.42:0/2738336665 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a18068380 0x7f5a1806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.693+0000 7f5a35e00640 1 --2- 172.21.15.42:0/2738336665 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a18068380 0x7f5a1806a840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5a280b2000 tx=0x7f5a280aa070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:10.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.697+0000 7f5a2e000640 1 -- 172.21.15.42:0/2738336665 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5a24055c90 con 0x7f5a380ce100 2024-05-28T13:59:10.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.865+0000 7f5a37200640 1 -- 172.21.15.42:0/2738336665 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a380b8230 con 0x7f5a380ce100 2024-05-28T13:59:10.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.869+0000 7f5a2e000640 1 -- 172.21.15.42:0/2738336665 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16737 (secure 0 0 0) 0x7f5a24059f90 con 0x7f5a380ce100 2024-05-28T13:59:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:10.853 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:59:03.992606+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:59:10.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.869+0000 7f5a37200640 1 -- 172.21.15.42:0/2738336665 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a18068380 msgr2=0x7f5a1806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:10.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.869+0000 7f5a37200640 1 --2- 172.21.15.42:0/2738336665 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a18068380 0x7f5a1806a840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5a280b2000 tx=0x7f5a280aa070 comp rx=0 tx=0).stop 2024-05-28T13:59:10.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.869+0000 7f5a37200640 1 -- 172.21.15.42:0/2738336665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a380ce100 msgr2=0x7f5a3815a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:10.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.869+0000 7f5a37200640 1 --2- 172.21.15.42:0/2738336665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a380ce100 0x7f5a3815a880 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400c450 tx=0x7f5a2400c850 comp rx=0 tx=0).stop 2024-05-28T13:59:10.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.869+0000 7f5a37200640 1 -- 172.21.15.42:0/2738336665 shutdown_connections 2024-05-28T13:59:10.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.869+0000 7f5a37200640 1 --2- 172.21.15.42:0/2738336665 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a18068380 0x7f5a1806a840 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.869+0000 7f5a37200640 1 --2- 172.21.15.42:0/2738336665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a380ce4e0 0x7f5a3815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.869+0000 7f5a37200640 1 --2- 172.21.15.42:0/2738336665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a380ce100 0x7f5a3815a880 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.869+0000 7f5a37200640 1 --2- 172.21.15.42:0/2738336665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a3806cd10 0x7f5a380c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.869+0000 7f5a37200640 1 -- 172.21.15.42:0/2738336665 >> 172.21.15.42:0/2738336665 conn(0x7f5a380b3da0 msgr2=0x7f5a380b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.869+0000 7f5a37200640 1 -- 172.21.15.42:0/2738336665 shutdown_connections 2024-05-28T13:59:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:10.869+0000 7f5a37200640 1 -- 172.21.15.42:0/2738336665 wait complete. 2024-05-28T13:59:10.875 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:59:10.875 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:59:11.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.033+0000 7ffab7200640 1 Processor -- start 2024-05-28T13:59:11.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.033+0000 7ffab7200640 1 -- start start 2024-05-28T13:59:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.033+0000 7ffab7200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffab006b9f0 0x7ffab006bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.033+0000 7ffab7200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffab00b4400 con 0x7ffab00b5ba0 2024-05-28T13:59:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.033+0000 7ffab7200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffab00b04e0 con 0x7ffab0145a60 2024-05-28T13:59:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.033+0000 7ffab7200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffab00b4700 con 0x7ffab006b9f0 2024-05-28T13:59:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.037+0000 7ffab5200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ffab00b5ba0 0x7ffab00d1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46066/0 (socket says 172.21.15.42:46066) 2024-05-28T13:59:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.037+0000 7ffab5200640 1 -- 172.21.15.42:0/3772814401 learned_addr learned my addr 172.21.15.42:0/3772814401 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.037+0000 7ffaafe00640 1 --2- 172.21.15.42:0/3772814401 >> v2:172.21.15.63:3300/0 conn(0x7ffab006b9f0 0x7ffab006bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.037+0000 7ffaaea00640 1 -- 172.21.15.42:0/3772814401 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1423180792 0 0) 0x7ffab00b4400 con 0x7ffab00b5ba0 2024-05-28T13:59:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.037+0000 7ffaaea00640 1 -- 172.21.15.42:0/3772814401 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa94003530 con 0x7ffab00b5ba0 2024-05-28T13:59:11.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.037+0000 7ffaafe00640 1 -- 172.21.15.42:0/3772814401 >> v1:172.21.15.63:6789/0 conn(0x7ffab0145a60 legacy=0x7ffab00d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.037+0000 7ffaaea00640 1 -- 172.21.15.42:0/3772814401 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2553573795 0 0) 0x7ffa9c001120 con 0x7ffab0145a60 2024-05-28T13:59:11.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.037+0000 7ffaafe00640 1 -- 172.21.15.42:0/3772814401 >> v1:172.21.15.42:6789/0 conn(0x7ffab00b5ba0 legacy=0x7ffab00d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.037+0000 7ffaafe00640 1 -- 172.21.15.42:0/3772814401 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab00d1f90 con 0x7ffab006b9f0 2024-05-28T13:59:11.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.037+0000 7ffaafe00640 1 --2- 172.21.15.42:0/3772814401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffab006b9f0 0x7ffab006bdd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00097c0 tx=0x7ffaa001c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f692ce1265f2207 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:11.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.037+0000 7ffaaea00640 1 -- 172.21.15.42:0/3772814401 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00b0070 con 0x7ffab006b9f0 2024-05-28T13:59:11.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.037+0000 7ffaaea00640 1 -- 172.21.15.42:0/3772814401 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa00ab600 con 0x7ffab006b9f0 2024-05-28T13:59:11.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.037+0000 7ffaaea00640 1 -- 172.21.15.42:0/3772814401 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00ab9d0 con 0x7ffab006b9f0 2024-05-28T13:59:11.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 -- 172.21.15.42:0/3772814401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffab006b9f0 msgr2=0x7ffab006bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 --2- 172.21.15.42:0/3772814401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffab006b9f0 0x7ffab006bdd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00097c0 tx=0x7ffaa001c380 comp rx=0 tx=0).stop 2024-05-28T13:59:11.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 -- 172.21.15.42:0/3772814401 shutdown_connections 2024-05-28T13:59:11.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 --2- 172.21.15.42:0/3772814401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffab006b9f0 0x7ffab006bdd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 -- 172.21.15.42:0/3772814401 >> 172.21.15.42:0/3772814401 conn(0x7ffab00b6880 msgr2=0x7ffab00b6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:11.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 -- 172.21.15.42:0/3772814401 shutdown_connections 2024-05-28T13:59:11.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 -- 172.21.15.42:0/3772814401 wait complete. 2024-05-28T13:59:11.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 Processor -- start 2024-05-28T13:59:11.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 -- start start 2024-05-28T13:59:11.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffab006b9f0 0x7ffab01418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab00b5ba0 0x7ffab015efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffab0145a60 0x7ffab01421a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffab00b0d20 con 0x7ffab00b5ba0 2024-05-28T13:59:11.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffab0145840 con 0x7ffab006b9f0 2024-05-28T13:59:11.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab7200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffab016ab00 con 0x7ffab0145a60 2024-05-28T13:59:11.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffaaf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab00b5ba0 0x7ffab015efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffaaf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab00b5ba0 0x7ffab015efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47998/0 (socket says 172.21.15.42:47998) 2024-05-28T13:59:11.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffaaf400640 1 -- 172.21.15.42:0/2343583902 learned_addr learned my addr 172.21.15.42:0/2343583902 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:11.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffaafe00640 1 --2- 172.21.15.42:0/2343583902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffab006b9f0 0x7ffab01418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffab5200640 1 --2- 172.21.15.42:0/2343583902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffab0145a60 0x7ffab01421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffaaf400640 1 -- 172.21.15.42:0/2343583902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffab0145a60 msgr2=0x7ffab01421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffaaf400640 1 --2- 172.21.15.42:0/2343583902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffab0145a60 0x7ffab01421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffaaf400640 1 -- 172.21.15.42:0/2343583902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffab006b9f0 msgr2=0x7ffab01418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.041+0000 7ffaaf400640 1 --2- 172.21.15.42:0/2343583902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffab006b9f0 0x7ffab01418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.045+0000 7ffaaf400640 1 -- 172.21.15.42:0/2343583902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab0133f70 con 0x7ffab00b5ba0 2024-05-28T13:59:11.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.045+0000 7ffaaf400640 1 --2- 172.21.15.42:0/2343583902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab00b5ba0 0x7ffab015efe0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00c750 tx=0x7ffa9c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:11.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.045+0000 7ffaa7e00640 1 -- 172.21.15.42:0/2343583902 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c017070 con 0x7ffab00b5ba0 2024-05-28T13:59:11.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.045+0000 7ffaa7e00640 1 -- 172.21.15.42:0/2343583902 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa9c005290 con 0x7ffab00b5ba0 2024-05-28T13:59:11.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.045+0000 7ffaa7e00640 1 -- 172.21.15.42:0/2343583902 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c003600 con 0x7ffab00b5ba0 2024-05-28T13:59:11.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.045+0000 7ffab7200640 1 -- 172.21.15.42:0/2343583902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab0134250 con 0x7ffab00b5ba0 2024-05-28T13:59:11.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.045+0000 7ffab7200640 1 -- 172.21.15.42:0/2343583902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab0134810 con 0x7ffab00b5ba0 2024-05-28T13:59:11.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.045+0000 7ffab7200640 1 -- 172.21.15.42:0/2343583902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab009f6c0 con 0x7ffab00b5ba0 2024-05-28T13:59:11.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.045+0000 7ffaa7e00640 1 -- 172.21.15.42:0/2343583902 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ffa9c01a070 con 0x7ffab00b5ba0 2024-05-28T13:59:11.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.045+0000 7ffaa7e00640 1 --2- 172.21.15.42:0/2343583902 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa900683b0 0x7ffa9006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.045+0000 7ffaa7e00640 1 -- 172.21.15.42:0/2343583902 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ffa9c085d40 con 0x7ffab00b5ba0 2024-05-28T13:59:11.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.045+0000 7ffaafe00640 1 --2- 172.21.15.42:0/2343583902 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa900683b0 0x7ffa9006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.049+0000 7ffaafe00640 1 --2- 172.21.15.42:0/2343583902 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa900683b0 0x7ffa9006a870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00ad000 tx=0x7ffaa0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:11.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.049+0000 7ffaa7e00640 1 -- 172.21.15.42:0/2343583902 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffab00b04e0 con 0x7ffab00b5ba0 2024-05-28T13:59:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.213+0000 7ffab7200640 1 -- 172.21.15.42:0/2343583902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffab004fdd0 con 0x7ffab00b5ba0 2024-05-28T13:59:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.217+0000 7ffaa7e00640 1 -- 172.21.15.42:0/2343583902 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7ffa9c055c20 con 0x7ffab00b5ba0 2024-05-28T13:59:11.201 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:59:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.217+0000 7ffab7200640 1 -- 172.21.15.42:0/2343583902 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa900683b0 msgr2=0x7ffa9006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.217+0000 7ffab7200640 1 --2- 172.21.15.42:0/2343583902 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa900683b0 0x7ffa9006a870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00ad000 tx=0x7ffaa0008040 comp rx=0 tx=0).stop 2024-05-28T13:59:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.217+0000 7ffab7200640 1 -- 172.21.15.42:0/2343583902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab00b5ba0 msgr2=0x7ffab015efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.217+0000 7ffab7200640 1 --2- 172.21.15.42:0/2343583902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab00b5ba0 0x7ffab015efe0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00c750 tx=0x7ffa9c00cc10 comp rx=0 tx=0).stop 2024-05-28T13:59:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.217+0000 7ffab7200640 1 -- 172.21.15.42:0/2343583902 shutdown_connections 2024-05-28T13:59:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.217+0000 7ffab7200640 1 --2- 172.21.15.42:0/2343583902 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa900683b0 0x7ffa9006a870 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.217+0000 7ffab7200640 1 --2- 172.21.15.42:0/2343583902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffab0145a60 0x7ffab01421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.217+0000 7ffab7200640 1 --2- 172.21.15.42:0/2343583902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab00b5ba0 0x7ffab015efe0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.217+0000 7ffab7200640 1 --2- 172.21.15.42:0/2343583902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffab006b9f0 0x7ffab01418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.217+0000 7ffab7200640 1 -- 172.21.15.42:0/2343583902 >> 172.21.15.42:0/2343583902 conn(0x7ffab00b6880 msgr2=0x7ffab013a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:11.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.217+0000 7ffab7200640 1 -- 172.21.15.42:0/2343583902 shutdown_connections 2024-05-28T13:59:11.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.217+0000 7ffab7200640 1 -- 172.21.15.42:0/2343583902 wait complete. 2024-05-28T13:59:11.222 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:59:11.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8aad200640 1 Processor -- start 2024-05-28T13:59:11.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8aad200640 1 -- start start 2024-05-28T13:59:11.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8aad200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8aa815cb50 0x7f8aa809b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8aad200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8aa80d11d0 0x7f8aa8166470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8aad200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aa809ba40 con 0x7f8aa815c770 2024-05-28T13:59:11.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8aad200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8aa80c6720 con 0x7f8aa815cb50 2024-05-28T13:59:11.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8aad200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8aa80c60d0 con 0x7f8aa80d11d0 2024-05-28T13:59:11.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8a9d400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8aa815cb50 0x7f8aa809b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8aa5e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8aa815c770 0x7f8aa80d0ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46076/0 (socket says 172.21.15.42:46076) 2024-05-28T13:59:11.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8a9d400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8aa815cb50 0x7f8aa809b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48014/0 (socket says 172.21.15.42:48014) 2024-05-28T13:59:11.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8aa5e00640 1 -- 172.21.15.42:0/925444569 learned_addr learned my addr 172.21.15.42:0/925444569 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:11.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8aa6800640 1 --2- 172.21.15.42:0/925444569 >> v2:172.21.15.63:3300/0 conn(0x7f8aa80d11d0 0x7f8aa8166470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8a9d400640 1 -- 172.21.15.42:0/925444569 >> v2:172.21.15.63:3300/0 conn(0x7f8aa80d11d0 msgr2=0x7f8aa8166470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:11.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8a9d400640 1 --2- 172.21.15.42:0/925444569 >> v2:172.21.15.63:3300/0 conn(0x7f8aa80d11d0 0x7f8aa8166470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8a9d400640 1 -- 172.21.15.42:0/925444569 >> v1:172.21.15.42:6789/0 conn(0x7f8aa815c770 legacy=0x7f8aa80d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8aa5400640 1 -- 172.21.15.42:0/925444569 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 754871707 0 0) 0x7f8aa809ba40 con 0x7f8aa815c770 2024-05-28T13:59:11.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.381+0000 7f8a9d400640 1 -- 172.21.15.42:0/925444569 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa80c63b0 con 0x7f8aa815cb50 2024-05-28T13:59:11.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.385+0000 7f8a9d400640 1 --2- 172.21.15.42:0/925444569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8aa815cb50 0x7f8aa809b460 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8160390 tx=0x7f8a9001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dfdfc37a3b8340 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:11.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.385+0000 7f8aa5400640 1 -- 172.21.15.42:0/925444569 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a900b5070 con 0x7f8aa815cb50 2024-05-28T13:59:11.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.385+0000 7f8aa5400640 1 -- 172.21.15.42:0/925444569 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a900a8970 con 0x7f8aa815cb50 2024-05-28T13:59:11.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.385+0000 7f8aa5400640 1 -- 172.21.15.42:0/925444569 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a900a8d40 con 0x7f8aa815cb50 2024-05-28T13:59:11.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.385+0000 7f8aad200640 1 -- 172.21.15.42:0/925444569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8aa815cb50 msgr2=0x7f8aa809b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.385+0000 7f8aad200640 1 --2- 172.21.15.42:0/925444569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8aa815cb50 0x7f8aa809b460 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8160390 tx=0x7f8a9001ec70 comp rx=0 tx=0).stop 2024-05-28T13:59:11.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.385+0000 7f8aad200640 1 -- 172.21.15.42:0/925444569 shutdown_connections 2024-05-28T13:59:11.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.385+0000 7f8aad200640 1 --2- 172.21.15.42:0/925444569 >> v2:172.21.15.63:3300/0 conn(0x7f8aa80d11d0 0x7f8aa8166470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.385+0000 7f8aad200640 1 --2- 172.21.15.42:0/925444569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8aa815cb50 0x7f8aa809b460 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.385+0000 7f8aad200640 1 -- 172.21.15.42:0/925444569 >> 172.21.15.42:0/925444569 conn(0x7f8aa80b3fd0 msgr2=0x7f8aa80b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:11.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.385+0000 7f8aad200640 1 -- 172.21.15.42:0/925444569 shutdown_connections 2024-05-28T13:59:11.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.385+0000 7f8aad200640 1 -- 172.21.15.42:0/925444569 wait complete. 2024-05-28T13:59:11.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aad200640 1 Processor -- start 2024-05-28T13:59:11.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aad200640 1 -- start start 2024-05-28T13:59:11.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aad200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8aa80d11d0 0x7f8aa80b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aad200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8aa815c770 0x7f8aa80b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aad200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8aa815cb50 0x7f8aa80b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aad200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa80c8bc0 con 0x7f8aa815cb50 2024-05-28T13:59:11.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aad200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa80ce2c0 con 0x7f8aa815c770 2024-05-28T13:59:11.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aad200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa80c8d40 con 0x7f8aa80d11d0 2024-05-28T13:59:11.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aa6800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8aa815cb50 0x7f8aa80b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aa6800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8aa815cb50 0x7f8aa80b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48022/0 (socket says 172.21.15.42:48022) 2024-05-28T13:59:11.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aa6800640 1 -- 172.21.15.42:0/379751223 learned_addr learned my addr 172.21.15.42:0/379751223 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:11.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aa5e00640 1 --2- 172.21.15.42:0/379751223 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8aa80d11d0 0x7f8aa80b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8a9d400640 1 --2- 172.21.15.42:0/379751223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8aa815c770 0x7f8aa80b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aa6800640 1 -- 172.21.15.42:0/379751223 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8aa80d11d0 msgr2=0x7f8aa80b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aa6800640 1 --2- 172.21.15.42:0/379751223 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8aa80d11d0 0x7f8aa80b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aa6800640 1 -- 172.21.15.42:0/379751223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8aa815c770 msgr2=0x7f8aa80b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aa6800640 1 --2- 172.21.15.42:0/379751223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8aa815c770 0x7f8aa80b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aa6800640 1 -- 172.21.15.42:0/379751223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa80b7440 con 0x7f8aa815cb50 2024-05-28T13:59:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aa6800640 1 --2- 172.21.15.42:0/379751223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8aa815cb50 0x7f8aa80b21d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0007e30 tx=0x7f8aa000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8a9ea00640 1 -- 172.21.15.42:0/379751223 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0014620 con 0x7f8aa815cb50 2024-05-28T13:59:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8a9ea00640 1 -- 172.21.15.42:0/379751223 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa0019070 con 0x7f8aa815cb50 2024-05-28T13:59:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8a9ea00640 1 -- 172.21.15.42:0/379751223 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0014a00 con 0x7f8aa815cb50 2024-05-28T13:59:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aad200640 1 -- 172.21.15.42:0/379751223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa80b7720 con 0x7f8aa815cb50 2024-05-28T13:59:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.389+0000 7f8aad200640 1 -- 172.21.15.42:0/379751223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa809cff0 con 0x7f8aa815cb50 2024-05-28T13:59:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.393+0000 7f8aad200640 1 -- 172.21.15.42:0/379751223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa80a04d0 con 0x7f8aa815cb50 2024-05-28T13:59:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.393+0000 7f8a9ea00640 1 -- 172.21.15.42:0/379751223 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8aa0014ba0 con 0x7f8aa815cb50 2024-05-28T13:59:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.393+0000 7f8a9ea00640 1 --2- 172.21.15.42:0/379751223 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a84068380 0x7f8a8406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.393+0000 7f8a9ea00640 1 -- 172.21.15.42:0/379751223 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f8aa0087260 con 0x7f8aa815cb50 2024-05-28T13:59:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.393+0000 7f8aa5e00640 1 --2- 172.21.15.42:0/379751223 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a84068380 0x7f8a8406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.393+0000 7f8aa5e00640 1 --2- 172.21.15.42:0/379751223 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a84068380 0x7f8a8406a840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8a98007fd0 tx=0x7f8a98007d60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:11.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.397+0000 7f8a9ea00640 1 -- 172.21.15.42:0/379751223 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8aa00570b0 con 0x7f8aa815cb50 2024-05-28T13:59:11.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.561+0000 7f8aad200640 1 -- 172.21.15.42:0/379751223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8aa815b190 con 0x7f8aa815cb50 2024-05-28T13:59:11.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.565+0000 7f8a9ea00640 1 -- 172.21.15.42:0/379751223 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f8aa005b3b0 con 0x7f8aa815cb50 2024-05-28T13:59:11.549 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:59:11.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.565+0000 7f8aad200640 1 -- 172.21.15.42:0/379751223 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a84068380 msgr2=0x7f8a8406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.565+0000 7f8aad200640 1 --2- 172.21.15.42:0/379751223 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a84068380 0x7f8a8406a840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8a98007fd0 tx=0x7f8a98007d60 comp rx=0 tx=0).stop 2024-05-28T13:59:11.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.565+0000 7f8aad200640 1 -- 172.21.15.42:0/379751223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8aa815cb50 msgr2=0x7f8aa80b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.565+0000 7f8aad200640 1 --2- 172.21.15.42:0/379751223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8aa815cb50 0x7f8aa80b21d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0007e30 tx=0x7f8aa000e4b0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.565+0000 7f8aad200640 1 -- 172.21.15.42:0/379751223 shutdown_connections 2024-05-28T13:59:11.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.565+0000 7f8aad200640 1 --2- 172.21.15.42:0/379751223 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a84068380 0x7f8a8406a840 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.565+0000 7f8aad200640 1 --2- 172.21.15.42:0/379751223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8aa815cb50 0x7f8aa80b21d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.565+0000 7f8aad200640 1 --2- 172.21.15.42:0/379751223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8aa815c770 0x7f8aa80b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.565+0000 7f8aad200640 1 --2- 172.21.15.42:0/379751223 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8aa80d11d0 0x7f8aa80b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.565+0000 7f8aad200640 1 -- 172.21.15.42:0/379751223 >> 172.21.15.42:0/379751223 conn(0x7f8aa80b3fd0 msgr2=0x7f8aa806ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:11.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.565+0000 7f8aad200640 1 -- 172.21.15.42:0/379751223 shutdown_connections 2024-05-28T13:59:11.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.565+0000 7f8aad200640 1 -- 172.21.15.42:0/379751223 wait complete. 2024-05-28T13:59:11.571 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:59:11.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc7a600640 1 Processor -- start 2024-05-28T13:59:11.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc7a600640 1 -- start start 2024-05-28T13:59:11.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc7a600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc7415c8a0 0x7fdc740cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc7a600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdc7409b160 0x7fdc740b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc7a600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc7415cc80 con 0x7fdc740ce090 2024-05-28T13:59:11.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc7a600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fdc740d0280 con 0x7fdc7409b160 2024-05-28T13:59:11.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc7a600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdc740d0560 con 0x7fdc7415c8a0 2024-05-28T13:59:11.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc73e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdc7409b160 0x7fdc740b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc73e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdc7409b160 0x7fdc740b52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48030/0 (socket says 172.21.15.42:48030) 2024-05-28T13:59:11.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc73e00640 1 -- 172.21.15.42:0/3753383044 learned_addr learned my addr 172.21.15.42:0/3753383044 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:11.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc73400640 1 --2- 172.21.15.42:0/3753383044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc7415c8a0 0x7fdc740cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc73e00640 1 -- 172.21.15.42:0/3753383044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc7415c8a0 msgr2=0x7fdc740cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:11.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc73e00640 1 --2- 172.21.15.42:0/3753383044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc7415c8a0 0x7fdc740cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc73e00640 1 -- 172.21.15.42:0/3753383044 >> v1:172.21.15.63:6789/0 conn(0x7fdc740ce090 legacy=0x7fdc7409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.733+0000 7fdc73e00640 1 -- 172.21.15.42:0/3753383044 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc740d0840 con 0x7fdc7409b160 2024-05-28T13:59:11.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.737+0000 7fdc73e00640 1 --2- 172.21.15.42:0/3753383044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc7409b160 0x7fdc740b52f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009e50 tx=0x7fdc6c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cb1e7d9352b869 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:11.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.737+0000 7fdc72a00640 1 -- 172.21.15.42:0/3753383044 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0a5700 con 0x7fdc7409b160 2024-05-28T13:59:11.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.737+0000 7fdc72a00640 1 -- 172.21.15.42:0/3753383044 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6c0a58a0 con 0x7fdc7409b160 2024-05-28T13:59:11.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.737+0000 7fdc72a00640 1 -- 172.21.15.42:0/3753383044 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0a5c70 con 0x7fdc7409b160 2024-05-28T13:59:11.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.737+0000 7fdc7a600640 1 -- 172.21.15.42:0/3753383044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc7409b160 msgr2=0x7fdc740b52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.737+0000 7fdc7a600640 1 --2- 172.21.15.42:0/3753383044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc7409b160 0x7fdc740b52f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009e50 tx=0x7fdc6c01ca30 comp rx=0 tx=0).stop 2024-05-28T13:59:11.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.737+0000 7fdc7a600640 1 -- 172.21.15.42:0/3753383044 shutdown_connections 2024-05-28T13:59:11.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.737+0000 7fdc7a600640 1 --2- 172.21.15.42:0/3753383044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc7409b160 0x7fdc740b52f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.737+0000 7fdc7a600640 1 --2- 172.21.15.42:0/3753383044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc7415c8a0 0x7fdc740cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.737+0000 7fdc7a600640 1 -- 172.21.15.42:0/3753383044 >> 172.21.15.42:0/3753383044 conn(0x7fdc740b3f90 msgr2=0x7fdc740b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:11.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.737+0000 7fdc7a600640 1 -- 172.21.15.42:0/3753383044 shutdown_connections 2024-05-28T13:59:11.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.737+0000 7fdc7a600640 1 -- 172.21.15.42:0/3753383044 wait complete. 2024-05-28T13:59:11.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc7a600640 1 Processor -- start 2024-05-28T13:59:11.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc7a600640 1 -- start start 2024-05-28T13:59:11.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc7a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc7409b160 0x7fdc7415d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc7a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc740ce090 0x7fdc740c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc7a600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc7415c8a0 0x7fdc740ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc7a600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdc740b6040 con 0x7fdc740ce090 2024-05-28T13:59:11.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc7a600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdc740d1530 con 0x7fdc7409b160 2024-05-28T13:59:11.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc7a600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdc740b61c0 con 0x7fdc7415c8a0 2024-05-28T13:59:11.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc6aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc740ce090 0x7fdc740c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc73e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc7415c8a0 0x7fdc740ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc6aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc740ce090 0x7fdc740c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48038/0 (socket says 172.21.15.42:48038) 2024-05-28T13:59:11.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc73400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc7409b160 0x7fdc7415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc6aa00640 1 -- 172.21.15.42:0/463278184 learned_addr learned my addr 172.21.15.42:0/463278184 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:11.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc73e00640 1 -- 172.21.15.42:0/463278184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc7409b160 msgr2=0x7fdc7415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc73e00640 1 --2- 172.21.15.42:0/463278184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc7409b160 0x7fdc7415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc73e00640 1 -- 172.21.15.42:0/463278184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc740ce090 msgr2=0x7fdc740c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc73e00640 1 --2- 172.21.15.42:0/463278184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc740ce090 0x7fdc740c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc73e00640 1 -- 172.21.15.42:0/463278184 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7409be60 con 0x7fdc7415c8a0 2024-05-28T13:59:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc73400640 1 --2- 172.21.15.42:0/463278184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc7409b160 0x7fdc7415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:59:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc6aa00640 1 --2- 172.21.15.42:0/463278184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc740ce090 0x7fdc740c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:59:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc73e00640 1 --2- 172.21.15.42:0/463278184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc7415c8a0 0x7fdc740ca0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0ab000 tx=0x7fdc6c0197d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc6be00640 1 -- 172.21.15.42:0/463278184 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c009510 con 0x7fdc7415c8a0 2024-05-28T13:59:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc6be00640 1 -- 172.21.15.42:0/463278184 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6c0b8470 con 0x7fdc7415c8a0 2024-05-28T13:59:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc6be00640 1 -- 172.21.15.42:0/463278184 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0b8820 con 0x7fdc7415c8a0 2024-05-28T13:59:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc7a600640 1 -- 172.21.15.42:0/463278184 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7415dfb0 con 0x7fdc7415c8a0 2024-05-28T13:59:11.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.741+0000 7fdc7a600640 1 -- 172.21.15.42:0/463278184 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7415e570 con 0x7fdc7415c8a0 2024-05-28T13:59:11.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.745+0000 7fdc7a600640 1 -- 172.21.15.42:0/463278184 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc74154260 con 0x7fdc7415c8a0 2024-05-28T13:59:11.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.745+0000 7fdc6be00640 1 -- 172.21.15.42:0/463278184 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdc6c0c0020 con 0x7fdc7415c8a0 2024-05-28T13:59:11.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.745+0000 7fdc6be00640 1 --2- 172.21.15.42:0/463278184 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc500683b0 0x7fdc5006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:11.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.745+0000 7fdc6be00640 1 -- 172.21.15.42:0/463278184 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fdc6c1211d0 con 0x7fdc7415c8a0 2024-05-28T13:59:11.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.745+0000 7fdc73400640 1 --2- 172.21.15.42:0/463278184 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc500683b0 0x7fdc5006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:11.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.745+0000 7fdc73400640 1 --2- 172.21.15.42:0/463278184 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc500683b0 0x7fdc5006a870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdc740c9780 tx=0x7fdc64009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:11.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.749+0000 7fdc6be00640 1 -- 172.21.15.42:0/463278184 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdc6c016620 con 0x7fdc7415c8a0 2024-05-28T13:59:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.913+0000 7fdc7a600640 1 -- 172.21.15.42:0/463278184 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdc7404fdd0 con 0x7fdc7415c8a0 2024-05-28T13:59:11.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.917+0000 7fdc6be00640 1 -- 172.21.15.42:0/463278184 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fdc6c0f1020 con 0x7fdc7415c8a0 2024-05-28T13:59:11.901 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:59:11.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.917+0000 7fdc7a600640 1 -- 172.21.15.42:0/463278184 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc500683b0 msgr2=0x7fdc5006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.917+0000 7fdc7a600640 1 --2- 172.21.15.42:0/463278184 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc500683b0 0x7fdc5006a870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdc740c9780 tx=0x7fdc64009190 comp rx=0 tx=0).stop 2024-05-28T13:59:11.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.917+0000 7fdc7a600640 1 -- 172.21.15.42:0/463278184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc7415c8a0 msgr2=0x7fdc740ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:11.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.917+0000 7fdc7a600640 1 --2- 172.21.15.42:0/463278184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc7415c8a0 0x7fdc740ca0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0ab000 tx=0x7fdc6c0197d0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.917+0000 7fdc7a600640 1 -- 172.21.15.42:0/463278184 shutdown_connections 2024-05-28T13:59:11.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.917+0000 7fdc7a600640 1 --2- 172.21.15.42:0/463278184 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc500683b0 0x7fdc5006a870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.917+0000 7fdc7a600640 1 --2- 172.21.15.42:0/463278184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc7415c8a0 0x7fdc740ca0d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.917+0000 7fdc7a600640 1 --2- 172.21.15.42:0/463278184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc740ce090 0x7fdc740c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.917+0000 7fdc7a600640 1 --2- 172.21.15.42:0/463278184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc7409b160 0x7fdc7415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.917+0000 7fdc7a600640 1 -- 172.21.15.42:0/463278184 >> 172.21.15.42:0/463278184 conn(0x7fdc740b3f90 msgr2=0x7fdc740b7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.917+0000 7fdc7a600640 1 -- 172.21.15.42:0/463278184 shutdown_connections 2024-05-28T13:59:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:11.917+0000 7fdc7a600640 1 -- 172.21.15.42:0/463278184 wait complete. 2024-05-28T13:59:11.923 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T13:59:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f937de00640 1 Processor -- start 2024-05-28T13:59:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f937de00640 1 -- start start 2024-05-28T13:59:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f937de00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9378145920 0x7f937806f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f937de00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f937806db40 0x7f9378071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f937de00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9378072300 0x7f9378160100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f937de00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9378072750 con 0x7f9378072300 2024-05-28T13:59:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f937de00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f937806dfa0 con 0x7f937806db40 2024-05-28T13:59:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f937de00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f937806e240 con 0x7f9378145920 2024-05-28T13:59:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f9377400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9378072300 0x7f9378160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f9377400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9378072300 0x7f9378160100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48054/0 (socket says 172.21.15.42:48054) 2024-05-28T13:59:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f9377400640 1 -- 172.21.15.42:0/2043626569 learned_addr learned my addr 172.21.15.42:0/2043626569 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f936fe00640 1 --2- 172.21.15.42:0/2043626569 >> v2:172.21.15.63:3300/0 conn(0x7f937806db40 0x7f9378071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f9376a00640 1 --2- 172.21.15.42:0/2043626569 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9378145920 0x7f937806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f9377400640 1 -- 172.21.15.42:0/2043626569 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9378145920 msgr2=0x7f937806f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f9377400640 1 --2- 172.21.15.42:0/2043626569 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9378145920 0x7f937806f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f9377400640 1 -- 172.21.15.42:0/2043626569 >> v2:172.21.15.63:3300/0 conn(0x7f937806db40 msgr2=0x7f9378071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f9377400640 1 --2- 172.21.15.42:0/2043626569 >> v2:172.21.15.63:3300/0 conn(0x7f937806db40 0x7f9378071dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f9377400640 1 -- 172.21.15.42:0/2043626569 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937806e520 con 0x7f9378072300 2024-05-28T13:59:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.081+0000 7f936fe00640 1 --2- 172.21.15.42:0/2043626569 >> v2:172.21.15.63:3300/0 conn(0x7f937806db40 0x7f9378071dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:12.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.085+0000 7f9377400640 1 --2- 172.21.15.42:0/2043626569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9378072300 0x7f9378160100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f937806c850 tx=0x7f937001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c20af8cdb498c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.085+0000 7f9376000640 1 -- 172.21.15.42:0/2043626569 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93700b0070 con 0x7f9378072300 2024-05-28T13:59:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.085+0000 7f9376000640 1 -- 172.21.15.42:0/2043626569 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93700ab420 con 0x7f9378072300 2024-05-28T13:59:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.085+0000 7f9376000640 1 -- 172.21.15.42:0/2043626569 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93700ab7f0 con 0x7f9378072300 2024-05-28T13:59:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.085+0000 7f937de00640 1 -- 172.21.15.42:0/2043626569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9378072300 msgr2=0x7f9378160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.085+0000 7f937de00640 1 --2- 172.21.15.42:0/2043626569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9378072300 0x7f9378160100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f937806c850 tx=0x7f937001c6a0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.085+0000 7f937de00640 1 -- 172.21.15.42:0/2043626569 shutdown_connections 2024-05-28T13:59:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.085+0000 7f937de00640 1 --2- 172.21.15.42:0/2043626569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9378072300 0x7f9378160100 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.085+0000 7f937de00640 1 --2- 172.21.15.42:0/2043626569 >> v2:172.21.15.63:3300/0 conn(0x7f937806db40 0x7f9378071dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.085+0000 7f937de00640 1 --2- 172.21.15.42:0/2043626569 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9378145920 0x7f937806f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.085+0000 7f937de00640 1 -- 172.21.15.42:0/2043626569 >> 172.21.15.42:0/2043626569 conn(0x7f937806b9f0 msgr2=0x7f937806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.085+0000 7f937de00640 1 -- 172.21.15.42:0/2043626569 shutdown_connections 2024-05-28T13:59:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.085+0000 7f937de00640 1 -- 172.21.15.42:0/2043626569 wait complete. 2024-05-28T13:59:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f937de00640 1 Processor -- start 2024-05-28T13:59:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f937de00640 1 -- start start 2024-05-28T13:59:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f937de00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f937806db40 0x7f93780b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f937de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9378072300 0x7f93780ac100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f937de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9378145920 0x7f93780aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f937de00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f937815d2a0 con 0x7f9378145920 2024-05-28T13:59:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f937de00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f937815d120 con 0x7f9378072300 2024-05-28T13:59:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f937de00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f937815d420 con 0x7f937806db40 2024-05-28T13:59:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f9377400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9378145920 0x7f93780aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f9377400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9378145920 0x7f93780aeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48056/0 (socket says 172.21.15.42:48056) 2024-05-28T13:59:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f9377400640 1 -- 172.21.15.42:0/3825732396 learned_addr learned my addr 172.21.15.42:0/3825732396 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f9376a00640 1 --2- 172.21.15.42:0/3825732396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f937806db40 0x7f93780b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f936fe00640 1 --2- 172.21.15.42:0/3825732396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9378072300 0x7f93780ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f9377400640 1 -- 172.21.15.42:0/3825732396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f937806db40 msgr2=0x7f93780b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f9377400640 1 --2- 172.21.15.42:0/3825732396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f937806db40 0x7f93780b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f9377400640 1 -- 172.21.15.42:0/3825732396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9378072300 msgr2=0x7f93780ac100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f9377400640 1 --2- 172.21.15.42:0/3825732396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9378072300 0x7f93780ac100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f9377400640 1 -- 172.21.15.42:0/3825732396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93780b0160 con 0x7f9378145920 2024-05-28T13:59:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f9377400640 1 --2- 172.21.15.42:0/3825732396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9378145920 0x7f93780aeae0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f93700ae040 tx=0x7f9370019440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f936f400640 1 -- 172.21.15.42:0/3825732396 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93700ba020 con 0x7f9378145920 2024-05-28T13:59:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f936f400640 1 -- 172.21.15.42:0/3825732396 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93700090d0 con 0x7f9378145920 2024-05-28T13:59:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f936f400640 1 -- 172.21.15.42:0/3825732396 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93700b0040 con 0x7f9378145920 2024-05-28T13:59:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f937de00640 1 -- 172.21.15.42:0/3825732396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93780b0400 con 0x7f9378145920 2024-05-28T13:59:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f937de00640 1 -- 172.21.15.42:0/3825732396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93780b09c0 con 0x7f9378145920 2024-05-28T13:59:12.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.089+0000 7f937de00640 1 -- 172.21.15.42:0/3825732396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9378073890 con 0x7f9378145920 2024-05-28T13:59:12.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.093+0000 7f936f400640 1 -- 172.21.15.42:0/3825732396 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f93700bd020 con 0x7f9378145920 2024-05-28T13:59:12.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.093+0000 7f936f400640 1 --2- 172.21.15.42:0/3825732396 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9354068380 0x7f935406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.093+0000 7f936f400640 1 -- 172.21.15.42:0/3825732396 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f9370120d30 con 0x7f9378145920 2024-05-28T13:59:12.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.093+0000 7f9376a00640 1 --2- 172.21.15.42:0/3825732396 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9354068380 0x7f935406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.093+0000 7f9376a00640 1 --2- 172.21.15.42:0/3825732396 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9354068380 0x7f935406a840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9368007940 tx=0x7f9368008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:12.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.097+0000 7f936f400640 1 -- 172.21.15.42:0/3825732396 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f93700c4030 con 0x7f9378145920 2024-05-28T13:59:12.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.273+0000 7f937de00640 1 -- 172.21.15.42:0/3825732396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f937804fdd0 con 0x7f9378145920 2024-05-28T13:59:12.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.277+0000 7f936f400640 1 -- 172.21.15.42:0/3825732396 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f93700165e0 con 0x7f9378145920 2024-05-28T13:59:12.261 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T13:59:12.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.277+0000 7f937de00640 1 -- 172.21.15.42:0/3825732396 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9354068380 msgr2=0x7f935406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.277+0000 7f937de00640 1 --2- 172.21.15.42:0/3825732396 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9354068380 0x7f935406a840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9368007940 tx=0x7f9368008040 comp rx=0 tx=0).stop 2024-05-28T13:59:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.277+0000 7f937de00640 1 -- 172.21.15.42:0/3825732396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9378145920 msgr2=0x7f93780aeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.277+0000 7f937de00640 1 --2- 172.21.15.42:0/3825732396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9378145920 0x7f93780aeae0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f93700ae040 tx=0x7f9370019440 comp rx=0 tx=0).stop 2024-05-28T13:59:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.277+0000 7f937de00640 1 -- 172.21.15.42:0/3825732396 shutdown_connections 2024-05-28T13:59:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.277+0000 7f937de00640 1 --2- 172.21.15.42:0/3825732396 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9354068380 0x7f935406a840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.277+0000 7f937de00640 1 --2- 172.21.15.42:0/3825732396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9378145920 0x7f93780aeae0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.277+0000 7f937de00640 1 --2- 172.21.15.42:0/3825732396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9378072300 0x7f93780ac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.277+0000 7f937de00640 1 --2- 172.21.15.42:0/3825732396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f937806db40 0x7f93780b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.277+0000 7f937de00640 1 -- 172.21.15.42:0/3825732396 >> 172.21.15.42:0/3825732396 conn(0x7f937806b9f0 msgr2=0x7f937806bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.277+0000 7f937de00640 1 -- 172.21.15.42:0/3825732396 shutdown_connections 2024-05-28T13:59:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.277+0000 7f937de00640 1 -- 172.21.15.42:0/3825732396 wait complete. 2024-05-28T13:59:12.283 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi063.front.sepia.ceph.com... 2024-05-28T13:59:12.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:12.283 DEBUG:teuthology.orchestra.run.smithi063:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-05-28T13:59:12.306 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T13:59:12.306 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T13:59:12.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:12.307 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-28T13:59:12.359 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-28T13:59:12.359 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T13:59:12.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.513+0000 7f07c0600640 1 Processor -- start 2024-05-28T13:59:12.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.513+0000 7f07c0600640 1 -- start start 2024-05-28T13:59:12.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07c0600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07b806f660 0x7f07b8073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07c0600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f07b80d0e00 0x7f07b806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07c0600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b815f280 con 0x7f07b815ba40 2024-05-28T13:59:12.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07c0600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f07b815f030 con 0x7f07b80d0e00 2024-05-28T13:59:12.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07c0600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f07b815f560 con 0x7f07b806f660 2024-05-28T13:59:12.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07bd200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f07b815ba40 0x7f07b80d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46082/0 (socket says 172.21.15.42:46082) 2024-05-28T13:59:12.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07bd200640 1 -- 172.21.15.42:0/3245768590 learned_addr learned my addr 172.21.15.42:0/3245768590 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:12.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07bdc00640 1 --2- 172.21.15.42:0/3245768590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07b806f660 0x7f07b8073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07be600640 1 --2- 172.21.15.42:0/3245768590 >> v2:172.21.15.63:3300/0 conn(0x7f07b80d0e00 0x7f07b806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 -- 172.21.15.42:0/3245768590 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 753792288 0 0) 0x7f07b815f280 con 0x7f07b815ba40 2024-05-28T13:59:12.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 -- 172.21.15.42:0/3245768590 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07a0003530 con 0x7f07b815ba40 2024-05-28T13:59:12.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 -- 172.21.15.42:0/3245768590 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 133326435 0 0) 0x7f07a0003530 con 0x7f07b815ba40 2024-05-28T13:59:12.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 -- 172.21.15.42:0/3245768590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07b806f660 msgr2=0x7f07b8073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:12.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 --2- 172.21.15.42:0/3245768590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07b806f660 0x7f07b8073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 -- 172.21.15.42:0/3245768590 >> v2:172.21.15.63:3300/0 conn(0x7f07b80d0e00 msgr2=0x7f07b806dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:12.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 --2- 172.21.15.42:0/3245768590 >> v2:172.21.15.63:3300/0 conn(0x7f07b80d0e00 0x7f07b806dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 -- 172.21.15.42:0/3245768590 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b815f840 con 0x7f07b815ba40 2024-05-28T13:59:12.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 -- 172.21.15.42:0/3245768590 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f07a4001680 con 0x7f07b815ba40 2024-05-28T13:59:12.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 -- 172.21.15.42:0/3245768590 >> v1:172.21.15.42:6789/0 conn(0x7f07b815ba40 legacy=0x7f07b80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:12.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 --2- 172.21.15.42:0/3245768590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07b806f660 0x7f07a00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 --2- 172.21.15.42:0/3245768590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07a0003f40 0x7f07a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 --2- 172.21.15.42:0/3245768590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07a0004d50 0x7f07a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 -- 172.21.15.42:0/3245768590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f07b815f030 con 0x7f07a0004d50 2024-05-28T13:59:12.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 -- 172.21.15.42:0/3245768590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07b815f560 con 0x7f07a0003f40 2024-05-28T13:59:12.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07b7e00640 1 -- 172.21.15.42:0/3245768590 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f07a000bab0 con 0x7f07b806f660 2024-05-28T13:59:12.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07be600640 1 --2- 172.21.15.42:0/3245768590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07a0004d50 0x7f07a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07bdc00640 1 --2- 172.21.15.42:0/3245768590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07b806f660 0x7f07a00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07bd200640 1 --2- 172.21.15.42:0/3245768590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07a0003f40 0x7f07a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07bdc00640 1 -- 172.21.15.42:0/3245768590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07a0003f40 msgr2=0x7f07a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:12.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07bdc00640 1 --2- 172.21.15.42:0/3245768590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07a0003f40 0x7f07a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07bdc00640 1 -- 172.21.15.42:0/3245768590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07a0004d50 msgr2=0x7f07a00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:12.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07bdc00640 1 --2- 172.21.15.42:0/3245768590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07a0004d50 0x7f07a00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07bdc00640 1 -- 172.21.15.42:0/3245768590 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a0003530 con 0x7f07b806f660 2024-05-28T13:59:12.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.517+0000 7f07be600640 1 --2- 172.21.15.42:0/3245768590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07a0004d50 0x7f07a00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:59:12.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.521+0000 7f07bdc00640 1 --2- 172.21.15.42:0/3245768590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07b806f660 0x7f07a00039c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009ae0 tx=0x7f07ac01ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7370ddf0205d82c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:12.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.521+0000 7f07b7e00640 1 -- 172.21.15.42:0/3245768590 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac0b4070 con 0x7f07b806f660 2024-05-28T13:59:12.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.521+0000 7f07b7e00640 1 -- 172.21.15.42:0/3245768590 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ac0a7b10 con 0x7f07b806f660 2024-05-28T13:59:12.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.521+0000 7f07b7e00640 1 -- 172.21.15.42:0/3245768590 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac0096b0 con 0x7f07b806f660 2024-05-28T13:59:12.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.521+0000 7f07c0600640 1 -- 172.21.15.42:0/3245768590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07b806f660 msgr2=0x7f07a00039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:12.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.521+0000 7f07c0600640 1 --2- 172.21.15.42:0/3245768590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07b806f660 0x7f07a00039c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009ae0 tx=0x7f07ac01ed10 comp rx=0 tx=0).stop 2024-05-28T13:59:12.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.521+0000 7f07be600640 1 -- 172.21.15.42:0/3245768590 reap_dead start 2024-05-28T13:59:12.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.521+0000 7f07c0600640 1 -- 172.21.15.42:0/3245768590 shutdown_connections 2024-05-28T13:59:12.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.521+0000 7f07c0600640 1 -- 172.21.15.42:0/3245768590 >> 172.21.15.42:0/3245768590 conn(0x7f07b806bdf0 msgr2=0x7f07b806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:12.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.521+0000 7f07c0600640 1 -- 172.21.15.42:0/3245768590 shutdown_connections 2024-05-28T13:59:12.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.521+0000 7f07c0600640 1 -- 172.21.15.42:0/3245768590 wait complete. 2024-05-28T13:59:12.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07c0600640 1 Processor -- start 2024-05-28T13:59:12.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07c0600640 1 -- start start 2024-05-28T13:59:12.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07c0600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07b806f660 0x7f07b814bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07c0600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07b80ccd00 0x7f07b80cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07c0600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b81e7f90 0x7f07b81e8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07c0600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8073f10 con 0x7f07b80ccd00 2024-05-28T13:59:12.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07c0600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8073d90 con 0x7f07b81e7f90 2024-05-28T13:59:12.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07c0600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f07b80b32e0 con 0x7f07b806f660 2024-05-28T13:59:12.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07bd200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07b80ccd00 0x7f07b80cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07bd200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07b80ccd00 0x7f07b80cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48078/0 (socket says 172.21.15.42:48078) 2024-05-28T13:59:12.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07bd200640 1 -- 172.21.15.42:0/370711790 learned_addr learned my addr 172.21.15.42:0/370711790 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:12.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07bdc00640 1 --2- 172.21.15.42:0/370711790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07b806f660 0x7f07b814bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07be600640 1 --2- 172.21.15.42:0/370711790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b81e7f90 0x7f07b81e8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07bd200640 1 -- 172.21.15.42:0/370711790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07b806f660 msgr2=0x7f07b814bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07bd200640 1 --2- 172.21.15.42:0/370711790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07b806f660 0x7f07b814bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07bd200640 1 -- 172.21.15.42:0/370711790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b81e7f90 msgr2=0x7f07b81e8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07bd200640 1 --2- 172.21.15.42:0/370711790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b81e7f90 0x7f07b81e8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07bd200640 1 -- 172.21.15.42:0/370711790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b81e73c0 con 0x7f07b80ccd00 2024-05-28T13:59:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07bd200640 1 --2- 172.21.15.42:0/370711790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07b80ccd00 0x7f07b80cd180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f07a4007340 tx=0x7f07a4007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07b5600640 1 -- 172.21.15.42:0/370711790 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a400f9c0 con 0x7f07b80ccd00 2024-05-28T13:59:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07b5600640 1 -- 172.21.15.42:0/370711790 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a4018070 con 0x7f07b80ccd00 2024-05-28T13:59:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07b5600640 1 -- 172.21.15.42:0/370711790 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a400fd60 con 0x7f07b80ccd00 2024-05-28T13:59:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07c0600640 1 -- 172.21.15.42:0/370711790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b814c130 con 0x7f07b80ccd00 2024-05-28T13:59:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.525+0000 7f07c0600640 1 -- 172.21.15.42:0/370711790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b80b0ae0 con 0x7f07b80ccd00 2024-05-28T13:59:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.529+0000 7f07c0600640 1 -- 172.21.15.42:0/370711790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b8068a90 con 0x7f07b80ccd00 2024-05-28T13:59:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.529+0000 7f07b5600640 1 -- 172.21.15.42:0/370711790 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f07a401d460 con 0x7f07b80ccd00 2024-05-28T13:59:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.529+0000 7f07b5600640 1 --2- 172.21.15.42:0/370711790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f07980683b0 0x7f079806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.529+0000 7f07b5600640 1 -- 172.21.15.42:0/370711790 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f07a4086810 con 0x7f07b80ccd00 2024-05-28T13:59:12.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.529+0000 7f07bdc00640 1 --2- 172.21.15.42:0/370711790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f07980683b0 0x7f079806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.529+0000 7f07bdc00640 1 --2- 172.21.15.42:0/370711790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f07980683b0 0x7f079806a870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0b1000 tx=0x7f07ac0046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:12.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.533+0000 7f07b5600640 1 -- 172.21.15.42:0/370711790 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f07a40566f0 con 0x7f07b80ccd00 2024-05-28T13:59:12.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.733+0000 7f07c0600640 1 -- 172.21.15.42:0/370711790 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07b8072bb0 con 0x7f07b80ccd00 2024-05-28T13:59:12.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.733+0000 7f07b5600640 1 -- 172.21.15.42:0/370711790 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f07a405a9f0 con 0x7f07b80ccd00 2024-05-28T13:59:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:12.720 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1409117123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1409117123},{"type":"v1","addr":"172.21.15.42:6837","nonce":1409117123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/3753517701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3753517701},{"type":"v1","addr":"172.21.15.42:6836","nonce":3753517701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/882866492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":882866492},{"type":"v1","addr":"172.21.15.63:6834","nonce":882866492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T13:58:23.075354+0000","modified":"2024-05-28T13:58:26.084895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/1552779700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1552779700},{"type":"v1","addr":"172.21.15.63:6835","nonce":1552779700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-05-28T13:59:12.720 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-05-28T13:59:12.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.737+0000 7f07c0600640 1 -- 172.21.15.42:0/370711790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f07980683b0 msgr2=0x7f079806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:12.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.737+0000 7f07c0600640 1 --2- 172.21.15.42:0/370711790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f07980683b0 0x7f079806a870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0b1000 tx=0x7f07ac0046d0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.737+0000 7f07c0600640 1 -- 172.21.15.42:0/370711790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07b80ccd00 msgr2=0x7f07b80cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:12.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.737+0000 7f07c0600640 1 --2- 172.21.15.42:0/370711790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07b80ccd00 0x7f07b80cd180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f07a4007340 tx=0x7f07a4007800 comp rx=0 tx=0).stop 2024-05-28T13:59:12.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.737+0000 7f07c0600640 1 -- 172.21.15.42:0/370711790 shutdown_connections 2024-05-28T13:59:12.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.737+0000 7f07c0600640 1 --2- 172.21.15.42:0/370711790 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f07980683b0 0x7f079806a870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.737+0000 7f07c0600640 1 --2- 172.21.15.42:0/370711790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b81e7f90 0x7f07b81e8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.737+0000 7f07c0600640 1 --2- 172.21.15.42:0/370711790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07b80ccd00 0x7f07b80cd180 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.737+0000 7f07c0600640 1 --2- 172.21.15.42:0/370711790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07b806f660 0x7f07b814bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.737+0000 7f07c0600640 1 -- 172.21.15.42:0/370711790 >> 172.21.15.42:0/370711790 conn(0x7f07b806bdf0 msgr2=0x7f07b806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:12.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.737+0000 7f07c0600640 1 -- 172.21.15.42:0/370711790 shutdown_connections 2024-05-28T13:59:12.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.737+0000 7f07c0600640 1 -- 172.21.15.42:0/370711790 wait complete. 2024-05-28T13:59:12.743 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:59:12.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.897+0000 7f9fc8200640 1 Processor -- start 2024-05-28T13:59:12.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.901+0000 7f9fc8200640 1 -- start start 2024-05-28T13:59:12.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.901+0000 7f9fc8200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9fc00997f0 0x7f9fc006f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.901+0000 7f9fc8200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00b5f10 0x7f9fc0073130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.901+0000 7f9fc8200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9fc0073670 0x7f9fc00b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.901+0000 7f9fc8200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9fc016f060 con 0x7f9fc00997f0 2024-05-28T13:59:12.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.901+0000 7f9fc8200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9fc016f340 con 0x7f9fc0073670 2024-05-28T13:59:12.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.901+0000 7f9fc8200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc016f620 con 0x7f9fc00b5f10 2024-05-28T13:59:12.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.901+0000 7f9fc5800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9fc00997f0 0x7f9fc006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.901+0000 7f9fc5800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9fc00997f0 0x7f9fc006f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48088/0 (socket says 172.21.15.42:48088) 2024-05-28T13:59:12.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.901+0000 7f9fc6200640 1 --2- 172.21.15.42:0/561209816 >> v2:172.21.15.63:3300/0 conn(0x7f9fc0073670 0x7f9fc00b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.901+0000 7f9fc4e00640 1 --2- 172.21.15.42:0/561209816 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00b5f10 0x7f9fc0073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.901+0000 7f9fc5800640 1 -- 172.21.15.42:0/561209816 learned_addr learned my addr 172.21.15.42:0/561209816 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:12.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.917+0000 7f9fc5800640 1 -- 172.21.15.42:0/561209816 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00b5f10 msgr2=0x7f9fc0073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:12.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.917+0000 7f9fc5800640 1 --2- 172.21.15.42:0/561209816 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00b5f10 0x7f9fc0073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.917+0000 7f9fc5800640 1 -- 172.21.15.42:0/561209816 >> v2:172.21.15.63:3300/0 conn(0x7f9fc0073670 msgr2=0x7f9fc00b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:12.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.917+0000 7f9fc5800640 1 --2- 172.21.15.42:0/561209816 >> v2:172.21.15.63:3300/0 conn(0x7f9fc0073670 0x7f9fc00b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.917+0000 7f9fc5800640 1 -- 172.21.15.42:0/561209816 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc016f900 con 0x7f9fc00997f0 2024-05-28T13:59:12.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.921+0000 7f9fc5800640 1 --2- 172.21.15.42:0/561209816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc00997f0 0x7f9fc006f730 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9fc016cd90 tx=0x7f9fb401e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c549e76dd8e084b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:12.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.921+0000 7f9fbfe00640 1 -- 172.21.15.42:0/561209816 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40ad070 con 0x7f9fc00997f0 2024-05-28T13:59:12.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.921+0000 7f9fbfe00640 1 -- 172.21.15.42:0/561209816 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb4004230 con 0x7f9fc00997f0 2024-05-28T13:59:12.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.921+0000 7f9fbfe00640 1 -- 172.21.15.42:0/561209816 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40a7650 con 0x7f9fc00997f0 2024-05-28T13:59:12.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.921+0000 7f9fc8200640 1 -- 172.21.15.42:0/561209816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc00997f0 msgr2=0x7f9fc006f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:12.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.921+0000 7f9fc8200640 1 --2- 172.21.15.42:0/561209816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc00997f0 0x7f9fc006f730 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9fc016cd90 tx=0x7f9fb401e8b0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.921+0000 7f9fc8200640 1 -- 172.21.15.42:0/561209816 shutdown_connections 2024-05-28T13:59:12.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.921+0000 7f9fc8200640 1 --2- 172.21.15.42:0/561209816 >> v2:172.21.15.63:3300/0 conn(0x7f9fc0073670 0x7f9fc00b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.921+0000 7f9fc8200640 1 --2- 172.21.15.42:0/561209816 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00b5f10 0x7f9fc0073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.921+0000 7f9fc8200640 1 --2- 172.21.15.42:0/561209816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc00997f0 0x7f9fc006f730 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.921+0000 7f9fc8200640 1 -- 172.21.15.42:0/561209816 >> 172.21.15.42:0/561209816 conn(0x7f9fc0072280 msgr2=0x7f9fc00b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:12.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.921+0000 7f9fc8200640 1 -- 172.21.15.42:0/561209816 shutdown_connections 2024-05-28T13:59:12.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.921+0000 7f9fc8200640 1 -- 172.21.15.42:0/561209816 wait complete. 2024-05-28T13:59:12.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc8200640 1 Processor -- start 2024-05-28T13:59:12.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc8200640 1 -- start start 2024-05-28T13:59:12.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc8200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc0073670 0x7f9fc008dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc8200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fc00997f0 0x7f9fc00b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc8200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00b5f10 0x7f9fc00b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc8200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc016ba60 con 0x7f9fc0073670 2024-05-28T13:59:12.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc8200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0073a80 con 0x7f9fc00997f0 2024-05-28T13:59:12.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc8200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc016bbe0 con 0x7f9fc00b5f10 2024-05-28T13:59:12.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc0073670 0x7f9fc008dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc0073670 0x7f9fc008dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48100/0 (socket says 172.21.15.42:48100) 2024-05-28T13:59:12.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc5800640 1 -- 172.21.15.42:0/2905411636 learned_addr learned my addr 172.21.15.42:0/2905411636 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:12.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc4e00640 1 --2- 172.21.15.42:0/2905411636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fc00997f0 0x7f9fc00b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc6200640 1 --2- 172.21.15.42:0/2905411636 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00b5f10 0x7f9fc00b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc5800640 1 -- 172.21.15.42:0/2905411636 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00b5f10 msgr2=0x7f9fc00b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:12.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc5800640 1 --2- 172.21.15.42:0/2905411636 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00b5f10 0x7f9fc00b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc5800640 1 -- 172.21.15.42:0/2905411636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fc00997f0 msgr2=0x7f9fc00b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:12.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc5800640 1 --2- 172.21.15.42:0/2905411636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fc00997f0 0x7f9fc00b21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:12.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc5800640 1 -- 172.21.15.42:0/2905411636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc00c9cb0 con 0x7f9fc0073670 2024-05-28T13:59:12.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc5800640 1 --2- 172.21.15.42:0/2905411636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc0073670 0x7f9fc008dd80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9fb401cf60 tx=0x7f9fb401b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:12.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fbd600640 1 -- 172.21.15.42:0/2905411636 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40be070 con 0x7f9fc0073670 2024-05-28T13:59:12.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fbd600640 1 -- 172.21.15.42:0/2905411636 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb4004030 con 0x7f9fc0073670 2024-05-28T13:59:12.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fbd600640 1 -- 172.21.15.42:0/2905411636 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40ad040 con 0x7f9fc0073670 2024-05-28T13:59:12.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc8200640 1 -- 172.21.15.42:0/2905411636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc0090da0 con 0x7f9fc0073670 2024-05-28T13:59:12.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.925+0000 7f9fc8200640 1 -- 172.21.15.42:0/2905411636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc00ca250 con 0x7f9fc0073670 2024-05-28T13:59:12.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.929+0000 7f9fc8200640 1 -- 172.21.15.42:0/2905411636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc00ca450 con 0x7f9fc0073670 2024-05-28T13:59:12.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.929+0000 7f9fbd600640 1 -- 172.21.15.42:0/2905411636 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9fb4003620 con 0x7f9fc0073670 2024-05-28T13:59:12.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.929+0000 7f9fbd600640 1 --2- 172.21.15.42:0/2905411636 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fa00683b0 0x7f9fa006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:12.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.929+0000 7f9fbd600640 1 -- 172.21.15.42:0/2905411636 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f9fb4120840 con 0x7f9fc0073670 2024-05-28T13:59:12.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.929+0000 7f9fc4e00640 1 --2- 172.21.15.42:0/2905411636 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fa00683b0 0x7f9fa006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:12.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.929+0000 7f9fc4e00640 1 --2- 172.21.15.42:0/2905411636 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fa00683b0 0x7f9fa006a870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00913a0 tx=0x7f9fac005ff0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:12.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:12.933+0000 7f9fbd600640 1 -- 172.21.15.42:0/2905411636 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9fc00ca450 con 0x7f9fc0073670 2024-05-28T13:59:13.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.101+0000 7f9fc8200640 1 -- 172.21.15.42:0/2905411636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fc0068a90 con 0x7f9fc0073670 2024-05-28T13:59:13.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.101+0000 7f9fbd600640 1 -- 172.21.15.42:0/2905411636 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16737 (secure 0 0 0) 0x7f9fb40185c0 con 0x7f9fc0073670 2024-05-28T13:59:13.087 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:13.087 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:59:03.992606+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:59:13.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.105+0000 7f9fc8200640 1 -- 172.21.15.42:0/2905411636 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fa00683b0 msgr2=0x7f9fa006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.105+0000 7f9fc8200640 1 --2- 172.21.15.42:0/2905411636 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fa00683b0 0x7f9fa006a870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00913a0 tx=0x7f9fac005ff0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.105+0000 7f9fc8200640 1 -- 172.21.15.42:0/2905411636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc0073670 msgr2=0x7f9fc008dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.105+0000 7f9fc8200640 1 --2- 172.21.15.42:0/2905411636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc0073670 0x7f9fc008dd80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9fb401cf60 tx=0x7f9fb401b650 comp rx=0 tx=0).stop 2024-05-28T13:59:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.105+0000 7f9fc8200640 1 -- 172.21.15.42:0/2905411636 shutdown_connections 2024-05-28T13:59:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.105+0000 7f9fc8200640 1 --2- 172.21.15.42:0/2905411636 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fa00683b0 0x7f9fa006a870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.105+0000 7f9fc8200640 1 --2- 172.21.15.42:0/2905411636 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00b5f10 0x7f9fc00b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.105+0000 7f9fc8200640 1 --2- 172.21.15.42:0/2905411636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fc00997f0 0x7f9fc00b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.105+0000 7f9fc8200640 1 --2- 172.21.15.42:0/2905411636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc0073670 0x7f9fc008dd80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.105+0000 7f9fc8200640 1 -- 172.21.15.42:0/2905411636 >> 172.21.15.42:0/2905411636 conn(0x7f9fc0072280 msgr2=0x7f9fc006f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.105+0000 7f9fc8200640 1 -- 172.21.15.42:0/2905411636 shutdown_connections 2024-05-28T13:59:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.105+0000 7f9fc8200640 1 -- 172.21.15.42:0/2905411636 wait complete. 2024-05-28T13:59:13.109 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:58:20.789974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:58:22.021080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:59:13.109 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:59:13.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f1412600640 1 Processor -- start 2024-05-28T13:59:13.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f1412600640 1 -- start start 2024-05-28T13:59:13.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f1412600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f140c144f80 0x7f140c072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f1412600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f140c06ee20 0x7f140c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f1412600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f140c15b9b0 0x7f140c15be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f1412600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f140c06f7c0 con 0x7f140c06ee20 2024-05-28T13:59:13.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f1412600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f140c0724d0 con 0x7f140c144f80 2024-05-28T13:59:13.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f1412600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f140c0afe40 con 0x7f140c15b9b0 2024-05-28T13:59:13.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f140aa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f140c06ee20 0x7f140c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f140aa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f140c06ee20 0x7f140c06f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48110/0 (socket says 172.21.15.42:48110) 2024-05-28T13:59:13.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f140aa00640 1 -- 172.21.15.42:0/874044922 learned_addr learned my addr 172.21.15.42:0/874044922 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:13.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f140be00640 1 --2- 172.21.15.42:0/874044922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f140c15b9b0 0x7f140c15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f140b400640 1 --2- 172.21.15.42:0/874044922 >> v2:172.21.15.63:3300/0 conn(0x7f140c144f80 0x7f140c072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f140aa00640 1 -- 172.21.15.42:0/874044922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f140c15b9b0 msgr2=0x7f140c15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:13.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f140aa00640 1 --2- 172.21.15.42:0/874044922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f140c15b9b0 0x7f140c15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f140aa00640 1 -- 172.21.15.42:0/874044922 >> v2:172.21.15.63:3300/0 conn(0x7f140c144f80 msgr2=0x7f140c072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:13.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f140aa00640 1 --2- 172.21.15.42:0/874044922 >> v2:172.21.15.63:3300/0 conn(0x7f140c144f80 0x7f140c072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.269+0000 7f140aa00640 1 -- 172.21.15.42:0/874044922 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140c15c380 con 0x7f140c06ee20 2024-05-28T13:59:13.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f140aa00640 1 --2- 172.21.15.42:0/874044922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f140c06ee20 0x7f140c06f280 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f140c071760 tx=0x7f13f801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5b3a266d2237f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:13.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f140a000640 1 -- 172.21.15.42:0/874044922 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f80b1070 con 0x7f140c06ee20 2024-05-28T13:59:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f140a000640 1 -- 172.21.15.42:0/874044922 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f80ac630 con 0x7f140c06ee20 2024-05-28T13:59:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f140a000640 1 -- 172.21.15.42:0/874044922 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f80a8050 con 0x7f140c06ee20 2024-05-28T13:59:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f1412600640 1 -- 172.21.15.42:0/874044922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f140c06ee20 msgr2=0x7f140c06f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f1412600640 1 --2- 172.21.15.42:0/874044922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f140c06ee20 0x7f140c06f280 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f140c071760 tx=0x7f13f801c520 comp rx=0 tx=0).stop 2024-05-28T13:59:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f1412600640 1 -- 172.21.15.42:0/874044922 shutdown_connections 2024-05-28T13:59:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f1412600640 1 --2- 172.21.15.42:0/874044922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f140c15b9b0 0x7f140c15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f1412600640 1 --2- 172.21.15.42:0/874044922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f140c06ee20 0x7f140c06f280 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f1412600640 1 --2- 172.21.15.42:0/874044922 >> v2:172.21.15.63:3300/0 conn(0x7f140c144f80 0x7f140c072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f1412600640 1 -- 172.21.15.42:0/874044922 >> 172.21.15.42:0/874044922 conn(0x7f140c0b21f0 msgr2=0x7f140c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f1412600640 1 -- 172.21.15.42:0/874044922 shutdown_connections 2024-05-28T13:59:13.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f1412600640 1 -- 172.21.15.42:0/874044922 wait complete. 2024-05-28T13:59:13.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f1412600640 1 Processor -- start 2024-05-28T13:59:13.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.273+0000 7f1412600640 1 -- start start 2024-05-28T13:59:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f1412600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f140c06ee20 0x7f140c13a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f1412600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f140c144f80 0x7f140c13d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f1412600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f140c15b9b0 0x7f140c143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f1412600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f140c072ea0 con 0x7f140c144f80 2024-05-28T13:59:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f1412600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f140c072160 con 0x7f140c06ee20 2024-05-28T13:59:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f1412600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f140c13f850 con 0x7f140c15b9b0 2024-05-28T13:59:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f140aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f140c144f80 0x7f140c13d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f140aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f140c144f80 0x7f140c13d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48122/0 (socket says 172.21.15.42:48122) 2024-05-28T13:59:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f140aa00640 1 -- 172.21.15.42:0/692065642 learned_addr learned my addr 172.21.15.42:0/692065642 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f140b400640 1 --2- 172.21.15.42:0/692065642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f140c06ee20 0x7f140c13a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f140be00640 1 --2- 172.21.15.42:0/692065642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f140c15b9b0 0x7f140c143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f140aa00640 1 -- 172.21.15.42:0/692065642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f140c15b9b0 msgr2=0x7f140c143730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f140aa00640 1 --2- 172.21.15.42:0/692065642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f140c15b9b0 0x7f140c143730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f140aa00640 1 -- 172.21.15.42:0/692065642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f140c06ee20 msgr2=0x7f140c13a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f140aa00640 1 --2- 172.21.15.42:0/692065642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f140c06ee20 0x7f140c13a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f140aa00640 1 -- 172.21.15.42:0/692065642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140c13e7b0 con 0x7f140c144f80 2024-05-28T13:59:13.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f140aa00640 1 --2- 172.21.15.42:0/692065642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f140c144f80 0x7f140c13d550 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f13f80af040 tx=0x7f13f80192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:13.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f1403400640 1 -- 172.21.15.42:0/692065642 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f80bc070 con 0x7f140c144f80 2024-05-28T13:59:13.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f1403400640 1 -- 172.21.15.42:0/692065642 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f80ab5e0 con 0x7f140c144f80 2024-05-28T13:59:13.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f1403400640 1 -- 172.21.15.42:0/692065642 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f80b1070 con 0x7f140c144f80 2024-05-28T13:59:13.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f1412600640 1 -- 172.21.15.42:0/692065642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140c13ea90 con 0x7f140c144f80 2024-05-28T13:59:13.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f1412600640 1 -- 172.21.15.42:0/692065642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f140c13f050 con 0x7f140c144f80 2024-05-28T13:59:13.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.277+0000 7f1412600640 1 -- 172.21.15.42:0/692065642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f140c0738b0 con 0x7f140c144f80 2024-05-28T13:59:13.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.281+0000 7f1403400640 1 -- 172.21.15.42:0/692065642 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f13f80ab9d0 con 0x7f140c144f80 2024-05-28T13:59:13.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.281+0000 7f1403400640 1 --2- 172.21.15.42:0/692065642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f13ec068380 0x7f13ec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.281+0000 7f1403400640 1 -- 172.21.15.42:0/692065642 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f13f8120400 con 0x7f140c144f80 2024-05-28T13:59:13.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.281+0000 7f140b400640 1 --2- 172.21.15.42:0/692065642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f13ec068380 0x7f13ec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.281+0000 7f140b400640 1 --2- 172.21.15.42:0/692065642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f13ec068380 0x7f13ec06a840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13fc007a60 tx=0x7f13fc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:13.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.285+0000 7f1403400640 1 -- 172.21.15.42:0/692065642 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f140c15c380 con 0x7f140c144f80 2024-05-28T13:59:13.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.465+0000 7f1412600640 1 -- 172.21.15.42:0/692065642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f140c04fdd0 con 0x7f140c144f80 2024-05-28T13:59:13.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.465+0000 7f1403400640 1 -- 172.21.15.42:0/692065642 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f13f8016110 con 0x7f140c144f80 2024-05-28T13:59:13.449 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:59:13.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.465+0000 7f1412600640 1 -- 172.21.15.42:0/692065642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f13ec068380 msgr2=0x7f13ec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.465+0000 7f1412600640 1 --2- 172.21.15.42:0/692065642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f13ec068380 0x7f13ec06a840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13fc007a60 tx=0x7f13fc008040 comp rx=0 tx=0).stop 2024-05-28T13:59:13.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.465+0000 7f1412600640 1 -- 172.21.15.42:0/692065642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f140c144f80 msgr2=0x7f140c13d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.465+0000 7f1412600640 1 --2- 172.21.15.42:0/692065642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f140c144f80 0x7f140c13d550 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f13f80af040 tx=0x7f13f80192c0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.465+0000 7f1412600640 1 -- 172.21.15.42:0/692065642 shutdown_connections 2024-05-28T13:59:13.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.465+0000 7f1412600640 1 --2- 172.21.15.42:0/692065642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f13ec068380 0x7f13ec06a840 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.465+0000 7f1412600640 1 --2- 172.21.15.42:0/692065642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f140c15b9b0 0x7f140c143730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.465+0000 7f1412600640 1 --2- 172.21.15.42:0/692065642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f140c144f80 0x7f140c13d550 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.465+0000 7f1412600640 1 --2- 172.21.15.42:0/692065642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f140c06ee20 0x7f140c13a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.465+0000 7f1412600640 1 -- 172.21.15.42:0/692065642 >> 172.21.15.42:0/692065642 conn(0x7f140c0b21f0 msgr2=0x7f140c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:13.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.465+0000 7f1412600640 1 -- 172.21.15.42:0/692065642 shutdown_connections 2024-05-28T13:59:13.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.465+0000 7f1412600640 1 -- 172.21.15.42:0/692065642 wait complete. 2024-05-28T13:59:13.471 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T13:59:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.629+0000 7f822fe00640 1 Processor -- start 2024-05-28T13:59:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.629+0000 7f822fe00640 1 -- start start 2024-05-28T13:59:13.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.629+0000 7f822fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82280ce3b0 0x7f82280ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.629+0000 7f822fe00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8228160230 con 0x7f8228071de0 2024-05-28T13:59:13.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.629+0000 7f822fe00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8228160510 con 0x7f822813fd10 2024-05-28T13:59:13.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.629+0000 7f822fe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82281607f0 con 0x7f82280ce3b0 2024-05-28T13:59:13.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.629+0000 7f822d400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8228071de0 0x7f82280721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46098/0 (socket says 172.21.15.42:46098) 2024-05-28T13:59:13.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.629+0000 7f822d400640 1 -- 172.21.15.42:0/1235443803 learned_addr learned my addr 172.21.15.42:0/1235443803 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:13.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.629+0000 7f822ca00640 1 --2- 172.21.15.42:0/1235443803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82280ce3b0 0x7f82280ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.629+0000 7f8227e00640 1 -- 172.21.15.42:0/1235443803 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 693230097 0 0) 0x7f8228160230 con 0x7f8228071de0 2024-05-28T13:59:13.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.629+0000 7f8227e00640 1 -- 172.21.15.42:0/1235443803 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8210003530 con 0x7f8228071de0 2024-05-28T13:59:13.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.629+0000 7f822ca00640 1 -- 172.21.15.42:0/1235443803 >> v1:172.21.15.63:6789/0 conn(0x7f822813fd10 legacy=0x7f82281401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.633+0000 7f8227e00640 1 -- 172.21.15.42:0/1235443803 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2049913741 0 0) 0x7f8228160510 con 0x7f822813fd10 2024-05-28T13:59:13.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.633+0000 7f822ca00640 1 -- 172.21.15.42:0/1235443803 >> v1:172.21.15.42:6789/0 conn(0x7f8228071de0 legacy=0x7f82280721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.633+0000 7f822ca00640 1 -- 172.21.15.42:0/1235443803 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8228160ad0 con 0x7f82280ce3b0 2024-05-28T13:59:13.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.633+0000 7f822ca00640 1 --2- 172.21.15.42:0/1235443803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82280ce3b0 0x7f82280ce810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82140097f0 tx=0x7f821401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57f7925e9d780ba5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:13.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.633+0000 7f8227e00640 1 -- 172.21.15.42:0/1235443803 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82140b0070 con 0x7f82280ce3b0 2024-05-28T13:59:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.633+0000 7f8227e00640 1 -- 172.21.15.42:0/1235443803 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82140a4e50 con 0x7f82280ce3b0 2024-05-28T13:59:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.633+0000 7f8227e00640 1 -- 172.21.15.42:0/1235443803 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82140ab820 con 0x7f82280ce3b0 2024-05-28T13:59:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.633+0000 7f822fe00640 1 -- 172.21.15.42:0/1235443803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82280ce3b0 msgr2=0x7f82280ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.633+0000 7f822fe00640 1 --2- 172.21.15.42:0/1235443803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82280ce3b0 0x7f82280ce810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82140097f0 tx=0x7f821401c3c0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.633+0000 7f822fe00640 1 -- 172.21.15.42:0/1235443803 shutdown_connections 2024-05-28T13:59:13.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.633+0000 7f822fe00640 1 --2- 172.21.15.42:0/1235443803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82280ce3b0 0x7f82280ce810 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.633+0000 7f822fe00640 1 -- 172.21.15.42:0/1235443803 >> 172.21.15.42:0/1235443803 conn(0x7f82280b21f0 msgr2=0x7f82280b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:13.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.633+0000 7f822fe00640 1 -- 172.21.15.42:0/1235443803 shutdown_connections 2024-05-28T13:59:13.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822fe00640 1 -- 172.21.15.42:0/1235443803 wait complete. 2024-05-28T13:59:13.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822fe00640 1 Processor -- start 2024-05-28T13:59:13.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822fe00640 1 -- start start 2024-05-28T13:59:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8228071de0 0x7f822815dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82280ce3b0 0x7f8228139fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f822813fd10 0x7f822815f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f822816a840 con 0x7f8228071de0 2024-05-28T13:59:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f822816a6c0 con 0x7f82280ce3b0 2024-05-28T13:59:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822fe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f822816a9c0 con 0x7f822813fd10 2024-05-28T13:59:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8228071de0 0x7f822815dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8228071de0 0x7f822815dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48134/0 (socket says 172.21.15.42:48134) 2024-05-28T13:59:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822d400640 1 -- 172.21.15.42:0/1090950941 learned_addr learned my addr 172.21.15.42:0/1090950941 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822ca00640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82280ce3b0 0x7f8228139fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822de00640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f822813fd10 0x7f822815f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822d400640 1 -- 172.21.15.42:0/1090950941 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f822813fd10 msgr2=0x7f822815f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822d400640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f822813fd10 0x7f822815f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822d400640 1 -- 172.21.15.42:0/1090950941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82280ce3b0 msgr2=0x7f8228139fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822d400640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82280ce3b0 0x7f8228139fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822d400640 1 -- 172.21.15.42:0/1090950941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822815cb50 con 0x7f8228071de0 2024-05-28T13:59:13.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822de00640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f822813fd10 0x7f822815f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822ca00640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82280ce3b0 0x7f8228139fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822d400640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8228071de0 0x7f822815dbf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f821c00c3c0 tx=0x7f821c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f8225600640 1 -- 172.21.15.42:0/1090950941 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c017070 con 0x7f8228071de0 2024-05-28T13:59:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f8225600640 1 -- 172.21.15.42:0/1090950941 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821c005cf0 con 0x7f8228071de0 2024-05-28T13:59:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822fe00640 1 -- 172.21.15.42:0/1090950941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822815ce30 con 0x7f8228071de0 2024-05-28T13:59:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f8225600640 1 -- 172.21.15.42:0/1090950941 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c0072f0 con 0x7f8228071de0 2024-05-28T13:59:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.637+0000 7f822fe00640 1 -- 172.21.15.42:0/1090950941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f822815d3f0 con 0x7f8228071de0 2024-05-28T13:59:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.641+0000 7f822fe00640 1 -- 172.21.15.42:0/1090950941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82280738b0 con 0x7f8228071de0 2024-05-28T13:59:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.641+0000 7f8225600640 1 -- 172.21.15.42:0/1090950941 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f821c01a070 con 0x7f8228071de0 2024-05-28T13:59:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.641+0000 7f8225600640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f82080683b0 0x7f820806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.641+0000 7f8225600640 1 -- 172.21.15.42:0/1090950941 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f821c085ef0 con 0x7f8228071de0 2024-05-28T13:59:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.641+0000 7f822ca00640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f82080683b0 0x7f820806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.641+0000 7f822ca00640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f82080683b0 0x7f820806a870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f82140ad000 tx=0x7f8214008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:13.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.645+0000 7f8225600640 1 -- 172.21.15.42:0/1090950941 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f821c056310 con 0x7f8228071de0 2024-05-28T13:59:13.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.809+0000 7f822fe00640 1 -- 172.21.15.42:0/1090950941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8228072ed0 con 0x7f8228071de0 2024-05-28T13:59:13.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.809+0000 7f8225600640 1 -- 172.21.15.42:0/1090950941 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f821c055c10 con 0x7f8228071de0 2024-05-28T13:59:13.795 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T13:59:13.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.813+0000 7f822fe00640 1 -- 172.21.15.42:0/1090950941 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f82080683b0 msgr2=0x7f820806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.813+0000 7f822fe00640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f82080683b0 0x7f820806a870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f82140ad000 tx=0x7f8214008040 comp rx=0 tx=0).stop 2024-05-28T13:59:13.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.813+0000 7f822fe00640 1 -- 172.21.15.42:0/1090950941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8228071de0 msgr2=0x7f822815dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.813+0000 7f822fe00640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8228071de0 0x7f822815dbf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f821c00c3c0 tx=0x7f821c00c880 comp rx=0 tx=0).stop 2024-05-28T13:59:13.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.813+0000 7f822fe00640 1 -- 172.21.15.42:0/1090950941 shutdown_connections 2024-05-28T13:59:13.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.813+0000 7f822fe00640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f82080683b0 0x7f820806a870 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.813+0000 7f822fe00640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f822813fd10 0x7f822815f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.813+0000 7f822fe00640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82280ce3b0 0x7f8228139fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.813+0000 7f822fe00640 1 --2- 172.21.15.42:0/1090950941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8228071de0 0x7f822815dbf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.813+0000 7f822fe00640 1 -- 172.21.15.42:0/1090950941 >> 172.21.15.42:0/1090950941 conn(0x7f82280b21f0 msgr2=0x7f82280721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:13.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.813+0000 7f822fe00640 1 -- 172.21.15.42:0/1090950941 shutdown_connections 2024-05-28T13:59:13.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.813+0000 7f822fe00640 1 -- 172.21.15.42:0/1090950941 wait complete. 2024-05-28T13:59:13.816 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T13:59:13.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.973+0000 7f8a3de00640 1 Processor -- start 2024-05-28T13:59:13.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.973+0000 7f8a3de00640 1 -- start start 2024-05-28T13:59:13.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.977+0000 7f8a3de00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8a380cce90 0x7f8a3815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.977+0000 7f8a3de00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a380b34f0 con 0x7f8a38072670 2024-05-28T13:59:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.977+0000 7f8a3de00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a3815ba90 con 0x7f8a380b3cd0 2024-05-28T13:59:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.977+0000 7f8a3de00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8a380b5870 con 0x7f8a380cce90 2024-05-28T13:59:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.977+0000 7f8a2fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8a380cce90 0x7f8a3815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.977+0000 7f8a2fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8a380cce90 0x7f8a3815b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48146/0 (socket says 172.21.15.42:48146) 2024-05-28T13:59:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.977+0000 7f8a2fe00640 1 -- 172.21.15.42:0/1602258632 learned_addr learned my addr 172.21.15.42:0/1602258632 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.977+0000 7f8a36a00640 1 --1- 172.21.15.42:0/1602258632 >> v1:172.21.15.42:6789/0 conn(0x7f8a38072670 0x7f8a38072a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46100/0 (socket says 172.21.15.42:46100) 2024-05-28T13:59:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.977+0000 7f8a2fe00640 1 -- 172.21.15.42:0/1602258632 >> v1:172.21.15.63:6789/0 conn(0x7f8a380b3cd0 legacy=0x7f8a3806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.977+0000 7f8a2fe00640 1 -- 172.21.15.42:0/1602258632 >> v1:172.21.15.42:6789/0 conn(0x7f8a38072670 legacy=0x7f8a38072a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.977+0000 7f8a2fe00640 1 -- 172.21.15.42:0/1602258632 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3815bd70 con 0x7f8a380cce90 2024-05-28T13:59:13.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a2fe00640 1 --2- 172.21.15.42:0/1602258632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a380cce90 0x7f8a3815b1f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009870 tx=0x7f8a2001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf9117eddf5dbc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:13.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a36000640 1 -- 172.21.15.42:0/1602258632 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a200b0070 con 0x7f8a380cce90 2024-05-28T13:59:13.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a36000640 1 -- 172.21.15.42:0/1602258632 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a200ab600 con 0x7f8a380cce90 2024-05-28T13:59:13.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a36000640 1 -- 172.21.15.42:0/1602258632 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a200ab9d0 con 0x7f8a380cce90 2024-05-28T13:59:13.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 -- 172.21.15.42:0/1602258632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a380cce90 msgr2=0x7f8a3815b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 --2- 172.21.15.42:0/1602258632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a380cce90 0x7f8a3815b1f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009870 tx=0x7f8a2001c520 comp rx=0 tx=0).stop 2024-05-28T13:59:13.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 -- 172.21.15.42:0/1602258632 shutdown_connections 2024-05-28T13:59:13.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 --2- 172.21.15.42:0/1602258632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a380cce90 0x7f8a3815b1f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 -- 172.21.15.42:0/1602258632 >> 172.21.15.42:0/1602258632 conn(0x7f8a380b21f0 msgr2=0x7f8a380b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:13.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 -- 172.21.15.42:0/1602258632 shutdown_connections 2024-05-28T13:59:13.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 -- 172.21.15.42:0/1602258632 wait complete. 2024-05-28T13:59:13.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 Processor -- start 2024-05-28T13:59:13.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 -- start start 2024-05-28T13:59:13.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a38072670 0x7f8a3814cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a380b3cd0 0x7f8a380af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a380cce90 0x7f8a3815d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38168d60 con 0x7f8a380b3cd0 2024-05-28T13:59:13.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38168be0 con 0x7f8a380cce90 2024-05-28T13:59:13.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.981+0000 7f8a3de00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8a38168ee0 con 0x7f8a38072670 2024-05-28T13:59:13.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a380b3cd0 0x7f8a380af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a380b3cd0 0x7f8a380af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48156/0 (socket says 172.21.15.42:48156) 2024-05-28T13:59:13.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a2fe00640 1 -- 172.21.15.42:0/2989597392 learned_addr learned my addr 172.21.15.42:0/2989597392 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:13.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a36a00640 1 --2- 172.21.15.42:0/2989597392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a38072670 0x7f8a3814cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a37400640 1 --2- 172.21.15.42:0/2989597392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a380cce90 0x7f8a3815d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a2fe00640 1 -- 172.21.15.42:0/2989597392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a38072670 msgr2=0x7f8a3814cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a2fe00640 1 --2- 172.21.15.42:0/2989597392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a38072670 0x7f8a3814cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a2fe00640 1 -- 172.21.15.42:0/2989597392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a380cce90 msgr2=0x7f8a3815d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a2fe00640 1 --2- 172.21.15.42:0/2989597392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a380cce90 0x7f8a3815d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a2fe00640 1 -- 172.21.15.42:0/2989597392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a380afc70 con 0x7f8a380b3cd0 2024-05-28T13:59:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a2fe00640 1 --2- 172.21.15.42:0/2989597392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a380b3cd0 0x7f8a380af730 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8a200ae040 tx=0x7f8a200192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a2f400640 1 -- 172.21.15.42:0/2989597392 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a200b0040 con 0x7f8a380b3cd0 2024-05-28T13:59:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a2f400640 1 -- 172.21.15.42:0/2989597392 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a200b6420 con 0x7f8a380b3cd0 2024-05-28T13:59:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a2f400640 1 -- 172.21.15.42:0/2989597392 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a200b6810 con 0x7f8a380b3cd0 2024-05-28T13:59:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a3de00640 1 -- 172.21.15.42:0/2989597392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3815a7d0 con 0x7f8a380b3cd0 2024-05-28T13:59:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a3de00640 1 -- 172.21.15.42:0/2989597392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a380aec50 con 0x7f8a380b3cd0 2024-05-28T13:59:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a3de00640 1 -- 172.21.15.42:0/2989597392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a380738b0 con 0x7f8a380b3cd0 2024-05-28T13:59:13.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.985+0000 7f8a2f400640 1 -- 172.21.15.42:0/2989597392 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8a200be020 con 0x7f8a380b3cd0 2024-05-28T13:59:13.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.989+0000 7f8a2f400640 1 --2- 172.21.15.42:0/2989597392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a14068380 0x7f8a1406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:13.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.989+0000 7f8a2f400640 1 -- 172.21.15.42:0/2989597392 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f8a20120620 con 0x7f8a380b3cd0 2024-05-28T13:59:13.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.989+0000 7f8a36a00640 1 --2- 172.21.15.42:0/2989597392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a14068380 0x7f8a1406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:13.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.989+0000 7f8a36a00640 1 --2- 172.21.15.42:0/2989597392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a14068380 0x7f8a1406a840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8a38151cf0 tx=0x7f8a28009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:13.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:13.993+0000 7f8a2f400640 1 -- 172.21.15.42:0/2989597392 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8a200c5020 con 0x7f8a380b3cd0 2024-05-28T13:59:14.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.157+0000 7f8a3de00640 1 -- 172.21.15.42:0/2989597392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8a3804fdd0 con 0x7f8a380b3cd0 2024-05-28T13:59:14.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.157+0000 7f8a2f400640 1 -- 172.21.15.42:0/2989597392 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f8a20016110 con 0x7f8a380b3cd0 2024-05-28T13:59:14.142 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T13:59:14.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.157+0000 7f8a3de00640 1 -- 172.21.15.42:0/2989597392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a14068380 msgr2=0x7f8a1406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.157+0000 7f8a3de00640 1 --2- 172.21.15.42:0/2989597392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a14068380 0x7f8a1406a840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8a38151cf0 tx=0x7f8a28009040 comp rx=0 tx=0).stop 2024-05-28T13:59:14.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.157+0000 7f8a3de00640 1 -- 172.21.15.42:0/2989597392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a380b3cd0 msgr2=0x7f8a380af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.157+0000 7f8a3de00640 1 --2- 172.21.15.42:0/2989597392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a380b3cd0 0x7f8a380af730 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8a200ae040 tx=0x7f8a200192c0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.157+0000 7f8a3de00640 1 -- 172.21.15.42:0/2989597392 shutdown_connections 2024-05-28T13:59:14.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.157+0000 7f8a3de00640 1 --2- 172.21.15.42:0/2989597392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a14068380 0x7f8a1406a840 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.157+0000 7f8a3de00640 1 --2- 172.21.15.42:0/2989597392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a380cce90 0x7f8a3815d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.157+0000 7f8a3de00640 1 --2- 172.21.15.42:0/2989597392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a380b3cd0 0x7f8a380af730 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.157+0000 7f8a3de00640 1 --2- 172.21.15.42:0/2989597392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a38072670 0x7f8a3814cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.157+0000 7f8a3de00640 1 -- 172.21.15.42:0/2989597392 >> 172.21.15.42:0/2989597392 conn(0x7f8a380b21f0 msgr2=0x7f8a380b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:14.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.157+0000 7f8a3de00640 1 -- 172.21.15.42:0/2989597392 shutdown_connections 2024-05-28T13:59:14.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.157+0000 7f8a3de00640 1 -- 172.21.15.42:0/2989597392 wait complete. 2024-05-28T13:59:14.163 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T13:59:14.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f266a800640 1 Processor -- start 2024-05-28T13:59:14.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f266a800640 1 -- start start 2024-05-28T13:59:14.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f266a800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f26640d0450 0x7f266409b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:14.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f266a800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f266415c880 0x7f26640b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:14.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f266a800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26640d0160 con 0x7f266415d680 2024-05-28T13:59:14.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f266a800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f266409b520 con 0x7f26640d0450 2024-05-28T13:59:14.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f266a800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f266409abf0 con 0x7f266415c880 2024-05-28T13:59:14.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f2669400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f26640d0450 0x7f266409b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:14.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f2669e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f266415d680 0x7f26640d0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46108/0 (socket says 172.21.15.42:46108) 2024-05-28T13:59:14.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f2669e00640 1 -- 172.21.15.42:0/3330908764 learned_addr learned my addr 172.21.15.42:0/3330908764 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:14.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f2668a00640 1 --2- 172.21.15.42:0/3330908764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f266415c880 0x7f26640b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:14.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f2669400640 1 -- 172.21.15.42:0/3330908764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f266415c880 msgr2=0x7f26640b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:14.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f2669400640 1 --2- 172.21.15.42:0/3330908764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f266415c880 0x7f26640b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f2669400640 1 -- 172.21.15.42:0/3330908764 >> v1:172.21.15.42:6789/0 conn(0x7f266415d680 legacy=0x7f26640d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f2669400640 1 -- 172.21.15.42:0/3330908764 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266409aed0 con 0x7f26640d0450 2024-05-28T13:59:14.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.321+0000 7f2663e00640 1 -- 172.21.15.42:0/3330908764 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2738351413 0 0) 0x7f26640d0160 con 0x7f266415d680 2024-05-28T13:59:14.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.325+0000 7f2669400640 1 --2- 172.21.15.42:0/3330908764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26640d0450 0x7f266409b9b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f26580097c0 tx=0x7f26580208b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbc65fcb4a76b58 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:14.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.325+0000 7f2663e00640 1 -- 172.21.15.42:0/3330908764 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580aa710 con 0x7f26640d0450 2024-05-28T13:59:14.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.325+0000 7f2663e00640 1 -- 172.21.15.42:0/3330908764 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26580aa8b0 con 0x7f26640d0450 2024-05-28T13:59:14.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.325+0000 7f2663e00640 1 -- 172.21.15.42:0/3330908764 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580aac80 con 0x7f26640d0450 2024-05-28T13:59:14.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.325+0000 7f266a800640 1 -- 172.21.15.42:0/3330908764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26640d0450 msgr2=0x7f266409b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.325+0000 7f266a800640 1 --2- 172.21.15.42:0/3330908764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26640d0450 0x7f266409b9b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f26580097c0 tx=0x7f26580208b0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.325+0000 7f266a800640 1 -- 172.21.15.42:0/3330908764 shutdown_connections 2024-05-28T13:59:14.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.325+0000 7f266a800640 1 --2- 172.21.15.42:0/3330908764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f266415c880 0x7f26640b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.325+0000 7f266a800640 1 --2- 172.21.15.42:0/3330908764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26640d0450 0x7f266409b9b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.325+0000 7f266a800640 1 -- 172.21.15.42:0/3330908764 >> 172.21.15.42:0/3330908764 conn(0x7f26640b3dd0 msgr2=0x7f26640b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:14.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.325+0000 7f266a800640 1 -- 172.21.15.42:0/3330908764 shutdown_connections 2024-05-28T13:59:14.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.325+0000 7f266a800640 1 -- 172.21.15.42:0/3330908764 wait complete. 2024-05-28T13:59:14.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.325+0000 7f266a800640 1 Processor -- start 2024-05-28T13:59:14.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.325+0000 7f266a800640 1 -- start start 2024-05-28T13:59:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f266a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26640d0450 0x7f26640c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f266a800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f266415c880 0x7f26640b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f266a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f266415d680 0x7f26640d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f266a800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f26640cd990 con 0x7f26640d0450 2024-05-28T13:59:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f266a800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26640cd810 con 0x7f266415d680 2024-05-28T13:59:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f266a800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f26640cdb10 con 0x7f266415c880 2024-05-28T13:59:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f2669400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26640d0450 0x7f26640c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f2669400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26640d0450 0x7f26640c5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48170/0 (socket says 172.21.15.42:48170) 2024-05-28T13:59:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f2669400640 1 -- 172.21.15.42:0/2675776190 learned_addr learned my addr 172.21.15.42:0/2675776190 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:14.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f2669400640 1 -- 172.21.15.42:0/2675776190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f266415c880 msgr2=0x7f26640b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f2669400640 1 --2- 172.21.15.42:0/2675776190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f266415c880 0x7f26640b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f2669400640 1 -- 172.21.15.42:0/2675776190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f266415d680 msgr2=0x7f26640d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f2669400640 1 --2- 172.21.15.42:0/2675776190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f266415d680 0x7f26640d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f2669400640 1 -- 172.21.15.42:0/2675776190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26640c64c0 con 0x7f26640d0450 2024-05-28T13:59:14.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f2669400640 1 --2- 172.21.15.42:0/2675776190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26640d0450 0x7f26640c5f80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f26580b4040 tx=0x7f265801d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f2661600640 1 -- 172.21.15.42:0/2675776190 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580b2a70 con 0x7f26640d0450 2024-05-28T13:59:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f2661600640 1 -- 172.21.15.42:0/2675776190 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26580c2070 con 0x7f26640d0450 2024-05-28T13:59:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f2661600640 1 -- 172.21.15.42:0/2675776190 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580b2e50 con 0x7f26640d0450 2024-05-28T13:59:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f266a800640 1 -- 172.21.15.42:0/2675776190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26640b5610 con 0x7f26640d0450 2024-05-28T13:59:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f266a800640 1 -- 172.21.15.42:0/2675776190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26640b5bd0 con 0x7f26640d0450 2024-05-28T13:59:14.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f266a800640 1 -- 172.21.15.42:0/2675776190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f266404fcf0 con 0x7f26640d0450 2024-05-28T13:59:14.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.329+0000 7f2661600640 1 -- 172.21.15.42:0/2675776190 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f26580c7460 con 0x7f26640d0450 2024-05-28T13:59:14.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.333+0000 7f2661600640 1 --2- 172.21.15.42:0/2675776190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2644068520 0x7f264406a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:14.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.333+0000 7f2661600640 1 -- 172.21.15.42:0/2675776190 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f26581258b0 con 0x7f26640d0450 2024-05-28T13:59:14.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.333+0000 7f2668a00640 1 --2- 172.21.15.42:0/2675776190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2644068520 0x7f264406a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:14.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.333+0000 7f2668a00640 1 --2- 172.21.15.42:0/2675776190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2644068520 0x7f264406a9e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f26640b6930 tx=0x7f2650008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:14.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.337+0000 7f2661600640 1 -- 172.21.15.42:0/2675776190 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f265801a4a0 con 0x7f26640d0450 2024-05-28T13:59:14.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.537+0000 7f266a800640 1 -- 172.21.15.42:0/2675776190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26640b4ce0 con 0x7f26640d0450 2024-05-28T13:59:14.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.537+0000 7f2661600640 1 -- 172.21.15.42:0/2675776190 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f26580f5e70 con 0x7f26640d0450 2024-05-28T13:59:14.524 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:14.524 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1409117123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1409117123},{"type":"v1","addr":"172.21.15.42:6837","nonce":1409117123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/3753517701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3753517701},{"type":"v1","addr":"172.21.15.42:6836","nonce":3753517701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/882866492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":882866492},{"type":"v1","addr":"172.21.15.63:6834","nonce":882866492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T13:58:23.075354+0000","modified":"2024-05-28T13:58:26.084895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/1552779700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1552779700},{"type":"v1","addr":"172.21.15.63:6835","nonce":1552779700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-05-28T13:59:14.525 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-05-28T13:59:14.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.541+0000 7f266a800640 1 -- 172.21.15.42:0/2675776190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2644068520 msgr2=0x7f264406a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.541+0000 7f266a800640 1 --2- 172.21.15.42:0/2675776190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2644068520 0x7f264406a9e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f26640b6930 tx=0x7f2650008040 comp rx=0 tx=0).stop 2024-05-28T13:59:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.541+0000 7f266a800640 1 -- 172.21.15.42:0/2675776190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26640d0450 msgr2=0x7f26640c5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.541+0000 7f266a800640 1 --2- 172.21.15.42:0/2675776190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26640d0450 0x7f26640c5f80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f26580b4040 tx=0x7f265801d650 comp rx=0 tx=0).stop 2024-05-28T13:59:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.541+0000 7f266a800640 1 -- 172.21.15.42:0/2675776190 shutdown_connections 2024-05-28T13:59:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.541+0000 7f266a800640 1 --2- 172.21.15.42:0/2675776190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2644068520 0x7f264406a9e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.541+0000 7f266a800640 1 --2- 172.21.15.42:0/2675776190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f266415d680 0x7f26640d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.541+0000 7f266a800640 1 --2- 172.21.15.42:0/2675776190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f266415c880 0x7f26640b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.541+0000 7f266a800640 1 --2- 172.21.15.42:0/2675776190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26640d0450 0x7f26640c5f80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.541+0000 7f266a800640 1 -- 172.21.15.42:0/2675776190 >> 172.21.15.42:0/2675776190 conn(0x7f26640b3dd0 msgr2=0x7f26640749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.541+0000 7f266a800640 1 -- 172.21.15.42:0/2675776190 shutdown_connections 2024-05-28T13:59:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.541+0000 7f266a800640 1 -- 172.21.15.42:0/2675776190 wait complete. 2024-05-28T13:59:14.545 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:59:14.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.701+0000 7fba94200640 1 Processor -- start 2024-05-28T13:59:14.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.701+0000 7fba94200640 1 -- start start 2024-05-28T13:59:14.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.701+0000 7fba94200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c06b9f0 0x7fba8c06be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:14.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.701+0000 7fba94200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba8c15eee0 con 0x7fba8c15eb00 2024-05-28T13:59:14.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.701+0000 7fba94200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba8c0ceb20 con 0x7fba8c0b5d80 2024-05-28T13:59:14.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.701+0000 7fba94200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fba8c06c390 con 0x7fba8c06b9f0 2024-05-28T13:59:14.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.701+0000 7fba91800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fba8c15eb00 0x7fba8c0cf8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46124/0 (socket says 172.21.15.42:46124) 2024-05-28T13:59:14.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.701+0000 7fba91800640 1 -- 172.21.15.42:0/2147326695 learned_addr learned my addr 172.21.15.42:0/2147326695 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.701+0000 7fba90e00640 1 --2- 172.21.15.42:0/2147326695 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c06b9f0 0x7fba8c06be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.701+0000 7fba8be00640 1 -- 172.21.15.42:0/2147326695 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 95599653 0 0) 0x7fba80001120 con 0x7fba8c15eb00 2024-05-28T13:59:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.701+0000 7fba8be00640 1 -- 172.21.15.42:0/2147326695 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba74003530 con 0x7fba8c15eb00 2024-05-28T13:59:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.705+0000 7fba8be00640 1 -- 172.21.15.42:0/2147326695 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2914881153 0 0) 0x7fba8c0ceb20 con 0x7fba8c0b5d80 2024-05-28T13:59:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.705+0000 7fba8be00640 1 -- 172.21.15.42:0/2147326695 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba80001120 con 0x7fba8c0b5d80 2024-05-28T13:59:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.705+0000 7fba8be00640 1 -- 172.21.15.42:0/2147326695 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2877901399 0 0) 0x7fba74003530 con 0x7fba8c15eb00 2024-05-28T13:59:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.705+0000 7fba90e00640 1 -- 172.21.15.42:0/2147326695 >> v1:172.21.15.63:6789/0 conn(0x7fba8c0b5d80 legacy=0x7fba8c0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.705+0000 7fba90e00640 1 -- 172.21.15.42:0/2147326695 >> v1:172.21.15.42:6789/0 conn(0x7fba8c15eb00 legacy=0x7fba8c0cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.705+0000 7fba90e00640 1 -- 172.21.15.42:0/2147326695 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c0cee00 con 0x7fba8c06b9f0 2024-05-28T13:59:14.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.705+0000 7fba90e00640 1 --2- 172.21.15.42:0/2147326695 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c06b9f0 0x7fba8c06be50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fba780097f0 tx=0x7fba7801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c804cb47c56bb6d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:14.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.705+0000 7fba8be00640 1 -- 172.21.15.42:0/2147326695 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba780b0070 con 0x7fba8c06b9f0 2024-05-28T13:59:14.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.705+0000 7fba8be00640 1 -- 172.21.15.42:0/2147326695 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba780ab400 con 0x7fba8c06b9f0 2024-05-28T13:59:14.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.705+0000 7fba8be00640 1 -- 172.21.15.42:0/2147326695 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80001850 con 0x7fba8c06b9f0 2024-05-28T13:59:14.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.705+0000 7fba94200640 1 -- 172.21.15.42:0/2147326695 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c06b9f0 msgr2=0x7fba8c06be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.705+0000 7fba94200640 1 --2- 172.21.15.42:0/2147326695 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c06b9f0 0x7fba8c06be50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fba780097f0 tx=0x7fba7801c400 comp rx=0 tx=0).stop 2024-05-28T13:59:14.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba94200640 1 -- 172.21.15.42:0/2147326695 shutdown_connections 2024-05-28T13:59:14.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba94200640 1 --2- 172.21.15.42:0/2147326695 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c06b9f0 0x7fba8c06be50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba94200640 1 -- 172.21.15.42:0/2147326695 >> 172.21.15.42:0/2147326695 conn(0x7fba8c0b68f0 msgr2=0x7fba8c0b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:14.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba94200640 1 -- 172.21.15.42:0/2147326695 shutdown_connections 2024-05-28T13:59:14.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba94200640 1 -- 172.21.15.42:0/2147326695 wait complete. 2024-05-28T13:59:14.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba94200640 1 Processor -- start 2024-05-28T13:59:14.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba94200640 1 -- start start 2024-05-28T13:59:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba94200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c06b9f0 0x7fba8c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba94200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c0b5d80 0x7fba8c156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba94200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8c15eb00 0x7fba8c157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba94200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c16cfb0 con 0x7fba8c15eb00 2024-05-28T13:59:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba94200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c16ce30 con 0x7fba8c06b9f0 2024-05-28T13:59:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba94200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fba8c16d130 con 0x7fba8c0b5d80 2024-05-28T13:59:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba92200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8c15eb00 0x7fba8c157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:14.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba92200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8c15eb00 0x7fba8c157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48186/0 (socket says 172.21.15.42:48186) 2024-05-28T13:59:14.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba92200640 1 -- 172.21.15.42:0/2075875550 learned_addr learned my addr 172.21.15.42:0/2075875550 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:14.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba90e00640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c0b5d80 0x7fba8c156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:14.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba91800640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c06b9f0 0x7fba8c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:14.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba92200640 1 -- 172.21.15.42:0/2075875550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c0b5d80 msgr2=0x7fba8c156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba92200640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c0b5d80 0x7fba8c156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba92200640 1 -- 172.21.15.42:0/2075875550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c06b9f0 msgr2=0x7fba8c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba92200640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c06b9f0 0x7fba8c156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba92200640 1 -- 172.21.15.42:0/2075875550 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c0b0280 con 0x7fba8c15eb00 2024-05-28T13:59:14.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba91800640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c06b9f0 0x7fba8c156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:14.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba92200640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8c15eb00 0x7fba8c157fc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fba8400c790 tx=0x7fba8400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:14.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.709+0000 7fba89600640 1 -- 172.21.15.42:0/2075875550 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84003b80 con 0x7fba8c15eb00 2024-05-28T13:59:14.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.713+0000 7fba89600640 1 -- 172.21.15.42:0/2075875550 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba84003d20 con 0x7fba8c15eb00 2024-05-28T13:59:14.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.713+0000 7fba94200640 1 -- 172.21.15.42:0/2075875550 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8c0affa0 con 0x7fba8c15eb00 2024-05-28T13:59:14.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.713+0000 7fba89600640 1 -- 172.21.15.42:0/2075875550 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8400aa50 con 0x7fba8c15eb00 2024-05-28T13:59:14.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.713+0000 7fba94200640 1 -- 172.21.15.42:0/2075875550 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba8c14fcb0 con 0x7fba8c15eb00 2024-05-28T13:59:14.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.713+0000 7fba94200640 1 -- 172.21.15.42:0/2075875550 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba8c14ffe0 con 0x7fba8c15eb00 2024-05-28T13:59:14.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.713+0000 7fba89600640 1 -- 172.21.15.42:0/2075875550 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fba8400abf0 con 0x7fba8c15eb00 2024-05-28T13:59:14.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.713+0000 7fba89600640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba6c0683b0 0x7fba6c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:14.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.713+0000 7fba89600640 1 -- 172.21.15.42:0/2075875550 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fba84086a50 con 0x7fba8c15eb00 2024-05-28T13:59:14.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.713+0000 7fba91800640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba6c0683b0 0x7fba6c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:14.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.717+0000 7fba91800640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba6c0683b0 0x7fba6c06a870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fba80001ec0 tx=0x7fba80009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:14.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.717+0000 7fba89600640 1 -- 172.21.15.42:0/2075875550 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fba84056ef0 con 0x7fba8c15eb00 2024-05-28T13:59:14.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.885+0000 7fba94200640 1 -- 172.21.15.42:0/2075875550 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba8c068a90 con 0x7fba8c15eb00 2024-05-28T13:59:14.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.885+0000 7fba89600640 1 -- 172.21.15.42:0/2075875550 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16737 (secure 0 0 0) 0x7fba840567f0 con 0x7fba8c15eb00 2024-05-28T13:59:14.870 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:14.870 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:59:03.992606+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:59:14.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.885+0000 7fba94200640 1 -- 172.21.15.42:0/2075875550 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba6c0683b0 msgr2=0x7fba6c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.885+0000 7fba94200640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba6c0683b0 0x7fba6c06a870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fba80001ec0 tx=0x7fba80009040 comp rx=0 tx=0).stop 2024-05-28T13:59:14.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.885+0000 7fba94200640 1 -- 172.21.15.42:0/2075875550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8c15eb00 msgr2=0x7fba8c157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:14.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.885+0000 7fba94200640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8c15eb00 0x7fba8c157fc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fba8400c790 tx=0x7fba8400cc50 comp rx=0 tx=0).stop 2024-05-28T13:59:14.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.885+0000 7fba94200640 1 -- 172.21.15.42:0/2075875550 shutdown_connections 2024-05-28T13:59:14.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.885+0000 7fba94200640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba6c0683b0 0x7fba6c06a870 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.885+0000 7fba94200640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8c15eb00 0x7fba8c157fc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.885+0000 7fba94200640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c0b5d80 0x7fba8c156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.885+0000 7fba94200640 1 --2- 172.21.15.42:0/2075875550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c06b9f0 0x7fba8c156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:14.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.885+0000 7fba94200640 1 -- 172.21.15.42:0/2075875550 >> 172.21.15.42:0/2075875550 conn(0x7fba8c0b68f0 msgr2=0x7fba8c0cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:14.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.885+0000 7fba94200640 1 -- 172.21.15.42:0/2075875550 shutdown_connections 2024-05-28T13:59:14.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:14.885+0000 7fba94200640 1 -- 172.21.15.42:0/2075875550 wait complete. 2024-05-28T13:59:14.892 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-28T13:59:14.892 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T13:59:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.049+0000 7f5165e00640 1 Processor -- start 2024-05-28T13:59:15.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.049+0000 7f5165e00640 1 -- start start 2024-05-28T13:59:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.049+0000 7f5165e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f51600d16e0 0x7f51600d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.049+0000 7f5165e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51600b4720 con 0x7f5160162010 2024-05-28T13:59:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.049+0000 7f5165e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51600d51a0 con 0x7f51600b35e0 2024-05-28T13:59:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.049+0000 7f5165e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f51600d5480 con 0x7f51600d16e0 2024-05-28T13:59:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.049+0000 7f515ea00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f5160162010 0x7f51600b4010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46128/0 (socket says 172.21.15.42:46128) 2024-05-28T13:59:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.049+0000 7f515ea00640 1 -- 172.21.15.42:0/3792814020 learned_addr learned my addr 172.21.15.42:0/3792814020 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.049+0000 7f515f400640 1 --2- 172.21.15.42:0/3792814020 >> v2:172.21.15.63:3300/0 conn(0x7f51600d16e0 0x7f51600d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.049+0000 7f515d600640 1 -- 172.21.15.42:0/3792814020 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2557944813 0 0) 0x7f51600b4720 con 0x7f5160162010 2024-05-28T13:59:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.049+0000 7f515d600640 1 -- 172.21.15.42:0/3792814020 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5144003530 con 0x7f5160162010 2024-05-28T13:59:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.049+0000 7f515f400640 1 -- 172.21.15.42:0/3792814020 >> v1:172.21.15.63:6789/0 conn(0x7f51600b35e0 legacy=0x7f51600d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.049+0000 7f515f400640 1 -- 172.21.15.42:0/3792814020 >> v1:172.21.15.42:6789/0 conn(0x7f5160162010 legacy=0x7f51600b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.049+0000 7f515f400640 1 -- 172.21.15.42:0/3792814020 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51600cf660 con 0x7f51600d16e0 2024-05-28T13:59:15.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.053+0000 7f515f400640 1 --2- 172.21.15.42:0/3792814020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51600d16e0 0x7f51600d3360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5158009dc0 tx=0x7f515801c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac3c5adcabf3958 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:15.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.053+0000 7f515d600640 1 -- 172.21.15.42:0/3792814020 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51580b2070 con 0x7f51600d16e0 2024-05-28T13:59:15.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.053+0000 7f515d600640 1 -- 172.21.15.42:0/3792814020 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51580a58b0 con 0x7f51600d16e0 2024-05-28T13:59:15.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.053+0000 7f515d600640 1 -- 172.21.15.42:0/3792814020 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51580a5c60 con 0x7f51600d16e0 2024-05-28T13:59:15.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.053+0000 7f5165e00640 1 -- 172.21.15.42:0/3792814020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51600d16e0 msgr2=0x7f51600d3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.053+0000 7f5165e00640 1 --2- 172.21.15.42:0/3792814020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51600d16e0 0x7f51600d3360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5158009dc0 tx=0x7f515801c890 comp rx=0 tx=0).stop 2024-05-28T13:59:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.053+0000 7f5165e00640 1 -- 172.21.15.42:0/3792814020 shutdown_connections 2024-05-28T13:59:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.053+0000 7f5165e00640 1 --2- 172.21.15.42:0/3792814020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51600d16e0 0x7f51600d3360 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.053+0000 7f5165e00640 1 -- 172.21.15.42:0/3792814020 >> 172.21.15.42:0/3792814020 conn(0x7f516008bfe0 msgr2=0x7f51600d46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.053+0000 7f5165e00640 1 -- 172.21.15.42:0/3792814020 shutdown_connections 2024-05-28T13:59:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.053+0000 7f5165e00640 1 -- 172.21.15.42:0/3792814020 wait complete. 2024-05-28T13:59:15.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f5165e00640 1 Processor -- start 2024-05-28T13:59:15.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f5165e00640 1 -- start start 2024-05-28T13:59:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f5165e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51600b35e0 0x7f51600b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f5165e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51600d16e0 0x7f5160154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f5165e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5160162010 0x7f516006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f5165e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f51600cf400 con 0x7f51600d16e0 2024-05-28T13:59:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f5165e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f51600cf280 con 0x7f51600b35e0 2024-05-28T13:59:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f5165e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f516016ca80 con 0x7f5160162010 2024-05-28T13:59:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f515e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51600d16e0 0x7f5160154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f515e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51600d16e0 0x7f5160154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48192/0 (socket says 172.21.15.42:48192) 2024-05-28T13:59:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f515e000640 1 -- 172.21.15.42:0/2307057299 learned_addr learned my addr 172.21.15.42:0/2307057299 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f515ea00640 1 --2- 172.21.15.42:0/2307057299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51600b35e0 0x7f51600b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f515f400640 1 --2- 172.21.15.42:0/2307057299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5160162010 0x7f516006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f515e000640 1 -- 172.21.15.42:0/2307057299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5160162010 msgr2=0x7f516006cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f515e000640 1 --2- 172.21.15.42:0/2307057299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5160162010 0x7f516006cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f515e000640 1 -- 172.21.15.42:0/2307057299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51600b35e0 msgr2=0x7f51600b7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f515e000640 1 --2- 172.21.15.42:0/2307057299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51600b35e0 0x7f51600b7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f515e000640 1 -- 172.21.15.42:0/2307057299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51601599c0 con 0x7f51600d16e0 2024-05-28T13:59:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f515e000640 1 --2- 172.21.15.42:0/2307057299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51600d16e0 0x7f5160154f60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f514c00c460 tx=0x7f514c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:15.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f5157400640 1 -- 172.21.15.42:0/2307057299 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c016020 con 0x7f51600d16e0 2024-05-28T13:59:15.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f5157400640 1 -- 172.21.15.42:0/2307057299 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f514c0052d0 con 0x7f51600d16e0 2024-05-28T13:59:15.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f5157400640 1 -- 172.21.15.42:0/2307057299 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c007260 con 0x7f51600d16e0 2024-05-28T13:59:15.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f5165e00640 1 -- 172.21.15.42:0/2307057299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5160159ca0 con 0x7f51600d16e0 2024-05-28T13:59:15.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.057+0000 7f5165e00640 1 -- 172.21.15.42:0/2307057299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f516015caf0 con 0x7f51600d16e0 2024-05-28T13:59:15.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.061+0000 7f5165e00640 1 -- 172.21.15.42:0/2307057299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5160068a90 con 0x7f51600d16e0 2024-05-28T13:59:15.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.061+0000 7f5157400640 1 -- 172.21.15.42:0/2307057299 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f514c007400 con 0x7f51600d16e0 2024-05-28T13:59:15.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.061+0000 7f5157400640 1 --2- 172.21.15.42:0/2307057299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5140068380 0x7f514006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.061+0000 7f5157400640 1 -- 172.21.15.42:0/2307057299 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f514c086460 con 0x7f51600d16e0 2024-05-28T13:59:15.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.061+0000 7f515ea00640 1 --2- 172.21.15.42:0/2307057299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5140068380 0x7f514006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:15.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.061+0000 7f515ea00640 1 --2- 172.21.15.42:0/2307057299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5140068380 0x7f514006a840 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f51500011f0 tx=0x7f5150009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:15.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.065+0000 7f5157400640 1 -- 172.21.15.42:0/2307057299 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f514c020020 con 0x7f51600d16e0 2024-05-28T13:59:15.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.265+0000 7f5165e00640 1 -- 172.21.15.42:0/2307057299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51600d2650 con 0x7f51600d16e0 2024-05-28T13:59:15.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.265+0000 7f5157400640 1 -- 172.21.15.42:0/2307057299 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f514c005480 con 0x7f51600d16e0 2024-05-28T13:59:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:15.251 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1409117123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1409117123},{"type":"v1","addr":"172.21.15.42:6837","nonce":1409117123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/3753517701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3753517701},{"type":"v1","addr":"172.21.15.42:6836","nonce":3753517701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/882866492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":882866492},{"type":"v1","addr":"172.21.15.63:6834","nonce":882866492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T13:58:23.075354+0000","modified":"2024-05-28T13:58:26.084895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/1552779700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1552779700},{"type":"v1","addr":"172.21.15.63:6835","nonce":1552779700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-05-28T13:59:15.251 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-05-28T13:59:15.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.269+0000 7f5165e00640 1 -- 172.21.15.42:0/2307057299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5140068380 msgr2=0x7f514006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.269+0000 7f5165e00640 1 --2- 172.21.15.42:0/2307057299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5140068380 0x7f514006a840 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f51500011f0 tx=0x7f5150009040 comp rx=0 tx=0).stop 2024-05-28T13:59:15.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.269+0000 7f5165e00640 1 -- 172.21.15.42:0/2307057299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51600d16e0 msgr2=0x7f5160154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.269+0000 7f5165e00640 1 --2- 172.21.15.42:0/2307057299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51600d16e0 0x7f5160154f60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f514c00c460 tx=0x7f514c00c920 comp rx=0 tx=0).stop 2024-05-28T13:59:15.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.269+0000 7f5165e00640 1 -- 172.21.15.42:0/2307057299 shutdown_connections 2024-05-28T13:59:15.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.269+0000 7f5165e00640 1 --2- 172.21.15.42:0/2307057299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5140068380 0x7f514006a840 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.269+0000 7f5165e00640 1 --2- 172.21.15.42:0/2307057299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5160162010 0x7f516006cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.269+0000 7f5165e00640 1 --2- 172.21.15.42:0/2307057299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51600d16e0 0x7f5160154f60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.269+0000 7f5165e00640 1 --2- 172.21.15.42:0/2307057299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51600b35e0 0x7f51600b7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.269+0000 7f5165e00640 1 -- 172.21.15.42:0/2307057299 >> 172.21.15.42:0/2307057299 conn(0x7f516008bfe0 msgr2=0x7f51600b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:15.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.269+0000 7f5165e00640 1 -- 172.21.15.42:0/2307057299 shutdown_connections 2024-05-28T13:59:15.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.269+0000 7f5165e00640 1 -- 172.21.15.42:0/2307057299 wait complete. 2024-05-28T13:59:15.273 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T13:59:15.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.429+0000 7fc44f000640 1 Processor -- start 2024-05-28T13:59:15.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.429+0000 7fc44f000640 1 -- start start 2024-05-28T13:59:15.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.429+0000 7fc44f000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc448162fb0 0x7fc4480b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.429+0000 7fc44f000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc44806d930 con 0x7fc4480b3610 2024-05-28T13:59:15.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.429+0000 7fc44f000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4480d3040 con 0x7fc44806e4d0 2024-05-28T13:59:15.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.429+0000 7fc44f000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc4480d4ff0 con 0x7fc448162fb0 2024-05-28T13:59:15.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.429+0000 7fc447e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc448162fb0 0x7fc4480b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:15.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.429+0000 7fc43f400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc4480b3610 0x7fc44806cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46136/0 (socket says 172.21.15.42:46136) 2024-05-28T13:59:15.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.429+0000 7fc43f400640 1 -- 172.21.15.42:0/1568611101 learned_addr learned my addr 172.21.15.42:0/1568611101 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:15.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.429+0000 7fc447e00640 1 -- 172.21.15.42:0/1568611101 >> v1:172.21.15.63:6789/0 conn(0x7fc44806e4d0 legacy=0x7fc44806d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:15.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.429+0000 7fc447e00640 1 -- 172.21.15.42:0/1568611101 >> v1:172.21.15.42:6789/0 conn(0x7fc4480b3610 legacy=0x7fc44806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.429+0000 7fc447e00640 1 -- 172.21.15.42:0/1568611101 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44806dbc0 con 0x7fc448162fb0 2024-05-28T13:59:15.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.433+0000 7fc447e00640 1 --2- 172.21.15.42:0/1568611101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc448162fb0 0x7fc4480b61b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc4380097c0 tx=0x7fc43801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0362dcce91c8a52 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:15.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.433+0000 7fc447400640 1 -- 172.21.15.42:0/1568611101 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4380b0070 con 0x7fc448162fb0 2024-05-28T13:59:15.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.433+0000 7fc447400640 1 -- 172.21.15.42:0/1568611101 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4380ab600 con 0x7fc448162fb0 2024-05-28T13:59:15.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.433+0000 7fc447400640 1 -- 172.21.15.42:0/1568611101 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4380ab9d0 con 0x7fc448162fb0 2024-05-28T13:59:15.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.433+0000 7fc44f000640 1 -- 172.21.15.42:0/1568611101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc448162fb0 msgr2=0x7fc4480b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.433+0000 7fc44f000640 1 --2- 172.21.15.42:0/1568611101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc448162fb0 0x7fc4480b61b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc4380097c0 tx=0x7fc43801c460 comp rx=0 tx=0).stop 2024-05-28T13:59:15.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.433+0000 7fc44f000640 1 -- 172.21.15.42:0/1568611101 shutdown_connections 2024-05-28T13:59:15.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.433+0000 7fc44f000640 1 --2- 172.21.15.42:0/1568611101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc448162fb0 0x7fc4480b61b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.433+0000 7fc44f000640 1 -- 172.21.15.42:0/1568611101 >> 172.21.15.42:0/1568611101 conn(0x7fc44808bfe0 msgr2=0x7fc4480d4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:15.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.433+0000 7fc44f000640 1 -- 172.21.15.42:0/1568611101 shutdown_connections 2024-05-28T13:59:15.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.433+0000 7fc44f000640 1 -- 172.21.15.42:0/1568611101 wait complete. 2024-05-28T13:59:15.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc44f000640 1 Processor -- start 2024-05-28T13:59:15.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc44f000640 1 -- start start 2024-05-28T13:59:15.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc44f000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc44806e4d0 0x7fc4480b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc44f000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4480b3610 0x7fc4480ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc44f000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc448162fb0 0x7fc448157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc44f000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc4480b3b60 con 0x7fc4480b3610 2024-05-28T13:59:15.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc44f000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc4480b6b00 con 0x7fc44806e4d0 2024-05-28T13:59:15.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc44f000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc4480b3ce0 con 0x7fc448162fb0 2024-05-28T13:59:15.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc43f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4480b3610 0x7fc4480ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:15.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc43f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4480b3610 0x7fc4480ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48206/0 (socket says 172.21.15.42:48206) 2024-05-28T13:59:15.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc43f400640 1 -- 172.21.15.42:0/173030395 learned_addr learned my addr 172.21.15.42:0/173030395 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:15.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc447e00640 1 --2- 172.21.15.42:0/173030395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc44806e4d0 0x7fc4480b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:15.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc43f400640 1 -- 172.21.15.42:0/173030395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc448162fb0 msgr2=0x7fc448157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc43f400640 1 --2- 172.21.15.42:0/173030395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc448162fb0 0x7fc448157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc43f400640 1 -- 172.21.15.42:0/173030395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc44806e4d0 msgr2=0x7fc4480b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc43f400640 1 --2- 172.21.15.42:0/173030395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc44806e4d0 0x7fc4480b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc43f400640 1 -- 172.21.15.42:0/173030395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4480b9e00 con 0x7fc4480b3610 2024-05-28T13:59:15.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc43f400640 1 --2- 172.21.15.42:0/173030395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4480b3610 0x7fc4480ca9f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc43000c420 tx=0x7fc43000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:15.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc444c00640 1 -- 172.21.15.42:0/173030395 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc430017070 con 0x7fc4480b3610 2024-05-28T13:59:15.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc444c00640 1 -- 172.21.15.42:0/173030395 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc430005cf0 con 0x7fc4480b3610 2024-05-28T13:59:15.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc444c00640 1 -- 172.21.15.42:0/173030395 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4300072f0 con 0x7fc4480b3610 2024-05-28T13:59:15.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc44f000640 1 -- 172.21.15.42:0/173030395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4480ba0e0 con 0x7fc4480b3610 2024-05-28T13:59:15.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc44f000640 1 -- 172.21.15.42:0/173030395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc448157ba0 con 0x7fc4480b3610 2024-05-28T13:59:15.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.437+0000 7fc44f000640 1 -- 172.21.15.42:0/173030395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc448068a90 con 0x7fc4480b3610 2024-05-28T13:59:15.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.441+0000 7fc444c00640 1 -- 172.21.15.42:0/173030395 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc43001a070 con 0x7fc4480b3610 2024-05-28T13:59:15.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.441+0000 7fc444c00640 1 --2- 172.21.15.42:0/173030395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc424068380 0x7fc42406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.441+0000 7fc444c00640 1 -- 172.21.15.42:0/173030395 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc430085db0 con 0x7fc4480b3610 2024-05-28T13:59:15.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.441+0000 7fc447e00640 1 --2- 172.21.15.42:0/173030395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc424068380 0x7fc42406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:15.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.441+0000 7fc447e00640 1 --2- 172.21.15.42:0/173030395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc424068380 0x7fc42406a840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc4380ad000 tx=0x7fc4380a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:15.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.445+0000 7fc444c00640 1 -- 172.21.15.42:0/173030395 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc43008a070 con 0x7fc4480b3610 2024-05-28T13:59:15.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.645+0000 7fc44f000640 1 -- 172.21.15.42:0/173030395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc44808e4a0 con 0x7fc4480b3610 2024-05-28T13:59:15.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.645+0000 7fc444c00640 1 -- 172.21.15.42:0/173030395 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7fc430055c90 con 0x7fc4480b3610 2024-05-28T13:59:15.630 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:15.631 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1409117123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1409117123},{"type":"v1","addr":"172.21.15.42:6837","nonce":1409117123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/3753517701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3753517701},{"type":"v1","addr":"172.21.15.42:6836","nonce":3753517701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/882866492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":882866492},{"type":"v1","addr":"172.21.15.63:6834","nonce":882866492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T13:58:23.075354+0000","modified":"2024-05-28T13:58:26.084895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/1552779700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1552779700},{"type":"v1","addr":"172.21.15.63:6835","nonce":1552779700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-05-28T13:59:15.631 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-05-28T13:59:15.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.645+0000 7fc44f000640 1 -- 172.21.15.42:0/173030395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc424068380 msgr2=0x7fc42406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.645+0000 7fc44f000640 1 --2- 172.21.15.42:0/173030395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc424068380 0x7fc42406a840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc4380ad000 tx=0x7fc4380a6000 comp rx=0 tx=0).stop 2024-05-28T13:59:15.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.645+0000 7fc44f000640 1 -- 172.21.15.42:0/173030395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4480b3610 msgr2=0x7fc4480ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.645+0000 7fc44f000640 1 --2- 172.21.15.42:0/173030395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4480b3610 0x7fc4480ca9f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc43000c420 tx=0x7fc43000c8e0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.645+0000 7fc44f000640 1 -- 172.21.15.42:0/173030395 shutdown_connections 2024-05-28T13:59:15.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.645+0000 7fc44f000640 1 --2- 172.21.15.42:0/173030395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc424068380 0x7fc42406a840 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.645+0000 7fc44f000640 1 --2- 172.21.15.42:0/173030395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc448162fb0 0x7fc448157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.645+0000 7fc44f000640 1 --2- 172.21.15.42:0/173030395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4480b3610 0x7fc4480ca9f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.645+0000 7fc44f000640 1 --2- 172.21.15.42:0/173030395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc44806e4d0 0x7fc4480b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.645+0000 7fc44f000640 1 -- 172.21.15.42:0/173030395 >> 172.21.15.42:0/173030395 conn(0x7fc44808bfe0 msgr2=0x7fc44815aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:15.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.645+0000 7fc44f000640 1 -- 172.21.15.42:0/173030395 shutdown_connections 2024-05-28T13:59:15.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.645+0000 7fc44f000640 1 -- 172.21.15.42:0/173030395 wait complete. 2024-05-28T13:59:15.653 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:59:15.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee9000640 1 Processor -- start 2024-05-28T13:59:15.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee9000640 1 -- start start 2024-05-28T13:59:15.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee9000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbee40ce1b0 0x7fbee41672e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee9000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee40cfd80 con 0x7fbee415a460 2024-05-28T13:59:15.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee9000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee415cea0 con 0x7fbee406c390 2024-05-28T13:59:15.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee9000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbee415cc60 con 0x7fbee40ce1b0 2024-05-28T13:59:15.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee1e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fbee415a460 0x7fbee40cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46144/0 (socket says 172.21.15.42:46144) 2024-05-28T13:59:15.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee1e00640 1 -- 172.21.15.42:0/850243519 learned_addr learned my addr 172.21.15.42:0/850243519 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:15.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee2800640 1 --2- 172.21.15.42:0/850243519 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbee40ce1b0 0x7fbee41672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:15.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee1400640 1 -- 172.21.15.42:0/850243519 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4231717166 0 0) 0x7fbee40cfd80 con 0x7fbee415a460 2024-05-28T13:59:15.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee1400640 1 -- 172.21.15.42:0/850243519 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbec8003530 con 0x7fbee415a460 2024-05-28T13:59:15.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee1400640 1 -- 172.21.15.42:0/850243519 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4265876975 0 0) 0x7fbee415cea0 con 0x7fbee406c390 2024-05-28T13:59:15.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee1400640 1 -- 172.21.15.42:0/850243519 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbee40cfd80 con 0x7fbee406c390 2024-05-28T13:59:15.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee2800640 1 -- 172.21.15.42:0/850243519 >> v1:172.21.15.63:6789/0 conn(0x7fbee406c390 legacy=0x7fbee40cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee1400640 1 -- 172.21.15.42:0/850243519 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 330635924 0 0) 0x7fbec8003530 con 0x7fbee415a460 2024-05-28T13:59:15.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee2800640 1 -- 172.21.15.42:0/850243519 >> v1:172.21.15.42:6789/0 conn(0x7fbee415a460 legacy=0x7fbee40cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.809+0000 7fbee2800640 1 -- 172.21.15.42:0/850243519 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee415d180 con 0x7fbee40ce1b0 2024-05-28T13:59:15.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.813+0000 7fbee2800640 1 --2- 172.21.15.42:0/850243519 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbee40ce1b0 0x7fbee41672e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009dc0 tx=0x7fbedc01c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85b21062c3dd78a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:15.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.813+0000 7fbee1400640 1 -- 172.21.15.42:0/850243519 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0ad070 con 0x7fbee40ce1b0 2024-05-28T13:59:15.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.813+0000 7fbee1400640 1 -- 172.21.15.42:0/850243519 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbedc0a9460 con 0x7fbee40ce1b0 2024-05-28T13:59:15.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.813+0000 7fbee1400640 1 -- 172.21.15.42:0/850243519 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4001680 con 0x7fbee40ce1b0 2024-05-28T13:59:15.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.813+0000 7fbee9000640 1 -- 172.21.15.42:0/850243519 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbee40ce1b0 msgr2=0x7fbee41672e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.813+0000 7fbee9000640 1 --2- 172.21.15.42:0/850243519 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbee40ce1b0 0x7fbee41672e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009dc0 tx=0x7fbedc01c820 comp rx=0 tx=0).stop 2024-05-28T13:59:15.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.813+0000 7fbee9000640 1 -- 172.21.15.42:0/850243519 shutdown_connections 2024-05-28T13:59:15.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.813+0000 7fbee9000640 1 --2- 172.21.15.42:0/850243519 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbee40ce1b0 0x7fbee41672e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.813+0000 7fbee9000640 1 -- 172.21.15.42:0/850243519 >> 172.21.15.42:0/850243519 conn(0x7fbee40b2f60 msgr2=0x7fbee40b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:15.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.813+0000 7fbee9000640 1 -- 172.21.15.42:0/850243519 shutdown_connections 2024-05-28T13:59:15.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.813+0000 7fbee9000640 1 -- 172.21.15.42:0/850243519 wait complete. 2024-05-28T13:59:15.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee9000640 1 Processor -- start 2024-05-28T13:59:15.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee9000640 1 -- start start 2024-05-28T13:59:15.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee9000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee406c390 0x7fbee415fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee9000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee40ce1b0 0x7fbee4152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee9000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbee415a460 0x7fbee4152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee9000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbee416a660 con 0x7fbee406c390 2024-05-28T13:59:15.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee9000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbee406c7c0 con 0x7fbee40ce1b0 2024-05-28T13:59:15.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee9000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbee416a7e0 con 0x7fbee415a460 2024-05-28T13:59:15.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee406c390 0x7fbee415fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:15.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee406c390 0x7fbee415fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48222/0 (socket says 172.21.15.42:48222) 2024-05-28T13:59:15.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee1e00640 1 -- 172.21.15.42:0/1618607076 learned_addr learned my addr 172.21.15.42:0/1618607076 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:15.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbed9400640 1 --2- 172.21.15.42:0/1618607076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee40ce1b0 0x7fbee4152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:15.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee2800640 1 --2- 172.21.15.42:0/1618607076 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbee415a460 0x7fbee4152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:15.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee1e00640 1 -- 172.21.15.42:0/1618607076 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbee415a460 msgr2=0x7fbee4152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee1e00640 1 --2- 172.21.15.42:0/1618607076 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbee415a460 0x7fbee4152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee1e00640 1 -- 172.21.15.42:0/1618607076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee40ce1b0 msgr2=0x7fbee4152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee1e00640 1 --2- 172.21.15.42:0/1618607076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee40ce1b0 0x7fbee4152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee1e00640 1 -- 172.21.15.42:0/1618607076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee41535a0 con 0x7fbee406c390 2024-05-28T13:59:15.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee1e00640 1 --2- 172.21.15.42:0/1618607076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee406c390 0x7fbee415fc30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbed400c780 tx=0x7fbed400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:15.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbedaa00640 1 -- 172.21.15.42:0/1618607076 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4017070 con 0x7fbee406c390 2024-05-28T13:59:15.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbedaa00640 1 -- 172.21.15.42:0/1618607076 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed4005b00 con 0x7fbee406c390 2024-05-28T13:59:15.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee9000640 1 -- 172.21.15.42:0/1618607076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee4153880 con 0x7fbee406c390 2024-05-28T13:59:15.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbedaa00640 1 -- 172.21.15.42:0/1618607076 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4007070 con 0x7fbee406c390 2024-05-28T13:59:15.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee9000640 1 -- 172.21.15.42:0/1618607076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee40ae750 con 0x7fbee406c390 2024-05-28T13:59:15.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.817+0000 7fbee9000640 1 -- 172.21.15.42:0/1618607076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee4068a90 con 0x7fbee406c390 2024-05-28T13:59:15.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.821+0000 7fbedaa00640 1 -- 172.21.15.42:0/1618607076 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbed401a070 con 0x7fbee406c390 2024-05-28T13:59:15.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.821+0000 7fbedaa00640 1 --2- 172.21.15.42:0/1618607076 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbec0068380 0x7fbec006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:15.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.821+0000 7fbedaa00640 1 -- 172.21.15.42:0/1618607076 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbed4086090 con 0x7fbee406c390 2024-05-28T13:59:15.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.821+0000 7fbed9400640 1 --2- 172.21.15.42:0/1618607076 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbec0068380 0x7fbec006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:15.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.821+0000 7fbed9400640 1 --2- 172.21.15.42:0/1618607076 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbec0068380 0x7fbec006a840 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbee4156050 tx=0x7fbecc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:15.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.825+0000 7fbedaa00640 1 -- 172.21.15.42:0/1618607076 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbed4056540 con 0x7fbee406c390 2024-05-28T13:59:15.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.989+0000 7fbee9000640 1 -- 172.21.15.42:0/1618607076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbee406dd80 con 0x7fbee406c390 2024-05-28T13:59:15.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.989+0000 7fbedaa00640 1 -- 172.21.15.42:0/1618607076 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16737 (secure 0 0 0) 0x7fbed4055e40 con 0x7fbee406c390 2024-05-28T13:59:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:15.976 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:59:03.992606+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:58:20.789974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-28T13:58:22.021080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:59:15.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.993+0000 7fbee9000640 1 -- 172.21.15.42:0/1618607076 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbec0068380 msgr2=0x7fbec006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.993+0000 7fbee9000640 1 --2- 172.21.15.42:0/1618607076 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbec0068380 0x7fbec006a840 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbee4156050 tx=0x7fbecc009040 comp rx=0 tx=0).stop 2024-05-28T13:59:15.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.993+0000 7fbee9000640 1 -- 172.21.15.42:0/1618607076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee406c390 msgr2=0x7fbee415fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:15.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.993+0000 7fbee9000640 1 --2- 172.21.15.42:0/1618607076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee406c390 0x7fbee415fc30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbed400c780 tx=0x7fbed400cc40 comp rx=0 tx=0).stop 2024-05-28T13:59:15.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.993+0000 7fbee9000640 1 -- 172.21.15.42:0/1618607076 shutdown_connections 2024-05-28T13:59:15.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.993+0000 7fbee9000640 1 --2- 172.21.15.42:0/1618607076 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbec0068380 0x7fbec006a840 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.993+0000 7fbee9000640 1 --2- 172.21.15.42:0/1618607076 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbee415a460 0x7fbee4152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.993+0000 7fbee9000640 1 --2- 172.21.15.42:0/1618607076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbee40ce1b0 0x7fbee4152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.993+0000 7fbee9000640 1 --2- 172.21.15.42:0/1618607076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee406c390 0x7fbee415fc30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:15.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.993+0000 7fbee9000640 1 -- 172.21.15.42:0/1618607076 >> 172.21.15.42:0/1618607076 conn(0x7fbee40b2f60 msgr2=0x7fbee41599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:15.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.993+0000 7fbee9000640 1 -- 172.21.15.42:0/1618607076 shutdown_connections 2024-05-28T13:59:15.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:15.993+0000 7fbee9000640 1 -- 172.21.15.42:0/1618607076 wait complete. 2024-05-28T13:59:15.998 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-28T13:59:16.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.153+0000 7f5a8fe00640 1 Processor -- start 2024-05-28T13:59:16.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.153+0000 7f5a8fe00640 1 -- start start 2024-05-28T13:59:16.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a8806f030 0x7f5a8806db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:16.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8fe00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a8806f480 con 0x7f5a88145920 2024-05-28T13:59:16.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8fe00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a8806e240 con 0x7f5a880724d0 2024-05-28T13:59:16.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8fe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5a8806e520 con 0x7f5a8806f030 2024-05-28T13:59:16.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8d400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f5a88145920 0x7f5a88071dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46154/0 (socket says 172.21.15.42:46154) 2024-05-28T13:59:16.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8d400640 1 -- 172.21.15.42:0/3028121629 learned_addr learned my addr 172.21.15.42:0/3028121629 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:16.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 -- 172.21.15.42:0/3028121629 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1850620866 0 0) 0x7f5a8806f480 con 0x7f5a88145920 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 -- 172.21.15.42:0/3028121629 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a70003530 con 0x7f5a88145920 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8de00640 1 --2- 172.21.15.42:0/3028121629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a8806f030 0x7f5a8806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 -- 172.21.15.42:0/3028121629 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3524383031 0 0) 0x7f5a70003530 con 0x7f5a88145920 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 -- 172.21.15.42:0/3028121629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a8806f030 msgr2=0x7f5a8806db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 --2- 172.21.15.42:0/3028121629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a8806f030 0x7f5a8806db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 -- 172.21.15.42:0/3028121629 >> v1:172.21.15.63:6789/0 conn(0x7f5a880724d0 legacy=0x7f5a8815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 -- 172.21.15.42:0/3028121629 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8806e800 con 0x7f5a88145920 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 -- 172.21.15.42:0/3028121629 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f5a7c001680 con 0x7f5a88145920 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 -- 172.21.15.42:0/3028121629 >> v1:172.21.15.42:6789/0 conn(0x7f5a88145920 legacy=0x7f5a88071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 --2- 172.21.15.42:0/3028121629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a700039f0 0x7f5a70003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 --2- 172.21.15.42:0/3028121629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a8806f030 0x7f5a70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 --2- 172.21.15.42:0/3028121629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70004d50 0x7f5a700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 -- 172.21.15.42:0/3028121629 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8806e520 con 0x7f5a70004d50 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 -- 172.21.15.42:0/3028121629 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7000bb00 con 0x7f5a700039f0 2024-05-28T13:59:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a87e00640 1 -- 172.21.15.42:0/3028121629 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5a7000bd00 con 0x7f5a8806f030 2024-05-28T13:59:16.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8de00640 1 --2- 172.21.15.42:0/3028121629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70004d50 0x7f5a700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:16.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8d400640 1 --2- 172.21.15.42:0/3028121629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a700039f0 0x7f5a70003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:16.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8ca00640 1 --2- 172.21.15.42:0/3028121629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a8806f030 0x7f5a70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:16.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8de00640 1 -- 172.21.15.42:0/3028121629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a8806f030 msgr2=0x7f5a70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:16.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8de00640 1 --2- 172.21.15.42:0/3028121629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a8806f030 0x7f5a70004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8de00640 1 -- 172.21.15.42:0/3028121629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a700039f0 msgr2=0x7f5a70003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:16.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8de00640 1 --2- 172.21.15.42:0/3028121629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a700039f0 0x7f5a70003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.157+0000 7f5a8de00640 1 -- 172.21.15.42:0/3028121629 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a70003530 con 0x7f5a70004d50 2024-05-28T13:59:16.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.161+0000 7f5a8de00640 1 --2- 172.21.15.42:0/3028121629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70004d50 0x7f5a700051d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009fc0 tx=0x7f5a8001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c434f92152cf4049 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:16.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.161+0000 7f5a87e00640 1 -- 172.21.15.42:0/3028121629 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a800b2070 con 0x7f5a70004d50 2024-05-28T13:59:16.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.161+0000 7f5a87e00640 1 -- 172.21.15.42:0/3028121629 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a800a5c90 con 0x7f5a70004d50 2024-05-28T13:59:16.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.161+0000 7f5a87e00640 1 -- 172.21.15.42:0/3028121629 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80009c10 con 0x7f5a70004d50 2024-05-28T13:59:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.161+0000 7f5a8fe00640 1 -- 172.21.15.42:0/3028121629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70004d50 msgr2=0x7f5a700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.161+0000 7f5a8fe00640 1 --2- 172.21.15.42:0/3028121629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70004d50 0x7f5a700051d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009fc0 tx=0x7f5a8001cc80 comp rx=0 tx=0).stop 2024-05-28T13:59:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.161+0000 7f5a8de00640 1 -- 172.21.15.42:0/3028121629 reap_dead start 2024-05-28T13:59:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.161+0000 7f5a8fe00640 1 -- 172.21.15.42:0/3028121629 shutdown_connections 2024-05-28T13:59:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.161+0000 7f5a8fe00640 1 -- 172.21.15.42:0/3028121629 >> 172.21.15.42:0/3028121629 conn(0x7f5a8806b9f0 msgr2=0x7f5a8806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.161+0000 7f5a8fe00640 1 -- 172.21.15.42:0/3028121629 shutdown_connections 2024-05-28T13:59:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.161+0000 7f5a8fe00640 1 -- 172.21.15.42:0/3028121629 wait complete. 2024-05-28T13:59:16.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.161+0000 7f5a8fe00640 1 Processor -- start 2024-05-28T13:59:16.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.161+0000 7f5a8fe00640 1 -- start start 2024-05-28T13:59:16.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a70004d50 0x7f5a881f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:16.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a880b35f0 0x7f5a881f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:16.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a880b4b90 0x7f5a881e7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:16.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8815cb30 con 0x7f5a880b35f0 2024-05-28T13:59:16.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8806db40 con 0x7f5a70004d50 2024-05-28T13:59:16.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8fe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5a880c57d0 con 0x7f5a880b4b90 2024-05-28T13:59:16.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a880b35f0 0x7f5a881f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:16.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a880b35f0 0x7f5a881f4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48238/0 (socket says 172.21.15.42:48238) 2024-05-28T13:59:16.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8ca00640 1 -- 172.21.15.42:0/2082904507 learned_addr learned my addr 172.21.15.42:0/2082904507 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:16.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8de00640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a880b4b90 0x7f5a881e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8d400640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a70004d50 0x7f5a881f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8ca00640 1 -- 172.21.15.42:0/2082904507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a880b4b90 msgr2=0x7f5a881e7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8ca00640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a880b4b90 0x7f5a881e7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8ca00640 1 -- 172.21.15.42:0/2082904507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a70004d50 msgr2=0x7f5a881f5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8ca00640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a70004d50 0x7f5a881f5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8ca00640 1 -- 172.21.15.42:0/2082904507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a880b3af0 con 0x7f5a880b35f0 2024-05-28T13:59:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8de00640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a880b4b90 0x7f5a881e7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8d400640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a70004d50 0x7f5a881f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8ca00640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a880b35f0 0x7f5a881f4170 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400e550 tx=0x7f5a7400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a85600640 1 -- 172.21.15.42:0/2082904507 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a740052e0 con 0x7f5a880b35f0 2024-05-28T13:59:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a85600640 1 -- 172.21.15.42:0/2082904507 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a74018070 con 0x7f5a880b35f0 2024-05-28T13:59:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a85600640 1 -- 172.21.15.42:0/2082904507 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a74003820 con 0x7f5a880b35f0 2024-05-28T13:59:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8fe00640 1 -- 172.21.15.42:0/2082904507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a880b3dd0 con 0x7f5a880b35f0 2024-05-28T13:59:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8fe00640 1 -- 172.21.15.42:0/2082904507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a880b4390 con 0x7f5a880b35f0 2024-05-28T13:59:16.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.165+0000 7f5a8fe00640 1 -- 172.21.15.42:0/2082904507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a88073890 con 0x7f5a880b35f0 2024-05-28T13:59:16.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.169+0000 7f5a85600640 1 -- 172.21.15.42:0/2082904507 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5a740039c0 con 0x7f5a880b35f0 2024-05-28T13:59:16.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.169+0000 7f5a85600640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a68068380 0x7f5a6806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:16.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.169+0000 7f5a85600640 1 -- 172.21.15.42:0/2082904507 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5a74086850 con 0x7f5a880b35f0 2024-05-28T13:59:16.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.169+0000 7f5a8d400640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a68068380 0x7f5a6806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.169+0000 7f5a8d400640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a68068380 0x7f5a6806a840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c001b90 tx=0x7f5a7c001b20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:16.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.173+0000 7f5a85600640 1 -- 172.21.15.42:0/2082904507 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5a74056cf0 con 0x7f5a880b35f0 2024-05-28T13:59:16.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.373+0000 7f5a8fe00640 1 -- 172.21.15.42:0/2082904507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5a8804fdd0 con 0x7f5a880b35f0 2024-05-28T13:59:16.359 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-05-28T13:59:16.373+0000 7f248b600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-28T13:59:16.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.377+0000 7f5a85600640 1 -- 172.21.15.42:0/2082904507 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f5a74005480 con 0x7f5a880b35f0 2024-05-28T13:59:16.361 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-28T13:59:16.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.377+0000 7f5a8fe00640 1 -- 172.21.15.42:0/2082904507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a68068380 msgr2=0x7f5a6806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:16.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.377+0000 7f5a8fe00640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a68068380 0x7f5a6806a840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c001b90 tx=0x7f5a7c001b20 comp rx=0 tx=0).stop 2024-05-28T13:59:16.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.377+0000 7f5a8fe00640 1 -- 172.21.15.42:0/2082904507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a880b35f0 msgr2=0x7f5a881f4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:16.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.377+0000 7f5a8fe00640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a880b35f0 0x7f5a881f4170 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400e550 tx=0x7f5a7400ea10 comp rx=0 tx=0).stop 2024-05-28T13:59:16.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.377+0000 7f5a8fe00640 1 -- 172.21.15.42:0/2082904507 shutdown_connections 2024-05-28T13:59:16.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.377+0000 7f5a8fe00640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a68068380 0x7f5a6806a840 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.377+0000 7f5a8fe00640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a880b4b90 0x7f5a881e7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.377+0000 7f5a8fe00640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a880b35f0 0x7f5a881f4170 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.377+0000 7f5a8fe00640 1 --2- 172.21.15.42:0/2082904507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a70004d50 0x7f5a881f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.377+0000 7f5a8fe00640 1 -- 172.21.15.42:0/2082904507 >> 172.21.15.42:0/2082904507 conn(0x7f5a8806b9f0 msgr2=0x7f5a880b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:16.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.377+0000 7f5a8fe00640 1 -- 172.21.15.42:0/2082904507 shutdown_connections 2024-05-28T13:59:16.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.381+0000 7f5a8fe00640 1 -- 172.21.15.42:0/2082904507 wait complete. 2024-05-28T13:59:16.373 INFO:tasks.ceph.mds.b.smithi063.stderr: -3342> 2024-05-28T13:58:27.044+0000 7f88d8e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-28T13:59:16.385 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-28T13:59:16.425 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T13:59:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.541+0000 7f4186600640 1 Processor -- start 2024-05-28T13:59:16.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.541+0000 7f4186600640 1 -- start start 2024-05-28T13:59:16.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.545+0000 7f4186600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4180144f80 0x7f4180072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:16.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.545+0000 7f4186600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f418006ee20 0x7f418006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:16.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.545+0000 7f4186600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41800afe40 con 0x7f418015b9b0 2024-05-28T13:59:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.545+0000 7f4186600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f418006f7c0 con 0x7f418006ee20 2024-05-28T13:59:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.545+0000 7f4186600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f41800724d0 con 0x7f4180144f80 2024-05-28T13:59:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.545+0000 7f4177e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f418006ee20 0x7f418006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.545+0000 7f417fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f418015b9b0 0x7f418015be40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46166/0 (socket says 172.21.15.42:46166) 2024-05-28T13:59:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.545+0000 7f417fe00640 1 -- 172.21.15.42:0/3436668110 learned_addr learned my addr 172.21.15.42:0/3436668110 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.545+0000 7f417f400640 1 --2- 172.21.15.42:0/3436668110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4180144f80 0x7f4180072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.545+0000 7f4177e00640 1 -- 172.21.15.42:0/3436668110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4180144f80 msgr2=0x7f4180072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.545+0000 7f4177e00640 1 --2- 172.21.15.42:0/3436668110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4180144f80 0x7f4180072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.545+0000 7f4177e00640 1 -- 172.21.15.42:0/3436668110 >> v1:172.21.15.42:6789/0 conn(0x7f418015b9b0 legacy=0x7f418015be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.545+0000 7f4177e00640 1 -- 172.21.15.42:0/3436668110 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418015c600 con 0x7f418006ee20 2024-05-28T13:59:16.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.545+0000 7f417ea00640 1 -- 172.21.15.42:0/3436668110 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2028432344 0 0) 0x7f41800afe40 con 0x7f418015b9b0 2024-05-28T13:59:16.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f4177e00640 1 --2- 172.21.15.42:0/3436668110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418006ee20 0x7f418006f280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f41680097c0 tx=0x7f416801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891e9b684c617c41 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:16.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f417ea00640 1 -- 172.21.15.42:0/3436668110 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41680b2070 con 0x7f418006ee20 2024-05-28T13:59:16.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f417ea00640 1 -- 172.21.15.42:0/3436668110 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41680ad640 con 0x7f418006ee20 2024-05-28T13:59:16.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f417ea00640 1 -- 172.21.15.42:0/3436668110 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41680ada10 con 0x7f418006ee20 2024-05-28T13:59:16.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f4186600640 1 -- 172.21.15.42:0/3436668110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418006ee20 msgr2=0x7f418006f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:16.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f4186600640 1 --2- 172.21.15.42:0/3436668110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418006ee20 0x7f418006f280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f41680097c0 tx=0x7f416801c4a0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f4186600640 1 -- 172.21.15.42:0/3436668110 shutdown_connections 2024-05-28T13:59:16.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f4186600640 1 --2- 172.21.15.42:0/3436668110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418006ee20 0x7f418006f280 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f4186600640 1 --2- 172.21.15.42:0/3436668110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4180144f80 0x7f4180072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f4186600640 1 -- 172.21.15.42:0/3436668110 >> 172.21.15.42:0/3436668110 conn(0x7f41800b21f0 msgr2=0x7f41800b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:16.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f4186600640 1 -- 172.21.15.42:0/3436668110 shutdown_connections 2024-05-28T13:59:16.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f4186600640 1 -- 172.21.15.42:0/3436668110 wait complete. 2024-05-28T13:59:16.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f4186600640 1 Processor -- start 2024-05-28T13:59:16.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f4186600640 1 -- start start 2024-05-28T13:59:16.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f4186600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418006ee20 0x7f41801549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:16.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.549+0000 7f4186600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4180144f80 0x7f418013a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:16.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f4186600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418015b9b0 0x7f4180142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:16.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f4186600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4180072ea0 con 0x7f418006ee20 2024-05-28T13:59:16.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f4186600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f41800b4010 con 0x7f418015b9b0 2024-05-28T13:59:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f4186600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f418015d700 con 0x7f4180144f80 2024-05-28T13:59:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f417f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418006ee20 0x7f41801549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f417f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418006ee20 0x7f41801549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48258/0 (socket says 172.21.15.42:48258) 2024-05-28T13:59:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f417f400640 1 -- 172.21.15.42:0/2624842102 learned_addr learned my addr 172.21.15.42:0/2624842102 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f4177e00640 1 --2- 172.21.15.42:0/2624842102 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4180144f80 0x7f418013a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f417fe00640 1 --2- 172.21.15.42:0/2624842102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418015b9b0 0x7f4180142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f417f400640 1 -- 172.21.15.42:0/2624842102 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4180144f80 msgr2=0x7f418013a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f417f400640 1 --2- 172.21.15.42:0/2624842102 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4180144f80 0x7f418013a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f417f400640 1 -- 172.21.15.42:0/2624842102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418015b9b0 msgr2=0x7f4180142b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f417f400640 1 --2- 172.21.15.42:0/2624842102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418015b9b0 0x7f4180142b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f417f400640 1 -- 172.21.15.42:0/2624842102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418015d9e0 con 0x7f418006ee20 2024-05-28T13:59:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f417f400640 1 --2- 172.21.15.42:0/2624842102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418006ee20 0x7f41801549d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f417000e930 tx=0x7f417000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f4177400640 1 -- 172.21.15.42:0/2624842102 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170017070 con 0x7f418006ee20 2024-05-28T13:59:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f4177400640 1 -- 172.21.15.42:0/2624842102 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f417000ca00 con 0x7f418006ee20 2024-05-28T13:59:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f4186600640 1 -- 172.21.15.42:0/2624842102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418015dcc0 con 0x7f418006ee20 2024-05-28T13:59:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f4177400640 1 -- 172.21.15.42:0/2624842102 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417000cdb0 con 0x7f418006ee20 2024-05-28T13:59:16.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f4186600640 1 -- 172.21.15.42:0/2624842102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4180154f10 con 0x7f418006ee20 2024-05-28T13:59:16.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f4186600640 1 -- 172.21.15.42:0/2624842102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41800738b0 con 0x7f418006ee20 2024-05-28T13:59:16.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.553+0000 7f4177400640 1 -- 172.21.15.42:0/2624842102 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f417001a070 con 0x7f418006ee20 2024-05-28T13:59:16.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.557+0000 7f4177400640 1 --2- 172.21.15.42:0/2624842102 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f415c0683b0 0x7f415c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:16.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.557+0000 7f4177400640 1 -- 172.21.15.42:0/2624842102 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5911+0+0 (secure 0 0 0) 0x7f4170086b20 con 0x7f418006ee20 2024-05-28T13:59:16.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.557+0000 7f4177e00640 1 --2- 172.21.15.42:0/2624842102 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f415c0683b0 0x7f415c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:16.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.557+0000 7f4177e00640 1 --2- 172.21.15.42:0/2624842102 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f415c0683b0 0x7f415c06a870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f41680af000 tx=0x7f4168008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:16.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.561+0000 7f4177400640 1 -- 172.21.15.42:0/2624842102 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4170056950 con 0x7f418006ee20 2024-05-28T13:59:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:16.761+0000 7f4186600640 1 -- 172.21.15.42:0/2624842102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f418004fdd0 con 0x7f418006ee20 2024-05-28T13:59:17.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.381+0000 7f4177400640 1 -- 172.21.15.42:0/2624842102 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f417005ac50 con 0x7f418006ee20 2024-05-28T13:59:17.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.385+0000 7f4186600640 1 -- 172.21.15.42:0/2624842102 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f415c0683b0 msgr2=0x7f415c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:17.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.385+0000 7f4186600640 1 --2- 172.21.15.42:0/2624842102 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f415c0683b0 0x7f415c06a870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f41680af000 tx=0x7f4168008040 comp rx=0 tx=0).stop 2024-05-28T13:59:17.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.385+0000 7f4186600640 1 -- 172.21.15.42:0/2624842102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418006ee20 msgr2=0x7f41801549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:17.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.385+0000 7f4186600640 1 --2- 172.21.15.42:0/2624842102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418006ee20 0x7f41801549d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f417000e930 tx=0x7f417000edf0 comp rx=0 tx=0).stop 2024-05-28T13:59:17.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.385+0000 7f4186600640 1 -- 172.21.15.42:0/2624842102 shutdown_connections 2024-05-28T13:59:17.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.385+0000 7f4186600640 1 --2- 172.21.15.42:0/2624842102 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f415c0683b0 0x7f415c06a870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:17.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.385+0000 7f4186600640 1 --2- 172.21.15.42:0/2624842102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418015b9b0 0x7f4180142b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:17.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.385+0000 7f4186600640 1 --2- 172.21.15.42:0/2624842102 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4180144f80 0x7f418013a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:17.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.385+0000 7f4186600640 1 --2- 172.21.15.42:0/2624842102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418006ee20 0x7f41801549d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:17.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.385+0000 7f4186600640 1 -- 172.21.15.42:0/2624842102 >> 172.21.15.42:0/2624842102 conn(0x7f41800b21f0 msgr2=0x7f418015a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:17.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.385+0000 7f4186600640 1 -- 172.21.15.42:0/2624842102 shutdown_connections 2024-05-28T13:59:17.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.385+0000 7f4186600640 1 -- 172.21.15.42:0/2624842102 wait complete. 2024-05-28T13:59:17.393 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-28T13:59:17.432 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T13:59:17.444 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T13:59:17.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.557+0000 7f5677200640 1 Processor -- start 2024-05-28T13:59:17.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.557+0000 7f5677200640 1 -- start start 2024-05-28T13:59:17.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f5677200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f567015eb10 0x7f56700cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:17.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f5677200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56700b3a50 0x7f56700cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:17.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f5677200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56700cf190 con 0x7f567006b9f0 2024-05-28T13:59:17.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f5677200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f56700d05e0 con 0x7f567015eb10 2024-05-28T13:59:17.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f5677200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f56700d0810 con 0x7f56700b3a50 2024-05-28T13:59:17.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f566fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f567015eb10 0x7f56700cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:17.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f5675200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f567006b9f0 0x7f56700ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:43778/0 (socket says 172.21.15.42:43778) 2024-05-28T13:59:17.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f5675200640 1 -- 172.21.15.42:0/2959489499 learned_addr learned my addr 172.21.15.42:0/2959489499 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:17.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f5667400640 1 --2- 172.21.15.42:0/2959489499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56700b3a50 0x7f56700cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:17.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f566f400640 1 -- 172.21.15.42:0/2959489499 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1492534877 0 0) 0x7f56700cf190 con 0x7f567006b9f0 2024-05-28T13:59:17.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f566f400640 1 -- 172.21.15.42:0/2959489499 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5654003530 con 0x7f567006b9f0 2024-05-28T13:59:17.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f566fe00640 1 -- 172.21.15.42:0/2959489499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56700b3a50 msgr2=0x7f56700cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:17.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f566fe00640 1 --2- 172.21.15.42:0/2959489499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56700b3a50 0x7f56700cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:17.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f566fe00640 1 -- 172.21.15.42:0/2959489499 >> v1:172.21.15.42:6789/0 conn(0x7f567006b9f0 legacy=0x7f56700ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:17.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.561+0000 7f566fe00640 1 -- 172.21.15.42:0/2959489499 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56700d0af0 con 0x7f567015eb10 2024-05-28T13:59:17.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.565+0000 7f566fe00640 1 --2- 172.21.15.42:0/2959489499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f567015eb10 0x7f56700cd520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f56600097c0 tx=0x7f566001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e96135ae3ab60 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:17.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.565+0000 7f566f400640 1 -- 172.21.15.42:0/2959489499 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56600b0070 con 0x7f567015eb10 2024-05-28T13:59:17.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.565+0000 7f566f400640 1 -- 172.21.15.42:0/2959489499 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56600ab600 con 0x7f567015eb10 2024-05-28T13:59:17.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.565+0000 7f566f400640 1 -- 172.21.15.42:0/2959489499 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56600ab9b0 con 0x7f567015eb10 2024-05-28T13:59:17.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.565+0000 7f5677200640 1 -- 172.21.15.42:0/2959489499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f567015eb10 msgr2=0x7f56700cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:17.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.565+0000 7f5677200640 1 --2- 172.21.15.42:0/2959489499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f567015eb10 0x7f56700cd520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f56600097c0 tx=0x7f566001c410 comp rx=0 tx=0).stop 2024-05-28T13:59:17.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.565+0000 7f5677200640 1 -- 172.21.15.42:0/2959489499 shutdown_connections 2024-05-28T13:59:17.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.565+0000 7f5677200640 1 --2- 172.21.15.42:0/2959489499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56700b3a50 0x7f56700cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:17.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.565+0000 7f5677200640 1 --2- 172.21.15.42:0/2959489499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f567015eb10 0x7f56700cd520 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:17.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.565+0000 7f5677200640 1 -- 172.21.15.42:0/2959489499 >> 172.21.15.42:0/2959489499 conn(0x7f56700b2f60 msgr2=0x7f56700b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:17.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.565+0000 7f5677200640 1 -- 172.21.15.42:0/2959489499 shutdown_connections 2024-05-28T13:59:17.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.565+0000 7f5677200640 1 -- 172.21.15.42:0/2959489499 wait complete. 2024-05-28T13:59:17.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.565+0000 7f5677200640 1 Processor -- start 2024-05-28T13:59:17.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.565+0000 7f5677200640 1 -- start start 2024-05-28T13:59:17.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5677200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f567006b9f0 0x7f5670156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:17.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5677200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56700b3a50 0x7f56700af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:17.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5677200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f567015eb10 0x7f56700ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:17.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5677200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f567006c210 con 0x7f56700b3a50 2024-05-28T13:59:17.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5677200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56700cec40 con 0x7f567006b9f0 2024-05-28T13:59:17.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5677200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5670156ec0 con 0x7f567015eb10 2024-05-28T13:59:17.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5667400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56700b3a50 0x7f56700af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:17.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5667400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56700b3a50 0x7f56700af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43904/0 (socket says 172.21.15.42:43904) 2024-05-28T13:59:17.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5667400640 1 -- 172.21.15.42:0/4012915179 learned_addr learned my addr 172.21.15.42:0/4012915179 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:17.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f566fe00640 1 --2- 172.21.15.42:0/4012915179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f567006b9f0 0x7f5670156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:17.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5667400640 1 -- 172.21.15.42:0/4012915179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f567015eb10 msgr2=0x7f56700ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:17.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5667400640 1 --2- 172.21.15.42:0/4012915179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f567015eb10 0x7f56700ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:17.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5667400640 1 -- 172.21.15.42:0/4012915179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f567006b9f0 msgr2=0x7f5670156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:17.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5667400640 1 --2- 172.21.15.42:0/4012915179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f567006b9f0 0x7f5670156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:17.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5667400640 1 -- 172.21.15.42:0/4012915179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567015f100 con 0x7f56700b3a50 2024-05-28T13:59:17.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f566fe00640 1 --2- 172.21.15.42:0/4012915179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f567006b9f0 0x7f5670156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:17.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5667400640 1 --2- 172.21.15.42:0/4012915179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56700b3a50 0x7f56700af8b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f565800e870 tx=0x7f565800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:17.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f566cc00640 1 -- 172.21.15.42:0/4012915179 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658017070 con 0x7f56700b3a50 2024-05-28T13:59:17.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f566cc00640 1 -- 172.21.15.42:0/4012915179 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f565800c940 con 0x7f56700b3a50 2024-05-28T13:59:17.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f566cc00640 1 -- 172.21.15.42:0/4012915179 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565800ccf0 con 0x7f56700b3a50 2024-05-28T13:59:17.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5677200640 1 -- 172.21.15.42:0/4012915179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567015f3e0 con 0x7f56700b3a50 2024-05-28T13:59:17.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5677200640 1 -- 172.21.15.42:0/4012915179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567015f9a0 con 0x7f56700b3a50 2024-05-28T13:59:17.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f5677200640 1 -- 172.21.15.42:0/4012915179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5670068a90 con 0x7f56700b3a50 2024-05-28T13:59:17.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.569+0000 7f566cc00640 1 -- 172.21.15.42:0/4012915179 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f565801a070 con 0x7f56700b3a50 2024-05-28T13:59:17.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.573+0000 7f566cc00640 1 --2- 172.21.15.42:0/4012915179 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f564c068380 0x7f564c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:17.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.573+0000 7f566cc00640 1 -- 172.21.15.42:0/4012915179 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5911+0+0 (secure 0 0 0) 0x7f5658086970 con 0x7f56700b3a50 2024-05-28T13:59:17.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.573+0000 7f566fe00640 1 --2- 172.21.15.42:0/4012915179 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f564c068380 0x7f564c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:17.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.573+0000 7f566fe00640 1 --2- 172.21.15.42:0/4012915179 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f564c068380 0x7f564c06a840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f56600ad000 tx=0x7f5660008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:17.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.577+0000 7f566cc00640 1 -- 172.21.15.42:0/4012915179 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5658056e50 con 0x7f56700b3a50 2024-05-28T13:59:17.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:17.741+0000 7f5677200640 1 -- 172.21.15.42:0/4012915179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f56700cb5b0 con 0x7f56700b3a50 2024-05-28T13:59:18.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.385+0000 7f566cc00640 1 -- 172.21.15.42:0/4012915179 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7f5658056750 con 0x7f56700b3a50 2024-05-28T13:59:18.369 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-05-28T13:59:18.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.389+0000 7f5677200640 1 -- 172.21.15.42:0/4012915179 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f564c068380 msgr2=0x7f564c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:18.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.389+0000 7f5677200640 1 --2- 172.21.15.42:0/4012915179 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f564c068380 0x7f564c06a840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f56600ad000 tx=0x7f5660008040 comp rx=0 tx=0).stop 2024-05-28T13:59:18.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.389+0000 7f5677200640 1 -- 172.21.15.42:0/4012915179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56700b3a50 msgr2=0x7f56700af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:18.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.389+0000 7f5677200640 1 --2- 172.21.15.42:0/4012915179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56700b3a50 0x7f56700af8b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f565800e870 tx=0x7f565800ec70 comp rx=0 tx=0).stop 2024-05-28T13:59:18.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.389+0000 7f5677200640 1 -- 172.21.15.42:0/4012915179 shutdown_connections 2024-05-28T13:59:18.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.389+0000 7f5677200640 1 --2- 172.21.15.42:0/4012915179 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f564c068380 0x7f564c06a840 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:18.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.389+0000 7f5677200640 1 --2- 172.21.15.42:0/4012915179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f567015eb10 0x7f56700ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:18.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.389+0000 7f5677200640 1 --2- 172.21.15.42:0/4012915179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56700b3a50 0x7f56700af8b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:18.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.389+0000 7f5677200640 1 --2- 172.21.15.42:0/4012915179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f567006b9f0 0x7f5670156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:18.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.389+0000 7f5677200640 1 -- 172.21.15.42:0/4012915179 >> 172.21.15.42:0/4012915179 conn(0x7f56700b2f60 msgr2=0x7f567015b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:18.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.389+0000 7f5677200640 1 -- 172.21.15.42:0/4012915179 shutdown_connections 2024-05-28T13:59:18.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.389+0000 7f5677200640 1 -- 172.21.15.42:0/4012915179 wait complete. 2024-05-28T13:59:18.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-28T13:59:18.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.565+0000 7f6035400640 1 Processor -- start 2024-05-28T13:59:18.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.565+0000 7f6035400640 1 -- start start 2024-05-28T13:59:18.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.565+0000 7f6035400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f603009b160 0x7f60300b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:18.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.565+0000 7f6035400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f603009bbd0 con 0x7f603015fb50 2024-05-28T13:59:18.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.565+0000 7f6035400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60300cdc00 con 0x7f603015ff30 2024-05-28T13:59:18.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.565+0000 7f6035400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f60300ce250 con 0x7f603009b160 2024-05-28T13:59:18.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.565+0000 7f6025e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f603009b160 0x7f60300b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:18.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.565+0000 7f602e800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f603015fb50 0x7f603009aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:43794/0 (socket says 172.21.15.42:43794) 2024-05-28T13:59:18.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.565+0000 7f602e800640 1 -- 172.21.15.42:0/3141352617 learned_addr learned my addr 172.21.15.42:0/3141352617 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:18.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.565+0000 7f6025e00640 1 -- 172.21.15.42:0/3141352617 >> v1:172.21.15.63:6789/0 conn(0x7f603015ff30 legacy=0x7f60300b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:18.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.565+0000 7f6025e00640 1 -- 172.21.15.42:0/3141352617 >> v1:172.21.15.42:6789/0 conn(0x7f603015fb50 legacy=0x7f603009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:18.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.565+0000 7f6025e00640 1 -- 172.21.15.42:0/3141352617 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60300cdee0 con 0x7f603009b160 2024-05-28T13:59:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.569+0000 7f6025e00640 1 --2- 172.21.15.42:0/3141352617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f603009b160 0x7f60300b1e50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f60180097c0 tx=0x7f601801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc41b9a50d02ed95 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.569+0000 7f602de00640 1 -- 172.21.15.42:0/3141352617 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60180ab5f0 con 0x7f603009b160 2024-05-28T13:59:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.569+0000 7f602de00640 1 -- 172.21.15.42:0/3141352617 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60180b0070 con 0x7f603009b160 2024-05-28T13:59:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.569+0000 7f602de00640 1 -- 172.21.15.42:0/3141352617 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60180ab9b0 con 0x7f603009b160 2024-05-28T13:59:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.569+0000 7f6035400640 1 -- 172.21.15.42:0/3141352617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f603009b160 msgr2=0x7f60300b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.569+0000 7f6035400640 1 --2- 172.21.15.42:0/3141352617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f603009b160 0x7f60300b1e50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f60180097c0 tx=0x7f601801c3d0 comp rx=0 tx=0).stop 2024-05-28T13:59:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.569+0000 7f6035400640 1 -- 172.21.15.42:0/3141352617 shutdown_connections 2024-05-28T13:59:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.569+0000 7f6035400640 1 --2- 172.21.15.42:0/3141352617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f603009b160 0x7f60300b1e50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.569+0000 7f6035400640 1 -- 172.21.15.42:0/3141352617 >> 172.21.15.42:0/3141352617 conn(0x7f60300b3f50 msgr2=0x7f60300b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.569+0000 7f6035400640 1 -- 172.21.15.42:0/3141352617 shutdown_connections 2024-05-28T13:59:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.569+0000 7f6035400640 1 -- 172.21.15.42:0/3141352617 wait complete. 2024-05-28T13:59:18.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6035400640 1 Processor -- start 2024-05-28T13:59:18.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6035400640 1 -- start start 2024-05-28T13:59:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6035400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f603009b160 0x7f603006e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6035400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f603015fb50 0x7f603006d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6035400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f603015ff30 0x7f6030158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6035400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f603016a7d0 con 0x7f603015ff30 2024-05-28T13:59:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6035400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f603016a650 con 0x7f603009b160 2024-05-28T13:59:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6035400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f603016a950 con 0x7f603015fb50 2024-05-28T13:59:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f602f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f603015ff30 0x7f6030158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f602f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f603015ff30 0x7f6030158de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43928/0 (socket says 172.21.15.42:43928) 2024-05-28T13:59:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f602f200640 1 -- 172.21.15.42:0/492271862 learned_addr learned my addr 172.21.15.42:0/492271862 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f602e800640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f603009b160 0x7f603006e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6025e00640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f603015fb50 0x7f603006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f602f200640 1 -- 172.21.15.42:0/492271862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f603015fb50 msgr2=0x7f603006d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f602f200640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f603015fb50 0x7f603006d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f602f200640 1 -- 172.21.15.42:0/492271862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f603009b160 msgr2=0x7f603006e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f602f200640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f603009b160 0x7f603006e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f602f200640 1 -- 172.21.15.42:0/492271862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60300b5100 con 0x7f603015ff30 2024-05-28T13:59:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6025e00640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f603015fb50 0x7f603006d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f602e800640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f603009b160 0x7f603006e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:18.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f602f200640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f603015ff30 0x7f6030158de0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f602800b970 tx=0x7f602800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:18.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6027400640 1 -- 172.21.15.42:0/492271862 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028019070 con 0x7f603015ff30 2024-05-28T13:59:18.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6027400640 1 -- 172.21.15.42:0/492271862 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f602800cb40 con 0x7f603015ff30 2024-05-28T13:59:18.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6027400640 1 -- 172.21.15.42:0/492271862 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028014440 con 0x7f603015ff30 2024-05-28T13:59:18.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6035400640 1 -- 172.21.15.42:0/492271862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60300b53e0 con 0x7f603015ff30 2024-05-28T13:59:18.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6035400640 1 -- 172.21.15.42:0/492271862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60300b59a0 con 0x7f603015ff30 2024-05-28T13:59:18.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.573+0000 7f6035400640 1 -- 172.21.15.42:0/492271862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6030068a90 con 0x7f603015ff30 2024-05-28T13:59:18.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.577+0000 7f6027400640 1 -- 172.21.15.42:0/492271862 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f602801c070 con 0x7f603015ff30 2024-05-28T13:59:18.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.577+0000 7f6027400640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f600c068380 0x7f600c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:18.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.577+0000 7f6027400640 1 -- 172.21.15.42:0/492271862 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5473+0+0 (secure 0 0 0) 0x7f60280865c0 con 0x7f603015ff30 2024-05-28T13:59:18.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.577+0000 7f602e800640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f600c068380 0x7f600c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:18.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.577+0000 7f602e800640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f600c068380 0x7f600c06a840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6020000f70 tx=0x7f6020009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.581+0000 7f6027400640 1 -- 172.21.15.42:0/492271862 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6028056bc0 con 0x7f603015ff30 2024-05-28T13:59:18.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:18.749+0000 7f6035400640 1 -- 172.21.15.42:0/492271862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f60300c92f0 con 0x7f603015ff30 2024-05-28T13:59:19.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.389+0000 7f6027400640 1 -- 172.21.15.42:0/492271862 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7f60280564c0 con 0x7f603015ff30 2024-05-28T13:59:19.375 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-05-28T13:59:19.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.393+0000 7f6035400640 1 -- 172.21.15.42:0/492271862 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f600c068380 msgr2=0x7f600c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:19.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.393+0000 7f6035400640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f600c068380 0x7f600c06a840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6020000f70 tx=0x7f6020009040 comp rx=0 tx=0).stop 2024-05-28T13:59:19.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.393+0000 7f6035400640 1 -- 172.21.15.42:0/492271862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f603015ff30 msgr2=0x7f6030158de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:19.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.393+0000 7f6035400640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f603015ff30 0x7f6030158de0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f602800b970 tx=0x7f602800be30 comp rx=0 tx=0).stop 2024-05-28T13:59:19.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.393+0000 7f6035400640 1 -- 172.21.15.42:0/492271862 shutdown_connections 2024-05-28T13:59:19.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.393+0000 7f6035400640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f600c068380 0x7f600c06a840 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:19.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.393+0000 7f6035400640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f603015ff30 0x7f6030158de0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:19.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.393+0000 7f6035400640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f603015fb50 0x7f603006d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:19.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.393+0000 7f6035400640 1 --2- 172.21.15.42:0/492271862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f603009b160 0x7f603006e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:19.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.393+0000 7f6035400640 1 -- 172.21.15.42:0/492271862 >> 172.21.15.42:0/492271862 conn(0x7f60300b3f50 msgr2=0x7f60300af710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:19.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.393+0000 7f6035400640 1 -- 172.21.15.42:0/492271862 shutdown_connections 2024-05-28T13:59:19.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.393+0000 7f6035400640 1 -- 172.21.15.42:0/492271862 wait complete. 2024-05-28T13:59:19.407 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-28T13:59:19.407 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-28T13:59:19.408 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-28T13:59:19.408 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T13:59:19.408 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-28T13:59:19.408 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-28T13:59:19.409 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-28T13:59:19.409 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T13:59:19.409 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-28T13:59:19.410 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-28T13:59:19.410 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-28T13:59:19.410 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T13:59:19.410 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-28T13:59:19.411 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-28T13:59:19.411 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-28T13:59:19.411 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T13:59:19.421 INFO:tasks.ceph.mds.a:Stopped 2024-05-28T13:59:19.421 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-28T13:59:19.424 INFO:tasks.ceph.mds.c:Stopped 2024-05-28T13:59:19.424 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-28T13:59:19.427 INFO:tasks.ceph.mds.b:Stopped 2024-05-28T13:59:19.427 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-28T13:59:19.428 INFO:tasks.ceph.mds.d:Stopped 2024-05-28T13:59:19.428 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-28T13:59:19.470 INFO:tasks.ceph.mds.a:Started 2024-05-28T13:59:19.471 DEBUG:teuthology.parallel:result is None 2024-05-28T13:59:19.473 INFO:tasks.ceph.mds.c:Started 2024-05-28T13:59:19.473 DEBUG:teuthology.parallel:result is None 2024-05-28T13:59:19.474 INFO:tasks.ceph.mds.b:Started 2024-05-28T13:59:19.474 DEBUG:teuthology.parallel:result is None 2024-05-28T13:59:19.476 INFO:tasks.ceph.mds.d:Started 2024-05-28T13:59:19.476 DEBUG:teuthology.parallel:result is None 2024-05-28T13:59:19.476 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-28T13:59:19.563 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-05-28T13:59:19.563 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-05-28T13:59:19.566 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T13:59:19.566 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-28T13:59:19.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.641+0000 7fcfac800640 1 Processor -- start 2024-05-28T13:59:19.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.641+0000 7fcfac800640 1 -- start start 2024-05-28T13:59:19.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.645+0000 7fcfac800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa4162130 0x7fcfa40b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:19.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.645+0000 7fcfac800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcfa40cddc0 0x7fcfa409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:19.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.645+0000 7fcfac800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfa406dc50 con 0x7fcfa40d0870 2024-05-28T13:59:19.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.645+0000 7fcfac800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fcfa406df30 con 0x7fcfa40cddc0 2024-05-28T13:59:19.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.645+0000 7fcfac800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa406e210 con 0x7fcfa4162130 2024-05-28T13:59:19.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.645+0000 7fcfaa800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcfa40cddc0 0x7fcfa409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:19.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.645+0000 7fcfa9400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fcfa40d0870 0x7fcfa40d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:43802/0 (socket says 172.21.15.42:43802) 2024-05-28T13:59:19.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.645+0000 7fcfa9400640 1 -- 172.21.15.42:0/486360844 learned_addr learned my addr 172.21.15.42:0/486360844 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:19.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.645+0000 7fcfa9e00640 1 --2- 172.21.15.42:0/486360844 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa4162130 0x7fcfa40b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:19.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.645+0000 7fcfaa800640 1 -- 172.21.15.42:0/486360844 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa4162130 msgr2=0x7fcfa40b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:19.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.645+0000 7fcfaa800640 1 --2- 172.21.15.42:0/486360844 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa4162130 0x7fcfa40b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:19.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.645+0000 7fcfaa800640 1 -- 172.21.15.42:0/486360844 >> v1:172.21.15.42:6789/0 conn(0x7fcfa40d0870 legacy=0x7fcfa40d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:19.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.645+0000 7fcfaa800640 1 -- 172.21.15.42:0/486360844 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa406e4f0 con 0x7fcfa40cddc0 2024-05-28T13:59:19.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.645+0000 7fcfa8a00640 1 -- 172.21.15.42:0/486360844 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 383747696 0 0) 0x7fcfa406dc50 con 0x7fcfa40d0870 2024-05-28T13:59:19.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.649+0000 7fcfaa800640 1 --2- 172.21.15.42:0/486360844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa40cddc0 0x7fcfa409aa50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009d90 tx=0x7fcfa001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4115def3016a149a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:19.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.649+0000 7fcfa8a00640 1 -- 172.21.15.42:0/486360844 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00b2070 con 0x7fcfa40cddc0 2024-05-28T13:59:19.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.649+0000 7fcfa8a00640 1 -- 172.21.15.42:0/486360844 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa00a5940 con 0x7fcfa40cddc0 2024-05-28T13:59:19.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.649+0000 7fcfa8a00640 1 -- 172.21.15.42:0/486360844 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00a5d10 con 0x7fcfa40cddc0 2024-05-28T13:59:19.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.649+0000 7fcfac800640 1 -- 172.21.15.42:0/486360844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa40cddc0 msgr2=0x7fcfa409aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:19.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.649+0000 7fcfac800640 1 --2- 172.21.15.42:0/486360844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa40cddc0 0x7fcfa409aa50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009d90 tx=0x7fcfa001c970 comp rx=0 tx=0).stop 2024-05-28T13:59:19.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.649+0000 7fcfac800640 1 -- 172.21.15.42:0/486360844 shutdown_connections 2024-05-28T13:59:19.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.649+0000 7fcfac800640 1 --2- 172.21.15.42:0/486360844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa40cddc0 0x7fcfa409aa50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:19.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.649+0000 7fcfac800640 1 --2- 172.21.15.42:0/486360844 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa4162130 0x7fcfa40b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:19.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.649+0000 7fcfac800640 1 -- 172.21.15.42:0/486360844 >> 172.21.15.42:0/486360844 conn(0x7fcfa40b4050 msgr2=0x7fcfa40b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:19.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.649+0000 7fcfac800640 1 -- 172.21.15.42:0/486360844 shutdown_connections 2024-05-28T13:59:19.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.649+0000 7fcfac800640 1 -- 172.21.15.42:0/486360844 wait complete. 2024-05-28T13:59:19.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.649+0000 7fcfac800640 1 Processor -- start 2024-05-28T13:59:19.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.649+0000 7fcfac800640 1 -- start start 2024-05-28T13:59:19.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfac800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa40cddc0 0x7fcfa415cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:19.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfac800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa40d0870 0x7fcfa41549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:19.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfac800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa4162130 0x7fcfa415a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:19.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfac800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa409bb30 con 0x7fcfa40d0870 2024-05-28T13:59:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfac800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa409b9b0 con 0x7fcfa40cddc0 2024-05-28T13:59:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfac800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa409bcb0 con 0x7fcfa4162130 2024-05-28T13:59:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfa9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa40d0870 0x7fcfa41549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfa9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa40d0870 0x7fcfa41549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43968/0 (socket says 172.21.15.42:43968) 2024-05-28T13:59:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfaa800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa4162130 0x7fcfa415a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfa9400640 1 -- 172.21.15.42:0/3250831514 learned_addr learned my addr 172.21.15.42:0/3250831514 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfa9e00640 1 --2- 172.21.15.42:0/3250831514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa40cddc0 0x7fcfa415cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfaa800640 1 -- 172.21.15.42:0/3250831514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa40cddc0 msgr2=0x7fcfa415cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfaa800640 1 --2- 172.21.15.42:0/3250831514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa40cddc0 0x7fcfa415cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfaa800640 1 -- 172.21.15.42:0/3250831514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa40d0870 msgr2=0x7fcfa41549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfaa800640 1 --2- 172.21.15.42:0/3250831514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa40d0870 0x7fcfa41549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfaa800640 1 -- 172.21.15.42:0/3250831514 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa415d7d0 con 0x7fcfa4162130 2024-05-28T13:59:19.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfa9400640 1 --2- 172.21.15.42:0/3250831514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa40d0870 0x7fcfa41549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:59:19.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfaa800640 1 --2- 172.21.15.42:0/3250831514 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa4162130 0x7fcfa415a180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00ac000 tx=0x7fcfa0019710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:19.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcf9e000640 1 -- 172.21.15.42:0/3250831514 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00bd020 con 0x7fcfa4162130 2024-05-28T13:59:19.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcf9e000640 1 -- 172.21.15.42:0/3250831514 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa00b3040 con 0x7fcfa4162130 2024-05-28T13:59:19.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfac800640 1 -- 172.21.15.42:0/3250831514 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa415dab0 con 0x7fcfa4162130 2024-05-28T13:59:19.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcf9e000640 1 -- 172.21.15.42:0/3250831514 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00b2050 con 0x7fcfa4162130 2024-05-28T13:59:19.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfac800640 1 -- 172.21.15.42:0/3250831514 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa415e070 con 0x7fcfa4162130 2024-05-28T13:59:19.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcfac800640 1 -- 172.21.15.42:0/3250831514 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa40a0650 con 0x7fcfa4162130 2024-05-28T13:59:19.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.653+0000 7fcf9e000640 1 -- 172.21.15.42:0/3250831514 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcfa00c0070 con 0x7fcfa4162130 2024-05-28T13:59:19.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.657+0000 7fcf9e000640 1 --2- 172.21.15.42:0/3250831514 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf88068380 0x7fcf8806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:19.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.657+0000 7fcf9e000640 1 -- 172.21.15.42:0/3250831514 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 4943+0+0 (secure 0 0 0) 0x7fcfa0120a60 con 0x7fcfa4162130 2024-05-28T13:59:19.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.657+0000 7fcfa9e00640 1 --2- 172.21.15.42:0/3250831514 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf88068380 0x7fcf8806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:19.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.657+0000 7fcfa9e00640 1 --2- 172.21.15.42:0/3250831514 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf88068380 0x7fcf8806a840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009ac0 tx=0x7fcf98009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:19.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.661+0000 7fcf9e000640 1 -- 172.21.15.42:0/3250831514 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcfa0016560 con 0x7fcfa4162130 2024-05-28T13:59:19.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:19.853+0000 7fcfac800640 1 -- 172.21.15.42:0/3250831514 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fcfa40d1180 con 0x7fcfa4162130 2024-05-28T13:59:20.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.409+0000 7fcf9e000640 1 -- 172.21.15.42:0/3250831514 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) ==== 97+0+0 (secure 0 0 0) 0x7fcfa00f0cc0 con 0x7fcfa4162130 2024-05-28T13:59:20.396 INFO:teuthology.orchestra.run.smithi042.stderr: removed all blocklist entries 2024-05-28T13:59:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.413+0000 7fcfac800640 1 -- 172.21.15.42:0/3250831514 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf88068380 msgr2=0x7fcf8806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.413+0000 7fcfac800640 1 --2- 172.21.15.42:0/3250831514 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf88068380 0x7fcf8806a840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009ac0 tx=0x7fcf98009190 comp rx=0 tx=0).stop 2024-05-28T13:59:20.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.413+0000 7fcfac800640 1 -- 172.21.15.42:0/3250831514 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa4162130 msgr2=0x7fcfa415a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.413+0000 7fcfac800640 1 --2- 172.21.15.42:0/3250831514 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa4162130 0x7fcfa415a180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00ac000 tx=0x7fcfa0019710 comp rx=0 tx=0).stop 2024-05-28T13:59:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.413+0000 7fcfac800640 1 -- 172.21.15.42:0/3250831514 shutdown_connections 2024-05-28T13:59:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.413+0000 7fcfac800640 1 --2- 172.21.15.42:0/3250831514 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf88068380 0x7fcf8806a840 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.413+0000 7fcfac800640 1 --2- 172.21.15.42:0/3250831514 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa4162130 0x7fcfa415a180 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.413+0000 7fcfac800640 1 --2- 172.21.15.42:0/3250831514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa40d0870 0x7fcfa41549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.413+0000 7fcfac800640 1 --2- 172.21.15.42:0/3250831514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa40cddc0 0x7fcfa415cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.413+0000 7fcfac800640 1 -- 172.21.15.42:0/3250831514 >> 172.21.15.42:0/3250831514 conn(0x7fcfa40b4050 msgr2=0x7fcfa40b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.413+0000 7fcfac800640 1 -- 172.21.15.42:0/3250831514 shutdown_connections 2024-05-28T13:59:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.413+0000 7fcfac800640 1 -- 172.21.15.42:0/3250831514 wait complete. 2024-05-28T13:59:20.421 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-28T13:59:20.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.577+0000 7f6dbe600640 1 Processor -- start 2024-05-28T13:59:20.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.577+0000 7f6dbe600640 1 -- start start 2024-05-28T13:59:20.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.577+0000 7f6dbe600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6db80b42e0 0x7f6db815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:20.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.577+0000 7f6dbe600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6db8067810 0x7f6db806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:20.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.577+0000 7f6dbe600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6db806eaa0 con 0x7f6db80d0420 2024-05-28T13:59:20.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.577+0000 7f6dbe600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6db806ed80 con 0x7f6db8067810 2024-05-28T13:59:20.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.577+0000 7f6dbe600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6db806f060 con 0x7f6db80b42e0 2024-05-28T13:59:20.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.581+0000 7f6db7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6db8067810 0x7f6db806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:20.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.581+0000 7f6db7400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f6db80d0420 0x7f6db80b3bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:43808/0 (socket says 172.21.15.42:43808) 2024-05-28T13:59:20.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.581+0000 7f6db7400640 1 -- 172.21.15.42:0/3120585926 learned_addr learned my addr 172.21.15.42:0/3120585926 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:20.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.581+0000 7f6daea00640 1 --2- 172.21.15.42:0/3120585926 >> v2:172.21.15.63:3300/0 conn(0x7f6db80b42e0 0x7f6db815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:20.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.581+0000 7f6db7e00640 1 -- 172.21.15.42:0/3120585926 >> v2:172.21.15.63:3300/0 conn(0x7f6db80b42e0 msgr2=0x7f6db815ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:20.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.581+0000 7f6db7e00640 1 --2- 172.21.15.42:0/3120585926 >> v2:172.21.15.63:3300/0 conn(0x7f6db80b42e0 0x7f6db815ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.581+0000 7f6db7e00640 1 -- 172.21.15.42:0/3120585926 >> v1:172.21.15.42:6789/0 conn(0x7f6db80d0420 legacy=0x7f6db80b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:20.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.581+0000 7f6db7e00640 1 -- 172.21.15.42:0/3120585926 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db806f340 con 0x7f6db8067810 2024-05-28T13:59:20.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.581+0000 7f6db6a00640 1 -- 172.21.15.42:0/3120585926 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2668787786 0 0) 0x7f6db806eaa0 con 0x7f6db80d0420 2024-05-28T13:59:20.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.581+0000 7f6daea00640 1 --2- 172.21.15.42:0/3120585926 >> v2:172.21.15.63:3300/0 conn(0x7f6db80b42e0 0x7f6db815ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:20.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.581+0000 7f6db7e00640 1 --2- 172.21.15.42:0/3120585926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6db8067810 0x7f6db806bc90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6db80b3850 tx=0x7f6db001d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ecc5e2f3b736f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:20.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.581+0000 7f6db6a00640 1 -- 172.21.15.42:0/3120585926 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db00b2070 con 0x7f6db8067810 2024-05-28T13:59:20.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.581+0000 7f6db6a00640 1 -- 172.21.15.42:0/3120585926 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db00a5870 con 0x7f6db8067810 2024-05-28T13:59:20.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.581+0000 7f6db6a00640 1 -- 172.21.15.42:0/3120585926 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db00a5c40 con 0x7f6db8067810 2024-05-28T13:59:20.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 -- 172.21.15.42:0/3120585926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6db8067810 msgr2=0x7f6db806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:20.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 --2- 172.21.15.42:0/3120585926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6db8067810 0x7f6db806bc90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6db80b3850 tx=0x7f6db001d300 comp rx=0 tx=0).stop 2024-05-28T13:59:20.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 -- 172.21.15.42:0/3120585926 shutdown_connections 2024-05-28T13:59:20.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 --2- 172.21.15.42:0/3120585926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6db8067810 0x7f6db806bc90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 --2- 172.21.15.42:0/3120585926 >> v2:172.21.15.63:3300/0 conn(0x7f6db80b42e0 0x7f6db815ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 -- 172.21.15.42:0/3120585926 >> 172.21.15.42:0/3120585926 conn(0x7f6db80d0b40 msgr2=0x7f6db80d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:20.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 -- 172.21.15.42:0/3120585926 shutdown_connections 2024-05-28T13:59:20.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 -- 172.21.15.42:0/3120585926 wait complete. 2024-05-28T13:59:20.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 Processor -- start 2024-05-28T13:59:20.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 -- start start 2024-05-28T13:59:20.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db8067810 0x7f6db8149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:20.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6db80b42e0 0x7f6db80aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:20.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6db80d0420 0x7f6db8152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:20.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6db806cb60 con 0x7f6db80d0420 2024-05-28T13:59:20.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6db806c9e0 con 0x7f6db8067810 2024-05-28T13:59:20.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6dbe600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6db806cce0 con 0x7f6db80b42e0 2024-05-28T13:59:20.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6db7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6db80d0420 0x7f6db8152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:20.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6db7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6db80d0420 0x7f6db8152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43996/0 (socket says 172.21.15.42:43996) 2024-05-28T13:59:20.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6db7e00640 1 -- 172.21.15.42:0/3786612532 learned_addr learned my addr 172.21.15.42:0/3786612532 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:20.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6daea00640 1 --2- 172.21.15.42:0/3786612532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6db80b42e0 0x7f6db80aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:20.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6db7400640 1 --2- 172.21.15.42:0/3786612532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db8067810 0x7f6db8149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:20.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6db7e00640 1 -- 172.21.15.42:0/3786612532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6db80b42e0 msgr2=0x7f6db80aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:20.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6db7e00640 1 --2- 172.21.15.42:0/3786612532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6db80b42e0 0x7f6db80aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6db7e00640 1 -- 172.21.15.42:0/3786612532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db8067810 msgr2=0x7f6db8149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:20.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6db7e00640 1 --2- 172.21.15.42:0/3786612532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db8067810 0x7f6db8149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.585+0000 7f6db7e00640 1 -- 172.21.15.42:0/3786612532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db8151e90 con 0x7f6db80d0420 2024-05-28T13:59:20.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.589+0000 7f6db7e00640 1 --2- 172.21.15.42:0/3786612532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6db80d0420 0x7f6db8152170 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6db00b0040 tx=0x7f6db0019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:20.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.589+0000 7f6dafe00640 1 -- 172.21.15.42:0/3786612532 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db00bc030 con 0x7f6db80d0420 2024-05-28T13:59:20.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.589+0000 7f6dafe00640 1 -- 172.21.15.42:0/3786612532 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db00ad440 con 0x7f6db80d0420 2024-05-28T13:59:20.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.589+0000 7f6dafe00640 1 -- 172.21.15.42:0/3786612532 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db00b2040 con 0x7f6db80d0420 2024-05-28T13:59:20.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.589+0000 7f6dbe600640 1 -- 172.21.15.42:0/3786612532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db81515f0 con 0x7f6db80d0420 2024-05-28T13:59:20.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.589+0000 7f6dbe600640 1 -- 172.21.15.42:0/3786612532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db8151bb0 con 0x7f6db80d0420 2024-05-28T13:59:20.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.589+0000 7f6dbe600640 1 -- 172.21.15.42:0/3786612532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db8068a90 con 0x7f6db80d0420 2024-05-28T13:59:20.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.589+0000 7f6dafe00640 1 -- 172.21.15.42:0/3786612532 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6db00bf070 con 0x7f6db80d0420 2024-05-28T13:59:20.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.589+0000 7f6dafe00640 1 --2- 172.21.15.42:0/3786612532 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d94068380 0x7f6d9406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:20.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.589+0000 7f6dafe00640 1 -- 172.21.15.42:0/3786612532 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f6db0120f80 con 0x7f6db80d0420 2024-05-28T13:59:20.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.589+0000 7f6db7400640 1 --2- 172.21.15.42:0/3786612532 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d94068380 0x7f6d9406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:20.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.593+0000 7f6db7400640 1 --2- 172.21.15.42:0/3786612532 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d94068380 0x7f6d9406a840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6da0000f70 tx=0x7f6da0007c10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:20.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.593+0000 7f6dafe00640 1 -- 172.21.15.42:0/3786612532 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6db0016520 con 0x7f6db80d0420 2024-05-28T13:59:20.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.817+0000 7f6dbe600640 1 -- 172.21.15.42:0/3786612532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6db815e750 con 0x7f6db80d0420 2024-05-28T13:59:20.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.817+0000 7f6dafe00640 1 -- 172.21.15.42:0/3786612532 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v7) ==== 80+0+6645 (secure 0 0 0) 0x7f6db00f1800 con 0x7f6db80d0420 2024-05-28T13:59:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:20.802 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T13:59:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_dump": [ 2024-05-28T13:59:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.a", 2024-05-28T13:59:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmE+3RLRAADuTKGfKeVafGeLpgr9WKUg==", 2024-05-28T13:59:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T13:59:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T13:59:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T13:59:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.b", 2024-05-28T13:59:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVm1rmVOhAAV46KzDJnRfDZYIZanS0g2g==", 2024-05-28T13:59:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T13:59:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T13:59:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T13:59:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.c", 2024-05-28T13:59:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVme4aYNRAAVxJTlugnxtFTexUclF/IwA==", 2024-05-28T13:59:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T13:59:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T13:59:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T13:59:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.d", 2024-05-28T13:59:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmeAdBBhAAr4IVFQpjQXFVlnFTHnPV9w==", 2024-05-28T13:59:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T13:59:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T13:59:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T13:59:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.0", 2024-05-28T13:59:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDB4lVmGMDBJhAA3jIyR7XKLGXEgimIGNsw6A==", 2024-05-28T13:59:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T13:59:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T13:59:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.1", 2024-05-28T13:59:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDE4lVmxMisLxAAcIz7GPlkZwvW5ig+DbAVGg==", 2024-05-28T13:59:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T13:59:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T13:59:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.2", 2024-05-28T13:59:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDH4lVmSSk4OhAAmLlLysM4Im/Kew4Xbq2/OA==", 2024-05-28T13:59:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T13:59:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T13:59:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.3", 2024-05-28T13:59:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDL4lVmvzrYBhAAiAAOQXZBuTsMVYa1vE5rkg==", 2024-05-28T13:59:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T13:59:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T13:59:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.4", 2024-05-28T13:59:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDQ4lVmrqnTOhAA2pIfgBT9LJDIhp+GgFy8Gg==", 2024-05-28T13:59:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T13:59:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T13:59:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.5", 2024-05-28T13:59:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDU4lVmYpsQCxAAoTK2aMBPrHKvsUnNOTj/Dw==", 2024-05-28T13:59:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T13:59:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T13:59:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.6", 2024-05-28T13:59:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDX4lVm6TCAFxAAs3rkDyKcLhWrn/EkIo0bVA==", 2024-05-28T13:59:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T13:59:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T13:59:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.7", 2024-05-28T13:59:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDa4lVmr5NwIRAA7Lb70VYV82dr0Frn8CCUXg==", 2024-05-28T13:59:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T13:59:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T13:59:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.0", 2024-05-28T13:59:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw==", 2024-05-28T13:59:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T13:59:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T13:59:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T13:59:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T13:59:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.1", 2024-05-28T13:59:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmcfkLFBAADNyODKTtYAlFWquZ/LeeqQ==", 2024-05-28T13:59:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T13:59:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T13:59:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T13:59:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T13:59:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.2", 2024-05-28T13:59:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmoxx+EBAA8mLNak5AwRmWQIgkYtMOrg==", 2024-05-28T13:59:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T13:59:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T13:59:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T13:59:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T13:59:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.3", 2024-05-28T13:59:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmxSUsGhAAbgr1eOh/sMfrEONKKHFo0g==", 2024-05-28T13:59:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T13:59:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T13:59:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T13:59:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T13:59:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.admin", 2024-05-28T13:59:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmOSBPERAAt4x2DBal4PnqxqeWQutLDQ==", 2024-05-28T13:59:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T13:59:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T13:59:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T13:59:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.bootstrap-mds", 2024-05-28T13:59:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDl4lVm25zPJxAAEFuCeC+ntV3jRJQD5eDxvw==", 2024-05-28T13:59:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile bootstrap-mds" 2024-05-28T13:59:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.bootstrap-mgr", 2024-05-28T13:59:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDl4lVm+ajPJxAA9ogkbcEqggZNSqKD1b5Nqw==", 2024-05-28T13:59:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-28T13:59:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.bootstrap-osd", 2024-05-28T13:59:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDl4lVmuLHPJxAAkipIXGVNBZg+ncRaTQwyrg==", 2024-05-28T13:59:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile bootstrap-osd" 2024-05-28T13:59:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.bootstrap-rbd", 2024-05-28T13:59:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDl4lVmcbrPJxAAbiT195lURmcFmET38zrNQg==", 2024-05-28T13:59:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-28T13:59:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-28T13:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDl4lVmdMPPJxAAoyNpQX+KLJwPPt+dUw8Lsw==", 2024-05-28T13:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-28T13:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.bootstrap-rgw", 2024-05-28T13:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDl4lVmwczPJxAAEDwui+k8dpcCetl+kSpRSw==", 2024-05-28T13:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-28T13:59:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.x", 2024-05-28T13:59:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmNO5tKRAA70hPLwoE83keovlq2rolxQ==", 2024-05-28T13:59:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T13:59:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T13:59:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.y", 2024-05-28T13:59:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmxTVyJRAATMLAAqKne20YvV537uXZ6g==", 2024-05-28T13:59:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T13:59:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T13:59:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T13:59:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T13:59:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-28T13:59:20.825 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T13:59:20.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.817+0000 7f6dbe600640 1 -- 172.21.15.42:0/3786612532 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d94068380 msgr2=0x7f6d9406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:20.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.817+0000 7f6dbe600640 1 --2- 172.21.15.42:0/3786612532 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d94068380 0x7f6d9406a840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6da0000f70 tx=0x7f6da0007c10 comp rx=0 tx=0).stop 2024-05-28T13:59:20.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.817+0000 7f6dbe600640 1 -- 172.21.15.42:0/3786612532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6db80d0420 msgr2=0x7f6db8152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.817+0000 7f6dbe600640 1 --2- 172.21.15.42:0/3786612532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6db80d0420 0x7f6db8152170 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6db00b0040 tx=0x7f6db0019640 comp rx=0 tx=0).stop 2024-05-28T13:59:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.817+0000 7f6dbe600640 1 -- 172.21.15.42:0/3786612532 shutdown_connections 2024-05-28T13:59:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.817+0000 7f6dbe600640 1 --2- 172.21.15.42:0/3786612532 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d94068380 0x7f6d9406a840 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.817+0000 7f6dbe600640 1 --2- 172.21.15.42:0/3786612532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6db80d0420 0x7f6db8152170 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.817+0000 7f6dbe600640 1 --2- 172.21.15.42:0/3786612532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6db80b42e0 0x7f6db80aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.817+0000 7f6dbe600640 1 --2- 172.21.15.42:0/3786612532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db8067810 0x7f6db8149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.817+0000 7f6dbe600640 1 -- 172.21.15.42:0/3786612532 >> 172.21.15.42:0/3786612532 conn(0x7f6db80d0b40 msgr2=0x7f6db815ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.817+0000 7f6dbe600640 1 -- 172.21.15.42:0/3786612532 shutdown_connections 2024-05-28T13:59:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.817+0000 7f6dbe600640 1 -- 172.21.15.42:0/3786612532 wait complete. 2024-05-28T13:59:20.828 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-28T13:59:20.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4f2c00640 1 Processor -- start 2024-05-28T13:59:20.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4f2c00640 1 -- start start 2024-05-28T13:59:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4f2c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4ec09aa50 0x7fc4ec09ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4f2c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc4ec0d15c0 0x7fc4ec0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4f2c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4ec09b370 con 0x7fc4ec0b4bc0 2024-05-28T13:59:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4f2c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc4ec0ccee0 con 0x7fc4ec0d15c0 2024-05-28T13:59:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4f2c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc4ec1622a0 con 0x7fc4ec09aa50 2024-05-28T13:59:20.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4f0c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc4ec0d15c0 0x7fc4ec0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:20.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4ebe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4ec09aa50 0x7fc4ec09ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:20.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4f0c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc4ec0d15c0 0x7fc4ec0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:35038/0 (socket says 172.21.15.42:35038) 2024-05-28T13:59:20.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4ebe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4ec09aa50 0x7fc4ec09ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:49856/0 (socket says 172.21.15.42:49856) 2024-05-28T13:59:20.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4f0c00640 1 -- 172.21.15.42:0/3964851214 learned_addr learned my addr 172.21.15.42:0/3964851214 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:20.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4e3400640 1 --1- 172.21.15.42:0/3964851214 >> v1:172.21.15.63:6789/0 conn(0x7fc4ec0b4bc0 0x7fc4ec0cd370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:40296/0 (socket says 172.21.15.42:40296) 2024-05-28T13:59:20.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4ebe00640 1 -- 172.21.15.42:0/3964851214 >> v2:172.21.15.63:3300/0 conn(0x7fc4ec0d15c0 msgr2=0x7fc4ec0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:20.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4ebe00640 1 --2- 172.21.15.42:0/3964851214 >> v2:172.21.15.63:3300/0 conn(0x7fc4ec0d15c0 0x7fc4ec0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4ebe00640 1 -- 172.21.15.42:0/3964851214 >> v1:172.21.15.63:6789/0 conn(0x7fc4ec0b4bc0 legacy=0x7fc4ec0cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:20.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4ebe00640 1 -- 172.21.15.42:0/3964851214 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec09b620 con 0x7fc4ec09aa50 2024-05-28T13:59:20.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.985+0000 7fc4f0c00640 1 --2- 172.21.15.42:0/3964851214 >> v2:172.21.15.63:3300/0 conn(0x7fc4ec0d15c0 0x7fc4ec0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:59:20.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.989+0000 7fc4ebe00640 1 --2- 172.21.15.42:0/3964851214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4ec09aa50 0x7fc4ec09ae30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0097c0 tx=0x7fc4dc01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34902d8e78fdcfe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:20.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.989+0000 7fc4eb400640 1 -- 172.21.15.42:0/3964851214 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0a7070 con 0x7fc4ec09aa50 2024-05-28T13:59:20.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.989+0000 7fc4eb400640 1 -- 172.21.15.42:0/3964851214 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4dc0b0070 con 0x7fc4ec09aa50 2024-05-28T13:59:20.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.989+0000 7fc4eb400640 1 -- 172.21.15.42:0/3964851214 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0ab890 con 0x7fc4ec09aa50 2024-05-28T13:59:20.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.989+0000 7fc4f2c00640 1 -- 172.21.15.42:0/3964851214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4ec09aa50 msgr2=0x7fc4ec09ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:20.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.989+0000 7fc4f2c00640 1 --2- 172.21.15.42:0/3964851214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4ec09aa50 0x7fc4ec09ae30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0097c0 tx=0x7fc4dc01c470 comp rx=0 tx=0).stop 2024-05-28T13:59:20.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.989+0000 7fc4f2c00640 1 -- 172.21.15.42:0/3964851214 shutdown_connections 2024-05-28T13:59:20.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.989+0000 7fc4f2c00640 1 --2- 172.21.15.42:0/3964851214 >> v2:172.21.15.63:3300/0 conn(0x7fc4ec0d15c0 0x7fc4ec0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.989+0000 7fc4f2c00640 1 --2- 172.21.15.42:0/3964851214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4ec09aa50 0x7fc4ec09ae30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.989+0000 7fc4f2c00640 1 -- 172.21.15.42:0/3964851214 >> 172.21.15.42:0/3964851214 conn(0x7fc4ec0b4010 msgr2=0x7fc4ec0b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:20.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.989+0000 7fc4f2c00640 1 -- 172.21.15.42:0/3964851214 shutdown_connections 2024-05-28T13:59:20.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.989+0000 7fc4f2c00640 1 -- 172.21.15.42:0/3964851214 wait complete. 2024-05-28T13:59:20.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f2c00640 1 Processor -- start 2024-05-28T13:59:20.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f2c00640 1 -- start start 2024-05-28T13:59:20.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f2c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4ec09aa50 0x7fc4ec153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:20.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f2c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4ec0b4bc0 0x7fc4ec15d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:20.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f2c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4ec0d15c0 0x7fc4ec150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:20.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f2c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec0ce410 con 0x7fc4ec0d15c0 2024-05-28T13:59:20.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f2c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec0ce290 con 0x7fc4ec09aa50 2024-05-28T13:59:20.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f2c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc4ec0ce590 con 0x7fc4ec0b4bc0 2024-05-28T13:59:20.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4ec0d15c0 0x7fc4ec150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:20.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4ec0d15c0 0x7fc4ec150900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43998/0 (socket says 172.21.15.42:43998) 2024-05-28T13:59:20.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4ebe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4ec09aa50 0x7fc4ec153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:20.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f0c00640 1 -- 172.21.15.42:0/1451180980 learned_addr learned my addr 172.21.15.42:0/1451180980 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:20.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4e3400640 1 --2- 172.21.15.42:0/1451180980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4ec0b4bc0 0x7fc4ec15d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:20.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f0c00640 1 -- 172.21.15.42:0/1451180980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4ec0b4bc0 msgr2=0x7fc4ec15d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:20.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f0c00640 1 --2- 172.21.15.42:0/1451180980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4ec0b4bc0 0x7fc4ec15d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f0c00640 1 -- 172.21.15.42:0/1451180980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4ec09aa50 msgr2=0x7fc4ec153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:20.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f0c00640 1 --2- 172.21.15.42:0/1451180980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4ec09aa50 0x7fc4ec153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:20.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f0c00640 1 -- 172.21.15.42:0/1451180980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec15d860 con 0x7fc4ec0d15c0 2024-05-28T13:59:20.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f0c00640 1 --2- 172.21.15.42:0/1451180980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4ec0d15c0 0x7fc4ec150900 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400efc0 tx=0x7fc4e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:20.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4e8c00640 1 -- 172.21.15.42:0/1451180980 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e400cbe0 con 0x7fc4ec0d15c0 2024-05-28T13:59:20.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4e8c00640 1 -- 172.21.15.42:0/1451180980 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e400cd80 con 0x7fc4ec0d15c0 2024-05-28T13:59:20.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4e8c00640 1 -- 172.21.15.42:0/1451180980 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4003bc0 con 0x7fc4ec0d15c0 2024-05-28T13:59:20.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f2c00640 1 -- 172.21.15.42:0/1451180980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ec15db40 con 0x7fc4ec0d15c0 2024-05-28T13:59:20.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.993+0000 7fc4f2c00640 1 -- 172.21.15.42:0/1451180980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ec15e100 con 0x7fc4ec0d15c0 2024-05-28T13:59:20.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.997+0000 7fc4f2c00640 1 -- 172.21.15.42:0/1451180980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4ec0a0630 con 0x7fc4ec0d15c0 2024-05-28T13:59:20.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.997+0000 7fc4e8c00640 1 -- 172.21.15.42:0/1451180980 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc4e401b030 con 0x7fc4ec0d15c0 2024-05-28T13:59:20.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.997+0000 7fc4e8c00640 1 --2- 172.21.15.42:0/1451180980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc4c8068380 0x7fc4c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:20.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.997+0000 7fc4e8c00640 1 -- 172.21.15.42:0/1451180980 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fc4e40869d0 con 0x7fc4ec0d15c0 2024-05-28T13:59:20.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.997+0000 7fc4ebe00640 1 --2- 172.21.15.42:0/1451180980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc4c8068380 0x7fc4c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:20.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:20.997+0000 7fc4ebe00640 1 --2- 172.21.15.42:0/1451180980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc4c8068380 0x7fc4c806a840 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0af000 tx=0x7fc4dc0a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:20.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.001+0000 7fc4e8c00640 1 -- 172.21.15.42:0/1451180980 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc4e4022020 con 0x7fc4ec0d15c0 2024-05-28T13:59:21.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.225+0000 7fc4f2c00640 1 -- 172.21.15.42:0/1451180980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fc4ec09aec0 con 0x7fc4ec0d15c0 2024-05-28T13:59:21.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.229+0000 7fc4e8c00640 1 -- 172.21.15.42:0/1451180980 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fc4e4010040 con 0x7fc4ec0d15c0 2024-05-28T13:59:21.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.229+0000 7fc4f2c00640 1 -- 172.21.15.42:0/1451180980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc4c8068380 msgr2=0x7fc4c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:21.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.229+0000 7fc4f2c00640 1 --2- 172.21.15.42:0/1451180980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc4c8068380 0x7fc4c806a840 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0af000 tx=0x7fc4dc0a7000 comp rx=0 tx=0).stop 2024-05-28T13:59:21.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.229+0000 7fc4f2c00640 1 -- 172.21.15.42:0/1451180980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4ec0d15c0 msgr2=0x7fc4ec150900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:21.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.229+0000 7fc4f2c00640 1 --2- 172.21.15.42:0/1451180980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4ec0d15c0 0x7fc4ec150900 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400efc0 tx=0x7fc4e400c490 comp rx=0 tx=0).stop 2024-05-28T13:59:21.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.229+0000 7fc4f2c00640 1 -- 172.21.15.42:0/1451180980 shutdown_connections 2024-05-28T13:59:21.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.229+0000 7fc4f2c00640 1 --2- 172.21.15.42:0/1451180980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc4c8068380 0x7fc4c806a840 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.229+0000 7fc4f2c00640 1 --2- 172.21.15.42:0/1451180980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4ec0d15c0 0x7fc4ec150900 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.229+0000 7fc4f2c00640 1 --2- 172.21.15.42:0/1451180980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4ec0b4bc0 0x7fc4ec15d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.229+0000 7fc4f2c00640 1 --2- 172.21.15.42:0/1451180980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4ec09aa50 0x7fc4ec153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.229+0000 7fc4f2c00640 1 -- 172.21.15.42:0/1451180980 >> 172.21.15.42:0/1451180980 conn(0x7fc4ec0b4010 msgr2=0x7fc4ec158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:21.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.229+0000 7fc4f2c00640 1 -- 172.21.15.42:0/1451180980 shutdown_connections 2024-05-28T13:59:21.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.229+0000 7fc4f2c00640 1 -- 172.21.15.42:0/1451180980 wait complete. 2024-05-28T13:59:21.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-28T13:59:21.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.389+0000 7f8a10000640 1 Processor -- start 2024-05-28T13:59:21.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.389+0000 7f8a10000640 1 -- start start 2024-05-28T13:59:21.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.393+0000 7f8a10000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8a0815e820 0x7f8a080b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:21.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.393+0000 7f8a10000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8a080ae6a0 0x7f8a0806b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:21.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.393+0000 7f8a10000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a080b6590 con 0x7f8a080ae240 2024-05-28T13:59:21.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.393+0000 7f8a10000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8a0806c870 con 0x7f8a0815e820 2024-05-28T13:59:21.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.393+0000 7f8a10000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8a080b5d30 con 0x7f8a080ae6a0 2024-05-28T13:59:21.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.393+0000 7f8a0d600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8a0815e820 0x7f8a080b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:21.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.393+0000 7f8a0d600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8a0815e820 0x7f8a080b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44000/0 (socket says 172.21.15.42:44000) 2024-05-28T13:59:21.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.393+0000 7f8a0d600640 1 -- 172.21.15.42:0/3008369045 learned_addr learned my addr 172.21.15.42:0/3008369045 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:21.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.393+0000 7f8a0e000640 1 --2- 172.21.15.42:0/3008369045 >> v2:172.21.15.63:3300/0 conn(0x7f8a080ae6a0 0x7f8a0806b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:21.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.393+0000 7f8a0d600640 1 -- 172.21.15.42:0/3008369045 >> v2:172.21.15.63:3300/0 conn(0x7f8a080ae6a0 msgr2=0x7f8a0806b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:21.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.393+0000 7f8a0d600640 1 --2- 172.21.15.42:0/3008369045 >> v2:172.21.15.63:3300/0 conn(0x7f8a080ae6a0 0x7f8a0806b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.393+0000 7f8a0d600640 1 -- 172.21.15.42:0/3008369045 >> v1:172.21.15.63:6789/0 conn(0x7f8a080ae240 legacy=0x7f8a080b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:21.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.393+0000 7f8a0d600640 1 -- 172.21.15.42:0/3008369045 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0815ec00 con 0x7f8a0815e820 2024-05-28T13:59:21.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.397+0000 7f8a0d600640 1 --2- 172.21.15.42:0/3008369045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a0815e820 0x7f8a080b4260 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8a080b0670 tx=0x7f89fc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e52d49bc6f5e35 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:21.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.397+0000 7f8a07e00640 1 -- 172.21.15.42:0/3008369045 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0b0070 con 0x7f8a0815e820 2024-05-28T13:59:21.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.397+0000 7f8a07e00640 1 -- 172.21.15.42:0/3008369045 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89fc0ab600 con 0x7f8a0815e820 2024-05-28T13:59:21.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.397+0000 7f8a07e00640 1 -- 172.21.15.42:0/3008369045 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0ab9d0 con 0x7f8a0815e820 2024-05-28T13:59:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.397+0000 7f8a10000640 1 -- 172.21.15.42:0/3008369045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a0815e820 msgr2=0x7f8a080b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.397+0000 7f8a10000640 1 --2- 172.21.15.42:0/3008369045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a0815e820 0x7f8a080b4260 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8a080b0670 tx=0x7f89fc01c4e0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.397+0000 7f8a10000640 1 -- 172.21.15.42:0/3008369045 shutdown_connections 2024-05-28T13:59:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.397+0000 7f8a10000640 1 --2- 172.21.15.42:0/3008369045 >> v2:172.21.15.63:3300/0 conn(0x7f8a080ae6a0 0x7f8a0806b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.397+0000 7f8a10000640 1 --2- 172.21.15.42:0/3008369045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a0815e820 0x7f8a080b4260 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.397+0000 7f8a10000640 1 -- 172.21.15.42:0/3008369045 >> 172.21.15.42:0/3008369045 conn(0x7f8a080b6780 msgr2=0x7f8a080b6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:21.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.397+0000 7f8a10000640 1 -- 172.21.15.42:0/3008369045 shutdown_connections 2024-05-28T13:59:21.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.397+0000 7f8a10000640 1 -- 172.21.15.42:0/3008369045 wait complete. 2024-05-28T13:59:21.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.397+0000 7f8a10000640 1 Processor -- start 2024-05-28T13:59:21.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.397+0000 7f8a10000640 1 -- start start 2024-05-28T13:59:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a10000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a080ae240 0x7f8a08157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a10000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a080ae6a0 0x7f8a08156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a10000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a0815e820 0x7f8a0806f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a10000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a080b47a0 con 0x7f8a080ae6a0 2024-05-28T13:59:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a10000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0806c340 con 0x7f8a080ae240 2024-05-28T13:59:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a10000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0815edd0 con 0x7f8a0815e820 2024-05-28T13:59:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a0cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a080ae6a0 0x7f8a08156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a0cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a080ae6a0 0x7f8a08156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44010/0 (socket says 172.21.15.42:44010) 2024-05-28T13:59:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a0cc00640 1 -- 172.21.15.42:0/2710067342 learned_addr learned my addr 172.21.15.42:0/2710067342 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a0d600640 1 --2- 172.21.15.42:0/2710067342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a080ae240 0x7f8a08157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:21.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a0e000640 1 --2- 172.21.15.42:0/2710067342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a0815e820 0x7f8a0806f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a0cc00640 1 -- 172.21.15.42:0/2710067342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a0815e820 msgr2=0x7f8a0806f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a0cc00640 1 --2- 172.21.15.42:0/2710067342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a0815e820 0x7f8a0806f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a0cc00640 1 -- 172.21.15.42:0/2710067342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a080ae240 msgr2=0x7f8a08157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a0cc00640 1 --2- 172.21.15.42:0/2710067342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a080ae240 0x7f8a08157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a0cc00640 1 -- 172.21.15.42:0/2710067342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0809ef20 con 0x7f8a080ae6a0 2024-05-28T13:59:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a0cc00640 1 --2- 172.21.15.42:0/2710067342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a080ae6a0 0x7f8a08156480 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f89f400c450 tx=0x7f89f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a05600640 1 -- 172.21.15.42:0/2710067342 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4012830 con 0x7f8a080ae6a0 2024-05-28T13:59:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a05600640 1 -- 172.21.15.42:0/2710067342 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f4017070 con 0x7f8a080ae6a0 2024-05-28T13:59:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a10000640 1 -- 172.21.15.42:0/2710067342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0815f020 con 0x7f8a080ae6a0 2024-05-28T13:59:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a05600640 1 -- 172.21.15.42:0/2710067342 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4012c30 con 0x7f8a080ae6a0 2024-05-28T13:59:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a10000640 1 -- 172.21.15.42:0/2710067342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a081614a0 con 0x7f8a080ae6a0 2024-05-28T13:59:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.401+0000 7f8a10000640 1 -- 172.21.15.42:0/2710067342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0809f5c0 con 0x7f8a080ae6a0 2024-05-28T13:59:21.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.405+0000 7f8a05600640 1 -- 172.21.15.42:0/2710067342 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f89f401c460 con 0x7f8a080ae6a0 2024-05-28T13:59:21.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.405+0000 7f8a05600640 1 --2- 172.21.15.42:0/2710067342 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89e8068380 0x7f89e806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:21.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.405+0000 7f8a05600640 1 -- 172.21.15.42:0/2710067342 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f89f4086e00 con 0x7f8a080ae6a0 2024-05-28T13:59:21.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.405+0000 7f8a0d600640 1 --2- 172.21.15.42:0/2710067342 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89e8068380 0x7f89e806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:21.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.405+0000 7f8a0d600640 1 --2- 172.21.15.42:0/2710067342 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89e8068380 0x7f89e806a840 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0ad000 tx=0x7f89fc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.405+0000 7f8a05600640 1 -- 172.21.15.42:0/2710067342 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f89f4057130 con 0x7f8a080ae6a0 2024-05-28T13:59:21.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.633+0000 7f8a10000640 1 -- 172.21.15.42:0/2710067342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f8a0804fdd0 con 0x7f8a080ae6a0 2024-05-28T13:59:21.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.633+0000 7f8a05600640 1 -- 172.21.15.42:0/2710067342 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f89f405b430 con 0x7f8a080ae6a0 2024-05-28T13:59:21.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.637+0000 7f8a10000640 1 -- 172.21.15.42:0/2710067342 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89e8068380 msgr2=0x7f89e806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:21.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.637+0000 7f8a10000640 1 --2- 172.21.15.42:0/2710067342 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89e8068380 0x7f89e806a840 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0ad000 tx=0x7f89fc008040 comp rx=0 tx=0).stop 2024-05-28T13:59:21.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.637+0000 7f8a10000640 1 -- 172.21.15.42:0/2710067342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a080ae6a0 msgr2=0x7f8a08156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:21.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.637+0000 7f8a10000640 1 --2- 172.21.15.42:0/2710067342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a080ae6a0 0x7f8a08156480 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f89f400c450 tx=0x7f89f400c910 comp rx=0 tx=0).stop 2024-05-28T13:59:21.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.637+0000 7f8a10000640 1 -- 172.21.15.42:0/2710067342 shutdown_connections 2024-05-28T13:59:21.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.637+0000 7f8a10000640 1 --2- 172.21.15.42:0/2710067342 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89e8068380 0x7f89e806a840 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.637+0000 7f8a10000640 1 --2- 172.21.15.42:0/2710067342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a0815e820 0x7f8a0806f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.637+0000 7f8a10000640 1 --2- 172.21.15.42:0/2710067342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a080ae6a0 0x7f8a08156480 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.637+0000 7f8a10000640 1 --2- 172.21.15.42:0/2710067342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a080ae240 0x7f8a08157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.637+0000 7f8a10000640 1 -- 172.21.15.42:0/2710067342 >> 172.21.15.42:0/2710067342 conn(0x7f8a080b6780 msgr2=0x7f8a080b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:21.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.637+0000 7f8a10000640 1 -- 172.21.15.42:0/2710067342 shutdown_connections 2024-05-28T13:59:21.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.637+0000 7f8a10000640 1 -- 172.21.15.42:0/2710067342 wait complete. 2024-05-28T13:59:21.642 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-28T13:59:21.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc86aa00640 1 Processor -- start 2024-05-28T13:59:21.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc86aa00640 1 -- start start 2024-05-28T13:59:21.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc86aa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc86406d110 0x7fc86406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:21.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc86aa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc86406f3f0 0x7fc864160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:21.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc86aa00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc86406fc20 con 0x7fc864155a00 2024-05-28T13:59:21.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc86aa00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc86415ff90 con 0x7fc86406d110 2024-05-28T13:59:21.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc86aa00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc864160270 con 0x7fc86406f3f0 2024-05-28T13:59:21.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc863e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc864155a00 0x7fc86406ca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:43816/0 (socket says 172.21.15.42:43816) 2024-05-28T13:59:21.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc863e00640 1 -- 172.21.15.42:0/1332574200 learned_addr learned my addr 172.21.15.42:0/1332574200 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:21.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc85b400640 1 --2- 172.21.15.42:0/1332574200 >> v2:172.21.15.42:3300/0 conn(0x7fc86406d110 0x7fc86406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:21.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc868a00640 1 --2- 172.21.15.42:0/1332574200 >> v2:172.21.15.63:3300/0 conn(0x7fc86406f3f0 0x7fc864160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:21.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc863400640 1 -- 172.21.15.42:0/1332574200 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 58856889 0 0) 0x7fc86406fc20 con 0x7fc864155a00 2024-05-28T13:59:21.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc863400640 1 -- 172.21.15.42:0/1332574200 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc848003530 con 0x7fc864155a00 2024-05-28T13:59:21.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc85b400640 1 -- 172.21.15.42:0/1332574200 >> v2:172.21.15.63:3300/0 conn(0x7fc86406f3f0 msgr2=0x7fc864160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:21.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc85b400640 1 --2- 172.21.15.42:0/1332574200 >> v2:172.21.15.63:3300/0 conn(0x7fc86406f3f0 0x7fc864160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc85b400640 1 -- 172.21.15.42:0/1332574200 >> v1:172.21.15.42:6789/0 conn(0x7fc864155a00 legacy=0x7fc86406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:21.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.801+0000 7fc85b400640 1 -- 172.21.15.42:0/1332574200 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc864160550 con 0x7fc86406d110 2024-05-28T13:59:21.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.805+0000 7fc85b400640 1 --2- 172.21.15.42:0/1332574200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc86406d110 0x7fc86406eeb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc864073ed0 tx=0x7fc84c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd398e1ca65fc144 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:21.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.805+0000 7fc863400640 1 -- 172.21.15.42:0/1332574200 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0b1070 con 0x7fc86406d110 2024-05-28T13:59:21.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.805+0000 7fc863400640 1 -- 172.21.15.42:0/1332574200 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc84c0ac5f0 con 0x7fc86406d110 2024-05-28T13:59:21.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.805+0000 7fc863400640 1 -- 172.21.15.42:0/1332574200 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0ac9c0 con 0x7fc86406d110 2024-05-28T13:59:21.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.805+0000 7fc86aa00640 1 -- 172.21.15.42:0/1332574200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc86406d110 msgr2=0x7fc86406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:21.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.805+0000 7fc86aa00640 1 --2- 172.21.15.42:0/1332574200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc86406d110 0x7fc86406eeb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc864073ed0 tx=0x7fc84c01c480 comp rx=0 tx=0).stop 2024-05-28T13:59:21.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.805+0000 7fc86aa00640 1 -- 172.21.15.42:0/1332574200 shutdown_connections 2024-05-28T13:59:21.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.805+0000 7fc86aa00640 1 --2- 172.21.15.42:0/1332574200 >> v2:172.21.15.63:3300/0 conn(0x7fc86406f3f0 0x7fc864160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.805+0000 7fc86aa00640 1 --2- 172.21.15.42:0/1332574200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc86406d110 0x7fc86406eeb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.805+0000 7fc86aa00640 1 -- 172.21.15.42:0/1332574200 >> 172.21.15.42:0/1332574200 conn(0x7fc8640725e0 msgr2=0x7fc8640729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:21.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.805+0000 7fc86aa00640 1 -- 172.21.15.42:0/1332574200 shutdown_connections 2024-05-28T13:59:21.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.805+0000 7fc86aa00640 1 -- 172.21.15.42:0/1332574200 wait complete. 2024-05-28T13:59:21.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.805+0000 7fc86aa00640 1 Processor -- start 2024-05-28T13:59:21.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc86aa00640 1 -- start start 2024-05-28T13:59:21.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc86aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc86406d110 0x7fc864149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:21.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc86aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc86406f3f0 0x7fc8640af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:21.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc86aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc864155a00 0x7fc86406bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:21.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc86aa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc864161e30 con 0x7fc864155a00 2024-05-28T13:59:21.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc86aa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc864161cb0 con 0x7fc86406d110 2024-05-28T13:59:21.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc86aa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc864161fb0 con 0x7fc86406f3f0 2024-05-28T13:59:21.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc868a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc864155a00 0x7fc86406bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:21.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc868a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc864155a00 0x7fc86406bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44038/0 (socket says 172.21.15.42:44038) 2024-05-28T13:59:21.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc868a00640 1 -- 172.21.15.42:0/2590201833 learned_addr learned my addr 172.21.15.42:0/2590201833 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:21.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc863e00640 1 --2- 172.21.15.42:0/2590201833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc86406d110 0x7fc864149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:21.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc85b400640 1 --2- 172.21.15.42:0/2590201833 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc86406f3f0 0x7fc8640af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:21.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc868a00640 1 -- 172.21.15.42:0/2590201833 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc86406f3f0 msgr2=0x7fc8640af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:21.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc868a00640 1 --2- 172.21.15.42:0/2590201833 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc86406f3f0 0x7fc8640af670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc868a00640 1 -- 172.21.15.42:0/2590201833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc86406d110 msgr2=0x7fc864149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:21.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc868a00640 1 --2- 172.21.15.42:0/2590201833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc86406d110 0x7fc864149940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:21.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc868a00640 1 -- 172.21.15.42:0/2590201833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8641586a0 con 0x7fc864155a00 2024-05-28T13:59:21.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc868a00640 1 --2- 172.21.15.42:0/2590201833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc864155a00 0x7fc86406bf90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc85c007e60 tx=0x7fc85c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:21.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc860c00640 1 -- 172.21.15.42:0/2590201833 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c014420 con 0x7fc864155a00 2024-05-28T13:59:21.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc860c00640 1 -- 172.21.15.42:0/2590201833 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc85c019070 con 0x7fc864155a00 2024-05-28T13:59:21.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc860c00640 1 -- 172.21.15.42:0/2590201833 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c014820 con 0x7fc864155a00 2024-05-28T13:59:21.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc86aa00640 1 -- 172.21.15.42:0/2590201833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc864158980 con 0x7fc864155a00 2024-05-28T13:59:21.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc86aa00640 1 -- 172.21.15.42:0/2590201833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc86414a260 con 0x7fc864155a00 2024-05-28T13:59:21.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.809+0000 7fc86aa00640 1 -- 172.21.15.42:0/2590201833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc864068310 con 0x7fc864155a00 2024-05-28T13:59:21.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.813+0000 7fc860c00640 1 -- 172.21.15.42:0/2590201833 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc85c0149c0 con 0x7fc864155a00 2024-05-28T13:59:21.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.813+0000 7fc860c00640 1 --2- 172.21.15.42:0/2590201833 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc840068380 0x7fc84006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:21.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.813+0000 7fc860c00640 1 -- 172.21.15.42:0/2590201833 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fc85c086ea0 con 0x7fc864155a00 2024-05-28T13:59:21.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.813+0000 7fc863e00640 1 --2- 172.21.15.42:0/2590201833 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc840068380 0x7fc84006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:21.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.813+0000 7fc863e00640 1 --2- 172.21.15.42:0/2590201833 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc840068380 0x7fc84006a840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc854005890 tx=0x7fc854009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:21.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:21.817+0000 7fc860c00640 1 -- 172.21.15.42:0/2590201833 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc85c0577a0 con 0x7fc864155a00 2024-05-28T13:59:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.041+0000 7fc86aa00640 1 -- 172.21.15.42:0/2590201833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fc864073f20 con 0x7fc864155a00 2024-05-28T13:59:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.041+0000 7fc860c00640 1 -- 172.21.15.42:0/2590201833 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fc85c0570a0 con 0x7fc864155a00 2024-05-28T13:59:22.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.045+0000 7fc86aa00640 1 -- 172.21.15.42:0/2590201833 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc840068380 msgr2=0x7fc84006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.045+0000 7fc86aa00640 1 --2- 172.21.15.42:0/2590201833 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc840068380 0x7fc84006a840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc854005890 tx=0x7fc854009040 comp rx=0 tx=0).stop 2024-05-28T13:59:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.045+0000 7fc86aa00640 1 -- 172.21.15.42:0/2590201833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc864155a00 msgr2=0x7fc86406bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.045+0000 7fc86aa00640 1 --2- 172.21.15.42:0/2590201833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc864155a00 0x7fc86406bf90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc85c007e60 tx=0x7fc85c00e4b0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.045+0000 7fc86aa00640 1 -- 172.21.15.42:0/2590201833 shutdown_connections 2024-05-28T13:59:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.045+0000 7fc86aa00640 1 --2- 172.21.15.42:0/2590201833 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc840068380 0x7fc84006a840 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.045+0000 7fc86aa00640 1 --2- 172.21.15.42:0/2590201833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc864155a00 0x7fc86406bf90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.045+0000 7fc86aa00640 1 --2- 172.21.15.42:0/2590201833 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc86406f3f0 0x7fc8640af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.045+0000 7fc86aa00640 1 --2- 172.21.15.42:0/2590201833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc86406d110 0x7fc864149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.045+0000 7fc86aa00640 1 -- 172.21.15.42:0/2590201833 >> 172.21.15.42:0/2590201833 conn(0x7fc8640725e0 msgr2=0x7fc8640b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.045+0000 7fc86aa00640 1 -- 172.21.15.42:0/2590201833 shutdown_connections 2024-05-28T13:59:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.045+0000 7fc86aa00640 1 -- 172.21.15.42:0/2590201833 wait complete. 2024-05-28T13:59:22.050 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-28T13:59:22.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.205+0000 7f7a44000640 1 Processor -- start 2024-05-28T13:59:22.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.205+0000 7f7a44000640 1 -- start start 2024-05-28T13:59:22.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a44000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a3c145920 0x7f7a3c06f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:22.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a44000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a3c06dfa0 con 0x7f7a3c0724d0 2024-05-28T13:59:22.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a44000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a3c06e240 con 0x7f7a3c06db40 2024-05-28T13:59:22.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a44000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a3c06e520 con 0x7f7a3c145920 2024-05-28T13:59:22.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a42000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f7a3c0724d0 0x7f7a3c15d120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:43832/0 (socket says 172.21.15.42:43832) 2024-05-28T13:59:22.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a42000640 1 -- 172.21.15.42:0/3624059277 learned_addr learned my addr 172.21.15.42:0/3624059277 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:22.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a41600640 1 --2- 172.21.15.42:0/3624059277 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a3c145920 0x7f7a3c06f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:22.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a3be00640 1 -- 172.21.15.42:0/3624059277 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2109320056 0 0) 0x7f7a3c06dfa0 con 0x7f7a3c0724d0 2024-05-28T13:59:22.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a3be00640 1 -- 172.21.15.42:0/3624059277 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a24003530 con 0x7f7a3c0724d0 2024-05-28T13:59:22.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a3be00640 1 -- 172.21.15.42:0/3624059277 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2950684417 0 0) 0x7f7a3c06e240 con 0x7f7a3c06db40 2024-05-28T13:59:22.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a3be00640 1 -- 172.21.15.42:0/3624059277 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a3c06dfa0 con 0x7f7a3c06db40 2024-05-28T13:59:22.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a41600640 1 -- 172.21.15.42:0/3624059277 >> v1:172.21.15.63:6789/0 conn(0x7f7a3c06db40 legacy=0x7f7a3c071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:22.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a3be00640 1 -- 172.21.15.42:0/3624059277 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3131415506 0 0) 0x7f7a24003530 con 0x7f7a3c0724d0 2024-05-28T13:59:22.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a41600640 1 -- 172.21.15.42:0/3624059277 >> v1:172.21.15.42:6789/0 conn(0x7f7a3c0724d0 legacy=0x7f7a3c15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:22.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.209+0000 7f7a41600640 1 -- 172.21.15.42:0/3624059277 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3c06e800 con 0x7f7a3c145920 2024-05-28T13:59:22.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.213+0000 7f7a41600640 1 --2- 172.21.15.42:0/3624059277 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a3c145920 0x7f7a3c06f030 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7a300097f0 tx=0x7f7a3001c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19712574c0e00e92 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:22.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.213+0000 7f7a3be00640 1 -- 172.21.15.42:0/3624059277 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a300b0070 con 0x7f7a3c145920 2024-05-28T13:59:22.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.213+0000 7f7a3be00640 1 -- 172.21.15.42:0/3624059277 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a300ab400 con 0x7f7a3c145920 2024-05-28T13:59:22.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.213+0000 7f7a3be00640 1 -- 172.21.15.42:0/3624059277 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34001a00 con 0x7f7a3c145920 2024-05-28T13:59:22.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.213+0000 7f7a44000640 1 -- 172.21.15.42:0/3624059277 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a3c145920 msgr2=0x7f7a3c06f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:22.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.213+0000 7f7a44000640 1 --2- 172.21.15.42:0/3624059277 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a3c145920 0x7f7a3c06f030 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7a300097f0 tx=0x7f7a3001c370 comp rx=0 tx=0).stop 2024-05-28T13:59:22.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.213+0000 7f7a44000640 1 -- 172.21.15.42:0/3624059277 shutdown_connections 2024-05-28T13:59:22.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.213+0000 7f7a44000640 1 --2- 172.21.15.42:0/3624059277 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a3c145920 0x7f7a3c06f030 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.213+0000 7f7a44000640 1 -- 172.21.15.42:0/3624059277 >> 172.21.15.42:0/3624059277 conn(0x7f7a3c06b9f0 msgr2=0x7f7a3c06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:22.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.213+0000 7f7a44000640 1 -- 172.21.15.42:0/3624059277 shutdown_connections 2024-05-28T13:59:22.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.213+0000 7f7a44000640 1 -- 172.21.15.42:0/3624059277 wait complete. 2024-05-28T13:59:22.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.213+0000 7f7a44000640 1 Processor -- start 2024-05-28T13:59:22.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.213+0000 7f7a44000640 1 -- start start 2024-05-28T13:59:22.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a44000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a3c06db40 0x7f7a3c167cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:22.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a44000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a3c0724d0 0x7f7a3c13cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:22.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a44000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3c145920 0x7f7a3c167650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:22.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a44000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c16a260 con 0x7f7a3c145920 2024-05-28T13:59:22.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a44000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c15e310 con 0x7f7a3c06db40 2024-05-28T13:59:22.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a44000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a3c16a3e0 con 0x7f7a3c0724d0 2024-05-28T13:59:22.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a42000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3c145920 0x7f7a3c167650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:22.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a42000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3c145920 0x7f7a3c167650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44046/0 (socket says 172.21.15.42:44046) 2024-05-28T13:59:22.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a42000640 1 -- 172.21.15.42:0/2829104287 learned_addr learned my addr 172.21.15.42:0/2829104287 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:22.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a40c00640 1 --2- 172.21.15.42:0/2829104287 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a3c0724d0 0x7f7a3c13cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:22.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a41600640 1 --2- 172.21.15.42:0/2829104287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a3c06db40 0x7f7a3c167cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:22.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a42000640 1 -- 172.21.15.42:0/2829104287 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a3c0724d0 msgr2=0x7f7a3c13cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:22.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a42000640 1 --2- 172.21.15.42:0/2829104287 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a3c0724d0 0x7f7a3c13cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a42000640 1 -- 172.21.15.42:0/2829104287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a3c06db40 msgr2=0x7f7a3c167cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:22.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a42000640 1 --2- 172.21.15.42:0/2829104287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a3c06db40 0x7f7a3c167cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a42000640 1 -- 172.21.15.42:0/2829104287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3c0b40a0 con 0x7f7a3c145920 2024-05-28T13:59:22.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a42000640 1 --2- 172.21.15.42:0/2829104287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3c145920 0x7f7a3c167650 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400cbe0 tx=0x7f7a3400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:22.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a39600640 1 -- 172.21.15.42:0/2829104287 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3400ad40 con 0x7f7a3c145920 2024-05-28T13:59:22.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a39600640 1 -- 172.21.15.42:0/2829104287 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a34012400 con 0x7f7a3c145920 2024-05-28T13:59:22.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a39600640 1 -- 172.21.15.42:0/2829104287 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34012830 con 0x7f7a3c145920 2024-05-28T13:59:22.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a44000640 1 -- 172.21.15.42:0/2829104287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3c0b4380 con 0x7f7a3c145920 2024-05-28T13:59:22.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a44000640 1 -- 172.21.15.42:0/2829104287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a3c0b4940 con 0x7f7a3c145920 2024-05-28T13:59:22.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a44000640 1 -- 172.21.15.42:0/2829104287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3c073890 con 0x7f7a3c145920 2024-05-28T13:59:22.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.217+0000 7f7a39600640 1 -- 172.21.15.42:0/2829104287 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7a340129d0 con 0x7f7a3c145920 2024-05-28T13:59:22.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.221+0000 7f7a39600640 1 --2- 172.21.15.42:0/2829104287 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a1c068380 0x7f7a1c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:22.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.221+0000 7f7a39600640 1 -- 172.21.15.42:0/2829104287 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f7a34086d40 con 0x7f7a3c145920 2024-05-28T13:59:22.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.221+0000 7f7a41600640 1 --2- 172.21.15.42:0/2829104287 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a1c068380 0x7f7a1c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:22.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.221+0000 7f7a41600640 1 --2- 172.21.15.42:0/2829104287 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a1c068380 0x7f7a1c06a840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a300ad000 tx=0x7f7a300a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:22.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.225+0000 7f7a39600640 1 -- 172.21.15.42:0/2829104287 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7a34056ff0 con 0x7f7a3c145920 2024-05-28T13:59:22.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.445+0000 7f7a44000640 1 -- 172.21.15.42:0/2829104287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f7a3c0aff40 con 0x7f7a3c145920 2024-05-28T13:59:22.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.445+0000 7f7a39600640 1 -- 172.21.15.42:0/2829104287 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7f7a3405b2f0 con 0x7f7a3c145920 2024-05-28T13:59:22.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.449+0000 7f7a44000640 1 -- 172.21.15.42:0/2829104287 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a1c068380 msgr2=0x7f7a1c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:22.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.449+0000 7f7a44000640 1 --2- 172.21.15.42:0/2829104287 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a1c068380 0x7f7a1c06a840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a300ad000 tx=0x7f7a300a6000 comp rx=0 tx=0).stop 2024-05-28T13:59:22.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.449+0000 7f7a44000640 1 -- 172.21.15.42:0/2829104287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3c145920 msgr2=0x7f7a3c167650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:22.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.449+0000 7f7a44000640 1 --2- 172.21.15.42:0/2829104287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3c145920 0x7f7a3c167650 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400cbe0 tx=0x7f7a3400a430 comp rx=0 tx=0).stop 2024-05-28T13:59:22.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.449+0000 7f7a44000640 1 -- 172.21.15.42:0/2829104287 shutdown_connections 2024-05-28T13:59:22.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.449+0000 7f7a44000640 1 --2- 172.21.15.42:0/2829104287 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a1c068380 0x7f7a1c06a840 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.449+0000 7f7a44000640 1 --2- 172.21.15.42:0/2829104287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3c145920 0x7f7a3c167650 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.449+0000 7f7a44000640 1 --2- 172.21.15.42:0/2829104287 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a3c0724d0 0x7f7a3c13cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.449+0000 7f7a44000640 1 --2- 172.21.15.42:0/2829104287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a3c06db40 0x7f7a3c167cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.449+0000 7f7a44000640 1 -- 172.21.15.42:0/2829104287 >> 172.21.15.42:0/2829104287 conn(0x7f7a3c06b9f0 msgr2=0x7f7a3c0b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:22.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.449+0000 7f7a44000640 1 -- 172.21.15.42:0/2829104287 shutdown_connections 2024-05-28T13:59:22.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.449+0000 7f7a44000640 1 -- 172.21.15.42:0/2829104287 wait complete. 2024-05-28T13:59:22.452 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-28T13:59:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0daf400640 1 Processor -- start 2024-05-28T13:59:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0daf400640 1 -- start start 2024-05-28T13:59:22.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0daf400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da8072170 0x7f0da815b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0daf400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0da80afa40 0x7f0da8072550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0daf400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0da80b4e70 0x7f0da815e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0daf400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0da8072a90 con 0x7f0da80b4e70 2024-05-28T13:59:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0daf400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0da815b620 con 0x7f0da80afa40 2024-05-28T13:59:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0daf400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0da815b900 con 0x7f0da8072170 2024-05-28T13:59:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0dad400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0da80b4e70 0x7f0da815e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0dad400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0da80b4e70 0x7f0da815e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44056/0 (socket says 172.21.15.42:44056) 2024-05-28T13:59:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0dad400640 1 -- 172.21.15.42:0/2806896381 learned_addr learned my addr 172.21.15.42:0/2806896381 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0da7e00640 1 --2- 172.21.15.42:0/2806896381 >> v2:172.21.15.63:3300/0 conn(0x7f0da80afa40 0x7f0da8072550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0daca00640 1 --2- 172.21.15.42:0/2806896381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da8072170 0x7f0da815b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0dad400640 1 -- 172.21.15.42:0/2806896381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da8072170 msgr2=0x7f0da815b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0dad400640 1 --2- 172.21.15.42:0/2806896381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da8072170 0x7f0da815b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0dad400640 1 -- 172.21.15.42:0/2806896381 >> v2:172.21.15.63:3300/0 conn(0x7f0da80afa40 msgr2=0x7f0da8072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0dad400640 1 --2- 172.21.15.42:0/2806896381 >> v2:172.21.15.63:3300/0 conn(0x7f0da80afa40 0x7f0da8072550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.609+0000 7f0dad400640 1 -- 172.21.15.42:0/2806896381 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da815bbe0 con 0x7f0da80b4e70 2024-05-28T13:59:22.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.613+0000 7f0dad400640 1 --2- 172.21.15.42:0/2806896381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80b4e70 0x7f0da815e900 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0da8068670 tx=0x7f0da001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3f98be558e6071 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:22.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.613+0000 7f0da7400640 1 -- 172.21.15.42:0/2806896381 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da00a56d0 con 0x7f0da80b4e70 2024-05-28T13:59:22.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.613+0000 7f0da7400640 1 -- 172.21.15.42:0/2806896381 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da00a5870 con 0x7f0da80b4e70 2024-05-28T13:59:22.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.613+0000 7f0da7400640 1 -- 172.21.15.42:0/2806896381 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da00a5c40 con 0x7f0da80b4e70 2024-05-28T13:59:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.613+0000 7f0daf400640 1 -- 172.21.15.42:0/2806896381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80b4e70 msgr2=0x7f0da815e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.613+0000 7f0daf400640 1 --2- 172.21.15.42:0/2806896381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80b4e70 0x7f0da815e900 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0da8068670 tx=0x7f0da001c9c0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.613+0000 7f0daf400640 1 -- 172.21.15.42:0/2806896381 shutdown_connections 2024-05-28T13:59:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.613+0000 7f0daf400640 1 --2- 172.21.15.42:0/2806896381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80b4e70 0x7f0da815e900 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.613+0000 7f0daf400640 1 --2- 172.21.15.42:0/2806896381 >> v2:172.21.15.63:3300/0 conn(0x7f0da80afa40 0x7f0da8072550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.613+0000 7f0daf400640 1 --2- 172.21.15.42:0/2806896381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da8072170 0x7f0da815b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.613+0000 7f0daf400640 1 -- 172.21.15.42:0/2806896381 >> 172.21.15.42:0/2806896381 conn(0x7f0da80d07c0 msgr2=0x7f0da80d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.613+0000 7f0daf400640 1 -- 172.21.15.42:0/2806896381 shutdown_connections 2024-05-28T13:59:22.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.613+0000 7f0daf400640 1 -- 172.21.15.42:0/2806896381 wait complete. 2024-05-28T13:59:22.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0daf400640 1 Processor -- start 2024-05-28T13:59:22.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0daf400640 1 -- start start 2024-05-28T13:59:22.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0daf400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da8072170 0x7f0da81483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:22.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0daf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80afa40 0x7f0da81f9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:22.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0daf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da80b4e70 0x7f0da8155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:22.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0daf400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0da815f1d0 con 0x7f0da80afa40 2024-05-28T13:59:22.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0daf400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0da815f050 con 0x7f0da80b4e70 2024-05-28T13:59:22.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0daf400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0da815f350 con 0x7f0da8072170 2024-05-28T13:59:22.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0da7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80afa40 0x7f0da81f9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:22.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0da7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80afa40 0x7f0da81f9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44072/0 (socket says 172.21.15.42:44072) 2024-05-28T13:59:22.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0da7e00640 1 -- 172.21.15.42:0/2146405420 learned_addr learned my addr 172.21.15.42:0/2146405420 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:22.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0daca00640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da8072170 0x7f0da81483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:22.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0dad400640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da80b4e70 0x7f0da8155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:22.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0da7e00640 1 -- 172.21.15.42:0/2146405420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da8072170 msgr2=0x7f0da81483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:22.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0da7e00640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da8072170 0x7f0da81483a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0da7e00640 1 -- 172.21.15.42:0/2146405420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da80b4e70 msgr2=0x7f0da8155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:22.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0da7e00640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da80b4e70 0x7f0da8155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0da7e00640 1 -- 172.21.15.42:0/2146405420 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da8152ce0 con 0x7f0da80afa40 2024-05-28T13:59:22.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0daca00640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da8072170 0x7f0da81483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:22.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0da7e00640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80afa40 0x7f0da81f9690 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400e930 tx=0x7f0d9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:22.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0da4c00640 1 -- 172.21.15.42:0/2146405420 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94017070 con 0x7f0da80afa40 2024-05-28T13:59:22.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0da4c00640 1 -- 172.21.15.42:0/2146405420 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9400ca00 con 0x7f0da80afa40 2024-05-28T13:59:22.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0da4c00640 1 -- 172.21.15.42:0/2146405420 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9400cdf0 con 0x7f0da80afa40 2024-05-28T13:59:22.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0daf400640 1 -- 172.21.15.42:0/2146405420 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da81fa0e0 con 0x7f0da80afa40 2024-05-28T13:59:22.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.617+0000 7f0daf400640 1 -- 172.21.15.42:0/2146405420 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da8152fa0 con 0x7f0da80afa40 2024-05-28T13:59:22.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.621+0000 7f0daf400640 1 -- 172.21.15.42:0/2146405420 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da8073bb0 con 0x7f0da80afa40 2024-05-28T13:59:22.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.621+0000 7f0da4c00640 1 -- 172.21.15.42:0/2146405420 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0d9401a070 con 0x7f0da80afa40 2024-05-28T13:59:22.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.621+0000 7f0da4c00640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d88068380 0x7f0d8806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:22.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.621+0000 7f0da4c00640 1 -- 172.21.15.42:0/2146405420 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f0d94086930 con 0x7f0da80afa40 2024-05-28T13:59:22.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.621+0000 7f0daca00640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d88068380 0x7f0d8806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:22.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.621+0000 7f0daca00640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d88068380 0x7f0d8806a840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009bb0 tx=0x7f0d9c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.625+0000 7f0da4c00640 1 -- 172.21.15.42:0/2146405420 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0d94056ca0 con 0x7f0da80afa40 2024-05-28T13:59:22.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.853+0000 7f0daf400640 1 -- 172.21.15.42:0/2146405420 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f0da8072870 con 0x7f0da80afa40 2024-05-28T13:59:22.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.853+0000 7f0da4c00640 1 -- 172.21.15.42:0/2146405420 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v12) ==== 97+0+0 (secure 0 0 0) 0x7f0d9405afa0 con 0x7f0da80afa40 2024-05-28T13:59:22.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.857+0000 7f0daf400640 1 -- 172.21.15.42:0/2146405420 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d88068380 msgr2=0x7f0d8806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:22.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.857+0000 7f0daf400640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d88068380 0x7f0d8806a840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009bb0 tx=0x7f0d9c008040 comp rx=0 tx=0).stop 2024-05-28T13:59:22.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.857+0000 7f0daf400640 1 -- 172.21.15.42:0/2146405420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80afa40 msgr2=0x7f0da81f9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:22.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.857+0000 7f0daf400640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80afa40 0x7f0da81f9690 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400e930 tx=0x7f0d9400edf0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.857+0000 7f0daf400640 1 -- 172.21.15.42:0/2146405420 shutdown_connections 2024-05-28T13:59:22.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.857+0000 7f0daf400640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d88068380 0x7f0d8806a840 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.857+0000 7f0daf400640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da80b4e70 0x7f0da8155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.857+0000 7f0daf400640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80afa40 0x7f0da81f9690 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.857+0000 7f0daf400640 1 --2- 172.21.15.42:0/2146405420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da8072170 0x7f0da81483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:22.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.857+0000 7f0daf400640 1 -- 172.21.15.42:0/2146405420 >> 172.21.15.42:0/2146405420 conn(0x7f0da80d07c0 msgr2=0x7f0da80ce380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:22.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.857+0000 7f0daf400640 1 -- 172.21.15.42:0/2146405420 shutdown_connections 2024-05-28T13:59:22.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:22.857+0000 7f0daf400640 1 -- 172.21.15.42:0/2146405420 wait complete. 2024-05-28T13:59:22.861 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-28T13:59:23.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.017+0000 7f96fac00640 1 Processor -- start 2024-05-28T13:59:23.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.017+0000 7f96fac00640 1 -- start start 2024-05-28T13:59:23.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.017+0000 7f96fac00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f96f415bf30 0x7f96f415c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.017+0000 7f96fac00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f96f40cf230 0x7f96f4072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.017+0000 7f96fac00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96f40cfc10 con 0x7f96f4072ba0 2024-05-28T13:59:23.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.017+0000 7f96fac00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f96f40721d0 con 0x7f96f415bf30 2024-05-28T13:59:23.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.017+0000 7f96fac00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f96f40cfef0 con 0x7f96f40cf230 2024-05-28T13:59:23.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.017+0000 7f96f3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f96f415bf30 0x7f96f415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.017+0000 7f96f8c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f96f4072ba0 0x7f96f406ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:43834/0 (socket says 172.21.15.42:43834) 2024-05-28T13:59:23.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.017+0000 7f96f8c00640 1 -- 172.21.15.42:0/1366218556 learned_addr learned my addr 172.21.15.42:0/1366218556 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:23.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.021+0000 7f96eb400640 1 --2- 172.21.15.42:0/1366218556 >> v2:172.21.15.63:3300/0 conn(0x7f96f40cf230 0x7f96f4072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.021+0000 7f96f3e00640 1 -- 172.21.15.42:0/1366218556 >> v2:172.21.15.63:3300/0 conn(0x7f96f40cf230 msgr2=0x7f96f4072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:23.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.021+0000 7f96f3e00640 1 --2- 172.21.15.42:0/1366218556 >> v2:172.21.15.63:3300/0 conn(0x7f96f40cf230 0x7f96f4072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.021+0000 7f96f3e00640 1 -- 172.21.15.42:0/1366218556 >> v1:172.21.15.42:6789/0 conn(0x7f96f4072ba0 legacy=0x7f96f406ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.021+0000 7f96f3e00640 1 -- 172.21.15.42:0/1366218556 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f40d01d0 con 0x7f96f415bf30 2024-05-28T13:59:23.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.021+0000 7f96f3400640 1 -- 172.21.15.42:0/1366218556 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689546203 0 0) 0x7f96f40cfc10 con 0x7f96f4072ba0 2024-05-28T13:59:23.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.021+0000 7f96f3e00640 1 --2- 172.21.15.42:0/1366218556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f415bf30 0x7f96f415c310 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f96f40b0130 tx=0x7f96e401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db47f8b6b716fb51 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:23.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.021+0000 7f96f3400640 1 -- 172.21.15.42:0/1366218556 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e40b0070 con 0x7f96f415bf30 2024-05-28T13:59:23.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.021+0000 7f96f3400640 1 -- 172.21.15.42:0/1366218556 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e40ab600 con 0x7f96f415bf30 2024-05-28T13:59:23.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.021+0000 7f96f3400640 1 -- 172.21.15.42:0/1366218556 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e40ab9d0 con 0x7f96f415bf30 2024-05-28T13:59:23.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.021+0000 7f96fac00640 1 -- 172.21.15.42:0/1366218556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f415bf30 msgr2=0x7f96f415c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.021+0000 7f96fac00640 1 --2- 172.21.15.42:0/1366218556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f415bf30 0x7f96f415c310 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f96f40b0130 tx=0x7f96e401c450 comp rx=0 tx=0).stop 2024-05-28T13:59:23.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96fac00640 1 -- 172.21.15.42:0/1366218556 shutdown_connections 2024-05-28T13:59:23.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96fac00640 1 --2- 172.21.15.42:0/1366218556 >> v2:172.21.15.63:3300/0 conn(0x7f96f40cf230 0x7f96f4072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96fac00640 1 --2- 172.21.15.42:0/1366218556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f415bf30 0x7f96f415c310 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96fac00640 1 -- 172.21.15.42:0/1366218556 >> 172.21.15.42:0/1366218556 conn(0x7f96f406b9f0 msgr2=0x7f96f406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:23.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96fac00640 1 -- 172.21.15.42:0/1366218556 shutdown_connections 2024-05-28T13:59:23.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96fac00640 1 -- 172.21.15.42:0/1366218556 wait complete. 2024-05-28T13:59:23.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96fac00640 1 Processor -- start 2024-05-28T13:59:23.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96fac00640 1 -- start start 2024-05-28T13:59:23.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96fac00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96f4072ba0 0x7f96f413adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96fac00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96f40cf230 0x7f96f414a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96fac00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f415bf30 0x7f96f4153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96fac00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96f406f4e0 con 0x7f96f415bf30 2024-05-28T13:59:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96fac00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f96f406f360 con 0x7f96f40cf230 2024-05-28T13:59:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96fac00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f96f40cf610 con 0x7f96f4072ba0 2024-05-28T13:59:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96f8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f415bf30 0x7f96f4153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96f8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f415bf30 0x7f96f4153870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44082/0 (socket says 172.21.15.42:44082) 2024-05-28T13:59:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96f8c00640 1 -- 172.21.15.42:0/2947086000 learned_addr learned my addr 172.21.15.42:0/2947086000 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96f3e00640 1 --2- 172.21.15.42:0/2947086000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96f4072ba0 0x7f96f413adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96eb400640 1 --2- 172.21.15.42:0/2947086000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96f40cf230 0x7f96f414a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96f8c00640 1 -- 172.21.15.42:0/2947086000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96f4072ba0 msgr2=0x7f96f413adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96f8c00640 1 --2- 172.21.15.42:0/2947086000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96f4072ba0 0x7f96f413adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96f8c00640 1 -- 172.21.15.42:0/2947086000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96f40cf230 msgr2=0x7f96f414a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96f8c00640 1 --2- 172.21.15.42:0/2947086000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96f40cf230 0x7f96f414a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96f8c00640 1 -- 172.21.15.42:0/2947086000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f4150900 con 0x7f96f415bf30 2024-05-28T13:59:23.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.025+0000 7f96f8c00640 1 --2- 172.21.15.42:0/2947086000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f415bf30 0x7f96f4153870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00c7b0 tx=0x7f96ec00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:23.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.029+0000 7f96f0c00640 1 -- 172.21.15.42:0/2947086000 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec018070 con 0x7f96f415bf30 2024-05-28T13:59:23.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.029+0000 7f96f0c00640 1 -- 172.21.15.42:0/2947086000 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ec00a640 con 0x7f96f415bf30 2024-05-28T13:59:23.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.029+0000 7f96fac00640 1 -- 172.21.15.42:0/2947086000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f4150be0 con 0x7f96f415bf30 2024-05-28T13:59:23.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.029+0000 7f96f0c00640 1 -- 172.21.15.42:0/2947086000 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec00aa70 con 0x7f96f415bf30 2024-05-28T13:59:23.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.029+0000 7f96fac00640 1 -- 172.21.15.42:0/2947086000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f40afb50 con 0x7f96f415bf30 2024-05-28T13:59:23.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.029+0000 7f96fac00640 1 -- 172.21.15.42:0/2947086000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f414ab30 con 0x7f96f415bf30 2024-05-28T13:59:23.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.029+0000 7f96f0c00640 1 -- 172.21.15.42:0/2947086000 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f96ec01b070 con 0x7f96f415bf30 2024-05-28T13:59:23.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.029+0000 7f96f0c00640 1 --2- 172.21.15.42:0/2947086000 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f96d00683b0 0x7f96d006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.029+0000 7f96f0c00640 1 -- 172.21.15.42:0/2947086000 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f96ec085b70 con 0x7f96f415bf30 2024-05-28T13:59:23.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.029+0000 7f96f3e00640 1 --2- 172.21.15.42:0/2947086000 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f96d00683b0 0x7f96d006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.029+0000 7f96f3e00640 1 --2- 172.21.15.42:0/2947086000 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f96d00683b0 0x7f96d006a870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f96e40ad000 tx=0x7f96e40a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:23.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.033+0000 7f96f0c00640 1 -- 172.21.15.42:0/2947086000 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f96ec055e20 con 0x7f96f415bf30 2024-05-28T13:59:23.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.257+0000 7f96fac00640 1 -- 172.21.15.42:0/2947086000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f96f4073890 con 0x7f96f415bf30 2024-05-28T13:59:23.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.261+0000 7f96f0c00640 1 -- 172.21.15.42:0/2947086000 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v13) ==== 90+0+0 (secure 0 0 0) 0x7f96ec05a120 con 0x7f96f415bf30 2024-05-28T13:59:23.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.261+0000 7f96fac00640 1 -- 172.21.15.42:0/2947086000 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f96d00683b0 msgr2=0x7f96d006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.261+0000 7f96fac00640 1 --2- 172.21.15.42:0/2947086000 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f96d00683b0 0x7f96d006a870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f96e40ad000 tx=0x7f96e40a6000 comp rx=0 tx=0).stop 2024-05-28T13:59:23.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.261+0000 7f96fac00640 1 -- 172.21.15.42:0/2947086000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f415bf30 msgr2=0x7f96f4153870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.261+0000 7f96fac00640 1 --2- 172.21.15.42:0/2947086000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f415bf30 0x7f96f4153870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00c7b0 tx=0x7f96ec00cc70 comp rx=0 tx=0).stop 2024-05-28T13:59:23.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.261+0000 7f96fac00640 1 -- 172.21.15.42:0/2947086000 shutdown_connections 2024-05-28T13:59:23.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.261+0000 7f96fac00640 1 --2- 172.21.15.42:0/2947086000 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f96d00683b0 0x7f96d006a870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.261+0000 7f96fac00640 1 --2- 172.21.15.42:0/2947086000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f415bf30 0x7f96f4153870 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.261+0000 7f96fac00640 1 --2- 172.21.15.42:0/2947086000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96f40cf230 0x7f96f414a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.261+0000 7f96fac00640 1 --2- 172.21.15.42:0/2947086000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96f4072ba0 0x7f96f413adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.261+0000 7f96fac00640 1 -- 172.21.15.42:0/2947086000 >> 172.21.15.42:0/2947086000 conn(0x7f96f406b9f0 msgr2=0x7f96f415e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:23.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.261+0000 7f96fac00640 1 -- 172.21.15.42:0/2947086000 shutdown_connections 2024-05-28T13:59:23.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.261+0000 7f96fac00640 1 -- 172.21.15.42:0/2947086000 wait complete. 2024-05-28T13:59:23.268 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:59:23.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.425+0000 7f70dbc00640 1 Processor -- start 2024-05-28T13:59:23.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.425+0000 7f70dbc00640 1 -- start start 2024-05-28T13:59:23.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.425+0000 7f70dbc00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f70d409bbd0 0x7f70d409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.425+0000 7f70dbc00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70d40d0870 con 0x7f70d415c990 2024-05-28T13:59:23.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.425+0000 7f70dbc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70d40cd300 con 0x7f70d409af90 2024-05-28T13:59:23.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.425+0000 7f70dbc00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f70d409b400 con 0x7f70d409bbd0 2024-05-28T13:59:23.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.425+0000 7f70d9200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f70d415c990 0x7f70d40d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:43850/0 (socket says 172.21.15.42:43850) 2024-05-28T13:59:23.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.425+0000 7f70d9200640 1 -- 172.21.15.42:0/775382312 learned_addr learned my addr 172.21.15.42:0/775382312 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:23.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.425+0000 7f70d3e00640 1 --2- 172.21.15.42:0/775382312 >> v2:172.21.15.63:3300/0 conn(0x7f70d409bbd0 0x7f70d409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.425+0000 7f70d3400640 1 -- 172.21.15.42:0/775382312 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3791850750 0 0) 0x7f70d40d0870 con 0x7f70d415c990 2024-05-28T13:59:23.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.429+0000 7f70d3400640 1 -- 172.21.15.42:0/775382312 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70bc003530 con 0x7f70d415c990 2024-05-28T13:59:23.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.429+0000 7f70d3e00640 1 -- 172.21.15.42:0/775382312 >> v1:172.21.15.63:6789/0 conn(0x7f70d409af90 legacy=0x7f70d40b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.429+0000 7f70d3e00640 1 -- 172.21.15.42:0/775382312 >> v1:172.21.15.42:6789/0 conn(0x7f70d415c990 legacy=0x7f70d40d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.429+0000 7f70d3e00640 1 -- 172.21.15.42:0/775382312 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d40cd5e0 con 0x7f70d409bbd0 2024-05-28T13:59:23.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.429+0000 7f70d3e00640 1 --2- 172.21.15.42:0/775382312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70d409bbd0 0x7f70d409aa50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f70c00097c0 tx=0x7f70c001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65cee3794c365ca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.429+0000 7f70d3400640 1 -- 172.21.15.42:0/775382312 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c00b1070 con 0x7f70d409bbd0 2024-05-28T13:59:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.429+0000 7f70d3400640 1 -- 172.21.15.42:0/775382312 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c00ac5f0 con 0x7f70d409bbd0 2024-05-28T13:59:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.429+0000 7f70d3400640 1 -- 172.21.15.42:0/775382312 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c00ac9c0 con 0x7f70d409bbd0 2024-05-28T13:59:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.429+0000 7f70dbc00640 1 -- 172.21.15.42:0/775382312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70d409bbd0 msgr2=0x7f70d409aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.429+0000 7f70dbc00640 1 --2- 172.21.15.42:0/775382312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70d409bbd0 0x7f70d409aa50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f70c00097c0 tx=0x7f70c001c4a0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70dbc00640 1 -- 172.21.15.42:0/775382312 shutdown_connections 2024-05-28T13:59:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70dbc00640 1 --2- 172.21.15.42:0/775382312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70d409bbd0 0x7f70d409aa50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70dbc00640 1 -- 172.21.15.42:0/775382312 >> 172.21.15.42:0/775382312 conn(0x7f70d40b4080 msgr2=0x7f70d40b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70dbc00640 1 -- 172.21.15.42:0/775382312 shutdown_connections 2024-05-28T13:59:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70dbc00640 1 -- 172.21.15.42:0/775382312 wait complete. 2024-05-28T13:59:23.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70dbc00640 1 Processor -- start 2024-05-28T13:59:23.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70dbc00640 1 -- start start 2024-05-28T13:59:23.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70dbc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70d409af90 0x7f70d406de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70dbc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70d415c990 0x7f70d41525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70dbc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70d40c9110 0x7f70d4070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70dbc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f70d40ce3a0 con 0x7f70d40c9110 2024-05-28T13:59:23.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70dbc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f70d416a750 con 0x7f70d409af90 2024-05-28T13:59:23.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70dbc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f70d4155840 con 0x7f70d415c990 2024-05-28T13:59:23.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70d9200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70d409af90 0x7f70d406de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70d9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70d40c9110 0x7f70d4070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70d9200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70d409af90 0x7f70d406de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:35152/0 (socket says 172.21.15.42:35152) 2024-05-28T13:59:23.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70d9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70d40c9110 0x7f70d4070030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44092/0 (socket says 172.21.15.42:44092) 2024-05-28T13:59:23.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70d9200640 1 -- 172.21.15.42:0/4042942656 learned_addr learned my addr 172.21.15.42:0/4042942656 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:23.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70d3e00640 1 --2- 172.21.15.42:0/4042942656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70d415c990 0x7f70d41525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70d9c00640 1 -- 172.21.15.42:0/4042942656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70d415c990 msgr2=0x7f70d41525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70d9c00640 1 --2- 172.21.15.42:0/4042942656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70d415c990 0x7f70d41525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70d9c00640 1 -- 172.21.15.42:0/4042942656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70d409af90 msgr2=0x7f70d406de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70d9c00640 1 --2- 172.21.15.42:0/4042942656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70d409af90 0x7f70d406de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70d9c00640 1 -- 172.21.15.42:0/4042942656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d406e360 con 0x7f70d40c9110 2024-05-28T13:59:23.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.433+0000 7f70d9200640 1 --2- 172.21.15.42:0/4042942656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70d409af90 0x7f70d406de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:59:23.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.437+0000 7f70d9c00640 1 --2- 172.21.15.42:0/4042942656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70d40c9110 0x7f70d4070030 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00c730 tx=0x7f70cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:23.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.437+0000 7f70d0c00640 1 -- 172.21.15.42:0/4042942656 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc016020 con 0x7f70d40c9110 2024-05-28T13:59:23.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.437+0000 7f70d0c00640 1 -- 172.21.15.42:0/4042942656 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70cc006d60 con 0x7f70d40c9110 2024-05-28T13:59:23.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.437+0000 7f70d0c00640 1 -- 172.21.15.42:0/4042942656 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc003bc0 con 0x7f70d40c9110 2024-05-28T13:59:23.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.437+0000 7f70dbc00640 1 -- 172.21.15.42:0/4042942656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d406e600 con 0x7f70d40c9110 2024-05-28T13:59:23.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.437+0000 7f70dbc00640 1 -- 172.21.15.42:0/4042942656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d4156cc0 con 0x7f70d40c9110 2024-05-28T13:59:23.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.437+0000 7f70dbc00640 1 -- 172.21.15.42:0/4042942656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d4068a90 con 0x7f70d40c9110 2024-05-28T13:59:23.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.437+0000 7f70d0c00640 1 -- 172.21.15.42:0/4042942656 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f70cc0050d0 con 0x7f70d40c9110 2024-05-28T13:59:23.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.437+0000 7f70d0c00640 1 --2- 172.21.15.42:0/4042942656 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70b4068380 0x7f70b406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.437+0000 7f70d0c00640 1 -- 172.21.15.42:0/4042942656 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f70cc086030 con 0x7f70d40c9110 2024-05-28T13:59:23.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.437+0000 7f70d9200640 1 --2- 172.21.15.42:0/4042942656 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70b4068380 0x7f70b406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.441+0000 7f70d9200640 1 --2- 172.21.15.42:0/4042942656 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70b4068380 0x7f70b406a840 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f70c80011f0 tx=0x7f70c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:23.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.441+0000 7f70d0c00640 1 -- 172.21.15.42:0/4042942656 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f70cc020020 con 0x7f70d40c9110 2024-05-28T13:59:23.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.613+0000 7f70dbc00640 1 -- 172.21.15.42:0/4042942656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70d40ce520 con 0x7f70d40c9110 2024-05-28T13:59:23.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.613+0000 7f70d0c00640 1 -- 172.21.15.42:0/4042942656 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+12831 (secure 0 0 0) 0x7f70cc0562e0 con 0x7f70d40c9110 2024-05-28T13:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":30,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:59:20.416209+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:59:23.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.617+0000 7f70dbc00640 1 -- 172.21.15.42:0/4042942656 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70b4068380 msgr2=0x7f70b406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.617+0000 7f70dbc00640 1 --2- 172.21.15.42:0/4042942656 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70b4068380 0x7f70b406a840 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f70c80011f0 tx=0x7f70c8009040 comp rx=0 tx=0).stop 2024-05-28T13:59:23.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.617+0000 7f70dbc00640 1 -- 172.21.15.42:0/4042942656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70d40c9110 msgr2=0x7f70d4070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.617+0000 7f70dbc00640 1 --2- 172.21.15.42:0/4042942656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70d40c9110 0x7f70d4070030 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00c730 tx=0x7f70cc00cbf0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.617+0000 7f70dbc00640 1 -- 172.21.15.42:0/4042942656 shutdown_connections 2024-05-28T13:59:23.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.617+0000 7f70dbc00640 1 --2- 172.21.15.42:0/4042942656 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70b4068380 0x7f70b406a840 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.617+0000 7f70dbc00640 1 --2- 172.21.15.42:0/4042942656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70d40c9110 0x7f70d4070030 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.617+0000 7f70dbc00640 1 --2- 172.21.15.42:0/4042942656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70d415c990 0x7f70d41525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.617+0000 7f70dbc00640 1 --2- 172.21.15.42:0/4042942656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70d409af90 0x7f70d406de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.617+0000 7f70dbc00640 1 -- 172.21.15.42:0/4042942656 >> 172.21.15.42:0/4042942656 conn(0x7f70d40b4080 msgr2=0x7f70d40d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:23.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.617+0000 7f70dbc00640 1 -- 172.21.15.42:0/4042942656 shutdown_connections 2024-05-28T13:59:23.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.617+0000 7f70dbc00640 1 -- 172.21.15.42:0/4042942656 wait complete. 2024-05-28T13:59:23.621 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T13:59:23.621 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T13:59:23.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c81c00640 1 Processor -- start 2024-05-28T13:59:23.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c81c00640 1 -- start start 2024-05-28T13:59:23.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c81c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8c7c0ce3b0 0x7f8c7c0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c81c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8c7c13fd10 0x7f8c7c140180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c81c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c7c160230 con 0x7f8c7c071de0 2024-05-28T13:59:23.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c81c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8c7c160510 con 0x7f8c7c0ce3b0 2024-05-28T13:59:23.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c81c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8c7c1607f0 con 0x7f8c7c13fd10 2024-05-28T13:59:23.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c7aa00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8c7c071de0 0x7f8c7c0721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:43854/0 (socket says 172.21.15.42:43854) 2024-05-28T13:59:23.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c7aa00640 1 -- 172.21.15.42:0/3980414465 learned_addr learned my addr 172.21.15.42:0/3980414465 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:23.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c72000640 1 --2- 172.21.15.42:0/3980414465 >> v2:172.21.15.42:3300/0 conn(0x7f8c7c0ce3b0 0x7f8c7c0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c7b400640 1 --2- 172.21.15.42:0/3980414465 >> v2:172.21.15.63:3300/0 conn(0x7f8c7c13fd10 0x7f8c7c140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c7a000640 1 -- 172.21.15.42:0/3980414465 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 568288027 0 0) 0x7f8c7c160230 con 0x7f8c7c071de0 2024-05-28T13:59:23.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c7a000640 1 -- 172.21.15.42:0/3980414465 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c60003530 con 0x7f8c7c071de0 2024-05-28T13:59:23.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c72000640 1 -- 172.21.15.42:0/3980414465 >> v2:172.21.15.63:3300/0 conn(0x7f8c7c13fd10 msgr2=0x7f8c7c140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:23.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c72000640 1 --2- 172.21.15.42:0/3980414465 >> v2:172.21.15.63:3300/0 conn(0x7f8c7c13fd10 0x7f8c7c140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c72000640 1 -- 172.21.15.42:0/3980414465 >> v1:172.21.15.42:6789/0 conn(0x7f8c7c071de0 legacy=0x7f8c7c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.781+0000 7f8c72000640 1 -- 172.21.15.42:0/3980414465 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c160ad0 con 0x7f8c7c0ce3b0 2024-05-28T13:59:23.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.785+0000 7f8c72000640 1 --2- 172.21.15.42:0/3980414465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c7c0ce3b0 0x7f8c7c0ce810 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c071680 tx=0x7f8c6401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b911d4799f91ff server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:23.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.785+0000 7f8c7a000640 1 -- 172.21.15.42:0/3980414465 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c640b0070 con 0x7f8c7c0ce3b0 2024-05-28T13:59:23.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.785+0000 7f8c7a000640 1 -- 172.21.15.42:0/3980414465 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c640ab600 con 0x7f8c7c0ce3b0 2024-05-28T13:59:23.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.785+0000 7f8c7a000640 1 -- 172.21.15.42:0/3980414465 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c640ab9d0 con 0x7f8c7c0ce3b0 2024-05-28T13:59:23.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.785+0000 7f8c81c00640 1 -- 172.21.15.42:0/3980414465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c7c0ce3b0 msgr2=0x7f8c7c0ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.785+0000 7f8c81c00640 1 --2- 172.21.15.42:0/3980414465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c7c0ce3b0 0x7f8c7c0ce810 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c071680 tx=0x7f8c6401c360 comp rx=0 tx=0).stop 2024-05-28T13:59:23.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.785+0000 7f8c81c00640 1 -- 172.21.15.42:0/3980414465 shutdown_connections 2024-05-28T13:59:23.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.785+0000 7f8c81c00640 1 --2- 172.21.15.42:0/3980414465 >> v2:172.21.15.63:3300/0 conn(0x7f8c7c13fd10 0x7f8c7c140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.785+0000 7f8c81c00640 1 --2- 172.21.15.42:0/3980414465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c7c0ce3b0 0x7f8c7c0ce810 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.785+0000 7f8c81c00640 1 -- 172.21.15.42:0/3980414465 >> 172.21.15.42:0/3980414465 conn(0x7f8c7c0b21f0 msgr2=0x7f8c7c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:23.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.785+0000 7f8c81c00640 1 -- 172.21.15.42:0/3980414465 shutdown_connections 2024-05-28T13:59:23.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.785+0000 7f8c81c00640 1 -- 172.21.15.42:0/3980414465 wait complete. 2024-05-28T13:59:23.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c81c00640 1 Processor -- start 2024-05-28T13:59:23.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c81c00640 1 -- start start 2024-05-28T13:59:23.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c81c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c7c071de0 0x7f8c7c1554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c81c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c7c0ce3b0 0x7f8c7c1543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c81c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c7c13fd10 0x7f8c7c1455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c81c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c169830 con 0x7f8c7c13fd10 2024-05-28T13:59:23.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c81c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c1696b0 con 0x7f8c7c0ce3b0 2024-05-28T13:59:23.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c81c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8c7c1699b0 con 0x7f8c7c071de0 2024-05-28T13:59:23.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c7b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c7c13fd10 0x7f8c7c1455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c7b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c7c13fd10 0x7f8c7c1455f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44106/0 (socket says 172.21.15.42:44106) 2024-05-28T13:59:23.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c7b400640 1 -- 172.21.15.42:0/1449950517 learned_addr learned my addr 172.21.15.42:0/1449950517 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:23.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c72000640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c7c0ce3b0 0x7f8c7c1543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c7aa00640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c7c071de0 0x7f8c7c1554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c7b400640 1 -- 172.21.15.42:0/1449950517 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c7c071de0 msgr2=0x7f8c7c1554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c7b400640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c7c071de0 0x7f8c7c1554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c7b400640 1 -- 172.21.15.42:0/1449950517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c7c0ce3b0 msgr2=0x7f8c7c1543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c7b400640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c7c0ce3b0 0x7f8c7c1543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c7b400640 1 -- 172.21.15.42:0/1449950517 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c1382d0 con 0x7f8c7c13fd10 2024-05-28T13:59:23.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c72000640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c7c0ce3b0 0x7f8c7c1543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:23.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c7aa00640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c7c071de0 0x7f8c7c1554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:23.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c7b400640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c7c13fd10 0x7f8c7c1455f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009d30 tx=0x7f8c7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:23.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c73400640 1 -- 172.21.15.42:0/1449950517 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74018020 con 0x7f8c7c13fd10 2024-05-28T13:59:23.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c73400640 1 -- 172.21.15.42:0/1449950517 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c74014480 con 0x7f8c7c13fd10 2024-05-28T13:59:23.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c81c00640 1 -- 172.21.15.42:0/1449950517 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7c155bf0 con 0x7f8c7c13fd10 2024-05-28T13:59:23.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c73400640 1 -- 172.21.15.42:0/1449950517 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74014870 con 0x7f8c7c13fd10 2024-05-28T13:59:23.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.789+0000 7f8c81c00640 1 -- 172.21.15.42:0/1449950517 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c7c1561b0 con 0x7f8c7c13fd10 2024-05-28T13:59:23.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.793+0000 7f8c81c00640 1 -- 172.21.15.42:0/1449950517 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c7c0738b0 con 0x7f8c7c13fd10 2024-05-28T13:59:23.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.793+0000 7f8c73400640 1 -- 172.21.15.42:0/1449950517 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8c7401b020 con 0x7f8c7c13fd10 2024-05-28T13:59:23.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.793+0000 7f8c73400640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c58068380 0x7f8c5806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:23.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.793+0000 7f8c73400640 1 -- 172.21.15.42:0/1449950517 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f8c740870b0 con 0x7f8c7c13fd10 2024-05-28T13:59:23.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.793+0000 7f8c7aa00640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c58068380 0x7f8c5806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:23.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.793+0000 7f8c7aa00640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c58068380 0x7f8c5806a840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c005890 tx=0x7f8c6c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:23.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.797+0000 7f8c73400640 1 -- 172.21.15.42:0/1449950517 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8c74022070 con 0x7f8c7c13fd10 2024-05-28T13:59:23.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.965+0000 7f8c81c00640 1 -- 172.21.15.42:0/1449950517 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8c7c15a4c0 con 0x7f8c7c13fd10 2024-05-28T13:59:23.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.965+0000 7f8c73400640 1 -- 172.21.15.42:0/1449950517 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) ==== 93+0+10 (secure 0 0 0) 0x7f8c7c1382d0 con 0x7f8c7c13fd10 2024-05-28T13:59:23.952 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T13:59:23.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.969+0000 7f8c81c00640 1 -- 172.21.15.42:0/1449950517 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c58068380 msgr2=0x7f8c5806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.969+0000 7f8c81c00640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c58068380 0x7f8c5806a840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c005890 tx=0x7f8c6c009040 comp rx=0 tx=0).stop 2024-05-28T13:59:23.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.969+0000 7f8c81c00640 1 -- 172.21.15.42:0/1449950517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c7c13fd10 msgr2=0x7f8c7c1455f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:23.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.969+0000 7f8c81c00640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c7c13fd10 0x7f8c7c1455f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009d30 tx=0x7f8c7400c490 comp rx=0 tx=0).stop 2024-05-28T13:59:23.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.969+0000 7f8c81c00640 1 -- 172.21.15.42:0/1449950517 shutdown_connections 2024-05-28T13:59:23.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.969+0000 7f8c81c00640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c58068380 0x7f8c5806a840 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.969+0000 7f8c81c00640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c7c13fd10 0x7f8c7c1455f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.969+0000 7f8c81c00640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c7c0ce3b0 0x7f8c7c1543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.969+0000 7f8c81c00640 1 --2- 172.21.15.42:0/1449950517 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c7c071de0 0x7f8c7c1554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:23.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.969+0000 7f8c81c00640 1 -- 172.21.15.42:0/1449950517 >> 172.21.15.42:0/1449950517 conn(0x7f8c7c0b21f0 msgr2=0x7f8c7c0b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:23.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.969+0000 7f8c81c00640 1 -- 172.21.15.42:0/1449950517 shutdown_connections 2024-05-28T13:59:23.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:23.969+0000 7f8c81c00640 1 -- 172.21.15.42:0/1449950517 wait complete. 2024-05-28T13:59:23.974 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-28T13:59:24.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.129+0000 7f2fb8e00640 1 Processor -- start 2024-05-28T13:59:24.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.129+0000 7f2fb8e00640 1 -- start start 2024-05-28T13:59:24.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.129+0000 7f2fb8e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2fb415e970 0x7f2fb40d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:24.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.129+0000 7f2fb8e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb406ff60 0x7f2fb40ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:24.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.129+0000 7f2fb8e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2fb40cd040 0x7f2fb4162500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:24.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.129+0000 7f2fb8e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f2fb40d0640 con 0x7f2fb40cd040 2024-05-28T13:59:24.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.129+0000 7f2fb8e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2fb40d1160 con 0x7f2fb415e970 2024-05-28T13:59:24.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.129+0000 7f2fb8e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb4070740 con 0x7f2fb406ff60 2024-05-28T13:59:24.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fb2800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2fb40cd040 0x7f2fb4162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:24.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fb2800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2fb40cd040 0x7f2fb4162500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44110/0 (socket says 172.21.15.42:44110) 2024-05-28T13:59:24.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fb2800640 1 -- 172.21.15.42:0/4055222692 learned_addr learned my addr 172.21.15.42:0/4055222692 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:24.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fb1e00640 1 --2- 172.21.15.42:0/4055222692 >> v2:172.21.15.63:3300/0 conn(0x7f2fb415e970 0x7f2fb40d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fa9400640 1 --2- 172.21.15.42:0/4055222692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb406ff60 0x7f2fb40ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fb2800640 1 -- 172.21.15.42:0/4055222692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb406ff60 msgr2=0x7f2fb40ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fb2800640 1 --2- 172.21.15.42:0/4055222692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb406ff60 0x7f2fb40ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fb2800640 1 -- 172.21.15.42:0/4055222692 >> v2:172.21.15.63:3300/0 conn(0x7f2fb415e970 msgr2=0x7f2fb40d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fb2800640 1 --2- 172.21.15.42:0/4055222692 >> v2:172.21.15.63:3300/0 conn(0x7f2fb415e970 0x7f2fb40d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fb2800640 1 -- 172.21.15.42:0/4055222692 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb40d0870 con 0x7f2fb40cd040 2024-05-28T13:59:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fa9400640 1 --2- 172.21.15.42:0/4055222692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb406ff60 0x7f2fb40ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:24.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fb2800640 1 --2- 172.21.15.42:0/4055222692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb40cd040 0x7f2fb4162500 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40732d0 tx=0x7f2fac01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f24f75aeb44797f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:24.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fb1400640 1 -- 172.21.15.42:0/4055222692 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0a57b0 con 0x7f2fb40cd040 2024-05-28T13:59:24.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fb1400640 1 -- 172.21.15.42:0/4055222692 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fac0a5950 con 0x7f2fb40cd040 2024-05-28T13:59:24.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.133+0000 7f2fb1400640 1 -- 172.21.15.42:0/4055222692 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0a5d20 con 0x7f2fb40cd040 2024-05-28T13:59:24.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 -- 172.21.15.42:0/4055222692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb40cd040 msgr2=0x7f2fb4162500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:24.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 --2- 172.21.15.42:0/4055222692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb40cd040 0x7f2fb4162500 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40732d0 tx=0x7f2fac01c930 comp rx=0 tx=0).stop 2024-05-28T13:59:24.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 -- 172.21.15.42:0/4055222692 shutdown_connections 2024-05-28T13:59:24.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 --2- 172.21.15.42:0/4055222692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb40cd040 0x7f2fb4162500 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 --2- 172.21.15.42:0/4055222692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb406ff60 0x7f2fb40ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 --2- 172.21.15.42:0/4055222692 >> v2:172.21.15.63:3300/0 conn(0x7f2fb415e970 0x7f2fb40d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 -- 172.21.15.42:0/4055222692 >> 172.21.15.42:0/4055222692 conn(0x7f2fb40b3fa0 msgr2=0x7f2fb40b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:24.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 -- 172.21.15.42:0/4055222692 shutdown_connections 2024-05-28T13:59:24.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 -- 172.21.15.42:0/4055222692 wait complete. 2024-05-28T13:59:24.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 Processor -- start 2024-05-28T13:59:24.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 -- start start 2024-05-28T13:59:24.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fb406ff60 0x7f2fb4153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:24.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb40cd040 0x7f2fb41549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:24.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb415e970 0x7f2fb4157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:24.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb40704c0 con 0x7f2fb415e970 2024-05-28T13:59:24.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4070340 con 0x7f2fb406ff60 2024-05-28T13:59:24.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb8e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb409aa50 con 0x7f2fb40cd040 2024-05-28T13:59:24.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fb406ff60 0x7f2fb4153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:24.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb415e970 0x7f2fb4157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:24.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fb406ff60 0x7f2fb4153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:35172/0 (socket says 172.21.15.42:35172) 2024-05-28T13:59:24.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb1e00640 1 -- 172.21.15.42:0/3113016887 learned_addr learned my addr 172.21.15.42:0/3113016887 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:24.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fa9400640 1 --2- 172.21.15.42:0/3113016887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb40cd040 0x7f2fb41549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:24.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb2800640 1 -- 172.21.15.42:0/3113016887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb40cd040 msgr2=0x7f2fb41549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:24.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb2800640 1 --2- 172.21.15.42:0/3113016887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb40cd040 0x7f2fb41549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb2800640 1 -- 172.21.15.42:0/3113016887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fb406ff60 msgr2=0x7f2fb4153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:24.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb2800640 1 --2- 172.21.15.42:0/3113016887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fb406ff60 0x7f2fb4153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.137+0000 7f2fb2800640 1 -- 172.21.15.42:0/3113016887 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb4153c40 con 0x7f2fb415e970 2024-05-28T13:59:24.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.141+0000 7f2fb2800640 1 --2- 172.21.15.42:0/3113016887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb415e970 0x7f2fb4157a40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0ab000 tx=0x7f2fac0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:24.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.141+0000 7f2faaa00640 1 -- 172.21.15.42:0/3113016887 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0bc020 con 0x7f2fb415e970 2024-05-28T13:59:24.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.141+0000 7f2faaa00640 1 -- 172.21.15.42:0/3113016887 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fac0b8670 con 0x7f2fb415e970 2024-05-28T13:59:24.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.141+0000 7f2faaa00640 1 -- 172.21.15.42:0/3113016887 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0b8a40 con 0x7f2fb415e970 2024-05-28T13:59:24.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.141+0000 7f2fb8e00640 1 -- 172.21.15.42:0/3113016887 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb4153ee0 con 0x7f2fb415e970 2024-05-28T13:59:24.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.141+0000 7f2fb8e00640 1 -- 172.21.15.42:0/3113016887 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb415cbc0 con 0x7f2fb415e970 2024-05-28T13:59:24.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.141+0000 7f2fb8e00640 1 -- 172.21.15.42:0/3113016887 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb4068a90 con 0x7f2fb415e970 2024-05-28T13:59:24.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.141+0000 7f2faaa00640 1 -- 172.21.15.42:0/3113016887 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2fac0bf020 con 0x7f2fb415e970 2024-05-28T13:59:24.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.141+0000 7f2faaa00640 1 --2- 172.21.15.42:0/3113016887 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2f90068380 0x7f2f9006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:24.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.141+0000 7f2faaa00640 1 -- 172.21.15.42:0/3113016887 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f2fac121660 con 0x7f2fb415e970 2024-05-28T13:59:24.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.141+0000 7f2fb1e00640 1 --2- 172.21.15.42:0/3113016887 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2f90068380 0x7f2f9006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:24.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.145+0000 7f2fb1e00640 1 --2- 172.21.15.42:0/3113016887 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2f90068380 0x7f2f9006a840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009b00 tx=0x7f2fa4009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:24.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.145+0000 7f2faaa00640 1 -- 172.21.15.42:0/3113016887 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2fac0c6030 con 0x7f2fb415e970 2024-05-28T13:59:24.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.313+0000 7f2fb8e00640 1 -- 172.21.15.42:0/3113016887 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2fb40d1fd0 con 0x7f2fb415e970 2024-05-28T13:59:24.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.313+0000 7f2faaa00640 1 -- 172.21.15.42:0/3113016887 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) ==== 84+0+65 (secure 0 0 0) 0x7f2fac0165e0 con 0x7f2fb415e970 2024-05-28T13:59:24.300 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:24.300 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T13:59:24.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:24.300 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-05-28T13:59:24.300 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-05-28T13:59:24.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:24.300 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T13:59:24.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.317+0000 7f2fb8e00640 1 -- 172.21.15.42:0/3113016887 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2f90068380 msgr2=0x7f2f9006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:24.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.317+0000 7f2fb8e00640 1 --2- 172.21.15.42:0/3113016887 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2f90068380 0x7f2f9006a840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009b00 tx=0x7f2fa4009190 comp rx=0 tx=0).stop 2024-05-28T13:59:24.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.317+0000 7f2fb8e00640 1 -- 172.21.15.42:0/3113016887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb415e970 msgr2=0x7f2fb4157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:24.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.317+0000 7f2fb8e00640 1 --2- 172.21.15.42:0/3113016887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb415e970 0x7f2fb4157a40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0ab000 tx=0x7f2fac0196d0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.317+0000 7f2fb8e00640 1 -- 172.21.15.42:0/3113016887 shutdown_connections 2024-05-28T13:59:24.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.317+0000 7f2fb8e00640 1 --2- 172.21.15.42:0/3113016887 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2f90068380 0x7f2f9006a840 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.317+0000 7f2fb8e00640 1 --2- 172.21.15.42:0/3113016887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb415e970 0x7f2fb4157a40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.317+0000 7f2fb8e00640 1 --2- 172.21.15.42:0/3113016887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb40cd040 0x7f2fb41549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.317+0000 7f2fb8e00640 1 --2- 172.21.15.42:0/3113016887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fb406ff60 0x7f2fb4153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.317+0000 7f2fb8e00640 1 -- 172.21.15.42:0/3113016887 >> 172.21.15.42:0/3113016887 conn(0x7f2fb40b3fa0 msgr2=0x7f2fb40cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:24.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.317+0000 7f2fb8e00640 1 -- 172.21.15.42:0/3113016887 shutdown_connections 2024-05-28T13:59:24.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.317+0000 7f2fb8e00640 1 -- 172.21.15.42:0/3113016887 wait complete. 2024-05-28T13:59:24.321 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-28T13:59:24.321 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-28T13:59:24.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.481+0000 7fc89ec00640 1 Processor -- start 2024-05-28T13:59:24.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.481+0000 7fc89ec00640 1 -- start start 2024-05-28T13:59:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.481+0000 7fc89ec00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc89806ee80 0x7fc89806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.481+0000 7fc89ec00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc8980af360 0x7fc8980af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.481+0000 7fc89ec00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8981570a0 con 0x7fc8980d06f0 2024-05-28T13:59:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.481+0000 7fc89ec00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc89815cf00 con 0x7fc8980af360 2024-05-28T13:59:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.481+0000 7fc89ec00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc89815d1e0 con 0x7fc89806ee80 2024-05-28T13:59:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.481+0000 7fc88fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc8980af360 0x7fc8980af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.481+0000 7fc89cc00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc8980d06f0 0x7fc898160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:43862/0 (socket says 172.21.15.42:43862) 2024-05-28T13:59:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.481+0000 7fc89cc00640 1 -- 172.21.15.42:0/2112585810 learned_addr learned my addr 172.21.15.42:0/2112585810 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.481+0000 7fc897e00640 1 --2- 172.21.15.42:0/2112585810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc89806ee80 0x7fc89806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc88fe00640 1 -- 172.21.15.42:0/2112585810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc89806ee80 msgr2=0x7fc89806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc88fe00640 1 --2- 172.21.15.42:0/2112585810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc89806ee80 0x7fc89806dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc88fe00640 1 -- 172.21.15.42:0/2112585810 >> v1:172.21.15.42:6789/0 conn(0x7fc8980d06f0 legacy=0x7fc898160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc88fe00640 1 -- 172.21.15.42:0/2112585810 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89815d4c0 con 0x7fc8980af360 2024-05-28T13:59:24.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc88fe00640 1 --2- 172.21.15.42:0/2112585810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8980af360 0x7fc8980af7e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc8800097c0 tx=0x7fc88001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdaccf6ee2255b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:24.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc897400640 1 -- 172.21.15.42:0/2112585810 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8800b0070 con 0x7fc8980af360 2024-05-28T13:59:24.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc897400640 1 -- 172.21.15.42:0/2112585810 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8800ab600 con 0x7fc8980af360 2024-05-28T13:59:24.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc897400640 1 -- 172.21.15.42:0/2112585810 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8800a4e60 con 0x7fc8980af360 2024-05-28T13:59:24.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc89ec00640 1 -- 172.21.15.42:0/2112585810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8980af360 msgr2=0x7fc8980af7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:24.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc89ec00640 1 --2- 172.21.15.42:0/2112585810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8980af360 0x7fc8980af7e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc8800097c0 tx=0x7fc88001c380 comp rx=0 tx=0).stop 2024-05-28T13:59:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc89ec00640 1 -- 172.21.15.42:0/2112585810 shutdown_connections 2024-05-28T13:59:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc89ec00640 1 --2- 172.21.15.42:0/2112585810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8980af360 0x7fc8980af7e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc89ec00640 1 --2- 172.21.15.42:0/2112585810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc89806ee80 0x7fc89806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc89ec00640 1 -- 172.21.15.42:0/2112585810 >> 172.21.15.42:0/2112585810 conn(0x7fc89806bdf0 msgr2=0x7fc89806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc89ec00640 1 -- 172.21.15.42:0/2112585810 shutdown_connections 2024-05-28T13:59:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.485+0000 7fc89ec00640 1 -- 172.21.15.42:0/2112585810 wait complete. 2024-05-28T13:59:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89ec00640 1 Processor -- start 2024-05-28T13:59:24.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89ec00640 1 -- start start 2024-05-28T13:59:24.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89ec00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc89806ee80 0x7fc898151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:24.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89ec00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8980af360 0x7fc898152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:24.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89ec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8980d06f0 0x7fc898153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:24.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89ec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc8980d0f10 con 0x7fc8980d06f0 2024-05-28T13:59:24.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89ec00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc898160fa0 con 0x7fc89806ee80 2024-05-28T13:59:24.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89ec00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc898073a90 con 0x7fc8980af360 2024-05-28T13:59:24.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8980d06f0 0x7fc898153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:24.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8980d06f0 0x7fc898153040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44132/0 (socket says 172.21.15.42:44132) 2024-05-28T13:59:24.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89cc00640 1 -- 172.21.15.42:0/2878633101 learned_addr learned my addr 172.21.15.42:0/2878633101 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:24.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc897e00640 1 --2- 172.21.15.42:0/2878633101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc89806ee80 0x7fc898151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:24.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc88fe00640 1 --2- 172.21.15.42:0/2878633101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8980af360 0x7fc898152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:24.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89cc00640 1 -- 172.21.15.42:0/2878633101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8980af360 msgr2=0x7fc898152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:24.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89cc00640 1 --2- 172.21.15.42:0/2878633101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8980af360 0x7fc898152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89cc00640 1 -- 172.21.15.42:0/2878633101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc89806ee80 msgr2=0x7fc898151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:24.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89cc00640 1 --2- 172.21.15.42:0/2878633101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc89806ee80 0x7fc898151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:24.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89cc00640 1 -- 172.21.15.42:0/2878633101 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc898151cc0 con 0x7fc8980d06f0 2024-05-28T13:59:24.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89cc00640 1 --2- 172.21.15.42:0/2878633101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8980d06f0 0x7fc898153040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc89000cae0 tx=0x7fc89000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:24.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc894c00640 1 -- 172.21.15.42:0/2878633101 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890017070 con 0x7fc8980d06f0 2024-05-28T13:59:24.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc894c00640 1 -- 172.21.15.42:0/2878633101 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc89000ac40 con 0x7fc8980d06f0 2024-05-28T13:59:24.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc89ec00640 1 -- 172.21.15.42:0/2878633101 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc898072e80 con 0x7fc8980d06f0 2024-05-28T13:59:24.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.489+0000 7fc894c00640 1 -- 172.21.15.42:0/2878633101 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890012400 con 0x7fc8980d06f0 2024-05-28T13:59:24.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.493+0000 7fc89ec00640 1 -- 172.21.15.42:0/2878633101 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc898073440 con 0x7fc8980d06f0 2024-05-28T13:59:24.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.493+0000 7fc89ec00640 1 -- 172.21.15.42:0/2878633101 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc898068a90 con 0x7fc8980d06f0 2024-05-28T13:59:24.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.493+0000 7fc894c00640 1 -- 172.21.15.42:0/2878633101 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc89001a070 con 0x7fc8980d06f0 2024-05-28T13:59:24.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.493+0000 7fc894c00640 1 --2- 172.21.15.42:0/2878633101 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc874068380 0x7fc87406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:24.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.493+0000 7fc894c00640 1 -- 172.21.15.42:0/2878633101 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fc890086390 con 0x7fc8980d06f0 2024-05-28T13:59:24.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.493+0000 7fc897e00640 1 --2- 172.21.15.42:0/2878633101 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc874068380 0x7fc87406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:24.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.493+0000 7fc897e00640 1 --2- 172.21.15.42:0/2878633101 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc874068380 0x7fc87406a840 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc8981fb340 tx=0x7fc888009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:24.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.497+0000 7fc894c00640 1 -- 172.21.15.42:0/2878633101 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc8900565b0 con 0x7fc8980d06f0 2024-05-28T13:59:24.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:24.661+0000 7fc89ec00640 1 -- 172.21.15.42:0/2878633101 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc8980b5160 con 0x7fc8980d06f0 2024-05-28T13:59:25.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.425+0000 7fc894c00640 1 -- 172.21.15.42:0/2878633101 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) ==== 138+0+0 (secure 0 0 0) 0x7fc89005a8b0 con 0x7fc8980d06f0 2024-05-28T13:59:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-05-28T13:59:25.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.425+0000 7fc89ec00640 1 -- 172.21.15.42:0/2878633101 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc874068380 msgr2=0x7fc87406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:25.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.425+0000 7fc89ec00640 1 --2- 172.21.15.42:0/2878633101 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc874068380 0x7fc87406a840 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc8981fb340 tx=0x7fc888009190 comp rx=0 tx=0).stop 2024-05-28T13:59:25.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.425+0000 7fc89ec00640 1 -- 172.21.15.42:0/2878633101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8980d06f0 msgr2=0x7fc898153040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:25.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.425+0000 7fc89ec00640 1 --2- 172.21.15.42:0/2878633101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8980d06f0 0x7fc898153040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc89000cae0 tx=0x7fc89000a060 comp rx=0 tx=0).stop 2024-05-28T13:59:25.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.425+0000 7fc89ec00640 1 -- 172.21.15.42:0/2878633101 shutdown_connections 2024-05-28T13:59:25.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.425+0000 7fc89ec00640 1 --2- 172.21.15.42:0/2878633101 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc874068380 0x7fc87406a840 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:25.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.425+0000 7fc89ec00640 1 --2- 172.21.15.42:0/2878633101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8980d06f0 0x7fc898153040 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:25.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.425+0000 7fc89ec00640 1 --2- 172.21.15.42:0/2878633101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8980af360 0x7fc898152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:25.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.425+0000 7fc89ec00640 1 --2- 172.21.15.42:0/2878633101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc89806ee80 0x7fc898151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.425+0000 7fc89ec00640 1 -- 172.21.15.42:0/2878633101 >> 172.21.15.42:0/2878633101 conn(0x7fc89806bdf0 msgr2=0x7fc89806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.425+0000 7fc89ec00640 1 -- 172.21.15.42:0/2878633101 shutdown_connections 2024-05-28T13:59:25.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.425+0000 7fc89ec00640 1 -- 172.21.15.42:0/2878633101 wait complete. 2024-05-28T13:59:25.439 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-28T13:59:25.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.601+0000 7fac1aa00640 1 Processor -- start 2024-05-28T13:59:25.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.601+0000 7fac1aa00640 1 -- start start 2024-05-28T13:59:25.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.601+0000 7fac1aa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fac14071de0 0x7fac140721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:25.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.601+0000 7fac1aa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fac140ce3b0 0x7fac140ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.601+0000 7fac1aa00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac14072700 con 0x7fac14141d10 2024-05-28T13:59:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.601+0000 7fac1aa00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fac14160230 con 0x7fac140ce3b0 2024-05-28T13:59:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.601+0000 7fac1aa00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fac14160510 con 0x7fac14071de0 2024-05-28T13:59:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.605+0000 7fac0b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fac140ce3b0 0x7fac140ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.605+0000 7fac18a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fac14141d10 0x7fac14142160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:43870/0 (socket says 172.21.15.42:43870) 2024-05-28T13:59:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.605+0000 7fac18a00640 1 -- 172.21.15.42:0/76055255 learned_addr learned my addr 172.21.15.42:0/76055255 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.605+0000 7fac13e00640 1 --2- 172.21.15.42:0/76055255 >> v2:172.21.15.63:3300/0 conn(0x7fac14071de0 0x7fac140721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.605+0000 7fac0b400640 1 -- 172.21.15.42:0/76055255 >> v2:172.21.15.63:3300/0 conn(0x7fac14071de0 msgr2=0x7fac140721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.605+0000 7fac0b400640 1 --2- 172.21.15.42:0/76055255 >> v2:172.21.15.63:3300/0 conn(0x7fac14071de0 0x7fac140721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.605+0000 7fac0b400640 1 -- 172.21.15.42:0/76055255 >> v1:172.21.15.42:6789/0 conn(0x7fac14141d10 legacy=0x7fac14142160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.605+0000 7fac0b400640 1 -- 172.21.15.42:0/76055255 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac141607f0 con 0x7fac140ce3b0 2024-05-28T13:59:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.605+0000 7fac13400640 1 -- 172.21.15.42:0/76055255 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2508323866 0 0) 0x7fac14072700 con 0x7fac14141d10 2024-05-28T13:59:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.605+0000 7fac0b400640 1 --2- 172.21.15.42:0/76055255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac140ce3b0 0x7fac140ce810 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fac1406cbf0 tx=0x7fabfc01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafcc6c846cc8a64 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.605+0000 7fac13400640 1 -- 172.21.15.42:0/76055255 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0b0070 con 0x7fac140ce3b0 2024-05-28T13:59:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.605+0000 7fac13400640 1 -- 172.21.15.42:0/76055255 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabfc0ab600 con 0x7fac140ce3b0 2024-05-28T13:59:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.605+0000 7fac13400640 1 -- 172.21.15.42:0/76055255 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0ab9d0 con 0x7fac140ce3b0 2024-05-28T13:59:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 -- 172.21.15.42:0/76055255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac140ce3b0 msgr2=0x7fac140ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 --2- 172.21.15.42:0/76055255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac140ce3b0 0x7fac140ce810 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fac1406cbf0 tx=0x7fabfc01c360 comp rx=0 tx=0).stop 2024-05-28T13:59:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 -- 172.21.15.42:0/76055255 shutdown_connections 2024-05-28T13:59:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 --2- 172.21.15.42:0/76055255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac140ce3b0 0x7fac140ce810 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 --2- 172.21.15.42:0/76055255 >> v2:172.21.15.63:3300/0 conn(0x7fac14071de0 0x7fac140721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 -- 172.21.15.42:0/76055255 >> 172.21.15.42:0/76055255 conn(0x7fac140b21f0 msgr2=0x7fac140b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 -- 172.21.15.42:0/76055255 shutdown_connections 2024-05-28T13:59:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 -- 172.21.15.42:0/76055255 wait complete. 2024-05-28T13:59:25.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 Processor -- start 2024-05-28T13:59:25.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 -- start start 2024-05-28T13:59:25.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac14071de0 0x7fac1415f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:25.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac140ce3b0 0x7fac14155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:25.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac14141d10 0x7fac1415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:25.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fac14169e30 con 0x7fac14071de0 2024-05-28T13:59:25.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fac14169cb0 con 0x7fac14141d10 2024-05-28T13:59:25.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac1aa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fac14169fb0 con 0x7fac140ce3b0 2024-05-28T13:59:25.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac13e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac14071de0 0x7fac1415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:25.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac13e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac14071de0 0x7fac1415f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44154/0 (socket says 172.21.15.42:44154) 2024-05-28T13:59:25.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac13e00640 1 -- 172.21.15.42:0/930272490 learned_addr learned my addr 172.21.15.42:0/930272490 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:25.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac13e00640 1 -- 172.21.15.42:0/930272490 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac140ce3b0 msgr2=0x7fac14155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:25.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac13e00640 1 --2- 172.21.15.42:0/930272490 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac140ce3b0 0x7fac14155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:25.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac13e00640 1 -- 172.21.15.42:0/930272490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac14141d10 msgr2=0x7fac1415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:25.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac13e00640 1 --2- 172.21.15.42:0/930272490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac14141d10 0x7fac1415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:25.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac13e00640 1 -- 172.21.15.42:0/930272490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac140683a0 con 0x7fac14071de0 2024-05-28T13:59:25.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.609+0000 7fac13e00640 1 --2- 172.21.15.42:0/930272490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac14071de0 0x7fac1415f790 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fac0400e870 tx=0x7fac0400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:25.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.613+0000 7fac10c00640 1 -- 172.21.15.42:0/930272490 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04017070 con 0x7fac14071de0 2024-05-28T13:59:25.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.613+0000 7fac10c00640 1 -- 172.21.15.42:0/930272490 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac0400c940 con 0x7fac14071de0 2024-05-28T13:59:25.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.613+0000 7fac10c00640 1 -- 172.21.15.42:0/930272490 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0400ccf0 con 0x7fac14071de0 2024-05-28T13:59:25.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.613+0000 7fac1aa00640 1 -- 172.21.15.42:0/930272490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac14139870 con 0x7fac14071de0 2024-05-28T13:59:25.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.613+0000 7fac1aa00640 1 -- 172.21.15.42:0/930272490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac14134b40 con 0x7fac14071de0 2024-05-28T13:59:25.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.613+0000 7fac1aa00640 1 -- 172.21.15.42:0/930272490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac140738b0 con 0x7fac14071de0 2024-05-28T13:59:25.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.613+0000 7fac10c00640 1 -- 172.21.15.42:0/930272490 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fac0401a070 con 0x7fac14071de0 2024-05-28T13:59:25.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.617+0000 7fac10c00640 1 --2- 172.21.15.42:0/930272490 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fabf0068380 0x7fabf006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:25.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.617+0000 7fac10c00640 1 -- 172.21.15.42:0/930272490 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5227+0+0 (secure 0 0 0) 0x7fac04086f50 con 0x7fac14071de0 2024-05-28T13:59:25.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.617+0000 7fac0b400640 1 --2- 172.21.15.42:0/930272490 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fabf0068380 0x7fabf006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:25.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.617+0000 7fac0b400640 1 --2- 172.21.15.42:0/930272490 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fabf0068380 0x7fabf006a840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fabfc0ad000 tx=0x7fabfc004740 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:25.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.621+0000 7fac10c00640 1 -- 172.21.15.42:0/930272490 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fac040576d0 con 0x7fac14071de0 2024-05-28T13:59:25.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:25.785+0000 7fac1aa00640 1 -- 172.21.15.42:0/930272490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fac1415fe70 con 0x7fac14071de0 2024-05-28T13:59:26.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.429+0000 7fac10c00640 1 -- 172.21.15.42:0/930272490 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) ==== 170+0+0 (secure 0 0 0) 0x7fac04056fd0 con 0x7fac14071de0 2024-05-28T13:59:26.413 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-05-28T13:59:26.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.429+0000 7fac1aa00640 1 -- 172.21.15.42:0/930272490 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fabf0068380 msgr2=0x7fabf006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:26.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.429+0000 7fac1aa00640 1 --2- 172.21.15.42:0/930272490 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fabf0068380 0x7fabf006a840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fabfc0ad000 tx=0x7fabfc004740 comp rx=0 tx=0).stop 2024-05-28T13:59:26.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.429+0000 7fac1aa00640 1 -- 172.21.15.42:0/930272490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac14071de0 msgr2=0x7fac1415f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:26.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.429+0000 7fac1aa00640 1 --2- 172.21.15.42:0/930272490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac14071de0 0x7fac1415f790 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fac0400e870 tx=0x7fac0400ec70 comp rx=0 tx=0).stop 2024-05-28T13:59:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.429+0000 7fac1aa00640 1 -- 172.21.15.42:0/930272490 shutdown_connections 2024-05-28T13:59:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.429+0000 7fac1aa00640 1 --2- 172.21.15.42:0/930272490 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fabf0068380 0x7fabf006a840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.429+0000 7fac1aa00640 1 --2- 172.21.15.42:0/930272490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac14141d10 0x7fac1415ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.429+0000 7fac1aa00640 1 --2- 172.21.15.42:0/930272490 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac140ce3b0 0x7fac14155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.429+0000 7fac1aa00640 1 --2- 172.21.15.42:0/930272490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac14071de0 0x7fac1415f790 secure :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fac0400e870 tx=0x7fac0400ec70 comp rx=0 tx=0).stop 2024-05-28T13:59:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.429+0000 7fac1aa00640 1 -- 172.21.15.42:0/930272490 >> 172.21.15.42:0/930272490 conn(0x7fac140b21f0 msgr2=0x7fac1406e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.433+0000 7fac1aa00640 1 -- 172.21.15.42:0/930272490 shutdown_connections 2024-05-28T13:59:26.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.433+0000 7fac1aa00640 1 -- 172.21.15.42:0/930272490 wait complete. 2024-05-28T13:59:26.459 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-28T13:59:26.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a07600640 1 Processor -- start 2024-05-28T13:59:26.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a07600640 1 -- start start 2024-05-28T13:59:26.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a07600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8a00071de0 0x7f8a000721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:26.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a07600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8a000ce3b0 0x7f8a000ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:26.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a07600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a00141d10 0x7f8a001421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:26.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a07600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8a00072700 con 0x7f8a00071de0 2024-05-28T13:59:26.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a07600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8a00160230 con 0x7f8a000ce3b0 2024-05-28T13:59:26.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a07600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8a00160510 con 0x7f8a00141d10 2024-05-28T13:59:26.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a04c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8a00071de0 0x7f8a000721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:26.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a04c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8a00071de0 0x7f8a000721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44164/0 (socket says 172.21.15.42:44164) 2024-05-28T13:59:26.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a04c00640 1 -- 172.21.15.42:0/3274808047 learned_addr learned my addr 172.21.15.42:0/3274808047 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:26.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f89ffe00640 1 --2- 172.21.15.42:0/3274808047 >> v2:172.21.15.63:3300/0 conn(0x7f8a000ce3b0 0x7f8a000ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:26.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a05600640 1 --2- 172.21.15.42:0/3274808047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a00141d10 0x7f8a001421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:26.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a04c00640 1 -- 172.21.15.42:0/3274808047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a00141d10 msgr2=0x7f8a001421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:26.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a04c00640 1 --2- 172.21.15.42:0/3274808047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a00141d10 0x7f8a001421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:26.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a04c00640 1 -- 172.21.15.42:0/3274808047 >> v2:172.21.15.63:3300/0 conn(0x7f8a000ce3b0 msgr2=0x7f8a000ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:26.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a04c00640 1 --2- 172.21.15.42:0/3274808047 >> v2:172.21.15.63:3300/0 conn(0x7f8a000ce3b0 0x7f8a000ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:26.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.621+0000 7f8a04c00640 1 -- 172.21.15.42:0/3274808047 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a001607f0 con 0x7f8a00071de0 2024-05-28T13:59:26.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.625+0000 7f8a04c00640 1 --2- 172.21.15.42:0/3274808047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a00071de0 0x7f8a000721c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8a00073280 tx=0x7f89f401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17fe14ddf451fdf server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:26.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.625+0000 7f89ff400640 1 -- 172.21.15.42:0/3274808047 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f40b0070 con 0x7f8a00071de0 2024-05-28T13:59:26.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.625+0000 7f89ff400640 1 -- 172.21.15.42:0/3274808047 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f40a4e50 con 0x7f8a00071de0 2024-05-28T13:59:26.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.625+0000 7f89ff400640 1 -- 172.21.15.42:0/3274808047 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f40ab820 con 0x7f8a00071de0 2024-05-28T13:59:26.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.625+0000 7f8a07600640 1 -- 172.21.15.42:0/3274808047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a00071de0 msgr2=0x7f8a000721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:26.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.625+0000 7f8a07600640 1 --2- 172.21.15.42:0/3274808047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a00071de0 0x7f8a000721c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8a00073280 tx=0x7f89f401c400 comp rx=0 tx=0).stop 2024-05-28T13:59:26.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.625+0000 7f8a07600640 1 -- 172.21.15.42:0/3274808047 shutdown_connections 2024-05-28T13:59:26.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.625+0000 7f8a07600640 1 --2- 172.21.15.42:0/3274808047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a00141d10 0x7f8a001421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:26.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.625+0000 7f8a07600640 1 --2- 172.21.15.42:0/3274808047 >> v2:172.21.15.63:3300/0 conn(0x7f8a000ce3b0 0x7f8a000ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:26.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.625+0000 7f8a07600640 1 --2- 172.21.15.42:0/3274808047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a00071de0 0x7f8a000721c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:26.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.625+0000 7f8a07600640 1 -- 172.21.15.42:0/3274808047 >> 172.21.15.42:0/3274808047 conn(0x7f8a000b21f0 msgr2=0x7f8a000b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:26.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a07600640 1 -- 172.21.15.42:0/3274808047 shutdown_connections 2024-05-28T13:59:26.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a07600640 1 -- 172.21.15.42:0/3274808047 wait complete. 2024-05-28T13:59:26.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a07600640 1 Processor -- start 2024-05-28T13:59:26.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a07600640 1 -- start start 2024-05-28T13:59:26.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a07600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a00071de0 0x7f8a001fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:26.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a07600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a000ce3b0 0x7f8a0015ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:26.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a07600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a00141d10 0x7f8a0013c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:26.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a07600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00140520 con 0x7f8a00071de0 2024-05-28T13:59:26.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a07600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0013f8c0 con 0x7f8a00141d10 2024-05-28T13:59:26.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a07600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8a001406a0 con 0x7f8a000ce3b0 2024-05-28T13:59:26.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a04c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a00071de0 0x7f8a001fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:26.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a04c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a00071de0 0x7f8a001fbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44166/0 (socket says 172.21.15.42:44166) 2024-05-28T13:59:26.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a04c00640 1 -- 172.21.15.42:0/935186057 learned_addr learned my addr 172.21.15.42:0/935186057 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:26.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a05600640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a00141d10 0x7f8a0013c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:26.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f89ffe00640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a000ce3b0 0x7f8a0015ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:26.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a04c00640 1 -- 172.21.15.42:0/935186057 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a000ce3b0 msgr2=0x7f8a0015ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:26.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a04c00640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a000ce3b0 0x7f8a0015ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:26.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a04c00640 1 -- 172.21.15.42:0/935186057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a00141d10 msgr2=0x7f8a0013c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:26.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a04c00640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a00141d10 0x7f8a0013c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:26.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a04c00640 1 -- 172.21.15.42:0/935186057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0015efa0 con 0x7f8a00071de0 2024-05-28T13:59:26.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a05600640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a00141d10 0x7f8a0013c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:26.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a04c00640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a00071de0 0x7f8a001fbc60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f89f40097c0 tx=0x7f89f40191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:26.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f89fcc00640 1 -- 172.21.15.42:0/935186057 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f40b0040 con 0x7f8a00071de0 2024-05-28T13:59:26.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f89fcc00640 1 -- 172.21.15.42:0/935186057 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f40b9070 con 0x7f8a00071de0 2024-05-28T13:59:26.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f89fcc00640 1 -- 172.21.15.42:0/935186057 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f40b4440 con 0x7f8a00071de0 2024-05-28T13:59:26.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.629+0000 7f8a07600640 1 -- 172.21.15.42:0/935186057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0013e4d0 con 0x7f8a00071de0 2024-05-28T13:59:26.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.633+0000 7f8a07600640 1 -- 172.21.15.42:0/935186057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0013ea90 con 0x7f8a00071de0 2024-05-28T13:59:26.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.633+0000 7f8a07600640 1 -- 172.21.15.42:0/935186057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a000738b0 con 0x7f8a00071de0 2024-05-28T13:59:26.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.633+0000 7f89fcc00640 1 -- 172.21.15.42:0/935186057 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f89f40b45e0 con 0x7f8a00071de0 2024-05-28T13:59:26.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.633+0000 7f89fcc00640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89e0068380 0x7f89e006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:26.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.633+0000 7f89fcc00640 1 -- 172.21.15.42:0/935186057 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5609+0+0 (secure 0 0 0) 0x7f89f4120ff0 con 0x7f8a00071de0 2024-05-28T13:59:26.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.633+0000 7f89ffe00640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89e0068380 0x7f89e006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:26.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.633+0000 7f89ffe00640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89e0068380 0x7f89e006a840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8a00156600 tx=0x7f89ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:26.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.637+0000 7f89fcc00640 1 -- 172.21.15.42:0/935186057 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f89f4016110 con 0x7f8a00071de0 2024-05-28T13:59:26.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:26.841+0000 7f8a07600640 1 -- 172.21.15.42:0/935186057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8a0004fdd0 con 0x7f8a00071de0 2024-05-28T13:59:27.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.441+0000 7f89fcc00640 1 -- 172.21.15.42:0/935186057 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f89f40f1630 con 0x7f8a00071de0 2024-05-28T13:59:27.429 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 4 and data pool 5 2024-05-28T13:59:27.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.445+0000 7f8a07600640 1 -- 172.21.15.42:0/935186057 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89e0068380 msgr2=0x7f89e006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:27.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.445+0000 7f8a07600640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89e0068380 0x7f89e006a840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8a00156600 tx=0x7f89ec008040 comp rx=0 tx=0).stop 2024-05-28T13:59:27.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.445+0000 7f8a07600640 1 -- 172.21.15.42:0/935186057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a00071de0 msgr2=0x7f8a001fbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.445+0000 7f8a07600640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a00071de0 0x7f8a001fbc60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f89f40097c0 tx=0x7f89f40191a0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.449+0000 7f8a07600640 1 -- 172.21.15.42:0/935186057 shutdown_connections 2024-05-28T13:59:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.449+0000 7f8a07600640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89e0068380 0x7f89e006a840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.449+0000 7f8a07600640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a00141d10 0x7f8a0013c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.449+0000 7f8a07600640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a000ce3b0 0x7f8a0015ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.449+0000 7f8a07600640 1 --2- 172.21.15.42:0/935186057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a00071de0 0x7f8a001fbc60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.449+0000 7f8a07600640 1 -- 172.21.15.42:0/935186057 >> 172.21.15.42:0/935186057 conn(0x7f8a000b21f0 msgr2=0x7f8a000b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:27.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.449+0000 7f8a07600640 1 -- 172.21.15.42:0/935186057 shutdown_connections 2024-05-28T13:59:27.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.449+0000 7f8a07600640 1 -- 172.21.15.42:0/935186057 wait complete. 2024-05-28T13:59:27.481 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:59:27.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff7200640 1 Processor -- start 2024-05-28T13:59:27.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff7200640 1 -- start start 2024-05-28T13:59:27.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff7200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faff809b160 0x7faff809ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:27.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff7200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faff8070840 0x7faff80d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:27.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff7200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff815ffe0 con 0x7faff815fc00 2024-05-28T13:59:27.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff7200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7faff80d1180 con 0x7faff8070840 2024-05-28T13:59:27.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff7200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faff8070de0 con 0x7faff809b160 2024-05-28T13:59:27.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff6800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faff8070840 0x7faff80d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:27.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff6800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faff8070840 0x7faff80d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41586/0 (socket says 172.21.15.42:41586) 2024-05-28T13:59:27.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff6800640 1 -- 172.21.15.42:0/2264173649 learned_addr learned my addr 172.21.15.42:0/2264173649 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:27.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff5400640 1 --2- 172.21.15.42:0/2264173649 >> v2:172.21.15.63:3300/0 conn(0x7faff809b160 0x7faff809ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:27.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff6800640 1 -- 172.21.15.42:0/2264173649 >> v2:172.21.15.63:3300/0 conn(0x7faff809b160 msgr2=0x7faff809ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:27.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff6800640 1 --2- 172.21.15.42:0/2264173649 >> v2:172.21.15.63:3300/0 conn(0x7faff809b160 0x7faff809ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff6800640 1 -- 172.21.15.42:0/2264173649 >> v1:172.21.15.63:6789/0 conn(0x7faff815fc00 legacy=0x7faff809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:27.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.641+0000 7faff6800640 1 -- 172.21.15.42:0/2264173649 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff80713c0 con 0x7faff8070840 2024-05-28T13:59:27.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.645+0000 7faff6800640 1 --2- 172.21.15.42:0/2264173649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff8070840 0x7faff80d0c40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faff80c5be0 tx=0x7faff001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53416d868c2dfad server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:27.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.645+0000 7faff4a00640 1 -- 172.21.15.42:0/2264173649 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff00a56c0 con 0x7faff8070840 2024-05-28T13:59:27.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.645+0000 7faff4a00640 1 -- 172.21.15.42:0/2264173649 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff00a5860 con 0x7faff8070840 2024-05-28T13:59:27.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.645+0000 7faff4a00640 1 -- 172.21.15.42:0/2264173649 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff00a5c30 con 0x7faff8070840 2024-05-28T13:59:27.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.645+0000 7faff7200640 1 -- 172.21.15.42:0/2264173649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff8070840 msgr2=0x7faff80d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:27.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.645+0000 7faff7200640 1 --2- 172.21.15.42:0/2264173649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff8070840 0x7faff80d0c40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faff80c5be0 tx=0x7faff001c9f0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.645+0000 7faff7200640 1 -- 172.21.15.42:0/2264173649 shutdown_connections 2024-05-28T13:59:27.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.645+0000 7faff7200640 1 --2- 172.21.15.42:0/2264173649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff8070840 0x7faff80d0c40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.645+0000 7faff7200640 1 --2- 172.21.15.42:0/2264173649 >> v2:172.21.15.63:3300/0 conn(0x7faff809b160 0x7faff809ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.645+0000 7faff7200640 1 -- 172.21.15.42:0/2264173649 >> 172.21.15.42:0/2264173649 conn(0x7faff80b3d70 msgr2=0x7faff80b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:27.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.645+0000 7faff7200640 1 -- 172.21.15.42:0/2264173649 shutdown_connections 2024-05-28T13:59:27.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.645+0000 7faff7200640 1 -- 172.21.15.42:0/2264173649 wait complete. 2024-05-28T13:59:27.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.645+0000 7faff7200640 1 Processor -- start 2024-05-28T13:59:27.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.645+0000 7faff7200640 1 -- start start 2024-05-28T13:59:27.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff8070840 0x7faff8158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:27.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff7200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff809b160 0x7faff80b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:27.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff815fc00 0x7faff80b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:27.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff7200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faff80cd840 con 0x7faff8070840 2024-05-28T13:59:27.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff7200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faff80cd6c0 con 0x7faff815fc00 2024-05-28T13:59:27.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff7200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faff80cd9c0 con 0x7faff809b160 2024-05-28T13:59:27.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff8070840 0x7faff8158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:27.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff8070840 0x7faff8158a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41596/0 (socket says 172.21.15.42:41596) 2024-05-28T13:59:27.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff5e00640 1 -- 172.21.15.42:0/529334383 learned_addr learned my addr 172.21.15.42:0/529334383 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:27.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff6800640 1 --2- 172.21.15.42:0/529334383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff815fc00 0x7faff80b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:27.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff5400640 1 --2- 172.21.15.42:0/529334383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff809b160 0x7faff80b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:27.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff5e00640 1 -- 172.21.15.42:0/529334383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff809b160 msgr2=0x7faff80b7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:27.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff5e00640 1 --2- 172.21.15.42:0/529334383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff809b160 0x7faff80b7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff5e00640 1 -- 172.21.15.42:0/529334383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff815fc00 msgr2=0x7faff80b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:27.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff5e00640 1 --2- 172.21.15.42:0/529334383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff815fc00 0x7faff80b6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff5e00640 1 -- 172.21.15.42:0/529334383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff80b4ea0 con 0x7faff8070840 2024-05-28T13:59:27.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff5e00640 1 --2- 172.21.15.42:0/529334383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff8070840 0x7faff8158a80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fafe800c450 tx=0x7fafe800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:27.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7fafee000640 1 -- 172.21.15.42:0/529334383 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8016020 con 0x7faff8070840 2024-05-28T13:59:27.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7fafee000640 1 -- 172.21.15.42:0/529334383 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe80052d0 con 0x7faff8070840 2024-05-28T13:59:27.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7fafee000640 1 -- 172.21.15.42:0/529334383 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8007260 con 0x7faff8070840 2024-05-28T13:59:27.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff7200640 1 -- 172.21.15.42:0/529334383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff80b5180 con 0x7faff8070840 2024-05-28T13:59:27.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff7200640 1 -- 172.21.15.42:0/529334383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff80b5740 con 0x7faff8070840 2024-05-28T13:59:27.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.649+0000 7faff7200640 1 -- 172.21.15.42:0/529334383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff804fcf0 con 0x7faff8070840 2024-05-28T13:59:27.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.653+0000 7fafee000640 1 -- 172.21.15.42:0/529334383 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fafe8007400 con 0x7faff8070840 2024-05-28T13:59:27.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.653+0000 7fafee000640 1 --2- 172.21.15.42:0/529334383 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fafd8068380 0x7fafd806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:27.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.653+0000 7fafee000640 1 -- 172.21.15.42:0/529334383 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fafe80862f0 con 0x7faff8070840 2024-05-28T13:59:27.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.653+0000 7faff5400640 1 --2- 172.21.15.42:0/529334383 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fafd8068380 0x7fafd806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:27.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.653+0000 7faff5400640 1 --2- 172.21.15.42:0/529334383 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fafd8068380 0x7fafd806a840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faff815c010 tx=0x7fafe4009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:27.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.657+0000 7fafee000640 1 -- 172.21.15.42:0/529334383 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fafe8020020 con 0x7faff8070840 2024-05-28T13:59:27.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.825+0000 7faff7200640 1 -- 172.21.15.42:0/529334383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faff81673a0 con 0x7faff8070840 2024-05-28T13:59:27.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.825+0000 7fafee000640 1 -- 172.21.15.42:0/529334383 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16622 (secure 0 0 0) 0x7fafe8005480 con 0x7faff8070840 2024-05-28T13:59:27.810 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:27.810 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":33,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:59:26.845129+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:59:24.673711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-28T13:59:25.795430+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:59:27.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.825+0000 7faff7200640 1 -- 172.21.15.42:0/529334383 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fafd8068380 msgr2=0x7fafd806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:27.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.825+0000 7faff7200640 1 --2- 172.21.15.42:0/529334383 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fafd8068380 0x7fafd806a840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faff815c010 tx=0x7fafe4009190 comp rx=0 tx=0).stop 2024-05-28T13:59:27.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.825+0000 7faff7200640 1 -- 172.21.15.42:0/529334383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff8070840 msgr2=0x7faff8158a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:27.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.825+0000 7faff7200640 1 --2- 172.21.15.42:0/529334383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff8070840 0x7faff8158a80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fafe800c450 tx=0x7fafe800c910 comp rx=0 tx=0).stop 2024-05-28T13:59:27.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.825+0000 7faff7200640 1 -- 172.21.15.42:0/529334383 shutdown_connections 2024-05-28T13:59:27.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.825+0000 7faff7200640 1 --2- 172.21.15.42:0/529334383 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fafd8068380 0x7fafd806a840 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.825+0000 7faff7200640 1 --2- 172.21.15.42:0/529334383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff815fc00 0x7faff80b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.825+0000 7faff7200640 1 --2- 172.21.15.42:0/529334383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff809b160 0x7faff80b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.825+0000 7faff7200640 1 --2- 172.21.15.42:0/529334383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff8070840 0x7faff8158a80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.825+0000 7faff7200640 1 -- 172.21.15.42:0/529334383 >> 172.21.15.42:0/529334383 conn(0x7faff80b3d70 msgr2=0x7faff80b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:27.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.825+0000 7faff7200640 1 -- 172.21.15.42:0/529334383 shutdown_connections 2024-05-28T13:59:27.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.825+0000 7faff7200640 1 -- 172.21.15.42:0/529334383 wait complete. 2024-05-28T13:59:27.831 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:59:27.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d6cc00640 1 Processor -- start 2024-05-28T13:59:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d6cc00640 1 -- start start 2024-05-28T13:59:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d6cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d68071620 0x7f6d6806d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d6cc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6d6806da50 0x7f6d6815dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d6cc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d68070860 con 0x7f6d680ce570 2024-05-28T13:59:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d6cc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6d68070b40 con 0x7f6d6806da50 2024-05-28T13:59:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d6cc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6d68070e20 con 0x7f6d68071620 2024-05-28T13:59:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d67e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6d6806da50 0x7f6d6815dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d67e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6d6806da50 0x7f6d6815dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41602/0 (socket says 172.21.15.42:41602) 2024-05-28T13:59:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d67e00640 1 -- 172.21.15.42:0/2519523685 learned_addr learned my addr 172.21.15.42:0/2519523685 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d5ea00640 1 --2- 172.21.15.42:0/2519523685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d68071620 0x7f6d6806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d67e00640 1 -- 172.21.15.42:0/2519523685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d68071620 msgr2=0x7f6d6806d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d67e00640 1 --2- 172.21.15.42:0/2519523685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d68071620 0x7f6d6806d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d67e00640 1 -- 172.21.15.42:0/2519523685 >> v1:172.21.15.63:6789/0 conn(0x7f6d680ce570 legacy=0x7f6d6809b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:27.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d67e00640 1 -- 172.21.15.42:0/2519523685 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d68071100 con 0x7f6d6806da50 2024-05-28T13:59:27.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.989+0000 7f6d5ea00640 1 --2- 172.21.15.42:0/2519523685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d68071620 0x7f6d6806d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:27.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.993+0000 7f6d67e00640 1 --2- 172.21.15.42:0/2519523685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d6806da50 0x7f6d6815dfe0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009870 tx=0x7f6d6001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf5cd177d041377 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:27.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.993+0000 7f6d66a00640 1 -- 172.21.15.42:0/2519523685 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d600b0070 con 0x7f6d6806da50 2024-05-28T13:59:27.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.993+0000 7f6d66a00640 1 -- 172.21.15.42:0/2519523685 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d600a4e50 con 0x7f6d6806da50 2024-05-28T13:59:27.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.993+0000 7f6d66a00640 1 -- 172.21.15.42:0/2519523685 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d600ab620 con 0x7f6d6806da50 2024-05-28T13:59:27.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.993+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2519523685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d6806da50 msgr2=0x7f6d6815dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:27.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.993+0000 7f6d6cc00640 1 --2- 172.21.15.42:0/2519523685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d6806da50 0x7f6d6815dfe0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009870 tx=0x7f6d6001c400 comp rx=0 tx=0).stop 2024-05-28T13:59:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.993+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2519523685 shutdown_connections 2024-05-28T13:59:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.993+0000 7f6d6cc00640 1 --2- 172.21.15.42:0/2519523685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d6806da50 0x7f6d6815dfe0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.993+0000 7f6d6cc00640 1 --2- 172.21.15.42:0/2519523685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d68071620 0x7f6d6806d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.993+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2519523685 >> 172.21.15.42:0/2519523685 conn(0x7f6d680b3e60 msgr2=0x7f6d680b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.993+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2519523685 shutdown_connections 2024-05-28T13:59:27.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.993+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2519523685 wait complete. 2024-05-28T13:59:27.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.993+0000 7f6d6cc00640 1 Processor -- start 2024-05-28T13:59:27.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.993+0000 7f6d6cc00640 1 -- start start 2024-05-28T13:59:27.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d6cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d6806da50 0x7f6d68159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:27.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d6cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d68071620 0x7f6d680b7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:27.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d6cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d680ce570 0x7f6d680b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:27.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d6cc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6d680ccb40 con 0x7f6d68071620 2024-05-28T13:59:27.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d6cc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6d680cc9c0 con 0x7f6d680ce570 2024-05-28T13:59:27.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d6cc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6d680cccc0 con 0x7f6d6806da50 2024-05-28T13:59:27.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d5ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d68071620 0x7f6d680b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:27.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d5ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d68071620 0x7f6d680b7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41606/0 (socket says 172.21.15.42:41606) 2024-05-28T13:59:27.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d5ea00640 1 -- 172.21.15.42:0/2125666425 learned_addr learned my addr 172.21.15.42:0/2125666425 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:27.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d67400640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d6806da50 0x7f6d68159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:27.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d67e00640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d680ce570 0x7f6d680b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:27.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d5ea00640 1 -- 172.21.15.42:0/2125666425 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d6806da50 msgr2=0x7f6d68159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:27.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d5ea00640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d6806da50 0x7f6d68159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d5ea00640 1 -- 172.21.15.42:0/2125666425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d680ce570 msgr2=0x7f6d680b7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:27.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d5ea00640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d680ce570 0x7f6d680b7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:27.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d5ea00640 1 -- 172.21.15.42:0/2125666425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d680b51e0 con 0x7f6d68071620 2024-05-28T13:59:27.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d67400640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d6806da50 0x7f6d68159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:27.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d5ea00640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d68071620 0x7f6d680b7550 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c7c0 tx=0x7f6d5000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:27.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d5fe00640 1 -- 172.21.15.42:0/2125666425 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50007a30 con 0x7f6d68071620 2024-05-28T13:59:27.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d5fe00640 1 -- 172.21.15.42:0/2125666425 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d50007bd0 con 0x7f6d68071620 2024-05-28T13:59:27.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d5fe00640 1 -- 172.21.15.42:0/2125666425 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d500113f0 con 0x7f6d68071620 2024-05-28T13:59:27.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2125666425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d680b54c0 con 0x7f6d68071620 2024-05-28T13:59:27.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2125666425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d680d0f40 con 0x7f6d68071620 2024-05-28T13:59:27.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2125666425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d6804fcf0 con 0x7f6d68071620 2024-05-28T13:59:27.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:27.997+0000 7f6d5fe00640 1 -- 172.21.15.42:0/2125666425 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6d50011590 con 0x7f6d68071620 2024-05-28T13:59:27.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.001+0000 7f6d5fe00640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d44068380 0x7f6d4406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:27.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.001+0000 7f6d5fe00640 1 -- 172.21.15.42:0/2125666425 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6d50086be0 con 0x7f6d68071620 2024-05-28T13:59:27.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.001+0000 7f6d67400640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d44068380 0x7f6d4406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:27.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.001+0000 7f6d67400640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d44068380 0x7f6d4406a840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009fc0 tx=0x7f6d58009550 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:27.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.005+0000 7f6d5fe00640 1 -- 172.21.15.42:0/2125666425 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6d50015020 con 0x7f6d68071620 2024-05-28T13:59:28.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.173+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2125666425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d680b2f10 con 0x7f6d68071620 2024-05-28T13:59:28.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.173+0000 7f6d5fe00640 1 -- 172.21.15.42:0/2125666425 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16622 (secure 0 0 0) 0x7f6d50057190 con 0x7f6d68071620 2024-05-28T13:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":33,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:59:26.845129+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:59:24.673711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-28T13:59:25.795430+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:59:28.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.173+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2125666425 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d44068380 msgr2=0x7f6d4406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:28.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.173+0000 7f6d6cc00640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d44068380 0x7f6d4406a840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009fc0 tx=0x7f6d58009550 comp rx=0 tx=0).stop 2024-05-28T13:59:28.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.173+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2125666425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d68071620 msgr2=0x7f6d680b7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:28.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.173+0000 7f6d6cc00640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d68071620 0x7f6d680b7550 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c7c0 tx=0x7f6d5000cc80 comp rx=0 tx=0).stop 2024-05-28T13:59:28.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.173+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2125666425 shutdown_connections 2024-05-28T13:59:28.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.173+0000 7f6d6cc00640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d44068380 0x7f6d4406a840 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:28.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.173+0000 7f6d6cc00640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d680ce570 0x7f6d680b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:28.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.173+0000 7f6d6cc00640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d68071620 0x7f6d680b7550 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:28.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.173+0000 7f6d6cc00640 1 --2- 172.21.15.42:0/2125666425 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d6806da50 0x7f6d68159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:28.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.173+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2125666425 >> 172.21.15.42:0/2125666425 conn(0x7f6d680b3e60 msgr2=0x7f6d680b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:28.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.173+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2125666425 shutdown_connections 2024-05-28T13:59:28.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.173+0000 7f6d6cc00640 1 -- 172.21.15.42:0/2125666425 wait complete. 2024-05-28T13:59:28.180 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-28T13:59:28.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.341+0000 7f0eeac00640 1 Processor -- start 2024-05-28T13:59:28.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.341+0000 7f0eeac00640 1 -- start start 2024-05-28T13:59:28.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.341+0000 7f0eeac00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0ee406c100 0x7f0ee4169510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:28.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.341+0000 7f0eeac00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ee40b4440 con 0x7f0ee40b2240 2024-05-28T13:59:28.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.341+0000 7f0eeac00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ee40b0f10 con 0x7f0ee415e8c0 2024-05-28T13:59:28.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.341+0000 7f0eeac00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0ee40ae420 con 0x7f0ee406c100 2024-05-28T13:59:28.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.341+0000 7f0ee8c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0ee406c100 0x7f0ee4169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:28.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.341+0000 7f0edbe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0ee40b2240 0x7f0ee406b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51870/0 (socket says 172.21.15.42:51870) 2024-05-28T13:59:28.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.341+0000 7f0edbe00640 1 -- 172.21.15.42:0/2312992392 learned_addr learned my addr 172.21.15.42:0/2312992392 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:28.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.341+0000 7f0ee8c00640 1 -- 172.21.15.42:0/2312992392 >> v1:172.21.15.63:6789/0 conn(0x7f0ee415e8c0 legacy=0x7f0ee40b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:28.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.341+0000 7f0ee8c00640 1 -- 172.21.15.42:0/2312992392 >> v1:172.21.15.42:6789/0 conn(0x7f0ee40b2240 legacy=0x7f0ee406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:28.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.341+0000 7f0ee8c00640 1 -- 172.21.15.42:0/2312992392 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee40ae670 con 0x7f0ee406c100 2024-05-28T13:59:28.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.345+0000 7f0ee8c00640 1 --2- 172.21.15.42:0/2312992392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee406c100 0x7f0ee4169510 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009d90 tx=0x7f0edc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bc2314c2596143 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:28.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.345+0000 7f0ee3400640 1 -- 172.21.15.42:0/2312992392 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc0b2070 con 0x7f0ee406c100 2024-05-28T13:59:28.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.345+0000 7f0ee3400640 1 -- 172.21.15.42:0/2312992392 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0edc0a57a0 con 0x7f0ee406c100 2024-05-28T13:59:28.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.345+0000 7f0ee3400640 1 -- 172.21.15.42:0/2312992392 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc0a5b70 con 0x7f0ee406c100 2024-05-28T13:59:28.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.345+0000 7f0eeac00640 1 -- 172.21.15.42:0/2312992392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee406c100 msgr2=0x7f0ee4169510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:28.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.345+0000 7f0eeac00640 1 --2- 172.21.15.42:0/2312992392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee406c100 0x7f0ee4169510 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009d90 tx=0x7f0edc01c8e0 comp rx=0 tx=0).stop 2024-05-28T13:59:28.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.345+0000 7f0eeac00640 1 -- 172.21.15.42:0/2312992392 shutdown_connections 2024-05-28T13:59:28.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.345+0000 7f0eeac00640 1 --2- 172.21.15.42:0/2312992392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee406c100 0x7f0ee4169510 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:28.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.345+0000 7f0eeac00640 1 -- 172.21.15.42:0/2312992392 >> 172.21.15.42:0/2312992392 conn(0x7f0ee40b6960 msgr2=0x7f0ee40b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:28.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.345+0000 7f0eeac00640 1 -- 172.21.15.42:0/2312992392 shutdown_connections 2024-05-28T13:59:28.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.345+0000 7f0eeac00640 1 -- 172.21.15.42:0/2312992392 wait complete. 2024-05-28T13:59:28.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.345+0000 7f0eeac00640 1 Processor -- start 2024-05-28T13:59:28.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.345+0000 7f0eeac00640 1 -- start start 2024-05-28T13:59:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0eeac00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee406c100 0x7f0ee414faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0eeac00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee40b2240 0x7f0ee414ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0eeac00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee415e8c0 0x7f0ee415d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0eeac00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee415ee20 con 0x7f0ee40b2240 2024-05-28T13:59:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0eeac00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee415eca0 con 0x7f0ee415e8c0 2024-05-28T13:59:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0eeac00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee416c680 con 0x7f0ee406c100 2024-05-28T13:59:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0edbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee40b2240 0x7f0ee414ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0ee3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee406c100 0x7f0ee414faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0edbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee40b2240 0x7f0ee414ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41628/0 (socket says 172.21.15.42:41628) 2024-05-28T13:59:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0edbe00640 1 -- 172.21.15.42:0/2797276430 learned_addr learned my addr 172.21.15.42:0/2797276430 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0ee8c00640 1 --2- 172.21.15.42:0/2797276430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee415e8c0 0x7f0ee415d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:28.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0edbe00640 1 -- 172.21.15.42:0/2797276430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee406c100 msgr2=0x7f0ee414faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:28.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0edbe00640 1 --2- 172.21.15.42:0/2797276430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee406c100 0x7f0ee414faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:28.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0edbe00640 1 -- 172.21.15.42:0/2797276430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee415e8c0 msgr2=0x7f0ee415d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:28.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0edbe00640 1 --2- 172.21.15.42:0/2797276430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee415e8c0 0x7f0ee415d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:28.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0edbe00640 1 -- 172.21.15.42:0/2797276430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee415a2a0 con 0x7f0ee40b2240 2024-05-28T13:59:28.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0edbe00640 1 --2- 172.21.15.42:0/2797276430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee40b2240 0x7f0ee414ea20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00c530 tx=0x7f0ecc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:28.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0ee0c00640 1 -- 172.21.15.42:0/2797276430 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc017070 con 0x7f0ee40b2240 2024-05-28T13:59:28.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0ee0c00640 1 -- 172.21.15.42:0/2797276430 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ecc005cf0 con 0x7f0ee40b2240 2024-05-28T13:59:28.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0ee0c00640 1 -- 172.21.15.42:0/2797276430 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc007270 con 0x7f0ee40b2240 2024-05-28T13:59:28.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0eeac00640 1 -- 172.21.15.42:0/2797276430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee4150e70 con 0x7f0ee40b2240 2024-05-28T13:59:28.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0eeac00640 1 -- 172.21.15.42:0/2797276430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee41505d0 con 0x7f0ee40b2240 2024-05-28T13:59:28.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.349+0000 7f0eeac00640 1 -- 172.21.15.42:0/2797276430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee409f7a0 con 0x7f0ee40b2240 2024-05-28T13:59:28.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.353+0000 7f0ee0c00640 1 -- 172.21.15.42:0/2797276430 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0ecc01a070 con 0x7f0ee40b2240 2024-05-28T13:59:28.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.353+0000 7f0ee0c00640 1 --2- 172.21.15.42:0/2797276430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ec0068380 0x7f0ec006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:28.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.353+0000 7f0ee0c00640 1 -- 172.21.15.42:0/2797276430 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0ecc085c50 con 0x7f0ee40b2240 2024-05-28T13:59:28.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.353+0000 7f0ee3e00640 1 --2- 172.21.15.42:0/2797276430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ec0068380 0x7f0ec006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:28.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.353+0000 7f0ee3e00640 1 --2- 172.21.15.42:0/2797276430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ec0068380 0x7f0ec006a840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4007fd0 tx=0x7f0ed4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.357+0000 7f0ee0c00640 1 -- 172.21.15.42:0/2797276430 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ee415a2a0 con 0x7f0ee40b2240 2024-05-28T13:59:28.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:28.561+0000 7f0eeac00640 1 -- 172.21.15.42:0/2797276430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0ee404fdd0 con 0x7f0ee40b2240 2024-05-28T13:59:29.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.469+0000 7f0ee0c00640 1 -- 172.21.15.42:0/2797276430 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f0ee40b4440 con 0x7f0ee40b2240 2024-05-28T13:59:29.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.473+0000 7f0eeac00640 1 -- 172.21.15.42:0/2797276430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ec0068380 msgr2=0x7f0ec006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:29.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.473+0000 7f0eeac00640 1 --2- 172.21.15.42:0/2797276430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ec0068380 0x7f0ec006a840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4007fd0 tx=0x7f0ed4009040 comp rx=0 tx=0).stop 2024-05-28T13:59:29.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.473+0000 7f0eeac00640 1 -- 172.21.15.42:0/2797276430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee40b2240 msgr2=0x7f0ee414ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:29.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.473+0000 7f0eeac00640 1 --2- 172.21.15.42:0/2797276430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee40b2240 0x7f0ee414ea20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00c530 tx=0x7f0ecc00c9f0 comp rx=0 tx=0).stop 2024-05-28T13:59:29.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.473+0000 7f0eeac00640 1 -- 172.21.15.42:0/2797276430 shutdown_connections 2024-05-28T13:59:29.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.473+0000 7f0eeac00640 1 --2- 172.21.15.42:0/2797276430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ec0068380 0x7f0ec006a840 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:29.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.473+0000 7f0eeac00640 1 --2- 172.21.15.42:0/2797276430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee415e8c0 0x7f0ee415d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:29.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.473+0000 7f0eeac00640 1 --2- 172.21.15.42:0/2797276430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee40b2240 0x7f0ee414ea20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:29.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.473+0000 7f0eeac00640 1 --2- 172.21.15.42:0/2797276430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee406c100 0x7f0ee414faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:29.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.473+0000 7f0eeac00640 1 -- 172.21.15.42:0/2797276430 >> 172.21.15.42:0/2797276430 conn(0x7f0ee40b6960 msgr2=0x7f0ee40b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:29.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.473+0000 7f0eeac00640 1 -- 172.21.15.42:0/2797276430 shutdown_connections 2024-05-28T13:59:29.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.473+0000 7f0eeac00640 1 -- 172.21.15.42:0/2797276430 wait complete. 2024-05-28T13:59:29.482 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-28T13:59:29.482 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-28T13:59:29.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.645+0000 7f6e9b800640 1 Processor -- start 2024-05-28T13:59:29.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.645+0000 7f6e9b800640 1 -- start start 2024-05-28T13:59:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.649+0000 7f6e9b800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6e9406ee20 0x7f6e9406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.649+0000 7f6e9b800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6e9415b9b0 0x7f6e9415be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.649+0000 7f6e9b800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e940afe40 con 0x7f6e94144f80 2024-05-28T13:59:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.649+0000 7f6e9b800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6e9406f7c0 con 0x7f6e9406ee20 2024-05-28T13:59:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.649+0000 7f6e9b800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6e940724d0 con 0x7f6e9415b9b0 2024-05-28T13:59:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.649+0000 7f6e8be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6e9406ee20 0x7f6e9406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.649+0000 7f6e98e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f6e94144f80 0x7f6e94072960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51886/0 (socket says 172.21.15.42:51886) 2024-05-28T13:59:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.649+0000 7f6e98e00640 1 -- 172.21.15.42:0/3808634376 learned_addr learned my addr 172.21.15.42:0/3808634376 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.649+0000 7f6e99800640 1 --2- 172.21.15.42:0/3808634376 >> v2:172.21.15.63:3300/0 conn(0x7f6e9415b9b0 0x7f6e9415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:29.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.649+0000 7f6e8be00640 1 -- 172.21.15.42:0/3808634376 >> v2:172.21.15.63:3300/0 conn(0x7f6e9415b9b0 msgr2=0x7f6e9415be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:29.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.649+0000 7f6e8be00640 1 --2- 172.21.15.42:0/3808634376 >> v2:172.21.15.63:3300/0 conn(0x7f6e9415b9b0 0x7f6e9415be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:29.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.649+0000 7f6e8be00640 1 -- 172.21.15.42:0/3808634376 >> v1:172.21.15.42:6789/0 conn(0x7f6e94144f80 legacy=0x7f6e94072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:29.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.649+0000 7f6e8be00640 1 -- 172.21.15.42:0/3808634376 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9415c340 con 0x7f6e9406ee20 2024-05-28T13:59:29.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.653+0000 7f6e8be00640 1 --2- 172.21.15.42:0/3808634376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e9406ee20 0x7f6e9406f280 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6e94160060 tx=0x7f6e7c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fbad181a55112d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.653+0000 7f6e93e00640 1 -- 172.21.15.42:0/3808634376 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0b0070 con 0x7f6e9406ee20 2024-05-28T13:59:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.653+0000 7f6e93e00640 1 -- 172.21.15.42:0/3808634376 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e7c0ab600 con 0x7f6e9406ee20 2024-05-28T13:59:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.653+0000 7f6e93e00640 1 -- 172.21.15.42:0/3808634376 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0ab9d0 con 0x7f6e9406ee20 2024-05-28T13:59:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.653+0000 7f6e9b800640 1 -- 172.21.15.42:0/3808634376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e9406ee20 msgr2=0x7f6e9406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.653+0000 7f6e9b800640 1 --2- 172.21.15.42:0/3808634376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e9406ee20 0x7f6e9406f280 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6e94160060 tx=0x7f6e7c01c480 comp rx=0 tx=0).stop 2024-05-28T13:59:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.653+0000 7f6e9b800640 1 -- 172.21.15.42:0/3808634376 shutdown_connections 2024-05-28T13:59:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.653+0000 7f6e9b800640 1 --2- 172.21.15.42:0/3808634376 >> v2:172.21.15.63:3300/0 conn(0x7f6e9415b9b0 0x7f6e9415be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.653+0000 7f6e9b800640 1 --2- 172.21.15.42:0/3808634376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e9406ee20 0x7f6e9406f280 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:29.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.653+0000 7f6e9b800640 1 -- 172.21.15.42:0/3808634376 >> 172.21.15.42:0/3808634376 conn(0x7f6e940b21f0 msgr2=0x7f6e940b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:29.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.653+0000 7f6e9b800640 1 -- 172.21.15.42:0/3808634376 shutdown_connections 2024-05-28T13:59:29.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.653+0000 7f6e9b800640 1 -- 172.21.15.42:0/3808634376 wait complete. 2024-05-28T13:59:29.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.653+0000 7f6e9b800640 1 Processor -- start 2024-05-28T13:59:29.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.653+0000 7f6e9b800640 1 -- start start 2024-05-28T13:59:29.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e9b800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e9406ee20 0x7f6e9415aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:29.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e9b800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e94144f80 0x7f6e9415a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:29.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e9b800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e9415b9b0 0x7f6e94142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:29.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e9b800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6e940b4010 con 0x7f6e9406ee20 2024-05-28T13:59:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e9b800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6e94072160 con 0x7f6e9415b9b0 2024-05-28T13:59:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e9b800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6e940b4190 con 0x7f6e94144f80 2024-05-28T13:59:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e98e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e9406ee20 0x7f6e9415aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e98e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e9406ee20 0x7f6e9415aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41652/0 (socket says 172.21.15.42:41652) 2024-05-28T13:59:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e98e00640 1 -- 172.21.15.42:0/3914932306 learned_addr learned my addr 172.21.15.42:0/3914932306 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e99800640 1 --2- 172.21.15.42:0/3914932306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e9415b9b0 0x7f6e94142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e8be00640 1 --2- 172.21.15.42:0/3914932306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e94144f80 0x7f6e9415a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e98e00640 1 -- 172.21.15.42:0/3914932306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e94144f80 msgr2=0x7f6e9415a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e98e00640 1 --2- 172.21.15.42:0/3914932306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e94144f80 0x7f6e9415a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e98e00640 1 -- 172.21.15.42:0/3914932306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e9415b9b0 msgr2=0x7f6e94142300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e98e00640 1 --2- 172.21.15.42:0/3914932306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e9415b9b0 0x7f6e94142300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e98e00640 1 -- 172.21.15.42:0/3914932306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e94142840 con 0x7f6e9406ee20 2024-05-28T13:59:29.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e98e00640 1 --2- 172.21.15.42:0/3914932306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e9406ee20 0x7f6e9415aed0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400c450 tx=0x7f6e8400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:29.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e91600640 1 -- 172.21.15.42:0/3914932306 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84017070 con 0x7f6e9406ee20 2024-05-28T13:59:29.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e91600640 1 -- 172.21.15.42:0/3914932306 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e84005cf0 con 0x7f6e9406ee20 2024-05-28T13:59:29.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e9b800640 1 -- 172.21.15.42:0/3914932306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e94142ac0 con 0x7f6e9406ee20 2024-05-28T13:59:29.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e91600640 1 -- 172.21.15.42:0/3914932306 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84007290 con 0x7f6e9406ee20 2024-05-28T13:59:29.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e9b800640 1 -- 172.21.15.42:0/3914932306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e94143080 con 0x7f6e9406ee20 2024-05-28T13:59:29.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e9b800640 1 -- 172.21.15.42:0/3914932306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e940738b0 con 0x7f6e9406ee20 2024-05-28T13:59:29.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.657+0000 7f6e91600640 1 -- 172.21.15.42:0/3914932306 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6e8401a070 con 0x7f6e9406ee20 2024-05-28T13:59:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.661+0000 7f6e91600640 1 --2- 172.21.15.42:0/3914932306 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6e700683b0 0x7f6e7006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.661+0000 7f6e91600640 1 -- 172.21.15.42:0/3914932306 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f6e84085d40 con 0x7f6e9406ee20 2024-05-28T13:59:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.661+0000 7f6e8be00640 1 --2- 172.21.15.42:0/3914932306 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6e700683b0 0x7f6e7006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.661+0000 7f6e8be00640 1 --2- 172.21.15.42:0/3914932306 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6e700683b0 0x7f6e7006a870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0ad000 tx=0x7f6e7c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:29.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.665+0000 7f6e91600640 1 -- 172.21.15.42:0/3914932306 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6e94142840 con 0x7f6e9406ee20 2024-05-28T13:59:29.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:29.865+0000 7f6e9b800640 1 -- 172.21.15.42:0/3914932306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6e9404fdd0 con 0x7f6e9406ee20 2024-05-28T13:59:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.473+0000 7f6e91600640 1 -- 172.21.15.42:0/3914932306 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f6e84055b80 con 0x7f6e9406ee20 2024-05-28T13:59:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.473+0000 7f6e9b800640 1 -- 172.21.15.42:0/3914932306 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6e700683b0 msgr2=0x7f6e7006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.473+0000 7f6e9b800640 1 --2- 172.21.15.42:0/3914932306 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6e700683b0 0x7f6e7006a870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0ad000 tx=0x7f6e7c008040 comp rx=0 tx=0).stop 2024-05-28T13:59:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.473+0000 7f6e9b800640 1 -- 172.21.15.42:0/3914932306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e9406ee20 msgr2=0x7f6e9415aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.473+0000 7f6e9b800640 1 --2- 172.21.15.42:0/3914932306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e9406ee20 0x7f6e9415aed0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400c450 tx=0x7f6e8400c910 comp rx=0 tx=0).stop 2024-05-28T13:59:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.473+0000 7f6e9b800640 1 -- 172.21.15.42:0/3914932306 shutdown_connections 2024-05-28T13:59:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.473+0000 7f6e9b800640 1 --2- 172.21.15.42:0/3914932306 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6e700683b0 0x7f6e7006a870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.473+0000 7f6e9b800640 1 --2- 172.21.15.42:0/3914932306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e9415b9b0 0x7f6e94142300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.473+0000 7f6e9b800640 1 --2- 172.21.15.42:0/3914932306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e94144f80 0x7f6e9415a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.473+0000 7f6e9b800640 1 --2- 172.21.15.42:0/3914932306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e9406ee20 0x7f6e9415aed0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.473+0000 7f6e9b800640 1 -- 172.21.15.42:0/3914932306 >> 172.21.15.42:0/3914932306 conn(0x7f6e940b21f0 msgr2=0x7f6e940cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.477+0000 7f6e9b800640 1 -- 172.21.15.42:0/3914932306 shutdown_connections 2024-05-28T13:59:30.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.477+0000 7f6e9b800640 1 -- 172.21.15.42:0/3914932306 wait complete. 2024-05-28T13:59:30.481 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-28T13:59:30.482 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T13:59:30.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.637+0000 7fba8cc00640 1 Processor -- start 2024-05-28T13:59:30.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.637+0000 7fba8cc00640 1 -- start start 2024-05-28T13:59:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.637+0000 7fba8cc00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fba8815c530 0x7fba8806c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.637+0000 7fba8cc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fba8815c910 0x7fba880b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.637+0000 7fba8cc00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba880b23b0 con 0x7fba88160c00 2024-05-28T13:59:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.637+0000 7fba8cc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fba88160420 con 0x7fba8815c910 2024-05-28T13:59:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.637+0000 7fba8cc00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fba880b2650 con 0x7fba8815c530 2024-05-28T13:59:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.637+0000 7fba87e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fba88160c00 0x7fba880d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51898/0 (socket says 172.21.15.42:51898) 2024-05-28T13:59:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.637+0000 7fba87e00640 1 -- 172.21.15.42:0/569286533 learned_addr learned my addr 172.21.15.42:0/569286533 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.637+0000 7fba7fe00640 1 --2- 172.21.15.42:0/569286533 >> v2:172.21.15.42:3300/0 conn(0x7fba8815c910 0x7fba880b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.637+0000 7fba87400640 1 --2- 172.21.15.42:0/569286533 >> v2:172.21.15.63:3300/0 conn(0x7fba8815c530 0x7fba8806c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba7fe00640 1 -- 172.21.15.42:0/569286533 >> v2:172.21.15.63:3300/0 conn(0x7fba8815c530 msgr2=0x7fba8806c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba7fe00640 1 --2- 172.21.15.42:0/569286533 >> v2:172.21.15.63:3300/0 conn(0x7fba8815c530 0x7fba8806c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba7fe00640 1 -- 172.21.15.42:0/569286533 >> v1:172.21.15.42:6789/0 conn(0x7fba88160c00 legacy=0x7fba880d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:30.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba7fe00640 1 -- 172.21.15.42:0/569286533 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba880b2930 con 0x7fba8815c910 2024-05-28T13:59:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba7fe00640 1 --2- 172.21.15.42:0/569286533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8815c910 0x7fba880b1e70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fba880cecb0 tx=0x7fba7001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f748af68acc93f24 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba86a00640 1 -- 172.21.15.42:0/569286533 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba700b1070 con 0x7fba8815c910 2024-05-28T13:59:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba86a00640 1 -- 172.21.15.42:0/569286533 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba700ac5f0 con 0x7fba8815c910 2024-05-28T13:59:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba86a00640 1 -- 172.21.15.42:0/569286533 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba700ac9c0 con 0x7fba8815c910 2024-05-28T13:59:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba8cc00640 1 -- 172.21.15.42:0/569286533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8815c910 msgr2=0x7fba880b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba8cc00640 1 --2- 172.21.15.42:0/569286533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8815c910 0x7fba880b1e70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fba880cecb0 tx=0x7fba7001c480 comp rx=0 tx=0).stop 2024-05-28T13:59:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba8cc00640 1 -- 172.21.15.42:0/569286533 shutdown_connections 2024-05-28T13:59:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba8cc00640 1 --2- 172.21.15.42:0/569286533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8815c910 0x7fba880b1e70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba8cc00640 1 --2- 172.21.15.42:0/569286533 >> v2:172.21.15.63:3300/0 conn(0x7fba8815c530 0x7fba8806c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:30.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba8cc00640 1 -- 172.21.15.42:0/569286533 >> 172.21.15.42:0/569286533 conn(0x7fba8809aa50 msgr2=0x7fba880b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:30.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba8cc00640 1 -- 172.21.15.42:0/569286533 shutdown_connections 2024-05-28T13:59:30.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.641+0000 7fba8cc00640 1 -- 172.21.15.42:0/569286533 wait complete. 2024-05-28T13:59:30.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba8cc00640 1 Processor -- start 2024-05-28T13:59:30.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba8cc00640 1 -- start start 2024-05-28T13:59:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba8cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8815c530 0x7fba88151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba8cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8815c910 0x7fba881f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba8cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba88160c00 0x7fba8813ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba8cc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fba880b16b0 con 0x7fba88160c00 2024-05-28T13:59:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba8cc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba880b1530 con 0x7fba8815c530 2024-05-28T13:59:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba8cc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fba881695e0 con 0x7fba8815c910 2024-05-28T13:59:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba87e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba88160c00 0x7fba8813ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:30.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba87400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8815c530 0x7fba88151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:30.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba87e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba88160c00 0x7fba8813ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41666/0 (socket says 172.21.15.42:41666) 2024-05-28T13:59:30.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba87e00640 1 -- 172.21.15.42:0/3554357067 learned_addr learned my addr 172.21.15.42:0/3554357067 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:30.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba7fe00640 1 --2- 172.21.15.42:0/3554357067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8815c910 0x7fba881f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba87e00640 1 -- 172.21.15.42:0/3554357067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8815c910 msgr2=0x7fba881f5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba87e00640 1 --2- 172.21.15.42:0/3554357067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8815c910 0x7fba881f5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba87e00640 1 -- 172.21.15.42:0/3554357067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8815c530 msgr2=0x7fba88151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba87e00640 1 --2- 172.21.15.42:0/3554357067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8815c530 0x7fba88151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba87e00640 1 -- 172.21.15.42:0/3554357067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8813f9d0 con 0x7fba88160c00 2024-05-28T13:59:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba87e00640 1 --2- 172.21.15.42:0/3554357067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba88160c00 0x7fba8813ef20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba80002240 tx=0x7fba8000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba7f400640 1 -- 172.21.15.42:0/3554357067 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80017070 con 0x7fba88160c00 2024-05-28T13:59:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba7f400640 1 -- 172.21.15.42:0/3554357067 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8000abd0 con 0x7fba88160c00 2024-05-28T13:59:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba7f400640 1 -- 172.21.15.42:0/3554357067 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80012400 con 0x7fba88160c00 2024-05-28T13:59:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba8cc00640 1 -- 172.21.15.42:0/3554357067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8813fcb0 con 0x7fba88160c00 2024-05-28T13:59:30.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.645+0000 7fba8cc00640 1 -- 172.21.15.42:0/3554357067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba881f5a80 con 0x7fba88160c00 2024-05-28T13:59:30.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.649+0000 7fba8cc00640 1 -- 172.21.15.42:0/3554357067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba8804fcf0 con 0x7fba88160c00 2024-05-28T13:59:30.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.649+0000 7fba7f400640 1 -- 172.21.15.42:0/3554357067 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fba8001a070 con 0x7fba88160c00 2024-05-28T13:59:30.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.649+0000 7fba7f400640 1 --2- 172.21.15.42:0/3554357067 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba64068380 0x7fba6406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:30.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.649+0000 7fba7f400640 1 -- 172.21.15.42:0/3554357067 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fba80086630 con 0x7fba88160c00 2024-05-28T13:59:30.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.649+0000 7fba87400640 1 --2- 172.21.15.42:0/3554357067 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba64068380 0x7fba6406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:30.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.649+0000 7fba87400640 1 --2- 172.21.15.42:0/3554357067 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba64068380 0x7fba6406a840 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fba78009b60 tx=0x7fba78009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:30.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.653+0000 7fba7f400640 1 -- 172.21.15.42:0/3554357067 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fba80056500 con 0x7fba88160c00 2024-05-28T13:59:30.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.853+0000 7fba8cc00640 1 -- 172.21.15.42:0/3554357067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba880b71a0 con 0x7fba88160c00 2024-05-28T13:59:30.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.853+0000 7fba7f400640 1 -- 172.21.15.42:0/3554357067 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7fba8005a800 con 0x7fba88160c00 2024-05-28T13:59:30.838 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:30.839 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2044753493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2044753493},{"type":"v1","addr":"172.21.15.42:6835","nonce":2044753493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3332210819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3332210819},{"type":"v1","addr":"172.21.15.42:6837","nonce":3332210819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":54782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1050322479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1050322479},{"type":"v1","addr":"172.21.15.63:6834","nonce":1050322479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T13:59:26.845096+0000","modified":"2024-05-28T13:59:29.868153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":54785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54785":{"gid":54785,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/2114729398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2114729398},{"type":"v1","addr":"172.21.15.63:6835","nonce":2114729398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":54785,"qdb_cluster":[54785]},"id":2}]} 2024-05-28T13:59:30.839 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 17 2024-05-28T13:59:30.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.853+0000 7fba8cc00640 1 -- 172.21.15.42:0/3554357067 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba64068380 msgr2=0x7fba6406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:30.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.853+0000 7fba8cc00640 1 --2- 172.21.15.42:0/3554357067 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba64068380 0x7fba6406a840 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fba78009b60 tx=0x7fba78009190 comp rx=0 tx=0).stop 2024-05-28T13:59:30.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.853+0000 7fba8cc00640 1 -- 172.21.15.42:0/3554357067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba88160c00 msgr2=0x7fba8813ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:30.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.853+0000 7fba8cc00640 1 --2- 172.21.15.42:0/3554357067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba88160c00 0x7fba8813ef20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba80002240 tx=0x7fba8000cf00 comp rx=0 tx=0).stop 2024-05-28T13:59:30.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.853+0000 7fba8cc00640 1 -- 172.21.15.42:0/3554357067 shutdown_connections 2024-05-28T13:59:30.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.853+0000 7fba8cc00640 1 --2- 172.21.15.42:0/3554357067 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba64068380 0x7fba6406a840 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:30.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.853+0000 7fba8cc00640 1 --2- 172.21.15.42:0/3554357067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba88160c00 0x7fba8813ef20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:30.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.853+0000 7fba8cc00640 1 --2- 172.21.15.42:0/3554357067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8815c910 0x7fba881f5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:30.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.853+0000 7fba8cc00640 1 --2- 172.21.15.42:0/3554357067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8815c530 0x7fba88151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:30.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.853+0000 7fba8cc00640 1 -- 172.21.15.42:0/3554357067 >> 172.21.15.42:0/3554357067 conn(0x7fba8809aa50 msgr2=0x7fba880af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:30.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.857+0000 7fba8cc00640 1 -- 172.21.15.42:0/3554357067 shutdown_connections 2024-05-28T13:59:30.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:30.857+0000 7fba8cc00640 1 -- 172.21.15.42:0/3554357067 wait complete. 2024-05-28T13:59:30.859 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T13:59:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.013+0000 7f8b33000640 1 Processor -- start 2024-05-28T13:59:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.013+0000 7f8b33000640 1 -- start start 2024-05-28T13:59:31.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.017+0000 7f8b33000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8b2c0d0420 0x7f8b2c15ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.017+0000 7f8b33000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8b2c0b42e0 0x7f8b2c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.017+0000 7f8b33000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b2c06eaa0 con 0x7f8b2c067810 2024-05-28T13:59:31.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.017+0000 7f8b33000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8b2c06ed80 con 0x7f8b2c0b42e0 2024-05-28T13:59:31.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.017+0000 7f8b33000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8b2c06f060 con 0x7f8b2c0d0420 2024-05-28T13:59:31.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.017+0000 7f8b31000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8b2c0b42e0 0x7f8b2c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.017+0000 7f8b31000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8b2c0b42e0 0x7f8b2c06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41672/0 (socket says 172.21.15.42:41672) 2024-05-28T13:59:31.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.017+0000 7f8b31000640 1 -- 172.21.15.42:0/1645900945 learned_addr learned my addr 172.21.15.42:0/1645900945 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:31.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.017+0000 7f8b2be00640 1 --2- 172.21.15.42:0/1645900945 >> v2:172.21.15.63:3300/0 conn(0x7f8b2c0d0420 0x7f8b2c15ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.017+0000 7f8b31000640 1 -- 172.21.15.42:0/1645900945 >> v2:172.21.15.63:3300/0 conn(0x7f8b2c0d0420 msgr2=0x7f8b2c15ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:31.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.017+0000 7f8b31000640 1 --2- 172.21.15.42:0/1645900945 >> v2:172.21.15.63:3300/0 conn(0x7f8b2c0d0420 0x7f8b2c15ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.017+0000 7f8b31000640 1 -- 172.21.15.42:0/1645900945 >> v1:172.21.15.63:6789/0 conn(0x7f8b2c067810 legacy=0x7f8b2c0b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.017+0000 7f8b31000640 1 -- 172.21.15.42:0/1645900945 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c06f340 con 0x7f8b2c0b42e0 2024-05-28T13:59:31.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.021+0000 7f8b31000640 1 --2- 172.21.15.42:0/1645900945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b2c0b42e0 0x7f8b2c06bc90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c072290 tx=0x7f8b2401d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2ed7451db53657 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:31.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.021+0000 7f8b2b400640 1 -- 172.21.15.42:0/1645900945 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b240a5730 con 0x7f8b2c0b42e0 2024-05-28T13:59:31.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.021+0000 7f8b2b400640 1 -- 172.21.15.42:0/1645900945 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b240a58d0 con 0x7f8b2c0b42e0 2024-05-28T13:59:31.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.021+0000 7f8b2b400640 1 -- 172.21.15.42:0/1645900945 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b240a5ca0 con 0x7f8b2c0b42e0 2024-05-28T13:59:31.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.021+0000 7f8b33000640 1 -- 172.21.15.42:0/1645900945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b2c0b42e0 msgr2=0x7f8b2c06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.021+0000 7f8b33000640 1 --2- 172.21.15.42:0/1645900945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b2c0b42e0 0x7f8b2c06bc90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c072290 tx=0x7f8b2401d3c0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.021+0000 7f8b33000640 1 -- 172.21.15.42:0/1645900945 shutdown_connections 2024-05-28T13:59:31.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.021+0000 7f8b33000640 1 --2- 172.21.15.42:0/1645900945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b2c0b42e0 0x7f8b2c06bc90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.021+0000 7f8b33000640 1 --2- 172.21.15.42:0/1645900945 >> v2:172.21.15.63:3300/0 conn(0x7f8b2c0d0420 0x7f8b2c15ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.021+0000 7f8b33000640 1 -- 172.21.15.42:0/1645900945 >> 172.21.15.42:0/1645900945 conn(0x7f8b2c0d0b40 msgr2=0x7f8b2c0d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:31.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.021+0000 7f8b33000640 1 -- 172.21.15.42:0/1645900945 shutdown_connections 2024-05-28T13:59:31.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.021+0000 7f8b33000640 1 -- 172.21.15.42:0/1645900945 wait complete. 2024-05-28T13:59:31.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.021+0000 7f8b33000640 1 Processor -- start 2024-05-28T13:59:31.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.021+0000 7f8b33000640 1 -- start start 2024-05-28T13:59:31.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b33000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b2c067810 0x7f8b2c13c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b33000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b2c0b42e0 0x7f8b2c14ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b33000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b2c0d0420 0x7f8b2c14d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b33000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c06cb60 con 0x7f8b2c0d0420 2024-05-28T13:59:31.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b33000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c06c9e0 con 0x7f8b2c0b42e0 2024-05-28T13:59:31.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b33000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8b2c06cce0 con 0x7f8b2c067810 2024-05-28T13:59:31.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b31000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b2c0d0420 0x7f8b2c14d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b31000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b2c0d0420 0x7f8b2c14d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41674/0 (socket says 172.21.15.42:41674) 2024-05-28T13:59:31.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b2be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b2c067810 0x7f8b2c13c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b31000640 1 -- 172.21.15.42:0/674745233 learned_addr learned my addr 172.21.15.42:0/674745233 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:31.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b23400640 1 --2- 172.21.15.42:0/674745233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b2c0b42e0 0x7f8b2c14ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b31000640 1 -- 172.21.15.42:0/674745233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b2c067810 msgr2=0x7f8b2c13c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b31000640 1 --2- 172.21.15.42:0/674745233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b2c067810 0x7f8b2c13c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b31000640 1 -- 172.21.15.42:0/674745233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b2c0b42e0 msgr2=0x7f8b2c14ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b31000640 1 --2- 172.21.15.42:0/674745233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b2c0b42e0 0x7f8b2c14ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b31000640 1 -- 172.21.15.42:0/674745233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c153a20 con 0x7f8b2c0d0420 2024-05-28T13:59:31.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b31000640 1 --2- 172.21.15.42:0/674745233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b2c0d0420 0x7f8b2c14d210 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8b240af040 tx=0x7f8b24019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:31.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b28c00640 1 -- 172.21.15.42:0/674745233 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b240bc030 con 0x7f8b2c0d0420 2024-05-28T13:59:31.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b28c00640 1 -- 172.21.15.42:0/674745233 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b240b8470 con 0x7f8b2c0d0420 2024-05-28T13:59:31.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b28c00640 1 -- 172.21.15.42:0/674745233 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b240b8880 con 0x7f8b2c0d0420 2024-05-28T13:59:31.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b33000640 1 -- 172.21.15.42:0/674745233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c153d00 con 0x7f8b2c0d0420 2024-05-28T13:59:31.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b33000640 1 -- 172.21.15.42:0/674745233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c1542c0 con 0x7f8b2c0d0420 2024-05-28T13:59:31.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b33000640 1 -- 172.21.15.42:0/674745233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2c068a90 con 0x7f8b2c0d0420 2024-05-28T13:59:31.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.025+0000 7f8b28c00640 1 -- 172.21.15.42:0/674745233 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8b240bf070 con 0x7f8b2c0d0420 2024-05-28T13:59:31.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.029+0000 7f8b28c00640 1 --2- 172.21.15.42:0/674745233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8b08068380 0x7f8b0806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.029+0000 7f8b28c00640 1 -- 172.21.15.42:0/674745233 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8b24121670 con 0x7f8b2c0d0420 2024-05-28T13:59:31.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.029+0000 7f8b2be00640 1 --2- 172.21.15.42:0/674745233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8b08068380 0x7f8b0806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.029+0000 7f8b2be00640 1 --2- 172.21.15.42:0/674745233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8b08068380 0x7f8b0806a840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009ac0 tx=0x7f8b1c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:31.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.033+0000 7f8b28c00640 1 -- 172.21.15.42:0/674745233 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8b240c6070 con 0x7f8b2c0d0420 2024-05-28T13:59:31.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.197+0000 7f8b33000640 1 -- 172.21.15.42:0/674745233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b2c154500 con 0x7f8b2c0d0420 2024-05-28T13:59:31.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.201+0000 7f8b28c00640 1 -- 172.21.15.42:0/674745233 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16609 (secure 0 0 0) 0x7f8b240165e0 con 0x7f8b2c0d0420 2024-05-28T13:59:31.185 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:31.185 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":34,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:59:28.461435+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:59:24.673711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-28T13:59:25.795430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T13:59:31.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.201+0000 7f8b33000640 1 -- 172.21.15.42:0/674745233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8b08068380 msgr2=0x7f8b0806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.201+0000 7f8b33000640 1 --2- 172.21.15.42:0/674745233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8b08068380 0x7f8b0806a840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009ac0 tx=0x7f8b1c009190 comp rx=0 tx=0).stop 2024-05-28T13:59:31.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.201+0000 7f8b33000640 1 -- 172.21.15.42:0/674745233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b2c0d0420 msgr2=0x7f8b2c14d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.201+0000 7f8b33000640 1 --2- 172.21.15.42:0/674745233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b2c0d0420 0x7f8b2c14d210 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8b240af040 tx=0x7f8b24019700 comp rx=0 tx=0).stop 2024-05-28T13:59:31.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.201+0000 7f8b33000640 1 -- 172.21.15.42:0/674745233 shutdown_connections 2024-05-28T13:59:31.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.201+0000 7f8b33000640 1 --2- 172.21.15.42:0/674745233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8b08068380 0x7f8b0806a840 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.201+0000 7f8b33000640 1 --2- 172.21.15.42:0/674745233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b2c0d0420 0x7f8b2c14d210 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.201+0000 7f8b33000640 1 --2- 172.21.15.42:0/674745233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b2c0b42e0 0x7f8b2c14ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.201+0000 7f8b33000640 1 --2- 172.21.15.42:0/674745233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b2c067810 0x7f8b2c13c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.201+0000 7f8b33000640 1 -- 172.21.15.42:0/674745233 >> 172.21.15.42:0/674745233 conn(0x7f8b2c0d0b40 msgr2=0x7f8b2c15ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:31.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.201+0000 7f8b33000640 1 -- 172.21.15.42:0/674745233 shutdown_connections 2024-05-28T13:59:31.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.201+0000 7f8b33000640 1 -- 172.21.15.42:0/674745233 wait complete. 2024-05-28T13:59:31.208 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-28T13:59:31.208 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T13:59:31.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.365+0000 7f5382800640 1 Processor -- start 2024-05-28T13:59:31.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.365+0000 7f5382800640 1 -- start start 2024-05-28T13:59:31.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.365+0000 7f5382800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f537c09bbd0 0x7f537c09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.365+0000 7f5382800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f537c0d0870 con 0x7f537c15c990 2024-05-28T13:59:31.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.365+0000 7f5382800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f537c0cd300 con 0x7f537c09af90 2024-05-28T13:59:31.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.365+0000 7f5382800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f537c09b400 con 0x7f537c09bbd0 2024-05-28T13:59:31.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.365+0000 7f537b400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f537c15c990 0x7f537c0d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51902/0 (socket says 172.21.15.42:51902) 2024-05-28T13:59:31.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.365+0000 7f537b400640 1 -- 172.21.15.42:0/1727361724 learned_addr learned my addr 172.21.15.42:0/1727361724 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:31.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.365+0000 7f537aa00640 1 --2- 172.21.15.42:0/1727361724 >> v2:172.21.15.42:3300/0 conn(0x7f537c09bbd0 0x7f537c09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.365+0000 7f537a000640 1 -- 172.21.15.42:0/1727361724 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1076499949 0 0) 0x7f537c0d0870 con 0x7f537c15c990 2024-05-28T13:59:31.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.365+0000 7f537a000640 1 -- 172.21.15.42:0/1727361724 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5360003530 con 0x7f537c15c990 2024-05-28T13:59:31.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.365+0000 7f537aa00640 1 -- 172.21.15.42:0/1727361724 >> v1:172.21.15.63:6789/0 conn(0x7f537c09af90 legacy=0x7f537c0b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.365+0000 7f537aa00640 1 -- 172.21.15.42:0/1727361724 >> v1:172.21.15.42:6789/0 conn(0x7f537c15c990 legacy=0x7f537c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.365+0000 7f537aa00640 1 -- 172.21.15.42:0/1727361724 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c0cd5e0 con 0x7f537c09bbd0 2024-05-28T13:59:31.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.369+0000 7f537aa00640 1 --2- 172.21.15.42:0/1727361724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f537c09bbd0 0x7f537c09aa50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f53680097c0 tx=0x7f536801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed100716ba46554e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:31.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.369+0000 7f537a000640 1 -- 172.21.15.42:0/1727361724 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53680b1070 con 0x7f537c09bbd0 2024-05-28T13:59:31.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.369+0000 7f537a000640 1 -- 172.21.15.42:0/1727361724 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53680ac5f0 con 0x7f537c09bbd0 2024-05-28T13:59:31.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.369+0000 7f537a000640 1 -- 172.21.15.42:0/1727361724 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53680ac9c0 con 0x7f537c09bbd0 2024-05-28T13:59:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.369+0000 7f5382800640 1 -- 172.21.15.42:0/1727361724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f537c09bbd0 msgr2=0x7f537c09aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.369+0000 7f5382800640 1 --2- 172.21.15.42:0/1727361724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f537c09bbd0 0x7f537c09aa50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f53680097c0 tx=0x7f536801c4a0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.369+0000 7f5382800640 1 -- 172.21.15.42:0/1727361724 shutdown_connections 2024-05-28T13:59:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.369+0000 7f5382800640 1 --2- 172.21.15.42:0/1727361724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f537c09bbd0 0x7f537c09aa50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.369+0000 7f5382800640 1 -- 172.21.15.42:0/1727361724 >> 172.21.15.42:0/1727361724 conn(0x7f537c0b4080 msgr2=0x7f537c0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.369+0000 7f5382800640 1 -- 172.21.15.42:0/1727361724 shutdown_connections 2024-05-28T13:59:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.369+0000 7f5382800640 1 -- 172.21.15.42:0/1727361724 wait complete. 2024-05-28T13:59:31.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f5382800640 1 Processor -- start 2024-05-28T13:59:31.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f5382800640 1 -- start start 2024-05-28T13:59:31.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f5382800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f537c09af90 0x7f537c0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f5382800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f537c15c990 0x7f537c0ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f5382800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537c1547c0 0x7f537c153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f5382800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f537c0ce3a0 con 0x7f537c09af90 2024-05-28T13:59:31.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f5382800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f537c16a750 con 0x7f537c15c990 2024-05-28T13:59:31.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f5382800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f537c0cb2f0 con 0x7f537c1547c0 2024-05-28T13:59:31.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f537b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f537c09af90 0x7f537c0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f537b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f537c09af90 0x7f537c0cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41694/0 (socket says 172.21.15.42:41694) 2024-05-28T13:59:31.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f537b400640 1 -- 172.21.15.42:0/1500510214 learned_addr learned my addr 172.21.15.42:0/1500510214 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:31.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f537b400640 1 -- 172.21.15.42:0/1500510214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537c1547c0 msgr2=0x7f537c153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f537be00640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537c1547c0 0x7f537c153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f537aa00640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f537c15c990 0x7f537c0ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f537b400640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537c1547c0 0x7f537c153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f537b400640 1 -- 172.21.15.42:0/1500510214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f537c15c990 msgr2=0x7f537c0ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f537b400640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f537c15c990 0x7f537c0ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f537b400640 1 -- 172.21.15.42:0/1500510214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c0ca6e0 con 0x7f537c09af90 2024-05-28T13:59:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f537be00640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537c1547c0 0x7f537c153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f537aa00640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f537c15c990 0x7f537c0ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f537b400640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f537c09af90 0x7f537c0cb780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f536c00c500 tx=0x7f536c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f5373400640 1 -- 172.21.15.42:0/1500510214 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c017070 con 0x7f537c09af90 2024-05-28T13:59:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f5373400640 1 -- 172.21.15.42:0/1500510214 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f536c005cf0 con 0x7f537c09af90 2024-05-28T13:59:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f5373400640 1 -- 172.21.15.42:0/1500510214 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c0072f0 con 0x7f537c09af90 2024-05-28T13:59:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f5382800640 1 -- 172.21.15.42:0/1500510214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537c0ca9c0 con 0x7f537c09af90 2024-05-28T13:59:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f5382800640 1 -- 172.21.15.42:0/1500510214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537c0caf80 con 0x7f537c09af90 2024-05-28T13:59:31.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.373+0000 7f5382800640 1 -- 172.21.15.42:0/1500510214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537c068a90 con 0x7f537c09af90 2024-05-28T13:59:31.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.377+0000 7f5373400640 1 -- 172.21.15.42:0/1500510214 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f536c01a070 con 0x7f537c09af90 2024-05-28T13:59:31.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.377+0000 7f5373400640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f535c068380 0x7f535c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.377+0000 7f5373400640 1 -- 172.21.15.42:0/1500510214 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f536c085d80 con 0x7f537c09af90 2024-05-28T13:59:31.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.377+0000 7f537aa00640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f535c068380 0x7f535c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.377+0000 7f537aa00640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f535c068380 0x7f535c06a840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f53680ae000 tx=0x7f5368008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:31.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.381+0000 7f5373400640 1 -- 172.21.15.42:0/1500510214 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f536c055ce0 con 0x7f537c09af90 2024-05-28T13:59:31.407 INFO:tasks.ceph.mds.d.smithi063.stderr:2024-05-28T13:59:31.423+0000 7f1ec6e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-28T13:59:31.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.545+0000 7f5382800640 1 -- 172.21.15.42:0/1500510214 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f537c0ce520 con 0x7f535c068380 2024-05-28T13:59:31.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.549+0000 7f5373400640 1 -- 172.21.15.42:0/1500510214 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f537c0ce520 con 0x7f535c068380 2024-05-28T13:59:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:31.541 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T13:59:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.553+0000 7f5382800640 1 -- 172.21.15.42:0/1500510214 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f535c068380 msgr2=0x7f535c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.553+0000 7f5382800640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f535c068380 0x7f535c06a840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f53680ae000 tx=0x7f5368008040 comp rx=0 tx=0).stop 2024-05-28T13:59:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.553+0000 7f5382800640 1 -- 172.21.15.42:0/1500510214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f537c09af90 msgr2=0x7f537c0cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.553+0000 7f5382800640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f537c09af90 0x7f537c0cb780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f536c00c500 tx=0x7f536c00c9c0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.553+0000 7f5382800640 1 -- 172.21.15.42:0/1500510214 shutdown_connections 2024-05-28T13:59:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.553+0000 7f5382800640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f535c068380 0x7f535c06a840 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.553+0000 7f5382800640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f537c1547c0 0x7f537c153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.553+0000 7f5382800640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f537c15c990 0x7f537c0ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.553+0000 7f5382800640 1 --2- 172.21.15.42:0/1500510214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f537c09af90 0x7f537c0cb780 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.553+0000 7f5382800640 1 -- 172.21.15.42:0/1500510214 >> 172.21.15.42:0/1500510214 conn(0x7f537c0b4080 msgr2=0x7f537c0b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:31.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.557+0000 7f5382800640 1 -- 172.21.15.42:0/1500510214 shutdown_connections 2024-05-28T13:59:31.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.557+0000 7f5382800640 1 -- 172.21.15.42:0/1500510214 wait complete. 2024-05-28T13:59:31.560 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-05-28T13:59:30.868873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237040,"kb_used_data":4208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737680,"statfs":{"total":773094113280,"available":772851384320,"internally_reserved":0,"allocated":4308992,"data_stored":2383188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.441200"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057333+0000","last_change":"2024-05-28T13:59:27.480267+0000","last_active":"2024-05-28T13:59:29.057333+0000","last_peered":"2024-05-28T13:59:29.057333+0000","last_clean":"2024-05-28T13:59:29.057333+0000","last_became_active":"2024-05-28T13:59:27.479892+0000","last_became_peered":"2024-05-28T13:59:27.479892+0000","last_unstale":"2024-05-28T13:59:29.057333+0000","last_undegraded":"2024-05-28T13:59:29.057333+0000","last_fullsized":"2024-05-28T13:59:29.057333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:16.522449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.470294+0000","last_change":"2024-05-28T13:59:27.471521+0000","last_active":"2024-05-28T13:59:28.470294+0000","last_peered":"2024-05-28T13:59:28.470294+0000","last_clean":"2024-05-28T13:59:28.470294+0000","last_became_active":"2024-05-28T13:59:27.471250+0000","last_became_peered":"2024-05-28T13:59:27.471250+0000","last_unstale":"2024-05-28T13:59:28.470294+0000","last_undegraded":"2024-05-28T13:59:28.470294+0000","last_fullsized":"2024-05-28T13:59:28.470294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:46.412620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.058034+0000","last_change":"2024-05-28T13:59:27.480775+0000","last_active":"2024-05-28T13:59:29.058034+0000","last_peered":"2024-05-28T13:59:29.058034+0000","last_clean":"2024-05-28T13:59:29.058034+0000","last_became_active":"2024-05-28T13:59:27.480513+0000","last_became_peered":"2024-05-28T13:59:27.480513+0000","last_unstale":"2024-05-28T13:59:29.058034+0000","last_undegraded":"2024-05-28T13:59:29.058034+0000","last_fullsized":"2024-05-28T13:59:29.058034+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:40.078224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056058+0000","last_change":"2024-05-28T13:59:27.471276+0000","last_active":"2024-05-28T13:59:29.056058+0000","last_peered":"2024-05-28T13:59:29.056058+0000","last_clean":"2024-05-28T13:59:29.056058+0000","last_became_active":"2024-05-28T13:59:27.471007+0000","last_became_peered":"2024-05-28T13:59:27.471007+0000","last_unstale":"2024-05-28T13:59:29.056058+0000","last_undegraded":"2024-05-28T13:59:29.056058+0000","last_fullsized":"2024-05-28T13:59:29.056058+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:40:52.295785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056268+0000","last_change":"2024-05-28T13:59:27.464933+0000","last_active":"2024-05-28T13:59:29.056268+0000","last_peered":"2024-05-28T13:59:29.056268+0000","last_clean":"2024-05-28T13:59:29.056268+0000","last_became_active":"2024-05-28T13:59:27.464641+0000","last_became_peered":"2024-05-28T13:59:27.464641+0000","last_unstale":"2024-05-28T13:59:29.056268+0000","last_undegraded":"2024-05-28T13:59:29.056268+0000","last_fullsized":"2024-05-28T13:59:29.056268+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:00:48.840009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469391+0000","last_change":"2024-05-28T13:59:27.477705+0000","last_active":"2024-05-28T13:59:28.469391+0000","last_peered":"2024-05-28T13:59:28.469391+0000","last_clean":"2024-05-28T13:59:28.469391+0000","last_became_active":"2024-05-28T13:59:27.477406+0000","last_became_peered":"2024-05-28T13:59:27.477406+0000","last_unstale":"2024-05-28T13:59:28.469391+0000","last_undegraded":"2024-05-28T13:59:28.469391+0000","last_fullsized":"2024-05-28T13:59:28.469391+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:13:14.315389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473550+0000","last_change":"2024-05-28T13:59:27.484864+0000","last_active":"2024-05-28T13:59:28.473550+0000","last_peered":"2024-05-28T13:59:28.473550+0000","last_clean":"2024-05-28T13:59:28.473550+0000","last_became_active":"2024-05-28T13:59:27.484214+0000","last_became_peered":"2024-05-28T13:59:27.484214+0000","last_unstale":"2024-05-28T13:59:28.473550+0000","last_undegraded":"2024-05-28T13:59:28.473550+0000","last_fullsized":"2024-05-28T13:59:28.473550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:14:36.728353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473130+0000","last_change":"2024-05-28T13:59:27.484311+0000","last_active":"2024-05-28T13:59:28.473130+0000","last_peered":"2024-05-28T13:59:28.473130+0000","last_clean":"2024-05-28T13:59:28.473130+0000","last_became_active":"2024-05-28T13:59:27.483949+0000","last_became_peered":"2024-05-28T13:59:27.483949+0000","last_unstale":"2024-05-28T13:59:28.473130+0000","last_undegraded":"2024-05-28T13:59:28.473130+0000","last_fullsized":"2024-05-28T13:59:28.473130+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:26:42.728298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472376+0000","last_change":"2024-05-28T13:59:27.476975+0000","last_active":"2024-05-28T13:59:28.472376+0000","last_peered":"2024-05-28T13:59:28.472376+0000","last_clean":"2024-05-28T13:59:28.472376+0000","last_became_active":"2024-05-28T13:59:27.476603+0000","last_became_peered":"2024-05-28T13:59:27.476603+0000","last_unstale":"2024-05-28T13:59:28.472376+0000","last_undegraded":"2024-05-28T13:59:28.472376+0000","last_fullsized":"2024-05-28T13:59:28.472376+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:16:26.601146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056875+0000","last_change":"2024-05-28T13:59:27.465768+0000","last_active":"2024-05-28T13:59:29.056875+0000","last_peered":"2024-05-28T13:59:29.056875+0000","last_clean":"2024-05-28T13:59:29.056875+0000","last_became_active":"2024-05-28T13:59:27.465474+0000","last_became_peered":"2024-05-28T13:59:27.465474+0000","last_unstale":"2024-05-28T13:59:29.056875+0000","last_undegraded":"2024-05-28T13:59:29.056875+0000","last_fullsized":"2024-05-28T13:59:29.056875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:06:38.471358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473293+0000","last_change":"2024-05-28T13:59:27.490188+0000","last_active":"2024-05-28T13:59:28.473293+0000","last_peered":"2024-05-28T13:59:28.473293+0000","last_clean":"2024-05-28T13:59:28.473293+0000","last_became_active":"2024-05-28T13:59:27.489876+0000","last_became_peered":"2024-05-28T13:59:27.489876+0000","last_unstale":"2024-05-28T13:59:28.473293+0000","last_undegraded":"2024-05-28T13:59:28.473293+0000","last_fullsized":"2024-05-28T13:59:28.473293+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:04:33.810167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057606+0000","last_change":"2024-05-28T13:59:27.475535+0000","last_active":"2024-05-28T13:59:29.057606+0000","last_peered":"2024-05-28T13:59:29.057606+0000","last_clean":"2024-05-28T13:59:29.057606+0000","last_became_active":"2024-05-28T13:59:27.475249+0000","last_became_peered":"2024-05-28T13:59:27.475249+0000","last_unstale":"2024-05-28T13:59:29.057606+0000","last_undegraded":"2024-05-28T13:59:29.057606+0000","last_fullsized":"2024-05-28T13:59:29.057606+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:13:14.315389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056421+0000","last_change":"2024-05-28T13:59:27.468303+0000","last_active":"2024-05-28T13:59:29.056421+0000","last_peered":"2024-05-28T13:59:29.056421+0000","last_clean":"2024-05-28T13:59:29.056421+0000","last_became_active":"2024-05-28T13:59:27.467958+0000","last_became_peered":"2024-05-28T13:59:27.467958+0000","last_unstale":"2024-05-28T13:59:29.056421+0000","last_undegraded":"2024-05-28T13:59:29.056421+0000","last_fullsized":"2024-05-28T13:59:29.056421+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:10:43.129378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469624+0000","last_change":"2024-05-28T13:59:27.473200+0000","last_active":"2024-05-28T13:59:28.469624+0000","last_peered":"2024-05-28T13:59:28.469624+0000","last_clean":"2024-05-28T13:59:28.469624+0000","last_became_active":"2024-05-28T13:59:27.472936+0000","last_became_peered":"2024-05-28T13:59:27.472936+0000","last_unstale":"2024-05-28T13:59:28.469624+0000","last_undegraded":"2024-05-28T13:59:28.469624+0000","last_fullsized":"2024-05-28T13:59:28.469624+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:17:00.358853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057112+0000","last_change":"2024-05-28T13:59:27.472137+0000","last_active":"2024-05-28T13:59:29.057112+0000","last_peered":"2024-05-28T13:59:29.057112+0000","last_clean":"2024-05-28T13:59:29.057112+0000","last_became_active":"2024-05-28T13:59:27.471877+0000","last_became_peered":"2024-05-28T13:59:27.471877+0000","last_unstale":"2024-05-28T13:59:29.057112+0000","last_undegraded":"2024-05-28T13:59:29.057112+0000","last_fullsized":"2024-05-28T13:59:29.057112+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:21:32.285503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473519+0000","last_change":"2024-05-28T13:59:27.487690+0000","last_active":"2024-05-28T13:59:28.473519+0000","last_peered":"2024-05-28T13:59:28.473519+0000","last_clean":"2024-05-28T13:59:28.473519+0000","last_became_active":"2024-05-28T13:59:27.485340+0000","last_became_peered":"2024-05-28T13:59:27.485340+0000","last_unstale":"2024-05-28T13:59:28.473519+0000","last_undegraded":"2024-05-28T13:59:28.473519+0000","last_fullsized":"2024-05-28T13:59:28.473519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:01:29.710266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473399+0000","last_change":"2024-05-28T13:59:27.493997+0000","last_active":"2024-05-28T13:59:28.473399+0000","last_peered":"2024-05-28T13:59:28.473399+0000","last_clean":"2024-05-28T13:59:28.473399+0000","last_became_active":"2024-05-28T13:59:27.493597+0000","last_became_peered":"2024-05-28T13:59:27.493597+0000","last_unstale":"2024-05-28T13:59:28.473399+0000","last_undegraded":"2024-05-28T13:59:28.473399+0000","last_fullsized":"2024-05-28T13:59:28.473399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:17:00.358853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.470342+0000","last_change":"2024-05-28T13:59:27.467719+0000","last_active":"2024-05-28T13:59:28.470342+0000","last_peered":"2024-05-28T13:59:28.470342+0000","last_clean":"2024-05-28T13:59:28.470342+0000","last_became_active":"2024-05-28T13:59:27.467403+0000","last_became_peered":"2024-05-28T13:59:27.467403+0000","last_unstale":"2024-05-28T13:59:28.470342+0000","last_undegraded":"2024-05-28T13:59:28.470342+0000","last_fullsized":"2024-05-28T13:59:28.470342+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:40.078224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.470128+0000","last_change":"2024-05-28T13:59:27.468124+0000","last_active":"2024-05-28T13:59:28.470128+0000","last_peered":"2024-05-28T13:59:28.470128+0000","last_clean":"2024-05-28T13:59:28.470128+0000","last_became_active":"2024-05-28T13:59:27.467867+0000","last_became_peered":"2024-05-28T13:59:27.467867+0000","last_unstale":"2024-05-28T13:59:28.470128+0000","last_undegraded":"2024-05-28T13:59:28.470128+0000","last_fullsized":"2024-05-28T13:59:28.470128+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:40:50.994223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056875+0000","last_change":"2024-05-28T13:59:27.477291+0000","last_active":"2024-05-28T13:59:29.056875+0000","last_peered":"2024-05-28T13:59:29.056875+0000","last_clean":"2024-05-28T13:59:29.056875+0000","last_became_active":"2024-05-28T13:59:27.476988+0000","last_became_peered":"2024-05-28T13:59:27.476988+0000","last_unstale":"2024-05-28T13:59:29.056875+0000","last_undegraded":"2024-05-28T13:59:29.056875+0000","last_fullsized":"2024-05-28T13:59:29.056875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:56:41.726103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472487+0000","last_change":"2024-05-28T13:59:27.478881+0000","last_active":"2024-05-28T13:59:28.472487+0000","last_peered":"2024-05-28T13:59:28.472487+0000","last_clean":"2024-05-28T13:59:28.472487+0000","last_became_active":"2024-05-28T13:59:27.478572+0000","last_became_peered":"2024-05-28T13:59:27.478572+0000","last_unstale":"2024-05-28T13:59:28.472487+0000","last_undegraded":"2024-05-28T13:59:28.472487+0000","last_fullsized":"2024-05-28T13:59:28.472487+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:58:31.249506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472169+0000","last_change":"2024-05-28T13:59:27.471276+0000","last_active":"2024-05-28T13:59:28.472169+0000","last_peered":"2024-05-28T13:59:28.472169+0000","last_clean":"2024-05-28T13:59:28.472169+0000","last_became_active":"2024-05-28T13:59:27.470165+0000","last_became_peered":"2024-05-28T13:59:27.470165+0000","last_unstale":"2024-05-28T13:59:28.472169+0000","last_undegraded":"2024-05-28T13:59:28.472169+0000","last_fullsized":"2024-05-28T13:59:28.472169+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:39.802345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.055769+0000","last_change":"2024-05-28T13:59:27.474945+0000","last_active":"2024-05-28T13:59:29.055769+0000","last_peered":"2024-05-28T13:59:29.055769+0000","last_clean":"2024-05-28T13:59:29.055769+0000","last_became_active":"2024-05-28T13:59:27.474483+0000","last_became_peered":"2024-05-28T13:59:27.474483+0000","last_unstale":"2024-05-28T13:59:29.055769+0000","last_undegraded":"2024-05-28T13:59:29.055769+0000","last_fullsized":"2024-05-28T13:59:29.055769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:39.802345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056879+0000","last_change":"2024-05-28T13:59:27.468399+0000","last_active":"2024-05-28T13:59:29.056879+0000","last_peered":"2024-05-28T13:59:29.056879+0000","last_clean":"2024-05-28T13:59:29.056879+0000","last_became_active":"2024-05-28T13:59:27.468083+0000","last_became_peered":"2024-05-28T13:59:27.468083+0000","last_unstale":"2024-05-28T13:59:29.056879+0000","last_undegraded":"2024-05-28T13:59:29.056879+0000","last_fullsized":"2024-05-28T13:59:29.056879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:26:42.728298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057005+0000","last_change":"2024-05-28T13:59:27.485177+0000","last_active":"2024-05-28T13:59:29.057005+0000","last_peered":"2024-05-28T13:59:29.057005+0000","last_clean":"2024-05-28T13:59:29.057005+0000","last_became_active":"2024-05-28T13:59:27.484565+0000","last_became_peered":"2024-05-28T13:59:27.484565+0000","last_unstale":"2024-05-28T13:59:29.057005+0000","last_undegraded":"2024-05-28T13:59:29.057005+0000","last_fullsized":"2024-05-28T13:59:29.057005+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:08:14.981461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.470449+0000","last_change":"2024-05-28T13:59:27.484860+0000","last_active":"2024-05-28T13:59:28.470449+0000","last_peered":"2024-05-28T13:59:28.470449+0000","last_clean":"2024-05-28T13:59:28.470449+0000","last_became_active":"2024-05-28T13:59:27.484433+0000","last_became_peered":"2024-05-28T13:59:27.484433+0000","last_unstale":"2024-05-28T13:59:28.470449+0000","last_undegraded":"2024-05-28T13:59:28.470449+0000","last_fullsized":"2024-05-28T13:59:28.470449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:32:29.021149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056447+0000","last_change":"2024-05-28T13:59:27.472965+0000","last_active":"2024-05-28T13:59:29.056447+0000","last_peered":"2024-05-28T13:59:29.056447+0000","last_clean":"2024-05-28T13:59:29.056447+0000","last_became_active":"2024-05-28T13:59:27.472683+0000","last_became_peered":"2024-05-28T13:59:27.472683+0000","last_unstale":"2024-05-28T13:59:29.056447+0000","last_undegraded":"2024-05-28T13:59:29.056447+0000","last_fullsized":"2024-05-28T13:59:29.056447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:01:36.630567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057856+0000","last_change":"2024-05-28T13:59:27.477064+0000","last_active":"2024-05-28T13:59:29.057856+0000","last_peered":"2024-05-28T13:59:29.057856+0000","last_clean":"2024-05-28T13:59:29.057856+0000","last_became_active":"2024-05-28T13:59:27.476756+0000","last_became_peered":"2024-05-28T13:59:27.476756+0000","last_unstale":"2024-05-28T13:59:29.057856+0000","last_undegraded":"2024-05-28T13:59:29.057856+0000","last_fullsized":"2024-05-28T13:59:29.057856+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:01:29.710266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472650+0000","last_change":"2024-05-28T13:59:27.465607+0000","last_active":"2024-05-28T13:59:28.472650+0000","last_peered":"2024-05-28T13:59:28.472650+0000","last_clean":"2024-05-28T13:59:28.472650+0000","last_became_active":"2024-05-28T13:59:27.465262+0000","last_became_peered":"2024-05-28T13:59:27.465262+0000","last_unstale":"2024-05-28T13:59:28.472650+0000","last_undegraded":"2024-05-28T13:59:28.472650+0000","last_fullsized":"2024-05-28T13:59:28.472650+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:01:24.296794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473593+0000","last_change":"2024-05-28T13:59:27.488169+0000","last_active":"2024-05-28T13:59:28.473593+0000","last_peered":"2024-05-28T13:59:28.473593+0000","last_clean":"2024-05-28T13:59:28.473593+0000","last_became_active":"2024-05-28T13:59:27.487907+0000","last_became_peered":"2024-05-28T13:59:27.487907+0000","last_unstale":"2024-05-28T13:59:28.473593+0000","last_undegraded":"2024-05-28T13:59:28.473593+0000","last_fullsized":"2024-05-28T13:59:28.473593+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:40.078224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057201+0000","last_change":"2024-05-28T13:59:27.480083+0000","last_active":"2024-05-28T13:59:29.057201+0000","last_peered":"2024-05-28T13:59:29.057201+0000","last_clean":"2024-05-28T13:59:29.057201+0000","last_became_active":"2024-05-28T13:59:27.479797+0000","last_became_peered":"2024-05-28T13:59:27.479797+0000","last_unstale":"2024-05-28T13:59:29.057201+0000","last_undegraded":"2024-05-28T13:59:29.057201+0000","last_fullsized":"2024-05-28T13:59:29.057201+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:11:08.637868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473109+0000","last_change":"2024-05-28T13:59:27.494251+0000","last_active":"2024-05-28T13:59:28.473109+0000","last_peered":"2024-05-28T13:59:28.473109+0000","last_clean":"2024-05-28T13:59:28.473109+0000","last_became_active":"2024-05-28T13:59:27.493726+0000","last_became_peered":"2024-05-28T13:59:27.493726+0000","last_unstale":"2024-05-28T13:59:28.473109+0000","last_undegraded":"2024-05-28T13:59:28.473109+0000","last_fullsized":"2024-05-28T13:59:28.473109+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:08:14.981461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.055959+0000","last_change":"2024-05-28T13:59:27.487065+0000","last_active":"2024-05-28T13:59:29.055959+0000","last_peered":"2024-05-28T13:59:29.055959+0000","last_clean":"2024-05-28T13:59:29.055959+0000","last_became_active":"2024-05-28T13:59:27.486748+0000","last_became_peered":"2024-05-28T13:59:27.486748+0000","last_unstale":"2024-05-28T13:59:29.055959+0000","last_undegraded":"2024-05-28T13:59:29.055959+0000","last_fullsized":"2024-05-28T13:59:29.055959+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:24:17.371296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057455+0000","last_change":"2024-05-28T13:59:26.457091+0000","last_active":"2024-05-28T13:59:29.057455+0000","last_peered":"2024-05-28T13:59:29.057455+0000","last_clean":"2024-05-28T13:59:29.057455+0000","last_became_active":"2024-05-28T13:59:26.456803+0000","last_became_peered":"2024-05-28T13:59:26.456803+0000","last_unstale":"2024-05-28T13:59:29.057455+0000","last_undegraded":"2024-05-28T13:59:29.057455+0000","last_fullsized":"2024-05-28T13:59:29.057455+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:10:42.124255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057363+0000","last_change":"2024-05-28T13:59:27.484036+0000","last_active":"2024-05-28T13:59:29.057363+0000","last_peered":"2024-05-28T13:59:29.057363+0000","last_clean":"2024-05-28T13:59:29.057363+0000","last_became_active":"2024-05-28T13:59:27.483763+0000","last_became_peered":"2024-05-28T13:59:27.483763+0000","last_unstale":"2024-05-28T13:59:29.057363+0000","last_undegraded":"2024-05-28T13:59:29.057363+0000","last_fullsized":"2024-05-28T13:59:29.057363+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:59:25.794233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056695+0000","last_change":"2024-05-28T13:59:26.462516+0000","last_active":"2024-05-28T13:59:29.056695+0000","last_peered":"2024-05-28T13:59:29.056695+0000","last_clean":"2024-05-28T13:59:29.056695+0000","last_became_active":"2024-05-28T13:59:26.462205+0000","last_became_peered":"2024-05-28T13:59:26.462205+0000","last_unstale":"2024-05-28T13:59:29.056695+0000","last_undegraded":"2024-05-28T13:59:29.056695+0000","last_fullsized":"2024-05-28T13:59:29.056695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:51:03.448869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473028+0000","last_change":"2024-05-28T13:59:27.470403+0000","last_active":"2024-05-28T13:59:28.473028+0000","last_peered":"2024-05-28T13:59:28.473028+0000","last_clean":"2024-05-28T13:59:28.473028+0000","last_became_active":"2024-05-28T13:59:27.469358+0000","last_became_peered":"2024-05-28T13:59:27.469358+0000","last_unstale":"2024-05-28T13:59:28.473028+0000","last_undegraded":"2024-05-28T13:59:28.473028+0000","last_fullsized":"2024-05-28T13:59:28.473028+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:18:11.375412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.470093+0000","last_change":"2024-05-28T13:59:26.457801+0000","last_active":"2024-05-28T13:59:28.470093+0000","last_peered":"2024-05-28T13:59:28.470093+0000","last_clean":"2024-05-28T13:59:28.470093+0000","last_became_active":"2024-05-28T13:59:26.457432+0000","last_became_peered":"2024-05-28T13:59:26.457432+0000","last_unstale":"2024-05-28T13:59:28.470093+0000","last_undegraded":"2024-05-28T13:59:28.470093+0000","last_fullsized":"2024-05-28T13:59:28.470093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:38.797222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056565+0000","last_change":"2024-05-28T13:59:27.473321+0000","last_active":"2024-05-28T13:59:29.056565+0000","last_peered":"2024-05-28T13:59:29.056565+0000","last_clean":"2024-05-28T13:59:29.056565+0000","last_became_active":"2024-05-28T13:59:27.473006+0000","last_became_peered":"2024-05-28T13:59:27.473006+0000","last_unstale":"2024-05-28T13:59:29.056565+0000","last_undegraded":"2024-05-28T13:59:29.056565+0000","last_fullsized":"2024-05-28T13:59:29.056565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:11:24.829033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056730+0000","last_change":"2024-05-28T13:59:26.461782+0000","last_active":"2024-05-28T13:59:29.056730+0000","last_peered":"2024-05-28T13:59:29.056730+0000","last_clean":"2024-05-28T13:59:29.056730+0000","last_became_active":"2024-05-28T13:59:26.461315+0000","last_became_peered":"2024-05-28T13:59:26.461315+0000","last_unstale":"2024-05-28T13:59:29.056730+0000","last_undegraded":"2024-05-28T13:59:29.056730+0000","last_fullsized":"2024-05-28T13:59:29.056730+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:29:57.955171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473552+0000","last_change":"2024-05-28T13:59:27.493269+0000","last_active":"2024-05-28T13:59:28.473552+0000","last_peered":"2024-05-28T13:59:28.473552+0000","last_clean":"2024-05-28T13:59:28.473552+0000","last_became_active":"2024-05-28T13:59:27.492980+0000","last_became_peered":"2024-05-28T13:59:27.492980+0000","last_unstale":"2024-05-28T13:59:28.473552+0000","last_undegraded":"2024-05-28T13:59:28.473552+0000","last_fullsized":"2024-05-28T13:59:28.473552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:40:50.994223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472754+0000","last_change":"2024-05-28T13:59:26.455974+0000","last_active":"2024-05-28T13:59:28.472754+0000","last_peered":"2024-05-28T13:59:28.472754+0000","last_clean":"2024-05-28T13:59:28.472754+0000","last_became_active":"2024-05-28T13:59:26.455680+0000","last_became_peered":"2024-05-28T13:59:26.455680+0000","last_unstale":"2024-05-28T13:59:28.472754+0000","last_undegraded":"2024-05-28T13:59:28.472754+0000","last_fullsized":"2024-05-28T13:59:28.472754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:14:43.321207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057552+0000","last_change":"2024-05-28T13:59:27.472454+0000","last_active":"2024-05-28T13:59:29.057552+0000","last_peered":"2024-05-28T13:59:29.057552+0000","last_clean":"2024-05-28T13:59:29.057552+0000","last_became_active":"2024-05-28T13:59:27.472107+0000","last_became_peered":"2024-05-28T13:59:27.472107+0000","last_unstale":"2024-05-28T13:59:29.057552+0000","last_undegraded":"2024-05-28T13:59:29.057552+0000","last_fullsized":"2024-05-28T13:59:29.057552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:04:33.810167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472283+0000","last_change":"2024-05-28T13:59:26.455099+0000","last_active":"2024-05-28T13:59:28.472283+0000","last_peered":"2024-05-28T13:59:28.472283+0000","last_clean":"2024-05-28T13:59:28.472283+0000","last_became_active":"2024-05-28T13:59:26.454339+0000","last_became_peered":"2024-05-28T13:59:26.454339+0000","last_unstale":"2024-05-28T13:59:28.472283+0000","last_undegraded":"2024-05-28T13:59:28.472283+0000","last_fullsized":"2024-05-28T13:59:28.472283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:14:19.236068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.471613+0000","last_change":"2024-05-28T13:59:27.478265+0000","last_active":"2024-05-28T13:59:28.471613+0000","last_peered":"2024-05-28T13:59:28.471613+0000","last_clean":"2024-05-28T13:59:28.471613+0000","last_became_active":"2024-05-28T13:59:27.475778+0000","last_became_peered":"2024-05-28T13:59:27.475778+0000","last_unstale":"2024-05-28T13:59:28.471613+0000","last_undegraded":"2024-05-28T13:59:28.471613+0000","last_fullsized":"2024-05-28T13:59:28.471613+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:16.522449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469351+0000","last_change":"2024-05-28T13:59:26.451905+0000","last_active":"2024-05-28T13:59:28.469351+0000","last_peered":"2024-05-28T13:59:28.469351+0000","last_clean":"2024-05-28T13:59:28.469351+0000","last_became_active":"2024-05-28T13:59:26.451616+0000","last_became_peered":"2024-05-28T13:59:26.451616+0000","last_unstale":"2024-05-28T13:59:28.469351+0000","last_undegraded":"2024-05-28T13:59:28.469351+0000","last_fullsized":"2024-05-28T13:59:28.469351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:15.517326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.055876+0000","last_change":"2024-05-28T13:59:27.466715+0000","last_active":"2024-05-28T13:59:29.055876+0000","last_peered":"2024-05-28T13:59:29.055876+0000","last_clean":"2024-05-28T13:59:29.055876+0000","last_became_active":"2024-05-28T13:59:27.466435+0000","last_became_peered":"2024-05-28T13:59:27.466435+0000","last_unstale":"2024-05-28T13:59:29.055876+0000","last_undegraded":"2024-05-28T13:59:29.055876+0000","last_fullsized":"2024-05-28T13:59:29.055876+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:18:11.375412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.471729+0000","last_change":"2024-05-28T13:59:26.452068+0000","last_active":"2024-05-28T13:59:28.471729+0000","last_peered":"2024-05-28T13:59:28.471729+0000","last_clean":"2024-05-28T13:59:28.471729+0000","last_became_active":"2024-05-28T13:59:26.451700+0000","last_became_peered":"2024-05-28T13:59:26.451700+0000","last_unstale":"2024-05-28T13:59:28.471729+0000","last_undegraded":"2024-05-28T13:59:28.471729+0000","last_fullsized":"2024-05-28T13:59:28.471729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:06:03.772836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057857+0000","last_change":"2024-05-28T13:59:27.470828+0000","last_active":"2024-05-28T13:59:29.057857+0000","last_peered":"2024-05-28T13:59:29.057857+0000","last_clean":"2024-05-28T13:59:29.057857+0000","last_became_active":"2024-05-28T13:59:27.470556+0000","last_became_peered":"2024-05-28T13:59:27.470556+0000","last_unstale":"2024-05-28T13:59:29.057857+0000","last_undegraded":"2024-05-28T13:59:29.057857+0000","last_fullsized":"2024-05-28T13:59:29.057857+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:14:36.728353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.470403+0000","last_change":"2024-05-28T13:59:26.455937+0000","last_active":"2024-05-28T13:59:28.470403+0000","last_peered":"2024-05-28T13:59:28.470403+0000","last_clean":"2024-05-28T13:59:28.470403+0000","last_became_active":"2024-05-28T13:59:26.453999+0000","last_became_peered":"2024-05-28T13:59:26.453999+0000","last_unstale":"2024-05-28T13:59:28.470403+0000","last_undegraded":"2024-05-28T13:59:28.470403+0000","last_fullsized":"2024-05-28T13:59:28.470403+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:24:16.366173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.470467+0000","last_change":"2024-05-28T13:59:27.478424+0000","last_active":"2024-05-28T13:59:28.470467+0000","last_peered":"2024-05-28T13:59:28.470467+0000","last_clean":"2024-05-28T13:59:28.470467+0000","last_became_active":"2024-05-28T13:59:27.477986+0000","last_became_peered":"2024-05-28T13:59:27.477986+0000","last_unstale":"2024-05-28T13:59:28.470467+0000","last_undegraded":"2024-05-28T13:59:28.470467+0000","last_fullsized":"2024-05-28T13:59:28.470467+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:22:27.057001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056089+0000","last_change":"2024-05-28T13:59:26.454732+0000","last_active":"2024-05-28T13:59:29.056089+0000","last_peered":"2024-05-28T13:59:29.056089+0000","last_clean":"2024-05-28T13:59:29.056089+0000","last_became_active":"2024-05-28T13:59:26.454392+0000","last_became_peered":"2024-05-28T13:59:26.454392+0000","last_unstale":"2024-05-28T13:59:29.056089+0000","last_undegraded":"2024-05-28T13:59:29.056089+0000","last_fullsized":"2024-05-28T13:59:29.056089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:09:06.066324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469966+0000","last_change":"2024-05-28T13:59:27.484330+0000","last_active":"2024-05-28T13:59:28.469966+0000","last_peered":"2024-05-28T13:59:28.469966+0000","last_clean":"2024-05-28T13:59:28.469966+0000","last_became_active":"2024-05-28T13:59:27.483976+0000","last_became_peered":"2024-05-28T13:59:27.483976+0000","last_unstale":"2024-05-28T13:59:28.469966+0000","last_undegraded":"2024-05-28T13:59:28.469966+0000","last_fullsized":"2024-05-28T13:59:28.469966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:14:36.728353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057933+0000","last_change":"2024-05-28T13:59:26.460226+0000","last_active":"2024-05-28T13:59:29.057933+0000","last_peered":"2024-05-28T13:59:29.057933+0000","last_clean":"2024-05-28T13:59:29.057933+0000","last_became_active":"2024-05-28T13:59:26.459871+0000","last_became_peered":"2024-05-28T13:59:26.459871+0000","last_unstale":"2024-05-28T13:59:29.057933+0000","last_undegraded":"2024-05-28T13:59:29.057933+0000","last_fullsized":"2024-05-28T13:59:29.057933+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:16:25.596023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057743+0000","last_change":"2024-05-28T13:59:27.481776+0000","last_active":"2024-05-28T13:59:29.057743+0000","last_peered":"2024-05-28T13:59:29.057743+0000","last_clean":"2024-05-28T13:59:29.057743+0000","last_became_active":"2024-05-28T13:59:27.481344+0000","last_became_peered":"2024-05-28T13:59:27.481344+0000","last_unstale":"2024-05-28T13:59:29.057743+0000","last_undegraded":"2024-05-28T13:59:29.057743+0000","last_fullsized":"2024-05-28T13:59:29.057743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:17:00.358853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469920+0000","last_change":"2024-05-28T13:59:26.457474+0000","last_active":"2024-05-28T13:59:28.469920+0000","last_peered":"2024-05-28T13:59:28.469920+0000","last_clean":"2024-05-28T13:59:28.469920+0000","last_became_active":"2024-05-28T13:59:26.457177+0000","last_became_peered":"2024-05-28T13:59:26.457177+0000","last_unstale":"2024-05-28T13:59:28.469920+0000","last_undegraded":"2024-05-28T13:59:28.469920+0000","last_fullsized":"2024-05-28T13:59:28.469920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:16:25.596023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.055630+0000","last_change":"2024-05-28T13:59:27.465166+0000","last_active":"2024-05-28T13:59:29.055630+0000","last_peered":"2024-05-28T13:59:29.055630+0000","last_clean":"2024-05-28T13:59:29.055630+0000","last_became_active":"2024-05-28T13:59:27.464834+0000","last_became_peered":"2024-05-28T13:59:27.464834+0000","last_unstale":"2024-05-28T13:59:29.055630+0000","last_undegraded":"2024-05-28T13:59:29.055630+0000","last_fullsized":"2024-05-28T13:59:29.055630+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:01:24.296794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472874+0000","last_change":"2024-05-28T13:59:26.451954+0000","last_active":"2024-05-28T13:59:28.472874+0000","last_peered":"2024-05-28T13:59:28.472874+0000","last_clean":"2024-05-28T13:59:28.472874+0000","last_became_active":"2024-05-28T13:59:26.451673+0000","last_became_peered":"2024-05-28T13:59:26.451673+0000","last_unstale":"2024-05-28T13:59:28.472874+0000","last_undegraded":"2024-05-28T13:59:28.472874+0000","last_fullsized":"2024-05-28T13:59:28.472874+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:45:57.009013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469783+0000","last_change":"2024-05-28T13:59:27.467486+0000","last_active":"2024-05-28T13:59:28.469783+0000","last_peered":"2024-05-28T13:59:28.469783+0000","last_clean":"2024-05-28T13:59:28.469783+0000","last_became_active":"2024-05-28T13:59:27.467189+0000","last_became_peered":"2024-05-28T13:59:27.467189+0000","last_unstale":"2024-05-28T13:59:28.469783+0000","last_undegraded":"2024-05-28T13:59:28.469783+0000","last_fullsized":"2024-05-28T13:59:28.469783+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:01:29.710266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.055594+0000","last_change":"2024-05-28T13:59:26.450030+0000","last_active":"2024-05-28T13:59:29.055594+0000","last_peered":"2024-05-28T13:59:29.055594+0000","last_clean":"2024-05-28T13:59:29.055594+0000","last_became_active":"2024-05-28T13:59:26.449518+0000","last_became_peered":"2024-05-28T13:59:26.449518+0000","last_unstale":"2024-05-28T13:59:29.055594+0000","last_undegraded":"2024-05-28T13:59:29.055594+0000","last_fullsized":"2024-05-28T13:59:29.055594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:56:40.720980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473319+0000","last_change":"2024-05-28T13:59:27.493147+0000","last_active":"2024-05-28T13:59:28.473319+0000","last_peered":"2024-05-28T13:59:28.473319+0000","last_clean":"2024-05-28T13:59:28.473319+0000","last_became_active":"2024-05-28T13:59:27.492868+0000","last_became_peered":"2024-05-28T13:59:27.492868+0000","last_unstale":"2024-05-28T13:59:28.473319+0000","last_undegraded":"2024-05-28T13:59:28.473319+0000","last_fullsized":"2024-05-28T13:59:28.473319+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:13:14.315389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469438+0000","last_change":"2024-05-28T13:59:26.460063+0000","last_active":"2024-05-28T13:59:28.469438+0000","last_peered":"2024-05-28T13:59:28.469438+0000","last_clean":"2024-05-28T13:59:28.469438+0000","last_became_active":"2024-05-28T13:59:26.459777+0000","last_became_peered":"2024-05-28T13:59:26.459777+0000","last_unstale":"2024-05-28T13:59:28.469438+0000","last_undegraded":"2024-05-28T13:59:28.469438+0000","last_fullsized":"2024-05-28T13:59:28.469438+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:59:24.789111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057000+0000","last_change":"2024-05-28T13:59:27.485209+0000","last_active":"2024-05-28T13:59:29.057000+0000","last_peered":"2024-05-28T13:59:29.057000+0000","last_clean":"2024-05-28T13:59:29.057000+0000","last_became_active":"2024-05-28T13:59:27.484797+0000","last_became_peered":"2024-05-28T13:59:27.484797+0000","last_unstale":"2024-05-28T13:59:29.057000+0000","last_undegraded":"2024-05-28T13:59:29.057000+0000","last_fullsized":"2024-05-28T13:59:29.057000+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:00:07.636952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469215+0000","last_change":"2024-05-28T13:59:26.455232+0000","last_active":"2024-05-28T13:59:28.469215+0000","last_peered":"2024-05-28T13:59:28.469215+0000","last_clean":"2024-05-28T13:59:28.469215+0000","last_became_active":"2024-05-28T13:59:26.454943+0000","last_became_peered":"2024-05-28T13:59:26.454943+0000","last_unstale":"2024-05-28T13:59:28.469215+0000","last_undegraded":"2024-05-28T13:59:28.469215+0000","last_fullsized":"2024-05-28T13:59:28.469215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:35:08.660538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472481+0000","last_change":"2024-05-28T13:59:27.481270+0000","last_active":"2024-05-28T13:59:28.472481+0000","last_peered":"2024-05-28T13:59:28.472481+0000","last_clean":"2024-05-28T13:59:28.472481+0000","last_became_active":"2024-05-28T13:59:27.480948+0000","last_became_peered":"2024-05-28T13:59:27.480948+0000","last_unstale":"2024-05-28T13:59:28.472481+0000","last_undegraded":"2024-05-28T13:59:28.472481+0000","last_fullsized":"2024-05-28T13:59:28.472481+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:24:17.371296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057108+0000","last_change":"2024-05-28T13:59:26.463761+0000","last_active":"2024-05-28T13:59:29.057108+0000","last_peered":"2024-05-28T13:59:29.057108+0000","last_clean":"2024-05-28T13:59:29.057108+0000","last_became_active":"2024-05-28T13:59:26.463398+0000","last_became_peered":"2024-05-28T13:59:26.463398+0000","last_unstale":"2024-05-28T13:59:29.057108+0000","last_undegraded":"2024-05-28T13:59:29.057108+0000","last_fullsized":"2024-05-28T13:59:29.057108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:51:03.448869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057961+0000","last_change":"2024-05-28T13:59:27.480086+0000","last_active":"2024-05-28T13:59:29.057961+0000","last_peered":"2024-05-28T13:59:29.057961+0000","last_clean":"2024-05-28T13:59:29.057961+0000","last_became_active":"2024-05-28T13:59:27.479834+0000","last_became_peered":"2024-05-28T13:59:27.479834+0000","last_unstale":"2024-05-28T13:59:29.057961+0000","last_undegraded":"2024-05-28T13:59:29.057961+0000","last_fullsized":"2024-05-28T13:59:29.057961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:40:50.994223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472547+0000","last_change":"2024-05-28T13:59:26.445326+0000","last_active":"2024-05-28T13:59:28.472547+0000","last_peered":"2024-05-28T13:59:28.472547+0000","last_clean":"2024-05-28T13:59:28.472547+0000","last_became_active":"2024-05-28T13:59:26.445038+0000","last_became_peered":"2024-05-28T13:59:26.445038+0000","last_unstale":"2024-05-28T13:59:28.472547+0000","last_undegraded":"2024-05-28T13:59:28.472547+0000","last_fullsized":"2024-05-28T13:59:28.472547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:21.322022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056214+0000","last_change":"2024-05-28T13:59:27.471440+0000","last_active":"2024-05-28T13:59:29.056214+0000","last_peered":"2024-05-28T13:59:29.056214+0000","last_clean":"2024-05-28T13:59:29.056214+0000","last_became_active":"2024-05-28T13:59:27.471141+0000","last_became_peered":"2024-05-28T13:59:27.471141+0000","last_unstale":"2024-05-28T13:59:29.056214+0000","last_undegraded":"2024-05-28T13:59:29.056214+0000","last_fullsized":"2024-05-28T13:59:29.056214+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:35:52.842901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473047+0000","last_change":"2024-05-28T13:59:26.458570+0000","last_active":"2024-05-28T13:59:28.473047+0000","last_peered":"2024-05-28T13:59:28.473047+0000","last_clean":"2024-05-28T13:59:28.473047+0000","last_became_active":"2024-05-28T13:59:26.458216+0000","last_became_peered":"2024-05-28T13:59:26.458216+0000","last_unstale":"2024-05-28T13:59:28.473047+0000","last_undegraded":"2024-05-28T13:59:28.473047+0000","last_fullsized":"2024-05-28T13:59:28.473047+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:54:53.495272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056809+0000","last_change":"2024-05-28T13:59:27.488471+0000","last_active":"2024-05-28T13:59:29.056809+0000","last_peered":"2024-05-28T13:59:29.056809+0000","last_clean":"2024-05-28T13:59:29.056809+0000","last_became_active":"2024-05-28T13:59:27.488091+0000","last_became_peered":"2024-05-28T13:59:27.488091+0000","last_unstale":"2024-05-28T13:59:29.056809+0000","last_undegraded":"2024-05-28T13:59:29.056809+0000","last_fullsized":"2024-05-28T13:59:29.056809+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:45:29.329612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473068+0000","last_change":"2024-05-28T13:59:26.455262+0000","last_active":"2024-05-28T13:59:28.473068+0000","last_peered":"2024-05-28T13:59:28.473068+0000","last_clean":"2024-05-28T13:59:28.473068+0000","last_became_active":"2024-05-28T13:59:26.453974+0000","last_became_peered":"2024-05-28T13:59:26.453974+0000","last_unstale":"2024-05-28T13:59:28.473068+0000","last_undegraded":"2024-05-28T13:59:28.473068+0000","last_fullsized":"2024-05-28T13:59:28.473068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:55:44.250628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056379+0000","last_change":"2024-05-28T13:59:27.476482+0000","last_active":"2024-05-28T13:59:29.056379+0000","last_peered":"2024-05-28T13:59:29.056379+0000","last_clean":"2024-05-28T13:59:29.056379+0000","last_became_active":"2024-05-28T13:59:27.475253+0000","last_became_peered":"2024-05-28T13:59:27.475253+0000","last_unstale":"2024-05-28T13:59:29.056379+0000","last_undegraded":"2024-05-28T13:59:29.056379+0000","last_fullsized":"2024-05-28T13:59:29.056379+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:23:17.062435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056081+0000","last_change":"2024-05-28T13:59:26.460366+0000","last_active":"2024-05-28T13:59:29.056081+0000","last_peered":"2024-05-28T13:59:29.056081+0000","last_clean":"2024-05-28T13:59:29.056081+0000","last_became_active":"2024-05-28T13:59:26.460168+0000","last_became_peered":"2024-05-28T13:59:26.460168+0000","last_unstale":"2024-05-28T13:59:29.056081+0000","last_undegraded":"2024-05-28T13:59:29.056081+0000","last_fullsized":"2024-05-28T13:59:29.056081+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:45:28.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056571+0000","last_change":"2024-05-28T13:59:27.464552+0000","last_active":"2024-05-28T13:59:29.056571+0000","last_peered":"2024-05-28T13:59:29.056571+0000","last_clean":"2024-05-28T13:59:29.056571+0000","last_became_active":"2024-05-28T13:59:27.463984+0000","last_became_peered":"2024-05-28T13:59:27.463984+0000","last_unstale":"2024-05-28T13:59:29.056571+0000","last_undegraded":"2024-05-28T13:59:29.056571+0000","last_fullsized":"2024-05-28T13:59:29.056571+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:09:29.776916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057203+0000","last_change":"2024-05-28T13:59:26.455086+0000","last_active":"2024-05-28T13:59:29.057203+0000","last_peered":"2024-05-28T13:59:29.057203+0000","last_clean":"2024-05-28T13:59:29.057203+0000","last_became_active":"2024-05-28T13:59:26.454817+0000","last_became_peered":"2024-05-28T13:59:26.454817+0000","last_unstale":"2024-05-28T13:59:29.057203+0000","last_undegraded":"2024-05-28T13:59:29.057203+0000","last_fullsized":"2024-05-28T13:59:29.057203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:23:16.057312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056751+0000","last_change":"2024-05-28T13:59:27.469324+0000","last_active":"2024-05-28T13:59:29.056751+0000","last_peered":"2024-05-28T13:59:29.056751+0000","last_clean":"2024-05-28T13:59:29.056751+0000","last_became_active":"2024-05-28T13:59:27.466052+0000","last_became_peered":"2024-05-28T13:59:27.466052+0000","last_unstale":"2024-05-28T13:59:29.056751+0000","last_undegraded":"2024-05-28T13:59:29.056751+0000","last_fullsized":"2024-05-28T13:59:29.056751+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:51:04.453992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.471727+0000","last_change":"2024-05-28T13:59:26.448462+0000","last_active":"2024-05-28T13:59:28.471727+0000","last_peered":"2024-05-28T13:59:28.471727+0000","last_clean":"2024-05-28T13:59:28.471727+0000","last_became_active":"2024-05-28T13:59:26.448212+0000","last_became_peered":"2024-05-28T13:59:26.448212+0000","last_unstale":"2024-05-28T13:59:28.471727+0000","last_undegraded":"2024-05-28T13:59:28.471727+0000","last_fullsized":"2024-05-28T13:59:28.471727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:00:47.834887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473296+0000","last_change":"2024-05-28T13:59:27.484434+0000","last_active":"2024-05-28T13:59:28.473296+0000","last_peered":"2024-05-28T13:59:28.473296+0000","last_clean":"2024-05-28T13:59:28.473296+0000","last_became_active":"2024-05-28T13:59:27.483863+0000","last_became_peered":"2024-05-28T13:59:27.483863+0000","last_unstale":"2024-05-28T13:59:28.473296+0000","last_undegraded":"2024-05-28T13:59:28.473296+0000","last_fullsized":"2024-05-28T13:59:28.473296+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:21:32.285503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056998+0000","last_change":"2024-05-28T13:59:26.456427+0000","last_active":"2024-05-28T13:59:29.056998+0000","last_peered":"2024-05-28T13:59:29.056998+0000","last_clean":"2024-05-28T13:59:29.056998+0000","last_became_active":"2024-05-28T13:59:26.455597+0000","last_became_peered":"2024-05-28T13:59:26.455597+0000","last_unstale":"2024-05-28T13:59:29.056998+0000","last_undegraded":"2024-05-28T13:59:29.056998+0000","last_fullsized":"2024-05-28T13:59:29.056998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:38.797222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056796+0000","last_change":"2024-05-28T13:59:27.470381+0000","last_active":"2024-05-28T13:59:29.056796+0000","last_peered":"2024-05-28T13:59:29.056796+0000","last_clean":"2024-05-28T13:59:29.056796+0000","last_became_active":"2024-05-28T13:59:27.469832+0000","last_became_peered":"2024-05-28T13:59:27.469832+0000","last_unstale":"2024-05-28T13:59:29.056796+0000","last_undegraded":"2024-05-28T13:59:29.056796+0000","last_fullsized":"2024-05-28T13:59:29.056796+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:09:07.071447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057094+0000","last_change":"2024-05-28T13:59:26.459166+0000","last_active":"2024-05-28T13:59:29.057094+0000","last_peered":"2024-05-28T13:59:29.057094+0000","last_clean":"2024-05-28T13:59:29.057094+0000","last_became_active":"2024-05-28T13:59:26.458777+0000","last_became_peered":"2024-05-28T13:59:26.458777+0000","last_unstale":"2024-05-28T13:59:29.057094+0000","last_undegraded":"2024-05-28T13:59:29.057094+0000","last_fullsized":"2024-05-28T13:59:29.057094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:09:06.066324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057974+0000","last_change":"2024-05-28T13:59:27.482817+0000","last_active":"2024-05-28T13:59:29.057974+0000","last_peered":"2024-05-28T13:59:29.057974+0000","last_clean":"2024-05-28T13:59:29.057974+0000","last_became_active":"2024-05-28T13:59:27.481904+0000","last_became_peered":"2024-05-28T13:59:27.481904+0000","last_unstale":"2024-05-28T13:59:29.057974+0000","last_undegraded":"2024-05-28T13:59:29.057974+0000","last_fullsized":"2024-05-28T13:59:29.057974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:46.412620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.455904+0000","last_change":"2024-05-28T13:59:26.459384+0000","last_active":"2024-05-28T13:59:29.455904+0000","last_peered":"2024-05-28T13:59:29.455904+0000","last_clean":"2024-05-28T13:59:29.455904+0000","last_became_active":"2024-05-28T13:59:26.459117+0000","last_became_peered":"2024-05-28T13:59:26.459117+0000","last_unstale":"2024-05-28T13:59:29.455904+0000","last_undegraded":"2024-05-28T13:59:29.455904+0000","last_fullsized":"2024-05-28T13:59:29.455904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:56:40.720980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.055831+0000","last_change":"2024-05-28T13:59:27.473996+0000","last_active":"2024-05-28T13:59:29.055831+0000","last_peered":"2024-05-28T13:59:29.055831+0000","last_clean":"2024-05-28T13:59:29.055831+0000","last_became_active":"2024-05-28T13:59:27.473702+0000","last_became_peered":"2024-05-28T13:59:27.473702+0000","last_unstale":"2024-05-28T13:59:29.055831+0000","last_undegraded":"2024-05-28T13:59:29.055831+0000","last_fullsized":"2024-05-28T13:59:29.055831+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:16:26.601146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056397+0000","last_change":"2024-05-28T13:59:26.453566+0000","last_active":"2024-05-28T13:59:29.056397+0000","last_peered":"2024-05-28T13:59:29.056397+0000","last_clean":"2024-05-28T13:59:29.056397+0000","last_became_active":"2024-05-28T13:59:26.453166+0000","last_became_peered":"2024-05-28T13:59:26.453166+0000","last_unstale":"2024-05-28T13:59:29.056397+0000","last_undegraded":"2024-05-28T13:59:29.056397+0000","last_fullsized":"2024-05-28T13:59:29.056397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:45:28.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057163+0000","last_change":"2024-05-28T13:59:27.478988+0000","last_active":"2024-05-28T13:59:29.057163+0000","last_peered":"2024-05-28T13:59:29.057163+0000","last_clean":"2024-05-28T13:59:29.057163+0000","last_became_active":"2024-05-28T13:59:27.478706+0000","last_became_peered":"2024-05-28T13:59:27.478706+0000","last_unstale":"2024-05-28T13:59:29.057163+0000","last_undegraded":"2024-05-28T13:59:29.057163+0000","last_fullsized":"2024-05-28T13:59:29.057163+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:35:09.665661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056220+0000","last_change":"2024-05-28T13:59:26.450630+0000","last_active":"2024-05-28T13:59:29.056220+0000","last_peered":"2024-05-28T13:59:29.056220+0000","last_clean":"2024-05-28T13:59:29.056220+0000","last_became_active":"2024-05-28T13:59:26.450335+0000","last_became_peered":"2024-05-28T13:59:26.450335+0000","last_unstale":"2024-05-28T13:59:29.056220+0000","last_undegraded":"2024-05-28T13:59:29.056220+0000","last_fullsized":"2024-05-28T13:59:29.056220+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:00:06.631829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056127+0000","last_change":"2024-05-28T13:59:27.470786+0000","last_active":"2024-05-28T13:59:29.056127+0000","last_peered":"2024-05-28T13:59:29.056127+0000","last_clean":"2024-05-28T13:59:29.056127+0000","last_became_active":"2024-05-28T13:59:27.470519+0000","last_became_peered":"2024-05-28T13:59:27.470519+0000","last_unstale":"2024-05-28T13:59:29.056127+0000","last_undegraded":"2024-05-28T13:59:29.056127+0000","last_fullsized":"2024-05-28T13:59:29.056127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:58:31.249506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.055785+0000","last_change":"2024-05-28T13:59:26.450196+0000","last_active":"2024-05-28T13:59:29.055785+0000","last_peered":"2024-05-28T13:59:29.055785+0000","last_clean":"2024-05-28T13:59:29.055785+0000","last_became_active":"2024-05-28T13:59:26.449781+0000","last_became_peered":"2024-05-28T13:59:26.449781+0000","last_unstale":"2024-05-28T13:59:29.055785+0000","last_undegraded":"2024-05-28T13:59:29.055785+0000","last_fullsized":"2024-05-28T13:59:29.055785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:00:06.631829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469223+0000","last_change":"2024-05-28T13:59:27.467807+0000","last_active":"2024-05-28T13:59:28.469223+0000","last_peered":"2024-05-28T13:59:28.469223+0000","last_clean":"2024-05-28T13:59:28.469223+0000","last_became_active":"2024-05-28T13:59:27.467575+0000","last_became_peered":"2024-05-28T13:59:27.467575+0000","last_unstale":"2024-05-28T13:59:28.469223+0000","last_undegraded":"2024-05-28T13:59:28.469223+0000","last_fullsized":"2024-05-28T13:59:28.469223+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:04:33.810167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.471846+0000","last_change":"2024-05-28T13:59:26.445766+0000","last_active":"2024-05-28T13:59:28.471846+0000","last_peered":"2024-05-28T13:59:28.471846+0000","last_clean":"2024-05-28T13:59:28.471846+0000","last_became_active":"2024-05-28T13:59:26.444829+0000","last_became_peered":"2024-05-28T13:59:26.444829+0000","last_unstale":"2024-05-28T13:59:28.471846+0000","last_undegraded":"2024-05-28T13:59:28.471846+0000","last_fullsized":"2024-05-28T13:59:28.471846+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:23:16.057312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.471614+0000","last_change":"2024-05-28T13:59:27.465500+0000","last_active":"2024-05-28T13:59:28.471614+0000","last_peered":"2024-05-28T13:59:28.471614+0000","last_clean":"2024-05-28T13:59:28.471614+0000","last_became_active":"2024-05-28T13:59:27.465185+0000","last_became_peered":"2024-05-28T13:59:27.465185+0000","last_unstale":"2024-05-28T13:59:28.471614+0000","last_undegraded":"2024-05-28T13:59:28.471614+0000","last_fullsized":"2024-05-28T13:59:28.471614+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:59:25.794233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469268+0000","last_change":"2024-05-28T13:59:26.455269+0000","last_active":"2024-05-28T13:59:28.469268+0000","last_peered":"2024-05-28T13:59:28.469268+0000","last_clean":"2024-05-28T13:59:28.469268+0000","last_became_active":"2024-05-28T13:59:26.454950+0000","last_became_peered":"2024-05-28T13:59:26.454950+0000","last_unstale":"2024-05-28T13:59:28.469268+0000","last_undegraded":"2024-05-28T13:59:28.469268+0000","last_fullsized":"2024-05-28T13:59:28.469268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:11:07.632745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473553+0000","last_change":"2024-05-28T13:59:27.484463+0000","last_active":"2024-05-28T13:59:28.473553+0000","last_peered":"2024-05-28T13:59:28.473553+0000","last_clean":"2024-05-28T13:59:28.473553+0000","last_became_active":"2024-05-28T13:59:27.484154+0000","last_became_peered":"2024-05-28T13:59:27.484154+0000","last_unstale":"2024-05-28T13:59:28.473553+0000","last_undegraded":"2024-05-28T13:59:28.473553+0000","last_fullsized":"2024-05-28T13:59:28.473553+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:46.412620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056963+0000","last_change":"2024-05-28T13:59:26.451972+0000","last_active":"2024-05-28T13:59:29.056963+0000","last_peered":"2024-05-28T13:59:29.056963+0000","last_clean":"2024-05-28T13:59:29.056963+0000","last_became_active":"2024-05-28T13:59:26.451403+0000","last_became_peered":"2024-05-28T13:59:26.451403+0000","last_unstale":"2024-05-28T13:59:29.056963+0000","last_undegraded":"2024-05-28T13:59:29.056963+0000","last_fullsized":"2024-05-28T13:59:29.056963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:00:47.834887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473416+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T13:59:28.473416+0000","last_peered":"2024-05-28T13:59:28.473416+0000","last_clean":"2024-05-28T13:59:28.473416+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T13:59:28.473416+0000","last_undegraded":"2024-05-28T13:59:28.473416+0000","last_fullsized":"2024-05-28T13:59:28.473416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":741737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":151951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":152913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":376832,"data_stored":145049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":152913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":978944,"data_stored":741647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T13:59:31.565 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T13:59:31.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7683200640 1 Processor -- start 2024-05-28T13:59:31.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7683200640 1 -- start start 2024-05-28T13:59:31.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7683200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0450 0x7f768409b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7683200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f768415d680 0x7f76840b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7683200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76840d0160 con 0x7f768415c880 2024-05-28T13:59:31.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7683200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f768409b520 con 0x7f768415d680 2024-05-28T13:59:31.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7683200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f768409abf0 con 0x7f76840d0450 2024-05-28T13:59:31.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7682800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f768415d680 0x7f76840b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7681e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0450 0x7f768409b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7682800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f768415d680 0x7f76840b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:54078/0 (socket says 172.21.15.42:54078) 2024-05-28T13:59:31.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7682800640 1 -- 172.21.15.42:0/4087740548 learned_addr learned my addr 172.21.15.42:0/4087740548 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:31.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7682800640 1 -- 172.21.15.42:0/4087740548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0450 msgr2=0x7f768409b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:31.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7682800640 1 --2- 172.21.15.42:0/4087740548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0450 0x7f768409b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7682800640 1 -- 172.21.15.42:0/4087740548 >> v1:172.21.15.63:6789/0 conn(0x7f768415c880 legacy=0x7f76840d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7682800640 1 -- 172.21.15.42:0/4087740548 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768409aed0 con 0x7f768415d680 2024-05-28T13:59:31.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.757+0000 7f7681400640 1 -- 172.21.15.42:0/4087740548 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 820017761 0 0) 0x7f76840d0160 con 0x7f768415c880 2024-05-28T13:59:31.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.761+0000 7f7682800640 1 --2- 172.21.15.42:0/4087740548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768415d680 0x7f76840b16e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f767c009e50 tx=0x7f767c01ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6deedfcd03945144 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:31.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.761+0000 7f7681400640 1 -- 172.21.15.42:0/4087740548 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c0a5790 con 0x7f768415d680 2024-05-28T13:59:31.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.761+0000 7f7681400640 1 -- 172.21.15.42:0/4087740548 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f767c0a5930 con 0x7f768415d680 2024-05-28T13:59:31.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.761+0000 7f7681400640 1 -- 172.21.15.42:0/4087740548 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c0a5d00 con 0x7f768415d680 2024-05-28T13:59:31.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.761+0000 7f7683200640 1 -- 172.21.15.42:0/4087740548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768415d680 msgr2=0x7f76840b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.761+0000 7f7683200640 1 --2- 172.21.15.42:0/4087740548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768415d680 0x7f76840b16e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f767c009e50 tx=0x7f767c01ca30 comp rx=0 tx=0).stop 2024-05-28T13:59:31.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.761+0000 7f7683200640 1 -- 172.21.15.42:0/4087740548 shutdown_connections 2024-05-28T13:59:31.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.761+0000 7f7683200640 1 --2- 172.21.15.42:0/4087740548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768415d680 0x7f76840b16e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.761+0000 7f7683200640 1 --2- 172.21.15.42:0/4087740548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0450 0x7f768409b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.761+0000 7f7683200640 1 -- 172.21.15.42:0/4087740548 >> 172.21.15.42:0/4087740548 conn(0x7f76840b3dd0 msgr2=0x7f76840b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:31.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.761+0000 7f7683200640 1 -- 172.21.15.42:0/4087740548 shutdown_connections 2024-05-28T13:59:31.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.761+0000 7f7683200640 1 -- 172.21.15.42:0/4087740548 wait complete. 2024-05-28T13:59:31.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7683200640 1 Processor -- start 2024-05-28T13:59:31.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7683200640 1 -- start start 2024-05-28T13:59:31.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7683200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0450 0x7f7684153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7683200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768415c880 0x7f76840c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7683200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f768415d680 0x7f76841656c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7683200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f76840cd9d0 con 0x7f768415d680 2024-05-28T13:59:31.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7683200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76840cd850 con 0x7f768415c880 2024-05-28T13:59:31.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7683200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76840cdb50 con 0x7f76840d0450 2024-05-28T13:59:31.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7682800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f768415d680 0x7f76841656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7682800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f768415d680 0x7f76841656c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41710/0 (socket says 172.21.15.42:41710) 2024-05-28T13:59:31.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7682800640 1 -- 172.21.15.42:0/3718489351 learned_addr learned my addr 172.21.15.42:0/3718489351 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:31.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7679400640 1 --2- 172.21.15.42:0/3718489351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768415c880 0x7f76840c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7681e00640 1 --2- 172.21.15.42:0/3718489351 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0450 0x7f7684153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7682800640 1 -- 172.21.15.42:0/3718489351 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0450 msgr2=0x7f7684153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7682800640 1 --2- 172.21.15.42:0/3718489351 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0450 0x7f7684153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7682800640 1 -- 172.21.15.42:0/3718489351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768415c880 msgr2=0x7f76840c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7682800640 1 --2- 172.21.15.42:0/3718489351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768415c880 0x7f76840c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7682800640 1 -- 172.21.15.42:0/3718489351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76840ca5a0 con 0x7f768415d680 2024-05-28T13:59:31.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7682800640 1 --2- 172.21.15.42:0/3718489351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f768415d680 0x7f76841656c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f767c01a660 tx=0x7f767c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:31.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f767aa00640 1 -- 172.21.15.42:0/3718489351 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c0b8880 con 0x7f768415d680 2024-05-28T13:59:31.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f767aa00640 1 -- 172.21.15.42:0/3718489351 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f767c0be070 con 0x7f768415d680 2024-05-28T13:59:31.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f767aa00640 1 -- 172.21.15.42:0/3718489351 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c0b8c40 con 0x7f768415d680 2024-05-28T13:59:31.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7683200640 1 -- 172.21.15.42:0/3718489351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76840ca880 con 0x7f768415d680 2024-05-28T13:59:31.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7683200640 1 -- 172.21.15.42:0/3718489351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76841541a0 con 0x7f768415d680 2024-05-28T13:59:31.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.765+0000 7f7683200640 1 -- 172.21.15.42:0/3718489351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f768404fcf0 con 0x7f768415d680 2024-05-28T13:59:31.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.769+0000 7f767aa00640 1 -- 172.21.15.42:0/3718489351 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f767c0c3460 con 0x7f768415d680 2024-05-28T13:59:31.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.769+0000 7f767aa00640 1 --2- 172.21.15.42:0/3718489351 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f76600683b0 0x7f766006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:31.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.769+0000 7f767aa00640 1 -- 172.21.15.42:0/3718489351 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f767c121a40 con 0x7f768415d680 2024-05-28T13:59:31.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.769+0000 7f7681e00640 1 --2- 172.21.15.42:0/3718489351 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f76600683b0 0x7f766006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:31.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.769+0000 7f7681e00640 1 --2- 172.21.15.42:0/3718489351 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f76600683b0 0x7f766006a870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7684166350 tx=0x7f7674009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:31.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.773+0000 7f767aa00640 1 -- 172.21.15.42:0/3718489351 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f767c016620 con 0x7f768415d680 2024-05-28T13:59:31.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.937+0000 7f7683200640 1 -- 172.21.15.42:0/3718489351 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76840b4ce0 con 0x7f76600683b0 2024-05-28T13:59:31.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.945+0000 7f767aa00640 1 -- 172.21.15.42:0/3718489351 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f76840b4ce0 con 0x7f76600683b0 2024-05-28T13:59:31.932 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:31.933 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T13:59:31.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.949+0000 7f7683200640 1 -- 172.21.15.42:0/3718489351 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f76600683b0 msgr2=0x7f766006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.949+0000 7f7683200640 1 --2- 172.21.15.42:0/3718489351 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f76600683b0 0x7f766006a870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7684166350 tx=0x7f7674009190 comp rx=0 tx=0).stop 2024-05-28T13:59:31.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.949+0000 7f7683200640 1 -- 172.21.15.42:0/3718489351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f768415d680 msgr2=0x7f76841656c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:31.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.949+0000 7f7683200640 1 --2- 172.21.15.42:0/3718489351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f768415d680 0x7f76841656c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f767c01a660 tx=0x7f767c0197d0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.949+0000 7f7683200640 1 -- 172.21.15.42:0/3718489351 shutdown_connections 2024-05-28T13:59:31.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.949+0000 7f7683200640 1 --2- 172.21.15.42:0/3718489351 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f76600683b0 0x7f766006a870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.949+0000 7f7683200640 1 --2- 172.21.15.42:0/3718489351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f768415d680 0x7f76841656c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.949+0000 7f7683200640 1 --2- 172.21.15.42:0/3718489351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f768415c880 0x7f76840c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.949+0000 7f7683200640 1 --2- 172.21.15.42:0/3718489351 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0450 0x7f7684153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:31.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.949+0000 7f7683200640 1 -- 172.21.15.42:0/3718489351 >> 172.21.15.42:0/3718489351 conn(0x7f76840b3dd0 msgr2=0x7f76840706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:31.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.949+0000 7f7683200640 1 -- 172.21.15.42:0/3718489351 shutdown_connections 2024-05-28T13:59:31.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:31.949+0000 7f7683200640 1 -- 172.21.15.42:0/3718489351 wait complete. 2024-05-28T13:59:31.953 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-05-28T13:59:30.868873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237040,"kb_used_data":4208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737680,"statfs":{"total":773094113280,"available":772851384320,"internally_reserved":0,"allocated":4308992,"data_stored":2383188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.441200"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057333+0000","last_change":"2024-05-28T13:59:27.480267+0000","last_active":"2024-05-28T13:59:29.057333+0000","last_peered":"2024-05-28T13:59:29.057333+0000","last_clean":"2024-05-28T13:59:29.057333+0000","last_became_active":"2024-05-28T13:59:27.479892+0000","last_became_peered":"2024-05-28T13:59:27.479892+0000","last_unstale":"2024-05-28T13:59:29.057333+0000","last_undegraded":"2024-05-28T13:59:29.057333+0000","last_fullsized":"2024-05-28T13:59:29.057333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:16.522449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.470294+0000","last_change":"2024-05-28T13:59:27.471521+0000","last_active":"2024-05-28T13:59:28.470294+0000","last_peered":"2024-05-28T13:59:28.470294+0000","last_clean":"2024-05-28T13:59:28.470294+0000","last_became_active":"2024-05-28T13:59:27.471250+0000","last_became_peered":"2024-05-28T13:59:27.471250+0000","last_unstale":"2024-05-28T13:59:28.470294+0000","last_undegraded":"2024-05-28T13:59:28.470294+0000","last_fullsized":"2024-05-28T13:59:28.470294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:46.412620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.058034+0000","last_change":"2024-05-28T13:59:27.480775+0000","last_active":"2024-05-28T13:59:29.058034+0000","last_peered":"2024-05-28T13:59:29.058034+0000","last_clean":"2024-05-28T13:59:29.058034+0000","last_became_active":"2024-05-28T13:59:27.480513+0000","last_became_peered":"2024-05-28T13:59:27.480513+0000","last_unstale":"2024-05-28T13:59:29.058034+0000","last_undegraded":"2024-05-28T13:59:29.058034+0000","last_fullsized":"2024-05-28T13:59:29.058034+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:40.078224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056058+0000","last_change":"2024-05-28T13:59:27.471276+0000","last_active":"2024-05-28T13:59:29.056058+0000","last_peered":"2024-05-28T13:59:29.056058+0000","last_clean":"2024-05-28T13:59:29.056058+0000","last_became_active":"2024-05-28T13:59:27.471007+0000","last_became_peered":"2024-05-28T13:59:27.471007+0000","last_unstale":"2024-05-28T13:59:29.056058+0000","last_undegraded":"2024-05-28T13:59:29.056058+0000","last_fullsized":"2024-05-28T13:59:29.056058+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:40:52.295785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056268+0000","last_change":"2024-05-28T13:59:27.464933+0000","last_active":"2024-05-28T13:59:29.056268+0000","last_peered":"2024-05-28T13:59:29.056268+0000","last_clean":"2024-05-28T13:59:29.056268+0000","last_became_active":"2024-05-28T13:59:27.464641+0000","last_became_peered":"2024-05-28T13:59:27.464641+0000","last_unstale":"2024-05-28T13:59:29.056268+0000","last_undegraded":"2024-05-28T13:59:29.056268+0000","last_fullsized":"2024-05-28T13:59:29.056268+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:00:48.840009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469391+0000","last_change":"2024-05-28T13:59:27.477705+0000","last_active":"2024-05-28T13:59:28.469391+0000","last_peered":"2024-05-28T13:59:28.469391+0000","last_clean":"2024-05-28T13:59:28.469391+0000","last_became_active":"2024-05-28T13:59:27.477406+0000","last_became_peered":"2024-05-28T13:59:27.477406+0000","last_unstale":"2024-05-28T13:59:28.469391+0000","last_undegraded":"2024-05-28T13:59:28.469391+0000","last_fullsized":"2024-05-28T13:59:28.469391+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:13:14.315389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473550+0000","last_change":"2024-05-28T13:59:27.484864+0000","last_active":"2024-05-28T13:59:28.473550+0000","last_peered":"2024-05-28T13:59:28.473550+0000","last_clean":"2024-05-28T13:59:28.473550+0000","last_became_active":"2024-05-28T13:59:27.484214+0000","last_became_peered":"2024-05-28T13:59:27.484214+0000","last_unstale":"2024-05-28T13:59:28.473550+0000","last_undegraded":"2024-05-28T13:59:28.473550+0000","last_fullsized":"2024-05-28T13:59:28.473550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:14:36.728353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473130+0000","last_change":"2024-05-28T13:59:27.484311+0000","last_active":"2024-05-28T13:59:28.473130+0000","last_peered":"2024-05-28T13:59:28.473130+0000","last_clean":"2024-05-28T13:59:28.473130+0000","last_became_active":"2024-05-28T13:59:27.483949+0000","last_became_peered":"2024-05-28T13:59:27.483949+0000","last_unstale":"2024-05-28T13:59:28.473130+0000","last_undegraded":"2024-05-28T13:59:28.473130+0000","last_fullsized":"2024-05-28T13:59:28.473130+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:26:42.728298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472376+0000","last_change":"2024-05-28T13:59:27.476975+0000","last_active":"2024-05-28T13:59:28.472376+0000","last_peered":"2024-05-28T13:59:28.472376+0000","last_clean":"2024-05-28T13:59:28.472376+0000","last_became_active":"2024-05-28T13:59:27.476603+0000","last_became_peered":"2024-05-28T13:59:27.476603+0000","last_unstale":"2024-05-28T13:59:28.472376+0000","last_undegraded":"2024-05-28T13:59:28.472376+0000","last_fullsized":"2024-05-28T13:59:28.472376+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:16:26.601146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056875+0000","last_change":"2024-05-28T13:59:27.465768+0000","last_active":"2024-05-28T13:59:29.056875+0000","last_peered":"2024-05-28T13:59:29.056875+0000","last_clean":"2024-05-28T13:59:29.056875+0000","last_became_active":"2024-05-28T13:59:27.465474+0000","last_became_peered":"2024-05-28T13:59:27.465474+0000","last_unstale":"2024-05-28T13:59:29.056875+0000","last_undegraded":"2024-05-28T13:59:29.056875+0000","last_fullsized":"2024-05-28T13:59:29.056875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:06:38.471358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473293+0000","last_change":"2024-05-28T13:59:27.490188+0000","last_active":"2024-05-28T13:59:28.473293+0000","last_peered":"2024-05-28T13:59:28.473293+0000","last_clean":"2024-05-28T13:59:28.473293+0000","last_became_active":"2024-05-28T13:59:27.489876+0000","last_became_peered":"2024-05-28T13:59:27.489876+0000","last_unstale":"2024-05-28T13:59:28.473293+0000","last_undegraded":"2024-05-28T13:59:28.473293+0000","last_fullsized":"2024-05-28T13:59:28.473293+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:04:33.810167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057606+0000","last_change":"2024-05-28T13:59:27.475535+0000","last_active":"2024-05-28T13:59:29.057606+0000","last_peered":"2024-05-28T13:59:29.057606+0000","last_clean":"2024-05-28T13:59:29.057606+0000","last_became_active":"2024-05-28T13:59:27.475249+0000","last_became_peered":"2024-05-28T13:59:27.475249+0000","last_unstale":"2024-05-28T13:59:29.057606+0000","last_undegraded":"2024-05-28T13:59:29.057606+0000","last_fullsized":"2024-05-28T13:59:29.057606+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:13:14.315389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056421+0000","last_change":"2024-05-28T13:59:27.468303+0000","last_active":"2024-05-28T13:59:29.056421+0000","last_peered":"2024-05-28T13:59:29.056421+0000","last_clean":"2024-05-28T13:59:29.056421+0000","last_became_active":"2024-05-28T13:59:27.467958+0000","last_became_peered":"2024-05-28T13:59:27.467958+0000","last_unstale":"2024-05-28T13:59:29.056421+0000","last_undegraded":"2024-05-28T13:59:29.056421+0000","last_fullsized":"2024-05-28T13:59:29.056421+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:10:43.129378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469624+0000","last_change":"2024-05-28T13:59:27.473200+0000","last_active":"2024-05-28T13:59:28.469624+0000","last_peered":"2024-05-28T13:59:28.469624+0000","last_clean":"2024-05-28T13:59:28.469624+0000","last_became_active":"2024-05-28T13:59:27.472936+0000","last_became_peered":"2024-05-28T13:59:27.472936+0000","last_unstale":"2024-05-28T13:59:28.469624+0000","last_undegraded":"2024-05-28T13:59:28.469624+0000","last_fullsized":"2024-05-28T13:59:28.469624+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:17:00.358853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057112+0000","last_change":"2024-05-28T13:59:27.472137+0000","last_active":"2024-05-28T13:59:29.057112+0000","last_peered":"2024-05-28T13:59:29.057112+0000","last_clean":"2024-05-28T13:59:29.057112+0000","last_became_active":"2024-05-28T13:59:27.471877+0000","last_became_peered":"2024-05-28T13:59:27.471877+0000","last_unstale":"2024-05-28T13:59:29.057112+0000","last_undegraded":"2024-05-28T13:59:29.057112+0000","last_fullsized":"2024-05-28T13:59:29.057112+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:21:32.285503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473519+0000","last_change":"2024-05-28T13:59:27.487690+0000","last_active":"2024-05-28T13:59:28.473519+0000","last_peered":"2024-05-28T13:59:28.473519+0000","last_clean":"2024-05-28T13:59:28.473519+0000","last_became_active":"2024-05-28T13:59:27.485340+0000","last_became_peered":"2024-05-28T13:59:27.485340+0000","last_unstale":"2024-05-28T13:59:28.473519+0000","last_undegraded":"2024-05-28T13:59:28.473519+0000","last_fullsized":"2024-05-28T13:59:28.473519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:01:29.710266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473399+0000","last_change":"2024-05-28T13:59:27.493997+0000","last_active":"2024-05-28T13:59:28.473399+0000","last_peered":"2024-05-28T13:59:28.473399+0000","last_clean":"2024-05-28T13:59:28.473399+0000","last_became_active":"2024-05-28T13:59:27.493597+0000","last_became_peered":"2024-05-28T13:59:27.493597+0000","last_unstale":"2024-05-28T13:59:28.473399+0000","last_undegraded":"2024-05-28T13:59:28.473399+0000","last_fullsized":"2024-05-28T13:59:28.473399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:17:00.358853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.470342+0000","last_change":"2024-05-28T13:59:27.467719+0000","last_active":"2024-05-28T13:59:28.470342+0000","last_peered":"2024-05-28T13:59:28.470342+0000","last_clean":"2024-05-28T13:59:28.470342+0000","last_became_active":"2024-05-28T13:59:27.467403+0000","last_became_peered":"2024-05-28T13:59:27.467403+0000","last_unstale":"2024-05-28T13:59:28.470342+0000","last_undegraded":"2024-05-28T13:59:28.470342+0000","last_fullsized":"2024-05-28T13:59:28.470342+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:40.078224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.470128+0000","last_change":"2024-05-28T13:59:27.468124+0000","last_active":"2024-05-28T13:59:28.470128+0000","last_peered":"2024-05-28T13:59:28.470128+0000","last_clean":"2024-05-28T13:59:28.470128+0000","last_became_active":"2024-05-28T13:59:27.467867+0000","last_became_peered":"2024-05-28T13:59:27.467867+0000","last_unstale":"2024-05-28T13:59:28.470128+0000","last_undegraded":"2024-05-28T13:59:28.470128+0000","last_fullsized":"2024-05-28T13:59:28.470128+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:40:50.994223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056875+0000","last_change":"2024-05-28T13:59:27.477291+0000","last_active":"2024-05-28T13:59:29.056875+0000","last_peered":"2024-05-28T13:59:29.056875+0000","last_clean":"2024-05-28T13:59:29.056875+0000","last_became_active":"2024-05-28T13:59:27.476988+0000","last_became_peered":"2024-05-28T13:59:27.476988+0000","last_unstale":"2024-05-28T13:59:29.056875+0000","last_undegraded":"2024-05-28T13:59:29.056875+0000","last_fullsized":"2024-05-28T13:59:29.056875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:56:41.726103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472487+0000","last_change":"2024-05-28T13:59:27.478881+0000","last_active":"2024-05-28T13:59:28.472487+0000","last_peered":"2024-05-28T13:59:28.472487+0000","last_clean":"2024-05-28T13:59:28.472487+0000","last_became_active":"2024-05-28T13:59:27.478572+0000","last_became_peered":"2024-05-28T13:59:27.478572+0000","last_unstale":"2024-05-28T13:59:28.472487+0000","last_undegraded":"2024-05-28T13:59:28.472487+0000","last_fullsized":"2024-05-28T13:59:28.472487+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:58:31.249506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472169+0000","last_change":"2024-05-28T13:59:27.471276+0000","last_active":"2024-05-28T13:59:28.472169+0000","last_peered":"2024-05-28T13:59:28.472169+0000","last_clean":"2024-05-28T13:59:28.472169+0000","last_became_active":"2024-05-28T13:59:27.470165+0000","last_became_peered":"2024-05-28T13:59:27.470165+0000","last_unstale":"2024-05-28T13:59:28.472169+0000","last_undegraded":"2024-05-28T13:59:28.472169+0000","last_fullsized":"2024-05-28T13:59:28.472169+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:39.802345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.055769+0000","last_change":"2024-05-28T13:59:27.474945+0000","last_active":"2024-05-28T13:59:29.055769+0000","last_peered":"2024-05-28T13:59:29.055769+0000","last_clean":"2024-05-28T13:59:29.055769+0000","last_became_active":"2024-05-28T13:59:27.474483+0000","last_became_peered":"2024-05-28T13:59:27.474483+0000","last_unstale":"2024-05-28T13:59:29.055769+0000","last_undegraded":"2024-05-28T13:59:29.055769+0000","last_fullsized":"2024-05-28T13:59:29.055769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:39.802345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056879+0000","last_change":"2024-05-28T13:59:27.468399+0000","last_active":"2024-05-28T13:59:29.056879+0000","last_peered":"2024-05-28T13:59:29.056879+0000","last_clean":"2024-05-28T13:59:29.056879+0000","last_became_active":"2024-05-28T13:59:27.468083+0000","last_became_peered":"2024-05-28T13:59:27.468083+0000","last_unstale":"2024-05-28T13:59:29.056879+0000","last_undegraded":"2024-05-28T13:59:29.056879+0000","last_fullsized":"2024-05-28T13:59:29.056879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:26:42.728298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057005+0000","last_change":"2024-05-28T13:59:27.485177+0000","last_active":"2024-05-28T13:59:29.057005+0000","last_peered":"2024-05-28T13:59:29.057005+0000","last_clean":"2024-05-28T13:59:29.057005+0000","last_became_active":"2024-05-28T13:59:27.484565+0000","last_became_peered":"2024-05-28T13:59:27.484565+0000","last_unstale":"2024-05-28T13:59:29.057005+0000","last_undegraded":"2024-05-28T13:59:29.057005+0000","last_fullsized":"2024-05-28T13:59:29.057005+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:08:14.981461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.470449+0000","last_change":"2024-05-28T13:59:27.484860+0000","last_active":"2024-05-28T13:59:28.470449+0000","last_peered":"2024-05-28T13:59:28.470449+0000","last_clean":"2024-05-28T13:59:28.470449+0000","last_became_active":"2024-05-28T13:59:27.484433+0000","last_became_peered":"2024-05-28T13:59:27.484433+0000","last_unstale":"2024-05-28T13:59:28.470449+0000","last_undegraded":"2024-05-28T13:59:28.470449+0000","last_fullsized":"2024-05-28T13:59:28.470449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:32:29.021149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056447+0000","last_change":"2024-05-28T13:59:27.472965+0000","last_active":"2024-05-28T13:59:29.056447+0000","last_peered":"2024-05-28T13:59:29.056447+0000","last_clean":"2024-05-28T13:59:29.056447+0000","last_became_active":"2024-05-28T13:59:27.472683+0000","last_became_peered":"2024-05-28T13:59:27.472683+0000","last_unstale":"2024-05-28T13:59:29.056447+0000","last_undegraded":"2024-05-28T13:59:29.056447+0000","last_fullsized":"2024-05-28T13:59:29.056447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:01:36.630567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057856+0000","last_change":"2024-05-28T13:59:27.477064+0000","last_active":"2024-05-28T13:59:29.057856+0000","last_peered":"2024-05-28T13:59:29.057856+0000","last_clean":"2024-05-28T13:59:29.057856+0000","last_became_active":"2024-05-28T13:59:27.476756+0000","last_became_peered":"2024-05-28T13:59:27.476756+0000","last_unstale":"2024-05-28T13:59:29.057856+0000","last_undegraded":"2024-05-28T13:59:29.057856+0000","last_fullsized":"2024-05-28T13:59:29.057856+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:01:29.710266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472650+0000","last_change":"2024-05-28T13:59:27.465607+0000","last_active":"2024-05-28T13:59:28.472650+0000","last_peered":"2024-05-28T13:59:28.472650+0000","last_clean":"2024-05-28T13:59:28.472650+0000","last_became_active":"2024-05-28T13:59:27.465262+0000","last_became_peered":"2024-05-28T13:59:27.465262+0000","last_unstale":"2024-05-28T13:59:28.472650+0000","last_undegraded":"2024-05-28T13:59:28.472650+0000","last_fullsized":"2024-05-28T13:59:28.472650+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:01:24.296794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473593+0000","last_change":"2024-05-28T13:59:27.488169+0000","last_active":"2024-05-28T13:59:28.473593+0000","last_peered":"2024-05-28T13:59:28.473593+0000","last_clean":"2024-05-28T13:59:28.473593+0000","last_became_active":"2024-05-28T13:59:27.487907+0000","last_became_peered":"2024-05-28T13:59:27.487907+0000","last_unstale":"2024-05-28T13:59:28.473593+0000","last_undegraded":"2024-05-28T13:59:28.473593+0000","last_fullsized":"2024-05-28T13:59:28.473593+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:40.078224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057201+0000","last_change":"2024-05-28T13:59:27.480083+0000","last_active":"2024-05-28T13:59:29.057201+0000","last_peered":"2024-05-28T13:59:29.057201+0000","last_clean":"2024-05-28T13:59:29.057201+0000","last_became_active":"2024-05-28T13:59:27.479797+0000","last_became_peered":"2024-05-28T13:59:27.479797+0000","last_unstale":"2024-05-28T13:59:29.057201+0000","last_undegraded":"2024-05-28T13:59:29.057201+0000","last_fullsized":"2024-05-28T13:59:29.057201+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:11:08.637868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473109+0000","last_change":"2024-05-28T13:59:27.494251+0000","last_active":"2024-05-28T13:59:28.473109+0000","last_peered":"2024-05-28T13:59:28.473109+0000","last_clean":"2024-05-28T13:59:28.473109+0000","last_became_active":"2024-05-28T13:59:27.493726+0000","last_became_peered":"2024-05-28T13:59:27.493726+0000","last_unstale":"2024-05-28T13:59:28.473109+0000","last_undegraded":"2024-05-28T13:59:28.473109+0000","last_fullsized":"2024-05-28T13:59:28.473109+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:08:14.981461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.055959+0000","last_change":"2024-05-28T13:59:27.487065+0000","last_active":"2024-05-28T13:59:29.055959+0000","last_peered":"2024-05-28T13:59:29.055959+0000","last_clean":"2024-05-28T13:59:29.055959+0000","last_became_active":"2024-05-28T13:59:27.486748+0000","last_became_peered":"2024-05-28T13:59:27.486748+0000","last_unstale":"2024-05-28T13:59:29.055959+0000","last_undegraded":"2024-05-28T13:59:29.055959+0000","last_fullsized":"2024-05-28T13:59:29.055959+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:24:17.371296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057455+0000","last_change":"2024-05-28T13:59:26.457091+0000","last_active":"2024-05-28T13:59:29.057455+0000","last_peered":"2024-05-28T13:59:29.057455+0000","last_clean":"2024-05-28T13:59:29.057455+0000","last_became_active":"2024-05-28T13:59:26.456803+0000","last_became_peered":"2024-05-28T13:59:26.456803+0000","last_unstale":"2024-05-28T13:59:29.057455+0000","last_undegraded":"2024-05-28T13:59:29.057455+0000","last_fullsized":"2024-05-28T13:59:29.057455+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:10:42.124255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057363+0000","last_change":"2024-05-28T13:59:27.484036+0000","last_active":"2024-05-28T13:59:29.057363+0000","last_peered":"2024-05-28T13:59:29.057363+0000","last_clean":"2024-05-28T13:59:29.057363+0000","last_became_active":"2024-05-28T13:59:27.483763+0000","last_became_peered":"2024-05-28T13:59:27.483763+0000","last_unstale":"2024-05-28T13:59:29.057363+0000","last_undegraded":"2024-05-28T13:59:29.057363+0000","last_fullsized":"2024-05-28T13:59:29.057363+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:59:25.794233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056695+0000","last_change":"2024-05-28T13:59:26.462516+0000","last_active":"2024-05-28T13:59:29.056695+0000","last_peered":"2024-05-28T13:59:29.056695+0000","last_clean":"2024-05-28T13:59:29.056695+0000","last_became_active":"2024-05-28T13:59:26.462205+0000","last_became_peered":"2024-05-28T13:59:26.462205+0000","last_unstale":"2024-05-28T13:59:29.056695+0000","last_undegraded":"2024-05-28T13:59:29.056695+0000","last_fullsized":"2024-05-28T13:59:29.056695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:51:03.448869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473028+0000","last_change":"2024-05-28T13:59:27.470403+0000","last_active":"2024-05-28T13:59:28.473028+0000","last_peered":"2024-05-28T13:59:28.473028+0000","last_clean":"2024-05-28T13:59:28.473028+0000","last_became_active":"2024-05-28T13:59:27.469358+0000","last_became_peered":"2024-05-28T13:59:27.469358+0000","last_unstale":"2024-05-28T13:59:28.473028+0000","last_undegraded":"2024-05-28T13:59:28.473028+0000","last_fullsized":"2024-05-28T13:59:28.473028+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:18:11.375412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.470093+0000","last_change":"2024-05-28T13:59:26.457801+0000","last_active":"2024-05-28T13:59:28.470093+0000","last_peered":"2024-05-28T13:59:28.470093+0000","last_clean":"2024-05-28T13:59:28.470093+0000","last_became_active":"2024-05-28T13:59:26.457432+0000","last_became_peered":"2024-05-28T13:59:26.457432+0000","last_unstale":"2024-05-28T13:59:28.470093+0000","last_undegraded":"2024-05-28T13:59:28.470093+0000","last_fullsized":"2024-05-28T13:59:28.470093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:38.797222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056565+0000","last_change":"2024-05-28T13:59:27.473321+0000","last_active":"2024-05-28T13:59:29.056565+0000","last_peered":"2024-05-28T13:59:29.056565+0000","last_clean":"2024-05-28T13:59:29.056565+0000","last_became_active":"2024-05-28T13:59:27.473006+0000","last_became_peered":"2024-05-28T13:59:27.473006+0000","last_unstale":"2024-05-28T13:59:29.056565+0000","last_undegraded":"2024-05-28T13:59:29.056565+0000","last_fullsized":"2024-05-28T13:59:29.056565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:11:24.829033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056730+0000","last_change":"2024-05-28T13:59:26.461782+0000","last_active":"2024-05-28T13:59:29.056730+0000","last_peered":"2024-05-28T13:59:29.056730+0000","last_clean":"2024-05-28T13:59:29.056730+0000","last_became_active":"2024-05-28T13:59:26.461315+0000","last_became_peered":"2024-05-28T13:59:26.461315+0000","last_unstale":"2024-05-28T13:59:29.056730+0000","last_undegraded":"2024-05-28T13:59:29.056730+0000","last_fullsized":"2024-05-28T13:59:29.056730+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:29:57.955171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473552+0000","last_change":"2024-05-28T13:59:27.493269+0000","last_active":"2024-05-28T13:59:28.473552+0000","last_peered":"2024-05-28T13:59:28.473552+0000","last_clean":"2024-05-28T13:59:28.473552+0000","last_became_active":"2024-05-28T13:59:27.492980+0000","last_became_peered":"2024-05-28T13:59:27.492980+0000","last_unstale":"2024-05-28T13:59:28.473552+0000","last_undegraded":"2024-05-28T13:59:28.473552+0000","last_fullsized":"2024-05-28T13:59:28.473552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:40:50.994223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472754+0000","last_change":"2024-05-28T13:59:26.455974+0000","last_active":"2024-05-28T13:59:28.472754+0000","last_peered":"2024-05-28T13:59:28.472754+0000","last_clean":"2024-05-28T13:59:28.472754+0000","last_became_active":"2024-05-28T13:59:26.455680+0000","last_became_peered":"2024-05-28T13:59:26.455680+0000","last_unstale":"2024-05-28T13:59:28.472754+0000","last_undegraded":"2024-05-28T13:59:28.472754+0000","last_fullsized":"2024-05-28T13:59:28.472754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:14:43.321207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057552+0000","last_change":"2024-05-28T13:59:27.472454+0000","last_active":"2024-05-28T13:59:29.057552+0000","last_peered":"2024-05-28T13:59:29.057552+0000","last_clean":"2024-05-28T13:59:29.057552+0000","last_became_active":"2024-05-28T13:59:27.472107+0000","last_became_peered":"2024-05-28T13:59:27.472107+0000","last_unstale":"2024-05-28T13:59:29.057552+0000","last_undegraded":"2024-05-28T13:59:29.057552+0000","last_fullsized":"2024-05-28T13:59:29.057552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:04:33.810167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472283+0000","last_change":"2024-05-28T13:59:26.455099+0000","last_active":"2024-05-28T13:59:28.472283+0000","last_peered":"2024-05-28T13:59:28.472283+0000","last_clean":"2024-05-28T13:59:28.472283+0000","last_became_active":"2024-05-28T13:59:26.454339+0000","last_became_peered":"2024-05-28T13:59:26.454339+0000","last_unstale":"2024-05-28T13:59:28.472283+0000","last_undegraded":"2024-05-28T13:59:28.472283+0000","last_fullsized":"2024-05-28T13:59:28.472283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:14:19.236068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.471613+0000","last_change":"2024-05-28T13:59:27.478265+0000","last_active":"2024-05-28T13:59:28.471613+0000","last_peered":"2024-05-28T13:59:28.471613+0000","last_clean":"2024-05-28T13:59:28.471613+0000","last_became_active":"2024-05-28T13:59:27.475778+0000","last_became_peered":"2024-05-28T13:59:27.475778+0000","last_unstale":"2024-05-28T13:59:28.471613+0000","last_undegraded":"2024-05-28T13:59:28.471613+0000","last_fullsized":"2024-05-28T13:59:28.471613+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:16.522449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469351+0000","last_change":"2024-05-28T13:59:26.451905+0000","last_active":"2024-05-28T13:59:28.469351+0000","last_peered":"2024-05-28T13:59:28.469351+0000","last_clean":"2024-05-28T13:59:28.469351+0000","last_became_active":"2024-05-28T13:59:26.451616+0000","last_became_peered":"2024-05-28T13:59:26.451616+0000","last_unstale":"2024-05-28T13:59:28.469351+0000","last_undegraded":"2024-05-28T13:59:28.469351+0000","last_fullsized":"2024-05-28T13:59:28.469351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:15.517326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.055876+0000","last_change":"2024-05-28T13:59:27.466715+0000","last_active":"2024-05-28T13:59:29.055876+0000","last_peered":"2024-05-28T13:59:29.055876+0000","last_clean":"2024-05-28T13:59:29.055876+0000","last_became_active":"2024-05-28T13:59:27.466435+0000","last_became_peered":"2024-05-28T13:59:27.466435+0000","last_unstale":"2024-05-28T13:59:29.055876+0000","last_undegraded":"2024-05-28T13:59:29.055876+0000","last_fullsized":"2024-05-28T13:59:29.055876+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:18:11.375412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.471729+0000","last_change":"2024-05-28T13:59:26.452068+0000","last_active":"2024-05-28T13:59:28.471729+0000","last_peered":"2024-05-28T13:59:28.471729+0000","last_clean":"2024-05-28T13:59:28.471729+0000","last_became_active":"2024-05-28T13:59:26.451700+0000","last_became_peered":"2024-05-28T13:59:26.451700+0000","last_unstale":"2024-05-28T13:59:28.471729+0000","last_undegraded":"2024-05-28T13:59:28.471729+0000","last_fullsized":"2024-05-28T13:59:28.471729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:06:03.772836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057857+0000","last_change":"2024-05-28T13:59:27.470828+0000","last_active":"2024-05-28T13:59:29.057857+0000","last_peered":"2024-05-28T13:59:29.057857+0000","last_clean":"2024-05-28T13:59:29.057857+0000","last_became_active":"2024-05-28T13:59:27.470556+0000","last_became_peered":"2024-05-28T13:59:27.470556+0000","last_unstale":"2024-05-28T13:59:29.057857+0000","last_undegraded":"2024-05-28T13:59:29.057857+0000","last_fullsized":"2024-05-28T13:59:29.057857+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:14:36.728353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.470403+0000","last_change":"2024-05-28T13:59:26.455937+0000","last_active":"2024-05-28T13:59:28.470403+0000","last_peered":"2024-05-28T13:59:28.470403+0000","last_clean":"2024-05-28T13:59:28.470403+0000","last_became_active":"2024-05-28T13:59:26.453999+0000","last_became_peered":"2024-05-28T13:59:26.453999+0000","last_unstale":"2024-05-28T13:59:28.470403+0000","last_undegraded":"2024-05-28T13:59:28.470403+0000","last_fullsized":"2024-05-28T13:59:28.470403+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:24:16.366173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.470467+0000","last_change":"2024-05-28T13:59:27.478424+0000","last_active":"2024-05-28T13:59:28.470467+0000","last_peered":"2024-05-28T13:59:28.470467+0000","last_clean":"2024-05-28T13:59:28.470467+0000","last_became_active":"2024-05-28T13:59:27.477986+0000","last_became_peered":"2024-05-28T13:59:27.477986+0000","last_unstale":"2024-05-28T13:59:28.470467+0000","last_undegraded":"2024-05-28T13:59:28.470467+0000","last_fullsized":"2024-05-28T13:59:28.470467+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:22:27.057001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056089+0000","last_change":"2024-05-28T13:59:26.454732+0000","last_active":"2024-05-28T13:59:29.056089+0000","last_peered":"2024-05-28T13:59:29.056089+0000","last_clean":"2024-05-28T13:59:29.056089+0000","last_became_active":"2024-05-28T13:59:26.454392+0000","last_became_peered":"2024-05-28T13:59:26.454392+0000","last_unstale":"2024-05-28T13:59:29.056089+0000","last_undegraded":"2024-05-28T13:59:29.056089+0000","last_fullsized":"2024-05-28T13:59:29.056089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:09:06.066324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469966+0000","last_change":"2024-05-28T13:59:27.484330+0000","last_active":"2024-05-28T13:59:28.469966+0000","last_peered":"2024-05-28T13:59:28.469966+0000","last_clean":"2024-05-28T13:59:28.469966+0000","last_became_active":"2024-05-28T13:59:27.483976+0000","last_became_peered":"2024-05-28T13:59:27.483976+0000","last_unstale":"2024-05-28T13:59:28.469966+0000","last_undegraded":"2024-05-28T13:59:28.469966+0000","last_fullsized":"2024-05-28T13:59:28.469966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:14:36.728353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057933+0000","last_change":"2024-05-28T13:59:26.460226+0000","last_active":"2024-05-28T13:59:29.057933+0000","last_peered":"2024-05-28T13:59:29.057933+0000","last_clean":"2024-05-28T13:59:29.057933+0000","last_became_active":"2024-05-28T13:59:26.459871+0000","last_became_peered":"2024-05-28T13:59:26.459871+0000","last_unstale":"2024-05-28T13:59:29.057933+0000","last_undegraded":"2024-05-28T13:59:29.057933+0000","last_fullsized":"2024-05-28T13:59:29.057933+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:16:25.596023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057743+0000","last_change":"2024-05-28T13:59:27.481776+0000","last_active":"2024-05-28T13:59:29.057743+0000","last_peered":"2024-05-28T13:59:29.057743+0000","last_clean":"2024-05-28T13:59:29.057743+0000","last_became_active":"2024-05-28T13:59:27.481344+0000","last_became_peered":"2024-05-28T13:59:27.481344+0000","last_unstale":"2024-05-28T13:59:29.057743+0000","last_undegraded":"2024-05-28T13:59:29.057743+0000","last_fullsized":"2024-05-28T13:59:29.057743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:17:00.358853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469920+0000","last_change":"2024-05-28T13:59:26.457474+0000","last_active":"2024-05-28T13:59:28.469920+0000","last_peered":"2024-05-28T13:59:28.469920+0000","last_clean":"2024-05-28T13:59:28.469920+0000","last_became_active":"2024-05-28T13:59:26.457177+0000","last_became_peered":"2024-05-28T13:59:26.457177+0000","last_unstale":"2024-05-28T13:59:28.469920+0000","last_undegraded":"2024-05-28T13:59:28.469920+0000","last_fullsized":"2024-05-28T13:59:28.469920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:16:25.596023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.055630+0000","last_change":"2024-05-28T13:59:27.465166+0000","last_active":"2024-05-28T13:59:29.055630+0000","last_peered":"2024-05-28T13:59:29.055630+0000","last_clean":"2024-05-28T13:59:29.055630+0000","last_became_active":"2024-05-28T13:59:27.464834+0000","last_became_peered":"2024-05-28T13:59:27.464834+0000","last_unstale":"2024-05-28T13:59:29.055630+0000","last_undegraded":"2024-05-28T13:59:29.055630+0000","last_fullsized":"2024-05-28T13:59:29.055630+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:01:24.296794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472874+0000","last_change":"2024-05-28T13:59:26.451954+0000","last_active":"2024-05-28T13:59:28.472874+0000","last_peered":"2024-05-28T13:59:28.472874+0000","last_clean":"2024-05-28T13:59:28.472874+0000","last_became_active":"2024-05-28T13:59:26.451673+0000","last_became_peered":"2024-05-28T13:59:26.451673+0000","last_unstale":"2024-05-28T13:59:28.472874+0000","last_undegraded":"2024-05-28T13:59:28.472874+0000","last_fullsized":"2024-05-28T13:59:28.472874+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:45:57.009013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469783+0000","last_change":"2024-05-28T13:59:27.467486+0000","last_active":"2024-05-28T13:59:28.469783+0000","last_peered":"2024-05-28T13:59:28.469783+0000","last_clean":"2024-05-28T13:59:28.469783+0000","last_became_active":"2024-05-28T13:59:27.467189+0000","last_became_peered":"2024-05-28T13:59:27.467189+0000","last_unstale":"2024-05-28T13:59:28.469783+0000","last_undegraded":"2024-05-28T13:59:28.469783+0000","last_fullsized":"2024-05-28T13:59:28.469783+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:01:29.710266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.055594+0000","last_change":"2024-05-28T13:59:26.450030+0000","last_active":"2024-05-28T13:59:29.055594+0000","last_peered":"2024-05-28T13:59:29.055594+0000","last_clean":"2024-05-28T13:59:29.055594+0000","last_became_active":"2024-05-28T13:59:26.449518+0000","last_became_peered":"2024-05-28T13:59:26.449518+0000","last_unstale":"2024-05-28T13:59:29.055594+0000","last_undegraded":"2024-05-28T13:59:29.055594+0000","last_fullsized":"2024-05-28T13:59:29.055594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:56:40.720980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473319+0000","last_change":"2024-05-28T13:59:27.493147+0000","last_active":"2024-05-28T13:59:28.473319+0000","last_peered":"2024-05-28T13:59:28.473319+0000","last_clean":"2024-05-28T13:59:28.473319+0000","last_became_active":"2024-05-28T13:59:27.492868+0000","last_became_peered":"2024-05-28T13:59:27.492868+0000","last_unstale":"2024-05-28T13:59:28.473319+0000","last_undegraded":"2024-05-28T13:59:28.473319+0000","last_fullsized":"2024-05-28T13:59:28.473319+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:13:14.315389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469438+0000","last_change":"2024-05-28T13:59:26.460063+0000","last_active":"2024-05-28T13:59:28.469438+0000","last_peered":"2024-05-28T13:59:28.469438+0000","last_clean":"2024-05-28T13:59:28.469438+0000","last_became_active":"2024-05-28T13:59:26.459777+0000","last_became_peered":"2024-05-28T13:59:26.459777+0000","last_unstale":"2024-05-28T13:59:28.469438+0000","last_undegraded":"2024-05-28T13:59:28.469438+0000","last_fullsized":"2024-05-28T13:59:28.469438+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:59:24.789111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057000+0000","last_change":"2024-05-28T13:59:27.485209+0000","last_active":"2024-05-28T13:59:29.057000+0000","last_peered":"2024-05-28T13:59:29.057000+0000","last_clean":"2024-05-28T13:59:29.057000+0000","last_became_active":"2024-05-28T13:59:27.484797+0000","last_became_peered":"2024-05-28T13:59:27.484797+0000","last_unstale":"2024-05-28T13:59:29.057000+0000","last_undegraded":"2024-05-28T13:59:29.057000+0000","last_fullsized":"2024-05-28T13:59:29.057000+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:00:07.636952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469215+0000","last_change":"2024-05-28T13:59:26.455232+0000","last_active":"2024-05-28T13:59:28.469215+0000","last_peered":"2024-05-28T13:59:28.469215+0000","last_clean":"2024-05-28T13:59:28.469215+0000","last_became_active":"2024-05-28T13:59:26.454943+0000","last_became_peered":"2024-05-28T13:59:26.454943+0000","last_unstale":"2024-05-28T13:59:28.469215+0000","last_undegraded":"2024-05-28T13:59:28.469215+0000","last_fullsized":"2024-05-28T13:59:28.469215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:35:08.660538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472481+0000","last_change":"2024-05-28T13:59:27.481270+0000","last_active":"2024-05-28T13:59:28.472481+0000","last_peered":"2024-05-28T13:59:28.472481+0000","last_clean":"2024-05-28T13:59:28.472481+0000","last_became_active":"2024-05-28T13:59:27.480948+0000","last_became_peered":"2024-05-28T13:59:27.480948+0000","last_unstale":"2024-05-28T13:59:28.472481+0000","last_undegraded":"2024-05-28T13:59:28.472481+0000","last_fullsized":"2024-05-28T13:59:28.472481+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:24:17.371296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057108+0000","last_change":"2024-05-28T13:59:26.463761+0000","last_active":"2024-05-28T13:59:29.057108+0000","last_peered":"2024-05-28T13:59:29.057108+0000","last_clean":"2024-05-28T13:59:29.057108+0000","last_became_active":"2024-05-28T13:59:26.463398+0000","last_became_peered":"2024-05-28T13:59:26.463398+0000","last_unstale":"2024-05-28T13:59:29.057108+0000","last_undegraded":"2024-05-28T13:59:29.057108+0000","last_fullsized":"2024-05-28T13:59:29.057108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:51:03.448869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057961+0000","last_change":"2024-05-28T13:59:27.480086+0000","last_active":"2024-05-28T13:59:29.057961+0000","last_peered":"2024-05-28T13:59:29.057961+0000","last_clean":"2024-05-28T13:59:29.057961+0000","last_became_active":"2024-05-28T13:59:27.479834+0000","last_became_peered":"2024-05-28T13:59:27.479834+0000","last_unstale":"2024-05-28T13:59:29.057961+0000","last_undegraded":"2024-05-28T13:59:29.057961+0000","last_fullsized":"2024-05-28T13:59:29.057961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:40:50.994223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.472547+0000","last_change":"2024-05-28T13:59:26.445326+0000","last_active":"2024-05-28T13:59:28.472547+0000","last_peered":"2024-05-28T13:59:28.472547+0000","last_clean":"2024-05-28T13:59:28.472547+0000","last_became_active":"2024-05-28T13:59:26.445038+0000","last_became_peered":"2024-05-28T13:59:26.445038+0000","last_unstale":"2024-05-28T13:59:28.472547+0000","last_undegraded":"2024-05-28T13:59:28.472547+0000","last_fullsized":"2024-05-28T13:59:28.472547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:21.322022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056214+0000","last_change":"2024-05-28T13:59:27.471440+0000","last_active":"2024-05-28T13:59:29.056214+0000","last_peered":"2024-05-28T13:59:29.056214+0000","last_clean":"2024-05-28T13:59:29.056214+0000","last_became_active":"2024-05-28T13:59:27.471141+0000","last_became_peered":"2024-05-28T13:59:27.471141+0000","last_unstale":"2024-05-28T13:59:29.056214+0000","last_undegraded":"2024-05-28T13:59:29.056214+0000","last_fullsized":"2024-05-28T13:59:29.056214+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:35:52.842901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473047+0000","last_change":"2024-05-28T13:59:26.458570+0000","last_active":"2024-05-28T13:59:28.473047+0000","last_peered":"2024-05-28T13:59:28.473047+0000","last_clean":"2024-05-28T13:59:28.473047+0000","last_became_active":"2024-05-28T13:59:26.458216+0000","last_became_peered":"2024-05-28T13:59:26.458216+0000","last_unstale":"2024-05-28T13:59:28.473047+0000","last_undegraded":"2024-05-28T13:59:28.473047+0000","last_fullsized":"2024-05-28T13:59:28.473047+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:54:53.495272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056809+0000","last_change":"2024-05-28T13:59:27.488471+0000","last_active":"2024-05-28T13:59:29.056809+0000","last_peered":"2024-05-28T13:59:29.056809+0000","last_clean":"2024-05-28T13:59:29.056809+0000","last_became_active":"2024-05-28T13:59:27.488091+0000","last_became_peered":"2024-05-28T13:59:27.488091+0000","last_unstale":"2024-05-28T13:59:29.056809+0000","last_undegraded":"2024-05-28T13:59:29.056809+0000","last_fullsized":"2024-05-28T13:59:29.056809+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:45:29.329612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473068+0000","last_change":"2024-05-28T13:59:26.455262+0000","last_active":"2024-05-28T13:59:28.473068+0000","last_peered":"2024-05-28T13:59:28.473068+0000","last_clean":"2024-05-28T13:59:28.473068+0000","last_became_active":"2024-05-28T13:59:26.453974+0000","last_became_peered":"2024-05-28T13:59:26.453974+0000","last_unstale":"2024-05-28T13:59:28.473068+0000","last_undegraded":"2024-05-28T13:59:28.473068+0000","last_fullsized":"2024-05-28T13:59:28.473068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:55:44.250628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056379+0000","last_change":"2024-05-28T13:59:27.476482+0000","last_active":"2024-05-28T13:59:29.056379+0000","last_peered":"2024-05-28T13:59:29.056379+0000","last_clean":"2024-05-28T13:59:29.056379+0000","last_became_active":"2024-05-28T13:59:27.475253+0000","last_became_peered":"2024-05-28T13:59:27.475253+0000","last_unstale":"2024-05-28T13:59:29.056379+0000","last_undegraded":"2024-05-28T13:59:29.056379+0000","last_fullsized":"2024-05-28T13:59:29.056379+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:23:17.062435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056081+0000","last_change":"2024-05-28T13:59:26.460366+0000","last_active":"2024-05-28T13:59:29.056081+0000","last_peered":"2024-05-28T13:59:29.056081+0000","last_clean":"2024-05-28T13:59:29.056081+0000","last_became_active":"2024-05-28T13:59:26.460168+0000","last_became_peered":"2024-05-28T13:59:26.460168+0000","last_unstale":"2024-05-28T13:59:29.056081+0000","last_undegraded":"2024-05-28T13:59:29.056081+0000","last_fullsized":"2024-05-28T13:59:29.056081+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:45:28.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056571+0000","last_change":"2024-05-28T13:59:27.464552+0000","last_active":"2024-05-28T13:59:29.056571+0000","last_peered":"2024-05-28T13:59:29.056571+0000","last_clean":"2024-05-28T13:59:29.056571+0000","last_became_active":"2024-05-28T13:59:27.463984+0000","last_became_peered":"2024-05-28T13:59:27.463984+0000","last_unstale":"2024-05-28T13:59:29.056571+0000","last_undegraded":"2024-05-28T13:59:29.056571+0000","last_fullsized":"2024-05-28T13:59:29.056571+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:09:29.776916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057203+0000","last_change":"2024-05-28T13:59:26.455086+0000","last_active":"2024-05-28T13:59:29.057203+0000","last_peered":"2024-05-28T13:59:29.057203+0000","last_clean":"2024-05-28T13:59:29.057203+0000","last_became_active":"2024-05-28T13:59:26.454817+0000","last_became_peered":"2024-05-28T13:59:26.454817+0000","last_unstale":"2024-05-28T13:59:29.057203+0000","last_undegraded":"2024-05-28T13:59:29.057203+0000","last_fullsized":"2024-05-28T13:59:29.057203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:23:16.057312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056751+0000","last_change":"2024-05-28T13:59:27.469324+0000","last_active":"2024-05-28T13:59:29.056751+0000","last_peered":"2024-05-28T13:59:29.056751+0000","last_clean":"2024-05-28T13:59:29.056751+0000","last_became_active":"2024-05-28T13:59:27.466052+0000","last_became_peered":"2024-05-28T13:59:27.466052+0000","last_unstale":"2024-05-28T13:59:29.056751+0000","last_undegraded":"2024-05-28T13:59:29.056751+0000","last_fullsized":"2024-05-28T13:59:29.056751+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:51:04.453992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.471727+0000","last_change":"2024-05-28T13:59:26.448462+0000","last_active":"2024-05-28T13:59:28.471727+0000","last_peered":"2024-05-28T13:59:28.471727+0000","last_clean":"2024-05-28T13:59:28.471727+0000","last_became_active":"2024-05-28T13:59:26.448212+0000","last_became_peered":"2024-05-28T13:59:26.448212+0000","last_unstale":"2024-05-28T13:59:28.471727+0000","last_undegraded":"2024-05-28T13:59:28.471727+0000","last_fullsized":"2024-05-28T13:59:28.471727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:00:47.834887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473296+0000","last_change":"2024-05-28T13:59:27.484434+0000","last_active":"2024-05-28T13:59:28.473296+0000","last_peered":"2024-05-28T13:59:28.473296+0000","last_clean":"2024-05-28T13:59:28.473296+0000","last_became_active":"2024-05-28T13:59:27.483863+0000","last_became_peered":"2024-05-28T13:59:27.483863+0000","last_unstale":"2024-05-28T13:59:28.473296+0000","last_undegraded":"2024-05-28T13:59:28.473296+0000","last_fullsized":"2024-05-28T13:59:28.473296+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:21:32.285503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056998+0000","last_change":"2024-05-28T13:59:26.456427+0000","last_active":"2024-05-28T13:59:29.056998+0000","last_peered":"2024-05-28T13:59:29.056998+0000","last_clean":"2024-05-28T13:59:29.056998+0000","last_became_active":"2024-05-28T13:59:26.455597+0000","last_became_peered":"2024-05-28T13:59:26.455597+0000","last_unstale":"2024-05-28T13:59:29.056998+0000","last_undegraded":"2024-05-28T13:59:29.056998+0000","last_fullsized":"2024-05-28T13:59:29.056998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:38.797222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056796+0000","last_change":"2024-05-28T13:59:27.470381+0000","last_active":"2024-05-28T13:59:29.056796+0000","last_peered":"2024-05-28T13:59:29.056796+0000","last_clean":"2024-05-28T13:59:29.056796+0000","last_became_active":"2024-05-28T13:59:27.469832+0000","last_became_peered":"2024-05-28T13:59:27.469832+0000","last_unstale":"2024-05-28T13:59:29.056796+0000","last_undegraded":"2024-05-28T13:59:29.056796+0000","last_fullsized":"2024-05-28T13:59:29.056796+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:09:07.071447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057094+0000","last_change":"2024-05-28T13:59:26.459166+0000","last_active":"2024-05-28T13:59:29.057094+0000","last_peered":"2024-05-28T13:59:29.057094+0000","last_clean":"2024-05-28T13:59:29.057094+0000","last_became_active":"2024-05-28T13:59:26.458777+0000","last_became_peered":"2024-05-28T13:59:26.458777+0000","last_unstale":"2024-05-28T13:59:29.057094+0000","last_undegraded":"2024-05-28T13:59:29.057094+0000","last_fullsized":"2024-05-28T13:59:29.057094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:09:06.066324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057974+0000","last_change":"2024-05-28T13:59:27.482817+0000","last_active":"2024-05-28T13:59:29.057974+0000","last_peered":"2024-05-28T13:59:29.057974+0000","last_clean":"2024-05-28T13:59:29.057974+0000","last_became_active":"2024-05-28T13:59:27.481904+0000","last_became_peered":"2024-05-28T13:59:27.481904+0000","last_unstale":"2024-05-28T13:59:29.057974+0000","last_undegraded":"2024-05-28T13:59:29.057974+0000","last_fullsized":"2024-05-28T13:59:29.057974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:46.412620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.455904+0000","last_change":"2024-05-28T13:59:26.459384+0000","last_active":"2024-05-28T13:59:29.455904+0000","last_peered":"2024-05-28T13:59:29.455904+0000","last_clean":"2024-05-28T13:59:29.455904+0000","last_became_active":"2024-05-28T13:59:26.459117+0000","last_became_peered":"2024-05-28T13:59:26.459117+0000","last_unstale":"2024-05-28T13:59:29.455904+0000","last_undegraded":"2024-05-28T13:59:29.455904+0000","last_fullsized":"2024-05-28T13:59:29.455904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:56:40.720980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.055831+0000","last_change":"2024-05-28T13:59:27.473996+0000","last_active":"2024-05-28T13:59:29.055831+0000","last_peered":"2024-05-28T13:59:29.055831+0000","last_clean":"2024-05-28T13:59:29.055831+0000","last_became_active":"2024-05-28T13:59:27.473702+0000","last_became_peered":"2024-05-28T13:59:27.473702+0000","last_unstale":"2024-05-28T13:59:29.055831+0000","last_undegraded":"2024-05-28T13:59:29.055831+0000","last_fullsized":"2024-05-28T13:59:29.055831+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:16:26.601146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056397+0000","last_change":"2024-05-28T13:59:26.453566+0000","last_active":"2024-05-28T13:59:29.056397+0000","last_peered":"2024-05-28T13:59:29.056397+0000","last_clean":"2024-05-28T13:59:29.056397+0000","last_became_active":"2024-05-28T13:59:26.453166+0000","last_became_peered":"2024-05-28T13:59:26.453166+0000","last_unstale":"2024-05-28T13:59:29.056397+0000","last_undegraded":"2024-05-28T13:59:29.056397+0000","last_fullsized":"2024-05-28T13:59:29.056397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:45:28.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.057163+0000","last_change":"2024-05-28T13:59:27.478988+0000","last_active":"2024-05-28T13:59:29.057163+0000","last_peered":"2024-05-28T13:59:29.057163+0000","last_clean":"2024-05-28T13:59:29.057163+0000","last_became_active":"2024-05-28T13:59:27.478706+0000","last_became_peered":"2024-05-28T13:59:27.478706+0000","last_unstale":"2024-05-28T13:59:29.057163+0000","last_undegraded":"2024-05-28T13:59:29.057163+0000","last_fullsized":"2024-05-28T13:59:29.057163+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:35:09.665661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056220+0000","last_change":"2024-05-28T13:59:26.450630+0000","last_active":"2024-05-28T13:59:29.056220+0000","last_peered":"2024-05-28T13:59:29.056220+0000","last_clean":"2024-05-28T13:59:29.056220+0000","last_became_active":"2024-05-28T13:59:26.450335+0000","last_became_peered":"2024-05-28T13:59:26.450335+0000","last_unstale":"2024-05-28T13:59:29.056220+0000","last_undegraded":"2024-05-28T13:59:29.056220+0000","last_fullsized":"2024-05-28T13:59:29.056220+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:00:06.631829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056127+0000","last_change":"2024-05-28T13:59:27.470786+0000","last_active":"2024-05-28T13:59:29.056127+0000","last_peered":"2024-05-28T13:59:29.056127+0000","last_clean":"2024-05-28T13:59:29.056127+0000","last_became_active":"2024-05-28T13:59:27.470519+0000","last_became_peered":"2024-05-28T13:59:27.470519+0000","last_unstale":"2024-05-28T13:59:29.056127+0000","last_undegraded":"2024-05-28T13:59:29.056127+0000","last_fullsized":"2024-05-28T13:59:29.056127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:58:31.249506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.055785+0000","last_change":"2024-05-28T13:59:26.450196+0000","last_active":"2024-05-28T13:59:29.055785+0000","last_peered":"2024-05-28T13:59:29.055785+0000","last_clean":"2024-05-28T13:59:29.055785+0000","last_became_active":"2024-05-28T13:59:26.449781+0000","last_became_peered":"2024-05-28T13:59:26.449781+0000","last_unstale":"2024-05-28T13:59:29.055785+0000","last_undegraded":"2024-05-28T13:59:29.055785+0000","last_fullsized":"2024-05-28T13:59:29.055785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:00:06.631829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469223+0000","last_change":"2024-05-28T13:59:27.467807+0000","last_active":"2024-05-28T13:59:28.469223+0000","last_peered":"2024-05-28T13:59:28.469223+0000","last_clean":"2024-05-28T13:59:28.469223+0000","last_became_active":"2024-05-28T13:59:27.467575+0000","last_became_peered":"2024-05-28T13:59:27.467575+0000","last_unstale":"2024-05-28T13:59:28.469223+0000","last_undegraded":"2024-05-28T13:59:28.469223+0000","last_fullsized":"2024-05-28T13:59:28.469223+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:04:33.810167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.471846+0000","last_change":"2024-05-28T13:59:26.445766+0000","last_active":"2024-05-28T13:59:28.471846+0000","last_peered":"2024-05-28T13:59:28.471846+0000","last_clean":"2024-05-28T13:59:28.471846+0000","last_became_active":"2024-05-28T13:59:26.444829+0000","last_became_peered":"2024-05-28T13:59:26.444829+0000","last_unstale":"2024-05-28T13:59:28.471846+0000","last_undegraded":"2024-05-28T13:59:28.471846+0000","last_fullsized":"2024-05-28T13:59:28.471846+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:23:16.057312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.471614+0000","last_change":"2024-05-28T13:59:27.465500+0000","last_active":"2024-05-28T13:59:28.471614+0000","last_peered":"2024-05-28T13:59:28.471614+0000","last_clean":"2024-05-28T13:59:28.471614+0000","last_became_active":"2024-05-28T13:59:27.465185+0000","last_became_peered":"2024-05-28T13:59:27.465185+0000","last_unstale":"2024-05-28T13:59:28.471614+0000","last_undegraded":"2024-05-28T13:59:28.471614+0000","last_fullsized":"2024-05-28T13:59:28.471614+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:59:25.794233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.469268+0000","last_change":"2024-05-28T13:59:26.455269+0000","last_active":"2024-05-28T13:59:28.469268+0000","last_peered":"2024-05-28T13:59:28.469268+0000","last_clean":"2024-05-28T13:59:28.469268+0000","last_became_active":"2024-05-28T13:59:26.454950+0000","last_became_peered":"2024-05-28T13:59:26.454950+0000","last_unstale":"2024-05-28T13:59:28.469268+0000","last_undegraded":"2024-05-28T13:59:28.469268+0000","last_fullsized":"2024-05-28T13:59:28.469268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:11:07.632745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473553+0000","last_change":"2024-05-28T13:59:27.484463+0000","last_active":"2024-05-28T13:59:28.473553+0000","last_peered":"2024-05-28T13:59:28.473553+0000","last_clean":"2024-05-28T13:59:28.473553+0000","last_became_active":"2024-05-28T13:59:27.484154+0000","last_became_peered":"2024-05-28T13:59:27.484154+0000","last_unstale":"2024-05-28T13:59:28.473553+0000","last_undegraded":"2024-05-28T13:59:28.473553+0000","last_fullsized":"2024-05-28T13:59:28.473553+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:26.427045+0000","last_clean_scrub_stamp":"2024-05-28T13:59:26.427045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:12:46.412620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:29.056963+0000","last_change":"2024-05-28T13:59:26.451972+0000","last_active":"2024-05-28T13:59:29.056963+0000","last_peered":"2024-05-28T13:59:29.056963+0000","last_clean":"2024-05-28T13:59:29.056963+0000","last_became_active":"2024-05-28T13:59:26.451403+0000","last_became_peered":"2024-05-28T13:59:26.451403+0000","last_unstale":"2024-05-28T13:59:29.056963+0000","last_undegraded":"2024-05-28T13:59:29.056963+0000","last_fullsized":"2024-05-28T13:59:29.056963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:59:25.421922+0000","last_clean_scrub_stamp":"2024-05-28T13:59:25.421922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:00:47.834887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-28T13:59:28.473416+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T13:59:28.473416+0000","last_peered":"2024-05-28T13:59:28.473416+0000","last_clean":"2024-05-28T13:59:28.473416+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T13:59:28.473416+0000","last_undegraded":"2024-05-28T13:59:28.473416+0000","last_fullsized":"2024-05-28T13:59:28.473416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":741737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":151951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":152913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":376832,"data_stored":145049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":152913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":978944,"data_stored":741647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T13:59:31.957 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-28T13:59:31.957 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-28T13:59:32.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.157+0000 7f60e0e00640 1 Processor -- start 2024-05-28T13:59:32.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.157+0000 7f60e0e00640 1 -- start start 2024-05-28T13:59:32.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60e0e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f60dc0b0e80 0x7f60dc06cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60e0e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f60dc09b8e0 0x7f60dc09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60e0e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60dc09af90 0x7f60dc15e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60e0e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f60dc09b3c0 con 0x7f60dc09b8e0 2024-05-28T13:59:32.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60e0e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f60dc0d0770 con 0x7f60dc0b0e80 2024-05-28T13:59:32.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60e0e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60dc0d0a10 con 0x7f60dc09af90 2024-05-28T13:59:32.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60d1400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f60dc09b8e0 0x7f60dc09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60d1400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f60dc09b8e0 0x7f60dc09aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41720/0 (socket says 172.21.15.42:41720) 2024-05-28T13:59:32.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60d1400640 1 -- 172.21.15.42:0/3921667641 learned_addr learned my addr 172.21.15.42:0/3921667641 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:32.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60da800640 1 --2- 172.21.15.42:0/3921667641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60dc09af90 0x7f60dc15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60d9e00640 1 --2- 172.21.15.42:0/3921667641 >> v2:172.21.15.63:3300/0 conn(0x7f60dc0b0e80 0x7f60dc06cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60d1400640 1 -- 172.21.15.42:0/3921667641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60dc09af90 msgr2=0x7f60dc15e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:32.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60d1400640 1 --2- 172.21.15.42:0/3921667641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60dc09af90 0x7f60dc15e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60d1400640 1 -- 172.21.15.42:0/3921667641 >> v2:172.21.15.63:3300/0 conn(0x7f60dc0b0e80 msgr2=0x7f60dc06cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:32.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60d1400640 1 --2- 172.21.15.42:0/3921667641 >> v2:172.21.15.63:3300/0 conn(0x7f60dc0b0e80 0x7f60dc06cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.161+0000 7f60d1400640 1 -- 172.21.15.42:0/3921667641 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc0d0cf0 con 0x7f60dc09b8e0 2024-05-28T13:59:32.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60d1400640 1 --2- 172.21.15.42:0/3921667641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60dc09b8e0 0x7f60dc09aa50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f60dc072fd0 tx=0x7f60c401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53e93fae9d40540 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:32.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60d9400640 1 -- 172.21.15.42:0/3921667641 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40b0070 con 0x7f60dc09b8e0 2024-05-28T13:59:32.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60d9400640 1 -- 172.21.15.42:0/3921667641 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c40ab600 con 0x7f60dc09b8e0 2024-05-28T13:59:32.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60d9400640 1 -- 172.21.15.42:0/3921667641 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40ab9d0 con 0x7f60dc09b8e0 2024-05-28T13:59:32.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60e0e00640 1 -- 172.21.15.42:0/3921667641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60dc09b8e0 msgr2=0x7f60dc09aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:32.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60e0e00640 1 --2- 172.21.15.42:0/3921667641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60dc09b8e0 0x7f60dc09aa50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f60dc072fd0 tx=0x7f60c401c520 comp rx=0 tx=0).stop 2024-05-28T13:59:32.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60e0e00640 1 -- 172.21.15.42:0/3921667641 shutdown_connections 2024-05-28T13:59:32.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60e0e00640 1 --2- 172.21.15.42:0/3921667641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60dc09af90 0x7f60dc15e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60e0e00640 1 --2- 172.21.15.42:0/3921667641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60dc09b8e0 0x7f60dc09aa50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60e0e00640 1 --2- 172.21.15.42:0/3921667641 >> v2:172.21.15.63:3300/0 conn(0x7f60dc0b0e80 0x7f60dc06cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60e0e00640 1 -- 172.21.15.42:0/3921667641 >> 172.21.15.42:0/3921667641 conn(0x7f60dc0b3f90 msgr2=0x7f60dc0b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:32.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60e0e00640 1 -- 172.21.15.42:0/3921667641 shutdown_connections 2024-05-28T13:59:32.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60e0e00640 1 -- 172.21.15.42:0/3921667641 wait complete. 2024-05-28T13:59:32.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60e0e00640 1 Processor -- start 2024-05-28T13:59:32.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.165+0000 7f60e0e00640 1 -- start start 2024-05-28T13:59:32.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60e0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60dc09af90 0x7f60dc0c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60e0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60dc09b8e0 0x7f60dc0c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60e0e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60dc0b0e80 0x7f60dc0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60e0e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc162410 con 0x7f60dc09b8e0 2024-05-28T13:59:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60e0e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc0d1a90 con 0x7f60dc09af90 2024-05-28T13:59:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60e0e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60dc162590 con 0x7f60dc0b0e80 2024-05-28T13:59:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60dc09b8e0 0x7f60dc0c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60dc09b8e0 0x7f60dc0c60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41724/0 (socket says 172.21.15.42:41724) 2024-05-28T13:59:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d1400640 1 -- 172.21.15.42:0/2856058224 learned_addr learned my addr 172.21.15.42:0/2856058224 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d9e00640 1 --2- 172.21.15.42:0/2856058224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60dc09af90 0x7f60dc0c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d1400640 1 -- 172.21.15.42:0/2856058224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60dc0b0e80 msgr2=0x7f60dc0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d1400640 1 --2- 172.21.15.42:0/2856058224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60dc0b0e80 0x7f60dc0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d1400640 1 -- 172.21.15.42:0/2856058224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60dc09af90 msgr2=0x7f60dc0c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d1400640 1 --2- 172.21.15.42:0/2856058224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60dc09af90 0x7f60dc0c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d1400640 1 -- 172.21.15.42:0/2856058224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc15b930 con 0x7f60dc09b8e0 2024-05-28T13:59:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d9e00640 1 --2- 172.21.15.42:0/2856058224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60dc09af90 0x7f60dc0c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T13:59:32.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d1400640 1 --2- 172.21.15.42:0/2856058224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60dc09b8e0 0x7f60dc0c60b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f60c401abd0 tx=0x7f60c40192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:32.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d2a00640 1 -- 172.21.15.42:0/2856058224 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40b0040 con 0x7f60dc09b8e0 2024-05-28T13:59:32.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d2a00640 1 -- 172.21.15.42:0/2856058224 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c40b4420 con 0x7f60dc09b8e0 2024-05-28T13:59:32.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d2a00640 1 -- 172.21.15.42:0/2856058224 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40b47f0 con 0x7f60dc09b8e0 2024-05-28T13:59:32.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60e0e00640 1 -- 172.21.15.42:0/2856058224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60dc15bc10 con 0x7f60dc09b8e0 2024-05-28T13:59:32.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60e0e00640 1 -- 172.21.15.42:0/2856058224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60dc15c1d0 con 0x7f60dc09b8e0 2024-05-28T13:59:32.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60e0e00640 1 -- 172.21.15.42:0/2856058224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60dc0a0550 con 0x7f60dc09b8e0 2024-05-28T13:59:32.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.169+0000 7f60d2a00640 1 -- 172.21.15.42:0/2856058224 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f60c40b4990 con 0x7f60dc09b8e0 2024-05-28T13:59:32.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.173+0000 7f60d2a00640 1 --2- 172.21.15.42:0/2856058224 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60b8068380 0x7f60b806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.173+0000 7f60d2a00640 1 -- 172.21.15.42:0/2856058224 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f60c41205c0 con 0x7f60dc09b8e0 2024-05-28T13:59:32.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.173+0000 7f60d9e00640 1 --2- 172.21.15.42:0/2856058224 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60b8068380 0x7f60b806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.173+0000 7f60d9e00640 1 --2- 172.21.15.42:0/2856058224 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60b8068380 0x7f60b806a840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f60cc007440 tx=0x7f60cc00b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:32.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.177+0000 7f60d2a00640 1 -- 172.21.15.42:0/2856058224 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f60c4016110 con 0x7f60dc09b8e0 2024-05-28T13:59:32.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.401+0000 7f60e0e00640 1 -- 172.21.15.42:0/2856058224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f60dc0ceaa0 con 0x7f60dc09b8e0 2024-05-28T13:59:32.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.401+0000 7f60d2a00640 1 -- 172.21.15.42:0/2856058224 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f60c40f0b70 con 0x7f60dc09b8e0 2024-05-28T13:59:32.388 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-05-28T13:59:32.388 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw== 2024-05-28T13:59:32.388 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow" 2024-05-28T13:59:32.388 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-05-28T13:59:32.388 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-28T13:59:32.389 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.0 2024-05-28T13:59:32.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.405+0000 7f60e0e00640 1 -- 172.21.15.42:0/2856058224 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60b8068380 msgr2=0x7f60b806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:32.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.405+0000 7f60e0e00640 1 --2- 172.21.15.42:0/2856058224 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60b8068380 0x7f60b806a840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f60cc007440 tx=0x7f60cc00b040 comp rx=0 tx=0).stop 2024-05-28T13:59:32.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.405+0000 7f60e0e00640 1 -- 172.21.15.42:0/2856058224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60dc09b8e0 msgr2=0x7f60dc0c60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:32.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.405+0000 7f60e0e00640 1 --2- 172.21.15.42:0/2856058224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60dc09b8e0 0x7f60dc0c60b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f60c401abd0 tx=0x7f60c40192c0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.405+0000 7f60e0e00640 1 -- 172.21.15.42:0/2856058224 shutdown_connections 2024-05-28T13:59:32.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.405+0000 7f60e0e00640 1 --2- 172.21.15.42:0/2856058224 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60b8068380 0x7f60b806a840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.405+0000 7f60e0e00640 1 --2- 172.21.15.42:0/2856058224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60dc0b0e80 0x7f60dc0712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.405+0000 7f60e0e00640 1 --2- 172.21.15.42:0/2856058224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60dc09b8e0 0x7f60dc0c60b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.405+0000 7f60e0e00640 1 --2- 172.21.15.42:0/2856058224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60dc09af90 0x7f60dc0c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.405+0000 7f60e0e00640 1 -- 172.21.15.42:0/2856058224 >> 172.21.15.42:0/2856058224 conn(0x7f60dc0b3f90 msgr2=0x7f60dc0af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:32.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.405+0000 7f60e0e00640 1 -- 172.21.15.42:0/2856058224 shutdown_connections 2024-05-28T13:59:32.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.405+0000 7f60e0e00640 1 -- 172.21.15.42:0/2856058224 wait complete. 2024-05-28T13:59:32.410 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T13:59:32.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.565+0000 7fae85200640 1 Processor -- start 2024-05-28T13:59:32.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.565+0000 7fae85200640 1 -- start start 2024-05-28T13:59:32.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.565+0000 7fae85200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae800afa50 0x7fae8006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.565+0000 7fae85200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fae80156e50 0x7fae800d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.565+0000 7fae85200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae8015cb90 con 0x7fae80156a70 2024-05-28T13:59:32.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.565+0000 7fae85200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fae8015ce50 con 0x7fae80156e50 2024-05-28T13:59:32.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.565+0000 7fae85200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fae8015d130 con 0x7fae800afa50 2024-05-28T13:59:32.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.565+0000 7fae7de00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fae80156a70 0x7fae800af340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51906/0 (socket says 172.21.15.42:51906) 2024-05-28T13:59:32.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.565+0000 7fae7de00640 1 -- 172.21.15.42:0/2858084235 learned_addr learned my addr 172.21.15.42:0/2858084235 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:32.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.565+0000 7fae7e800640 1 --2- 172.21.15.42:0/2858084235 >> v2:172.21.15.63:3300/0 conn(0x7fae80156e50 0x7fae800d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.565+0000 7fae75400640 1 --2- 172.21.15.42:0/2858084235 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae800afa50 0x7fae8006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 -- 172.21.15.42:0/2858084235 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2697415157 0 0) 0x7fae8015cb90 con 0x7fae80156a70 2024-05-28T13:59:32.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 -- 172.21.15.42:0/2858084235 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae64003530 con 0x7fae80156a70 2024-05-28T13:59:32.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 -- 172.21.15.42:0/2858084235 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3323826602 0 0) 0x7fae64003530 con 0x7fae80156a70 2024-05-28T13:59:32.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 -- 172.21.15.42:0/2858084235 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae800afa50 msgr2=0x7fae8006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:32.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 --2- 172.21.15.42:0/2858084235 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae800afa50 0x7fae8006dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 -- 172.21.15.42:0/2858084235 >> v2:172.21.15.63:3300/0 conn(0x7fae80156e50 msgr2=0x7fae800d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:32.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 --2- 172.21.15.42:0/2858084235 >> v2:172.21.15.63:3300/0 conn(0x7fae80156e50 0x7fae800d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 -- 172.21.15.42:0/2858084235 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8015d410 con 0x7fae80156a70 2024-05-28T13:59:32.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 -- 172.21.15.42:0/2858084235 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fae70001680 con 0x7fae80156a70 2024-05-28T13:59:32.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 -- 172.21.15.42:0/2858084235 >> v1:172.21.15.42:6789/0 conn(0x7fae80156a70 legacy=0x7fae800af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:32.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 --2- 172.21.15.42:0/2858084235 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae800afa50 0x7fae640039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 --2- 172.21.15.42:0/2858084235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64003f40 0x7fae64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 --2- 172.21.15.42:0/2858084235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae64004d50 0x7fae640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 -- 172.21.15.42:0/2858084235 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fae8015ce50 con 0x7fae64004d50 2024-05-28T13:59:32.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 -- 172.21.15.42:0/2858084235 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae8015d130 con 0x7fae64003f40 2024-05-28T13:59:32.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7d400640 1 -- 172.21.15.42:0/2858084235 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fae6400b9d0 con 0x7fae800afa50 2024-05-28T13:59:32.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7e800640 1 --2- 172.21.15.42:0/2858084235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae64004d50 0x7fae640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7de00640 1 --2- 172.21.15.42:0/2858084235 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae800afa50 0x7fae640039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae75400640 1 --2- 172.21.15.42:0/2858084235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64003f40 0x7fae64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7e800640 1 -- 172.21.15.42:0/2858084235 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae800afa50 msgr2=0x7fae640039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:32.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7e800640 1 --2- 172.21.15.42:0/2858084235 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae800afa50 0x7fae640039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7e800640 1 -- 172.21.15.42:0/2858084235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64003f40 msgr2=0x7fae64004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:32.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7e800640 1 --2- 172.21.15.42:0/2858084235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64003f40 0x7fae64004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.569+0000 7fae7e800640 1 -- 172.21.15.42:0/2858084235 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae64003530 con 0x7fae64004d50 2024-05-28T13:59:32.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae7e800640 1 --2- 172.21.15.42:0/2858084235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae64004d50 0x7fae640051d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fae78009f90 tx=0x7fae7801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6dcdf2f62109afe server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:32.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae7d400640 1 -- 172.21.15.42:0/2858084235 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae780b2070 con 0x7fae64004d50 2024-05-28T13:59:32.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae7d400640 1 -- 172.21.15.42:0/2858084235 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae780a5c90 con 0x7fae64004d50 2024-05-28T13:59:32.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae7d400640 1 -- 172.21.15.42:0/2858084235 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78009c10 con 0x7fae64004d50 2024-05-28T13:59:32.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae85200640 1 -- 172.21.15.42:0/2858084235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae64004d50 msgr2=0x7fae640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:32.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae85200640 1 --2- 172.21.15.42:0/2858084235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae64004d50 0x7fae640051d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fae78009f90 tx=0x7fae7801cc20 comp rx=0 tx=0).stop 2024-05-28T13:59:32.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae7e800640 1 -- 172.21.15.42:0/2858084235 reap_dead start 2024-05-28T13:59:32.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae85200640 1 -- 172.21.15.42:0/2858084235 shutdown_connections 2024-05-28T13:59:32.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae85200640 1 -- 172.21.15.42:0/2858084235 >> 172.21.15.42:0/2858084235 conn(0x7fae8006bdf0 msgr2=0x7fae8006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:32.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae85200640 1 -- 172.21.15.42:0/2858084235 shutdown_connections 2024-05-28T13:59:32.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae85200640 1 -- 172.21.15.42:0/2858084235 wait complete. 2024-05-28T13:59:32.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae85200640 1 Processor -- start 2024-05-28T13:59:32.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae85200640 1 -- start start 2024-05-28T13:59:32.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae85200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64004d50 0x7fae80148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae85200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae80155e10 0x7fae80071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae85200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae80073630 0x7fae8015b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae85200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fae8006dee0 con 0x7fae80073630 2024-05-28T13:59:32.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae85200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae8006dd60 con 0x7fae64004d50 2024-05-28T13:59:32.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae85200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fae8006e060 con 0x7fae80155e10 2024-05-28T13:59:32.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.573+0000 7fae7e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae80073630 0x7fae8015b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae7e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae80073630 0x7fae8015b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41742/0 (socket says 172.21.15.42:41742) 2024-05-28T13:59:32.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae7e800640 1 -- 172.21.15.42:0/951018260 learned_addr learned my addr 172.21.15.42:0/951018260 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:32.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae7de00640 1 --2- 172.21.15.42:0/951018260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64004d50 0x7fae80148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae75400640 1 --2- 172.21.15.42:0/951018260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae80155e10 0x7fae80071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae7e800640 1 -- 172.21.15.42:0/951018260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae80155e10 msgr2=0x7fae80071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:32.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae7e800640 1 --2- 172.21.15.42:0/951018260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae80155e10 0x7fae80071d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae7e800640 1 -- 172.21.15.42:0/951018260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64004d50 msgr2=0x7fae80148850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:32.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae7e800640 1 --2- 172.21.15.42:0/951018260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64004d50 0x7fae80148850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae7e800640 1 -- 172.21.15.42:0/951018260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae80148d90 con 0x7fae80073630 2024-05-28T13:59:32.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae7e800640 1 --2- 172.21.15.42:0/951018260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae80073630 0x7fae8015b800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fae780b0040 tx=0x7fae78009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:32.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae76a00640 1 -- 172.21.15.42:0/951018260 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae780b2040 con 0x7fae80073630 2024-05-28T13:59:32.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae76a00640 1 -- 172.21.15.42:0/951018260 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae780bc070 con 0x7fae80073630 2024-05-28T13:59:32.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae85200640 1 -- 172.21.15.42:0/951018260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae80072590 con 0x7fae80073630 2024-05-28T13:59:32.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae76a00640 1 -- 172.21.15.42:0/951018260 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78009c10 con 0x7fae80073630 2024-05-28T13:59:32.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae85200640 1 -- 172.21.15.42:0/951018260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae80072b50 con 0x7fae80073630 2024-05-28T13:59:32.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae85200640 1 -- 172.21.15.42:0/951018260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae80068a90 con 0x7fae80073630 2024-05-28T13:59:32.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae76a00640 1 -- 172.21.15.42:0/951018260 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fae780adcc0 con 0x7fae80073630 2024-05-28T13:59:32.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae76a00640 1 --2- 172.21.15.42:0/951018260 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fae5c068380 0x7fae5c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:32.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.577+0000 7fae76a00640 1 -- 172.21.15.42:0/951018260 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fae781213a0 con 0x7fae80073630 2024-05-28T13:59:32.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.581+0000 7fae7de00640 1 --2- 172.21.15.42:0/951018260 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fae5c068380 0x7fae5c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:32.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.581+0000 7fae7de00640 1 --2- 172.21.15.42:0/951018260 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fae5c068380 0x7fae5c06a840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae70000c10 tx=0x7fae70007900 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:32.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.585+0000 7fae76a00640 1 -- 172.21.15.42:0/951018260 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fae78016990 con 0x7fae80073630 2024-05-28T13:59:32.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.781+0000 7fae85200640 1 -- 172.21.15.42:0/951018260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae800b5160 con 0x7fae80073630 2024-05-28T13:59:32.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.781+0000 7fae76a00640 1 -- 172.21.15.42:0/951018260 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7fae780f1980 con 0x7fae80073630 2024-05-28T13:59:32.768 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T13:59:32.768 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2044753493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2044753493},{"type":"v1","addr":"172.21.15.42:6835","nonce":2044753493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3332210819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3332210819},{"type":"v1","addr":"172.21.15.42:6837","nonce":3332210819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":54782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1050322479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1050322479},{"type":"v1","addr":"172.21.15.63:6834","nonce":1050322479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T13:59:26.845096+0000","modified":"2024-05-28T13:59:29.868153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":54785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54785":{"gid":54785,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/2114729398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2114729398},{"type":"v1","addr":"172.21.15.63:6835","nonce":2114729398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":54785,"qdb_cluster":[54785]},"id":2}]} 2024-05-28T13:59:32.769 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 17 2024-05-28T13:59:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.785+0000 7fae85200640 1 -- 172.21.15.42:0/951018260 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fae5c068380 msgr2=0x7fae5c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.785+0000 7fae85200640 1 --2- 172.21.15.42:0/951018260 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fae5c068380 0x7fae5c06a840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae70000c10 tx=0x7fae70007900 comp rx=0 tx=0).stop 2024-05-28T13:59:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.785+0000 7fae85200640 1 -- 172.21.15.42:0/951018260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae80073630 msgr2=0x7fae8015b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.785+0000 7fae85200640 1 --2- 172.21.15.42:0/951018260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae80073630 0x7fae8015b800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fae780b0040 tx=0x7fae78009750 comp rx=0 tx=0).stop 2024-05-28T13:59:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.785+0000 7fae85200640 1 -- 172.21.15.42:0/951018260 shutdown_connections 2024-05-28T13:59:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.785+0000 7fae85200640 1 --2- 172.21.15.42:0/951018260 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fae5c068380 0x7fae5c06a840 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.785+0000 7fae85200640 1 --2- 172.21.15.42:0/951018260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae80073630 0x7fae8015b800 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.785+0000 7fae85200640 1 --2- 172.21.15.42:0/951018260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae80155e10 0x7fae80071d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.785+0000 7fae85200640 1 --2- 172.21.15.42:0/951018260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae64004d50 0x7fae80148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:32.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.785+0000 7fae85200640 1 -- 172.21.15.42:0/951018260 >> 172.21.15.42:0/951018260 conn(0x7fae8006bdf0 msgr2=0x7fae800b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:32.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.785+0000 7fae85200640 1 -- 172.21.15.42:0/951018260 shutdown_connections 2024-05-28T13:59:32.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:32.785+0000 7fae85200640 1 -- 172.21.15.42:0/951018260 wait complete. 2024-05-28T13:59:33.192 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-28T13:59:26.845096+0000', 'modified': '2024-05-28T13:59:29.868153+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 54785}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_54785': {'gid': 54785, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6835/2114729398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6833', 'nonce': 2114729398}, {'type': 'v1', 'addr': '172.21.15.63:6835', 'nonce': 2114729398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 54785, 'qdb_cluster': [54785]} 2024-05-28T13:59:33.192 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-28T13:59:33.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-28T13:59:33.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59b2c00640 1 Processor -- start 2024-05-28T13:59:33.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59b2c00640 1 -- start start 2024-05-28T13:59:33.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59b2c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f59ac15bf30 0x7f59ac15c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59b2c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59ac0cf230 0x7f59ac072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59b2c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59ac0cfc10 con 0x7f59ac072ba0 2024-05-28T13:59:33.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59b2c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f59ac0721d0 con 0x7f59ac15bf30 2024-05-28T13:59:33.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59b2c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f59ac0cfef0 con 0x7f59ac0cf230 2024-05-28T13:59:33.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59abe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f59ac15bf30 0x7f59ac15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59a3400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59ac0cf230 0x7f59ac072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59abe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f59ac15bf30 0x7f59ac15c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:54142/0 (socket says 172.21.15.42:54142) 2024-05-28T13:59:33.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59a3400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59ac0cf230 0x7f59ac072660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:60980/0 (socket says 172.21.15.42:60980) 2024-05-28T13:59:33.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59abe00640 1 -- 172.21.15.42:0/2915984699 learned_addr learned my addr 172.21.15.42:0/2915984699 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:33.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59b0c00640 1 --1- 172.21.15.42:0/2915984699 >> v1:172.21.15.63:6789/0 conn(0x7f59ac072ba0 0x7f59ac06ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:54364/0 (socket says 172.21.15.42:54364) 2024-05-28T13:59:33.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59a3400640 1 -- 172.21.15.42:0/2915984699 >> v2:172.21.15.63:3300/0 conn(0x7f59ac15bf30 msgr2=0x7f59ac15c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:33.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59a3400640 1 --2- 172.21.15.42:0/2915984699 >> v2:172.21.15.63:3300/0 conn(0x7f59ac15bf30 0x7f59ac15c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59a3400640 1 -- 172.21.15.42:0/2915984699 >> v1:172.21.15.63:6789/0 conn(0x7f59ac072ba0 legacy=0x7f59ac06ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.353+0000 7f59a3400640 1 -- 172.21.15.42:0/2915984699 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac0d01d0 con 0x7f59ac0cf230 2024-05-28T13:59:33.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.357+0000 7f59a3400640 1 --2- 172.21.15.42:0/2915984699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59ac0cf230 0x7f59ac072660 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5994009d90 tx=0x7f599401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5feb5814281b5c6a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:33.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.357+0000 7f59ab400640 1 -- 172.21.15.42:0/2915984699 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59940b2070 con 0x7f59ac0cf230 2024-05-28T13:59:33.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.357+0000 7f59ab400640 1 -- 172.21.15.42:0/2915984699 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59940a5920 con 0x7f59ac0cf230 2024-05-28T13:59:33.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.357+0000 7f59ab400640 1 -- 172.21.15.42:0/2915984699 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59940a5cf0 con 0x7f59ac0cf230 2024-05-28T13:59:33.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.357+0000 7f59b2c00640 1 -- 172.21.15.42:0/2915984699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59ac0cf230 msgr2=0x7f59ac072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.357+0000 7f59b2c00640 1 --2- 172.21.15.42:0/2915984699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59ac0cf230 0x7f59ac072660 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5994009d90 tx=0x7f599401c940 comp rx=0 tx=0).stop 2024-05-28T13:59:33.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.357+0000 7f59b2c00640 1 -- 172.21.15.42:0/2915984699 shutdown_connections 2024-05-28T13:59:33.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.357+0000 7f59b2c00640 1 --2- 172.21.15.42:0/2915984699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59ac0cf230 0x7f59ac072660 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.357+0000 7f59b2c00640 1 --2- 172.21.15.42:0/2915984699 >> v2:172.21.15.63:3300/0 conn(0x7f59ac15bf30 0x7f59ac15c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.357+0000 7f59b2c00640 1 -- 172.21.15.42:0/2915984699 >> 172.21.15.42:0/2915984699 conn(0x7f59ac06b9f0 msgr2=0x7f59ac06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:33.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.357+0000 7f59b2c00640 1 -- 172.21.15.42:0/2915984699 shutdown_connections 2024-05-28T13:59:33.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.357+0000 7f59b2c00640 1 -- 172.21.15.42:0/2915984699 wait complete. 2024-05-28T13:59:33.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59b2c00640 1 Processor -- start 2024-05-28T13:59:33.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59b2c00640 1 -- start start 2024-05-28T13:59:33.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59b2c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59ac072ba0 0x7f59ac14a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59b2c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59ac0cf230 0x7f59ac14a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59b2c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59ac15bf30 0x7f59ac14ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59b2c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac06f4e0 con 0x7f59ac0cf230 2024-05-28T13:59:33.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59b2c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac06f360 con 0x7f59ac072ba0 2024-05-28T13:59:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59b2c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f59ac0cf690 con 0x7f59ac15bf30 2024-05-28T13:59:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59a3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59ac0cf230 0x7f59ac14a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59a3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59ac0cf230 0x7f59ac14a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41752/0 (socket says 172.21.15.42:41752) 2024-05-28T13:59:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59a3400640 1 -- 172.21.15.42:0/3729891380 learned_addr learned my addr 172.21.15.42:0/3729891380 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59abe00640 1 --2- 172.21.15.42:0/3729891380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59ac072ba0 0x7f59ac14a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59b0c00640 1 --2- 172.21.15.42:0/3729891380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59ac15bf30 0x7f59ac14ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59a3400640 1 -- 172.21.15.42:0/3729891380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59ac15bf30 msgr2=0x7f59ac14ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59a3400640 1 --2- 172.21.15.42:0/3729891380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59ac15bf30 0x7f59ac14ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59a3400640 1 -- 172.21.15.42:0/3729891380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59ac072ba0 msgr2=0x7f59ac14a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59a3400640 1 --2- 172.21.15.42:0/3729891380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59ac072ba0 0x7f59ac14a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59a3400640 1 -- 172.21.15.42:0/3729891380 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac14b750 con 0x7f59ac0cf230 2024-05-28T13:59:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59a3400640 1 --2- 172.21.15.42:0/3729891380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59ac0cf230 0x7f59ac14a6f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f59940b0040 tx=0x7f59940196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59a8c00640 1 -- 172.21.15.42:0/3729891380 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59940b2050 con 0x7f59ac0cf230 2024-05-28T13:59:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59a8c00640 1 -- 172.21.15.42:0/3729891380 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59940be070 con 0x7f59ac0cf230 2024-05-28T13:59:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59b2c00640 1 -- 172.21.15.42:0/3729891380 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ac152930 con 0x7f59ac0cf230 2024-05-28T13:59:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59a8c00640 1 -- 172.21.15.42:0/3729891380 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59940ada70 con 0x7f59ac0cf230 2024-05-28T13:59:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.361+0000 7f59b2c00640 1 -- 172.21.15.42:0/3729891380 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ac152ef0 con 0x7f59ac0cf230 2024-05-28T13:59:33.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.365+0000 7f59a8c00640 1 -- 172.21.15.42:0/3729891380 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f59940adc10 con 0x7f59ac0cf230 2024-05-28T13:59:33.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.365+0000 7f59a8c00640 1 --2- 172.21.15.42:0/3729891380 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5988068380 0x7f598806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.365+0000 7f59a8c00640 1 -- 172.21.15.42:0/3729891380 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5994121bb0 con 0x7f59ac0cf230 2024-05-28T13:59:33.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.365+0000 7f59b5fbc000 1 Processor -- start 2024-05-28T13:59:33.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.365+0000 7f59abe00640 1 --2- 172.21.15.42:0/3729891380 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5988068380 0x7f598806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.365+0000 7f59b5fbc000 1 -- start start 2024-05-28T13:59:33.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.365+0000 7f59abe00640 1 --2- 172.21.15.42:0/3729891380 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5988068380 0x7f598806a840 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f599c009b30 tx=0x7f599c009240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:33.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.365+0000 7f59b5fbc000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8fbc120 0x5558c8fbc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.365+0000 7f59b5fbc000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbca40 0x5558c8f6f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.365+0000 7f59b5fbc000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8f6f650 0x5558c8f6fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5558c8dc0420 con 0x5558c8fbca40 2024-05-28T13:59:33.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5558c8d5abf0 con 0x5558c8fbc120 2024-05-28T13:59:33.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5558c8dbc3e0 con 0x5558c8f6f650 2024-05-28T13:59:33.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59a3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbca40 0x5558c8f6f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59a3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbca40 0x5558c8f6f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41760/0 (socket says 172.21.15.42:41760) 2024-05-28T13:59:33.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59a3400640 1 -- 172.21.15.42:0/2352986831 learned_addr learned my addr 172.21.15.42:0/2352986831 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:33.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59a3400640 1 -- 172.21.15.42:0/2352986831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8f6f650 msgr2=0x5558c8f6fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:33.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59a3400640 1 --2- 172.21.15.42:0/2352986831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8f6f650 0x5558c8f6fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59a3400640 1 -- 172.21.15.42:0/2352986831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8fbc120 msgr2=0x5558c8fbc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:33.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59a3400640 1 --2- 172.21.15.42:0/2352986831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8fbc120 0x5558c8fbc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59a3400640 1 -- 172.21.15.42:0/2352986831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558c8e68450 con 0x5558c8fbca40 2024-05-28T13:59:33.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59a3400640 1 --2- 172.21.15.42:0/2352986831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbca40 0x5558c8f6f110 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f59940a5fd0 tx=0x7f599401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12063a5be5aa46bc server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:33.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59a0c00640 1 -- 172.21.15.42:0/2352986831 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599410a0d0 con 0x5558c8fbca40 2024-05-28T13:59:33.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59a0c00640 1 -- 172.21.15.42:0/2352986831 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59940b3040 con 0x5558c8fbca40 2024-05-28T13:59:33.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59a0c00640 1 -- 172.21.15.42:0/2352986831 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59940fa6d0 con 0x5558c8fbca40 2024-05-28T13:59:33.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 -- 172.21.15.42:0/2352986831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbca40 msgr2=0x5558c8f6f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/2352986831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbca40 0x5558c8f6f110 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f59940a5fd0 tx=0x7f599401c940 comp rx=0 tx=0).stop 2024-05-28T13:59:33.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 -- 172.21.15.42:0/2352986831 shutdown_connections 2024-05-28T13:59:33.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/2352986831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8f6f650 0x5558c8f6fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/2352986831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbca40 0x5558c8f6f110 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/2352986831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8fbc120 0x5558c8fbc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 -- 172.21.15.42:0/2352986831 >> 172.21.15.42:0/2352986831 conn(0x5558c8f9cec0 msgr2=0x5558c8f9f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:33.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 -- 172.21.15.42:0/2352986831 shutdown_connections 2024-05-28T13:59:33.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 -- 172.21.15.42:0/2352986831 wait complete. 2024-05-28T13:59:33.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 -- start start 2024-05-28T13:59:33.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 Processor -- start 2024-05-28T13:59:33.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8f6f650 0x5558c8f74b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8fbc120 0x5558c8f75040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8fbca40 0x5558c8f75580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5558c8d60ec0 con 0x5558c8f6f650 2024-05-28T13:59:33.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5558c8e41120 con 0x5558c8fbc120 2024-05-28T13:59:33.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5558c8dcb350 con 0x5558c8fbca40 2024-05-28T13:59:33.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59b5fbc000 10 client.0 _ll_register_callbacks cb 0x5558c8d9d4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T13:59:33.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.369+0000 7f59abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8f6f650 0x5558c8f74b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8f6f650 0x5558c8f74b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41772/0 (socket says 172.21.15.42:41772) 2024-05-28T13:59:33.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59abe00640 1 -- 172.21.15.42:0/529927358 learned_addr learned my addr 172.21.15.42:0/529927358 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:33.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59a3400640 1 --2- 172.21.15.42:0/529927358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8fbc120 0x5558c8f75040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59b0c00640 1 --2- 172.21.15.42:0/529927358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8fbca40 0x5558c8f75580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59abe00640 1 -- 172.21.15.42:0/529927358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8fbca40 msgr2=0x5558c8f75580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:33.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59abe00640 1 --2- 172.21.15.42:0/529927358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8fbca40 0x5558c8f75580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59abe00640 1 -- 172.21.15.42:0/529927358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8fbc120 msgr2=0x5558c8f75040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:33.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59abe00640 1 --2- 172.21.15.42:0/529927358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8fbc120 0x5558c8f75040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59abe00640 1 -- 172.21.15.42:0/529927358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558c8e05f80 con 0x5558c8f6f650 2024-05-28T13:59:33.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59abe00640 1 --2- 172.21.15.42:0/529927358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8f6f650 0x5558c8f74b00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f599c0149c0 tx=0x7f599c0149f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8392c27025697b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:33.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f5999600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T13:59:33.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f5999600640 1 -- 172.21.15.42:0/529927358 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c025020 con 0x5558c8f6f650 2024-05-28T13:59:33.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f5999600640 1 -- 172.21.15.42:0/529927358 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f599c0213f0 con 0x5558c8f6f650 2024-05-28T13:59:33.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59b5fbc000 1 -- 172.21.15.42:0/529927358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59940b3040 con 0x5558c8f6f650 2024-05-28T13:59:33.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f5999600640 1 -- 172.21.15.42:0/529927358 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f599c01d040 con 0x5558c8f6f650 2024-05-28T13:59:33.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59b5fbc000 10 client.5403 fetch_fsmap learned FSMap version 17 2024-05-28T13:59:33.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59b5fbc000 1 -- 172.21.15.42:0/529927358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5558c8d99f00 con 0x5558c8f6f650 2024-05-28T13:59:33.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f5999600640 1 -- 172.21.15.42:0/529927358 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59940b3040 con 0x5558c8f6f650 2024-05-28T13:59:33.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f5999600640 1 -- 172.21.15.42:0/529927358 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f599c020030 con 0x5558c8f6f650 2024-05-28T13:59:33.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59b5fbc000 10 client.5403 fetch_fsmap finished waiting for FSMap version 17 2024-05-28T13:59:33.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59b5fbc000 10 client.5403 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-28T13:59:33.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59b5fbc000 20 client.5403 populate_metadata read hostname 'smithi042' 2024-05-28T13:59:33.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/529927358 >> [v2:172.21.15.63:6833/2114729398,v1:172.21.15.63:6835/2114729398] conn(0x5558c8fcb3c0 0x5558c8fcd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59b5fbc000 4 client.5403 mds_command: new command op to 54785 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-28T13:59:33.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59b5fbc000 1 -- 172.21.15.42:0/529927358 --> [v2:172.21.15.63:6833/2114729398,v1:172.21.15.63:6835/2114729398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558c8d59bc0 con 0x5558c8fcb3c0 2024-05-28T13:59:33.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59a3400640 1 --2- 172.21.15.42:0/529927358 >> [v2:172.21.15.63:6833/2114729398,v1:172.21.15.63:6835/2114729398] conn(0x5558c8fcb3c0 0x5558c8fcd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f59a3400640 1 --2- 172.21.15.42:0/529927358 >> [v2:172.21.15.63:6833/2114729398,v1:172.21.15.63:6835/2114729398] conn(0x5558c8fcb3c0 0x5558c8fcd7b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b863e2a4e9dcbd06 server_cookie=6142a6713873ea9 in_seq=0 out_seq=0 2024-05-28T13:59:33.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f5999600640 10 client.5403 ms_handle_connect on v2:172.21.15.63:6833/2114729398 2024-05-28T13:59:33.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f5999600640 1 -- 172.21.15.42:0/529927358 <== mds.0 v2:172.21.15.63:6833/2114729398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558c8d59bc0 con 0x5558c8fcb3c0 2024-05-28T13:59:33.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.373+0000 7f5999600640 10 client.5403 handle_command_reply: tid=0 2024-05-28T13:59:33.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.377+0000 7f59b5fbc000 1 client.5403 shutdown 2024-05-28T13:59:33.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.377+0000 7f59b5fbc000 1 -- 172.21.15.42:0/529927358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8f6f650 msgr2=0x5558c8f74b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.377+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/529927358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8f6f650 0x5558c8f74b00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f599c0149c0 tx=0x7f599c0149f0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.377+0000 7f59b5fbc000 1 -- 172.21.15.42:0/529927358 shutdown_connections 2024-05-28T13:59:33.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.377+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/529927358 >> [v2:172.21.15.63:6833/2114729398,v1:172.21.15.63:6835/2114729398] conn(0x5558c8fcb3c0 0x5558c8fcd7b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.377+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/529927358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8fbca40 0x5558c8f75580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.377+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/529927358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8fbc120 0x5558c8f75040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.377+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/529927358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8f6f650 0x5558c8f74b00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.377+0000 7f5999600640 0 client.5403 ms_handle_reset on v2:172.21.15.63:6833/2114729398 2024-05-28T13:59:33.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.377+0000 7f59b5fbc000 1 -- 172.21.15.42:0/529927358 >> 172.21.15.42:0/529927358 conn(0x5558c8f9cec0 msgr2=0x5558c8f9f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:33.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.377+0000 7f59b5fbc000 1 -- 172.21.15.42:0/529927358 shutdown_connections 2024-05-28T13:59:33.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.377+0000 7f59b5fbc000 1 -- 172.21.15.42:0/529927358 wait complete. 2024-05-28T13:59:33.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.377+0000 7f59b5fbc000 20 client.5403 trim_cache size 0 max 16384 2024-05-28T13:59:33.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.393+0000 7f59b5fbc000 1 Processor -- start 2024-05-28T13:59:33.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.393+0000 7f59b5fbc000 1 -- start start 2024-05-28T13:59:33.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.393+0000 7f59b5fbc000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8f6f650 0x5558c8f6e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.393+0000 7f59b5fbc000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbc120 0x5558c8f6ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.393+0000 7f59b5fbc000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8fbca40 0x5558c8fcd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.393+0000 7f59b5fbc000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac202a00 con 0x5558c8fbc120 2024-05-28T13:59:33.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.393+0000 7f59b5fbc000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac155430 con 0x5558c8f6f650 2024-05-28T13:59:33.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.393+0000 7f59b5fbc000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f59ac155a80 con 0x5558c8fbca40 2024-05-28T13:59:33.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.393+0000 7f59a3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbc120 0x5558c8f6ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.393+0000 7f59a3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbc120 0x5558c8f6ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41786/0 (socket says 172.21.15.42:41786) 2024-05-28T13:59:33.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.393+0000 7f59a3400640 1 -- 172.21.15.42:0/1176650940 learned_addr learned my addr 172.21.15.42:0/1176650940 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:33.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.393+0000 7f59b0c00640 1 --2- 172.21.15.42:0/1176650940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8fbca40 0x5558c8fcd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.393+0000 7f59abe00640 1 --2- 172.21.15.42:0/1176650940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8f6f650 0x5558c8f6e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59a3400640 1 -- 172.21.15.42:0/1176650940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8fbca40 msgr2=0x5558c8fcd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:33.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59a3400640 1 --2- 172.21.15.42:0/1176650940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8fbca40 0x5558c8fcd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59a3400640 1 -- 172.21.15.42:0/1176650940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8f6f650 msgr2=0x5558c8f6e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:33.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59a3400640 1 --2- 172.21.15.42:0/1176650940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8f6f650 0x5558c8f6e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59a3400640 1 -- 172.21.15.42:0/1176650940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558c8fbc870 con 0x5558c8fbc120 2024-05-28T13:59:33.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59abe00640 1 --2- 172.21.15.42:0/1176650940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8f6f650 0x5558c8f6e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:33.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59a3400640 1 --2- 172.21.15.42:0/1176650940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbc120 0x5558c8f6ed50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f59940a5fd0 tx=0x7f59940c5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d162b76f8a6a7bff server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:33.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f599be00640 1 -- 172.21.15.42:0/1176650940 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59940fa6d0 con 0x5558c8fbc120 2024-05-28T13:59:33.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f599be00640 1 -- 172.21.15.42:0/1176650940 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59940efe10 con 0x5558c8fbc120 2024-05-28T13:59:33.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f599be00640 1 -- 172.21.15.42:0/1176650940 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59940ee180 con 0x5558c8fbc120 2024-05-28T13:59:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 -- 172.21.15.42:0/1176650940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbc120 msgr2=0x5558c8f6ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/1176650940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbc120 0x5558c8f6ed50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f59940a5fd0 tx=0x7f59940c5000 comp rx=0 tx=0).stop 2024-05-28T13:59:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 -- 172.21.15.42:0/1176650940 shutdown_connections 2024-05-28T13:59:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/1176650940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8fbca40 0x5558c8fcd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/1176650940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbc120 0x5558c8f6ed50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/1176650940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8f6f650 0x5558c8f6e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 -- 172.21.15.42:0/1176650940 >> 172.21.15.42:0/1176650940 conn(0x5558c8f9cec0 msgr2=0x5558c8f6e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 -- 172.21.15.42:0/1176650940 shutdown_connections 2024-05-28T13:59:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 -- 172.21.15.42:0/1176650940 wait complete. 2024-05-28T13:59:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 -- start start 2024-05-28T13:59:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 Processor -- start 2024-05-28T13:59:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8f6f650 0x5558c8fd0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8fbc120 0x5558c8fe7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbca40 0x5558c8fe7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5558c8dbf730 con 0x5558c8fbca40 2024-05-28T13:59:33.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5558c8d8c9e0 con 0x5558c8fbc120 2024-05-28T13:59:33.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5558c8dc4a50 con 0x5558c8f6f650 2024-05-28T13:59:33.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b5fbc000 10 client.0 _ll_register_callbacks cb 0x7f599c0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T13:59:33.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbca40 0x5558c8fe7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbca40 0x5558c8fe7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41802/0 (socket says 172.21.15.42:41802) 2024-05-28T13:59:33.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b0c00640 1 -- 172.21.15.42:0/2746920064 learned_addr learned my addr 172.21.15.42:0/2746920064 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:33.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59a3400640 1 --2- 172.21.15.42:0/2746920064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8fbc120 0x5558c8fe7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59abe00640 1 --2- 172.21.15.42:0/2746920064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8f6f650 0x5558c8fd0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b0c00640 1 -- 172.21.15.42:0/2746920064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8f6f650 msgr2=0x5558c8fd0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:33.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b0c00640 1 --2- 172.21.15.42:0/2746920064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8f6f650 0x5558c8fd0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b0c00640 1 -- 172.21.15.42:0/2746920064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8fbc120 msgr2=0x5558c8fe7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:33.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b0c00640 1 --2- 172.21.15.42:0/2746920064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8fbc120 0x5558c8fe7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59b0c00640 1 -- 172.21.15.42:0/2746920064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558c8d46e80 con 0x5558c8fbca40 2024-05-28T13:59:33.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.397+0000 7f59abe00640 1 --2- 172.21.15.42:0/2746920064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8f6f650 0x5558c8fd0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:33.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b0c00640 1 --2- 172.21.15.42:0/2746920064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbca40 0x5558c8fe7880 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f59a40075f0 tx=0x7f59a4001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1675489b020ea67e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:33.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f5999600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T13:59:33.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f5999600640 1 -- 172.21.15.42:0/2746920064 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4016630 con 0x5558c8fbca40 2024-05-28T13:59:33.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f5999600640 1 -- 172.21.15.42:0/2746920064 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f59a4010040 con 0x5558c8fbca40 2024-05-28T13:59:33.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 -- 172.21.15.42:0/2746920064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59940efe10 con 0x5558c8fbca40 2024-05-28T13:59:33.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f5999600640 1 -- 172.21.15.42:0/2746920064 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f59a40167d0 con 0x5558c8fbca40 2024-05-28T13:59:33.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 10 client.5409 fetch_fsmap learned FSMap version 17 2024-05-28T13:59:33.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 -- 172.21.15.42:0/2746920064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5558c8d88b30 con 0x5558c8fbca40 2024-05-28T13:59:33.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f5999600640 1 -- 172.21.15.42:0/2746920064 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59940efe10 con 0x5558c8fbca40 2024-05-28T13:59:33.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f5999600640 1 -- 172.21.15.42:0/2746920064 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f59a4016970 con 0x5558c8fbca40 2024-05-28T13:59:33.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 10 client.5409 fetch_fsmap finished waiting for FSMap version 17 2024-05-28T13:59:33.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 10 client.5409 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-28T13:59:33.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 20 client.5409 populate_metadata read hostname 'smithi042' 2024-05-28T13:59:33.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/2746920064 >> [v2:172.21.15.63:6833/2114729398,v1:172.21.15.63:6835/2114729398] conn(0x5558c8fe7dc0 0x5558c902afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 4 client.5409 mds_command: new command op to 54785 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-28T13:59:33.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 -- 172.21.15.42:0/2746920064 --> [v2:172.21.15.63:6833/2114729398,v1:172.21.15.63:6835/2114729398] -- command(tid 0: {"prefix": "status"}) -- 0x5558c8dcde40 con 0x5558c8fe7dc0 2024-05-28T13:59:33.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59abe00640 1 --2- 172.21.15.42:0/2746920064 >> [v2:172.21.15.63:6833/2114729398,v1:172.21.15.63:6835/2114729398] conn(0x5558c8fe7dc0 0x5558c902afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59abe00640 1 --2- 172.21.15.42:0/2746920064 >> [v2:172.21.15.63:6833/2114729398,v1:172.21.15.63:6835/2114729398] conn(0x5558c8fe7dc0 0x5558c902afb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a55f21e37be88c server_cookie=9f9a18007cd5a0e6 in_seq=0 out_seq=0 2024-05-28T13:59:33.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f5999600640 10 client.5409 ms_handle_connect on v2:172.21.15.63:6833/2114729398 2024-05-28T13:59:33.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f5999600640 1 -- 172.21.15.42:0/2746920064 <== mds.0 v2:172.21.15.63:6833/2114729398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5558c8dcde40 con 0x5558c8fe7dc0 2024-05-28T13:59:33.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f5999600640 10 client.5409 handle_command_reply: tid=0 2024-05-28T13:59:33.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 client.5409 shutdown 2024-05-28T13:59:33.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 -- 172.21.15.42:0/2746920064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbca40 msgr2=0x5558c8fe7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/2746920064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbca40 0x5558c8fe7880 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f59a40075f0 tx=0x7f59a4001610 comp rx=0 tx=0).stop 2024-05-28T13:59:33.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 -- 172.21.15.42:0/2746920064 shutdown_connections 2024-05-28T13:59:33.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/2746920064 >> [v2:172.21.15.63:6833/2114729398,v1:172.21.15.63:6835/2114729398] conn(0x5558c8fe7dc0 0x5558c902afb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/2746920064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558c8fbca40 0x5558c8fe7880 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/2746920064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558c8fbc120 0x5558c8fe7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 --2- 172.21.15.42:0/2746920064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5558c8f6f650 0x5558c8fd0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f5999600640 0 client.5409 ms_handle_reset on v2:172.21.15.63:6833/2114729398 2024-05-28T13:59:33.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 -- 172.21.15.42:0/2746920064 >> 172.21.15.42:0/2746920064 conn(0x5558c8f9cec0 msgr2=0x5558c8f6e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:33.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 -- 172.21.15.42:0/2746920064 shutdown_connections 2024-05-28T13:59:33.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 1 -- 172.21.15.42:0/2746920064 wait complete. 2024-05-28T13:59:33.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b5fbc000 20 client.5409 trim_cache size 0 max 16384 2024-05-28T13:59:33.391 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T13:59:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T13:59:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-05-28T13:59:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 54785, 2024-05-28T13:59:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-28T13:59:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-28T13:59:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-28T13:59:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 5.9567484850000003, 2024-05-28T13:59:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 17, 2024-05-28T13:59:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 34, 2024-05-28T13:59:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 34, 2024-05-28T13:59:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 13.816777105 2024-05-28T13:59:33.392 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T13:59:33.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b2c00640 1 -- 172.21.15.42:0/3729891380 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5988068380 msgr2=0x7f598806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b2c00640 1 --2- 172.21.15.42:0/3729891380 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5988068380 0x7f598806a840 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f599c009b30 tx=0x7f599c009240 comp rx=0 tx=0).stop 2024-05-28T13:59:33.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b2c00640 1 -- 172.21.15.42:0/3729891380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59ac0cf230 msgr2=0x7f59ac14a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b2c00640 1 --2- 172.21.15.42:0/3729891380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59ac0cf230 0x7f59ac14a6f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f59940b0040 tx=0x7f59940196e0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b2c00640 1 -- 172.21.15.42:0/3729891380 shutdown_connections 2024-05-28T13:59:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b2c00640 1 --2- 172.21.15.42:0/3729891380 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5988068380 0x7f598806a840 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b2c00640 1 --2- 172.21.15.42:0/3729891380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59ac15bf30 0x7f59ac14ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b2c00640 1 --2- 172.21.15.42:0/3729891380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59ac0cf230 0x7f59ac14a6f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b2c00640 1 --2- 172.21.15.42:0/3729891380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59ac072ba0 0x7f59ac14a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b2c00640 1 -- 172.21.15.42:0/3729891380 >> 172.21.15.42:0/3729891380 conn(0x7f59ac06b9f0 msgr2=0x7f59ac06bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b2c00640 1 -- 172.21.15.42:0/3729891380 shutdown_connections 2024-05-28T13:59:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.401+0000 7f59b2c00640 1 -- 172.21.15.42:0/3729891380 wait complete. 2024-05-28T13:59:33.407 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-28T13:59:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.561+0000 7f73db800640 1 Processor -- start 2024-05-28T13:59:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.561+0000 7f73db800640 1 -- start start 2024-05-28T13:59:33.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.565+0000 7f73db800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f73d40b6490 0x7f73d406be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.565+0000 7f73db800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d415eee0 con 0x7f73d406b9f0 2024-05-28T13:59:33.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.565+0000 7f73db800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d40ceb20 con 0x7f73d415eb00 2024-05-28T13:59:33.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.565+0000 7f73db800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f73d40cee00 con 0x7f73d40b6490 2024-05-28T13:59:33.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.565+0000 7f73d9800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f73d40b6490 0x7f73d406be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.565+0000 7f73d3e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f73d406b9f0 0x7f73d40b5d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51918/0 (socket says 172.21.15.42:51918) 2024-05-28T13:59:33.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.565+0000 7f73d3e00640 1 -- 172.21.15.42:0/2446407752 learned_addr learned my addr 172.21.15.42:0/2446407752 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:33.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.565+0000 7f73d9800640 1 -- 172.21.15.42:0/2446407752 >> v1:172.21.15.63:6789/0 conn(0x7f73d415eb00 legacy=0x7f73d40cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.565+0000 7f73d9800640 1 -- 172.21.15.42:0/2446407752 >> v1:172.21.15.42:6789/0 conn(0x7f73d406b9f0 legacy=0x7f73d40b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.565+0000 7f73d9800640 1 -- 172.21.15.42:0/2446407752 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d40b5580 con 0x7f73d40b6490 2024-05-28T13:59:33.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.565+0000 7f73d3400640 1 -- 172.21.15.42:0/2446407752 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 224827342 0 0) 0x7f73c0001120 con 0x7f73d406b9f0 2024-05-28T13:59:33.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73d9800640 1 --2- 172.21.15.42:0/2446407752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d40b6490 0x7f73d406be80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009d90 tx=0x7f73cc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56475e00f5bd0e63 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:33.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73d3400640 1 -- 172.21.15.42:0/2446407752 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc0b2070 con 0x7f73d40b6490 2024-05-28T13:59:33.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73d3400640 1 -- 172.21.15.42:0/2446407752 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73cc0a5830 con 0x7f73d40b6490 2024-05-28T13:59:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73d3400640 1 -- 172.21.15.42:0/2446407752 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc0a5c00 con 0x7f73d40b6490 2024-05-28T13:59:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 -- 172.21.15.42:0/2446407752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d40b6490 msgr2=0x7f73d406be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 --2- 172.21.15.42:0/2446407752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d40b6490 0x7f73d406be80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009d90 tx=0x7f73cc01c970 comp rx=0 tx=0).stop 2024-05-28T13:59:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 -- 172.21.15.42:0/2446407752 shutdown_connections 2024-05-28T13:59:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 --2- 172.21.15.42:0/2446407752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d40b6490 0x7f73d406be80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 -- 172.21.15.42:0/2446407752 >> 172.21.15.42:0/2446407752 conn(0x7f73d40b68f0 msgr2=0x7f73d40b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:33.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 -- 172.21.15.42:0/2446407752 shutdown_connections 2024-05-28T13:59:33.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 -- 172.21.15.42:0/2446407752 wait complete. 2024-05-28T13:59:33.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 Processor -- start 2024-05-28T13:59:33.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 -- start start 2024-05-28T13:59:33.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73d406b9f0 0x7f73d413e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d40b6490 0x7f73d4160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73d415eb00 0x7f73d415d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f73d416c620 con 0x7f73d40b6490 2024-05-28T13:59:33.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f73d40ce8d0 con 0x7f73d406b9f0 2024-05-28T13:59:33.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73db800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f73d416c7a0 con 0x7f73d415eb00 2024-05-28T13:59:33.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73d3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d40b6490 0x7f73d4160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73d3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d40b6490 0x7f73d4160d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41822/0 (socket says 172.21.15.42:41822) 2024-05-28T13:59:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73d3e00640 1 -- 172.21.15.42:0/341828507 learned_addr learned my addr 172.21.15.42:0/341828507 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73d9800640 1 --2- 172.21.15.42:0/341828507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73d415eb00 0x7f73d415d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.569+0000 7f73d8e00640 1 --2- 172.21.15.42:0/341828507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73d406b9f0 0x7f73d413e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.573+0000 7f73d3e00640 1 -- 172.21.15.42:0/341828507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73d415eb00 msgr2=0x7f73d415d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.573+0000 7f73d3e00640 1 --2- 172.21.15.42:0/341828507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73d415eb00 0x7f73d415d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.573+0000 7f73d3e00640 1 -- 172.21.15.42:0/341828507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73d406b9f0 msgr2=0x7f73d413e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.573+0000 7f73d3e00640 1 --2- 172.21.15.42:0/341828507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73d406b9f0 0x7f73d413e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.573+0000 7f73d3e00640 1 -- 172.21.15.42:0/341828507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d415fcd0 con 0x7f73d40b6490 2024-05-28T13:59:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.573+0000 7f73d3e00640 1 --2- 172.21.15.42:0/341828507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d40b6490 0x7f73d4160d70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f73c000c7b0 tx=0x7f73c000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.573+0000 7f73d0c00640 1 -- 172.21.15.42:0/341828507 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0018070 con 0x7f73d40b6490 2024-05-28T13:59:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.573+0000 7f73d0c00640 1 -- 172.21.15.42:0/341828507 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c00072d0 con 0x7f73d40b6490 2024-05-28T13:59:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.573+0000 7f73d0c00640 1 -- 172.21.15.42:0/341828507 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c00047f0 con 0x7f73d40b6490 2024-05-28T13:59:33.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.573+0000 7f73db800640 1 -- 172.21.15.42:0/341828507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d415ffb0 con 0x7f73d40b6490 2024-05-28T13:59:33.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.573+0000 7f73db800640 1 -- 172.21.15.42:0/341828507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d4160570 con 0x7f73d40b6490 2024-05-28T13:59:33.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.573+0000 7f73db800640 1 -- 172.21.15.42:0/341828507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d4068a90 con 0x7f73d40b6490 2024-05-28T13:59:33.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.573+0000 7f73d0c00640 1 -- 172.21.15.42:0/341828507 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f73c001b070 con 0x7f73d40b6490 2024-05-28T13:59:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.573+0000 7f73d0c00640 1 --2- 172.21.15.42:0/341828507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73b40683d0 0x7f73b406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.577+0000 7f73d0c00640 1 -- 172.21.15.42:0/341828507 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f73c0085cd0 con 0x7f73d40b6490 2024-05-28T13:59:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.577+0000 7f73d8e00640 1 --2- 172.21.15.42:0/341828507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73b40683d0 0x7f73b406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.577+0000 7f73d8e00640 1 --2- 172.21.15.42:0/341828507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73b40683d0 0x7f73b406a890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f73c8007fd0 tx=0x7f73c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:33.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.577+0000 7f73d0c00640 1 -- 172.21.15.42:0/341828507 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f73d415eee0 con 0x7f73d40b6490 2024-05-28T13:59:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.777+0000 7f73db800640 1 -- 172.21.15.42:0/341828507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f73d4074be0 con 0x7f73d40b6490 2024-05-28T13:59:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.781+0000 7f73d0c00640 1 -- 172.21.15.42:0/341828507 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f73c00076c0 con 0x7f73d40b6490 2024-05-28T13:59:33.765 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-28T13:59:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.781+0000 7f73db800640 1 -- 172.21.15.42:0/341828507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73b40683d0 msgr2=0x7f73b406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.781+0000 7f73db800640 1 --2- 172.21.15.42:0/341828507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73b40683d0 0x7f73b406a890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f73c8007fd0 tx=0x7f73c8009040 comp rx=0 tx=0).stop 2024-05-28T13:59:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.781+0000 7f73db800640 1 -- 172.21.15.42:0/341828507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d40b6490 msgr2=0x7f73d4160d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.781+0000 7f73db800640 1 --2- 172.21.15.42:0/341828507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d40b6490 0x7f73d4160d70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f73c000c7b0 tx=0x7f73c000cc70 comp rx=0 tx=0).stop 2024-05-28T13:59:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.781+0000 7f73db800640 1 -- 172.21.15.42:0/341828507 shutdown_connections 2024-05-28T13:59:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.781+0000 7f73db800640 1 --2- 172.21.15.42:0/341828507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73b40683d0 0x7f73b406a890 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.781+0000 7f73db800640 1 --2- 172.21.15.42:0/341828507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73d415eb00 0x7f73d415d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.781+0000 7f73db800640 1 --2- 172.21.15.42:0/341828507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d40b6490 0x7f73d4160d70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.781+0000 7f73db800640 1 --2- 172.21.15.42:0/341828507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73d406b9f0 0x7f73d413e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.781+0000 7f73db800640 1 -- 172.21.15.42:0/341828507 >> 172.21.15.42:0/341828507 conn(0x7f73d40b68f0 msgr2=0x7f73d40b5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.781+0000 7f73db800640 1 -- 172.21.15.42:0/341828507 shutdown_connections 2024-05-28T13:59:33.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:33.781+0000 7f73db800640 1 -- 172.21.15.42:0/341828507 wait complete. 2024-05-28T13:59:33.787 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-28T13:59:33.787 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-28T13:59:33.787 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-05-28T13:59:33.787 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-28T13:59:33.787 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-28T13:59:33.787 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-28T13:59:33.787 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-28T13:59:33.787 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-28T13:59:33.788 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T13:59:33.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:33.788 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-05-28T13:59:33.798 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T13:59:33.798 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T13:59:33.798 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T13:59:33.798 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:59:33.798 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-05-28T13:59:33.798 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:59:33.798 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:59:33.798 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-05-28T13:59:33.798 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-05-28T13:59:33.798 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:59:33.798 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-05-28T13:59:33.799 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-05-28T13:59:33.799 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-28T13:59:33.799 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:59:33.799 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-05-28T13:59:33.799 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-28T13:59:33.799 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42373sec preferred_lft 42373sec 2024-05-28T13:59:33.799 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2024-05-28T13:59:33.799 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:59:33.799 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T13:59:33.799 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:59:33.799 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-05-28T13:59:33.799 INFO:teuthology.orchestra.run.smithi042.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-28T13:59:33.799 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 02:42:65:2b:63:73 brd ff:ff:ff:ff:ff:ff 2024-05-28T13:59:33.800 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-28T13:59:33.800 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:59:33.800 INFO:teuthology.orchestra.run.smithi042.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-28T13:59:33.800 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-28T13:59:33.800 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-28T13:59:33.800 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:59:33.800 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-28T13:59:33.800 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:59:33.800 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T13:59:33.800 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:59:33.800 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-28T13:59:33.800 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:59:33.800 INFO:teuthology.orchestra.run.smithi042.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T13:59:33.800 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T13:59:33.801 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-28T13:59:33.801 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T13:59:33.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:33.801 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-28T13:59:33.862 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-05-28T13:59:33.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:33.864 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-05-28T13:59:33.921 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-05-28T13:59:33.921 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-28T13:59:33.921 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-05-28T13:59:33.922 INFO:teuthology.orchestra.run.smithi042.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-28T13:59:33.922 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-05-28T13:59:33.922 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-28T13:59:33.922 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T13:59:33.922 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T13:59:33.922 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-28T13:59:33.922 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-28T13:59:33.922 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-28T13:59:33.922 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T13:59:34.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T13:59:34.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:59:34.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:34.060 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-28T13:59:34.070 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-28T13:59:34.070 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-28T13:59:34.070 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T13:59:34.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:34.070 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-05-28T13:59:34.132 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-28T13:59:34.132 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T13:59:34.132 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-28T13:59:34.184 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-28T13:59:34.184 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T13:59:34.184 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-28T13:59:34.235 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T13:59:34.296 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T13:59:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T13:59:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T13:59:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T13:59:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T13:59:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T13:59:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T13:59:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T13:59:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T13:59:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T13:59:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T13:59:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T13:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T13:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T13:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T13:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T13:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T13:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T13:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T13:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T13:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T13:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T13:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T13:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T13:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T13:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T13:59:34.356 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T13:59:34.356 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T13:59:34.356 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T13:59:34.356 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T13:59:34.356 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:59:34.356 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:59:34.356 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T13:59:34.356 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T13:59:34.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:34.357 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T13:59:34.408 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-28T13:59:34.408 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-28T13:59:34.459 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T13:59:34.479 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T13:59:34.534 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T13:59:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T13:59:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T13:59:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T13:59:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T13:59:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T13:59:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T13:59:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T13:59:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T13:59:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T13:59:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T13:59:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T13:59:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T13:59:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T13:59:34.536 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T13:59:34.536 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T13:59:34.536 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T13:59:34.536 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T13:59:34.536 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T13:59:34.536 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T13:59:34.536 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T13:59:34.536 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T13:59:34.536 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T13:59:34.536 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T13:59:34.536 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T13:59:34.536 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T13:59:34.536 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T13:59:34.537 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T13:59:34.537 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T13:59:34.537 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:59:34.537 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T13:59:34.537 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T13:59:34.537 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T13:59:34.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:34.538 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T13:59:34.558 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-05-28T13:59:34.573+0000 7f169140f580 -1 init, newargv = 0x556a7dca9e70 newargc=15 2024-05-28T13:59:34.558 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[36920]: starting ceph client 2024-05-28T13:59:34.565 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[36920]: starting fuse 2024-05-28T13:59:34.567 INFO:teuthology.orchestra.run.smithi042.stdout:44 2024-05-28T13:59:34.567 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-28T13:59:34.567 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-05-28T13:59:34.567 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-05-28T13:59:34.567 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-05-28T13:59:34.567 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-05-28T13:59:34.567 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-05-28T13:59:34.567 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T13:59:34.567 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-05-28T13:59:34.568 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-28T13:59:34.568 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-05-28T13:59:34.568 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-28T13:59:34.568 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T13:59:34.568 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-05-28T13:59:34.568 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-05-28T13:59:34.568 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-05-28T13:59:34.568 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-05-28T13:59:34.568 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T13:59:34.568 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-05-28T13:59:34.568 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-28T13:59:34.568 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-28T13:59:34.568 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-28T13:59:34.568 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-05-28T13:59:34.569 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-05-28T13:59:34.569 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-05-28T13:59:34.569 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-28T13:59:34.569 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T13:59:34.569 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-05-28T13:59:34.569 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-05-28T13:59:34.696 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.36920.asok 2024-05-28T13:59:34.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T13:59:34.708 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36920.asok 2024-05-28T13:59:34.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:34.709 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36920.asok status 2024-05-28T13:59:34.888 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T13:59:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-05-28T13:59:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-28T13:59:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-28T13:59:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-05-28T13:59:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-05-28T13:59:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-28T13:59:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "36920", 2024-05-28T13:59:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-05-28T13:59:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-05-28T13:59:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-05-28T13:59:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5424, 2024-05-28T13:59:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-05-28T13:59:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-05-28T13:59:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-05-28T13:59:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 5424 2024-05-28T13:59:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T13:59:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T13:59:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T13:59:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1504728598 2024-05-28T13:59:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T13:59:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T13:59:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T13:59:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1504728598 2024-05-28T13:59:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T13:59:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.5424 192.168.0.1:0/1504728598", 2024-05-28T13:59:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/1504728598", 2024-05-28T13:59:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-05-28T13:59:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 17, 2024-05-28T13:59:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 34, 2024-05-28T13:59:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-05-28T13:59:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-05-28T13:59:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-05-28T13:59:34.891 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T13:59:34.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:34.904 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-05-28T13:59:34.904 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-28T13:59:34.914 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-05-28T13:59:34.914 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-28T13:59:34.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:34.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-28T13:59:34.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-28T13:59:35.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.181+0000 7fa023800640 1 Processor -- start 2024-05-28T13:59:35.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.181+0000 7fa023800640 1 -- start start 2024-05-28T13:59:35.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.181+0000 7fa023800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa01c0d0e00 0x7fa01c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:35.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.181+0000 7fa023800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c072d40 0x7fa01c0aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:35.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.181+0000 7fa023800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa01c0ade40 con 0x7fa01c15b350 2024-05-28T13:59:35.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.185+0000 7fa023800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa01c0ae9f0 con 0x7fa01c0d0e00 2024-05-28T13:59:35.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.185+0000 7fa023800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa01c0ae0c0 con 0x7fa01c072d40 2024-05-28T13:59:35.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.185+0000 7fa020e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa01c15b350 0x7fa01c0d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51924/0 (socket says 172.21.15.42:51924) 2024-05-28T13:59:35.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.185+0000 7fa020e00640 1 -- 172.21.15.42:0/563169944 learned_addr learned my addr 172.21.15.42:0/563169944 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:35.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.185+0000 7fa013e00640 1 --2- 172.21.15.42:0/563169944 >> v2:172.21.15.63:3300/0 conn(0x7fa01c0d0e00 0x7fa01c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:35.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.185+0000 7fa021800640 1 --2- 172.21.15.42:0/563169944 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c072d40 0x7fa01c0aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:35.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.185+0000 7fa01be00640 1 -- 172.21.15.42:0/563169944 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 569435333 0 0) 0x7fa00c001120 con 0x7fa01c15b350 2024-05-28T13:59:35.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.185+0000 7fa01be00640 1 -- 172.21.15.42:0/563169944 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa000003530 con 0x7fa01c15b350 2024-05-28T13:59:35.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.185+0000 7fa013e00640 1 -- 172.21.15.42:0/563169944 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c072d40 msgr2=0x7fa01c0aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:35.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.185+0000 7fa013e00640 1 --2- 172.21.15.42:0/563169944 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c072d40 0x7fa01c0aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.185+0000 7fa013e00640 1 -- 172.21.15.42:0/563169944 >> v1:172.21.15.42:6789/0 conn(0x7fa01c15b350 legacy=0x7fa01c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:35.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.185+0000 7fa013e00640 1 -- 172.21.15.42:0/563169944 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01c0ae3a0 con 0x7fa01c0d0e00 2024-05-28T13:59:35.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.185+0000 7fa021800640 1 --2- 172.21.15.42:0/563169944 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c072d40 0x7fa01c0aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:59:35.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.185+0000 7fa013e00640 1 --2- 172.21.15.42:0/563169944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c0d0e00 0x7fa01c06dd40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa0040097c0 tx=0x7fa00401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ea52c9061c8b2d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:35.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa01be00640 1 -- 172.21.15.42:0/563169944 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0040b0070 con 0x7fa01c0d0e00 2024-05-28T13:59:35.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa01be00640 1 -- 172.21.15.42:0/563169944 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0040ab600 con 0x7fa01c0d0e00 2024-05-28T13:59:35.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa01be00640 1 -- 172.21.15.42:0/563169944 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0040ab9d0 con 0x7fa01c0d0e00 2024-05-28T13:59:35.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 -- 172.21.15.42:0/563169944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c0d0e00 msgr2=0x7fa01c06dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:35.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 --2- 172.21.15.42:0/563169944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c0d0e00 0x7fa01c06dd40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa0040097c0 tx=0x7fa00401c410 comp rx=0 tx=0).stop 2024-05-28T13:59:35.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 -- 172.21.15.42:0/563169944 shutdown_connections 2024-05-28T13:59:35.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 --2- 172.21.15.42:0/563169944 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c072d40 0x7fa01c0aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 --2- 172.21.15.42:0/563169944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c0d0e00 0x7fa01c06dd40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 -- 172.21.15.42:0/563169944 >> 172.21.15.42:0/563169944 conn(0x7fa01c06bdf0 msgr2=0x7fa01c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:35.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 -- 172.21.15.42:0/563169944 shutdown_connections 2024-05-28T13:59:35.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 -- 172.21.15.42:0/563169944 wait complete. 2024-05-28T13:59:35.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 Processor -- start 2024-05-28T13:59:35.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 -- start start 2024-05-28T13:59:35.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c072d40 0x7fa01c1f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:35.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c0d0e00 0x7fa01c09b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:35.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa01c15b350 0x7fa01c1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:35.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c0b3460 con 0x7fa01c15b350 2024-05-28T13:59:35.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c0b32e0 con 0x7fa01c0d0e00 2024-05-28T13:59:35.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa023800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa01c0b35e0 con 0x7fa01c072d40 2024-05-28T13:59:35.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa021800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa01c15b350 0x7fa01c1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:35.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa021800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa01c15b350 0x7fa01c1510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41836/0 (socket says 172.21.15.42:41836) 2024-05-28T13:59:35.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.189+0000 7fa021800640 1 -- 172.21.15.42:0/2280289678 learned_addr learned my addr 172.21.15.42:0/2280289678 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:35.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa020e00640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c072d40 0x7fa01c1f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:35.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa013e00640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c0d0e00 0x7fa01c09b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:35.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa021800640 1 -- 172.21.15.42:0/2280289678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c072d40 msgr2=0x7fa01c1f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:35.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa021800640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c072d40 0x7fa01c1f79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa021800640 1 -- 172.21.15.42:0/2280289678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c0d0e00 msgr2=0x7fa01c09b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:35.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa021800640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c0d0e00 0x7fa01c09b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa021800640 1 -- 172.21.15.42:0/2280289678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01c09a370 con 0x7fa01c15b350 2024-05-28T13:59:35.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa013e00640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c0d0e00 0x7fa01c09b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:35.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa020e00640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c072d40 0x7fa01c1f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:35.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa021800640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa01c15b350 0x7fa01c1510e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa01400efb0 tx=0x7fa01400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:35.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa019600640 1 -- 172.21.15.42:0/2280289678 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014018020 con 0x7fa01c15b350 2024-05-28T13:59:35.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa019600640 1 -- 172.21.15.42:0/2280289678 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa014014480 con 0x7fa01c15b350 2024-05-28T13:59:35.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa023800640 1 -- 172.21.15.42:0/2280289678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01c1f82c0 con 0x7fa01c15b350 2024-05-28T13:59:35.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa019600640 1 -- 172.21.15.42:0/2280289678 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014014850 con 0x7fa01c15b350 2024-05-28T13:59:35.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa023800640 1 -- 172.21.15.42:0/2280289678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa01c09bc80 con 0x7fa01c15b350 2024-05-28T13:59:35.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa023800640 1 -- 172.21.15.42:0/2280289678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa01c1f8040 con 0x7fa01c15b350 2024-05-28T13:59:35.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa019600640 1 -- 172.21.15.42:0/2280289678 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa01401b020 con 0x7fa01c15b350 2024-05-28T13:59:35.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa019600640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9ff8068380 0x7f9ff806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:35.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa019600640 1 -- 172.21.15.42:0/2280289678 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa014087720 con 0x7fa01c15b350 2024-05-28T13:59:35.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.193+0000 7fa020e00640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9ff8068380 0x7f9ff806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:35.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.197+0000 7fa020e00640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9ff8068380 0x7f9ff806a840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa00c005890 tx=0x7fa00c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:35.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.197+0000 7fa019600640 1 -- 172.21.15.42:0/2280289678 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa01c1f8040 con 0x7fa01c15b350 2024-05-28T13:59:35.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.397+0000 7fa023800640 1 -- 172.21.15.42:0/2280289678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fa01c068a90 con 0x7fa01c15b350 2024-05-28T13:59:35.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.501+0000 7fa019600640 1 -- 172.21.15.42:0/2280289678 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) ==== 136+0+0 (secure 0 0 0) 0x7fa014022030 con 0x7fa01c15b350 2024-05-28T13:59:35.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.505+0000 7fa023800640 1 -- 172.21.15.42:0/2280289678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9ff8068380 msgr2=0x7f9ff806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:35.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.505+0000 7fa023800640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9ff8068380 0x7f9ff806a840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa00c005890 tx=0x7fa00c009040 comp rx=0 tx=0).stop 2024-05-28T13:59:35.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.505+0000 7fa023800640 1 -- 172.21.15.42:0/2280289678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa01c15b350 msgr2=0x7fa01c1510e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:35.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.505+0000 7fa023800640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa01c15b350 0x7fa01c1510e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa01400efb0 tx=0x7fa01400c490 comp rx=0 tx=0).stop 2024-05-28T13:59:35.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.505+0000 7fa023800640 1 -- 172.21.15.42:0/2280289678 shutdown_connections 2024-05-28T13:59:35.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.505+0000 7fa023800640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9ff8068380 0x7f9ff806a840 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.505+0000 7fa023800640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa01c15b350 0x7fa01c1510e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.505+0000 7fa023800640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c0d0e00 0x7fa01c09b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.505+0000 7fa023800640 1 --2- 172.21.15.42:0/2280289678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c072d40 0x7fa01c1f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.505+0000 7fa023800640 1 -- 172.21.15.42:0/2280289678 >> 172.21.15.42:0/2280289678 conn(0x7fa01c06bdf0 msgr2=0x7fa01c1535b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:35.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.505+0000 7fa023800640 1 -- 172.21.15.42:0/2280289678 shutdown_connections 2024-05-28T13:59:35.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.505+0000 7fa023800640 1 -- 172.21.15.42:0/2280289678 wait complete. 2024-05-28T13:59:35.509 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T13:59:35.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.665+0000 7fdcbd800640 1 Processor -- start 2024-05-28T13:59:35.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.665+0000 7fdcbd800640 1 -- start start 2024-05-28T13:59:35.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.665+0000 7fdcbd800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcb806f030 0x7fdcb806db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:35.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.669+0000 7fdcbd800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcb806f480 con 0x7fdcb8145920 2024-05-28T13:59:35.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.669+0000 7fdcbd800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcb806e240 con 0x7fdcb80724d0 2024-05-28T13:59:35.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.669+0000 7fdcbd800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdcb806e520 con 0x7fdcb806f030 2024-05-28T13:59:35.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.669+0000 7fdcb6a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fdcb8145920 0x7fdcb8071dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51934/0 (socket says 172.21.15.42:51934) 2024-05-28T13:59:35.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.669+0000 7fdcb6a00640 1 -- 172.21.15.42:0/2341167781 learned_addr learned my addr 172.21.15.42:0/2341167781 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T13:59:35.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.669+0000 7fdcb7400640 1 --2- 172.21.15.42:0/2341167781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcb806f030 0x7fdcb806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:35.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.669+0000 7fdcb6000640 1 -- 172.21.15.42:0/2341167781 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3859988017 0 0) 0x7fdcb806f480 con 0x7fdcb8145920 2024-05-28T13:59:35.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.669+0000 7fdcb6000640 1 -- 172.21.15.42:0/2341167781 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc9c003530 con 0x7fdcb8145920 2024-05-28T13:59:35.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.669+0000 7fdcb6000640 1 -- 172.21.15.42:0/2341167781 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2416062806 0 0) 0x7fdcb806e240 con 0x7fdcb80724d0 2024-05-28T13:59:35.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.669+0000 7fdcb6000640 1 -- 172.21.15.42:0/2341167781 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcb806f480 con 0x7fdcb80724d0 2024-05-28T13:59:35.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.669+0000 7fdcb7400640 1 -- 172.21.15.42:0/2341167781 >> v1:172.21.15.63:6789/0 conn(0x7fdcb80724d0 legacy=0x7fdcb815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:35.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.669+0000 7fdcb7400640 1 -- 172.21.15.42:0/2341167781 >> v1:172.21.15.42:6789/0 conn(0x7fdcb8145920 legacy=0x7fdcb8071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:35.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.669+0000 7fdcb6000640 1 -- 172.21.15.42:0/2341167781 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3786022183 0 0) 0x7fdc9c003530 con 0x7fdcb8145920 2024-05-28T13:59:35.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.669+0000 7fdcb7400640 1 -- 172.21.15.42:0/2341167781 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb806e800 con 0x7fdcb806f030 2024-05-28T13:59:35.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcb7400640 1 --2- 172.21.15.42:0/2341167781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcb806f030 0x7fdcb806db40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009dc0 tx=0x7fdcb001c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d0ada67f2e6b981 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcb6000640 1 -- 172.21.15.42:0/2341167781 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00b2070 con 0x7fdcb806f030 2024-05-28T13:59:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcb6000640 1 -- 172.21.15.42:0/2341167781 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb00a5570 con 0x7fdcb806f030 2024-05-28T13:59:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcb6000640 1 -- 172.21.15.42:0/2341167781 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8001680 con 0x7fdcb806f030 2024-05-28T13:59:35.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 -- 172.21.15.42:0/2341167781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcb806f030 msgr2=0x7fdcb806db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 --2- 172.21.15.42:0/2341167781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcb806f030 0x7fdcb806db40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009dc0 tx=0x7fdcb001c820 comp rx=0 tx=0).stop 2024-05-28T13:59:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 -- 172.21.15.42:0/2341167781 shutdown_connections 2024-05-28T13:59:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 --2- 172.21.15.42:0/2341167781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcb806f030 0x7fdcb806db40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 -- 172.21.15.42:0/2341167781 >> 172.21.15.42:0/2341167781 conn(0x7fdcb806b9f0 msgr2=0x7fdcb806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 -- 172.21.15.42:0/2341167781 shutdown_connections 2024-05-28T13:59:35.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 -- 172.21.15.42:0/2341167781 wait complete. 2024-05-28T13:59:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 Processor -- start 2024-05-28T13:59:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 -- start start 2024-05-28T13:59:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcb806f030 0x7fdcb81f8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcb80724d0 0x7fdcb81435e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcb8145920 0x7fdcb8143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb815e3d0 con 0x7fdcb8145920 2024-05-28T13:59:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb815e250 con 0x7fdcb806f030 2024-05-28T13:59:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcbd800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdcb816a260 con 0x7fdcb80724d0 2024-05-28T13:59:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcb7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcb8145920 0x7fdcb8143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcb7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcb8145920 0x7fdcb8143b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41840/0 (socket says 172.21.15.42:41840) 2024-05-28T13:59:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.673+0000 7fdcb7400640 1 -- 172.21.15.42:0/302915804 learned_addr learned my addr 172.21.15.42:0/302915804 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcb6a00640 1 --2- 172.21.15.42:0/302915804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcb806f030 0x7fdcb81f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcafe00640 1 --2- 172.21.15.42:0/302915804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcb80724d0 0x7fdcb81435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcb7400640 1 -- 172.21.15.42:0/302915804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcb80724d0 msgr2=0x7fdcb81435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcb7400640 1 --2- 172.21.15.42:0/302915804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcb80724d0 0x7fdcb81435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcb7400640 1 -- 172.21.15.42:0/302915804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcb806f030 msgr2=0x7fdcb81f8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:35.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcb7400640 1 --2- 172.21.15.42:0/302915804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcb806f030 0x7fdcb81f8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcb7400640 1 -- 172.21.15.42:0/302915804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb80ab850 con 0x7fdcb8145920 2024-05-28T13:59:35.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcb7400640 1 --2- 172.21.15.42:0/302915804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcb8145920 0x7fdcb8143b20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00b0040 tx=0x7fdcb00195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:35.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcaea00640 1 -- 172.21.15.42:0/302915804 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00c0070 con 0x7fdcb8145920 2024-05-28T13:59:35.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcaea00640 1 -- 172.21.15.42:0/302915804 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb00039a0 con 0x7fdcb8145920 2024-05-28T13:59:35.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcaea00640 1 -- 172.21.15.42:0/302915804 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00b2040 con 0x7fdcb8145920 2024-05-28T13:59:35.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcbd800640 1 -- 172.21.15.42:0/302915804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb80abb30 con 0x7fdcb8145920 2024-05-28T13:59:35.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcbd800640 1 -- 172.21.15.42:0/302915804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb80ac0f0 con 0x7fdcb8145920 2024-05-28T13:59:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcbd800640 1 -- 172.21.15.42:0/302915804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb8073890 con 0x7fdcb8145920 2024-05-28T13:59:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcaea00640 1 -- 172.21.15.42:0/302915804 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdcb0004030 con 0x7fdcb8145920 2024-05-28T13:59:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.677+0000 7fdcaea00640 1 --2- 172.21.15.42:0/302915804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc94068380 0x7fdc9406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.681+0000 7fdcaea00640 1 -- 172.21.15.42:0/302915804 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fdcb0120900 con 0x7fdcb8145920 2024-05-28T13:59:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.681+0000 7fdcb6a00640 1 --2- 172.21.15.42:0/302915804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc94068380 0x7fdc9406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.681+0000 7fdcb6a00640 1 --2- 172.21.15.42:0/302915804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc94068380 0x7fdc9406a840 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdca8000fe0 tx=0x7fdca8001160 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:35.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.685+0000 7fdcaea00640 1 -- 172.21.15.42:0/302915804 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdcb00164a0 con 0x7fdcb8145920 2024-05-28T13:59:35.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.873+0000 7fdcbd800640 1 -- 172.21.15.42:0/302915804 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fdcb81f8ea0 con 0x7fdc94068380 2024-05-28T13:59:35.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.873+0000 7fdcaea00640 1 -- 172.21.15.42:0/302915804 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fdcb81f8ea0 con 0x7fdc94068380 2024-05-28T13:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T13:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T13:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T13:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T13:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T13:59:35.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.877+0000 7fdcbd800640 1 -- 172.21.15.42:0/302915804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc94068380 msgr2=0x7fdc9406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:35.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.877+0000 7fdcbd800640 1 --2- 172.21.15.42:0/302915804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc94068380 0x7fdc9406a840 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdca8000fe0 tx=0x7fdca8001160 comp rx=0 tx=0).stop 2024-05-28T13:59:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.877+0000 7fdcbd800640 1 -- 172.21.15.42:0/302915804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcb8145920 msgr2=0x7fdcb8143b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.877+0000 7fdcbd800640 1 --2- 172.21.15.42:0/302915804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcb8145920 0x7fdcb8143b20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00b0040 tx=0x7fdcb00195c0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.877+0000 7fdcbd800640 1 -- 172.21.15.42:0/302915804 shutdown_connections 2024-05-28T13:59:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.877+0000 7fdcbd800640 1 --2- 172.21.15.42:0/302915804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc94068380 0x7fdc9406a840 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.877+0000 7fdcbd800640 1 --2- 172.21.15.42:0/302915804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcb8145920 0x7fdcb8143b20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.877+0000 7fdcbd800640 1 --2- 172.21.15.42:0/302915804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcb80724d0 0x7fdcb81435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.877+0000 7fdcbd800640 1 --2- 172.21.15.42:0/302915804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcb806f030 0x7fdcb81f8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.877+0000 7fdcbd800640 1 -- 172.21.15.42:0/302915804 >> 172.21.15.42:0/302915804 conn(0x7fdcb806b9f0 msgr2=0x7fdcb80cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.877+0000 7fdcbd800640 1 -- 172.21.15.42:0/302915804 shutdown_connections 2024-05-28T13:59:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:35.877+0000 7fdcbd800640 1 -- 172.21.15.42:0/302915804 wait complete. 2024-05-28T13:59:35.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T13:59:36.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7dfe00640 1 Processor -- start 2024-05-28T13:59:36.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7dfe00640 1 -- start start 2024-05-28T13:59:36.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7dfe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc7e00cf540 0x7fc7e00cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e00707e0 0x7fc7e00cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7dfe00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e0160170 con 0x7fc7e0145d50 2024-05-28T13:59:36.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7dfe00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e0160450 con 0x7fc7e00cf540 2024-05-28T13:59:36.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7dfe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e0160730 con 0x7fc7e00707e0 2024-05-28T13:59:36.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7dea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc7e00cf540 0x7fc7e00cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7dea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc7e00cf540 0x7fc7e00cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41856/0 (socket says 172.21.15.42:41856) 2024-05-28T13:59:36.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7dea00640 1 -- 172.21.15.42:0/1846052917 learned_addr learned my addr 172.21.15.42:0/1846052917 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:36.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7df400640 1 --2- 172.21.15.42:0/1846052917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e00707e0 0x7fc7e00cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7dea00640 1 -- 172.21.15.42:0/1846052917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e00707e0 msgr2=0x7fc7e00cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:36.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7dea00640 1 --2- 172.21.15.42:0/1846052917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e00707e0 0x7fc7e00cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7dea00640 1 -- 172.21.15.42:0/1846052917 >> v1:172.21.15.63:6789/0 conn(0x7fc7e0145d50 legacy=0x7fc7e0071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.041+0000 7fc7dea00640 1 -- 172.21.15.42:0/1846052917 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e00d0ed0 con 0x7fc7e00cf540 2024-05-28T13:59:36.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.045+0000 7fc7dea00640 1 --2- 172.21.15.42:0/1846052917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7e00cf540 0x7fc7e00cc940 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0009870 tx=0x7fc7d001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bab6e5119264fd server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:36.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.045+0000 7fc7de000640 1 -- 172.21.15.42:0/1846052917 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d00b0070 con 0x7fc7e00cf540 2024-05-28T13:59:36.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.045+0000 7fc7de000640 1 -- 172.21.15.42:0/1846052917 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d00ab600 con 0x7fc7e00cf540 2024-05-28T13:59:36.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.045+0000 7fc7de000640 1 -- 172.21.15.42:0/1846052917 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d00ab9d0 con 0x7fc7e00cf540 2024-05-28T13:59:36.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.045+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1846052917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7e00cf540 msgr2=0x7fc7e00cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.045+0000 7fc7dfe00640 1 --2- 172.21.15.42:0/1846052917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7e00cf540 0x7fc7e00cc940 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0009870 tx=0x7fc7d001c4d0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.045+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1846052917 shutdown_connections 2024-05-28T13:59:36.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.045+0000 7fc7dfe00640 1 --2- 172.21.15.42:0/1846052917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e00707e0 0x7fc7e00cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.045+0000 7fc7dfe00640 1 --2- 172.21.15.42:0/1846052917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7e00cf540 0x7fc7e00cc940 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.045+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1846052917 >> 172.21.15.42:0/1846052917 conn(0x7fc7e006fcc0 msgr2=0x7fc7e00700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:36.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.045+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1846052917 shutdown_connections 2024-05-28T13:59:36.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.045+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1846052917 wait complete. 2024-05-28T13:59:36.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7dfe00640 1 Processor -- start 2024-05-28T13:59:36.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7dfe00640 1 -- start start 2024-05-28T13:59:36.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e00707e0 0x7fc7e0154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e00cf540 0x7fc7e015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7e0145d50 0x7fc7e015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7dfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0169350 con 0x7fc7e0145d50 2024-05-28T13:59:36.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7dfe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e00d1100 con 0x7fc7e00cf540 2024-05-28T13:59:36.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7dfe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e01694d0 con 0x7fc7e00707e0 2024-05-28T13:59:36.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7df400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7e0145d50 0x7fc7e015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7df400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7e0145d50 0x7fc7e015ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41864/0 (socket says 172.21.15.42:41864) 2024-05-28T13:59:36.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7df400640 1 -- 172.21.15.42:0/1728264993 learned_addr learned my addr 172.21.15.42:0/1728264993 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:36.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7dea00640 1 --2- 172.21.15.42:0/1728264993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e00707e0 0x7fc7e0154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7d7e00640 1 --2- 172.21.15.42:0/1728264993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e00cf540 0x7fc7e015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7df400640 1 -- 172.21.15.42:0/1728264993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e00707e0 msgr2=0x7fc7e0154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7df400640 1 --2- 172.21.15.42:0/1728264993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e00707e0 0x7fc7e0154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7df400640 1 -- 172.21.15.42:0/1728264993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e00cf540 msgr2=0x7fc7e015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7df400640 1 --2- 172.21.15.42:0/1728264993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e00cf540 0x7fc7e015c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7df400640 1 -- 172.21.15.42:0/1728264993 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e0158530 con 0x7fc7e0145d50 2024-05-28T13:59:36.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7df400640 1 --2- 172.21.15.42:0/1728264993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7e0145d50 0x7fc7e015ccf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8007de0 tx=0x7fc7d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:36.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7d6a00640 1 -- 172.21.15.42:0/1728264993 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8014620 con 0x7fc7e0145d50 2024-05-28T13:59:36.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7d6a00640 1 -- 172.21.15.42:0/1728264993 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d8019070 con 0x7fc7e0145d50 2024-05-28T13:59:36.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7d6a00640 1 -- 172.21.15.42:0/1728264993 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8014a20 con 0x7fc7e0145d50 2024-05-28T13:59:36.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.049+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1728264993 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e0158810 con 0x7fc7e0145d50 2024-05-28T13:59:36.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.053+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1728264993 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e0158dd0 con 0x7fc7e0145d50 2024-05-28T13:59:36.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.053+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1728264993 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e004fcf0 con 0x7fc7e0145d50 2024-05-28T13:59:36.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.053+0000 7fc7d6a00640 1 -- 172.21.15.42:0/1728264993 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc7d8014bc0 con 0x7fc7e0145d50 2024-05-28T13:59:36.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.053+0000 7fc7d6a00640 1 --2- 172.21.15.42:0/1728264993 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7bc0683d0 0x7fc7bc06a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.053+0000 7fc7d6a00640 1 -- 172.21.15.42:0/1728264993 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc7d8087160 con 0x7fc7e0145d50 2024-05-28T13:59:36.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.053+0000 7fc7dea00640 1 --2- 172.21.15.42:0/1728264993 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7bc0683d0 0x7fc7bc06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.053+0000 7fc7dea00640 1 --2- 172.21.15.42:0/1728264993 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7bc0683d0 0x7fc7bc06a890 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00ad000 tx=0x7fc7d0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:36.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.057+0000 7fc7d6a00640 1 -- 172.21.15.42:0/1728264993 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc7d8057140 con 0x7fc7e0145d50 2024-05-28T13:59:36.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.221+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1728264993 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc7e006ed10 con 0x7fc7e0145d50 2024-05-28T13:59:36.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.225+0000 7fc7d6a00640 1 -- 172.21.15.42:0/1728264993 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fc7d805b440 con 0x7fc7e0145d50 2024-05-28T13:59:36.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.225+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1728264993 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7bc0683d0 msgr2=0x7fc7bc06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.225+0000 7fc7dfe00640 1 --2- 172.21.15.42:0/1728264993 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7bc0683d0 0x7fc7bc06a890 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00ad000 tx=0x7fc7d0008040 comp rx=0 tx=0).stop 2024-05-28T13:59:36.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.225+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1728264993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7e0145d50 msgr2=0x7fc7e015ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.225+0000 7fc7dfe00640 1 --2- 172.21.15.42:0/1728264993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7e0145d50 0x7fc7e015ccf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8007de0 tx=0x7fc7d800e4b0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.225+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1728264993 shutdown_connections 2024-05-28T13:59:36.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.225+0000 7fc7dfe00640 1 --2- 172.21.15.42:0/1728264993 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7bc0683d0 0x7fc7bc06a890 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.225+0000 7fc7dfe00640 1 --2- 172.21.15.42:0/1728264993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7e0145d50 0x7fc7e015ccf0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.225+0000 7fc7dfe00640 1 --2- 172.21.15.42:0/1728264993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e00cf540 0x7fc7e015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.225+0000 7fc7dfe00640 1 --2- 172.21.15.42:0/1728264993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e00707e0 0x7fc7e0154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.225+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1728264993 >> 172.21.15.42:0/1728264993 conn(0x7fc7e006fcc0 msgr2=0x7fc7e0144ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:36.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.225+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1728264993 shutdown_connections 2024-05-28T13:59:36.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.225+0000 7fc7dfe00640 1 -- 172.21.15.42:0/1728264993 wait complete. 2024-05-28T13:59:36.230 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-28T13:59:36.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.389+0000 7f37dbc00640 1 Processor -- start 2024-05-28T13:59:36.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.389+0000 7f37dbc00640 1 -- start start 2024-05-28T13:59:36.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.389+0000 7f37dbc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37d415eb10 0x7f37d40cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.389+0000 7f37dbc00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f37d40b3a50 0x7f37d40cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.389+0000 7f37dbc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37d40cf190 con 0x7f37d406b9f0 2024-05-28T13:59:36.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.389+0000 7f37dbc00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f37d40d05e0 con 0x7f37d40b3a50 2024-05-28T13:59:36.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.389+0000 7f37dbc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f37d40d0810 con 0x7f37d415eb10 2024-05-28T13:59:36.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.389+0000 7f37d9200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37d415eb10 0x7f37d40cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.389+0000 7f37cbe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f37d40b3a50 0x7f37d40cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.389+0000 7f37cbe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f37d40b3a50 0x7f37d40cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:54248/0 (socket says 172.21.15.42:54248) 2024-05-28T13:59:36.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.389+0000 7f37d9200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37d415eb10 0x7f37d40cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:32878/0 (socket says 172.21.15.42:32878) 2024-05-28T13:59:36.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.389+0000 7f37cbe00640 1 -- 172.21.15.42:0/4231922922 learned_addr learned my addr 172.21.15.42:0/4231922922 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:36.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.389+0000 7f37d9c00640 1 --1- 172.21.15.42:0/4231922922 >> v1:172.21.15.63:6789/0 conn(0x7f37d406b9f0 0x7f37d40ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:54410/0 (socket says 172.21.15.42:54410) 2024-05-28T13:59:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37cbe00640 1 -- 172.21.15.42:0/4231922922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37d415eb10 msgr2=0x7f37d40cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37cbe00640 1 --2- 172.21.15.42:0/4231922922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37d415eb10 0x7f37d40cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37cbe00640 1 -- 172.21.15.42:0/4231922922 >> v1:172.21.15.63:6789/0 conn(0x7f37d406b9f0 legacy=0x7f37d40ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37cbe00640 1 -- 172.21.15.42:0/4231922922 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d40d0af0 con 0x7f37d40b3a50 2024-05-28T13:59:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37d3e00640 1 -- 172.21.15.42:0/4231922922 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3809340611 0 0) 0x7f37d40cf190 con 0x7f37d406b9f0 2024-05-28T13:59:36.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37cbe00640 1 --2- 172.21.15.42:0/4231922922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37d40b3a50 0x7f37d40cb9e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009870 tx=0x7f37bc01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2c14f6f378a1b58 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:36.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37d3e00640 1 -- 172.21.15.42:0/4231922922 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0b0070 con 0x7f37d40b3a50 2024-05-28T13:59:36.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37d3e00640 1 -- 172.21.15.42:0/4231922922 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37bc0ab600 con 0x7f37d40b3a50 2024-05-28T13:59:36.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37d3e00640 1 -- 172.21.15.42:0/4231922922 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0ab9d0 con 0x7f37d40b3a50 2024-05-28T13:59:36.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37dbc00640 1 -- 172.21.15.42:0/4231922922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37d40b3a50 msgr2=0x7f37d40cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37dbc00640 1 --2- 172.21.15.42:0/4231922922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37d40b3a50 0x7f37d40cb9e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009870 tx=0x7f37bc01c4d0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37dbc00640 1 -- 172.21.15.42:0/4231922922 shutdown_connections 2024-05-28T13:59:36.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37dbc00640 1 --2- 172.21.15.42:0/4231922922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37d40b3a50 0x7f37d40cb9e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37dbc00640 1 --2- 172.21.15.42:0/4231922922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37d415eb10 0x7f37d40cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37dbc00640 1 -- 172.21.15.42:0/4231922922 >> 172.21.15.42:0/4231922922 conn(0x7f37d40b2f60 msgr2=0x7f37d40b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:36.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37dbc00640 1 -- 172.21.15.42:0/4231922922 shutdown_connections 2024-05-28T13:59:36.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.393+0000 7f37dbc00640 1 -- 172.21.15.42:0/4231922922 wait complete. 2024-05-28T13:59:36.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37dbc00640 1 Processor -- start 2024-05-28T13:59:36.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37dbc00640 1 -- start start 2024-05-28T13:59:36.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37dbc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37d406b9f0 0x7f37d415dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37dbc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37d40b3a50 0x7f37d415ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37dbc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37d415eb10 0x7f37d4160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37dbc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f37d406c250 con 0x7f37d406b9f0 2024-05-28T13:59:36.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37dbc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f37d40cec40 con 0x7f37d415eb10 2024-05-28T13:59:36.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37dbc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f37d416bdc0 con 0x7f37d40b3a50 2024-05-28T13:59:36.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37d9200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37d406b9f0 0x7f37d415dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37d9200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37d406b9f0 0x7f37d415dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41874/0 (socket says 172.21.15.42:41874) 2024-05-28T13:59:36.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37d9200640 1 -- 172.21.15.42:0/1488462190 learned_addr learned my addr 172.21.15.42:0/1488462190 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:36.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37d9c00640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37d415eb10 0x7f37d4160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37cbe00640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37d40b3a50 0x7f37d415ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37d9200640 1 -- 172.21.15.42:0/1488462190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37d40b3a50 msgr2=0x7f37d415ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37d9200640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37d40b3a50 0x7f37d415ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37d9200640 1 -- 172.21.15.42:0/1488462190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37d415eb10 msgr2=0x7f37d4160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37d9200640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37d415eb10 0x7f37d4160760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37d9200640 1 -- 172.21.15.42:0/1488462190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d415d150 con 0x7f37d406b9f0 2024-05-28T13:59:36.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37cbe00640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37d40b3a50 0x7f37d415ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T13:59:36.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37d9200640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37d406b9f0 0x7f37d415dd90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f37c400e870 tx=0x7f37c400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:36.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37d1600640 1 -- 172.21.15.42:0/1488462190 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c4017070 con 0x7f37d406b9f0 2024-05-28T13:59:36.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37d1600640 1 -- 172.21.15.42:0/1488462190 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c400c940 con 0x7f37d406b9f0 2024-05-28T13:59:36.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37d1600640 1 -- 172.21.15.42:0/1488462190 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c400ccf0 con 0x7f37d406b9f0 2024-05-28T13:59:36.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37dbc00640 1 -- 172.21.15.42:0/1488462190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d415f9a0 con 0x7f37d406b9f0 2024-05-28T13:59:36.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.397+0000 7f37dbc00640 1 -- 172.21.15.42:0/1488462190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d415ff60 con 0x7f37d406b9f0 2024-05-28T13:59:36.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.401+0000 7f37dbc00640 1 -- 172.21.15.42:0/1488462190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d4068a90 con 0x7f37d406b9f0 2024-05-28T13:59:36.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.401+0000 7f37d1600640 1 -- 172.21.15.42:0/1488462190 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f37c401a070 con 0x7f37d406b9f0 2024-05-28T13:59:36.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.401+0000 7f37d1600640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f37b00683b0 0x7f37b006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.401+0000 7f37d1600640 1 -- 172.21.15.42:0/1488462190 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f37c4086c30 con 0x7f37d406b9f0 2024-05-28T13:59:36.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.401+0000 7f37cbe00640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f37b00683b0 0x7f37b006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.401+0000 7f37cbe00640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f37b00683b0 0x7f37b006a870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0ad000 tx=0x7f37bc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:36.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.405+0000 7f37d1600640 1 -- 172.21.15.42:0/1488462190 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f37c4056b00 con 0x7f37d406b9f0 2024-05-28T13:59:36.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.597+0000 7f37dbc00640 1 -- 172.21.15.42:0/1488462190 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f37d4160ca0 con 0x7f37b00683b0 2024-05-28T13:59:36.587 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:59:36.601+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T13:59:36.587 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:59:36.601+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T13:59:36.587 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:59:36.601+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T13:59:36.588 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:59:36.601+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T13:59:36.588 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T13:59:36.601+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T13:59:36.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.749+0000 7f37d1600640 1 -- 172.21.15.42:0/1488462190 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f37bc0ab480 con 0x7f37b00683b0 2024-05-28T13:59:36.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.749+0000 7f37dbc00640 1 -- 172.21.15.42:0/1488462190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f37b00683b0 msgr2=0x7f37b006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.749+0000 7f37dbc00640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f37b00683b0 0x7f37b006a870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0ad000 tx=0x7f37bc008040 comp rx=0 tx=0).stop 2024-05-28T13:59:36.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.749+0000 7f37dbc00640 1 -- 172.21.15.42:0/1488462190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37d406b9f0 msgr2=0x7f37d415dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.749+0000 7f37dbc00640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37d406b9f0 0x7f37d415dd90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f37c400e870 tx=0x7f37c400ec70 comp rx=0 tx=0).stop 2024-05-28T13:59:36.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.749+0000 7f37dbc00640 1 -- 172.21.15.42:0/1488462190 shutdown_connections 2024-05-28T13:59:36.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.749+0000 7f37dbc00640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f37b00683b0 0x7f37b006a870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.749+0000 7f37dbc00640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37d415eb10 0x7f37d4160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.749+0000 7f37dbc00640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37d40b3a50 0x7f37d415ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.749+0000 7f37dbc00640 1 --2- 172.21.15.42:0/1488462190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37d406b9f0 0x7f37d415dd90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.749+0000 7f37dbc00640 1 -- 172.21.15.42:0/1488462190 >> 172.21.15.42:0/1488462190 conn(0x7f37d40b2f60 msgr2=0x7f37d415b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:36.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.749+0000 7f37dbc00640 1 -- 172.21.15.42:0/1488462190 shutdown_connections 2024-05-28T13:59:36.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.749+0000 7f37dbc00640 1 -- 172.21.15.42:0/1488462190 wait complete. 2024-05-28T13:59:36.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T13:59:36.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.913+0000 7f6d97e00640 1 Processor -- start 2024-05-28T13:59:36.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.913+0000 7f6d97e00640 1 -- start start 2024-05-28T13:59:36.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.917+0000 7f6d97e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6d90144f80 0x7f6d90072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.917+0000 7f6d97e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9006ee20 0x7f6d9006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.917+0000 7f6d97e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d900afe40 con 0x7f6d9015b9b0 2024-05-28T13:59:36.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.917+0000 7f6d97e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6d9006f7c0 con 0x7f6d90144f80 2024-05-28T13:59:36.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.917+0000 7f6d97e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6d900724d0 con 0x7f6d9006ee20 2024-05-28T13:59:36.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.917+0000 7f6d95400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6d90144f80 0x7f6d90072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.917+0000 7f6d95400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6d90144f80 0x7f6d90072960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41908/0 (socket says 172.21.15.42:41908) 2024-05-28T13:59:36.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.917+0000 7f6d95400640 1 -- 172.21.15.42:0/3005579587 learned_addr learned my addr 172.21.15.42:0/3005579587 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:36.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.917+0000 7f6d94a00640 1 --2- 172.21.15.42:0/3005579587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9006ee20 0x7f6d9006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.917+0000 7f6d95400640 1 -- 172.21.15.42:0/3005579587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9006ee20 msgr2=0x7f6d9006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T13:59:36.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.917+0000 7f6d95400640 1 --2- 172.21.15.42:0/3005579587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9006ee20 0x7f6d9006f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.917+0000 7f6d95400640 1 -- 172.21.15.42:0/3005579587 >> v1:172.21.15.63:6789/0 conn(0x7f6d9015b9b0 legacy=0x7f6d9015be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.917+0000 7f6d95400640 1 -- 172.21.15.42:0/3005579587 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9015c600 con 0x7f6d90144f80 2024-05-28T13:59:36.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.921+0000 7f6d95400640 1 --2- 172.21.15.42:0/3005579587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d90144f80 0x7f6d90072960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009870 tx=0x7f6d8401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4201776f548b9d63 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:36.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.921+0000 7f6d8fe00640 1 -- 172.21.15.42:0/3005579587 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d840ab650 con 0x7f6d90144f80 2024-05-28T13:59:36.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.921+0000 7f6d8fe00640 1 -- 172.21.15.42:0/3005579587 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d840b0070 con 0x7f6d90144f80 2024-05-28T13:59:36.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.921+0000 7f6d8fe00640 1 -- 172.21.15.42:0/3005579587 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d840aba10 con 0x7f6d90144f80 2024-05-28T13:59:36.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.921+0000 7f6d97e00640 1 -- 172.21.15.42:0/3005579587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d90144f80 msgr2=0x7f6d90072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.921+0000 7f6d97e00640 1 --2- 172.21.15.42:0/3005579587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d90144f80 0x7f6d90072960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009870 tx=0x7f6d8401c560 comp rx=0 tx=0).stop 2024-05-28T13:59:36.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.921+0000 7f6d97e00640 1 -- 172.21.15.42:0/3005579587 shutdown_connections 2024-05-28T13:59:36.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.921+0000 7f6d97e00640 1 --2- 172.21.15.42:0/3005579587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9006ee20 0x7f6d9006f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.921+0000 7f6d97e00640 1 --2- 172.21.15.42:0/3005579587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d90144f80 0x7f6d90072960 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.921+0000 7f6d97e00640 1 -- 172.21.15.42:0/3005579587 >> 172.21.15.42:0/3005579587 conn(0x7f6d900b21f0 msgr2=0x7f6d900b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:36.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.921+0000 7f6d97e00640 1 -- 172.21.15.42:0/3005579587 shutdown_connections 2024-05-28T13:59:36.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.921+0000 7f6d97e00640 1 -- 172.21.15.42:0/3005579587 wait complete. 2024-05-28T13:59:36.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.921+0000 7f6d97e00640 1 Processor -- start 2024-05-28T13:59:36.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.921+0000 7f6d97e00640 1 -- start start 2024-05-28T13:59:36.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d97e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9006ee20 0x7f6d90141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d97e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90144f80 0x7f6d901404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d97e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9015b9b0 0x7f6d900aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d97e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6d90072ea0 con 0x7f6d9015b9b0 2024-05-28T13:59:36.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d97e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6d900b4010 con 0x7f6d90144f80 2024-05-28T13:59:36.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d97e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6d90142330 con 0x7f6d9006ee20 2024-05-28T13:59:36.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d95e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9015b9b0 0x7f6d900aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d95e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9015b9b0 0x7f6d900aed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41922/0 (socket says 172.21.15.42:41922) 2024-05-28T13:59:36.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d95e00640 1 -- 172.21.15.42:0/3520746604 learned_addr learned my addr 172.21.15.42:0/3520746604 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T13:59:36.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d95400640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9006ee20 0x7f6d90141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d94a00640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90144f80 0x7f6d901404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d95e00640 1 -- 172.21.15.42:0/3520746604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9006ee20 msgr2=0x7f6d90141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d95e00640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9006ee20 0x7f6d90141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d95e00640 1 -- 172.21.15.42:0/3520746604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90144f80 msgr2=0x7f6d901404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:36.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d95e00640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90144f80 0x7f6d901404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:36.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d95e00640 1 -- 172.21.15.42:0/3520746604 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d90142610 con 0x7f6d9015b9b0 2024-05-28T13:59:36.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d95400640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9006ee20 0x7f6d90141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T13:59:36.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d95e00640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9015b9b0 0x7f6d900aed70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800c730 tx=0x7f6d8800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:36.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d8d600640 1 -- 172.21.15.42:0/3520746604 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88003b80 con 0x7f6d9015b9b0 2024-05-28T13:59:36.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d8d600640 1 -- 172.21.15.42:0/3520746604 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d88003d20 con 0x7f6d9015b9b0 2024-05-28T13:59:36.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d8d600640 1 -- 172.21.15.42:0/3520746604 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800aa30 con 0x7f6d9015b9b0 2024-05-28T13:59:36.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d97e00640 1 -- 172.21.15.42:0/3520746604 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d901428f0 con 0x7f6d9015b9b0 2024-05-28T13:59:36.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d97e00640 1 -- 172.21.15.42:0/3520746604 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d90142eb0 con 0x7f6d9015b9b0 2024-05-28T13:59:36.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.925+0000 7f6d97e00640 1 -- 172.21.15.42:0/3520746604 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d900738b0 con 0x7f6d9015b9b0 2024-05-28T13:59:36.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.929+0000 7f6d8d600640 1 -- 172.21.15.42:0/3520746604 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6d8800abd0 con 0x7f6d9015b9b0 2024-05-28T13:59:36.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.929+0000 7f6d8d600640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d70068380 0x7f6d7006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T13:59:36.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.929+0000 7f6d8d600640 1 -- 172.21.15.42:0/3520746604 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f6d88086850 con 0x7f6d9015b9b0 2024-05-28T13:59:36.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.929+0000 7f6d95400640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d70068380 0x7f6d7006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T13:59:36.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.929+0000 7f6d95400640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d70068380 0x7f6d7006a840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6d840af000 tx=0x7f6d840a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T13:59:36.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:36.933+0000 7f6d8d600640 1 -- 172.21.15.42:0/3520746604 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6d88056e00 con 0x7f6d9015b9b0 2024-05-28T13:59:37.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:37.121+0000 7f6d97e00640 1 -- 172.21.15.42:0/3520746604 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f6d9004fdd0 con 0x7f6d70068380 2024-05-28T13:59:37.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:37.129+0000 7f6d8d600640 1 -- 172.21.15.42:0/3520746604 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f6d9004fdd0 con 0x7f6d70068380 2024-05-28T13:59:37.113 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25 2024-05-28T13:59:37.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:37.129+0000 7f6d97e00640 1 -- 172.21.15.42:0/3520746604 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d70068380 msgr2=0x7f6d7006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:37.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:37.129+0000 7f6d97e00640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d70068380 0x7f6d7006a840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6d840af000 tx=0x7f6d840a7000 comp rx=0 tx=0).stop 2024-05-28T13:59:37.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:37.129+0000 7f6d97e00640 1 -- 172.21.15.42:0/3520746604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9015b9b0 msgr2=0x7f6d900aed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T13:59:37.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:37.129+0000 7f6d97e00640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9015b9b0 0x7f6d900aed70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800c730 tx=0x7f6d8800cbf0 comp rx=0 tx=0).stop 2024-05-28T13:59:37.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:37.129+0000 7f6d97e00640 1 -- 172.21.15.42:0/3520746604 shutdown_connections 2024-05-28T13:59:37.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:37.129+0000 7f6d97e00640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d70068380 0x7f6d7006a840 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:37.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:37.129+0000 7f6d97e00640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9015b9b0 0x7f6d900aed70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:37.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:37.129+0000 7f6d97e00640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90144f80 0x7f6d901404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:37.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:37.129+0000 7f6d97e00640 1 --2- 172.21.15.42:0/3520746604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9006ee20 0x7f6d90141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T13:59:37.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:37.129+0000 7f6d97e00640 1 -- 172.21.15.42:0/3520746604 >> 172.21.15.42:0/3520746604 conn(0x7f6d900b21f0 msgr2=0x7f6d90155810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T13:59:37.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:37.129+0000 7f6d97e00640 1 -- 172.21.15.42:0/3520746604 shutdown_connections 2024-05-28T13:59:37.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T13:59:37.129+0000 7f6d97e00640 1 -- 172.21.15.42:0/3520746604 wait complete. 2024-05-28T13:59:37.134 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25 2024-05-28T13:59:37.134 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:59:37.134 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:59:37.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:37.143 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T13:59:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T13:59:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T13:59:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.06214 s, 152 MB/s 2024-05-28T13:59:44.271 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:59:44.271 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:59:44.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:44.280 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T13:59:47.489 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T13:59:47.493 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T13:59:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T13:59:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T13:59:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.87543 s, 156 MB/s 2024-05-28T13:59:51.213 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:59:51.213 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:59:51.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:51.222 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T13:59:58.251 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T13:59:58.251 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T13:59:58.251 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.97382 s, 154 MB/s 2024-05-28T13:59:58.252 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T13:59:58.253 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T13:59:58.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T13:59:58.262 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:00:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:00:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:00:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.91629 s, 155 MB/s 2024-05-28T14:00:05.239 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:00:05.239 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:00:05.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:00:05.249 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:00:12.143 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:00:12.143 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:00:12.143 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.83837 s, 157 MB/s 2024-05-28T14:00:12.144 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:00:12.144 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:00:12.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:00:12.153 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:00:17.563 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:00:17.567 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:00:19.094 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:00:19.094 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:00:19.094 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.88215 s, 156 MB/s 2024-05-28T14:00:19.095 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:00:19.095 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:00:19.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:00:19.105 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:00:25.998 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:00:25.998 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:00:25.998 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.83733 s, 157 MB/s 2024-05-28T14:00:25.999 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:00:25.999 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:00:26.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:00:26.008 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:00:32.976 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:00:32.976 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:00:32.976 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.90814 s, 155 MB/s 2024-05-28T14:00:32.977 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:00:32.978 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:00:32.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:00:32.986 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:00:39.945 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:00:39.946 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:00:39.946 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.90006 s, 156 MB/s 2024-05-28T14:00:39.947 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:00:39.947 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:00:39.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:00:39.956 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:00:46.912 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:00:46.913 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:00:46.913 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.9003 s, 156 MB/s 2024-05-28T14:00:46.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-28T14:00:47.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.072+0000 7ff0bf600640 1 Processor -- start 2024-05-28T14:00:47.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.072+0000 7ff0bf600640 1 -- start start 2024-05-28T14:00:47.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.076+0000 7ff0bf600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff0b806ee80 0x7ff0b806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:47.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.076+0000 7ff0bf600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff0b80af360 0x7ff0b80af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:47.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.076+0000 7ff0bf600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b81570a0 con 0x7ff0b80d06f0 2024-05-28T14:00:47.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.076+0000 7ff0bf600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff0b815cf00 con 0x7ff0b806ee80 2024-05-28T14:00:47.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.076+0000 7ff0bf600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff0b815d1e0 con 0x7ff0b80af360 2024-05-28T14:00:47.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.076+0000 7ff0bcc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff0b806ee80 0x7ff0b806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:47.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.076+0000 7ff0bd600640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff0b80d06f0 0x7ff0b8160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:41552/0 (socket says 172.21.15.42:41552) 2024-05-28T14:00:47.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.076+0000 7ff0bd600640 1 -- 172.21.15.42:0/3758325782 learned_addr learned my addr 172.21.15.42:0/3758325782 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:00:47.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.076+0000 7ff0b7e00640 1 --2- 172.21.15.42:0/3758325782 >> v2:172.21.15.63:3300/0 conn(0x7ff0b80af360 0x7ff0b80af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:47.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.076+0000 7ff0bcc00640 1 -- 172.21.15.42:0/3758325782 >> v2:172.21.15.63:3300/0 conn(0x7ff0b80af360 msgr2=0x7ff0b80af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:00:47.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.076+0000 7ff0bcc00640 1 --2- 172.21.15.42:0/3758325782 >> v2:172.21.15.63:3300/0 conn(0x7ff0b80af360 0x7ff0b80af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.076+0000 7ff0bcc00640 1 -- 172.21.15.42:0/3758325782 >> v1:172.21.15.42:6789/0 conn(0x7ff0b80d06f0 legacy=0x7ff0b8160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:47.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.076+0000 7ff0bcc00640 1 -- 172.21.15.42:0/3758325782 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b815d4c0 con 0x7ff0b806ee80 2024-05-28T14:00:47.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.076+0000 7ff0b7e00640 1 --2- 172.21.15.42:0/3758325782 >> v2:172.21.15.63:3300/0 conn(0x7ff0b80af360 0x7ff0b80af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:00:47.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.080+0000 7ff0bcc00640 1 --2- 172.21.15.42:0/3758325782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0b806ee80 0x7ff0b806dd40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8073fc0 tx=0x7ff0ac01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabe49449a82edd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:47.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.080+0000 7ff0b7400640 1 -- 172.21.15.42:0/3758325782 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac0b0070 con 0x7ff0b806ee80 2024-05-28T14:00:47.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.080+0000 7ff0b7400640 1 -- 172.21.15.42:0/3758325782 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0ac0ab600 con 0x7ff0b806ee80 2024-05-28T14:00:47.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.080+0000 7ff0b7400640 1 -- 172.21.15.42:0/3758325782 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac0ab9d0 con 0x7ff0b806ee80 2024-05-28T14:00:47.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.080+0000 7ff0bf600640 1 -- 172.21.15.42:0/3758325782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0b806ee80 msgr2=0x7ff0b806dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:47.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.080+0000 7ff0bf600640 1 --2- 172.21.15.42:0/3758325782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0b806ee80 0x7ff0b806dd40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8073fc0 tx=0x7ff0ac01c3c0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.080+0000 7ff0bf600640 1 -- 172.21.15.42:0/3758325782 shutdown_connections 2024-05-28T14:00:47.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.080+0000 7ff0bf600640 1 --2- 172.21.15.42:0/3758325782 >> v2:172.21.15.63:3300/0 conn(0x7ff0b80af360 0x7ff0b80af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.080+0000 7ff0bf600640 1 --2- 172.21.15.42:0/3758325782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0b806ee80 0x7ff0b806dd40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.080+0000 7ff0bf600640 1 -- 172.21.15.42:0/3758325782 >> 172.21.15.42:0/3758325782 conn(0x7ff0b806bdf0 msgr2=0x7ff0b806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:47.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.080+0000 7ff0bf600640 1 -- 172.21.15.42:0/3758325782 shutdown_connections 2024-05-28T14:00:47.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.080+0000 7ff0bf600640 1 -- 172.21.15.42:0/3758325782 wait complete. 2024-05-28T14:00:47.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.080+0000 7ff0bf600640 1 Processor -- start 2024-05-28T14:00:47.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.080+0000 7ff0bf600640 1 -- start start 2024-05-28T14:00:47.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bf600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b806ee80 0x7ff0b80738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:47.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bf600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0b80af360 0x7ff0b815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:47.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bf600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0b80d06f0 0x7ff0b81fb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:47.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bf600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b80d1060 con 0x7ff0b80d06f0 2024-05-28T14:00:47.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bf600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b80d0ee0 con 0x7ff0b806ee80 2024-05-28T14:00:47.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bf600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b8160fa0 con 0x7ff0b80af360 2024-05-28T14:00:47.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bd600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0b80d06f0 0x7ff0b81fb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:47.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bd600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0b80d06f0 0x7ff0b81fb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57182/0 (socket says 172.21.15.42:57182) 2024-05-28T14:00:47.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bd600640 1 -- 172.21.15.42:0/667074859 learned_addr learned my addr 172.21.15.42:0/667074859 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:00:47.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0b7e00640 1 --2- 172.21.15.42:0/667074859 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0b80af360 0x7ff0b815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:47.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bcc00640 1 --2- 172.21.15.42:0/667074859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b806ee80 0x7ff0b80738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:47.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bd600640 1 -- 172.21.15.42:0/667074859 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0b80af360 msgr2=0x7ff0b815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:47.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bd600640 1 --2- 172.21.15.42:0/667074859 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0b80af360 0x7ff0b815b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bd600640 1 -- 172.21.15.42:0/667074859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b806ee80 msgr2=0x7ff0b80738e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:47.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bd600640 1 --2- 172.21.15.42:0/667074859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b806ee80 0x7ff0b80738e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bd600640 1 -- 172.21.15.42:0/667074859 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b815c0c0 con 0x7ff0b80d06f0 2024-05-28T14:00:47.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bd600640 1 --2- 172.21.15.42:0/667074859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0b80d06f0 0x7ff0b81fb6b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000cae0 tx=0x7ff0b000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:47.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0b4c00640 1 -- 172.21.15.42:0/667074859 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0017070 con 0x7ff0b80d06f0 2024-05-28T14:00:47.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0b4c00640 1 -- 172.21.15.42:0/667074859 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b000ac40 con 0x7ff0b80d06f0 2024-05-28T14:00:47.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0b4c00640 1 -- 172.21.15.42:0/667074859 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0012400 con 0x7ff0b80d06f0 2024-05-28T14:00:47.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bf600640 1 -- 172.21.15.42:0/667074859 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b815c3a0 con 0x7ff0b80d06f0 2024-05-28T14:00:47.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bf600640 1 -- 172.21.15.42:0/667074859 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b81fb190 con 0x7ff0b80d06f0 2024-05-28T14:00:47.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0bf600640 1 -- 172.21.15.42:0/667074859 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b8068a90 con 0x7ff0b80d06f0 2024-05-28T14:00:47.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.084+0000 7ff0b4c00640 1 -- 172.21.15.42:0/667074859 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 6) ==== 88038+0+0 (secure 0 0 0) 0x7ff0b001a070 con 0x7ff0b80d06f0 2024-05-28T14:00:47.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.088+0000 7ff0b4c00640 1 --2- 172.21.15.42:0/667074859 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff098068410 0x7ff09806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:47.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.088+0000 7ff0b4c00640 1 -- 172.21.15.42:0/667074859 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ff0b00866d0 con 0x7ff0b80d06f0 2024-05-28T14:00:47.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.088+0000 7ff0bcc00640 1 --2- 172.21.15.42:0/667074859 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff098068410 0x7ff09806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:47.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.088+0000 7ff0bcc00640 1 --2- 172.21.15.42:0/667074859 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff098068410 0x7ff09806a8d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0ad000 tx=0x7ff0ac0a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:47.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.092+0000 7ff0b4c00640 1 -- 172.21.15.42:0/667074859 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff0b00565e0 con 0x7ff0b80d06f0 2024-05-28T14:00:47.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.280+0000 7ff0bf600640 1 -- 172.21.15.42:0/667074859 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7ff0b80b5160 con 0x7ff098068410 2024-05-28T14:00:47.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.296+0000 7ff0b4c00640 1 -- 172.21.15.42:0/667074859 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff0b80b5160 con 0x7ff098068410 2024-05-28T14:00:47.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.296+0000 7ff0bf600640 1 -- 172.21.15.42:0/667074859 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff098068410 msgr2=0x7ff09806a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:47.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.296+0000 7ff0bf600640 1 --2- 172.21.15.42:0/667074859 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff098068410 0x7ff09806a8d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0ad000 tx=0x7ff0ac0a6000 comp rx=0 tx=0).stop 2024-05-28T14:00:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.296+0000 7ff0bf600640 1 -- 172.21.15.42:0/667074859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0b80d06f0 msgr2=0x7ff0b81fb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.296+0000 7ff0bf600640 1 --2- 172.21.15.42:0/667074859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0b80d06f0 0x7ff0b81fb6b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000cae0 tx=0x7ff0b000a060 comp rx=0 tx=0).stop 2024-05-28T14:00:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.296+0000 7ff0bf600640 1 -- 172.21.15.42:0/667074859 shutdown_connections 2024-05-28T14:00:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.296+0000 7ff0bf600640 1 --2- 172.21.15.42:0/667074859 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff098068410 0x7ff09806a8d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.296+0000 7ff0bf600640 1 --2- 172.21.15.42:0/667074859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0b80d06f0 0x7ff0b81fb6b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.296+0000 7ff0bf600640 1 --2- 172.21.15.42:0/667074859 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0b80af360 0x7ff0b815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.296+0000 7ff0bf600640 1 --2- 172.21.15.42:0/667074859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b806ee80 0x7ff0b80738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.296+0000 7ff0bf600640 1 -- 172.21.15.42:0/667074859 >> 172.21.15.42:0/667074859 conn(0x7ff0b806bdf0 msgr2=0x7ff0b806f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.296+0000 7ff0bf600640 1 -- 172.21.15.42:0/667074859 shutdown_connections 2024-05-28T14:00:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.296+0000 7ff0bf600640 1 -- 172.21.15.42:0/667074859 wait complete. 2024-05-28T14:00:47.302 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:00:47.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd3200640 1 Processor -- start 2024-05-28T14:00:47.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd3200640 1 -- start start 2024-05-28T14:00:47.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd3200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4ccc0700a0 0x7f4ccc070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:47.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd3200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ccc0b02d0 0x7f4ccc160a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:47.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd3200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ccc0d19c0 con 0x7f4ccc0d0660 2024-05-28T14:00:47.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd3200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f4ccc0d00c0 con 0x7f4ccc0700a0 2024-05-28T14:00:47.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd3200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4ccc070d20 con 0x7f4ccc0b02d0 2024-05-28T14:00:47.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd1e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4ccc0700a0 0x7f4ccc070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:47.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd1e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4ccc0700a0 0x7f4ccc070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46662/0 (socket says 172.21.15.42:46662) 2024-05-28T14:00:47.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd1e00640 1 -- 172.21.15.42:0/1792375661 learned_addr learned my addr 172.21.15.42:0/1792375661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:00:47.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd2800640 1 --2- 172.21.15.42:0/1792375661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ccc0b02d0 0x7f4ccc160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:47.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd1e00640 1 -- 172.21.15.42:0/1792375661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ccc0b02d0 msgr2=0x7f4ccc160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:00:47.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd1e00640 1 --2- 172.21.15.42:0/1792375661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ccc0b02d0 0x7f4ccc160a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd1e00640 1 -- 172.21.15.42:0/1792375661 >> v1:172.21.15.63:6789/0 conn(0x7f4ccc0d0660 legacy=0x7f4ccc160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:47.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.460+0000 7f4cd1e00640 1 -- 172.21.15.42:0/1792375661 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ccc0709c0 con 0x7f4ccc0700a0 2024-05-28T14:00:47.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.464+0000 7f4cd1e00640 1 --2- 172.21.15.42:0/1792375661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ccc0700a0 0x7f4ccc070480 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009870 tx=0x7f4cc001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b2c837ba23f300 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:47.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.464+0000 7f4cd0a00640 1 -- 172.21.15.42:0/1792375661 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc00b0070 con 0x7f4ccc0700a0 2024-05-28T14:00:47.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.464+0000 7f4cd0a00640 1 -- 172.21.15.42:0/1792375661 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc00ab600 con 0x7f4ccc0700a0 2024-05-28T14:00:47.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.464+0000 7f4cd0a00640 1 -- 172.21.15.42:0/1792375661 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc00ab9d0 con 0x7f4ccc0700a0 2024-05-28T14:00:47.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.464+0000 7f4cd3200640 1 -- 172.21.15.42:0/1792375661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ccc0700a0 msgr2=0x7f4ccc070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:47.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.464+0000 7f4cd3200640 1 --2- 172.21.15.42:0/1792375661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ccc0700a0 0x7f4ccc070480 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009870 tx=0x7f4cc001c440 comp rx=0 tx=0).stop 2024-05-28T14:00:47.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.464+0000 7f4cd3200640 1 -- 172.21.15.42:0/1792375661 shutdown_connections 2024-05-28T14:00:47.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.464+0000 7f4cd3200640 1 --2- 172.21.15.42:0/1792375661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ccc0b02d0 0x7f4ccc160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.464+0000 7f4cd3200640 1 --2- 172.21.15.42:0/1792375661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ccc0700a0 0x7f4ccc070480 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.464+0000 7f4cd3200640 1 -- 172.21.15.42:0/1792375661 >> 172.21.15.42:0/1792375661 conn(0x7f4ccc0cd540 msgr2=0x7f4ccc0cd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:47.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.464+0000 7f4cd3200640 1 -- 172.21.15.42:0/1792375661 shutdown_connections 2024-05-28T14:00:47.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.464+0000 7f4cd3200640 1 -- 172.21.15.42:0/1792375661 wait complete. 2024-05-28T14:00:47.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd3200640 1 Processor -- start 2024-05-28T14:00:47.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd3200640 1 -- start start 2024-05-28T14:00:47.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd3200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ccc0700a0 0x7f4ccc147090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:47.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd3200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ccc0b02d0 0x7f4ccc14fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:47.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ccc0d0660 0x7f4ccc14b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:47.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd3200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc15ca50 con 0x7f4ccc0b02d0 2024-05-28T14:00:47.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd3200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc0cfde0 con 0x7f4ccc0d0660 2024-05-28T14:00:47.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd3200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4ccc15cbd0 con 0x7f4ccc0700a0 2024-05-28T14:00:47.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd1e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ccc0700a0 0x7f4ccc147090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:47.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd1e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ccc0700a0 0x7f4ccc147090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:54632/0 (socket says 172.21.15.42:54632) 2024-05-28T14:00:47.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd1e00640 1 -- 172.21.15.42:0/3248674848 learned_addr learned my addr 172.21.15.42:0/3248674848 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:00:47.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd1400640 1 --2- 172.21.15.42:0/3248674848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ccc0b02d0 0x7f4ccc14fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:47.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd2800640 1 --2- 172.21.15.42:0/3248674848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ccc0d0660 0x7f4ccc14b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:47.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd1e00640 1 -- 172.21.15.42:0/3248674848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ccc0d0660 msgr2=0x7f4ccc14b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:47.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd1e00640 1 --2- 172.21.15.42:0/3248674848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ccc0d0660 0x7f4ccc14b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd1e00640 1 -- 172.21.15.42:0/3248674848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ccc0b02d0 msgr2=0x7f4ccc14fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:47.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd1e00640 1 --2- 172.21.15.42:0/3248674848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ccc0b02d0 0x7f4ccc14fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd1e00640 1 -- 172.21.15.42:0/3248674848 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ccc14da80 con 0x7f4ccc0700a0 2024-05-28T14:00:47.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.468+0000 7f4cd1e00640 1 --2- 172.21.15.42:0/3248674848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ccc0700a0 0x7f4ccc147090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4cc00ae040 tx=0x7f4cc001ac80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:47.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.472+0000 7f4cc6000640 1 -- 172.21.15.42:0/3248674848 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc00ba020 con 0x7f4ccc0700a0 2024-05-28T14:00:47.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.472+0000 7f4cc6000640 1 -- 172.21.15.42:0/3248674848 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc00b6420 con 0x7f4ccc0700a0 2024-05-28T14:00:47.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.472+0000 7f4cc6000640 1 -- 172.21.15.42:0/3248674848 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc00b0040 con 0x7f4ccc0700a0 2024-05-28T14:00:47.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.472+0000 7f4cd3200640 1 -- 172.21.15.42:0/3248674848 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ccc14c1a0 con 0x7f4ccc0700a0 2024-05-28T14:00:47.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.472+0000 7f4cd3200640 1 -- 172.21.15.42:0/3248674848 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ccc14e020 con 0x7f4ccc0700a0 2024-05-28T14:00:47.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.472+0000 7f4cd3200640 1 -- 172.21.15.42:0/3248674848 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ccc04fcf0 con 0x7f4ccc0700a0 2024-05-28T14:00:47.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.472+0000 7f4cc6000640 1 -- 172.21.15.42:0/3248674848 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 6) ==== 88038+0+0 (secure 0 0 0) 0x7f4cc00bd020 con 0x7f4ccc0700a0 2024-05-28T14:00:47.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.472+0000 7f4cc6000640 1 --2- 172.21.15.42:0/3248674848 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4cb0068410 0x7f4cb006a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:47.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.472+0000 7f4cc6000640 1 -- 172.21.15.42:0/3248674848 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f4cc0120b20 con 0x7f4ccc0700a0 2024-05-28T14:00:47.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.472+0000 7f4cd1400640 1 --2- 172.21.15.42:0/3248674848 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4cb0068410 0x7f4cb006a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:47.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.476+0000 7f4cd1400640 1 --2- 172.21.15.42:0/3248674848 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4cb0068410 0x7f4cb006a8d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc1516e0 tx=0x7f4cbc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:47.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.480+0000 7f4cc6000640 1 -- 172.21.15.42:0/3248674848 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4cc00c4030 con 0x7f4ccc0700a0 2024-05-28T14:00:47.610 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:00:47.613 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:00:47.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.672+0000 7f4cd3200640 1 -- 172.21.15.42:0/3248674848 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f4ccc160080 con 0x7f4cb0068410 2024-05-28T14:00:47.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.676+0000 7f4cc6000640 1 -- 172.21.15.42:0/3248674848 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f4ccc160080 con 0x7f4cb0068410 2024-05-28T14:00:47.662 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25 2024-05-28T14:00:47.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.680+0000 7f4cd3200640 1 -- 172.21.15.42:0/3248674848 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4cb0068410 msgr2=0x7f4cb006a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:47.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.680+0000 7f4cd3200640 1 --2- 172.21.15.42:0/3248674848 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4cb0068410 0x7f4cb006a8d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc1516e0 tx=0x7f4cbc009040 comp rx=0 tx=0).stop 2024-05-28T14:00:47.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.680+0000 7f4cd3200640 1 -- 172.21.15.42:0/3248674848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ccc0700a0 msgr2=0x7f4ccc147090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:47.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.680+0000 7f4cd3200640 1 --2- 172.21.15.42:0/3248674848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ccc0700a0 0x7f4ccc147090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4cc00ae040 tx=0x7f4cc001ac80 comp rx=0 tx=0).stop 2024-05-28T14:00:47.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.680+0000 7f4cd3200640 1 -- 172.21.15.42:0/3248674848 shutdown_connections 2024-05-28T14:00:47.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.680+0000 7f4cd3200640 1 --2- 172.21.15.42:0/3248674848 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4cb0068410 0x7f4cb006a8d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.680+0000 7f4cd3200640 1 --2- 172.21.15.42:0/3248674848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ccc0d0660 0x7f4ccc14b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.680+0000 7f4cd3200640 1 --2- 172.21.15.42:0/3248674848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ccc0b02d0 0x7f4ccc14fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.680+0000 7f4cd3200640 1 --2- 172.21.15.42:0/3248674848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ccc0700a0 0x7f4ccc147090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:47.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.680+0000 7f4cd3200640 1 -- 172.21.15.42:0/3248674848 >> 172.21.15.42:0/3248674848 conn(0x7f4ccc0cd540 msgr2=0x7f4ccc0ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:47.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.680+0000 7f4cd3200640 1 -- 172.21.15.42:0/3248674848 shutdown_connections 2024-05-28T14:00:47.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:47.680+0000 7f4cd3200640 1 -- 172.21.15.42:0/3248674848 wait complete. 2024-05-28T14:00:47.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:00:47.683 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25 2024-05-28T14:00:47.683 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:00:47.753 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25 2024-05-28T14:00:47.753 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="10654449664" 2024-05-28T14:00:47.753 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:00:47.754 INFO:tasks.cephfs.test_volumes:r_size = 10654449664 exp_size = 10737418240 2024-05-28T14:00:49.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:00:49.755 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25 2024-05-28T14:00:49.755 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:00:49.770 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/6d82dfac-4efe-4f5c-92c5-4f2ac03cba25 2024-05-28T14:00:49.770 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="10737418240" 2024-05-28T14:00:49.770 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:00:49.771 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-05-28T14:00:49.771 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-05-28T14:00:49.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.980+0000 7f281ae00640 1 Processor -- start 2024-05-28T14:00:49.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.980+0000 7f281ae00640 1 -- start start 2024-05-28T14:00:49.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f281ae00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2814071de0 0x7f28140721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:49.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f281ae00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f28140ce3b0 0x7f28140ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:49.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f281ae00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2814141d10 0x7f28141421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:49.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f281ae00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f2814072700 con 0x7f28140ce3b0 2024-05-28T14:00:49.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f281ae00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2814160230 con 0x7f2814071de0 2024-05-28T14:00:49.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f281ae00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2814160510 con 0x7f2814141d10 2024-05-28T14:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f280b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f28140ce3b0 0x7f28140ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f280b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f28140ce3b0 0x7f28140ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46676/0 (socket says 172.21.15.42:46676) 2024-05-28T14:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f280b400640 1 -- 172.21.15.42:0/2467752398 learned_addr learned my addr 172.21.15.42:0/2467752398 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f2813e00640 1 --2- 172.21.15.42:0/2467752398 >> v2:172.21.15.63:3300/0 conn(0x7f2814071de0 0x7f28140721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f2818e00640 1 --2- 172.21.15.42:0/2467752398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2814141d10 0x7f28141421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f280b400640 1 -- 172.21.15.42:0/2467752398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2814141d10 msgr2=0x7f28141421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f280b400640 1 --2- 172.21.15.42:0/2467752398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2814141d10 0x7f28141421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f280b400640 1 -- 172.21.15.42:0/2467752398 >> v2:172.21.15.63:3300/0 conn(0x7f2814071de0 msgr2=0x7f28140721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f280b400640 1 --2- 172.21.15.42:0/2467752398 >> v2:172.21.15.63:3300/0 conn(0x7f2814071de0 0x7f28140721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.984+0000 7f280b400640 1 -- 172.21.15.42:0/2467752398 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28141607f0 con 0x7f28140ce3b0 2024-05-28T14:00:49.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f280b400640 1 --2- 172.21.15.42:0/2467752398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28140ce3b0 0x7f28140ce810 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f281406cbf0 tx=0x7f27fc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f102c33414b82e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:49.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f2813400640 1 -- 172.21.15.42:0/2467752398 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0b0070 con 0x7f28140ce3b0 2024-05-28T14:00:49.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f2813400640 1 -- 172.21.15.42:0/2467752398 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27fc0a4e50 con 0x7f28140ce3b0 2024-05-28T14:00:49.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f2813400640 1 -- 172.21.15.42:0/2467752398 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0ab820 con 0x7f28140ce3b0 2024-05-28T14:00:49.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f281ae00640 1 -- 172.21.15.42:0/2467752398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28140ce3b0 msgr2=0x7f28140ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:49.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f281ae00640 1 --2- 172.21.15.42:0/2467752398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28140ce3b0 0x7f28140ce810 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f281406cbf0 tx=0x7f27fc01c400 comp rx=0 tx=0).stop 2024-05-28T14:00:49.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f281ae00640 1 -- 172.21.15.42:0/2467752398 shutdown_connections 2024-05-28T14:00:49.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f281ae00640 1 --2- 172.21.15.42:0/2467752398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2814141d10 0x7f28141421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:49.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f281ae00640 1 --2- 172.21.15.42:0/2467752398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28140ce3b0 0x7f28140ce810 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:49.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f281ae00640 1 --2- 172.21.15.42:0/2467752398 >> v2:172.21.15.63:3300/0 conn(0x7f2814071de0 0x7f28140721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:49.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f281ae00640 1 -- 172.21.15.42:0/2467752398 >> 172.21.15.42:0/2467752398 conn(0x7f28140b21f0 msgr2=0x7f28140b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:49.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f281ae00640 1 -- 172.21.15.42:0/2467752398 shutdown_connections 2024-05-28T14:00:49.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f281ae00640 1 -- 172.21.15.42:0/2467752398 wait complete. 2024-05-28T14:00:49.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f281ae00640 1 Processor -- start 2024-05-28T14:00:49.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.988+0000 7f281ae00640 1 -- start start 2024-05-28T14:00:49.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f281ae00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2814071de0 0x7f28141533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:49.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f281ae00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28140ce3b0 0x7f281413c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f281ae00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2814141d10 0x7f281413cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f281ae00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2814140520 con 0x7f2814141d10 2024-05-28T14:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f281ae00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f281413f8c0 con 0x7f2814071de0 2024-05-28T14:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f281ae00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f28141406a0 con 0x7f28140ce3b0 2024-05-28T14:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2818e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2814141d10 0x7f281413cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2813e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2814071de0 0x7f28141533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2818e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2814141d10 0x7f281413cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46690/0 (socket says 172.21.15.42:46690) 2024-05-28T14:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2813e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2814071de0 0x7f28141533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:40842/0 (socket says 172.21.15.42:40842) 2024-05-28T14:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2818e00640 1 -- 172.21.15.42:0/3267482065 learned_addr learned my addr 172.21.15.42:0/3267482065 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f280b400640 1 --2- 172.21.15.42:0/3267482065 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28140ce3b0 0x7f281413c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2818e00640 1 -- 172.21.15.42:0/3267482065 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28140ce3b0 msgr2=0x7f281413c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:49.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2818e00640 1 --2- 172.21.15.42:0/3267482065 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28140ce3b0 0x7f281413c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:49.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2818e00640 1 -- 172.21.15.42:0/3267482065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2814071de0 msgr2=0x7f28141533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:49.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2818e00640 1 --2- 172.21.15.42:0/3267482065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2814071de0 0x7f28141533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:49.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2818e00640 1 -- 172.21.15.42:0/3267482065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28140b42e0 con 0x7f2814141d10 2024-05-28T14:00:49.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2818e00640 1 --2- 172.21.15.42:0/3267482065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2814141d10 0x7f281413cda0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f280c007e90 tx=0x7f280c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:49.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2810c00640 1 -- 172.21.15.42:0/3267482065 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c014420 con 0x7f2814141d10 2024-05-28T14:00:49.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2810c00640 1 -- 172.21.15.42:0/3267482065 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f280c019070 con 0x7f2814141d10 2024-05-28T14:00:49.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2810c00640 1 -- 172.21.15.42:0/3267482065 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c014800 con 0x7f2814141d10 2024-05-28T14:00:49.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f281ae00640 1 -- 172.21.15.42:0/3267482065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28140b45c0 con 0x7f2814141d10 2024-05-28T14:00:49.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f281ae00640 1 -- 172.21.15.42:0/3267482065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28140b4b80 con 0x7f2814141d10 2024-05-28T14:00:49.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f281ae00640 1 -- 172.21.15.42:0/3267482065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28140738b0 con 0x7f2814141d10 2024-05-28T14:00:49.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.992+0000 7f2810c00640 1 -- 172.21.15.42:0/3267482065 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 6) ==== 88038+0+0 (secure 0 0 0) 0x7f280c0149a0 con 0x7f2814141d10 2024-05-28T14:00:49.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.996+0000 7f2810c00640 1 --2- 172.21.15.42:0/3267482065 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f27f0068410 0x7f27f006a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:49.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.996+0000 7f2810c00640 1 -- 172.21.15.42:0/3267482065 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f280c087260 con 0x7f2814141d10 2024-05-28T14:00:49.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.996+0000 7f2813e00640 1 --2- 172.21.15.42:0/3267482065 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f27f0068410 0x7f27f006a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:49.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:49.996+0000 7f2813e00640 1 --2- 172.21.15.42:0/3267482065 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f27f0068410 0x7f27f006a8d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f28040045f0 tx=0x7f2804008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:49.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.000+0000 7f2810c00640 1 -- 172.21.15.42:0/3267482065 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f280c057050 con 0x7f2814141d10 2024-05-28T14:00:50.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.196+0000 7f281ae00640 1 -- 172.21.15.42:0/3267482065 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f2814072ed0 con 0x7f27f0068410 2024-05-28T14:00:50.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.264+0000 7f2810c00640 1 -- 172.21.15.42:0/3267482065 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2814072ed0 con 0x7f27f0068410 2024-05-28T14:00:50.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.264+0000 7f281ae00640 1 -- 172.21.15.42:0/3267482065 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f27f0068410 msgr2=0x7f27f006a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:50.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.264+0000 7f281ae00640 1 --2- 172.21.15.42:0/3267482065 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f27f0068410 0x7f27f006a8d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f28040045f0 tx=0x7f2804008040 comp rx=0 tx=0).stop 2024-05-28T14:00:50.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.264+0000 7f281ae00640 1 -- 172.21.15.42:0/3267482065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2814141d10 msgr2=0x7f281413cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:50.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.264+0000 7f281ae00640 1 --2- 172.21.15.42:0/3267482065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2814141d10 0x7f281413cda0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f280c007e90 tx=0x7f280c00e4b0 comp rx=0 tx=0).stop 2024-05-28T14:00:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.268+0000 7f281ae00640 1 -- 172.21.15.42:0/3267482065 shutdown_connections 2024-05-28T14:00:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.268+0000 7f281ae00640 1 --2- 172.21.15.42:0/3267482065 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f27f0068410 0x7f27f006a8d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.268+0000 7f281ae00640 1 --2- 172.21.15.42:0/3267482065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2814141d10 0x7f281413cda0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.268+0000 7f281ae00640 1 --2- 172.21.15.42:0/3267482065 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28140ce3b0 0x7f281413c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.268+0000 7f281ae00640 1 --2- 172.21.15.42:0/3267482065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2814071de0 0x7f28141533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.268+0000 7f281ae00640 1 -- 172.21.15.42:0/3267482065 >> 172.21.15.42:0/3267482065 conn(0x7f28140b21f0 msgr2=0x7f28140ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.268+0000 7f281ae00640 1 -- 172.21.15.42:0/3267482065 shutdown_connections 2024-05-28T14:00:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.268+0000 7f281ae00640 1 -- 172.21.15.42:0/3267482065 wait complete. 2024-05-28T14:00:50.254 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:00:50.268+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:00:50.254 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:00:50.268+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:00:50.254 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:00:50.268+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:00:50.254 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:00:50.268+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:00:50.254 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:00:50.268+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:00:50.273 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:00:50.288+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:00:50.273 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:00:50.288+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:00:50.273 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:00:50.288+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:00:50.273 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:00:50.288+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:00:50.273 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:00:50.288+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:00:50.277 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:00:50.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91ba000640 1 Processor -- start 2024-05-28T14:00:50.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91ba000640 1 -- start start 2024-05-28T14:00:50.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91ba000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b4156c00 0x7f91b406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:50.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91ba000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f91b415d8d0 0x7f91b40aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:50.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91ba000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91b406f6a0 con 0x7f91b40af500 2024-05-28T14:00:50.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91ba000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f91b406e600 con 0x7f91b415d8d0 2024-05-28T14:00:50.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91ba000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f91b406e8e0 con 0x7f91b4156c00 2024-05-28T14:00:50.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91b3e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f91b40af500 0x7f91b41601a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38898/0 (socket says 172.21.15.42:38898) 2024-05-28T14:00:50.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91b3e00640 1 -- 172.21.15.42:0/3944797307 learned_addr learned my addr 172.21.15.42:0/3944797307 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:00:50.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91b2a00640 1 --2- 172.21.15.42:0/3944797307 >> v2:172.21.15.63:3300/0 conn(0x7f91b415d8d0 0x7f91b40aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:50.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91b3400640 1 --2- 172.21.15.42:0/3944797307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b4156c00 0x7f91b406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:50.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91b2000640 1 -- 172.21.15.42:0/3944797307 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3807897685 0 0) 0x7f91b406f6a0 con 0x7f91b40af500 2024-05-28T14:00:50.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91b2000640 1 -- 172.21.15.42:0/3944797307 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9198003530 con 0x7f91b40af500 2024-05-28T14:00:50.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91b2a00640 1 -- 172.21.15.42:0/3944797307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b4156c00 msgr2=0x7f91b406dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:00:50.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91b2a00640 1 --2- 172.21.15.42:0/3944797307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b4156c00 0x7f91b406dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:50.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91b2a00640 1 -- 172.21.15.42:0/3944797307 >> v1:172.21.15.42:6789/0 conn(0x7f91b40af500 legacy=0x7f91b41601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:50.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91b2a00640 1 -- 172.21.15.42:0/3944797307 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b406ebc0 con 0x7f91b415d8d0 2024-05-28T14:00:50.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.436+0000 7f91b3400640 1 --2- 172.21.15.42:0/3944797307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b4156c00 0x7f91b406dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:00:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.440+0000 7f91b2a00640 1 --2- 172.21.15.42:0/3944797307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91b415d8d0 0x7f91b40aefc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f91a00097c0 tx=0x7f91a001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e95eb1a56a4acbdb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.440+0000 7f91b2000640 1 -- 172.21.15.42:0/3944797307 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a00b0070 con 0x7f91b415d8d0 2024-05-28T14:00:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.440+0000 7f91b2000640 1 -- 172.21.15.42:0/3944797307 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a00ab600 con 0x7f91b415d8d0 2024-05-28T14:00:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.440+0000 7f91b2000640 1 -- 172.21.15.42:0/3944797307 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a00ab9d0 con 0x7f91b415d8d0 2024-05-28T14:00:50.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.440+0000 7f91ba000640 1 -- 172.21.15.42:0/3944797307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91b415d8d0 msgr2=0x7f91b40aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:50.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.440+0000 7f91ba000640 1 --2- 172.21.15.42:0/3944797307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91b415d8d0 0x7f91b40aefc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f91a00097c0 tx=0x7f91a001c410 comp rx=0 tx=0).stop 2024-05-28T14:00:50.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.440+0000 7f91ba000640 1 -- 172.21.15.42:0/3944797307 shutdown_connections 2024-05-28T14:00:50.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.440+0000 7f91ba000640 1 --2- 172.21.15.42:0/3944797307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91b415d8d0 0x7f91b40aefc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:50.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.440+0000 7f91ba000640 1 --2- 172.21.15.42:0/3944797307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b4156c00 0x7f91b406dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:50.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.440+0000 7f91ba000640 1 -- 172.21.15.42:0/3944797307 >> 172.21.15.42:0/3944797307 conn(0x7f91b406bff0 msgr2=0x7f91b40b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:50.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.440+0000 7f91ba000640 1 -- 172.21.15.42:0/3944797307 shutdown_connections 2024-05-28T14:00:50.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.440+0000 7f91ba000640 1 -- 172.21.15.42:0/3944797307 wait complete. 2024-05-28T14:00:50.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91ba000640 1 Processor -- start 2024-05-28T14:00:50.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91ba000640 1 -- start start 2024-05-28T14:00:50.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91ba000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b40af500 0x7f91b41559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:50.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91ba000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91b4156c00 0x7f91b414e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:50.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91ba000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91b415d8d0 0x7f91b41491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:50.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91ba000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f91b406e240 con 0x7f91b415d8d0 2024-05-28T14:00:50.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91ba000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f91b40af910 con 0x7f91b4156c00 2024-05-28T14:00:50.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91ba000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f91b406e3c0 con 0x7f91b40af500 2024-05-28T14:00:50.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91b3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91b415d8d0 0x7f91b41491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:50.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91b3400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b40af500 0x7f91b41559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:50.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91b3400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b40af500 0x7f91b41559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:54698/0 (socket says 172.21.15.42:54698) 2024-05-28T14:00:50.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91b3400640 1 -- 172.21.15.42:0/3899303261 learned_addr learned my addr 172.21.15.42:0/3899303261 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:00:50.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91b3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91b415d8d0 0x7f91b41491f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46744/0 (socket says 172.21.15.42:46744) 2024-05-28T14:00:50.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91b2a00640 1 --2- 172.21.15.42:0/3899303261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91b4156c00 0x7f91b414e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:50.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91b3400640 1 -- 172.21.15.42:0/3899303261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91b4156c00 msgr2=0x7f91b414e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:50.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91b3400640 1 --2- 172.21.15.42:0/3899303261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91b4156c00 0x7f91b414e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:50.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91b3400640 1 -- 172.21.15.42:0/3899303261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91b415d8d0 msgr2=0x7f91b41491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:50.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91b3400640 1 --2- 172.21.15.42:0/3899303261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91b415d8d0 0x7f91b41491f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:50.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91b3400640 1 -- 172.21.15.42:0/3899303261 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b414b630 con 0x7f91b40af500 2024-05-28T14:00:50.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91b3e00640 1 --2- 172.21.15.42:0/3899303261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91b415d8d0 0x7f91b41491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:00:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91b3400640 1 --2- 172.21.15.42:0/3899303261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b40af500 0x7f91b41559a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f91a400efc0 tx=0x7f91a400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91ab400640 1 -- 172.21.15.42:0/3899303261 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4018020 con 0x7f91b40af500 2024-05-28T14:00:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91ab400640 1 -- 172.21.15.42:0/3899303261 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a4014680 con 0x7f91b40af500 2024-05-28T14:00:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91ab400640 1 -- 172.21.15.42:0/3899303261 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4014a50 con 0x7f91b40af500 2024-05-28T14:00:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91ba000640 1 -- 172.21.15.42:0/3899303261 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b41552a0 con 0x7f91b40af500 2024-05-28T14:00:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.444+0000 7f91ba000640 1 -- 172.21.15.42:0/3899303261 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b414abd0 con 0x7f91b40af500 2024-05-28T14:00:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.448+0000 7f91ba000640 1 -- 172.21.15.42:0/3899303261 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b4068a90 con 0x7f91b40af500 2024-05-28T14:00:50.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.448+0000 7f91ab400640 1 -- 172.21.15.42:0/3899303261 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 6) ==== 88038+0+0 (secure 0 0 0) 0x7f91a401b020 con 0x7f91b40af500 2024-05-28T14:00:50.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.448+0000 7f91ab400640 1 --2- 172.21.15.42:0/3899303261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9194068410 0x7f919406a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:50.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.448+0000 7f91ab400640 1 -- 172.21.15.42:0/3899303261 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f91a4087770 con 0x7f91b40af500 2024-05-28T14:00:50.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.448+0000 7f91b2a00640 1 --2- 172.21.15.42:0/3899303261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9194068410 0x7f919406a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:50.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.448+0000 7f91b2a00640 1 --2- 172.21.15.42:0/3899303261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9194068410 0x7f919406a8d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f91a00ad000 tx=0x7f91a00a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:50.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.452+0000 7f91ab400640 1 -- 172.21.15.42:0/3899303261 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f91a4022030 con 0x7f91b40af500 2024-05-28T14:00:50.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.672+0000 7f91ba000640 1 -- 172.21.15.42:0/3899303261 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f91b40b2a40 con 0x7f91b40af500 2024-05-28T14:00:50.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.676+0000 7f91ab400640 1 -- 172.21.15.42:0/3899303261 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1831 (secure 0 0 0) 0x7f91a4010040 con 0x7f91b40af500 2024-05-28T14:00:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:00:50.661 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:00:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:00:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:00:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:00:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:00:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:00:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:00:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:00:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:00:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:00:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:00:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:00:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:00:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:00:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:00:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 171, 2024-05-28T14:00:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:00:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:00:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:00:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:00:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:00:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 34, 2024-05-28T14:00:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:00:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:00:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:00:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:00:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:00:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:00:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:00:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:00:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:00:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:00:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:00:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:00:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:00:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:00:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 2494, 2024-05-28T14:00:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 10356854174, 2024-05-28T14:00:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 20833968128, 2024-05-28T14:00:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 752260145152, 2024-05-28T14:00:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:00:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 129304410, 2024-05-28T14:00:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:00:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 32 2024-05-28T14:00:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:00:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:00:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2, 2024-05-28T14:00:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:00:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:00:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:00:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:00:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:00:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 2, 2024-05-28T14:00:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:00:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:00:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:00:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 54785 2024-05-28T14:00:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:00:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:00:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:00:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:00:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:00:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:00:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:00:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:00:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:00:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:00:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:00:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 8, 2024-05-28T14:00:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:00:32.881524+0000", 2024-05-28T14:00:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:00:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": {} 2024-05-28T14:00:50.668 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:00:50.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.676+0000 7f91ba000640 1 -- 172.21.15.42:0/3899303261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9194068410 msgr2=0x7f919406a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:50.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.676+0000 7f91ba000640 1 --2- 172.21.15.42:0/3899303261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9194068410 0x7f919406a8d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f91a00ad000 tx=0x7f91a00a6000 comp rx=0 tx=0).stop 2024-05-28T14:00:50.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.676+0000 7f91ba000640 1 -- 172.21.15.42:0/3899303261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b40af500 msgr2=0x7f91b41559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:50.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.676+0000 7f91ba000640 1 --2- 172.21.15.42:0/3899303261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b40af500 0x7f91b41559a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f91a400efc0 tx=0x7f91a400c490 comp rx=0 tx=0).stop 2024-05-28T14:00:50.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.676+0000 7f91ba000640 1 -- 172.21.15.42:0/3899303261 shutdown_connections 2024-05-28T14:00:50.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.676+0000 7f91ba000640 1 --2- 172.21.15.42:0/3899303261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9194068410 0x7f919406a8d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:50.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.676+0000 7f91ba000640 1 --2- 172.21.15.42:0/3899303261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91b415d8d0 0x7f91b41491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:50.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.676+0000 7f91ba000640 1 --2- 172.21.15.42:0/3899303261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91b4156c00 0x7f91b414e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:50.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.676+0000 7f91ba000640 1 --2- 172.21.15.42:0/3899303261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b40af500 0x7f91b41559a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:50.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.676+0000 7f91ba000640 1 -- 172.21.15.42:0/3899303261 >> 172.21.15.42:0/3899303261 conn(0x7f91b406bff0 msgr2=0x7f91b40b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:50.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.676+0000 7f91ba000640 1 -- 172.21.15.42:0/3899303261 shutdown_connections 2024-05-28T14:00:50.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:50.676+0000 7f91ba000640 1 -- 172.21.15.42:0/3899303261 wait complete. 2024-05-28T14:00:52.683 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:00:52.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.848+0000 7fee99200640 1 Processor -- start 2024-05-28T14:00:52.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.848+0000 7fee99200640 1 -- start start 2024-05-28T14:00:52.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.848+0000 7fee99200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fee940af6d0 0x7fee94167530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:52.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.848+0000 7fee99200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee9406f6a0 con 0x7fee9415d8d0 2024-05-28T14:00:52.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.848+0000 7fee99200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee9406e600 con 0x7fee94156c00 2024-05-28T14:00:52.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.848+0000 7fee99200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fee9406e8e0 con 0x7fee940af6d0 2024-05-28T14:00:52.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.848+0000 7fee89400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fee9415d8d0 0x7fee940aefc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38906/0 (socket says 172.21.15.42:38906) 2024-05-28T14:00:52.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.848+0000 7fee89400640 1 -- 172.21.15.42:0/484250064 learned_addr learned my addr 172.21.15.42:0/484250064 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:00:52.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.848+0000 7fee92800640 1 --2- 172.21.15.42:0/484250064 >> v2:172.21.15.63:3300/0 conn(0x7fee940af6d0 0x7fee94167530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:52.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.848+0000 7fee91400640 1 -- 172.21.15.42:0/484250064 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 318143687 0 0) 0x7fee9406f6a0 con 0x7fee9415d8d0 2024-05-28T14:00:52.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.848+0000 7fee91400640 1 -- 172.21.15.42:0/484250064 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee78003530 con 0x7fee9415d8d0 2024-05-28T14:00:52.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91400640 1 -- 172.21.15.42:0/484250064 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3103586213 0 0) 0x7fee78003530 con 0x7fee9415d8d0 2024-05-28T14:00:52.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91400640 1 -- 172.21.15.42:0/484250064 >> v2:172.21.15.63:3300/0 conn(0x7fee940af6d0 msgr2=0x7fee94167530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:00:52.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91400640 1 --2- 172.21.15.42:0/484250064 >> v2:172.21.15.63:3300/0 conn(0x7fee940af6d0 0x7fee94167530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:52.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91400640 1 -- 172.21.15.42:0/484250064 >> v1:172.21.15.63:6789/0 conn(0x7fee94156c00 legacy=0x7fee9406dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:52.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91400640 1 -- 172.21.15.42:0/484250064 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9406ebc0 con 0x7fee9415d8d0 2024-05-28T14:00:52.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91400640 1 -- 172.21.15.42:0/484250064 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fee7c001680 con 0x7fee9415d8d0 2024-05-28T14:00:52.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91400640 1 -- 172.21.15.42:0/484250064 >> v1:172.21.15.42:6789/0 conn(0x7fee9415d8d0 legacy=0x7fee940aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:52.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91400640 1 --2- 172.21.15.42:0/484250064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee780039c0 0x7fee78003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:52.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91400640 1 --2- 172.21.15.42:0/484250064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee78004810 0x7fee78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:52.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91400640 1 --2- 172.21.15.42:0/484250064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee780051d0 0x7fee7800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:52.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91400640 1 -- 172.21.15.42:0/484250064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fee9406e8e0 con 0x7fee780039c0 2024-05-28T14:00:52.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91400640 1 -- 172.21.15.42:0/484250064 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee7800c170 con 0x7fee780051d0 2024-05-28T14:00:52.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91400640 1 -- 172.21.15.42:0/484250064 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fee7800c370 con 0x7fee78004810 2024-05-28T14:00:52.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91e00640 1 --2- 172.21.15.42:0/484250064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee780039c0 0x7fee78003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:52.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee89400640 1 --2- 172.21.15.42:0/484250064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee78004810 0x7fee78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:52.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee92800640 1 --2- 172.21.15.42:0/484250064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee780051d0 0x7fee7800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:52.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91e00640 1 -- 172.21.15.42:0/484250064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee78004810 msgr2=0x7fee78004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:00:52.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91e00640 1 --2- 172.21.15.42:0/484250064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee78004810 0x7fee78004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:52.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91e00640 1 -- 172.21.15.42:0/484250064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee780051d0 msgr2=0x7fee7800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:00:52.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91e00640 1 --2- 172.21.15.42:0/484250064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee780051d0 0x7fee7800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:52.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91e00640 1 -- 172.21.15.42:0/484250064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee78003530 con 0x7fee780039c0 2024-05-28T14:00:52.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee92800640 1 -- 172.21.15.42:0/484250064 reap_dead start 2024-05-28T14:00:52.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.852+0000 7fee91e00640 1 --2- 172.21.15.42:0/484250064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee780039c0 0x7fee78003de0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fee84001e70 tx=0x7fee8401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7681b2fca1e1bb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:52.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee91400640 1 -- 172.21.15.42:0/484250064 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840b0070 con 0x7fee780039c0 2024-05-28T14:00:52.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee91400640 1 -- 172.21.15.42:0/484250064 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee840a7070 con 0x7fee780039c0 2024-05-28T14:00:52.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee91400640 1 -- 172.21.15.42:0/484250064 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84001a90 con 0x7fee780039c0 2024-05-28T14:00:52.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 -- 172.21.15.42:0/484250064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee780039c0 msgr2=0x7fee78003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:52.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 --2- 172.21.15.42:0/484250064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee780039c0 0x7fee78003de0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fee84001e70 tx=0x7fee8401c760 comp rx=0 tx=0).stop 2024-05-28T14:00:52.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 -- 172.21.15.42:0/484250064 shutdown_connections 2024-05-28T14:00:52.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 --2- 172.21.15.42:0/484250064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee780039c0 0x7fee78003de0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:52.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 -- 172.21.15.42:0/484250064 >> 172.21.15.42:0/484250064 conn(0x7fee9406bff0 msgr2=0x7fee940b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:52.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 -- 172.21.15.42:0/484250064 shutdown_connections 2024-05-28T14:00:52.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 -- 172.21.15.42:0/484250064 wait complete. 2024-05-28T14:00:52.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 Processor -- start 2024-05-28T14:00:52.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 -- start start 2024-05-28T14:00:52.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee780039c0 0x7fee9414a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:52.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee9415c090 0x7fee94159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:52.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee94159ad0 0x7fee941571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:52.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fee94160320 con 0x7fee780039c0 2024-05-28T14:00:52.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee941601a0 con 0x7fee9415c090 2024-05-28T14:00:52.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee99200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fee941604a0 con 0x7fee94159ad0 2024-05-28T14:00:52.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee91e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee780039c0 0x7fee9414a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:52.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee91e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee780039c0 0x7fee9414a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46776/0 (socket says 172.21.15.42:46776) 2024-05-28T14:00:52.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee91e00640 1 -- 172.21.15.42:0/114269650 learned_addr learned my addr 172.21.15.42:0/114269650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:00:52.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee92800640 1 --2- 172.21.15.42:0/114269650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee94159ad0 0x7fee941571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:52.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.856+0000 7fee89400640 1 --2- 172.21.15.42:0/114269650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee9415c090 0x7fee94159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:52.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee91e00640 1 -- 172.21.15.42:0/114269650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee94159ad0 msgr2=0x7fee941571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:52.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee91e00640 1 --2- 172.21.15.42:0/114269650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee94159ad0 0x7fee941571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:52.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee91e00640 1 -- 172.21.15.42:0/114269650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee9415c090 msgr2=0x7fee94159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:52.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee91e00640 1 --2- 172.21.15.42:0/114269650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee9415c090 0x7fee94159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:52.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee91e00640 1 -- 172.21.15.42:0/114269650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9414af10 con 0x7fee780039c0 2024-05-28T14:00:52.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee91e00640 1 --2- 172.21.15.42:0/114269650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee780039c0 0x7fee9414a9d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fee8401c590 tx=0x7fee78005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:52.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee8aa00640 1 -- 172.21.15.42:0/114269650 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840b9070 con 0x7fee780039c0 2024-05-28T14:00:52.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee8aa00640 1 -- 172.21.15.42:0/114269650 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee84003c40 con 0x7fee780039c0 2024-05-28T14:00:52.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee99200640 1 -- 172.21.15.42:0/114269650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee940b3750 con 0x7fee780039c0 2024-05-28T14:00:52.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee8aa00640 1 -- 172.21.15.42:0/114269650 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840b0040 con 0x7fee780039c0 2024-05-28T14:00:52.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee99200640 1 -- 172.21.15.42:0/114269650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee940b3d10 con 0x7fee780039c0 2024-05-28T14:00:52.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee99200640 1 -- 172.21.15.42:0/114269650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee94068a90 con 0x7fee780039c0 2024-05-28T14:00:52.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee8aa00640 1 -- 172.21.15.42:0/114269650 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7fee840050d0 con 0x7fee780039c0 2024-05-28T14:00:52.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee8aa00640 1 --2- 172.21.15.42:0/114269650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fee70068530 0x7fee7006a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:52.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee8aa00640 1 -- 172.21.15.42:0/114269650 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fee84120a90 con 0x7fee780039c0 2024-05-28T14:00:52.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.860+0000 7fee89400640 1 --2- 172.21.15.42:0/114269650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fee70068530 0x7fee7006a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:52.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.864+0000 7fee89400640 1 --2- 172.21.15.42:0/114269650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fee70068530 0x7fee7006a9f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fee94073d10 tx=0x7fee7c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:52.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:52.864+0000 7fee8aa00640 1 -- 172.21.15.42:0/114269650 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fee84016470 con 0x7fee780039c0 2024-05-28T14:00:53.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:53.088+0000 7fee99200640 1 -- 172.21.15.42:0/114269650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fee940b2a40 con 0x7fee780039c0 2024-05-28T14:00:53.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:53.088+0000 7fee8aa00640 1 -- 172.21.15.42:0/114269650 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1831 (secure 0 0 0) 0x7fee840f0660 con 0x7fee780039c0 2024-05-28T14:00:53.075 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:00:53.075 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:00:53.075 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:00:53.075 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:00:53.075 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:00:53.075 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:00:53.075 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:00:53.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:53.075 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:00:53.075 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:00:53.076 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:00:53.076 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:00:53.076 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:00:53.076 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:53.076 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:00:53.076 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:00:53.076 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:00:53.076 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:00:53.076 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:53.076 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 174, 2024-05-28T14:00:53.076 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:00:53.076 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:00:53.076 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:00:53.076 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:00:53.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:53.077 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:00:53.077 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 34, 2024-05-28T14:00:53.077 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:00:53.077 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:00:53.077 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:00:53.077 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:00:53.077 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:00:53.077 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:00:53.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:53.077 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:00:53.077 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:00:53.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:00:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:00:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:00:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:00:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:00:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:00:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 2585, 2024-05-28T14:00:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 10738546674, 2024-05-28T14:00:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 21744156672, 2024-05-28T14:00:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 751349956608, 2024-05-28T14:00:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:00:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 161103949, 2024-05-28T14:00:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:00:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 40 2024-05-28T14:00:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:00:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:00:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2, 2024-05-28T14:00:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:00:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:00:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:00:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:00:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:00:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 2, 2024-05-28T14:00:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:00:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:00:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:00:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 54785 2024-05-28T14:00:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:00:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:00:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:00:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:00:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:00:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:00:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:00:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:00:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:00:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:00:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:00:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 8, 2024-05-28T14:00:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:00:32.881524+0000", 2024-05-28T14:00:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:00:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": {} 2024-05-28T14:00:53.081 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:00:53.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:53.092+0000 7fee99200640 1 -- 172.21.15.42:0/114269650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fee70068530 msgr2=0x7fee7006a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:53.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:53.092+0000 7fee99200640 1 --2- 172.21.15.42:0/114269650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fee70068530 0x7fee7006a9f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fee94073d10 tx=0x7fee7c009040 comp rx=0 tx=0).stop 2024-05-28T14:00:53.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:53.092+0000 7fee99200640 1 -- 172.21.15.42:0/114269650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee780039c0 msgr2=0x7fee9414a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:53.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:53.092+0000 7fee99200640 1 --2- 172.21.15.42:0/114269650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee780039c0 0x7fee9414a9d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fee8401c590 tx=0x7fee78005840 comp rx=0 tx=0).stop 2024-05-28T14:00:53.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:53.092+0000 7fee99200640 1 -- 172.21.15.42:0/114269650 shutdown_connections 2024-05-28T14:00:53.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:53.092+0000 7fee99200640 1 --2- 172.21.15.42:0/114269650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fee70068530 0x7fee7006a9f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:53.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:53.092+0000 7fee99200640 1 --2- 172.21.15.42:0/114269650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee94159ad0 0x7fee941571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:53.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:53.092+0000 7fee99200640 1 --2- 172.21.15.42:0/114269650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee9415c090 0x7fee94159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:53.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:53.092+0000 7fee99200640 1 --2- 172.21.15.42:0/114269650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee780039c0 0x7fee9414a9d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:53.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:53.092+0000 7fee99200640 1 -- 172.21.15.42:0/114269650 >> 172.21.15.42:0/114269650 conn(0x7fee9406bff0 msgr2=0x7fee940afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:53.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:53.092+0000 7fee99200640 1 -- 172.21.15.42:0/114269650 shutdown_connections 2024-05-28T14:00:53.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:53.092+0000 7fee99200640 1 -- 172.21.15.42:0/114269650 wait complete. 2024-05-28T14:00:55.097 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:00:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f386d600640 1 Processor -- start 2024-05-28T14:00:55.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f386d600640 1 -- start start 2024-05-28T14:00:55.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f386d600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38680cf6a0 0x7f38681609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:55.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f386d600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f386815d170 con 0x7f386815ca70 2024-05-28T14:00:55.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f386d600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f386815d450 con 0x7f386815a160 2024-05-28T14:00:55.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f386d600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3868160f10 con 0x7f38680cf6a0 2024-05-28T14:00:55.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f3867e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f386815a160 0x7f38680cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:41034/0 (socket says 172.21.15.42:41034) 2024-05-28T14:00:55.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f3867e00640 1 -- 172.21.15.42:0/2274316588 learned_addr learned my addr 172.21.15.42:0/2274316588 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:00:55.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f386cc00640 1 --2- 172.21.15.42:0/2274316588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38680cf6a0 0x7f38681609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:55.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f3866a00640 1 -- 172.21.15.42:0/2274316588 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1639728665 0 0) 0x7f386815d450 con 0x7f386815a160 2024-05-28T14:00:55.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f3866a00640 1 -- 172.21.15.42:0/2274316588 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f384c003530 con 0x7f386815a160 2024-05-28T14:00:55.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f3866a00640 1 -- 172.21.15.42:0/2274316588 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4269207023 0 0) 0x7f386815d170 con 0x7f386815ca70 2024-05-28T14:00:55.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f386cc00640 1 -- 172.21.15.42:0/2274316588 >> v1:172.21.15.63:6789/0 conn(0x7f386815a160 legacy=0x7f38680cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:55.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f386cc00640 1 -- 172.21.15.42:0/2274316588 >> v1:172.21.15.42:6789/0 conn(0x7f386815ca70 legacy=0x7f38680cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:55.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.360+0000 7f386cc00640 1 -- 172.21.15.42:0/2274316588 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386815d730 con 0x7f38680cf6a0 2024-05-28T14:00:55.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.364+0000 7f386cc00640 1 --2- 172.21.15.42:0/2274316588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38680cf6a0 0x7f38681609d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3860009dc0 tx=0x7f386001c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea15a398bcfc15d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:55.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.364+0000 7f3866a00640 1 -- 172.21.15.42:0/2274316588 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38600b2070 con 0x7f38680cf6a0 2024-05-28T14:00:55.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.364+0000 7f3866a00640 1 -- 172.21.15.42:0/2274316588 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38600a5850 con 0x7f38680cf6a0 2024-05-28T14:00:55.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.364+0000 7f3866a00640 1 -- 172.21.15.42:0/2274316588 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38600a5c20 con 0x7f38680cf6a0 2024-05-28T14:00:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.364+0000 7f386d600640 1 -- 172.21.15.42:0/2274316588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38680cf6a0 msgr2=0x7f38681609d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.364+0000 7f386d600640 1 --2- 172.21.15.42:0/2274316588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38680cf6a0 0x7f38681609d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3860009dc0 tx=0x7f386001c990 comp rx=0 tx=0).stop 2024-05-28T14:00:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.368+0000 7f386d600640 1 -- 172.21.15.42:0/2274316588 shutdown_connections 2024-05-28T14:00:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.368+0000 7f386d600640 1 --2- 172.21.15.42:0/2274316588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38680cf6a0 0x7f38681609d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.368+0000 7f386d600640 1 -- 172.21.15.42:0/2274316588 >> 172.21.15.42:0/2274316588 conn(0x7f38680b3ec0 msgr2=0x7f38680b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.368+0000 7f386d600640 1 -- 172.21.15.42:0/2274316588 shutdown_connections 2024-05-28T14:00:55.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.368+0000 7f386d600640 1 -- 172.21.15.42:0/2274316588 wait complete. 2024-05-28T14:00:55.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.368+0000 7f386d600640 1 Processor -- start 2024-05-28T14:00:55.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.368+0000 7f386d600640 1 -- start start 2024-05-28T14:00:55.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f386d600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38680cf6a0 0x7f386815f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:55.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f386d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f386815a160 0x7f38681543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:55.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f386d600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386815ca70 0x7f3868154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:55.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f386d600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f386816a5b0 con 0x7f386815a160 2024-05-28T14:00:55.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f386d600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f386815cee0 con 0x7f386815ca70 2024-05-28T14:00:55.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f386d600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f386816a730 con 0x7f38680cf6a0 2024-05-28T14:00:55.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f3867400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f386815a160 0x7f38681543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:55.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f3867400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f386815a160 0x7f38681543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46784/0 (socket says 172.21.15.42:46784) 2024-05-28T14:00:55.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f3867400640 1 -- 172.21.15.42:0/4160943048 learned_addr learned my addr 172.21.15.42:0/4160943048 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:00:55.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f386cc00640 1 --2- 172.21.15.42:0/4160943048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386815ca70 0x7f3868154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:55.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f386cc00640 1 -- 172.21.15.42:0/4160943048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38680cf6a0 msgr2=0x7f386815f1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T14:00:55.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f386cc00640 1 --2- 172.21.15.42:0/4160943048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38680cf6a0 0x7f386815f1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:55.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f386cc00640 1 -- 172.21.15.42:0/4160943048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f386815a160 msgr2=0x7f38681543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:55.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f386cc00640 1 --2- 172.21.15.42:0/4160943048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f386815a160 0x7f38681543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:55.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f386cc00640 1 -- 172.21.15.42:0/4160943048 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38681671b0 con 0x7f386815ca70 2024-05-28T14:00:55.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.372+0000 7f386cc00640 1 --2- 172.21.15.42:0/4160943048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386815ca70 0x7f3868154930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f38600ac000 tx=0x7f3860019730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:55.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.376+0000 7f385fe00640 1 -- 172.21.15.42:0/4160943048 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38600be070 con 0x7f386815ca70 2024-05-28T14:00:55.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.376+0000 7f385fe00640 1 -- 172.21.15.42:0/4160943048 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3860003ba0 con 0x7f386815ca70 2024-05-28T14:00:55.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.376+0000 7f385fe00640 1 -- 172.21.15.42:0/4160943048 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38600b2040 con 0x7f386815ca70 2024-05-28T14:00:55.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.376+0000 7f386d600640 1 -- 172.21.15.42:0/4160943048 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3868167490 con 0x7f386815ca70 2024-05-28T14:00:55.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.376+0000 7f386d600640 1 -- 172.21.15.42:0/4160943048 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3868167a50 con 0x7f386815ca70 2024-05-28T14:00:55.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.376+0000 7f386d600640 1 -- 172.21.15.42:0/4160943048 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3868068a90 con 0x7f386815ca70 2024-05-28T14:00:55.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.380+0000 7f385fe00640 1 -- 172.21.15.42:0/4160943048 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7f38600a5710 con 0x7f386815ca70 2024-05-28T14:00:55.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.380+0000 7f385fe00640 1 --2- 172.21.15.42:0/4160943048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3848070e10 0x7f38480732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:55.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.380+0000 7f385fe00640 1 -- 172.21.15.42:0/4160943048 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f3860121060 con 0x7f386815ca70 2024-05-28T14:00:55.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.380+0000 7f3867e00640 1 --2- 172.21.15.42:0/4160943048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3848070e10 0x7f38480732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:55.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.388+0000 7f3867e00640 1 --2- 172.21.15.42:0/4160943048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3848070e10 0x7f38480732d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3858007fd0 tx=0x7f3858007900 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:55.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.388+0000 7f385fe00640 1 -- 172.21.15.42:0/4160943048 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3860016580 con 0x7f386815ca70 2024-05-28T14:00:55.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.836+0000 7f386d600640 1 -- 172.21.15.42:0/4160943048 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3868071030 con 0x7f386815ca70 2024-05-28T14:00:55.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.836+0000 7f385fe00640 1 -- 172.21.15.42:0/4160943048 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2096 (secure 0 0 0) 0x7f38600f1380 con 0x7f386815ca70 2024-05-28T14:00:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:00:55.823 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:00:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:00:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:00:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:00:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:00:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:00:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:00:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:00:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:00:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:00:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:00:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:00:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:00:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:00:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:00:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 177, 2024-05-28T14:00:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:00:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 34, 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:00:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:00:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:00:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:00:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:00:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:00:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:00:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 2589, 2024-05-28T14:00:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 10738608726, 2024-05-28T14:00:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 21744304128, 2024-05-28T14:00:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 751349809152, 2024-05-28T14:00:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:00:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 170, 2024-05-28T14:00:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 95411240, 2024-05-28T14:00:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:00:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 26 2024-05-28T14:00:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:00:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:00:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2, 2024-05-28T14:00:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:00:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:00:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:00:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:00:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:00:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 2, 2024-05-28T14:00:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:00:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:00:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:00:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 54785 2024-05-28T14:00:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:00:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 8, 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:00:32.881524+0000", 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:00:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:00:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: "2634ebf7-b822-4613-b0f8-227d147fb4c4": { 2024-05-28T14:00:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "1 ongoing clones; Avg progress is 0.0% (0s)\n [............................] ", 2024-05-28T14:00:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0, 2024-05-28T14:00:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:00:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:00:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:00:55.832 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:00:55.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.840+0000 7f386d600640 1 -- 172.21.15.42:0/4160943048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3848070e10 msgr2=0x7f38480732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:55.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.840+0000 7f386d600640 1 --2- 172.21.15.42:0/4160943048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3848070e10 0x7f38480732d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3858007fd0 tx=0x7f3858007900 comp rx=0 tx=0).stop 2024-05-28T14:00:55.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.840+0000 7f386d600640 1 -- 172.21.15.42:0/4160943048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386815ca70 msgr2=0x7f3868154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:55.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.840+0000 7f386d600640 1 --2- 172.21.15.42:0/4160943048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386815ca70 0x7f3868154930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f38600ac000 tx=0x7f3860019730 comp rx=0 tx=0).stop 2024-05-28T14:00:55.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.840+0000 7f386d600640 1 -- 172.21.15.42:0/4160943048 shutdown_connections 2024-05-28T14:00:55.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.840+0000 7f386d600640 1 --2- 172.21.15.42:0/4160943048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3848070e10 0x7f38480732d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:55.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.840+0000 7f386d600640 1 --2- 172.21.15.42:0/4160943048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386815ca70 0x7f3868154930 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:55.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.840+0000 7f386d600640 1 --2- 172.21.15.42:0/4160943048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f386815a160 0x7f38681543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:55.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.840+0000 7f386d600640 1 --2- 172.21.15.42:0/4160943048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38680cf6a0 0x7f386815f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:55.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.840+0000 7f386d600640 1 -- 172.21.15.42:0/4160943048 >> 172.21.15.42:0/4160943048 conn(0x7f38680b3ec0 msgr2=0x7f38680cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:55.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.840+0000 7f386d600640 1 -- 172.21.15.42:0/4160943048 shutdown_connections 2024-05-28T14:00:55.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:55.840+0000 7f386d600640 1 -- 172.21.15.42:0/4160943048 wait complete. 2024-05-28T14:00:55.889 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-05-28T14:00:56.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.196+0000 7fad0ea00640 1 Processor -- start 2024-05-28T14:00:56.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.196+0000 7fad0ea00640 1 -- start start 2024-05-28T14:00:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.200+0000 7fad0ea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fad08071080 0x7fad0809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.200+0000 7fad0ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fad0809af90 0x7fad081687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.200+0000 7fad0ea00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad0806ffe0 con 0x7fad0815e800 2024-05-28T14:00:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.200+0000 7fad0ea00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fad0809ba90 con 0x7fad0809af90 2024-05-28T14:00:56.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.200+0000 7fad0ea00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fad0809b3e0 con 0x7fad08071080 2024-05-28T14:00:56.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.204+0000 7fad07e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fad0815e800 0x7fad080d12c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:41038/0 (socket says 172.21.15.42:41038) 2024-05-28T14:00:56.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.204+0000 7fad07e00640 1 -- 172.21.15.42:0/2888361070 learned_addr learned my addr 172.21.15.42:0/2888361070 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:00:56.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.204+0000 7fad0ca00640 1 --2- 172.21.15.42:0/2888361070 >> v2:172.21.15.42:3300/0 conn(0x7fad0809af90 0x7fad081687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:56.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.204+0000 7fad07400640 1 --2- 172.21.15.42:0/2888361070 >> v2:172.21.15.63:3300/0 conn(0x7fad08071080 0x7fad0809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:56.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.204+0000 7fad0ca00640 1 -- 172.21.15.42:0/2888361070 >> v2:172.21.15.63:3300/0 conn(0x7fad08071080 msgr2=0x7fad0809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:00:56.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.204+0000 7fad0ca00640 1 --2- 172.21.15.42:0/2888361070 >> v2:172.21.15.63:3300/0 conn(0x7fad08071080 0x7fad0809aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:56.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.204+0000 7fad0ca00640 1 -- 172.21.15.42:0/2888361070 >> v1:172.21.15.63:6789/0 conn(0x7fad0815e800 legacy=0x7fad080d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:56.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.204+0000 7fad0ca00640 1 -- 172.21.15.42:0/2888361070 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad080702c0 con 0x7fad0809af90 2024-05-28T14:00:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.208+0000 7fad0ca00640 1 --2- 172.21.15.42:0/2888361070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad0809af90 0x7fad081687a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fad08075560 tx=0x7fad0001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4872069b4a810f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.208+0000 7fad06a00640 1 -- 172.21.15.42:0/2888361070 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad000b2070 con 0x7fad0809af90 2024-05-28T14:00:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.208+0000 7fad06a00640 1 -- 172.21.15.42:0/2888361070 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad000a5850 con 0x7fad0809af90 2024-05-28T14:00:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.208+0000 7fad06a00640 1 -- 172.21.15.42:0/2888361070 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad000a5c20 con 0x7fad0809af90 2024-05-28T14:00:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.208+0000 7fad0ea00640 1 -- 172.21.15.42:0/2888361070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad0809af90 msgr2=0x7fad081687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.208+0000 7fad0ea00640 1 --2- 172.21.15.42:0/2888361070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad0809af90 0x7fad081687a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fad08075560 tx=0x7fad0001c8a0 comp rx=0 tx=0).stop 2024-05-28T14:00:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.212+0000 7fad0ea00640 1 -- 172.21.15.42:0/2888361070 shutdown_connections 2024-05-28T14:00:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.212+0000 7fad0ea00640 1 --2- 172.21.15.42:0/2888361070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad0809af90 0x7fad081687a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.212+0000 7fad0ea00640 1 --2- 172.21.15.42:0/2888361070 >> v2:172.21.15.63:3300/0 conn(0x7fad08071080 0x7fad0809aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:56.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.212+0000 7fad0ea00640 1 -- 172.21.15.42:0/2888361070 >> 172.21.15.42:0/2888361070 conn(0x7fad080b4060 msgr2=0x7fad080b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.212+0000 7fad0ea00640 1 -- 172.21.15.42:0/2888361070 shutdown_connections 2024-05-28T14:00:56.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.216+0000 7fad0ea00640 1 -- 172.21.15.42:0/2888361070 wait complete. 2024-05-28T14:00:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.216+0000 7fad0ea00640 1 Processor -- start 2024-05-28T14:00:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.216+0000 7fad0ea00640 1 -- start start 2024-05-28T14:00:56.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad0ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad08071080 0x7fad0815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:56.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad0ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad0809af90 0x7fad0815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:56.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad0ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad0815e800 0x7fad08154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:56.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad0ea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fad0815ed60 con 0x7fad0809af90 2024-05-28T14:00:56.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad0ea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fad0815ebe0 con 0x7fad08071080 2024-05-28T14:00:56.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad0ea00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fad081694f0 con 0x7fad0815e800 2024-05-28T14:00:56.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad07e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad08071080 0x7fad0815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:56.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad07400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad0809af90 0x7fad0815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:56.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad07400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad0809af90 0x7fad0815d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46802/0 (socket says 172.21.15.42:46802) 2024-05-28T14:00:56.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad07400640 1 -- 172.21.15.42:0/3438727733 learned_addr learned my addr 172.21.15.42:0/3438727733 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:00:56.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad0ca00640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad0815e800 0x7fad08154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad07e00640 1 -- 172.21.15.42:0/3438727733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad0815e800 msgr2=0x7fad08154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad07e00640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad0815e800 0x7fad08154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad07e00640 1 -- 172.21.15.42:0/3438727733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad0809af90 msgr2=0x7fad0815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad07e00640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad0809af90 0x7fad0815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.220+0000 7fad07e00640 1 -- 172.21.15.42:0/3438727733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad080b3740 con 0x7fad08071080 2024-05-28T14:00:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.224+0000 7fad07400640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad0809af90 0x7fad0815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:00:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.224+0000 7fad0ca00640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad0815e800 0x7fad08154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:00:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.224+0000 7fad07e00640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad08071080 0x7fad0815cf90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7facf800c3c0 tx=0x7facf800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.224+0000 7facffe00640 1 -- 172.21.15.42:0/3438727733 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8005cf0 con 0x7fad08071080 2024-05-28T14:00:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.224+0000 7facffe00640 1 -- 172.21.15.42:0/3438727733 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf80047f0 con 0x7fad08071080 2024-05-28T14:00:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.224+0000 7facffe00640 1 -- 172.21.15.42:0/3438727733 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8004be0 con 0x7fad08071080 2024-05-28T14:00:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.224+0000 7fad0ea00640 1 -- 172.21.15.42:0/3438727733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0809fe90 con 0x7fad08071080 2024-05-28T14:00:56.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.224+0000 7fad0ea00640 1 -- 172.21.15.42:0/3438727733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0809fb40 con 0x7fad08071080 2024-05-28T14:00:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.228+0000 7fad0ea00640 1 -- 172.21.15.42:0/3438727733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad080a05e0 con 0x7fad08071080 2024-05-28T14:00:56.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.228+0000 7facffe00640 1 -- 172.21.15.42:0/3438727733 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7facf801a020 con 0x7fad08071080 2024-05-28T14:00:56.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.232+0000 7facffe00640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7face8068530 0x7face806a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:56.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.232+0000 7facffe00640 1 -- 172.21.15.42:0/3438727733 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7facf8086a40 con 0x7fad08071080 2024-05-28T14:00:56.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.232+0000 7fad07400640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7face8068530 0x7face806a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:56.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.232+0000 7fad07400640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7face8068530 0x7face806a9f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fad08169ad0 tx=0x7facf4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:56.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.240+0000 7facffe00640 1 -- 172.21.15.42:0/3438727733 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7facf8021070 con 0x7fad08071080 2024-05-28T14:00:56.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.660+0000 7fad0ea00640 1 -- 172.21.15.42:0/3438727733 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fad0804fdd0 con 0x7face8068530 2024-05-28T14:00:56.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.772+0000 7facffe00640 1 -- 172.21.15.42:0/3438727733 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fad0804fdd0 con 0x7face8068530 2024-05-28T14:00:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.772+0000 7fad0ea00640 1 -- 172.21.15.42:0/3438727733 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7face8068530 msgr2=0x7face806a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.772+0000 7fad0ea00640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7face8068530 0x7face806a9f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fad08169ad0 tx=0x7facf4008040 comp rx=0 tx=0).stop 2024-05-28T14:00:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.772+0000 7fad0ea00640 1 -- 172.21.15.42:0/3438727733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad08071080 msgr2=0x7fad0815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.772+0000 7fad0ea00640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad08071080 0x7fad0815cf90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7facf800c3c0 tx=0x7facf800c880 comp rx=0 tx=0).stop 2024-05-28T14:00:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.772+0000 7fad0ea00640 1 -- 172.21.15.42:0/3438727733 shutdown_connections 2024-05-28T14:00:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.772+0000 7fad0ea00640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7face8068530 0x7face806a9f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.772+0000 7fad0ea00640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad0815e800 0x7fad08154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.772+0000 7fad0ea00640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad0809af90 0x7fad0815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.772+0000 7fad0ea00640 1 --2- 172.21.15.42:0/3438727733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad08071080 0x7fad0815cf90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.772+0000 7fad0ea00640 1 -- 172.21.15.42:0/3438727733 >> 172.21.15.42:0/3438727733 conn(0x7fad080b4060 msgr2=0x7fad081596e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:56.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.772+0000 7fad0ea00640 1 -- 172.21.15.42:0/3438727733 shutdown_connections 2024-05-28T14:00:56.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.772+0000 7fad0ea00640 1 -- 172.21.15.42:0/3438727733 wait complete. 2024-05-28T14:00:56.778 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:00:56.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.932+0000 7fc002400640 1 Processor -- start 2024-05-28T14:00:56.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.932+0000 7fc002400640 1 -- start start 2024-05-28T14:00:56.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.936+0000 7fc002400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbffc0b4e30 0x7fbffc0b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:56.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.936+0000 7fc002400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbffc06dce0 con 0x7fbffc166310 2024-05-28T14:00:56.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.936+0000 7fc002400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbffc06dfc0 con 0x7fbffc06f210 2024-05-28T14:00:56.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.936+0000 7fc002400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbffc06e2a0 con 0x7fbffc0b4e30 2024-05-28T14:00:56.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.936+0000 7fbffbe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fbffc166310 0x7fbffc168700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38934/0 (socket says 172.21.15.42:38934) 2024-05-28T14:00:56.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.936+0000 7fbffbe00640 1 -- 172.21.15.42:0/51775669 learned_addr learned my addr 172.21.15.42:0/51775669 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:00:56.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.936+0000 7fbff2a00640 1 --2- 172.21.15.42:0/51775669 >> v2:172.21.15.63:3300/0 conn(0x7fbffc0b4e30 0x7fbffc0b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:56.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.936+0000 7fbffaa00640 1 -- 172.21.15.42:0/51775669 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 53975575 0 0) 0x7fbffc06dce0 con 0x7fbffc166310 2024-05-28T14:00:56.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.936+0000 7fbffaa00640 1 -- 172.21.15.42:0/51775669 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfe0003530 con 0x7fbffc166310 2024-05-28T14:00:56.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.936+0000 7fbffaa00640 1 -- 172.21.15.42:0/51775669 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 917268298 0 0) 0x7fbffc06dfc0 con 0x7fbffc06f210 2024-05-28T14:00:56.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.936+0000 7fbffaa00640 1 -- 172.21.15.42:0/51775669 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbffc06dce0 con 0x7fbffc06f210 2024-05-28T14:00:56.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.936+0000 7fbff2a00640 1 -- 172.21.15.42:0/51775669 >> v1:172.21.15.63:6789/0 conn(0x7fbffc06f210 legacy=0x7fbffc06f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:56.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.936+0000 7fbff2a00640 1 -- 172.21.15.42:0/51775669 >> v1:172.21.15.42:6789/0 conn(0x7fbffc166310 legacy=0x7fbffc168700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:56.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.936+0000 7fbff2a00640 1 -- 172.21.15.42:0/51775669 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc06e580 con 0x7fbffc0b4e30 2024-05-28T14:00:56.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.940+0000 7fbff2a00640 1 --2- 172.21.15.42:0/51775669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbffc0b4e30 0x7fbffc0b52c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40097f0 tx=0x7fbfe401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16d622802d8e13b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:56.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.940+0000 7fbffaa00640 1 -- 172.21.15.42:0/51775669 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe40b1070 con 0x7fbffc0b4e30 2024-05-28T14:00:56.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.940+0000 7fbffaa00640 1 -- 172.21.15.42:0/51775669 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe40ac5f0 con 0x7fbffc0b4e30 2024-05-28T14:00:56.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.940+0000 7fbffaa00640 1 -- 172.21.15.42:0/51775669 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe40ac9c0 con 0x7fbffc0b4e30 2024-05-28T14:00:56.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.940+0000 7fc002400640 1 -- 172.21.15.42:0/51775669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbffc0b4e30 msgr2=0x7fbffc0b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:56.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.940+0000 7fc002400640 1 --2- 172.21.15.42:0/51775669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbffc0b4e30 0x7fbffc0b52c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40097f0 tx=0x7fbfe401c470 comp rx=0 tx=0).stop 2024-05-28T14:00:56.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.940+0000 7fc002400640 1 -- 172.21.15.42:0/51775669 shutdown_connections 2024-05-28T14:00:56.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.940+0000 7fc002400640 1 --2- 172.21.15.42:0/51775669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbffc0b4e30 0x7fbffc0b52c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:56.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.940+0000 7fc002400640 1 -- 172.21.15.42:0/51775669 >> 172.21.15.42:0/51775669 conn(0x7fbffc06b9f0 msgr2=0x7fbffc06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:56.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.940+0000 7fc002400640 1 -- 172.21.15.42:0/51775669 shutdown_connections 2024-05-28T14:00:56.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.940+0000 7fc002400640 1 -- 172.21.15.42:0/51775669 wait complete. 2024-05-28T14:00:56.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.940+0000 7fc002400640 1 Processor -- start 2024-05-28T14:00:56.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.940+0000 7fc002400640 1 -- start start 2024-05-28T14:00:56.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fc002400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbffc06f210 0x7fbffc15e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:56.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fc002400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbffc0b4e30 0x7fbffc160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:56.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fc002400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbffc166310 0x7fbffc160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:56.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fc002400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc0afdd0 con 0x7fbffc166310 2024-05-28T14:00:56.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fc002400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc0afc50 con 0x7fbffc06f210 2024-05-28T14:00:56.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fc002400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbffc16c640 con 0x7fbffc0b4e30 2024-05-28T14:00:56.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fbffbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbffc166310 0x7fbffc160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:56.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fbffbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbffc166310 0x7fbffc160840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46818/0 (socket says 172.21.15.42:46818) 2024-05-28T14:00:56.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fbffbe00640 1 -- 172.21.15.42:0/3270567047 learned_addr learned my addr 172.21.15.42:0/3270567047 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:00:56.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fbffb400640 1 --2- 172.21.15.42:0/3270567047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbffc06f210 0x7fbffc15e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:56.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fbff2a00640 1 --2- 172.21.15.42:0/3270567047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbffc0b4e30 0x7fbffc160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:56.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fbffbe00640 1 -- 172.21.15.42:0/3270567047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbffc0b4e30 msgr2=0x7fbffc160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:56.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fbffbe00640 1 --2- 172.21.15.42:0/3270567047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbffc0b4e30 0x7fbffc160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:56.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fbffbe00640 1 -- 172.21.15.42:0/3270567047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbffc06f210 msgr2=0x7fbffc15e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:56.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fbffbe00640 1 --2- 172.21.15.42:0/3270567047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbffc06f210 0x7fbffc15e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:56.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fbffbe00640 1 -- 172.21.15.42:0/3270567047 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc1f7ab0 con 0x7fbffc166310 2024-05-28T14:00:56.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fbffbe00640 1 --2- 172.21.15.42:0/3270567047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbffc166310 0x7fbffc160840 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbff400ca80 tx=0x7fbff400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:56.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fbff3e00640 1 -- 172.21.15.42:0/3270567047 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4017070 con 0x7fbffc166310 2024-05-28T14:00:56.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fbff3e00640 1 -- 172.21.15.42:0/3270567047 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff400ac30 con 0x7fbffc166310 2024-05-28T14:00:56.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fbff3e00640 1 -- 172.21.15.42:0/3270567047 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4012400 con 0x7fbffc166310 2024-05-28T14:00:56.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fc002400640 1 -- 172.21.15.42:0/3270567047 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc160e70 con 0x7fbffc166310 2024-05-28T14:00:56.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fc002400640 1 -- 172.21.15.42:0/3270567047 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc161430 con 0x7fbffc166310 2024-05-28T14:00:56.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.944+0000 7fc002400640 1 -- 172.21.15.42:0/3270567047 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbffc073890 con 0x7fbffc166310 2024-05-28T14:00:56.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.948+0000 7fbff3e00640 1 -- 172.21.15.42:0/3270567047 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7fbff401a070 con 0x7fbffc166310 2024-05-28T14:00:56.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.948+0000 7fbff3e00640 1 --2- 172.21.15.42:0/3270567047 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfd8068530 0x7fbfd806a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:56.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.948+0000 7fbff3e00640 1 -- 172.21.15.42:0/3270567047 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fbff40869d0 con 0x7fbffc166310 2024-05-28T14:00:56.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.948+0000 7fbffb400640 1 --2- 172.21.15.42:0/3270567047 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfd8068530 0x7fbfd806a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:56.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.948+0000 7fbffb400640 1 --2- 172.21.15.42:0/3270567047 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfd8068530 0x7fbfd806a9f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0017c0 tx=0x7fbfec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:56.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:56.952+0000 7fbff3e00640 1 -- 172.21.15.42:0/3270567047 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbff4056640 con 0x7fbffc166310 2024-05-28T14:00:57.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.140+0000 7fc002400640 1 -- 172.21.15.42:0/3270567047 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fbffc04fdd0 con 0x7fbfd8068530 2024-05-28T14:00:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.144+0000 7fbff3e00640 1 -- 172.21.15.42:0/3270567047 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fbffc04fdd0 con 0x7fbfd8068530 2024-05-28T14:00:57.129 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:00:57.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:00:57.129 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:00:57.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:00:57.129 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:00:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.144+0000 7fc002400640 1 -- 172.21.15.42:0/3270567047 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfd8068530 msgr2=0x7fbfd806a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.144+0000 7fc002400640 1 --2- 172.21.15.42:0/3270567047 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfd8068530 0x7fbfd806a9f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0017c0 tx=0x7fbfec009040 comp rx=0 tx=0).stop 2024-05-28T14:00:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.144+0000 7fc002400640 1 -- 172.21.15.42:0/3270567047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbffc166310 msgr2=0x7fbffc160840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.144+0000 7fc002400640 1 --2- 172.21.15.42:0/3270567047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbffc166310 0x7fbffc160840 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbff400ca80 tx=0x7fbff400cf40 comp rx=0 tx=0).stop 2024-05-28T14:00:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.144+0000 7fc002400640 1 -- 172.21.15.42:0/3270567047 shutdown_connections 2024-05-28T14:00:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.144+0000 7fc002400640 1 --2- 172.21.15.42:0/3270567047 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfd8068530 0x7fbfd806a9f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.144+0000 7fc002400640 1 --2- 172.21.15.42:0/3270567047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbffc166310 0x7fbffc160840 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.144+0000 7fc002400640 1 --2- 172.21.15.42:0/3270567047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbffc0b4e30 0x7fbffc160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.144+0000 7fc002400640 1 --2- 172.21.15.42:0/3270567047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbffc06f210 0x7fbffc15e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.144+0000 7fc002400640 1 -- 172.21.15.42:0/3270567047 >> 172.21.15.42:0/3270567047 conn(0x7fbffc06b9f0 msgr2=0x7fbffc146560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.144+0000 7fc002400640 1 -- 172.21.15.42:0/3270567047 shutdown_connections 2024-05-28T14:00:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.144+0000 7fc002400640 1 -- 172.21.15.42:0/3270567047 wait complete. 2024-05-28T14:00:57.150 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:00:57.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8efc600640 1 Processor -- start 2024-05-28T14:00:57.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8efc600640 1 -- start start 2024-05-28T14:00:57.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8efc600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef40ce610 0x7f8ef409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:57.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8efc600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8ef40b4ab0 0x7f8ef409af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:57.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8efc600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef40d0750 con 0x7f8ef409b4d0 2024-05-28T14:00:57.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8efc600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8ef40d0a30 con 0x7f8ef40b4ab0 2024-05-28T14:00:57.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8efc600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef40d0d10 con 0x7f8ef40ce610 2024-05-28T14:00:57.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8efa600640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8ef409b4d0 0x7f8ef41626c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38948/0 (socket says 172.21.15.42:38948) 2024-05-28T14:00:57.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8efa600640 1 -- 172.21.15.42:0/1225908410 learned_addr learned my addr 172.21.15.42:0/1225908410 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:00:57.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8ef9c00640 1 --2- 172.21.15.42:0/1225908410 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef40ce610 0x7f8ef409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:57.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8ef9200640 1 --2- 172.21.15.42:0/1225908410 >> v2:172.21.15.63:3300/0 conn(0x7f8ef40b4ab0 0x7f8ef409af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:57.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8ef3e00640 1 -- 172.21.15.42:0/1225908410 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4172902143 0 0) 0x7f8ef40d0750 con 0x7f8ef409b4d0 2024-05-28T14:00:57.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8ef3e00640 1 -- 172.21.15.42:0/1225908410 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8edc003530 con 0x7f8ef409b4d0 2024-05-28T14:00:57.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8ef9c00640 1 -- 172.21.15.42:0/1225908410 >> v2:172.21.15.63:3300/0 conn(0x7f8ef40b4ab0 msgr2=0x7f8ef409af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:00:57.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8ef9c00640 1 --2- 172.21.15.42:0/1225908410 >> v2:172.21.15.63:3300/0 conn(0x7f8ef40b4ab0 0x7f8ef409af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:57.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8ef9c00640 1 -- 172.21.15.42:0/1225908410 >> v1:172.21.15.42:6789/0 conn(0x7f8ef409b4d0 legacy=0x7f8ef41626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:57.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.304+0000 7f8ef9c00640 1 -- 172.21.15.42:0/1225908410 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef40d0ff0 con 0x7f8ef40ce610 2024-05-28T14:00:57.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.308+0000 7f8ef9200640 1 --2- 172.21.15.42:0/1225908410 >> v2:172.21.15.63:3300/0 conn(0x7f8ef40b4ab0 0x7f8ef409af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:00:57.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.308+0000 7f8ef9c00640 1 --2- 172.21.15.42:0/1225908410 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef40ce610 0x7f8ef409aa50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80097c0 tx=0x7f8ee801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c462c9dbd1dd05e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.308+0000 7f8ef3e00640 1 -- 172.21.15.42:0/1225908410 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80b1070 con 0x7f8ef40ce610 2024-05-28T14:00:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.308+0000 7f8ef3e00640 1 -- 172.21.15.42:0/1225908410 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee80ac410 con 0x7f8ef40ce610 2024-05-28T14:00:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.308+0000 7f8ef3e00640 1 -- 172.21.15.42:0/1225908410 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80ac7c0 con 0x7f8ef40ce610 2024-05-28T14:00:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.308+0000 7f8efc600640 1 -- 172.21.15.42:0/1225908410 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef40ce610 msgr2=0x7f8ef409aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.308+0000 7f8efc600640 1 --2- 172.21.15.42:0/1225908410 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef40ce610 0x7f8ef409aa50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80097c0 tx=0x7f8ee801c4a0 comp rx=0 tx=0).stop 2024-05-28T14:00:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.308+0000 7f8efc600640 1 -- 172.21.15.42:0/1225908410 shutdown_connections 2024-05-28T14:00:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.308+0000 7f8efc600640 1 --2- 172.21.15.42:0/1225908410 >> v2:172.21.15.63:3300/0 conn(0x7f8ef40b4ab0 0x7f8ef409af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.308+0000 7f8efc600640 1 --2- 172.21.15.42:0/1225908410 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef40ce610 0x7f8ef409aa50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.308+0000 7f8efc600640 1 -- 172.21.15.42:0/1225908410 >> 172.21.15.42:0/1225908410 conn(0x7f8ef40b3fc0 msgr2=0x7f8ef40b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.308+0000 7f8efc600640 1 -- 172.21.15.42:0/1225908410 shutdown_connections 2024-05-28T14:00:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.308+0000 7f8efc600640 1 -- 172.21.15.42:0/1225908410 wait complete. 2024-05-28T14:00:57.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8efc600640 1 Processor -- start 2024-05-28T14:00:57.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8efc600640 1 -- start start 2024-05-28T14:00:57.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8efc600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef409b4d0 0x7f8ef40b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:57.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8efc600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef40b4ab0 0x7f8ef415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:57.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8efc600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ef40ce610 0x7f8ef40af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:57.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8efc600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef416bd90 con 0x7f8ef409b4d0 2024-05-28T14:00:57.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8efc600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef416bc10 con 0x7f8ef40ce610 2024-05-28T14:00:57.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8efc600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef416bf10 con 0x7f8ef40b4ab0 2024-05-28T14:00:57.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8ef9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef409b4d0 0x7f8ef40b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:57.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8ef9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef409b4d0 0x7f8ef40b66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46834/0 (socket says 172.21.15.42:46834) 2024-05-28T14:00:57.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8ef9c00640 1 -- 172.21.15.42:0/733632574 learned_addr learned my addr 172.21.15.42:0/733632574 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:00:57.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8efa600640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ef40ce610 0x7f8ef40af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:57.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8ef9200640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef40b4ab0 0x7f8ef415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:57.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8ef9c00640 1 -- 172.21.15.42:0/733632574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef40b4ab0 msgr2=0x7f8ef415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:57.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8ef9c00640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef40b4ab0 0x7f8ef415a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:57.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8ef9c00640 1 -- 172.21.15.42:0/733632574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ef40ce610 msgr2=0x7f8ef40af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:57.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8ef9c00640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ef40ce610 0x7f8ef40af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:57.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8ef9c00640 1 -- 172.21.15.42:0/733632574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef40b70e0 con 0x7f8ef409b4d0 2024-05-28T14:00:57.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8efa600640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ef40ce610 0x7f8ef40af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:00:57.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8ef9c00640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef409b4d0 0x7f8ef40b66b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8ee801ab50 tx=0x7f8ee8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:57.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8ef1600640 1 -- 172.21.15.42:0/733632574 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80bb070 con 0x7f8ef409b4d0 2024-05-28T14:00:57.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8ef1600640 1 -- 172.21.15.42:0/733632574 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee80accc0 con 0x7f8ef409b4d0 2024-05-28T14:00:57.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8efc600640 1 -- 172.21.15.42:0/733632574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef415a980 con 0x7f8ef409b4d0 2024-05-28T14:00:57.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8ef1600640 1 -- 172.21.15.42:0/733632574 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80b1040 con 0x7f8ef409b4d0 2024-05-28T14:00:57.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.312+0000 7f8efc600640 1 -- 172.21.15.42:0/733632574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef40b7350 con 0x7f8ef409b4d0 2024-05-28T14:00:57.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.316+0000 7f8efc600640 1 -- 172.21.15.42:0/733632574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef415b000 con 0x7f8ef409b4d0 2024-05-28T14:00:57.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.316+0000 7f8ef1600640 1 -- 172.21.15.42:0/733632574 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7f8ee80be070 con 0x7f8ef409b4d0 2024-05-28T14:00:57.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.316+0000 7f8ef1600640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ed4068530 0x7f8ed406a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:00:57.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.316+0000 7f8ef1600640 1 -- 172.21.15.42:0/733632574 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8ee8120fe0 con 0x7f8ef409b4d0 2024-05-28T14:00:57.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.316+0000 7f8ef9200640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ed4068530 0x7f8ed406a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:00:57.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.316+0000 7f8ef9200640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ed4068530 0x7f8ed406a9f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40c9500 tx=0x7f8ee0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:00:57.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.320+0000 7f8ef1600640 1 -- 172.21.15.42:0/733632574 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ee8016090 con 0x7f8ef409b4d0 2024-05-28T14:00:57.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.548+0000 7f8efc600640 1 -- 172.21.15.42:0/733632574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8ef4068a90 con 0x7f8ef409b4d0 2024-05-28T14:00:57.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.548+0000 7f8ef1600640 1 -- 172.21.15.42:0/733632574 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2096 (secure 0 0 0) 0x7f8ee80f1380 con 0x7f8ef409b4d0 2024-05-28T14:00:57.534 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:00:57.534 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:00:57.534 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:00:57.534 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:00:57.534 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:00:57.534 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:00:57.534 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:00:57.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:57.534 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:00:57.534 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:00:57.534 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:00:57.535 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:00:57.535 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:00:57.535 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:57.535 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:00:57.535 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:00:57.535 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:00:57.535 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:00:57.535 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:57.535 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 178, 2024-05-28T14:00:57.535 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:00:57.535 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:00:57.535 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:00:57.535 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:00:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:00:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 34, 2024-05-28T14:00:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:00:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:00:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:00:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:00:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:00:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:00:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:00:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:00:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:00:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:00:57.537 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:00:57.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:00:57.537 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:57.537 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:00:57.537 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:00:57.537 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 2589, 2024-05-28T14:00:57.537 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 10738608726, 2024-05-28T14:00:57.537 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 21744304128, 2024-05-28T14:00:57.537 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 751349809152, 2024-05-28T14:00:57.537 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:00:57.537 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 170, 2024-05-28T14:00:57.537 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 95411240, 2024-05-28T14:00:57.537 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:00:57.537 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 26 2024-05-28T14:00:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:00:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:00:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2, 2024-05-28T14:00:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:00:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:00:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:00:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:00:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:00:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 2, 2024-05-28T14:00:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:00:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:00:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:00:57.538 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 54785 2024-05-28T14:00:57.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:00:57.539 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:57.539 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:00:57.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:57.539 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:00:57.539 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:00:57.539 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:00:57.539 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:00:57.539 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:00:57.539 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:00:57.539 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:00:57.539 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:00:57.539 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:00:57.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:57.540 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:00:57.540 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 8, 2024-05-28T14:00:57.540 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:00:32.881524+0000", 2024-05-28T14:00:57.540 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:00:57.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:00:57.540 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:00:57.540 INFO:teuthology.orchestra.run.smithi042.stdout: "2634ebf7-b822-4613-b0f8-227d147fb4c4": { 2024-05-28T14:00:57.540 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "1 ongoing clones; Avg progress is 0.0% (0s)\n [............................] ", 2024-05-28T14:00:57.540 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0, 2024-05-28T14:00:57.540 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:00:57.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:00:57.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:00:57.540 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:00:57.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.548+0000 7f8efc600640 1 -- 172.21.15.42:0/733632574 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ed4068530 msgr2=0x7f8ed406a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:57.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.548+0000 7f8efc600640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ed4068530 0x7f8ed406a9f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40c9500 tx=0x7f8ee0008040 comp rx=0 tx=0).stop 2024-05-28T14:00:57.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.548+0000 7f8efc600640 1 -- 172.21.15.42:0/733632574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef409b4d0 msgr2=0x7f8ef40b66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:00:57.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.548+0000 7f8efc600640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef409b4d0 0x7f8ef40b66b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8ee801ab50 tx=0x7f8ee8019240 comp rx=0 tx=0).stop 2024-05-28T14:00:57.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.548+0000 7f8efc600640 1 -- 172.21.15.42:0/733632574 shutdown_connections 2024-05-28T14:00:57.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.548+0000 7f8efc600640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ed4068530 0x7f8ed406a9f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:57.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.548+0000 7f8efc600640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ef40ce610 0x7f8ef40af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:57.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.548+0000 7f8efc600640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ef40b4ab0 0x7f8ef415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:57.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.548+0000 7f8efc600640 1 --2- 172.21.15.42:0/733632574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ef409b4d0 0x7f8ef40b66b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:00:57.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.548+0000 7f8efc600640 1 -- 172.21.15.42:0/733632574 >> 172.21.15.42:0/733632574 conn(0x7f8ef40b3fc0 msgr2=0x7f8ef40cd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:00:57.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.548+0000 7f8efc600640 1 -- 172.21.15.42:0/733632574 shutdown_connections 2024-05-28T14:00:57.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:00:57.548+0000 7f8efc600640 1 -- 172.21.15.42:0/733632574 wait complete. 2024-05-28T14:01:02.556 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:01:02.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.724+0000 7f983aa00640 1 Processor -- start 2024-05-28T14:01:02.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.724+0000 7f983aa00640 1 -- start start 2024-05-28T14:01:02.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.724+0000 7f983aa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9834156c00 0x7f983406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:02.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.724+0000 7f983aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98340af6d0 0x7f983415dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:02.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.724+0000 7f983aa00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f983406f6a0 con 0x7f983415d8d0 2024-05-28T14:01:02.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.724+0000 7f983aa00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f983406e600 con 0x7f9834156c00 2024-05-28T14:01:02.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.724+0000 7f983aa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f983406e8e0 con 0x7f98340af6d0 2024-05-28T14:01:02.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.728+0000 7f9833e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9834156c00 0x7f983406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:02.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.728+0000 7f9833e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9834156c00 0x7f983406dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:49826/0 (socket says 172.21.15.42:49826) 2024-05-28T14:01:02.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.728+0000 7f9833e00640 1 -- 172.21.15.42:0/804311591 learned_addr learned my addr 172.21.15.42:0/804311591 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:02.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.728+0000 7f9838a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98340af6d0 0x7f983415dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:02.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.728+0000 7f9833400640 1 -- 172.21.15.42:0/804311591 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3149979952 0 0) 0x7f983406f6a0 con 0x7f983415d8d0 2024-05-28T14:01:02.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.728+0000 7f9833400640 1 -- 172.21.15.42:0/804311591 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9818003530 con 0x7f983415d8d0 2024-05-28T14:01:02.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.728+0000 7f9833e00640 1 -- 172.21.15.42:0/804311591 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98340af6d0 msgr2=0x7f983415dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:02.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.728+0000 7f9833e00640 1 --2- 172.21.15.42:0/804311591 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98340af6d0 0x7f983415dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:02.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.728+0000 7f9833e00640 1 -- 172.21.15.42:0/804311591 >> v1:172.21.15.63:6789/0 conn(0x7f983415d8d0 legacy=0x7f98340aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:02.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.728+0000 7f9833e00640 1 -- 172.21.15.42:0/804311591 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983406ebc0 con 0x7f9834156c00 2024-05-28T14:01:02.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.728+0000 7f9838a00640 1 --2- 172.21.15.42:0/804311591 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98340af6d0 0x7f983415dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:01:02.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.728+0000 7f9833e00640 1 --2- 172.21.15.42:0/804311591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9834156c00 0x7f983406dd00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9824009870 tx=0x7f982401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cd1aaae1ef50e58 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:02.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.728+0000 7f9833400640 1 -- 172.21.15.42:0/804311591 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98240b0070 con 0x7f9834156c00 2024-05-28T14:01:02.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.728+0000 7f9833400640 1 -- 172.21.15.42:0/804311591 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98240ab600 con 0x7f9834156c00 2024-05-28T14:01:02.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f9833400640 1 -- 172.21.15.42:0/804311591 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98240ab9d0 con 0x7f9834156c00 2024-05-28T14:01:02.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 -- 172.21.15.42:0/804311591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9834156c00 msgr2=0x7f983406dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:02.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 --2- 172.21.15.42:0/804311591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9834156c00 0x7f983406dd00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9824009870 tx=0x7f982401c4d0 comp rx=0 tx=0).stop 2024-05-28T14:01:02.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 -- 172.21.15.42:0/804311591 shutdown_connections 2024-05-28T14:01:02.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 --2- 172.21.15.42:0/804311591 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98340af6d0 0x7f983415dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:02.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 --2- 172.21.15.42:0/804311591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9834156c00 0x7f983406dd00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:02.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 -- 172.21.15.42:0/804311591 >> 172.21.15.42:0/804311591 conn(0x7f983406bff0 msgr2=0x7f98340b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:02.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 -- 172.21.15.42:0/804311591 shutdown_connections 2024-05-28T14:01:02.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 -- 172.21.15.42:0/804311591 wait complete. 2024-05-28T14:01:02.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 Processor -- start 2024-05-28T14:01:02.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 -- start start 2024-05-28T14:01:02.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98340af6d0 0x7f9834099060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:02.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9834156c00 0x7f983409c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:02.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f983415d8d0 0x7f983409cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:02.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f983415ec40 con 0x7f983415d8d0 2024-05-28T14:01:02.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f983415eac0 con 0x7f9834156c00 2024-05-28T14:01:02.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f983aa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f983415edc0 con 0x7f98340af6d0 2024-05-28T14:01:02.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f9838a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f983415d8d0 0x7f983409cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:02.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f9838a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f983415d8d0 0x7f983409cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49100/0 (socket says 172.21.15.42:49100) 2024-05-28T14:01:02.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f9838a00640 1 -- 172.21.15.42:0/2757970906 learned_addr learned my addr 172.21.15.42:0/2757970906 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f9833e00640 1 --2- 172.21.15.42:0/2757970906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98340af6d0 0x7f9834099060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f982be00640 1 --2- 172.21.15.42:0/2757970906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9834156c00 0x7f983409c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f9838a00640 1 -- 172.21.15.42:0/2757970906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98340af6d0 msgr2=0x7f9834099060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f9838a00640 1 --2- 172.21.15.42:0/2757970906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98340af6d0 0x7f9834099060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.732+0000 7f9838a00640 1 -- 172.21.15.42:0/2757970906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9834156c00 msgr2=0x7f983409c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.736+0000 7f9838a00640 1 --2- 172.21.15.42:0/2757970906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9834156c00 0x7f983409c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.736+0000 7f9838a00640 1 -- 172.21.15.42:0/2757970906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983409d6f0 con 0x7f983415d8d0 2024-05-28T14:01:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.736+0000 7f9838a00640 1 --2- 172.21.15.42:0/2757970906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f983415d8d0 0x7f983409cec0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f982c00efc0 tx=0x7f982c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:02.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.736+0000 7f9830c00640 1 -- 172.21.15.42:0/2757970906 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c018030 con 0x7f983415d8d0 2024-05-28T14:01:02.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.736+0000 7f9830c00640 1 -- 172.21.15.42:0/2757970906 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f982c00cdf0 con 0x7f983415d8d0 2024-05-28T14:01:02.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.736+0000 7f9830c00640 1 -- 172.21.15.42:0/2757970906 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c003bc0 con 0x7f983415d8d0 2024-05-28T14:01:02.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.736+0000 7f983aa00640 1 -- 172.21.15.42:0/2757970906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f983409d9d0 con 0x7f983415d8d0 2024-05-28T14:01:02.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.736+0000 7f983aa00640 1 -- 172.21.15.42:0/2757970906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f983409df90 con 0x7f983415d8d0 2024-05-28T14:01:02.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.736+0000 7f983aa00640 1 -- 172.21.15.42:0/2757970906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9834068a90 con 0x7f983415d8d0 2024-05-28T14:01:02.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.736+0000 7f9830c00640 1 -- 172.21.15.42:0/2757970906 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7f982c01b070 con 0x7f983415d8d0 2024-05-28T14:01:02.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.736+0000 7f9830c00640 1 --2- 172.21.15.42:0/2757970906 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9810068530 0x7f981006a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:02.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.740+0000 7f9830c00640 1 -- 172.21.15.42:0/2757970906 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f982c087670 con 0x7f983415d8d0 2024-05-28T14:01:02.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.740+0000 7f9833e00640 1 --2- 172.21.15.42:0/2757970906 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9810068530 0x7f981006a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:02.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.740+0000 7f9833e00640 1 --2- 172.21.15.42:0/2757970906 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9810068530 0x7f981006a9f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f98240ad000 tx=0x7f9824008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:02.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.744+0000 7f9830c00640 1 -- 172.21.15.42:0/2757970906 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f982c010040 con 0x7f983415d8d0 2024-05-28T14:01:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.968+0000 7f983aa00640 1 -- 172.21.15.42:0/2757970906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f98340b2a40 con 0x7f983415d8d0 2024-05-28T14:01:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.968+0000 7f9830c00640 1 -- 172.21.15.42:0/2757970906 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2122 (secure 0 0 0) 0x7f982c057240 con 0x7f983415d8d0 2024-05-28T14:01:02.955 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:02.955 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:01:02.955 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:01:02.955 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:01:02.955 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:01:02.955 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:01:02.955 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:01:02.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:02.956 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:01:02.956 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:01:02.956 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:01:02.956 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:01:02.956 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:01:02.956 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:02.956 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:01:02.956 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:01:02.956 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:01:02.956 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:01:02.956 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:02.956 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 184, 2024-05-28T14:01:02.956 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:01:02.956 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:01:02.957 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:01:02.957 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:01:02.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:02.957 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:01:02.957 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 34, 2024-05-28T14:01:02.957 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:01:02.957 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:01:02.957 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:01:02.957 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:01:02.957 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:01:02.957 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:01:02.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:02.957 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:01:02.957 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:01:02.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:02.958 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:01:02.958 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:01:02.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:02.958 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:02.958 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:01:02.958 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:01:02.958 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 2846, 2024-05-28T14:01:02.958 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 11812453533, 2024-05-28T14:01:02.958 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 23892017152, 2024-05-28T14:01:02.958 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 749202096128, 2024-05-28T14:01:02.958 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 89460401, 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 121276941, 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 22, 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 34 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2, 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 2, 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:01:02.959 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 54785 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:02.960 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:01:02.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:02.961 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:01:02.961 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 8, 2024-05-28T14:01:02.961 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:00:32.881524+0000", 2024-05-28T14:01:02.961 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:01:02.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:02.961 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:01:02.961 INFO:teuthology.orchestra.run.smithi042.stdout: "2634ebf7-b822-4613-b0f8-227d147fb4c4": { 2024-05-28T14:01:02.961 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 27.0 seconds before exiting (6s)\n [============================] ", 2024-05-28T14:01:02.961 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:01:02.961 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:01:02.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:02.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:02.961 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:01:02.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.972+0000 7f983aa00640 1 -- 172.21.15.42:0/2757970906 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9810068530 msgr2=0x7f981006a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:02.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.972+0000 7f983aa00640 1 --2- 172.21.15.42:0/2757970906 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9810068530 0x7f981006a9f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f98240ad000 tx=0x7f9824008040 comp rx=0 tx=0).stop 2024-05-28T14:01:02.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.972+0000 7f983aa00640 1 -- 172.21.15.42:0/2757970906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f983415d8d0 msgr2=0x7f983409cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:02.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.972+0000 7f983aa00640 1 --2- 172.21.15.42:0/2757970906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f983415d8d0 0x7f983409cec0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f982c00efc0 tx=0x7f982c00c490 comp rx=0 tx=0).stop 2024-05-28T14:01:02.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.972+0000 7f983aa00640 1 -- 172.21.15.42:0/2757970906 shutdown_connections 2024-05-28T14:01:02.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.972+0000 7f983aa00640 1 --2- 172.21.15.42:0/2757970906 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9810068530 0x7f981006a9f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:02.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.972+0000 7f983aa00640 1 --2- 172.21.15.42:0/2757970906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f983415d8d0 0x7f983409cec0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:02.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.972+0000 7f983aa00640 1 --2- 172.21.15.42:0/2757970906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9834156c00 0x7f983409c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:02.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.972+0000 7f983aa00640 1 --2- 172.21.15.42:0/2757970906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98340af6d0 0x7f9834099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:02.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.972+0000 7f983aa00640 1 -- 172.21.15.42:0/2757970906 >> 172.21.15.42:0/2757970906 conn(0x7f983406bff0 msgr2=0x7f983414ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:02.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.972+0000 7f983aa00640 1 -- 172.21.15.42:0/2757970906 shutdown_connections 2024-05-28T14:01:02.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:02.972+0000 7f983aa00640 1 -- 172.21.15.42:0/2757970906 wait complete. 2024-05-28T14:01:07.977 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:01:08.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.144+0000 7f9f88a00640 1 Processor -- start 2024-05-28T14:01:08.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.144+0000 7f9f88a00640 1 -- start start 2024-05-28T14:01:08.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.144+0000 7f9f88a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f84158f30 0x7f9f84072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:08.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.144+0000 7f9f88a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9f8406ee10 0x7f9f8406f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:08.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.144+0000 7f9f88a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8415c120 con 0x7f9f840b4f00 2024-05-28T14:01:08.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.144+0000 7f9f88a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8415c9b0 con 0x7f9f8406ee10 2024-05-28T14:01:08.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.144+0000 7f9f88a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8415c3f0 con 0x7f9f84158f30 2024-05-28T14:01:08.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.144+0000 7f9f82800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9f8406ee10 0x7f9f8406f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:08.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.144+0000 7f9f82800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9f8406ee10 0x7f9f8406f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58506/0 (socket says 172.21.15.42:58506) 2024-05-28T14:01:08.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.144+0000 7f9f82800640 1 -- 172.21.15.42:0/3113180630 learned_addr learned my addr 172.21.15.42:0/3113180630 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:08.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.144+0000 7f9f81e00640 1 --2- 172.21.15.42:0/3113180630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f84158f30 0x7f9f84072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:08.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f82800640 1 -- 172.21.15.42:0/3113180630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f84158f30 msgr2=0x7f9f84072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:08.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f82800640 1 --2- 172.21.15.42:0/3113180630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f84158f30 0x7f9f84072610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:08.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f82800640 1 -- 172.21.15.42:0/3113180630 >> v1:172.21.15.63:6789/0 conn(0x7f9f840b4f00 legacy=0x7f9f8415ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:08.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f82800640 1 -- 172.21.15.42:0/3113180630 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8415c6d0 con 0x7f9f8406ee10 2024-05-28T14:01:08.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f81e00640 1 --2- 172.21.15.42:0/3113180630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f84158f30 0x7f9f84072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:08.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f82800640 1 --2- 172.21.15.42:0/3113180630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f8406ee10 0x7f9f8406f2a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009870 tx=0x7f9f7c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3aae79923eda6d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:08.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f81400640 1 -- 172.21.15.42:0/3113180630 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0b1070 con 0x7f9f8406ee10 2024-05-28T14:01:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f81400640 1 -- 172.21.15.42:0/3113180630 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f7c0ac620 con 0x7f9f8406ee10 2024-05-28T14:01:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f81400640 1 -- 172.21.15.42:0/3113180630 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0ac9f0 con 0x7f9f8406ee10 2024-05-28T14:01:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f88a00640 1 -- 172.21.15.42:0/3113180630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f8406ee10 msgr2=0x7f9f8406f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f88a00640 1 --2- 172.21.15.42:0/3113180630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f8406ee10 0x7f9f8406f2a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009870 tx=0x7f9f7c01c4d0 comp rx=0 tx=0).stop 2024-05-28T14:01:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f88a00640 1 -- 172.21.15.42:0/3113180630 shutdown_connections 2024-05-28T14:01:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f88a00640 1 --2- 172.21.15.42:0/3113180630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f8406ee10 0x7f9f8406f2a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f88a00640 1 --2- 172.21.15.42:0/3113180630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f84158f30 0x7f9f84072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f88a00640 1 -- 172.21.15.42:0/3113180630 >> 172.21.15.42:0/3113180630 conn(0x7f9f8406bff0 msgr2=0x7f9f840b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f88a00640 1 -- 172.21.15.42:0/3113180630 shutdown_connections 2024-05-28T14:01:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.148+0000 7f9f88a00640 1 -- 172.21.15.42:0/3113180630 wait complete. 2024-05-28T14:01:08.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f88a00640 1 Processor -- start 2024-05-28T14:01:08.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f88a00640 1 -- start start 2024-05-28T14:01:08.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f88a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f8406ee10 0x7f9f84152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:08.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f88a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f840b4f00 0x7f9f84150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:08.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f88a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f84158f30 0x7f9f840afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:08.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f88a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9f84072cd0 con 0x7f9f840b4f00 2024-05-28T14:01:08.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f88a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f84072b50 con 0x7f9f84158f30 2024-05-28T14:01:08.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f88a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9f840b39f0 con 0x7f9f8406ee10 2024-05-28T14:01:08.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f79400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f840b4f00 0x7f9f84150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:08.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f79400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f840b4f00 0x7f9f84150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58510/0 (socket says 172.21.15.42:58510) 2024-05-28T14:01:08.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f79400640 1 -- 172.21.15.42:0/2321012960 learned_addr learned my addr 172.21.15.42:0/2321012960 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:08.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f81e00640 1 --2- 172.21.15.42:0/2321012960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f8406ee10 0x7f9f84152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:08.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f82800640 1 --2- 172.21.15.42:0/2321012960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f84158f30 0x7f9f840afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:08.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f79400640 1 -- 172.21.15.42:0/2321012960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f8406ee10 msgr2=0x7f9f84152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:08.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f79400640 1 --2- 172.21.15.42:0/2321012960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f8406ee10 0x7f9f84152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:08.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f79400640 1 -- 172.21.15.42:0/2321012960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f84158f30 msgr2=0x7f9f840afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:08.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f79400640 1 --2- 172.21.15.42:0/2321012960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f84158f30 0x7f9f840afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:08.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f79400640 1 -- 172.21.15.42:0/2321012960 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f84151040 con 0x7f9f840b4f00 2024-05-28T14:01:08.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f79400640 1 --2- 172.21.15.42:0/2321012960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f840b4f00 0x7f9f84150270 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c790 tx=0x7f9f6c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:08.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f7aa00640 1 -- 172.21.15.42:0/2321012960 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c016020 con 0x7f9f840b4f00 2024-05-28T14:01:08.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f7aa00640 1 -- 172.21.15.42:0/2321012960 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f6c006d60 con 0x7f9f840b4f00 2024-05-28T14:01:08.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f88a00640 1 -- 172.21.15.42:0/2321012960 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f84151320 con 0x7f9f840b4f00 2024-05-28T14:01:08.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.152+0000 7f9f7aa00640 1 -- 172.21.15.42:0/2321012960 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c003bc0 con 0x7f9f840b4f00 2024-05-28T14:01:08.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.156+0000 7f9f88a00640 1 -- 172.21.15.42:0/2321012960 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f841518e0 con 0x7f9f840b4f00 2024-05-28T14:01:08.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.156+0000 7f9f88a00640 1 -- 172.21.15.42:0/2321012960 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f84068a90 con 0x7f9f840b4f00 2024-05-28T14:01:08.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.156+0000 7f9f7aa00640 1 -- 172.21.15.42:0/2321012960 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7f9f6c00a850 con 0x7f9f840b4f00 2024-05-28T14:01:08.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.156+0000 7f9f7aa00640 1 --2- 172.21.15.42:0/2321012960 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f60068530 0x7f9f6006a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:08.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.156+0000 7f9f7aa00640 1 -- 172.21.15.42:0/2321012960 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f9f6c086830 con 0x7f9f840b4f00 2024-05-28T14:01:08.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.156+0000 7f9f81e00640 1 --2- 172.21.15.42:0/2321012960 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f60068530 0x7f9f6006a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:08.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.156+0000 7f9f81e00640 1 --2- 172.21.15.42:0/2321012960 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f60068530 0x7f9f6006a9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9f740045f0 tx=0x7f9f74008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:08.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.160+0000 7f9f7aa00640 1 -- 172.21.15.42:0/2321012960 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9f6c020020 con 0x7f9f840b4f00 2024-05-28T14:01:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.388+0000 7f9f88a00640 1 -- 172.21.15.42:0/2321012960 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9f84072910 con 0x7f9f840b4f00 2024-05-28T14:01:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.392+0000 7f9f7aa00640 1 -- 172.21.15.42:0/2321012960 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2122 (secure 0 0 0) 0x7f9f6c0565b0 con 0x7f9f840b4f00 2024-05-28T14:01:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:08.377 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:01:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:01:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:01:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:01:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:01:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:01:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:01:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:01:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:01:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:01:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:01:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:01:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:01:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:01:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:01:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 189, 2024-05-28T14:01:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:01:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:01:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:01:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:01:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:01:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 34, 2024-05-28T14:01:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:01:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:01:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:01:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:01:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:01:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:01:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:01:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:01:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:01:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:01:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 2845, 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 11812474326, 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 23892041728, 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 749202071552, 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 89458676, 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 89469937, 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 22, 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 25 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2, 2024-05-28T14:01:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:01:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:01:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:01:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:01:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 2, 2024-05-28T14:01:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:01:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:01:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:01:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 54785 2024-05-28T14:01:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:01:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:01:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:01:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:01:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:01:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:01:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:01:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:01:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:01:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:01:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 8, 2024-05-28T14:01:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:00:32.881524+0000", 2024-05-28T14:01:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:01:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:01:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: "2634ebf7-b822-4613-b0f8-227d147fb4c4": { 2024-05-28T14:01:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 21.0 seconds before exiting (12s)\n [============================] ", 2024-05-28T14:01:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:01:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:01:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:08.383 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:01:08.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.392+0000 7f9f88a00640 1 -- 172.21.15.42:0/2321012960 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f60068530 msgr2=0x7f9f6006a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:08.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.392+0000 7f9f88a00640 1 --2- 172.21.15.42:0/2321012960 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f60068530 0x7f9f6006a9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9f740045f0 tx=0x7f9f74008040 comp rx=0 tx=0).stop 2024-05-28T14:01:08.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.392+0000 7f9f88a00640 1 -- 172.21.15.42:0/2321012960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f840b4f00 msgr2=0x7f9f84150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:08.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.392+0000 7f9f88a00640 1 --2- 172.21.15.42:0/2321012960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f840b4f00 0x7f9f84150270 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c790 tx=0x7f9f6c00cc50 comp rx=0 tx=0).stop 2024-05-28T14:01:08.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.392+0000 7f9f88a00640 1 -- 172.21.15.42:0/2321012960 shutdown_connections 2024-05-28T14:01:08.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.392+0000 7f9f88a00640 1 --2- 172.21.15.42:0/2321012960 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f60068530 0x7f9f6006a9f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:08.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.392+0000 7f9f88a00640 1 --2- 172.21.15.42:0/2321012960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f84158f30 0x7f9f840afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:08.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.392+0000 7f9f88a00640 1 --2- 172.21.15.42:0/2321012960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f840b4f00 0x7f9f84150270 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:08.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.392+0000 7f9f88a00640 1 --2- 172.21.15.42:0/2321012960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f8406ee10 0x7f9f84152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:08.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.392+0000 7f9f88a00640 1 -- 172.21.15.42:0/2321012960 >> 172.21.15.42:0/2321012960 conn(0x7f9f8406bff0 msgr2=0x7f9f840ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:08.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.392+0000 7f9f88a00640 1 -- 172.21.15.42:0/2321012960 shutdown_connections 2024-05-28T14:01:08.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:08.392+0000 7f9f88a00640 1 -- 172.21.15.42:0/2321012960 wait complete. 2024-05-28T14:01:13.399 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:01:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.564+0000 7f7ed5e00640 1 Processor -- start 2024-05-28T14:01:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.564+0000 7f7ed5e00640 1 -- start start 2024-05-28T14:01:13.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ed5e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7ed006c430 0x7f7ed0160e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ed00d0b40 0x7f7ed015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ed5e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed01603a0 con 0x7f7ed00d0760 2024-05-28T14:01:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ed5e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed00cd760 con 0x7f7ed006c430 2024-05-28T14:01:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ed5e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed0160640 con 0x7f7ed00d0b40 2024-05-28T14:01:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ed4a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f7ed00d0760 0x7f7ed00ccd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:40158/0 (socket says 172.21.15.42:40158) 2024-05-28T14:01:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ed4a00640 1 -- 172.21.15.42:0/2007146191 learned_addr learned my addr 172.21.15.42:0/2007146191 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ed5400640 1 --2- 172.21.15.42:0/2007146191 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ed00d0b40 0x7f7ed015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ec7e00640 1 --2- 172.21.15.42:0/2007146191 >> v2:172.21.15.63:3300/0 conn(0x7f7ed006c430 0x7f7ed0160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ecfe00640 1 -- 172.21.15.42:0/2007146191 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2340787825 0 0) 0x7f7ed01603a0 con 0x7f7ed00d0760 2024-05-28T14:01:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ecfe00640 1 -- 172.21.15.42:0/2007146191 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7eb4003530 con 0x7f7ed00d0760 2024-05-28T14:01:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ed5400640 1 -- 172.21.15.42:0/2007146191 >> v2:172.21.15.63:3300/0 conn(0x7f7ed006c430 msgr2=0x7f7ed0160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:13.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ed5400640 1 --2- 172.21.15.42:0/2007146191 >> v2:172.21.15.63:3300/0 conn(0x7f7ed006c430 0x7f7ed0160e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:13.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ed5400640 1 -- 172.21.15.42:0/2007146191 >> v1:172.21.15.42:6789/0 conn(0x7f7ed00d0760 legacy=0x7f7ed00ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:13.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.568+0000 7f7ed5400640 1 -- 172.21.15.42:0/2007146191 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed0160920 con 0x7f7ed00d0b40 2024-05-28T14:01:13.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.572+0000 7f7ed5400640 1 --2- 172.21.15.42:0/2007146191 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ed00d0b40 0x7f7ed015c1e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009d90 tx=0x7f7ec801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14e69bca8c61ff20 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:13.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.572+0000 7f7ecfe00640 1 -- 172.21.15.42:0/2007146191 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80b2070 con 0x7f7ed00d0b40 2024-05-28T14:01:13.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.572+0000 7f7ecfe00640 1 -- 172.21.15.42:0/2007146191 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec80a5a10 con 0x7f7ed00d0b40 2024-05-28T14:01:13.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.572+0000 7f7ecfe00640 1 -- 172.21.15.42:0/2007146191 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80a5dc0 con 0x7f7ed00d0b40 2024-05-28T14:01:13.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.572+0000 7f7ed5e00640 1 -- 172.21.15.42:0/2007146191 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ed00d0b40 msgr2=0x7f7ed015c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:13.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.572+0000 7f7ed5e00640 1 --2- 172.21.15.42:0/2007146191 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ed00d0b40 0x7f7ed015c1e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009d90 tx=0x7f7ec801c970 comp rx=0 tx=0).stop 2024-05-28T14:01:13.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.572+0000 7f7ed5e00640 1 -- 172.21.15.42:0/2007146191 shutdown_connections 2024-05-28T14:01:13.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.572+0000 7f7ed5e00640 1 --2- 172.21.15.42:0/2007146191 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ed00d0b40 0x7f7ed015c1e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:13.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.572+0000 7f7ed5e00640 1 --2- 172.21.15.42:0/2007146191 >> v2:172.21.15.63:3300/0 conn(0x7f7ed006c430 0x7f7ed0160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:13.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.572+0000 7f7ed5e00640 1 -- 172.21.15.42:0/2007146191 >> 172.21.15.42:0/2007146191 conn(0x7f7ed009aa50 msgr2=0x7f7ed00b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:13.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.572+0000 7f7ed5e00640 1 -- 172.21.15.42:0/2007146191 shutdown_connections 2024-05-28T14:01:13.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.572+0000 7f7ed5e00640 1 -- 172.21.15.42:0/2007146191 wait complete. 2024-05-28T14:01:13.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.572+0000 7f7ed5e00640 1 Processor -- start 2024-05-28T14:01:13.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.572+0000 7f7ed5e00640 1 -- start start 2024-05-28T14:01:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ed006c430 0x7f7ed01fb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ed00d0760 0x7f7ed0152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ed00d0b40 0x7f7ed01fac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ed5e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed00cf820 con 0x7f7ed00d0760 2024-05-28T14:01:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ed5e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed00cf6a0 con 0x7f7ed00d0b40 2024-05-28T14:01:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ed5e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed00cf9a0 con 0x7f7ed006c430 2024-05-28T14:01:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ec7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ed00d0760 0x7f7ed0152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ec7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ed00d0760 0x7f7ed0152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58520/0 (socket says 172.21.15.42:58520) 2024-05-28T14:01:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ec7e00640 1 -- 172.21.15.42:0/3885778823 learned_addr learned my addr 172.21.15.42:0/3885778823 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ed5400640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ed00d0b40 0x7f7ed01fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ed4a00640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ed006c430 0x7f7ed01fb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ec7e00640 1 -- 172.21.15.42:0/3885778823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ed006c430 msgr2=0x7f7ed01fb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ec7e00640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ed006c430 0x7f7ed01fb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ec7e00640 1 -- 172.21.15.42:0/3885778823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ed00d0b40 msgr2=0x7f7ed01fac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:13.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ec7e00640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ed00d0b40 0x7f7ed01fac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:13.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ec7e00640 1 -- 172.21.15.42:0/3885778823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed0155bb0 con 0x7f7ed00d0760 2024-05-28T14:01:13.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ed4a00640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ed006c430 0x7f7ed01fb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:13.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ed5400640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ed00d0b40 0x7f7ed01fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:13.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ec7e00640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ed00d0760 0x7f7ed0152610 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800e840 tx=0x7f7eb800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ecd600640 1 -- 172.21.15.42:0/3885778823 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8017070 con 0x7f7ed00d0760 2024-05-28T14:01:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ecd600640 1 -- 172.21.15.42:0/3885778823 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb800ee90 con 0x7f7ed00d0760 2024-05-28T14:01:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ecd600640 1 -- 172.21.15.42:0/3885778823 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb800cb60 con 0x7f7ed00d0760 2024-05-28T14:01:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ed5e00640 1 -- 172.21.15.42:0/3885778823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed0155e90 con 0x7f7ed00d0760 2024-05-28T14:01:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ed5e00640 1 -- 172.21.15.42:0/3885778823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed0152b50 con 0x7f7ed00d0760 2024-05-28T14:01:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ed5e00640 1 -- 172.21.15.42:0/3885778823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed0068a90 con 0x7f7ed00d0760 2024-05-28T14:01:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.576+0000 7f7ecd600640 1 -- 172.21.15.42:0/3885778823 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7f7eb801a070 con 0x7f7ed00d0760 2024-05-28T14:01:13.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.580+0000 7f7ecd600640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7eac068530 0x7f7eac06a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:13.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.580+0000 7f7ecd600640 1 -- 172.21.15.42:0/3885778823 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7eb8086f10 con 0x7f7ed00d0760 2024-05-28T14:01:13.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.580+0000 7f7ed4a00640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7eac068530 0x7f7eac06a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:13.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.580+0000 7f7ed4a00640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7eac068530 0x7f7eac06a9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0001210 tx=0x7f7ec0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.584+0000 7f7ecd600640 1 -- 172.21.15.42:0/3885778823 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7eb8057240 con 0x7f7ed00d0760 2024-05-28T14:01:13.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.816+0000 7f7ed5e00640 1 -- 172.21.15.42:0/3885778823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7ed00b7240 con 0x7f7ed00d0760 2024-05-28T14:01:13.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.816+0000 7f7ecd600640 1 -- 172.21.15.42:0/3885778823 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2110 (secure 0 0 0) 0x7f7eb8056b40 con 0x7f7ed00d0760 2024-05-28T14:01:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:13.801 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:01:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:01:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:01:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:01:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:01:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:01:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:01:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:01:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:01:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:01:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:01:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 195, 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 34, 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:01:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 2845, 2024-05-28T14:01:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 11812496236, 2024-05-28T14:01:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 23892082688, 2024-05-28T14:01:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 749202030592, 2024-05-28T14:01:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:01:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 85, 2024-05-28T14:01:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 4265, 2024-05-28T14:01:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:01:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 1 2024-05-28T14:01:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:01:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:01:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2, 2024-05-28T14:01:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:01:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:01:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 2, 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 54785 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:01:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:01:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:01:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:01:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:01:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:01:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 8, 2024-05-28T14:01:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:00:32.881524+0000", 2024-05-28T14:01:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:01:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:01:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "2634ebf7-b822-4613-b0f8-227d147fb4c4": { 2024-05-28T14:01:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 15.0 seconds before exiting (18s)\n [============================] ", 2024-05-28T14:01:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:01:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:01:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:13.807 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:01:13.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.816+0000 7f7ed5e00640 1 -- 172.21.15.42:0/3885778823 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7eac068530 msgr2=0x7f7eac06a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:13.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.816+0000 7f7ed5e00640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7eac068530 0x7f7eac06a9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0001210 tx=0x7f7ec0009040 comp rx=0 tx=0).stop 2024-05-28T14:01:13.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.816+0000 7f7ed5e00640 1 -- 172.21.15.42:0/3885778823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ed00d0760 msgr2=0x7f7ed0152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:13.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.816+0000 7f7ed5e00640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ed00d0760 0x7f7ed0152610 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800e840 tx=0x7f7eb800ec40 comp rx=0 tx=0).stop 2024-05-28T14:01:13.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.816+0000 7f7ed5e00640 1 -- 172.21.15.42:0/3885778823 shutdown_connections 2024-05-28T14:01:13.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.816+0000 7f7ed5e00640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7eac068530 0x7f7eac06a9f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:13.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.816+0000 7f7ed5e00640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ed00d0b40 0x7f7ed01fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:13.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.816+0000 7f7ed5e00640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ed00d0760 0x7f7ed0152610 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:13.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.816+0000 7f7ed5e00640 1 --2- 172.21.15.42:0/3885778823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ed006c430 0x7f7ed01fb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:13.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.816+0000 7f7ed5e00640 1 -- 172.21.15.42:0/3885778823 >> 172.21.15.42:0/3885778823 conn(0x7f7ed009aa50 msgr2=0x7f7ed015b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:13.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.816+0000 7f7ed5e00640 1 -- 172.21.15.42:0/3885778823 shutdown_connections 2024-05-28T14:01:13.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:13.816+0000 7f7ed5e00640 1 -- 172.21.15.42:0/3885778823 wait complete. 2024-05-28T14:01:17.665 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:01:17.669 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:01:18.823 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:01:18.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.988+0000 7f1b03000640 1 Processor -- start 2024-05-28T14:01:18.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.988+0000 7f1b03000640 1 -- start start 2024-05-28T14:01:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.992+0000 7f1b03000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1afc15ea00 0x7f1afc0ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.992+0000 7f1b03000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1afc06b9f0 0x7f1afc06be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.992+0000 7f1b03000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1afc0ae8f0 con 0x7f1afc0b0300 2024-05-28T14:01:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.992+0000 7f1b03000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1afc06c340 con 0x7f1afc15ea00 2024-05-28T14:01:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.992+0000 7f1b03000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1afc0b07e0 con 0x7f1afc06b9f0 2024-05-28T14:01:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.992+0000 7f1afbe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1afc15ea00 0x7f1afc0ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.992+0000 7f1afbe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1afc15ea00 0x7f1afc0ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57166/0 (socket says 172.21.15.42:57166) 2024-05-28T14:01:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.992+0000 7f1afbe00640 1 -- 172.21.15.42:0/1279564366 learned_addr learned my addr 172.21.15.42:0/1279564366 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:18.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.992+0000 7f1b01000640 1 --2- 172.21.15.42:0/1279564366 >> v2:172.21.15.63:3300/0 conn(0x7f1afc06b9f0 0x7f1afc06be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:18.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.992+0000 7f1afbe00640 1 -- 172.21.15.42:0/1279564366 >> v2:172.21.15.63:3300/0 conn(0x7f1afc06b9f0 msgr2=0x7f1afc06be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:18.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.992+0000 7f1afbe00640 1 --2- 172.21.15.42:0/1279564366 >> v2:172.21.15.63:3300/0 conn(0x7f1afc06b9f0 0x7f1afc06be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:18.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.992+0000 7f1afbe00640 1 -- 172.21.15.42:0/1279564366 >> v1:172.21.15.63:6789/0 conn(0x7f1afc0b0300 legacy=0x7f1afc0b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:18.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.992+0000 7f1afbe00640 1 -- 172.21.15.42:0/1279564366 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc0b0ac0 con 0x7f1afc15ea00 2024-05-28T14:01:18.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.992+0000 7f1b01000640 1 --2- 172.21.15.42:0/1279564366 >> v2:172.21.15.63:3300/0 conn(0x7f1afc06b9f0 0x7f1afc06be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:18.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1afbe00640 1 --2- 172.21.15.42:0/1279564366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1afc15ea00 0x7f1afc0ae3b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0b4140 tx=0x7f1aec01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e510703ba032a0cc server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:18.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1afb400640 1 -- 172.21.15.42:0/1279564366 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec0aa610 con 0x7f1afc15ea00 2024-05-28T14:01:18.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1afb400640 1 -- 172.21.15.42:0/1279564366 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aec0af070 con 0x7f1afc15ea00 2024-05-28T14:01:18.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1afb400640 1 -- 172.21.15.42:0/1279564366 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec0aa9d0 con 0x7f1afc15ea00 2024-05-28T14:01:18.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1b03000640 1 -- 172.21.15.42:0/1279564366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1afc15ea00 msgr2=0x7f1afc0ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:18.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1b03000640 1 --2- 172.21.15.42:0/1279564366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1afc15ea00 0x7f1afc0ae3b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0b4140 tx=0x7f1aec01c4b0 comp rx=0 tx=0).stop 2024-05-28T14:01:18.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1b03000640 1 -- 172.21.15.42:0/1279564366 shutdown_connections 2024-05-28T14:01:18.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1b03000640 1 --2- 172.21.15.42:0/1279564366 >> v2:172.21.15.63:3300/0 conn(0x7f1afc06b9f0 0x7f1afc06be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:18.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1b03000640 1 --2- 172.21.15.42:0/1279564366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1afc15ea00 0x7f1afc0ae3b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:18.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1b03000640 1 -- 172.21.15.42:0/1279564366 >> 172.21.15.42:0/1279564366 conn(0x7f1afc0b6b10 msgr2=0x7f1afc09aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:18.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1b03000640 1 -- 172.21.15.42:0/1279564366 shutdown_connections 2024-05-28T14:01:18.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1b03000640 1 -- 172.21.15.42:0/1279564366 wait complete. 2024-05-28T14:01:18.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1b03000640 1 Processor -- start 2024-05-28T14:01:18.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1b03000640 1 -- start start 2024-05-28T14:01:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1b03000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1afc06b9f0 0x7f1afc09e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:18.996+0000 7f1b03000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1afc0b0300 0x7f1afc14a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1b03000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1afc15ea00 0x7f1afc14e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1b03000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc06ff80 con 0x7f1afc0b0300 2024-05-28T14:01:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1b03000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc0b0da0 con 0x7f1afc15ea00 2024-05-28T14:01:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1b03000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1afc070100 con 0x7f1afc06b9f0 2024-05-28T14:01:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1af3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1afc0b0300 0x7f1afc14a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1af3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1afc0b0300 0x7f1afc14a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57170/0 (socket says 172.21.15.42:57170) 2024-05-28T14:01:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1af3e00640 1 -- 172.21.15.42:0/1661932139 learned_addr learned my addr 172.21.15.42:0/1661932139 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1b01000640 1 --2- 172.21.15.42:0/1661932139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1afc15ea00 0x7f1afc14e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1afbe00640 1 --2- 172.21.15.42:0/1661932139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1afc06b9f0 0x7f1afc09e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1af3e00640 1 -- 172.21.15.42:0/1661932139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1afc06b9f0 msgr2=0x7f1afc09e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1af3e00640 1 --2- 172.21.15.42:0/1661932139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1afc06b9f0 0x7f1afc09e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:18.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1af3e00640 1 -- 172.21.15.42:0/1661932139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1afc15ea00 msgr2=0x7f1afc14e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:18.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1af3e00640 1 --2- 172.21.15.42:0/1661932139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1afc15ea00 0x7f1afc14e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:18.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1af3e00640 1 -- 172.21.15.42:0/1661932139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc09ea90 con 0x7f1afc0b0300 2024-05-28T14:01:18.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1af3e00640 1 --2- 172.21.15.42:0/1661932139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1afc0b0300 0x7f1afc14a9f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400c410 tx=0x7f1ae400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:18.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1af8c00640 1 -- 172.21.15.42:0/1661932139 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4011790 con 0x7f1afc0b0300 2024-05-28T14:01:18.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1af8c00640 1 -- 172.21.15.42:0/1661932139 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae4017070 con 0x7f1afc0b0300 2024-05-28T14:01:18.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1b03000640 1 -- 172.21.15.42:0/1661932139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1afc09ed70 con 0x7f1afc0b0300 2024-05-28T14:01:18.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1af8c00640 1 -- 172.21.15.42:0/1661932139 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4011b70 con 0x7f1afc0b0300 2024-05-28T14:01:18.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1b03000640 1 -- 172.21.15.42:0/1661932139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1afc0b2580 con 0x7f1afc0b0300 2024-05-28T14:01:18.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1b03000640 1 -- 172.21.15.42:0/1661932139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1afc0b2780 con 0x7f1afc0b0300 2024-05-28T14:01:18.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.000+0000 7f1af8c00640 1 -- 172.21.15.42:0/1661932139 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7f1ae4011d10 con 0x7f1afc0b0300 2024-05-28T14:01:18.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.004+0000 7f1af8c00640 1 --2- 172.21.15.42:0/1661932139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ad8068530 0x7f1ad806a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:18.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.004+0000 7f1af8c00640 1 -- 172.21.15.42:0/1661932139 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f1ae4086880 con 0x7f1afc0b0300 2024-05-28T14:01:18.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.004+0000 7f1afbe00640 1 --2- 172.21.15.42:0/1661932139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ad8068530 0x7f1ad806a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:18.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.004+0000 7f1afbe00640 1 --2- 172.21.15.42:0/1661932139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ad8068530 0x7f1ad806a9f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0ae000 tx=0x7f1aec0a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:18.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.008+0000 7f1af8c00640 1 -- 172.21.15.42:0/1661932139 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1afc0b2780 con 0x7f1afc0b0300 2024-05-28T14:01:19.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.228+0000 7f1b03000640 1 -- 172.21.15.42:0/1661932139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1afc068a90 con 0x7f1afc0b0300 2024-05-28T14:01:19.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.228+0000 7f1af8c00640 1 -- 172.21.15.42:0/1661932139 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2080 (secure 0 0 0) 0x7f1ae4021020 con 0x7f1afc0b0300 2024-05-28T14:01:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:19.215 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:01:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:01:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:01:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:01:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:01:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:01:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:01:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:01:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:01:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:01:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:01:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:01:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:01:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:01:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:01:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 200, 2024-05-28T14:01:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:01:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:01:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:01:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:01:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:01:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 34, 2024-05-28T14:01:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:01:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:01:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:01:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:01:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:01:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:01:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:01:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:01:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:01:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:01:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:01:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:01:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 2845, 2024-05-28T14:01:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 11812496236, 2024-05-28T14:01:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 23892082688, 2024-05-28T14:01:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 749202030592, 2024-05-28T14:01:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:01:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 2047, 2024-05-28T14:01:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:01:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:01:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:01:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:01:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2, 2024-05-28T14:01:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:01:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:01:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:01:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:01:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 2, 2024-05-28T14:01:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:01:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:01:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:01:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 54785 2024-05-28T14:01:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:01:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:01:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:01:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:01:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:01:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:01:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:01:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:01:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:01:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:01:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 8, 2024-05-28T14:01:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:00:32.881524+0000", 2024-05-28T14:01:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:01:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:01:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "2634ebf7-b822-4613-b0f8-227d147fb4c4": { 2024-05-28T14:01:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 11.0 seconds before exiting (22s)\n [============================] ", 2024-05-28T14:01:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:01:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:01:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:19.221 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:01:19.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.232+0000 7f1b03000640 1 -- 172.21.15.42:0/1661932139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ad8068530 msgr2=0x7f1ad806a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:19.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.232+0000 7f1b03000640 1 --2- 172.21.15.42:0/1661932139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ad8068530 0x7f1ad806a9f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0ae000 tx=0x7f1aec0a7000 comp rx=0 tx=0).stop 2024-05-28T14:01:19.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.232+0000 7f1b03000640 1 -- 172.21.15.42:0/1661932139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1afc0b0300 msgr2=0x7f1afc14a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:19.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.232+0000 7f1b03000640 1 --2- 172.21.15.42:0/1661932139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1afc0b0300 0x7f1afc14a9f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400c410 tx=0x7f1ae400c8d0 comp rx=0 tx=0).stop 2024-05-28T14:01:19.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.232+0000 7f1b03000640 1 -- 172.21.15.42:0/1661932139 shutdown_connections 2024-05-28T14:01:19.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.232+0000 7f1b03000640 1 --2- 172.21.15.42:0/1661932139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ad8068530 0x7f1ad806a9f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:19.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.232+0000 7f1b03000640 1 --2- 172.21.15.42:0/1661932139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1afc15ea00 0x7f1afc14e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:19.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.232+0000 7f1b03000640 1 --2- 172.21.15.42:0/1661932139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1afc0b0300 0x7f1afc14a9f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:19.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.232+0000 7f1b03000640 1 --2- 172.21.15.42:0/1661932139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1afc06b9f0 0x7f1afc09e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:19.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.232+0000 7f1b03000640 1 -- 172.21.15.42:0/1661932139 >> 172.21.15.42:0/1661932139 conn(0x7f1afc0b6b10 msgr2=0x7f1afc0b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:19.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.232+0000 7f1b03000640 1 -- 172.21.15.42:0/1661932139 shutdown_connections 2024-05-28T14:01:19.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:19.232+0000 7f1b03000640 1 -- 172.21.15.42:0/1661932139 wait complete. 2024-05-28T14:01:24.237 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:01:24.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f680b600640 1 Processor -- start 2024-05-28T14:01:24.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f680b600640 1 -- start start 2024-05-28T14:01:24.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f680b600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68040cd290 0x7f680415d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:24.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f680b600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6804160330 0x7f68041607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:24.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f680b600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f680406c4b0 con 0x7f68040cd670 2024-05-28T14:01:24.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f680b600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f680415dd40 con 0x7f6804160330 2024-05-28T14:01:24.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f680b600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f680415d370 con 0x7f68040cd290 2024-05-28T14:01:24.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f6809600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6804160330 0x7f68041607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:24.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f6803e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f68040cd670 0x7f680406b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55188/0 (socket says 172.21.15.42:55188) 2024-05-28T14:01:24.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f6803e00640 1 -- 172.21.15.42:0/1701389436 learned_addr learned my addr 172.21.15.42:0/1701389436 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:24.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f6808c00640 1 --2- 172.21.15.42:0/1701389436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68040cd290 0x7f680415d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:24.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f6803400640 1 -- 172.21.15.42:0/1701389436 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2022467724 0 0) 0x7f680406c4b0 con 0x7f68040cd670 2024-05-28T14:01:24.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f6809600640 1 -- 172.21.15.42:0/1701389436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68040cd290 msgr2=0x7f680415d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:24.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f6809600640 1 --2- 172.21.15.42:0/1701389436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68040cd290 0x7f680415d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:24.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f6809600640 1 -- 172.21.15.42:0/1701389436 >> v1:172.21.15.42:6789/0 conn(0x7f68040cd670 legacy=0x7f680406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:24.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.404+0000 7f6809600640 1 -- 172.21.15.42:0/1701389436 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680406c790 con 0x7f6804160330 2024-05-28T14:01:24.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.408+0000 7f6809600640 1 --2- 172.21.15.42:0/1701389436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6804160330 0x7f68041607a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009d90 tx=0x7f67fc01d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e224feac4d2c8ae server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:24.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.408+0000 7f6803400640 1 -- 172.21.15.42:0/1701389436 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0b2070 con 0x7f6804160330 2024-05-28T14:01:24.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.408+0000 7f6803400640 1 -- 172.21.15.42:0/1701389436 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67fc0a58b0 con 0x7f6804160330 2024-05-28T14:01:24.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.408+0000 7f6803400640 1 -- 172.21.15.42:0/1701389436 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0a5c80 con 0x7f6804160330 2024-05-28T14:01:24.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.408+0000 7f680b600640 1 -- 172.21.15.42:0/1701389436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6804160330 msgr2=0x7f68041607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:24.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.408+0000 7f680b600640 1 --2- 172.21.15.42:0/1701389436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6804160330 0x7f68041607a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009d90 tx=0x7f67fc01d340 comp rx=0 tx=0).stop 2024-05-28T14:01:24.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.408+0000 7f680b600640 1 -- 172.21.15.42:0/1701389436 shutdown_connections 2024-05-28T14:01:24.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.408+0000 7f680b600640 1 --2- 172.21.15.42:0/1701389436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6804160330 0x7f68041607a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:24.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.408+0000 7f680b600640 1 --2- 172.21.15.42:0/1701389436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68040cd290 0x7f680415d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:24.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.408+0000 7f680b600640 1 -- 172.21.15.42:0/1701389436 >> 172.21.15.42:0/1701389436 conn(0x7f680409aa50 msgr2=0x7f68040b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:24.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.408+0000 7f680b600640 1 -- 172.21.15.42:0/1701389436 shutdown_connections 2024-05-28T14:01:24.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.408+0000 7f680b600640 1 -- 172.21.15.42:0/1701389436 wait complete. 2024-05-28T14:01:24.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f680b600640 1 Processor -- start 2024-05-28T14:01:24.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f680b600640 1 -- start start 2024-05-28T14:01:24.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f680b600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68040cd290 0x7f68040c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:24.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f680b600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68040cd670 0x7f6804151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f680b600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6804160330 0x7f680415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f680b600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f680406c100 con 0x7f68040cd290 2024-05-28T14:01:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f680b600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68040cdab0 con 0x7f68040cd670 2024-05-28T14:01:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f680b600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f68040c5410 con 0x7f6804160330 2024-05-28T14:01:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6808c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68040cd290 0x7f68040c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6808c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68040cd290 0x7f68040c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57178/0 (socket says 172.21.15.42:57178) 2024-05-28T14:01:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6808c00640 1 -- 172.21.15.42:0/3653277501 learned_addr learned my addr 172.21.15.42:0/3653277501 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6803e00640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68040cd670 0x7f6804151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6808c00640 1 -- 172.21.15.42:0/3653277501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6804160330 msgr2=0x7f680415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6809600640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6804160330 0x7f680415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6808c00640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6804160330 0x7f680415b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6808c00640 1 -- 172.21.15.42:0/3653277501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68040cd670 msgr2=0x7f6804151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6808c00640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68040cd670 0x7f6804151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:24.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6808c00640 1 -- 172.21.15.42:0/3653277501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6804070e90 con 0x7f68040cd290 2024-05-28T14:01:24.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6803e00640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68040cd670 0x7f6804151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:24.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6809600640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6804160330 0x7f680415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:24.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6808c00640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68040cd290 0x7f68040c77a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f67f800d620 tx=0x7f67f800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:24.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6800c00640 1 -- 172.21.15.42:0/3653277501 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f800b620 con 0x7f68040cd290 2024-05-28T14:01:24.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6800c00640 1 -- 172.21.15.42:0/3653277501 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f8019070 con 0x7f68040cd290 2024-05-28T14:01:24.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f6800c00640 1 -- 172.21.15.42:0/3653277501 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f800b9e0 con 0x7f68040cd290 2024-05-28T14:01:24.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f680b600640 1 -- 172.21.15.42:0/3653277501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68040c50e0 con 0x7f68040cd290 2024-05-28T14:01:24.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f680b600640 1 -- 172.21.15.42:0/3653277501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6804070bb0 con 0x7f68040cd290 2024-05-28T14:01:24.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.412+0000 7f680b600640 1 -- 172.21.15.42:0/3653277501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6804068a90 con 0x7f68040cd290 2024-05-28T14:01:24.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.416+0000 7f6800c00640 1 -- 172.21.15.42:0/3653277501 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7f67f800bb80 con 0x7f68040cd290 2024-05-28T14:01:24.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.416+0000 7f6800c00640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f67e4068530 0x7f67e406a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:24.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.416+0000 7f6800c00640 1 -- 172.21.15.42:0/3653277501 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f67f80870c0 con 0x7f68040cd290 2024-05-28T14:01:24.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.416+0000 7f6803e00640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f67e4068530 0x7f67e406a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:24.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.416+0000 7f6803e00640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f67e4068530 0x7f67e406a9f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6804156be0 tx=0x7f67f0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:24.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.420+0000 7f6800c00640 1 -- 172.21.15.42:0/3653277501 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f67f80573f0 con 0x7f68040cd290 2024-05-28T14:01:24.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.644+0000 7f680b600640 1 -- 172.21.15.42:0/3653277501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f680409af30 con 0x7f68040cd290 2024-05-28T14:01:24.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.644+0000 7f6800c00640 1 -- 172.21.15.42:0/3653277501 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2078 (secure 0 0 0) 0x7f67f8056cf0 con 0x7f68040cd290 2024-05-28T14:01:24.629 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:24.629 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:01:24.629 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:01:24.629 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:01:24.629 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:01:24.629 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:01:24.630 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:01:24.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:24.630 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:01:24.630 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:01:24.630 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:01:24.630 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:01:24.630 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:01:24.630 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:24.630 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:01:24.630 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:01:24.630 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:01:24.630 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:01:24.630 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:24.630 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 205, 2024-05-28T14:01:24.631 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:01:24.631 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:01:24.631 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:01:24.631 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:01:24.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:24.631 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:01:24.631 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 34, 2024-05-28T14:01:24.631 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:01:24.631 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:01:24.631 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:01:24.631 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:01:24.631 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:01:24.631 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:01:24.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 2845, 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 11812500170, 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 23892090880, 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 749202022400, 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:01:24.632 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 341, 2024-05-28T14:01:24.633 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:01:24.633 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:01:24.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:24.633 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:01:24.633 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:01:24.633 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2, 2024-05-28T14:01:24.633 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:01:24.633 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:01:24.633 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:01:24.633 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:01:24.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:24.633 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 2, 2024-05-28T14:01:24.633 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:01:24.633 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:01:24.634 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:01:24.634 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 54785 2024-05-28T14:01:24.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:24.634 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:24.634 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:01:24.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:24.634 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:01:24.634 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:01:24.634 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:01:24.634 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:01:24.634 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:01:24.634 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:01:24.634 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:01:24.635 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:24.635 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:01:24.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:24.635 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:01:24.635 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 8, 2024-05-28T14:01:24.635 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:00:32.881524+0000", 2024-05-28T14:01:24.635 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:01:24.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:24.635 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:01:24.635 INFO:teuthology.orchestra.run.smithi042.stdout: "2634ebf7-b822-4613-b0f8-227d147fb4c4": { 2024-05-28T14:01:24.635 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 5.0 seconds before exiting (28s)\n [============================] ", 2024-05-28T14:01:24.635 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:01:24.635 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:01:24.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:24.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:24.636 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:01:24.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.644+0000 7f680b600640 1 -- 172.21.15.42:0/3653277501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f67e4068530 msgr2=0x7f67e406a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:24.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.644+0000 7f680b600640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f67e4068530 0x7f67e406a9f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6804156be0 tx=0x7f67f0009040 comp rx=0 tx=0).stop 2024-05-28T14:01:24.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.644+0000 7f680b600640 1 -- 172.21.15.42:0/3653277501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68040cd290 msgr2=0x7f68040c77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:24.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.644+0000 7f680b600640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68040cd290 0x7f68040c77a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f67f800d620 tx=0x7f67f800dae0 comp rx=0 tx=0).stop 2024-05-28T14:01:24.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.644+0000 7f680b600640 1 -- 172.21.15.42:0/3653277501 shutdown_connections 2024-05-28T14:01:24.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.644+0000 7f680b600640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f67e4068530 0x7f67e406a9f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:24.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.644+0000 7f680b600640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6804160330 0x7f680415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:24.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.644+0000 7f680b600640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68040cd670 0x7f6804151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:24.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.644+0000 7f680b600640 1 --2- 172.21.15.42:0/3653277501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68040cd290 0x7f68040c77a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:24.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.644+0000 7f680b600640 1 -- 172.21.15.42:0/3653277501 >> 172.21.15.42:0/3653277501 conn(0x7f680409aa50 msgr2=0x7f68040b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:24.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.644+0000 7f680b600640 1 -- 172.21.15.42:0/3653277501 shutdown_connections 2024-05-28T14:01:24.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:24.644+0000 7f680b600640 1 -- 172.21.15.42:0/3653277501 wait complete. 2024-05-28T14:01:29.652 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:01:29.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa3000640 1 Processor -- start 2024-05-28T14:01:29.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa3000640 1 -- start start 2024-05-28T14:01:29.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa3000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba9c15a160 0x7fba9c15ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:29.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa3000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fba9c0cef90 0x7fba9c0cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:29.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa3000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba9c15d170 con 0x7fba9c1609d0 2024-05-28T14:01:29.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa3000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fba9c0cf920 con 0x7fba9c0cef90 2024-05-28T14:01:29.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa3000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fba9c15da10 con 0x7fba9c15a160 2024-05-28T14:01:29.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa2600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fba9c0cef90 0x7fba9c0cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:29.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba9c15a160 0x7fba9c15ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:29.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba9c15a160 0x7fba9c15ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:44972/0 (socket says 172.21.15.42:44972) 2024-05-28T14:01:29.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa2600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fba9c0cef90 0x7fba9c0cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:42548/0 (socket says 172.21.15.42:42548) 2024-05-28T14:01:29.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa1c00640 1 -- 172.21.15.42:0/2733417158 learned_addr learned my addr 172.21.15.42:0/2733417158 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:29.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa1200640 1 --1- 172.21.15.42:0/2733417158 >> v1:172.21.15.63:6789/0 conn(0x7fba9c1609d0 0x7fba9c0cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:55446/0 (socket says 172.21.15.42:55446) 2024-05-28T14:01:29.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa2600640 1 -- 172.21.15.42:0/2733417158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba9c15a160 msgr2=0x7fba9c15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:29.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa2600640 1 --2- 172.21.15.42:0/2733417158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba9c15a160 0x7fba9c15ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:29.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa2600640 1 -- 172.21.15.42:0/2733417158 >> v1:172.21.15.63:6789/0 conn(0x7fba9c1609d0 legacy=0x7fba9c0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:29.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa2600640 1 -- 172.21.15.42:0/2733417158 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9c15d450 con 0x7fba9c0cef90 2024-05-28T14:01:29.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.820+0000 7fbaa1c00640 1 --2- 172.21.15.42:0/2733417158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba9c15a160 0x7fba9c15ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:01:29.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.824+0000 7fbaa2600640 1 --2- 172.21.15.42:0/2733417158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba9c0cef90 0x7fba9c0cf3e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba94009d90 tx=0x7fba9401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=719e6e546a42deb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:29.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.824+0000 7fba9be00640 1 -- 172.21.15.42:0/2733417158 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba940b2070 con 0x7fba9c0cef90 2024-05-28T14:01:29.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.824+0000 7fba9be00640 1 -- 172.21.15.42:0/2733417158 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba940a5940 con 0x7fba9c0cef90 2024-05-28T14:01:29.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.824+0000 7fba9be00640 1 -- 172.21.15.42:0/2733417158 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba940a5d10 con 0x7fba9c0cef90 2024-05-28T14:01:29.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.824+0000 7fbaa3000640 1 -- 172.21.15.42:0/2733417158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba9c0cef90 msgr2=0x7fba9c0cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:29.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.824+0000 7fbaa3000640 1 --2- 172.21.15.42:0/2733417158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba9c0cef90 0x7fba9c0cf3e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba94009d90 tx=0x7fba9401c970 comp rx=0 tx=0).stop 2024-05-28T14:01:29.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.824+0000 7fbaa3000640 1 -- 172.21.15.42:0/2733417158 shutdown_connections 2024-05-28T14:01:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.824+0000 7fbaa3000640 1 --2- 172.21.15.42:0/2733417158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba9c0cef90 0x7fba9c0cf3e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.824+0000 7fbaa3000640 1 --2- 172.21.15.42:0/2733417158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba9c15a160 0x7fba9c15ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.824+0000 7fbaa3000640 1 -- 172.21.15.42:0/2733417158 >> 172.21.15.42:0/2733417158 conn(0x7fba9c0b3ec0 msgr2=0x7fba9c0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.824+0000 7fbaa3000640 1 -- 172.21.15.42:0/2733417158 shutdown_connections 2024-05-28T14:01:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.824+0000 7fbaa3000640 1 -- 172.21.15.42:0/2733417158 wait complete. 2024-05-28T14:01:29.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa3000640 1 Processor -- start 2024-05-28T14:01:29.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa3000640 1 -- start start 2024-05-28T14:01:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa3000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba9c0cef90 0x7fba9c151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa3000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba9c15a160 0x7fba9c15def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa3000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba9c1609d0 0x7fba9c157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa3000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c169390 con 0x7fba9c15a160 2024-05-28T14:01:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa3000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c160dd0 con 0x7fba9c1609d0 2024-05-28T14:01:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa3000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fba9c169510 con 0x7fba9c0cef90 2024-05-28T14:01:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba9c15a160 0x7fba9c15def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba9c0cef90 0x7fba9c151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba9c15a160 0x7fba9c15def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57570/0 (socket says 172.21.15.42:57570) 2024-05-28T14:01:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa1200640 1 -- 172.21.15.42:0/3930732842 learned_addr learned my addr 172.21.15.42:0/3930732842 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa2600640 1 --2- 172.21.15.42:0/3930732842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba9c1609d0 0x7fba9c157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa1200640 1 -- 172.21.15.42:0/3930732842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba9c0cef90 msgr2=0x7fba9c151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa1200640 1 --2- 172.21.15.42:0/3930732842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba9c0cef90 0x7fba9c151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa1200640 1 -- 172.21.15.42:0/3930732842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba9c1609d0 msgr2=0x7fba9c157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa1200640 1 --2- 172.21.15.42:0/3930732842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba9c1609d0 0x7fba9c157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa1200640 1 -- 172.21.15.42:0/3930732842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9c15e430 con 0x7fba9c15a160 2024-05-28T14:01:29.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa1200640 1 --2- 172.21.15.42:0/3930732842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba9c15a160 0x7fba9c15def0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fba8800c3a0 tx=0x7fba8800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:29.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fba99600640 1 -- 172.21.15.42:0/3930732842 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba880074e0 con 0x7fba9c15a160 2024-05-28T14:01:29.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fba99600640 1 -- 172.21.15.42:0/3930732842 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba88007680 con 0x7fba9c15a160 2024-05-28T14:01:29.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa3000640 1 -- 172.21.15.42:0/3930732842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9c1524f0 con 0x7fba9c15a160 2024-05-28T14:01:29.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fba99600640 1 -- 172.21.15.42:0/3930732842 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88004a30 con 0x7fba9c15a160 2024-05-28T14:01:29.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.828+0000 7fbaa3000640 1 -- 172.21.15.42:0/3930732842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba9c157bd0 con 0x7fba9c15a160 2024-05-28T14:01:29.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.832+0000 7fbaa3000640 1 -- 172.21.15.42:0/3930732842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba9c068a90 con 0x7fba9c15a160 2024-05-28T14:01:29.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.832+0000 7fba99600640 1 -- 172.21.15.42:0/3930732842 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7fba8801a020 con 0x7fba9c15a160 2024-05-28T14:01:29.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.832+0000 7fba99600640 1 --2- 172.21.15.42:0/3930732842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba7c068530 0x7fba7c06a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:29.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.832+0000 7fba99600640 1 -- 172.21.15.42:0/3930732842 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fba880862c0 con 0x7fba9c15a160 2024-05-28T14:01:29.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.832+0000 7fbaa1c00640 1 --2- 172.21.15.42:0/3930732842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba7c068530 0x7fba7c06a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:29.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.832+0000 7fbaa1c00640 1 --2- 172.21.15.42:0/3930732842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba7c068530 0x7fba7c06a9f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fba90006cb0 tx=0x7fba90009240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:29.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:29.836+0000 7fba99600640 1 -- 172.21.15.42:0/3930732842 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fba88056040 con 0x7fba9c15a160 2024-05-28T14:01:30.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.064+0000 7fbaa3000640 1 -- 172.21.15.42:0/3930732842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fba9c071030 con 0x7fba9c15a160 2024-05-28T14:01:30.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.064+0000 7fba99600640 1 -- 172.21.15.42:0/3930732842 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1824 (secure 0 0 0) 0x7fba880860e0 con 0x7fba9c15a160 2024-05-28T14:01:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:30.052 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:01:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:01:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:01:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:01:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:01:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:01:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:30.052 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:01:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:01:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:01:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:01:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:01:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:01:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:01:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:01:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:01:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 211, 2024-05-28T14:01:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:01:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:01:30.053 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:01:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:01:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:01:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 34, 2024-05-28T14:01:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:01:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:01:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:01:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:01:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:01:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:01:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:30.054 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:01:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:01:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:01:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:01:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:01:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:01:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 2845, 2024-05-28T14:01:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 11812500170, 2024-05-28T14:01:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 23892090880, 2024-05-28T14:01:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 749202022400, 2024-05-28T14:01:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:01:30.055 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 341, 2024-05-28T14:01:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:01:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:01:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:01:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:01:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 2, 2024-05-28T14:01:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:01:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:01:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:01:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:01:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 2, 2024-05-28T14:01:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:01:30.056 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:01:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:01:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 54785 2024-05-28T14:01:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:01:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:01:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:01:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:01:30.057 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:01:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:01:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:01:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:01:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:01:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:01:30.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:01:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 8, 2024-05-28T14:01:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:00:32.881524+0000", 2024-05-28T14:01:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:01:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:30.059 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": {} 2024-05-28T14:01:30.059 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:01:30.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.068+0000 7fbaa3000640 1 -- 172.21.15.42:0/3930732842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba7c068530 msgr2=0x7fba7c06a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.068+0000 7fbaa3000640 1 --2- 172.21.15.42:0/3930732842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba7c068530 0x7fba7c06a9f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fba90006cb0 tx=0x7fba90009240 comp rx=0 tx=0).stop 2024-05-28T14:01:30.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.068+0000 7fbaa3000640 1 -- 172.21.15.42:0/3930732842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba9c15a160 msgr2=0x7fba9c15def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.068+0000 7fbaa3000640 1 --2- 172.21.15.42:0/3930732842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba9c15a160 0x7fba9c15def0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fba8800c3a0 tx=0x7fba8800c860 comp rx=0 tx=0).stop 2024-05-28T14:01:30.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.068+0000 7fbaa3000640 1 -- 172.21.15.42:0/3930732842 shutdown_connections 2024-05-28T14:01:30.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.068+0000 7fbaa3000640 1 --2- 172.21.15.42:0/3930732842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba7c068530 0x7fba7c06a9f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.068+0000 7fbaa3000640 1 --2- 172.21.15.42:0/3930732842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba9c1609d0 0x7fba9c157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.068+0000 7fbaa3000640 1 --2- 172.21.15.42:0/3930732842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba9c15a160 0x7fba9c15def0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.068+0000 7fbaa3000640 1 --2- 172.21.15.42:0/3930732842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba9c0cef90 0x7fba9c151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.068+0000 7fbaa3000640 1 -- 172.21.15.42:0/3930732842 >> 172.21.15.42:0/3930732842 conn(0x7fba9c0b3ec0 msgr2=0x7fba9c152fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:30.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.068+0000 7fbaa3000640 1 -- 172.21.15.42:0/3930732842 shutdown_connections 2024-05-28T14:01:30.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.068+0000 7fbaa3000640 1 -- 172.21.15.42:0/3930732842 wait complete. 2024-05-28T14:01:30.074 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-28T14:01:30.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35dde00640 1 Processor -- start 2024-05-28T14:01:30.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35dde00640 1 -- start start 2024-05-28T14:01:30.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35dde00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815fcf0 0x7f35d80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35dde00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f35d806c9d0 0x7f35d80b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35dde00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d80d1480 con 0x7f35d80b2af0 2024-05-28T14:01:30.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35dde00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f35d80b3b80 con 0x7f35d806c9d0 2024-05-28T14:01:30.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35dde00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f35d80b1e40 con 0x7f35d815fcf0 2024-05-28T14:01:30.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35d6a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815fcf0 0x7f35d80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:30.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35ce000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f35d806c9d0 0x7f35d80b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:30.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35d6a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815fcf0 0x7f35d80b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:44992/0 (socket says 172.21.15.42:44992) 2024-05-28T14:01:30.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35d6a00640 1 -- 172.21.15.42:0/1513715180 learned_addr learned my addr 172.21.15.42:0/1513715180 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:30.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35d6a00640 1 -- 172.21.15.42:0/1513715180 >> v2:172.21.15.63:3300/0 conn(0x7f35d806c9d0 msgr2=0x7f35d80b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:30.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35d6a00640 1 --2- 172.21.15.42:0/1513715180 >> v2:172.21.15.63:3300/0 conn(0x7f35d806c9d0 0x7f35d80b25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35d6a00640 1 -- 172.21.15.42:0/1513715180 >> v1:172.21.15.63:6789/0 conn(0x7f35d80b2af0 legacy=0x7f35d809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.232+0000 7f35d6a00640 1 -- 172.21.15.42:0/1513715180 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d80d1760 con 0x7f35d815fcf0 2024-05-28T14:01:30.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.236+0000 7f35d6a00640 1 --2- 172.21.15.42:0/1513715180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815fcf0 0x7f35d80b3410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009870 tx=0x7f35c801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99d7a243505fdb5c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:30.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.236+0000 7f35d6000640 1 -- 172.21.15.42:0/1513715180 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c80a7070 con 0x7f35d815fcf0 2024-05-28T14:01:30.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.236+0000 7f35d6000640 1 -- 172.21.15.42:0/1513715180 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c80b0070 con 0x7f35d815fcf0 2024-05-28T14:01:30.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.236+0000 7f35d6000640 1 -- 172.21.15.42:0/1513715180 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c80ab890 con 0x7f35d815fcf0 2024-05-28T14:01:30.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.236+0000 7f35dde00640 1 -- 172.21.15.42:0/1513715180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815fcf0 msgr2=0x7f35d80b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.236+0000 7f35dde00640 1 --2- 172.21.15.42:0/1513715180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815fcf0 0x7f35d80b3410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009870 tx=0x7f35c801c560 comp rx=0 tx=0).stop 2024-05-28T14:01:30.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.236+0000 7f35dde00640 1 -- 172.21.15.42:0/1513715180 shutdown_connections 2024-05-28T14:01:30.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.236+0000 7f35dde00640 1 --2- 172.21.15.42:0/1513715180 >> v2:172.21.15.63:3300/0 conn(0x7f35d806c9d0 0x7f35d80b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.236+0000 7f35dde00640 1 --2- 172.21.15.42:0/1513715180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815fcf0 0x7f35d80b3410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.236+0000 7f35dde00640 1 -- 172.21.15.42:0/1513715180 >> 172.21.15.42:0/1513715180 conn(0x7f35d80b3ff0 msgr2=0x7f35d80b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:30.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.236+0000 7f35dde00640 1 -- 172.21.15.42:0/1513715180 shutdown_connections 2024-05-28T14:01:30.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.236+0000 7f35dde00640 1 -- 172.21.15.42:0/1513715180 wait complete. 2024-05-28T14:01:30.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35dde00640 1 Processor -- start 2024-05-28T14:01:30.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35dde00640 1 -- start start 2024-05-28T14:01:30.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35dde00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35d806c9d0 0x7f35d814c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35dde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35d80b2af0 0x7f35d8149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35dde00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815fcf0 0x7f35d814a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35dde00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f35d816bc70 con 0x7f35d80b2af0 2024-05-28T14:01:30.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35dde00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f35d816baf0 con 0x7f35d806c9d0 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35dde00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f35d816bdf0 con 0x7f35d815fcf0 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35ce000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35d80b2af0 0x7f35d8149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35ce000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35d80b2af0 0x7f35d8149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57584/0 (socket says 172.21.15.42:57584) 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35ce000640 1 -- 172.21.15.42:0/3687096734 learned_addr learned my addr 172.21.15.42:0/3687096734 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35d6a00640 1 --2- 172.21.15.42:0/3687096734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35d806c9d0 0x7f35d814c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35d7400640 1 --2- 172.21.15.42:0/3687096734 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815fcf0 0x7f35d814a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35ce000640 1 -- 172.21.15.42:0/3687096734 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815fcf0 msgr2=0x7f35d814a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35ce000640 1 --2- 172.21.15.42:0/3687096734 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815fcf0 0x7f35d814a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35ce000640 1 -- 172.21.15.42:0/3687096734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35d806c9d0 msgr2=0x7f35d814c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35ce000640 1 --2- 172.21.15.42:0/3687096734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35d806c9d0 0x7f35d814c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35ce000640 1 -- 172.21.15.42:0/3687096734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d81f91f0 con 0x7f35d80b2af0 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35ce000640 1 --2- 172.21.15.42:0/3687096734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35d80b2af0 0x7f35d8149f50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f35c000e870 tx=0x7f35c000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35cfe00640 1 -- 172.21.15.42:0/3687096734 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0017070 con 0x7f35d80b2af0 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35cfe00640 1 -- 172.21.15.42:0/3687096734 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c000c940 con 0x7f35d80b2af0 2024-05-28T14:01:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35dde00640 1 -- 172.21.15.42:0/3687096734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d814e8d0 con 0x7f35d80b2af0 2024-05-28T14:01:30.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35cfe00640 1 -- 172.21.15.42:0/3687096734 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c000ccf0 con 0x7f35d80b2af0 2024-05-28T14:01:30.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.240+0000 7f35dde00640 1 -- 172.21.15.42:0/3687096734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d814ee50 con 0x7f35d80b2af0 2024-05-28T14:01:30.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.244+0000 7f35dde00640 1 -- 172.21.15.42:0/3687096734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d80a0650 con 0x7f35d80b2af0 2024-05-28T14:01:30.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.244+0000 7f35cfe00640 1 -- 172.21.15.42:0/3687096734 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7f35c001a070 con 0x7f35d80b2af0 2024-05-28T14:01:30.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.244+0000 7f35cfe00640 1 --2- 172.21.15.42:0/3687096734 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f35b4068530 0x7f35b406a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.244+0000 7f35cfe00640 1 -- 172.21.15.42:0/3687096734 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f35c0086a90 con 0x7f35d80b2af0 2024-05-28T14:01:30.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.244+0000 7f35d6a00640 1 --2- 172.21.15.42:0/3687096734 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f35b4068530 0x7f35b406a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.244+0000 7f35d6a00640 1 --2- 172.21.15.42:0/3687096734 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f35b4068530 0x7f35b406a9f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f35c80af000 tx=0x7f35c80a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.248+0000 7f35cfe00640 1 -- 172.21.15.42:0/3687096734 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f35c0056820 con 0x7f35d80b2af0 2024-05-28T14:01:30.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.436+0000 7f35dde00640 1 -- 172.21.15.42:0/3687096734 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35d804fdd0 con 0x7f35b4068530 2024-05-28T14:01:30.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.444+0000 7f35cfe00640 1 -- 172.21.15.42:0/3687096734 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f35d804fdd0 con 0x7f35b4068530 2024-05-28T14:01:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:30.429 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:01:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sv1" 2024-05-28T14:01:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ss1clone1" 2024-05-28T14:01:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:30.430 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:01:30.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.444+0000 7f35dde00640 1 -- 172.21.15.42:0/3687096734 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f35b4068530 msgr2=0x7f35b406a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.444+0000 7f35dde00640 1 --2- 172.21.15.42:0/3687096734 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f35b4068530 0x7f35b406a9f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f35c80af000 tx=0x7f35c80a7000 comp rx=0 tx=0).stop 2024-05-28T14:01:30.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.444+0000 7f35dde00640 1 -- 172.21.15.42:0/3687096734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35d80b2af0 msgr2=0x7f35d8149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.444+0000 7f35dde00640 1 --2- 172.21.15.42:0/3687096734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35d80b2af0 0x7f35d8149f50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f35c000e870 tx=0x7f35c000ec70 comp rx=0 tx=0).stop 2024-05-28T14:01:30.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.444+0000 7f35dde00640 1 -- 172.21.15.42:0/3687096734 shutdown_connections 2024-05-28T14:01:30.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.444+0000 7f35dde00640 1 --2- 172.21.15.42:0/3687096734 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f35b4068530 0x7f35b406a9f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.444+0000 7f35dde00640 1 --2- 172.21.15.42:0/3687096734 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815fcf0 0x7f35d814a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.444+0000 7f35dde00640 1 --2- 172.21.15.42:0/3687096734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35d80b2af0 0x7f35d8149f50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.444+0000 7f35dde00640 1 --2- 172.21.15.42:0/3687096734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35d806c9d0 0x7f35d814c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.444+0000 7f35dde00640 1 -- 172.21.15.42:0/3687096734 >> 172.21.15.42:0/3687096734 conn(0x7f35d80b3ff0 msgr2=0x7f35d80afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:30.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.444+0000 7f35dde00640 1 -- 172.21.15.42:0/3687096734 shutdown_connections 2024-05-28T14:01:30.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.444+0000 7f35dde00640 1 -- 172.21.15.42:0/3687096734 wait complete. 2024-05-28T14:01:30.450 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-28T14:01:30.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.608+0000 7f2ac6800640 1 Processor -- start 2024-05-28T14:01:30.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.608+0000 7f2ac6800640 1 -- start start 2024-05-28T14:01:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.608+0000 7f2ac6800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ac015b350 0x7f2ac006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.608+0000 7f2ac6800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2ac00d0e00 0x7f2ac00aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.608+0000 7f2ac6800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac00ade40 con 0x7f2ac0072d40 2024-05-28T14:01:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.608+0000 7f2ac6800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f2ac00ae9f0 con 0x7f2ac00d0e00 2024-05-28T14:01:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.608+0000 7f2ac6800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac00ae0c0 con 0x7f2ac015b350 2024-05-28T14:01:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.612+0000 7f2abfe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2ac00d0e00 0x7f2ac00aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.612+0000 7f2abf400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ac015b350 0x7f2ac006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.612+0000 7f2abfe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2ac00d0e00 0x7f2ac00aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57590/0 (socket says 172.21.15.42:57590) 2024-05-28T14:01:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.612+0000 7f2abfe00640 1 -- 172.21.15.42:0/2755554216 learned_addr learned my addr 172.21.15.42:0/2755554216 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.612+0000 7f2abfe00640 1 -- 172.21.15.42:0/2755554216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ac015b350 msgr2=0x7f2ac006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.612+0000 7f2abfe00640 1 --2- 172.21.15.42:0/2755554216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ac015b350 0x7f2ac006dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.612+0000 7f2abfe00640 1 -- 172.21.15.42:0/2755554216 >> v1:172.21.15.63:6789/0 conn(0x7f2ac0072d40 legacy=0x7f2ac00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.612+0000 7f2abfe00640 1 -- 172.21.15.42:0/2755554216 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac00ae3a0 con 0x7f2ac00d0e00 2024-05-28T14:01:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.612+0000 7f2abf400640 1 --2- 172.21.15.42:0/2755554216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ac015b350 0x7f2ac006dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:01:30.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.612+0000 7f2abfe00640 1 --2- 172.21.15.42:0/2755554216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ac00d0e00 0x7f2ac00aee80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009e50 tx=0x7f2ab801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7344d5d18b8b32d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:30.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.612+0000 7f2abea00640 1 -- 172.21.15.42:0/2755554216 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab80a56e0 con 0x7f2ac00d0e00 2024-05-28T14:01:30.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.612+0000 7f2abea00640 1 -- 172.21.15.42:0/2755554216 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab80a5880 con 0x7f2ac00d0e00 2024-05-28T14:01:30.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.612+0000 7f2abea00640 1 -- 172.21.15.42:0/2755554216 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab80a5c50 con 0x7f2ac00d0e00 2024-05-28T14:01:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 -- 172.21.15.42:0/2755554216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ac00d0e00 msgr2=0x7f2ac00aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 --2- 172.21.15.42:0/2755554216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ac00d0e00 0x7f2ac00aee80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009e50 tx=0x7f2ab801c9a0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 -- 172.21.15.42:0/2755554216 shutdown_connections 2024-05-28T14:01:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 --2- 172.21.15.42:0/2755554216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ac00d0e00 0x7f2ac00aee80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 --2- 172.21.15.42:0/2755554216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ac015b350 0x7f2ac006dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 -- 172.21.15.42:0/2755554216 >> 172.21.15.42:0/2755554216 conn(0x7f2ac006bdf0 msgr2=0x7f2ac006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 -- 172.21.15.42:0/2755554216 shutdown_connections 2024-05-28T14:01:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 -- 172.21.15.42:0/2755554216 wait complete. 2024-05-28T14:01:30.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 Processor -- start 2024-05-28T14:01:30.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 -- start start 2024-05-28T14:01:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ac0072d40 0x7f2ac0154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ac00d0e00 0x7f2ac0160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ac015b350 0x7f2ac0161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac0168470 con 0x7f2ac0072d40 2024-05-28T14:01:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01682f0 con 0x7f2ac015b350 2024-05-28T14:01:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ac6800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac01685f0 con 0x7f2ac00d0e00 2024-05-28T14:01:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2abf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ac0072d40 0x7f2ac0154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2abf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ac0072d40 0x7f2ac0154340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57606/0 (socket says 172.21.15.42:57606) 2024-05-28T14:01:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2abf400640 1 -- 172.21.15.42:0/4069274593 learned_addr learned my addr 172.21.15.42:0/4069274593 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2ab7e00640 1 --2- 172.21.15.42:0/4069274593 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ac00d0e00 0x7f2ac0160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2abfe00640 1 --2- 172.21.15.42:0/4069274593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ac015b350 0x7f2ac0161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2abf400640 1 -- 172.21.15.42:0/4069274593 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ac00d0e00 msgr2=0x7f2ac0160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2abf400640 1 --2- 172.21.15.42:0/4069274593 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ac00d0e00 0x7f2ac0160b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2abf400640 1 -- 172.21.15.42:0/4069274593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ac015b350 msgr2=0x7f2ac0161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2abf400640 1 --2- 172.21.15.42:0/4069274593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ac015b350 0x7f2ac0161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.616+0000 7f2abf400640 1 -- 172.21.15.42:0/4069274593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac015cd50 con 0x7f2ac0072d40 2024-05-28T14:01:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.620+0000 7f2abf400640 1 --2- 172.21.15.42:0/4069274593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ac0072d40 0x7f2ac0154340 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000e9b0 tx=0x7f2ab000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.620+0000 7f2ab7400640 1 -- 172.21.15.42:0/4069274593 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0017070 con 0x7f2ac0072d40 2024-05-28T14:01:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.620+0000 7f2ab7400640 1 -- 172.21.15.42:0/4069274593 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab000ca00 con 0x7f2ac0072d40 2024-05-28T14:01:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.620+0000 7f2ac6800640 1 -- 172.21.15.42:0/4069274593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac015d030 con 0x7f2ac0072d40 2024-05-28T14:01:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.620+0000 7f2ab7400640 1 -- 172.21.15.42:0/4069274593 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab000cdd0 con 0x7f2ac0072d40 2024-05-28T14:01:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.620+0000 7f2ac6800640 1 -- 172.21.15.42:0/4069274593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac015d5f0 con 0x7f2ac0072d40 2024-05-28T14:01:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.620+0000 7f2ac6800640 1 -- 172.21.15.42:0/4069274593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac0068a90 con 0x7f2ac0072d40 2024-05-28T14:01:30.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.620+0000 7f2ab7400640 1 -- 172.21.15.42:0/4069274593 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7f2ab001a070 con 0x7f2ac0072d40 2024-05-28T14:01:30.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.620+0000 7f2ab7400640 1 --2- 172.21.15.42:0/4069274593 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2a9c068530 0x7f2a9c06a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.620+0000 7f2ab7400640 1 -- 172.21.15.42:0/4069274593 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f2ab0086f30 con 0x7f2ac0072d40 2024-05-28T14:01:30.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.620+0000 7f2ab7e00640 1 --2- 172.21.15.42:0/4069274593 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2a9c068530 0x7f2a9c06a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:30.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.624+0000 7f2ab7e00640 1 --2- 172.21.15.42:0/4069274593 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2a9c068530 0x7f2a9c06a9f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2ac015a730 tx=0x7f2aa8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:30.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.624+0000 7f2ab7400640 1 -- 172.21.15.42:0/4069274593 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2ab0057260 con 0x7f2ac0072d40 2024-05-28T14:01:30.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.816+0000 7f2ac6800640 1 -- 172.21.15.42:0/4069274593 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ac00b1880 con 0x7f2a9c068530 2024-05-28T14:01:30.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.836+0000 7f2ab7400640 1 -- 172.21.15.42:0/4069274593 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2ac00b1880 con 0x7f2a9c068530 2024-05-28T14:01:30.821 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:30.821 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:01:30.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:30.821 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ss1" 2024-05-28T14:01:30.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:30.821 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:01:30.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.836+0000 7f2ac6800640 1 -- 172.21.15.42:0/4069274593 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2a9c068530 msgr2=0x7f2a9c06a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.836+0000 7f2ac6800640 1 --2- 172.21.15.42:0/4069274593 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2a9c068530 0x7f2a9c06a9f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2ac015a730 tx=0x7f2aa8009040 comp rx=0 tx=0).stop 2024-05-28T14:01:30.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.836+0000 7f2ac6800640 1 -- 172.21.15.42:0/4069274593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ac0072d40 msgr2=0x7f2ac0154340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.836+0000 7f2ac6800640 1 --2- 172.21.15.42:0/4069274593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ac0072d40 0x7f2ac0154340 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000e9b0 tx=0x7f2ab000ee70 comp rx=0 tx=0).stop 2024-05-28T14:01:30.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.836+0000 7f2ac6800640 1 -- 172.21.15.42:0/4069274593 shutdown_connections 2024-05-28T14:01:30.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.836+0000 7f2ac6800640 1 --2- 172.21.15.42:0/4069274593 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2a9c068530 0x7f2a9c06a9f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.836+0000 7f2ac6800640 1 --2- 172.21.15.42:0/4069274593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ac015b350 0x7f2ac0161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.836+0000 7f2ac6800640 1 --2- 172.21.15.42:0/4069274593 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ac00d0e00 0x7f2ac0160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.836+0000 7f2ac6800640 1 --2- 172.21.15.42:0/4069274593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ac0072d40 0x7f2ac0154340 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.836+0000 7f2ac6800640 1 -- 172.21.15.42:0/4069274593 >> 172.21.15.42:0/4069274593 conn(0x7f2ac006bdf0 msgr2=0x7f2ac00b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:30.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.836+0000 7f2ac6800640 1 -- 172.21.15.42:0/4069274593 shutdown_connections 2024-05-28T14:01:30.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:30.836+0000 7f2ac6800640 1 -- 172.21.15.42:0/4069274593 wait complete. 2024-05-28T14:01:30.841 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:01:30.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.004+0000 7f14f3c00640 1 Processor -- start 2024-05-28T14:01:30.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.004+0000 7f14f3c00640 1 -- start start 2024-05-28T14:01:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.004+0000 7f14f3c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14ec072670 0x7f14ec072a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.004+0000 7f14f3c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f14ec0cce90 0x7f14ec15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.004+0000 7f14f3c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14ec072f90 con 0x7f14ec0b3cd0 2024-05-28T14:01:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.004+0000 7f14f3c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f14ec0b5870 con 0x7f14ec0cce90 2024-05-28T14:01:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.004+0000 7f14f3c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f14ec0b34f0 con 0x7f14ec072670 2024-05-28T14:01:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.004+0000 7f14e3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f14ec0cce90 0x7f14ec15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.004+0000 7f14f1c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f14ec0b3cd0 0x7f14ec06dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:49716/0 (socket says 172.21.15.42:49716) 2024-05-28T14:01:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.004+0000 7f14f1c00640 1 -- 172.21.15.42:0/2562381035 learned_addr learned my addr 172.21.15.42:0/2562381035 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.004+0000 7f14f1200640 1 --2- 172.21.15.42:0/2562381035 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14ec072670 0x7f14ec072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14e3e00640 1 -- 172.21.15.42:0/2562381035 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14ec072670 msgr2=0x7f14ec072a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14e3e00640 1 --2- 172.21.15.42:0/2562381035 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14ec072670 0x7f14ec072a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14e3e00640 1 -- 172.21.15.42:0/2562381035 >> v1:172.21.15.42:6789/0 conn(0x7f14ec0b3cd0 legacy=0x7f14ec06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14e3e00640 1 -- 172.21.15.42:0/2562381035 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec15ba90 con 0x7f14ec0cce90 2024-05-28T14:01:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14ebe00640 1 -- 172.21.15.42:0/2562381035 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3884534998 0 0) 0x7f14ec072f90 con 0x7f14ec0b3cd0 2024-05-28T14:01:30.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14e3e00640 1 --2- 172.21.15.42:0/2562381035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec0cce90 0x7f14ec15b1f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f14d400a960 tx=0x7f14d401d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cd4f2632db02a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:30.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14ebe00640 1 -- 172.21.15.42:0/2562381035 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d40b3070 con 0x7f14ec0cce90 2024-05-28T14:01:30.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14ebe00640 1 -- 172.21.15.42:0/2562381035 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d40a6740 con 0x7f14ec0cce90 2024-05-28T14:01:30.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14ebe00640 1 -- 172.21.15.42:0/2562381035 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d400b040 con 0x7f14ec0cce90 2024-05-28T14:01:30.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14f3c00640 1 -- 172.21.15.42:0/2562381035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec0cce90 msgr2=0x7f14ec15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:30.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14f3c00640 1 --2- 172.21.15.42:0/2562381035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec0cce90 0x7f14ec15b1f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f14d400a960 tx=0x7f14d401d870 comp rx=0 tx=0).stop 2024-05-28T14:01:30.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14f3c00640 1 -- 172.21.15.42:0/2562381035 shutdown_connections 2024-05-28T14:01:30.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14f3c00640 1 --2- 172.21.15.42:0/2562381035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec0cce90 0x7f14ec15b1f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14f3c00640 1 --2- 172.21.15.42:0/2562381035 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14ec072670 0x7f14ec072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:30.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.008+0000 7f14f3c00640 1 -- 172.21.15.42:0/2562381035 >> 172.21.15.42:0/2562381035 conn(0x7f14ec0b21f0 msgr2=0x7f14ec0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:30.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f3c00640 1 -- 172.21.15.42:0/2562381035 shutdown_connections 2024-05-28T14:01:30.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f3c00640 1 -- 172.21.15.42:0/2562381035 wait complete. 2024-05-28T14:01:30.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f3c00640 1 Processor -- start 2024-05-28T14:01:30.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f3c00640 1 -- start start 2024-05-28T14:01:30.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f3c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec072670 0x7f14ec15df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f3c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14ec0b3cd0 0x7f14ec151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:30.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f3c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14ec0cce90 0x7f14ec157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f3c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec167bd0 con 0x7f14ec072670 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f3c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec15b730 con 0x7f14ec0b3cd0 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f3c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f14ec167d50 con 0x7f14ec0cce90 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec072670 0x7f14ec15df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec072670 0x7f14ec15df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57622/0 (socket says 172.21.15.42:57622) 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f1200640 1 -- 172.21.15.42:0/4255377351 learned_addr learned my addr 172.21.15.42:0/4255377351 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14e3e00640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14ec0b3cd0 0x7f14ec151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f1c00640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14ec0cce90 0x7f14ec157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f1200640 1 -- 172.21.15.42:0/4255377351 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14ec0cce90 msgr2=0x7f14ec157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f1200640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14ec0cce90 0x7f14ec157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f1200640 1 -- 172.21.15.42:0/4255377351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14ec0b3cd0 msgr2=0x7f14ec151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f1200640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14ec0b3cd0 0x7f14ec151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f1200640 1 -- 172.21.15.42:0/4255377351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec15ce80 con 0x7f14ec072670 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f1c00640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14ec0cce90 0x7f14ec157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14f1200640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec072670 0x7f14ec15df20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00d800 tx=0x7f14dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:31.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.012+0000 7f14e9600640 1 -- 172.21.15.42:0/4255377351 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc00b6c0 con 0x7f14ec072670 2024-05-28T14:01:31.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.016+0000 7f14e9600640 1 -- 172.21.15.42:0/4255377351 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14dc019070 con 0x7f14ec072670 2024-05-28T14:01:31.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.016+0000 7f14f3c00640 1 -- 172.21.15.42:0/4255377351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ec15d160 con 0x7f14ec072670 2024-05-28T14:01:31.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.016+0000 7f14e9600640 1 -- 172.21.15.42:0/4255377351 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc00ba60 con 0x7f14ec072670 2024-05-28T14:01:31.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.016+0000 7f14f3c00640 1 -- 172.21.15.42:0/4255377351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ec15d720 con 0x7f14ec072670 2024-05-28T14:01:31.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.016+0000 7f14f3c00640 1 -- 172.21.15.42:0/4255377351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ec04fdd0 con 0x7f14ec072670 2024-05-28T14:01:31.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.016+0000 7f14e9600640 1 -- 172.21.15.42:0/4255377351 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7f14dc00bc00 con 0x7f14ec072670 2024-05-28T14:01:31.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.016+0000 7f14e9600640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f14c8068560 0x7f14c806aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.016+0000 7f14e9600640 1 -- 172.21.15.42:0/4255377351 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f14dc087020 con 0x7f14ec072670 2024-05-28T14:01:31.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.016+0000 7f14e3e00640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f14c8068560 0x7f14c806aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.016+0000 7f14e3e00640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f14c8068560 0x7f14c806aa20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f14d40b0000 tx=0x7f14d400c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:31.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.020+0000 7f14e9600640 1 -- 172.21.15.42:0/4255377351 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f14dc0573f0 con 0x7f14ec072670 2024-05-28T14:01:31.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.216+0000 7f14f3c00640 1 -- 172.21.15.42:0/4255377351 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f14ec06f400 con 0x7f14c8068560 2024-05-28T14:01:31.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.248+0000 7f14e9600640 1 -- 172.21.15.42:0/4255377351 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f14ec06f400 con 0x7f14c8068560 2024-05-28T14:01:31.232 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.248+0000 7f14f3c00640 1 -- 172.21.15.42:0/4255377351 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f14c8068560 msgr2=0x7f14c806aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.248+0000 7f14f3c00640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f14c8068560 0x7f14c806aa20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f14d40b0000 tx=0x7f14d400c040 comp rx=0 tx=0).stop 2024-05-28T14:01:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.248+0000 7f14f3c00640 1 -- 172.21.15.42:0/4255377351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec072670 msgr2=0x7f14ec15df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.248+0000 7f14f3c00640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec072670 0x7f14ec15df20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00d800 tx=0x7f14dc00dcc0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.248+0000 7f14f3c00640 1 -- 172.21.15.42:0/4255377351 shutdown_connections 2024-05-28T14:01:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.248+0000 7f14f3c00640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f14c8068560 0x7f14c806aa20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.248+0000 7f14f3c00640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14ec0cce90 0x7f14ec157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.248+0000 7f14f3c00640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14ec0b3cd0 0x7f14ec151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.248+0000 7f14f3c00640 1 --2- 172.21.15.42:0/4255377351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec072670 0x7f14ec15df20 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.248+0000 7f14f3c00640 1 -- 172.21.15.42:0/4255377351 >> 172.21.15.42:0/4255377351 conn(0x7f14ec0b21f0 msgr2=0x7f14ec153190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.248+0000 7f14f3c00640 1 -- 172.21.15.42:0/4255377351 shutdown_connections 2024-05-28T14:01:31.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.248+0000 7f14f3c00640 1 -- 172.21.15.42:0/4255377351 wait complete. 2024-05-28T14:01:31.254 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-28T14:01:31.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.412+0000 7f528cc00640 1 Processor -- start 2024-05-28T14:01:31.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.412+0000 7f528cc00640 1 -- start start 2024-05-28T14:01:31.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.412+0000 7f528cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f528809b160 0x7f528806cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.412+0000 7f528cc00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52880d0770 con 0x7f52880b0e80 2024-05-28T14:01:31.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.412+0000 7f528cc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52880d0a10 con 0x7f528809b8e0 2024-05-28T14:01:31.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.412+0000 7f528cc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f52880d0cf0 con 0x7f528809b160 2024-05-28T14:01:31.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.412+0000 7f5285e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f52880b0e80 0x7f528809aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:49730/0 (socket says 172.21.15.42:49730) 2024-05-28T14:01:31.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.412+0000 7f5285e00640 1 -- 172.21.15.42:0/4016973984 learned_addr learned my addr 172.21.15.42:0/4016973984 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:31.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.412+0000 7f527fe00640 1 --2- 172.21.15.42:0/4016973984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f528809b160 0x7f528806cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3106420666 0 0) 0x7f52880d0770 con 0x7f52880b0e80 2024-05-28T14:01:31.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f526c003530 con 0x7f52880b0e80 2024-05-28T14:01:31.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1029329925 0 0) 0x7f52880d0a10 con 0x7f528809b8e0 2024-05-28T14:01:31.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52880d0770 con 0x7f528809b8e0 2024-05-28T14:01:31.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4128594328 0 0) 0x7f526c003530 con 0x7f52880b0e80 2024-05-28T14:01:31.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f528809b160 msgr2=0x7f528806cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:31.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 --2- 172.21.15.42:0/4016973984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f528809b160 0x7f528806cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 >> v1:172.21.15.63:6789/0 conn(0x7f528809b8e0 legacy=0x7f528815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52880d0fd0 con 0x7f52880b0e80 2024-05-28T14:01:31.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f5278001680 con 0x7f52880b0e80 2024-05-28T14:01:31.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 >> v1:172.21.15.42:6789/0 conn(0x7f52880b0e80 legacy=0x7f528809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 --2- 172.21.15.42:0/4016973984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f526c003b80 0x7f526c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 --2- 172.21.15.42:0/4016973984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f528809b160 0x7f526c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 --2- 172.21.15.42:0/4016973984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f526c004d50 0x7f526c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f52880d0cf0 con 0x7f526c003b80 2024-05-28T14:01:31.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f526c00bab0 con 0x7f526c004d50 2024-05-28T14:01:31.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f526c00bcb0 con 0x7f528809b160 2024-05-28T14:01:31.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285e00640 1 --2- 172.21.15.42:0/4016973984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f526c003b80 0x7f526c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f527fe00640 1 --2- 172.21.15.42:0/4016973984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f528809b160 0x7f526c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5286800640 1 --2- 172.21.15.42:0/4016973984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f526c004d50 0x7f526c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285e00640 1 -- 172.21.15.42:0/4016973984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f528809b160 msgr2=0x7f526c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:31.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285e00640 1 --2- 172.21.15.42:0/4016973984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f528809b160 0x7f526c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285e00640 1 -- 172.21.15.42:0/4016973984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f526c004d50 msgr2=0x7f526c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:31.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285e00640 1 --2- 172.21.15.42:0/4016973984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f526c004d50 0x7f526c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5285e00640 1 -- 172.21.15.42:0/4016973984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526c003530 con 0x7f526c003b80 2024-05-28T14:01:31.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.416+0000 7f5286800640 1 --2- 172.21.15.42:0/4016973984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f526c004d50 0x7f526c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:31.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f5285e00640 1 --2- 172.21.15.42:0/4016973984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f526c003b80 0x7f526c003f80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f527800c1c0 tx=0x7f527801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c59809f9eb17d65 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:31.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278007cf0 con 0x7f526c003b80 2024-05-28T14:01:31.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52780a9020 con 0x7f526c003b80 2024-05-28T14:01:31.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f5285400640 1 -- 172.21.15.42:0/4016973984 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278005b00 con 0x7f526c003b80 2024-05-28T14:01:31.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f528cc00640 1 -- 172.21.15.42:0/4016973984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f526c003b80 msgr2=0x7f526c003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f528cc00640 1 --2- 172.21.15.42:0/4016973984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f526c003b80 0x7f526c003f80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f527800c1c0 tx=0x7f527801cc20 comp rx=0 tx=0).stop 2024-05-28T14:01:31.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f5286800640 1 -- 172.21.15.42:0/4016973984 reap_dead start 2024-05-28T14:01:31.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f528cc00640 1 -- 172.21.15.42:0/4016973984 shutdown_connections 2024-05-28T14:01:31.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f528cc00640 1 -- 172.21.15.42:0/4016973984 >> 172.21.15.42:0/4016973984 conn(0x7f52880b3f90 msgr2=0x7f52880b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:31.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f528cc00640 1 -- 172.21.15.42:0/4016973984 shutdown_connections 2024-05-28T14:01:31.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f528cc00640 1 -- 172.21.15.42:0/4016973984 wait complete. 2024-05-28T14:01:31.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f528cc00640 1 Processor -- start 2024-05-28T14:01:31.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f528cc00640 1 -- start start 2024-05-28T14:01:31.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f528cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f526c003b80 0x7f528815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f528cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5288161270 0x7f52881616f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f528cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5288070820 0x7f5288070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f528cc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f52880d1a90 con 0x7f526c003b80 2024-05-28T14:01:31.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f528cc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f52880b01e0 con 0x7f5288070820 2024-05-28T14:01:31.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f528cc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f528815ec30 con 0x7f5288161270 2024-05-28T14:01:31.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f5285e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f526c003b80 0x7f528815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f5285e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f526c003b80 0x7f528815bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57646/0 (socket says 172.21.15.42:57646) 2024-05-28T14:01:31.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.420+0000 7f5285e00640 1 -- 172.21.15.42:0/2496600670 learned_addr learned my addr 172.21.15.42:0/2496600670 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:31.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f5286800640 1 --2- 172.21.15.42:0/2496600670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5288070820 0x7f5288070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f527fe00640 1 --2- 172.21.15.42:0/2496600670 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5288161270 0x7f52881616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f5285e00640 1 -- 172.21.15.42:0/2496600670 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5288161270 msgr2=0x7f52881616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f5285e00640 1 --2- 172.21.15.42:0/2496600670 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5288161270 0x7f52881616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f5285e00640 1 -- 172.21.15.42:0/2496600670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5288070820 msgr2=0x7f5288070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f5285e00640 1 --2- 172.21.15.42:0/2496600670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5288070820 0x7f5288070ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f5285e00640 1 -- 172.21.15.42:0/2496600670 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52880c6300 con 0x7f526c003b80 2024-05-28T14:01:31.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f5285e00640 1 --2- 172.21.15.42:0/2496600670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f526c003b80 0x7f528815bac0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f52780ab040 tx=0x7f526c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:31.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f527e000640 1 -- 172.21.15.42:0/2496600670 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278004a70 con 0x7f526c003b80 2024-05-28T14:01:31.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f527e000640 1 -- 172.21.15.42:0/2496600670 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5278004c10 con 0x7f526c003b80 2024-05-28T14:01:31.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f527e000640 1 -- 172.21.15.42:0/2496600670 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52780a57f0 con 0x7f526c003b80 2024-05-28T14:01:31.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f528cc00640 1 -- 172.21.15.42:0/2496600670 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52880c65e0 con 0x7f526c003b80 2024-05-28T14:01:31.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f528cc00640 1 -- 172.21.15.42:0/2496600670 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52880c6ba0 con 0x7f526c003b80 2024-05-28T14:01:31.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f528cc00640 1 -- 172.21.15.42:0/2496600670 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52880ceaa0 con 0x7f526c003b80 2024-05-28T14:01:31.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.424+0000 7f527e000640 1 -- 172.21.15.42:0/2496600670 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7f52780a59b0 con 0x7f526c003b80 2024-05-28T14:01:31.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.428+0000 7f527e000640 1 --2- 172.21.15.42:0/2496600670 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5264068530 0x7f526406a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.428+0000 7f527e000640 1 -- 172.21.15.42:0/2496600670 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f52781201b0 con 0x7f526c003b80 2024-05-28T14:01:31.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.428+0000 7f527fe00640 1 --2- 172.21.15.42:0/2496600670 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5264068530 0x7f526406a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.428+0000 7f527fe00640 1 --2- 172.21.15.42:0/2496600670 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5264068530 0x7f526406a9f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5288071630 tx=0x7f5270009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:31.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.432+0000 7f527e000640 1 -- 172.21.15.42:0/2496600670 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5278016930 con 0x7f526c003b80 2024-05-28T14:01:31.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.620+0000 7f528cc00640 1 -- 172.21.15.42:0/2496600670 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f52880b16a0 con 0x7f5264068530 2024-05-28T14:01:31.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.644+0000 7f527e000640 1 -- 172.21.15.42:0/2496600670 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f52880b16a0 con 0x7f5264068530 2024-05-28T14:01:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f528cc00640 1 -- 172.21.15.42:0/2496600670 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5264068530 msgr2=0x7f526406a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f528cc00640 1 --2- 172.21.15.42:0/2496600670 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5264068530 0x7f526406a9f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5288071630 tx=0x7f5270009190 comp rx=0 tx=0).stop 2024-05-28T14:01:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f528cc00640 1 -- 172.21.15.42:0/2496600670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f526c003b80 msgr2=0x7f528815bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f528cc00640 1 --2- 172.21.15.42:0/2496600670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f526c003b80 0x7f528815bac0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f52780ab040 tx=0x7f526c005840 comp rx=0 tx=0).stop 2024-05-28T14:01:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f528cc00640 1 -- 172.21.15.42:0/2496600670 shutdown_connections 2024-05-28T14:01:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f528cc00640 1 --2- 172.21.15.42:0/2496600670 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5264068530 0x7f526406a9f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f528cc00640 1 --2- 172.21.15.42:0/2496600670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5288070820 0x7f5288070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f528cc00640 1 --2- 172.21.15.42:0/2496600670 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5288161270 0x7f52881616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f528cc00640 1 --2- 172.21.15.42:0/2496600670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f526c003b80 0x7f528815bac0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f528cc00640 1 -- 172.21.15.42:0/2496600670 >> 172.21.15.42:0/2496600670 conn(0x7f52880b3f90 msgr2=0x7f5288169d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:31.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f528cc00640 1 -- 172.21.15.42:0/2496600670 shutdown_connections 2024-05-28T14:01:31.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f528cc00640 1 -- 172.21.15.42:0/2496600670 wait complete. 2024-05-28T14:01:31.635 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:01:31.636 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:01:31.636 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:01:31.636 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:01:31.636 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:01:31.648+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:01:31.655 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-05-28T14:01:31.656 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:01:31.668+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:01:31.656 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:01:31.668+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:01:31.656 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:01:31.668+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:01:31.656 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:01:31.668+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:01:31.656 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:01:31.668+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:01:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.824+0000 7f8258e00640 1 Processor -- start 2024-05-28T14:01:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.824+0000 7f8258e00640 1 -- start start 2024-05-28T14:01:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.824+0000 7f8258e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f825415fcc0 0x7f82540d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.824+0000 7f8258e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82540d0fd0 con 0x7f825409aa50 2024-05-28T14:01:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.824+0000 7f8258e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82540d1250 con 0x7f825409b5c0 2024-05-28T14:01:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.824+0000 7f8258e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f82540b5510 con 0x7f825415fcc0 2024-05-28T14:01:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.824+0000 7f8251e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f825415fcc0 0x7f82540d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.824+0000 7f824be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f825409aa50 0x7f825409aeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:49732/0 (socket says 172.21.15.42:49732) 2024-05-28T14:01:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.824+0000 7f824be00640 1 -- 172.21.15.42:0/4201516315 learned_addr learned my addr 172.21.15.42:0/4201516315 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.824+0000 7f8251e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f825415fcc0 0x7f82540d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57702/0 (socket says 172.21.15.42:57702) 2024-05-28T14:01:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.824+0000 7f8251400640 1 -- 172.21.15.42:0/4201516315 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4276586527 0 0) 0x7f823c001120 con 0x7f825409aa50 2024-05-28T14:01:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.824+0000 7f8251e00640 1 -- 172.21.15.42:0/4201516315 >> v1:172.21.15.63:6789/0 conn(0x7f825409b5c0 legacy=0x7f82540b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.824+0000 7f8251e00640 1 -- 172.21.15.42:0/4201516315 >> v1:172.21.15.42:6789/0 conn(0x7f825409aa50 legacy=0x7f825409aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.824+0000 7f8251e00640 1 -- 172.21.15.42:0/4201516315 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82540b57f0 con 0x7f825415fcc0 2024-05-28T14:01:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.828+0000 7f8251e00640 1 --2- 172.21.15.42:0/4201516315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f825415fcc0 0x7f82540d0a90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f82440097c0 tx=0x7f824401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8665b7efe68950c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.828+0000 7f8251400640 1 -- 172.21.15.42:0/4201516315 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82440b0070 con 0x7f825415fcc0 2024-05-28T14:01:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.828+0000 7f8251400640 1 -- 172.21.15.42:0/4201516315 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82440ab600 con 0x7f825415fcc0 2024-05-28T14:01:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.828+0000 7f8251400640 1 -- 172.21.15.42:0/4201516315 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82440ab9d0 con 0x7f825415fcc0 2024-05-28T14:01:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.828+0000 7f8258e00640 1 -- 172.21.15.42:0/4201516315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f825415fcc0 msgr2=0x7f82540d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.828+0000 7f8258e00640 1 --2- 172.21.15.42:0/4201516315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f825415fcc0 0x7f82540d0a90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f82440097c0 tx=0x7f824401c4a0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.828+0000 7f8258e00640 1 -- 172.21.15.42:0/4201516315 shutdown_connections 2024-05-28T14:01:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.828+0000 7f8258e00640 1 --2- 172.21.15.42:0/4201516315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f825415fcc0 0x7f82540d0a90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.828+0000 7f8258e00640 1 -- 172.21.15.42:0/4201516315 >> 172.21.15.42:0/4201516315 conn(0x7f82540b3fa0 msgr2=0x7f82540b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.828+0000 7f8258e00640 1 -- 172.21.15.42:0/4201516315 shutdown_connections 2024-05-28T14:01:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.828+0000 7f8258e00640 1 -- 172.21.15.42:0/4201516315 wait complete. 2024-05-28T14:01:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f8258e00640 1 Processor -- start 2024-05-28T14:01:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f8258e00640 1 -- start start 2024-05-28T14:01:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f8258e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f825409aa50 0x7f82540cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f8258e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f825409b5c0 0x7f82541594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f8258e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f825415fcc0 0x7f8254159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f8258e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f82540b7320 con 0x7f825409b5c0 2024-05-28T14:01:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f8258e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82540b71a0 con 0x7f825415fcc0 2024-05-28T14:01:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f8258e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f825416a280 con 0x7f825409aa50 2024-05-28T14:01:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f824be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f825409b5c0 0x7f82541594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f8251e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f825409aa50 0x7f82540cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f824be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f825409b5c0 0x7f82541594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57704/0 (socket says 172.21.15.42:57704) 2024-05-28T14:01:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f824be00640 1 -- 172.21.15.42:0/1318611789 learned_addr learned my addr 172.21.15.42:0/1318611789 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f8252800640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f825415fcc0 0x7f8254159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f824be00640 1 -- 172.21.15.42:0/1318611789 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f825409aa50 msgr2=0x7f82540cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f824be00640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f825409aa50 0x7f82540cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f824be00640 1 -- 172.21.15.42:0/1318611789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f825415fcc0 msgr2=0x7f8254159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f824be00640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f825415fcc0 0x7f8254159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f824be00640 1 -- 172.21.15.42:0/1318611789 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825415a050 con 0x7f825409b5c0 2024-05-28T14:01:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f8252800640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f825415fcc0 0x7f8254159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:01:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f8251e00640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f825409aa50 0x7f82540cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:01:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f824be00640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f825409b5c0 0x7f82541594e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f823c00c410 tx=0x7f823c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f824a000640 1 -- 172.21.15.42:0/1318611789 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c0074e0 con 0x7f825409b5c0 2024-05-28T14:01:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f824a000640 1 -- 172.21.15.42:0/1318611789 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f823c0047f0 con 0x7f825409b5c0 2024-05-28T14:01:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f824a000640 1 -- 172.21.15.42:0/1318611789 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c007680 con 0x7f825409b5c0 2024-05-28T14:01:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f8258e00640 1 -- 172.21.15.42:0/1318611789 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825415a330 con 0x7f825409b5c0 2024-05-28T14:01:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.832+0000 7f8258e00640 1 -- 172.21.15.42:0/1318611789 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825415a8f0 con 0x7f825409b5c0 2024-05-28T14:01:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.836+0000 7f8258e00640 1 -- 172.21.15.42:0/1318611789 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8254068a90 con 0x7f825409b5c0 2024-05-28T14:01:31.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.836+0000 7f824a000640 1 -- 172.21.15.42:0/1318611789 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7f823c011070 con 0x7f825409b5c0 2024-05-28T14:01:31.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.836+0000 7f824a000640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8230068530 0x7f823006a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:31.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.836+0000 7f824a000640 1 -- 172.21.15.42:0/1318611789 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f823c0867f0 con 0x7f825409b5c0 2024-05-28T14:01:31.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.836+0000 7f8251e00640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8230068530 0x7f823006a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:31.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.836+0000 7f8251e00640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8230068530 0x7f823006a9f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f82440ad000 tx=0x7f82440a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:31.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:31.844+0000 7f824a000640 1 -- 172.21.15.42:0/1318611789 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f82540d0fd0 con 0x7f825409b5c0 2024-05-28T14:01:32.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.036+0000 7f8258e00640 1 -- 172.21.15.42:0/1318611789 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f82540691a0 con 0x7f8230068530 2024-05-28T14:01:32.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.056+0000 7f824a000640 1 -- 172.21.15.42:0/1318611789 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f82540691a0 con 0x7f8230068530 2024-05-28T14:01:32.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.060+0000 7f8258e00640 1 -- 172.21.15.42:0/1318611789 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8230068530 msgr2=0x7f823006a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:32.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.060+0000 7f8258e00640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8230068530 0x7f823006a9f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f82440ad000 tx=0x7f82440a6000 comp rx=0 tx=0).stop 2024-05-28T14:01:32.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.060+0000 7f8258e00640 1 -- 172.21.15.42:0/1318611789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f825409b5c0 msgr2=0x7f82541594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.060+0000 7f8258e00640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f825409b5c0 0x7f82541594e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f823c00c410 tx=0x7f823c00c8d0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.060+0000 7f8258e00640 1 -- 172.21.15.42:0/1318611789 shutdown_connections 2024-05-28T14:01:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.060+0000 7f8258e00640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8230068530 0x7f823006a9f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.060+0000 7f8258e00640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f825415fcc0 0x7f8254159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.060+0000 7f8258e00640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f825409b5c0 0x7f82541594e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.060+0000 7f8258e00640 1 --2- 172.21.15.42:0/1318611789 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f825409aa50 0x7f82540cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.060+0000 7f8258e00640 1 -- 172.21.15.42:0/1318611789 >> 172.21.15.42:0/1318611789 conn(0x7f82540b3fa0 msgr2=0x7f82540ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.060+0000 7f8258e00640 1 -- 172.21.15.42:0/1318611789 shutdown_connections 2024-05-28T14:01:32.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.060+0000 7f8258e00640 1 -- 172.21.15.42:0/1318611789 wait complete. 2024-05-28T14:01:32.068 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-28T14:01:32.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-28T14:01:32.126 INFO:teuthology.orchestra.run.smithi042.stdout:2 2024-05-28T14:01:32.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:01:32.134 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T14:01:32.134 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:01:32.227 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:01:32.285 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T14:01:32.286 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:01:32.286 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:01:32.286 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:01:32.295 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:01:32.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.492+0000 7faac3600640 1 Processor -- start 2024-05-28T14:01:32.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.492+0000 7faac3600640 1 -- start start 2024-05-28T14:01:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.492+0000 7faac3600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc15c990 0x7faabc0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.492+0000 7faac3600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faabc09bbd0 0x7faabc09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.492+0000 7faac3600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faabc0d06a0 0x7faabc09af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.492+0000 7faac3600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7faabc15cd70 con 0x7faabc0d06a0 2024-05-28T14:01:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.492+0000 7faac3600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faabc0cd300 con 0x7faabc09bbd0 2024-05-28T14:01:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.492+0000 7faac3600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faabc0cd5e0 con 0x7faabc15c990 2024-05-28T14:01:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.492+0000 7faac1600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faabc0d06a0 0x7faabc09af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.492+0000 7faac1600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faabc0d06a0 0x7faabc09af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57714/0 (socket says 172.21.15.42:57714) 2024-05-28T14:01:32.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.492+0000 7faac1600640 1 -- 172.21.15.42:0/3845114966 learned_addr learned my addr 172.21.15.42:0/3845114966 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:32.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.496+0000 7faac0c00640 1 --2- 172.21.15.42:0/3845114966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc15c990 0x7faabc0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:32.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.496+0000 7faabbe00640 1 --2- 172.21.15.42:0/3845114966 >> v2:172.21.15.63:3300/0 conn(0x7faabc09bbd0 0x7faabc09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:32.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.496+0000 7faac1600640 1 -- 172.21.15.42:0/3845114966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc15c990 msgr2=0x7faabc0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:32.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.496+0000 7faac1600640 1 --2- 172.21.15.42:0/3845114966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc15c990 0x7faabc0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.496+0000 7faac1600640 1 -- 172.21.15.42:0/3845114966 >> v2:172.21.15.63:3300/0 conn(0x7faabc09bbd0 msgr2=0x7faabc09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:32.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.496+0000 7faac1600640 1 --2- 172.21.15.42:0/3845114966 >> v2:172.21.15.63:3300/0 conn(0x7faabc09bbd0 0x7faabc09aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.496+0000 7faac1600640 1 -- 172.21.15.42:0/3845114966 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc0cd8c0 con 0x7faabc0d06a0 2024-05-28T14:01:32.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.496+0000 7faabbe00640 1 --2- 172.21.15.42:0/3845114966 >> v2:172.21.15.63:3300/0 conn(0x7faabc09bbd0 0x7faabc09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:32.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.496+0000 7faac1600640 1 --2- 172.21.15.42:0/3845114966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc0d06a0 0x7faabc09af90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7faabc0735e0 tx=0x7faab401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f86000b4ec299df server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:32.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.496+0000 7faabb400640 1 -- 172.21.15.42:0/3845114966 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab40a5840 con 0x7faabc0d06a0 2024-05-28T14:01:32.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.496+0000 7faabb400640 1 -- 172.21.15.42:0/3845114966 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab40a59e0 con 0x7faabc0d06a0 2024-05-28T14:01:32.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.496+0000 7faabb400640 1 -- 172.21.15.42:0/3845114966 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab40a5db0 con 0x7faabc0d06a0 2024-05-28T14:01:32.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.496+0000 7faac3600640 1 -- 172.21.15.42:0/3845114966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc0d06a0 msgr2=0x7faabc09af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:32.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.496+0000 7faac3600640 1 --2- 172.21.15.42:0/3845114966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc0d06a0 0x7faabc09af90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7faabc0735e0 tx=0x7faab401c9c0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 -- 172.21.15.42:0/3845114966 shutdown_connections 2024-05-28T14:01:32.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 --2- 172.21.15.42:0/3845114966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc0d06a0 0x7faabc09af90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 --2- 172.21.15.42:0/3845114966 >> v2:172.21.15.63:3300/0 conn(0x7faabc09bbd0 0x7faabc09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 --2- 172.21.15.42:0/3845114966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc15c990 0x7faabc0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 -- 172.21.15.42:0/3845114966 >> 172.21.15.42:0/3845114966 conn(0x7faabc0b4080 msgr2=0x7faabc0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:32.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 -- 172.21.15.42:0/3845114966 shutdown_connections 2024-05-28T14:01:32.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 -- 172.21.15.42:0/3845114966 wait complete. 2024-05-28T14:01:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 Processor -- start 2024-05-28T14:01:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 -- start start 2024-05-28T14:01:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc0d06a0 0x7faabc0c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faabc15c990 0x7faabc0c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc0c87c0 0x7faabc159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faabc160d90 con 0x7faabc0d06a0 2024-05-28T14:01:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faabc160c10 con 0x7faabc15c990 2024-05-28T14:01:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faabc160f10 con 0x7faabc0c87c0 2024-05-28T14:01:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc0d06a0 0x7faabc0c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc0d06a0 0x7faabc0c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57730/0 (socket says 172.21.15.42:57730) 2024-05-28T14:01:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac0c00640 1 -- 172.21.15.42:0/3734373026 learned_addr learned my addr 172.21.15.42:0/3734373026 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac1600640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc0c87c0 0x7faabc159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faabbe00640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faabc15c990 0x7faabc0c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac0c00640 1 -- 172.21.15.42:0/3734373026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc0c87c0 msgr2=0x7faabc159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac0c00640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc0c87c0 0x7faabc159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac0c00640 1 -- 172.21.15.42:0/3734373026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faabc15c990 msgr2=0x7faabc0c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac0c00640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faabc15c990 0x7faabc0c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac0c00640 1 -- 172.21.15.42:0/3734373026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc0c6e30 con 0x7faabc0d06a0 2024-05-28T14:01:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faabbe00640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faabc15c990 0x7faabc0c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac0c00640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc0d06a0 0x7faabc0c68f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faab000d5f0 tx=0x7faab000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faab8c00640 1 -- 172.21.15.42:0/3734373026 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00125f0 con 0x7faabc0d06a0 2024-05-28T14:01:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faab8c00640 1 -- 172.21.15.42:0/3734373026 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab0019070 con 0x7faabc0d06a0 2024-05-28T14:01:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faab8c00640 1 -- 172.21.15.42:0/3734373026 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00129b0 con 0x7faabc0d06a0 2024-05-28T14:01:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.500+0000 7faac3600640 1 -- 172.21.15.42:0/3734373026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faabc15a370 con 0x7faabc0d06a0 2024-05-28T14:01:32.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.504+0000 7faac3600640 1 -- 172.21.15.42:0/3734373026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faabc15a930 con 0x7faabc0d06a0 2024-05-28T14:01:32.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.504+0000 7faac3600640 1 -- 172.21.15.42:0/3734373026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faabc068a90 con 0x7faabc0d06a0 2024-05-28T14:01:32.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.504+0000 7faab8c00640 1 -- 172.21.15.42:0/3734373026 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7faab0012b50 con 0x7faabc0d06a0 2024-05-28T14:01:32.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.504+0000 7faab8c00640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa9c068530 0x7faa9c06a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:32.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.504+0000 7faab8c00640 1 -- 172.21.15.42:0/3734373026 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7faab00871a0 con 0x7faabc0d06a0 2024-05-28T14:01:32.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.504+0000 7faabbe00640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa9c068530 0x7faa9c06a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:32.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.504+0000 7faabbe00640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa9c068530 0x7faa9c06a9f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faaa8006fd0 tx=0x7faaa8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.508+0000 7faab8c00640 1 -- 172.21.15.42:0/3734373026 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faab0057440 con 0x7faabc0d06a0 2024-05-28T14:01:32.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.676+0000 7faac3600640 1 -- 172.21.15.42:0/3734373026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faabc0b4ce0 con 0x7faabc0d06a0 2024-05-28T14:01:32.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.676+0000 7faab8c00640 1 -- 172.21.15.42:0/3734373026 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16609 (secure 0 0 0) 0x7faab0056d40 con 0x7faabc0d06a0 2024-05-28T14:01:32.663 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:32.663 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":34,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T13:59:28.461435+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:59:24.673711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-28T13:59:25.795430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:01:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.680+0000 7faac3600640 1 -- 172.21.15.42:0/3734373026 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa9c068530 msgr2=0x7faa9c06a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.680+0000 7faac3600640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa9c068530 0x7faa9c06a9f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faaa8006fd0 tx=0x7faaa8008040 comp rx=0 tx=0).stop 2024-05-28T14:01:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.680+0000 7faac3600640 1 -- 172.21.15.42:0/3734373026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc0d06a0 msgr2=0x7faabc0c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.680+0000 7faac3600640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc0d06a0 0x7faabc0c68f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faab000d5f0 tx=0x7faab000dab0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.680+0000 7faac3600640 1 -- 172.21.15.42:0/3734373026 shutdown_connections 2024-05-28T14:01:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.680+0000 7faac3600640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa9c068530 0x7faa9c06a9f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.680+0000 7faac3600640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc0c87c0 0x7faabc159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.680+0000 7faac3600640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faabc15c990 0x7faabc0c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.680+0000 7faac3600640 1 --2- 172.21.15.42:0/3734373026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc0d06a0 0x7faabc0c68f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.680+0000 7faac3600640 1 -- 172.21.15.42:0/3734373026 >> 172.21.15.42:0/3734373026 conn(0x7faabc0b4080 msgr2=0x7faabc15f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.680+0000 7faac3600640 1 -- 172.21.15.42:0/3734373026 shutdown_connections 2024-05-28T14:01:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.680+0000 7faac3600640 1 -- 172.21.15.42:0/3734373026 wait complete. 2024-05-28T14:01:32.684 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:59:24.673711+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:59:25.795430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:01:32.685 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:01:32.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.844+0000 7f5b36200640 1 Processor -- start 2024-05-28T14:01:32.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.844+0000 7f5b36200640 1 -- start start 2024-05-28T14:01:32.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.844+0000 7f5b36200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b300b0e80 0x7f5b3006cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:32.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.844+0000 7f5b36200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5b3009b8e0 0x7f5b3009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:32.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.844+0000 7f5b36200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b300d0770 con 0x7f5b3009af90 2024-05-28T14:01:32.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.844+0000 7f5b36200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5b3009b400 con 0x7f5b3009b8e0 2024-05-28T14:01:32.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.844+0000 7f5b36200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5b300d0a10 con 0x7f5b300b0e80 2024-05-28T14:01:32.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.844+0000 7f5b27e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5b3009b8e0 0x7f5b3009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:32.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.844+0000 7f5b2fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f5b3009af90 0x7f5b3015e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:49742/0 (socket says 172.21.15.42:49742) 2024-05-28T14:01:32.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.844+0000 7f5b2fe00640 1 -- 172.21.15.42:0/2039695650 learned_addr learned my addr 172.21.15.42:0/2039695650 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:32.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.844+0000 7f5b27e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5b3009b8e0 0x7f5b3009aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57742/0 (socket says 172.21.15.42:57742) 2024-05-28T14:01:32.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.844+0000 7f5b2f400640 1 --2- 172.21.15.42:0/2039695650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b300b0e80 0x7f5b3006cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b27e00640 1 -- 172.21.15.42:0/2039695650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b300b0e80 msgr2=0x7f5b3006cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b27e00640 1 --2- 172.21.15.42:0/2039695650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b300b0e80 0x7f5b3006cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b27e00640 1 -- 172.21.15.42:0/2039695650 >> v1:172.21.15.42:6789/0 conn(0x7f5b3009af90 legacy=0x7f5b3015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b27e00640 1 -- 172.21.15.42:0/2039695650 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b300d0cf0 con 0x7f5b3009b8e0 2024-05-28T14:01:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b2f400640 1 --2- 172.21.15.42:0/2039695650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b300b0e80 0x7f5b3006cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b27e00640 1 --2- 172.21.15.42:0/2039695650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3009b8e0 0x7f5b3009aa50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5b180097c0 tx=0x7f5b1801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0f4b059bd08097 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b2ea00640 1 -- 172.21.15.42:0/2039695650 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b180b0070 con 0x7f5b3009b8e0 2024-05-28T14:01:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b2ea00640 1 -- 172.21.15.42:0/2039695650 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b180ab600 con 0x7f5b3009b8e0 2024-05-28T14:01:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b2ea00640 1 -- 172.21.15.42:0/2039695650 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b180ab9d0 con 0x7f5b3009b8e0 2024-05-28T14:01:32.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b36200640 1 -- 172.21.15.42:0/2039695650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3009b8e0 msgr2=0x7f5b3009aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:32.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b36200640 1 --2- 172.21.15.42:0/2039695650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3009b8e0 0x7f5b3009aa50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5b180097c0 tx=0x7f5b1801c4a0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b36200640 1 -- 172.21.15.42:0/2039695650 shutdown_connections 2024-05-28T14:01:32.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b36200640 1 --2- 172.21.15.42:0/2039695650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3009b8e0 0x7f5b3009aa50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b36200640 1 --2- 172.21.15.42:0/2039695650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b300b0e80 0x7f5b3006cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.848+0000 7f5b36200640 1 -- 172.21.15.42:0/2039695650 >> 172.21.15.42:0/2039695650 conn(0x7f5b300b3f90 msgr2=0x7f5b300b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:32.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b36200640 1 -- 172.21.15.42:0/2039695650 shutdown_connections 2024-05-28T14:01:32.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b36200640 1 -- 172.21.15.42:0/2039695650 wait complete. 2024-05-28T14:01:32.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b36200640 1 Processor -- start 2024-05-28T14:01:32.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b36200640 1 -- start start 2024-05-28T14:01:32.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b36200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3009af90 0x7f5b30154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:32.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b36200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b3009b8e0 0x7f5b301563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:32.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b36200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b300b0e80 0x7f5b301fb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:32.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b36200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3015ec30 con 0x7f5b3009af90 2024-05-28T14:01:32.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b36200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5b300d1a90 con 0x7f5b300b0e80 2024-05-28T14:01:32.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b36200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5b300af9e0 con 0x7f5b3009b8e0 2024-05-28T14:01:32.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b2f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3009af90 0x7f5b30154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:32.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b2f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3009af90 0x7f5b30154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57750/0 (socket says 172.21.15.42:57750) 2024-05-28T14:01:32.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b2f400640 1 -- 172.21.15.42:0/3143868589 learned_addr learned my addr 172.21.15.42:0/3143868589 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:32.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b2f400640 1 -- 172.21.15.42:0/3143868589 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b3009b8e0 msgr2=0x7f5b301563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:32.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b2f400640 1 --2- 172.21.15.42:0/3143868589 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b3009b8e0 0x7f5b301563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b2f400640 1 -- 172.21.15.42:0/3143868589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b300b0e80 msgr2=0x7f5b301fb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:32.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b2f400640 1 --2- 172.21.15.42:0/3143868589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b300b0e80 0x7f5b301fb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:32.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b2f400640 1 -- 172.21.15.42:0/3143868589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b30070fa0 con 0x7f5b3009af90 2024-05-28T14:01:32.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b2f400640 1 --2- 172.21.15.42:0/3143868589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3009af90 0x7f5b30154480 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000d800 tx=0x7f5b2000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b27400640 1 -- 172.21.15.42:0/3143868589 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20018020 con 0x7f5b3009af90 2024-05-28T14:01:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b27400640 1 -- 172.21.15.42:0/3143868589 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b200096c0 con 0x7f5b3009af90 2024-05-28T14:01:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b27400640 1 -- 172.21.15.42:0/3143868589 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20009ab0 con 0x7f5b3009af90 2024-05-28T14:01:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b36200640 1 -- 172.21.15.42:0/3143868589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b30161c10 con 0x7f5b3009af90 2024-05-28T14:01:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.852+0000 7f5b36200640 1 -- 172.21.15.42:0/3143868589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b300b5a60 con 0x7f5b3009af90 2024-05-28T14:01:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.856+0000 7f5b36200640 1 -- 172.21.15.42:0/3143868589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b300b5880 con 0x7f5b3009af90 2024-05-28T14:01:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.856+0000 7f5b27400640 1 -- 172.21.15.42:0/3143868589 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 88140+0+0 (secure 0 0 0) 0x7f5b2001b020 con 0x7f5b3009af90 2024-05-28T14:01:32.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.856+0000 7f5b27400640 1 --2- 172.21.15.42:0/3143868589 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b0c068530 0x7f5b0c06a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.856+0000 7f5b27400640 1 -- 172.21.15.42:0/3143868589 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5b20087220 con 0x7f5b3009af90 2024-05-28T14:01:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.856+0000 7f5b27e00640 1 --2- 172.21.15.42:0/3143868589 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b0c068530 0x7f5b0c06a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.856+0000 7f5b27e00640 1 --2- 172.21.15.42:0/3143868589 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b0c068530 0x7f5b0c06a9f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b180ad000 tx=0x7f5b18004740 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:32.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.860+0000 7f5b27400640 1 -- 172.21.15.42:0/3143868589 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b300b5880 con 0x7f5b3009af90 2024-05-28T14:01:32.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:32.968+0000 7f5b27400640 1 -- 172.21.15.42:0/3143868589 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f5b200534c0 con 0x7f5b3009af90 2024-05-28T14:01:33.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.032+0000 7f5b36200640 1 -- 172.21.15.42:0/3143868589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5b300a0550 con 0x7f5b3009af90 2024-05-28T14:01:33.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.032+0000 7f5b27400640 1 -- 172.21.15.42:0/3143868589 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f5b20022020 con 0x7f5b3009af90 2024-05-28T14:01:33.020 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:01:33.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.036+0000 7f5b36200640 1 -- 172.21.15.42:0/3143868589 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b0c068530 msgr2=0x7f5b0c06a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.036+0000 7f5b36200640 1 --2- 172.21.15.42:0/3143868589 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b0c068530 0x7f5b0c06a9f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b180ad000 tx=0x7f5b18004740 comp rx=0 tx=0).stop 2024-05-28T14:01:33.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.036+0000 7f5b36200640 1 -- 172.21.15.42:0/3143868589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3009af90 msgr2=0x7f5b30154480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.036+0000 7f5b36200640 1 --2- 172.21.15.42:0/3143868589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3009af90 0x7f5b30154480 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000d800 tx=0x7f5b2000dcc0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.036+0000 7f5b36200640 1 -- 172.21.15.42:0/3143868589 shutdown_connections 2024-05-28T14:01:33.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.036+0000 7f5b36200640 1 --2- 172.21.15.42:0/3143868589 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b0c068530 0x7f5b0c06a9f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.036+0000 7f5b36200640 1 --2- 172.21.15.42:0/3143868589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b300b0e80 0x7f5b301fb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.036+0000 7f5b36200640 1 --2- 172.21.15.42:0/3143868589 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b3009b8e0 0x7f5b301563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.036+0000 7f5b36200640 1 --2- 172.21.15.42:0/3143868589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3009af90 0x7f5b30154480 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.036+0000 7f5b36200640 1 -- 172.21.15.42:0/3143868589 >> 172.21.15.42:0/3143868589 conn(0x7f5b300b3f90 msgr2=0x7f5b300b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:33.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.036+0000 7f5b36200640 1 -- 172.21.15.42:0/3143868589 shutdown_connections 2024-05-28T14:01:33.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.036+0000 7f5b36200640 1 -- 172.21.15.42:0/3143868589 wait complete. 2024-05-28T14:01:33.042 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:01:33.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.200+0000 7ff6ea400640 1 Processor -- start 2024-05-28T14:01:33.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.200+0000 7ff6ea400640 1 -- start start 2024-05-28T14:01:33.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.200+0000 7ff6ea400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff6e4071de0 0x7ff6e40721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.200+0000 7ff6ea400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6e40ce3b0 0x7ff6e40ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.200+0000 7ff6ea400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff6e4141d10 0x7ff6e41421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.200+0000 7ff6ea400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff6e4072700 con 0x7ff6e4071de0 2024-05-28T14:01:33.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.200+0000 7ff6ea400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff6e4160230 con 0x7ff6e4141d10 2024-05-28T14:01:33.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.200+0000 7ff6ea400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e4160510 con 0x7ff6e40ce3b0 2024-05-28T14:01:33.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.200+0000 7ff6e3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff6e4071de0 0x7ff6e40721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.200+0000 7ff6e3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff6e4071de0 0x7ff6e40721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57752/0 (socket says 172.21.15.42:57752) 2024-05-28T14:01:33.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.200+0000 7ff6e3400640 1 -- 172.21.15.42:0/3414087163 learned_addr learned my addr 172.21.15.42:0/3414087163 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:33.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.204+0000 7ff6dbe00640 1 --2- 172.21.15.42:0/3414087163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6e40ce3b0 0x7ff6e40ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.204+0000 7ff6e3e00640 1 --2- 172.21.15.42:0/3414087163 >> v2:172.21.15.63:3300/0 conn(0x7ff6e4141d10 0x7ff6e41421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.204+0000 7ff6e3400640 1 -- 172.21.15.42:0/3414087163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6e40ce3b0 msgr2=0x7ff6e40ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:33.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.204+0000 7ff6e3400640 1 --2- 172.21.15.42:0/3414087163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6e40ce3b0 0x7ff6e40ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.204+0000 7ff6e3400640 1 -- 172.21.15.42:0/3414087163 >> v2:172.21.15.63:3300/0 conn(0x7ff6e4141d10 msgr2=0x7ff6e41421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:33.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.204+0000 7ff6e3400640 1 --2- 172.21.15.42:0/3414087163 >> v2:172.21.15.63:3300/0 conn(0x7ff6e4141d10 0x7ff6e41421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.204+0000 7ff6e3400640 1 -- 172.21.15.42:0/3414087163 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e41607f0 con 0x7ff6e4071de0 2024-05-28T14:01:33.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.204+0000 7ff6e3400640 1 --2- 172.21.15.42:0/3414087163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6e4071de0 0x7ff6e40721c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40702b0 tx=0x7ff6d401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f0c571d8a580c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:33.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.204+0000 7ff6e2a00640 1 -- 172.21.15.42:0/3414087163 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40b0070 con 0x7ff6e4071de0 2024-05-28T14:01:33.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.204+0000 7ff6e2a00640 1 -- 172.21.15.42:0/3414087163 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d40a4e50 con 0x7ff6e4071de0 2024-05-28T14:01:33.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.204+0000 7ff6e2a00640 1 -- 172.21.15.42:0/3414087163 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40ab820 con 0x7ff6e4071de0 2024-05-28T14:01:33.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 -- 172.21.15.42:0/3414087163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6e4071de0 msgr2=0x7ff6e40721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 --2- 172.21.15.42:0/3414087163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6e4071de0 0x7ff6e40721c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40702b0 tx=0x7ff6d401c400 comp rx=0 tx=0).stop 2024-05-28T14:01:33.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 -- 172.21.15.42:0/3414087163 shutdown_connections 2024-05-28T14:01:33.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 --2- 172.21.15.42:0/3414087163 >> v2:172.21.15.63:3300/0 conn(0x7ff6e4141d10 0x7ff6e41421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 --2- 172.21.15.42:0/3414087163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6e40ce3b0 0x7ff6e40ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 --2- 172.21.15.42:0/3414087163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6e4071de0 0x7ff6e40721c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 -- 172.21.15.42:0/3414087163 >> 172.21.15.42:0/3414087163 conn(0x7ff6e40b21f0 msgr2=0x7ff6e40b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:33.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 -- 172.21.15.42:0/3414087163 shutdown_connections 2024-05-28T14:01:33.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 -- 172.21.15.42:0/3414087163 wait complete. 2024-05-28T14:01:33.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 Processor -- start 2024-05-28T14:01:33.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 -- start start 2024-05-28T14:01:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6e4071de0 0x7ff6e41f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6e40ce3b0 0x7ff6e413efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6e4141d10 0x7ff6e413d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e4140520 con 0x7ff6e4071de0 2024-05-28T14:01:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e413f8c0 con 0x7ff6e4141d10 2024-05-28T14:01:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6ea400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e41406a0 con 0x7ff6e40ce3b0 2024-05-28T14:01:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6e3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6e4071de0 0x7ff6e41f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6e3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6e4071de0 0x7ff6e41f78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57758/0 (socket says 172.21.15.42:57758) 2024-05-28T14:01:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6e3400640 1 -- 172.21.15.42:0/3293651417 learned_addr learned my addr 172.21.15.42:0/3293651417 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6e3e00640 1 --2- 172.21.15.42:0/3293651417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6e4141d10 0x7ff6e413d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6dbe00640 1 --2- 172.21.15.42:0/3293651417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6e40ce3b0 0x7ff6e413efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6e3400640 1 -- 172.21.15.42:0/3293651417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6e40ce3b0 msgr2=0x7ff6e413efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6e3400640 1 --2- 172.21.15.42:0/3293651417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6e40ce3b0 0x7ff6e413efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6e3400640 1 -- 172.21.15.42:0/3293651417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6e4141d10 msgr2=0x7ff6e413d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6e3400640 1 --2- 172.21.15.42:0/3293651417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6e4141d10 0x7ff6e413d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6e3400640 1 -- 172.21.15.42:0/3293651417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e413df00 con 0x7ff6e4071de0 2024-05-28T14:01:33.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.208+0000 7ff6e3400640 1 --2- 172.21.15.42:0/3293651417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6e4071de0 0x7ff6e41f78d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40ae040 tx=0x7ff6d40191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:33.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.212+0000 7ff6db400640 1 -- 172.21.15.42:0/3293651417 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40b0040 con 0x7ff6e4071de0 2024-05-28T14:01:33.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.212+0000 7ff6db400640 1 -- 172.21.15.42:0/3293651417 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d40b9070 con 0x7ff6e4071de0 2024-05-28T14:01:33.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.212+0000 7ff6ea400640 1 -- 172.21.15.42:0/3293651417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e413e1e0 con 0x7ff6e4071de0 2024-05-28T14:01:33.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.212+0000 7ff6db400640 1 -- 172.21.15.42:0/3293651417 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40b4440 con 0x7ff6e4071de0 2024-05-28T14:01:33.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.212+0000 7ff6ea400640 1 -- 172.21.15.42:0/3293651417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e413e7a0 con 0x7ff6e4071de0 2024-05-28T14:01:33.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.212+0000 7ff6ea400640 1 -- 172.21.15.42:0/3293651417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6e40738b0 con 0x7ff6e4071de0 2024-05-28T14:01:33.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.212+0000 7ff6db400640 1 -- 172.21.15.42:0/3293651417 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7ff6d40b45e0 con 0x7ff6e4071de0 2024-05-28T14:01:33.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.212+0000 7ff6db400640 1 --2- 172.21.15.42:0/3293651417 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff6c0068680 0x7ff6c006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.212+0000 7ff6db400640 1 -- 172.21.15.42:0/3293651417 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ff6d4121530 con 0x7ff6e4071de0 2024-05-28T14:01:33.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.212+0000 7ff6dbe00640 1 --2- 172.21.15.42:0/3293651417 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff6c0068680 0x7ff6c006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.216+0000 7ff6dbe00640 1 --2- 172.21.15.42:0/3293651417 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff6c0068680 0x7ff6c006ab40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff6e41fb330 tx=0x7ff6cc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:33.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.216+0000 7ff6db400640 1 -- 172.21.15.42:0/3293651417 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff6d4016110 con 0x7ff6e4071de0 2024-05-28T14:01:33.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.384+0000 7ff6ea400640 1 -- 172.21.15.42:0/3293651417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff6e404fdd0 con 0x7ff6e4071de0 2024-05-28T14:01:33.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.384+0000 7ff6db400640 1 -- 172.21.15.42:0/3293651417 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7ff6d40f1700 con 0x7ff6e4071de0 2024-05-28T14:01:33.371 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:01:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.388+0000 7ff6ea400640 1 -- 172.21.15.42:0/3293651417 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff6c0068680 msgr2=0x7ff6c006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.388+0000 7ff6ea400640 1 --2- 172.21.15.42:0/3293651417 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff6c0068680 0x7ff6c006ab40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff6e41fb330 tx=0x7ff6cc008040 comp rx=0 tx=0).stop 2024-05-28T14:01:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.388+0000 7ff6ea400640 1 -- 172.21.15.42:0/3293651417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6e4071de0 msgr2=0x7ff6e41f78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.388+0000 7ff6ea400640 1 --2- 172.21.15.42:0/3293651417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6e4071de0 0x7ff6e41f78d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40ae040 tx=0x7ff6d40191a0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.388+0000 7ff6ea400640 1 -- 172.21.15.42:0/3293651417 shutdown_connections 2024-05-28T14:01:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.388+0000 7ff6ea400640 1 --2- 172.21.15.42:0/3293651417 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff6c0068680 0x7ff6c006ab40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.388+0000 7ff6ea400640 1 --2- 172.21.15.42:0/3293651417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6e4141d10 0x7ff6e413d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.388+0000 7ff6ea400640 1 --2- 172.21.15.42:0/3293651417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6e40ce3b0 0x7ff6e413efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.388+0000 7ff6ea400640 1 --2- 172.21.15.42:0/3293651417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6e4071de0 0x7ff6e41f78d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.388+0000 7ff6ea400640 1 -- 172.21.15.42:0/3293651417 >> 172.21.15.42:0/3293651417 conn(0x7ff6e40b21f0 msgr2=0x7ff6e406dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:33.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.388+0000 7ff6ea400640 1 -- 172.21.15.42:0/3293651417 shutdown_connections 2024-05-28T14:01:33.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.388+0000 7ff6ea400640 1 -- 172.21.15.42:0/3293651417 wait complete. 2024-05-28T14:01:33.393 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:01:33.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f8855200640 1 Processor -- start 2024-05-28T14:01:33.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f8855200640 1 -- start start 2024-05-28T14:01:33.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f8855200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8850148090 0x7f88500b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f8855200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f88500b4c10 0x7f8850071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f8855200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88500b3690 con 0x7f8850072320 2024-05-28T14:01:33.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f8855200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8850148470 con 0x7f88500b4c10 2024-05-28T14:01:33.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f8855200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8850072770 con 0x7f8850148090 2024-05-28T14:01:33.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f8847e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f88500b4c10 0x7f8850071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f8847e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f88500b4c10 0x7f8850071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57770/0 (socket says 172.21.15.42:57770) 2024-05-28T14:01:33.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f8847e00640 1 -- 172.21.15.42:0/2748052704 learned_addr learned my addr 172.21.15.42:0/2748052704 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:33.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f884de00640 1 --2- 172.21.15.42:0/2748052704 >> v2:172.21.15.63:3300/0 conn(0x7f8850148090 0x7f88500b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f8847e00640 1 -- 172.21.15.42:0/2748052704 >> v2:172.21.15.63:3300/0 conn(0x7f8850148090 msgr2=0x7f88500b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:33.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f8847e00640 1 --2- 172.21.15.42:0/2748052704 >> v2:172.21.15.63:3300/0 conn(0x7f8850148090 0x7f88500b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f8847e00640 1 -- 172.21.15.42:0/2748052704 >> v1:172.21.15.63:6789/0 conn(0x7f8850072320 legacy=0x7f88501623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.552+0000 7f8847e00640 1 -- 172.21.15.42:0/2748052704 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88500b3970 con 0x7f88500b4c10 2024-05-28T14:01:33.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.556+0000 7f8847e00640 1 --2- 172.21.15.42:0/2748052704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88500b4c10 0x7f8850071de0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8850071730 tx=0x7f883801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e69433ea7e2984 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:33.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.556+0000 7f884d400640 1 -- 172.21.15.42:0/2748052704 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88380b2070 con 0x7f88500b4c10 2024-05-28T14:01:33.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.556+0000 7f884d400640 1 -- 172.21.15.42:0/2748052704 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88380ad680 con 0x7f88500b4c10 2024-05-28T14:01:33.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.556+0000 7f884d400640 1 -- 172.21.15.42:0/2748052704 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88380ada50 con 0x7f88500b4c10 2024-05-28T14:01:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.556+0000 7f8855200640 1 -- 172.21.15.42:0/2748052704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88500b4c10 msgr2=0x7f8850071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.556+0000 7f8855200640 1 --2- 172.21.15.42:0/2748052704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88500b4c10 0x7f8850071de0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8850071730 tx=0x7f883801c490 comp rx=0 tx=0).stop 2024-05-28T14:01:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.556+0000 7f8855200640 1 -- 172.21.15.42:0/2748052704 shutdown_connections 2024-05-28T14:01:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.556+0000 7f8855200640 1 --2- 172.21.15.42:0/2748052704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88500b4c10 0x7f8850071de0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.556+0000 7f8855200640 1 --2- 172.21.15.42:0/2748052704 >> v2:172.21.15.63:3300/0 conn(0x7f8850148090 0x7f88500b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.556+0000 7f8855200640 1 -- 172.21.15.42:0/2748052704 >> 172.21.15.42:0/2748052704 conn(0x7f88500b21f0 msgr2=0x7f88500b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.556+0000 7f8855200640 1 -- 172.21.15.42:0/2748052704 shutdown_connections 2024-05-28T14:01:33.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.556+0000 7f8855200640 1 -- 172.21.15.42:0/2748052704 wait complete. 2024-05-28T14:01:33.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8855200640 1 Processor -- start 2024-05-28T14:01:33.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8855200640 1 -- start start 2024-05-28T14:01:33.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8855200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8850072320 0x7f885015ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8855200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8850148090 0x7f88500b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8855200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8850158150 0x7f8850161700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8855200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f88500b57e0 con 0x7f8850072320 2024-05-28T14:01:33.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8855200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88500b5660 con 0x7f8850148090 2024-05-28T14:01:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8855200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f885016b540 con 0x7f8850158150 2024-05-28T14:01:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f884de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8850072320 0x7f885015ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f884de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8850072320 0x7f885015ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57782/0 (socket says 172.21.15.42:57782) 2024-05-28T14:01:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f884de00640 1 -- 172.21.15.42:0/2703250392 learned_addr learned my addr 172.21.15.42:0/2703250392 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8847e00640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8850148090 0x7f88500b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f884e800640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8850158150 0x7f8850161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f884de00640 1 -- 172.21.15.42:0/2703250392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8850158150 msgr2=0x7f8850161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f884de00640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8850158150 0x7f8850161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f884de00640 1 -- 172.21.15.42:0/2703250392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8850148090 msgr2=0x7f88500b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f884de00640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8850148090 0x7f88500b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f884de00640 1 -- 172.21.15.42:0/2703250392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8850160660 con 0x7f8850072320 2024-05-28T14:01:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8847e00640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8850148090 0x7f88500b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f884de00640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8850072320 0x7f885015ff60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f884000e8a0 tx=0x7f884000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:33.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8846000640 1 -- 172.21.15.42:0/2703250392 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840017070 con 0x7f8850072320 2024-05-28T14:01:33.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8846000640 1 -- 172.21.15.42:0/2703250392 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f884000c940 con 0x7f8850072320 2024-05-28T14:01:33.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8846000640 1 -- 172.21.15.42:0/2703250392 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884000ccf0 con 0x7f8850072320 2024-05-28T14:01:33.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8855200640 1 -- 172.21.15.42:0/2703250392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8850160940 con 0x7f8850072320 2024-05-28T14:01:33.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8855200640 1 -- 172.21.15.42:0/2703250392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8850160f00 con 0x7f8850072320 2024-05-28T14:01:33.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.560+0000 7f8855200640 1 -- 172.21.15.42:0/2703250392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88500738b0 con 0x7f8850072320 2024-05-28T14:01:33.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.564+0000 7f8846000640 1 -- 172.21.15.42:0/2703250392 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f884001a070 con 0x7f8850072320 2024-05-28T14:01:33.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.564+0000 7f8846000640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f882c068680 0x7f882c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.564+0000 7f8846000640 1 -- 172.21.15.42:0/2703250392 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8840086fe0 con 0x7f8850072320 2024-05-28T14:01:33.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.564+0000 7f8847e00640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f882c068680 0x7f882c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.564+0000 7f8847e00640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f882c068680 0x7f882c06ab40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f88380af000 tx=0x7f8838008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:33.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.568+0000 7f8846000640 1 -- 172.21.15.42:0/2703250392 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f88400572e0 con 0x7f8850072320 2024-05-28T14:01:33.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.732+0000 7f8855200640 1 -- 172.21.15.42:0/2703250392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f885004fdd0 con 0x7f8850072320 2024-05-28T14:01:33.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.732+0000 7f8846000640 1 -- 172.21.15.42:0/2703250392 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f8840056be0 con 0x7f8850072320 2024-05-28T14:01:33.717 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:01:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.732+0000 7f8855200640 1 -- 172.21.15.42:0/2703250392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f882c068680 msgr2=0x7f882c06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.732+0000 7f8855200640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f882c068680 0x7f882c06ab40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f88380af000 tx=0x7f8838008040 comp rx=0 tx=0).stop 2024-05-28T14:01:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.732+0000 7f8855200640 1 -- 172.21.15.42:0/2703250392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8850072320 msgr2=0x7f885015ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.732+0000 7f8855200640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8850072320 0x7f885015ff60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f884000e8a0 tx=0x7f884000eca0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.732+0000 7f8855200640 1 -- 172.21.15.42:0/2703250392 shutdown_connections 2024-05-28T14:01:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.732+0000 7f8855200640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f882c068680 0x7f882c06ab40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.732+0000 7f8855200640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8850158150 0x7f8850161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.732+0000 7f8855200640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8850148090 0x7f88500b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.732+0000 7f8855200640 1 --2- 172.21.15.42:0/2703250392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8850072320 0x7f885015ff60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.732+0000 7f8855200640 1 -- 172.21.15.42:0/2703250392 >> 172.21.15.42:0/2703250392 conn(0x7f88500b21f0 msgr2=0x7f88501415a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.732+0000 7f8855200640 1 -- 172.21.15.42:0/2703250392 shutdown_connections 2024-05-28T14:01:33.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.732+0000 7f8855200640 1 -- 172.21.15.42:0/2703250392 wait complete. 2024-05-28T14:01:33.739 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T14:01:33.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.892+0000 7f8e67a00640 1 Processor -- start 2024-05-28T14:01:33.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.892+0000 7f8e67a00640 1 -- start start 2024-05-28T14:01:33.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e67a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e6015bf30 0x7f8e6015c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e67a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8e600cf230 0x7f8e60072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e67a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e600cfc10 con 0x7f8e60072ba0 2024-05-28T14:01:33.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e67a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8e600721d0 con 0x7f8e600cf230 2024-05-28T14:01:33.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e67a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8e600cfef0 con 0x7f8e6015bf30 2024-05-28T14:01:33.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e65000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e6015bf30 0x7f8e6015c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e5fe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8e600cf230 0x7f8e60072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e65a00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f8e60072ba0 0x7f8e6006ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:55520/0 (socket says 172.21.15.42:55520) 2024-05-28T14:01:33.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e65a00640 1 -- 172.21.15.42:0/1323161492 learned_addr learned my addr 172.21.15.42:0/1323161492 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:33.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e65000640 1 --2- 172.21.15.42:0/1323161492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e6015bf30 0x7f8e6015c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:45194/0 (socket says 172.21.15.42:45194) 2024-05-28T14:01:33.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e65000640 1 -- 172.21.15.42:0/1323161492 >> v2:172.21.15.63:3300/0 conn(0x7f8e600cf230 msgr2=0x7f8e60072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:33.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e65000640 1 --2- 172.21.15.42:0/1323161492 >> v2:172.21.15.63:3300/0 conn(0x7f8e600cf230 0x7f8e60072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e65000640 1 -- 172.21.15.42:0/1323161492 >> v1:172.21.15.63:6789/0 conn(0x7f8e60072ba0 legacy=0x7f8e6006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e65000640 1 -- 172.21.15.42:0/1323161492 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e600d01d0 con 0x7f8e6015bf30 2024-05-28T14:01:33.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.896+0000 7f8e5fe00640 1 --2- 172.21.15.42:0/1323161492 >> v2:172.21.15.63:3300/0 conn(0x7f8e600cf230 0x7f8e60072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:01:33.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.900+0000 7f8e65000640 1 --2- 172.21.15.42:0/1323161492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e6015bf30 0x7f8e6015c310 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8e540097c0 tx=0x7f8e5401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a855b73685af3ee server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:33.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.900+0000 7f8e5f400640 1 -- 172.21.15.42:0/1323161492 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e540b2070 con 0x7f8e6015bf30 2024-05-28T14:01:33.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.900+0000 7f8e5f400640 1 -- 172.21.15.42:0/1323161492 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e540ad640 con 0x7f8e6015bf30 2024-05-28T14:01:33.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.900+0000 7f8e5f400640 1 -- 172.21.15.42:0/1323161492 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e540ada10 con 0x7f8e6015bf30 2024-05-28T14:01:33.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.900+0000 7f8e67a00640 1 -- 172.21.15.42:0/1323161492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e6015bf30 msgr2=0x7f8e6015c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.900+0000 7f8e67a00640 1 --2- 172.21.15.42:0/1323161492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e6015bf30 0x7f8e6015c310 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8e540097c0 tx=0x7f8e5401c470 comp rx=0 tx=0).stop 2024-05-28T14:01:33.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.900+0000 7f8e67a00640 1 -- 172.21.15.42:0/1323161492 shutdown_connections 2024-05-28T14:01:33.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.900+0000 7f8e67a00640 1 --2- 172.21.15.42:0/1323161492 >> v2:172.21.15.63:3300/0 conn(0x7f8e600cf230 0x7f8e60072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.900+0000 7f8e67a00640 1 --2- 172.21.15.42:0/1323161492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e6015bf30 0x7f8e6015c310 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.900+0000 7f8e67a00640 1 -- 172.21.15.42:0/1323161492 >> 172.21.15.42:0/1323161492 conn(0x7f8e6006b9f0 msgr2=0x7f8e6006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:33.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.900+0000 7f8e67a00640 1 -- 172.21.15.42:0/1323161492 shutdown_connections 2024-05-28T14:01:33.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.900+0000 7f8e67a00640 1 -- 172.21.15.42:0/1323161492 wait complete. 2024-05-28T14:01:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.900+0000 7f8e67a00640 1 Processor -- start 2024-05-28T14:01:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e67a00640 1 -- start start 2024-05-28T14:01:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e67a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e60072ba0 0x7f8e600b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e67a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e600cf230 0x7f8e60149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e67a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e6015bf30 0x7f8e601f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e67a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6006f4e0 con 0x7f8e600cf230 2024-05-28T14:01:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e67a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6006f360 con 0x7f8e60072ba0 2024-05-28T14:01:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e67a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8e600cf610 con 0x7f8e6015bf30 2024-05-28T14:01:33.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e5fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e600cf230 0x7f8e60149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e5fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e600cf230 0x7f8e60149fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57792/0 (socket says 172.21.15.42:57792) 2024-05-28T14:01:33.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e5fe00640 1 -- 172.21.15.42:0/3700362486 learned_addr learned my addr 172.21.15.42:0/3700362486 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:33.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e65a00640 1 --2- 172.21.15.42:0/3700362486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e6015bf30 0x7f8e601f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e65000640 1 --2- 172.21.15.42:0/3700362486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e60072ba0 0x7f8e600b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e5fe00640 1 -- 172.21.15.42:0/3700362486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e6015bf30 msgr2=0x7f8e601f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e5fe00640 1 --2- 172.21.15.42:0/3700362486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e6015bf30 0x7f8e601f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e5fe00640 1 -- 172.21.15.42:0/3700362486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e60072ba0 msgr2=0x7f8e600b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:33.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e5fe00640 1 --2- 172.21.15.42:0/3700362486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e60072ba0 0x7f8e600b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:33.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e5fe00640 1 -- 172.21.15.42:0/3700362486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e60155440 con 0x7f8e600cf230 2024-05-28T14:01:33.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e5fe00640 1 --2- 172.21.15.42:0/3700362486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e600cf230 0x7f8e60149fd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00ebe0 tx=0x7f8e4c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:33.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e5cc00640 1 -- 172.21.15.42:0/3700362486 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c017070 con 0x7f8e600cf230 2024-05-28T14:01:33.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e5cc00640 1 -- 172.21.15.42:0/3700362486 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e4c00cc30 con 0x7f8e600cf230 2024-05-28T14:01:33.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e67a00640 1 -- 172.21.15.42:0/3700362486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e60155720 con 0x7f8e600cf230 2024-05-28T14:01:33.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e5cc00640 1 -- 172.21.15.42:0/3700362486 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c013470 con 0x7f8e600cf230 2024-05-28T14:01:33.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e67a00640 1 -- 172.21.15.42:0/3700362486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e60155ce0 con 0x7f8e600cf230 2024-05-28T14:01:33.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.904+0000 7f8e67a00640 1 -- 172.21.15.42:0/3700362486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e60073890 con 0x7f8e600cf230 2024-05-28T14:01:33.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.908+0000 7f8e5cc00640 1 -- 172.21.15.42:0/3700362486 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f8e4c01a070 con 0x7f8e600cf230 2024-05-28T14:01:33.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.908+0000 7f8e5cc00640 1 --2- 172.21.15.42:0/3700362486 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e40068650 0x7f8e4006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:33.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.908+0000 7f8e5cc00640 1 -- 172.21.15.42:0/3700362486 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8e4c0870d0 con 0x7f8e600cf230 2024-05-28T14:01:33.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.908+0000 7f8e65000640 1 --2- 172.21.15.42:0/3700362486 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e40068650 0x7f8e4006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:33.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.908+0000 7f8e65000640 1 --2- 172.21.15.42:0/3700362486 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e40068650 0x7f8e4006ab10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8e540af000 tx=0x7f8e54008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:33.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:33.912+0000 7f8e5cc00640 1 -- 172.21.15.42:0/3700362486 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8e4c056cb0 con 0x7f8e600cf230 2024-05-28T14:01:34.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.088+0000 7f8e67a00640 1 -- 172.21.15.42:0/3700362486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8e6004fdd0 con 0x7f8e600cf230 2024-05-28T14:01:34.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.088+0000 7f8e5cc00640 1 -- 172.21.15.42:0/3700362486 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7f8e4c05afb0 con 0x7f8e600cf230 2024-05-28T14:01:34.074 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T14:01:34.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.092+0000 7f8e67a00640 1 -- 172.21.15.42:0/3700362486 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e40068650 msgr2=0x7f8e4006ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:34.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.092+0000 7f8e67a00640 1 --2- 172.21.15.42:0/3700362486 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e40068650 0x7f8e4006ab10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8e540af000 tx=0x7f8e54008040 comp rx=0 tx=0).stop 2024-05-28T14:01:34.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.092+0000 7f8e67a00640 1 -- 172.21.15.42:0/3700362486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e600cf230 msgr2=0x7f8e60149fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:34.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.092+0000 7f8e67a00640 1 --2- 172.21.15.42:0/3700362486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e600cf230 0x7f8e60149fd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00ebe0 tx=0x7f8e4c00c520 comp rx=0 tx=0).stop 2024-05-28T14:01:34.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.092+0000 7f8e67a00640 1 -- 172.21.15.42:0/3700362486 shutdown_connections 2024-05-28T14:01:34.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.092+0000 7f8e67a00640 1 --2- 172.21.15.42:0/3700362486 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e40068650 0x7f8e4006ab10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:34.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.092+0000 7f8e67a00640 1 --2- 172.21.15.42:0/3700362486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e6015bf30 0x7f8e601f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:34.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.092+0000 7f8e67a00640 1 --2- 172.21.15.42:0/3700362486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e600cf230 0x7f8e60149fd0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:34.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.092+0000 7f8e67a00640 1 --2- 172.21.15.42:0/3700362486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e60072ba0 0x7f8e600b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:34.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.092+0000 7f8e67a00640 1 -- 172.21.15.42:0/3700362486 >> 172.21.15.42:0/3700362486 conn(0x7f8e6006b9f0 msgr2=0x7f8e6006bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:34.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.092+0000 7f8e67a00640 1 -- 172.21.15.42:0/3700362486 shutdown_connections 2024-05-28T14:01:34.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.092+0000 7f8e67a00640 1 -- 172.21.15.42:0/3700362486 wait complete. 2024-05-28T14:01:34.096 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-05-28T14:01:34.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:34.096 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-28T14:01:34.115 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[36920]: fuse finished with error 0 and tester_r 0 2024-05-28T14:01:34.117 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:01:34.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:34.117 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-28T14:01:34.287 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:01:34.287 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:01:34.287 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:01:34.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:01:34.300 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:01:34.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:34.300 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-28T14:01:34.354 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-28T14:01:34.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:01:34.355 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:01:34.355 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:01:34.355 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:01:34.364 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:01:34.364 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:01:34.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:34.364 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-28T14:01:34.419 INFO:teuthology.orchestra.run.smithi042.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-28T14:01:34.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:01:34.420 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:01:34.420 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:01:34.420 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:01:34.429 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:01:34.429 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:01:34.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:34.430 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-28T14:01:34.478 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-28T14:01:34.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:01:34.479 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:01:34.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.680+0000 7f413d400640 1 Processor -- start 2024-05-28T14:01:34.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.680+0000 7f413d400640 1 -- start start 2024-05-28T14:01:34.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.684+0000 7f413d400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f413815c730 0x7f41380b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:34.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.684+0000 7f413d400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41380cca90 0x7f413806be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:34.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.684+0000 7f413d400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41380cdb80 con 0x7f413806b9f0 2024-05-28T14:01:34.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.684+0000 7f413d400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f41380cde60 con 0x7f413815c730 2024-05-28T14:01:34.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.684+0000 7f413d400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f41380ce140 con 0x7f41380cca90 2024-05-28T14:01:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.684+0000 7f4135e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f413806b9f0 0x7f41381601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:55532/0 (socket says 172.21.15.42:55532) 2024-05-28T14:01:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.684+0000 7f4135e00640 1 -- 172.21.15.42:0/3326602184 learned_addr learned my addr 172.21.15.42:0/3326602184 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.684+0000 7f4136800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f413815c730 0x7f41380b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.684+0000 7f4136800640 1 -- 172.21.15.42:0/3326602184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41380cca90 msgr2=0x7f413806be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.684+0000 7f4136800640 1 --2- 172.21.15.42:0/3326602184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41380cca90 0x7f413806be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.684+0000 7f4136800640 1 -- 172.21.15.42:0/3326602184 >> v1:172.21.15.63:6789/0 conn(0x7f413806b9f0 legacy=0x7f41381601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.684+0000 7f4135400640 1 -- 172.21.15.42:0/3326602184 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2448584601 0 0) 0x7f41380cdb80 con 0x7f413806b9f0 2024-05-28T14:01:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.684+0000 7f4136800640 1 -- 172.21.15.42:0/3326602184 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41380ce420 con 0x7f413815c730 2024-05-28T14:01:34.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.688+0000 7f4136800640 1 --2- 172.21.15.42:0/3326602184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f413815c730 0x7f41380b36e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f41280097c0 tx=0x7f412801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91a40545081e35e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:34.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.688+0000 7f4135400640 1 -- 172.21.15.42:0/3326602184 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41280b0070 con 0x7f413815c730 2024-05-28T14:01:34.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.688+0000 7f4135400640 1 -- 172.21.15.42:0/3326602184 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41280ab600 con 0x7f413815c730 2024-05-28T14:01:34.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.688+0000 7f4135400640 1 -- 172.21.15.42:0/3326602184 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41280ab9d0 con 0x7f413815c730 2024-05-28T14:01:34.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.692+0000 7f413d400640 1 -- 172.21.15.42:0/3326602184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f413815c730 msgr2=0x7f41380b36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:34.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.692+0000 7f413d400640 1 --2- 172.21.15.42:0/3326602184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f413815c730 0x7f41380b36e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f41280097c0 tx=0x7f412801c410 comp rx=0 tx=0).stop 2024-05-28T14:01:34.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.692+0000 7f413d400640 1 -- 172.21.15.42:0/3326602184 shutdown_connections 2024-05-28T14:01:34.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.692+0000 7f413d400640 1 --2- 172.21.15.42:0/3326602184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41380cca90 0x7f413806be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:34.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.692+0000 7f413d400640 1 --2- 172.21.15.42:0/3326602184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f413815c730 0x7f41380b36e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:34.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.692+0000 7f413d400640 1 -- 172.21.15.42:0/3326602184 >> 172.21.15.42:0/3326602184 conn(0x7f413809aa50 msgr2=0x7f41380b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:34.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.692+0000 7f413d400640 1 -- 172.21.15.42:0/3326602184 shutdown_connections 2024-05-28T14:01:34.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.692+0000 7f413d400640 1 -- 172.21.15.42:0/3326602184 wait complete. 2024-05-28T14:01:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f413d400640 1 Processor -- start 2024-05-28T14:01:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f413d400640 1 -- start start 2024-05-28T14:01:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f413d400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f413806b9f0 0x7f413814f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f413d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41380cca90 0x7f41380b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f413d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f413815c730 0x7f413814c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f413d400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f41380c6240 con 0x7f413815c730 2024-05-28T14:01:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f413d400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f41380b0980 con 0x7f41380cca90 2024-05-28T14:01:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f413d400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f41380c63c0 con 0x7f413806b9f0 2024-05-28T14:01:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f4136800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f413806b9f0 0x7f413814f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f4136800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f413806b9f0 0x7f413814f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:45222/0 (socket says 172.21.15.42:45222) 2024-05-28T14:01:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f4136800640 1 -- 172.21.15.42:0/3886082066 learned_addr learned my addr 172.21.15.42:0/3886082066 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f4135e00640 1 --2- 172.21.15.42:0/3886082066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41380cca90 0x7f41380b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f4137200640 1 --2- 172.21.15.42:0/3886082066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f413815c730 0x7f413814c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:34.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f4135e00640 1 -- 172.21.15.42:0/3886082066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f413806b9f0 msgr2=0x7f413814f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:34.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f4135e00640 1 --2- 172.21.15.42:0/3886082066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f413806b9f0 0x7f413814f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:34.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f4135e00640 1 -- 172.21.15.42:0/3886082066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f413815c730 msgr2=0x7f413814c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:34.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f4135e00640 1 --2- 172.21.15.42:0/3886082066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f413815c730 0x7f413814c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:34.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f4135e00640 1 -- 172.21.15.42:0/3886082066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413814cd20 con 0x7f41380cca90 2024-05-28T14:01:34.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.696+0000 7f4135e00640 1 --2- 172.21.15.42:0/3886082066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41380cca90 0x7f41380b69c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4124000c10 tx=0x7f412400c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:34.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.700+0000 7f412ea00640 1 -- 172.21.15.42:0/3886082066 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41240074e0 con 0x7f41380cca90 2024-05-28T14:01:34.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.700+0000 7f412ea00640 1 -- 172.21.15.42:0/3886082066 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4124007680 con 0x7f41380cca90 2024-05-28T14:01:34.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.700+0000 7f412ea00640 1 -- 172.21.15.42:0/3886082066 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124004a10 con 0x7f41380cca90 2024-05-28T14:01:34.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.700+0000 7f413d400640 1 -- 172.21.15.42:0/3886082066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41380c9da0 con 0x7f41380cca90 2024-05-28T14:01:34.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.700+0000 7f413d400640 1 -- 172.21.15.42:0/3886082066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41380ca360 con 0x7f41380cca90 2024-05-28T14:01:34.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.700+0000 7f413d400640 1 -- 172.21.15.42:0/3886082066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41380691a0 con 0x7f41380cca90 2024-05-28T14:01:34.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.704+0000 7f412ea00640 1 -- 172.21.15.42:0/3886082066 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f4124011070 con 0x7f41380cca90 2024-05-28T14:01:34.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.704+0000 7f412ea00640 1 --2- 172.21.15.42:0/3886082066 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4118068720 0x7f411806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:34.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.704+0000 7f412ea00640 1 -- 172.21.15.42:0/3886082066 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f41240868a0 con 0x7f41380cca90 2024-05-28T14:01:34.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.704+0000 7f4136800640 1 --2- 172.21.15.42:0/3886082066 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4118068720 0x7f411806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:34.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.704+0000 7f4136800640 1 --2- 172.21.15.42:0/3886082066 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4118068720 0x7f411806abe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f41280097c0 tx=0x7f4128008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:34.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:34.712+0000 7f412ea00640 1 -- 172.21.15.42:0/3886082066 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4124056470 con 0x7f41380cca90 2024-05-28T14:01:35.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.092+0000 7f413d400640 1 -- 172.21.15.42:0/3886082066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41380ccfc0 con 0x7f41380cca90 2024-05-28T14:01:35.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.096+0000 7f412ea00640 1 -- 172.21.15.42:0/3886082066 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5187 (secure 0 0 0) 0x7f412405a770 con 0x7f41380cca90 2024-05-28T14:01:35.082 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:35.082 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2044753493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2044753493},{"type":"v1","addr":"172.21.15.42:6835","nonce":2044753493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3332210819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3332210819},{"type":"v1","addr":"172.21.15.42:6837","nonce":3332210819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":54782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1050322479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1050322479},{"type":"v1","addr":"172.21.15.63:6834","nonce":1050322479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T13:59:26.845096+0000","modified":"2024-05-28T13:59:29.868153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":54785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54785":{"gid":54785,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/2114729398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2114729398},{"type":"v1","addr":"172.21.15.63:6835","nonce":2114729398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":54785,"qdb_cluster":[54785]},"id":2}]} 2024-05-28T14:01:35.082 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 18 2024-05-28T14:01:35.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.100+0000 7f413d400640 1 -- 172.21.15.42:0/3886082066 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4118068720 msgr2=0x7f411806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:35.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.100+0000 7f413d400640 1 --2- 172.21.15.42:0/3886082066 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4118068720 0x7f411806abe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f41280097c0 tx=0x7f4128008040 comp rx=0 tx=0).stop 2024-05-28T14:01:35.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.100+0000 7f413d400640 1 -- 172.21.15.42:0/3886082066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41380cca90 msgr2=0x7f41380b69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:35.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.100+0000 7f413d400640 1 --2- 172.21.15.42:0/3886082066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41380cca90 0x7f41380b69c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4124000c10 tx=0x7f412400c8e0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.100+0000 7f413d400640 1 -- 172.21.15.42:0/3886082066 shutdown_connections 2024-05-28T14:01:35.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.100+0000 7f413d400640 1 --2- 172.21.15.42:0/3886082066 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4118068720 0x7f411806abe0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.100+0000 7f413d400640 1 --2- 172.21.15.42:0/3886082066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f413815c730 0x7f413814c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.100+0000 7f413d400640 1 --2- 172.21.15.42:0/3886082066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41380cca90 0x7f41380b69c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.100+0000 7f413d400640 1 --2- 172.21.15.42:0/3886082066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f413806b9f0 0x7f413814f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.100+0000 7f413d400640 1 -- 172.21.15.42:0/3886082066 >> 172.21.15.42:0/3886082066 conn(0x7f413809aa50 msgr2=0x7f41380b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:35.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.100+0000 7f413d400640 1 -- 172.21.15.42:0/3886082066 shutdown_connections 2024-05-28T14:01:35.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.100+0000 7f413d400640 1 -- 172.21.15.42:0/3886082066 wait complete. 2024-05-28T14:01:35.121 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:01:35.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.352+0000 7f0163c00640 1 Processor -- start 2024-05-28T14:01:35.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.352+0000 7f0163c00640 1 -- start start 2024-05-28T14:01:35.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0163c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f015c162b30 0x7f015c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0163c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f015c0d0c30 0x7f015c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0163c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f015c0728e0 0x7f015c0b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0163c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f015c0d1090 con 0x7f015c0728e0 2024-05-28T14:01:35.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0163c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f015c0730d0 con 0x7f015c162b30 2024-05-28T14:01:35.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0163c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f015c0733b0 con 0x7f015c0d0c30 2024-05-28T14:01:35.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0161c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f015c0728e0 0x7f015c0b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0161200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f015c162b30 0x7f015c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0161c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f015c0728e0 0x7f015c0b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57816/0 (socket says 172.21.15.42:57816) 2024-05-28T14:01:35.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0161c00640 1 -- 172.21.15.42:0/3624418568 learned_addr learned my addr 172.21.15.42:0/3624418568 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:35.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f015be00640 1 --2- 172.21.15.42:0/3624418568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f015c0d0c30 0x7f015c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0161c00640 1 -- 172.21.15.42:0/3624418568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f015c0d0c30 msgr2=0x7f015c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:35.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0161c00640 1 --2- 172.21.15.42:0/3624418568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f015c0d0c30 0x7f015c0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0161c00640 1 -- 172.21.15.42:0/3624418568 >> v2:172.21.15.63:3300/0 conn(0x7f015c162b30 msgr2=0x7f015c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:35.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0161c00640 1 --2- 172.21.15.42:0/3624418568 >> v2:172.21.15.63:3300/0 conn(0x7f015c162b30 0x7f015c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0161c00640 1 -- 172.21.15.42:0/3624418568 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c073690 con 0x7f015c0728e0 2024-05-28T14:01:35.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.356+0000 7f0161200640 1 --2- 172.21.15.42:0/3624418568 >> v2:172.21.15.63:3300/0 conn(0x7f015c162b30 0x7f015c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:01:35.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.360+0000 7f0161c00640 1 --2- 172.21.15.42:0/3624418568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f015c0728e0 0x7f015c0b31e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f015c0b44a0 tx=0x7f014c01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553d1141637cdf77 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:35.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.360+0000 7f015b400640 1 -- 172.21.15.42:0/3624418568 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c0b0070 con 0x7f015c0728e0 2024-05-28T14:01:35.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.360+0000 7f015b400640 1 -- 172.21.15.42:0/3624418568 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f014c0ab600 con 0x7f015c0728e0 2024-05-28T14:01:35.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.360+0000 7f015b400640 1 -- 172.21.15.42:0/3624418568 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c0ab9d0 con 0x7f015c0728e0 2024-05-28T14:01:35.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.364+0000 7f0163c00640 1 -- 172.21.15.42:0/3624418568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f015c0728e0 msgr2=0x7f015c0b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:35.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.364+0000 7f0163c00640 1 --2- 172.21.15.42:0/3624418568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f015c0728e0 0x7f015c0b31e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f015c0b44a0 tx=0x7f014c01c230 comp rx=0 tx=0).stop 2024-05-28T14:01:35.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.364+0000 7f0163c00640 1 -- 172.21.15.42:0/3624418568 shutdown_connections 2024-05-28T14:01:35.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.364+0000 7f0163c00640 1 --2- 172.21.15.42:0/3624418568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f015c0728e0 0x7f015c0b31e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.364+0000 7f0163c00640 1 --2- 172.21.15.42:0/3624418568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f015c0d0c30 0x7f015c0723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.364+0000 7f0163c00640 1 --2- 172.21.15.42:0/3624418568 >> v2:172.21.15.63:3300/0 conn(0x7f015c162b30 0x7f015c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.364+0000 7f0163c00640 1 -- 172.21.15.42:0/3624418568 >> 172.21.15.42:0/3624418568 conn(0x7f015c06bdf0 msgr2=0x7f015c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:35.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.364+0000 7f0163c00640 1 -- 172.21.15.42:0/3624418568 shutdown_connections 2024-05-28T14:01:35.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.364+0000 7f0163c00640 1 -- 172.21.15.42:0/3624418568 wait complete. 2024-05-28T14:01:35.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.368+0000 7f0163c00640 1 Processor -- start 2024-05-28T14:01:35.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0163c00640 1 -- start start 2024-05-28T14:01:35.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0163c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f015c0728e0 0x7f015c1568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0163c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f015c0d0c30 0x7f015c2034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0163c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f015c162b30 0x7f015c153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0163c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f015c0ad230 con 0x7f015c0728e0 2024-05-28T14:01:35.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0163c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f015c072d70 con 0x7f015c162b30 2024-05-28T14:01:35.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0163c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f015c0ad3b0 con 0x7f015c0d0c30 2024-05-28T14:01:35.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0161200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f015c0728e0 0x7f015c1568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0161200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f015c0728e0 0x7f015c1568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57822/0 (socket says 172.21.15.42:57822) 2024-05-28T14:01:35.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0161200640 1 -- 172.21.15.42:0/3337326377 learned_addr learned my addr 172.21.15.42:0/3337326377 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:35.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0161c00640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f015c162b30 0x7f015c153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f015be00640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f015c0d0c30 0x7f015c2034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0161200640 1 -- 172.21.15.42:0/3337326377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f015c0d0c30 msgr2=0x7f015c2034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:35.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0161200640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f015c0d0c30 0x7f015c2034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0161200640 1 -- 172.21.15.42:0/3337326377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f015c162b30 msgr2=0x7f015c153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:35.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0161200640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f015c162b30 0x7f015c153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0161200640 1 -- 172.21.15.42:0/3337326377 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c1606b0 con 0x7f015c0728e0 2024-05-28T14:01:35.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f015be00640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f015c0d0c30 0x7f015c2034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:35.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0161c00640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f015c162b30 0x7f015c153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:35.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.372+0000 7f0161200640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f015c0728e0 0x7f015c1568a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0154009ec0 tx=0x7f015400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:35.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.376+0000 7f0158c00640 1 -- 172.21.15.42:0/3337326377 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154018020 con 0x7f015c0728e0 2024-05-28T14:01:35.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.376+0000 7f0158c00640 1 -- 172.21.15.42:0/3337326377 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0154014680 con 0x7f015c0728e0 2024-05-28T14:01:35.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.376+0000 7f0158c00640 1 -- 172.21.15.42:0/3337326377 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154014a50 con 0x7f015c0728e0 2024-05-28T14:01:35.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.376+0000 7f0163c00640 1 -- 172.21.15.42:0/3337326377 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015c160990 con 0x7f015c0728e0 2024-05-28T14:01:35.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.376+0000 7f0163c00640 1 -- 172.21.15.42:0/3337326377 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015c156380 con 0x7f015c0728e0 2024-05-28T14:01:35.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.376+0000 7f0163c00640 1 -- 172.21.15.42:0/3337326377 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015c068a90 con 0x7f015c0728e0 2024-05-28T14:01:35.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.376+0000 7f0158c00640 1 -- 172.21.15.42:0/3337326377 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f015401b020 con 0x7f015c0728e0 2024-05-28T14:01:35.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.380+0000 7f0158c00640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f01380686a0 0x7f013806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.380+0000 7f0158c00640 1 -- 172.21.15.42:0/3337326377 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5765+0+0 (secure 0 0 0) 0x7f0154087d10 con 0x7f015c0728e0 2024-05-28T14:01:35.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.380+0000 7f015be00640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f01380686a0 0x7f013806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.380+0000 7f015be00640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f01380686a0 0x7f013806ab60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f015c06c4f0 tx=0x7f0144008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:35.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.384+0000 7f0158c00640 1 -- 172.21.15.42:0/3337326377 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0154022030 con 0x7f015c0728e0 2024-05-28T14:01:35.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.676+0000 7f0163c00640 1 -- 172.21.15.42:0/3337326377 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f015c06cd20 con 0x7f015c0728e0 2024-05-28T14:01:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.676+0000 7f0158c00640 1 -- 172.21.15.42:0/3337326377 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16717 (secure 0 0 0) 0x7f0154010040 con 0x7f015c0728e0 2024-05-28T14:01:35.664 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:35.664 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":35,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:01:34.969835+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:59:24.673711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-28T13:59:25.795430+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":35,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:01:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.680+0000 7f0163c00640 1 -- 172.21.15.42:0/3337326377 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f01380686a0 msgr2=0x7f013806ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.680+0000 7f0163c00640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f01380686a0 0x7f013806ab60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f015c06c4f0 tx=0x7f0144008040 comp rx=0 tx=0).stop 2024-05-28T14:01:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.680+0000 7f0163c00640 1 -- 172.21.15.42:0/3337326377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f015c0728e0 msgr2=0x7f015c1568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.680+0000 7f0163c00640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f015c0728e0 0x7f015c1568a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0154009ec0 tx=0x7f015400c490 comp rx=0 tx=0).stop 2024-05-28T14:01:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.680+0000 7f0163c00640 1 -- 172.21.15.42:0/3337326377 shutdown_connections 2024-05-28T14:01:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.680+0000 7f0163c00640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f01380686a0 0x7f013806ab60 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f015c06c4f0 tx=0x7f0144008040 comp rx=0 tx=0).stop 2024-05-28T14:01:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.680+0000 7f0163c00640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f015c162b30 0x7f015c153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.680+0000 7f0163c00640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f015c0d0c30 0x7f015c2034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.680+0000 7f0163c00640 1 --2- 172.21.15.42:0/3337326377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f015c0728e0 0x7f015c1568a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.680+0000 7f0163c00640 1 -- 172.21.15.42:0/3337326377 >> 172.21.15.42:0/3337326377 conn(0x7f015c06bdf0 msgr2=0x7f015c0aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:35.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.680+0000 7f0163c00640 1 -- 172.21.15.42:0/3337326377 shutdown_connections 2024-05-28T14:01:35.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.684+0000 7f0163c00640 1 -- 172.21.15.42:0/3337326377 wait complete. 2024-05-28T14:01:35.703 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T13:59:24.673711+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T13:59:25.795430+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 35, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:01:35.703 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:01:35.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.940+0000 7f5c07e00640 1 Processor -- start 2024-05-28T14:01:35.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.940+0000 7f5c07e00640 1 -- start start 2024-05-28T14:01:35.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c07e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5c0015ba70 0x7f5c00154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c07e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0006f2c0 0x7f5c0006f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c07e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c00072710 con 0x7f5c000b2d10 2024-05-28T14:01:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c05400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f5c000b2d10 0x7f5c00071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:49758/0 (socket says 172.21.15.42:49758) 2024-05-28T14:01:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c05400640 1 -- 172.21.15.42:0/4073523568 learned_addr learned my addr 172.21.15.42:0/4073523568 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:35.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c04a00640 1 --2- 172.21.15.42:0/4073523568 >> v2:172.21.15.63:3300/0 conn(0x7f5c0015ba70 0x7f5c00154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c05e00640 1 --2- 172.21.15.42:0/4073523568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0006f2c0 0x7f5c0006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c07e00640 1 -- 172.21.15.42:0/4073523568 --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5c000724f0 con 0x7f5c0015ba70 2024-05-28T14:01:35.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c07e00640 1 -- 172.21.15.42:0/4073523568 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5c0006fc70 con 0x7f5c0006f2c0 2024-05-28T14:01:35.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 -- 172.21.15.42:0/4073523568 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3038902615 0 0) 0x7f5c00072710 con 0x7f5c000b2d10 2024-05-28T14:01:35.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 -- 172.21.15.42:0/4073523568 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5be8003530 con 0x7f5c000b2d10 2024-05-28T14:01:35.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 -- 172.21.15.42:0/4073523568 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1228744207 0 0) 0x7f5be8003530 con 0x7f5c000b2d10 2024-05-28T14:01:35.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 -- 172.21.15.42:0/4073523568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0006f2c0 msgr2=0x7f5c0006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:35.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 --2- 172.21.15.42:0/4073523568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0006f2c0 0x7f5c0006f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 -- 172.21.15.42:0/4073523568 >> v2:172.21.15.63:3300/0 conn(0x7f5c0015ba70 msgr2=0x7f5c00154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:35.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 --2- 172.21.15.42:0/4073523568 >> v2:172.21.15.63:3300/0 conn(0x7f5c0015ba70 0x7f5c00154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 -- 172.21.15.42:0/4073523568 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0015e330 con 0x7f5c000b2d10 2024-05-28T14:01:35.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 -- 172.21.15.42:0/4073523568 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f5bf4001680 con 0x7f5c000b2d10 2024-05-28T14:01:35.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 -- 172.21.15.42:0/4073523568 >> v1:172.21.15.42:6789/0 conn(0x7f5c000b2d10 legacy=0x7f5c00071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:35.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 --2- 172.21.15.42:0/4073523568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be80039c0 0x7f5be8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 --2- 172.21.15.42:0/4073523568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5be8004810 0x7f5be8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 --2- 172.21.15.42:0/4073523568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80051d0 0x7f5be800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 -- 172.21.15.42:0/4073523568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5c000724f0 con 0x7f5be8004810 2024-05-28T14:01:35.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 -- 172.21.15.42:0/4073523568 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0006fc70 con 0x7f5be80039c0 2024-05-28T14:01:35.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5bffe00640 1 -- 172.21.15.42:0/4073523568 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5be800bf40 con 0x7f5be80051d0 2024-05-28T14:01:35.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c05e00640 1 --2- 172.21.15.42:0/4073523568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0006f2c0 0x7f5c0006f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:01:35.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c04a00640 1 --2- 172.21.15.42:0/4073523568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5be8004810 0x7f5be8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c05e00640 1 --2- 172.21.15.42:0/4073523568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80051d0 0x7f5be800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c05400640 1 --2- 172.21.15.42:0/4073523568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be80039c0 0x7f5be8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c05e00640 1 -- 172.21.15.42:0/4073523568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be80039c0 msgr2=0x7f5be8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:35.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c05e00640 1 --2- 172.21.15.42:0/4073523568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be80039c0 0x7f5be8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c05e00640 1 -- 172.21.15.42:0/4073523568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5be8004810 msgr2=0x7f5be8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c05e00640 1 --2- 172.21.15.42:0/4073523568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5be8004810 0x7f5be8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.944+0000 7f5c05e00640 1 -- 172.21.15.42:0/4073523568 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be8003530 con 0x7f5be80051d0 2024-05-28T14:01:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.948+0000 7f5c05e00640 1 --2- 172.21.15.42:0/4073523568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80051d0 0x7f5be800b9d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c0006f6a0 tx=0x7f5bf801cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a3ad97ab2551e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.952+0000 7f5bffe00640 1 -- 172.21.15.42:0/4073523568 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8009d60 con 0x7f5be80051d0 2024-05-28T14:01:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.952+0000 7f5bffe00640 1 -- 172.21.15.42:0/4073523568 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf80a5d90 con 0x7f5be80051d0 2024-05-28T14:01:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.952+0000 7f5bffe00640 1 -- 172.21.15.42:0/4073523568 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8003bc0 con 0x7f5be80051d0 2024-05-28T14:01:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.952+0000 7f5c07e00640 1 -- 172.21.15.42:0/4073523568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80051d0 msgr2=0x7f5be800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.952+0000 7f5c07e00640 1 --2- 172.21.15.42:0/4073523568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80051d0 0x7f5be800b9d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c0006f6a0 tx=0x7f5bf801cce0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.952+0000 7f5c05e00640 1 -- 172.21.15.42:0/4073523568 reap_dead start 2024-05-28T14:01:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.952+0000 7f5c07e00640 1 -- 172.21.15.42:0/4073523568 shutdown_connections 2024-05-28T14:01:35.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.952+0000 7f5c07e00640 1 -- 172.21.15.42:0/4073523568 >> 172.21.15.42:0/4073523568 conn(0x7f5c000b21f0 msgr2=0x7f5c000b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:35.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.952+0000 7f5c07e00640 1 -- 172.21.15.42:0/4073523568 shutdown_connections 2024-05-28T14:01:35.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.952+0000 7f5c07e00640 1 -- 172.21.15.42:0/4073523568 wait complete. 2024-05-28T14:01:35.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c07e00640 1 Processor -- start 2024-05-28T14:01:35.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c07e00640 1 -- start start 2024-05-28T14:01:35.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c07e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80051d0 0x7f5c001501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c07e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c00158850 0x7f5c00158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c07e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c000b4400 0x7f5c000b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c07e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0015ebd0 con 0x7f5c00158850 2024-05-28T14:01:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c07e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0015e8f0 con 0x7f5c000b4400 2024-05-28T14:01:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c07e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5c0015f0f0 con 0x7f5be80051d0 2024-05-28T14:01:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c04a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c00158850 0x7f5c00158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c04a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c00158850 0x7f5c00158cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57828/0 (socket says 172.21.15.42:57828) 2024-05-28T14:01:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c04a00640 1 -- 172.21.15.42:0/1858033114 learned_addr learned my addr 172.21.15.42:0/1858033114 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c05400640 1 --2- 172.21.15.42:0/1858033114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80051d0 0x7f5c001501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c04a00640 1 -- 172.21.15.42:0/1858033114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80051d0 msgr2=0x7f5c001501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c04a00640 1 --2- 172.21.15.42:0/1858033114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80051d0 0x7f5c001501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c04a00640 1 -- 172.21.15.42:0/1858033114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c000b4400 msgr2=0x7f5c000b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c04a00640 1 --2- 172.21.15.42:0/1858033114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c000b4400 0x7f5c000b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:35.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c04a00640 1 -- 172.21.15.42:0/1858033114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c001577f0 con 0x7f5c00158850 2024-05-28T14:01:35.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c05400640 1 --2- 172.21.15.42:0/1858033114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80051d0 0x7f5c001501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:35.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c04a00640 1 --2- 172.21.15.42:0/1858033114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c00158850 0x7f5c00158cd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5bec010940 tx=0x7f5bec010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:35.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5bfd600640 1 -- 172.21.15.42:0/1858033114 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec01a020 con 0x7f5c00158850 2024-05-28T14:01:35.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c07e00640 1 -- 172.21.15.42:0/1858033114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c00157a90 con 0x7f5c00158850 2024-05-28T14:01:35.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5bfd600640 1 -- 172.21.15.42:0/1858033114 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bec00e9a0 con 0x7f5c00158850 2024-05-28T14:01:35.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5c07e00640 1 -- 172.21.15.42:0/1858033114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c00158050 con 0x7f5c00158850 2024-05-28T14:01:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.956+0000 7f5bfd600640 1 -- 172.21.15.42:0/1858033114 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec00ed70 con 0x7f5c00158850 2024-05-28T14:01:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.960+0000 7f5c07e00640 1 -- 172.21.15.42:0/1858033114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bdc0052a0 con 0x7f5c00158850 2024-05-28T14:01:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.960+0000 7f5bfd600640 1 -- 172.21.15.42:0/1858033114 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f5bec01d020 con 0x7f5c00158850 2024-05-28T14:01:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.960+0000 7f5bfd600640 1 --2- 172.21.15.42:0/1858033114 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5be0068650 0x7f5be006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.960+0000 7f5bfd600640 1 -- 172.21.15.42:0/1858033114 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5765+0+0 (secure 0 0 0) 0x7f5bec087260 con 0x7f5c00158850 2024-05-28T14:01:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.960+0000 7f5c05400640 1 --2- 172.21.15.42:0/1858033114 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5be0068650 0x7f5be006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:35.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.964+0000 7f5c05400640 1 --2- 172.21.15.42:0/1858033114 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5be0068650 0x7f5be006ab10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4000c10 tx=0x7f5bf4001b00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:35.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:35.972+0000 7f5bfd600640 1 -- 172.21.15.42:0/1858033114 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5bec024070 con 0x7f5c00158850 2024-05-28T14:01:36.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.244+0000 7f5c07e00640 1 -- 172.21.15.42:0/1858033114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5bdc005860 con 0x7f5c00158850 2024-05-28T14:01:36.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.244+0000 7f5bfd600640 1 -- 172.21.15.42:0/1858033114 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f5bec057450 con 0x7f5c00158850 2024-05-28T14:01:36.231 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:01:36.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.248+0000 7f5c07e00640 1 -- 172.21.15.42:0/1858033114 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5be0068650 msgr2=0x7f5be006ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:36.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.248+0000 7f5c07e00640 1 --2- 172.21.15.42:0/1858033114 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5be0068650 0x7f5be006ab10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4000c10 tx=0x7f5bf4001b00 comp rx=0 tx=0).stop 2024-05-28T14:01:36.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.248+0000 7f5c07e00640 1 -- 172.21.15.42:0/1858033114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c00158850 msgr2=0x7f5c00158cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:36.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.248+0000 7f5c07e00640 1 --2- 172.21.15.42:0/1858033114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c00158850 0x7f5c00158cd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5bec010940 tx=0x7f5bec010e00 comp rx=0 tx=0).stop 2024-05-28T14:01:36.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.248+0000 7f5c07e00640 1 -- 172.21.15.42:0/1858033114 shutdown_connections 2024-05-28T14:01:36.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.248+0000 7f5c07e00640 1 --2- 172.21.15.42:0/1858033114 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5be0068650 0x7f5be006ab10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.248+0000 7f5c07e00640 1 --2- 172.21.15.42:0/1858033114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c000b4400 0x7f5c000b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.248+0000 7f5c07e00640 1 --2- 172.21.15.42:0/1858033114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c00158850 0x7f5c00158cd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.248+0000 7f5c07e00640 1 --2- 172.21.15.42:0/1858033114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80051d0 0x7f5c001501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.248+0000 7f5c07e00640 1 -- 172.21.15.42:0/1858033114 >> 172.21.15.42:0/1858033114 conn(0x7f5c000b21f0 msgr2=0x7f5c0015d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:36.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.248+0000 7f5c07e00640 1 -- 172.21.15.42:0/1858033114 shutdown_connections 2024-05-28T14:01:36.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.248+0000 7f5c07e00640 1 -- 172.21.15.42:0/1858033114 wait complete. 2024-05-28T14:01:36.253 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:01:36.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.408+0000 7f90b7a00640 1 Processor -- start 2024-05-28T14:01:36.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.408+0000 7f90b7a00640 1 -- start start 2024-05-28T14:01:36.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.412+0000 7f90b7a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f90b0144f80 0x7f90b0072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:36.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.412+0000 7f90b7a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b015c0c0 0x7f90b006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:36.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.412+0000 7f90b7a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b00afe40 con 0x7f90b006ee20 2024-05-28T14:01:36.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.412+0000 7f90b7a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f90b006f7c0 con 0x7f90b0144f80 2024-05-28T14:01:36.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.412+0000 7f90b7a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90b00724d0 con 0x7f90b015c0c0 2024-05-28T14:01:36.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.412+0000 7f90a7e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f90b006ee20 0x7f90b015b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:49760/0 (socket says 172.21.15.42:49760) 2024-05-28T14:01:36.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.412+0000 7f90a7e00640 1 -- 172.21.15.42:0/2553164126 learned_addr learned my addr 172.21.15.42:0/2553164126 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:36.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.412+0000 7f90b5000640 1 --2- 172.21.15.42:0/2553164126 >> v2:172.21.15.63:3300/0 conn(0x7f90b0144f80 0x7f90b0072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:36.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.412+0000 7f90b5a00640 1 --2- 172.21.15.42:0/2553164126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b015c0c0 0x7f90b006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:36.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.412+0000 7f90afe00640 1 -- 172.21.15.42:0/2553164126 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1668819703 0 0) 0x7f90b00afe40 con 0x7f90b006ee20 2024-05-28T14:01:36.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.412+0000 7f90b5000640 1 -- 172.21.15.42:0/2553164126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b015c0c0 msgr2=0x7f90b006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:36.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.412+0000 7f90b5000640 1 --2- 172.21.15.42:0/2553164126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b015c0c0 0x7f90b006f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.412+0000 7f90b5000640 1 -- 172.21.15.42:0/2553164126 >> v1:172.21.15.42:6789/0 conn(0x7f90b006ee20 legacy=0x7f90b015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:36.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.412+0000 7f90b5000640 1 -- 172.21.15.42:0/2553164126 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b015c600 con 0x7f90b0144f80 2024-05-28T14:01:36.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.416+0000 7f90b5000640 1 --2- 172.21.15.42:0/2553164126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b0144f80 0x7f90b0072960 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f90a00097c0 tx=0x7f90a001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fb3e46a77e45185 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:36.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.416+0000 7f90afe00640 1 -- 172.21.15.42:0/2553164126 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a00b2070 con 0x7f90b0144f80 2024-05-28T14:01:36.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.416+0000 7f90afe00640 1 -- 172.21.15.42:0/2553164126 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a00ad640 con 0x7f90b0144f80 2024-05-28T14:01:36.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.416+0000 7f90afe00640 1 -- 172.21.15.42:0/2553164126 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a00ada10 con 0x7f90b0144f80 2024-05-28T14:01:36.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.416+0000 7f90b7a00640 1 -- 172.21.15.42:0/2553164126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b0144f80 msgr2=0x7f90b0072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:36.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.416+0000 7f90b7a00640 1 --2- 172.21.15.42:0/2553164126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b0144f80 0x7f90b0072960 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f90a00097c0 tx=0x7f90a001c4a0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.416+0000 7f90b7a00640 1 -- 172.21.15.42:0/2553164126 shutdown_connections 2024-05-28T14:01:36.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.416+0000 7f90b7a00640 1 --2- 172.21.15.42:0/2553164126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b015c0c0 0x7f90b006f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.416+0000 7f90b7a00640 1 --2- 172.21.15.42:0/2553164126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b0144f80 0x7f90b0072960 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.416+0000 7f90b7a00640 1 -- 172.21.15.42:0/2553164126 >> 172.21.15.42:0/2553164126 conn(0x7f90b00b21f0 msgr2=0x7f90b00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:36.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.416+0000 7f90b7a00640 1 -- 172.21.15.42:0/2553164126 shutdown_connections 2024-05-28T14:01:36.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.416+0000 7f90b7a00640 1 -- 172.21.15.42:0/2553164126 wait complete. 2024-05-28T14:01:36.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.416+0000 7f90b7a00640 1 Processor -- start 2024-05-28T14:01:36.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.416+0000 7f90b7a00640 1 -- start start 2024-05-28T14:01:36.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90b7a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b006ee20 0x7f90b013c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:36.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90b7a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b0144f80 0x7f90b01f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:36.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90b7a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b015c0c0 0x7f90b00a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:36.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90b7a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0072ea0 con 0x7f90b0144f80 2024-05-28T14:01:36.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90b7a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90b00b4010 con 0x7f90b015c0c0 2024-05-28T14:01:36.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90b7a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90b00aef80 con 0x7f90b006ee20 2024-05-28T14:01:36.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90a7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b0144f80 0x7f90b01f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:36.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90a7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b0144f80 0x7f90b01f73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57840/0 (socket says 172.21.15.42:57840) 2024-05-28T14:01:36.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90a7e00640 1 -- 172.21.15.42:0/3739205310 learned_addr learned my addr 172.21.15.42:0/3739205310 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:36.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90b5000640 1 --2- 172.21.15.42:0/3739205310 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b006ee20 0x7f90b013c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:36.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90b5a00640 1 --2- 172.21.15.42:0/3739205310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b015c0c0 0x7f90b00a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:36.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90a7e00640 1 -- 172.21.15.42:0/3739205310 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b006ee20 msgr2=0x7f90b013c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:36.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90a7e00640 1 --2- 172.21.15.42:0/3739205310 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b006ee20 0x7f90b013c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90a7e00640 1 -- 172.21.15.42:0/3739205310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b015c0c0 msgr2=0x7f90b00a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:36.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90a7e00640 1 --2- 172.21.15.42:0/3739205310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b015c0c0 0x7f90b00a94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90a7e00640 1 -- 172.21.15.42:0/3739205310 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b015e000 con 0x7f90b0144f80 2024-05-28T14:01:36.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90a7e00640 1 --2- 172.21.15.42:0/3739205310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b0144f80 0x7f90b01f73f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f909800c4a0 tx=0x7f909800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:36.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90ad600640 1 -- 172.21.15.42:0/3739205310 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098018070 con 0x7f90b0144f80 2024-05-28T14:01:36.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90ad600640 1 -- 172.21.15.42:0/3739205310 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90980074e0 con 0x7f90b0144f80 2024-05-28T14:01:36.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90ad600640 1 -- 172.21.15.42:0/3739205310 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098004a30 con 0x7f90b0144f80 2024-05-28T14:01:36.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90b7a00640 1 -- 172.21.15.42:0/3739205310 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b015e2e0 con 0x7f90b0144f80 2024-05-28T14:01:36.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90b7a00640 1 -- 172.21.15.42:0/3739205310 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b015e8a0 con 0x7f90b0144f80 2024-05-28T14:01:36.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90b7a00640 1 -- 172.21.15.42:0/3739205310 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b00738b0 con 0x7f90b0144f80 2024-05-28T14:01:36.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.420+0000 7f90ad600640 1 -- 172.21.15.42:0/3739205310 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f909801b070 con 0x7f90b0144f80 2024-05-28T14:01:36.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.424+0000 7f90ad600640 1 --2- 172.21.15.42:0/3739205310 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f908c068650 0x7f908c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:36.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.424+0000 7f90ad600640 1 -- 172.21.15.42:0/3739205310 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5765+0+0 (secure 0 0 0) 0x7f9098085ff0 con 0x7f90b0144f80 2024-05-28T14:01:36.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.424+0000 7f90b5000640 1 --2- 172.21.15.42:0/3739205310 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f908c068650 0x7f908c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:36.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.424+0000 7f90b5000640 1 --2- 172.21.15.42:0/3739205310 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f908c068650 0x7f908c06ab10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90a00af000 tx=0x7f90a00a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:36.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.428+0000 7f90ad600640 1 -- 172.21.15.42:0/3739205310 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9098055c20 con 0x7f90b0144f80 2024-05-28T14:01:36.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.588+0000 7f90b7a00640 1 -- 172.21.15.42:0/3739205310 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f90b004fdd0 con 0x7f90b0144f80 2024-05-28T14:01:36.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.588+0000 7f90ad600640 1 -- 172.21.15.42:0/3739205310 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f9098059f20 con 0x7f90b0144f80 2024-05-28T14:01:36.576 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:01:36.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.592+0000 7f90b7a00640 1 -- 172.21.15.42:0/3739205310 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f908c068650 msgr2=0x7f908c06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:36.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.592+0000 7f90b7a00640 1 --2- 172.21.15.42:0/3739205310 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f908c068650 0x7f908c06ab10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90a00af000 tx=0x7f90a00a7070 comp rx=0 tx=0).stop 2024-05-28T14:01:36.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.592+0000 7f90b7a00640 1 -- 172.21.15.42:0/3739205310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b0144f80 msgr2=0x7f90b01f73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:36.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.592+0000 7f90b7a00640 1 --2- 172.21.15.42:0/3739205310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b0144f80 0x7f90b01f73f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f909800c4a0 tx=0x7f909800c960 comp rx=0 tx=0).stop 2024-05-28T14:01:36.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.592+0000 7f90b7a00640 1 -- 172.21.15.42:0/3739205310 shutdown_connections 2024-05-28T14:01:36.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.592+0000 7f90b7a00640 1 --2- 172.21.15.42:0/3739205310 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f908c068650 0x7f908c06ab10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.592+0000 7f90b7a00640 1 --2- 172.21.15.42:0/3739205310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b015c0c0 0x7f90b00a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.592+0000 7f90b7a00640 1 --2- 172.21.15.42:0/3739205310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b0144f80 0x7f90b01f73f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.592+0000 7f90b7a00640 1 --2- 172.21.15.42:0/3739205310 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b006ee20 0x7f90b013c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.592+0000 7f90b7a00640 1 -- 172.21.15.42:0/3739205310 >> 172.21.15.42:0/3739205310 conn(0x7f90b00b21f0 msgr2=0x7f90b015b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:36.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.592+0000 7f90b7a00640 1 -- 172.21.15.42:0/3739205310 shutdown_connections 2024-05-28T14:01:36.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.592+0000 7f90b7a00640 1 -- 172.21.15.42:0/3739205310 wait complete. 2024-05-28T14:01:36.597 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:01:36.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.756+0000 7f9014200640 1 Processor -- start 2024-05-28T14:01:36.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.756+0000 7f9014200640 1 -- start start 2024-05-28T14:01:36.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.756+0000 7f9014200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f900c071de0 0x7f900c0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:36.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.756+0000 7f9014200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f900c13fd10 0x7f900c1401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:36.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.756+0000 7f9014200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900c072700 con 0x7f900c0ce3b0 2024-05-28T14:01:36.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.756+0000 7f9014200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f900c160230 con 0x7f900c13fd10 2024-05-28T14:01:36.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.756+0000 7f9014200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f900c160510 con 0x7f900c071de0 2024-05-28T14:01:36.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.756+0000 7f9012200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f900c13fd10 0x7f900c1401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:36.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.760+0000 7f9011800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f900c071de0 0x7f900c0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:36.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.760+0000 7f9010e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f900c0ce3b0 0x7f900c0ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:55538/0 (socket says 172.21.15.42:55538) 2024-05-28T14:01:36.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.760+0000 7f9012200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f900c13fd10 0x7f900c1401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:42836/0 (socket says 172.21.15.42:42836) 2024-05-28T14:01:36.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.760+0000 7f9010e00640 1 -- 172.21.15.42:0/2621267280 learned_addr learned my addr 172.21.15.42:0/2621267280 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:36.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.760+0000 7f9012200640 1 -- 172.21.15.42:0/2621267280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f900c071de0 msgr2=0x7f900c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:36.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.760+0000 7f9012200640 1 --2- 172.21.15.42:0/2621267280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f900c071de0 0x7f900c0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.760+0000 7f9012200640 1 -- 172.21.15.42:0/2621267280 >> v1:172.21.15.63:6789/0 conn(0x7f900c0ce3b0 legacy=0x7f900c0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:36.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.760+0000 7f9012200640 1 -- 172.21.15.42:0/2621267280 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c1607f0 con 0x7f900c13fd10 2024-05-28T14:01:36.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.760+0000 7f900be00640 1 -- 172.21.15.42:0/2621267280 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3993459241 0 0) 0x7f900c072700 con 0x7f900c0ce3b0 2024-05-28T14:01:36.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.760+0000 7f9012200640 1 --2- 172.21.15.42:0/2621267280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f900c13fd10 0x7f900c1401a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9004009d90 tx=0x7f900401c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8e5454924be2321 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:36.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.760+0000 7f900be00640 1 -- 172.21.15.42:0/2621267280 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90040b2070 con 0x7f900c13fd10 2024-05-28T14:01:36.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.760+0000 7f900be00640 1 -- 172.21.15.42:0/2621267280 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90040a5710 con 0x7f900c13fd10 2024-05-28T14:01:36.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.760+0000 7f900be00640 1 -- 172.21.15.42:0/2621267280 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90040a5ae0 con 0x7f900c13fd10 2024-05-28T14:01:36.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 -- 172.21.15.42:0/2621267280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f900c13fd10 msgr2=0x7f900c1401a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:36.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 --2- 172.21.15.42:0/2621267280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f900c13fd10 0x7f900c1401a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9004009d90 tx=0x7f900401c850 comp rx=0 tx=0).stop 2024-05-28T14:01:36.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 -- 172.21.15.42:0/2621267280 shutdown_connections 2024-05-28T14:01:36.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 --2- 172.21.15.42:0/2621267280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f900c13fd10 0x7f900c1401a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 --2- 172.21.15.42:0/2621267280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f900c071de0 0x7f900c0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 -- 172.21.15.42:0/2621267280 >> 172.21.15.42:0/2621267280 conn(0x7f900c0b21f0 msgr2=0x7f900c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:36.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 -- 172.21.15.42:0/2621267280 shutdown_connections 2024-05-28T14:01:36.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 -- 172.21.15.42:0/2621267280 wait complete. 2024-05-28T14:01:36.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 Processor -- start 2024-05-28T14:01:36.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 -- start start 2024-05-28T14:01:36.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f900c071de0 0x7f900c0ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:36.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f900c0ce3b0 0x7f900c1fb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:36.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f900c13fd10 0x7f900c1fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:36.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f900c169830 con 0x7f900c0ce3b0 2024-05-28T14:01:36.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f900c1696b0 con 0x7f900c071de0 2024-05-28T14:01:36.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9014200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f900c1699b0 con 0x7f900c13fd10 2024-05-28T14:01:36.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9010e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f900c0ce3b0 0x7f900c1fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:36.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9010e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f900c0ce3b0 0x7f900c1fb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57854/0 (socket says 172.21.15.42:57854) 2024-05-28T14:01:36.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9010e00640 1 -- 172.21.15.42:0/2999782137 learned_addr learned my addr 172.21.15.42:0/2999782137 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:36.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9012200640 1 --2- 172.21.15.42:0/2999782137 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f900c13fd10 0x7f900c1fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:36.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9011800640 1 --2- 172.21.15.42:0/2999782137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f900c071de0 0x7f900c0ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:36.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9010e00640 1 -- 172.21.15.42:0/2999782137 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f900c13fd10 msgr2=0x7f900c1fbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:36.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9010e00640 1 --2- 172.21.15.42:0/2999782137 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f900c13fd10 0x7f900c1fbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9010e00640 1 -- 172.21.15.42:0/2999782137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f900c071de0 msgr2=0x7f900c0ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:36.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9010e00640 1 --2- 172.21.15.42:0/2999782137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f900c071de0 0x7f900c0ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.764+0000 7f9010e00640 1 -- 172.21.15.42:0/2999782137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c1fb510 con 0x7f900c0ce3b0 2024-05-28T14:01:36.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.768+0000 7f9010e00640 1 --2- 172.21.15.42:0/2999782137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f900c0ce3b0 0x7f900c1fb780 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80011f0 tx=0x7f8ff800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:36.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.768+0000 7f9009600640 1 -- 172.21.15.42:0/2999782137 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8018070 con 0x7f900c0ce3b0 2024-05-28T14:01:36.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.768+0000 7f9009600640 1 -- 172.21.15.42:0/2999782137 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff80074e0 con 0x7f900c0ce3b0 2024-05-28T14:01:36.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.768+0000 7f9014200640 1 -- 172.21.15.42:0/2999782137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c0aed40 con 0x7f900c0ce3b0 2024-05-28T14:01:36.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.768+0000 7f9009600640 1 -- 172.21.15.42:0/2999782137 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8004a10 con 0x7f900c0ce3b0 2024-05-28T14:01:36.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.768+0000 7f9014200640 1 -- 172.21.15.42:0/2999782137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c0b3840 con 0x7f900c0ce3b0 2024-05-28T14:01:36.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.768+0000 7f9014200640 1 -- 172.21.15.42:0/2999782137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c0738b0 con 0x7f900c0ce3b0 2024-05-28T14:01:36.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.768+0000 7f9009600640 1 -- 172.21.15.42:0/2999782137 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f8ff801b070 con 0x7f900c0ce3b0 2024-05-28T14:01:36.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.768+0000 7f9009600640 1 --2- 172.21.15.42:0/2999782137 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fec068650 0x7f8fec06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:36.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.768+0000 7f9009600640 1 -- 172.21.15.42:0/2999782137 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5765+0+0 (secure 0 0 0) 0x7f8ff8086020 con 0x7f900c0ce3b0 2024-05-28T14:01:36.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.768+0000 7f9011800640 1 --2- 172.21.15.42:0/2999782137 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fec068650 0x7f8fec06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:36.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.772+0000 7f9011800640 1 --2- 172.21.15.42:0/2999782137 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fec068650 0x7f8fec06ab10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9000009b60 tx=0x7f9000009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:36.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.772+0000 7f9009600640 1 -- 172.21.15.42:0/2999782137 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ff8055c90 con 0x7f900c0ce3b0 2024-05-28T14:01:36.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.936+0000 7f9014200640 1 -- 172.21.15.42:0/2999782137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f900c04fdd0 con 0x7f900c0ce3b0 2024-05-28T14:01:36.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.940+0000 7f9009600640 1 -- 172.21.15.42:0/2999782137 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7f8ff8059f90 con 0x7f900c0ce3b0 2024-05-28T14:01:36.924 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:01:36.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.940+0000 7f9014200640 1 -- 172.21.15.42:0/2999782137 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fec068650 msgr2=0x7f8fec06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:36.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.940+0000 7f9014200640 1 --2- 172.21.15.42:0/2999782137 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fec068650 0x7f8fec06ab10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9000009b60 tx=0x7f9000009190 comp rx=0 tx=0).stop 2024-05-28T14:01:36.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.940+0000 7f9014200640 1 -- 172.21.15.42:0/2999782137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f900c0ce3b0 msgr2=0x7f900c1fb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:36.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.940+0000 7f9014200640 1 --2- 172.21.15.42:0/2999782137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f900c0ce3b0 0x7f900c1fb780 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80011f0 tx=0x7f8ff800c9b0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.940+0000 7f9014200640 1 -- 172.21.15.42:0/2999782137 shutdown_connections 2024-05-28T14:01:36.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.940+0000 7f9014200640 1 --2- 172.21.15.42:0/2999782137 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fec068650 0x7f8fec06ab10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.940+0000 7f9014200640 1 --2- 172.21.15.42:0/2999782137 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f900c13fd10 0x7f900c1fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.940+0000 7f9014200640 1 --2- 172.21.15.42:0/2999782137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f900c0ce3b0 0x7f900c1fb780 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.940+0000 7f9014200640 1 --2- 172.21.15.42:0/2999782137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f900c071de0 0x7f900c0ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:36.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.940+0000 7f9014200640 1 -- 172.21.15.42:0/2999782137 >> 172.21.15.42:0/2999782137 conn(0x7f900c0b21f0 msgr2=0x7f900c15a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:36.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.940+0000 7f9014200640 1 -- 172.21.15.42:0/2999782137 shutdown_connections 2024-05-28T14:01:36.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:36.940+0000 7f9014200640 1 -- 172.21.15.42:0/2999782137 wait complete. 2024-05-28T14:01:36.946 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:01:37.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac35e00640 1 Processor -- start 2024-05-28T14:01:37.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac35e00640 1 -- start start 2024-05-28T14:01:37.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac35e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fac300d0450 0x7fac3009b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac35e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fac3015c880 0x7fac300b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac35e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac3015d680 0x7fac300d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac35e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fac3009b520 con 0x7fac3015c880 2024-05-28T14:01:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac35e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fac300d0d70 con 0x7fac300d0450 2024-05-28T14:01:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac35e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fac300d0160 con 0x7fac3015d680 2024-05-28T14:01:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac2fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fac3015c880 0x7fac300b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac2fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fac3015c880 0x7fac300b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57858/0 (socket says 172.21.15.42:57858) 2024-05-28T14:01:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac2fe00640 1 -- 172.21.15.42:0/1955388867 learned_addr learned my addr 172.21.15.42:0/1955388867 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac35400640 1 --2- 172.21.15.42:0/1955388867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac3015d680 0x7fac300d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac34a00640 1 --2- 172.21.15.42:0/1955388867 >> v2:172.21.15.63:3300/0 conn(0x7fac300d0450 0x7fac3009b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac2fe00640 1 -- 172.21.15.42:0/1955388867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac3015d680 msgr2=0x7fac300d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac2fe00640 1 --2- 172.21.15.42:0/1955388867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac3015d680 0x7fac300d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac2fe00640 1 -- 172.21.15.42:0/1955388867 >> v2:172.21.15.63:3300/0 conn(0x7fac300d0450 msgr2=0x7fac3009b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac2fe00640 1 --2- 172.21.15.42:0/1955388867 >> v2:172.21.15.63:3300/0 conn(0x7fac300d0450 0x7fac3009b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.104+0000 7fac2fe00640 1 -- 172.21.15.42:0/1955388867 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3009abf0 con 0x7fac3015c880 2024-05-28T14:01:37.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac2fe00640 1 --2- 172.21.15.42:0/1955388867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac3015c880 0x7fac300b16e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fac30073020 tx=0x7fac1c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37c70358f679bdb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:37.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac2f400640 1 -- 172.21.15.42:0/1955388867 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0ab5f0 con 0x7fac3015c880 2024-05-28T14:01:37.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac2f400640 1 -- 172.21.15.42:0/1955388867 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac1c0b0070 con 0x7fac3015c880 2024-05-28T14:01:37.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac2f400640 1 -- 172.21.15.42:0/1955388867 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0ab9b0 con 0x7fac3015c880 2024-05-28T14:01:37.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac35e00640 1 -- 172.21.15.42:0/1955388867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac3015c880 msgr2=0x7fac300b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac35e00640 1 --2- 172.21.15.42:0/1955388867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac3015c880 0x7fac300b16e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fac30073020 tx=0x7fac1c01c520 comp rx=0 tx=0).stop 2024-05-28T14:01:37.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac35e00640 1 -- 172.21.15.42:0/1955388867 shutdown_connections 2024-05-28T14:01:37.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac35e00640 1 --2- 172.21.15.42:0/1955388867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac3015d680 0x7fac300d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac35e00640 1 --2- 172.21.15.42:0/1955388867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac3015c880 0x7fac300b16e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac35e00640 1 --2- 172.21.15.42:0/1955388867 >> v2:172.21.15.63:3300/0 conn(0x7fac300d0450 0x7fac3009b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac35e00640 1 -- 172.21.15.42:0/1955388867 >> 172.21.15.42:0/1955388867 conn(0x7fac300b3dd0 msgr2=0x7fac300b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:37.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac35e00640 1 -- 172.21.15.42:0/1955388867 shutdown_connections 2024-05-28T14:01:37.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac35e00640 1 -- 172.21.15.42:0/1955388867 wait complete. 2024-05-28T14:01:37.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac35e00640 1 Processor -- start 2024-05-28T14:01:37.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.108+0000 7fac35e00640 1 -- start start 2024-05-28T14:01:37.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac35e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac300d0450 0x7fac3015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac35e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac3015c880 0x7fac3015f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac35e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac3015d680 0x7fac3014a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac35e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fac30160d60 con 0x7fac3015c880 2024-05-28T14:01:37.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac35e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fac30160be0 con 0x7fac300d0450 2024-05-28T14:01:37.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac35e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fac30160ee0 con 0x7fac3015d680 2024-05-28T14:01:37.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac3015c880 0x7fac3015f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac3015c880 0x7fac3015f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57868/0 (socket says 172.21.15.42:57868) 2024-05-28T14:01:37.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac2fe00640 1 -- 172.21.15.42:0/1142538917 learned_addr learned my addr 172.21.15.42:0/1142538917 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:37.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac34a00640 1 --2- 172.21.15.42:0/1142538917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac300d0450 0x7fac3015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac35400640 1 --2- 172.21.15.42:0/1142538917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac3015d680 0x7fac3014a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac2fe00640 1 -- 172.21.15.42:0/1142538917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac3015d680 msgr2=0x7fac3014a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac2fe00640 1 --2- 172.21.15.42:0/1142538917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac3015d680 0x7fac3014a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac2fe00640 1 -- 172.21.15.42:0/1142538917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac300d0450 msgr2=0x7fac3015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac2fe00640 1 --2- 172.21.15.42:0/1142538917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac300d0450 0x7fac3015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac2fe00640 1 -- 172.21.15.42:0/1142538917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3014b270 con 0x7fac3015c880 2024-05-28T14:01:37.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac2fe00640 1 --2- 172.21.15.42:0/1142538917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac3015c880 0x7fac3015f1d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fac1c01abd0 tx=0x7fac1c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:37.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac2cc00640 1 -- 172.21.15.42:0/1142538917 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0be3f0 con 0x7fac3015c880 2024-05-28T14:01:37.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac2cc00640 1 -- 172.21.15.42:0/1142538917 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac1c0be590 con 0x7fac3015c880 2024-05-28T14:01:37.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac2cc00640 1 -- 172.21.15.42:0/1142538917 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0be980 con 0x7fac3015c880 2024-05-28T14:01:37.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac35e00640 1 -- 172.21.15.42:0/1142538917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac300af0f0 con 0x7fac3015c880 2024-05-28T14:01:37.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac35e00640 1 -- 172.21.15.42:0/1142538917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac300af6b0 con 0x7fac3015c880 2024-05-28T14:01:37.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.112+0000 7fac35e00640 1 -- 172.21.15.42:0/1142538917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac30147ff0 con 0x7fac3015c880 2024-05-28T14:01:37.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.116+0000 7fac2cc00640 1 -- 172.21.15.42:0/1142538917 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7fac1c0b43f0 con 0x7fac3015c880 2024-05-28T14:01:37.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.116+0000 7fac2cc00640 1 --2- 172.21.15.42:0/1142538917 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac10068650 0x7fac1006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.116+0000 7fac2cc00640 1 -- 172.21.15.42:0/1142538917 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5765+0+0 (secure 0 0 0) 0x7fac1c121140 con 0x7fac3015c880 2024-05-28T14:01:37.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.116+0000 7fac34a00640 1 --2- 172.21.15.42:0/1142538917 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac10068650 0x7fac1006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.116+0000 7fac34a00640 1 --2- 172.21.15.42:0/1142538917 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac10068650 0x7fac1006ab10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fac30070ec0 tx=0x7fac24009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:37.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.120+0000 7fac2cc00640 1 -- 172.21.15.42:0/1142538917 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fac30147ff0 con 0x7fac3015c880 2024-05-28T14:01:37.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.316+0000 7fac35e00640 1 -- 172.21.15.42:0/1142538917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac300ce780 con 0x7fac3015c880 2024-05-28T14:01:37.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.316+0000 7fac2cc00640 1 -- 172.21.15.42:0/1142538917 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5187 (secure 0 0 0) 0x7fac1c0c4060 con 0x7fac3015c880 2024-05-28T14:01:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:37.304 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2044753493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2044753493},{"type":"v1","addr":"172.21.15.42:6835","nonce":2044753493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3332210819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3332210819},{"type":"v1","addr":"172.21.15.42:6837","nonce":3332210819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":54782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1050322479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1050322479},{"type":"v1","addr":"172.21.15.63:6834","nonce":1050322479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T13:59:26.845096+0000","modified":"2024-05-28T13:59:29.868153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":54785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54785":{"gid":54785,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/2114729398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2114729398},{"type":"v1","addr":"172.21.15.63:6835","nonce":2114729398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":54785,"qdb_cluster":[54785]},"id":2}]} 2024-05-28T14:01:37.304 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 18 2024-05-28T14:01:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.320+0000 7fac35e00640 1 -- 172.21.15.42:0/1142538917 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac10068650 msgr2=0x7fac1006ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.320+0000 7fac35e00640 1 --2- 172.21.15.42:0/1142538917 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac10068650 0x7fac1006ab10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fac30070ec0 tx=0x7fac24009190 comp rx=0 tx=0).stop 2024-05-28T14:01:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.320+0000 7fac35e00640 1 -- 172.21.15.42:0/1142538917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac3015c880 msgr2=0x7fac3015f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.320+0000 7fac35e00640 1 --2- 172.21.15.42:0/1142538917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac3015c880 0x7fac3015f1d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fac1c01abd0 tx=0x7fac1c01ad60 comp rx=0 tx=0).stop 2024-05-28T14:01:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.320+0000 7fac35e00640 1 -- 172.21.15.42:0/1142538917 shutdown_connections 2024-05-28T14:01:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.320+0000 7fac35e00640 1 --2- 172.21.15.42:0/1142538917 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac10068650 0x7fac1006ab10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.320+0000 7fac35e00640 1 --2- 172.21.15.42:0/1142538917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac3015d680 0x7fac3014a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.320+0000 7fac35e00640 1 --2- 172.21.15.42:0/1142538917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac3015c880 0x7fac3015f1d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.320+0000 7fac35e00640 1 --2- 172.21.15.42:0/1142538917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac300d0450 0x7fac3015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.320+0000 7fac35e00640 1 -- 172.21.15.42:0/1142538917 >> 172.21.15.42:0/1142538917 conn(0x7fac300b3dd0 msgr2=0x7fac300712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.320+0000 7fac35e00640 1 -- 172.21.15.42:0/1142538917 shutdown_connections 2024-05-28T14:01:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.320+0000 7fac35e00640 1 -- 172.21.15.42:0/1142538917 wait complete. 2024-05-28T14:01:37.325 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:01:37.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.480+0000 7f4ae4e00640 1 Processor -- start 2024-05-28T14:01:37.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.480+0000 7f4ae4e00640 1 -- start start 2024-05-28T14:01:37.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.480+0000 7f4ae4e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ae015e990 0x7f4ae009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.480+0000 7f4ae4e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4ae00b1a80 0x7f4ae009af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.480+0000 7f4ae4e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ae00d0330 con 0x7f4ae006d780 2024-05-28T14:01:37.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.480+0000 7f4ae4e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f4ae006d2f0 con 0x7f4ae00b1a80 2024-05-28T14:01:37.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.480+0000 7f4ae4e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae006cca0 con 0x7f4ae015e990 2024-05-28T14:01:37.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.484+0000 7f4ad7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4ae00b1a80 0x7f4ae009af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.484+0000 7f4ad7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4ae00b1a80 0x7f4ae009af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34288/0 (socket says 172.21.15.42:34288) 2024-05-28T14:01:37.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.484+0000 7f4ad7e00640 1 -- 172.21.15.42:0/2540086284 learned_addr learned my addr 172.21.15.42:0/2540086284 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:37.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.484+0000 7f4adde00640 1 --2- 172.21.15.42:0/2540086284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ae015e990 0x7f4ae009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.484+0000 7f4ad7e00640 1 -- 172.21.15.42:0/2540086284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ae015e990 msgr2=0x7f4ae009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:37.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.484+0000 7f4ad7e00640 1 --2- 172.21.15.42:0/2540086284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ae015e990 0x7f4ae009aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.484+0000 7f4ad7e00640 1 -- 172.21.15.42:0/2540086284 >> v1:172.21.15.63:6789/0 conn(0x7f4ae006d780 legacy=0x7f4ae006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:37.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.484+0000 7f4ad7e00640 1 -- 172.21.15.42:0/2540086284 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae006cf80 con 0x7f4ae00b1a80 2024-05-28T14:01:37.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.484+0000 7f4adde00640 1 --2- 172.21.15.42:0/2540086284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ae015e990 0x7f4ae009aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:37.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.484+0000 7f4ad7e00640 1 --2- 172.21.15.42:0/2540086284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ae00b1a80 0x7f4ae009af90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009870 tx=0x7f4ac801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e1451eef821ed server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:37.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.484+0000 7f4add400640 1 -- 172.21.15.42:0/2540086284 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80b0070 con 0x7f4ae00b1a80 2024-05-28T14:01:37.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.484+0000 7f4add400640 1 -- 172.21.15.42:0/2540086284 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac80ab600 con 0x7f4ae00b1a80 2024-05-28T14:01:37.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.484+0000 7f4add400640 1 -- 172.21.15.42:0/2540086284 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80ab9d0 con 0x7f4ae00b1a80 2024-05-28T14:01:37.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2540086284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ae00b1a80 msgr2=0x7f4ae009af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 --2- 172.21.15.42:0/2540086284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ae00b1a80 0x7f4ae009af90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009870 tx=0x7f4ac801c490 comp rx=0 tx=0).stop 2024-05-28T14:01:37.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2540086284 shutdown_connections 2024-05-28T14:01:37.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 --2- 172.21.15.42:0/2540086284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ae00b1a80 0x7f4ae009af90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 --2- 172.21.15.42:0/2540086284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ae015e990 0x7f4ae009aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2540086284 >> 172.21.15.42:0/2540086284 conn(0x7f4ae00b3fa0 msgr2=0x7f4ae00b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:37.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2540086284 shutdown_connections 2024-05-28T14:01:37.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2540086284 wait complete. 2024-05-28T14:01:37.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 Processor -- start 2024-05-28T14:01:37.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 -- start start 2024-05-28T14:01:37.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ae006d780 0x7f4ae014de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ae00b1a80 0x7f4ae015c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ae015e990 0x7f4ae0154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae016b950 con 0x7f4ae00b1a80 2024-05-28T14:01:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae016b7d0 con 0x7f4ae015e990 2024-05-28T14:01:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ae4e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae016bad0 con 0x7f4ae006d780 2024-05-28T14:01:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ad7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ae00b1a80 0x7f4ae015c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ad7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ae00b1a80 0x7f4ae015c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34304/0 (socket says 172.21.15.42:34304) 2024-05-28T14:01:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ad7e00640 1 -- 172.21.15.42:0/2542950804 learned_addr learned my addr 172.21.15.42:0/2542950804 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4adde00640 1 --2- 172.21.15.42:0/2542950804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ae006d780 0x7f4ae014de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ade800640 1 --2- 172.21.15.42:0/2542950804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ae015e990 0x7f4ae0154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ad7e00640 1 -- 172.21.15.42:0/2542950804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ae006d780 msgr2=0x7f4ae014de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ad7e00640 1 --2- 172.21.15.42:0/2542950804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ae006d780 0x7f4ae014de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ad7e00640 1 -- 172.21.15.42:0/2542950804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ae015e990 msgr2=0x7f4ae0154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ad7e00640 1 --2- 172.21.15.42:0/2542950804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ae015e990 0x7f4ae0154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.488+0000 7f4ad7e00640 1 -- 172.21.15.42:0/2542950804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae01561d0 con 0x7f4ae00b1a80 2024-05-28T14:01:37.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.492+0000 7f4ad7e00640 1 --2- 172.21.15.42:0/2542950804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ae00b1a80 0x7f4ae015c3d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4ac801cd30 tx=0x7f4ac8019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:37.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.492+0000 7f4ad6000640 1 -- 172.21.15.42:0/2542950804 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80b0040 con 0x7f4ae00b1a80 2024-05-28T14:01:37.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.492+0000 7f4ad6000640 1 -- 172.21.15.42:0/2542950804 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac80b4640 con 0x7f4ae00b1a80 2024-05-28T14:01:37.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.492+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2542950804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae01564b0 con 0x7f4ae00b1a80 2024-05-28T14:01:37.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.492+0000 7f4ad6000640 1 -- 172.21.15.42:0/2542950804 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80b4a30 con 0x7f4ae00b1a80 2024-05-28T14:01:37.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.492+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2542950804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae0155520 con 0x7f4ae00b1a80 2024-05-28T14:01:37.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.492+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2542950804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae0068a90 con 0x7f4ae00b1a80 2024-05-28T14:01:37.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.492+0000 7f4ad6000640 1 -- 172.21.15.42:0/2542950804 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f4ac80ab4c0 con 0x7f4ae00b1a80 2024-05-28T14:01:37.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.492+0000 7f4ad6000640 1 --2- 172.21.15.42:0/2542950804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4abc068650 0x7f4abc06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.492+0000 7f4ad6000640 1 -- 172.21.15.42:0/2542950804 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5765+0+0 (secure 0 0 0) 0x7f4ac8120ad0 con 0x7f4ae00b1a80 2024-05-28T14:01:37.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.492+0000 7f4adde00640 1 --2- 172.21.15.42:0/2542950804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4abc068650 0x7f4abc06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.496+0000 7f4adde00640 1 --2- 172.21.15.42:0/2542950804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4abc068650 0x7f4abc06ab10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000ab60 tx=0x7f4ad0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:37.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.496+0000 7f4ad6000640 1 -- 172.21.15.42:0/2542950804 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4ac8016110 con 0x7f4ae00b1a80 2024-05-28T14:01:37.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.660+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2542950804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ae00cebf0 con 0x7f4ae00b1a80 2024-05-28T14:01:37.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.664+0000 7f4ad6000640 1 -- 172.21.15.42:0/2542950804 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16717 (secure 0 0 0) 0x7f4ac80f05f0 con 0x7f4ae00b1a80 2024-05-28T14:01:37.649 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:37.649 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":35,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:01:34.969835+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:59:24.673711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-28T13:59:25.795430+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":35,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:01:37.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.664+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2542950804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4abc068650 msgr2=0x7f4abc06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.664+0000 7f4ae4e00640 1 --2- 172.21.15.42:0/2542950804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4abc068650 0x7f4abc06ab10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000ab60 tx=0x7f4ad0008040 comp rx=0 tx=0).stop 2024-05-28T14:01:37.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.664+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2542950804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ae00b1a80 msgr2=0x7f4ae015c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.664+0000 7f4ae4e00640 1 --2- 172.21.15.42:0/2542950804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ae00b1a80 0x7f4ae015c3d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4ac801cd30 tx=0x7f4ac8019230 comp rx=0 tx=0).stop 2024-05-28T14:01:37.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.664+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2542950804 shutdown_connections 2024-05-28T14:01:37.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.664+0000 7f4ae4e00640 1 --2- 172.21.15.42:0/2542950804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4abc068650 0x7f4abc06ab10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.664+0000 7f4ae4e00640 1 --2- 172.21.15.42:0/2542950804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ae015e990 0x7f4ae0154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.664+0000 7f4ae4e00640 1 --2- 172.21.15.42:0/2542950804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ae00b1a80 0x7f4ae015c3d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.664+0000 7f4ae4e00640 1 --2- 172.21.15.42:0/2542950804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ae006d780 0x7f4ae014de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.664+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2542950804 >> 172.21.15.42:0/2542950804 conn(0x7f4ae00b3fa0 msgr2=0x7f4ae0157d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:37.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.664+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2542950804 shutdown_connections 2024-05-28T14:01:37.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.664+0000 7f4ae4e00640 1 -- 172.21.15.42:0/2542950804 wait complete. 2024-05-28T14:01:37.670 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-28T14:01:37.671 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:01:37.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f7200640 1 Processor -- start 2024-05-28T14:01:37.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f7200640 1 -- start start 2024-05-28T14:01:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f7200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99f806ff10 0x7f99f815e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f7200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f80d0660 con 0x7f99f8070370 2024-05-28T14:01:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f7200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f80d0940 con 0x7f99f8070800 2024-05-28T14:01:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f7200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f99f809ba60 con 0x7f99f806ff10 2024-05-28T14:01:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f6800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f99f8070370 0x7f99f815ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51502/0 (socket says 172.21.15.42:51502) 2024-05-28T14:01:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f6800640 1 -- 172.21.15.42:0/4118855431 learned_addr learned my addr 172.21.15.42:0/4118855431 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:37.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99efe00640 1 --2- 172.21.15.42:0/4118855431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99f806ff10 0x7f99f815e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1427446103 0 0) 0x7f99f80d0660 con 0x7f99f8070370 2024-05-28T14:01:37.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99dc003530 con 0x7f99f8070370 2024-05-28T14:01:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2238484715 0 0) 0x7f99f80d0940 con 0x7f99f8070800 2024-05-28T14:01:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99f80d0660 con 0x7f99f8070800 2024-05-28T14:01:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2043993768 0 0) 0x7f99dc003530 con 0x7f99f8070370 2024-05-28T14:01:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99f806ff10 msgr2=0x7f99f815e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 --2- 172.21.15.42:0/4118855431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99f806ff10 0x7f99f815e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 >> v1:172.21.15.63:6789/0 conn(0x7f99f8070800 legacy=0x7f99f8070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f80d0c20 con 0x7f99f8070370 2024-05-28T14:01:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f99f0001a00 con 0x7f99f8070370 2024-05-28T14:01:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 >> v1:172.21.15.42:6789/0 conn(0x7f99f8070370 legacy=0x7f99f815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 --2- 172.21.15.42:0/4118855431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99dc003b80 0x7f99dc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 --2- 172.21.15.42:0/4118855431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99f806ff10 0x7f99dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 --2- 172.21.15.42:0/4118855431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc004d50 0x7f99dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f99f809ba60 con 0x7f99dc004d50 2024-05-28T14:01:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc00ba80 con 0x7f99dc003b80 2024-05-28T14:01:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f99dc00bc80 con 0x7f99f806ff10 2024-05-28T14:01:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f6800640 1 --2- 172.21.15.42:0/4118855431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc004d50 0x7f99dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f5e00640 1 --2- 172.21.15.42:0/4118855431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99dc003b80 0x7f99dc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99efe00640 1 --2- 172.21.15.42:0/4118855431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99f806ff10 0x7f99dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f6800640 1 -- 172.21.15.42:0/4118855431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99f806ff10 msgr2=0x7f99dc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f6800640 1 --2- 172.21.15.42:0/4118855431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99f806ff10 0x7f99dc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f6800640 1 -- 172.21.15.42:0/4118855431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99dc003b80 msgr2=0x7f99dc003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f6800640 1 --2- 172.21.15.42:0/4118855431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99dc003b80 0x7f99dc003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.828+0000 7f99f6800640 1 -- 172.21.15.42:0/4118855431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc003530 con 0x7f99dc004d50 2024-05-28T14:01:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.832+0000 7f99f6800640 1 --2- 172.21.15.42:0/4118855431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc004d50 0x7f99dc0051d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f99f000a800 tx=0x7f99f001fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733f81434f82a679 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.832+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f000a430 con 0x7f99dc004d50 2024-05-28T14:01:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.832+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f00aa690 con 0x7f99dc004d50 2024-05-28T14:01:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.832+0000 7f99f5400640 1 -- 172.21.15.42:0/4118855431 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f00aaa80 con 0x7f99dc004d50 2024-05-28T14:01:37.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.832+0000 7f99f7200640 1 -- 172.21.15.42:0/4118855431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc004d50 msgr2=0x7f99dc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.832+0000 7f99f7200640 1 --2- 172.21.15.42:0/4118855431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc004d50 0x7f99dc0051d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f99f000a800 tx=0x7f99f001fa60 comp rx=0 tx=0).stop 2024-05-28T14:01:37.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.832+0000 7f99f6800640 1 -- 172.21.15.42:0/4118855431 reap_dead start 2024-05-28T14:01:37.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.832+0000 7f99f7200640 1 -- 172.21.15.42:0/4118855431 shutdown_connections 2024-05-28T14:01:37.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.832+0000 7f99f7200640 1 -- 172.21.15.42:0/4118855431 >> 172.21.15.42:0/4118855431 conn(0x7f99f80b3dd0 msgr2=0x7f99f80b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:37.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.832+0000 7f99f7200640 1 -- 172.21.15.42:0/4118855431 shutdown_connections 2024-05-28T14:01:37.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.832+0000 7f99f7200640 1 -- 172.21.15.42:0/4118855431 wait complete. 2024-05-28T14:01:37.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99f7200640 1 Processor -- start 2024-05-28T14:01:37.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99f7200640 1 -- start start 2024-05-28T14:01:37.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99f7200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99dc004d50 0x7f99f814d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99f7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f80b7960 0x7f99f80b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99f7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99f80b2f10 0x7f99f80b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99f7200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f99f80d1700 con 0x7f99f80b7960 2024-05-28T14:01:37.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99f7200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99f80d11e0 con 0x7f99f80b2f10 2024-05-28T14:01:37.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99f7200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f99f80b1890 con 0x7f99dc004d50 2024-05-28T14:01:37.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99efe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f80b7960 0x7f99f80b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99efe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f80b7960 0x7f99f80b6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34324/0 (socket says 172.21.15.42:34324) 2024-05-28T14:01:37.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99efe00640 1 -- 172.21.15.42:0/1559674841 learned_addr learned my addr 172.21.15.42:0/1559674841 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:37.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99efe00640 1 -- 172.21.15.42:0/1559674841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99dc004d50 msgr2=0x7f99f814d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99efe00640 1 --2- 172.21.15.42:0/1559674841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99dc004d50 0x7f99f814d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99efe00640 1 -- 172.21.15.42:0/1559674841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99f80b2f10 msgr2=0x7f99f80b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:37.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99efe00640 1 --2- 172.21.15.42:0/1559674841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99f80b2f10 0x7f99f80b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:37.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99efe00640 1 -- 172.21.15.42:0/1559674841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f80b6a80 con 0x7f99f80b7960 2024-05-28T14:01:37.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99efe00640 1 --2- 172.21.15.42:0/1559674841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f80b7960 0x7f99f80b6180 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f99e0011a00 tx=0x7f99e0011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:37.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99ee000640 1 -- 172.21.15.42:0/1559674841 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e000fa10 con 0x7f99f80b7960 2024-05-28T14:01:37.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99ee000640 1 -- 172.21.15.42:0/1559674841 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e000fbb0 con 0x7f99f80b7960 2024-05-28T14:01:37.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99ee000640 1 -- 172.21.15.42:0/1559674841 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0014440 con 0x7f99f80b7960 2024-05-28T14:01:37.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99f7200640 1 -- 172.21.15.42:0/1559674841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f80b6d60 con 0x7f99f80b7960 2024-05-28T14:01:37.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99f7200640 1 -- 172.21.15.42:0/1559674841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f80b7320 con 0x7f99f80b7960 2024-05-28T14:01:37.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.836+0000 7f99f7200640 1 -- 172.21.15.42:0/1559674841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f804fcf0 con 0x7f99f80b7960 2024-05-28T14:01:37.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.840+0000 7f99ee000640 1 -- 172.21.15.42:0/1559674841 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f99e0012040 con 0x7f99f80b7960 2024-05-28T14:01:37.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.840+0000 7f99ee000640 1 --2- 172.21.15.42:0/1559674841 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99d4068680 0x7f99d406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:37.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.840+0000 7f99ee000640 1 -- 172.21.15.42:0/1559674841 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5765+0+0 (secure 0 0 0) 0x7f99e00873f0 con 0x7f99f80b7960 2024-05-28T14:01:37.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.840+0000 7f99f5e00640 1 --2- 172.21.15.42:0/1559674841 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99d4068680 0x7f99d406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:37.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.840+0000 7f99f5e00640 1 --2- 172.21.15.42:0/1559674841 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99d4068680 0x7f99d406ab40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f99e8007fd0 tx=0x7f99e8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:37.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:37.844+0000 7f99ee000640 1 -- 172.21.15.42:0/1559674841 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f99e0019030 con 0x7f99f80b7960 2024-05-28T14:01:38.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.044+0000 7f99f7200640 1 -- 172.21.15.42:0/1559674841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99f80b4ce0 con 0x7f99f80b7960 2024-05-28T14:01:38.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.044+0000 7f99ee000640 1 -- 172.21.15.42:0/1559674841 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5187 (secure 0 0 0) 0x7f99e001c360 con 0x7f99f80b7960 2024-05-28T14:01:38.031 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:38.031 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2044753493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2044753493},{"type":"v1","addr":"172.21.15.42:6835","nonce":2044753493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3332210819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3332210819},{"type":"v1","addr":"172.21.15.42:6837","nonce":3332210819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":54782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1050322479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1050322479},{"type":"v1","addr":"172.21.15.63:6834","nonce":1050322479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T13:59:26.845096+0000","modified":"2024-05-28T13:59:29.868153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":54785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54785":{"gid":54785,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/2114729398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2114729398},{"type":"v1","addr":"172.21.15.63:6835","nonce":2114729398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":54785,"qdb_cluster":[54785]},"id":2}]} 2024-05-28T14:01:38.032 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 18 2024-05-28T14:01:38.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.048+0000 7f99f7200640 1 -- 172.21.15.42:0/1559674841 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99d4068680 msgr2=0x7f99d406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.048+0000 7f99f7200640 1 --2- 172.21.15.42:0/1559674841 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99d4068680 0x7f99d406ab40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f99e8007fd0 tx=0x7f99e8009040 comp rx=0 tx=0).stop 2024-05-28T14:01:38.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.048+0000 7f99f7200640 1 -- 172.21.15.42:0/1559674841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f80b7960 msgr2=0x7f99f80b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.048+0000 7f99f7200640 1 --2- 172.21.15.42:0/1559674841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f80b7960 0x7f99f80b6180 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f99e0011a00 tx=0x7f99e0011ec0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.048+0000 7f99f7200640 1 -- 172.21.15.42:0/1559674841 shutdown_connections 2024-05-28T14:01:38.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.048+0000 7f99f7200640 1 --2- 172.21.15.42:0/1559674841 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99d4068680 0x7f99d406ab40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.048+0000 7f99f7200640 1 --2- 172.21.15.42:0/1559674841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99f80b2f10 0x7f99f80b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.048+0000 7f99f7200640 1 --2- 172.21.15.42:0/1559674841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99f80b7960 0x7f99f80b6180 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.048+0000 7f99f7200640 1 --2- 172.21.15.42:0/1559674841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99dc004d50 0x7f99f814d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.048+0000 7f99f7200640 1 -- 172.21.15.42:0/1559674841 >> 172.21.15.42:0/1559674841 conn(0x7f99f80b3dd0 msgr2=0x7f99f806dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:38.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.048+0000 7f99f7200640 1 -- 172.21.15.42:0/1559674841 shutdown_connections 2024-05-28T14:01:38.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.048+0000 7f99f7200640 1 -- 172.21.15.42:0/1559674841 wait complete. 2024-05-28T14:01:38.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:01:38.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.208+0000 7fa2b7c00640 1 Processor -- start 2024-05-28T14:01:38.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.208+0000 7fa2b7c00640 1 -- start start 2024-05-28T14:01:38.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.208+0000 7fa2b7c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2b006f660 0x7fa2b0073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.208+0000 7fa2b7c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2b015f280 con 0x7fa2b00d0e00 2024-05-28T14:01:38.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.208+0000 7fa2b7c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2b015f560 con 0x7fa2b015ba40 2024-05-28T14:01:38.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.208+0000 7fa2b7c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b015f030 con 0x7fa2b006f660 2024-05-28T14:01:38.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.208+0000 7fa2b5c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa2b00d0e00 0x7fa2b006dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51514/0 (socket says 172.21.15.42:51514) 2024-05-28T14:01:38.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.208+0000 7fa2b5c00640 1 -- 172.21.15.42:0/1695205321 learned_addr learned my addr 172.21.15.42:0/1695205321 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:38.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2b5200640 1 --2- 172.21.15.42:0/1695205321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2b006f660 0x7fa2b0073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3236620967 0 0) 0x7fa2b015f280 con 0x7fa2b00d0e00 2024-05-28T14:01:38.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa294003530 con 0x7fa2b00d0e00 2024-05-28T14:01:38.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1593284380 0 0) 0x7fa2b015f560 con 0x7fa2b015ba40 2024-05-28T14:01:38.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2b015f280 con 0x7fa2b015ba40 2024-05-28T14:01:38.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2065385218 0 0) 0x7fa294003530 con 0x7fa2b00d0e00 2024-05-28T14:01:38.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2b006f660 msgr2=0x7fa2b0073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:38.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 --2- 172.21.15.42:0/1695205321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2b006f660 0x7fa2b0073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 >> v1:172.21.15.63:6789/0 conn(0x7fa2b015ba40 legacy=0x7fa2b00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b015f840 con 0x7fa2b00d0e00 2024-05-28T14:01:38.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fa2a8001a00 con 0x7fa2b00d0e00 2024-05-28T14:01:38.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2b5200640 1 --2- 172.21.15.42:0/1695205321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2b006f660 0x7fa2b0073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:01:38.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 >> v1:172.21.15.42:6789/0 conn(0x7fa2b00d0e00 legacy=0x7fa2b006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 --2- 172.21.15.42:0/1695205321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa294003b80 0x7fa294003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 --2- 172.21.15.42:0/1695205321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2b006f660 0x7fa294004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 --2- 172.21.15.42:0/1695205321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa294004d50 0x7fa2940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b015f030 con 0x7fa294003b80 2024-05-28T14:01:38.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa29400dd70 con 0x7fa294004d50 2024-05-28T14:01:38.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa29400df70 con 0x7fa2b006f660 2024-05-28T14:01:38.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2b5200640 1 --2- 172.21.15.42:0/1695205321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa294003b80 0x7fa294003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2b5c00640 1 --2- 172.21.15.42:0/1695205321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa294004d50 0x7fa2940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2a7e00640 1 --2- 172.21.15.42:0/1695205321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2b006f660 0x7fa294004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2b5200640 1 -- 172.21.15.42:0/1695205321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2b006f660 msgr2=0x7fa294004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:38.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2b5200640 1 --2- 172.21.15.42:0/1695205321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2b006f660 0x7fa294004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2b5200640 1 -- 172.21.15.42:0/1695205321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa294004d50 msgr2=0x7fa2940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:38.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2b5200640 1 --2- 172.21.15.42:0/1695205321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa294004d50 0x7fa2940051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.212+0000 7fa2b5200640 1 -- 172.21.15.42:0/1695205321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa294003530 con 0x7fa294003b80 2024-05-28T14:01:38.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.216+0000 7fa2b5200640 1 --2- 172.21.15.42:0/1695205321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa294003b80 0x7fa294003f80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009cb0 tx=0x7fa2a001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73c644f3ecd2b29 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:38.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.216+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00b2070 con 0x7fa294003b80 2024-05-28T14:01:38.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.216+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a00a58a0 con 0x7fa294003b80 2024-05-28T14:01:38.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.216+0000 7fa2afe00640 1 -- 172.21.15.42:0/1695205321 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0009880 con 0x7fa294003b80 2024-05-28T14:01:38.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.216+0000 7fa2b7c00640 1 -- 172.21.15.42:0/1695205321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa294003b80 msgr2=0x7fa294003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.216+0000 7fa2b7c00640 1 --2- 172.21.15.42:0/1695205321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa294003b80 0x7fa294003f80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009cb0 tx=0x7fa2a001cbe0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.216+0000 7fa2b5c00640 1 -- 172.21.15.42:0/1695205321 reap_dead start 2024-05-28T14:01:38.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.216+0000 7fa2b7c00640 1 -- 172.21.15.42:0/1695205321 shutdown_connections 2024-05-28T14:01:38.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.216+0000 7fa2b7c00640 1 -- 172.21.15.42:0/1695205321 >> 172.21.15.42:0/1695205321 conn(0x7fa2b006bdf0 msgr2=0x7fa2b006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:38.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.216+0000 7fa2b7c00640 1 -- 172.21.15.42:0/1695205321 shutdown_connections 2024-05-28T14:01:38.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.216+0000 7fa2b7c00640 1 -- 172.21.15.42:0/1695205321 wait complete. 2024-05-28T14:01:38.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.216+0000 7fa2b7c00640 1 Processor -- start 2024-05-28T14:01:38.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.216+0000 7fa2b7c00640 1 -- start start 2024-05-28T14:01:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b7c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa294003b80 0x7fa2b00b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b7c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2b0154910 0x7fa2b01509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b7c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2b00afcf0 0x7fa2b00b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b7c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b00b32e0 con 0x7fa294003b80 2024-05-28T14:01:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b7c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b006ec80 con 0x7fa2b00afcf0 2024-05-28T14:01:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b7c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b00b3460 con 0x7fa2b0154910 2024-05-28T14:01:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa294003b80 0x7fa2b00b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa294003b80 0x7fa2b00b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34354/0 (socket says 172.21.15.42:34354) 2024-05-28T14:01:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b5200640 1 -- 172.21.15.42:0/536289121 learned_addr learned my addr 172.21.15.42:0/536289121 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b5200640 1 -- 172.21.15.42:0/536289121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2b0154910 msgr2=0x7fa2b01509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b5200640 1 --2- 172.21.15.42:0/536289121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2b0154910 0x7fa2b01509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b5200640 1 -- 172.21.15.42:0/536289121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2b00afcf0 msgr2=0x7fa2b00b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b5200640 1 --2- 172.21.15.42:0/536289121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2b00afcf0 0x7fa2b00b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b5200640 1 -- 172.21.15.42:0/536289121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b0150ee0 con 0x7fa294003b80 2024-05-28T14:01:38.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b5200640 1 --2- 172.21.15.42:0/536289121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa294003b80 0x7fa2b00b1690 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa2a001ca10 tx=0x7fa294005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:38.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2ad600640 1 -- 172.21.15.42:0/536289121 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00b2040 con 0x7fa294003b80 2024-05-28T14:01:38.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2ad600640 1 -- 172.21.15.42:0/536289121 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a0009400 con 0x7fa294003b80 2024-05-28T14:01:38.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2ad600640 1 -- 172.21.15.42:0/536289121 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0004020 con 0x7fa294003b80 2024-05-28T14:01:38.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b7c00640 1 -- 172.21.15.42:0/536289121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b00b05f0 con 0x7fa294003b80 2024-05-28T14:01:38.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b7c00640 1 -- 172.21.15.42:0/536289121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b00b0bb0 con 0x7fa294003b80 2024-05-28T14:01:38.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2b7c00640 1 -- 172.21.15.42:0/536289121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b0068a90 con 0x7fa294003b80 2024-05-28T14:01:38.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.220+0000 7fa2ad600640 1 -- 172.21.15.42:0/536289121 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7fa2a00c0030 con 0x7fa294003b80 2024-05-28T14:01:38.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.224+0000 7fa2ad600640 1 --2- 172.21.15.42:0/536289121 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa28c068680 0x7fa28c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.224+0000 7fa2ad600640 1 -- 172.21.15.42:0/536289121 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5765+0+0 (secure 0 0 0) 0x7fa2a0120a10 con 0x7fa294003b80 2024-05-28T14:01:38.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.224+0000 7fa2a7e00640 1 --2- 172.21.15.42:0/536289121 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa28c068680 0x7fa28c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.224+0000 7fa2a7e00640 1 --2- 172.21.15.42:0/536289121 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa28c068680 0x7fa28c06ab40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa2980010f0 tx=0x7fa298009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:38.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.228+0000 7fa2ad600640 1 -- 172.21.15.42:0/536289121 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa2a00c7060 con 0x7fa294003b80 2024-05-28T14:01:38.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.428+0000 7fa2b7c00640 1 -- 172.21.15.42:0/536289121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2b0072bb0 con 0x7fa294003b80 2024-05-28T14:01:38.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.428+0000 7fa2ad600640 1 -- 172.21.15.42:0/536289121 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5187 (secure 0 0 0) 0x7fa2a0016920 con 0x7fa294003b80 2024-05-28T14:01:38.413 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:38.413 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2044753493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2044753493},{"type":"v1","addr":"172.21.15.42:6835","nonce":2044753493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3332210819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3332210819},{"type":"v1","addr":"172.21.15.42:6837","nonce":3332210819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":54782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1050322479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1050322479},{"type":"v1","addr":"172.21.15.63:6834","nonce":1050322479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T13:59:26.845096+0000","modified":"2024-05-28T13:59:29.868153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":54785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54785":{"gid":54785,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/2114729398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2114729398},{"type":"v1","addr":"172.21.15.63:6835","nonce":2114729398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":54785,"qdb_cluster":[54785]},"id":2}]} 2024-05-28T14:01:38.414 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 18 2024-05-28T14:01:38.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.428+0000 7fa2b7c00640 1 -- 172.21.15.42:0/536289121 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa28c068680 msgr2=0x7fa28c06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.428+0000 7fa2b7c00640 1 --2- 172.21.15.42:0/536289121 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa28c068680 0x7fa28c06ab40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa2980010f0 tx=0x7fa298009040 comp rx=0 tx=0).stop 2024-05-28T14:01:38.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.428+0000 7fa2b7c00640 1 -- 172.21.15.42:0/536289121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa294003b80 msgr2=0x7fa2b00b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.428+0000 7fa2b7c00640 1 --2- 172.21.15.42:0/536289121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa294003b80 0x7fa2b00b1690 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa2a001ca10 tx=0x7fa294005840 comp rx=0 tx=0).stop 2024-05-28T14:01:38.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.428+0000 7fa2b7c00640 1 -- 172.21.15.42:0/536289121 shutdown_connections 2024-05-28T14:01:38.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.428+0000 7fa2b7c00640 1 --2- 172.21.15.42:0/536289121 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa28c068680 0x7fa28c06ab40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.428+0000 7fa2b7c00640 1 --2- 172.21.15.42:0/536289121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2b00afcf0 0x7fa2b00b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.428+0000 7fa2b7c00640 1 --2- 172.21.15.42:0/536289121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2b0154910 0x7fa2b01509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.428+0000 7fa2b7c00640 1 --2- 172.21.15.42:0/536289121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa294003b80 0x7fa2b00b1690 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.428+0000 7fa2b7c00640 1 -- 172.21.15.42:0/536289121 >> 172.21.15.42:0/536289121 conn(0x7fa2b006bdf0 msgr2=0x7fa2b01528d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:38.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.428+0000 7fa2b7c00640 1 -- 172.21.15.42:0/536289121 shutdown_connections 2024-05-28T14:01:38.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.428+0000 7fa2b7c00640 1 -- 172.21.15.42:0/536289121 wait complete. 2024-05-28T14:01:38.434 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:01:38.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.588+0000 7efc4d200640 1 Processor -- start 2024-05-28T14:01:38.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.588+0000 7efc4d200640 1 -- start start 2024-05-28T14:01:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc4d200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc480722b0 0x7efc4806fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc4d200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efc480b38a0 0x7efc480b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc4d200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc4815f320 con 0x7efc4815eb60 2024-05-28T14:01:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc4d200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7efc4815f600 con 0x7efc480b38a0 2024-05-28T14:01:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc4d200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efc4815f8e0 con 0x7efc480722b0 2024-05-28T14:01:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc3ea00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7efc4815eb60 0x7efc4806dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51528/0 (socket says 172.21.15.42:51528) 2024-05-28T14:01:38.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc3ea00640 1 -- 172.21.15.42:0/1586393383 learned_addr learned my addr 172.21.15.42:0/1586393383 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:38.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc47e00640 1 --2- 172.21.15.42:0/1586393383 >> v2:172.21.15.42:3300/0 conn(0x7efc480b38a0 0x7efc480b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc47400640 1 --2- 172.21.15.42:0/1586393383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc480722b0 0x7efc4806fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc46a00640 1 -- 172.21.15.42:0/1586393383 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3635355193 0 0) 0x7efc4815f320 con 0x7efc4815eb60 2024-05-28T14:01:38.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc46a00640 1 -- 172.21.15.42:0/1586393383 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc2c003530 con 0x7efc4815eb60 2024-05-28T14:01:38.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc47e00640 1 -- 172.21.15.42:0/1586393383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc480722b0 msgr2=0x7efc4806fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:38.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc47e00640 1 --2- 172.21.15.42:0/1586393383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc480722b0 0x7efc4806fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc47e00640 1 -- 172.21.15.42:0/1586393383 >> v1:172.21.15.42:6789/0 conn(0x7efc4815eb60 legacy=0x7efc4806dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.592+0000 7efc47e00640 1 -- 172.21.15.42:0/1586393383 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc480b3380 con 0x7efc480b38a0 2024-05-28T14:01:38.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.596+0000 7efc47e00640 1 --2- 172.21.15.42:0/1586393383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc480b38a0 0x7efc480b3d10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efc400097c0 tx=0x7efc4001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973101dbc8208439 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:38.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.596+0000 7efc46a00640 1 -- 172.21.15.42:0/1586393383 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc400b0070 con 0x7efc480b38a0 2024-05-28T14:01:38.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.596+0000 7efc46a00640 1 -- 172.21.15.42:0/1586393383 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc400ab600 con 0x7efc480b38a0 2024-05-28T14:01:38.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.596+0000 7efc46a00640 1 -- 172.21.15.42:0/1586393383 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc400ab9b0 con 0x7efc480b38a0 2024-05-28T14:01:38.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.596+0000 7efc4d200640 1 -- 172.21.15.42:0/1586393383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc480b38a0 msgr2=0x7efc480b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.596+0000 7efc4d200640 1 --2- 172.21.15.42:0/1586393383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc480b38a0 0x7efc480b3d10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efc400097c0 tx=0x7efc4001c410 comp rx=0 tx=0).stop 2024-05-28T14:01:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.596+0000 7efc4d200640 1 -- 172.21.15.42:0/1586393383 shutdown_connections 2024-05-28T14:01:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.596+0000 7efc4d200640 1 --2- 172.21.15.42:0/1586393383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc480b38a0 0x7efc480b3d10 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.596+0000 7efc4d200640 1 --2- 172.21.15.42:0/1586393383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc480722b0 0x7efc4806fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.596+0000 7efc4d200640 1 -- 172.21.15.42:0/1586393383 >> 172.21.15.42:0/1586393383 conn(0x7efc4806bbc0 msgr2=0x7efc4806bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.596+0000 7efc4d200640 1 -- 172.21.15.42:0/1586393383 shutdown_connections 2024-05-28T14:01:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.596+0000 7efc4d200640 1 -- 172.21.15.42:0/1586393383 wait complete. 2024-05-28T14:01:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.596+0000 7efc4d200640 1 Processor -- start 2024-05-28T14:01:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.596+0000 7efc4d200640 1 -- start start 2024-05-28T14:01:38.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc4d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc480722b0 0x7efc48156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc4d200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc480b38a0 0x7efc4814e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc4d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc4815eb60 0x7efc48157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc4d200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efc4816b440 con 0x7efc480722b0 2024-05-28T14:01:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc4d200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efc4816b2c0 con 0x7efc4815eb60 2024-05-28T14:01:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc4d200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efc4816b5c0 con 0x7efc480b38a0 2024-05-28T14:01:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc47400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc480722b0 0x7efc48156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc47400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc480722b0 0x7efc48156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34374/0 (socket says 172.21.15.42:34374) 2024-05-28T14:01:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc47400640 1 -- 172.21.15.42:0/4247991541 learned_addr learned my addr 172.21.15.42:0/4247991541 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc47400640 1 -- 172.21.15.42:0/4247991541 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc480b38a0 msgr2=0x7efc4814e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc47400640 1 --2- 172.21.15.42:0/4247991541 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc480b38a0 0x7efc4814e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc47400640 1 -- 172.21.15.42:0/4247991541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc4815eb60 msgr2=0x7efc48157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc47400640 1 --2- 172.21.15.42:0/4247991541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc4815eb60 0x7efc48157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc47400640 1 -- 172.21.15.42:0/4247991541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4814ec40 con 0x7efc480722b0 2024-05-28T14:01:38.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc47400640 1 --2- 172.21.15.42:0/4247991541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc480722b0 0x7efc48156350 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efc38009d10 tx=0x7efc3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:38.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc3fe00640 1 -- 172.21.15.42:0/4247991541 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38018020 con 0x7efc480722b0 2024-05-28T14:01:38.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc3fe00640 1 -- 172.21.15.42:0/4247991541 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc38014480 con 0x7efc480722b0 2024-05-28T14:01:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc3fe00640 1 -- 172.21.15.42:0/4247991541 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38014830 con 0x7efc480722b0 2024-05-28T14:01:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc4d200640 1 -- 172.21.15.42:0/4247991541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc48154b80 con 0x7efc480722b0 2024-05-28T14:01:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc4d200640 1 -- 172.21.15.42:0/4247991541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc48155140 con 0x7efc480722b0 2024-05-28T14:01:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc4d200640 1 -- 172.21.15.42:0/4247991541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4804fcf0 con 0x7efc480722b0 2024-05-28T14:01:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.600+0000 7efc3fe00640 1 -- 172.21.15.42:0/4247991541 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7efc3801b020 con 0x7efc480722b0 2024-05-28T14:01:38.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.604+0000 7efc3fe00640 1 --2- 172.21.15.42:0/4247991541 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efc24068680 0x7efc2406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.604+0000 7efc3fe00640 1 -- 172.21.15.42:0/4247991541 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5765+0+0 (secure 0 0 0) 0x7efc380879e0 con 0x7efc480722b0 2024-05-28T14:01:38.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.604+0000 7efc3ea00640 1 --2- 172.21.15.42:0/4247991541 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efc24068680 0x7efc2406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.604+0000 7efc3ea00640 1 --2- 172.21.15.42:0/4247991541 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efc24068680 0x7efc2406ab40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efc30007fd0 tx=0x7efc30007900 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:38.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.608+0000 7efc3fe00640 1 -- 172.21.15.42:0/4247991541 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efc38022070 con 0x7efc480722b0 2024-05-28T14:01:38.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.772+0000 7efc4d200640 1 -- 172.21.15.42:0/4247991541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc4806f3a0 con 0x7efc480722b0 2024-05-28T14:01:38.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.772+0000 7efc3fe00640 1 -- 172.21.15.42:0/4247991541 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16717 (secure 0 0 0) 0x7efc38010040 con 0x7efc480722b0 2024-05-28T14:01:38.758 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:38.758 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":35,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:01:34.969835+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-28T13:59:24.673711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-28T13:59:25.795430+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":35,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:01:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.776+0000 7efc4d200640 1 -- 172.21.15.42:0/4247991541 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efc24068680 msgr2=0x7efc2406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.776+0000 7efc4d200640 1 --2- 172.21.15.42:0/4247991541 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efc24068680 0x7efc2406ab40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efc30007fd0 tx=0x7efc30007900 comp rx=0 tx=0).stop 2024-05-28T14:01:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.776+0000 7efc4d200640 1 -- 172.21.15.42:0/4247991541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc480722b0 msgr2=0x7efc48156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.776+0000 7efc4d200640 1 --2- 172.21.15.42:0/4247991541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc480722b0 0x7efc48156350 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efc38009d10 tx=0x7efc3800c490 comp rx=0 tx=0).stop 2024-05-28T14:01:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.776+0000 7efc4d200640 1 -- 172.21.15.42:0/4247991541 shutdown_connections 2024-05-28T14:01:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.776+0000 7efc4d200640 1 --2- 172.21.15.42:0/4247991541 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efc24068680 0x7efc2406ab40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.776+0000 7efc4d200640 1 --2- 172.21.15.42:0/4247991541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc4815eb60 0x7efc48157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.776+0000 7efc4d200640 1 --2- 172.21.15.42:0/4247991541 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc480b38a0 0x7efc4814e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.776+0000 7efc4d200640 1 --2- 172.21.15.42:0/4247991541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc480722b0 0x7efc48156350 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.776+0000 7efc4d200640 1 -- 172.21.15.42:0/4247991541 >> 172.21.15.42:0/4247991541 conn(0x7efc4806bbc0 msgr2=0x7efc4806e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.776+0000 7efc4d200640 1 -- 172.21.15.42:0/4247991541 shutdown_connections 2024-05-28T14:01:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.776+0000 7efc4d200640 1 -- 172.21.15.42:0/4247991541 wait complete. 2024-05-28T14:01:38.781 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-28T14:01:38.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.936+0000 7f896ea00640 1 Processor -- start 2024-05-28T14:01:38.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.936+0000 7f896ea00640 1 -- start start 2024-05-28T14:01:38.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.936+0000 7f896ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f896815bf30 0x7f896815c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.936+0000 7f896ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f89680cf230 0x7f8968072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.936+0000 7f896ea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8968072ba0 0x7f896806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.936+0000 7f896ea00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f89680721d0 con 0x7f89680cf230 2024-05-28T14:01:38.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.936+0000 7f896ea00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f89680cfc10 con 0x7f8968072ba0 2024-05-28T14:01:38.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.936+0000 7f896ea00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f89680cfef0 con 0x7f896815bf30 2024-05-28T14:01:38.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f895fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f89680cf230 0x7f8968072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f895fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f89680cf230 0x7f8968072660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34382/0 (socket says 172.21.15.42:34382) 2024-05-28T14:01:38.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f895fe00640 1 -- 172.21.15.42:0/1080383025 learned_addr learned my addr 172.21.15.42:0/1080383025 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:38.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f896ca00640 1 --2- 172.21.15.42:0/1080383025 >> v2:172.21.15.63:3300/0 conn(0x7f8968072ba0 0x7f896806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f8967e00640 1 --2- 172.21.15.42:0/1080383025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f896815bf30 0x7f896815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f895fe00640 1 -- 172.21.15.42:0/1080383025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f896815bf30 msgr2=0x7f896815c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:38.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f895fe00640 1 --2- 172.21.15.42:0/1080383025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f896815bf30 0x7f896815c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f895fe00640 1 -- 172.21.15.42:0/1080383025 >> v2:172.21.15.63:3300/0 conn(0x7f8968072ba0 msgr2=0x7f896806ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:38.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f895fe00640 1 --2- 172.21.15.42:0/1080383025 >> v2:172.21.15.63:3300/0 conn(0x7f8968072ba0 0x7f896806ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f895fe00640 1 -- 172.21.15.42:0/1080383025 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89680d01d0 con 0x7f89680cf230 2024-05-28T14:01:38.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f8967e00640 1 --2- 172.21.15.42:0/1080383025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f896815bf30 0x7f896815c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:38.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f895fe00640 1 --2- 172.21.15.42:0/1080383025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89680cf230 0x7f8968072660 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8968071d50 tx=0x7f895001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f0ec560898255f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:38.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f8967400640 1 -- 172.21.15.42:0/1080383025 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89500b1070 con 0x7f89680cf230 2024-05-28T14:01:38.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f8967400640 1 -- 172.21.15.42:0/1080383025 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89500ac630 con 0x7f89680cf230 2024-05-28T14:01:38.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.940+0000 7f8967400640 1 -- 172.21.15.42:0/1080383025 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89500a8050 con 0x7f89680cf230 2024-05-28T14:01:38.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 -- 172.21.15.42:0/1080383025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89680cf230 msgr2=0x7f8968072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 --2- 172.21.15.42:0/1080383025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89680cf230 0x7f8968072660 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8968071d50 tx=0x7f895001c4f0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 -- 172.21.15.42:0/1080383025 shutdown_connections 2024-05-28T14:01:38.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 --2- 172.21.15.42:0/1080383025 >> v2:172.21.15.63:3300/0 conn(0x7f8968072ba0 0x7f896806ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 --2- 172.21.15.42:0/1080383025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89680cf230 0x7f8968072660 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 --2- 172.21.15.42:0/1080383025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f896815bf30 0x7f896815c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 -- 172.21.15.42:0/1080383025 >> 172.21.15.42:0/1080383025 conn(0x7f896806b9f0 msgr2=0x7f896806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:38.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 -- 172.21.15.42:0/1080383025 shutdown_connections 2024-05-28T14:01:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 -- 172.21.15.42:0/1080383025 wait complete. 2024-05-28T14:01:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 Processor -- start 2024-05-28T14:01:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 -- start start 2024-05-28T14:01:38.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8968072ba0 0x7f896814ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89680cf230 0x7f896815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f896815bf30 0x7f896815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f896815f020 con 0x7f8968072ba0 2024-05-28T14:01:38.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f896806f5a0 con 0x7f896815bf30 2024-05-28T14:01:38.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ea00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f896815f1a0 con 0x7f89680cf230 2024-05-28T14:01:38.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f8967e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8968072ba0 0x7f896814ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f8967e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8968072ba0 0x7f896814ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34384/0 (socket says 172.21.15.42:34384) 2024-05-28T14:01:38.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f8967e00640 1 -- 172.21.15.42:0/832173750 learned_addr learned my addr 172.21.15.42:0/832173750 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:38.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ca00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f896815bf30 0x7f896815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f895fe00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89680cf230 0x7f896815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f8967e00640 1 -- 172.21.15.42:0/832173750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89680cf230 msgr2=0x7f896815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f8967e00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89680cf230 0x7f896815a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f8967e00640 1 -- 172.21.15.42:0/832173750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f896815bf30 msgr2=0x7f896815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:38.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f8967e00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f896815bf30 0x7f896815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:38.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f8967e00640 1 -- 172.21.15.42:0/832173750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89681fbcd0 con 0x7f8968072ba0 2024-05-28T14:01:38.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.944+0000 7f896ca00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f896815bf30 0x7f896815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:38.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.948+0000 7f8967e00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8968072ba0 0x7f896814ada0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f895800c820 tx=0x7f895800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:38.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.948+0000 7f8964c00640 1 -- 172.21.15.42:0/832173750 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958015070 con 0x7f8968072ba0 2024-05-28T14:01:38.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.948+0000 7f8964c00640 1 -- 172.21.15.42:0/832173750 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8958007a30 con 0x7f8968072ba0 2024-05-28T14:01:38.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.948+0000 7f8964c00640 1 -- 172.21.15.42:0/832173750 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958007e00 con 0x7f8968072ba0 2024-05-28T14:01:38.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.948+0000 7f896ea00640 1 -- 172.21.15.42:0/832173750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8968152b80 con 0x7f8968072ba0 2024-05-28T14:01:38.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.948+0000 7f896ea00640 1 -- 172.21.15.42:0/832173750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89681532b0 con 0x7f8968072ba0 2024-05-28T14:01:38.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.948+0000 7f896ea00640 1 -- 172.21.15.42:0/832173750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8968073890 con 0x7f8968072ba0 2024-05-28T14:01:38.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.948+0000 7f8964c00640 1 -- 172.21.15.42:0/832173750 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f8958018070 con 0x7f8968072ba0 2024-05-28T14:01:38.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.948+0000 7f8964c00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8944068650 0x7f894406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:38.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.948+0000 7f8964c00640 1 -- 172.21.15.42:0/832173750 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5765+0+0 (secure 0 0 0) 0x7f8958086d90 con 0x7f8968072ba0 2024-05-28T14:01:38.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.948+0000 7f895fe00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8944068650 0x7f894406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:38.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.952+0000 7f895fe00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8944068650 0x7f894406ab10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f89500ae000 tx=0x7f8950008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:38.952+0000 7f8964c00640 1 -- 172.21.15.42:0/832173750 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8958056f00 con 0x7f8968072ba0 2024-05-28T14:01:39.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.152+0000 7f896ea00640 1 -- 172.21.15.42:0/832173750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f896815c630 con 0x7f8968072ba0 2024-05-28T14:01:39.140 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-05-28T14:01:39.156+0000 7f248b600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-28T14:01:39.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.156+0000 7f8964c00640 1 -- 172.21.15.42:0/832173750 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f895805d0d0 con 0x7f8968072ba0 2024-05-28T14:01:39.142 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-28T14:01:39.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.160+0000 7f896ea00640 1 -- 172.21.15.42:0/832173750 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8944068650 msgr2=0x7f894406ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:39.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.160+0000 7f896ea00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8944068650 0x7f894406ab10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f89500ae000 tx=0x7f8950008040 comp rx=0 tx=0).stop 2024-05-28T14:01:39.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.160+0000 7f896ea00640 1 -- 172.21.15.42:0/832173750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8968072ba0 msgr2=0x7f896814ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:39.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.160+0000 7f896ea00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8968072ba0 0x7f896814ada0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f895800c820 tx=0x7f895800cce0 comp rx=0 tx=0).stop 2024-05-28T14:01:39.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.160+0000 7f896ea00640 1 -- 172.21.15.42:0/832173750 shutdown_connections 2024-05-28T14:01:39.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.160+0000 7f896ea00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8944068650 0x7f894406ab10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:39.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.160+0000 7f896ea00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f896815bf30 0x7f896815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:39.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.160+0000 7f896ea00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89680cf230 0x7f896815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:39.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.160+0000 7f896ea00640 1 --2- 172.21.15.42:0/832173750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8968072ba0 0x7f896814ada0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:39.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.160+0000 7f896ea00640 1 -- 172.21.15.42:0/832173750 >> 172.21.15.42:0/832173750 conn(0x7f896806b9f0 msgr2=0x7f896806bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:39.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.160+0000 7f896ea00640 1 -- 172.21.15.42:0/832173750 shutdown_connections 2024-05-28T14:01:39.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.160+0000 7f896ea00640 1 -- 172.21.15.42:0/832173750 wait complete. 2024-05-28T14:01:39.166 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-28T14:01:39.219 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-28T14:01:39.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc643400640 1 Processor -- start 2024-05-28T14:01:39.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc643400640 1 -- start start 2024-05-28T14:01:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc643400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c15c0c0 0x7fc63c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc643400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc63c0afe40 con 0x7fc63c144f80 2024-05-28T14:01:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc643400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc63c15c600 con 0x7fc63c06ee20 2024-05-28T14:01:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc643400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc63c06f7c0 con 0x7fc63c15c0c0 2024-05-28T14:01:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc640a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc63c144f80 0x7fc63c072960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51540/0 (socket says 172.21.15.42:51540) 2024-05-28T14:01:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc640a00640 1 -- 172.21.15.42:0/2374965179 learned_addr learned my addr 172.21.15.42:0/2374965179 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc641400640 1 --2- 172.21.15.42:0/2374965179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c15c0c0 0x7fc63c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc63b400640 1 -- 172.21.15.42:0/2374965179 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1284019171 0 0) 0x7fc63c0afe40 con 0x7fc63c144f80 2024-05-28T14:01:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc63b400640 1 -- 172.21.15.42:0/2374965179 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc624003530 con 0x7fc63c144f80 2024-05-28T14:01:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc63b400640 1 -- 172.21.15.42:0/2374965179 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 392732306 0 0) 0x7fc63c15c600 con 0x7fc63c06ee20 2024-05-28T14:01:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc63b400640 1 -- 172.21.15.42:0/2374965179 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc63c0afe40 con 0x7fc63c06ee20 2024-05-28T14:01:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc641400640 1 -- 172.21.15.42:0/2374965179 >> v1:172.21.15.63:6789/0 conn(0x7fc63c06ee20 legacy=0x7fc63c15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc641400640 1 -- 172.21.15.42:0/2374965179 >> v1:172.21.15.42:6789/0 conn(0x7fc63c144f80 legacy=0x7fc63c072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.324+0000 7fc641400640 1 -- 172.21.15.42:0/2374965179 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c15c8e0 con 0x7fc63c15c0c0 2024-05-28T14:01:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.328+0000 7fc641400640 1 --2- 172.21.15.42:0/2374965179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c15c0c0 0x7fc63c06f280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc634009dc0 tx=0x7fc63401eb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10f4e6fa38fd054f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.328+0000 7fc63b400640 1 -- 172.21.15.42:0/2374965179 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6340b4070 con 0x7fc63c15c0c0 2024-05-28T14:01:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.328+0000 7fc63b400640 1 -- 172.21.15.42:0/2374965179 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6340a7a30 con 0x7fc63c15c0c0 2024-05-28T14:01:39.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.328+0000 7fc63b400640 1 -- 172.21.15.42:0/2374965179 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6340a7e00 con 0x7fc63c15c0c0 2024-05-28T14:01:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.328+0000 7fc643400640 1 -- 172.21.15.42:0/2374965179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c15c0c0 msgr2=0x7fc63c06f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.328+0000 7fc643400640 1 --2- 172.21.15.42:0/2374965179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c15c0c0 0x7fc63c06f280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc634009dc0 tx=0x7fc63401eb60 comp rx=0 tx=0).stop 2024-05-28T14:01:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.328+0000 7fc643400640 1 -- 172.21.15.42:0/2374965179 shutdown_connections 2024-05-28T14:01:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.328+0000 7fc643400640 1 --2- 172.21.15.42:0/2374965179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c15c0c0 0x7fc63c06f280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.328+0000 7fc643400640 1 -- 172.21.15.42:0/2374965179 >> 172.21.15.42:0/2374965179 conn(0x7fc63c0b21f0 msgr2=0x7fc63c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.328+0000 7fc643400640 1 -- 172.21.15.42:0/2374965179 shutdown_connections 2024-05-28T14:01:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.328+0000 7fc643400640 1 -- 172.21.15.42:0/2374965179 wait complete. 2024-05-28T14:01:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc643400640 1 Processor -- start 2024-05-28T14:01:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc643400640 1 -- start start 2024-05-28T14:01:39.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc643400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c06ee20 0x7fc63c1f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:39.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc643400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc63c144f80 0x7fc63c0ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:39.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc643400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc63c15c0c0 0x7fc63c0ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:39.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc643400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c0724d0 con 0x7fc63c15c0c0 2024-05-28T14:01:39.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc643400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c0b4260 con 0x7fc63c144f80 2024-05-28T14:01:39.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc643400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc63c06dd10 con 0x7fc63c06ee20 2024-05-28T14:01:39.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc641400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc63c15c0c0 0x7fc63c0ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:39.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc640a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c06ee20 0x7fc63c1f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:39.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc641400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc63c15c0c0 0x7fc63c0ae990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34394/0 (socket says 172.21.15.42:34394) 2024-05-28T14:01:39.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc641400640 1 -- 172.21.15.42:0/2128247581 learned_addr learned my addr 172.21.15.42:0/2128247581 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:39.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc63be00640 1 --2- 172.21.15.42:0/2128247581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc63c144f80 0x7fc63c0ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:39.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc641400640 1 -- 172.21.15.42:0/2128247581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c06ee20 msgr2=0x7fc63c1f74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:39.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc641400640 1 --2- 172.21.15.42:0/2128247581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c06ee20 0x7fc63c1f74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:39.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc641400640 1 -- 172.21.15.42:0/2128247581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc63c144f80 msgr2=0x7fc63c0ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:39.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc641400640 1 --2- 172.21.15.42:0/2128247581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc63c144f80 0x7fc63c0ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:39.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc641400640 1 -- 172.21.15.42:0/2128247581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c1f79f0 con 0x7fc63c15c0c0 2024-05-28T14:01:39.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc641400640 1 --2- 172.21.15.42:0/2128247581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc63c15c0c0 0x7fc63c0ae990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc63401d210 tx=0x7fc63401b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc638c00640 1 -- 172.21.15.42:0/2128247581 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6340b4040 con 0x7fc63c15c0c0 2024-05-28T14:01:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc638c00640 1 -- 172.21.15.42:0/2128247581 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6340bd070 con 0x7fc63c15c0c0 2024-05-28T14:01:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc638c00640 1 -- 172.21.15.42:0/2128247581 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6340af680 con 0x7fc63c15c0c0 2024-05-28T14:01:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc643400640 1 -- 172.21.15.42:0/2128247581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63c0af210 con 0x7fc63c15c0c0 2024-05-28T14:01:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.332+0000 7fc643400640 1 -- 172.21.15.42:0/2128247581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63c0ab960 con 0x7fc63c15c0c0 2024-05-28T14:01:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.336+0000 7fc643400640 1 -- 172.21.15.42:0/2128247581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc63c0738b0 con 0x7fc63c15c0c0 2024-05-28T14:01:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.336+0000 7fc638c00640 1 -- 172.21.15.42:0/2128247581 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7fc6340af820 con 0x7fc63c15c0c0 2024-05-28T14:01:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.336+0000 7fc638c00640 1 --2- 172.21.15.42:0/2128247581 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc61c068650 0x7fc61c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.336+0000 7fc638c00640 1 -- 172.21.15.42:0/2128247581 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5823+0+0 (secure 0 0 0) 0x7fc6341210e0 con 0x7fc63c15c0c0 2024-05-28T14:01:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.336+0000 7fc640a00640 1 --2- 172.21.15.42:0/2128247581 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc61c068650 0x7fc61c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.336+0000 7fc640a00640 1 --2- 172.21.15.42:0/2128247581 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc61c068650 0x7fc61c06ab10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc6300011f0 tx=0x7fc630007d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.340+0000 7fc638c00640 1 -- 172.21.15.42:0/2128247581 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc634018750 con 0x7fc63c15c0c0 2024-05-28T14:01:39.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:39.536+0000 7fc643400640 1 -- 172.21.15.42:0/2128247581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc63c0b1ad0 con 0x7fc63c15c0c0 2024-05-28T14:01:40.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.160+0000 7fc638c00640 1 -- 172.21.15.42:0/2128247581 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7fc6340f0e00 con 0x7fc63c15c0c0 2024-05-28T14:01:40.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.160+0000 7fc643400640 1 -- 172.21.15.42:0/2128247581 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc61c068650 msgr2=0x7fc61c06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:40.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.160+0000 7fc643400640 1 --2- 172.21.15.42:0/2128247581 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc61c068650 0x7fc61c06ab10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc6300011f0 tx=0x7fc630007d40 comp rx=0 tx=0).stop 2024-05-28T14:01:40.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.160+0000 7fc643400640 1 -- 172.21.15.42:0/2128247581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc63c15c0c0 msgr2=0x7fc63c0ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:40.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.160+0000 7fc643400640 1 --2- 172.21.15.42:0/2128247581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc63c15c0c0 0x7fc63c0ae990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc63401d210 tx=0x7fc63401b900 comp rx=0 tx=0).stop 2024-05-28T14:01:40.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.160+0000 7fc643400640 1 -- 172.21.15.42:0/2128247581 shutdown_connections 2024-05-28T14:01:40.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.160+0000 7fc643400640 1 --2- 172.21.15.42:0/2128247581 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc61c068650 0x7fc61c06ab10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:40.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.160+0000 7fc643400640 1 --2- 172.21.15.42:0/2128247581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc63c15c0c0 0x7fc63c0ae990 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:40.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.160+0000 7fc643400640 1 --2- 172.21.15.42:0/2128247581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc63c144f80 0x7fc63c0ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:40.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.160+0000 7fc643400640 1 --2- 172.21.15.42:0/2128247581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c06ee20 0x7fc63c1f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:40.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.160+0000 7fc643400640 1 -- 172.21.15.42:0/2128247581 >> 172.21.15.42:0/2128247581 conn(0x7fc63c0b21f0 msgr2=0x7fc63c0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:40.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.160+0000 7fc643400640 1 -- 172.21.15.42:0/2128247581 shutdown_connections 2024-05-28T14:01:40.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.160+0000 7fc643400640 1 -- 172.21.15.42:0/2128247581 wait complete. 2024-05-28T14:01:40.170 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-28T14:01:40.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f6172600640 1 Processor -- start 2024-05-28T14:01:40.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f6172600640 1 -- start start 2024-05-28T14:01:40.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f6172600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f616c15b350 0x7f616c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:40.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f6172600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f616c0d0e00 0x7f616c0aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:40.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f6172600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f616c0ade40 con 0x7f616c072d40 2024-05-28T14:01:40.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f6172600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f616c0ae9f0 con 0x7f616c15b350 2024-05-28T14:01:40.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f6172600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f616c0ae0c0 con 0x7f616c0d0e00 2024-05-28T14:01:40.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f6162a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f616c072d40 0x7f616c0d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51554/0 (socket says 172.21.15.42:51554) 2024-05-28T14:01:40.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f6162a00640 1 -- 172.21.15.42:0/1953513552 learned_addr learned my addr 172.21.15.42:0/1953513552 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:40.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f616b400640 1 --2- 172.21.15.42:0/1953513552 >> v2:172.21.15.42:3300/0 conn(0x7f616c15b350 0x7f616c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:40.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f616be00640 1 --2- 172.21.15.42:0/1953513552 >> v2:172.21.15.63:3300/0 conn(0x7f616c0d0e00 0x7f616c0aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:40.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f616aa00640 1 -- 172.21.15.42:0/1953513552 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2646481654 0 0) 0x7f6154001120 con 0x7f616c072d40 2024-05-28T14:01:40.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f616aa00640 1 -- 172.21.15.42:0/1953513552 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6150003530 con 0x7f616c072d40 2024-05-28T14:01:40.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f616b400640 1 -- 172.21.15.42:0/1953513552 >> v2:172.21.15.63:3300/0 conn(0x7f616c0d0e00 msgr2=0x7f616c0aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:40.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f616b400640 1 --2- 172.21.15.42:0/1953513552 >> v2:172.21.15.63:3300/0 conn(0x7f616c0d0e00 0x7f616c0aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:40.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f616b400640 1 -- 172.21.15.42:0/1953513552 >> v1:172.21.15.42:6789/0 conn(0x7f616c072d40 legacy=0x7f616c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:40.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f616b400640 1 -- 172.21.15.42:0/1953513552 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616c0ae3a0 con 0x7f616c15b350 2024-05-28T14:01:40.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.332+0000 7f616be00640 1 --2- 172.21.15.42:0/1953513552 >> v2:172.21.15.63:3300/0 conn(0x7f616c0d0e00 0x7f616c0aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:40.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.336+0000 7f616b400640 1 --2- 172.21.15.42:0/1953513552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f616c15b350 0x7f616c06dd40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f616c0ad980 tx=0x7f615c01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e5f473a29267d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:40.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.336+0000 7f616aa00640 1 -- 172.21.15.42:0/1953513552 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c0b2070 con 0x7f616c15b350 2024-05-28T14:01:40.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.336+0000 7f616aa00640 1 -- 172.21.15.42:0/1953513552 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f615c0a5750 con 0x7f616c15b350 2024-05-28T14:01:40.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.336+0000 7f616aa00640 1 -- 172.21.15.42:0/1953513552 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c0a5b20 con 0x7f616c15b350 2024-05-28T14:01:40.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.336+0000 7f6172600640 1 -- 172.21.15.42:0/1953513552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f616c15b350 msgr2=0x7f616c06dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:40.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.336+0000 7f6172600640 1 --2- 172.21.15.42:0/1953513552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f616c15b350 0x7f616c06dd40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f616c0ad980 tx=0x7f615c01c810 comp rx=0 tx=0).stop 2024-05-28T14:01:40.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.336+0000 7f6172600640 1 -- 172.21.15.42:0/1953513552 shutdown_connections 2024-05-28T14:01:40.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.336+0000 7f6172600640 1 --2- 172.21.15.42:0/1953513552 >> v2:172.21.15.63:3300/0 conn(0x7f616c0d0e00 0x7f616c0aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:40.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.336+0000 7f6172600640 1 --2- 172.21.15.42:0/1953513552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f616c15b350 0x7f616c06dd40 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:40.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.336+0000 7f6172600640 1 -- 172.21.15.42:0/1953513552 >> 172.21.15.42:0/1953513552 conn(0x7f616c06bdf0 msgr2=0x7f616c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:40.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.336+0000 7f6172600640 1 -- 172.21.15.42:0/1953513552 shutdown_connections 2024-05-28T14:01:40.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.336+0000 7f6172600640 1 -- 172.21.15.42:0/1953513552 wait complete. 2024-05-28T14:01:40.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6172600640 1 Processor -- start 2024-05-28T14:01:40.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6172600640 1 -- start start 2024-05-28T14:01:40.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6172600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f616c072d40 0x7f616c153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6172600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f616c0d0e00 0x7f616c154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6172600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f616c15b350 0x7f616c1549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6172600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f616c168470 con 0x7f616c072d40 2024-05-28T14:01:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6172600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f616c1682f0 con 0x7f616c15b350 2024-05-28T14:01:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6172600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f616c1685f0 con 0x7f616c0d0e00 2024-05-28T14:01:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f616b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f616c072d40 0x7f616c153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f616b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f616c072d40 0x7f616c153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34422/0 (socket says 172.21.15.42:34422) 2024-05-28T14:01:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f616b400640 1 -- 172.21.15.42:0/1201871987 learned_addr learned my addr 172.21.15.42:0/1201871987 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6162a00640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f616c0d0e00 0x7f616c154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f616be00640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f616c15b350 0x7f616c1549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f616b400640 1 -- 172.21.15.42:0/1201871987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f616c0d0e00 msgr2=0x7f616c154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:40.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f616b400640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f616c0d0e00 0x7f616c154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:40.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f616b400640 1 -- 172.21.15.42:0/1201871987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f616c15b350 msgr2=0x7f616c1549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:40.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f616b400640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f616c15b350 0x7f616c1549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:40.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f616b400640 1 -- 172.21.15.42:0/1201871987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616c0b2980 con 0x7f616c072d40 2024-05-28T14:01:40.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6162a00640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f616c0d0e00 0x7f616c154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:40.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f616b400640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f616c072d40 0x7f616c153d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f615c0b0040 tx=0x7f615c0195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:40.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6163e00640 1 -- 172.21.15.42:0/1201871987 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c0b2050 con 0x7f616c072d40 2024-05-28T14:01:40.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6163e00640 1 -- 172.21.15.42:0/1201871987 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f615c0be070 con 0x7f616c072d40 2024-05-28T14:01:40.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6163e00640 1 -- 172.21.15.42:0/1201871987 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c0ad870 con 0x7f616c072d40 2024-05-28T14:01:40.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6172600640 1 -- 172.21.15.42:0/1201871987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616c15c580 con 0x7f616c072d40 2024-05-28T14:01:40.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.340+0000 7f6172600640 1 -- 172.21.15.42:0/1201871987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f616c09b4c0 con 0x7f616c072d40 2024-05-28T14:01:40.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.344+0000 7f6172600640 1 -- 172.21.15.42:0/1201871987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f616c068a90 con 0x7f616c072d40 2024-05-28T14:01:40.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.344+0000 7f6163e00640 1 -- 172.21.15.42:0/1201871987 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f615c0ada10 con 0x7f616c072d40 2024-05-28T14:01:40.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.344+0000 7f6163e00640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6148068650 0x7f614806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:40.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.344+0000 7f6163e00640 1 -- 172.21.15.42:0/1201871987 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5823+0+0 (secure 0 0 0) 0x7f615c122290 con 0x7f616c072d40 2024-05-28T14:01:40.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.344+0000 7f6162a00640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6148068650 0x7f614806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:40.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.344+0000 7f6162a00640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6148068650 0x7f614806ab10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f616c0cfdc0 tx=0x7f6154009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:40.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.348+0000 7f6163e00640 1 -- 172.21.15.42:0/1201871987 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f615c016520 con 0x7f616c072d40 2024-05-28T14:01:40.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:40.516+0000 7f6172600640 1 -- 172.21.15.42:0/1201871987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f616c15cc90 con 0x7f616c072d40 2024-05-28T14:01:41.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.164+0000 7f6163e00640 1 -- 172.21.15.42:0/1201871987 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) ==== 181+0+0 (secure 0 0 0) 0x7f615c0add00 con 0x7f616c072d40 2024-05-28T14:01:41.152 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-05-28T14:01:41.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.168+0000 7f6172600640 1 -- 172.21.15.42:0/1201871987 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6148068650 msgr2=0x7f614806ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:41.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.168+0000 7f6172600640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6148068650 0x7f614806ab10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f616c0cfdc0 tx=0x7f6154009040 comp rx=0 tx=0).stop 2024-05-28T14:01:41.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.168+0000 7f6172600640 1 -- 172.21.15.42:0/1201871987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f616c072d40 msgr2=0x7f616c153d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:41.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.168+0000 7f6172600640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f616c072d40 0x7f616c153d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f615c0b0040 tx=0x7f615c0195b0 comp rx=0 tx=0).stop 2024-05-28T14:01:41.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.168+0000 7f6172600640 1 -- 172.21.15.42:0/1201871987 shutdown_connections 2024-05-28T14:01:41.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.168+0000 7f6172600640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6148068650 0x7f614806ab10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:41.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.168+0000 7f6172600640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f616c15b350 0x7f616c1549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:41.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.168+0000 7f6172600640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f616c0d0e00 0x7f616c154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:41.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.168+0000 7f6172600640 1 --2- 172.21.15.42:0/1201871987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f616c072d40 0x7f616c153d90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:41.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.168+0000 7f6172600640 1 -- 172.21.15.42:0/1201871987 >> 172.21.15.42:0/1201871987 conn(0x7f616c06bdf0 msgr2=0x7f616c0b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:41.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.168+0000 7f6172600640 1 -- 172.21.15.42:0/1201871987 shutdown_connections 2024-05-28T14:01:41.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.168+0000 7f6172600640 1 -- 172.21.15.42:0/1201871987 wait complete. 2024-05-28T14:01:41.181 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-28T14:01:41.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.344+0000 7ff3b0800640 1 Processor -- start 2024-05-28T14:01:41.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.344+0000 7ff3b0800640 1 -- start start 2024-05-28T14:01:41.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3b0800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3a809b160 0x7ff3a8070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:41.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3b0800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3a806ff10 con 0x7ff3a806c940 2024-05-28T14:01:41.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3b0800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3a80701b0 con 0x7ff3a806e740 2024-05-28T14:01:41.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3b0800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a806eb20 con 0x7ff3a809b160 2024-05-28T14:01:41.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3ad400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff3a806c940 0x7ff3a809aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51568/0 (socket says 172.21.15.42:51568) 2024-05-28T14:01:41.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3ad400640 1 -- 172.21.15.42:0/1395791730 learned_addr learned my addr 172.21.15.42:0/1395791730 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:41.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3ae800640 1 --2- 172.21.15.42:0/1395791730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3a809b160 0x7ff3a8070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 -- 172.21.15.42:0/1395791730 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3880346728 0 0) 0x7ff3a806ff10 con 0x7ff3a806c940 2024-05-28T14:01:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 -- 172.21.15.42:0/1395791730 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff390003530 con 0x7ff3a806c940 2024-05-28T14:01:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 -- 172.21.15.42:0/1395791730 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1327896572 0 0) 0x7ff390003530 con 0x7ff3a806c940 2024-05-28T14:01:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 -- 172.21.15.42:0/1395791730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3a809b160 msgr2=0x7ff3a8070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 --2- 172.21.15.42:0/1395791730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3a809b160 0x7ff3a8070f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 -- 172.21.15.42:0/1395791730 >> v1:172.21.15.63:6789/0 conn(0x7ff3a806e740 legacy=0x7ff3a80cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 -- 172.21.15.42:0/1395791730 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a8070490 con 0x7ff3a806c940 2024-05-28T14:01:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 -- 172.21.15.42:0/1395791730 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7ff398001680 con 0x7ff3a806c940 2024-05-28T14:01:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 -- 172.21.15.42:0/1395791730 >> v1:172.21.15.42:6789/0 conn(0x7ff3a806c940 legacy=0x7ff3a809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 --2- 172.21.15.42:0/1395791730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3a809b160 0x7ff3900039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 --2- 172.21.15.42:0/1395791730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff390003f70 0x7ff390004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 --2- 172.21.15.42:0/1395791730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff390004d50 0x7ff3900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 -- 172.21.15.42:0/1395791730 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a806eb20 con 0x7ff390004d50 2024-05-28T14:01:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 -- 172.21.15.42:0/1395791730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff39000bb60 con 0x7ff390003f70 2024-05-28T14:01:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3aca00640 1 -- 172.21.15.42:0/1395791730 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff39000bd60 con 0x7ff3a809b160 2024-05-28T14:01:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3ae800640 1 --2- 172.21.15.42:0/1395791730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff390004d50 0x7ff3900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3ade00640 1 --2- 172.21.15.42:0/1395791730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3a809b160 0x7ff3900039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3ad400640 1 --2- 172.21.15.42:0/1395791730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff390003f70 0x7ff390004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3ae800640 1 -- 172.21.15.42:0/1395791730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3a809b160 msgr2=0x7ff3900039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3ae800640 1 --2- 172.21.15.42:0/1395791730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3a809b160 0x7ff3900039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3ae800640 1 -- 172.21.15.42:0/1395791730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff390003f70 msgr2=0x7ff390004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3ae800640 1 --2- 172.21.15.42:0/1395791730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff390003f70 0x7ff390004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3ae800640 1 -- 172.21.15.42:0/1395791730 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff390003530 con 0x7ff390004d50 2024-05-28T14:01:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.348+0000 7ff3ad400640 1 --2- 172.21.15.42:0/1395791730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff390003f70 0x7ff390004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:41.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.352+0000 7ff3ae800640 1 --2- 172.21.15.42:0/1395791730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff390004d50 0x7ff3900051d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80733b0 tx=0x7ff3a401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9e2da3e314dcd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:41.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.352+0000 7ff3aca00640 1 -- 172.21.15.42:0/1395791730 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40b2070 con 0x7ff390004d50 2024-05-28T14:01:41.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.352+0000 7ff3aca00640 1 -- 172.21.15.42:0/1395791730 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3a40a5c30 con 0x7ff390004d50 2024-05-28T14:01:41.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.352+0000 7ff3aca00640 1 -- 172.21.15.42:0/1395791730 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4009c10 con 0x7ff390004d50 2024-05-28T14:01:41.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.352+0000 7ff3b0800640 1 -- 172.21.15.42:0/1395791730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff390004d50 msgr2=0x7ff3900051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:41.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.352+0000 7ff3b0800640 1 --2- 172.21.15.42:0/1395791730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff390004d50 0x7ff3900051d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80733b0 tx=0x7ff3a401cc20 comp rx=0 tx=0).stop 2024-05-28T14:01:41.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.352+0000 7ff3ae800640 1 -- 172.21.15.42:0/1395791730 reap_dead start 2024-05-28T14:01:41.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.352+0000 7ff3b0800640 1 -- 172.21.15.42:0/1395791730 shutdown_connections 2024-05-28T14:01:41.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.352+0000 7ff3b0800640 1 -- 172.21.15.42:0/1395791730 >> 172.21.15.42:0/1395791730 conn(0x7ff3a80b3fa0 msgr2=0x7ff3a80b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:41.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.352+0000 7ff3b0800640 1 -- 172.21.15.42:0/1395791730 shutdown_connections 2024-05-28T14:01:41.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.352+0000 7ff3b0800640 1 -- 172.21.15.42:0/1395791730 wait complete. 2024-05-28T14:01:41.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.352+0000 7ff3b0800640 1 Processor -- start 2024-05-28T14:01:41.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.352+0000 7ff3b0800640 1 -- start start 2024-05-28T14:01:41.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3b0800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80cafc0 0x7ff3a81f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:41.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3b0800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3a80cb3a0 0x7ff3a81565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:41.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3b0800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a8156b30 0x7ff3a80b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:41.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3b0800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a80ce2e0 con 0x7ff3a80cafc0 2024-05-28T14:01:41.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3b0800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a8070a50 con 0x7ff3a8156b30 2024-05-28T14:01:41.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3b0800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a809baa0 con 0x7ff3a80cb3a0 2024-05-28T14:01:41.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3ade00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80cafc0 0x7ff3a81f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:41.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3ade00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80cafc0 0x7ff3a81f7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34444/0 (socket says 172.21.15.42:34444) 2024-05-28T14:01:41.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3ade00640 1 -- 172.21.15.42:0/87925800 learned_addr learned my addr 172.21.15.42:0/87925800 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:41.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3ad400640 1 --2- 172.21.15.42:0/87925800 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3a80cb3a0 0x7ff3a81565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:41.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3ae800640 1 --2- 172.21.15.42:0/87925800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a8156b30 0x7ff3a80b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:41.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3ade00640 1 -- 172.21.15.42:0/87925800 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3a80cb3a0 msgr2=0x7ff3a81565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:41.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3ade00640 1 --2- 172.21.15.42:0/87925800 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3a80cb3a0 0x7ff3a81565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:41.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3ade00640 1 -- 172.21.15.42:0/87925800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a8156b30 msgr2=0x7ff3a80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:41.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3ade00640 1 --2- 172.21.15.42:0/87925800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a8156b30 0x7ff3a80b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:41.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3ade00640 1 -- 172.21.15.42:0/87925800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a80c9720 con 0x7ff3a80cafc0 2024-05-28T14:01:41.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3ade00640 1 --2- 172.21.15.42:0/87925800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80cafc0 0x7ff3a81f7b00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00f7f0 tx=0x7ff39c00fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:41.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3a2000640 1 -- 172.21.15.42:0/87925800 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0050d0 con 0x7ff3a80cafc0 2024-05-28T14:01:41.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3a2000640 1 -- 172.21.15.42:0/87925800 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff39c018070 con 0x7ff3a80cafc0 2024-05-28T14:01:41.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3b0800640 1 -- 172.21.15.42:0/87925800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a80c9a00 con 0x7ff3a80cafc0 2024-05-28T14:01:41.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3a2000640 1 -- 172.21.15.42:0/87925800 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00d730 con 0x7ff3a80cafc0 2024-05-28T14:01:41.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3b0800640 1 -- 172.21.15.42:0/87925800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a80c9fc0 con 0x7ff3a80cafc0 2024-05-28T14:01:41.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.356+0000 7ff3b0800640 1 -- 172.21.15.42:0/87925800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a8068a90 con 0x7ff3a80cafc0 2024-05-28T14:01:41.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.360+0000 7ff3a2000640 1 -- 172.21.15.42:0/87925800 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7ff39c00d8d0 con 0x7ff3a80cafc0 2024-05-28T14:01:41.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.360+0000 7ff3a2000640 1 --2- 172.21.15.42:0/87925800 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff38c068650 0x7ff38c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:41.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.360+0000 7ff3a2000640 1 -- 172.21.15.42:0/87925800 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5385+0+0 (secure 0 0 0) 0x7ff39c0869b0 con 0x7ff3a80cafc0 2024-05-28T14:01:41.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.360+0000 7ff3ad400640 1 --2- 172.21.15.42:0/87925800 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff38c068650 0x7ff38c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:41.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.360+0000 7ff3ad400640 1 --2- 172.21.15.42:0/87925800 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff38c068650 0x7ff38c06ab10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff3980017f0 tx=0x7ff398001500 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:41.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.364+0000 7ff3a2000640 1 -- 172.21.15.42:0/87925800 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff39c056760 con 0x7ff3a80cafc0 2024-05-28T14:01:41.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:41.532+0000 7ff3b0800640 1 -- 172.21.15.42:0/87925800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff3a8156f10 con 0x7ff3a80cafc0 2024-05-28T14:01:42.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.168+0000 7ff3a2000640 1 -- 172.21.15.42:0/87925800 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) ==== 169+0+0 (secure 0 0 0) 0x7ff39c05aa60 con 0x7ff3a80cafc0 2024-05-28T14:01:42.154 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-05-28T14:01:42.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.172+0000 7ff3b0800640 1 -- 172.21.15.42:0/87925800 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff38c068650 msgr2=0x7ff38c06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:42.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.172+0000 7ff3b0800640 1 --2- 172.21.15.42:0/87925800 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff38c068650 0x7ff38c06ab10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff3980017f0 tx=0x7ff398001500 comp rx=0 tx=0).stop 2024-05-28T14:01:42.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.172+0000 7ff3b0800640 1 -- 172.21.15.42:0/87925800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80cafc0 msgr2=0x7ff3a81f7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:42.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.172+0000 7ff3b0800640 1 --2- 172.21.15.42:0/87925800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80cafc0 0x7ff3a81f7b00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00f7f0 tx=0x7ff39c00fcb0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.172+0000 7ff3b0800640 1 -- 172.21.15.42:0/87925800 shutdown_connections 2024-05-28T14:01:42.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.172+0000 7ff3b0800640 1 --2- 172.21.15.42:0/87925800 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff38c068650 0x7ff38c06ab10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.172+0000 7ff3b0800640 1 --2- 172.21.15.42:0/87925800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a8156b30 0x7ff3a80b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.172+0000 7ff3b0800640 1 --2- 172.21.15.42:0/87925800 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3a80cb3a0 0x7ff3a81565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.172+0000 7ff3b0800640 1 --2- 172.21.15.42:0/87925800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a80cafc0 0x7ff3a81f7b00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.172+0000 7ff3b0800640 1 -- 172.21.15.42:0/87925800 >> 172.21.15.42:0/87925800 conn(0x7ff3a80b3fa0 msgr2=0x7ff3a8161d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:42.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.172+0000 7ff3b0800640 1 -- 172.21.15.42:0/87925800 shutdown_connections 2024-05-28T14:01:42.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.176+0000 7ff3b0800640 1 -- 172.21.15.42:0/87925800 wait complete. 2024-05-28T14:01:42.196 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-28T14:01:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.356+0000 7fb7a5400640 1 Processor -- start 2024-05-28T14:01:42.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.356+0000 7fb7a5400640 1 -- start start 2024-05-28T14:01:42.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.356+0000 7fb7a5400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb7a0071de0 0x7fb7a00721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:42.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.356+0000 7fb7a5400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7a0072700 con 0x7fb7a013fd10 2024-05-28T14:01:42.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.356+0000 7fb7a5400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7a0160230 con 0x7fb7a00ce3b0 2024-05-28T14:01:42.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.356+0000 7fb7a5400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb7a0160510 con 0x7fb7a0071de0 2024-05-28T14:01:42.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.356+0000 7fb79f200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb7a013fd10 0x7fb7a0140160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51580/0 (socket says 172.21.15.42:51580) 2024-05-28T14:01:42.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.356+0000 7fb79f200640 1 -- 172.21.15.42:0/2547779973 learned_addr learned my addr 172.21.15.42:0/2547779973 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:42.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.356+0000 7fb79e800640 1 --2- 172.21.15.42:0/2547779973 >> v2:172.21.15.42:3300/0 conn(0x7fb7a0071de0 0x7fb7a00721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:42.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.356+0000 7fb79de00640 1 -- 172.21.15.42:0/2547779973 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3920484579 0 0) 0x7fb7a0072700 con 0x7fb7a013fd10 2024-05-28T14:01:42.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.356+0000 7fb79de00640 1 -- 172.21.15.42:0/2547779973 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb784003530 con 0x7fb7a013fd10 2024-05-28T14:01:42.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.356+0000 7fb79e800640 1 -- 172.21.15.42:0/2547779973 >> v1:172.21.15.63:6789/0 conn(0x7fb7a00ce3b0 legacy=0x7fb7a00ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:42.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.356+0000 7fb79e800640 1 -- 172.21.15.42:0/2547779973 >> v1:172.21.15.42:6789/0 conn(0x7fb7a013fd10 legacy=0x7fb7a0140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:42.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.356+0000 7fb79e800640 1 -- 172.21.15.42:0/2547779973 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a01607f0 con 0x7fb7a0071de0 2024-05-28T14:01:42.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.360+0000 7fb79e800640 1 --2- 172.21.15.42:0/2547779973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7a0071de0 0x7fb7a00721c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb7900097c0 tx=0x7fb79001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80dfa9ffb4daff2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:42.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.360+0000 7fb79de00640 1 -- 172.21.15.42:0/2547779973 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7900b0070 con 0x7fb7a0071de0 2024-05-28T14:01:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.360+0000 7fb79de00640 1 -- 172.21.15.42:0/2547779973 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7900ab600 con 0x7fb7a0071de0 2024-05-28T14:01:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.360+0000 7fb79de00640 1 -- 172.21.15.42:0/2547779973 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7900ab9d0 con 0x7fb7a0071de0 2024-05-28T14:01:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.360+0000 7fb7a5400640 1 -- 172.21.15.42:0/2547779973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7a0071de0 msgr2=0x7fb7a00721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.360+0000 7fb7a5400640 1 --2- 172.21.15.42:0/2547779973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7a0071de0 0x7fb7a00721c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb7900097c0 tx=0x7fb79001c380 comp rx=0 tx=0).stop 2024-05-28T14:01:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.360+0000 7fb7a5400640 1 -- 172.21.15.42:0/2547779973 shutdown_connections 2024-05-28T14:01:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.360+0000 7fb7a5400640 1 --2- 172.21.15.42:0/2547779973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7a0071de0 0x7fb7a00721c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.360+0000 7fb7a5400640 1 -- 172.21.15.42:0/2547779973 >> 172.21.15.42:0/2547779973 conn(0x7fb7a00b21f0 msgr2=0x7fb7a00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.360+0000 7fb7a5400640 1 -- 172.21.15.42:0/2547779973 shutdown_connections 2024-05-28T14:01:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.360+0000 7fb7a5400640 1 -- 172.21.15.42:0/2547779973 wait complete. 2024-05-28T14:01:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb7a5400640 1 Processor -- start 2024-05-28T14:01:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb7a5400640 1 -- start start 2024-05-28T14:01:42.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb7a5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a0071de0 0x7fb7a015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:42.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb7a5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7a00ce3b0 0x7fb7a013d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:42.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb7a5400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7a013fd10 0x7fb7a0137af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:42.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb7a5400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a016a840 con 0x7fb7a00ce3b0 2024-05-28T14:01:42.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb7a5400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a016a6c0 con 0x7fb7a0071de0 2024-05-28T14:01:42.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb7a5400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb7a016a9c0 con 0x7fb7a013fd10 2024-05-28T14:01:42.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb795e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7a00ce3b0 0x7fb7a013d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:42.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb795e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7a00ce3b0 0x7fb7a013d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34466/0 (socket says 172.21.15.42:34466) 2024-05-28T14:01:42.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb795e00640 1 -- 172.21.15.42:0/2295787831 learned_addr learned my addr 172.21.15.42:0/2295787831 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:42.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb79e800640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a0071de0 0x7fb7a015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:42.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb79f200640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7a013fd10 0x7fb7a0137af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:42.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb795e00640 1 -- 172.21.15.42:0/2295787831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7a013fd10 msgr2=0x7fb7a0137af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:42.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb795e00640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7a013fd10 0x7fb7a0137af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb795e00640 1 -- 172.21.15.42:0/2295787831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a0071de0 msgr2=0x7fb7a015c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:42.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb795e00640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a0071de0 0x7fb7a015c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb795e00640 1 -- 172.21.15.42:0/2295787831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a0138030 con 0x7fb7a00ce3b0 2024-05-28T14:01:42.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb79f200640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7a013fd10 0x7fb7a0137af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:42.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb795e00640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7a00ce3b0 0x7fb7a013d3f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb78800c420 tx=0x7fb78800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:42.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb797400640 1 -- 172.21.15.42:0/2295787831 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7880052d0 con 0x7fb7a00ce3b0 2024-05-28T14:01:42.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb797400640 1 -- 172.21.15.42:0/2295787831 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb788005470 con 0x7fb7a00ce3b0 2024-05-28T14:01:42.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb797400640 1 -- 172.21.15.42:0/2295787831 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788007260 con 0x7fb7a00ce3b0 2024-05-28T14:01:42.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb7a5400640 1 -- 172.21.15.42:0/2295787831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a013db20 con 0x7fb7a00ce3b0 2024-05-28T14:01:42.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb7a5400640 1 -- 172.21.15.42:0/2295787831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a013bb70 con 0x7fb7a00ce3b0 2024-05-28T14:01:42.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.364+0000 7fb7a5400640 1 -- 172.21.15.42:0/2295787831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a00738b0 con 0x7fb7a00ce3b0 2024-05-28T14:01:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.368+0000 7fb797400640 1 -- 172.21.15.42:0/2295787831 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7fb788007400 con 0x7fb7a00ce3b0 2024-05-28T14:01:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.368+0000 7fb797400640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb77c068650 0x7fb77c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.368+0000 7fb797400640 1 -- 172.21.15.42:0/2295787831 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4971+0+0 (secure 0 0 0) 0x7fb788085fe0 con 0x7fb7a00ce3b0 2024-05-28T14:01:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.368+0000 7fb79e800640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb77c068650 0x7fb77c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.368+0000 7fb79e800640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb77c068650 0x7fb77c06ab10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb7900ad000 tx=0x7fb7900a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:42.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.372+0000 7fb797400640 1 -- 172.21.15.42:0/2295787831 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb788055ea0 con 0x7fb7a00ce3b0 2024-05-28T14:01:42.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.552+0000 7fb7a5400640 1 -- 172.21.15.42:0/2295787831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb7a004fdd0 con 0x7fb7a00ce3b0 2024-05-28T14:01:42.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.552+0000 7fb797400640 1 -- 172.21.15.42:0/2295787831 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fb78805a1a0 con 0x7fb7a00ce3b0 2024-05-28T14:01:42.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.556+0000 7fb7a5400640 1 -- 172.21.15.42:0/2295787831 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb77c068650 msgr2=0x7fb77c06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:42.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.556+0000 7fb7a5400640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb77c068650 0x7fb77c06ab10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb7900ad000 tx=0x7fb7900a6000 comp rx=0 tx=0).stop 2024-05-28T14:01:42.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.556+0000 7fb7a5400640 1 -- 172.21.15.42:0/2295787831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7a00ce3b0 msgr2=0x7fb7a013d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:42.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.556+0000 7fb7a5400640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7a00ce3b0 0x7fb7a013d3f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb78800c420 tx=0x7fb78800c8e0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.556+0000 7fb7a5400640 1 -- 172.21.15.42:0/2295787831 shutdown_connections 2024-05-28T14:01:42.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.556+0000 7fb7a5400640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb77c068650 0x7fb77c06ab10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.556+0000 7fb7a5400640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7a013fd10 0x7fb7a0137af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.556+0000 7fb7a5400640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7a00ce3b0 0x7fb7a013d3f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.556+0000 7fb7a5400640 1 --2- 172.21.15.42:0/2295787831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a0071de0 0x7fb7a015c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.556+0000 7fb7a5400640 1 -- 172.21.15.42:0/2295787831 >> 172.21.15.42:0/2295787831 conn(0x7fb7a00b21f0 msgr2=0x7fb7a01455f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:42.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.556+0000 7fb7a5400640 1 -- 172.21.15.42:0/2295787831 shutdown_connections 2024-05-28T14:01:42.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.556+0000 7fb7a5400640 1 -- 172.21.15.42:0/2295787831 wait complete. 2024-05-28T14:01:42.562 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone' 2024-05-28T14:01:42.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f320f400640 1 Processor -- start 2024-05-28T14:01:42.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f320f400640 1 -- start start 2024-05-28T14:01:42.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f320f400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f320815c990 0x7f32080d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:42.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f320f400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32080cd300 con 0x7f320809bbd0 2024-05-28T14:01:42.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f320f400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32080cd5e0 con 0x7f32080d06a0 2024-05-28T14:01:42.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f320f400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f320815cd70 con 0x7f320815c990 2024-05-28T14:01:42.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f320809bbd0 0x7f320809aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51582/0 (socket says 172.21.15.42:51582) 2024-05-28T14:01:42.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207e00640 1 -- 172.21.15.42:0/3629901010 learned_addr learned my addr 172.21.15.42:0/3629901010 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f320ca00640 1 --2- 172.21.15.42:0/3629901010 >> v2:172.21.15.63:3300/0 conn(0x7f320815c990 0x7f32080d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3365832213 0 0) 0x7f32080cd300 con 0x7f320809bbd0 2024-05-28T14:01:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31f0003530 con 0x7f320809bbd0 2024-05-28T14:01:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2181967462 0 0) 0x7f32080cd5e0 con 0x7f32080d06a0 2024-05-28T14:01:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32080cd300 con 0x7f32080d06a0 2024-05-28T14:01:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2176290111 0 0) 0x7f31f0003530 con 0x7f320809bbd0 2024-05-28T14:01:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 >> v2:172.21.15.63:3300/0 conn(0x7f320815c990 msgr2=0x7f32080d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 --2- 172.21.15.42:0/3629901010 >> v2:172.21.15.63:3300/0 conn(0x7f320815c990 0x7f32080d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 >> v1:172.21.15.63:6789/0 conn(0x7f32080d06a0 legacy=0x7f32080b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32080cd8c0 con 0x7f320809bbd0 2024-05-28T14:01:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f31f4001680 con 0x7f320809bbd0 2024-05-28T14:01:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 >> v1:172.21.15.42:6789/0 conn(0x7f320809bbd0 legacy=0x7f320809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 --2- 172.21.15.42:0/3629901010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31f0003ba0 0x7f31f0003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 --2- 172.21.15.42:0/3629901010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f0004810 0x7f31f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 --2- 172.21.15.42:0/3629901010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31f00051d0 0x7f31f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f31f000c0a0 con 0x7f31f00051d0 2024-05-28T14:01:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f31f000c220 con 0x7f31f0004810 2024-05-28T14:01:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f31f000c420 con 0x7f31f0003ba0 2024-05-28T14:01:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f320ca00640 1 --2- 172.21.15.42:0/3629901010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31f0003ba0 0x7f31f0003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f320d400640 1 --2- 172.21.15.42:0/3629901010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31f00051d0 0x7f31f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.728+0000 7f3207e00640 1 --2- 172.21.15.42:0/3629901010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f0004810 0x7f31f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f320d400640 1 -- 172.21.15.42:0/3629901010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31f0003ba0 msgr2=0x7f31f0003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f320d400640 1 --2- 172.21.15.42:0/3629901010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31f0003ba0 0x7f31f0003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f320d400640 1 -- 172.21.15.42:0/3629901010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f0004810 msgr2=0x7f31f0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f320d400640 1 --2- 172.21.15.42:0/3629901010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f0004810 0x7f31f0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f320d400640 1 -- 172.21.15.42:0/3629901010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f0003530 con 0x7f31f00051d0 2024-05-28T14:01:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f320d400640 1 -- 172.21.15.42:0/3629901010 reap_dead start 2024-05-28T14:01:42.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f320d400640 1 --2- 172.21.15.42:0/3629901010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31f00051d0 0x7f31f000b9d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f32080739d0 tx=0x7f320001c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4d15775e47c02c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:42.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200001c20 con 0x7f31f00051d0 2024-05-28T14:01:42.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3200003b80 con 0x7f31f00051d0 2024-05-28T14:01:42.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f3207400640 1 -- 172.21.15.42:0/3629901010 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32000050d0 con 0x7f31f00051d0 2024-05-28T14:01:42.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f320f400640 1 -- 172.21.15.42:0/3629901010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31f00051d0 msgr2=0x7f31f000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:42.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f320f400640 1 --2- 172.21.15.42:0/3629901010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31f00051d0 0x7f31f000b9d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f32080739d0 tx=0x7f320001c700 comp rx=0 tx=0).stop 2024-05-28T14:01:42.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f320f400640 1 -- 172.21.15.42:0/3629901010 shutdown_connections 2024-05-28T14:01:42.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f320f400640 1 --2- 172.21.15.42:0/3629901010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31f00051d0 0x7f31f000b9d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f320f400640 1 -- 172.21.15.42:0/3629901010 >> 172.21.15.42:0/3629901010 conn(0x7f32080b4080 msgr2=0x7f32080b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:42.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.732+0000 7f320f400640 1 -- 172.21.15.42:0/3629901010 shutdown_connections 2024-05-28T14:01:42.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f320f400640 1 -- 172.21.15.42:0/3629901010 wait complete. 2024-05-28T14:01:42.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f320f400640 1 Processor -- start 2024-05-28T14:01:42.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f320f400640 1 -- start start 2024-05-28T14:01:42.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f320f400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31f00051d0 0x7f32080b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:42.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f320f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32080c9740 0x7f32080c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:42.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f320f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f320815a710 0x7f320815abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:42.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f320f400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f32080cde80 con 0x7f32080c9740 2024-05-28T14:01:42.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f320f400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32080cd160 con 0x7f320815a710 2024-05-28T14:01:42.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f320f400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f32080ce3a0 con 0x7f31f00051d0 2024-05-28T14:01:42.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f3207e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32080c9740 0x7f32080c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:42.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f3207e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32080c9740 0x7f32080c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34494/0 (socket says 172.21.15.42:34494) 2024-05-28T14:01:42.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f3207e00640 1 -- 172.21.15.42:0/1402450798 learned_addr learned my addr 172.21.15.42:0/1402450798 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:42.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f320ca00640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31f00051d0 0x7f32080b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:42.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f320d400640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f320815a710 0x7f320815abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:42.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f3207e00640 1 -- 172.21.15.42:0/1402450798 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31f00051d0 msgr2=0x7f32080b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:42.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f3207e00640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31f00051d0 0x7f32080b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f3207e00640 1 -- 172.21.15.42:0/1402450798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f320815a710 msgr2=0x7f320815abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:42.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f3207e00640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f320815a710 0x7f320815abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:42.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f3207e00640 1 -- 172.21.15.42:0/1402450798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32080b5b20 con 0x7f32080c9740 2024-05-28T14:01:42.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f320d400640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f320815a710 0x7f320815abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:42.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f3207e00640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32080c9740 0x7f32080c8170 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f31f40124c0 tx=0x7f31f40128c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:42.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f3204c00640 1 -- 172.21.15.42:0/1402450798 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f401c070 con 0x7f32080c9740 2024-05-28T14:01:42.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f3204c00640 1 -- 172.21.15.42:0/1402450798 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f4014770 con 0x7f32080c9740 2024-05-28T14:01:42.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f3204c00640 1 -- 172.21.15.42:0/1402450798 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4014b60 con 0x7f32080c9740 2024-05-28T14:01:42.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f320f400640 1 -- 172.21.15.42:0/1402450798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32080b5e00 con 0x7f32080c9740 2024-05-28T14:01:42.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.736+0000 7f320f400640 1 -- 172.21.15.42:0/1402450798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32080b63c0 con 0x7f32080c9740 2024-05-28T14:01:42.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.740+0000 7f320f400640 1 -- 172.21.15.42:0/1402450798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3208068a90 con 0x7f32080c9740 2024-05-28T14:01:42.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.740+0000 7f3204c00640 1 -- 172.21.15.42:0/1402450798 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f31f4014d00 con 0x7f32080c9740 2024-05-28T14:01:42.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.740+0000 7f3204c00640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f31e8068650 0x7f31e806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:42.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.740+0000 7f3204c00640 1 -- 172.21.15.42:0/1402450798 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4971+0+0 (secure 0 0 0) 0x7f31f4089460 con 0x7f32080c9740 2024-05-28T14:01:42.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.740+0000 7f320ca00640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f31e8068650 0x7f31e806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:42.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.740+0000 7f320ca00640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f31e8068650 0x7f31e806ab10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009b60 tx=0x7f31fc009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:42.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.744+0000 7f3204c00640 1 -- 172.21.15.42:0/1402450798 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f31f4059970 con 0x7f32080c9740 2024-05-28T14:01:42.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:42.968+0000 7f320f400640 1 -- 172.21.15.42:0/1402450798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone"]} v 0) -- 0x7f320806fe30 con 0x7f32080c9740 2024-05-28T14:01:43.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.180+0000 7f3204c00640 1 -- 172.21.15.42:0/1402450798 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone"]}]=0 v171) ==== 160+0+0 (secure 0 0 0) 0x7f31f4059270 con 0x7f32080c9740 2024-05-28T14:01:43.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.184+0000 7f320f400640 1 -- 172.21.15.42:0/1402450798 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f31e8068650 msgr2=0x7f31e806ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.184+0000 7f320f400640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f31e8068650 0x7f31e806ab10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009b60 tx=0x7f31fc009190 comp rx=0 tx=0).stop 2024-05-28T14:01:43.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.184+0000 7f320f400640 1 -- 172.21.15.42:0/1402450798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32080c9740 msgr2=0x7f32080c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.184+0000 7f320f400640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32080c9740 0x7f32080c8170 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f31f40124c0 tx=0x7f31f40128c0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.184+0000 7f320f400640 1 -- 172.21.15.42:0/1402450798 shutdown_connections 2024-05-28T14:01:43.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.184+0000 7f320f400640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f31e8068650 0x7f31e806ab10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.184+0000 7f320f400640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f320815a710 0x7f320815abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.184+0000 7f320f400640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32080c9740 0x7f32080c8170 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.184+0000 7f320f400640 1 --2- 172.21.15.42:0/1402450798 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31f00051d0 0x7f32080b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.184+0000 7f320f400640 1 -- 172.21.15.42:0/1402450798 >> 172.21.15.42:0/1402450798 conn(0x7f32080b4080 msgr2=0x7f3208071360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:43.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.184+0000 7f320f400640 1 -- 172.21.15.42:0/1402450798 shutdown_connections 2024-05-28T14:01:43.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.184+0000 7f320f400640 1 -- 172.21.15.42:0/1402450798 wait complete. 2024-05-28T14:01:43.190 INFO:tasks.cephfs_test_runner:Test that when one clone is in progress, one progress bar is printed ... ok 2024-05-28T14:01:43.190 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bars_for_4_clones (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed when number of clone jobs is 2024-05-28T14:01:43.190 INFO:tasks.cephfs_test_runner:test_progress_bars_for_4_clones (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-28T14:01:43.190 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:01:43.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.344+0000 7f90fde00640 1 Processor -- start 2024-05-28T14:01:43.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.344+0000 7f90fde00640 1 -- start start 2024-05-28T14:01:43.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.348+0000 7f90fde00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90f806f660 0x7f90f8073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:43.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.348+0000 7f90fde00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f90f80d0e00 0x7f90f806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:43.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.348+0000 7f90fde00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90f815f280 con 0x7f90f815ba40 2024-05-28T14:01:43.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.348+0000 7f90fde00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f90f815f030 con 0x7f90f80d0e00 2024-05-28T14:01:43.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.348+0000 7f90fde00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90f815f560 con 0x7f90f806f660 2024-05-28T14:01:43.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.348+0000 7f90f7400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f90f80d0e00 0x7f90f806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:43.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.348+0000 7f90f6a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90f806f660 0x7f90f8073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:43.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.348+0000 7f90f7400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f90f80d0e00 0x7f90f806dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34508/0 (socket says 172.21.15.42:34508) 2024-05-28T14:01:43.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.348+0000 7f90f7400640 1 -- 172.21.15.42:0/3795168050 learned_addr learned my addr 172.21.15.42:0/3795168050 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:43.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.348+0000 7f90f7400640 1 -- 172.21.15.42:0/3795168050 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90f806f660 msgr2=0x7f90f8073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:43.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.348+0000 7f90f7400640 1 --2- 172.21.15.42:0/3795168050 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90f806f660 0x7f90f8073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.348+0000 7f90f7400640 1 -- 172.21.15.42:0/3795168050 >> v1:172.21.15.63:6789/0 conn(0x7f90f815ba40 legacy=0x7f90f80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.348+0000 7f90f7400640 1 -- 172.21.15.42:0/3795168050 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f815f840 con 0x7f90f80d0e00 2024-05-28T14:01:43.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.352+0000 7f90f7400640 1 --2- 172.21.15.42:0/3795168050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f80d0e00 0x7f90f806dd60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009870 tx=0x7f90f001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5aeb6d8ec3d31c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:43.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.352+0000 7f90f6000640 1 -- 172.21.15.42:0/3795168050 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f00ab5f0 con 0x7f90f80d0e00 2024-05-28T14:01:43.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.352+0000 7f90f6000640 1 -- 172.21.15.42:0/3795168050 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f00b0070 con 0x7f90f80d0e00 2024-05-28T14:01:43.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.352+0000 7f90f6000640 1 -- 172.21.15.42:0/3795168050 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f00ab9b0 con 0x7f90f80d0e00 2024-05-28T14:01:43.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.352+0000 7f90fde00640 1 -- 172.21.15.42:0/3795168050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f80d0e00 msgr2=0x7f90f806dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.352+0000 7f90fde00640 1 --2- 172.21.15.42:0/3795168050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f80d0e00 0x7f90f806dd60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009870 tx=0x7f90f001c410 comp rx=0 tx=0).stop 2024-05-28T14:01:43.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.352+0000 7f90fde00640 1 -- 172.21.15.42:0/3795168050 shutdown_connections 2024-05-28T14:01:43.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.352+0000 7f90fde00640 1 --2- 172.21.15.42:0/3795168050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f80d0e00 0x7f90f806dd60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.352+0000 7f90fde00640 1 --2- 172.21.15.42:0/3795168050 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90f806f660 0x7f90f8073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.352+0000 7f90fde00640 1 -- 172.21.15.42:0/3795168050 >> 172.21.15.42:0/3795168050 conn(0x7f90f806bdf0 msgr2=0x7f90f806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:43.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.352+0000 7f90fde00640 1 -- 172.21.15.42:0/3795168050 shutdown_connections 2024-05-28T14:01:43.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.352+0000 7f90fde00640 1 -- 172.21.15.42:0/3795168050 wait complete. 2024-05-28T14:01:43.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.352+0000 7f90fde00640 1 Processor -- start 2024-05-28T14:01:43.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.352+0000 7f90fde00640 1 -- start start 2024-05-28T14:01:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90fde00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90f806f660 0x7f90f814dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90fde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f80d0e00 0x7f90f809a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90fde00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f80b25d0 0x7f90f815cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90fde00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f90f8168550 con 0x7f90f80d0e00 2024-05-28T14:01:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90fde00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90f81683d0 con 0x7f90f80b25d0 2024-05-28T14:01:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90fde00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90f81686d0 con 0x7f90f806f660 2024-05-28T14:01:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90ee000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f80d0e00 0x7f90f809a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90ee000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f80d0e00 0x7f90f809a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34510/0 (socket says 172.21.15.42:34510) 2024-05-28T14:01:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90ee000640 1 -- 172.21.15.42:0/2687127846 learned_addr learned my addr 172.21.15.42:0/2687127846 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90f6a00640 1 --2- 172.21.15.42:0/2687127846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90f806f660 0x7f90f814dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90f7400640 1 --2- 172.21.15.42:0/2687127846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f80b25d0 0x7f90f815cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90ee000640 1 -- 172.21.15.42:0/2687127846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90f806f660 msgr2=0x7f90f814dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90ee000640 1 --2- 172.21.15.42:0/2687127846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90f806f660 0x7f90f814dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90ee000640 1 -- 172.21.15.42:0/2687127846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f80b25d0 msgr2=0x7f90f815cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90ee000640 1 --2- 172.21.15.42:0/2687127846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f80b25d0 0x7f90f815cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90ee000640 1 -- 172.21.15.42:0/2687127846 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f80b1530 con 0x7f90f80d0e00 2024-05-28T14:01:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90ee000640 1 --2- 172.21.15.42:0/2687127846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f80d0e00 0x7f90f809a2d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f90e000c370 tx=0x7f90e000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90efe00640 1 -- 172.21.15.42:0/2687127846 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e00052d0 con 0x7f90f80d0e00 2024-05-28T14:01:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90efe00640 1 -- 172.21.15.42:0/2687127846 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e0005470 con 0x7f90f80d0e00 2024-05-28T14:01:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90efe00640 1 -- 172.21.15.42:0/2687127846 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0007260 con 0x7f90f80d0e00 2024-05-28T14:01:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90fde00640 1 -- 172.21.15.42:0/2687127846 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f80b1810 con 0x7f90f80d0e00 2024-05-28T14:01:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90fde00640 1 -- 172.21.15.42:0/2687127846 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f80b1dd0 con 0x7f90f80d0e00 2024-05-28T14:01:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90fde00640 1 -- 172.21.15.42:0/2687127846 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f8068a90 con 0x7f90f80d0e00 2024-05-28T14:01:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.356+0000 7f90efe00640 1 -- 172.21.15.42:0/2687127846 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f90e0007400 con 0x7f90f80d0e00 2024-05-28T14:01:43.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.360+0000 7f90efe00640 1 --2- 172.21.15.42:0/2687127846 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f90d4068650 0x7f90d406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:43.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.360+0000 7f90efe00640 1 -- 172.21.15.42:0/2687127846 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4971+0+0 (secure 0 0 0) 0x7f90e0085f20 con 0x7f90f80d0e00 2024-05-28T14:01:43.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.360+0000 7f90f6a00640 1 --2- 172.21.15.42:0/2687127846 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f90d4068650 0x7f90d406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:43.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.360+0000 7f90f6a00640 1 --2- 172.21.15.42:0/2687127846 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f90d4068650 0x7f90d406ab10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f90e80113e0 tx=0x7f90e800b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:43.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.364+0000 7f90efe00640 1 -- 172.21.15.42:0/2687127846 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f90e0055e20 con 0x7f90f80d0e00 2024-05-28T14:01:43.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.532+0000 7f90fde00640 1 -- 172.21.15.42:0/2687127846 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90f8072bb0 con 0x7f90f80d0e00 2024-05-28T14:01:43.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.532+0000 7f90efe00640 1 -- 172.21.15.42:0/2687127846 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+13003 (secure 0 0 0) 0x7f90e005a120 con 0x7f90f80d0e00 2024-05-28T14:01:43.518 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:43.518 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":38,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:01:42.169442+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6835/2114729398":"2024-05-29T14:01:39.158280+0000","172.21.15.63:6833/2114729398":"2024-05-29T14:01:39.158280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:01:43.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.536+0000 7f90fde00640 1 -- 172.21.15.42:0/2687127846 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f90d4068650 msgr2=0x7f90d406ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.536+0000 7f90fde00640 1 --2- 172.21.15.42:0/2687127846 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f90d4068650 0x7f90d406ab10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f90e80113e0 tx=0x7f90e800b040 comp rx=0 tx=0).stop 2024-05-28T14:01:43.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.536+0000 7f90fde00640 1 -- 172.21.15.42:0/2687127846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f80d0e00 msgr2=0x7f90f809a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.536+0000 7f90fde00640 1 --2- 172.21.15.42:0/2687127846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f80d0e00 0x7f90f809a2d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f90e000c370 tx=0x7f90e000c830 comp rx=0 tx=0).stop 2024-05-28T14:01:43.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.536+0000 7f90fde00640 1 -- 172.21.15.42:0/2687127846 shutdown_connections 2024-05-28T14:01:43.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.536+0000 7f90fde00640 1 --2- 172.21.15.42:0/2687127846 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f90d4068650 0x7f90d406ab10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.536+0000 7f90fde00640 1 --2- 172.21.15.42:0/2687127846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f80b25d0 0x7f90f815cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.536+0000 7f90fde00640 1 --2- 172.21.15.42:0/2687127846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f80d0e00 0x7f90f809a2d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.536+0000 7f90fde00640 1 --2- 172.21.15.42:0/2687127846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90f806f660 0x7f90f814dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.536+0000 7f90fde00640 1 -- 172.21.15.42:0/2687127846 >> 172.21.15.42:0/2687127846 conn(0x7f90f806bdf0 msgr2=0x7f90f80b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:43.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.536+0000 7f90fde00640 1 -- 172.21.15.42:0/2687127846 shutdown_connections 2024-05-28T14:01:43.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.536+0000 7f90fde00640 1 -- 172.21.15.42:0/2687127846 wait complete. 2024-05-28T14:01:43.540 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:01:43.540 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:01:43.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.696+0000 7f1446000640 1 Processor -- start 2024-05-28T14:01:43.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.700+0000 7f1446000640 1 -- start start 2024-05-28T14:01:43.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.700+0000 7f1446000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f144006cfb0 0x7f144015f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:43.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.700+0000 7f1446000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f144015fe30 con 0x7f144009aa50 2024-05-28T14:01:43.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.700+0000 7f1446000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f144009bb00 con 0x7f14400b32f0 2024-05-28T14:01:43.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.700+0000 7f1446000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f14400d0800 con 0x7f144006cfb0 2024-05-28T14:01:43.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.700+0000 7f143f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f144006cfb0 0x7f144015f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:43.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.700+0000 7f1437e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f144009aa50 0x7f144006d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51588/0 (socket says 172.21.15.42:51588) 2024-05-28T14:01:43.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.700+0000 7f1437e00640 1 -- 172.21.15.42:0/2485045069 learned_addr learned my addr 172.21.15.42:0/2485045069 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:43.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.700+0000 7f143f400640 1 -- 172.21.15.42:0/2485045069 >> v1:172.21.15.63:6789/0 conn(0x7f14400b32f0 legacy=0x7f14400b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.700+0000 7f143f400640 1 -- 172.21.15.42:0/2485045069 >> v1:172.21.15.42:6789/0 conn(0x7f144009aa50 legacy=0x7f144006d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.700+0000 7f143f400640 1 -- 172.21.15.42:0/2485045069 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144015c8c0 con 0x7f144006cfb0 2024-05-28T14:01:43.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.704+0000 7f143f400640 1 --2- 172.21.15.42:0/2485045069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f144006cfb0 0x7f144015f8f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f14300097c0 tx=0x7f143001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a176fc7e9057e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:43.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.704+0000 7f143ea00640 1 -- 172.21.15.42:0/2485045069 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14300b1070 con 0x7f144006cfb0 2024-05-28T14:01:43.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.704+0000 7f143ea00640 1 -- 172.21.15.42:0/2485045069 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14300ac410 con 0x7f144006cfb0 2024-05-28T14:01:43.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.704+0000 7f143ea00640 1 -- 172.21.15.42:0/2485045069 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14300ac7e0 con 0x7f144006cfb0 2024-05-28T14:01:43.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.704+0000 7f1446000640 1 -- 172.21.15.42:0/2485045069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f144006cfb0 msgr2=0x7f144015f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.704+0000 7f1446000640 1 --2- 172.21.15.42:0/2485045069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f144006cfb0 0x7f144015f8f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f14300097c0 tx=0x7f143001c4a0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.704+0000 7f1446000640 1 -- 172.21.15.42:0/2485045069 shutdown_connections 2024-05-28T14:01:43.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.704+0000 7f1446000640 1 --2- 172.21.15.42:0/2485045069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f144006cfb0 0x7f144015f8f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.704+0000 7f1446000640 1 -- 172.21.15.42:0/2485045069 >> 172.21.15.42:0/2485045069 conn(0x7f14400b40d0 msgr2=0x7f14400b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:43.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.704+0000 7f1446000640 1 -- 172.21.15.42:0/2485045069 shutdown_connections 2024-05-28T14:01:43.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.704+0000 7f1446000640 1 -- 172.21.15.42:0/2485045069 wait complete. 2024-05-28T14:01:43.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.704+0000 7f1446000640 1 Processor -- start 2024-05-28T14:01:43.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.704+0000 7f1446000640 1 -- start start 2024-05-28T14:01:43.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1446000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f144006cfb0 0x7f144015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:43.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1446000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f144009aa50 0x7f144015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:43.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1446000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14400b32f0 0x7f144015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:43.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1446000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f14400b3e10 con 0x7f144009aa50 2024-05-28T14:01:43.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1446000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f144009b820 con 0x7f14400b32f0 2024-05-28T14:01:43.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1446000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f14400b1f40 con 0x7f144006cfb0 2024-05-28T14:01:43.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1437e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f144009aa50 0x7f144015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:43.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1437e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f144009aa50 0x7f144015a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34532/0 (socket says 172.21.15.42:34532) 2024-05-28T14:01:43.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1437e00640 1 -- 172.21.15.42:0/1412103072 learned_addr learned my addr 172.21.15.42:0/1412103072 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:43.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f143f400640 1 --2- 172.21.15.42:0/1412103072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f144006cfb0 0x7f144015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:43.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f143fe00640 1 --2- 172.21.15.42:0/1412103072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14400b32f0 0x7f144015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:43.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1437e00640 1 -- 172.21.15.42:0/1412103072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f144006cfb0 msgr2=0x7f144015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1437e00640 1 --2- 172.21.15.42:0/1412103072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f144006cfb0 0x7f144015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1437e00640 1 -- 172.21.15.42:0/1412103072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14400b32f0 msgr2=0x7f144015ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1437e00640 1 --2- 172.21.15.42:0/1412103072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14400b32f0 0x7f144015ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1437e00640 1 -- 172.21.15.42:0/1412103072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1440156dc0 con 0x7f144009aa50 2024-05-28T14:01:43.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1437e00640 1 --2- 172.21.15.42:0/1412103072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f144009aa50 0x7f144015a630 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f142800c500 tx=0x7f142800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:43.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1437400640 1 -- 172.21.15.42:0/1412103072 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428017070 con 0x7f144009aa50 2024-05-28T14:01:43.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1437400640 1 -- 172.21.15.42:0/1412103072 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1428005cf0 con 0x7f144009aa50 2024-05-28T14:01:43.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1446000640 1 -- 172.21.15.42:0/1412103072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14401570a0 con 0x7f144009aa50 2024-05-28T14:01:43.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1437400640 1 -- 172.21.15.42:0/1412103072 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14280072f0 con 0x7f144009aa50 2024-05-28T14:01:43.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1446000640 1 -- 172.21.15.42:0/1412103072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1440157660 con 0x7f144009aa50 2024-05-28T14:01:43.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1446000640 1 -- 172.21.15.42:0/1412103072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14400a06c0 con 0x7f144009aa50 2024-05-28T14:01:43.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.708+0000 7f1437400640 1 -- 172.21.15.42:0/1412103072 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f142801a070 con 0x7f144009aa50 2024-05-28T14:01:43.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.712+0000 7f1437400640 1 --2- 172.21.15.42:0/1412103072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f141c068650 0x7f141c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:43.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.712+0000 7f1437400640 1 -- 172.21.15.42:0/1412103072 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4971+0+0 (secure 0 0 0) 0x7f1428085e00 con 0x7f144009aa50 2024-05-28T14:01:43.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.712+0000 7f143f400640 1 --2- 172.21.15.42:0/1412103072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f141c068650 0x7f141c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:43.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.712+0000 7f143f400640 1 --2- 172.21.15.42:0/1412103072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f141c068650 0x7f141c06ab10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f14300ae000 tx=0x7f1430008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:43.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.716+0000 7f1437400640 1 -- 172.21.15.42:0/1412103072 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f144015fe30 con 0x7f144009aa50 2024-05-28T14:01:43.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.880+0000 7f1446000640 1 -- 172.21.15.42:0/1412103072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f144004fdd0 con 0x7f144009aa50 2024-05-28T14:01:43.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.880+0000 7f1437400640 1 -- 172.21.15.42:0/1412103072 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f1428055d50 con 0x7f144009aa50 2024-05-28T14:01:43.866 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:01:43.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.884+0000 7f1446000640 1 -- 172.21.15.42:0/1412103072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f141c068650 msgr2=0x7f141c06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.884+0000 7f1446000640 1 --2- 172.21.15.42:0/1412103072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f141c068650 0x7f141c06ab10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f14300ae000 tx=0x7f1430008040 comp rx=0 tx=0).stop 2024-05-28T14:01:43.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.884+0000 7f1446000640 1 -- 172.21.15.42:0/1412103072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f144009aa50 msgr2=0x7f144015a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:43.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.884+0000 7f1446000640 1 --2- 172.21.15.42:0/1412103072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f144009aa50 0x7f144015a630 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f142800c500 tx=0x7f142800c9c0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.884+0000 7f1446000640 1 -- 172.21.15.42:0/1412103072 shutdown_connections 2024-05-28T14:01:43.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.884+0000 7f1446000640 1 --2- 172.21.15.42:0/1412103072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f141c068650 0x7f141c06ab10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.884+0000 7f1446000640 1 --2- 172.21.15.42:0/1412103072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14400b32f0 0x7f144015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.884+0000 7f1446000640 1 --2- 172.21.15.42:0/1412103072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f144009aa50 0x7f144015a630 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.884+0000 7f1446000640 1 --2- 172.21.15.42:0/1412103072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f144006cfb0 0x7f144015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:43.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.884+0000 7f1446000640 1 -- 172.21.15.42:0/1412103072 >> 172.21.15.42:0/1412103072 conn(0x7f14400b40d0 msgr2=0x7f14400717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:43.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.884+0000 7f1446000640 1 -- 172.21.15.42:0/1412103072 shutdown_connections 2024-05-28T14:01:43.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:43.884+0000 7f1446000640 1 -- 172.21.15.42:0/1412103072 wait complete. 2024-05-28T14:01:43.888 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones' 2024-05-28T14:01:44.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f874ce00640 1 Processor -- start 2024-05-28T14:01:44.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f874ce00640 1 -- start start 2024-05-28T14:01:44.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f874ce00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87480b1b00 0x7f87480b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:44.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f874ce00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8748147b40 0x7f874806b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:44.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f874ce00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87480b5bd0 con 0x7f874806bf30 2024-05-28T14:01:44.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f874ce00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8748145590 con 0x7f8748147b40 2024-05-28T14:01:44.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f874ce00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8748145870 con 0x7f87480b1b00 2024-05-28T14:01:44.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f8746a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8748147b40 0x7f874806b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:44.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f8746a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8748147b40 0x7f874806b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34536/0 (socket says 172.21.15.42:34536) 2024-05-28T14:01:44.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f8746a00640 1 -- 172.21.15.42:0/3647584283 learned_addr learned my addr 172.21.15.42:0/3647584283 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:44.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f8747400640 1 --2- 172.21.15.42:0/3647584283 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87480b1b00 0x7f87480b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:44.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f8746a00640 1 -- 172.21.15.42:0/3647584283 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87480b1b00 msgr2=0x7f87480b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:44.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f8746a00640 1 --2- 172.21.15.42:0/3647584283 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87480b1b00 0x7f87480b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:44.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f8746a00640 1 -- 172.21.15.42:0/3647584283 >> v1:172.21.15.63:6789/0 conn(0x7f874806bf30 legacy=0x7f8748162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:44.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f8746a00640 1 -- 172.21.15.42:0/3647584283 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87480b5e70 con 0x7f8748147b40 2024-05-28T14:01:44.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.044+0000 7f8747400640 1 --2- 172.21.15.42:0/3647584283 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87480b1b00 0x7f87480b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:44.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.048+0000 7f8746a00640 1 --2- 172.21.15.42:0/3647584283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8748147b40 0x7f874806b9f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8734009870 tx=0x7f873401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fcb27f3e643835 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:44.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.048+0000 7f8746000640 1 -- 172.21.15.42:0/3647584283 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87340ab650 con 0x7f8748147b40 2024-05-28T14:01:44.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.048+0000 7f8746000640 1 -- 172.21.15.42:0/3647584283 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87340b0070 con 0x7f8748147b40 2024-05-28T14:01:44.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.048+0000 7f8746000640 1 -- 172.21.15.42:0/3647584283 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87340aba10 con 0x7f8748147b40 2024-05-28T14:01:44.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.048+0000 7f874ce00640 1 -- 172.21.15.42:0/3647584283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8748147b40 msgr2=0x7f874806b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:44.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.048+0000 7f874ce00640 1 --2- 172.21.15.42:0/3647584283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8748147b40 0x7f874806b9f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8734009870 tx=0x7f873401c560 comp rx=0 tx=0).stop 2024-05-28T14:01:44.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.048+0000 7f874ce00640 1 -- 172.21.15.42:0/3647584283 shutdown_connections 2024-05-28T14:01:44.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.048+0000 7f874ce00640 1 --2- 172.21.15.42:0/3647584283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8748147b40 0x7f874806b9f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:44.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.048+0000 7f874ce00640 1 --2- 172.21.15.42:0/3647584283 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87480b1b00 0x7f87480b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:44.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.048+0000 7f874ce00640 1 -- 172.21.15.42:0/3647584283 >> 172.21.15.42:0/3647584283 conn(0x7f87480b67a0 msgr2=0x7f87480b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:44.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.048+0000 7f874ce00640 1 -- 172.21.15.42:0/3647584283 shutdown_connections 2024-05-28T14:01:44.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.048+0000 7f874ce00640 1 -- 172.21.15.42:0/3647584283 wait complete. 2024-05-28T14:01:44.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.048+0000 7f874ce00640 1 Processor -- start 2024-05-28T14:01:44.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.048+0000 7f874ce00640 1 -- start start 2024-05-28T14:01:44.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f874ce00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f874806bf30 0x7f87481419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:44.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f874ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87480b1b00 0x7f8748142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:44.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f874ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8748147b40 0x7f874815c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:44.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f874ce00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f874816b660 con 0x7f8748147b40 2024-05-28T14:01:44.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f874ce00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f874816b4e0 con 0x7f87480b1b00 2024-05-28T14:01:44.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f874ce00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f874816b7e0 con 0x7f874806bf30 2024-05-28T14:01:44.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f8747e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8748147b40 0x7f874815c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:44.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f8746a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87480b1b00 0x7f8748142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:44.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f8747400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f874806bf30 0x7f87481419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:44.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f8746a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87480b1b00 0x7f8748142180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:42432/0 (socket says 172.21.15.42:42432) 2024-05-28T14:01:44.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f8746a00640 1 -- 172.21.15.42:0/692542853 learned_addr learned my addr 172.21.15.42:0/692542853 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:44.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f8746a00640 1 -- 172.21.15.42:0/692542853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f874806bf30 msgr2=0x7f87481419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:44.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f8746a00640 1 --2- 172.21.15.42:0/692542853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f874806bf30 0x7f87481419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:44.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f8746a00640 1 -- 172.21.15.42:0/692542853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8748147b40 msgr2=0x7f874815c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:44.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f8746a00640 1 --2- 172.21.15.42:0/692542853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8748147b40 0x7f874815c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:44.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f8746a00640 1 -- 172.21.15.42:0/692542853 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8748160f90 con 0x7f87480b1b00 2024-05-28T14:01:44.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f8747e00640 1 --2- 172.21.15.42:0/692542853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8748147b40 0x7f874815c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:01:44.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f8746a00640 1 --2- 172.21.15.42:0/692542853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87480b1b00 0x7f8748142180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f87340ad040 tx=0x7f873401ada0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:44.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f873f400640 1 -- 172.21.15.42:0/692542853 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87340be5f0 con 0x7f87480b1b00 2024-05-28T14:01:44.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f873f400640 1 -- 172.21.15.42:0/692542853 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87340be790 con 0x7f87480b1b00 2024-05-28T14:01:44.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f874ce00640 1 -- 172.21.15.42:0/692542853 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8748161270 con 0x7f87480b1b00 2024-05-28T14:01:44.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f873f400640 1 -- 172.21.15.42:0/692542853 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87340beb80 con 0x7f87480b1b00 2024-05-28T14:01:44.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f874ce00640 1 -- 172.21.15.42:0/692542853 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8748141f10 con 0x7f87480b1b00 2024-05-28T14:01:44.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.052+0000 7f874ce00640 1 -- 172.21.15.42:0/692542853 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f874813f880 con 0x7f87480b1b00 2024-05-28T14:01:44.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.056+0000 7f873f400640 1 -- 172.21.15.42:0/692542853 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f87340b43f0 con 0x7f87480b1b00 2024-05-28T14:01:44.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.056+0000 7f873f400640 1 --2- 172.21.15.42:0/692542853 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8728068650 0x7f872806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:44.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.056+0000 7f873f400640 1 -- 172.21.15.42:0/692542853 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4971+0+0 (secure 0 0 0) 0x7f8734121a20 con 0x7f87480b1b00 2024-05-28T14:01:44.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.056+0000 7f8747400640 1 --2- 172.21.15.42:0/692542853 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8728068650 0x7f872806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:44.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.056+0000 7f8747400640 1 --2- 172.21.15.42:0/692542853 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8728068650 0x7f872806ab10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f874813eb60 tx=0x7f8738008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:44.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.060+0000 7f873f400640 1 -- 172.21.15.42:0/692542853 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f874813f880 con 0x7f87480b1b00 2024-05-28T14:01:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:44.284+0000 7f874ce00640 1 -- 172.21.15.42:0/692542853 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones"]} v 0) -- 0x7f874804fcf0 con 0x7f87480b1b00 2024-05-28T14:01:45.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.196+0000 7f873f400640 1 -- 172.21.15.42:0/692542853 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones"]}]=0 v173) ==== 165+0+0 (secure 0 0 0) 0x7f87340c4060 con 0x7f87480b1b00 2024-05-28T14:01:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.196+0000 7f874ce00640 1 -- 172.21.15.42:0/692542853 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8728068650 msgr2=0x7f872806ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.196+0000 7f874ce00640 1 --2- 172.21.15.42:0/692542853 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8728068650 0x7f872806ab10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f874813eb60 tx=0x7f8738008040 comp rx=0 tx=0).stop 2024-05-28T14:01:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.196+0000 7f874ce00640 1 -- 172.21.15.42:0/692542853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87480b1b00 msgr2=0x7f8748142180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.196+0000 7f874ce00640 1 --2- 172.21.15.42:0/692542853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87480b1b00 0x7f8748142180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f87340ad040 tx=0x7f873401ada0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.196+0000 7f874ce00640 1 -- 172.21.15.42:0/692542853 shutdown_connections 2024-05-28T14:01:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.196+0000 7f874ce00640 1 --2- 172.21.15.42:0/692542853 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8728068650 0x7f872806ab10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.196+0000 7f874ce00640 1 --2- 172.21.15.42:0/692542853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8748147b40 0x7f874815c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.196+0000 7f874ce00640 1 --2- 172.21.15.42:0/692542853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87480b1b00 0x7f8748142180 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.196+0000 7f874ce00640 1 --2- 172.21.15.42:0/692542853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f874806bf30 0x7f87481419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.196+0000 7f874ce00640 1 -- 172.21.15.42:0/692542853 >> 172.21.15.42:0/692542853 conn(0x7f87480b67a0 msgr2=0x7f8748070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.196+0000 7f874ce00640 1 -- 172.21.15.42:0/692542853 shutdown_connections 2024-05-28T14:01:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.196+0000 7f874ce00640 1 -- 172.21.15.42:0/692542853 wait complete. 2024-05-28T14:01:45.208 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T14:01:45.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0111000640 1 Processor -- start 2024-05-28T14:01:45.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0111000640 1 -- start start 2024-05-28T14:01:45.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0111000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c071de0 0x7f010c0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:45.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0111000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f010c0ce3b0 0x7f010c0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:45.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0111000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f010c141d10 0x7f010c1421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:45.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0111000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f010c072700 con 0x7f010c0ce3b0 2024-05-28T14:01:45.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0111000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f010c160230 con 0x7f010c141d10 2024-05-28T14:01:45.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0111000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f010c160510 con 0x7f010c071de0 2024-05-28T14:01:45.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0103e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f010c0ce3b0 0x7f010c0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:45.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0103e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f010c0ce3b0 0x7f010c0ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34552/0 (socket says 172.21.15.42:34552) 2024-05-28T14:01:45.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0103e00640 1 -- 172.21.15.42:0/3154899464 learned_addr learned my addr 172.21.15.42:0/3154899464 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:45.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0109e00640 1 --2- 172.21.15.42:0/3154899464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c071de0 0x7f010c0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:45.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f010a800640 1 --2- 172.21.15.42:0/3154899464 >> v2:172.21.15.63:3300/0 conn(0x7f010c141d10 0x7f010c1421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:45.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0103e00640 1 -- 172.21.15.42:0/3154899464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c071de0 msgr2=0x7f010c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:45.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0103e00640 1 --2- 172.21.15.42:0/3154899464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c071de0 0x7f010c0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0103e00640 1 -- 172.21.15.42:0/3154899464 >> v2:172.21.15.63:3300/0 conn(0x7f010c141d10 msgr2=0x7f010c1421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:45.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0103e00640 1 --2- 172.21.15.42:0/3154899464 >> v2:172.21.15.63:3300/0 conn(0x7f010c141d10 0x7f010c1421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.372+0000 7f0103e00640 1 -- 172.21.15.42:0/3154899464 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c1607f0 con 0x7f010c0ce3b0 2024-05-28T14:01:45.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.376+0000 7f0103e00640 1 --2- 172.21.15.42:0/3154899464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f010c0ce3b0 0x7f010c0ce810 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f010c0702b0 tx=0x7f00f401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82c51fb4ac5dd3f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:45.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.376+0000 7f0109400640 1 -- 172.21.15.42:0/3154899464 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f40b0070 con 0x7f010c0ce3b0 2024-05-28T14:01:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.376+0000 7f0109400640 1 -- 172.21.15.42:0/3154899464 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f40a4e50 con 0x7f010c0ce3b0 2024-05-28T14:01:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.376+0000 7f0109400640 1 -- 172.21.15.42:0/3154899464 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f40ab820 con 0x7f010c0ce3b0 2024-05-28T14:01:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.376+0000 7f0111000640 1 -- 172.21.15.42:0/3154899464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f010c0ce3b0 msgr2=0x7f010c0ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.376+0000 7f0111000640 1 --2- 172.21.15.42:0/3154899464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f010c0ce3b0 0x7f010c0ce810 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f010c0702b0 tx=0x7f00f401c400 comp rx=0 tx=0).stop 2024-05-28T14:01:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.376+0000 7f0111000640 1 -- 172.21.15.42:0/3154899464 shutdown_connections 2024-05-28T14:01:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.376+0000 7f0111000640 1 --2- 172.21.15.42:0/3154899464 >> v2:172.21.15.63:3300/0 conn(0x7f010c141d10 0x7f010c1421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.376+0000 7f0111000640 1 --2- 172.21.15.42:0/3154899464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f010c0ce3b0 0x7f010c0ce810 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.376+0000 7f0111000640 1 --2- 172.21.15.42:0/3154899464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c071de0 0x7f010c0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.376+0000 7f0111000640 1 -- 172.21.15.42:0/3154899464 >> 172.21.15.42:0/3154899464 conn(0x7f010c0b21f0 msgr2=0x7f010c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.376+0000 7f0111000640 1 -- 172.21.15.42:0/3154899464 shutdown_connections 2024-05-28T14:01:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.376+0000 7f0111000640 1 -- 172.21.15.42:0/3154899464 wait complete. 2024-05-28T14:01:45.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0111000640 1 Processor -- start 2024-05-28T14:01:45.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0111000640 1 -- start start 2024-05-28T14:01:45.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0111000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c071de0 0x7f010c06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:45.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0111000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f010c0ce3b0 0x7f010c146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:45.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0111000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f010c141d10 0x7f010c149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:45.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0111000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f010c140520 con 0x7f010c0ce3b0 2024-05-28T14:01:45.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0111000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f010c13f8c0 con 0x7f010c141d10 2024-05-28T14:01:45.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0111000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f010c1406a0 con 0x7f010c071de0 2024-05-28T14:01:45.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0103e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f010c0ce3b0 0x7f010c146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:45.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0103e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f010c0ce3b0 0x7f010c146000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34554/0 (socket says 172.21.15.42:34554) 2024-05-28T14:01:45.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0103e00640 1 -- 172.21.15.42:0/4222460397 learned_addr learned my addr 172.21.15.42:0/4222460397 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:45.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0109e00640 1 --2- 172.21.15.42:0/4222460397 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c071de0 0x7f010c06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:45.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f010a800640 1 --2- 172.21.15.42:0/4222460397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f010c141d10 0x7f010c149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:45.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0103e00640 1 -- 172.21.15.42:0/4222460397 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c071de0 msgr2=0x7f010c06e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:45.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0103e00640 1 --2- 172.21.15.42:0/4222460397 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c071de0 0x7f010c06e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0103e00640 1 -- 172.21.15.42:0/4222460397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f010c141d10 msgr2=0x7f010c149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:45.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0103e00640 1 --2- 172.21.15.42:0/4222460397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f010c141d10 0x7f010c149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0103e00640 1 -- 172.21.15.42:0/4222460397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c06ef30 con 0x7f010c0ce3b0 2024-05-28T14:01:45.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0103e00640 1 --2- 172.21.15.42:0/4222460397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f010c0ce3b0 0x7f010c146000 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f00f40ae040 tx=0x7f00f40191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:45.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0102000640 1 -- 172.21.15.42:0/4222460397 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f40b0040 con 0x7f010c0ce3b0 2024-05-28T14:01:45.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0102000640 1 -- 172.21.15.42:0/4222460397 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f40b9070 con 0x7f010c0ce3b0 2024-05-28T14:01:45.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0102000640 1 -- 172.21.15.42:0/4222460397 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f40b4440 con 0x7f010c0ce3b0 2024-05-28T14:01:45.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.380+0000 7f0111000640 1 -- 172.21.15.42:0/4222460397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010c06f210 con 0x7f010c0ce3b0 2024-05-28T14:01:45.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.384+0000 7f0111000640 1 -- 172.21.15.42:0/4222460397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f010c13c290 con 0x7f010c0ce3b0 2024-05-28T14:01:45.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.384+0000 7f0111000640 1 -- 172.21.15.42:0/4222460397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010c0738b0 con 0x7f010c0ce3b0 2024-05-28T14:01:45.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.384+0000 7f0102000640 1 -- 172.21.15.42:0/4222460397 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f00f40b45e0 con 0x7f010c0ce3b0 2024-05-28T14:01:45.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.384+0000 7f0102000640 1 --2- 172.21.15.42:0/4222460397 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00e8068650 0x7f00e806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:45.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.384+0000 7f0102000640 1 -- 172.21.15.42:0/4222460397 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4971+0+0 (secure 0 0 0) 0x7f00f4120770 con 0x7f010c0ce3b0 2024-05-28T14:01:45.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.384+0000 7f0109e00640 1 --2- 172.21.15.42:0/4222460397 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00e8068650 0x7f00e806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:45.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.384+0000 7f0109e00640 1 --2- 172.21.15.42:0/4222460397 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00e8068650 0x7f00e806ab10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0045f0 tx=0x7f00fc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:45.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.388+0000 7f0102000640 1 -- 172.21.15.42:0/4222460397 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f00f4016110 con 0x7f010c0ce3b0 2024-05-28T14:01:45.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.556+0000 7f0111000640 1 -- 172.21.15.42:0/4222460397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f010c04fdd0 con 0x7f010c0ce3b0 2024-05-28T14:01:45.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.560+0000 7f0102000640 1 -- 172.21.15.42:0/4222460397 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f00f40f0b70 con 0x7f010c0ce3b0 2024-05-28T14:01:45.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.560+0000 7f0111000640 1 -- 172.21.15.42:0/4222460397 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00e8068650 msgr2=0x7f00e806ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:45.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.560+0000 7f0111000640 1 --2- 172.21.15.42:0/4222460397 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00e8068650 0x7f00e806ab10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0045f0 tx=0x7f00fc008040 comp rx=0 tx=0).stop 2024-05-28T14:01:45.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.560+0000 7f0111000640 1 -- 172.21.15.42:0/4222460397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f010c0ce3b0 msgr2=0x7f010c146000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:45.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.564+0000 7f0111000640 1 --2- 172.21.15.42:0/4222460397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f010c0ce3b0 0x7f010c146000 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f00f40ae040 tx=0x7f00f40191a0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.564+0000 7f0111000640 1 -- 172.21.15.42:0/4222460397 shutdown_connections 2024-05-28T14:01:45.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.564+0000 7f0111000640 1 --2- 172.21.15.42:0/4222460397 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00e8068650 0x7f00e806ab10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.564+0000 7f0111000640 1 --2- 172.21.15.42:0/4222460397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f010c141d10 0x7f010c149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.564+0000 7f0111000640 1 --2- 172.21.15.42:0/4222460397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f010c0ce3b0 0x7f010c146000 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.564+0000 7f0111000640 1 --2- 172.21.15.42:0/4222460397 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c071de0 0x7f010c06e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:45.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.564+0000 7f0111000640 1 -- 172.21.15.42:0/4222460397 >> 172.21.15.42:0/4222460397 conn(0x7f010c0b21f0 msgr2=0x7f010c0b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:45.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.564+0000 7f0111000640 1 -- 172.21.15.42:0/4222460397 shutdown_connections 2024-05-28T14:01:45.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:45.564+0000 7f0111000640 1 -- 172.21.15.42:0/4222460397 wait complete. 2024-05-28T14:01:45.568 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T14:01:45.568 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:01:45.654 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:01:45.761 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T14:01:45.761 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:01:45.761 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:01:45.770 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:01:45.770 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:01:45.770 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:01:45.780 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:01:45.780 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:01:45.780 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:01:45.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:01:45.820 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:01:45.820 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:01:45.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:01:45.831 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-28T14:01:45.831 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:01:46.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.020+0000 7f7935400640 1 Processor -- start 2024-05-28T14:01:46.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.020+0000 7f7935400640 1 -- start start 2024-05-28T14:01:46.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.024+0000 7f7935400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7930162b30 0x7f79300d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:46.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.024+0000 7f7935400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7930072ab0 0x7f79300ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:46.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.024+0000 7f7935400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79300730d0 con 0x7f79300d0c30 2024-05-28T14:01:46.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.024+0000 7f7935400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f79300733b0 con 0x7f7930162b30 2024-05-28T14:01:46.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.024+0000 7f7935400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7930073690 con 0x7f7930072ab0 2024-05-28T14:01:46.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.024+0000 7f792e800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7930162b30 0x7f79300d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:46.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.024+0000 7f7925e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f79300d0c30 0x7f79300723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51596/0 (socket says 172.21.15.42:51596) 2024-05-28T14:01:46.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.024+0000 7f7925e00640 1 -- 172.21.15.42:0/1565833828 learned_addr learned my addr 172.21.15.42:0/1565833828 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:46.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.024+0000 7f792f200640 1 --2- 172.21.15.42:0/1565833828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7930072ab0 0x7f79300ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:46.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.024+0000 7f792e800640 1 -- 172.21.15.42:0/1565833828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7930072ab0 msgr2=0x7f79300ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:46.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.024+0000 7f792e800640 1 --2- 172.21.15.42:0/1565833828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7930072ab0 0x7f79300ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.024+0000 7f792e800640 1 -- 172.21.15.42:0/1565833828 >> v1:172.21.15.42:6789/0 conn(0x7f79300d0c30 legacy=0x7f79300723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:46.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.024+0000 7f792e800640 1 -- 172.21.15.42:0/1565833828 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7930073970 con 0x7f7930162b30 2024-05-28T14:01:46.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.024+0000 7f792de00640 1 -- 172.21.15.42:0/1565833828 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 754884967 0 0) 0x7f79300730d0 con 0x7f79300d0c30 2024-05-28T14:01:46.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.028+0000 7f792e800640 1 --2- 172.21.15.42:0/1565833828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7930162b30 0x7f79300d06f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f79200097c0 tx=0x7f792001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d17f24549e6087f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:46.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.028+0000 7f792de00640 1 -- 172.21.15.42:0/1565833828 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79200b0070 con 0x7f7930162b30 2024-05-28T14:01:46.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.028+0000 7f792de00640 1 -- 172.21.15.42:0/1565833828 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79200ab600 con 0x7f7930162b30 2024-05-28T14:01:46.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.028+0000 7f792de00640 1 -- 172.21.15.42:0/1565833828 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79200ab9d0 con 0x7f7930162b30 2024-05-28T14:01:46.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.028+0000 7f7935400640 1 -- 172.21.15.42:0/1565833828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7930162b30 msgr2=0x7f79300d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:46.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.028+0000 7f7935400640 1 --2- 172.21.15.42:0/1565833828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7930162b30 0x7f79300d06f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f79200097c0 tx=0x7f792001c4a0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.028+0000 7f7935400640 1 -- 172.21.15.42:0/1565833828 shutdown_connections 2024-05-28T14:01:46.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.028+0000 7f7935400640 1 --2- 172.21.15.42:0/1565833828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7930072ab0 0x7f79300ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.028+0000 7f7935400640 1 --2- 172.21.15.42:0/1565833828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7930162b30 0x7f79300d06f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.028+0000 7f7935400640 1 -- 172.21.15.42:0/1565833828 >> 172.21.15.42:0/1565833828 conn(0x7f793006bdf0 msgr2=0x7f793006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:46.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.028+0000 7f7935400640 1 -- 172.21.15.42:0/1565833828 shutdown_connections 2024-05-28T14:01:46.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.028+0000 7f7935400640 1 -- 172.21.15.42:0/1565833828 wait complete. 2024-05-28T14:01:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.028+0000 7f7935400640 1 Processor -- start 2024-05-28T14:01:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.028+0000 7f7935400640 1 -- start start 2024-05-28T14:01:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7935400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7930072ab0 0x7f7930150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7935400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79300d0c30 0x7f7930158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7935400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7930162b30 0x7f7930158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7935400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f79300ae310 con 0x7f79300d0c30 2024-05-28T14:01:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7935400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79300ae190 con 0x7f7930162b30 2024-05-28T14:01:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7935400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f79300ae490 con 0x7f7930072ab0 2024-05-28T14:01:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7925e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79300d0c30 0x7f7930158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7925e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79300d0c30 0x7f7930158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34586/0 (socket says 172.21.15.42:34586) 2024-05-28T14:01:46.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7925e00640 1 -- 172.21.15.42:0/3384842916 learned_addr learned my addr 172.21.15.42:0/3384842916 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:46.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f792e800640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7930072ab0 0x7f7930150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:46.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f792f200640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7930162b30 0x7f7930158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7925e00640 1 -- 172.21.15.42:0/3384842916 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7930072ab0 msgr2=0x7f7930150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7925e00640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7930072ab0 0x7f7930150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7925e00640 1 -- 172.21.15.42:0/3384842916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7930162b30 msgr2=0x7f7930158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7925e00640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7930162b30 0x7f7930158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7925e00640 1 -- 172.21.15.42:0/3384842916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793014ebd0 con 0x7f79300d0c30 2024-05-28T14:01:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f792e800640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7930072ab0 0x7f7930150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f792f200640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7930162b30 0x7f7930158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7925e00640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79300d0c30 0x7f7930158120 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f791800c520 tx=0x7f791800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7927400640 1 -- 172.21.15.42:0/3384842916 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918018070 con 0x7f79300d0c30 2024-05-28T14:01:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7927400640 1 -- 172.21.15.42:0/3384842916 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79180074e0 con 0x7f79300d0c30 2024-05-28T14:01:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7927400640 1 -- 172.21.15.42:0/3384842916 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79180049f0 con 0x7f79300d0c30 2024-05-28T14:01:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7935400640 1 -- 172.21.15.42:0/3384842916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793014eeb0 con 0x7f79300d0c30 2024-05-28T14:01:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7935400640 1 -- 172.21.15.42:0/3384842916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f793014f470 con 0x7f79300d0c30 2024-05-28T14:01:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7935400640 1 -- 172.21.15.42:0/3384842916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7930068a90 con 0x7f79300d0c30 2024-05-28T14:01:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.032+0000 7f7927400640 1 -- 172.21.15.42:0/3384842916 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f7918004b90 con 0x7f79300d0c30 2024-05-28T14:01:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.036+0000 7f7927400640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f790c068650 0x7f790c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:46.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.036+0000 7f7927400640 1 -- 172.21.15.42:0/3384842916 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4971+0+0 (secure 0 0 0) 0x7f7918085e10 con 0x7f79300d0c30 2024-05-28T14:01:46.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.036+0000 7f792e800640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f790c068650 0x7f790c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:46.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.036+0000 7f792e800640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f790c068650 0x7f790c06ab10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f79200ad000 tx=0x7f79200a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:46.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.040+0000 7f7927400640 1 -- 172.21.15.42:0/3384842916 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7918089030 con 0x7f79300d0c30 2024-05-28T14:01:46.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.240+0000 7f7935400640 1 -- 172.21.15.42:0/3384842916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7930161430 con 0x7f79300d0c30 2024-05-28T14:01:46.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.240+0000 7f7927400640 1 -- 172.21.15.42:0/3384842916 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3871 (secure 0 0 0) 0x7f7918055c80 con 0x7f79300d0c30 2024-05-28T14:01:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:46.228 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2044753493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2044753493},{"type":"v1","addr":"172.21.15.42:6835","nonce":2044753493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3332210819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3332210819},{"type":"v1","addr":"172.21.15.42:6837","nonce":3332210819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":54782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1050322479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1050322479},{"type":"v1","addr":"172.21.15.63:6834","nonce":1050322479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":55124,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1825251196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1825251196},{"type":"v1","addr":"172.21.15.63:6835","nonce":1825251196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-05-28T14:01:46.228 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 20 2024-05-28T14:01:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.244+0000 7f7935400640 1 -- 172.21.15.42:0/3384842916 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f790c068650 msgr2=0x7f790c06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.244+0000 7f7935400640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f790c068650 0x7f790c06ab10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f79200ad000 tx=0x7f79200a6000 comp rx=0 tx=0).stop 2024-05-28T14:01:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.244+0000 7f7935400640 1 -- 172.21.15.42:0/3384842916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79300d0c30 msgr2=0x7f7930158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.244+0000 7f7935400640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79300d0c30 0x7f7930158120 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f791800c520 tx=0x7f791800c9e0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.244+0000 7f7935400640 1 -- 172.21.15.42:0/3384842916 shutdown_connections 2024-05-28T14:01:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.244+0000 7f7935400640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f790c068650 0x7f790c06ab10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.244+0000 7f7935400640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7930162b30 0x7f7930158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.244+0000 7f7935400640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79300d0c30 0x7f7930158120 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.244+0000 7f7935400640 1 --2- 172.21.15.42:0/3384842916 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7930072ab0 0x7f7930150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.244+0000 7f7935400640 1 -- 172.21.15.42:0/3384842916 >> 172.21.15.42:0/3384842916 conn(0x7f793006bdf0 msgr2=0x7f79300b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:46.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.244+0000 7f7935400640 1 -- 172.21.15.42:0/3384842916 shutdown_connections 2024-05-28T14:01:46.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.244+0000 7f7935400640 1 -- 172.21.15.42:0/3384842916 wait complete. 2024-05-28T14:01:46.249 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-28T14:01:46.249 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-28T14:01:46.250 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-28T14:01:46.250 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:01:46.250 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-28T14:01:46.250 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-28T14:01:46.250 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-28T14:01:46.250 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:01:46.250 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-28T14:01:46.251 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-28T14:01:46.251 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-28T14:01:46.251 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:01:46.251 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-28T14:01:46.251 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-28T14:01:46.251 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-28T14:01:46.252 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:01:46.264 INFO:tasks.ceph.mds.a:Stopped 2024-05-28T14:01:46.264 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-28T14:01:46.265 INFO:tasks.ceph.mds.c:Stopped 2024-05-28T14:01:46.265 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-28T14:01:46.267 INFO:tasks.ceph.mds.b:Stopped 2024-05-28T14:01:46.267 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-28T14:01:46.268 INFO:tasks.ceph.mds.d:Stopped 2024-05-28T14:01:46.268 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-28T14:01:46.314 INFO:tasks.ceph.mds.a:Started 2024-05-28T14:01:46.314 DEBUG:teuthology.parallel:result is None 2024-05-28T14:01:46.315 INFO:tasks.ceph.mds.c:Started 2024-05-28T14:01:46.316 DEBUG:teuthology.parallel:result is None 2024-05-28T14:01:46.317 INFO:tasks.ceph.mds.b:Started 2024-05-28T14:01:46.317 DEBUG:teuthology.parallel:result is None 2024-05-28T14:01:46.319 INFO:tasks.ceph.mds.d:Started 2024-05-28T14:01:46.319 DEBUG:teuthology.parallel:result is None 2024-05-28T14:01:46.320 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-28T14:01:46.401 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-05-28T14:01:46.401 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-05-28T14:01:46.411 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T14:01:46.413 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-28T14:01:46.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f48ad400640 1 Processor -- start 2024-05-28T14:01:46.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f48ad400640 1 -- start start 2024-05-28T14:01:46.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f48ad400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48a806c100 0x7f48a80b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:46.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f48ad400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f48a80ce2f0 0x7f48a80ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:46.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f48ad400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48a815cdf0 con 0x7f48a815ca10 2024-05-28T14:01:46.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f48ad400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f48a80b2eb0 con 0x7f48a80ce2f0 2024-05-28T14:01:46.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f48ad400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f48a80b18d0 con 0x7f48a806c100 2024-05-28T14:01:46.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f48a6800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f48a815ca10 0x7f48a806b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51620/0 (socket says 172.21.15.42:51620) 2024-05-28T14:01:46.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f48a6800640 1 -- 172.21.15.42:0/2507220568 learned_addr learned my addr 172.21.15.42:0/2507220568 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:46.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f489de00640 1 --2- 172.21.15.42:0/2507220568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48a806c100 0x7f48a80b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:46.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f48a7200640 1 --2- 172.21.15.42:0/2507220568 >> v2:172.21.15.63:3300/0 conn(0x7f48a80ce2f0 0x7f48a80ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:46.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f48a5e00640 1 -- 172.21.15.42:0/2507220568 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 882157980 0 0) 0x7f48a815cdf0 con 0x7f48a815ca10 2024-05-28T14:01:46.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f48a5e00640 1 -- 172.21.15.42:0/2507220568 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f488c003530 con 0x7f48a815ca10 2024-05-28T14:01:46.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f489de00640 1 -- 172.21.15.42:0/2507220568 >> v2:172.21.15.63:3300/0 conn(0x7f48a80ce2f0 msgr2=0x7f48a80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:46.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f489de00640 1 --2- 172.21.15.42:0/2507220568 >> v2:172.21.15.63:3300/0 conn(0x7f48a80ce2f0 0x7f48a80ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f489de00640 1 -- 172.21.15.42:0/2507220568 >> v1:172.21.15.42:6789/0 conn(0x7f48a815ca10 legacy=0x7f48a806b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:46.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f489de00640 1 -- 172.21.15.42:0/2507220568 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a80b1bb0 con 0x7f48a806c100 2024-05-28T14:01:46.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.480+0000 7f48a7200640 1 --2- 172.21.15.42:0/2507220568 >> v2:172.21.15.63:3300/0 conn(0x7f48a80ce2f0 0x7f48a80ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:01:46.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.484+0000 7f489de00640 1 --2- 172.21.15.42:0/2507220568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48a806c100 0x7f48a80b2970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f48900097c0 tx=0x7f489001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98e7b91ac6b7803 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.484+0000 7f48a5e00640 1 -- 172.21.15.42:0/2507220568 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48900a7070 con 0x7f48a806c100 2024-05-28T14:01:46.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.484+0000 7f48a5e00640 1 -- 172.21.15.42:0/2507220568 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48900b0070 con 0x7f48a806c100 2024-05-28T14:01:46.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.484+0000 7f48a5e00640 1 -- 172.21.15.42:0/2507220568 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48900ab890 con 0x7f48a806c100 2024-05-28T14:01:46.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.484+0000 7f48ad400640 1 -- 172.21.15.42:0/2507220568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48a806c100 msgr2=0x7f48a80b2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:46.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.484+0000 7f48ad400640 1 --2- 172.21.15.42:0/2507220568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48a806c100 0x7f48a80b2970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f48900097c0 tx=0x7f489001c470 comp rx=0 tx=0).stop 2024-05-28T14:01:46.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.484+0000 7f48ad400640 1 -- 172.21.15.42:0/2507220568 shutdown_connections 2024-05-28T14:01:46.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.484+0000 7f48ad400640 1 --2- 172.21.15.42:0/2507220568 >> v2:172.21.15.63:3300/0 conn(0x7f48a80ce2f0 0x7f48a80ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.484+0000 7f48ad400640 1 --2- 172.21.15.42:0/2507220568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48a806c100 0x7f48a80b2970 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.484+0000 7f48ad400640 1 -- 172.21.15.42:0/2507220568 >> 172.21.15.42:0/2507220568 conn(0x7f48a809aa50 msgr2=0x7f48a80b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:46.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.484+0000 7f48ad400640 1 -- 172.21.15.42:0/2507220568 shutdown_connections 2024-05-28T14:01:46.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.484+0000 7f48ad400640 1 -- 172.21.15.42:0/2507220568 wait complete. 2024-05-28T14:01:46.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48ad400640 1 Processor -- start 2024-05-28T14:01:46.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48ad400640 1 -- start start 2024-05-28T14:01:46.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48ad400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48a806c100 0x7f48a80c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:46.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48ad400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a80ce2f0 0x7f48a80c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:46.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48ad400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a815ca10 0x7f48a80c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:46.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48ad400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f48a80aeae0 con 0x7f48a815ca10 2024-05-28T14:01:46.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48ad400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48a80cecc0 con 0x7f48a80ce2f0 2024-05-28T14:01:46.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48ad400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f48a80aec60 con 0x7f48a806c100 2024-05-28T14:01:46.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48a7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a815ca10 0x7f48a80c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:46.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48a7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a815ca10 0x7f48a80c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34618/0 (socket says 172.21.15.42:34618) 2024-05-28T14:01:46.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f489de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a80ce2f0 0x7f48a80c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:46.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48a7200640 1 -- 172.21.15.42:0/1861780503 learned_addr learned my addr 172.21.15.42:0/1861780503 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:46.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48a6800640 1 --2- 172.21.15.42:0/1861780503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48a806c100 0x7f48a80c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:46.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f489de00640 1 -- 172.21.15.42:0/1861780503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48a806c100 msgr2=0x7f48a80c3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:46.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f489de00640 1 --2- 172.21.15.42:0/1861780503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48a806c100 0x7f48a80c3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f489de00640 1 -- 172.21.15.42:0/1861780503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a815ca10 msgr2=0x7f48a80c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:46.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f489de00640 1 --2- 172.21.15.42:0/1861780503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a815ca10 0x7f48a80c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:46.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f489de00640 1 -- 172.21.15.42:0/1861780503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a80c4200 con 0x7f48a80ce2f0 2024-05-28T14:01:46.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48a7200640 1 --2- 172.21.15.42:0/1861780503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a815ca10 0x7f48a80c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:01:46.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f489de00640 1 --2- 172.21.15.42:0/1861780503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a80ce2f0 0x7f48a80c7540 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f48900ac000 tx=0x7f489001acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:46.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f489f400640 1 -- 172.21.15.42:0/1861780503 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48900bf610 con 0x7f48a80ce2f0 2024-05-28T14:01:46.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f489f400640 1 -- 172.21.15.42:0/1861780503 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48900bf7f0 con 0x7f48a80ce2f0 2024-05-28T14:01:46.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f489f400640 1 -- 172.21.15.42:0/1861780503 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48900bfbe0 con 0x7f48a80ce2f0 2024-05-28T14:01:46.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48ad400640 1 -- 172.21.15.42:0/1861780503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a80c9b90 con 0x7f48a80ce2f0 2024-05-28T14:01:46.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48ad400640 1 -- 172.21.15.42:0/1861780503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a80c50a0 con 0x7f48a80ce2f0 2024-05-28T14:01:46.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.488+0000 7f48ad400640 1 -- 172.21.15.42:0/1861780503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a809f650 con 0x7f48a80ce2f0 2024-05-28T14:01:46.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.492+0000 7f489f400640 1 -- 172.21.15.42:0/1861780503 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f48900bd020 con 0x7f48a80ce2f0 2024-05-28T14:01:46.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.492+0000 7f489f400640 1 --2- 172.21.15.42:0/1861780503 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4884068650 0x7f488406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:46.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.492+0000 7f489f400640 1 -- 172.21.15.42:0/1861780503 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4971+0+0 (secure 0 0 0) 0x7f4890120e90 con 0x7f48a80ce2f0 2024-05-28T14:01:46.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.492+0000 7f48a6800640 1 --2- 172.21.15.42:0/1861780503 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4884068650 0x7f488406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:46.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.492+0000 7f48a6800640 1 --2- 172.21.15.42:0/1861780503 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4884068650 0x7f488406ab10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4898007fd0 tx=0x7f4898009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:46.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.496+0000 7f489f400640 1 -- 172.21.15.42:0/1861780503 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f48900c6030 con 0x7f48a80ce2f0 2024-05-28T14:01:46.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:46.672+0000 7f48ad400640 1 -- 172.21.15.42:0/1861780503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f48a804fdd0 con 0x7f48a80ce2f0 2024-05-28T14:01:47.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.212+0000 7f489f400640 1 -- 172.21.15.42:0/1861780503 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) ==== 97+0+0 (secure 0 0 0) 0x7f4890016090 con 0x7f48a80ce2f0 2024-05-28T14:01:47.199 INFO:teuthology.orchestra.run.smithi042.stderr: removed all blocklist entries 2024-05-28T14:01:47.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.216+0000 7f48ad400640 1 -- 172.21.15.42:0/1861780503 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4884068650 msgr2=0x7f488406ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.216+0000 7f48ad400640 1 --2- 172.21.15.42:0/1861780503 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4884068650 0x7f488406ab10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4898007fd0 tx=0x7f4898009040 comp rx=0 tx=0).stop 2024-05-28T14:01:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.216+0000 7f48ad400640 1 -- 172.21.15.42:0/1861780503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a80ce2f0 msgr2=0x7f48a80c7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.216+0000 7f48ad400640 1 --2- 172.21.15.42:0/1861780503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a80ce2f0 0x7f48a80c7540 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f48900ac000 tx=0x7f489001acb0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.216+0000 7f48ad400640 1 -- 172.21.15.42:0/1861780503 shutdown_connections 2024-05-28T14:01:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.216+0000 7f48ad400640 1 --2- 172.21.15.42:0/1861780503 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4884068650 0x7f488406ab10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.216+0000 7f48ad400640 1 --2- 172.21.15.42:0/1861780503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a815ca10 0x7f48a80c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.216+0000 7f48ad400640 1 --2- 172.21.15.42:0/1861780503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a80ce2f0 0x7f48a80c7540 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.216+0000 7f48ad400640 1 --2- 172.21.15.42:0/1861780503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48a806c100 0x7f48a80c3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.216+0000 7f48ad400640 1 -- 172.21.15.42:0/1861780503 >> 172.21.15.42:0/1861780503 conn(0x7f48a809aa50 msgr2=0x7f48a806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.216+0000 7f48ad400640 1 -- 172.21.15.42:0/1861780503 shutdown_connections 2024-05-28T14:01:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.216+0000 7f48ad400640 1 -- 172.21.15.42:0/1861780503 wait complete. 2024-05-28T14:01:47.227 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-28T14:01:47.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.388+0000 7f84a9200640 1 Processor -- start 2024-05-28T14:01:47.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.388+0000 7f84a9200640 1 -- start start 2024-05-28T14:01:47.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.388+0000 7f84a9200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f84a409bbd0 0x7f84a409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:47.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.388+0000 7f84a9200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84a40d0870 con 0x7f84a409af90 2024-05-28T14:01:47.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.388+0000 7f84a9200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84a40cd300 con 0x7f84a415c990 2024-05-28T14:01:47.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.388+0000 7f84a9200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f84a409b400 con 0x7f84a409bbd0 2024-05-28T14:01:47.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.388+0000 7f84a2800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f84a409af90 0x7f84a40b53e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36722/0 (socket says 172.21.15.42:36722) 2024-05-28T14:01:47.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.388+0000 7f84a2800640 1 -- 172.21.15.42:0/376239549 learned_addr learned my addr 172.21.15.42:0/376239549 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:47.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.388+0000 7f849be00640 1 --2- 172.21.15.42:0/376239549 >> v2:172.21.15.63:3300/0 conn(0x7f84a409bbd0 0x7f84a409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:47.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.388+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 19850513 0 0) 0x7f84a40d0870 con 0x7f84a409af90 2024-05-28T14:01:47.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.388+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8488003530 con 0x7f84a409af90 2024-05-28T14:01:47.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.388+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 508837441 0 0) 0x7f84a40cd300 con 0x7f84a415c990 2024-05-28T14:01:47.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.388+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84a40d0870 con 0x7f84a415c990 2024-05-28T14:01:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.388+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2413739681 0 0) 0x7f8488003530 con 0x7f84a409af90 2024-05-28T14:01:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 >> v2:172.21.15.63:3300/0 conn(0x7f84a409bbd0 msgr2=0x7f84a409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 --2- 172.21.15.42:0/376239549 >> v2:172.21.15.63:3300/0 conn(0x7f84a409bbd0 0x7f84a409aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 >> v1:172.21.15.63:6789/0 conn(0x7f84a415c990 legacy=0x7f84a40d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a40cd5e0 con 0x7f84a409af90 2024-05-28T14:01:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f849c001a00 con 0x7f84a409af90 2024-05-28T14:01:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 >> v1:172.21.15.42:6789/0 conn(0x7f84a409af90 legacy=0x7f84a40b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 --2- 172.21.15.42:0/376239549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8488003ba0 0x7f8488003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 --2- 172.21.15.42:0/376239549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8488004810 0x7f8488004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 --2- 172.21.15.42:0/376239549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84880051d0 0x7f848800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f84a409b400 con 0x7f8488004810 2024-05-28T14:01:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f848800c0a0 con 0x7f84880051d0 2024-05-28T14:01:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f848800c2a0 con 0x7f8488003ba0 2024-05-28T14:01:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f849be00640 1 --2- 172.21.15.42:0/376239549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8488004810 0x7f8488004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1e00640 1 --2- 172.21.15.42:0/376239549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8488003ba0 0x7f8488003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a2800640 1 --2- 172.21.15.42:0/376239549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84880051d0 0x7f848800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f849be00640 1 -- 172.21.15.42:0/376239549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8488003ba0 msgr2=0x7f8488003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f849be00640 1 --2- 172.21.15.42:0/376239549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8488003ba0 0x7f8488003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f849be00640 1 -- 172.21.15.42:0/376239549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84880051d0 msgr2=0x7f848800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f849be00640 1 --2- 172.21.15.42:0/376239549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84880051d0 0x7f848800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f849be00640 1 -- 172.21.15.42:0/376239549 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8488003530 con 0x7f8488004810 2024-05-28T14:01:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a2800640 1 -- 172.21.15.42:0/376239549 reap_dead start 2024-05-28T14:01:47.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f849be00640 1 --2- 172.21.15.42:0/376239549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8488004810 0x7f8488004c90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f848c009ae0 tx=0x7f848c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0ace3998a80aa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:47.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c0b2070 con 0x7f8488004810 2024-05-28T14:01:47.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848c0a5660 con 0x7f8488004810 2024-05-28T14:01:47.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.392+0000 7f84a1400640 1 -- 172.21.15.42:0/376239549 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c0096b0 con 0x7f8488004810 2024-05-28T14:01:47.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 -- 172.21.15.42:0/376239549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8488004810 msgr2=0x7f8488004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 --2- 172.21.15.42:0/376239549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8488004810 0x7f8488004c90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f848c009ae0 tx=0x7f848c01ca70 comp rx=0 tx=0).stop 2024-05-28T14:01:47.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 -- 172.21.15.42:0/376239549 shutdown_connections 2024-05-28T14:01:47.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 --2- 172.21.15.42:0/376239549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8488004810 0x7f8488004c90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 -- 172.21.15.42:0/376239549 >> 172.21.15.42:0/376239549 conn(0x7f84a40b4080 msgr2=0x7f84a40b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:47.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 -- 172.21.15.42:0/376239549 shutdown_connections 2024-05-28T14:01:47.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 -- 172.21.15.42:0/376239549 wait complete. 2024-05-28T14:01:47.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 Processor -- start 2024-05-28T14:01:47.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 -- start start 2024-05-28T14:01:47.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8488004810 0x7f84a415f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:47.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84a4154f20 0x7f84a415ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:47.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40cae30 0x7f84a4070e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:47.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f84a40cdba0 con 0x7f84a40cae30 2024-05-28T14:01:47.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f84a40cde80 con 0x7f84a4154f20 2024-05-28T14:01:47.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a9200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f84a40ce3a0 con 0x7f8488004810 2024-05-28T14:01:47.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40cae30 0x7f84a4070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:47.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40cae30 0x7f84a4070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54976/0 (socket says 172.21.15.42:54976) 2024-05-28T14:01:47.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a2800640 1 -- 172.21.15.42:0/238428794 learned_addr learned my addr 172.21.15.42:0/238428794 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:47.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a1e00640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8488004810 0x7f84a415f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:47.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f849be00640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84a4154f20 0x7f84a415ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:47.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a2800640 1 -- 172.21.15.42:0/238428794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8488004810 msgr2=0x7f84a415f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a2800640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8488004810 0x7f84a415f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a2800640 1 -- 172.21.15.42:0/238428794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84a4154f20 msgr2=0x7f84a415ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a2800640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84a4154f20 0x7f84a415ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a2800640 1 -- 172.21.15.42:0/238428794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a415fc60 con 0x7f84a40cae30 2024-05-28T14:01:47.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f84a1e00640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8488004810 0x7f84a415f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:47.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.396+0000 7f849be00640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84a4154f20 0x7f84a415ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:47.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.400+0000 7f84a2800640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40cae30 0x7f84a4070e80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f84a406fbc0 tx=0x7f849c007050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:47.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.400+0000 7f849a000640 1 -- 172.21.15.42:0/238428794 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c0078a0 con 0x7f84a40cae30 2024-05-28T14:01:47.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.400+0000 7f849a000640 1 -- 172.21.15.42:0/238428794 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f849c007a60 con 0x7f84a40cae30 2024-05-28T14:01:47.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.400+0000 7f849a000640 1 -- 172.21.15.42:0/238428794 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c01a660 con 0x7f84a40cae30 2024-05-28T14:01:47.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.400+0000 7f84a9200640 1 -- 172.21.15.42:0/238428794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a40ca350 con 0x7f84a40cae30 2024-05-28T14:01:47.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.400+0000 7f84a9200640 1 -- 172.21.15.42:0/238428794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a40ca910 con 0x7f84a40cae30 2024-05-28T14:01:47.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.400+0000 7f84a9200640 1 -- 172.21.15.42:0/238428794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a4068a90 con 0x7f84a40cae30 2024-05-28T14:01:47.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.400+0000 7f849a000640 1 -- 172.21.15.42:0/238428794 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f849c0193c0 con 0x7f84a40cae30 2024-05-28T14:01:47.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.400+0000 7f849a000640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8480068680 0x7f848006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:47.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.400+0000 7f849a000640 1 -- 172.21.15.42:0/238428794 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4885+0+0 (secure 0 0 0) 0x7f849c08a420 con 0x7f84a40cae30 2024-05-28T14:01:47.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.400+0000 7f84a1e00640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8480068680 0x7f848006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:47.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.404+0000 7f84a1e00640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8480068680 0x7f848006ab40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8494007af0 tx=0x7f8494007a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:47.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.404+0000 7f849a000640 1 -- 172.21.15.42:0/238428794 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f849c05a790 con 0x7f84a40cae30 2024-05-28T14:01:47.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.624+0000 7f84a9200640 1 -- 172.21.15.42:0/238428794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f84a40b4ce0 con 0x7f84a40cae30 2024-05-28T14:01:47.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.624+0000 7f849a000640 1 -- 172.21.15.42:0/238428794 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f849c05a090 con 0x7f84a40cae30 2024-05-28T14:01:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:47.612 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:01:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_dump": [ 2024-05-28T14:01:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.a", 2024-05-28T14:01:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmE+3RLRAADuTKGfKeVafGeLpgr9WKUg==", 2024-05-28T14:01:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:01:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:01:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:01:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.b", 2024-05-28T14:01:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVm1rmVOhAAV46KzDJnRfDZYIZanS0g2g==", 2024-05-28T14:01:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:01:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:01:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:01:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.c", 2024-05-28T14:01:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVme4aYNRAAVxJTlugnxtFTexUclF/IwA==", 2024-05-28T14:01:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:01:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:01:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:01:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.615 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.d", 2024-05-28T14:01:47.615 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmeAdBBhAAr4IVFQpjQXFVlnFTHnPV9w==", 2024-05-28T14:01:47.615 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.615 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:01:47.615 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:01:47.615 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:01:47.615 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.615 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.0", 2024-05-28T14:01:47.615 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDB4lVmGMDBJhAA3jIyR7XKLGXEgimIGNsw6A==", 2024-05-28T14:01:47.615 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.616 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:01:47.616 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:01:47.616 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.616 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.1", 2024-05-28T14:01:47.616 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDE4lVmxMisLxAAcIz7GPlkZwvW5ig+DbAVGg==", 2024-05-28T14:01:47.616 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.616 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:01:47.616 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:01:47.616 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.617 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.2", 2024-05-28T14:01:47.617 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDH4lVmSSk4OhAAmLlLysM4Im/Kew4Xbq2/OA==", 2024-05-28T14:01:47.617 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.617 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:01:47.617 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:01:47.617 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.617 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.3", 2024-05-28T14:01:47.617 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDL4lVmvzrYBhAAiAAOQXZBuTsMVYa1vE5rkg==", 2024-05-28T14:01:47.617 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.617 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:01:47.618 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:01:47.618 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.618 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.4", 2024-05-28T14:01:47.618 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDQ4lVmrqnTOhAA2pIfgBT9LJDIhp+GgFy8Gg==", 2024-05-28T14:01:47.618 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.618 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:01:47.618 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:01:47.618 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.5", 2024-05-28T14:01:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDU4lVmYpsQCxAAoTK2aMBPrHKvsUnNOTj/Dw==", 2024-05-28T14:01:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:01:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:01:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.6", 2024-05-28T14:01:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDX4lVm6TCAFxAAs3rkDyKcLhWrn/EkIo0bVA==", 2024-05-28T14:01:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:01:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:01:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.7", 2024-05-28T14:01:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDa4lVmr5NwIRAA7Lb70VYV82dr0Frn8CCUXg==", 2024-05-28T14:01:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:01:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:01:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.0", 2024-05-28T14:01:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw==", 2024-05-28T14:01:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:01:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-05-28T14:01:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-28T14:01:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.1", 2024-05-28T14:01:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmcfkLFBAADNyODKTtYAlFWquZ/LeeqQ==", 2024-05-28T14:01:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:01:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:01:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:01:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:01:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.2", 2024-05-28T14:01:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmoxx+EBAA8mLNak5AwRmWQIgkYtMOrg==", 2024-05-28T14:01:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:01:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:01:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:01:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:01:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.3", 2024-05-28T14:01:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmxSUsGhAAbgr1eOh/sMfrEONKKHFo0g==", 2024-05-28T14:01:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:01:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:01:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:01:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:01:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.admin", 2024-05-28T14:01:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmOSBPERAAt4x2DBal4PnqxqeWQutLDQ==", 2024-05-28T14:01:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:01:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:01:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:01:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.x", 2024-05-28T14:01:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmNO5tKRAA70hPLwoE83keovlq2rolxQ==", 2024-05-28T14:01:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:01:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T14:01:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.y", 2024-05-28T14:01:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmxTVyJRAATMLAAqKne20YvV537uXZ6g==", 2024-05-28T14:01:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:01:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:01:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T14:01:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:01:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-28T14:01:47.626 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:01:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.628+0000 7f84a9200640 1 -- 172.21.15.42:0/238428794 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8480068680 msgr2=0x7f848006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.628+0000 7f84a9200640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8480068680 0x7f848006ab40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8494007af0 tx=0x7f8494007a40 comp rx=0 tx=0).stop 2024-05-28T14:01:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.628+0000 7f84a9200640 1 -- 172.21.15.42:0/238428794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40cae30 msgr2=0x7f84a4070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.628+0000 7f84a9200640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40cae30 0x7f84a4070e80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f84a406fbc0 tx=0x7f849c007050 comp rx=0 tx=0).stop 2024-05-28T14:01:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.628+0000 7f84a9200640 1 -- 172.21.15.42:0/238428794 shutdown_connections 2024-05-28T14:01:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.628+0000 7f84a9200640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8480068680 0x7f848006ab40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.628+0000 7f84a9200640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40cae30 0x7f84a4070e80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.628+0000 7f84a9200640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84a4154f20 0x7f84a415ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.628+0000 7f84a9200640 1 --2- 172.21.15.42:0/238428794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8488004810 0x7f84a415f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.628+0000 7f84a9200640 1 -- 172.21.15.42:0/238428794 >> 172.21.15.42:0/238428794 conn(0x7f84a40b4080 msgr2=0x7f84a40ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.628+0000 7f84a9200640 1 -- 172.21.15.42:0/238428794 shutdown_connections 2024-05-28T14:01:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.628+0000 7f84a9200640 1 -- 172.21.15.42:0/238428794 wait complete. 2024-05-28T14:01:47.633 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:01:47.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.788+0000 7f6efd200640 1 Processor -- start 2024-05-28T14:01:47.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.788+0000 7f6efd200640 1 -- start start 2024-05-28T14:01:47.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6efd200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6ef80d0420 0x7f6ef815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:47.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6efd200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef806eaa0 con 0x7f6ef80b42e0 2024-05-28T14:01:47.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6efd200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef806ed80 con 0x7f6ef8067810 2024-05-28T14:01:47.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6efd200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6ef806f060 con 0x7f6ef80d0420 2024-05-28T14:01:47.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6ef6800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f6ef80b42e0 0x7f6ef806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36726/0 (socket says 172.21.15.42:36726) 2024-05-28T14:01:47.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6ef6800640 1 -- 172.21.15.42:0/323055941 learned_addr learned my addr 172.21.15.42:0/323055941 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:47.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6ef5e00640 1 --2- 172.21.15.42:0/323055941 >> v2:172.21.15.63:3300/0 conn(0x7f6ef80d0420 0x7f6ef815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:47.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6ef5400640 1 -- 172.21.15.42:0/323055941 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3591834596 0 0) 0x7f6ef806eaa0 con 0x7f6ef80b42e0 2024-05-28T14:01:47.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6ef5400640 1 -- 172.21.15.42:0/323055941 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6edc003530 con 0x7f6ef80b42e0 2024-05-28T14:01:47.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6ef5400640 1 -- 172.21.15.42:0/323055941 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 737141410 0 0) 0x7f6ef806ed80 con 0x7f6ef8067810 2024-05-28T14:01:47.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6ef5400640 1 -- 172.21.15.42:0/323055941 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ef806eaa0 con 0x7f6ef8067810 2024-05-28T14:01:47.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6ef5400640 1 -- 172.21.15.42:0/323055941 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3942313959 0 0) 0x7f6edc003530 con 0x7f6ef80b42e0 2024-05-28T14:01:47.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6ef5e00640 1 -- 172.21.15.42:0/323055941 >> v1:172.21.15.63:6789/0 conn(0x7f6ef8067810 legacy=0x7f6ef80b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6ef5e00640 1 -- 172.21.15.42:0/323055941 >> v1:172.21.15.42:6789/0 conn(0x7f6ef80b42e0 legacy=0x7f6ef806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.792+0000 7f6ef5e00640 1 -- 172.21.15.42:0/323055941 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef806f340 con 0x7f6ef80d0420 2024-05-28T14:01:47.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.796+0000 7f6ef5e00640 1 --2- 172.21.15.42:0/323055941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef80d0420 0x7f6ef815ed10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009d90 tx=0x7f6ee801d260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbf516a5ff7580a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:47.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.796+0000 7f6ef5400640 1 -- 172.21.15.42:0/323055941 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee80b2070 con 0x7f6ef80d0420 2024-05-28T14:01:47.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.796+0000 7f6ef5400640 1 -- 172.21.15.42:0/323055941 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee80a5550 con 0x7f6ef80d0420 2024-05-28T14:01:47.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.796+0000 7f6ef5400640 1 -- 172.21.15.42:0/323055941 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0001680 con 0x7f6ef80d0420 2024-05-28T14:01:47.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.796+0000 7f6efd200640 1 -- 172.21.15.42:0/323055941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef80d0420 msgr2=0x7f6ef815ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.796+0000 7f6efd200640 1 --2- 172.21.15.42:0/323055941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef80d0420 0x7f6ef815ed10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009d90 tx=0x7f6ee801d260 comp rx=0 tx=0).stop 2024-05-28T14:01:47.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.796+0000 7f6efd200640 1 -- 172.21.15.42:0/323055941 shutdown_connections 2024-05-28T14:01:47.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.796+0000 7f6efd200640 1 --2- 172.21.15.42:0/323055941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef80d0420 0x7f6ef815ed10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.796+0000 7f6efd200640 1 -- 172.21.15.42:0/323055941 >> 172.21.15.42:0/323055941 conn(0x7f6ef80d0b40 msgr2=0x7f6ef80d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:47.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.796+0000 7f6efd200640 1 -- 172.21.15.42:0/323055941 shutdown_connections 2024-05-28T14:01:47.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.796+0000 7f6efd200640 1 -- 172.21.15.42:0/323055941 wait complete. 2024-05-28T14:01:47.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.796+0000 7f6efd200640 1 Processor -- start 2024-05-28T14:01:47.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.796+0000 7f6efd200640 1 -- start start 2024-05-28T14:01:47.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6efd200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ef8067810 0x7f6ef814b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:47.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6efd200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef80b42e0 0x7f6ef8154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:47.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6efd200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ef80d0420 0x7f6ef8162500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:47.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6efd200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef806cd30 con 0x7f6ef80d0420 2024-05-28T14:01:47.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6efd200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef806cbb0 con 0x7f6ef80b42e0 2024-05-28T14:01:47.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6efd200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef806ceb0 con 0x7f6ef8067810 2024-05-28T14:01:47.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6ef6800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ef80d0420 0x7f6ef8162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:47.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6ef6800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ef80d0420 0x7f6ef8162500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54988/0 (socket says 172.21.15.42:54988) 2024-05-28T14:01:47.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6ef6800640 1 -- 172.21.15.42:0/3930477956 learned_addr learned my addr 172.21.15.42:0/3930477956 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:47.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6ef5e00640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ef8067810 0x7f6ef814b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:47.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6eed400640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef80b42e0 0x7f6ef8154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:47.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6ef6800640 1 -- 172.21.15.42:0/3930477956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ef8067810 msgr2=0x7f6ef814b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6ef6800640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ef8067810 0x7f6ef814b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6ef6800640 1 -- 172.21.15.42:0/3930477956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef80b42e0 msgr2=0x7f6ef8154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6ef6800640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef80b42e0 0x7f6ef8154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6ef6800640 1 -- 172.21.15.42:0/3930477956 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef8157f10 con 0x7f6ef80d0420 2024-05-28T14:01:47.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6eed400640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef80b42e0 0x7f6ef8154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:47.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6ef6800640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ef80d0420 0x7f6ef8162500 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000c750 tx=0x7f6ef000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:47.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6eeea00640 1 -- 172.21.15.42:0/3930477956 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0005ae0 con 0x7f6ef80d0420 2024-05-28T14:01:47.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6eeea00640 1 -- 172.21.15.42:0/3930477956 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef0005c80 con 0x7f6ef80d0420 2024-05-28T14:01:47.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6eeea00640 1 -- 172.21.15.42:0/3930477956 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00072b0 con 0x7f6ef80d0420 2024-05-28T14:01:47.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6efd200640 1 -- 172.21.15.42:0/3930477956 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef8153c70 con 0x7f6ef80d0420 2024-05-28T14:01:47.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6efd200640 1 -- 172.21.15.42:0/3930477956 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef8154230 con 0x7f6ef80d0420 2024-05-28T14:01:47.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6efd200640 1 -- 172.21.15.42:0/3930477956 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef8068a90 con 0x7f6ef80d0420 2024-05-28T14:01:47.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.800+0000 7f6eeea00640 1 -- 172.21.15.42:0/3930477956 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f6ef0007450 con 0x7f6ef80d0420 2024-05-28T14:01:47.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.804+0000 7f6eeea00640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ed4068680 0x7f6ed406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:47.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.804+0000 7f6eeea00640 1 -- 172.21.15.42:0/3930477956 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4885+0+0 (secure 0 0 0) 0x7f6ef00867e0 con 0x7f6ef80d0420 2024-05-28T14:01:47.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.804+0000 7f6ef5e00640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ed4068680 0x7f6ed406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:47.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.804+0000 7f6ef5e00640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ed4068680 0x7f6ed406ab40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80af000 tx=0x7f6ee8004ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:47.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.808+0000 7f6eeea00640 1 -- 172.21.15.42:0/3930477956 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6ef0056e10 con 0x7f6ef80d0420 2024-05-28T14:01:47.831 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:01:47.834 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:01:47.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.976+0000 7f6efd200640 1 -- 172.21.15.42:0/3930477956 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ef80d12a0 con 0x7f6ef80d0420 2024-05-28T14:01:47.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.976+0000 7f6eeea00640 1 -- 172.21.15.42:0/3930477956 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+12874 (secure 0 0 0) 0x7f6ef0056710 con 0x7f6ef80d0420 2024-05-28T14:01:47.962 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:47.962 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":42,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:01:47.221367+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:01:47.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.976+0000 7f6efd200640 1 -- 172.21.15.42:0/3930477956 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ed4068680 msgr2=0x7f6ed406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.976+0000 7f6efd200640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ed4068680 0x7f6ed406ab40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80af000 tx=0x7f6ee8004ac0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.976+0000 7f6efd200640 1 -- 172.21.15.42:0/3930477956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ef80d0420 msgr2=0x7f6ef8162500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:47.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.976+0000 7f6efd200640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ef80d0420 0x7f6ef8162500 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000c750 tx=0x7f6ef000cc10 comp rx=0 tx=0).stop 2024-05-28T14:01:47.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.976+0000 7f6efd200640 1 -- 172.21.15.42:0/3930477956 shutdown_connections 2024-05-28T14:01:47.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.976+0000 7f6efd200640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ed4068680 0x7f6ed406ab40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.976+0000 7f6efd200640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ef80d0420 0x7f6ef8162500 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.976+0000 7f6efd200640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ef80b42e0 0x7f6ef8154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.976+0000 7f6efd200640 1 --2- 172.21.15.42:0/3930477956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ef8067810 0x7f6ef814b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:47.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.976+0000 7f6efd200640 1 -- 172.21.15.42:0/3930477956 >> 172.21.15.42:0/3930477956 conn(0x7f6ef80d0b40 msgr2=0x7f6ef80b3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:47.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.980+0000 7f6efd200640 1 -- 172.21.15.42:0/3930477956 shutdown_connections 2024-05-28T14:01:47.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:47.980+0000 7f6efd200640 1 -- 172.21.15.42:0/3930477956 wait complete. 2024-05-28T14:01:47.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:01:47.984 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:01:48.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fccb6c00640 1 Processor -- start 2024-05-28T14:01:48.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fccb6c00640 1 -- start start 2024-05-28T14:01:48.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fccb6c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fccb015ec50 0x7fccb00b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fccb6c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb00b7ba0 0x7fccb00b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fccb6c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccb00d1a80 con 0x7fccb015e870 2024-05-28T14:01:48.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fccb6c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fccb00cd500 con 0x7fccb015ec50 2024-05-28T14:01:48.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fccb6c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fccb00ccbd0 con 0x7fccb00b7ba0 2024-05-28T14:01:48.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fcca7400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fccb015ec50 0x7fccb00b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fcca7400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fccb015ec50 0x7fccb00b7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54990/0 (socket says 172.21.15.42:54990) 2024-05-28T14:01:48.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fcca7400640 1 -- 172.21.15.42:0/1042782023 learned_addr learned my addr 172.21.15.42:0/1042782023 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:48.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fccb4c00640 1 --2- 172.21.15.42:0/1042782023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb00b7ba0 0x7fccb00b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fcca7400640 1 -- 172.21.15.42:0/1042782023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb00b7ba0 msgr2=0x7fccb00b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:48.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fcca7400640 1 --2- 172.21.15.42:0/1042782023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb00b7ba0 0x7fccb00b8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fcca7400640 1 -- 172.21.15.42:0/1042782023 >> v1:172.21.15.63:6789/0 conn(0x7fccb015e870 legacy=0x7fccb00d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:48.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fcca7400640 1 -- 172.21.15.42:0/1042782023 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb00cceb0 con 0x7fccb015ec50 2024-05-28T14:01:48.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.144+0000 7fccb4c00640 1 --2- 172.21.15.42:0/1042782023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb00b7ba0 0x7fccb00b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:48.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.148+0000 7fcca7400640 1 --2- 172.21.15.42:0/1042782023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb015ec50 0x7fccb00b7660 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009870 tx=0x7fcc9801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7716da5521cde8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:48.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.148+0000 7fccaf400640 1 -- 172.21.15.42:0/1042782023 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc980b5070 con 0x7fccb015ec50 2024-05-28T14:01:48.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.148+0000 7fccaf400640 1 -- 172.21.15.42:0/1042782023 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc980a89d0 con 0x7fccb015ec50 2024-05-28T14:01:48.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.148+0000 7fccaf400640 1 -- 172.21.15.42:0/1042782023 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc980a8da0 con 0x7fccb015ec50 2024-05-28T14:01:48.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.148+0000 7fccb6c00640 1 -- 172.21.15.42:0/1042782023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb015ec50 msgr2=0x7fccb00b7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.148+0000 7fccb6c00640 1 --2- 172.21.15.42:0/1042782023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb015ec50 0x7fccb00b7660 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009870 tx=0x7fcc9801ed10 comp rx=0 tx=0).stop 2024-05-28T14:01:48.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.148+0000 7fccb6c00640 1 -- 172.21.15.42:0/1042782023 shutdown_connections 2024-05-28T14:01:48.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.148+0000 7fccb6c00640 1 --2- 172.21.15.42:0/1042782023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb00b7ba0 0x7fccb00b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.148+0000 7fccb6c00640 1 --2- 172.21.15.42:0/1042782023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb015ec50 0x7fccb00b7660 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.148+0000 7fccb6c00640 1 -- 172.21.15.42:0/1042782023 >> 172.21.15.42:0/1042782023 conn(0x7fccb00b3fd0 msgr2=0x7fccb00b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:48.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.148+0000 7fccb6c00640 1 -- 172.21.15.42:0/1042782023 shutdown_connections 2024-05-28T14:01:48.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.148+0000 7fccb6c00640 1 -- 172.21.15.42:0/1042782023 wait complete. 2024-05-28T14:01:48.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccb6c00640 1 Processor -- start 2024-05-28T14:01:48.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccb6c00640 1 -- start start 2024-05-28T14:01:48.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccb6c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb00b7ba0 0x7fccb00c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccb6c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb015e870 0x7fccb01f7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccb6c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccb015ec50 0x7fccb015fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccb6c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fccb00b5330 con 0x7fccb00b7ba0 2024-05-28T14:01:48.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccb6c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fccb009aa50 con 0x7fccb015ec50 2024-05-28T14:01:48.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccb6c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fccb00c8f60 con 0x7fccb015e870 2024-05-28T14:01:48.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccafe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb00b7ba0 0x7fccb00c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccafe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb00b7ba0 0x7fccb00c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55006/0 (socket says 172.21.15.42:55006) 2024-05-28T14:01:48.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccafe00640 1 -- 172.21.15.42:0/4202395528 learned_addr learned my addr 172.21.15.42:0/4202395528 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:48.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fcca7400640 1 --2- 172.21.15.42:0/4202395528 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb015e870 0x7fccb01f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccb4c00640 1 --2- 172.21.15.42:0/4202395528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccb015ec50 0x7fccb015fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccafe00640 1 -- 172.21.15.42:0/4202395528 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb015e870 msgr2=0x7fccb01f7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccafe00640 1 --2- 172.21.15.42:0/4202395528 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb015e870 0x7fccb01f7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccafe00640 1 -- 172.21.15.42:0/4202395528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccb015ec50 msgr2=0x7fccb015fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccafe00640 1 --2- 172.21.15.42:0/4202395528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccb015ec50 0x7fccb015fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccafe00640 1 -- 172.21.15.42:0/4202395528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb01f7c70 con 0x7fccb00b7ba0 2024-05-28T14:01:48.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccafe00640 1 --2- 172.21.15.42:0/4202395528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb00b7ba0 0x7fccb00c8a20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcca000c7c0 tx=0x7fcca000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:48.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccacc00640 1 -- 172.21.15.42:0/4202395528 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0016020 con 0x7fccb00b7ba0 2024-05-28T14:01:48.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccacc00640 1 -- 172.21.15.42:0/4202395528 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca000aa50 con 0x7fccb00b7ba0 2024-05-28T14:01:48.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccacc00640 1 -- 172.21.15.42:0/4202395528 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0012420 con 0x7fccb00b7ba0 2024-05-28T14:01:48.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccb6c00640 1 -- 172.21.15.42:0/4202395528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb00c8470 con 0x7fccb00b7ba0 2024-05-28T14:01:48.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccb6c00640 1 -- 172.21.15.42:0/4202395528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb015f470 con 0x7fccb00b7ba0 2024-05-28T14:01:48.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.152+0000 7fccb6c00640 1 -- 172.21.15.42:0/4202395528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb015f890 con 0x7fccb00b7ba0 2024-05-28T14:01:48.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.156+0000 7fccacc00640 1 -- 172.21.15.42:0/4202395528 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7fcca0019020 con 0x7fccb00b7ba0 2024-05-28T14:01:48.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.156+0000 7fccacc00640 1 --2- 172.21.15.42:0/4202395528 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcc8c068650 0x7fcc8c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.156+0000 7fccacc00640 1 -- 172.21.15.42:0/4202395528 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4885+0+0 (secure 0 0 0) 0x7fcca0086ae0 con 0x7fccb00b7ba0 2024-05-28T14:01:48.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.156+0000 7fcca7400640 1 --2- 172.21.15.42:0/4202395528 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcc8c068650 0x7fcc8c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.156+0000 7fcca7400640 1 --2- 172.21.15.42:0/4202395528 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcc8c068650 0x7fcc8c06ab10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcc980b2000 tx=0x7fcc98008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:48.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.160+0000 7fccacc00640 1 -- 172.21.15.42:0/4202395528 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcca0020030 con 0x7fccb00b7ba0 2024-05-28T14:01:48.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.328+0000 7fccb6c00640 1 -- 172.21.15.42:0/4202395528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fccb00b4660 con 0x7fccb00b7ba0 2024-05-28T14:01:48.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.328+0000 7fccacc00640 1 -- 172.21.15.42:0/4202395528 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7fcca00570d0 con 0x7fccb00b7ba0 2024-05-28T14:01:48.313 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:01:48.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.328+0000 7fccb6c00640 1 -- 172.21.15.42:0/4202395528 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcc8c068650 msgr2=0x7fcc8c06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.328+0000 7fccb6c00640 1 --2- 172.21.15.42:0/4202395528 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcc8c068650 0x7fcc8c06ab10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcc980b2000 tx=0x7fcc98008040 comp rx=0 tx=0).stop 2024-05-28T14:01:48.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.328+0000 7fccb6c00640 1 -- 172.21.15.42:0/4202395528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb00b7ba0 msgr2=0x7fccb00c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.328+0000 7fccb6c00640 1 --2- 172.21.15.42:0/4202395528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb00b7ba0 0x7fccb00c8a20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcca000c7c0 tx=0x7fcca000cc80 comp rx=0 tx=0).stop 2024-05-28T14:01:48.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.328+0000 7fccb6c00640 1 -- 172.21.15.42:0/4202395528 shutdown_connections 2024-05-28T14:01:48.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.328+0000 7fccb6c00640 1 --2- 172.21.15.42:0/4202395528 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcc8c068650 0x7fcc8c06ab10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.328+0000 7fccb6c00640 1 --2- 172.21.15.42:0/4202395528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccb015ec50 0x7fccb015fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.328+0000 7fccb6c00640 1 --2- 172.21.15.42:0/4202395528 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb015e870 0x7fccb01f7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.328+0000 7fccb6c00640 1 --2- 172.21.15.42:0/4202395528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb00b7ba0 0x7fccb00c8a20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.328+0000 7fccb6c00640 1 -- 172.21.15.42:0/4202395528 >> 172.21.15.42:0/4202395528 conn(0x7fccb00b3fd0 msgr2=0x7fccb00cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:48.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.328+0000 7fccb6c00640 1 -- 172.21.15.42:0/4202395528 shutdown_connections 2024-05-28T14:01:48.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.328+0000 7fccb6c00640 1 -- 172.21.15.42:0/4202395528 wait complete. 2024-05-28T14:01:48.335 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-28T14:01:48.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.488+0000 7ff066c00640 1 Processor -- start 2024-05-28T14:01:48.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.488+0000 7ff066c00640 1 -- start start 2024-05-28T14:01:48.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.492+0000 7ff066c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff0600b14c0 0x7ff0600b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.492+0000 7ff066c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0600b18a0 0x7ff0600b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.492+0000 7ff066c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff060163410 con 0x7ff0600d0540 2024-05-28T14:01:48.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.492+0000 7ff066c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff0600b2b40 con 0x7ff0600b14c0 2024-05-28T14:01:48.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.492+0000 7ff066c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff060162650 con 0x7ff0600b18a0 2024-05-28T14:01:48.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.492+0000 7ff065800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff0600b14c0 0x7ff0600b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.492+0000 7ff066200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff0600d0540 0x7ff0600d09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36730/0 (socket says 172.21.15.42:36730) 2024-05-28T14:01:48.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.492+0000 7ff066200640 1 -- 172.21.15.42:0/1255857230 learned_addr learned my addr 172.21.15.42:0/1255857230 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:48.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.492+0000 7ff064e00640 1 --2- 172.21.15.42:0/1255857230 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0600b18a0 0x7ff0600b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.492+0000 7ff065800640 1 -- 172.21.15.42:0/1255857230 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0600b18a0 msgr2=0x7ff0600b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:48.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.492+0000 7ff065800640 1 --2- 172.21.15.42:0/1255857230 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0600b18a0 0x7ff0600b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.492+0000 7ff065800640 1 -- 172.21.15.42:0/1255857230 >> v1:172.21.15.42:6789/0 conn(0x7ff0600d0540 legacy=0x7ff0600d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.492+0000 7ff065800640 1 -- 172.21.15.42:0/1255857230 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff060162930 con 0x7ff0600b14c0 2024-05-28T14:01:48.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.496+0000 7ff065800640 1 --2- 172.21.15.42:0/1255857230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600b14c0 0x7ff0600b1fb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff0540097c0 tx=0x7ff05401c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e774dee7b5e4567 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:48.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.496+0000 7ff05fe00640 1 -- 172.21.15.42:0/1255857230 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0540b0070 con 0x7ff0600b14c0 2024-05-28T14:01:48.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.496+0000 7ff05fe00640 1 -- 172.21.15.42:0/1255857230 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff054004230 con 0x7ff0600b14c0 2024-05-28T14:01:48.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.496+0000 7ff05fe00640 1 -- 172.21.15.42:0/1255857230 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054003620 con 0x7ff0600b14c0 2024-05-28T14:01:48.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.496+0000 7ff066c00640 1 -- 172.21.15.42:0/1255857230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600b14c0 msgr2=0x7ff0600b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.496+0000 7ff066c00640 1 --2- 172.21.15.42:0/1255857230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600b14c0 0x7ff0600b1fb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff0540097c0 tx=0x7ff05401c350 comp rx=0 tx=0).stop 2024-05-28T14:01:48.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.496+0000 7ff066c00640 1 -- 172.21.15.42:0/1255857230 shutdown_connections 2024-05-28T14:01:48.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.496+0000 7ff066c00640 1 --2- 172.21.15.42:0/1255857230 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0600b18a0 0x7ff0600b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.496+0000 7ff066c00640 1 --2- 172.21.15.42:0/1255857230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600b14c0 0x7ff0600b1fb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.496+0000 7ff066c00640 1 -- 172.21.15.42:0/1255857230 >> 172.21.15.42:0/1255857230 conn(0x7ff06009aa50 msgr2=0x7ff0600b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:48.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.496+0000 7ff066c00640 1 -- 172.21.15.42:0/1255857230 shutdown_connections 2024-05-28T14:01:48.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.496+0000 7ff066c00640 1 -- 172.21.15.42:0/1255857230 wait complete. 2024-05-28T14:01:48.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.496+0000 7ff066c00640 1 Processor -- start 2024-05-28T14:01:48.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.496+0000 7ff066c00640 1 -- start start 2024-05-28T14:01:48.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff066c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0600b14c0 0x7ff06015f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff066c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600b18a0 0x7ff0600ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff066c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0600d0540 0x7ff06015fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff066c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff0600ce030 con 0x7ff0600b18a0 2024-05-28T14:01:48.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff066c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff0600cdeb0 con 0x7ff0600b14c0 2024-05-28T14:01:48.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff066c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff0600ce1b0 con 0x7ff0600d0540 2024-05-28T14:01:48.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff064e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600b18a0 0x7ff0600ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff064e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600b18a0 0x7ff0600ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55032/0 (socket says 172.21.15.42:55032) 2024-05-28T14:01:48.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff064e00640 1 -- 172.21.15.42:0/605222190 learned_addr learned my addr 172.21.15.42:0/605222190 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff065800640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0600b14c0 0x7ff06015f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff066200640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0600d0540 0x7ff06015fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff064e00640 1 -- 172.21.15.42:0/605222190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0600d0540 msgr2=0x7ff06015fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff064e00640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0600d0540 0x7ff06015fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff064e00640 1 -- 172.21.15.42:0/605222190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0600b14c0 msgr2=0x7ff06015f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff064e00640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0600b14c0 0x7ff06015f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff064e00640 1 -- 172.21.15.42:0/605222190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff060156160 con 0x7ff0600b18a0 2024-05-28T14:01:48.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff066200640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0600d0540 0x7ff06015fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:48.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff064e00640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600b18a0 0x7ff0600ccaf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00e900 tx=0x7ff04c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:48.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff05d600640 1 -- 172.21.15.42:0/605222190 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c017070 con 0x7ff0600b18a0 2024-05-28T14:01:48.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff05d600640 1 -- 172.21.15.42:0/605222190 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff04c00ca00 con 0x7ff0600b18a0 2024-05-28T14:01:48.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff05d600640 1 -- 172.21.15.42:0/605222190 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c00cdf0 con 0x7ff0600b18a0 2024-05-28T14:01:48.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff066c00640 1 -- 172.21.15.42:0/605222190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0600cd030 con 0x7ff0600b18a0 2024-05-28T14:01:48.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff066c00640 1 -- 172.21.15.42:0/605222190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06014dc10 con 0x7ff0600b18a0 2024-05-28T14:01:48.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff066c00640 1 -- 172.21.15.42:0/605222190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff06004fcf0 con 0x7ff0600b18a0 2024-05-28T14:01:48.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.500+0000 7ff05d600640 1 -- 172.21.15.42:0/605222190 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7ff04c01a070 con 0x7ff0600b18a0 2024-05-28T14:01:48.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.504+0000 7ff05d600640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff040068650 0x7ff04006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.504+0000 7ff05d600640 1 -- 172.21.15.42:0/605222190 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4885+0+0 (secure 0 0 0) 0x7ff04c086ae0 con 0x7ff0600b18a0 2024-05-28T14:01:48.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.504+0000 7ff065800640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff040068650 0x7ff04006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.504+0000 7ff065800640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff040068650 0x7ff04006ab10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff0540ad000 tx=0x7ff054008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:48.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.508+0000 7ff05d600640 1 -- 172.21.15.42:0/605222190 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff04c057080 con 0x7ff0600b18a0 2024-05-28T14:01:48.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.672+0000 7ff066c00640 1 -- 172.21.15.42:0/605222190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff060070940 con 0x7ff0600b18a0 2024-05-28T14:01:48.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.672+0000 7ff05d600640 1 -- 172.21.15.42:0/605222190 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) ==== 84+0+65 (secure 0 0 0) 0x7ff04c056980 con 0x7ff0600b18a0 2024-05-28T14:01:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:48.658 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:01:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-05-28T14:01:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-05-28T14:01:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:48.659 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:01:48.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.672+0000 7ff066c00640 1 -- 172.21.15.42:0/605222190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff040068650 msgr2=0x7ff04006ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.672+0000 7ff066c00640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff040068650 0x7ff04006ab10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff0540ad000 tx=0x7ff054008040 comp rx=0 tx=0).stop 2024-05-28T14:01:48.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.672+0000 7ff066c00640 1 -- 172.21.15.42:0/605222190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600b18a0 msgr2=0x7ff0600ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.672+0000 7ff066c00640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600b18a0 0x7ff0600ccaf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00e900 tx=0x7ff04c00edc0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.676+0000 7ff066c00640 1 -- 172.21.15.42:0/605222190 shutdown_connections 2024-05-28T14:01:48.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.676+0000 7ff066c00640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff040068650 0x7ff04006ab10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.676+0000 7ff066c00640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0600d0540 0x7ff06015fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.676+0000 7ff066c00640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600b18a0 0x7ff0600ccaf0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.676+0000 7ff066c00640 1 --2- 172.21.15.42:0/605222190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0600b14c0 0x7ff06015f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.676+0000 7ff066c00640 1 -- 172.21.15.42:0/605222190 >> 172.21.15.42:0/605222190 conn(0x7ff06009aa50 msgr2=0x7ff0601581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:48.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.676+0000 7ff066c00640 1 -- 172.21.15.42:0/605222190 shutdown_connections 2024-05-28T14:01:48.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.676+0000 7ff066c00640 1 -- 172.21.15.42:0/605222190 wait complete. 2024-05-28T14:01:48.680 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-28T14:01:48.680 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-28T14:01:48.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69ff400640 1 Processor -- start 2024-05-28T14:01:48.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69ff400640 1 -- start start 2024-05-28T14:01:48.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69ff400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f69f815aab0 0x7f69f80ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69ff400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69f806eda0 0x7f69f8073690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69ff400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69f806dff0 con 0x7f69f8073220 2024-05-28T14:01:48.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69ff400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f69f8074050 con 0x7f69f815aab0 2024-05-28T14:01:48.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69ff400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f69f8073d70 con 0x7f69f806eda0 2024-05-28T14:01:48.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69fd400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69f806eda0 0x7f69f8073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69fca00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f69f815aab0 0x7f69f80ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69efe00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f69f8073220 0x7f69f806e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:41524/0 (socket says 172.21.15.42:41524) 2024-05-28T14:01:48.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69efe00640 1 -- 172.21.15.42:0/1160544347 learned_addr learned my addr 172.21.15.42:0/1160544347 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:48.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69fd400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69f806eda0 0x7f69f8073690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:36754/0 (socket says 172.21.15.42:36754) 2024-05-28T14:01:48.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69fca00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f69f815aab0 0x7f69f80ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:35470/0 (socket says 172.21.15.42:35470) 2024-05-28T14:01:48.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69fca00640 1 -- 172.21.15.42:0/1160544347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69f806eda0 msgr2=0x7f69f8073690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:48.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69fca00640 1 --2- 172.21.15.42:0/1160544347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69f806eda0 0x7f69f8073690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69fca00640 1 -- 172.21.15.42:0/1160544347 >> v1:172.21.15.63:6789/0 conn(0x7f69f8073220 legacy=0x7f69f806e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69fca00640 1 -- 172.21.15.42:0/1160544347 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f80b3310 con 0x7f69f815aab0 2024-05-28T14:01:48.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.840+0000 7f69fd400640 1 --2- 172.21.15.42:0/1160544347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69f806eda0 0x7f69f8073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:01:48.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.844+0000 7f69fca00640 1 --2- 172.21.15.42:0/1160544347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69f815aab0 0x7f69f80ad680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f69e80097c0 tx=0x7f69e801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=850db6e17a8e95b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:48.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.844+0000 7f69f7e00640 1 -- 172.21.15.42:0/1160544347 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e80b2070 con 0x7f69f815aab0 2024-05-28T14:01:48.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.844+0000 7f69f7e00640 1 -- 172.21.15.42:0/1160544347 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e80a9070 con 0x7f69f815aab0 2024-05-28T14:01:48.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.844+0000 7f69f7e00640 1 -- 172.21.15.42:0/1160544347 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e80ad880 con 0x7f69f815aab0 2024-05-28T14:01:48.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.844+0000 7f69ff400640 1 -- 172.21.15.42:0/1160544347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69f815aab0 msgr2=0x7f69f80ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.844+0000 7f69ff400640 1 --2- 172.21.15.42:0/1160544347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69f815aab0 0x7f69f80ad680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f69e80097c0 tx=0x7f69e801c3e0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.844+0000 7f69ff400640 1 -- 172.21.15.42:0/1160544347 shutdown_connections 2024-05-28T14:01:48.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.844+0000 7f69ff400640 1 --2- 172.21.15.42:0/1160544347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69f806eda0 0x7f69f8073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.844+0000 7f69ff400640 1 --2- 172.21.15.42:0/1160544347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69f815aab0 0x7f69f80ad680 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.844+0000 7f69ff400640 1 -- 172.21.15.42:0/1160544347 >> 172.21.15.42:0/1160544347 conn(0x7f69f806bdf0 msgr2=0x7f69f806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:48.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.844+0000 7f69ff400640 1 -- 172.21.15.42:0/1160544347 shutdown_connections 2024-05-28T14:01:48.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.844+0000 7f69ff400640 1 -- 172.21.15.42:0/1160544347 wait complete. 2024-05-28T14:01:48.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69ff400640 1 Processor -- start 2024-05-28T14:01:48.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69ff400640 1 -- start start 2024-05-28T14:01:48.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69ff400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f806eda0 0x7f69f8147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69ff400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69f8073220 0x7f69f81581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69ff400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69f815aab0 0x7f69f8158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69ff400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69f80b3f70 con 0x7f69f806eda0 2024-05-28T14:01:48.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69ff400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f69f80b3df0 con 0x7f69f815aab0 2024-05-28T14:01:48.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69ff400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f69f80b40f0 con 0x7f69f8073220 2024-05-28T14:01:48.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69fca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f806eda0 0x7f69f8147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69fca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f806eda0 0x7f69f8147780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55042/0 (socket says 172.21.15.42:55042) 2024-05-28T14:01:48.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69fca00640 1 -- 172.21.15.42:0/3705158334 learned_addr learned my addr 172.21.15.42:0/3705158334 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:48.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69fd400640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69f815aab0 0x7f69f8158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69efe00640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69f8073220 0x7f69f81581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69fca00640 1 -- 172.21.15.42:0/3705158334 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69f8073220 msgr2=0x7f69f81581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69fca00640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69f8073220 0x7f69f81581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69fca00640 1 -- 172.21.15.42:0/3705158334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69f815aab0 msgr2=0x7f69f8158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:48.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69fca00640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69f815aab0 0x7f69f8158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:48.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69fca00640 1 -- 172.21.15.42:0/3705158334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f8147cc0 con 0x7f69f806eda0 2024-05-28T14:01:48.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69fd400640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69f815aab0 0x7f69f8158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:48.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69fca00640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f806eda0 0x7f69f8147780 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f69e801a010 tx=0x7f69e801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:48.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69f5600640 1 -- 172.21.15.42:0/3705158334 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e80b2050 con 0x7f69f806eda0 2024-05-28T14:01:48.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69f5600640 1 -- 172.21.15.42:0/3705158334 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e80be070 con 0x7f69f806eda0 2024-05-28T14:01:48.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69f5600640 1 -- 172.21.15.42:0/3705158334 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e80b9690 con 0x7f69f806eda0 2024-05-28T14:01:48.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69ff400640 1 -- 172.21.15.42:0/3705158334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f81491f0 con 0x7f69f806eda0 2024-05-28T14:01:48.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.848+0000 7f69ff400640 1 -- 172.21.15.42:0/3705158334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f814f870 con 0x7f69f806eda0 2024-05-28T14:01:48.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.852+0000 7f69ff400640 1 -- 172.21.15.42:0/3705158334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f8068a90 con 0x7f69f806eda0 2024-05-28T14:01:48.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.852+0000 7f69f5600640 1 -- 172.21.15.42:0/3705158334 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f69e80ab070 con 0x7f69f806eda0 2024-05-28T14:01:48.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.852+0000 7f69f5600640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f69d4068650 0x7f69d406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:48.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.852+0000 7f69f5600640 1 -- 172.21.15.42:0/3705158334 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4885+0+0 (secure 0 0 0) 0x7f69e8121150 con 0x7f69f806eda0 2024-05-28T14:01:48.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.852+0000 7f69efe00640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f69d4068650 0x7f69d406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:48.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.852+0000 7f69efe00640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f69d4068650 0x7f69d406ab10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69f8159110 tx=0x7f69e0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:48.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:48.856+0000 7f69f5600640 1 -- 172.21.15.42:0/3705158334 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f69f81491f0 con 0x7f69f806eda0 2024-05-28T14:01:49.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.028+0000 7f69ff400640 1 -- 172.21.15.42:0/3705158334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f69f80b0fb0 con 0x7f69f806eda0 2024-05-28T14:01:49.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.216+0000 7f69f5600640 1 -- 172.21.15.42:0/3705158334 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) ==== 138+0+0 (secure 0 0 0) 0x7f69e8016090 con 0x7f69f806eda0 2024-05-28T14:01:49.203 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-05-28T14:01:49.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.220+0000 7f69ff400640 1 -- 172.21.15.42:0/3705158334 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f69d4068650 msgr2=0x7f69d406ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.220+0000 7f69ff400640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f69d4068650 0x7f69d406ab10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69f8159110 tx=0x7f69e0009040 comp rx=0 tx=0).stop 2024-05-28T14:01:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.220+0000 7f69ff400640 1 -- 172.21.15.42:0/3705158334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f806eda0 msgr2=0x7f69f8147780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.220+0000 7f69ff400640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f806eda0 0x7f69f8147780 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f69e801a010 tx=0x7f69e801ac20 comp rx=0 tx=0).stop 2024-05-28T14:01:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.220+0000 7f69ff400640 1 -- 172.21.15.42:0/3705158334 shutdown_connections 2024-05-28T14:01:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.220+0000 7f69ff400640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f69d4068650 0x7f69d406ab10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.220+0000 7f69ff400640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69f815aab0 0x7f69f8158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.220+0000 7f69ff400640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69f8073220 0x7f69f81581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.220+0000 7f69ff400640 1 --2- 172.21.15.42:0/3705158334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f806eda0 0x7f69f8147780 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.220+0000 7f69ff400640 1 -- 172.21.15.42:0/3705158334 >> 172.21.15.42:0/3705158334 conn(0x7f69f806bdf0 msgr2=0x7f69f8159880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.220+0000 7f69ff400640 1 -- 172.21.15.42:0/3705158334 shutdown_connections 2024-05-28T14:01:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.220+0000 7f69ff400640 1 -- 172.21.15.42:0/3705158334 wait complete. 2024-05-28T14:01:49.231 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-28T14:01:49.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.392+0000 7fa1b1000640 1 Processor -- start 2024-05-28T14:01:49.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.392+0000 7fa1b1000640 1 -- start start 2024-05-28T14:01:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.392+0000 7fa1b1000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa1ac06ee20 0x7fa1ac06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.392+0000 7fa1b1000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa1ac15b9b0 0x7fa1ac15be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.392+0000 7fa1b1000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1ac0afe40 con 0x7fa1ac144f80 2024-05-28T14:01:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.392+0000 7fa1b1000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa1ac06f7c0 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.392+0000 7fa1b1000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa1ac0724d0 con 0x7fa1ac06ee20 2024-05-28T14:01:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.396+0000 7fa1aa800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa1ac15b9b0 0x7fa1ac15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.396+0000 7fa1a9e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa1ac144f80 0x7fa1ac072960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36746/0 (socket says 172.21.15.42:36746) 2024-05-28T14:01:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.396+0000 7fa1a9e00640 1 -- 172.21.15.42:0/3568324070 learned_addr learned my addr 172.21.15.42:0/3568324070 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.396+0000 7fa1a1400640 1 --2- 172.21.15.42:0/3568324070 >> v2:172.21.15.63:3300/0 conn(0x7fa1ac06ee20 0x7fa1ac06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.396+0000 7fa1aa800640 1 -- 172.21.15.42:0/3568324070 >> v2:172.21.15.63:3300/0 conn(0x7fa1ac06ee20 msgr2=0x7fa1ac06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.396+0000 7fa1aa800640 1 --2- 172.21.15.42:0/3568324070 >> v2:172.21.15.63:3300/0 conn(0x7fa1ac06ee20 0x7fa1ac06f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.396+0000 7fa1aa800640 1 -- 172.21.15.42:0/3568324070 >> v1:172.21.15.42:6789/0 conn(0x7fa1ac144f80 legacy=0x7fa1ac072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.396+0000 7fa1aa800640 1 -- 172.21.15.42:0/3568324070 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac15c340 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.396+0000 7fa1a1400640 1 --2- 172.21.15.42:0/3568324070 >> v2:172.21.15.63:3300/0 conn(0x7fa1ac06ee20 0x7fa1ac06f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.396+0000 7fa1aa800640 1 --2- 172.21.15.42:0/3568324070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1ac15b9b0 0x7fa1ac15be00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac075500 tx=0x7fa1a401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b292d2cdc0531de9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.396+0000 7fa1a9400640 1 -- 172.21.15.42:0/3568324070 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40b2070 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.396+0000 7fa1a9400640 1 -- 172.21.15.42:0/3568324070 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a40a57f0 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1a9400640 1 -- 172.21.15.42:0/3568324070 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40a5bc0 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 -- 172.21.15.42:0/3568324070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1ac15b9b0 msgr2=0x7fa1ac15be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 --2- 172.21.15.42:0/3568324070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1ac15b9b0 0x7fa1ac15be00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac075500 tx=0x7fa1a401c930 comp rx=0 tx=0).stop 2024-05-28T14:01:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 -- 172.21.15.42:0/3568324070 shutdown_connections 2024-05-28T14:01:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 --2- 172.21.15.42:0/3568324070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1ac15b9b0 0x7fa1ac15be00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 --2- 172.21.15.42:0/3568324070 >> v2:172.21.15.63:3300/0 conn(0x7fa1ac06ee20 0x7fa1ac06f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 -- 172.21.15.42:0/3568324070 >> 172.21.15.42:0/3568324070 conn(0x7fa1ac0b21f0 msgr2=0x7fa1ac0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 -- 172.21.15.42:0/3568324070 shutdown_connections 2024-05-28T14:01:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 -- 172.21.15.42:0/3568324070 wait complete. 2024-05-28T14:01:49.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 Processor -- start 2024-05-28T14:01:49.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 -- start start 2024-05-28T14:01:49.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1ac06ee20 0x7fa1ac15a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:49.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac144f80 0x7fa1ac15dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:49.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1ac15b9b0 0x7fa1ac15e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:49.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac0b4010 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac072160 con 0x7fa1ac06ee20 2024-05-28T14:01:49.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1b1000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ac0b4190 con 0x7fa1ac144f80 2024-05-28T14:01:49.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1aa800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1ac15b9b0 0x7fa1ac15e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:49.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1aa800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1ac15b9b0 0x7fa1ac15e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55068/0 (socket says 172.21.15.42:55068) 2024-05-28T14:01:49.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1aa800640 1 -- 172.21.15.42:0/2833400758 learned_addr learned my addr 172.21.15.42:0/2833400758 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:49.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1a9e00640 1 --2- 172.21.15.42:0/2833400758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1ac06ee20 0x7fa1ac15a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:49.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.400+0000 7fa1a1400640 1 --2- 172.21.15.42:0/2833400758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac144f80 0x7fa1ac15dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:49.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.404+0000 7fa1aa800640 1 -- 172.21.15.42:0/2833400758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac144f80 msgr2=0x7fa1ac15dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:49.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.404+0000 7fa1aa800640 1 --2- 172.21.15.42:0/2833400758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac144f80 0x7fa1ac15dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:49.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.404+0000 7fa1aa800640 1 -- 172.21.15.42:0/2833400758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1ac06ee20 msgr2=0x7fa1ac15a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.404+0000 7fa1aa800640 1 --2- 172.21.15.42:0/2833400758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1ac06ee20 0x7fa1ac15a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.404+0000 7fa1aa800640 1 -- 172.21.15.42:0/2833400758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac140ca0 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.404+0000 7fa1aa800640 1 --2- 172.21.15.42:0/2833400758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1ac15b9b0 0x7fa1ac15e100 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40b0040 tx=0x7fa1a40196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.404+0000 7fa1a2a00640 1 -- 172.21.15.42:0/2833400758 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40b2050 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.404+0000 7fa1a2a00640 1 -- 172.21.15.42:0/2833400758 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a40be070 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.404+0000 7fa1a2a00640 1 -- 172.21.15.42:0/2833400758 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40ada90 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.404+0000 7fa1b1000640 1 -- 172.21.15.42:0/2833400758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ac13eb80 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.404+0000 7fa1b1000640 1 -- 172.21.15.42:0/2833400758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ac13f140 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.404+0000 7fa1b1000640 1 -- 172.21.15.42:0/2833400758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ac0738b0 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.404+0000 7fa1a2a00640 1 -- 172.21.15.42:0/2833400758 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7fa1a40adc30 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.408+0000 7fa1a2a00640 1 --2- 172.21.15.42:0/2833400758 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa188068680 0x7fa18806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:49.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.408+0000 7fa1a2a00640 1 -- 172.21.15.42:0/2833400758 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5255+0+0 (secure 0 0 0) 0x7fa1a4122760 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.408+0000 7fa1a9e00640 1 --2- 172.21.15.42:0/2833400758 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa188068680 0x7fa18806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:49.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.408+0000 7fa1a9e00640 1 --2- 172.21.15.42:0/2833400758 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa188068680 0x7fa18806ab40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa19c000f70 tx=0x7fa19c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:49.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.408+0000 7fa1a2a00640 1 -- 172.21.15.42:0/2833400758 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa1a4016520 con 0x7fa1ac15b9b0 2024-05-28T14:01:49.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:49.576+0000 7fa1b1000640 1 -- 172.21.15.42:0/2833400758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa1ac04fdd0 con 0x7fa1ac15b9b0 2024-05-28T14:01:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.220+0000 7fa1a2a00640 1 -- 172.21.15.42:0/2833400758 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) ==== 170+0+0 (secure 0 0 0) 0x7fa1a40f2d80 con 0x7fa1ac15b9b0 2024-05-28T14:01:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-05-28T14:01:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.224+0000 7fa1b1000640 1 -- 172.21.15.42:0/2833400758 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa188068680 msgr2=0x7fa18806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.224+0000 7fa1b1000640 1 --2- 172.21.15.42:0/2833400758 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa188068680 0x7fa18806ab40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa19c000f70 tx=0x7fa19c009040 comp rx=0 tx=0).stop 2024-05-28T14:01:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.224+0000 7fa1b1000640 1 -- 172.21.15.42:0/2833400758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1ac15b9b0 msgr2=0x7fa1ac15e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.224+0000 7fa1b1000640 1 --2- 172.21.15.42:0/2833400758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1ac15b9b0 0x7fa1ac15e100 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40b0040 tx=0x7fa1a40196d0 comp rx=0 tx=0).stop 2024-05-28T14:01:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.224+0000 7fa1b1000640 1 -- 172.21.15.42:0/2833400758 shutdown_connections 2024-05-28T14:01:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.224+0000 7fa1b1000640 1 --2- 172.21.15.42:0/2833400758 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa188068680 0x7fa18806ab40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.224+0000 7fa1b1000640 1 --2- 172.21.15.42:0/2833400758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1ac15b9b0 0x7fa1ac15e100 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.224+0000 7fa1b1000640 1 --2- 172.21.15.42:0/2833400758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac144f80 0x7fa1ac15dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.224+0000 7fa1b1000640 1 --2- 172.21.15.42:0/2833400758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1ac06ee20 0x7fa1ac15a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.224+0000 7fa1b1000640 1 -- 172.21.15.42:0/2833400758 >> 172.21.15.42:0/2833400758 conn(0x7fa1ac0b21f0 msgr2=0x7fa1ac0cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.224+0000 7fa1b1000640 1 -- 172.21.15.42:0/2833400758 shutdown_connections 2024-05-28T14:01:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.224+0000 7fa1b1000640 1 -- 172.21.15.42:0/2833400758 wait complete. 2024-05-28T14:01:50.250 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-28T14:01:50.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f4585000640 1 Processor -- start 2024-05-28T14:01:50.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f4585000640 1 -- start start 2024-05-28T14:01:50.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f4585000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f45800ce3b0 0x7f45800ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:50.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f4585000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f458013fd10 0x7f4580140180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:50.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f4585000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4580160230 con 0x7f4580071de0 2024-05-28T14:01:50.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f4585000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f4580160510 con 0x7f45800ce3b0 2024-05-28T14:01:50.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f4585000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f45801607f0 con 0x7f458013fd10 2024-05-28T14:01:50.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f457de00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f4580071de0 0x7f45800721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36756/0 (socket says 172.21.15.42:36756) 2024-05-28T14:01:50.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f457de00640 1 -- 172.21.15.42:0/2865888145 learned_addr learned my addr 172.21.15.42:0/2865888145 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:50.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f4577e00640 1 --2- 172.21.15.42:0/2865888145 >> v2:172.21.15.42:3300/0 conn(0x7f45800ce3b0 0x7f45800ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:50.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f457e800640 1 --2- 172.21.15.42:0/2865888145 >> v2:172.21.15.63:3300/0 conn(0x7f458013fd10 0x7f4580140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:50.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f4577e00640 1 -- 172.21.15.42:0/2865888145 >> v2:172.21.15.63:3300/0 conn(0x7f458013fd10 msgr2=0x7f4580140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:50.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f4577e00640 1 --2- 172.21.15.42:0/2865888145 >> v2:172.21.15.63:3300/0 conn(0x7f458013fd10 0x7f4580140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:50.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f4577e00640 1 -- 172.21.15.42:0/2865888145 >> v1:172.21.15.42:6789/0 conn(0x7f4580071de0 legacy=0x7f45800721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:50.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f4577e00640 1 -- 172.21.15.42:0/2865888145 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4580160ad0 con 0x7f45800ce3b0 2024-05-28T14:01:50.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f457e800640 1 --2- 172.21.15.42:0/2865888145 >> v2:172.21.15.63:3300/0 conn(0x7f458013fd10 0x7f4580140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:50.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.412+0000 7f457d400640 1 -- 172.21.15.42:0/2865888145 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 311295629 0 0) 0x7f4580160230 con 0x7f4580071de0 2024-05-28T14:01:50.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.416+0000 7f4577e00640 1 --2- 172.21.15.42:0/2865888145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45800ce3b0 0x7f45800ce810 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4580071680 tx=0x7f456801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3551d4b649819a3f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:50.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.416+0000 7f457d400640 1 -- 172.21.15.42:0/2865888145 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680b0070 con 0x7f45800ce3b0 2024-05-28T14:01:50.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.416+0000 7f457d400640 1 -- 172.21.15.42:0/2865888145 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45680ab600 con 0x7f45800ce3b0 2024-05-28T14:01:50.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.416+0000 7f457d400640 1 -- 172.21.15.42:0/2865888145 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680ab9d0 con 0x7f45800ce3b0 2024-05-28T14:01:50.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.416+0000 7f4585000640 1 -- 172.21.15.42:0/2865888145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45800ce3b0 msgr2=0x7f45800ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:50.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.416+0000 7f4585000640 1 --2- 172.21.15.42:0/2865888145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45800ce3b0 0x7f45800ce810 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4580071680 tx=0x7f456801c360 comp rx=0 tx=0).stop 2024-05-28T14:01:50.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.416+0000 7f4585000640 1 -- 172.21.15.42:0/2865888145 shutdown_connections 2024-05-28T14:01:50.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.416+0000 7f4585000640 1 --2- 172.21.15.42:0/2865888145 >> v2:172.21.15.63:3300/0 conn(0x7f458013fd10 0x7f4580140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:50.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.416+0000 7f4585000640 1 --2- 172.21.15.42:0/2865888145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45800ce3b0 0x7f45800ce810 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:50.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.416+0000 7f4585000640 1 -- 172.21.15.42:0/2865888145 >> 172.21.15.42:0/2865888145 conn(0x7f45800b21f0 msgr2=0x7f45800b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.416+0000 7f4585000640 1 -- 172.21.15.42:0/2865888145 shutdown_connections 2024-05-28T14:01:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.416+0000 7f4585000640 1 -- 172.21.15.42:0/2865888145 wait complete. 2024-05-28T14:01:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.416+0000 7f4585000640 1 Processor -- start 2024-05-28T14:01:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.416+0000 7f4585000640 1 -- start start 2024-05-28T14:01:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f4585000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4580071de0 0x7f4580158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f4585000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45800ce3b0 0x7f4580155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f4585000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458013fd10 0x7f45801571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f4585000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4580169830 con 0x7f458013fd10 2024-05-28T14:01:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f4585000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f45801696b0 con 0x7f4580071de0 2024-05-28T14:01:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f4585000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f45801699b0 con 0x7f45800ce3b0 2024-05-28T14:01:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f457e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458013fd10 0x7f45801571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f457e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458013fd10 0x7f45801571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55082/0 (socket says 172.21.15.42:55082) 2024-05-28T14:01:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f457e800640 1 -- 172.21.15.42:0/3434205592 learned_addr learned my addr 172.21.15.42:0/3434205592 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f4577e00640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45800ce3b0 0x7f4580155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f457de00640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4580071de0 0x7f4580158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f457e800640 1 -- 172.21.15.42:0/3434205592 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45800ce3b0 msgr2=0x7f4580155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f457e800640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45800ce3b0 0x7f4580155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f457e800640 1 -- 172.21.15.42:0/3434205592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4580071de0 msgr2=0x7f4580158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f457e800640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4580071de0 0x7f4580158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f457e800640 1 -- 172.21.15.42:0/3434205592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4580157a60 con 0x7f458013fd10 2024-05-28T14:01:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f457de00640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4580071de0 0x7f4580158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f457e800640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458013fd10 0x7f45801571d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4578007e20 tx=0x7f457800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f4576000640 1 -- 172.21.15.42:0/3434205592 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457801a070 con 0x7f458013fd10 2024-05-28T14:01:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f4576000640 1 -- 172.21.15.42:0/3434205592 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4578003ba0 con 0x7f458013fd10 2024-05-28T14:01:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f4576000640 1 -- 172.21.15.42:0/3434205592 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457800de80 con 0x7f458013fd10 2024-05-28T14:01:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f4585000640 1 -- 172.21.15.42:0/3434205592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4580157d40 con 0x7f458013fd10 2024-05-28T14:01:50.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f4585000640 1 -- 172.21.15.42:0/3434205592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4580158300 con 0x7f458013fd10 2024-05-28T14:01:50.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.420+0000 7f4585000640 1 -- 172.21.15.42:0/3434205592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45800738b0 con 0x7f458013fd10 2024-05-28T14:01:50.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.424+0000 7f4576000640 1 -- 172.21.15.42:0/3434205592 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f457801d070 con 0x7f458013fd10 2024-05-28T14:01:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.424+0000 7f4576000640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f455c068650 0x7f455c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.424+0000 7f4576000640 1 -- 172.21.15.42:0/3434205592 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5637+0+0 (secure 0 0 0) 0x7f4578087560 con 0x7f458013fd10 2024-05-28T14:01:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.424+0000 7f457de00640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f455c068650 0x7f455c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.424+0000 7f457de00640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f455c068650 0x7f455c06ab10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4570007fd0 tx=0x7f4570009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:50.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.428+0000 7f4576000640 1 -- 172.21.15.42:0/3434205592 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4578024020 con 0x7f458013fd10 2024-05-28T14:01:50.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:50.628+0000 7f4585000640 1 -- 172.21.15.42:0/3434205592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f458004fdd0 con 0x7f458013fd10 2024-05-28T14:01:51.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.236+0000 7f4576000640 1 -- 172.21.15.42:0/3434205592 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v23) ==== 172+0+0 (secure 0 0 0) 0x7f4578010070 con 0x7f458013fd10 2024-05-28T14:01:51.223 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 6 and data pool 7 2024-05-28T14:01:51.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.240+0000 7f4585000640 1 -- 172.21.15.42:0/3434205592 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f455c068650 msgr2=0x7f455c06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.240+0000 7f4585000640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f455c068650 0x7f455c06ab10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4570007fd0 tx=0x7f4570009040 comp rx=0 tx=0).stop 2024-05-28T14:01:51.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.240+0000 7f4585000640 1 -- 172.21.15.42:0/3434205592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458013fd10 msgr2=0x7f45801571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.240+0000 7f4585000640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458013fd10 0x7f45801571d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4578007e20 tx=0x7f457800d4a0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.240+0000 7f4585000640 1 -- 172.21.15.42:0/3434205592 shutdown_connections 2024-05-28T14:01:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.240+0000 7f4585000640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f455c068650 0x7f455c06ab10 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4570007fd0 tx=0x7f4570009040 comp rx=0 tx=0).stop 2024-05-28T14:01:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.240+0000 7f4585000640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458013fd10 0x7f45801571d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.240+0000 7f4585000640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45800ce3b0 0x7f4580155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.240+0000 7f4585000640 1 --2- 172.21.15.42:0/3434205592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4580071de0 0x7f4580158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.240+0000 7f4585000640 1 -- 172.21.15.42:0/3434205592 >> 172.21.15.42:0/3434205592 conn(0x7f45800b21f0 msgr2=0x7f45800b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.240+0000 7f4585000640 1 -- 172.21.15.42:0/3434205592 shutdown_connections 2024-05-28T14:01:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.240+0000 7f4585000640 1 -- 172.21.15.42:0/3434205592 wait complete. 2024-05-28T14:01:51.267 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:01:51.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.428+0000 7f7e37a00640 1 Processor -- start 2024-05-28T14:01:51.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.428+0000 7f7e37a00640 1 -- start start 2024-05-28T14:01:51.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.428+0000 7f7e37a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7e3006f660 0x7f7e30073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:51.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.428+0000 7f7e37a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e3015f280 con 0x7f7e300d0e00 2024-05-28T14:01:51.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.428+0000 7f7e37a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e3015f560 con 0x7f7e3015ba40 2024-05-28T14:01:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.428+0000 7f7e37a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f7e3015f030 con 0x7f7e3006f660 2024-05-28T14:01:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.428+0000 7f7e35000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7e3006f660 0x7f7e30073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.428+0000 7f7e35a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f7e300d0e00 0x7f7e3006dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36762/0 (socket says 172.21.15.42:36762) 2024-05-28T14:01:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.428+0000 7f7e35a00640 1 -- 172.21.15.42:0/298263005 learned_addr learned my addr 172.21.15.42:0/298263005 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.428+0000 7f7e35000640 1 -- 172.21.15.42:0/298263005 >> v1:172.21.15.63:6789/0 conn(0x7f7e3015ba40 legacy=0x7f7e300d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.428+0000 7f7e35000640 1 -- 172.21.15.42:0/298263005 >> v1:172.21.15.42:6789/0 conn(0x7f7e300d0e00 legacy=0x7f7e3006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.428+0000 7f7e35000640 1 -- 172.21.15.42:0/298263005 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3015f840 con 0x7f7e3006f660 2024-05-28T14:01:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.428+0000 7f7e2f400640 1 -- 172.21.15.42:0/298263005 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1625092114 0 0) 0x7f7e3015f280 con 0x7f7e300d0e00 2024-05-28T14:01:51.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.432+0000 7f7e35000640 1 --2- 172.21.15.42:0/298263005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e3006f660 0x7f7e30073980 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7e240097c0 tx=0x7f7e2401c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e855c2ad0b6c8d76 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:51.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.432+0000 7f7e2f400640 1 -- 172.21.15.42:0/298263005 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e240b0070 con 0x7f7e3006f660 2024-05-28T14:01:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.432+0000 7f7e2f400640 1 -- 172.21.15.42:0/298263005 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e240ab600 con 0x7f7e3006f660 2024-05-28T14:01:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.432+0000 7f7e2f400640 1 -- 172.21.15.42:0/298263005 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e240ab9d0 con 0x7f7e3006f660 2024-05-28T14:01:51.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.432+0000 7f7e37a00640 1 -- 172.21.15.42:0/298263005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e3006f660 msgr2=0x7f7e30073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.432+0000 7f7e37a00640 1 --2- 172.21.15.42:0/298263005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e3006f660 0x7f7e30073980 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7e240097c0 tx=0x7f7e2401c350 comp rx=0 tx=0).stop 2024-05-28T14:01:51.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.432+0000 7f7e37a00640 1 -- 172.21.15.42:0/298263005 shutdown_connections 2024-05-28T14:01:51.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.432+0000 7f7e37a00640 1 --2- 172.21.15.42:0/298263005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e3006f660 0x7f7e30073980 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.432+0000 7f7e37a00640 1 -- 172.21.15.42:0/298263005 >> 172.21.15.42:0/298263005 conn(0x7f7e3006bdf0 msgr2=0x7f7e3006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:51.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.432+0000 7f7e37a00640 1 -- 172.21.15.42:0/298263005 shutdown_connections 2024-05-28T14:01:51.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.432+0000 7f7e37a00640 1 -- 172.21.15.42:0/298263005 wait complete. 2024-05-28T14:01:51.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.432+0000 7f7e37a00640 1 Processor -- start 2024-05-28T14:01:51.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e37a00640 1 -- start start 2024-05-28T14:01:51.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e37a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e3006f660 0x7f7e301471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:51.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e37a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e300d0e00 0x7f7e300ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:51.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e37a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e300ceef0 0x7f7e3014fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:51.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e37a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7e30169560 con 0x7f7e300ceef0 2024-05-28T14:01:51.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e37a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7e301693e0 con 0x7f7e3006f660 2024-05-28T14:01:51.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e37a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7e301696e0 con 0x7f7e300d0e00 2024-05-28T14:01:51.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e35a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e300ceef0 0x7f7e3014fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:51.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e35000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e3006f660 0x7f7e301471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:51.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e35a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e300ceef0 0x7f7e3014fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55108/0 (socket says 172.21.15.42:55108) 2024-05-28T14:01:51.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e35a00640 1 -- 172.21.15.42:0/507404187 learned_addr learned my addr 172.21.15.42:0/507404187 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:51.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e2fe00640 1 --2- 172.21.15.42:0/507404187 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e300d0e00 0x7f7e300ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:51.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e35a00640 1 -- 172.21.15.42:0/507404187 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e300d0e00 msgr2=0x7f7e300ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e35a00640 1 --2- 172.21.15.42:0/507404187 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e300d0e00 0x7f7e300ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e35a00640 1 -- 172.21.15.42:0/507404187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e3006f660 msgr2=0x7f7e301471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e35a00640 1 --2- 172.21.15.42:0/507404187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e3006f660 0x7f7e301471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e35a00640 1 -- 172.21.15.42:0/507404187 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e300cec90 con 0x7f7e300ceef0 2024-05-28T14:01:51.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e35a00640 1 --2- 172.21.15.42:0/507404187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e300ceef0 0x7f7e3014fd30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800c7d0 tx=0x7f7e2800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e2cc00640 1 -- 172.21.15.42:0/507404187 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28018070 con 0x7f7e300ceef0 2024-05-28T14:01:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e2cc00640 1 -- 172.21.15.42:0/507404187 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e2800a640 con 0x7f7e300ceef0 2024-05-28T14:01:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e37a00640 1 -- 172.21.15.42:0/507404187 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e300cf3d0 con 0x7f7e300ceef0 2024-05-28T14:01:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e2cc00640 1 -- 172.21.15.42:0/507404187 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800aa70 con 0x7f7e300ceef0 2024-05-28T14:01:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e37a00640 1 -- 172.21.15.42:0/507404187 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e301f8940 con 0x7f7e300ceef0 2024-05-28T14:01:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.436+0000 7f7e37a00640 1 -- 172.21.15.42:0/507404187 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e30068a90 con 0x7f7e300ceef0 2024-05-28T14:01:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.440+0000 7f7e2cc00640 1 -- 172.21.15.42:0/507404187 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f7e2801b070 con 0x7f7e300ceef0 2024-05-28T14:01:51.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.440+0000 7f7e2cc00640 1 --2- 172.21.15.42:0/507404187 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e10068650 0x7f7e1006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:51.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.440+0000 7f7e2cc00640 1 -- 172.21.15.42:0/507404187 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5737+0+0 (secure 0 0 0) 0x7f7e28086360 con 0x7f7e300ceef0 2024-05-28T14:01:51.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.440+0000 7f7e35000640 1 --2- 172.21.15.42:0/507404187 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e10068650 0x7f7e1006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:51.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.440+0000 7f7e35000640 1 --2- 172.21.15.42:0/507404187 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e10068650 0x7f7e1006ab10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7e240ad000 tx=0x7f7e240a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:51.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.444+0000 7f7e2cc00640 1 -- 172.21.15.42:0/507404187 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7e28055f20 con 0x7f7e300ceef0 2024-05-28T14:01:51.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.608+0000 7f7e37a00640 1 -- 172.21.15.42:0/507404187 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e30072bb0 con 0x7f7e300ceef0 2024-05-28T14:01:51.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.608+0000 7f7e2cc00640 1 -- 172.21.15.42:0/507404187 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16609 (secure 0 0 0) 0x7f7e2805a220 con 0x7f7e300ceef0 2024-05-28T14:01:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:51.594 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":45,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:01:50.630771+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:01:49.041938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-28T14:01:49.591543+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:01:51.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.608+0000 7f7e37a00640 1 -- 172.21.15.42:0/507404187 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e10068650 msgr2=0x7f7e1006ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.608+0000 7f7e37a00640 1 --2- 172.21.15.42:0/507404187 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e10068650 0x7f7e1006ab10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7e240ad000 tx=0x7f7e240a6000 comp rx=0 tx=0).stop 2024-05-28T14:01:51.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.608+0000 7f7e37a00640 1 -- 172.21.15.42:0/507404187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e300ceef0 msgr2=0x7f7e3014fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.608+0000 7f7e37a00640 1 --2- 172.21.15.42:0/507404187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e300ceef0 0x7f7e3014fd30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800c7d0 tx=0x7f7e2800cc90 comp rx=0 tx=0).stop 2024-05-28T14:01:51.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.612+0000 7f7e37a00640 1 -- 172.21.15.42:0/507404187 shutdown_connections 2024-05-28T14:01:51.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.612+0000 7f7e37a00640 1 --2- 172.21.15.42:0/507404187 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e10068650 0x7f7e1006ab10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.612+0000 7f7e37a00640 1 --2- 172.21.15.42:0/507404187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e300ceef0 0x7f7e3014fd30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.612+0000 7f7e37a00640 1 --2- 172.21.15.42:0/507404187 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e300d0e00 0x7f7e300ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.612+0000 7f7e37a00640 1 --2- 172.21.15.42:0/507404187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e3006f660 0x7f7e301471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.612+0000 7f7e37a00640 1 -- 172.21.15.42:0/507404187 >> 172.21.15.42:0/507404187 conn(0x7f7e3006bdf0 msgr2=0x7f7e300732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:51.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.612+0000 7f7e37a00640 1 -- 172.21.15.42:0/507404187 shutdown_connections 2024-05-28T14:01:51.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.612+0000 7f7e37a00640 1 -- 172.21.15.42:0/507404187 wait complete. 2024-05-28T14:01:51.616 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:01:51.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.772+0000 7f131e800640 1 Processor -- start 2024-05-28T14:01:51.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.772+0000 7f131e800640 1 -- start start 2024-05-28T14:01:51.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f131e800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f131806c940 0x7f131809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:51.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f131e800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f131809af90 0x7f1318070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:51.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f131e800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f131806ff10 con 0x7f131806e740 2024-05-28T14:01:51.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f131e800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f131806eb20 con 0x7f131809af90 2024-05-28T14:01:51.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f131e800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f13180701b0 con 0x7f131806c940 2024-05-28T14:01:51.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f1317e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f131809af90 0x7f1318070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:51.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f1317400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f131806e740 0x7f13180cdbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36772/0 (socket says 172.21.15.42:36772) 2024-05-28T14:01:51.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f1317400640 1 -- 172.21.15.42:0/4005131325 learned_addr learned my addr 172.21.15.42:0/4005131325 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:51.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f130fe00640 1 --2- 172.21.15.42:0/4005131325 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f131806c940 0x7f131809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:51.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f1316a00640 1 -- 172.21.15.42:0/4005131325 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1532489796 0 0) 0x7f131806ff10 con 0x7f131806e740 2024-05-28T14:01:51.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f1316a00640 1 -- 172.21.15.42:0/4005131325 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12fc003530 con 0x7f131806e740 2024-05-28T14:01:51.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f1317e00640 1 -- 172.21.15.42:0/4005131325 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f131806c940 msgr2=0x7f131809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:51.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f1317e00640 1 --2- 172.21.15.42:0/4005131325 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f131806c940 0x7f131809aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f1317e00640 1 -- 172.21.15.42:0/4005131325 >> v1:172.21.15.42:6789/0 conn(0x7f131806e740 legacy=0x7f13180cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.776+0000 7f1317e00640 1 -- 172.21.15.42:0/4005131325 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1318070490 con 0x7f131809af90 2024-05-28T14:01:51.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f1317e00640 1 --2- 172.21.15.42:0/4005131325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131809af90 0x7f1318070f70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1310009d90 tx=0x7f131001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299f35948071286b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f1316a00640 1 -- 172.21.15.42:0/4005131325 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13100b2070 con 0x7f131809af90 2024-05-28T14:01:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f1316a00640 1 -- 172.21.15.42:0/4005131325 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13100a5890 con 0x7f131809af90 2024-05-28T14:01:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f1316a00640 1 -- 172.21.15.42:0/4005131325 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13100a5c60 con 0x7f131809af90 2024-05-28T14:01:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 -- 172.21.15.42:0/4005131325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131809af90 msgr2=0x7f1318070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 --2- 172.21.15.42:0/4005131325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131809af90 0x7f1318070f70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1310009d90 tx=0x7f131001c8e0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 -- 172.21.15.42:0/4005131325 shutdown_connections 2024-05-28T14:01:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 --2- 172.21.15.42:0/4005131325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131809af90 0x7f1318070f70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 --2- 172.21.15.42:0/4005131325 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f131806c940 0x7f131809aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 -- 172.21.15.42:0/4005131325 >> 172.21.15.42:0/4005131325 conn(0x7f13180b3fa0 msgr2=0x7f13180b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:51.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 -- 172.21.15.42:0/4005131325 shutdown_connections 2024-05-28T14:01:51.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 -- 172.21.15.42:0/4005131325 wait complete. 2024-05-28T14:01:51.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 Processor -- start 2024-05-28T14:01:51.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 -- start start 2024-05-28T14:01:51.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f131806c940 0x7f13180c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:51.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f131806e740 0x7f13180cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:51.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131809af90 0x7f1318155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:51.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f131815e120 con 0x7f131809af90 2024-05-28T14:01:51.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f131809baa0 con 0x7f131806e740 2024-05-28T14:01:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.780+0000 7f131e800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f131815e2a0 con 0x7f131806c940 2024-05-28T14:01:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f1317e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131809af90 0x7f1318155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f1317e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131809af90 0x7f1318155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55132/0 (socket says 172.21.15.42:55132) 2024-05-28T14:01:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f1317e00640 1 -- 172.21.15.42:0/2513329367 learned_addr learned my addr 172.21.15.42:0/2513329367 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f1317400640 1 --2- 172.21.15.42:0/2513329367 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f131806c940 0x7f13180c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f130fe00640 1 --2- 172.21.15.42:0/2513329367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f131806e740 0x7f13180cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f1317e00640 1 -- 172.21.15.42:0/2513329367 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f131806c940 msgr2=0x7f13180c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f1317e00640 1 --2- 172.21.15.42:0/2513329367 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f131806c940 0x7f13180c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f1317e00640 1 -- 172.21.15.42:0/2513329367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f131806e740 msgr2=0x7f13180cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f1317e00640 1 --2- 172.21.15.42:0/2513329367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f131806e740 0x7f13180cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f1317e00640 1 -- 172.21.15.42:0/2513329367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13180ca690 con 0x7f131809af90 2024-05-28T14:01:51.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f1317e00640 1 --2- 172.21.15.42:0/2513329367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131809af90 0x7f1318155900 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f13100b0040 tx=0x7f1310019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:51.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f130f400640 1 -- 172.21.15.42:0/2513329367 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13100bd020 con 0x7f131809af90 2024-05-28T14:01:51.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f130f400640 1 -- 172.21.15.42:0/2513329367 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13100b8470 con 0x7f131809af90 2024-05-28T14:01:51.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f130f400640 1 -- 172.21.15.42:0/2513329367 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13100b2040 con 0x7f131809af90 2024-05-28T14:01:51.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f131e800640 1 -- 172.21.15.42:0/2513329367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13180ca970 con 0x7f131809af90 2024-05-28T14:01:51.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f131e800640 1 -- 172.21.15.42:0/2513329367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13180caf30 con 0x7f131809af90 2024-05-28T14:01:51.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f131e800640 1 -- 172.21.15.42:0/2513329367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1318068a90 con 0x7f131809af90 2024-05-28T14:01:51.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.784+0000 7f130f400640 1 -- 172.21.15.42:0/2513329367 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f13100c0020 con 0x7f131809af90 2024-05-28T14:01:51.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.788+0000 7f130f400640 1 --2- 172.21.15.42:0/2513329367 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12f4068650 0x7f12f406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:51.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.788+0000 7f130f400640 1 -- 172.21.15.42:0/2513329367 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5737+0+0 (secure 0 0 0) 0x7f1310121c80 con 0x7f131809af90 2024-05-28T14:01:51.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.788+0000 7f1317400640 1 --2- 172.21.15.42:0/2513329367 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12f4068650 0x7f12f406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:51.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.788+0000 7f1317400640 1 --2- 172.21.15.42:0/2513329367 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12f4068650 0x7f12f406ab10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1308005890 tx=0x7f1308009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.792+0000 7f130f400640 1 -- 172.21.15.42:0/2513329367 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f13100c7030 con 0x7f131809af90 2024-05-28T14:01:51.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.960+0000 7f131e800640 1 -- 172.21.15.42:0/2513329367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13180b2870 con 0x7f131809af90 2024-05-28T14:01:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.960+0000 7f130f400640 1 -- 172.21.15.42:0/2513329367 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16609 (secure 0 0 0) 0x7f1310016560 con 0x7f131809af90 2024-05-28T14:01:51.946 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:51.946 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":45,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:01:50.630771+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:01:49.041938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-28T14:01:49.591543+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:01:51.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.960+0000 7f131e800640 1 -- 172.21.15.42:0/2513329367 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12f4068650 msgr2=0x7f12f406ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.960+0000 7f131e800640 1 --2- 172.21.15.42:0/2513329367 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12f4068650 0x7f12f406ab10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1308005890 tx=0x7f1308009040 comp rx=0 tx=0).stop 2024-05-28T14:01:51.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.960+0000 7f131e800640 1 -- 172.21.15.42:0/2513329367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131809af90 msgr2=0x7f1318155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:51.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.960+0000 7f131e800640 1 --2- 172.21.15.42:0/2513329367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131809af90 0x7f1318155900 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f13100b0040 tx=0x7f1310019680 comp rx=0 tx=0).stop 2024-05-28T14:01:51.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.960+0000 7f131e800640 1 -- 172.21.15.42:0/2513329367 shutdown_connections 2024-05-28T14:01:51.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.960+0000 7f131e800640 1 --2- 172.21.15.42:0/2513329367 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12f4068650 0x7f12f406ab10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.960+0000 7f131e800640 1 --2- 172.21.15.42:0/2513329367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131809af90 0x7f1318155900 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.960+0000 7f131e800640 1 --2- 172.21.15.42:0/2513329367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f131806e740 0x7f13180cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.960+0000 7f131e800640 1 --2- 172.21.15.42:0/2513329367 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f131806c940 0x7f13180c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:51.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.960+0000 7f131e800640 1 -- 172.21.15.42:0/2513329367 >> 172.21.15.42:0/2513329367 conn(0x7f13180b3fa0 msgr2=0x7f13180cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:51.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.960+0000 7f131e800640 1 -- 172.21.15.42:0/2513329367 shutdown_connections 2024-05-28T14:01:51.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:51.960+0000 7f131e800640 1 -- 172.21.15.42:0/2513329367 wait complete. 2024-05-28T14:01:51.967 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-28T14:01:52.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.124+0000 7f2b91c00640 1 Processor -- start 2024-05-28T14:01:52.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.124+0000 7f2b91c00640 1 -- start start 2024-05-28T14:01:52.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b91c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2b8c140d20 0x7f2b8c167490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:52.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b91c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b8c160230 con 0x7f2b8c0ce3b0 2024-05-28T14:01:52.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b91c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b8c160510 con 0x7f2b8c071de0 2024-05-28T14:01:52.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b91c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2b8c1607f0 con 0x7f2b8c140d20 2024-05-28T14:01:52.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b82000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f2b8c0ce3b0 0x7f2b8c0ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36776/0 (socket says 172.21.15.42:36776) 2024-05-28T14:01:52.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b82000640 1 -- 172.21.15.42:0/244414473 learned_addr learned my addr 172.21.15.42:0/244414473 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:52.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8b400640 1 --2- 172.21.15.42:0/244414473 >> v2:172.21.15.63:3300/0 conn(0x7f2b8c140d20 0x7f2b8c167490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:52.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 -- 172.21.15.42:0/244414473 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 777715834 0 0) 0x7f2b8c160230 con 0x7f2b8c0ce3b0 2024-05-28T14:01:52.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 -- 172.21.15.42:0/244414473 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b70003530 con 0x7f2b8c0ce3b0 2024-05-28T14:01:52.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 -- 172.21.15.42:0/244414473 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4070443596 0 0) 0x7f2b70003530 con 0x7f2b8c0ce3b0 2024-05-28T14:01:52.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 -- 172.21.15.42:0/244414473 >> v2:172.21.15.63:3300/0 conn(0x7f2b8c140d20 msgr2=0x7f2b8c167490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:52.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 --2- 172.21.15.42:0/244414473 >> v2:172.21.15.63:3300/0 conn(0x7f2b8c140d20 0x7f2b8c167490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:52.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 -- 172.21.15.42:0/244414473 >> v1:172.21.15.63:6789/0 conn(0x7f2b8c071de0 legacy=0x7f2b8c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:52.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 -- 172.21.15.42:0/244414473 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c160ad0 con 0x7f2b8c0ce3b0 2024-05-28T14:01:52.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 -- 172.21.15.42:0/244414473 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f2b74001680 con 0x7f2b8c0ce3b0 2024-05-28T14:01:52.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 -- 172.21.15.42:0/244414473 >> v1:172.21.15.42:6789/0 conn(0x7f2b8c0ce3b0 legacy=0x7f2b8c0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:52.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 --2- 172.21.15.42:0/244414473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b700039c0 0x7f2b70003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:52.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 --2- 172.21.15.42:0/244414473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b70004810 0x7f2b70004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:52.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 --2- 172.21.15.42:0/244414473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b700051d0 0x7f2b7000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:52.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 -- 172.21.15.42:0/244414473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1607f0 con 0x7f2b700039c0 2024-05-28T14:01:52.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 -- 172.21.15.42:0/244414473 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7000c120 con 0x7f2b700051d0 2024-05-28T14:01:52.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8a000640 1 -- 172.21.15.42:0/244414473 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7000c320 con 0x7f2b70004810 2024-05-28T14:01:52.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8aa00640 1 --2- 172.21.15.42:0/244414473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b700039c0 0x7f2b70003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:52.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b82000640 1 --2- 172.21.15.42:0/244414473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b70004810 0x7f2b70004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:52.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8b400640 1 --2- 172.21.15.42:0/244414473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b700051d0 0x7f2b7000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:52.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8aa00640 1 -- 172.21.15.42:0/244414473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b70004810 msgr2=0x7f2b70004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:52.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8aa00640 1 --2- 172.21.15.42:0/244414473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b70004810 0x7f2b70004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:52.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8aa00640 1 -- 172.21.15.42:0/244414473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b700051d0 msgr2=0x7f2b7000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:52.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8aa00640 1 --2- 172.21.15.42:0/244414473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b700051d0 0x7f2b7000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:52.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8aa00640 1 -- 172.21.15.42:0/244414473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b70003530 con 0x7f2b700039c0 2024-05-28T14:01:52.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b82000640 1 --2- 172.21.15.42:0/244414473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b70004810 0x7f2b70004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:52.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8b400640 1 --2- 172.21.15.42:0/244414473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b700051d0 0x7f2b7000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:52.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.128+0000 7f2b8b400640 1 -- 172.21.15.42:0/244414473 reap_dead start 2024-05-28T14:01:52.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.132+0000 7f2b8aa00640 1 --2- 172.21.15.42:0/244414473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b700039c0 0x7f2b70003da0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c001910 tx=0x7f2b7c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d30e2fe348dcc server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:52.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.132+0000 7f2b8a000640 1 -- 172.21.15.42:0/244414473 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c0b0070 con 0x7f2b700039c0 2024-05-28T14:01:52.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.132+0000 7f2b8a000640 1 -- 172.21.15.42:0/244414473 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b7c0ab620 con 0x7f2b700039c0 2024-05-28T14:01:52.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.132+0000 7f2b8a000640 1 -- 172.21.15.42:0/244414473 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c001510 con 0x7f2b700039c0 2024-05-28T14:01:52.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.132+0000 7f2b91c00640 1 -- 172.21.15.42:0/244414473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b700039c0 msgr2=0x7f2b70003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:52.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.132+0000 7f2b91c00640 1 --2- 172.21.15.42:0/244414473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b700039c0 0x7f2b70003da0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c001910 tx=0x7f2b7c01c560 comp rx=0 tx=0).stop 2024-05-28T14:01:52.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.132+0000 7f2b91c00640 1 -- 172.21.15.42:0/244414473 shutdown_connections 2024-05-28T14:01:52.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.132+0000 7f2b91c00640 1 --2- 172.21.15.42:0/244414473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b700039c0 0x7f2b70003da0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:52.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.132+0000 7f2b91c00640 1 -- 172.21.15.42:0/244414473 >> 172.21.15.42:0/244414473 conn(0x7f2b8c0b21f0 msgr2=0x7f2b8c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:52.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.132+0000 7f2b91c00640 1 -- 172.21.15.42:0/244414473 shutdown_connections 2024-05-28T14:01:52.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.132+0000 7f2b91c00640 1 -- 172.21.15.42:0/244414473 wait complete. 2024-05-28T14:01:52.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.132+0000 7f2b91c00640 1 Processor -- start 2024-05-28T14:01:52.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b91c00640 1 -- start start 2024-05-28T14:01:52.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b91c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b700039c0 0x7f2b8c15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:52.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b91c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b8c13dda0 0x7f2b8c15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:52.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b91c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b8c15f220 0x7f2b8c155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:52.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b91c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1420a0 con 0x7f2b8c13dda0 2024-05-28T14:01:52.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b91c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c141f20 con 0x7f2b700039c0 2024-05-28T14:01:52.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b91c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b8c142220 con 0x7f2b8c15f220 2024-05-28T14:01:52.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b82000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b8c13dda0 0x7f2b8c15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:52.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b82000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b8c13dda0 0x7f2b8c15ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55150/0 (socket says 172.21.15.42:55150) 2024-05-28T14:01:52.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b82000640 1 -- 172.21.15.42:0/2333589561 learned_addr learned my addr 172.21.15.42:0/2333589561 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:52.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b8aa00640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b700039c0 0x7f2b8c15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:52.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b8b400640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b8c15f220 0x7f2b8c155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:52.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b82000640 1 -- 172.21.15.42:0/2333589561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b8c15f220 msgr2=0x7f2b8c155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:52.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b82000640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b8c15f220 0x7f2b8c155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:52.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b82000640 1 -- 172.21.15.42:0/2333589561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b700039c0 msgr2=0x7f2b8c15e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:52.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b82000640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b700039c0 0x7f2b8c15e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:52.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b82000640 1 -- 172.21.15.42:0/2333589561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c15d310 con 0x7f2b8c13dda0 2024-05-28T14:01:52.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b8b400640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b8c15f220 0x7f2b8c155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:52.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b82000640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b8c13dda0 0x7f2b8c15ca10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2b74001cf0 tx=0x7f2b7400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:52.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b83400640 1 -- 172.21.15.42:0/2333589561 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7400ee00 con 0x7f2b8c13dda0 2024-05-28T14:01:52.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b83400640 1 -- 172.21.15.42:0/2333589561 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b74018070 con 0x7f2b8c13dda0 2024-05-28T14:01:52.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b83400640 1 -- 172.21.15.42:0/2333589561 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7400ca70 con 0x7f2b8c13dda0 2024-05-28T14:01:52.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b91c00640 1 -- 172.21.15.42:0/2333589561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8c15d5f0 con 0x7f2b8c13dda0 2024-05-28T14:01:52.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b91c00640 1 -- 172.21.15.42:0/2333589561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8c15dbb0 con 0x7f2b8c13dda0 2024-05-28T14:01:52.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.136+0000 7f2b91c00640 1 -- 172.21.15.42:0/2333589561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b8c0738b0 con 0x7f2b8c13dda0 2024-05-28T14:01:52.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.140+0000 7f2b83400640 1 -- 172.21.15.42:0/2333589561 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f2b7400cc10 con 0x7f2b8c13dda0 2024-05-28T14:01:52.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.140+0000 7f2b83400640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b68068650 0x7f2b6806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:52.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.140+0000 7f2b83400640 1 -- 172.21.15.42:0/2333589561 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5737+0+0 (secure 0 0 0) 0x7f2b74086b70 con 0x7f2b8c13dda0 2024-05-28T14:01:52.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.140+0000 7f2b8aa00640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b68068650 0x7f2b6806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:52.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.140+0000 7f2b8aa00640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b68068650 0x7f2b6806ab10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0ad000 tx=0x7f2b7c001330 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:52.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.144+0000 7f2b83400640 1 -- 172.21.15.42:0/2333589561 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2b74056d30 con 0x7f2b8c13dda0 2024-05-28T14:01:52.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:52.348+0000 7f2b91c00640 1 -- 172.21.15.42:0/2333589561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2b8c04fdd0 con 0x7f2b8c13dda0 2024-05-28T14:01:53.201 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-28T14:01:53.214+0000 7f1a0d400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-28T14:01:53.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.272+0000 7f2b83400640 1 -- 172.21.15.42:0/2333589561 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f2b74056630 con 0x7f2b8c13dda0 2024-05-28T14:01:53.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.272+0000 7f2b91c00640 1 -- 172.21.15.42:0/2333589561 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b68068650 msgr2=0x7f2b6806ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:53.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.272+0000 7f2b91c00640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b68068650 0x7f2b6806ab10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0ad000 tx=0x7f2b7c001330 comp rx=0 tx=0).stop 2024-05-28T14:01:53.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.272+0000 7f2b91c00640 1 -- 172.21.15.42:0/2333589561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b8c13dda0 msgr2=0x7f2b8c15ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:53.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.272+0000 7f2b91c00640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b8c13dda0 0x7f2b8c15ca10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2b74001cf0 tx=0x7f2b7400ec10 comp rx=0 tx=0).stop 2024-05-28T14:01:53.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.272+0000 7f2b91c00640 1 -- 172.21.15.42:0/2333589561 shutdown_connections 2024-05-28T14:01:53.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.272+0000 7f2b91c00640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b68068650 0x7f2b6806ab10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:53.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.272+0000 7f2b91c00640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b8c15f220 0x7f2b8c155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:53.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.272+0000 7f2b91c00640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b8c13dda0 0x7f2b8c15ca10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:53.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.272+0000 7f2b91c00640 1 --2- 172.21.15.42:0/2333589561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b700039c0 0x7f2b8c15e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:53.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.272+0000 7f2b91c00640 1 -- 172.21.15.42:0/2333589561 >> 172.21.15.42:0/2333589561 conn(0x7f2b8c0b21f0 msgr2=0x7f2b8c145870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:53.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.272+0000 7f2b91c00640 1 -- 172.21.15.42:0/2333589561 shutdown_connections 2024-05-28T14:01:53.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.272+0000 7f2b91c00640 1 -- 172.21.15.42:0/2333589561 wait complete. 2024-05-28T14:01:53.282 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-28T14:01:53.283 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-28T14:01:53.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.444+0000 7faccfe00640 1 Processor -- start 2024-05-28T14:01:53.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.444+0000 7faccfe00640 1 -- start start 2024-05-28T14:01:53.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.448+0000 7faccfe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7facc8071de0 0x7facc80721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:53.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.448+0000 7faccfe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facc80ce3b0 0x7facc80ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:53.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.448+0000 7faccfe00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facc8072700 con 0x7facc8141d10 2024-05-28T14:01:53.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.448+0000 7faccfe00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7facc8160230 con 0x7facc8071de0 2024-05-28T14:01:53.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.448+0000 7faccfe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7facc8160510 con 0x7facc80ce3b0 2024-05-28T14:01:53.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.448+0000 7faccde00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7facc8141d10 0x7facc81421a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36780/0 (socket says 172.21.15.42:36780) 2024-05-28T14:01:53.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.448+0000 7faccde00640 1 -- 172.21.15.42:0/492817448 learned_addr learned my addr 172.21.15.42:0/492817448 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:53.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.448+0000 7faccd400640 1 --2- 172.21.15.42:0/492817448 >> v2:172.21.15.42:3300/0 conn(0x7facc8071de0 0x7facc80721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:53.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.448+0000 7faccca00640 1 --2- 172.21.15.42:0/492817448 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facc80ce3b0 0x7facc80ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:53.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.448+0000 7faccd400640 1 -- 172.21.15.42:0/492817448 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facc80ce3b0 msgr2=0x7facc80ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:53.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.448+0000 7facc7e00640 1 -- 172.21.15.42:0/492817448 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1259683193 0 0) 0x7facc8072700 con 0x7facc8141d10 2024-05-28T14:01:53.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.448+0000 7faccd400640 1 --2- 172.21.15.42:0/492817448 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facc80ce3b0 0x7facc80ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:53.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.448+0000 7faccd400640 1 -- 172.21.15.42:0/492817448 >> v1:172.21.15.42:6789/0 conn(0x7facc8141d10 legacy=0x7facc81421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:53.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.448+0000 7faccd400640 1 -- 172.21.15.42:0/492817448 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc81607f0 con 0x7facc8071de0 2024-05-28T14:01:53.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.452+0000 7faccd400640 1 --2- 172.21.15.42:0/492817448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc8071de0 0x7facc80721c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7facbc0097c0 tx=0x7facbc01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101f9288511c8726 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:53.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.452+0000 7facc7e00640 1 -- 172.21.15.42:0/492817448 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc0b0070 con 0x7facc8071de0 2024-05-28T14:01:53.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.452+0000 7facc7e00640 1 -- 172.21.15.42:0/492817448 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facbc0ab600 con 0x7facc8071de0 2024-05-28T14:01:53.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.452+0000 7facc7e00640 1 -- 172.21.15.42:0/492817448 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc0ab9d0 con 0x7facc8071de0 2024-05-28T14:01:53.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.452+0000 7faccfe00640 1 -- 172.21.15.42:0/492817448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc8071de0 msgr2=0x7facc80721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:53.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.452+0000 7faccfe00640 1 --2- 172.21.15.42:0/492817448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc8071de0 0x7facc80721c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7facbc0097c0 tx=0x7facbc01c380 comp rx=0 tx=0).stop 2024-05-28T14:01:53.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.452+0000 7faccfe00640 1 -- 172.21.15.42:0/492817448 shutdown_connections 2024-05-28T14:01:53.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.452+0000 7faccfe00640 1 --2- 172.21.15.42:0/492817448 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facc80ce3b0 0x7facc80ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:53.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.452+0000 7faccfe00640 1 --2- 172.21.15.42:0/492817448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc8071de0 0x7facc80721c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:53.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.452+0000 7faccfe00640 1 -- 172.21.15.42:0/492817448 >> 172.21.15.42:0/492817448 conn(0x7facc80b21f0 msgr2=0x7facc80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:53.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.452+0000 7faccfe00640 1 -- 172.21.15.42:0/492817448 shutdown_connections 2024-05-28T14:01:53.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.452+0000 7faccfe00640 1 -- 172.21.15.42:0/492817448 wait complete. 2024-05-28T14:01:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.452+0000 7faccfe00640 1 Processor -- start 2024-05-28T14:01:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.452+0000 7faccfe00640 1 -- start start 2024-05-28T14:01:53.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccfe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facc8071de0 0x7facc813cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:53.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc80ce3b0 0x7facc81fb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:53.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facc8141d10 0x7facc81fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:53.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7facc8169e30 con 0x7facc80ce3b0 2024-05-28T14:01:53.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccfe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7facc8169cb0 con 0x7facc8141d10 2024-05-28T14:01:53.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccfe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7facc8169fb0 con 0x7facc8071de0 2024-05-28T14:01:53.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc80ce3b0 0x7facc81fb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:53.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc80ce3b0 0x7facc81fb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55176/0 (socket says 172.21.15.42:55176) 2024-05-28T14:01:53.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccca00640 1 -- 172.21.15.42:0/2149415364 learned_addr learned my addr 172.21.15.42:0/2149415364 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccd400640 1 --2- 172.21.15.42:0/2149415364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facc8071de0 0x7facc813cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccde00640 1 --2- 172.21.15.42:0/2149415364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facc8141d10 0x7facc81fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccca00640 1 -- 172.21.15.42:0/2149415364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facc8071de0 msgr2=0x7facc813cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccca00640 1 --2- 172.21.15.42:0/2149415364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facc8071de0 0x7facc813cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccca00640 1 -- 172.21.15.42:0/2149415364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facc8141d10 msgr2=0x7facc81fbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccca00640 1 --2- 172.21.15.42:0/2149415364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facc8141d10 0x7facc81fbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccca00640 1 -- 172.21.15.42:0/2149415364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc80b4c00 con 0x7facc80ce3b0 2024-05-28T14:01:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccca00640 1 --2- 172.21.15.42:0/2149415364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc80ce3b0 0x7facc81fb6e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7facb400e870 tx=0x7facb400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7facc5600640 1 -- 172.21.15.42:0/2149415364 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4017070 con 0x7facc80ce3b0 2024-05-28T14:01:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7facc5600640 1 -- 172.21.15.42:0/2149415364 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb400c940 con 0x7facc80ce3b0 2024-05-28T14:01:53.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccfe00640 1 -- 172.21.15.42:0/2149415364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc80b4ee0 con 0x7facc80ce3b0 2024-05-28T14:01:53.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7facc5600640 1 -- 172.21.15.42:0/2149415364 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb400ccf0 con 0x7facc80ce3b0 2024-05-28T14:01:53.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccfe00640 1 -- 172.21.15.42:0/2149415364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc8139ee0 con 0x7facc80ce3b0 2024-05-28T14:01:53.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7faccfe00640 1 -- 172.21.15.42:0/2149415364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc80b55a0 con 0x7facc80ce3b0 2024-05-28T14:01:53.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.456+0000 7facc5600640 1 -- 172.21.15.42:0/2149415364 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7facb401a070 con 0x7facc80ce3b0 2024-05-28T14:01:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.460+0000 7facc5600640 1 --2- 172.21.15.42:0/2149415364 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faca8068650 0x7faca806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.460+0000 7facc5600640 1 -- 172.21.15.42:0/2149415364 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7facb4087040 con 0x7facc80ce3b0 2024-05-28T14:01:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.460+0000 7faccd400640 1 --2- 172.21.15.42:0/2149415364 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faca8068650 0x7faca806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.460+0000 7faccd400640 1 --2- 172.21.15.42:0/2149415364 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faca8068650 0x7faca806ab10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7facbc0ad000 tx=0x7facbc0a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:53.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.464+0000 7facc5600640 1 -- 172.21.15.42:0/2149415364 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7facc80b55a0 con 0x7facc80ce3b0 2024-05-28T14:01:53.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:53.664+0000 7faccfe00640 1 -- 172.21.15.42:0/2149415364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7facc80738b0 con 0x7facc80ce3b0 2024-05-28T14:01:54.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.272+0000 7facc5600640 1 -- 172.21.15.42:0/2149415364 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7facb40572e0 con 0x7facc80ce3b0 2024-05-28T14:01:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.276+0000 7faccfe00640 1 -- 172.21.15.42:0/2149415364 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faca8068650 msgr2=0x7faca806ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.276+0000 7faccfe00640 1 --2- 172.21.15.42:0/2149415364 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faca8068650 0x7faca806ab10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7facbc0ad000 tx=0x7facbc0a6000 comp rx=0 tx=0).stop 2024-05-28T14:01:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.276+0000 7faccfe00640 1 -- 172.21.15.42:0/2149415364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc80ce3b0 msgr2=0x7facc81fb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.276+0000 7faccfe00640 1 --2- 172.21.15.42:0/2149415364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc80ce3b0 0x7facc81fb6e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7facb400e870 tx=0x7facb400ec70 comp rx=0 tx=0).stop 2024-05-28T14:01:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.276+0000 7faccfe00640 1 -- 172.21.15.42:0/2149415364 shutdown_connections 2024-05-28T14:01:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.276+0000 7faccfe00640 1 --2- 172.21.15.42:0/2149415364 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faca8068650 0x7faca806ab10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.276+0000 7faccfe00640 1 --2- 172.21.15.42:0/2149415364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facc8141d10 0x7facc81fbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.276+0000 7faccfe00640 1 --2- 172.21.15.42:0/2149415364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc80ce3b0 0x7facc81fb6e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.276+0000 7faccfe00640 1 --2- 172.21.15.42:0/2149415364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facc8071de0 0x7facc813cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.276+0000 7faccfe00640 1 -- 172.21.15.42:0/2149415364 >> 172.21.15.42:0/2149415364 conn(0x7facc80b21f0 msgr2=0x7facc80b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:54.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.276+0000 7faccfe00640 1 -- 172.21.15.42:0/2149415364 shutdown_connections 2024-05-28T14:01:54.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.276+0000 7faccfe00640 1 -- 172.21.15.42:0/2149415364 wait complete. 2024-05-28T14:01:54.285 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-28T14:01:54.285 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:01:54.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.444+0000 7fe8eaa00640 1 Processor -- start 2024-05-28T14:01:54.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.444+0000 7fe8eaa00640 1 -- start start 2024-05-28T14:01:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.444+0000 7fe8eaa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe8e415e970 0x7fe8e40d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.444+0000 7fe8eaa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe8e406ff60 0x7fe8e40ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.444+0000 7fe8eaa00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e4070740 con 0x7fe8e40cd040 2024-05-28T14:01:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.444+0000 7fe8eaa00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e40d0640 con 0x7fe8e406ff60 2024-05-28T14:01:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.444+0000 7fe8eaa00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e40d1160 con 0x7fe8e415e970 2024-05-28T14:01:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.448+0000 7fe8dbe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe8e406ff60 0x7fe8e40ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.448+0000 7fe8e8a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe8e40cd040 0x7fe8e4162500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36792/0 (socket says 172.21.15.42:36792) 2024-05-28T14:01:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.448+0000 7fe8dbe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe8e406ff60 0x7fe8e40ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55184/0 (socket says 172.21.15.42:55184) 2024-05-28T14:01:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.448+0000 7fe8e8a00640 1 -- 172.21.15.42:0/112754320 learned_addr learned my addr 172.21.15.42:0/112754320 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:54.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.448+0000 7fe8e3e00640 1 --2- 172.21.15.42:0/112754320 >> v2:172.21.15.63:3300/0 conn(0x7fe8e415e970 0x7fe8e40d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:54.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.448+0000 7fe8dbe00640 1 -- 172.21.15.42:0/112754320 >> v2:172.21.15.63:3300/0 conn(0x7fe8e415e970 msgr2=0x7fe8e40d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:54.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.448+0000 7fe8dbe00640 1 --2- 172.21.15.42:0/112754320 >> v2:172.21.15.63:3300/0 conn(0x7fe8e415e970 0x7fe8e40d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.448+0000 7fe8dbe00640 1 -- 172.21.15.42:0/112754320 >> v1:172.21.15.42:6789/0 conn(0x7fe8e40cd040 legacy=0x7fe8e4162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:54.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.448+0000 7fe8dbe00640 1 -- 172.21.15.42:0/112754320 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e40d0870 con 0x7fe8e406ff60 2024-05-28T14:01:54.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.448+0000 7fe8e3e00640 1 --2- 172.21.15.42:0/112754320 >> v2:172.21.15.63:3300/0 conn(0x7fe8e415e970 0x7fe8e40d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:54.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.448+0000 7fe8dbe00640 1 --2- 172.21.15.42:0/112754320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e406ff60 0x7fe8e40ccb00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40732d0 tx=0x7fe8cc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6452fe4ef8219efd server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:54.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.448+0000 7fe8e3400640 1 -- 172.21.15.42:0/112754320 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0b0070 con 0x7fe8e406ff60 2024-05-28T14:01:54.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.448+0000 7fe8e3400640 1 -- 172.21.15.42:0/112754320 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8cc0ab600 con 0x7fe8e406ff60 2024-05-28T14:01:54.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.448+0000 7fe8e3400640 1 -- 172.21.15.42:0/112754320 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0ab9d0 con 0x7fe8e406ff60 2024-05-28T14:01:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 -- 172.21.15.42:0/112754320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e406ff60 msgr2=0x7fe8e40ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 --2- 172.21.15.42:0/112754320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e406ff60 0x7fe8e40ccb00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40732d0 tx=0x7fe8cc01c3c0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 -- 172.21.15.42:0/112754320 shutdown_connections 2024-05-28T14:01:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 --2- 172.21.15.42:0/112754320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e406ff60 0x7fe8e40ccb00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 --2- 172.21.15.42:0/112754320 >> v2:172.21.15.63:3300/0 conn(0x7fe8e415e970 0x7fe8e40d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 -- 172.21.15.42:0/112754320 >> 172.21.15.42:0/112754320 conn(0x7fe8e40b3fa0 msgr2=0x7fe8e40b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 -- 172.21.15.42:0/112754320 shutdown_connections 2024-05-28T14:01:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 -- 172.21.15.42:0/112754320 wait complete. 2024-05-28T14:01:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 Processor -- start 2024-05-28T14:01:54.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 -- start start 2024-05-28T14:01:54.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8e406ff60 0x7fe8e415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:54.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e40cd040 0x7fe8e40b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:54.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8e415e970 0x7fe8e414f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:54.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e416b840 con 0x7fe8e40cd040 2024-05-28T14:01:54.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e40703c0 con 0x7fe8e406ff60 2024-05-28T14:01:54.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8eaa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e416b9c0 con 0x7fe8e415e970 2024-05-28T14:01:54.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8dbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e40cd040 0x7fe8e40b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:54.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8dbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e40cd040 0x7fe8e40b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55200/0 (socket says 172.21.15.42:55200) 2024-05-28T14:01:54.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8dbe00640 1 -- 172.21.15.42:0/3204118577 learned_addr learned my addr 172.21.15.42:0/3204118577 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:54.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8e3e00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8e406ff60 0x7fe8e415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:54.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8e8a00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8e415e970 0x7fe8e414f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:54.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8dbe00640 1 -- 172.21.15.42:0/3204118577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8e415e970 msgr2=0x7fe8e414f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:54.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8dbe00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8e415e970 0x7fe8e414f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8dbe00640 1 -- 172.21.15.42:0/3204118577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8e406ff60 msgr2=0x7fe8e415ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:54.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8dbe00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8e406ff60 0x7fe8e415ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8dbe00640 1 -- 172.21.15.42:0/3204118577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e40af400 con 0x7fe8e40cd040 2024-05-28T14:01:54.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.452+0000 7fe8e8a00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8e415e970 0x7fe8e414f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:54.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.456+0000 7fe8dbe00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e40cd040 0x7fe8e40b13a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0ae040 tx=0x7fe8cc019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:54.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.456+0000 7fe8e0c00640 1 -- 172.21.15.42:0/3204118577 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0b8020 con 0x7fe8e40cd040 2024-05-28T14:01:54.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.456+0000 7fe8e0c00640 1 -- 172.21.15.42:0/3204118577 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8cc0b4410 con 0x7fe8e40cd040 2024-05-28T14:01:54.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.456+0000 7fe8eaa00640 1 -- 172.21.15.42:0/3204118577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e40af6a0 con 0x7fe8e40cd040 2024-05-28T14:01:54.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.456+0000 7fe8e0c00640 1 -- 172.21.15.42:0/3204118577 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0b0040 con 0x7fe8e40cd040 2024-05-28T14:01:54.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.456+0000 7fe8eaa00640 1 -- 172.21.15.42:0/3204118577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e40afc60 con 0x7fe8e40cd040 2024-05-28T14:01:54.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.456+0000 7fe8eaa00640 1 -- 172.21.15.42:0/3204118577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e4068a90 con 0x7fe8e40cd040 2024-05-28T14:01:54.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.456+0000 7fe8e0c00640 1 -- 172.21.15.42:0/3204118577 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7fe8cc0bb020 con 0x7fe8e40cd040 2024-05-28T14:01:54.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.456+0000 7fe8e0c00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe8c0068650 0x7fe8c006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:54.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.456+0000 7fe8e0c00640 1 -- 172.21.15.42:0/3204118577 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7fe8cc120c30 con 0x7fe8e40cd040 2024-05-28T14:01:54.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.456+0000 7fe8e3e00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe8c0068650 0x7fe8c006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:54.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.460+0000 7fe8e3e00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe8c0068650 0x7fe8c006ab10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4006fd0 tx=0x7fe8d4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:54.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.460+0000 7fe8e0c00640 1 -- 172.21.15.42:0/3204118577 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe8cc0c2030 con 0x7fe8e40cd040 2024-05-28T14:01:54.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.660+0000 7fe8eaa00640 1 -- 172.21.15.42:0/3204118577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8e40d1fd0 con 0x7fe8e40cd040 2024-05-28T14:01:54.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.664+0000 7fe8e0c00640 1 -- 172.21.15.42:0/3204118577 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5180 (secure 0 0 0) 0x7fe8cc016070 con 0x7fe8e40cd040 2024-05-28T14:01:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:54.648 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3694688875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3694688875},{"type":"v1","addr":"172.21.15.63:6835","nonce":3694688875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5799,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/199639725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":199639725},{"type":"v1","addr":"172.21.15.42:6836","nonce":199639725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1043559054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1043559054},{"type":"v1","addr":"172.21.15.42:6837","nonce":1043559054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:01:50.630732+0000","modified":"2024-05-28T14:01:53.667008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":55184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55184":{"gid":55184,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6833/754542999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":754542999},{"type":"v1","addr":"172.21.15.63:6833","nonce":754542999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":55184,"qdb_cluster":[55184]},"id":3}]} 2024-05-28T14:01:54.649 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-05-28T14:01:54.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.664+0000 7fe8eaa00640 1 -- 172.21.15.42:0/3204118577 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe8c0068650 msgr2=0x7fe8c006ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:54.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.664+0000 7fe8eaa00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe8c0068650 0x7fe8c006ab10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4006fd0 tx=0x7fe8d4008040 comp rx=0 tx=0).stop 2024-05-28T14:01:54.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.664+0000 7fe8eaa00640 1 -- 172.21.15.42:0/3204118577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e40cd040 msgr2=0x7fe8e40b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:54.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.664+0000 7fe8eaa00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e40cd040 0x7fe8e40b13a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0ae040 tx=0x7fe8cc019160 comp rx=0 tx=0).stop 2024-05-28T14:01:54.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.664+0000 7fe8eaa00640 1 -- 172.21.15.42:0/3204118577 shutdown_connections 2024-05-28T14:01:54.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.664+0000 7fe8eaa00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe8c0068650 0x7fe8c006ab10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.664+0000 7fe8eaa00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8e415e970 0x7fe8e414f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.664+0000 7fe8eaa00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e40cd040 0x7fe8e40b13a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.664+0000 7fe8eaa00640 1 --2- 172.21.15.42:0/3204118577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8e406ff60 0x7fe8e415ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.664+0000 7fe8eaa00640 1 -- 172.21.15.42:0/3204118577 >> 172.21.15.42:0/3204118577 conn(0x7fe8e40b3fa0 msgr2=0x7fe8e415d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:54.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.664+0000 7fe8eaa00640 1 -- 172.21.15.42:0/3204118577 shutdown_connections 2024-05-28T14:01:54.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.664+0000 7fe8eaa00640 1 -- 172.21.15.42:0/3204118577 wait complete. 2024-05-28T14:01:54.670 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:01:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb80e000640 1 Processor -- start 2024-05-28T14:01:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb80e000640 1 -- start start 2024-05-28T14:01:54.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb80e000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb808155810 0x7fb8080cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb80e000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb80815b7d0 0x7fb80815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb80e000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb80815c290 con 0x7fb8080b4e00 2024-05-28T14:01:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb80e000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb8080cd840 con 0x7fb80815b7d0 2024-05-28T14:01:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb80e000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb80815c570 con 0x7fb808155810 2024-05-28T14:01:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb80d600640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb8080b4e00 0x7fb8080cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36800/0 (socket says 172.21.15.42:36800) 2024-05-28T14:01:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb80d600640 1 -- 172.21.15.42:0/1968532398 learned_addr learned my addr 172.21.15.42:0/1968532398 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb807e00640 1 --2- 172.21.15.42:0/1968532398 >> v2:172.21.15.63:3300/0 conn(0x7fb80815b7d0 0x7fb80815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb80cc00640 1 --2- 172.21.15.42:0/1968532398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb808155810 0x7fb8080cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb807400640 1 -- 172.21.15.42:0/1968532398 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2435277630 0 0) 0x7fb80815c290 con 0x7fb8080b4e00 2024-05-28T14:01:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb807400640 1 -- 172.21.15.42:0/1968532398 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7f0003530 con 0x7fb8080b4e00 2024-05-28T14:01:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb80cc00640 1 -- 172.21.15.42:0/1968532398 >> v2:172.21.15.63:3300/0 conn(0x7fb80815b7d0 msgr2=0x7fb80815bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb80cc00640 1 --2- 172.21.15.42:0/1968532398 >> v2:172.21.15.63:3300/0 conn(0x7fb80815b7d0 0x7fb80815bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb80cc00640 1 -- 172.21.15.42:0/1968532398 >> v1:172.21.15.42:6789/0 conn(0x7fb8080b4e00 legacy=0x7fb8080cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:54.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb80cc00640 1 -- 172.21.15.42:0/1968532398 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8080cd4d0 con 0x7fb808155810 2024-05-28T14:01:54.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.828+0000 7fb807e00640 1 --2- 172.21.15.42:0/1968532398 >> v2:172.21.15.63:3300/0 conn(0x7fb80815b7d0 0x7fb80815bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:01:54.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.832+0000 7fb80cc00640 1 --2- 172.21.15.42:0/1968532398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb808155810 0x7fb8080cdcd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0097c0 tx=0x7fb7fc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b7e05420f200195 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:54.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.832+0000 7fb807400640 1 -- 172.21.15.42:0/1968532398 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0aa620 con 0x7fb808155810 2024-05-28T14:01:54.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.832+0000 7fb807400640 1 -- 172.21.15.42:0/1968532398 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7fc0af070 con 0x7fb808155810 2024-05-28T14:01:54.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.832+0000 7fb807400640 1 -- 172.21.15.42:0/1968532398 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0aa9e0 con 0x7fb808155810 2024-05-28T14:01:54.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.832+0000 7fb80e000640 1 -- 172.21.15.42:0/1968532398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb808155810 msgr2=0x7fb8080cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:54.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.832+0000 7fb80e000640 1 --2- 172.21.15.42:0/1968532398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb808155810 0x7fb8080cdcd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0097c0 tx=0x7fb7fc01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.832+0000 7fb80e000640 1 -- 172.21.15.42:0/1968532398 shutdown_connections 2024-05-28T14:01:54.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.832+0000 7fb80e000640 1 --2- 172.21.15.42:0/1968532398 >> v2:172.21.15.63:3300/0 conn(0x7fb80815b7d0 0x7fb80815bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.832+0000 7fb80e000640 1 --2- 172.21.15.42:0/1968532398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb808155810 0x7fb8080cdcd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.832+0000 7fb80e000640 1 -- 172.21.15.42:0/1968532398 >> 172.21.15.42:0/1968532398 conn(0x7fb808073310 msgr2=0x7fb80806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:54.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.832+0000 7fb80e000640 1 -- 172.21.15.42:0/1968532398 shutdown_connections 2024-05-28T14:01:54.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.832+0000 7fb80e000640 1 -- 172.21.15.42:0/1968532398 wait complete. 2024-05-28T14:01:54.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80e000640 1 Processor -- start 2024-05-28T14:01:54.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80e000640 1 -- start start 2024-05-28T14:01:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8080b4e00 0x7fb80815ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80e000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb808155810 0x7fb8081488c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb80815b7d0 0x7fb808147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80e000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb8080ce390 con 0x7fb8080b4e00 2024-05-28T14:01:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80e000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8080ce210 con 0x7fb80815b7d0 2024-05-28T14:01:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80e000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb8080b04f0 con 0x7fb808155810 2024-05-28T14:01:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8080b4e00 0x7fb80815ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8080b4e00 0x7fb80815ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55210/0 (socket says 172.21.15.42:55210) 2024-05-28T14:01:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80cc00640 1 -- 172.21.15.42:0/2386155883 learned_addr learned my addr 172.21.15.42:0/2386155883 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80d600640 1 --2- 172.21.15.42:0/2386155883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb80815b7d0 0x7fb808147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb807e00640 1 --2- 172.21.15.42:0/2386155883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb808155810 0x7fb8081488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80cc00640 1 -- 172.21.15.42:0/2386155883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb808155810 msgr2=0x7fb8081488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:54.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80cc00640 1 --2- 172.21.15.42:0/2386155883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb808155810 0x7fb8081488c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80cc00640 1 -- 172.21.15.42:0/2386155883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb80815b7d0 msgr2=0x7fb808147340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:54.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80cc00640 1 --2- 172.21.15.42:0/2386155883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb80815b7d0 0x7fb808147340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80cc00640 1 -- 172.21.15.42:0/2386155883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8081492a0 con 0x7fb8080b4e00 2024-05-28T14:01:54.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80cc00640 1 --2- 172.21.15.42:0/2386155883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8080b4e00 0x7fb80815ded0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0ac040 tx=0x7fb7fc01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb804c00640 1 -- 172.21.15.42:0/2386155883 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0aabc0 con 0x7fb8080b4e00 2024-05-28T14:01:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb804c00640 1 -- 172.21.15.42:0/2386155883 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7fc0aad60 con 0x7fb8080b4e00 2024-05-28T14:01:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80e000640 1 -- 172.21.15.42:0/2386155883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb808149580 con 0x7fb8080b4e00 2024-05-28T14:01:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb804c00640 1 -- 172.21.15.42:0/2386155883 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0be650 con 0x7fb8080b4e00 2024-05-28T14:01:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80e000640 1 -- 172.21.15.42:0/2386155883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb808149b40 con 0x7fb8080b4e00 2024-05-28T14:01:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.836+0000 7fb80e000640 1 -- 172.21.15.42:0/2386155883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb80804fcf0 con 0x7fb8080b4e00 2024-05-28T14:01:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.840+0000 7fb804c00640 1 -- 172.21.15.42:0/2386155883 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7fb7fc0bc020 con 0x7fb8080b4e00 2024-05-28T14:01:54.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.840+0000 7fb804c00640 1 --2- 172.21.15.42:0/2386155883 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb7e8068650 0x7fb7e806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:54.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.840+0000 7fb804c00640 1 -- 172.21.15.42:0/2386155883 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7fb7fc121270 con 0x7fb8080b4e00 2024-05-28T14:01:54.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.840+0000 7fb807e00640 1 --2- 172.21.15.42:0/2386155883 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb7e8068650 0x7fb7e806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:54.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.840+0000 7fb807e00640 1 --2- 172.21.15.42:0/2386155883 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb7e8068650 0x7fb7e806ab10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009580 tx=0x7fb7f4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:54.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:54.844+0000 7fb804c00640 1 -- 172.21.15.42:0/2386155883 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb7fc016090 con 0x7fb8080b4e00 2024-05-28T14:01:54.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.008+0000 7fb80e000640 1 -- 172.21.15.42:0/2386155883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb80814d5b0 con 0x7fb8080b4e00 2024-05-28T14:01:54.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.008+0000 7fb804c00640 1 -- 172.21.15.42:0/2386155883 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16596 (secure 0 0 0) 0x7fb7fc0f13b0 con 0x7fb8080b4e00 2024-05-28T14:01:54.996 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:54.996 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":46,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:01:52.253262+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:01:49.041938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-28T14:01:49.591543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:01:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.012+0000 7fb80e000640 1 -- 172.21.15.42:0/2386155883 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb7e8068650 msgr2=0x7fb7e806ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.012+0000 7fb80e000640 1 --2- 172.21.15.42:0/2386155883 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb7e8068650 0x7fb7e806ab10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009580 tx=0x7fb7f4008040 comp rx=0 tx=0).stop 2024-05-28T14:01:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.012+0000 7fb80e000640 1 -- 172.21.15.42:0/2386155883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8080b4e00 msgr2=0x7fb80815ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.012+0000 7fb80e000640 1 --2- 172.21.15.42:0/2386155883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8080b4e00 0x7fb80815ded0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0ac040 tx=0x7fb7fc01ace0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.012+0000 7fb80e000640 1 -- 172.21.15.42:0/2386155883 shutdown_connections 2024-05-28T14:01:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.012+0000 7fb80e000640 1 --2- 172.21.15.42:0/2386155883 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb7e8068650 0x7fb7e806ab10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.012+0000 7fb80e000640 1 --2- 172.21.15.42:0/2386155883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb80815b7d0 0x7fb808147340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.012+0000 7fb80e000640 1 --2- 172.21.15.42:0/2386155883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb808155810 0x7fb8081488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.012+0000 7fb80e000640 1 --2- 172.21.15.42:0/2386155883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8080b4e00 0x7fb80815ded0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:54.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.012+0000 7fb80e000640 1 -- 172.21.15.42:0/2386155883 >> 172.21.15.42:0/2386155883 conn(0x7fb808073310 msgr2=0x7fb8080b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:54.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.012+0000 7fb80e000640 1 -- 172.21.15.42:0/2386155883 shutdown_connections 2024-05-28T14:01:54.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.012+0000 7fb80e000640 1 -- 172.21.15.42:0/2386155883 wait complete. 2024-05-28T14:01:55.017 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-28T14:01:55.018 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:01:55.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.172+0000 7f3d41a00640 1 Processor -- start 2024-05-28T14:01:55.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.172+0000 7f3d41a00640 1 -- start start 2024-05-28T14:01:55.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.172+0000 7f3d41a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3d3c06f660 0x7f3d3c073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.172+0000 7f3d41a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3c15ba40 0x7f3d3c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.172+0000 7f3d41a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3d3c072dd0 0x7f3d3c0d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.172+0000 7f3d41a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f3d3c15f030 con 0x7f3d3c072dd0 2024-05-28T14:01:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.172+0000 7f3d41a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3d3c15f280 con 0x7f3d3c06f660 2024-05-28T14:01:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.172+0000 7f3d41a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3c15f560 con 0x7f3d3c15ba40 2024-05-28T14:01:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.172+0000 7f3d3b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3d3c072dd0 0x7f3d3c0d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.172+0000 7f3d3b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3d3c072dd0 0x7f3d3c0d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55226/0 (socket says 172.21.15.42:55226) 2024-05-28T14:01:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.172+0000 7f3d3b400640 1 -- 172.21.15.42:0/2130738867 learned_addr learned my addr 172.21.15.42:0/2130738867 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d32000640 1 --2- 172.21.15.42:0/2130738867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3c15ba40 0x7f3d3c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d3aa00640 1 --2- 172.21.15.42:0/2130738867 >> v2:172.21.15.63:3300/0 conn(0x7f3d3c06f660 0x7f3d3c073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d3b400640 1 -- 172.21.15.42:0/2130738867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3c15ba40 msgr2=0x7f3d3c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d3b400640 1 --2- 172.21.15.42:0/2130738867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3c15ba40 0x7f3d3c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d3b400640 1 -- 172.21.15.42:0/2130738867 >> v2:172.21.15.63:3300/0 conn(0x7f3d3c06f660 msgr2=0x7f3d3c073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d3b400640 1 --2- 172.21.15.42:0/2130738867 >> v2:172.21.15.63:3300/0 conn(0x7f3d3c06f660 0x7f3d3c073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d3b400640 1 -- 172.21.15.42:0/2130738867 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c15f840 con 0x7f3d3c072dd0 2024-05-28T14:01:55.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d3b400640 1 --2- 172.21.15.42:0/2130738867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3c072dd0 0x7f3d3c0d0c30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0d13a0 tx=0x7f3d3401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375e17fa2778da01 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d3a000640 1 -- 172.21.15.42:0/2130738867 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d340a5700 con 0x7f3d3c072dd0 2024-05-28T14:01:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d3a000640 1 -- 172.21.15.42:0/2130738867 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d340a58a0 con 0x7f3d3c072dd0 2024-05-28T14:01:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d3a000640 1 -- 172.21.15.42:0/2130738867 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d340a5c70 con 0x7f3d3c072dd0 2024-05-28T14:01:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d41a00640 1 -- 172.21.15.42:0/2130738867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3c072dd0 msgr2=0x7f3d3c0d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d41a00640 1 --2- 172.21.15.42:0/2130738867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3c072dd0 0x7f3d3c0d0c30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0d13a0 tx=0x7f3d3401c8a0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d41a00640 1 -- 172.21.15.42:0/2130738867 shutdown_connections 2024-05-28T14:01:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.176+0000 7f3d41a00640 1 --2- 172.21.15.42:0/2130738867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3c072dd0 0x7f3d3c0d0c30 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d41a00640 1 --2- 172.21.15.42:0/2130738867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3c15ba40 0x7f3d3c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d41a00640 1 --2- 172.21.15.42:0/2130738867 >> v2:172.21.15.63:3300/0 conn(0x7f3d3c06f660 0x7f3d3c073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d41a00640 1 -- 172.21.15.42:0/2130738867 >> 172.21.15.42:0/2130738867 conn(0x7f3d3c06bdf0 msgr2=0x7f3d3c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d41a00640 1 -- 172.21.15.42:0/2130738867 shutdown_connections 2024-05-28T14:01:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d41a00640 1 -- 172.21.15.42:0/2130738867 wait complete. 2024-05-28T14:01:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d41a00640 1 Processor -- start 2024-05-28T14:01:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d41a00640 1 -- start start 2024-05-28T14:01:55.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d41a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3c06f660 0x7f3d3c099840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d41a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3c072dd0 0x7f3d3c154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d41a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d3c1551c0 0x7f3d3c09a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d41a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c0b4d80 con 0x7f3d3c072dd0 2024-05-28T14:01:55.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d41a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c0b4c00 con 0x7f3d3c1551c0 2024-05-28T14:01:55.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d41a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3c0b4f00 con 0x7f3d3c06f660 2024-05-28T14:01:55.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d32000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3c072dd0 0x7f3d3c154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d32000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3c072dd0 0x7f3d3c154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55236/0 (socket says 172.21.15.42:55236) 2024-05-28T14:01:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d32000640 1 -- 172.21.15.42:0/3439848781 learned_addr learned my addr 172.21.15.42:0/3439848781 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d3aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3c06f660 0x7f3d3c099840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d3b400640 1 --2- 172.21.15.42:0/3439848781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d3c1551c0 0x7f3d3c09a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d32000640 1 -- 172.21.15.42:0/3439848781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3c06f660 msgr2=0x7f3d3c099840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d32000640 1 --2- 172.21.15.42:0/3439848781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3c06f660 0x7f3d3c099840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d32000640 1 -- 172.21.15.42:0/3439848781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d3c1551c0 msgr2=0x7f3d3c09a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d32000640 1 --2- 172.21.15.42:0/3439848781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d3c1551c0 0x7f3d3c09a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d32000640 1 -- 172.21.15.42:0/3439848781 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c0cf5b0 con 0x7f3d3c072dd0 2024-05-28T14:01:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d32000640 1 --2- 172.21.15.42:0/3439848781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3c072dd0 0x7f3d3c154c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400d620 tx=0x7f3d2400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d33400640 1 -- 172.21.15.42:0/3439848781 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2400b620 con 0x7f3d3c072dd0 2024-05-28T14:01:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d33400640 1 -- 172.21.15.42:0/3439848781 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d24019070 con 0x7f3d3c072dd0 2024-05-28T14:01:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d33400640 1 -- 172.21.15.42:0/3439848781 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2400b9e0 con 0x7f3d3c072dd0 2024-05-28T14:01:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.180+0000 7f3d41a00640 1 -- 172.21.15.42:0/3439848781 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3c0b2030 con 0x7f3d3c072dd0 2024-05-28T14:01:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.184+0000 7f3d41a00640 1 -- 172.21.15.42:0/3439848781 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3c13af80 con 0x7f3d3c072dd0 2024-05-28T14:01:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.184+0000 7f3d41a00640 1 -- 172.21.15.42:0/3439848781 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d3c068a90 con 0x7f3d3c072dd0 2024-05-28T14:01:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.184+0000 7f3d33400640 1 -- 172.21.15.42:0/3439848781 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f3d2400bb80 con 0x7f3d3c072dd0 2024-05-28T14:01:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.184+0000 7f3d33400640 1 --2- 172.21.15.42:0/3439848781 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d18068680 0x7f3d1806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.184+0000 7f3d33400640 1 -- 172.21.15.42:0/3439848781 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7f3d24087070 con 0x7f3d3c072dd0 2024-05-28T14:01:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.184+0000 7f3d3aa00640 1 --2- 172.21.15.42:0/3439848781 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d18068680 0x7f3d1806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.188+0000 7f3d3aa00640 1 --2- 172.21.15.42:0/3439848781 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d18068680 0x7f3d1806ab40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0045f0 tx=0x7f3d2c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.188+0000 7f3d33400640 1 -- 172.21.15.42:0/3439848781 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3d3c0b2030 con 0x7f3d3c072dd0 2024-05-28T14:01:55.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.352+0000 7f3d41a00640 1 -- 172.21.15.42:0/3439848781 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d3c072bb0 con 0x7f3d18068680 2024-05-28T14:01:55.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.360+0000 7f3d33400640 1 -- 172.21.15.42:0/3439848781 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251958 (secure 0 0 0) 0x7f3d3c072bb0 con 0x7f3d18068680 2024-05-28T14:01:55.348 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:55.350 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:01:55.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.364+0000 7f3d41a00640 1 -- 172.21.15.42:0/3439848781 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d18068680 msgr2=0x7f3d1806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.364+0000 7f3d41a00640 1 --2- 172.21.15.42:0/3439848781 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d18068680 0x7f3d1806ab40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0045f0 tx=0x7f3d2c008040 comp rx=0 tx=0).stop 2024-05-28T14:01:55.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.364+0000 7f3d41a00640 1 -- 172.21.15.42:0/3439848781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3c072dd0 msgr2=0x7f3d3c154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.364+0000 7f3d41a00640 1 --2- 172.21.15.42:0/3439848781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3c072dd0 0x7f3d3c154c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400d620 tx=0x7f3d2400dae0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.364+0000 7f3d41a00640 1 -- 172.21.15.42:0/3439848781 shutdown_connections 2024-05-28T14:01:55.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.364+0000 7f3d41a00640 1 --2- 172.21.15.42:0/3439848781 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d18068680 0x7f3d1806ab40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.364+0000 7f3d41a00640 1 --2- 172.21.15.42:0/3439848781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d3c1551c0 0x7f3d3c09a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.364+0000 7f3d41a00640 1 --2- 172.21.15.42:0/3439848781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d3c072dd0 0x7f3d3c154c80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.364+0000 7f3d41a00640 1 --2- 172.21.15.42:0/3439848781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d3c06f660 0x7f3d3c099840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.364+0000 7f3d41a00640 1 -- 172.21.15.42:0/3439848781 >> 172.21.15.42:0/3439848781 conn(0x7f3d3c06bdf0 msgr2=0x7f3d3c06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.364+0000 7f3d41a00640 1 -- 172.21.15.42:0/3439848781 shutdown_connections 2024-05-28T14:01:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.364+0000 7f3d41a00640 1 -- 172.21.15.42:0/3439848781 wait complete. 2024-05-28T14:01:55.369 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-05-28T14:01:54.899922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":292320,"kb_used_data":5412,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754682400,"statfs":{"total":773094113280,"available":772794777600,"internally_reserved":0,"allocated":5541888,"data_stored":2935876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":293654066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.678690"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279909+0000","last_change":"2024-05-28T14:01:51.279990+0000","last_active":"2024-05-28T14:01:54.279909+0000","last_peered":"2024-05-28T14:01:54.279909+0000","last_clean":"2024-05-28T14:01:54.279909+0000","last_became_active":"2024-05-28T14:01:51.279688+0000","last_became_peered":"2024-05-28T14:01:51.279688+0000","last_unstale":"2024-05-28T14:01:54.279909+0000","last_undegraded":"2024-05-28T14:01:54.279909+0000","last_fullsized":"2024-05-28T14:01:54.279909+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:30:06.684281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262852+0000","last_change":"2024-05-28T14:01:51.287112+0000","last_active":"2024-05-28T14:01:52.262852+0000","last_peered":"2024-05-28T14:01:52.262852+0000","last_clean":"2024-05-28T14:01:52.262852+0000","last_became_active":"2024-05-28T14:01:51.285855+0000","last_became_peered":"2024-05-28T14:01:51.285855+0000","last_unstale":"2024-05-28T14:01:52.262852+0000","last_undegraded":"2024-05-28T14:01:52.262852+0000","last_fullsized":"2024-05-28T14:01:52.262852+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:59:32.660570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263061+0000","last_change":"2024-05-28T14:01:51.262992+0000","last_active":"2024-05-28T14:01:52.263061+0000","last_peered":"2024-05-28T14:01:52.263061+0000","last_clean":"2024-05-28T14:01:52.263061+0000","last_became_active":"2024-05-28T14:01:51.262698+0000","last_became_peered":"2024-05-28T14:01:51.262698+0000","last_unstale":"2024-05-28T14:01:52.263061+0000","last_undegraded":"2024-05-28T14:01:52.263061+0000","last_fullsized":"2024-05-28T14:01:52.263061+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:54:18.537342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279077+0000","last_change":"2024-05-28T14:01:51.284598+0000","last_active":"2024-05-28T14:01:54.279077+0000","last_peered":"2024-05-28T14:01:54.279077+0000","last_clean":"2024-05-28T14:01:54.279077+0000","last_became_active":"2024-05-28T14:01:51.281894+0000","last_became_peered":"2024-05-28T14:01:51.281894+0000","last_unstale":"2024-05-28T14:01:54.279077+0000","last_undegraded":"2024-05-28T14:01:54.279077+0000","last_fullsized":"2024-05-28T14:01:54.279077+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:29:46.724802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263044+0000","last_change":"2024-05-28T14:01:51.287119+0000","last_active":"2024-05-28T14:01:52.263044+0000","last_peered":"2024-05-28T14:01:52.263044+0000","last_clean":"2024-05-28T14:01:52.263044+0000","last_became_active":"2024-05-28T14:01:51.286771+0000","last_became_peered":"2024-05-28T14:01:51.286771+0000","last_unstale":"2024-05-28T14:01:52.263044+0000","last_undegraded":"2024-05-28T14:01:52.263044+0000","last_fullsized":"2024-05-28T14:01:52.263044+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:42:52.419012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.253925+0000","last_change":"2024-05-28T14:01:51.262667+0000","last_active":"2024-05-28T14:01:53.253925+0000","last_peered":"2024-05-28T14:01:53.253925+0000","last_clean":"2024-05-28T14:01:53.253925+0000","last_became_active":"2024-05-28T14:01:51.262353+0000","last_became_peered":"2024-05-28T14:01:51.262353+0000","last_unstale":"2024-05-28T14:01:53.253925+0000","last_undegraded":"2024-05-28T14:01:53.253925+0000","last_fullsized":"2024-05-28T14:01:53.253925+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:55.017591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279358+0000","last_change":"2024-05-28T14:01:51.273056+0000","last_active":"2024-05-28T14:01:54.279358+0000","last_peered":"2024-05-28T14:01:54.279358+0000","last_clean":"2024-05-28T14:01:54.279358+0000","last_became_active":"2024-05-28T14:01:51.272752+0000","last_became_peered":"2024-05-28T14:01:51.272752+0000","last_unstale":"2024-05-28T14:01:54.279358+0000","last_undegraded":"2024-05-28T14:01:54.279358+0000","last_fullsized":"2024-05-28T14:01:54.279358+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:47:43.795047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.277851+0000","last_change":"2024-05-28T14:01:51.280585+0000","last_active":"2024-05-28T14:01:54.277851+0000","last_peered":"2024-05-28T14:01:54.277851+0000","last_clean":"2024-05-28T14:01:54.277851+0000","last_became_active":"2024-05-28T14:01:51.280319+0000","last_became_peered":"2024-05-28T14:01:51.280319+0000","last_unstale":"2024-05-28T14:01:54.277851+0000","last_undegraded":"2024-05-28T14:01:54.277851+0000","last_fullsized":"2024-05-28T14:01:54.277851+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:34:30.307831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279402+0000","last_change":"2024-05-28T14:01:51.285809+0000","last_active":"2024-05-28T14:01:54.279402+0000","last_peered":"2024-05-28T14:01:54.279402+0000","last_clean":"2024-05-28T14:01:54.279402+0000","last_became_active":"2024-05-28T14:01:51.284118+0000","last_became_peered":"2024-05-28T14:01:51.284118+0000","last_unstale":"2024-05-28T14:01:54.279402+0000","last_undegraded":"2024-05-28T14:01:54.279402+0000","last_fullsized":"2024-05-28T14:01:54.279402+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:10:42.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278712+0000","last_change":"2024-05-28T14:01:51.280306+0000","last_active":"2024-05-28T14:01:54.278712+0000","last_peered":"2024-05-28T14:01:54.278712+0000","last_clean":"2024-05-28T14:01:54.278712+0000","last_became_active":"2024-05-28T14:01:51.279990+0000","last_became_peered":"2024-05-28T14:01:51.279990+0000","last_unstale":"2024-05-28T14:01:54.278712+0000","last_undegraded":"2024-05-28T14:01:54.278712+0000","last_fullsized":"2024-05-28T14:01:54.278712+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:55:31.334010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278317+0000","last_change":"2024-05-28T14:01:51.285883+0000","last_active":"2024-05-28T14:01:54.278317+0000","last_peered":"2024-05-28T14:01:54.278317+0000","last_clean":"2024-05-28T14:01:54.278317+0000","last_became_active":"2024-05-28T14:01:51.284441+0000","last_became_peered":"2024-05-28T14:01:51.284441+0000","last_unstale":"2024-05-28T14:01:54.278317+0000","last_undegraded":"2024-05-28T14:01:54.278317+0000","last_fullsized":"2024-05-28T14:01:54.278317+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:26.331819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263055+0000","last_change":"2024-05-28T14:01:51.271958+0000","last_active":"2024-05-28T14:01:52.263055+0000","last_peered":"2024-05-28T14:01:52.263055+0000","last_clean":"2024-05-28T14:01:52.263055+0000","last_became_active":"2024-05-28T14:01:51.271664+0000","last_became_peered":"2024-05-28T14:01:51.271664+0000","last_unstale":"2024-05-28T14:01:52.263055+0000","last_undegraded":"2024-05-28T14:01:52.263055+0000","last_fullsized":"2024-05-28T14:01:52.263055+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:02:45.432903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278790+0000","last_change":"2024-05-28T14:01:51.265344+0000","last_active":"2024-05-28T14:01:54.278790+0000","last_peered":"2024-05-28T14:01:54.278790+0000","last_clean":"2024-05-28T14:01:54.278790+0000","last_became_active":"2024-05-28T14:01:51.265065+0000","last_became_peered":"2024-05-28T14:01:51.265065+0000","last_unstale":"2024-05-28T14:01:54.278790+0000","last_undegraded":"2024-05-28T14:01:54.278790+0000","last_fullsized":"2024-05-28T14:01:54.278790+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:11:37.087786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278672+0000","last_change":"2024-05-28T14:01:51.285217+0000","last_active":"2024-05-28T14:01:54.278672+0000","last_peered":"2024-05-28T14:01:54.278672+0000","last_clean":"2024-05-28T14:01:54.278672+0000","last_became_active":"2024-05-28T14:01:51.284885+0000","last_became_peered":"2024-05-28T14:01:51.284885+0000","last_unstale":"2024-05-28T14:01:54.278672+0000","last_undegraded":"2024-05-28T14:01:54.278672+0000","last_fullsized":"2024-05-28T14:01:54.278672+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:45:04.831620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.253736+0000","last_change":"2024-05-28T14:01:51.258743+0000","last_active":"2024-05-28T14:01:53.253736+0000","last_peered":"2024-05-28T14:01:53.253736+0000","last_clean":"2024-05-28T14:01:53.253736+0000","last_became_active":"2024-05-28T14:01:51.258473+0000","last_became_peered":"2024-05-28T14:01:51.258473+0000","last_unstale":"2024-05-28T14:01:53.253736+0000","last_undegraded":"2024-05-28T14:01:53.253736+0000","last_fullsized":"2024-05-28T14:01:53.253736+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:44:30.868394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014127+0000","last_change":"2024-05-28T14:01:51.286917+0000","last_active":"2024-05-28T14:01:53.014127+0000","last_peered":"2024-05-28T14:01:53.014127+0000","last_clean":"2024-05-28T14:01:53.014127+0000","last_became_active":"2024-05-28T14:01:51.285949+0000","last_became_peered":"2024-05-28T14:01:51.285949+0000","last_unstale":"2024-05-28T14:01:53.014127+0000","last_undegraded":"2024-05-28T14:01:53.014127+0000","last_fullsized":"2024-05-28T14:01:53.014127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:24:16.440714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263546+0000","last_change":"2024-05-28T14:01:51.274811+0000","last_active":"2024-05-28T14:01:52.263546+0000","last_peered":"2024-05-28T14:01:52.263546+0000","last_clean":"2024-05-28T14:01:52.263546+0000","last_became_active":"2024-05-28T14:01:51.274520+0000","last_became_peered":"2024-05-28T14:01:51.274520+0000","last_unstale":"2024-05-28T14:01:52.263546+0000","last_undegraded":"2024-05-28T14:01:52.263546+0000","last_fullsized":"2024-05-28T14:01:52.263546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:58:30.793739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263140+0000","last_change":"2024-05-28T14:01:51.273089+0000","last_active":"2024-05-28T14:01:52.263140+0000","last_peered":"2024-05-28T14:01:52.263140+0000","last_clean":"2024-05-28T14:01:52.263140+0000","last_became_active":"2024-05-28T14:01:51.272801+0000","last_became_peered":"2024-05-28T14:01:51.272801+0000","last_unstale":"2024-05-28T14:01:52.263140+0000","last_undegraded":"2024-05-28T14:01:52.263140+0000","last_fullsized":"2024-05-28T14:01:52.263140+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:26.331819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014855+0000","last_change":"2024-05-28T14:01:51.285668+0000","last_active":"2024-05-28T14:01:53.014855+0000","last_peered":"2024-05-28T14:01:53.014855+0000","last_clean":"2024-05-28T14:01:53.014855+0000","last_became_active":"2024-05-28T14:01:51.285356+0000","last_became_peered":"2024-05-28T14:01:51.285356+0000","last_unstale":"2024-05-28T14:01:53.014855+0000","last_undegraded":"2024-05-28T14:01:53.014855+0000","last_fullsized":"2024-05-28T14:01:53.014855+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:53:22.132470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278371+0000","last_change":"2024-05-28T14:01:51.268861+0000","last_active":"2024-05-28T14:01:54.278371+0000","last_peered":"2024-05-28T14:01:54.278371+0000","last_clean":"2024-05-28T14:01:54.278371+0000","last_became_active":"2024-05-28T14:01:51.268577+0000","last_became_peered":"2024-05-28T14:01:51.268577+0000","last_unstale":"2024-05-28T14:01:54.278371+0000","last_undegraded":"2024-05-28T14:01:54.278371+0000","last_fullsized":"2024-05-28T14:01:54.278371+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:23.608502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.253921+0000","last_change":"2024-05-28T14:01:51.278198+0000","last_active":"2024-05-28T14:01:53.253921+0000","last_peered":"2024-05-28T14:01:53.253921+0000","last_clean":"2024-05-28T14:01:53.253921+0000","last_became_active":"2024-05-28T14:01:51.277883+0000","last_became_peered":"2024-05-28T14:01:51.277883+0000","last_unstale":"2024-05-28T14:01:53.253921+0000","last_undegraded":"2024-05-28T14:01:53.253921+0000","last_fullsized":"2024-05-28T14:01:53.253921+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:50:22.503284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278822+0000","last_change":"2024-05-28T14:01:51.281113+0000","last_active":"2024-05-28T14:01:54.278822+0000","last_peered":"2024-05-28T14:01:54.278822+0000","last_clean":"2024-05-28T14:01:54.278822+0000","last_became_active":"2024-05-28T14:01:51.280833+0000","last_became_peered":"2024-05-28T14:01:51.280833+0000","last_unstale":"2024-05-28T14:01:54.278822+0000","last_undegraded":"2024-05-28T14:01:54.278822+0000","last_fullsized":"2024-05-28T14:01:54.278822+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:58:30.793739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014337+0000","last_change":"2024-05-28T14:01:51.280307+0000","last_active":"2024-05-28T14:01:53.014337+0000","last_peered":"2024-05-28T14:01:53.014337+0000","last_clean":"2024-05-28T14:01:53.014337+0000","last_became_active":"2024-05-28T14:01:51.279591+0000","last_became_peered":"2024-05-28T14:01:51.279591+0000","last_unstale":"2024-05-28T14:01:53.014337+0000","last_undegraded":"2024-05-28T14:01:53.014337+0000","last_fullsized":"2024-05-28T14:01:53.014337+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:33:55.389532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014172+0000","last_change":"2024-05-28T14:01:51.281163+0000","last_active":"2024-05-28T14:01:53.014172+0000","last_peered":"2024-05-28T14:01:53.014172+0000","last_clean":"2024-05-28T14:01:53.014172+0000","last_became_active":"2024-05-28T14:01:51.280877+0000","last_became_peered":"2024-05-28T14:01:51.280877+0000","last_unstale":"2024-05-28T14:01:53.014172+0000","last_undegraded":"2024-05-28T14:01:53.014172+0000","last_fullsized":"2024-05-28T14:01:53.014172+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:11:37.087786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014770+0000","last_change":"2024-05-28T14:01:51.285012+0000","last_active":"2024-05-28T14:01:53.014770+0000","last_peered":"2024-05-28T14:01:53.014770+0000","last_clean":"2024-05-28T14:01:53.014770+0000","last_became_active":"2024-05-28T14:01:51.284685+0000","last_became_peered":"2024-05-28T14:01:51.284685+0000","last_unstale":"2024-05-28T14:01:53.014770+0000","last_undegraded":"2024-05-28T14:01:53.014770+0000","last_fullsized":"2024-05-28T14:01:53.014770+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:07.570864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278558+0000","last_change":"2024-05-28T14:01:51.254804+0000","last_active":"2024-05-28T14:01:54.278558+0000","last_peered":"2024-05-28T14:01:54.278558+0000","last_clean":"2024-05-28T14:01:54.278558+0000","last_became_active":"2024-05-28T14:01:51.254509+0000","last_became_peered":"2024-05-28T14:01:51.254509+0000","last_unstale":"2024-05-28T14:01:54.278558+0000","last_undegraded":"2024-05-28T14:01:54.278558+0000","last_fullsized":"2024-05-28T14:01:54.278558+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:28:09.087529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263248+0000","last_change":"2024-05-28T14:01:51.272978+0000","last_active":"2024-05-28T14:01:52.263248+0000","last_peered":"2024-05-28T14:01:52.263248+0000","last_clean":"2024-05-28T14:01:52.263248+0000","last_became_active":"2024-05-28T14:01:51.272040+0000","last_became_peered":"2024-05-28T14:01:51.272040+0000","last_unstale":"2024-05-28T14:01:52.263248+0000","last_undegraded":"2024-05-28T14:01:52.263248+0000","last_fullsized":"2024-05-28T14:01:52.263248+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:45:04.831620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263279+0000","last_change":"2024-05-28T14:01:51.271937+0000","last_active":"2024-05-28T14:01:52.263279+0000","last_peered":"2024-05-28T14:01:52.263279+0000","last_clean":"2024-05-28T14:01:52.263279+0000","last_became_active":"2024-05-28T14:01:51.271633+0000","last_became_peered":"2024-05-28T14:01:51.271633+0000","last_unstale":"2024-05-28T14:01:52.263279+0000","last_undegraded":"2024-05-28T14:01:52.263279+0000","last_fullsized":"2024-05-28T14:01:52.263279+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:47:43.795047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014441+0000","last_change":"2024-05-28T14:01:51.282053+0000","last_active":"2024-05-28T14:01:53.014441+0000","last_peered":"2024-05-28T14:01:53.014441+0000","last_clean":"2024-05-28T14:01:53.014441+0000","last_became_active":"2024-05-28T14:01:51.281778+0000","last_became_peered":"2024-05-28T14:01:51.281778+0000","last_unstale":"2024-05-28T14:01:53.014441+0000","last_undegraded":"2024-05-28T14:01:53.014441+0000","last_fullsized":"2024-05-28T14:01:53.014441+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:19:11.175912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262089+0000","last_change":"2024-05-28T14:01:51.268383+0000","last_active":"2024-05-28T14:01:52.262089+0000","last_peered":"2024-05-28T14:01:52.262089+0000","last_clean":"2024-05-28T14:01:52.262089+0000","last_became_active":"2024-05-28T14:01:51.267929+0000","last_became_peered":"2024-05-28T14:01:51.267929+0000","last_unstale":"2024-05-28T14:01:52.262089+0000","last_undegraded":"2024-05-28T14:01:52.262089+0000","last_fullsized":"2024-05-28T14:01:52.262089+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:07:41.855434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.253896+0000","last_change":"2024-05-28T14:01:51.263422+0000","last_active":"2024-05-28T14:01:53.253896+0000","last_peered":"2024-05-28T14:01:53.253896+0000","last_clean":"2024-05-28T14:01:53.253896+0000","last_became_active":"2024-05-28T14:01:51.263160+0000","last_became_peered":"2024-05-28T14:01:51.263160+0000","last_unstale":"2024-05-28T14:01:53.253896+0000","last_undegraded":"2024-05-28T14:01:53.253896+0000","last_fullsized":"2024-05-28T14:01:53.253896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:10:16.843029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.277858+0000","last_change":"2024-05-28T14:01:51.269810+0000","last_active":"2024-05-28T14:01:54.277858+0000","last_peered":"2024-05-28T14:01:54.277858+0000","last_clean":"2024-05-28T14:01:54.277858+0000","last_became_active":"2024-05-28T14:01:51.269522+0000","last_became_peered":"2024-05-28T14:01:51.269522+0000","last_unstale":"2024-05-28T14:01:54.277858+0000","last_undegraded":"2024-05-28T14:01:54.277858+0000","last_fullsized":"2024-05-28T14:01:54.277858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:50:37.152282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.257874+0000","last_change":"2024-05-28T14:01:51.279615+0000","last_active":"2024-05-28T14:01:52.257874+0000","last_peered":"2024-05-28T14:01:52.257874+0000","last_clean":"2024-05-28T14:01:52.257874+0000","last_became_active":"2024-05-28T14:01:51.279345+0000","last_became_peered":"2024-05-28T14:01:51.279345+0000","last_unstale":"2024-05-28T14:01:52.257874+0000","last_undegraded":"2024-05-28T14:01:52.257874+0000","last_fullsized":"2024-05-28T14:01:52.257874+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:21:00.662269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254297+0000","last_change":"2024-05-28T14:01:50.247611+0000","last_active":"2024-05-28T14:01:53.254297+0000","last_peered":"2024-05-28T14:01:53.254297+0000","last_clean":"2024-05-28T14:01:53.254297+0000","last_became_active":"2024-05-28T14:01:50.246921+0000","last_became_peered":"2024-05-28T14:01:50.246921+0000","last_unstale":"2024-05-28T14:01:53.254297+0000","last_undegraded":"2024-05-28T14:01:53.254297+0000","last_fullsized":"2024-05-28T14:01:53.254297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:34:19.001030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014826+0000","last_change":"2024-05-28T14:01:51.281147+0000","last_active":"2024-05-28T14:01:53.014826+0000","last_peered":"2024-05-28T14:01:53.014826+0000","last_clean":"2024-05-28T14:01:53.014826+0000","last_became_active":"2024-05-28T14:01:51.280829+0000","last_became_peered":"2024-05-28T14:01:51.280829+0000","last_unstale":"2024-05-28T14:01:53.014826+0000","last_undegraded":"2024-05-28T14:01:53.014826+0000","last_fullsized":"2024-05-28T14:01:53.014826+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:39:26.852227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263373+0000","last_change":"2024-05-28T14:01:50.235938+0000","last_active":"2024-05-28T14:01:52.263373+0000","last_peered":"2024-05-28T14:01:52.263373+0000","last_clean":"2024-05-28T14:01:52.263373+0000","last_became_active":"2024-05-28T14:01:50.235580+0000","last_became_peered":"2024-05-28T14:01:50.235580+0000","last_unstale":"2024-05-28T14:01:52.263373+0000","last_undegraded":"2024-05-28T14:01:52.263373+0000","last_fullsized":"2024-05-28T14:01:52.263373+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:05.536748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263233+0000","last_change":"2024-05-28T14:01:51.282705+0000","last_active":"2024-05-28T14:01:52.263233+0000","last_peered":"2024-05-28T14:01:52.263233+0000","last_clean":"2024-05-28T14:01:52.263233+0000","last_became_active":"2024-05-28T14:01:51.282399+0000","last_became_peered":"2024-05-28T14:01:51.282399+0000","last_unstale":"2024-05-28T14:01:52.263233+0000","last_undegraded":"2024-05-28T14:01:52.263233+0000","last_fullsized":"2024-05-28T14:01:52.263233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:16:53.615918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.257884+0000","last_change":"2024-05-28T14:01:50.257519+0000","last_active":"2024-05-28T14:01:52.257884+0000","last_peered":"2024-05-28T14:01:52.257884+0000","last_clean":"2024-05-28T14:01:52.257884+0000","last_became_active":"2024-05-28T14:01:50.257237+0000","last_became_peered":"2024-05-28T14:01:50.257237+0000","last_unstale":"2024-05-28T14:01:52.257884+0000","last_undegraded":"2024-05-28T14:01:52.257884+0000","last_fullsized":"2024-05-28T14:01:52.257884+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:37:38.552292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.257935+0000","last_change":"2024-05-28T14:01:51.276500+0000","last_active":"2024-05-28T14:01:52.257935+0000","last_peered":"2024-05-28T14:01:52.257935+0000","last_clean":"2024-05-28T14:01:52.257935+0000","last_became_active":"2024-05-28T14:01:51.276213+0000","last_became_peered":"2024-05-28T14:01:51.276213+0000","last_unstale":"2024-05-28T14:01:52.257935+0000","last_undegraded":"2024-05-28T14:01:52.257935+0000","last_fullsized":"2024-05-28T14:01:52.257935+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:36:21.006195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279525+0000","last_change":"2024-05-28T14:01:50.252200+0000","last_active":"2024-05-28T14:01:54.279525+0000","last_peered":"2024-05-28T14:01:54.279525+0000","last_clean":"2024-05-28T14:01:54.279525+0000","last_became_active":"2024-05-28T14:01:50.250940+0000","last_became_peered":"2024-05-28T14:01:50.250940+0000","last_unstale":"2024-05-28T14:01:54.279525+0000","last_undegraded":"2024-05-28T14:01:54.279525+0000","last_fullsized":"2024-05-28T14:01:54.279525+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:20:52.388988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262100+0000","last_change":"2024-05-28T14:01:51.281796+0000","last_active":"2024-05-28T14:01:52.262100+0000","last_peered":"2024-05-28T14:01:52.262100+0000","last_clean":"2024-05-28T14:01:52.262100+0000","last_became_active":"2024-05-28T14:01:51.281330+0000","last_became_peered":"2024-05-28T14:01:51.281330+0000","last_unstale":"2024-05-28T14:01:52.262100+0000","last_undegraded":"2024-05-28T14:01:52.262100+0000","last_fullsized":"2024-05-28T14:01:52.262100+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:33:07.768264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254296+0000","last_change":"2024-05-28T14:01:50.242765+0000","last_active":"2024-05-28T14:01:53.254296+0000","last_peered":"2024-05-28T14:01:53.254296+0000","last_clean":"2024-05-28T14:01:53.254296+0000","last_became_active":"2024-05-28T14:01:50.241957+0000","last_became_peered":"2024-05-28T14:01:50.241957+0000","last_unstale":"2024-05-28T14:01:53.254296+0000","last_undegraded":"2024-05-28T14:01:53.254296+0000","last_fullsized":"2024-05-28T14:01:53.254296+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:38:40.676552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279142+0000","last_change":"2024-05-28T14:01:51.254960+0000","last_active":"2024-05-28T14:01:54.279142+0000","last_peered":"2024-05-28T14:01:54.279142+0000","last_clean":"2024-05-28T14:01:54.279142+0000","last_became_active":"2024-05-28T14:01:51.254654+0000","last_became_peered":"2024-05-28T14:01:51.254654+0000","last_unstale":"2024-05-28T14:01:54.279142+0000","last_undegraded":"2024-05-28T14:01:54.279142+0000","last_fullsized":"2024-05-28T14:01:54.279142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:23:35.885615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262184+0000","last_change":"2024-05-28T14:01:50.248702+0000","last_active":"2024-05-28T14:01:52.262184+0000","last_peered":"2024-05-28T14:01:52.262184+0000","last_clean":"2024-05-28T14:01:52.262184+0000","last_became_active":"2024-05-28T14:01:50.247906+0000","last_became_peered":"2024-05-28T14:01:50.247906+0000","last_unstale":"2024-05-28T14:01:52.262184+0000","last_undegraded":"2024-05-28T14:01:52.262184+0000","last_fullsized":"2024-05-28T14:01:52.262184+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:11:36.080061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278027+0000","last_change":"2024-05-28T14:01:51.280366+0000","last_active":"2024-05-28T14:01:54.278027+0000","last_peered":"2024-05-28T14:01:54.278027+0000","last_clean":"2024-05-28T14:01:54.278027+0000","last_became_active":"2024-05-28T14:01:51.280080+0000","last_became_peered":"2024-05-28T14:01:51.280080+0000","last_unstale":"2024-05-28T14:01:54.278027+0000","last_undegraded":"2024-05-28T14:01:54.278027+0000","last_fullsized":"2024-05-28T14:01:54.278027+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:33:07.768264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262431+0000","last_change":"2024-05-28T14:01:50.255043+0000","last_active":"2024-05-28T14:01:52.262431+0000","last_peered":"2024-05-28T14:01:52.262431+0000","last_clean":"2024-05-28T14:01:52.262431+0000","last_became_active":"2024-05-28T14:01:50.253917+0000","last_became_peered":"2024-05-28T14:01:50.253917+0000","last_unstale":"2024-05-28T14:01:52.262431+0000","last_undegraded":"2024-05-28T14:01:52.262431+0000","last_fullsized":"2024-05-28T14:01:52.262431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:10:41.806648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014335+0000","last_change":"2024-05-28T14:01:51.280547+0000","last_active":"2024-05-28T14:01:53.014335+0000","last_peered":"2024-05-28T14:01:53.014335+0000","last_clean":"2024-05-28T14:01:53.014335+0000","last_became_active":"2024-05-28T14:01:51.279941+0000","last_became_peered":"2024-05-28T14:01:51.279941+0000","last_unstale":"2024-05-28T14:01:53.014335+0000","last_undegraded":"2024-05-28T14:01:53.014335+0000","last_fullsized":"2024-05-28T14:01:53.014335+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:22:41.196856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278147+0000","last_change":"2024-05-28T14:01:50.253530+0000","last_active":"2024-05-28T14:01:54.278147+0000","last_peered":"2024-05-28T14:01:54.278147+0000","last_clean":"2024-05-28T14:01:54.278147+0000","last_became_active":"2024-05-28T14:01:50.253218+0000","last_became_peered":"2024-05-28T14:01:50.253218+0000","last_unstale":"2024-05-28T14:01:54.278147+0000","last_undegraded":"2024-05-28T14:01:54.278147+0000","last_fullsized":"2024-05-28T14:01:54.278147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:37:16.273440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263305+0000","last_change":"2024-05-28T14:01:51.281785+0000","last_active":"2024-05-28T14:01:52.263305+0000","last_peered":"2024-05-28T14:01:52.263305+0000","last_clean":"2024-05-28T14:01:52.263305+0000","last_became_active":"2024-05-28T14:01:51.281460+0000","last_became_peered":"2024-05-28T14:01:51.281460+0000","last_unstale":"2024-05-28T14:01:52.263305+0000","last_undegraded":"2024-05-28T14:01:52.263305+0000","last_fullsized":"2024-05-28T14:01:52.263305+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:55:31.334010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263585+0000","last_change":"2024-05-28T14:01:50.244865+0000","last_active":"2024-05-28T14:01:52.263585+0000","last_peered":"2024-05-28T14:01:52.263585+0000","last_clean":"2024-05-28T14:01:52.263585+0000","last_became_active":"2024-05-28T14:01:50.244523+0000","last_became_peered":"2024-05-28T14:01:50.244523+0000","last_unstale":"2024-05-28T14:01:52.263585+0000","last_undegraded":"2024-05-28T14:01:52.263585+0000","last_fullsized":"2024-05-28T14:01:52.263585+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:19:10.168187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.280009+0000","last_change":"2024-05-28T14:01:51.277015+0000","last_active":"2024-05-28T14:01:54.280009+0000","last_peered":"2024-05-28T14:01:54.280009+0000","last_clean":"2024-05-28T14:01:54.280009+0000","last_became_active":"2024-05-28T14:01:51.276689+0000","last_became_peered":"2024-05-28T14:01:51.276689+0000","last_unstale":"2024-05-28T14:01:54.280009+0000","last_undegraded":"2024-05-28T14:01:54.280009+0000","last_fullsized":"2024-05-28T14:01:54.280009+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:37:36.755762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278932+0000","last_change":"2024-05-28T14:01:50.255287+0000","last_active":"2024-05-28T14:01:54.278932+0000","last_peered":"2024-05-28T14:01:54.278932+0000","last_clean":"2024-05-28T14:01:54.278932+0000","last_became_active":"2024-05-28T14:01:50.254641+0000","last_became_peered":"2024-05-28T14:01:50.254641+0000","last_unstale":"2024-05-28T14:01:54.278932+0000","last_undegraded":"2024-05-28T14:01:54.278932+0000","last_fullsized":"2024-05-28T14:01:54.278932+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:50:36.144557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254337+0000","last_change":"2024-05-28T14:01:51.273569+0000","last_active":"2024-05-28T14:01:53.254337+0000","last_peered":"2024-05-28T14:01:53.254337+0000","last_clean":"2024-05-28T14:01:53.254337+0000","last_became_active":"2024-05-28T14:01:51.273292+0000","last_became_peered":"2024-05-28T14:01:51.273292+0000","last_unstale":"2024-05-28T14:01:53.254337+0000","last_undegraded":"2024-05-28T14:01:53.254337+0000","last_fullsized":"2024-05-28T14:01:53.254337+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:19:09.628626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279757+0000","last_change":"2024-05-28T14:01:50.253565+0000","last_active":"2024-05-28T14:01:54.279757+0000","last_peered":"2024-05-28T14:01:54.279757+0000","last_clean":"2024-05-28T14:01:54.279757+0000","last_became_active":"2024-05-28T14:01:50.253125+0000","last_became_peered":"2024-05-28T14:01:50.253125+0000","last_unstale":"2024-05-28T14:01:54.279757+0000","last_undegraded":"2024-05-28T14:01:54.279757+0000","last_fullsized":"2024-05-28T14:01:54.279757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:12:21.809608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279668+0000","last_change":"2024-05-28T14:01:51.273964+0000","last_active":"2024-05-28T14:01:54.279668+0000","last_peered":"2024-05-28T14:01:54.279668+0000","last_clean":"2024-05-28T14:01:54.279668+0000","last_became_active":"2024-05-28T14:01:51.273576+0000","last_became_peered":"2024-05-28T14:01:51.273576+0000","last_unstale":"2024-05-28T14:01:54.279668+0000","last_undegraded":"2024-05-28T14:01:54.279668+0000","last_fullsized":"2024-05-28T14:01:54.279668+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:07:10.966218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278413+0000","last_change":"2024-05-28T14:01:50.237388+0000","last_active":"2024-05-28T14:01:54.278413+0000","last_peered":"2024-05-28T14:01:54.278413+0000","last_clean":"2024-05-28T14:01:54.278413+0000","last_became_active":"2024-05-28T14:01:50.236288+0000","last_became_peered":"2024-05-28T14:01:50.236288+0000","last_unstale":"2024-05-28T14:01:54.278413+0000","last_undegraded":"2024-05-28T14:01:54.278413+0000","last_fullsized":"2024-05-28T14:01:54.278413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:13:46.276215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014331+0000","last_change":"2024-05-28T14:01:51.285350+0000","last_active":"2024-05-28T14:01:53.014331+0000","last_peered":"2024-05-28T14:01:53.014331+0000","last_clean":"2024-05-28T14:01:53.014331+0000","last_became_active":"2024-05-28T14:01:51.284913+0000","last_became_peered":"2024-05-28T14:01:51.284913+0000","last_unstale":"2024-05-28T14:01:53.014331+0000","last_undegraded":"2024-05-28T14:01:53.014331+0000","last_fullsized":"2024-05-28T14:01:53.014331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:06.544473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254315+0000","last_change":"2024-05-28T14:01:50.241618+0000","last_active":"2024-05-28T14:01:53.254315+0000","last_peered":"2024-05-28T14:01:53.254315+0000","last_clean":"2024-05-28T14:01:53.254315+0000","last_became_active":"2024-05-28T14:01:50.240635+0000","last_became_peered":"2024-05-28T14:01:50.240635+0000","last_unstale":"2024-05-28T14:01:53.254315+0000","last_undegraded":"2024-05-28T14:01:53.254315+0000","last_fullsized":"2024-05-28T14:01:53.254315+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:08:35.087398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279417+0000","last_change":"2024-05-28T14:01:51.265946+0000","last_active":"2024-05-28T14:01:54.279417+0000","last_peered":"2024-05-28T14:01:54.279417+0000","last_clean":"2024-05-28T14:01:54.279417+0000","last_became_active":"2024-05-28T14:01:51.265651+0000","last_became_peered":"2024-05-28T14:01:51.265651+0000","last_unstale":"2024-05-28T14:01:54.279417+0000","last_undegraded":"2024-05-28T14:01:54.279417+0000","last_fullsized":"2024-05-28T14:01:54.279417+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:34:20.008756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279504+0000","last_change":"2024-05-28T14:01:50.251360+0000","last_active":"2024-05-28T14:01:54.279504+0000","last_peered":"2024-05-28T14:01:54.279504+0000","last_clean":"2024-05-28T14:01:54.279504+0000","last_became_active":"2024-05-28T14:01:50.251058+0000","last_became_peered":"2024-05-28T14:01:50.251058+0000","last_unstale":"2024-05-28T14:01:54.279504+0000","last_undegraded":"2024-05-28T14:01:54.279504+0000","last_fullsized":"2024-05-28T14:01:54.279504+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:10:41.806648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.253736+0000","last_change":"2024-05-28T14:01:51.260428+0000","last_active":"2024-05-28T14:01:53.253736+0000","last_peered":"2024-05-28T14:01:53.253736+0000","last_clean":"2024-05-28T14:01:53.253736+0000","last_became_active":"2024-05-28T14:01:51.260117+0000","last_became_peered":"2024-05-28T14:01:51.260117+0000","last_unstale":"2024-05-28T14:01:53.253736+0000","last_undegraded":"2024-05-28T14:01:53.253736+0000","last_fullsized":"2024-05-28T14:01:53.253736+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:37:36.755762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254128+0000","last_change":"2024-05-28T14:01:50.240951+0000","last_active":"2024-05-28T14:01:53.254128+0000","last_peered":"2024-05-28T14:01:53.254128+0000","last_clean":"2024-05-28T14:01:53.254128+0000","last_became_active":"2024-05-28T14:01:50.240657+0000","last_became_peered":"2024-05-28T14:01:50.240657+0000","last_unstale":"2024-05-28T14:01:53.254128+0000","last_undegraded":"2024-05-28T14:01:53.254128+0000","last_fullsized":"2024-05-28T14:01:53.254128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:12:21.809608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279439+0000","last_change":"2024-05-28T14:01:51.278672+0000","last_active":"2024-05-28T14:01:54.279439+0000","last_peered":"2024-05-28T14:01:54.279439+0000","last_clean":"2024-05-28T14:01:54.279439+0000","last_became_active":"2024-05-28T14:01:51.277254+0000","last_became_peered":"2024-05-28T14:01:51.277254+0000","last_unstale":"2024-05-28T14:01:54.279439+0000","last_undegraded":"2024-05-28T14:01:54.279439+0000","last_fullsized":"2024-05-28T14:01:54.279439+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:38:34.772356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":22,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278263+0000","last_change":"2024-05-28T14:01:50.249113+0000","last_active":"2024-05-28T14:01:54.278263+0000","last_peered":"2024-05-28T14:01:54.278263+0000","last_clean":"2024-05-28T14:01:54.278263+0000","last_became_active":"2024-05-28T14:01:50.248646+0000","last_became_peered":"2024-05-28T14:01:50.248646+0000","last_unstale":"2024-05-28T14:01:54.278263+0000","last_undegraded":"2024-05-28T14:01:54.278263+0000","last_fullsized":"2024-05-28T14:01:54.278263+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:22.600777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278773+0000","last_change":"2024-05-28T14:01:51.259618+0000","last_active":"2024-05-28T14:01:54.278773+0000","last_peered":"2024-05-28T14:01:54.278773+0000","last_clean":"2024-05-28T14:01:54.278773+0000","last_became_active":"2024-05-28T14:01:51.259327+0000","last_became_peered":"2024-05-28T14:01:51.259327+0000","last_unstale":"2024-05-28T14:01:54.278773+0000","last_undegraded":"2024-05-28T14:01:54.278773+0000","last_fullsized":"2024-05-28T14:01:54.278773+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:24:16.440714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279048+0000","last_change":"2024-05-28T14:01:50.244480+0000","last_active":"2024-05-28T14:01:54.279048+0000","last_peered":"2024-05-28T14:01:54.279048+0000","last_clean":"2024-05-28T14:01:54.279048+0000","last_became_active":"2024-05-28T14:01:50.243160+0000","last_became_peered":"2024-05-28T14:01:50.243160+0000","last_unstale":"2024-05-28T14:01:54.279048+0000","last_undegraded":"2024-05-28T14:01:54.279048+0000","last_fullsized":"2024-05-28T14:01:54.279048+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:50:33.689225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279492+0000","last_change":"2024-05-28T14:01:51.259313+0000","last_active":"2024-05-28T14:01:54.279492+0000","last_peered":"2024-05-28T14:01:54.279492+0000","last_clean":"2024-05-28T14:01:54.279492+0000","last_became_active":"2024-05-28T14:01:51.258231+0000","last_became_peered":"2024-05-28T14:01:51.258231+0000","last_unstale":"2024-05-28T14:01:54.279492+0000","last_undegraded":"2024-05-28T14:01:54.279492+0000","last_fullsized":"2024-05-28T14:01:54.279492+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:12:22.817333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263375+0000","last_change":"2024-05-28T14:01:50.254749+0000","last_active":"2024-05-28T14:01:52.263375+0000","last_peered":"2024-05-28T14:01:52.263375+0000","last_clean":"2024-05-28T14:01:52.263375+0000","last_became_active":"2024-05-28T14:01:50.254428+0000","last_became_peered":"2024-05-28T14:01:50.254428+0000","last_unstale":"2024-05-28T14:01:52.263375+0000","last_undegraded":"2024-05-28T14:01:52.263375+0000","last_fullsized":"2024-05-28T14:01:52.263375+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:33:54.381807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014520+0000","last_change":"2024-05-28T14:01:51.286227+0000","last_active":"2024-05-28T14:01:53.014520+0000","last_peered":"2024-05-28T14:01:53.014520+0000","last_clean":"2024-05-28T14:01:53.014520+0000","last_became_active":"2024-05-28T14:01:51.284627+0000","last_became_peered":"2024-05-28T14:01:51.284627+0000","last_unstale":"2024-05-28T14:01:53.014520+0000","last_undegraded":"2024-05-28T14:01:53.014520+0000","last_fullsized":"2024-05-28T14:01:53.014520+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:06:41.596599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262665+0000","last_change":"2024-05-28T14:01:50.258828+0000","last_active":"2024-05-28T14:01:52.262665+0000","last_peered":"2024-05-28T14:01:52.262665+0000","last_clean":"2024-05-28T14:01:52.262665+0000","last_became_active":"2024-05-28T14:01:50.258581+0000","last_became_peered":"2024-05-28T14:01:50.258581+0000","last_unstale":"2024-05-28T14:01:52.262665+0000","last_undegraded":"2024-05-28T14:01:52.262665+0000","last_fullsized":"2024-05-28T14:01:52.262665+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:23:34.877889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014698+0000","last_change":"2024-05-28T14:01:51.282904+0000","last_active":"2024-05-28T14:01:53.014698+0000","last_peered":"2024-05-28T14:01:53.014698+0000","last_clean":"2024-05-28T14:01:53.014698+0000","last_became_active":"2024-05-28T14:01:51.282479+0000","last_became_peered":"2024-05-28T14:01:51.282479+0000","last_unstale":"2024-05-28T14:01:53.014698+0000","last_undegraded":"2024-05-28T14:01:53.014698+0000","last_fullsized":"2024-05-28T14:01:53.014698+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:37:39.560017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254197+0000","last_change":"2024-05-28T14:01:50.247282+0000","last_active":"2024-05-28T14:01:53.254197+0000","last_peered":"2024-05-28T14:01:53.254197+0000","last_clean":"2024-05-28T14:01:53.254197+0000","last_became_active":"2024-05-28T14:01:50.246963+0000","last_became_peered":"2024-05-28T14:01:50.246963+0000","last_unstale":"2024-05-28T14:01:53.254197+0000","last_undegraded":"2024-05-28T14:01:53.254197+0000","last_fullsized":"2024-05-28T14:01:53.254197+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:20:52.388988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.253798+0000","last_change":"2024-05-28T14:01:51.258939+0000","last_active":"2024-05-28T14:01:53.253798+0000","last_peered":"2024-05-28T14:01:53.253798+0000","last_clean":"2024-05-28T14:01:53.253798+0000","last_became_active":"2024-05-28T14:01:51.258612+0000","last_became_peered":"2024-05-28T14:01:51.258612+0000","last_unstale":"2024-05-28T14:01:53.253798+0000","last_undegraded":"2024-05-28T14:01:53.253798+0000","last_fullsized":"2024-05-28T14:01:53.253798+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:04:37.469912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254447+0000","last_change":"2024-05-28T14:01:50.242434+0000","last_active":"2024-05-28T14:01:53.254447+0000","last_peered":"2024-05-28T14:01:53.254447+0000","last_clean":"2024-05-28T14:01:53.254447+0000","last_became_active":"2024-05-28T14:01:50.241077+0000","last_became_peered":"2024-05-28T14:01:50.241077+0000","last_unstale":"2024-05-28T14:01:53.254447+0000","last_undegraded":"2024-05-28T14:01:53.254447+0000","last_fullsized":"2024-05-28T14:01:53.254447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:30:39.256190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.257691+0000","last_change":"2024-05-28T14:01:51.278971+0000","last_active":"2024-05-28T14:01:52.257691+0000","last_peered":"2024-05-28T14:01:52.257691+0000","last_clean":"2024-05-28T14:01:52.257691+0000","last_became_active":"2024-05-28T14:01:51.278718+0000","last_became_peered":"2024-05-28T14:01:51.278718+0000","last_unstale":"2024-05-28T14:01:52.257691+0000","last_undegraded":"2024-05-28T14:01:52.257691+0000","last_fullsized":"2024-05-28T14:01:52.257691+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:32:16.010129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262423+0000","last_change":"2024-05-28T14:01:50.249518+0000","last_active":"2024-05-28T14:01:52.262423+0000","last_peered":"2024-05-28T14:01:52.262423+0000","last_clean":"2024-05-28T14:01:52.262423+0000","last_became_active":"2024-05-28T14:01:50.246638+0000","last_became_peered":"2024-05-28T14:01:50.246638+0000","last_unstale":"2024-05-28T14:01:52.262423+0000","last_undegraded":"2024-05-28T14:01:52.262423+0000","last_fullsized":"2024-05-28T14:01:52.262423+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:22:14.538409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.277993+0000","last_change":"2024-05-28T14:01:51.267456+0000","last_active":"2024-05-28T14:01:54.277993+0000","last_peered":"2024-05-28T14:01:54.277993+0000","last_clean":"2024-05-28T14:01:54.277993+0000","last_became_active":"2024-05-28T14:01:51.267083+0000","last_became_peered":"2024-05-28T14:01:51.267083+0000","last_unstale":"2024-05-28T14:01:54.277993+0000","last_undegraded":"2024-05-28T14:01:54.277993+0000","last_fullsized":"2024-05-28T14:01:54.277993+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:37:17.281165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.277928+0000","last_change":"2024-05-28T14:01:50.255380+0000","last_active":"2024-05-28T14:01:54.277928+0000","last_peered":"2024-05-28T14:01:54.277928+0000","last_clean":"2024-05-28T14:01:54.277928+0000","last_became_active":"2024-05-28T14:01:50.255076+0000","last_became_peered":"2024-05-28T14:01:50.255076+0000","last_unstale":"2024-05-28T14:01:54.277928+0000","last_undegraded":"2024-05-28T14:01:54.277928+0000","last_fullsized":"2024-05-28T14:01:54.277928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:06:22.496792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.277866+0000","last_change":"2024-05-28T14:01:51.281100+0000","last_active":"2024-05-28T14:01:54.277866+0000","last_peered":"2024-05-28T14:01:54.277866+0000","last_clean":"2024-05-28T14:01:54.277866+0000","last_became_active":"2024-05-28T14:01:51.280804+0000","last_became_peered":"2024-05-28T14:01:51.280804+0000","last_unstale":"2024-05-28T14:01:54.277866+0000","last_undegraded":"2024-05-28T14:01:54.277866+0000","last_fullsized":"2024-05-28T14:01:54.277866+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:07:41.855434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279350+0000","last_change":"2024-05-28T14:01:50.248761+0000","last_active":"2024-05-28T14:01:54.279350+0000","last_peered":"2024-05-28T14:01:54.279350+0000","last_clean":"2024-05-28T14:01:54.279350+0000","last_became_active":"2024-05-28T14:01:50.248430+0000","last_became_peered":"2024-05-28T14:01:50.248430+0000","last_unstale":"2024-05-28T14:01:54.279350+0000","last_undegraded":"2024-05-28T14:01:54.279350+0000","last_fullsized":"2024-05-28T14:01:54.279350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:23:34.877889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014876+0000","last_change":"2024-05-28T14:01:51.289347+0000","last_active":"2024-05-28T14:01:53.014876+0000","last_peered":"2024-05-28T14:01:53.014876+0000","last_clean":"2024-05-28T14:01:53.014876+0000","last_became_active":"2024-05-28T14:01:51.289063+0000","last_became_peered":"2024-05-28T14:01:51.289063+0000","last_unstale":"2024-05-28T14:01:53.014876+0000","last_undegraded":"2024-05-28T14:01:53.014876+0000","last_fullsized":"2024-05-28T14:01:53.014876+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:32:16.010129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254155+0000","last_change":"2024-05-28T14:01:50.239351+0000","last_active":"2024-05-28T14:01:53.254155+0000","last_peered":"2024-05-28T14:01:53.254155+0000","last_clean":"2024-05-28T14:01:53.254155+0000","last_became_active":"2024-05-28T14:01:50.239057+0000","last_became_peered":"2024-05-28T14:01:50.239057+0000","last_unstale":"2024-05-28T14:01:53.254155+0000","last_undegraded":"2024-05-28T14:01:53.254155+0000","last_fullsized":"2024-05-28T14:01:53.254155+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:46:08.867269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278273+0000","last_change":"2024-05-28T14:01:51.258455+0000","last_active":"2024-05-28T14:01:54.278273+0000","last_peered":"2024-05-28T14:01:54.278273+0000","last_clean":"2024-05-28T14:01:54.278273+0000","last_became_active":"2024-05-28T14:01:51.258153+0000","last_became_peered":"2024-05-28T14:01:51.258153+0000","last_unstale":"2024-05-28T14:01:54.278273+0000","last_undegraded":"2024-05-28T14:01:54.278273+0000","last_fullsized":"2024-05-28T14:01:54.278273+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:54:18.537342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279757+0000","last_change":"2024-05-28T14:01:50.253758+0000","last_active":"2024-05-28T14:01:54.279757+0000","last_peered":"2024-05-28T14:01:54.279757+0000","last_clean":"2024-05-28T14:01:54.279757+0000","last_became_active":"2024-05-28T14:01:50.253358+0000","last_became_peered":"2024-05-28T14:01:50.253358+0000","last_unstale":"2024-05-28T14:01:54.279757+0000","last_undegraded":"2024-05-28T14:01:54.279757+0000","last_fullsized":"2024-05-28T14:01:54.279757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:46:08.867269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279667+0000","last_change":"2024-05-28T14:01:51.271166+0000","last_active":"2024-05-28T14:01:54.279667+0000","last_peered":"2024-05-28T14:01:54.279667+0000","last_clean":"2024-05-28T14:01:54.279667+0000","last_became_active":"2024-05-28T14:01:51.270617+0000","last_became_peered":"2024-05-28T14:01:51.270617+0000","last_unstale":"2024-05-28T14:01:54.279667+0000","last_undegraded":"2024-05-28T14:01:54.279667+0000","last_fullsized":"2024-05-28T14:01:54.279667+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:25:20.426709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.257572+0000","last_change":"2024-05-28T14:01:50.257500+0000","last_active":"2024-05-28T14:01:52.257572+0000","last_peered":"2024-05-28T14:01:52.257572+0000","last_clean":"2024-05-28T14:01:52.257572+0000","last_became_active":"2024-05-28T14:01:50.254671+0000","last_became_peered":"2024-05-28T14:01:50.254671+0000","last_unstale":"2024-05-28T14:01:52.257572+0000","last_undegraded":"2024-05-28T14:01:52.257572+0000","last_fullsized":"2024-05-28T14:01:52.257572+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:06:40.588874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254002+0000","last_change":"2024-05-28T14:01:51.278235+0000","last_active":"2024-05-28T14:01:53.254002+0000","last_peered":"2024-05-28T14:01:53.254002+0000","last_clean":"2024-05-28T14:01:53.254002+0000","last_became_active":"2024-05-28T14:01:51.277922+0000","last_became_peered":"2024-05-28T14:01:51.277922+0000","last_unstale":"2024-05-28T14:01:53.254002+0000","last_undegraded":"2024-05-28T14:01:53.254002+0000","last_fullsized":"2024-05-28T14:01:53.254002+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:02:45.159807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279536+0000","last_change":"2024-05-28T14:01:50.250535+0000","last_active":"2024-05-28T14:01:54.279536+0000","last_peered":"2024-05-28T14:01:54.279536+0000","last_clean":"2024-05-28T14:01:54.279536+0000","last_became_active":"2024-05-28T14:01:50.250233+0000","last_became_peered":"2024-05-28T14:01:50.250233+0000","last_unstale":"2024-05-28T14:01:54.279536+0000","last_undegraded":"2024-05-28T14:01:54.279536+0000","last_fullsized":"2024-05-28T14:01:54.279536+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:34:19.001030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254070+0000","last_change":"2024-05-28T14:01:51.271813+0000","last_active":"2024-05-28T14:01:53.254070+0000","last_peered":"2024-05-28T14:01:53.254070+0000","last_clean":"2024-05-28T14:01:53.254070+0000","last_became_active":"2024-05-28T14:01:51.271494+0000","last_became_peered":"2024-05-28T14:01:51.271494+0000","last_unstale":"2024-05-28T14:01:53.254070+0000","last_undegraded":"2024-05-28T14:01:53.254070+0000","last_fullsized":"2024-05-28T14:01:53.254070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:44:58.614519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262300+0000","last_change":"2024-05-28T14:01:50.240591+0000","last_active":"2024-05-28T14:01:52.262300+0000","last_peered":"2024-05-28T14:01:52.262300+0000","last_clean":"2024-05-28T14:01:52.262300+0000","last_became_active":"2024-05-28T14:01:50.240282+0000","last_became_peered":"2024-05-28T14:01:50.240282+0000","last_unstale":"2024-05-28T14:01:52.262300+0000","last_undegraded":"2024-05-28T14:01:52.262300+0000","last_fullsized":"2024-05-28T14:01:52.262300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:22:40.189131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262656+0000","last_change":"2024-05-28T14:01:51.271622+0000","last_active":"2024-05-28T14:01:52.262656+0000","last_peered":"2024-05-28T14:01:52.262656+0000","last_clean":"2024-05-28T14:01:52.262656+0000","last_became_active":"2024-05-28T14:01:51.271332+0000","last_became_peered":"2024-05-28T14:01:51.271332+0000","last_unstale":"2024-05-28T14:01:52.262656+0000","last_undegraded":"2024-05-28T14:01:52.262656+0000","last_fullsized":"2024-05-28T14:01:52.262656+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:30:40.263915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.257465+0000","last_change":"2024-05-28T14:01:50.251927+0000","last_active":"2024-05-28T14:01:52.257465+0000","last_peered":"2024-05-28T14:01:52.257465+0000","last_clean":"2024-05-28T14:01:52.257465+0000","last_became_active":"2024-05-28T14:01:50.251129+0000","last_became_peered":"2024-05-28T14:01:50.251129+0000","last_unstale":"2024-05-28T14:01:52.257465+0000","last_undegraded":"2024-05-28T14:01:52.257465+0000","last_fullsized":"2024-05-28T14:01:52.257465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:19:10.168187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262090+0000","last_change":"2024-05-28T14:01:51.266774+0000","last_active":"2024-05-28T14:01:52.262090+0000","last_peered":"2024-05-28T14:01:52.262090+0000","last_clean":"2024-05-28T14:01:52.262090+0000","last_became_active":"2024-05-28T14:01:51.266199+0000","last_became_peered":"2024-05-28T14:01:51.266199+0000","last_unstale":"2024-05-28T14:01:52.262090+0000","last_undegraded":"2024-05-28T14:01:52.262090+0000","last_fullsized":"2024-05-28T14:01:52.262090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:34:30.307831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278373+0000","last_change":"2024-05-28T14:01:50.248895+0000","last_active":"2024-05-28T14:01:54.278373+0000","last_peered":"2024-05-28T14:01:54.278373+0000","last_clean":"2024-05-28T14:01:54.278373+0000","last_became_active":"2024-05-28T14:01:50.247794+0000","last_became_peered":"2024-05-28T14:01:50.247794+0000","last_unstale":"2024-05-28T14:01:54.278373+0000","last_undegraded":"2024-05-28T14:01:54.278373+0000","last_fullsized":"2024-05-28T14:01:54.278373+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:28:08.079804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278870+0000","last_change":"2024-05-28T14:01:51.266696+0000","last_active":"2024-05-28T14:01:54.278870+0000","last_peered":"2024-05-28T14:01:54.278870+0000","last_clean":"2024-05-28T14:01:54.278870+0000","last_became_active":"2024-05-28T14:01:51.266422+0000","last_became_peered":"2024-05-28T14:01:51.266422+0000","last_unstale":"2024-05-28T14:01:54.278870+0000","last_undegraded":"2024-05-28T14:01:54.278870+0000","last_fullsized":"2024-05-28T14:01:54.278870+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:22:15.546134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262926+0000","last_change":"2024-05-28T14:01:50.249960+0000","last_active":"2024-05-28T14:01:52.262926+0000","last_peered":"2024-05-28T14:01:52.262926+0000","last_clean":"2024-05-28T14:01:52.262926+0000","last_became_active":"2024-05-28T14:01:50.249563+0000","last_became_peered":"2024-05-28T14:01:50.249563+0000","last_unstale":"2024-05-28T14:01:52.262926+0000","last_undegraded":"2024-05-28T14:01:52.262926+0000","last_fullsized":"2024-05-28T14:01:52.262926+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:29:45.717077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":216,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279214+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:01:54.279214+0000","last_peered":"2024-05-28T14:01:54.279214+0000","last_clean":"2024-05-28T14:01:54.279214+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:01:54.279214+0000","last_undegraded":"2024-05-28T14:01:54.279214+0000","last_fullsized":"2024-05-28T14:01:54.279214+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36448,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335392,"statfs":{"total":96636764160,"available":96599441408,"internally_reserved":0,"allocated":1126400,"data_stored":805293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":548864,"data_stored":221835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35876,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335964,"statfs":{"total":96636764160,"available":96600027136,"internally_reserved":0,"allocated":544768,"data_stored":221233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35884,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335956,"statfs":{"total":96636764160,"available":96600018944,"internally_reserved":0,"allocated":552960,"data_stored":222775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40044,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331796,"statfs":{"total":96636764160,"available":96595759104,"internally_reserved":0,"allocated":552960,"data_stored":221801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35876,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335964,"statfs":{"total":96636764160,"available":96600027136,"internally_reserved":0,"allocated":544768,"data_stored":216469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40032,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331808,"statfs":{"total":96636764160,"available":96595771392,"internally_reserved":0,"allocated":544768,"data_stored":221233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36440,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335400,"statfs":{"total":96636764160,"available":96599449600,"internally_reserved":0,"allocated":1126400,"data_stored":805237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:01:55.377 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:01:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f679e200640 1 Processor -- start 2024-05-28T14:01:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f679e200640 1 -- start start 2024-05-28T14:01:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f679e200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f679806c100 0x7f67980b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f679e200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6798147b40 0x7f6798145d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f679e200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67980b5bd0 con 0x7f67980b1b00 2024-05-28T14:01:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f679e200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6798145590 con 0x7f679806c100 2024-05-28T14:01:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f679e200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6798145870 con 0x7f6798147b40 2024-05-28T14:01:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f6797e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f679806c100 0x7f67980b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f679ce00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f67980b1b00 0x7f679806b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36814/0 (socket says 172.21.15.42:36814) 2024-05-28T14:01:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f679ce00640 1 -- 172.21.15.42:0/4252480763 learned_addr learned my addr 172.21.15.42:0/4252480763 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f679d800640 1 --2- 172.21.15.42:0/4252480763 >> v2:172.21.15.63:3300/0 conn(0x7f6798147b40 0x7f6798145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f6797e00640 1 -- 172.21.15.42:0/4252480763 >> v2:172.21.15.63:3300/0 conn(0x7f6798147b40 msgr2=0x7f6798145d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:55.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f6797e00640 1 --2- 172.21.15.42:0/4252480763 >> v2:172.21.15.63:3300/0 conn(0x7f6798147b40 0x7f6798145d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f6797e00640 1 -- 172.21.15.42:0/4252480763 >> v1:172.21.15.42:6789/0 conn(0x7f67980b1b00 legacy=0x7f679806b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f6797e00640 1 -- 172.21.15.42:0/4252480763 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67980b5e70 con 0x7f679806c100 2024-05-28T14:01:55.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f679d800640 1 --2- 172.21.15.42:0/4252480763 >> v2:172.21.15.63:3300/0 conn(0x7f6798147b40 0x7f6798145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:55.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.572+0000 7f6797400640 1 -- 172.21.15.42:0/4252480763 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2372998257 0 0) 0x7f67980b5bd0 con 0x7f67980b1b00 2024-05-28T14:01:55.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.576+0000 7f6797e00640 1 --2- 172.21.15.42:0/4252480763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f679806c100 0x7f67980b4280 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f679806ecf0 tx=0x7f678401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0197ec2533e9b7d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:55.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.576+0000 7f6797400640 1 -- 172.21.15.42:0/4252480763 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67840a7070 con 0x7f679806c100 2024-05-28T14:01:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.576+0000 7f6797400640 1 -- 172.21.15.42:0/4252480763 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67840b0070 con 0x7f679806c100 2024-05-28T14:01:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.576+0000 7f6797400640 1 -- 172.21.15.42:0/4252480763 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67840ab890 con 0x7f679806c100 2024-05-28T14:01:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.576+0000 7f679e200640 1 -- 172.21.15.42:0/4252480763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f679806c100 msgr2=0x7f67980b4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.576+0000 7f679e200640 1 --2- 172.21.15.42:0/4252480763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f679806c100 0x7f67980b4280 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f679806ecf0 tx=0x7f678401c450 comp rx=0 tx=0).stop 2024-05-28T14:01:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.576+0000 7f679e200640 1 -- 172.21.15.42:0/4252480763 shutdown_connections 2024-05-28T14:01:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.576+0000 7f679e200640 1 --2- 172.21.15.42:0/4252480763 >> v2:172.21.15.63:3300/0 conn(0x7f6798147b40 0x7f6798145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.576+0000 7f679e200640 1 --2- 172.21.15.42:0/4252480763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f679806c100 0x7f67980b4280 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.576+0000 7f679e200640 1 -- 172.21.15.42:0/4252480763 >> 172.21.15.42:0/4252480763 conn(0x7f67980b67a0 msgr2=0x7f67980b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.576+0000 7f679e200640 1 -- 172.21.15.42:0/4252480763 shutdown_connections 2024-05-28T14:01:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.576+0000 7f679e200640 1 -- 172.21.15.42:0/4252480763 wait complete. 2024-05-28T14:01:55.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.576+0000 7f679e200640 1 Processor -- start 2024-05-28T14:01:55.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.576+0000 7f679e200640 1 -- start start 2024-05-28T14:01:55.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f679806c100 0x7f679809e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67980b1b00 0x7f6798140f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6798147b40 0x7f67980ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679e200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6798146bd0 con 0x7f679806c100 2024-05-28T14:01:55.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679e200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6798146a50 con 0x7f67980b1b00 2024-05-28T14:01:55.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679e200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6798146d50 con 0x7f6798147b40 2024-05-28T14:01:55.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f679806c100 0x7f679809e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f679806c100 0x7f679809e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55244/0 (socket says 172.21.15.42:55244) 2024-05-28T14:01:55.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679ce00640 1 -- 172.21.15.42:0/3695209268 learned_addr learned my addr 172.21.15.42:0/3695209268 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:55.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f6797e00640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67980b1b00 0x7f6798140f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679ce00640 1 -- 172.21.15.42:0/3695209268 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6798147b40 msgr2=0x7f67980ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679d800640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6798147b40 0x7f67980ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679ce00640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6798147b40 0x7f67980ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679ce00640 1 -- 172.21.15.42:0/3695209268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67980b1b00 msgr2=0x7f6798140f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679ce00640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67980b1b00 0x7f6798140f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679ce00640 1 -- 172.21.15.42:0/3695209268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679809d480 con 0x7f679806c100 2024-05-28T14:01:55.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f6797e00640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67980b1b00 0x7f6798140f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:55.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679d800640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6798147b40 0x7f67980ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:55.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679ce00640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f679806c100 0x7f679809e520 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f678c00c550 tx=0x7f678c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:55.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f6794c00640 1 -- 172.21.15.42:0/3695209268 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c017070 con 0x7f679806c100 2024-05-28T14:01:55.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f6794c00640 1 -- 172.21.15.42:0/3695209268 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f678c005d10 con 0x7f679806c100 2024-05-28T14:01:55.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f6794c00640 1 -- 172.21.15.42:0/3695209268 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c007330 con 0x7f679806c100 2024-05-28T14:01:55.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679e200640 1 -- 172.21.15.42:0/3695209268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679809d760 con 0x7f679806c100 2024-05-28T14:01:55.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679e200640 1 -- 172.21.15.42:0/3695209268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f679809dd20 con 0x7f679806c100 2024-05-28T14:01:55.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f679e200640 1 -- 172.21.15.42:0/3695209268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f679804fcf0 con 0x7f679806c100 2024-05-28T14:01:55.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.580+0000 7f6794c00640 1 -- 172.21.15.42:0/3695209268 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f678c01a070 con 0x7f679806c100 2024-05-28T14:01:55.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.584+0000 7f6794c00640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6778068680 0x7f677806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.584+0000 7f6794c00640 1 -- 172.21.15.42:0/3695209268 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7f678c0862b0 con 0x7f679806c100 2024-05-28T14:01:55.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.584+0000 7f6797e00640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6778068680 0x7f677806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.584+0000 7f6797e00640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6778068680 0x7f677806ab40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f67840af000 tx=0x7f6784008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:55.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.588+0000 7f6794c00640 1 -- 172.21.15.42:0/3695209268 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f678c056470 con 0x7f679806c100 2024-05-28T14:01:55.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.748+0000 7f679e200640 1 -- 172.21.15.42:0/3695209268 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67980b6d90 con 0x7f6778068680 2024-05-28T14:01:55.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.756+0000 7f6794c00640 1 -- 172.21.15.42:0/3695209268 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251958 (secure 0 0 0) 0x7f67980b6d90 con 0x7f6778068680 2024-05-28T14:01:55.744 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:55.746 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:01:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.760+0000 7f679e200640 1 -- 172.21.15.42:0/3695209268 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6778068680 msgr2=0x7f677806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.760+0000 7f679e200640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6778068680 0x7f677806ab40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f67840af000 tx=0x7f6784008040 comp rx=0 tx=0).stop 2024-05-28T14:01:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.760+0000 7f679e200640 1 -- 172.21.15.42:0/3695209268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f679806c100 msgr2=0x7f679809e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.760+0000 7f679e200640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f679806c100 0x7f679809e520 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f678c00c550 tx=0x7f678c00ca10 comp rx=0 tx=0).stop 2024-05-28T14:01:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.760+0000 7f679e200640 1 -- 172.21.15.42:0/3695209268 shutdown_connections 2024-05-28T14:01:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.760+0000 7f679e200640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6778068680 0x7f677806ab40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.760+0000 7f679e200640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6798147b40 0x7f67980ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.760+0000 7f679e200640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67980b1b00 0x7f6798140f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.760+0000 7f679e200640 1 --2- 172.21.15.42:0/3695209268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f679806c100 0x7f679809e520 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.760+0000 7f679e200640 1 -- 172.21.15.42:0/3695209268 >> 172.21.15.42:0/3695209268 conn(0x7f67980b67a0 msgr2=0x7f67980aec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:55.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.760+0000 7f679e200640 1 -- 172.21.15.42:0/3695209268 shutdown_connections 2024-05-28T14:01:55.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.760+0000 7f679e200640 1 -- 172.21.15.42:0/3695209268 wait complete. 2024-05-28T14:01:55.764 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-05-28T14:01:54.899922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":292320,"kb_used_data":5412,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754682400,"statfs":{"total":773094113280,"available":772794777600,"internally_reserved":0,"allocated":5541888,"data_stored":2935876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":293654066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.678690"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279909+0000","last_change":"2024-05-28T14:01:51.279990+0000","last_active":"2024-05-28T14:01:54.279909+0000","last_peered":"2024-05-28T14:01:54.279909+0000","last_clean":"2024-05-28T14:01:54.279909+0000","last_became_active":"2024-05-28T14:01:51.279688+0000","last_became_peered":"2024-05-28T14:01:51.279688+0000","last_unstale":"2024-05-28T14:01:54.279909+0000","last_undegraded":"2024-05-28T14:01:54.279909+0000","last_fullsized":"2024-05-28T14:01:54.279909+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:30:06.684281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262852+0000","last_change":"2024-05-28T14:01:51.287112+0000","last_active":"2024-05-28T14:01:52.262852+0000","last_peered":"2024-05-28T14:01:52.262852+0000","last_clean":"2024-05-28T14:01:52.262852+0000","last_became_active":"2024-05-28T14:01:51.285855+0000","last_became_peered":"2024-05-28T14:01:51.285855+0000","last_unstale":"2024-05-28T14:01:52.262852+0000","last_undegraded":"2024-05-28T14:01:52.262852+0000","last_fullsized":"2024-05-28T14:01:52.262852+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:59:32.660570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263061+0000","last_change":"2024-05-28T14:01:51.262992+0000","last_active":"2024-05-28T14:01:52.263061+0000","last_peered":"2024-05-28T14:01:52.263061+0000","last_clean":"2024-05-28T14:01:52.263061+0000","last_became_active":"2024-05-28T14:01:51.262698+0000","last_became_peered":"2024-05-28T14:01:51.262698+0000","last_unstale":"2024-05-28T14:01:52.263061+0000","last_undegraded":"2024-05-28T14:01:52.263061+0000","last_fullsized":"2024-05-28T14:01:52.263061+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:54:18.537342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279077+0000","last_change":"2024-05-28T14:01:51.284598+0000","last_active":"2024-05-28T14:01:54.279077+0000","last_peered":"2024-05-28T14:01:54.279077+0000","last_clean":"2024-05-28T14:01:54.279077+0000","last_became_active":"2024-05-28T14:01:51.281894+0000","last_became_peered":"2024-05-28T14:01:51.281894+0000","last_unstale":"2024-05-28T14:01:54.279077+0000","last_undegraded":"2024-05-28T14:01:54.279077+0000","last_fullsized":"2024-05-28T14:01:54.279077+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:29:46.724802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263044+0000","last_change":"2024-05-28T14:01:51.287119+0000","last_active":"2024-05-28T14:01:52.263044+0000","last_peered":"2024-05-28T14:01:52.263044+0000","last_clean":"2024-05-28T14:01:52.263044+0000","last_became_active":"2024-05-28T14:01:51.286771+0000","last_became_peered":"2024-05-28T14:01:51.286771+0000","last_unstale":"2024-05-28T14:01:52.263044+0000","last_undegraded":"2024-05-28T14:01:52.263044+0000","last_fullsized":"2024-05-28T14:01:52.263044+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:42:52.419012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.253925+0000","last_change":"2024-05-28T14:01:51.262667+0000","last_active":"2024-05-28T14:01:53.253925+0000","last_peered":"2024-05-28T14:01:53.253925+0000","last_clean":"2024-05-28T14:01:53.253925+0000","last_became_active":"2024-05-28T14:01:51.262353+0000","last_became_peered":"2024-05-28T14:01:51.262353+0000","last_unstale":"2024-05-28T14:01:53.253925+0000","last_undegraded":"2024-05-28T14:01:53.253925+0000","last_fullsized":"2024-05-28T14:01:53.253925+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:55.017591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279358+0000","last_change":"2024-05-28T14:01:51.273056+0000","last_active":"2024-05-28T14:01:54.279358+0000","last_peered":"2024-05-28T14:01:54.279358+0000","last_clean":"2024-05-28T14:01:54.279358+0000","last_became_active":"2024-05-28T14:01:51.272752+0000","last_became_peered":"2024-05-28T14:01:51.272752+0000","last_unstale":"2024-05-28T14:01:54.279358+0000","last_undegraded":"2024-05-28T14:01:54.279358+0000","last_fullsized":"2024-05-28T14:01:54.279358+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:47:43.795047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.277851+0000","last_change":"2024-05-28T14:01:51.280585+0000","last_active":"2024-05-28T14:01:54.277851+0000","last_peered":"2024-05-28T14:01:54.277851+0000","last_clean":"2024-05-28T14:01:54.277851+0000","last_became_active":"2024-05-28T14:01:51.280319+0000","last_became_peered":"2024-05-28T14:01:51.280319+0000","last_unstale":"2024-05-28T14:01:54.277851+0000","last_undegraded":"2024-05-28T14:01:54.277851+0000","last_fullsized":"2024-05-28T14:01:54.277851+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:34:30.307831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279402+0000","last_change":"2024-05-28T14:01:51.285809+0000","last_active":"2024-05-28T14:01:54.279402+0000","last_peered":"2024-05-28T14:01:54.279402+0000","last_clean":"2024-05-28T14:01:54.279402+0000","last_became_active":"2024-05-28T14:01:51.284118+0000","last_became_peered":"2024-05-28T14:01:51.284118+0000","last_unstale":"2024-05-28T14:01:54.279402+0000","last_undegraded":"2024-05-28T14:01:54.279402+0000","last_fullsized":"2024-05-28T14:01:54.279402+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:10:42.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278712+0000","last_change":"2024-05-28T14:01:51.280306+0000","last_active":"2024-05-28T14:01:54.278712+0000","last_peered":"2024-05-28T14:01:54.278712+0000","last_clean":"2024-05-28T14:01:54.278712+0000","last_became_active":"2024-05-28T14:01:51.279990+0000","last_became_peered":"2024-05-28T14:01:51.279990+0000","last_unstale":"2024-05-28T14:01:54.278712+0000","last_undegraded":"2024-05-28T14:01:54.278712+0000","last_fullsized":"2024-05-28T14:01:54.278712+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:55:31.334010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278317+0000","last_change":"2024-05-28T14:01:51.285883+0000","last_active":"2024-05-28T14:01:54.278317+0000","last_peered":"2024-05-28T14:01:54.278317+0000","last_clean":"2024-05-28T14:01:54.278317+0000","last_became_active":"2024-05-28T14:01:51.284441+0000","last_became_peered":"2024-05-28T14:01:51.284441+0000","last_unstale":"2024-05-28T14:01:54.278317+0000","last_undegraded":"2024-05-28T14:01:54.278317+0000","last_fullsized":"2024-05-28T14:01:54.278317+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:26.331819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263055+0000","last_change":"2024-05-28T14:01:51.271958+0000","last_active":"2024-05-28T14:01:52.263055+0000","last_peered":"2024-05-28T14:01:52.263055+0000","last_clean":"2024-05-28T14:01:52.263055+0000","last_became_active":"2024-05-28T14:01:51.271664+0000","last_became_peered":"2024-05-28T14:01:51.271664+0000","last_unstale":"2024-05-28T14:01:52.263055+0000","last_undegraded":"2024-05-28T14:01:52.263055+0000","last_fullsized":"2024-05-28T14:01:52.263055+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:02:45.432903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278790+0000","last_change":"2024-05-28T14:01:51.265344+0000","last_active":"2024-05-28T14:01:54.278790+0000","last_peered":"2024-05-28T14:01:54.278790+0000","last_clean":"2024-05-28T14:01:54.278790+0000","last_became_active":"2024-05-28T14:01:51.265065+0000","last_became_peered":"2024-05-28T14:01:51.265065+0000","last_unstale":"2024-05-28T14:01:54.278790+0000","last_undegraded":"2024-05-28T14:01:54.278790+0000","last_fullsized":"2024-05-28T14:01:54.278790+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:11:37.087786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278672+0000","last_change":"2024-05-28T14:01:51.285217+0000","last_active":"2024-05-28T14:01:54.278672+0000","last_peered":"2024-05-28T14:01:54.278672+0000","last_clean":"2024-05-28T14:01:54.278672+0000","last_became_active":"2024-05-28T14:01:51.284885+0000","last_became_peered":"2024-05-28T14:01:51.284885+0000","last_unstale":"2024-05-28T14:01:54.278672+0000","last_undegraded":"2024-05-28T14:01:54.278672+0000","last_fullsized":"2024-05-28T14:01:54.278672+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:45:04.831620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.253736+0000","last_change":"2024-05-28T14:01:51.258743+0000","last_active":"2024-05-28T14:01:53.253736+0000","last_peered":"2024-05-28T14:01:53.253736+0000","last_clean":"2024-05-28T14:01:53.253736+0000","last_became_active":"2024-05-28T14:01:51.258473+0000","last_became_peered":"2024-05-28T14:01:51.258473+0000","last_unstale":"2024-05-28T14:01:53.253736+0000","last_undegraded":"2024-05-28T14:01:53.253736+0000","last_fullsized":"2024-05-28T14:01:53.253736+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:44:30.868394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014127+0000","last_change":"2024-05-28T14:01:51.286917+0000","last_active":"2024-05-28T14:01:53.014127+0000","last_peered":"2024-05-28T14:01:53.014127+0000","last_clean":"2024-05-28T14:01:53.014127+0000","last_became_active":"2024-05-28T14:01:51.285949+0000","last_became_peered":"2024-05-28T14:01:51.285949+0000","last_unstale":"2024-05-28T14:01:53.014127+0000","last_undegraded":"2024-05-28T14:01:53.014127+0000","last_fullsized":"2024-05-28T14:01:53.014127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:24:16.440714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263546+0000","last_change":"2024-05-28T14:01:51.274811+0000","last_active":"2024-05-28T14:01:52.263546+0000","last_peered":"2024-05-28T14:01:52.263546+0000","last_clean":"2024-05-28T14:01:52.263546+0000","last_became_active":"2024-05-28T14:01:51.274520+0000","last_became_peered":"2024-05-28T14:01:51.274520+0000","last_unstale":"2024-05-28T14:01:52.263546+0000","last_undegraded":"2024-05-28T14:01:52.263546+0000","last_fullsized":"2024-05-28T14:01:52.263546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:58:30.793739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263140+0000","last_change":"2024-05-28T14:01:51.273089+0000","last_active":"2024-05-28T14:01:52.263140+0000","last_peered":"2024-05-28T14:01:52.263140+0000","last_clean":"2024-05-28T14:01:52.263140+0000","last_became_active":"2024-05-28T14:01:51.272801+0000","last_became_peered":"2024-05-28T14:01:51.272801+0000","last_unstale":"2024-05-28T14:01:52.263140+0000","last_undegraded":"2024-05-28T14:01:52.263140+0000","last_fullsized":"2024-05-28T14:01:52.263140+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:26.331819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014855+0000","last_change":"2024-05-28T14:01:51.285668+0000","last_active":"2024-05-28T14:01:53.014855+0000","last_peered":"2024-05-28T14:01:53.014855+0000","last_clean":"2024-05-28T14:01:53.014855+0000","last_became_active":"2024-05-28T14:01:51.285356+0000","last_became_peered":"2024-05-28T14:01:51.285356+0000","last_unstale":"2024-05-28T14:01:53.014855+0000","last_undegraded":"2024-05-28T14:01:53.014855+0000","last_fullsized":"2024-05-28T14:01:53.014855+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:53:22.132470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278371+0000","last_change":"2024-05-28T14:01:51.268861+0000","last_active":"2024-05-28T14:01:54.278371+0000","last_peered":"2024-05-28T14:01:54.278371+0000","last_clean":"2024-05-28T14:01:54.278371+0000","last_became_active":"2024-05-28T14:01:51.268577+0000","last_became_peered":"2024-05-28T14:01:51.268577+0000","last_unstale":"2024-05-28T14:01:54.278371+0000","last_undegraded":"2024-05-28T14:01:54.278371+0000","last_fullsized":"2024-05-28T14:01:54.278371+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:23.608502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.253921+0000","last_change":"2024-05-28T14:01:51.278198+0000","last_active":"2024-05-28T14:01:53.253921+0000","last_peered":"2024-05-28T14:01:53.253921+0000","last_clean":"2024-05-28T14:01:53.253921+0000","last_became_active":"2024-05-28T14:01:51.277883+0000","last_became_peered":"2024-05-28T14:01:51.277883+0000","last_unstale":"2024-05-28T14:01:53.253921+0000","last_undegraded":"2024-05-28T14:01:53.253921+0000","last_fullsized":"2024-05-28T14:01:53.253921+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:50:22.503284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278822+0000","last_change":"2024-05-28T14:01:51.281113+0000","last_active":"2024-05-28T14:01:54.278822+0000","last_peered":"2024-05-28T14:01:54.278822+0000","last_clean":"2024-05-28T14:01:54.278822+0000","last_became_active":"2024-05-28T14:01:51.280833+0000","last_became_peered":"2024-05-28T14:01:51.280833+0000","last_unstale":"2024-05-28T14:01:54.278822+0000","last_undegraded":"2024-05-28T14:01:54.278822+0000","last_fullsized":"2024-05-28T14:01:54.278822+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:58:30.793739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014337+0000","last_change":"2024-05-28T14:01:51.280307+0000","last_active":"2024-05-28T14:01:53.014337+0000","last_peered":"2024-05-28T14:01:53.014337+0000","last_clean":"2024-05-28T14:01:53.014337+0000","last_became_active":"2024-05-28T14:01:51.279591+0000","last_became_peered":"2024-05-28T14:01:51.279591+0000","last_unstale":"2024-05-28T14:01:53.014337+0000","last_undegraded":"2024-05-28T14:01:53.014337+0000","last_fullsized":"2024-05-28T14:01:53.014337+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:33:55.389532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014172+0000","last_change":"2024-05-28T14:01:51.281163+0000","last_active":"2024-05-28T14:01:53.014172+0000","last_peered":"2024-05-28T14:01:53.014172+0000","last_clean":"2024-05-28T14:01:53.014172+0000","last_became_active":"2024-05-28T14:01:51.280877+0000","last_became_peered":"2024-05-28T14:01:51.280877+0000","last_unstale":"2024-05-28T14:01:53.014172+0000","last_undegraded":"2024-05-28T14:01:53.014172+0000","last_fullsized":"2024-05-28T14:01:53.014172+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:11:37.087786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014770+0000","last_change":"2024-05-28T14:01:51.285012+0000","last_active":"2024-05-28T14:01:53.014770+0000","last_peered":"2024-05-28T14:01:53.014770+0000","last_clean":"2024-05-28T14:01:53.014770+0000","last_became_active":"2024-05-28T14:01:51.284685+0000","last_became_peered":"2024-05-28T14:01:51.284685+0000","last_unstale":"2024-05-28T14:01:53.014770+0000","last_undegraded":"2024-05-28T14:01:53.014770+0000","last_fullsized":"2024-05-28T14:01:53.014770+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:07.570864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278558+0000","last_change":"2024-05-28T14:01:51.254804+0000","last_active":"2024-05-28T14:01:54.278558+0000","last_peered":"2024-05-28T14:01:54.278558+0000","last_clean":"2024-05-28T14:01:54.278558+0000","last_became_active":"2024-05-28T14:01:51.254509+0000","last_became_peered":"2024-05-28T14:01:51.254509+0000","last_unstale":"2024-05-28T14:01:54.278558+0000","last_undegraded":"2024-05-28T14:01:54.278558+0000","last_fullsized":"2024-05-28T14:01:54.278558+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:28:09.087529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263248+0000","last_change":"2024-05-28T14:01:51.272978+0000","last_active":"2024-05-28T14:01:52.263248+0000","last_peered":"2024-05-28T14:01:52.263248+0000","last_clean":"2024-05-28T14:01:52.263248+0000","last_became_active":"2024-05-28T14:01:51.272040+0000","last_became_peered":"2024-05-28T14:01:51.272040+0000","last_unstale":"2024-05-28T14:01:52.263248+0000","last_undegraded":"2024-05-28T14:01:52.263248+0000","last_fullsized":"2024-05-28T14:01:52.263248+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:45:04.831620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263279+0000","last_change":"2024-05-28T14:01:51.271937+0000","last_active":"2024-05-28T14:01:52.263279+0000","last_peered":"2024-05-28T14:01:52.263279+0000","last_clean":"2024-05-28T14:01:52.263279+0000","last_became_active":"2024-05-28T14:01:51.271633+0000","last_became_peered":"2024-05-28T14:01:51.271633+0000","last_unstale":"2024-05-28T14:01:52.263279+0000","last_undegraded":"2024-05-28T14:01:52.263279+0000","last_fullsized":"2024-05-28T14:01:52.263279+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:47:43.795047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014441+0000","last_change":"2024-05-28T14:01:51.282053+0000","last_active":"2024-05-28T14:01:53.014441+0000","last_peered":"2024-05-28T14:01:53.014441+0000","last_clean":"2024-05-28T14:01:53.014441+0000","last_became_active":"2024-05-28T14:01:51.281778+0000","last_became_peered":"2024-05-28T14:01:51.281778+0000","last_unstale":"2024-05-28T14:01:53.014441+0000","last_undegraded":"2024-05-28T14:01:53.014441+0000","last_fullsized":"2024-05-28T14:01:53.014441+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:19:11.175912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262089+0000","last_change":"2024-05-28T14:01:51.268383+0000","last_active":"2024-05-28T14:01:52.262089+0000","last_peered":"2024-05-28T14:01:52.262089+0000","last_clean":"2024-05-28T14:01:52.262089+0000","last_became_active":"2024-05-28T14:01:51.267929+0000","last_became_peered":"2024-05-28T14:01:51.267929+0000","last_unstale":"2024-05-28T14:01:52.262089+0000","last_undegraded":"2024-05-28T14:01:52.262089+0000","last_fullsized":"2024-05-28T14:01:52.262089+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:07:41.855434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.253896+0000","last_change":"2024-05-28T14:01:51.263422+0000","last_active":"2024-05-28T14:01:53.253896+0000","last_peered":"2024-05-28T14:01:53.253896+0000","last_clean":"2024-05-28T14:01:53.253896+0000","last_became_active":"2024-05-28T14:01:51.263160+0000","last_became_peered":"2024-05-28T14:01:51.263160+0000","last_unstale":"2024-05-28T14:01:53.253896+0000","last_undegraded":"2024-05-28T14:01:53.253896+0000","last_fullsized":"2024-05-28T14:01:53.253896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:10:16.843029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.277858+0000","last_change":"2024-05-28T14:01:51.269810+0000","last_active":"2024-05-28T14:01:54.277858+0000","last_peered":"2024-05-28T14:01:54.277858+0000","last_clean":"2024-05-28T14:01:54.277858+0000","last_became_active":"2024-05-28T14:01:51.269522+0000","last_became_peered":"2024-05-28T14:01:51.269522+0000","last_unstale":"2024-05-28T14:01:54.277858+0000","last_undegraded":"2024-05-28T14:01:54.277858+0000","last_fullsized":"2024-05-28T14:01:54.277858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:50:37.152282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.257874+0000","last_change":"2024-05-28T14:01:51.279615+0000","last_active":"2024-05-28T14:01:52.257874+0000","last_peered":"2024-05-28T14:01:52.257874+0000","last_clean":"2024-05-28T14:01:52.257874+0000","last_became_active":"2024-05-28T14:01:51.279345+0000","last_became_peered":"2024-05-28T14:01:51.279345+0000","last_unstale":"2024-05-28T14:01:52.257874+0000","last_undegraded":"2024-05-28T14:01:52.257874+0000","last_fullsized":"2024-05-28T14:01:52.257874+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:21:00.662269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254297+0000","last_change":"2024-05-28T14:01:50.247611+0000","last_active":"2024-05-28T14:01:53.254297+0000","last_peered":"2024-05-28T14:01:53.254297+0000","last_clean":"2024-05-28T14:01:53.254297+0000","last_became_active":"2024-05-28T14:01:50.246921+0000","last_became_peered":"2024-05-28T14:01:50.246921+0000","last_unstale":"2024-05-28T14:01:53.254297+0000","last_undegraded":"2024-05-28T14:01:53.254297+0000","last_fullsized":"2024-05-28T14:01:53.254297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:34:19.001030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014826+0000","last_change":"2024-05-28T14:01:51.281147+0000","last_active":"2024-05-28T14:01:53.014826+0000","last_peered":"2024-05-28T14:01:53.014826+0000","last_clean":"2024-05-28T14:01:53.014826+0000","last_became_active":"2024-05-28T14:01:51.280829+0000","last_became_peered":"2024-05-28T14:01:51.280829+0000","last_unstale":"2024-05-28T14:01:53.014826+0000","last_undegraded":"2024-05-28T14:01:53.014826+0000","last_fullsized":"2024-05-28T14:01:53.014826+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:39:26.852227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263373+0000","last_change":"2024-05-28T14:01:50.235938+0000","last_active":"2024-05-28T14:01:52.263373+0000","last_peered":"2024-05-28T14:01:52.263373+0000","last_clean":"2024-05-28T14:01:52.263373+0000","last_became_active":"2024-05-28T14:01:50.235580+0000","last_became_peered":"2024-05-28T14:01:50.235580+0000","last_unstale":"2024-05-28T14:01:52.263373+0000","last_undegraded":"2024-05-28T14:01:52.263373+0000","last_fullsized":"2024-05-28T14:01:52.263373+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:05.536748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263233+0000","last_change":"2024-05-28T14:01:51.282705+0000","last_active":"2024-05-28T14:01:52.263233+0000","last_peered":"2024-05-28T14:01:52.263233+0000","last_clean":"2024-05-28T14:01:52.263233+0000","last_became_active":"2024-05-28T14:01:51.282399+0000","last_became_peered":"2024-05-28T14:01:51.282399+0000","last_unstale":"2024-05-28T14:01:52.263233+0000","last_undegraded":"2024-05-28T14:01:52.263233+0000","last_fullsized":"2024-05-28T14:01:52.263233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:16:53.615918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.257884+0000","last_change":"2024-05-28T14:01:50.257519+0000","last_active":"2024-05-28T14:01:52.257884+0000","last_peered":"2024-05-28T14:01:52.257884+0000","last_clean":"2024-05-28T14:01:52.257884+0000","last_became_active":"2024-05-28T14:01:50.257237+0000","last_became_peered":"2024-05-28T14:01:50.257237+0000","last_unstale":"2024-05-28T14:01:52.257884+0000","last_undegraded":"2024-05-28T14:01:52.257884+0000","last_fullsized":"2024-05-28T14:01:52.257884+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:37:38.552292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.257935+0000","last_change":"2024-05-28T14:01:51.276500+0000","last_active":"2024-05-28T14:01:52.257935+0000","last_peered":"2024-05-28T14:01:52.257935+0000","last_clean":"2024-05-28T14:01:52.257935+0000","last_became_active":"2024-05-28T14:01:51.276213+0000","last_became_peered":"2024-05-28T14:01:51.276213+0000","last_unstale":"2024-05-28T14:01:52.257935+0000","last_undegraded":"2024-05-28T14:01:52.257935+0000","last_fullsized":"2024-05-28T14:01:52.257935+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:36:21.006195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279525+0000","last_change":"2024-05-28T14:01:50.252200+0000","last_active":"2024-05-28T14:01:54.279525+0000","last_peered":"2024-05-28T14:01:54.279525+0000","last_clean":"2024-05-28T14:01:54.279525+0000","last_became_active":"2024-05-28T14:01:50.250940+0000","last_became_peered":"2024-05-28T14:01:50.250940+0000","last_unstale":"2024-05-28T14:01:54.279525+0000","last_undegraded":"2024-05-28T14:01:54.279525+0000","last_fullsized":"2024-05-28T14:01:54.279525+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:20:52.388988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262100+0000","last_change":"2024-05-28T14:01:51.281796+0000","last_active":"2024-05-28T14:01:52.262100+0000","last_peered":"2024-05-28T14:01:52.262100+0000","last_clean":"2024-05-28T14:01:52.262100+0000","last_became_active":"2024-05-28T14:01:51.281330+0000","last_became_peered":"2024-05-28T14:01:51.281330+0000","last_unstale":"2024-05-28T14:01:52.262100+0000","last_undegraded":"2024-05-28T14:01:52.262100+0000","last_fullsized":"2024-05-28T14:01:52.262100+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:33:07.768264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254296+0000","last_change":"2024-05-28T14:01:50.242765+0000","last_active":"2024-05-28T14:01:53.254296+0000","last_peered":"2024-05-28T14:01:53.254296+0000","last_clean":"2024-05-28T14:01:53.254296+0000","last_became_active":"2024-05-28T14:01:50.241957+0000","last_became_peered":"2024-05-28T14:01:50.241957+0000","last_unstale":"2024-05-28T14:01:53.254296+0000","last_undegraded":"2024-05-28T14:01:53.254296+0000","last_fullsized":"2024-05-28T14:01:53.254296+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:38:40.676552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279142+0000","last_change":"2024-05-28T14:01:51.254960+0000","last_active":"2024-05-28T14:01:54.279142+0000","last_peered":"2024-05-28T14:01:54.279142+0000","last_clean":"2024-05-28T14:01:54.279142+0000","last_became_active":"2024-05-28T14:01:51.254654+0000","last_became_peered":"2024-05-28T14:01:51.254654+0000","last_unstale":"2024-05-28T14:01:54.279142+0000","last_undegraded":"2024-05-28T14:01:54.279142+0000","last_fullsized":"2024-05-28T14:01:54.279142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:23:35.885615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262184+0000","last_change":"2024-05-28T14:01:50.248702+0000","last_active":"2024-05-28T14:01:52.262184+0000","last_peered":"2024-05-28T14:01:52.262184+0000","last_clean":"2024-05-28T14:01:52.262184+0000","last_became_active":"2024-05-28T14:01:50.247906+0000","last_became_peered":"2024-05-28T14:01:50.247906+0000","last_unstale":"2024-05-28T14:01:52.262184+0000","last_undegraded":"2024-05-28T14:01:52.262184+0000","last_fullsized":"2024-05-28T14:01:52.262184+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:11:36.080061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278027+0000","last_change":"2024-05-28T14:01:51.280366+0000","last_active":"2024-05-28T14:01:54.278027+0000","last_peered":"2024-05-28T14:01:54.278027+0000","last_clean":"2024-05-28T14:01:54.278027+0000","last_became_active":"2024-05-28T14:01:51.280080+0000","last_became_peered":"2024-05-28T14:01:51.280080+0000","last_unstale":"2024-05-28T14:01:54.278027+0000","last_undegraded":"2024-05-28T14:01:54.278027+0000","last_fullsized":"2024-05-28T14:01:54.278027+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:33:07.768264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262431+0000","last_change":"2024-05-28T14:01:50.255043+0000","last_active":"2024-05-28T14:01:52.262431+0000","last_peered":"2024-05-28T14:01:52.262431+0000","last_clean":"2024-05-28T14:01:52.262431+0000","last_became_active":"2024-05-28T14:01:50.253917+0000","last_became_peered":"2024-05-28T14:01:50.253917+0000","last_unstale":"2024-05-28T14:01:52.262431+0000","last_undegraded":"2024-05-28T14:01:52.262431+0000","last_fullsized":"2024-05-28T14:01:52.262431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:10:41.806648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014335+0000","last_change":"2024-05-28T14:01:51.280547+0000","last_active":"2024-05-28T14:01:53.014335+0000","last_peered":"2024-05-28T14:01:53.014335+0000","last_clean":"2024-05-28T14:01:53.014335+0000","last_became_active":"2024-05-28T14:01:51.279941+0000","last_became_peered":"2024-05-28T14:01:51.279941+0000","last_unstale":"2024-05-28T14:01:53.014335+0000","last_undegraded":"2024-05-28T14:01:53.014335+0000","last_fullsized":"2024-05-28T14:01:53.014335+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:22:41.196856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278147+0000","last_change":"2024-05-28T14:01:50.253530+0000","last_active":"2024-05-28T14:01:54.278147+0000","last_peered":"2024-05-28T14:01:54.278147+0000","last_clean":"2024-05-28T14:01:54.278147+0000","last_became_active":"2024-05-28T14:01:50.253218+0000","last_became_peered":"2024-05-28T14:01:50.253218+0000","last_unstale":"2024-05-28T14:01:54.278147+0000","last_undegraded":"2024-05-28T14:01:54.278147+0000","last_fullsized":"2024-05-28T14:01:54.278147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:37:16.273440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263305+0000","last_change":"2024-05-28T14:01:51.281785+0000","last_active":"2024-05-28T14:01:52.263305+0000","last_peered":"2024-05-28T14:01:52.263305+0000","last_clean":"2024-05-28T14:01:52.263305+0000","last_became_active":"2024-05-28T14:01:51.281460+0000","last_became_peered":"2024-05-28T14:01:51.281460+0000","last_unstale":"2024-05-28T14:01:52.263305+0000","last_undegraded":"2024-05-28T14:01:52.263305+0000","last_fullsized":"2024-05-28T14:01:52.263305+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:55:31.334010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263585+0000","last_change":"2024-05-28T14:01:50.244865+0000","last_active":"2024-05-28T14:01:52.263585+0000","last_peered":"2024-05-28T14:01:52.263585+0000","last_clean":"2024-05-28T14:01:52.263585+0000","last_became_active":"2024-05-28T14:01:50.244523+0000","last_became_peered":"2024-05-28T14:01:50.244523+0000","last_unstale":"2024-05-28T14:01:52.263585+0000","last_undegraded":"2024-05-28T14:01:52.263585+0000","last_fullsized":"2024-05-28T14:01:52.263585+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:19:10.168187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.280009+0000","last_change":"2024-05-28T14:01:51.277015+0000","last_active":"2024-05-28T14:01:54.280009+0000","last_peered":"2024-05-28T14:01:54.280009+0000","last_clean":"2024-05-28T14:01:54.280009+0000","last_became_active":"2024-05-28T14:01:51.276689+0000","last_became_peered":"2024-05-28T14:01:51.276689+0000","last_unstale":"2024-05-28T14:01:54.280009+0000","last_undegraded":"2024-05-28T14:01:54.280009+0000","last_fullsized":"2024-05-28T14:01:54.280009+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:37:36.755762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278932+0000","last_change":"2024-05-28T14:01:50.255287+0000","last_active":"2024-05-28T14:01:54.278932+0000","last_peered":"2024-05-28T14:01:54.278932+0000","last_clean":"2024-05-28T14:01:54.278932+0000","last_became_active":"2024-05-28T14:01:50.254641+0000","last_became_peered":"2024-05-28T14:01:50.254641+0000","last_unstale":"2024-05-28T14:01:54.278932+0000","last_undegraded":"2024-05-28T14:01:54.278932+0000","last_fullsized":"2024-05-28T14:01:54.278932+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:50:36.144557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254337+0000","last_change":"2024-05-28T14:01:51.273569+0000","last_active":"2024-05-28T14:01:53.254337+0000","last_peered":"2024-05-28T14:01:53.254337+0000","last_clean":"2024-05-28T14:01:53.254337+0000","last_became_active":"2024-05-28T14:01:51.273292+0000","last_became_peered":"2024-05-28T14:01:51.273292+0000","last_unstale":"2024-05-28T14:01:53.254337+0000","last_undegraded":"2024-05-28T14:01:53.254337+0000","last_fullsized":"2024-05-28T14:01:53.254337+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:19:09.628626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279757+0000","last_change":"2024-05-28T14:01:50.253565+0000","last_active":"2024-05-28T14:01:54.279757+0000","last_peered":"2024-05-28T14:01:54.279757+0000","last_clean":"2024-05-28T14:01:54.279757+0000","last_became_active":"2024-05-28T14:01:50.253125+0000","last_became_peered":"2024-05-28T14:01:50.253125+0000","last_unstale":"2024-05-28T14:01:54.279757+0000","last_undegraded":"2024-05-28T14:01:54.279757+0000","last_fullsized":"2024-05-28T14:01:54.279757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:12:21.809608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279668+0000","last_change":"2024-05-28T14:01:51.273964+0000","last_active":"2024-05-28T14:01:54.279668+0000","last_peered":"2024-05-28T14:01:54.279668+0000","last_clean":"2024-05-28T14:01:54.279668+0000","last_became_active":"2024-05-28T14:01:51.273576+0000","last_became_peered":"2024-05-28T14:01:51.273576+0000","last_unstale":"2024-05-28T14:01:54.279668+0000","last_undegraded":"2024-05-28T14:01:54.279668+0000","last_fullsized":"2024-05-28T14:01:54.279668+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:07:10.966218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278413+0000","last_change":"2024-05-28T14:01:50.237388+0000","last_active":"2024-05-28T14:01:54.278413+0000","last_peered":"2024-05-28T14:01:54.278413+0000","last_clean":"2024-05-28T14:01:54.278413+0000","last_became_active":"2024-05-28T14:01:50.236288+0000","last_became_peered":"2024-05-28T14:01:50.236288+0000","last_unstale":"2024-05-28T14:01:54.278413+0000","last_undegraded":"2024-05-28T14:01:54.278413+0000","last_fullsized":"2024-05-28T14:01:54.278413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:13:46.276215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014331+0000","last_change":"2024-05-28T14:01:51.285350+0000","last_active":"2024-05-28T14:01:53.014331+0000","last_peered":"2024-05-28T14:01:53.014331+0000","last_clean":"2024-05-28T14:01:53.014331+0000","last_became_active":"2024-05-28T14:01:51.284913+0000","last_became_peered":"2024-05-28T14:01:51.284913+0000","last_unstale":"2024-05-28T14:01:53.014331+0000","last_undegraded":"2024-05-28T14:01:53.014331+0000","last_fullsized":"2024-05-28T14:01:53.014331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:06.544473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254315+0000","last_change":"2024-05-28T14:01:50.241618+0000","last_active":"2024-05-28T14:01:53.254315+0000","last_peered":"2024-05-28T14:01:53.254315+0000","last_clean":"2024-05-28T14:01:53.254315+0000","last_became_active":"2024-05-28T14:01:50.240635+0000","last_became_peered":"2024-05-28T14:01:50.240635+0000","last_unstale":"2024-05-28T14:01:53.254315+0000","last_undegraded":"2024-05-28T14:01:53.254315+0000","last_fullsized":"2024-05-28T14:01:53.254315+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:08:35.087398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279417+0000","last_change":"2024-05-28T14:01:51.265946+0000","last_active":"2024-05-28T14:01:54.279417+0000","last_peered":"2024-05-28T14:01:54.279417+0000","last_clean":"2024-05-28T14:01:54.279417+0000","last_became_active":"2024-05-28T14:01:51.265651+0000","last_became_peered":"2024-05-28T14:01:51.265651+0000","last_unstale":"2024-05-28T14:01:54.279417+0000","last_undegraded":"2024-05-28T14:01:54.279417+0000","last_fullsized":"2024-05-28T14:01:54.279417+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:34:20.008756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279504+0000","last_change":"2024-05-28T14:01:50.251360+0000","last_active":"2024-05-28T14:01:54.279504+0000","last_peered":"2024-05-28T14:01:54.279504+0000","last_clean":"2024-05-28T14:01:54.279504+0000","last_became_active":"2024-05-28T14:01:50.251058+0000","last_became_peered":"2024-05-28T14:01:50.251058+0000","last_unstale":"2024-05-28T14:01:54.279504+0000","last_undegraded":"2024-05-28T14:01:54.279504+0000","last_fullsized":"2024-05-28T14:01:54.279504+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:10:41.806648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.253736+0000","last_change":"2024-05-28T14:01:51.260428+0000","last_active":"2024-05-28T14:01:53.253736+0000","last_peered":"2024-05-28T14:01:53.253736+0000","last_clean":"2024-05-28T14:01:53.253736+0000","last_became_active":"2024-05-28T14:01:51.260117+0000","last_became_peered":"2024-05-28T14:01:51.260117+0000","last_unstale":"2024-05-28T14:01:53.253736+0000","last_undegraded":"2024-05-28T14:01:53.253736+0000","last_fullsized":"2024-05-28T14:01:53.253736+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:37:36.755762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254128+0000","last_change":"2024-05-28T14:01:50.240951+0000","last_active":"2024-05-28T14:01:53.254128+0000","last_peered":"2024-05-28T14:01:53.254128+0000","last_clean":"2024-05-28T14:01:53.254128+0000","last_became_active":"2024-05-28T14:01:50.240657+0000","last_became_peered":"2024-05-28T14:01:50.240657+0000","last_unstale":"2024-05-28T14:01:53.254128+0000","last_undegraded":"2024-05-28T14:01:53.254128+0000","last_fullsized":"2024-05-28T14:01:53.254128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:12:21.809608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279439+0000","last_change":"2024-05-28T14:01:51.278672+0000","last_active":"2024-05-28T14:01:54.279439+0000","last_peered":"2024-05-28T14:01:54.279439+0000","last_clean":"2024-05-28T14:01:54.279439+0000","last_became_active":"2024-05-28T14:01:51.277254+0000","last_became_peered":"2024-05-28T14:01:51.277254+0000","last_unstale":"2024-05-28T14:01:54.279439+0000","last_undegraded":"2024-05-28T14:01:54.279439+0000","last_fullsized":"2024-05-28T14:01:54.279439+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:38:34.772356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":22,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278263+0000","last_change":"2024-05-28T14:01:50.249113+0000","last_active":"2024-05-28T14:01:54.278263+0000","last_peered":"2024-05-28T14:01:54.278263+0000","last_clean":"2024-05-28T14:01:54.278263+0000","last_became_active":"2024-05-28T14:01:50.248646+0000","last_became_peered":"2024-05-28T14:01:50.248646+0000","last_unstale":"2024-05-28T14:01:54.278263+0000","last_undegraded":"2024-05-28T14:01:54.278263+0000","last_fullsized":"2024-05-28T14:01:54.278263+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:48:22.600777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278773+0000","last_change":"2024-05-28T14:01:51.259618+0000","last_active":"2024-05-28T14:01:54.278773+0000","last_peered":"2024-05-28T14:01:54.278773+0000","last_clean":"2024-05-28T14:01:54.278773+0000","last_became_active":"2024-05-28T14:01:51.259327+0000","last_became_peered":"2024-05-28T14:01:51.259327+0000","last_unstale":"2024-05-28T14:01:54.278773+0000","last_undegraded":"2024-05-28T14:01:54.278773+0000","last_fullsized":"2024-05-28T14:01:54.278773+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:24:16.440714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279048+0000","last_change":"2024-05-28T14:01:50.244480+0000","last_active":"2024-05-28T14:01:54.279048+0000","last_peered":"2024-05-28T14:01:54.279048+0000","last_clean":"2024-05-28T14:01:54.279048+0000","last_became_active":"2024-05-28T14:01:50.243160+0000","last_became_peered":"2024-05-28T14:01:50.243160+0000","last_unstale":"2024-05-28T14:01:54.279048+0000","last_undegraded":"2024-05-28T14:01:54.279048+0000","last_fullsized":"2024-05-28T14:01:54.279048+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:50:33.689225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279492+0000","last_change":"2024-05-28T14:01:51.259313+0000","last_active":"2024-05-28T14:01:54.279492+0000","last_peered":"2024-05-28T14:01:54.279492+0000","last_clean":"2024-05-28T14:01:54.279492+0000","last_became_active":"2024-05-28T14:01:51.258231+0000","last_became_peered":"2024-05-28T14:01:51.258231+0000","last_unstale":"2024-05-28T14:01:54.279492+0000","last_undegraded":"2024-05-28T14:01:54.279492+0000","last_fullsized":"2024-05-28T14:01:54.279492+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:12:22.817333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.263375+0000","last_change":"2024-05-28T14:01:50.254749+0000","last_active":"2024-05-28T14:01:52.263375+0000","last_peered":"2024-05-28T14:01:52.263375+0000","last_clean":"2024-05-28T14:01:52.263375+0000","last_became_active":"2024-05-28T14:01:50.254428+0000","last_became_peered":"2024-05-28T14:01:50.254428+0000","last_unstale":"2024-05-28T14:01:52.263375+0000","last_undegraded":"2024-05-28T14:01:52.263375+0000","last_fullsized":"2024-05-28T14:01:52.263375+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:33:54.381807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014520+0000","last_change":"2024-05-28T14:01:51.286227+0000","last_active":"2024-05-28T14:01:53.014520+0000","last_peered":"2024-05-28T14:01:53.014520+0000","last_clean":"2024-05-28T14:01:53.014520+0000","last_became_active":"2024-05-28T14:01:51.284627+0000","last_became_peered":"2024-05-28T14:01:51.284627+0000","last_unstale":"2024-05-28T14:01:53.014520+0000","last_undegraded":"2024-05-28T14:01:53.014520+0000","last_fullsized":"2024-05-28T14:01:53.014520+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:06:41.596599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262665+0000","last_change":"2024-05-28T14:01:50.258828+0000","last_active":"2024-05-28T14:01:52.262665+0000","last_peered":"2024-05-28T14:01:52.262665+0000","last_clean":"2024-05-28T14:01:52.262665+0000","last_became_active":"2024-05-28T14:01:50.258581+0000","last_became_peered":"2024-05-28T14:01:50.258581+0000","last_unstale":"2024-05-28T14:01:52.262665+0000","last_undegraded":"2024-05-28T14:01:52.262665+0000","last_fullsized":"2024-05-28T14:01:52.262665+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:23:34.877889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014698+0000","last_change":"2024-05-28T14:01:51.282904+0000","last_active":"2024-05-28T14:01:53.014698+0000","last_peered":"2024-05-28T14:01:53.014698+0000","last_clean":"2024-05-28T14:01:53.014698+0000","last_became_active":"2024-05-28T14:01:51.282479+0000","last_became_peered":"2024-05-28T14:01:51.282479+0000","last_unstale":"2024-05-28T14:01:53.014698+0000","last_undegraded":"2024-05-28T14:01:53.014698+0000","last_fullsized":"2024-05-28T14:01:53.014698+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:37:39.560017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254197+0000","last_change":"2024-05-28T14:01:50.247282+0000","last_active":"2024-05-28T14:01:53.254197+0000","last_peered":"2024-05-28T14:01:53.254197+0000","last_clean":"2024-05-28T14:01:53.254197+0000","last_became_active":"2024-05-28T14:01:50.246963+0000","last_became_peered":"2024-05-28T14:01:50.246963+0000","last_unstale":"2024-05-28T14:01:53.254197+0000","last_undegraded":"2024-05-28T14:01:53.254197+0000","last_fullsized":"2024-05-28T14:01:53.254197+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:20:52.388988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.253798+0000","last_change":"2024-05-28T14:01:51.258939+0000","last_active":"2024-05-28T14:01:53.253798+0000","last_peered":"2024-05-28T14:01:53.253798+0000","last_clean":"2024-05-28T14:01:53.253798+0000","last_became_active":"2024-05-28T14:01:51.258612+0000","last_became_peered":"2024-05-28T14:01:51.258612+0000","last_unstale":"2024-05-28T14:01:53.253798+0000","last_undegraded":"2024-05-28T14:01:53.253798+0000","last_fullsized":"2024-05-28T14:01:53.253798+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:04:37.469912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254447+0000","last_change":"2024-05-28T14:01:50.242434+0000","last_active":"2024-05-28T14:01:53.254447+0000","last_peered":"2024-05-28T14:01:53.254447+0000","last_clean":"2024-05-28T14:01:53.254447+0000","last_became_active":"2024-05-28T14:01:50.241077+0000","last_became_peered":"2024-05-28T14:01:50.241077+0000","last_unstale":"2024-05-28T14:01:53.254447+0000","last_undegraded":"2024-05-28T14:01:53.254447+0000","last_fullsized":"2024-05-28T14:01:53.254447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:30:39.256190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.257691+0000","last_change":"2024-05-28T14:01:51.278971+0000","last_active":"2024-05-28T14:01:52.257691+0000","last_peered":"2024-05-28T14:01:52.257691+0000","last_clean":"2024-05-28T14:01:52.257691+0000","last_became_active":"2024-05-28T14:01:51.278718+0000","last_became_peered":"2024-05-28T14:01:51.278718+0000","last_unstale":"2024-05-28T14:01:52.257691+0000","last_undegraded":"2024-05-28T14:01:52.257691+0000","last_fullsized":"2024-05-28T14:01:52.257691+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:32:16.010129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262423+0000","last_change":"2024-05-28T14:01:50.249518+0000","last_active":"2024-05-28T14:01:52.262423+0000","last_peered":"2024-05-28T14:01:52.262423+0000","last_clean":"2024-05-28T14:01:52.262423+0000","last_became_active":"2024-05-28T14:01:50.246638+0000","last_became_peered":"2024-05-28T14:01:50.246638+0000","last_unstale":"2024-05-28T14:01:52.262423+0000","last_undegraded":"2024-05-28T14:01:52.262423+0000","last_fullsized":"2024-05-28T14:01:52.262423+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:22:14.538409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.277993+0000","last_change":"2024-05-28T14:01:51.267456+0000","last_active":"2024-05-28T14:01:54.277993+0000","last_peered":"2024-05-28T14:01:54.277993+0000","last_clean":"2024-05-28T14:01:54.277993+0000","last_became_active":"2024-05-28T14:01:51.267083+0000","last_became_peered":"2024-05-28T14:01:51.267083+0000","last_unstale":"2024-05-28T14:01:54.277993+0000","last_undegraded":"2024-05-28T14:01:54.277993+0000","last_fullsized":"2024-05-28T14:01:54.277993+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:37:17.281165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.277928+0000","last_change":"2024-05-28T14:01:50.255380+0000","last_active":"2024-05-28T14:01:54.277928+0000","last_peered":"2024-05-28T14:01:54.277928+0000","last_clean":"2024-05-28T14:01:54.277928+0000","last_became_active":"2024-05-28T14:01:50.255076+0000","last_became_peered":"2024-05-28T14:01:50.255076+0000","last_unstale":"2024-05-28T14:01:54.277928+0000","last_undegraded":"2024-05-28T14:01:54.277928+0000","last_fullsized":"2024-05-28T14:01:54.277928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:06:22.496792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.277866+0000","last_change":"2024-05-28T14:01:51.281100+0000","last_active":"2024-05-28T14:01:54.277866+0000","last_peered":"2024-05-28T14:01:54.277866+0000","last_clean":"2024-05-28T14:01:54.277866+0000","last_became_active":"2024-05-28T14:01:51.280804+0000","last_became_peered":"2024-05-28T14:01:51.280804+0000","last_unstale":"2024-05-28T14:01:54.277866+0000","last_undegraded":"2024-05-28T14:01:54.277866+0000","last_fullsized":"2024-05-28T14:01:54.277866+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:07:41.855434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279350+0000","last_change":"2024-05-28T14:01:50.248761+0000","last_active":"2024-05-28T14:01:54.279350+0000","last_peered":"2024-05-28T14:01:54.279350+0000","last_clean":"2024-05-28T14:01:54.279350+0000","last_became_active":"2024-05-28T14:01:50.248430+0000","last_became_peered":"2024-05-28T14:01:50.248430+0000","last_unstale":"2024-05-28T14:01:54.279350+0000","last_undegraded":"2024-05-28T14:01:54.279350+0000","last_fullsized":"2024-05-28T14:01:54.279350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:23:34.877889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.014876+0000","last_change":"2024-05-28T14:01:51.289347+0000","last_active":"2024-05-28T14:01:53.014876+0000","last_peered":"2024-05-28T14:01:53.014876+0000","last_clean":"2024-05-28T14:01:53.014876+0000","last_became_active":"2024-05-28T14:01:51.289063+0000","last_became_peered":"2024-05-28T14:01:51.289063+0000","last_unstale":"2024-05-28T14:01:53.014876+0000","last_undegraded":"2024-05-28T14:01:53.014876+0000","last_fullsized":"2024-05-28T14:01:53.014876+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:32:16.010129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254155+0000","last_change":"2024-05-28T14:01:50.239351+0000","last_active":"2024-05-28T14:01:53.254155+0000","last_peered":"2024-05-28T14:01:53.254155+0000","last_clean":"2024-05-28T14:01:53.254155+0000","last_became_active":"2024-05-28T14:01:50.239057+0000","last_became_peered":"2024-05-28T14:01:50.239057+0000","last_unstale":"2024-05-28T14:01:53.254155+0000","last_undegraded":"2024-05-28T14:01:53.254155+0000","last_fullsized":"2024-05-28T14:01:53.254155+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:46:08.867269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278273+0000","last_change":"2024-05-28T14:01:51.258455+0000","last_active":"2024-05-28T14:01:54.278273+0000","last_peered":"2024-05-28T14:01:54.278273+0000","last_clean":"2024-05-28T14:01:54.278273+0000","last_became_active":"2024-05-28T14:01:51.258153+0000","last_became_peered":"2024-05-28T14:01:51.258153+0000","last_unstale":"2024-05-28T14:01:54.278273+0000","last_undegraded":"2024-05-28T14:01:54.278273+0000","last_fullsized":"2024-05-28T14:01:54.278273+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:54:18.537342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279757+0000","last_change":"2024-05-28T14:01:50.253758+0000","last_active":"2024-05-28T14:01:54.279757+0000","last_peered":"2024-05-28T14:01:54.279757+0000","last_clean":"2024-05-28T14:01:54.279757+0000","last_became_active":"2024-05-28T14:01:50.253358+0000","last_became_peered":"2024-05-28T14:01:50.253358+0000","last_unstale":"2024-05-28T14:01:54.279757+0000","last_undegraded":"2024-05-28T14:01:54.279757+0000","last_fullsized":"2024-05-28T14:01:54.279757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:46:08.867269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279667+0000","last_change":"2024-05-28T14:01:51.271166+0000","last_active":"2024-05-28T14:01:54.279667+0000","last_peered":"2024-05-28T14:01:54.279667+0000","last_clean":"2024-05-28T14:01:54.279667+0000","last_became_active":"2024-05-28T14:01:51.270617+0000","last_became_peered":"2024-05-28T14:01:51.270617+0000","last_unstale":"2024-05-28T14:01:54.279667+0000","last_undegraded":"2024-05-28T14:01:54.279667+0000","last_fullsized":"2024-05-28T14:01:54.279667+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:25:20.426709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.257572+0000","last_change":"2024-05-28T14:01:50.257500+0000","last_active":"2024-05-28T14:01:52.257572+0000","last_peered":"2024-05-28T14:01:52.257572+0000","last_clean":"2024-05-28T14:01:52.257572+0000","last_became_active":"2024-05-28T14:01:50.254671+0000","last_became_peered":"2024-05-28T14:01:50.254671+0000","last_unstale":"2024-05-28T14:01:52.257572+0000","last_undegraded":"2024-05-28T14:01:52.257572+0000","last_fullsized":"2024-05-28T14:01:52.257572+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:06:40.588874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254002+0000","last_change":"2024-05-28T14:01:51.278235+0000","last_active":"2024-05-28T14:01:53.254002+0000","last_peered":"2024-05-28T14:01:53.254002+0000","last_clean":"2024-05-28T14:01:53.254002+0000","last_became_active":"2024-05-28T14:01:51.277922+0000","last_became_peered":"2024-05-28T14:01:51.277922+0000","last_unstale":"2024-05-28T14:01:53.254002+0000","last_undegraded":"2024-05-28T14:01:53.254002+0000","last_fullsized":"2024-05-28T14:01:53.254002+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:02:45.159807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279536+0000","last_change":"2024-05-28T14:01:50.250535+0000","last_active":"2024-05-28T14:01:54.279536+0000","last_peered":"2024-05-28T14:01:54.279536+0000","last_clean":"2024-05-28T14:01:54.279536+0000","last_became_active":"2024-05-28T14:01:50.250233+0000","last_became_peered":"2024-05-28T14:01:50.250233+0000","last_unstale":"2024-05-28T14:01:54.279536+0000","last_undegraded":"2024-05-28T14:01:54.279536+0000","last_fullsized":"2024-05-28T14:01:54.279536+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:34:19.001030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:53.254070+0000","last_change":"2024-05-28T14:01:51.271813+0000","last_active":"2024-05-28T14:01:53.254070+0000","last_peered":"2024-05-28T14:01:53.254070+0000","last_clean":"2024-05-28T14:01:53.254070+0000","last_became_active":"2024-05-28T14:01:51.271494+0000","last_became_peered":"2024-05-28T14:01:51.271494+0000","last_unstale":"2024-05-28T14:01:53.254070+0000","last_undegraded":"2024-05-28T14:01:53.254070+0000","last_fullsized":"2024-05-28T14:01:53.254070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:44:58.614519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262300+0000","last_change":"2024-05-28T14:01:50.240591+0000","last_active":"2024-05-28T14:01:52.262300+0000","last_peered":"2024-05-28T14:01:52.262300+0000","last_clean":"2024-05-28T14:01:52.262300+0000","last_became_active":"2024-05-28T14:01:50.240282+0000","last_became_peered":"2024-05-28T14:01:50.240282+0000","last_unstale":"2024-05-28T14:01:52.262300+0000","last_undegraded":"2024-05-28T14:01:52.262300+0000","last_fullsized":"2024-05-28T14:01:52.262300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:22:40.189131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262656+0000","last_change":"2024-05-28T14:01:51.271622+0000","last_active":"2024-05-28T14:01:52.262656+0000","last_peered":"2024-05-28T14:01:52.262656+0000","last_clean":"2024-05-28T14:01:52.262656+0000","last_became_active":"2024-05-28T14:01:51.271332+0000","last_became_peered":"2024-05-28T14:01:51.271332+0000","last_unstale":"2024-05-28T14:01:52.262656+0000","last_undegraded":"2024-05-28T14:01:52.262656+0000","last_fullsized":"2024-05-28T14:01:52.262656+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:30:40.263915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.257465+0000","last_change":"2024-05-28T14:01:50.251927+0000","last_active":"2024-05-28T14:01:52.257465+0000","last_peered":"2024-05-28T14:01:52.257465+0000","last_clean":"2024-05-28T14:01:52.257465+0000","last_became_active":"2024-05-28T14:01:50.251129+0000","last_became_peered":"2024-05-28T14:01:50.251129+0000","last_unstale":"2024-05-28T14:01:52.257465+0000","last_undegraded":"2024-05-28T14:01:52.257465+0000","last_fullsized":"2024-05-28T14:01:52.257465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:19:10.168187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262090+0000","last_change":"2024-05-28T14:01:51.266774+0000","last_active":"2024-05-28T14:01:52.262090+0000","last_peered":"2024-05-28T14:01:52.262090+0000","last_clean":"2024-05-28T14:01:52.262090+0000","last_became_active":"2024-05-28T14:01:51.266199+0000","last_became_peered":"2024-05-28T14:01:51.266199+0000","last_unstale":"2024-05-28T14:01:52.262090+0000","last_undegraded":"2024-05-28T14:01:52.262090+0000","last_fullsized":"2024-05-28T14:01:52.262090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:34:30.307831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278373+0000","last_change":"2024-05-28T14:01:50.248895+0000","last_active":"2024-05-28T14:01:54.278373+0000","last_peered":"2024-05-28T14:01:54.278373+0000","last_clean":"2024-05-28T14:01:54.278373+0000","last_became_active":"2024-05-28T14:01:50.247794+0000","last_became_peered":"2024-05-28T14:01:50.247794+0000","last_unstale":"2024-05-28T14:01:54.278373+0000","last_undegraded":"2024-05-28T14:01:54.278373+0000","last_fullsized":"2024-05-28T14:01:54.278373+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:28:08.079804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.278870+0000","last_change":"2024-05-28T14:01:51.266696+0000","last_active":"2024-05-28T14:01:54.278870+0000","last_peered":"2024-05-28T14:01:54.278870+0000","last_clean":"2024-05-28T14:01:54.278870+0000","last_became_active":"2024-05-28T14:01:51.266422+0000","last_became_peered":"2024-05-28T14:01:51.266422+0000","last_unstale":"2024-05-28T14:01:54.278870+0000","last_undegraded":"2024-05-28T14:01:54.278870+0000","last_fullsized":"2024-05-28T14:01:54.278870+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:50.220564+0000","last_clean_scrub_stamp":"2024-05-28T14:01:50.220564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:22:15.546134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:52.262926+0000","last_change":"2024-05-28T14:01:50.249960+0000","last_active":"2024-05-28T14:01:52.262926+0000","last_peered":"2024-05-28T14:01:52.262926+0000","last_clean":"2024-05-28T14:01:52.262926+0000","last_became_active":"2024-05-28T14:01:50.249563+0000","last_became_peered":"2024-05-28T14:01:50.249563+0000","last_unstale":"2024-05-28T14:01:52.262926+0000","last_undegraded":"2024-05-28T14:01:52.262926+0000","last_fullsized":"2024-05-28T14:01:52.262926+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:01:49.212839+0000","last_clean_scrub_stamp":"2024-05-28T14:01:49.212839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:29:45.717077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":216,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-28T14:01:54.279214+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:01:54.279214+0000","last_peered":"2024-05-28T14:01:54.279214+0000","last_clean":"2024-05-28T14:01:54.279214+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:01:54.279214+0000","last_undegraded":"2024-05-28T14:01:54.279214+0000","last_fullsized":"2024-05-28T14:01:54.279214+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36448,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335392,"statfs":{"total":96636764160,"available":96599441408,"internally_reserved":0,"allocated":1126400,"data_stored":805293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":548864,"data_stored":221835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35876,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335964,"statfs":{"total":96636764160,"available":96600027136,"internally_reserved":0,"allocated":544768,"data_stored":221233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35884,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335956,"statfs":{"total":96636764160,"available":96600018944,"internally_reserved":0,"allocated":552960,"data_stored":222775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40044,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331796,"statfs":{"total":96636764160,"available":96595759104,"internally_reserved":0,"allocated":552960,"data_stored":221801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35876,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335964,"statfs":{"total":96636764160,"available":96600027136,"internally_reserved":0,"allocated":544768,"data_stored":216469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40032,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331808,"statfs":{"total":96636764160,"available":96595771392,"internally_reserved":0,"allocated":544768,"data_stored":221233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36440,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335400,"statfs":{"total":96636764160,"available":96599449600,"internally_reserved":0,"allocated":1126400,"data_stored":805237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:01:55.769 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-28T14:01:55.769 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-28T14:01:55.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.972+0000 7f9fbfe00640 1 Processor -- start 2024-05-28T14:01:55.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.972+0000 7f9fbfe00640 1 -- start start 2024-05-28T14:01:55.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.976+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00cf540 0x7f9fc00cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.976+0000 7f9fbfe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9fc00707e0 0x7f9fc00cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.976+0000 7f9fbfe00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fc0160170 con 0x7f9fc0145d50 2024-05-28T14:01:55.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.976+0000 7f9fbfe00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9fc0160450 con 0x7f9fc00707e0 2024-05-28T14:01:55.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.976+0000 7f9fbfe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc0160730 con 0x7f9fc00cf540 2024-05-28T14:01:55.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.976+0000 7f9fbea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00cf540 0x7f9fc00cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.976+0000 7f9fbf400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9fc00707e0 0x7f9fc00cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.976+0000 7f9fbea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00cf540 0x7f9fc00cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:36886/0 (socket says 172.21.15.42:36886) 2024-05-28T14:01:55.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.976+0000 7f9fbea00640 1 -- 172.21.15.42:0/3527344613 learned_addr learned my addr 172.21.15.42:0/3527344613 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:55.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.976+0000 7f9fbea00640 1 -- 172.21.15.42:0/3527344613 >> v2:172.21.15.63:3300/0 conn(0x7f9fc00707e0 msgr2=0x7f9fc00cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:55.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.976+0000 7f9fbea00640 1 --2- 172.21.15.42:0/3527344613 >> v2:172.21.15.63:3300/0 conn(0x7f9fc00707e0 0x7f9fc00cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.976+0000 7f9fbea00640 1 -- 172.21.15.42:0/3527344613 >> v1:172.21.15.63:6789/0 conn(0x7f9fc0145d50 legacy=0x7f9fc0071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.976+0000 7f9fbea00640 1 -- 172.21.15.42:0/3527344613 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc00d0ed0 con 0x7f9fc00cf540 2024-05-28T14:01:55.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbea00640 1 --2- 172.21.15.42:0/3527344613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00cf540 0x7f9fc00cc940 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009e50 tx=0x7f9fb001c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3103bc3b94679146 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:55.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbe000640 1 -- 172.21.15.42:0/3527344613 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00ac020 con 0x7f9fc00cf540 2024-05-28T14:01:55.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbe000640 1 -- 172.21.15.42:0/3527344613 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb00a6480 con 0x7f9fc00cf540 2024-05-28T14:01:55.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbe000640 1 -- 172.21.15.42:0/3527344613 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00a6850 con 0x7f9fc00cf540 2024-05-28T14:01:55.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3527344613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00cf540 msgr2=0x7f9fc00cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 --2- 172.21.15.42:0/3527344613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00cf540 0x7f9fc00cc940 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009e50 tx=0x7f9fb001c9a0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3527344613 shutdown_connections 2024-05-28T14:01:55.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 --2- 172.21.15.42:0/3527344613 >> v2:172.21.15.63:3300/0 conn(0x7f9fc00707e0 0x7f9fc00cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 --2- 172.21.15.42:0/3527344613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00cf540 0x7f9fc00cc940 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3527344613 >> 172.21.15.42:0/3527344613 conn(0x7f9fc006fcc0 msgr2=0x7f9fc00700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:55.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3527344613 shutdown_connections 2024-05-28T14:01:55.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3527344613 wait complete. 2024-05-28T14:01:55.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 Processor -- start 2024-05-28T14:01:55.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 -- start start 2024-05-28T14:01:55.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00707e0 0x7f9fc00c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fc00cf540 0x7f9fc0158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc0145d50 0x7f9fc0158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0169350 con 0x7f9fc0145d50 2024-05-28T14:01:55.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc00d1100 con 0x7f9fc00cf540 2024-05-28T14:01:55.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.980+0000 7f9fbfe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc01694d0 con 0x7f9fc00707e0 2024-05-28T14:01:55.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc0145d50 0x7f9fc0158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc0145d50 0x7f9fc0158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55248/0 (socket says 172.21.15.42:55248) 2024-05-28T14:01:55.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fbf400640 1 -- 172.21.15.42:0/3289867259 learned_addr learned my addr 172.21.15.42:0/3289867259 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:55.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fbea00640 1 --2- 172.21.15.42:0/3289867259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00707e0 0x7f9fc00c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fb6000640 1 --2- 172.21.15.42:0/3289867259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fc00cf540 0x7f9fc0158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fbf400640 1 -- 172.21.15.42:0/3289867259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00707e0 msgr2=0x7f9fc00c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fbf400640 1 --2- 172.21.15.42:0/3289867259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00707e0 0x7f9fc00c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fbf400640 1 -- 172.21.15.42:0/3289867259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fc00cf540 msgr2=0x7f9fc0158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fbf400640 1 --2- 172.21.15.42:0/3289867259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fc00cf540 0x7f9fc0158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fbf400640 1 -- 172.21.15.42:0/3289867259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc00c69e0 con 0x7f9fc0145d50 2024-05-28T14:01:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fbf400640 1 --2- 172.21.15.42:0/3289867259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc0145d50 0x7f9fc0158540 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800e880 tx=0x7f9fb800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fb7400640 1 -- 172.21.15.42:0/3289867259 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8017070 con 0x7f9fc0145d50 2024-05-28T14:01:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fb7400640 1 -- 172.21.15.42:0/3289867259 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb800c940 con 0x7f9fc0145d50 2024-05-28T14:01:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fb7400640 1 -- 172.21.15.42:0/3289867259 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb800cd10 con 0x7f9fc0145d50 2024-05-28T14:01:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3289867259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc00cbe30 con 0x7f9fc0145d50 2024-05-28T14:01:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3289867259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc00cc3f0 con 0x7f9fc0145d50 2024-05-28T14:01:55.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3289867259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc004fcf0 con 0x7f9fc0145d50 2024-05-28T14:01:55.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fb7400640 1 -- 172.21.15.42:0/3289867259 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f9fb801a070 con 0x7f9fc0145d50 2024-05-28T14:01:55.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.984+0000 7f9fb7400640 1 --2- 172.21.15.42:0/3289867259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f9c068650 0x7f9f9c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:55.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.988+0000 7f9fb7400640 1 -- 172.21.15.42:0/3289867259 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7f9fb8086d30 con 0x7f9fc0145d50 2024-05-28T14:01:55.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.988+0000 7f9fbea00640 1 --2- 172.21.15.42:0/3289867259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f9c068650 0x7f9f9c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:55.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.988+0000 7f9fbea00640 1 --2- 172.21.15.42:0/3289867259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f9c068650 0x7f9f9c06ab10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00ad000 tx=0x7f9fb00a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:55.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:55.992+0000 7f9fb7400640 1 -- 172.21.15.42:0/3289867259 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9fb8056980 con 0x7f9fc0145d50 2024-05-28T14:01:56.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.216+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3289867259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9fc01558b0 con 0x7f9fc0145d50 2024-05-28T14:01:56.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.216+0000 7f9fb7400640 1 -- 172.21.15.42:0/3289867259 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f9fb805ac80 con 0x7f9fc0145d50 2024-05-28T14:01:56.203 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-05-28T14:01:56.203 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw== 2024-05-28T14:01:56.203 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow" 2024-05-28T14:01:56.203 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-05-28T14:01:56.203 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-28T14:01:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.0 2024-05-28T14:01:56.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.220+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3289867259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f9c068650 msgr2=0x7f9f9c06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:56.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.220+0000 7f9fbfe00640 1 --2- 172.21.15.42:0/3289867259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f9c068650 0x7f9f9c06ab10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00ad000 tx=0x7f9fb00a7000 comp rx=0 tx=0).stop 2024-05-28T14:01:56.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.220+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3289867259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc0145d50 msgr2=0x7f9fc0158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:56.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.220+0000 7f9fbfe00640 1 --2- 172.21.15.42:0/3289867259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc0145d50 0x7f9fc0158540 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800e880 tx=0x7f9fb800ed40 comp rx=0 tx=0).stop 2024-05-28T14:01:56.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.220+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3289867259 shutdown_connections 2024-05-28T14:01:56.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.220+0000 7f9fbfe00640 1 --2- 172.21.15.42:0/3289867259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f9c068650 0x7f9f9c06ab10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:56.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.220+0000 7f9fbfe00640 1 --2- 172.21.15.42:0/3289867259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc0145d50 0x7f9fc0158540 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:56.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.220+0000 7f9fbfe00640 1 --2- 172.21.15.42:0/3289867259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fc00cf540 0x7f9fc0158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:56.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.220+0000 7f9fbfe00640 1 --2- 172.21.15.42:0/3289867259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fc00707e0 0x7f9fc00c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:56.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.220+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3289867259 >> 172.21.15.42:0/3289867259 conn(0x7f9fc006fcc0 msgr2=0x7f9fc00b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:56.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.220+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3289867259 shutdown_connections 2024-05-28T14:01:56.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.220+0000 7f9fbfe00640 1 -- 172.21.15.42:0/3289867259 wait complete. 2024-05-28T14:01:56.224 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:01:56.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.380+0000 7fe565000640 1 Processor -- start 2024-05-28T14:01:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.380+0000 7fe565000640 1 -- start start 2024-05-28T14:01:56.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe565000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5600d0420 0x7fe56015ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:56.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe565000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe560067810 0x7fe5600b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:56.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe565000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe56006eaa0 con 0x7fe5600b4110 2024-05-28T14:01:56.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe565000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe56006ed80 con 0x7fe560067810 2024-05-28T14:01:56.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe565000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe56006f060 con 0x7fe5600d0420 2024-05-28T14:01:56.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe55e800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe5600b4110 0x7fe560162350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36820/0 (socket says 172.21.15.42:36820) 2024-05-28T14:01:56.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe55e800640 1 -- 172.21.15.42:0/3883619405 learned_addr learned my addr 172.21.15.42:0/3883619405 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe55de00640 1 --2- 172.21.15.42:0/3883619405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5600d0420 0x7fe56015ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe555400640 1 --2- 172.21.15.42:0/3883619405 >> v2:172.21.15.63:3300/0 conn(0x7fe560067810 0x7fe5600b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe55d400640 1 -- 172.21.15.42:0/3883619405 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1362835680 0 0) 0x7fe56006eaa0 con 0x7fe5600b4110 2024-05-28T14:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe55d400640 1 -- 172.21.15.42:0/3883619405 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe544003530 con 0x7fe5600b4110 2024-05-28T14:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe55de00640 1 -- 172.21.15.42:0/3883619405 >> v2:172.21.15.63:3300/0 conn(0x7fe560067810 msgr2=0x7fe5600b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe55de00640 1 --2- 172.21.15.42:0/3883619405 >> v2:172.21.15.63:3300/0 conn(0x7fe560067810 0x7fe5600b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe55de00640 1 -- 172.21.15.42:0/3883619405 >> v1:172.21.15.42:6789/0 conn(0x7fe5600b4110 legacy=0x7fe560162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.384+0000 7fe55de00640 1 -- 172.21.15.42:0/3883619405 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56006f340 con 0x7fe5600d0420 2024-05-28T14:01:56.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.388+0000 7fe55de00640 1 --2- 172.21.15.42:0/3883619405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5600d0420 0x7fe56015ed10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe5500097c0 tx=0x7fe55001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba9523c0406c6f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:56.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.388+0000 7fe55d400640 1 -- 172.21.15.42:0/3883619405 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5500b0070 con 0x7fe5600d0420 2024-05-28T14:01:56.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.388+0000 7fe55d400640 1 -- 172.21.15.42:0/3883619405 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5500ab600 con 0x7fe5600d0420 2024-05-28T14:01:56.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.388+0000 7fe55d400640 1 -- 172.21.15.42:0/3883619405 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5500ab9b0 con 0x7fe5600d0420 2024-05-28T14:01:56.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.388+0000 7fe565000640 1 -- 172.21.15.42:0/3883619405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5600d0420 msgr2=0x7fe56015ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:56.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.388+0000 7fe565000640 1 --2- 172.21.15.42:0/3883619405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5600d0420 0x7fe56015ed10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe5500097c0 tx=0x7fe55001c410 comp rx=0 tx=0).stop 2024-05-28T14:01:56.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.388+0000 7fe565000640 1 -- 172.21.15.42:0/3883619405 shutdown_connections 2024-05-28T14:01:56.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.388+0000 7fe565000640 1 --2- 172.21.15.42:0/3883619405 >> v2:172.21.15.63:3300/0 conn(0x7fe560067810 0x7fe5600b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:56.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.388+0000 7fe565000640 1 --2- 172.21.15.42:0/3883619405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5600d0420 0x7fe56015ed10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:56.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.388+0000 7fe565000640 1 -- 172.21.15.42:0/3883619405 >> 172.21.15.42:0/3883619405 conn(0x7fe5600d0b40 msgr2=0x7fe5600d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:56.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.388+0000 7fe565000640 1 -- 172.21.15.42:0/3883619405 shutdown_connections 2024-05-28T14:01:56.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.388+0000 7fe565000640 1 -- 172.21.15.42:0/3883619405 wait complete. 2024-05-28T14:01:56.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.388+0000 7fe565000640 1 Processor -- start 2024-05-28T14:01:56.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.388+0000 7fe565000640 1 -- start start 2024-05-28T14:01:56.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe565000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe560067810 0x7fe56014f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:56.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe565000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5600b4110 0x7fe560152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:56.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe565000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5600d0420 0x7fe560152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:56.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe565000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe56016b640 con 0x7fe560067810 2024-05-28T14:01:56.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe565000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe56016b4c0 con 0x7fe5600b4110 2024-05-28T14:01:56.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe565000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe56016b7c0 con 0x7fe5600d0420 2024-05-28T14:01:56.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe55de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe560067810 0x7fe56014f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:56.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe55de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe560067810 0x7fe56014f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55252/0 (socket says 172.21.15.42:55252) 2024-05-28T14:01:56.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe55de00640 1 -- 172.21.15.42:0/3239846275 learned_addr learned my addr 172.21.15.42:0/3239846275 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:56.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe555400640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5600b4110 0x7fe560152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:56.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe55e800640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5600d0420 0x7fe560152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:56.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe55de00640 1 -- 172.21.15.42:0/3239846275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5600d0420 msgr2=0x7fe560152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:56.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe55de00640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5600d0420 0x7fe560152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:56.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe55de00640 1 -- 172.21.15.42:0/3239846275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5600b4110 msgr2=0x7fe560152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:56.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe55de00640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5600b4110 0x7fe560152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:56.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe55de00640 1 -- 172.21.15.42:0/3239846275 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56014ff00 con 0x7fe560067810 2024-05-28T14:01:56.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe555400640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5600b4110 0x7fe560152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:56.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe55de00640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe560067810 0x7fe56014f520 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe5500ae040 tx=0x7fe5500191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:56.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe556a00640 1 -- 172.21.15.42:0/3239846275 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5500bb020 con 0x7fe560067810 2024-05-28T14:01:56.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe556a00640 1 -- 172.21.15.42:0/3239846275 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5500ab9b0 con 0x7fe560067810 2024-05-28T14:01:56.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe556a00640 1 -- 172.21.15.42:0/3239846275 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5500b0040 con 0x7fe560067810 2024-05-28T14:01:56.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe565000640 1 -- 172.21.15.42:0/3239846275 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5601501e0 con 0x7fe560067810 2024-05-28T14:01:56.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe565000640 1 -- 172.21.15.42:0/3239846275 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5600af070 con 0x7fe560067810 2024-05-28T14:01:56.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.392+0000 7fe565000640 1 -- 172.21.15.42:0/3239846275 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe560068a90 con 0x7fe560067810 2024-05-28T14:01:56.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.396+0000 7fe556a00640 1 -- 172.21.15.42:0/3239846275 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7fe5500be020 con 0x7fe560067810 2024-05-28T14:01:56.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.396+0000 7fe556a00640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe53c0686a0 0x7fe53c06ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:56.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.396+0000 7fe556a00640 1 -- 172.21.15.42:0/3239846275 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7fe550121300 con 0x7fe560067810 2024-05-28T14:01:56.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.396+0000 7fe555400640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe53c0686a0 0x7fe53c06ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:56.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.396+0000 7fe555400640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe53c0686a0 0x7fe53c06ab60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe56015e260 tx=0x7fe548008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:56.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.400+0000 7fe556a00640 1 -- 172.21.15.42:0/3239846275 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe5500c5030 con 0x7fe560067810 2024-05-28T14:01:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.596+0000 7fe565000640 1 -- 172.21.15.42:0/3239846275 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe560152350 con 0x7fe560067810 2024-05-28T14:01:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.600+0000 7fe556a00640 1 -- 172.21.15.42:0/3239846275 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5180 (secure 0 0 0) 0x7fe550016090 con 0x7fe560067810 2024-05-28T14:01:56.585 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:01:56.585 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3694688875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3694688875},{"type":"v1","addr":"172.21.15.63:6835","nonce":3694688875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5799,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/199639725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":199639725},{"type":"v1","addr":"172.21.15.42:6836","nonce":199639725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1043559054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1043559054},{"type":"v1","addr":"172.21.15.42:6837","nonce":1043559054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:01:50.630732+0000","modified":"2024-05-28T14:01:53.667008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":55184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55184":{"gid":55184,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6833/754542999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":754542999},{"type":"v1","addr":"172.21.15.63:6833","nonce":754542999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":55184,"qdb_cluster":[55184]},"id":3}]} 2024-05-28T14:01:56.585 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-05-28T14:01:56.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.600+0000 7fe565000640 1 -- 172.21.15.42:0/3239846275 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe53c0686a0 msgr2=0x7fe53c06ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:56.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.600+0000 7fe565000640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe53c0686a0 0x7fe53c06ab60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe56015e260 tx=0x7fe548008040 comp rx=0 tx=0).stop 2024-05-28T14:01:56.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.600+0000 7fe565000640 1 -- 172.21.15.42:0/3239846275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe560067810 msgr2=0x7fe56014f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:56.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.600+0000 7fe565000640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe560067810 0x7fe56014f520 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe5500ae040 tx=0x7fe5500191b0 comp rx=0 tx=0).stop 2024-05-28T14:01:56.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.600+0000 7fe565000640 1 -- 172.21.15.42:0/3239846275 shutdown_connections 2024-05-28T14:01:56.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.600+0000 7fe565000640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe53c0686a0 0x7fe53c06ab60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:56.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.600+0000 7fe565000640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5600d0420 0x7fe560152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:56.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.600+0000 7fe565000640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5600b4110 0x7fe560152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:56.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.600+0000 7fe565000640 1 --2- 172.21.15.42:0/3239846275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe560067810 0x7fe56014f520 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:56.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.600+0000 7fe565000640 1 -- 172.21.15.42:0/3239846275 >> 172.21.15.42:0/3239846275 conn(0x7fe5600d0b40 msgr2=0x7fe56015d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:56.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.600+0000 7fe565000640 1 -- 172.21.15.42:0/3239846275 shutdown_connections 2024-05-28T14:01:56.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:56.600+0000 7fe565000640 1 -- 172.21.15.42:0/3239846275 wait complete. 2024-05-28T14:01:57.011 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-28T14:01:50.630732+0000', 'modified': '2024-05-28T14:01:53.667008+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 55184}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_55184': {'gid': 55184, 'name': 'b', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6833/754542999', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6832', 'nonce': 754542999}, {'type': 'v1', 'addr': '172.21.15.63:6833', 'nonce': 754542999}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 55184, 'qdb_cluster': [55184]} 2024-05-28T14:01:57.013 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-28T14:01:57.013 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-28T14:01:57.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd80ba00640 1 Processor -- start 2024-05-28T14:01:57.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd80ba00640 1 -- start start 2024-05-28T14:01:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd80ba00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd8040cf230 0x7fd804072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd80ba00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd804072ba0 0x7fd80406ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd80ba00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8040cfc10 con 0x7fd80415bf30 2024-05-28T14:01:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd80ba00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd8040721d0 con 0x7fd804072ba0 2024-05-28T14:01:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd80ba00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd8040cfef0 con 0x7fd8040cf230 2024-05-28T14:01:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd809a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd804072ba0 0x7fd80406ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd809000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd80415bf30 0x7fd80415c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36836/0 (socket says 172.21.15.42:36836) 2024-05-28T14:01:57.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd809000640 1 -- 172.21.15.42:0/3127908224 learned_addr learned my addr 172.21.15.42:0/3127908224 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:57.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd7fbe00640 1 --2- 172.21.15.42:0/3127908224 >> v2:172.21.15.63:3300/0 conn(0x7fd8040cf230 0x7fd804072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd809a00640 1 -- 172.21.15.42:0/3127908224 >> v2:172.21.15.63:3300/0 conn(0x7fd8040cf230 msgr2=0x7fd804072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:57.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd809a00640 1 --2- 172.21.15.42:0/3127908224 >> v2:172.21.15.63:3300/0 conn(0x7fd8040cf230 0x7fd804072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd809a00640 1 -- 172.21.15.42:0/3127908224 >> v1:172.21.15.42:6789/0 conn(0x7fd80415bf30 legacy=0x7fd80415c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd809a00640 1 -- 172.21.15.42:0/3127908224 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8040d01d0 con 0x7fd804072ba0 2024-05-28T14:01:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.176+0000 7fd803e00640 1 -- 172.21.15.42:0/3127908224 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2232039850 0 0) 0x7fd8040cfc10 con 0x7fd80415bf30 2024-05-28T14:01:57.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.180+0000 7fd809a00640 1 --2- 172.21.15.42:0/3127908224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd804072ba0 0x7fd80406ec50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd8040b0130 tx=0x7fd7fc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d553e678193371ff server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:57.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.180+0000 7fd803e00640 1 -- 172.21.15.42:0/3127908224 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0b2070 con 0x7fd804072ba0 2024-05-28T14:01:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.180+0000 7fd803e00640 1 -- 172.21.15.42:0/3127908224 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7fc0a57f0 con 0x7fd804072ba0 2024-05-28T14:01:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.180+0000 7fd803e00640 1 -- 172.21.15.42:0/3127908224 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0a5bc0 con 0x7fd804072ba0 2024-05-28T14:01:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.180+0000 7fd80ba00640 1 -- 172.21.15.42:0/3127908224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd804072ba0 msgr2=0x7fd80406ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.180+0000 7fd80ba00640 1 --2- 172.21.15.42:0/3127908224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd804072ba0 0x7fd80406ec50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd8040b0130 tx=0x7fd7fc01c930 comp rx=0 tx=0).stop 2024-05-28T14:01:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.180+0000 7fd80ba00640 1 -- 172.21.15.42:0/3127908224 shutdown_connections 2024-05-28T14:01:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.180+0000 7fd80ba00640 1 --2- 172.21.15.42:0/3127908224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd804072ba0 0x7fd80406ec50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.180+0000 7fd80ba00640 1 --2- 172.21.15.42:0/3127908224 >> v2:172.21.15.63:3300/0 conn(0x7fd8040cf230 0x7fd804072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.180+0000 7fd80ba00640 1 -- 172.21.15.42:0/3127908224 >> 172.21.15.42:0/3127908224 conn(0x7fd80406b9f0 msgr2=0x7fd80406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.180+0000 7fd80ba00640 1 -- 172.21.15.42:0/3127908224 shutdown_connections 2024-05-28T14:01:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.180+0000 7fd80ba00640 1 -- 172.21.15.42:0/3127908224 wait complete. 2024-05-28T14:01:57.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd80ba00640 1 Processor -- start 2024-05-28T14:01:57.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd80ba00640 1 -- start start 2024-05-28T14:01:57.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd80ba00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd804072ba0 0x7fd8040b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd80ba00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8040cf230 0x7fd8040aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd80ba00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd80415bf30 0x7fd8040b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd80ba00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd804168330 con 0x7fd8040cf230 2024-05-28T14:01:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd80ba00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd8041681b0 con 0x7fd80415bf30 2024-05-28T14:01:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd80ba00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd8041684b0 con 0x7fd804072ba0 2024-05-28T14:01:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8040cf230 0x7fd8040aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd809000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd804072ba0 0x7fd8040b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8040cf230 0x7fd8040aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55270/0 (socket says 172.21.15.42:55270) 2024-05-28T14:01:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd7fbe00640 1 -- 172.21.15.42:0/1891511163 learned_addr learned my addr 172.21.15.42:0/1891511163 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd809a00640 1 --2- 172.21.15.42:0/1891511163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd80415bf30 0x7fd8040b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd7fbe00640 1 -- 172.21.15.42:0/1891511163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd804072ba0 msgr2=0x7fd8040b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd7fbe00640 1 --2- 172.21.15.42:0/1891511163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd804072ba0 0x7fd8040b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd7fbe00640 1 -- 172.21.15.42:0/1891511163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd80415bf30 msgr2=0x7fd8040b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd7fbe00640 1 --2- 172.21.15.42:0/1891511163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd80415bf30 0x7fd8040b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd7fbe00640 1 -- 172.21.15.42:0/1891511163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8040b0960 con 0x7fd8040cf230 2024-05-28T14:01:57.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd7fbe00640 1 --2- 172.21.15.42:0/1891511163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8040cf230 0x7fd8040aede0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00d800 tx=0x7fd7ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:57.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd801600640 1 -- 172.21.15.42:0/1891511163 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00b6c0 con 0x7fd8040cf230 2024-05-28T14:01:57.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd801600640 1 -- 172.21.15.42:0/1891511163 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7ec019070 con 0x7fd8040cf230 2024-05-28T14:01:57.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd801600640 1 -- 172.21.15.42:0/1891511163 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00ba60 con 0x7fd8040cf230 2024-05-28T14:01:57.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd80ba00640 1 -- 172.21.15.42:0/1891511163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8040b0c40 con 0x7fd8040cf230 2024-05-28T14:01:57.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.184+0000 7fd80ba00640 1 -- 172.21.15.42:0/1891511163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8040b1200 con 0x7fd8040cf230 2024-05-28T14:01:57.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd801600640 1 -- 172.21.15.42:0/1891511163 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7fd7ec00bc00 con 0x7fd8040cf230 2024-05-28T14:01:57.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd801600640 1 --2- 172.21.15.42:0/1891511163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7e0068650 0x7fd7e006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd801600640 1 -- 172.21.15.42:0/1891511163 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7fd7ec087160 con 0x7fd8040cf230 2024-05-28T14:01:57.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd809000640 1 --2- 172.21.15.42:0/1891511163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7e0068650 0x7fd7e006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd80ef99000 1 Processor -- start 2024-05-28T14:01:57.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd80ef99000 1 -- start start 2024-05-28T14:01:57.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd809000640 1 --2- 172.21.15.42:0/1891511163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7e0068650 0x7fd7e006ab10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4000f70 tx=0x7fd7f4007df0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:57.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd80ef99000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096059500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd80ef99000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059a40 0x55b09600c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd80ef99000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b09600c650 0x55b09600ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd80ef99000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b095e5d420 con 0x55b096059120 2024-05-28T14:01:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd80ef99000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b095df7bf0 con 0x55b09600c650 2024-05-28T14:01:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd80ef99000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b095e593e0 con 0x55b096059a40 2024-05-28T14:01:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd809000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096059500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd809000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096059500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55280/0 (socket says 172.21.15.42:55280) 2024-05-28T14:01:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.188+0000 7fd809000640 1 -- 172.21.15.42:0/1191722277 learned_addr learned my addr 172.21.15.42:0/1191722277 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd809a00640 1 --2- 172.21.15.42:0/1191722277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b09600c650 0x55b09600ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd7fbe00640 1 --2- 172.21.15.42:0/1191722277 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059a40 0x55b09600c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd809000640 1 -- 172.21.15.42:0/1191722277 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059a40 msgr2=0x55b09600c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd809000640 1 --2- 172.21.15.42:0/1191722277 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059a40 0x55b09600c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd809000640 1 -- 172.21.15.42:0/1191722277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b09600c650 msgr2=0x55b09600ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd809000640 1 --2- 172.21.15.42:0/1191722277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b09600c650 0x55b09600ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd809000640 1 -- 172.21.15.42:0/1191722277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b095f05450 con 0x55b096059120 2024-05-28T14:01:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd809a00640 1 --2- 172.21.15.42:0/1191722277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b09600c650 0x55b09600ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd809000640 1 --2- 172.21.15.42:0/1191722277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096059500 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40077f0 tx=0x7fd7f4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295f38e82fa5d8cc server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd7f9600640 1 -- 172.21.15.42:0/1191722277 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4026070 con 0x55b096059120 2024-05-28T14:01:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd7f9600640 1 -- 172.21.15.42:0/1191722277 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f4001d30 con 0x55b096059120 2024-05-28T14:01:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd7f9600640 1 -- 172.21.15.42:0/1191722277 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4013d70 con 0x55b096059120 2024-05-28T14:01:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 -- 172.21.15.42:0/1191722277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 msgr2=0x55b096059500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 --2- 172.21.15.42:0/1191722277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096059500 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40077f0 tx=0x7fd7f4012000 comp rx=0 tx=0).stop 2024-05-28T14:01:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 -- 172.21.15.42:0/1191722277 shutdown_connections 2024-05-28T14:01:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 --2- 172.21.15.42:0/1191722277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b09600c650 0x55b09600ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 --2- 172.21.15.42:0/1191722277 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059a40 0x55b09600c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 --2- 172.21.15.42:0/1191722277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096059500 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 -- 172.21.15.42:0/1191722277 >> 172.21.15.42:0/1191722277 conn(0x55b096039ec0 msgr2=0x55b09603c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 -- 172.21.15.42:0/1191722277 shutdown_connections 2024-05-28T14:01:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 -- 172.21.15.42:0/1191722277 wait complete. 2024-05-28T14:01:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 -- start start 2024-05-28T14:01:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 Processor -- start 2024-05-28T14:01:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b09600c650 0x55b096011b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096012080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059a40 0x55b0960125c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b095dfdec0 con 0x55b096059120 2024-05-28T14:01:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b095ede120 con 0x55b09600c650 2024-05-28T14:01:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b095e68350 con 0x55b096059a40 2024-05-28T14:01:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd80ef99000 10 client.0 _ll_register_callbacks cb 0x55b095e3a4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T14:01:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096012080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096012080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55296/0 (socket says 172.21.15.42:55296) 2024-05-28T14:01:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd7fbe00640 1 -- 172.21.15.42:0/3319906011 learned_addr learned my addr 172.21.15.42:0/3319906011 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd809000640 1 --2- 172.21.15.42:0/3319906011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b09600c650 0x55b096011b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.192+0000 7fd809a00640 1 --2- 172.21.15.42:0/3319906011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059a40 0x55b0960125c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7fbe00640 1 -- 172.21.15.42:0/3319906011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059a40 msgr2=0x55b0960125c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7fbe00640 1 --2- 172.21.15.42:0/3319906011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059a40 0x55b0960125c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7fbe00640 1 -- 172.21.15.42:0/3319906011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b09600c650 msgr2=0x55b096011b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7fbe00640 1 --2- 172.21.15.42:0/3319906011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b09600c650 0x55b096011b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7fbe00640 1 -- 172.21.15.42:0/3319906011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b095ea2f80 con 0x55b096059120 2024-05-28T14:01:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd809a00640 1 --2- 172.21.15.42:0/3319906011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059a40 0x55b0960125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7fbe00640 1 --2- 172.21.15.42:0/3319906011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096012080 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00b4b0 tx=0x7fd7ec05b170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56f429a61421e0d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7f2000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T14:01:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7f2000640 1 -- 172.21.15.42:0/3319906011 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec056400 con 0x55b096059120 2024-05-28T14:01:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd80ef99000 1 -- 172.21.15.42:0/3319906011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b095df6bc0 con 0x55b096059120 2024-05-28T14:01:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7f2000640 1 -- 172.21.15.42:0/3319906011 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7fd7ec085470 con 0x55b096059120 2024-05-28T14:01:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7f2000640 1 -- 172.21.15.42:0/3319906011 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd7ec075050 con 0x55b096059120 2024-05-28T14:01:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd80ef99000 10 client.5952 fetch_fsmap learned FSMap version 27 2024-05-28T14:01:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd80ef99000 1 -- 172.21.15.42:0/3319906011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55b095e36f00 con 0x55b096059120 2024-05-28T14:01:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7f2000640 1 -- 172.21.15.42:0/3319906011 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b095df6bc0 con 0x55b096059120 2024-05-28T14:01:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7f2000640 1 -- 172.21.15.42:0/3319906011 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7fd7ec013070 con 0x55b096059120 2024-05-28T14:01:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd80ef99000 10 client.5952 fetch_fsmap finished waiting for FSMap version 27 2024-05-28T14:01:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd80ef99000 10 client.5952 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-28T14:01:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd80ef99000 20 client.5952 populate_metadata read hostname 'smithi042' 2024-05-28T14:01:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd80ef99000 1 --2- 172.21.15.42:0/3319906011 >> [v2:172.21.15.63:6832/754542999,v1:172.21.15.63:6833/754542999] conn(0x55b0960683c0 0x55b09606a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd80ef99000 4 client.5952 mds_command: new command op to 55184 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-28T14:01:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd80ef99000 1 -- 172.21.15.42:0/3319906011 --> [v2:172.21.15.63:6832/754542999,v1:172.21.15.63:6833/754542999] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b095e6ae40 con 0x55b0960683c0 2024-05-28T14:01:57.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd809000640 1 --2- 172.21.15.42:0/3319906011 >> [v2:172.21.15.63:6832/754542999,v1:172.21.15.63:6833/754542999] conn(0x55b0960683c0 0x55b09606a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd809000640 1 --2- 172.21.15.42:0/3319906011 >> [v2:172.21.15.63:6832/754542999,v1:172.21.15.63:6833/754542999] conn(0x55b0960683c0 0x55b09606a7b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c35feba20f515bb9 server_cookie=65fba20fbeac77d5 in_seq=0 out_seq=0 2024-05-28T14:01:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7f2000640 10 client.5952 ms_handle_connect on v2:172.21.15.63:6832/754542999 2024-05-28T14:01:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7f2000640 1 -- 172.21.15.42:0/3319906011 <== mds.0 v2:172.21.15.63:6832/754542999 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b095e6ae40 con 0x55b0960683c0 2024-05-28T14:01:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd7f2000640 10 client.5952 handle_command_reply: tid=0 2024-05-28T14:01:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.196+0000 7fd80ef99000 1 client.5952 shutdown 2024-05-28T14:01:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.200+0000 7fd80ef99000 1 -- 172.21.15.42:0/3319906011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 msgr2=0x55b096012080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.200+0000 7fd80ef99000 1 --2- 172.21.15.42:0/3319906011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096012080 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00b4b0 tx=0x7fd7ec05b170 comp rx=0 tx=0).stop 2024-05-28T14:01:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.200+0000 7fd80ef99000 1 -- 172.21.15.42:0/3319906011 shutdown_connections 2024-05-28T14:01:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.200+0000 7fd80ef99000 1 --2- 172.21.15.42:0/3319906011 >> [v2:172.21.15.63:6832/754542999,v1:172.21.15.63:6833/754542999] conn(0x55b0960683c0 0x55b09606a7b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.200+0000 7fd80ef99000 1 --2- 172.21.15.42:0/3319906011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059a40 0x55b0960125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.200+0000 7fd80ef99000 1 --2- 172.21.15.42:0/3319906011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096012080 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.200+0000 7fd7f2000640 0 client.5952 ms_handle_reset on v2:172.21.15.63:6832/754542999 2024-05-28T14:01:57.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.200+0000 7fd80ef99000 1 --2- 172.21.15.42:0/3319906011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b09600c650 0x55b096011b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.200+0000 7fd80ef99000 1 -- 172.21.15.42:0/3319906011 >> 172.21.15.42:0/3319906011 conn(0x55b096039ec0 msgr2=0x55b09603c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:57.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.200+0000 7fd80ef99000 1 -- 172.21.15.42:0/3319906011 shutdown_connections 2024-05-28T14:01:57.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.200+0000 7fd80ef99000 1 -- 172.21.15.42:0/3319906011 wait complete. 2024-05-28T14:01:57.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.200+0000 7fd80ef99000 20 client.5952 trim_cache size 0 max 16384 2024-05-28T14:01:57.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd80ef99000 1 Processor -- start 2024-05-28T14:01:57.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd80ef99000 1 -- start start 2024-05-28T14:01:57.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd80ef99000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b09600c650 0x55b09600b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd80ef99000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059120 0x55b09600bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd80ef99000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b096059a40 0x55b09606a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd80ef99000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd80414fb80 con 0x55b09600c650 2024-05-28T14:01:57.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd80ef99000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd80414eee0 con 0x55b096059a40 2024-05-28T14:01:57.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd80ef99000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd80414f530 con 0x55b096059120 2024-05-28T14:01:57.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd809000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b09600c650 0x55b09600b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd809000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b09600c650 0x55b09600b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55310/0 (socket says 172.21.15.42:55310) 2024-05-28T14:01:57.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd809000640 1 -- 172.21.15.42:0/3838034288 learned_addr learned my addr 172.21.15.42:0/3838034288 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:57.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd7fbe00640 1 --2- 172.21.15.42:0/3838034288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059120 0x55b09600bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd809a00640 1 --2- 172.21.15.42:0/3838034288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b096059a40 0x55b09606a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd809000640 1 -- 172.21.15.42:0/3838034288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059120 msgr2=0x55b09600bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:57.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd809000640 1 --2- 172.21.15.42:0/3838034288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059120 0x55b09600bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd809000640 1 -- 172.21.15.42:0/3838034288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b096059a40 msgr2=0x55b09606a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:57.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd809000640 1 --2- 172.21.15.42:0/3838034288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b096059a40 0x55b09606a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd809000640 1 -- 172.21.15.42:0/3838034288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b095de3e80 con 0x55b09600c650 2024-05-28T14:01:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd7fbe00640 1 --2- 172.21.15.42:0/3838034288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059120 0x55b09600bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.216+0000 7fd809a00640 1 --2- 172.21.15.42:0/3838034288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b096059a40 0x55b09606a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:01:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd809000640 1 --2- 172.21.15.42:0/3838034288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b09600c650 0x55b09600b810 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4004f90 tx=0x7fd7f4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64dc26e1cfbd9b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd7f8c00640 1 -- 172.21.15.42:0/3838034288 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4026070 con 0x55b09600c650 2024-05-28T14:01:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd7f8c00640 1 -- 172.21.15.42:0/3838034288 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f4032020 con 0x55b09600c650 2024-05-28T14:01:57.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd7f8c00640 1 -- 172.21.15.42:0/3838034288 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f40321d0 con 0x55b09600c650 2024-05-28T14:01:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 -- 172.21.15.42:0/3838034288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b09600c650 msgr2=0x55b09600b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 --2- 172.21.15.42:0/3838034288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b09600c650 0x55b09600b810 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4004f90 tx=0x7fd7f4013000 comp rx=0 tx=0).stop 2024-05-28T14:01:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 -- 172.21.15.42:0/3838034288 shutdown_connections 2024-05-28T14:01:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 --2- 172.21.15.42:0/3838034288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b096059a40 0x55b09606a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 --2- 172.21.15.42:0/3838034288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b096059120 0x55b09600bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 --2- 172.21.15.42:0/3838034288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b09600c650 0x55b09600b810 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 -- 172.21.15.42:0/3838034288 >> 172.21.15.42:0/3838034288 conn(0x55b096039ec0 msgr2=0x55b09600b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 -- 172.21.15.42:0/3838034288 shutdown_connections 2024-05-28T14:01:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 -- 172.21.15.42:0/3838034288 wait complete. 2024-05-28T14:01:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 -- start start 2024-05-28T14:01:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 Processor -- start 2024-05-28T14:01:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b09600c650 0x55b09606d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096084340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b096059a40 0x55b096084880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b095e5c730 con 0x55b096059120 2024-05-28T14:01:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b095e299e0 con 0x55b096059a40 2024-05-28T14:01:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b095e61a50 con 0x55b09600c650 2024-05-28T14:01:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd80ef99000 10 client.0 _ll_register_callbacks cb 0x7fd7ec0870b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T14:01:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096084340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096084340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55316/0 (socket says 172.21.15.42:55316) 2024-05-28T14:01:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd7fbe00640 1 -- 172.21.15.42:0/1544971241 learned_addr learned my addr 172.21.15.42:0/1544971241 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:57.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd809000640 1 --2- 172.21.15.42:0/1544971241 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b09600c650 0x55b09606d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd809a00640 1 --2- 172.21.15.42:0/1544971241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b096059a40 0x55b096084880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd7fbe00640 1 -- 172.21.15.42:0/1544971241 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b09600c650 msgr2=0x55b09606d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:57.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd7fbe00640 1 --2- 172.21.15.42:0/1544971241 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b09600c650 0x55b09606d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd7fbe00640 1 -- 172.21.15.42:0/1544971241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b096059a40 msgr2=0x55b096084880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:57.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd7fbe00640 1 --2- 172.21.15.42:0/1544971241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b096059a40 0x55b096084880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd7fbe00640 1 -- 172.21.15.42:0/1544971241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b095e25b30 con 0x55b096059120 2024-05-28T14:01:57.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd7fbe00640 1 --2- 172.21.15.42:0/1544971241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096084340 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0073a0 tx=0x7fd7ec056c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c325c8453e6a8b9b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:57.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.220+0000 7fd7f2000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T14:01:57.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd7f2000640 1 -- 172.21.15.42:0/1544971241 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec053050 con 0x55b096059120 2024-05-28T14:01:57.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd7f2000640 1 -- 172.21.15.42:0/1544971241 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7fd7ec085680 con 0x55b096059120 2024-05-28T14:01:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 -- 172.21.15.42:0/1544971241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7f4032020 con 0x55b096059120 2024-05-28T14:01:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd7f2000640 1 -- 172.21.15.42:0/1544971241 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd7ec05d2b0 con 0x55b096059120 2024-05-28T14:01:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 10 client.5958 fetch_fsmap learned FSMap version 27 2024-05-28T14:01:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 -- 172.21.15.42:0/1544971241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55b095ede5b0 con 0x55b096059120 2024-05-28T14:01:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd7f2000640 1 -- 172.21.15.42:0/1544971241 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f4032020 con 0x55b096059120 2024-05-28T14:01:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd7f2000640 1 -- 172.21.15.42:0/1544971241 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7fd7ec05d450 con 0x55b096059120 2024-05-28T14:01:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 10 client.5958 fetch_fsmap finished waiting for FSMap version 27 2024-05-28T14:01:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 10 client.5958 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-28T14:01:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 20 client.5958 populate_metadata read hostname 'smithi042' 2024-05-28T14:01:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 --2- 172.21.15.42:0/1544971241 >> [v2:172.21.15.63:6832/754542999,v1:172.21.15.63:6833/754542999] conn(0x55b096084dc0 0x55b0960c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 4 client.5958 mds_command: new command op to 55184 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-28T14:01:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 -- 172.21.15.42:0/1544971241 --> [v2:172.21.15.63:6832/754542999,v1:172.21.15.63:6833/754542999] -- command(tid 0: {"prefix": "status"}) -- 0x55b095e64cb0 con 0x55b096084dc0 2024-05-28T14:01:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd809000640 1 --2- 172.21.15.42:0/1544971241 >> [v2:172.21.15.63:6832/754542999,v1:172.21.15.63:6833/754542999] conn(0x55b096084dc0 0x55b0960c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd809000640 1 --2- 172.21.15.42:0/1544971241 >> [v2:172.21.15.63:6832/754542999,v1:172.21.15.63:6833/754542999] conn(0x55b096084dc0 0x55b0960c7fb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc73ada36b4e4e61 server_cookie=eb1d8a41944d3ebd in_seq=0 out_seq=0 2024-05-28T14:01:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd7f2000640 10 client.5958 ms_handle_connect on v2:172.21.15.63:6832/754542999 2024-05-28T14:01:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd7f2000640 1 -- 172.21.15.42:0/1544971241 <== mds.0 v2:172.21.15.63:6832/754542999 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b095e64cb0 con 0x55b096084dc0 2024-05-28T14:01:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd7f2000640 10 client.5958 handle_command_reply: tid=0 2024-05-28T14:01:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 client.5958 shutdown 2024-05-28T14:01:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 -- 172.21.15.42:0/1544971241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 msgr2=0x55b096084340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 --2- 172.21.15.42:0/1544971241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096084340 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0073a0 tx=0x7fd7ec056c00 comp rx=0 tx=0).stop 2024-05-28T14:01:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 -- 172.21.15.42:0/1544971241 shutdown_connections 2024-05-28T14:01:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 --2- 172.21.15.42:0/1544971241 >> [v2:172.21.15.63:6832/754542999,v1:172.21.15.63:6833/754542999] conn(0x55b096084dc0 0x55b0960c7fb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 --2- 172.21.15.42:0/1544971241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b096059a40 0x55b096084880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 --2- 172.21.15.42:0/1544971241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b096059120 0x55b096084340 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 --2- 172.21.15.42:0/1544971241 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b09600c650 0x55b09606d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd7f2000640 0 client.5958 ms_handle_reset on v2:172.21.15.63:6832/754542999 2024-05-28T14:01:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 -- 172.21.15.42:0/1544971241 >> 172.21.15.42:0/1544971241 conn(0x55b096039ec0 msgr2=0x55b09600b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 -- 172.21.15.42:0/1544971241 shutdown_connections 2024-05-28T14:01:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 1 -- 172.21.15.42:0/1544971241 wait complete. 2024-05-28T14:01:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ef99000 20 client.5958 trim_cache size 0 max 16384 2024-05-28T14:01:57.218 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:01:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:01:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-05-28T14:01:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 55184, 2024-05-28T14:01:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-28T14:01:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-28T14:01:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-28T14:01:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 5.9840537380000001, 2024-05-28T14:01:57.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 27, 2024-05-28T14:01:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 46, 2024-05-28T14:01:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 46, 2024-05-28T14:01:57.219 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 10.794656487999999 2024-05-28T14:01:57.219 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:01:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ba00640 1 -- 172.21.15.42:0/1891511163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7e0068650 msgr2=0x7fd7e006ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ba00640 1 --2- 172.21.15.42:0/1891511163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7e0068650 0x7fd7e006ab10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4000f70 tx=0x7fd7f4007df0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ba00640 1 -- 172.21.15.42:0/1891511163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8040cf230 msgr2=0x7fd8040aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ba00640 1 --2- 172.21.15.42:0/1891511163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8040cf230 0x7fd8040aede0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00d800 tx=0x7fd7ec00dcc0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ba00640 1 -- 172.21.15.42:0/1891511163 shutdown_connections 2024-05-28T14:01:57.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ba00640 1 --2- 172.21.15.42:0/1891511163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7e0068650 0x7fd7e006ab10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ba00640 1 --2- 172.21.15.42:0/1891511163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd80415bf30 0x7fd8040b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ba00640 1 --2- 172.21.15.42:0/1891511163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8040cf230 0x7fd8040aede0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ba00640 1 --2- 172.21.15.42:0/1891511163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd804072ba0 0x7fd8040b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ba00640 1 -- 172.21.15.42:0/1891511163 >> 172.21.15.42:0/1891511163 conn(0x7fd80406b9f0 msgr2=0x7fd80406bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:57.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ba00640 1 -- 172.21.15.42:0/1891511163 shutdown_connections 2024-05-28T14:01:57.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.224+0000 7fd80ba00640 1 -- 172.21.15.42:0/1891511163 wait complete. 2024-05-28T14:01:57.230 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-28T14:01:57.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa7a800640 1 Processor -- start 2024-05-28T14:01:57.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa7a800640 1 -- start start 2024-05-28T14:01:57.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa7a800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffa7415c990 0x7ffa740d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa7a800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffa740d06a0 0x7ffa740b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa7a800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa740cd300 con 0x7ffa7409bbd0 2024-05-28T14:01:57.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa7a800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ffa7415cd70 con 0x7ffa740d06a0 2024-05-28T14:01:57.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa7a800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffa740cd5e0 con 0x7ffa7415c990 2024-05-28T14:01:57.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa73e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffa740d06a0 0x7ffa740b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa73e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffa740d06a0 0x7ffa740b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47832/0 (socket says 172.21.15.42:47832) 2024-05-28T14:01:57.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa73e00640 1 -- 172.21.15.42:0/2086286959 learned_addr learned my addr 172.21.15.42:0/2086286959 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:57.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa73400640 1 --2- 172.21.15.42:0/2086286959 >> v2:172.21.15.63:3300/0 conn(0x7ffa7415c990 0x7ffa740d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa73e00640 1 -- 172.21.15.42:0/2086286959 >> v2:172.21.15.63:3300/0 conn(0x7ffa7415c990 msgr2=0x7ffa740d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:57.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa73e00640 1 --2- 172.21.15.42:0/2086286959 >> v2:172.21.15.63:3300/0 conn(0x7ffa7415c990 0x7ffa740d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa73e00640 1 -- 172.21.15.42:0/2086286959 >> v1:172.21.15.63:6789/0 conn(0x7ffa7409bbd0 legacy=0x7ffa7409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.388+0000 7ffa73e00640 1 -- 172.21.15.42:0/2086286959 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa740cd8c0 con 0x7ffa740d06a0 2024-05-28T14:01:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.392+0000 7ffa73e00640 1 --2- 172.21.15.42:0/2086286959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa740d06a0 0x7ffa740b53e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ffa74073350 tx=0x7ffa6c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8325ef8ed28f7cc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.392+0000 7ffa72a00640 1 -- 172.21.15.42:0/2086286959 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0a5880 con 0x7ffa740d06a0 2024-05-28T14:01:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.392+0000 7ffa72a00640 1 -- 172.21.15.42:0/2086286959 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c0a5a20 con 0x7ffa740d06a0 2024-05-28T14:01:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.392+0000 7ffa72a00640 1 -- 172.21.15.42:0/2086286959 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0a5df0 con 0x7ffa740d06a0 2024-05-28T14:01:57.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.392+0000 7ffa7a800640 1 -- 172.21.15.42:0/2086286959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa740d06a0 msgr2=0x7ffa740b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.392+0000 7ffa7a800640 1 --2- 172.21.15.42:0/2086286959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa740d06a0 0x7ffa740b53e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ffa74073350 tx=0x7ffa6c01c9c0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.392+0000 7ffa7a800640 1 -- 172.21.15.42:0/2086286959 shutdown_connections 2024-05-28T14:01:57.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.392+0000 7ffa7a800640 1 --2- 172.21.15.42:0/2086286959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa740d06a0 0x7ffa740b53e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.392+0000 7ffa7a800640 1 --2- 172.21.15.42:0/2086286959 >> v2:172.21.15.63:3300/0 conn(0x7ffa7415c990 0x7ffa740d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.392+0000 7ffa7a800640 1 -- 172.21.15.42:0/2086286959 >> 172.21.15.42:0/2086286959 conn(0x7ffa740b4080 msgr2=0x7ffa740b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:57.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.392+0000 7ffa7a800640 1 -- 172.21.15.42:0/2086286959 shutdown_connections 2024-05-28T14:01:57.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.392+0000 7ffa7a800640 1 -- 172.21.15.42:0/2086286959 wait complete. 2024-05-28T14:01:57.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa7a800640 1 Processor -- start 2024-05-28T14:01:57.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa7a800640 1 -- start start 2024-05-28T14:01:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa7a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa740d06a0 0x7ffa74150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa7a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa7415c990 0x7ffa740c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa7a800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa74157780 0x7ffa74150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa7a800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7409b2e0 con 0x7ffa7415c990 2024-05-28T14:01:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa7a800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7409b160 con 0x7ffa740d06a0 2024-05-28T14:01:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa7a800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa740b6130 con 0x7ffa74157780 2024-05-28T14:01:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa6be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa7415c990 0x7ffa740c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa6be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa7415c990 0x7ffa740c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47836/0 (socket says 172.21.15.42:47836) 2024-05-28T14:01:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa6be00640 1 -- 172.21.15.42:0/720009250 learned_addr learned my addr 172.21.15.42:0/720009250 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa73400640 1 --2- 172.21.15.42:0/720009250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa740d06a0 0x7ffa74150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa73e00640 1 --2- 172.21.15.42:0/720009250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa74157780 0x7ffa74150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa6be00640 1 -- 172.21.15.42:0/720009250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa74157780 msgr2=0x7ffa74150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa6be00640 1 --2- 172.21.15.42:0/720009250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa74157780 0x7ffa74150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa6be00640 1 -- 172.21.15.42:0/720009250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa740d06a0 msgr2=0x7ffa74150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa6be00640 1 --2- 172.21.15.42:0/720009250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa740d06a0 0x7ffa74150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa6be00640 1 -- 172.21.15.42:0/720009250 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa74152440 con 0x7ffa7415c990 2024-05-28T14:01:57.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa6be00640 1 --2- 172.21.15.42:0/720009250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa7415c990 0x7ffa740c8b40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00c420 tx=0x7ffa5c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:57.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa6b400640 1 -- 172.21.15.42:0/720009250 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0052d0 con 0x7ffa7415c990 2024-05-28T14:01:57.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa6b400640 1 -- 172.21.15.42:0/720009250 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5c005470 con 0x7ffa7415c990 2024-05-28T14:01:57.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa6b400640 1 -- 172.21.15.42:0/720009250 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c007260 con 0x7ffa7415c990 2024-05-28T14:01:57.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa7a800640 1 -- 172.21.15.42:0/720009250 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa741512b0 con 0x7ffa7415c990 2024-05-28T14:01:57.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa7a800640 1 -- 172.21.15.42:0/720009250 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa74151870 con 0x7ffa7415c990 2024-05-28T14:01:57.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.396+0000 7ffa7a800640 1 -- 172.21.15.42:0/720009250 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa74068a90 con 0x7ffa7415c990 2024-05-28T14:01:57.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.400+0000 7ffa6b400640 1 -- 172.21.15.42:0/720009250 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7ffa5c007400 con 0x7ffa7415c990 2024-05-28T14:01:57.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.400+0000 7ffa6b400640 1 --2- 172.21.15.42:0/720009250 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa50068680 0x7ffa5006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:57.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.400+0000 7ffa6b400640 1 -- 172.21.15.42:0/720009250 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7ffa5c086330 con 0x7ffa7415c990 2024-05-28T14:01:57.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.400+0000 7ffa73400640 1 --2- 172.21.15.42:0/720009250 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa50068680 0x7ffa5006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:57.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.400+0000 7ffa73400640 1 --2- 172.21.15.42:0/720009250 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa50068680 0x7ffa5006ab40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009a50 tx=0x7ffa64008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:57.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.404+0000 7ffa6b400640 1 -- 172.21.15.42:0/720009250 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffa74152440 con 0x7ffa7415c990 2024-05-28T14:01:57.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.612+0000 7ffa7a800640 1 -- 172.21.15.42:0/720009250 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffa740b4ce0 con 0x7ffa7415c990 2024-05-28T14:01:57.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.612+0000 7ffa6b400640 1 -- 172.21.15.42:0/720009250 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7ffa5c056540 con 0x7ffa7415c990 2024-05-28T14:01:57.597 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-28T14:01:57.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.612+0000 7ffa7a800640 1 -- 172.21.15.42:0/720009250 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa50068680 msgr2=0x7ffa5006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.612+0000 7ffa7a800640 1 --2- 172.21.15.42:0/720009250 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa50068680 0x7ffa5006ab40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009a50 tx=0x7ffa64008040 comp rx=0 tx=0).stop 2024-05-28T14:01:57.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.612+0000 7ffa7a800640 1 -- 172.21.15.42:0/720009250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa7415c990 msgr2=0x7ffa740c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:57.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.612+0000 7ffa7a800640 1 --2- 172.21.15.42:0/720009250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa7415c990 0x7ffa740c8b40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00c420 tx=0x7ffa5c00c8e0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.612+0000 7ffa7a800640 1 -- 172.21.15.42:0/720009250 shutdown_connections 2024-05-28T14:01:57.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.612+0000 7ffa7a800640 1 --2- 172.21.15.42:0/720009250 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa50068680 0x7ffa5006ab40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.612+0000 7ffa7a800640 1 --2- 172.21.15.42:0/720009250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa74157780 0x7ffa74150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.612+0000 7ffa7a800640 1 --2- 172.21.15.42:0/720009250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa7415c990 0x7ffa740c8b40 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.612+0000 7ffa7a800640 1 --2- 172.21.15.42:0/720009250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa740d06a0 0x7ffa74150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:57.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.612+0000 7ffa7a800640 1 -- 172.21.15.42:0/720009250 >> 172.21.15.42:0/720009250 conn(0x7ffa740b4080 msgr2=0x7ffa7409aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:57.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.612+0000 7ffa7a800640 1 -- 172.21.15.42:0/720009250 shutdown_connections 2024-05-28T14:01:57.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57.612+0000 7ffa7a800640 1 -- 172.21.15.42:0/720009250 wait complete. 2024-05-28T14:01:57.618 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-28T14:01:57.618 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-28T14:01:57.618 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-05-28T14:01:57.618 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-28T14:01:57.618 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-28T14:01:57.618 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-28T14:01:57.618 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-28T14:01:57.618 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-28T14:01:57.618 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:01:57.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:57.619 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-05-28T14:01:57.628 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T14:01:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T14:01:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T14:01:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:01:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-05-28T14:01:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:01:57.629 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:01:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-05-28T14:01:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-05-28T14:01:57.629 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:01:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-05-28T14:01:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-05-28T14:01:57.629 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-28T14:01:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:01:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-05-28T14:01:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-28T14:01:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42230sec preferred_lft 42230sec 2024-05-28T14:01:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2024-05-28T14:01:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:01:57.630 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:01:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:01:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-05-28T14:01:57.630 INFO:teuthology.orchestra.run.smithi042.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-28T14:01:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 02:42:65:2b:63:73 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:01:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-28T14:01:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:01:57.630 INFO:teuthology.orchestra.run.smithi042.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-28T14:01:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-28T14:01:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-28T14:01:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:01:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-28T14:01:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:01:57.630 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T14:01:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:01:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-28T14:01:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:01:57.631 INFO:teuthology.orchestra.run.smithi042.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T14:01:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:01:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-28T14:01:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:01:57.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:57.631 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-28T14:01:57.693 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-05-28T14:01:57.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:57.696 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-05-28T14:01:57.749 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-05-28T14:01:57.749 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-28T14:01:57.749 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-05-28T14:01:57.749 INFO:teuthology.orchestra.run.smithi042.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-28T14:01:57.749 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-05-28T14:01:57.749 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-28T14:01:57.750 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T14:01:57.750 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T14:01:57.750 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-28T14:01:57.750 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-28T14:01:57.750 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-28T14:01:57.750 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T14:01:57.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T14:01:57.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:01:57.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:57.888 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-28T14:01:57.897 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-28T14:01:57.897 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-28T14:01:57.898 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:01:57.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:57.898 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-05-28T14:01:57.960 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-28T14:01:57.960 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T14:01:57.960 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-28T14:01:58.012 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:01:58.012 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T14:01:58.012 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-28T14:01:58.063 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:01:58.123 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:01:58.181 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:01:58.181 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:01:58.181 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:01:58.181 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:01:58.181 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:01:58.181 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:01:58.181 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:01:58.181 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:01:58.181 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:01:58.182 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:01:58.182 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:01:58.182 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:01:58.182 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:01:58.182 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:01:58.182 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:01:58.182 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:01:58.182 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:01:58.182 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:01:58.182 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:01:58.182 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:01:58.182 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:01:58.182 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:01:58.183 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:01:58.183 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:01:58.183 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:01:58.183 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:01:58.183 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:01:58.183 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:01:58.183 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:01:58.183 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:01:58.183 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:01:58.183 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:01:58.183 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:01:58.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:58.184 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:01:58.239 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-28T14:01:58.240 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-28T14:01:58.290 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:01:58.310 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:01:58.365 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:01:58.365 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:01:58.365 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:01:58.365 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:01:58.365 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:01:58.365 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:01:58.365 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:01:58.365 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:01:58.366 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:01:58.366 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:01:58.366 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:01:58.366 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:01:58.366 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:01:58.366 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:01:58.366 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:01:58.366 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:01:58.366 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:01:58.366 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:01:58.366 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:01:58.366 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:01:58.366 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:01:58.367 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:01:58.367 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:01:58.367 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:01:58.367 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:01:58.367 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:01:58.367 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:01:58.367 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:01:58.367 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:01:58.367 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:01:58.367 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:01:58.367 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:01:58.367 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:01:58.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:58.368 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:01:58.387 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-05-28T14:01:58.400+0000 7fe8fc83a580 -1 init, newargv = 0x55ae3aa99af0 newargc=15 2024-05-28T14:01:58.387 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[39057]: starting ceph client 2024-05-28T14:01:58.394 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[39057]: starting fuse 2024-05-28T14:01:58.395 INFO:teuthology.orchestra.run.smithi042.stdout:44 2024-05-28T14:01:58.395 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-28T14:01:58.396 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-05-28T14:01:58.396 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-05-28T14:01:58.396 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-05-28T14:01:58.396 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-05-28T14:01:58.396 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-05-28T14:01:58.396 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:01:58.396 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-05-28T14:01:58.396 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-28T14:01:58.396 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-05-28T14:01:58.396 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-28T14:01:58.396 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:01:58.396 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-05-28T14:01:58.397 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-05-28T14:01:58.527 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.39057.asok 2024-05-28T14:01:58.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:01:58.538 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39057.asok 2024-05-28T14:01:58.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:58.539 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39057.asok status 2024-05-28T14:01:58.712 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:01:58.712 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-05-28T14:01:58.712 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-28T14:01:58.712 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-28T14:01:58.712 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-05-28T14:01:58.712 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-05-28T14:01:58.712 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-28T14:01:58.712 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "39057", 2024-05-28T14:01:58.712 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-05-28T14:01:58.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:58.713 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-05-28T14:01:58.713 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-05-28T14:01:58.713 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5967, 2024-05-28T14:01:58.713 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-05-28T14:01:58.713 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-05-28T14:01:58.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-05-28T14:01:58.713 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 5967 2024-05-28T14:01:58.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:58.713 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T14:01:58.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T14:01:58.713 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T14:01:58.713 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 782932083 2024-05-28T14:01:58.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:58.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:58.714 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T14:01:58.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T14:01:58.714 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T14:01:58.714 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 782932083 2024-05-28T14:01:58.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:01:58.714 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.5967 192.168.0.1:0/782932083", 2024-05-28T14:01:58.714 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/782932083", 2024-05-28T14:01:58.714 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-05-28T14:01:58.714 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 27, 2024-05-28T14:01:58.714 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 46, 2024-05-28T14:01:58.714 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-05-28T14:01:58.714 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-05-28T14:01:58.715 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-05-28T14:01:58.715 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:01:58.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:58.728 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-05-28T14:01:58.728 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-28T14:01:58.737 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-05-28T14:01:58.737 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-28T14:01:58.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:01:58.737 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-28T14:01:58.800 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-28T14:01:58.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.000+0000 7f4458a00640 1 Processor -- start 2024-05-28T14:01:58.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.000+0000 7f4458a00640 1 -- start start 2024-05-28T14:01:58.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4458a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f445406cbd0 0x7f445406cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:58.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4458a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44540afb20 0x7f445415cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:58.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4458a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44540732e0 con 0x7f445415d450 2024-05-28T14:01:58.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4458a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f44540735c0 con 0x7f445406cbd0 2024-05-28T14:01:58.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4458a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f44540ce110 con 0x7f44540afb20 2024-05-28T14:01:58.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4452800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f445415d450 0x7f44540d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53740/0 (socket says 172.21.15.42:53740) 2024-05-28T14:01:58.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4452800640 1 -- 172.21.15.42:0/3019003624 learned_addr learned my addr 172.21.15.42:0/3019003624 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:58.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4451e00640 1 --2- 172.21.15.42:0/3019003624 >> v2:172.21.15.63:3300/0 conn(0x7f445406cbd0 0x7f445406cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:58.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4449400640 1 --2- 172.21.15.42:0/3019003624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44540afb20 0x7f445415cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:58.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4451400640 1 -- 172.21.15.42:0/3019003624 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1178695530 0 0) 0x7f44540732e0 con 0x7f445415d450 2024-05-28T14:01:58.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4451400640 1 -- 172.21.15.42:0/3019003624 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4438003530 con 0x7f445415d450 2024-05-28T14:01:58.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4449400640 1 -- 172.21.15.42:0/3019003624 >> v2:172.21.15.63:3300/0 conn(0x7f445406cbd0 msgr2=0x7f445406cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:58.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4449400640 1 --2- 172.21.15.42:0/3019003624 >> v2:172.21.15.63:3300/0 conn(0x7f445406cbd0 0x7f445406cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:58.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4449400640 1 -- 172.21.15.42:0/3019003624 >> v1:172.21.15.42:6789/0 conn(0x7f445415d450 legacy=0x7f44540d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:58.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.004+0000 7f4449400640 1 -- 172.21.15.42:0/3019003624 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44540ce3f0 con 0x7f44540afb20 2024-05-28T14:01:58.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.008+0000 7f4449400640 1 --2- 172.21.15.42:0/3019003624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44540afb20 0x7f445415cf10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f443c0097c0 tx=0x7f443c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c568668cfcab26bd server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:58.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.008+0000 7f4451400640 1 -- 172.21.15.42:0/3019003624 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0b1070 con 0x7f44540afb20 2024-05-28T14:01:58.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.008+0000 7f4451400640 1 -- 172.21.15.42:0/3019003624 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f443c0ac410 con 0x7f44540afb20 2024-05-28T14:01:58.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.008+0000 7f4451400640 1 -- 172.21.15.42:0/3019003624 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0ac7c0 con 0x7f44540afb20 2024-05-28T14:01:58.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.008+0000 7f4458a00640 1 -- 172.21.15.42:0/3019003624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44540afb20 msgr2=0x7f445415cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:58.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.008+0000 7f4458a00640 1 --2- 172.21.15.42:0/3019003624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44540afb20 0x7f445415cf10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f443c0097c0 tx=0x7f443c01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:01:58.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.008+0000 7f4458a00640 1 -- 172.21.15.42:0/3019003624 shutdown_connections 2024-05-28T14:01:58.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.008+0000 7f4458a00640 1 --2- 172.21.15.42:0/3019003624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44540afb20 0x7f445415cf10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:58.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.008+0000 7f4458a00640 1 --2- 172.21.15.42:0/3019003624 >> v2:172.21.15.63:3300/0 conn(0x7f445406cbd0 0x7f445406cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:58.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.008+0000 7f4458a00640 1 -- 172.21.15.42:0/3019003624 >> 172.21.15.42:0/3019003624 conn(0x7f44540725b0 msgr2=0x7f44540729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:58.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.008+0000 7f4458a00640 1 -- 172.21.15.42:0/3019003624 shutdown_connections 2024-05-28T14:01:58.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.008+0000 7f4458a00640 1 -- 172.21.15.42:0/3019003624 wait complete. 2024-05-28T14:01:58.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.008+0000 7f4458a00640 1 Processor -- start 2024-05-28T14:01:58.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.008+0000 7f4458a00640 1 -- start start 2024-05-28T14:01:58.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4458a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f445406cbd0 0x7f445414efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:58.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4458a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44540afb20 0x7f44540b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:58.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4458a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f445415d450 0x7f44540b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:58.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4458a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f44541614c0 con 0x7f44540afb20 2024-05-28T14:01:58.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4458a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4454161340 con 0x7f445415d450 2024-05-28T14:01:58.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4458a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4454161640 con 0x7f445406cbd0 2024-05-28T14:01:58.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4449400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44540afb20 0x7f44540b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:58.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4449400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44540afb20 0x7f44540b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47852/0 (socket says 172.21.15.42:47852) 2024-05-28T14:01:58.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4449400640 1 -- 172.21.15.42:0/1093675429 learned_addr learned my addr 172.21.15.42:0/1093675429 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:58.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4451e00640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f445406cbd0 0x7f445414efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:58.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4452800640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f445415d450 0x7f44540b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:58.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4449400640 1 -- 172.21.15.42:0/1093675429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f445406cbd0 msgr2=0x7f445414efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:58.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4449400640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f445406cbd0 0x7f445414efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:58.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4449400640 1 -- 172.21.15.42:0/1093675429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f445415d450 msgr2=0x7f44540b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4449400640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f445415d450 0x7f44540b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4449400640 1 -- 172.21.15.42:0/1093675429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445406cfb0 con 0x7f44540afb20 2024-05-28T14:01:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4452800640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f445415d450 0x7f44540b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:01:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4449400640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44540afb20 0x7f44540b0810 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f443c0af040 tx=0x7f443c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f444aa00640 1 -- 172.21.15.42:0/1093675429 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0b9020 con 0x7f44540afb20 2024-05-28T14:01:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f444aa00640 1 -- 172.21.15.42:0/1093675429 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f443c0accc0 con 0x7f44540afb20 2024-05-28T14:01:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4458a00640 1 -- 172.21.15.42:0/1093675429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4454158220 con 0x7f44540afb20 2024-05-28T14:01:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f444aa00640 1 -- 172.21.15.42:0/1093675429 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0b1040 con 0x7f44540afb20 2024-05-28T14:01:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4458a00640 1 -- 172.21.15.42:0/1093675429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4454159ed0 con 0x7f44540afb20 2024-05-28T14:01:59.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.012+0000 7f4458a00640 1 -- 172.21.15.42:0/1093675429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4454068a90 con 0x7f44540afb20 2024-05-28T14:01:59.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.016+0000 7f444aa00640 1 -- 172.21.15.42:0/1093675429 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f443c0bc020 con 0x7f44540afb20 2024-05-28T14:01:59.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.016+0000 7f444aa00640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4430068650 0x7f443006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.016+0000 7f444aa00640 1 -- 172.21.15.42:0/1093675429 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7f443c1207f0 con 0x7f44540afb20 2024-05-28T14:01:59.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.016+0000 7f4451e00640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4430068650 0x7f443006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.016+0000 7f4451e00640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4430068650 0x7f443006ab10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4444011340 tx=0x7f444400b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:59.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.020+0000 7f444aa00640 1 -- 172.21.15.42:0/1093675429 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f443c0c3030 con 0x7f44540afb20 2024-05-28T14:01:59.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.220+0000 7f4458a00640 1 -- 172.21.15.42:0/1093675429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f44540703a0 con 0x7f44540afb20 2024-05-28T14:01:59.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.296+0000 7f444aa00640 1 -- 172.21.15.42:0/1093675429 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v28) ==== 136+0+0 (secure 0 0 0) 0x7f443c016090 con 0x7f44540afb20 2024-05-28T14:01:59.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.296+0000 7f4458a00640 1 -- 172.21.15.42:0/1093675429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4430068650 msgr2=0x7f443006ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:59.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.296+0000 7f4458a00640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4430068650 0x7f443006ab10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4444011340 tx=0x7f444400b040 comp rx=0 tx=0).stop 2024-05-28T14:01:59.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.296+0000 7f4458a00640 1 -- 172.21.15.42:0/1093675429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44540afb20 msgr2=0x7f44540b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:59.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.296+0000 7f4458a00640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44540afb20 0x7f44540b0810 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f443c0af040 tx=0x7f443c019240 comp rx=0 tx=0).stop 2024-05-28T14:01:59.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.296+0000 7f4458a00640 1 -- 172.21.15.42:0/1093675429 shutdown_connections 2024-05-28T14:01:59.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.296+0000 7f4458a00640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4430068650 0x7f443006ab10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.296+0000 7f4458a00640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f445415d450 0x7f44540b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.296+0000 7f4458a00640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44540afb20 0x7f44540b0810 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.296+0000 7f4458a00640 1 --2- 172.21.15.42:0/1093675429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f445406cbd0 0x7f445414efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.296+0000 7f4458a00640 1 -- 172.21.15.42:0/1093675429 >> 172.21.15.42:0/1093675429 conn(0x7f44540725b0 msgr2=0x7f44540d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:59.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.296+0000 7f4458a00640 1 -- 172.21.15.42:0/1093675429 shutdown_connections 2024-05-28T14:01:59.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.296+0000 7f4458a00640 1 -- 172.21.15.42:0/1093675429 wait complete. 2024-05-28T14:01:59.303 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:01:59.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8988e00640 1 Processor -- start 2024-05-28T14:01:59.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8988e00640 1 -- start start 2024-05-28T14:01:59.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8988e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f898406ff10 0x7f898415e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8988e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8984070370 0x7f898415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8988e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89840d0660 con 0x7f8984070800 2024-05-28T14:01:59.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8988e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f898409ba60 con 0x7f8984070370 2024-05-28T14:01:59.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8988e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f89840d0940 con 0x7f898406ff10 2024-05-28T14:01:59.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8983400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8984070800 0x7f8984070be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53756/0 (socket says 172.21.15.42:53756) 2024-05-28T14:01:59.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8983400640 1 -- 172.21.15.42:0/4030551255 learned_addr learned my addr 172.21.15.42:0/4030551255 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:01:59.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8983e00640 1 --2- 172.21.15.42:0/4030551255 >> v2:172.21.15.63:3300/0 conn(0x7f8984070370 0x7f898415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f897aa00640 1 --2- 172.21.15.42:0/4030551255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f898406ff10 0x7f898415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 -- 172.21.15.42:0/4030551255 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1029382884 0 0) 0x7f89840d0660 con 0x7f8984070800 2024-05-28T14:01:59.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 -- 172.21.15.42:0/4030551255 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8968003530 con 0x7f8984070800 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 -- 172.21.15.42:0/4030551255 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2678630131 0 0) 0x7f8968003530 con 0x7f8984070800 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 -- 172.21.15.42:0/4030551255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f898406ff10 msgr2=0x7f898415e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 --2- 172.21.15.42:0/4030551255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f898406ff10 0x7f898415e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 -- 172.21.15.42:0/4030551255 >> v2:172.21.15.63:3300/0 conn(0x7f8984070370 msgr2=0x7f898415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 --2- 172.21.15.42:0/4030551255 >> v2:172.21.15.63:3300/0 conn(0x7f8984070370 0x7f898415ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 -- 172.21.15.42:0/4030551255 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89840d0c20 con 0x7f8984070800 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 -- 172.21.15.42:0/4030551255 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f8974001a00 con 0x7f8984070800 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 -- 172.21.15.42:0/4030551255 >> v1:172.21.15.42:6789/0 conn(0x7f8984070800 legacy=0x7f8984070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 --2- 172.21.15.42:0/4030551255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f898406ff10 0x7f89680039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 --2- 172.21.15.42:0/4030551255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8968003f40 0x7f8968004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 --2- 172.21.15.42:0/4030551255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8968004d50 0x7f89680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 -- 172.21.15.42:0/4030551255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f898409ba60 con 0x7f8968004d50 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 -- 172.21.15.42:0/4030551255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f89840d0940 con 0x7f8968003f40 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8982a00640 1 -- 172.21.15.42:0/4030551255 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f896800b9d0 con 0x7f898406ff10 2024-05-28T14:01:59.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8983e00640 1 --2- 172.21.15.42:0/4030551255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8968004d50 0x7f89680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f897aa00640 1 --2- 172.21.15.42:0/4030551255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8968003f40 0x7f8968004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.460+0000 7f8983400640 1 --2- 172.21.15.42:0/4030551255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f898406ff10 0x7f89680039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8983e00640 1 -- 172.21.15.42:0/4030551255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f898406ff10 msgr2=0x7f89680039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:59.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8983e00640 1 --2- 172.21.15.42:0/4030551255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f898406ff10 0x7f89680039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8983e00640 1 -- 172.21.15.42:0/4030551255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8968003f40 msgr2=0x7f8968004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:59.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8983e00640 1 --2- 172.21.15.42:0/4030551255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8968003f40 0x7f8968004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8983e00640 1 -- 172.21.15.42:0/4030551255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8968003530 con 0x7f8968004d50 2024-05-28T14:01:59.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8983e00640 1 --2- 172.21.15.42:0/4030551255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8968004d50 0x7f89680051d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f897c009ae0 tx=0x7f897c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74960d31fc222649 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:59.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8982a00640 1 -- 172.21.15.42:0/4030551255 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c0b2070 con 0x7f8968004d50 2024-05-28T14:01:59.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8982a00640 1 -- 172.21.15.42:0/4030551255 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f897c0a56a0 con 0x7f8968004d50 2024-05-28T14:01:59.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8982a00640 1 -- 172.21.15.42:0/4030551255 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c0096b0 con 0x7f8968004d50 2024-05-28T14:01:59.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8988e00640 1 -- 172.21.15.42:0/4030551255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8968004d50 msgr2=0x7f89680051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:59.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8988e00640 1 --2- 172.21.15.42:0/4030551255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8968004d50 0x7f89680051d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f897c009ae0 tx=0x7f897c01ca50 comp rx=0 tx=0).stop 2024-05-28T14:01:59.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8983e00640 1 -- 172.21.15.42:0/4030551255 reap_dead start 2024-05-28T14:01:59.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8988e00640 1 -- 172.21.15.42:0/4030551255 shutdown_connections 2024-05-28T14:01:59.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8988e00640 1 -- 172.21.15.42:0/4030551255 >> 172.21.15.42:0/4030551255 conn(0x7f89840b3dd0 msgr2=0x7f89840b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:59.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8988e00640 1 -- 172.21.15.42:0/4030551255 shutdown_connections 2024-05-28T14:01:59.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.464+0000 7f8988e00640 1 -- 172.21.15.42:0/4030551255 wait complete. 2024-05-28T14:01:59.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8988e00640 1 Processor -- start 2024-05-28T14:01:59.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8988e00640 1 -- start start 2024-05-28T14:01:59.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8988e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8968004d50 0x7f89840b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8988e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89840af050 0x7f89841508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8988e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89840b1080 0x7f89840b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8988e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f89840d1700 con 0x7f89840b1080 2024-05-28T14:01:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8988e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f89840d11e0 con 0x7f89840af050 2024-05-28T14:01:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8988e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f89840b1890 con 0x7f8968004d50 2024-05-28T14:01:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8983e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89840b1080 0x7f89840b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8983e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89840b1080 0x7f89840b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47858/0 (socket says 172.21.15.42:47858) 2024-05-28T14:01:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8983e00640 1 -- 172.21.15.42:0/2414284009 learned_addr learned my addr 172.21.15.42:0/2414284009 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8983400640 1 --2- 172.21.15.42:0/2414284009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8968004d50 0x7f89840b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f897aa00640 1 --2- 172.21.15.42:0/2414284009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89840af050 0x7f89841508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8983e00640 1 -- 172.21.15.42:0/2414284009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8968004d50 msgr2=0x7f89840b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8983e00640 1 --2- 172.21.15.42:0/2414284009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8968004d50 0x7f89840b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8983e00640 1 -- 172.21.15.42:0/2414284009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89840af050 msgr2=0x7f89841508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8983e00640 1 --2- 172.21.15.42:0/2414284009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89840af050 0x7f89841508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8983e00640 1 -- 172.21.15.42:0/2414284009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89840af6e0 con 0x7f89840b1080 2024-05-28T14:01:59.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8983e00640 1 --2- 172.21.15.42:0/2414284009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89840b1080 0x7f89840b5130 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f897c009ae0 tx=0x7f8968005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:59.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f897be00640 1 -- 172.21.15.42:0/2414284009 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c0b2040 con 0x7f89840b1080 2024-05-28T14:01:59.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f897be00640 1 -- 172.21.15.42:0/2414284009 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f897c000e50 con 0x7f89840b1080 2024-05-28T14:01:59.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f897be00640 1 -- 172.21.15.42:0/2414284009 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c0096b0 con 0x7f89840b1080 2024-05-28T14:01:59.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8988e00640 1 -- 172.21.15.42:0/2414284009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89840af9c0 con 0x7f89840b1080 2024-05-28T14:01:59.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.468+0000 7f8988e00640 1 -- 172.21.15.42:0/2414284009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89840aff80 con 0x7f89840b1080 2024-05-28T14:01:59.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.472+0000 7f8988e00640 1 -- 172.21.15.42:0/2414284009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898404fcf0 con 0x7f89840b1080 2024-05-28T14:01:59.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.472+0000 7f897be00640 1 -- 172.21.15.42:0/2414284009 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f897c004240 con 0x7f89840b1080 2024-05-28T14:01:59.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.472+0000 7f897be00640 1 --2- 172.21.15.42:0/2414284009 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8960068650 0x7f896006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.472+0000 7f897be00640 1 -- 172.21.15.42:0/2414284009 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7f897c120590 con 0x7f89840b1080 2024-05-28T14:01:59.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.472+0000 7f8983400640 1 --2- 172.21.15.42:0/2414284009 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8960068650 0x7f896006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.472+0000 7f8983400640 1 --2- 172.21.15.42:0/2414284009 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8960068650 0x7f896006ab10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8974000f90 tx=0x7f8974006040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:59.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.476+0000 7f897be00640 1 -- 172.21.15.42:0/2414284009 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f897c0167c0 con 0x7f89840b1080 2024-05-28T14:01:59.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.664+0000 7f8988e00640 1 -- 172.21.15.42:0/2414284009 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f89840d1d50 con 0x7f8960068650 2024-05-28T14:01:59.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.668+0000 7f897be00640 1 -- 172.21.15.42:0/2414284009 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f89840d1d50 con 0x7f8960068650 2024-05-28T14:01:59.652 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:01:59.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:01:59.653 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:01:59.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:01:59.653 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:01:59.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.668+0000 7f8988e00640 1 -- 172.21.15.42:0/2414284009 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8960068650 msgr2=0x7f896006ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:59.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.668+0000 7f8988e00640 1 --2- 172.21.15.42:0/2414284009 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8960068650 0x7f896006ab10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8974000f90 tx=0x7f8974006040 comp rx=0 tx=0).stop 2024-05-28T14:01:59.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.668+0000 7f8988e00640 1 -- 172.21.15.42:0/2414284009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89840b1080 msgr2=0x7f89840b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:59.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.668+0000 7f8988e00640 1 --2- 172.21.15.42:0/2414284009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89840b1080 0x7f89840b5130 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f897c009ae0 tx=0x7f8968005840 comp rx=0 tx=0).stop 2024-05-28T14:01:59.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.668+0000 7f8988e00640 1 -- 172.21.15.42:0/2414284009 shutdown_connections 2024-05-28T14:01:59.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.668+0000 7f8988e00640 1 --2- 172.21.15.42:0/2414284009 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8960068650 0x7f896006ab10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.668+0000 7f8988e00640 1 --2- 172.21.15.42:0/2414284009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89840b1080 0x7f89840b5130 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.668+0000 7f8988e00640 1 --2- 172.21.15.42:0/2414284009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89840af050 0x7f89841508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.668+0000 7f8988e00640 1 --2- 172.21.15.42:0/2414284009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8968004d50 0x7f89840b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.668+0000 7f8988e00640 1 -- 172.21.15.42:0/2414284009 >> 172.21.15.42:0/2414284009 conn(0x7f89840b3dd0 msgr2=0x7f898415e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:59.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.668+0000 7f8988e00640 1 -- 172.21.15.42:0/2414284009 shutdown_connections 2024-05-28T14:01:59.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.668+0000 7f8988e00640 1 -- 172.21.15.42:0/2414284009 wait complete. 2024-05-28T14:01:59.674 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T14:01:59.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.832+0000 7f4069200640 1 Processor -- start 2024-05-28T14:01:59.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.832+0000 7f4069200640 1 -- start start 2024-05-28T14:01:59.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.832+0000 7f4069200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406415c6c0 0x7f40640d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.832+0000 7f4069200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f40640d0fb0 0x7f4064070930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.832+0000 7f4069200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406406fe90 con 0x7f406409aa50 2024-05-28T14:01:59.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.832+0000 7f4069200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f406409ba10 con 0x7f40640d0fb0 2024-05-28T14:01:59.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.832+0000 7f4069200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4064070130 con 0x7f406415c6c0 2024-05-28T14:01:59.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.836+0000 7f4063400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406415c6c0 0x7f40640d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.836+0000 7f4063e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f40640d0fb0 0x7f4064070930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.836+0000 7f4063400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406415c6c0 0x7f40640d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:41690/0 (socket says 172.21.15.42:41690) 2024-05-28T14:01:59.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.836+0000 7f4063400640 1 -- 172.21.15.42:0/1295347740 learned_addr learned my addr 172.21.15.42:0/1295347740 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:59.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.836+0000 7f4063400640 1 -- 172.21.15.42:0/1295347740 >> v2:172.21.15.63:3300/0 conn(0x7f40640d0fb0 msgr2=0x7f4064070930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:01:59.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.836+0000 7f4063400640 1 --2- 172.21.15.42:0/1295347740 >> v2:172.21.15.63:3300/0 conn(0x7f40640d0fb0 0x7f4064070930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.836+0000 7f4063400640 1 -- 172.21.15.42:0/1295347740 >> v1:172.21.15.63:6789/0 conn(0x7f406409aa50 legacy=0x7f40640d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:59.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.836+0000 7f4063400640 1 -- 172.21.15.42:0/1295347740 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4064070410 con 0x7f406415c6c0 2024-05-28T14:01:59.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.836+0000 7f4063400640 1 --2- 172.21.15.42:0/1295347740 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406415c6c0 0x7f40640d0360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4054009870 tx=0x7f405401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b216140ec6cca75 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:59.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4062a00640 1 -- 172.21.15.42:0/1295347740 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540a7070 con 0x7f406415c6c0 2024-05-28T14:01:59.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4062a00640 1 -- 172.21.15.42:0/1295347740 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40540b0070 con 0x7f406415c6c0 2024-05-28T14:01:59.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4062a00640 1 -- 172.21.15.42:0/1295347740 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540ab890 con 0x7f406415c6c0 2024-05-28T14:01:59.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 -- 172.21.15.42:0/1295347740 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406415c6c0 msgr2=0x7f40640d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:59.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 --2- 172.21.15.42:0/1295347740 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406415c6c0 0x7f40640d0360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4054009870 tx=0x7f405401c560 comp rx=0 tx=0).stop 2024-05-28T14:01:59.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 -- 172.21.15.42:0/1295347740 shutdown_connections 2024-05-28T14:01:59.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 --2- 172.21.15.42:0/1295347740 >> v2:172.21.15.63:3300/0 conn(0x7f40640d0fb0 0x7f4064070930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 --2- 172.21.15.42:0/1295347740 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406415c6c0 0x7f40640d0360 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 -- 172.21.15.42:0/1295347740 >> 172.21.15.42:0/1295347740 conn(0x7f40640b3dd0 msgr2=0x7f40640b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:01:59.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 -- 172.21.15.42:0/1295347740 shutdown_connections 2024-05-28T14:01:59.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 -- 172.21.15.42:0/1295347740 wait complete. 2024-05-28T14:01:59.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 Processor -- start 2024-05-28T14:01:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 -- start start 2024-05-28T14:01:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f406409aa50 0x7f4064150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40640d0fb0 0x7f40641fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406415c6c0 0x7f40641fb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4064071a50 con 0x7f406409aa50 2024-05-28T14:01:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f40640718d0 con 0x7f40640d0fb0 2024-05-28T14:01:59.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4069200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4064071bd0 con 0x7f406415c6c0 2024-05-28T14:01:59.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4063400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f406409aa50 0x7f4064150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4063400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f406409aa50 0x7f4064150300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47868/0 (socket says 172.21.15.42:47868) 2024-05-28T14:01:59.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4063400640 1 -- 172.21.15.42:0/3515130585 learned_addr learned my addr 172.21.15.42:0/3515130585 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:01:59.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f405be00640 1 --2- 172.21.15.42:0/3515130585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40640d0fb0 0x7f40641fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.840+0000 7f4063e00640 1 --2- 172.21.15.42:0/3515130585 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406415c6c0 0x7f40641fb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f4063400640 1 -- 172.21.15.42:0/3515130585 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406415c6c0 msgr2=0x7f40641fb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:59.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f4063400640 1 --2- 172.21.15.42:0/3515130585 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406415c6c0 0x7f40641fb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f4063400640 1 -- 172.21.15.42:0/3515130585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40640d0fb0 msgr2=0x7f40641fafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:01:59.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f4063400640 1 --2- 172.21.15.42:0/3515130585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40640d0fb0 0x7f40641fafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:01:59.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f4063400640 1 -- 172.21.15.42:0/3515130585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40640b5480 con 0x7f406409aa50 2024-05-28T14:01:59.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f4063400640 1 --2- 172.21.15.42:0/3515130585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f406409aa50 0x7f4064150300 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f40540ad040 tx=0x7f4054019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:59.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f405b400640 1 -- 172.21.15.42:0/3515130585 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540bf610 con 0x7f406409aa50 2024-05-28T14:01:59.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f405b400640 1 -- 172.21.15.42:0/3515130585 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40540bf7f0 con 0x7f406409aa50 2024-05-28T14:01:59.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f405b400640 1 -- 172.21.15.42:0/3515130585 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540bfbe0 con 0x7f406409aa50 2024-05-28T14:01:59.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f4069200640 1 -- 172.21.15.42:0/3515130585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40640b5ae0 con 0x7f406409aa50 2024-05-28T14:01:59.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f4069200640 1 -- 172.21.15.42:0/3515130585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40640b60a0 con 0x7f406409aa50 2024-05-28T14:01:59.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f4069200640 1 -- 172.21.15.42:0/3515130585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406404fcf0 con 0x7f406409aa50 2024-05-28T14:01:59.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f405b400640 1 -- 172.21.15.42:0/3515130585 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f40540bd020 con 0x7f406409aa50 2024-05-28T14:01:59.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f405b400640 1 --2- 172.21.15.42:0/3515130585 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4040068680 0x7f404006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:01:59.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f405b400640 1 -- 172.21.15.42:0/3515130585 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7f40541213b0 con 0x7f406409aa50 2024-05-28T14:01:59.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.844+0000 7f405be00640 1 --2- 172.21.15.42:0/3515130585 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4040068680 0x7f404006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:01:59.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.848+0000 7f405be00640 1 --2- 172.21.15.42:0/3515130585 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4040068680 0x7f404006ab40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f40641faf30 tx=0x7f404c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:01:59.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:01:59.848+0000 7f405b400640 1 -- 172.21.15.42:0/3515130585 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4054016150 con 0x7f406409aa50 2024-05-28T14:02:00.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.016+0000 7f4069200640 1 -- 172.21.15.42:0/3515130585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f40640cead0 con 0x7f406409aa50 2024-05-28T14:02:00.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.016+0000 7f405b400640 1 -- 172.21.15.42:0/3515130585 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f40540f0f80 con 0x7f406409aa50 2024-05-28T14:02:00.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.020+0000 7f4069200640 1 -- 172.21.15.42:0/3515130585 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4040068680 msgr2=0x7f404006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.020+0000 7f4069200640 1 --2- 172.21.15.42:0/3515130585 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4040068680 0x7f404006ab40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f40641faf30 tx=0x7f404c009040 comp rx=0 tx=0).stop 2024-05-28T14:02:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.020+0000 7f4069200640 1 -- 172.21.15.42:0/3515130585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f406409aa50 msgr2=0x7f4064150300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.020+0000 7f4069200640 1 --2- 172.21.15.42:0/3515130585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f406409aa50 0x7f4064150300 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f40540ad040 tx=0x7f4054019300 comp rx=0 tx=0).stop 2024-05-28T14:02:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.020+0000 7f4069200640 1 -- 172.21.15.42:0/3515130585 shutdown_connections 2024-05-28T14:02:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.020+0000 7f4069200640 1 --2- 172.21.15.42:0/3515130585 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4040068680 0x7f404006ab40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.020+0000 7f4069200640 1 --2- 172.21.15.42:0/3515130585 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406415c6c0 0x7f40641fb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.020+0000 7f4069200640 1 --2- 172.21.15.42:0/3515130585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40640d0fb0 0x7f40641fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.020+0000 7f4069200640 1 --2- 172.21.15.42:0/3515130585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f406409aa50 0x7f4064150300 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.020+0000 7f4069200640 1 -- 172.21.15.42:0/3515130585 >> 172.21.15.42:0/3515130585 conn(0x7f40640b3dd0 msgr2=0x7f40640d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:02:00.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.020+0000 7f4069200640 1 -- 172.21.15.42:0/3515130585 shutdown_connections 2024-05-28T14:02:00.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.020+0000 7f4069200640 1 -- 172.21.15.42:0/3515130585 wait complete. 2024-05-28T14:02:00.025 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-28T14:02:00.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.184+0000 7f45b5600640 1 Processor -- start 2024-05-28T14:02:00.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.184+0000 7f45b5600640 1 -- start start 2024-05-28T14:02:00.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.184+0000 7f45b5600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f45b006dc20 0x7f45b00d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:02:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.184+0000 7f45b5600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f45b006cd10 0x7f45b00cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:02:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.184+0000 7f45b5600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45b00d1970 con 0x7f45b015e840 2024-05-28T14:02:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.184+0000 7f45b5600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f45b00b5300 con 0x7f45b006dc20 2024-05-28T14:02:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.184+0000 7f45b5600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f45b00d1740 con 0x7f45b006cd10 2024-05-28T14:02:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.184+0000 7f45a6000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f45b006dc20 0x7f45b00d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:02:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.184+0000 7f45a6000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f45b006dc20 0x7f45b00d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47876/0 (socket says 172.21.15.42:47876) 2024-05-28T14:02:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.184+0000 7f45a6000640 1 -- 172.21.15.42:0/2721830141 learned_addr learned my addr 172.21.15.42:0/2721830141 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:02:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.184+0000 7f45af400640 1 --2- 172.21.15.42:0/2721830141 >> v2:172.21.15.63:3300/0 conn(0x7f45b006cd10 0x7f45b00cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:02:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45a6000640 1 -- 172.21.15.42:0/2721830141 >> v2:172.21.15.63:3300/0 conn(0x7f45b006cd10 msgr2=0x7f45b00cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:02:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45a6000640 1 --2- 172.21.15.42:0/2721830141 >> v2:172.21.15.63:3300/0 conn(0x7f45b006cd10 0x7f45b00cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45a6000640 1 -- 172.21.15.42:0/2721830141 >> v1:172.21.15.42:6789/0 conn(0x7f45b015e840 legacy=0x7f45b006d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45a6000640 1 -- 172.21.15.42:0/2721830141 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b00ccca0 con 0x7f45b006dc20 2024-05-28T14:02:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45a6000640 1 --2- 172.21.15.42:0/2721830141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45b006dc20 0x7f45b00d1200 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f45b00b8560 tx=0x7f459801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06f1701d8e67557 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:02:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45ae000640 1 -- 172.21.15.42:0/2721830141 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45980b5070 con 0x7f45b006dc20 2024-05-28T14:02:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45ae000640 1 -- 172.21.15.42:0/2721830141 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45980a8a10 con 0x7f45b006dc20 2024-05-28T14:02:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45ae000640 1 -- 172.21.15.42:0/2721830141 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45980a8de0 con 0x7f45b006dc20 2024-05-28T14:02:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45b5600640 1 -- 172.21.15.42:0/2721830141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45b006dc20 msgr2=0x7f45b00d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45b5600640 1 --2- 172.21.15.42:0/2721830141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45b006dc20 0x7f45b00d1200 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f45b00b8560 tx=0x7f459801ed10 comp rx=0 tx=0).stop 2024-05-28T14:02:00.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45b5600640 1 -- 172.21.15.42:0/2721830141 shutdown_connections 2024-05-28T14:02:00.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45b5600640 1 --2- 172.21.15.42:0/2721830141 >> v2:172.21.15.63:3300/0 conn(0x7f45b006cd10 0x7f45b00cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45b5600640 1 --2- 172.21.15.42:0/2721830141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45b006dc20 0x7f45b00d1200 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45b5600640 1 -- 172.21.15.42:0/2721830141 >> 172.21.15.42:0/2721830141 conn(0x7f45b00b3fa0 msgr2=0x7f45b00b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:02:00.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45b5600640 1 -- 172.21.15.42:0/2721830141 shutdown_connections 2024-05-28T14:02:00.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.188+0000 7f45b5600640 1 -- 172.21.15.42:0/2721830141 wait complete. 2024-05-28T14:02:00.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45b5600640 1 Processor -- start 2024-05-28T14:02:00.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45b5600640 1 -- start start 2024-05-28T14:02:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45b5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45b006cd10 0x7f45b0160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:02:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45b5600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45b006dc20 0x7f45b01516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:02:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45b5600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45b015e840 0x7f45b015ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:02:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45b5600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f45b015ec20 con 0x7f45b006cd10 2024-05-28T14:02:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45b5600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f45b00cd260 con 0x7f45b006dc20 2024-05-28T14:02:00.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45b5600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f45b015eda0 con 0x7f45b015e840 2024-05-28T14:02:00.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45aea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45b006cd10 0x7f45b0160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:02:00.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45aea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45b006cd10 0x7f45b0160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47892/0 (socket says 172.21.15.42:47892) 2024-05-28T14:02:00.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45a6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45b006dc20 0x7f45b01516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:02:00.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45af400640 1 --2- 172.21.15.42:0/2302066411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45b015e840 0x7f45b015ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:02:00.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45aea00640 1 -- 172.21.15.42:0/2302066411 learned_addr learned my addr 172.21.15.42:0/2302066411 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:02:00.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45a6000640 1 -- 172.21.15.42:0/2302066411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45b015e840 msgr2=0x7f45b015ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45a6000640 1 --2- 172.21.15.42:0/2302066411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45b015e840 0x7f45b015ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45a6000640 1 -- 172.21.15.42:0/2302066411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45b006cd10 msgr2=0x7f45b0160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45a6000640 1 --2- 172.21.15.42:0/2302066411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45b006cd10 0x7f45b0160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45a6000640 1 -- 172.21.15.42:0/2302066411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b00c5fe0 con 0x7f45b006dc20 2024-05-28T14:02:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45aea00640 1 --2- 172.21.15.42:0/2302066411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45b006cd10 0x7f45b0160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:02:00.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.192+0000 7f45a6000640 1 --2- 172.21.15.42:0/2302066411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45b006dc20 0x7f45b01516e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f45980af000 tx=0x7f459801bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:02:00.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.196+0000 7f45a7400640 1 -- 172.21.15.42:0/2302066411 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45980b5070 con 0x7f45b006dc20 2024-05-28T14:02:00.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.196+0000 7f45a7400640 1 -- 172.21.15.42:0/2302066411 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45980b0b50 con 0x7f45b006dc20 2024-05-28T14:02:00.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.196+0000 7f45a7400640 1 -- 172.21.15.42:0/2302066411 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45980bb410 con 0x7f45b006dc20 2024-05-28T14:02:00.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.196+0000 7f45b5600640 1 -- 172.21.15.42:0/2302066411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b00c62a0 con 0x7f45b006dc20 2024-05-28T14:02:00.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.196+0000 7f45b5600640 1 -- 172.21.15.42:0/2302066411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b00c6860 con 0x7f45b006dc20 2024-05-28T14:02:00.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.196+0000 7f45b5600640 1 -- 172.21.15.42:0/2302066411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b0068a90 con 0x7f45b006dc20 2024-05-28T14:02:00.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.196+0000 7f45a7400640 1 -- 172.21.15.42:0/2302066411 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7f45980c2060 con 0x7f45b006dc20 2024-05-28T14:02:00.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.196+0000 7f45a7400640 1 --2- 172.21.15.42:0/2302066411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f458c0686a0 0x7f458c06ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:02:00.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.196+0000 7f45a7400640 1 -- 172.21.15.42:0/2302066411 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7f4598123d50 con 0x7f45b006dc20 2024-05-28T14:02:00.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.196+0000 7f45aea00640 1 --2- 172.21.15.42:0/2302066411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f458c0686a0 0x7f458c06ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:02:00.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.200+0000 7f45aea00640 1 --2- 172.21.15.42:0/2302066411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f458c0686a0 0x7f458c06ab60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f45a0007fd0 tx=0x7f45a0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:02:00.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.200+0000 7f45a7400640 1 -- 172.21.15.42:0/2302066411 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4598018870 con 0x7f45b006dc20 2024-05-28T14:02:00.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.392+0000 7f45b5600640 1 -- 172.21.15.42:0/2302066411 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f45b00d1d80 con 0x7f458c0686a0 2024-05-28T14:02:00.383 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:02:00.396+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:02:00.383 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:02:00.396+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:02:00.383 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:02:00.396+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:02:00.383 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:02:00.396+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:02:00.383 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:02:00.396+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:02:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.428+0000 7f45a7400640 1 -- 172.21.15.42:0/2302066411 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f45b00d1d80 con 0x7f458c0686a0 2024-05-28T14:02:00.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.428+0000 7f45b5600640 1 -- 172.21.15.42:0/2302066411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f458c0686a0 msgr2=0x7f458c06ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.428+0000 7f45b5600640 1 --2- 172.21.15.42:0/2302066411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f458c0686a0 0x7f458c06ab60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f45a0007fd0 tx=0x7f45a0009040 comp rx=0 tx=0).stop 2024-05-28T14:02:00.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.428+0000 7f45b5600640 1 -- 172.21.15.42:0/2302066411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45b006dc20 msgr2=0x7f45b01516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.428+0000 7f45b5600640 1 --2- 172.21.15.42:0/2302066411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45b006dc20 0x7f45b01516e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f45980af000 tx=0x7f459801bab0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.428+0000 7f45b5600640 1 -- 172.21.15.42:0/2302066411 shutdown_connections 2024-05-28T14:02:00.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.428+0000 7f45b5600640 1 --2- 172.21.15.42:0/2302066411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f458c0686a0 0x7f458c06ab60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.428+0000 7f45b5600640 1 --2- 172.21.15.42:0/2302066411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45b015e840 0x7f45b015ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.428+0000 7f45b5600640 1 --2- 172.21.15.42:0/2302066411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45b006dc20 0x7f45b01516e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.428+0000 7f45b5600640 1 --2- 172.21.15.42:0/2302066411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45b006cd10 0x7f45b0160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.428+0000 7f45b5600640 1 -- 172.21.15.42:0/2302066411 >> 172.21.15.42:0/2302066411 conn(0x7f45b00b3fa0 msgr2=0x7f45b00cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:02:00.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.428+0000 7f45b5600640 1 -- 172.21.15.42:0/2302066411 shutdown_connections 2024-05-28T14:02:00.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.428+0000 7f45b5600640 1 -- 172.21.15.42:0/2302066411 wait complete. 2024-05-28T14:02:00.435 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:02:00.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.592+0000 7ff3e5a00640 1 Processor -- start 2024-05-28T14:02:00.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.592+0000 7ff3e5a00640 1 -- start start 2024-05-28T14:02:00.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.596+0000 7ff3e5a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff3e0071de0 0x7ff3e00721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:02:00.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.596+0000 7ff3e5a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e0072700 con 0x7ff3e013fd10 2024-05-28T14:02:00.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.596+0000 7ff3e5a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e0160230 con 0x7ff3e00ce3b0 2024-05-28T14:02:00.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.596+0000 7ff3e5a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff3e0160510 con 0x7ff3e0071de0 2024-05-28T14:02:00.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.596+0000 7ff3dea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff3e0071de0 0x7ff3e00721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:02:00.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.596+0000 7ff3df400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff3e013fd10 0x7ff3e0140160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53786/0 (socket says 172.21.15.42:53786) 2024-05-28T14:02:00.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.596+0000 7ff3df400640 1 -- 172.21.15.42:0/3001547567 learned_addr learned my addr 172.21.15.42:0/3001547567 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:02:00.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.596+0000 7ff3dea00640 1 --2- 172.21.15.42:0/3001547567 >> v2:172.21.15.42:3300/0 conn(0x7ff3e0071de0 0x7ff3e00721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47928/0 (socket says 172.21.15.42:47928) 2024-05-28T14:02:00.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.596+0000 7ff3dea00640 1 -- 172.21.15.42:0/3001547567 >> v1:172.21.15.63:6789/0 conn(0x7ff3e00ce3b0 legacy=0x7ff3e00ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.596+0000 7ff3dea00640 1 -- 172.21.15.42:0/3001547567 >> v1:172.21.15.42:6789/0 conn(0x7ff3e013fd10 legacy=0x7ff3e0140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.596+0000 7ff3dea00640 1 -- 172.21.15.42:0/3001547567 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e01607f0 con 0x7ff3e0071de0 2024-05-28T14:02:00.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.596+0000 7ff3de000640 1 -- 172.21.15.42:0/3001547567 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 10732206 0 0) 0x7ff3e0072700 con 0x7ff3e013fd10 2024-05-28T14:02:00.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.596+0000 7ff3dea00640 1 --2- 172.21.15.42:0/3001547567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3e0071de0 0x7ff3e00721c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00097c0 tx=0x7ff3d001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b630e707b060b5a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:02:00.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3de000640 1 -- 172.21.15.42:0/3001547567 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00b0070 con 0x7ff3e0071de0 2024-05-28T14:02:00.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3de000640 1 -- 172.21.15.42:0/3001547567 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d00ab600 con 0x7ff3e0071de0 2024-05-28T14:02:00.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3de000640 1 -- 172.21.15.42:0/3001547567 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00ab9d0 con 0x7ff3e0071de0 2024-05-28T14:02:00.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 -- 172.21.15.42:0/3001547567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3e0071de0 msgr2=0x7ff3e00721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 --2- 172.21.15.42:0/3001547567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3e0071de0 0x7ff3e00721c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00097c0 tx=0x7ff3d001c380 comp rx=0 tx=0).stop 2024-05-28T14:02:00.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 -- 172.21.15.42:0/3001547567 shutdown_connections 2024-05-28T14:02:00.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 --2- 172.21.15.42:0/3001547567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3e0071de0 0x7ff3e00721c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 -- 172.21.15.42:0/3001547567 >> 172.21.15.42:0/3001547567 conn(0x7ff3e00b21f0 msgr2=0x7ff3e00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:02:00.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 -- 172.21.15.42:0/3001547567 shutdown_connections 2024-05-28T14:02:00.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 -- 172.21.15.42:0/3001547567 wait complete. 2024-05-28T14:02:00.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 Processor -- start 2024-05-28T14:02:00.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 -- start start 2024-05-28T14:02:00.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3e0071de0 0x7ff3e0159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:02:00.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3e00ce3b0 0x7ff3e015baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:02:00.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3e013fd10 0x7ff3e013aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:02:00.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e016a840 con 0x7ff3e013fd10 2024-05-28T14:02:00.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e016a6c0 con 0x7ff3e0071de0 2024-05-28T14:02:00.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3e5a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e016a9c0 con 0x7ff3e00ce3b0 2024-05-28T14:02:00.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3df400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3e013fd10 0x7ff3e013aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:02:00.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3df400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3e013fd10 0x7ff3e013aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47934/0 (socket says 172.21.15.42:47934) 2024-05-28T14:02:00.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3df400640 1 -- 172.21.15.42:0/2914281189 learned_addr learned my addr 172.21.15.42:0/2914281189 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:02:00.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3dea00640 1 --2- 172.21.15.42:0/2914281189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3e0071de0 0x7ff3e0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:02:00.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.600+0000 7ff3d7e00640 1 --2- 172.21.15.42:0/2914281189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3e00ce3b0 0x7ff3e015baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:02:00.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.604+0000 7ff3df400640 1 -- 172.21.15.42:0/2914281189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3e00ce3b0 msgr2=0x7ff3e015baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.604+0000 7ff3df400640 1 --2- 172.21.15.42:0/2914281189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3e00ce3b0 0x7ff3e015baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.604+0000 7ff3df400640 1 -- 172.21.15.42:0/2914281189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3e0071de0 msgr2=0x7ff3e0159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.604+0000 7ff3df400640 1 --2- 172.21.15.42:0/2914281189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3e0071de0 0x7ff3e0159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.604+0000 7ff3df400640 1 -- 172.21.15.42:0/2914281189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e015c030 con 0x7ff3e013fd10 2024-05-28T14:02:00.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.604+0000 7ff3df400640 1 --2- 172.21.15.42:0/2914281189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3e013fd10 0x7ff3e013aee0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c7d0 tx=0x7ff3d800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:02:00.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.604+0000 7ff3d6a00640 1 -- 172.21.15.42:0/2914281189 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8018070 con 0x7ff3e013fd10 2024-05-28T14:02:00.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.604+0000 7ff3d6a00640 1 -- 172.21.15.42:0/2914281189 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d800a640 con 0x7ff3e013fd10 2024-05-28T14:02:00.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.604+0000 7ff3d6a00640 1 -- 172.21.15.42:0/2914281189 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d800aa70 con 0x7ff3e013fd10 2024-05-28T14:02:00.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.604+0000 7ff3e5a00640 1 -- 172.21.15.42:0/2914281189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e0152e20 con 0x7ff3e013fd10 2024-05-28T14:02:00.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.604+0000 7ff3e5a00640 1 -- 172.21.15.42:0/2914281189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e01533e0 con 0x7ff3e013fd10 2024-05-28T14:02:00.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.604+0000 7ff3e5a00640 1 -- 172.21.15.42:0/2914281189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e00738b0 con 0x7ff3e013fd10 2024-05-28T14:02:00.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.604+0000 7ff3d6a00640 1 -- 172.21.15.42:0/2914281189 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 88242+0+0 (secure 0 0 0) 0x7ff3d801b070 con 0x7ff3e013fd10 2024-05-28T14:02:00.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.608+0000 7ff3d6a00640 1 --2- 172.21.15.42:0/2914281189 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3bc068650 0x7ff3bc06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:02:00.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.608+0000 7ff3d6a00640 1 -- 172.21.15.42:0/2914281189 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5737+0+0 (secure 0 0 0) 0x7ff3d80862d0 con 0x7ff3e013fd10 2024-05-28T14:02:00.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.608+0000 7ff3dea00640 1 --2- 172.21.15.42:0/2914281189 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3bc068650 0x7ff3bc06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:02:00.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.608+0000 7ff3dea00640 1 --2- 172.21.15.42:0/2914281189 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3bc068650 0x7ff3bc06ab10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00ad000 tx=0x7ff3d00a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:02:00.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.612+0000 7ff3d6a00640 1 -- 172.21.15.42:0/2914281189 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff3d8055f20 con 0x7ff3e013fd10 2024-05-28T14:02:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.804+0000 7ff3e5a00640 1 -- 172.21.15.42:0/2914281189 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7ff3e0072ed0 con 0x7ff3bc068650 2024-05-28T14:02:00.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.808+0000 7ff3d6a00640 1 -- 172.21.15.42:0/2914281189 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7ff3e0072ed0 con 0x7ff3bc068650 2024-05-28T14:02:00.792 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43 2024-05-28T14:02:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.808+0000 7ff3e5a00640 1 -- 172.21.15.42:0/2914281189 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3bc068650 msgr2=0x7ff3bc06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.808+0000 7ff3e5a00640 1 --2- 172.21.15.42:0/2914281189 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3bc068650 0x7ff3bc06ab10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00ad000 tx=0x7ff3d00a6000 comp rx=0 tx=0).stop 2024-05-28T14:02:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.808+0000 7ff3e5a00640 1 -- 172.21.15.42:0/2914281189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3e013fd10 msgr2=0x7ff3e013aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:02:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.808+0000 7ff3e5a00640 1 --2- 172.21.15.42:0/2914281189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3e013fd10 0x7ff3e013aee0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c7d0 tx=0x7ff3d800cc90 comp rx=0 tx=0).stop 2024-05-28T14:02:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.808+0000 7ff3e5a00640 1 -- 172.21.15.42:0/2914281189 shutdown_connections 2024-05-28T14:02:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.808+0000 7ff3e5a00640 1 --2- 172.21.15.42:0/2914281189 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3bc068650 0x7ff3bc06ab10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.808+0000 7ff3e5a00640 1 --2- 172.21.15.42:0/2914281189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3e013fd10 0x7ff3e013aee0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.808+0000 7ff3e5a00640 1 --2- 172.21.15.42:0/2914281189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3e00ce3b0 0x7ff3e015baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.808+0000 7ff3e5a00640 1 --2- 172.21.15.42:0/2914281189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3e0071de0 0x7ff3e0159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:02:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.808+0000 7ff3e5a00640 1 -- 172.21.15.42:0/2914281189 >> 172.21.15.42:0/2914281189 conn(0x7ff3e00b21f0 msgr2=0x7ff3e006f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:02:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.808+0000 7ff3e5a00640 1 -- 172.21.15.42:0/2914281189 shutdown_connections 2024-05-28T14:02:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:02:00.808+0000 7ff3e5a00640 1 -- 172.21.15.42:0/2914281189 wait complete. 2024-05-28T14:02:00.814 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43 2024-05-28T14:02:00.814 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:02:00.814 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:02:00.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:02:00.823 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:02:08.160 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:02:08.160 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:02:08.160 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.27575 s, 148 MB/s 2024-05-28T14:02:08.161 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:02:08.162 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:02:08.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:02:08.171 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:02:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:02:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:02:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.798 s, 158 MB/s 2024-05-28T14:02:15.023 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:02:15.023 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:02:15.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:02:15.032 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:02:18.087 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:02:18.091 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:02:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:02:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:02:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.81018 s, 158 MB/s 2024-05-28T14:02:21.899 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:02:21.899 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:02:21.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:02:21.910 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:02:28.821 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:02:28.821 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:02:28.821 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.85115 s, 157 MB/s 2024-05-28T14:02:28.823 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:02:28.823 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:02:28.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:02:28.831 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:02:35.747 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:02:35.747 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:02:35.747 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.85478 s, 157 MB/s 2024-05-28T14:02:35.749 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:02:35.749 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:02:35.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:02:35.758 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:02:42.685 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:02:42.685 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:02:42.685 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.87251 s, 156 MB/s 2024-05-28T14:02:42.686 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:02:42.686 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:02:42.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:02:42.696 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:02:48.150 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:02:48.153 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:02:49.588 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:02:49.589 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:02:49.589 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.83647 s, 157 MB/s 2024-05-28T14:02:49.590 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:02:49.590 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:02:49.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:02:49.599 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:02:56.576 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:02:56.576 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:02:56.577 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.92066 s, 155 MB/s 2024-05-28T14:02:56.578 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:02:56.578 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:02:56.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:02:56.587 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:03:03.555 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:03:03.555 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:03:03.555 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.91377 s, 155 MB/s 2024-05-28T14:03:03.555 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:03:03.555 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:03:03.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:03:03.565 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:03:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:03:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:03:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.89145 s, 156 MB/s 2024-05-28T14:03:10.516 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-28T14:03:10.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.675+0000 7f3ba6800640 1 Processor -- start 2024-05-28T14:03:10.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.675+0000 7f3ba6800640 1 -- start start 2024-05-28T14:03:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.675+0000 7f3ba6800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3ba006ee80 0x7f3ba006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.675+0000 7f3ba6800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ba01570a0 con 0x7f3ba00af360 2024-05-28T14:03:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.675+0000 7f3ba6800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ba015cf00 con 0x7f3ba00d0e00 2024-05-28T14:03:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.675+0000 7f3ba6800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3ba015d1e0 con 0x7f3ba006ee80 2024-05-28T14:03:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.675+0000 7f3b97e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f3ba00af360 0x7f3ba00d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46302/0 (socket says 172.21.15.42:46302) 2024-05-28T14:03:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.675+0000 7f3b97e00640 1 -- 172.21.15.42:0/3817941008 learned_addr learned my addr 172.21.15.42:0/3817941008 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:03:10.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.675+0000 7f3b9f400640 1 --2- 172.21.15.42:0/3817941008 >> v2:172.21.15.63:3300/0 conn(0x7f3ba006ee80 0x7f3ba006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:10.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.679+0000 7f3b9ea00640 1 -- 172.21.15.42:0/3817941008 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1303339679 0 0) 0x7f3ba015cf00 con 0x7f3ba00d0e00 2024-05-28T14:03:10.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.679+0000 7f3b9ea00640 1 -- 172.21.15.42:0/3817941008 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b84003530 con 0x7f3ba00d0e00 2024-05-28T14:03:10.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.679+0000 7f3b9ea00640 1 -- 172.21.15.42:0/3817941008 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1813471804 0 0) 0x7f3ba01570a0 con 0x7f3ba00af360 2024-05-28T14:03:10.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.679+0000 7f3b9ea00640 1 -- 172.21.15.42:0/3817941008 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ba015cf00 con 0x7f3ba00af360 2024-05-28T14:03:10.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.679+0000 7f3b9f400640 1 -- 172.21.15.42:0/3817941008 >> v1:172.21.15.63:6789/0 conn(0x7f3ba00d0e00 legacy=0x7f3ba0160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:10.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.679+0000 7f3b9f400640 1 -- 172.21.15.42:0/3817941008 >> v1:172.21.15.42:6789/0 conn(0x7f3ba00af360 legacy=0x7f3ba00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:10.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.679+0000 7f3b9f400640 1 -- 172.21.15.42:0/3817941008 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba015d4c0 con 0x7f3ba006ee80 2024-05-28T14:03:10.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.679+0000 7f3b9f400640 1 --2- 172.21.15.42:0/3817941008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ba006ee80 0x7f3ba006dd40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3b900097c0 tx=0x7f3b9001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efca03f4b06f0a46 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:10.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.679+0000 7f3b9ea00640 1 -- 172.21.15.42:0/3817941008 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b900ab600 con 0x7f3ba006ee80 2024-05-28T14:03:10.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.679+0000 7f3b9ea00640 1 -- 172.21.15.42:0/3817941008 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b900b0070 con 0x7f3ba006ee80 2024-05-28T14:03:10.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.679+0000 7f3b9ea00640 1 -- 172.21.15.42:0/3817941008 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b900ab9c0 con 0x7f3ba006ee80 2024-05-28T14:03:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 -- 172.21.15.42:0/3817941008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ba006ee80 msgr2=0x7f3ba006dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 --2- 172.21.15.42:0/3817941008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ba006ee80 0x7f3ba006dd40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3b900097c0 tx=0x7f3b9001c3e0 comp rx=0 tx=0).stop 2024-05-28T14:03:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 -- 172.21.15.42:0/3817941008 shutdown_connections 2024-05-28T14:03:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 --2- 172.21.15.42:0/3817941008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ba006ee80 0x7f3ba006dd40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 -- 172.21.15.42:0/3817941008 >> 172.21.15.42:0/3817941008 conn(0x7f3ba006bdf0 msgr2=0x7f3ba006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 -- 172.21.15.42:0/3817941008 shutdown_connections 2024-05-28T14:03:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 -- 172.21.15.42:0/3817941008 wait complete. 2024-05-28T14:03:10.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 Processor -- start 2024-05-28T14:03:10.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 -- start start 2024-05-28T14:03:10.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba006ee80 0x7f3ba00b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:10.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ba00af360 0x7f3ba00737c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:10.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ba00d0e00 0x7f3ba0099860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba015cce0 con 0x7f3ba006ee80 2024-05-28T14:03:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba015cb60 con 0x7f3ba00d0e00 2024-05-28T14:03:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3ba6800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba0160fa0 con 0x7f3ba00af360 2024-05-28T14:03:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3b9f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba006ee80 0x7f3ba00b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3b9f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba006ee80 0x7f3ba00b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45760/0 (socket says 172.21.15.42:45760) 2024-05-28T14:03:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3b9f400640 1 -- 172.21.15.42:0/391341706 learned_addr learned my addr 172.21.15.42:0/391341706 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3b9fe00640 1 --2- 172.21.15.42:0/391341706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ba00d0e00 0x7f3ba0099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3b97e00640 1 --2- 172.21.15.42:0/391341706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ba00af360 0x7f3ba00737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3b9f400640 1 -- 172.21.15.42:0/391341706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ba00af360 msgr2=0x7f3ba00737c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3b9f400640 1 --2- 172.21.15.42:0/391341706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ba00af360 0x7f3ba00737c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3b9f400640 1 -- 172.21.15.42:0/391341706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ba00d0e00 msgr2=0x7f3ba0099860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3b9f400640 1 --2- 172.21.15.42:0/391341706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ba00d0e00 0x7f3ba0099860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.683+0000 7f3b9f400640 1 -- 172.21.15.42:0/391341706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba00b2f40 con 0x7f3ba006ee80 2024-05-28T14:03:10.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.687+0000 7f3b9f400640 1 --2- 172.21.15.42:0/391341706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba006ee80 0x7f3ba00b2880 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3b90018ff0 tx=0x7f3b9001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:10.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.687+0000 7f3b97400640 1 -- 172.21.15.42:0/391341706 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b900bf410 con 0x7f3ba006ee80 2024-05-28T14:03:10.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.687+0000 7f3b97400640 1 -- 172.21.15.42:0/391341706 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b900bf5b0 con 0x7f3ba006ee80 2024-05-28T14:03:10.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.687+0000 7f3b97400640 1 -- 172.21.15.42:0/391341706 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b900bf9a0 con 0x7f3ba006ee80 2024-05-28T14:03:10.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.687+0000 7f3ba6800640 1 -- 172.21.15.42:0/391341706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba00b3220 con 0x7f3ba006ee80 2024-05-28T14:03:10.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.687+0000 7f3ba6800640 1 -- 172.21.15.42:0/391341706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba0150490 con 0x7f3ba006ee80 2024-05-28T14:03:10.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.687+0000 7f3ba6800640 1 -- 172.21.15.42:0/391341706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba0068a90 con 0x7f3ba006ee80 2024-05-28T14:03:10.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.687+0000 7f3b97400640 1 -- 172.21.15.42:0/391341706 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 10) ==== 88242+0+0 (secure 0 0 0) 0x7f3b900bd020 con 0x7f3ba006ee80 2024-05-28T14:03:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.687+0000 7f3b97400640 1 --2- 172.21.15.42:0/391341706 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b7c068650 0x7f3b7c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.687+0000 7f3b97400640 1 -- 172.21.15.42:0/391341706 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f3b90121380 con 0x7f3ba006ee80 2024-05-28T14:03:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.687+0000 7f3b97e00640 1 --2- 172.21.15.42:0/391341706 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b7c068650 0x7f3b7c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.691+0000 7f3b97e00640 1 --2- 172.21.15.42:0/391341706 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b7c068650 0x7f3b7c06ab10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0153820 tx=0x7f3b88009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.691+0000 7f3b97400640 1 -- 172.21.15.42:0/391341706 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3b90016090 con 0x7f3ba006ee80 2024-05-28T14:03:10.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.887+0000 7f3ba6800640 1 -- 172.21.15.42:0/391341706 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f3ba01506d0 con 0x7f3b7c068650 2024-05-28T14:03:10.878 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:10.891+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:10.878 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:10.891+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:10.878 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:10.891+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:10.879 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:10.891+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:10.879 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:10.891+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:10.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.911+0000 7f3b97400640 1 -- 172.21.15.42:0/391341706 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3b88001cf0 con 0x7f3b7c068650 2024-05-28T14:03:10.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.915+0000 7f3ba6800640 1 -- 172.21.15.42:0/391341706 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b7c068650 msgr2=0x7f3b7c06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:10.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.915+0000 7f3ba6800640 1 --2- 172.21.15.42:0/391341706 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b7c068650 0x7f3b7c06ab10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0153820 tx=0x7f3b88009040 comp rx=0 tx=0).stop 2024-05-28T14:03:10.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.915+0000 7f3ba6800640 1 -- 172.21.15.42:0/391341706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba006ee80 msgr2=0x7f3ba00b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:10.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.915+0000 7f3ba6800640 1 --2- 172.21.15.42:0/391341706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba006ee80 0x7f3ba00b2880 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3b90018ff0 tx=0x7f3b9001ac20 comp rx=0 tx=0).stop 2024-05-28T14:03:10.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.915+0000 7f3ba6800640 1 -- 172.21.15.42:0/391341706 shutdown_connections 2024-05-28T14:03:10.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.915+0000 7f3ba6800640 1 --2- 172.21.15.42:0/391341706 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b7c068650 0x7f3b7c06ab10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:10.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.915+0000 7f3ba6800640 1 --2- 172.21.15.42:0/391341706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ba00d0e00 0x7f3ba0099860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:10.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.915+0000 7f3ba6800640 1 --2- 172.21.15.42:0/391341706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ba00af360 0x7f3ba00737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:10.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.915+0000 7f3ba6800640 1 --2- 172.21.15.42:0/391341706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba006ee80 0x7f3ba00b2880 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:10.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.915+0000 7f3ba6800640 1 -- 172.21.15.42:0/391341706 >> 172.21.15.42:0/391341706 conn(0x7f3ba006bdf0 msgr2=0x7f3ba015f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:10.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.915+0000 7f3ba6800640 1 -- 172.21.15.42:0/391341706 shutdown_connections 2024-05-28T14:03:10.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:10.915+0000 7f3ba6800640 1 -- 172.21.15.42:0/391341706 wait complete. 2024-05-28T14:03:10.920 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:03:11.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.075+0000 7f7a60e00640 1 Processor -- start 2024-05-28T14:03:11.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.075+0000 7f7a60e00640 1 -- start start 2024-05-28T14:03:11.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a60e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7a5c0ce3b0 0x7f7a5c0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:11.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a60e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7a5c13fd10 0x7f7a5c140180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:11.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a60e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a5c160230 con 0x7f7a5c071de0 2024-05-28T14:03:11.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a60e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f7a5c160510 con 0x7f7a5c0ce3b0 2024-05-28T14:03:11.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a60e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7a5c1607f0 con 0x7f7a5c13fd10 2024-05-28T14:03:11.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a59e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f7a5c071de0 0x7f7a5c0721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46314/0 (socket says 172.21.15.42:46314) 2024-05-28T14:03:11.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a59e00640 1 -- 172.21.15.42:0/533950280 learned_addr learned my addr 172.21.15.42:0/533950280 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:03:11.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a53e00640 1 --2- 172.21.15.42:0/533950280 >> v2:172.21.15.42:3300/0 conn(0x7f7a5c0ce3b0 0x7f7a5c0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:11.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a5a800640 1 --2- 172.21.15.42:0/533950280 >> v2:172.21.15.63:3300/0 conn(0x7f7a5c13fd10 0x7f7a5c140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:11.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a59400640 1 -- 172.21.15.42:0/533950280 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1654219823 0 0) 0x7f7a5c160230 con 0x7f7a5c071de0 2024-05-28T14:03:11.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a59400640 1 -- 172.21.15.42:0/533950280 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a40003530 con 0x7f7a5c071de0 2024-05-28T14:03:11.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a53e00640 1 -- 172.21.15.42:0/533950280 >> v2:172.21.15.63:3300/0 conn(0x7f7a5c13fd10 msgr2=0x7f7a5c140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:11.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a53e00640 1 --2- 172.21.15.42:0/533950280 >> v2:172.21.15.63:3300/0 conn(0x7f7a5c13fd10 0x7f7a5c140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:11.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a53e00640 1 -- 172.21.15.42:0/533950280 >> v1:172.21.15.42:6789/0 conn(0x7f7a5c071de0 legacy=0x7f7a5c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:11.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.079+0000 7f7a53e00640 1 -- 172.21.15.42:0/533950280 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c160ad0 con 0x7f7a5c0ce3b0 2024-05-28T14:03:11.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.083+0000 7f7a53e00640 1 --2- 172.21.15.42:0/533950280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a5c0ce3b0 0x7f7a5c0ce810 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c071680 tx=0x7f7a4401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a600c7afe26b3eb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:11.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.083+0000 7f7a59400640 1 -- 172.21.15.42:0/533950280 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a440b0070 con 0x7f7a5c0ce3b0 2024-05-28T14:03:11.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.083+0000 7f7a59400640 1 -- 172.21.15.42:0/533950280 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a440ab600 con 0x7f7a5c0ce3b0 2024-05-28T14:03:11.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.083+0000 7f7a59400640 1 -- 172.21.15.42:0/533950280 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a440ab9d0 con 0x7f7a5c0ce3b0 2024-05-28T14:03:11.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.083+0000 7f7a60e00640 1 -- 172.21.15.42:0/533950280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a5c0ce3b0 msgr2=0x7f7a5c0ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:11.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.083+0000 7f7a60e00640 1 --2- 172.21.15.42:0/533950280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a5c0ce3b0 0x7f7a5c0ce810 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c071680 tx=0x7f7a4401c360 comp rx=0 tx=0).stop 2024-05-28T14:03:11.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.083+0000 7f7a60e00640 1 -- 172.21.15.42:0/533950280 shutdown_connections 2024-05-28T14:03:11.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.083+0000 7f7a60e00640 1 --2- 172.21.15.42:0/533950280 >> v2:172.21.15.63:3300/0 conn(0x7f7a5c13fd10 0x7f7a5c140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:11.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.083+0000 7f7a60e00640 1 --2- 172.21.15.42:0/533950280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a5c0ce3b0 0x7f7a5c0ce810 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:11.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.083+0000 7f7a60e00640 1 -- 172.21.15.42:0/533950280 >> 172.21.15.42:0/533950280 conn(0x7f7a5c0b21f0 msgr2=0x7f7a5c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:11.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.083+0000 7f7a60e00640 1 -- 172.21.15.42:0/533950280 shutdown_connections 2024-05-28T14:03:11.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.083+0000 7f7a60e00640 1 -- 172.21.15.42:0/533950280 wait complete. 2024-05-28T14:03:11.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.083+0000 7f7a60e00640 1 Processor -- start 2024-05-28T14:03:11.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.083+0000 7f7a60e00640 1 -- start start 2024-05-28T14:03:11.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a60e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a5c071de0 0x7f7a5c1551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:11.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a60e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a5c0ce3b0 0x7f7a5c0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:11.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a60e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a5c13fd10 0x7f7a5c06f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:11.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a60e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c169830 con 0x7f7a5c13fd10 2024-05-28T14:03:11.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a60e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1696b0 con 0x7f7a5c0ce3b0 2024-05-28T14:03:11.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a60e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5c1699b0 con 0x7f7a5c071de0 2024-05-28T14:03:11.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a5a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a5c13fd10 0x7f7a5c06f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a5a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a5c13fd10 0x7f7a5c06f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45796/0 (socket says 172.21.15.42:45796) 2024-05-28T14:03:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a5a800640 1 -- 172.21.15.42:0/3339509532 learned_addr learned my addr 172.21.15.42:0/3339509532 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a59e00640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a5c071de0 0x7f7a5c1551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a53e00640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a5c0ce3b0 0x7f7a5c0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a5a800640 1 -- 172.21.15.42:0/3339509532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a5c071de0 msgr2=0x7f7a5c1551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a5a800640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a5c071de0 0x7f7a5c1551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a5a800640 1 -- 172.21.15.42:0/3339509532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a5c0ce3b0 msgr2=0x7f7a5c0b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a5a800640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a5c0ce3b0 0x7f7a5c0b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a5a800640 1 -- 172.21.15.42:0/3339509532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c06e360 con 0x7f7a5c13fd10 2024-05-28T14:03:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a53e00640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a5c0ce3b0 0x7f7a5c0b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:03:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a5a800640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a5c13fd10 0x7f7a5c06f400 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7a54007e60 tx=0x7f7a5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:11.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a52000640 1 -- 172.21.15.42:0/3339509532 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54014420 con 0x7f7a5c13fd10 2024-05-28T14:03:11.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a52000640 1 -- 172.21.15.42:0/3339509532 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a54019070 con 0x7f7a5c13fd10 2024-05-28T14:03:11.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a52000640 1 -- 172.21.15.42:0/3339509532 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54014820 con 0x7f7a5c13fd10 2024-05-28T14:03:11.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a60e00640 1 -- 172.21.15.42:0/3339509532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5c06e640 con 0x7f7a5c13fd10 2024-05-28T14:03:11.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a60e00640 1 -- 172.21.15.42:0/3339509532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5c06ec00 con 0x7f7a5c13fd10 2024-05-28T14:03:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a60e00640 1 -- 172.21.15.42:0/3339509532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a5c0738b0 con 0x7f7a5c13fd10 2024-05-28T14:03:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.087+0000 7f7a52000640 1 -- 172.21.15.42:0/3339509532 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 10) ==== 88242+0+0 (secure 0 0 0) 0x7f7a540149c0 con 0x7f7a5c13fd10 2024-05-28T14:03:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.091+0000 7f7a52000640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a38068650 0x7f7a3806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.091+0000 7f7a52000640 1 -- 172.21.15.42:0/3339509532 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f7a540876d0 con 0x7f7a5c13fd10 2024-05-28T14:03:11.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.091+0000 7f7a59e00640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a38068650 0x7f7a3806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:11.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.091+0000 7f7a59e00640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a38068650 0x7f7a3806ab10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c005890 tx=0x7f7a4c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:11.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.095+0000 7f7a52000640 1 -- 172.21.15.42:0/3339509532 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7a540578a0 con 0x7f7a5c13fd10 2024-05-28T14:03:11.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.291+0000 7f7a60e00640 1 -- 172.21.15.42:0/3339509532 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f7a5c04fdd0 con 0x7f7a38068650 2024-05-28T14:03:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.295+0000 7f7a52000640 1 -- 172.21.15.42:0/3339509532 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f7a5c04fdd0 con 0x7f7a38068650 2024-05-28T14:03:11.282 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43 2024-05-28T14:03:11.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.295+0000 7f7a60e00640 1 -- 172.21.15.42:0/3339509532 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a38068650 msgr2=0x7f7a3806ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:11.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.295+0000 7f7a60e00640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a38068650 0x7f7a3806ab10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c005890 tx=0x7f7a4c009040 comp rx=0 tx=0).stop 2024-05-28T14:03:11.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.295+0000 7f7a60e00640 1 -- 172.21.15.42:0/3339509532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a5c13fd10 msgr2=0x7f7a5c06f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:11.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.295+0000 7f7a60e00640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a5c13fd10 0x7f7a5c06f400 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7a54007e60 tx=0x7f7a5400e4b0 comp rx=0 tx=0).stop 2024-05-28T14:03:11.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.299+0000 7f7a60e00640 1 -- 172.21.15.42:0/3339509532 shutdown_connections 2024-05-28T14:03:11.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.299+0000 7f7a60e00640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a38068650 0x7f7a3806ab10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:11.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.299+0000 7f7a60e00640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a5c13fd10 0x7f7a5c06f400 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:11.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.299+0000 7f7a60e00640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a5c0ce3b0 0x7f7a5c0b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:11.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.299+0000 7f7a60e00640 1 --2- 172.21.15.42:0/3339509532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a5c071de0 0x7f7a5c1551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:11.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.299+0000 7f7a60e00640 1 -- 172.21.15.42:0/3339509532 >> 172.21.15.42:0/3339509532 conn(0x7f7a5c0b21f0 msgr2=0x7f7a5c156590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:11.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.299+0000 7f7a60e00640 1 -- 172.21.15.42:0/3339509532 shutdown_connections 2024-05-28T14:03:11.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:11.299+0000 7f7a60e00640 1 -- 172.21.15.42:0/3339509532 wait complete. 2024-05-28T14:03:11.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:03:11.303 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43 2024-05-28T14:03:11.303 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:03:11.317 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43 2024-05-28T14:03:11.318 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="10654449664" 2024-05-28T14:03:11.318 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:03:11.318 INFO:tasks.cephfs.test_volumes:r_size = 10654449664 exp_size = 10737418240 2024-05-28T14:03:13.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:03:13.318 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43 2024-05-28T14:03:13.319 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:03:13.334 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/ac15c1b6-ac99-45b0-86f0-5d72da092f43 2024-05-28T14:03:13.334 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="10737418240" 2024-05-28T14:03:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:03:13.335 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-05-28T14:03:13.335 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone85311 2024-05-28T14:03:13.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.547+0000 7f554a400640 1 Processor -- start 2024-05-28T14:03:13.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.547+0000 7f554a400640 1 -- start start 2024-05-28T14:03:13.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.547+0000 7f554a400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5544145d50 0x7f55440cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:13.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.547+0000 7f554a400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55440cd050 0x7f55440d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:13.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.547+0000 7f554a400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5544160170 con 0x7f55440cf540 2024-05-28T14:03:13.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.547+0000 7f554a400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5544160450 con 0x7f5544145d50 2024-05-28T14:03:13.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.547+0000 7f554a400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5544160730 con 0x7f55440cd050 2024-05-28T14:03:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.547+0000 7f553be00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5544145d50 0x7f55440cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.547+0000 7f5549a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55440cd050 0x7f55440d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.547+0000 7f5549a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55440cd050 0x7f55440d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:48776/0 (socket says 172.21.15.42:48776) 2024-05-28T14:03:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.547+0000 7f5549a00640 1 -- 172.21.15.42:0/3432305720 learned_addr learned my addr 172.21.15.42:0/3432305720 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.547+0000 7f553be00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5544145d50 0x7f55440cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:49250/0 (socket says 172.21.15.42:49250) 2024-05-28T14:03:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.551+0000 7f5549a00640 1 -- 172.21.15.42:0/3432305720 >> v2:172.21.15.63:3300/0 conn(0x7f5544145d50 msgr2=0x7f55440cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.551+0000 7f5549a00640 1 --2- 172.21.15.42:0/3432305720 >> v2:172.21.15.63:3300/0 conn(0x7f5544145d50 0x7f55440cfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.551+0000 7f5549a00640 1 -- 172.21.15.42:0/3432305720 >> v1:172.21.15.63:6789/0 conn(0x7f55440cf540 legacy=0x7f55440cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.551+0000 7f5549a00640 1 -- 172.21.15.42:0/3432305720 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55440707e0 con 0x7f55440cd050 2024-05-28T14:03:13.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.551+0000 7f5549a00640 1 --2- 172.21.15.42:0/3432305720 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55440cd050 0x7f55440d0400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f553c009e50 tx=0x7f553c01c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85cf4b8fc1989ac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:13.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.551+0000 7f5543e00640 1 -- 172.21.15.42:0/3432305720 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c0a5550 con 0x7f55440cd050 2024-05-28T14:03:13.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.551+0000 7f5543e00640 1 -- 172.21.15.42:0/3432305720 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553c0a56f0 con 0x7f55440cd050 2024-05-28T14:03:13.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.551+0000 7f5543e00640 1 -- 172.21.15.42:0/3432305720 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c0a5ac0 con 0x7f55440cd050 2024-05-28T14:03:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 -- 172.21.15.42:0/3432305720 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55440cd050 msgr2=0x7f55440d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 --2- 172.21.15.42:0/3432305720 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55440cd050 0x7f55440d0400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f553c009e50 tx=0x7f553c01c9a0 comp rx=0 tx=0).stop 2024-05-28T14:03:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 -- 172.21.15.42:0/3432305720 shutdown_connections 2024-05-28T14:03:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 --2- 172.21.15.42:0/3432305720 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55440cd050 0x7f55440d0400 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 --2- 172.21.15.42:0/3432305720 >> v2:172.21.15.63:3300/0 conn(0x7f5544145d50 0x7f55440cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 -- 172.21.15.42:0/3432305720 >> 172.21.15.42:0/3432305720 conn(0x7f554406fcc0 msgr2=0x7f55440700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 -- 172.21.15.42:0/3432305720 shutdown_connections 2024-05-28T14:03:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 -- 172.21.15.42:0/3432305720 wait complete. 2024-05-28T14:03:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 Processor -- start 2024-05-28T14:03:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 -- start start 2024-05-28T14:03:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55440cd050 0x7f5544153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55440cf540 0x7f5544157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5544145d50 0x7f5544157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55441694d0 con 0x7f55440cd050 2024-05-28T14:03:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5544169350 con 0x7f5544145d50 2024-05-28T14:03:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f554a400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5544169650 con 0x7f55440cf540 2024-05-28T14:03:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f5549000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55440cd050 0x7f5544153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f5549000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55440cd050 0x7f5544153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45812/0 (socket says 172.21.15.42:45812) 2024-05-28T14:03:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f5549000640 1 -- 172.21.15.42:0/2358743792 learned_addr learned my addr 172.21.15.42:0/2358743792 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f553be00640 1 --2- 172.21.15.42:0/2358743792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55440cf540 0x7f5544157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f5549a00640 1 --2- 172.21.15.42:0/2358743792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5544145d50 0x7f5544157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f5549000640 1 -- 172.21.15.42:0/2358743792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55440cf540 msgr2=0x7f5544157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f5549000640 1 --2- 172.21.15.42:0/2358743792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55440cf540 0x7f5544157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f5549000640 1 -- 172.21.15.42:0/2358743792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5544145d50 msgr2=0x7f5544157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f5549000640 1 --2- 172.21.15.42:0/2358743792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5544145d50 0x7f5544157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.555+0000 7f5549000640 1 -- 172.21.15.42:0/2358743792 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55440b3810 con 0x7f55440cd050 2024-05-28T14:03:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.559+0000 7f5549000640 1 --2- 172.21.15.42:0/2358743792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55440cd050 0x7f5544153470 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f553400c450 tx=0x7f553400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.559+0000 7f5541600640 1 -- 172.21.15.42:0/2358743792 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534016020 con 0x7f55440cd050 2024-05-28T14:03:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.559+0000 7f5541600640 1 -- 172.21.15.42:0/2358743792 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55340052d0 con 0x7f55440cd050 2024-05-28T14:03:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.559+0000 7f554a400640 1 -- 172.21.15.42:0/2358743792 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55440b3af0 con 0x7f55440cd050 2024-05-28T14:03:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.559+0000 7f5541600640 1 -- 172.21.15.42:0/2358743792 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534007260 con 0x7f55440cd050 2024-05-28T14:03:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.559+0000 7f554a400640 1 -- 172.21.15.42:0/2358743792 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5544154a10 con 0x7f55440cd050 2024-05-28T14:03:13.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.559+0000 7f554a400640 1 -- 172.21.15.42:0/2358743792 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5544154c10 con 0x7f55440cd050 2024-05-28T14:03:13.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.559+0000 7f5541600640 1 -- 172.21.15.42:0/2358743792 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 11) ==== 88293+0+0 (secure 0 0 0) 0x7f5534007400 con 0x7f55440cd050 2024-05-28T14:03:13.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.559+0000 7f5541600640 1 --2- 172.21.15.42:0/2358743792 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5520068650 0x7f552006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:13.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.559+0000 7f5541600640 1 -- 172.21.15.42:0/2358743792 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f5534086780 con 0x7f55440cd050 2024-05-28T14:03:13.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.559+0000 7f553be00640 1 --2- 172.21.15.42:0/2358743792 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5520068650 0x7f552006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.563+0000 7f553be00640 1 --2- 172.21.15.42:0/2358743792 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5520068650 0x7f552006ab10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f55440d1d10 tx=0x7f552c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:13.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.567+0000 7f5541600640 1 -- 172.21.15.42:0/2358743792 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5544154c10 con 0x7f55440cd050 2024-05-28T14:03:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.763+0000 7f554a400640 1 -- 172.21.15.42:0/2358743792 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone85311", "target": ["mon-mgr", ""]}) -- 0x7f55440c8710 con 0x7f5520068650 2024-05-28T14:03:13.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.831+0000 7f5541600640 1 -- 172.21.15.42:0/2358743792 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f55440c8710 con 0x7f5520068650 2024-05-28T14:03:13.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.831+0000 7f554a400640 1 -- 172.21.15.42:0/2358743792 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5520068650 msgr2=0x7f552006ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:13.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.831+0000 7f554a400640 1 --2- 172.21.15.42:0/2358743792 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5520068650 0x7f552006ab10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f55440d1d10 tx=0x7f552c009190 comp rx=0 tx=0).stop 2024-05-28T14:03:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.831+0000 7f554a400640 1 -- 172.21.15.42:0/2358743792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55440cd050 msgr2=0x7f5544153470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.831+0000 7f554a400640 1 --2- 172.21.15.42:0/2358743792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55440cd050 0x7f5544153470 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f553400c450 tx=0x7f553400c910 comp rx=0 tx=0).stop 2024-05-28T14:03:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.831+0000 7f554a400640 1 -- 172.21.15.42:0/2358743792 shutdown_connections 2024-05-28T14:03:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.831+0000 7f554a400640 1 --2- 172.21.15.42:0/2358743792 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5520068650 0x7f552006ab10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.831+0000 7f554a400640 1 --2- 172.21.15.42:0/2358743792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5544145d50 0x7f5544157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.831+0000 7f554a400640 1 --2- 172.21.15.42:0/2358743792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55440cf540 0x7f5544157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.831+0000 7f554a400640 1 --2- 172.21.15.42:0/2358743792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55440cd050 0x7f5544153470 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.831+0000 7f554a400640 1 -- 172.21.15.42:0/2358743792 >> 172.21.15.42:0/2358743792 conn(0x7f554406fcc0 msgr2=0x7f55440c6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.831+0000 7f554a400640 1 -- 172.21.15.42:0/2358743792 shutdown_connections 2024-05-28T14:03:13.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:13.831+0000 7f554a400640 1 -- 172.21.15.42:0/2358743792 wait complete. 2024-05-28T14:03:13.820 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:13.835+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:13.820 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:13.835+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:13.820 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:13.835+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:13.820 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:13.835+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:13.820 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:13.835+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:13.837 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:13.851+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:13.837 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:13.851+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:13.837 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:13.851+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:13.837 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:13.851+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:13.837 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:13.851+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:13.840 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone85312 2024-05-28T14:03:13.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.007+0000 7f0c23e00640 1 Processor -- start 2024-05-28T14:03:13.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.007+0000 7f0c23e00640 1 -- start start 2024-05-28T14:03:13.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c23e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0c1c09b8e0 0x7f0c1c06cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:13.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c23e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c1c0d0770 con 0x7f0c1c09b160 2024-05-28T14:03:13.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c23e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c1c0d0a10 con 0x7f0c1c0b0e80 2024-05-28T14:03:13.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c23e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1c0d0cf0 con 0x7f0c1c09b8e0 2024-05-28T14:03:13.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c20a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0c1c09b160 0x7f0c1c15e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46320/0 (socket says 172.21.15.42:46320) 2024-05-28T14:03:13.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c20a00640 1 -- 172.21.15.42:0/4052375928 learned_addr learned my addr 172.21.15.42:0/4052375928 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:03:13.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c21e00640 1 --2- 172.21.15.42:0/4052375928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0c1c09b8e0 0x7f0c1c06cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:13.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c1be00640 1 -- 172.21.15.42:0/4052375928 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 541025167 0 0) 0x7f0c1c0d0770 con 0x7f0c1c09b160 2024-05-28T14:03:13.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c1be00640 1 -- 172.21.15.42:0/4052375928 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c04003530 con 0x7f0c1c09b160 2024-05-28T14:03:13.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c1be00640 1 -- 172.21.15.42:0/4052375928 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3494220809 0 0) 0x7f0c1c0d0a10 con 0x7f0c1c0b0e80 2024-05-28T14:03:13.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c1be00640 1 -- 172.21.15.42:0/4052375928 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c1c0d0770 con 0x7f0c1c0b0e80 2024-05-28T14:03:13.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c21e00640 1 -- 172.21.15.42:0/4052375928 >> v1:172.21.15.63:6789/0 conn(0x7f0c1c0b0e80 legacy=0x7f0c1c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:13.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c21e00640 1 -- 172.21.15.42:0/4052375928 >> v1:172.21.15.42:6789/0 conn(0x7f0c1c09b160 legacy=0x7f0c1c15e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:13.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c1be00640 1 -- 172.21.15.42:0/4052375928 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1568790456 0 0) 0x7f0c04003530 con 0x7f0c1c09b160 2024-05-28T14:03:13.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.011+0000 7f0c21e00640 1 -- 172.21.15.42:0/4052375928 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c0d0fd0 con 0x7f0c1c09b8e0 2024-05-28T14:03:14.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c21e00640 1 --2- 172.21.15.42:0/4052375928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0c1c09b8e0 0x7f0c1c06cb00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009dc0 tx=0x7f0c1401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96ac24ff24b394ce server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:14.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c1be00640 1 -- 172.21.15.42:0/4052375928 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08001680 con 0x7f0c1c09b8e0 2024-05-28T14:03:14.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c1be00640 1 -- 172.21.15.42:0/4052375928 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c140a6500 con 0x7f0c1c09b8e0 2024-05-28T14:03:14.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c1be00640 1 -- 172.21.15.42:0/4052375928 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c140a68d0 con 0x7f0c1c09b8e0 2024-05-28T14:03:14.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c23e00640 1 -- 172.21.15.42:0/4052375928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0c1c09b8e0 msgr2=0x7f0c1c06cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:14.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c23e00640 1 --2- 172.21.15.42:0/4052375928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0c1c09b8e0 0x7f0c1c06cb00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009dc0 tx=0x7f0c1401c8b0 comp rx=0 tx=0).stop 2024-05-28T14:03:14.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c23e00640 1 -- 172.21.15.42:0/4052375928 shutdown_connections 2024-05-28T14:03:14.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c23e00640 1 --2- 172.21.15.42:0/4052375928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0c1c09b8e0 0x7f0c1c06cb00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:14.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c23e00640 1 -- 172.21.15.42:0/4052375928 >> 172.21.15.42:0/4052375928 conn(0x7f0c1c0b3f90 msgr2=0x7f0c1c0b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:14.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c23e00640 1 -- 172.21.15.42:0/4052375928 shutdown_connections 2024-05-28T14:03:14.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c23e00640 1 -- 172.21.15.42:0/4052375928 wait complete. 2024-05-28T14:03:14.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c23e00640 1 Processor -- start 2024-05-28T14:03:14.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c23e00640 1 -- start start 2024-05-28T14:03:14.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c23e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0c1c09b160 0x7f0c1c0c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:14.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c23e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c1c09b8e0 0x7f0c1c09ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:14.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.015+0000 7f0c23e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c1c0b0e80 0x7f0c1c15c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c23e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c15ec30 con 0x7f0c1c0b0e80 2024-05-28T14:03:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c23e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c0d1a90 con 0x7f0c1c09b8e0 2024-05-28T14:03:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c23e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1c16cf90 con 0x7f0c1c09b160 2024-05-28T14:03:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c21e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c1c0b0e80 0x7f0c1c15c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c21e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c1c0b0e80 0x7f0c1c15c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45868/0 (socket says 172.21.15.42:45868) 2024-05-28T14:03:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c21e00640 1 -- 172.21.15.42:0/1475308803 learned_addr learned my addr 172.21.15.42:0/1475308803 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c21400640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0c1c09b160 0x7f0c1c0c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c20a00640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c1c09b8e0 0x7f0c1c09ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c21e00640 1 -- 172.21.15.42:0/1475308803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0c1c09b160 msgr2=0x7f0c1c0c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c21e00640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0c1c09b160 0x7f0c1c0c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c21e00640 1 -- 172.21.15.42:0/1475308803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c1c09b8e0 msgr2=0x7f0c1c09ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c21e00640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c1c09b8e0 0x7f0c1c09ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c21e00640 1 -- 172.21.15.42:0/1475308803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c09f5b0 con 0x7f0c1c0b0e80 2024-05-28T14:03:14.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c20a00640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c1c09b8e0 0x7f0c1c09ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:03:14.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c21e00640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c1c0b0e80 0x7f0c1c15c350 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0c1401cd90 tx=0x7f0c14019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:14.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c19600640 1 -- 172.21.15.42:0/1475308803 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14009450 con 0x7f0c1c0b0e80 2024-05-28T14:03:14.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c19600640 1 -- 172.21.15.42:0/1475308803 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c140c0070 con 0x7f0c1c0b0e80 2024-05-28T14:03:14.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c19600640 1 -- 172.21.15.42:0/1475308803 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c140ba910 con 0x7f0c1c0b0e80 2024-05-28T14:03:14.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c23e00640 1 -- 172.21.15.42:0/1475308803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1c09f890 con 0x7f0c1c0b0e80 2024-05-28T14:03:14.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c23e00640 1 -- 172.21.15.42:0/1475308803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1c15be30 con 0x7f0c1c0b0e80 2024-05-28T14:03:14.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c23e00640 1 -- 172.21.15.42:0/1475308803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1c0a0550 con 0x7f0c1c0b0e80 2024-05-28T14:03:14.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.019+0000 7f0c19600640 1 -- 172.21.15.42:0/1475308803 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 11) ==== 88293+0+0 (secure 0 0 0) 0x7f0c140baab0 con 0x7f0c1c0b0e80 2024-05-28T14:03:14.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.023+0000 7f0c19600640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0bfc0686e0 0x7f0bfc06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:14.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.023+0000 7f0c19600640 1 -- 172.21.15.42:0/1475308803 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f0c14122470 con 0x7f0c1c0b0e80 2024-05-28T14:03:14.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.023+0000 7f0c21400640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0bfc0686e0 0x7f0bfc06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:14.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.023+0000 7f0c21400640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0bfc0686e0 0x7f0bfc06aba0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0c100011f0 tx=0x7f0c10009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:14.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.027+0000 7f0c19600640 1 -- 172.21.15.42:0/1475308803 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0c140164a0 con 0x7f0c1c0b0e80 2024-05-28T14:03:14.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.223+0000 7f0c23e00640 1 -- 172.21.15.42:0/1475308803 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone85312", "target": ["mon-mgr", ""]}) -- 0x7f0c1c16a550 con 0x7f0bfc0686e0 2024-05-28T14:03:14.820 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:14.835+0000 7f076a600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:14.821 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:14.835+0000 7f076a600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:14.821 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:14.835+0000 7f076a600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:14.821 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:14.835+0000 7f076a600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:14.821 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:14.835+0000 7f076a600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:14.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.943+0000 7f0c19600640 1 -- 172.21.15.42:0/1475308803 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0c1c16a550 con 0x7f0bfc0686e0 2024-05-28T14:03:14.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.947+0000 7f0c23e00640 1 -- 172.21.15.42:0/1475308803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0bfc0686e0 msgr2=0x7f0bfc06aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:14.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.947+0000 7f0c23e00640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0bfc0686e0 0x7f0bfc06aba0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0c100011f0 tx=0x7f0c10009040 comp rx=0 tx=0).stop 2024-05-28T14:03:14.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.947+0000 7f0c23e00640 1 -- 172.21.15.42:0/1475308803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c1c0b0e80 msgr2=0x7f0c1c15c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:14.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.947+0000 7f0c23e00640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c1c0b0e80 0x7f0c1c15c350 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0c1401cd90 tx=0x7f0c14019650 comp rx=0 tx=0).stop 2024-05-28T14:03:14.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.947+0000 7f0c23e00640 1 -- 172.21.15.42:0/1475308803 shutdown_connections 2024-05-28T14:03:14.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.947+0000 7f0c23e00640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0bfc0686e0 0x7f0bfc06aba0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:14.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.947+0000 7f0c23e00640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c1c0b0e80 0x7f0c1c15c350 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:14.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.947+0000 7f0c23e00640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c1c09b8e0 0x7f0c1c09ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:14.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.947+0000 7f0c23e00640 1 --2- 172.21.15.42:0/1475308803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0c1c09b160 0x7f0c1c0c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:14.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.947+0000 7f0c23e00640 1 -- 172.21.15.42:0/1475308803 >> 172.21.15.42:0/1475308803 conn(0x7f0c1c0b3f90 msgr2=0x7f0c1c160dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:14.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.947+0000 7f0c23e00640 1 -- 172.21.15.42:0/1475308803 shutdown_connections 2024-05-28T14:03:14.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:14.951+0000 7f0c23e00640 1 -- 172.21.15.42:0/1475308803 wait complete. 2024-05-28T14:03:14.953 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:14.967+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:14.953 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:14.967+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:14.953 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:14.967+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:14.954 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:14.967+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:14.954 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:14.967+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:14.996 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone85313 2024-05-28T14:03:15.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.307+0000 7fa086600640 1 Processor -- start 2024-05-28T14:03:15.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.307+0000 7fa086600640 1 -- start start 2024-05-28T14:03:15.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa086600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa080162fe0 0x7fa0800cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:15.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa086600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa0800b04a0 0x7fa0800b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:15.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa086600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0800b25c0 con 0x7fa08006b9f0 2024-05-28T14:03:15.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa086600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa0800b2e60 con 0x7fa0800b04a0 2024-05-28T14:03:15.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa086600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa0800b1090 con 0x7fa080162fe0 2024-05-28T14:03:15.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa07fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa08006b9f0 0x7fa0800b1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:46332/0 (socket says 172.21.15.42:46332) 2024-05-28T14:03:15.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa07fe00640 1 -- 172.21.15.42:0/2642772939 learned_addr learned my addr 172.21.15.42:0/2642772939 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:03:15.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa076a00640 1 --2- 172.21.15.42:0/2642772939 >> v2:172.21.15.42:3300/0 conn(0x7fa0800b04a0 0x7fa0800b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:15.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa07f400640 1 --2- 172.21.15.42:0/2642772939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa080162fe0 0x7fa0800cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa07ea00640 1 -- 172.21.15.42:0/2642772939 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 365772863 0 0) 0x7fa0800b25c0 con 0x7fa08006b9f0 2024-05-28T14:03:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa07ea00640 1 -- 172.21.15.42:0/2642772939 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa064003530 con 0x7fa08006b9f0 2024-05-28T14:03:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa076a00640 1 -- 172.21.15.42:0/2642772939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa080162fe0 msgr2=0x7fa0800cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa076a00640 1 --2- 172.21.15.42:0/2642772939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa080162fe0 0x7fa0800cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa07ea00640 1 -- 172.21.15.42:0/2642772939 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1123575994 0 0) 0x7fa064003530 con 0x7fa08006b9f0 2024-05-28T14:03:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa076a00640 1 -- 172.21.15.42:0/2642772939 >> v1:172.21.15.42:6789/0 conn(0x7fa08006b9f0 legacy=0x7fa0800b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.311+0000 7fa076a00640 1 -- 172.21.15.42:0/2642772939 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0800b28a0 con 0x7fa0800b04a0 2024-05-28T14:03:15.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.323+0000 7fa076a00640 1 --2- 172.21.15.42:0/2642772939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0800b04a0 0x7fa0800b0920 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa0680097c0 tx=0x7fa06801c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6428fbd5f02efd server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:15.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.323+0000 7fa07ea00640 1 -- 172.21.15.42:0/2642772939 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0680b0070 con 0x7fa0800b04a0 2024-05-28T14:03:15.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.323+0000 7fa07ea00640 1 -- 172.21.15.42:0/2642772939 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0680ab600 con 0x7fa0800b04a0 2024-05-28T14:03:15.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.323+0000 7fa07ea00640 1 -- 172.21.15.42:0/2642772939 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0680ab9d0 con 0x7fa0800b04a0 2024-05-28T14:03:15.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.327+0000 7fa086600640 1 -- 172.21.15.42:0/2642772939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0800b04a0 msgr2=0x7fa0800b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:15.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.327+0000 7fa086600640 1 --2- 172.21.15.42:0/2642772939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0800b04a0 0x7fa0800b0920 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa0680097c0 tx=0x7fa06801c350 comp rx=0 tx=0).stop 2024-05-28T14:03:15.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.327+0000 7fa086600640 1 -- 172.21.15.42:0/2642772939 shutdown_connections 2024-05-28T14:03:15.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.327+0000 7fa086600640 1 --2- 172.21.15.42:0/2642772939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0800b04a0 0x7fa0800b0920 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:15.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.327+0000 7fa086600640 1 --2- 172.21.15.42:0/2642772939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa080162fe0 0x7fa0800cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:15.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.327+0000 7fa086600640 1 -- 172.21.15.42:0/2642772939 >> 172.21.15.42:0/2642772939 conn(0x7fa08009aa50 msgr2=0x7fa0800b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.331+0000 7fa086600640 1 -- 172.21.15.42:0/2642772939 shutdown_connections 2024-05-28T14:03:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.331+0000 7fa086600640 1 -- 172.21.15.42:0/2642772939 wait complete. 2024-05-28T14:03:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.335+0000 7fa086600640 1 Processor -- start 2024-05-28T14:03:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.335+0000 7fa086600640 1 -- start start 2024-05-28T14:03:15.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa086600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa08006b9f0 0x7fa0800cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:15.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa086600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0800b04a0 0x7fa080143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:15.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa086600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa080162fe0 0x7fa0801564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:15.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa086600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa0800b3f70 con 0x7fa080162fe0 2024-05-28T14:03:15.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa086600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0800b3df0 con 0x7fa08006b9f0 2024-05-28T14:03:15.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa086600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa08016fb70 con 0x7fa0800b04a0 2024-05-28T14:03:15.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa07fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa080162fe0 0x7fa0801564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:15.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa07fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa080162fe0 0x7fa0801564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45908/0 (socket says 172.21.15.42:45908) 2024-05-28T14:03:15.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa07fe00640 1 -- 172.21.15.42:0/3805711767 learned_addr learned my addr 172.21.15.42:0/3805711767 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:15.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa076a00640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0800b04a0 0x7fa080143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:15.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa07f400640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa08006b9f0 0x7fa0800cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:15.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa07fe00640 1 -- 172.21.15.42:0/3805711767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0800b04a0 msgr2=0x7fa080143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:15.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa07fe00640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0800b04a0 0x7fa080143fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:15.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa07fe00640 1 -- 172.21.15.42:0/3805711767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa08006b9f0 msgr2=0x7fa0800cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:15.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa07fe00640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa08006b9f0 0x7fa0800cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:15.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.339+0000 7fa07fe00640 1 -- 172.21.15.42:0/3805711767 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0800cdec0 con 0x7fa080162fe0 2024-05-28T14:03:15.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.343+0000 7fa076a00640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0800b04a0 0x7fa080143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:03:15.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.343+0000 7fa07f400640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa08006b9f0 0x7fa0800cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:03:15.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.343+0000 7fa07fe00640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa080162fe0 0x7fa0801564b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa07800c410 tx=0x7fa078001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:15.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.343+0000 7fa077e00640 1 -- 172.21.15.42:0/3805711767 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078017070 con 0x7fa080162fe0 2024-05-28T14:03:15.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.343+0000 7fa077e00640 1 -- 172.21.15.42:0/3805711767 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa078005d10 con 0x7fa080162fe0 2024-05-28T14:03:15.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.343+0000 7fa077e00640 1 -- 172.21.15.42:0/3805711767 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078007310 con 0x7fa080162fe0 2024-05-28T14:03:15.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.343+0000 7fa086600640 1 -- 172.21.15.42:0/3805711767 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0800cb3c0 con 0x7fa080162fe0 2024-05-28T14:03:15.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.343+0000 7fa086600640 1 -- 172.21.15.42:0/3805711767 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0801559d0 con 0x7fa080162fe0 2024-05-28T14:03:15.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.347+0000 7fa086600640 1 -- 172.21.15.42:0/3805711767 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08009f650 con 0x7fa080162fe0 2024-05-28T14:03:15.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.347+0000 7fa077e00640 1 -- 172.21.15.42:0/3805711767 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 11) ==== 88293+0+0 (secure 0 0 0) 0x7fa07801a070 con 0x7fa080162fe0 2024-05-28T14:03:15.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.347+0000 7fa077e00640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa05c068730 0x7fa05c06abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:15.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.347+0000 7fa077e00640 1 -- 172.21.15.42:0/3805711767 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fa078086220 con 0x7fa080162fe0 2024-05-28T14:03:15.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.347+0000 7fa07f400640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa05c068730 0x7fa05c06abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:15.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.351+0000 7fa07f400640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa05c068730 0x7fa05c06abf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa070009fd0 tx=0x7fa07000b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.355+0000 7fa077e00640 1 -- 172.21.15.42:0/3805711767 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa078056500 con 0x7fa080162fe0 2024-05-28T14:03:15.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.507+0000 7fa077e00640 1 -- 172.21.15.42:0/3805711767 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 12) ==== 88446+0+0 (secure 0 0 0) 0x7fa078052470 con 0x7fa080162fe0 2024-05-28T14:03:15.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:15.783+0000 7fa086600640 1 -- 172.21.15.42:0/3805711767 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone85313", "target": ["mon-mgr", ""]}) -- 0x7fa08006fe00 con 0x7fa05c068730 2024-05-28T14:03:16.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.447+0000 7fa077e00640 1 -- 172.21.15.42:0/3805711767 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa08006fe00 con 0x7fa05c068730 2024-05-28T14:03:16.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.451+0000 7fa086600640 1 -- 172.21.15.42:0/3805711767 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa05c068730 msgr2=0x7fa05c06abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:16.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.451+0000 7fa086600640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa05c068730 0x7fa05c06abf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa070009fd0 tx=0x7fa07000b040 comp rx=0 tx=0).stop 2024-05-28T14:03:16.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.451+0000 7fa086600640 1 -- 172.21.15.42:0/3805711767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa080162fe0 msgr2=0x7fa0801564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:16.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.451+0000 7fa086600640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa080162fe0 0x7fa0801564b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa07800c410 tx=0x7fa078001e80 comp rx=0 tx=0).stop 2024-05-28T14:03:16.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.451+0000 7fa086600640 1 -- 172.21.15.42:0/3805711767 shutdown_connections 2024-05-28T14:03:16.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.451+0000 7fa086600640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa05c068730 0x7fa05c06abf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:16.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.451+0000 7fa086600640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa080162fe0 0x7fa0801564b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:16.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.451+0000 7fa086600640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0800b04a0 0x7fa080143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:16.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.451+0000 7fa086600640 1 --2- 172.21.15.42:0/3805711767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa08006b9f0 0x7fa0800cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:16.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.451+0000 7fa086600640 1 -- 172.21.15.42:0/3805711767 >> 172.21.15.42:0/3805711767 conn(0x7fa08009aa50 msgr2=0x7fa080159fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:16.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.455+0000 7fa086600640 1 -- 172.21.15.42:0/3805711767 shutdown_connections 2024-05-28T14:03:16.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.455+0000 7fa086600640 1 -- 172.21.15.42:0/3805711767 wait complete. 2024-05-28T14:03:16.519 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone85314 2024-05-28T14:03:16.525 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:16.535+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:16.525 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:16.535+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:16.525 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:16.535+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:16.525 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:16.535+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:16.525 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:16.535+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:16.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.923+0000 7fb650c00640 1 Processor -- start 2024-05-28T14:03:16.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.923+0000 7fb650c00640 1 -- start start 2024-05-28T14:03:16.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.923+0000 7fb650c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb644050dd0 0x7fb6440398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:16.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.923+0000 7fb650c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb6440e0490 0x7fb64403a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:16.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.923+0000 7fb650c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb644039f60 con 0x7fb6440e00b0 2024-05-28T14:03:16.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.923+0000 7fb650c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb64403ac50 con 0x7fb644050dd0 2024-05-28T14:03:16.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.923+0000 7fb650c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb644053060 con 0x7fb6440e0490 2024-05-28T14:03:16.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.927+0000 7fb64aa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb644050dd0 0x7fb6440398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:16.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.927+0000 7fb64aa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb644050dd0 0x7fb6440398f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45944/0 (socket says 172.21.15.42:45944) 2024-05-28T14:03:16.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.927+0000 7fb64aa00640 1 -- 172.21.15.42:0/95118122 learned_addr learned my addr 172.21.15.42:0/95118122 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:16.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.927+0000 7fb652200640 1 --2- 172.21.15.42:0/95118122 >> v2:172.21.15.63:3300/0 conn(0x7fb6440e0490 0x7fb64403a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:16.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.927+0000 7fb64aa00640 1 -- 172.21.15.42:0/95118122 >> v2:172.21.15.63:3300/0 conn(0x7fb6440e0490 msgr2=0x7fb64403a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:16.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.927+0000 7fb64aa00640 1 --2- 172.21.15.42:0/95118122 >> v2:172.21.15.63:3300/0 conn(0x7fb6440e0490 0x7fb64403a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:16.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.927+0000 7fb64aa00640 1 -- 172.21.15.42:0/95118122 >> v1:172.21.15.63:6789/0 conn(0x7fb6440e00b0 legacy=0x7fb6440506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:16.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.927+0000 7fb64aa00640 1 -- 172.21.15.42:0/95118122 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb644052a10 con 0x7fb644050dd0 2024-05-28T14:03:16.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.935+0000 7fb64aa00640 1 --2- 172.21.15.42:0/95118122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb644050dd0 0x7fb6440398f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb6440487a0 tx=0x7fb63801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e036e29c65d70d37 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:16.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.935+0000 7fb64a000640 1 -- 172.21.15.42:0/95118122 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6380b0070 con 0x7fb644050dd0 2024-05-28T14:03:16.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.935+0000 7fb64a000640 1 -- 172.21.15.42:0/95118122 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6380ab600 con 0x7fb644050dd0 2024-05-28T14:03:16.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.935+0000 7fb64a000640 1 -- 172.21.15.42:0/95118122 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6380ab9d0 con 0x7fb644050dd0 2024-05-28T14:03:16.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.939+0000 7fb650c00640 1 -- 172.21.15.42:0/95118122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb644050dd0 msgr2=0x7fb6440398f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:16.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.939+0000 7fb650c00640 1 --2- 172.21.15.42:0/95118122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb644050dd0 0x7fb6440398f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb6440487a0 tx=0x7fb63801c3c0 comp rx=0 tx=0).stop 2024-05-28T14:03:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.939+0000 7fb650c00640 1 -- 172.21.15.42:0/95118122 shutdown_connections 2024-05-28T14:03:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.939+0000 7fb650c00640 1 --2- 172.21.15.42:0/95118122 >> v2:172.21.15.63:3300/0 conn(0x7fb6440e0490 0x7fb64403a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.939+0000 7fb650c00640 1 --2- 172.21.15.42:0/95118122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb644050dd0 0x7fb6440398f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.939+0000 7fb650c00640 1 -- 172.21.15.42:0/95118122 >> 172.21.15.42:0/95118122 conn(0x7fb644054ca0 msgr2=0x7fb6440550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.939+0000 7fb650c00640 1 -- 172.21.15.42:0/95118122 shutdown_connections 2024-05-28T14:03:16.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.943+0000 7fb650c00640 1 -- 172.21.15.42:0/95118122 wait complete. 2024-05-28T14:03:16.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.947+0000 7fb650c00640 1 Processor -- start 2024-05-28T14:03:16.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.947+0000 7fb650c00640 1 -- start start 2024-05-28T14:03:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.951+0000 7fb650c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb644050dd0 0x7fb64404a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.955+0000 7fb650c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6440e00b0 0x7fb64402d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.955+0000 7fb650c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6440e0490 0x7fb64402d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.955+0000 7fb650c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb644007570 con 0x7fb6440e00b0 2024-05-28T14:03:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.955+0000 7fb650c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb6440073f0 con 0x7fb6440e0490 2024-05-28T14:03:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.955+0000 7fb650c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb6440076f0 con 0x7fb644050dd0 2024-05-28T14:03:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.955+0000 7fb64b400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb644050dd0 0x7fb64404a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.955+0000 7fb64b400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb644050dd0 0x7fb64404a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:48908/0 (socket says 172.21.15.42:48908) 2024-05-28T14:03:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.955+0000 7fb64b400640 1 -- 172.21.15.42:0/1522964988 learned_addr learned my addr 172.21.15.42:0/1522964988 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.955+0000 7fb652200640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6440e0490 0x7fb64402d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:16.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.955+0000 7fb64aa00640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6440e00b0 0x7fb64402d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:16.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.955+0000 7fb64b400640 1 -- 172.21.15.42:0/1522964988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6440e0490 msgr2=0x7fb64402d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:16.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.955+0000 7fb64b400640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6440e0490 0x7fb64402d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:16.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.959+0000 7fb64b400640 1 -- 172.21.15.42:0/1522964988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6440e00b0 msgr2=0x7fb64402d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:16.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.959+0000 7fb64b400640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6440e00b0 0x7fb64402d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:16.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.959+0000 7fb64b400640 1 -- 172.21.15.42:0/1522964988 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64404ad90 con 0x7fb644050dd0 2024-05-28T14:03:16.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.959+0000 7fb652200640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6440e0490 0x7fb64402d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:03:16.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.959+0000 7fb64b400640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb644050dd0 0x7fb64404a850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb64000b410 tx=0x7fb64000b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:16.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.959+0000 7fb63f400640 1 -- 172.21.15.42:0/1522964988 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640015030 con 0x7fb644050dd0 2024-05-28T14:03:16.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.959+0000 7fb63f400640 1 -- 172.21.15.42:0/1522964988 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb640004020 con 0x7fb644050dd0 2024-05-28T14:03:16.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.959+0000 7fb63f400640 1 -- 172.21.15.42:0/1522964988 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640004450 con 0x7fb644050dd0 2024-05-28T14:03:16.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.963+0000 7fb650c00640 1 -- 172.21.15.42:0/1522964988 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6440f0970 con 0x7fb644050dd0 2024-05-28T14:03:16.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.963+0000 7fb650c00640 1 -- 172.21.15.42:0/1522964988 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6440eea80 con 0x7fb644050dd0 2024-05-28T14:03:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.963+0000 7fb650c00640 1 -- 172.21.15.42:0/1522964988 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6440054a0 con 0x7fb644050dd0 2024-05-28T14:03:16.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.979+0000 7fb63f400640 1 -- 172.21.15.42:0/1522964988 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 12) ==== 88446+0+0 (secure 0 0 0) 0x7fb640018070 con 0x7fb644050dd0 2024-05-28T14:03:16.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.979+0000 7fb63f400640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb62c068910 0x7fb62c06add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:16.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.983+0000 7fb64aa00640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb62c068910 0x7fb62c06add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:16.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.987+0000 7fb63f400640 1 -- 172.21.15.42:0/1522964988 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fb640054600 con 0x7fb644050dd0 2024-05-28T14:03:16.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:16.995+0000 7fb63f400640 1 -- 172.21.15.42:0/1522964988 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb640056c60 con 0x7fb644050dd0 2024-05-28T14:03:16.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:17.003+0000 7fb64aa00640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb62c068910 0x7fb62c06add0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb6380ad000 tx=0x7fb638008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:17.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:17.523+0000 7fb650c00640 1 -- 172.21.15.42:0/1522964988 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone85314", "target": ["mon-mgr", ""]}) -- 0x7fb644037b80 con 0x7fb62c068910 2024-05-28T14:03:17.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:17.547+0000 7fb63f400640 1 -- 172.21.15.42:0/1522964988 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mgrmap(e 13) ==== 88497+0+0 (secure 0 0 0) 0x7fb640018070 con 0x7fb644050dd0 2024-05-28T14:03:18.208 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:03:18.220 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:03:18.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.303+0000 7fb63f400640 1 -- 172.21.15.42:0/1522964988 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb644037b80 con 0x7fb62c068910 2024-05-28T14:03:18.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.307+0000 7fb650c00640 1 -- 172.21.15.42:0/1522964988 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb62c068910 msgr2=0x7fb62c06add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:18.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.307+0000 7fb650c00640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb62c068910 0x7fb62c06add0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb6380ad000 tx=0x7fb638008040 comp rx=0 tx=0).stop 2024-05-28T14:03:18.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.307+0000 7fb650c00640 1 -- 172.21.15.42:0/1522964988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb644050dd0 msgr2=0x7fb64404a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:18.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.307+0000 7fb650c00640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb644050dd0 0x7fb64404a850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb64000b410 tx=0x7fb64000b8d0 comp rx=0 tx=0).stop 2024-05-28T14:03:18.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.307+0000 7fb650c00640 1 -- 172.21.15.42:0/1522964988 shutdown_connections 2024-05-28T14:03:18.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.307+0000 7fb650c00640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb62c068910 0x7fb62c06add0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:18.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.307+0000 7fb650c00640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6440e0490 0x7fb64402d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:18.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.307+0000 7fb650c00640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6440e00b0 0x7fb64402d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:18.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.307+0000 7fb650c00640 1 --2- 172.21.15.42:0/1522964988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb644050dd0 0x7fb64404a850 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:18.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.307+0000 7fb650c00640 1 -- 172.21.15.42:0/1522964988 >> 172.21.15.42:0/1522964988 conn(0x7fb644054ca0 msgr2=0x7fb64404e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:18.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.307+0000 7fb650c00640 1 -- 172.21.15.42:0/1522964988 shutdown_connections 2024-05-28T14:03:18.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.307+0000 7fb650c00640 1 -- 172.21.15.42:0/1522964988 wait complete. 2024-05-28T14:03:18.383 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:03:18.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.767+0000 7f5e06400640 1 Processor -- start 2024-05-28T14:03:18.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.767+0000 7f5e06400640 1 -- start start 2024-05-28T14:03:18.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.767+0000 7f5e06400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000d0760 0x7f5e00160e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:18.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.767+0000 7f5e06400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5e000d0b40 0x7f5e0015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:18.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.771+0000 7f5e06400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e001603a0 con 0x7f5e0006c430 2024-05-28T14:03:18.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.771+0000 7f5e06400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5e000cd760 con 0x7f5e000d0b40 2024-05-28T14:03:18.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.771+0000 7f5e06400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5e00160640 con 0x7f5e000d0760 2024-05-28T14:03:18.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.771+0000 7f5e05000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000d0760 0x7f5e00160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:18.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.771+0000 7f5e05000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000d0760 0x7f5e00160e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:39476/0 (socket says 172.21.15.42:39476) 2024-05-28T14:03:18.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.771+0000 7f5e05000640 1 -- 172.21.15.42:0/4101661872 learned_addr learned my addr 172.21.15.42:0/4101661872 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:18.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.771+0000 7f5e05a00640 1 --2- 172.21.15.42:0/4101661872 >> v2:172.21.15.63:3300/0 conn(0x7f5e000d0b40 0x7f5e0015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:18.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.771+0000 7f5e05a00640 1 -- 172.21.15.42:0/4101661872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000d0760 msgr2=0x7f5e00160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:18.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.771+0000 7f5dff400640 1 -- 172.21.15.42:0/4101661872 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 492921759 0 0) 0x7f5e001603a0 con 0x7f5e0006c430 2024-05-28T14:03:18.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.771+0000 7f5e05a00640 1 --2- 172.21.15.42:0/4101661872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000d0760 0x7f5e00160e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:18.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.771+0000 7f5e05a00640 1 -- 172.21.15.42:0/4101661872 >> v1:172.21.15.63:6789/0 conn(0x7f5e0006c430 legacy=0x7f5e000ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:18.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.771+0000 7f5e05a00640 1 -- 172.21.15.42:0/4101661872 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e00160920 con 0x7f5e000d0b40 2024-05-28T14:03:18.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.775+0000 7f5e05a00640 1 --2- 172.21.15.42:0/4101661872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e000d0b40 0x7f5e0015c1e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009d90 tx=0x7f5df801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2973fffe57af252 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:18.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.775+0000 7f5dff400640 1 -- 172.21.15.42:0/4101661872 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80b2070 con 0x7f5e000d0b40 2024-05-28T14:03:18.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.775+0000 7f5dff400640 1 -- 172.21.15.42:0/4101661872 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df80a5a10 con 0x7f5e000d0b40 2024-05-28T14:03:18.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.775+0000 7f5dff400640 1 -- 172.21.15.42:0/4101661872 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80b2070 con 0x7f5e000d0b40 2024-05-28T14:03:18.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.779+0000 7f5e06400640 1 -- 172.21.15.42:0/4101661872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e000d0b40 msgr2=0x7f5e0015c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:18.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.779+0000 7f5e06400640 1 --2- 172.21.15.42:0/4101661872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e000d0b40 0x7f5e0015c1e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009d90 tx=0x7f5df801c970 comp rx=0 tx=0).stop 2024-05-28T14:03:18.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.779+0000 7f5e06400640 1 -- 172.21.15.42:0/4101661872 shutdown_connections 2024-05-28T14:03:18.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.779+0000 7f5e06400640 1 --2- 172.21.15.42:0/4101661872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e000d0b40 0x7f5e0015c1e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:18.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.779+0000 7f5e06400640 1 --2- 172.21.15.42:0/4101661872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000d0760 0x7f5e00160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:18.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.779+0000 7f5e06400640 1 -- 172.21.15.42:0/4101661872 >> 172.21.15.42:0/4101661872 conn(0x7f5e0009aa50 msgr2=0x7f5e000b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:18.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.779+0000 7f5e06400640 1 -- 172.21.15.42:0/4101661872 shutdown_connections 2024-05-28T14:03:18.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.779+0000 7f5e06400640 1 -- 172.21.15.42:0/4101661872 wait complete. 2024-05-28T14:03:18.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.783+0000 7f5e06400640 1 Processor -- start 2024-05-28T14:03:18.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.783+0000 7f5e06400640 1 -- start start 2024-05-28T14:03:18.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.783+0000 7f5e06400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0006c430 0x7f5e000b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.787+0000 7f5e06400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e000d0760 0x7f5e000b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.787+0000 7f5e06400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000d0b40 0x7f5e000c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.787+0000 7f5e06400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e000cf820 con 0x7f5e0006c430 2024-05-28T14:03:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.787+0000 7f5e06400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e000cf6a0 con 0x7f5e000d0760 2024-05-28T14:03:18.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.787+0000 7f5e06400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5e000cf9a0 con 0x7f5e000d0b40 2024-05-28T14:03:18.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.787+0000 7f5dffe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e000d0760 0x7f5e000b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:18.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.787+0000 7f5dffe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e000d0760 0x7f5e000b68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:41176/0 (socket says 172.21.15.42:41176) 2024-05-28T14:03:18.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.787+0000 7f5dffe00640 1 -- 172.21.15.42:0/2042384931 learned_addr learned my addr 172.21.15.42:0/2042384931 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:18.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.787+0000 7f5e05a00640 1 --2- 172.21.15.42:0/2042384931 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000d0b40 0x7f5e000c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:18.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.787+0000 7f5dffe00640 1 -- 172.21.15.42:0/2042384931 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000d0b40 msgr2=0x7f5e000c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:18.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.787+0000 7f5dffe00640 1 --2- 172.21.15.42:0/2042384931 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000d0b40 0x7f5e000c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:18.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.787+0000 7f5dffe00640 1 -- 172.21.15.42:0/2042384931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0006c430 msgr2=0x7f5e000b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:18.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.787+0000 7f5dffe00640 1 --2- 172.21.15.42:0/2042384931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0006c430 0x7f5e000b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:18.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.787+0000 7f5dffe00640 1 -- 172.21.15.42:0/2042384931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0006e730 con 0x7f5e000d0760 2024-05-28T14:03:18.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.791+0000 7f5dffe00640 1 --2- 172.21.15.42:0/2042384931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e000d0760 0x7f5e000b68f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5df000c3f0 tx=0x7f5df000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:18.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.795+0000 7f5dfcc00640 1 -- 172.21.15.42:0/2042384931 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0017070 con 0x7f5e000d0760 2024-05-28T14:03:18.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.795+0000 7f5dfcc00640 1 -- 172.21.15.42:0/2042384931 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df00052f0 con 0x7f5e000d0760 2024-05-28T14:03:18.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.795+0000 7f5dfcc00640 1 -- 172.21.15.42:0/2042384931 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0003860 con 0x7f5e000d0760 2024-05-28T14:03:18.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.795+0000 7f5e06400640 1 -- 172.21.15.42:0/2042384931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e00068270 con 0x7f5e000d0760 2024-05-28T14:03:18.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.795+0000 7f5e06400640 1 -- 172.21.15.42:0/2042384931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e000b6e30 con 0x7f5e000d0760 2024-05-28T14:03:18.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.799+0000 7f5dfcc00640 1 -- 172.21.15.42:0/2042384931 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 88497+0+0 (secure 0 0 0) 0x7f5df001a070 con 0x7f5e000d0760 2024-05-28T14:03:18.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.803+0000 7f5e06400640 1 -- 172.21.15.42:0/2042384931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e00068a90 con 0x7f5e000d0760 2024-05-28T14:03:18.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.803+0000 7f5dfcc00640 1 --2- 172.21.15.42:0/2042384931 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ddc068970 0x7f5ddc06ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:18.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.803+0000 7f5dfcc00640 1 -- 172.21.15.42:0/2042384931 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f5df0086660 con 0x7f5e000d0760 2024-05-28T14:03:18.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.811+0000 7f5dfcc00640 1 -- 172.21.15.42:0/2042384931 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5df0055e90 con 0x7f5e000d0760 2024-05-28T14:03:18.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.887+0000 7f5e05000640 1 --2- 172.21.15.42:0/2042384931 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ddc068970 0x7f5ddc06ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:18.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:18.895+0000 7f5e05000640 1 --2- 172.21.15.42:0/2042384931 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ddc068970 0x7f5ddc06ae30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5de80094a0 tx=0x7f5de80093f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:19.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.327+0000 7f5e06400640 1 -- 172.21.15.42:0/2042384931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5e00067b30 con 0x7f5e000d0760 2024-05-28T14:03:19.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.327+0000 7f5dfcc00640 1 -- 172.21.15.42:0/2042384931 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2097 (secure 0 0 0) 0x7f5df005a190 con 0x7f5e000d0760 2024-05-28T14:03:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:03:19.315 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:03:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:03:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:03:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:03:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:03:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:03:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:03:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:03:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:03:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:03:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:03:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 320, 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 48, 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:03:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:03:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 2588, 2024-05-28T14:03:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 10738627745, 2024-05-28T14:03:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 21792587776, 2024-05-28T14:03:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 751301525504, 2024-05-28T14:03:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:03:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 170, 2024-05-28T14:03:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 78281842, 2024-05-28T14:03:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:03:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 21 2024-05-28T14:03:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:03:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 28, 2024-05-28T14:03:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3, 2024-05-28T14:03:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 3, 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 55184 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:03:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 10, 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:02:58.913274+0000", 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:03:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "7d0de869-e5f8-45bf-a6c9-cad4b1fd3c49": { 2024-05-28T14:03:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "2 ongoing clones; Avg progress is 0.0% (1s)\n [............................] ", 2024-05-28T14:03:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0, 2024-05-28T14:03:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:03:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:19.321 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:03:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.331+0000 7f5e06400640 1 -- 172.21.15.42:0/2042384931 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ddc068970 msgr2=0x7f5ddc06ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.331+0000 7f5e06400640 1 --2- 172.21.15.42:0/2042384931 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ddc068970 0x7f5ddc06ae30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5de80094a0 tx=0x7f5de80093f0 comp rx=0 tx=0).stop 2024-05-28T14:03:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.331+0000 7f5e06400640 1 -- 172.21.15.42:0/2042384931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e000d0760 msgr2=0x7f5e000b68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.331+0000 7f5e06400640 1 --2- 172.21.15.42:0/2042384931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e000d0760 0x7f5e000b68f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5df000c3f0 tx=0x7f5df000c8b0 comp rx=0 tx=0).stop 2024-05-28T14:03:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.331+0000 7f5e06400640 1 -- 172.21.15.42:0/2042384931 shutdown_connections 2024-05-28T14:03:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.331+0000 7f5e06400640 1 --2- 172.21.15.42:0/2042384931 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ddc068970 0x7f5ddc06ae30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.331+0000 7f5e06400640 1 --2- 172.21.15.42:0/2042384931 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000d0b40 0x7f5e000c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.331+0000 7f5e06400640 1 --2- 172.21.15.42:0/2042384931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e000d0760 0x7f5e000b68f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.331+0000 7f5e06400640 1 --2- 172.21.15.42:0/2042384931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0006c430 0x7f5e000b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.331+0000 7f5e06400640 1 -- 172.21.15.42:0/2042384931 >> 172.21.15.42:0/2042384931 conn(0x7f5e0009aa50 msgr2=0x7f5e000cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.331+0000 7f5e06400640 1 -- 172.21.15.42:0/2042384931 shutdown_connections 2024-05-28T14:03:19.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.331+0000 7f5e06400640 1 -- 172.21.15.42:0/2042384931 wait complete. 2024-05-28T14:03:19.381 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone85311 2024-05-28T14:03:19.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.811+0000 7f558fa00640 1 Processor -- start 2024-05-28T14:03:19.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.811+0000 7f558fa00640 1 -- start start 2024-05-28T14:03:19.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.811+0000 7f558fa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55880b2430 0x7f55880b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:19.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.811+0000 7f558fa00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55880cbaa0 con 0x7f558815ed70 2024-05-28T14:03:19.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.811+0000 7f558fa00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55880b1ec0 con 0x7f55880ce3e0 2024-05-28T14:03:19.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.811+0000 7f558fa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f55880cfcd0 con 0x7f55880b2430 2024-05-28T14:03:19.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f558d000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f558815ed70 0x7f55880cdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55496/0 (socket says 172.21.15.42:55496) 2024-05-28T14:03:19.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f558d000640 1 -- 172.21.15.42:0/868463472 learned_addr learned my addr 172.21.15.42:0/868463472 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:03:19.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 -- 172.21.15.42:0/868463472 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3501132913 0 0) 0x7f55880b1ec0 con 0x7f55880ce3e0 2024-05-28T14:03:19.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 -- 172.21.15.42:0/868463472 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5570003530 con 0x7f55880ce3e0 2024-05-28T14:03:19.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 -- 172.21.15.42:0/868463472 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3512809027 0 0) 0x7f5570003530 con 0x7f55880ce3e0 2024-05-28T14:03:19.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 -- 172.21.15.42:0/868463472 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55880b2430 msgr2=0x7f55880b18e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-28T14:03:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 --2- 172.21.15.42:0/868463472 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55880b2430 0x7f55880b18e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 -- 172.21.15.42:0/868463472 >> v1:172.21.15.42:6789/0 conn(0x7f558815ed70 legacy=0x7f55880cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 -- 172.21.15.42:0/868463472 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55880b21a0 con 0x7f55880ce3e0 2024-05-28T14:03:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 -- 172.21.15.42:0/868463472 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f5574001680 con 0x7f55880ce3e0 2024-05-28T14:03:19.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 -- 172.21.15.42:0/868463472 >> v1:172.21.15.63:6789/0 conn(0x7f55880ce3e0 legacy=0x7f55880d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:19.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 --2- 172.21.15.42:0/868463472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55700039c0 0x7f5570003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:19.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 --2- 172.21.15.42:0/868463472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5570004810 0x7f5570004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:19.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 --2- 172.21.15.42:0/868463472 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5570005220 0x7f557000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:19.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 -- 172.21.15.42:0/868463472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55880cfcd0 con 0x7f5570004810 2024-05-28T14:03:19.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 -- 172.21.15.42:0/868463472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f55700148e0 con 0x7f55700039c0 2024-05-28T14:03:19.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587400640 1 -- 172.21.15.42:0/868463472 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5570014ae0 con 0x7f5570005220 2024-05-28T14:03:19.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587e00640 1 --2- 172.21.15.42:0/868463472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5570004810 0x7f5570004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:19.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f558da00640 1 --2- 172.21.15.42:0/868463472 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5570005220 0x7f557000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:19.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587e00640 1 -- 172.21.15.42:0/868463472 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5570005220 msgr2=0x7f557000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:19.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587e00640 1 --2- 172.21.15.42:0/868463472 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5570005220 0x7f557000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:19.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f558d000640 1 --2- 172.21.15.42:0/868463472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55700039c0 0x7f5570003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:19.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587e00640 1 -- 172.21.15.42:0/868463472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55700039c0 msgr2=0x7f5570003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:19.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587e00640 1 --2- 172.21.15.42:0/868463472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55700039c0 0x7f5570003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:19.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f5587e00640 1 -- 172.21.15.42:0/868463472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55880b21a0 con 0x7f5570004810 2024-05-28T14:03:19.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.815+0000 7f558d000640 1 --2- 172.21.15.42:0/868463472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55700039c0 0x7f5570003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:03:19.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.823+0000 7f5587e00640 1 --2- 172.21.15.42:0/868463472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5570004810 0x7f5570004c90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f557400c260 tx=0x7f557401f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4124b9979f9775b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:19.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.823+0000 7f5587400640 1 -- 172.21.15.42:0/868463472 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55740b4070 con 0x7f5570004810 2024-05-28T14:03:19.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.823+0000 7f5587400640 1 -- 172.21.15.42:0/868463472 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55740af600 con 0x7f5570004810 2024-05-28T14:03:19.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.823+0000 7f5587400640 1 -- 172.21.15.42:0/868463472 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574007d50 con 0x7f5570004810 2024-05-28T14:03:19.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.823+0000 7f558fa00640 1 -- 172.21.15.42:0/868463472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5570004810 msgr2=0x7f5570004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:19.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.823+0000 7f558fa00640 1 --2- 172.21.15.42:0/868463472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5570004810 0x7f5570004c90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f557400c260 tx=0x7f557401f450 comp rx=0 tx=0).stop 2024-05-28T14:03:19.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.823+0000 7f558da00640 1 -- 172.21.15.42:0/868463472 reap_dead start 2024-05-28T14:03:19.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.823+0000 7f558fa00640 1 -- 172.21.15.42:0/868463472 shutdown_connections 2024-05-28T14:03:19.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.823+0000 7f558fa00640 1 -- 172.21.15.42:0/868463472 >> 172.21.15.42:0/868463472 conn(0x7f558809aa50 msgr2=0x7f55880b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:19.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.823+0000 7f558fa00640 1 -- 172.21.15.42:0/868463472 shutdown_connections 2024-05-28T14:03:19.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.831+0000 7f558fa00640 1 -- 172.21.15.42:0/868463472 wait complete. 2024-05-28T14:03:19.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.839+0000 7f558fa00640 1 Processor -- start 2024-05-28T14:03:19.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.839+0000 7f558fa00640 1 -- start start 2024-05-28T14:03:19.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.839+0000 7f558fa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5570004810 0x7f55880b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:19.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.839+0000 7f558fa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5588152200 0x7f55880aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:19.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.839+0000 7f558fa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55880b6320 0x7f55880b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:19.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.839+0000 7f558fa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55880cecf0 con 0x7f55880b6320 2024-05-28T14:03:19.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.839+0000 7f558fa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f55880cf960 con 0x7f5570004810 2024-05-28T14:03:19.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.839+0000 7f558fa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f55880af220 con 0x7f5588152200 2024-05-28T14:03:19.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.843+0000 7f558da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55880b6320 0x7f55880b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:19.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.843+0000 7f558d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5570004810 0x7f55880b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:19.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.843+0000 7f558d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5570004810 0x7f55880b01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:41196/0 (socket says 172.21.15.42:41196) 2024-05-28T14:03:19.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.843+0000 7f558d000640 1 -- 172.21.15.42:0/89217829 learned_addr learned my addr 172.21.15.42:0/89217829 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:19.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.843+0000 7f558d000640 1 -- 172.21.15.42:0/89217829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5588152200 msgr2=0x7f55880aece0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T14:03:19.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.843+0000 7f558d000640 1 --2- 172.21.15.42:0/89217829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5588152200 0x7f55880aece0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:19.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.843+0000 7f558d000640 1 -- 172.21.15.42:0/89217829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55880b6320 msgr2=0x7f55880b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:19.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.843+0000 7f558d000640 1 --2- 172.21.15.42:0/89217829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55880b6320 0x7f55880b67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:19.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.843+0000 7f558d000640 1 -- 172.21.15.42:0/89217829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55880b3f00 con 0x7f5570004810 2024-05-28T14:03:19.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.843+0000 7f558da00640 1 --2- 172.21.15.42:0/89217829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55880b6320 0x7f55880b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:03:19.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.843+0000 7f558d000640 1 --2- 172.21.15.42:0/89217829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5570004810 0x7f55880b01b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f557c00d5e0 tx=0x7f557c00daa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:19.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.851+0000 7f5584c00640 1 -- 172.21.15.42:0/89217829 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c005cf0 con 0x7f5570004810 2024-05-28T14:03:19.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.851+0000 7f5584c00640 1 -- 172.21.15.42:0/89217829 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f557c017070 con 0x7f5570004810 2024-05-28T14:03:19.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.851+0000 7f5584c00640 1 -- 172.21.15.42:0/89217829 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c007270 con 0x7f5570004810 2024-05-28T14:03:19.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.855+0000 7f558fa00640 1 -- 172.21.15.42:0/89217829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55880af8c0 con 0x7f5570004810 2024-05-28T14:03:19.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.855+0000 7f558fa00640 1 -- 172.21.15.42:0/89217829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5588156b70 con 0x7f5570004810 2024-05-28T14:03:19.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.855+0000 7f558fa00640 1 -- 172.21.15.42:0/89217829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558809f650 con 0x7f5570004810 2024-05-28T14:03:19.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.867+0000 7f5584c00640 1 -- 172.21.15.42:0/89217829 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 14) ==== 88548+0+0 (secure 0 0 0) 0x7f557c007410 con 0x7f5570004810 2024-05-28T14:03:19.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.867+0000 7f5584c00640 1 --2- 172.21.15.42:0/89217829 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5568068a80 0x7f556806af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:19.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.867+0000 7f5584c00640 1 -- 172.21.15.42:0/89217829 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f557c088120 con 0x7f5570004810 2024-05-28T14:03:19.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.871+0000 7f5584c00640 1 -- 172.21.15.42:0/89217829 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f557c0884d0 con 0x7f5570004810 2024-05-28T14:03:19.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:19.899+0000 7f5587e00640 1 --2- 172.21.15.42:0/89217829 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5568068a80 0x7f556806af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:20.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:20.043+0000 7f5587e00640 1 --2- 172.21.15.42:0/89217829 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5568068a80 0x7f556806af40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f55740b1000 tx=0x7f55740aa000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:20.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:20.563+0000 7f558fa00640 1 -- 172.21.15.42:0/89217829 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone85311", "target": ["mon-mgr", ""]}) -- 0x7f558804fdd0 con 0x7f5568068a80 2024-05-28T14:03:20.727 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:20.735+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:20.727 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:20.735+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:20.728 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:20.735+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:20.728 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:20.735+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:20.728 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:03:20.735+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:03:21.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:21.691+0000 7f5584c00640 1 -- 172.21.15.42:0/89217829 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7f557c053c10 con 0x7f5570004810 2024-05-28T14:03:22.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:22.731+0000 7f5584c00640 1 -- 172.21.15.42:0/89217829 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f558804fdd0 con 0x7f5568068a80 2024-05-28T14:03:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:22.739+0000 7f558fa00640 1 -- 172.21.15.42:0/89217829 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5568068a80 msgr2=0x7f556806af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:22.739+0000 7f558fa00640 1 --2- 172.21.15.42:0/89217829 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5568068a80 0x7f556806af40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f55740b1000 tx=0x7f55740aa000 comp rx=0 tx=0).stop 2024-05-28T14:03:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:22.739+0000 7f558fa00640 1 -- 172.21.15.42:0/89217829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5570004810 msgr2=0x7f55880b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:22.739+0000 7f558fa00640 1 --2- 172.21.15.42:0/89217829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5570004810 0x7f55880b01b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f557c00d5e0 tx=0x7f557c00daa0 comp rx=0 tx=0).stop 2024-05-28T14:03:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:22.747+0000 7f558fa00640 1 -- 172.21.15.42:0/89217829 shutdown_connections 2024-05-28T14:03:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:22.747+0000 7f558fa00640 1 --2- 172.21.15.42:0/89217829 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5568068a80 0x7f556806af40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:22.747+0000 7f558fa00640 1 --2- 172.21.15.42:0/89217829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55880b6320 0x7f55880b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:22.747+0000 7f558fa00640 1 --2- 172.21.15.42:0/89217829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5588152200 0x7f55880aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:22.747+0000 7f558fa00640 1 --2- 172.21.15.42:0/89217829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5570004810 0x7f55880b01b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:22.747+0000 7f558fa00640 1 -- 172.21.15.42:0/89217829 >> 172.21.15.42:0/89217829 conn(0x7f558809aa50 msgr2=0x7f558806ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:22.747+0000 7f558fa00640 1 -- 172.21.15.42:0/89217829 shutdown_connections 2024-05-28T14:03:22.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:22.747+0000 7f558fa00640 1 -- 172.21.15.42:0/89217829 wait complete. 2024-05-28T14:03:22.793 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone85312 2024-05-28T14:03:23.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.287+0000 7fca97200640 1 Processor -- start 2024-05-28T14:03:23.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.291+0000 7fca97200640 1 -- start start 2024-05-28T14:03:23.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.291+0000 7fca97200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca980b2880 0x7fca9815aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:23.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.291+0000 7fca97200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fca980af630 0x7fca980b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.295+0000 7fca95e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca980b2880 0x7fca9815aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.295+0000 7fca95e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca980b2880 0x7fca9815aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:39540/0 (socket says 172.21.15.42:39540) 2024-05-28T14:03:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.295+0000 7fca95e00640 1 -- 172.21.15.42:0/3052414696 learned_addr learned my addr 172.21.15.42:0/3052414696 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:23.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.295+0000 7fca97200640 1 -- 172.21.15.42:0/3052414696 --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca980b4af0 con 0x7fca980b5040 2024-05-28T14:03:23.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.295+0000 7fca97200640 1 -- 172.21.15.42:0/3052414696 --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fca98073950 con 0x7fca980af630 2024-05-28T14:03:23.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.295+0000 7fca97200640 1 -- 172.21.15.42:0/3052414696 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fca980b0b00 con 0x7fca980b2880 2024-05-28T14:03:23.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.295+0000 7fca95400640 1 --2- 172.21.15.42:0/3052414696 >> v2:172.21.15.42:3300/0 conn(0x7fca980af630 0x7fca980b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:23.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.299+0000 7fca95400640 1 -- 172.21.15.42:0/3052414696 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca980b2880 msgr2=0x7fca9815aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:23.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.299+0000 7fca95400640 1 --2- 172.21.15.42:0/3052414696 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca980b2880 0x7fca9815aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:23.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.299+0000 7fca95400640 1 -- 172.21.15.42:0/3052414696 >> v1:172.21.15.63:6789/0 conn(0x7fca980b5040 legacy=0x7fca9815e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:23.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.299+0000 7fca95400640 1 -- 172.21.15.42:0/3052414696 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca980b4d90 con 0x7fca980af630 2024-05-28T14:03:23.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.307+0000 7fca95400640 1 --2- 172.21.15.42:0/3052414696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca980af630 0x7fca980b0df0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fca840097c0 tx=0x7fca8401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f71940495946ee3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:23.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.307+0000 7fca94a00640 1 -- 172.21.15.42:0/3052414696 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca840b0070 con 0x7fca980af630 2024-05-28T14:03:23.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.307+0000 7fca94a00640 1 -- 172.21.15.42:0/3052414696 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca840ab600 con 0x7fca980af630 2024-05-28T14:03:23.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.307+0000 7fca94a00640 1 -- 172.21.15.42:0/3052414696 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca840ab9d0 con 0x7fca980af630 2024-05-28T14:03:23.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.307+0000 7fca97200640 1 -- 172.21.15.42:0/3052414696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca980af630 msgr2=0x7fca980b0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:23.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.307+0000 7fca97200640 1 --2- 172.21.15.42:0/3052414696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca980af630 0x7fca980b0df0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fca840097c0 tx=0x7fca8401c410 comp rx=0 tx=0).stop 2024-05-28T14:03:23.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.311+0000 7fca97200640 1 -- 172.21.15.42:0/3052414696 shutdown_connections 2024-05-28T14:03:23.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.311+0000 7fca97200640 1 --2- 172.21.15.42:0/3052414696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca980af630 0x7fca980b0df0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:23.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.311+0000 7fca97200640 1 --2- 172.21.15.42:0/3052414696 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca980b2880 0x7fca9815aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:23.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.311+0000 7fca97200640 1 -- 172.21.15.42:0/3052414696 >> 172.21.15.42:0/3052414696 conn(0x7fca980b1d60 msgr2=0x7fca980b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:23.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.311+0000 7fca97200640 1 -- 172.21.15.42:0/3052414696 shutdown_connections 2024-05-28T14:03:23.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.311+0000 7fca97200640 1 -- 172.21.15.42:0/3052414696 wait complete. 2024-05-28T14:03:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.315+0000 7fca97200640 1 Processor -- start 2024-05-28T14:03:23.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.323+0000 7fca97200640 1 -- start start 2024-05-28T14:03:23.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.323+0000 7fca97200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca980af630 0x7fca9814b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:23.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.323+0000 7fca97200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca980b2880 0x7fca9814d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:23.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.327+0000 7fca97200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca980b5040 0x7fca9815fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:23.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.327+0000 7fca97200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fca98073dc0 con 0x7fca980b2880 2024-05-28T14:03:23.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.327+0000 7fca97200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fca980b1740 con 0x7fca980af630 2024-05-28T14:03:23.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.327+0000 7fca97200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fca98073f40 con 0x7fca980b5040 2024-05-28T14:03:23.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.331+0000 7fca96800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca980b5040 0x7fca9815fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:23.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.331+0000 7fca96800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca980b5040 0x7fca9815fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:39542/0 (socket says 172.21.15.42:39542) 2024-05-28T14:03:23.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.331+0000 7fca96800640 1 -- 172.21.15.42:0/2765703779 learned_addr learned my addr 172.21.15.42:0/2765703779 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:23.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.331+0000 7fca95e00640 1 --2- 172.21.15.42:0/2765703779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca980af630 0x7fca9814b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:23.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.331+0000 7fca95400640 1 --2- 172.21.15.42:0/2765703779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca980b2880 0x7fca9814d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:23.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.331+0000 7fca95e00640 1 -- 172.21.15.42:0/2765703779 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca980b5040 msgr2=0x7fca9815fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:23.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.331+0000 7fca95e00640 1 --2- 172.21.15.42:0/2765703779 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca980b5040 0x7fca9815fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:23.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.331+0000 7fca95e00640 1 -- 172.21.15.42:0/2765703779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca980b2880 msgr2=0x7fca9814d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:23.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.331+0000 7fca95e00640 1 --2- 172.21.15.42:0/2765703779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca980b2880 0x7fca9814d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:23.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.331+0000 7fca95e00640 1 -- 172.21.15.42:0/2765703779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca980adee0 con 0x7fca980af630 2024-05-28T14:03:23.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.331+0000 7fca95e00640 1 --2- 172.21.15.42:0/2765703779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca980af630 0x7fca9814b950 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fca8800e870 tx=0x7fca8800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.335+0000 7fca8e000640 1 -- 172.21.15.42:0/2765703779 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88017070 con 0x7fca980af630 2024-05-28T14:03:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.335+0000 7fca8e000640 1 -- 172.21.15.42:0/2765703779 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca8800c940 con 0x7fca980af630 2024-05-28T14:03:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.335+0000 7fca8e000640 1 -- 172.21.15.42:0/2765703779 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88017070 con 0x7fca980af630 2024-05-28T14:03:23.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.335+0000 7fca97200640 1 -- 172.21.15.42:0/2765703779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca980ae1c0 con 0x7fca980af630 2024-05-28T14:03:23.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.339+0000 7fca97200640 1 -- 172.21.15.42:0/2765703779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca980ae780 con 0x7fca980af630 2024-05-28T14:03:23.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.339+0000 7fca83400640 1 -- 172.21.15.42:0/2765703779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca6c0052a0 con 0x7fca980af630 2024-05-28T14:03:23.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.343+0000 7fca8e000640 1 -- 172.21.15.42:0/2765703779 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7fca8801a070 con 0x7fca980af630 2024-05-28T14:03:23.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.343+0000 7fca8e000640 1 --2- 172.21.15.42:0/2765703779 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fca78068c40 0x7fca7806b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:23.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.347+0000 7fca8e000640 1 -- 172.21.15.42:0/2765703779 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fca88054d60 con 0x7fca980af630 2024-05-28T14:03:23.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.355+0000 7fca8e000640 1 -- 172.21.15.42:0/2765703779 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fca88056cc0 con 0x7fca980af630 2024-05-28T14:03:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.363+0000 7fca95400640 1 --2- 172.21.15.42:0/2765703779 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fca78068c40 0x7fca7806b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:23.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.387+0000 7fca95400640 1 --2- 172.21.15.42:0/2765703779 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fca78068c40 0x7fca7806b100 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fca840ad000 tx=0x7fca84008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:23.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:23.847+0000 7fca83400640 1 -- 172.21.15.42:0/2765703779 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone85312", "target": ["mon-mgr", ""]}) -- 0x7fca6c000bd0 con 0x7fca78068c40 2024-05-28T14:03:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:24.567+0000 7fca8e000640 1 -- 172.21.15.42:0/2765703779 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fca6c000bd0 con 0x7fca78068c40 2024-05-28T14:03:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:24.591+0000 7fca83400640 1 -- 172.21.15.42:0/2765703779 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fca78068c40 msgr2=0x7fca7806b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:24.591+0000 7fca83400640 1 --2- 172.21.15.42:0/2765703779 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fca78068c40 0x7fca7806b100 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fca840ad000 tx=0x7fca84008040 comp rx=0 tx=0).stop 2024-05-28T14:03:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:24.591+0000 7fca83400640 1 -- 172.21.15.42:0/2765703779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca980af630 msgr2=0x7fca9814b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:24.591+0000 7fca83400640 1 --2- 172.21.15.42:0/2765703779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca980af630 0x7fca9814b950 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fca8800e870 tx=0x7fca8800ed30 comp rx=0 tx=0).stop 2024-05-28T14:03:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:24.591+0000 7fca83400640 1 -- 172.21.15.42:0/2765703779 shutdown_connections 2024-05-28T14:03:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:24.591+0000 7fca83400640 1 --2- 172.21.15.42:0/2765703779 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fca78068c40 0x7fca7806b100 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:24.591+0000 7fca83400640 1 --2- 172.21.15.42:0/2765703779 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca980b5040 0x7fca9815fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:24.591+0000 7fca83400640 1 --2- 172.21.15.42:0/2765703779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca980b2880 0x7fca9814d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:24.591+0000 7fca83400640 1 --2- 172.21.15.42:0/2765703779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca980af630 0x7fca9814b950 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fca8800e870 tx=0x7fca8800ed30 comp rx=0 tx=0).stop 2024-05-28T14:03:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:24.591+0000 7fca83400640 1 -- 172.21.15.42:0/2765703779 >> 172.21.15.42:0/2765703779 conn(0x7fca980b1d60 msgr2=0x7fca980b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:24.595+0000 7fca83400640 1 -- 172.21.15.42:0/2765703779 shutdown_connections 2024-05-28T14:03:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:24.599+0000 7fca83400640 1 -- 172.21.15.42:0/2765703779 wait complete. 2024-05-28T14:03:24.679 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone85313 2024-05-28T14:03:25.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.051+0000 7fcfa5200640 1 Processor -- start 2024-05-28T14:03:25.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.051+0000 7fcfa5200640 1 -- start start 2024-05-28T14:03:25.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.051+0000 7fcfa5200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcfa00b3fc0 0x7fcfa006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:25.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.051+0000 7fcfa5200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa0167070 0x7fcfa00cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:25.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.051+0000 7fcfa5200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfa0162f60 con 0x7fcfa00d0320 2024-05-28T14:03:25.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.051+0000 7fcfa5200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fcfa006f5e0 con 0x7fcfa00b3fc0 2024-05-28T14:03:25.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.051+0000 7fcfa5200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa00b4630 con 0x7fcfa0167070 2024-05-28T14:03:25.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.055+0000 7fcf9ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcfa00b3fc0 0x7fcfa006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:25.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.055+0000 7fcf9ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcfa00b3fc0 0x7fcfa006e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54268/0 (socket says 172.21.15.42:54268) 2024-05-28T14:03:25.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.055+0000 7fcf9ea00640 1 -- 172.21.15.42:0/2682855072 learned_addr learned my addr 172.21.15.42:0/2682855072 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:25.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.055+0000 7fcf9f400640 1 --1- 172.21.15.42:0/2682855072 >> v1:172.21.15.42:6789/0 conn(0x7fcfa00d0320 0x7fcfa00d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55504/0 (socket says 172.21.15.42:55504) 2024-05-28T14:03:25.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.055+0000 7fcf9ea00640 1 -- 172.21.15.42:0/2682855072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa0167070 msgr2=0x7fcfa00cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:25.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.055+0000 7fcf9ea00640 1 --2- 172.21.15.42:0/2682855072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa0167070 0x7fcfa00cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:25.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.055+0000 7fcf9ea00640 1 -- 172.21.15.42:0/2682855072 >> v1:172.21.15.42:6789/0 conn(0x7fcfa00d0320 legacy=0x7fcfa00d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:25.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.055+0000 7fcf9ea00640 1 -- 172.21.15.42:0/2682855072 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa00b4910 con 0x7fcfa00b3fc0 2024-05-28T14:03:25.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.059+0000 7fcf9ea00640 1 --2- 172.21.15.42:0/2682855072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa00b3fc0 0x7fcfa006e4a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009870 tx=0x7fcf8c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b851f065b5b679be server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:25.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.059+0000 7fcf9e000640 1 -- 172.21.15.42:0/2682855072 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c0b1070 con 0x7fcfa00b3fc0 2024-05-28T14:03:25.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.059+0000 7fcf9e000640 1 -- 172.21.15.42:0/2682855072 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf8c0a8050 con 0x7fcfa00b3fc0 2024-05-28T14:03:25.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.059+0000 7fcf9e000640 1 -- 172.21.15.42:0/2682855072 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c0ac860 con 0x7fcfa00b3fc0 2024-05-28T14:03:25.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.059+0000 7fcfa5200640 1 -- 172.21.15.42:0/2682855072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa00b3fc0 msgr2=0x7fcfa006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:25.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.059+0000 7fcfa5200640 1 --2- 172.21.15.42:0/2682855072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa00b3fc0 0x7fcfa006e4a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009870 tx=0x7fcf8c01c4d0 comp rx=0 tx=0).stop 2024-05-28T14:03:25.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.059+0000 7fcfa5200640 1 -- 172.21.15.42:0/2682855072 shutdown_connections 2024-05-28T14:03:25.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.059+0000 7fcfa5200640 1 --2- 172.21.15.42:0/2682855072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa0167070 0x7fcfa00cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:25.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.059+0000 7fcfa5200640 1 --2- 172.21.15.42:0/2682855072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa00b3fc0 0x7fcfa006e4a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:25.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.059+0000 7fcfa5200640 1 -- 172.21.15.42:0/2682855072 >> 172.21.15.42:0/2682855072 conn(0x7fcfa006bbc0 msgr2=0x7fcfa006bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:25.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.063+0000 7fcfa5200640 1 -- 172.21.15.42:0/2682855072 shutdown_connections 2024-05-28T14:03:25.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.063+0000 7fcfa5200640 1 -- 172.21.15.42:0/2682855072 wait complete. 2024-05-28T14:03:25.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.071+0000 7fcfa5200640 1 Processor -- start 2024-05-28T14:03:25.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.071+0000 7fcfa5200640 1 -- start start 2024-05-28T14:03:25.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.071+0000 7fcfa5200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa00b3fc0 0x7fcfa0160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:25.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.071+0000 7fcfa5200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa00d0320 0x7fcfa0159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:25.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.071+0000 7fcfa5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa0167070 0x7fcfa015a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:25.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.071+0000 7fcfa5200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa016fbd0 con 0x7fcfa0167070 2024-05-28T14:03:25.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.071+0000 7fcfa5200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa016fa50 con 0x7fcfa00d0320 2024-05-28T14:03:25.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.071+0000 7fcfa5200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa016fd50 con 0x7fcfa00b3fc0 2024-05-28T14:03:25.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.075+0000 7fcf9ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa00d0320 0x7fcfa0159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:25.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.075+0000 7fcf9ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa00d0320 0x7fcfa0159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:41228/0 (socket says 172.21.15.42:41228) 2024-05-28T14:03:25.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.075+0000 7fcf9ea00640 1 -- 172.21.15.42:0/1700596199 learned_addr learned my addr 172.21.15.42:0/1700596199 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:25.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.079+0000 7fcf9fe00640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa0167070 0x7fcfa015a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:25.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.083+0000 7fcf9f400640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa00b3fc0 0x7fcfa0160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:25.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.083+0000 7fcf9fe00640 1 -- 172.21.15.42:0/1700596199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa00b3fc0 msgr2=0x7fcfa0160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:25.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.083+0000 7fcf9fe00640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa00b3fc0 0x7fcfa0160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:25.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.083+0000 7fcf9fe00640 1 -- 172.21.15.42:0/1700596199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa00d0320 msgr2=0x7fcfa0159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:25.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.083+0000 7fcf9fe00640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa00d0320 0x7fcfa0159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:25.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.083+0000 7fcf9fe00640 1 -- 172.21.15.42:0/1700596199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa00b3230 con 0x7fcfa0167070 2024-05-28T14:03:25.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.083+0000 7fcf9f400640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa00b3fc0 0x7fcfa0160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:03:25.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.083+0000 7fcf9fe00640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa0167070 0x7fcfa015a280 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800be20 tx=0x7fcf9800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:25.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.083+0000 7fcf97400640 1 -- 172.21.15.42:0/1700596199 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98019070 con 0x7fcfa0167070 2024-05-28T14:03:25.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.083+0000 7fcf97400640 1 -- 172.21.15.42:0/1700596199 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf98014440 con 0x7fcfa0167070 2024-05-28T14:03:25.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.083+0000 7fcf97400640 1 -- 172.21.15.42:0/1700596199 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98014830 con 0x7fcfa0167070 2024-05-28T14:03:25.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.083+0000 7fcfa5200640 1 -- 172.21.15.42:0/1700596199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa00b2180 con 0x7fcfa0167070 2024-05-28T14:03:25.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.083+0000 7fcfa5200640 1 -- 172.21.15.42:0/1700596199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa00b2740 con 0x7fcfa0167070 2024-05-28T14:03:25.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.087+0000 7fcfa5200640 1 -- 172.21.15.42:0/1700596199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa004fcf0 con 0x7fcfa0167070 2024-05-28T14:03:25.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.091+0000 7fcf97400640 1 -- 172.21.15.42:0/1700596199 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7fcf9801c070 con 0x7fcfa0167070 2024-05-28T14:03:25.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.095+0000 7fcf97400640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf80068a90 0x7fcf8006af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:25.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.099+0000 7fcf97400640 1 -- 172.21.15.42:0/1700596199 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fcf98087800 con 0x7fcfa0167070 2024-05-28T14:03:25.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.103+0000 7fcf97400640 1 -- 172.21.15.42:0/1700596199 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcf98023020 con 0x7fcfa0167070 2024-05-28T14:03:25.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.111+0000 7fcf9f400640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf80068a90 0x7fcf8006af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:25.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.167+0000 7fcf9f400640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf80068a90 0x7fcf8006af50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcfa01ff190 tx=0x7fcf90009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:25.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:25.575+0000 7fcfa5200640 1 -- 172.21.15.42:0/1700596199 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone85313", "target": ["mon-mgr", ""]}) -- 0x7fcfa00b2a80 con 0x7fcf80068a90 2024-05-28T14:03:25.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.007+0000 7fcf97400640 1 -- 172.21.15.42:0/1700596199 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcfa00b2a80 con 0x7fcf80068a90 2024-05-28T14:03:25.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.011+0000 7fcfa5200640 1 -- 172.21.15.42:0/1700596199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf80068a90 msgr2=0x7fcf8006af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:25.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.011+0000 7fcfa5200640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf80068a90 0x7fcf8006af50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcfa01ff190 tx=0x7fcf90009040 comp rx=0 tx=0).stop 2024-05-28T14:03:25.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.011+0000 7fcfa5200640 1 -- 172.21.15.42:0/1700596199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa0167070 msgr2=0x7fcfa015a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:25.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.011+0000 7fcfa5200640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa0167070 0x7fcfa015a280 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800be20 tx=0x7fcf9800bf10 comp rx=0 tx=0).stop 2024-05-28T14:03:25.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.011+0000 7fcfa5200640 1 -- 172.21.15.42:0/1700596199 shutdown_connections 2024-05-28T14:03:25.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.011+0000 7fcfa5200640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf80068a90 0x7fcf8006af50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:25.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.011+0000 7fcfa5200640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfa0167070 0x7fcfa015a280 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:25.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.011+0000 7fcfa5200640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa00d0320 0x7fcfa0159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:25.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.011+0000 7fcfa5200640 1 --2- 172.21.15.42:0/1700596199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa00b3fc0 0x7fcfa0160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:25.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.011+0000 7fcfa5200640 1 -- 172.21.15.42:0/1700596199 >> 172.21.15.42:0/1700596199 conn(0x7fcfa006bbc0 msgr2=0x7fcfa0160a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:25.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.011+0000 7fcfa5200640 1 -- 172.21.15.42:0/1700596199 shutdown_connections 2024-05-28T14:03:25.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.011+0000 7fcfa5200640 1 -- 172.21.15.42:0/1700596199 wait complete. 2024-05-28T14:03:26.054 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone85314 2024-05-28T14:03:26.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.359+0000 7fd10f000640 1 Processor -- start 2024-05-28T14:03:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.359+0000 7fd10f000640 1 -- start start 2024-05-28T14:03:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.363+0000 7fd10f000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd10815b570 0x7fd1080cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.363+0000 7fd10f000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd10815b950 0x7fd1080b2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.363+0000 7fd10f000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1080b2f20 con 0x7fd1080cdde0 2024-05-28T14:03:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.363+0000 7fd10f000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd1080b44e0 con 0x7fd10815b950 2024-05-28T14:03:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.363+0000 7fd10f000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd1080b47c0 con 0x7fd10815b570 2024-05-28T14:03:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.363+0000 7fd107400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd10815b950 0x7fd1080b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.363+0000 7fd10d000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd1080cdde0 0x7fd1080d0890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55514/0 (socket says 172.21.15.42:55514) 2024-05-28T14:03:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.363+0000 7fd10d000640 1 -- 172.21.15.42:0/373514829 learned_addr learned my addr 172.21.15.42:0/373514829 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:03:26.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.363+0000 7fd107e00640 1 --2- 172.21.15.42:0/373514829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd10815b570 0x7fd1080cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:26.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.363+0000 7fd107400640 1 -- 172.21.15.42:0/373514829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd10815b570 msgr2=0x7fd1080cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:26.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.363+0000 7fd107400640 1 --2- 172.21.15.42:0/373514829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd10815b570 0x7fd1080cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:26.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.363+0000 7fd107400640 1 -- 172.21.15.42:0/373514829 >> v1:172.21.15.42:6789/0 conn(0x7fd1080cdde0 legacy=0x7fd1080d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:26.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.363+0000 7fd107400640 1 -- 172.21.15.42:0/373514829 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1080b4aa0 con 0x7fd10815b950 2024-05-28T14:03:26.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.367+0000 7fd107400640 1 --2- 172.21.15.42:0/373514829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd10815b950 0x7fd1080b2820 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80097c0 tx=0x7fd0f801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fab7de04901b72 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:26.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.371+0000 7fd106a00640 1 -- 172.21.15.42:0/373514829 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80b0070 con 0x7fd10815b950 2024-05-28T14:03:26.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.371+0000 7fd106a00640 1 -- 172.21.15.42:0/373514829 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f80ab600 con 0x7fd10815b950 2024-05-28T14:03:26.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.371+0000 7fd106a00640 1 -- 172.21.15.42:0/373514829 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80ab9d0 con 0x7fd10815b950 2024-05-28T14:03:26.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.375+0000 7fd10f000640 1 -- 172.21.15.42:0/373514829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd10815b950 msgr2=0x7fd1080b2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:26.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.375+0000 7fd10f000640 1 --2- 172.21.15.42:0/373514829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd10815b950 0x7fd1080b2820 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80097c0 tx=0x7fd0f801c410 comp rx=0 tx=0).stop 2024-05-28T14:03:26.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.375+0000 7fd10f000640 1 -- 172.21.15.42:0/373514829 shutdown_connections 2024-05-28T14:03:26.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.375+0000 7fd10f000640 1 --2- 172.21.15.42:0/373514829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd10815b950 0x7fd1080b2820 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:26.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.375+0000 7fd10f000640 1 --2- 172.21.15.42:0/373514829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd10815b570 0x7fd1080cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:26.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.375+0000 7fd10f000640 1 -- 172.21.15.42:0/373514829 >> 172.21.15.42:0/373514829 conn(0x7fd1080b6960 msgr2=0x7fd1080b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:26.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd10f000640 1 -- 172.21.15.42:0/373514829 shutdown_connections 2024-05-28T14:03:26.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd10f000640 1 -- 172.21.15.42:0/373514829 wait complete. 2024-05-28T14:03:26.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd10f000640 1 Processor -- start 2024-05-28T14:03:26.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd10f000640 1 -- start start 2024-05-28T14:03:26.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd10f000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1080cdde0 0x7fd108148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:26.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd10f000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd10815b570 0x7fd10814cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:26.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd10f000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd10815b950 0x7fd108159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:26.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd10f000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd10806b9f0 con 0x7fd10815b570 2024-05-28T14:03:26.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd10f000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd10806bb70 con 0x7fd10815b950 2024-05-28T14:03:26.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd10f000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd10806bcf0 con 0x7fd1080cdde0 2024-05-28T14:03:26.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd107400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd10815b570 0x7fd10814cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:26.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd107400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd10815b570 0x7fd10814cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54290/0 (socket says 172.21.15.42:54290) 2024-05-28T14:03:26.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd107400640 1 -- 172.21.15.42:0/3005423120 learned_addr learned my addr 172.21.15.42:0/3005423120 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:26.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd107e00640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1080cdde0 0x7fd108148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:26.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd10d000640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd10815b950 0x7fd108159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:26.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd107400640 1 -- 172.21.15.42:0/3005423120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1080cdde0 msgr2=0x7fd108148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:26.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd107400640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1080cdde0 0x7fd108148f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:26.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd107400640 1 -- 172.21.15.42:0/3005423120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd10815b950 msgr2=0x7fd108159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:26.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd107400640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd10815b950 0x7fd108159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:26.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.379+0000 7fd107400640 1 -- 172.21.15.42:0/3005423120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1081494b0 con 0x7fd10815b570 2024-05-28T14:03:26.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.383+0000 7fd10d000640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd10815b950 0x7fd108159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:03:26.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.383+0000 7fd107400640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd10815b570 0x7fd10814cf80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8019020 tx=0x7fd0f80191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:26.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.383+0000 7fd0ffe00640 1 -- 172.21.15.42:0/3005423120 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80b9020 con 0x7fd10815b570 2024-05-28T14:03:26.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.383+0000 7fd0ffe00640 1 -- 172.21.15.42:0/3005423120 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f80b5680 con 0x7fd10815b570 2024-05-28T14:03:26.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.383+0000 7fd0ffe00640 1 -- 172.21.15.42:0/3005423120 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80b0040 con 0x7fd10815b570 2024-05-28T14:03:26.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.383+0000 7fd10f000640 1 -- 172.21.15.42:0/3005423120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10814edd0 con 0x7fd10815b570 2024-05-28T14:03:26.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.383+0000 7fd10f000640 1 -- 172.21.15.42:0/3005423120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1081f7dd0 con 0x7fd10815b570 2024-05-28T14:03:26.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.383+0000 7fd10f000640 1 -- 172.21.15.42:0/3005423120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd10809f7a0 con 0x7fd10815b570 2024-05-28T14:03:26.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.387+0000 7fd0ffe00640 1 -- 172.21.15.42:0/3005423120 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7fd0f80bc020 con 0x7fd10815b570 2024-05-28T14:03:26.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.387+0000 7fd0ffe00640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd0e8068a40 0x7fd0e806af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:26.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.387+0000 7fd0ffe00640 1 -- 172.21.15.42:0/3005423120 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fd0f8121160 con 0x7fd10815b570 2024-05-28T14:03:26.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.387+0000 7fd107e00640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd0e8068a40 0x7fd0e806af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:26.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.387+0000 7fd107e00640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd0e8068a40 0x7fd0e806af00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4007a60 tx=0x7fd0f4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:26.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.391+0000 7fd0ffe00640 1 -- 172.21.15.42:0/3005423120 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd0f8016090 con 0x7fd10815b570 2024-05-28T14:03:26.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.787+0000 7fd10f000640 1 -- 172.21.15.42:0/3005423120 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone85314", "target": ["mon-mgr", ""]}) -- 0x7fd10806e7b0 con 0x7fd0e8068a40 2024-05-28T14:03:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.967+0000 7fd0ffe00640 1 -- 172.21.15.42:0/3005423120 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd10806e7b0 con 0x7fd0e8068a40 2024-05-28T14:03:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.967+0000 7fd10f000640 1 -- 172.21.15.42:0/3005423120 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd0e8068a40 msgr2=0x7fd0e806af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.967+0000 7fd10f000640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd0e8068a40 0x7fd0e806af00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4007a60 tx=0x7fd0f4008040 comp rx=0 tx=0).stop 2024-05-28T14:03:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.967+0000 7fd10f000640 1 -- 172.21.15.42:0/3005423120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd10815b570 msgr2=0x7fd10814cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.967+0000 7fd10f000640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd10815b570 0x7fd10814cf80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8019020 tx=0x7fd0f80191b0 comp rx=0 tx=0).stop 2024-05-28T14:03:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.967+0000 7fd10f000640 1 -- 172.21.15.42:0/3005423120 shutdown_connections 2024-05-28T14:03:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.967+0000 7fd10f000640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd0e8068a40 0x7fd0e806af00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.967+0000 7fd10f000640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd10815b950 0x7fd108159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:26.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.967+0000 7fd10f000640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd10815b570 0x7fd10814cf80 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:26.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.967+0000 7fd10f000640 1 --2- 172.21.15.42:0/3005423120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1080cdde0 0x7fd108148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:26.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.967+0000 7fd10f000640 1 -- 172.21.15.42:0/3005423120 >> 172.21.15.42:0/3005423120 conn(0x7fd1080b6960 msgr2=0x7fd10806f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:26.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.967+0000 7fd10f000640 1 -- 172.21.15.42:0/3005423120 shutdown_connections 2024-05-28T14:03:26.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:26.967+0000 7fd10f000640 1 -- 172.21.15.42:0/3005423120 wait complete. 2024-05-28T14:03:26.975 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:03:27.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef94a00640 1 Processor -- start 2024-05-28T14:03:27.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef94a00640 1 -- start start 2024-05-28T14:03:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef94a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fef900cf230 0x7fef90072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef94a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef90072ba0 0x7fef9006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef94a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef900cfc10 con 0x7fef9015bf30 2024-05-28T14:03:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef94a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fef900721d0 con 0x7fef900cf230 2024-05-28T14:03:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef94a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef900cfef0 con 0x7fef90072ba0 2024-05-28T14:03:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef8d400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fef900cf230 0x7fef90072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef8d400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fef900cf230 0x7fef90072660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54304/0 (socket says 172.21.15.42:54304) 2024-05-28T14:03:27.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef8de00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fef9015bf30 0x7fef9015c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55518/0 (socket says 172.21.15.42:55518) 2024-05-28T14:03:27.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef8d400640 1 -- 172.21.15.42:0/3984447290 learned_addr learned my addr 172.21.15.42:0/3984447290 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:27.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef8e800640 1 --2- 172.21.15.42:0/3984447290 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef90072ba0 0x7fef9006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:27.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef8d400640 1 -- 172.21.15.42:0/3984447290 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef90072ba0 msgr2=0x7fef9006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:27.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef8d400640 1 --2- 172.21.15.42:0/3984447290 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef90072ba0 0x7fef9006ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef8d400640 1 -- 172.21.15.42:0/3984447290 >> v1:172.21.15.42:6789/0 conn(0x7fef9015bf30 legacy=0x7fef9015c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:27.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef8ca00640 1 -- 172.21.15.42:0/3984447290 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4023723711 0 0) 0x7fef900cfc10 con 0x7fef9015bf30 2024-05-28T14:03:27.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.131+0000 7fef8d400640 1 -- 172.21.15.42:0/3984447290 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef900d01d0 con 0x7fef900cf230 2024-05-28T14:03:27.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.135+0000 7fef8d400640 1 --2- 172.21.15.42:0/3984447290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef900cf230 0x7fef90072660 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009870 tx=0x7fef7c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fdbe3ba9b1aa284 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:27.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.135+0000 7fef8ca00640 1 -- 172.21.15.42:0/3984447290 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0a7070 con 0x7fef900cf230 2024-05-28T14:03:27.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.135+0000 7fef8ca00640 1 -- 172.21.15.42:0/3984447290 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef7c0b0070 con 0x7fef900cf230 2024-05-28T14:03:27.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.135+0000 7fef8ca00640 1 -- 172.21.15.42:0/3984447290 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0ab890 con 0x7fef900cf230 2024-05-28T14:03:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.135+0000 7fef94a00640 1 -- 172.21.15.42:0/3984447290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef900cf230 msgr2=0x7fef90072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.135+0000 7fef94a00640 1 --2- 172.21.15.42:0/3984447290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef900cf230 0x7fef90072660 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009870 tx=0x7fef7c01c560 comp rx=0 tx=0).stop 2024-05-28T14:03:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.135+0000 7fef94a00640 1 -- 172.21.15.42:0/3984447290 shutdown_connections 2024-05-28T14:03:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.135+0000 7fef94a00640 1 --2- 172.21.15.42:0/3984447290 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef90072ba0 0x7fef9006ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.135+0000 7fef94a00640 1 --2- 172.21.15.42:0/3984447290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef900cf230 0x7fef90072660 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.135+0000 7fef94a00640 1 -- 172.21.15.42:0/3984447290 >> 172.21.15.42:0/3984447290 conn(0x7fef9006b9f0 msgr2=0x7fef9006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.135+0000 7fef94a00640 1 -- 172.21.15.42:0/3984447290 shutdown_connections 2024-05-28T14:03:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.135+0000 7fef94a00640 1 -- 172.21.15.42:0/3984447290 wait complete. 2024-05-28T14:03:27.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef94a00640 1 Processor -- start 2024-05-28T14:03:27.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef94a00640 1 -- start start 2024-05-28T14:03:27.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef94a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef90072ba0 0x7fef9013cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef94a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef900cf230 0x7fef901f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef94a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9015bf30 0x7fef901faff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef94a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fef90168540 con 0x7fef90072ba0 2024-05-28T14:03:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef94a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef901683c0 con 0x7fef9015bf30 2024-05-28T14:03:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef94a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef901686c0 con 0x7fef900cf230 2024-05-28T14:03:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef8de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef90072ba0 0x7fef9013cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef8de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef90072ba0 0x7fef9013cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54318/0 (socket says 172.21.15.42:54318) 2024-05-28T14:03:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef8de00640 1 -- 172.21.15.42:0/2282261928 learned_addr learned my addr 172.21.15.42:0/2282261928 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef8d400640 1 --2- 172.21.15.42:0/2282261928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef900cf230 0x7fef901f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef8e800640 1 --2- 172.21.15.42:0/2282261928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9015bf30 0x7fef901faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef8de00640 1 -- 172.21.15.42:0/2282261928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef900cf230 msgr2=0x7fef901f83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef8de00640 1 --2- 172.21.15.42:0/2282261928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef900cf230 0x7fef901f83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef8de00640 1 -- 172.21.15.42:0/2282261928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9015bf30 msgr2=0x7fef901faff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef8de00640 1 --2- 172.21.15.42:0/2282261928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9015bf30 0x7fef901faff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef8de00640 1 -- 172.21.15.42:0/2282261928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef901fb8f0 con 0x7fef90072ba0 2024-05-28T14:03:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef8de00640 1 --2- 172.21.15.42:0/2282261928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef90072ba0 0x7fef9013cce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c480 tx=0x7fef8000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef86000640 1 -- 172.21.15.42:0/2282261928 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80018070 con 0x7fef90072ba0 2024-05-28T14:03:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef86000640 1 -- 172.21.15.42:0/2282261928 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef80007500 con 0x7fef90072ba0 2024-05-28T14:03:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef94a00640 1 -- 172.21.15.42:0/2282261928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef901fbbd0 con 0x7fef90072ba0 2024-05-28T14:03:27.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef86000640 1 -- 172.21.15.42:0/2282261928 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80004a30 con 0x7fef90072ba0 2024-05-28T14:03:27.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef94a00640 1 -- 172.21.15.42:0/2282261928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9013d220 con 0x7fef90072ba0 2024-05-28T14:03:27.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.139+0000 7fef94a00640 1 -- 172.21.15.42:0/2282261928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef901f79d0 con 0x7fef90072ba0 2024-05-28T14:03:27.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.143+0000 7fef86000640 1 -- 172.21.15.42:0/2282261928 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7fef8001b070 con 0x7fef90072ba0 2024-05-28T14:03:27.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.143+0000 7fef86000640 1 --2- 172.21.15.42:0/2282261928 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef70068a40 0x7fef7006af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:27.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.143+0000 7fef86000640 1 -- 172.21.15.42:0/2282261928 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fef800868a0 con 0x7fef90072ba0 2024-05-28T14:03:27.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.143+0000 7fef8d400640 1 --2- 172.21.15.42:0/2282261928 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef70068a40 0x7fef7006af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:27.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.143+0000 7fef8d400640 1 --2- 172.21.15.42:0/2282261928 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef70068a40 0x7fef7006af00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0af000 tx=0x7fef7c0a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:27.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.147+0000 7fef86000640 1 -- 172.21.15.42:0/2282261928 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fef80055de0 con 0x7fef90072ba0 2024-05-28T14:03:27.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.347+0000 7fef94a00640 1 -- 172.21.15.42:0/2282261928 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fef90073890 con 0x7fef70068a40 2024-05-28T14:03:27.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.347+0000 7fef86000640 1 -- 172.21.15.42:0/2282261928 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fef90073890 con 0x7fef70068a40 2024-05-28T14:03:27.334 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:03:27.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:27.334 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:03:27.334 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:27.334 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:03:27.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.351+0000 7fef94a00640 1 -- 172.21.15.42:0/2282261928 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef70068a40 msgr2=0x7fef7006af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:27.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.351+0000 7fef94a00640 1 --2- 172.21.15.42:0/2282261928 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef70068a40 0x7fef7006af00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0af000 tx=0x7fef7c0a7000 comp rx=0 tx=0).stop 2024-05-28T14:03:27.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.351+0000 7fef94a00640 1 -- 172.21.15.42:0/2282261928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef90072ba0 msgr2=0x7fef9013cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:27.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.351+0000 7fef94a00640 1 --2- 172.21.15.42:0/2282261928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef90072ba0 0x7fef9013cce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c480 tx=0x7fef8000c940 comp rx=0 tx=0).stop 2024-05-28T14:03:27.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.351+0000 7fef94a00640 1 -- 172.21.15.42:0/2282261928 shutdown_connections 2024-05-28T14:03:27.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.351+0000 7fef94a00640 1 --2- 172.21.15.42:0/2282261928 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef70068a40 0x7fef7006af00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.351+0000 7fef94a00640 1 --2- 172.21.15.42:0/2282261928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9015bf30 0x7fef901faff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.351+0000 7fef94a00640 1 --2- 172.21.15.42:0/2282261928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef900cf230 0x7fef901f83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.351+0000 7fef94a00640 1 --2- 172.21.15.42:0/2282261928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef90072ba0 0x7fef9013cce0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.351+0000 7fef94a00640 1 -- 172.21.15.42:0/2282261928 >> 172.21.15.42:0/2282261928 conn(0x7fef9006b9f0 msgr2=0x7fef90155670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:27.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.351+0000 7fef94a00640 1 -- 172.21.15.42:0/2282261928 shutdown_connections 2024-05-28T14:03:27.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.351+0000 7fef94a00640 1 -- 172.21.15.42:0/2282261928 wait complete. 2024-05-28T14:03:27.355 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:03:27.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.507+0000 7f6423800640 1 Processor -- start 2024-05-28T14:03:27.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.507+0000 7f6423800640 1 -- start start 2024-05-28T14:03:27.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f6423800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f641c0af270 0x7f641c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:27.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f6423800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f641c0b6530 0x7f641c0b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:27.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f6423800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f641c0b1df0 con 0x7f641c06b9f0 2024-05-28T14:03:27.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f6423800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f641c0b20d0 con 0x7f641c0af270 2024-05-28T14:03:27.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f6423800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f641c0b23b0 con 0x7f641c0b6530 2024-05-28T14:03:27.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f6420e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f641c06b9f0 0x7f641c06bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:57634/0 (socket says 172.21.15.42:57634) 2024-05-28T14:03:27.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f6420e00640 1 -- 172.21.15.42:0/2619852738 learned_addr learned my addr 172.21.15.42:0/2619852738 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:03:27.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f6413e00640 1 --2- 172.21.15.42:0/2619852738 >> v2:172.21.15.63:3300/0 conn(0x7f641c0af270 0x7f641c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:27.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f6421800640 1 --2- 172.21.15.42:0/2619852738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f641c0b6530 0x7f641c0b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:27.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f641be00640 1 -- 172.21.15.42:0/2619852738 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 508828744 0 0) 0x7f641c0b1df0 con 0x7f641c06b9f0 2024-05-28T14:03:27.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f641be00640 1 -- 172.21.15.42:0/2619852738 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6400003530 con 0x7f641c06b9f0 2024-05-28T14:03:27.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f6413e00640 1 -- 172.21.15.42:0/2619852738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f641c0b6530 msgr2=0x7f641c0b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:27.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f6413e00640 1 --2- 172.21.15.42:0/2619852738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f641c0b6530 0x7f641c0b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f6413e00640 1 -- 172.21.15.42:0/2619852738 >> v1:172.21.15.42:6789/0 conn(0x7f641c06b9f0 legacy=0x7f641c06bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:27.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.511+0000 7f6413e00640 1 -- 172.21.15.42:0/2619852738 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c0b2690 con 0x7f641c0af270 2024-05-28T14:03:27.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.515+0000 7f6413e00640 1 --2- 172.21.15.42:0/2619852738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f641c0af270 0x7f641c0b5ff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f64040097c0 tx=0x7f640401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af5bade61a7fb78c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:27.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.515+0000 7f641be00640 1 -- 172.21.15.42:0/2619852738 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64040aa5f0 con 0x7f641c0af270 2024-05-28T14:03:27.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.515+0000 7f641be00640 1 -- 172.21.15.42:0/2619852738 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64040af070 con 0x7f641c0af270 2024-05-28T14:03:27.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.515+0000 7f641be00640 1 -- 172.21.15.42:0/2619852738 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64040aa9b0 con 0x7f641c0af270 2024-05-28T14:03:27.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.515+0000 7f6423800640 1 -- 172.21.15.42:0/2619852738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f641c0af270 msgr2=0x7f641c0b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:27.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.515+0000 7f6423800640 1 --2- 172.21.15.42:0/2619852738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f641c0af270 0x7f641c0b5ff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f64040097c0 tx=0x7f640401c470 comp rx=0 tx=0).stop 2024-05-28T14:03:27.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.515+0000 7f6423800640 1 -- 172.21.15.42:0/2619852738 shutdown_connections 2024-05-28T14:03:27.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.515+0000 7f6423800640 1 --2- 172.21.15.42:0/2619852738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f641c0b6530 0x7f641c0b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.515+0000 7f6423800640 1 --2- 172.21.15.42:0/2619852738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f641c0af270 0x7f641c0b5ff0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.515+0000 7f6423800640 1 -- 172.21.15.42:0/2619852738 >> 172.21.15.42:0/2619852738 conn(0x7f641c073290 msgr2=0x7f641c0b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:27.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.515+0000 7f6423800640 1 -- 172.21.15.42:0/2619852738 shutdown_connections 2024-05-28T14:03:27.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.515+0000 7f6423800640 1 -- 172.21.15.42:0/2619852738 wait complete. 2024-05-28T14:03:27.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.515+0000 7f6423800640 1 Processor -- start 2024-05-28T14:03:27.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.515+0000 7f6423800640 1 -- start start 2024-05-28T14:03:27.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6423800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f641c06b9f0 0x7f641c14f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:27.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6423800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f641c0af270 0x7f641c14d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:27.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6423800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f641c0b6530 0x7f641c146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:27.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6423800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f641c0b19f0 con 0x7f641c0af270 2024-05-28T14:03:27.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6423800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f641c0b1870 con 0x7f641c06b9f0 2024-05-28T14:03:27.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6423800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f641c0b1b70 con 0x7f641c0b6530 2024-05-28T14:03:27.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6413e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f641c0af270 0x7f641c14d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:27.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6413e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f641c0af270 0x7f641c14d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37188/0 (socket says 172.21.15.42:37188) 2024-05-28T14:03:27.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6413e00640 1 -- 172.21.15.42:0/1859613769 learned_addr learned my addr 172.21.15.42:0/1859613769 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:27.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6420e00640 1 --2- 172.21.15.42:0/1859613769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f641c06b9f0 0x7f641c14f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:27.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6421800640 1 --2- 172.21.15.42:0/1859613769 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f641c0b6530 0x7f641c146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:27.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6413e00640 1 -- 172.21.15.42:0/1859613769 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f641c0b6530 msgr2=0x7f641c146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:27.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6413e00640 1 --2- 172.21.15.42:0/1859613769 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f641c0b6530 0x7f641c146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6413e00640 1 -- 172.21.15.42:0/1859613769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f641c06b9f0 msgr2=0x7f641c14f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:27.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6413e00640 1 --2- 172.21.15.42:0/1859613769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f641c06b9f0 0x7f641c14f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6413e00640 1 -- 172.21.15.42:0/1859613769 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c15b570 con 0x7f641c0af270 2024-05-28T14:03:27.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6413e00640 1 --2- 172.21.15.42:0/1859613769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f641c0af270 0x7f641c14d540 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f64040ac040 tx=0x7f640401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:27.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6419600640 1 -- 172.21.15.42:0/1859613769 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64040be610 con 0x7f641c0af270 2024-05-28T14:03:27.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6419600640 1 -- 172.21.15.42:0/1859613769 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64040be7b0 con 0x7f641c0af270 2024-05-28T14:03:27.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6419600640 1 -- 172.21.15.42:0/1859613769 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64040beb60 con 0x7f641c0af270 2024-05-28T14:03:27.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6423800640 1 -- 172.21.15.42:0/1859613769 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641c1466b0 con 0x7f641c0af270 2024-05-28T14:03:27.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6423800640 1 -- 172.21.15.42:0/1859613769 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f641c15b290 con 0x7f641c0af270 2024-05-28T14:03:27.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6423800640 1 -- 172.21.15.42:0/1859613769 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f641c068a90 con 0x7f641c0af270 2024-05-28T14:03:27.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.519+0000 7f6419600640 1 -- 172.21.15.42:0/1859613769 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7f64040bc020 con 0x7f641c0af270 2024-05-28T14:03:27.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.523+0000 7f6419600640 1 --2- 172.21.15.42:0/1859613769 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f63f8068a40 0x7f63f806af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:27.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.523+0000 7f6419600640 1 -- 172.21.15.42:0/1859613769 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f6404121b40 con 0x7f641c0af270 2024-05-28T14:03:27.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.523+0000 7f6420e00640 1 --2- 172.21.15.42:0/1859613769 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f63f8068a40 0x7f63f806af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:27.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.523+0000 7f6420e00640 1 --2- 172.21.15.42:0/1859613769 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f63f8068a40 0x7f63f806af00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f640c005890 tx=0x7f640c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:27.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.527+0000 7f6419600640 1 -- 172.21.15.42:0/1859613769 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f64040c5030 con 0x7f641c0af270 2024-05-28T14:03:27.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.751+0000 7f6423800640 1 -- 172.21.15.42:0/1859613769 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f641c06c7d0 con 0x7f641c0af270 2024-05-28T14:03:27.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.751+0000 7f6419600640 1 -- 172.21.15.42:0/1859613769 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2145 (secure 0 0 0) 0x7f641c1466b0 con 0x7f641c0af270 2024-05-28T14:03:27.739 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:03:27.739 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:03:27.739 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:03:27.739 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:03:27.739 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:03:27.739 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:03:27.739 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:03:27.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:27.739 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:03:27.739 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:03:27.739 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:03:27.739 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:03:27.739 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:03:27.740 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:27.740 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:03:27.740 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:03:27.740 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:03:27.740 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:03:27.740 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:27.740 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 329, 2024-05-28T14:03:27.740 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:03:27.740 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:03:27.740 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:03:27.740 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:03:27.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:27.740 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:03:27.740 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 48, 2024-05-28T14:03:27.741 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:03:27.741 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:03:27.741 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:03:27.741 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:03:27.741 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:03:27.741 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:03:27.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:27.741 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:03:27.741 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:03:27.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:27.741 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:03:27.741 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:03:27.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:27.742 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:27.742 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:03:27.742 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:03:27.742 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4038, 2024-05-28T14:03:27.742 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 16787325971, 2024-05-28T14:03:27.742 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 33240084480, 2024-05-28T14:03:27.742 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 739854028800, 2024-05-28T14:03:27.742 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:03:27.742 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 511896767, 2024-05-28T14:03:27.742 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 503563826, 2024-05-28T14:03:27.742 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 124, 2024-05-28T14:03:27.742 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 136 2024-05-28T14:03:27.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:27.743 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:03:27.743 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 28, 2024-05-28T14:03:27.743 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3, 2024-05-28T14:03:27.743 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:03:27.743 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:03:27.743 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:03:27.743 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:03:27.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:27.743 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 3, 2024-05-28T14:03:27.743 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:03:27.743 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:03:27.743 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:03:27.743 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 55184 2024-05-28T14:03:27.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:27.744 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:27.744 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:03:27.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:27.744 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:03:27.744 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:03:27.744 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:03:27.744 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:03:27.744 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:03:27.744 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:03:27.744 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:03:27.744 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:27.744 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:03:27.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:27.745 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:03:27.745 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 10, 2024-05-28T14:03:27.745 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:02:58.913274+0000", 2024-05-28T14:03:27.745 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:03:27.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:27.745 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:03:27.745 INFO:teuthology.orchestra.run.smithi042.stdout: "7d0de869-e5f8-45bf-a6c9-cad4b1fd3c49": { 2024-05-28T14:03:27.745 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "1 ongoing clones; Avg progress is 13.672% (11s)\n [===.........................] (remaining: 71s)", 2024-05-28T14:03:27.745 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.13672000169754028, 2024-05-28T14:03:27.745 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:03:27.745 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:27.745 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:27.745 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:03:27.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.755+0000 7f6423800640 1 -- 172.21.15.42:0/1859613769 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f63f8068a40 msgr2=0x7f63f806af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:27.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.755+0000 7f6423800640 1 --2- 172.21.15.42:0/1859613769 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f63f8068a40 0x7f63f806af00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f640c005890 tx=0x7f640c009040 comp rx=0 tx=0).stop 2024-05-28T14:03:27.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.755+0000 7f6423800640 1 -- 172.21.15.42:0/1859613769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f641c0af270 msgr2=0x7f641c14d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:27.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.755+0000 7f6423800640 1 --2- 172.21.15.42:0/1859613769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f641c0af270 0x7f641c14d540 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f64040ac040 tx=0x7f640401acb0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.755+0000 7f6423800640 1 -- 172.21.15.42:0/1859613769 shutdown_connections 2024-05-28T14:03:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.755+0000 7f6423800640 1 --2- 172.21.15.42:0/1859613769 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f63f8068a40 0x7f63f806af00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.755+0000 7f6423800640 1 --2- 172.21.15.42:0/1859613769 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f641c0b6530 0x7f641c146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.755+0000 7f6423800640 1 --2- 172.21.15.42:0/1859613769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f641c0af270 0x7f641c14d540 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.755+0000 7f6423800640 1 --2- 172.21.15.42:0/1859613769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f641c06b9f0 0x7f641c14f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.755+0000 7f6423800640 1 -- 172.21.15.42:0/1859613769 >> 172.21.15.42:0/1859613769 conn(0x7f641c073290 msgr2=0x7f641c0af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.755+0000 7f6423800640 1 -- 172.21.15.42:0/1859613769 shutdown_connections 2024-05-28T14:03:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:27.755+0000 7f6423800640 1 -- 172.21.15.42:0/1859613769 wait complete. 2024-05-28T14:03:32.762 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:03:32.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f213b200640 1 Processor -- start 2024-05-28T14:03:32.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f213b200640 1 -- start start 2024-05-28T14:03:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f213b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f213c0d2e80 0x7f213c0d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f213b200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f213c15ea80 0x7f213c15ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f213b200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f213c09bc20 con 0x7f213c15e6a0 2024-05-28T14:03:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f213b200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f213c0d06d0 con 0x7f213c15ea80 2024-05-28T14:03:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f213b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f213c0d09b0 con 0x7f213c0d2e80 2024-05-28T14:03:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f2139400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f213c0d2e80 0x7f213c0d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f2139400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f213c0d2e80 0x7f213c0d1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:33530/0 (socket says 172.21.15.42:33530) 2024-05-28T14:03:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f2139400640 1 -- 172.21.15.42:0/2722953750 learned_addr learned my addr 172.21.15.42:0/2722953750 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f213a800640 1 --2- 172.21.15.42:0/2722953750 >> v2:172.21.15.63:3300/0 conn(0x7f213c15ea80 0x7f213c15ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:32.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f2139400640 1 -- 172.21.15.42:0/2722953750 >> v2:172.21.15.63:3300/0 conn(0x7f213c15ea80 msgr2=0x7f213c15ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:32.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f2139400640 1 --2- 172.21.15.42:0/2722953750 >> v2:172.21.15.63:3300/0 conn(0x7f213c15ea80 0x7f213c15ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:32.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f2139400640 1 -- 172.21.15.42:0/2722953750 >> v1:172.21.15.63:6789/0 conn(0x7f213c15e6a0 legacy=0x7f213c09b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:32.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.927+0000 7f2139400640 1 -- 172.21.15.42:0/2722953750 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c0d0c90 con 0x7f213c0d2e80 2024-05-28T14:03:32.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.931+0000 7f2139400640 1 --2- 172.21.15.42:0/2722953750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f213c0d2e80 0x7f213c0d1490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2128009870 tx=0x7f212801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bac0069e3f0b774 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:32.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.931+0000 7f2138a00640 1 -- 172.21.15.42:0/2722953750 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21280a9610 con 0x7f213c0d2e80 2024-05-28T14:03:32.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.931+0000 7f2138a00640 1 -- 172.21.15.42:0/2722953750 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21280a97b0 con 0x7f213c0d2e80 2024-05-28T14:03:32.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.931+0000 7f2138a00640 1 -- 172.21.15.42:0/2722953750 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21280a9b80 con 0x7f213c0d2e80 2024-05-28T14:03:32.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.931+0000 7f213b200640 1 -- 172.21.15.42:0/2722953750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f213c0d2e80 msgr2=0x7f213c0d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:32.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.931+0000 7f213b200640 1 --2- 172.21.15.42:0/2722953750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f213c0d2e80 0x7f213c0d1490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2128009870 tx=0x7f212801c530 comp rx=0 tx=0).stop 2024-05-28T14:03:32.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.931+0000 7f213b200640 1 -- 172.21.15.42:0/2722953750 shutdown_connections 2024-05-28T14:03:32.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.931+0000 7f213b200640 1 --2- 172.21.15.42:0/2722953750 >> v2:172.21.15.63:3300/0 conn(0x7f213c15ea80 0x7f213c15ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:32.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.931+0000 7f213b200640 1 --2- 172.21.15.42:0/2722953750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f213c0d2e80 0x7f213c0d1490 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:32.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.931+0000 7f213b200640 1 -- 172.21.15.42:0/2722953750 >> 172.21.15.42:0/2722953750 conn(0x7f213c0d22f0 msgr2=0x7f213c0d2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:32.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.931+0000 7f213b200640 1 -- 172.21.15.42:0/2722953750 shutdown_connections 2024-05-28T14:03:32.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.931+0000 7f213b200640 1 -- 172.21.15.42:0/2722953750 wait complete. 2024-05-28T14:03:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f213b200640 1 Processor -- start 2024-05-28T14:03:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f213b200640 1 -- start start 2024-05-28T14:03:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f213b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f213c0d2e80 0x7f213c155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f213b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f213c15e6a0 0x7f213c156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f213b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f213c15ea80 0x7f213c0c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f213b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f213c06d6a0 con 0x7f213c0d2e80 2024-05-28T14:03:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f213b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f213c06d520 con 0x7f213c15e6a0 2024-05-28T14:03:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f213b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f213c15b8b0 con 0x7f213c15ea80 2024-05-28T14:03:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f2139e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f213c0d2e80 0x7f213c155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f2139e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f213c0d2e80 0x7f213c155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37196/0 (socket says 172.21.15.42:37196) 2024-05-28T14:03:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f2139e00640 1 -- 172.21.15.42:0/551568566 learned_addr learned my addr 172.21.15.42:0/551568566 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f2139400640 1 --2- 172.21.15.42:0/551568566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f213c15e6a0 0x7f213c156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f213a800640 1 --2- 172.21.15.42:0/551568566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f213c15ea80 0x7f213c0c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f2139e00640 1 -- 172.21.15.42:0/551568566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f213c15ea80 msgr2=0x7f213c0c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f2139e00640 1 --2- 172.21.15.42:0/551568566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f213c15ea80 0x7f213c0c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f2139e00640 1 -- 172.21.15.42:0/551568566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f213c15e6a0 msgr2=0x7f213c156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f2139e00640 1 --2- 172.21.15.42:0/551568566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f213c15e6a0 0x7f213c156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f2139e00640 1 -- 172.21.15.42:0/551568566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c14d2a0 con 0x7f213c0d2e80 2024-05-28T14:03:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f2139e00640 1 --2- 172.21.15.42:0/551568566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f213c0d2e80 0x7f213c155f30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f212c00c450 tx=0x7f212c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f2132000640 1 -- 172.21.15.42:0/551568566 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c016020 con 0x7f213c0d2e80 2024-05-28T14:03:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f2132000640 1 -- 172.21.15.42:0/551568566 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f212c0052d0 con 0x7f213c0d2e80 2024-05-28T14:03:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f213b200640 1 -- 172.21.15.42:0/551568566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213c14d580 con 0x7f213c0d2e80 2024-05-28T14:03:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f2132000640 1 -- 172.21.15.42:0/551568566 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c007260 con 0x7f213c0d2e80 2024-05-28T14:03:32.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f213b200640 1 -- 172.21.15.42:0/551568566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f213c15bde0 con 0x7f213c0d2e80 2024-05-28T14:03:32.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.935+0000 7f213b200640 1 -- 172.21.15.42:0/551568566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f213c068a90 con 0x7f213c0d2e80 2024-05-28T14:03:32.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.939+0000 7f2132000640 1 -- 172.21.15.42:0/551568566 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7f212c007400 con 0x7f213c0d2e80 2024-05-28T14:03:32.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.939+0000 7f2132000640 1 --2- 172.21.15.42:0/551568566 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f211c068a40 0x7f211c06af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:32.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.939+0000 7f2132000640 1 -- 172.21.15.42:0/551568566 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f212c086ee0 con 0x7f213c0d2e80 2024-05-28T14:03:32.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.939+0000 7f2139400640 1 --2- 172.21.15.42:0/551568566 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f211c068a40 0x7f211c06af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:32.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.939+0000 7f2139400640 1 --2- 172.21.15.42:0/551568566 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f211c068a40 0x7f211c06af00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f21280ae000 tx=0x7f21280b3040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:32.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:32.943+0000 7f2132000640 1 -- 172.21.15.42:0/551568566 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f212c020020 con 0x7f213c0d2e80 2024-05-28T14:03:33.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:33.163+0000 7f213b200640 1 -- 172.21.15.42:0/551568566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f213c0b0330 con 0x7f213c0d2e80 2024-05-28T14:03:33.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:33.167+0000 7f2132000640 1 -- 172.21.15.42:0/551568566 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2127 (secure 0 0 0) 0x7f212c005480 con 0x7f213c0d2e80 2024-05-28T14:03:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:03:33.152 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:03:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:03:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:03:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:03:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:03:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:03:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:03:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:03:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 334, 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:03:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 48, 2024-05-28T14:03:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:03:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:03:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:03:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:03:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:03:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:03:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:03:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:03:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:03:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:03:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4373, 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 18205143015, 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 36745637888, 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 736348475392, 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 627308689, 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 621780943, 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 153, 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 170 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 28, 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3, 2024-05-28T14:03:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 3, 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 55184 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:03:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 10, 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:02:58.913274+0000", 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "7d0de869-e5f8-45bf-a6c9-cad4b1fd3c49": { 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 27.0 seconds before exiting (15s)\n [============================] ", 2024-05-28T14:03:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:03:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:03:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:33.158 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:03:33.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:33.167+0000 7f213b200640 1 -- 172.21.15.42:0/551568566 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f211c068a40 msgr2=0x7f211c06af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:33.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:33.167+0000 7f213b200640 1 --2- 172.21.15.42:0/551568566 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f211c068a40 0x7f211c06af00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f21280ae000 tx=0x7f21280b3040 comp rx=0 tx=0).stop 2024-05-28T14:03:33.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:33.167+0000 7f213b200640 1 -- 172.21.15.42:0/551568566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f213c0d2e80 msgr2=0x7f213c155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:33.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:33.167+0000 7f213b200640 1 --2- 172.21.15.42:0/551568566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f213c0d2e80 0x7f213c155f30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f212c00c450 tx=0x7f212c00c910 comp rx=0 tx=0).stop 2024-05-28T14:03:33.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:33.167+0000 7f213b200640 1 -- 172.21.15.42:0/551568566 shutdown_connections 2024-05-28T14:03:33.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:33.167+0000 7f213b200640 1 --2- 172.21.15.42:0/551568566 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f211c068a40 0x7f211c06af00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:33.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:33.167+0000 7f213b200640 1 --2- 172.21.15.42:0/551568566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f213c15ea80 0x7f213c0c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:33.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:33.167+0000 7f213b200640 1 --2- 172.21.15.42:0/551568566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f213c15e6a0 0x7f213c156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:33.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:33.167+0000 7f213b200640 1 --2- 172.21.15.42:0/551568566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f213c0d2e80 0x7f213c155f30 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:33.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:33.167+0000 7f213b200640 1 -- 172.21.15.42:0/551568566 >> 172.21.15.42:0/551568566 conn(0x7f213c0d22f0 msgr2=0x7f213c0b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:33.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:33.167+0000 7f213b200640 1 -- 172.21.15.42:0/551568566 shutdown_connections 2024-05-28T14:03:33.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:33.167+0000 7f213b200640 1 -- 172.21.15.42:0/551568566 wait complete. 2024-05-28T14:03:38.174 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:03:38.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.339+0000 7fbd3ae00640 1 Processor -- start 2024-05-28T14:03:38.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.339+0000 7fbd3ae00640 1 -- start start 2024-05-28T14:03:38.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.339+0000 7fbd3ae00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbd34156c00 0x7fbd3406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:38.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.339+0000 7fbd3ae00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbd3415d8d0 0x7fbd340aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:38.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.339+0000 7fbd3ae00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd3406f6a0 con 0x7fbd340af500 2024-05-28T14:03:38.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.339+0000 7fbd3ae00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fbd3406e600 con 0x7fbd34156c00 2024-05-28T14:03:38.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.339+0000 7fbd3ae00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbd3406e8e0 con 0x7fbd3415d8d0 2024-05-28T14:03:38.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.343+0000 7fbd33e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbd34156c00 0x7fbd3406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:38.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.343+0000 7fbd38e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fbd340af500 0x7fbd341601a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:52352/0 (socket says 172.21.15.42:52352) 2024-05-28T14:03:38.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.343+0000 7fbd38e00640 1 -- 172.21.15.42:0/2423131411 learned_addr learned my addr 172.21.15.42:0/2423131411 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:03:38.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.343+0000 7fbd2be00640 1 --2- 172.21.15.42:0/2423131411 >> v2:172.21.15.63:3300/0 conn(0x7fbd3415d8d0 0x7fbd340aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.343+0000 7fbd33e00640 1 -- 172.21.15.42:0/2423131411 >> v2:172.21.15.63:3300/0 conn(0x7fbd3415d8d0 msgr2=0x7fbd340aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.343+0000 7fbd33e00640 1 --2- 172.21.15.42:0/2423131411 >> v2:172.21.15.63:3300/0 conn(0x7fbd3415d8d0 0x7fbd340aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.343+0000 7fbd33e00640 1 -- 172.21.15.42:0/2423131411 >> v1:172.21.15.42:6789/0 conn(0x7fbd340af500 legacy=0x7fbd341601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.343+0000 7fbd33e00640 1 -- 172.21.15.42:0/2423131411 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3406ebc0 con 0x7fbd34156c00 2024-05-28T14:03:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.343+0000 7fbd33400640 1 -- 172.21.15.42:0/2423131411 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3012517330 0 0) 0x7fbd3406f6a0 con 0x7fbd340af500 2024-05-28T14:03:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.343+0000 7fbd2be00640 1 --2- 172.21.15.42:0/2423131411 >> v2:172.21.15.63:3300/0 conn(0x7fbd3415d8d0 0x7fbd340aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:03:38.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.343+0000 7fbd33e00640 1 --2- 172.21.15.42:0/2423131411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd34156c00 0x7fbd3406dd00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbd340741e0 tx=0x7fbd2401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd83cbe7398d1d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:38.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.343+0000 7fbd33400640 1 -- 172.21.15.42:0/2423131411 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd240b0070 con 0x7fbd34156c00 2024-05-28T14:03:38.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.343+0000 7fbd33400640 1 -- 172.21.15.42:0/2423131411 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd240a4e50 con 0x7fbd34156c00 2024-05-28T14:03:38.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.343+0000 7fbd33400640 1 -- 172.21.15.42:0/2423131411 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd240ab820 con 0x7fbd34156c00 2024-05-28T14:03:38.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2423131411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd34156c00 msgr2=0x7fbd3406dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:38.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 --2- 172.21.15.42:0/2423131411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd34156c00 0x7fbd3406dd00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbd340741e0 tx=0x7fbd2401c3f0 comp rx=0 tx=0).stop 2024-05-28T14:03:38.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2423131411 shutdown_connections 2024-05-28T14:03:38.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 --2- 172.21.15.42:0/2423131411 >> v2:172.21.15.63:3300/0 conn(0x7fbd3415d8d0 0x7fbd340aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:38.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 --2- 172.21.15.42:0/2423131411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd34156c00 0x7fbd3406dd00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:38.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2423131411 >> 172.21.15.42:0/2423131411 conn(0x7fbd3406bff0 msgr2=0x7fbd340b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:38.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2423131411 shutdown_connections 2024-05-28T14:03:38.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2423131411 wait complete. 2024-05-28T14:03:38.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 Processor -- start 2024-05-28T14:03:38.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 -- start start 2024-05-28T14:03:38.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd340af500 0x7fbd3414ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:38.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd34156c00 0x7fbd340726f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:38.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd3415d8d0 0x7fbd3415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:38.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3406e240 con 0x7fbd340af500 2024-05-28T14:03:38.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd340af910 con 0x7fbd3415d8d0 2024-05-28T14:03:38.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd3ae00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3406e3c0 con 0x7fbd34156c00 2024-05-28T14:03:38.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd33e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd340af500 0x7fbd3414ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:38.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd33e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd340af500 0x7fbd3414ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57122/0 (socket says 172.21.15.42:57122) 2024-05-28T14:03:38.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd33e00640 1 -- 172.21.15.42:0/2567275157 learned_addr learned my addr 172.21.15.42:0/2567275157 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:38.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd2be00640 1 --2- 172.21.15.42:0/2567275157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd34156c00 0x7fbd340726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:38.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd38e00640 1 --2- 172.21.15.42:0/2567275157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd3415d8d0 0x7fbd3415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:38.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd33e00640 1 -- 172.21.15.42:0/2567275157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd34156c00 msgr2=0x7fbd340726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:38.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd33e00640 1 --2- 172.21.15.42:0/2567275157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd34156c00 0x7fbd340726f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:38.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd33e00640 1 -- 172.21.15.42:0/2567275157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd3415d8d0 msgr2=0x7fbd3415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:38.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.347+0000 7fbd33e00640 1 --2- 172.21.15.42:0/2567275157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd3415d8d0 0x7fbd3415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:38.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.351+0000 7fbd33e00640 1 -- 172.21.15.42:0/2567275157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd340730d0 con 0x7fbd340af500 2024-05-28T14:03:38.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.351+0000 7fbd33e00640 1 --2- 172.21.15.42:0/2567275157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd340af500 0x7fbd3414ebb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbd240ae040 tx=0x7fbd2401ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:38.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.351+0000 7fbd30c00640 1 -- 172.21.15.42:0/2567275157 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd240b0050 con 0x7fbd340af500 2024-05-28T14:03:38.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.351+0000 7fbd30c00640 1 -- 172.21.15.42:0/2567275157 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd240bc070 con 0x7fbd340af500 2024-05-28T14:03:38.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.351+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2567275157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd340733b0 con 0x7fbd340af500 2024-05-28T14:03:38.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.351+0000 7fbd30c00640 1 -- 172.21.15.42:0/2567275157 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd240b78b0 con 0x7fbd340af500 2024-05-28T14:03:38.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.351+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2567275157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd34073970 con 0x7fbd340af500 2024-05-28T14:03:38.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.351+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2567275157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd34073ca0 con 0x7fbd340af500 2024-05-28T14:03:38.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.351+0000 7fbd30c00640 1 -- 172.21.15.42:0/2567275157 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7fbd240b7a50 con 0x7fbd340af500 2024-05-28T14:03:38.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.351+0000 7fbd30c00640 1 --2- 172.21.15.42:0/2567275157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd10068a40 0x7fbd1006af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:38.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.351+0000 7fbd30c00640 1 -- 172.21.15.42:0/2567275157 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fbd24121ce0 con 0x7fbd340af500 2024-05-28T14:03:38.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.351+0000 7fbd2be00640 1 --2- 172.21.15.42:0/2567275157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd10068a40 0x7fbd1006af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:38.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.355+0000 7fbd2be00640 1 --2- 172.21.15.42:0/2567275157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd10068a40 0x7fbd1006af00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbd34154790 tx=0x7fbd1c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:38.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.355+0000 7fbd30c00640 1 -- 172.21.15.42:0/2567275157 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbd24016070 con 0x7fbd340af500 2024-05-28T14:03:38.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.579+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2567275157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbd34068a90 con 0x7fbd340af500 2024-05-28T14:03:38.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.579+0000 7fbd30c00640 1 -- 172.21.15.42:0/2567275157 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2125 (secure 0 0 0) 0x7fbd240f1860 con 0x7fbd340af500 2024-05-28T14:03:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:03:38.566 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:03:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:03:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:03:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:03:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:03:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:03:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:03:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:03:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:03:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:03:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:03:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:03:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:03:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:03:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:03:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 339, 2024-05-28T14:03:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:03:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:03:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:03:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:03:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:03:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 48, 2024-05-28T14:03:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:03:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:03:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:03:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:03:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:03:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:03:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:03:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:03:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:03:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:03:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:03:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:03:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4372, 2024-05-28T14:03:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 18205192935, 2024-05-28T14:03:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 36745715712, 2024-05-28T14:03:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 736348397568, 2024-05-28T14:03:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:03:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 115313529, 2024-05-28T14:03:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 118126809, 2024-05-28T14:03:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 28, 2024-05-28T14:03:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 35 2024-05-28T14:03:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:03:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 28, 2024-05-28T14:03:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3, 2024-05-28T14:03:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:03:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:03:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:03:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:03:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 3, 2024-05-28T14:03:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:03:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:03:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:03:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 55184 2024-05-28T14:03:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:03:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:03:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:03:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:03:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:03:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:03:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:03:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:03:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:03:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:03:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 10, 2024-05-28T14:03:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:02:58.913274+0000", 2024-05-28T14:03:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:03:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:03:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "7d0de869-e5f8-45bf-a6c9-cad4b1fd3c49": { 2024-05-28T14:03:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 21.0 seconds before exiting (21s)\n [============================] ", 2024-05-28T14:03:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:03:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:03:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:38.573 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:03:38.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.583+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2567275157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd10068a40 msgr2=0x7fbd1006af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:38.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.583+0000 7fbd3ae00640 1 --2- 172.21.15.42:0/2567275157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd10068a40 0x7fbd1006af00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbd34154790 tx=0x7fbd1c008040 comp rx=0 tx=0).stop 2024-05-28T14:03:38.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.583+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2567275157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd340af500 msgr2=0x7fbd3414ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:38.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.583+0000 7fbd3ae00640 1 --2- 172.21.15.42:0/2567275157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd340af500 0x7fbd3414ebb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbd240ae040 tx=0x7fbd2401ac30 comp rx=0 tx=0).stop 2024-05-28T14:03:38.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.583+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2567275157 shutdown_connections 2024-05-28T14:03:38.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.583+0000 7fbd3ae00640 1 --2- 172.21.15.42:0/2567275157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd10068a40 0x7fbd1006af00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:38.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.583+0000 7fbd3ae00640 1 --2- 172.21.15.42:0/2567275157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd3415d8d0 0x7fbd3415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:38.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.583+0000 7fbd3ae00640 1 --2- 172.21.15.42:0/2567275157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd34156c00 0x7fbd340726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:38.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.583+0000 7fbd3ae00640 1 --2- 172.21.15.42:0/2567275157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd340af500 0x7fbd3414ebb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:38.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.583+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2567275157 >> 172.21.15.42:0/2567275157 conn(0x7fbd3406bff0 msgr2=0x7fbd340afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:38.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.583+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2567275157 shutdown_connections 2024-05-28T14:03:38.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:38.583+0000 7fbd3ae00640 1 -- 172.21.15.42:0/2567275157 wait complete. 2024-05-28T14:03:43.589 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:03:43.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.755+0000 7f0b00c00640 1 Processor -- start 2024-05-28T14:03:43.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.755+0000 7f0b00c00640 1 -- start start 2024-05-28T14:03:43.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.755+0000 7f0b00c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0afc0af6d0 0x7f0afc167530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:43.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.755+0000 7f0b00c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0afc06f6a0 con 0x7f0afc15d8d0 2024-05-28T14:03:43.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.755+0000 7f0b00c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0afc06e600 con 0x7f0afc156c00 2024-05-28T14:03:43.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.755+0000 7f0b00c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0afc06e8e0 con 0x7f0afc0af6d0 2024-05-28T14:03:43.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.755+0000 7f0afa800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0afc0af6d0 0x7f0afc167530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:43.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.755+0000 7f0afa800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0afc0af6d0 0x7f0afc167530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57126/0 (socket says 172.21.15.42:57126) 2024-05-28T14:03:43.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.755+0000 7f0afa800640 1 -- 172.21.15.42:0/3511949554 learned_addr learned my addr 172.21.15.42:0/3511949554 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:43.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.755+0000 7f0af9400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0afc15d8d0 0x7f0afc0aefc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:52364/0 (socket says 172.21.15.42:52364) 2024-05-28T14:03:43.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.759+0000 7f0afa800640 1 -- 172.21.15.42:0/3511949554 >> v1:172.21.15.63:6789/0 conn(0x7f0afc156c00 legacy=0x7f0afc06dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:43.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.759+0000 7f0afa800640 1 -- 172.21.15.42:0/3511949554 >> v1:172.21.15.42:6789/0 conn(0x7f0afc15d8d0 legacy=0x7f0afc0aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:43.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.759+0000 7f0afa800640 1 -- 172.21.15.42:0/3511949554 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0afc06ebc0 con 0x7f0afc0af6d0 2024-05-28T14:03:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.759+0000 7f0afa800640 1 --2- 172.21.15.42:0/3511949554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0af6d0 0x7f0afc167530 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009e50 tx=0x7f0af401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a9e24e4fed6765 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.759+0000 7f0af8a00640 1 -- 172.21.15.42:0/3511949554 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af40a5660 con 0x7f0afc0af6d0 2024-05-28T14:03:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.759+0000 7f0af8a00640 1 -- 172.21.15.42:0/3511949554 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0af40a5800 con 0x7f0afc0af6d0 2024-05-28T14:03:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.759+0000 7f0af8a00640 1 -- 172.21.15.42:0/3511949554 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af40a5bd0 con 0x7f0afc0af6d0 2024-05-28T14:03:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.759+0000 7f0b00c00640 1 -- 172.21.15.42:0/3511949554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0af6d0 msgr2=0x7f0afc167530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.759+0000 7f0b00c00640 1 --2- 172.21.15.42:0/3511949554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0af6d0 0x7f0afc167530 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009e50 tx=0x7f0af401c9a0 comp rx=0 tx=0).stop 2024-05-28T14:03:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.759+0000 7f0b00c00640 1 -- 172.21.15.42:0/3511949554 shutdown_connections 2024-05-28T14:03:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0b00c00640 1 --2- 172.21.15.42:0/3511949554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0af6d0 0x7f0afc167530 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0b00c00640 1 -- 172.21.15.42:0/3511949554 >> 172.21.15.42:0/3511949554 conn(0x7f0afc06bff0 msgr2=0x7f0afc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0b00c00640 1 -- 172.21.15.42:0/3511949554 shutdown_connections 2024-05-28T14:03:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0b00c00640 1 -- 172.21.15.42:0/3511949554 wait complete. 2024-05-28T14:03:43.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0b00c00640 1 Processor -- start 2024-05-28T14:03:43.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0b00c00640 1 -- start start 2024-05-28T14:03:43.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0b00c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0afc0af6d0 0x7f0afc155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:43.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0b00c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc156c00 0x7f0afc150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:43.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0b00c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0afc15d8d0 0x7f0afc148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:43.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0b00c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc16a820 con 0x7f0afc156c00 2024-05-28T14:03:43.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0b00c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc16a6a0 con 0x7f0afc0af6d0 2024-05-28T14:03:43.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0b00c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0afc16a9a0 con 0x7f0afc15d8d0 2024-05-28T14:03:43.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0af9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc156c00 0x7f0afc150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:43.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0af9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc156c00 0x7f0afc150fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57132/0 (socket says 172.21.15.42:57132) 2024-05-28T14:03:43.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0af9400640 1 -- 172.21.15.42:0/3335776355 learned_addr learned my addr 172.21.15.42:0/3335776355 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:43.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0af9e00640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0afc0af6d0 0x7f0afc155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:43.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0afa800640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0afc15d8d0 0x7f0afc148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:43.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0af9400640 1 -- 172.21.15.42:0/3335776355 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0afc15d8d0 msgr2=0x7f0afc148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:43.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0af9400640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0afc15d8d0 0x7f0afc148690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:43.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0af9400640 1 -- 172.21.15.42:0/3335776355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0afc0af6d0 msgr2=0x7f0afc155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:43.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0af9400640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0afc0af6d0 0x7f0afc155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:43.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0af9400640 1 -- 172.21.15.42:0/3335776355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0afc1670f0 con 0x7f0afc156c00 2024-05-28T14:03:43.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0af9e00640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0afc0af6d0 0x7f0afc155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:03:43.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0afa800640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0afc15d8d0 0x7f0afc148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:03:43.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0af9400640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc156c00 0x7f0afc150fc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800c430 tx=0x7f0ae800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:43.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0af2000640 1 -- 172.21.15.42:0/3335776355 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae80074e0 con 0x7f0afc156c00 2024-05-28T14:03:43.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.763+0000 7f0af2000640 1 -- 172.21.15.42:0/3335776355 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae8007680 con 0x7f0afc156c00 2024-05-28T14:03:43.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.767+0000 7f0b00c00640 1 -- 172.21.15.42:0/3335776355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0afc1507c0 con 0x7f0afc156c00 2024-05-28T14:03:43.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.767+0000 7f0af2000640 1 -- 172.21.15.42:0/3335776355 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8004a10 con 0x7f0afc156c00 2024-05-28T14:03:43.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.767+0000 7f0b00c00640 1 -- 172.21.15.42:0/3335776355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0afc148bd0 con 0x7f0afc156c00 2024-05-28T14:03:43.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.767+0000 7f0b00c00640 1 -- 172.21.15.42:0/3335776355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0afc068a90 con 0x7f0afc156c00 2024-05-28T14:03:43.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.767+0000 7f0af2000640 1 -- 172.21.15.42:0/3335776355 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7f0ae801a020 con 0x7f0afc156c00 2024-05-28T14:03:43.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.767+0000 7f0af2000640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0adc068a40 0x7f0adc06af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:43.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.767+0000 7f0af2000640 1 -- 172.21.15.42:0/3335776355 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f0ae8086e00 con 0x7f0afc156c00 2024-05-28T14:03:43.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.767+0000 7f0af9e00640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0adc068a40 0x7f0adc06af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:43.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.767+0000 7f0af9e00640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0adc068a40 0x7f0adc06af00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0aec001650 tx=0x7f0aec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:43.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.771+0000 7f0af2000640 1 -- 172.21.15.42:0/3335776355 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ae8010300 con 0x7f0afc156c00 2024-05-28T14:03:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.999+0000 7f0b00c00640 1 -- 172.21.15.42:0/3335776355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0afc0b2a40 con 0x7f0afc156c00 2024-05-28T14:03:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.999+0000 7f0af2000640 1 -- 172.21.15.42:0/3335776355 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2111 (secure 0 0 0) 0x7f0ae8056990 con 0x7f0afc156c00 2024-05-28T14:03:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:03:43.985 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:03:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:03:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:03:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:03:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:03:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:03:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:03:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:03:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:03:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:03:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:03:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:03:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:03:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:03:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:03:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 345, 2024-05-28T14:03:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:03:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:03:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:03:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:03:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:03:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 48, 2024-05-28T14:03:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:03:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:03:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:03:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:03:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:03:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:03:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:03:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:03:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:03:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:03:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:03:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:03:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4372, 2024-05-28T14:03:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 18205220586, 2024-05-28T14:03:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 36745773056, 2024-05-28T14:03:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 736348340224, 2024-05-28T14:03:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:03:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 85, 2024-05-28T14:03:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 7080, 2024-05-28T14:03:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:03:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 1 2024-05-28T14:03:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:03:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 28, 2024-05-28T14:03:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3, 2024-05-28T14:03:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:03:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:03:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:03:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:03:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 3, 2024-05-28T14:03:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:03:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:03:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:03:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 55184 2024-05-28T14:03:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:03:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:03:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:03:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:03:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:03:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:03:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:03:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 10, 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:02:58.913274+0000", 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "7d0de869-e5f8-45bf-a6c9-cad4b1fd3c49": { 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 15.0 seconds before exiting (27s)\n [============================] ", 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:43.993 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:03:43.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.999+0000 7f0b00c00640 1 -- 172.21.15.42:0/3335776355 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0adc068a40 msgr2=0x7f0adc06af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:43.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.999+0000 7f0b00c00640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0adc068a40 0x7f0adc06af00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0aec001650 tx=0x7f0aec009040 comp rx=0 tx=0).stop 2024-05-28T14:03:43.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.999+0000 7f0b00c00640 1 -- 172.21.15.42:0/3335776355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc156c00 msgr2=0x7f0afc150fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.999+0000 7f0b00c00640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc156c00 0x7f0afc150fc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800c430 tx=0x7f0ae800c8f0 comp rx=0 tx=0).stop 2024-05-28T14:03:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.999+0000 7f0b00c00640 1 -- 172.21.15.42:0/3335776355 shutdown_connections 2024-05-28T14:03:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.999+0000 7f0b00c00640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0adc068a40 0x7f0adc06af00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.999+0000 7f0b00c00640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0afc15d8d0 0x7f0afc148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.999+0000 7f0b00c00640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc156c00 0x7f0afc150fc0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.999+0000 7f0b00c00640 1 --2- 172.21.15.42:0/3335776355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0afc0af6d0 0x7f0afc155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.999+0000 7f0b00c00640 1 -- 172.21.15.42:0/3335776355 >> 172.21.15.42:0/3335776355 conn(0x7f0afc06bff0 msgr2=0x7f0afc0aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.999+0000 7f0b00c00640 1 -- 172.21.15.42:0/3335776355 shutdown_connections 2024-05-28T14:03:43.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:43.999+0000 7f0b00c00640 1 -- 172.21.15.42:0/3335776355 wait complete. 2024-05-28T14:03:48.283 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:03:48.286 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:03:49.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:03:49.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.171+0000 7fd418400640 1 Processor -- start 2024-05-28T14:03:49.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.171+0000 7fd418400640 1 -- start start 2024-05-28T14:03:49.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd418400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4100b2a30 0x7fd41015d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:49.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd418400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd41006be50 0x7fd41015e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:49.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd418400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd41006c400 con 0x7fd41006b9f0 2024-05-28T14:03:49.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd418400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd41006c6e0 con 0x7fd41006be50 2024-05-28T14:03:49.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd418400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd41015d080 con 0x7fd4100b2a30 2024-05-28T14:03:49.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd415000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd41006b9f0 0x7fd4100cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:35662/0 (socket says 172.21.15.42:35662) 2024-05-28T14:03:49.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd415000640 1 -- 172.21.15.42:0/578082271 learned_addr learned my addr 172.21.15.42:0/578082271 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:03:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd416400640 1 --2- 172.21.15.42:0/578082271 >> v2:172.21.15.63:3300/0 conn(0x7fd41006be50 0x7fd41015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd415a00640 1 --2- 172.21.15.42:0/578082271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4100b2a30 0x7fd41015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 -- 172.21.15.42:0/578082271 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465344286 0 0) 0x7fd41006c400 con 0x7fd41006b9f0 2024-05-28T14:03:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 -- 172.21.15.42:0/578082271 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3f8003530 con 0x7fd41006b9f0 2024-05-28T14:03:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 -- 172.21.15.42:0/578082271 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 600466003 0 0) 0x7fd3f8003530 con 0x7fd41006b9f0 2024-05-28T14:03:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 -- 172.21.15.42:0/578082271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4100b2a30 msgr2=0x7fd41015d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 --2- 172.21.15.42:0/578082271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4100b2a30 0x7fd41015d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 -- 172.21.15.42:0/578082271 >> v2:172.21.15.63:3300/0 conn(0x7fd41006be50 msgr2=0x7fd41015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 --2- 172.21.15.42:0/578082271 >> v2:172.21.15.63:3300/0 conn(0x7fd41006be50 0x7fd41015e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 -- 172.21.15.42:0/578082271 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41015d360 con 0x7fd41006b9f0 2024-05-28T14:03:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 -- 172.21.15.42:0/578082271 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fd3fc0016a0 con 0x7fd41006b9f0 2024-05-28T14:03:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 -- 172.21.15.42:0/578082271 >> v1:172.21.15.42:6789/0 conn(0x7fd41006b9f0 legacy=0x7fd4100cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 --2- 172.21.15.42:0/578082271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3f80039c0 0x7fd3f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 --2- 172.21.15.42:0/578082271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f8004810 0x7fd3f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 --2- 172.21.15.42:0/578082271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4100b2a30 0x7fd3f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 -- 172.21.15.42:0/578082271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd41006c6e0 con 0x7fd3f8004810 2024-05-28T14:03:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 -- 172.21.15.42:0/578082271 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd41015d080 con 0x7fd3f80039c0 2024-05-28T14:03:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd40fe00640 1 -- 172.21.15.42:0/578082271 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f800b9d0 con 0x7fd4100b2a30 2024-05-28T14:03:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd415000640 1 --2- 172.21.15.42:0/578082271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f8004810 0x7fd3f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd415a00640 1 --2- 172.21.15.42:0/578082271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3f80039c0 0x7fd3f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd416400640 1 --2- 172.21.15.42:0/578082271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4100b2a30 0x7fd3f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd415000640 1 -- 172.21.15.42:0/578082271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4100b2a30 msgr2=0x7fd3f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd415000640 1 --2- 172.21.15.42:0/578082271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4100b2a30 0x7fd3f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd415000640 1 -- 172.21.15.42:0/578082271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3f80039c0 msgr2=0x7fd3f8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:49.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd415000640 1 --2- 172.21.15.42:0/578082271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3f80039c0 0x7fd3f8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:49.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.175+0000 7fd415000640 1 -- 172.21.15.42:0/578082271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f8003530 con 0x7fd3f8004810 2024-05-28T14:03:49.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.179+0000 7fd415000640 1 --2- 172.21.15.42:0/578082271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f8004810 0x7fd3f8004c90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00c210 tx=0x7fd3fc01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6203f7d05d208ba server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:49.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.179+0000 7fd40fe00640 1 -- 172.21.15.42:0/578082271 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc007d70 con 0x7fd3f8004810 2024-05-28T14:03:49.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.179+0000 7fd40fe00640 1 -- 172.21.15.42:0/578082271 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3fc005ae0 con 0x7fd3f8004810 2024-05-28T14:03:49.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.179+0000 7fd40fe00640 1 -- 172.21.15.42:0/578082271 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc007070 con 0x7fd3f8004810 2024-05-28T14:03:49.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.179+0000 7fd418400640 1 -- 172.21.15.42:0/578082271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f8004810 msgr2=0x7fd3f8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:49.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.179+0000 7fd418400640 1 --2- 172.21.15.42:0/578082271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f8004810 0x7fd3f8004c90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00c210 tx=0x7fd3fc01f3f0 comp rx=0 tx=0).stop 2024-05-28T14:03:49.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.179+0000 7fd416400640 1 -- 172.21.15.42:0/578082271 reap_dead start 2024-05-28T14:03:49.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.179+0000 7fd418400640 1 -- 172.21.15.42:0/578082271 shutdown_connections 2024-05-28T14:03:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.179+0000 7fd418400640 1 -- 172.21.15.42:0/578082271 >> 172.21.15.42:0/578082271 conn(0x7fd41009aa50 msgr2=0x7fd4100b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.179+0000 7fd418400640 1 -- 172.21.15.42:0/578082271 shutdown_connections 2024-05-28T14:03:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.179+0000 7fd418400640 1 -- 172.21.15.42:0/578082271 wait complete. 2024-05-28T14:03:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd418400640 1 Processor -- start 2024-05-28T14:03:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd418400640 1 -- start start 2024-05-28T14:03:49.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd418400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f8004810 0x7fd4101f8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:49.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd418400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4101f4290 0x7fd4100c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:49.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd418400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4100c7150 0x7fd41015b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:49.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd418400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd4100b3e50 con 0x7fd3f8004810 2024-05-28T14:03:49.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd418400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd4100b3cd0 con 0x7fd4101f4290 2024-05-28T14:03:49.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd418400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd4100cd770 con 0x7fd4100c7150 2024-05-28T14:03:49.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd415a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f8004810 0x7fd4101f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:49.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd415a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f8004810 0x7fd4101f8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48986/0 (socket says 172.21.15.42:48986) 2024-05-28T14:03:49.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd415a00640 1 -- 172.21.15.42:0/407915761 learned_addr learned my addr 172.21.15.42:0/407915761 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:49.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd416400640 1 --2- 172.21.15.42:0/407915761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4100c7150 0x7fd41015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:49.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd415000640 1 --2- 172.21.15.42:0/407915761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4101f4290 0x7fd4100c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:49.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd415a00640 1 -- 172.21.15.42:0/407915761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4100c7150 msgr2=0x7fd41015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:49.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd415a00640 1 --2- 172.21.15.42:0/407915761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4100c7150 0x7fd41015b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:49.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd415a00640 1 -- 172.21.15.42:0/407915761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4101f4290 msgr2=0x7fd4100c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:49.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd415a00640 1 --2- 172.21.15.42:0/407915761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4101f4290 0x7fd4100c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:49.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd415a00640 1 -- 172.21.15.42:0/407915761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4100d0a50 con 0x7fd3f8004810 2024-05-28T14:03:49.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd415a00640 1 --2- 172.21.15.42:0/407915761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f8004810 0x7fd4101f8290 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd404011960 tx=0x7fd404011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:49.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd40d600640 1 -- 172.21.15.42:0/407915761 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40400faf0 con 0x7fd3f8004810 2024-05-28T14:03:49.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd40d600640 1 -- 172.21.15.42:0/407915761 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd40400fc90 con 0x7fd3f8004810 2024-05-28T14:03:49.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd418400640 1 -- 172.21.15.42:0/407915761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4101f87d0 con 0x7fd3f8004810 2024-05-28T14:03:49.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd40d600640 1 -- 172.21.15.42:0/407915761 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404014440 con 0x7fd3f8004810 2024-05-28T14:03:49.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd418400640 1 -- 172.21.15.42:0/407915761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4101f4aa0 con 0x7fd3f8004810 2024-05-28T14:03:49.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.183+0000 7fd418400640 1 -- 172.21.15.42:0/407915761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd410068a90 con 0x7fd3f8004810 2024-05-28T14:03:49.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.187+0000 7fd40d600640 1 -- 172.21.15.42:0/407915761 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7fd404012040 con 0x7fd3f8004810 2024-05-28T14:03:49.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.187+0000 7fd40d600640 1 --2- 172.21.15.42:0/407915761 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3f0068a40 0x7fd3f006af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:49.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.187+0000 7fd40d600640 1 -- 172.21.15.42:0/407915761 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fd404087c00 con 0x7fd3f8004810 2024-05-28T14:03:49.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.187+0000 7fd415000640 1 --2- 172.21.15.42:0/407915761 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3f0068a40 0x7fd3f006af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:49.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.187+0000 7fd415000640 1 --2- 172.21.15.42:0/407915761 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3f0068a40 0x7fd3f006af00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0ae000 tx=0x7fd3fc001a70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:49.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.191+0000 7fd40d600640 1 -- 172.21.15.42:0/407915761 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd4101f87d0 con 0x7fd3f8004810 2024-05-28T14:03:49.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.415+0000 7fd418400640 1 -- 172.21.15.42:0/407915761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd410070560 con 0x7fd3f8004810 2024-05-28T14:03:49.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.415+0000 7fd40d600640 1 -- 172.21.15.42:0/407915761 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2080 (secure 0 0 0) 0x7fd4100d0a50 con 0x7fd3f8004810 2024-05-28T14:03:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:03:49.403 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:03:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:03:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:03:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:03:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:03:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:03:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:03:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:03:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:03:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:03:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:03:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:03:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:03:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:03:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:03:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 350, 2024-05-28T14:03:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:03:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:03:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:03:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:03:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:03:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 48, 2024-05-28T14:03:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:03:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:03:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:03:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:03:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:03:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:03:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:03:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:03:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:03:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:03:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:03:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:03:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4372, 2024-05-28T14:03:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 18205220586, 2024-05-28T14:03:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 36745773056, 2024-05-28T14:03:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 736348340224, 2024-05-28T14:03:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:03:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 2559, 2024-05-28T14:03:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:03:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:03:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:03:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 28, 2024-05-28T14:03:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3, 2024-05-28T14:03:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:03:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:03:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:03:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:03:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 3, 2024-05-28T14:03:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:03:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:03:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:03:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 55184 2024-05-28T14:03:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:03:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:03:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:03:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:03:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:03:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:03:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:03:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:03:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:03:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:03:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 10, 2024-05-28T14:03:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:02:58.913274+0000", 2024-05-28T14:03:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:03:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:03:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "7d0de869-e5f8-45bf-a6c9-cad4b1fd3c49": { 2024-05-28T14:03:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 9.0 seconds before exiting (33s)\n [============================] ", 2024-05-28T14:03:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:03:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:03:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:49.415 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:03:49.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.415+0000 7fd418400640 1 -- 172.21.15.42:0/407915761 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3f0068a40 msgr2=0x7fd3f006af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:49.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.415+0000 7fd418400640 1 --2- 172.21.15.42:0/407915761 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3f0068a40 0x7fd3f006af00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0ae000 tx=0x7fd3fc001a70 comp rx=0 tx=0).stop 2024-05-28T14:03:49.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.415+0000 7fd418400640 1 -- 172.21.15.42:0/407915761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f8004810 msgr2=0x7fd4101f8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:49.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.415+0000 7fd418400640 1 --2- 172.21.15.42:0/407915761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f8004810 0x7fd4101f8290 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd404011960 tx=0x7fd404011e20 comp rx=0 tx=0).stop 2024-05-28T14:03:49.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.415+0000 7fd418400640 1 -- 172.21.15.42:0/407915761 shutdown_connections 2024-05-28T14:03:49.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.415+0000 7fd418400640 1 --2- 172.21.15.42:0/407915761 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3f0068a40 0x7fd3f006af00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:49.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.415+0000 7fd418400640 1 --2- 172.21.15.42:0/407915761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4100c7150 0x7fd41015b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:49.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.415+0000 7fd418400640 1 --2- 172.21.15.42:0/407915761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4101f4290 0x7fd4100c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:49.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.415+0000 7fd418400640 1 --2- 172.21.15.42:0/407915761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f8004810 0x7fd4101f8290 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:49.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.415+0000 7fd418400640 1 -- 172.21.15.42:0/407915761 >> 172.21.15.42:0/407915761 conn(0x7fd41009aa50 msgr2=0x7fd4100ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:49.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.419+0000 7fd418400640 1 -- 172.21.15.42:0/407915761 shutdown_connections 2024-05-28T14:03:49.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:49.419+0000 7fd418400640 1 -- 172.21.15.42:0/407915761 wait complete. 2024-05-28T14:03:54.423 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:03:54.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.591+0000 7f1e2c800640 1 Processor -- start 2024-05-28T14:03:54.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.591+0000 7f1e2c800640 1 -- start start 2024-05-28T14:03:54.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.591+0000 7f1e2c800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1e24070c10 0x7f1e2406f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:54.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.591+0000 7f1e2c800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e2406fb10 con 0x7f1e2415e7e0 2024-05-28T14:03:54.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.591+0000 7f1e2c800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e240d0210 con 0x7f1e2406f0a0 2024-05-28T14:03:54.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.591+0000 7f1e2c800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1e240b3ed0 con 0x7f1e24070c10 2024-05-28T14:03:54.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.591+0000 7f1e29400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1e24070c10 0x7f1e2406f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:54.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.591+0000 7f1e29400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1e24070c10 0x7f1e2406f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49000/0 (socket says 172.21.15.42:49000) 2024-05-28T14:03:54.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.591+0000 7f1e2a800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1e2415e7e0 0x7f1e240afc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:35666/0 (socket says 172.21.15.42:35666) 2024-05-28T14:03:54.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.591+0000 7f1e29400640 1 -- 172.21.15.42:0/1502043481 learned_addr learned my addr 172.21.15.42:0/1502043481 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:54.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.591+0000 7f1e29400640 1 -- 172.21.15.42:0/1502043481 >> v1:172.21.15.63:6789/0 conn(0x7f1e2406f0a0 legacy=0x7f1e240cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:54.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.591+0000 7f1e29400640 1 -- 172.21.15.42:0/1502043481 >> v1:172.21.15.42:6789/0 conn(0x7f1e2415e7e0 legacy=0x7f1e240afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:54.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.591+0000 7f1e29400640 1 -- 172.21.15.42:0/1502043481 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e240b3bf0 con 0x7f1e24070c10 2024-05-28T14:03:54.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.591+0000 7f1e28a00640 1 -- 172.21.15.42:0/1502043481 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 441091554 0 0) 0x7f1e2406fb10 con 0x7f1e2415e7e0 2024-05-28T14:03:54.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.595+0000 7f1e29400640 1 --2- 172.21.15.42:0/1502043481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e24070c10 0x7f1e2406f480 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009870 tx=0x7f1e1401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cf114019b172be server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:54.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.595+0000 7f1e28a00640 1 -- 172.21.15.42:0/1502043481 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e140b0070 con 0x7f1e24070c10 2024-05-28T14:03:54.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.595+0000 7f1e28a00640 1 -- 172.21.15.42:0/1502043481 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e140ab600 con 0x7f1e24070c10 2024-05-28T14:03:54.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.595+0000 7f1e28a00640 1 -- 172.21.15.42:0/1502043481 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e140ab9d0 con 0x7f1e24070c10 2024-05-28T14:03:54.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.595+0000 7f1e2c800640 1 -- 172.21.15.42:0/1502043481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e24070c10 msgr2=0x7f1e2406f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:54.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.595+0000 7f1e2c800640 1 --2- 172.21.15.42:0/1502043481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e24070c10 0x7f1e2406f480 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009870 tx=0x7f1e1401c4d0 comp rx=0 tx=0).stop 2024-05-28T14:03:54.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.595+0000 7f1e2c800640 1 -- 172.21.15.42:0/1502043481 shutdown_connections 2024-05-28T14:03:54.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.595+0000 7f1e2c800640 1 --2- 172.21.15.42:0/1502043481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e24070c10 0x7f1e2406f480 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:54.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.595+0000 7f1e2c800640 1 -- 172.21.15.42:0/1502043481 >> 172.21.15.42:0/1502043481 conn(0x7f1e240b2f60 msgr2=0x7f1e240b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:54.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.595+0000 7f1e2c800640 1 -- 172.21.15.42:0/1502043481 shutdown_connections 2024-05-28T14:03:54.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.595+0000 7f1e2c800640 1 -- 172.21.15.42:0/1502043481 wait complete. 2024-05-28T14:03:54.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e2c800640 1 Processor -- start 2024-05-28T14:03:54.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e2c800640 1 -- start start 2024-05-28T14:03:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e2c800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e2406f0a0 0x7f1e2414c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e2c800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e24070c10 0x7f1e240c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e2c800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e2415e7e0 0x7f1e240c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e2c800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e240706c0 con 0x7f1e24070c10 2024-05-28T14:03:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e2c800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1e240b0720 con 0x7f1e2415e7e0 2024-05-28T14:03:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e2c800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1e24070840 con 0x7f1e2406f0a0 2024-05-28T14:03:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e29400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e24070c10 0x7f1e240c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e29400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e24070c10 0x7f1e240c87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49002/0 (socket says 172.21.15.42:49002) 2024-05-28T14:03:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e29400640 1 -- 172.21.15.42:0/311682133 learned_addr learned my addr 172.21.15.42:0/311682133 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:03:54.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e2a800640 1 --2- 172.21.15.42:0/311682133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e2415e7e0 0x7f1e240c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:54.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e29e00640 1 --2- 172.21.15.42:0/311682133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e2406f0a0 0x7f1e2414c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:54.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e29400640 1 -- 172.21.15.42:0/311682133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e2406f0a0 msgr2=0x7f1e2414c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:54.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e29400640 1 --2- 172.21.15.42:0/311682133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e2406f0a0 0x7f1e2414c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:54.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e29400640 1 -- 172.21.15.42:0/311682133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e2415e7e0 msgr2=0x7f1e240c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:54.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e29400640 1 --2- 172.21.15.42:0/311682133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e2415e7e0 0x7f1e240c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:54.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e29400640 1 -- 172.21.15.42:0/311682133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2415a9e0 con 0x7f1e24070c10 2024-05-28T14:03:54.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e29400640 1 --2- 172.21.15.42:0/311682133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e24070c10 0x7f1e240c87f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1e140ae040 tx=0x7f1e1401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:54.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e1e000640 1 -- 172.21.15.42:0/311682133 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e140b0040 con 0x7f1e24070c10 2024-05-28T14:03:54.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e1e000640 1 -- 172.21.15.42:0/311682133 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e140bc070 con 0x7f1e24070c10 2024-05-28T14:03:54.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e2c800640 1 -- 172.21.15.42:0/311682133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e240c8d30 con 0x7f1e24070c10 2024-05-28T14:03:54.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e1e000640 1 -- 172.21.15.42:0/311682133 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e140b6880 con 0x7f1e24070c10 2024-05-28T14:03:54.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e2c800640 1 -- 172.21.15.42:0/311682133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e2415af60 con 0x7f1e24070c10 2024-05-28T14:03:54.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.599+0000 7f1e2c800640 1 -- 172.21.15.42:0/311682133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e24068a90 con 0x7f1e24070c10 2024-05-28T14:03:54.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.603+0000 7f1e1e000640 1 -- 172.21.15.42:0/311682133 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7f1e140b6a20 con 0x7f1e24070c10 2024-05-28T14:03:54.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.603+0000 7f1e1e000640 1 --2- 172.21.15.42:0/311682133 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1e08068a40 0x7f1e0806af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:54.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.603+0000 7f1e1e000640 1 -- 172.21.15.42:0/311682133 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f1e141218c0 con 0x7f1e24070c10 2024-05-28T14:03:54.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.603+0000 7f1e29e00640 1 --2- 172.21.15.42:0/311682133 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1e08068a40 0x7f1e0806af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:54.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.603+0000 7f1e29e00640 1 --2- 172.21.15.42:0/311682133 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1e08068a40 0x7f1e0806af00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1e18007fd0 tx=0x7f1e18009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:03:54.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.607+0000 7f1e1e000640 1 -- 172.21.15.42:0/311682133 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e14016150 con 0x7f1e24070c10 2024-05-28T14:03:54.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.831+0000 7f1e2c800640 1 -- 172.21.15.42:0/311682133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1e240cd8d0 con 0x7f1e24070c10 2024-05-28T14:03:54.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.831+0000 7f1e1e000640 1 -- 172.21.15.42:0/311682133 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2079 (secure 0 0 0) 0x7f1e140f0eb0 con 0x7f1e24070c10 2024-05-28T14:03:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:03:54.817 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:03:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:03:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:03:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:03:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:03:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:03:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:03:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:03:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:03:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:03:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:03:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:03:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:03:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:03:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:03:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 356, 2024-05-28T14:03:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:03:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:03:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:03:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:03:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 48, 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:03:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:03:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:03:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4372, 2024-05-28T14:03:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 18205223192, 2024-05-28T14:03:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 36745781248, 2024-05-28T14:03:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 736348332032, 2024-05-28T14:03:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:03:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 341, 2024-05-28T14:03:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:03:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:03:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:03:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 28, 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3, 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 3, 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 55184 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:03:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 10, 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:02:58.913274+0000", 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:03:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:03:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:03:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: "7d0de869-e5f8-45bf-a6c9-cad4b1fd3c49": { 2024-05-28T14:03:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 5.0 seconds before exiting (37s)\n [============================] ", 2024-05-28T14:03:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:03:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:03:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:03:54.823 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:03:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.831+0000 7f1e2c800640 1 -- 172.21.15.42:0/311682133 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1e08068a40 msgr2=0x7f1e0806af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.831+0000 7f1e2c800640 1 --2- 172.21.15.42:0/311682133 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1e08068a40 0x7f1e0806af00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1e18007fd0 tx=0x7f1e18009040 comp rx=0 tx=0).stop 2024-05-28T14:03:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.831+0000 7f1e2c800640 1 -- 172.21.15.42:0/311682133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e24070c10 msgr2=0x7f1e240c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.831+0000 7f1e2c800640 1 --2- 172.21.15.42:0/311682133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e24070c10 0x7f1e240c87f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1e140ae040 tx=0x7f1e1401ad10 comp rx=0 tx=0).stop 2024-05-28T14:03:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.831+0000 7f1e2c800640 1 -- 172.21.15.42:0/311682133 shutdown_connections 2024-05-28T14:03:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.831+0000 7f1e2c800640 1 --2- 172.21.15.42:0/311682133 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1e08068a40 0x7f1e0806af00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.831+0000 7f1e2c800640 1 --2- 172.21.15.42:0/311682133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e2415e7e0 0x7f1e240c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.831+0000 7f1e2c800640 1 --2- 172.21.15.42:0/311682133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e24070c10 0x7f1e240c87f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.831+0000 7f1e2c800640 1 --2- 172.21.15.42:0/311682133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e2406f0a0 0x7f1e2414c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.831+0000 7f1e2c800640 1 -- 172.21.15.42:0/311682133 >> 172.21.15.42:0/311682133 conn(0x7f1e240b2f60 msgr2=0x7f1e2415dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:03:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.831+0000 7f1e2c800640 1 -- 172.21.15.42:0/311682133 shutdown_connections 2024-05-28T14:03:54.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:03:54.831+0000 7f1e2c800640 1 -- 172.21.15.42:0/311682133 wait complete. 2024-05-28T14:03:59.839 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:03:59.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa29600640 1 Processor -- start 2024-05-28T14:03:59.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa29600640 1 -- start start 2024-05-28T14:03:59.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa29600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faa2415c430 0x7faa240cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:59.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa29600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faa24160b20 0x7faa2415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:03:59.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa29600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa2415d820 con 0x7faa2415ff80 2024-05-28T14:03:59.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa29600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7faa240cf160 con 0x7faa24160b20 2024-05-28T14:03:59.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa29600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faa240b0430 con 0x7faa2415c430 2024-05-28T14:03:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa28c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faa24160b20 0x7faa2415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa1be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7faa2415ff80 0x7faa24160410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:54320/0 (socket says 172.21.15.42:54320) 2024-05-28T14:03:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa1be00640 1 -- 172.21.15.42:0/527207654 learned_addr learned my addr 172.21.15.42:0/527207654 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:03:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa23e00640 1 --2- 172.21.15.42:0/527207654 >> v2:172.21.15.63:3300/0 conn(0x7faa2415c430 0x7faa240cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:03:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa28c00640 1 -- 172.21.15.42:0/527207654 >> v2:172.21.15.63:3300/0 conn(0x7faa2415c430 msgr2=0x7faa240cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:03:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa28c00640 1 --2- 172.21.15.42:0/527207654 >> v2:172.21.15.63:3300/0 conn(0x7faa2415c430 0x7faa240cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:03:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa28c00640 1 -- 172.21.15.42:0/527207654 >> v1:172.21.15.42:6789/0 conn(0x7faa2415ff80 legacy=0x7faa24160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:03:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa23400640 1 -- 172.21.15.42:0/527207654 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4092825486 0 0) 0x7faa2415d820 con 0x7faa2415ff80 2024-05-28T14:03:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.011+0000 7faa28c00640 1 -- 172.21.15.42:0/527207654 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa240ce550 con 0x7faa24160b20 2024-05-28T14:04:00.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.015+0000 7faa28c00640 1 --2- 172.21.15.42:0/527207654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa24160b20 0x7faa2415dbc0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7faa240b3940 tx=0x7faa1c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae5ef4cafee6029 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:00.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.015+0000 7faa23400640 1 -- 172.21.15.42:0/527207654 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0b2070 con 0x7faa24160b20 2024-05-28T14:04:00.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.015+0000 7faa23400640 1 -- 172.21.15.42:0/527207654 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1c0a59c0 con 0x7faa24160b20 2024-05-28T14:04:00.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.015+0000 7faa23400640 1 -- 172.21.15.42:0/527207654 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0a5d90 con 0x7faa24160b20 2024-05-28T14:04:00.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.015+0000 7faa29600640 1 -- 172.21.15.42:0/527207654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa24160b20 msgr2=0x7faa2415dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.015+0000 7faa29600640 1 --2- 172.21.15.42:0/527207654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa24160b20 0x7faa2415dbc0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7faa240b3940 tx=0x7faa1c01c900 comp rx=0 tx=0).stop 2024-05-28T14:04:00.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.015+0000 7faa29600640 1 -- 172.21.15.42:0/527207654 shutdown_connections 2024-05-28T14:04:00.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.015+0000 7faa29600640 1 --2- 172.21.15.42:0/527207654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa24160b20 0x7faa2415dbc0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.015+0000 7faa29600640 1 --2- 172.21.15.42:0/527207654 >> v2:172.21.15.63:3300/0 conn(0x7faa2415c430 0x7faa240cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.015+0000 7faa29600640 1 -- 172.21.15.42:0/527207654 >> 172.21.15.42:0/527207654 conn(0x7faa24073280 msgr2=0x7faa2406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:00.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.015+0000 7faa29600640 1 -- 172.21.15.42:0/527207654 shutdown_connections 2024-05-28T14:04:00.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.015+0000 7faa29600640 1 -- 172.21.15.42:0/527207654 wait complete. 2024-05-28T14:04:00.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.015+0000 7faa29600640 1 Processor -- start 2024-05-28T14:04:00.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa29600640 1 -- start start 2024-05-28T14:04:00.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa29600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa2415c430 0x7faa24150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa29600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2415ff80 0x7faa2406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa29600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa24160b20 0x7faa2406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa29600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faa2415ea90 con 0x7faa24160b20 2024-05-28T14:04:00.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa29600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa2415e910 con 0x7faa2415ff80 2024-05-28T14:04:00.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa29600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faa2415ec10 con 0x7faa2415c430 2024-05-28T14:04:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa28c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa24160b20 0x7faa2406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa28c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa24160b20 0x7faa2406ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47556/0 (socket says 172.21.15.42:47556) 2024-05-28T14:04:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa28c00640 1 -- 172.21.15.42:0/1617370772 learned_addr learned my addr 172.21.15.42:0/1617370772 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa1be00640 1 --2- 172.21.15.42:0/1617370772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2415ff80 0x7faa2406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa23e00640 1 --2- 172.21.15.42:0/1617370772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa2415c430 0x7faa24150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa28c00640 1 -- 172.21.15.42:0/1617370772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa2415c430 msgr2=0x7faa24150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa28c00640 1 --2- 172.21.15.42:0/1617370772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa2415c430 0x7faa24150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa28c00640 1 -- 172.21.15.42:0/1617370772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2415ff80 msgr2=0x7faa2406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa28c00640 1 --2- 172.21.15.42:0/1617370772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2415ff80 0x7faa2406c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa28c00640 1 -- 172.21.15.42:0/1617370772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa24153460 con 0x7faa24160b20 2024-05-28T14:04:00.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa28c00640 1 --2- 172.21.15.42:0/1617370772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa24160b20 0x7faa2406ca80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7faa1c019510 tx=0x7faa1c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:00.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa20c00640 1 -- 172.21.15.42:0/1617370772 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0bd020 con 0x7faa24160b20 2024-05-28T14:04:00.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa20c00640 1 -- 172.21.15.42:0/1617370772 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1c0b3040 con 0x7faa24160b20 2024-05-28T14:04:00.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa29600640 1 -- 172.21.15.42:0/1617370772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa24153740 con 0x7faa24160b20 2024-05-28T14:04:00.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa20c00640 1 -- 172.21.15.42:0/1617370772 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0b2050 con 0x7faa24160b20 2024-05-28T14:04:00.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa29600640 1 -- 172.21.15.42:0/1617370772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa24153d00 con 0x7faa24160b20 2024-05-28T14:04:00.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.019+0000 7faa29600640 1 -- 172.21.15.42:0/1617370772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa240691a0 con 0x7faa24160b20 2024-05-28T14:04:00.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.023+0000 7faa20c00640 1 -- 172.21.15.42:0/1617370772 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7faa1c0ad650 con 0x7faa24160b20 2024-05-28T14:04:00.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.023+0000 7faa20c00640 1 --2- 172.21.15.42:0/1617370772 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa00068a40 0x7faa0006af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.023+0000 7faa20c00640 1 -- 172.21.15.42:0/1617370772 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7faa1c1220e0 con 0x7faa24160b20 2024-05-28T14:04:00.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.023+0000 7faa23e00640 1 --2- 172.21.15.42:0/1617370772 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa00068a40 0x7faa0006af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.023+0000 7faa23e00640 1 --2- 172.21.15.42:0/1617370772 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa00068a40 0x7faa0006af00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa14009b60 tx=0x7faa14009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:00.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.027+0000 7faa20c00640 1 -- 172.21.15.42:0/1617370772 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faa1c0c7020 con 0x7faa24160b20 2024-05-28T14:04:00.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.251+0000 7faa29600640 1 -- 172.21.15.42:0/1617370772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7faa24067d00 con 0x7faa24160b20 2024-05-28T14:04:00.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.251+0000 7faa20c00640 1 -- 172.21.15.42:0/1617370772 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1825 (secure 0 0 0) 0x7faa1c016520 con 0x7faa24160b20 2024-05-28T14:04:00.239 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:00.239 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:04:00.239 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:04:00.239 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:04:00.239 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:04:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:04:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:04:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:04:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:04:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:04:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:04:00.240 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:04:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:04:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:04:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:04:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:04:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:04:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:04:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 361, 2024-05-28T14:04:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:04:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:04:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:04:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:04:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:04:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 48, 2024-05-28T14:04:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:04:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:04:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:04:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:04:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:04:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:04:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:04:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:04:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:04:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:04:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:00.242 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:04:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:04:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:04:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4372, 2024-05-28T14:04:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 18205223192, 2024-05-28T14:04:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 36745781248, 2024-05-28T14:04:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 736348332032, 2024-05-28T14:04:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:04:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 341, 2024-05-28T14:04:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:04:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:04:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:04:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 28, 2024-05-28T14:04:00.243 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 3, 2024-05-28T14:04:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:04:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:04:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:04:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:04:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 3, 2024-05-28T14:04:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:04:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:04:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:04:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 55184 2024-05-28T14:04:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:04:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:04:00.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:04:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:04:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:04:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:04:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:04:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:04:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:04:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:04:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:04:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:04:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 10, 2024-05-28T14:04:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:02:58.913274+0000", 2024-05-28T14:04:00.245 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:04:00.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:00.246 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": {} 2024-05-28T14:04:00.246 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:04:00.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.255+0000 7faa29600640 1 -- 172.21.15.42:0/1617370772 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa00068a40 msgr2=0x7faa0006af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.255+0000 7faa29600640 1 --2- 172.21.15.42:0/1617370772 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa00068a40 0x7faa0006af00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa14009b60 tx=0x7faa14009190 comp rx=0 tx=0).stop 2024-05-28T14:04:00.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.255+0000 7faa29600640 1 -- 172.21.15.42:0/1617370772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa24160b20 msgr2=0x7faa2406ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.255+0000 7faa29600640 1 --2- 172.21.15.42:0/1617370772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa24160b20 0x7faa2406ca80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7faa1c019510 tx=0x7faa1c0196a0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.255+0000 7faa29600640 1 -- 172.21.15.42:0/1617370772 shutdown_connections 2024-05-28T14:04:00.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.255+0000 7faa29600640 1 --2- 172.21.15.42:0/1617370772 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa00068a40 0x7faa0006af00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.255+0000 7faa29600640 1 --2- 172.21.15.42:0/1617370772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa24160b20 0x7faa2406ca80 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.255+0000 7faa29600640 1 --2- 172.21.15.42:0/1617370772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2415ff80 0x7faa2406c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.255+0000 7faa29600640 1 --2- 172.21.15.42:0/1617370772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa2415c430 0x7faa24150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.255+0000 7faa29600640 1 -- 172.21.15.42:0/1617370772 >> 172.21.15.42:0/1617370772 conn(0x7faa24073280 msgr2=0x7faa240b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:00.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.255+0000 7faa29600640 1 -- 172.21.15.42:0/1617370772 shutdown_connections 2024-05-28T14:04:00.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.255+0000 7faa29600640 1 -- 172.21.15.42:0/1617370772 wait complete. 2024-05-28T14:04:00.260 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-28T14:04:00.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.415+0000 7f00c9000640 1 Processor -- start 2024-05-28T14:04:00.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.419+0000 7f00c9000640 1 -- start start 2024-05-28T14:04:00.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.419+0000 7f00c9000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00c415c770 0x7f00c409b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.419+0000 7f00c9000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f00c40d11d0 0x7f00c4166470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.419+0000 7f00c9000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00c409ba40 con 0x7f00c415cb50 2024-05-28T14:04:00.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.419+0000 7f00c9000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f00c40c6720 con 0x7f00c40d11d0 2024-05-28T14:04:00.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.419+0000 7f00c9000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f00c40c60d0 con 0x7f00c415c770 2024-05-28T14:04:00.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.419+0000 7f00c2800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f00c40d11d0 0x7f00c4166470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.419+0000 7f00b9400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f00c415cb50 0x7f00c40d0ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:54324/0 (socket says 172.21.15.42:54324) 2024-05-28T14:04:00.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.419+0000 7f00b9400640 1 -- 172.21.15.42:0/1689076818 learned_addr learned my addr 172.21.15.42:0/1689076818 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:00.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.419+0000 7f00c1e00640 1 --2- 172.21.15.42:0/1689076818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00c415c770 0x7f00c409b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.419+0000 7f00c2800640 1 -- 172.21.15.42:0/1689076818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00c415c770 msgr2=0x7f00c409b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:00.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.419+0000 7f00c2800640 1 --2- 172.21.15.42:0/1689076818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00c415c770 0x7f00c409b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.419+0000 7f00c2800640 1 -- 172.21.15.42:0/1689076818 >> v1:172.21.15.42:6789/0 conn(0x7f00c415cb50 legacy=0x7f00c40d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.419+0000 7f00c2800640 1 -- 172.21.15.42:0/1689076818 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c40c63b0 con 0x7f00c40d11d0 2024-05-28T14:04:00.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.423+0000 7f00c2800640 1 --2- 172.21.15.42:0/1689076818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c40d11d0 0x7f00c4166470 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0097c0 tx=0x7f00bc01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459687ff251c1ab0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:00.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.423+0000 7f00c1400640 1 -- 172.21.15.42:0/1689076818 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc0b5070 con 0x7f00c40d11d0 2024-05-28T14:04:00.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.423+0000 7f00c1400640 1 -- 172.21.15.42:0/1689076818 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00bc0a8990 con 0x7f00c40d11d0 2024-05-28T14:04:00.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.423+0000 7f00c1400640 1 -- 172.21.15.42:0/1689076818 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc0a8d60 con 0x7f00c40d11d0 2024-05-28T14:04:00.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.423+0000 7f00c9000640 1 -- 172.21.15.42:0/1689076818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c40d11d0 msgr2=0x7f00c4166470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.423+0000 7f00c9000640 1 --2- 172.21.15.42:0/1689076818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c40d11d0 0x7f00c4166470 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0097c0 tx=0x7f00bc01ec90 comp rx=0 tx=0).stop 2024-05-28T14:04:00.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.423+0000 7f00c9000640 1 -- 172.21.15.42:0/1689076818 shutdown_connections 2024-05-28T14:04:00.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.423+0000 7f00c9000640 1 --2- 172.21.15.42:0/1689076818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c40d11d0 0x7f00c4166470 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.423+0000 7f00c9000640 1 --2- 172.21.15.42:0/1689076818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00c415c770 0x7f00c409b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.423+0000 7f00c9000640 1 -- 172.21.15.42:0/1689076818 >> 172.21.15.42:0/1689076818 conn(0x7f00c40b3fd0 msgr2=0x7f00c40b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:00.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.423+0000 7f00c9000640 1 -- 172.21.15.42:0/1689076818 shutdown_connections 2024-05-28T14:04:00.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.423+0000 7f00c9000640 1 -- 172.21.15.42:0/1689076818 wait complete. 2024-05-28T14:04:00.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.423+0000 7f00c9000640 1 Processor -- start 2024-05-28T14:04:00.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.423+0000 7f00c9000640 1 -- start start 2024-05-28T14:04:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c9000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c40d11d0 0x7f00c40b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c9000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00c415c770 0x7f00c40b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c9000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00c415cb50 0x7f00c41f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c9000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f00c40c8bc0 con 0x7f00c40d11d0 2024-05-28T14:04:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c9000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f00c40ce2c0 con 0x7f00c415cb50 2024-05-28T14:04:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c9000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f00c40c8d40 con 0x7f00c415c770 2024-05-28T14:04:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c40d11d0 0x7f00c40b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c40d11d0 0x7f00c40b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47584/0 (socket says 172.21.15.42:47584) 2024-05-28T14:04:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c1e00640 1 -- 172.21.15.42:0/3170982090 learned_addr learned my addr 172.21.15.42:0/3170982090 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:00.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00b9400640 1 --2- 172.21.15.42:0/3170982090 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00c415c770 0x7f00c40b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c2800640 1 --2- 172.21.15.42:0/3170982090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00c415cb50 0x7f00c41f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c1e00640 1 -- 172.21.15.42:0/3170982090 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00c415c770 msgr2=0x7f00c40b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c1e00640 1 --2- 172.21.15.42:0/3170982090 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00c415c770 0x7f00c40b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c1e00640 1 -- 172.21.15.42:0/3170982090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00c415cb50 msgr2=0x7f00c41f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c1e00640 1 --2- 172.21.15.42:0/3170982090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00c415cb50 0x7f00c41f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c1e00640 1 -- 172.21.15.42:0/3170982090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c40c9750 con 0x7f00c40d11d0 2024-05-28T14:04:00.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c1e00640 1 --2- 172.21.15.42:0/3170982090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c40d11d0 0x7f00c40b69e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009db0 tx=0x7f00b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:00.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00baa00640 1 -- 172.21.15.42:0/3170982090 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4018020 con 0x7f00c40d11d0 2024-05-28T14:04:00.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00baa00640 1 -- 172.21.15.42:0/3170982090 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b4014480 con 0x7f00c40d11d0 2024-05-28T14:04:00.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00baa00640 1 -- 172.21.15.42:0/3170982090 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4014850 con 0x7f00c40d11d0 2024-05-28T14:04:00.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c9000640 1 -- 172.21.15.42:0/3170982090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c40c99f0 con 0x7f00c40d11d0 2024-05-28T14:04:00.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c9000640 1 -- 172.21.15.42:0/3170982090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c40c9fb0 con 0x7f00c40d11d0 2024-05-28T14:04:00.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00c9000640 1 -- 172.21.15.42:0/3170982090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c40a04d0 con 0x7f00c40d11d0 2024-05-28T14:04:00.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.427+0000 7f00baa00640 1 -- 172.21.15.42:0/3170982090 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7f00b401b020 con 0x7f00c40d11d0 2024-05-28T14:04:00.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.431+0000 7f00baa00640 1 --2- 172.21.15.42:0/3170982090 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00a0068ac0 0x7f00a006af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.431+0000 7f00baa00640 1 -- 172.21.15.42:0/3170982090 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f00b40880b0 con 0x7f00c40d11d0 2024-05-28T14:04:00.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.431+0000 7f00b9400640 1 --2- 172.21.15.42:0/3170982090 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00a0068ac0 0x7f00a006af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.431+0000 7f00b9400640 1 --2- 172.21.15.42:0/3170982090 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00a0068ac0 0x7f00a006af80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f00c4157980 tx=0x7f00ac007e90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:00.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.435+0000 7f00baa00640 1 -- 172.21.15.42:0/3170982090 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f00b4022070 con 0x7f00c40d11d0 2024-05-28T14:04:00.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.623+0000 7f00c9000640 1 -- 172.21.15.42:0/3170982090 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00c4160160 con 0x7f00a0068ac0 2024-05-28T14:04:00.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.627+0000 7f00baa00640 1 -- 172.21.15.42:0/3170982090 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+234 (secure 0 0 0) 0x7f00c4160160 con 0x7f00a0068ac0 2024-05-28T14:04:00.614 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:00.614 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:04:00.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:00.614 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sv1" 2024-05-28T14:04:00.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:00.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:00.615 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone85312" 2024-05-28T14:04:00.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:00.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:00.615 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone85311" 2024-05-28T14:04:00.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:00.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:00.615 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone85314" 2024-05-28T14:04:00.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:00.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:00.615 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone85313" 2024-05-28T14:04:00.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:00.615 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:04:00.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.631+0000 7f00c9000640 1 -- 172.21.15.42:0/3170982090 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00a0068ac0 msgr2=0x7f00a006af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.631+0000 7f00c9000640 1 --2- 172.21.15.42:0/3170982090 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00a0068ac0 0x7f00a006af80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f00c4157980 tx=0x7f00ac007e90 comp rx=0 tx=0).stop 2024-05-28T14:04:00.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.631+0000 7f00c9000640 1 -- 172.21.15.42:0/3170982090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c40d11d0 msgr2=0x7f00c40b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.631+0000 7f00c9000640 1 --2- 172.21.15.42:0/3170982090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c40d11d0 0x7f00c40b69e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009db0 tx=0x7f00b400c490 comp rx=0 tx=0).stop 2024-05-28T14:04:00.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.631+0000 7f00c9000640 1 -- 172.21.15.42:0/3170982090 shutdown_connections 2024-05-28T14:04:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.631+0000 7f00c9000640 1 --2- 172.21.15.42:0/3170982090 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00a0068ac0 0x7f00a006af80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.631+0000 7f00c9000640 1 --2- 172.21.15.42:0/3170982090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00c415cb50 0x7f00c41f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.631+0000 7f00c9000640 1 --2- 172.21.15.42:0/3170982090 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00c415c770 0x7f00c40b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.631+0000 7f00c9000640 1 --2- 172.21.15.42:0/3170982090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00c40d11d0 0x7f00c40b69e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.631+0000 7f00c9000640 1 -- 172.21.15.42:0/3170982090 >> 172.21.15.42:0/3170982090 conn(0x7f00c40b3fd0 msgr2=0x7f00c40c6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.631+0000 7f00c9000640 1 -- 172.21.15.42:0/3170982090 shutdown_connections 2024-05-28T14:04:00.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.631+0000 7f00c9000640 1 -- 172.21.15.42:0/3170982090 wait complete. 2024-05-28T14:04:00.636 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-28T14:04:00.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e0d200640 1 Processor -- start 2024-05-28T14:04:00.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e0d200640 1 -- start start 2024-05-28T14:04:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e0d200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e0815b350 0x7f3e0806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e0d200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3e080d0e00 0x7f3e080aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e0d200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e080ade40 con 0x7f3e08072d40 2024-05-28T14:04:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e0d200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f3e080ae9f0 con 0x7f3e080d0e00 2024-05-28T14:04:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e0d200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3e080ae0c0 con 0x7f3e0815b350 2024-05-28T14:04:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e06800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3e080d0e00 0x7f3e080aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e06800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3e080d0e00 0x7f3e080aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47592/0 (socket says 172.21.15.42:47592) 2024-05-28T14:04:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e06800640 1 -- 172.21.15.42:0/54361075 learned_addr learned my addr 172.21.15.42:0/54361075 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e05e00640 1 --2- 172.21.15.42:0/54361075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e0815b350 0x7f3e0806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e06800640 1 -- 172.21.15.42:0/54361075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e0815b350 msgr2=0x7f3e0806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e06800640 1 --2- 172.21.15.42:0/54361075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e0815b350 0x7f3e0806dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e06800640 1 -- 172.21.15.42:0/54361075 >> v1:172.21.15.63:6789/0 conn(0x7f3e08072d40 legacy=0x7f3e080d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e06800640 1 -- 172.21.15.42:0/54361075 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e080ae3a0 con 0x7f3e080d0e00 2024-05-28T14:04:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.795+0000 7f3e05e00640 1 --2- 172.21.15.42:0/54361075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e0815b350 0x7f3e0806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.799+0000 7f3e06800640 1 --2- 172.21.15.42:0/54361075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e080d0e00 0x7f3e080aee80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009e50 tx=0x7f3e0001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2224d30eb6ee24 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.799+0000 7f3e05400640 1 -- 172.21.15.42:0/54361075 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e000a56e0 con 0x7f3e080d0e00 2024-05-28T14:04:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.799+0000 7f3e05400640 1 -- 172.21.15.42:0/54361075 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e000a5880 con 0x7f3e080d0e00 2024-05-28T14:04:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.799+0000 7f3e05400640 1 -- 172.21.15.42:0/54361075 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e000a5c50 con 0x7f3e080d0e00 2024-05-28T14:04:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.799+0000 7f3e0d200640 1 -- 172.21.15.42:0/54361075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e080d0e00 msgr2=0x7f3e080aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.799+0000 7f3e0d200640 1 --2- 172.21.15.42:0/54361075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e080d0e00 0x7f3e080aee80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009e50 tx=0x7f3e0001c9a0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.799+0000 7f3e0d200640 1 -- 172.21.15.42:0/54361075 shutdown_connections 2024-05-28T14:04:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.799+0000 7f3e0d200640 1 --2- 172.21.15.42:0/54361075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e080d0e00 0x7f3e080aee80 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.799+0000 7f3e0d200640 1 --2- 172.21.15.42:0/54361075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e0815b350 0x7f3e0806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.799+0000 7f3e0d200640 1 -- 172.21.15.42:0/54361075 >> 172.21.15.42:0/54361075 conn(0x7f3e0806bdf0 msgr2=0x7f3e0806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.799+0000 7f3e0d200640 1 -- 172.21.15.42:0/54361075 shutdown_connections 2024-05-28T14:04:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.799+0000 7f3e0d200640 1 -- 172.21.15.42:0/54361075 wait complete. 2024-05-28T14:04:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e0d200640 1 Processor -- start 2024-05-28T14:04:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e0d200640 1 -- start start 2024-05-28T14:04:00.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e0d200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e08072d40 0x7f3e08153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e0d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e080d0e00 0x7f3e08159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e0d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e0815b350 0x7f3e08155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e0d200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3e08168470 con 0x7f3e080d0e00 2024-05-28T14:04:00.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e0d200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3e081682f0 con 0x7f3e0815b350 2024-05-28T14:04:00.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e0d200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3e081685f0 con 0x7f3e08072d40 2024-05-28T14:04:00.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e05e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e08072d40 0x7f3e08153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e05e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e08072d40 0x7f3e08153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:38920/0 (socket says 172.21.15.42:38920) 2024-05-28T14:04:00.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3dfd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e080d0e00 0x7f3e08159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e05e00640 1 -- 172.21.15.42:0/2269055776 learned_addr learned my addr 172.21.15.42:0/2269055776 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:00.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e06800640 1 --2- 172.21.15.42:0/2269055776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e0815b350 0x7f3e08155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e05e00640 1 -- 172.21.15.42:0/2269055776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e0815b350 msgr2=0x7f3e08155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e05e00640 1 --2- 172.21.15.42:0/2269055776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e0815b350 0x7f3e08155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e05e00640 1 -- 172.21.15.42:0/2269055776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e080d0e00 msgr2=0x7f3e08159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:00.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e05e00640 1 --2- 172.21.15.42:0/2269055776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e080d0e00 0x7f3e08159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:00.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e05e00640 1 -- 172.21.15.42:0/2269055776 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0815c9f0 con 0x7f3e08072d40 2024-05-28T14:04:00.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e05e00640 1 --2- 172.21.15.42:0/2269055776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e08072d40 0x7f3e08153a60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3df800d800 tx=0x7f3df800dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:00.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3dfea00640 1 -- 172.21.15.42:0/2269055776 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8018020 con 0x7f3e08072d40 2024-05-28T14:04:00.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3dfea00640 1 -- 172.21.15.42:0/2269055776 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df80096c0 con 0x7f3e08072d40 2024-05-28T14:04:00.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3dfea00640 1 -- 172.21.15.42:0/2269055776 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8009ab0 con 0x7f3e08072d40 2024-05-28T14:04:00.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e0d200640 1 -- 172.21.15.42:0/2269055776 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0815ccd0 con 0x7f3e08072d40 2024-05-28T14:04:00.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e0d200640 1 -- 172.21.15.42:0/2269055776 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e08159f00 con 0x7f3e08072d40 2024-05-28T14:04:00.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.803+0000 7f3e0d200640 1 -- 172.21.15.42:0/2269055776 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e0815d1f0 con 0x7f3e08072d40 2024-05-28T14:04:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.807+0000 7f3dfea00640 1 -- 172.21.15.42:0/2269055776 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7f3df801b020 con 0x7f3e08072d40 2024-05-28T14:04:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.807+0000 7f3dfea00640 1 --2- 172.21.15.42:0/2269055776 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de4068a40 0x7f3de406af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.807+0000 7f3dfea00640 1 -- 172.21.15.42:0/2269055776 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f3df8087cb0 con 0x7f3e08072d40 2024-05-28T14:04:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.807+0000 7f3dfd400640 1 --2- 172.21.15.42:0/2269055776 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de4068a40 0x7f3de406af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.807+0000 7f3dfd400640 1 --2- 172.21.15.42:0/2269055776 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de4068a40 0x7f3de406af00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3e08159350 tx=0x7f3df0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:00.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:00.811+0000 7f3dfea00640 1 -- 172.21.15.42:0/2269055776 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3e0815d1f0 con 0x7f3e08072d40 2024-05-28T14:04:00.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.007+0000 7f3e0d200640 1 -- 172.21.15.42:0/2269055776 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3e08068a90 con 0x7f3de4068a40 2024-05-28T14:04:01.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.019+0000 7f3dfea00640 1 -- 172.21.15.42:0/2269055776 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f3e08068a90 con 0x7f3de4068a40 2024-05-28T14:04:01.008 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:01.008 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:04:01.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:01.008 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ss1" 2024-05-28T14:04:01.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:01.008 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:04:01.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.023+0000 7f3e0d200640 1 -- 172.21.15.42:0/2269055776 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de4068a40 msgr2=0x7f3de406af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.023+0000 7f3e0d200640 1 --2- 172.21.15.42:0/2269055776 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de4068a40 0x7f3de406af00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3e08159350 tx=0x7f3df0009040 comp rx=0 tx=0).stop 2024-05-28T14:04:01.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.023+0000 7f3e0d200640 1 -- 172.21.15.42:0/2269055776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e08072d40 msgr2=0x7f3e08153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.023+0000 7f3e0d200640 1 --2- 172.21.15.42:0/2269055776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e08072d40 0x7f3e08153a60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3df800d800 tx=0x7f3df800dcc0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.023+0000 7f3e0d200640 1 -- 172.21.15.42:0/2269055776 shutdown_connections 2024-05-28T14:04:01.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.023+0000 7f3e0d200640 1 --2- 172.21.15.42:0/2269055776 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de4068a40 0x7f3de406af00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.023+0000 7f3e0d200640 1 --2- 172.21.15.42:0/2269055776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e0815b350 0x7f3e08155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.023+0000 7f3e0d200640 1 --2- 172.21.15.42:0/2269055776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e080d0e00 0x7f3e08159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.023+0000 7f3e0d200640 1 --2- 172.21.15.42:0/2269055776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e08072d40 0x7f3e08153a60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.023+0000 7f3e0d200640 1 -- 172.21.15.42:0/2269055776 >> 172.21.15.42:0/2269055776 conn(0x7f3e0806bdf0 msgr2=0x7f3e0815dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:01.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.023+0000 7f3e0d200640 1 -- 172.21.15.42:0/2269055776 shutdown_connections 2024-05-28T14:04:01.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.023+0000 7f3e0d200640 1 -- 172.21.15.42:0/2269055776 wait complete. 2024-05-28T14:04:01.030 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:04:01.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb037200640 1 Processor -- start 2024-05-28T14:04:01.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb037200640 1 -- start start 2024-05-28T14:04:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb037200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb030148090 0x7fb0300b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb037200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb0300724f0 0x7fb0300aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb037200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0300b3690 con 0x7fb0300b4c10 2024-05-28T14:04:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb037200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb030148470 con 0x7fb0300724f0 2024-05-28T14:04:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb037200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb0300b4730 con 0x7fb030148090 2024-05-28T14:04:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb035200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb0300724f0 0x7fb0300aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb035200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb0300724f0 0x7fb0300aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47610/0 (socket says 172.21.15.42:47610) 2024-05-28T14:04:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb035200640 1 -- 172.21.15.42:0/3939784418 learned_addr learned my addr 172.21.15.42:0/3939784418 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:01.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb02fe00640 1 --2- 172.21.15.42:0/3939784418 >> v2:172.21.15.63:3300/0 conn(0x7fb030148090 0x7fb0300b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb035200640 1 -- 172.21.15.42:0/3939784418 >> v2:172.21.15.63:3300/0 conn(0x7fb030148090 msgr2=0x7fb0300b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:01.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb035200640 1 --2- 172.21.15.42:0/3939784418 >> v2:172.21.15.63:3300/0 conn(0x7fb030148090 0x7fb0300b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb035200640 1 -- 172.21.15.42:0/3939784418 >> v1:172.21.15.63:6789/0 conn(0x7fb0300b4c10 legacy=0x7fb030071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.191+0000 7fb035200640 1 -- 172.21.15.42:0/3939784418 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0300b3970 con 0x7fb0300724f0 2024-05-28T14:04:01.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.195+0000 7fb035200640 1 --2- 172.21.15.42:0/3939784418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0300724f0 0x7fb0300aeb40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb030071730 tx=0x7fb02801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3928e23af19653fc server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.195+0000 7fb02f400640 1 -- 172.21.15.42:0/3939784418 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0280a57f0 con 0x7fb0300724f0 2024-05-28T14:04:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.195+0000 7fb02f400640 1 -- 172.21.15.42:0/3939784418 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0280a5990 con 0x7fb0300724f0 2024-05-28T14:04:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.195+0000 7fb02f400640 1 -- 172.21.15.42:0/3939784418 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0280a5d60 con 0x7fb0300724f0 2024-05-28T14:04:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.195+0000 7fb037200640 1 -- 172.21.15.42:0/3939784418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0300724f0 msgr2=0x7fb0300aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.195+0000 7fb037200640 1 --2- 172.21.15.42:0/3939784418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0300724f0 0x7fb0300aeb40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb030071730 tx=0x7fb02801c930 comp rx=0 tx=0).stop 2024-05-28T14:04:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.195+0000 7fb037200640 1 -- 172.21.15.42:0/3939784418 shutdown_connections 2024-05-28T14:04:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.195+0000 7fb037200640 1 --2- 172.21.15.42:0/3939784418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0300724f0 0x7fb0300aeb40 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.195+0000 7fb037200640 1 --2- 172.21.15.42:0/3939784418 >> v2:172.21.15.63:3300/0 conn(0x7fb030148090 0x7fb0300b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.195+0000 7fb037200640 1 -- 172.21.15.42:0/3939784418 >> 172.21.15.42:0/3939784418 conn(0x7fb0300b21f0 msgr2=0x7fb0300b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.195+0000 7fb037200640 1 -- 172.21.15.42:0/3939784418 shutdown_connections 2024-05-28T14:04:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.195+0000 7fb037200640 1 -- 172.21.15.42:0/3939784418 wait complete. 2024-05-28T14:04:01.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb037200640 1 Processor -- start 2024-05-28T14:04:01.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb037200640 1 -- start start 2024-05-28T14:04:01.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb037200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0300724f0 0x7fb030159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb037200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb030148090 0x7fb030162dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb037200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb030143f00 0x7fb03006de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb037200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb0300b57e0 con 0x7fb030143f00 2024-05-28T14:04:01.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb037200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0300b5660 con 0x7fb0300724f0 2024-05-28T14:04:01.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb037200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0300af890 con 0x7fb030148090 2024-05-28T14:04:01.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb035200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb030143f00 0x7fb03006de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb035200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb030143f00 0x7fb03006de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47626/0 (socket says 172.21.15.42:47626) 2024-05-28T14:04:01.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb035200640 1 -- 172.21.15.42:0/494357019 learned_addr learned my addr 172.21.15.42:0/494357019 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:01.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb02fe00640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0300724f0 0x7fb030159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb027400640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb030148090 0x7fb030162dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb035200640 1 -- 172.21.15.42:0/494357019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb030148090 msgr2=0x7fb030162dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb035200640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb030148090 0x7fb030162dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb035200640 1 -- 172.21.15.42:0/494357019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0300724f0 msgr2=0x7fb030159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb035200640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0300724f0 0x7fb030159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb035200640 1 -- 172.21.15.42:0/494357019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03012e9c0 con 0x7fb030143f00 2024-05-28T14:04:01.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb027400640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb030148090 0x7fb030162dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:01.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb035200640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb030143f00 0x7fb03006de10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb0280af040 tx=0x7fb0280196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:01.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb02cc00640 1 -- 172.21.15.42:0/494357019 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0280bc030 con 0x7fb030143f00 2024-05-28T14:04:01.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb02cc00640 1 -- 172.21.15.42:0/494357019 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0280b8470 con 0x7fb030143f00 2024-05-28T14:04:01.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb02cc00640 1 -- 172.21.15.42:0/494357019 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0280b8840 con 0x7fb030143f00 2024-05-28T14:04:01.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb037200640 1 -- 172.21.15.42:0/494357019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03012eca0 con 0x7fb030143f00 2024-05-28T14:04:01.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.199+0000 7fb037200640 1 -- 172.21.15.42:0/494357019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03012f260 con 0x7fb030143f00 2024-05-28T14:04:01.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.203+0000 7fb037200640 1 -- 172.21.15.42:0/494357019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0300738b0 con 0x7fb030143f00 2024-05-28T14:04:01.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.203+0000 7fb02cc00640 1 -- 172.21.15.42:0/494357019 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7fb0280bf070 con 0x7fb030143f00 2024-05-28T14:04:01.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.203+0000 7fb02cc00640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb00c068a40 0x7fb00c06af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.203+0000 7fb02cc00640 1 -- 172.21.15.42:0/494357019 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fb028121d90 con 0x7fb030143f00 2024-05-28T14:04:01.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.203+0000 7fb02fe00640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb00c068a40 0x7fb00c06af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.203+0000 7fb02fe00640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb00c068a40 0x7fb00c06af00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb03006ebf0 tx=0x7fb020009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:01.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.207+0000 7fb02cc00640 1 -- 172.21.15.42:0/494357019 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb028128080 con 0x7fb030143f00 2024-05-28T14:04:01.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.403+0000 7fb037200640 1 -- 172.21.15.42:0/494357019 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb030169140 con 0x7fb00c068a40 2024-05-28T14:04:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.435+0000 7fb02cc00640 1 -- 172.21.15.42:0/494357019 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb030169140 con 0x7fb00c068a40 2024-05-28T14:04:01.422 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:01.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.439+0000 7fb037200640 1 -- 172.21.15.42:0/494357019 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb00c068a40 msgr2=0x7fb00c06af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.439+0000 7fb037200640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb00c068a40 0x7fb00c06af00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb03006ebf0 tx=0x7fb020009190 comp rx=0 tx=0).stop 2024-05-28T14:04:01.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.439+0000 7fb037200640 1 -- 172.21.15.42:0/494357019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb030143f00 msgr2=0x7fb03006de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.439+0000 7fb037200640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb030143f00 0x7fb03006de10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb0280af040 tx=0x7fb0280196d0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.439+0000 7fb037200640 1 -- 172.21.15.42:0/494357019 shutdown_connections 2024-05-28T14:04:01.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.439+0000 7fb037200640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb00c068a40 0x7fb00c06af00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.439+0000 7fb037200640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb030143f00 0x7fb03006de10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.439+0000 7fb037200640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb030148090 0x7fb030162dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.439+0000 7fb037200640 1 --2- 172.21.15.42:0/494357019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0300724f0 0x7fb030159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.439+0000 7fb037200640 1 -- 172.21.15.42:0/494357019 >> 172.21.15.42:0/494357019 conn(0x7fb0300b21f0 msgr2=0x7fb0300b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:01.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.439+0000 7fb037200640 1 -- 172.21.15.42:0/494357019 shutdown_connections 2024-05-28T14:04:01.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.439+0000 7fb037200640 1 -- 172.21.15.42:0/494357019 wait complete. 2024-05-28T14:04:01.443 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-28T14:04:01.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.599+0000 7f0ed4200640 1 Processor -- start 2024-05-28T14:04:01.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.599+0000 7f0ed4200640 1 -- start start 2024-05-28T14:04:01.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.599+0000 7f0ed4200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0ecc071de0 0x7f0ecc0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.599+0000 7f0ed4200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0ecc13fd10 0x7f0ecc140160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.599+0000 7f0ed4200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ecc072700 con 0x7f0ecc0ce3b0 2024-05-28T14:04:01.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.599+0000 7f0ed4200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0ecc160230 con 0x7f0ecc13fd10 2024-05-28T14:04:01.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.599+0000 7f0ed4200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0ecc160510 con 0x7f0ecc071de0 2024-05-28T14:04:01.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.603+0000 7f0ed2200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0ecc13fd10 0x7f0ecc140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.603+0000 7f0ed2200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0ecc13fd10 0x7f0ecc140160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47634/0 (socket says 172.21.15.42:47634) 2024-05-28T14:04:01.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.603+0000 7f0ed2200640 1 -- 172.21.15.42:0/933794153 learned_addr learned my addr 172.21.15.42:0/933794153 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:01.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.603+0000 7f0ed1800640 1 --2- 172.21.15.42:0/933794153 >> v2:172.21.15.63:3300/0 conn(0x7f0ecc071de0 0x7f0ecc0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.603+0000 7f0ed2200640 1 -- 172.21.15.42:0/933794153 >> v2:172.21.15.63:3300/0 conn(0x7f0ecc071de0 msgr2=0x7f0ecc0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:01.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.603+0000 7f0ed2200640 1 --2- 172.21.15.42:0/933794153 >> v2:172.21.15.63:3300/0 conn(0x7f0ecc071de0 0x7f0ecc0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.603+0000 7f0ed2200640 1 -- 172.21.15.42:0/933794153 >> v1:172.21.15.63:6789/0 conn(0x7f0ecc0ce3b0 legacy=0x7f0ecc0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:01.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.603+0000 7f0ed2200640 1 -- 172.21.15.42:0/933794153 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ecc1607f0 con 0x7f0ecc13fd10 2024-05-28T14:04:01.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.603+0000 7f0ed2200640 1 --2- 172.21.15.42:0/933794153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc13fd10 0x7f0ecc140160 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc06cbf0 tx=0x7f0ec401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f092728a7b0b4ef server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:01.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.603+0000 7f0ecbe00640 1 -- 172.21.15.42:0/933794153 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40a5550 con 0x7f0ecc13fd10 2024-05-28T14:04:01.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.603+0000 7f0ecbe00640 1 -- 172.21.15.42:0/933794153 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec40a56f0 con 0x7f0ecc13fd10 2024-05-28T14:04:01.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.603+0000 7f0ecbe00640 1 -- 172.21.15.42:0/933794153 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40a5ac0 con 0x7f0ecc13fd10 2024-05-28T14:04:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 -- 172.21.15.42:0/933794153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc13fd10 msgr2=0x7f0ecc140160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 --2- 172.21.15.42:0/933794153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc13fd10 0x7f0ecc140160 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc06cbf0 tx=0x7f0ec401c890 comp rx=0 tx=0).stop 2024-05-28T14:04:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 -- 172.21.15.42:0/933794153 shutdown_connections 2024-05-28T14:04:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 --2- 172.21.15.42:0/933794153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc13fd10 0x7f0ecc140160 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 --2- 172.21.15.42:0/933794153 >> v2:172.21.15.63:3300/0 conn(0x7f0ecc071de0 0x7f0ecc0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 -- 172.21.15.42:0/933794153 >> 172.21.15.42:0/933794153 conn(0x7f0ecc0b21f0 msgr2=0x7f0ecc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 -- 172.21.15.42:0/933794153 shutdown_connections 2024-05-28T14:04:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 -- 172.21.15.42:0/933794153 wait complete. 2024-05-28T14:04:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 Processor -- start 2024-05-28T14:04:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 -- start start 2024-05-28T14:04:01.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ecc071de0 0x7f0ecc0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc0ce3b0 0x7f0ecc15ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc13fd10 0x7f0ecc148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc169830 con 0x7f0ecc13fd10 2024-05-28T14:04:01.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1696b0 con 0x7f0ecc0ce3b0 2024-05-28T14:04:01.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed4200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0ecc1699b0 con 0x7f0ecc071de0 2024-05-28T14:04:01.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed2200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc13fd10 0x7f0ecc148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed2200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc13fd10 0x7f0ecc148a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47646/0 (socket says 172.21.15.42:47646) 2024-05-28T14:04:01.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed2200640 1 -- 172.21.15.42:0/740896590 learned_addr learned my addr 172.21.15.42:0/740896590 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:01.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed1800640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ecc071de0 0x7f0ecc0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed0e00640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc0ce3b0 0x7f0ecc15ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed2200640 1 -- 172.21.15.42:0/740896590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ecc071de0 msgr2=0x7f0ecc0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.607+0000 7f0ed2200640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ecc071de0 0x7f0ecc0b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ed2200640 1 -- 172.21.15.42:0/740896590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc0ce3b0 msgr2=0x7f0ecc15ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ed2200640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc0ce3b0 0x7f0ecc15ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ed2200640 1 -- 172.21.15.42:0/740896590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ecc0b4310 con 0x7f0ecc13fd10 2024-05-28T14:04:01.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ed1800640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ecc071de0 0x7f0ecc0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:01.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ed0e00640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc0ce3b0 0x7f0ecc15ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:01.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ed2200640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc13fd10 0x7f0ecc148a80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0ec401a4c0 tx=0x7f0ec4019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:01.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ec9600640 1 -- 172.21.15.42:0/740896590 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40bc020 con 0x7f0ecc13fd10 2024-05-28T14:04:01.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ec9600640 1 -- 172.21.15.42:0/740896590 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec40b8690 con 0x7f0ecc13fd10 2024-05-28T14:04:01.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ed4200640 1 -- 172.21.15.42:0/740896590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ecc0b45f0 con 0x7f0ecc13fd10 2024-05-28T14:04:01.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ec9600640 1 -- 172.21.15.42:0/740896590 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4009510 con 0x7f0ecc13fd10 2024-05-28T14:04:01.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ed4200640 1 -- 172.21.15.42:0/740896590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ecc0b4bb0 con 0x7f0ecc13fd10 2024-05-28T14:04:01.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ed4200640 1 -- 172.21.15.42:0/740896590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ecc0738b0 con 0x7f0ecc13fd10 2024-05-28T14:04:01.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ec9600640 1 -- 172.21.15.42:0/740896590 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7f0ec40bf020 con 0x7f0ecc13fd10 2024-05-28T14:04:01.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ec9600640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0eac068a70 0x7f0eac06af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ec9600640 1 -- 172.21.15.42:0/740896590 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f0ec41223e0 con 0x7f0ecc13fd10 2024-05-28T14:04:01.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.611+0000 7f0ed1800640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0eac068a70 0x7f0eac06af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.615+0000 7f0ed1800640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0eac068a70 0x7f0eac06af30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00097a0 tx=0x7f0ec0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:01.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.615+0000 7f0ec9600640 1 -- 172.21.15.42:0/740896590 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ec40165a0 con 0x7f0ecc13fd10 2024-05-28T14:04:01.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.807+0000 7f0ed4200640 1 -- 172.21.15.42:0/740896590 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f0ecc072ed0 con 0x7f0eac068a70 2024-05-28T14:04:01.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.827+0000 7f0ec9600640 1 -- 172.21.15.42:0/740896590 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0ecc072ed0 con 0x7f0eac068a70 2024-05-28T14:04:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.827+0000 7f0ed4200640 1 -- 172.21.15.42:0/740896590 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0eac068a70 msgr2=0x7f0eac06af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.827+0000 7f0ed4200640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0eac068a70 0x7f0eac06af30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00097a0 tx=0x7f0ec0008040 comp rx=0 tx=0).stop 2024-05-28T14:04:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.827+0000 7f0ed4200640 1 -- 172.21.15.42:0/740896590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc13fd10 msgr2=0x7f0ecc148a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.827+0000 7f0ed4200640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc13fd10 0x7f0ecc148a80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0ec401a4c0 tx=0x7f0ec4019630 comp rx=0 tx=0).stop 2024-05-28T14:04:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.827+0000 7f0ed4200640 1 -- 172.21.15.42:0/740896590 shutdown_connections 2024-05-28T14:04:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.827+0000 7f0ed4200640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0eac068a70 0x7f0eac06af30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.827+0000 7f0ed4200640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc13fd10 0x7f0ecc148a80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.827+0000 7f0ed4200640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc0ce3b0 0x7f0ecc15ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.827+0000 7f0ed4200640 1 --2- 172.21.15.42:0/740896590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ecc071de0 0x7f0ecc0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.827+0000 7f0ed4200640 1 -- 172.21.15.42:0/740896590 >> 172.21.15.42:0/740896590 conn(0x7f0ecc0b21f0 msgr2=0x7f0ecc0b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:01.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.827+0000 7f0ed4200640 1 -- 172.21.15.42:0/740896590 shutdown_connections 2024-05-28T14:04:01.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.827+0000 7f0ed4200640 1 -- 172.21.15.42:0/740896590 wait complete. 2024-05-28T14:04:01.817 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:01.831+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:01.817 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:01.831+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:01.817 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:01.831+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:01.817 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:01.831+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:01.817 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:01.831+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:01.835 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:01.847+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:01.835 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:01.847+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:01.835 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:01.847+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:01.835 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:01.847+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:01.835 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:01.847+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:01.838 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone85312 2024-05-28T14:04:01.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:01.999+0000 7fb461e00640 1 Processor -- start 2024-05-28T14:04:01.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.003+0000 7fb461e00640 1 -- start start 2024-05-28T14:04:01.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.003+0000 7fb461e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb45c070870 0x7fb45c15fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.003+0000 7fb461e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb45c06f7f0 con 0x7fb45c156ae0 2024-05-28T14:04:01.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.003+0000 7fb461e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb45c160dd0 con 0x7fb45c15cf70 2024-05-28T14:04:01.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.003+0000 7fb461e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb45c160530 con 0x7fb45c070870 2024-05-28T14:04:01.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.003+0000 7fb453e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb45c070870 0x7fb45c15fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.003+0000 7fb460a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb45c156ae0 0x7fb45c0ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:54336/0 (socket says 172.21.15.42:54336) 2024-05-28T14:04:01.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.003+0000 7fb460a00640 1 -- 172.21.15.42:0/457554924 learned_addr learned my addr 172.21.15.42:0/457554924 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:01.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.003+0000 7fb453e00640 1 -- 172.21.15.42:0/457554924 >> v1:172.21.15.63:6789/0 conn(0x7fb45c15cf70 legacy=0x7fb45c0b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.003+0000 7fb453e00640 1 -- 172.21.15.42:0/457554924 >> v1:172.21.15.42:6789/0 conn(0x7fb45c156ae0 legacy=0x7fb45c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.003+0000 7fb453e00640 1 -- 172.21.15.42:0/457554924 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c160810 con 0x7fb45c070870 2024-05-28T14:04:01.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.007+0000 7fb453e00640 1 --2- 172.21.15.42:0/457554924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c070870 0x7fb45c15fea0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb4440097c0 tx=0x7fb44401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bef6a1d76babf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:01.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.007+0000 7fb45be00640 1 -- 172.21.15.42:0/457554924 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4440b1070 con 0x7fb45c070870 2024-05-28T14:04:01.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.007+0000 7fb45be00640 1 -- 172.21.15.42:0/457554924 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4440ac5f0 con 0x7fb45c070870 2024-05-28T14:04:01.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.007+0000 7fb45be00640 1 -- 172.21.15.42:0/457554924 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4440ac9c0 con 0x7fb45c070870 2024-05-28T14:04:01.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.007+0000 7fb461e00640 1 -- 172.21.15.42:0/457554924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c070870 msgr2=0x7fb45c15fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.007+0000 7fb461e00640 1 --2- 172.21.15.42:0/457554924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c070870 0x7fb45c15fea0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb4440097c0 tx=0x7fb44401c4a0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.007+0000 7fb461e00640 1 -- 172.21.15.42:0/457554924 shutdown_connections 2024-05-28T14:04:01.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.007+0000 7fb461e00640 1 --2- 172.21.15.42:0/457554924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c070870 0x7fb45c15fea0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.007+0000 7fb461e00640 1 -- 172.21.15.42:0/457554924 >> 172.21.15.42:0/457554924 conn(0x7fb45c072dc0 msgr2=0x7fb45c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:01.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.007+0000 7fb461e00640 1 -- 172.21.15.42:0/457554924 shutdown_connections 2024-05-28T14:04:01.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.007+0000 7fb461e00640 1 -- 172.21.15.42:0/457554924 wait complete. 2024-05-28T14:04:01.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.007+0000 7fb461e00640 1 Processor -- start 2024-05-28T14:04:01.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461e00640 1 -- start start 2024-05-28T14:04:01.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45c070870 0x7fb45c14a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb45c156ae0 0x7fb45c06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c15cf70 0x7fb45c06bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:01.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c0b66b0 con 0x7fb45c15cf70 2024-05-28T14:04:01.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c0b6530 con 0x7fb45c070870 2024-05-28T14:04:01.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb45c15d380 con 0x7fb45c156ae0 2024-05-28T14:04:01.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c15cf70 0x7fb45c06bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb460a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45c070870 0x7fb45c14a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c15cf70 0x7fb45c06bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47714/0 (socket says 172.21.15.42:47714) 2024-05-28T14:04:01.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461400640 1 -- 172.21.15.42:0/509623663 learned_addr learned my addr 172.21.15.42:0/509623663 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:01.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb453e00640 1 --2- 172.21.15.42:0/509623663 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb45c156ae0 0x7fb45c06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:01.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461400640 1 -- 172.21.15.42:0/509623663 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb45c156ae0 msgr2=0x7fb45c06b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461400640 1 --2- 172.21.15.42:0/509623663 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb45c156ae0 0x7fb45c06b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461400640 1 -- 172.21.15.42:0/509623663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45c070870 msgr2=0x7fb45c14a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:01.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461400640 1 --2- 172.21.15.42:0/509623663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45c070870 0x7fb45c14a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:01.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461400640 1 -- 172.21.15.42:0/509623663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c15e8f0 con 0x7fb45c15cf70 2024-05-28T14:04:01.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461400640 1 --2- 172.21.15.42:0/509623663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c15cf70 0x7fb45c06bf30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb45400c730 tx=0x7fb45400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:01.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb459600640 1 -- 172.21.15.42:0/509623663 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454003b80 con 0x7fb45c15cf70 2024-05-28T14:04:01.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb459600640 1 -- 172.21.15.42:0/509623663 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb454003d20 con 0x7fb45c15cf70 2024-05-28T14:04:01.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461e00640 1 -- 172.21.15.42:0/509623663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45c14f6f0 con 0x7fb45c15cf70 2024-05-28T14:04:01.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb459600640 1 -- 172.21.15.42:0/509623663 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45400aa30 con 0x7fb45c15cf70 2024-05-28T14:04:01.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461e00640 1 -- 172.21.15.42:0/509623663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45c14fc70 con 0x7fb45c15cf70 2024-05-28T14:04:01.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.011+0000 7fb461e00640 1 -- 172.21.15.42:0/509623663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45c0691a0 con 0x7fb45c15cf70 2024-05-28T14:04:02.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.015+0000 7fb459600640 1 -- 172.21.15.42:0/509623663 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7fb45400abd0 con 0x7fb45c15cf70 2024-05-28T14:04:02.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.015+0000 7fb459600640 1 --2- 172.21.15.42:0/509623663 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb438068a70 0x7fb43806af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:02.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.015+0000 7fb459600640 1 -- 172.21.15.42:0/509623663 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fb4540871e0 con 0x7fb45c15cf70 2024-05-28T14:04:02.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.015+0000 7fb460a00640 1 --2- 172.21.15.42:0/509623663 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb438068a70 0x7fb43806af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:02.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.015+0000 7fb460a00640 1 --2- 172.21.15.42:0/509623663 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb438068a70 0x7fb43806af30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb44c005890 tx=0x7fb44c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:02.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.019+0000 7fb459600640 1 -- 172.21.15.42:0/509623663 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb4540568b0 con 0x7fb45c15cf70 2024-05-28T14:04:02.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.211+0000 7fb461e00640 1 -- 172.21.15.42:0/509623663 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone85312", "target": ["mon-mgr", ""]}) -- 0x7fb45c14fe40 con 0x7fb438068a70 2024-05-28T14:04:02.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.231+0000 7fb459600640 1 -- 172.21.15.42:0/509623663 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb45c14fe40 con 0x7fb438068a70 2024-05-28T14:04:02.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.235+0000 7fb461e00640 1 -- 172.21.15.42:0/509623663 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb438068a70 msgr2=0x7fb43806af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:02.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.235+0000 7fb461e00640 1 --2- 172.21.15.42:0/509623663 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb438068a70 0x7fb43806af30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb44c005890 tx=0x7fb44c009040 comp rx=0 tx=0).stop 2024-05-28T14:04:02.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.235+0000 7fb461e00640 1 -- 172.21.15.42:0/509623663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c15cf70 msgr2=0x7fb45c06bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:02.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.235+0000 7fb461e00640 1 --2- 172.21.15.42:0/509623663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c15cf70 0x7fb45c06bf30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb45400c730 tx=0x7fb45400cbf0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.235+0000 7fb461e00640 1 -- 172.21.15.42:0/509623663 shutdown_connections 2024-05-28T14:04:02.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.235+0000 7fb461e00640 1 --2- 172.21.15.42:0/509623663 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb438068a70 0x7fb43806af30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.235+0000 7fb461e00640 1 --2- 172.21.15.42:0/509623663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c15cf70 0x7fb45c06bf30 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.235+0000 7fb461e00640 1 --2- 172.21.15.42:0/509623663 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb45c156ae0 0x7fb45c06b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.235+0000 7fb461e00640 1 --2- 172.21.15.42:0/509623663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45c070870 0x7fb45c14a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.235+0000 7fb461e00640 1 -- 172.21.15.42:0/509623663 >> 172.21.15.42:0/509623663 conn(0x7fb45c072dc0 msgr2=0x7fb45c1528a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:02.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.235+0000 7fb461e00640 1 -- 172.21.15.42:0/509623663 shutdown_connections 2024-05-28T14:04:02.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.235+0000 7fb461e00640 1 -- 172.21.15.42:0/509623663 wait complete. 2024-05-28T14:04:02.243 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone85311 2024-05-28T14:04:02.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.403+0000 7fe670000640 1 Processor -- start 2024-05-28T14:04:02.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.403+0000 7fe670000640 1 -- start start 2024-05-28T14:04:02.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe670000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66806ee80 0x7fe66806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:02.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe670000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe6680af360 0x7fe6680af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:02.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe670000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6681570a0 con 0x7fe6680d06f0 2024-05-28T14:04:02.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe670000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe66815cf00 con 0x7fe6680af360 2024-05-28T14:04:02.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe670000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe66815d1e0 con 0x7fe66806ee80 2024-05-28T14:04:02.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe66e000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe6680d06f0 0x7fe668160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:54342/0 (socket says 172.21.15.42:54342) 2024-05-28T14:04:02.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe66e000640 1 -- 172.21.15.42:0/4283124993 learned_addr learned my addr 172.21.15.42:0/4283124993 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:02.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe66cc00640 1 --2- 172.21.15.42:0/4283124993 >> v2:172.21.15.63:3300/0 conn(0x7fe6680af360 0x7fe6680af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:02.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe66d600640 1 --2- 172.21.15.42:0/4283124993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66806ee80 0x7fe66806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:02.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe667e00640 1 -- 172.21.15.42:0/4283124993 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3328813860 0 0) 0x7fe6681570a0 con 0x7fe6680d06f0 2024-05-28T14:04:02.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe667e00640 1 -- 172.21.15.42:0/4283124993 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe650003530 con 0x7fe6680d06f0 2024-05-28T14:04:02.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe66cc00640 1 -- 172.21.15.42:0/4283124993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66806ee80 msgr2=0x7fe66806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:02.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe66cc00640 1 --2- 172.21.15.42:0/4283124993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66806ee80 0x7fe66806dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe66cc00640 1 -- 172.21.15.42:0/4283124993 >> v1:172.21.15.42:6789/0 conn(0x7fe6680d06f0 legacy=0x7fe668160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:02.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe66cc00640 1 -- 172.21.15.42:0/4283124993 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66815d4c0 con 0x7fe6680af360 2024-05-28T14:04:02.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.407+0000 7fe66d600640 1 --2- 172.21.15.42:0/4283124993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66806ee80 0x7fe66806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:04:02.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.411+0000 7fe66cc00640 1 --2- 172.21.15.42:0/4283124993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6680af360 0x7fe6680af7e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe6540097c0 tx=0x7fe65401c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39954cf20b88d251 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:02.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.411+0000 7fe667e00640 1 -- 172.21.15.42:0/4283124993 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6540b0070 con 0x7fe6680af360 2024-05-28T14:04:02.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.411+0000 7fe667e00640 1 -- 172.21.15.42:0/4283124993 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6540ab600 con 0x7fe6680af360 2024-05-28T14:04:02.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.411+0000 7fe667e00640 1 -- 172.21.15.42:0/4283124993 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6540a4e60 con 0x7fe6680af360 2024-05-28T14:04:02.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.411+0000 7fe670000640 1 -- 172.21.15.42:0/4283124993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6680af360 msgr2=0x7fe6680af7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:02.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.411+0000 7fe670000640 1 --2- 172.21.15.42:0/4283124993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6680af360 0x7fe6680af7e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe6540097c0 tx=0x7fe65401c380 comp rx=0 tx=0).stop 2024-05-28T14:04:02.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.411+0000 7fe670000640 1 -- 172.21.15.42:0/4283124993 shutdown_connections 2024-05-28T14:04:02.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.411+0000 7fe670000640 1 --2- 172.21.15.42:0/4283124993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6680af360 0x7fe6680af7e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.411+0000 7fe670000640 1 --2- 172.21.15.42:0/4283124993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66806ee80 0x7fe66806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.411+0000 7fe670000640 1 -- 172.21.15.42:0/4283124993 >> 172.21.15.42:0/4283124993 conn(0x7fe66806bdf0 msgr2=0x7fe66806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:02.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.411+0000 7fe670000640 1 -- 172.21.15.42:0/4283124993 shutdown_connections 2024-05-28T14:04:02.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.411+0000 7fe670000640 1 -- 172.21.15.42:0/4283124993 wait complete. 2024-05-28T14:04:02.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.411+0000 7fe670000640 1 Processor -- start 2024-05-28T14:04:02.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.411+0000 7fe670000640 1 -- start start 2024-05-28T14:04:02.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe670000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66806ee80 0x7fe668155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:02.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe670000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6680af360 0x7fe668099900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:02.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe670000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6680d06f0 0x7fe668166340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:02.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe670000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe6680d0f10 con 0x7fe66806ee80 2024-05-28T14:04:02.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe670000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe668160fa0 con 0x7fe6680d06f0 2024-05-28T14:04:02.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe670000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe6681400d0 con 0x7fe6680af360 2024-05-28T14:04:02.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe66d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66806ee80 0x7fe668155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:02.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe66d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66806ee80 0x7fe668155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47716/0 (socket says 172.21.15.42:47716) 2024-05-28T14:04:02.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe66d600640 1 -- 172.21.15.42:0/2202686640 learned_addr learned my addr 172.21.15.42:0/2202686640 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:02.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe66cc00640 1 --2- 172.21.15.42:0/2202686640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6680af360 0x7fe668099900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:02.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe66e000640 1 --2- 172.21.15.42:0/2202686640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6680d06f0 0x7fe668166340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:02.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe66d600640 1 -- 172.21.15.42:0/2202686640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6680af360 msgr2=0x7fe668099900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:02.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe66d600640 1 --2- 172.21.15.42:0/2202686640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6680af360 0x7fe668099900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe66d600640 1 -- 172.21.15.42:0/2202686640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6680d06f0 msgr2=0x7fe668166340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:02.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe66d600640 1 --2- 172.21.15.42:0/2202686640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6680d06f0 0x7fe668166340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe66d600640 1 -- 172.21.15.42:0/2202686640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe668071ec0 con 0x7fe66806ee80 2024-05-28T14:04:02.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe66d600640 1 --2- 172.21.15.42:0/2202686640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66806ee80 0x7fe668155430 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00e980 tx=0x7fe65c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:02.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe665600640 1 -- 172.21.15.42:0/2202686640 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c017070 con 0x7fe66806ee80 2024-05-28T14:04:02.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe665600640 1 -- 172.21.15.42:0/2202686640 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65c00ca00 con 0x7fe66806ee80 2024-05-28T14:04:02.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe670000640 1 -- 172.21.15.42:0/2202686640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6680721a0 con 0x7fe66806ee80 2024-05-28T14:04:02.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe665600640 1 -- 172.21.15.42:0/2202686640 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c00cdd0 con 0x7fe66806ee80 2024-05-28T14:04:02.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe670000640 1 -- 172.21.15.42:0/2202686640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe668155970 con 0x7fe66806ee80 2024-05-28T14:04:02.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.415+0000 7fe670000640 1 -- 172.21.15.42:0/2202686640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe668068a90 con 0x7fe66806ee80 2024-05-28T14:04:02.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.419+0000 7fe665600640 1 -- 172.21.15.42:0/2202686640 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7fe65c01a070 con 0x7fe66806ee80 2024-05-28T14:04:02.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.419+0000 7fe665600640 1 --2- 172.21.15.42:0/2202686640 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe648068a70 0x7fe64806af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:02.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.419+0000 7fe665600640 1 -- 172.21.15.42:0/2202686640 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fe65c087460 con 0x7fe66806ee80 2024-05-28T14:04:02.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.419+0000 7fe66cc00640 1 --2- 172.21.15.42:0/2202686640 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe648068a70 0x7fe64806af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:02.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.419+0000 7fe66cc00640 1 --2- 172.21.15.42:0/2202686640 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe648068a70 0x7fe64806af30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe6540ad000 tx=0x7fe654008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:02.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.423+0000 7fe665600640 1 -- 172.21.15.42:0/2202686640 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe65c0569a0 con 0x7fe66806ee80 2024-05-28T14:04:02.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.619+0000 7fe670000640 1 -- 172.21.15.42:0/2202686640 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone85311", "target": ["mon-mgr", ""]}) -- 0x7fe6680b5160 con 0x7fe648068a70 2024-05-28T14:04:02.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.639+0000 7fe665600640 1 -- 172.21.15.42:0/2202686640 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe6680b5160 con 0x7fe648068a70 2024-05-28T14:04:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.639+0000 7fe670000640 1 -- 172.21.15.42:0/2202686640 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe648068a70 msgr2=0x7fe64806af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.639+0000 7fe670000640 1 --2- 172.21.15.42:0/2202686640 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe648068a70 0x7fe64806af30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe6540ad000 tx=0x7fe654008040 comp rx=0 tx=0).stop 2024-05-28T14:04:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.639+0000 7fe670000640 1 -- 172.21.15.42:0/2202686640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66806ee80 msgr2=0x7fe668155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.639+0000 7fe670000640 1 --2- 172.21.15.42:0/2202686640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66806ee80 0x7fe668155430 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00e980 tx=0x7fe65c00ee40 comp rx=0 tx=0).stop 2024-05-28T14:04:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.639+0000 7fe670000640 1 -- 172.21.15.42:0/2202686640 shutdown_connections 2024-05-28T14:04:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.639+0000 7fe670000640 1 --2- 172.21.15.42:0/2202686640 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe648068a70 0x7fe64806af30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.639+0000 7fe670000640 1 --2- 172.21.15.42:0/2202686640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6680d06f0 0x7fe668166340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.639+0000 7fe670000640 1 --2- 172.21.15.42:0/2202686640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6680af360 0x7fe668099900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.639+0000 7fe670000640 1 --2- 172.21.15.42:0/2202686640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66806ee80 0x7fe668155430 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.643+0000 7fe670000640 1 -- 172.21.15.42:0/2202686640 >> 172.21.15.42:0/2202686640 conn(0x7fe66806bdf0 msgr2=0x7fe668156670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.643+0000 7fe670000640 1 -- 172.21.15.42:0/2202686640 shutdown_connections 2024-05-28T14:04:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.643+0000 7fe670000640 1 -- 172.21.15.42:0/2202686640 wait complete. 2024-05-28T14:04:02.650 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone85314 2024-05-28T14:04:02.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.811+0000 7fe37a400640 1 Processor -- start 2024-05-28T14:04:02.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.811+0000 7fe37a400640 1 -- start start 2024-05-28T14:04:02.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.811+0000 7fe37a400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe37406dc20 0x7fe3740d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:02.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.811+0000 7fe37a400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe37406cd10 0x7fe3740cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:02.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.811+0000 7fe37a400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3740d1970 con 0x7fe37415e840 2024-05-28T14:04:02.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.811+0000 7fe37a400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fe3740b5300 con 0x7fe37406cd10 2024-05-28T14:04:02.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.811+0000 7fe37a400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe3740d1740 con 0x7fe37406dc20 2024-05-28T14:04:02.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.811+0000 7fe373e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe37406cd10 0x7fe3740cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:02.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.811+0000 7fe373e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe37406cd10 0x7fe3740cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47720/0 (socket says 172.21.15.42:47720) 2024-05-28T14:04:02.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.811+0000 7fe373e00640 1 -- 172.21.15.42:0/4097328381 learned_addr learned my addr 172.21.15.42:0/4097328381 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:02.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.811+0000 7fe36be00640 1 --2- 172.21.15.42:0/4097328381 >> v2:172.21.15.63:3300/0 conn(0x7fe37406dc20 0x7fe3740d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:02.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.815+0000 7fe373e00640 1 -- 172.21.15.42:0/4097328381 >> v2:172.21.15.63:3300/0 conn(0x7fe37406dc20 msgr2=0x7fe3740d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:02.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.815+0000 7fe373e00640 1 --2- 172.21.15.42:0/4097328381 >> v2:172.21.15.63:3300/0 conn(0x7fe37406dc20 0x7fe3740d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.815+0000 7fe373e00640 1 -- 172.21.15.42:0/4097328381 >> v1:172.21.15.63:6789/0 conn(0x7fe37415e840 legacy=0x7fe37406d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:02.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.815+0000 7fe373e00640 1 -- 172.21.15.42:0/4097328381 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3740ccca0 con 0x7fe37406cd10 2024-05-28T14:04:02.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.819+0000 7fe373e00640 1 --2- 172.21.15.42:0/4097328381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37406cd10 0x7fe3740cd780 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe3740d3560 tx=0x7fe36c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4592d1ea3e162ae server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:02.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.819+0000 7fe372a00640 1 -- 172.21.15.42:0/4097328381 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0b4070 con 0x7fe37406cd10 2024-05-28T14:04:02.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.819+0000 7fe372a00640 1 -- 172.21.15.42:0/4097328381 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36c0af400 con 0x7fe37406cd10 2024-05-28T14:04:02.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.819+0000 7fe372a00640 1 -- 172.21.15.42:0/4097328381 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0af7d0 con 0x7fe37406cd10 2024-05-28T14:04:02.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.819+0000 7fe37a400640 1 -- 172.21.15.42:0/4097328381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37406cd10 msgr2=0x7fe3740cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:02.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.819+0000 7fe37a400640 1 --2- 172.21.15.42:0/4097328381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37406cd10 0x7fe3740cd780 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe3740d3560 tx=0x7fe36c01f1e0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.819+0000 7fe37a400640 1 -- 172.21.15.42:0/4097328381 shutdown_connections 2024-05-28T14:04:02.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.819+0000 7fe37a400640 1 --2- 172.21.15.42:0/4097328381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37406cd10 0x7fe3740cd780 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.819+0000 7fe37a400640 1 --2- 172.21.15.42:0/4097328381 >> v2:172.21.15.63:3300/0 conn(0x7fe37406dc20 0x7fe3740d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.819+0000 7fe37a400640 1 -- 172.21.15.42:0/4097328381 >> 172.21.15.42:0/4097328381 conn(0x7fe3740b3fa0 msgr2=0x7fe3740b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.819+0000 7fe37a400640 1 -- 172.21.15.42:0/4097328381 shutdown_connections 2024-05-28T14:04:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.819+0000 7fe37a400640 1 -- 172.21.15.42:0/4097328381 wait complete. 2024-05-28T14:04:02.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.819+0000 7fe37a400640 1 Processor -- start 2024-05-28T14:04:02.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.819+0000 7fe37a400640 1 -- start start 2024-05-28T14:04:02.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe37a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37406cd10 0x7fe3741f8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe37a400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe37406dc20 0x7fe37414e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe37a400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37415e840 0x7fe3740cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe37a400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe37415ec20 con 0x7fe37406cd10 2024-05-28T14:04:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe37a400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe3740cd260 con 0x7fe37415e840 2024-05-28T14:04:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe37a400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe37415eda0 con 0x7fe37406dc20 2024-05-28T14:04:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe373400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37406cd10 0x7fe3741f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe373400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37406cd10 0x7fe3741f8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47728/0 (socket says 172.21.15.42:47728) 2024-05-28T14:04:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe373400640 1 -- 172.21.15.42:0/3048388009 learned_addr learned my addr 172.21.15.42:0/3048388009 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe373e00640 1 --2- 172.21.15.42:0/3048388009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37415e840 0x7fe3740cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe36be00640 1 --2- 172.21.15.42:0/3048388009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe37406dc20 0x7fe37414e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe373400640 1 -- 172.21.15.42:0/3048388009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe37406dc20 msgr2=0x7fe37414e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe373400640 1 --2- 172.21.15.42:0/3048388009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe37406dc20 0x7fe37414e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe373400640 1 -- 172.21.15.42:0/3048388009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37415e840 msgr2=0x7fe3740cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe373400640 1 --2- 172.21.15.42:0/3048388009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37415e840 0x7fe3740cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe373400640 1 -- 172.21.15.42:0/3048388009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3740cb6d0 con 0x7fe37406cd10 2024-05-28T14:04:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe373400640 1 --2- 172.21.15.42:0/3048388009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37406cd10 0x7fe3741f8670 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe36400c370 tx=0x7fe36400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe36b400640 1 -- 172.21.15.42:0/3048388009 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3640052d0 con 0x7fe37406cd10 2024-05-28T14:04:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe36b400640 1 -- 172.21.15.42:0/3048388009 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe364005470 con 0x7fe37406cd10 2024-05-28T14:04:02.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe36b400640 1 -- 172.21.15.42:0/3048388009 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364007260 con 0x7fe37406cd10 2024-05-28T14:04:02.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe37a400640 1 -- 172.21.15.42:0/3048388009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3741f7b90 con 0x7fe37406cd10 2024-05-28T14:04:02.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe37a400640 1 -- 172.21.15.42:0/3048388009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3740cb9b0 con 0x7fe37406cd10 2024-05-28T14:04:02.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe37a400640 1 -- 172.21.15.42:0/3048388009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe374068a90 con 0x7fe37406cd10 2024-05-28T14:04:02.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.823+0000 7fe36b400640 1 -- 172.21.15.42:0/3048388009 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 88599+0+0 (secure 0 0 0) 0x7fe364007400 con 0x7fe37406cd10 2024-05-28T14:04:02.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.827+0000 7fe36b400640 1 --2- 172.21.15.42:0/3048388009 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe350068a40 0x7fe35006af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:02.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.827+0000 7fe36b400640 1 -- 172.21.15.42:0/3048388009 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fe364086ad0 con 0x7fe37406cd10 2024-05-28T14:04:02.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.827+0000 7fe36be00640 1 --2- 172.21.15.42:0/3048388009 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe350068a40 0x7fe35006af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:02.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.827+0000 7fe36be00640 1 --2- 172.21.15.42:0/3048388009 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe350068a40 0x7fe35006af00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe3741578f0 tx=0x7fe35c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:02.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:02.831+0000 7fe36b400640 1 -- 172.21.15.42:0/3048388009 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe364055f80 con 0x7fe37406cd10 2024-05-28T14:04:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.023+0000 7fe37a400640 1 -- 172.21.15.42:0/3048388009 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone85314", "target": ["mon-mgr", ""]}) -- 0x7fe3740cfde0 con 0x7fe350068a40 2024-05-28T14:04:03.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.043+0000 7fe36b400640 1 -- 172.21.15.42:0/3048388009 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe3740cfde0 con 0x7fe350068a40 2024-05-28T14:04:03.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.047+0000 7fe37a400640 1 -- 172.21.15.42:0/3048388009 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe350068a40 msgr2=0x7fe35006af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:03.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.047+0000 7fe37a400640 1 --2- 172.21.15.42:0/3048388009 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe350068a40 0x7fe35006af00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe3741578f0 tx=0x7fe35c008040 comp rx=0 tx=0).stop 2024-05-28T14:04:03.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.047+0000 7fe37a400640 1 -- 172.21.15.42:0/3048388009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37406cd10 msgr2=0x7fe3741f8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:03.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.047+0000 7fe37a400640 1 --2- 172.21.15.42:0/3048388009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37406cd10 0x7fe3741f8670 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe36400c370 tx=0x7fe36400c830 comp rx=0 tx=0).stop 2024-05-28T14:04:03.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.047+0000 7fe37a400640 1 -- 172.21.15.42:0/3048388009 shutdown_connections 2024-05-28T14:04:03.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.047+0000 7fe37a400640 1 --2- 172.21.15.42:0/3048388009 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe350068a40 0x7fe35006af00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.047+0000 7fe37a400640 1 --2- 172.21.15.42:0/3048388009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37415e840 0x7fe3740cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.047+0000 7fe37a400640 1 --2- 172.21.15.42:0/3048388009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe37406dc20 0x7fe37414e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.047+0000 7fe37a400640 1 --2- 172.21.15.42:0/3048388009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37406cd10 0x7fe3741f8670 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.047+0000 7fe37a400640 1 -- 172.21.15.42:0/3048388009 >> 172.21.15.42:0/3048388009 conn(0x7fe3740b3fa0 msgr2=0x7fe3740cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:03.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.047+0000 7fe37a400640 1 -- 172.21.15.42:0/3048388009 shutdown_connections 2024-05-28T14:04:03.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.047+0000 7fe37a400640 1 -- 172.21.15.42:0/3048388009 wait complete. 2024-05-28T14:04:03.057 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone85313 2024-05-28T14:04:03.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8607e00640 1 Processor -- start 2024-05-28T14:04:03.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8607e00640 1 -- start start 2024-05-28T14:04:03.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8607e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f860015e840 0x7f86000d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:03.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8607e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f860006dc20 0x7f86000cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:03.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8607e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86000d1970 con 0x7f860006cd10 2024-05-28T14:04:03.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8607e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f86000b5300 con 0x7f860015e840 2024-05-28T14:04:03.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8607e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f86000d1740 con 0x7f860006dc20 2024-05-28T14:04:03.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8604a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f860006cd10 0x7f860006d510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:54350/0 (socket says 172.21.15.42:54350) 2024-05-28T14:04:03.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8604a00640 1 -- 172.21.15.42:0/2867972225 learned_addr learned my addr 172.21.15.42:0/2867972225 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:03.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8605e00640 1 --2- 172.21.15.42:0/2867972225 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f860006dc20 0x7f86000cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:03.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8605400640 1 --2- 172.21.15.42:0/2867972225 >> v2:172.21.15.63:3300/0 conn(0x7f860015e840 0x7f86000d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:03.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8605e00640 1 -- 172.21.15.42:0/2867972225 >> v2:172.21.15.63:3300/0 conn(0x7f860015e840 msgr2=0x7f86000d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:03.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8605e00640 1 --2- 172.21.15.42:0/2867972225 >> v2:172.21.15.63:3300/0 conn(0x7f860015e840 0x7f86000d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8605e00640 1 -- 172.21.15.42:0/2867972225 >> v1:172.21.15.42:6789/0 conn(0x7f860006cd10 legacy=0x7f860006d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:03.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.219+0000 7f8605e00640 1 -- 172.21.15.42:0/2867972225 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86000ccca0 con 0x7f860006dc20 2024-05-28T14:04:03.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.223+0000 7f8605e00640 1 --2- 172.21.15.42:0/2867972225 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f860006dc20 0x7f86000cd780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009d90 tx=0x7f85f801f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecb618931b3a186f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:03.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.223+0000 7f85ffe00640 1 -- 172.21.15.42:0/2867972225 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f80b4070 con 0x7f860006dc20 2024-05-28T14:04:03.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.223+0000 7f85ffe00640 1 -- 172.21.15.42:0/2867972225 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f80af600 con 0x7f860006dc20 2024-05-28T14:04:03.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.223+0000 7f85ffe00640 1 -- 172.21.15.42:0/2867972225 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f80af9d0 con 0x7f860006dc20 2024-05-28T14:04:03.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.223+0000 7f8607e00640 1 -- 172.21.15.42:0/2867972225 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f860006dc20 msgr2=0x7f86000cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:03.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.223+0000 7f8607e00640 1 --2- 172.21.15.42:0/2867972225 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f860006dc20 0x7f86000cd780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009d90 tx=0x7f85f801f150 comp rx=0 tx=0).stop 2024-05-28T14:04:03.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.223+0000 7f8607e00640 1 -- 172.21.15.42:0/2867972225 shutdown_connections 2024-05-28T14:04:03.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.223+0000 7f8607e00640 1 --2- 172.21.15.42:0/2867972225 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f860006dc20 0x7f86000cd780 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.223+0000 7f8607e00640 1 --2- 172.21.15.42:0/2867972225 >> v2:172.21.15.63:3300/0 conn(0x7f860015e840 0x7f86000d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.223+0000 7f8607e00640 1 -- 172.21.15.42:0/2867972225 >> 172.21.15.42:0/2867972225 conn(0x7f86000b3fa0 msgr2=0x7f86000b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:03.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.223+0000 7f8607e00640 1 -- 172.21.15.42:0/2867972225 shutdown_connections 2024-05-28T14:04:03.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.223+0000 7f8607e00640 1 -- 172.21.15.42:0/2867972225 wait complete. 2024-05-28T14:04:03.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8607e00640 1 Processor -- start 2024-05-28T14:04:03.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8607e00640 1 -- start start 2024-05-28T14:04:03.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8607e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f860006cd10 0x7f8600157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:03.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8607e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f860006dc20 0x7f860015aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:03.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8607e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f860015e840 0x7f860015c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:03.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8607e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f860015ec20 con 0x7f860006dc20 2024-05-28T14:04:03.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8607e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86000cd260 con 0x7f860006cd10 2024-05-28T14:04:03.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8607e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f860015eda0 con 0x7f860015e840 2024-05-28T14:04:03.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8604a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f860006dc20 0x7f860015aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:03.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8605400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f860006cd10 0x7f8600157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:03.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8604a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f860006dc20 0x7f860015aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47734/0 (socket says 172.21.15.42:47734) 2024-05-28T14:04:03.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8604a00640 1 -- 172.21.15.42:0/1682449276 learned_addr learned my addr 172.21.15.42:0/1682449276 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:03.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8605e00640 1 --2- 172.21.15.42:0/1682449276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f860015e840 0x7f860015c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:03.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8605e00640 1 -- 172.21.15.42:0/1682449276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f860006cd10 msgr2=0x7f8600157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:03.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8605e00640 1 --2- 172.21.15.42:0/1682449276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f860006cd10 0x7f8600157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8605e00640 1 -- 172.21.15.42:0/1682449276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f860006dc20 msgr2=0x7f860015aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:03.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8605e00640 1 --2- 172.21.15.42:0/1682449276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f860006dc20 0x7f860015aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8605e00640 1 -- 172.21.15.42:0/1682449276 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8600159a60 con 0x7f860015e840 2024-05-28T14:04:03.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8604a00640 1 --2- 172.21.15.42:0/1682449276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f860006dc20 0x7f860015aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:04:03.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8605e00640 1 --2- 172.21.15.42:0/1682449276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f860015e840 0x7f860015c090 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f85f80af000 tx=0x7f85f801bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:03.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f85fd600640 1 -- 172.21.15.42:0/1682449276 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f80b4040 con 0x7f860015e840 2024-05-28T14:04:03.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f85fd600640 1 -- 172.21.15.42:0/1682449276 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f80af400 con 0x7f860015e840 2024-05-28T14:04:03.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8607e00640 1 -- 172.21.15.42:0/1682449276 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8600159d40 con 0x7f860015e840 2024-05-28T14:04:03.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f85fd600640 1 -- 172.21.15.42:0/1682449276 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8009450 con 0x7f860015e840 2024-05-28T14:04:03.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.227+0000 7f8607e00640 1 -- 172.21.15.42:0/1682449276 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f860015a300 con 0x7f860015e840 2024-05-28T14:04:03.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.231+0000 7f8607e00640 1 -- 172.21.15.42:0/1682449276 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8600068a90 con 0x7f860015e840 2024-05-28T14:04:03.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.231+0000 7f85fd600640 1 -- 172.21.15.42:0/1682449276 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f85f80c3020 con 0x7f860015e840 2024-05-28T14:04:03.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.231+0000 7f85fd600640 1 --2- 172.21.15.42:0/1682449276 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85e0068b90 0x7f85e006b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:03.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.231+0000 7f85fd600640 1 -- 172.21.15.42:0/1682449276 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f85f8125700 con 0x7f860015e840 2024-05-28T14:04:03.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.231+0000 7f8605400640 1 --2- 172.21.15.42:0/1682449276 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85e0068b90 0x7f85e006b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:03.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.231+0000 7f8605400640 1 --2- 172.21.15.42:0/1682449276 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85e0068b90 0x7f85e006b050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009ac0 tx=0x7f85f4009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:03.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.235+0000 7f85fd600640 1 -- 172.21.15.42:0/1682449276 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f85f80ca020 con 0x7f860015e840 2024-05-28T14:04:03.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.427+0000 7f8607e00640 1 -- 172.21.15.42:0/1682449276 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone85313", "target": ["mon-mgr", ""]}) -- 0x7f8600158060 con 0x7f85e0068b90 2024-05-28T14:04:03.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.443+0000 7f85fd600640 1 -- 172.21.15.42:0/1682449276 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8600158060 con 0x7f85e0068b90 2024-05-28T14:04:03.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.447+0000 7f8607e00640 1 -- 172.21.15.42:0/1682449276 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85e0068b90 msgr2=0x7f85e006b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:03.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.447+0000 7f8607e00640 1 --2- 172.21.15.42:0/1682449276 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85e0068b90 0x7f85e006b050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009ac0 tx=0x7f85f4009190 comp rx=0 tx=0).stop 2024-05-28T14:04:03.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.447+0000 7f8607e00640 1 -- 172.21.15.42:0/1682449276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f860015e840 msgr2=0x7f860015c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:03.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.447+0000 7f8607e00640 1 --2- 172.21.15.42:0/1682449276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f860015e840 0x7f860015c090 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f85f80af000 tx=0x7f85f801bef0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.447+0000 7f8607e00640 1 -- 172.21.15.42:0/1682449276 shutdown_connections 2024-05-28T14:04:03.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.447+0000 7f8607e00640 1 --2- 172.21.15.42:0/1682449276 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85e0068b90 0x7f85e006b050 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.447+0000 7f8607e00640 1 --2- 172.21.15.42:0/1682449276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f860015e840 0x7f860015c090 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.447+0000 7f8607e00640 1 --2- 172.21.15.42:0/1682449276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f860006dc20 0x7f860015aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.447+0000 7f8607e00640 1 --2- 172.21.15.42:0/1682449276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f860006cd10 0x7f8600157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.447+0000 7f8607e00640 1 -- 172.21.15.42:0/1682449276 >> 172.21.15.42:0/1682449276 conn(0x7f86000b3fa0 msgr2=0x7f860006d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:03.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.447+0000 7f8607e00640 1 -- 172.21.15.42:0/1682449276 shutdown_connections 2024-05-28T14:04:03.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.447+0000 7f8607e00640 1 -- 172.21.15.42:0/1682449276 wait complete. 2024-05-28T14:04:03.455 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-28T14:04:03.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-28T14:04:03.513 INFO:teuthology.orchestra.run.smithi042.stdout:2 2024-05-28T14:04:03.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:04:03.520 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T14:04:03.521 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:04:03.634 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:04:03.724 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T14:04:03.724 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:04:03.724 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:04:03.724 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:04:03.734 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:04:03.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd46600640 1 Processor -- start 2024-05-28T14:04:03.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd46600640 1 -- start start 2024-05-28T14:04:03.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd46600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbd4015c990 0x7fbd400d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:03.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd46600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd400d06a0 0x7fbd400b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:03.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd46600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd400cd300 con 0x7fbd4009bbd0 2024-05-28T14:04:03.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd46600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fbd4015cd70 con 0x7fbd4015c990 2024-05-28T14:04:03.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd46600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbd400cd5e0 con 0x7fbd400d06a0 2024-05-28T14:04:03.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd3f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbd4015c990 0x7fbd400d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:03.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd3f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbd4015c990 0x7fbd400d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47742/0 (socket says 172.21.15.42:47742) 2024-05-28T14:04:03.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd3f400640 1 -- 172.21.15.42:0/1719672285 learned_addr learned my addr 172.21.15.42:0/1719672285 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:03.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd3fe00640 1 --2- 172.21.15.42:0/1719672285 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd400d06a0 0x7fbd400b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:03.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd3f400640 1 -- 172.21.15.42:0/1719672285 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd400d06a0 msgr2=0x7fbd400b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:03.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd3f400640 1 --2- 172.21.15.42:0/1719672285 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd400d06a0 0x7fbd400b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd3f400640 1 -- 172.21.15.42:0/1719672285 >> v1:172.21.15.63:6789/0 conn(0x7fbd4009bbd0 legacy=0x7fbd4009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:03.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd3f400640 1 -- 172.21.15.42:0/1719672285 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd400cd8c0 con 0x7fbd4015c990 2024-05-28T14:04:03.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.931+0000 7fbd3fe00640 1 --2- 172.21.15.42:0/1719672285 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd400d06a0 0x7fbd400b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:03.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.935+0000 7fbd3f400640 1 --2- 172.21.15.42:0/1719672285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4015c990 0x7fbd400d0160 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009870 tx=0x7fbd3001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361f4621a5ee24ff server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:03.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.935+0000 7fbd3ea00640 1 -- 172.21.15.42:0/1719672285 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd300aa610 con 0x7fbd4015c990 2024-05-28T14:04:03.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.935+0000 7fbd3ea00640 1 -- 172.21.15.42:0/1719672285 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd300af070 con 0x7fbd4015c990 2024-05-28T14:04:03.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.935+0000 7fbd3ea00640 1 -- 172.21.15.42:0/1719672285 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd300aa9d0 con 0x7fbd4015c990 2024-05-28T14:04:03.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.935+0000 7fbd46600640 1 -- 172.21.15.42:0/1719672285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4015c990 msgr2=0x7fbd400d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:03.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.935+0000 7fbd46600640 1 --2- 172.21.15.42:0/1719672285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4015c990 0x7fbd400d0160 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009870 tx=0x7fbd3001c4f0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.935+0000 7fbd46600640 1 -- 172.21.15.42:0/1719672285 shutdown_connections 2024-05-28T14:04:03.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.935+0000 7fbd46600640 1 --2- 172.21.15.42:0/1719672285 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd400d06a0 0x7fbd400b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.935+0000 7fbd46600640 1 --2- 172.21.15.42:0/1719672285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4015c990 0x7fbd400d0160 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.935+0000 7fbd46600640 1 -- 172.21.15.42:0/1719672285 >> 172.21.15.42:0/1719672285 conn(0x7fbd400b4080 msgr2=0x7fbd400b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:03.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.935+0000 7fbd46600640 1 -- 172.21.15.42:0/1719672285 shutdown_connections 2024-05-28T14:04:03.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.935+0000 7fbd46600640 1 -- 172.21.15.42:0/1719672285 wait complete. 2024-05-28T14:04:03.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd46600640 1 Processor -- start 2024-05-28T14:04:03.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd46600640 1 -- start start 2024-05-28T14:04:03.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd46600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd400d06a0 0x7fbd4014cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:03.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd46600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd4015c990 0x7fbd4014d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:03.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd46600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd4014eb30 0x7fbd4015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:03.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd46600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4009b2e0 con 0x7fbd400d06a0 2024-05-28T14:04:03.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd46600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4009b160 con 0x7fbd4015c990 2024-05-28T14:04:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd46600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbd400b6130 con 0x7fbd4014eb30 2024-05-28T14:04:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd3f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd400d06a0 0x7fbd4014cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd3f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd400d06a0 0x7fbd4014cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47756/0 (socket says 172.21.15.42:47756) 2024-05-28T14:04:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd3f400640 1 -- 172.21.15.42:0/476072837 learned_addr learned my addr 172.21.15.42:0/476072837 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd36a00640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd4015c990 0x7fbd4014d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd3fe00640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd4014eb30 0x7fbd4015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd3f400640 1 -- 172.21.15.42:0/476072837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd4014eb30 msgr2=0x7fbd4015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd3f400640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd4014eb30 0x7fbd4015adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd3f400640 1 -- 172.21.15.42:0/476072837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd4015c990 msgr2=0x7fbd4014d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd3f400640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd4015c990 0x7fbd4014d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd3f400640 1 -- 172.21.15.42:0/476072837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4014de60 con 0x7fbd400d06a0 2024-05-28T14:04:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd36a00640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd4015c990 0x7fbd4014d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:03.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd3f400640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd400d06a0 0x7fbd4014cd50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbd3001cd90 tx=0x7fbd3001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:03.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd37e00640 1 -- 172.21.15.42:0/476072837 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd300bd5f0 con 0x7fbd400d06a0 2024-05-28T14:04:03.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd37e00640 1 -- 172.21.15.42:0/476072837 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd300bd7b0 con 0x7fbd400d06a0 2024-05-28T14:04:03.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd37e00640 1 -- 172.21.15.42:0/476072837 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd300bdba0 con 0x7fbd400d06a0 2024-05-28T14:04:03.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd46600640 1 -- 172.21.15.42:0/476072837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4014e140 con 0x7fbd400d06a0 2024-05-28T14:04:03.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd46600640 1 -- 172.21.15.42:0/476072837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4014e700 con 0x7fbd400d06a0 2024-05-28T14:04:03.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.939+0000 7fbd46600640 1 -- 172.21.15.42:0/476072837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd40068a90 con 0x7fbd400d06a0 2024-05-28T14:04:03.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.943+0000 7fbd37e00640 1 -- 172.21.15.42:0/476072837 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fbd300b33f0 con 0x7fbd400d06a0 2024-05-28T14:04:03.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.943+0000 7fbd37e00640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd1c068b60 0x7fbd1c06b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:03.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.943+0000 7fbd37e00640 1 -- 172.21.15.42:0/476072837 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fbd30121fc0 con 0x7fbd400d06a0 2024-05-28T14:04:03.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.943+0000 7fbd36a00640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd1c068b60 0x7fbd1c06b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:03.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.943+0000 7fbd36a00640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd1c068b60 0x7fbd1c06b020 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbd4015bab0 tx=0x7fbd28009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:03.947+0000 7fbd37e00640 1 -- 172.21.15.42:0/476072837 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbd300c3060 con 0x7fbd400d06a0 2024-05-28T14:04:04.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.115+0000 7fbd46600640 1 -- 172.21.15.42:0/476072837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd400b4ce0 con 0x7fbd400d06a0 2024-05-28T14:04:04.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.119+0000 7fbd37e00640 1 -- 172.21.15.42:0/476072837 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16748 (secure 0 0 0) 0x7fbd300a7070 con 0x7fbd400d06a0 2024-05-28T14:04:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:04.104 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":48,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:02:04.318755+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:01:49.041938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-28T14:01:49.591543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.1b","mappings":[{"from":0,"to":4}]},{"pgid":"7.30","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:04:04.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.119+0000 7fbd46600640 1 -- 172.21.15.42:0/476072837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd1c068b60 msgr2=0x7fbd1c06b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:04.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.119+0000 7fbd46600640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd1c068b60 0x7fbd1c06b020 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbd4015bab0 tx=0x7fbd28009040 comp rx=0 tx=0).stop 2024-05-28T14:04:04.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.119+0000 7fbd46600640 1 -- 172.21.15.42:0/476072837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd400d06a0 msgr2=0x7fbd4014cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:04.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.119+0000 7fbd46600640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd400d06a0 0x7fbd4014cd50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbd3001cd90 tx=0x7fbd3001ad30 comp rx=0 tx=0).stop 2024-05-28T14:04:04.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.119+0000 7fbd46600640 1 -- 172.21.15.42:0/476072837 shutdown_connections 2024-05-28T14:04:04.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.119+0000 7fbd46600640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd1c068b60 0x7fbd1c06b020 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.119+0000 7fbd46600640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd4014eb30 0x7fbd4015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.119+0000 7fbd46600640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd4015c990 0x7fbd4014d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.119+0000 7fbd46600640 1 --2- 172.21.15.42:0/476072837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd400d06a0 0x7fbd4014cd50 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.119+0000 7fbd46600640 1 -- 172.21.15.42:0/476072837 >> 172.21.15.42:0/476072837 conn(0x7fbd400b4080 msgr2=0x7fbd401590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:04.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.119+0000 7fbd46600640 1 -- 172.21.15.42:0/476072837 shutdown_connections 2024-05-28T14:04:04.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.119+0000 7fbd46600640 1 -- 172.21.15.42:0/476072837 wait complete. 2024-05-28T14:04:04.126 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T14:01:49.041938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T14:01:49.591543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:04:04.126 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:04:04.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.283+0000 7f792e800640 1 Processor -- start 2024-05-28T14:04:04.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.283+0000 7f792e800640 1 -- start start 2024-05-28T14:04:04.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.287+0000 7f792e800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f792806ee20 0x7f792806f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:04.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.287+0000 7f792e800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79280afe40 con 0x7f792815b9b0 2024-05-28T14:04:04.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.287+0000 7f792e800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f792815c600 con 0x7f7928144f80 2024-05-28T14:04:04.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.287+0000 7f792e800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f792806f7c0 con 0x7f792806ee20 2024-05-28T14:04:04.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.287+0000 7f7927e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f792815b9b0 0x7f792815be40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:54358/0 (socket says 172.21.15.42:54358) 2024-05-28T14:04:04.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.287+0000 7f7927e00640 1 -- 172.21.15.42:0/1932669555 learned_addr learned my addr 172.21.15.42:0/1932669555 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:04.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.287+0000 7f791ea00640 1 --2- 172.21.15.42:0/1932669555 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f792806ee20 0x7f792806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:04.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.287+0000 7f7926a00640 1 -- 172.21.15.42:0/1932669555 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2919750086 0 0) 0x7f79280afe40 con 0x7f792815b9b0 2024-05-28T14:04:04.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.287+0000 7f7926a00640 1 -- 172.21.15.42:0/1932669555 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f790c005740 con 0x7f792815b9b0 2024-05-28T14:04:04.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.287+0000 7f7926a00640 1 -- 172.21.15.42:0/1932669555 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1432493563 0 0) 0x7f792815c600 con 0x7f7928144f80 2024-05-28T14:04:04.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.287+0000 7f7926a00640 1 -- 172.21.15.42:0/1932669555 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79280afe40 con 0x7f7928144f80 2024-05-28T14:04:04.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.287+0000 7f791ea00640 1 -- 172.21.15.42:0/1932669555 >> v1:172.21.15.63:6789/0 conn(0x7f7928144f80 legacy=0x7f7928072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:04.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.287+0000 7f791ea00640 1 -- 172.21.15.42:0/1932669555 >> v1:172.21.15.42:6789/0 conn(0x7f792815b9b0 legacy=0x7f792815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:04.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.287+0000 7f791ea00640 1 -- 172.21.15.42:0/1932669555 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792815c8e0 con 0x7f792806ee20 2024-05-28T14:04:04.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f791ea00640 1 --2- 172.21.15.42:0/1932669555 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f792806ee20 0x7f792806f280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f79100097f0 tx=0x7f791001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95b5edf75f5ed527 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:04.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f7926a00640 1 -- 172.21.15.42:0/1932669555 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100b0070 con 0x7f792806ee20 2024-05-28T14:04:04.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f7926a00640 1 -- 172.21.15.42:0/1932669555 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79100ab600 con 0x7f792806ee20 2024-05-28T14:04:04.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f7926a00640 1 -- 172.21.15.42:0/1932669555 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100ab9d0 con 0x7f792806ee20 2024-05-28T14:04:04.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 -- 172.21.15.42:0/1932669555 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f792806ee20 msgr2=0x7f792806f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:04.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 --2- 172.21.15.42:0/1932669555 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f792806ee20 0x7f792806f280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f79100097f0 tx=0x7f791001c4e0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 -- 172.21.15.42:0/1932669555 shutdown_connections 2024-05-28T14:04:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 --2- 172.21.15.42:0/1932669555 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f792806ee20 0x7f792806f280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 -- 172.21.15.42:0/1932669555 >> 172.21.15.42:0/1932669555 conn(0x7f79280b21f0 msgr2=0x7f79280b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 -- 172.21.15.42:0/1932669555 shutdown_connections 2024-05-28T14:04:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 -- 172.21.15.42:0/1932669555 wait complete. 2024-05-28T14:04:04.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 Processor -- start 2024-05-28T14:04:04.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 -- start start 2024-05-28T14:04:04.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f792806ee20 0x7f792813f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:04.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7928144f80 0x7f79281f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:04.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f792815b9b0 0x7f7928140c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:04.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f79280724d0 con 0x7f792806ee20 2024-05-28T14:04:04.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79280b4260 con 0x7f7928144f80 2024-05-28T14:04:04.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f792e800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f792806dd10 con 0x7f792815b9b0 2024-05-28T14:04:04.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f7927400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f792806ee20 0x7f792813f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:04.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f7927400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f792806ee20 0x7f792813f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47770/0 (socket says 172.21.15.42:47770) 2024-05-28T14:04:04.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f7927400640 1 -- 172.21.15.42:0/687804798 learned_addr learned my addr 172.21.15.42:0/687804798 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:04.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.291+0000 7f791ea00640 1 --2- 172.21.15.42:0/687804798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7928144f80 0x7f79281f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:04.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f7927e00640 1 --2- 172.21.15.42:0/687804798 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f792815b9b0 0x7f7928140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:04.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f7927400640 1 -- 172.21.15.42:0/687804798 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f792815b9b0 msgr2=0x7f7928140c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:04.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f7927400640 1 --2- 172.21.15.42:0/687804798 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f792815b9b0 0x7f7928140c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f7927400640 1 -- 172.21.15.42:0/687804798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7928144f80 msgr2=0x7f79281f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:04.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f7927400640 1 --2- 172.21.15.42:0/687804798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7928144f80 0x7f79281f2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f7927400640 1 -- 172.21.15.42:0/687804798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7928158cd0 con 0x7f792806ee20 2024-05-28T14:04:04.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f7927400640 1 --2- 172.21.15.42:0/687804798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f792806ee20 0x7f792813f690 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f791800c870 tx=0x7f791800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:04.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f791fe00640 1 -- 172.21.15.42:0/687804798 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918003b80 con 0x7f792806ee20 2024-05-28T14:04:04.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f791fe00640 1 -- 172.21.15.42:0/687804798 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7918003d20 con 0x7f792806ee20 2024-05-28T14:04:04.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f792e800640 1 -- 172.21.15.42:0/687804798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7928158fb0 con 0x7f792806ee20 2024-05-28T14:04:04.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f791fe00640 1 -- 172.21.15.42:0/687804798 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791800aa30 con 0x7f792806ee20 2024-05-28T14:04:04.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f792e800640 1 -- 172.21.15.42:0/687804798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7928159570 con 0x7f792806ee20 2024-05-28T14:04:04.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f792e800640 1 -- 172.21.15.42:0/687804798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79280738b0 con 0x7f792806ee20 2024-05-28T14:04:04.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f791fe00640 1 -- 172.21.15.42:0/687804798 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f791800abd0 con 0x7f792806ee20 2024-05-28T14:04:04.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f791fe00640 1 --2- 172.21.15.42:0/687804798 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7904068b60 0x7f790406b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:04.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f791fe00640 1 -- 172.21.15.42:0/687804798 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f7918087b90 con 0x7f792806ee20 2024-05-28T14:04:04.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.295+0000 7f791ea00640 1 --2- 172.21.15.42:0/687804798 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7904068b60 0x7f790406b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:04.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.299+0000 7f791ea00640 1 --2- 172.21.15.42:0/687804798 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7904068b60 0x7f790406b020 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f79100ad000 tx=0x7f79100a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:04.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.299+0000 7f791fe00640 1 -- 172.21.15.42:0/687804798 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7918056a80 con 0x7f792806ee20 2024-05-28T14:04:04.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.583+0000 7f792e800640 1 -- 172.21.15.42:0/687804798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f792815dce0 con 0x7f792806ee20 2024-05-28T14:04:04.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.583+0000 7f791fe00640 1 -- 172.21.15.42:0/687804798 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f791805ad80 con 0x7f792806ee20 2024-05-28T14:04:04.570 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:04:04.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.591+0000 7f792e800640 1 -- 172.21.15.42:0/687804798 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7904068b60 msgr2=0x7f790406b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:04.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.591+0000 7f792e800640 1 --2- 172.21.15.42:0/687804798 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7904068b60 0x7f790406b020 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f79100ad000 tx=0x7f79100a6000 comp rx=0 tx=0).stop 2024-05-28T14:04:04.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.591+0000 7f792e800640 1 -- 172.21.15.42:0/687804798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f792806ee20 msgr2=0x7f792813f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:04.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.591+0000 7f792e800640 1 --2- 172.21.15.42:0/687804798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f792806ee20 0x7f792813f690 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f791800c870 tx=0x7f791800cd30 comp rx=0 tx=0).stop 2024-05-28T14:04:04.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.591+0000 7f792e800640 1 -- 172.21.15.42:0/687804798 shutdown_connections 2024-05-28T14:04:04.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.591+0000 7f792e800640 1 --2- 172.21.15.42:0/687804798 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7904068b60 0x7f790406b020 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.591+0000 7f792e800640 1 --2- 172.21.15.42:0/687804798 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f792815b9b0 0x7f7928140c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.591+0000 7f792e800640 1 --2- 172.21.15.42:0/687804798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7928144f80 0x7f79281f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.591+0000 7f792e800640 1 --2- 172.21.15.42:0/687804798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f792806ee20 0x7f792813f690 secure :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f791800c870 tx=0x7f791800cd30 comp rx=0 tx=0).stop 2024-05-28T14:04:04.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.591+0000 7f792e800640 1 -- 172.21.15.42:0/687804798 >> 172.21.15.42:0/687804798 conn(0x7f79280b21f0 msgr2=0x7f792815d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:04.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.591+0000 7f792e800640 1 -- 172.21.15.42:0/687804798 shutdown_connections 2024-05-28T14:04:04.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.591+0000 7f792e800640 1 -- 172.21.15.42:0/687804798 wait complete. 2024-05-28T14:04:04.621 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:04:04.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.887+0000 7f4860600640 1 Processor -- start 2024-05-28T14:04:04.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.887+0000 7f4860600640 1 -- start start 2024-05-28T14:04:04.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.891+0000 7f4860600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f485806ec00 0x7f48580b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:04.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.891+0000 7f4860600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48580b2a40 0x7f48580adb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:04.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.891+0000 7f4860600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f485806f050 con 0x7f485815e620 2024-05-28T14:04:04.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.891+0000 7f4860600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f48580b2e90 con 0x7f485806ec00 2024-05-28T14:04:04.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.891+0000 7f4860600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f48580ad8c0 con 0x7f48580b2a40 2024-05-28T14:04:04.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.891+0000 7f485d200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f485806ec00 0x7f48580b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:04.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.891+0000 7f485d200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f485806ec00 0x7f48580b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:43884/0 (socket says 172.21.15.42:43884) 2024-05-28T14:04:04.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.891+0000 7f485d200640 1 -- 172.21.15.42:0/2951991523 learned_addr learned my addr 172.21.15.42:0/2951991523 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:04.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.891+0000 7f485e600640 1 --2- 172.21.15.42:0/2951991523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48580b2a40 0x7f48580adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:04.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.891+0000 7f485e600640 1 -- 172.21.15.42:0/2951991523 >> v2:172.21.15.63:3300/0 conn(0x7f485806ec00 msgr2=0x7f48580b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:04.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.891+0000 7f485e600640 1 --2- 172.21.15.42:0/2951991523 >> v2:172.21.15.63:3300/0 conn(0x7f485806ec00 0x7f48580b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.891+0000 7f485e600640 1 -- 172.21.15.42:0/2951991523 >> v1:172.21.15.63:6789/0 conn(0x7f485815e620 legacy=0x7f48580b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:04.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.891+0000 7f485e600640 1 -- 172.21.15.42:0/2951991523 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485806f2e0 con 0x7f48580b2a40 2024-05-28T14:04:04.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.895+0000 7f485e600640 1 --2- 172.21.15.42:0/2951991523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48580b2a40 0x7f48580adb70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4850009d90 tx=0x7f485001c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2763e5b9016ebe2e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:04.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.899+0000 7f4857e00640 1 -- 172.21.15.42:0/2951991523 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500a6700 con 0x7f48580b2a40 2024-05-28T14:04:04.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.899+0000 7f4857e00640 1 -- 172.21.15.42:0/2951991523 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48500a68a0 con 0x7f48580b2a40 2024-05-28T14:04:04.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.899+0000 7f4857e00640 1 -- 172.21.15.42:0/2951991523 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500a6c70 con 0x7f48580b2a40 2024-05-28T14:04:04.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.899+0000 7f4860600640 1 -- 172.21.15.42:0/2951991523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48580b2a40 msgr2=0x7f48580adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:04.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.899+0000 7f4860600640 1 --2- 172.21.15.42:0/2951991523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48580b2a40 0x7f48580adb70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4850009d90 tx=0x7f485001c800 comp rx=0 tx=0).stop 2024-05-28T14:04:04.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.899+0000 7f4860600640 1 -- 172.21.15.42:0/2951991523 shutdown_connections 2024-05-28T14:04:04.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.899+0000 7f4860600640 1 --2- 172.21.15.42:0/2951991523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48580b2a40 0x7f48580adb70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.899+0000 7f4860600640 1 --2- 172.21.15.42:0/2951991523 >> v2:172.21.15.63:3300/0 conn(0x7f485806ec00 0x7f48580b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.899+0000 7f4860600640 1 -- 172.21.15.42:0/2951991523 >> 172.21.15.42:0/2951991523 conn(0x7f48580d07c0 msgr2=0x7f48580d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:04.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.899+0000 7f4860600640 1 -- 172.21.15.42:0/2951991523 shutdown_connections 2024-05-28T14:04:04.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.899+0000 7f4860600640 1 -- 172.21.15.42:0/2951991523 wait complete. 2024-05-28T14:04:04.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.903+0000 7f4860600640 1 Processor -- start 2024-05-28T14:04:04.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.903+0000 7f4860600640 1 -- start start 2024-05-28T14:04:04.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.903+0000 7f4860600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f485806ec00 0x7f48581fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:04.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.903+0000 7f4860600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48580b2a40 0x7f4858149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:04.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.903+0000 7f4860600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f485815e620 0x7f48580b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:04.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.903+0000 7f4860600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f485806fc10 con 0x7f485815e620 2024-05-28T14:04:04.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.903+0000 7f4860600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f485815ec10 con 0x7f48580b2a40 2024-05-28T14:04:04.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.903+0000 7f4860600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f48580ae8c0 con 0x7f485806ec00 2024-05-28T14:04:04.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.903+0000 7f485dc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f485806ec00 0x7f48581fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:04.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.903+0000 7f485dc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f485806ec00 0x7f48581fb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:39080/0 (socket says 172.21.15.42:39080) 2024-05-28T14:04:04.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.903+0000 7f485dc00640 1 -- 172.21.15.42:0/3096205531 learned_addr learned my addr 172.21.15.42:0/3096205531 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:04.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.903+0000 7f485d200640 1 --2- 172.21.15.42:0/3096205531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48580b2a40 0x7f4858149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:04.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.903+0000 7f485e600640 1 --2- 172.21.15.42:0/3096205531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f485815e620 0x7f48580b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.907+0000 7f485d200640 1 -- 172.21.15.42:0/3096205531 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f485806ec00 msgr2=0x7f48581fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.907+0000 7f485d200640 1 --2- 172.21.15.42:0/3096205531 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f485806ec00 0x7f48581fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.907+0000 7f485d200640 1 -- 172.21.15.42:0/3096205531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f485815e620 msgr2=0x7f48580b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.907+0000 7f485d200640 1 --2- 172.21.15.42:0/3096205531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f485815e620 0x7f48580b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.907+0000 7f485d200640 1 -- 172.21.15.42:0/3096205531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48580b1b80 con 0x7f48580b2a40 2024-05-28T14:04:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.907+0000 7f485d200640 1 --2- 172.21.15.42:0/3096205531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48580b2a40 0x7f4858149b00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f484400e630 tx=0x7f484400eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.907+0000 7f4855600640 1 -- 172.21.15.42:0/3096205531 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844017070 con 0x7f48580b2a40 2024-05-28T14:04:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.907+0000 7f4855600640 1 -- 172.21.15.42:0/3096205531 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f484400c620 con 0x7f48580b2a40 2024-05-28T14:04:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.907+0000 7f4860600640 1 -- 172.21.15.42:0/3096205531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4858148d40 con 0x7f48580b2a40 2024-05-28T14:04:04.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.907+0000 7f4855600640 1 -- 172.21.15.42:0/3096205531 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484400f040 con 0x7f48580b2a40 2024-05-28T14:04:04.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.907+0000 7f4860600640 1 -- 172.21.15.42:0/3096205531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4858149300 con 0x7f48580b2a40 2024-05-28T14:04:04.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.911+0000 7f4855600640 1 -- 172.21.15.42:0/3096205531 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f484401a070 con 0x7f48580b2a40 2024-05-28T14:04:04.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.911+0000 7f4860600640 1 -- 172.21.15.42:0/3096205531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48340052a0 con 0x7f48580b2a40 2024-05-28T14:04:04.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.911+0000 7f4855600640 1 --2- 172.21.15.42:0/3096205531 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4838068c30 0x7f483806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:04.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.911+0000 7f4855600640 1 -- 172.21.15.42:0/3096205531 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f48440876e0 con 0x7f48580b2a40 2024-05-28T14:04:04.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.911+0000 7f485dc00640 1 --2- 172.21.15.42:0/3096205531 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4838068c30 0x7f483806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:04.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.911+0000 7f485dc00640 1 --2- 172.21.15.42:0/3096205531 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4838068c30 0x7f483806b0f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f484c001890 tx=0x7f484c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:04.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:04.919+0000 7f4855600640 1 -- 172.21.15.42:0/3096205531 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4844056650 con 0x7f48580b2a40 2024-05-28T14:04:05.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.223+0000 7f4860600640 1 -- 172.21.15.42:0/3096205531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4834005860 con 0x7f48580b2a40 2024-05-28T14:04:05.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.223+0000 7f4855600640 1 -- 172.21.15.42:0/3096205531 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) ==== 104+0+11 (secure 0 0 0) 0x7f484405a950 con 0x7f48580b2a40 2024-05-28T14:04:05.212 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:04:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.227+0000 7f4860600640 1 -- 172.21.15.42:0/3096205531 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4838068c30 msgr2=0x7f483806b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.227+0000 7f4860600640 1 --2- 172.21.15.42:0/3096205531 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4838068c30 0x7f483806b0f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f484c001890 tx=0x7f484c009040 comp rx=0 tx=0).stop 2024-05-28T14:04:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.227+0000 7f4860600640 1 -- 172.21.15.42:0/3096205531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48580b2a40 msgr2=0x7f4858149b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.227+0000 7f4860600640 1 --2- 172.21.15.42:0/3096205531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48580b2a40 0x7f4858149b00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f484400e630 tx=0x7f484400eaf0 comp rx=0 tx=0).stop 2024-05-28T14:04:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.227+0000 7f4860600640 1 -- 172.21.15.42:0/3096205531 shutdown_connections 2024-05-28T14:04:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.227+0000 7f4860600640 1 --2- 172.21.15.42:0/3096205531 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4838068c30 0x7f483806b0f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.227+0000 7f4860600640 1 --2- 172.21.15.42:0/3096205531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f485815e620 0x7f48580b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.227+0000 7f4860600640 1 --2- 172.21.15.42:0/3096205531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48580b2a40 0x7f4858149b00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.227+0000 7f4860600640 1 --2- 172.21.15.42:0/3096205531 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f485806ec00 0x7f48581fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:05.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.227+0000 7f4860600640 1 -- 172.21.15.42:0/3096205531 >> 172.21.15.42:0/3096205531 conn(0x7f48580d07c0 msgr2=0x7f48580b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:05.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.227+0000 7f4860600640 1 -- 172.21.15.42:0/3096205531 shutdown_connections 2024-05-28T14:04:05.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.231+0000 7f4860600640 1 -- 172.21.15.42:0/3096205531 wait complete. 2024-05-28T14:04:05.246 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:04:05.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.471+0000 7f3c4fa00640 1 Processor -- start 2024-05-28T14:04:05.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.471+0000 7f3c4fa00640 1 -- start start 2024-05-28T14:04:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.475+0000 7f3c4fa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3c480afce0 0x7f3c48071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.475+0000 7f3c4fa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3c480b5410 0x7f3c48072320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.475+0000 7f3c4fa00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c480aef20 con 0x7f3c48144e60 2024-05-28T14:04:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.475+0000 7f3c4fa00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f3c480af850 con 0x7f3c480b5410 2024-05-28T14:04:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.475+0000 7f3c4fa00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3c480af200 con 0x7f3c480afce0 2024-05-28T14:04:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.475+0000 7f3c4da00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3c480b5410 0x7f3c48072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.475+0000 7f3c4da00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3c480b5410 0x7f3c48072320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47798/0 (socket says 172.21.15.42:47798) 2024-05-28T14:04:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.475+0000 7f3c4da00640 1 -- 172.21.15.42:0/1383270 learned_addr learned my addr 172.21.15.42:0/1383270 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.475+0000 7f3c4da00640 1 -- 172.21.15.42:0/1383270 >> v2:172.21.15.63:3300/0 conn(0x7f3c480afce0 msgr2=0x7f3c48071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.475+0000 7f3c4da00640 1 --2- 172.21.15.42:0/1383270 >> v2:172.21.15.63:3300/0 conn(0x7f3c480afce0 0x7f3c48071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.475+0000 7f3c4da00640 1 -- 172.21.15.42:0/1383270 >> v1:172.21.15.63:6789/0 conn(0x7f3c48144e60 legacy=0x7f3c480b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.475+0000 7f3c4da00640 1 -- 172.21.15.42:0/1383270 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c480af4e0 con 0x7f3c480b5410 2024-05-28T14:04:05.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.479+0000 7f3c4da00640 1 --2- 172.21.15.42:0/1383270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c480b5410 0x7f3c48072320 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3c48071c30 tx=0x7f3c4001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a224bcf9ab550021 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:05.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.479+0000 7f3c47400640 1 -- 172.21.15.42:0/1383270 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400a5890 con 0x7f3c480b5410 2024-05-28T14:04:05.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.479+0000 7f3c47400640 1 -- 172.21.15.42:0/1383270 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c400a5a30 con 0x7f3c480b5410 2024-05-28T14:04:05.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.479+0000 7f3c47400640 1 -- 172.21.15.42:0/1383270 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400a5e00 con 0x7f3c480b5410 2024-05-28T14:04:05.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.483+0000 7f3c4fa00640 1 -- 172.21.15.42:0/1383270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c480b5410 msgr2=0x7f3c48072320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:05.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.483+0000 7f3c4fa00640 1 --2- 172.21.15.42:0/1383270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c480b5410 0x7f3c48072320 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3c48071c30 tx=0x7f3c4001c970 comp rx=0 tx=0).stop 2024-05-28T14:04:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.483+0000 7f3c4fa00640 1 -- 172.21.15.42:0/1383270 shutdown_connections 2024-05-28T14:04:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.483+0000 7f3c4fa00640 1 --2- 172.21.15.42:0/1383270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c480b5410 0x7f3c48072320 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.483+0000 7f3c4fa00640 1 --2- 172.21.15.42:0/1383270 >> v2:172.21.15.63:3300/0 conn(0x7f3c480afce0 0x7f3c48071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.483+0000 7f3c4fa00640 1 -- 172.21.15.42:0/1383270 >> 172.21.15.42:0/1383270 conn(0x7f3c480b21f0 msgr2=0x7f3c480b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:05.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.483+0000 7f3c4fa00640 1 -- 172.21.15.42:0/1383270 shutdown_connections 2024-05-28T14:04:05.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.483+0000 7f3c4fa00640 1 -- 172.21.15.42:0/1383270 wait complete. 2024-05-28T14:04:05.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.487+0000 7f3c4fa00640 1 Processor -- start 2024-05-28T14:04:05.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.487+0000 7f3c4fa00640 1 -- start start 2024-05-28T14:04:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.487+0000 7f3c4fa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c480afce0 0x7f3c480ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.487+0000 7f3c4fa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c480b5410 0x7f3c48143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.487+0000 7f3c4fa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c48144e60 0x7f3c480a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.487+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c480b5410 0x7f3c48143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.487+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c480b5410 0x7f3c48143da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47812/0 (socket says 172.21.15.42:47812) 2024-05-28T14:04:05.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.487+0000 7f3c4d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c480afce0 0x7f3c480ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:05.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.487+0000 7f3c4d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c480afce0 0x7f3c480ad560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:43914/0 (socket says 172.21.15.42:43914) 2024-05-28T14:04:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.491+0000 7f3c4fa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4806e8b0 con 0x7f3c480b5410 2024-05-28T14:04:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.491+0000 7f3c4fa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4806e730 con 0x7f3c480afce0 2024-05-28T14:04:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.491+0000 7f3c4fa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3c4806ea30 con 0x7f3c48144e60 2024-05-28T14:04:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.491+0000 7f3c47e00640 1 -- 172.21.15.42:0/2901601602 learned_addr learned my addr 172.21.15.42:0/2901601602 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.491+0000 7f3c4d000640 1 -- 172.21.15.42:0/2901601602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c48144e60 msgr2=0x7f3c480a9f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T14:04:05.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.491+0000 7f3c4d000640 1 --2- 172.21.15.42:0/2901601602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c48144e60 0x7f3c480a9f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:05.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.491+0000 7f3c4d000640 1 -- 172.21.15.42:0/2901601602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c480b5410 msgr2=0x7f3c48143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:05.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.491+0000 7f3c4d000640 1 --2- 172.21.15.42:0/2901601602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c480b5410 0x7f3c48143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:05.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.491+0000 7f3c4d000640 1 -- 172.21.15.42:0/2901601602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c480cdcf0 con 0x7f3c480afce0 2024-05-28T14:04:05.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.495+0000 7f3c47e00640 1 --2- 172.21.15.42:0/2901601602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c480b5410 0x7f3c48143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:04:05.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.495+0000 7f3c4d000640 1 --2- 172.21.15.42:0/2901601602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c480afce0 0x7f3c480ad560 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800c390 tx=0x7f3c3800c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.499+0000 7f3c44c00640 1 -- 172.21.15.42:0/2901601602 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38011790 con 0x7f3c480afce0 2024-05-28T14:04:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.499+0000 7f3c44c00640 1 -- 172.21.15.42:0/2901601602 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c38017070 con 0x7f3c480afce0 2024-05-28T14:04:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.499+0000 7f3c44c00640 1 -- 172.21.15.42:0/2901601602 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38011b70 con 0x7f3c480afce0 2024-05-28T14:04:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.499+0000 7f3c4fa00640 1 -- 172.21.15.42:0/2901601602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c481442e0 con 0x7f3c480afce0 2024-05-28T14:04:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.499+0000 7f3c4fa00640 1 -- 172.21.15.42:0/2901601602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c480cdf90 con 0x7f3c480afce0 2024-05-28T14:04:05.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.499+0000 7f3c4fa00640 1 -- 172.21.15.42:0/2901601602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c480738b0 con 0x7f3c480afce0 2024-05-28T14:04:05.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.503+0000 7f3c44c00640 1 -- 172.21.15.42:0/2901601602 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f3c38011d10 con 0x7f3c480afce0 2024-05-28T14:04:05.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.503+0000 7f3c44c00640 1 --2- 172.21.15.42:0/2901601602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3c28068bb0 0x7f3c2806b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:05.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.503+0000 7f3c44c00640 1 -- 172.21.15.42:0/2901601602 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f3c38087880 con 0x7f3c480afce0 2024-05-28T14:04:05.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.503+0000 7f3c47e00640 1 --2- 172.21.15.42:0/2901601602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3c28068bb0 0x7f3c2806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:05.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.503+0000 7f3c47e00640 1 --2- 172.21.15.42:0/2901601602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3c28068bb0 0x7f3c2806b070 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3c48162e00 tx=0x7f3c34009110 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:05.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.511+0000 7f3c44c00640 1 -- 172.21.15.42:0/2901601602 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3c38056ed0 con 0x7f3c480afce0 2024-05-28T14:04:05.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.811+0000 7f3c4fa00640 1 -- 172.21.15.42:0/2901601602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3c480aa4b0 con 0x7f3c480afce0 2024-05-28T14:04:05.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.815+0000 7f3c44c00640 1 -- 172.21.15.42:0/2901601602 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) ==== 100+0+11 (secure 0 0 0) 0x7f3c380567d0 con 0x7f3c480afce0 2024-05-28T14:04:05.800 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:04:05.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.815+0000 7f3c4fa00640 1 -- 172.21.15.42:0/2901601602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3c28068bb0 msgr2=0x7f3c2806b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:05.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.815+0000 7f3c4fa00640 1 --2- 172.21.15.42:0/2901601602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3c28068bb0 0x7f3c2806b070 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3c48162e00 tx=0x7f3c34009110 comp rx=0 tx=0).stop 2024-05-28T14:04:05.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.815+0000 7f3c4fa00640 1 -- 172.21.15.42:0/2901601602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c480afce0 msgr2=0x7f3c480ad560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:05.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.815+0000 7f3c4fa00640 1 --2- 172.21.15.42:0/2901601602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c480afce0 0x7f3c480ad560 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800c390 tx=0x7f3c3800c790 comp rx=0 tx=0).stop 2024-05-28T14:04:05.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.815+0000 7f3c4fa00640 1 -- 172.21.15.42:0/2901601602 shutdown_connections 2024-05-28T14:04:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.815+0000 7f3c4fa00640 1 --2- 172.21.15.42:0/2901601602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3c28068bb0 0x7f3c2806b070 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.815+0000 7f3c4fa00640 1 --2- 172.21.15.42:0/2901601602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c48144e60 0x7f3c480a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.815+0000 7f3c4fa00640 1 --2- 172.21.15.42:0/2901601602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c480b5410 0x7f3c48143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.815+0000 7f3c4fa00640 1 --2- 172.21.15.42:0/2901601602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c480afce0 0x7f3c480ad560 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.815+0000 7f3c4fa00640 1 -- 172.21.15.42:0/2901601602 >> 172.21.15.42:0/2901601602 conn(0x7f3c480b21f0 msgr2=0x7f3c48162300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:05.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.815+0000 7f3c4fa00640 1 -- 172.21.15.42:0/2901601602 shutdown_connections 2024-05-28T14:04:05.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:05.815+0000 7f3c4fa00640 1 -- 172.21.15.42:0/2901601602 wait complete. 2024-05-28T14:04:05.841 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T14:04:06.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.079+0000 7fc2e4000640 1 Processor -- start 2024-05-28T14:04:06.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.079+0000 7fc2e4000640 1 -- start start 2024-05-28T14:04:06.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.079+0000 7fc2e4000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2dc15fdf0 0x7fc2dc0cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:06.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.079+0000 7fc2e4000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc2dc0d0ea0 0x7fc2dc0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:06.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.079+0000 7fc2e4000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2dc0ce400 con 0x7fc2dc09baf0 2024-05-28T14:04:06.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.079+0000 7fc2e4000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc2dc0ce6e0 con 0x7fc2dc0d0ea0 2024-05-28T14:04:06.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.079+0000 7fc2e4000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc2dc0b1ea0 con 0x7fc2dc15fdf0 2024-05-28T14:04:06.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.079+0000 7fc2e0c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc2dc09baf0 0x7fc2dc0d0790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:54374/0 (socket says 172.21.15.42:54374) 2024-05-28T14:04:06.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.079+0000 7fc2e0c00640 1 -- 172.21.15.42:0/1002399012 learned_addr learned my addr 172.21.15.42:0/1002399012 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:06.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.083+0000 7fc2e1600640 1 --2- 172.21.15.42:0/1002399012 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2dc15fdf0 0x7fc2dc0cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:06.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.083+0000 7fc2e2000640 1 --2- 172.21.15.42:0/1002399012 >> v2:172.21.15.42:3300/0 conn(0x7fc2dc0d0ea0 0x7fc2dc0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:06.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.083+0000 7fc2dbe00640 1 -- 172.21.15.42:0/1002399012 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2037357864 0 0) 0x7fc2dc0ce400 con 0x7fc2dc09baf0 2024-05-28T14:04:06.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.083+0000 7fc2dbe00640 1 -- 172.21.15.42:0/1002399012 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2c4003530 con 0x7fc2dc09baf0 2024-05-28T14:04:06.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.083+0000 7fc2e1600640 1 -- 172.21.15.42:0/1002399012 >> v2:172.21.15.42:3300/0 conn(0x7fc2dc0d0ea0 msgr2=0x7fc2dc0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:06.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.083+0000 7fc2e1600640 1 --2- 172.21.15.42:0/1002399012 >> v2:172.21.15.42:3300/0 conn(0x7fc2dc0d0ea0 0x7fc2dc0b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:06.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.083+0000 7fc2e1600640 1 -- 172.21.15.42:0/1002399012 >> v1:172.21.15.42:6789/0 conn(0x7fc2dc09baf0 legacy=0x7fc2dc0d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:06.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.083+0000 7fc2e1600640 1 -- 172.21.15.42:0/1002399012 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2dc0b2180 con 0x7fc2dc15fdf0 2024-05-28T14:04:06.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.087+0000 7fc2e1600640 1 --2- 172.21.15.42:0/1002399012 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2dc15fdf0 0x7fc2dc0cdbd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009d90 tx=0x7fc2d401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a9041a41f6cf15 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:06.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.087+0000 7fc2dbe00640 1 -- 172.21.15.42:0/1002399012 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d40b2070 con 0x7fc2dc15fdf0 2024-05-28T14:04:06.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.087+0000 7fc2dbe00640 1 -- 172.21.15.42:0/1002399012 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2d40a57a0 con 0x7fc2dc15fdf0 2024-05-28T14:04:06.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.087+0000 7fc2dbe00640 1 -- 172.21.15.42:0/1002399012 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d40a5b70 con 0x7fc2dc15fdf0 2024-05-28T14:04:06.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.091+0000 7fc2e4000640 1 -- 172.21.15.42:0/1002399012 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2dc15fdf0 msgr2=0x7fc2dc0cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:06.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.091+0000 7fc2e4000640 1 --2- 172.21.15.42:0/1002399012 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2dc15fdf0 0x7fc2dc0cdbd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009d90 tx=0x7fc2d401c900 comp rx=0 tx=0).stop 2024-05-28T14:04:06.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.091+0000 7fc2e4000640 1 -- 172.21.15.42:0/1002399012 shutdown_connections 2024-05-28T14:04:06.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.091+0000 7fc2e4000640 1 --2- 172.21.15.42:0/1002399012 >> v2:172.21.15.42:3300/0 conn(0x7fc2dc0d0ea0 0x7fc2dc0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:06.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.091+0000 7fc2e4000640 1 --2- 172.21.15.42:0/1002399012 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2dc15fdf0 0x7fc2dc0cdbd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:06.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.091+0000 7fc2e4000640 1 -- 172.21.15.42:0/1002399012 >> 172.21.15.42:0/1002399012 conn(0x7fc2dc0b4050 msgr2=0x7fc2dc0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:06.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.091+0000 7fc2e4000640 1 -- 172.21.15.42:0/1002399012 shutdown_connections 2024-05-28T14:04:06.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.091+0000 7fc2e4000640 1 -- 172.21.15.42:0/1002399012 wait complete. 2024-05-28T14:04:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e4000640 1 Processor -- start 2024-05-28T14:04:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e4000640 1 -- start start 2024-05-28T14:04:06.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e4000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2dc09baf0 0x7fc2dc153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:06.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e4000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2dc0d0ea0 0x7fc2dc0c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:06.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e4000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2dc15fdf0 0x7fc2dc0c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:06.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e4000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc0b3830 con 0x7fc2dc0d0ea0 2024-05-28T14:04:06.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e4000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc0b36b0 con 0x7fc2dc15fdf0 2024-05-28T14:04:06.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e4000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc2dc0b39b0 con 0x7fc2dc09baf0 2024-05-28T14:04:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2dc0d0ea0 0x7fc2dc0c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2dc0d0ea0 0x7fc2dc0c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47832/0 (socket says 172.21.15.42:47832) 2024-05-28T14:04:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e0c00640 1 -- 172.21.15.42:0/1829158862 learned_addr learned my addr 172.21.15.42:0/1829158862 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e2000640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2dc15fdf0 0x7fc2dc0c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e1600640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2dc09baf0 0x7fc2dc153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e0c00640 1 -- 172.21.15.42:0/1829158862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2dc09baf0 msgr2=0x7fc2dc153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e0c00640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2dc09baf0 0x7fc2dc153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e0c00640 1 -- 172.21.15.42:0/1829158862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2dc15fdf0 msgr2=0x7fc2dc0c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e0c00640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2dc15fdf0 0x7fc2dc0c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:06.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.095+0000 7fc2e0c00640 1 -- 172.21.15.42:0/1829158862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2dc0c68a0 con 0x7fc2dc0d0ea0 2024-05-28T14:04:06.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.099+0000 7fc2e1600640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2dc09baf0 0x7fc2dc153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.099+0000 7fc2e0c00640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2dc0d0ea0 0x7fc2dc0c5fa0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800c4d0 tx=0x7fc2c800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.099+0000 7fc2d9600640 1 -- 172.21.15.42:0/1829158862 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8017070 con 0x7fc2dc0d0ea0 2024-05-28T14:04:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.099+0000 7fc2d9600640 1 -- 172.21.15.42:0/1829158862 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c8005cf0 con 0x7fc2dc0d0ea0 2024-05-28T14:04:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.099+0000 7fc2d9600640 1 -- 172.21.15.42:0/1829158862 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8007270 con 0x7fc2dc0d0ea0 2024-05-28T14:04:06.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.099+0000 7fc2e4000640 1 -- 172.21.15.42:0/1829158862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2dc0c6b80 con 0x7fc2dc0d0ea0 2024-05-28T14:04:06.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.099+0000 7fc2e4000640 1 -- 172.21.15.42:0/1829158862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2dc0c7140 con 0x7fc2dc0d0ea0 2024-05-28T14:04:06.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.103+0000 7fc2e4000640 1 -- 172.21.15.42:0/1829158862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2dc0a0650 con 0x7fc2dc0d0ea0 2024-05-28T14:04:06.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.103+0000 7fc2d9600640 1 -- 172.21.15.42:0/1829158862 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fc2c801a070 con 0x7fc2dc0d0ea0 2024-05-28T14:04:06.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.107+0000 7fc2d9600640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc2bc068bb0 0x7fc2bc06b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:06.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.107+0000 7fc2d9600640 1 -- 172.21.15.42:0/1829158862 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fc2c8086e70 con 0x7fc2dc0d0ea0 2024-05-28T14:04:06.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.107+0000 7fc2e1600640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc2bc068bb0 0x7fc2bc06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:06.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.107+0000 7fc2e1600640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc2bc068bb0 0x7fc2bc06b070 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40af000 tx=0x7fc2d40a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:06.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.115+0000 7fc2d9600640 1 -- 172.21.15.42:0/1829158862 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc2c8055f30 con 0x7fc2dc0d0ea0 2024-05-28T14:04:06.459 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T14:04:06.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.471+0000 7fc2e4000640 1 -- 172.21.15.42:0/1829158862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc2dc04fdd0 con 0x7fc2dc0d0ea0 2024-05-28T14:04:06.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.471+0000 7fc2d9600640 1 -- 172.21.15.42:0/1829158862 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) ==== 80+0+0 (secure 0 0 0) 0x7fc2c805a230 con 0x7fc2dc0d0ea0 2024-05-28T14:04:06.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.475+0000 7fc2e4000640 1 -- 172.21.15.42:0/1829158862 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc2bc068bb0 msgr2=0x7fc2bc06b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:06.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.475+0000 7fc2e4000640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc2bc068bb0 0x7fc2bc06b070 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40af000 tx=0x7fc2d40a7070 comp rx=0 tx=0).stop 2024-05-28T14:04:06.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.475+0000 7fc2e4000640 1 -- 172.21.15.42:0/1829158862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2dc0d0ea0 msgr2=0x7fc2dc0c5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:06.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.475+0000 7fc2e4000640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2dc0d0ea0 0x7fc2dc0c5fa0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800c4d0 tx=0x7fc2c800c990 comp rx=0 tx=0).stop 2024-05-28T14:04:06.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.475+0000 7fc2e4000640 1 -- 172.21.15.42:0/1829158862 shutdown_connections 2024-05-28T14:04:06.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.475+0000 7fc2e4000640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc2bc068bb0 0x7fc2bc06b070 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:06.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.475+0000 7fc2e4000640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2dc15fdf0 0x7fc2dc0c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:06.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.475+0000 7fc2e4000640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2dc0d0ea0 0x7fc2dc0c5fa0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:06.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.475+0000 7fc2e4000640 1 --2- 172.21.15.42:0/1829158862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2dc09baf0 0x7fc2dc153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:06.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.475+0000 7fc2e4000640 1 -- 172.21.15.42:0/1829158862 >> 172.21.15.42:0/1829158862 conn(0x7fc2dc0b4050 msgr2=0x7fc2dc0cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:06.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.475+0000 7fc2e4000640 1 -- 172.21.15.42:0/1829158862 shutdown_connections 2024-05-28T14:04:06.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:06.475+0000 7fc2e4000640 1 -- 172.21.15.42:0/1829158862 wait complete. 2024-05-28T14:04:06.494 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-05-28T14:04:06.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:06.495 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-28T14:04:06.540 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:04:06.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:06.540 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-28T14:04:06.543 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[39057]: fuse finished with error 0 and tester_r 0 2024-05-28T14:04:06.684 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:04:06.684 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:04:06.684 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:04:06.697 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:04:06.697 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:04:06.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:06.698 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-28T14:04:06.753 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-28T14:04:06.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:04:06.754 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:04:06.755 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:04:06.755 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:04:06.767 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:04:06.767 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:04:06.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:06.767 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-28T14:04:06.819 INFO:teuthology.orchestra.run.smithi042.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-28T14:04:06.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:04:06.821 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:04:06.821 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:04:06.821 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:04:06.837 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:04:06.837 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:04:06.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:06.837 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-28T14:04:06.893 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-28T14:04:06.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:04:06.894 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:04:07.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c7000640 1 Processor -- start 2024-05-28T14:04:07.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c7000640 1 -- start start 2024-05-28T14:04:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c7000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f79c0070800 0x7f79c0070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c7000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f79c006ff10 0x7f79c009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c7000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c00d0660 con 0x7f79c00703a0 2024-05-28T14:04:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c7000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f79c00d0940 con 0x7f79c0070800 2024-05-28T14:04:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c7000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f79c00d0c20 con 0x7f79c006ff10 2024-05-28T14:04:07.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c6600640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f79c00703a0 0x7f79c009af90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:54390/0 (socket says 172.21.15.42:54390) 2024-05-28T14:04:07.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c5c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f79c0070800 0x7f79c0070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c6600640 1 -- 172.21.15.42:0/369775010 learned_addr learned my addr 172.21.15.42:0/369775010 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:07.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c5200640 1 --2- 172.21.15.42:0/369775010 >> v2:172.21.15.63:3300/0 conn(0x7f79c006ff10 0x7f79c009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c5c00640 1 -- 172.21.15.42:0/369775010 >> v2:172.21.15.63:3300/0 conn(0x7f79c006ff10 msgr2=0x7f79c009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:07.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c5c00640 1 --2- 172.21.15.42:0/369775010 >> v2:172.21.15.63:3300/0 conn(0x7f79c006ff10 0x7f79c009aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c5c00640 1 -- 172.21.15.42:0/369775010 >> v1:172.21.15.42:6789/0 conn(0x7f79c00703a0 legacy=0x7f79c009af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79c5c00640 1 -- 172.21.15.42:0/369775010 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c00d0f00 con 0x7f79c0070800 2024-05-28T14:04:07.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.051+0000 7f79bfe00640 1 -- 172.21.15.42:0/369775010 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3375056288 0 0) 0x7f79c00d0660 con 0x7f79c00703a0 2024-05-28T14:04:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.055+0000 7f79c5c00640 1 --2- 172.21.15.42:0/369775010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79c0070800 0x7f79c0070be0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f79b40097c0 tx=0x7f79b401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce45e4536423679 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.055+0000 7f79bfe00640 1 -- 172.21.15.42:0/369775010 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b40b0070 con 0x7f79c0070800 2024-05-28T14:04:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.055+0000 7f79bfe00640 1 -- 172.21.15.42:0/369775010 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b40ab600 con 0x7f79c0070800 2024-05-28T14:04:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.055+0000 7f79bfe00640 1 -- 172.21.15.42:0/369775010 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b40ab9d0 con 0x7f79c0070800 2024-05-28T14:04:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.055+0000 7f79c7000640 1 -- 172.21.15.42:0/369775010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79c0070800 msgr2=0x7f79c0070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.055+0000 7f79c7000640 1 --2- 172.21.15.42:0/369775010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79c0070800 0x7f79c0070be0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f79b40097c0 tx=0x7f79b401c460 comp rx=0 tx=0).stop 2024-05-28T14:04:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.055+0000 7f79c7000640 1 -- 172.21.15.42:0/369775010 shutdown_connections 2024-05-28T14:04:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.055+0000 7f79c7000640 1 --2- 172.21.15.42:0/369775010 >> v2:172.21.15.63:3300/0 conn(0x7f79c006ff10 0x7f79c009aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.055+0000 7f79c7000640 1 --2- 172.21.15.42:0/369775010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79c0070800 0x7f79c0070be0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.055+0000 7f79c7000640 1 -- 172.21.15.42:0/369775010 >> 172.21.15.42:0/369775010 conn(0x7f79c00b3dd0 msgr2=0x7f79c00b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.055+0000 7f79c7000640 1 -- 172.21.15.42:0/369775010 shutdown_connections 2024-05-28T14:04:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.055+0000 7f79c7000640 1 -- 172.21.15.42:0/369775010 wait complete. 2024-05-28T14:04:07.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.055+0000 7f79c7000640 1 Processor -- start 2024-05-28T14:04:07.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.055+0000 7f79c7000640 1 -- start start 2024-05-28T14:04:07.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c7000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79c006ff10 0x7f79c006dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c7000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79c00703a0 0x7f79c00ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c7000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79c0070800 0x7f79c00ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c7000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f79c00b1a10 con 0x7f79c006ff10 2024-05-28T14:04:07.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c7000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79c00b1890 con 0x7f79c0070800 2024-05-28T14:04:07.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c7000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f79c016bde0 con 0x7f79c00703a0 2024-05-28T14:04:07.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c5c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79c006ff10 0x7f79c006dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c5c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79c006ff10 0x7f79c006dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47842/0 (socket says 172.21.15.42:47842) 2024-05-28T14:04:07.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c5c00640 1 -- 172.21.15.42:0/741212193 learned_addr learned my addr 172.21.15.42:0/741212193 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:07.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c6600640 1 --2- 172.21.15.42:0/741212193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79c0070800 0x7f79c00ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c5200640 1 --2- 172.21.15.42:0/741212193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79c00703a0 0x7f79c00ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c5c00640 1 -- 172.21.15.42:0/741212193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79c00703a0 msgr2=0x7f79c00ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c5c00640 1 --2- 172.21.15.42:0/741212193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79c00703a0 0x7f79c00ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c5c00640 1 -- 172.21.15.42:0/741212193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79c0070800 msgr2=0x7f79c00ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c5c00640 1 --2- 172.21.15.42:0/741212193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79c0070800 0x7f79c00ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c5c00640 1 -- 172.21.15.42:0/741212193 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c00b3040 con 0x7f79c006ff10 2024-05-28T14:04:07.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c5c00640 1 --2- 172.21.15.42:0/741212193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79c006ff10 0x7f79c006dc70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f79b40ae040 tx=0x7f79b401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79bd600640 1 -- 172.21.15.42:0/741212193 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b40b0050 con 0x7f79c006ff10 2024-05-28T14:04:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79bd600640 1 -- 172.21.15.42:0/741212193 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b40bc070 con 0x7f79c006ff10 2024-05-28T14:04:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79bd600640 1 -- 172.21.15.42:0/741212193 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4006e20 con 0x7f79c006ff10 2024-05-28T14:04:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c7000640 1 -- 172.21.15.42:0/741212193 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c014d2e0 con 0x7f79c006ff10 2024-05-28T14:04:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c7000640 1 -- 172.21.15.42:0/741212193 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c0152f90 con 0x7f79c006ff10 2024-05-28T14:04:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79c7000640 1 -- 172.21.15.42:0/741212193 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c004fcf0 con 0x7f79c006ff10 2024-05-28T14:04:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.059+0000 7f79bd600640 1 -- 172.21.15.42:0/741212193 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f79b40b5870 con 0x7f79c006ff10 2024-05-28T14:04:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.063+0000 7f79bd600640 1 --2- 172.21.15.42:0/741212193 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f79a0068b90 0x7f79a006b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.063+0000 7f79bd600640 1 -- 172.21.15.42:0/741212193 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f79b4121fd0 con 0x7f79c006ff10 2024-05-28T14:04:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.063+0000 7f79c5200640 1 --2- 172.21.15.42:0/741212193 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f79a0068b90 0x7f79a006b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.063+0000 7f79c5200640 1 --2- 172.21.15.42:0/741212193 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f79a0068b90 0x7f79a006b050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f79c00cdf10 tx=0x7f79ac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:07.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.067+0000 7f79bd600640 1 -- 172.21.15.42:0/741212193 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f79b4016050 con 0x7f79c006ff10 2024-05-28T14:04:07.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.271+0000 7f79c7000640 1 -- 172.21.15.42:0/741212193 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79c01693a0 con 0x7f79c006ff10 2024-05-28T14:04:07.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.271+0000 7f79bd600640 1 -- 172.21.15.42:0/741212193 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5180 (secure 0 0 0) 0x7f79b40f18a0 con 0x7f79c006ff10 2024-05-28T14:04:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:07.257 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3694688875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3694688875},{"type":"v1","addr":"172.21.15.63:6835","nonce":3694688875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5799,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/199639725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":199639725},{"type":"v1","addr":"172.21.15.42:6836","nonce":199639725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1043559054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1043559054},{"type":"v1","addr":"172.21.15.42:6837","nonce":1043559054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:01:50.630732+0000","modified":"2024-05-28T14:01:53.667008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":55184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55184":{"gid":55184,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6833/754542999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":754542999},{"type":"v1","addr":"172.21.15.63:6833","nonce":754542999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":55184,"qdb_cluster":[55184]},"id":3}]} 2024-05-28T14:04:07.257 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 28 2024-05-28T14:04:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.271+0000 7f79c7000640 1 -- 172.21.15.42:0/741212193 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f79a0068b90 msgr2=0x7f79a006b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.271+0000 7f79c7000640 1 --2- 172.21.15.42:0/741212193 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f79a0068b90 0x7f79a006b050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f79c00cdf10 tx=0x7f79ac008040 comp rx=0 tx=0).stop 2024-05-28T14:04:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.271+0000 7f79c7000640 1 -- 172.21.15.42:0/741212193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79c006ff10 msgr2=0x7f79c006dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.271+0000 7f79c7000640 1 --2- 172.21.15.42:0/741212193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79c006ff10 0x7f79c006dc70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f79b40ae040 tx=0x7f79b401aca0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.271+0000 7f79c7000640 1 -- 172.21.15.42:0/741212193 shutdown_connections 2024-05-28T14:04:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.271+0000 7f79c7000640 1 --2- 172.21.15.42:0/741212193 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f79a0068b90 0x7f79a006b050 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.271+0000 7f79c7000640 1 --2- 172.21.15.42:0/741212193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79c0070800 0x7f79c00ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.271+0000 7f79c7000640 1 --2- 172.21.15.42:0/741212193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79c00703a0 0x7f79c00ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.271+0000 7f79c7000640 1 --2- 172.21.15.42:0/741212193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79c006ff10 0x7f79c006dc70 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.271+0000 7f79c7000640 1 -- 172.21.15.42:0/741212193 >> 172.21.15.42:0/741212193 conn(0x7f79c00b3dd0 msgr2=0x7f79c014ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.271+0000 7f79c7000640 1 -- 172.21.15.42:0/741212193 shutdown_connections 2024-05-28T14:04:07.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.271+0000 7f79c7000640 1 -- 172.21.15.42:0/741212193 wait complete. 2024-05-28T14:04:07.280 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:04:07.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64ef400640 1 Processor -- start 2024-05-28T14:04:07.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64ef400640 1 -- start start 2024-05-28T14:04:07.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64ef400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64e80ce860 0x7f64e806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64ef400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f64e80d0900 0x7f64e8162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64ef400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f64e806dcf0 0x7f64e8162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64ef400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f64e80d19f0 con 0x7f64e80d0900 2024-05-28T14:04:07.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64ef400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f64e80d0de0 con 0x7f64e806dcf0 2024-05-28T14:04:07.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64ef400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f64e80d10c0 con 0x7f64e80ce860 2024-05-28T14:04:07.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64dfe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f64e80d0900 0x7f64e8162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64eca00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64e80ce860 0x7f64e806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64dfe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f64e80d0900 0x7f64e8162570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35092/0 (socket says 172.21.15.42:35092) 2024-05-28T14:04:07.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64ed400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f64e806dcf0 0x7f64e8162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64eca00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64e80ce860 0x7f64e806d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:40550/0 (socket says 172.21.15.42:40550) 2024-05-28T14:04:07.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64dfe00640 1 -- 172.21.15.42:0/1810436398 learned_addr learned my addr 172.21.15.42:0/1810436398 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:07.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64dfe00640 1 -- 172.21.15.42:0/1810436398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64e80ce860 msgr2=0x7f64e806d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:07.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64dfe00640 1 --2- 172.21.15.42:0/1810436398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64e80ce860 0x7f64e806d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64dfe00640 1 -- 172.21.15.42:0/1810436398 >> v2:172.21.15.63:3300/0 conn(0x7f64e806dcf0 msgr2=0x7f64e8162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:07.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64dfe00640 1 --2- 172.21.15.42:0/1810436398 >> v2:172.21.15.63:3300/0 conn(0x7f64e806dcf0 0x7f64e8162ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.439+0000 7f64dfe00640 1 -- 172.21.15.42:0/1810436398 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e80d13a0 con 0x7f64e80d0900 2024-05-28T14:04:07.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.443+0000 7f64dfe00640 1 --2- 172.21.15.42:0/1810436398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e80d0900 0x7f64e8162570 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f64e8073870 tx=0x7f64d001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3d24a13f6d52e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:07.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.443+0000 7f64e7e00640 1 -- 172.21.15.42:0/1810436398 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d00ab640 con 0x7f64e80d0900 2024-05-28T14:04:07.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.443+0000 7f64e7e00640 1 -- 172.21.15.42:0/1810436398 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64d00b0070 con 0x7f64e80d0900 2024-05-28T14:04:07.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.443+0000 7f64e7e00640 1 -- 172.21.15.42:0/1810436398 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d00aba00 con 0x7f64e80d0900 2024-05-28T14:04:07.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.443+0000 7f64ef400640 1 -- 172.21.15.42:0/1810436398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e80d0900 msgr2=0x7f64e8162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.443+0000 7f64ef400640 1 --2- 172.21.15.42:0/1810436398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e80d0900 0x7f64e8162570 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f64e8073870 tx=0x7f64d001c460 comp rx=0 tx=0).stop 2024-05-28T14:04:07.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.443+0000 7f64ef400640 1 -- 172.21.15.42:0/1810436398 shutdown_connections 2024-05-28T14:04:07.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.443+0000 7f64ef400640 1 --2- 172.21.15.42:0/1810436398 >> v2:172.21.15.63:3300/0 conn(0x7f64e806dcf0 0x7f64e8162ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.443+0000 7f64ef400640 1 --2- 172.21.15.42:0/1810436398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e80d0900 0x7f64e8162570 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.443+0000 7f64ef400640 1 --2- 172.21.15.42:0/1810436398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64e80ce860 0x7f64e806d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.443+0000 7f64ef400640 1 -- 172.21.15.42:0/1810436398 >> 172.21.15.42:0/1810436398 conn(0x7f64e80b4080 msgr2=0x7f64e80b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:07.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.443+0000 7f64ef400640 1 -- 172.21.15.42:0/1810436398 shutdown_connections 2024-05-28T14:04:07.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.443+0000 7f64ef400640 1 -- 172.21.15.42:0/1810436398 wait complete. 2024-05-28T14:04:07.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64ef400640 1 Processor -- start 2024-05-28T14:04:07.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64ef400640 1 -- start start 2024-05-28T14:04:07.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64ef400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e806dcf0 0x7f64e81548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64ef400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64e80ce860 0x7f64e815f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64ef400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64e80d0900 0x7f64e814f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64ef400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f64e815eda0 con 0x7f64e806dcf0 2024-05-28T14:04:07.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64ef400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64e815ec20 con 0x7f64e80d0900 2024-05-28T14:04:07.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64ef400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f64e809aa50 con 0x7f64e80ce860 2024-05-28T14:04:07.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64eca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e806dcf0 0x7f64e81548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64eca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e806dcf0 0x7f64e81548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35098/0 (socket says 172.21.15.42:35098) 2024-05-28T14:04:07.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64eca00640 1 -- 172.21.15.42:0/3344897872 learned_addr learned my addr 172.21.15.42:0/3344897872 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:07.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64dfe00640 1 --2- 172.21.15.42:0/3344897872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64e80ce860 0x7f64e815f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64ed400640 1 --2- 172.21.15.42:0/3344897872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64e80d0900 0x7f64e814f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64eca00640 1 -- 172.21.15.42:0/3344897872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64e80ce860 msgr2=0x7f64e815f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64eca00640 1 --2- 172.21.15.42:0/3344897872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64e80ce860 0x7f64e815f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64eca00640 1 -- 172.21.15.42:0/3344897872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64e80d0900 msgr2=0x7f64e814f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64eca00640 1 --2- 172.21.15.42:0/3344897872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64e80d0900 0x7f64e814f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64eca00640 1 -- 172.21.15.42:0/3344897872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e8155130 con 0x7f64e806dcf0 2024-05-28T14:04:07.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64eca00640 1 --2- 172.21.15.42:0/3344897872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e806dcf0 0x7f64e81548b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f64d800e930 tx=0x7f64d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:07.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64e5600640 1 -- 172.21.15.42:0/3344897872 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8017070 con 0x7f64e806dcf0 2024-05-28T14:04:07.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64e5600640 1 -- 172.21.15.42:0/3344897872 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64d800ca00 con 0x7f64e806dcf0 2024-05-28T14:04:07.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64e5600640 1 -- 172.21.15.42:0/3344897872 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d800cdb0 con 0x7f64e806dcf0 2024-05-28T14:04:07.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64ef400640 1 -- 172.21.15.42:0/3344897872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e80c6d80 con 0x7f64e806dcf0 2024-05-28T14:04:07.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.447+0000 7f64ef400640 1 -- 172.21.15.42:0/3344897872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e80c7340 con 0x7f64e806dcf0 2024-05-28T14:04:07.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.451+0000 7f64ef400640 1 -- 172.21.15.42:0/3344897872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e8068a90 con 0x7f64e806dcf0 2024-05-28T14:04:07.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.451+0000 7f64e5600640 1 -- 172.21.15.42:0/3344897872 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f64d801a070 con 0x7f64e806dcf0 2024-05-28T14:04:07.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.451+0000 7f64e5600640 1 --2- 172.21.15.42:0/3344897872 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f64c4068b60 0x7f64c406b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.451+0000 7f64e5600640 1 -- 172.21.15.42:0/3344897872 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5851+0+0 (secure 0 0 0) 0x7f64d8087fa0 con 0x7f64e806dcf0 2024-05-28T14:04:07.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.451+0000 7f64dfe00640 1 --2- 172.21.15.42:0/3344897872 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f64c4068b60 0x7f64c406b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.451+0000 7f64dfe00640 1 --2- 172.21.15.42:0/3344897872 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f64c4068b60 0x7f64c406b020 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f64d00af000 tx=0x7f64d00a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:07.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.455+0000 7f64e5600640 1 -- 172.21.15.42:0/3344897872 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f64d8056fb0 con 0x7f64e806dcf0 2024-05-28T14:04:07.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.627+0000 7f64ef400640 1 -- 172.21.15.42:0/3344897872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64e80b4ce0 con 0x7f64e806dcf0 2024-05-28T14:04:07.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.627+0000 7f64e5600640 1 -- 172.21.15.42:0/3344897872 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16857 (secure 0 0 0) 0x7f64d805b2b0 con 0x7f64e806dcf0 2024-05-28T14:04:07.613 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:07.613 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":49,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:04:07.083179+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:01:49.041938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-28T14:01:49.591543+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.1b","mappings":[{"from":0,"to":4}]},{"pgid":"7.30","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:04:07.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.627+0000 7f64ef400640 1 -- 172.21.15.42:0/3344897872 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f64c4068b60 msgr2=0x7f64c406b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.627+0000 7f64ef400640 1 --2- 172.21.15.42:0/3344897872 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f64c4068b60 0x7f64c406b020 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f64d00af000 tx=0x7f64d00a7000 comp rx=0 tx=0).stop 2024-05-28T14:04:07.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.627+0000 7f64ef400640 1 -- 172.21.15.42:0/3344897872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e806dcf0 msgr2=0x7f64e81548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.627+0000 7f64ef400640 1 --2- 172.21.15.42:0/3344897872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e806dcf0 0x7f64e81548b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f64d800e930 tx=0x7f64d800edf0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.627+0000 7f64ef400640 1 -- 172.21.15.42:0/3344897872 shutdown_connections 2024-05-28T14:04:07.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.627+0000 7f64ef400640 1 --2- 172.21.15.42:0/3344897872 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f64c4068b60 0x7f64c406b020 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.627+0000 7f64ef400640 1 --2- 172.21.15.42:0/3344897872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64e80d0900 0x7f64e814f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.627+0000 7f64ef400640 1 --2- 172.21.15.42:0/3344897872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64e80ce860 0x7f64e815f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.627+0000 7f64ef400640 1 --2- 172.21.15.42:0/3344897872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e806dcf0 0x7f64e81548b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.627+0000 7f64ef400640 1 -- 172.21.15.42:0/3344897872 >> 172.21.15.42:0/3344897872 conn(0x7f64e80b4080 msgr2=0x7f64e80703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.627+0000 7f64ef400640 1 -- 172.21.15.42:0/3344897872 shutdown_connections 2024-05-28T14:04:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.627+0000 7f64ef400640 1 -- 172.21.15.42:0/3344897872 wait complete. 2024-05-28T14:04:07.635 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T14:01:49.041938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T14:01:49.591543+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 49, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:04:07.635 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:04:07.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.791+0000 7f0d35400640 1 Processor -- start 2024-05-28T14:04:07.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.791+0000 7f0d35400640 1 -- start start 2024-05-28T14:04:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.795+0000 7f0d35400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0d300cf540 0x7f0d300cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.795+0000 7f0d35400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d30160170 con 0x7f0d300707e0 2024-05-28T14:04:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.795+0000 7f0d35400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d30160450 con 0x7f0d30145d50 2024-05-28T14:04:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.795+0000 7f0d35400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0d30160730 con 0x7f0d300cf540 2024-05-28T14:04:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.795+0000 7f0d2fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0d300cf540 0x7f0d300cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.795+0000 7f0d34a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0d300707e0 0x7f0d30071e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47556/0 (socket says 172.21.15.42:47556) 2024-05-28T14:04:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.795+0000 7f0d34a00640 1 -- 172.21.15.42:0/3776275173 learned_addr learned my addr 172.21.15.42:0/3776275173 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.795+0000 7f0d2f400640 1 -- 172.21.15.42:0/3776275173 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1512002215 0 0) 0x7f0d30160170 con 0x7f0d300707e0 2024-05-28T14:04:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.795+0000 7f0d2f400640 1 -- 172.21.15.42:0/3776275173 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d14003530 con 0x7f0d300707e0 2024-05-28T14:04:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.795+0000 7f0d2fe00640 1 -- 172.21.15.42:0/3776275173 >> v1:172.21.15.63:6789/0 conn(0x7f0d30145d50 legacy=0x7f0d30071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.795+0000 7f0d2fe00640 1 -- 172.21.15.42:0/3776275173 >> v1:172.21.15.42:6789/0 conn(0x7f0d300707e0 legacy=0x7f0d30071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.795+0000 7f0d2fe00640 1 -- 172.21.15.42:0/3776275173 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d30072530 con 0x7f0d300cf540 2024-05-28T14:04:07.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d2fe00640 1 --2- 172.21.15.42:0/3776275173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d300cf540 0x7f0d300cc940 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009d90 tx=0x7f0d2001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e6e981a6407595 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:07.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d2f400640 1 -- 172.21.15.42:0/3776275173 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d200a66a0 con 0x7f0d300cf540 2024-05-28T14:04:07.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d2f400640 1 -- 172.21.15.42:0/3776275173 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d200a6840 con 0x7f0d300cf540 2024-05-28T14:04:07.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d2f400640 1 -- 172.21.15.42:0/3776275173 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d200a6c10 con 0x7f0d300cf540 2024-05-28T14:04:07.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 -- 172.21.15.42:0/3776275173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d300cf540 msgr2=0x7f0d300cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 --2- 172.21.15.42:0/3776275173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d300cf540 0x7f0d300cc940 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009d90 tx=0x7f0d2001c8e0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 -- 172.21.15.42:0/3776275173 shutdown_connections 2024-05-28T14:04:07.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 --2- 172.21.15.42:0/3776275173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d300cf540 0x7f0d300cc940 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 -- 172.21.15.42:0/3776275173 >> 172.21.15.42:0/3776275173 conn(0x7f0d3006fcc0 msgr2=0x7f0d300700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:07.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 -- 172.21.15.42:0/3776275173 shutdown_connections 2024-05-28T14:04:07.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 -- 172.21.15.42:0/3776275173 wait complete. 2024-05-28T14:04:07.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 Processor -- start 2024-05-28T14:04:07.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 -- start start 2024-05-28T14:04:07.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d300707e0 0x7f0d300c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d300cf540 0x7f0d300c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d30145d50 0x7f0d300c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3016ab70 con 0x7f0d300707e0 2024-05-28T14:04:07.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0d300d10e0 con 0x7f0d300cf540 2024-05-28T14:04:07.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.799+0000 7f0d35400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3016acf0 con 0x7f0d30145d50 2024-05-28T14:04:07.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d300707e0 0x7f0d300c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d34a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d30145d50 0x7f0d300c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d300707e0 0x7f0d300c73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35124/0 (socket says 172.21.15.42:35124) 2024-05-28T14:04:07.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d2fe00640 1 -- 172.21.15.42:0/3655199040 learned_addr learned my addr 172.21.15.42:0/3655199040 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:07.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d27400640 1 --2- 172.21.15.42:0/3655199040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d300cf540 0x7f0d300c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d2fe00640 1 -- 172.21.15.42:0/3655199040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d30145d50 msgr2=0x7f0d300c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d2fe00640 1 --2- 172.21.15.42:0/3655199040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d30145d50 0x7f0d300c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d2fe00640 1 -- 172.21.15.42:0/3655199040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d300cf540 msgr2=0x7f0d300c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d2fe00640 1 --2- 172.21.15.42:0/3655199040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d300cf540 0x7f0d300c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d2fe00640 1 -- 172.21.15.42:0/3655199040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d300b3660 con 0x7f0d300707e0 2024-05-28T14:04:07.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d2fe00640 1 --2- 172.21.15.42:0/3655199040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d300707e0 0x7f0d300c73b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0d2001a510 tx=0x7f0d20019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:07.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d2cc00640 1 -- 172.21.15.42:0/3655199040 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d200a7890 con 0x7f0d300707e0 2024-05-28T14:04:07.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d2cc00640 1 -- 172.21.15.42:0/3655199040 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d200bf070 con 0x7f0d300707e0 2024-05-28T14:04:07.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d35400640 1 -- 172.21.15.42:0/3655199040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d300b3380 con 0x7f0d300707e0 2024-05-28T14:04:07.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d2cc00640 1 -- 172.21.15.42:0/3655199040 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d200a7c90 con 0x7f0d300707e0 2024-05-28T14:04:07.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d35400640 1 -- 172.21.15.42:0/3655199040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d300c6bb0 con 0x7f0d300707e0 2024-05-28T14:04:07.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d35400640 1 -- 172.21.15.42:0/3655199040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3004fcf0 con 0x7f0d300707e0 2024-05-28T14:04:07.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.803+0000 7f0d2cc00640 1 -- 172.21.15.42:0/3655199040 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f0d200c3410 con 0x7f0d300707e0 2024-05-28T14:04:07.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.807+0000 7f0d2cc00640 1 --2- 172.21.15.42:0/3655199040 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d0c068b60 0x7f0d0c06b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:07.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.807+0000 7f0d2cc00640 1 -- 172.21.15.42:0/3655199040 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5851+0+0 (secure 0 0 0) 0x7f0d201228d0 con 0x7f0d300707e0 2024-05-28T14:04:07.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.807+0000 7f0d27400640 1 --2- 172.21.15.42:0/3655199040 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d0c068b60 0x7f0d0c06b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:07.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.807+0000 7f0d27400640 1 --2- 172.21.15.42:0/3655199040 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d0c068b60 0x7f0d0c06b020 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0d300ccbd0 tx=0x7f0d18009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:07.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.811+0000 7f0d2cc00640 1 -- 172.21.15.42:0/3655199040 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0d200b7070 con 0x7f0d300707e0 2024-05-28T14:04:07.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.975+0000 7f0d35400640 1 -- 172.21.15.42:0/3655199040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d300c7c10 con 0x7f0d300707e0 2024-05-28T14:04:07.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.975+0000 7f0d2cc00640 1 -- 172.21.15.42:0/3655199040 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) ==== 93+0+10 (secure 0 0 0) 0x7f0d200ac030 con 0x7f0d300707e0 2024-05-28T14:04:07.960 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:04:07.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.975+0000 7f0d35400640 1 -- 172.21.15.42:0/3655199040 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d0c068b60 msgr2=0x7f0d0c06b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.975+0000 7f0d35400640 1 --2- 172.21.15.42:0/3655199040 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d0c068b60 0x7f0d0c06b020 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0d300ccbd0 tx=0x7f0d18009040 comp rx=0 tx=0).stop 2024-05-28T14:04:07.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.975+0000 7f0d35400640 1 -- 172.21.15.42:0/3655199040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d300707e0 msgr2=0x7f0d300c73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:07.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.975+0000 7f0d35400640 1 --2- 172.21.15.42:0/3655199040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d300707e0 0x7f0d300c73b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0d2001a510 tx=0x7f0d20019680 comp rx=0 tx=0).stop 2024-05-28T14:04:07.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.975+0000 7f0d35400640 1 -- 172.21.15.42:0/3655199040 shutdown_connections 2024-05-28T14:04:07.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.975+0000 7f0d35400640 1 --2- 172.21.15.42:0/3655199040 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d0c068b60 0x7f0d0c06b020 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.975+0000 7f0d35400640 1 --2- 172.21.15.42:0/3655199040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d30145d50 0x7f0d300c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.975+0000 7f0d35400640 1 --2- 172.21.15.42:0/3655199040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d300cf540 0x7f0d300c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.975+0000 7f0d35400640 1 --2- 172.21.15.42:0/3655199040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d300707e0 0x7f0d300c73b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:07.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.975+0000 7f0d35400640 1 -- 172.21.15.42:0/3655199040 >> 172.21.15.42:0/3655199040 conn(0x7f0d3006fcc0 msgr2=0x7f0d30156600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:07.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.975+0000 7f0d35400640 1 -- 172.21.15.42:0/3655199040 shutdown_connections 2024-05-28T14:04:07.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:07.975+0000 7f0d35400640 1 -- 172.21.15.42:0/3655199040 wait complete. 2024-05-28T14:04:07.982 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:04:08.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.139+0000 7f22f3600640 1 Processor -- start 2024-05-28T14:04:08.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.139+0000 7f22f3600640 1 -- start start 2024-05-28T14:04:08.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.143+0000 7f22f3600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f22ec15cb50 0x7f22ec09b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.143+0000 7f22f3600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22ec0d11d0 0x7f22ec166470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.143+0000 7f22f3600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22ec09ba40 con 0x7f22ec15c770 2024-05-28T14:04:08.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.143+0000 7f22f3600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f22ec0c6720 con 0x7f22ec15cb50 2024-05-28T14:04:08.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.143+0000 7f22f3600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec0c60d0 con 0x7f22ec0d11d0 2024-05-28T14:04:08.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.143+0000 7f22e3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f22ec15cb50 0x7f22ec09b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.143+0000 7f22e3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f22ec15cb50 0x7f22ec09b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35140/0 (socket says 172.21.15.42:35140) 2024-05-28T14:04:08.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.143+0000 7f22e3e00640 1 -- 172.21.15.42:0/2762002254 learned_addr learned my addr 172.21.15.42:0/2762002254 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:08.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.143+0000 7f22f1600640 1 --2- 172.21.15.42:0/2762002254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22ec0d11d0 0x7f22ec166470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.143+0000 7f22e3e00640 1 -- 172.21.15.42:0/2762002254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22ec0d11d0 msgr2=0x7f22ec166470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:08.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.143+0000 7f22e3e00640 1 --2- 172.21.15.42:0/2762002254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22ec0d11d0 0x7f22ec166470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.143+0000 7f22e3e00640 1 -- 172.21.15.42:0/2762002254 >> v1:172.21.15.63:6789/0 conn(0x7f22ec15c770 legacy=0x7f22ec0d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.143+0000 7f22e3e00640 1 -- 172.21.15.42:0/2762002254 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec0c63b0 con 0x7f22ec15cb50 2024-05-28T14:04:08.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22e3e00640 1 --2- 172.21.15.42:0/2762002254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22ec15cb50 0x7f22ec09b460 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009870 tx=0x7f22d401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5f6894fa19d10f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:08.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22ebe00640 1 -- 172.21.15.42:0/2762002254 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d40b5070 con 0x7f22ec15cb50 2024-05-28T14:04:08.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22ebe00640 1 -- 172.21.15.42:0/2762002254 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d40a8a50 con 0x7f22ec15cb50 2024-05-28T14:04:08.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22ebe00640 1 -- 172.21.15.42:0/2762002254 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d40a8e20 con 0x7f22ec15cb50 2024-05-28T14:04:08.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 -- 172.21.15.42:0/2762002254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22ec15cb50 msgr2=0x7f22ec09b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 --2- 172.21.15.42:0/2762002254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22ec15cb50 0x7f22ec09b460 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009870 tx=0x7f22d401ed50 comp rx=0 tx=0).stop 2024-05-28T14:04:08.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 -- 172.21.15.42:0/2762002254 shutdown_connections 2024-05-28T14:04:08.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 --2- 172.21.15.42:0/2762002254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22ec0d11d0 0x7f22ec166470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 --2- 172.21.15.42:0/2762002254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22ec15cb50 0x7f22ec09b460 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 -- 172.21.15.42:0/2762002254 >> 172.21.15.42:0/2762002254 conn(0x7f22ec0b3fd0 msgr2=0x7f22ec0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:08.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 -- 172.21.15.42:0/2762002254 shutdown_connections 2024-05-28T14:04:08.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 -- 172.21.15.42:0/2762002254 wait complete. 2024-05-28T14:04:08.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 Processor -- start 2024-05-28T14:04:08.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 -- start start 2024-05-28T14:04:08.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22ec0d11d0 0x7f22ec15e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22ec15c770 0x7f22ec1ee890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22ec15cb50 0x7f22ec1f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec0c8bc0 con 0x7f22ec0d11d0 2024-05-28T14:04:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec0ce2c0 con 0x7f22ec15cb50 2024-05-28T14:04:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f3600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec0c8d40 con 0x7f22ec15c770 2024-05-28T14:04:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22ec0d11d0 0x7f22ec15e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22ec0d11d0 0x7f22ec15e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35146/0 (socket says 172.21.15.42:35146) 2024-05-28T14:04:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f0c00640 1 -- 172.21.15.42:0/3801608785 learned_addr learned my addr 172.21.15.42:0/3801608785 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22e3e00640 1 --2- 172.21.15.42:0/3801608785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22ec15c770 0x7f22ec1ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.147+0000 7f22f1600640 1 --2- 172.21.15.42:0/3801608785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22ec15cb50 0x7f22ec1f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22f0c00640 1 -- 172.21.15.42:0/3801608785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22ec15c770 msgr2=0x7f22ec1ee890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22f0c00640 1 --2- 172.21.15.42:0/3801608785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22ec15c770 0x7f22ec1ee890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22f0c00640 1 -- 172.21.15.42:0/3801608785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22ec15cb50 msgr2=0x7f22ec1f5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22f0c00640 1 --2- 172.21.15.42:0/3801608785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22ec15cb50 0x7f22ec1f5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22f0c00640 1 -- 172.21.15.42:0/3801608785 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec1f57c0 con 0x7f22ec0d11d0 2024-05-28T14:04:08.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22f0c00640 1 --2- 172.21.15.42:0/3801608785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22ec0d11d0 0x7f22ec15e070 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c420 tx=0x7f22dc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:08.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22e9600640 1 -- 172.21.15.42:0/3801608785 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0052d0 con 0x7f22ec0d11d0 2024-05-28T14:04:08.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22e9600640 1 -- 172.21.15.42:0/3801608785 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22dc005470 con 0x7f22ec0d11d0 2024-05-28T14:04:08.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22e9600640 1 -- 172.21.15.42:0/3801608785 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc007260 con 0x7f22ec0d11d0 2024-05-28T14:04:08.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22f3600640 1 -- 172.21.15.42:0/3801608785 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ec0caba0 con 0x7f22ec0d11d0 2024-05-28T14:04:08.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22f3600640 1 -- 172.21.15.42:0/3801608785 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ec0cb160 con 0x7f22ec0d11d0 2024-05-28T14:04:08.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22f3600640 1 -- 172.21.15.42:0/3801608785 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ec0a04d0 con 0x7f22ec0d11d0 2024-05-28T14:04:08.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22e9600640 1 -- 172.21.15.42:0/3801608785 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f22dc007400 con 0x7f22ec0d11d0 2024-05-28T14:04:08.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22e9600640 1 --2- 172.21.15.42:0/3801608785 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22c8068b60 0x7f22c806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.151+0000 7f22e9600640 1 -- 172.21.15.42:0/3801608785 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5851+0+0 (secure 0 0 0) 0x7f22dc0871a0 con 0x7f22ec0d11d0 2024-05-28T14:04:08.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.155+0000 7f22e3e00640 1 --2- 172.21.15.42:0/3801608785 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22c8068b60 0x7f22c806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.155+0000 7f22e3e00640 1 --2- 172.21.15.42:0/3801608785 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22c8068b60 0x7f22c806b020 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f22d40b2000 tx=0x7f22d4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:08.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.159+0000 7f22e9600640 1 -- 172.21.15.42:0/3801608785 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f22ec1f57c0 con 0x7f22ec0d11d0 2024-05-28T14:04:08.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.323+0000 7f22f3600640 1 -- 172.21.15.42:0/3801608785 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f22ec09abc0 con 0x7f22ec0d11d0 2024-05-28T14:04:08.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.323+0000 7f22e9600640 1 -- 172.21.15.42:0/3801608785 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) ==== 104+0+11 (secure 0 0 0) 0x7f22dc056060 con 0x7f22ec0d11d0 2024-05-28T14:04:08.308 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:04:08.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.323+0000 7f22f3600640 1 -- 172.21.15.42:0/3801608785 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22c8068b60 msgr2=0x7f22c806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.323+0000 7f22f3600640 1 --2- 172.21.15.42:0/3801608785 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22c8068b60 0x7f22c806b020 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f22d40b2000 tx=0x7f22d4009040 comp rx=0 tx=0).stop 2024-05-28T14:04:08.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.323+0000 7f22f3600640 1 -- 172.21.15.42:0/3801608785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22ec0d11d0 msgr2=0x7f22ec15e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.323+0000 7f22f3600640 1 --2- 172.21.15.42:0/3801608785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22ec0d11d0 0x7f22ec15e070 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c420 tx=0x7f22dc00c8e0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.323+0000 7f22f3600640 1 -- 172.21.15.42:0/3801608785 shutdown_connections 2024-05-28T14:04:08.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.323+0000 7f22f3600640 1 --2- 172.21.15.42:0/3801608785 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22c8068b60 0x7f22c806b020 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.323+0000 7f22f3600640 1 --2- 172.21.15.42:0/3801608785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22ec15cb50 0x7f22ec1f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.323+0000 7f22f3600640 1 --2- 172.21.15.42:0/3801608785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22ec15c770 0x7f22ec1ee890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.323+0000 7f22f3600640 1 --2- 172.21.15.42:0/3801608785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22ec0d11d0 0x7f22ec15e070 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.323+0000 7f22f3600640 1 -- 172.21.15.42:0/3801608785 >> 172.21.15.42:0/3801608785 conn(0x7f22ec0b3fd0 msgr2=0x7f22ec06fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:08.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.323+0000 7f22f3600640 1 -- 172.21.15.42:0/3801608785 shutdown_connections 2024-05-28T14:04:08.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.323+0000 7f22f3600640 1 -- 172.21.15.42:0/3801608785 wait complete. 2024-05-28T14:04:08.330 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:04:08.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d40e00640 1 Processor -- start 2024-05-28T14:04:08.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d40e00640 1 -- start start 2024-05-28T14:04:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d40e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0d3c06ee20 0x7f0d3c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d40e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0d3c15b9b0 0x7f0d3c15be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d40e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d3c0afe40 con 0x7f0d3c144f80 2024-05-28T14:04:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d40e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0d3c06f7c0 con 0x7f0d3c06ee20 2024-05-28T14:04:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d40e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0d3c0724d0 con 0x7f0d3c15b9b0 2024-05-28T14:04:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d31400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0d3c06ee20 0x7f0d3c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d31400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0d3c06ee20 0x7f0d3c06f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35162/0 (socket says 172.21.15.42:35162) 2024-05-28T14:04:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d31400640 1 -- 172.21.15.42:0/1113094262 learned_addr learned my addr 172.21.15.42:0/1113094262 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d3a800640 1 --2- 172.21.15.42:0/1113094262 >> v2:172.21.15.63:3300/0 conn(0x7f0d3c15b9b0 0x7f0d3c15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d31400640 1 -- 172.21.15.42:0/1113094262 >> v2:172.21.15.63:3300/0 conn(0x7f0d3c15b9b0 msgr2=0x7f0d3c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d31400640 1 --2- 172.21.15.42:0/1113094262 >> v2:172.21.15.63:3300/0 conn(0x7f0d3c15b9b0 0x7f0d3c15be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d31400640 1 -- 172.21.15.42:0/1113094262 >> v1:172.21.15.63:6789/0 conn(0x7f0d3c144f80 legacy=0x7f0d3c072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.487+0000 7f0d31400640 1 -- 172.21.15.42:0/1113094262 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c15c340 con 0x7f0d3c06ee20 2024-05-28T14:04:08.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.491+0000 7f0d31400640 1 --2- 172.21.15.42:0/1113094262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d3c06ee20 0x7f0d3c06f280 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c160060 tx=0x7f0d2401e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693526491bf62043 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.491+0000 7f0d39400640 1 -- 172.21.15.42:0/1113094262 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d240ab400 con 0x7f0d3c06ee20 2024-05-28T14:04:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.491+0000 7f0d39400640 1 -- 172.21.15.42:0/1113094262 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d240ab5a0 con 0x7f0d3c06ee20 2024-05-28T14:04:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.491+0000 7f0d39400640 1 -- 172.21.15.42:0/1113094262 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d240ab970 con 0x7f0d3c06ee20 2024-05-28T14:04:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.491+0000 7f0d40e00640 1 -- 172.21.15.42:0/1113094262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d3c06ee20 msgr2=0x7f0d3c06f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.491+0000 7f0d40e00640 1 --2- 172.21.15.42:0/1113094262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d3c06ee20 0x7f0d3c06f280 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c160060 tx=0x7f0d2401e730 comp rx=0 tx=0).stop 2024-05-28T14:04:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.491+0000 7f0d40e00640 1 -- 172.21.15.42:0/1113094262 shutdown_connections 2024-05-28T14:04:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.491+0000 7f0d40e00640 1 --2- 172.21.15.42:0/1113094262 >> v2:172.21.15.63:3300/0 conn(0x7f0d3c15b9b0 0x7f0d3c15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.491+0000 7f0d40e00640 1 --2- 172.21.15.42:0/1113094262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d3c06ee20 0x7f0d3c06f280 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.491+0000 7f0d40e00640 1 -- 172.21.15.42:0/1113094262 >> 172.21.15.42:0/1113094262 conn(0x7f0d3c0b21f0 msgr2=0x7f0d3c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:08.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.491+0000 7f0d40e00640 1 -- 172.21.15.42:0/1113094262 shutdown_connections 2024-05-28T14:04:08.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.491+0000 7f0d40e00640 1 -- 172.21.15.42:0/1113094262 wait complete. 2024-05-28T14:04:08.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d40e00640 1 Processor -- start 2024-05-28T14:04:08.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d40e00640 1 -- start start 2024-05-28T14:04:08.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d40e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d3c06ee20 0x7f0d3c15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d40e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d3c144f80 0x7f0d3c15dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d40e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d3c15b9b0 0x7f0d3c0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d40e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c0b4010 con 0x7f0d3c15b9b0 2024-05-28T14:04:08.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d40e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c072160 con 0x7f0d3c06ee20 2024-05-28T14:04:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d40e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3c0b4190 con 0x7f0d3c144f80 2024-05-28T14:04:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d3a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d3c15b9b0 0x7f0d3c0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d3a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d3c15b9b0 0x7f0d3c0cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35166/0 (socket says 172.21.15.42:35166) 2024-05-28T14:04:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d3a800640 1 -- 172.21.15.42:0/1918729868 learned_addr learned my addr 172.21.15.42:0/1918729868 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d31400640 1 --2- 172.21.15.42:0/1918729868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d3c144f80 0x7f0d3c15dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d39e00640 1 --2- 172.21.15.42:0/1918729868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d3c06ee20 0x7f0d3c15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d3a800640 1 -- 172.21.15.42:0/1918729868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d3c144f80 msgr2=0x7f0d3c15dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d3a800640 1 --2- 172.21.15.42:0/1918729868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d3c144f80 0x7f0d3c15dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d3a800640 1 -- 172.21.15.42:0/1918729868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d3c06ee20 msgr2=0x7f0d3c15d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d3a800640 1 --2- 172.21.15.42:0/1918729868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d3c06ee20 0x7f0d3c15d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d3a800640 1 -- 172.21.15.42:0/1918729868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c15e290 con 0x7f0d3c15b9b0 2024-05-28T14:04:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d3a800640 1 --2- 172.21.15.42:0/1918729868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d3c15b9b0 0x7f0d3c0cf270 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400efc0 tx=0x7f0d3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:08.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d32a00640 1 -- 172.21.15.42:0/1918729868 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34018020 con 0x7f0d3c15b9b0 2024-05-28T14:04:08.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d32a00640 1 -- 172.21.15.42:0/1918729868 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d34014480 con 0x7f0d3c15b9b0 2024-05-28T14:04:08.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d32a00640 1 -- 172.21.15.42:0/1918729868 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34014870 con 0x7f0d3c15b9b0 2024-05-28T14:04:08.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d40e00640 1 -- 172.21.15.42:0/1918729868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c15e570 con 0x7f0d3c15b9b0 2024-05-28T14:04:08.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.495+0000 7f0d40e00640 1 -- 172.21.15.42:0/1918729868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c0ad960 con 0x7f0d3c15b9b0 2024-05-28T14:04:08.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.499+0000 7f0d40e00640 1 -- 172.21.15.42:0/1918729868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3c15ebf0 con 0x7f0d3c15b9b0 2024-05-28T14:04:08.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.499+0000 7f0d32a00640 1 -- 172.21.15.42:0/1918729868 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f0d3401b020 con 0x7f0d3c15b9b0 2024-05-28T14:04:08.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.499+0000 7f0d32a00640 1 --2- 172.21.15.42:0/1918729868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d18068b60 0x7f0d1806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.499+0000 7f0d32a00640 1 -- 172.21.15.42:0/1918729868 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5851+0+0 (secure 0 0 0) 0x7f0d34088720 con 0x7f0d3c15b9b0 2024-05-28T14:04:08.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.499+0000 7f0d39e00640 1 --2- 172.21.15.42:0/1918729868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d18068b60 0x7f0d1806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.499+0000 7f0d39e00640 1 --2- 172.21.15.42:0/1918729868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d18068b60 0x7f0d1806b020 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0d0340 tx=0x7f0d2c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:08.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.503+0000 7f0d32a00640 1 -- 172.21.15.42:0/1918729868 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0d3c15ebf0 con 0x7f0d3c15b9b0 2024-05-28T14:04:08.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.667+0000 7f0d40e00640 1 -- 172.21.15.42:0/1918729868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0d3c0adb30 con 0x7f0d3c15b9b0 2024-05-28T14:04:08.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.667+0000 7f0d32a00640 1 -- 172.21.15.42:0/1918729868 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) ==== 100+0+11 (secure 0 0 0) 0x7f0d34022030 con 0x7f0d3c15b9b0 2024-05-28T14:04:08.653 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:04:08.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.667+0000 7f0d40e00640 1 -- 172.21.15.42:0/1918729868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d18068b60 msgr2=0x7f0d1806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.667+0000 7f0d40e00640 1 --2- 172.21.15.42:0/1918729868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d18068b60 0x7f0d1806b020 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0d0340 tx=0x7f0d2c009040 comp rx=0 tx=0).stop 2024-05-28T14:04:08.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.667+0000 7f0d40e00640 1 -- 172.21.15.42:0/1918729868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d3c15b9b0 msgr2=0x7f0d3c0cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.667+0000 7f0d40e00640 1 --2- 172.21.15.42:0/1918729868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d3c15b9b0 0x7f0d3c0cf270 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400efc0 tx=0x7f0d3400c490 comp rx=0 tx=0).stop 2024-05-28T14:04:08.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.667+0000 7f0d40e00640 1 -- 172.21.15.42:0/1918729868 shutdown_connections 2024-05-28T14:04:08.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.667+0000 7f0d40e00640 1 --2- 172.21.15.42:0/1918729868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d18068b60 0x7f0d1806b020 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.667+0000 7f0d40e00640 1 --2- 172.21.15.42:0/1918729868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d3c15b9b0 0x7f0d3c0cf270 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.667+0000 7f0d40e00640 1 --2- 172.21.15.42:0/1918729868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d3c144f80 0x7f0d3c15dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.667+0000 7f0d40e00640 1 --2- 172.21.15.42:0/1918729868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d3c06ee20 0x7f0d3c15d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.667+0000 7f0d40e00640 1 -- 172.21.15.42:0/1918729868 >> 172.21.15.42:0/1918729868 conn(0x7f0d3c0b21f0 msgr2=0x7f0d3c1409e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:08.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.667+0000 7f0d40e00640 1 -- 172.21.15.42:0/1918729868 shutdown_connections 2024-05-28T14:04:08.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.667+0000 7f0d40e00640 1 -- 172.21.15.42:0/1918729868 wait complete. 2024-05-28T14:04:08.675 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:04:08.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe883200640 1 Processor -- start 2024-05-28T14:04:08.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe883200640 1 -- start start 2024-05-28T14:04:08.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe883200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe87c15c990 0x7fe87c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe883200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe87c09bbd0 0x7fe87c09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe883200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe87c0d06a0 0x7fe87c09af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe883200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fe87c15cd70 con 0x7fe87c15c990 2024-05-28T14:04:08.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe883200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe87c0cd300 con 0x7fe87c09bbd0 2024-05-28T14:04:08.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe883200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe87c0cd5e0 con 0x7fe87c0d06a0 2024-05-28T14:04:08.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe87be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe87c15c990 0x7fe87c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe87be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe87c15c990 0x7fe87c0d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35172/0 (socket says 172.21.15.42:35172) 2024-05-28T14:04:08.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe87be00640 1 -- 172.21.15.42:0/2871111407 learned_addr learned my addr 172.21.15.42:0/2871111407 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:08.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe881200640 1 --2- 172.21.15.42:0/2871111407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe87c0d06a0 0x7fe87c09af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe873400640 1 --2- 172.21.15.42:0/2871111407 >> v2:172.21.15.63:3300/0 conn(0x7fe87c09bbd0 0x7fe87c09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe87be00640 1 -- 172.21.15.42:0/2871111407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe87c0d06a0 msgr2=0x7fe87c09af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:08.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe87be00640 1 --2- 172.21.15.42:0/2871111407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe87c0d06a0 0x7fe87c09af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe87be00640 1 -- 172.21.15.42:0/2871111407 >> v2:172.21.15.63:3300/0 conn(0x7fe87c09bbd0 msgr2=0x7fe87c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:08.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe87be00640 1 --2- 172.21.15.42:0/2871111407 >> v2:172.21.15.63:3300/0 conn(0x7fe87c09bbd0 0x7fe87c09aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.831+0000 7fe87be00640 1 -- 172.21.15.42:0/2871111407 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87c0cd8c0 con 0x7fe87c15c990 2024-05-28T14:04:08.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.835+0000 7fe87be00640 1 --2- 172.21.15.42:0/2871111407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe87c15c990 0x7fe87c0d0160 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe87c072560 tx=0x7fe86c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834f2547f4f6f10c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:08.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.835+0000 7fe87b400640 1 -- 172.21.15.42:0/2871111407 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0a5840 con 0x7fe87c15c990 2024-05-28T14:04:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.835+0000 7fe87b400640 1 -- 172.21.15.42:0/2871111407 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86c0a59e0 con 0x7fe87c15c990 2024-05-28T14:04:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.835+0000 7fe87b400640 1 -- 172.21.15.42:0/2871111407 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0a5db0 con 0x7fe87c15c990 2024-05-28T14:04:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.835+0000 7fe883200640 1 -- 172.21.15.42:0/2871111407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe87c15c990 msgr2=0x7fe87c0d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.835+0000 7fe883200640 1 --2- 172.21.15.42:0/2871111407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe87c15c990 0x7fe87c0d0160 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe87c072560 tx=0x7fe86c01c9c0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.835+0000 7fe883200640 1 -- 172.21.15.42:0/2871111407 shutdown_connections 2024-05-28T14:04:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.835+0000 7fe883200640 1 --2- 172.21.15.42:0/2871111407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe87c0d06a0 0x7fe87c09af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.835+0000 7fe883200640 1 --2- 172.21.15.42:0/2871111407 >> v2:172.21.15.63:3300/0 conn(0x7fe87c09bbd0 0x7fe87c09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.835+0000 7fe883200640 1 --2- 172.21.15.42:0/2871111407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe87c15c990 0x7fe87c0d0160 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.835+0000 7fe883200640 1 -- 172.21.15.42:0/2871111407 >> 172.21.15.42:0/2871111407 conn(0x7fe87c0b4080 msgr2=0x7fe87c0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.835+0000 7fe883200640 1 -- 172.21.15.42:0/2871111407 shutdown_connections 2024-05-28T14:04:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.835+0000 7fe883200640 1 -- 172.21.15.42:0/2871111407 wait complete. 2024-05-28T14:04:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe883200640 1 Processor -- start 2024-05-28T14:04:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe883200640 1 -- start start 2024-05-28T14:04:08.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe883200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe87c0d06a0 0x7fe87c15b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe883200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe87c15c990 0x7fe87c14cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe883200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe87c14d410 0x7fe87c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe883200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c160d90 con 0x7fe87c0d06a0 2024-05-28T14:04:08.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe883200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c160c10 con 0x7fe87c15c990 2024-05-28T14:04:08.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe883200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe87c160f10 con 0x7fe87c14d410 2024-05-28T14:04:08.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe87be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe87c0d06a0 0x7fe87c15b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe87be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe87c0d06a0 0x7fe87c15b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35176/0 (socket says 172.21.15.42:35176) 2024-05-28T14:04:08.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe87be00640 1 -- 172.21.15.42:0/2551246990 learned_addr learned my addr 172.21.15.42:0/2551246990 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:08.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe881200640 1 --2- 172.21.15.42:0/2551246990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe87c14d410 0x7fe87c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe873400640 1 --2- 172.21.15.42:0/2551246990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe87c15c990 0x7fe87c14cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe87be00640 1 -- 172.21.15.42:0/2551246990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe87c14d410 msgr2=0x7fe87c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe87be00640 1 --2- 172.21.15.42:0/2551246990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe87c14d410 0x7fe87c156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe87be00640 1 -- 172.21.15.42:0/2551246990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe87c15c990 msgr2=0x7fe87c14cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:08.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe87be00640 1 --2- 172.21.15.42:0/2551246990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe87c15c990 0x7fe87c14cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:08.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe87be00640 1 -- 172.21.15.42:0/2551246990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87c0cfe20 con 0x7fe87c0d06a0 2024-05-28T14:04:08.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe87be00640 1 --2- 172.21.15.42:0/2551246990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe87c0d06a0 0x7fe87c15b3b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe86c01d260 tx=0x7fe86c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:08.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe878c00640 1 -- 172.21.15.42:0/2551246990 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0b8670 con 0x7fe87c0d06a0 2024-05-28T14:04:08.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe878c00640 1 -- 172.21.15.42:0/2551246990 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86c0be070 con 0x7fe87c0d06a0 2024-05-28T14:04:08.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe878c00640 1 -- 172.21.15.42:0/2551246990 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0b8a70 con 0x7fe87c0d06a0 2024-05-28T14:04:08.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe883200640 1 -- 172.21.15.42:0/2551246990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe87c0cc470 con 0x7fe87c0d06a0 2024-05-28T14:04:08.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.839+0000 7fe883200640 1 -- 172.21.15.42:0/2551246990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe87c0cc190 con 0x7fe87c0d06a0 2024-05-28T14:04:08.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.843+0000 7fe883200640 1 -- 172.21.15.42:0/2551246990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe87c068a90 con 0x7fe87c0d06a0 2024-05-28T14:04:08.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.843+0000 7fe878c00640 1 -- 172.21.15.42:0/2551246990 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fe86c0b8c10 con 0x7fe87c0d06a0 2024-05-28T14:04:08.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.843+0000 7fe878c00640 1 --2- 172.21.15.42:0/2551246990 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe858068b60 0x7fe85806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:08.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.843+0000 7fe878c00640 1 -- 172.21.15.42:0/2551246990 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5851+0+0 (secure 0 0 0) 0x7fe86c1228d0 con 0x7fe87c0d06a0 2024-05-28T14:04:08.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.843+0000 7fe873400640 1 --2- 172.21.15.42:0/2551246990 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe858068b60 0x7fe85806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:08.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.843+0000 7fe873400640 1 --2- 172.21.15.42:0/2551246990 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe858068b60 0x7fe85806b020 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe87c1faab0 tx=0x7fe864009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:08.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:08.847+0000 7fe878c00640 1 -- 172.21.15.42:0/2551246990 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe86c0165e0 con 0x7fe87c0d06a0 2024-05-28T14:04:09.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.047+0000 7fe883200640 1 -- 172.21.15.42:0/2551246990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe87c0b4ce0 con 0x7fe87c0d06a0 2024-05-28T14:04:09.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.047+0000 7fe878c00640 1 -- 172.21.15.42:0/2551246990 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5180 (secure 0 0 0) 0x7fe86c0f1ba0 con 0x7fe87c0d06a0 2024-05-28T14:04:09.035 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:09.035 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3694688875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3694688875},{"type":"v1","addr":"172.21.15.63:6835","nonce":3694688875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5799,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/199639725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":199639725},{"type":"v1","addr":"172.21.15.42:6836","nonce":199639725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1043559054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1043559054},{"type":"v1","addr":"172.21.15.42:6837","nonce":1043559054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:01:50.630732+0000","modified":"2024-05-28T14:01:53.667008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":55184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55184":{"gid":55184,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6833/754542999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":754542999},{"type":"v1","addr":"172.21.15.63:6833","nonce":754542999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":55184,"qdb_cluster":[55184]},"id":3}]} 2024-05-28T14:04:09.035 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 28 2024-05-28T14:04:09.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.051+0000 7fe883200640 1 -- 172.21.15.42:0/2551246990 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe858068b60 msgr2=0x7fe85806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.051+0000 7fe883200640 1 --2- 172.21.15.42:0/2551246990 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe858068b60 0x7fe85806b020 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe87c1faab0 tx=0x7fe864009190 comp rx=0 tx=0).stop 2024-05-28T14:04:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.051+0000 7fe883200640 1 -- 172.21.15.42:0/2551246990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe87c0d06a0 msgr2=0x7fe87c15b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.051+0000 7fe883200640 1 --2- 172.21.15.42:0/2551246990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe87c0d06a0 0x7fe87c15b3b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe86c01d260 tx=0x7fe86c019760 comp rx=0 tx=0).stop 2024-05-28T14:04:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.051+0000 7fe883200640 1 -- 172.21.15.42:0/2551246990 shutdown_connections 2024-05-28T14:04:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.051+0000 7fe883200640 1 --2- 172.21.15.42:0/2551246990 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe858068b60 0x7fe85806b020 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.051+0000 7fe883200640 1 --2- 172.21.15.42:0/2551246990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe87c14d410 0x7fe87c156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.051+0000 7fe883200640 1 --2- 172.21.15.42:0/2551246990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe87c15c990 0x7fe87c14cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.051+0000 7fe883200640 1 --2- 172.21.15.42:0/2551246990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe87c0d06a0 0x7fe87c15b3b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.051+0000 7fe883200640 1 -- 172.21.15.42:0/2551246990 >> 172.21.15.42:0/2551246990 conn(0x7fe87c0b4080 msgr2=0x7fe87c09aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.051+0000 7fe883200640 1 -- 172.21.15.42:0/2551246990 shutdown_connections 2024-05-28T14:04:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.051+0000 7fe883200640 1 -- 172.21.15.42:0/2551246990 wait complete. 2024-05-28T14:04:09.057 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:04:09.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.211+0000 7f4e81a00640 1 Processor -- start 2024-05-28T14:04:09.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.211+0000 7f4e81a00640 1 -- start start 2024-05-28T14:04:09.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.211+0000 7f4e81a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e7c156a70 0x7f4e7c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.211+0000 7f4e81a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4e7c156e50 0x7f4e7c0af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.211+0000 7f4e81a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4e7c0af880 0x7f4e7c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.211+0000 7f4e81a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f4e7c15cb90 con 0x7f4e7c156e50 2024-05-28T14:04:09.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.211+0000 7f4e81a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4e7c15ce50 con 0x7f4e7c0af880 2024-05-28T14:04:09.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.211+0000 7f4e81a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7c15d130 con 0x7f4e7c156a70 2024-05-28T14:04:09.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e73e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4e7c156e50 0x7f4e7c0af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e73e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4e7c156e50 0x7f4e7c0af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35180/0 (socket says 172.21.15.42:35180) 2024-05-28T14:04:09.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e73e00640 1 -- 172.21.15.42:0/3493293958 learned_addr learned my addr 172.21.15.42:0/3493293958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:09.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e7b400640 1 --2- 172.21.15.42:0/3493293958 >> v2:172.21.15.63:3300/0 conn(0x7f4e7c0af880 0x7f4e7c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e7aa00640 1 --2- 172.21.15.42:0/3493293958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e7c156a70 0x7f4e7c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e73e00640 1 -- 172.21.15.42:0/3493293958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e7c156a70 msgr2=0x7f4e7c06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:09.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e73e00640 1 --2- 172.21.15.42:0/3493293958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e7c156a70 0x7f4e7c06dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e73e00640 1 -- 172.21.15.42:0/3493293958 >> v2:172.21.15.63:3300/0 conn(0x7f4e7c0af880 msgr2=0x7f4e7c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:09.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e73e00640 1 --2- 172.21.15.42:0/3493293958 >> v2:172.21.15.63:3300/0 conn(0x7f4e7c0af880 0x7f4e7c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e73e00640 1 -- 172.21.15.42:0/3493293958 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c15d410 con 0x7f4e7c156e50 2024-05-28T14:04:09.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e7b400640 1 --2- 172.21.15.42:0/3493293958 >> v2:172.21.15.63:3300/0 conn(0x7f4e7c0af880 0x7f4e7c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:09.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e73e00640 1 --2- 172.21.15.42:0/3493293958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c156e50 0x7f4e7c0af340 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0d13d0 tx=0x7f4e6401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8088fa8d4006ff0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:09.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e7a000640 1 -- 172.21.15.42:0/3493293958 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e640ab5f0 con 0x7f4e7c156e50 2024-05-28T14:04:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e7a000640 1 -- 172.21.15.42:0/3493293958 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e640b0070 con 0x7f4e7c156e50 2024-05-28T14:04:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.215+0000 7f4e7a000640 1 -- 172.21.15.42:0/3493293958 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e640ab9b0 con 0x7f4e7c156e50 2024-05-28T14:04:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 -- 172.21.15.42:0/3493293958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c156e50 msgr2=0x7f4e7c0af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 --2- 172.21.15.42:0/3493293958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c156e50 0x7f4e7c0af340 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0d13d0 tx=0x7f4e6401c520 comp rx=0 tx=0).stop 2024-05-28T14:04:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 -- 172.21.15.42:0/3493293958 shutdown_connections 2024-05-28T14:04:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 --2- 172.21.15.42:0/3493293958 >> v2:172.21.15.63:3300/0 conn(0x7f4e7c0af880 0x7f4e7c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 --2- 172.21.15.42:0/3493293958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c156e50 0x7f4e7c0af340 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 --2- 172.21.15.42:0/3493293958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e7c156a70 0x7f4e7c06dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 -- 172.21.15.42:0/3493293958 >> 172.21.15.42:0/3493293958 conn(0x7f4e7c06bdf0 msgr2=0x7f4e7c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 -- 172.21.15.42:0/3493293958 shutdown_connections 2024-05-28T14:04:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 -- 172.21.15.42:0/3493293958 wait complete. 2024-05-28T14:04:09.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 Processor -- start 2024-05-28T14:04:09.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 -- start start 2024-05-28T14:04:09.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e7c0af880 0x7f4e7c14b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c156a70 0x7f4e7c1543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e7c156e50 0x7f4e7c15a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c160eb0 con 0x7f4e7c156a70 2024-05-28T14:04:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c160d30 con 0x7f4e7c156e50 2024-05-28T14:04:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e81a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7c161030 con 0x7f4e7c0af880 2024-05-28T14:04:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e73e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c156a70 0x7f4e7c1543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e73e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c156a70 0x7f4e7c1543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35186/0 (socket says 172.21.15.42:35186) 2024-05-28T14:04:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e73e00640 1 -- 172.21.15.42:0/2363305008 learned_addr learned my addr 172.21.15.42:0/2363305008 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e7b400640 1 --2- 172.21.15.42:0/2363305008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e7c156e50 0x7f4e7c15a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e7aa00640 1 --2- 172.21.15.42:0/2363305008 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e7c0af880 0x7f4e7c14b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e73e00640 1 -- 172.21.15.42:0/2363305008 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e7c0af880 msgr2=0x7f4e7c14b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e73e00640 1 --2- 172.21.15.42:0/2363305008 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e7c0af880 0x7f4e7c14b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e73e00640 1 -- 172.21.15.42:0/2363305008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e7c156e50 msgr2=0x7f4e7c15a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e73e00640 1 --2- 172.21.15.42:0/2363305008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e7c156e50 0x7f4e7c15a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.219+0000 7f4e73e00640 1 -- 172.21.15.42:0/2363305008 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c154cf0 con 0x7f4e7c156a70 2024-05-28T14:04:09.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.223+0000 7f4e73e00640 1 --2- 172.21.15.42:0/2363305008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c156a70 0x7f4e7c1543a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4e6401abd0 tx=0x7f4e6401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:09.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.223+0000 7f4e72a00640 1 -- 172.21.15.42:0/2363305008 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e640b4800 con 0x7f4e7c156a70 2024-05-28T14:04:09.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.223+0000 7f4e72a00640 1 -- 172.21.15.42:0/2363305008 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e640b49a0 con 0x7f4e7c156a70 2024-05-28T14:04:09.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.223+0000 7f4e81a00640 1 -- 172.21.15.42:0/2363305008 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c154fd0 con 0x7f4e7c156a70 2024-05-28T14:04:09.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.223+0000 7f4e72a00640 1 -- 172.21.15.42:0/2363305008 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e640b4d90 con 0x7f4e7c156a70 2024-05-28T14:04:09.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.223+0000 7f4e81a00640 1 -- 172.21.15.42:0/2363305008 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c155870 con 0x7f4e7c156a70 2024-05-28T14:04:09.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.223+0000 7f4e81a00640 1 -- 172.21.15.42:0/2363305008 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e7c068a90 con 0x7f4e7c156a70 2024-05-28T14:04:09.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.223+0000 7f4e72a00640 1 -- 172.21.15.42:0/2363305008 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f4e640b3380 con 0x7f4e7c156a70 2024-05-28T14:04:09.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.223+0000 7f4e72a00640 1 --2- 172.21.15.42:0/2363305008 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4e58068b60 0x7f4e5806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.223+0000 7f4e72a00640 1 -- 172.21.15.42:0/2363305008 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5851+0+0 (secure 0 0 0) 0x7f4e64121b40 con 0x7f4e7c156a70 2024-05-28T14:04:09.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.223+0000 7f4e7aa00640 1 --2- 172.21.15.42:0/2363305008 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4e58068b60 0x7f4e5806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.223+0000 7f4e7aa00640 1 --2- 172.21.15.42:0/2363305008 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4e58068b60 0x7f4e5806b020 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c073e30 tx=0x7f4e6c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:09.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.227+0000 7f4e72a00640 1 -- 172.21.15.42:0/2363305008 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4e640c4060 con 0x7f4e7c156a70 2024-05-28T14:04:09.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.395+0000 7f4e81a00640 1 -- 172.21.15.42:0/2363305008 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e7c15c1a0 con 0x7f4e7c156a70 2024-05-28T14:04:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.395+0000 7f4e72a00640 1 -- 172.21.15.42:0/2363305008 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16857 (secure 0 0 0) 0x7f4e64016110 con 0x7f4e7c156a70 2024-05-28T14:04:09.381 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:09.381 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":49,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:04:07.083179+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:01:49.041938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-28T14:01:49.591543+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.1b","mappings":[{"from":0,"to":4}]},{"pgid":"7.30","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:04:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.395+0000 7f4e81a00640 1 -- 172.21.15.42:0/2363305008 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4e58068b60 msgr2=0x7f4e5806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.395+0000 7f4e81a00640 1 --2- 172.21.15.42:0/2363305008 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4e58068b60 0x7f4e5806b020 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c073e30 tx=0x7f4e6c008040 comp rx=0 tx=0).stop 2024-05-28T14:04:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.395+0000 7f4e81a00640 1 -- 172.21.15.42:0/2363305008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c156a70 msgr2=0x7f4e7c1543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.395+0000 7f4e81a00640 1 --2- 172.21.15.42:0/2363305008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c156a70 0x7f4e7c1543a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4e6401abd0 tx=0x7f4e6401ad60 comp rx=0 tx=0).stop 2024-05-28T14:04:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.395+0000 7f4e81a00640 1 -- 172.21.15.42:0/2363305008 shutdown_connections 2024-05-28T14:04:09.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.395+0000 7f4e81a00640 1 --2- 172.21.15.42:0/2363305008 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4e58068b60 0x7f4e5806b020 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.395+0000 7f4e81a00640 1 --2- 172.21.15.42:0/2363305008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e7c156e50 0x7f4e7c15a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.395+0000 7f4e81a00640 1 --2- 172.21.15.42:0/2363305008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c156a70 0x7f4e7c1543a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.395+0000 7f4e81a00640 1 --2- 172.21.15.42:0/2363305008 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e7c0af880 0x7f4e7c14b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.395+0000 7f4e81a00640 1 -- 172.21.15.42:0/2363305008 >> 172.21.15.42:0/2363305008 conn(0x7f4e7c06bdf0 msgr2=0x7f4e7c06f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:09.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.395+0000 7f4e81a00640 1 -- 172.21.15.42:0/2363305008 shutdown_connections 2024-05-28T14:04:09.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.395+0000 7f4e81a00640 1 -- 172.21.15.42:0/2363305008 wait complete. 2024-05-28T14:04:09.402 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-28T14:04:09.403 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:04:09.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.555+0000 7efe36800640 1 Processor -- start 2024-05-28T14:04:09.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.559+0000 7efe36800640 1 -- start start 2024-05-28T14:04:09.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.559+0000 7efe36800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efe300ad680 0x7efe3015dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.559+0000 7efe36800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efe3006e1a0 0x7efe300732d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.559+0000 7efe36800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe30073bb0 con 0x7efe300d06f0 2024-05-28T14:04:09.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.559+0000 7efe36800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7efe300b31e0 con 0x7efe3006e1a0 2024-05-28T14:04:09.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.559+0000 7efe36800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efe3015e4e0 con 0x7efe300ad680 2024-05-28T14:04:09.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.559+0000 7efe2ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efe3006e1a0 0x7efe300732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.559+0000 7efe2fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7efe300d06f0 0x7efe3006e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47560/0 (socket says 172.21.15.42:47560) 2024-05-28T14:04:09.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.559+0000 7efe2fe00640 1 -- 172.21.15.42:0/3173103506 learned_addr learned my addr 172.21.15.42:0/3173103506 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:09.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.559+0000 7efe2f400640 1 --2- 172.21.15.42:0/3173103506 >> v2:172.21.15.63:3300/0 conn(0x7efe300ad680 0x7efe3015dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.559+0000 7efe2ea00640 1 -- 172.21.15.42:0/3173103506 >> v2:172.21.15.63:3300/0 conn(0x7efe300ad680 msgr2=0x7efe3015dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:09.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.559+0000 7efe2ea00640 1 --2- 172.21.15.42:0/3173103506 >> v2:172.21.15.63:3300/0 conn(0x7efe300ad680 0x7efe3015dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.559+0000 7efe2ea00640 1 -- 172.21.15.42:0/3173103506 >> v1:172.21.15.42:6789/0 conn(0x7efe300d06f0 legacy=0x7efe3006e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.559+0000 7efe2ea00640 1 -- 172.21.15.42:0/3173103506 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe30073e90 con 0x7efe3006e1a0 2024-05-28T14:04:09.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.563+0000 7efe2ea00640 1 --2- 172.21.15.42:0/3173103506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3006e1a0 0x7efe300732d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7efe300d1330 tx=0x7efe1c01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418639cc206162c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:09.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.563+0000 7efe2e000640 1 -- 172.21.15.42:0/3173103506 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c0b0070 con 0x7efe3006e1a0 2024-05-28T14:04:09.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.563+0000 7efe2e000640 1 -- 172.21.15.42:0/3173103506 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe1c0ab600 con 0x7efe3006e1a0 2024-05-28T14:04:09.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.563+0000 7efe2e000640 1 -- 172.21.15.42:0/3173103506 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c0ab9d0 con 0x7efe3006e1a0 2024-05-28T14:04:09.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.563+0000 7efe36800640 1 -- 172.21.15.42:0/3173103506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3006e1a0 msgr2=0x7efe300732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.563+0000 7efe36800640 1 --2- 172.21.15.42:0/3173103506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3006e1a0 0x7efe300732d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7efe300d1330 tx=0x7efe1c01c330 comp rx=0 tx=0).stop 2024-05-28T14:04:09.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.563+0000 7efe36800640 1 -- 172.21.15.42:0/3173103506 shutdown_connections 2024-05-28T14:04:09.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.563+0000 7efe36800640 1 --2- 172.21.15.42:0/3173103506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3006e1a0 0x7efe300732d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.563+0000 7efe36800640 1 --2- 172.21.15.42:0/3173103506 >> v2:172.21.15.63:3300/0 conn(0x7efe300ad680 0x7efe3015dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.563+0000 7efe36800640 1 -- 172.21.15.42:0/3173103506 >> 172.21.15.42:0/3173103506 conn(0x7efe3006bdf0 msgr2=0x7efe3006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:09.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.563+0000 7efe36800640 1 -- 172.21.15.42:0/3173103506 shutdown_connections 2024-05-28T14:04:09.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.563+0000 7efe36800640 1 -- 172.21.15.42:0/3173103506 wait complete. 2024-05-28T14:04:09.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.563+0000 7efe36800640 1 Processor -- start 2024-05-28T14:04:09.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.563+0000 7efe36800640 1 -- start start 2024-05-28T14:04:09.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe36800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3006e1a0 0x7efe3014f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe36800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe300ad680 0x7efe30151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe36800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe300d06f0 0x7efe30151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe36800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efe30168430 con 0x7efe3006e1a0 2024-05-28T14:04:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe36800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe301682b0 con 0x7efe300d06f0 2024-05-28T14:04:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe36800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efe301685b0 con 0x7efe300ad680 2024-05-28T14:04:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe2f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3006e1a0 0x7efe3014f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe2f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3006e1a0 0x7efe3014f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35212/0 (socket says 172.21.15.42:35212) 2024-05-28T14:04:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe2f400640 1 -- 172.21.15.42:0/1594514311 learned_addr learned my addr 172.21.15.42:0/1594514311 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe2ea00640 1 --2- 172.21.15.42:0/1594514311 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe300ad680 0x7efe30151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe2fe00640 1 --2- 172.21.15.42:0/1594514311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe300d06f0 0x7efe30151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe2f400640 1 -- 172.21.15.42:0/1594514311 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe300ad680 msgr2=0x7efe30151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe2f400640 1 --2- 172.21.15.42:0/1594514311 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe300ad680 0x7efe30151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe2f400640 1 -- 172.21.15.42:0/1594514311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe300d06f0 msgr2=0x7efe30151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe2f400640 1 --2- 172.21.15.42:0/1594514311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe300d06f0 0x7efe30151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe2f400640 1 -- 172.21.15.42:0/1594514311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe30153180 con 0x7efe3006e1a0 2024-05-28T14:04:09.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe2f400640 1 --2- 172.21.15.42:0/1594514311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3006e1a0 0x7efe3014f800 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7efe2000e870 tx=0x7efe2000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:09.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe27400640 1 -- 172.21.15.42:0/1594514311 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20017070 con 0x7efe3006e1a0 2024-05-28T14:04:09.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe27400640 1 -- 172.21.15.42:0/1594514311 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2000c940 con 0x7efe3006e1a0 2024-05-28T14:04:09.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe27400640 1 -- 172.21.15.42:0/1594514311 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2000ccf0 con 0x7efe3006e1a0 2024-05-28T14:04:09.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe36800640 1 -- 172.21.15.42:0/1594514311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe301520e0 con 0x7efe3006e1a0 2024-05-28T14:04:09.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe36800640 1 -- 172.21.15.42:0/1594514311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe301526a0 con 0x7efe3006e1a0 2024-05-28T14:04:09.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.567+0000 7efe36800640 1 -- 172.21.15.42:0/1594514311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe30068a90 con 0x7efe3006e1a0 2024-05-28T14:04:09.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.571+0000 7efe27400640 1 -- 172.21.15.42:0/1594514311 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7efe2001a070 con 0x7efe3006e1a0 2024-05-28T14:04:09.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.571+0000 7efe27400640 1 --2- 172.21.15.42:0/1594514311 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe10068b90 0x7efe1006b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.571+0000 7efe27400640 1 -- 172.21.15.42:0/1594514311 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5851+0+0 (secure 0 0 0) 0x7efe20087c40 con 0x7efe3006e1a0 2024-05-28T14:04:09.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.571+0000 7efe2ea00640 1 --2- 172.21.15.42:0/1594514311 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe10068b90 0x7efe1006b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.571+0000 7efe2ea00640 1 --2- 172.21.15.42:0/1594514311 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe10068b90 0x7efe1006b050 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0ad000 tx=0x7efe1c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.575+0000 7efe27400640 1 -- 172.21.15.42:0/1594514311 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efe200570c0 con 0x7efe3006e1a0 2024-05-28T14:04:09.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.775+0000 7efe36800640 1 -- 172.21.15.42:0/1594514311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe30072350 con 0x7efe3006e1a0 2024-05-28T14:04:09.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.775+0000 7efe27400640 1 -- 172.21.15.42:0/1594514311 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5180 (secure 0 0 0) 0x7efe200569c0 con 0x7efe3006e1a0 2024-05-28T14:04:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:09.762 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3694688875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3694688875},{"type":"v1","addr":"172.21.15.63:6835","nonce":3694688875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5799,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/199639725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":199639725},{"type":"v1","addr":"172.21.15.42:6836","nonce":199639725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1043559054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1043559054},{"type":"v1","addr":"172.21.15.42:6837","nonce":1043559054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:01:50.630732+0000","modified":"2024-05-28T14:01:53.667008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":55184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55184":{"gid":55184,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6833/754542999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":754542999},{"type":"v1","addr":"172.21.15.63:6833","nonce":754542999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":55184,"qdb_cluster":[55184]},"id":3}]} 2024-05-28T14:04:09.762 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 28 2024-05-28T14:04:09.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.779+0000 7efe36800640 1 -- 172.21.15.42:0/1594514311 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe10068b90 msgr2=0x7efe1006b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.779+0000 7efe36800640 1 --2- 172.21.15.42:0/1594514311 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe10068b90 0x7efe1006b050 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0ad000 tx=0x7efe1c008040 comp rx=0 tx=0).stop 2024-05-28T14:04:09.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.779+0000 7efe36800640 1 -- 172.21.15.42:0/1594514311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3006e1a0 msgr2=0x7efe3014f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.779+0000 7efe36800640 1 --2- 172.21.15.42:0/1594514311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3006e1a0 0x7efe3014f800 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7efe2000e870 tx=0x7efe2000ec70 comp rx=0 tx=0).stop 2024-05-28T14:04:09.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.779+0000 7efe36800640 1 -- 172.21.15.42:0/1594514311 shutdown_connections 2024-05-28T14:04:09.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.779+0000 7efe36800640 1 --2- 172.21.15.42:0/1594514311 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe10068b90 0x7efe1006b050 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.779+0000 7efe36800640 1 --2- 172.21.15.42:0/1594514311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe300d06f0 0x7efe30151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.779+0000 7efe36800640 1 --2- 172.21.15.42:0/1594514311 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe300ad680 0x7efe30151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.779+0000 7efe36800640 1 --2- 172.21.15.42:0/1594514311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3006e1a0 0x7efe3014f800 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.779+0000 7efe36800640 1 -- 172.21.15.42:0/1594514311 >> 172.21.15.42:0/1594514311 conn(0x7efe3006bdf0 msgr2=0x7efe3006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:09.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.779+0000 7efe36800640 1 -- 172.21.15.42:0/1594514311 shutdown_connections 2024-05-28T14:04:09.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.779+0000 7efe36800640 1 -- 172.21.15.42:0/1594514311 wait complete. 2024-05-28T14:04:09.784 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:04:09.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.939+0000 7f0dcf800640 1 Processor -- start 2024-05-28T14:04:09.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.939+0000 7f0dcf800640 1 -- start start 2024-05-28T14:04:09.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.939+0000 7f0dcf800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0dc815e970 0x7f0dc80d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.939+0000 7f0dcf800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0dc806ff60 0x7f0dc80ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.939+0000 7f0dcf800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc8070740 con 0x7f0dc80cd040 2024-05-28T14:04:09.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.939+0000 7f0dcf800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0dc80d0640 con 0x7f0dc815e970 2024-05-28T14:04:09.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.939+0000 7f0dcf800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc80d1160 con 0x7f0dc806ff60 2024-05-28T14:04:09.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.939+0000 7f0dc7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0dc806ff60 0x7f0dc80ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.939+0000 7f0dcce00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0dc815e970 0x7f0dc80d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.939+0000 7f0dc7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0dc806ff60 0x7f0dc80ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:40626/0 (socket says 172.21.15.42:40626) 2024-05-28T14:04:09.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.939+0000 7f0dc7e00640 1 -- 172.21.15.42:0/611838312 learned_addr learned my addr 172.21.15.42:0/611838312 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:09.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.943+0000 7f0dcce00640 1 -- 172.21.15.42:0/611838312 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0dc806ff60 msgr2=0x7f0dc80ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:09.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.943+0000 7f0dcce00640 1 --2- 172.21.15.42:0/611838312 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0dc806ff60 0x7f0dc80ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.943+0000 7f0dcce00640 1 -- 172.21.15.42:0/611838312 >> v1:172.21.15.63:6789/0 conn(0x7f0dc80cd040 legacy=0x7f0dc8162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.943+0000 7f0dcce00640 1 -- 172.21.15.42:0/611838312 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc80d0870 con 0x7f0dc815e970 2024-05-28T14:04:09.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.943+0000 7f0dcce00640 1 --2- 172.21.15.42:0/611838312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dc815e970 0x7f0dc80d15f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0097c0 tx=0x7f0dbc01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5c92eb27f10a559 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:09.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.943+0000 7f0dc7400640 1 -- 172.21.15.42:0/611838312 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0b0070 con 0x7f0dc815e970 2024-05-28T14:04:09.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.943+0000 7f0dc7400640 1 -- 172.21.15.42:0/611838312 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dbc0ab600 con 0x7f0dc815e970 2024-05-28T14:04:09.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.943+0000 7f0dc7400640 1 -- 172.21.15.42:0/611838312 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0ab9d0 con 0x7f0dc815e970 2024-05-28T14:04:09.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 -- 172.21.15.42:0/611838312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dc815e970 msgr2=0x7f0dc80d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 --2- 172.21.15.42:0/611838312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dc815e970 0x7f0dc80d15f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0097c0 tx=0x7f0dbc01c3e0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 -- 172.21.15.42:0/611838312 shutdown_connections 2024-05-28T14:04:09.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 --2- 172.21.15.42:0/611838312 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0dc806ff60 0x7f0dc80ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 --2- 172.21.15.42:0/611838312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dc815e970 0x7f0dc80d15f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 -- 172.21.15.42:0/611838312 >> 172.21.15.42:0/611838312 conn(0x7f0dc80b3fa0 msgr2=0x7f0dc80b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:09.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 -- 172.21.15.42:0/611838312 shutdown_connections 2024-05-28T14:04:09.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 -- 172.21.15.42:0/611838312 wait complete. 2024-05-28T14:04:09.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 Processor -- start 2024-05-28T14:04:09.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 -- start start 2024-05-28T14:04:09.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0dc806ff60 0x7f0dc80b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dc80cd040 0x7f0dc80b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dc815e970 0x7f0dc80c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc80704c0 con 0x7f0dc815e970 2024-05-28T14:04:09.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8070340 con 0x7f0dc80cd040 2024-05-28T14:04:09.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcf800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc816b840 con 0x7f0dc806ff60 2024-05-28T14:04:09.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dc815e970 0x7f0dc80c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dc815e970 0x7f0dc80c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35224/0 (socket says 172.21.15.42:35224) 2024-05-28T14:04:09.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcd800640 1 -- 172.21.15.42:0/149416259 learned_addr learned my addr 172.21.15.42:0/149416259 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:09.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcce00640 1 --2- 172.21.15.42:0/149416259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0dc806ff60 0x7f0dc80b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dc7e00640 1 --2- 172.21.15.42:0/149416259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dc80cd040 0x7f0dc80b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcd800640 1 -- 172.21.15.42:0/149416259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0dc806ff60 msgr2=0x7f0dc80b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcd800640 1 --2- 172.21.15.42:0/149416259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0dc806ff60 0x7f0dc80b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcd800640 1 -- 172.21.15.42:0/149416259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dc80cd040 msgr2=0x7f0dc80b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:09.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcd800640 1 --2- 172.21.15.42:0/149416259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dc80cd040 0x7f0dc80b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:09.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcd800640 1 -- 172.21.15.42:0/149416259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc80b1980 con 0x7f0dc815e970 2024-05-28T14:04:09.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.947+0000 7f0dcd800640 1 --2- 172.21.15.42:0/149416259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dc815e970 0x7f0dc80c5fa0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000c730 tx=0x7f0dc000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:09.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.951+0000 7f0dc4c00640 1 -- 172.21.15.42:0/149416259 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0003b80 con 0x7f0dc815e970 2024-05-28T14:04:09.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.951+0000 7f0dc4c00640 1 -- 172.21.15.42:0/149416259 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc0003d20 con 0x7f0dc815e970 2024-05-28T14:04:09.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.951+0000 7f0dcf800640 1 -- 172.21.15.42:0/149416259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc80c64e0 con 0x7f0dc815e970 2024-05-28T14:04:09.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.951+0000 7f0dc4c00640 1 -- 172.21.15.42:0/149416259 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc000aa30 con 0x7f0dc815e970 2024-05-28T14:04:09.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.951+0000 7f0dcf800640 1 -- 172.21.15.42:0/149416259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc80af650 con 0x7f0dc815e970 2024-05-28T14:04:09.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.951+0000 7f0dcf800640 1 -- 172.21.15.42:0/149416259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc8068a90 con 0x7f0dc815e970 2024-05-28T14:04:09.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.951+0000 7f0dc4c00640 1 -- 172.21.15.42:0/149416259 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f0dc000abd0 con 0x7f0dc815e970 2024-05-28T14:04:09.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.951+0000 7f0dc4c00640 1 --2- 172.21.15.42:0/149416259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0da8068b60 0x7f0da806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:09.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.951+0000 7f0dc4c00640 1 -- 172.21.15.42:0/149416259 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5851+0+0 (secure 0 0 0) 0x7f0dc0087a70 con 0x7f0dc815e970 2024-05-28T14:04:09.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.951+0000 7f0dcce00640 1 --2- 172.21.15.42:0/149416259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0da8068b60 0x7f0da806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:09.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.955+0000 7f0dcce00640 1 --2- 172.21.15.42:0/149416259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0da8068b60 0x7f0da806b020 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0ad000 tx=0x7f0dbc0a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:09.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:09.955+0000 7f0dc4c00640 1 -- 172.21.15.42:0/149416259 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0dc0056930 con 0x7f0dc815e970 2024-05-28T14:04:10.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.159+0000 7f0dcf800640 1 -- 172.21.15.42:0/149416259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dc80b4d20 con 0x7f0dc815e970 2024-05-28T14:04:10.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.159+0000 7f0dc4c00640 1 -- 172.21.15.42:0/149416259 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5180 (secure 0 0 0) 0x7f0dc005ac30 con 0x7f0dc815e970 2024-05-28T14:04:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:10.146 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3694688875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3694688875},{"type":"v1","addr":"172.21.15.63:6835","nonce":3694688875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5799,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/199639725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":199639725},{"type":"v1","addr":"172.21.15.42:6836","nonce":199639725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1043559054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1043559054},{"type":"v1","addr":"172.21.15.42:6837","nonce":1043559054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:01:50.630732+0000","modified":"2024-05-28T14:01:53.667008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":55184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55184":{"gid":55184,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6833/754542999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":754542999},{"type":"v1","addr":"172.21.15.63:6833","nonce":754542999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":55184,"qdb_cluster":[55184]},"id":3}]} 2024-05-28T14:04:10.146 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 28 2024-05-28T14:04:10.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.163+0000 7f0dcf800640 1 -- 172.21.15.42:0/149416259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0da8068b60 msgr2=0x7f0da806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:10.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.163+0000 7f0dcf800640 1 --2- 172.21.15.42:0/149416259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0da8068b60 0x7f0da806b020 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0ad000 tx=0x7f0dbc0a6000 comp rx=0 tx=0).stop 2024-05-28T14:04:10.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.163+0000 7f0dcf800640 1 -- 172.21.15.42:0/149416259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dc815e970 msgr2=0x7f0dc80c5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:10.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.163+0000 7f0dcf800640 1 --2- 172.21.15.42:0/149416259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dc815e970 0x7f0dc80c5fa0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000c730 tx=0x7f0dc000cbf0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.163+0000 7f0dcf800640 1 -- 172.21.15.42:0/149416259 shutdown_connections 2024-05-28T14:04:10.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.163+0000 7f0dcf800640 1 --2- 172.21.15.42:0/149416259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0da8068b60 0x7f0da806b020 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.163+0000 7f0dcf800640 1 --2- 172.21.15.42:0/149416259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dc815e970 0x7f0dc80c5fa0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.163+0000 7f0dcf800640 1 --2- 172.21.15.42:0/149416259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dc80cd040 0x7f0dc80b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.163+0000 7f0dcf800640 1 --2- 172.21.15.42:0/149416259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0dc806ff60 0x7f0dc80b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.163+0000 7f0dcf800640 1 -- 172.21.15.42:0/149416259 >> 172.21.15.42:0/149416259 conn(0x7f0dc80b3fa0 msgr2=0x7f0dc809aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:10.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.163+0000 7f0dcf800640 1 -- 172.21.15.42:0/149416259 shutdown_connections 2024-05-28T14:04:10.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.163+0000 7f0dcf800640 1 -- 172.21.15.42:0/149416259 wait complete. 2024-05-28T14:04:10.168 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:04:10.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89aac00640 1 Processor -- start 2024-05-28T14:04:10.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89aac00640 1 -- start start 2024-05-28T14:04:10.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89aac00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f89a406f660 0x7f89a4073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:10.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89aac00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f89a40d0e00 0x7f89a406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:10.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89aac00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89a415f280 con 0x7f89a415ba40 2024-05-28T14:04:10.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89aac00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f89a415f030 con 0x7f89a40d0e00 2024-05-28T14:04:10.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89aac00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f89a415f560 con 0x7f89a406f660 2024-05-28T14:04:10.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f899b400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f89a415ba40 0x7f89a40d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47562/0 (socket says 172.21.15.42:47562) 2024-05-28T14:04:10.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f899b400640 1 -- 172.21.15.42:0/4119263059 learned_addr learned my addr 172.21.15.42:0/4119263059 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89a8c00640 1 --2- 172.21.15.42:0/4119263059 >> v2:172.21.15.42:3300/0 conn(0x7f89a40d0e00 0x7f89a406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89a3e00640 1 --2- 172.21.15.42:0/4119263059 >> v2:172.21.15.63:3300/0 conn(0x7f89a406f660 0x7f89a4073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89a3400640 1 -- 172.21.15.42:0/4119263059 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3054878094 0 0) 0x7f89a415f280 con 0x7f89a415ba40 2024-05-28T14:04:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89a3400640 1 -- 172.21.15.42:0/4119263059 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8988003530 con 0x7f89a415ba40 2024-05-28T14:04:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89a8c00640 1 -- 172.21.15.42:0/4119263059 >> v2:172.21.15.63:3300/0 conn(0x7f89a406f660 msgr2=0x7f89a4073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89a8c00640 1 --2- 172.21.15.42:0/4119263059 >> v2:172.21.15.63:3300/0 conn(0x7f89a406f660 0x7f89a4073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89a8c00640 1 -- 172.21.15.42:0/4119263059 >> v1:172.21.15.42:6789/0 conn(0x7f89a415ba40 legacy=0x7f89a40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.323+0000 7f89a8c00640 1 -- 172.21.15.42:0/4119263059 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a415f840 con 0x7f89a40d0e00 2024-05-28T14:04:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.327+0000 7f89a8c00640 1 --2- 172.21.15.42:0/4119263059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a40d0e00 0x7f89a406dd60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f89a40d13a0 tx=0x7f899c01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b862fa8ff30e4d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.327+0000 7f89a3400640 1 -- 172.21.15.42:0/4119263059 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c0b1070 con 0x7f89a40d0e00 2024-05-28T14:04:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.327+0000 7f89a3400640 1 -- 172.21.15.42:0/4119263059 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f899c0ac640 con 0x7f89a40d0e00 2024-05-28T14:04:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.327+0000 7f89a3400640 1 -- 172.21.15.42:0/4119263059 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c0aca10 con 0x7f89a40d0e00 2024-05-28T14:04:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.327+0000 7f89aac00640 1 -- 172.21.15.42:0/4119263059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a40d0e00 msgr2=0x7f89a406dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.327+0000 7f89aac00640 1 --2- 172.21.15.42:0/4119263059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a40d0e00 0x7f89a406dd60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f89a40d13a0 tx=0x7f899c01c330 comp rx=0 tx=0).stop 2024-05-28T14:04:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.327+0000 7f89aac00640 1 -- 172.21.15.42:0/4119263059 shutdown_connections 2024-05-28T14:04:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.327+0000 7f89aac00640 1 --2- 172.21.15.42:0/4119263059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a40d0e00 0x7f89a406dd60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.327+0000 7f89aac00640 1 --2- 172.21.15.42:0/4119263059 >> v2:172.21.15.63:3300/0 conn(0x7f89a406f660 0x7f89a4073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.327+0000 7f89aac00640 1 -- 172.21.15.42:0/4119263059 >> 172.21.15.42:0/4119263059 conn(0x7f89a406bdf0 msgr2=0x7f89a406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.327+0000 7f89aac00640 1 -- 172.21.15.42:0/4119263059 shutdown_connections 2024-05-28T14:04:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.327+0000 7f89aac00640 1 -- 172.21.15.42:0/4119263059 wait complete. 2024-05-28T14:04:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89aac00640 1 Processor -- start 2024-05-28T14:04:10.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89aac00640 1 -- start start 2024-05-28T14:04:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89aac00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a406f660 0x7f89a4099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89aac00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a40d0e00 0x7f89a414eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89aac00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a4149fc0 0x7f89a414df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89aac00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f89a4166540 con 0x7f89a4149fc0 2024-05-28T14:04:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89aac00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f89a41663c0 con 0x7f89a40d0e00 2024-05-28T14:04:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89aac00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f89a41666c0 con 0x7f89a406f660 2024-05-28T14:04:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89a8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a4149fc0 0x7f89a414df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89a8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a4149fc0 0x7f89a414df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35246/0 (socket says 172.21.15.42:35246) 2024-05-28T14:04:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89a8c00640 1 -- 172.21.15.42:0/1792138038 learned_addr learned my addr 172.21.15.42:0/1792138038 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89a3e00640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a406f660 0x7f89a4099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f899b400640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a40d0e00 0x7f89a414eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:10.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89a8c00640 1 -- 172.21.15.42:0/1792138038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a406f660 msgr2=0x7f89a4099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89a8c00640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a406f660 0x7f89a4099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89a8c00640 1 -- 172.21.15.42:0/1792138038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a40d0e00 msgr2=0x7f89a414eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89a8c00640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a40d0e00 0x7f89a414eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89a8c00640 1 -- 172.21.15.42:0/1792138038 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a414f070 con 0x7f89a4149fc0 2024-05-28T14:04:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f899b400640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a40d0e00 0x7f89a414eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89a3e00640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a406f660 0x7f89a4099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89a8c00640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a4149fc0 0x7f89a414df50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f899c0af040 tx=0x7f899c0190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89a0c00640 1 -- 172.21.15.42:0/1792138038 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c0b1040 con 0x7f89a4149fc0 2024-05-28T14:04:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89a0c00640 1 -- 172.21.15.42:0/1792138038 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f899c0ba070 con 0x7f89a4149fc0 2024-05-28T14:04:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89a0c00640 1 -- 172.21.15.42:0/1792138038 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c0b5650 con 0x7f89a4149fc0 2024-05-28T14:04:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89aac00640 1 -- 172.21.15.42:0/1792138038 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a414a580 con 0x7f89a4149fc0 2024-05-28T14:04:10.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.331+0000 7f89aac00640 1 -- 172.21.15.42:0/1792138038 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a4153a00 con 0x7f89a4149fc0 2024-05-28T14:04:10.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.335+0000 7f89aac00640 1 -- 172.21.15.42:0/1792138038 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a4068a90 con 0x7f89a4149fc0 2024-05-28T14:04:10.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.335+0000 7f89a0c00640 1 -- 172.21.15.42:0/1792138038 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f899c0b57f0 con 0x7f89a4149fc0 2024-05-28T14:04:10.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.335+0000 7f89a0c00640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8980068b60 0x7f898006b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:10.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.335+0000 7f89a0c00640 1 -- 172.21.15.42:0/1792138038 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5851+0+0 (secure 0 0 0) 0x7f899c121720 con 0x7f89a4149fc0 2024-05-28T14:04:10.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.335+0000 7f89a3e00640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8980068b60 0x7f898006b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:10.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.335+0000 7f89a3e00640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8980068b60 0x7f898006b020 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f89940074c0 tx=0x7f899400b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:10.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.339+0000 7f89a0c00640 1 -- 172.21.15.42:0/1792138038 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f899c016070 con 0x7f89a4149fc0 2024-05-28T14:04:10.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.507+0000 7f89aac00640 1 -- 172.21.15.42:0/1792138038 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89a4072bb0 con 0x7f89a4149fc0 2024-05-28T14:04:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.507+0000 7f89a0c00640 1 -- 172.21.15.42:0/1792138038 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16857 (secure 0 0 0) 0x7f899c0f1100 con 0x7f89a4149fc0 2024-05-28T14:04:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:10.495 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":49,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:04:07.083179+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:01:49.041938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-28T14:01:49.591543+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.1b","mappings":[{"from":0,"to":4}]},{"pgid":"7.30","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:04:10.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.511+0000 7f89aac00640 1 -- 172.21.15.42:0/1792138038 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8980068b60 msgr2=0x7f898006b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:10.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.511+0000 7f89aac00640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8980068b60 0x7f898006b020 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f89940074c0 tx=0x7f899400b040 comp rx=0 tx=0).stop 2024-05-28T14:04:10.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.511+0000 7f89aac00640 1 -- 172.21.15.42:0/1792138038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a4149fc0 msgr2=0x7f89a414df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:10.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.511+0000 7f89aac00640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a4149fc0 0x7f89a414df50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f899c0af040 tx=0x7f899c0190d0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.511+0000 7f89aac00640 1 -- 172.21.15.42:0/1792138038 shutdown_connections 2024-05-28T14:04:10.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.511+0000 7f89aac00640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8980068b60 0x7f898006b020 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.511+0000 7f89aac00640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a4149fc0 0x7f89a414df50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.511+0000 7f89aac00640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a40d0e00 0x7f89a414eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.511+0000 7f89aac00640 1 --2- 172.21.15.42:0/1792138038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a406f660 0x7f89a4099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.511+0000 7f89aac00640 1 -- 172.21.15.42:0/1792138038 >> 172.21.15.42:0/1792138038 conn(0x7f89a406bdf0 msgr2=0x7f89a415a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:10.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.511+0000 7f89aac00640 1 -- 172.21.15.42:0/1792138038 shutdown_connections 2024-05-28T14:04:10.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.511+0000 7f89aac00640 1 -- 172.21.15.42:0/1792138038 wait complete. 2024-05-28T14:04:10.516 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-28T14:04:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.671+0000 7f8bd2600640 1 Processor -- start 2024-05-28T14:04:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.671+0000 7f8bd2600640 1 -- start start 2024-05-28T14:04:10.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.671+0000 7f8bd2600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8bcc0ae240 0x7f8bcc0ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:10.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.671+0000 7f8bd2600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bcc0b1640 0x7f8bcc0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:10.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.671+0000 7f8bd2600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bcc0b6590 con 0x7f8bcc15e820 2024-05-28T14:04:10.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.671+0000 7f8bd2600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8bcc06c870 con 0x7f8bcc0ae240 2024-05-28T14:04:10.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.671+0000 7f8bd2600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc0b5d30 con 0x7f8bcc0b1640 2024-05-28T14:04:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.671+0000 7f8bcb400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8bcc15e820 0x7f8bcc0b4260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47570/0 (socket says 172.21.15.42:47570) 2024-05-28T14:04:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.671+0000 7f8bcb400640 1 -- 172.21.15.42:0/2250776186 learned_addr learned my addr 172.21.15.42:0/2250776186 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.675+0000 7f8bc3e00640 1 --2- 172.21.15.42:0/2250776186 >> v2:172.21.15.63:3300/0 conn(0x7f8bcc0ae240 0x7f8bcc0ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.675+0000 7f8bcbe00640 1 --2- 172.21.15.42:0/2250776186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bcc0b1640 0x7f8bcc0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.675+0000 7f8bcaa00640 1 -- 172.21.15.42:0/2250776186 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1107181264 0 0) 0x7f8bcc0b6590 con 0x7f8bcc15e820 2024-05-28T14:04:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.675+0000 7f8bcaa00640 1 -- 172.21.15.42:0/2250776186 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bb0003530 con 0x7f8bcc15e820 2024-05-28T14:04:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.675+0000 7f8bcaa00640 1 -- 172.21.15.42:0/2250776186 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 899886218 0 0) 0x7f8bb0003530 con 0x7f8bcc15e820 2024-05-28T14:04:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.675+0000 7f8bcbe00640 1 -- 172.21.15.42:0/2250776186 >> v2:172.21.15.63:3300/0 conn(0x7f8bcc0ae240 msgr2=0x7f8bcc0ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.675+0000 7f8bcbe00640 1 --2- 172.21.15.42:0/2250776186 >> v2:172.21.15.63:3300/0 conn(0x7f8bcc0ae240 0x7f8bcc0ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.675+0000 7f8bcbe00640 1 -- 172.21.15.42:0/2250776186 >> v1:172.21.15.42:6789/0 conn(0x7f8bcc15e820 legacy=0x7f8bcc0b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.675+0000 7f8bcbe00640 1 -- 172.21.15.42:0/2250776186 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc15ec00 con 0x7f8bcc0b1640 2024-05-28T14:04:10.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.675+0000 7f8bc3e00640 1 --2- 172.21.15.42:0/2250776186 >> v2:172.21.15.63:3300/0 conn(0x7f8bcc0ae240 0x7f8bcc0ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:04:10.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.675+0000 7f8bcbe00640 1 --2- 172.21.15.42:0/2250776186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bcc0b1640 0x7f8bcc0b1ad0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009d90 tx=0x7f8bc401c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae52341e74743e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:10.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.675+0000 7f8bcaa00640 1 -- 172.21.15.42:0/2250776186 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40ac070 con 0x7f8bcc0b1640 2024-05-28T14:04:10.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.675+0000 7f8bcaa00640 1 -- 172.21.15.42:0/2250776186 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc40a64a0 con 0x7f8bcc0b1640 2024-05-28T14:04:10.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.675+0000 7f8bcaa00640 1 -- 172.21.15.42:0/2250776186 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc001680 con 0x7f8bcc0b1640 2024-05-28T14:04:10.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 -- 172.21.15.42:0/2250776186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bcc0b1640 msgr2=0x7f8bcc0b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:10.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 --2- 172.21.15.42:0/2250776186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bcc0b1640 0x7f8bcc0b1ad0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009d90 tx=0x7f8bc401c890 comp rx=0 tx=0).stop 2024-05-28T14:04:10.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 -- 172.21.15.42:0/2250776186 shutdown_connections 2024-05-28T14:04:10.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 --2- 172.21.15.42:0/2250776186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bcc0b1640 0x7f8bcc0b1ad0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 --2- 172.21.15.42:0/2250776186 >> v2:172.21.15.63:3300/0 conn(0x7f8bcc0ae240 0x7f8bcc0ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 -- 172.21.15.42:0/2250776186 >> 172.21.15.42:0/2250776186 conn(0x7f8bcc0b6780 msgr2=0x7f8bcc0b6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:10.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 -- 172.21.15.42:0/2250776186 shutdown_connections 2024-05-28T14:04:10.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 -- 172.21.15.42:0/2250776186 wait complete. 2024-05-28T14:04:10.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 Processor -- start 2024-05-28T14:04:10.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 -- start start 2024-05-28T14:04:10.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bcc0ae240 0x7f8bcc13f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:10.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bcc0b1640 0x7f8bcc15ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bcc15e820 0x7f8bcc15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0b2850 con 0x7f8bcc0ae240 2024-05-28T14:04:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0b26d0 con 0x7f8bcc15e820 2024-05-28T14:04:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bd2600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc16b510 con 0x7f8bcc0b1640 2024-05-28T14:04:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bcb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bcc0ae240 0x7f8bcc13f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bcb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bcc0ae240 0x7f8bcc13f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35262/0 (socket says 172.21.15.42:35262) 2024-05-28T14:04:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bcb400640 1 -- 172.21.15.42:0/309611680 learned_addr learned my addr 172.21.15.42:0/309611680 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bc3e00640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bcc0b1640 0x7f8bcc15ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bcbe00640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bcc15e820 0x7f8bcc15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bcb400640 1 -- 172.21.15.42:0/309611680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bcc0b1640 msgr2=0x7f8bcc15ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bcb400640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bcc0b1640 0x7f8bcc15ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bcb400640 1 -- 172.21.15.42:0/309611680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bcc15e820 msgr2=0x7f8bcc15b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:10.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bcb400640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bcc15e820 0x7f8bcc15b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bcb400640 1 -- 172.21.15.42:0/309611680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc1540f0 con 0x7f8bcc0ae240 2024-05-28T14:04:10.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bc3e00640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bcc0b1640 0x7f8bcc15ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:10.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.679+0000 7f8bcbe00640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bcc15e820 0x7f8bcc15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:10.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.683+0000 7f8bcb400640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bcc0ae240 0x7f8bcc13f6c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00c750 tx=0x7f8bbc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:10.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.683+0000 7f8bc3400640 1 -- 172.21.15.42:0/309611680 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc005ae0 con 0x7f8bcc0ae240 2024-05-28T14:04:10.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.683+0000 7f8bc3400640 1 -- 172.21.15.42:0/309611680 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bbc005ca0 con 0x7f8bcc0ae240 2024-05-28T14:04:10.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.683+0000 7f8bc3400640 1 -- 172.21.15.42:0/309611680 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0072b0 con 0x7f8bcc0ae240 2024-05-28T14:04:10.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.683+0000 7f8bd2600640 1 -- 172.21.15.42:0/309611680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bcc1543d0 con 0x7f8bcc0ae240 2024-05-28T14:04:10.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.683+0000 7f8bd2600640 1 -- 172.21.15.42:0/309611680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bcc154eb0 con 0x7f8bcc0ae240 2024-05-28T14:04:10.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.683+0000 7f8bd2600640 1 -- 172.21.15.42:0/309611680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bcc09f5c0 con 0x7f8bcc0ae240 2024-05-28T14:04:10.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.683+0000 7f8bc3400640 1 -- 172.21.15.42:0/309611680 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f8bbc007450 con 0x7f8bcc0ae240 2024-05-28T14:04:10.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.683+0000 7f8bc3400640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ba8068b60 0x7f8ba806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:10.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.683+0000 7f8bc3400640 1 -- 172.21.15.42:0/309611680 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5851+0+0 (secure 0 0 0) 0x7f8bbc087ad0 con 0x7f8bcc0ae240 2024-05-28T14:04:10.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.683+0000 7f8bc3e00640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ba8068b60 0x7f8ba806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:10.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.687+0000 7f8bc3e00640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ba8068b60 0x7f8ba806b020 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc15b8c0 tx=0x7f8bb4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:10.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.687+0000 7f8bc3400640 1 -- 172.21.15.42:0/309611680 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8bbc010300 con 0x7f8bcc0ae240 2024-05-28T14:04:10.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.887+0000 7f8bd2600640 1 -- 172.21.15.42:0/309611680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8bcc04fdd0 con 0x7f8bcc0ae240 2024-05-28T14:04:10.875 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-05-28T14:04:10.887+0000 7f248b600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-28T14:04:10.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.891+0000 7f8bc3400640 1 -- 172.21.15.42:0/309611680 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f8bbc057070 con 0x7f8bcc0ae240 2024-05-28T14:04:10.877 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-28T14:04:10.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.895+0000 7f8bd2600640 1 -- 172.21.15.42:0/309611680 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ba8068b60 msgr2=0x7f8ba806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:10.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.895+0000 7f8bd2600640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ba8068b60 0x7f8ba806b020 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc15b8c0 tx=0x7f8bb4008040 comp rx=0 tx=0).stop 2024-05-28T14:04:10.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.895+0000 7f8bd2600640 1 -- 172.21.15.42:0/309611680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bcc0ae240 msgr2=0x7f8bcc13f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:10.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.895+0000 7f8bd2600640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bcc0ae240 0x7f8bcc13f6c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00c750 tx=0x7f8bbc00cc10 comp rx=0 tx=0).stop 2024-05-28T14:04:10.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.895+0000 7f8bd2600640 1 -- 172.21.15.42:0/309611680 shutdown_connections 2024-05-28T14:04:10.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.895+0000 7f8bd2600640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ba8068b60 0x7f8ba806b020 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.895+0000 7f8bd2600640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bcc15e820 0x7f8bcc15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.895+0000 7f8bd2600640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bcc0b1640 0x7f8bcc15ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.895+0000 7f8bd2600640 1 --2- 172.21.15.42:0/309611680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bcc0ae240 0x7f8bcc13f6c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:10.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.895+0000 7f8bd2600640 1 -- 172.21.15.42:0/309611680 >> 172.21.15.42:0/309611680 conn(0x7f8bcc0b6780 msgr2=0x7f8bcc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:10.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.895+0000 7f8bd2600640 1 -- 172.21.15.42:0/309611680 shutdown_connections 2024-05-28T14:04:10.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:10.895+0000 7f8bd2600640 1 -- 172.21.15.42:0/309611680 wait complete. 2024-05-28T14:04:10.901 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-28T14:04:10.951 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T14:04:11.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29b0800640 1 Processor -- start 2024-05-28T14:04:11.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29b0800640 1 -- start start 2024-05-28T14:04:11.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29b0800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29a80d0d40 0x7f29a809b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:11.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29b0800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29a80d11f0 con 0x7f29a806d3e0 2024-05-28T14:04:11.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29b0800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29a80d14d0 con 0x7f29a81603f0 2024-05-28T14:04:11.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29b0800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f29a809ac80 con 0x7f29a80d0d40 2024-05-28T14:04:11.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29ade00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f29a806d3e0 0x7f29a806d7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47580/0 (socket says 172.21.15.42:47580) 2024-05-28T14:04:11.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29ade00640 1 -- 172.21.15.42:0/32748133 learned_addr learned my addr 172.21.15.42:0/32748133 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:11.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29ae800640 1 --2- 172.21.15.42:0/32748133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29a80d0d40 0x7f29a809b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:11.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29aca00640 1 -- 172.21.15.42:0/32748133 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3613741268 0 0) 0x7f29a80d11f0 con 0x7f29a806d3e0 2024-05-28T14:04:11.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29aca00640 1 -- 172.21.15.42:0/32748133 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2990003530 con 0x7f29a806d3e0 2024-05-28T14:04:11.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29ae800640 1 -- 172.21.15.42:0/32748133 >> v1:172.21.15.63:6789/0 conn(0x7f29a81603f0 legacy=0x7f29a80d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:11.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29aca00640 1 -- 172.21.15.42:0/32748133 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3668243267 0 0) 0x7f29a80d14d0 con 0x7f29a81603f0 2024-05-28T14:04:11.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29ae800640 1 -- 172.21.15.42:0/32748133 >> v1:172.21.15.42:6789/0 conn(0x7f29a806d3e0 legacy=0x7f29a806d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:11.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.059+0000 7f29ae800640 1 -- 172.21.15.42:0/32748133 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a80d17b0 con 0x7f29a80d0d40 2024-05-28T14:04:11.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.063+0000 7f29ae800640 1 --2- 172.21.15.42:0/32748133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29a80d0d40 0x7f29a809b110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009dc0 tx=0x7f29a401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1f98799c2881654 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:11.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.063+0000 7f29aca00640 1 -- 172.21.15.42:0/32748133 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a40b2070 con 0x7f29a80d0d40 2024-05-28T14:04:11.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.063+0000 7f29aca00640 1 -- 172.21.15.42:0/32748133 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a40a5850 con 0x7f29a80d0d40 2024-05-28T14:04:11.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.063+0000 7f29aca00640 1 -- 172.21.15.42:0/32748133 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a809be60 con 0x7f29a80d0d40 2024-05-28T14:04:11.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.063+0000 7f29b0800640 1 -- 172.21.15.42:0/32748133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29a80d0d40 msgr2=0x7f29a809b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:11.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.063+0000 7f29b0800640 1 --2- 172.21.15.42:0/32748133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29a80d0d40 0x7f29a809b110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009dc0 tx=0x7f29a401c990 comp rx=0 tx=0).stop 2024-05-28T14:04:11.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.063+0000 7f29b0800640 1 -- 172.21.15.42:0/32748133 shutdown_connections 2024-05-28T14:04:11.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.063+0000 7f29b0800640 1 --2- 172.21.15.42:0/32748133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29a80d0d40 0x7f29a809b110 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:11.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.063+0000 7f29b0800640 1 -- 172.21.15.42:0/32748133 >> 172.21.15.42:0/32748133 conn(0x7f29a80b3f90 msgr2=0x7f29a80b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:11.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.063+0000 7f29b0800640 1 -- 172.21.15.42:0/32748133 shutdown_connections 2024-05-28T14:04:11.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.063+0000 7f29b0800640 1 -- 172.21.15.42:0/32748133 wait complete. 2024-05-28T14:04:11.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29b0800640 1 Processor -- start 2024-05-28T14:04:11.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29b0800640 1 -- start start 2024-05-28T14:04:11.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29b0800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29a806d3e0 0x7f29a8154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:11.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29b0800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29a80d0d40 0x7f29a81567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:11.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29b0800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29a81603f0 0x7f29a8151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:11.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29b0800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f29a816cc10 con 0x7f29a806d3e0 2024-05-28T14:04:11.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29b0800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29a816ca90 con 0x7f29a81603f0 2024-05-28T14:04:11.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29b0800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f29a816cd90 con 0x7f29a80d0d40 2024-05-28T14:04:11.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29ade00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29a806d3e0 0x7f29a8154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:11.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29ade00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29a806d3e0 0x7f29a8154b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35274/0 (socket says 172.21.15.42:35274) 2024-05-28T14:04:11.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29ade00640 1 -- 172.21.15.42:0/162221259 learned_addr learned my addr 172.21.15.42:0/162221259 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:11.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29ae800640 1 --2- 172.21.15.42:0/162221259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29a81603f0 0x7f29a8151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:11.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29ad400640 1 --2- 172.21.15.42:0/162221259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29a80d0d40 0x7f29a81567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:11.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29ade00640 1 -- 172.21.15.42:0/162221259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29a80d0d40 msgr2=0x7f29a81567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:11.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29ade00640 1 --2- 172.21.15.42:0/162221259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29a80d0d40 0x7f29a81567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:11.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29ade00640 1 -- 172.21.15.42:0/162221259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29a81603f0 msgr2=0x7f29a8151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:11.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29ade00640 1 --2- 172.21.15.42:0/162221259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29a81603f0 0x7f29a8151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:11.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29ade00640 1 -- 172.21.15.42:0/162221259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a80b1800 con 0x7f29a806d3e0 2024-05-28T14:04:11.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29ade00640 1 --2- 172.21.15.42:0/162221259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29a806d3e0 0x7f29a8154b80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f299c00c3f0 tx=0x7f299c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:11.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29a2000640 1 -- 172.21.15.42:0/162221259 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c017070 con 0x7f29a806d3e0 2024-05-28T14:04:11.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29a2000640 1 -- 172.21.15.42:0/162221259 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f299c005cf0 con 0x7f29a806d3e0 2024-05-28T14:04:11.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29a2000640 1 -- 172.21.15.42:0/162221259 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c0072f0 con 0x7f29a806d3e0 2024-05-28T14:04:11.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29b0800640 1 -- 172.21.15.42:0/162221259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a80afe30 con 0x7f29a806d3e0 2024-05-28T14:04:11.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29b0800640 1 -- 172.21.15.42:0/162221259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a8155d00 con 0x7f29a806d3e0 2024-05-28T14:04:11.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.067+0000 7f29b0800640 1 -- 172.21.15.42:0/162221259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a80a0550 con 0x7f29a806d3e0 2024-05-28T14:04:11.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.071+0000 7f29a2000640 1 -- 172.21.15.42:0/162221259 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f299c01a070 con 0x7f29a806d3e0 2024-05-28T14:04:11.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.071+0000 7f29a2000640 1 --2- 172.21.15.42:0/162221259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f298c068b60 0x7f298c06b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:11.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.071+0000 7f29a2000640 1 -- 172.21.15.42:0/162221259 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5909+0+0 (secure 0 0 0) 0x7f299c087090 con 0x7f29a806d3e0 2024-05-28T14:04:11.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.071+0000 7f29ad400640 1 --2- 172.21.15.42:0/162221259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f298c068b60 0x7f298c06b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:11.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.071+0000 7f29ad400640 1 --2- 172.21.15.42:0/162221259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f298c068b60 0x7f298c06b020 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f29a81574e0 tx=0x7f2998009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:11.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.075+0000 7f29a2000640 1 -- 172.21.15.42:0/162221259 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f299c055fa0 con 0x7f29a806d3e0 2024-05-28T14:04:11.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.275+0000 7f29b0800640 1 -- 172.21.15.42:0/162221259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f29a804fdd0 con 0x7f29a806d3e0 2024-05-28T14:04:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.895+0000 7f29a2000640 1 -- 172.21.15.42:0/162221259 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7f299c05a2a0 con 0x7f29a806d3e0 2024-05-28T14:04:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.899+0000 7f29b0800640 1 -- 172.21.15.42:0/162221259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f298c068b60 msgr2=0x7f298c06b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.899+0000 7f29b0800640 1 --2- 172.21.15.42:0/162221259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f298c068b60 0x7f298c06b020 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f29a81574e0 tx=0x7f2998009040 comp rx=0 tx=0).stop 2024-05-28T14:04:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.899+0000 7f29b0800640 1 -- 172.21.15.42:0/162221259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29a806d3e0 msgr2=0x7f29a8154b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.899+0000 7f29b0800640 1 --2- 172.21.15.42:0/162221259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29a806d3e0 0x7f29a8154b80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f299c00c3f0 tx=0x7f299c00c8b0 comp rx=0 tx=0).stop 2024-05-28T14:04:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.899+0000 7f29b0800640 1 -- 172.21.15.42:0/162221259 shutdown_connections 2024-05-28T14:04:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.899+0000 7f29b0800640 1 --2- 172.21.15.42:0/162221259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f298c068b60 0x7f298c06b020 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.899+0000 7f29b0800640 1 --2- 172.21.15.42:0/162221259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29a81603f0 0x7f29a8151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.899+0000 7f29b0800640 1 --2- 172.21.15.42:0/162221259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29a80d0d40 0x7f29a81567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.899+0000 7f29b0800640 1 --2- 172.21.15.42:0/162221259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29a806d3e0 0x7f29a8154b80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:11.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.899+0000 7f29b0800640 1 -- 172.21.15.42:0/162221259 >> 172.21.15.42:0/162221259 conn(0x7f29a80b3f90 msgr2=0x7f29a80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:11.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.899+0000 7f29b0800640 1 -- 172.21.15.42:0/162221259 shutdown_connections 2024-05-28T14:04:11.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:11.899+0000 7f29b0800640 1 -- 172.21.15.42:0/162221259 wait complete. 2024-05-28T14:04:11.911 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-28T14:04:12.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb359400640 1 Processor -- start 2024-05-28T14:04:12.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb359400640 1 -- start start 2024-05-28T14:04:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb359400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb35415e840 0x7fb3540d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb359400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb35406cd10 0x7fb35406d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb359400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3540d1970 con 0x7fb35406da50 2024-05-28T14:04:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb359400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb3540b5300 con 0x7fb35406cd10 2024-05-28T14:04:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb359400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb3540d1740 con 0x7fb35415e840 2024-05-28T14:04:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb34be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb35406cd10 0x7fb35406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb34be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb35406cd10 0x7fb35406d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35276/0 (socket says 172.21.15.42:35276) 2024-05-28T14:04:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb34be00640 1 -- 172.21.15.42:0/573154197 learned_addr learned my addr 172.21.15.42:0/573154197 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb352800640 1 --2- 172.21.15.42:0/573154197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb35415e840 0x7fb3540d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb34be00640 1 -- 172.21.15.42:0/573154197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb35415e840 msgr2=0x7fb3540d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb34be00640 1 --2- 172.21.15.42:0/573154197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb35415e840 0x7fb3540d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb34be00640 1 -- 172.21.15.42:0/573154197 >> v1:172.21.15.63:6789/0 conn(0x7fb35406da50 legacy=0x7fb35409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:12.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb34be00640 1 -- 172.21.15.42:0/573154197 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3540ccca0 con 0x7fb35406cd10 2024-05-28T14:04:12.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.075+0000 7fb352800640 1 --2- 172.21.15.42:0/573154197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb35415e840 0x7fb3540d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.079+0000 7fb34be00640 1 --2- 172.21.15.42:0/573154197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb35406cd10 0x7fb35406d510 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009870 tx=0x7fb33c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837a6491a2d9d8f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.079+0000 7fb351e00640 1 -- 172.21.15.42:0/573154197 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0b5070 con 0x7fb35406cd10 2024-05-28T14:04:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.079+0000 7fb351e00640 1 -- 172.21.15.42:0/573154197 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33c0a89d0 con 0x7fb35406cd10 2024-05-28T14:04:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.079+0000 7fb351e00640 1 -- 172.21.15.42:0/573154197 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0a8da0 con 0x7fb35406cd10 2024-05-28T14:04:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.079+0000 7fb359400640 1 -- 172.21.15.42:0/573154197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb35406cd10 msgr2=0x7fb35406d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.079+0000 7fb359400640 1 --2- 172.21.15.42:0/573154197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb35406cd10 0x7fb35406d510 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009870 tx=0x7fb33c01ed10 comp rx=0 tx=0).stop 2024-05-28T14:04:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.079+0000 7fb359400640 1 -- 172.21.15.42:0/573154197 shutdown_connections 2024-05-28T14:04:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.079+0000 7fb359400640 1 --2- 172.21.15.42:0/573154197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb35406cd10 0x7fb35406d510 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.079+0000 7fb359400640 1 --2- 172.21.15.42:0/573154197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb35415e840 0x7fb3540d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.079+0000 7fb359400640 1 -- 172.21.15.42:0/573154197 >> 172.21.15.42:0/573154197 conn(0x7fb3540b3fa0 msgr2=0x7fb3540b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.079+0000 7fb359400640 1 -- 172.21.15.42:0/573154197 shutdown_connections 2024-05-28T14:04:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.079+0000 7fb359400640 1 -- 172.21.15.42:0/573154197 wait complete. 2024-05-28T14:04:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb359400640 1 Processor -- start 2024-05-28T14:04:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb359400640 1 -- start start 2024-05-28T14:04:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb359400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb35406cd10 0x7fb3540c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb359400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb35406da50 0x7fb35409cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb359400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb35415e840 0x7fb354153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb359400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb3540cd260 con 0x7fb35406da50 2024-05-28T14:04:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb359400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb35406dea0 con 0x7fb35406cd10 2024-05-28T14:04:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb359400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb35415ec20 con 0x7fb35415e840 2024-05-28T14:04:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb34be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb35406da50 0x7fb35409cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb34be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb35406da50 0x7fb35409cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35284/0 (socket says 172.21.15.42:35284) 2024-05-28T14:04:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb34be00640 1 -- 172.21.15.42:0/1913778688 learned_addr learned my addr 172.21.15.42:0/1913778688 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb352800640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb35406cd10 0x7fb3540c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb34be00640 1 -- 172.21.15.42:0/1913778688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb35415e840 msgr2=0x7fb354153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb353200640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb35415e840 0x7fb354153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb34be00640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb35415e840 0x7fb354153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb34be00640 1 -- 172.21.15.42:0/1913778688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb35406cd10 msgr2=0x7fb3540c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb34be00640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb35406cd10 0x7fb3540c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb34be00640 1 -- 172.21.15.42:0/1913778688 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb354154290 con 0x7fb35406da50 2024-05-28T14:04:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb352800640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb35406cd10 0x7fb3540c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb353200640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb35415e840 0x7fb354153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb34be00640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb35406da50 0x7fb35409cbd0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb33c01d3c0 tx=0x7fb33c01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb34aa00640 1 -- 172.21.15.42:0/1913778688 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0b5070 con 0x7fb35406da50 2024-05-28T14:04:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb34aa00640 1 -- 172.21.15.42:0/1913778688 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33c0b0b40 con 0x7fb35406da50 2024-05-28T14:04:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb34aa00640 1 -- 172.21.15.42:0/1913778688 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0bb410 con 0x7fb35406da50 2024-05-28T14:04:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb359400640 1 -- 172.21.15.42:0/1913778688 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb354154570 con 0x7fb35406da50 2024-05-28T14:04:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.083+0000 7fb359400640 1 -- 172.21.15.42:0/1913778688 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb354154b30 con 0x7fb35406da50 2024-05-28T14:04:12.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.087+0000 7fb359400640 1 -- 172.21.15.42:0/1913778688 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb354068a90 con 0x7fb35406da50 2024-05-28T14:04:12.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.087+0000 7fb34aa00640 1 -- 172.21.15.42:0/1913778688 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fb33c0c2060 con 0x7fb35406da50 2024-05-28T14:04:12.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.087+0000 7fb34aa00640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb330068b60 0x7fb33006b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:12.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.087+0000 7fb34aa00640 1 -- 172.21.15.42:0/1913778688 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5909+0+0 (secure 0 0 0) 0x7fb33c124b60 con 0x7fb35406da50 2024-05-28T14:04:12.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.087+0000 7fb352800640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb330068b60 0x7fb33006b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:12.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.087+0000 7fb352800640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb330068b60 0x7fb33006b020 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb344008890 tx=0x7fb344008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:12.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.091+0000 7fb34aa00640 1 -- 172.21.15.42:0/1913778688 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb33c018870 con 0x7fb35406da50 2024-05-28T14:04:12.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.255+0000 7fb359400640 1 -- 172.21.15.42:0/1913778688 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb35406d620 con 0x7fb35406da50 2024-05-28T14:04:12.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.503+0000 7fb34aa00640 1 -- 172.21.15.42:0/1913778688 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) ==== 181+0+0 (secure 0 0 0) 0x7fb33c0f44d0 con 0x7fb35406da50 2024-05-28T14:04:12.490 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-05-28T14:04:12.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.507+0000 7fb359400640 1 -- 172.21.15.42:0/1913778688 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb330068b60 msgr2=0x7fb33006b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:12.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.507+0000 7fb359400640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb330068b60 0x7fb33006b020 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb344008890 tx=0x7fb344008040 comp rx=0 tx=0).stop 2024-05-28T14:04:12.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.507+0000 7fb359400640 1 -- 172.21.15.42:0/1913778688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb35406da50 msgr2=0x7fb35409cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:12.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.507+0000 7fb359400640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb35406da50 0x7fb35409cbd0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb33c01d3c0 tx=0x7fb33c01bab0 comp rx=0 tx=0).stop 2024-05-28T14:04:12.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.507+0000 7fb359400640 1 -- 172.21.15.42:0/1913778688 shutdown_connections 2024-05-28T14:04:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.507+0000 7fb359400640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb330068b60 0x7fb33006b020 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.507+0000 7fb359400640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb35415e840 0x7fb354153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.507+0000 7fb359400640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb35406da50 0x7fb35409cbd0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.507+0000 7fb359400640 1 --2- 172.21.15.42:0/1913778688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb35406cd10 0x7fb3540c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.507+0000 7fb359400640 1 -- 172.21.15.42:0/1913778688 >> 172.21.15.42:0/1913778688 conn(0x7fb3540b3fa0 msgr2=0x7fb3540d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.507+0000 7fb359400640 1 -- 172.21.15.42:0/1913778688 shutdown_connections 2024-05-28T14:04:12.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.507+0000 7fb359400640 1 -- 172.21.15.42:0/1913778688 wait complete. 2024-05-28T14:04:12.517 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-28T14:04:12.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.675+0000 7f47bfe00640 1 Processor -- start 2024-05-28T14:04:12.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.675+0000 7f47bfe00640 1 -- start start 2024-05-28T14:04:12.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.675+0000 7f47bfe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f47c015c4d0 0x7f47c00cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:12.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.675+0000 7f47bfe00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c015c8b0 con 0x7f47c00c5e20 2024-05-28T14:04:12.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.675+0000 7f47bfe00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c00d11d0 con 0x7f47c00cfea0 2024-05-28T14:04:12.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.675+0000 7f47bfe00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f47c00cf8f0 con 0x7f47c015c4d0 2024-05-28T14:04:12.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.675+0000 7f47bf400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f47c00c5e20 0x7f47c00d0ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47594/0 (socket says 172.21.15.42:47594) 2024-05-28T14:04:12.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.675+0000 7f47bf400640 1 -- 172.21.15.42:0/2885694723 learned_addr learned my addr 172.21.15.42:0/2885694723 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:12.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.675+0000 7f47bea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f47c015c4d0 0x7f47c00cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:12.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.679+0000 7f47bea00640 1 -- 172.21.15.42:0/2885694723 >> v1:172.21.15.63:6789/0 conn(0x7f47c00cfea0 legacy=0x7f47c00d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:12.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.679+0000 7f47bea00640 1 -- 172.21.15.42:0/2885694723 >> v1:172.21.15.42:6789/0 conn(0x7f47c00c5e20 legacy=0x7f47c00d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:12.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.679+0000 7f47bea00640 1 -- 172.21.15.42:0/2885694723 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c00cece0 con 0x7f47c015c4d0 2024-05-28T14:04:12.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.679+0000 7f47bea00640 1 --2- 172.21.15.42:0/2885694723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c015c4d0 0x7f47c00cc920 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f47b00097c0 tx=0x7f47b001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c7261efc159d4d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:12.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.679+0000 7f47be000640 1 -- 172.21.15.42:0/2885694723 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b00ab430 con 0x7f47c015c4d0 2024-05-28T14:04:12.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.679+0000 7f47be000640 1 -- 172.21.15.42:0/2885694723 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b00b0070 con 0x7f47c015c4d0 2024-05-28T14:04:12.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.679+0000 7f47be000640 1 -- 172.21.15.42:0/2885694723 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b00ab7f0 con 0x7f47c015c4d0 2024-05-28T14:04:12.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.679+0000 7f47bfe00640 1 -- 172.21.15.42:0/2885694723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c015c4d0 msgr2=0x7f47c00cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:12.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.679+0000 7f47bfe00640 1 --2- 172.21.15.42:0/2885694723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c015c4d0 0x7f47c00cc920 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f47b00097c0 tx=0x7f47b001c430 comp rx=0 tx=0).stop 2024-05-28T14:04:12.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.679+0000 7f47bfe00640 1 -- 172.21.15.42:0/2885694723 shutdown_connections 2024-05-28T14:04:12.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.679+0000 7f47bfe00640 1 --2- 172.21.15.42:0/2885694723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c015c4d0 0x7f47c00cc920 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:12.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.679+0000 7f47bfe00640 1 -- 172.21.15.42:0/2885694723 >> 172.21.15.42:0/2885694723 conn(0x7f47c00b3ec0 msgr2=0x7f47c00b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:12.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.679+0000 7f47bfe00640 1 -- 172.21.15.42:0/2885694723 shutdown_connections 2024-05-28T14:04:12.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.679+0000 7f47bfe00640 1 -- 172.21.15.42:0/2885694723 wait complete. 2024-05-28T14:04:12.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bfe00640 1 Processor -- start 2024-05-28T14:04:12.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bfe00640 1 -- start start 2024-05-28T14:04:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c00c5e20 0x7f47c015e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47c00cfea0 0x7f47c0155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47c015c4d0 0x7f47c015b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f47c00ccfe0 con 0x7f47c00c5e20 2024-05-28T14:04:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bfe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f47c00cce60 con 0x7f47c015c4d0 2024-05-28T14:04:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bfe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f47c00cd160 con 0x7f47c00cfea0 2024-05-28T14:04:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c00c5e20 0x7f47c015e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c00c5e20 0x7f47c015e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35290/0 (socket says 172.21.15.42:35290) 2024-05-28T14:04:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bea00640 1 -- 172.21.15.42:0/3569345875 learned_addr learned my addr 172.21.15.42:0/3569345875 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bea00640 1 -- 172.21.15.42:0/3569345875 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47c00cfea0 msgr2=0x7f47c0155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bea00640 1 --2- 172.21.15.42:0/3569345875 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47c00cfea0 0x7f47c0155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bea00640 1 -- 172.21.15.42:0/3569345875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47c015c4d0 msgr2=0x7f47c015b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bf400640 1 --2- 172.21.15.42:0/3569345875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47c015c4d0 0x7f47c015b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bea00640 1 --2- 172.21.15.42:0/3569345875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47c015c4d0 0x7f47c015b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bea00640 1 -- 172.21.15.42:0/3569345875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c015eff0 con 0x7f47c00c5e20 2024-05-28T14:04:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bf400640 1 --2- 172.21.15.42:0/3569345875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47c015c4d0 0x7f47c015b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bea00640 1 --2- 172.21.15.42:0/3569345875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c00c5e20 0x7f47c015e980 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f47b00ad040 tx=0x7f47b001ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47b6a00640 1 -- 172.21.15.42:0/3569345875 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b00b5810 con 0x7f47c00c5e20 2024-05-28T14:04:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47b6a00640 1 -- 172.21.15.42:0/3569345875 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b00b59b0 con 0x7f47c00c5e20 2024-05-28T14:04:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47b6a00640 1 -- 172.21.15.42:0/3569345875 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b00b5da0 con 0x7f47c00c5e20 2024-05-28T14:04:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bfe00640 1 -- 172.21.15.42:0/3569345875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c015f2d0 con 0x7f47c00c5e20 2024-05-28T14:04:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.683+0000 7f47bfe00640 1 -- 172.21.15.42:0/3569345875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c015d4f0 con 0x7f47c00c5e20 2024-05-28T14:04:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.687+0000 7f47bfe00640 1 -- 172.21.15.42:0/3569345875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c0068a90 con 0x7f47c00c5e20 2024-05-28T14:04:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.687+0000 7f47b6a00640 1 -- 172.21.15.42:0/3569345875 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f47b00bd020 con 0x7f47c00c5e20 2024-05-28T14:04:12.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.687+0000 7f47b6a00640 1 --2- 172.21.15.42:0/3569345875 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f479c068b90 0x7f479c06b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:12.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.687+0000 7f47b6a00640 1 -- 172.21.15.42:0/3569345875 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5471+0+0 (secure 0 0 0) 0x7f47b00a8030 con 0x7f47c00c5e20 2024-05-28T14:04:12.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.687+0000 7f47b7e00640 1 --2- 172.21.15.42:0/3569345875 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f479c068b90 0x7f479c06b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:12.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.687+0000 7f47b7e00640 1 --2- 172.21.15.42:0/3569345875 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f479c068b90 0x7f479c06b050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f47a8007fd0 tx=0x7f47a80079a0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:12.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.691+0000 7f47b6a00640 1 -- 172.21.15.42:0/3569345875 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f47b00c4060 con 0x7f47c00c5e20 2024-05-28T14:04:12.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:12.863+0000 7f47bfe00640 1 -- 172.21.15.42:0/3569345875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f47c006e9a0 con 0x7f47c00c5e20 2024-05-28T14:04:13.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.507+0000 7f47b6a00640 1 -- 172.21.15.42:0/3569345875 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) ==== 169+0+0 (secure 0 0 0) 0x7f47b0016050 con 0x7f47c00c5e20 2024-05-28T14:04:13.496 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-05-28T14:04:13.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.515+0000 7f47bfe00640 1 -- 172.21.15.42:0/3569345875 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f479c068b90 msgr2=0x7f479c06b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:13.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.515+0000 7f47bfe00640 1 --2- 172.21.15.42:0/3569345875 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f479c068b90 0x7f479c06b050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f47a8007fd0 tx=0x7f47a80079a0 comp rx=0 tx=0).stop 2024-05-28T14:04:13.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.515+0000 7f47bfe00640 1 -- 172.21.15.42:0/3569345875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c00c5e20 msgr2=0x7f47c015e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:13.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.515+0000 7f47bfe00640 1 --2- 172.21.15.42:0/3569345875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c00c5e20 0x7f47c015e980 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f47b00ad040 tx=0x7f47b001ac70 comp rx=0 tx=0).stop 2024-05-28T14:04:13.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.515+0000 7f47bfe00640 1 -- 172.21.15.42:0/3569345875 shutdown_connections 2024-05-28T14:04:13.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.515+0000 7f47bfe00640 1 --2- 172.21.15.42:0/3569345875 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f479c068b90 0x7f479c06b050 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:13.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.515+0000 7f47bfe00640 1 --2- 172.21.15.42:0/3569345875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47c015c4d0 0x7f47c015b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:13.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.515+0000 7f47bfe00640 1 --2- 172.21.15.42:0/3569345875 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47c00cfea0 0x7f47c0155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:13.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.515+0000 7f47bfe00640 1 --2- 172.21.15.42:0/3569345875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c00c5e20 0x7f47c015e980 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:13.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.515+0000 7f47bfe00640 1 -- 172.21.15.42:0/3569345875 >> 172.21.15.42:0/3569345875 conn(0x7f47c00b3ec0 msgr2=0x7f47c006d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:13.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.515+0000 7f47bfe00640 1 -- 172.21.15.42:0/3569345875 shutdown_connections 2024-05-28T14:04:13.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.515+0000 7f47bfe00640 1 -- 172.21.15.42:0/3569345875 wait complete. 2024-05-28T14:04:13.531 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-28T14:04:13.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.691+0000 7f12c9600640 1 Processor -- start 2024-05-28T14:04:13.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.691+0000 7f12c9600640 1 -- start start 2024-05-28T14:04:13.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.691+0000 7f12c9600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f12c406ee20 0x7f12c406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:13.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.691+0000 7f12c9600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12c40afe40 con 0x7f12c4144f80 2024-05-28T14:04:13.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.691+0000 7f12c9600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12c415c600 con 0x7f12c415b9b0 2024-05-28T14:04:13.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.691+0000 7f12c9600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f12c406f7c0 con 0x7f12c406ee20 2024-05-28T14:04:13.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.695+0000 7f12bbe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f12c406ee20 0x7f12c406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:13.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.695+0000 7f12c2a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f12c4144f80 0x7f12c4072960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47608/0 (socket says 172.21.15.42:47608) 2024-05-28T14:04:13.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.695+0000 7f12c2a00640 1 -- 172.21.15.42:0/4150494513 learned_addr learned my addr 172.21.15.42:0/4150494513 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:13.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.695+0000 7f12bbe00640 1 -- 172.21.15.42:0/4150494513 >> v1:172.21.15.63:6789/0 conn(0x7f12c415b9b0 legacy=0x7f12c415be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:13.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.695+0000 7f12bbe00640 1 -- 172.21.15.42:0/4150494513 >> v1:172.21.15.42:6789/0 conn(0x7f12c4144f80 legacy=0x7f12c4072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:13.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.695+0000 7f12c2000640 1 -- 172.21.15.42:0/4150494513 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 795476552 0 0) 0x7f12c40afe40 con 0x7f12c4144f80 2024-05-28T14:04:13.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.695+0000 7f12bbe00640 1 -- 172.21.15.42:0/4150494513 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c415c8e0 con 0x7f12c406ee20 2024-05-28T14:04:13.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.695+0000 7f12bbe00640 1 --2- 172.21.15.42:0/4150494513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c406ee20 0x7f12c406f280 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0097c0 tx=0x7f12ac01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b61f11128ac11b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:13.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.695+0000 7f12c2000640 1 -- 172.21.15.42:0/4150494513 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac0b0070 con 0x7f12c406ee20 2024-05-28T14:04:13.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.695+0000 7f12c2000640 1 -- 172.21.15.42:0/4150494513 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12ac0ab600 con 0x7f12c406ee20 2024-05-28T14:04:13.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.695+0000 7f12c2000640 1 -- 172.21.15.42:0/4150494513 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac0ab9d0 con 0x7f12c406ee20 2024-05-28T14:04:13.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 -- 172.21.15.42:0/4150494513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c406ee20 msgr2=0x7f12c406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:13.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 --2- 172.21.15.42:0/4150494513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c406ee20 0x7f12c406f280 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0097c0 tx=0x7f12ac01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:04:13.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 -- 172.21.15.42:0/4150494513 shutdown_connections 2024-05-28T14:04:13.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 --2- 172.21.15.42:0/4150494513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c406ee20 0x7f12c406f280 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:13.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 -- 172.21.15.42:0/4150494513 >> 172.21.15.42:0/4150494513 conn(0x7f12c40b21f0 msgr2=0x7f12c40b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:13.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 -- 172.21.15.42:0/4150494513 shutdown_connections 2024-05-28T14:04:13.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 -- 172.21.15.42:0/4150494513 wait complete. 2024-05-28T14:04:13.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 Processor -- start 2024-05-28T14:04:13.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 -- start start 2024-05-28T14:04:13.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12c406ee20 0x7f12c4142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:13.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c4144f80 0x7f12c4141210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:13.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12c415b9b0 0x7f12c4141750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:13.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f12c40724d0 con 0x7f12c4144f80 2024-05-28T14:04:13.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f12c40b4260 con 0x7f12c415b9b0 2024-05-28T14:04:13.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c9600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f12c4169140 con 0x7f12c406ee20 2024-05-28T14:04:13.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12bbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c4144f80 0x7f12c4141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:13.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12bbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c4144f80 0x7f12c4141210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35302/0 (socket says 172.21.15.42:35302) 2024-05-28T14:04:13.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12bbe00640 1 -- 172.21.15.42:0/3740189031 learned_addr learned my addr 172.21.15.42:0/3740189031 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:13.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c2a00640 1 --2- 172.21.15.42:0/3740189031 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12c406ee20 0x7f12c4142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:13.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12c3400640 1 --2- 172.21.15.42:0/3740189031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12c415b9b0 0x7f12c4141750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:13.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12bbe00640 1 -- 172.21.15.42:0/3740189031 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12c406ee20 msgr2=0x7f12c4142fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:13.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12bbe00640 1 --2- 172.21.15.42:0/3740189031 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12c406ee20 0x7f12c4142fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:13.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12bbe00640 1 -- 172.21.15.42:0/3740189031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12c415b9b0 msgr2=0x7f12c4141750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:13.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12bbe00640 1 --2- 172.21.15.42:0/3740189031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12c415b9b0 0x7f12c4141750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:13.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.699+0000 7f12bbe00640 1 -- 172.21.15.42:0/3740189031 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c4141c90 con 0x7f12c4144f80 2024-05-28T14:04:13.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.703+0000 7f12bbe00640 1 --2- 172.21.15.42:0/3740189031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c4144f80 0x7f12c4141210 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0097c0 tx=0x7f12ac01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:13.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.703+0000 7f12baa00640 1 -- 172.21.15.42:0/3740189031 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac0b0050 con 0x7f12c4144f80 2024-05-28T14:04:13.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.703+0000 7f12baa00640 1 -- 172.21.15.42:0/3740189031 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12ac0b1040 con 0x7f12c4144f80 2024-05-28T14:04:13.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.703+0000 7f12baa00640 1 -- 172.21.15.42:0/3740189031 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac004260 con 0x7f12c4144f80 2024-05-28T14:04:13.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.703+0000 7f12c9600640 1 -- 172.21.15.42:0/3740189031 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c4141f40 con 0x7f12c4144f80 2024-05-28T14:04:13.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.703+0000 7f12c9600640 1 -- 172.21.15.42:0/3740189031 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c4142500 con 0x7f12c4144f80 2024-05-28T14:04:13.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.703+0000 7f12c9600640 1 -- 172.21.15.42:0/3740189031 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c406fce0 con 0x7f12c4144f80 2024-05-28T14:04:13.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.703+0000 7f12baa00640 1 -- 172.21.15.42:0/3740189031 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f12ac0be060 con 0x7f12c4144f80 2024-05-28T14:04:13.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.703+0000 7f12baa00640 1 --2- 172.21.15.42:0/3740189031 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12a0068b60 0x7f12a006b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:13.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.703+0000 7f12baa00640 1 -- 172.21.15.42:0/3740189031 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f12ac1210b0 con 0x7f12c4144f80 2024-05-28T14:04:13.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.703+0000 7f12c2a00640 1 --2- 172.21.15.42:0/3740189031 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12a0068b60 0x7f12a006b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:13.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.707+0000 7f12c2a00640 1 --2- 172.21.15.42:0/3740189031 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12a0068b60 0x7f12a006b020 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f12b4000f70 tx=0x7f12b4007240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:13.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.707+0000 7f12baa00640 1 -- 172.21.15.42:0/3740189031 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f12ac016090 con 0x7f12c4144f80 2024-05-28T14:04:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.875+0000 7f12c9600640 1 -- 172.21.15.42:0/3740189031 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f12c40738b0 con 0x7f12c4144f80 2024-05-28T14:04:13.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.875+0000 7f12baa00640 1 -- 172.21.15.42:0/3740189031 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f12ac0f0620 con 0x7f12c4144f80 2024-05-28T14:04:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.879+0000 7f12c9600640 1 -- 172.21.15.42:0/3740189031 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12a0068b60 msgr2=0x7f12a006b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.879+0000 7f12c9600640 1 --2- 172.21.15.42:0/3740189031 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12a0068b60 0x7f12a006b020 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f12b4000f70 tx=0x7f12b4007240 comp rx=0 tx=0).stop 2024-05-28T14:04:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.879+0000 7f12c9600640 1 -- 172.21.15.42:0/3740189031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c4144f80 msgr2=0x7f12c4141210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.879+0000 7f12c9600640 1 --2- 172.21.15.42:0/3740189031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c4144f80 0x7f12c4141210 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0097c0 tx=0x7f12ac01ace0 comp rx=0 tx=0).stop 2024-05-28T14:04:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.879+0000 7f12c9600640 1 -- 172.21.15.42:0/3740189031 shutdown_connections 2024-05-28T14:04:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.879+0000 7f12c9600640 1 --2- 172.21.15.42:0/3740189031 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12a0068b60 0x7f12a006b020 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.879+0000 7f12c9600640 1 --2- 172.21.15.42:0/3740189031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12c415b9b0 0x7f12c4141750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:13.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.879+0000 7f12c9600640 1 --2- 172.21.15.42:0/3740189031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c4144f80 0x7f12c4141210 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.879+0000 7f12c9600640 1 --2- 172.21.15.42:0/3740189031 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12c406ee20 0x7f12c4142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.879+0000 7f12c9600640 1 -- 172.21.15.42:0/3740189031 >> 172.21.15.42:0/3740189031 conn(0x7f12c40b21f0 msgr2=0x7f12c406e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.879+0000 7f12c9600640 1 -- 172.21.15.42:0/3740189031 shutdown_connections 2024-05-28T14:04:13.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:13.879+0000 7f12c9600640 1 -- 172.21.15.42:0/3740189031 wait complete. 2024-05-28T14:04:13.886 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones' 2024-05-28T14:04:14.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f71400640 1 Processor -- start 2024-05-28T14:04:14.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f71400640 1 -- start start 2024-05-28T14:04:14.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f71400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6f6c156a70 0x7f6f6c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:14.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f71400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6f6c156e50 0x7f6f6c0af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:14.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f71400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f6c0afc80 con 0x7f6f6c0af880 2024-05-28T14:04:14.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f71400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6f6c15cb90 con 0x7f6f6c156a70 2024-05-28T14:04:14.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f71400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6f6c15ce50 con 0x7f6f6c156e50 2024-05-28T14:04:14.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f6a800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6f6c156a70 0x7f6f6c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:14.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f6a800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6f6c156a70 0x7f6f6c06dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35316/0 (socket says 172.21.15.42:35316) 2024-05-28T14:04:14.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f6a800640 1 -- 172.21.15.42:0/3754705853 learned_addr learned my addr 172.21.15.42:0/3754705853 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:14.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f63e00640 1 --2- 172.21.15.42:0/3754705853 >> v2:172.21.15.63:3300/0 conn(0x7f6f6c156e50 0x7f6f6c0af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:14.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f6a800640 1 -- 172.21.15.42:0/3754705853 >> v2:172.21.15.63:3300/0 conn(0x7f6f6c156e50 msgr2=0x7f6f6c0af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:14.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f6a800640 1 --2- 172.21.15.42:0/3754705853 >> v2:172.21.15.63:3300/0 conn(0x7f6f6c156e50 0x7f6f6c0af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f6a800640 1 -- 172.21.15.42:0/3754705853 >> v1:172.21.15.63:6789/0 conn(0x7f6f6c0af880 legacy=0x7f6f6c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:14.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f6a800640 1 -- 172.21.15.42:0/3754705853 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c15d130 con 0x7f6f6c156a70 2024-05-28T14:04:14.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.043+0000 7f6f63e00640 1 --2- 172.21.15.42:0/3754705853 >> v2:172.21.15.63:3300/0 conn(0x7f6f6c156e50 0x7f6f6c0af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:14.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.047+0000 7f6f6a800640 1 --2- 172.21.15.42:0/3754705853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c156a70 0x7f6f6c06dd60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0701f0 tx=0x7f6f5c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4e416d041d2400 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:14.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.047+0000 7f6f69e00640 1 -- 172.21.15.42:0/3754705853 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0a6040 con 0x7f6f6c156a70 2024-05-28T14:04:14.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.047+0000 7f6f69e00640 1 -- 172.21.15.42:0/3754705853 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c0af070 con 0x7f6f6c156a70 2024-05-28T14:04:14.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.047+0000 7f6f69e00640 1 -- 172.21.15.42:0/3754705853 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0aa860 con 0x7f6f6c156a70 2024-05-28T14:04:14.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.047+0000 7f6f71400640 1 -- 172.21.15.42:0/3754705853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c156a70 msgr2=0x7f6f6c06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:14.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.047+0000 7f6f71400640 1 --2- 172.21.15.42:0/3754705853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c156a70 0x7f6f6c06dd60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0701f0 tx=0x7f6f5c01c4e0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.047+0000 7f6f71400640 1 -- 172.21.15.42:0/3754705853 shutdown_connections 2024-05-28T14:04:14.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.047+0000 7f6f71400640 1 --2- 172.21.15.42:0/3754705853 >> v2:172.21.15.63:3300/0 conn(0x7f6f6c156e50 0x7f6f6c0af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.047+0000 7f6f71400640 1 --2- 172.21.15.42:0/3754705853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c156a70 0x7f6f6c06dd60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.047+0000 7f6f71400640 1 -- 172.21.15.42:0/3754705853 >> 172.21.15.42:0/3754705853 conn(0x7f6f6c06bdf0 msgr2=0x7f6f6c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:14.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.047+0000 7f6f71400640 1 -- 172.21.15.42:0/3754705853 shutdown_connections 2024-05-28T14:04:14.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.047+0000 7f6f71400640 1 -- 172.21.15.42:0/3754705853 wait complete. 2024-05-28T14:04:14.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.047+0000 7f6f71400640 1 Processor -- start 2024-05-28T14:04:14.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.047+0000 7f6f71400640 1 -- start start 2024-05-28T14:04:14.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f71400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f6c0af880 0x7f6f6c14aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:14.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f71400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c156a70 0x7f6f6c154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:14.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f71400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f6c156e50 0x7f6f6c155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:14.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f71400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c160b40 con 0x7f6f6c156a70 2024-05-28T14:04:14.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f71400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c0d1010 con 0x7f6f6c156e50 2024-05-28T14:04:14.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f71400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6c160cc0 con 0x7f6f6c0af880 2024-05-28T14:04:14.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f63e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c156a70 0x7f6f6c154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:14.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f63e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c156a70 0x7f6f6c154b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35322/0 (socket says 172.21.15.42:35322) 2024-05-28T14:04:14.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f63e00640 1 -- 172.21.15.42:0/2587514858 learned_addr learned my addr 172.21.15.42:0/2587514858 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:14.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f6b200640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f6c156e50 0x7f6f6c155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:14.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f6a800640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f6c0af880 0x7f6f6c14aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:14.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f63e00640 1 -- 172.21.15.42:0/2587514858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f6c0af880 msgr2=0x7f6f6c14aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:14.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f63e00640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f6c0af880 0x7f6f6c14aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f63e00640 1 -- 172.21.15.42:0/2587514858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f6c156e50 msgr2=0x7f6f6c155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:14.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f63e00640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f6c156e50 0x7f6f6c155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f63e00640 1 -- 172.21.15.42:0/2587514858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c15ada0 con 0x7f6f6c156a70 2024-05-28T14:04:14.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f6a800640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f6c0af880 0x7f6f6c14aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:14.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f63e00640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c156a70 0x7f6f6c154b20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400c7f0 tx=0x7f6f5400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:14.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f62a00640 1 -- 172.21.15.42:0/2587514858 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54015070 con 0x7f6f6c156a70 2024-05-28T14:04:14.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f62a00640 1 -- 172.21.15.42:0/2587514858 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f54007820 con 0x7f6f6c156a70 2024-05-28T14:04:14.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f62a00640 1 -- 172.21.15.42:0/2587514858 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54007c10 con 0x7f6f6c156a70 2024-05-28T14:04:14.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f71400640 1 -- 172.21.15.42:0/2587514858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6c073400 con 0x7f6f6c156a70 2024-05-28T14:04:14.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f71400640 1 -- 172.21.15.42:0/2587514858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6c0739c0 con 0x7f6f6c156a70 2024-05-28T14:04:14.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f71400640 1 -- 172.21.15.42:0/2587514858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6c068a90 con 0x7f6f6c156a70 2024-05-28T14:04:14.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.051+0000 7f6f62a00640 1 -- 172.21.15.42:0/2587514858 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f6f54018070 con 0x7f6f6c156a70 2024-05-28T14:04:14.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.055+0000 7f6f62a00640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f48068b60 0x7f6f4806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:14.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.055+0000 7f6f62a00640 1 -- 172.21.15.42:0/2587514858 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f6f54085510 con 0x7f6f6c156a70 2024-05-28T14:04:14.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.055+0000 7f6f6a800640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f48068b60 0x7f6f4806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:14.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.055+0000 7f6f6a800640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f48068b60 0x7f6f4806b020 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0ae000 tx=0x7f6f5c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:14.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.059+0000 7f6f62a00640 1 -- 172.21.15.42:0/2587514858 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6f540573c0 con 0x7f6f6c156a70 2024-05-28T14:04:14.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.279+0000 7f6f71400640 1 -- 172.21.15.42:0/2587514858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones"]} v 0) -- 0x7f6f6c0b5160 con 0x7f6f6c156a70 2024-05-28T14:04:14.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.511+0000 7f6f62a00640 1 -- 172.21.15.42:0/2587514858 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones"]}]=0 v274) ==== 162+0+0 (secure 0 0 0) 0x7f6f54056cc0 con 0x7f6f6c156a70 2024-05-28T14:04:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.519+0000 7f6f71400640 1 -- 172.21.15.42:0/2587514858 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f48068b60 msgr2=0x7f6f4806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.519+0000 7f6f71400640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f48068b60 0x7f6f4806b020 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0ae000 tx=0x7f6f5c008040 comp rx=0 tx=0).stop 2024-05-28T14:04:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.519+0000 7f6f71400640 1 -- 172.21.15.42:0/2587514858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c156a70 msgr2=0x7f6f6c154b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.519+0000 7f6f71400640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c156a70 0x7f6f6c154b20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400c7f0 tx=0x7f6f5400ccb0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.519+0000 7f6f71400640 1 -- 172.21.15.42:0/2587514858 shutdown_connections 2024-05-28T14:04:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.519+0000 7f6f71400640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f48068b60 0x7f6f4806b020 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.519+0000 7f6f71400640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f6c156e50 0x7f6f6c155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.519+0000 7f6f71400640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c156a70 0x7f6f6c154b20 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.519+0000 7f6f71400640 1 --2- 172.21.15.42:0/2587514858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f6c0af880 0x7f6f6c14aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.519+0000 7f6f71400640 1 -- 172.21.15.42:0/2587514858 >> 172.21.15.42:0/2587514858 conn(0x7f6f6c06bdf0 msgr2=0x7f6f6c166d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.519+0000 7f6f71400640 1 -- 172.21.15.42:0/2587514858 shutdown_connections 2024-05-28T14:04:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.519+0000 7f6f71400640 1 -- 172.21.15.42:0/2587514858 wait complete. 2024-05-28T14:04:14.537 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed when number of clone jobs is ... ok 2024-05-28T14:04:14.537 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bars_for_7_clones (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that 2 progress bars are printed when number of clone jobs is 2024-05-28T14:04:14.538 INFO:tasks.cephfs_test_runner:test_progress_bars_for_7_clones (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-28T14:04:14.538 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:04:14.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.691+0000 7f6189a00640 1 Processor -- start 2024-05-28T14:04:14.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.691+0000 7f6189a00640 1 -- start start 2024-05-28T14:04:14.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6189a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f61840d0e00 0x7f61841663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:14.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6189a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f618415f280 con 0x7f618406f660 2024-05-28T14:04:14.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6189a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f618415f560 con 0x7f618415ba40 2024-05-28T14:04:14.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6189a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f618415f030 con 0x7f61840d0e00 2024-05-28T14:04:14.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f618406f660 0x7f6184073980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47616/0 (socket says 172.21.15.42:47616) 2024-05-28T14:04:14.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182a00640 1 -- 172.21.15.42:0/836377894 learned_addr learned my addr 172.21.15.42:0/836377894 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:14.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6183400640 1 --2- 172.21.15.42:0/836377894 >> v2:172.21.15.63:3300/0 conn(0x7f61840d0e00 0x7f61841663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:14.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 -- 172.21.15.42:0/836377894 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2540841501 0 0) 0x7f618415f280 con 0x7f618406f660 2024-05-28T14:04:14.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 -- 172.21.15.42:0/836377894 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6168005740 con 0x7f618406f660 2024-05-28T14:04:14.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 -- 172.21.15.42:0/836377894 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 30015484 0 0) 0x7f6168005740 con 0x7f618406f660 2024-05-28T14:04:14.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 -- 172.21.15.42:0/836377894 >> v2:172.21.15.63:3300/0 conn(0x7f61840d0e00 msgr2=0x7f61841663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:14.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 --2- 172.21.15.42:0/836377894 >> v2:172.21.15.63:3300/0 conn(0x7f61840d0e00 0x7f61841663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 -- 172.21.15.42:0/836377894 >> v1:172.21.15.63:6789/0 conn(0x7f618415ba40 legacy=0x7f61840d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:14.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 -- 172.21.15.42:0/836377894 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618415f840 con 0x7f618406f660 2024-05-28T14:04:14.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 -- 172.21.15.42:0/836377894 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f6174001680 con 0x7f618406f660 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 -- 172.21.15.42:0/836377894 >> v1:172.21.15.42:6789/0 conn(0x7f618406f660 legacy=0x7f6184073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 --2- 172.21.15.42:0/836377894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6168005bd0 0x7f6168005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 --2- 172.21.15.42:0/836377894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6168006a20 0x7f6168006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 --2- 172.21.15.42:0/836377894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6168007500 0x7f616800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 -- 172.21.15.42:0/836377894 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f618415f030 con 0x7f6168007500 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 -- 172.21.15.42:0/836377894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f616800e3d0 con 0x7f6168005bd0 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182000640 1 -- 172.21.15.42:0/836377894 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f616800e5d0 con 0x7f6168006a20 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6183400640 1 --2- 172.21.15.42:0/836377894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6168007500 0x7f616800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6182a00640 1 --2- 172.21.15.42:0/836377894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6168005bd0 0x7f6168005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f617a000640 1 --2- 172.21.15.42:0/836377894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6168006a20 0x7f6168006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6183400640 1 -- 172.21.15.42:0/836377894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6168006a20 msgr2=0x7f6168006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6183400640 1 --2- 172.21.15.42:0/836377894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6168006a20 0x7f6168006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6183400640 1 -- 172.21.15.42:0/836377894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6168005bd0 msgr2=0x7f6168005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6183400640 1 --2- 172.21.15.42:0/836377894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6168005bd0 0x7f6168005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.695+0000 7f6183400640 1 -- 172.21.15.42:0/836377894 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6168005740 con 0x7f6168007500 2024-05-28T14:04:14.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.699+0000 7f6183400640 1 -- 172.21.15.42:0/836377894 reap_dead start 2024-05-28T14:04:14.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.699+0000 7f6183400640 1 --2- 172.21.15.42:0/836377894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6168007500 0x7f616800dbe0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f617c009ab0 tx=0x7f617c01ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e84e0b6b19dcb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:14.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.699+0000 7f6182000640 1 -- 172.21.15.42:0/836377894 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c0096b0 con 0x7f6168007500 2024-05-28T14:04:14.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.699+0000 7f6182000640 1 -- 172.21.15.42:0/836377894 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f617c004020 con 0x7f6168007500 2024-05-28T14:04:14.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.699+0000 7f6182000640 1 -- 172.21.15.42:0/836377894 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c004410 con 0x7f6168007500 2024-05-28T14:04:14.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.699+0000 7f6189a00640 1 -- 172.21.15.42:0/836377894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6168007500 msgr2=0x7f616800dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:14.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.699+0000 7f6189a00640 1 --2- 172.21.15.42:0/836377894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6168007500 0x7f616800dbe0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f617c009ab0 tx=0x7f617c01ca60 comp rx=0 tx=0).stop 2024-05-28T14:04:14.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.699+0000 7f6189a00640 1 -- 172.21.15.42:0/836377894 shutdown_connections 2024-05-28T14:04:14.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.699+0000 7f6189a00640 1 --2- 172.21.15.42:0/836377894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6168007500 0x7f616800dbe0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.699+0000 7f6189a00640 1 -- 172.21.15.42:0/836377894 >> 172.21.15.42:0/836377894 conn(0x7f618406bdf0 msgr2=0x7f618406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:14.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.699+0000 7f6189a00640 1 -- 172.21.15.42:0/836377894 shutdown_connections 2024-05-28T14:04:14.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.699+0000 7f6189a00640 1 -- 172.21.15.42:0/836377894 wait complete. 2024-05-28T14:04:14.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.699+0000 7f6189a00640 1 Processor -- start 2024-05-28T14:04:14.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.699+0000 7f6189a00640 1 -- start start 2024-05-28T14:04:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f6189a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6168007500 0x7f6184143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f6189a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6184146180 0x7f6184144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f6189a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61841552a0 0x7f6184155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f6189a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61840d0a80 con 0x7f6184146180 2024-05-28T14:04:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f6189a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f61840d0900 con 0x7f6168007500 2024-05-28T14:04:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f6189a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f61840d0c00 con 0x7f61841552a0 2024-05-28T14:04:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f617a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6184146180 0x7f6184144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f617a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6184146180 0x7f6184144dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35348/0 (socket says 172.21.15.42:35348) 2024-05-28T14:04:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f617a000640 1 -- 172.21.15.42:0/3760764559 learned_addr learned my addr 172.21.15.42:0/3760764559 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f6183400640 1 --2- 172.21.15.42:0/3760764559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61841552a0 0x7f6184155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:14.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f6182a00640 1 --2- 172.21.15.42:0/3760764559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6168007500 0x7f6184143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:14.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f617a000640 1 -- 172.21.15.42:0/3760764559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61841552a0 msgr2=0x7f6184155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:14.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f617a000640 1 --2- 172.21.15.42:0/3760764559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61841552a0 0x7f6184155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f617a000640 1 -- 172.21.15.42:0/3760764559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6168007500 msgr2=0x7f6184143670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:14.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f617a000640 1 --2- 172.21.15.42:0/3760764559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6168007500 0x7f6184143670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f617a000640 1 -- 172.21.15.42:0/3760764559 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6184146600 con 0x7f6184146180 2024-05-28T14:04:14.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f617a000640 1 --2- 172.21.15.42:0/3760764559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6184146180 0x7f6184144dd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f616c00e880 tx=0x7f616c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:14.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f617b400640 1 -- 172.21.15.42:0/3760764559 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c003b90 con 0x7f6184146180 2024-05-28T14:04:14.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f617b400640 1 -- 172.21.15.42:0/3760764559 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616c018070 con 0x7f6184146180 2024-05-28T14:04:14.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f617b400640 1 -- 172.21.15.42:0/3760764559 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c0050d0 con 0x7f6184146180 2024-05-28T14:04:14.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f6189a00640 1 -- 172.21.15.42:0/3760764559 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6184143d30 con 0x7f6184146180 2024-05-28T14:04:14.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f6189a00640 1 -- 172.21.15.42:0/3760764559 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61841442f0 con 0x7f6184146180 2024-05-28T14:04:14.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f6189a00640 1 -- 172.21.15.42:0/3760764559 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6184068a90 con 0x7f6184146180 2024-05-28T14:04:14.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.703+0000 7f617b400640 1 -- 172.21.15.42:0/3760764559 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f616c005270 con 0x7f6184146180 2024-05-28T14:04:14.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.707+0000 7f617b400640 1 --2- 172.21.15.42:0/3760764559 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6160068b90 0x7f616006b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:14.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.707+0000 7f617b400640 1 -- 172.21.15.42:0/3760764559 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f616c0877d0 con 0x7f6184146180 2024-05-28T14:04:14.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.707+0000 7f6182a00640 1 --2- 172.21.15.42:0/3760764559 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6160068b90 0x7f616006b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:14.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.707+0000 7f6182a00640 1 --2- 172.21.15.42:0/3760764559 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6160068b90 0x7f616006b050 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6184143540 tx=0x7f6174007900 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:14.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.711+0000 7f617b400640 1 -- 172.21.15.42:0/3760764559 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f616c057050 con 0x7f6184146180 2024-05-28T14:04:14.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.875+0000 7f6189a00640 1 -- 172.21.15.42:0/3760764559 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6184072bb0 con 0x7f6184146180 2024-05-28T14:04:14.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.879+0000 7f617b400640 1 -- 172.21.15.42:0/3760764559 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+13001 (secure 0 0 0) 0x7f616c056950 con 0x7f6184146180 2024-05-28T14:04:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:14.864 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":52,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:04:13.508766+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6833/754542999":"2024-05-29T14:04:10.893617+0000","172.21.15.63:6832/754542999":"2024-05-29T14:04:10.893617+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:04:14.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.879+0000 7f6189a00640 1 -- 172.21.15.42:0/3760764559 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6160068b90 msgr2=0x7f616006b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:14.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.879+0000 7f6189a00640 1 --2- 172.21.15.42:0/3760764559 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6160068b90 0x7f616006b050 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6184143540 tx=0x7f6174007900 comp rx=0 tx=0).stop 2024-05-28T14:04:14.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.879+0000 7f6189a00640 1 -- 172.21.15.42:0/3760764559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6184146180 msgr2=0x7f6184144dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:14.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.879+0000 7f6189a00640 1 --2- 172.21.15.42:0/3760764559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6184146180 0x7f6184144dd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f616c00e880 tx=0x7f616c00ec80 comp rx=0 tx=0).stop 2024-05-28T14:04:14.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.879+0000 7f6189a00640 1 -- 172.21.15.42:0/3760764559 shutdown_connections 2024-05-28T14:04:14.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.879+0000 7f6189a00640 1 --2- 172.21.15.42:0/3760764559 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6160068b90 0x7f616006b050 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.879+0000 7f6189a00640 1 --2- 172.21.15.42:0/3760764559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61841552a0 0x7f6184155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.879+0000 7f6189a00640 1 --2- 172.21.15.42:0/3760764559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6184146180 0x7f6184144dd0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.879+0000 7f6189a00640 1 --2- 172.21.15.42:0/3760764559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6168007500 0x7f6184143670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:14.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.879+0000 7f6189a00640 1 -- 172.21.15.42:0/3760764559 >> 172.21.15.42:0/3760764559 conn(0x7f618406bdf0 msgr2=0x7f61840adfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:14.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.879+0000 7f6189a00640 1 -- 172.21.15.42:0/3760764559 shutdown_connections 2024-05-28T14:04:14.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:14.879+0000 7f6189a00640 1 -- 172.21.15.42:0/3760764559 wait complete. 2024-05-28T14:04:14.885 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:04:14.885 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:04:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.043+0000 7faf36a00640 1 Processor -- start 2024-05-28T14:04:15.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.043+0000 7faf36a00640 1 -- start start 2024-05-28T14:04:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf36a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faf300b0e80 0x7faf3006cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf36a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf3009b8e0 0x7faf3009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf36a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf300d0770 con 0x7faf3009af90 2024-05-28T14:04:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf36a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faf300d0a10 con 0x7faf300b0e80 2024-05-28T14:04:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf36a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faf300d0cf0 con 0x7faf3009b8e0 2024-05-28T14:04:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf34a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7faf3009af90 0x7faf3015e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47628/0 (socket says 172.21.15.42:47628) 2024-05-28T14:04:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf34a00640 1 -- 172.21.15.42:0/2407547198 learned_addr learned my addr 172.21.15.42:0/2407547198 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf2fe00640 1 --2- 172.21.15.42:0/2407547198 >> v2:172.21.15.63:3300/0 conn(0x7faf300b0e80 0x7faf3006cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:15.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf27400640 1 --2- 172.21.15.42:0/2407547198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf3009b8e0 0x7faf3009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:15.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf2f400640 1 -- 172.21.15.42:0/2407547198 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 356499177 0 0) 0x7faf300d0770 con 0x7faf3009af90 2024-05-28T14:04:15.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf2f400640 1 -- 172.21.15.42:0/2407547198 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf14003530 con 0x7faf3009af90 2024-05-28T14:04:15.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf2fe00640 1 -- 172.21.15.42:0/2407547198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf3009b8e0 msgr2=0x7faf3009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:15.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf2fe00640 1 --2- 172.21.15.42:0/2407547198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf3009b8e0 0x7faf3009aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf2fe00640 1 -- 172.21.15.42:0/2407547198 >> v1:172.21.15.42:6789/0 conn(0x7faf3009af90 legacy=0x7faf3015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:15.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.047+0000 7faf2fe00640 1 -- 172.21.15.42:0/2407547198 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf300d0fd0 con 0x7faf300b0e80 2024-05-28T14:04:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf2fe00640 1 --2- 172.21.15.42:0/2407547198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf300b0e80 0x7faf3006cb00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faf200097c0 tx=0x7faf2001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dd70c068b532a6a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf2f400640 1 -- 172.21.15.42:0/2407547198 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf200b0070 con 0x7faf300b0e80 2024-05-28T14:04:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf2f400640 1 -- 172.21.15.42:0/2407547198 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf200ab600 con 0x7faf300b0e80 2024-05-28T14:04:15.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf2f400640 1 -- 172.21.15.42:0/2407547198 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf200ab9b0 con 0x7faf300b0e80 2024-05-28T14:04:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 -- 172.21.15.42:0/2407547198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf300b0e80 msgr2=0x7faf3006cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 --2- 172.21.15.42:0/2407547198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf300b0e80 0x7faf3006cb00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faf200097c0 tx=0x7faf2001c4a0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 -- 172.21.15.42:0/2407547198 shutdown_connections 2024-05-28T14:04:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 --2- 172.21.15.42:0/2407547198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf3009b8e0 0x7faf3009aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 --2- 172.21.15.42:0/2407547198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf300b0e80 0x7faf3006cb00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 -- 172.21.15.42:0/2407547198 >> 172.21.15.42:0/2407547198 conn(0x7faf300b3f90 msgr2=0x7faf300b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 -- 172.21.15.42:0/2407547198 shutdown_connections 2024-05-28T14:04:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 -- 172.21.15.42:0/2407547198 wait complete. 2024-05-28T14:04:15.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 Processor -- start 2024-05-28T14:04:15.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 -- start start 2024-05-28T14:04:15.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf3009af90 0x7faf301574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf3009b8e0 0x7faf3015c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf300b0e80 0x7faf300b6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faf3015ec30 con 0x7faf3009af90 2024-05-28T14:04:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf3009b3a0 con 0x7faf3009b8e0 2024-05-28T14:04:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf36a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faf300af9e0 con 0x7faf300b0e80 2024-05-28T14:04:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.051+0000 7faf2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf3009af90 0x7faf301574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf3009af90 0x7faf301574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35362/0 (socket says 172.21.15.42:35362) 2024-05-28T14:04:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf2fe00640 1 -- 172.21.15.42:0/3722295280 learned_addr learned my addr 172.21.15.42:0/3722295280 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf27400640 1 --2- 172.21.15.42:0/3722295280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf3009b8e0 0x7faf3015c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf34a00640 1 --2- 172.21.15.42:0/3722295280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf300b0e80 0x7faf300b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf2fe00640 1 -- 172.21.15.42:0/3722295280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf300b0e80 msgr2=0x7faf300b6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf2fe00640 1 --2- 172.21.15.42:0/3722295280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf300b0e80 0x7faf300b6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf2fe00640 1 -- 172.21.15.42:0/3722295280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf3009b8e0 msgr2=0x7faf3015c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:15.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf2fe00640 1 --2- 172.21.15.42:0/3722295280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf3009b8e0 0x7faf3015c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf2fe00640 1 -- 172.21.15.42:0/3722295280 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3015df30 con 0x7faf3009af90 2024-05-28T14:04:15.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf2fe00640 1 --2- 172.21.15.42:0/3722295280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf3009af90 0x7faf301574e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7faf2001a0d0 tx=0x7faf20019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:15.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf2cc00640 1 -- 172.21.15.42:0/3722295280 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf200b8020 con 0x7faf3009af90 2024-05-28T14:04:15.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf2cc00640 1 -- 172.21.15.42:0/3722295280 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf200b4410 con 0x7faf3009af90 2024-05-28T14:04:15.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf36a00640 1 -- 172.21.15.42:0/3722295280 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3015e210 con 0x7faf3009af90 2024-05-28T14:04:15.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf2cc00640 1 -- 172.21.15.42:0/3722295280 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf200b0040 con 0x7faf3009af90 2024-05-28T14:04:15.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf36a00640 1 -- 172.21.15.42:0/3722295280 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf3015bc10 con 0x7faf3009af90 2024-05-28T14:04:15.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf36a00640 1 -- 172.21.15.42:0/3722295280 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf300a0550 con 0x7faf3009af90 2024-05-28T14:04:15.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf2cc00640 1 -- 172.21.15.42:0/3722295280 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7faf200bb020 con 0x7faf3009af90 2024-05-28T14:04:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf2cc00640 1 --2- 172.21.15.42:0/3722295280 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf0c068b90 0x7faf0c06b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.055+0000 7faf2cc00640 1 -- 172.21.15.42:0/3722295280 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7faf20121050 con 0x7faf3009af90 2024-05-28T14:04:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.059+0000 7faf27400640 1 --2- 172.21.15.42:0/3722295280 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf0c068b90 0x7faf0c06b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.059+0000 7faf27400640 1 --2- 172.21.15.42:0/3722295280 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf0c068b90 0x7faf0c06b050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faf18009af0 tx=0x7faf18008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:15.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.063+0000 7faf2cc00640 1 -- 172.21.15.42:0/3722295280 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faf200c2030 con 0x7faf3009af90 2024-05-28T14:04:15.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.223+0000 7faf36a00640 1 -- 172.21.15.42:0/3722295280 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faf300b16a0 con 0x7faf3009af90 2024-05-28T14:04:15.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.227+0000 7faf2cc00640 1 -- 172.21.15.42:0/3722295280 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7faf20016090 con 0x7faf3009af90 2024-05-28T14:04:15.212 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:04:15.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.227+0000 7faf36a00640 1 -- 172.21.15.42:0/3722295280 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf0c068b90 msgr2=0x7faf0c06b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:15.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.227+0000 7faf36a00640 1 --2- 172.21.15.42:0/3722295280 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf0c068b90 0x7faf0c06b050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faf18009af0 tx=0x7faf18008040 comp rx=0 tx=0).stop 2024-05-28T14:04:15.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.227+0000 7faf36a00640 1 -- 172.21.15.42:0/3722295280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf3009af90 msgr2=0x7faf301574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:15.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.227+0000 7faf36a00640 1 --2- 172.21.15.42:0/3722295280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf3009af90 0x7faf301574e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7faf2001a0d0 tx=0x7faf20019240 comp rx=0 tx=0).stop 2024-05-28T14:04:15.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.227+0000 7faf36a00640 1 -- 172.21.15.42:0/3722295280 shutdown_connections 2024-05-28T14:04:15.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.227+0000 7faf36a00640 1 --2- 172.21.15.42:0/3722295280 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf0c068b90 0x7faf0c06b050 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.227+0000 7faf36a00640 1 --2- 172.21.15.42:0/3722295280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf300b0e80 0x7faf300b6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.227+0000 7faf36a00640 1 --2- 172.21.15.42:0/3722295280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf3009b8e0 0x7faf3015c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.227+0000 7faf36a00640 1 --2- 172.21.15.42:0/3722295280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf3009af90 0x7faf301574e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.227+0000 7faf36a00640 1 -- 172.21.15.42:0/3722295280 >> 172.21.15.42:0/3722295280 conn(0x7faf300b3f90 msgr2=0x7faf3006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:15.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.227+0000 7faf36a00640 1 -- 172.21.15.42:0/3722295280 shutdown_connections 2024-05-28T14:04:15.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.227+0000 7faf36a00640 1 -- 172.21.15.42:0/3722295280 wait complete. 2024-05-28T14:04:15.233 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones' 2024-05-28T14:04:15.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.387+0000 7fb910000640 1 Processor -- start 2024-05-28T14:04:15.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.387+0000 7fb910000640 1 -- start start 2024-05-28T14:04:15.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb910000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb90815ba40 0x7fb9080d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:15.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb910000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb908072dd0 0x7fb9080d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:15.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb910000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb90815f280 con 0x7fb90806f660 2024-05-28T14:04:15.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb910000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb90815f030 con 0x7fb908072dd0 2024-05-28T14:04:15.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb910000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb90815f560 con 0x7fb90815ba40 2024-05-28T14:04:15.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb90cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb90815ba40 0x7fb9080d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:15.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb90e000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb908072dd0 0x7fb9080d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:15.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb90e000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb908072dd0 0x7fb9080d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:59432/0 (socket says 172.21.15.42:59432) 2024-05-28T14:04:15.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb90cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb90815ba40 0x7fb9080d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:40778/0 (socket says 172.21.15.42:40778) 2024-05-28T14:04:15.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb90e000640 1 -- 172.21.15.42:0/2282603365 learned_addr learned my addr 172.21.15.42:0/2282603365 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:15.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb90cc00640 1 -- 172.21.15.42:0/2282603365 >> v2:172.21.15.63:3300/0 conn(0x7fb908072dd0 msgr2=0x7fb9080d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:15.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb90cc00640 1 --2- 172.21.15.42:0/2282603365 >> v2:172.21.15.63:3300/0 conn(0x7fb908072dd0 0x7fb9080d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb90cc00640 1 -- 172.21.15.42:0/2282603365 >> v1:172.21.15.63:6789/0 conn(0x7fb90806f660 legacy=0x7fb908073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:15.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb90cc00640 1 -- 172.21.15.42:0/2282603365 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90815f840 con 0x7fb90815ba40 2024-05-28T14:04:15.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb90e000640 1 --2- 172.21.15.42:0/2282603365 >> v2:172.21.15.63:3300/0 conn(0x7fb908072dd0 0x7fb9080d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:04:15.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.391+0000 7fb907e00640 1 -- 172.21.15.42:0/2282603365 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 436296570 0 0) 0x7fb90815f280 con 0x7fb90806f660 2024-05-28T14:04:15.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb90cc00640 1 --2- 172.21.15.42:0/2282603365 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb90815ba40 0x7fb9080d06f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40097c0 tx=0x7fb8f401c030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c820475e5ddd18f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb907e00640 1 -- 172.21.15.42:0/2282603365 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40b1070 con 0x7fb90815ba40 2024-05-28T14:04:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb907e00640 1 -- 172.21.15.42:0/2282603365 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f40a4870 con 0x7fb90815ba40 2024-05-28T14:04:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb907e00640 1 -- 172.21.15.42:0/2282603365 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40a4c40 con 0x7fb90815ba40 2024-05-28T14:04:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 -- 172.21.15.42:0/2282603365 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb90815ba40 msgr2=0x7fb9080d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 --2- 172.21.15.42:0/2282603365 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb90815ba40 0x7fb9080d06f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40097c0 tx=0x7fb8f401c030 comp rx=0 tx=0).stop 2024-05-28T14:04:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 -- 172.21.15.42:0/2282603365 shutdown_connections 2024-05-28T14:04:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 --2- 172.21.15.42:0/2282603365 >> v2:172.21.15.63:3300/0 conn(0x7fb908072dd0 0x7fb9080d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 --2- 172.21.15.42:0/2282603365 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb90815ba40 0x7fb9080d06f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 -- 172.21.15.42:0/2282603365 >> 172.21.15.42:0/2282603365 conn(0x7fb90806bdf0 msgr2=0x7fb90806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 -- 172.21.15.42:0/2282603365 shutdown_connections 2024-05-28T14:04:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 -- 172.21.15.42:0/2282603365 wait complete. 2024-05-28T14:04:15.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 Processor -- start 2024-05-28T14:04:15.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 -- start start 2024-05-28T14:04:15.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb90806f660 0x7fb908154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:15.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908072dd0 0x7fb908099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:15.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9080b1170 0x7fb90815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:15.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9080b3670 con 0x7fb908072dd0 2024-05-28T14:04:15.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb9080b34f0 con 0x7fb9080b1170 2024-05-28T14:04:15.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.395+0000 7fb910000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb9080b37f0 con 0x7fb90806f660 2024-05-28T14:04:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb90cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908072dd0 0x7fb908099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb90cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908072dd0 0x7fb908099920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35370/0 (socket says 172.21.15.42:35370) 2024-05-28T14:04:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb90cc00640 1 -- 172.21.15.42:0/1691091977 learned_addr learned my addr 172.21.15.42:0/1691091977 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb90e000640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9080b1170 0x7fb90815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb90d600640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb90806f660 0x7fb908154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb90cc00640 1 -- 172.21.15.42:0/1691091977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb90806f660 msgr2=0x7fb908154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb90cc00640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb90806f660 0x7fb908154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb90cc00640 1 -- 172.21.15.42:0/1691091977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9080b1170 msgr2=0x7fb90815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb90cc00640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9080b1170 0x7fb90815aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb90cc00640 1 -- 172.21.15.42:0/1691091977 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb908153220 con 0x7fb908072dd0 2024-05-28T14:04:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb90e000640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9080b1170 0x7fb90815aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:15.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb90d600640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb90806f660 0x7fb908154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:15.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb90cc00640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908072dd0 0x7fb908099920 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40af040 tx=0x7fb8f40190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:15.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb905600640 1 -- 172.21.15.42:0/1691091977 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40b1040 con 0x7fb908072dd0 2024-05-28T14:04:15.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb905600640 1 -- 172.21.15.42:0/1691091977 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f40ac870 con 0x7fb908072dd0 2024-05-28T14:04:15.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb905600640 1 -- 172.21.15.42:0/1691091977 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40acc80 con 0x7fb908072dd0 2024-05-28T14:04:15.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb910000640 1 -- 172.21.15.42:0/1691091977 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb908155440 con 0x7fb908072dd0 2024-05-28T14:04:15.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb910000640 1 -- 172.21.15.42:0/1691091977 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9080ce3c0 con 0x7fb908072dd0 2024-05-28T14:04:15.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb910000640 1 -- 172.21.15.42:0/1691091977 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9081533f0 con 0x7fb908072dd0 2024-05-28T14:04:15.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.399+0000 7fb905600640 1 -- 172.21.15.42:0/1691091977 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fb8f40be400 con 0x7fb908072dd0 2024-05-28T14:04:15.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.403+0000 7fb905600640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb8e8068b60 0x7fb8e806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:15.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.403+0000 7fb905600640 1 -- 172.21.15.42:0/1691091977 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7fb8f41209a0 con 0x7fb908072dd0 2024-05-28T14:04:15.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.403+0000 7fb90d600640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb8e8068b60 0x7fb8e806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:15.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.403+0000 7fb90d600640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb8e8068b60 0x7fb8e806b020 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc000f70 tx=0x7fb8fc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:15.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.407+0000 7fb905600640 1 -- 172.21.15.42:0/1691091977 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb8f4016090 con 0x7fb908072dd0 2024-05-28T14:04:15.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:15.631+0000 7fb910000640 1 -- 172.21.15.42:0/1691091977 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones"]} v 0) -- 0x7fb9081661a0 con 0x7fb908072dd0 2024-05-28T14:04:16.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.519+0000 7fb905600640 1 -- 172.21.15.42:0/1691091977 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones"]}]=0 v276) ==== 165+0+0 (secure 0 0 0) 0x7fb8f40b32f0 con 0x7fb908072dd0 2024-05-28T14:04:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.523+0000 7fb910000640 1 -- 172.21.15.42:0/1691091977 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb8e8068b60 msgr2=0x7fb8e806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.523+0000 7fb910000640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb8e8068b60 0x7fb8e806b020 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc000f70 tx=0x7fb8fc009040 comp rx=0 tx=0).stop 2024-05-28T14:04:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.523+0000 7fb910000640 1 -- 172.21.15.42:0/1691091977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908072dd0 msgr2=0x7fb908099920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.523+0000 7fb910000640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908072dd0 0x7fb908099920 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40af040 tx=0x7fb8f40190f0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.523+0000 7fb910000640 1 -- 172.21.15.42:0/1691091977 shutdown_connections 2024-05-28T14:04:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.523+0000 7fb910000640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb8e8068b60 0x7fb8e806b020 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.523+0000 7fb910000640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9080b1170 0x7fb90815aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.523+0000 7fb910000640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908072dd0 0x7fb908099920 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.523+0000 7fb910000640 1 --2- 172.21.15.42:0/1691091977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb90806f660 0x7fb908154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.523+0000 7fb910000640 1 -- 172.21.15.42:0/1691091977 >> 172.21.15.42:0/1691091977 conn(0x7fb90806bdf0 msgr2=0x7fb9080cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.523+0000 7fb910000640 1 -- 172.21.15.42:0/1691091977 shutdown_connections 2024-05-28T14:04:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.523+0000 7fb910000640 1 -- 172.21.15.42:0/1691091977 wait complete. 2024-05-28T14:04:16.534 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T14:04:16.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.695+0000 7fea9da00640 1 Processor -- start 2024-05-28T14:04:16.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.695+0000 7fea9da00640 1 -- start start 2024-05-28T14:04:16.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.699+0000 7fea9da00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea98071de0 0x7fea980721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:16.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.699+0000 7fea9da00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fea980ce3b0 0x7fea980ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:16.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.699+0000 7fea9da00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea98072700 con 0x7fea98141d10 2024-05-28T14:04:16.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.699+0000 7fea9da00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fea98160230 con 0x7fea980ce3b0 2024-05-28T14:04:16.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.699+0000 7fea9da00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea98160510 con 0x7fea98071de0 2024-05-28T14:04:16.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.699+0000 7fea8e000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fea980ce3b0 0x7fea980ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:16.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.699+0000 7fea8e000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fea980ce3b0 0x7fea980ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35372/0 (socket says 172.21.15.42:35372) 2024-05-28T14:04:16.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.699+0000 7fea8e000640 1 -- 172.21.15.42:0/206794387 learned_addr learned my addr 172.21.15.42:0/206794387 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:16.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.699+0000 7fea96a00640 1 --2- 172.21.15.42:0/206794387 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea98071de0 0x7fea980721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:16.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.699+0000 7fea8e000640 1 -- 172.21.15.42:0/206794387 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea98071de0 msgr2=0x7fea980721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:16.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.699+0000 7fea8e000640 1 --2- 172.21.15.42:0/206794387 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea98071de0 0x7fea980721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.699+0000 7fea8e000640 1 -- 172.21.15.42:0/206794387 >> v1:172.21.15.63:6789/0 conn(0x7fea98141d10 legacy=0x7fea981421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:16.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.699+0000 7fea8e000640 1 -- 172.21.15.42:0/206794387 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea981607f0 con 0x7fea980ce3b0 2024-05-28T14:04:16.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.699+0000 7fea96a00640 1 --2- 172.21.15.42:0/206794387 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea98071de0 0x7fea980721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:16.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.703+0000 7fea8e000640 1 --2- 172.21.15.42:0/206794387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea980ce3b0 0x7fea980ce810 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fea80009870 tx=0x7fea8001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3958ea37151775 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:16.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.703+0000 7fea96000640 1 -- 172.21.15.42:0/206794387 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800b0070 con 0x7fea980ce3b0 2024-05-28T14:04:16.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.703+0000 7fea96000640 1 -- 172.21.15.42:0/206794387 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea800a4e50 con 0x7fea980ce3b0 2024-05-28T14:04:16.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.703+0000 7fea96000640 1 -- 172.21.15.42:0/206794387 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800ab820 con 0x7fea980ce3b0 2024-05-28T14:04:16.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.703+0000 7fea9da00640 1 -- 172.21.15.42:0/206794387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea980ce3b0 msgr2=0x7fea980ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:16.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.703+0000 7fea9da00640 1 --2- 172.21.15.42:0/206794387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea980ce3b0 0x7fea980ce810 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fea80009870 tx=0x7fea8001c400 comp rx=0 tx=0).stop 2024-05-28T14:04:16.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.703+0000 7fea9da00640 1 -- 172.21.15.42:0/206794387 shutdown_connections 2024-05-28T14:04:16.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.703+0000 7fea9da00640 1 --2- 172.21.15.42:0/206794387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea980ce3b0 0x7fea980ce810 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.703+0000 7fea9da00640 1 --2- 172.21.15.42:0/206794387 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea98071de0 0x7fea980721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.703+0000 7fea9da00640 1 -- 172.21.15.42:0/206794387 >> 172.21.15.42:0/206794387 conn(0x7fea980b21f0 msgr2=0x7fea980b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:16.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.703+0000 7fea9da00640 1 -- 172.21.15.42:0/206794387 shutdown_connections 2024-05-28T14:04:16.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.703+0000 7fea9da00640 1 -- 172.21.15.42:0/206794387 wait complete. 2024-05-28T14:04:16.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.703+0000 7fea9da00640 1 Processor -- start 2024-05-28T14:04:16.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.703+0000 7fea9da00640 1 -- start start 2024-05-28T14:04:16.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea9da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea98071de0 0x7fea9806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:16.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea9da00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea980ce3b0 0x7fea980b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:16.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea9da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea98141d10 0x7fea9815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:16.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea9da00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea98169e30 con 0x7fea98071de0 2024-05-28T14:04:16.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea9da00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea98169cb0 con 0x7fea98141d10 2024-05-28T14:04:16.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea9da00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea98169fb0 con 0x7fea980ce3b0 2024-05-28T14:04:16.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea96a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea98071de0 0x7fea9806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:16.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea96a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea98071de0 0x7fea9806eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35388/0 (socket says 172.21.15.42:35388) 2024-05-28T14:04:16.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea96a00640 1 -- 172.21.15.42:0/4045059170 learned_addr learned my addr 172.21.15.42:0/4045059170 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:16.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea8e000640 1 --2- 172.21.15.42:0/4045059170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea980ce3b0 0x7fea980b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:16.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea97400640 1 --2- 172.21.15.42:0/4045059170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea98141d10 0x7fea9815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:16.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea96a00640 1 -- 172.21.15.42:0/4045059170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea980ce3b0 msgr2=0x7fea980b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:16.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea96a00640 1 --2- 172.21.15.42:0/4045059170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea980ce3b0 0x7fea980b4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea96a00640 1 -- 172.21.15.42:0/4045059170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea98141d10 msgr2=0x7fea9815b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:16.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea96a00640 1 --2- 172.21.15.42:0/4045059170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea98141d10 0x7fea9815b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea96a00640 1 -- 172.21.15.42:0/4045059170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea980b39b0 con 0x7fea98071de0 2024-05-28T14:04:16.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea96a00640 1 --2- 172.21.15.42:0/4045059170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea98071de0 0x7fea9806eee0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fea8800c820 tx=0x7fea8800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:16.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea8f400640 1 -- 172.21.15.42:0/4045059170 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88015070 con 0x7fea98071de0 2024-05-28T14:04:16.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea8f400640 1 -- 172.21.15.42:0/4045059170 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea88007a30 con 0x7fea98071de0 2024-05-28T14:04:16.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea8f400640 1 -- 172.21.15.42:0/4045059170 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88007e40 con 0x7fea98071de0 2024-05-28T14:04:16.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea9da00640 1 -- 172.21.15.42:0/4045059170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea98139720 con 0x7fea98071de0 2024-05-28T14:04:16.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea9da00640 1 -- 172.21.15.42:0/4045059170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9806e6e0 con 0x7fea98071de0 2024-05-28T14:04:16.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea9da00640 1 -- 172.21.15.42:0/4045059170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea980738b0 con 0x7fea98071de0 2024-05-28T14:04:16.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.707+0000 7fea8f400640 1 -- 172.21.15.42:0/4045059170 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fea88018070 con 0x7fea98071de0 2024-05-28T14:04:16.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.711+0000 7fea8f400640 1 --2- 172.21.15.42:0/4045059170 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea74068b60 0x7fea7406b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:16.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.711+0000 7fea8f400640 1 -- 172.21.15.42:0/4045059170 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7fea88087870 con 0x7fea98071de0 2024-05-28T14:04:16.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.711+0000 7fea8e000640 1 --2- 172.21.15.42:0/4045059170 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea74068b60 0x7fea7406b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:16.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.711+0000 7fea8e000640 1 --2- 172.21.15.42:0/4045059170 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea74068b60 0x7fea7406b020 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fea800ad000 tx=0x7fea800a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:16.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.715+0000 7fea8f400640 1 -- 172.21.15.42:0/4045059170 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fea88056a20 con 0x7fea98071de0 2024-05-28T14:04:16.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.879+0000 7fea9da00640 1 -- 172.21.15.42:0/4045059170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fea9804fdd0 con 0x7fea98071de0 2024-05-28T14:04:16.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.883+0000 7fea8f400640 1 -- 172.21.15.42:0/4045059170 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fea8805ad20 con 0x7fea98071de0 2024-05-28T14:04:16.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.887+0000 7fea9da00640 1 -- 172.21.15.42:0/4045059170 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea74068b60 msgr2=0x7fea7406b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:16.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.887+0000 7fea9da00640 1 --2- 172.21.15.42:0/4045059170 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea74068b60 0x7fea7406b020 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fea800ad000 tx=0x7fea800a6000 comp rx=0 tx=0).stop 2024-05-28T14:04:16.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.887+0000 7fea9da00640 1 -- 172.21.15.42:0/4045059170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea98071de0 msgr2=0x7fea9806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:16.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.887+0000 7fea9da00640 1 --2- 172.21.15.42:0/4045059170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea98071de0 0x7fea9806eee0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fea8800c820 tx=0x7fea8800cce0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.887+0000 7fea9da00640 1 -- 172.21.15.42:0/4045059170 shutdown_connections 2024-05-28T14:04:16.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.887+0000 7fea9da00640 1 --2- 172.21.15.42:0/4045059170 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea74068b60 0x7fea7406b020 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.887+0000 7fea9da00640 1 --2- 172.21.15.42:0/4045059170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea98141d10 0x7fea9815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.887+0000 7fea9da00640 1 --2- 172.21.15.42:0/4045059170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea980ce3b0 0x7fea980b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.887+0000 7fea9da00640 1 --2- 172.21.15.42:0/4045059170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea98071de0 0x7fea9806eee0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:16.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.887+0000 7fea9da00640 1 -- 172.21.15.42:0/4045059170 >> 172.21.15.42:0/4045059170 conn(0x7fea980b21f0 msgr2=0x7fea980aeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:16.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.887+0000 7fea9da00640 1 -- 172.21.15.42:0/4045059170 shutdown_connections 2024-05-28T14:04:16.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:16.887+0000 7fea9da00640 1 -- 172.21.15.42:0/4045059170 wait complete. 2024-05-28T14:04:16.893 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T14:04:16.894 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:04:16.986 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:04:17.093 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T14:04:17.093 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:04:17.093 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:04:17.102 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:04:17.102 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:04:17.102 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:04:17.111 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:04:17.112 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:04:17.112 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:04:17.151 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:04:17.151 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:04:17.151 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:04:17.162 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:04:17.162 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-28T14:04:17.162 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:04:17.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.355+0000 7f2479a00640 1 Processor -- start 2024-05-28T14:04:17.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.355+0000 7f2479a00640 1 -- start start 2024-05-28T14:04:17.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.355+0000 7f2479a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f247415c880 0x7f247409b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:17.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.355+0000 7f2479a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f247415d680 0x7f24740b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:17.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.355+0000 7f2479a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24740d0160 con 0x7f24740d0450 2024-05-28T14:04:17.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.355+0000 7f2479a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f247409b520 con 0x7f247415c880 2024-05-28T14:04:17.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.355+0000 7f2479a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f247409abf0 con 0x7f247415d680 2024-05-28T14:04:17.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.355+0000 7f246b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f247415c880 0x7f247409b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:17.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.355+0000 7f2473e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f24740d0450 0x7f24740d0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47644/0 (socket says 172.21.15.42:47644) 2024-05-28T14:04:17.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.355+0000 7f2473e00640 1 -- 172.21.15.42:0/2366271601 learned_addr learned my addr 172.21.15.42:0/2366271601 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:17.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.355+0000 7f2479000640 1 --2- 172.21.15.42:0/2366271601 >> v2:172.21.15.63:3300/0 conn(0x7f247415d680 0x7f24740b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f246b400640 1 -- 172.21.15.42:0/2366271601 >> v2:172.21.15.63:3300/0 conn(0x7f247415d680 msgr2=0x7f24740b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f246b400640 1 --2- 172.21.15.42:0/2366271601 >> v2:172.21.15.63:3300/0 conn(0x7f247415d680 0x7f24740b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f246b400640 1 -- 172.21.15.42:0/2366271601 >> v1:172.21.15.42:6789/0 conn(0x7f24740d0450 legacy=0x7f24740d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f246b400640 1 -- 172.21.15.42:0/2366271601 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247409aed0 con 0x7f247415c880 2024-05-28T14:04:17.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f246b400640 1 --2- 172.21.15.42:0/2366271601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f247415c880 0x7f247409b9b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2474073550 tx=0x7f245c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ad7a6d73cc1720 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:17.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f2473400640 1 -- 172.21.15.42:0/2366271601 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c0b0070 con 0x7f247415c880 2024-05-28T14:04:17.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f2473400640 1 -- 172.21.15.42:0/2366271601 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f245c0ab600 con 0x7f247415c880 2024-05-28T14:04:17.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f2473400640 1 -- 172.21.15.42:0/2366271601 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c0ab9d0 con 0x7f247415c880 2024-05-28T14:04:17.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f2479a00640 1 -- 172.21.15.42:0/2366271601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f247415c880 msgr2=0x7f247409b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:17.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f2479a00640 1 --2- 172.21.15.42:0/2366271601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f247415c880 0x7f247409b9b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2474073550 tx=0x7f245c01c3f0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f2479a00640 1 -- 172.21.15.42:0/2366271601 shutdown_connections 2024-05-28T14:04:17.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f2479a00640 1 --2- 172.21.15.42:0/2366271601 >> v2:172.21.15.63:3300/0 conn(0x7f247415d680 0x7f24740b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f2479a00640 1 --2- 172.21.15.42:0/2366271601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f247415c880 0x7f247409b9b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f2479a00640 1 -- 172.21.15.42:0/2366271601 >> 172.21.15.42:0/2366271601 conn(0x7f24740b3dd0 msgr2=0x7f24740b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:17.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f2479a00640 1 -- 172.21.15.42:0/2366271601 shutdown_connections 2024-05-28T14:04:17.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.359+0000 7f2479a00640 1 -- 172.21.15.42:0/2366271601 wait complete. 2024-05-28T14:04:17.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f2479a00640 1 Processor -- start 2024-05-28T14:04:17.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f2479a00640 1 -- start start 2024-05-28T14:04:17.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f2479a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24740d0450 0x7f24741fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:17.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f2479a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f247415c880 0x7f24740c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:17.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f2479a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f247415d680 0x7f247406e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:17.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f2479a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f24740cd990 con 0x7f247415c880 2024-05-28T14:04:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f2479a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24740cd810 con 0x7f247415d680 2024-05-28T14:04:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f2479a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f24740cdb10 con 0x7f24740d0450 2024-05-28T14:04:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f246b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f247415c880 0x7f24740c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f246b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f247415c880 0x7f24740c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35418/0 (socket says 172.21.15.42:35418) 2024-05-28T14:04:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f246b400640 1 -- 172.21.15.42:0/4168115819 learned_addr learned my addr 172.21.15.42:0/4168115819 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f2473e00640 1 --2- 172.21.15.42:0/4168115819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24740d0450 0x7f24741fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f2479000640 1 --2- 172.21.15.42:0/4168115819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f247415d680 0x7f247406e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f246b400640 1 -- 172.21.15.42:0/4168115819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24740d0450 msgr2=0x7f24741fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f246b400640 1 --2- 172.21.15.42:0/4168115819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24740d0450 0x7f24741fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f246b400640 1 -- 172.21.15.42:0/4168115819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f247415d680 msgr2=0x7f247406e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f246b400640 1 --2- 172.21.15.42:0/4168115819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f247415d680 0x7f247406e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f246b400640 1 -- 172.21.15.42:0/4168115819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247406eab0 con 0x7f247415c880 2024-05-28T14:04:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.363+0000 7f246b400640 1 --2- 172.21.15.42:0/4168115819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f247415c880 0x7f24740c7440 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f245c0ae040 tx=0x7f245c019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.367+0000 7f2470c00640 1 -- 172.21.15.42:0/4168115819 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c0b8020 con 0x7f247415c880 2024-05-28T14:04:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.367+0000 7f2470c00640 1 -- 172.21.15.42:0/4168115819 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f245c0b4610 con 0x7f247415c880 2024-05-28T14:04:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.367+0000 7f2470c00640 1 -- 172.21.15.42:0/4168115819 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c0b0040 con 0x7f247415c880 2024-05-28T14:04:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.367+0000 7f2479a00640 1 -- 172.21.15.42:0/4168115819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24740c8c50 con 0x7f247415c880 2024-05-28T14:04:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.367+0000 7f2479a00640 1 -- 172.21.15.42:0/4168115819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24740c7e90 con 0x7f247415c880 2024-05-28T14:04:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.367+0000 7f2479a00640 1 -- 172.21.15.42:0/4168115819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f247404fcf0 con 0x7f247415c880 2024-05-28T14:04:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.367+0000 7f2470c00640 1 -- 172.21.15.42:0/4168115819 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f245c0bb020 con 0x7f247415c880 2024-05-28T14:04:17.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.367+0000 7f2470c00640 1 --2- 172.21.15.42:0/4168115819 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2450068b60 0x7f245006b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:17.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.367+0000 7f2470c00640 1 -- 172.21.15.42:0/4168115819 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f245c1211c0 con 0x7f247415c880 2024-05-28T14:04:17.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.367+0000 7f2473e00640 1 --2- 172.21.15.42:0/4168115819 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2450068b60 0x7f245006b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:17.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.367+0000 7f2473e00640 1 --2- 172.21.15.42:0/4168115819 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2450068b60 0x7f245006b020 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2464000f70 tx=0x7f2464007e90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:17.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.371+0000 7f2470c00640 1 -- 172.21.15.42:0/4168115819 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f245c0c2030 con 0x7f247415c880 2024-05-28T14:04:17.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.571+0000 7f2479a00640 1 -- 172.21.15.42:0/4168115819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24740ce780 con 0x7f247415c880 2024-05-28T14:04:17.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.571+0000 7f2470c00640 1 -- 172.21.15.42:0/4168115819 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+3867 (secure 0 0 0) 0x7f245c016070 con 0x7f247415c880 2024-05-28T14:04:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:17.559 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3694688875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3694688875},{"type":"v1","addr":"172.21.15.63:6835","nonce":3694688875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5799,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/199639725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":199639725},{"type":"v1","addr":"172.21.15.42:6836","nonce":199639725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1043559054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1043559054},{"type":"v1","addr":"172.21.15.42:6837","nonce":1043559054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":55532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/2086860026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":2086860026},{"type":"v1","addr":"172.21.15.63:6833","nonce":2086860026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[]} 2024-05-28T14:04:17.559 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 30 2024-05-28T14:04:17.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.575+0000 7f2479a00640 1 -- 172.21.15.42:0/4168115819 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2450068b60 msgr2=0x7f245006b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:17.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.575+0000 7f2479a00640 1 --2- 172.21.15.42:0/4168115819 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2450068b60 0x7f245006b020 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2464000f70 tx=0x7f2464007e90 comp rx=0 tx=0).stop 2024-05-28T14:04:17.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.575+0000 7f2479a00640 1 -- 172.21.15.42:0/4168115819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f247415c880 msgr2=0x7f24740c7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:17.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.575+0000 7f2479a00640 1 --2- 172.21.15.42:0/4168115819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f247415c880 0x7f24740c7440 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f245c0ae040 tx=0x7f245c019190 comp rx=0 tx=0).stop 2024-05-28T14:04:17.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.575+0000 7f2479a00640 1 -- 172.21.15.42:0/4168115819 shutdown_connections 2024-05-28T14:04:17.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.575+0000 7f2479a00640 1 --2- 172.21.15.42:0/4168115819 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2450068b60 0x7f245006b020 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.575+0000 7f2479a00640 1 --2- 172.21.15.42:0/4168115819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f247415d680 0x7f247406e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.575+0000 7f2479a00640 1 --2- 172.21.15.42:0/4168115819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f247415c880 0x7f24740c7440 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.575+0000 7f2479a00640 1 --2- 172.21.15.42:0/4168115819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24740d0450 0x7f24741fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.575+0000 7f2479a00640 1 -- 172.21.15.42:0/4168115819 >> 172.21.15.42:0/4168115819 conn(0x7f24740b3dd0 msgr2=0x7f24740749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:17.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.575+0000 7f2479a00640 1 -- 172.21.15.42:0/4168115819 shutdown_connections 2024-05-28T14:04:17.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.575+0000 7f2479a00640 1 -- 172.21.15.42:0/4168115819 wait complete. 2024-05-28T14:04:17.581 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-28T14:04:17.581 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-28T14:04:17.581 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-28T14:04:17.581 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:04:17.581 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-28T14:04:17.581 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-28T14:04:17.582 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-28T14:04:17.582 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:04:17.582 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-28T14:04:17.582 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-28T14:04:17.582 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-28T14:04:17.582 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:04:17.583 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-28T14:04:17.583 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-28T14:04:17.583 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-28T14:04:17.583 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:04:17.596 INFO:tasks.ceph.mds.c:Stopped 2024-05-28T14:04:17.596 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-28T14:04:17.597 INFO:tasks.ceph.mds.a:Stopped 2024-05-28T14:04:17.597 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-28T14:04:17.599 INFO:tasks.ceph.mds.b:Stopped 2024-05-28T14:04:17.600 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-28T14:04:17.600 INFO:tasks.ceph.mds.d:Stopped 2024-05-28T14:04:17.601 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-28T14:04:17.642 INFO:tasks.ceph.mds.c:Started 2024-05-28T14:04:17.643 DEBUG:teuthology.parallel:result is None 2024-05-28T14:04:17.643 INFO:tasks.ceph.mds.a:Started 2024-05-28T14:04:17.643 DEBUG:teuthology.parallel:result is None 2024-05-28T14:04:17.644 INFO:tasks.ceph.mds.b:Started 2024-05-28T14:04:17.644 DEBUG:teuthology.parallel:result is None 2024-05-28T14:04:17.646 INFO:tasks.ceph.mds.d:Started 2024-05-28T14:04:17.646 DEBUG:teuthology.parallel:result is None 2024-05-28T14:04:17.647 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-28T14:04:17.728 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-05-28T14:04:17.729 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-05-28T14:04:17.737 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-28T14:04:17.737 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T14:04:17.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87bc800640 1 Processor -- start 2024-05-28T14:04:17.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87bc800640 1 -- start start 2024-05-28T14:04:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87bc800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f87b415fdf0 0x7f87b40cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87bc800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87b40d0ea0 0x7f87b40b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87bc800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87b40ce400 con 0x7f87b409baf0 2024-05-28T14:04:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87bc800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f87b40ce6e0 con 0x7f87b415fdf0 2024-05-28T14:04:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87bc800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f87b40b1ea0 con 0x7f87b40d0ea0 2024-05-28T14:04:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87b9e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f87b415fdf0 0x7f87b40cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87b9e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f87b415fdf0 0x7f87b40cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34338/0 (socket says 172.21.15.42:34338) 2024-05-28T14:04:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87b9e00640 1 -- 172.21.15.42:0/1349833026 learned_addr learned my addr 172.21.15.42:0/1349833026 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87ba800640 1 --2- 172.21.15.42:0/1349833026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87b40d0ea0 0x7f87b40b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87b9e00640 1 -- 172.21.15.42:0/1349833026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87b40d0ea0 msgr2=0x7f87b40b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87b9e00640 1 --2- 172.21.15.42:0/1349833026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87b40d0ea0 0x7f87b40b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87b9e00640 1 -- 172.21.15.42:0/1349833026 >> v1:172.21.15.63:6789/0 conn(0x7f87b409baf0 legacy=0x7f87b40d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:17.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.807+0000 7f87b9e00640 1 -- 172.21.15.42:0/1349833026 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b40b2180 con 0x7f87b415fdf0 2024-05-28T14:04:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.811+0000 7f87b9e00640 1 --2- 172.21.15.42:0/1349833026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b415fdf0 0x7f87b40cdbd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009870 tx=0x7f87a801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9956e138485f6caa server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.811+0000 7f87b8a00640 1 -- 172.21.15.42:0/1349833026 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a80aa620 con 0x7f87b415fdf0 2024-05-28T14:04:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.811+0000 7f87b8a00640 1 -- 172.21.15.42:0/1349833026 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a80af070 con 0x7f87b415fdf0 2024-05-28T14:04:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.811+0000 7f87b8a00640 1 -- 172.21.15.42:0/1349833026 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a80aa9e0 con 0x7f87b415fdf0 2024-05-28T14:04:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.811+0000 7f87bc800640 1 -- 172.21.15.42:0/1349833026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b415fdf0 msgr2=0x7f87b40cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.811+0000 7f87bc800640 1 --2- 172.21.15.42:0/1349833026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b415fdf0 0x7f87b40cdbd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009870 tx=0x7f87a801c4a0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.811+0000 7f87bc800640 1 -- 172.21.15.42:0/1349833026 shutdown_connections 2024-05-28T14:04:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.811+0000 7f87bc800640 1 --2- 172.21.15.42:0/1349833026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87b40d0ea0 0x7f87b40b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.811+0000 7f87bc800640 1 --2- 172.21.15.42:0/1349833026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b415fdf0 0x7f87b40cdbd0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.811+0000 7f87bc800640 1 -- 172.21.15.42:0/1349833026 >> 172.21.15.42:0/1349833026 conn(0x7f87b40b4050 msgr2=0x7f87b40b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.811+0000 7f87bc800640 1 -- 172.21.15.42:0/1349833026 shutdown_connections 2024-05-28T14:04:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.811+0000 7f87bc800640 1 -- 172.21.15.42:0/1349833026 wait complete. 2024-05-28T14:04:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87bc800640 1 Processor -- start 2024-05-28T14:04:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87bc800640 1 -- start start 2024-05-28T14:04:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87bc800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87b409baf0 0x7f87b4154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87bc800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40d0ea0 0x7f87b4157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87bc800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b415fdf0 0x7f87b4157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87bc800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f87b40b3830 con 0x7f87b40d0ea0 2024-05-28T14:04:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87bc800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f87b40b36b0 con 0x7f87b415fdf0 2024-05-28T14:04:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87bc800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f87b40b39b0 con 0x7f87b409baf0 2024-05-28T14:04:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87b9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40d0ea0 0x7f87b4157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87b9e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87b409baf0 0x7f87b4154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87b9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40d0ea0 0x7f87b4157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34350/0 (socket says 172.21.15.42:34350) 2024-05-28T14:04:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87b9400640 1 -- 172.21.15.42:0/2118696052 learned_addr learned my addr 172.21.15.42:0/2118696052 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87b9e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87b409baf0 0x7f87b4154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:49052/0 (socket says 172.21.15.42:49052) 2024-05-28T14:04:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87ba800640 1 --2- 172.21.15.42:0/2118696052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b415fdf0 0x7f87b4157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87b9400640 1 -- 172.21.15.42:0/2118696052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87b409baf0 msgr2=0x7f87b4154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87b9400640 1 --2- 172.21.15.42:0/2118696052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87b409baf0 0x7f87b4154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87b9400640 1 -- 172.21.15.42:0/2118696052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b415fdf0 msgr2=0x7f87b4157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87b9400640 1 --2- 172.21.15.42:0/2118696052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b415fdf0 0x7f87b4157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87b9400640 1 -- 172.21.15.42:0/2118696052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b40b59b0 con 0x7f87b40d0ea0 2024-05-28T14:04:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87b9400640 1 --2- 172.21.15.42:0/2118696052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40d0ea0 0x7f87b4157580 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c400 tx=0x7f87a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:17.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87ae000640 1 -- 172.21.15.42:0/2118696052 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4017070 con 0x7f87b40d0ea0 2024-05-28T14:04:17.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87ae000640 1 -- 172.21.15.42:0/2118696052 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a4005310 con 0x7f87b40d0ea0 2024-05-28T14:04:17.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87bc800640 1 -- 172.21.15.42:0/2118696052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b415d0a0 con 0x7f87b40d0ea0 2024-05-28T14:04:17.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87ae000640 1 -- 172.21.15.42:0/2118696052 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a40054c0 con 0x7f87b40d0ea0 2024-05-28T14:04:17.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.815+0000 7f87bc800640 1 -- 172.21.15.42:0/2118696052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b415d660 con 0x7f87b40d0ea0 2024-05-28T14:04:17.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.819+0000 7f87bc800640 1 -- 172.21.15.42:0/2118696052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b40a0650 con 0x7f87b40d0ea0 2024-05-28T14:04:17.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.819+0000 7f87ae000640 1 -- 172.21.15.42:0/2118696052 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f87a401a070 con 0x7f87b40d0ea0 2024-05-28T14:04:17.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.819+0000 7f87ae000640 1 --2- 172.21.15.42:0/2118696052 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8798068b90 0x7f879806b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:17.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.819+0000 7f87ae000640 1 -- 172.21.15.42:0/2118696052 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f87a4086c20 con 0x7f87b40d0ea0 2024-05-28T14:04:17.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.819+0000 7f87b9e00640 1 --2- 172.21.15.42:0/2118696052 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8798068b90 0x7f879806b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:17.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.819+0000 7f87b9e00640 1 --2- 172.21.15.42:0/2118696052 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8798068b90 0x7f879806b050 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f87a80ae000 tx=0x7f87a80a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:17.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:17.823+0000 7f87ae000640 1 -- 172.21.15.42:0/2118696052 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f87b40b59b0 con 0x7f87b40d0ea0 2024-05-28T14:04:17.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:18.003+0000 7f87bc800640 1 -- 172.21.15.42:0/2118696052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f87b404fdd0 con 0x7f87b40d0ea0 2024-05-28T14:04:18.439 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:04:18.443 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:04:18.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:18.899+0000 7f87ae000640 1 -- 172.21.15.42:0/2118696052 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v57) ==== 97+0+0 (secure 0 0 0) 0x7f87a4086e20 con 0x7f87b40d0ea0 2024-05-28T14:04:18.885 INFO:teuthology.orchestra.run.smithi042.stderr: removed all blocklist entries 2024-05-28T14:04:18.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:18.899+0000 7f87bc800640 1 -- 172.21.15.42:0/2118696052 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8798068b90 msgr2=0x7f879806b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:18.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:18.899+0000 7f87bc800640 1 --2- 172.21.15.42:0/2118696052 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8798068b90 0x7f879806b050 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f87a80ae000 tx=0x7f87a80a7000 comp rx=0 tx=0).stop 2024-05-28T14:04:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:18.899+0000 7f87bc800640 1 -- 172.21.15.42:0/2118696052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40d0ea0 msgr2=0x7f87b4157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:18.899+0000 7f87bc800640 1 --2- 172.21.15.42:0/2118696052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40d0ea0 0x7f87b4157580 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c400 tx=0x7f87a400c8c0 comp rx=0 tx=0).stop 2024-05-28T14:04:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:18.899+0000 7f87bc800640 1 -- 172.21.15.42:0/2118696052 shutdown_connections 2024-05-28T14:04:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:18.899+0000 7f87bc800640 1 --2- 172.21.15.42:0/2118696052 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8798068b90 0x7f879806b050 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:18.899+0000 7f87bc800640 1 --2- 172.21.15.42:0/2118696052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b415fdf0 0x7f87b4157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:18.899+0000 7f87bc800640 1 --2- 172.21.15.42:0/2118696052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40d0ea0 0x7f87b4157580 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:18.899+0000 7f87bc800640 1 --2- 172.21.15.42:0/2118696052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87b409baf0 0x7f87b4154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:18.899+0000 7f87bc800640 1 -- 172.21.15.42:0/2118696052 >> 172.21.15.42:0/2118696052 conn(0x7f87b40b4050 msgr2=0x7f87b40af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:18.903+0000 7f87bc800640 1 -- 172.21.15.42:0/2118696052 shutdown_connections 2024-05-28T14:04:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:18.903+0000 7f87bc800640 1 -- 172.21.15.42:0/2118696052 wait complete. 2024-05-28T14:04:18.915 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-28T14:04:19.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb6d000640 1 Processor -- start 2024-05-28T14:04:19.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb6d000640 1 -- start start 2024-05-28T14:04:19.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb6d000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb680d0450 0x7ffb6809b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb6d000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffb6815c880 0x7ffb680b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb6d000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb680d0160 con 0x7ffb6815d680 2024-05-28T14:04:19.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb6d000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ffb6809b520 con 0x7ffb6815c880 2024-05-28T14:04:19.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb6d000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffb6809abf0 con 0x7ffb680d0450 2024-05-28T14:04:19.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb5fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffb6815c880 0x7ffb680b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb5fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffb6815c880 0x7ffb680b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34352/0 (socket says 172.21.15.42:34352) 2024-05-28T14:04:19.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb5fe00640 1 -- 172.21.15.42:0/2544928532 learned_addr learned my addr 172.21.15.42:0/2544928532 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:19.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb67400640 1 --2- 172.21.15.42:0/2544928532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb680d0450 0x7ffb6809b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb5fe00640 1 -- 172.21.15.42:0/2544928532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb680d0450 msgr2=0x7ffb6809b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:19.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb5fe00640 1 --2- 172.21.15.42:0/2544928532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb680d0450 0x7ffb6809b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb5fe00640 1 -- 172.21.15.42:0/2544928532 >> v1:172.21.15.63:6789/0 conn(0x7ffb6815d680 legacy=0x7ffb680d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.075+0000 7ffb5fe00640 1 -- 172.21.15.42:0/2544928532 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6809aed0 con 0x7ffb6815c880 2024-05-28T14:04:19.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.079+0000 7ffb5fe00640 1 --2- 172.21.15.42:0/2544928532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb6815c880 0x7ffb680b16e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009870 tx=0x7ffb5001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1058c3dfa5e44f86 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:19.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.079+0000 7ffb66a00640 1 -- 172.21.15.42:0/2544928532 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb500ab5f0 con 0x7ffb6815c880 2024-05-28T14:04:19.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.079+0000 7ffb66a00640 1 -- 172.21.15.42:0/2544928532 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb500b0070 con 0x7ffb6815c880 2024-05-28T14:04:19.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.079+0000 7ffb66a00640 1 -- 172.21.15.42:0/2544928532 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb500ab9b0 con 0x7ffb6815c880 2024-05-28T14:04:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.079+0000 7ffb6d000640 1 -- 172.21.15.42:0/2544928532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb6815c880 msgr2=0x7ffb680b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.079+0000 7ffb6d000640 1 --2- 172.21.15.42:0/2544928532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb6815c880 0x7ffb680b16e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009870 tx=0x7ffb5001c560 comp rx=0 tx=0).stop 2024-05-28T14:04:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.079+0000 7ffb6d000640 1 -- 172.21.15.42:0/2544928532 shutdown_connections 2024-05-28T14:04:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.079+0000 7ffb6d000640 1 --2- 172.21.15.42:0/2544928532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb6815c880 0x7ffb680b16e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.079+0000 7ffb6d000640 1 --2- 172.21.15.42:0/2544928532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb680d0450 0x7ffb6809b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.079+0000 7ffb6d000640 1 -- 172.21.15.42:0/2544928532 >> 172.21.15.42:0/2544928532 conn(0x7ffb680b3dd0 msgr2=0x7ffb680b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.079+0000 7ffb6d000640 1 -- 172.21.15.42:0/2544928532 shutdown_connections 2024-05-28T14:04:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.079+0000 7ffb6d000640 1 -- 172.21.15.42:0/2544928532 wait complete. 2024-05-28T14:04:19.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb6d000640 1 Processor -- start 2024-05-28T14:04:19.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb6d000640 1 -- start start 2024-05-28T14:04:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb6d000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb680d0450 0x7ffb680b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb6d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb6815c880 0x7ffb6815b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb6d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb6815d680 0x7ffb68156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb6d000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffb680cd9d0 con 0x7ffb6815c880 2024-05-28T14:04:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb6d000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffb680cd850 con 0x7ffb6815d680 2024-05-28T14:04:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb6d000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffb680cdb50 con 0x7ffb680d0450 2024-05-28T14:04:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb5fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb6815c880 0x7ffb6815b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb5fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb6815c880 0x7ffb6815b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34354/0 (socket says 172.21.15.42:34354) 2024-05-28T14:04:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb5fe00640 1 -- 172.21.15.42:0/1780563972 learned_addr learned my addr 172.21.15.42:0/1780563972 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb67400640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb680d0450 0x7ffb680b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb67e00640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb6815d680 0x7ffb68156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb5fe00640 1 -- 172.21.15.42:0/1780563972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb680d0450 msgr2=0x7ffb680b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb5fe00640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb680d0450 0x7ffb680b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb5fe00640 1 -- 172.21.15.42:0/1780563972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb6815d680 msgr2=0x7ffb68156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb5fe00640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb6815d680 0x7ffb68156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb5fe00640 1 -- 172.21.15.42:0/1780563972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb680af1f0 con 0x7ffb6815c880 2024-05-28T14:04:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb67e00640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb6815d680 0x7ffb68156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb5fe00640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb6815c880 0x7ffb6815b050 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ffb500ad040 tx=0x7ffb5001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb5f400640 1 -- 172.21.15.42:0/1780563972 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb500b4800 con 0x7ffb6815c880 2024-05-28T14:04:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb5f400640 1 -- 172.21.15.42:0/1780563972 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb500b49a0 con 0x7ffb6815c880 2024-05-28T14:04:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb5f400640 1 -- 172.21.15.42:0/1780563972 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb500b4d90 con 0x7ffb6815c880 2024-05-28T14:04:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb6d000640 1 -- 172.21.15.42:0/1780563972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb680af4d0 con 0x7ffb6815c880 2024-05-28T14:04:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.083+0000 7ffb6d000640 1 -- 172.21.15.42:0/1780563972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb680afa90 con 0x7ffb6815c880 2024-05-28T14:04:19.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.087+0000 7ffb6d000640 1 -- 172.21.15.42:0/1780563972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6804fcf0 con 0x7ffb6815c880 2024-05-28T14:04:19.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.087+0000 7ffb5f400640 1 -- 172.21.15.42:0/1780563972 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7ffb500b3380 con 0x7ffb6815c880 2024-05-28T14:04:19.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.087+0000 7ffb5f400640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb44068b60 0x7ffb4406b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.087+0000 7ffb5f400640 1 -- 172.21.15.42:0/1780563972 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4885+0+0 (secure 0 0 0) 0x7ffb50121510 con 0x7ffb6815c880 2024-05-28T14:04:19.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.087+0000 7ffb67400640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb44068b60 0x7ffb4406b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.087+0000 7ffb67400640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb44068b60 0x7ffb4406b020 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffb580079c0 tx=0x7ffb58008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:19.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.091+0000 7ffb5f400640 1 -- 172.21.15.42:0/1780563972 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffb50016150 con 0x7ffb6815c880 2024-05-28T14:04:19.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.311+0000 7ffb6d000640 1 -- 172.21.15.42:0/1780563972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ffb680b4aa0 con 0x7ffb6815c880 2024-05-28T14:04:19.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.311+0000 7ffb5f400640 1 -- 172.21.15.42:0/1780563972 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7ffb500f0d70 con 0x7ffb6815c880 2024-05-28T14:04:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:19.298 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:04:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_dump": [ 2024-05-28T14:04:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.a", 2024-05-28T14:04:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmE+3RLRAADuTKGfKeVafGeLpgr9WKUg==", 2024-05-28T14:04:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:04:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:04:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:04:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.b", 2024-05-28T14:04:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVm1rmVOhAAV46KzDJnRfDZYIZanS0g2g==", 2024-05-28T14:04:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:04:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:04:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:04:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.c", 2024-05-28T14:04:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVme4aYNRAAVxJTlugnxtFTexUclF/IwA==", 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.d", 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmeAdBBhAAr4IVFQpjQXFVlnFTHnPV9w==", 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:04:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.0", 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDB4lVmGMDBJhAA3jIyR7XKLGXEgimIGNsw6A==", 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.1", 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDE4lVmxMisLxAAcIz7GPlkZwvW5ig+DbAVGg==", 2024-05-28T14:04:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:04:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:04:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.2", 2024-05-28T14:04:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDH4lVmSSk4OhAAmLlLysM4Im/Kew4Xbq2/OA==", 2024-05-28T14:04:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:04:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:04:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.3", 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDL4lVmvzrYBhAAiAAOQXZBuTsMVYa1vE5rkg==", 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.4", 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDQ4lVmrqnTOhAA2pIfgBT9LJDIhp+GgFy8Gg==", 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.5", 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDU4lVmYpsQCxAAoTK2aMBPrHKvsUnNOTj/Dw==", 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.6", 2024-05-28T14:04:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDX4lVm6TCAFxAAs3rkDyKcLhWrn/EkIo0bVA==", 2024-05-28T14:04:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:04:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:04:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.7", 2024-05-28T14:04:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDa4lVmr5NwIRAA7Lb70VYV82dr0Frn8CCUXg==", 2024-05-28T14:04:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:04:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:04:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.0", 2024-05-28T14:04:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw==", 2024-05-28T14:04:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:04:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-05-28T14:04:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-28T14:04:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.1", 2024-05-28T14:04:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmcfkLFBAADNyODKTtYAlFWquZ/LeeqQ==", 2024-05-28T14:04:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:04:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:04:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:04:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:04:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.2", 2024-05-28T14:04:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmoxx+EBAA8mLNak5AwRmWQIgkYtMOrg==", 2024-05-28T14:04:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:04:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:04:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:04:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:04:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.3", 2024-05-28T14:04:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmxSUsGhAAbgr1eOh/sMfrEONKKHFo0g==", 2024-05-28T14:04:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:04:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:04:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:04:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:04:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.admin", 2024-05-28T14:04:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmOSBPERAAt4x2DBal4PnqxqeWQutLDQ==", 2024-05-28T14:04:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:04:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:04:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:04:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.x", 2024-05-28T14:04:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmNO5tKRAA70hPLwoE83keovlq2rolxQ==", 2024-05-28T14:04:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:04:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T14:04:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.y", 2024-05-28T14:04:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmxTVyJRAATMLAAqKne20YvV537uXZ6g==", 2024-05-28T14:04:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:04:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:04:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T14:04:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:04:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-28T14:04:19.311 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:04:19.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.311+0000 7ffb6d000640 1 -- 172.21.15.42:0/1780563972 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb44068b60 msgr2=0x7ffb4406b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.311+0000 7ffb6d000640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb44068b60 0x7ffb4406b020 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffb580079c0 tx=0x7ffb58008040 comp rx=0 tx=0).stop 2024-05-28T14:04:19.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.311+0000 7ffb6d000640 1 -- 172.21.15.42:0/1780563972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb6815c880 msgr2=0x7ffb6815b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.311+0000 7ffb6d000640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb6815c880 0x7ffb6815b050 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ffb500ad040 tx=0x7ffb5001ada0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.311+0000 7ffb6d000640 1 -- 172.21.15.42:0/1780563972 shutdown_connections 2024-05-28T14:04:19.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.311+0000 7ffb6d000640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb44068b60 0x7ffb4406b020 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.311+0000 7ffb6d000640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb6815d680 0x7ffb68156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.311+0000 7ffb6d000640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb6815c880 0x7ffb6815b050 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.311+0000 7ffb6d000640 1 --2- 172.21.15.42:0/1780563972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb680d0450 0x7ffb680b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.311+0000 7ffb6d000640 1 -- 172.21.15.42:0/1780563972 >> 172.21.15.42:0/1780563972 conn(0x7ffb680b3dd0 msgr2=0x7ffb680715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:19.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.311+0000 7ffb6d000640 1 -- 172.21.15.42:0/1780563972 shutdown_connections 2024-05-28T14:04:19.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.311+0000 7ffb6d000640 1 -- 172.21.15.42:0/1780563972 wait complete. 2024-05-28T14:04:19.319 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:04:19.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.475+0000 7f1adf200640 1 Processor -- start 2024-05-28T14:04:19.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.475+0000 7f1adf200640 1 -- start start 2024-05-28T14:04:19.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.475+0000 7f1adf200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1ad80ce860 0x7f1ad806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.475+0000 7f1adf200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ad80d0de0 con 0x7f1ad80d0900 2024-05-28T14:04:19.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.475+0000 7f1adf200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ad80d10c0 con 0x7f1ad8162c80 2024-05-28T14:04:19.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.475+0000 7f1adf200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1ad80d19f0 con 0x7f1ad80ce860 2024-05-28T14:04:19.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.475+0000 7f1ad7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1ad80ce860 0x7f1ad806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.475+0000 7f1ad7400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1ad80d0900 0x7f1ad8162570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:52582/0 (socket says 172.21.15.42:52582) 2024-05-28T14:04:19.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.475+0000 7f1ad7400640 1 -- 172.21.15.42:0/949676028 learned_addr learned my addr 172.21.15.42:0/949676028 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:19.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.475+0000 7f1ad7e00640 1 -- 172.21.15.42:0/949676028 >> v1:172.21.15.63:6789/0 conn(0x7f1ad8162c80 legacy=0x7f1ad815e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.475+0000 7f1ad7e00640 1 -- 172.21.15.42:0/949676028 >> v1:172.21.15.42:6789/0 conn(0x7f1ad80d0900 legacy=0x7f1ad8162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.475+0000 7f1ad7e00640 1 -- 172.21.15.42:0/949676028 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad80d13a0 con 0x7f1ad80ce860 2024-05-28T14:04:19.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.475+0000 7f1ad6a00640 1 -- 172.21.15.42:0/949676028 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2071948469 0 0) 0x7f1ad80d0de0 con 0x7f1ad80d0900 2024-05-28T14:04:19.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.479+0000 7f1ad7e00640 1 --2- 172.21.15.42:0/949676028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad80ce860 0x7f1ad806d7b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80097c0 tx=0x7f1ac801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efdc77df271d7ec server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:19.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.479+0000 7f1ad6a00640 1 -- 172.21.15.42:0/949676028 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac80b0070 con 0x7f1ad80ce860 2024-05-28T14:04:19.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.479+0000 7f1ad6a00640 1 -- 172.21.15.42:0/949676028 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac80ab600 con 0x7f1ad80ce860 2024-05-28T14:04:19.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.479+0000 7f1ad6a00640 1 -- 172.21.15.42:0/949676028 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac80ab9d0 con 0x7f1ad80ce860 2024-05-28T14:04:19.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.479+0000 7f1adf200640 1 -- 172.21.15.42:0/949676028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad80ce860 msgr2=0x7f1ad806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.479+0000 7f1adf200640 1 --2- 172.21.15.42:0/949676028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad80ce860 0x7f1ad806d7b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80097c0 tx=0x7f1ac801c3e0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.479+0000 7f1adf200640 1 -- 172.21.15.42:0/949676028 shutdown_connections 2024-05-28T14:04:19.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.479+0000 7f1adf200640 1 --2- 172.21.15.42:0/949676028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad80ce860 0x7f1ad806d7b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.479+0000 7f1adf200640 1 -- 172.21.15.42:0/949676028 >> 172.21.15.42:0/949676028 conn(0x7f1ad80b4080 msgr2=0x7f1ad80b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:19.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.479+0000 7f1adf200640 1 -- 172.21.15.42:0/949676028 shutdown_connections 2024-05-28T14:04:19.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.479+0000 7f1adf200640 1 -- 172.21.15.42:0/949676028 wait complete. 2024-05-28T14:04:19.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.479+0000 7f1adf200640 1 Processor -- start 2024-05-28T14:04:19.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.479+0000 7f1adf200640 1 -- start start 2024-05-28T14:04:19.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1adf200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad80ce860 0x7f1ad81558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1adf200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad80d0900 0x7f1ad80b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1adf200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad8162c80 0x7f1ad815f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1adf200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad815eca0 con 0x7f1ad80d0900 2024-05-28T14:04:19.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1adf200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad815eb20 con 0x7f1ad8162c80 2024-05-28T14:04:19.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1adf200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad815ee20 con 0x7f1ad80ce860 2024-05-28T14:04:19.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1ad7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad80d0900 0x7f1ad80b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1ad7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad80d0900 0x7f1ad80b7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34380/0 (socket says 172.21.15.42:34380) 2024-05-28T14:04:19.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1ad7400640 1 -- 172.21.15.42:0/892022919 learned_addr learned my addr 172.21.15.42:0/892022919 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:19.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1ad7e00640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad80ce860 0x7f1ad81558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1add200640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad8162c80 0x7f1ad815f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1ad7400640 1 -- 172.21.15.42:0/892022919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad80ce860 msgr2=0x7f1ad81558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1ad7400640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad80ce860 0x7f1ad81558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1ad7400640 1 -- 172.21.15.42:0/892022919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad8162c80 msgr2=0x7f1ad815f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1ad7400640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad8162c80 0x7f1ad815f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1ad7400640 1 -- 172.21.15.42:0/892022919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad80b5570 con 0x7f1ad80d0900 2024-05-28T14:04:19.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1add200640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad8162c80 0x7f1ad815f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:19.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1ad7400640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad80d0900 0x7f1ad80b7b90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400c4a0 tx=0x7f1ac400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:19.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1acfe00640 1 -- 172.21.15.42:0/892022919 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4018070 con 0x7f1ad80d0900 2024-05-28T14:04:19.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1acfe00640 1 -- 172.21.15.42:0/892022919 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac40074e0 con 0x7f1ad80d0900 2024-05-28T14:04:19.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1adf200640 1 -- 172.21.15.42:0/892022919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad80b5bc0 con 0x7f1ad80d0900 2024-05-28T14:04:19.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1acfe00640 1 -- 172.21.15.42:0/892022919 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4004a30 con 0x7f1ad80d0900 2024-05-28T14:04:19.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1adf200640 1 -- 172.21.15.42:0/892022919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad80b6180 con 0x7f1ad80d0900 2024-05-28T14:04:19.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.483+0000 7f1adf200640 1 -- 172.21.15.42:0/892022919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad80ca4d0 con 0x7f1ad80d0900 2024-05-28T14:04:19.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.487+0000 7f1acfe00640 1 -- 172.21.15.42:0/892022919 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f1ac401b070 con 0x7f1ad80d0900 2024-05-28T14:04:19.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.487+0000 7f1acfe00640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ab8068b60 0x7f1ab806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.487+0000 7f1acfe00640 1 -- 172.21.15.42:0/892022919 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4885+0+0 (secure 0 0 0) 0x7f1ac4086b80 con 0x7f1ad80d0900 2024-05-28T14:04:19.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.487+0000 7f1ad7e00640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ab8068b60 0x7f1ab806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.487+0000 7f1ad7e00640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ab8068b60 0x7f1ab806b020 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80ad000 tx=0x7f1ac8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:19.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.491+0000 7f1acfe00640 1 -- 172.21.15.42:0/892022919 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ac4055e00 con 0x7f1ad80d0900 2024-05-28T14:04:19.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.659+0000 7f1adf200640 1 -- 172.21.15.42:0/892022919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ad80b4ce0 con 0x7f1ad80d0900 2024-05-28T14:04:19.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.659+0000 7f1acfe00640 1 -- 172.21.15.42:0/892022919 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+12874 (secure 0 0 0) 0x7f1ac405a100 con 0x7f1ad80d0900 2024-05-28T14:04:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:19.645 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":57,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:04:18.900605+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:04:19.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.659+0000 7f1adf200640 1 -- 172.21.15.42:0/892022919 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ab8068b60 msgr2=0x7f1ab806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.659+0000 7f1adf200640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ab8068b60 0x7f1ab806b020 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80ad000 tx=0x7f1ac8008040 comp rx=0 tx=0).stop 2024-05-28T14:04:19.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.659+0000 7f1adf200640 1 -- 172.21.15.42:0/892022919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad80d0900 msgr2=0x7f1ad80b7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.659+0000 7f1adf200640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad80d0900 0x7f1ad80b7b90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400c4a0 tx=0x7f1ac400c960 comp rx=0 tx=0).stop 2024-05-28T14:04:19.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.659+0000 7f1adf200640 1 -- 172.21.15.42:0/892022919 shutdown_connections 2024-05-28T14:04:19.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.659+0000 7f1adf200640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ab8068b60 0x7f1ab806b020 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.659+0000 7f1adf200640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad8162c80 0x7f1ad815f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.659+0000 7f1adf200640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad80d0900 0x7f1ad80b7b90 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.659+0000 7f1adf200640 1 --2- 172.21.15.42:0/892022919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad80ce860 0x7f1ad81558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.659+0000 7f1adf200640 1 -- 172.21.15.42:0/892022919 >> 172.21.15.42:0/892022919 conn(0x7f1ad80b4080 msgr2=0x7f1ad806ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:19.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.659+0000 7f1adf200640 1 -- 172.21.15.42:0/892022919 shutdown_connections 2024-05-28T14:04:19.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.659+0000 7f1adf200640 1 -- 172.21.15.42:0/892022919 wait complete. 2024-05-28T14:04:19.668 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:04:19.668 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:04:19.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.823+0000 7fa705e00640 1 Processor -- start 2024-05-28T14:04:19.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.827+0000 7fa705e00640 1 -- start start 2024-05-28T14:04:19.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.827+0000 7fa705e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa700144f80 0x7fa700072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.827+0000 7fa705e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7000afe40 con 0x7fa70015c0c0 2024-05-28T14:04:19.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.827+0000 7fa705e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa70015c600 con 0x7fa70006ee20 2024-05-28T14:04:19.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.827+0000 7fa705e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa7000724d0 con 0x7fa700144f80 2024-05-28T14:04:19.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.827+0000 7fa6fea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa700144f80 0x7fa700072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.827+0000 7fa6ff400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa70015c0c0 0x7fa7000b3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:52584/0 (socket says 172.21.15.42:52584) 2024-05-28T14:04:19.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.827+0000 7fa6ff400640 1 -- 172.21.15.42:0/601105292 learned_addr learned my addr 172.21.15.42:0/601105292 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:19.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.827+0000 7fa6fea00640 1 -- 172.21.15.42:0/601105292 >> v1:172.21.15.63:6789/0 conn(0x7fa70006ee20 legacy=0x7fa70015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:19.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.827+0000 7fa6fea00640 1 -- 172.21.15.42:0/601105292 >> v1:172.21.15.42:6789/0 conn(0x7fa70015c0c0 legacy=0x7fa7000b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.827+0000 7fa6fea00640 1 -- 172.21.15.42:0/601105292 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70015c8e0 con 0x7fa700144f80 2024-05-28T14:04:19.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.831+0000 7fa6fea00640 1 --2- 172.21.15.42:0/601105292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa700144f80 0x7fa700072960 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00097c0 tx=0x7fa6f001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751a0efaaf3485eb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:19.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.831+0000 7fa6fe000640 1 -- 172.21.15.42:0/601105292 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f00b0070 con 0x7fa700144f80 2024-05-28T14:04:19.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.831+0000 7fa6fe000640 1 -- 172.21.15.42:0/601105292 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6f00ab600 con 0x7fa700144f80 2024-05-28T14:04:19.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.831+0000 7fa6fe000640 1 -- 172.21.15.42:0/601105292 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f00ab9d0 con 0x7fa700144f80 2024-05-28T14:04:19.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.831+0000 7fa705e00640 1 -- 172.21.15.42:0/601105292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa700144f80 msgr2=0x7fa700072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.831+0000 7fa705e00640 1 --2- 172.21.15.42:0/601105292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa700144f80 0x7fa700072960 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00097c0 tx=0x7fa6f001c460 comp rx=0 tx=0).stop 2024-05-28T14:04:19.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.831+0000 7fa705e00640 1 -- 172.21.15.42:0/601105292 shutdown_connections 2024-05-28T14:04:19.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.831+0000 7fa705e00640 1 --2- 172.21.15.42:0/601105292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa700144f80 0x7fa700072960 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.831+0000 7fa705e00640 1 -- 172.21.15.42:0/601105292 >> 172.21.15.42:0/601105292 conn(0x7fa7000b21f0 msgr2=0x7fa7000b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:19.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.831+0000 7fa705e00640 1 -- 172.21.15.42:0/601105292 shutdown_connections 2024-05-28T14:04:19.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.831+0000 7fa705e00640 1 -- 172.21.15.42:0/601105292 wait complete. 2024-05-28T14:04:19.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.831+0000 7fa705e00640 1 Processor -- start 2024-05-28T14:04:19.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.831+0000 7fa705e00640 1 -- start start 2024-05-28T14:04:19.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa705e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa70006ee20 0x7fa70015d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa705e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa700144f80 0x7fa70015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa705e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa70015c0c0 0x7fa7000b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa705e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa700072ea0 con 0x7fa70006ee20 2024-05-28T14:04:19.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa705e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa700169950 con 0x7fa70015c0c0 2024-05-28T14:04:19.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa705e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa7001fb520 con 0x7fa700144f80 2024-05-28T14:04:19.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6fea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa70006ee20 0x7fa70015d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6fea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa70006ee20 0x7fa70015d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34398/0 (socket says 172.21.15.42:34398) 2024-05-28T14:04:19.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6fea00640 1 -- 172.21.15.42:0/3189019511 learned_addr learned my addr 172.21.15.42:0/3189019511 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:19.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6f6000640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa700144f80 0x7fa70015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6ff400640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa70015c0c0 0x7fa7000b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6fea00640 1 -- 172.21.15.42:0/3189019511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa700144f80 msgr2=0x7fa70015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6fea00640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa700144f80 0x7fa70015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6fea00640 1 -- 172.21.15.42:0/3189019511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa70015c0c0 msgr2=0x7fa7000b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:19.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6fea00640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa70015c0c0 0x7fa7000b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:19.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6fea00640 1 -- 172.21.15.42:0/3189019511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70015b700 con 0x7fa70006ee20 2024-05-28T14:04:19.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6ff400640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa70015c0c0 0x7fa7000b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6fea00640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa70006ee20 0x7fa70015d580 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0019070 tx=0x7fa6f001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6f7e00640 1 -- 172.21.15.42:0/3189019511 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f00b0050 con 0x7fa70006ee20 2024-05-28T14:04:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6f7e00640 1 -- 172.21.15.42:0/3189019511 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6f00bc070 con 0x7fa70006ee20 2024-05-28T14:04:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa6f7e00640 1 -- 172.21.15.42:0/3189019511 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f00b5a70 con 0x7fa70006ee20 2024-05-28T14:04:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa705e00640 1 -- 172.21.15.42:0/3189019511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7001fb770 con 0x7fa70006ee20 2024-05-28T14:04:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa705e00640 1 -- 172.21.15.42:0/3189019511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7001fbd30 con 0x7fa70006ee20 2024-05-28T14:04:19.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.835+0000 7fa705e00640 1 -- 172.21.15.42:0/3189019511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7000738b0 con 0x7fa70006ee20 2024-05-28T14:04:19.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.839+0000 7fa6f7e00640 1 -- 172.21.15.42:0/3189019511 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fa6f00ab480 con 0x7fa70006ee20 2024-05-28T14:04:19.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.839+0000 7fa6f7e00640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa6dc068b60 0x7fa6dc06b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:19.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.839+0000 7fa6f7e00640 1 -- 172.21.15.42:0/3189019511 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4885+0+0 (secure 0 0 0) 0x7fa6f0121e40 con 0x7fa70006ee20 2024-05-28T14:04:19.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.839+0000 7fa6f6000640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa6dc068b60 0x7fa6dc06b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:19.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.839+0000 7fa6f6000640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa6dc068b60 0x7fa6dc06b020 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa7000cdce0 tx=0x7fa6e8007e40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:19.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:19.843+0000 7fa6f7e00640 1 -- 172.21.15.42:0/3189019511 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa6f0128080 con 0x7fa70006ee20 2024-05-28T14:04:19.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.011+0000 7fa705e00640 1 -- 172.21.15.42:0/3189019511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa70004fdd0 con 0x7fa70006ee20 2024-05-28T14:04:19.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.011+0000 7fa6f7e00640 1 -- 172.21.15.42:0/3189019511 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) ==== 93+0+10 (secure 0 0 0) 0x7fa6f0016050 con 0x7fa70006ee20 2024-05-28T14:04:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:04:19.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.011+0000 7fa705e00640 1 -- 172.21.15.42:0/3189019511 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa6dc068b60 msgr2=0x7fa6dc06b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.011+0000 7fa705e00640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa6dc068b60 0x7fa6dc06b020 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa7000cdce0 tx=0x7fa6e8007e40 comp rx=0 tx=0).stop 2024-05-28T14:04:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.011+0000 7fa705e00640 1 -- 172.21.15.42:0/3189019511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa70006ee20 msgr2=0x7fa70015d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.011+0000 7fa705e00640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa70006ee20 0x7fa70015d580 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0019070 tx=0x7fa6f001aca0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.011+0000 7fa705e00640 1 -- 172.21.15.42:0/3189019511 shutdown_connections 2024-05-28T14:04:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.011+0000 7fa705e00640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa6dc068b60 0x7fa6dc06b020 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.011+0000 7fa705e00640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa70015c0c0 0x7fa7000b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.011+0000 7fa705e00640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa700144f80 0x7fa70015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.011+0000 7fa705e00640 1 --2- 172.21.15.42:0/3189019511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa70006ee20 0x7fa70015d580 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.011+0000 7fa705e00640 1 -- 172.21.15.42:0/3189019511 >> 172.21.15.42:0/3189019511 conn(0x7fa7000b21f0 msgr2=0x7fa70015f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.015+0000 7fa705e00640 1 -- 172.21.15.42:0/3189019511 shutdown_connections 2024-05-28T14:04:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.015+0000 7fa705e00640 1 -- 172.21.15.42:0/3189019511 wait complete. 2024-05-28T14:04:20.019 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-28T14:04:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a96000640 1 Processor -- start 2024-05-28T14:04:20.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a96000640 1 -- start start 2024-05-28T14:04:20.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a96000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8a90156a70 0x7f8a9006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:20.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a96000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8a90156e50 0x7f8a900af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:20.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a96000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a900afc80 con 0x7f8a900af880 2024-05-28T14:04:20.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a96000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9015cb90 con 0x7f8a90156a70 2024-05-28T14:04:20.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a96000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9015ce50 con 0x7f8a90156e50 2024-05-28T14:04:20.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a8f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8a90156a70 0x7f8a9006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:20.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a8f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8a90156a70 0x7f8a9006dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34408/0 (socket says 172.21.15.42:34408) 2024-05-28T14:04:20.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a8f400640 1 -- 172.21.15.42:0/2451089764 learned_addr learned my addr 172.21.15.42:0/2451089764 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:20.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a86a00640 1 --2- 172.21.15.42:0/2451089764 >> v2:172.21.15.63:3300/0 conn(0x7f8a90156e50 0x7f8a900af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:20.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a8f400640 1 -- 172.21.15.42:0/2451089764 >> v2:172.21.15.63:3300/0 conn(0x7f8a90156e50 msgr2=0x7f8a900af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:20.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a8f400640 1 --2- 172.21.15.42:0/2451089764 >> v2:172.21.15.63:3300/0 conn(0x7f8a90156e50 0x7f8a900af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a8f400640 1 -- 172.21.15.42:0/2451089764 >> v1:172.21.15.63:6789/0 conn(0x7f8a900af880 legacy=0x7f8a900d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:20.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a8f400640 1 -- 172.21.15.42:0/2451089764 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9015d130 con 0x7f8a90156a70 2024-05-28T14:04:20.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.175+0000 7f8a86a00640 1 --2- 172.21.15.42:0/2451089764 >> v2:172.21.15.63:3300/0 conn(0x7f8a90156e50 0x7f8a900af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.179+0000 7f8a8f400640 1 --2- 172.21.15.42:0/2451089764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a90156a70 0x7f8a9006dd60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8a900701f0 tx=0x7f8a8001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238f12007d921d6b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.179+0000 7f8a8ea00640 1 -- 172.21.15.42:0/2451089764 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a800a6040 con 0x7f8a90156a70 2024-05-28T14:04:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.179+0000 7f8a8ea00640 1 -- 172.21.15.42:0/2451089764 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a800af070 con 0x7f8a90156a70 2024-05-28T14:04:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.179+0000 7f8a8ea00640 1 -- 172.21.15.42:0/2451089764 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a800aa860 con 0x7f8a90156a70 2024-05-28T14:04:20.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.179+0000 7f8a96000640 1 -- 172.21.15.42:0/2451089764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a90156a70 msgr2=0x7f8a9006dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:20.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.179+0000 7f8a96000640 1 --2- 172.21.15.42:0/2451089764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a90156a70 0x7f8a9006dd60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8a900701f0 tx=0x7f8a8001c4e0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.179+0000 7f8a96000640 1 -- 172.21.15.42:0/2451089764 shutdown_connections 2024-05-28T14:04:20.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.179+0000 7f8a96000640 1 --2- 172.21.15.42:0/2451089764 >> v2:172.21.15.63:3300/0 conn(0x7f8a90156e50 0x7f8a900af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.179+0000 7f8a96000640 1 --2- 172.21.15.42:0/2451089764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a90156a70 0x7f8a9006dd60 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.179+0000 7f8a96000640 1 -- 172.21.15.42:0/2451089764 >> 172.21.15.42:0/2451089764 conn(0x7f8a9006bdf0 msgr2=0x7f8a9006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:20.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.179+0000 7f8a96000640 1 -- 172.21.15.42:0/2451089764 shutdown_connections 2024-05-28T14:04:20.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.179+0000 7f8a96000640 1 -- 172.21.15.42:0/2451089764 wait complete. 2024-05-28T14:04:20.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a96000640 1 Processor -- start 2024-05-28T14:04:20.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a96000640 1 -- start start 2024-05-28T14:04:20.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a96000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a900af880 0x7f8a90150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:20.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a96000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a90156a70 0x7f8a9015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:20.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a96000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a90156e50 0x7f8a9014e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:20.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a96000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a90160b40 con 0x7f8a900af880 2024-05-28T14:04:20.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a96000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a900d1010 con 0x7f8a90156a70 2024-05-28T14:04:20.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a96000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8a90160cc0 con 0x7f8a90156e50 2024-05-28T14:04:20.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a8f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a900af880 0x7f8a90150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:20.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a8f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a900af880 0x7f8a90150c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34416/0 (socket says 172.21.15.42:34416) 2024-05-28T14:04:20.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a8f400640 1 -- 172.21.15.42:0/3364958064 learned_addr learned my addr 172.21.15.42:0/3364958064 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:20.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a8fe00640 1 --2- 172.21.15.42:0/3364958064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a90156e50 0x7f8a9014e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:20.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a86a00640 1 --2- 172.21.15.42:0/3364958064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a90156a70 0x7f8a9015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:20.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a8f400640 1 -- 172.21.15.42:0/3364958064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a90156e50 msgr2=0x7f8a9014e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:20.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a8f400640 1 --2- 172.21.15.42:0/3364958064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a90156e50 0x7f8a9014e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a8f400640 1 -- 172.21.15.42:0/3364958064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a90156a70 msgr2=0x7f8a9015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:20.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a8f400640 1 --2- 172.21.15.42:0/3364958064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a90156a70 0x7f8a9015ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a8f400640 1 -- 172.21.15.42:0/3364958064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9015f2c0 con 0x7f8a900af880 2024-05-28T14:04:20.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a8f400640 1 --2- 172.21.15.42:0/3364958064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a900af880 0x7f8a90150c90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8a8001a110 tx=0x7f8a8001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:20.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a87e00640 1 -- 172.21.15.42:0/3364958064 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a800bd5f0 con 0x7f8a900af880 2024-05-28T14:04:20.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a87e00640 1 -- 172.21.15.42:0/3364958064 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a800bd790 con 0x7f8a900af880 2024-05-28T14:04:20.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a87e00640 1 -- 172.21.15.42:0/3364958064 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a800bdb40 con 0x7f8a900af880 2024-05-28T14:04:20.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a96000640 1 -- 172.21.15.42:0/3364958064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9015f820 con 0x7f8a900af880 2024-05-28T14:04:20.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a96000640 1 -- 172.21.15.42:0/3364958064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9015a8e0 con 0x7f8a900af880 2024-05-28T14:04:20.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.183+0000 7f8a96000640 1 -- 172.21.15.42:0/3364958064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a90068a90 con 0x7f8a900af880 2024-05-28T14:04:20.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.187+0000 7f8a87e00640 1 -- 172.21.15.42:0/3364958064 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f8a800c6430 con 0x7f8a900af880 2024-05-28T14:04:20.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.187+0000 7f8a87e00640 1 --2- 172.21.15.42:0/3364958064 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a6c068b60 0x7f8a6c06b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:20.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.187+0000 7f8a87e00640 1 -- 172.21.15.42:0/3364958064 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4885+0+0 (secure 0 0 0) 0x7f8a80122110 con 0x7f8a900af880 2024-05-28T14:04:20.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.187+0000 7f8a86a00640 1 --2- 172.21.15.42:0/3364958064 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a6c068b60 0x7f8a6c06b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:20.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.187+0000 7f8a86a00640 1 --2- 172.21.15.42:0/3364958064 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a6c068b60 0x7f8a6c06b020 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8a78006fd0 tx=0x7f8a78008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:20.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.191+0000 7f8a87e00640 1 -- 172.21.15.42:0/3364958064 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8a9015f2c0 con 0x7f8a900af880 2024-05-28T14:04:20.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.359+0000 7f8a96000640 1 -- 172.21.15.42:0/3364958064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8a90151200 con 0x7f8a900af880 2024-05-28T14:04:20.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.359+0000 7f8a87e00640 1 -- 172.21.15.42:0/3364958064 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v57) ==== 84+0+65 (secure 0 0 0) 0x7f8a800160d0 con 0x7f8a900af880 2024-05-28T14:04:20.344 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:20.344 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:04:20.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:20.345 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-05-28T14:04:20.345 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-05-28T14:04:20.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:20.345 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:04:20.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.359+0000 7f8a96000640 1 -- 172.21.15.42:0/3364958064 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a6c068b60 msgr2=0x7f8a6c06b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:20.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.359+0000 7f8a96000640 1 --2- 172.21.15.42:0/3364958064 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a6c068b60 0x7f8a6c06b020 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8a78006fd0 tx=0x7f8a78008040 comp rx=0 tx=0).stop 2024-05-28T14:04:20.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.359+0000 7f8a96000640 1 -- 172.21.15.42:0/3364958064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a900af880 msgr2=0x7f8a90150c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:20.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.359+0000 7f8a96000640 1 --2- 172.21.15.42:0/3364958064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a900af880 0x7f8a90150c90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8a8001a110 tx=0x7f8a8001ad20 comp rx=0 tx=0).stop 2024-05-28T14:04:20.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.359+0000 7f8a96000640 1 -- 172.21.15.42:0/3364958064 shutdown_connections 2024-05-28T14:04:20.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.359+0000 7f8a96000640 1 --2- 172.21.15.42:0/3364958064 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8a6c068b60 0x7f8a6c06b020 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.359+0000 7f8a96000640 1 --2- 172.21.15.42:0/3364958064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a90156e50 0x7f8a9014e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.359+0000 7f8a96000640 1 --2- 172.21.15.42:0/3364958064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a90156a70 0x7f8a9015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.359+0000 7f8a96000640 1 --2- 172.21.15.42:0/3364958064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a900af880 0x7f8a90150c90 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.359+0000 7f8a96000640 1 -- 172.21.15.42:0/3364958064 >> 172.21.15.42:0/3364958064 conn(0x7f8a9006bdf0 msgr2=0x7f8a900b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:20.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.359+0000 7f8a96000640 1 -- 172.21.15.42:0/3364958064 shutdown_connections 2024-05-28T14:04:20.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.359+0000 7f8a96000640 1 -- 172.21.15.42:0/3364958064 wait complete. 2024-05-28T14:04:20.366 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-28T14:04:20.366 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-28T14:04:20.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.523+0000 7fb940c00640 1 Processor -- start 2024-05-28T14:04:20.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.523+0000 7fb940c00640 1 -- start start 2024-05-28T14:04:20.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.527+0000 7fb940c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb93c0d0e00 0x7fb93c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:20.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.527+0000 7fb940c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb93c072d40 0x7fb93c0aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:20.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.527+0000 7fb940c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb93c0ade40 con 0x7fb93c15b350 2024-05-28T14:04:20.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.527+0000 7fb940c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb93c0ae9f0 con 0x7fb93c0d0e00 2024-05-28T14:04:20.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.527+0000 7fb940c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb93c0ae0c0 con 0x7fb93c072d40 2024-05-28T14:04:20.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.527+0000 7fb933e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb93c0d0e00 0x7fb93c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:20.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.527+0000 7fb933e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb93c0d0e00 0x7fb93c06dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34428/0 (socket says 172.21.15.42:34428) 2024-05-28T14:04:20.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.527+0000 7fb933e00640 1 -- 172.21.15.42:0/3351906265 learned_addr learned my addr 172.21.15.42:0/3351906265 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:20.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.527+0000 7fb93a800640 1 --2- 172.21.15.42:0/3351906265 >> v2:172.21.15.63:3300/0 conn(0x7fb93c072d40 0x7fb93c0aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:20.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.527+0000 7fb933e00640 1 -- 172.21.15.42:0/3351906265 >> v2:172.21.15.63:3300/0 conn(0x7fb93c072d40 msgr2=0x7fb93c0aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:20.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.527+0000 7fb933e00640 1 --2- 172.21.15.42:0/3351906265 >> v2:172.21.15.63:3300/0 conn(0x7fb93c072d40 0x7fb93c0aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.527+0000 7fb933e00640 1 -- 172.21.15.42:0/3351906265 >> v1:172.21.15.63:6789/0 conn(0x7fb93c15b350 legacy=0x7fb93c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:20.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.527+0000 7fb933e00640 1 -- 172.21.15.42:0/3351906265 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c0ae3a0 con 0x7fb93c0d0e00 2024-05-28T14:04:20.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb933e00640 1 --2- 172.21.15.42:0/3351906265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93c0d0e00 0x7fb93c06dd40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0ad980 tx=0x7fb92401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3e7d4d5a304518 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:20.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb939400640 1 -- 172.21.15.42:0/3351906265 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240b0070 con 0x7fb93c0d0e00 2024-05-28T14:04:20.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb939400640 1 -- 172.21.15.42:0/3351906265 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9240ab600 con 0x7fb93c0d0e00 2024-05-28T14:04:20.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb939400640 1 -- 172.21.15.42:0/3351906265 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240ab9d0 con 0x7fb93c0d0e00 2024-05-28T14:04:20.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 -- 172.21.15.42:0/3351906265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93c0d0e00 msgr2=0x7fb93c06dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:20.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 --2- 172.21.15.42:0/3351906265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93c0d0e00 0x7fb93c06dd40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0ad980 tx=0x7fb92401c400 comp rx=0 tx=0).stop 2024-05-28T14:04:20.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 -- 172.21.15.42:0/3351906265 shutdown_connections 2024-05-28T14:04:20.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 --2- 172.21.15.42:0/3351906265 >> v2:172.21.15.63:3300/0 conn(0x7fb93c072d40 0x7fb93c0aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 --2- 172.21.15.42:0/3351906265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93c0d0e00 0x7fb93c06dd40 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 -- 172.21.15.42:0/3351906265 >> 172.21.15.42:0/3351906265 conn(0x7fb93c06bdf0 msgr2=0x7fb93c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:20.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 -- 172.21.15.42:0/3351906265 shutdown_connections 2024-05-28T14:04:20.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 -- 172.21.15.42:0/3351906265 wait complete. 2024-05-28T14:04:20.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 Processor -- start 2024-05-28T14:04:20.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 -- start start 2024-05-28T14:04:20.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93c072d40 0x7fb93c1fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:20.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb93c0d0e00 0x7fb93c14d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:20.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb93c15b350 0x7fb93c164950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:20.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c168470 con 0x7fb93c072d40 2024-05-28T14:04:20.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1682f0 con 0x7fb93c15b350 2024-05-28T14:04:20.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.531+0000 7fb940c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb93c1685f0 con 0x7fb93c0d0e00 2024-05-28T14:04:20.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb939e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93c072d40 0x7fb93c1fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:20.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb939e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93c072d40 0x7fb93c1fbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34436/0 (socket says 172.21.15.42:34436) 2024-05-28T14:04:20.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb939e00640 1 -- 172.21.15.42:0/1883523199 learned_addr learned my addr 172.21.15.42:0/1883523199 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:20.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb933e00640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb93c0d0e00 0x7fb93c14d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:20.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb93a800640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb93c15b350 0x7fb93c164950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:20.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb939e00640 1 -- 172.21.15.42:0/1883523199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb93c0d0e00 msgr2=0x7fb93c14d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:20.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb939e00640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb93c0d0e00 0x7fb93c14d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb939e00640 1 -- 172.21.15.42:0/1883523199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb93c15b350 msgr2=0x7fb93c164950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:20.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb939e00640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb93c15b350 0x7fb93c164950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:20.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb939e00640 1 -- 172.21.15.42:0/1883523199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c14ddd0 con 0x7fb93c072d40 2024-05-28T14:04:20.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb93a800640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb93c15b350 0x7fb93c164950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:20.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb939e00640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93c072d40 0x7fb93c1fbae0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00c370 tx=0x7fb92c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:20.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb932000640 1 -- 172.21.15.42:0/1883523199 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0052d0 con 0x7fb93c072d40 2024-05-28T14:04:20.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb932000640 1 -- 172.21.15.42:0/1883523199 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb92c005470 con 0x7fb93c072d40 2024-05-28T14:04:20.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb932000640 1 -- 172.21.15.42:0/1883523199 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c007260 con 0x7fb93c072d40 2024-05-28T14:04:20.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb940c00640 1 -- 172.21.15.42:0/1883523199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93c14e7d0 con 0x7fb93c072d40 2024-05-28T14:04:20.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb940c00640 1 -- 172.21.15.42:0/1883523199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93c14ed90 con 0x7fb93c072d40 2024-05-28T14:04:20.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb940c00640 1 -- 172.21.15.42:0/1883523199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb93c068a90 con 0x7fb93c072d40 2024-05-28T14:04:20.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.535+0000 7fb932000640 1 -- 172.21.15.42:0/1883523199 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fb92c007400 con 0x7fb93c072d40 2024-05-28T14:04:20.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.539+0000 7fb932000640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb918068b60 0x7fb91806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:20.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.539+0000 7fb932000640 1 -- 172.21.15.42:0/1883523199 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4885+0+0 (secure 0 0 0) 0x7fb92c084780 con 0x7fb93c072d40 2024-05-28T14:04:20.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.539+0000 7fb933e00640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb918068b60 0x7fb91806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:20.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.539+0000 7fb933e00640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb918068b60 0x7fb91806b020 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb9240ad000 tx=0x7fb924008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.543+0000 7fb932000640 1 -- 172.21.15.42:0/1883523199 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb93c0ade40 con 0x7fb93c072d40 2024-05-28T14:04:20.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:20.707+0000 7fb940c00640 1 -- 172.21.15.42:0/1883523199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb93c0b2d20 con 0x7fb93c072d40 2024-05-28T14:04:21.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.539+0000 7fb932000640 1 -- 172.21.15.42:0/1883523199 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v58) ==== 138+0+0 (secure 0 0 0) 0x7fb92c055fe0 con 0x7fb93c072d40 2024-05-28T14:04:21.525 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-05-28T14:04:21.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.543+0000 7fb940c00640 1 -- 172.21.15.42:0/1883523199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb918068b60 msgr2=0x7fb91806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:21.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.543+0000 7fb940c00640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb918068b60 0x7fb91806b020 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb9240ad000 tx=0x7fb924008040 comp rx=0 tx=0).stop 2024-05-28T14:04:21.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.543+0000 7fb940c00640 1 -- 172.21.15.42:0/1883523199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93c072d40 msgr2=0x7fb93c1fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:21.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.543+0000 7fb940c00640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93c072d40 0x7fb93c1fbae0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00c370 tx=0x7fb92c00c830 comp rx=0 tx=0).stop 2024-05-28T14:04:21.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.543+0000 7fb940c00640 1 -- 172.21.15.42:0/1883523199 shutdown_connections 2024-05-28T14:04:21.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.543+0000 7fb940c00640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb918068b60 0x7fb91806b020 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:21.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.543+0000 7fb940c00640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb93c15b350 0x7fb93c164950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:21.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.543+0000 7fb940c00640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb93c0d0e00 0x7fb93c14d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:21.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.543+0000 7fb940c00640 1 --2- 172.21.15.42:0/1883523199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93c072d40 0x7fb93c1fbae0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:21.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.543+0000 7fb940c00640 1 -- 172.21.15.42:0/1883523199 >> 172.21.15.42:0/1883523199 conn(0x7fb93c06bdf0 msgr2=0x7fb93c0b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:21.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.543+0000 7fb940c00640 1 -- 172.21.15.42:0/1883523199 shutdown_connections 2024-05-28T14:04:21.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.543+0000 7fb940c00640 1 -- 172.21.15.42:0/1883523199 wait complete. 2024-05-28T14:04:21.557 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-28T14:04:21.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.723+0000 7f256f600640 1 Processor -- start 2024-05-28T14:04:21.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.723+0000 7f256f600640 1 -- start start 2024-05-28T14:04:21.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.723+0000 7f256f600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2568071de0 0x7f25680721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:21.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.723+0000 7f256f600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f25680ce3b0 0x7f25680ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:21.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.723+0000 7f256f600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2568072700 con 0x7f2568141d10 2024-05-28T14:04:21.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.723+0000 7f256f600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2568160230 con 0x7f25680ce3b0 2024-05-28T14:04:21.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.723+0000 7f256f600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2568160510 con 0x7f2568071de0 2024-05-28T14:04:21.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.727+0000 7f2567e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f25680ce3b0 0x7f25680ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:21.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.727+0000 7f256cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2568071de0 0x7f25680721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:21.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.727+0000 7f2567e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f25680ce3b0 0x7f25680ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:44696/0 (socket says 172.21.15.42:44696) 2024-05-28T14:04:21.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.727+0000 7f2567e00640 1 -- 172.21.15.42:0/2938409875 learned_addr learned my addr 172.21.15.42:0/2938409875 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:21.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.727+0000 7f2567e00640 1 -- 172.21.15.42:0/2938409875 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2568071de0 msgr2=0x7f25680721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:21.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.727+0000 7f2567e00640 1 --2- 172.21.15.42:0/2938409875 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2568071de0 0x7f25680721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:21.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.727+0000 7f2567e00640 1 -- 172.21.15.42:0/2938409875 >> v1:172.21.15.63:6789/0 conn(0x7f2568141d10 legacy=0x7f25681421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:21.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.727+0000 7f2567e00640 1 -- 172.21.15.42:0/2938409875 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25681607f0 con 0x7f25680ce3b0 2024-05-28T14:04:21.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.727+0000 7f2567e00640 1 --2- 172.21.15.42:0/2938409875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25680ce3b0 0x7f25680ce810 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2554009870 tx=0x7f255401c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6aaa43bff61273 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:21.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.727+0000 7f2567400640 1 -- 172.21.15.42:0/2938409875 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25540b0070 con 0x7f25680ce3b0 2024-05-28T14:04:21.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.727+0000 7f2567400640 1 -- 172.21.15.42:0/2938409875 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25540ab600 con 0x7f25680ce3b0 2024-05-28T14:04:21.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.727+0000 7f2567400640 1 -- 172.21.15.42:0/2938409875 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25540ab9d0 con 0x7f25680ce3b0 2024-05-28T14:04:21.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 -- 172.21.15.42:0/2938409875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25680ce3b0 msgr2=0x7f25680ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:21.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 --2- 172.21.15.42:0/2938409875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25680ce3b0 0x7f25680ce810 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2554009870 tx=0x7f255401c440 comp rx=0 tx=0).stop 2024-05-28T14:04:21.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 -- 172.21.15.42:0/2938409875 shutdown_connections 2024-05-28T14:04:21.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 --2- 172.21.15.42:0/2938409875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25680ce3b0 0x7f25680ce810 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:21.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 --2- 172.21.15.42:0/2938409875 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2568071de0 0x7f25680721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:21.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 -- 172.21.15.42:0/2938409875 >> 172.21.15.42:0/2938409875 conn(0x7f25680b21f0 msgr2=0x7f25680b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:21.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 -- 172.21.15.42:0/2938409875 shutdown_connections 2024-05-28T14:04:21.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 -- 172.21.15.42:0/2938409875 wait complete. 2024-05-28T14:04:21.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 Processor -- start 2024-05-28T14:04:21.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 -- start start 2024-05-28T14:04:21.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568071de0 0x7f2568156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:21.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f25680ce3b0 0x7f256815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:21.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2568141d10 0x7f256815b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:21.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2568169e30 con 0x7f2568141d10 2024-05-28T14:04:21.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2568169cb0 con 0x7f2568071de0 2024-05-28T14:04:21.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256f600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2568169fb0 con 0x7f25680ce3b0 2024-05-28T14:04:21.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2568141d10 0x7f256815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:21.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568071de0 0x7f2568156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:21.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f2567e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f25680ce3b0 0x7f256815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:21.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2568141d10 0x7f256815b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34448/0 (socket says 172.21.15.42:34448) 2024-05-28T14:04:21.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.731+0000 7f256d600640 1 -- 172.21.15.42:0/3367175890 learned_addr learned my addr 172.21.15.42:0/3367175890 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:21.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.735+0000 7f2567e00640 1 -- 172.21.15.42:0/3367175890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568071de0 msgr2=0x7f2568156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:21.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.735+0000 7f2567e00640 1 --2- 172.21.15.42:0/3367175890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568071de0 0x7f2568156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:21.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.735+0000 7f2567e00640 1 -- 172.21.15.42:0/3367175890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2568141d10 msgr2=0x7f256815b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:21.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.735+0000 7f2567e00640 1 --2- 172.21.15.42:0/3367175890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2568141d10 0x7f256815b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:21.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.735+0000 7f2567e00640 1 -- 172.21.15.42:0/3367175890 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2568139bc0 con 0x7f25680ce3b0 2024-05-28T14:04:21.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.735+0000 7f256d600640 1 --2- 172.21.15.42:0/3367175890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2568141d10 0x7f256815b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:04:21.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.735+0000 7f2567e00640 1 --2- 172.21.15.42:0/3367175890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f25680ce3b0 0x7f256815ab50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f256806ffc0 tx=0x7f255401ac80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:21.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.735+0000 7f2564c00640 1 -- 172.21.15.42:0/3367175890 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25540ba020 con 0x7f25680ce3b0 2024-05-28T14:04:21.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.735+0000 7f2564c00640 1 -- 172.21.15.42:0/3367175890 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25540b6420 con 0x7f25680ce3b0 2024-05-28T14:04:21.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.735+0000 7f2564c00640 1 -- 172.21.15.42:0/3367175890 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25540b0040 con 0x7f25680ce3b0 2024-05-28T14:04:21.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.735+0000 7f256f600640 1 -- 172.21.15.42:0/3367175890 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2568139ea0 con 0x7f25680ce3b0 2024-05-28T14:04:21.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.735+0000 7f256f600640 1 -- 172.21.15.42:0/3367175890 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f256806e930 con 0x7f25680ce3b0 2024-05-28T14:04:21.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.735+0000 7f256f600640 1 -- 172.21.15.42:0/3367175890 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25680738b0 con 0x7f25680ce3b0 2024-05-28T14:04:21.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.735+0000 7f2564c00640 1 -- 172.21.15.42:0/3367175890 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f25540bd020 con 0x7f25680ce3b0 2024-05-28T14:04:21.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.739+0000 7f2564c00640 1 --2- 172.21.15.42:0/3367175890 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2548068b60 0x7f254806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:21.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.739+0000 7f2564c00640 1 -- 172.21.15.42:0/3367175890 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(58..58 src has 1..58) ==== 5255+0+0 (secure 0 0 0) 0x7f25541216f0 con 0x7f25680ce3b0 2024-05-28T14:04:21.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.739+0000 7f256cc00640 1 --2- 172.21.15.42:0/3367175890 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2548068b60 0x7f254806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:21.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.739+0000 7f256cc00640 1 --2- 172.21.15.42:0/3367175890 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2548068b60 0x7f254806b020 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f255c009ac0 tx=0x7f255c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:21.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.743+0000 7f2564c00640 1 -- 172.21.15.42:0/3367175890 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f25540c4030 con 0x7f25680ce3b0 2024-05-28T14:04:21.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:21.907+0000 7f256f600640 1 -- 172.21.15.42:0/3367175890 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f256804fdd0 con 0x7f25680ce3b0 2024-05-28T14:04:22.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.547+0000 7f2564c00640 1 -- 172.21.15.42:0/3367175890 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v59) ==== 170+0+0 (secure 0 0 0) 0x7f2554016150 con 0x7f25680ce3b0 2024-05-28T14:04:22.533 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-05-28T14:04:22.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.551+0000 7f256f600640 1 -- 172.21.15.42:0/3367175890 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2548068b60 msgr2=0x7f254806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:22.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.551+0000 7f256f600640 1 --2- 172.21.15.42:0/3367175890 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2548068b60 0x7f254806b020 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f255c009ac0 tx=0x7f255c009190 comp rx=0 tx=0).stop 2024-05-28T14:04:22.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.551+0000 7f256f600640 1 -- 172.21.15.42:0/3367175890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f25680ce3b0 msgr2=0x7f256815ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:22.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.551+0000 7f256f600640 1 --2- 172.21.15.42:0/3367175890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f25680ce3b0 0x7f256815ab50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f256806ffc0 tx=0x7f255401ac80 comp rx=0 tx=0).stop 2024-05-28T14:04:22.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.559+0000 7f256f600640 1 -- 172.21.15.42:0/3367175890 shutdown_connections 2024-05-28T14:04:22.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.559+0000 7f256f600640 1 --2- 172.21.15.42:0/3367175890 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2548068b60 0x7f254806b020 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:22.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.559+0000 7f256f600640 1 --2- 172.21.15.42:0/3367175890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2568141d10 0x7f256815b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:22.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.559+0000 7f256f600640 1 --2- 172.21.15.42:0/3367175890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f25680ce3b0 0x7f256815ab50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:22.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.559+0000 7f256f600640 1 --2- 172.21.15.42:0/3367175890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568071de0 0x7f2568156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:22.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.559+0000 7f256f600640 1 -- 172.21.15.42:0/3367175890 >> 172.21.15.42:0/3367175890 conn(0x7f25680b21f0 msgr2=0x7f2568141130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:22.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.563+0000 7f256f600640 1 -- 172.21.15.42:0/3367175890 shutdown_connections 2024-05-28T14:04:22.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.563+0000 7f256f600640 1 -- 172.21.15.42:0/3367175890 wait complete. 2024-05-28T14:04:22.579 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-28T14:04:22.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2163600640 1 Processor -- start 2024-05-28T14:04:22.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2163600640 1 -- start start 2024-05-28T14:04:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2163600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f215c15e8c0 0x7f215c0b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2163600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f215c0b0650 0x7f215c06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2163600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f215c0b4440 con 0x7f215c06bf30 2024-05-28T14:04:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2163600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f215c0ae420 con 0x7f215c0b0650 2024-05-28T14:04:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2163600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f215c06c370 con 0x7f215c15e8c0 2024-05-28T14:04:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2153e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f215c0b0650 0x7f215c06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2153e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f215c0b0650 0x7f215c06b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34450/0 (socket says 172.21.15.42:34450) 2024-05-28T14:04:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2153e00640 1 -- 172.21.15.42:0/3834344185 learned_addr learned my addr 172.21.15.42:0/3834344185 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2160c00640 1 --2- 172.21.15.42:0/3834344185 >> v2:172.21.15.63:3300/0 conn(0x7f215c15e8c0 0x7f215c0b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2153e00640 1 -- 172.21.15.42:0/3834344185 >> v2:172.21.15.63:3300/0 conn(0x7f215c15e8c0 msgr2=0x7f215c0b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2153e00640 1 --2- 172.21.15.42:0/3834344185 >> v2:172.21.15.63:3300/0 conn(0x7f215c15e8c0 0x7f215c0b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2153e00640 1 -- 172.21.15.42:0/3834344185 >> v1:172.21.15.42:6789/0 conn(0x7f215c06bf30 legacy=0x7f215c0cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2153e00640 1 -- 172.21.15.42:0/3834344185 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c0b0f10 con 0x7f215c0b0650 2024-05-28T14:04:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.739+0000 7f2160c00640 1 --2- 172.21.15.42:0/3834344185 >> v2:172.21.15.63:3300/0 conn(0x7f215c15e8c0 0x7f215c0b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:22.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.743+0000 7f2153e00640 1 --2- 172.21.15.42:0/3834344185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c0b0650 0x7f215c06b9f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f215c0b41b0 tx=0x7f214401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5121993f2522e307 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:22.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.743+0000 7f215be00640 1 -- 172.21.15.42:0/3834344185 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21440b0070 con 0x7f215c0b0650 2024-05-28T14:04:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.743+0000 7f215be00640 1 -- 172.21.15.42:0/3834344185 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2144004230 con 0x7f215c0b0650 2024-05-28T14:04:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.743+0000 7f215be00640 1 -- 172.21.15.42:0/3834344185 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144003620 con 0x7f215c0b0650 2024-05-28T14:04:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.743+0000 7f2163600640 1 -- 172.21.15.42:0/3834344185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c0b0650 msgr2=0x7f215c06b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.743+0000 7f2163600640 1 --2- 172.21.15.42:0/3834344185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c0b0650 0x7f215c06b9f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f215c0b41b0 tx=0x7f214401c460 comp rx=0 tx=0).stop 2024-05-28T14:04:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.743+0000 7f2163600640 1 -- 172.21.15.42:0/3834344185 shutdown_connections 2024-05-28T14:04:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.743+0000 7f2163600640 1 --2- 172.21.15.42:0/3834344185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c0b0650 0x7f215c06b9f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.743+0000 7f2163600640 1 --2- 172.21.15.42:0/3834344185 >> v2:172.21.15.63:3300/0 conn(0x7f215c15e8c0 0x7f215c0b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.743+0000 7f2163600640 1 -- 172.21.15.42:0/3834344185 >> 172.21.15.42:0/3834344185 conn(0x7f215c0b6960 msgr2=0x7f215c0b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.743+0000 7f2163600640 1 -- 172.21.15.42:0/3834344185 shutdown_connections 2024-05-28T14:04:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.743+0000 7f2163600640 1 -- 172.21.15.42:0/3834344185 wait complete. 2024-05-28T14:04:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2163600640 1 Processor -- start 2024-05-28T14:04:22.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2163600640 1 -- start start 2024-05-28T14:04:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2163600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c06bf30 0x7f215c151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2163600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0b0650 0x7f215c159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2163600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15e8c0 0x7f215c153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2163600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f215c15eca0 con 0x7f215c06bf30 2024-05-28T14:04:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2163600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f215c0b0160 con 0x7f215c15e8c0 2024-05-28T14:04:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2163600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f215c15ee20 con 0x7f215c0b0650 2024-05-28T14:04:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2160c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c06bf30 0x7f215c151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2160c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c06bf30 0x7f215c151450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34462/0 (socket says 172.21.15.42:34462) 2024-05-28T14:04:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2160c00640 1 -- 172.21.15.42:0/2247902400 learned_addr learned my addr 172.21.15.42:0/2247902400 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2161600640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15e8c0 0x7f215c153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2153e00640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0b0650 0x7f215c159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2160c00640 1 -- 172.21.15.42:0/2247902400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0b0650 msgr2=0x7f215c159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2160c00640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0b0650 0x7f215c159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2160c00640 1 -- 172.21.15.42:0/2247902400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15e8c0 msgr2=0x7f215c153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2160c00640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15e8c0 0x7f215c153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2160c00640 1 -- 172.21.15.42:0/2247902400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c151dc0 con 0x7f215c06bf30 2024-05-28T14:04:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2161600640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15e8c0 0x7f215c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2160c00640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c06bf30 0x7f215c151450 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f214c00d830 tx=0x7f214c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2159600640 1 -- 172.21.15.42:0/2247902400 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c00b6c0 con 0x7f215c06bf30 2024-05-28T14:04:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2159600640 1 -- 172.21.15.42:0/2247902400 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214c019070 con 0x7f215c06bf30 2024-05-28T14:04:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2159600640 1 -- 172.21.15.42:0/2247902400 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c00baa0 con 0x7f215c06bf30 2024-05-28T14:04:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2163600640 1 -- 172.21.15.42:0/2247902400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215c157260 con 0x7f215c06bf30 2024-05-28T14:04:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.747+0000 7f2163600640 1 -- 172.21.15.42:0/2247902400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f215c157820 con 0x7f215c06bf30 2024-05-28T14:04:22.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.751+0000 7f2163600640 1 -- 172.21.15.42:0/2247902400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f215c09f7a0 con 0x7f215c06bf30 2024-05-28T14:04:22.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.751+0000 7f2159600640 1 -- 172.21.15.42:0/2247902400 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f214c028430 con 0x7f215c06bf30 2024-05-28T14:04:22.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.751+0000 7f2159600640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2138068b60 0x7f213806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:22.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.751+0000 7f2159600640 1 -- 172.21.15.42:0/2247902400 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5637+0+0 (secure 0 0 0) 0x7f214c0888d0 con 0x7f215c06bf30 2024-05-28T14:04:22.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.751+0000 7f2153e00640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2138068b60 0x7f213806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:22.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.751+0000 7f2153e00640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2138068b60 0x7f213806b020 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f21440ad000 tx=0x7f2144009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:22.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.755+0000 7f2159600640 1 -- 172.21.15.42:0/2247902400 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f215c151dc0 con 0x7f215c06bf30 2024-05-28T14:04:22.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:22.955+0000 7f2163600640 1 -- 172.21.15.42:0/2247902400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f215c04fdd0 con 0x7f215c06bf30 2024-05-28T14:04:23.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.575+0000 7f2159600640 1 -- 172.21.15.42:0/2247902400 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v32) ==== 172+0+0 (secure 0 0 0) 0x7f214c0583d0 con 0x7f215c06bf30 2024-05-28T14:04:23.564 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 8 and data pool 9 2024-05-28T14:04:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.583+0000 7f2163600640 1 -- 172.21.15.42:0/2247902400 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2138068b60 msgr2=0x7f213806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.583+0000 7f2163600640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2138068b60 0x7f213806b020 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f21440ad000 tx=0x7f2144009040 comp rx=0 tx=0).stop 2024-05-28T14:04:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.583+0000 7f2163600640 1 -- 172.21.15.42:0/2247902400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c06bf30 msgr2=0x7f215c151450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.583+0000 7f2163600640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c06bf30 0x7f215c151450 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f214c00d830 tx=0x7f214c00dcf0 comp rx=0 tx=0).stop 2024-05-28T14:04:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.587+0000 7f2163600640 1 -- 172.21.15.42:0/2247902400 shutdown_connections 2024-05-28T14:04:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.587+0000 7f2163600640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2138068b60 0x7f213806b020 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.587+0000 7f2163600640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15e8c0 0x7f215c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.587+0000 7f2163600640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0b0650 0x7f215c159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.587+0000 7f2163600640 1 --2- 172.21.15.42:0/2247902400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c06bf30 0x7f215c151450 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.587+0000 7f2163600640 1 -- 172.21.15.42:0/2247902400 >> 172.21.15.42:0/2247902400 conn(0x7f215c0b6960 msgr2=0x7f215c15d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.587+0000 7f2163600640 1 -- 172.21.15.42:0/2247902400 shutdown_connections 2024-05-28T14:04:23.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.587+0000 7f2163600640 1 -- 172.21.15.42:0/2247902400 wait complete. 2024-05-28T14:04:23.604 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:04:23.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85f1c00640 1 Processor -- start 2024-05-28T14:04:23.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85f1c00640 1 -- start start 2024-05-28T14:04:23.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85f1c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f85ec15e9e0 0x7f85ec0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:23.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85f1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec0d0e40 0x7f85ec0b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:23.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85f1c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85ec0cdbd0 con 0x7f85ec0ce990 2024-05-28T14:04:23.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85f1c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f85ec0b2b70 con 0x7f85ec15e9e0 2024-05-28T14:04:23.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85f1c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f85ec15edc0 con 0x7f85ec0d0e40 2024-05-28T14:04:23.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85eaa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f85ec15e9e0 0x7f85ec0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:23.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85eaa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f85ec15e9e0 0x7f85ec0b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34464/0 (socket says 172.21.15.42:34464) 2024-05-28T14:04:23.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85eaa00640 1 -- 172.21.15.42:0/2669343557 learned_addr learned my addr 172.21.15.42:0/2669343557 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:23.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85e3e00640 1 --1- 172.21.15.42:0/2669343557 >> v1:172.21.15.42:6789/0 conn(0x7f85ec0ce990 0x7f85ec0d0730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:52600/0 (socket says 172.21.15.42:52600) 2024-05-28T14:04:23.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85eb400640 1 --2- 172.21.15.42:0/2669343557 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec0d0e40 0x7f85ec0b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:23.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85eaa00640 1 -- 172.21.15.42:0/2669343557 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec0d0e40 msgr2=0x7f85ec0b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:23.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85eaa00640 1 --2- 172.21.15.42:0/2669343557 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec0d0e40 0x7f85ec0b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:23.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85eaa00640 1 -- 172.21.15.42:0/2669343557 >> v1:172.21.15.42:6789/0 conn(0x7f85ec0ce990 legacy=0x7f85ec0d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:23.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85ea000640 1 -- 172.21.15.42:0/2669343557 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3819561187 0 0) 0x7f85ec0cdbd0 con 0x7f85ec0ce990 2024-05-28T14:04:23.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.763+0000 7f85eaa00640 1 -- 172.21.15.42:0/2669343557 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec0cdeb0 con 0x7f85ec15e9e0 2024-05-28T14:04:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.767+0000 7f85eaa00640 1 --2- 172.21.15.42:0/2669343557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec15e9e0 0x7f85ec0b20f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009870 tx=0x7f85dc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dd92295dda9753 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.767+0000 7f85ea000640 1 -- 172.21.15.42:0/2669343557 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0b0070 con 0x7f85ec15e9e0 2024-05-28T14:04:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.767+0000 7f85ea000640 1 -- 172.21.15.42:0/2669343557 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85dc0ab600 con 0x7f85ec15e9e0 2024-05-28T14:04:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.767+0000 7f85ea000640 1 -- 172.21.15.42:0/2669343557 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0ab9d0 con 0x7f85ec15e9e0 2024-05-28T14:04:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.767+0000 7f85f1c00640 1 -- 172.21.15.42:0/2669343557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec15e9e0 msgr2=0x7f85ec0b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.767+0000 7f85f1c00640 1 --2- 172.21.15.42:0/2669343557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec15e9e0 0x7f85ec0b20f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009870 tx=0x7f85dc01c440 comp rx=0 tx=0).stop 2024-05-28T14:04:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.767+0000 7f85f1c00640 1 -- 172.21.15.42:0/2669343557 shutdown_connections 2024-05-28T14:04:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.767+0000 7f85f1c00640 1 --2- 172.21.15.42:0/2669343557 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec0d0e40 0x7f85ec0b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.767+0000 7f85f1c00640 1 --2- 172.21.15.42:0/2669343557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec15e9e0 0x7f85ec0b20f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.767+0000 7f85f1c00640 1 -- 172.21.15.42:0/2669343557 >> 172.21.15.42:0/2669343557 conn(0x7f85ec09aa50 msgr2=0x7f85ec0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.767+0000 7f85f1c00640 1 -- 172.21.15.42:0/2669343557 shutdown_connections 2024-05-28T14:04:23.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.767+0000 7f85f1c00640 1 -- 172.21.15.42:0/2669343557 wait complete. 2024-05-28T14:04:23.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85f1c00640 1 Processor -- start 2024-05-28T14:04:23.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85f1c00640 1 -- start start 2024-05-28T14:04:23.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85f1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec0ce990 0x7f85ec1587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:23.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85f1c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec0d0e40 0x7f85ec150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:23.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85f1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85ec15e9e0 0x7f85ec15c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:23.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85f1c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec0b15b0 con 0x7f85ec0d0e40 2024-05-28T14:04:23.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85f1c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec0b1430 con 0x7f85ec15e9e0 2024-05-28T14:04:23.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85f1c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f85ec0b1730 con 0x7f85ec0ce990 2024-05-28T14:04:23.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85e3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec0d0e40 0x7f85ec150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:23.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85e3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec0d0e40 0x7f85ec150d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34480/0 (socket says 172.21.15.42:34480) 2024-05-28T14:04:23.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85e3e00640 1 -- 172.21.15.42:0/2891663809 learned_addr learned my addr 172.21.15.42:0/2891663809 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:23.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85eaa00640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec0ce990 0x7f85ec1587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:23.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85eb400640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85ec15e9e0 0x7f85ec15c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:23.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85e3e00640 1 -- 172.21.15.42:0/2891663809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec0ce990 msgr2=0x7f85ec1587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:23.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85e3e00640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec0ce990 0x7f85ec1587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:23.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85e3e00640 1 -- 172.21.15.42:0/2891663809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85ec15e9e0 msgr2=0x7f85ec15c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:23.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85e3e00640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85ec15e9e0 0x7f85ec15c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:23.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85e3e00640 1 -- 172.21.15.42:0/2891663809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec158d00 con 0x7f85ec0d0e40 2024-05-28T14:04:23.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85eaa00640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec0ce990 0x7f85ec1587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:23.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85e3e00640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec0d0e40 0x7f85ec150d50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f85d400c480 tx=0x7f85d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:23.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85e2a00640 1 -- 172.21.15.42:0/2891663809 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4018070 con 0x7f85ec0d0e40 2024-05-28T14:04:23.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85e2a00640 1 -- 172.21.15.42:0/2891663809 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d4007500 con 0x7f85ec0d0e40 2024-05-28T14:04:23.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85f1c00640 1 -- 172.21.15.42:0/2891663809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ec15d200 con 0x7f85ec0d0e40 2024-05-28T14:04:23.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85e2a00640 1 -- 172.21.15.42:0/2891663809 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4004a30 con 0x7f85ec0d0e40 2024-05-28T14:04:23.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85f1c00640 1 -- 172.21.15.42:0/2891663809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ec15d7c0 con 0x7f85ec0d0e40 2024-05-28T14:04:23.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.771+0000 7f85f1c00640 1 -- 172.21.15.42:0/2891663809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85ec068a90 con 0x7f85ec0d0e40 2024-05-28T14:04:23.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.775+0000 7f85e2a00640 1 -- 172.21.15.42:0/2891663809 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f85d4004bd0 con 0x7f85ec0d0e40 2024-05-28T14:04:23.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.775+0000 7f85e2a00640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85c8068b60 0x7f85c806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:23.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.775+0000 7f85e2a00640 1 -- 172.21.15.42:0/2891663809 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f85d4086f10 con 0x7f85ec0d0e40 2024-05-28T14:04:23.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.775+0000 7f85eaa00640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85c8068b60 0x7f85c806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:23.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.775+0000 7f85eaa00640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85c8068b60 0x7f85c806b020 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0ad000 tx=0x7f85dc0a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:23.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.779+0000 7f85e2a00640 1 -- 172.21.15.42:0/2891663809 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f85d40564a0 con 0x7f85ec0d0e40 2024-05-28T14:04:23.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.947+0000 7f85f1c00640 1 -- 172.21.15.42:0/2891663809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85ec0b7390 con 0x7f85ec0d0e40 2024-05-28T14:04:23.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.947+0000 7f85e2a00640 1 -- 172.21.15.42:0/2891663809 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16661 (secure 0 0 0) 0x7f85d4055da0 con 0x7f85ec0d0e40 2024-05-28T14:04:23.935 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:23.935 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":60,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:04:22.960486+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:04:20.721059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-28T14:04:21.922127+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:04:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.951+0000 7f85f1c00640 1 -- 172.21.15.42:0/2891663809 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85c8068b60 msgr2=0x7f85c806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.951+0000 7f85f1c00640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85c8068b60 0x7f85c806b020 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0ad000 tx=0x7f85dc0a6000 comp rx=0 tx=0).stop 2024-05-28T14:04:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.951+0000 7f85f1c00640 1 -- 172.21.15.42:0/2891663809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec0d0e40 msgr2=0x7f85ec150d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.951+0000 7f85f1c00640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec0d0e40 0x7f85ec150d50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f85d400c480 tx=0x7f85d400c940 comp rx=0 tx=0).stop 2024-05-28T14:04:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.951+0000 7f85f1c00640 1 -- 172.21.15.42:0/2891663809 shutdown_connections 2024-05-28T14:04:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.951+0000 7f85f1c00640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85c8068b60 0x7f85c806b020 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.951+0000 7f85f1c00640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85ec15e9e0 0x7f85ec15c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.951+0000 7f85f1c00640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec0d0e40 0x7f85ec150d50 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.951+0000 7f85f1c00640 1 --2- 172.21.15.42:0/2891663809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec0ce990 0x7f85ec1587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.951+0000 7f85f1c00640 1 -- 172.21.15.42:0/2891663809 >> 172.21.15.42:0/2891663809 conn(0x7f85ec09aa50 msgr2=0x7f85ec15b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.951+0000 7f85f1c00640 1 -- 172.21.15.42:0/2891663809 shutdown_connections 2024-05-28T14:04:23.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:23.951+0000 7f85f1c00640 1 -- 172.21.15.42:0/2891663809 wait complete. 2024-05-28T14:04:23.956 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:04:24.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc042c00640 1 Processor -- start 2024-05-28T14:04:24.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc042c00640 1 -- start start 2024-05-28T14:04:24.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc042c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc03c0ce860 0x7fc03c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:24.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc042c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c162c80 0x7fc03c15e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:24.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc042c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc03c0d0de0 con 0x7fc03c0d0900 2024-05-28T14:04:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc042c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc03c0d19f0 con 0x7fc03c0ce860 2024-05-28T14:04:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc042c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc03c0d10c0 con 0x7fc03c162c80 2024-05-28T14:04:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc033400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc03c0d0900 0x7fc03c162570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:52604/0 (socket says 172.21.15.42:52604) 2024-05-28T14:04:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc033400640 1 -- 172.21.15.42:0/2825013810 learned_addr learned my addr 172.21.15.42:0/2825013810 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc03be00640 1 --2- 172.21.15.42:0/2825013810 >> v2:172.21.15.63:3300/0 conn(0x7fc03c0ce860 0x7fc03c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc040c00640 1 --2- 172.21.15.42:0/2825013810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c162c80 0x7fc03c15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc03b400640 1 -- 172.21.15.42:0/2825013810 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4130216710 0 0) 0x7fc03c0d0de0 con 0x7fc03c0d0900 2024-05-28T14:04:24.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc03b400640 1 -- 172.21.15.42:0/2825013810 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc020003530 con 0x7fc03c0d0900 2024-05-28T14:04:24.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc03be00640 1 -- 172.21.15.42:0/2825013810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c162c80 msgr2=0x7fc03c15e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:24.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc03be00640 1 --2- 172.21.15.42:0/2825013810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c162c80 0x7fc03c15e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:24.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc03be00640 1 -- 172.21.15.42:0/2825013810 >> v1:172.21.15.42:6789/0 conn(0x7fc03c0d0900 legacy=0x7fc03c162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:24.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.111+0000 7fc03be00640 1 -- 172.21.15.42:0/2825013810 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c0d13a0 con 0x7fc03c0ce860 2024-05-28T14:04:24.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.115+0000 7fc03be00640 1 --2- 172.21.15.42:0/2825013810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0ce860 0x7fc03c06d7b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009d90 tx=0x7fc02c01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d6d610dda4a94a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:24.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.115+0000 7fc03b400640 1 -- 172.21.15.42:0/2825013810 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0b2070 con 0x7fc03c0ce860 2024-05-28T14:04:24.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.115+0000 7fc03b400640 1 -- 172.21.15.42:0/2825013810 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02c0a5990 con 0x7fc03c0ce860 2024-05-28T14:04:24.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.115+0000 7fc03b400640 1 -- 172.21.15.42:0/2825013810 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0a5d60 con 0x7fc03c0ce860 2024-05-28T14:04:24.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.115+0000 7fc042c00640 1 -- 172.21.15.42:0/2825013810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0ce860 msgr2=0x7fc03c06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:24.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.115+0000 7fc042c00640 1 --2- 172.21.15.42:0/2825013810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0ce860 0x7fc03c06d7b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009d90 tx=0x7fc02c01c8b0 comp rx=0 tx=0).stop 2024-05-28T14:04:24.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.115+0000 7fc042c00640 1 -- 172.21.15.42:0/2825013810 shutdown_connections 2024-05-28T14:04:24.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.115+0000 7fc042c00640 1 --2- 172.21.15.42:0/2825013810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c162c80 0x7fc03c15e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:24.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.115+0000 7fc042c00640 1 --2- 172.21.15.42:0/2825013810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0ce860 0x7fc03c06d7b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.115+0000 7fc042c00640 1 -- 172.21.15.42:0/2825013810 >> 172.21.15.42:0/2825013810 conn(0x7fc03c0b4080 msgr2=0x7fc03c0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.115+0000 7fc042c00640 1 -- 172.21.15.42:0/2825013810 shutdown_connections 2024-05-28T14:04:24.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.115+0000 7fc042c00640 1 -- 172.21.15.42:0/2825013810 wait complete. 2024-05-28T14:04:24.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc042c00640 1 Processor -- start 2024-05-28T14:04:24.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc042c00640 1 -- start start 2024-05-28T14:04:24.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc042c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0ce860 0x7fc03c151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:24.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc042c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0d0900 0x7fc03c15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:24.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc042c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c162c80 0x7fc03c15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:24.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc042c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c0ccf70 con 0x7fc03c162c80 2024-05-28T14:04:24.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc042c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c0ccdf0 con 0x7fc03c0ce860 2024-05-28T14:04:24.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc042c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc03c0cd0f0 con 0x7fc03c0d0900 2024-05-28T14:04:24.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc040c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c162c80 0x7fc03c15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:24.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc03be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0ce860 0x7fc03c151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:24.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc033400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0d0900 0x7fc03c15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:24.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc03be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0ce860 0x7fc03c151300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:44762/0 (socket says 172.21.15.42:44762) 2024-05-28T14:04:24.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc03be00640 1 -- 172.21.15.42:0/3911064073 learned_addr learned my addr 172.21.15.42:0/3911064073 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:24.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc040c00640 1 -- 172.21.15.42:0/3911064073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0d0900 msgr2=0x7fc03c15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:24.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc040c00640 1 --2- 172.21.15.42:0/3911064073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0d0900 0x7fc03c15a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:24.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc040c00640 1 -- 172.21.15.42:0/3911064073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0ce860 msgr2=0x7fc03c151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:24.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc040c00640 1 --2- 172.21.15.42:0/3911064073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0ce860 0x7fc03c151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:24.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc040c00640 1 -- 172.21.15.42:0/3911064073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c157a60 con 0x7fc03c162c80 2024-05-28T14:04:24.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc040c00640 1 --2- 172.21.15.42:0/3911064073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c162c80 0x7fc03c15a6b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc03400e7f0 tx=0x7fc03400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:24.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc038c00640 1 -- 172.21.15.42:0/3911064073 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034017070 con 0x7fc03c162c80 2024-05-28T14:04:24.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc038c00640 1 -- 172.21.15.42:0/3911064073 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03400c730 con 0x7fc03c162c80 2024-05-28T14:04:24.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc038c00640 1 -- 172.21.15.42:0/3911064073 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03400cb00 con 0x7fc03c162c80 2024-05-28T14:04:24.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc042c00640 1 -- 172.21.15.42:0/3911064073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03c155ba0 con 0x7fc03c162c80 2024-05-28T14:04:24.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.119+0000 7fc042c00640 1 -- 172.21.15.42:0/3911064073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc03c156160 con 0x7fc03c162c80 2024-05-28T14:04:24.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.123+0000 7fc042c00640 1 -- 172.21.15.42:0/3911064073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03c068a90 con 0x7fc03c162c80 2024-05-28T14:04:24.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.123+0000 7fc038c00640 1 -- 172.21.15.42:0/3911064073 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fc03401a070 con 0x7fc03c162c80 2024-05-28T14:04:24.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.123+0000 7fc038c00640 1 --2- 172.21.15.42:0/3911064073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc018068b60 0x7fc01806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:24.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.123+0000 7fc038c00640 1 -- 172.21.15.42:0/3911064073 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fc034087a60 con 0x7fc03c162c80 2024-05-28T14:04:24.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.123+0000 7fc03be00640 1 --2- 172.21.15.42:0/3911064073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc018068b60 0x7fc01806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:24.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.123+0000 7fc03be00640 1 --2- 172.21.15.42:0/3911064073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc018068b60 0x7fc01806b020 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009d90 tx=0x7fc02c0a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:24.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.127+0000 7fc038c00640 1 -- 172.21.15.42:0/3911064073 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc034056990 con 0x7fc03c162c80 2024-05-28T14:04:24.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.295+0000 7fc042c00640 1 -- 172.21.15.42:0/3911064073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc03c0b4ce0 con 0x7fc03c162c80 2024-05-28T14:04:24.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.295+0000 7fc038c00640 1 -- 172.21.15.42:0/3911064073 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16661 (secure 0 0 0) 0x7fc03405be20 con 0x7fc03c162c80 2024-05-28T14:04:24.282 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:24.282 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":60,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:04:22.960486+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:04:20.721059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-28T14:04:21.922127+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:04:24.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.299+0000 7fc042c00640 1 -- 172.21.15.42:0/3911064073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc018068b60 msgr2=0x7fc01806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:24.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.299+0000 7fc042c00640 1 --2- 172.21.15.42:0/3911064073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc018068b60 0x7fc01806b020 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009d90 tx=0x7fc02c0a7070 comp rx=0 tx=0).stop 2024-05-28T14:04:24.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.299+0000 7fc042c00640 1 -- 172.21.15.42:0/3911064073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c162c80 msgr2=0x7fc03c15a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:24.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.299+0000 7fc042c00640 1 --2- 172.21.15.42:0/3911064073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c162c80 0x7fc03c15a6b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc03400e7f0 tx=0x7fc03400ecb0 comp rx=0 tx=0).stop 2024-05-28T14:04:24.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.299+0000 7fc042c00640 1 -- 172.21.15.42:0/3911064073 shutdown_connections 2024-05-28T14:04:24.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.299+0000 7fc042c00640 1 --2- 172.21.15.42:0/3911064073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc018068b60 0x7fc01806b020 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:24.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.299+0000 7fc042c00640 1 --2- 172.21.15.42:0/3911064073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c162c80 0x7fc03c15a6b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:24.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.299+0000 7fc042c00640 1 --2- 172.21.15.42:0/3911064073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0d0900 0x7fc03c15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:24.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.299+0000 7fc042c00640 1 --2- 172.21.15.42:0/3911064073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0ce860 0x7fc03c151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:24.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.299+0000 7fc042c00640 1 -- 172.21.15.42:0/3911064073 >> 172.21.15.42:0/3911064073 conn(0x7fc03c0b4080 msgr2=0x7fc03c09aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:24.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.299+0000 7fc042c00640 1 -- 172.21.15.42:0/3911064073 shutdown_connections 2024-05-28T14:04:24.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.299+0000 7fc042c00640 1 -- 172.21.15.42:0/3911064073 wait complete. 2024-05-28T14:04:24.305 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-28T14:04:24.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.463+0000 7fdf7fe00640 1 Processor -- start 2024-05-28T14:04:24.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.463+0000 7fdf7fe00640 1 -- start start 2024-05-28T14:04:24.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.463+0000 7fdf7fe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdf800d4c00 0x7fdf800d3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:24.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.463+0000 7fdf7fe00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf800cedd0 con 0x7fdf800b6680 2024-05-28T14:04:24.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.463+0000 7fdf7fe00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf800d26a0 con 0x7fdf8015e7f0 2024-05-28T14:04:24.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.463+0000 7fdf7fe00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdf800d3270 con 0x7fdf800d4c00 2024-05-28T14:04:24.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.463+0000 7fdf76000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fdf800b6680 0x7fdf8006c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:52620/0 (socket says 172.21.15.42:52620) 2024-05-28T14:04:24.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.463+0000 7fdf76000640 1 -- 172.21.15.42:0/636251842 learned_addr learned my addr 172.21.15.42:0/636251842 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.463+0000 7fdf7f400640 1 --2- 172.21.15.42:0/636251842 >> v2:172.21.15.63:3300/0 conn(0x7fdf800d4c00 0x7fdf800d3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.467+0000 7fdf7e000640 1 -- 172.21.15.42:0/636251842 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3674264755 0 0) 0x7fdf800cedd0 con 0x7fdf800b6680 2024-05-28T14:04:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.467+0000 7fdf7e000640 1 -- 172.21.15.42:0/636251842 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf64003530 con 0x7fdf800b6680 2024-05-28T14:04:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.467+0000 7fdf7e000640 1 -- 172.21.15.42:0/636251842 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800060446 0 0) 0x7fdf800d26a0 con 0x7fdf8015e7f0 2024-05-28T14:04:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.467+0000 7fdf7e000640 1 -- 172.21.15.42:0/636251842 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf800cedd0 con 0x7fdf8015e7f0 2024-05-28T14:04:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.467+0000 7fdf7f400640 1 -- 172.21.15.42:0/636251842 >> v1:172.21.15.63:6789/0 conn(0x7fdf8015e7f0 legacy=0x7fdf800b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.467+0000 7fdf7f400640 1 -- 172.21.15.42:0/636251842 >> v1:172.21.15.42:6789/0 conn(0x7fdf800b6680 legacy=0x7fdf8006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.467+0000 7fdf7f400640 1 -- 172.21.15.42:0/636251842 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf800d2940 con 0x7fdf800d4c00 2024-05-28T14:04:24.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.467+0000 7fdf7f400640 1 --2- 172.21.15.42:0/636251842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf800d4c00 0x7fdf800d3700 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009d90 tx=0x7fdf8006d750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=384c9771f7a3da00 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:24.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.467+0000 7fdf7e000640 1 -- 172.21.15.42:0/636251842 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf780b2070 con 0x7fdf800d4c00 2024-05-28T14:04:24.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.467+0000 7fdf7e000640 1 -- 172.21.15.42:0/636251842 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf780a5830 con 0x7fdf800d4c00 2024-05-28T14:04:24.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.467+0000 7fdf7e000640 1 -- 172.21.15.42:0/636251842 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf780a5c00 con 0x7fdf800d4c00 2024-05-28T14:04:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 -- 172.21.15.42:0/636251842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf800d4c00 msgr2=0x7fdf800d3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 --2- 172.21.15.42:0/636251842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf800d4c00 0x7fdf800d3700 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009d90 tx=0x7fdf8006d750 comp rx=0 tx=0).stop 2024-05-28T14:04:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 -- 172.21.15.42:0/636251842 shutdown_connections 2024-05-28T14:04:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 --2- 172.21.15.42:0/636251842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf800d4c00 0x7fdf800d3700 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 -- 172.21.15.42:0/636251842 >> 172.21.15.42:0/636251842 conn(0x7fdf8008bfe0 msgr2=0x7fdf800d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:24.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 -- 172.21.15.42:0/636251842 shutdown_connections 2024-05-28T14:04:24.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 -- 172.21.15.42:0/636251842 wait complete. 2024-05-28T14:04:24.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 Processor -- start 2024-05-28T14:04:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 -- start start 2024-05-28T14:04:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf800b6680 0x7fdf80154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf800d4c00 0x7fdf800c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf800c8560 0x7fdf801566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8016c910 con 0x7fdf800c8560 2024-05-28T14:04:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8016c790 con 0x7fdf800d4c00 2024-05-28T14:04:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7fe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8016ca90 con 0x7fdf800b6680 2024-05-28T14:04:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf800c8560 0x7fdf801566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf800c8560 0x7fdf801566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34508/0 (socket says 172.21.15.42:34508) 2024-05-28T14:04:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7f400640 1 -- 172.21.15.42:0/1728246333 learned_addr learned my addr 172.21.15.42:0/1728246333 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf76000640 1 --2- 172.21.15.42:0/1728246333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf800d4c00 0x7fdf800c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7ea00640 1 --2- 172.21.15.42:0/1728246333 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf800b6680 0x7fdf80154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7f400640 1 -- 172.21.15.42:0/1728246333 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf800b6680 msgr2=0x7fdf80154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7f400640 1 --2- 172.21.15.42:0/1728246333 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf800b6680 0x7fdf80154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7f400640 1 -- 172.21.15.42:0/1728246333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf800d4c00 msgr2=0x7fdf800c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7f400640 1 --2- 172.21.15.42:0/1728246333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf800d4c00 0x7fdf800c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:24.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7f400640 1 -- 172.21.15.42:0/1728246333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf800c8a00 con 0x7fdf800c8560 2024-05-28T14:04:24.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.471+0000 7fdf7f400640 1 --2- 172.21.15.42:0/1728246333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf800c8560 0x7fdf801566d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fdf78019580 tx=0x7fdf78019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:24.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.475+0000 7fdf77e00640 1 -- 172.21.15.42:0/1728246333 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf780b2040 con 0x7fdf800c8560 2024-05-28T14:04:24.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.475+0000 7fdf77e00640 1 -- 172.21.15.42:0/1728246333 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf780a5630 con 0x7fdf800c8560 2024-05-28T14:04:24.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.475+0000 7fdf77e00640 1 -- 172.21.15.42:0/1728246333 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf780b86b0 con 0x7fdf800c8560 2024-05-28T14:04:24.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.475+0000 7fdf7fe00640 1 -- 172.21.15.42:0/1728246333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf800c8ce0 con 0x7fdf800c8560 2024-05-28T14:04:24.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.475+0000 7fdf7fe00640 1 -- 172.21.15.42:0/1728246333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf800c92a0 con 0x7fdf800c8560 2024-05-28T14:04:24.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.475+0000 7fdf7fe00640 1 -- 172.21.15.42:0/1728246333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf8004fcf0 con 0x7fdf800c8560 2024-05-28T14:04:24.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.475+0000 7fdf77e00640 1 -- 172.21.15.42:0/1728246333 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fdf780c1020 con 0x7fdf800c8560 2024-05-28T14:04:24.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.475+0000 7fdf77e00640 1 --2- 172.21.15.42:0/1728246333 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf5c068b90 0x7fdf5c06b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:24.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.475+0000 7fdf77e00640 1 -- 172.21.15.42:0/1728246333 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fdf78122900 con 0x7fdf800c8560 2024-05-28T14:04:24.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.475+0000 7fdf7ea00640 1 --2- 172.21.15.42:0/1728246333 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf5c068b90 0x7fdf5c06b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:24.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.479+0000 7fdf7ea00640 1 --2- 172.21.15.42:0/1728246333 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf5c068b90 0x7fdf5c06b050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdf700015e0 tx=0x7fdf70009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:24.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.479+0000 7fdf77e00640 1 -- 172.21.15.42:0/1728246333 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdf780c8020 con 0x7fdf800c8560 2024-05-28T14:04:24.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:24.683+0000 7fdf7fe00640 1 -- 172.21.15.42:0/1728246333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdf800d4260 con 0x7fdf800c8560 2024-05-28T14:04:25.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.587+0000 7fdf77e00640 1 -- 172.21.15.42:0/1728246333 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7fdf78016560 con 0x7fdf800c8560 2024-05-28T14:04:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.587+0000 7fdf7fe00640 1 -- 172.21.15.42:0/1728246333 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf5c068b90 msgr2=0x7fdf5c06b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.587+0000 7fdf7fe00640 1 --2- 172.21.15.42:0/1728246333 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf5c068b90 0x7fdf5c06b050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdf700015e0 tx=0x7fdf70009040 comp rx=0 tx=0).stop 2024-05-28T14:04:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.587+0000 7fdf7fe00640 1 -- 172.21.15.42:0/1728246333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf800c8560 msgr2=0x7fdf801566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.587+0000 7fdf7fe00640 1 --2- 172.21.15.42:0/1728246333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf800c8560 0x7fdf801566d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fdf78019580 tx=0x7fdf78019710 comp rx=0 tx=0).stop 2024-05-28T14:04:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.587+0000 7fdf7fe00640 1 -- 172.21.15.42:0/1728246333 shutdown_connections 2024-05-28T14:04:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.587+0000 7fdf7fe00640 1 --2- 172.21.15.42:0/1728246333 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf5c068b90 0x7fdf5c06b050 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:25.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.587+0000 7fdf7fe00640 1 --2- 172.21.15.42:0/1728246333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf800c8560 0x7fdf801566d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:25.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.587+0000 7fdf7fe00640 1 --2- 172.21.15.42:0/1728246333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf800d4c00 0x7fdf800c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:25.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.587+0000 7fdf7fe00640 1 --2- 172.21.15.42:0/1728246333 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf800b6680 0x7fdf80154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:25.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.587+0000 7fdf7fe00640 1 -- 172.21.15.42:0/1728246333 >> 172.21.15.42:0/1728246333 conn(0x7fdf8008bfe0 msgr2=0x7fdf800b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:25.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.587+0000 7fdf7fe00640 1 -- 172.21.15.42:0/1728246333 shutdown_connections 2024-05-28T14:04:25.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.587+0000 7fdf7fe00640 1 -- 172.21.15.42:0/1728246333 wait complete. 2024-05-28T14:04:25.599 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-28T14:04:25.599 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-28T14:04:25.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.763+0000 7fd956800640 1 Processor -- start 2024-05-28T14:04:25.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.763+0000 7fd956800640 1 -- start start 2024-05-28T14:04:25.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.763+0000 7fd956800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd950071de0 0x7fd9500721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:25.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.763+0000 7fd956800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd950072700 con 0x7fd9500ce3b0 2024-05-28T14:04:25.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.763+0000 7fd956800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd950160230 con 0x7fd95013fd10 2024-05-28T14:04:25.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.763+0000 7fd956800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd950160510 con 0x7fd950071de0 2024-05-28T14:04:25.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.763+0000 7fd94f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd950071de0 0x7fd9500721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:25.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.763+0000 7fd94ea00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd9500ce3b0 0x7fd9500ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:52624/0 (socket says 172.21.15.42:52624) 2024-05-28T14:04:25.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.763+0000 7fd94ea00640 1 -- 172.21.15.42:0/156333572 learned_addr learned my addr 172.21.15.42:0/156333572 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:25.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.763+0000 7fd94e000640 1 -- 172.21.15.42:0/156333572 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2385572691 0 0) 0x7fd950072700 con 0x7fd9500ce3b0 2024-05-28T14:04:25.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.763+0000 7fd94e000640 1 -- 172.21.15.42:0/156333572 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd934003530 con 0x7fd9500ce3b0 2024-05-28T14:04:25.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.763+0000 7fd94f400640 1 -- 172.21.15.42:0/156333572 >> v1:172.21.15.63:6789/0 conn(0x7fd95013fd10 legacy=0x7fd950140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:25.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.763+0000 7fd94f400640 1 -- 172.21.15.42:0/156333572 >> v1:172.21.15.42:6789/0 conn(0x7fd9500ce3b0 legacy=0x7fd9500ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:25.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.763+0000 7fd94f400640 1 -- 172.21.15.42:0/156333572 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9501607f0 con 0x7fd950071de0 2024-05-28T14:04:25.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.767+0000 7fd94f400640 1 --2- 172.21.15.42:0/156333572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd950071de0 0x7fd9500721c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd9400097c0 tx=0x7fd94001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d07d7144754bd64 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:25.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.767+0000 7fd94e000640 1 -- 172.21.15.42:0/156333572 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9400b0070 con 0x7fd950071de0 2024-05-28T14:04:25.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.767+0000 7fd94e000640 1 -- 172.21.15.42:0/156333572 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9400ab600 con 0x7fd950071de0 2024-05-28T14:04:25.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.767+0000 7fd94e000640 1 -- 172.21.15.42:0/156333572 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9400ab9b0 con 0x7fd950071de0 2024-05-28T14:04:25.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.767+0000 7fd956800640 1 -- 172.21.15.42:0/156333572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd950071de0 msgr2=0x7fd9500721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:25.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.767+0000 7fd956800640 1 --2- 172.21.15.42:0/156333572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd950071de0 0x7fd9500721c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd9400097c0 tx=0x7fd94001c380 comp rx=0 tx=0).stop 2024-05-28T14:04:25.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.767+0000 7fd956800640 1 -- 172.21.15.42:0/156333572 shutdown_connections 2024-05-28T14:04:25.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.767+0000 7fd956800640 1 --2- 172.21.15.42:0/156333572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd950071de0 0x7fd9500721c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:25.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.767+0000 7fd956800640 1 -- 172.21.15.42:0/156333572 >> 172.21.15.42:0/156333572 conn(0x7fd9500b21f0 msgr2=0x7fd9500b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:25.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.767+0000 7fd956800640 1 -- 172.21.15.42:0/156333572 shutdown_connections 2024-05-28T14:04:25.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.767+0000 7fd956800640 1 -- 172.21.15.42:0/156333572 wait complete. 2024-05-28T14:04:25.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd956800640 1 Processor -- start 2024-05-28T14:04:25.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd956800640 1 -- start start 2024-05-28T14:04:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd956800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd950071de0 0x7fd95013a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd956800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9500ce3b0 0x7fd950139c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd956800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd95013fd10 0x7fd9501571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd956800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd95016a840 con 0x7fd9500ce3b0 2024-05-28T14:04:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd956800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd95016a6c0 con 0x7fd95013fd10 2024-05-28T14:04:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd956800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd95016a9c0 con 0x7fd950071de0 2024-05-28T14:04:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd94ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9500ce3b0 0x7fd950139c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd94ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9500ce3b0 0x7fd950139c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34526/0 (socket says 172.21.15.42:34526) 2024-05-28T14:04:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd94ea00640 1 -- 172.21.15.42:0/2760279071 learned_addr learned my addr 172.21.15.42:0/2760279071 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:25.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd94f400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd950071de0 0x7fd95013a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd94fe00640 1 --2- 172.21.15.42:0/2760279071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd95013fd10 0x7fd9501571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd94ea00640 1 -- 172.21.15.42:0/2760279071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd950071de0 msgr2=0x7fd95013a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd94ea00640 1 --2- 172.21.15.42:0/2760279071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd950071de0 0x7fd95013a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd94ea00640 1 -- 172.21.15.42:0/2760279071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd95013fd10 msgr2=0x7fd9501571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd94ea00640 1 --2- 172.21.15.42:0/2760279071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd95013fd10 0x7fd9501571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd94ea00640 1 -- 172.21.15.42:0/2760279071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9500b4730 con 0x7fd9500ce3b0 2024-05-28T14:04:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd94ea00640 1 --2- 172.21.15.42:0/2760279071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9500ce3b0 0x7fd950139c20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00c4d0 tx=0x7fd93c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd947400640 1 -- 172.21.15.42:0/2760279071 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c017070 con 0x7fd9500ce3b0 2024-05-28T14:04:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd947400640 1 -- 172.21.15.42:0/2760279071 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd93c005cf0 con 0x7fd9500ce3b0 2024-05-28T14:04:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd947400640 1 -- 172.21.15.42:0/2760279071 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c007270 con 0x7fd9500ce3b0 2024-05-28T14:04:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd956800640 1 -- 172.21.15.42:0/2760279071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9500b4a10 con 0x7fd9500ce3b0 2024-05-28T14:04:25.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd956800640 1 -- 172.21.15.42:0/2760279071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd95013a160 con 0x7fd9500ce3b0 2024-05-28T14:04:25.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.771+0000 7fd956800640 1 -- 172.21.15.42:0/2760279071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd950156b80 con 0x7fd9500ce3b0 2024-05-28T14:04:25.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.775+0000 7fd947400640 1 -- 172.21.15.42:0/2760279071 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fd93c01a070 con 0x7fd9500ce3b0 2024-05-28T14:04:25.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.775+0000 7fd947400640 1 --2- 172.21.15.42:0/2760279071 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd930068b60 0x7fd93006b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:25.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.775+0000 7fd947400640 1 -- 172.21.15.42:0/2760279071 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fd93c086f60 con 0x7fd9500ce3b0 2024-05-28T14:04:25.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.775+0000 7fd94f400640 1 --2- 172.21.15.42:0/2760279071 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd930068b60 0x7fd93006b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:25.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.775+0000 7fd94f400640 1 --2- 172.21.15.42:0/2760279071 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd930068b60 0x7fd93006b020 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd9400ad000 tx=0x7fd9400a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:25.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.779+0000 7fd947400640 1 -- 172.21.15.42:0/2760279071 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd950156b80 con 0x7fd9500ce3b0 2024-05-28T14:04:25.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:25.979+0000 7fd956800640 1 -- 172.21.15.42:0/2760279071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd9500738b0 con 0x7fd9500ce3b0 2024-05-28T14:04:26.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.587+0000 7fd947400640 1 -- 172.21.15.42:0/2760279071 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7fd93c055f20 con 0x7fd9500ce3b0 2024-05-28T14:04:26.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.591+0000 7fd956800640 1 -- 172.21.15.42:0/2760279071 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd930068b60 msgr2=0x7fd93006b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:26.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.591+0000 7fd956800640 1 --2- 172.21.15.42:0/2760279071 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd930068b60 0x7fd93006b020 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd9400ad000 tx=0x7fd9400a6000 comp rx=0 tx=0).stop 2024-05-28T14:04:26.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.591+0000 7fd956800640 1 -- 172.21.15.42:0/2760279071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9500ce3b0 msgr2=0x7fd950139c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:26.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.591+0000 7fd956800640 1 --2- 172.21.15.42:0/2760279071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9500ce3b0 0x7fd950139c20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00c4d0 tx=0x7fd93c00c990 comp rx=0 tx=0).stop 2024-05-28T14:04:26.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.591+0000 7fd956800640 1 -- 172.21.15.42:0/2760279071 shutdown_connections 2024-05-28T14:04:26.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.591+0000 7fd956800640 1 --2- 172.21.15.42:0/2760279071 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd930068b60 0x7fd93006b020 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.591+0000 7fd956800640 1 --2- 172.21.15.42:0/2760279071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd95013fd10 0x7fd9501571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.591+0000 7fd956800640 1 --2- 172.21.15.42:0/2760279071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9500ce3b0 0x7fd950139c20 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.591+0000 7fd956800640 1 --2- 172.21.15.42:0/2760279071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd950071de0 0x7fd95013a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.591+0000 7fd956800640 1 -- 172.21.15.42:0/2760279071 >> 172.21.15.42:0/2760279071 conn(0x7fd9500b21f0 msgr2=0x7fd9500b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:26.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.591+0000 7fd956800640 1 -- 172.21.15.42:0/2760279071 shutdown_connections 2024-05-28T14:04:26.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.591+0000 7fd956800640 1 -- 172.21.15.42:0/2760279071 wait complete. 2024-05-28T14:04:26.598 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-28T14:04:26.598 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:04:26.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.755+0000 7fea7d600640 1 Processor -- start 2024-05-28T14:04:26.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.755+0000 7fea7d600640 1 -- start start 2024-05-28T14:04:26.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea7d600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78156a70 0x7fea7806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:26.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea7d600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fea780afa50 0x7fea780d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:26.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea7d600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea7815cb90 con 0x7fea78156e50 2024-05-28T14:04:26.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea7d600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fea7815ce50 con 0x7fea780afa50 2024-05-28T14:04:26.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea7d600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea7815d130 con 0x7fea78156a70 2024-05-28T14:04:26.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea76000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fea78156e50 0x7fea780af340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:52626/0 (socket says 172.21.15.42:52626) 2024-05-28T14:04:26.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea76000640 1 -- 172.21.15.42:0/3257727483 learned_addr learned my addr 172.21.15.42:0/3257727483 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:26.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea76a00640 1 --2- 172.21.15.42:0/3257727483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78156a70 0x7fea7806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:26.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea77400640 1 --2- 172.21.15.42:0/3257727483 >> v2:172.21.15.63:3300/0 conn(0x7fea780afa50 0x7fea780d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:26.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 -- 172.21.15.42:0/3257727483 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4257193514 0 0) 0x7fea7815cb90 con 0x7fea78156e50 2024-05-28T14:04:26.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 -- 172.21.15.42:0/3257727483 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea5c003530 con 0x7fea78156e50 2024-05-28T14:04:26.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 -- 172.21.15.42:0/3257727483 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2813157541 0 0) 0x7fea5c003530 con 0x7fea78156e50 2024-05-28T14:04:26.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 -- 172.21.15.42:0/3257727483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78156a70 msgr2=0x7fea7806dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:26.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 --2- 172.21.15.42:0/3257727483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78156a70 0x7fea7806dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 -- 172.21.15.42:0/3257727483 >> v2:172.21.15.63:3300/0 conn(0x7fea780afa50 msgr2=0x7fea780d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:26.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 --2- 172.21.15.42:0/3257727483 >> v2:172.21.15.63:3300/0 conn(0x7fea780afa50 0x7fea780d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 -- 172.21.15.42:0/3257727483 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7815d410 con 0x7fea78156e50 2024-05-28T14:04:26.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 -- 172.21.15.42:0/3257727483 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fea64001680 con 0x7fea78156e50 2024-05-28T14:04:26.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 -- 172.21.15.42:0/3257727483 >> v1:172.21.15.42:6789/0 conn(0x7fea78156e50 legacy=0x7fea780af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:26.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 --2- 172.21.15.42:0/3257727483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78156a70 0x7fea5c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:26.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 --2- 172.21.15.42:0/3257727483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea5c003f40 0x7fea5c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:26.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 --2- 172.21.15.42:0/3257727483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c004d50 0x7fea5c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:26.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 -- 172.21.15.42:0/3257727483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea7815ce50 con 0x7fea5c003f40 2024-05-28T14:04:26.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 -- 172.21.15.42:0/3257727483 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea7815d130 con 0x7fea5c004d50 2024-05-28T14:04:26.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea75600640 1 -- 172.21.15.42:0/3257727483 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c00b9d0 con 0x7fea78156a70 2024-05-28T14:04:26.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea76000640 1 --2- 172.21.15.42:0/3257727483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea5c003f40 0x7fea5c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:26.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea77400640 1 --2- 172.21.15.42:0/3257727483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c004d50 0x7fea5c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:26.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea76a00640 1 --2- 172.21.15.42:0/3257727483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78156a70 0x7fea5c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:26.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea76000640 1 -- 172.21.15.42:0/3257727483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78156a70 msgr2=0x7fea5c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:26.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea76000640 1 --2- 172.21.15.42:0/3257727483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78156a70 0x7fea5c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea76000640 1 -- 172.21.15.42:0/3257727483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c004d50 msgr2=0x7fea5c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:26.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea76000640 1 --2- 172.21.15.42:0/3257727483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c004d50 0x7fea5c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea76000640 1 -- 172.21.15.42:0/3257727483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c003530 con 0x7fea5c003f40 2024-05-28T14:04:26.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.759+0000 7fea76a00640 1 --2- 172.21.15.42:0/3257727483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78156a70 0x7fea5c0039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:26.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.763+0000 7fea76000640 1 --2- 172.21.15.42:0/3257727483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea5c003f40 0x7fea5c004810 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fea6400c240 tx=0x7fea6401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e526acfb87b67063 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:26.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.763+0000 7fea75600640 1 -- 172.21.15.42:0/3257727483 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64007d40 con 0x7fea5c003f40 2024-05-28T14:04:26.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.763+0000 7fea75600640 1 -- 172.21.15.42:0/3257727483 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea64005b00 con 0x7fea5c003f40 2024-05-28T14:04:26.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.763+0000 7fea75600640 1 -- 172.21.15.42:0/3257727483 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64007070 con 0x7fea5c003f40 2024-05-28T14:04:26.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.763+0000 7fea7d600640 1 -- 172.21.15.42:0/3257727483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea5c003f40 msgr2=0x7fea5c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:26.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.763+0000 7fea7d600640 1 --2- 172.21.15.42:0/3257727483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea5c003f40 0x7fea5c004810 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fea6400c240 tx=0x7fea6401f3b0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.763+0000 7fea77400640 1 -- 172.21.15.42:0/3257727483 reap_dead start 2024-05-28T14:04:26.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.763+0000 7fea7d600640 1 -- 172.21.15.42:0/3257727483 shutdown_connections 2024-05-28T14:04:26.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.763+0000 7fea7d600640 1 -- 172.21.15.42:0/3257727483 >> 172.21.15.42:0/3257727483 conn(0x7fea7806bdf0 msgr2=0x7fea7806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:26.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.763+0000 7fea7d600640 1 -- 172.21.15.42:0/3257727483 shutdown_connections 2024-05-28T14:04:26.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.763+0000 7fea7d600640 1 -- 172.21.15.42:0/3257727483 wait complete. 2024-05-28T14:04:26.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.763+0000 7fea7d600640 1 Processor -- start 2024-05-28T14:04:26.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.763+0000 7fea7d600640 1 -- start start 2024-05-28T14:04:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea7d600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c003f40 0x7fea781515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea7d600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78142760 0x7fea78157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea7d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea78157790 0x7fea781f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea7d600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea7806dee0 con 0x7fea78157790 2024-05-28T14:04:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea7d600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea7806dd60 con 0x7fea5c003f40 2024-05-28T14:04:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea7d600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea7806e060 con 0x7fea78142760 2024-05-28T14:04:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea77400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea78157790 0x7fea781f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea77400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea78157790 0x7fea781f2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34536/0 (socket says 172.21.15.42:34536) 2024-05-28T14:04:26.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea77400640 1 -- 172.21.15.42:0/1793865411 learned_addr learned my addr 172.21.15.42:0/1793865411 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:26.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea76a00640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c003f40 0x7fea781515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:26.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea76000640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78142760 0x7fea78157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:26.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea77400640 1 -- 172.21.15.42:0/1793865411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78142760 msgr2=0x7fea78157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:26.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea77400640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78142760 0x7fea78157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea77400640 1 -- 172.21.15.42:0/1793865411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c003f40 msgr2=0x7fea781515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:26.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea77400640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c003f40 0x7fea781515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea77400640 1 -- 172.21.15.42:0/1793865411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7815a1f0 con 0x7fea78157790 2024-05-28T14:04:26.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea76000640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78142760 0x7fea78157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:26.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea77400640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea78157790 0x7fea781f2d30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fea780b4780 tx=0x7fea7000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea6ea00640 1 -- 172.21.15.42:0/1793865411 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70019070 con 0x7fea78157790 2024-05-28T14:04:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea6ea00640 1 -- 172.21.15.42:0/1793865411 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea70011e90 con 0x7fea78157790 2024-05-28T14:04:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea7d600640 1 -- 172.21.15.42:0/1793865411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea78154440 con 0x7fea78157790 2024-05-28T14:04:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea6ea00640 1 -- 172.21.15.42:0/1793865411 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70003c00 con 0x7fea78157790 2024-05-28T14:04:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea7d600640 1 -- 172.21.15.42:0/1793865411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea78154a00 con 0x7fea78157790 2024-05-28T14:04:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.767+0000 7fea7d600640 1 -- 172.21.15.42:0/1793865411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea78068a90 con 0x7fea78157790 2024-05-28T14:04:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.771+0000 7fea6ea00640 1 -- 172.21.15.42:0/1793865411 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fea7001c070 con 0x7fea78157790 2024-05-28T14:04:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.771+0000 7fea6ea00640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea58068b60 0x7fea5806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.771+0000 7fea6ea00640 1 -- 172.21.15.42:0/1793865411 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fea70088640 con 0x7fea78157790 2024-05-28T14:04:26.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.771+0000 7fea76a00640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea58068b60 0x7fea5806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:26.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.771+0000 7fea76a00640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea58068b60 0x7fea5806b020 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fea78151f90 tx=0x7fea68008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:26.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.775+0000 7fea6ea00640 1 -- 172.21.15.42:0/1793865411 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fea70010040 con 0x7fea78157790 2024-05-28T14:04:26.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.975+0000 7fea7d600640 1 -- 172.21.15.42:0/1793865411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea780b5160 con 0x7fea78157790 2024-05-28T14:04:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.975+0000 7fea6ea00640 1 -- 172.21.15.42:0/1793865411 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5178 (secure 0 0 0) 0x7fea70057c80 con 0x7fea78157790 2024-05-28T14:04:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:26.962 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1310016671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1310016671},{"type":"v1","addr":"172.21.15.63:6834","nonce":1310016671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1602237641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1602237641},{"type":"v1","addr":"172.21.15.42:6835","nonce":1602237641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/142807607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":142807607},{"type":"v1","addr":"172.21.15.42:6837","nonce":142807607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:04:22.960438+0000","modified":"2024-05-28T14:04:25.983153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3311861302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3311861302},{"type":"v1","addr":"172.21.15.63:6835","nonce":3311861302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6420,"qdb_cluster":[6420]},"id":4}]} 2024-05-28T14:04:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 36 2024-05-28T14:04:26.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.979+0000 7fea7d600640 1 -- 172.21.15.42:0/1793865411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea58068b60 msgr2=0x7fea5806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:26.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.979+0000 7fea7d600640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea58068b60 0x7fea5806b020 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fea78151f90 tx=0x7fea68008040 comp rx=0 tx=0).stop 2024-05-28T14:04:26.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.979+0000 7fea7d600640 1 -- 172.21.15.42:0/1793865411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea78157790 msgr2=0x7fea781f2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:26.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.979+0000 7fea7d600640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea78157790 0x7fea781f2d30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fea780b4780 tx=0x7fea7000f580 comp rx=0 tx=0).stop 2024-05-28T14:04:26.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.979+0000 7fea7d600640 1 -- 172.21.15.42:0/1793865411 shutdown_connections 2024-05-28T14:04:26.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.979+0000 7fea7d600640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea58068b60 0x7fea5806b020 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.979+0000 7fea7d600640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea78157790 0x7fea781f2d30 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.979+0000 7fea7d600640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea78142760 0x7fea78157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.979+0000 7fea7d600640 1 --2- 172.21.15.42:0/1793865411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c003f40 0x7fea781515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:26.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.979+0000 7fea7d600640 1 -- 172.21.15.42:0/1793865411 >> 172.21.15.42:0/1793865411 conn(0x7fea7806bdf0 msgr2=0x7fea7806f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:26.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.979+0000 7fea7d600640 1 -- 172.21.15.42:0/1793865411 shutdown_connections 2024-05-28T14:04:26.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:26.979+0000 7fea7d600640 1 -- 172.21.15.42:0/1793865411 wait complete. 2024-05-28T14:04:26.984 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:04:27.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f15000640 1 Processor -- start 2024-05-28T14:04:27.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f15000640 1 -- start start 2024-05-28T14:04:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f15000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0f1009aa50 0x7f0f1009aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f15000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0f100cdda0 0x7f0f100ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f15000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f100ccd00 con 0x7f0f1015ea80 2024-05-28T14:04:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f15000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0f1015ee60 con 0x7f0f100cdda0 2024-05-28T14:04:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f15000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0f100ccfe0 con 0x7f0f1009aa50 2024-05-28T14:04:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f0de00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0f1015ea80 0x7f0f100d12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:52634/0 (socket says 172.21.15.42:52634) 2024-05-28T14:04:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f0de00640 1 -- 172.21.15.42:0/3246888769 learned_addr learned my addr 172.21.15.42:0/3246888769 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f0e800640 1 --2- 172.21.15.42:0/3246888769 >> v2:172.21.15.42:3300/0 conn(0x7f0f100cdda0 0x7f0f100ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f05400640 1 --2- 172.21.15.42:0/3246888769 >> v2:172.21.15.63:3300/0 conn(0x7f0f1009aa50 0x7f0f1009aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f0d400640 1 -- 172.21.15.42:0/3246888769 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2234814317 0 0) 0x7f0f100ccd00 con 0x7f0f1015ea80 2024-05-28T14:04:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f0d400640 1 -- 172.21.15.42:0/3246888769 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ef4003530 con 0x7f0f1015ea80 2024-05-28T14:04:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f0e800640 1 -- 172.21.15.42:0/3246888769 >> v2:172.21.15.63:3300/0 conn(0x7f0f1009aa50 msgr2=0x7f0f1009aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f0e800640 1 --2- 172.21.15.42:0/3246888769 >> v2:172.21.15.63:3300/0 conn(0x7f0f1009aa50 0x7f0f1009aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f0e800640 1 -- 172.21.15.42:0/3246888769 >> v1:172.21.15.42:6789/0 conn(0x7f0f1015ea80 legacy=0x7f0f100d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f0e800640 1 -- 172.21.15.42:0/3246888769 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f100cd2c0 con 0x7f0f100cdda0 2024-05-28T14:04:27.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.139+0000 7f0f05400640 1 --2- 172.21.15.42:0/3246888769 >> v2:172.21.15.63:3300/0 conn(0x7f0f1009aa50 0x7f0f1009aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:27.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.143+0000 7f0f0e800640 1 --2- 172.21.15.42:0/3246888769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f100cdda0 0x7f0f100ce230 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0f10069790 tx=0x7f0f0801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226b8654f8d2fa2d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:27.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.143+0000 7f0f0d400640 1 -- 172.21.15.42:0/3246888769 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f080b0070 con 0x7f0f100cdda0 2024-05-28T14:04:27.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.143+0000 7f0f0d400640 1 -- 172.21.15.42:0/3246888769 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f080ab600 con 0x7f0f100cdda0 2024-05-28T14:04:27.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.143+0000 7f0f0d400640 1 -- 172.21.15.42:0/3246888769 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f080ab9d0 con 0x7f0f100cdda0 2024-05-28T14:04:27.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.143+0000 7f0f15000640 1 -- 172.21.15.42:0/3246888769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f100cdda0 msgr2=0x7f0f100ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.143+0000 7f0f15000640 1 --2- 172.21.15.42:0/3246888769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f100cdda0 0x7f0f100ce230 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0f10069790 tx=0x7f0f0801c480 comp rx=0 tx=0).stop 2024-05-28T14:04:27.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.143+0000 7f0f15000640 1 -- 172.21.15.42:0/3246888769 shutdown_connections 2024-05-28T14:04:27.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.143+0000 7f0f15000640 1 --2- 172.21.15.42:0/3246888769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f100cdda0 0x7f0f100ce230 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.143+0000 7f0f15000640 1 --2- 172.21.15.42:0/3246888769 >> v2:172.21.15.63:3300/0 conn(0x7f0f1009aa50 0x7f0f1009aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.143+0000 7f0f15000640 1 -- 172.21.15.42:0/3246888769 >> 172.21.15.42:0/3246888769 conn(0x7f0f100b3f90 msgr2=0x7f0f100b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:27.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.143+0000 7f0f15000640 1 -- 172.21.15.42:0/3246888769 shutdown_connections 2024-05-28T14:04:27.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.143+0000 7f0f15000640 1 -- 172.21.15.42:0/3246888769 wait complete. 2024-05-28T14:04:27.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f15000640 1 Processor -- start 2024-05-28T14:04:27.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f15000640 1 -- start start 2024-05-28T14:04:27.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f15000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f1009aa50 0x7f0f10154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f15000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f100cdda0 0x7f0f101589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f15000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f1015ea80 0x7f0f10158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f15000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0f100d19f0 con 0x7f0f100cdda0 2024-05-28T14:04:27.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f15000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0f100d1b70 con 0x7f0f1015ea80 2024-05-28T14:04:27.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f15000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0f100b19f0 con 0x7f0f1009aa50 2024-05-28T14:04:27.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f05400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f100cdda0 0x7f0f101589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f05400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f100cdda0 0x7f0f101589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34548/0 (socket says 172.21.15.42:34548) 2024-05-28T14:04:27.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f05400640 1 -- 172.21.15.42:0/2930060948 learned_addr learned my addr 172.21.15.42:0/2930060948 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:27.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f0e800640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f1015ea80 0x7f0f10158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f0de00640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f1009aa50 0x7f0f10154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f05400640 1 -- 172.21.15.42:0/2930060948 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f1009aa50 msgr2=0x7f0f10154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f05400640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f1009aa50 0x7f0f10154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f05400640 1 -- 172.21.15.42:0/2930060948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f1015ea80 msgr2=0x7f0f10158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f05400640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f1015ea80 0x7f0f10158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f05400640 1 -- 172.21.15.42:0/2930060948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f101594f0 con 0x7f0f100cdda0 2024-05-28T14:04:27.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f0e800640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f1015ea80 0x7f0f10158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:27.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f0de00640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f1009aa50 0x7f0f10154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:27.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f05400640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f100cdda0 0x7f0f101589a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009fc0 tx=0x7f0ef800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:27.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f06a00640 1 -- 172.21.15.42:0/2930060948 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef801a070 con 0x7f0f100cdda0 2024-05-28T14:04:27.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f06a00640 1 -- 172.21.15.42:0/2930060948 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef8003ba0 con 0x7f0f100cdda0 2024-05-28T14:04:27.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f06a00640 1 -- 172.21.15.42:0/2930060948 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8004030 con 0x7f0f100cdda0 2024-05-28T14:04:27.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f15000640 1 -- 172.21.15.42:0/2930060948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f101597d0 con 0x7f0f100cdda0 2024-05-28T14:04:27.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.147+0000 7f0f15000640 1 -- 172.21.15.42:0/2930060948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f10159d90 con 0x7f0f100cdda0 2024-05-28T14:04:27.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.151+0000 7f0f15000640 1 -- 172.21.15.42:0/2930060948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f10068a90 con 0x7f0f100cdda0 2024-05-28T14:04:27.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.151+0000 7f0f06a00640 1 -- 172.21.15.42:0/2930060948 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f0ef801d070 con 0x7f0f100cdda0 2024-05-28T14:04:27.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.151+0000 7f0f06a00640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0eec068b60 0x7f0eec06b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.151+0000 7f0f06a00640 1 -- 172.21.15.42:0/2930060948 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f0ef8088040 con 0x7f0f100cdda0 2024-05-28T14:04:27.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.151+0000 7f0f0de00640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0eec068b60 0x7f0eec06b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.151+0000 7f0f0de00640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0eec068b60 0x7f0eec06b020 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0f00005890 tx=0x7f0f00009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:27.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.155+0000 7f0f06a00640 1 -- 172.21.15.42:0/2930060948 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ef8010070 con 0x7f0f100cdda0 2024-05-28T14:04:27.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.319+0000 7f0f15000640 1 -- 172.21.15.42:0/2930060948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f10071bf0 con 0x7f0f100cdda0 2024-05-28T14:04:27.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.323+0000 7f0f06a00640 1 -- 172.21.15.42:0/2930060948 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16648 (secure 0 0 0) 0x7f0ef8056e60 con 0x7f0f100cdda0 2024-05-28T14:04:27.309 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:27.309 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":61,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:04:24.558637+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:04:20.721059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-28T14:04:21.922127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:04:27.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.323+0000 7f0f15000640 1 -- 172.21.15.42:0/2930060948 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0eec068b60 msgr2=0x7f0eec06b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.323+0000 7f0f15000640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0eec068b60 0x7f0eec06b020 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0f00005890 tx=0x7f0f00009040 comp rx=0 tx=0).stop 2024-05-28T14:04:27.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.323+0000 7f0f15000640 1 -- 172.21.15.42:0/2930060948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f100cdda0 msgr2=0x7f0f101589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.323+0000 7f0f15000640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f100cdda0 0x7f0f101589a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009fc0 tx=0x7f0ef800d560 comp rx=0 tx=0).stop 2024-05-28T14:04:27.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.323+0000 7f0f15000640 1 -- 172.21.15.42:0/2930060948 shutdown_connections 2024-05-28T14:04:27.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.323+0000 7f0f15000640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0eec068b60 0x7f0eec06b020 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.323+0000 7f0f15000640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f1015ea80 0x7f0f10158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.323+0000 7f0f15000640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f100cdda0 0x7f0f101589a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.323+0000 7f0f15000640 1 --2- 172.21.15.42:0/2930060948 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f1009aa50 0x7f0f10154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.323+0000 7f0f15000640 1 -- 172.21.15.42:0/2930060948 >> 172.21.15.42:0/2930060948 conn(0x7f0f100b3f90 msgr2=0x7f0f1015ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:27.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.323+0000 7f0f15000640 1 -- 172.21.15.42:0/2930060948 shutdown_connections 2024-05-28T14:04:27.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.323+0000 7f0f15000640 1 -- 172.21.15.42:0/2930060948 wait complete. 2024-05-28T14:04:27.332 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-28T14:04:27.333 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:04:27.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.487+0000 7f4220c00640 1 Processor -- start 2024-05-28T14:04:27.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.487+0000 7f4220c00640 1 -- start start 2024-05-28T14:04:27.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.487+0000 7f4220c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f421c156a70 0x7f421c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.487+0000 7f4220c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f421c156e50 0x7f421c0af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.487+0000 7f4220c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f421c15cb90 con 0x7f421c0af880 2024-05-28T14:04:27.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.487+0000 7f4220c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f421c15ce50 con 0x7f421c156e50 2024-05-28T14:04:27.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.487+0000 7f4220c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f421c15d130 con 0x7f421c156a70 2024-05-28T14:04:27.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.487+0000 7f421a800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f421c0af880 0x7f421c0d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51232/0 (socket says 172.21.15.42:51232) 2024-05-28T14:04:27.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.487+0000 7f421a800640 1 -- 172.21.15.42:0/1414693812 learned_addr learned my addr 172.21.15.42:0/1414693812 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:27.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.487+0000 7f4213e00640 1 --2- 172.21.15.42:0/1414693812 >> v2:172.21.15.63:3300/0 conn(0x7f421c156e50 0x7f421c0af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.487+0000 7f4219e00640 1 --2- 172.21.15.42:0/1414693812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f421c156a70 0x7f421c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.491+0000 7f4219400640 1 -- 172.21.15.42:0/1414693812 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2195124626 0 0) 0x7f421c15cb90 con 0x7f421c0af880 2024-05-28T14:04:27.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.491+0000 7f4219400640 1 -- 172.21.15.42:0/1414693812 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4200003530 con 0x7f421c0af880 2024-05-28T14:04:27.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.491+0000 7f4219e00640 1 -- 172.21.15.42:0/1414693812 >> v2:172.21.15.63:3300/0 conn(0x7f421c156e50 msgr2=0x7f421c0af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:27.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.491+0000 7f4219e00640 1 --2- 172.21.15.42:0/1414693812 >> v2:172.21.15.63:3300/0 conn(0x7f421c156e50 0x7f421c0af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.491+0000 7f4219e00640 1 -- 172.21.15.42:0/1414693812 >> v1:172.21.15.42:6789/0 conn(0x7f421c0af880 legacy=0x7f421c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.491+0000 7f4219e00640 1 -- 172.21.15.42:0/1414693812 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c15d410 con 0x7f421c156a70 2024-05-28T14:04:27.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.491+0000 7f4213e00640 1 --2- 172.21.15.42:0/1414693812 >> v2:172.21.15.63:3300/0 conn(0x7f421c156e50 0x7f421c0af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:04:27.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.491+0000 7f4219e00640 1 --2- 172.21.15.42:0/1414693812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f421c156a70 0x7f421c06dd60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f420c0097c0 tx=0x7f420c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ee4923e24ea0bf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:27.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.491+0000 7f4219400640 1 -- 172.21.15.42:0/1414693812 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c0b1070 con 0x7f421c156a70 2024-05-28T14:04:27.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.491+0000 7f4219400640 1 -- 172.21.15.42:0/1414693812 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420c0ac410 con 0x7f421c156a70 2024-05-28T14:04:27.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.491+0000 7f4219400640 1 -- 172.21.15.42:0/1414693812 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c0ac7e0 con 0x7f421c156a70 2024-05-28T14:04:27.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 -- 172.21.15.42:0/1414693812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f421c156a70 msgr2=0x7f421c06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 --2- 172.21.15.42:0/1414693812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f421c156a70 0x7f421c06dd60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f420c0097c0 tx=0x7f420c01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 -- 172.21.15.42:0/1414693812 shutdown_connections 2024-05-28T14:04:27.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 --2- 172.21.15.42:0/1414693812 >> v2:172.21.15.63:3300/0 conn(0x7f421c156e50 0x7f421c0af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 --2- 172.21.15.42:0/1414693812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f421c156a70 0x7f421c06dd60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 -- 172.21.15.42:0/1414693812 >> 172.21.15.42:0/1414693812 conn(0x7f421c06bdf0 msgr2=0x7f421c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:27.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 -- 172.21.15.42:0/1414693812 shutdown_connections 2024-05-28T14:04:27.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 -- 172.21.15.42:0/1414693812 wait complete. 2024-05-28T14:04:27.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 Processor -- start 2024-05-28T14:04:27.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 -- start start 2024-05-28T14:04:27.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f421c0af880 0x7f421c14d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f421c156a70 0x7f421c15f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f421c156e50 0x7f421c15f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f421c160b40 con 0x7f421c0af880 2024-05-28T14:04:27.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f421c0d1010 con 0x7f421c156e50 2024-05-28T14:04:27.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4220c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f421c160cc0 con 0x7f421c156a70 2024-05-28T14:04:27.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4219e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f421c0af880 0x7f421c14d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4219e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f421c0af880 0x7f421c14d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60864/0 (socket says 172.21.15.42:60864) 2024-05-28T14:04:27.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4219e00640 1 -- 172.21.15.42:0/717995070 learned_addr learned my addr 172.21.15.42:0/717995070 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:27.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4213e00640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f421c156a70 0x7f421c15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f421a800640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f421c156e50 0x7f421c15f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4219e00640 1 -- 172.21.15.42:0/717995070 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f421c156a70 msgr2=0x7f421c15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4219e00640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f421c156a70 0x7f421c15f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4219e00640 1 -- 172.21.15.42:0/717995070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f421c156e50 msgr2=0x7f421c15f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4219e00640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f421c156e50 0x7f421c15f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4219e00640 1 -- 172.21.15.42:0/717995070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c157660 con 0x7f421c0af880 2024-05-28T14:04:27.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.495+0000 7f4213e00640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f421c156a70 0x7f421c15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:27.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.499+0000 7f4219e00640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f421c0af880 0x7f421c14d930 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f420c01a0d0 tx=0x7f420c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:27.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.499+0000 7f4212000640 1 -- 172.21.15.42:0/717995070 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c0b9020 con 0x7f421c0af880 2024-05-28T14:04:27.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.499+0000 7f4212000640 1 -- 172.21.15.42:0/717995070 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420c0b5410 con 0x7f421c0af880 2024-05-28T14:04:27.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.499+0000 7f4220c00640 1 -- 172.21.15.42:0/717995070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421c157380 con 0x7f421c0af880 2024-05-28T14:04:27.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.499+0000 7f4212000640 1 -- 172.21.15.42:0/717995070 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c0b1040 con 0x7f421c0af880 2024-05-28T14:04:27.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.499+0000 7f4220c00640 1 -- 172.21.15.42:0/717995070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421c14de70 con 0x7f421c0af880 2024-05-28T14:04:27.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.499+0000 7f4220c00640 1 -- 172.21.15.42:0/717995070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421c068a90 con 0x7f421c0af880 2024-05-28T14:04:27.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.499+0000 7f4212000640 1 -- 172.21.15.42:0/717995070 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f420c0bc020 con 0x7f421c0af880 2024-05-28T14:04:27.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.499+0000 7f4212000640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f41f8068b60 0x7f41f806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.499+0000 7f4212000640 1 -- 172.21.15.42:0/717995070 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f420c121400 con 0x7f421c0af880 2024-05-28T14:04:27.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.499+0000 7f4213e00640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f41f8068b60 0x7f41f806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.503+0000 7f4213e00640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f41f8068b60 0x7f41f806b020 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f421c1f8690 tx=0x7f4204008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:27.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.503+0000 7f4212000640 1 -- 172.21.15.42:0/717995070 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f420c0c3030 con 0x7f421c0af880 2024-05-28T14:04:27.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.671+0000 7f4220c00640 1 -- 172.21.15.42:0/717995070 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f421c14f9e0 con 0x7f41f8068b60 2024-05-28T14:04:27.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.679+0000 7f4212000640 1 -- 172.21.15.42:0/717995070 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251910 (secure 0 0 0) 0x7f4204010840 con 0x7f41f8068b60 2024-05-28T14:04:27.667 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:27.668 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:04:27.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.683+0000 7f4220c00640 1 -- 172.21.15.42:0/717995070 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f41f8068b60 msgr2=0x7f41f806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.683+0000 7f4220c00640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f41f8068b60 0x7f41f806b020 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f421c1f8690 tx=0x7f4204008040 comp rx=0 tx=0).stop 2024-05-28T14:04:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.683+0000 7f4220c00640 1 -- 172.21.15.42:0/717995070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f421c0af880 msgr2=0x7f421c14d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.683+0000 7f4220c00640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f421c0af880 0x7f421c14d930 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f420c01a0d0 tx=0x7f420c019240 comp rx=0 tx=0).stop 2024-05-28T14:04:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.683+0000 7f4220c00640 1 -- 172.21.15.42:0/717995070 shutdown_connections 2024-05-28T14:04:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.683+0000 7f4220c00640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f41f8068b60 0x7f41f806b020 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.683+0000 7f4220c00640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f421c156e50 0x7f421c15f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.683+0000 7f4220c00640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f421c156a70 0x7f421c15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.683+0000 7f4220c00640 1 --2- 172.21.15.42:0/717995070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f421c0af880 0x7f421c14d930 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.683+0000 7f4220c00640 1 -- 172.21.15.42:0/717995070 >> 172.21.15.42:0/717995070 conn(0x7f421c06bdf0 msgr2=0x7f421c0b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.683+0000 7f4220c00640 1 -- 172.21.15.42:0/717995070 shutdown_connections 2024-05-28T14:04:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.683+0000 7f4220c00640 1 -- 172.21.15.42:0/717995070 wait complete. 2024-05-28T14:04:27.687 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-05-28T14:04:26.944166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356304,"kb_used_data":6856,"kb_used_omap":12,"kb_used_meta":349299,"kb_avail":754618416,"statfs":{"total":773094113280,"available":772729257984,"internally_reserved":0,"allocated":7020544,"data_stored":3605124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":357682738},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.400031"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.563010+0000","last_change":"2024-05-28T14:04:23.593610+0000","last_active":"2024-05-28T14:04:24.563010+0000","last_peered":"2024-05-28T14:04:24.563010+0000","last_clean":"2024-05-28T14:04:24.563010+0000","last_became_active":"2024-05-28T14:04:23.593289+0000","last_became_peered":"2024-05-28T14:04:23.593289+0000","last_unstale":"2024-05-28T14:04:24.563010+0000","last_undegraded":"2024-05-28T14:04:24.563010+0000","last_fullsized":"2024-05-28T14:04:24.563010+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:57.872099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588414+0000","last_change":"2024-05-28T14:04:23.583536+0000","last_active":"2024-05-28T14:04:23.588414+0000","last_peered":"2024-05-28T14:04:23.588414+0000","last_clean":"2024-05-28T14:04:23.588414+0000","last_became_active":"2024-05-28T14:04:23.583284+0000","last_became_peered":"2024-05-28T14:04:23.583284+0000","last_unstale":"2024-05-28T14:04:23.588414+0000","last_undegraded":"2024-05-28T14:04:23.588414+0000","last_fullsized":"2024-05-28T14:04:23.588414+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:03:35.986922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588441+0000","last_change":"2024-05-28T14:04:23.581101+0000","last_active":"2024-05-28T14:04:23.588441+0000","last_peered":"2024-05-28T14:04:23.588441+0000","last_clean":"2024-05-28T14:04:23.588441+0000","last_became_active":"2024-05-28T14:04:23.580815+0000","last_became_peered":"2024-05-28T14:04:23.580815+0000","last_unstale":"2024-05-28T14:04:23.588441+0000","last_undegraded":"2024-05-28T14:04:23.588441+0000","last_fullsized":"2024-05-28T14:04:23.588441+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:59:05.852542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571147+0000","last_change":"2024-05-28T14:04:23.574659+0000","last_active":"2024-05-28T14:04:24.571147+0000","last_peered":"2024-05-28T14:04:24.571147+0000","last_clean":"2024-05-28T14:04:24.571147+0000","last_became_active":"2024-05-28T14:04:23.574298+0000","last_became_peered":"2024-05-28T14:04:23.574298+0000","last_unstale":"2024-05-28T14:04:24.571147+0000","last_undegraded":"2024-05-28T14:04:24.571147+0000","last_fullsized":"2024-05-28T14:04:24.571147+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:27:45.268308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.562649+0000","last_change":"2024-05-28T14:04:23.589646+0000","last_active":"2024-05-28T14:04:24.562649+0000","last_peered":"2024-05-28T14:04:24.562649+0000","last_clean":"2024-05-28T14:04:24.562649+0000","last_became_active":"2024-05-28T14:04:23.585954+0000","last_became_peered":"2024-05-28T14:04:23.585954+0000","last_unstale":"2024-05-28T14:04:24.562649+0000","last_undegraded":"2024-05-28T14:04:24.562649+0000","last_fullsized":"2024-05-28T14:04:24.562649+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:48:13.320287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590433+0000","last_change":"2024-05-28T14:04:23.581910+0000","last_active":"2024-05-28T14:04:24.590433+0000","last_peered":"2024-05-28T14:04:24.590433+0000","last_clean":"2024-05-28T14:04:24.590433+0000","last_became_active":"2024-05-28T14:04:23.581609+0000","last_became_peered":"2024-05-28T14:04:23.581609+0000","last_unstale":"2024-05-28T14:04:24.590433+0000","last_undegraded":"2024-05-28T14:04:24.590433+0000","last_fullsized":"2024-05-28T14:04:24.590433+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:20:19.960393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.590710+0000","last_change":"2024-05-28T14:04:23.583917+0000","last_active":"2024-05-28T14:04:23.590710+0000","last_peered":"2024-05-28T14:04:23.590710+0000","last_clean":"2024-05-28T14:04:23.590710+0000","last_became_active":"2024-05-28T14:04:23.583493+0000","last_became_peered":"2024-05-28T14:04:23.583493+0000","last_unstale":"2024-05-28T14:04:23.590710+0000","last_undegraded":"2024-05-28T14:04:23.590710+0000","last_fullsized":"2024-05-28T14:04:23.590710+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:05:47.954430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588064+0000","last_change":"2024-05-28T14:04:23.576473+0000","last_active":"2024-05-28T14:04:23.588064+0000","last_peered":"2024-05-28T14:04:23.588064+0000","last_clean":"2024-05-28T14:04:23.588064+0000","last_became_active":"2024-05-28T14:04:23.576146+0000","last_became_peered":"2024-05-28T14:04:23.576146+0000","last_unstale":"2024-05-28T14:04:23.588064+0000","last_undegraded":"2024-05-28T14:04:23.588064+0000","last_fullsized":"2024-05-28T14:04:23.588064+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:57:02.905276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.579928+0000","last_change":"2024-05-28T14:04:23.577882+0000","last_active":"2024-05-28T14:04:23.579928+0000","last_peered":"2024-05-28T14:04:23.579928+0000","last_clean":"2024-05-28T14:04:23.579928+0000","last_became_active":"2024-05-28T14:04:23.577557+0000","last_became_peered":"2024-05-28T14:04:23.577557+0000","last_unstale":"2024-05-28T14:04:23.579928+0000","last_undegraded":"2024-05-28T14:04:23.579928+0000","last_fullsized":"2024-05-28T14:04:23.579928+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:29:22.224097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.594974+0000","last_change":"2024-05-28T14:04:23.582604+0000","last_active":"2024-05-28T14:04:23.594974+0000","last_peered":"2024-05-28T14:04:23.594974+0000","last_clean":"2024-05-28T14:04:23.594974+0000","last_became_active":"2024-05-28T14:04:23.582302+0000","last_became_peered":"2024-05-28T14:04:23.582302+0000","last_unstale":"2024-05-28T14:04:23.594974+0000","last_undegraded":"2024-05-28T14:04:23.594974+0000","last_fullsized":"2024-05-28T14:04:23.594974+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:03:44.309364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.595884+0000","last_change":"2024-05-28T14:04:23.589273+0000","last_active":"2024-05-28T14:04:23.595884+0000","last_peered":"2024-05-28T14:04:23.595884+0000","last_clean":"2024-05-28T14:04:23.595884+0000","last_became_active":"2024-05-28T14:04:23.588934+0000","last_became_peered":"2024-05-28T14:04:23.588934+0000","last_unstale":"2024-05-28T14:04:23.595884+0000","last_undegraded":"2024-05-28T14:04:23.595884+0000","last_fullsized":"2024-05-28T14:04:23.595884+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:46:52.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571106+0000","last_change":"2024-05-28T14:04:23.577217+0000","last_active":"2024-05-28T14:04:24.571106+0000","last_peered":"2024-05-28T14:04:24.571106+0000","last_clean":"2024-05-28T14:04:24.571106+0000","last_became_active":"2024-05-28T14:04:23.576880+0000","last_became_peered":"2024-05-28T14:04:23.576880+0000","last_unstale":"2024-05-28T14:04:24.571106+0000","last_undegraded":"2024-05-28T14:04:24.571106+0000","last_fullsized":"2024-05-28T14:04:24.571106+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:50:54.504716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.585295+0000","last_change":"2024-05-28T14:04:23.582966+0000","last_active":"2024-05-28T14:04:23.585295+0000","last_peered":"2024-05-28T14:04:23.585295+0000","last_clean":"2024-05-28T14:04:23.585295+0000","last_became_active":"2024-05-28T14:04:23.582695+0000","last_became_peered":"2024-05-28T14:04:23.582695+0000","last_unstale":"2024-05-28T14:04:23.585295+0000","last_undegraded":"2024-05-28T14:04:23.585295+0000","last_fullsized":"2024-05-28T14:04:23.585295+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:47:25.790192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588503+0000","last_change":"2024-05-28T14:04:23.582534+0000","last_active":"2024-05-28T14:04:23.588503+0000","last_peered":"2024-05-28T14:04:23.588503+0000","last_clean":"2024-05-28T14:04:23.588503+0000","last_became_active":"2024-05-28T14:04:23.582248+0000","last_became_peered":"2024-05-28T14:04:23.582248+0000","last_unstale":"2024-05-28T14:04:23.588503+0000","last_undegraded":"2024-05-28T14:04:23.588503+0000","last_fullsized":"2024-05-28T14:04:23.588503+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:20:27.689523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.580932+0000","last_change":"2024-05-28T14:04:23.575701+0000","last_active":"2024-05-28T14:04:23.580932+0000","last_peered":"2024-05-28T14:04:23.580932+0000","last_clean":"2024-05-28T14:04:23.580932+0000","last_became_active":"2024-05-28T14:04:23.575393+0000","last_became_peered":"2024-05-28T14:04:23.575393+0000","last_unstale":"2024-05-28T14:04:23.580932+0000","last_undegraded":"2024-05-28T14:04:23.580932+0000","last_fullsized":"2024-05-28T14:04:23.580932+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:22:52.389811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590360+0000","last_change":"2024-05-28T14:04:23.584296+0000","last_active":"2024-05-28T14:04:24.590360+0000","last_peered":"2024-05-28T14:04:24.590360+0000","last_clean":"2024-05-28T14:04:24.590360+0000","last_became_active":"2024-05-28T14:04:23.583997+0000","last_became_peered":"2024-05-28T14:04:23.583997+0000","last_unstale":"2024-05-28T14:04:24.590360+0000","last_undegraded":"2024-05-28T14:04:24.590360+0000","last_fullsized":"2024-05-28T14:04:24.590360+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:59:31.979919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.587700+0000","last_change":"2024-05-28T14:04:23.579745+0000","last_active":"2024-05-28T14:04:23.587700+0000","last_peered":"2024-05-28T14:04:23.587700+0000","last_clean":"2024-05-28T14:04:23.587700+0000","last_became_active":"2024-05-28T14:04:23.579481+0000","last_became_peered":"2024-05-28T14:04:23.579481+0000","last_unstale":"2024-05-28T14:04:23.587700+0000","last_undegraded":"2024-05-28T14:04:23.587700+0000","last_fullsized":"2024-05-28T14:04:23.587700+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:51:55.173568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.595535+0000","last_change":"2024-05-28T14:04:23.589867+0000","last_active":"2024-05-28T14:04:23.595535+0000","last_peered":"2024-05-28T14:04:23.595535+0000","last_clean":"2024-05-28T14:04:23.595535+0000","last_became_active":"2024-05-28T14:04:23.589596+0000","last_became_peered":"2024-05-28T14:04:23.589596+0000","last_unstale":"2024-05-28T14:04:23.595535+0000","last_undegraded":"2024-05-28T14:04:23.595535+0000","last_fullsized":"2024-05-28T14:04:23.595535+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:22:11.458492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.562845+0000","last_change":"2024-05-28T14:04:23.593765+0000","last_active":"2024-05-28T14:04:24.562845+0000","last_peered":"2024-05-28T14:04:24.562845+0000","last_clean":"2024-05-28T14:04:24.562845+0000","last_became_active":"2024-05-28T14:04:23.593455+0000","last_became_peered":"2024-05-28T14:04:23.593455+0000","last_unstale":"2024-05-28T14:04:24.562845+0000","last_undegraded":"2024-05-28T14:04:24.562845+0000","last_fullsized":"2024-05-28T14:04:24.562845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:06:07.071310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.587974+0000","last_change":"2024-05-28T14:04:23.580408+0000","last_active":"2024-05-28T14:04:23.587974+0000","last_peered":"2024-05-28T14:04:23.587974+0000","last_clean":"2024-05-28T14:04:23.587974+0000","last_became_active":"2024-05-28T14:04:23.577918+0000","last_became_peered":"2024-05-28T14:04:23.577918+0000","last_unstale":"2024-05-28T14:04:23.587974+0000","last_undegraded":"2024-05-28T14:04:23.587974+0000","last_fullsized":"2024-05-28T14:04:23.587974+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:22:12.704246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.587853+0000","last_change":"2024-05-28T14:04:23.583279+0000","last_active":"2024-05-28T14:04:23.587853+0000","last_peered":"2024-05-28T14:04:23.587853+0000","last_clean":"2024-05-28T14:04:23.587853+0000","last_became_active":"2024-05-28T14:04:23.583012+0000","last_became_peered":"2024-05-28T14:04:23.583012+0000","last_unstale":"2024-05-28T14:04:23.587853+0000","last_undegraded":"2024-05-28T14:04:23.587853+0000","last_fullsized":"2024-05-28T14:04:23.587853+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:07:44.967029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588117+0000","last_change":"2024-05-28T14:04:23.585380+0000","last_active":"2024-05-28T14:04:23.588117+0000","last_peered":"2024-05-28T14:04:23.588117+0000","last_clean":"2024-05-28T14:04:23.588117+0000","last_became_active":"2024-05-28T14:04:23.582647+0000","last_became_peered":"2024-05-28T14:04:23.582647+0000","last_unstale":"2024-05-28T14:04:23.588117+0000","last_undegraded":"2024-05-28T14:04:23.588117+0000","last_fullsized":"2024-05-28T14:04:23.588117+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:17:01.328216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.594366+0000","last_change":"2024-05-28T14:04:23.590713+0000","last_active":"2024-05-28T14:04:23.594366+0000","last_peered":"2024-05-28T14:04:23.594366+0000","last_clean":"2024-05-28T14:04:23.594366+0000","last_became_active":"2024-05-28T14:04:23.590441+0000","last_became_peered":"2024-05-28T14:04:23.590441+0000","last_unstale":"2024-05-28T14:04:23.594366+0000","last_undegraded":"2024-05-28T14:04:23.594366+0000","last_fullsized":"2024-05-28T14:04:23.594366+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:16:28.733076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.583617+0000","last_change":"2024-05-28T14:04:23.580197+0000","last_active":"2024-05-28T14:04:23.583617+0000","last_peered":"2024-05-28T14:04:23.583617+0000","last_clean":"2024-05-28T14:04:23.583617+0000","last_became_active":"2024-05-28T14:04:23.579864+0000","last_became_peered":"2024-05-28T14:04:23.579864+0000","last_unstale":"2024-05-28T14:04:23.583617+0000","last_undegraded":"2024-05-28T14:04:23.583617+0000","last_fullsized":"2024-05-28T14:04:23.583617+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:46:52.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.563123+0000","last_change":"2024-05-28T14:04:23.583792+0000","last_active":"2024-05-28T14:04:24.563123+0000","last_peered":"2024-05-28T14:04:24.563123+0000","last_clean":"2024-05-28T14:04:24.563123+0000","last_became_active":"2024-05-28T14:04:23.583325+0000","last_became_peered":"2024-05-28T14:04:23.583325+0000","last_unstale":"2024-05-28T14:04:24.563123+0000","last_undegraded":"2024-05-28T14:04:24.563123+0000","last_fullsized":"2024-05-28T14:04:24.563123+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:11:27.816943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.570949+0000","last_change":"2024-05-28T14:04:23.578927+0000","last_active":"2024-05-28T14:04:24.570949+0000","last_peered":"2024-05-28T14:04:24.570949+0000","last_clean":"2024-05-28T14:04:24.570949+0000","last_became_active":"2024-05-28T14:04:23.576566+0000","last_became_peered":"2024-05-28T14:04:23.576566+0000","last_unstale":"2024-05-28T14:04:24.570949+0000","last_undegraded":"2024-05-28T14:04:24.570949+0000","last_fullsized":"2024-05-28T14:04:24.570949+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:38:06.191955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590941+0000","last_change":"2024-05-28T14:04:23.585543+0000","last_active":"2024-05-28T14:04:24.590941+0000","last_peered":"2024-05-28T14:04:24.590941+0000","last_clean":"2024-05-28T14:04:24.590941+0000","last_became_active":"2024-05-28T14:04:23.585265+0000","last_became_peered":"2024-05-28T14:04:23.585265+0000","last_unstale":"2024-05-28T14:04:24.590941+0000","last_undegraded":"2024-05-28T14:04:24.590941+0000","last_fullsized":"2024-05-28T14:04:24.590941+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:06:07.558092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.562904+0000","last_change":"2024-05-28T14:04:23.590936+0000","last_active":"2024-05-28T14:04:24.562904+0000","last_peered":"2024-05-28T14:04:24.562904+0000","last_clean":"2024-05-28T14:04:24.562904+0000","last_became_active":"2024-05-28T14:04:23.583135+0000","last_became_peered":"2024-05-28T14:04:23.583135+0000","last_unstale":"2024-05-28T14:04:24.562904+0000","last_undegraded":"2024-05-28T14:04:24.562904+0000","last_fullsized":"2024-05-28T14:04:24.562904+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:09:01.172689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588085+0000","last_change":"2024-05-28T14:04:23.585146+0000","last_active":"2024-05-28T14:04:23.588085+0000","last_peered":"2024-05-28T14:04:23.588085+0000","last_clean":"2024-05-28T14:04:23.588085+0000","last_became_active":"2024-05-28T14:04:23.584842+0000","last_became_peered":"2024-05-28T14:04:23.584842+0000","last_unstale":"2024-05-28T14:04:23.588085+0000","last_undegraded":"2024-05-28T14:04:23.588085+0000","last_fullsized":"2024-05-28T14:04:23.588085+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:57:03.942825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.581766+0000","last_change":"2024-05-28T14:04:23.579833+0000","last_active":"2024-05-28T14:04:23.581766+0000","last_peered":"2024-05-28T14:04:23.581766+0000","last_clean":"2024-05-28T14:04:23.581766+0000","last_became_active":"2024-05-28T14:04:23.579461+0000","last_became_peered":"2024-05-28T14:04:23.579461+0000","last_unstale":"2024-05-28T14:04:23.581766+0000","last_undegraded":"2024-05-28T14:04:23.581766+0000","last_fullsized":"2024-05-28T14:04:23.581766+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:12:00.209164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.562644+0000","last_change":"2024-05-28T14:04:23.574780+0000","last_active":"2024-05-28T14:04:24.562644+0000","last_peered":"2024-05-28T14:04:24.562644+0000","last_clean":"2024-05-28T14:04:24.562644+0000","last_became_active":"2024-05-28T14:04:23.574504+0000","last_became_peered":"2024-05-28T14:04:23.574504+0000","last_unstale":"2024-05-28T14:04:24.562644+0000","last_undegraded":"2024-05-28T14:04:24.562644+0000","last_fullsized":"2024-05-28T14:04:24.562644+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:23:07.598206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590431+0000","last_change":"2024-05-28T14:04:23.588278+0000","last_active":"2024-05-28T14:04:24.590431+0000","last_peered":"2024-05-28T14:04:24.590431+0000","last_clean":"2024-05-28T14:04:24.590431+0000","last_became_active":"2024-05-28T14:04:23.588005+0000","last_became_peered":"2024-05-28T14:04:23.588005+0000","last_unstale":"2024-05-28T14:04:24.590431+0000","last_undegraded":"2024-05-28T14:04:24.590431+0000","last_fullsized":"2024-05-28T14:04:24.590431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:38:06.191955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590788+0000","last_change":"2024-05-28T14:04:23.579518+0000","last_active":"2024-05-28T14:04:24.590788+0000","last_peered":"2024-05-28T14:04:24.590788+0000","last_clean":"2024-05-28T14:04:24.590788+0000","last_became_active":"2024-05-28T14:04:23.579136+0000","last_became_peered":"2024-05-28T14:04:23.579136+0000","last_unstale":"2024-05-28T14:04:24.590788+0000","last_undegraded":"2024-05-28T14:04:24.590788+0000","last_fullsized":"2024-05-28T14:04:24.590788+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:51:52.497265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"60'4","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590845+0000","last_change":"2024-05-28T14:04:22.567917+0000","last_active":"2024-05-28T14:04:24.590845+0000","last_peered":"2024-05-28T14:04:24.590845+0000","last_clean":"2024-05-28T14:04:24.590845+0000","last_became_active":"2024-05-28T14:04:22.567634+0000","last_became_peered":"2024-05-28T14:04:22.567634+0000","last_unstale":"2024-05-28T14:04:24.590845+0000","last_undegraded":"2024-05-28T14:04:24.590845+0000","last_fullsized":"2024-05-28T14:04:24.590845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:17:46.877238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.594706+0000","last_change":"2024-05-28T14:04:23.584443+0000","last_active":"2024-05-28T14:04:23.594706+0000","last_peered":"2024-05-28T14:04:23.594706+0000","last_clean":"2024-05-28T14:04:23.594706+0000","last_became_active":"2024-05-28T14:04:23.582096+0000","last_became_peered":"2024-05-28T14:04:23.582096+0000","last_unstale":"2024-05-28T14:04:23.594706+0000","last_undegraded":"2024-05-28T14:04:23.594706+0000","last_fullsized":"2024-05-28T14:04:23.594706+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:56:35.913850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.602693+0000","last_change":"2024-05-28T14:04:22.571536+0000","last_active":"2024-05-28T14:04:23.602693+0000","last_peered":"2024-05-28T14:04:23.602693+0000","last_clean":"2024-05-28T14:04:23.602693+0000","last_became_active":"2024-05-28T14:04:22.571268+0000","last_became_peered":"2024-05-28T14:04:22.571268+0000","last_unstale":"2024-05-28T14:04:23.602693+0000","last_undegraded":"2024-05-28T14:04:23.602693+0000","last_fullsized":"2024-05-28T14:04:23.602693+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:59.738093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590642+0000","last_change":"2024-05-28T14:04:23.579400+0000","last_active":"2024-05-28T14:04:24.590642+0000","last_peered":"2024-05-28T14:04:24.590642+0000","last_clean":"2024-05-28T14:04:24.590642+0000","last_became_active":"2024-05-28T14:04:23.579102+0000","last_became_peered":"2024-05-28T14:04:23.579102+0000","last_unstale":"2024-05-28T14:04:24.590642+0000","last_undegraded":"2024-05-28T14:04:24.590642+0000","last_fullsized":"2024-05-28T14:04:24.590642+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:47:28.172031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590989+0000","last_change":"2024-05-28T14:04:22.569071+0000","last_active":"2024-05-28T14:04:24.590989+0000","last_peered":"2024-05-28T14:04:24.590989+0000","last_clean":"2024-05-28T14:04:24.590989+0000","last_became_active":"2024-05-28T14:04:22.568655+0000","last_became_peered":"2024-05-28T14:04:22.568655+0000","last_unstale":"2024-05-28T14:04:24.590989+0000","last_undegraded":"2024-05-28T14:04:24.590989+0000","last_fullsized":"2024-05-28T14:04:24.590989+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:13.052691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590773+0000","last_change":"2024-05-28T14:04:23.580842+0000","last_active":"2024-05-28T14:04:24.590773+0000","last_peered":"2024-05-28T14:04:24.590773+0000","last_clean":"2024-05-28T14:04:24.590773+0000","last_became_active":"2024-05-28T14:04:23.580548+0000","last_became_peered":"2024-05-28T14:04:23.580548+0000","last_unstale":"2024-05-28T14:04:24.590773+0000","last_undegraded":"2024-05-28T14:04:24.590773+0000","last_fullsized":"2024-05-28T14:04:24.590773+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:27:45.268308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571192+0000","last_change":"2024-05-28T14:04:22.569314+0000","last_active":"2024-05-28T14:04:24.571192+0000","last_peered":"2024-05-28T14:04:24.571192+0000","last_clean":"2024-05-28T14:04:24.571192+0000","last_became_active":"2024-05-28T14:04:22.569104+0000","last_became_peered":"2024-05-28T14:04:22.569104+0000","last_unstale":"2024-05-28T14:04:24.571192+0000","last_undegraded":"2024-05-28T14:04:24.571192+0000","last_fullsized":"2024-05-28T14:04:24.571192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:11:59.201758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.587968+0000","last_change":"2024-05-28T14:04:23.582632+0000","last_active":"2024-05-28T14:04:23.587968+0000","last_peered":"2024-05-28T14:04:23.587968+0000","last_clean":"2024-05-28T14:04:23.587968+0000","last_became_active":"2024-05-28T14:04:23.582290+0000","last_became_peered":"2024-05-28T14:04:23.582290+0000","last_unstale":"2024-05-28T14:04:23.587968+0000","last_undegraded":"2024-05-28T14:04:23.587968+0000","last_fullsized":"2024-05-28T14:04:23.587968+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:11:47.754807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.603730+0000","last_change":"2024-05-28T14:04:22.573692+0000","last_active":"2024-05-28T14:04:23.603730+0000","last_peered":"2024-05-28T14:04:23.603730+0000","last_clean":"2024-05-28T14:04:23.603730+0000","last_became_active":"2024-05-28T14:04:22.573412+0000","last_became_peered":"2024-05-28T14:04:22.573412+0000","last_unstale":"2024-05-28T14:04:23.603730+0000","last_undegraded":"2024-05-28T14:04:23.603730+0000","last_fullsized":"2024-05-28T14:04:23.603730+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:40:22.524069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.587165+0000","last_change":"2024-05-28T14:04:23.577017+0000","last_active":"2024-05-28T14:04:23.587165+0000","last_peered":"2024-05-28T14:04:23.587165+0000","last_clean":"2024-05-28T14:04:23.587165+0000","last_became_active":"2024-05-28T14:04:23.576634+0000","last_became_peered":"2024-05-28T14:04:23.576634+0000","last_unstale":"2024-05-28T14:04:23.587165+0000","last_undegraded":"2024-05-28T14:04:23.587165+0000","last_fullsized":"2024-05-28T14:04:23.587165+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:55.492821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590458+0000","last_change":"2024-05-28T14:04:22.566294+0000","last_active":"2024-05-28T14:04:24.590458+0000","last_peered":"2024-05-28T14:04:24.590458+0000","last_clean":"2024-05-28T14:04:24.590458+0000","last_became_active":"2024-05-28T14:04:22.566021+0000","last_became_peered":"2024-05-28T14:04:22.566021+0000","last_unstale":"2024-05-28T14:04:24.590458+0000","last_undegraded":"2024-05-28T14:04:24.590458+0000","last_fullsized":"2024-05-28T14:04:24.590458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:11.736974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.597058+0000","last_change":"2024-05-28T14:04:23.591906+0000","last_active":"2024-05-28T14:04:23.597058+0000","last_peered":"2024-05-28T14:04:23.597058+0000","last_clean":"2024-05-28T14:04:23.597058+0000","last_became_active":"2024-05-28T14:04:23.591600+0000","last_became_peered":"2024-05-28T14:04:23.591600+0000","last_unstale":"2024-05-28T14:04:23.597058+0000","last_undegraded":"2024-05-28T14:04:23.597058+0000","last_fullsized":"2024-05-28T14:04:23.597058+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:51:11.966539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.570759+0000","last_change":"2024-05-28T14:04:22.573550+0000","last_active":"2024-05-28T14:04:24.570759+0000","last_peered":"2024-05-28T14:04:24.570759+0000","last_clean":"2024-05-28T14:04:24.570759+0000","last_became_active":"2024-05-28T14:04:22.573237+0000","last_became_peered":"2024-05-28T14:04:22.573237+0000","last_unstale":"2024-05-28T14:04:24.570759+0000","last_undegraded":"2024-05-28T14:04:24.570759+0000","last_fullsized":"2024-05-28T14:04:24.570759+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:43:50.366314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588294+0000","last_change":"2024-05-28T14:04:23.586746+0000","last_active":"2024-05-28T14:04:23.588294+0000","last_peered":"2024-05-28T14:04:23.588294+0000","last_clean":"2024-05-28T14:04:23.588294+0000","last_became_active":"2024-05-28T14:04:23.585939+0000","last_became_peered":"2024-05-28T14:04:23.585939+0000","last_unstale":"2024-05-28T14:04:23.588294+0000","last_undegraded":"2024-05-28T14:04:23.588294+0000","last_fullsized":"2024-05-28T14:04:23.588294+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:36:31.735646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.565265+0000","last_change":"2024-05-28T14:04:22.567872+0000","last_active":"2024-05-28T14:04:23.565265+0000","last_peered":"2024-05-28T14:04:23.565265+0000","last_clean":"2024-05-28T14:04:23.565265+0000","last_became_active":"2024-05-28T14:04:22.567276+0000","last_became_peered":"2024-05-28T14:04:22.567276+0000","last_unstale":"2024-05-28T14:04:23.565265+0000","last_undegraded":"2024-05-28T14:04:23.565265+0000","last_fullsized":"2024-05-28T14:04:23.565265+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:11:46.843859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.590945+0000","last_change":"2024-05-28T14:04:23.584063+0000","last_active":"2024-05-28T14:04:23.590945+0000","last_peered":"2024-05-28T14:04:23.590945+0000","last_clean":"2024-05-28T14:04:23.590945+0000","last_became_active":"2024-05-28T14:04:23.583760+0000","last_became_peered":"2024-05-28T14:04:23.583760+0000","last_unstale":"2024-05-28T14:04:23.590945+0000","last_undegraded":"2024-05-28T14:04:23.590945+0000","last_fullsized":"2024-05-28T14:04:23.590945+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:12.744380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.574931+0000","last_change":"2024-05-28T14:04:22.577953+0000","last_active":"2024-05-28T14:04:23.574931+0000","last_peered":"2024-05-28T14:04:23.574931+0000","last_clean":"2024-05-28T14:04:23.574931+0000","last_became_active":"2024-05-28T14:04:22.577701+0000","last_became_peered":"2024-05-28T14:04:22.577701+0000","last_unstale":"2024-05-28T14:04:23.574931+0000","last_undegraded":"2024-05-28T14:04:23.574931+0000","last_fullsized":"2024-05-28T14:04:23.574931+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:24:52.150831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.592925+0000","last_change":"2024-05-28T14:04:23.589068+0000","last_active":"2024-05-28T14:04:23.592925+0000","last_peered":"2024-05-28T14:04:23.592925+0000","last_clean":"2024-05-28T14:04:23.592925+0000","last_became_active":"2024-05-28T14:04:23.588691+0000","last_became_peered":"2024-05-28T14:04:23.588691+0000","last_unstale":"2024-05-28T14:04:23.592925+0000","last_undegraded":"2024-05-28T14:04:23.592925+0000","last_fullsized":"2024-05-28T14:04:23.592925+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:59:03.398286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.562974+0000","last_change":"2024-05-28T14:04:22.572888+0000","last_active":"2024-05-28T14:04:23.562974+0000","last_peered":"2024-05-28T14:04:23.562974+0000","last_clean":"2024-05-28T14:04:23.562974+0000","last_became_active":"2024-05-28T14:04:22.572134+0000","last_became_peered":"2024-05-28T14:04:22.572134+0000","last_unstale":"2024-05-28T14:04:23.562974+0000","last_undegraded":"2024-05-28T14:04:23.562974+0000","last_fullsized":"2024-05-28T14:04:23.562974+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:20:18.952987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.582764+0000","last_change":"2024-05-28T14:04:23.580368+0000","last_active":"2024-05-28T14:04:23.582764+0000","last_peered":"2024-05-28T14:04:23.582764+0000","last_clean":"2024-05-28T14:04:23.582764+0000","last_became_active":"2024-05-28T14:04:23.578796+0000","last_became_peered":"2024-05-28T14:04:23.578796+0000","last_unstale":"2024-05-28T14:04:23.582764+0000","last_undegraded":"2024-05-28T14:04:23.582764+0000","last_fullsized":"2024-05-28T14:04:23.582764+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:55:40.974607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"60'3","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.604319+0000","last_change":"2024-05-28T14:04:22.573719+0000","last_active":"2024-05-28T14:04:23.604319+0000","last_peered":"2024-05-28T14:04:23.604319+0000","last_clean":"2024-05-28T14:04:23.604319+0000","last_became_active":"2024-05-28T14:04:22.573397+0000","last_became_peered":"2024-05-28T14:04:22.573397+0000","last_unstale":"2024-05-28T14:04:23.604319+0000","last_undegraded":"2024-05-28T14:04:23.604319+0000","last_fullsized":"2024-05-28T14:04:23.604319+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:13.052691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.593377+0000","last_change":"2024-05-28T14:04:23.588306+0000","last_active":"2024-05-28T14:04:23.593377+0000","last_peered":"2024-05-28T14:04:23.593377+0000","last_clean":"2024-05-28T14:04:23.593377+0000","last_became_active":"2024-05-28T14:04:23.587977+0000","last_became_peered":"2024-05-28T14:04:23.587977+0000","last_unstale":"2024-05-28T14:04:23.593377+0000","last_undegraded":"2024-05-28T14:04:23.593377+0000","last_fullsized":"2024-05-28T14:04:23.593377+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:22:49.692478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"60'6","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.607668+0000","last_change":"2024-05-28T14:04:22.564496+0000","last_active":"2024-05-28T14:04:23.607668+0000","last_peered":"2024-05-28T14:04:23.607668+0000","last_clean":"2024-05-28T14:04:23.607668+0000","last_became_active":"2024-05-28T14:04:22.564108+0000","last_became_peered":"2024-05-28T14:04:22.564108+0000","last_unstale":"2024-05-28T14:04:23.607668+0000","last_undegraded":"2024-05-28T14:04:23.607668+0000","last_fullsized":"2024-05-28T14:04:23.607668+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:42:43.825729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.570656+0000","last_change":"2024-05-28T14:04:23.585131+0000","last_active":"2024-05-28T14:04:24.570656+0000","last_peered":"2024-05-28T14:04:24.570656+0000","last_clean":"2024-05-28T14:04:24.570656+0000","last_became_active":"2024-05-28T14:04:23.576783+0000","last_became_peered":"2024-05-28T14:04:23.576783+0000","last_unstale":"2024-05-28T14:04:24.570656+0000","last_undegraded":"2024-05-28T14:04:24.570656+0000","last_fullsized":"2024-05-28T14:04:24.570656+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:59:31.979919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.601989+0000","last_change":"2024-05-28T14:04:22.572445+0000","last_active":"2024-05-28T14:04:23.601989+0000","last_peered":"2024-05-28T14:04:23.601989+0000","last_clean":"2024-05-28T14:04:23.601989+0000","last_became_active":"2024-05-28T14:04:22.572128+0000","last_became_peered":"2024-05-28T14:04:22.572128+0000","last_unstale":"2024-05-28T14:04:23.601989+0000","last_undegraded":"2024-05-28T14:04:23.601989+0000","last_fullsized":"2024-05-28T14:04:23.601989+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:47:27.164625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.594549+0000","last_change":"2024-05-28T14:04:23.591567+0000","last_active":"2024-05-28T14:04:23.594549+0000","last_peered":"2024-05-28T14:04:23.594549+0000","last_clean":"2024-05-28T14:04:23.594549+0000","last_became_active":"2024-05-28T14:04:23.591297+0000","last_became_peered":"2024-05-28T14:04:23.591297+0000","last_unstale":"2024-05-28T14:04:23.594549+0000","last_undegraded":"2024-05-28T14:04:23.594549+0000","last_fullsized":"2024-05-28T14:04:23.594549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:51:16.768786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.574694+0000","last_change":"2024-05-28T14:04:22.571901+0000","last_active":"2024-05-28T14:04:23.574694+0000","last_peered":"2024-05-28T14:04:23.574694+0000","last_clean":"2024-05-28T14:04:23.574694+0000","last_became_active":"2024-05-28T14:04:22.571442+0000","last_became_peered":"2024-05-28T14:04:22.571442+0000","last_unstale":"2024-05-28T14:04:23.574694+0000","last_undegraded":"2024-05-28T14:04:23.574694+0000","last_fullsized":"2024-05-28T14:04:23.574694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:03:43.301958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.590858+0000","last_change":"2024-05-28T14:04:23.584036+0000","last_active":"2024-05-28T14:04:23.590858+0000","last_peered":"2024-05-28T14:04:23.590858+0000","last_clean":"2024-05-28T14:04:23.590858+0000","last_became_active":"2024-05-28T14:04:23.583720+0000","last_became_peered":"2024-05-28T14:04:23.583720+0000","last_unstale":"2024-05-28T14:04:23.590858+0000","last_undegraded":"2024-05-28T14:04:23.590858+0000","last_fullsized":"2024-05-28T14:04:23.590858+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:16:03.356703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571052+0000","last_change":"2024-05-28T14:04:22.572924+0000","last_active":"2024-05-28T14:04:24.571052+0000","last_peered":"2024-05-28T14:04:24.571052+0000","last_clean":"2024-05-28T14:04:24.571052+0000","last_became_active":"2024-05-28T14:04:22.572660+0000","last_became_peered":"2024-05-28T14:04:22.572660+0000","last_unstale":"2024-05-28T14:04:24.571052+0000","last_undegraded":"2024-05-28T14:04:24.571052+0000","last_fullsized":"2024-05-28T14:04:24.571052+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:24:52.150831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.596644+0000","last_change":"2024-05-28T14:04:23.593510+0000","last_active":"2024-05-28T14:04:23.596644+0000","last_peered":"2024-05-28T14:04:23.596644+0000","last_clean":"2024-05-28T14:04:23.596644+0000","last_became_active":"2024-05-28T14:04:23.593194+0000","last_became_peered":"2024-05-28T14:04:23.593194+0000","last_unstale":"2024-05-28T14:04:23.596644+0000","last_undegraded":"2024-05-28T14:04:23.596644+0000","last_fullsized":"2024-05-28T14:04:23.596644+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:38:06.191955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"60'8","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.606162+0000","last_change":"2024-05-28T14:04:22.575319+0000","last_active":"2024-05-28T14:04:23.606162+0000","last_peered":"2024-05-28T14:04:23.606162+0000","last_clean":"2024-05-28T14:04:23.606162+0000","last_became_active":"2024-05-28T14:04:22.575041+0000","last_became_peered":"2024-05-28T14:04:22.575041+0000","last_unstale":"2024-05-28T14:04:23.606162+0000","last_undegraded":"2024-05-28T14:04:23.606162+0000","last_fullsized":"2024-05-28T14:04:23.606162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:43:50.366314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.595556+0000","last_change":"2024-05-28T14:04:23.592647+0000","last_active":"2024-05-28T14:04:23.595556+0000","last_peered":"2024-05-28T14:04:23.595556+0000","last_clean":"2024-05-28T14:04:23.595556+0000","last_became_active":"2024-05-28T14:04:23.592303+0000","last_became_peered":"2024-05-28T14:04:23.592303+0000","last_unstale":"2024-05-28T14:04:23.595556+0000","last_undegraded":"2024-05-28T14:04:23.595556+0000","last_fullsized":"2024-05-28T14:04:23.595556+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:04:41.618441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.602343+0000","last_change":"2024-05-28T14:04:22.569944+0000","last_active":"2024-05-28T14:04:23.602343+0000","last_peered":"2024-05-28T14:04:23.602343+0000","last_clean":"2024-05-28T14:04:23.602343+0000","last_became_active":"2024-05-28T14:04:22.569475+0000","last_became_peered":"2024-05-28T14:04:22.569475+0000","last_unstale":"2024-05-28T14:04:23.602343+0000","last_undegraded":"2024-05-28T14:04:23.602343+0000","last_fullsized":"2024-05-28T14:04:23.602343+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:27:44.260902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.570980+0000","last_change":"2024-05-28T14:04:23.581095+0000","last_active":"2024-05-28T14:04:24.570980+0000","last_peered":"2024-05-28T14:04:24.570980+0000","last_clean":"2024-05-28T14:04:24.570980+0000","last_became_active":"2024-05-28T14:04:23.577030+0000","last_became_peered":"2024-05-28T14:04:23.577030+0000","last_unstale":"2024-05-28T14:04:24.570980+0000","last_undegraded":"2024-05-28T14:04:24.570980+0000","last_fullsized":"2024-05-28T14:04:24.570980+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:27:00.745499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590817+0000","last_change":"2024-05-28T14:04:22.574606+0000","last_active":"2024-05-28T14:04:24.590817+0000","last_peered":"2024-05-28T14:04:24.590817+0000","last_clean":"2024-05-28T14:04:24.590817+0000","last_became_active":"2024-05-28T14:04:22.574301+0000","last_became_peered":"2024-05-28T14:04:22.574301+0000","last_unstale":"2024-05-28T14:04:24.590817+0000","last_undegraded":"2024-05-28T14:04:24.590817+0000","last_fullsized":"2024-05-28T14:04:24.590817+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:49:37.723962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.576042+0000","last_change":"2024-05-28T14:04:23.570505+0000","last_active":"2024-05-28T14:04:23.576042+0000","last_peered":"2024-05-28T14:04:23.576042+0000","last_clean":"2024-05-28T14:04:23.576042+0000","last_became_active":"2024-05-28T14:04:23.570190+0000","last_became_peered":"2024-05-28T14:04:23.570190+0000","last_unstale":"2024-05-28T14:04:23.576042+0000","last_undegraded":"2024-05-28T14:04:23.576042+0000","last_fullsized":"2024-05-28T14:04:23.576042+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:51:52.497265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.570949+0000","last_change":"2024-05-28T14:04:22.572545+0000","last_active":"2024-05-28T14:04:24.570949+0000","last_peered":"2024-05-28T14:04:24.570949+0000","last_clean":"2024-05-28T14:04:24.570949+0000","last_became_active":"2024-05-28T14:04:22.572261+0000","last_became_peered":"2024-05-28T14:04:22.572261+0000","last_unstale":"2024-05-28T14:04:24.570949+0000","last_undegraded":"2024-05-28T14:04:24.570949+0000","last_fullsized":"2024-05-28T14:04:24.570949+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:17:46.877238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.593397+0000","last_change":"2024-05-28T14:04:23.588599+0000","last_active":"2024-05-28T14:04:23.593397+0000","last_peered":"2024-05-28T14:04:23.593397+0000","last_clean":"2024-05-28T14:04:23.593397+0000","last_became_active":"2024-05-28T14:04:23.588242+0000","last_became_peered":"2024-05-28T14:04:23.588242+0000","last_unstale":"2024-05-28T14:04:23.593397+0000","last_undegraded":"2024-05-28T14:04:23.593397+0000","last_fullsized":"2024-05-28T14:04:23.593397+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:50:44.894296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590825+0000","last_change":"2024-05-28T14:04:22.567302+0000","last_active":"2024-05-28T14:04:24.590825+0000","last_peered":"2024-05-28T14:04:24.590825+0000","last_clean":"2024-05-28T14:04:24.590825+0000","last_became_active":"2024-05-28T14:04:22.566844+0000","last_became_peered":"2024-05-28T14:04:22.566844+0000","last_unstale":"2024-05-28T14:04:24.590825+0000","last_undegraded":"2024-05-28T14:04:24.590825+0000","last_fullsized":"2024-05-28T14:04:24.590825+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:24:52.150831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571241+0000","last_change":"2024-05-28T14:04:23.581675+0000","last_active":"2024-05-28T14:04:24.571241+0000","last_peered":"2024-05-28T14:04:24.571241+0000","last_clean":"2024-05-28T14:04:24.571241+0000","last_became_active":"2024-05-28T14:04:23.577287+0000","last_became_peered":"2024-05-28T14:04:23.577287+0000","last_unstale":"2024-05-28T14:04:24.571241+0000","last_undegraded":"2024-05-28T14:04:24.571241+0000","last_fullsized":"2024-05-28T14:04:24.571241+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:51:52.497265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.600742+0000","last_change":"2024-05-28T14:04:22.563880+0000","last_active":"2024-05-28T14:04:23.600742+0000","last_peered":"2024-05-28T14:04:23.600742+0000","last_clean":"2024-05-28T14:04:23.600742+0000","last_became_active":"2024-05-28T14:04:22.563540+0000","last_became_peered":"2024-05-28T14:04:22.563540+0000","last_unstale":"2024-05-28T14:04:23.600742+0000","last_undegraded":"2024-05-28T14:04:23.600742+0000","last_fullsized":"2024-05-28T14:04:23.600742+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:50:53.497310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590671+0000","last_change":"2024-05-28T14:04:23.584691+0000","last_active":"2024-05-28T14:04:24.590671+0000","last_peered":"2024-05-28T14:04:24.590671+0000","last_clean":"2024-05-28T14:04:24.590671+0000","last_became_active":"2024-05-28T14:04:23.584375+0000","last_became_peered":"2024-05-28T14:04:23.584375+0000","last_unstale":"2024-05-28T14:04:24.590671+0000","last_undegraded":"2024-05-28T14:04:24.590671+0000","last_fullsized":"2024-05-28T14:04:24.590671+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:50:54.504716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.569839+0000","last_change":"2024-05-28T14:04:22.573373+0000","last_active":"2024-05-28T14:04:23.569839+0000","last_peered":"2024-05-28T14:04:23.569839+0000","last_clean":"2024-05-28T14:04:23.569839+0000","last_became_active":"2024-05-28T14:04:22.573101+0000","last_became_peered":"2024-05-28T14:04:22.573101+0000","last_unstale":"2024-05-28T14:04:23.569839+0000","last_undegraded":"2024-05-28T14:04:23.569839+0000","last_fullsized":"2024-05-28T14:04:23.569839+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:53:13.328390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590434+0000","last_change":"2024-05-28T14:04:23.578247+0000","last_active":"2024-05-28T14:04:24.590434+0000","last_peered":"2024-05-28T14:04:24.590434+0000","last_clean":"2024-05-28T14:04:24.590434+0000","last_became_active":"2024-05-28T14:04:23.577888+0000","last_became_peered":"2024-05-28T14:04:23.577888+0000","last_unstale":"2024-05-28T14:04:24.590434+0000","last_undegraded":"2024-05-28T14:04:24.590434+0000","last_fullsized":"2024-05-28T14:04:24.590434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:27:00.745499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.570554+0000","last_change":"2024-05-28T14:04:22.570054+0000","last_active":"2024-05-28T14:04:24.570554+0000","last_peered":"2024-05-28T14:04:24.570554+0000","last_clean":"2024-05-28T14:04:24.570554+0000","last_became_active":"2024-05-28T14:04:22.569689+0000","last_became_peered":"2024-05-28T14:04:22.569689+0000","last_unstale":"2024-05-28T14:04:24.570554+0000","last_undegraded":"2024-05-28T14:04:24.570554+0000","last_fullsized":"2024-05-28T14:04:24.570554+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:13.052691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571048+0000","last_change":"2024-05-28T14:04:23.582490+0000","last_active":"2024-05-28T14:04:24.571048+0000","last_peered":"2024-05-28T14:04:24.571048+0000","last_clean":"2024-05-28T14:04:24.571048+0000","last_became_active":"2024-05-28T14:04:23.582082+0000","last_became_peered":"2024-05-28T14:04:23.582082+0000","last_unstale":"2024-05-28T14:04:24.571048+0000","last_undegraded":"2024-05-28T14:04:24.571048+0000","last_fullsized":"2024-05-28T14:04:24.571048+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:47:28.172031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.604711+0000","last_change":"2024-05-28T14:04:22.573572+0000","last_active":"2024-05-28T14:04:23.604711+0000","last_peered":"2024-05-28T14:04:23.604711+0000","last_clean":"2024-05-28T14:04:23.604711+0000","last_became_active":"2024-05-28T14:04:22.573350+0000","last_became_peered":"2024-05-28T14:04:22.573350+0000","last_unstale":"2024-05-28T14:04:23.604711+0000","last_undegraded":"2024-05-28T14:04:23.604711+0000","last_fullsized":"2024-05-28T14:04:23.604711+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:15:11.019258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571027+0000","last_change":"2024-05-28T14:04:23.577025+0000","last_active":"2024-05-28T14:04:24.571027+0000","last_peered":"2024-05-28T14:04:24.571027+0000","last_clean":"2024-05-28T14:04:24.571027+0000","last_became_active":"2024-05-28T14:04:23.576475+0000","last_became_peered":"2024-05-28T14:04:23.576475+0000","last_unstale":"2024-05-28T14:04:24.571027+0000","last_undegraded":"2024-05-28T14:04:24.571027+0000","last_fullsized":"2024-05-28T14:04:24.571027+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:20:19.960393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"61'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.592607+0000","last_change":"2024-05-28T14:04:22.566851+0000","last_active":"2024-05-28T14:04:24.592607+0000","last_peered":"2024-05-28T14:04:24.592607+0000","last_clean":"2024-05-28T14:04:24.592607+0000","last_became_active":"2024-05-28T14:04:22.566574+0000","last_became_peered":"2024-05-28T14:04:22.566574+0000","last_unstale":"2024-05-28T14:04:24.592607+0000","last_undegraded":"2024-05-28T14:04:24.592607+0000","last_fullsized":"2024-05-28T14:04:24.592607+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:43:50.366314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.582740+0000","last_change":"2024-05-28T14:04:23.578787+0000","last_active":"2024-05-28T14:04:23.582740+0000","last_peered":"2024-05-28T14:04:23.582740+0000","last_clean":"2024-05-28T14:04:23.582740+0000","last_became_active":"2024-05-28T14:04:23.578399+0000","last_became_peered":"2024-05-28T14:04:23.578399+0000","last_unstale":"2024-05-28T14:04:23.582740+0000","last_undegraded":"2024-05-28T14:04:23.582740+0000","last_fullsized":"2024-05-28T14:04:23.582740+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:56:01.585123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.574904+0000","last_change":"2024-05-28T14:04:22.565649+0000","last_active":"2024-05-28T14:04:23.574904+0000","last_peered":"2024-05-28T14:04:23.574904+0000","last_clean":"2024-05-28T14:04:23.574904+0000","last_became_active":"2024-05-28T14:04:22.565328+0000","last_became_peered":"2024-05-28T14:04:22.565328+0000","last_unstale":"2024-05-28T14:04:23.574904+0000","last_undegraded":"2024-05-28T14:04:23.574904+0000","last_fullsized":"2024-05-28T14:04:23.574904+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:30:05.374724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590907+0000","last_change":"2024-05-28T14:04:23.581565+0000","last_active":"2024-05-28T14:04:24.590907+0000","last_peered":"2024-05-28T14:04:24.590907+0000","last_clean":"2024-05-28T14:04:24.590907+0000","last_became_active":"2024-05-28T14:04:23.581289+0000","last_became_peered":"2024-05-28T14:04:23.581289+0000","last_unstale":"2024-05-28T14:04:24.590907+0000","last_undegraded":"2024-05-28T14:04:24.590907+0000","last_fullsized":"2024-05-28T14:04:24.590907+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:55.492821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571275+0000","last_change":"2024-05-28T14:04:22.566709+0000","last_active":"2024-05-28T14:04:24.571275+0000","last_peered":"2024-05-28T14:04:24.571275+0000","last_clean":"2024-05-28T14:04:24.571275+0000","last_became_active":"2024-05-28T14:04:22.566268+0000","last_became_peered":"2024-05-28T14:04:22.566268+0000","last_unstale":"2024-05-28T14:04:24.571275+0000","last_undegraded":"2024-05-28T14:04:24.571275+0000","last_fullsized":"2024-05-28T14:04:24.571275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:59:04.845137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.587256+0000","last_change":"2024-05-28T14:04:23.577401+0000","last_active":"2024-05-28T14:04:23.587256+0000","last_peered":"2024-05-28T14:04:23.587256+0000","last_clean":"2024-05-28T14:04:23.587256+0000","last_became_active":"2024-05-28T14:04:23.576327+0000","last_became_peered":"2024-05-28T14:04:23.576327+0000","last_unstale":"2024-05-28T14:04:23.587256+0000","last_undegraded":"2024-05-28T14:04:23.587256+0000","last_fullsized":"2024-05-28T14:04:23.587256+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:06:07.558092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.569723+0000","last_change":"2024-05-28T14:04:22.568496+0000","last_active":"2024-05-28T14:04:23.569723+0000","last_peered":"2024-05-28T14:04:23.569723+0000","last_clean":"2024-05-28T14:04:23.569723+0000","last_became_active":"2024-05-28T14:04:22.568188+0000","last_became_peered":"2024-05-28T14:04:22.568188+0000","last_unstale":"2024-05-28T14:04:23.569723+0000","last_undegraded":"2024-05-28T14:04:23.569723+0000","last_fullsized":"2024-05-28T14:04:23.569723+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:32:30.371408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.596641+0000","last_change":"2024-05-28T14:04:23.589242+0000","last_active":"2024-05-28T14:04:23.596641+0000","last_peered":"2024-05-28T14:04:23.596641+0000","last_clean":"2024-05-28T14:04:23.596641+0000","last_became_active":"2024-05-28T14:04:23.588848+0000","last_became_peered":"2024-05-28T14:04:23.588848+0000","last_unstale":"2024-05-28T14:04:23.596641+0000","last_undegraded":"2024-05-28T14:04:23.596641+0000","last_fullsized":"2024-05-28T14:04:23.596641+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:59:31.979919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.567117+0000","last_change":"2024-05-28T14:04:22.570228+0000","last_active":"2024-05-28T14:04:23.567117+0000","last_peered":"2024-05-28T14:04:23.567117+0000","last_clean":"2024-05-28T14:04:23.567117+0000","last_became_active":"2024-05-28T14:04:22.569964+0000","last_became_peered":"2024-05-28T14:04:22.569964+0000","last_unstale":"2024-05-28T14:04:23.567117+0000","last_undegraded":"2024-05-28T14:04:23.567117+0000","last_fullsized":"2024-05-28T14:04:23.567117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:08:45.861158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.594873+0000","last_change":"2024-05-28T14:04:23.582926+0000","last_active":"2024-05-28T14:04:23.594873+0000","last_peered":"2024-05-28T14:04:23.594873+0000","last_clean":"2024-05-28T14:04:23.594873+0000","last_became_active":"2024-05-28T14:04:23.582625+0000","last_became_peered":"2024-05-28T14:04:23.582625+0000","last_unstale":"2024-05-28T14:04:23.594873+0000","last_undegraded":"2024-05-28T14:04:23.594873+0000","last_fullsized":"2024-05-28T14:04:23.594873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:30:06.382130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.571330+0000","last_change":"2024-05-28T14:04:22.565568+0000","last_active":"2024-05-28T14:04:23.571330+0000","last_peered":"2024-05-28T14:04:23.571330+0000","last_clean":"2024-05-28T14:04:23.571330+0000","last_became_active":"2024-05-28T14:04:22.565262+0000","last_became_peered":"2024-05-28T14:04:22.565262+0000","last_unstale":"2024-05-28T14:04:23.571330+0000","last_undegraded":"2024-05-28T14:04:23.571330+0000","last_fullsized":"2024-05-28T14:04:23.571330+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:47:13.344736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588315+0000","last_change":"2024-05-28T14:04:23.586885+0000","last_active":"2024-05-28T14:04:23.588315+0000","last_peered":"2024-05-28T14:04:23.588315+0000","last_clean":"2024-05-28T14:04:23.588315+0000","last_became_active":"2024-05-28T14:04:23.586437+0000","last_became_peered":"2024-05-28T14:04:23.586437+0000","last_unstale":"2024-05-28T14:04:23.588315+0000","last_undegraded":"2024-05-28T14:04:23.588315+0000","last_fullsized":"2024-05-28T14:04:23.588315+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:02:55.459572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.562710+0000","last_change":"2024-05-28T14:04:22.564493+0000","last_active":"2024-05-28T14:04:23.562710+0000","last_peered":"2024-05-28T14:04:23.562710+0000","last_clean":"2024-05-28T14:04:23.562710+0000","last_became_active":"2024-05-28T14:04:22.564185+0000","last_became_peered":"2024-05-28T14:04:22.564185+0000","last_unstale":"2024-05-28T14:04:23.562710+0000","last_undegraded":"2024-05-28T14:04:23.562710+0000","last_fullsized":"2024-05-28T14:04:23.562710+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:38:05.184549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.587629+0000","last_change":"2024-05-28T14:04:23.577031+0000","last_active":"2024-05-28T14:04:23.587629+0000","last_peered":"2024-05-28T14:04:23.587629+0000","last_clean":"2024-05-28T14:04:23.587629+0000","last_became_active":"2024-05-28T14:04:23.576752+0000","last_became_peered":"2024-05-28T14:04:23.576752+0000","last_unstale":"2024-05-28T14:04:23.587629+0000","last_undegraded":"2024-05-28T14:04:23.587629+0000","last_fullsized":"2024-05-28T14:04:23.587629+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:10:22.343727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.576593+0000","last_change":"2024-05-28T14:04:22.574129+0000","last_active":"2024-05-28T14:04:23.576593+0000","last_peered":"2024-05-28T14:04:23.576593+0000","last_clean":"2024-05-28T14:04:23.576593+0000","last_became_active":"2024-05-28T14:04:22.572893+0000","last_became_peered":"2024-05-28T14:04:22.572893+0000","last_unstale":"2024-05-28T14:04:23.576593+0000","last_undegraded":"2024-05-28T14:04:23.576593+0000","last_fullsized":"2024-05-28T14:04:23.576593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:56:34.906444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":243,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.570574+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:04:23.570574+0000","last_peered":"2024-05-28T14:04:23.570574+0000","last_clean":"2024-05-28T14:04:23.570574+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:04:23.570574+0000","last_undegraded":"2024-05-28T14:04:23.570574+0000","last_fullsized":"2024-05-28T14:04:23.570574+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":1323008,"data_stored":892974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45780,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326060,"statfs":{"total":96636764160,"available":96589885440,"internally_reserved":0,"allocated":720896,"data_stored":302446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40220,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331620,"statfs":{"total":96636764160,"available":96595578880,"internally_reserved":0,"allocated":729088,"data_stored":303098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45792,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326048,"statfs":{"total":96636764160,"available":96589873152,"internally_reserved":0,"allocated":733184,"data_stored":304058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49948,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321892,"statfs":{"total":96636764160,"available":96585617408,"internally_reserved":0,"allocated":729088,"data_stored":302502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45780,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326060,"statfs":{"total":96636764160,"available":96589885440,"internally_reserved":0,"allocated":729088,"data_stored":302502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49936,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321904,"statfs":{"total":96636764160,"available":96585629696,"internally_reserved":0,"allocated":724992,"data_stored":302536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50532,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321308,"statfs":{"total":96636764160,"available":96585019392,"internally_reserved":0,"allocated":1331200,"data_stored":895008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:04:27.692 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:04:27.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77f8400640 1 Processor -- start 2024-05-28T14:04:27.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77f8400640 1 -- start start 2024-05-28T14:04:27.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77f8400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f015c990 0x7f77f00d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77f8400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77f00cd300 con 0x7f77f009bbd0 2024-05-28T14:04:27.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77f8400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77f00cd5e0 con 0x7f77f00d06a0 2024-05-28T14:04:27.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77f8400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f77f015cd70 con 0x7f77f015c990 2024-05-28T14:04:27.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77f5000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f77f009bbd0 0x7f77f009aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51246/0 (socket says 172.21.15.42:51246) 2024-05-28T14:04:27.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77f5000640 1 -- 172.21.15.42:0/3884116399 learned_addr learned my addr 172.21.15.42:0/3884116399 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:27.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77f5a00640 1 --2- 172.21.15.42:0/3884116399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f015c990 0x7f77f00d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77efe00640 1 -- 172.21.15.42:0/3884116399 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2125397027 0 0) 0x7f77f00cd300 con 0x7f77f009bbd0 2024-05-28T14:04:27.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77efe00640 1 -- 172.21.15.42:0/3884116399 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77d8003530 con 0x7f77f009bbd0 2024-05-28T14:04:27.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77efe00640 1 -- 172.21.15.42:0/3884116399 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 112659226 0 0) 0x7f77f00cd5e0 con 0x7f77f00d06a0 2024-05-28T14:04:27.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77efe00640 1 -- 172.21.15.42:0/3884116399 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77f00cd300 con 0x7f77f00d06a0 2024-05-28T14:04:27.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77f5a00640 1 -- 172.21.15.42:0/3884116399 >> v1:172.21.15.63:6789/0 conn(0x7f77f00d06a0 legacy=0x7f77f00b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77f5a00640 1 -- 172.21.15.42:0/3884116399 >> v1:172.21.15.42:6789/0 conn(0x7f77f009bbd0 legacy=0x7f77f009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77efe00640 1 -- 172.21.15.42:0/3884116399 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1456790791 0 0) 0x7f77d8003530 con 0x7f77f009bbd0 2024-05-28T14:04:27.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.887+0000 7f77f5a00640 1 -- 172.21.15.42:0/3884116399 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f00cd8c0 con 0x7f77f015c990 2024-05-28T14:04:27.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.891+0000 7f77f5a00640 1 --2- 172.21.15.42:0/3884116399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f015c990 0x7f77f00d0160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f77e40097f0 tx=0x7f77e401c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cbc04b7e4c4da7f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:27.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.891+0000 7f77efe00640 1 -- 172.21.15.42:0/3884116399 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e40b0070 con 0x7f77f015c990 2024-05-28T14:04:27.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.891+0000 7f77efe00640 1 -- 172.21.15.42:0/3884116399 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e4004030 con 0x7f77f015c990 2024-05-28T14:04:27.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.891+0000 7f77efe00640 1 -- 172.21.15.42:0/3884116399 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc001680 con 0x7f77f015c990 2024-05-28T14:04:27.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.891+0000 7f77f8400640 1 -- 172.21.15.42:0/3884116399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f015c990 msgr2=0x7f77f00d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.891+0000 7f77f8400640 1 --2- 172.21.15.42:0/3884116399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f015c990 0x7f77f00d0160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f77e40097f0 tx=0x7f77e401c3d0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.891+0000 7f77f8400640 1 -- 172.21.15.42:0/3884116399 shutdown_connections 2024-05-28T14:04:27.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.891+0000 7f77f8400640 1 --2- 172.21.15.42:0/3884116399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f015c990 0x7f77f00d0160 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.891+0000 7f77f8400640 1 -- 172.21.15.42:0/3884116399 >> 172.21.15.42:0/3884116399 conn(0x7f77f00b4080 msgr2=0x7f77f00b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:27.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.891+0000 7f77f8400640 1 -- 172.21.15.42:0/3884116399 shutdown_connections 2024-05-28T14:04:27.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.891+0000 7f77f8400640 1 -- 172.21.15.42:0/3884116399 wait complete. 2024-05-28T14:04:27.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f8400640 1 Processor -- start 2024-05-28T14:04:27.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f8400640 1 -- start start 2024-05-28T14:04:27.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f8400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f00d06a0 0x7f77f015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f8400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f015c990 0x7f77f00c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f8400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77f00ca070 0x7f77f00ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f8400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f77f00ce3a0 con 0x7f77f015c990 2024-05-28T14:04:27.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f8400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77f009b160 con 0x7f77f00ca070 2024-05-28T14:04:27.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f8400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f77f00c7d60 con 0x7f77f00d06a0 2024-05-28T14:04:27.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f015c990 0x7f77f00c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f015c990 0x7f77f00c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60868/0 (socket says 172.21.15.42:60868) 2024-05-28T14:04:27.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f5000640 1 -- 172.21.15.42:0/186800900 learned_addr learned my addr 172.21.15.42:0/186800900 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:27.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f6400640 1 --2- 172.21.15.42:0/186800900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77f00ca070 0x7f77f00ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f5a00640 1 --2- 172.21.15.42:0/186800900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f00d06a0 0x7f77f015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f5000640 1 -- 172.21.15.42:0/186800900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f00d06a0 msgr2=0x7f77f015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f5000640 1 --2- 172.21.15.42:0/186800900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f00d06a0 0x7f77f015ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f5000640 1 -- 172.21.15.42:0/186800900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77f00ca070 msgr2=0x7f77f00ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:27.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f5000640 1 --2- 172.21.15.42:0/186800900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77f00ca070 0x7f77f00ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:27.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f5000640 1 -- 172.21.15.42:0/186800900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f00c7150 con 0x7f77f015c990 2024-05-28T14:04:27.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f5000640 1 --2- 172.21.15.42:0/186800900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f015c990 0x7f77f00c81f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00c7b0 tx=0x7f77dc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77ed600640 1 -- 172.21.15.42:0/186800900 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc017070 con 0x7f77f015c990 2024-05-28T14:04:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77ed600640 1 -- 172.21.15.42:0/186800900 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77dc005b20 con 0x7f77f015c990 2024-05-28T14:04:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77ed600640 1 -- 172.21.15.42:0/186800900 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc007070 con 0x7f77f015c990 2024-05-28T14:04:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f8400640 1 -- 172.21.15.42:0/186800900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f00c7430 con 0x7f77f015c990 2024-05-28T14:04:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.895+0000 7f77f8400640 1 -- 172.21.15.42:0/186800900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f00c79f0 con 0x7f77f015c990 2024-05-28T14:04:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.899+0000 7f77f8400640 1 -- 172.21.15.42:0/186800900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f0068a90 con 0x7f77f015c990 2024-05-28T14:04:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.899+0000 7f77ed600640 1 -- 172.21.15.42:0/186800900 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f77dc01a070 con 0x7f77f015c990 2024-05-28T14:04:27.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.899+0000 7f77ed600640 1 --2- 172.21.15.42:0/186800900 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f77d0068b60 0x7f77d006b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:27.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.899+0000 7f77ed600640 1 -- 172.21.15.42:0/186800900 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f77dc0870b0 con 0x7f77f015c990 2024-05-28T14:04:27.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.899+0000 7f77f5a00640 1 --2- 172.21.15.42:0/186800900 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f77d0068b60 0x7f77d006b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:27.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.899+0000 7f77f5a00640 1 --2- 172.21.15.42:0/186800900 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f77d0068b60 0x7f77d006b020 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f77e40ad000 tx=0x7f77e40a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:27.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:27.903+0000 7f77ed600640 1 -- 172.21.15.42:0/186800900 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f77dc056070 con 0x7f77f015c990 2024-05-28T14:04:28.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.067+0000 7f77f8400640 1 -- 172.21.15.42:0/186800900 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f77f00ce520 con 0x7f77d0068b60 2024-05-28T14:04:28.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.075+0000 7f77ed600640 1 -- 172.21.15.42:0/186800900 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251910 (secure 0 0 0) 0x7f77f00ce520 con 0x7f77d0068b60 2024-05-28T14:04:28.064 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:28.065 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:04:28.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.079+0000 7f77f8400640 1 -- 172.21.15.42:0/186800900 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f77d0068b60 msgr2=0x7f77d006b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:28.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.079+0000 7f77f8400640 1 --2- 172.21.15.42:0/186800900 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f77d0068b60 0x7f77d006b020 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f77e40ad000 tx=0x7f77e40a6000 comp rx=0 tx=0).stop 2024-05-28T14:04:28.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.079+0000 7f77f8400640 1 -- 172.21.15.42:0/186800900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f015c990 msgr2=0x7f77f00c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:28.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.079+0000 7f77f8400640 1 --2- 172.21.15.42:0/186800900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f015c990 0x7f77f00c81f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00c7b0 tx=0x7f77dc00cc70 comp rx=0 tx=0).stop 2024-05-28T14:04:28.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.079+0000 7f77f8400640 1 -- 172.21.15.42:0/186800900 shutdown_connections 2024-05-28T14:04:28.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.079+0000 7f77f8400640 1 --2- 172.21.15.42:0/186800900 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f77d0068b60 0x7f77d006b020 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.079+0000 7f77f8400640 1 --2- 172.21.15.42:0/186800900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77f00ca070 0x7f77f00ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.079+0000 7f77f8400640 1 --2- 172.21.15.42:0/186800900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f015c990 0x7f77f00c81f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.079+0000 7f77f8400640 1 --2- 172.21.15.42:0/186800900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f00d06a0 0x7f77f015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.079+0000 7f77f8400640 1 -- 172.21.15.42:0/186800900 >> 172.21.15.42:0/186800900 conn(0x7f77f00b4080 msgr2=0x7f77f00b7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:28.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.079+0000 7f77f8400640 1 -- 172.21.15.42:0/186800900 shutdown_connections 2024-05-28T14:04:28.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.079+0000 7f77f8400640 1 -- 172.21.15.42:0/186800900 wait complete. 2024-05-28T14:04:28.084 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-05-28T14:04:26.944166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356304,"kb_used_data":6856,"kb_used_omap":12,"kb_used_meta":349299,"kb_avail":754618416,"statfs":{"total":773094113280,"available":772729257984,"internally_reserved":0,"allocated":7020544,"data_stored":3605124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":357682738},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.400031"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.563010+0000","last_change":"2024-05-28T14:04:23.593610+0000","last_active":"2024-05-28T14:04:24.563010+0000","last_peered":"2024-05-28T14:04:24.563010+0000","last_clean":"2024-05-28T14:04:24.563010+0000","last_became_active":"2024-05-28T14:04:23.593289+0000","last_became_peered":"2024-05-28T14:04:23.593289+0000","last_unstale":"2024-05-28T14:04:24.563010+0000","last_undegraded":"2024-05-28T14:04:24.563010+0000","last_fullsized":"2024-05-28T14:04:24.563010+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:57.872099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588414+0000","last_change":"2024-05-28T14:04:23.583536+0000","last_active":"2024-05-28T14:04:23.588414+0000","last_peered":"2024-05-28T14:04:23.588414+0000","last_clean":"2024-05-28T14:04:23.588414+0000","last_became_active":"2024-05-28T14:04:23.583284+0000","last_became_peered":"2024-05-28T14:04:23.583284+0000","last_unstale":"2024-05-28T14:04:23.588414+0000","last_undegraded":"2024-05-28T14:04:23.588414+0000","last_fullsized":"2024-05-28T14:04:23.588414+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:03:35.986922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588441+0000","last_change":"2024-05-28T14:04:23.581101+0000","last_active":"2024-05-28T14:04:23.588441+0000","last_peered":"2024-05-28T14:04:23.588441+0000","last_clean":"2024-05-28T14:04:23.588441+0000","last_became_active":"2024-05-28T14:04:23.580815+0000","last_became_peered":"2024-05-28T14:04:23.580815+0000","last_unstale":"2024-05-28T14:04:23.588441+0000","last_undegraded":"2024-05-28T14:04:23.588441+0000","last_fullsized":"2024-05-28T14:04:23.588441+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:59:05.852542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571147+0000","last_change":"2024-05-28T14:04:23.574659+0000","last_active":"2024-05-28T14:04:24.571147+0000","last_peered":"2024-05-28T14:04:24.571147+0000","last_clean":"2024-05-28T14:04:24.571147+0000","last_became_active":"2024-05-28T14:04:23.574298+0000","last_became_peered":"2024-05-28T14:04:23.574298+0000","last_unstale":"2024-05-28T14:04:24.571147+0000","last_undegraded":"2024-05-28T14:04:24.571147+0000","last_fullsized":"2024-05-28T14:04:24.571147+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:27:45.268308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.562649+0000","last_change":"2024-05-28T14:04:23.589646+0000","last_active":"2024-05-28T14:04:24.562649+0000","last_peered":"2024-05-28T14:04:24.562649+0000","last_clean":"2024-05-28T14:04:24.562649+0000","last_became_active":"2024-05-28T14:04:23.585954+0000","last_became_peered":"2024-05-28T14:04:23.585954+0000","last_unstale":"2024-05-28T14:04:24.562649+0000","last_undegraded":"2024-05-28T14:04:24.562649+0000","last_fullsized":"2024-05-28T14:04:24.562649+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:48:13.320287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590433+0000","last_change":"2024-05-28T14:04:23.581910+0000","last_active":"2024-05-28T14:04:24.590433+0000","last_peered":"2024-05-28T14:04:24.590433+0000","last_clean":"2024-05-28T14:04:24.590433+0000","last_became_active":"2024-05-28T14:04:23.581609+0000","last_became_peered":"2024-05-28T14:04:23.581609+0000","last_unstale":"2024-05-28T14:04:24.590433+0000","last_undegraded":"2024-05-28T14:04:24.590433+0000","last_fullsized":"2024-05-28T14:04:24.590433+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:20:19.960393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.590710+0000","last_change":"2024-05-28T14:04:23.583917+0000","last_active":"2024-05-28T14:04:23.590710+0000","last_peered":"2024-05-28T14:04:23.590710+0000","last_clean":"2024-05-28T14:04:23.590710+0000","last_became_active":"2024-05-28T14:04:23.583493+0000","last_became_peered":"2024-05-28T14:04:23.583493+0000","last_unstale":"2024-05-28T14:04:23.590710+0000","last_undegraded":"2024-05-28T14:04:23.590710+0000","last_fullsized":"2024-05-28T14:04:23.590710+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:05:47.954430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588064+0000","last_change":"2024-05-28T14:04:23.576473+0000","last_active":"2024-05-28T14:04:23.588064+0000","last_peered":"2024-05-28T14:04:23.588064+0000","last_clean":"2024-05-28T14:04:23.588064+0000","last_became_active":"2024-05-28T14:04:23.576146+0000","last_became_peered":"2024-05-28T14:04:23.576146+0000","last_unstale":"2024-05-28T14:04:23.588064+0000","last_undegraded":"2024-05-28T14:04:23.588064+0000","last_fullsized":"2024-05-28T14:04:23.588064+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:57:02.905276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.579928+0000","last_change":"2024-05-28T14:04:23.577882+0000","last_active":"2024-05-28T14:04:23.579928+0000","last_peered":"2024-05-28T14:04:23.579928+0000","last_clean":"2024-05-28T14:04:23.579928+0000","last_became_active":"2024-05-28T14:04:23.577557+0000","last_became_peered":"2024-05-28T14:04:23.577557+0000","last_unstale":"2024-05-28T14:04:23.579928+0000","last_undegraded":"2024-05-28T14:04:23.579928+0000","last_fullsized":"2024-05-28T14:04:23.579928+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:29:22.224097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.594974+0000","last_change":"2024-05-28T14:04:23.582604+0000","last_active":"2024-05-28T14:04:23.594974+0000","last_peered":"2024-05-28T14:04:23.594974+0000","last_clean":"2024-05-28T14:04:23.594974+0000","last_became_active":"2024-05-28T14:04:23.582302+0000","last_became_peered":"2024-05-28T14:04:23.582302+0000","last_unstale":"2024-05-28T14:04:23.594974+0000","last_undegraded":"2024-05-28T14:04:23.594974+0000","last_fullsized":"2024-05-28T14:04:23.594974+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:03:44.309364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.595884+0000","last_change":"2024-05-28T14:04:23.589273+0000","last_active":"2024-05-28T14:04:23.595884+0000","last_peered":"2024-05-28T14:04:23.595884+0000","last_clean":"2024-05-28T14:04:23.595884+0000","last_became_active":"2024-05-28T14:04:23.588934+0000","last_became_peered":"2024-05-28T14:04:23.588934+0000","last_unstale":"2024-05-28T14:04:23.595884+0000","last_undegraded":"2024-05-28T14:04:23.595884+0000","last_fullsized":"2024-05-28T14:04:23.595884+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:46:52.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571106+0000","last_change":"2024-05-28T14:04:23.577217+0000","last_active":"2024-05-28T14:04:24.571106+0000","last_peered":"2024-05-28T14:04:24.571106+0000","last_clean":"2024-05-28T14:04:24.571106+0000","last_became_active":"2024-05-28T14:04:23.576880+0000","last_became_peered":"2024-05-28T14:04:23.576880+0000","last_unstale":"2024-05-28T14:04:24.571106+0000","last_undegraded":"2024-05-28T14:04:24.571106+0000","last_fullsized":"2024-05-28T14:04:24.571106+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:50:54.504716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.585295+0000","last_change":"2024-05-28T14:04:23.582966+0000","last_active":"2024-05-28T14:04:23.585295+0000","last_peered":"2024-05-28T14:04:23.585295+0000","last_clean":"2024-05-28T14:04:23.585295+0000","last_became_active":"2024-05-28T14:04:23.582695+0000","last_became_peered":"2024-05-28T14:04:23.582695+0000","last_unstale":"2024-05-28T14:04:23.585295+0000","last_undegraded":"2024-05-28T14:04:23.585295+0000","last_fullsized":"2024-05-28T14:04:23.585295+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:47:25.790192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588503+0000","last_change":"2024-05-28T14:04:23.582534+0000","last_active":"2024-05-28T14:04:23.588503+0000","last_peered":"2024-05-28T14:04:23.588503+0000","last_clean":"2024-05-28T14:04:23.588503+0000","last_became_active":"2024-05-28T14:04:23.582248+0000","last_became_peered":"2024-05-28T14:04:23.582248+0000","last_unstale":"2024-05-28T14:04:23.588503+0000","last_undegraded":"2024-05-28T14:04:23.588503+0000","last_fullsized":"2024-05-28T14:04:23.588503+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:20:27.689523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.580932+0000","last_change":"2024-05-28T14:04:23.575701+0000","last_active":"2024-05-28T14:04:23.580932+0000","last_peered":"2024-05-28T14:04:23.580932+0000","last_clean":"2024-05-28T14:04:23.580932+0000","last_became_active":"2024-05-28T14:04:23.575393+0000","last_became_peered":"2024-05-28T14:04:23.575393+0000","last_unstale":"2024-05-28T14:04:23.580932+0000","last_undegraded":"2024-05-28T14:04:23.580932+0000","last_fullsized":"2024-05-28T14:04:23.580932+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:22:52.389811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590360+0000","last_change":"2024-05-28T14:04:23.584296+0000","last_active":"2024-05-28T14:04:24.590360+0000","last_peered":"2024-05-28T14:04:24.590360+0000","last_clean":"2024-05-28T14:04:24.590360+0000","last_became_active":"2024-05-28T14:04:23.583997+0000","last_became_peered":"2024-05-28T14:04:23.583997+0000","last_unstale":"2024-05-28T14:04:24.590360+0000","last_undegraded":"2024-05-28T14:04:24.590360+0000","last_fullsized":"2024-05-28T14:04:24.590360+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:59:31.979919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.587700+0000","last_change":"2024-05-28T14:04:23.579745+0000","last_active":"2024-05-28T14:04:23.587700+0000","last_peered":"2024-05-28T14:04:23.587700+0000","last_clean":"2024-05-28T14:04:23.587700+0000","last_became_active":"2024-05-28T14:04:23.579481+0000","last_became_peered":"2024-05-28T14:04:23.579481+0000","last_unstale":"2024-05-28T14:04:23.587700+0000","last_undegraded":"2024-05-28T14:04:23.587700+0000","last_fullsized":"2024-05-28T14:04:23.587700+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:51:55.173568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.595535+0000","last_change":"2024-05-28T14:04:23.589867+0000","last_active":"2024-05-28T14:04:23.595535+0000","last_peered":"2024-05-28T14:04:23.595535+0000","last_clean":"2024-05-28T14:04:23.595535+0000","last_became_active":"2024-05-28T14:04:23.589596+0000","last_became_peered":"2024-05-28T14:04:23.589596+0000","last_unstale":"2024-05-28T14:04:23.595535+0000","last_undegraded":"2024-05-28T14:04:23.595535+0000","last_fullsized":"2024-05-28T14:04:23.595535+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:22:11.458492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.562845+0000","last_change":"2024-05-28T14:04:23.593765+0000","last_active":"2024-05-28T14:04:24.562845+0000","last_peered":"2024-05-28T14:04:24.562845+0000","last_clean":"2024-05-28T14:04:24.562845+0000","last_became_active":"2024-05-28T14:04:23.593455+0000","last_became_peered":"2024-05-28T14:04:23.593455+0000","last_unstale":"2024-05-28T14:04:24.562845+0000","last_undegraded":"2024-05-28T14:04:24.562845+0000","last_fullsized":"2024-05-28T14:04:24.562845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:06:07.071310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.587974+0000","last_change":"2024-05-28T14:04:23.580408+0000","last_active":"2024-05-28T14:04:23.587974+0000","last_peered":"2024-05-28T14:04:23.587974+0000","last_clean":"2024-05-28T14:04:23.587974+0000","last_became_active":"2024-05-28T14:04:23.577918+0000","last_became_peered":"2024-05-28T14:04:23.577918+0000","last_unstale":"2024-05-28T14:04:23.587974+0000","last_undegraded":"2024-05-28T14:04:23.587974+0000","last_fullsized":"2024-05-28T14:04:23.587974+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:22:12.704246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.587853+0000","last_change":"2024-05-28T14:04:23.583279+0000","last_active":"2024-05-28T14:04:23.587853+0000","last_peered":"2024-05-28T14:04:23.587853+0000","last_clean":"2024-05-28T14:04:23.587853+0000","last_became_active":"2024-05-28T14:04:23.583012+0000","last_became_peered":"2024-05-28T14:04:23.583012+0000","last_unstale":"2024-05-28T14:04:23.587853+0000","last_undegraded":"2024-05-28T14:04:23.587853+0000","last_fullsized":"2024-05-28T14:04:23.587853+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:07:44.967029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588117+0000","last_change":"2024-05-28T14:04:23.585380+0000","last_active":"2024-05-28T14:04:23.588117+0000","last_peered":"2024-05-28T14:04:23.588117+0000","last_clean":"2024-05-28T14:04:23.588117+0000","last_became_active":"2024-05-28T14:04:23.582647+0000","last_became_peered":"2024-05-28T14:04:23.582647+0000","last_unstale":"2024-05-28T14:04:23.588117+0000","last_undegraded":"2024-05-28T14:04:23.588117+0000","last_fullsized":"2024-05-28T14:04:23.588117+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:17:01.328216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.594366+0000","last_change":"2024-05-28T14:04:23.590713+0000","last_active":"2024-05-28T14:04:23.594366+0000","last_peered":"2024-05-28T14:04:23.594366+0000","last_clean":"2024-05-28T14:04:23.594366+0000","last_became_active":"2024-05-28T14:04:23.590441+0000","last_became_peered":"2024-05-28T14:04:23.590441+0000","last_unstale":"2024-05-28T14:04:23.594366+0000","last_undegraded":"2024-05-28T14:04:23.594366+0000","last_fullsized":"2024-05-28T14:04:23.594366+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:16:28.733076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.583617+0000","last_change":"2024-05-28T14:04:23.580197+0000","last_active":"2024-05-28T14:04:23.583617+0000","last_peered":"2024-05-28T14:04:23.583617+0000","last_clean":"2024-05-28T14:04:23.583617+0000","last_became_active":"2024-05-28T14:04:23.579864+0000","last_became_peered":"2024-05-28T14:04:23.579864+0000","last_unstale":"2024-05-28T14:04:23.583617+0000","last_undegraded":"2024-05-28T14:04:23.583617+0000","last_fullsized":"2024-05-28T14:04:23.583617+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:46:52.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.563123+0000","last_change":"2024-05-28T14:04:23.583792+0000","last_active":"2024-05-28T14:04:24.563123+0000","last_peered":"2024-05-28T14:04:24.563123+0000","last_clean":"2024-05-28T14:04:24.563123+0000","last_became_active":"2024-05-28T14:04:23.583325+0000","last_became_peered":"2024-05-28T14:04:23.583325+0000","last_unstale":"2024-05-28T14:04:24.563123+0000","last_undegraded":"2024-05-28T14:04:24.563123+0000","last_fullsized":"2024-05-28T14:04:24.563123+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:11:27.816943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.570949+0000","last_change":"2024-05-28T14:04:23.578927+0000","last_active":"2024-05-28T14:04:24.570949+0000","last_peered":"2024-05-28T14:04:24.570949+0000","last_clean":"2024-05-28T14:04:24.570949+0000","last_became_active":"2024-05-28T14:04:23.576566+0000","last_became_peered":"2024-05-28T14:04:23.576566+0000","last_unstale":"2024-05-28T14:04:24.570949+0000","last_undegraded":"2024-05-28T14:04:24.570949+0000","last_fullsized":"2024-05-28T14:04:24.570949+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:38:06.191955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590941+0000","last_change":"2024-05-28T14:04:23.585543+0000","last_active":"2024-05-28T14:04:24.590941+0000","last_peered":"2024-05-28T14:04:24.590941+0000","last_clean":"2024-05-28T14:04:24.590941+0000","last_became_active":"2024-05-28T14:04:23.585265+0000","last_became_peered":"2024-05-28T14:04:23.585265+0000","last_unstale":"2024-05-28T14:04:24.590941+0000","last_undegraded":"2024-05-28T14:04:24.590941+0000","last_fullsized":"2024-05-28T14:04:24.590941+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:06:07.558092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.562904+0000","last_change":"2024-05-28T14:04:23.590936+0000","last_active":"2024-05-28T14:04:24.562904+0000","last_peered":"2024-05-28T14:04:24.562904+0000","last_clean":"2024-05-28T14:04:24.562904+0000","last_became_active":"2024-05-28T14:04:23.583135+0000","last_became_peered":"2024-05-28T14:04:23.583135+0000","last_unstale":"2024-05-28T14:04:24.562904+0000","last_undegraded":"2024-05-28T14:04:24.562904+0000","last_fullsized":"2024-05-28T14:04:24.562904+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:09:01.172689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588085+0000","last_change":"2024-05-28T14:04:23.585146+0000","last_active":"2024-05-28T14:04:23.588085+0000","last_peered":"2024-05-28T14:04:23.588085+0000","last_clean":"2024-05-28T14:04:23.588085+0000","last_became_active":"2024-05-28T14:04:23.584842+0000","last_became_peered":"2024-05-28T14:04:23.584842+0000","last_unstale":"2024-05-28T14:04:23.588085+0000","last_undegraded":"2024-05-28T14:04:23.588085+0000","last_fullsized":"2024-05-28T14:04:23.588085+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:57:03.942825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.581766+0000","last_change":"2024-05-28T14:04:23.579833+0000","last_active":"2024-05-28T14:04:23.581766+0000","last_peered":"2024-05-28T14:04:23.581766+0000","last_clean":"2024-05-28T14:04:23.581766+0000","last_became_active":"2024-05-28T14:04:23.579461+0000","last_became_peered":"2024-05-28T14:04:23.579461+0000","last_unstale":"2024-05-28T14:04:23.581766+0000","last_undegraded":"2024-05-28T14:04:23.581766+0000","last_fullsized":"2024-05-28T14:04:23.581766+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:12:00.209164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.562644+0000","last_change":"2024-05-28T14:04:23.574780+0000","last_active":"2024-05-28T14:04:24.562644+0000","last_peered":"2024-05-28T14:04:24.562644+0000","last_clean":"2024-05-28T14:04:24.562644+0000","last_became_active":"2024-05-28T14:04:23.574504+0000","last_became_peered":"2024-05-28T14:04:23.574504+0000","last_unstale":"2024-05-28T14:04:24.562644+0000","last_undegraded":"2024-05-28T14:04:24.562644+0000","last_fullsized":"2024-05-28T14:04:24.562644+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:23:07.598206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590431+0000","last_change":"2024-05-28T14:04:23.588278+0000","last_active":"2024-05-28T14:04:24.590431+0000","last_peered":"2024-05-28T14:04:24.590431+0000","last_clean":"2024-05-28T14:04:24.590431+0000","last_became_active":"2024-05-28T14:04:23.588005+0000","last_became_peered":"2024-05-28T14:04:23.588005+0000","last_unstale":"2024-05-28T14:04:24.590431+0000","last_undegraded":"2024-05-28T14:04:24.590431+0000","last_fullsized":"2024-05-28T14:04:24.590431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:38:06.191955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590788+0000","last_change":"2024-05-28T14:04:23.579518+0000","last_active":"2024-05-28T14:04:24.590788+0000","last_peered":"2024-05-28T14:04:24.590788+0000","last_clean":"2024-05-28T14:04:24.590788+0000","last_became_active":"2024-05-28T14:04:23.579136+0000","last_became_peered":"2024-05-28T14:04:23.579136+0000","last_unstale":"2024-05-28T14:04:24.590788+0000","last_undegraded":"2024-05-28T14:04:24.590788+0000","last_fullsized":"2024-05-28T14:04:24.590788+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:51:52.497265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"60'4","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590845+0000","last_change":"2024-05-28T14:04:22.567917+0000","last_active":"2024-05-28T14:04:24.590845+0000","last_peered":"2024-05-28T14:04:24.590845+0000","last_clean":"2024-05-28T14:04:24.590845+0000","last_became_active":"2024-05-28T14:04:22.567634+0000","last_became_peered":"2024-05-28T14:04:22.567634+0000","last_unstale":"2024-05-28T14:04:24.590845+0000","last_undegraded":"2024-05-28T14:04:24.590845+0000","last_fullsized":"2024-05-28T14:04:24.590845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:17:46.877238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.594706+0000","last_change":"2024-05-28T14:04:23.584443+0000","last_active":"2024-05-28T14:04:23.594706+0000","last_peered":"2024-05-28T14:04:23.594706+0000","last_clean":"2024-05-28T14:04:23.594706+0000","last_became_active":"2024-05-28T14:04:23.582096+0000","last_became_peered":"2024-05-28T14:04:23.582096+0000","last_unstale":"2024-05-28T14:04:23.594706+0000","last_undegraded":"2024-05-28T14:04:23.594706+0000","last_fullsized":"2024-05-28T14:04:23.594706+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:56:35.913850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.602693+0000","last_change":"2024-05-28T14:04:22.571536+0000","last_active":"2024-05-28T14:04:23.602693+0000","last_peered":"2024-05-28T14:04:23.602693+0000","last_clean":"2024-05-28T14:04:23.602693+0000","last_became_active":"2024-05-28T14:04:22.571268+0000","last_became_peered":"2024-05-28T14:04:22.571268+0000","last_unstale":"2024-05-28T14:04:23.602693+0000","last_undegraded":"2024-05-28T14:04:23.602693+0000","last_fullsized":"2024-05-28T14:04:23.602693+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:59.738093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590642+0000","last_change":"2024-05-28T14:04:23.579400+0000","last_active":"2024-05-28T14:04:24.590642+0000","last_peered":"2024-05-28T14:04:24.590642+0000","last_clean":"2024-05-28T14:04:24.590642+0000","last_became_active":"2024-05-28T14:04:23.579102+0000","last_became_peered":"2024-05-28T14:04:23.579102+0000","last_unstale":"2024-05-28T14:04:24.590642+0000","last_undegraded":"2024-05-28T14:04:24.590642+0000","last_fullsized":"2024-05-28T14:04:24.590642+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:47:28.172031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590989+0000","last_change":"2024-05-28T14:04:22.569071+0000","last_active":"2024-05-28T14:04:24.590989+0000","last_peered":"2024-05-28T14:04:24.590989+0000","last_clean":"2024-05-28T14:04:24.590989+0000","last_became_active":"2024-05-28T14:04:22.568655+0000","last_became_peered":"2024-05-28T14:04:22.568655+0000","last_unstale":"2024-05-28T14:04:24.590989+0000","last_undegraded":"2024-05-28T14:04:24.590989+0000","last_fullsized":"2024-05-28T14:04:24.590989+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:13.052691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590773+0000","last_change":"2024-05-28T14:04:23.580842+0000","last_active":"2024-05-28T14:04:24.590773+0000","last_peered":"2024-05-28T14:04:24.590773+0000","last_clean":"2024-05-28T14:04:24.590773+0000","last_became_active":"2024-05-28T14:04:23.580548+0000","last_became_peered":"2024-05-28T14:04:23.580548+0000","last_unstale":"2024-05-28T14:04:24.590773+0000","last_undegraded":"2024-05-28T14:04:24.590773+0000","last_fullsized":"2024-05-28T14:04:24.590773+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:27:45.268308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571192+0000","last_change":"2024-05-28T14:04:22.569314+0000","last_active":"2024-05-28T14:04:24.571192+0000","last_peered":"2024-05-28T14:04:24.571192+0000","last_clean":"2024-05-28T14:04:24.571192+0000","last_became_active":"2024-05-28T14:04:22.569104+0000","last_became_peered":"2024-05-28T14:04:22.569104+0000","last_unstale":"2024-05-28T14:04:24.571192+0000","last_undegraded":"2024-05-28T14:04:24.571192+0000","last_fullsized":"2024-05-28T14:04:24.571192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:11:59.201758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.587968+0000","last_change":"2024-05-28T14:04:23.582632+0000","last_active":"2024-05-28T14:04:23.587968+0000","last_peered":"2024-05-28T14:04:23.587968+0000","last_clean":"2024-05-28T14:04:23.587968+0000","last_became_active":"2024-05-28T14:04:23.582290+0000","last_became_peered":"2024-05-28T14:04:23.582290+0000","last_unstale":"2024-05-28T14:04:23.587968+0000","last_undegraded":"2024-05-28T14:04:23.587968+0000","last_fullsized":"2024-05-28T14:04:23.587968+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:11:47.754807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.603730+0000","last_change":"2024-05-28T14:04:22.573692+0000","last_active":"2024-05-28T14:04:23.603730+0000","last_peered":"2024-05-28T14:04:23.603730+0000","last_clean":"2024-05-28T14:04:23.603730+0000","last_became_active":"2024-05-28T14:04:22.573412+0000","last_became_peered":"2024-05-28T14:04:22.573412+0000","last_unstale":"2024-05-28T14:04:23.603730+0000","last_undegraded":"2024-05-28T14:04:23.603730+0000","last_fullsized":"2024-05-28T14:04:23.603730+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:40:22.524069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.587165+0000","last_change":"2024-05-28T14:04:23.577017+0000","last_active":"2024-05-28T14:04:23.587165+0000","last_peered":"2024-05-28T14:04:23.587165+0000","last_clean":"2024-05-28T14:04:23.587165+0000","last_became_active":"2024-05-28T14:04:23.576634+0000","last_became_peered":"2024-05-28T14:04:23.576634+0000","last_unstale":"2024-05-28T14:04:23.587165+0000","last_undegraded":"2024-05-28T14:04:23.587165+0000","last_fullsized":"2024-05-28T14:04:23.587165+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:55.492821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590458+0000","last_change":"2024-05-28T14:04:22.566294+0000","last_active":"2024-05-28T14:04:24.590458+0000","last_peered":"2024-05-28T14:04:24.590458+0000","last_clean":"2024-05-28T14:04:24.590458+0000","last_became_active":"2024-05-28T14:04:22.566021+0000","last_became_peered":"2024-05-28T14:04:22.566021+0000","last_unstale":"2024-05-28T14:04:24.590458+0000","last_undegraded":"2024-05-28T14:04:24.590458+0000","last_fullsized":"2024-05-28T14:04:24.590458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:11.736974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.597058+0000","last_change":"2024-05-28T14:04:23.591906+0000","last_active":"2024-05-28T14:04:23.597058+0000","last_peered":"2024-05-28T14:04:23.597058+0000","last_clean":"2024-05-28T14:04:23.597058+0000","last_became_active":"2024-05-28T14:04:23.591600+0000","last_became_peered":"2024-05-28T14:04:23.591600+0000","last_unstale":"2024-05-28T14:04:23.597058+0000","last_undegraded":"2024-05-28T14:04:23.597058+0000","last_fullsized":"2024-05-28T14:04:23.597058+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:51:11.966539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.570759+0000","last_change":"2024-05-28T14:04:22.573550+0000","last_active":"2024-05-28T14:04:24.570759+0000","last_peered":"2024-05-28T14:04:24.570759+0000","last_clean":"2024-05-28T14:04:24.570759+0000","last_became_active":"2024-05-28T14:04:22.573237+0000","last_became_peered":"2024-05-28T14:04:22.573237+0000","last_unstale":"2024-05-28T14:04:24.570759+0000","last_undegraded":"2024-05-28T14:04:24.570759+0000","last_fullsized":"2024-05-28T14:04:24.570759+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:43:50.366314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588294+0000","last_change":"2024-05-28T14:04:23.586746+0000","last_active":"2024-05-28T14:04:23.588294+0000","last_peered":"2024-05-28T14:04:23.588294+0000","last_clean":"2024-05-28T14:04:23.588294+0000","last_became_active":"2024-05-28T14:04:23.585939+0000","last_became_peered":"2024-05-28T14:04:23.585939+0000","last_unstale":"2024-05-28T14:04:23.588294+0000","last_undegraded":"2024-05-28T14:04:23.588294+0000","last_fullsized":"2024-05-28T14:04:23.588294+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:36:31.735646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.565265+0000","last_change":"2024-05-28T14:04:22.567872+0000","last_active":"2024-05-28T14:04:23.565265+0000","last_peered":"2024-05-28T14:04:23.565265+0000","last_clean":"2024-05-28T14:04:23.565265+0000","last_became_active":"2024-05-28T14:04:22.567276+0000","last_became_peered":"2024-05-28T14:04:22.567276+0000","last_unstale":"2024-05-28T14:04:23.565265+0000","last_undegraded":"2024-05-28T14:04:23.565265+0000","last_fullsized":"2024-05-28T14:04:23.565265+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:11:46.843859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.590945+0000","last_change":"2024-05-28T14:04:23.584063+0000","last_active":"2024-05-28T14:04:23.590945+0000","last_peered":"2024-05-28T14:04:23.590945+0000","last_clean":"2024-05-28T14:04:23.590945+0000","last_became_active":"2024-05-28T14:04:23.583760+0000","last_became_peered":"2024-05-28T14:04:23.583760+0000","last_unstale":"2024-05-28T14:04:23.590945+0000","last_undegraded":"2024-05-28T14:04:23.590945+0000","last_fullsized":"2024-05-28T14:04:23.590945+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:12.744380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.574931+0000","last_change":"2024-05-28T14:04:22.577953+0000","last_active":"2024-05-28T14:04:23.574931+0000","last_peered":"2024-05-28T14:04:23.574931+0000","last_clean":"2024-05-28T14:04:23.574931+0000","last_became_active":"2024-05-28T14:04:22.577701+0000","last_became_peered":"2024-05-28T14:04:22.577701+0000","last_unstale":"2024-05-28T14:04:23.574931+0000","last_undegraded":"2024-05-28T14:04:23.574931+0000","last_fullsized":"2024-05-28T14:04:23.574931+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:24:52.150831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.592925+0000","last_change":"2024-05-28T14:04:23.589068+0000","last_active":"2024-05-28T14:04:23.592925+0000","last_peered":"2024-05-28T14:04:23.592925+0000","last_clean":"2024-05-28T14:04:23.592925+0000","last_became_active":"2024-05-28T14:04:23.588691+0000","last_became_peered":"2024-05-28T14:04:23.588691+0000","last_unstale":"2024-05-28T14:04:23.592925+0000","last_undegraded":"2024-05-28T14:04:23.592925+0000","last_fullsized":"2024-05-28T14:04:23.592925+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:59:03.398286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.562974+0000","last_change":"2024-05-28T14:04:22.572888+0000","last_active":"2024-05-28T14:04:23.562974+0000","last_peered":"2024-05-28T14:04:23.562974+0000","last_clean":"2024-05-28T14:04:23.562974+0000","last_became_active":"2024-05-28T14:04:22.572134+0000","last_became_peered":"2024-05-28T14:04:22.572134+0000","last_unstale":"2024-05-28T14:04:23.562974+0000","last_undegraded":"2024-05-28T14:04:23.562974+0000","last_fullsized":"2024-05-28T14:04:23.562974+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:20:18.952987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.582764+0000","last_change":"2024-05-28T14:04:23.580368+0000","last_active":"2024-05-28T14:04:23.582764+0000","last_peered":"2024-05-28T14:04:23.582764+0000","last_clean":"2024-05-28T14:04:23.582764+0000","last_became_active":"2024-05-28T14:04:23.578796+0000","last_became_peered":"2024-05-28T14:04:23.578796+0000","last_unstale":"2024-05-28T14:04:23.582764+0000","last_undegraded":"2024-05-28T14:04:23.582764+0000","last_fullsized":"2024-05-28T14:04:23.582764+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:55:40.974607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"60'3","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.604319+0000","last_change":"2024-05-28T14:04:22.573719+0000","last_active":"2024-05-28T14:04:23.604319+0000","last_peered":"2024-05-28T14:04:23.604319+0000","last_clean":"2024-05-28T14:04:23.604319+0000","last_became_active":"2024-05-28T14:04:22.573397+0000","last_became_peered":"2024-05-28T14:04:22.573397+0000","last_unstale":"2024-05-28T14:04:23.604319+0000","last_undegraded":"2024-05-28T14:04:23.604319+0000","last_fullsized":"2024-05-28T14:04:23.604319+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:13.052691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.593377+0000","last_change":"2024-05-28T14:04:23.588306+0000","last_active":"2024-05-28T14:04:23.593377+0000","last_peered":"2024-05-28T14:04:23.593377+0000","last_clean":"2024-05-28T14:04:23.593377+0000","last_became_active":"2024-05-28T14:04:23.587977+0000","last_became_peered":"2024-05-28T14:04:23.587977+0000","last_unstale":"2024-05-28T14:04:23.593377+0000","last_undegraded":"2024-05-28T14:04:23.593377+0000","last_fullsized":"2024-05-28T14:04:23.593377+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:22:49.692478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"60'6","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.607668+0000","last_change":"2024-05-28T14:04:22.564496+0000","last_active":"2024-05-28T14:04:23.607668+0000","last_peered":"2024-05-28T14:04:23.607668+0000","last_clean":"2024-05-28T14:04:23.607668+0000","last_became_active":"2024-05-28T14:04:22.564108+0000","last_became_peered":"2024-05-28T14:04:22.564108+0000","last_unstale":"2024-05-28T14:04:23.607668+0000","last_undegraded":"2024-05-28T14:04:23.607668+0000","last_fullsized":"2024-05-28T14:04:23.607668+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:42:43.825729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.570656+0000","last_change":"2024-05-28T14:04:23.585131+0000","last_active":"2024-05-28T14:04:24.570656+0000","last_peered":"2024-05-28T14:04:24.570656+0000","last_clean":"2024-05-28T14:04:24.570656+0000","last_became_active":"2024-05-28T14:04:23.576783+0000","last_became_peered":"2024-05-28T14:04:23.576783+0000","last_unstale":"2024-05-28T14:04:24.570656+0000","last_undegraded":"2024-05-28T14:04:24.570656+0000","last_fullsized":"2024-05-28T14:04:24.570656+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:59:31.979919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.601989+0000","last_change":"2024-05-28T14:04:22.572445+0000","last_active":"2024-05-28T14:04:23.601989+0000","last_peered":"2024-05-28T14:04:23.601989+0000","last_clean":"2024-05-28T14:04:23.601989+0000","last_became_active":"2024-05-28T14:04:22.572128+0000","last_became_peered":"2024-05-28T14:04:22.572128+0000","last_unstale":"2024-05-28T14:04:23.601989+0000","last_undegraded":"2024-05-28T14:04:23.601989+0000","last_fullsized":"2024-05-28T14:04:23.601989+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:47:27.164625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.594549+0000","last_change":"2024-05-28T14:04:23.591567+0000","last_active":"2024-05-28T14:04:23.594549+0000","last_peered":"2024-05-28T14:04:23.594549+0000","last_clean":"2024-05-28T14:04:23.594549+0000","last_became_active":"2024-05-28T14:04:23.591297+0000","last_became_peered":"2024-05-28T14:04:23.591297+0000","last_unstale":"2024-05-28T14:04:23.594549+0000","last_undegraded":"2024-05-28T14:04:23.594549+0000","last_fullsized":"2024-05-28T14:04:23.594549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:51:16.768786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.574694+0000","last_change":"2024-05-28T14:04:22.571901+0000","last_active":"2024-05-28T14:04:23.574694+0000","last_peered":"2024-05-28T14:04:23.574694+0000","last_clean":"2024-05-28T14:04:23.574694+0000","last_became_active":"2024-05-28T14:04:22.571442+0000","last_became_peered":"2024-05-28T14:04:22.571442+0000","last_unstale":"2024-05-28T14:04:23.574694+0000","last_undegraded":"2024-05-28T14:04:23.574694+0000","last_fullsized":"2024-05-28T14:04:23.574694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:03:43.301958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.590858+0000","last_change":"2024-05-28T14:04:23.584036+0000","last_active":"2024-05-28T14:04:23.590858+0000","last_peered":"2024-05-28T14:04:23.590858+0000","last_clean":"2024-05-28T14:04:23.590858+0000","last_became_active":"2024-05-28T14:04:23.583720+0000","last_became_peered":"2024-05-28T14:04:23.583720+0000","last_unstale":"2024-05-28T14:04:23.590858+0000","last_undegraded":"2024-05-28T14:04:23.590858+0000","last_fullsized":"2024-05-28T14:04:23.590858+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:16:03.356703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571052+0000","last_change":"2024-05-28T14:04:22.572924+0000","last_active":"2024-05-28T14:04:24.571052+0000","last_peered":"2024-05-28T14:04:24.571052+0000","last_clean":"2024-05-28T14:04:24.571052+0000","last_became_active":"2024-05-28T14:04:22.572660+0000","last_became_peered":"2024-05-28T14:04:22.572660+0000","last_unstale":"2024-05-28T14:04:24.571052+0000","last_undegraded":"2024-05-28T14:04:24.571052+0000","last_fullsized":"2024-05-28T14:04:24.571052+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:24:52.150831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.596644+0000","last_change":"2024-05-28T14:04:23.593510+0000","last_active":"2024-05-28T14:04:23.596644+0000","last_peered":"2024-05-28T14:04:23.596644+0000","last_clean":"2024-05-28T14:04:23.596644+0000","last_became_active":"2024-05-28T14:04:23.593194+0000","last_became_peered":"2024-05-28T14:04:23.593194+0000","last_unstale":"2024-05-28T14:04:23.596644+0000","last_undegraded":"2024-05-28T14:04:23.596644+0000","last_fullsized":"2024-05-28T14:04:23.596644+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:38:06.191955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"60'8","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.606162+0000","last_change":"2024-05-28T14:04:22.575319+0000","last_active":"2024-05-28T14:04:23.606162+0000","last_peered":"2024-05-28T14:04:23.606162+0000","last_clean":"2024-05-28T14:04:23.606162+0000","last_became_active":"2024-05-28T14:04:22.575041+0000","last_became_peered":"2024-05-28T14:04:22.575041+0000","last_unstale":"2024-05-28T14:04:23.606162+0000","last_undegraded":"2024-05-28T14:04:23.606162+0000","last_fullsized":"2024-05-28T14:04:23.606162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:43:50.366314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.595556+0000","last_change":"2024-05-28T14:04:23.592647+0000","last_active":"2024-05-28T14:04:23.595556+0000","last_peered":"2024-05-28T14:04:23.595556+0000","last_clean":"2024-05-28T14:04:23.595556+0000","last_became_active":"2024-05-28T14:04:23.592303+0000","last_became_peered":"2024-05-28T14:04:23.592303+0000","last_unstale":"2024-05-28T14:04:23.595556+0000","last_undegraded":"2024-05-28T14:04:23.595556+0000","last_fullsized":"2024-05-28T14:04:23.595556+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:04:41.618441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.602343+0000","last_change":"2024-05-28T14:04:22.569944+0000","last_active":"2024-05-28T14:04:23.602343+0000","last_peered":"2024-05-28T14:04:23.602343+0000","last_clean":"2024-05-28T14:04:23.602343+0000","last_became_active":"2024-05-28T14:04:22.569475+0000","last_became_peered":"2024-05-28T14:04:22.569475+0000","last_unstale":"2024-05-28T14:04:23.602343+0000","last_undegraded":"2024-05-28T14:04:23.602343+0000","last_fullsized":"2024-05-28T14:04:23.602343+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:27:44.260902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.570980+0000","last_change":"2024-05-28T14:04:23.581095+0000","last_active":"2024-05-28T14:04:24.570980+0000","last_peered":"2024-05-28T14:04:24.570980+0000","last_clean":"2024-05-28T14:04:24.570980+0000","last_became_active":"2024-05-28T14:04:23.577030+0000","last_became_peered":"2024-05-28T14:04:23.577030+0000","last_unstale":"2024-05-28T14:04:24.570980+0000","last_undegraded":"2024-05-28T14:04:24.570980+0000","last_fullsized":"2024-05-28T14:04:24.570980+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:27:00.745499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590817+0000","last_change":"2024-05-28T14:04:22.574606+0000","last_active":"2024-05-28T14:04:24.590817+0000","last_peered":"2024-05-28T14:04:24.590817+0000","last_clean":"2024-05-28T14:04:24.590817+0000","last_became_active":"2024-05-28T14:04:22.574301+0000","last_became_peered":"2024-05-28T14:04:22.574301+0000","last_unstale":"2024-05-28T14:04:24.590817+0000","last_undegraded":"2024-05-28T14:04:24.590817+0000","last_fullsized":"2024-05-28T14:04:24.590817+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:49:37.723962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.576042+0000","last_change":"2024-05-28T14:04:23.570505+0000","last_active":"2024-05-28T14:04:23.576042+0000","last_peered":"2024-05-28T14:04:23.576042+0000","last_clean":"2024-05-28T14:04:23.576042+0000","last_became_active":"2024-05-28T14:04:23.570190+0000","last_became_peered":"2024-05-28T14:04:23.570190+0000","last_unstale":"2024-05-28T14:04:23.576042+0000","last_undegraded":"2024-05-28T14:04:23.576042+0000","last_fullsized":"2024-05-28T14:04:23.576042+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:51:52.497265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.570949+0000","last_change":"2024-05-28T14:04:22.572545+0000","last_active":"2024-05-28T14:04:24.570949+0000","last_peered":"2024-05-28T14:04:24.570949+0000","last_clean":"2024-05-28T14:04:24.570949+0000","last_became_active":"2024-05-28T14:04:22.572261+0000","last_became_peered":"2024-05-28T14:04:22.572261+0000","last_unstale":"2024-05-28T14:04:24.570949+0000","last_undegraded":"2024-05-28T14:04:24.570949+0000","last_fullsized":"2024-05-28T14:04:24.570949+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:17:46.877238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.593397+0000","last_change":"2024-05-28T14:04:23.588599+0000","last_active":"2024-05-28T14:04:23.593397+0000","last_peered":"2024-05-28T14:04:23.593397+0000","last_clean":"2024-05-28T14:04:23.593397+0000","last_became_active":"2024-05-28T14:04:23.588242+0000","last_became_peered":"2024-05-28T14:04:23.588242+0000","last_unstale":"2024-05-28T14:04:23.593397+0000","last_undegraded":"2024-05-28T14:04:23.593397+0000","last_fullsized":"2024-05-28T14:04:23.593397+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:50:44.894296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590825+0000","last_change":"2024-05-28T14:04:22.567302+0000","last_active":"2024-05-28T14:04:24.590825+0000","last_peered":"2024-05-28T14:04:24.590825+0000","last_clean":"2024-05-28T14:04:24.590825+0000","last_became_active":"2024-05-28T14:04:22.566844+0000","last_became_peered":"2024-05-28T14:04:22.566844+0000","last_unstale":"2024-05-28T14:04:24.590825+0000","last_undegraded":"2024-05-28T14:04:24.590825+0000","last_fullsized":"2024-05-28T14:04:24.590825+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:24:52.150831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571241+0000","last_change":"2024-05-28T14:04:23.581675+0000","last_active":"2024-05-28T14:04:24.571241+0000","last_peered":"2024-05-28T14:04:24.571241+0000","last_clean":"2024-05-28T14:04:24.571241+0000","last_became_active":"2024-05-28T14:04:23.577287+0000","last_became_peered":"2024-05-28T14:04:23.577287+0000","last_unstale":"2024-05-28T14:04:24.571241+0000","last_undegraded":"2024-05-28T14:04:24.571241+0000","last_fullsized":"2024-05-28T14:04:24.571241+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:51:52.497265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.600742+0000","last_change":"2024-05-28T14:04:22.563880+0000","last_active":"2024-05-28T14:04:23.600742+0000","last_peered":"2024-05-28T14:04:23.600742+0000","last_clean":"2024-05-28T14:04:23.600742+0000","last_became_active":"2024-05-28T14:04:22.563540+0000","last_became_peered":"2024-05-28T14:04:22.563540+0000","last_unstale":"2024-05-28T14:04:23.600742+0000","last_undegraded":"2024-05-28T14:04:23.600742+0000","last_fullsized":"2024-05-28T14:04:23.600742+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:50:53.497310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590671+0000","last_change":"2024-05-28T14:04:23.584691+0000","last_active":"2024-05-28T14:04:24.590671+0000","last_peered":"2024-05-28T14:04:24.590671+0000","last_clean":"2024-05-28T14:04:24.590671+0000","last_became_active":"2024-05-28T14:04:23.584375+0000","last_became_peered":"2024-05-28T14:04:23.584375+0000","last_unstale":"2024-05-28T14:04:24.590671+0000","last_undegraded":"2024-05-28T14:04:24.590671+0000","last_fullsized":"2024-05-28T14:04:24.590671+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:50:54.504716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.569839+0000","last_change":"2024-05-28T14:04:22.573373+0000","last_active":"2024-05-28T14:04:23.569839+0000","last_peered":"2024-05-28T14:04:23.569839+0000","last_clean":"2024-05-28T14:04:23.569839+0000","last_became_active":"2024-05-28T14:04:22.573101+0000","last_became_peered":"2024-05-28T14:04:22.573101+0000","last_unstale":"2024-05-28T14:04:23.569839+0000","last_undegraded":"2024-05-28T14:04:23.569839+0000","last_fullsized":"2024-05-28T14:04:23.569839+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:53:13.328390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590434+0000","last_change":"2024-05-28T14:04:23.578247+0000","last_active":"2024-05-28T14:04:24.590434+0000","last_peered":"2024-05-28T14:04:24.590434+0000","last_clean":"2024-05-28T14:04:24.590434+0000","last_became_active":"2024-05-28T14:04:23.577888+0000","last_became_peered":"2024-05-28T14:04:23.577888+0000","last_unstale":"2024-05-28T14:04:24.590434+0000","last_undegraded":"2024-05-28T14:04:24.590434+0000","last_fullsized":"2024-05-28T14:04:24.590434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:27:00.745499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.570554+0000","last_change":"2024-05-28T14:04:22.570054+0000","last_active":"2024-05-28T14:04:24.570554+0000","last_peered":"2024-05-28T14:04:24.570554+0000","last_clean":"2024-05-28T14:04:24.570554+0000","last_became_active":"2024-05-28T14:04:22.569689+0000","last_became_peered":"2024-05-28T14:04:22.569689+0000","last_unstale":"2024-05-28T14:04:24.570554+0000","last_undegraded":"2024-05-28T14:04:24.570554+0000","last_fullsized":"2024-05-28T14:04:24.570554+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:13.052691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571048+0000","last_change":"2024-05-28T14:04:23.582490+0000","last_active":"2024-05-28T14:04:24.571048+0000","last_peered":"2024-05-28T14:04:24.571048+0000","last_clean":"2024-05-28T14:04:24.571048+0000","last_became_active":"2024-05-28T14:04:23.582082+0000","last_became_peered":"2024-05-28T14:04:23.582082+0000","last_unstale":"2024-05-28T14:04:24.571048+0000","last_undegraded":"2024-05-28T14:04:24.571048+0000","last_fullsized":"2024-05-28T14:04:24.571048+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:47:28.172031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.604711+0000","last_change":"2024-05-28T14:04:22.573572+0000","last_active":"2024-05-28T14:04:23.604711+0000","last_peered":"2024-05-28T14:04:23.604711+0000","last_clean":"2024-05-28T14:04:23.604711+0000","last_became_active":"2024-05-28T14:04:22.573350+0000","last_became_peered":"2024-05-28T14:04:22.573350+0000","last_unstale":"2024-05-28T14:04:23.604711+0000","last_undegraded":"2024-05-28T14:04:23.604711+0000","last_fullsized":"2024-05-28T14:04:23.604711+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:15:11.019258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571027+0000","last_change":"2024-05-28T14:04:23.577025+0000","last_active":"2024-05-28T14:04:24.571027+0000","last_peered":"2024-05-28T14:04:24.571027+0000","last_clean":"2024-05-28T14:04:24.571027+0000","last_became_active":"2024-05-28T14:04:23.576475+0000","last_became_peered":"2024-05-28T14:04:23.576475+0000","last_unstale":"2024-05-28T14:04:24.571027+0000","last_undegraded":"2024-05-28T14:04:24.571027+0000","last_fullsized":"2024-05-28T14:04:24.571027+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:20:19.960393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"61'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.592607+0000","last_change":"2024-05-28T14:04:22.566851+0000","last_active":"2024-05-28T14:04:24.592607+0000","last_peered":"2024-05-28T14:04:24.592607+0000","last_clean":"2024-05-28T14:04:24.592607+0000","last_became_active":"2024-05-28T14:04:22.566574+0000","last_became_peered":"2024-05-28T14:04:22.566574+0000","last_unstale":"2024-05-28T14:04:24.592607+0000","last_undegraded":"2024-05-28T14:04:24.592607+0000","last_fullsized":"2024-05-28T14:04:24.592607+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:43:50.366314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.582740+0000","last_change":"2024-05-28T14:04:23.578787+0000","last_active":"2024-05-28T14:04:23.582740+0000","last_peered":"2024-05-28T14:04:23.582740+0000","last_clean":"2024-05-28T14:04:23.582740+0000","last_became_active":"2024-05-28T14:04:23.578399+0000","last_became_peered":"2024-05-28T14:04:23.578399+0000","last_unstale":"2024-05-28T14:04:23.582740+0000","last_undegraded":"2024-05-28T14:04:23.582740+0000","last_fullsized":"2024-05-28T14:04:23.582740+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:56:01.585123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.574904+0000","last_change":"2024-05-28T14:04:22.565649+0000","last_active":"2024-05-28T14:04:23.574904+0000","last_peered":"2024-05-28T14:04:23.574904+0000","last_clean":"2024-05-28T14:04:23.574904+0000","last_became_active":"2024-05-28T14:04:22.565328+0000","last_became_peered":"2024-05-28T14:04:22.565328+0000","last_unstale":"2024-05-28T14:04:23.574904+0000","last_undegraded":"2024-05-28T14:04:23.574904+0000","last_fullsized":"2024-05-28T14:04:23.574904+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:30:05.374724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.590907+0000","last_change":"2024-05-28T14:04:23.581565+0000","last_active":"2024-05-28T14:04:24.590907+0000","last_peered":"2024-05-28T14:04:24.590907+0000","last_clean":"2024-05-28T14:04:24.590907+0000","last_became_active":"2024-05-28T14:04:23.581289+0000","last_became_peered":"2024-05-28T14:04:23.581289+0000","last_unstale":"2024-05-28T14:04:24.590907+0000","last_undegraded":"2024-05-28T14:04:24.590907+0000","last_fullsized":"2024-05-28T14:04:24.590907+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:55.492821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-28T14:04:24.571275+0000","last_change":"2024-05-28T14:04:22.566709+0000","last_active":"2024-05-28T14:04:24.571275+0000","last_peered":"2024-05-28T14:04:24.571275+0000","last_clean":"2024-05-28T14:04:24.571275+0000","last_became_active":"2024-05-28T14:04:22.566268+0000","last_became_peered":"2024-05-28T14:04:22.566268+0000","last_unstale":"2024-05-28T14:04:24.571275+0000","last_undegraded":"2024-05-28T14:04:24.571275+0000","last_fullsized":"2024-05-28T14:04:24.571275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:59:04.845137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.587256+0000","last_change":"2024-05-28T14:04:23.577401+0000","last_active":"2024-05-28T14:04:23.587256+0000","last_peered":"2024-05-28T14:04:23.587256+0000","last_clean":"2024-05-28T14:04:23.587256+0000","last_became_active":"2024-05-28T14:04:23.576327+0000","last_became_peered":"2024-05-28T14:04:23.576327+0000","last_unstale":"2024-05-28T14:04:23.587256+0000","last_undegraded":"2024-05-28T14:04:23.587256+0000","last_fullsized":"2024-05-28T14:04:23.587256+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:06:07.558092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.569723+0000","last_change":"2024-05-28T14:04:22.568496+0000","last_active":"2024-05-28T14:04:23.569723+0000","last_peered":"2024-05-28T14:04:23.569723+0000","last_clean":"2024-05-28T14:04:23.569723+0000","last_became_active":"2024-05-28T14:04:22.568188+0000","last_became_peered":"2024-05-28T14:04:22.568188+0000","last_unstale":"2024-05-28T14:04:23.569723+0000","last_undegraded":"2024-05-28T14:04:23.569723+0000","last_fullsized":"2024-05-28T14:04:23.569723+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:32:30.371408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.596641+0000","last_change":"2024-05-28T14:04:23.589242+0000","last_active":"2024-05-28T14:04:23.596641+0000","last_peered":"2024-05-28T14:04:23.596641+0000","last_clean":"2024-05-28T14:04:23.596641+0000","last_became_active":"2024-05-28T14:04:23.588848+0000","last_became_peered":"2024-05-28T14:04:23.588848+0000","last_unstale":"2024-05-28T14:04:23.596641+0000","last_undegraded":"2024-05-28T14:04:23.596641+0000","last_fullsized":"2024-05-28T14:04:23.596641+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:59:31.979919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.567117+0000","last_change":"2024-05-28T14:04:22.570228+0000","last_active":"2024-05-28T14:04:23.567117+0000","last_peered":"2024-05-28T14:04:23.567117+0000","last_clean":"2024-05-28T14:04:23.567117+0000","last_became_active":"2024-05-28T14:04:22.569964+0000","last_became_peered":"2024-05-28T14:04:22.569964+0000","last_unstale":"2024-05-28T14:04:23.567117+0000","last_undegraded":"2024-05-28T14:04:23.567117+0000","last_fullsized":"2024-05-28T14:04:23.567117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:08:45.861158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.594873+0000","last_change":"2024-05-28T14:04:23.582926+0000","last_active":"2024-05-28T14:04:23.594873+0000","last_peered":"2024-05-28T14:04:23.594873+0000","last_clean":"2024-05-28T14:04:23.594873+0000","last_became_active":"2024-05-28T14:04:23.582625+0000","last_became_peered":"2024-05-28T14:04:23.582625+0000","last_unstale":"2024-05-28T14:04:23.594873+0000","last_undegraded":"2024-05-28T14:04:23.594873+0000","last_fullsized":"2024-05-28T14:04:23.594873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:30:06.382130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.571330+0000","last_change":"2024-05-28T14:04:22.565568+0000","last_active":"2024-05-28T14:04:23.571330+0000","last_peered":"2024-05-28T14:04:23.571330+0000","last_clean":"2024-05-28T14:04:23.571330+0000","last_became_active":"2024-05-28T14:04:22.565262+0000","last_became_peered":"2024-05-28T14:04:22.565262+0000","last_unstale":"2024-05-28T14:04:23.571330+0000","last_undegraded":"2024-05-28T14:04:23.571330+0000","last_fullsized":"2024-05-28T14:04:23.571330+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:47:13.344736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.588315+0000","last_change":"2024-05-28T14:04:23.586885+0000","last_active":"2024-05-28T14:04:23.588315+0000","last_peered":"2024-05-28T14:04:23.588315+0000","last_clean":"2024-05-28T14:04:23.588315+0000","last_became_active":"2024-05-28T14:04:23.586437+0000","last_became_peered":"2024-05-28T14:04:23.586437+0000","last_unstale":"2024-05-28T14:04:23.588315+0000","last_undegraded":"2024-05-28T14:04:23.588315+0000","last_fullsized":"2024-05-28T14:04:23.588315+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:02:55.459572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.562710+0000","last_change":"2024-05-28T14:04:22.564493+0000","last_active":"2024-05-28T14:04:23.562710+0000","last_peered":"2024-05-28T14:04:23.562710+0000","last_clean":"2024-05-28T14:04:23.562710+0000","last_became_active":"2024-05-28T14:04:22.564185+0000","last_became_peered":"2024-05-28T14:04:22.564185+0000","last_unstale":"2024-05-28T14:04:23.562710+0000","last_undegraded":"2024-05-28T14:04:23.562710+0000","last_fullsized":"2024-05-28T14:04:23.562710+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:38:05.184549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.587629+0000","last_change":"2024-05-28T14:04:23.577031+0000","last_active":"2024-05-28T14:04:23.587629+0000","last_peered":"2024-05-28T14:04:23.587629+0000","last_clean":"2024-05-28T14:04:23.587629+0000","last_became_active":"2024-05-28T14:04:23.576752+0000","last_became_peered":"2024-05-28T14:04:23.576752+0000","last_unstale":"2024-05-28T14:04:23.587629+0000","last_undegraded":"2024-05-28T14:04:23.587629+0000","last_fullsized":"2024-05-28T14:04:23.587629+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:22.543350+0000","last_clean_scrub_stamp":"2024-05-28T14:04:22.543350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:10:22.343727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.576593+0000","last_change":"2024-05-28T14:04:22.574129+0000","last_active":"2024-05-28T14:04:23.576593+0000","last_peered":"2024-05-28T14:04:23.576593+0000","last_clean":"2024-05-28T14:04:23.576593+0000","last_became_active":"2024-05-28T14:04:22.572893+0000","last_became_peered":"2024-05-28T14:04:22.572893+0000","last_unstale":"2024-05-28T14:04:23.576593+0000","last_undegraded":"2024-05-28T14:04:23.576593+0000","last_fullsized":"2024-05-28T14:04:23.576593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:04:21.535944+0000","last_clean_scrub_stamp":"2024-05-28T14:04:21.535944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:56:34.906444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":243,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-28T14:04:23.570574+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:04:23.570574+0000","last_peered":"2024-05-28T14:04:23.570574+0000","last_clean":"2024-05-28T14:04:23.570574+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:04:23.570574+0000","last_undegraded":"2024-05-28T14:04:23.570574+0000","last_fullsized":"2024-05-28T14:04:23.570574+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":1323008,"data_stored":892974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45780,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326060,"statfs":{"total":96636764160,"available":96589885440,"internally_reserved":0,"allocated":720896,"data_stored":302446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40220,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331620,"statfs":{"total":96636764160,"available":96595578880,"internally_reserved":0,"allocated":729088,"data_stored":303098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45792,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326048,"statfs":{"total":96636764160,"available":96589873152,"internally_reserved":0,"allocated":733184,"data_stored":304058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49948,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321892,"statfs":{"total":96636764160,"available":96585617408,"internally_reserved":0,"allocated":729088,"data_stored":302502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45780,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326060,"statfs":{"total":96636764160,"available":96589885440,"internally_reserved":0,"allocated":729088,"data_stored":302502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49936,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321904,"statfs":{"total":96636764160,"available":96585629696,"internally_reserved":0,"allocated":724992,"data_stored":302536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50532,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321308,"statfs":{"total":96636764160,"available":96585019392,"internally_reserved":0,"allocated":1331200,"data_stored":895008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:04:28.088 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-28T14:04:28.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-28T14:04:28.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.291+0000 7fc2ac200640 1 Processor -- start 2024-05-28T14:04:28.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.291+0000 7fc2ac200640 1 -- start start 2024-05-28T14:04:28.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2ac200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2a409b160 0x7fc2a406cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:28.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2ac200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc2a409b8e0 0x7fc2a415e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2ac200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2a40d0770 con 0x7fc2a40b0e80 2024-05-28T14:04:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2ac200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc2a40d0a10 con 0x7fc2a409b8e0 2024-05-28T14:04:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2ac200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a40d0cf0 con 0x7fc2a409b160 2024-05-28T14:04:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2a9800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc2a40b0e80 0x7fc2a409aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51254/0 (socket says 172.21.15.42:51254) 2024-05-28T14:04:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2a9800640 1 -- 172.21.15.42:0/3434505733 learned_addr learned my addr 172.21.15.42:0/3434505733 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2aa200640 1 --2- 172.21.15.42:0/3434505733 >> v2:172.21.15.63:3300/0 conn(0x7fc2a409b8e0 0x7fc2a415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2a8e00640 1 --2- 172.21.15.42:0/3434505733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2a409b160 0x7fc2a406cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2a3e00640 1 -- 172.21.15.42:0/3434505733 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3435765478 0 0) 0x7fc2a40d0770 con 0x7fc2a40b0e80 2024-05-28T14:04:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2a3e00640 1 -- 172.21.15.42:0/3434505733 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc28c003530 con 0x7fc2a40b0e80 2024-05-28T14:04:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2aa200640 1 -- 172.21.15.42:0/3434505733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2a409b160 msgr2=0x7fc2a406cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2aa200640 1 --2- 172.21.15.42:0/3434505733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2a409b160 0x7fc2a406cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2aa200640 1 -- 172.21.15.42:0/3434505733 >> v1:172.21.15.42:6789/0 conn(0x7fc2a40b0e80 legacy=0x7fc2a409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:28.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2aa200640 1 -- 172.21.15.42:0/3434505733 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a40d0fd0 con 0x7fc2a409b8e0 2024-05-28T14:04:28.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.295+0000 7fc2aa200640 1 --2- 172.21.15.42:0/3434505733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a409b8e0 0x7fc2a415e110 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009d90 tx=0x7fc29c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79ef21409d27f40 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:28.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2a3e00640 1 -- 172.21.15.42:0/3434505733 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0b2070 con 0x7fc2a409b8e0 2024-05-28T14:04:28.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2a3e00640 1 -- 172.21.15.42:0/3434505733 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc29c0a5830 con 0x7fc2a409b8e0 2024-05-28T14:04:28.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2a3e00640 1 -- 172.21.15.42:0/3434505733 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0a5c00 con 0x7fc2a409b8e0 2024-05-28T14:04:28.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 -- 172.21.15.42:0/3434505733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a409b8e0 msgr2=0x7fc2a415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:28.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 --2- 172.21.15.42:0/3434505733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a409b8e0 0x7fc2a415e110 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009d90 tx=0x7fc29c01c970 comp rx=0 tx=0).stop 2024-05-28T14:04:28.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 -- 172.21.15.42:0/3434505733 shutdown_connections 2024-05-28T14:04:28.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 --2- 172.21.15.42:0/3434505733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a409b8e0 0x7fc2a415e110 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 --2- 172.21.15.42:0/3434505733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2a409b160 0x7fc2a406cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 -- 172.21.15.42:0/3434505733 >> 172.21.15.42:0/3434505733 conn(0x7fc2a40b3f90 msgr2=0x7fc2a40b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:28.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 -- 172.21.15.42:0/3434505733 shutdown_connections 2024-05-28T14:04:28.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 -- 172.21.15.42:0/3434505733 wait complete. 2024-05-28T14:04:28.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 Processor -- start 2024-05-28T14:04:28.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 -- start start 2024-05-28T14:04:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2a409b160 0x7fc2a41542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a409b8e0 0x7fc2a415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2a40b0e80 0x7fc2a415f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a40af350 con 0x7fc2a409b160 2024-05-28T14:04:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a40af1d0 con 0x7fc2a409b8e0 2024-05-28T14:04:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2ac200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a40af4d0 con 0x7fc2a40b0e80 2024-05-28T14:04:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2a9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2a409b160 0x7fc2a41542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2a9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2a409b160 0x7fc2a41542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60882/0 (socket says 172.21.15.42:60882) 2024-05-28T14:04:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.299+0000 7fc2a9800640 1 -- 172.21.15.42:0/2975723538 learned_addr learned my addr 172.21.15.42:0/2975723538 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:28.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2aa200640 1 --2- 172.21.15.42:0/2975723538 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2a40b0e80 0x7fc2a415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:28.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2a8e00640 1 --2- 172.21.15.42:0/2975723538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a409b8e0 0x7fc2a415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:28.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2a9800640 1 -- 172.21.15.42:0/2975723538 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2a40b0e80 msgr2=0x7fc2a415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:28.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2a9800640 1 --2- 172.21.15.42:0/2975723538 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2a40b0e80 0x7fc2a415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2a9800640 1 -- 172.21.15.42:0/2975723538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a409b8e0 msgr2=0x7fc2a415ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:28.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2a9800640 1 --2- 172.21.15.42:0/2975723538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a409b8e0 0x7fc2a415ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2a9800640 1 -- 172.21.15.42:0/2975723538 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a40b5090 con 0x7fc2a409b160 2024-05-28T14:04:28.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2a9800640 1 --2- 172.21.15.42:0/2975723538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2a409b160 0x7fc2a41542d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc29800c3c0 tx=0x7fc29800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:28.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2a1600640 1 -- 172.21.15.42:0/2975723538 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298017070 con 0x7fc2a409b160 2024-05-28T14:04:28.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2a1600640 1 -- 172.21.15.42:0/2975723538 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc298005cf0 con 0x7fc2a409b160 2024-05-28T14:04:28.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2a1600640 1 -- 172.21.15.42:0/2975723538 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2980072f0 con 0x7fc2a409b160 2024-05-28T14:04:28.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2ac200640 1 -- 172.21.15.42:0/2975723538 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a40b5370 con 0x7fc2a409b160 2024-05-28T14:04:28.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2ac200640 1 -- 172.21.15.42:0/2975723538 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a40b5930 con 0x7fc2a409b160 2024-05-28T14:04:28.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2ac200640 1 -- 172.21.15.42:0/2975723538 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a40a0550 con 0x7fc2a409b160 2024-05-28T14:04:28.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2a1600640 1 -- 172.21.15.42:0/2975723538 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fc29801a070 con 0x7fc2a409b160 2024-05-28T14:04:28.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2a1600640 1 --2- 172.21.15.42:0/2975723538 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc284068b60 0x7fc28406b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:28.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2a1600640 1 -- 172.21.15.42:0/2975723538 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fc2980847a0 con 0x7fc2a409b160 2024-05-28T14:04:28.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.303+0000 7fc2a8e00640 1 --2- 172.21.15.42:0/2975723538 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc284068b60 0x7fc28406b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:28.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.307+0000 7fc2a8e00640 1 --2- 172.21.15.42:0/2975723538 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc284068b60 0x7fc28406b020 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc2a406c030 tx=0x7fc290008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:28.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.307+0000 7fc2a1600640 1 -- 172.21.15.42:0/2975723538 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc298056650 con 0x7fc2a409b160 2024-05-28T14:04:28.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.531+0000 7fc2ac200640 1 -- 172.21.15.42:0/2975723538 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc2a404fdd0 con 0x7fc2a409b160 2024-05-28T14:04:28.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.535+0000 7fc2a1600640 1 -- 172.21.15.42:0/2975723538 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7fc298055f50 con 0x7fc2a409b160 2024-05-28T14:04:28.521 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-05-28T14:04:28.521 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw== 2024-05-28T14:04:28.522 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow" 2024-05-28T14:04:28.522 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-05-28T14:04:28.522 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-28T14:04:28.522 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.0 2024-05-28T14:04:28.522 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-28T14:04:28.540+0000 7f0797c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-28T14:04:28.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.535+0000 7fc2ac200640 1 -- 172.21.15.42:0/2975723538 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc284068b60 msgr2=0x7fc28406b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:28.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.535+0000 7fc2ac200640 1 --2- 172.21.15.42:0/2975723538 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc284068b60 0x7fc28406b020 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc2a406c030 tx=0x7fc290008040 comp rx=0 tx=0).stop 2024-05-28T14:04:28.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.535+0000 7fc2ac200640 1 -- 172.21.15.42:0/2975723538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2a409b160 msgr2=0x7fc2a41542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:28.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.535+0000 7fc2ac200640 1 --2- 172.21.15.42:0/2975723538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2a409b160 0x7fc2a41542d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc29800c3c0 tx=0x7fc29800c880 comp rx=0 tx=0).stop 2024-05-28T14:04:28.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.535+0000 7fc2ac200640 1 -- 172.21.15.42:0/2975723538 shutdown_connections 2024-05-28T14:04:28.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.535+0000 7fc2ac200640 1 --2- 172.21.15.42:0/2975723538 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc284068b60 0x7fc28406b020 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.535+0000 7fc2ac200640 1 --2- 172.21.15.42:0/2975723538 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2a40b0e80 0x7fc2a415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.535+0000 7fc2ac200640 1 --2- 172.21.15.42:0/2975723538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2a409b8e0 0x7fc2a415ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.535+0000 7fc2ac200640 1 --2- 172.21.15.42:0/2975723538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2a409b160 0x7fc2a41542d0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.539+0000 7fc2ac200640 1 -- 172.21.15.42:0/2975723538 >> 172.21.15.42:0/2975723538 conn(0x7fc2a40b3f90 msgr2=0x7fc2a406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:28.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.539+0000 7fc2ac200640 1 -- 172.21.15.42:0/2975723538 shutdown_connections 2024-05-28T14:04:28.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.539+0000 7fc2ac200640 1 -- 172.21.15.42:0/2975723538 wait complete. 2024-05-28T14:04:28.543 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:04:28.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4685c00640 1 Processor -- start 2024-05-28T14:04:28.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4685c00640 1 -- start start 2024-05-28T14:04:28.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4685c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f468015c990 0x7f46800d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:28.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4685c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f468009bbd0 0x7f468009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:28.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4685c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46800d06a0 0x7f468009af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:28.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4685c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f468015cd70 con 0x7f468009bbd0 2024-05-28T14:04:28.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4685c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f46800cd300 con 0x7f468015c990 2024-05-28T14:04:28.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4685c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46800cd5e0 con 0x7f46800d06a0 2024-05-28T14:04:28.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4677e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f468009bbd0 0x7f468009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:28.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4677e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f468009bbd0 0x7f468009aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60896/0 (socket says 172.21.15.42:60896) 2024-05-28T14:04:28.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4677e00640 1 -- 172.21.15.42:0/2899852862 learned_addr learned my addr 172.21.15.42:0/2899852862 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:28.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f467ea00640 1 --2- 172.21.15.42:0/2899852862 >> v2:172.21.15.63:3300/0 conn(0x7f468015c990 0x7f46800d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:28.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f467f400640 1 --2- 172.21.15.42:0/2899852862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46800d06a0 0x7f468009af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:28.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4677e00640 1 -- 172.21.15.42:0/2899852862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46800d06a0 msgr2=0x7f468009af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:28.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4677e00640 1 --2- 172.21.15.42:0/2899852862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46800d06a0 0x7f468009af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4677e00640 1 -- 172.21.15.42:0/2899852862 >> v2:172.21.15.63:3300/0 conn(0x7f468015c990 msgr2=0x7f46800d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:28.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4677e00640 1 --2- 172.21.15.42:0/2899852862 >> v2:172.21.15.63:3300/0 conn(0x7f468015c990 0x7f46800d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f4677e00640 1 -- 172.21.15.42:0/2899852862 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46800cd8c0 con 0x7f468009bbd0 2024-05-28T14:04:28.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.699+0000 7f467f400640 1 --2- 172.21.15.42:0/2899852862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46800d06a0 0x7f468009af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:28.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.703+0000 7f4677e00640 1 --2- 172.21.15.42:0/2899852862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f468009bbd0 0x7f468009aa50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4680073350 tx=0x7f466801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2437ab96f50d81b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:28.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.703+0000 7f467e000640 1 -- 172.21.15.42:0/2899852862 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46680aa610 con 0x7f468009bbd0 2024-05-28T14:04:28.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.703+0000 7f467e000640 1 -- 172.21.15.42:0/2899852862 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46680af070 con 0x7f468009bbd0 2024-05-28T14:04:28.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.703+0000 7f467e000640 1 -- 172.21.15.42:0/2899852862 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46680aa9d0 con 0x7f468009bbd0 2024-05-28T14:04:28.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.703+0000 7f4685c00640 1 -- 172.21.15.42:0/2899852862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f468009bbd0 msgr2=0x7f468009aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:28.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.703+0000 7f4685c00640 1 --2- 172.21.15.42:0/2899852862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f468009bbd0 0x7f468009aa50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4680073350 tx=0x7f466801c4f0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.703+0000 7f4685c00640 1 -- 172.21.15.42:0/2899852862 shutdown_connections 2024-05-28T14:04:28.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.703+0000 7f4685c00640 1 --2- 172.21.15.42:0/2899852862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46800d06a0 0x7f468009af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.703+0000 7f4685c00640 1 --2- 172.21.15.42:0/2899852862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f468009bbd0 0x7f468009aa50 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.703+0000 7f4685c00640 1 --2- 172.21.15.42:0/2899852862 >> v2:172.21.15.63:3300/0 conn(0x7f468015c990 0x7f46800d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.703+0000 7f4685c00640 1 -- 172.21.15.42:0/2899852862 >> 172.21.15.42:0/2899852862 conn(0x7f46800b4080 msgr2=0x7f46800b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:28.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.703+0000 7f4685c00640 1 -- 172.21.15.42:0/2899852862 shutdown_connections 2024-05-28T14:04:28.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.703+0000 7f4685c00640 1 -- 172.21.15.42:0/2899852862 wait complete. 2024-05-28T14:04:28.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4685c00640 1 Processor -- start 2024-05-28T14:04:28.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4685c00640 1 -- start start 2024-05-28T14:04:28.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4685c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46800d06a0 0x7f46800cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:28.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4685c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f468015c990 0x7f468015af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:28.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4685c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f468015b4c0 0x7f46801584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:28.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4685c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4680160d90 con 0x7f468015b4c0 2024-05-28T14:04:28.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4685c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4680160c10 con 0x7f46800d06a0 2024-05-28T14:04:28.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4685c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4680160f10 con 0x7f468015c990 2024-05-28T14:04:28.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f467f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f468015b4c0 0x7f46801584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:28.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f467f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f468015b4c0 0x7f46801584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60900/0 (socket says 172.21.15.42:60900) 2024-05-28T14:04:28.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f467f400640 1 -- 172.21.15.42:0/3432728096 learned_addr learned my addr 172.21.15.42:0/3432728096 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:28.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f467ea00640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46800d06a0 0x7f46800cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:28.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4677e00640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f468015c990 0x7f468015af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:28.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f467f400640 1 -- 172.21.15.42:0/3432728096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f468015c990 msgr2=0x7f468015af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f467f400640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f468015c990 0x7f468015af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f467f400640 1 -- 172.21.15.42:0/3432728096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46800d06a0 msgr2=0x7f46800cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f467f400640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46800d06a0 0x7f46800cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f467f400640 1 -- 172.21.15.42:0/3432728096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46800ca170 con 0x7f468015b4c0 2024-05-28T14:04:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4677e00640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f468015c990 0x7f468015af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f467f400640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f468015b4c0 0x7f46801584c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4678007df0 tx=0x7f467800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4676a00640 1 -- 172.21.15.42:0/3432728096 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467801a070 con 0x7f468015b4c0 2024-05-28T14:04:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4676a00640 1 -- 172.21.15.42:0/3432728096 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4678015400 con 0x7f468015b4c0 2024-05-28T14:04:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4676a00640 1 -- 172.21.15.42:0/3432728096 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46780157f0 con 0x7f468015b4c0 2024-05-28T14:04:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4685c00640 1 -- 172.21.15.42:0/3432728096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46800ca450 con 0x7f468015b4c0 2024-05-28T14:04:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.707+0000 7f4685c00640 1 -- 172.21.15.42:0/3432728096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46800caa10 con 0x7f468015b4c0 2024-05-28T14:04:28.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.711+0000 7f4685c00640 1 -- 172.21.15.42:0/3432728096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4680068a90 con 0x7f468015b4c0 2024-05-28T14:04:28.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.711+0000 7f4676a00640 1 -- 172.21.15.42:0/3432728096 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f467801d070 con 0x7f468015b4c0 2024-05-28T14:04:28.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.711+0000 7f4676a00640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f465c068b60 0x7f465c06b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:28.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.711+0000 7f4676a00640 1 -- 172.21.15.42:0/3432728096 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f4678087f90 con 0x7f468015b4c0 2024-05-28T14:04:28.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.711+0000 7f467ea00640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f465c068b60 0x7f465c06b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:28.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.711+0000 7f467ea00640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f465c068b60 0x7f465c06b020 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4670009ac0 tx=0x7f4670009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:28.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.715+0000 7f4676a00640 1 -- 172.21.15.42:0/3432728096 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f467808b030 con 0x7f468015b4c0 2024-05-28T14:04:28.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.915+0000 7f4685c00640 1 -- 172.21.15.42:0/3432728096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46800b4ce0 con 0x7f468015b4c0 2024-05-28T14:04:28.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.915+0000 7f4676a00640 1 -- 172.21.15.42:0/3432728096 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5178 (secure 0 0 0) 0x7f4678010070 con 0x7f468015b4c0 2024-05-28T14:04:28.902 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:28.902 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1310016671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1310016671},{"type":"v1","addr":"172.21.15.63:6834","nonce":1310016671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1602237641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1602237641},{"type":"v1","addr":"172.21.15.42:6835","nonce":1602237641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/142807607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":142807607},{"type":"v1","addr":"172.21.15.42:6837","nonce":142807607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:04:22.960438+0000","modified":"2024-05-28T14:04:25.983153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3311861302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3311861302},{"type":"v1","addr":"172.21.15.63:6835","nonce":3311861302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6420,"qdb_cluster":[6420]},"id":4}]} 2024-05-28T14:04:28.902 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 36 2024-05-28T14:04:28.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.919+0000 7f4685c00640 1 -- 172.21.15.42:0/3432728096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f465c068b60 msgr2=0x7f465c06b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:28.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.919+0000 7f4685c00640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f465c068b60 0x7f465c06b020 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4670009ac0 tx=0x7f4670009190 comp rx=0 tx=0).stop 2024-05-28T14:04:28.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.919+0000 7f4685c00640 1 -- 172.21.15.42:0/3432728096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f468015b4c0 msgr2=0x7f46801584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:28.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.919+0000 7f4685c00640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f468015b4c0 0x7f46801584c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4678007df0 tx=0x7f467800d4a0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.919+0000 7f4685c00640 1 -- 172.21.15.42:0/3432728096 shutdown_connections 2024-05-28T14:04:28.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.919+0000 7f4685c00640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f465c068b60 0x7f465c06b020 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.919+0000 7f4685c00640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f468015b4c0 0x7f46801584c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.919+0000 7f4685c00640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f468015c990 0x7f468015af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.919+0000 7f4685c00640 1 --2- 172.21.15.42:0/3432728096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46800d06a0 0x7f46800cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:28.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.919+0000 7f4685c00640 1 -- 172.21.15.42:0/3432728096 >> 172.21.15.42:0/3432728096 conn(0x7f46800b4080 msgr2=0x7f468009aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:28.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.919+0000 7f4685c00640 1 -- 172.21.15.42:0/3432728096 shutdown_connections 2024-05-28T14:04:28.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:28.919+0000 7f4685c00640 1 -- 172.21.15.42:0/3432728096 wait complete. 2024-05-28T14:04:29.325 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-28T14:04:22.960438+0000', 'modified': '2024-05-28T14:04:25.983153+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6420}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6420': {'gid': 6420, 'name': 'b', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6835/3311861302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6833', 'nonce': 3311861302}, {'type': 'v1', 'addr': '172.21.15.63:6835', 'nonce': 3311861302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6420, 'qdb_cluster': [6420]} 2024-05-28T14:04:29.326 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-28T14:04:29.326 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-28T14:04:29.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9d1a00640 1 Processor -- start 2024-05-28T14:04:29.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9d1a00640 1 -- start start 2024-05-28T14:04:29.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9d1a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff9cc0b4bc0 0x7ff9cc0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9d1a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9cc0ce380 0x7ff9cc0d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9d1a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9cc1622a0 con 0x7ff9cc09aa50 2024-05-28T14:04:29.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9d1a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff9cc0ccee0 con 0x7ff9cc0b4bc0 2024-05-28T14:04:29.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9d1a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9cc09b620 con 0x7ff9cc0ce380 2024-05-28T14:04:29.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9cb400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9cc0ce380 0x7ff9cc0d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9c2000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff9cc0b4bc0 0x7ff9cc0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9cb400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9cc0ce380 0x7ff9cc0d0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:37410/0 (socket says 172.21.15.42:37410) 2024-05-28T14:04:29.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9cb400640 1 -- 172.21.15.42:0/3824835476 learned_addr learned my addr 172.21.15.42:0/3824835476 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:29.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9cb400640 1 -- 172.21.15.42:0/3824835476 >> v2:172.21.15.63:3300/0 conn(0x7ff9cc0b4bc0 msgr2=0x7ff9cc0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:29.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9cb400640 1 --2- 172.21.15.42:0/3824835476 >> v2:172.21.15.63:3300/0 conn(0x7ff9cc0b4bc0 0x7ff9cc0d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9cb400640 1 -- 172.21.15.42:0/3824835476 >> v1:172.21.15.63:6789/0 conn(0x7ff9cc09aa50 legacy=0x7ff9cc09ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.487+0000 7ff9cb400640 1 -- 172.21.15.42:0/3824835476 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc09b900 con 0x7ff9cc0ce380 2024-05-28T14:04:29.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.491+0000 7ff9cb400640 1 --2- 172.21.15.42:0/3824835476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9cc0ce380 0x7ff9cc0d0af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009e50 tx=0x7ff9c401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5bdf8e81f4c68db server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:29.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.491+0000 7ff9ca000640 1 -- 172.21.15.42:0/3824835476 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40a58d0 con 0x7ff9cc0ce380 2024-05-28T14:04:29.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.491+0000 7ff9ca000640 1 -- 172.21.15.42:0/3824835476 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c40a5a70 con 0x7ff9cc0ce380 2024-05-28T14:04:29.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.491+0000 7ff9ca000640 1 -- 172.21.15.42:0/3824835476 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40a5e40 con 0x7ff9cc0ce380 2024-05-28T14:04:29.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.491+0000 7ff9d1a00640 1 -- 172.21.15.42:0/3824835476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9cc0ce380 msgr2=0x7ff9cc0d0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.491+0000 7ff9d1a00640 1 --2- 172.21.15.42:0/3824835476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9cc0ce380 0x7ff9cc0d0af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009e50 tx=0x7ff9c401ca00 comp rx=0 tx=0).stop 2024-05-28T14:04:29.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.491+0000 7ff9d1a00640 1 -- 172.21.15.42:0/3824835476 shutdown_connections 2024-05-28T14:04:29.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.491+0000 7ff9d1a00640 1 --2- 172.21.15.42:0/3824835476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9cc0ce380 0x7ff9cc0d0af0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.491+0000 7ff9d1a00640 1 --2- 172.21.15.42:0/3824835476 >> v2:172.21.15.63:3300/0 conn(0x7ff9cc0b4bc0 0x7ff9cc0d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.491+0000 7ff9d1a00640 1 -- 172.21.15.42:0/3824835476 >> 172.21.15.42:0/3824835476 conn(0x7ff9cc0b4010 msgr2=0x7ff9cc0b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:29.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.491+0000 7ff9d1a00640 1 -- 172.21.15.42:0/3824835476 shutdown_connections 2024-05-28T14:04:29.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.491+0000 7ff9d1a00640 1 -- 172.21.15.42:0/3824835476 wait complete. 2024-05-28T14:04:29.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9d1a00640 1 Processor -- start 2024-05-28T14:04:29.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9d1a00640 1 -- start start 2024-05-28T14:04:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9d1a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc09aa50 0x7ff9cc0cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9d1a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9cc0b4bc0 0x7ff9cc0b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9d1a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9cc0ce380 0x7ff9cc0b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9d1a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc0d1bd0 con 0x7ff9cc09aa50 2024-05-28T14:04:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9d1a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc0d1a50 con 0x7ff9cc0ce380 2024-05-28T14:04:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9d1a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9cc0d1d50 con 0x7ff9cc0b4bc0 2024-05-28T14:04:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc09aa50 0x7ff9cc0cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc09aa50 0x7ff9cc0cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60904/0 (socket says 172.21.15.42:60904) 2024-05-28T14:04:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9caa00640 1 -- 172.21.15.42:0/3675144294 learned_addr learned my addr 172.21.15.42:0/3675144294 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9c2000640 1 --2- 172.21.15.42:0/3675144294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9cc0b4bc0 0x7ff9cc0b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9cb400640 1 --2- 172.21.15.42:0/3675144294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9cc0ce380 0x7ff9cc0b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9caa00640 1 -- 172.21.15.42:0/3675144294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9cc0b4bc0 msgr2=0x7ff9cc0b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9caa00640 1 --2- 172.21.15.42:0/3675144294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9cc0b4bc0 0x7ff9cc0b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9caa00640 1 -- 172.21.15.42:0/3675144294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9cc0ce380 msgr2=0x7ff9cc0b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9caa00640 1 --2- 172.21.15.42:0/3675144294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9cc0ce380 0x7ff9cc0b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9caa00640 1 -- 172.21.15.42:0/3675144294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc15e1d0 con 0x7ff9cc09aa50 2024-05-28T14:04:29.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9caa00640 1 --2- 172.21.15.42:0/3675144294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc09aa50 0x7ff9cc0cbfc0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00c370 tx=0x7ff9bc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:29.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9c3400640 1 -- 172.21.15.42:0/3675144294 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0052d0 con 0x7ff9cc09aa50 2024-05-28T14:04:29.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9c3400640 1 -- 172.21.15.42:0/3675144294 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9bc005470 con 0x7ff9cc09aa50 2024-05-28T14:04:29.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9c3400640 1 -- 172.21.15.42:0/3675144294 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc007260 con 0x7ff9cc09aa50 2024-05-28T14:04:29.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9d1a00640 1 -- 172.21.15.42:0/3675144294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9cc15e4b0 con 0x7ff9cc09aa50 2024-05-28T14:04:29.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.495+0000 7ff9d1a00640 1 -- 172.21.15.42:0/3675144294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9cc15ea70 con 0x7ff9cc09aa50 2024-05-28T14:04:29.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9c3400640 1 -- 172.21.15.42:0/3675144294 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7ff9bc007400 con 0x7ff9cc09aa50 2024-05-28T14:04:29.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9c3400640 1 --2- 172.21.15.42:0/3675144294 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9a8068b60 0x7ff9a806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9c3400640 1 -- 172.21.15.42:0/3675144294 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7ff9bc086fc0 con 0x7ff9cc09aa50 2024-05-28T14:04:29.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9c2000640 1 --2- 172.21.15.42:0/3675144294 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9a8068b60 0x7ff9a806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9d4f3a000 1 Processor -- start 2024-05-28T14:04:29.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9d4f3a000 1 -- start start 2024-05-28T14:04:29.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9c2000640 1 --2- 172.21.15.42:0/3675144294 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9a8068b60 0x7ff9a806b020 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc15fb80 tx=0x7ff9b4009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:29.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9d4f3a000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 0x55d9b46b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9d4f3a000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 0x55d9b4665110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9d4f3a000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b4665a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9d4f3a000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d9b44b6420 con 0x55d9b4665650 2024-05-28T14:04:29.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9d4f3a000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d9b4450bf0 con 0x55d9b46b2a40 2024-05-28T14:04:29.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9d4f3a000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d9b44b23e0 con 0x55d9b46b2120 2024-05-28T14:04:29.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9cb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b4665a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9cb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b4665a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60918/0 (socket says 172.21.15.42:60918) 2024-05-28T14:04:29.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9cb400640 1 -- 172.21.15.42:0/3366404210 learned_addr learned my addr 172.21.15.42:0/3366404210 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:29.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9c2000640 1 --2- 172.21.15.42:0/3366404210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 0x55d9b4665110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9caa00640 1 --2- 172.21.15.42:0/3366404210 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 0x55d9b46b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9cb400640 1 -- 172.21.15.42:0/3366404210 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 msgr2=0x55d9b46b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:29.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9cb400640 1 --2- 172.21.15.42:0/3366404210 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 0x55d9b46b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9cb400640 1 -- 172.21.15.42:0/3366404210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 msgr2=0x55d9b4665110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:29.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9cb400640 1 --2- 172.21.15.42:0/3366404210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 0x55d9b4665110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9cb400640 1 -- 172.21.15.42:0/3366404210 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b455e450 con 0x55d9b4665650 2024-05-28T14:04:29.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.499+0000 7ff9cb400640 1 --2- 172.21.15.42:0/3366404210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b4665a30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009040 tx=0x7ff9c4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efec5f5c4f23ae7a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:29.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9bb400640 1 -- 172.21.15.42:0/3366404210 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40bc070 con 0x55d9b4665650 2024-05-28T14:04:29.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9bb400640 1 -- 172.21.15.42:0/3366404210 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c40a58d0 con 0x55d9b4665650 2024-05-28T14:04:29.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9bb400640 1 -- 172.21.15.42:0/3366404210 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40a5a80 con 0x55d9b4665650 2024-05-28T14:04:29.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/3366404210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 msgr2=0x55d9b4665a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/3366404210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b4665a30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009040 tx=0x7ff9c4004ac0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/3366404210 shutdown_connections 2024-05-28T14:04:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/3366404210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b4665a30 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/3366404210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 0x55d9b4665110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/3366404210 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 0x55d9b46b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/3366404210 >> 172.21.15.42:0/3366404210 conn(0x55d9b4692ec0 msgr2=0x55d9b46952b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/3366404210 shutdown_connections 2024-05-28T14:04:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/3366404210 wait complete. 2024-05-28T14:04:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 -- start start 2024-05-28T14:04:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 Processor -- start 2024-05-28T14:04:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b466ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2120 0x55d9b466b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2a40 0x55d9b466b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d9b4456ec0 con 0x55d9b4665650 2024-05-28T14:04:29.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d9b4537120 con 0x55d9b46b2120 2024-05-28T14:04:29.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d9b44c1350 con 0x55d9b46b2a40 2024-05-28T14:04:29.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 10 client.0 _ll_register_callbacks cb 0x55d9b44934d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T14:04:29.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b466ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b466ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60932/0 (socket says 172.21.15.42:60932) 2024-05-28T14:04:29.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9caa00640 1 -- 172.21.15.42:0/2629713245 learned_addr learned my addr 172.21.15.42:0/2629713245 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:29.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9c2000640 1 --2- 172.21.15.42:0/2629713245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2120 0x55d9b466b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9cb400640 1 --2- 172.21.15.42:0/2629713245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2a40 0x55d9b466b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9caa00640 1 -- 172.21.15.42:0/2629713245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2a40 msgr2=0x55d9b466b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:29.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9caa00640 1 --2- 172.21.15.42:0/2629713245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2a40 0x55d9b466b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9caa00640 1 -- 172.21.15.42:0/2629713245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2120 msgr2=0x55d9b466b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:29.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9caa00640 1 --2- 172.21.15.42:0/2629713245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2120 0x55d9b466b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9caa00640 1 -- 172.21.15.42:0/2629713245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b44fbf80 con 0x55d9b4665650 2024-05-28T14:04:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9caa00640 1 --2- 172.21.15.42:0/2629713245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b466ab40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00cd30 tx=0x7ff9bc055c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c6742218cbb249 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9afe00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T14:04:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9afe00640 1 -- 172.21.15.42:0/2629713245 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc082060 con 0x55d9b4665650 2024-05-28T14:04:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9afe00640 1 -- 172.21.15.42:0/2629713245 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7ff9bc06e3e0 con 0x55d9b4665650 2024-05-28T14:04:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.503+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2629713245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c40a58d0 con 0x55d9b4665650 2024-05-28T14:04:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9afe00640 1 -- 172.21.15.42:0/2629713245 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff9bc073050 con 0x55d9b4665650 2024-05-28T14:04:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 10 client.6552 fetch_fsmap learned FSMap version 36 2024-05-28T14:04:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2629713245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55d9b448ff00 con 0x55d9b4665650 2024-05-28T14:04:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9afe00640 1 -- 172.21.15.42:0/2629713245 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c40a58d0 con 0x55d9b4665650 2024-05-28T14:04:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9afe00640 1 -- 172.21.15.42:0/2629713245 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7ff9bc055410 con 0x55d9b4665650 2024-05-28T14:04:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 10 client.6552 fetch_fsmap finished waiting for FSMap version 36 2024-05-28T14:04:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 10 client.6552 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-28T14:04:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 20 client.6552 populate_metadata read hostname 'smithi042' 2024-05-28T14:04:29.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/2629713245 >> [v2:172.21.15.63:6833/3311861302,v1:172.21.15.63:6835/3311861302] conn(0x55d9b46c13c0 0x55d9b46c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 4 client.6552 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-28T14:04:29.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2629713245 --> [v2:172.21.15.63:6833/3311861302,v1:172.21.15.63:6835/3311861302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9b444fbc0 con 0x55d9b46c13c0 2024-05-28T14:04:29.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9c2000640 1 --2- 172.21.15.42:0/2629713245 >> [v2:172.21.15.63:6833/3311861302,v1:172.21.15.63:6835/3311861302] conn(0x55d9b46c13c0 0x55d9b46c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9c2000640 1 --2- 172.21.15.42:0/2629713245 >> [v2:172.21.15.63:6833/3311861302,v1:172.21.15.63:6835/3311861302] conn(0x55d9b46c13c0 0x55d9b46c37b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2efd526fb669963a server_cookie=64898065ec74bd35 in_seq=0 out_seq=0 2024-05-28T14:04:29.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9afe00640 10 client.6552 ms_handle_connect on v2:172.21.15.63:6833/3311861302 2024-05-28T14:04:29.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9afe00640 1 -- 172.21.15.42:0/2629713245 <== mds.0 v2:172.21.15.63:6833/3311861302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9b444fbc0 con 0x55d9b46c13c0 2024-05-28T14:04:29.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9afe00640 10 client.6552 handle_command_reply: tid=0 2024-05-28T14:04:29.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 1 client.6552 shutdown 2024-05-28T14:04:29.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2629713245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 msgr2=0x55d9b466ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/2629713245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b466ab40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00cd30 tx=0x7ff9bc055c30 comp rx=0 tx=0).stop 2024-05-28T14:04:29.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2629713245 shutdown_connections 2024-05-28T14:04:29.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/2629713245 >> [v2:172.21.15.63:6833/3311861302,v1:172.21.15.63:6835/3311861302] conn(0x55d9b46c13c0 0x55d9b46c37b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/2629713245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2a40 0x55d9b466b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/2629713245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2120 0x55d9b466b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/2629713245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b466ab40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9afe00640 0 client.6552 ms_handle_reset on v2:172.21.15.63:6833/3311861302 2024-05-28T14:04:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2629713245 >> 172.21.15.42:0/2629713245 conn(0x55d9b4692ec0 msgr2=0x55d9b46952b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2629713245 shutdown_connections 2024-05-28T14:04:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2629713245 wait complete. 2024-05-28T14:04:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.507+0000 7ff9d4f3a000 20 client.6552 trim_cache size 0 max 16384 2024-05-28T14:04:29.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 Processor -- start 2024-05-28T14:04:29.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 -- start start 2024-05-28T14:04:29.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b4664810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 0x55d9b4664d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 0x55d9b46c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc0cbd20 con 0x55d9b4665650 2024-05-28T14:04:29.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc0cb080 con 0x55d9b46b2a40 2024-05-28T14:04:29.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9cc0cb6d0 con 0x55d9b46b2120 2024-05-28T14:04:29.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b4664810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b4664810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60936/0 (socket says 172.21.15.42:60936) 2024-05-28T14:04:29.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9caa00640 1 -- 172.21.15.42:0/504958067 learned_addr learned my addr 172.21.15.42:0/504958067 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:29.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9cb400640 1 --2- 172.21.15.42:0/504958067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 0x55d9b46c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9c2000640 1 --2- 172.21.15.42:0/504958067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 0x55d9b4664d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9caa00640 1 -- 172.21.15.42:0/504958067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 msgr2=0x55d9b4664d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9caa00640 1 --2- 172.21.15.42:0/504958067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 0x55d9b4664d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9caa00640 1 -- 172.21.15.42:0/504958067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 msgr2=0x55d9b46c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9caa00640 1 --2- 172.21.15.42:0/504958067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 0x55d9b46c37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9caa00640 1 -- 172.21.15.42:0/504958067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b443ce80 con 0x55d9b4665650 2024-05-28T14:04:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9caa00640 1 --2- 172.21.15.42:0/504958067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b4664810 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc052420 tx=0x7ff9bc06c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a88d3f43004c5a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9baa00640 1 -- 172.21.15.42:0/504958067 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc06e1e0 con 0x55d9b4665650 2024-05-28T14:04:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9baa00640 1 -- 172.21.15.42:0/504958067 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9bc06e030 con 0x55d9b4665650 2024-05-28T14:04:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9baa00640 1 -- 172.21.15.42:0/504958067 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc051d10 con 0x55d9b4665650 2024-05-28T14:04:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/504958067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 msgr2=0x55d9b4664810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/504958067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b4664810 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc052420 tx=0x7ff9bc06c000 comp rx=0 tx=0).stop 2024-05-28T14:04:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/504958067 shutdown_connections 2024-05-28T14:04:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/504958067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 0x55d9b46c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/504958067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 0x55d9b4664d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/504958067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b4664810 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/504958067 >> 172.21.15.42:0/504958067 conn(0x55d9b4692ec0 msgr2=0x55d9b4664100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/504958067 shutdown_connections 2024-05-28T14:04:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.527+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/504958067 wait complete. 2024-05-28T14:04:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 1 -- start start 2024-05-28T14:04:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 1 Processor -- start 2024-05-28T14:04:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b46c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 0x55d9b46dd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 0x55d9b46dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d9b44b5730 con 0x55d9b4665650 2024-05-28T14:04:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d9b44829e0 con 0x55d9b46b2a40 2024-05-28T14:04:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d9b44baa50 con 0x55d9b46b2120 2024-05-28T14:04:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 10 client.0 _ll_register_callbacks cb 0x7ff9bc0559f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T14:04:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b46c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b46c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60952/0 (socket says 172.21.15.42:60952) 2024-05-28T14:04:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9caa00640 1 -- 172.21.15.42:0/2335704124 learned_addr learned my addr 172.21.15.42:0/2335704124 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9cb400640 1 --2- 172.21.15.42:0/2335704124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 0x55d9b46dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9c2000640 1 --2- 172.21.15.42:0/2335704124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 0x55d9b46dd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9caa00640 1 -- 172.21.15.42:0/2335704124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 msgr2=0x55d9b46dd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9caa00640 1 --2- 172.21.15.42:0/2335704124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 0x55d9b46dd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9caa00640 1 -- 172.21.15.42:0/2335704124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 msgr2=0x55d9b46dd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9caa00640 1 --2- 172.21.15.42:0/2335704124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 0x55d9b46dd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9caa00640 1 -- 172.21.15.42:0/2335704124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b447eb30 con 0x55d9b4665650 2024-05-28T14:04:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9caa00640 1 --2- 172.21.15.42:0/2335704124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b46c61f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc054270 tx=0x7ff9bc084930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff491a6c33b230 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9afe00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T14:04:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9afe00640 1 -- 172.21.15.42:0/2335704124 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc054e80 con 0x55d9b4665650 2024-05-28T14:04:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9afe00640 1 -- 172.21.15.42:0/2335704124 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7ff9bc004cd0 con 0x55d9b4665650 2024-05-28T14:04:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2335704124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9bc06e030 con 0x55d9b4665650 2024-05-28T14:04:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9afe00640 1 -- 172.21.15.42:0/2335704124 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff9bc082030 con 0x55d9b4665650 2024-05-28T14:04:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 10 client.6558 fetch_fsmap learned FSMap version 36 2024-05-28T14:04:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2335704124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55d9b45375b0 con 0x55d9b4665650 2024-05-28T14:04:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9afe00640 1 -- 172.21.15.42:0/2335704124 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9bc06e030 con 0x55d9b4665650 2024-05-28T14:04:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9afe00640 1 -- 172.21.15.42:0/2335704124 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7ff9bc055020 con 0x55d9b4665650 2024-05-28T14:04:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 10 client.6558 fetch_fsmap finished waiting for FSMap version 36 2024-05-28T14:04:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 10 client.6558 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-28T14:04:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 20 client.6558 populate_metadata read hostname 'smithi042' 2024-05-28T14:04:29.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/2335704124 >> [v2:172.21.15.63:6833/3311861302,v1:172.21.15.63:6835/3311861302] conn(0x55d9b46dddc0 0x55d9b4720fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 4 client.6558 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-28T14:04:29.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2335704124 --> [v2:172.21.15.63:6833/3311861302,v1:172.21.15.63:6835/3311861302] -- command(tid 0: {"prefix": "status"}) -- 0x55d9b44c3e40 con 0x55d9b46dddc0 2024-05-28T14:04:29.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9c2000640 1 --2- 172.21.15.42:0/2335704124 >> [v2:172.21.15.63:6833/3311861302,v1:172.21.15.63:6835/3311861302] conn(0x55d9b46dddc0 0x55d9b4720fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9c2000640 1 --2- 172.21.15.42:0/2335704124 >> [v2:172.21.15.63:6833/3311861302,v1:172.21.15.63:6835/3311861302] conn(0x55d9b46dddc0 0x55d9b4720fb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c6885c312e47b4 server_cookie=f1231cc82be4734b in_seq=0 out_seq=0 2024-05-28T14:04:29.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.531+0000 7ff9afe00640 10 client.6558 ms_handle_connect on v2:172.21.15.63:6833/3311861302 2024-05-28T14:04:29.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9afe00640 1 -- 172.21.15.42:0/2335704124 <== mds.0 v2:172.21.15.63:6833/3311861302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55d9b44c3e40 con 0x55d9b46dddc0 2024-05-28T14:04:29.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9afe00640 10 client.6558 handle_command_reply: tid=0 2024-05-28T14:04:29.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d4f3a000 1 client.6558 shutdown 2024-05-28T14:04:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2335704124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 msgr2=0x55d9b46c61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/2335704124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b46c61f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc054270 tx=0x7ff9bc084930 comp rx=0 tx=0).stop 2024-05-28T14:04:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2335704124 shutdown_connections 2024-05-28T14:04:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/2335704124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d9b46b2a40 0x55d9b46dd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/2335704124 >> [v2:172.21.15.63:6833/3311861302,v1:172.21.15.63:6835/3311861302] conn(0x55d9b46dddc0 0x55d9b4720fb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/2335704124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d9b46b2120 0x55d9b46dd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d4f3a000 1 --2- 172.21.15.42:0/2335704124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9b4665650 0x55d9b46c61f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9afe00640 0 client.6558 ms_handle_reset on v2:172.21.15.63:6833/3311861302 2024-05-28T14:04:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2335704124 >> 172.21.15.42:0/2335704124 conn(0x55d9b4692ec0 msgr2=0x55d9b4664810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2335704124 shutdown_connections 2024-05-28T14:04:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d4f3a000 1 -- 172.21.15.42:0/2335704124 wait complete. 2024-05-28T14:04:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d4f3a000 20 client.6558 trim_cache size 0 max 16384 2024-05-28T14:04:29.526 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:04:29.526 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:04:29.526 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-05-28T14:04:29.526 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 6420, 2024-05-28T14:04:29.526 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-28T14:04:29.526 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-28T14:04:29.526 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-28T14:04:29.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 5.9582337719999998, 2024-05-28T14:04:29.526 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 36, 2024-05-28T14:04:29.527 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 61, 2024-05-28T14:04:29.527 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 61, 2024-05-28T14:04:29.527 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 11.777550243 2024-05-28T14:04:29.527 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:04:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d1a00640 1 -- 172.21.15.42:0/3675144294 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9a8068b60 msgr2=0x7ff9a806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d1a00640 1 --2- 172.21.15.42:0/3675144294 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9a8068b60 0x7ff9a806b020 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc15fb80 tx=0x7ff9b4009190 comp rx=0 tx=0).stop 2024-05-28T14:04:29.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d1a00640 1 -- 172.21.15.42:0/3675144294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc09aa50 msgr2=0x7ff9cc0cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d1a00640 1 --2- 172.21.15.42:0/3675144294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc09aa50 0x7ff9cc0cbfc0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00c370 tx=0x7ff9bc00c830 comp rx=0 tx=0).stop 2024-05-28T14:04:29.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d1a00640 1 -- 172.21.15.42:0/3675144294 shutdown_connections 2024-05-28T14:04:29.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d1a00640 1 --2- 172.21.15.42:0/3675144294 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9a8068b60 0x7ff9a806b020 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d1a00640 1 --2- 172.21.15.42:0/3675144294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9cc0ce380 0x7ff9cc0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d1a00640 1 --2- 172.21.15.42:0/3675144294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9cc0b4bc0 0x7ff9cc0b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d1a00640 1 --2- 172.21.15.42:0/3675144294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc09aa50 0x7ff9cc0cbfc0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d1a00640 1 -- 172.21.15.42:0/3675144294 >> 172.21.15.42:0/3675144294 conn(0x7ff9cc0b4010 msgr2=0x7ff9cc06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:29.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d1a00640 1 -- 172.21.15.42:0/3675144294 shutdown_connections 2024-05-28T14:04:29.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.535+0000 7ff9d1a00640 1 -- 172.21.15.42:0/3675144294 wait complete. 2024-05-28T14:04:29.540 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-28T14:04:29.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.695+0000 7f5a76400640 1 Processor -- start 2024-05-28T14:04:29.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.695+0000 7f5a76400640 1 -- start start 2024-05-28T14:04:29.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.695+0000 7f5a76400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5a70156a70 0x7f5a7006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.695+0000 7f5a76400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5a70156e50 0x7f5a700af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.695+0000 7f5a76400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a700afc80 con 0x7f5a700af880 2024-05-28T14:04:29.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.695+0000 7f5a76400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5a7015cb90 con 0x7f5a70156a70 2024-05-28T14:04:29.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.695+0000 7f5a76400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5a7015ce50 con 0x7f5a70156e50 2024-05-28T14:04:29.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.699+0000 7f5a6f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5a70156a70 0x7f5a7006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.699+0000 7f5a67e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5a70156e50 0x7f5a700af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.699+0000 7f5a6f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5a70156a70 0x7f5a7006dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60964/0 (socket says 172.21.15.42:60964) 2024-05-28T14:04:29.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.699+0000 7f5a6f400640 1 -- 172.21.15.42:0/3081463758 learned_addr learned my addr 172.21.15.42:0/3081463758 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:29.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.699+0000 7f5a6fe00640 1 --1- 172.21.15.42:0/3081463758 >> v1:172.21.15.63:6789/0 conn(0x7f5a700af880 0x7f5a700d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:35938/0 (socket says 172.21.15.42:35938) 2024-05-28T14:04:29.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.699+0000 7f5a6f400640 1 -- 172.21.15.42:0/3081463758 >> v2:172.21.15.63:3300/0 conn(0x7f5a70156e50 msgr2=0x7f5a700af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:29.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.699+0000 7f5a6f400640 1 --2- 172.21.15.42:0/3081463758 >> v2:172.21.15.63:3300/0 conn(0x7f5a70156e50 0x7f5a700af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.699+0000 7f5a6f400640 1 -- 172.21.15.42:0/3081463758 >> v1:172.21.15.63:6789/0 conn(0x7f5a700af880 legacy=0x7f5a700d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.699+0000 7f5a6f400640 1 -- 172.21.15.42:0/3081463758 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7015d130 con 0x7f5a70156a70 2024-05-28T14:04:29.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.699+0000 7f5a67e00640 1 --2- 172.21.15.42:0/3081463758 >> v2:172.21.15.63:3300/0 conn(0x7f5a70156e50 0x7f5a700af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:04:29.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.699+0000 7f5a6f400640 1 --2- 172.21.15.42:0/3081463758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70156a70 0x7f5a7006dd60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5a700701f0 tx=0x7f5a6001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee64eb6e8dd267e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.699+0000 7f5a6ea00640 1 -- 172.21.15.42:0/3081463758 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a600aa680 con 0x7f5a70156a70 2024-05-28T14:04:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.699+0000 7f5a6ea00640 1 -- 172.21.15.42:0/3081463758 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a600af070 con 0x7f5a70156a70 2024-05-28T14:04:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.699+0000 7f5a6ea00640 1 -- 172.21.15.42:0/3081463758 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a600aaa40 con 0x7f5a70156a70 2024-05-28T14:04:29.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 -- 172.21.15.42:0/3081463758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70156a70 msgr2=0x7f5a7006dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 --2- 172.21.15.42:0/3081463758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70156a70 0x7f5a7006dd60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5a700701f0 tx=0x7f5a6001c520 comp rx=0 tx=0).stop 2024-05-28T14:04:29.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 -- 172.21.15.42:0/3081463758 shutdown_connections 2024-05-28T14:04:29.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 --2- 172.21.15.42:0/3081463758 >> v2:172.21.15.63:3300/0 conn(0x7f5a70156e50 0x7f5a700af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 --2- 172.21.15.42:0/3081463758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70156a70 0x7f5a7006dd60 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 -- 172.21.15.42:0/3081463758 >> 172.21.15.42:0/3081463758 conn(0x7f5a7006bdf0 msgr2=0x7f5a7006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:29.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 -- 172.21.15.42:0/3081463758 shutdown_connections 2024-05-28T14:04:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 -- 172.21.15.42:0/3081463758 wait complete. 2024-05-28T14:04:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 Processor -- start 2024-05-28T14:04:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 -- start start 2024-05-28T14:04:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a700af880 0x7f5a70152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70156a70 0x7f5a700cf360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a70156e50 0x7f5a70152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70160b40 con 0x7f5a70156a70 2024-05-28T14:04:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a700d1010 con 0x7f5a700af880 2024-05-28T14:04:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a76400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5a70160cc0 con 0x7f5a70156e50 2024-05-28T14:04:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a67e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70156a70 0x7f5a700cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a67e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70156a70 0x7f5a700cf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60974/0 (socket says 172.21.15.42:60974) 2024-05-28T14:04:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a67e00640 1 -- 172.21.15.42:0/300335642 learned_addr learned my addr 172.21.15.42:0/300335642 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a67e00640 1 -- 172.21.15.42:0/300335642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a70156e50 msgr2=0x7f5a70152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a6fe00640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a70156e50 0x7f5a70152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a6f400640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a700af880 0x7f5a70152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a67e00640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a70156e50 0x7f5a70152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a67e00640 1 -- 172.21.15.42:0/300335642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a700af880 msgr2=0x7f5a70152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a67e00640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a700af880 0x7f5a70152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a67e00640 1 -- 172.21.15.42:0/300335642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a70166cf0 con 0x7f5a70156a70 2024-05-28T14:04:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a6f400640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a700af880 0x7f5a70152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a6fe00640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a70156e50 0x7f5a70152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.703+0000 7f5a67e00640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70156a70 0x7f5a700cf360 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800e980 tx=0x7f5a5800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.707+0000 7f5a67400640 1 -- 172.21.15.42:0/300335642 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58017070 con 0x7f5a70156a70 2024-05-28T14:04:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.707+0000 7f5a67400640 1 -- 172.21.15.42:0/300335642 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5800ca00 con 0x7f5a70156a70 2024-05-28T14:04:29.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.707+0000 7f5a67400640 1 -- 172.21.15.42:0/300335642 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5800cdd0 con 0x7f5a70156a70 2024-05-28T14:04:29.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.707+0000 7f5a76400640 1 -- 172.21.15.42:0/300335642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a700739b0 con 0x7f5a70156a70 2024-05-28T14:04:29.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.707+0000 7f5a76400640 1 -- 172.21.15.42:0/300335642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a70166980 con 0x7f5a70156a70 2024-05-28T14:04:29.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.707+0000 7f5a76400640 1 -- 172.21.15.42:0/300335642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a70068a90 con 0x7f5a70156a70 2024-05-28T14:04:29.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.707+0000 7f5a67400640 1 -- 172.21.15.42:0/300335642 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f5a5801a070 con 0x7f5a70156a70 2024-05-28T14:04:29.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.707+0000 7f5a67400640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a4c068b60 0x7f5a4c06b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:29.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.707+0000 7f5a67400640 1 -- 172.21.15.42:0/300335642 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f5a58087de0 con 0x7f5a70156a70 2024-05-28T14:04:29.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.707+0000 7f5a6f400640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a4c068b60 0x7f5a4c06b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:29.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.711+0000 7f5a6f400640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a4c068b60 0x7f5a4c06b020 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5a600ae000 tx=0x7f5a60008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:29.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.711+0000 7f5a67400640 1 -- 172.21.15.42:0/300335642 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5a58057940 con 0x7f5a70156a70 2024-05-28T14:04:29.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.911+0000 7f5a76400640 1 -- 172.21.15.42:0/300335642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5a700cec10 con 0x7f5a70156a70 2024-05-28T14:04:29.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.911+0000 7f5a67400640 1 -- 172.21.15.42:0/300335642 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f5a58057240 con 0x7f5a70156a70 2024-05-28T14:04:29.956 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-28T14:04:29.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.915+0000 7f5a76400640 1 -- 172.21.15.42:0/300335642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a4c068b60 msgr2=0x7f5a4c06b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.915+0000 7f5a76400640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a4c068b60 0x7f5a4c06b020 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5a600ae000 tx=0x7f5a60008040 comp rx=0 tx=0).stop 2024-05-28T14:04:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.915+0000 7f5a76400640 1 -- 172.21.15.42:0/300335642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70156a70 msgr2=0x7f5a700cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.915+0000 7f5a76400640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70156a70 0x7f5a700cf360 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800e980 tx=0x7f5a5800ee40 comp rx=0 tx=0).stop 2024-05-28T14:04:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.915+0000 7f5a76400640 1 -- 172.21.15.42:0/300335642 shutdown_connections 2024-05-28T14:04:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.915+0000 7f5a76400640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5a4c068b60 0x7f5a4c06b020 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.915+0000 7f5a76400640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a70156e50 0x7f5a70152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.915+0000 7f5a76400640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a70156a70 0x7f5a700cf360 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.915+0000 7f5a76400640 1 --2- 172.21.15.42:0/300335642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a700af880 0x7f5a70152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.915+0000 7f5a76400640 1 -- 172.21.15.42:0/300335642 >> 172.21.15.42:0/300335642 conn(0x7f5a7006bdf0 msgr2=0x7f5a700afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.915+0000 7f5a76400640 1 -- 172.21.15.42:0/300335642 shutdown_connections 2024-05-28T14:04:29.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:29.915+0000 7f5a76400640 1 -- 172.21.15.42:0/300335642 wait complete. 2024-05-28T14:04:29.958 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-28T14:04:29.959 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-28T14:04:29.959 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-05-28T14:04:29.959 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-28T14:04:29.959 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-28T14:04:29.959 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-28T14:04:29.959 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-28T14:04:29.960 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-28T14:04:29.960 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:04:29.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:29.960 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-05-28T14:04:29.977 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T14:04:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T14:04:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T14:04:29.977 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:04:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-05-28T14:04:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:04:29.978 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:04:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-05-28T14:04:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-05-28T14:04:29.978 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:04:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-05-28T14:04:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-05-28T14:04:29.978 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-28T14:04:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:04:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-05-28T14:04:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-28T14:04:29.978 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42077sec preferred_lft 42077sec 2024-05-28T14:04:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2024-05-28T14:04:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:04:29.979 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:04:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:04:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-05-28T14:04:29.979 INFO:teuthology.orchestra.run.smithi042.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-28T14:04:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 02:42:65:2b:63:73 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:04:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-28T14:04:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:04:29.979 INFO:teuthology.orchestra.run.smithi042.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-28T14:04:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-28T14:04:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-28T14:04:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:04:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-28T14:04:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:04:29.980 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T14:04:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:04:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-28T14:04:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:04:29.980 INFO:teuthology.orchestra.run.smithi042.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T14:04:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:04:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-28T14:04:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:04:29.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:29.981 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-28T14:04:30.038 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-05-28T14:04:30.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:30.041 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-05-28T14:04:30.097 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-05-28T14:04:30.097 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-28T14:04:30.097 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-05-28T14:04:30.097 INFO:teuthology.orchestra.run.smithi042.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-28T14:04:30.097 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-05-28T14:04:30.098 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-28T14:04:30.098 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T14:04:30.098 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T14:04:30.098 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-28T14:04:30.098 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-28T14:04:30.098 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-28T14:04:30.098 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T14:04:30.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T14:04:30.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:04:30.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:30.235 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-28T14:04:30.244 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-28T14:04:30.245 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-28T14:04:30.245 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:04:30.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:30.245 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-05-28T14:04:30.311 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-28T14:04:30.311 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T14:04:30.311 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-28T14:04:30.363 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:04:30.363 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T14:04:30.363 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-28T14:04:30.414 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:04:30.476 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:04:30.529 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:04:30.529 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:04:30.529 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:04:30.529 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:04:30.529 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:04:30.529 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:04:30.529 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:04:30.530 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:04:30.530 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:04:30.530 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:04:30.530 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:04:30.530 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:04:30.530 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:04:30.530 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:04:30.530 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:04:30.530 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:04:30.530 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:04:30.530 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:04:30.530 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:04:30.531 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:04:30.531 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:04:30.531 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:04:30.531 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:04:30.531 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:04:30.531 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:04:30.531 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:04:30.531 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:04:30.531 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:04:30.531 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:04:30.531 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:04:30.531 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:04:30.531 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:04:30.532 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:04:30.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:30.533 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:04:30.587 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-28T14:04:30.587 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-28T14:04:30.633 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:04:30.654 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:04:30.708 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:04:30.709 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:04:30.709 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:04:30.709 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:04:30.709 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:04:30.709 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:04:30.709 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:04:30.709 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:04:30.709 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:04:30.709 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:04:30.709 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:04:30.709 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:04:30.710 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:04:30.710 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:04:30.710 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:04:30.710 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:04:30.710 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:04:30.710 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:04:30.710 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:04:30.710 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:04:30.710 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:04:30.710 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:04:30.710 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:04:30.710 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:04:30.710 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:04:30.711 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:04:30.711 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:04:30.711 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:04:30.711 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:04:30.711 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:04:30.711 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:04:30.711 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:04:30.711 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:04:30.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:30.712 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:04:30.729 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-05-28T14:04:30.743+0000 7f76e2de8580 -1 init, newargv = 0x555cfe875b60 newargc=15 2024-05-28T14:04:30.730 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[41448]: starting ceph client 2024-05-28T14:04:30.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[41448]: starting fuse 2024-05-28T14:04:30.738 INFO:teuthology.orchestra.run.smithi042.stdout:44 2024-05-28T14:04:30.738 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-05-28T14:04:30.739 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:04:30.740 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-05-28T14:04:30.740 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-28T14:04:30.740 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-28T14:04:30.740 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-28T14:04:30.740 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-05-28T14:04:30.740 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-05-28T14:04:30.740 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-05-28T14:04:30.740 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-28T14:04:30.740 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:04:30.740 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-05-28T14:04:30.740 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-05-28T14:04:30.864 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.41448.asok 2024-05-28T14:04:30.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:04:30.875 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41448.asok 2024-05-28T14:04:30.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:30.875 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41448.asok status 2024-05-28T14:04:31.055 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:04:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-05-28T14:04:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-28T14:04:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-28T14:04:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-05-28T14:04:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-05-28T14:04:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-28T14:04:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "41448", 2024-05-28T14:04:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-05-28T14:04:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-05-28T14:04:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-05-28T14:04:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 6573, 2024-05-28T14:04:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-05-28T14:04:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-05-28T14:04:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-05-28T14:04:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 6573 2024-05-28T14:04:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T14:04:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T14:04:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T14:04:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3310764712 2024-05-28T14:04:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T14:04:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T14:04:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T14:04:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3310764712 2024-05-28T14:04:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:04:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.6573 192.168.0.1:0/3310764712", 2024-05-28T14:04:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/3310764712", 2024-05-28T14:04:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-05-28T14:04:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 36, 2024-05-28T14:04:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 61, 2024-05-28T14:04:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-05-28T14:04:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-05-28T14:04:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-05-28T14:04:31.058 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:04:31.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:31.072 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-05-28T14:04:31.072 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-28T14:04:31.081 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-05-28T14:04:31.081 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-28T14:04:31.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:31.082 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-28T14:04:31.143 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-28T14:04:31.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.343+0000 7f1031000640 1 Processor -- start 2024-05-28T14:04:31.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f1031000640 1 -- start start 2024-05-28T14:04:31.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f1031000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f102c15e460 0x7f102c0d2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:31.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f1031000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f102c0d1520 0x7f102c0d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:31.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f1031000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f102c0ced30 con 0x7f102c0d1be0 2024-05-28T14:04:31.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f1031000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f102c162190 con 0x7f102c15e460 2024-05-28T14:04:31.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f1031000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f102c162470 con 0x7f102c0d1520 2024-05-28T14:04:31.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f102b400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f102c15e460 0x7f102c0d2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:31.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f1023e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f102c0d1520 0x7f102c0d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:31.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f102b400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f102c15e460 0x7f102c0d2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:53550/0 (socket says 172.21.15.42:53550) 2024-05-28T14:04:31.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f102b400640 1 -- 172.21.15.42:0/3928287164 learned_addr learned my addr 172.21.15.42:0/3928287164 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:31.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f102be00640 1 --1- 172.21.15.42:0/3928287164 >> v1:172.21.15.63:6789/0 conn(0x7f102c0d1be0 0x7f102c0714d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:35940/0 (socket says 172.21.15.42:35940) 2024-05-28T14:04:31.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f1023e00640 1 -- 172.21.15.42:0/3928287164 >> v2:172.21.15.63:3300/0 conn(0x7f102c15e460 msgr2=0x7f102c0d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:31.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f1023e00640 1 --2- 172.21.15.42:0/3928287164 >> v2:172.21.15.63:3300/0 conn(0x7f102c15e460 0x7f102c0d2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f1023e00640 1 -- 172.21.15.42:0/3928287164 >> v1:172.21.15.63:6789/0 conn(0x7f102c0d1be0 legacy=0x7f102c0714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:31.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f1023e00640 1 -- 172.21.15.42:0/3928287164 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c162750 con 0x7f102c0d1520 2024-05-28T14:04:31.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f102aa00640 1 -- 172.21.15.42:0/3928287164 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2645574006 0 0) 0x7f10240014a0 con 0x7f102c0d1be0 2024-05-28T14:04:31.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.347+0000 7f102b400640 1 --2- 172.21.15.42:0/3928287164 >> v2:172.21.15.63:3300/0 conn(0x7f102c15e460 0x7f102c0d2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:04:31.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.351+0000 7f1023e00640 1 --2- 172.21.15.42:0/3928287164 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f102c0d1520 0x7f102c0d2780 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f10140097c0 tx=0x7f101401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b63ae1ebcea3da55 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:31.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.351+0000 7f102aa00640 1 -- 172.21.15.42:0/3928287164 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10140b0070 con 0x7f102c0d1520 2024-05-28T14:04:31.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.351+0000 7f102aa00640 1 -- 172.21.15.42:0/3928287164 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10140ab600 con 0x7f102c0d1520 2024-05-28T14:04:31.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.351+0000 7f102aa00640 1 -- 172.21.15.42:0/3928287164 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10140ab9d0 con 0x7f102c0d1520 2024-05-28T14:04:31.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.351+0000 7f1031000640 1 -- 172.21.15.42:0/3928287164 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f102c0d1520 msgr2=0x7f102c0d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:31.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.351+0000 7f1031000640 1 --2- 172.21.15.42:0/3928287164 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f102c0d1520 0x7f102c0d2780 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f10140097c0 tx=0x7f101401c4a0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.351+0000 7f1031000640 1 -- 172.21.15.42:0/3928287164 shutdown_connections 2024-05-28T14:04:31.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.351+0000 7f1031000640 1 --2- 172.21.15.42:0/3928287164 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f102c0d1520 0x7f102c0d2780 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.351+0000 7f1031000640 1 --2- 172.21.15.42:0/3928287164 >> v2:172.21.15.63:3300/0 conn(0x7f102c15e460 0x7f102c0d2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.351+0000 7f1031000640 1 -- 172.21.15.42:0/3928287164 >> 172.21.15.42:0/3928287164 conn(0x7f102c06fd70 msgr2=0x7f102c070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:31.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.351+0000 7f1031000640 1 -- 172.21.15.42:0/3928287164 shutdown_connections 2024-05-28T14:04:31.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.351+0000 7f1031000640 1 -- 172.21.15.42:0/3928287164 wait complete. 2024-05-28T14:04:31.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1031000640 1 Processor -- start 2024-05-28T14:04:31.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1031000640 1 -- start start 2024-05-28T14:04:31.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1031000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f102c0d1520 0x7f102c15f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:31.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1031000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f102c0d1be0 0x7f102c0cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:31.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1031000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f102c15e460 0x7f102c1527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:31.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1031000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f102c06e1d0 con 0x7f102c0d1520 2024-05-28T14:04:31.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1031000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f102c15e840 con 0x7f102c15e460 2024-05-28T14:04:31.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1031000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f102c161190 con 0x7f102c0d1be0 2024-05-28T14:04:31.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f102b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f102c0d1520 0x7f102c15f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:31.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f102b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f102c0d1520 0x7f102c15f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60986/0 (socket says 172.21.15.42:60986) 2024-05-28T14:04:31.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f102b400640 1 -- 172.21.15.42:0/1921468507 learned_addr learned my addr 172.21.15.42:0/1921468507 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:31.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1023e00640 1 --2- 172.21.15.42:0/1921468507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f102c0d1be0 0x7f102c0cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:31.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f102be00640 1 --2- 172.21.15.42:0/1921468507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f102c15e460 0x7f102c1527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:31.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f102b400640 1 -- 172.21.15.42:0/1921468507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f102c0d1be0 msgr2=0x7f102c0cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:31.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f102b400640 1 --2- 172.21.15.42:0/1921468507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f102c0d1be0 0x7f102c0cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f102b400640 1 -- 172.21.15.42:0/1921468507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f102c15e460 msgr2=0x7f102c1527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:31.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f102b400640 1 --2- 172.21.15.42:0/1921468507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f102c15e460 0x7f102c1527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f102b400640 1 -- 172.21.15.42:0/1921468507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c160580 con 0x7f102c0d1520 2024-05-28T14:04:31.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f102b400640 1 --2- 172.21.15.42:0/1921468507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f102c0d1520 0x7f102c15f9f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f101c00e930 tx=0x7f101c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:31.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1023400640 1 -- 172.21.15.42:0/1921468507 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c017070 con 0x7f102c0d1520 2024-05-28T14:04:31.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1023400640 1 -- 172.21.15.42:0/1921468507 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101c00ca00 con 0x7f102c0d1520 2024-05-28T14:04:31.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1031000640 1 -- 172.21.15.42:0/1921468507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102c160860 con 0x7f102c0d1520 2024-05-28T14:04:31.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1023400640 1 -- 172.21.15.42:0/1921468507 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c00cdd0 con 0x7f102c0d1520 2024-05-28T14:04:31.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1031000640 1 -- 172.21.15.42:0/1921468507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102c160e20 con 0x7f102c0d1520 2024-05-28T14:04:31.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.355+0000 7f1031000640 1 -- 172.21.15.42:0/1921468507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102c068a90 con 0x7f102c0d1520 2024-05-28T14:04:31.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.359+0000 7f1023400640 1 -- 172.21.15.42:0/1921468507 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f101c01a070 con 0x7f102c0d1520 2024-05-28T14:04:31.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.359+0000 7f1023400640 1 --2- 172.21.15.42:0/1921468507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1008068b60 0x7f100806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:31.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.359+0000 7f1023400640 1 -- 172.21.15.42:0/1921468507 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f101c087ba0 con 0x7f102c0d1520 2024-05-28T14:04:31.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.359+0000 7f1023e00640 1 --2- 172.21.15.42:0/1921468507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1008068b60 0x7f100806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:31.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.359+0000 7f1023e00640 1 --2- 172.21.15.42:0/1921468507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1008068b60 0x7f100806b020 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f10140ad000 tx=0x7f1014008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:31.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.363+0000 7f1023400640 1 -- 172.21.15.42:0/1921468507 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f101c056a40 con 0x7f102c0d1520 2024-05-28T14:04:31.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.567+0000 7f1031000640 1 -- 172.21.15.42:0/1921468507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f102c0b6120 con 0x7f102c0d1520 2024-05-28T14:04:31.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.611+0000 7f1023400640 1 -- 172.21.15.42:0/1921468507 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v37) ==== 136+0+0 (secure 0 0 0) 0x7f101c05ad40 con 0x7f102c0d1520 2024-05-28T14:04:31.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.615+0000 7f1031000640 1 -- 172.21.15.42:0/1921468507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1008068b60 msgr2=0x7f100806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:31.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.615+0000 7f1031000640 1 --2- 172.21.15.42:0/1921468507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1008068b60 0x7f100806b020 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f10140ad000 tx=0x7f1014008040 comp rx=0 tx=0).stop 2024-05-28T14:04:31.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.615+0000 7f1031000640 1 -- 172.21.15.42:0/1921468507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f102c0d1520 msgr2=0x7f102c15f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:31.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.615+0000 7f1031000640 1 --2- 172.21.15.42:0/1921468507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f102c0d1520 0x7f102c15f9f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f101c00e930 tx=0x7f101c00edf0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.615+0000 7f1031000640 1 -- 172.21.15.42:0/1921468507 shutdown_connections 2024-05-28T14:04:31.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.615+0000 7f1031000640 1 --2- 172.21.15.42:0/1921468507 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1008068b60 0x7f100806b020 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.615+0000 7f1031000640 1 --2- 172.21.15.42:0/1921468507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f102c15e460 0x7f102c1527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.615+0000 7f1031000640 1 --2- 172.21.15.42:0/1921468507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f102c0d1be0 0x7f102c0cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.615+0000 7f1031000640 1 --2- 172.21.15.42:0/1921468507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f102c0d1520 0x7f102c15f9f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.615+0000 7f1031000640 1 -- 172.21.15.42:0/1921468507 >> 172.21.15.42:0/1921468507 conn(0x7f102c06fd70 msgr2=0x7f102c0b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:31.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.615+0000 7f1031000640 1 -- 172.21.15.42:0/1921468507 shutdown_connections 2024-05-28T14:04:31.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.615+0000 7f1031000640 1 -- 172.21.15.42:0/1921468507 wait complete. 2024-05-28T14:04:31.621 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:04:31.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc21d000640 1 Processor -- start 2024-05-28T14:04:31.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc21d000640 1 -- start start 2024-05-28T14:04:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc21d000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc21815bf30 0x7fc21815c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc21d000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc2180cf230 0x7fc218072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc21d000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2180cfc10 con 0x7fc218072ba0 2024-05-28T14:04:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc21d000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc2180721d0 con 0x7fc2180cf230 2024-05-28T14:04:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc21d000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc2180cfef0 con 0x7fc21815bf30 2024-05-28T14:04:31.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc216800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc218072ba0 0x7fc21806ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51268/0 (socket says 172.21.15.42:51268) 2024-05-28T14:04:31.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc216800640 1 -- 172.21.15.42:0/1274846727 learned_addr learned my addr 172.21.15.42:0/1274846727 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:31.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc20d400640 1 --2- 172.21.15.42:0/1274846727 >> v2:172.21.15.42:3300/0 conn(0x7fc2180cf230 0x7fc218072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:31.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc215e00640 1 --2- 172.21.15.42:0/1274846727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc21815bf30 0x7fc21815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:31.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc20d400640 1 -- 172.21.15.42:0/1274846727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc21815bf30 msgr2=0x7fc21815c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:31.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc20d400640 1 --2- 172.21.15.42:0/1274846727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc21815bf30 0x7fc21815c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc20d400640 1 -- 172.21.15.42:0/1274846727 >> v1:172.21.15.42:6789/0 conn(0x7fc218072ba0 legacy=0x7fc21806ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:31.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc20d400640 1 -- 172.21.15.42:0/1274846727 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2180d01d0 con 0x7fc2180cf230 2024-05-28T14:04:31.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.779+0000 7fc215400640 1 -- 172.21.15.42:0/1274846727 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3530435868 0 0) 0x7fc2180cfc10 con 0x7fc218072ba0 2024-05-28T14:04:31.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.783+0000 7fc20d400640 1 --2- 172.21.15.42:0/1274846727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2180cf230 0x7fc218072660 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc2000097c0 tx=0x7fc20001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6d8867247984f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:31.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.783+0000 7fc215400640 1 -- 172.21.15.42:0/1274846727 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2000b2070 con 0x7fc2180cf230 2024-05-28T14:04:31.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.783+0000 7fc215400640 1 -- 172.21.15.42:0/1274846727 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2000ad640 con 0x7fc2180cf230 2024-05-28T14:04:31.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.783+0000 7fc215400640 1 -- 172.21.15.42:0/1274846727 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2000ada10 con 0x7fc2180cf230 2024-05-28T14:04:31.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.783+0000 7fc21d000640 1 -- 172.21.15.42:0/1274846727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2180cf230 msgr2=0x7fc218072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:31.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.783+0000 7fc21d000640 1 --2- 172.21.15.42:0/1274846727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2180cf230 0x7fc218072660 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc2000097c0 tx=0x7fc20001c470 comp rx=0 tx=0).stop 2024-05-28T14:04:31.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.783+0000 7fc21d000640 1 -- 172.21.15.42:0/1274846727 shutdown_connections 2024-05-28T14:04:31.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.783+0000 7fc21d000640 1 --2- 172.21.15.42:0/1274846727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2180cf230 0x7fc218072660 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.783+0000 7fc21d000640 1 --2- 172.21.15.42:0/1274846727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc21815bf30 0x7fc21815c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.783+0000 7fc21d000640 1 -- 172.21.15.42:0/1274846727 >> 172.21.15.42:0/1274846727 conn(0x7fc21806b9f0 msgr2=0x7fc21806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:31.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.783+0000 7fc21d000640 1 -- 172.21.15.42:0/1274846727 shutdown_connections 2024-05-28T14:04:31.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.783+0000 7fc21d000640 1 -- 172.21.15.42:0/1274846727 wait complete. 2024-05-28T14:04:31.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc21d000640 1 Processor -- start 2024-05-28T14:04:31.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc21d000640 1 -- start start 2024-05-28T14:04:31.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc21d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc218072ba0 0x7fc21814bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:31.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc21d000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2180cf230 0x7fc2180a9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:31.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc21d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc21815bf30 0x7fc2180a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:31.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc21d000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc21806f4e0 con 0x7fc21815bf30 2024-05-28T14:04:31.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc21d000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc21806f360 con 0x7fc218072ba0 2024-05-28T14:04:31.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc21d000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc2180cf690 con 0x7fc2180cf230 2024-05-28T14:04:31.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc216800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc21815bf30 0x7fc2180a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:31.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc216800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc21815bf30 0x7fc2180a9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32782/0 (socket says 172.21.15.42:32782) 2024-05-28T14:04:31.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc216800640 1 -- 172.21.15.42:0/1320447564 learned_addr learned my addr 172.21.15.42:0/1320447564 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:31.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc20d400640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2180cf230 0x7fc2180a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc215e00640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc218072ba0 0x7fc21814bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc216800640 1 -- 172.21.15.42:0/1320447564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2180cf230 msgr2=0x7fc2180a9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc216800640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2180cf230 0x7fc2180a9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc216800640 1 -- 172.21.15.42:0/1320447564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc218072ba0 msgr2=0x7fc21814bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc216800640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc218072ba0 0x7fc21814bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc216800640 1 -- 172.21.15.42:0/1320447564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2180afb50 con 0x7fc21815bf30 2024-05-28T14:04:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc20d400640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2180cf230 0x7fc2180a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:04:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc216800640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc21815bf30 0x7fc2180a9e80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc21000c750 tx=0x7fc21000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc20ea00640 1 -- 172.21.15.42:0/1320447564 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210018070 con 0x7fc21815bf30 2024-05-28T14:04:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc20ea00640 1 -- 172.21.15.42:0/1320447564 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21000a640 con 0x7fc21815bf30 2024-05-28T14:04:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc20ea00640 1 -- 172.21.15.42:0/1320447564 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21000ce60 con 0x7fc21815bf30 2024-05-28T14:04:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc21d000640 1 -- 172.21.15.42:0/1320447564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2180b1940 con 0x7fc21815bf30 2024-05-28T14:04:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.787+0000 7fc21d000640 1 -- 172.21.15.42:0/1320447564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2180b1f00 con 0x7fc21815bf30 2024-05-28T14:04:31.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.791+0000 7fc21d000640 1 -- 172.21.15.42:0/1320447564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc218073890 con 0x7fc21815bf30 2024-05-28T14:04:31.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.791+0000 7fc20ea00640 1 -- 172.21.15.42:0/1320447564 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fc21001b070 con 0x7fc21815bf30 2024-05-28T14:04:31.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.791+0000 7fc20ea00640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc1f4068b90 0x7fc1f406b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:31.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.791+0000 7fc20ea00640 1 -- 172.21.15.42:0/1320447564 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fc210087100 con 0x7fc21815bf30 2024-05-28T14:04:31.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.791+0000 7fc215e00640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc1f4068b90 0x7fc1f406b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:31.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.791+0000 7fc215e00640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc1f4068b90 0x7fc1f406b050 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc2080079c0 tx=0x7fc208008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:31.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.795+0000 7fc20ea00640 1 -- 172.21.15.42:0/1320447564 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc210056070 con 0x7fc21815bf30 2024-05-28T14:04:31.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.983+0000 7fc21d000640 1 -- 172.21.15.42:0/1320447564 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fc2180b4f00 con 0x7fc1f4068b90 2024-05-28T14:04:31.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.983+0000 7fc20ea00640 1 -- 172.21.15.42:0/1320447564 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fc2180b4f00 con 0x7fc1f4068b90 2024-05-28T14:04:31.970 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:04:31.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:04:31.970 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:04:31.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:04:31.971 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:04:31.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.987+0000 7fc21d000640 1 -- 172.21.15.42:0/1320447564 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc1f4068b90 msgr2=0x7fc1f406b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:31.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.987+0000 7fc21d000640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc1f4068b90 0x7fc1f406b050 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc2080079c0 tx=0x7fc208008040 comp rx=0 tx=0).stop 2024-05-28T14:04:31.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.987+0000 7fc21d000640 1 -- 172.21.15.42:0/1320447564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc21815bf30 msgr2=0x7fc2180a9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:31.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.987+0000 7fc21d000640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc21815bf30 0x7fc2180a9e80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc21000c750 tx=0x7fc21000cc10 comp rx=0 tx=0).stop 2024-05-28T14:04:31.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.987+0000 7fc21d000640 1 -- 172.21.15.42:0/1320447564 shutdown_connections 2024-05-28T14:04:31.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.987+0000 7fc21d000640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc1f4068b90 0x7fc1f406b050 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.987+0000 7fc21d000640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc21815bf30 0x7fc2180a9e80 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.987+0000 7fc21d000640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2180cf230 0x7fc2180a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.987+0000 7fc21d000640 1 --2- 172.21.15.42:0/1320447564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc218072ba0 0x7fc21814bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:31.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.987+0000 7fc21d000640 1 -- 172.21.15.42:0/1320447564 >> 172.21.15.42:0/1320447564 conn(0x7fc21806b9f0 msgr2=0x7fc21806bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:31.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.987+0000 7fc21d000640 1 -- 172.21.15.42:0/1320447564 shutdown_connections 2024-05-28T14:04:31.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:31.987+0000 7fc21d000640 1 -- 172.21.15.42:0/1320447564 wait complete. 2024-05-28T14:04:31.993 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T14:04:32.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5698200640 1 Processor -- start 2024-05-28T14:04:32.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5698200640 1 -- start start 2024-05-28T14:04:32.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5698200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56900b3770 0x7f569006b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5698200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f56900b3bd0 0x7f569006bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5698200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56900d06e0 con 0x7f569015c800 2024-05-28T14:04:32.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5698200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f56900d1200 con 0x7f56900b3bd0 2024-05-28T14:04:32.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5698200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f56900d0960 con 0x7f56900b3770 2024-05-28T14:04:32.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5696200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f56900b3bd0 0x7f569006bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5695800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f569015c800 0x7f56900cbaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51276/0 (socket says 172.21.15.42:51276) 2024-05-28T14:04:32.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5695800640 1 -- 172.21.15.42:0/131365554 learned_addr learned my addr 172.21.15.42:0/131365554 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:32.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5694e00640 1 --2- 172.21.15.42:0/131365554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56900b3770 0x7f569006b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5696200640 1 -- 172.21.15.42:0/131365554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56900b3770 msgr2=0x7f569006b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:32.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5696200640 1 --2- 172.21.15.42:0/131365554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56900b3770 0x7f569006b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5696200640 1 -- 172.21.15.42:0/131365554 >> v1:172.21.15.42:6789/0 conn(0x7f569015c800 legacy=0x7f56900cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.151+0000 7f5696200640 1 -- 172.21.15.42:0/131365554 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56900d0c40 con 0x7f56900b3bd0 2024-05-28T14:04:32.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.155+0000 7f5696200640 1 --2- 172.21.15.42:0/131365554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56900b3bd0 0x7f569006bf30 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5688009d90 tx=0x7f568801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868ff24e0e9c4a1e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:32.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.155+0000 7f568fe00640 1 -- 172.21.15.42:0/131365554 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56880b2070 con 0x7f56900b3bd0 2024-05-28T14:04:32.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.155+0000 7f568fe00640 1 -- 172.21.15.42:0/131365554 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56880a5970 con 0x7f56900b3bd0 2024-05-28T14:04:32.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.155+0000 7f568fe00640 1 -- 172.21.15.42:0/131365554 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56880a5d40 con 0x7f56900b3bd0 2024-05-28T14:04:32.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.155+0000 7f5698200640 1 -- 172.21.15.42:0/131365554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56900b3bd0 msgr2=0x7f569006bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.155+0000 7f5698200640 1 --2- 172.21.15.42:0/131365554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56900b3bd0 0x7f569006bf30 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5688009d90 tx=0x7f568801c8b0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.155+0000 7f5698200640 1 -- 172.21.15.42:0/131365554 shutdown_connections 2024-05-28T14:04:32.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.155+0000 7f5698200640 1 --2- 172.21.15.42:0/131365554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56900b3bd0 0x7f569006bf30 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.155+0000 7f5698200640 1 --2- 172.21.15.42:0/131365554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56900b3770 0x7f569006b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.155+0000 7f5698200640 1 -- 172.21.15.42:0/131365554 >> 172.21.15.42:0/131365554 conn(0x7f569009aa50 msgr2=0x7f56900b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:32.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.155+0000 7f5698200640 1 -- 172.21.15.42:0/131365554 shutdown_connections 2024-05-28T14:04:32.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.155+0000 7f5698200640 1 -- 172.21.15.42:0/131365554 wait complete. 2024-05-28T14:04:32.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5698200640 1 Processor -- start 2024-05-28T14:04:32.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5698200640 1 -- start start 2024-05-28T14:04:32.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5698200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56900b3770 0x7f569015ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5698200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56900b3bd0 0x7f569014f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5698200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f569015c800 0x7f5690159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5698200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f56900b2310 con 0x7f56900b3770 2024-05-28T14:04:32.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5698200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56900b2190 con 0x7f569015c800 2024-05-28T14:04:32.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5698200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f56900ce900 con 0x7f56900b3bd0 2024-05-28T14:04:32.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5695800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56900b3770 0x7f569015ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5695800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56900b3770 0x7f569015ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32808/0 (socket says 172.21.15.42:32808) 2024-05-28T14:04:32.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5695800640 1 -- 172.21.15.42:0/2527640934 learned_addr learned my addr 172.21.15.42:0/2527640934 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:32.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5694e00640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56900b3bd0 0x7f569014f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5695800640 1 -- 172.21.15.42:0/2527640934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56900b3bd0 msgr2=0x7f569014f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5695800640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56900b3bd0 0x7f569014f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5695800640 1 -- 172.21.15.42:0/2527640934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f569015c800 msgr2=0x7f5690159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5696200640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f569015c800 0x7f5690159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5695800640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f569015c800 0x7f5690159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5695800640 1 -- 172.21.15.42:0/2527640934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569015a960 con 0x7f56900b3770 2024-05-28T14:04:32.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5694e00640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56900b3bd0 0x7f569014f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:32.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5696200640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f569015c800 0x7f5690159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:04:32.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5695800640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56900b3770 0x7f569015ba00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f568400c370 tx=0x7f568400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:32.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f568d600640 1 -- 172.21.15.42:0/2527640934 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684017070 con 0x7f56900b3770 2024-05-28T14:04:32.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f568d600640 1 -- 172.21.15.42:0/2527640934 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5684005cf0 con 0x7f56900b3770 2024-05-28T14:04:32.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f568d600640 1 -- 172.21.15.42:0/2527640934 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56840072f0 con 0x7f56900b3770 2024-05-28T14:04:32.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5698200640 1 -- 172.21.15.42:0/2527640934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569015ac40 con 0x7f56900b3770 2024-05-28T14:04:32.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.159+0000 7f5698200640 1 -- 172.21.15.42:0/2527640934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f569015b200 con 0x7f56900b3770 2024-05-28T14:04:32.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.163+0000 7f5698200640 1 -- 172.21.15.42:0/2527640934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569015b630 con 0x7f56900b3770 2024-05-28T14:04:32.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.163+0000 7f568d600640 1 -- 172.21.15.42:0/2527640934 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f568401a070 con 0x7f56900b3770 2024-05-28T14:04:32.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.163+0000 7f568d600640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5670068b60 0x7f567006b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.163+0000 7f568d600640 1 -- 172.21.15.42:0/2527640934 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f5684086f30 con 0x7f56900b3770 2024-05-28T14:04:32.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.163+0000 7f5694e00640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5670068b60 0x7f567006b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.163+0000 7f5694e00640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5670068b60 0x7f567006b020 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f56900c84b0 tx=0x7f567c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:32.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.167+0000 7f568d600640 1 -- 172.21.15.42:0/2527640934 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f568408b070 con 0x7f56900b3770 2024-05-28T14:04:32.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.331+0000 7f5698200640 1 -- 172.21.15.42:0/2527640934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f569009f650 con 0x7f56900b3770 2024-05-28T14:04:32.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.331+0000 7f568d600640 1 -- 172.21.15.42:0/2527640934 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f56900d06e0 con 0x7f56900b3770 2024-05-28T14:04:32.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.335+0000 7f5698200640 1 -- 172.21.15.42:0/2527640934 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5670068b60 msgr2=0x7f567006b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.335+0000 7f5698200640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5670068b60 0x7f567006b020 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f56900c84b0 tx=0x7f567c008040 comp rx=0 tx=0).stop 2024-05-28T14:04:32.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.335+0000 7f5698200640 1 -- 172.21.15.42:0/2527640934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56900b3770 msgr2=0x7f569015ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.335+0000 7f5698200640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56900b3770 0x7f569015ba00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f568400c370 tx=0x7f568400c830 comp rx=0 tx=0).stop 2024-05-28T14:04:32.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.335+0000 7f5698200640 1 -- 172.21.15.42:0/2527640934 shutdown_connections 2024-05-28T14:04:32.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.335+0000 7f5698200640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5670068b60 0x7f567006b020 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.335+0000 7f5698200640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f569015c800 0x7f5690159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.335+0000 7f5698200640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56900b3bd0 0x7f569014f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.335+0000 7f5698200640 1 --2- 172.21.15.42:0/2527640934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56900b3770 0x7f569015ba00 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.335+0000 7f5698200640 1 -- 172.21.15.42:0/2527640934 >> 172.21.15.42:0/2527640934 conn(0x7f569009aa50 msgr2=0x7f56900703c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:32.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.335+0000 7f5698200640 1 -- 172.21.15.42:0/2527640934 shutdown_connections 2024-05-28T14:04:32.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.335+0000 7f5698200640 1 -- 172.21.15.42:0/2527640934 wait complete. 2024-05-28T14:04:32.341 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-05-28T14:04:32.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.499+0000 7fad44200640 1 Processor -- start 2024-05-28T14:04:32.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.499+0000 7fad44200640 1 -- start start 2024-05-28T14:04:32.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.499+0000 7fad44200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fad3c0b2240 0x7fad3c06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.499+0000 7fad44200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad3c0b4440 con 0x7fad3c06bf30 2024-05-28T14:04:32.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.499+0000 7fad44200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad3c0b0f10 con 0x7fad3c15e8c0 2024-05-28T14:04:32.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.499+0000 7fad44200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fad3c0ae420 con 0x7fad3c0b2240 2024-05-28T14:04:32.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.499+0000 7fad42200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fad3c06bf30 0x7fad3c0af150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51280/0 (socket says 172.21.15.42:51280) 2024-05-28T14:04:32.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.499+0000 7fad42200640 1 -- 172.21.15.42:0/2086867326 learned_addr learned my addr 172.21.15.42:0/2086867326 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:32.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.499+0000 7fad40e00640 1 --2- 172.21.15.42:0/2086867326 >> v2:172.21.15.42:3300/0 conn(0x7fad3c0b2240 0x7fad3c06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.499+0000 7fad40e00640 1 -- 172.21.15.42:0/2086867326 >> v1:172.21.15.63:6789/0 conn(0x7fad3c15e8c0 legacy=0x7fad3c0b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.499+0000 7fad40e00640 1 -- 172.21.15.42:0/2086867326 >> v1:172.21.15.42:6789/0 conn(0x7fad3c06bf30 legacy=0x7fad3c0af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.499+0000 7fad40e00640 1 -- 172.21.15.42:0/2086867326 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c0ae670 con 0x7fad3c0b2240 2024-05-28T14:04:32.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.503+0000 7fad40e00640 1 --2- 172.21.15.42:0/2086867326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3c0b2240 0x7fad3c06b9f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fad280097c0 tx=0x7fad2801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e7f21459ef4142 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.503+0000 7fad3be00640 1 -- 172.21.15.42:0/2086867326 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad280b1070 con 0x7fad3c0b2240 2024-05-28T14:04:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.503+0000 7fad3be00640 1 -- 172.21.15.42:0/2086867326 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad280ac440 con 0x7fad3c0b2240 2024-05-28T14:04:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.503+0000 7fad3be00640 1 -- 172.21.15.42:0/2086867326 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad280ac810 con 0x7fad3c0b2240 2024-05-28T14:04:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.503+0000 7fad44200640 1 -- 172.21.15.42:0/2086867326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3c0b2240 msgr2=0x7fad3c06b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.503+0000 7fad44200640 1 --2- 172.21.15.42:0/2086867326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3c0b2240 0x7fad3c06b9f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fad280097c0 tx=0x7fad2801c410 comp rx=0 tx=0).stop 2024-05-28T14:04:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.503+0000 7fad44200640 1 -- 172.21.15.42:0/2086867326 shutdown_connections 2024-05-28T14:04:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.503+0000 7fad44200640 1 --2- 172.21.15.42:0/2086867326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3c0b2240 0x7fad3c06b9f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.503+0000 7fad44200640 1 -- 172.21.15.42:0/2086867326 >> 172.21.15.42:0/2086867326 conn(0x7fad3c0b6960 msgr2=0x7fad3c0b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.503+0000 7fad44200640 1 -- 172.21.15.42:0/2086867326 shutdown_connections 2024-05-28T14:04:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.503+0000 7fad44200640 1 -- 172.21.15.42:0/2086867326 wait complete. 2024-05-28T14:04:32.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad44200640 1 Processor -- start 2024-05-28T14:04:32.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad44200640 1 -- start start 2024-05-28T14:04:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad44200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad3c06bf30 0x7fad3c159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad44200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad3c0b2240 0x7fad3c154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad44200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3c15e8c0 0x7fad3c154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad44200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c15ee20 con 0x7fad3c15e8c0 2024-05-28T14:04:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad44200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c15eca0 con 0x7fad3c06bf30 2024-05-28T14:04:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad44200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fad3c16c530 con 0x7fad3c0b2240 2024-05-28T14:04:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad42200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3c15e8c0 0x7fad3c154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad42200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3c15e8c0 0x7fad3c154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32832/0 (socket says 172.21.15.42:32832) 2024-05-28T14:04:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad42200640 1 -- 172.21.15.42:0/3564821618 learned_addr learned my addr 172.21.15.42:0/3564821618 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad41800640 1 --2- 172.21.15.42:0/3564821618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad3c06bf30 0x7fad3c159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad40e00640 1 --2- 172.21.15.42:0/3564821618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad3c0b2240 0x7fad3c154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad42200640 1 -- 172.21.15.42:0/3564821618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad3c0b2240 msgr2=0x7fad3c154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad42200640 1 --2- 172.21.15.42:0/3564821618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad3c0b2240 0x7fad3c154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad42200640 1 -- 172.21.15.42:0/3564821618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad3c06bf30 msgr2=0x7fad3c159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad42200640 1 --2- 172.21.15.42:0/3564821618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad3c06bf30 0x7fad3c159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad42200640 1 -- 172.21.15.42:0/3564821618 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c14def0 con 0x7fad3c15e8c0 2024-05-28T14:04:32.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad42200640 1 --2- 172.21.15.42:0/3564821618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3c15e8c0 0x7fad3c154cc0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fad3400cbc0 tx=0x7fad3400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:32.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad39600640 1 -- 172.21.15.42:0/3564821618 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34017070 con 0x7fad3c15e8c0 2024-05-28T14:04:32.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad39600640 1 -- 172.21.15.42:0/3564821618 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad3400ad40 con 0x7fad3c15e8c0 2024-05-28T14:04:32.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad39600640 1 -- 172.21.15.42:0/3564821618 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34012600 con 0x7fad3c15e8c0 2024-05-28T14:04:32.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad44200640 1 -- 172.21.15.42:0/3564821618 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3c14e1d0 con 0x7fad3c15e8c0 2024-05-28T14:04:32.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.507+0000 7fad44200640 1 -- 172.21.15.42:0/3564821618 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad3c14e790 con 0x7fad3c15e8c0 2024-05-28T14:04:32.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.511+0000 7fad44200640 1 -- 172.21.15.42:0/3564821618 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad3c09f7a0 con 0x7fad3c15e8c0 2024-05-28T14:04:32.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.511+0000 7fad39600640 1 -- 172.21.15.42:0/3564821618 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fad3401a070 con 0x7fad3c15e8c0 2024-05-28T14:04:32.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.511+0000 7fad39600640 1 --2- 172.21.15.42:0/3564821618 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fad1c068b60 0x7fad1c06b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.511+0000 7fad39600640 1 -- 172.21.15.42:0/3564821618 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fad34087880 con 0x7fad3c15e8c0 2024-05-28T14:04:32.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.511+0000 7fad41800640 1 --2- 172.21.15.42:0/3564821618 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fad1c068b60 0x7fad1c06b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.511+0000 7fad41800640 1 --2- 172.21.15.42:0/3564821618 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fad1c068b60 0x7fad1c06b020 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0b4690 tx=0x7fad30009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:32.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.515+0000 7fad39600640 1 -- 172.21.15.42:0/3564821618 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fad3c0b4440 con 0x7fad3c15e8c0 2024-05-28T14:04:32.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.683+0000 7fad44200640 1 -- 172.21.15.42:0/3564821618 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7fad3c15bb80 con 0x7fad3c15e8c0 2024-05-28T14:04:32.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.687+0000 7fad39600640 1 -- 172.21.15.42:0/3564821618 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v15)=0 v15) ==== 136+0+0 (secure 0 0 0) 0x7fad34056840 con 0x7fad3c15e8c0 2024-05-28T14:04:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.687+0000 7fad44200640 1 -- 172.21.15.42:0/3564821618 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fad1c068b60 msgr2=0x7fad1c06b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.687+0000 7fad44200640 1 --2- 172.21.15.42:0/3564821618 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fad1c068b60 0x7fad1c06b020 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0b4690 tx=0x7fad30009040 comp rx=0 tx=0).stop 2024-05-28T14:04:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.687+0000 7fad44200640 1 -- 172.21.15.42:0/3564821618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3c15e8c0 msgr2=0x7fad3c154cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.687+0000 7fad44200640 1 --2- 172.21.15.42:0/3564821618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3c15e8c0 0x7fad3c154cc0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fad3400cbc0 tx=0x7fad3400a430 comp rx=0 tx=0).stop 2024-05-28T14:04:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.687+0000 7fad44200640 1 -- 172.21.15.42:0/3564821618 shutdown_connections 2024-05-28T14:04:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.687+0000 7fad44200640 1 --2- 172.21.15.42:0/3564821618 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fad1c068b60 0x7fad1c06b020 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.687+0000 7fad44200640 1 --2- 172.21.15.42:0/3564821618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3c15e8c0 0x7fad3c154cc0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.687+0000 7fad44200640 1 --2- 172.21.15.42:0/3564821618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad3c0b2240 0x7fad3c154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.687+0000 7fad44200640 1 --2- 172.21.15.42:0/3564821618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad3c06bf30 0x7fad3c159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.687+0000 7fad44200640 1 -- 172.21.15.42:0/3564821618 >> 172.21.15.42:0/3564821618 conn(0x7fad3c0b6960 msgr2=0x7fad3c06f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:32.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.687+0000 7fad44200640 1 -- 172.21.15.42:0/3564821618 shutdown_connections 2024-05-28T14:04:32.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.687+0000 7fad44200640 1 -- 172.21.15.42:0/3564821618 wait complete. 2024-05-28T14:04:32.689 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:32.703+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:32.689 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:32.703+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:32.689 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:32.703+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:32.689 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:32.703+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:32.689 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:32.703+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:32.696 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-28T14:04:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf8c00640 1 Processor -- start 2024-05-28T14:04:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf8c00640 1 -- start start 2024-05-28T14:04:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf8c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1cf406e740 0x7f1cf40cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf8c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cf40ce110 0x7f1cf409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf8c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1cf406c940 0x7f1cf409af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf8c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1cf406eb20 con 0x7f1cf406c940 2024-05-28T14:04:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf8c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1cf406ff10 con 0x7f1cf406e740 2024-05-28T14:04:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf8c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf40701b0 con 0x7f1cf40ce110 2024-05-28T14:04:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf2800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1cf406c940 0x7f1cf409af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf2800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1cf406c940 0x7f1cf409af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32858/0 (socket says 172.21.15.42:32858) 2024-05-28T14:04:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf2800640 1 -- 172.21.15.42:0/330529336 learned_addr learned my addr 172.21.15.42:0/330529336 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf1e00640 1 --2- 172.21.15.42:0/330529336 >> v2:172.21.15.63:3300/0 conn(0x7f1cf406e740 0x7f1cf40cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1ce9400640 1 --2- 172.21.15.42:0/330529336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cf40ce110 0x7f1cf409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf2800640 1 -- 172.21.15.42:0/330529336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cf40ce110 msgr2=0x7f1cf409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf2800640 1 --2- 172.21.15.42:0/330529336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cf40ce110 0x7f1cf409aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf2800640 1 -- 172.21.15.42:0/330529336 >> v2:172.21.15.63:3300/0 conn(0x7f1cf406e740 msgr2=0x7f1cf40cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf2800640 1 --2- 172.21.15.42:0/330529336 >> v2:172.21.15.63:3300/0 conn(0x7f1cf406e740 0x7f1cf40cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.855+0000 7f1cf2800640 1 -- 172.21.15.42:0/330529336 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf4070490 con 0x7f1cf406c940 2024-05-28T14:04:32.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.859+0000 7f1cf2800640 1 --2- 172.21.15.42:0/330529336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cf406c940 0x7f1cf409af90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1cf406f990 tx=0x7f1cec01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edc65f733e58e8a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:32.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.859+0000 7f1cf1400640 1 -- 172.21.15.42:0/330529336 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0a5840 con 0x7f1cf406c940 2024-05-28T14:04:32.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.859+0000 7f1cf1400640 1 -- 172.21.15.42:0/330529336 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cec0a59e0 con 0x7f1cf406c940 2024-05-28T14:04:32.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.859+0000 7f1cf1400640 1 -- 172.21.15.42:0/330529336 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0a5db0 con 0x7f1cf406c940 2024-05-28T14:04:32.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.859+0000 7f1cf8c00640 1 -- 172.21.15.42:0/330529336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cf406c940 msgr2=0x7f1cf409af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.859+0000 7f1cf8c00640 1 --2- 172.21.15.42:0/330529336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cf406c940 0x7f1cf409af90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1cf406f990 tx=0x7f1cec01c9c0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.859+0000 7f1cf8c00640 1 -- 172.21.15.42:0/330529336 shutdown_connections 2024-05-28T14:04:32.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.859+0000 7f1cf8c00640 1 --2- 172.21.15.42:0/330529336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cf406c940 0x7f1cf409af90 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.859+0000 7f1cf8c00640 1 --2- 172.21.15.42:0/330529336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cf40ce110 0x7f1cf409aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.859+0000 7f1cf8c00640 1 --2- 172.21.15.42:0/330529336 >> v2:172.21.15.63:3300/0 conn(0x7f1cf406e740 0x7f1cf40cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.859+0000 7f1cf8c00640 1 -- 172.21.15.42:0/330529336 >> 172.21.15.42:0/330529336 conn(0x7f1cf40b3fa0 msgr2=0x7f1cf40b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:32.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.859+0000 7f1cf8c00640 1 -- 172.21.15.42:0/330529336 shutdown_connections 2024-05-28T14:04:32.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.859+0000 7f1cf8c00640 1 -- 172.21.15.42:0/330529336 wait complete. 2024-05-28T14:04:32.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf8c00640 1 Processor -- start 2024-05-28T14:04:32.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf8c00640 1 -- start start 2024-05-28T14:04:32.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf8c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cf406c940 0x7f1cf415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cf406e740 0x7f1cf40c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cf40ce110 0x7f1cf415bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf8c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4071900 con 0x7f1cf406e740 2024-05-28T14:04:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf8c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4071780 con 0x7f1cf40ce110 2024-05-28T14:04:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf8c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf4071a80 con 0x7f1cf406c940 2024-05-28T14:04:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1ce9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cf406e740 0x7f1cf40c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cf40ce110 0x7f1cf415bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf1e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cf406c940 0x7f1cf415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1ce9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cf406e740 0x7f1cf40c74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32866/0 (socket says 172.21.15.42:32866) 2024-05-28T14:04:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cf40ce110 0x7f1cf415bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:53614/0 (socket says 172.21.15.42:53614) 2024-05-28T14:04:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1ce9400640 1 -- 172.21.15.42:0/1385478678 learned_addr learned my addr 172.21.15.42:0/1385478678 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf2800640 1 -- 172.21.15.42:0/1385478678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cf406c940 msgr2=0x7f1cf415c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf2800640 1 --2- 172.21.15.42:0/1385478678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cf406c940 0x7f1cf415c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf2800640 1 -- 172.21.15.42:0/1385478678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cf406e740 msgr2=0x7f1cf40c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf2800640 1 --2- 172.21.15.42:0/1385478678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cf406e740 0x7f1cf40c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf2800640 1 -- 172.21.15.42:0/1385478678 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf415cc60 con 0x7f1cf40ce110 2024-05-28T14:04:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1ce9400640 1 --2- 172.21.15.42:0/1385478678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cf406e740 0x7f1cf40c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:04:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf2800640 1 --2- 172.21.15.42:0/1385478678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cf40ce110 0x7f1cf415bb00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0ab000 tx=0x7f1cec019760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1ceaa00640 1 -- 172.21.15.42:0/1385478678 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0bc020 con 0x7f1cf40ce110 2024-05-28T14:04:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1ceaa00640 1 -- 172.21.15.42:0/1385478678 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cec0b8670 con 0x7f1cf40ce110 2024-05-28T14:04:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1ceaa00640 1 -- 172.21.15.42:0/1385478678 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0b8a60 con 0x7f1cf40ce110 2024-05-28T14:04:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf8c00640 1 -- 172.21.15.42:0/1385478678 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf415cf40 con 0x7f1cf40ce110 2024-05-28T14:04:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.863+0000 7f1cf8c00640 1 -- 172.21.15.42:0/1385478678 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf415b5e0 con 0x7f1cf40ce110 2024-05-28T14:04:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.867+0000 7f1cf8c00640 1 -- 172.21.15.42:0/1385478678 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf4068a90 con 0x7f1cf40ce110 2024-05-28T14:04:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.867+0000 7f1ceaa00640 1 -- 172.21.15.42:0/1385478678 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7f1cec0bf020 con 0x7f1cf40ce110 2024-05-28T14:04:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.867+0000 7f1ceaa00640 1 --2- 172.21.15.42:0/1385478678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1cd0068b60 0x7f1cd006b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.867+0000 7f1ceaa00640 1 -- 172.21.15.42:0/1385478678 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f1cec122c30 con 0x7f1cf40ce110 2024-05-28T14:04:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.867+0000 7f1cf1e00640 1 --2- 172.21.15.42:0/1385478678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1cd0068b60 0x7f1cd006b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:32.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.867+0000 7f1cf1e00640 1 --2- 172.21.15.42:0/1385478678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1cd0068b60 0x7f1cd006b020 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40b12b0 tx=0x7f1ce4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:32.871+0000 7f1ceaa00640 1 -- 172.21.15.42:0/1385478678 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1cec0c6030 con 0x7f1cf40ce110 2024-05-28T14:04:33.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.063+0000 7f1cf8c00640 1 -- 172.21.15.42:0/1385478678 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f1cf415adb0 con 0x7f1cd0068b60 2024-05-28T14:04:33.057 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:33.071+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:33.057 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:33.071+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:33.057 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:33.071+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:33.057 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:33.071+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:33.057 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:33.071+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:33.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.103+0000 7f1ceaa00640 1 -- 172.21.15.42:0/1385478678 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1cf415adb0 con 0x7f1cd0068b60 2024-05-28T14:04:33.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.103+0000 7f1cf8c00640 1 -- 172.21.15.42:0/1385478678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1cd0068b60 msgr2=0x7f1cd006b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:33.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.103+0000 7f1cf8c00640 1 --2- 172.21.15.42:0/1385478678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1cd0068b60 0x7f1cd006b020 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40b12b0 tx=0x7f1ce4008040 comp rx=0 tx=0).stop 2024-05-28T14:04:33.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.103+0000 7f1cf8c00640 1 -- 172.21.15.42:0/1385478678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cf40ce110 msgr2=0x7f1cf415bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:33.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.103+0000 7f1cf8c00640 1 --2- 172.21.15.42:0/1385478678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cf40ce110 0x7f1cf415bb00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0ab000 tx=0x7f1cec019760 comp rx=0 tx=0).stop 2024-05-28T14:04:33.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.103+0000 7f1cf8c00640 1 -- 172.21.15.42:0/1385478678 shutdown_connections 2024-05-28T14:04:33.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.103+0000 7f1cf8c00640 1 --2- 172.21.15.42:0/1385478678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1cd0068b60 0x7f1cd006b020 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:33.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.103+0000 7f1cf8c00640 1 --2- 172.21.15.42:0/1385478678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cf40ce110 0x7f1cf415bb00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:33.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.103+0000 7f1cf8c00640 1 --2- 172.21.15.42:0/1385478678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cf406e740 0x7f1cf40c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:33.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.103+0000 7f1cf8c00640 1 --2- 172.21.15.42:0/1385478678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cf406c940 0x7f1cf415c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:33.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.103+0000 7f1cf8c00640 1 -- 172.21.15.42:0/1385478678 >> 172.21.15.42:0/1385478678 conn(0x7f1cf40b3fa0 msgr2=0x7f1cf40cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:33.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.103+0000 7f1cf8c00640 1 -- 172.21.15.42:0/1385478678 shutdown_connections 2024-05-28T14:04:33.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.103+0000 7f1cf8c00640 1 -- 172.21.15.42:0/1385478678 wait complete. 2024-05-28T14:04:33.111 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:04:33.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2be600640 1 Processor -- start 2024-05-28T14:04:33.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2be600640 1 -- start start 2024-05-28T14:04:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2be600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb2b8071de0 0x7fb2b80721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2be600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2b8072700 con 0x7fb2b80ce3b0 2024-05-28T14:04:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2be600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2b8160230 con 0x7fb2b813fd10 2024-05-28T14:04:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2be600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb2b8160510 con 0x7fb2b8071de0 2024-05-28T14:04:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2afe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb2b80ce3b0 0x7fb2b80ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51284/0 (socket says 172.21.15.42:51284) 2024-05-28T14:04:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2afe00640 1 -- 172.21.15.42:0/1399227184 learned_addr learned my addr 172.21.15.42:0/1399227184 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:33.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b7400640 1 --2- 172.21.15.42:0/1399227184 >> v2:172.21.15.63:3300/0 conn(0x7fb2b8071de0 0x7fb2b80721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:33.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 -- 172.21.15.42:0/1399227184 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2335603458 0 0) 0x7fb2b8072700 con 0x7fb2b80ce3b0 2024-05-28T14:04:33.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 -- 172.21.15.42:0/1399227184 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb29c003530 con 0x7fb2b80ce3b0 2024-05-28T14:04:33.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 -- 172.21.15.42:0/1399227184 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2789079480 0 0) 0x7fb29c003530 con 0x7fb2b80ce3b0 2024-05-28T14:04:33.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 -- 172.21.15.42:0/1399227184 >> v2:172.21.15.63:3300/0 conn(0x7fb2b8071de0 msgr2=0x7fb2b80721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:33.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 --2- 172.21.15.42:0/1399227184 >> v2:172.21.15.63:3300/0 conn(0x7fb2b8071de0 0x7fb2b80721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:33.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 -- 172.21.15.42:0/1399227184 >> v1:172.21.15.63:6789/0 conn(0x7fb2b813fd10 legacy=0x7fb2b8140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 -- 172.21.15.42:0/1399227184 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b81607f0 con 0x7fb2b80ce3b0 2024-05-28T14:04:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 -- 172.21.15.42:0/1399227184 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fb2a0001680 con 0x7fb2b80ce3b0 2024-05-28T14:04:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 -- 172.21.15.42:0/1399227184 >> v1:172.21.15.42:6789/0 conn(0x7fb2b80ce3b0 legacy=0x7fb2b80ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 --2- 172.21.15.42:0/1399227184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb29c0039c0 0x7fb29c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 --2- 172.21.15.42:0/1399227184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb29c004810 0x7fb29c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 --2- 172.21.15.42:0/1399227184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb29c0052f0 0x7fb29c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 -- 172.21.15.42:0/1399227184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b8160510 con 0x7fb29c004810 2024-05-28T14:04:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 -- 172.21.15.42:0/1399227184 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c00c1c0 con 0x7fb29c0052f0 2024-05-28T14:04:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b6a00640 1 -- 172.21.15.42:0/1399227184 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb29c00c3c0 con 0x7fb29c0039c0 2024-05-28T14:04:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2afe00640 1 --2- 172.21.15.42:0/1399227184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb29c004810 0x7fb29c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b7e00640 1 --2- 172.21.15.42:0/1399227184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb29c0052f0 0x7fb29c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b7400640 1 --2- 172.21.15.42:0/1399227184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb29c0039c0 0x7fb29c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2afe00640 1 -- 172.21.15.42:0/1399227184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb29c0039c0 msgr2=0x7fb29c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2afe00640 1 --2- 172.21.15.42:0/1399227184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb29c0039c0 0x7fb29c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2afe00640 1 -- 172.21.15.42:0/1399227184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb29c0052f0 msgr2=0x7fb29c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2afe00640 1 --2- 172.21.15.42:0/1399227184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb29c0052f0 0x7fb29c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2afe00640 1 -- 172.21.15.42:0/1399227184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c003530 con 0x7fb29c004810 2024-05-28T14:04:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.271+0000 7fb2b7e00640 1 -- 172.21.15.42:0/1399227184 reap_dead start 2024-05-28T14:04:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.275+0000 7fb2afe00640 1 --2- 172.21.15.42:0/1399227184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb29c004810 0x7fb29c004c90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000c210 tx=0x7fb2a001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d670782289ffcdec server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:33.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.275+0000 7fb2b6a00640 1 -- 172.21.15.42:0/1399227184 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0007d20 con 0x7fb29c004810 2024-05-28T14:04:33.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.275+0000 7fb2b6a00640 1 -- 172.21.15.42:0/1399227184 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a00a9070 con 0x7fb29c004810 2024-05-28T14:04:33.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.275+0000 7fb2b6a00640 1 -- 172.21.15.42:0/1399227184 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0005d80 con 0x7fb29c004810 2024-05-28T14:04:33.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.275+0000 7fb2be600640 1 -- 172.21.15.42:0/1399227184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb29c004810 msgr2=0x7fb29c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:33.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.275+0000 7fb2be600640 1 --2- 172.21.15.42:0/1399227184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb29c004810 0x7fb29c004c90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000c210 tx=0x7fb2a001cc60 comp rx=0 tx=0).stop 2024-05-28T14:04:33.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.275+0000 7fb2be600640 1 -- 172.21.15.42:0/1399227184 shutdown_connections 2024-05-28T14:04:33.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.275+0000 7fb2be600640 1 --2- 172.21.15.42:0/1399227184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb29c004810 0x7fb29c004c90 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:33.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.275+0000 7fb2be600640 1 -- 172.21.15.42:0/1399227184 >> 172.21.15.42:0/1399227184 conn(0x7fb2b80b21f0 msgr2=0x7fb2b80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:33.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.275+0000 7fb2be600640 1 -- 172.21.15.42:0/1399227184 shutdown_connections 2024-05-28T14:04:33.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.275+0000 7fb2be600640 1 -- 172.21.15.42:0/1399227184 wait complete. 2024-05-28T14:04:33.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2be600640 1 Processor -- start 2024-05-28T14:04:33.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2be600640 1 -- start start 2024-05-28T14:04:33.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2be600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb29c004810 0x7fb2b815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:33.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2be600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2b815bda0 0x7fb2b81448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:33.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2be600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2b815f350 0x7fb2b815f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:33.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2be600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81420a0 con 0x7fb2b815bda0 2024-05-28T14:04:33.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2be600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b8141f20 con 0x7fb29c004810 2024-05-28T14:04:33.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2be600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b8142220 con 0x7fb2b815f350 2024-05-28T14:04:33.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2b815bda0 0x7fb2b81448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:33.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2b815bda0 0x7fb2b81448d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32906/0 (socket says 172.21.15.42:32906) 2024-05-28T14:04:33.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2afe00640 1 -- 172.21.15.42:0/1838867075 learned_addr learned my addr 172.21.15.42:0/1838867075 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:33.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2b7400640 1 --2- 172.21.15.42:0/1838867075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb29c004810 0x7fb2b815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:33.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2b7e00640 1 --2- 172.21.15.42:0/1838867075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2b815f350 0x7fb2b815f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:33.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2afe00640 1 -- 172.21.15.42:0/1838867075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2b815f350 msgr2=0x7fb2b815f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:33.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2afe00640 1 --2- 172.21.15.42:0/1838867075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2b815f350 0x7fb2b815f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:33.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2afe00640 1 -- 172.21.15.42:0/1838867075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb29c004810 msgr2=0x7fb2b815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:33.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2afe00640 1 --2- 172.21.15.42:0/1838867075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb29c004810 0x7fb2b815ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:33.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2afe00640 1 -- 172.21.15.42:0/1838867075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b815d010 con 0x7fb2b815bda0 2024-05-28T14:04:33.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2afe00640 1 --2- 172.21.15.42:0/1838867075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2b815bda0 0x7fb2b81448d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb2a001ca90 tx=0x7fb29c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:33.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2af400640 1 -- 172.21.15.42:0/1838867075 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0000f90 con 0x7fb2b815bda0 2024-05-28T14:04:33.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2af400640 1 -- 172.21.15.42:0/1838867075 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a0007180 con 0x7fb2b815bda0 2024-05-28T14:04:33.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2af400640 1 -- 172.21.15.42:0/1838867075 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0007550 con 0x7fb2b815bda0 2024-05-28T14:04:33.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2be600640 1 -- 172.21.15.42:0/1838867075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b815d2f0 con 0x7fb2b815bda0 2024-05-28T14:04:33.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.279+0000 7fb2be600640 1 -- 172.21.15.42:0/1838867075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b815d8b0 con 0x7fb2b815bda0 2024-05-28T14:04:33.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.283+0000 7fb2be600640 1 -- 172.21.15.42:0/1838867075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b80738b0 con 0x7fb2b815bda0 2024-05-28T14:04:33.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.283+0000 7fb2af400640 1 -- 172.21.15.42:0/1838867075 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 88701+0+0 (secure 0 0 0) 0x7fb2a00047f0 con 0x7fb2b815bda0 2024-05-28T14:04:33.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.283+0000 7fb2af400640 1 --2- 172.21.15.42:0/1838867075 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb294068b60 0x7fb29406b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:33.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.283+0000 7fb2af400640 1 -- 172.21.15.42:0/1838867075 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fb2a0120660 con 0x7fb2b815bda0 2024-05-28T14:04:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.283+0000 7fb2b7400640 1 --2- 172.21.15.42:0/1838867075 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb294068b60 0x7fb29406b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.283+0000 7fb2b7400640 1 --2- 172.21.15.42:0/1838867075 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb294068b60 0x7fb29406b020 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8051740 tx=0x7fb2a8009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:33.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.287+0000 7fb2af400640 1 -- 172.21.15.42:0/1838867075 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb2a00bf070 con 0x7fb2b815bda0 2024-05-28T14:04:33.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.479+0000 7fb2be600640 1 -- 172.21.15.42:0/1838867075 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fb2b804fdd0 con 0x7fb294068b60 2024-05-28T14:04:33.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.483+0000 7fb2af400640 1 -- 172.21.15.42:0/1838867075 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fb2b804fdd0 con 0x7fb294068b60 2024-05-28T14:04:33.470 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/19bb7fcc-5845-462e-83f1-114a41db1528 2024-05-28T14:04:33.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.487+0000 7fb2be600640 1 -- 172.21.15.42:0/1838867075 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb294068b60 msgr2=0x7fb29406b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:33.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.487+0000 7fb2be600640 1 --2- 172.21.15.42:0/1838867075 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb294068b60 0x7fb29406b020 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8051740 tx=0x7fb2a8009190 comp rx=0 tx=0).stop 2024-05-28T14:04:33.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.487+0000 7fb2be600640 1 -- 172.21.15.42:0/1838867075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2b815bda0 msgr2=0x7fb2b81448d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:33.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.487+0000 7fb2be600640 1 --2- 172.21.15.42:0/1838867075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2b815bda0 0x7fb2b81448d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb2a001ca90 tx=0x7fb29c005840 comp rx=0 tx=0).stop 2024-05-28T14:04:33.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.487+0000 7fb2be600640 1 -- 172.21.15.42:0/1838867075 shutdown_connections 2024-05-28T14:04:33.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.487+0000 7fb2be600640 1 --2- 172.21.15.42:0/1838867075 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb294068b60 0x7fb29406b020 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:33.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.487+0000 7fb2be600640 1 --2- 172.21.15.42:0/1838867075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2b815f350 0x7fb2b815f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:33.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.487+0000 7fb2be600640 1 --2- 172.21.15.42:0/1838867075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2b815bda0 0x7fb2b81448d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:33.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.487+0000 7fb2be600640 1 --2- 172.21.15.42:0/1838867075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb29c004810 0x7fb2b815ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:33.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.487+0000 7fb2be600640 1 -- 172.21.15.42:0/1838867075 >> 172.21.15.42:0/1838867075 conn(0x7fb2b80b21f0 msgr2=0x7fb2b80b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:33.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.487+0000 7fb2be600640 1 -- 172.21.15.42:0/1838867075 shutdown_connections 2024-05-28T14:04:33.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:33.487+0000 7fb2be600640 1 -- 172.21.15.42:0/1838867075 wait complete. 2024-05-28T14:04:33.491 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/19bb7fcc-5845-462e-83f1-114a41db1528 2024-05-28T14:04:33.491 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:04:33.492 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:04:33.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:33.501 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/19bb7fcc-5845-462e-83f1-114a41db1528/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:04:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:04:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:04:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.04059 s, 153 MB/s 2024-05-28T14:04:40.613 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:04:40.613 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:04:40.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:40.622 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/19bb7fcc-5845-462e-83f1-114a41db1528/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:04:47.512 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:04:47.512 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:04:47.513 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.83382 s, 157 MB/s 2024-05-28T14:04:47.514 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:04:47.514 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:04:47.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:47.523 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/19bb7fcc-5845-462e-83f1-114a41db1528/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:04:48.495 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:04:48.500 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:04:54.341 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:04:54.341 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:04:54.341 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.75748 s, 159 MB/s 2024-05-28T14:04:54.343 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-28T14:04:54.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9ac0400640 1 Processor -- start 2024-05-28T14:04:54.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9ac0400640 1 -- start start 2024-05-28T14:04:54.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9ac0400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ab80afa70 0x7f9ab806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:54.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9ac0400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9ab80d06f0 0x7f9ab80d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:54.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9ac0400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ab81570a0 con 0x7f9ab806ee80 2024-05-28T14:04:54.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9ac0400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab815cf00 con 0x7f9ab80d06f0 2024-05-28T14:04:54.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9ac0400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab815d1e0 con 0x7f9ab80afa70 2024-05-28T14:04:54.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9abe400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9ab80d06f0 0x7f9ab80d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:54.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9abda00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f9ab806ee80 0x7f9ab80af360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47022/0 (socket says 172.21.15.42:47022) 2024-05-28T14:04:54.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9abda00640 1 -- 172.21.15.42:0/206585325 learned_addr learned my addr 172.21.15.42:0/206585325 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:54.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9abd000640 1 --2- 172.21.15.42:0/206585325 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ab80afa70 0x7f9ab806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:54.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9abe400640 1 -- 172.21.15.42:0/206585325 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ab80afa70 msgr2=0x7f9ab806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:54.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9abe400640 1 --2- 172.21.15.42:0/206585325 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ab80afa70 0x7f9ab806dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9abe400640 1 -- 172.21.15.42:0/206585325 >> v1:172.21.15.42:6789/0 conn(0x7f9ab806ee80 legacy=0x7f9ab80af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:54.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9abe400640 1 -- 172.21.15.42:0/206585325 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab815d4c0 con 0x7f9ab80d06f0 2024-05-28T14:04:54.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.503+0000 7f9ab7e00640 1 -- 172.21.15.42:0/206585325 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 277857239 0 0) 0x7f9ab81570a0 con 0x7f9ab806ee80 2024-05-28T14:04:54.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.507+0000 7f9abe400640 1 --2- 172.21.15.42:0/206585325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab80d06f0 0x7f9ab80d0b80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00097c0 tx=0x7f9ab001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b4fdd5c436ee50 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:54.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.507+0000 7f9ab7e00640 1 -- 172.21.15.42:0/206585325 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00b0070 con 0x7f9ab80d06f0 2024-05-28T14:04:54.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.507+0000 7f9ab7e00640 1 -- 172.21.15.42:0/206585325 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab00ab600 con 0x7f9ab80d06f0 2024-05-28T14:04:54.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.507+0000 7f9ab7e00640 1 -- 172.21.15.42:0/206585325 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00a4e60 con 0x7f9ab80d06f0 2024-05-28T14:04:54.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.507+0000 7f9ac0400640 1 -- 172.21.15.42:0/206585325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab80d06f0 msgr2=0x7f9ab80d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:54.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.507+0000 7f9ac0400640 1 --2- 172.21.15.42:0/206585325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab80d06f0 0x7f9ab80d0b80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00097c0 tx=0x7f9ab001c380 comp rx=0 tx=0).stop 2024-05-28T14:04:54.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.507+0000 7f9ac0400640 1 -- 172.21.15.42:0/206585325 shutdown_connections 2024-05-28T14:04:54.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.507+0000 7f9ac0400640 1 --2- 172.21.15.42:0/206585325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab80d06f0 0x7f9ab80d0b80 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.507+0000 7f9ac0400640 1 --2- 172.21.15.42:0/206585325 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ab80afa70 0x7f9ab806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.507+0000 7f9ac0400640 1 -- 172.21.15.42:0/206585325 >> 172.21.15.42:0/206585325 conn(0x7f9ab806bdf0 msgr2=0x7f9ab806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:54.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.507+0000 7f9ac0400640 1 -- 172.21.15.42:0/206585325 shutdown_connections 2024-05-28T14:04:54.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.507+0000 7f9ac0400640 1 -- 172.21.15.42:0/206585325 wait complete. 2024-05-28T14:04:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9ac0400640 1 Processor -- start 2024-05-28T14:04:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9ac0400640 1 -- start start 2024-05-28T14:04:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9ac0400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ab806ee80 0x7f9ab8150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9ac0400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ab80afa70 0x7f9ab80ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9ac0400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab80d06f0 0x7f9ab8150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9ac0400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab8160e10 con 0x7f9ab80d06f0 2024-05-28T14:04:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9ac0400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab8160c90 con 0x7f9ab80afa70 2024-05-28T14:04:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9ac0400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab8160f90 con 0x7f9ab806ee80 2024-05-28T14:04:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9abe400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab80d06f0 0x7f9ab8150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9abe400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab80d06f0 0x7f9ab8150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34160/0 (socket says 172.21.15.42:34160) 2024-05-28T14:04:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9abe400640 1 -- 172.21.15.42:0/3272246942 learned_addr learned my addr 172.21.15.42:0/3272246942 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9abd000640 1 --2- 172.21.15.42:0/3272246942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ab80afa70 0x7f9ab80ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9abda00640 1 --2- 172.21.15.42:0/3272246942 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ab806ee80 0x7f9ab8150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9abe400640 1 -- 172.21.15.42:0/3272246942 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ab806ee80 msgr2=0x7f9ab8150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9abe400640 1 --2- 172.21.15.42:0/3272246942 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ab806ee80 0x7f9ab8150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9abe400640 1 -- 172.21.15.42:0/3272246942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ab80afa70 msgr2=0x7f9ab80ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9abe400640 1 --2- 172.21.15.42:0/3272246942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ab80afa70 0x7f9ab80ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9abe400640 1 -- 172.21.15.42:0/3272246942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab80ce7d0 con 0x7f9ab80d06f0 2024-05-28T14:04:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9abe400640 1 --2- 172.21.15.42:0/3272246942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab80d06f0 0x7f9ab8150f40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00097c0 tx=0x7f9ab001abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9ab5600640 1 -- 172.21.15.42:0/3272246942 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00b0040 con 0x7f9ab80d06f0 2024-05-28T14:04:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9ab5600640 1 -- 172.21.15.42:0/3272246942 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab00a4e60 con 0x7f9ab80d06f0 2024-05-28T14:04:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9ac0400640 1 -- 172.21.15.42:0/3272246942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab80cd730 con 0x7f9ab80d06f0 2024-05-28T14:04:54.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9ab5600640 1 -- 172.21.15.42:0/3272246942 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0006e20 con 0x7f9ab80d06f0 2024-05-28T14:04:54.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9ac0400640 1 -- 172.21.15.42:0/3272246942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab80cdcf0 con 0x7f9ab80d06f0 2024-05-28T14:04:54.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.511+0000 7f9ac0400640 1 -- 172.21.15.42:0/3272246942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab8068a90 con 0x7f9ab80d06f0 2024-05-28T14:04:54.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.515+0000 7f9ab5600640 1 -- 172.21.15.42:0/3272246942 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 88803+0+0 (secure 0 0 0) 0x7f9ab0004230 con 0x7f9ab80d06f0 2024-05-28T14:04:54.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.515+0000 7f9ab5600640 1 --2- 172.21.15.42:0/3272246942 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9a98068cb0 0x7f9a9806b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:54.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.515+0000 7f9ab5600640 1 -- 172.21.15.42:0/3272246942 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f9ab011fa40 con 0x7f9ab80d06f0 2024-05-28T14:04:54.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.515+0000 7f9abda00640 1 --2- 172.21.15.42:0/3272246942 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9a98068cb0 0x7f9a9806b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:54.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.515+0000 7f9abda00640 1 --2- 172.21.15.42:0/3272246942 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9a98068cb0 0x7f9a9806b170 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0012f0 tx=0x7f9aac001690 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:54.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.519+0000 7f9ab5600640 1 -- 172.21.15.42:0/3272246942 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9ab0016090 con 0x7f9ab80d06f0 2024-05-28T14:04:54.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.715+0000 7f9ac0400640 1 -- 172.21.15.42:0/3272246942 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f9ab80b5160 con 0x7f9a98068cb0 2024-05-28T14:04:54.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.727+0000 7f9ab5600640 1 -- 172.21.15.42:0/3272246942 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ab80b5160 con 0x7f9a98068cb0 2024-05-28T14:04:54.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.731+0000 7f9ac0400640 1 -- 172.21.15.42:0/3272246942 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9a98068cb0 msgr2=0x7f9a9806b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:54.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.731+0000 7f9ac0400640 1 --2- 172.21.15.42:0/3272246942 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9a98068cb0 0x7f9a9806b170 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0012f0 tx=0x7f9aac001690 comp rx=0 tx=0).stop 2024-05-28T14:04:54.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.731+0000 7f9ac0400640 1 -- 172.21.15.42:0/3272246942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab80d06f0 msgr2=0x7f9ab8150f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:54.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.731+0000 7f9ac0400640 1 --2- 172.21.15.42:0/3272246942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab80d06f0 0x7f9ab8150f40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00097c0 tx=0x7f9ab001abc0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.731+0000 7f9ac0400640 1 -- 172.21.15.42:0/3272246942 shutdown_connections 2024-05-28T14:04:54.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.731+0000 7f9ac0400640 1 --2- 172.21.15.42:0/3272246942 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9a98068cb0 0x7f9a9806b170 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.731+0000 7f9ac0400640 1 --2- 172.21.15.42:0/3272246942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab80d06f0 0x7f9ab8150f40 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.731+0000 7f9ac0400640 1 --2- 172.21.15.42:0/3272246942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ab80afa70 0x7f9ab80ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.731+0000 7f9ac0400640 1 --2- 172.21.15.42:0/3272246942 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ab806ee80 0x7f9ab8150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.731+0000 7f9ac0400640 1 -- 172.21.15.42:0/3272246942 >> 172.21.15.42:0/3272246942 conn(0x7f9ab806bdf0 msgr2=0x7f9ab80b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:54.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.731+0000 7f9ac0400640 1 -- 172.21.15.42:0/3272246942 shutdown_connections 2024-05-28T14:04:54.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.731+0000 7f9ac0400640 1 -- 172.21.15.42:0/3272246942 wait complete. 2024-05-28T14:04:54.737 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:04:54.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.895+0000 7f9c09a00640 1 Processor -- start 2024-05-28T14:04:54.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.895+0000 7f9c09a00640 1 -- start start 2024-05-28T14:04:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.895+0000 7f9c09a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9c0406ee20 0x7f9c0406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.895+0000 7f9c09a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c0415b9b0 0x7f9c0415be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.895+0000 7f9c09a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c040afe40 con 0x7f9c04144f80 2024-05-28T14:04:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.895+0000 7f9c09a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9c0406f7c0 con 0x7f9c0406ee20 2024-05-28T14:04:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.895+0000 7f9c09a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9c040724d0 con 0x7f9c0415b9b0 2024-05-28T14:04:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.895+0000 7f9c02a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f9c04144f80 0x7f9c04072960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47024/0 (socket says 172.21.15.42:47024) 2024-05-28T14:04:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.895+0000 7f9bfa000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9c0406ee20 0x7f9c0406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.895+0000 7f9c02a00640 1 -- 172.21.15.42:0/1874724306 learned_addr learned my addr 172.21.15.42:0/1874724306 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9c03400640 1 --2- 172.21.15.42:0/1874724306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c0415b9b0 0x7f9c0415be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9bfa000640 1 -- 172.21.15.42:0/1874724306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c0415b9b0 msgr2=0x7f9c0415be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9bfa000640 1 --2- 172.21.15.42:0/1874724306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c0415b9b0 0x7f9c0415be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9bfa000640 1 -- 172.21.15.42:0/1874724306 >> v1:172.21.15.42:6789/0 conn(0x7f9c04144f80 legacy=0x7f9c04072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9bfa000640 1 -- 172.21.15.42:0/1874724306 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0415c380 con 0x7f9c0406ee20 2024-05-28T14:04:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9c02000640 1 -- 172.21.15.42:0/1874724306 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 608862584 0 0) 0x7f9c040afe40 con 0x7f9c04144f80 2024-05-28T14:04:54.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9bfa000640 1 --2- 172.21.15.42:0/1874724306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c0406ee20 0x7f9c0406f280 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0097c0 tx=0x7f9bec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efaa1a20528df6de server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:54.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9c02000640 1 -- 172.21.15.42:0/1874724306 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0b0070 con 0x7f9c0406ee20 2024-05-28T14:04:54.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9c02000640 1 -- 172.21.15.42:0/1874724306 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bec0ab600 con 0x7f9c0406ee20 2024-05-28T14:04:54.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9c02000640 1 -- 172.21.15.42:0/1874724306 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0ab9d0 con 0x7f9c0406ee20 2024-05-28T14:04:54.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9c09a00640 1 -- 172.21.15.42:0/1874724306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c0406ee20 msgr2=0x7f9c0406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9c09a00640 1 --2- 172.21.15.42:0/1874724306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c0406ee20 0x7f9c0406f280 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0097c0 tx=0x7f9bec01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9c09a00640 1 -- 172.21.15.42:0/1874724306 shutdown_connections 2024-05-28T14:04:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9c09a00640 1 --2- 172.21.15.42:0/1874724306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c0415b9b0 0x7f9c0415be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9c09a00640 1 --2- 172.21.15.42:0/1874724306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c0406ee20 0x7f9c0406f280 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9c09a00640 1 -- 172.21.15.42:0/1874724306 >> 172.21.15.42:0/1874724306 conn(0x7f9c040b21f0 msgr2=0x7f9c040b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.899+0000 7f9c09a00640 1 -- 172.21.15.42:0/1874724306 shutdown_connections 2024-05-28T14:04:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c09a00640 1 -- 172.21.15.42:0/1874724306 wait complete. 2024-05-28T14:04:54.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c09a00640 1 Processor -- start 2024-05-28T14:04:54.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c09a00640 1 -- start start 2024-05-28T14:04:54.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c09a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c0406ee20 0x7f9c04154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:54.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c09a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c04144f80 0x7f9c041408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:54.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c09a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c0415b9b0 0x7f9c040b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:54.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c09a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9c040b4010 con 0x7f9c0406ee20 2024-05-28T14:04:54.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c09a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04072160 con 0x7f9c0415b9b0 2024-05-28T14:04:54.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c09a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9c040b4190 con 0x7f9c04144f80 2024-05-28T14:04:54.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c02a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c0406ee20 0x7f9c04154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c02a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c0406ee20 0x7f9c04154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34166/0 (socket says 172.21.15.42:34166) 2024-05-28T14:04:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c02a00640 1 -- 172.21.15.42:0/4284603762 learned_addr learned my addr 172.21.15.42:0/4284603762 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9bfa000640 1 --2- 172.21.15.42:0/4284603762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c04144f80 0x7f9c041408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c03400640 1 --2- 172.21.15.42:0/4284603762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c0415b9b0 0x7f9c040b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c02a00640 1 -- 172.21.15.42:0/4284603762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c04144f80 msgr2=0x7f9c041408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c02a00640 1 --2- 172.21.15.42:0/4284603762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c04144f80 0x7f9c041408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c02a00640 1 -- 172.21.15.42:0/4284603762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c0415b9b0 msgr2=0x7f9c040b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c02a00640 1 --2- 172.21.15.42:0/4284603762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c0415b9b0 0x7f9c040b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c02a00640 1 -- 172.21.15.42:0/4284603762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c04156570 con 0x7f9c0406ee20 2024-05-28T14:04:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c02a00640 1 --2- 172.21.15.42:0/4284603762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c0406ee20 0x7f9c04154ff0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40011f0 tx=0x7f9bf400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9bfb400640 1 -- 172.21.15.42:0/4284603762 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4018070 con 0x7f9c0406ee20 2024-05-28T14:04:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9bfb400640 1 -- 172.21.15.42:0/4284603762 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf40074e0 con 0x7f9c0406ee20 2024-05-28T14:04:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9bfb400640 1 -- 172.21.15.42:0/4284603762 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4004a10 con 0x7f9c0406ee20 2024-05-28T14:04:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c09a00640 1 -- 172.21.15.42:0/4284603762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0413fdd0 con 0x7f9c0406ee20 2024-05-28T14:04:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.903+0000 7f9c09a00640 1 -- 172.21.15.42:0/4284603762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0413a1f0 con 0x7f9c0406ee20 2024-05-28T14:04:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.907+0000 7f9c09a00640 1 -- 172.21.15.42:0/4284603762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c040738b0 con 0x7f9c0406ee20 2024-05-28T14:04:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.907+0000 7f9bfb400640 1 -- 172.21.15.42:0/4284603762 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 88803+0+0 (secure 0 0 0) 0x7f9bf401b070 con 0x7f9c0406ee20 2024-05-28T14:04:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.907+0000 7f9bfb400640 1 --2- 172.21.15.42:0/4284603762 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9be0068c80 0x7f9be006b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.907+0000 7f9bfb400640 1 -- 172.21.15.42:0/4284603762 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f9bf4084540 con 0x7f9c0406ee20 2024-05-28T14:04:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.907+0000 7f9bfa000640 1 --2- 172.21.15.42:0/4284603762 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9be0068c80 0x7f9be006b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.911+0000 7f9bfa000640 1 --2- 172.21.15.42:0/4284603762 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9be0068c80 0x7f9be006b140 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0ad000 tx=0x7f9bec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:54.911+0000 7f9bfb400640 1 -- 172.21.15.42:0/4284603762 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9bf4055f00 con 0x7f9c0406ee20 2024-05-28T14:04:55.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:55.103+0000 7f9c09a00640 1 -- 172.21.15.42:0/4284603762 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f9c0404fdd0 con 0x7f9be0068c80 2024-05-28T14:04:55.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:55.107+0000 7f9bfb400640 1 -- 172.21.15.42:0/4284603762 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f9c0404fdd0 con 0x7f9be0068c80 2024-05-28T14:04:55.094 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/19bb7fcc-5845-462e-83f1-114a41db1528 2024-05-28T14:04:55.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:55.111+0000 7f9c09a00640 1 -- 172.21.15.42:0/4284603762 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9be0068c80 msgr2=0x7f9be006b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:55.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:55.111+0000 7f9c09a00640 1 --2- 172.21.15.42:0/4284603762 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9be0068c80 0x7f9be006b140 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0ad000 tx=0x7f9bec008040 comp rx=0 tx=0).stop 2024-05-28T14:04:55.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:55.111+0000 7f9c09a00640 1 -- 172.21.15.42:0/4284603762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c0406ee20 msgr2=0x7f9c04154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:55.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:55.111+0000 7f9c09a00640 1 --2- 172.21.15.42:0/4284603762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c0406ee20 0x7f9c04154ff0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40011f0 tx=0x7f9bf400c9b0 comp rx=0 tx=0).stop 2024-05-28T14:04:55.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:55.111+0000 7f9c09a00640 1 -- 172.21.15.42:0/4284603762 shutdown_connections 2024-05-28T14:04:55.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:55.111+0000 7f9c09a00640 1 --2- 172.21.15.42:0/4284603762 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9be0068c80 0x7f9be006b140 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:55.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:55.111+0000 7f9c09a00640 1 --2- 172.21.15.42:0/4284603762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c0415b9b0 0x7f9c040b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:55.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:55.111+0000 7f9c09a00640 1 --2- 172.21.15.42:0/4284603762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c04144f80 0x7f9c041408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:55.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:55.111+0000 7f9c09a00640 1 --2- 172.21.15.42:0/4284603762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c0406ee20 0x7f9c04154ff0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:55.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:55.111+0000 7f9c09a00640 1 -- 172.21.15.42:0/4284603762 >> 172.21.15.42:0/4284603762 conn(0x7f9c040b21f0 msgr2=0x7f9c040cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:55.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:55.111+0000 7f9c09a00640 1 -- 172.21.15.42:0/4284603762 shutdown_connections 2024-05-28T14:04:55.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:55.111+0000 7f9c09a00640 1 -- 172.21.15.42:0/4284603762 wait complete. 2024-05-28T14:04:55.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:55.114 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/19bb7fcc-5845-462e-83f1-114a41db1528 2024-05-28T14:04:55.114 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:04:55.128 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/19bb7fcc-5845-462e-83f1-114a41db1528 2024-05-28T14:04:55.128 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3138256896" 2024-05-28T14:04:55.128 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:55.129 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-28T14:04:57.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:57.131 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/19bb7fcc-5845-462e-83f1-114a41db1528 2024-05-28T14:04:57.131 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:04:57.146 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/19bb7fcc-5845-462e-83f1-114a41db1528 2024-05-28T14:04:57.146 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3138256896" 2024-05-28T14:04:57.147 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:57.149 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-28T14:04:59.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:04:59.150 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/19bb7fcc-5845-462e-83f1-114a41db1528 2024-05-28T14:04:59.150 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:04:59.166 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/19bb7fcc-5845-462e-83f1-114a41db1528 2024-05-28T14:04:59.166 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3221225472" 2024-05-28T14:04:59.166 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:04:59.167 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-28T14:04:59.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone28221 2024-05-28T14:04:59.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.379+0000 7f0845e00640 1 Processor -- start 2024-05-28T14:04:59.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.379+0000 7f0845e00640 1 -- start start 2024-05-28T14:04:59.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.383+0000 7f0845e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0840144f80 0x7f0840072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:59.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.383+0000 7f0845e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f084015c0c0 0x7f084006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:59.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.383+0000 7f0845e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08400afe40 con 0x7f084006ee20 2024-05-28T14:04:59.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.383+0000 7f0845e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f084006f7c0 con 0x7f0840144f80 2024-05-28T14:04:59.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.383+0000 7f0845e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f08400724d0 con 0x7f084015c0c0 2024-05-28T14:04:59.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.383+0000 7f083ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0840144f80 0x7f0840072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:59.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.383+0000 7f0836000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f084006ee20 0x7f084015b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39312/0 (socket says 172.21.15.42:39312) 2024-05-28T14:04:59.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.383+0000 7f0836000640 1 -- 172.21.15.42:0/3153474599 learned_addr learned my addr 172.21.15.42:0/3153474599 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:59.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.383+0000 7f083f400640 1 --2- 172.21.15.42:0/3153474599 >> v2:172.21.15.63:3300/0 conn(0x7f084015c0c0 0x7f084006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:59.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.383+0000 7f083ea00640 1 -- 172.21.15.42:0/3153474599 >> v2:172.21.15.63:3300/0 conn(0x7f084015c0c0 msgr2=0x7f084006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:59.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.383+0000 7f083ea00640 1 --2- 172.21.15.42:0/3153474599 >> v2:172.21.15.63:3300/0 conn(0x7f084015c0c0 0x7f084006f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.383+0000 7f083ea00640 1 -- 172.21.15.42:0/3153474599 >> v1:172.21.15.42:6789/0 conn(0x7f084006ee20 legacy=0x7f084015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:59.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.383+0000 7f083ea00640 1 -- 172.21.15.42:0/3153474599 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084015c600 con 0x7f0840144f80 2024-05-28T14:04:59.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.383+0000 7f083e000640 1 -- 172.21.15.42:0/3153474599 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4175432415 0 0) 0x7f08400afe40 con 0x7f084006ee20 2024-05-28T14:04:59.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.387+0000 7f083ea00640 1 --2- 172.21.15.42:0/3153474599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0840144f80 0x7f0840072960 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0840160060 tx=0x7f083001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641c6a9afac5d0bd server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:59.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.387+0000 7f083e000640 1 -- 172.21.15.42:0/3153474599 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08300b0070 con 0x7f0840144f80 2024-05-28T14:04:59.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.387+0000 7f083e000640 1 -- 172.21.15.42:0/3153474599 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08300ab600 con 0x7f0840144f80 2024-05-28T14:04:59.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.387+0000 7f083e000640 1 -- 172.21.15.42:0/3153474599 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08300ab9d0 con 0x7f0840144f80 2024-05-28T14:04:59.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.387+0000 7f0845e00640 1 -- 172.21.15.42:0/3153474599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0840144f80 msgr2=0x7f0840072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:59.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.387+0000 7f0845e00640 1 --2- 172.21.15.42:0/3153474599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0840144f80 0x7f0840072960 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0840160060 tx=0x7f083001c480 comp rx=0 tx=0).stop 2024-05-28T14:04:59.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.387+0000 7f0845e00640 1 -- 172.21.15.42:0/3153474599 shutdown_connections 2024-05-28T14:04:59.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.387+0000 7f0845e00640 1 --2- 172.21.15.42:0/3153474599 >> v2:172.21.15.63:3300/0 conn(0x7f084015c0c0 0x7f084006f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.387+0000 7f0845e00640 1 --2- 172.21.15.42:0/3153474599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0840144f80 0x7f0840072960 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.387+0000 7f0845e00640 1 -- 172.21.15.42:0/3153474599 >> 172.21.15.42:0/3153474599 conn(0x7f08400b21f0 msgr2=0x7f08400b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:59.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.387+0000 7f0845e00640 1 -- 172.21.15.42:0/3153474599 shutdown_connections 2024-05-28T14:04:59.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.387+0000 7f0845e00640 1 -- 172.21.15.42:0/3153474599 wait complete. 2024-05-28T14:04:59.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.387+0000 7f0845e00640 1 Processor -- start 2024-05-28T14:04:59.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.387+0000 7f0845e00640 1 -- start start 2024-05-28T14:04:59.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0845e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f084006ee20 0x7f08400ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:59.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0845e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0840144f80 0x7f08400ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:59.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0845e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f084015c0c0 0x7f08401f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:59.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0845e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0840072ea0 con 0x7f084006ee20 2024-05-28T14:04:59.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0845e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f08400b4010 con 0x7f0840144f80 2024-05-28T14:04:59.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0845e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f08400aeba0 con 0x7f084015c0c0 2024-05-28T14:04:59.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f083ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f084006ee20 0x7f08400ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:59.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f083ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f084006ee20 0x7f08400ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56070/0 (socket says 172.21.15.42:56070) 2024-05-28T14:04:59.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f083ea00640 1 -- 172.21.15.42:0/4080299603 learned_addr learned my addr 172.21.15.42:0/4080299603 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:59.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0836000640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0840144f80 0x7f08400ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:59.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f083f400640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f084015c0c0 0x7f08401f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:59.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f083ea00640 1 -- 172.21.15.42:0/4080299603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f084015c0c0 msgr2=0x7f08401f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:59.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f083ea00640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f084015c0c0 0x7f08401f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f083ea00640 1 -- 172.21.15.42:0/4080299603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0840144f80 msgr2=0x7f08400ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:59.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f083ea00640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0840144f80 0x7f08400ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f083ea00640 1 -- 172.21.15.42:0/4080299603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084013f750 con 0x7f084006ee20 2024-05-28T14:04:59.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0836000640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0840144f80 0x7f08400ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:59.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f083f400640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f084015c0c0 0x7f08401f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:04:59.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f083ea00640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f084006ee20 0x7f08400ad8c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f08300ae040 tx=0x7f083001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:59.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0837e00640 1 -- 172.21.15.42:0/4080299603 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08300b0050 con 0x7f084006ee20 2024-05-28T14:04:59.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0837e00640 1 -- 172.21.15.42:0/4080299603 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08300bc070 con 0x7f084006ee20 2024-05-28T14:04:59.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0845e00640 1 -- 172.21.15.42:0/4080299603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084013f9f0 con 0x7f084006ee20 2024-05-28T14:04:59.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0837e00640 1 -- 172.21.15.42:0/4080299603 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08300b5830 con 0x7f084006ee20 2024-05-28T14:04:59.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0845e00640 1 -- 172.21.15.42:0/4080299603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0840140c90 con 0x7f084006ee20 2024-05-28T14:04:59.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0845e00640 1 -- 172.21.15.42:0/4080299603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08400738b0 con 0x7f084006ee20 2024-05-28T14:04:59.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.391+0000 7f0837e00640 1 -- 172.21.15.42:0/4080299603 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 88803+0+0 (secure 0 0 0) 0x7f08300b59d0 con 0x7f084006ee20 2024-05-28T14:04:59.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.395+0000 7f0837e00640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f081c068c80 0x7f081c06b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:59.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.395+0000 7f0837e00640 1 -- 172.21.15.42:0/4080299603 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f083011fb70 con 0x7f084006ee20 2024-05-28T14:04:59.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.395+0000 7f0836000640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f081c068c80 0x7f081c06b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:59.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.395+0000 7f0836000640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f081c068c80 0x7f081c06b140 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0828007fd0 tx=0x7f0828007c20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:59.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.399+0000 7f0837e00640 1 -- 172.21.15.42:0/4080299603 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0830016070 con 0x7f084006ee20 2024-05-28T14:04:59.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.591+0000 7f0845e00640 1 -- 172.21.15.42:0/4080299603 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone28221", "target": ["mon-mgr", ""]}) -- 0x7f084015b9b0 con 0x7f081c068c80 2024-05-28T14:04:59.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.667+0000 7f0837e00640 1 -- 172.21.15.42:0/4080299603 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0828007270 con 0x7f081c068c80 2024-05-28T14:04:59.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.667+0000 7f0845e00640 1 -- 172.21.15.42:0/4080299603 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f081c068c80 msgr2=0x7f081c06b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:59.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.667+0000 7f0845e00640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f081c068c80 0x7f081c06b140 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0828007fd0 tx=0x7f0828007c20 comp rx=0 tx=0).stop 2024-05-28T14:04:59.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.667+0000 7f0845e00640 1 -- 172.21.15.42:0/4080299603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f084006ee20 msgr2=0x7f08400ad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:59.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.667+0000 7f0845e00640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f084006ee20 0x7f08400ad8c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f08300ae040 tx=0x7f083001acc0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.671+0000 7f0845e00640 1 -- 172.21.15.42:0/4080299603 shutdown_connections 2024-05-28T14:04:59.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.671+0000 7f0845e00640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f081c068c80 0x7f081c06b140 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.671+0000 7f0845e00640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f084015c0c0 0x7f08401f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.671+0000 7f0845e00640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0840144f80 0x7f08400ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.671+0000 7f0845e00640 1 --2- 172.21.15.42:0/4080299603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f084006ee20 0x7f08400ad8c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.671+0000 7f0845e00640 1 -- 172.21.15.42:0/4080299603 >> 172.21.15.42:0/4080299603 conn(0x7f08400b21f0 msgr2=0x7f08401402d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:59.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.671+0000 7f0845e00640 1 -- 172.21.15.42:0/4080299603 shutdown_connections 2024-05-28T14:04:59.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.671+0000 7f0845e00640 1 -- 172.21.15.42:0/4080299603 wait complete. 2024-05-28T14:04:59.657 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:59.671+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:59.657 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:59.671+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:59.657 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:59.671+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:59.657 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:59.671+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:59.657 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:59.671+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:59.674 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:59.687+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:59.674 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:59.687+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:59.674 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:59.687+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:59.674 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:59.687+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:59.674 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:04:59.687+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:04:59.678 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone28222 2024-05-28T14:04:59.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff466800640 1 Processor -- start 2024-05-28T14:04:59.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff466800640 1 -- start start 2024-05-28T14:04:59.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff466800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff460144f80 0x7ff460072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:59.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff466800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff46015c0c0 0x7ff46006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:59.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff466800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4600afe40 con 0x7ff46006ee20 2024-05-28T14:04:59.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff466800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff46006f7c0 con 0x7ff46015c0c0 2024-05-28T14:04:59.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff466800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff4600724d0 con 0x7ff460144f80 2024-05-28T14:04:59.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff45fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff46015c0c0 0x7ff46006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:59.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff45ea00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff46006ee20 0x7ff46015b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39322/0 (socket says 172.21.15.42:39322) 2024-05-28T14:04:59.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff45ea00640 1 -- 172.21.15.42:0/1725604228 learned_addr learned my addr 172.21.15.42:0/1725604228 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:04:59.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff45f400640 1 --2- 172.21.15.42:0/1725604228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff460144f80 0x7ff460072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:59.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff45e000640 1 -- 172.21.15.42:0/1725604228 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2730225615 0 0) 0x7ff4600afe40 con 0x7ff46006ee20 2024-05-28T14:04:59.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff45e000640 1 -- 172.21.15.42:0/1725604228 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff444005740 con 0x7ff46006ee20 2024-05-28T14:04:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff45fe00640 1 -- 172.21.15.42:0/1725604228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff460144f80 msgr2=0x7ff460072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:04:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff45fe00640 1 --2- 172.21.15.42:0/1725604228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff460144f80 0x7ff460072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff45fe00640 1 -- 172.21.15.42:0/1725604228 >> v1:172.21.15.42:6789/0 conn(0x7ff46006ee20 legacy=0x7ff46015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.843+0000 7ff45fe00640 1 -- 172.21.15.42:0/1725604228 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46015c600 con 0x7ff46015c0c0 2024-05-28T14:04:59.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.847+0000 7ff45fe00640 1 --2- 172.21.15.42:0/1725604228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46015c0c0 0x7ff46006f280 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff458009d90 tx=0x7ff45801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75318ebabc5eb383 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:59.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.847+0000 7ff45e000640 1 -- 172.21.15.42:0/1725604228 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4580b2070 con 0x7ff46015c0c0 2024-05-28T14:04:59.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.847+0000 7ff45e000640 1 -- 172.21.15.42:0/1725604228 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4580a5920 con 0x7ff46015c0c0 2024-05-28T14:04:59.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.847+0000 7ff45e000640 1 -- 172.21.15.42:0/1725604228 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4580a5cf0 con 0x7ff46015c0c0 2024-05-28T14:04:59.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.847+0000 7ff466800640 1 -- 172.21.15.42:0/1725604228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46015c0c0 msgr2=0x7ff46006f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:59.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.847+0000 7ff466800640 1 --2- 172.21.15.42:0/1725604228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46015c0c0 0x7ff46006f280 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff458009d90 tx=0x7ff45801c970 comp rx=0 tx=0).stop 2024-05-28T14:04:59.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.847+0000 7ff466800640 1 -- 172.21.15.42:0/1725604228 shutdown_connections 2024-05-28T14:04:59.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.847+0000 7ff466800640 1 --2- 172.21.15.42:0/1725604228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46015c0c0 0x7ff46006f280 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.847+0000 7ff466800640 1 --2- 172.21.15.42:0/1725604228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff460144f80 0x7ff460072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.847+0000 7ff466800640 1 -- 172.21.15.42:0/1725604228 >> 172.21.15.42:0/1725604228 conn(0x7ff4600b21f0 msgr2=0x7ff4600b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:04:59.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.847+0000 7ff466800640 1 -- 172.21.15.42:0/1725604228 shutdown_connections 2024-05-28T14:04:59.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.847+0000 7ff466800640 1 -- 172.21.15.42:0/1725604228 wait complete. 2024-05-28T14:04:59.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.847+0000 7ff466800640 1 Processor -- start 2024-05-28T14:04:59.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff466800640 1 -- start start 2024-05-28T14:04:59.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff466800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff46006ee20 0x7ff4600b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:59.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff466800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff460144f80 0x7ff460136d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:59.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff466800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46015c0c0 0x7ff460154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:59.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff466800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff460072ea0 con 0x7ff46015c0c0 2024-05-28T14:04:59.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff466800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4600b4010 con 0x7ff46006ee20 2024-05-28T14:04:59.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff466800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff46015eb40 con 0x7ff460144f80 2024-05-28T14:04:59.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff45fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46015c0c0 0x7ff460154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:59.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff45fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46015c0c0 0x7ff460154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56142/0 (socket says 172.21.15.42:56142) 2024-05-28T14:04:59.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff45fe00640 1 -- 172.21.15.42:0/2411620619 learned_addr learned my addr 172.21.15.42:0/2411620619 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:04:59.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff45f400640 1 --2- 172.21.15.42:0/2411620619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff46006ee20 0x7ff4600b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:59.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff45ea00640 1 --2- 172.21.15.42:0/2411620619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff460144f80 0x7ff460136d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:59.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff45fe00640 1 -- 172.21.15.42:0/2411620619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff460144f80 msgr2=0x7ff460136d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:59.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff45fe00640 1 --2- 172.21.15.42:0/2411620619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff460144f80 0x7ff460136d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff45fe00640 1 -- 172.21.15.42:0/2411620619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff46006ee20 msgr2=0x7ff4600b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:04:59.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff45fe00640 1 --2- 172.21.15.42:0/2411620619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff46006ee20 0x7ff4600b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:04:59.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff45fe00640 1 -- 172.21.15.42:0/2411620619 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff460137250 con 0x7ff46015c0c0 2024-05-28T14:04:59.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff45fe00640 1 --2- 172.21.15.42:0/2411620619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46015c0c0 0x7ff460154e20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff4580b0040 tx=0x7ff458019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:59.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff457400640 1 -- 172.21.15.42:0/2411620619 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4580b2040 con 0x7ff46015c0c0 2024-05-28T14:04:59.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff457400640 1 -- 172.21.15.42:0/2411620619 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4580bf070 con 0x7ff46015c0c0 2024-05-28T14:04:59.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff457400640 1 -- 172.21.15.42:0/2411620619 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4580b8870 con 0x7ff46015c0c0 2024-05-28T14:04:59.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff466800640 1 -- 172.21.15.42:0/2411620619 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff460143950 con 0x7ff46015c0c0 2024-05-28T14:04:59.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff466800640 1 -- 172.21.15.42:0/2411620619 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff46015e4f0 con 0x7ff46015c0c0 2024-05-28T14:04:59.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.851+0000 7ff466800640 1 -- 172.21.15.42:0/2411620619 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4600738b0 con 0x7ff46015c0c0 2024-05-28T14:04:59.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.855+0000 7ff457400640 1 -- 172.21.15.42:0/2411620619 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 88803+0+0 (secure 0 0 0) 0x7ff4580a57a0 con 0x7ff46015c0c0 2024-05-28T14:04:59.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.855+0000 7ff457400640 1 --2- 172.21.15.42:0/2411620619 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff440068c80 0x7ff44006b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:04:59.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.855+0000 7ff457400640 1 -- 172.21.15.42:0/2411620619 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7ff4581203b0 con 0x7ff46015c0c0 2024-05-28T14:04:59.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.855+0000 7ff45f400640 1 --2- 172.21.15.42:0/2411620619 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff440068c80 0x7ff44006b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:04:59.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.855+0000 7ff45f400640 1 --2- 172.21.15.42:0/2411620619 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff440068c80 0x7ff44006b140 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff450007a60 tx=0x7ff450008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:04:59.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:04:59.859+0000 7ff457400640 1 -- 172.21.15.42:0/2411620619 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff458016560 con 0x7ff46015c0c0 2024-05-28T14:05:00.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:00.099+0000 7ff466800640 1 -- 172.21.15.42:0/2411620619 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone28222", "target": ["mon-mgr", ""]}) -- 0x7ff46004fdd0 con 0x7ff440068c80 2024-05-28T14:05:00.657 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:00.671+0000 7f073fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:00.657 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:00.671+0000 7f073fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:00.657 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:00.671+0000 7f073fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:00.657 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:00.671+0000 7f073fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:00.657 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:00.671+0000 7f073fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:00.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:00.755+0000 7ff457400640 1 -- 172.21.15.42:0/2411620619 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff46004fdd0 con 0x7ff440068c80 2024-05-28T14:05:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:00.759+0000 7ff466800640 1 -- 172.21.15.42:0/2411620619 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff440068c80 msgr2=0x7ff44006b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:00.759+0000 7ff466800640 1 --2- 172.21.15.42:0/2411620619 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff440068c80 0x7ff44006b140 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff450007a60 tx=0x7ff450008040 comp rx=0 tx=0).stop 2024-05-28T14:05:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:00.759+0000 7ff466800640 1 -- 172.21.15.42:0/2411620619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46015c0c0 msgr2=0x7ff460154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:00.759+0000 7ff466800640 1 --2- 172.21.15.42:0/2411620619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46015c0c0 0x7ff460154e20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff4580b0040 tx=0x7ff458019710 comp rx=0 tx=0).stop 2024-05-28T14:05:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:00.759+0000 7ff466800640 1 -- 172.21.15.42:0/2411620619 shutdown_connections 2024-05-28T14:05:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:00.759+0000 7ff466800640 1 --2- 172.21.15.42:0/2411620619 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff440068c80 0x7ff44006b140 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:00.759+0000 7ff466800640 1 --2- 172.21.15.42:0/2411620619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff46015c0c0 0x7ff460154e20 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:00.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:00.759+0000 7ff466800640 1 --2- 172.21.15.42:0/2411620619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff460144f80 0x7ff460136d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:00.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:00.759+0000 7ff466800640 1 --2- 172.21.15.42:0/2411620619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff46006ee20 0x7ff4600b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:00.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:00.759+0000 7ff466800640 1 -- 172.21.15.42:0/2411620619 >> 172.21.15.42:0/2411620619 conn(0x7ff4600b21f0 msgr2=0x7ff4600b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:00.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:00.759+0000 7ff466800640 1 -- 172.21.15.42:0/2411620619 shutdown_connections 2024-05-28T14:05:00.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:00.759+0000 7ff466800640 1 -- 172.21.15.42:0/2411620619 wait complete. 2024-05-28T14:05:00.756 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:00.771+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:00.757 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:00.771+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:00.757 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:00.771+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:00.757 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:00.771+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:00.757 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:00.771+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:00.773 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone28223 2024-05-28T14:05:01.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.099+0000 7f8643200640 1 Processor -- start 2024-05-28T14:05:01.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.099+0000 7f8643200640 1 -- start start 2024-05-28T14:05:01.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.103+0000 7f8643200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f86440b3fc0 0x7f864406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:01.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.103+0000 7f8643200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8644162f60 con 0x7f8644167070 2024-05-28T14:05:01.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.103+0000 7f8643200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f864406f5e0 con 0x7f86440d0320 2024-05-28T14:05:01.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.103+0000 7f8643200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f86440b4630 con 0x7f86440b3fc0 2024-05-28T14:05:01.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.103+0000 7f8641400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f86440b3fc0 0x7f864406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:01.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.103+0000 7f8641400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f86440b3fc0 0x7f864406e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56202/0 (socket says 172.21.15.42:56202) 2024-05-28T14:05:01.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.103+0000 7f8641400640 1 -- 172.21.15.42:0/768559015 learned_addr learned my addr 172.21.15.42:0/768559015 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:01.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.103+0000 7f8641400640 1 -- 172.21.15.42:0/768559015 >> v1:172.21.15.63:6789/0 conn(0x7f86440d0320 legacy=0x7f86440d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:01.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.103+0000 7f8641400640 1 -- 172.21.15.42:0/768559015 >> v1:172.21.15.42:6789/0 conn(0x7f8644167070 legacy=0x7f86440cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:01.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.103+0000 7f8641400640 1 -- 172.21.15.42:0/768559015 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86440b4910 con 0x7f86440b3fc0 2024-05-28T14:05:01.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.111+0000 7f8641400640 1 --2- 172.21.15.42:0/768559015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86440b3fc0 0x7f864406e4a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8630009870 tx=0x7f863001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b6d9572415b164 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:01.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.111+0000 7f8640a00640 1 -- 172.21.15.42:0/768559015 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86300b1070 con 0x7f86440b3fc0 2024-05-28T14:05:01.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.111+0000 7f8640a00640 1 -- 172.21.15.42:0/768559015 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86300ac600 con 0x7f86440b3fc0 2024-05-28T14:05:01.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.111+0000 7f8640a00640 1 -- 172.21.15.42:0/768559015 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86300ac9d0 con 0x7f86440b3fc0 2024-05-28T14:05:01.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.115+0000 7f8643200640 1 -- 172.21.15.42:0/768559015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86440b3fc0 msgr2=0x7f864406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:01.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.115+0000 7f8643200640 1 --2- 172.21.15.42:0/768559015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86440b3fc0 0x7f864406e4a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8630009870 tx=0x7f863001c490 comp rx=0 tx=0).stop 2024-05-28T14:05:01.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.115+0000 7f8643200640 1 -- 172.21.15.42:0/768559015 shutdown_connections 2024-05-28T14:05:01.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.115+0000 7f8643200640 1 --2- 172.21.15.42:0/768559015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86440b3fc0 0x7f864406e4a0 secure :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8630009870 tx=0x7f863001c490 comp rx=0 tx=0).stop 2024-05-28T14:05:01.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.115+0000 7f8643200640 1 -- 172.21.15.42:0/768559015 >> 172.21.15.42:0/768559015 conn(0x7f864406bbc0 msgr2=0x7f864406bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:01.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.115+0000 7f8643200640 1 -- 172.21.15.42:0/768559015 shutdown_connections 2024-05-28T14:05:01.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.119+0000 7f8643200640 1 -- 172.21.15.42:0/768559015 wait complete. 2024-05-28T14:05:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.119+0000 7f8643200640 1 Processor -- start 2024-05-28T14:05:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.119+0000 7f8643200640 1 -- start start 2024-05-28T14:05:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.123+0000 7f8643200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86440d0320 0x7f8644161d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.123+0000 7f8643200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8644167070 0x7f86442084f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.123+0000 7f8643200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8644208a30 0x7f8644208eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.123+0000 7f8643200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8644167600 con 0x7f8644208a30 2024-05-28T14:05:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.123+0000 7f8643200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8644167480 con 0x7f8644167070 2024-05-28T14:05:01.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.123+0000 7f8643200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8644171680 con 0x7f86440d0320 2024-05-28T14:05:01.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.123+0000 7f8642800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8644208a30 0x7f8644208eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:01.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.123+0000 7f8642800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8644208a30 0x7f8644208eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56212/0 (socket says 172.21.15.42:56212) 2024-05-28T14:05:01.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.123+0000 7f8642800640 1 -- 172.21.15.42:0/1653267886 learned_addr learned my addr 172.21.15.42:0/1653267886 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:01.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.123+0000 7f8641e00640 1 --2- 172.21.15.42:0/1653267886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86440d0320 0x7f8644161d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:01.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.127+0000 7f8642800640 1 -- 172.21.15.42:0/1653267886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86440d0320 msgr2=0x7f8644161d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:01.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.127+0000 7f8642800640 1 --2- 172.21.15.42:0/1653267886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86440d0320 0x7f8644161d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:01.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.127+0000 7f8642800640 1 -- 172.21.15.42:0/1653267886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8644167070 msgr2=0x7f86442084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:01.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.127+0000 7f8642800640 1 --2- 172.21.15.42:0/1653267886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8644167070 0x7f86442084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:01.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.127+0000 7f8642800640 1 -- 172.21.15.42:0/1653267886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8644164100 con 0x7f8644208a30 2024-05-28T14:05:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.127+0000 7f8642800640 1 --2- 172.21.15.42:0/1653267886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8644208a30 0x7f8644208eb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f863c00c740 tx=0x7f863c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:01.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.127+0000 7f863a000640 1 -- 172.21.15.42:0/1653267886 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c016020 con 0x7f8644208a30 2024-05-28T14:05:01.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.127+0000 7f863a000640 1 -- 172.21.15.42:0/1653267886 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f863c00ce50 con 0x7f8644208a30 2024-05-28T14:05:01.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.127+0000 7f863a000640 1 -- 172.21.15.42:0/1653267886 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c003980 con 0x7f8644208a30 2024-05-28T14:05:01.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.127+0000 7f8643200640 1 -- 172.21.15.42:0/1653267886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86441643e0 con 0x7f8644208a30 2024-05-28T14:05:01.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.127+0000 7f8643200640 1 -- 172.21.15.42:0/1653267886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86441649a0 con 0x7f8644208a30 2024-05-28T14:05:01.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.131+0000 7f8643200640 1 -- 172.21.15.42:0/1653267886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f861c0052a0 con 0x7f8644208a30 2024-05-28T14:05:01.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.131+0000 7f863a000640 1 -- 172.21.15.42:0/1653267886 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 88803+0+0 (secure 0 0 0) 0x7f863c003b20 con 0x7f8644208a30 2024-05-28T14:05:01.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.135+0000 7f863a000640 1 --2- 172.21.15.42:0/1653267886 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8624068c80 0x7f862406b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:01.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.135+0000 7f863a000640 1 -- 172.21.15.42:0/1653267886 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f863c084fa0 con 0x7f8644208a30 2024-05-28T14:05:01.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.143+0000 7f8641e00640 1 --2- 172.21.15.42:0/1653267886 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8624068c80 0x7f862406b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:01.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.143+0000 7f863a000640 1 -- 172.21.15.42:0/1653267886 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f863c020020 con 0x7f8644208a30 2024-05-28T14:05:01.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.147+0000 7f8641e00640 1 --2- 172.21.15.42:0/1653267886 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8624068c80 0x7f862406b140 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f863400b810 tx=0x7f8634008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:01.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.663+0000 7f8643200640 1 -- 172.21.15.42:0/1653267886 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone28223", "target": ["mon-mgr", ""]}) -- 0x7f861c000c90 con 0x7f8624068c80 2024-05-28T14:05:01.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:01.795+0000 7f863a000640 1 -- 172.21.15.42:0/1653267886 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 19) ==== 89007+0+0 (secure 0 0 0) 0x7f863c052f60 con 0x7f8644208a30 2024-05-28T14:05:02.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.207+0000 7f863a000640 1 -- 172.21.15.42:0/1653267886 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f861c000c90 con 0x7f8624068c80 2024-05-28T14:05:02.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.219+0000 7f8643200640 1 -- 172.21.15.42:0/1653267886 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8624068c80 msgr2=0x7f862406b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:02.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.219+0000 7f8643200640 1 --2- 172.21.15.42:0/1653267886 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8624068c80 0x7f862406b140 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f863400b810 tx=0x7f8634008040 comp rx=0 tx=0).stop 2024-05-28T14:05:02.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.219+0000 7f8643200640 1 -- 172.21.15.42:0/1653267886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8644208a30 msgr2=0x7f8644208eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:02.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.219+0000 7f8643200640 1 --2- 172.21.15.42:0/1653267886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8644208a30 0x7f8644208eb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f863c00c740 tx=0x7f863c00cc00 comp rx=0 tx=0).stop 2024-05-28T14:05:02.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.223+0000 7f8643200640 1 -- 172.21.15.42:0/1653267886 shutdown_connections 2024-05-28T14:05:02.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.223+0000 7f8643200640 1 --2- 172.21.15.42:0/1653267886 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8624068c80 0x7f862406b140 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:02.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.223+0000 7f8643200640 1 --2- 172.21.15.42:0/1653267886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8644208a30 0x7f8644208eb0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:02.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.223+0000 7f8643200640 1 --2- 172.21.15.42:0/1653267886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8644167070 0x7f86442084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:02.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.223+0000 7f8643200640 1 --2- 172.21.15.42:0/1653267886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86440d0320 0x7f8644161d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:02.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.223+0000 7f8643200640 1 -- 172.21.15.42:0/1653267886 >> 172.21.15.42:0/1653267886 conn(0x7f864406bbc0 msgr2=0x7f864406ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:02.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.223+0000 7f8643200640 1 -- 172.21.15.42:0/1653267886 shutdown_connections 2024-05-28T14:05:02.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.223+0000 7f8643200640 1 -- 172.21.15.42:0/1653267886 wait complete. 2024-05-28T14:05:02.300 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:02.311+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:02.300 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:02.311+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:02.300 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:02.311+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:02.300 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:02.311+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:02.300 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:02.311+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:02.327 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone28224 2024-05-28T14:05:02.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.755+0000 7fe44e800640 1 Processor -- start 2024-05-28T14:05:02.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.755+0000 7fe44e800640 1 -- start start 2024-05-28T14:05:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.759+0000 7fe44e800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4480cef00 0x7fe4480cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.759+0000 7fe44e800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe4480cf8a0 0x7fe4480b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.759+0000 7fe44e800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4480b0340 con 0x7fe4480b0af0 2024-05-28T14:05:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.759+0000 7fe44e800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe4480b0620 con 0x7fe4480cf8a0 2024-05-28T14:05:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.759+0000 7fe44e800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe4480b2920 con 0x7fe4480cef00 2024-05-28T14:05:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.759+0000 7fe44ca00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4480cef00 0x7fe4480cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.759+0000 7fe44ca00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4480cef00 0x7fe4480cf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:48042/0 (socket says 172.21.15.42:48042) 2024-05-28T14:05:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.759+0000 7fe44ca00640 1 -- 172.21.15.42:0/22032548 learned_addr learned my addr 172.21.15.42:0/22032548 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.759+0000 7fe44de00640 1 --2- 172.21.15.42:0/22032548 >> v2:172.21.15.63:3300/0 conn(0x7fe4480cf8a0 0x7fe4480b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.763+0000 7fe44de00640 1 -- 172.21.15.42:0/22032548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4480cef00 msgr2=0x7fe4480cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.763+0000 7fe44de00640 1 --2- 172.21.15.42:0/22032548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4480cef00 0x7fe4480cf360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.763+0000 7fe44de00640 1 -- 172.21.15.42:0/22032548 >> v1:172.21.15.63:6789/0 conn(0x7fe4480b0af0 legacy=0x7fe4480b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.763+0000 7fe44de00640 1 -- 172.21.15.42:0/22032548 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe448073a30 con 0x7fe4480cf8a0 2024-05-28T14:05:02.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.767+0000 7fe44de00640 1 --2- 172.21.15.42:0/22032548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4480cf8a0 0x7fe4480b3160 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe440009d90 tx=0x7fe44001d3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2dfcb7639d23f05 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:02.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.767+0000 7fe447e00640 1 -- 172.21.15.42:0/22032548 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4400b2070 con 0x7fe4480cf8a0 2024-05-28T14:05:02.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.767+0000 7fe447e00640 1 -- 172.21.15.42:0/22032548 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4400a5a50 con 0x7fe4480cf8a0 2024-05-28T14:05:02.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.767+0000 7fe447e00640 1 -- 172.21.15.42:0/22032548 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4400a5e20 con 0x7fe4480cf8a0 2024-05-28T14:05:02.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.767+0000 7fe44e800640 1 -- 172.21.15.42:0/22032548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4480cf8a0 msgr2=0x7fe4480b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:02.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.767+0000 7fe44e800640 1 --2- 172.21.15.42:0/22032548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4480cf8a0 0x7fe4480b3160 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe440009d90 tx=0x7fe44001d3d0 comp rx=0 tx=0).stop 2024-05-28T14:05:02.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.767+0000 7fe44e800640 1 -- 172.21.15.42:0/22032548 shutdown_connections 2024-05-28T14:05:02.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.767+0000 7fe44e800640 1 --2- 172.21.15.42:0/22032548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4480cf8a0 0x7fe4480b3160 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:02.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.767+0000 7fe44e800640 1 --2- 172.21.15.42:0/22032548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4480cef00 0x7fe4480cf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:02.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.767+0000 7fe44e800640 1 -- 172.21.15.42:0/22032548 >> 172.21.15.42:0/22032548 conn(0x7fe4480b1d60 msgr2=0x7fe4480b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:02.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.771+0000 7fe44e800640 1 -- 172.21.15.42:0/22032548 shutdown_connections 2024-05-28T14:05:02.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.771+0000 7fe44e800640 1 -- 172.21.15.42:0/22032548 wait complete. 2024-05-28T14:05:02.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.771+0000 7fe44e800640 1 Processor -- start 2024-05-28T14:05:02.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.771+0000 7fe44e800640 1 -- start start 2024-05-28T14:05:02.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44e800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4480b0af0 0x7fe448140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:02.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4480cef00 0x7fe44813e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:02.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4480cf8a0 0x7fe448152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:02.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4480b1760 con 0x7fe4480cf8a0 2024-05-28T14:05:02.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44e800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe4480b15e0 con 0x7fe4480cef00 2024-05-28T14:05:02.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44e800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe4480b4ed0 con 0x7fe4480b0af0 2024-05-28T14:05:02.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4480cf8a0 0x7fe448152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:02.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4480cf8a0 0x7fe448152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56226/0 (socket says 172.21.15.42:56226) 2024-05-28T14:05:02.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44de00640 1 -- 172.21.15.42:0/866377887 learned_addr learned my addr 172.21.15.42:0/866377887 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:02.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44ca00640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4480cef00 0x7fe44813e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:02.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44d400640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4480b0af0 0x7fe448140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:02.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44de00640 1 -- 172.21.15.42:0/866377887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4480b0af0 msgr2=0x7fe448140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:02.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44de00640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4480b0af0 0x7fe448140ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:02.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44de00640 1 -- 172.21.15.42:0/866377887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4480cef00 msgr2=0x7fe44813e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:02.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44de00640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4480cef00 0x7fe44813e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:02.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.775+0000 7fe44de00640 1 -- 172.21.15.42:0/866377887 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe448155cb0 con 0x7fe4480cf8a0 2024-05-28T14:05:02.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.779+0000 7fe44de00640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4480cf8a0 0x7fe448152e00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe44001a5a0 tx=0x7fe440019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:02.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.779+0000 7fe44d400640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4480b0af0 0x7fe448140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:05:02.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.779+0000 7fe445600640 1 -- 172.21.15.42:0/866377887 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4400b2050 con 0x7fe4480cf8a0 2024-05-28T14:05:02.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.779+0000 7fe445600640 1 -- 172.21.15.42:0/866377887 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4400be070 con 0x7fe4480cf8a0 2024-05-28T14:05:02.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.779+0000 7fe445600640 1 -- 172.21.15.42:0/866377887 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4400ade60 con 0x7fe4480cf8a0 2024-05-28T14:05:02.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.779+0000 7fe44e800640 1 -- 172.21.15.42:0/866377887 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe448154830 con 0x7fe4480cf8a0 2024-05-28T14:05:02.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.783+0000 7fe44e800640 1 -- 172.21.15.42:0/866377887 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe448154550 con 0x7fe4480cf8a0 2024-05-28T14:05:02.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.783+0000 7fe43aa00640 1 -- 172.21.15.42:0/866377887 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44804fcf0 con 0x7fe4480cf8a0 2024-05-28T14:05:02.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.791+0000 7fe445600640 1 -- 172.21.15.42:0/866377887 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) ==== 89007+0+0 (secure 0 0 0) 0x7fe4400b9450 con 0x7fe4480cf8a0 2024-05-28T14:05:02.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.791+0000 7fe445600640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe428068f90 0x7fe42806b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:02.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.799+0000 7fe445600640 1 -- 172.21.15.42:0/866377887 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fe4401206e0 con 0x7fe4480cf8a0 2024-05-28T14:05:02.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.811+0000 7fe445600640 1 -- 172.21.15.42:0/866377887 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe440128020 con 0x7fe4480cf8a0 2024-05-28T14:05:02.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.819+0000 7fe44d400640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe428068f90 0x7fe42806b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:02.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:02.819+0000 7fe44d400640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe428068f90 0x7fe42806b450 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe43c001650 tx=0x7fe43c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:03.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:03.395+0000 7fe43aa00640 1 -- 172.21.15.42:0/866377887 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone28224", "target": ["mon-mgr", ""]}) -- 0x7fe4480cc140 con 0x7fe428068f90 2024-05-28T14:05:03.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:03.775+0000 7fe445600640 1 -- 172.21.15.42:0/866377887 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 20) ==== 89058+0+0 (secure 0 0 0) 0x7fe4400ee660 con 0x7fe4480cf8a0 2024-05-28T14:05:04.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:04.671+0000 7fe445600640 1 -- 172.21.15.42:0/866377887 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe4480cc140 con 0x7fe428068f90 2024-05-28T14:05:04.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:04.675+0000 7fe43aa00640 1 -- 172.21.15.42:0/866377887 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe428068f90 msgr2=0x7fe42806b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:04.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:04.675+0000 7fe43aa00640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe428068f90 0x7fe42806b450 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe43c001650 tx=0x7fe43c009040 comp rx=0 tx=0).stop 2024-05-28T14:05:04.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:04.675+0000 7fe43aa00640 1 -- 172.21.15.42:0/866377887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4480cf8a0 msgr2=0x7fe448152e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:04.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:04.675+0000 7fe43aa00640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4480cf8a0 0x7fe448152e00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe44001a5a0 tx=0x7fe440019710 comp rx=0 tx=0).stop 2024-05-28T14:05:04.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:04.675+0000 7fe43aa00640 1 -- 172.21.15.42:0/866377887 shutdown_connections 2024-05-28T14:05:04.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:04.675+0000 7fe43aa00640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe428068f90 0x7fe42806b450 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:04.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:04.675+0000 7fe43aa00640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4480cf8a0 0x7fe448152e00 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:04.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:04.675+0000 7fe43aa00640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4480cef00 0x7fe44813e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:04.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:04.675+0000 7fe43aa00640 1 --2- 172.21.15.42:0/866377887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4480b0af0 0x7fe448140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:04.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:04.675+0000 7fe43aa00640 1 -- 172.21.15.42:0/866377887 >> 172.21.15.42:0/866377887 conn(0x7fe4480b1d60 msgr2=0x7fe44806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:04.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:04.675+0000 7fe43aa00640 1 -- 172.21.15.42:0/866377887 shutdown_connections 2024-05-28T14:05:04.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:04.675+0000 7fe43aa00640 1 -- 172.21.15.42:0/866377887 wait complete. 2024-05-28T14:05:04.716 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone28225 2024-05-28T14:05:05.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.307+0000 7f9c53e00640 1 Processor -- start 2024-05-28T14:05:05.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.307+0000 7f9c53e00640 1 -- start start 2024-05-28T14:05:05.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c53e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9c4c162d10 0x7f9c4c06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:05.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c53e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c4c0724f0 0x7f9c4c0b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:05.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c53e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c4c166800 con 0x7f9c4c0cfb40 2024-05-28T14:05:05.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c53e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9c4c1630f0 con 0x7f9c4c162d10 2024-05-28T14:05:05.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c53e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9c4c06f8c0 con 0x7f9c4c0724f0 2024-05-28T14:05:05.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c50a00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f9c4c0cfb40 0x7f9c4c071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:60576/0 (socket says 172.21.15.42:60576) 2024-05-28T14:05:05.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c50a00640 1 -- 172.21.15.42:0/4013264716 learned_addr learned my addr 172.21.15.42:0/4013264716 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:05:05.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c51e00640 1 --2- 172.21.15.42:0/4013264716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c4c0724f0 0x7f9c4c0b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:05.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c51400640 1 --2- 172.21.15.42:0/4013264716 >> v2:172.21.15.42:3300/0 conn(0x7f9c4c162d10 0x7f9c4c06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:05.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c4be00640 1 -- 172.21.15.42:0/4013264716 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3762644119 0 0) 0x7f9c4c166800 con 0x7f9c4c0cfb40 2024-05-28T14:05:05.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c4be00640 1 -- 172.21.15.42:0/4013264716 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c34003530 con 0x7f9c4c0cfb40 2024-05-28T14:05:05.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c51e00640 1 -- 172.21.15.42:0/4013264716 >> v2:172.21.15.42:3300/0 conn(0x7f9c4c162d10 msgr2=0x7f9c4c06f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:05.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c51e00640 1 --2- 172.21.15.42:0/4013264716 >> v2:172.21.15.42:3300/0 conn(0x7f9c4c162d10 0x7f9c4c06f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:05.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c51e00640 1 -- 172.21.15.42:0/4013264716 >> v1:172.21.15.63:6789/0 conn(0x7f9c4c0cfb40 legacy=0x7f9c4c071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:05.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.311+0000 7f9c51e00640 1 -- 172.21.15.42:0/4013264716 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c166ac0 con 0x7f9c4c0724f0 2024-05-28T14:05:05.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.323+0000 7f9c51e00640 1 --2- 172.21.15.42:0/4013264716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c4c0724f0 0x7f9c4c0b34f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9c400097c0 tx=0x7f9c4001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f0058fa03ac394 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:05.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.323+0000 7f9c4be00640 1 -- 172.21.15.42:0/4013264716 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c400b1070 con 0x7f9c4c0724f0 2024-05-28T14:05:05.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.323+0000 7f9c4be00640 1 -- 172.21.15.42:0/4013264716 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c400ac640 con 0x7f9c4c0724f0 2024-05-28T14:05:05.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.323+0000 7f9c4be00640 1 -- 172.21.15.42:0/4013264716 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c400aca10 con 0x7f9c4c0724f0 2024-05-28T14:05:05.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.331+0000 7f9c53e00640 1 -- 172.21.15.42:0/4013264716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c4c0724f0 msgr2=0x7f9c4c0b34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:05.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.331+0000 7f9c53e00640 1 --2- 172.21.15.42:0/4013264716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c4c0724f0 0x7f9c4c0b34f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9c400097c0 tx=0x7f9c4001c480 comp rx=0 tx=0).stop 2024-05-28T14:05:05.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.331+0000 7f9c53e00640 1 -- 172.21.15.42:0/4013264716 shutdown_connections 2024-05-28T14:05:05.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.331+0000 7f9c53e00640 1 --2- 172.21.15.42:0/4013264716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c4c0724f0 0x7f9c4c0b34f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:05.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.331+0000 7f9c53e00640 1 --2- 172.21.15.42:0/4013264716 >> v2:172.21.15.42:3300/0 conn(0x7f9c4c162d10 0x7f9c4c06f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:05.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.331+0000 7f9c53e00640 1 -- 172.21.15.42:0/4013264716 >> 172.21.15.42:0/4013264716 conn(0x7f9c4c0b21f0 msgr2=0x7f9c4c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:05.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.331+0000 7f9c53e00640 1 -- 172.21.15.42:0/4013264716 shutdown_connections 2024-05-28T14:05:05.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.331+0000 7f9c53e00640 1 -- 172.21.15.42:0/4013264716 wait complete. 2024-05-28T14:05:05.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.339+0000 7f9c53e00640 1 Processor -- start 2024-05-28T14:05:05.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.339+0000 7f9c53e00640 1 -- start start 2024-05-28T14:05:05.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.351+0000 7f9c53e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0724f0 0x7f9c4c150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:05.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.351+0000 7f9c51400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0724f0 0x7f9c4c150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:05.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.351+0000 7f9c51400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0724f0 0x7f9c4c150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56244/0 (socket says 172.21.15.42:56244) 2024-05-28T14:05:05.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.351+0000 7f9c51400640 1 -- 172.21.15.42:0/4394504 learned_addr learned my addr 172.21.15.42:0/4394504 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:05.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.351+0000 7f9c53e00640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c4c0cfb40 0x7f9c4c158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:05.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.351+0000 7f9c53e00640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c162d10 0x7f9c4c164b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:05.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.351+0000 7f9c53e00640 1 -- 172.21.15.42:0/4394504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1675a0 con 0x7f9c4c0724f0 2024-05-28T14:05:05.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.351+0000 7f9c51e00640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c162d10 0x7f9c4c164b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:05.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.351+0000 7f9c50a00640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c4c0cfb40 0x7f9c4c158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:05.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.351+0000 7f9c53e00640 1 -- 172.21.15.42:0/4394504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c0b4240 con 0x7f9c4c162d10 2024-05-28T14:05:05.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.351+0000 7f9c53e00640 1 -- 172.21.15.42:0/4394504 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9c4c15d5f0 con 0x7f9c4c0cfb40 2024-05-28T14:05:05.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.355+0000 7f9c50a00640 1 -- 172.21.15.42:0/4394504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c162d10 msgr2=0x7f9c4c164b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:05.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.355+0000 7f9c50a00640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c162d10 0x7f9c4c164b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:05.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.355+0000 7f9c50a00640 1 -- 172.21.15.42:0/4394504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0724f0 msgr2=0x7f9c4c150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:05.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.355+0000 7f9c50a00640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0724f0 0x7f9c4c150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:05.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.355+0000 7f9c50a00640 1 -- 172.21.15.42:0/4394504 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c15c9e0 con 0x7f9c4c0cfb40 2024-05-28T14:05:05.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.355+0000 7f9c51400640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0724f0 0x7f9c4c150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:05:05.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.359+0000 7f9c50a00640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c4c0cfb40 0x7f9c4c158290 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800c430 tx=0x7f9c3800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:05.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.359+0000 7f9c49600640 1 -- 172.21.15.42:0/4394504 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c380074e0 con 0x7f9c4c0cfb40 2024-05-28T14:05:05.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.359+0000 7f9c49600640 1 -- 172.21.15.42:0/4394504 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c38007680 con 0x7f9c4c0cfb40 2024-05-28T14:05:05.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.359+0000 7f9c49600640 1 -- 172.21.15.42:0/4394504 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38004a30 con 0x7f9c4c0cfb40 2024-05-28T14:05:05.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.363+0000 7f9c53e00640 1 -- 172.21.15.42:0/4394504 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4c15ccc0 con 0x7f9c4c0cfb40 2024-05-28T14:05:05.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.363+0000 7f9c53e00640 1 -- 172.21.15.42:0/4394504 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4c15d280 con 0x7f9c4c0cfb40 2024-05-28T14:05:05.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.363+0000 7f9c53e00640 1 -- 172.21.15.42:0/4394504 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c4c0738b0 con 0x7f9c4c0cfb40 2024-05-28T14:05:05.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.367+0000 7f9c49600640 1 -- 172.21.15.42:0/4394504 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 20) ==== 89058+0+0 (secure 0 0 0) 0x7f9c38011070 con 0x7f9c4c0cfb40 2024-05-28T14:05:05.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.367+0000 7f9c49600640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9c2c068fa0 0x7f9c2c06b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:05.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.367+0000 7f9c49600640 1 -- 172.21.15.42:0/4394504 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f9c38085130 con 0x7f9c4c0cfb40 2024-05-28T14:05:05.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.375+0000 7f9c49600640 1 -- 172.21.15.42:0/4394504 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9c38056840 con 0x7f9c4c0cfb40 2024-05-28T14:05:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.379+0000 7f9c51400640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9c2c068fa0 0x7f9c2c06b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:05.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.427+0000 7f9c51400640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9c2c068fa0 0x7f9c2c06b460 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9c440094c0 tx=0x7f9c4400b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:05.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:05.891+0000 7f9c53e00640 1 -- 172.21.15.42:0/4394504 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone28225", "target": ["mon-mgr", ""]}) -- 0x7f9c4c15d770 con 0x7f9c2c068fa0 2024-05-28T14:05:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.111+0000 7f9c49600640 1 -- 172.21.15.42:0/4394504 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9c4c15d770 con 0x7f9c2c068fa0 2024-05-28T14:05:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.119+0000 7f9c53e00640 1 -- 172.21.15.42:0/4394504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9c2c068fa0 msgr2=0x7f9c2c06b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:07.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.119+0000 7f9c53e00640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9c2c068fa0 0x7f9c2c06b460 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9c440094c0 tx=0x7f9c4400b040 comp rx=0 tx=0).stop 2024-05-28T14:05:07.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.119+0000 7f9c53e00640 1 -- 172.21.15.42:0/4394504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c4c0cfb40 msgr2=0x7f9c4c158290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:07.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.119+0000 7f9c53e00640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c4c0cfb40 0x7f9c4c158290 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800c430 tx=0x7f9c3800c8f0 comp rx=0 tx=0).stop 2024-05-28T14:05:07.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.119+0000 7f9c53e00640 1 -- 172.21.15.42:0/4394504 shutdown_connections 2024-05-28T14:05:07.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.119+0000 7f9c53e00640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9c2c068fa0 0x7f9c2c06b460 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:07.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.119+0000 7f9c53e00640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c162d10 0x7f9c4c164b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:07.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.119+0000 7f9c53e00640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c4c0cfb40 0x7f9c4c158290 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.119+0000 7f9c53e00640 1 --2- 172.21.15.42:0/4394504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0724f0 0x7f9c4c150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.119+0000 7f9c53e00640 1 -- 172.21.15.42:0/4394504 >> 172.21.15.42:0/4394504 conn(0x7f9c4c0b21f0 msgr2=0x7f9c4c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.119+0000 7f9c53e00640 1 -- 172.21.15.42:0/4394504 shutdown_connections 2024-05-28T14:05:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.123+0000 7f9c53e00640 1 -- 172.21.15.42:0/4394504 wait complete. 2024-05-28T14:05:07.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone28226 2024-05-28T14:05:07.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.687+0000 7f2324400640 1 Processor -- start 2024-05-28T14:05:07.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.687+0000 7f2324400640 1 -- start start 2024-05-28T14:05:07.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.687+0000 7f2324400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f231c06c100 0x7f231c0cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:07.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.687+0000 7f2324400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231c0b4280 0x7f231c0cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:07.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.687+0000 7f2324400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f231c0b2560 con 0x7f231c0b70d0 2024-05-28T14:05:07.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.687+0000 7f2324400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f231c0b2840 con 0x7f231c06c100 2024-05-28T14:05:07.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.687+0000 7f2324400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f231c0b2b20 con 0x7f231c0b4280 2024-05-28T14:05:07.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.687+0000 7f2321000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f231c06c100 0x7f231c0cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:07.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.687+0000 7f2321000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f231c06c100 0x7f231c0cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:40514/0 (socket says 172.21.15.42:40514) 2024-05-28T14:05:07.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.687+0000 7f2321000640 1 -- 172.21.15.42:0/1987144686 learned_addr learned my addr 172.21.15.42:0/1987144686 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:07.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.687+0000 7f2322400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231c0b4280 0x7f231c0cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:07.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.691+0000 7f2321000640 1 -- 172.21.15.42:0/1987144686 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231c0b4280 msgr2=0x7f231c0cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:07.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.691+0000 7f2321000640 1 --2- 172.21.15.42:0/1987144686 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231c0b4280 0x7f231c0cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:07.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.691+0000 7f2321000640 1 -- 172.21.15.42:0/1987144686 >> v1:172.21.15.63:6789/0 conn(0x7f231c0b70d0 legacy=0x7f231c06b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:07.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.691+0000 7f2321000640 1 -- 172.21.15.42:0/1987144686 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231c0cbbe0 con 0x7f231c06c100 2024-05-28T14:05:07.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.691+0000 7f2322400640 1 --2- 172.21.15.42:0/1987144686 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231c0b4280 0x7f231c0cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:05:07.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.695+0000 7f2321000640 1 --2- 172.21.15.42:0/1987144686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231c06c100 0x7f231c0cc3e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2308009870 tx=0x7f230801c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aa2b1bc73fc398e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:07.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.695+0000 7f231be00640 1 -- 172.21.15.42:0/1987144686 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23080b0070 con 0x7f231c06c100 2024-05-28T14:05:07.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.695+0000 7f231be00640 1 -- 172.21.15.42:0/1987144686 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23080ab600 con 0x7f231c06c100 2024-05-28T14:05:07.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.695+0000 7f231be00640 1 -- 172.21.15.42:0/1987144686 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23080ab9d0 con 0x7f231c06c100 2024-05-28T14:05:07.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.699+0000 7f2324400640 1 -- 172.21.15.42:0/1987144686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231c06c100 msgr2=0x7f231c0cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:07.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.699+0000 7f2324400640 1 --2- 172.21.15.42:0/1987144686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231c06c100 0x7f231c0cc3e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2308009870 tx=0x7f230801c520 comp rx=0 tx=0).stop 2024-05-28T14:05:07.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.703+0000 7f2324400640 1 -- 172.21.15.42:0/1987144686 shutdown_connections 2024-05-28T14:05:07.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.703+0000 7f2324400640 1 --2- 172.21.15.42:0/1987144686 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231c0b4280 0x7f231c0cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:07.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.703+0000 7f2324400640 1 --2- 172.21.15.42:0/1987144686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231c06c100 0x7f231c0cc3e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:07.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.703+0000 7f2324400640 1 -- 172.21.15.42:0/1987144686 >> 172.21.15.42:0/1987144686 conn(0x7f231c0b2f80 msgr2=0x7f231c0b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:07.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.703+0000 7f2324400640 1 -- 172.21.15.42:0/1987144686 shutdown_connections 2024-05-28T14:05:07.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.703+0000 7f2324400640 1 -- 172.21.15.42:0/1987144686 wait complete. 2024-05-28T14:05:07.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.711+0000 7f2324400640 1 Processor -- start 2024-05-28T14:05:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.715+0000 7f2324400640 1 -- start start 2024-05-28T14:05:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.715+0000 7f2324400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231c06c100 0x7f231c0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.715+0000 7f2324400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231c0b4280 0x7f231c1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.715+0000 7f2324400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f231c0b70d0 0x7f231c161d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.715+0000 7f2324400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f231c0ce220 con 0x7f231c0b70d0 2024-05-28T14:05:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.715+0000 7f2324400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f231c0ce0a0 con 0x7f231c0b4280 2024-05-28T14:05:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.715+0000 7f2324400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f231c0ce3a0 con 0x7f231c06c100 2024-05-28T14:05:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.715+0000 7f2321a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231c06c100 0x7f231c0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.715+0000 7f2321a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231c06c100 0x7f231c0c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:39604/0 (socket says 172.21.15.42:39604) 2024-05-28T14:05:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.715+0000 7f2321a00640 1 -- 172.21.15.42:0/958555460 learned_addr learned my addr 172.21.15.42:0/958555460 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.719+0000 7f2322400640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f231c0b70d0 0x7f231c161d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.719+0000 7f2321000640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231c0b4280 0x7f231c1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.719+0000 7f2321a00640 1 -- 172.21.15.42:0/958555460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231c0b4280 msgr2=0x7f231c1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.719+0000 7f2321a00640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231c0b4280 0x7f231c1617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.719+0000 7f2321a00640 1 -- 172.21.15.42:0/958555460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f231c0b70d0 msgr2=0x7f231c161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.719+0000 7f2321a00640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f231c0b70d0 0x7f231c161d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.719+0000 7f2321a00640 1 -- 172.21.15.42:0/958555460 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231c0b1b20 con 0x7f231c06c100 2024-05-28T14:05:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.719+0000 7f2321000640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231c0b4280 0x7f231c1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:05:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.719+0000 7f2322400640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f231c0b70d0 0x7f231c161d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:05:07.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.727+0000 7f2321a00640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231c06c100 0x7f231c0c9fe0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f230c00ba50 tx=0x7f230c00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:07.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.727+0000 7f2319600640 1 -- 172.21.15.42:0/958555460 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c019070 con 0x7f231c06c100 2024-05-28T14:05:07.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.727+0000 7f2319600640 1 -- 172.21.15.42:0/958555460 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230c00cc20 con 0x7f231c06c100 2024-05-28T14:05:07.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.727+0000 7f2319600640 1 -- 172.21.15.42:0/958555460 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c014680 con 0x7f231c06c100 2024-05-28T14:05:07.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.731+0000 7f2324400640 1 -- 172.21.15.42:0/958555460 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231c0ca950 con 0x7f231c06c100 2024-05-28T14:05:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.731+0000 7f2324400640 1 -- 172.21.15.42:0/958555460 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f231c0caf10 con 0x7f231c06c100 2024-05-28T14:05:07.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.735+0000 7f2319600640 1 -- 172.21.15.42:0/958555460 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 20) ==== 89058+0+0 (secure 0 0 0) 0x7f230c01c070 con 0x7f231c06c100 2024-05-28T14:05:07.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.735+0000 7f2319600640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22fc068fd0 0x7f22fc06b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:07.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.735+0000 7f2324400640 1 -- 172.21.15.42:0/958555460 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f231c09f5e0 con 0x7f231c06c100 2024-05-28T14:05:07.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.739+0000 7f2319600640 1 -- 172.21.15.42:0/958555460 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f230c0549e0 con 0x7f231c06c100 2024-05-28T14:05:07.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.747+0000 7f2319600640 1 -- 172.21.15.42:0/958555460 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f230c056940 con 0x7f231c06c100 2024-05-28T14:05:07.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.779+0000 7f2321000640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22fc068fd0 0x7f22fc06b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:07.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:07.783+0000 7f2321000640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22fc068fd0 0x7f22fc06b490 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f23080ad000 tx=0x7f2308008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:08.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:08.351+0000 7f2324400640 1 -- 172.21.15.42:0/958555460 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone28226", "target": ["mon-mgr", ""]}) -- 0x7f231c154e30 con 0x7f22fc068fd0 2024-05-28T14:05:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:09.451+0000 7f2319600640 1 -- 172.21.15.42:0/958555460 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f231c154e30 con 0x7f22fc068fd0 2024-05-28T14:05:09.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:09.455+0000 7f2324400640 1 -- 172.21.15.42:0/958555460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22fc068fd0 msgr2=0x7f22fc06b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:09.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:09.459+0000 7f2324400640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22fc068fd0 0x7f22fc06b490 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f23080ad000 tx=0x7f2308008040 comp rx=0 tx=0).stop 2024-05-28T14:05:09.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:09.459+0000 7f2324400640 1 -- 172.21.15.42:0/958555460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231c06c100 msgr2=0x7f231c0c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:09.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:09.459+0000 7f2324400640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231c06c100 0x7f231c0c9fe0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f230c00ba50 tx=0x7f230c00bf10 comp rx=0 tx=0).stop 2024-05-28T14:05:09.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:09.463+0000 7f2324400640 1 -- 172.21.15.42:0/958555460 shutdown_connections 2024-05-28T14:05:09.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:09.463+0000 7f2324400640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f22fc068fd0 0x7f22fc06b490 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:09.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:09.463+0000 7f2324400640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f231c0b70d0 0x7f231c161d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:09.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:09.463+0000 7f2324400640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231c0b4280 0x7f231c1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:09.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:09.463+0000 7f2324400640 1 --2- 172.21.15.42:0/958555460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231c06c100 0x7f231c0c9fe0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:09.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:09.463+0000 7f2324400640 1 -- 172.21.15.42:0/958555460 >> 172.21.15.42:0/958555460 conn(0x7f231c0b2f80 msgr2=0x7f231c0b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:09.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:09.467+0000 7f2324400640 1 -- 172.21.15.42:0/958555460 shutdown_connections 2024-05-28T14:05:09.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:09.467+0000 7f2324400640 1 -- 172.21.15.42:0/958555460 wait complete. 2024-05-28T14:05:09.505 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone28227 2024-05-28T14:05:10.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.023+0000 7f5d67200640 1 Processor -- start 2024-05-28T14:05:10.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.023+0000 7f5d67200640 1 -- start start 2024-05-28T14:05:10.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.027+0000 7f5d67200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5d680d0320 0x7f5d680b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:10.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.027+0000 7f5d67200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5d6806e4a0 0x7f5d680d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:10.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.027+0000 7f5d65400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5d6806e4a0 0x7f5d680d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:10.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.027+0000 7f5d67200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d68162f60 con 0x7f5d68167070 2024-05-28T14:05:10.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.027+0000 7f5d65400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5d6806e4a0 0x7f5d680d0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47656/0 (socket says 172.21.15.42:47656) 2024-05-28T14:05:10.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.027+0000 7f5d67200640 1 -- 172.21.15.42:0/3638396571 --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5d6806e900 con 0x7f5d6806e4a0 2024-05-28T14:05:10.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.027+0000 7f5d65400640 1 -- 172.21.15.42:0/3638396571 learned_addr learned my addr 172.21.15.42:0/3638396571 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:10.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.027+0000 7f5d67200640 1 -- 172.21.15.42:0/3638396571 --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5d680d0c40 con 0x7f5d680d0320 2024-05-28T14:05:10.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.027+0000 7f5d65400640 1 -- 172.21.15.42:0/3638396571 >> v2:172.21.15.63:3300/0 conn(0x7f5d680d0320 msgr2=0x7f5d680b3fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T14:05:10.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.027+0000 7f5d65400640 1 --2- 172.21.15.42:0/3638396571 >> v2:172.21.15.63:3300/0 conn(0x7f5d680d0320 0x7f5d680b3fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:10.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.027+0000 7f5d65400640 1 -- 172.21.15.42:0/3638396571 >> v1:172.21.15.42:6789/0 conn(0x7f5d68167070 legacy=0x7f5d680cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:10.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.027+0000 7f5d65400640 1 -- 172.21.15.42:0/3638396571 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6806f5e0 con 0x7f5d6806e4a0 2024-05-28T14:05:10.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.035+0000 7f5d65400640 1 --2- 172.21.15.42:0/3638396571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d6806e4a0 0x7f5d680d0700 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5d680d0e70 tx=0x7f5d5401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bd7f95f7140e11 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:10.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.039+0000 7f5d64a00640 1 -- 172.21.15.42:0/3638396571 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d540b0070 con 0x7f5d6806e4a0 2024-05-28T14:05:10.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.039+0000 7f5d64a00640 1 -- 172.21.15.42:0/3638396571 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d540ab600 con 0x7f5d6806e4a0 2024-05-28T14:05:10.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.039+0000 7f5d64a00640 1 -- 172.21.15.42:0/3638396571 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d540ab9d0 con 0x7f5d6806e4a0 2024-05-28T14:05:10.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.043+0000 7f5d67200640 1 -- 172.21.15.42:0/3638396571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d6806e4a0 msgr2=0x7f5d680d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:10.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.043+0000 7f5d67200640 1 --2- 172.21.15.42:0/3638396571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d6806e4a0 0x7f5d680d0700 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5d680d0e70 tx=0x7f5d5401c3e0 comp rx=0 tx=0).stop 2024-05-28T14:05:10.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.047+0000 7f5d67200640 1 -- 172.21.15.42:0/3638396571 shutdown_connections 2024-05-28T14:05:10.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.047+0000 7f5d67200640 1 --2- 172.21.15.42:0/3638396571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d6806e4a0 0x7f5d680d0700 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:10.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.047+0000 7f5d67200640 1 --2- 172.21.15.42:0/3638396571 >> v2:172.21.15.63:3300/0 conn(0x7f5d680d0320 0x7f5d680b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:10.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.047+0000 7f5d67200640 1 -- 172.21.15.42:0/3638396571 >> 172.21.15.42:0/3638396571 conn(0x7f5d6806bbc0 msgr2=0x7f5d6806bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:10.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.047+0000 7f5d67200640 1 -- 172.21.15.42:0/3638396571 shutdown_connections 2024-05-28T14:05:10.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.047+0000 7f5d67200640 1 -- 172.21.15.42:0/3638396571 wait complete. 2024-05-28T14:05:10.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.051+0000 7f5d67200640 1 Processor -- start 2024-05-28T14:05:10.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.051+0000 7f5d67200640 1 -- start start 2024-05-28T14:05:10.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.051+0000 7f5d67200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d6806e4a0 0x7f5d680999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:10.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.055+0000 7f5d67200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d680d0320 0x7f5d6815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:10.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.055+0000 7f5d67200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d68167070 0x7f5d680ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:10.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.055+0000 7f5d67200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5d68167500 con 0x7f5d68167070 2024-05-28T14:05:10.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.055+0000 7f5d67200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5d680b4ed0 con 0x7f5d6806e4a0 2024-05-28T14:05:10.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.055+0000 7f5d67200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5d68167680 con 0x7f5d680d0320 2024-05-28T14:05:10.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.055+0000 7f5d65400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d680d0320 0x7f5d6815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:10.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.055+0000 7f5d65400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d680d0320 0x7f5d6815ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:39610/0 (socket says 172.21.15.42:39610) 2024-05-28T14:05:10.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.055+0000 7f5d65400640 1 -- 172.21.15.42:0/3753169772 learned_addr learned my addr 172.21.15.42:0/3753169772 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:10.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.055+0000 7f5d65e00640 1 --2- 172.21.15.42:0/3753169772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d6806e4a0 0x7f5d680999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.055+0000 7f5d65400640 1 -- 172.21.15.42:0/3753169772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d6806e4a0 msgr2=0x7f5d680999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.055+0000 7f5d65400640 1 --2- 172.21.15.42:0/3753169772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d6806e4a0 0x7f5d680999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.059+0000 7f5d65400640 1 -- 172.21.15.42:0/3753169772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d68167070 msgr2=0x7f5d680ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.059+0000 7f5d65400640 1 --2- 172.21.15.42:0/3753169772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d68167070 0x7f5d680ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.059+0000 7f5d65400640 1 -- 172.21.15.42:0/3753169772 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d68099ef0 con 0x7f5d680d0320 2024-05-28T14:05:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.059+0000 7f5d65400640 1 --2- 172.21.15.42:0/3753169772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d680d0320 0x7f5d6815ea80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5d54018ff0 tx=0x7f5d54019180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.059+0000 7f5d5e000640 1 -- 172.21.15.42:0/3753169772 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d540b9020 con 0x7f5d680d0320 2024-05-28T14:05:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.059+0000 7f5d5e000640 1 -- 172.21.15.42:0/3753169772 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d540b1040 con 0x7f5d680d0320 2024-05-28T14:05:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.059+0000 7f5d5e000640 1 -- 172.21.15.42:0/3753169772 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d540b0050 con 0x7f5d680d0320 2024-05-28T14:05:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.063+0000 7f5d67200640 1 -- 172.21.15.42:0/3753169772 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d681636f0 con 0x7f5d680d0320 2024-05-28T14:05:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.063+0000 7f5d67200640 1 -- 172.21.15.42:0/3753169772 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d68163cb0 con 0x7f5d680d0320 2024-05-28T14:05:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.063+0000 7f5d53400640 1 -- 172.21.15.42:0/3753169772 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c0052a0 con 0x7f5d680d0320 2024-05-28T14:05:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.067+0000 7f5d5e000640 1 -- 172.21.15.42:0/3753169772 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 20) ==== 89058+0+0 (secure 0 0 0) 0x7f5d540bc020 con 0x7f5d680d0320 2024-05-28T14:05:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.067+0000 7f5d5e000640 1 --2- 172.21.15.42:0/3753169772 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5d48068fa0 0x7f5d4806b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.067+0000 7f5d5e000640 1 -- 172.21.15.42:0/3753169772 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f5d5411f360 con 0x7f5d680d0320 2024-05-28T14:05:10.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.075+0000 7f5d5e000640 1 -- 172.21.15.42:0/3753169772 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5d540160f0 con 0x7f5d680d0320 2024-05-28T14:05:10.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.091+0000 7f5d65e00640 1 --2- 172.21.15.42:0/3753169772 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5d48068fa0 0x7f5d4806b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:10.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.123+0000 7f5d65e00640 1 --2- 172.21.15.42:0/3753169772 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5d48068fa0 0x7f5d4806b460 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5d68164b60 tx=0x7f5d6000d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:10.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:10.687+0000 7f5d53400640 1 -- 172.21.15.42:0/3753169772 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone28227", "target": ["mon-mgr", ""]}) -- 0x7f5d3c000c90 con 0x7f5d48068fa0 2024-05-28T14:05:12.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.219+0000 7f5d5e000640 1 -- 172.21.15.42:0/3753169772 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5d3c000c90 con 0x7f5d48068fa0 2024-05-28T14:05:12.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.231+0000 7f5d53400640 1 -- 172.21.15.42:0/3753169772 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5d48068fa0 msgr2=0x7f5d4806b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:12.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.231+0000 7f5d53400640 1 --2- 172.21.15.42:0/3753169772 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5d48068fa0 0x7f5d4806b460 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5d68164b60 tx=0x7f5d6000d040 comp rx=0 tx=0).stop 2024-05-28T14:05:12.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.231+0000 7f5d53400640 1 -- 172.21.15.42:0/3753169772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d680d0320 msgr2=0x7f5d6815ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:12.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.231+0000 7f5d53400640 1 --2- 172.21.15.42:0/3753169772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d680d0320 0x7f5d6815ea80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5d54018ff0 tx=0x7f5d54019180 comp rx=0 tx=0).stop 2024-05-28T14:05:12.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.231+0000 7f5d53400640 1 -- 172.21.15.42:0/3753169772 shutdown_connections 2024-05-28T14:05:12.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.231+0000 7f5d53400640 1 --2- 172.21.15.42:0/3753169772 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5d48068fa0 0x7f5d4806b460 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:12.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.231+0000 7f5d53400640 1 --2- 172.21.15.42:0/3753169772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d68167070 0x7f5d680ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:12.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.231+0000 7f5d53400640 1 --2- 172.21.15.42:0/3753169772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d680d0320 0x7f5d6815ea80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:12.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.231+0000 7f5d53400640 1 --2- 172.21.15.42:0/3753169772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d6806e4a0 0x7f5d680999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:12.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.231+0000 7f5d53400640 1 -- 172.21.15.42:0/3753169772 >> 172.21.15.42:0/3753169772 conn(0x7f5d6806bbc0 msgr2=0x7f5d680b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:12.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.239+0000 7f5d53400640 1 -- 172.21.15.42:0/3753169772 shutdown_connections 2024-05-28T14:05:12.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.239+0000 7f5d53400640 1 -- 172.21.15.42:0/3753169772 wait complete. 2024-05-28T14:05:12.304 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:05:12.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.735+0000 7f7624200640 1 Processor -- start 2024-05-28T14:05:12.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.735+0000 7f7624200640 1 -- start start 2024-05-28T14:05:12.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.743+0000 7f7624200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f761c15c460 0x7f761c0b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:12.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.743+0000 7f7624200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f761c0b38f0 0x7f761c15feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:12.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.743+0000 7f7624200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f761c15fca0 con 0x7f761c15f770 2024-05-28T14:05:12.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.743+0000 7f7624200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f761c0b1bf0 con 0x7f761c0b38f0 2024-05-28T14:05:12.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.743+0000 7f7624200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f761c0b0c50 con 0x7f761c15c460 2024-05-28T14:05:12.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.743+0000 7f7621800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f761c15c460 0x7f761c0b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:12.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.743+0000 7f7621800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f761c15c460 0x7f761c0b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:40542/0 (socket says 172.21.15.42:40542) 2024-05-28T14:05:12.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.743+0000 7f7621800640 1 -- 172.21.15.42:0/474552878 learned_addr learned my addr 172.21.15.42:0/474552878 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:12.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.743+0000 7f7621800640 1 -- 172.21.15.42:0/474552878 >> v2:172.21.15.42:3300/0 conn(0x7f761c0b38f0 msgr2=0x7f761c15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:12.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.743+0000 7f7621800640 1 --2- 172.21.15.42:0/474552878 >> v2:172.21.15.42:3300/0 conn(0x7f761c0b38f0 0x7f761c15feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:12.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.743+0000 7f7621800640 1 -- 172.21.15.42:0/474552878 >> v1:172.21.15.42:6789/0 conn(0x7f761c15f770 legacy=0x7f761c0b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:12.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.743+0000 7f7621800640 1 -- 172.21.15.42:0/474552878 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c0b0360 con 0x7f761c15c460 2024-05-28T14:05:12.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.751+0000 7f7621800640 1 --2- 172.21.15.42:0/474552878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f761c15c460 0x7f761c0b5e10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f761c072e30 tx=0x7f7610024c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2610618ce2a7c274 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:12.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.755+0000 7f761be00640 1 -- 172.21.15.42:0/474552878 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76100ba070 con 0x7f761c15c460 2024-05-28T14:05:12.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.755+0000 7f761be00640 1 -- 172.21.15.42:0/474552878 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76100adb10 con 0x7f761c15c460 2024-05-28T14:05:12.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.755+0000 7f761be00640 1 -- 172.21.15.42:0/474552878 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76100ba070 con 0x7f761c15c460 2024-05-28T14:05:12.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.759+0000 7f7624200640 1 -- 172.21.15.42:0/474552878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f761c15c460 msgr2=0x7f761c0b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:12.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.763+0000 7f7624200640 1 --2- 172.21.15.42:0/474552878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f761c15c460 0x7f761c0b5e10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f761c072e30 tx=0x7f7610024c80 comp rx=0 tx=0).stop 2024-05-28T14:05:12.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.763+0000 7f7624200640 1 -- 172.21.15.42:0/474552878 shutdown_connections 2024-05-28T14:05:12.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.763+0000 7f7624200640 1 --2- 172.21.15.42:0/474552878 >> v2:172.21.15.42:3300/0 conn(0x7f761c0b38f0 0x7f761c15feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:12.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.763+0000 7f7624200640 1 --2- 172.21.15.42:0/474552878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f761c15c460 0x7f761c0b5e10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:12.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.763+0000 7f7624200640 1 -- 172.21.15.42:0/474552878 >> 172.21.15.42:0/474552878 conn(0x7f761c072220 msgr2=0x7f761c0b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:12.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.763+0000 7f7624200640 1 -- 172.21.15.42:0/474552878 shutdown_connections 2024-05-28T14:05:12.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.763+0000 7f7624200640 1 -- 172.21.15.42:0/474552878 wait complete. 2024-05-28T14:05:12.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.767+0000 7f7624200640 1 Processor -- start 2024-05-28T14:05:12.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.767+0000 7f7624200640 1 -- start start 2024-05-28T14:05:12.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.767+0000 7f7624200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f761c0b38f0 0x7f761c1f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:12.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.767+0000 7f7624200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f761c15c460 0x7f761c09a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:12.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.767+0000 7f7624200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f761c15f770 0x7f761c06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:12.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.767+0000 7f7624200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f761c160d80 con 0x7f761c0b38f0 2024-05-28T14:05:12.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.767+0000 7f7624200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f761c160c00 con 0x7f761c15c460 2024-05-28T14:05:12.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.767+0000 7f7624200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f761c160f00 con 0x7f761c15f770 2024-05-28T14:05:12.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.771+0000 7f7622200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f761c15f770 0x7f761c06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:12.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.771+0000 7f7621800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f761c0b38f0 0x7f761c1f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:12.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.771+0000 7f7621800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f761c0b38f0 0x7f761c1f29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47670/0 (socket says 172.21.15.42:47670) 2024-05-28T14:05:12.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.771+0000 7f7621800640 1 -- 172.21.15.42:0/2079004157 learned_addr learned my addr 172.21.15.42:0/2079004157 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:12.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.771+0000 7f7620e00640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f761c15c460 0x7f761c09a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:12.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.771+0000 7f7620e00640 1 -- 172.21.15.42:0/2079004157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f761c15f770 msgr2=0x7f761c06e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:12.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.775+0000 7f7620e00640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f761c15f770 0x7f761c06e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:12.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.775+0000 7f7620e00640 1 -- 172.21.15.42:0/2079004157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f761c0b38f0 msgr2=0x7f761c1f29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:12.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.775+0000 7f7620e00640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f761c0b38f0 0x7f761c1f29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:12.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.775+0000 7f7620e00640 1 -- 172.21.15.42:0/2079004157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c146580 con 0x7f761c15c460 2024-05-28T14:05:12.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.775+0000 7f7621800640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f761c0b38f0 0x7f761c1f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:05:12.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.775+0000 7f7622200640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f761c15f770 0x7f761c06e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:05:12.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.775+0000 7f7620e00640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f761c15c460 0x7f761c09a590 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f760800c360 tx=0x7f760800c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:12.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.779+0000 7f7619600640 1 -- 172.21.15.42:0/2079004157 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608017070 con 0x7f761c15c460 2024-05-28T14:05:12.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.779+0000 7f7619600640 1 -- 172.21.15.42:0/2079004157 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7608003600 con 0x7f761c15c460 2024-05-28T14:05:12.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.779+0000 7f7619600640 1 -- 172.21.15.42:0/2079004157 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608003a30 con 0x7f761c15c460 2024-05-28T14:05:12.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.779+0000 7f7624200640 1 -- 172.21.15.42:0/2079004157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761c146860 con 0x7f761c15c460 2024-05-28T14:05:12.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.779+0000 7f7624200640 1 -- 172.21.15.42:0/2079004157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761c146e20 con 0x7f761c15c460 2024-05-28T14:05:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.787+0000 7f7619600640 1 -- 172.21.15.42:0/2079004157 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 20) ==== 89058+0+0 (secure 0 0 0) 0x7f760801a070 con 0x7f761c15c460 2024-05-28T14:05:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.787+0000 7f7619600640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f75fc068fa0 0x7f75fc06b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.787+0000 7f7624200640 1 -- 172.21.15.42:0/2079004157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761c068a90 con 0x7f761c15c460 2024-05-28T14:05:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.787+0000 7f7619600640 1 -- 172.21.15.42:0/2079004157 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f7608084a00 con 0x7f761c15c460 2024-05-28T14:05:12.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.807+0000 7f7619600640 1 -- 172.21.15.42:0/2079004157 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f76080567f0 con 0x7f761c15c460 2024-05-28T14:05:12.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.823+0000 7f7621800640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f75fc068fa0 0x7f75fc06b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:12.875+0000 7f7621800640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f75fc068fa0 0x7f75fc06b460 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f761c158740 tx=0x7f76100af070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:13.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:13.535+0000 7f7624200640 1 -- 172.21.15.42:0/2079004157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f761c06df90 con 0x7f761c15c460 2024-05-28T14:05:13.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:13.543+0000 7f7619600640 1 -- 172.21.15.42:0/2079004157 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2402 (secure 0 0 0) 0x7f76080560f0 con 0x7f761c15c460 2024-05-28T14:05:13.531 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:05:13.532 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:05:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:05:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:05:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:05:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:05:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:05:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:05:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:05:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:05:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:05:13.532 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 434, 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 61, 2024-05-28T14:05:13.533 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:05:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:05:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:05:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:05:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:05:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:05:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:05:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:05:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:05:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:05:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:13.534 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 2186, 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 9019053470, 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 17818120192, 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 755275993088, 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 494206539, 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 482730301, 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 120, 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 131 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 37, 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4, 2024-05-28T14:05:13.535 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 4, 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 6420 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:05:13.536 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:05:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:05:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:05:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:05:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:05:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:05:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:05:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:05:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 10, 2024-05-28T14:05:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:02:58.913274+0000", 2024-05-28T14:05:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:05:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:13.537 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:05:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "58c696cb-4bfb-45e4-a3e5-43a164bc676d": { 2024-05-28T14:05:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "5 ongoing+pending clones; Avg progress is 36.667% (0s)\n [==========..................] ", 2024-05-28T14:05:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.36667001247406006, 2024-05-28T14:05:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:05:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "f63639d3-a8b8-449e-aca8-3322f44f1d24": { 2024-05-28T14:05:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "4 ongoing clones; Avg progress is 28.646% (9s)\n [========....................] (remaining: 23s)", 2024-05-28T14:05:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.28646001219749451, 2024-05-28T14:05:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:05:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:13.538 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:13.538 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:05:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:13.555+0000 7f7624200640 1 -- 172.21.15.42:0/2079004157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f75fc068fa0 msgr2=0x7f75fc06b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:13.555+0000 7f7624200640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f75fc068fa0 0x7f75fc06b460 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f761c158740 tx=0x7f76100af070 comp rx=0 tx=0).stop 2024-05-28T14:05:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:13.555+0000 7f7624200640 1 -- 172.21.15.42:0/2079004157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f761c15c460 msgr2=0x7f761c09a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:13.555+0000 7f7624200640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f761c15c460 0x7f761c09a590 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f760800c360 tx=0x7f760800c820 comp rx=0 tx=0).stop 2024-05-28T14:05:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:13.555+0000 7f7624200640 1 -- 172.21.15.42:0/2079004157 shutdown_connections 2024-05-28T14:05:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:13.555+0000 7f7624200640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f75fc068fa0 0x7f75fc06b460 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:13.555+0000 7f7624200640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f761c15f770 0x7f761c06e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:13.555+0000 7f7624200640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f761c15c460 0x7f761c09a590 secure :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f760800c360 tx=0x7f760800c820 comp rx=0 tx=0).stop 2024-05-28T14:05:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:13.555+0000 7f7624200640 1 --2- 172.21.15.42:0/2079004157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f761c0b38f0 0x7f761c1f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:13.555+0000 7f7624200640 1 -- 172.21.15.42:0/2079004157 >> 172.21.15.42:0/2079004157 conn(0x7f761c072220 msgr2=0x7f761c15b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:13.555+0000 7f7624200640 1 -- 172.21.15.42:0/2079004157 shutdown_connections 2024-05-28T14:05:13.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:13.559+0000 7f7624200640 1 -- 172.21.15.42:0/2079004157 wait complete. 2024-05-28T14:05:13.650 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone28221 2024-05-28T14:05:14.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.291+0000 7fac2d600640 1 Processor -- start 2024-05-28T14:05:14.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.291+0000 7fac2d600640 1 -- start start 2024-05-28T14:05:14.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.291+0000 7fac2d600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fac280cef00 0x7fac280cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:14.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.295+0000 7fac2d600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac280b0340 con 0x7fac280b0af0 2024-05-28T14:05:14.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.295+0000 7fac2d600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac280b2920 con 0x7fac280cf8a0 2024-05-28T14:05:14.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.295+0000 7fac2d600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fac280b0620 con 0x7fac280cef00 2024-05-28T14:05:14.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.295+0000 7fac27400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fac280cef00 0x7fac280cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:14.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.295+0000 7fac27400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fac280cef00 0x7fac280cf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:40558/0 (socket says 172.21.15.42:40558) 2024-05-28T14:05:14.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.295+0000 7fac27400640 1 -- 172.21.15.42:0/3616563648 learned_addr learned my addr 172.21.15.42:0/3616563648 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:14.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.299+0000 7fac27400640 1 -- 172.21.15.42:0/3616563648 >> v1:172.21.15.63:6789/0 conn(0x7fac280cf8a0 legacy=0x7fac280b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:14.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.299+0000 7fac27400640 1 -- 172.21.15.42:0/3616563648 >> v1:172.21.15.42:6789/0 conn(0x7fac280b0af0 legacy=0x7fac280b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:14.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.299+0000 7fac27400640 1 -- 172.21.15.42:0/3616563648 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac28073a30 con 0x7fac280cef00 2024-05-28T14:05:14.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.303+0000 7fac27400640 1 --2- 172.21.15.42:0/3616563648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac280cef00 0x7fac280cf360 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fac18009870 tx=0x7fac1801cfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89b7c3af58437664 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:14.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.307+0000 7fac26a00640 1 -- 172.21.15.42:0/3616563648 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac180a9400 con 0x7fac280cef00 2024-05-28T14:05:14.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.307+0000 7fac26a00640 1 -- 172.21.15.42:0/3616563648 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac180a95a0 con 0x7fac280cef00 2024-05-28T14:05:14.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.307+0000 7fac26a00640 1 -- 172.21.15.42:0/3616563648 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac180a9970 con 0x7fac280cef00 2024-05-28T14:05:14.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.307+0000 7fac2d600640 1 -- 172.21.15.42:0/3616563648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac280cef00 msgr2=0x7fac280cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:14.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.307+0000 7fac2d600640 1 --2- 172.21.15.42:0/3616563648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac280cef00 0x7fac280cf360 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fac18009870 tx=0x7fac1801cfa0 comp rx=0 tx=0).stop 2024-05-28T14:05:14.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.307+0000 7fac2d600640 1 -- 172.21.15.42:0/3616563648 shutdown_connections 2024-05-28T14:05:14.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.307+0000 7fac2d600640 1 --2- 172.21.15.42:0/3616563648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac280cef00 0x7fac280cf360 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:14.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.307+0000 7fac2d600640 1 -- 172.21.15.42:0/3616563648 >> 172.21.15.42:0/3616563648 conn(0x7fac280b1d60 msgr2=0x7fac280b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:14.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.307+0000 7fac2d600640 1 -- 172.21.15.42:0/3616563648 shutdown_connections 2024-05-28T14:05:14.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.307+0000 7fac2d600640 1 -- 172.21.15.42:0/3616563648 wait complete. 2024-05-28T14:05:14.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.311+0000 7fac2d600640 1 Processor -- start 2024-05-28T14:05:14.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.311+0000 7fac2d600640 1 -- start start 2024-05-28T14:05:14.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac2d600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac280b0af0 0x7fac281fcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:14.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac2d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280cef00 0x7fac280ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:14.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac2d600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac280cf8a0 0x7fac2815b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:14.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac2d600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fac280cfe20 con 0x7fac280cef00 2024-05-28T14:05:14.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac2d600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fac280cfca0 con 0x7fac280b0af0 2024-05-28T14:05:14.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac2d600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fac280b4ed0 con 0x7fac280cf8a0 2024-05-28T14:05:14.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac27e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac280b0af0 0x7fac281fcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:14.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac2cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac280cf8a0 0x7fac2815b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:14.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac27e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac280b0af0 0x7fac281fcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:40560/0 (socket says 172.21.15.42:40560) 2024-05-28T14:05:14.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac27e00640 1 -- 172.21.15.42:0/850832134 learned_addr learned my addr 172.21.15.42:0/850832134 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:14.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac27400640 1 --2- 172.21.15.42:0/850832134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280cef00 0x7fac280ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:14.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac2cc00640 1 -- 172.21.15.42:0/850832134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac280b0af0 msgr2=0x7fac281fcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:14.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac2cc00640 1 --2- 172.21.15.42:0/850832134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac280b0af0 0x7fac281fcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:14.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac2cc00640 1 -- 172.21.15.42:0/850832134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280cef00 msgr2=0x7fac280ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:14.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac2cc00640 1 --2- 172.21.15.42:0/850832134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280cef00 0x7fac280ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:14.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac2cc00640 1 -- 172.21.15.42:0/850832134 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac28153850 con 0x7fac280cf8a0 2024-05-28T14:05:14.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac27e00640 1 --2- 172.21.15.42:0/850832134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac280b0af0 0x7fac281fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:05:14.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac2cc00640 1 --2- 172.21.15.42:0/850832134 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac280cf8a0 0x7fac2815b550 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fac2000c730 tx=0x7fac2000cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:14.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac1fe00640 1 -- 172.21.15.42:0/850832134 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20016020 con 0x7fac280cf8a0 2024-05-28T14:05:14.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac1fe00640 1 -- 172.21.15.42:0/850832134 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac20006d60 con 0x7fac280cf8a0 2024-05-28T14:05:14.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.315+0000 7fac1fe00640 1 -- 172.21.15.42:0/850832134 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20003bc0 con 0x7fac280cf8a0 2024-05-28T14:05:14.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.319+0000 7fac2d600640 1 -- 172.21.15.42:0/850832134 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac28153b30 con 0x7fac280cf8a0 2024-05-28T14:05:14.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.319+0000 7fac2d600640 1 -- 172.21.15.42:0/850832134 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fac28151f60 con 0x7fac280cf8a0 2024-05-28T14:05:14.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.319+0000 7fac1fe00640 1 -- 172.21.15.42:0/850832134 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 20) ==== 89058+0+0 (secure 0 0 0) 0x7fac200050d0 con 0x7fac280cf8a0 2024-05-28T14:05:14.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.319+0000 7fac2d600640 1 -- 172.21.15.42:0/850832134 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac000052a0 con 0x7fac280cf8a0 2024-05-28T14:05:14.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.323+0000 7fac1fe00640 1 --2- 172.21.15.42:0/850832134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac08068f50 0x7fac0806b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.327+0000 7fac1fe00640 1 -- 172.21.15.42:0/850832134 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fac20085040 con 0x7fac280cf8a0 2024-05-28T14:05:14.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.327+0000 7fac1fe00640 1 -- 172.21.15.42:0/850832134 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fac20020020 con 0x7fac280cf8a0 2024-05-28T14:05:14.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.371+0000 7fac27e00640 1 --2- 172.21.15.42:0/850832134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac08068f50 0x7fac0806b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:14.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.379+0000 7fac27e00640 1 --2- 172.21.15.42:0/850832134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac08068f50 0x7fac0806b410 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fac28156ac0 tx=0x7fac140090e0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:14.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:14.903+0000 7fac2d600640 1 -- 172.21.15.42:0/850832134 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone28221", "target": ["mon-mgr", ""]}) -- 0x7fac00000bd0 con 0x7fac08068f50 2024-05-28T14:05:14.945 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:14.959+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:14.946 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:14.959+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:14.946 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:14.959+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:14.946 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:14.959+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:14.946 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:14.959+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:05:16.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.519+0000 7fac1fe00640 1 -- 172.21.15.42:0/850832134 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fac00000bd0 con 0x7fac08068f50 2024-05-28T14:05:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.523+0000 7fac2d600640 1 -- 172.21.15.42:0/850832134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac08068f50 msgr2=0x7fac0806b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:16.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.523+0000 7fac2d600640 1 --2- 172.21.15.42:0/850832134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac08068f50 0x7fac0806b410 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fac28156ac0 tx=0x7fac140090e0 comp rx=0 tx=0).stop 2024-05-28T14:05:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.523+0000 7fac2d600640 1 -- 172.21.15.42:0/850832134 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac280cf8a0 msgr2=0x7fac2815b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.523+0000 7fac2d600640 1 --2- 172.21.15.42:0/850832134 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac280cf8a0 0x7fac2815b550 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fac2000c730 tx=0x7fac2000cbf0 comp rx=0 tx=0).stop 2024-05-28T14:05:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.523+0000 7fac2d600640 1 -- 172.21.15.42:0/850832134 shutdown_connections 2024-05-28T14:05:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.523+0000 7fac2d600640 1 --2- 172.21.15.42:0/850832134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac08068f50 0x7fac0806b410 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.523+0000 7fac2d600640 1 --2- 172.21.15.42:0/850832134 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac280cf8a0 0x7fac2815b550 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.523+0000 7fac2d600640 1 --2- 172.21.15.42:0/850832134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280cef00 0x7fac280ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.523+0000 7fac2d600640 1 --2- 172.21.15.42:0/850832134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac280b0af0 0x7fac281fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.523+0000 7fac2d600640 1 -- 172.21.15.42:0/850832134 >> 172.21.15.42:0/850832134 conn(0x7fac280b1d60 msgr2=0x7fac280b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.523+0000 7fac2d600640 1 -- 172.21.15.42:0/850832134 shutdown_connections 2024-05-28T14:05:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.523+0000 7fac2d600640 1 -- 172.21.15.42:0/850832134 wait complete. 2024-05-28T14:05:16.600 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone28222 2024-05-28T14:05:16.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.931+0000 7feb80c00640 1 Processor -- start 2024-05-28T14:05:16.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.931+0000 7feb80c00640 1 -- start start 2024-05-28T14:05:16.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb80c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb7c0cfe50 0x7feb7c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:16.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb80c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7feb7c15add0 0x7feb7c15b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:16.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb79400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7feb7c15add0 0x7feb7c15b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:16.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb79400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7feb7c15add0 0x7feb7c15b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47714/0 (socket says 172.21.15.42:47714) 2024-05-28T14:05:16.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb79e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb7c0cfe50 0x7feb7c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:16.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb79e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb7c0cfe50 0x7feb7c06e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:39658/0 (socket says 172.21.15.42:39658) 2024-05-28T14:05:16.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb7a800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7feb7c071de0 0x7feb7c0b34f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:43060/0 (socket says 172.21.15.42:43060) 2024-05-28T14:05:16.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb80c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb7c15b850 con 0x7feb7c071de0 2024-05-28T14:05:16.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb80c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7feb7c15bb30 con 0x7feb7c15add0 2024-05-28T14:05:16.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb80c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feb7c06df20 con 0x7feb7c0cfe50 2024-05-28T14:05:16.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb79400640 1 -- 172.21.15.42:0/3946033735 learned_addr learned my addr 172.21.15.42:0/3946033735 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:16.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb79e00640 1 -- 172.21.15.42:0/3946033735 >> v2:172.21.15.42:3300/0 conn(0x7feb7c15add0 msgr2=0x7feb7c15b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:16.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb79e00640 1 --2- 172.21.15.42:0/3946033735 >> v2:172.21.15.42:3300/0 conn(0x7feb7c15add0 0x7feb7c15b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:16.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb79e00640 1 -- 172.21.15.42:0/3946033735 >> v1:172.21.15.42:6789/0 conn(0x7feb7c071de0 legacy=0x7feb7c0b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:16.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.935+0000 7feb79e00640 1 -- 172.21.15.42:0/3946033735 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7c06e200 con 0x7feb7c0cfe50 2024-05-28T14:05:16.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.943+0000 7feb79e00640 1 --2- 172.21.15.42:0/3946033735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb7c0cfe50 0x7feb7c06e720 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7feb74009d90 tx=0x7feb7401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49b22157748ebbfc server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:16.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.943+0000 7feb78a00640 1 -- 172.21.15.42:0/3946033735 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb740b2070 con 0x7feb7c0cfe50 2024-05-28T14:05:16.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.943+0000 7feb78a00640 1 -- 172.21.15.42:0/3946033735 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb740a5830 con 0x7feb7c0cfe50 2024-05-28T14:05:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.947+0000 7feb78a00640 1 -- 172.21.15.42:0/3946033735 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb740a5c00 con 0x7feb7c0cfe50 2024-05-28T14:05:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.947+0000 7feb80c00640 1 -- 172.21.15.42:0/3946033735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb7c0cfe50 msgr2=0x7feb7c06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.947+0000 7feb80c00640 1 --2- 172.21.15.42:0/3946033735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb7c0cfe50 0x7feb7c06e720 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7feb74009d90 tx=0x7feb7401c990 comp rx=0 tx=0).stop 2024-05-28T14:05:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.951+0000 7feb80c00640 1 -- 172.21.15.42:0/3946033735 shutdown_connections 2024-05-28T14:05:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.951+0000 7feb80c00640 1 --2- 172.21.15.42:0/3946033735 >> v2:172.21.15.42:3300/0 conn(0x7feb7c15add0 0x7feb7c15b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.951+0000 7feb80c00640 1 --2- 172.21.15.42:0/3946033735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb7c0cfe50 0x7feb7c06e720 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.951+0000 7feb80c00640 1 -- 172.21.15.42:0/3946033735 >> 172.21.15.42:0/3946033735 conn(0x7feb7c0b21f0 msgr2=0x7feb7c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:16.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.951+0000 7feb80c00640 1 -- 172.21.15.42:0/3946033735 shutdown_connections 2024-05-28T14:05:16.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.951+0000 7feb80c00640 1 -- 172.21.15.42:0/3946033735 wait complete. 2024-05-28T14:05:16.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.955+0000 7feb80c00640 1 Processor -- start 2024-05-28T14:05:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.955+0000 7feb80c00640 1 -- start start 2024-05-28T14:05:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.955+0000 7feb80c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb7c071de0 0x7feb7c140c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.955+0000 7feb80c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb7c0cfe50 0x7feb7c156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.955+0000 7feb80c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb7c15add0 0x7feb7c157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.955+0000 7feb80c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c072390 con 0x7feb7c15add0 2024-05-28T14:05:16.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.955+0000 7feb80c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c072210 con 0x7feb7c0cfe50 2024-05-28T14:05:16.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.955+0000 7feb80c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feb7c072510 con 0x7feb7c071de0 2024-05-28T14:05:16.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.955+0000 7feb7a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb7c15add0 0x7feb7c157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:16.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.955+0000 7feb7a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb7c15add0 0x7feb7c157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47724/0 (socket says 172.21.15.42:47724) 2024-05-28T14:05:16.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.955+0000 7feb7a800640 1 -- 172.21.15.42:0/4017371187 learned_addr learned my addr 172.21.15.42:0/4017371187 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:16.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.955+0000 7feb79400640 1 --2- 172.21.15.42:0/4017371187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb7c0cfe50 0x7feb7c156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:16.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.955+0000 7feb79e00640 1 --2- 172.21.15.42:0/4017371187 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb7c071de0 0x7feb7c140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:16.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.959+0000 7feb79400640 1 -- 172.21.15.42:0/4017371187 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb7c071de0 msgr2=0x7feb7c140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:16.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.959+0000 7feb79400640 1 --2- 172.21.15.42:0/4017371187 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb7c071de0 0x7feb7c140c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:16.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.959+0000 7feb79400640 1 -- 172.21.15.42:0/4017371187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb7c15add0 msgr2=0x7feb7c157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:16.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.959+0000 7feb79400640 1 --2- 172.21.15.42:0/4017371187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb7c15add0 0x7feb7c157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:16.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.959+0000 7feb79400640 1 -- 172.21.15.42:0/4017371187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7c152410 con 0x7feb7c0cfe50 2024-05-28T14:05:16.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.959+0000 7feb79400640 1 --2- 172.21.15.42:0/4017371187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb7c0cfe50 0x7feb7c156f30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7feb6800e870 tx=0x7feb6800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:16.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.959+0000 7feb72000640 1 -- 172.21.15.42:0/4017371187 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68017070 con 0x7feb7c0cfe50 2024-05-28T14:05:16.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.959+0000 7feb72000640 1 -- 172.21.15.42:0/4017371187 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6800ee60 con 0x7feb7c0cfe50 2024-05-28T14:05:16.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.959+0000 7feb72000640 1 -- 172.21.15.42:0/4017371187 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6800cb40 con 0x7feb7c0cfe50 2024-05-28T14:05:16.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.959+0000 7feb80c00640 1 -- 172.21.15.42:0/4017371187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7c1526f0 con 0x7feb7c0cfe50 2024-05-28T14:05:16.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.959+0000 7feb80c00640 1 -- 172.21.15.42:0/4017371187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7c152cb0 con 0x7feb7c0cfe50 2024-05-28T14:05:16.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.963+0000 7feb80c00640 1 -- 172.21.15.42:0/4017371187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7c0738b0 con 0x7feb7c0cfe50 2024-05-28T14:05:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.963+0000 7feb72000640 1 -- 172.21.15.42:0/4017371187 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 20) ==== 89058+0+0 (secure 0 0 0) 0x7feb6801a070 con 0x7feb7c0cfe50 2024-05-28T14:05:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.967+0000 7feb72000640 1 --2- 172.21.15.42:0/4017371187 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feb5c068fd0 0x7feb5c06b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.967+0000 7feb72000640 1 -- 172.21.15.42:0/4017371187 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7feb68085760 con 0x7feb7c0cfe50 2024-05-28T14:05:16.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.971+0000 7feb72000640 1 -- 172.21.15.42:0/4017371187 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feb68056f00 con 0x7feb7c0cfe50 2024-05-28T14:05:16.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.987+0000 7feb79e00640 1 --2- 172.21.15.42:0/4017371187 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feb5c068fd0 0x7feb5c06b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:16.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:16.991+0000 7feb79e00640 1 --2- 172.21.15.42:0/4017371187 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feb5c068fd0 0x7feb5c06b490 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7feb740af000 tx=0x7feb740a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:17.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:17.079+0000 7feb72000640 1 -- 172.21.15.42:0/4017371187 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 21) ==== 89109+0+0 (secure 0 0 0) 0x7feb6800cce0 con 0x7feb7c0cfe50 2024-05-28T14:05:17.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:17.463+0000 7feb80c00640 1 -- 172.21.15.42:0/4017371187 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone28222", "target": ["mon-mgr", ""]}) -- 0x7feb7c04fdd0 con 0x7feb5c068fd0 2024-05-28T14:05:17.778 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:17.791+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:05:17.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:17.795+0000 7feb72000640 1 -- 172.21.15.42:0/4017371187 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7feb7c04fdd0 con 0x7feb5c068fd0 2024-05-28T14:05:17.784 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:05:17.817 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:05:17.818 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone28223 2024-05-28T14:05:18.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.215+0000 7fabb3e00640 1 Processor -- start 2024-05-28T14:05:18.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.215+0000 7fabb3e00640 1 -- start start 2024-05-28T14:05:18.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.219+0000 7fabb3e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fabb40c58c0 0x7fabb40cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:18.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.219+0000 7fabb3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fabb406dc30 0x7fabb40d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:18.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.219+0000 7fabaa000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fabb406dc30 0x7fabb40d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:18.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.219+0000 7fabaa000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fabb406dc30 0x7fabb40d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60988/0 (socket says 172.21.15.42:60988) 2024-05-28T14:05:18.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.219+0000 7fabaa000640 1 -- 172.21.15.42:0/2299302638 learned_addr learned my addr 172.21.15.42:0/2299302638 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:18.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.219+0000 7fabb3e00640 1 --2- 172.21.15.42:0/2299302638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabb406cd10 0x7fabb40714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:18.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.219+0000 7fabb3e00640 1 -- 172.21.15.42:0/2299302638 --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fabb40ccd10 con 0x7fabb406dc30 2024-05-28T14:05:18.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.219+0000 7fabb3e00640 1 -- 172.21.15.42:0/2299302638 --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fabb40ccff0 con 0x7fabb40c58c0 2024-05-28T14:05:18.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.219+0000 7fabb2a00640 1 --2- 172.21.15.42:0/2299302638 >> v2:172.21.15.63:3300/0 conn(0x7fabb40c58c0 0x7fabb40cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:18.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.219+0000 7fabb3e00640 1 -- 172.21.15.42:0/2299302638 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fabb40cd2d0 con 0x7fabb406cd10 2024-05-28T14:05:18.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.219+0000 7fabb3400640 1 --2- 172.21.15.42:0/2299302638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabb406cd10 0x7fabb40714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:18.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.223+0000 7fabb2a00640 1 -- 172.21.15.42:0/2299302638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabb406cd10 msgr2=0x7fabb40714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:18.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.223+0000 7fabb2a00640 1 --2- 172.21.15.42:0/2299302638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabb406cd10 0x7fabb40714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:18.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.223+0000 7fabb2a00640 1 -- 172.21.15.42:0/2299302638 >> v2:172.21.15.42:3300/0 conn(0x7fabb406dc30 msgr2=0x7fabb40d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:18.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.223+0000 7fabb2a00640 1 --2- 172.21.15.42:0/2299302638 >> v2:172.21.15.42:3300/0 conn(0x7fabb406dc30 0x7fabb40d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:18.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.223+0000 7fabb2a00640 1 -- 172.21.15.42:0/2299302638 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb40cd5b0 con 0x7fabb40c58c0 2024-05-28T14:05:18.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.223+0000 7fabb3400640 1 --2- 172.21.15.42:0/2299302638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabb406cd10 0x7fabb40714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:05:18.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.231+0000 7fabb2a00640 1 --2- 172.21.15.42:0/2299302638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb40c58c0 0x7fabb40cff20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fabb406f7b0 tx=0x7faba401c1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e7fe6d6a41adb37 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:18.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.235+0000 7fabb2000640 1 -- 172.21.15.42:0/2299302638 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba40b0070 con 0x7fabb40c58c0 2024-05-28T14:05:18.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.235+0000 7fabb2000640 1 -- 172.21.15.42:0/2299302638 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba40ab600 con 0x7fabb40c58c0 2024-05-28T14:05:18.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.235+0000 7fabb2000640 1 -- 172.21.15.42:0/2299302638 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba40ab9d0 con 0x7fabb40c58c0 2024-05-28T14:05:18.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.235+0000 7fabb3e00640 1 -- 172.21.15.42:0/2299302638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb40c58c0 msgr2=0x7fabb40cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:18.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.235+0000 7fabb3e00640 1 --2- 172.21.15.42:0/2299302638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb40c58c0 0x7fabb40cff20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fabb406f7b0 tx=0x7faba401c1e0 comp rx=0 tx=0).stop 2024-05-28T14:05:18.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.239+0000 7fabb3e00640 1 -- 172.21.15.42:0/2299302638 shutdown_connections 2024-05-28T14:05:18.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.239+0000 7fabb3e00640 1 --2- 172.21.15.42:0/2299302638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabb406cd10 0x7fabb40714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:18.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.239+0000 7fabb3e00640 1 --2- 172.21.15.42:0/2299302638 >> v2:172.21.15.42:3300/0 conn(0x7fabb406dc30 0x7fabb40d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:18.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.239+0000 7fabb3e00640 1 --2- 172.21.15.42:0/2299302638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb40c58c0 0x7fabb40cff20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:18.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.239+0000 7fabb3e00640 1 -- 172.21.15.42:0/2299302638 >> 172.21.15.42:0/2299302638 conn(0x7fabb406fce0 msgr2=0x7fabb40700f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:18.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.239+0000 7fabb3e00640 1 -- 172.21.15.42:0/2299302638 shutdown_connections 2024-05-28T14:05:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.239+0000 7fabb3e00640 1 -- 172.21.15.42:0/2299302638 wait complete. 2024-05-28T14:05:18.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.247+0000 7fabb3e00640 1 Processor -- start 2024-05-28T14:05:18.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.247+0000 7fabb3e00640 1 -- start start 2024-05-28T14:05:18.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.251+0000 7fabb3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb406cd10 0x7fabb4159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:18.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.251+0000 7fabb3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabb406dc30 0x7fabb4158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:18.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.251+0000 7fabb3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabb40c58c0 0x7fabb415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:18.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.251+0000 7fabb3e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fabb40ce140 con 0x7fabb406dc30 2024-05-28T14:05:18.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.251+0000 7fabb3e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fabb40cdfc0 con 0x7fabb406cd10 2024-05-28T14:05:18.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.251+0000 7fabb3e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fabb40ce2c0 con 0x7fabb40c58c0 2024-05-28T14:05:18.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.251+0000 7fabaa000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabb406dc30 0x7fabb4158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:18.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.251+0000 7fabaa000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabb406dc30 0x7fabb4158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60994/0 (socket says 172.21.15.42:60994) 2024-05-28T14:05:18.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.251+0000 7fabaa000640 1 -- 172.21.15.42:0/3071469458 learned_addr learned my addr 172.21.15.42:0/3071469458 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:18.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.251+0000 7fabb3400640 1 --2- 172.21.15.42:0/3071469458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabb40c58c0 0x7fabb415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:18.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.251+0000 7fabb2a00640 1 --2- 172.21.15.42:0/3071469458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb406cd10 0x7fabb4159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:18.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.255+0000 7fabb3400640 1 -- 172.21.15.42:0/3071469458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb406cd10 msgr2=0x7fabb4159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:18.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.255+0000 7fabb3400640 1 --2- 172.21.15.42:0/3071469458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb406cd10 0x7fabb4159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:18.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.255+0000 7fabb3400640 1 -- 172.21.15.42:0/3071469458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabb406dc30 msgr2=0x7fabb4158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:18.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.255+0000 7fabb3400640 1 --2- 172.21.15.42:0/3071469458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabb406dc30 0x7fabb4158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:18.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.255+0000 7fabb3400640 1 -- 172.21.15.42:0/3071469458 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb4159cd0 con 0x7fabb40c58c0 2024-05-28T14:05:18.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.255+0000 7fabb3400640 1 --2- 172.21.15.42:0/3071469458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabb40c58c0 0x7fabb415b0b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fabac007d70 tx=0x7fabac00d4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.255+0000 7fabab400640 1 -- 172.21.15.42:0/3071469458 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac01a070 con 0x7fabb40c58c0 2024-05-28T14:05:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.255+0000 7fabab400640 1 -- 172.21.15.42:0/3071469458 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabac015400 con 0x7fabb40c58c0 2024-05-28T14:05:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.255+0000 7fabab400640 1 -- 172.21.15.42:0/3071469458 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac0157b0 con 0x7fabb40c58c0 2024-05-28T14:05:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.255+0000 7fabb3e00640 1 -- 172.21.15.42:0/3071469458 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb4159fb0 con 0x7fabb40c58c0 2024-05-28T14:05:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.255+0000 7fabb3e00640 1 -- 172.21.15.42:0/3071469458 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb415a570 con 0x7fabb40c58c0 2024-05-28T14:05:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.259+0000 7fabb3e00640 1 -- 172.21.15.42:0/3071469458 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb404fcf0 con 0x7fabb40c58c0 2024-05-28T14:05:18.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.263+0000 7fabab400640 1 -- 172.21.15.42:0/3071469458 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 21) ==== 89109+0+0 (secure 0 0 0) 0x7fabac01d070 con 0x7fabb40c58c0 2024-05-28T14:05:18.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.263+0000 7fabab400640 1 --2- 172.21.15.42:0/3071469458 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fab90068fe0 0x7fab9006b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:18.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.263+0000 7fabab400640 1 -- 172.21.15.42:0/3071469458 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fabac085380 con 0x7fabb40c58c0 2024-05-28T14:05:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.275+0000 7fabab400640 1 -- 172.21.15.42:0/3071469458 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fabac010070 con 0x7fabb40c58c0 2024-05-28T14:05:18.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.287+0000 7fabb2a00640 1 --2- 172.21.15.42:0/3071469458 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fab90068fe0 0x7fab9006b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:18.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.299+0000 7fabb2a00640 1 --2- 172.21.15.42:0/3071469458 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fab90068fe0 0x7fab9006b4a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faba40ad000 tx=0x7faba40a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:18.575 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:05:18.599 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:05:18.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:18.767+0000 7fabb3e00640 1 -- 172.21.15.42:0/3071469458 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone28223", "target": ["mon-mgr", ""]}) -- 0x7fabb41599c0 con 0x7fab90068fe0 2024-05-28T14:05:20.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:20.551+0000 7fabab400640 1 -- 172.21.15.42:0/3071469458 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fabb41599c0 con 0x7fab90068fe0 2024-05-28T14:05:20.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:20.563+0000 7fabb3e00640 1 -- 172.21.15.42:0/3071469458 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fab90068fe0 msgr2=0x7fab9006b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:20.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:20.563+0000 7fabb3e00640 1 --2- 172.21.15.42:0/3071469458 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fab90068fe0 0x7fab9006b4a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faba40ad000 tx=0x7faba40a6000 comp rx=0 tx=0).stop 2024-05-28T14:05:20.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:20.563+0000 7fabb3e00640 1 -- 172.21.15.42:0/3071469458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabb40c58c0 msgr2=0x7fabb415b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:20.563+0000 7fabb3e00640 1 --2- 172.21.15.42:0/3071469458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabb40c58c0 0x7fabb415b0b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fabac007d70 tx=0x7fabac00d4a0 comp rx=0 tx=0).stop 2024-05-28T14:05:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:20.563+0000 7fabb3e00640 1 -- 172.21.15.42:0/3071469458 shutdown_connections 2024-05-28T14:05:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:20.563+0000 7fabb3e00640 1 --2- 172.21.15.42:0/3071469458 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fab90068fe0 0x7fab9006b4a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:20.563+0000 7fabb3e00640 1 --2- 172.21.15.42:0/3071469458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabb40c58c0 0x7fabb415b0b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:20.563+0000 7fabb3e00640 1 --2- 172.21.15.42:0/3071469458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabb406dc30 0x7fabb4158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:20.563+0000 7fabb3e00640 1 --2- 172.21.15.42:0/3071469458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb406cd10 0x7fabb4159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:20.563+0000 7fabb3e00640 1 -- 172.21.15.42:0/3071469458 >> 172.21.15.42:0/3071469458 conn(0x7fabb406fce0 msgr2=0x7fabb40cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:20.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:20.563+0000 7fabb3e00640 1 -- 172.21.15.42:0/3071469458 shutdown_connections 2024-05-28T14:05:20.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:20.567+0000 7fabb3e00640 1 -- 172.21.15.42:0/3071469458 wait complete. 2024-05-28T14:05:20.658 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone28224 2024-05-28T14:05:21.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.074+0000 7f3b9fa00640 1 Processor -- start 2024-05-28T14:05:21.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.074+0000 7f3b9fa00640 1 -- start start 2024-05-28T14:05:21.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.082+0000 7f3b9fa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3b9815bad0 0x7f3b980ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:21.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.082+0000 7f3b9fa00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b980cf160 con 0x7f3b9815b6f0 2024-05-28T14:05:21.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.082+0000 7f3b9fa00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b980cf440 con 0x7f3b980b3a70 2024-05-28T14:05:21.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.082+0000 7f3b9fa00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f3b980cf720 con 0x7f3b9815bad0 2024-05-28T14:05:21.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.082+0000 7f3b9d000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f3b9815b6f0 0x7f3b980cdd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:49492/0 (socket says 172.21.15.42:49492) 2024-05-28T14:05:21.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.082+0000 7f3b9d000640 1 -- 172.21.15.42:0/1729117426 learned_addr learned my addr 172.21.15.42:0/1729117426 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:05:21.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.082+0000 7f3b8fe00640 1 --2- 172.21.15.42:0/1729117426 >> v2:172.21.15.42:3300/0 conn(0x7f3b9815bad0 0x7f3b980ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.086+0000 7f3b97e00640 1 -- 172.21.15.42:0/1729117426 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 347761203 0 0) 0x7f3b980cf160 con 0x7f3b9815b6f0 2024-05-28T14:05:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.086+0000 7f3b97e00640 1 -- 172.21.15.42:0/1729117426 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b78003530 con 0x7f3b9815b6f0 2024-05-28T14:05:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.086+0000 7f3b8fe00640 1 -- 172.21.15.42:0/1729117426 >> v1:172.21.15.63:6789/0 conn(0x7f3b980b3a70 legacy=0x7f3b980b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.086+0000 7f3b8fe00640 1 -- 172.21.15.42:0/1729117426 >> v1:172.21.15.42:6789/0 conn(0x7f3b9815b6f0 legacy=0x7f3b980cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:21.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.090+0000 7f3b8fe00640 1 -- 172.21.15.42:0/1729117426 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b980b0e90 con 0x7f3b9815bad0 2024-05-28T14:05:21.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.098+0000 7f3b8fe00640 1 --2- 172.21.15.42:0/1729117426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9815bad0 0x7f3b980ce460 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3b880097c0 tx=0x7f3b8801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83f3ba55dd05ca5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:21.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.098+0000 7f3b97e00640 1 -- 172.21.15.42:0/1729117426 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b880b0070 con 0x7f3b9815bad0 2024-05-28T14:05:21.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.098+0000 7f3b97e00640 1 -- 172.21.15.42:0/1729117426 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b880ab600 con 0x7f3b9815bad0 2024-05-28T14:05:21.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.102+0000 7f3b97e00640 1 -- 172.21.15.42:0/1729117426 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b880a4e60 con 0x7f3b9815bad0 2024-05-28T14:05:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.102+0000 7f3b9fa00640 1 -- 172.21.15.42:0/1729117426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9815bad0 msgr2=0x7f3b980ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.102+0000 7f3b9fa00640 1 --2- 172.21.15.42:0/1729117426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9815bad0 0x7f3b980ce460 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3b880097c0 tx=0x7f3b8801c3d0 comp rx=0 tx=0).stop 2024-05-28T14:05:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.102+0000 7f3b9fa00640 1 -- 172.21.15.42:0/1729117426 shutdown_connections 2024-05-28T14:05:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.102+0000 7f3b9fa00640 1 --2- 172.21.15.42:0/1729117426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9815bad0 0x7f3b980ce460 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.102+0000 7f3b9fa00640 1 -- 172.21.15.42:0/1729117426 >> 172.21.15.42:0/1729117426 conn(0x7f3b980b2f80 msgr2=0x7f3b980b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.102+0000 7f3b9fa00640 1 -- 172.21.15.42:0/1729117426 shutdown_connections 2024-05-28T14:05:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.102+0000 7f3b9fa00640 1 -- 172.21.15.42:0/1729117426 wait complete. 2024-05-28T14:05:21.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.110+0000 7f3b9fa00640 1 Processor -- start 2024-05-28T14:05:21.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.110+0000 7f3b9fa00640 1 -- start start 2024-05-28T14:05:21.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.114+0000 7f3b9fa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b980b3a70 0x7f3b9815a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.114+0000 7f3b9fa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9815b6f0 0x7f3b980cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.114+0000 7f3b9fa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9815bad0 0x7f3b980b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.114+0000 7f3b9fa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3b980b25a0 con 0x7f3b9815b6f0 2024-05-28T14:05:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.114+0000 7f3b9fa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b980cd290 con 0x7f3b980b3a70 2024-05-28T14:05:21.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.114+0000 7f3b9fa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3b98169410 con 0x7f3b9815bad0 2024-05-28T14:05:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.114+0000 7f3b9d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b980b3a70 0x7f3b9815a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.114+0000 7f3b8fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9815b6f0 0x7f3b980cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.114+0000 7f3b9da00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9815bad0 0x7f3b980b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.114+0000 7f3b9d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b980b3a70 0x7f3b9815a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:56820/0 (socket says 172.21.15.42:56820) 2024-05-28T14:05:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.114+0000 7f3b9d000640 1 -- 172.21.15.42:0/1066309969 learned_addr learned my addr 172.21.15.42:0/1066309969 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.118+0000 7f3b8fe00640 1 -- 172.21.15.42:0/1066309969 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9815bad0 msgr2=0x7f3b980b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.118+0000 7f3b8fe00640 1 --2- 172.21.15.42:0/1066309969 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9815bad0 0x7f3b980b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.118+0000 7f3b8fe00640 1 -- 172.21.15.42:0/1066309969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b980b3a70 msgr2=0x7f3b9815a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.118+0000 7f3b8fe00640 1 --2- 172.21.15.42:0/1066309969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b980b3a70 0x7f3b9815a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.118+0000 7f3b8fe00640 1 -- 172.21.15.42:0/1066309969 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9815a9a0 con 0x7f3b9815b6f0 2024-05-28T14:05:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.118+0000 7f3b9da00640 1 --2- 172.21.15.42:0/1066309969 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b9815bad0 0x7f3b980b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:05:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.118+0000 7f3b8fe00640 1 --2- 172.21.15.42:0/1066309969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9815b6f0 0x7f3b980cabe0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3b880ab000 tx=0x7f3b88019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:21.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.122+0000 7f3b95600640 1 -- 172.21.15.42:0/1066309969 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b880b0040 con 0x7f3b9815b6f0 2024-05-28T14:05:21.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.122+0000 7f3b95600640 1 -- 172.21.15.42:0/1066309969 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b880ab400 con 0x7f3b9815b6f0 2024-05-28T14:05:21.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.122+0000 7f3b95600640 1 -- 172.21.15.42:0/1066309969 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b880ab9b0 con 0x7f3b9815b6f0 2024-05-28T14:05:21.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.122+0000 7f3b9fa00640 1 -- 172.21.15.42:0/1066309969 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b980c9b40 con 0x7f3b9815b6f0 2024-05-28T14:05:21.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.122+0000 7f3b9fa00640 1 -- 172.21.15.42:0/1066309969 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b980ca100 con 0x7f3b9815b6f0 2024-05-28T14:05:21.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.126+0000 7f3b9fa00640 1 -- 172.21.15.42:0/1066309969 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b9809f5e0 con 0x7f3b9815b6f0 2024-05-28T14:05:21.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.142+0000 7f3b95600640 1 -- 172.21.15.42:0/1066309969 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) ==== 89109+0+0 (secure 0 0 0) 0x7f3b880bb030 con 0x7f3b9815b6f0 2024-05-28T14:05:21.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.146+0000 7f3b95600640 1 --2- 172.21.15.42:0/1066309969 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b740690b0 0x7f3b7406b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:21.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.150+0000 7f3b95600640 1 -- 172.21.15.42:0/1066309969 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f3b8811e630 con 0x7f3b9815b6f0 2024-05-28T14:05:21.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.150+0000 7f3b95600640 1 -- 172.21.15.42:0/1066309969 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3b880c2020 con 0x7f3b9815b6f0 2024-05-28T14:05:21.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.170+0000 7f3b9d000640 1 --2- 172.21.15.42:0/1066309969 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b740690b0 0x7f3b7406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:21.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.182+0000 7f3b9d000640 1 --2- 172.21.15.42:0/1066309969 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b740690b0 0x7f3b7406b570 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3b80007fd0 tx=0x7f3b80009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:21.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:21.630+0000 7f3b9fa00640 1 -- 172.21.15.42:0/1066309969 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone28224", "target": ["mon-mgr", ""]}) -- 0x7f3b9804fdd0 con 0x7f3b740690b0 2024-05-28T14:05:22.563 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:05:22.574+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:05:22.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.590+0000 7f3b95600640 1 -- 172.21.15.42:0/1066309969 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f3b9804fdd0 con 0x7f3b740690b0 2024-05-28T14:05:22.577 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:05:22.596 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:05:22.597 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone28225 2024-05-28T14:05:22.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.938+0000 7f4b56200640 1 Processor -- start 2024-05-28T14:05:22.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.938+0000 7f4b56200640 1 -- start start 2024-05-28T14:05:22.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.942+0000 7f4b56200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4b50162b80 0x7f4b500b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:22.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.942+0000 7f4b56200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4b5006dd10 0x7f4b50071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.942+0000 7f4b56200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b500b3d40 con 0x7f4b5006e1a0 2024-05-28T14:05:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.942+0000 7f4b56200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f4b500ce1c0 con 0x7f4b50162b80 2024-05-28T14:05:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.942+0000 7f4b56200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5006e760 con 0x7f4b5006dd10 2024-05-28T14:05:22.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.942+0000 7f4b4f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4b50162b80 0x7f4b500b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:22.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.942+0000 7f4b4ea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4b5006dd10 0x7f4b50071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:22.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.942+0000 7f4b4ea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4b5006dd10 0x7f4b50071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:56824/0 (socket says 172.21.15.42:56824) 2024-05-28T14:05:22.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.942+0000 7f4b4ea00640 1 -- 172.21.15.42:0/669255871 learned_addr learned my addr 172.21.15.42:0/669255871 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:22.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.942+0000 7f4b4fe00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f4b5006e1a0 0x7f4b500cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:46180/0 (socket says 172.21.15.42:46180) 2024-05-28T14:05:22.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.946+0000 7f4b4e000640 1 -- 172.21.15.42:0/669255871 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2152329355 0 0) 0x7f4b500b3d40 con 0x7f4b5006e1a0 2024-05-28T14:05:22.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.946+0000 7f4b4e000640 1 -- 172.21.15.42:0/669255871 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b34003530 con 0x7f4b5006e1a0 2024-05-28T14:05:22.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.946+0000 7f4b4e000640 1 -- 172.21.15.42:0/669255871 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2628084748 0 0) 0x7f4b34003530 con 0x7f4b5006e1a0 2024-05-28T14:05:22.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.946+0000 7f4b4e000640 1 -- 172.21.15.42:0/669255871 >> v2:172.21.15.63:3300/0 conn(0x7f4b5006dd10 msgr2=0x7f4b50071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:22.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.946+0000 7f4b4e000640 1 --2- 172.21.15.42:0/669255871 >> v2:172.21.15.63:3300/0 conn(0x7f4b5006dd10 0x7f4b50071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:22.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4e000640 1 -- 172.21.15.42:0/669255871 >> v2:172.21.15.42:3300/0 conn(0x7f4b50162b80 msgr2=0x7f4b500b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:22.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4e000640 1 --2- 172.21.15.42:0/669255871 >> v2:172.21.15.42:3300/0 conn(0x7f4b50162b80 0x7f4b500b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:22.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4e000640 1 -- 172.21.15.42:0/669255871 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5006ea40 con 0x7f4b5006e1a0 2024-05-28T14:05:22.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4e000640 1 -- 172.21.15.42:0/669255871 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f4b48001a00 con 0x7f4b5006e1a0 2024-05-28T14:05:22.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4e000640 1 -- 172.21.15.42:0/669255871 >> v1:172.21.15.63:6789/0 conn(0x7f4b5006e1a0 legacy=0x7f4b500cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:22.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4e000640 1 --2- 172.21.15.42:0/669255871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b340039c0 0x7f4b34003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:22.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4e000640 1 --2- 172.21.15.42:0/669255871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b34004810 0x7f4b34004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:22.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4e000640 1 --2- 172.21.15.42:0/669255871 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b340051d0 0x7f4b3400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:22.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4e000640 1 -- 172.21.15.42:0/669255871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b500ce1c0 con 0x7f4b340039c0 2024-05-28T14:05:22.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4e000640 1 -- 172.21.15.42:0/669255871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5006e760 con 0x7f4b34004810 2024-05-28T14:05:22.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4e000640 1 -- 172.21.15.42:0/669255871 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b3400bf60 con 0x7f4b340051d0 2024-05-28T14:05:22.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4f400640 1 --2- 172.21.15.42:0/669255871 >> v2:172.21.15.42:3300/0 conn(0x7f4b50162b80 0x7f4b500b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:05:22.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4ea00640 1 --2- 172.21.15.42:0/669255871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b34004810 0x7f4b34004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:22.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4f400640 1 --2- 172.21.15.42:0/669255871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b340039c0 0x7f4b34003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:22.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.950+0000 7f4b4fe00640 1 --2- 172.21.15.42:0/669255871 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b340051d0 0x7f4b3400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:22.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.954+0000 7f4b4ea00640 1 -- 172.21.15.42:0/669255871 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b340051d0 msgr2=0x7f4b3400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:22.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.954+0000 7f4b4ea00640 1 --2- 172.21.15.42:0/669255871 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b340051d0 0x7f4b3400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:22.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.954+0000 7f4b4ea00640 1 -- 172.21.15.42:0/669255871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b340039c0 msgr2=0x7f4b34003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:22.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.954+0000 7f4b4ea00640 1 --2- 172.21.15.42:0/669255871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b340039c0 0x7f4b34003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:22.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.954+0000 7f4b4ea00640 1 -- 172.21.15.42:0/669255871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5006ea40 con 0x7f4b34004810 2024-05-28T14:05:22.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.954+0000 7f4b4fe00640 1 -- 172.21.15.42:0/669255871 reap_dead start 2024-05-28T14:05:22.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.954+0000 7f4b4f400640 1 --2- 172.21.15.42:0/669255871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b340039c0 0x7f4b34003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:05:22.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.962+0000 7f4b4ea00640 1 --2- 172.21.15.42:0/669255871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b34004810 0x7f4b34004c90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009b90 tx=0x7f4b3c01cb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67abc29b5c3d4463 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:22.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.962+0000 7f4b4e000640 1 -- 172.21.15.42:0/669255871 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0b2070 con 0x7f4b34004810 2024-05-28T14:05:22.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.962+0000 7f4b4e000640 1 -- 172.21.15.42:0/669255871 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3c0a5510 con 0x7f4b34004810 2024-05-28T14:05:22.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.962+0000 7f4b56200640 1 -- 172.21.15.42:0/669255871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b34004810 msgr2=0x7f4b34004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:22.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.966+0000 7f4b4e000640 1 -- 172.21.15.42:0/669255871 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0b2070 con 0x7f4b34004810 2024-05-28T14:05:22.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.966+0000 7f4b56200640 1 --2- 172.21.15.42:0/669255871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b34004810 0x7f4b34004c90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009b90 tx=0x7f4b3c01cb30 comp rx=0 tx=0).stop 2024-05-28T14:05:22.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.966+0000 7f4b56200640 1 -- 172.21.15.42:0/669255871 shutdown_connections 2024-05-28T14:05:22.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.966+0000 7f4b56200640 1 --2- 172.21.15.42:0/669255871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b34004810 0x7f4b34004c90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:22.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.966+0000 7f4b56200640 1 -- 172.21.15.42:0/669255871 >> 172.21.15.42:0/669255871 conn(0x7f4b500b21f0 msgr2=0x7f4b500b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:22.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.966+0000 7f4b56200640 1 -- 172.21.15.42:0/669255871 shutdown_connections 2024-05-28T14:05:22.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.966+0000 7f4b56200640 1 -- 172.21.15.42:0/669255871 wait complete. 2024-05-28T14:05:22.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.970+0000 7f4b56200640 1 Processor -- start 2024-05-28T14:05:22.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.974+0000 7f4b56200640 1 -- start start 2024-05-28T14:05:22.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.974+0000 7f4b56200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b34004810 0x7f4b500b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:22.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.974+0000 7f4b56200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b500cb920 0x7f4b500cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:22.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.974+0000 7f4b56200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b50148f60 0x7f4b501493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:22.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.974+0000 7f4b56200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501630e0 con 0x7f4b34004810 2024-05-28T14:05:22.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.974+0000 7f4b56200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50162f60 con 0x7f4b500cb920 2024-05-28T14:05:22.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.974+0000 7f4b56200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b500cec60 con 0x7f4b50148f60 2024-05-28T14:05:22.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.974+0000 7f4b4ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b500cb920 0x7f4b500cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:22.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.978+0000 7f4b4ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b500cb920 0x7f4b500cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:56838/0 (socket says 172.21.15.42:56838) 2024-05-28T14:05:22.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.978+0000 7f4b4ea00640 1 -- 172.21.15.42:0/2753648012 learned_addr learned my addr 172.21.15.42:0/2753648012 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:22.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.978+0000 7f4b4fe00640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b50148f60 0x7f4b501493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:22.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.978+0000 7f4b4f400640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b34004810 0x7f4b500b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:22.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.978+0000 7f4b4ea00640 1 -- 172.21.15.42:0/2753648012 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b50148f60 msgr2=0x7f4b501493e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:22.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.978+0000 7f4b4ea00640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b50148f60 0x7f4b501493e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:22.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.978+0000 7f4b4ea00640 1 -- 172.21.15.42:0/2753648012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b34004810 msgr2=0x7f4b500b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:22.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.978+0000 7f4b4ea00640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b34004810 0x7f4b500b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:22.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.978+0000 7f4b4ea00640 1 -- 172.21.15.42:0/2753648012 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b50149d00 con 0x7f4b500cb920 2024-05-28T14:05:22.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.978+0000 7f4b4f400640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b34004810 0x7f4b500b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:05:22.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.978+0000 7f4b4ea00640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b500cb920 0x7f4b500cbda0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009b90 tx=0x7f4b3c0198d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:22.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.982+0000 7f4b47400640 1 -- 172.21.15.42:0/2753648012 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0b2040 con 0x7f4b500cb920 2024-05-28T14:05:22.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.982+0000 7f4b47400640 1 -- 172.21.15.42:0/2753648012 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3c00f000 con 0x7f4b500cb920 2024-05-28T14:05:22.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.982+0000 7f4b47400640 1 -- 172.21.15.42:0/2753648012 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c009760 con 0x7f4b500cb920 2024-05-28T14:05:22.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.982+0000 7f4b56200640 1 -- 172.21.15.42:0/2753648012 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b50149fe0 con 0x7f4b500cb920 2024-05-28T14:05:22.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.982+0000 7f4b56200640 1 -- 172.21.15.42:0/2753648012 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b500cc2e0 con 0x7f4b500cb920 2024-05-28T14:05:22.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.986+0000 7f4b56200640 1 -- 172.21.15.42:0/2753648012 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b280052a0 con 0x7f4b500cb920 2024-05-28T14:05:22.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.990+0000 7f4b47400640 1 -- 172.21.15.42:0/2753648012 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 21) ==== 89109+0+0 (secure 0 0 0) 0x7f4b3c0bf030 con 0x7f4b500cb920 2024-05-28T14:05:22.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.990+0000 7f4b47400640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b300690b0 0x7f4b3006b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:22.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:22.990+0000 7f4b47400640 1 -- 172.21.15.42:0/2753648012 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f4b3c11ebb0 con 0x7f4b500cb920 2024-05-28T14:05:22.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:23.002+0000 7f4b47400640 1 -- 172.21.15.42:0/2753648012 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4b3c016840 con 0x7f4b500cb920 2024-05-28T14:05:23.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:23.022+0000 7f4b4f400640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b300690b0 0x7f4b3006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:23.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:23.054+0000 7f4b4f400640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b300690b0 0x7f4b3006b570 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4b5006a7c0 tx=0x7f4b40008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:23.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:23.650+0000 7f4b56200640 1 -- 172.21.15.42:0/2753648012 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone28225", "target": ["mon-mgr", ""]}) -- 0x7f4b28000bd0 con 0x7f4b300690b0 2024-05-28T14:05:24.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.574+0000 7f4b47400640 1 -- 172.21.15.42:0/2753648012 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4b28000bd0 con 0x7f4b300690b0 2024-05-28T14:05:24.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.578+0000 7f4b56200640 1 -- 172.21.15.42:0/2753648012 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b300690b0 msgr2=0x7f4b3006b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:24.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.578+0000 7f4b56200640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b300690b0 0x7f4b3006b570 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4b5006a7c0 tx=0x7f4b40008040 comp rx=0 tx=0).stop 2024-05-28T14:05:24.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.578+0000 7f4b56200640 1 -- 172.21.15.42:0/2753648012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b500cb920 msgr2=0x7f4b500cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:24.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.578+0000 7f4b56200640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b500cb920 0x7f4b500cbda0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009b90 tx=0x7f4b3c0198d0 comp rx=0 tx=0).stop 2024-05-28T14:05:24.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.578+0000 7f4b56200640 1 -- 172.21.15.42:0/2753648012 shutdown_connections 2024-05-28T14:05:24.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.578+0000 7f4b56200640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b300690b0 0x7f4b3006b570 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:24.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.578+0000 7f4b56200640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b50148f60 0x7f4b501493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:24.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.578+0000 7f4b56200640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b500cb920 0x7f4b500cbda0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:24.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.578+0000 7f4b56200640 1 --2- 172.21.15.42:0/2753648012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b34004810 0x7f4b500b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:24.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.578+0000 7f4b56200640 1 -- 172.21.15.42:0/2753648012 >> 172.21.15.42:0/2753648012 conn(0x7f4b500b21f0 msgr2=0x7f4b50163260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:24.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.578+0000 7f4b56200640 1 -- 172.21.15.42:0/2753648012 shutdown_connections 2024-05-28T14:05:24.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.578+0000 7f4b56200640 1 -- 172.21.15.42:0/2753648012 wait complete. 2024-05-28T14:05:24.634 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone28226 2024-05-28T14:05:24.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.994+0000 7f3dfa800640 1 Processor -- start 2024-05-28T14:05:24.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:24.994+0000 7f3dfa800640 1 -- start start 2024-05-28T14:05:24.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.002+0000 7f3dfa800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3df40b39a0 0x7f3df415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:24.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.002+0000 7f3df2a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f3df4151860 0x7f3df415e3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:49504/0 (socket says 172.21.15.42:49504) 2024-05-28T14:05:24.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.002+0000 7f3df2000640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f3df40b34f0 0x7f3df415c930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:46196/0 (socket says 172.21.15.42:46196) 2024-05-28T14:05:24.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.002+0000 7f3dfa800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3df4151c40 con 0x7f3df4151860 2024-05-28T14:05:24.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.002+0000 7f3df2a00640 1 -- 172.21.15.42:0/387535377 learned_addr learned my addr 172.21.15.42:0/387535377 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:05:24.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.002+0000 7f3dfa800640 1 -- 172.21.15.42:0/387535377 --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3df40b3e10 con 0x7f3df40b34f0 2024-05-28T14:05:24.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.002+0000 7f3dfa800640 1 -- 172.21.15.42:0/387535377 --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f3df415ee20 con 0x7f3df40b39a0 2024-05-28T14:05:24.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.002+0000 7f3df3400640 1 --2- 172.21.15.42:0/387535377 >> v2:172.21.15.42:3300/0 conn(0x7f3df40b39a0 0x7f3df415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:24.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.006+0000 7f3df3400640 1 -- 172.21.15.42:0/387535377 >> v1:172.21.15.63:6789/0 conn(0x7f3df40b34f0 legacy=0x7f3df415c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:24.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.010+0000 7f3df3400640 1 -- 172.21.15.42:0/387535377 >> v1:172.21.15.42:6789/0 conn(0x7f3df4151860 legacy=0x7f3df415e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:25.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.014+0000 7f3df9200640 1 -- 172.21.15.42:0/387535377 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1255956024 0 0) 0x7f3df4151c40 con 0x7f3df4151860 2024-05-28T14:05:25.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.014+0000 7f3df3400640 1 -- 172.21.15.42:0/387535377 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df415f100 con 0x7f3df40b39a0 2024-05-28T14:05:25.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.018+0000 7f3df3400640 1 --2- 172.21.15.42:0/387535377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df40b39a0 0x7f3df415d040 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3df4151e10 tx=0x7f3df415dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03feab3279632b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:25.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.022+0000 7f3df9200640 1 -- 172.21.15.42:0/387535377 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de80b2070 con 0x7f3df40b39a0 2024-05-28T14:05:25.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.022+0000 7f3df9200640 1 -- 172.21.15.42:0/387535377 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3de80a59e0 con 0x7f3df40b39a0 2024-05-28T14:05:25.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.022+0000 7f3df9200640 1 -- 172.21.15.42:0/387535377 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de80a5db0 con 0x7f3df40b39a0 2024-05-28T14:05:25.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.022+0000 7f3dfa800640 1 -- 172.21.15.42:0/387535377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df40b39a0 msgr2=0x7f3df415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:25.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.022+0000 7f3dfa800640 1 --2- 172.21.15.42:0/387535377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df40b39a0 0x7f3df415d040 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3df4151e10 tx=0x7f3df415dd90 comp rx=0 tx=0).stop 2024-05-28T14:05:25.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.022+0000 7f3dfa800640 1 -- 172.21.15.42:0/387535377 shutdown_connections 2024-05-28T14:05:25.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.022+0000 7f3dfa800640 1 --2- 172.21.15.42:0/387535377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df40b39a0 0x7f3df415d040 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:25.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.022+0000 7f3dfa800640 1 -- 172.21.15.42:0/387535377 >> 172.21.15.42:0/387535377 conn(0x7f3df4072220 msgr2=0x7f3df40b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:25.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.022+0000 7f3dfa800640 1 -- 172.21.15.42:0/387535377 shutdown_connections 2024-05-28T14:05:25.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.022+0000 7f3dfa800640 1 -- 172.21.15.42:0/387535377 wait complete. 2024-05-28T14:05:25.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3dfa800640 1 Processor -- start 2024-05-28T14:05:25.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3dfa800640 1 -- start start 2024-05-28T14:05:25.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3dfa800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3df40b34f0 0x7f3df4099860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:25.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3dfa800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df40b39a0 0x7f3df4144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:25.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3dfa800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3df4151860 0x7f3df4145250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:25.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3dfa800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3df415eab0 con 0x7f3df40b39a0 2024-05-28T14:05:25.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3dfa800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3df40b1a90 con 0x7f3df4151860 2024-05-28T14:05:25.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3dfa800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3df4168500 con 0x7f3df40b34f0 2024-05-28T14:05:25.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3df2a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3df40b34f0 0x7f3df4099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:25.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3df2a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3df40b34f0 0x7f3df4099860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:37848/0 (socket says 172.21.15.42:37848) 2024-05-28T14:05:25.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3df2a00640 1 -- 172.21.15.42:0/3747428085 learned_addr learned my addr 172.21.15.42:0/3747428085 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:25.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3df2000640 1 --2- 172.21.15.42:0/3747428085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df40b39a0 0x7f3df4144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:25.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3df2a00640 1 -- 172.21.15.42:0/3747428085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3df4151860 msgr2=0x7f3df4145250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:25.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3df2a00640 1 --2- 172.21.15.42:0/3747428085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3df4151860 0x7f3df4145250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:25.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3df2a00640 1 -- 172.21.15.42:0/3747428085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df40b39a0 msgr2=0x7f3df4144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:25.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3df2a00640 1 --2- 172.21.15.42:0/3747428085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df40b39a0 0x7f3df4144d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:25.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.026+0000 7f3df2a00640 1 -- 172.21.15.42:0/3747428085 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df4147070 con 0x7f3df40b34f0 2024-05-28T14:05:25.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.030+0000 7f3df2a00640 1 --2- 172.21.15.42:0/3747428085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3df40b34f0 0x7f3df4099860 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3de000c410 tx=0x7f3de000c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:25.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.030+0000 7f3de7400640 1 -- 172.21.15.42:0/3747428085 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00074e0 con 0x7f3df40b34f0 2024-05-28T14:05:25.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.030+0000 7f3de7400640 1 -- 172.21.15.42:0/3747428085 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3de00047f0 con 0x7f3df40b34f0 2024-05-28T14:05:25.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.030+0000 7f3de7400640 1 -- 172.21.15.42:0/3747428085 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0007680 con 0x7f3df40b34f0 2024-05-28T14:05:25.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.030+0000 7f3dfa800640 1 -- 172.21.15.42:0/3747428085 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df41467d0 con 0x7f3df40b34f0 2024-05-28T14:05:25.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.030+0000 7f3dfa800640 1 -- 172.21.15.42:0/3747428085 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df4146d90 con 0x7f3df40b34f0 2024-05-28T14:05:25.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.034+0000 7f3dfa800640 1 -- 172.21.15.42:0/3747428085 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df4068a90 con 0x7f3df40b34f0 2024-05-28T14:05:25.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.034+0000 7f3de7400640 1 -- 172.21.15.42:0/3747428085 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 21) ==== 89109+0+0 (secure 0 0 0) 0x7f3de0011070 con 0x7f3df40b34f0 2024-05-28T14:05:25.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.038+0000 7f3de7400640 1 --2- 172.21.15.42:0/3747428085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3dd40690b0 0x7f3dd406b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:25.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.038+0000 7f3de7400640 1 -- 172.21.15.42:0/3747428085 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f3de0085240 con 0x7f3df40b34f0 2024-05-28T14:05:25.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.046+0000 7f3de7400640 1 -- 172.21.15.42:0/3747428085 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3de0056fd0 con 0x7f3df40b34f0 2024-05-28T14:05:25.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.062+0000 7f3df2000640 1 --2- 172.21.15.42:0/3747428085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3dd40690b0 0x7f3dd406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:25.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.102+0000 7f3df2000640 1 --2- 172.21.15.42:0/3747428085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3dd40690b0 0x7f3dd406b570 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3df41656e0 tx=0x7f3ddc007c90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:25.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:25.574+0000 7f3dfa800640 1 -- 172.21.15.42:0/3747428085 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone28226", "target": ["mon-mgr", ""]}) -- 0x7f3df415a1c0 con 0x7f3dd40690b0 2024-05-28T14:05:26.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.034+0000 7f3de7400640 1 -- 172.21.15.42:0/3747428085 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3df415a1c0 con 0x7f3dd40690b0 2024-05-28T14:05:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.042+0000 7f3dfa800640 1 -- 172.21.15.42:0/3747428085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3dd40690b0 msgr2=0x7f3dd406b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.042+0000 7f3dfa800640 1 --2- 172.21.15.42:0/3747428085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3dd40690b0 0x7f3dd406b570 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3df41656e0 tx=0x7f3ddc007c90 comp rx=0 tx=0).stop 2024-05-28T14:05:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.042+0000 7f3dfa800640 1 -- 172.21.15.42:0/3747428085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3df40b34f0 msgr2=0x7f3df4099860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:26.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.042+0000 7f3dfa800640 1 --2- 172.21.15.42:0/3747428085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3df40b34f0 0x7f3df4099860 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3de000c410 tx=0x7f3de000c8d0 comp rx=0 tx=0).stop 2024-05-28T14:05:26.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.042+0000 7f3dfa800640 1 -- 172.21.15.42:0/3747428085 shutdown_connections 2024-05-28T14:05:26.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.042+0000 7f3dfa800640 1 --2- 172.21.15.42:0/3747428085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3dd40690b0 0x7f3dd406b570 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:26.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.042+0000 7f3dfa800640 1 --2- 172.21.15.42:0/3747428085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3df4151860 0x7f3df4145250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:26.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.042+0000 7f3dfa800640 1 --2- 172.21.15.42:0/3747428085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3df40b39a0 0x7f3df4144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:26.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.042+0000 7f3dfa800640 1 --2- 172.21.15.42:0/3747428085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3df40b34f0 0x7f3df4099860 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:26.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.042+0000 7f3dfa800640 1 -- 172.21.15.42:0/3747428085 >> 172.21.15.42:0/3747428085 conn(0x7f3df4072220 msgr2=0x7f3df415b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:26.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.042+0000 7f3dfa800640 1 -- 172.21.15.42:0/3747428085 shutdown_connections 2024-05-28T14:05:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.042+0000 7f3dfa800640 1 -- 172.21.15.42:0/3747428085 wait complete. 2024-05-28T14:05:26.068 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone28227 2024-05-28T14:05:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f6967000640 1 Processor -- start 2024-05-28T14:05:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f6967000640 1 -- start start 2024-05-28T14:05:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f6967000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69600ce3e0 0x7f69600b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f6967000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69600cbaa0 con 0x7f69600b2430 2024-05-28T14:05:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f6967000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69600b1ec0 con 0x7f696015ed70 2024-05-28T14:05:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f6967000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f69600cfcd0 con 0x7f69600ce3e0 2024-05-28T14:05:26.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f6965000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f69600b2430 0x7f69600d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:49508/0 (socket says 172.21.15.42:49508) 2024-05-28T14:05:26.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f6965000640 1 -- 172.21.15.42:0/377764482 learned_addr learned my addr 172.21.15.42:0/377764482 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:05:26.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695f400640 1 --2- 172.21.15.42:0/377764482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69600ce3e0 0x7f69600b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:26.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 -- 172.21.15.42:0/377764482 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3988575230 0 0) 0x7f69580014a0 con 0x7f69600b2430 2024-05-28T14:05:26.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 -- 172.21.15.42:0/377764482 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6944003530 con 0x7f69600b2430 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 -- 172.21.15.42:0/377764482 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 615629911 0 0) 0x7f6944003530 con 0x7f69600b2430 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 -- 172.21.15.42:0/377764482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69600ce3e0 msgr2=0x7f69600b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 --2- 172.21.15.42:0/377764482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69600ce3e0 0x7f69600b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 -- 172.21.15.42:0/377764482 >> v1:172.21.15.63:6789/0 conn(0x7f696015ed70 legacy=0x7f69600cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 -- 172.21.15.42:0/377764482 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69600b21a0 con 0x7f69600b2430 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 -- 172.21.15.42:0/377764482 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f6958001bd0 con 0x7f69600b2430 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 -- 172.21.15.42:0/377764482 >> v1:172.21.15.42:6789/0 conn(0x7f69600b2430 legacy=0x7f69600d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 --2- 172.21.15.42:0/377764482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69440039f0 0x7f6944003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 --2- 172.21.15.42:0/377764482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69600ce3e0 0x7f6944004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 --2- 172.21.15.42:0/377764482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6944004d50 0x7f69440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 -- 172.21.15.42:0/377764482 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69600cfcd0 con 0x7f6944004d50 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 -- 172.21.15.42:0/377764482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f694400bb60 con 0x7f69440039f0 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695ea00640 1 -- 172.21.15.42:0/377764482 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f694400df40 con 0x7f69600ce3e0 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f6965000640 1 --2- 172.21.15.42:0/377764482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6944004d50 0x7f69440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695fe00640 1 --2- 172.21.15.42:0/377764482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69440039f0 0x7f6944003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.362+0000 7f695f400640 1 --2- 172.21.15.42:0/377764482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69600ce3e0 0x7f6944004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.366+0000 7f6965000640 1 -- 172.21.15.42:0/377764482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69600ce3e0 msgr2=0x7f6944004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.366+0000 7f6965000640 1 --2- 172.21.15.42:0/377764482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69600ce3e0 0x7f6944004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.366+0000 7f6965000640 1 -- 172.21.15.42:0/377764482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69440039f0 msgr2=0x7f6944003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.366+0000 7f6965000640 1 --2- 172.21.15.42:0/377764482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69440039f0 0x7f6944003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.366+0000 7f6965000640 1 -- 172.21.15.42:0/377764482 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6944003530 con 0x7f6944004d50 2024-05-28T14:05:26.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.366+0000 7f6965000640 1 --2- 172.21.15.42:0/377764482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6944004d50 0x7f69440051d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f695800aa80 tx=0x7f695801d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81285094f6e59873 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:26.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.370+0000 7f695ea00640 1 -- 172.21.15.42:0/377764482 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695800a670 con 0x7f6944004d50 2024-05-28T14:05:26.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.370+0000 7f695ea00640 1 -- 172.21.15.42:0/377764482 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69580ad430 con 0x7f6944004d50 2024-05-28T14:05:26.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.370+0000 7f695ea00640 1 -- 172.21.15.42:0/377764482 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69580ad840 con 0x7f6944004d50 2024-05-28T14:05:26.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.370+0000 7f6967000640 1 -- 172.21.15.42:0/377764482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6944004d50 msgr2=0x7f69440051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:26.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.370+0000 7f6967000640 1 --2- 172.21.15.42:0/377764482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6944004d50 0x7f69440051d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f695800aa80 tx=0x7f695801d380 comp rx=0 tx=0).stop 2024-05-28T14:05:26.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.370+0000 7f6965000640 1 -- 172.21.15.42:0/377764482 reap_dead start 2024-05-28T14:05:26.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.370+0000 7f6967000640 1 -- 172.21.15.42:0/377764482 shutdown_connections 2024-05-28T14:05:26.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.370+0000 7f6967000640 1 -- 172.21.15.42:0/377764482 >> 172.21.15.42:0/377764482 conn(0x7f696009aa50 msgr2=0x7f69600b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:26.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.370+0000 7f6967000640 1 -- 172.21.15.42:0/377764482 shutdown_connections 2024-05-28T14:05:26.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.370+0000 7f6967000640 1 -- 172.21.15.42:0/377764482 wait complete. 2024-05-28T14:05:26.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.374+0000 7f6967000640 1 Processor -- start 2024-05-28T14:05:26.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.374+0000 7f6967000640 1 -- start start 2024-05-28T14:05:26.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6967000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6944004d50 0x7f69601ed380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:26.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6967000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69600ca820 0x7f69600cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:26.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6967000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69600c8520 0x7f69600c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:26.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6967000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69600cecf0 con 0x7f69600c8520 2024-05-28T14:05:26.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6967000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f69600cf960 con 0x7f69600ca820 2024-05-28T14:05:26.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6967000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f69600c8ec0 con 0x7f6944004d50 2024-05-28T14:05:26.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6965000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69600c8520 0x7f69600c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:26.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6965000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69600c8520 0x7f69600c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32822/0 (socket says 172.21.15.42:32822) 2024-05-28T14:05:26.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6965000640 1 -- 172.21.15.42:0/671261826 learned_addr learned my addr 172.21.15.42:0/671261826 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:26.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f695f400640 1 --2- 172.21.15.42:0/671261826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69600ca820 0x7f69600cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:26.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6965000640 1 -- 172.21.15.42:0/671261826 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6944004d50 msgr2=0x7f69601ed380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:26.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6965000640 1 --2- 172.21.15.42:0/671261826 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6944004d50 0x7f69601ed380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:26.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6965000640 1 -- 172.21.15.42:0/671261826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69600ca820 msgr2=0x7f69600cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:26.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6965000640 1 --2- 172.21.15.42:0/671261826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69600ca820 0x7f69600cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:26.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6965000640 1 -- 172.21.15.42:0/671261826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69600cb3a0 con 0x7f69600c8520 2024-05-28T14:05:26.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6965000640 1 --2- 172.21.15.42:0/671261826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69600c8520 0x7f69600c8980 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f695800aa20 tx=0x7f6944005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:26.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6957e00640 1 -- 172.21.15.42:0/671261826 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69580be070 con 0x7f69600c8520 2024-05-28T14:05:26.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6957e00640 1 -- 172.21.15.42:0/671261826 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69580b3040 con 0x7f69600c8520 2024-05-28T14:05:26.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.378+0000 7f6957e00640 1 -- 172.21.15.42:0/671261826 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695800a670 con 0x7f69600c8520 2024-05-28T14:05:26.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.382+0000 7f6967000640 1 -- 172.21.15.42:0/671261826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69600cb680 con 0x7f69600c8520 2024-05-28T14:05:26.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.382+0000 7f6967000640 1 -- 172.21.15.42:0/671261826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69600af470 con 0x7f69600c8520 2024-05-28T14:05:26.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.382+0000 7f6967000640 1 -- 172.21.15.42:0/671261826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69600aef80 con 0x7f69600c8520 2024-05-28T14:05:26.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.382+0000 7f6957e00640 1 -- 172.21.15.42:0/671261826 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) ==== 89109+0+0 (secure 0 0 0) 0x7f69580ad430 con 0x7f69600c8520 2024-05-28T14:05:26.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.382+0000 7f6957e00640 1 --2- 172.21.15.42:0/671261826 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6940068fe0 0x7f694006b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:26.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.386+0000 7f6957e00640 1 -- 172.21.15.42:0/671261826 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f695811f670 con 0x7f69600c8520 2024-05-28T14:05:26.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.386+0000 7f695fe00640 1 --2- 172.21.15.42:0/671261826 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6940068fe0 0x7f694006b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:26.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.386+0000 7f695fe00640 1 --2- 172.21.15.42:0/671261826 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6940068fe0 0x7f694006b4a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6950007fd0 tx=0x7f6950009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:26.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.390+0000 7f6957e00640 1 -- 172.21.15.42:0/671261826 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f69600cbaa0 con 0x7f69600c8520 2024-05-28T14:05:26.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:26.798+0000 7f6967000640 1 -- 172.21.15.42:0/671261826 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone28227", "target": ["mon-mgr", ""]}) -- 0x7f69600cb850 con 0x7f6940068fe0 2024-05-28T14:05:27.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.042+0000 7f6957e00640 1 -- 172.21.15.42:0/671261826 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6950001440 con 0x7f6940068fe0 2024-05-28T14:05:27.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.042+0000 7f6967000640 1 -- 172.21.15.42:0/671261826 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6940068fe0 msgr2=0x7f694006b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:27.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.042+0000 7f6967000640 1 --2- 172.21.15.42:0/671261826 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6940068fe0 0x7f694006b4a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6950007fd0 tx=0x7f6950009040 comp rx=0 tx=0).stop 2024-05-28T14:05:27.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.042+0000 7f6967000640 1 -- 172.21.15.42:0/671261826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69600c8520 msgr2=0x7f69600c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:27.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.042+0000 7f6967000640 1 --2- 172.21.15.42:0/671261826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69600c8520 0x7f69600c8980 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f695800aa20 tx=0x7f6944005840 comp rx=0 tx=0).stop 2024-05-28T14:05:27.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.042+0000 7f6967000640 1 -- 172.21.15.42:0/671261826 shutdown_connections 2024-05-28T14:05:27.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.042+0000 7f6967000640 1 --2- 172.21.15.42:0/671261826 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6940068fe0 0x7f694006b4a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.042+0000 7f6967000640 1 --2- 172.21.15.42:0/671261826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69600c8520 0x7f69600c8980 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.042+0000 7f6967000640 1 --2- 172.21.15.42:0/671261826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69600ca820 0x7f69600cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.042+0000 7f6967000640 1 --2- 172.21.15.42:0/671261826 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6944004d50 0x7f69601ed380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.042+0000 7f6967000640 1 -- 172.21.15.42:0/671261826 >> 172.21.15.42:0/671261826 conn(0x7f696009aa50 msgr2=0x7f69600d1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.042+0000 7f6967000640 1 -- 172.21.15.42:0/671261826 shutdown_connections 2024-05-28T14:05:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.042+0000 7f6967000640 1 -- 172.21.15.42:0/671261826 wait complete. 2024-05-28T14:05:27.051 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:05:27.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd628600640 1 Processor -- start 2024-05-28T14:05:27.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd628600640 1 -- start start 2024-05-28T14:05:27.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd628600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd62015bf30 0x7fd62015c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:27.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd628600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6200cf690 0x7fd62006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:27.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd628600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6200cfc10 con 0x7fd6200cf230 2024-05-28T14:05:27.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd628600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd6200721d0 con 0x7fd62015bf30 2024-05-28T14:05:27.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd628600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd6200cfef0 con 0x7fd6200cf690 2024-05-28T14:05:27.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd626600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6200cf690 0x7fd62006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:27.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd625c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd62015bf30 0x7fd62015c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:27.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd625200640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fd6200cf230 0x7fd620072660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:46212/0 (socket says 172.21.15.42:46212) 2024-05-28T14:05:27.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd626600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6200cf690 0x7fd62006ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:37890/0 (socket says 172.21.15.42:37890) 2024-05-28T14:05:27.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd625200640 1 -- 172.21.15.42:0/2584483080 learned_addr learned my addr 172.21.15.42:0/2584483080 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:05:27.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd625c00640 1 -- 172.21.15.42:0/2584483080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6200cf690 msgr2=0x7fd62006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:27.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd625c00640 1 --2- 172.21.15.42:0/2584483080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6200cf690 0x7fd62006ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd625c00640 1 -- 172.21.15.42:0/2584483080 >> v1:172.21.15.63:6789/0 conn(0x7fd6200cf230 legacy=0x7fd620072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:27.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.210+0000 7fd625c00640 1 -- 172.21.15.42:0/2584483080 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6200d01d0 con 0x7fd62015bf30 2024-05-28T14:05:27.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.210+0000 7fd626600640 1 --2- 172.21.15.42:0/2584483080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6200cf690 0x7fd62006ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:05:27.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.206+0000 7fd61fe00640 1 -- 172.21.15.42:0/2584483080 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2613451382 0 0) 0x7fd6200cfc10 con 0x7fd6200cf230 2024-05-28T14:05:27.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.210+0000 7fd625c00640 1 --2- 172.21.15.42:0/2584483080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd62015bf30 0x7fd62015c310 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd6140097c0 tx=0x7fd61401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78800efe5cb254ea server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:27.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.210+0000 7fd61fe00640 1 -- 172.21.15.42:0/2584483080 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6140b0070 con 0x7fd62015bf30 2024-05-28T14:05:27.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.210+0000 7fd61fe00640 1 -- 172.21.15.42:0/2584483080 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6140ab600 con 0x7fd62015bf30 2024-05-28T14:05:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.210+0000 7fd61fe00640 1 -- 172.21.15.42:0/2584483080 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6140ab9d0 con 0x7fd62015bf30 2024-05-28T14:05:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.210+0000 7fd628600640 1 -- 172.21.15.42:0/2584483080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd62015bf30 msgr2=0x7fd62015c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.210+0000 7fd628600640 1 --2- 172.21.15.42:0/2584483080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd62015bf30 0x7fd62015c310 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd6140097c0 tx=0x7fd61401c3e0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.210+0000 7fd628600640 1 -- 172.21.15.42:0/2584483080 shutdown_connections 2024-05-28T14:05:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.210+0000 7fd628600640 1 --2- 172.21.15.42:0/2584483080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6200cf690 0x7fd62006ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.210+0000 7fd628600640 1 --2- 172.21.15.42:0/2584483080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd62015bf30 0x7fd62015c310 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.210+0000 7fd628600640 1 -- 172.21.15.42:0/2584483080 >> 172.21.15.42:0/2584483080 conn(0x7fd62006b9f0 msgr2=0x7fd62006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd628600640 1 -- 172.21.15.42:0/2584483080 shutdown_connections 2024-05-28T14:05:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd628600640 1 -- 172.21.15.42:0/2584483080 wait complete. 2024-05-28T14:05:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd628600640 1 Processor -- start 2024-05-28T14:05:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd628600640 1 -- start start 2024-05-28T14:05:27.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd628600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6200cf230 0x7fd62014ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:27.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd628600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6200cf690 0x7fd6200b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:27.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd628600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd62015bf30 0x7fd6200b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:27.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd628600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd62006dcc0 con 0x7fd6200cf690 2024-05-28T14:05:27.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd628600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd62006db40 con 0x7fd6200cf230 2024-05-28T14:05:27.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd628600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd62006de40 con 0x7fd62015bf30 2024-05-28T14:05:27.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd625c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6200cf230 0x7fd62014ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:27.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd625200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6200cf690 0x7fd6200b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:27.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd626600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd62015bf30 0x7fd6200b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:27.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd625c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6200cf230 0x7fd62014ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:56900/0 (socket says 172.21.15.42:56900) 2024-05-28T14:05:27.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd625c00640 1 -- 172.21.15.42:0/3993892661 learned_addr learned my addr 172.21.15.42:0/3993892661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:27.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd625c00640 1 -- 172.21.15.42:0/3993892661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd62015bf30 msgr2=0x7fd6200b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:27.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd625c00640 1 --2- 172.21.15.42:0/3993892661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd62015bf30 0x7fd6200b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd625c00640 1 -- 172.21.15.42:0/3993892661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6200cf690 msgr2=0x7fd6200b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:27.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd625c00640 1 --2- 172.21.15.42:0/3993892661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6200cf690 0x7fd6200b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd625c00640 1 -- 172.21.15.42:0/3993892661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd620165fd0 con 0x7fd6200cf230 2024-05-28T14:05:27.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd625200640 1 --2- 172.21.15.42:0/3993892661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6200cf690 0x7fd6200b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:05:27.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.214+0000 7fd625c00640 1 --2- 172.21.15.42:0/3993892661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6200cf230 0x7fd62014ef00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd61401a010 tx=0x7fd614019180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:27.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.218+0000 7fd61d600640 1 -- 172.21.15.42:0/3993892661 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6140b8020 con 0x7fd6200cf230 2024-05-28T14:05:27.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.218+0000 7fd61d600640 1 -- 172.21.15.42:0/3993892661 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6140b4410 con 0x7fd6200cf230 2024-05-28T14:05:27.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.218+0000 7fd628600640 1 -- 172.21.15.42:0/3993892661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6200ae560 con 0x7fd6200cf230 2024-05-28T14:05:27.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.218+0000 7fd61d600640 1 -- 172.21.15.42:0/3993892661 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6140b0040 con 0x7fd6200cf230 2024-05-28T14:05:27.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.218+0000 7fd628600640 1 -- 172.21.15.42:0/3993892661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6200aeb20 con 0x7fd6200cf230 2024-05-28T14:05:27.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.218+0000 7fd628600640 1 -- 172.21.15.42:0/3993892661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6200af300 con 0x7fd6200cf230 2024-05-28T14:05:27.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.218+0000 7fd61d600640 1 -- 172.21.15.42:0/3993892661 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 21) ==== 89109+0+0 (secure 0 0 0) 0x7fd6140bb020 con 0x7fd6200cf230 2024-05-28T14:05:27.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.218+0000 7fd61d600640 1 --2- 172.21.15.42:0/3993892661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd600068fe0 0x7fd60006b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:27.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.218+0000 7fd61d600640 1 -- 172.21.15.42:0/3993892661 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fd61411ef30 con 0x7fd6200cf230 2024-05-28T14:05:27.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.218+0000 7fd625200640 1 --2- 172.21.15.42:0/3993892661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd600068fe0 0x7fd60006b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:27.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.218+0000 7fd625200640 1 --2- 172.21.15.42:0/3993892661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd600068fe0 0x7fd60006b4a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd6200abff0 tx=0x7fd60c007d60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:27.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.222+0000 7fd61d600640 1 -- 172.21.15.42:0/3993892661 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd6140c2030 con 0x7fd6200cf230 2024-05-28T14:05:27.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.410+0000 7fd628600640 1 -- 172.21.15.42:0/3993892661 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fd6200aef50 con 0x7fd600068fe0 2024-05-28T14:05:27.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.410+0000 7fd61d600640 1 -- 172.21.15.42:0/3993892661 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fd6200aef50 con 0x7fd600068fe0 2024-05-28T14:05:27.398 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:05:27.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:27.398 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:05:27.398 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:27.399 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:05:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.414+0000 7fd628600640 1 -- 172.21.15.42:0/3993892661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd600068fe0 msgr2=0x7fd60006b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.414+0000 7fd628600640 1 --2- 172.21.15.42:0/3993892661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd600068fe0 0x7fd60006b4a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd6200abff0 tx=0x7fd60c007d60 comp rx=0 tx=0).stop 2024-05-28T14:05:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.414+0000 7fd628600640 1 -- 172.21.15.42:0/3993892661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6200cf230 msgr2=0x7fd62014ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.414+0000 7fd628600640 1 --2- 172.21.15.42:0/3993892661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6200cf230 0x7fd62014ef00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd61401a010 tx=0x7fd614019180 comp rx=0 tx=0).stop 2024-05-28T14:05:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.414+0000 7fd628600640 1 -- 172.21.15.42:0/3993892661 shutdown_connections 2024-05-28T14:05:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.414+0000 7fd628600640 1 --2- 172.21.15.42:0/3993892661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd600068fe0 0x7fd60006b4a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.414+0000 7fd628600640 1 --2- 172.21.15.42:0/3993892661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd62015bf30 0x7fd6200b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.414+0000 7fd628600640 1 --2- 172.21.15.42:0/3993892661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6200cf690 0x7fd6200b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.414+0000 7fd628600640 1 --2- 172.21.15.42:0/3993892661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6200cf230 0x7fd62014ef00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.414+0000 7fd628600640 1 -- 172.21.15.42:0/3993892661 >> 172.21.15.42:0/3993892661 conn(0x7fd62006b9f0 msgr2=0x7fd620151b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:27.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.414+0000 7fd628600640 1 -- 172.21.15.42:0/3993892661 shutdown_connections 2024-05-28T14:05:27.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.414+0000 7fd628600640 1 -- 172.21.15.42:0/3993892661 wait complete. 2024-05-28T14:05:27.421 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:05:27.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.574+0000 7f9f76800640 1 Processor -- start 2024-05-28T14:05:27.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.574+0000 7f9f76800640 1 -- start start 2024-05-28T14:05:27.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.574+0000 7f9f76800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9f7015e9d0 0x7f9f700d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:27.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.574+0000 7f9f76800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f700b52d0 0x7f9f700d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:27.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.574+0000 7f9f76800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f700ccce0 con 0x7f9f7009aa50 2024-05-28T14:05:27.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.574+0000 7f9f76800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9f700ccfc0 con 0x7f9f7015e9d0 2024-05-28T14:05:27.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.574+0000 7f9f76800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9f700cd2a0 con 0x7f9f700b52d0 2024-05-28T14:05:27.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.578+0000 7f9f6f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9f7015e9d0 0x7f9f700d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:27.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.578+0000 7f9f6f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9f7015e9d0 0x7f9f700d12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59358/0 (socket says 172.21.15.42:59358) 2024-05-28T14:05:27.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.578+0000 7f9f6f400640 1 -- 172.21.15.42:0/2127014027 learned_addr learned my addr 172.21.15.42:0/2127014027 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:27.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.578+0000 7f9f6ea00640 1 --2- 172.21.15.42:0/2127014027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f700b52d0 0x7f9f700d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:27.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.578+0000 7f9f6f400640 1 -- 172.21.15.42:0/2127014027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f700b52d0 msgr2=0x7f9f700d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:27.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.578+0000 7f9f6f400640 1 --2- 172.21.15.42:0/2127014027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f700b52d0 0x7f9f700d1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.578+0000 7f9f6f400640 1 -- 172.21.15.42:0/2127014027 >> v1:172.21.15.63:6789/0 conn(0x7f9f7009aa50 legacy=0x7f9f700b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:27.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.578+0000 7f9f6f400640 1 -- 172.21.15.42:0/2127014027 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f700cd580 con 0x7f9f7015e9d0 2024-05-28T14:05:27.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.578+0000 7f9f6f400640 1 --2- 172.21.15.42:0/2127014027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7015e9d0 0x7f9f700d12c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009870 tx=0x7f9f6001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c1e689f1074fdc server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:27.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f6e000640 1 -- 172.21.15.42:0/2127014027 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f600b2070 con 0x7f9f7015e9d0 2024-05-28T14:05:27.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f6e000640 1 -- 172.21.15.42:0/2127014027 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f600ad640 con 0x7f9f7015e9d0 2024-05-28T14:05:27.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f6e000640 1 -- 172.21.15.42:0/2127014027 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f600ada10 con 0x7f9f7015e9d0 2024-05-28T14:05:27.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 -- 172.21.15.42:0/2127014027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7015e9d0 msgr2=0x7f9f700d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:27.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 --2- 172.21.15.42:0/2127014027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7015e9d0 0x7f9f700d12c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009870 tx=0x7f9f6001c520 comp rx=0 tx=0).stop 2024-05-28T14:05:27.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 -- 172.21.15.42:0/2127014027 shutdown_connections 2024-05-28T14:05:27.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 --2- 172.21.15.42:0/2127014027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f700b52d0 0x7f9f700d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 --2- 172.21.15.42:0/2127014027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f7015e9d0 0x7f9f700d12c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 -- 172.21.15.42:0/2127014027 >> 172.21.15.42:0/2127014027 conn(0x7f9f700b3f70 msgr2=0x7f9f700b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:27.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 -- 172.21.15.42:0/2127014027 shutdown_connections 2024-05-28T14:05:27.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 -- 172.21.15.42:0/2127014027 wait complete. 2024-05-28T14:05:27.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 Processor -- start 2024-05-28T14:05:27.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 -- start start 2024-05-28T14:05:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f7009aa50 0x7f9f70150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f700b52d0 0x7f9f7015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f7015e9d0 0x7f9f700d0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9f700b3d30 con 0x7f9f700b52d0 2024-05-28T14:05:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f700b3bb0 con 0x7f9f7009aa50 2024-05-28T14:05:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f76800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9f700ce590 con 0x7f9f7015e9d0 2024-05-28T14:05:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f6ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f700b52d0 0x7f9f7015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f6ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f700b52d0 0x7f9f7015b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59362/0 (socket says 172.21.15.42:59362) 2024-05-28T14:05:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f6ea00640 1 -- 172.21.15.42:0/1826671658 learned_addr learned my addr 172.21.15.42:0/1826671658 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.582+0000 7f9f6fe00640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f7015e9d0 0x7f9f700d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f6f400640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f7009aa50 0x7f9f70150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f6ea00640 1 -- 172.21.15.42:0/1826671658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f7015e9d0 msgr2=0x7f9f700d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f6ea00640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f7015e9d0 0x7f9f700d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f6ea00640 1 -- 172.21.15.42:0/1826671658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f7009aa50 msgr2=0x7f9f70150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f6ea00640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f7009aa50 0x7f9f70150b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f6ea00640 1 -- 172.21.15.42:0/1826671658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7015bce0 con 0x7f9f700b52d0 2024-05-28T14:05:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f6fe00640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f7015e9d0 0x7f9f700d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:05:27.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f6ea00640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f700b52d0 0x7f9f7015b3e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00d800 tx=0x7f9f5c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:27.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f67400640 1 -- 172.21.15.42:0/1826671658 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00b6c0 con 0x7f9f700b52d0 2024-05-28T14:05:27.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f67400640 1 -- 172.21.15.42:0/1826671658 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5c019070 con 0x7f9f700b52d0 2024-05-28T14:05:27.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f67400640 1 -- 172.21.15.42:0/1826671658 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00baa0 con 0x7f9f700b52d0 2024-05-28T14:05:27.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f76800640 1 -- 172.21.15.42:0/1826671658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7015bfc0 con 0x7f9f700b52d0 2024-05-28T14:05:27.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f76800640 1 -- 172.21.15.42:0/1826671658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f701591a0 con 0x7f9f700b52d0 2024-05-28T14:05:27.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f76800640 1 -- 172.21.15.42:0/1826671658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f70068a90 con 0x7f9f700b52d0 2024-05-28T14:05:27.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.586+0000 7f9f67400640 1 -- 172.21.15.42:0/1826671658 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) ==== 89109+0+0 (secure 0 0 0) 0x7f9f5c00bc40 con 0x7f9f700b52d0 2024-05-28T14:05:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.590+0000 7f9f67400640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f50068fe0 0x7f9f5006b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.590+0000 7f9f67400640 1 -- 172.21.15.42:0/1826671658 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f9f5c085720 con 0x7f9f700b52d0 2024-05-28T14:05:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.590+0000 7f9f6f400640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f50068fe0 0x7f9f5006b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:27.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.590+0000 7f9f6f400640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f50068fe0 0x7f9f5006b4a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9f600af000 tx=0x7f9f60008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:27.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.594+0000 7f9f67400640 1 -- 172.21.15.42:0/1826671658 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9f5c057790 con 0x7f9f700b52d0 2024-05-28T14:05:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.826+0000 7f9f76800640 1 -- 172.21.15.42:0/1826671658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9f700b43f0 con 0x7f9f700b52d0 2024-05-28T14:05:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.826+0000 7f9f67400640 1 -- 172.21.15.42:0/1826671658 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2874 (secure 0 0 0) 0x7f9f5c057090 con 0x7f9f700b52d0 2024-05-28T14:05:27.815 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:05:27.815 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:05:27.816 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:05:27.816 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:05:27.816 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:05:27.816 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:05:27.816 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:05:27.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:27.816 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:05:27.816 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:05:27.816 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:05:27.816 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:05:27.816 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:05:27.816 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:27.816 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:05:27.816 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 449, 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 61, 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:05:27.817 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4635, 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 19279061182, 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 38216650752, 2024-05-28T14:05:27.818 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 734877462528, 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 563753206, 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 564152729, 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 137, 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 150 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 37, 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4, 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:27.819 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 4, 2024-05-28T14:05:27.820 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:05:27.820 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:05:27.820 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:05:27.820 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 6420 2024-05-28T14:05:27.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:27.820 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:27.820 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:05:27.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:27.820 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:05:27.820 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:05:27.820 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:05:27.820 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:05:27.820 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:05:27.820 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 11, 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:05:18.969158+0000", 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: "daemons": { 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: "summary": "", 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: "b": { 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:05:27.821 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: "58c696cb-4bfb-45e4-a3e5-43a164bc676d": { 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "1 ongoing+pending clones; Avg progress is 27.865% (16s)\n [=======.....................] (remaining: 43s)", 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.27864998579025269, 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: "f63639d3-a8b8-449e-aca8-3322f44f1d24": { 2024-05-28T14:05:27.822 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "1 ongoing clones; Avg progress is 27.865% (26s)\n [=======.....................] (remaining: 67s)", 2024-05-28T14:05:27.823 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.27864998579025269, 2024-05-28T14:05:27.823 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:05:27.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:27.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:27.823 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:05:27.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.830+0000 7f9f76800640 1 -- 172.21.15.42:0/1826671658 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f50068fe0 msgr2=0x7f9f5006b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:27.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.830+0000 7f9f76800640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f50068fe0 0x7f9f5006b4a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9f600af000 tx=0x7f9f60008040 comp rx=0 tx=0).stop 2024-05-28T14:05:27.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.830+0000 7f9f76800640 1 -- 172.21.15.42:0/1826671658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f700b52d0 msgr2=0x7f9f7015b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:27.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.830+0000 7f9f76800640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f700b52d0 0x7f9f7015b3e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00d800 tx=0x7f9f5c00dcc0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.830+0000 7f9f76800640 1 -- 172.21.15.42:0/1826671658 shutdown_connections 2024-05-28T14:05:27.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.830+0000 7f9f76800640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9f50068fe0 0x7f9f5006b4a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.830+0000 7f9f76800640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f7015e9d0 0x7f9f700d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.830+0000 7f9f76800640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f700b52d0 0x7f9f7015b3e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.830+0000 7f9f76800640 1 --2- 172.21.15.42:0/1826671658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f7009aa50 0x7f9f70150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:27.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.830+0000 7f9f76800640 1 -- 172.21.15.42:0/1826671658 >> 172.21.15.42:0/1826671658 conn(0x7f9f700b3f70 msgr2=0x7f9f700d1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:27.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.830+0000 7f9f76800640 1 -- 172.21.15.42:0/1826671658 shutdown_connections 2024-05-28T14:05:27.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:27.830+0000 7f9f76800640 1 -- 172.21.15.42:0/1826671658 wait complete. 2024-05-28T14:05:32.836 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:05:32.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.002+0000 7f7c79400640 1 Processor -- start 2024-05-28T14:05:32.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.002+0000 7f7c79400640 1 -- start start 2024-05-28T14:05:32.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c79400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7c7409b160 0x7f7c740b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:32.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c79400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c740ccce0 con 0x7f7c7415e9d0 2024-05-28T14:05:32.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c79400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c740ccfc0 con 0x7f7c740b52d0 2024-05-28T14:05:32.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c79400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7c740cd2a0 con 0x7f7c7409b160 2024-05-28T14:05:32.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c72800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f7c7415e9d0 0x7f7c740d12c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:54214/0 (socket says 172.21.15.42:54214) 2024-05-28T14:05:32.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c72800640 1 -- 172.21.15.42:0/1706861364 learned_addr learned my addr 172.21.15.42:0/1706861364 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:05:32.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c73200640 1 --2- 172.21.15.42:0/1706861364 >> v2:172.21.15.63:3300/0 conn(0x7f7c7409b160 0x7f7c740b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:32.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 115589286 0 0) 0x7f7c740ccce0 con 0x7f7c7415e9d0 2024-05-28T14:05:32.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c58003530 con 0x7f7c7415e9d0 2024-05-28T14:05:32.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3834864949 0 0) 0x7f7c740ccfc0 con 0x7f7c740b52d0 2024-05-28T14:05:32.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c740ccce0 con 0x7f7c740b52d0 2024-05-28T14:05:32.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2214661728 0 0) 0x7f7c58003530 con 0x7f7c7415e9d0 2024-05-28T14:05:32.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 >> v2:172.21.15.63:3300/0 conn(0x7f7c7409b160 msgr2=0x7f7c740b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:32.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 --2- 172.21.15.42:0/1706861364 >> v2:172.21.15.63:3300/0 conn(0x7f7c7409b160 0x7f7c740b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:32.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 >> v1:172.21.15.63:6789/0 conn(0x7f7c740b52d0 legacy=0x7f7c7409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:32.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c740cd580 con 0x7f7c7415e9d0 2024-05-28T14:05:32.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f7c64001680 con 0x7f7c7415e9d0 2024-05-28T14:05:32.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 >> v1:172.21.15.42:6789/0 conn(0x7f7c7415e9d0 legacy=0x7f7c740d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:32.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 --2- 172.21.15.42:0/1706861364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c58003b50 0x7f7c58003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 --2- 172.21.15.42:0/1706861364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c58004810 0x7f7c58004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 --2- 172.21.15.42:0/1706861364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c580051d0 0x7f7c5800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7c740cd2a0 con 0x7f7c58003b50 2024-05-28T14:05:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5800c050 con 0x7f7c58004810 2024-05-28T14:05:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5800c250 con 0x7f7c580051d0 2024-05-28T14:05:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c72800640 1 --2- 172.21.15.42:0/1706861364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c58003b50 0x7f7c58003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c69e00640 1 --2- 172.21.15.42:0/1706861364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c58004810 0x7f7c58004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c73200640 1 --2- 172.21.15.42:0/1706861364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c580051d0 0x7f7c5800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c72800640 1 -- 172.21.15.42:0/1706861364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c580051d0 msgr2=0x7f7c5800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c72800640 1 --2- 172.21.15.42:0/1706861364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c580051d0 0x7f7c5800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c72800640 1 -- 172.21.15.42:0/1706861364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c58004810 msgr2=0x7f7c58004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c72800640 1 --2- 172.21.15.42:0/1706861364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c58004810 0x7f7c58004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c72800640 1 -- 172.21.15.42:0/1706861364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c58003530 con 0x7f7c58003b50 2024-05-28T14:05:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.006+0000 7f7c73200640 1 -- 172.21.15.42:0/1706861364 reap_dead start 2024-05-28T14:05:32.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.010+0000 7f7c72800640 1 --2- 172.21.15.42:0/1706861364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c58003b50 0x7f7c58003f50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400c190 tx=0x7f7c6401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02f8b6cc219d66c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:32.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.010+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64007cd0 con 0x7f7c58003b50 2024-05-28T14:05:32.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.010+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c640a9020 con 0x7f7c58003b50 2024-05-28T14:05:32.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.010+0000 7f7c71e00640 1 -- 172.21.15.42:0/1706861364 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64005ae0 con 0x7f7c58003b50 2024-05-28T14:05:32.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.010+0000 7f7c79400640 1 -- 172.21.15.42:0/1706861364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c58003b50 msgr2=0x7f7c58003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:32.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.010+0000 7f7c79400640 1 --2- 172.21.15.42:0/1706861364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c58003b50 0x7f7c58003f50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400c190 tx=0x7f7c6401cc20 comp rx=0 tx=0).stop 2024-05-28T14:05:32.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.010+0000 7f7c79400640 1 -- 172.21.15.42:0/1706861364 shutdown_connections 2024-05-28T14:05:32.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.010+0000 7f7c79400640 1 --2- 172.21.15.42:0/1706861364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c58003b50 0x7f7c58003f50 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:32.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.010+0000 7f7c79400640 1 -- 172.21.15.42:0/1706861364 >> 172.21.15.42:0/1706861364 conn(0x7f7c740b3f70 msgr2=0x7f7c740b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:32.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.010+0000 7f7c79400640 1 -- 172.21.15.42:0/1706861364 shutdown_connections 2024-05-28T14:05:32.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.010+0000 7f7c79400640 1 -- 172.21.15.42:0/1706861364 wait complete. 2024-05-28T14:05:32.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.010+0000 7f7c79400640 1 Processor -- start 2024-05-28T14:05:32.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.010+0000 7f7c79400640 1 -- start start 2024-05-28T14:05:33.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c79400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c58003b50 0x7f7c74150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:33.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c79400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c7415f9d0 0x7f7c7415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:33.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c79400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c7415fe50 0x7f7c7415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:33.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c79400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7c740b3210 con 0x7f7c58003b50 2024-05-28T14:05:33.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c79400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7c740b3090 con 0x7f7c7415f9d0 2024-05-28T14:05:33.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c79400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7c740b3390 con 0x7f7c7415fe50 2024-05-28T14:05:33.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c72800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c58003b50 0x7f7c74150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:33.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c72800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c58003b50 0x7f7c74150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59388/0 (socket says 172.21.15.42:59388) 2024-05-28T14:05:33.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c72800640 1 -- 172.21.15.42:0/2811054764 learned_addr learned my addr 172.21.15.42:0/2811054764 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:33.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c69e00640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c7415f9d0 0x7f7c7415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:33.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c73200640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c7415fe50 0x7f7c7415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:33.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c72800640 1 -- 172.21.15.42:0/2811054764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c7415fe50 msgr2=0x7f7c7415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:33.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c72800640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c7415fe50 0x7f7c7415d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:33.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c72800640 1 -- 172.21.15.42:0/2811054764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c7415f9d0 msgr2=0x7f7c7415cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:33.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c72800640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c7415f9d0 0x7f7c7415cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:33.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c72800640 1 -- 172.21.15.42:0/2811054764 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c740ca680 con 0x7f7c58003b50 2024-05-28T14:05:33.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c73200640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c7415fe50 0x7f7c7415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:05:33.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c69e00640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c7415f9d0 0x7f7c7415cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:05:33.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c72800640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c58003b50 0x7f7c74150010 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7c6401ca50 tx=0x7f7c58005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:33.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c6b400640 1 -- 172.21.15.42:0/2811054764 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c640b4070 con 0x7f7c58003b50 2024-05-28T14:05:33.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c6b400640 1 -- 172.21.15.42:0/2811054764 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c640070d0 con 0x7f7c58003b50 2024-05-28T14:05:33.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c6b400640 1 -- 172.21.15.42:0/2811054764 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64007cd0 con 0x7f7c58003b50 2024-05-28T14:05:33.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c79400640 1 -- 172.21.15.42:0/2811054764 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c74153f30 con 0x7f7c58003b50 2024-05-28T14:05:33.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c79400640 1 -- 172.21.15.42:0/2811054764 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c741544f0 con 0x7f7c58003b50 2024-05-28T14:05:33.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.014+0000 7f7c79400640 1 -- 172.21.15.42:0/2811054764 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c74068a90 con 0x7f7c58003b50 2024-05-28T14:05:33.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.018+0000 7f7c6b400640 1 -- 172.21.15.42:0/2811054764 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) ==== 89109+0+0 (secure 0 0 0) 0x7f7c640bf060 con 0x7f7c58003b50 2024-05-28T14:05:33.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.018+0000 7f7c6b400640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c50068fe0 0x7f7c5006b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:33.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.018+0000 7f7c6b400640 1 -- 172.21.15.42:0/2811054764 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f7c6411e1d0 con 0x7f7c58003b50 2024-05-28T14:05:33.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.018+0000 7f7c69e00640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c50068fe0 0x7f7c5006b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:33.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.018+0000 7f7c69e00640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c50068fe0 0x7f7c5006b4a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7c7406ff60 tx=0x7f7c5c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:33.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.022+0000 7f7c6b400640 1 -- 172.21.15.42:0/2811054764 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7c64016930 con 0x7f7c58003b50 2024-05-28T14:05:33.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.250+0000 7f7c79400640 1 -- 172.21.15.42:0/2811054764 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7c7406bdc0 con 0x7f7c58003b50 2024-05-28T14:05:33.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.250+0000 7f7c6b400640 1 -- 172.21.15.42:0/2811054764 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2379 (secure 0 0 0) 0x7f7c640effc0 con 0x7f7c58003b50 2024-05-28T14:05:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:05:33.236 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:05:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:05:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:05:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:05:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:05:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:05:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:05:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:05:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:05:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:05:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:05:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:05:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:05:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:05:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:05:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 454, 2024-05-28T14:05:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:05:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:05:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:05:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:05:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:05:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 61, 2024-05-28T14:05:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:05:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:05:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:05:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:05:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4957, 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 20646570141, 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 41715187712, 2024-05-28T14:05:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 731378925568, 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 675036666, 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 678245247, 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 165, 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 184 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 37, 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4, 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 4, 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 6420 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:05:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:05:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:05:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:05:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 12, 2024-05-28T14:05:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:05:28.979186+0000", 2024-05-28T14:05:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:05:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:05:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "58c696cb-4bfb-45e4-a3e5-43a164bc676d": { 2024-05-28T14:05:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 27.0 seconds before exiting (20s)\n [============================] ", 2024-05-28T14:05:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:05:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:05:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "f63639d3-a8b8-449e-aca8-3322f44f1d24": { 2024-05-28T14:05:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 27.0 seconds before exiting (30s)\n [============================] ", 2024-05-28T14:05:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:05:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:05:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:33.243 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:05:33.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.250+0000 7f7c79400640 1 -- 172.21.15.42:0/2811054764 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c50068fe0 msgr2=0x7f7c5006b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:33.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.250+0000 7f7c79400640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c50068fe0 0x7f7c5006b4a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7c7406ff60 tx=0x7f7c5c009040 comp rx=0 tx=0).stop 2024-05-28T14:05:33.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.250+0000 7f7c79400640 1 -- 172.21.15.42:0/2811054764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c58003b50 msgr2=0x7f7c74150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:33.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.250+0000 7f7c79400640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c58003b50 0x7f7c74150010 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7c6401ca50 tx=0x7f7c58005840 comp rx=0 tx=0).stop 2024-05-28T14:05:33.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.250+0000 7f7c79400640 1 -- 172.21.15.42:0/2811054764 shutdown_connections 2024-05-28T14:05:33.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.250+0000 7f7c79400640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c50068fe0 0x7f7c5006b4a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:33.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.250+0000 7f7c79400640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c7415fe50 0x7f7c7415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:33.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.250+0000 7f7c79400640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c7415f9d0 0x7f7c7415cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:33.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.250+0000 7f7c79400640 1 --2- 172.21.15.42:0/2811054764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c58003b50 0x7f7c74150010 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:33.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.250+0000 7f7c79400640 1 -- 172.21.15.42:0/2811054764 >> 172.21.15.42:0/2811054764 conn(0x7f7c740b3f70 msgr2=0x7f7c74163e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:33.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.250+0000 7f7c79400640 1 -- 172.21.15.42:0/2811054764 shutdown_connections 2024-05-28T14:05:33.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:33.250+0000 7f7c79400640 1 -- 172.21.15.42:0/2811054764 wait complete. 2024-05-28T14:05:38.261 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:05:38.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.426+0000 7f64ad200640 1 Processor -- start 2024-05-28T14:05:38.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.426+0000 7f64ad200640 1 -- start start 2024-05-28T14:05:38.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64ad200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f64a81559d0 0x7f64a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:38.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64ad200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64a80b52d0 0x7f64a815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:38.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64ad200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64a80cfac0 con 0x7f64a815c0a0 2024-05-28T14:05:38.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64ad200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f64a80cfda0 con 0x7f64a81559d0 2024-05-28T14:05:38.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64ad200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f64a80d0080 con 0x7f64a80b52d0 2024-05-28T14:05:38.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a7e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f64a815c0a0 0x7f64a815edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34612/0 (socket says 172.21.15.42:34612) 2024-05-28T14:05:38.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a7e00640 1 -- 172.21.15.42:0/3529268181 learned_addr learned my addr 172.21.15.42:0/3529268181 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:05:38.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f649ea00640 1 --2- 172.21.15.42:0/3529268181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64a80b52d0 0x7f64a815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 -- 172.21.15.42:0/3529268181 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2382085195 0 0) 0x7f64a80cfac0 con 0x7f64a815c0a0 2024-05-28T14:05:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a7400640 1 --2- 172.21.15.42:0/3529268181 >> v2:172.21.15.63:3300/0 conn(0x7f64a81559d0 0x7f64a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 -- 172.21.15.42:0/3529268181 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f648c003530 con 0x7f64a815c0a0 2024-05-28T14:05:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 -- 172.21.15.42:0/3529268181 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1533065416 0 0) 0x7f648c003530 con 0x7f64a815c0a0 2024-05-28T14:05:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 -- 172.21.15.42:0/3529268181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64a80b52d0 msgr2=0x7f64a815bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 --2- 172.21.15.42:0/3529268181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64a80b52d0 0x7f64a815bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 -- 172.21.15.42:0/3529268181 >> v2:172.21.15.63:3300/0 conn(0x7f64a81559d0 msgr2=0x7f64a80b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 --2- 172.21.15.42:0/3529268181 >> v2:172.21.15.63:3300/0 conn(0x7f64a81559d0 0x7f64a80b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 -- 172.21.15.42:0/3529268181 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a80d0360 con 0x7f64a815c0a0 2024-05-28T14:05:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 -- 172.21.15.42:0/3529268181 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f64a0001a00 con 0x7f64a815c0a0 2024-05-28T14:05:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 -- 172.21.15.42:0/3529268181 >> v1:172.21.15.42:6789/0 conn(0x7f64a815c0a0 legacy=0x7f64a815edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 --2- 172.21.15.42:0/3529268181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64a80b52d0 0x7f648c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 --2- 172.21.15.42:0/3529268181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f648c003f40 0x7f648c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 --2- 172.21.15.42:0/3529268181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f648c004d50 0x7f648c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 -- 172.21.15.42:0/3529268181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f64a80cfda0 con 0x7f648c004d50 2024-05-28T14:05:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 -- 172.21.15.42:0/3529268181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64a80d0080 con 0x7f648c003f40 2024-05-28T14:05:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a6a00640 1 -- 172.21.15.42:0/3529268181 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f648c00b9d0 con 0x7f64a80b52d0 2024-05-28T14:05:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a7e00640 1 --2- 172.21.15.42:0/3529268181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f648c004d50 0x7f648c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a7400640 1 --2- 172.21.15.42:0/3529268181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64a80b52d0 0x7f648c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f649ea00640 1 --2- 172.21.15.42:0/3529268181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f648c003f40 0x7f648c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a7e00640 1 -- 172.21.15.42:0/3529268181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64a80b52d0 msgr2=0x7f648c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a7e00640 1 --2- 172.21.15.42:0/3529268181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64a80b52d0 0x7f648c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a7e00640 1 -- 172.21.15.42:0/3529268181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f648c003f40 msgr2=0x7f648c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a7e00640 1 --2- 172.21.15.42:0/3529268181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f648c003f40 0x7f648c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a7e00640 1 -- 172.21.15.42:0/3529268181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c003530 con 0x7f648c004d50 2024-05-28T14:05:38.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.430+0000 7f64a7400640 1 --2- 172.21.15.42:0/3529268181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64a80b52d0 0x7f648c0039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:05:38.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.434+0000 7f64a7e00640 1 --2- 172.21.15.42:0/3529268181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f648c004d50 0x7f648c0051d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f64a000a8a0 tx=0x7f64a001d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475d85487c3df042 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:38.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.434+0000 7f64a6a00640 1 -- 172.21.15.42:0/3529268181 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a000a4c0 con 0x7f648c004d50 2024-05-28T14:05:38.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.434+0000 7f64a6a00640 1 -- 172.21.15.42:0/3529268181 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a00a9020 con 0x7f648c004d50 2024-05-28T14:05:38.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.434+0000 7f64a6a00640 1 -- 172.21.15.42:0/3529268181 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a00072d0 con 0x7f648c004d50 2024-05-28T14:05:38.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.434+0000 7f64ad200640 1 -- 172.21.15.42:0/3529268181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f648c004d50 msgr2=0x7f648c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:38.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.434+0000 7f64ad200640 1 --2- 172.21.15.42:0/3529268181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f648c004d50 0x7f648c0051d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f64a000a8a0 tx=0x7f64a001d360 comp rx=0 tx=0).stop 2024-05-28T14:05:38.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.434+0000 7f64a7e00640 1 -- 172.21.15.42:0/3529268181 reap_dead start 2024-05-28T14:05:38.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.434+0000 7f64ad200640 1 -- 172.21.15.42:0/3529268181 shutdown_connections 2024-05-28T14:05:38.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.434+0000 7f64ad200640 1 -- 172.21.15.42:0/3529268181 >> 172.21.15.42:0/3529268181 conn(0x7f64a8073280 msgr2=0x7f64a806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:38.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.434+0000 7f64ad200640 1 -- 172.21.15.42:0/3529268181 shutdown_connections 2024-05-28T14:05:38.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.434+0000 7f64ad200640 1 -- 172.21.15.42:0/3529268181 wait complete. 2024-05-28T14:05:38.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f64ad200640 1 Processor -- start 2024-05-28T14:05:38.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f64ad200640 1 -- start start 2024-05-28T14:05:38.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f64ad200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f648c004d50 0x7f64a814de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:38.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f64ad200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64a8159730 0x7f64a81f0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:38.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f64ad200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64a81576d0 0x7f64a8157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:38.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f64ad200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f64a80cf770 con 0x7f64a8159730 2024-05-28T14:05:38.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f64ad200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64a815d990 con 0x7f64a81576d0 2024-05-28T14:05:38.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f64ad200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f64a815c530 con 0x7f648c004d50 2024-05-28T14:05:38.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f649ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64a8159730 0x7f64a81f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:38.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f649ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64a8159730 0x7f64a81f0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43548/0 (socket says 172.21.15.42:43548) 2024-05-28T14:05:38.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f649ea00640 1 -- 172.21.15.42:0/1466954710 learned_addr learned my addr 172.21.15.42:0/1466954710 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:38.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f64a7400640 1 --2- 172.21.15.42:0/1466954710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f648c004d50 0x7f64a814de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:38.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f64a7e00640 1 --2- 172.21.15.42:0/1466954710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64a81576d0 0x7f64a8157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:38.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f649ea00640 1 -- 172.21.15.42:0/1466954710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f648c004d50 msgr2=0x7f64a814de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:38.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f649ea00640 1 --2- 172.21.15.42:0/1466954710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f648c004d50 0x7f64a814de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:38.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f649ea00640 1 -- 172.21.15.42:0/1466954710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64a81576d0 msgr2=0x7f64a8157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:38.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f649ea00640 1 --2- 172.21.15.42:0/1466954710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64a81576d0 0x7f64a8157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:38.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f649ea00640 1 -- 172.21.15.42:0/1466954710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a81f0e70 con 0x7f64a8159730 2024-05-28T14:05:38.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f649ea00640 1 --2- 172.21.15.42:0/1466954710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64a8159730 0x7f64a81f0930 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6490011d80 tx=0x7f649000e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:38.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f649fe00640 1 -- 172.21.15.42:0/1466954710 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649001f070 con 0x7f64a8159730 2024-05-28T14:05:38.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f649fe00640 1 -- 172.21.15.42:0/1466954710 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f649000be10 con 0x7f64a8159730 2024-05-28T14:05:38.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f649fe00640 1 -- 172.21.15.42:0/1466954710 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649000f800 con 0x7f64a8159730 2024-05-28T14:05:38.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f64ad200640 1 -- 172.21.15.42:0/1466954710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a814e7e0 con 0x7f64a8159730 2024-05-28T14:05:38.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f64ad200640 1 -- 172.21.15.42:0/1466954710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a8159f60 con 0x7f64a8159730 2024-05-28T14:05:38.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.438+0000 7f64ad200640 1 -- 172.21.15.42:0/1466954710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a80691a0 con 0x7f64a8159730 2024-05-28T14:05:38.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.442+0000 7f649fe00640 1 -- 172.21.15.42:0/1466954710 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 89058+0+0 (secure 0 0 0) 0x7f6490022070 con 0x7f64a8159730 2024-05-28T14:05:38.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.442+0000 7f649fe00640 1 --2- 172.21.15.42:0/1466954710 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6484068f50 0x7f648406b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:38.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.442+0000 7f649fe00640 1 -- 172.21.15.42:0/1466954710 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f6490087ca0 con 0x7f64a8159730 2024-05-28T14:05:38.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.442+0000 7f64a7400640 1 --2- 172.21.15.42:0/1466954710 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6484068f50 0x7f648406b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:38.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.442+0000 7f64a7400640 1 --2- 172.21.15.42:0/1466954710 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6484068f50 0x7f648406b410 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f64a80b4820 tx=0x7f6498008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:38.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.446+0000 7f649fe00640 1 -- 172.21.15.42:0/1466954710 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f64a81f0e70 con 0x7f64a8159730 2024-05-28T14:05:38.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.670+0000 7f64ad200640 1 -- 172.21.15.42:0/1466954710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f64a8067d00 con 0x7f64a8159730 2024-05-28T14:05:38.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.670+0000 7f649fe00640 1 -- 172.21.15.42:0/1466954710 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2377 (secure 0 0 0) 0x7f6490059400 con 0x7f64a8159730 2024-05-28T14:05:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:05:38.658 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:05:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:05:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:05:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:05:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:05:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:05:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:05:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:05:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:05:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:05:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:05:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:05:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:05:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:05:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:05:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 459, 2024-05-28T14:05:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:05:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:05:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:05:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:05:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:05:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 61, 2024-05-28T14:05:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:05:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:05:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:05:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:05:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:05:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:05:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:05:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:05:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:05:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:05:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:05:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:05:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4957, 2024-05-28T14:05:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 20646613215, 2024-05-28T14:05:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 41715261440, 2024-05-28T14:05:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 731378851840, 2024-05-28T14:05:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:05:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 111104016, 2024-05-28T14:05:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 113918156, 2024-05-28T14:05:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 27, 2024-05-28T14:05:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 34 2024-05-28T14:05:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:05:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 37, 2024-05-28T14:05:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4, 2024-05-28T14:05:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:05:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:05:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:05:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:05:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 4, 2024-05-28T14:05:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:05:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:05:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:05:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 6420 2024-05-28T14:05:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 12, 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:05:28.979186+0000", 2024-05-28T14:05:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:05:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:05:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: "58c696cb-4bfb-45e4-a3e5-43a164bc676d": { 2024-05-28T14:05:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 22.0 seconds before exiting (25s)\n [============================] ", 2024-05-28T14:05:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:05:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:05:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: "f63639d3-a8b8-449e-aca8-3322f44f1d24": { 2024-05-28T14:05:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 22.0 seconds before exiting (35s)\n [============================] ", 2024-05-28T14:05:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:05:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:05:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:38.664 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:05:38.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.674+0000 7f64ad200640 1 -- 172.21.15.42:0/1466954710 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6484068f50 msgr2=0x7f648406b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:38.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.674+0000 7f64ad200640 1 --2- 172.21.15.42:0/1466954710 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6484068f50 0x7f648406b410 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f64a80b4820 tx=0x7f6498008040 comp rx=0 tx=0).stop 2024-05-28T14:05:38.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.674+0000 7f64ad200640 1 -- 172.21.15.42:0/1466954710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64a8159730 msgr2=0x7f64a81f0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:38.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.674+0000 7f64ad200640 1 --2- 172.21.15.42:0/1466954710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64a8159730 0x7f64a81f0930 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6490011d80 tx=0x7f649000e730 comp rx=0 tx=0).stop 2024-05-28T14:05:38.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.674+0000 7f64ad200640 1 -- 172.21.15.42:0/1466954710 shutdown_connections 2024-05-28T14:05:38.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.674+0000 7f64ad200640 1 --2- 172.21.15.42:0/1466954710 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6484068f50 0x7f648406b410 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:38.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.674+0000 7f64ad200640 1 --2- 172.21.15.42:0/1466954710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64a81576d0 0x7f64a8157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:38.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.674+0000 7f64ad200640 1 --2- 172.21.15.42:0/1466954710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64a8159730 0x7f64a81f0930 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:38.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.674+0000 7f64ad200640 1 --2- 172.21.15.42:0/1466954710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f648c004d50 0x7f64a814de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:38.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.674+0000 7f64ad200640 1 -- 172.21.15.42:0/1466954710 >> 172.21.15.42:0/1466954710 conn(0x7f64a8073280 msgr2=0x7f64a806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:38.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.674+0000 7f64ad200640 1 -- 172.21.15.42:0/1466954710 shutdown_connections 2024-05-28T14:05:38.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:38.674+0000 7f64ad200640 1 -- 172.21.15.42:0/1466954710 wait complete. 2024-05-28T14:05:43.680 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:05:43.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f88afa00640 1 Processor -- start 2024-05-28T14:05:43.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f88afa00640 1 -- start start 2024-05-28T14:05:43.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f88afa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f88a8158f30 0x7f88a8072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:43.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f88afa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88a80b4f00 0x7f88a815ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:43.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f88afa00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88a815c3f0 con 0x7f88a815bf50 2024-05-28T14:05:43.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f88afa00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f88a815c9b0 con 0x7f88a8158f30 2024-05-28T14:05:43.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f88afa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f88a815c6d0 con 0x7f88a80b4f00 2024-05-28T14:05:43.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f88ad000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f88a8158f30 0x7f88a8072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:43.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f88ad000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f88a8158f30 0x7f88a8072610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43564/0 (socket says 172.21.15.42:43564) 2024-05-28T14:05:43.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f88ad000640 1 -- 172.21.15.42:0/1693103425 learned_addr learned my addr 172.21.15.42:0/1693103425 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:43.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f88ad000640 1 -- 172.21.15.42:0/1693103425 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88a80b4f00 msgr2=0x7f88a815ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:43.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f889fe00640 1 --2- 172.21.15.42:0/1693103425 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88a80b4f00 0x7f88a815ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:43.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f88ad000640 1 --2- 172.21.15.42:0/1693103425 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88a80b4f00 0x7f88a815ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:43.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f88ad000640 1 -- 172.21.15.42:0/1693103425 >> v1:172.21.15.63:6789/0 conn(0x7f88a815bf50 legacy=0x7f88a806ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:43.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f88ad000640 1 -- 172.21.15.42:0/1693103425 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a8071e10 con 0x7f88a8158f30 2024-05-28T14:05:43.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.846+0000 7f889fe00640 1 --2- 172.21.15.42:0/1693103425 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88a80b4f00 0x7f88a815ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:05:43.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.850+0000 7f88ad000640 1 --2- 172.21.15.42:0/1693103425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88a8158f30 0x7f88a8072610 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8898009870 tx=0x7f889801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa7700762bca2d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:43.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.850+0000 7f88a7e00640 1 -- 172.21.15.42:0/1693103425 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88980b0070 con 0x7f88a8158f30 2024-05-28T14:05:43.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.850+0000 7f88a7e00640 1 -- 172.21.15.42:0/1693103425 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88980ab600 con 0x7f88a8158f30 2024-05-28T14:05:43.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.850+0000 7f88a7e00640 1 -- 172.21.15.42:0/1693103425 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88980ab9d0 con 0x7f88a8158f30 2024-05-28T14:05:43.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.850+0000 7f88afa00640 1 -- 172.21.15.42:0/1693103425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88a8158f30 msgr2=0x7f88a8072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:43.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.850+0000 7f88afa00640 1 --2- 172.21.15.42:0/1693103425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88a8158f30 0x7f88a8072610 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8898009870 tx=0x7f889801c460 comp rx=0 tx=0).stop 2024-05-28T14:05:43.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.850+0000 7f88afa00640 1 -- 172.21.15.42:0/1693103425 shutdown_connections 2024-05-28T14:05:43.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.850+0000 7f88afa00640 1 --2- 172.21.15.42:0/1693103425 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88a80b4f00 0x7f88a815ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:43.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.850+0000 7f88afa00640 1 --2- 172.21.15.42:0/1693103425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88a8158f30 0x7f88a8072610 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:43.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.850+0000 7f88afa00640 1 -- 172.21.15.42:0/1693103425 >> 172.21.15.42:0/1693103425 conn(0x7f88a806bff0 msgr2=0x7f88a80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:43.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.850+0000 7f88afa00640 1 -- 172.21.15.42:0/1693103425 shutdown_connections 2024-05-28T14:05:43.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.850+0000 7f88afa00640 1 -- 172.21.15.42:0/1693103425 wait complete. 2024-05-28T14:05:43.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88afa00640 1 Processor -- start 2024-05-28T14:05:43.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88afa00640 1 -- start start 2024-05-28T14:05:43.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88afa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a80b4f00 0x7f88a814a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:43.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88afa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88a8158f30 0x7f88a815cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:43.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88afa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88a815bf50 0x7f88a8164540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:43.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88afa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f88a806f520 con 0x7f88a815bf50 2024-05-28T14:05:43.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88afa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88a8072d60 con 0x7f88a80b4f00 2024-05-28T14:05:43.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88afa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f88a806f6a0 con 0x7f88a8158f30 2024-05-28T14:05:43.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88ada00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88a815bf50 0x7f88a8164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:43.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f889fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88a8158f30 0x7f88a815cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:43.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f889fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88a8158f30 0x7f88a815cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:53284/0 (socket says 172.21.15.42:53284) 2024-05-28T14:05:43.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f889fe00640 1 -- 172.21.15.42:0/908401362 learned_addr learned my addr 172.21.15.42:0/908401362 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:43.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88ad000640 1 --2- 172.21.15.42:0/908401362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a80b4f00 0x7f88a814a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:43.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88ada00640 1 -- 172.21.15.42:0/908401362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88a8158f30 msgr2=0x7f88a815cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:43.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88ada00640 1 --2- 172.21.15.42:0/908401362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88a8158f30 0x7f88a815cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:43.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88ada00640 1 -- 172.21.15.42:0/908401362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a80b4f00 msgr2=0x7f88a814a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:43.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88ada00640 1 --2- 172.21.15.42:0/908401362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a80b4f00 0x7f88a814a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:43.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88ada00640 1 -- 172.21.15.42:0/908401362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a815d5f0 con 0x7f88a815bf50 2024-05-28T14:05:43.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88ada00640 1 --2- 172.21.15.42:0/908401362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88a815bf50 0x7f88a8164540 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f88a000be00 tx=0x7f88a000bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:43.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88a5600640 1 -- 172.21.15.42:0/908401362 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0003980 con 0x7f88a815bf50 2024-05-28T14:05:43.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88a5600640 1 -- 172.21.15.42:0/908401362 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88a0003b20 con 0x7f88a815bf50 2024-05-28T14:05:43.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88a5600640 1 -- 172.21.15.42:0/908401362 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0004020 con 0x7f88a815bf50 2024-05-28T14:05:43.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.854+0000 7f88afa00640 1 -- 172.21.15.42:0/908401362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a815d8d0 con 0x7f88a815bf50 2024-05-28T14:05:43.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.858+0000 7f88afa00640 1 -- 172.21.15.42:0/908401362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a815de90 con 0x7f88a815bf50 2024-05-28T14:05:43.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.858+0000 7f88afa00640 1 -- 172.21.15.42:0/908401362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a8068a90 con 0x7f88a815bf50 2024-05-28T14:05:43.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.858+0000 7f88a5600640 1 -- 172.21.15.42:0/908401362 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 89058+0+0 (secure 0 0 0) 0x7f88a0006d10 con 0x7f88a815bf50 2024-05-28T14:05:43.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.858+0000 7f88a5600640 1 --2- 172.21.15.42:0/908401362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8884068f50 0x7f888406b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:43.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.858+0000 7f88a5600640 1 -- 172.21.15.42:0/908401362 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f88a0085380 con 0x7f88a815bf50 2024-05-28T14:05:43.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.858+0000 7f88ad000640 1 --2- 172.21.15.42:0/908401362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8884068f50 0x7f888406b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:43.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.858+0000 7f88ad000640 1 --2- 172.21.15.42:0/908401362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8884068f50 0x7f888406b410 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f88980ad000 tx=0x7f88980a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:43.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:43.862+0000 7f88a5600640 1 -- 172.21.15.42:0/908401362 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f88a0056a50 con 0x7f88a815bf50 2024-05-28T14:05:44.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:44.086+0000 7f88afa00640 1 -- 172.21.15.42:0/908401362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f88a80b53e0 con 0x7f88a815bf50 2024-05-28T14:05:44.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:44.090+0000 7f88a5600640 1 -- 172.21.15.42:0/908401362 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2333 (secure 0 0 0) 0x7f88a005ad50 con 0x7f88a815bf50 2024-05-28T14:05:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:05:44.075 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:05:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:05:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:05:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:05:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:05:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:05:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:05:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:05:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:05:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:05:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:05:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:05:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:05:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:05:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:05:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 465, 2024-05-28T14:05:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:05:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:05:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:05:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:05:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:05:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 61, 2024-05-28T14:05:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:05:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:05:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4957, 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 20646623192, 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 41715277824, 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 731378835456, 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 4862, 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 1 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 37, 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4, 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:05:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 4, 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 6420 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:05:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:05:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:05:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:05:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 12, 2024-05-28T14:05:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:05:28.979186+0000", 2024-05-28T14:05:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:05:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:05:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "58c696cb-4bfb-45e4-a3e5-43a164bc676d": { 2024-05-28T14:05:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 16.0 seconds before exiting (31s)\n [============================] ", 2024-05-28T14:05:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:05:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:05:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:44.082 INFO:teuthology.orchestra.run.smithi042.stdout: "f63639d3-a8b8-449e-aca8-3322f44f1d24": { 2024-05-28T14:05:44.082 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 16.0 seconds before exiting (41s)\n [============================] ", 2024-05-28T14:05:44.082 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:05:44.082 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:05:44.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:44.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:44.082 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:05:44.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:44.090+0000 7f88afa00640 1 -- 172.21.15.42:0/908401362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8884068f50 msgr2=0x7f888406b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:44.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:44.090+0000 7f88afa00640 1 --2- 172.21.15.42:0/908401362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8884068f50 0x7f888406b410 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f88980ad000 tx=0x7f88980a6000 comp rx=0 tx=0).stop 2024-05-28T14:05:44.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:44.090+0000 7f88afa00640 1 -- 172.21.15.42:0/908401362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88a815bf50 msgr2=0x7f88a8164540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:44.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:44.090+0000 7f88afa00640 1 --2- 172.21.15.42:0/908401362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88a815bf50 0x7f88a8164540 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f88a000be00 tx=0x7f88a000bef0 comp rx=0 tx=0).stop 2024-05-28T14:05:44.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:44.090+0000 7f88afa00640 1 -- 172.21.15.42:0/908401362 shutdown_connections 2024-05-28T14:05:44.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:44.090+0000 7f88afa00640 1 --2- 172.21.15.42:0/908401362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8884068f50 0x7f888406b410 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:44.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:44.090+0000 7f88afa00640 1 --2- 172.21.15.42:0/908401362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88a815bf50 0x7f88a8164540 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:44.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:44.090+0000 7f88afa00640 1 --2- 172.21.15.42:0/908401362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88a8158f30 0x7f88a815cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:44.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:44.090+0000 7f88afa00640 1 --2- 172.21.15.42:0/908401362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a80b4f00 0x7f88a814a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:44.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:44.090+0000 7f88afa00640 1 -- 172.21.15.42:0/908401362 >> 172.21.15.42:0/908401362 conn(0x7f88a806bff0 msgr2=0x7f88a80ae040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:44.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:44.090+0000 7f88afa00640 1 -- 172.21.15.42:0/908401362 shutdown_connections 2024-05-28T14:05:44.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:44.090+0000 7f88afa00640 1 -- 172.21.15.42:0/908401362 wait complete. 2024-05-28T14:05:48.686 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:05:48.690 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:05:49.098 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:05:49.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.262+0000 7f6fe0c00640 1 Processor -- start 2024-05-28T14:05:49.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.262+0000 7f6fe0c00640 1 -- start start 2024-05-28T14:05:49.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.262+0000 7f6fe0c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6fdc158f30 0x7f6fdc072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:49.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.262+0000 7f6fe0c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6fdc06ee10 0x7f6fdc06f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:49.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.262+0000 7f6fe0c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fdc15c120 con 0x7f6fdc0b4f00 2024-05-28T14:05:49.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.262+0000 7f6fe0c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6fdc06f7a0 con 0x7f6fdc158f30 2024-05-28T14:05:49.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.262+0000 7f6fe0c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6fdc15c9b0 con 0x7f6fdc06ee10 2024-05-28T14:05:49.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.266+0000 7f6fd9e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6fdc158f30 0x7f6fdc072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:49.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.266+0000 7f6fd9e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6fdc158f30 0x7f6fdc072610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34912/0 (socket says 172.21.15.42:34912) 2024-05-28T14:05:49.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.266+0000 7f6fd9e00640 1 -- 172.21.15.42:0/519261740 learned_addr learned my addr 172.21.15.42:0/519261740 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:49.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.266+0000 7f6fda800640 1 --2- 172.21.15.42:0/519261740 >> v2:172.21.15.63:3300/0 conn(0x7f6fdc06ee10 0x7f6fdc06f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:49.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.266+0000 7f6fd9e00640 1 -- 172.21.15.42:0/519261740 >> v2:172.21.15.63:3300/0 conn(0x7f6fdc06ee10 msgr2=0x7f6fdc06f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:49.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.266+0000 7f6fd9e00640 1 --2- 172.21.15.42:0/519261740 >> v2:172.21.15.63:3300/0 conn(0x7f6fdc06ee10 0x7f6fdc06f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:49.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.266+0000 7f6fd9e00640 1 -- 172.21.15.42:0/519261740 >> v1:172.21.15.63:6789/0 conn(0x7f6fdc0b4f00 legacy=0x7f6fdc15ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:49.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.266+0000 7f6fd9e00640 1 -- 172.21.15.42:0/519261740 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc15c3f0 con 0x7f6fdc158f30 2024-05-28T14:05:49.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.266+0000 7f6fd9e00640 1 --2- 172.21.15.42:0/519261740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fdc158f30 0x7f6fdc072610 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc073d40 tx=0x7f6fcc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9679b0dc44b05b22 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:49.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.266+0000 7f6fd9400640 1 -- 172.21.15.42:0/519261740 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc0ab660 con 0x7f6fdc158f30 2024-05-28T14:05:49.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.266+0000 7f6fd9400640 1 -- 172.21.15.42:0/519261740 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fcc0b0070 con 0x7f6fdc158f30 2024-05-28T14:05:49.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.266+0000 7f6fd9400640 1 -- 172.21.15.42:0/519261740 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc0aba20 con 0x7f6fdc158f30 2024-05-28T14:05:49.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 -- 172.21.15.42:0/519261740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fdc158f30 msgr2=0x7f6fdc072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:49.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 --2- 172.21.15.42:0/519261740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fdc158f30 0x7f6fdc072610 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc073d40 tx=0x7f6fcc01c460 comp rx=0 tx=0).stop 2024-05-28T14:05:49.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 -- 172.21.15.42:0/519261740 shutdown_connections 2024-05-28T14:05:49.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 --2- 172.21.15.42:0/519261740 >> v2:172.21.15.63:3300/0 conn(0x7f6fdc06ee10 0x7f6fdc06f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:49.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 --2- 172.21.15.42:0/519261740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fdc158f30 0x7f6fdc072610 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:49.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 -- 172.21.15.42:0/519261740 >> 172.21.15.42:0/519261740 conn(0x7f6fdc06bff0 msgr2=0x7f6fdc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:49.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 -- 172.21.15.42:0/519261740 shutdown_connections 2024-05-28T14:05:49.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 -- 172.21.15.42:0/519261740 wait complete. 2024-05-28T14:05:49.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 Processor -- start 2024-05-28T14:05:49.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 -- start start 2024-05-28T14:05:49.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fdc06ee10 0x7f6fdc154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:49.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fdc0b4f00 0x7f6fdc150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:49.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fdc158f30 0x7f6fdc153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:49.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc159490 con 0x7f6fdc06ee10 2024-05-28T14:05:49.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc159310 con 0x7f6fdc158f30 2024-05-28T14:05:49.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fe0c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6fdc072b50 con 0x7f6fdc0b4f00 2024-05-28T14:05:49.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fd9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fdc06ee10 0x7f6fdc154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:49.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fd9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fdc06ee10 0x7f6fdc154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34922/0 (socket says 172.21.15.42:34922) 2024-05-28T14:05:49.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fd9e00640 1 -- 172.21.15.42:0/1718417174 learned_addr learned my addr 172.21.15.42:0/1718417174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:49.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fda800640 1 --2- 172.21.15.42:0/1718417174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fdc158f30 0x7f6fdc153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:49.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fd1400640 1 --2- 172.21.15.42:0/1718417174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fdc0b4f00 0x7f6fdc150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:49.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fd9e00640 1 -- 172.21.15.42:0/1718417174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fdc0b4f00 msgr2=0x7f6fdc150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:49.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fd9e00640 1 --2- 172.21.15.42:0/1718417174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fdc0b4f00 0x7f6fdc150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:49.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fd9e00640 1 -- 172.21.15.42:0/1718417174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fdc158f30 msgr2=0x7f6fdc153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:49.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fd9e00640 1 --2- 172.21.15.42:0/1718417174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fdc158f30 0x7f6fdc153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:49.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.270+0000 7f6fd9e00640 1 -- 172.21.15.42:0/1718417174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc1514c0 con 0x7f6fdc06ee10 2024-05-28T14:05:49.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.274+0000 7f6fd9e00640 1 --2- 172.21.15.42:0/1718417174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fdc06ee10 0x7f6fdc154f40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0ad040 tx=0x7f6fcc01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:49.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.274+0000 7f6fd2a00640 1 -- 172.21.15.42:0/1718417174 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc0be5f0 con 0x7f6fdc06ee10 2024-05-28T14:05:49.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.274+0000 7f6fd2a00640 1 -- 172.21.15.42:0/1718417174 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fcc0be7b0 con 0x7f6fdc06ee10 2024-05-28T14:05:49.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.274+0000 7f6fd2a00640 1 -- 172.21.15.42:0/1718417174 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc0beba0 con 0x7f6fdc06ee10 2024-05-28T14:05:49.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.274+0000 7f6fe0c00640 1 -- 172.21.15.42:0/1718417174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc0ce0f0 con 0x7f6fdc06ee10 2024-05-28T14:05:49.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.274+0000 7f6fe0c00640 1 -- 172.21.15.42:0/1718417174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc0ce6b0 con 0x7f6fdc06ee10 2024-05-28T14:05:49.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.274+0000 7f6fe0c00640 1 -- 172.21.15.42:0/1718417174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc0ceaf0 con 0x7f6fdc06ee10 2024-05-28T14:05:49.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.274+0000 7f6fd2a00640 1 -- 172.21.15.42:0/1718417174 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 89058+0+0 (secure 0 0 0) 0x7f6fcc0b43f0 con 0x7f6fdc06ee10 2024-05-28T14:05:49.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.274+0000 7f6fd2a00640 1 --2- 172.21.15.42:0/1718417174 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6fb8068f50 0x7f6fb806b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:49.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.274+0000 7f6fd2a00640 1 -- 172.21.15.42:0/1718417174 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f6fcc120200 con 0x7f6fdc06ee10 2024-05-28T14:05:49.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.274+0000 7f6fd1400640 1 --2- 172.21.15.42:0/1718417174 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6fb8068f50 0x7f6fb806b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:49.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.278+0000 7f6fd1400640 1 --2- 172.21.15.42:0/1718417174 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6fb8068f50 0x7f6fb806b410 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc152330 tx=0x7f6fc4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.278+0000 7f6fd2a00640 1 -- 172.21.15.42:0/1718417174 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6fcc016110 con 0x7f6fdc06ee10 2024-05-28T14:05:49.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.502+0000 7f6fe0c00640 1 -- 172.21.15.42:0/1718417174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6fdc068a90 con 0x7f6fdc06ee10 2024-05-28T14:05:49.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.502+0000 7f6fd2a00640 1 -- 172.21.15.42:0/1718417174 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2333 (secure 0 0 0) 0x7f6fcc0f20d0 con 0x7f6fdc06ee10 2024-05-28T14:05:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:05:49.491 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:05:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:05:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:05:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:05:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:05:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:05:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:05:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:05:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:05:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:05:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:05:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:05:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:05:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:05:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:05:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 470, 2024-05-28T14:05:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 61, 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:05:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:05:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:05:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:05:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:05:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4957, 2024-05-28T14:05:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 20646628400, 2024-05-28T14:05:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 41715286016, 2024-05-28T14:05:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 731378827264, 2024-05-28T14:05:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:05:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 1535, 2024-05-28T14:05:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 37, 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4, 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 4, 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:05:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 6420 2024-05-28T14:05:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:05:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:05:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:05:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:05:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:05:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:05:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:05:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:05:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:05:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 12, 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:05:28.979186+0000", 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "58c696cb-4bfb-45e4-a3e5-43a164bc676d": { 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 10.0 seconds before exiting (37s)\n [============================] ", 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "f63639d3-a8b8-449e-aca8-3322f44f1d24": { 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 10.0 seconds before exiting (47s)\n [============================] ", 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:05:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:05:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:49.498 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:05:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.506+0000 7f6fe0c00640 1 -- 172.21.15.42:0/1718417174 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6fb8068f50 msgr2=0x7f6fb806b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.506+0000 7f6fe0c00640 1 --2- 172.21.15.42:0/1718417174 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6fb8068f50 0x7f6fb806b410 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc152330 tx=0x7f6fc4009040 comp rx=0 tx=0).stop 2024-05-28T14:05:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.506+0000 7f6fe0c00640 1 -- 172.21.15.42:0/1718417174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fdc06ee10 msgr2=0x7f6fdc154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.506+0000 7f6fe0c00640 1 --2- 172.21.15.42:0/1718417174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fdc06ee10 0x7f6fdc154f40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0ad040 tx=0x7f6fcc01aca0 comp rx=0 tx=0).stop 2024-05-28T14:05:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.506+0000 7f6fe0c00640 1 -- 172.21.15.42:0/1718417174 shutdown_connections 2024-05-28T14:05:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.506+0000 7f6fe0c00640 1 --2- 172.21.15.42:0/1718417174 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6fb8068f50 0x7f6fb806b410 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.506+0000 7f6fe0c00640 1 --2- 172.21.15.42:0/1718417174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fdc158f30 0x7f6fdc153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.506+0000 7f6fe0c00640 1 --2- 172.21.15.42:0/1718417174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fdc0b4f00 0x7f6fdc150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.506+0000 7f6fe0c00640 1 --2- 172.21.15.42:0/1718417174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fdc06ee10 0x7f6fdc154f40 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.506+0000 7f6fe0c00640 1 -- 172.21.15.42:0/1718417174 >> 172.21.15.42:0/1718417174 conn(0x7f6fdc06bff0 msgr2=0x7f6fdc0ae310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.506+0000 7f6fe0c00640 1 -- 172.21.15.42:0/1718417174 shutdown_connections 2024-05-28T14:05:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:49.506+0000 7f6fe0c00640 1 -- 172.21.15.42:0/1718417174 wait complete. 2024-05-28T14:05:54.514 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:05:54.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.678+0000 7fba65400640 1 Processor -- start 2024-05-28T14:05:54.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.682+0000 7fba65400640 1 -- start start 2024-05-28T14:05:54.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.682+0000 7fba65400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fba60156c00 0x7fba6006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:54.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.682+0000 7fba65400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6015d8d0 0x7fba600aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:54.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.682+0000 7fba65400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba6006f6a0 con 0x7fba600af500 2024-05-28T14:05:54.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.682+0000 7fba65400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fba6006e600 con 0x7fba60156c00 2024-05-28T14:05:54.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.682+0000 7fba65400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fba6006e8e0 con 0x7fba6015d8d0 2024-05-28T14:05:54.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.682+0000 7fba55e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6015d8d0 0x7fba600aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:54.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.682+0000 7fba55e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6015d8d0 0x7fba600aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:44942/0 (socket says 172.21.15.42:44942) 2024-05-28T14:05:54.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.682+0000 7fba55e00640 1 -- 172.21.15.42:0/3742888324 learned_addr learned my addr 172.21.15.42:0/3742888324 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:54.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.682+0000 7fba5e800640 1 --2- 172.21.15.42:0/3742888324 >> v2:172.21.15.63:3300/0 conn(0x7fba60156c00 0x7fba6006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:54.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.682+0000 7fba55e00640 1 -- 172.21.15.42:0/3742888324 >> v2:172.21.15.63:3300/0 conn(0x7fba60156c00 msgr2=0x7fba6006dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:05:54.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.682+0000 7fba55e00640 1 --2- 172.21.15.42:0/3742888324 >> v2:172.21.15.63:3300/0 conn(0x7fba60156c00 0x7fba6006dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:54.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.682+0000 7fba55e00640 1 -- 172.21.15.42:0/3742888324 >> v1:172.21.15.63:6789/0 conn(0x7fba600af500 legacy=0x7fba601601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:54.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.682+0000 7fba55e00640 1 -- 172.21.15.42:0/3742888324 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6006ebc0 con 0x7fba6015d8d0 2024-05-28T14:05:54.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.686+0000 7fba55e00640 1 --2- 172.21.15.42:0/3742888324 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6015d8d0 0x7fba600aefc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba48009870 tx=0x7fba4801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=998b13e08d62096f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:54.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.686+0000 7fba5de00640 1 -- 172.21.15.42:0/3742888324 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba480b0070 con 0x7fba6015d8d0 2024-05-28T14:05:54.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.686+0000 7fba5de00640 1 -- 172.21.15.42:0/3742888324 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba480ab600 con 0x7fba6015d8d0 2024-05-28T14:05:54.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.686+0000 7fba5de00640 1 -- 172.21.15.42:0/3742888324 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba480ab9d0 con 0x7fba6015d8d0 2024-05-28T14:05:54.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.686+0000 7fba65400640 1 -- 172.21.15.42:0/3742888324 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6015d8d0 msgr2=0x7fba600aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:54.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.686+0000 7fba65400640 1 --2- 172.21.15.42:0/3742888324 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6015d8d0 0x7fba600aefc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba48009870 tx=0x7fba4801c4d0 comp rx=0 tx=0).stop 2024-05-28T14:05:54.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.686+0000 7fba65400640 1 -- 172.21.15.42:0/3742888324 shutdown_connections 2024-05-28T14:05:54.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.686+0000 7fba65400640 1 --2- 172.21.15.42:0/3742888324 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6015d8d0 0x7fba600aefc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:54.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.686+0000 7fba65400640 1 --2- 172.21.15.42:0/3742888324 >> v2:172.21.15.63:3300/0 conn(0x7fba60156c00 0x7fba6006dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:54.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.686+0000 7fba65400640 1 -- 172.21.15.42:0/3742888324 >> 172.21.15.42:0/3742888324 conn(0x7fba6006bff0 msgr2=0x7fba600b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:54.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.686+0000 7fba65400640 1 -- 172.21.15.42:0/3742888324 shutdown_connections 2024-05-28T14:05:54.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.686+0000 7fba65400640 1 -- 172.21.15.42:0/3742888324 wait complete. 2024-05-28T14:05:54.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba65400640 1 Processor -- start 2024-05-28T14:05:54.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba65400640 1 -- start start 2024-05-28T14:05:54.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba65400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba600af500 0x7fba6013c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:54.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba65400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba60156c00 0x7fba600726f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:54.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba65400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6015d8d0 0x7fba6015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:54.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba65400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fba6006e240 con 0x7fba600af500 2024-05-28T14:05:54.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba65400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba600af910 con 0x7fba60156c00 2024-05-28T14:05:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba65400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fba6006e3c0 con 0x7fba6015d8d0 2024-05-28T14:05:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba5e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba600af500 0x7fba6013c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba5e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba600af500 0x7fba6013c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34930/0 (socket says 172.21.15.42:34930) 2024-05-28T14:05:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba5e800640 1 -- 172.21.15.42:0/3617198640 learned_addr learned my addr 172.21.15.42:0/3617198640 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:05:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba5f200640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6015d8d0 0x7fba6015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba55e00640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba60156c00 0x7fba600726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba5e800640 1 -- 172.21.15.42:0/3617198640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6015d8d0 msgr2=0x7fba6015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba5e800640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6015d8d0 0x7fba6015dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba5e800640 1 -- 172.21.15.42:0/3617198640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba60156c00 msgr2=0x7fba600726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba5e800640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba60156c00 0x7fba600726f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba5e800640 1 -- 172.21.15.42:0/3617198640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba60150e30 con 0x7fba600af500 2024-05-28T14:05:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba5f200640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6015d8d0 0x7fba6015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:05:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba55e00640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba60156c00 0x7fba600726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:05:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba5e800640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba600af500 0x7fba6013c6d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fba5000e870 tx=0x7fba5000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba57400640 1 -- 172.21.15.42:0/3617198640 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50017070 con 0x7fba600af500 2024-05-28T14:05:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba57400640 1 -- 172.21.15.42:0/3617198640 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba5000c940 con 0x7fba600af500 2024-05-28T14:05:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba65400640 1 -- 172.21.15.42:0/3617198640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba60151110 con 0x7fba600af500 2024-05-28T14:05:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba57400640 1 -- 172.21.15.42:0/3617198640 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5000ccf0 con 0x7fba600af500 2024-05-28T14:05:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba65400640 1 -- 172.21.15.42:0/3617198640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba60149ac0 con 0x7fba600af500 2024-05-28T14:05:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.690+0000 7fba65400640 1 -- 172.21.15.42:0/3617198640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba60068a90 con 0x7fba600af500 2024-05-28T14:05:54.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.694+0000 7fba57400640 1 -- 172.21.15.42:0/3617198640 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 89058+0+0 (secure 0 0 0) 0x7fba5001a070 con 0x7fba600af500 2024-05-28T14:05:54.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.694+0000 7fba57400640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba3c068f50 0x7fba3c06b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:05:54.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.694+0000 7fba57400640 1 -- 172.21.15.42:0/3617198640 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fba50085880 con 0x7fba600af500 2024-05-28T14:05:54.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.694+0000 7fba55e00640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba3c068f50 0x7fba3c06b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:05:54.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.694+0000 7fba55e00640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba3c068f50 0x7fba3c06b410 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fba480ad000 tx=0x7fba480a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:05:54.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.698+0000 7fba57400640 1 -- 172.21.15.42:0/3617198640 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fba50057630 con 0x7fba600af500 2024-05-28T14:05:54.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.922+0000 7fba65400640 1 -- 172.21.15.42:0/3617198640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fba600b2a40 con 0x7fba600af500 2024-05-28T14:05:54.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.922+0000 7fba57400640 1 -- 172.21.15.42:0/3617198640 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2330 (secure 0 0 0) 0x7fba50056f30 con 0x7fba600af500 2024-05-28T14:05:54.908 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:05:54.909 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:05:54.909 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:05:54.909 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:05:54.909 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:05:54.909 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:05:54.909 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:05:54.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:54.909 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:05:54.909 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:05:54.909 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:05:54.909 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:05:54.909 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:05:54.910 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:54.910 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:05:54.910 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:05:54.910 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:05:54.910 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:05:54.910 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:54.910 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 476, 2024-05-28T14:05:54.910 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:05:54.910 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:05:54.910 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:05:54.910 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:05:54.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:54.910 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:05:54.911 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 61, 2024-05-28T14:05:54.911 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:05:54.911 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:05:54.911 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:05:54.911 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:05:54.911 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:05:54.911 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:05:54.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:54.911 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:05:54.911 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:05:54.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:54.911 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:05:54.911 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:05:54.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:54.912 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:54.912 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:05:54.912 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:05:54.912 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4957, 2024-05-28T14:05:54.912 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 20646628400, 2024-05-28T14:05:54.912 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 41715286016, 2024-05-28T14:05:54.912 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 731378827264, 2024-05-28T14:05:54.912 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:05:54.912 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 597, 2024-05-28T14:05:54.912 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:05:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:05:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:05:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 37, 2024-05-28T14:05:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4, 2024-05-28T14:05:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:05:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:05:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:05:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:05:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:05:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 4, 2024-05-28T14:05:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:05:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:05:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:05:54.914 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 6420 2024-05-28T14:05:54.914 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:54.914 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:54.914 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:05:54.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:54.914 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:05:54.914 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:05:54.914 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:05:54.914 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:05:54.914 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:05:54.914 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:05:54.914 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:05:54.914 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 12, 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:05:28.979186+0000", 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: "58c696cb-4bfb-45e4-a3e5-43a164bc676d": { 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 6.0 seconds before exiting (41s)\n [============================] ", 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: "f63639d3-a8b8-449e-aca8-3322f44f1d24": { 2024-05-28T14:05:54.915 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 6.0 seconds before exiting (51s)\n [============================] ", 2024-05-28T14:05:54.916 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:05:54.916 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:05:54.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:54.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:05:54.916 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:05:54.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.922+0000 7fba65400640 1 -- 172.21.15.42:0/3617198640 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba3c068f50 msgr2=0x7fba3c06b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.922+0000 7fba65400640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba3c068f50 0x7fba3c06b410 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fba480ad000 tx=0x7fba480a6000 comp rx=0 tx=0).stop 2024-05-28T14:05:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.922+0000 7fba65400640 1 -- 172.21.15.42:0/3617198640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba600af500 msgr2=0x7fba6013c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:05:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.922+0000 7fba65400640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba600af500 0x7fba6013c6d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fba5000e870 tx=0x7fba5000ec70 comp rx=0 tx=0).stop 2024-05-28T14:05:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.922+0000 7fba65400640 1 -- 172.21.15.42:0/3617198640 shutdown_connections 2024-05-28T14:05:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.922+0000 7fba65400640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba3c068f50 0x7fba3c06b410 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.922+0000 7fba65400640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6015d8d0 0x7fba6015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.922+0000 7fba65400640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba60156c00 0x7fba600726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.922+0000 7fba65400640 1 --2- 172.21.15.42:0/3617198640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba600af500 0x7fba6013c6d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:05:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.922+0000 7fba65400640 1 -- 172.21.15.42:0/3617198640 >> 172.21.15.42:0/3617198640 conn(0x7fba6006bff0 msgr2=0x7fba600749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:05:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.922+0000 7fba65400640 1 -- 172.21.15.42:0/3617198640 shutdown_connections 2024-05-28T14:05:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:05:54.922+0000 7fba65400640 1 -- 172.21.15.42:0/3617198640 wait complete. 2024-05-28T14:05:59.931 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:06:00.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11c1c00640 1 Processor -- start 2024-05-28T14:06:00.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11c1c00640 1 -- start start 2024-05-28T14:06:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11c1c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f11bc156c00 0x7f11bc06dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11c1c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11bc06f6a0 con 0x7f11bc0af6d0 2024-05-28T14:06:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11c1c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11bc06e600 con 0x7f11bc15d8d0 2024-05-28T14:06:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11c1c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f11bc06e8e0 con 0x7f11bc156c00 2024-05-28T14:06:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11bb400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f11bc0af6d0 0x7f11bc15dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:45782/0 (socket says 172.21.15.42:45782) 2024-05-28T14:06:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11bb400640 1 -- 172.21.15.42:0/1079929106 learned_addr learned my addr 172.21.15.42:0/1079929106 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11baa00640 1 --2- 172.21.15.42:0/1079929106 >> v2:172.21.15.63:3300/0 conn(0x7f11bc156c00 0x7f11bc06dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11b9600640 1 -- 172.21.15.42:0/1079929106 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2067099365 0 0) 0x7f11bc06f6a0 con 0x7f11bc0af6d0 2024-05-28T14:06:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11b9600640 1 -- 172.21.15.42:0/1079929106 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11a0003530 con 0x7f11bc0af6d0 2024-05-28T14:06:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11b9600640 1 -- 172.21.15.42:0/1079929106 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4194615736 0 0) 0x7f11bc06e600 con 0x7f11bc15d8d0 2024-05-28T14:06:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11b9600640 1 -- 172.21.15.42:0/1079929106 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11bc06f6a0 con 0x7f11bc15d8d0 2024-05-28T14:06:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11baa00640 1 -- 172.21.15.42:0/1079929106 >> v1:172.21.15.63:6789/0 conn(0x7f11bc15d8d0 legacy=0x7f11bc0aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11baa00640 1 -- 172.21.15.42:0/1079929106 >> v1:172.21.15.42:6789/0 conn(0x7f11bc0af6d0 legacy=0x7f11bc15dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.098+0000 7f11baa00640 1 -- 172.21.15.42:0/1079929106 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc06ebc0 con 0x7f11bc156c00 2024-05-28T14:06:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.102+0000 7f11baa00640 1 --2- 172.21.15.42:0/1079929106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11bc156c00 0x7f11bc06dd00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0097c0 tx=0x7f11ac01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5be2acc98e27059 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:00.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.102+0000 7f11b9600640 1 -- 172.21.15.42:0/1079929106 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac0b0070 con 0x7f11bc156c00 2024-05-28T14:06:00.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.102+0000 7f11b9600640 1 -- 172.21.15.42:0/1079929106 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac0ab600 con 0x7f11bc156c00 2024-05-28T14:06:00.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.102+0000 7f11b9600640 1 -- 172.21.15.42:0/1079929106 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac0ab9d0 con 0x7f11bc156c00 2024-05-28T14:06:00.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.102+0000 7f11c1c00640 1 -- 172.21.15.42:0/1079929106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11bc156c00 msgr2=0x7f11bc06dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.102+0000 7f11c1c00640 1 --2- 172.21.15.42:0/1079929106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11bc156c00 0x7f11bc06dd00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0097c0 tx=0x7f11ac01c410 comp rx=0 tx=0).stop 2024-05-28T14:06:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.102+0000 7f11c1c00640 1 -- 172.21.15.42:0/1079929106 shutdown_connections 2024-05-28T14:06:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.102+0000 7f11c1c00640 1 --2- 172.21.15.42:0/1079929106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11bc156c00 0x7f11bc06dd00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.102+0000 7f11c1c00640 1 -- 172.21.15.42:0/1079929106 >> 172.21.15.42:0/1079929106 conn(0x7f11bc06bff0 msgr2=0x7f11bc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.102+0000 7f11c1c00640 1 -- 172.21.15.42:0/1079929106 shutdown_connections 2024-05-28T14:06:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.102+0000 7f11c1c00640 1 -- 172.21.15.42:0/1079929106 wait complete. 2024-05-28T14:06:00.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.102+0000 7f11c1c00640 1 Processor -- start 2024-05-28T14:06:00.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11c1c00640 1 -- start start 2024-05-28T14:06:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11c1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11bc0af6d0 0x7f11bc148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11c1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11bc156c00 0x7f11bc072210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11c1c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11bc15d8d0 0x7f11bc072750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11c1c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc16a6d0 con 0x7f11bc15d8d0 2024-05-28T14:06:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11c1c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc16a550 con 0x7f11bc156c00 2024-05-28T14:06:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11c1c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f11bc16a850 con 0x7f11bc0af6d0 2024-05-28T14:06:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11bb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11bc15d8d0 0x7f11bc072750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11baa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11bc0af6d0 0x7f11bc148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11bb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11bc15d8d0 0x7f11bc072750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54216/0 (socket says 172.21.15.42:54216) 2024-05-28T14:06:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11bb400640 1 -- 172.21.15.42:0/1236335432 learned_addr learned my addr 172.21.15.42:0/1236335432 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11ba000640 1 --2- 172.21.15.42:0/1236335432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11bc156c00 0x7f11bc072210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11bb400640 1 -- 172.21.15.42:0/1236335432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11bc0af6d0 msgr2=0x7f11bc148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11bb400640 1 --2- 172.21.15.42:0/1236335432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11bc0af6d0 0x7f11bc148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11bb400640 1 -- 172.21.15.42:0/1236335432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11bc156c00 msgr2=0x7f11bc072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11bb400640 1 --2- 172.21.15.42:0/1236335432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11bc156c00 0x7f11bc072210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11bb400640 1 -- 172.21.15.42:0/1236335432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc154070 con 0x7f11bc15d8d0 2024-05-28T14:06:00.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11baa00640 1 --2- 172.21.15.42:0/1236335432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11bc0af6d0 0x7f11bc148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:06:00.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11bb400640 1 --2- 172.21.15.42:0/1236335432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11bc15d8d0 0x7f11bc072750 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f11b400caa0 tx=0x7f11b400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:00.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11b2a00640 1 -- 172.21.15.42:0/1236335432 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4017070 con 0x7f11bc15d8d0 2024-05-28T14:06:00.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11b2a00640 1 -- 172.21.15.42:0/1236335432 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b400ac20 con 0x7f11bc15d8d0 2024-05-28T14:06:00.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11b2a00640 1 -- 172.21.15.42:0/1236335432 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4012640 con 0x7f11bc15d8d0 2024-05-28T14:06:00.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11c1c00640 1 -- 172.21.15.42:0/1236335432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc154350 con 0x7f11bc15d8d0 2024-05-28T14:06:00.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11c1c00640 1 -- 172.21.15.42:0/1236335432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11bc154910 con 0x7f11bc15d8d0 2024-05-28T14:06:00.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.106+0000 7f11c1c00640 1 -- 172.21.15.42:0/1236335432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11bc068a90 con 0x7f11bc15d8d0 2024-05-28T14:06:00.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.110+0000 7f11b2a00640 1 -- 172.21.15.42:0/1236335432 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 89058+0+0 (secure 0 0 0) 0x7f11b401a070 con 0x7f11bc15d8d0 2024-05-28T14:06:00.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.110+0000 7f11b2a00640 1 --2- 172.21.15.42:0/1236335432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f119c068f50 0x7f119c06b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.110+0000 7f11b2a00640 1 -- 172.21.15.42:0/1236335432 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f11b4085340 con 0x7f11bc15d8d0 2024-05-28T14:06:00.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.110+0000 7f11baa00640 1 --2- 172.21.15.42:0/1236335432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f119c068f50 0x7f119c06b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.110+0000 7f11baa00640 1 --2- 172.21.15.42:0/1236335432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f119c068f50 0x7f119c06b410 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0ad000 tx=0x7f11ac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:00.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.114+0000 7f11b2a00640 1 -- 172.21.15.42:0/1236335432 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f11b4056aa0 con 0x7f11bc15d8d0 2024-05-28T14:06:00.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.354+0000 7f11c1c00640 1 -- 172.21.15.42:0/1236335432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f11bc0705f0 con 0x7f11bc15d8d0 2024-05-28T14:06:00.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.354+0000 7f11b2a00640 1 -- 172.21.15.42:0/1236335432 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1731 (secure 0 0 0) 0x7f11b405ada0 con 0x7f11bc15d8d0 2024-05-28T14:06:00.340 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:00.340 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:06:00.340 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:06:00.340 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:06:00.340 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:06:00.340 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:06:00.340 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:06:00.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:00.341 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:06:00.341 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:06:00.341 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:06:00.341 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:06:00.341 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:06:00.341 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:06:00.341 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:06:00.341 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:06:00.341 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:06:00.341 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:06:00.341 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:06:00.341 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 481, 2024-05-28T14:06:00.342 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:06:00.342 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:06:00.342 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:06:00.342 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:06:00.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:00.342 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:06:00.342 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 61, 2024-05-28T14:06:00.342 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:06:00.342 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:06:00.342 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:06:00.342 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:06:00.342 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:06:00.342 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:06:00.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:00.343 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:06:00.343 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:06:00.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:00.343 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:06:00.343 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:06:00.343 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:00.343 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:06:00.343 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:06:00.343 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:06:00.343 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4957, 2024-05-28T14:06:00.343 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 20646628400, 2024-05-28T14:06:00.343 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 41715286016, 2024-05-28T14:06:00.343 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 731378827264, 2024-05-28T14:06:00.344 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280 2024-05-28T14:06:00.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:00.344 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:06:00.344 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 37, 2024-05-28T14:06:00.344 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4, 2024-05-28T14:06:00.344 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:06:00.344 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:06:00.344 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:06:00.344 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:06:00.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:00.344 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 4, 2024-05-28T14:06:00.344 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:06:00.344 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:06:00.345 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:06:00.345 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 6420 2024-05-28T14:06:00.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:00.345 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:06:00.345 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:06:00.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:00.345 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:06:00.345 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:06:00.345 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:06:00.345 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:06:00.345 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:06:00.345 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:06:00.346 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:06:00.346 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:06:00.346 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:06:00.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:00.346 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:06:00.346 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 12, 2024-05-28T14:06:00.346 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:05:28.979186+0000", 2024-05-28T14:06:00.346 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:06:00.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:00.346 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": {} 2024-05-28T14:06:00.346 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:06:00.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.354+0000 7f11c1c00640 1 -- 172.21.15.42:0/1236335432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f119c068f50 msgr2=0x7f119c06b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.354+0000 7f11c1c00640 1 --2- 172.21.15.42:0/1236335432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f119c068f50 0x7f119c06b410 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0ad000 tx=0x7f11ac008040 comp rx=0 tx=0).stop 2024-05-28T14:06:00.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.354+0000 7f11c1c00640 1 -- 172.21.15.42:0/1236335432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11bc15d8d0 msgr2=0x7f11bc072750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.354+0000 7f11c1c00640 1 --2- 172.21.15.42:0/1236335432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11bc15d8d0 0x7f11bc072750 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f11b400caa0 tx=0x7f11b400cf60 comp rx=0 tx=0).stop 2024-05-28T14:06:00.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.354+0000 7f11c1c00640 1 -- 172.21.15.42:0/1236335432 shutdown_connections 2024-05-28T14:06:00.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.354+0000 7f11c1c00640 1 --2- 172.21.15.42:0/1236335432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f119c068f50 0x7f119c06b410 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.354+0000 7f11c1c00640 1 --2- 172.21.15.42:0/1236335432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11bc15d8d0 0x7f11bc072750 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.354+0000 7f11c1c00640 1 --2- 172.21.15.42:0/1236335432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11bc156c00 0x7f11bc072210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.354+0000 7f11c1c00640 1 --2- 172.21.15.42:0/1236335432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11bc0af6d0 0x7f11bc148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.354+0000 7f11c1c00640 1 -- 172.21.15.42:0/1236335432 >> 172.21.15.42:0/1236335432 conn(0x7f11bc06bff0 msgr2=0x7f11bc15cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:00.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.354+0000 7f11c1c00640 1 -- 172.21.15.42:0/1236335432 shutdown_connections 2024-05-28T14:06:00.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.354+0000 7f11c1c00640 1 -- 172.21.15.42:0/1236335432 wait complete. 2024-05-28T14:06:00.362 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-28T14:06:00.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.518+0000 7f4e09800640 1 Processor -- start 2024-05-28T14:06:00.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.518+0000 7f4e09800640 1 -- start start 2024-05-28T14:06:00.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.518+0000 7f4e09800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4e04145d50 0x7f4e040cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.518+0000 7f4e09800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e040cd050 0x7f4e040d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.518+0000 7f4e09800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e04160170 con 0x7f4e040cf540 2024-05-28T14:06:00.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.518+0000 7f4e09800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f4e04160450 con 0x7f4e04145d50 2024-05-28T14:06:00.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.518+0000 7f4e09800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4e04160730 con 0x7f4e040cd050 2024-05-28T14:06:00.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.518+0000 7f4dfb400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4e04145d50 0x7f4e040cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.518+0000 7f4e08e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e040cd050 0x7f4e040d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.518+0000 7f4dfb400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4e04145d50 0x7f4e040cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54220/0 (socket says 172.21.15.42:54220) 2024-05-28T14:06:00.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.518+0000 7f4dfb400640 1 -- 172.21.15.42:0/2605390416 learned_addr learned my addr 172.21.15.42:0/2605390416 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:00.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.518+0000 7f4e03e00640 1 --1- 172.21.15.42:0/2605390416 >> v1:172.21.15.63:6789/0 conn(0x7f4e040cf540 0x7f4e040cc940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:43366/0 (socket says 172.21.15.42:43366) 2024-05-28T14:06:00.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.522+0000 7f4e08e00640 1 -- 172.21.15.42:0/2605390416 >> v2:172.21.15.42:3300/0 conn(0x7f4e04145d50 msgr2=0x7f4e040cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:00.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.522+0000 7f4e08e00640 1 --2- 172.21.15.42:0/2605390416 >> v2:172.21.15.42:3300/0 conn(0x7f4e04145d50 0x7f4e040cfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.522+0000 7f4e08e00640 1 -- 172.21.15.42:0/2605390416 >> v1:172.21.15.63:6789/0 conn(0x7f4e040cf540 legacy=0x7f4e040cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.522+0000 7f4e08e00640 1 -- 172.21.15.42:0/2605390416 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e040707e0 con 0x7f4e040cd050 2024-05-28T14:06:00.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.522+0000 7f4dfb400640 1 --2- 172.21.15.42:0/2605390416 >> v2:172.21.15.42:3300/0 conn(0x7f4e04145d50 0x7f4e040cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:06:00.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.522+0000 7f4e03400640 1 -- 172.21.15.42:0/2605390416 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3794696762 0 0) 0x7f4e04160170 con 0x7f4e040cf540 2024-05-28T14:06:00.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.522+0000 7f4e08e00640 1 --2- 172.21.15.42:0/2605390416 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e040cd050 0x7f4e040d0400 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0097c0 tx=0x7f4dfc01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4579118b0932687 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:00.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.522+0000 7f4e03400640 1 -- 172.21.15.42:0/2605390416 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc0b0070 con 0x7f4e040cd050 2024-05-28T14:06:00.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.522+0000 7f4e03400640 1 -- 172.21.15.42:0/2605390416 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dfc0ab600 con 0x7f4e040cd050 2024-05-28T14:06:00.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.522+0000 7f4e03400640 1 -- 172.21.15.42:0/2605390416 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc0ab9d0 con 0x7f4e040cd050 2024-05-28T14:06:00.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.522+0000 7f4e09800640 1 -- 172.21.15.42:0/2605390416 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e040cd050 msgr2=0x7f4e040d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.522+0000 7f4e09800640 1 --2- 172.21.15.42:0/2605390416 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e040cd050 0x7f4e040d0400 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0097c0 tx=0x7f4dfc01c410 comp rx=0 tx=0).stop 2024-05-28T14:06:00.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e09800640 1 -- 172.21.15.42:0/2605390416 shutdown_connections 2024-05-28T14:06:00.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e09800640 1 --2- 172.21.15.42:0/2605390416 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e040cd050 0x7f4e040d0400 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e09800640 1 --2- 172.21.15.42:0/2605390416 >> v2:172.21.15.42:3300/0 conn(0x7f4e04145d50 0x7f4e040cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e09800640 1 -- 172.21.15.42:0/2605390416 >> 172.21.15.42:0/2605390416 conn(0x7f4e0406fcc0 msgr2=0x7f4e040700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:00.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e09800640 1 -- 172.21.15.42:0/2605390416 shutdown_connections 2024-05-28T14:06:00.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e09800640 1 -- 172.21.15.42:0/2605390416 wait complete. 2024-05-28T14:06:00.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e09800640 1 Processor -- start 2024-05-28T14:06:00.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e09800640 1 -- start start 2024-05-28T14:06:00.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e09800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e040cd050 0x7f4e04154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e09800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e040cf540 0x7f4e04157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e09800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e04145d50 0x7f4e04157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e09800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4e041694d0 con 0x7f4e040cd050 2024-05-28T14:06:00.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e09800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e04169350 con 0x7f4e040cf540 2024-05-28T14:06:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e09800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4e04169650 con 0x7f4e04145d50 2024-05-28T14:06:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e03e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e040cd050 0x7f4e04154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e03e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e040cd050 0x7f4e04154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54234/0 (socket says 172.21.15.42:54234) 2024-05-28T14:06:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e03e00640 1 -- 172.21.15.42:0/840636388 learned_addr learned my addr 172.21.15.42:0/840636388 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4dfb400640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e040cf540 0x7f4e04157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e08e00640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e04145d50 0x7f4e04157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e03e00640 1 -- 172.21.15.42:0/840636388 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e04145d50 msgr2=0x7f4e04157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e03e00640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e04145d50 0x7f4e04157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e03e00640 1 -- 172.21.15.42:0/840636388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e040cf540 msgr2=0x7f4e04157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e03e00640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e040cf540 0x7f4e04157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e03e00640 1 -- 172.21.15.42:0/840636388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e04166000 con 0x7f4e040cd050 2024-05-28T14:06:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e08e00640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e04145d50 0x7f4e04157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:00.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e03e00640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e040cd050 0x7f4e04154c20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4df4001570 tx=0x7f4df400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.526+0000 7f4e00c00640 1 -- 172.21.15.42:0/840636388 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4018070 con 0x7f4e040cd050 2024-05-28T14:06:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.530+0000 7f4e00c00640 1 -- 172.21.15.42:0/840636388 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df400a640 con 0x7f4e040cd050 2024-05-28T14:06:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.530+0000 7f4e00c00640 1 -- 172.21.15.42:0/840636388 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df400aa30 con 0x7f4e040cd050 2024-05-28T14:06:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.530+0000 7f4e09800640 1 -- 172.21.15.42:0/840636388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e041662a0 con 0x7f4e040cd050 2024-05-28T14:06:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.530+0000 7f4e09800640 1 -- 172.21.15.42:0/840636388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e04166860 con 0x7f4e040cd050 2024-05-28T14:06:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.530+0000 7f4e09800640 1 -- 172.21.15.42:0/840636388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e0404fcf0 con 0x7f4e040cd050 2024-05-28T14:06:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.530+0000 7f4e00c00640 1 -- 172.21.15.42:0/840636388 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 89058+0+0 (secure 0 0 0) 0x7f4df400abd0 con 0x7f4e040cd050 2024-05-28T14:06:00.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.530+0000 7f4e00c00640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4de0068f50 0x7f4de006b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.530+0000 7f4e00c00640 1 -- 172.21.15.42:0/840636388 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f4df4084c50 con 0x7f4e040cd050 2024-05-28T14:06:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.530+0000 7f4dfb400640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4de0068f50 0x7f4de006b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.534+0000 7f4dfb400640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4de0068f50 0x7f4de006b410 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4e0415f840 tx=0x7f4dec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:00.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.534+0000 7f4e00c00640 1 -- 172.21.15.42:0/840636388 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4df4056a00 con 0x7f4e040cd050 2024-05-28T14:06:00.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.726+0000 7f4e09800640 1 -- 172.21.15.42:0/840636388 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e040cccc0 con 0x7f4de0068f50 2024-05-28T14:06:00.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.734+0000 7f4e00c00640 1 -- 172.21.15.42:0/840636388 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+381 (secure 0 0 0) 0x7f4e040cccc0 con 0x7f4de0068f50 2024-05-28T14:06:00.721 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:00.721 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:06:00.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:00.721 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sv1" 2024-05-28T14:06:00.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:00.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:00.721 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone28222" 2024-05-28T14:06:00.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:00.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:00.721 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone28221" 2024-05-28T14:06:00.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:00.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:00.722 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone28227" 2024-05-28T14:06:00.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:00.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:00.722 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone28226" 2024-05-28T14:06:00.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:00.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:00.722 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone28223" 2024-05-28T14:06:00.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:00.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:00.722 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone28225" 2024-05-28T14:06:00.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:00.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:00.723 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone28224" 2024-05-28T14:06:00.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:00.723 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:06:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.734+0000 7f4e09800640 1 -- 172.21.15.42:0/840636388 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4de0068f50 msgr2=0x7f4de006b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.734+0000 7f4e09800640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4de0068f50 0x7f4de006b410 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4e0415f840 tx=0x7f4dec008040 comp rx=0 tx=0).stop 2024-05-28T14:06:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.734+0000 7f4e09800640 1 -- 172.21.15.42:0/840636388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e040cd050 msgr2=0x7f4e04154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.734+0000 7f4e09800640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e040cd050 0x7f4e04154c20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4df4001570 tx=0x7f4df400cd20 comp rx=0 tx=0).stop 2024-05-28T14:06:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.734+0000 7f4e09800640 1 -- 172.21.15.42:0/840636388 shutdown_connections 2024-05-28T14:06:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.734+0000 7f4e09800640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4de0068f50 0x7f4de006b410 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.734+0000 7f4e09800640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e04145d50 0x7f4e04157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.734+0000 7f4e09800640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e040cf540 0x7f4e04157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.734+0000 7f4e09800640 1 --2- 172.21.15.42:0/840636388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e040cd050 0x7f4e04154c20 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.734+0000 7f4e09800640 1 -- 172.21.15.42:0/840636388 >> 172.21.15.42:0/840636388 conn(0x7f4e0406fcc0 msgr2=0x7f4e040c67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.734+0000 7f4e09800640 1 -- 172.21.15.42:0/840636388 shutdown_connections 2024-05-28T14:06:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.734+0000 7f4e09800640 1 -- 172.21.15.42:0/840636388 wait complete. 2024-05-28T14:06:00.743 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-28T14:06:00.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a50200640 1 Processor -- start 2024-05-28T14:06:00.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a50200640 1 -- start start 2024-05-28T14:06:00.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a50200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0a480afa70 0x7f0a4806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a50200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0a480d06f0 0x7f0a480d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a50200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a481570a0 con 0x7f0a4806ee80 2024-05-28T14:06:00.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a50200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0a4815cf00 con 0x7f0a480d06f0 2024-05-28T14:06:00.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a50200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0a4815d1e0 con 0x7f0a480afa70 2024-05-28T14:06:00.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a4e200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0a480d06f0 0x7f0a480d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a4e200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0a480d06f0 0x7f0a480d0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54238/0 (socket says 172.21.15.42:54238) 2024-05-28T14:06:00.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a4e200640 1 -- 172.21.15.42:0/3665092198 learned_addr learned my addr 172.21.15.42:0/3665092198 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:00.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a4ce00640 1 --2- 172.21.15.42:0/3665092198 >> v2:172.21.15.63:3300/0 conn(0x7f0a480afa70 0x7f0a4806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a4e200640 1 -- 172.21.15.42:0/3665092198 >> v2:172.21.15.63:3300/0 conn(0x7f0a480afa70 msgr2=0x7f0a4806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:00.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a4e200640 1 --2- 172.21.15.42:0/3665092198 >> v2:172.21.15.63:3300/0 conn(0x7f0a480afa70 0x7f0a4806dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a4e200640 1 -- 172.21.15.42:0/3665092198 >> v1:172.21.15.63:6789/0 conn(0x7f0a4806ee80 legacy=0x7f0a480af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:00.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a4e200640 1 -- 172.21.15.42:0/3665092198 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4815d4c0 con 0x7f0a480d06f0 2024-05-28T14:06:00.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.902+0000 7f0a4ce00640 1 --2- 172.21.15.42:0/3665092198 >> v2:172.21.15.63:3300/0 conn(0x7f0a480afa70 0x7f0a4806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:00.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.906+0000 7f0a4e200640 1 --2- 172.21.15.42:0/3665092198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a480d06f0 0x7f0a480d0ba0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f0a4809e510 tx=0x7f0a4001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7471e26defcb7d7c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.906+0000 7f0a47e00640 1 -- 172.21.15.42:0/3665092198 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a400a5630 con 0x7f0a480d06f0 2024-05-28T14:06:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.906+0000 7f0a47e00640 1 -- 172.21.15.42:0/3665092198 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a400a57d0 con 0x7f0a480d06f0 2024-05-28T14:06:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.906+0000 7f0a47e00640 1 -- 172.21.15.42:0/3665092198 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a400a5ba0 con 0x7f0a480d06f0 2024-05-28T14:06:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.906+0000 7f0a50200640 1 -- 172.21.15.42:0/3665092198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a480d06f0 msgr2=0x7f0a480d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.906+0000 7f0a50200640 1 --2- 172.21.15.42:0/3665092198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a480d06f0 0x7f0a480d0ba0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f0a4809e510 tx=0x7f0a4001c920 comp rx=0 tx=0).stop 2024-05-28T14:06:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.906+0000 7f0a50200640 1 -- 172.21.15.42:0/3665092198 shutdown_connections 2024-05-28T14:06:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.906+0000 7f0a50200640 1 --2- 172.21.15.42:0/3665092198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a480d06f0 0x7f0a480d0ba0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.906+0000 7f0a50200640 1 --2- 172.21.15.42:0/3665092198 >> v2:172.21.15.63:3300/0 conn(0x7f0a480afa70 0x7f0a4806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.906+0000 7f0a50200640 1 -- 172.21.15.42:0/3665092198 >> 172.21.15.42:0/3665092198 conn(0x7f0a4806bdf0 msgr2=0x7f0a4806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.906+0000 7f0a50200640 1 -- 172.21.15.42:0/3665092198 shutdown_connections 2024-05-28T14:06:00.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.906+0000 7f0a50200640 1 -- 172.21.15.42:0/3665092198 wait complete. 2024-05-28T14:06:00.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.906+0000 7f0a50200640 1 Processor -- start 2024-05-28T14:06:00.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.906+0000 7f0a50200640 1 -- start start 2024-05-28T14:06:00.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a50200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a4806ee80 0x7f0a481546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a50200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a480afa70 0x7f0a48072310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a50200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a480d06f0 0x7f0a4806f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a50200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48160e10 con 0x7f0a480d06f0 2024-05-28T14:06:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a50200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48160c90 con 0x7f0a4806ee80 2024-05-28T14:06:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a50200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0a48160f90 con 0x7f0a480afa70 2024-05-28T14:06:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a4e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a480d06f0 0x7f0a4806f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a4e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a480d06f0 0x7f0a4806f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54248/0 (socket says 172.21.15.42:54248) 2024-05-28T14:06:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a4e200640 1 -- 172.21.15.42:0/2312901421 learned_addr learned my addr 172.21.15.42:0/2312901421 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a4d800640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a4806ee80 0x7f0a481546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a4ce00640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a480afa70 0x7f0a48072310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a4e200640 1 -- 172.21.15.42:0/2312901421 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a480afa70 msgr2=0x7f0a48072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a4e200640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a480afa70 0x7f0a48072310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a4e200640 1 -- 172.21.15.42:0/2312901421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a4806ee80 msgr2=0x7f0a481546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a4e200640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a4806ee80 0x7f0a481546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a4e200640 1 -- 172.21.15.42:0/2312901421 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a48072b60 con 0x7f0a480d06f0 2024-05-28T14:06:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a4d800640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a4806ee80 0x7f0a481546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:00.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a4e200640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a480d06f0 0x7f0a4806f550 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0a4001a550 tx=0x7f0a400196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a45600640 1 -- 172.21.15.42:0/2312901421 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a400bc020 con 0x7f0a480d06f0 2024-05-28T14:06:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a45600640 1 -- 172.21.15.42:0/2312901421 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a400b8670 con 0x7f0a480d06f0 2024-05-28T14:06:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a45600640 1 -- 172.21.15.42:0/2312901421 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a400b8a40 con 0x7f0a480d06f0 2024-05-28T14:06:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a50200640 1 -- 172.21.15.42:0/2312901421 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a48072ea0 con 0x7f0a480d06f0 2024-05-28T14:06:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a50200640 1 -- 172.21.15.42:0/2312901421 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a480736e0 con 0x7f0a480d06f0 2024-05-28T14:06:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.910+0000 7f0a50200640 1 -- 172.21.15.42:0/2312901421 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a48068a90 con 0x7f0a480d06f0 2024-05-28T14:06:00.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.914+0000 7f0a45600640 1 -- 172.21.15.42:0/2312901421 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 89058+0+0 (secure 0 0 0) 0x7f0a400bf020 con 0x7f0a480d06f0 2024-05-28T14:06:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.914+0000 7f0a45600640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0a28068f50 0x7f0a2806b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.914+0000 7f0a45600640 1 -- 172.21.15.42:0/2312901421 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f0a40120640 con 0x7f0a480d06f0 2024-05-28T14:06:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.914+0000 7f0a4d800640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0a28068f50 0x7f0a2806b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.914+0000 7f0a4d800640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0a28068f50 0x7f0a2806b410 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0a480cc9a0 tx=0x7f0a3c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:00.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:00.918+0000 7f0a45600640 1 -- 172.21.15.42:0/2312901421 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0a400c6030 con 0x7f0a480d06f0 2024-05-28T14:06:01.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.110+0000 7f0a50200640 1 -- 172.21.15.42:0/2312901421 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0a480b5160 con 0x7f0a28068f50 2024-05-28T14:06:01.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.122+0000 7f0a45600640 1 -- 172.21.15.42:0/2312901421 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0a480b5160 con 0x7f0a28068f50 2024-05-28T14:06:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:01.111 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:06:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ss1" 2024-05-28T14:06:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:01.111 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:06:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.126+0000 7f0a50200640 1 -- 172.21.15.42:0/2312901421 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0a28068f50 msgr2=0x7f0a2806b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.126+0000 7f0a50200640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0a28068f50 0x7f0a2806b410 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0a480cc9a0 tx=0x7f0a3c009040 comp rx=0 tx=0).stop 2024-05-28T14:06:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.126+0000 7f0a50200640 1 -- 172.21.15.42:0/2312901421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a480d06f0 msgr2=0x7f0a4806f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.126+0000 7f0a50200640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a480d06f0 0x7f0a4806f550 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0a4001a550 tx=0x7f0a400196c0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.126+0000 7f0a50200640 1 -- 172.21.15.42:0/2312901421 shutdown_connections 2024-05-28T14:06:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.126+0000 7f0a50200640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0a28068f50 0x7f0a2806b410 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.126+0000 7f0a50200640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a480d06f0 0x7f0a4806f550 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.126+0000 7f0a50200640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a480afa70 0x7f0a48072310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.126+0000 7f0a50200640 1 --2- 172.21.15.42:0/2312901421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a4806ee80 0x7f0a481546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.126+0000 7f0a50200640 1 -- 172.21.15.42:0/2312901421 >> 172.21.15.42:0/2312901421 conn(0x7f0a4806bdf0 msgr2=0x7f0a48155f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:01.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.126+0000 7f0a50200640 1 -- 172.21.15.42:0/2312901421 shutdown_connections 2024-05-28T14:06:01.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.126+0000 7f0a50200640 1 -- 172.21.15.42:0/2312901421 wait complete. 2024-05-28T14:06:01.133 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:06:01.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6edd800640 1 Processor -- start 2024-05-28T14:06:01.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6edd800640 1 -- start start 2024-05-28T14:06:01.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6edd800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6ed8144f80 0x7f6ed8072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6edd800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ed815c0c0 0x7f6ed806f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6edd800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ed80afe40 con 0x7f6ed806ee20 2024-05-28T14:06:01.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6edd800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed806f7c0 con 0x7f6ed8144f80 2024-05-28T14:06:01.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6edd800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed80724d0 con 0x7f6ed815c0c0 2024-05-28T14:06:01.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6ed6a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6ed8144f80 0x7f6ed8072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6ed6a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6ed8144f80 0x7f6ed8072960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54250/0 (socket says 172.21.15.42:54250) 2024-05-28T14:06:01.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6ed6a00640 1 -- 172.21.15.42:0/1407742204 learned_addr learned my addr 172.21.15.42:0/1407742204 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:01.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6ed7400640 1 --2- 172.21.15.42:0/1407742204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ed815c0c0 0x7f6ed806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6ed6a00640 1 -- 172.21.15.42:0/1407742204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ed815c0c0 msgr2=0x7f6ed806f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:01.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6ed6a00640 1 --2- 172.21.15.42:0/1407742204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ed815c0c0 0x7f6ed806f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6ed6a00640 1 -- 172.21.15.42:0/1407742204 >> v1:172.21.15.63:6789/0 conn(0x7f6ed806ee20 legacy=0x7f6ed815b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:01.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6ed6a00640 1 -- 172.21.15.42:0/1407742204 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed815c600 con 0x7f6ed8144f80 2024-05-28T14:06:01.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.294+0000 7f6ed7400640 1 --2- 172.21.15.42:0/1407742204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ed815c0c0 0x7f6ed806f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:01.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.298+0000 7f6ed6a00640 1 --2- 172.21.15.42:0/1407742204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ed8144f80 0x7f6ed8072960 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009870 tx=0x7f6ec801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166002cb992f49ed server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:01.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.298+0000 7f6ed6000640 1 -- 172.21.15.42:0/1407742204 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80ab650 con 0x7f6ed8144f80 2024-05-28T14:06:01.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.298+0000 7f6ed6000640 1 -- 172.21.15.42:0/1407742204 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec80b0070 con 0x7f6ed8144f80 2024-05-28T14:06:01.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.298+0000 7f6ed6000640 1 -- 172.21.15.42:0/1407742204 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80aba10 con 0x7f6ed8144f80 2024-05-28T14:06:01.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.298+0000 7f6edd800640 1 -- 172.21.15.42:0/1407742204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ed8144f80 msgr2=0x7f6ed8072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:01.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.298+0000 7f6edd800640 1 --2- 172.21.15.42:0/1407742204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ed8144f80 0x7f6ed8072960 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009870 tx=0x7f6ec801c560 comp rx=0 tx=0).stop 2024-05-28T14:06:01.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.298+0000 7f6edd800640 1 -- 172.21.15.42:0/1407742204 shutdown_connections 2024-05-28T14:06:01.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.298+0000 7f6edd800640 1 --2- 172.21.15.42:0/1407742204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ed815c0c0 0x7f6ed806f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.298+0000 7f6edd800640 1 --2- 172.21.15.42:0/1407742204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ed8144f80 0x7f6ed8072960 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.298+0000 7f6edd800640 1 -- 172.21.15.42:0/1407742204 >> 172.21.15.42:0/1407742204 conn(0x7f6ed80b21f0 msgr2=0x7f6ed80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:01.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.298+0000 7f6edd800640 1 -- 172.21.15.42:0/1407742204 shutdown_connections 2024-05-28T14:06:01.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.298+0000 7f6edd800640 1 -- 172.21.15.42:0/1407742204 wait complete. 2024-05-28T14:06:01.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6edd800640 1 Processor -- start 2024-05-28T14:06:01.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6edd800640 1 -- start start 2024-05-28T14:06:01.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6edd800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ed806ee20 0x7f6ed80b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6edd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ed8144f80 0x7f6ed813faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6edd800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ed815c0c0 0x7f6ed8154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6edd800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8072ea0 con 0x7f6ed8144f80 2024-05-28T14:06:01.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6edd800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed80b4010 con 0x7f6ed815c0c0 2024-05-28T14:06:01.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6edd800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed815d540 con 0x7f6ed806ee20 2024-05-28T14:06:01.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ecfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ed8144f80 0x7f6ed813faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ecfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ed8144f80 0x7f6ed813faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54264/0 (socket says 172.21.15.42:54264) 2024-05-28T14:06:01.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ecfe00640 1 -- 172.21.15.42:0/2811716771 learned_addr learned my addr 172.21.15.42:0/2811716771 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:01.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ed6a00640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ed806ee20 0x7f6ed80b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ed7400640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ed815c0c0 0x7f6ed8154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ecfe00640 1 -- 172.21.15.42:0/2811716771 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ed806ee20 msgr2=0x7f6ed80b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:01.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ecfe00640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ed806ee20 0x7f6ed80b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ecfe00640 1 -- 172.21.15.42:0/2811716771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ed815c0c0 msgr2=0x7f6ed8154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:01.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ecfe00640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ed815c0c0 0x7f6ed8154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ecfe00640 1 -- 172.21.15.42:0/2811716771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed80b1f80 con 0x7f6ed8144f80 2024-05-28T14:06:01.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ed7400640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ed815c0c0 0x7f6ed8154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:01.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ecfe00640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ed8144f80 0x7f6ed813faf0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000c420 tx=0x7f6ec000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:01.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ecea00640 1 -- 172.21.15.42:0/2811716771 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00052d0 con 0x7f6ed8144f80 2024-05-28T14:06:01.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ecea00640 1 -- 172.21.15.42:0/2811716771 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec0005470 con 0x7f6ed8144f80 2024-05-28T14:06:01.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6ecea00640 1 -- 172.21.15.42:0/2811716771 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0007260 con 0x7f6ed8144f80 2024-05-28T14:06:01.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6edd800640 1 -- 172.21.15.42:0/2811716771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed8140030 con 0x7f6ed8144f80 2024-05-28T14:06:01.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6edd800640 1 -- 172.21.15.42:0/2811716771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed815cef0 con 0x7f6ed8144f80 2024-05-28T14:06:01.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.302+0000 7f6edd800640 1 -- 172.21.15.42:0/2811716771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed80738b0 con 0x7f6ed8144f80 2024-05-28T14:06:01.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.306+0000 7f6ecea00640 1 -- 172.21.15.42:0/2811716771 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 89058+0+0 (secure 0 0 0) 0x7f6ec0007400 con 0x7f6ed8144f80 2024-05-28T14:06:01.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.306+0000 7f6ecea00640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6eb4068f50 0x7f6eb406b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.306+0000 7f6ecea00640 1 -- 172.21.15.42:0/2811716771 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f6ec00876b0 con 0x7f6ed8144f80 2024-05-28T14:06:01.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.306+0000 7f6ed6a00640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6eb4068f50 0x7f6eb406b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.306+0000 7f6ed6a00640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6eb4068f50 0x7f6eb406b410 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80af000 tx=0x7f6ec80a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:01.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.310+0000 7f6ecea00640 1 -- 172.21.15.42:0/2811716771 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6ec00561d0 con 0x7f6ed8144f80 2024-05-28T14:06:01.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.506+0000 7f6edd800640 1 -- 172.21.15.42:0/2811716771 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ed804fdd0 con 0x7f6eb4068f50 2024-05-28T14:06:01.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.534+0000 7f6ecea00640 1 -- 172.21.15.42:0/2811716771 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6ed804fdd0 con 0x7f6eb4068f50 2024-05-28T14:06:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:01.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.538+0000 7f6edd800640 1 -- 172.21.15.42:0/2811716771 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6eb4068f50 msgr2=0x7f6eb406b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:01.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.538+0000 7f6edd800640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6eb4068f50 0x7f6eb406b410 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80af000 tx=0x7f6ec80a7000 comp rx=0 tx=0).stop 2024-05-28T14:06:01.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.538+0000 7f6edd800640 1 -- 172.21.15.42:0/2811716771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ed8144f80 msgr2=0x7f6ed813faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:01.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.538+0000 7f6edd800640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ed8144f80 0x7f6ed813faf0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000c420 tx=0x7f6ec000c8e0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.538+0000 7f6edd800640 1 -- 172.21.15.42:0/2811716771 shutdown_connections 2024-05-28T14:06:01.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.538+0000 7f6edd800640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6eb4068f50 0x7f6eb406b410 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.538+0000 7f6edd800640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ed815c0c0 0x7f6ed8154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.538+0000 7f6edd800640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ed8144f80 0x7f6ed813faf0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.538+0000 7f6edd800640 1 --2- 172.21.15.42:0/2811716771 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ed806ee20 0x7f6ed80b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.538+0000 7f6edd800640 1 -- 172.21.15.42:0/2811716771 >> 172.21.15.42:0/2811716771 conn(0x7f6ed80b21f0 msgr2=0x7f6ed80b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:01.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.538+0000 7f6edd800640 1 -- 172.21.15.42:0/2811716771 shutdown_connections 2024-05-28T14:06:01.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.538+0000 7f6edd800640 1 -- 172.21.15.42:0/2811716771 wait complete. 2024-05-28T14:06:01.545 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-28T14:06:01.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.702+0000 7fea7be00640 1 Processor -- start 2024-05-28T14:06:01.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.702+0000 7fea7be00640 1 -- start start 2024-05-28T14:06:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.702+0000 7fea7be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea74071de0 0x7fea740721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.702+0000 7fea7be00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fea7413fd10 0x7fea741401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.702+0000 7fea7be00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea74072700 con 0x7fea740ce3b0 2024-05-28T14:06:01.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.702+0000 7fea7be00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fea74160230 con 0x7fea7413fd10 2024-05-28T14:06:01.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.702+0000 7fea7be00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea74160510 con 0x7fea74071de0 2024-05-28T14:06:01.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea78a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fea740ce3b0 0x7fea740ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:45786/0 (socket says 172.21.15.42:45786) 2024-05-28T14:06:01.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea78a00640 1 -- 172.21.15.42:0/1491430224 learned_addr learned my addr 172.21.15.42:0/1491430224 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:01.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea79400640 1 --2- 172.21.15.42:0/1491430224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea74071de0 0x7fea740721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea79e00640 1 --2- 172.21.15.42:0/1491430224 >> v2:172.21.15.63:3300/0 conn(0x7fea7413fd10 0x7fea741401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 -- 172.21.15.42:0/1491430224 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 93349227 0 0) 0x7fea74072700 con 0x7fea740ce3b0 2024-05-28T14:06:01.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 -- 172.21.15.42:0/1491430224 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea5c003530 con 0x7fea740ce3b0 2024-05-28T14:06:01.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 -- 172.21.15.42:0/1491430224 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3409175693 0 0) 0x7fea5c003530 con 0x7fea740ce3b0 2024-05-28T14:06:01.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 -- 172.21.15.42:0/1491430224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea74071de0 msgr2=0x7fea740721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:01.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 --2- 172.21.15.42:0/1491430224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea74071de0 0x7fea740721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 -- 172.21.15.42:0/1491430224 >> v2:172.21.15.63:3300/0 conn(0x7fea7413fd10 msgr2=0x7fea741401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:01.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 --2- 172.21.15.42:0/1491430224 >> v2:172.21.15.63:3300/0 conn(0x7fea7413fd10 0x7fea741401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 -- 172.21.15.42:0/1491430224 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea741607f0 con 0x7fea740ce3b0 2024-05-28T14:06:01.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 -- 172.21.15.42:0/1491430224 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fea60001680 con 0x7fea740ce3b0 2024-05-28T14:06:01.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 -- 172.21.15.42:0/1491430224 >> v1:172.21.15.42:6789/0 conn(0x7fea740ce3b0 legacy=0x7fea740ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:01.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 --2- 172.21.15.42:0/1491430224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c0039c0 0x7fea5c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 --2- 172.21.15.42:0/1491430224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea5c004810 0x7fea5c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 --2- 172.21.15.42:0/1491430224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea74071de0 0x7fea5c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 -- 172.21.15.42:0/1491430224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea74160230 con 0x7fea5c004810 2024-05-28T14:06:01.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 -- 172.21.15.42:0/1491430224 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea74160510 con 0x7fea5c0039c0 2024-05-28T14:06:01.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea73e00640 1 -- 172.21.15.42:0/1491430224 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c00ba50 con 0x7fea74071de0 2024-05-28T14:06:01.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea78a00640 1 --2- 172.21.15.42:0/1491430224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea5c004810 0x7fea5c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea79e00640 1 --2- 172.21.15.42:0/1491430224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea74071de0 0x7fea5c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea79400640 1 --2- 172.21.15.42:0/1491430224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c0039c0 0x7fea5c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea78a00640 1 -- 172.21.15.42:0/1491430224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea74071de0 msgr2=0x7fea5c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:01.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea78a00640 1 --2- 172.21.15.42:0/1491430224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea74071de0 0x7fea5c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea78a00640 1 -- 172.21.15.42:0/1491430224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c0039c0 msgr2=0x7fea5c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:01.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea78a00640 1 --2- 172.21.15.42:0/1491430224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c0039c0 0x7fea5c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.706+0000 7fea78a00640 1 -- 172.21.15.42:0/1491430224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c003530 con 0x7fea5c004810 2024-05-28T14:06:01.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.710+0000 7fea78a00640 1 --2- 172.21.15.42:0/1491430224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea5c004810 0x7fea5c004c90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fea6000c2c0 tx=0x7fea6001cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9800074bf5a0b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:01.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.710+0000 7fea73e00640 1 -- 172.21.15.42:0/1491430224 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60007d60 con 0x7fea5c004810 2024-05-28T14:06:01.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.710+0000 7fea73e00640 1 -- 172.21.15.42:0/1491430224 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea600a9020 con 0x7fea5c004810 2024-05-28T14:06:01.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.710+0000 7fea73e00640 1 -- 172.21.15.42:0/1491430224 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60005ae0 con 0x7fea5c004810 2024-05-28T14:06:01.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.710+0000 7fea7be00640 1 -- 172.21.15.42:0/1491430224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea5c004810 msgr2=0x7fea5c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:01.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.710+0000 7fea7be00640 1 --2- 172.21.15.42:0/1491430224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea5c004810 0x7fea5c004c90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fea6000c2c0 tx=0x7fea6001cd40 comp rx=0 tx=0).stop 2024-05-28T14:06:01.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.710+0000 7fea79e00640 1 -- 172.21.15.42:0/1491430224 reap_dead start 2024-05-28T14:06:01.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.710+0000 7fea7be00640 1 -- 172.21.15.42:0/1491430224 shutdown_connections 2024-05-28T14:06:01.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.710+0000 7fea7be00640 1 -- 172.21.15.42:0/1491430224 >> 172.21.15.42:0/1491430224 conn(0x7fea740b21f0 msgr2=0x7fea740b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:01.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.710+0000 7fea7be00640 1 -- 172.21.15.42:0/1491430224 shutdown_connections 2024-05-28T14:06:01.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.710+0000 7fea7be00640 1 -- 172.21.15.42:0/1491430224 wait complete. 2024-05-28T14:06:01.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.710+0000 7fea7be00640 1 Processor -- start 2024-05-28T14:06:01.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.710+0000 7fea7be00640 1 -- start start 2024-05-28T14:06:01.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea7be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea5c004810 0x7fea7415be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea7be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea741323a0 0x7fea74132820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea7be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea7406ed20 0x7fea7406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea7be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea74167820 con 0x7fea741323a0 2024-05-28T14:06:01.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea7be00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea741676a0 con 0x7fea7406ed20 2024-05-28T14:06:01.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea7be00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea741679a0 con 0x7fea5c004810 2024-05-28T14:06:01.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea78a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea741323a0 0x7fea74132820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea78a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea741323a0 0x7fea74132820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54290/0 (socket says 172.21.15.42:54290) 2024-05-28T14:06:01.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea78a00640 1 -- 172.21.15.42:0/519657039 learned_addr learned my addr 172.21.15.42:0/519657039 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:01.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea79400640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea5c004810 0x7fea7415be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea79e00640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea7406ed20 0x7fea7406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea78a00640 1 -- 172.21.15.42:0/519657039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea5c004810 msgr2=0x7fea7415be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea78a00640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea5c004810 0x7fea7415be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea78a00640 1 -- 172.21.15.42:0/519657039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea7406ed20 msgr2=0x7fea7406f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea78a00640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea7406ed20 0x7fea7406f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea78a00640 1 -- 172.21.15.42:0/519657039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea740cda20 con 0x7fea741323a0 2024-05-28T14:06:01.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea79e00640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea7406ed20 0x7fea7406f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea78a00640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea741323a0 0x7fea74132820 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fea6001cb70 tx=0x7fea5c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea71600640 1 -- 172.21.15.42:0/519657039 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60001c90 con 0x7fea741323a0 2024-05-28T14:06:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea71600640 1 -- 172.21.15.42:0/519657039 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea600049f0 con 0x7fea741323a0 2024-05-28T14:06:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea71600640 1 -- 172.21.15.42:0/519657039 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60004de0 con 0x7fea741323a0 2024-05-28T14:06:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea7be00640 1 -- 172.21.15.42:0/519657039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7415c370 con 0x7fea741323a0 2024-05-28T14:06:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea7be00640 1 -- 172.21.15.42:0/519657039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea74164360 con 0x7fea741323a0 2024-05-28T14:06:01.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea7be00640 1 -- 172.21.15.42:0/519657039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea740738b0 con 0x7fea741323a0 2024-05-28T14:06:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.714+0000 7fea71600640 1 -- 172.21.15.42:0/519657039 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 89058+0+0 (secure 0 0 0) 0x7fea74140ef0 con 0x7fea741323a0 2024-05-28T14:06:01.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.718+0000 7fea71600640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea54068f80 0x7fea5406b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.718+0000 7fea71600640 1 -- 172.21.15.42:0/519657039 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fea6011e7c0 con 0x7fea741323a0 2024-05-28T14:06:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.718+0000 7fea79400640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea54068f80 0x7fea5406b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:01.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.718+0000 7fea79400640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea54068f80 0x7fea5406b440 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fea68009a80 tx=0x7fea68009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:01.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.722+0000 7fea71600640 1 -- 172.21.15.42:0/519657039 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fea740cda20 con 0x7fea741323a0 2024-05-28T14:06:01.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.910+0000 7fea7be00640 1 -- 172.21.15.42:0/519657039 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fea7404fdd0 con 0x7fea54068f80 2024-05-28T14:06:01.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.930+0000 7fea71600640 1 -- 172.21.15.42:0/519657039 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fea7404fdd0 con 0x7fea54068f80 2024-05-28T14:06:01.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.934+0000 7fea7be00640 1 -- 172.21.15.42:0/519657039 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea54068f80 msgr2=0x7fea5406b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:01.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.934+0000 7fea7be00640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea54068f80 0x7fea5406b440 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fea68009a80 tx=0x7fea68009190 comp rx=0 tx=0).stop 2024-05-28T14:06:01.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.934+0000 7fea7be00640 1 -- 172.21.15.42:0/519657039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea741323a0 msgr2=0x7fea74132820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:01.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.934+0000 7fea7be00640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea741323a0 0x7fea74132820 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fea6001cb70 tx=0x7fea5c005840 comp rx=0 tx=0).stop 2024-05-28T14:06:01.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.934+0000 7fea7be00640 1 -- 172.21.15.42:0/519657039 shutdown_connections 2024-05-28T14:06:01.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.934+0000 7fea7be00640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea54068f80 0x7fea5406b440 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.934+0000 7fea7be00640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea7406ed20 0x7fea7406f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.934+0000 7fea7be00640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea741323a0 0x7fea74132820 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.934+0000 7fea7be00640 1 --2- 172.21.15.42:0/519657039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea5c004810 0x7fea7415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:01.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.934+0000 7fea7be00640 1 -- 172.21.15.42:0/519657039 >> 172.21.15.42:0/519657039 conn(0x7fea740b21f0 msgr2=0x7fea740afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:01.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.934+0000 7fea7be00640 1 -- 172.21.15.42:0/519657039 shutdown_connections 2024-05-28T14:06:01.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:01.934+0000 7fea7be00640 1 -- 172.21.15.42:0/519657039 wait complete. 2024-05-28T14:06:01.921 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:01.934+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:01.921 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:01.934+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:01.921 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:01.934+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:01.921 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:01.934+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:01.922 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:01.934+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:01.939 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:01.954+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:01.939 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:01.954+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:01.939 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:01.954+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:01.939 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:01.954+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:01.939 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:01.954+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:01.941 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone28222 2024-05-28T14:06:02.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb7e00640 1 Processor -- start 2024-05-28T14:06:02.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb7e00640 1 -- start start 2024-05-28T14:06:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3bb815e460 0x7f3bb80d2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb7e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3bb80d1520 0x7f3bb80d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bb80d1be0 0x7f3bb80714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb7e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb80ced30 con 0x7f3bb815e460 2024-05-28T14:06:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb7e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb8162190 con 0x7f3bb80d1520 2024-05-28T14:06:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb7e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb8162470 con 0x7f3bb80d1be0 2024-05-28T14:06:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb6a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3bb815e460 0x7f3bb80d2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb7400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bb80d1be0 0x7f3bb80714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb6a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3bb815e460 0x7f3bb80d2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54318/0 (socket says 172.21.15.42:54318) 2024-05-28T14:06:02.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb6a00640 1 -- 172.21.15.42:0/1402880961 learned_addr learned my addr 172.21.15.42:0/1402880961 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:02.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bafe00640 1 --2- 172.21.15.42:0/1402880961 >> v2:172.21.15.63:3300/0 conn(0x7f3bb80d1520 0x7f3bb80d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb7400640 1 -- 172.21.15.42:0/1402880961 >> v2:172.21.15.63:3300/0 conn(0x7f3bb80d1520 msgr2=0x7f3bb80d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:02.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb7400640 1 --2- 172.21.15.42:0/1402880961 >> v2:172.21.15.63:3300/0 conn(0x7f3bb80d1520 0x7f3bb80d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb7400640 1 -- 172.21.15.42:0/1402880961 >> v2:172.21.15.42:3300/0 conn(0x7f3bb815e460 msgr2=0x7f3bb80d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:02.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb7400640 1 --2- 172.21.15.42:0/1402880961 >> v2:172.21.15.42:3300/0 conn(0x7f3bb815e460 0x7f3bb80d2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.106+0000 7f3bb7400640 1 -- 172.21.15.42:0/1402880961 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb8162750 con 0x7f3bb80d1be0 2024-05-28T14:06:02.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.110+0000 7f3bb7400640 1 --2- 172.21.15.42:0/1402880961 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bb80d1be0 0x7f3bb80714d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3bb806f540 tx=0x7f3bb001c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cba03913d993e26 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.110+0000 7f3bb6000640 1 -- 172.21.15.42:0/1402880961 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb00a8650 con 0x7f3bb80d1be0 2024-05-28T14:06:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.110+0000 7f3bb6000640 1 -- 172.21.15.42:0/1402880961 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb00a87f0 con 0x7f3bb80d1be0 2024-05-28T14:06:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.110+0000 7f3bb6000640 1 -- 172.21.15.42:0/1402880961 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb00a8bc0 con 0x7f3bb80d1be0 2024-05-28T14:06:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.110+0000 7f3bb7e00640 1 -- 172.21.15.42:0/1402880961 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bb80d1be0 msgr2=0x7f3bb80714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.110+0000 7f3bb7e00640 1 --2- 172.21.15.42:0/1402880961 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bb80d1be0 0x7f3bb80714d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3bb806f540 tx=0x7f3bb001c670 comp rx=0 tx=0).stop 2024-05-28T14:06:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.110+0000 7f3bb7e00640 1 -- 172.21.15.42:0/1402880961 shutdown_connections 2024-05-28T14:06:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.110+0000 7f3bb7e00640 1 --2- 172.21.15.42:0/1402880961 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bb80d1be0 0x7f3bb80714d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.110+0000 7f3bb7e00640 1 --2- 172.21.15.42:0/1402880961 >> v2:172.21.15.63:3300/0 conn(0x7f3bb80d1520 0x7f3bb80d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.110+0000 7f3bb7e00640 1 --2- 172.21.15.42:0/1402880961 >> v2:172.21.15.42:3300/0 conn(0x7f3bb815e460 0x7f3bb80d2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.110+0000 7f3bb7e00640 1 -- 172.21.15.42:0/1402880961 >> 172.21.15.42:0/1402880961 conn(0x7f3bb806fd70 msgr2=0x7f3bb8070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.110+0000 7f3bb7e00640 1 -- 172.21.15.42:0/1402880961 shutdown_connections 2024-05-28T14:06:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.110+0000 7f3bb7e00640 1 -- 172.21.15.42:0/1402880961 wait complete. 2024-05-28T14:06:02.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7e00640 1 Processor -- start 2024-05-28T14:06:02.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7e00640 1 -- start start 2024-05-28T14:06:02.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bb80d1520 0x7f3bb80b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bb80d1be0 0x7f3bb8155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb815e460 0x7f3bb813e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb815e9c0 con 0x7f3bb815e460 2024-05-28T14:06:02.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb815e840 con 0x7f3bb80d1be0 2024-05-28T14:06:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb806d1c0 con 0x7f3bb80d1520 2024-05-28T14:06:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb815e460 0x7f3bb813e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb815e460 0x7f3bb813e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54320/0 (socket says 172.21.15.42:54320) 2024-05-28T14:06:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7400640 1 -- 172.21.15.42:0/2681016487 learned_addr learned my addr 172.21.15.42:0/2681016487 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb6a00640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bb80d1520 0x7f3bb80b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bafe00640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bb80d1be0 0x7f3bb8155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7400640 1 -- 172.21.15.42:0/2681016487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bb80d1520 msgr2=0x7f3bb80b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7400640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bb80d1520 0x7f3bb80b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7400640 1 -- 172.21.15.42:0/2681016487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bb80d1be0 msgr2=0x7f3bb8155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7400640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bb80d1be0 0x7f3bb8155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7400640 1 -- 172.21.15.42:0/2681016487 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb80b2d80 con 0x7f3bb815e460 2024-05-28T14:06:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bafe00640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bb80d1be0 0x7f3bb8155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:02.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7400640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb815e460 0x7f3bb813e800 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3bb001ad20 tx=0x7f3bb0019410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:02.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3baea00640 1 -- 172.21.15.42:0/2681016487 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb00b0600 con 0x7f3bb815e460 2024-05-28T14:06:02.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3baea00640 1 -- 172.21.15.42:0/2681016487 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb00bf070 con 0x7f3bb815e460 2024-05-28T14:06:02.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7e00640 1 -- 172.21.15.42:0/2681016487 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb80b3000 con 0x7f3bb815e460 2024-05-28T14:06:02.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3baea00640 1 -- 172.21.15.42:0/2681016487 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb00b09c0 con 0x7f3bb815e460 2024-05-28T14:06:02.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7e00640 1 -- 172.21.15.42:0/2681016487 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb80b35c0 con 0x7f3bb815e460 2024-05-28T14:06:02.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.114+0000 7f3bb7e00640 1 -- 172.21.15.42:0/2681016487 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb8068a90 con 0x7f3bb815e460 2024-05-28T14:06:02.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.118+0000 7f3baea00640 1 -- 172.21.15.42:0/2681016487 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 89058+0+0 (secure 0 0 0) 0x7f3bb00a6040 con 0x7f3bb815e460 2024-05-28T14:06:02.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.118+0000 7f3baea00640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b94068f50 0x7f3b9406b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.118+0000 7f3baea00640 1 -- 172.21.15.42:0/2681016487 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f3bb0120580 con 0x7f3bb815e460 2024-05-28T14:06:02.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.118+0000 7f3bb6a00640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b94068f50 0x7f3b9406b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.118+0000 7f3bb6a00640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b94068f50 0x7f3b9406b410 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80099c0 tx=0x7f3ba8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:02.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.122+0000 7f3baea00640 1 -- 172.21.15.42:0/2681016487 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3bb80b2d80 con 0x7f3bb815e460 2024-05-28T14:06:02.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.314+0000 7f3bb7e00640 1 -- 172.21.15.42:0/2681016487 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone28222", "target": ["mon-mgr", ""]}) -- 0x7f3bb815bd80 con 0x7f3b94068f50 2024-05-28T14:06:02.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.338+0000 7f3baea00640 1 -- 172.21.15.42:0/2681016487 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3bb815bd80 con 0x7f3b94068f50 2024-05-28T14:06:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.338+0000 7f3bb7e00640 1 -- 172.21.15.42:0/2681016487 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b94068f50 msgr2=0x7f3b9406b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.338+0000 7f3bb7e00640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b94068f50 0x7f3b9406b410 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80099c0 tx=0x7f3ba8008040 comp rx=0 tx=0).stop 2024-05-28T14:06:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.338+0000 7f3bb7e00640 1 -- 172.21.15.42:0/2681016487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb815e460 msgr2=0x7f3bb813e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.338+0000 7f3bb7e00640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb815e460 0x7f3bb813e800 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3bb001ad20 tx=0x7f3bb0019410 comp rx=0 tx=0).stop 2024-05-28T14:06:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.338+0000 7f3bb7e00640 1 -- 172.21.15.42:0/2681016487 shutdown_connections 2024-05-28T14:06:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.338+0000 7f3bb7e00640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3b94068f50 0x7f3b9406b410 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.338+0000 7f3bb7e00640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb815e460 0x7f3bb813e800 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.338+0000 7f3bb7e00640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bb80d1be0 0x7f3bb8155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.338+0000 7f3bb7e00640 1 --2- 172.21.15.42:0/2681016487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bb80d1520 0x7f3bb80b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.338+0000 7f3bb7e00640 1 -- 172.21.15.42:0/2681016487 >> 172.21.15.42:0/2681016487 conn(0x7f3bb806fd70 msgr2=0x7f3bb815d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:02.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.342+0000 7f3bb7e00640 1 -- 172.21.15.42:0/2681016487 shutdown_connections 2024-05-28T14:06:02.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.342+0000 7f3bb7e00640 1 -- 172.21.15.42:0/2681016487 wait complete. 2024-05-28T14:06:02.350 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone28221 2024-05-28T14:06:02.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.510+0000 7f58d0a00640 1 Processor -- start 2024-05-28T14:06:02.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.510+0000 7f58d0a00640 1 -- start start 2024-05-28T14:06:02.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58d0a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc06b9f0 0x7f58cc06be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58d0a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f58cc1600b0 0x7f58cc160540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58d0a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58cc0b6410 con 0x7f58cc0b19d0 2024-05-28T14:06:02.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58d0a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f58cc0b2c60 con 0x7f58cc1600b0 2024-05-28T14:06:02.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58d0a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f58cc06c390 con 0x7f58cc06b9f0 2024-05-28T14:06:02.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58c2a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc06b9f0 0x7f58cc06be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58c2a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc06b9f0 0x7f58cc06be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:58524/0 (socket says 172.21.15.42:58524) 2024-05-28T14:06:02.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58c2a00640 1 -- 172.21.15.42:0/139169613 learned_addr learned my addr 172.21.15.42:0/139169613 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:02.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58cb400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f58cc0b19d0 0x7f58cc0b2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:43382/0 (socket says 172.21.15.42:43382) 2024-05-28T14:06:02.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58cbe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f58cc1600b0 0x7f58cc160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58c2a00640 1 -- 172.21.15.42:0/139169613 >> v2:172.21.15.63:3300/0 conn(0x7f58cc1600b0 msgr2=0x7f58cc160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:02.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58c2a00640 1 --2- 172.21.15.42:0/139169613 >> v2:172.21.15.63:3300/0 conn(0x7f58cc1600b0 0x7f58cc160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58c2a00640 1 -- 172.21.15.42:0/139169613 >> v1:172.21.15.63:6789/0 conn(0x7f58cc0b19d0 legacy=0x7f58cc0b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58caa00640 1 -- 172.21.15.42:0/139169613 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3603786604 0 0) 0x7f58cc0b6410 con 0x7f58cc0b19d0 2024-05-28T14:06:02.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58c2a00640 1 -- 172.21.15.42:0/139169613 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc0b1db0 con 0x7f58cc06b9f0 2024-05-28T14:06:02.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.514+0000 7f58cbe00640 1 --2- 172.21.15.42:0/139169613 >> v2:172.21.15.63:3300/0 conn(0x7f58cc1600b0 0x7f58cc160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:06:02.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58c2a00640 1 --2- 172.21.15.42:0/139169613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc06b9f0 0x7f58cc06be50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009870 tx=0x7f58b401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6173698a3a5fcc18 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:02.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58caa00640 1 -- 172.21.15.42:0/139169613 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b40ab650 con 0x7f58cc06b9f0 2024-05-28T14:06:02.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58caa00640 1 -- 172.21.15.42:0/139169613 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b40b0070 con 0x7f58cc06b9f0 2024-05-28T14:06:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58caa00640 1 -- 172.21.15.42:0/139169613 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b40aba10 con 0x7f58cc06b9f0 2024-05-28T14:06:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 -- 172.21.15.42:0/139169613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc06b9f0 msgr2=0x7f58cc06be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 --2- 172.21.15.42:0/139169613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc06b9f0 0x7f58cc06be50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009870 tx=0x7f58b401c560 comp rx=0 tx=0).stop 2024-05-28T14:06:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 -- 172.21.15.42:0/139169613 shutdown_connections 2024-05-28T14:06:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 --2- 172.21.15.42:0/139169613 >> v2:172.21.15.63:3300/0 conn(0x7f58cc1600b0 0x7f58cc160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 --2- 172.21.15.42:0/139169613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc06b9f0 0x7f58cc06be50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 -- 172.21.15.42:0/139169613 >> 172.21.15.42:0/139169613 conn(0x7f58cc0b6600 msgr2=0x7f58cc0b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:02.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 -- 172.21.15.42:0/139169613 shutdown_connections 2024-05-28T14:06:02.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 -- 172.21.15.42:0/139169613 wait complete. 2024-05-28T14:06:02.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 Processor -- start 2024-05-28T14:06:02.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 -- start start 2024-05-28T14:06:02.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc06b9f0 0x7f58cc15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58cc0b19d0 0x7f58cc09ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1600b0 0x7f58cc09ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc0b4310 con 0x7f58cc06b9f0 2024-05-28T14:06:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc0b5b40 con 0x7f58cc0b19d0 2024-05-28T14:06:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.518+0000 7f58d0a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f58cc16b360 con 0x7f58cc1600b0 2024-05-28T14:06:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58cb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc06b9f0 0x7f58cc15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58cb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc06b9f0 0x7f58cc15e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54322/0 (socket says 172.21.15.42:54322) 2024-05-28T14:06:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58cb400640 1 -- 172.21.15.42:0/1486377749 learned_addr learned my addr 172.21.15.42:0/1486377749 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58c2a00640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58cc0b19d0 0x7f58cc09ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58cbe00640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1600b0 0x7f58cc09ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58cb400640 1 -- 172.21.15.42:0/1486377749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1600b0 msgr2=0x7f58cc09ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58cb400640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1600b0 0x7f58cc09ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58cb400640 1 -- 172.21.15.42:0/1486377749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58cc0b19d0 msgr2=0x7f58cc09ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58cb400640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58cc0b19d0 0x7f58cc09ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58cb400640 1 -- 172.21.15.42:0/1486377749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc1401b0 con 0x7f58cc06b9f0 2024-05-28T14:06:02.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58c2a00640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58cc0b19d0 0x7f58cc09ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:02.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58cb400640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc06b9f0 0x7f58cc15e3a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0011f0 tx=0x7f58bc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:02.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58c3e00640 1 -- 172.21.15.42:0/1486377749 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc018070 con 0x7f58cc06b9f0 2024-05-28T14:06:02.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58c3e00640 1 -- 172.21.15.42:0/1486377749 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58bc0074e0 con 0x7f58cc06b9f0 2024-05-28T14:06:02.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58c3e00640 1 -- 172.21.15.42:0/1486377749 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc004a10 con 0x7f58cc06b9f0 2024-05-28T14:06:02.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58d0a00640 1 -- 172.21.15.42:0/1486377749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc140480 con 0x7f58cc06b9f0 2024-05-28T14:06:02.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58d0a00640 1 -- 172.21.15.42:0/1486377749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc140a40 con 0x7f58cc06b9f0 2024-05-28T14:06:02.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58d0a00640 1 -- 172.21.15.42:0/1486377749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58cc068a90 con 0x7f58cc06b9f0 2024-05-28T14:06:02.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.522+0000 7f58c3e00640 1 -- 172.21.15.42:0/1486377749 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 89058+0+0 (secure 0 0 0) 0x7f58bc004bb0 con 0x7f58cc06b9f0 2024-05-28T14:06:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.526+0000 7f58c3e00640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f58a8068f50 0x7f58a806b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.526+0000 7f58c3e00640 1 -- 172.21.15.42:0/1486377749 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f58bc084a90 con 0x7f58cc06b9f0 2024-05-28T14:06:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.526+0000 7f58c2a00640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f58a8068f50 0x7f58a806b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.526+0000 7f58c2a00640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f58a8068f50 0x7f58a806b410 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009870 tx=0x7f58b40a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:02.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.530+0000 7f58c3e00640 1 -- 172.21.15.42:0/1486377749 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f58bc056780 con 0x7f58cc06b9f0 2024-05-28T14:06:02.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.718+0000 7f58d0a00640 1 -- 172.21.15.42:0/1486377749 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone28221", "target": ["mon-mgr", ""]}) -- 0x7f58cc09c860 con 0x7f58a8068f50 2024-05-28T14:06:02.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.738+0000 7f58c3e00640 1 -- 172.21.15.42:0/1486377749 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f58cc09c860 con 0x7f58a8068f50 2024-05-28T14:06:02.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.742+0000 7f58d0a00640 1 -- 172.21.15.42:0/1486377749 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f58a8068f50 msgr2=0x7f58a806b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.742+0000 7f58d0a00640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f58a8068f50 0x7f58a806b410 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009870 tx=0x7f58b40a7000 comp rx=0 tx=0).stop 2024-05-28T14:06:02.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.742+0000 7f58d0a00640 1 -- 172.21.15.42:0/1486377749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc06b9f0 msgr2=0x7f58cc15e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.742+0000 7f58d0a00640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc06b9f0 0x7f58cc15e3a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0011f0 tx=0x7f58bc00c9b0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.742+0000 7f58d0a00640 1 -- 172.21.15.42:0/1486377749 shutdown_connections 2024-05-28T14:06:02.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.742+0000 7f58d0a00640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f58a8068f50 0x7f58a806b410 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.742+0000 7f58d0a00640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1600b0 0x7f58cc09ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.742+0000 7f58d0a00640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58cc0b19d0 0x7f58cc09ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.742+0000 7f58d0a00640 1 --2- 172.21.15.42:0/1486377749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58cc06b9f0 0x7f58cc15e3a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.742+0000 7f58d0a00640 1 -- 172.21.15.42:0/1486377749 >> 172.21.15.42:0/1486377749 conn(0x7f58cc0b6600 msgr2=0x7f58cc141ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:02.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.742+0000 7f58d0a00640 1 -- 172.21.15.42:0/1486377749 shutdown_connections 2024-05-28T14:06:02.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.742+0000 7f58d0a00640 1 -- 172.21.15.42:0/1486377749 wait complete. 2024-05-28T14:06:02.752 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone28227 2024-05-28T14:06:02.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.910+0000 7fc84a200640 1 Processor -- start 2024-05-28T14:06:02.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.910+0000 7fc84a200640 1 -- start start 2024-05-28T14:06:02.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.914+0000 7fc84a200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc8440b3a50 0x7fc8440cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.914+0000 7fc84a200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc84406b9f0 0x7fc84406be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.914+0000 7fc84a200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8440cf190 con 0x7fc84415eb10 2024-05-28T14:06:02.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.914+0000 7fc84a200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc8440d05e0 con 0x7fc8440b3a50 2024-05-28T14:06:02.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.914+0000 7fc84a200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc84406c380 con 0x7fc84406b9f0 2024-05-28T14:06:02.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.914+0000 7fc842a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc8440b3a50 0x7fc8440cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.914+0000 7fc843400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc84415eb10 0x7fc8440cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:45788/0 (socket says 172.21.15.42:45788) 2024-05-28T14:06:02.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.914+0000 7fc843400640 1 -- 172.21.15.42:0/319987144 learned_addr learned my addr 172.21.15.42:0/319987144 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:02.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.914+0000 7fc843e00640 1 --2- 172.21.15.42:0/319987144 >> v2:172.21.15.63:3300/0 conn(0x7fc84406b9f0 0x7fc84406be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.914+0000 7fc842a00640 1 -- 172.21.15.42:0/319987144 >> v2:172.21.15.63:3300/0 conn(0x7fc84406b9f0 msgr2=0x7fc84406be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:02.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.914+0000 7fc842a00640 1 --2- 172.21.15.42:0/319987144 >> v2:172.21.15.63:3300/0 conn(0x7fc84406b9f0 0x7fc84406be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.914+0000 7fc842a00640 1 -- 172.21.15.42:0/319987144 >> v1:172.21.15.42:6789/0 conn(0x7fc84415eb10 legacy=0x7fc8440cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.914+0000 7fc842a00640 1 -- 172.21.15.42:0/319987144 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8440d0810 con 0x7fc8440b3a50 2024-05-28T14:06:02.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.918+0000 7fc842a00640 1 --2- 172.21.15.42:0/319987144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8440b3a50 0x7fc8440cd520 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc8440b2ef0 tx=0x7fc83001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c692660a50a30d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:02.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.918+0000 7fc842000640 1 -- 172.21.15.42:0/319987144 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8300b0070 con 0x7fc8440b3a50 2024-05-28T14:06:02.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.918+0000 7fc842000640 1 -- 172.21.15.42:0/319987144 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8300ab600 con 0x7fc8440b3a50 2024-05-28T14:06:02.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.918+0000 7fc842000640 1 -- 172.21.15.42:0/319987144 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8300ab9d0 con 0x7fc8440b3a50 2024-05-28T14:06:02.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.918+0000 7fc84a200640 1 -- 172.21.15.42:0/319987144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8440b3a50 msgr2=0x7fc8440cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.918+0000 7fc84a200640 1 --2- 172.21.15.42:0/319987144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8440b3a50 0x7fc8440cd520 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc8440b2ef0 tx=0x7fc83001c3f0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.918+0000 7fc84a200640 1 -- 172.21.15.42:0/319987144 shutdown_connections 2024-05-28T14:06:02.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.918+0000 7fc84a200640 1 --2- 172.21.15.42:0/319987144 >> v2:172.21.15.63:3300/0 conn(0x7fc84406b9f0 0x7fc84406be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.918+0000 7fc84a200640 1 --2- 172.21.15.42:0/319987144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8440b3a50 0x7fc8440cd520 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.918+0000 7fc84a200640 1 -- 172.21.15.42:0/319987144 >> 172.21.15.42:0/319987144 conn(0x7fc8440b2f60 msgr2=0x7fc8440b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:02.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.918+0000 7fc84a200640 1 -- 172.21.15.42:0/319987144 shutdown_connections 2024-05-28T14:06:02.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.918+0000 7fc84a200640 1 -- 172.21.15.42:0/319987144 wait complete. 2024-05-28T14:06:02.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.918+0000 7fc84a200640 1 Processor -- start 2024-05-28T14:06:02.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.918+0000 7fc84a200640 1 -- start start 2024-05-28T14:06:02.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc84a200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc84406b9f0 0x7fc844156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc84a200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8440b3a50 0x7fc8441534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc84a200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc84415eb10 0x7fc8440b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc84a200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc8440ceab0 con 0x7fc8440b3a50 2024-05-28T14:06:02.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc84a200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8440ce930 con 0x7fc84406b9f0 2024-05-28T14:06:02.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc84a200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc8440cec30 con 0x7fc84415eb10 2024-05-28T14:06:02.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc842a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8440b3a50 0x7fc8441534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc842a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8440b3a50 0x7fc8441534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54328/0 (socket says 172.21.15.42:54328) 2024-05-28T14:06:02.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc842a00640 1 -- 172.21.15.42:0/3283884438 learned_addr learned my addr 172.21.15.42:0/3283884438 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:02.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc843e00640 1 --2- 172.21.15.42:0/3283884438 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc84415eb10 0x7fc8440b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc843400640 1 --2- 172.21.15.42:0/3283884438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc84406b9f0 0x7fc844156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc842a00640 1 -- 172.21.15.42:0/3283884438 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc84415eb10 msgr2=0x7fc8440b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc842a00640 1 --2- 172.21.15.42:0/3283884438 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc84415eb10 0x7fc8440b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc842a00640 1 -- 172.21.15.42:0/3283884438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc84406b9f0 msgr2=0x7fc844156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:02.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc842a00640 1 --2- 172.21.15.42:0/3283884438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc84406b9f0 0x7fc844156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:02.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc842a00640 1 -- 172.21.15.42:0/3283884438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8440aedf0 con 0x7fc8440b3a50 2024-05-28T14:06:02.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc842a00640 1 --2- 172.21.15.42:0/3283884438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8440b3a50 0x7fc8441534f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc8300ae040 tx=0x7fc830019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:02.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc83b400640 1 -- 172.21.15.42:0/3283884438 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8300b0040 con 0x7fc8440b3a50 2024-05-28T14:06:02.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc83b400640 1 -- 172.21.15.42:0/3283884438 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8300b9070 con 0x7fc8440b3a50 2024-05-28T14:06:02.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc83b400640 1 -- 172.21.15.42:0/3283884438 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830006e20 con 0x7fc8440b3a50 2024-05-28T14:06:02.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc84a200640 1 -- 172.21.15.42:0/3283884438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8440af090 con 0x7fc8440b3a50 2024-05-28T14:06:02.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc84a200640 1 -- 172.21.15.42:0/3283884438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8440af650 con 0x7fc8440b3a50 2024-05-28T14:06:02.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc84a200640 1 -- 172.21.15.42:0/3283884438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc844068a90 con 0x7fc8440b3a50 2024-05-28T14:06:02.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.922+0000 7fc83b400640 1 -- 172.21.15.42:0/3283884438 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 89058+0+0 (secure 0 0 0) 0x7fc8300b4650 con 0x7fc8440b3a50 2024-05-28T14:06:02.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.926+0000 7fc83b400640 1 --2- 172.21.15.42:0/3283884438 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc824068f50 0x7fc82406b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:02.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.926+0000 7fc83b400640 1 -- 172.21.15.42:0/3283884438 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fc83011f220 con 0x7fc8440b3a50 2024-05-28T14:06:02.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.926+0000 7fc843400640 1 --2- 172.21.15.42:0/3283884438 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc824068f50 0x7fc82406b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:02.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.926+0000 7fc843400640 1 --2- 172.21.15.42:0/3283884438 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc824068f50 0x7fc82406b410 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc8440cb690 tx=0x7fc834009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:02.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:02.930+0000 7fc83b400640 1 -- 172.21.15.42:0/3283884438 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc830016070 con 0x7fc8440b3a50 2024-05-28T14:06:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.122+0000 7fc84a200640 1 -- 172.21.15.42:0/3283884438 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone28227", "target": ["mon-mgr", ""]}) -- 0x7fc8440c8220 con 0x7fc824068f50 2024-05-28T14:06:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.142+0000 7fc83b400640 1 -- 172.21.15.42:0/3283884438 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc8440c8220 con 0x7fc824068f50 2024-05-28T14:06:03.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.146+0000 7fc84a200640 1 -- 172.21.15.42:0/3283884438 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc824068f50 msgr2=0x7fc82406b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.146+0000 7fc84a200640 1 --2- 172.21.15.42:0/3283884438 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc824068f50 0x7fc82406b410 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc8440cb690 tx=0x7fc834009040 comp rx=0 tx=0).stop 2024-05-28T14:06:03.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.146+0000 7fc84a200640 1 -- 172.21.15.42:0/3283884438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8440b3a50 msgr2=0x7fc8441534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.146+0000 7fc84a200640 1 --2- 172.21.15.42:0/3283884438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8440b3a50 0x7fc8441534f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc8300ae040 tx=0x7fc830019190 comp rx=0 tx=0).stop 2024-05-28T14:06:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.146+0000 7fc84a200640 1 -- 172.21.15.42:0/3283884438 shutdown_connections 2024-05-28T14:06:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.146+0000 7fc84a200640 1 --2- 172.21.15.42:0/3283884438 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc824068f50 0x7fc82406b410 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.146+0000 7fc84a200640 1 --2- 172.21.15.42:0/3283884438 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc84415eb10 0x7fc8440b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.146+0000 7fc84a200640 1 --2- 172.21.15.42:0/3283884438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8440b3a50 0x7fc8441534f0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.146+0000 7fc84a200640 1 --2- 172.21.15.42:0/3283884438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc84406b9f0 0x7fc844156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.146+0000 7fc84a200640 1 -- 172.21.15.42:0/3283884438 >> 172.21.15.42:0/3283884438 conn(0x7fc8440b2f60 msgr2=0x7fc8440d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.146+0000 7fc84a200640 1 -- 172.21.15.42:0/3283884438 shutdown_connections 2024-05-28T14:06:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.146+0000 7fc84a200640 1 -- 172.21.15.42:0/3283884438 wait complete. 2024-05-28T14:06:03.155 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone28226 2024-05-28T14:06:03.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba93600640 1 Processor -- start 2024-05-28T14:06:03.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba93600640 1 -- start start 2024-05-28T14:06:03.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba93600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fba8c0d0e00 0x7fba8c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:03.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba93600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba8c0ade40 con 0x7fba8c15b350 2024-05-28T14:06:03.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba93600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba8c0ae0c0 con 0x7fba8c072d40 2024-05-28T14:06:03.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba93600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fba8c0ae9f0 con 0x7fba8c0d0e00 2024-05-28T14:06:03.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba90c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fba8c15b350 0x7fba8c0d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:45800/0 (socket says 172.21.15.42:45800) 2024-05-28T14:06:03.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba90c00640 1 -- 172.21.15.42:0/2184424608 learned_addr learned my addr 172.21.15.42:0/2184424608 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8be00640 1 --2- 172.21.15.42:0/2184424608 >> v2:172.21.15.63:3300/0 conn(0x7fba8c0d0e00 0x7fba8c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 -- 172.21.15.42:0/2184424608 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 744136222 0 0) 0x7fba80001120 con 0x7fba8c15b350 2024-05-28T14:06:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 -- 172.21.15.42:0/2184424608 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba74003530 con 0x7fba8c15b350 2024-05-28T14:06:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 -- 172.21.15.42:0/2184424608 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3715458136 0 0) 0x7fba74003530 con 0x7fba8c15b350 2024-05-28T14:06:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 -- 172.21.15.42:0/2184424608 >> v2:172.21.15.63:3300/0 conn(0x7fba8c0d0e00 msgr2=0x7fba8c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 --2- 172.21.15.42:0/2184424608 >> v2:172.21.15.63:3300/0 conn(0x7fba8c0d0e00 0x7fba8c06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 -- 172.21.15.42:0/2184424608 >> v1:172.21.15.63:6789/0 conn(0x7fba8c072d40 legacy=0x7fba8c0aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 -- 172.21.15.42:0/2184424608 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c0ae3a0 con 0x7fba8c15b350 2024-05-28T14:06:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 -- 172.21.15.42:0/2184424608 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fba80001850 con 0x7fba8c15b350 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 -- 172.21.15.42:0/2184424608 >> v1:172.21.15.42:6789/0 conn(0x7fba8c15b350 legacy=0x7fba8c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 --2- 172.21.15.42:0/2184424608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba740039c0 0x7fba74003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 --2- 172.21.15.42:0/2184424608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74004810 0x7fba74004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 --2- 172.21.15.42:0/2184424608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba740051d0 0x7fba7400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 -- 172.21.15.42:0/2184424608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c0ae9f0 con 0x7fba74004810 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 -- 172.21.15.42:0/2184424608 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba7400c120 con 0x7fba740039c0 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8b400640 1 -- 172.21.15.42:0/2184424608 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fba7400c320 con 0x7fba740051d0 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8be00640 1 --2- 172.21.15.42:0/2184424608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74004810 0x7fba74004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba90c00640 1 --2- 172.21.15.42:0/2184424608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba740039c0 0x7fba74003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba91600640 1 --2- 172.21.15.42:0/2184424608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba740051d0 0x7fba7400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8be00640 1 -- 172.21.15.42:0/2184424608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba740051d0 msgr2=0x7fba7400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8be00640 1 --2- 172.21.15.42:0/2184424608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba740051d0 0x7fba7400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8be00640 1 -- 172.21.15.42:0/2184424608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba740039c0 msgr2=0x7fba74003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8be00640 1 --2- 172.21.15.42:0/2184424608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba740039c0 0x7fba74003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba8be00640 1 -- 172.21.15.42:0/2184424608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba74003530 con 0x7fba74004810 2024-05-28T14:06:03.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.314+0000 7fba91600640 1 --2- 172.21.15.42:0/2184424608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba740051d0 0x7fba7400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:03.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.318+0000 7fba91600640 1 -- 172.21.15.42:0/2184424608 reap_dead start 2024-05-28T14:06:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.318+0000 7fba8be00640 1 --2- 172.21.15.42:0/2184424608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74004810 0x7fba74004c90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fba78009ae0 tx=0x7fba7801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dce99cb366cac16 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.318+0000 7fba8b400640 1 -- 172.21.15.42:0/2184424608 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba780096b0 con 0x7fba74004810 2024-05-28T14:06:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.318+0000 7fba8b400640 1 -- 172.21.15.42:0/2184424608 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba780a8420 con 0x7fba74004810 2024-05-28T14:06:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.318+0000 7fba8b400640 1 -- 172.21.15.42:0/2184424608 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba780a8810 con 0x7fba74004810 2024-05-28T14:06:03.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.318+0000 7fba93600640 1 -- 172.21.15.42:0/2184424608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74004810 msgr2=0x7fba74004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.318+0000 7fba93600640 1 --2- 172.21.15.42:0/2184424608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74004810 0x7fba74004c90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fba78009ae0 tx=0x7fba7801ca70 comp rx=0 tx=0).stop 2024-05-28T14:06:03.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.318+0000 7fba93600640 1 -- 172.21.15.42:0/2184424608 shutdown_connections 2024-05-28T14:06:03.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.318+0000 7fba93600640 1 --2- 172.21.15.42:0/2184424608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74004810 0x7fba74004c90 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.318+0000 7fba93600640 1 -- 172.21.15.42:0/2184424608 >> 172.21.15.42:0/2184424608 conn(0x7fba8c06bdf0 msgr2=0x7fba8c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:03.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.318+0000 7fba93600640 1 -- 172.21.15.42:0/2184424608 shutdown_connections 2024-05-28T14:06:03.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.318+0000 7fba93600640 1 -- 172.21.15.42:0/2184424608 wait complete. 2024-05-28T14:06:03.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba93600640 1 Processor -- start 2024-05-28T14:06:03.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba93600640 1 -- start start 2024-05-28T14:06:03.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba93600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba74004810 0x7fba8c155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:03.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba93600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8c1f3390 0x7fba8c1588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:03.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba93600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c158e10 0x7fba8c151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:03.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba93600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c15f0d0 con 0x7fba8c1f3390 2024-05-28T14:06:03.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba93600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c15ef50 con 0x7fba74004810 2024-05-28T14:06:03.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba93600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fba8c0af670 con 0x7fba8c158e10 2024-05-28T14:06:03.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba8be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8c1f3390 0x7fba8c1588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:03.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba8be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8c1f3390 0x7fba8c1588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54342/0 (socket says 172.21.15.42:54342) 2024-05-28T14:06:03.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba8be00640 1 -- 172.21.15.42:0/1013146692 learned_addr learned my addr 172.21.15.42:0/1013146692 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:03.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba91600640 1 --2- 172.21.15.42:0/1013146692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c158e10 0x7fba8c151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:03.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba90c00640 1 --2- 172.21.15.42:0/1013146692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba74004810 0x7fba8c155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba8be00640 1 -- 172.21.15.42:0/1013146692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c158e10 msgr2=0x7fba8c151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba8be00640 1 --2- 172.21.15.42:0/1013146692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c158e10 0x7fba8c151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba8be00640 1 -- 172.21.15.42:0/1013146692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba74004810 msgr2=0x7fba8c155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba8be00640 1 --2- 172.21.15.42:0/1013146692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba74004810 0x7fba8c155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba8be00640 1 -- 172.21.15.42:0/1013146692 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c1f37f0 con 0x7fba8c1f3390 2024-05-28T14:06:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba8be00640 1 --2- 172.21.15.42:0/1013146692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8c1f3390 0x7fba8c1588d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fba7801c8a0 tx=0x7fba78019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba88c00640 1 -- 172.21.15.42:0/1013146692 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba780096b0 con 0x7fba8c1f3390 2024-05-28T14:06:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba88c00640 1 -- 172.21.15.42:0/1013146692 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba780b9440 con 0x7fba8c1f3390 2024-05-28T14:06:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba88c00640 1 -- 172.21.15.42:0/1013146692 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba780b97f0 con 0x7fba8c1f3390 2024-05-28T14:06:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba93600640 1 -- 172.21.15.42:0/1013146692 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8c154580 con 0x7fba8c1f3390 2024-05-28T14:06:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.322+0000 7fba93600640 1 -- 172.21.15.42:0/1013146692 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba8c154b40 con 0x7fba8c1f3390 2024-05-28T14:06:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.326+0000 7fba93600640 1 -- 172.21.15.42:0/1013146692 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba8c159220 con 0x7fba8c1f3390 2024-05-28T14:06:03.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.326+0000 7fba88c00640 1 -- 172.21.15.42:0/1013146692 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 89058+0+0 (secure 0 0 0) 0x7fba780b0400 con 0x7fba8c1f3390 2024-05-28T14:06:03.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.326+0000 7fba88c00640 1 --2- 172.21.15.42:0/1013146692 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba6c068f80 0x7fba6c06b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:03.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.326+0000 7fba88c00640 1 -- 172.21.15.42:0/1013146692 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fba7811f320 con 0x7fba8c1f3390 2024-05-28T14:06:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.326+0000 7fba90c00640 1 --2- 172.21.15.42:0/1013146692 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba6c068f80 0x7fba6c06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.326+0000 7fba90c00640 1 --2- 172.21.15.42:0/1013146692 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba6c068f80 0x7fba6c06b440 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fba800010b0 tx=0x7fba80007ab0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:03.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.330+0000 7fba88c00640 1 -- 172.21.15.42:0/1013146692 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fba780ac030 con 0x7fba8c1f3390 2024-05-28T14:06:03.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.522+0000 7fba93600640 1 -- 172.21.15.42:0/1013146692 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone28226", "target": ["mon-mgr", ""]}) -- 0x7fba8c158700 con 0x7fba6c068f80 2024-05-28T14:06:03.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.546+0000 7fba88c00640 1 -- 172.21.15.42:0/1013146692 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fba8c158700 con 0x7fba6c068f80 2024-05-28T14:06:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.546+0000 7fba93600640 1 -- 172.21.15.42:0/1013146692 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba6c068f80 msgr2=0x7fba6c06b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.546+0000 7fba93600640 1 --2- 172.21.15.42:0/1013146692 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba6c068f80 0x7fba6c06b440 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fba800010b0 tx=0x7fba80007ab0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.550+0000 7fba93600640 1 -- 172.21.15.42:0/1013146692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8c1f3390 msgr2=0x7fba8c1588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.550+0000 7fba93600640 1 --2- 172.21.15.42:0/1013146692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8c1f3390 0x7fba8c1588d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fba7801c8a0 tx=0x7fba78019810 comp rx=0 tx=0).stop 2024-05-28T14:06:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.550+0000 7fba93600640 1 -- 172.21.15.42:0/1013146692 shutdown_connections 2024-05-28T14:06:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.550+0000 7fba93600640 1 --2- 172.21.15.42:0/1013146692 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba6c068f80 0x7fba6c06b440 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.550+0000 7fba93600640 1 --2- 172.21.15.42:0/1013146692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba8c158e10 0x7fba8c151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.550+0000 7fba93600640 1 --2- 172.21.15.42:0/1013146692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba8c1f3390 0x7fba8c1588d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.550+0000 7fba93600640 1 --2- 172.21.15.42:0/1013146692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba74004810 0x7fba8c155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.550+0000 7fba93600640 1 -- 172.21.15.42:0/1013146692 >> 172.21.15.42:0/1013146692 conn(0x7fba8c06bdf0 msgr2=0x7fba8c0cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.550+0000 7fba93600640 1 -- 172.21.15.42:0/1013146692 shutdown_connections 2024-05-28T14:06:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.550+0000 7fba93600640 1 -- 172.21.15.42:0/1013146692 wait complete. 2024-05-28T14:06:03.559 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone28223 2024-05-28T14:06:03.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe0d400640 1 Processor -- start 2024-05-28T14:06:03.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe0d400640 1 -- start start 2024-05-28T14:06:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe0d400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe0806dc20 0x7efe080d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe0d400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efe0806cd10 0x7efe080cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe0d400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe080d1970 con 0x7efe0815e840 2024-05-28T14:06:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe0d400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efe080b5300 con 0x7efe0806cd10 2024-05-28T14:06:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe0d400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efe080d1740 con 0x7efe0806dc20 2024-05-28T14:06:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe05e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe0806dc20 0x7efe080d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe06800640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7efe0815e840 0x7efe0806d510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:43414/0 (socket says 172.21.15.42:43414) 2024-05-28T14:06:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe06800640 1 -- 172.21.15.42:0/1037111479 learned_addr learned my addr 172.21.15.42:0/1037111479 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe07200640 1 --2- 172.21.15.42:0/1037111479 >> v2:172.21.15.63:3300/0 conn(0x7efe0806cd10 0x7efe080cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:03.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe07200640 1 -- 172.21.15.42:0/1037111479 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe0806dc20 msgr2=0x7efe080d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:03.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe07200640 1 --2- 172.21.15.42:0/1037111479 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe0806dc20 0x7efe080d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe07200640 1 -- 172.21.15.42:0/1037111479 >> v1:172.21.15.63:6789/0 conn(0x7efe0815e840 legacy=0x7efe0806d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe05400640 1 -- 172.21.15.42:0/1037111479 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 410678187 0 0) 0x7efe080d1970 con 0x7efe0815e840 2024-05-28T14:06:03.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe07200640 1 -- 172.21.15.42:0/1037111479 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe080ccca0 con 0x7efe0806cd10 2024-05-28T14:06:03.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.726+0000 7efe05e00640 1 --2- 172.21.15.42:0/1037111479 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe0806dc20 0x7efe080d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:06:03.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.730+0000 7efe07200640 1 --2- 172.21.15.42:0/1037111479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe0806cd10 0x7efe080cd780 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efe00009d90 tx=0x7efe0001f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=664aed90e93b9016 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:03.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.730+0000 7efe05400640 1 -- 172.21.15.42:0/1037111479 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe000b4070 con 0x7efe0806cd10 2024-05-28T14:06:03.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.730+0000 7efe05400640 1 -- 172.21.15.42:0/1037111479 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe000af600 con 0x7efe0806cd10 2024-05-28T14:06:03.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.730+0000 7efe05400640 1 -- 172.21.15.42:0/1037111479 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe000af9d0 con 0x7efe0806cd10 2024-05-28T14:06:03.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.730+0000 7efe0d400640 1 -- 172.21.15.42:0/1037111479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe0806cd10 msgr2=0x7efe080cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.730+0000 7efe0d400640 1 --2- 172.21.15.42:0/1037111479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe0806cd10 0x7efe080cd780 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efe00009d90 tx=0x7efe0001f150 comp rx=0 tx=0).stop 2024-05-28T14:06:03.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.730+0000 7efe0d400640 1 -- 172.21.15.42:0/1037111479 shutdown_connections 2024-05-28T14:06:03.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.730+0000 7efe0d400640 1 --2- 172.21.15.42:0/1037111479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe0806cd10 0x7efe080cd780 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.730+0000 7efe0d400640 1 --2- 172.21.15.42:0/1037111479 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe0806dc20 0x7efe080d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.730+0000 7efe0d400640 1 -- 172.21.15.42:0/1037111479 >> 172.21.15.42:0/1037111479 conn(0x7efe080b3fa0 msgr2=0x7efe080b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:03.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.730+0000 7efe0d400640 1 -- 172.21.15.42:0/1037111479 shutdown_connections 2024-05-28T14:06:03.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.730+0000 7efe0d400640 1 -- 172.21.15.42:0/1037111479 wait complete. 2024-05-28T14:06:03.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe0d400640 1 Processor -- start 2024-05-28T14:06:03.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe0d400640 1 -- start start 2024-05-28T14:06:03.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe0d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe0806cd10 0x7efe08148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:03.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe0d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe0806dc20 0x7efe080c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:03.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe0d400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe0815e840 0x7efe0806fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:03.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe0d400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efe0815ec20 con 0x7efe0806dc20 2024-05-28T14:06:03.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe0d400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe080cd260 con 0x7efe0806cd10 2024-05-28T14:06:03.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe0d400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efe0815eda0 con 0x7efe0815e840 2024-05-28T14:06:03.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe05e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe0806dc20 0x7efe080c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:03.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe05e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe0806dc20 0x7efe080c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54346/0 (socket says 172.21.15.42:54346) 2024-05-28T14:06:03.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe05e00640 1 -- 172.21.15.42:0/3047479555 learned_addr learned my addr 172.21.15.42:0/3047479555 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:03.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe06800640 1 --2- 172.21.15.42:0/3047479555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe0806cd10 0x7efe08148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:03.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe07200640 1 --2- 172.21.15.42:0/3047479555 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe0815e840 0x7efe0806fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:03.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe05e00640 1 -- 172.21.15.42:0/3047479555 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe0815e840 msgr2=0x7efe0806fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe05e00640 1 --2- 172.21.15.42:0/3047479555 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe0815e840 0x7efe0806fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe05e00640 1 -- 172.21.15.42:0/3047479555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe0806cd10 msgr2=0x7efe08148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe05e00640 1 --2- 172.21.15.42:0/3047479555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe0806cd10 0x7efe08148580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe05e00640 1 -- 172.21.15.42:0/3047479555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe080c9db0 con 0x7efe0806dc20 2024-05-28T14:06:03.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe05e00640 1 --2- 172.21.15.42:0/3047479555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe0806dc20 0x7efe080c9580 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7efdf400ebe0 tx=0x7efdf400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:03.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efdfea00640 1 -- 172.21.15.42:0/3047479555 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4017070 con 0x7efe0806dc20 2024-05-28T14:06:03.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efdfea00640 1 -- 172.21.15.42:0/3047479555 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf400cc30 con 0x7efe0806dc20 2024-05-28T14:06:03.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efdfea00640 1 -- 172.21.15.42:0/3047479555 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4013470 con 0x7efe0806dc20 2024-05-28T14:06:03.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe0d400640 1 -- 172.21.15.42:0/3047479555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe080ca090 con 0x7efe0806dc20 2024-05-28T14:06:03.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe0d400640 1 -- 172.21.15.42:0/3047479555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe080cb230 con 0x7efe0806dc20 2024-05-28T14:06:03.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.734+0000 7efe0d400640 1 -- 172.21.15.42:0/3047479555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe08068a90 con 0x7efe0806dc20 2024-05-28T14:06:03.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.738+0000 7efdfea00640 1 -- 172.21.15.42:0/3047479555 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 89058+0+0 (secure 0 0 0) 0x7efdf401a070 con 0x7efe0806dc20 2024-05-28T14:06:03.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.738+0000 7efdfea00640 1 --2- 172.21.15.42:0/3047479555 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efde8068f50 0x7efde806b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:03.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.738+0000 7efdfea00640 1 -- 172.21.15.42:0/3047479555 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5793+0+0 (secure 0 0 0) 0x7efdf40854d0 con 0x7efe0806dc20 2024-05-28T14:06:03.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.738+0000 7efe06800640 1 --2- 172.21.15.42:0/3047479555 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efde8068f50 0x7efde806b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:03.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.738+0000 7efe06800640 1 --2- 172.21.15.42:0/3047479555 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efde8068f50 0x7efde806b410 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efdf8000f70 tx=0x7efdf8007df0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:03.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.742+0000 7efdfea00640 1 -- 172.21.15.42:0/3047479555 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efdf4056c30 con 0x7efe0806dc20 2024-05-28T14:06:03.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.926+0000 7efdfea00640 1 -- 172.21.15.42:0/3047479555 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7efdf40532a0 con 0x7efe0806dc20 2024-05-28T14:06:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.938+0000 7efe0d400640 1 -- 172.21.15.42:0/3047479555 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone28223", "target": ["mon-mgr", ""]}) -- 0x7efe080cdd70 con 0x7efde8068f50 2024-05-28T14:06:03.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.958+0000 7efdfea00640 1 -- 172.21.15.42:0/3047479555 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efe080cdd70 con 0x7efde8068f50 2024-05-28T14:06:03.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.958+0000 7efe0d400640 1 -- 172.21.15.42:0/3047479555 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efde8068f50 msgr2=0x7efde806b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.958+0000 7efe0d400640 1 --2- 172.21.15.42:0/3047479555 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efde8068f50 0x7efde806b410 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efdf8000f70 tx=0x7efdf8007df0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.958+0000 7efe0d400640 1 -- 172.21.15.42:0/3047479555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe0806dc20 msgr2=0x7efe080c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:03.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.958+0000 7efe0d400640 1 --2- 172.21.15.42:0/3047479555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe0806dc20 0x7efe080c9580 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7efdf400ebe0 tx=0x7efdf400c520 comp rx=0 tx=0).stop 2024-05-28T14:06:03.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.958+0000 7efe0d400640 1 -- 172.21.15.42:0/3047479555 shutdown_connections 2024-05-28T14:06:03.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.958+0000 7efe0d400640 1 --2- 172.21.15.42:0/3047479555 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efde8068f50 0x7efde806b410 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.958+0000 7efe0d400640 1 --2- 172.21.15.42:0/3047479555 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe0815e840 0x7efe0806fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.958+0000 7efe0d400640 1 --2- 172.21.15.42:0/3047479555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe0806dc20 0x7efe080c9580 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.958+0000 7efe0d400640 1 --2- 172.21.15.42:0/3047479555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe0806cd10 0x7efe08148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:03.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.958+0000 7efe0d400640 1 -- 172.21.15.42:0/3047479555 >> 172.21.15.42:0/3047479555 conn(0x7efe080b3fa0 msgr2=0x7efe080b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:03.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.958+0000 7efe0d400640 1 -- 172.21.15.42:0/3047479555 shutdown_connections 2024-05-28T14:06:03.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:03.958+0000 7efe0d400640 1 -- 172.21.15.42:0/3047479555 wait complete. 2024-05-28T14:06:03.968 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone28225 2024-05-28T14:06:04.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.126+0000 7f90ba200640 1 Processor -- start 2024-05-28T14:06:04.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.126+0000 7f90ba200640 1 -- start start 2024-05-28T14:06:04.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.126+0000 7f90ba200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b415ed00 0x7f90b415f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:04.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.126+0000 7f90ba200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f90b40d06f0 0x7f90b40d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:04.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.126+0000 7f90ba200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b415f7d0 con 0x7f90b406e5f0 2024-05-28T14:06:04.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.126+0000 7f90ba200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f90b415fab0 con 0x7f90b40d06f0 2024-05-28T14:06:04.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.126+0000 7f90ba200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90b41624c0 con 0x7f90b415ed00 2024-05-28T14:06:04.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.126+0000 7f90abe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b415ed00 0x7f90b415f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:04.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.126+0000 7f90b3e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f90b40d06f0 0x7f90b40d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:04.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.126+0000 7f90b3400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f90b406e5f0 0x7f90b4162cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:43416/0 (socket says 172.21.15.42:43416) 2024-05-28T14:06:04.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.126+0000 7f90b3400640 1 -- 172.21.15.42:0/428848206 learned_addr learned my addr 172.21.15.42:0/428848206 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:04.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.130+0000 7f90b3e00640 1 -- 172.21.15.42:0/428848206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b415ed00 msgr2=0x7f90b415f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:04.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.130+0000 7f90b3e00640 1 --2- 172.21.15.42:0/428848206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b415ed00 0x7f90b415f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.130+0000 7f90b3e00640 1 -- 172.21.15.42:0/428848206 >> v1:172.21.15.63:6789/0 conn(0x7f90b406e5f0 legacy=0x7f90b4162cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:04.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.130+0000 7f90b3e00640 1 -- 172.21.15.42:0/428848206 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b41627a0 con 0x7f90b40d06f0 2024-05-28T14:06:04.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.130+0000 7f90b3e00640 1 --2- 172.21.15.42:0/428848206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b40d06f0 0x7f90b40d0b80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009d90 tx=0x7f90ac01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe3fbbfb9ba221c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:04.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.130+0000 7f90b2a00640 1 -- 172.21.15.42:0/428848206 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac0b1070 con 0x7f90b40d06f0 2024-05-28T14:06:04.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.130+0000 7f90b2a00640 1 -- 172.21.15.42:0/428848206 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90ac0a49b0 con 0x7f90b40d06f0 2024-05-28T14:06:04.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.130+0000 7f90b2a00640 1 -- 172.21.15.42:0/428848206 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac0a4d80 con 0x7f90b40d06f0 2024-05-28T14:06:04.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 -- 172.21.15.42:0/428848206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b40d06f0 msgr2=0x7f90b40d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:04.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 --2- 172.21.15.42:0/428848206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b40d06f0 0x7f90b40d0b80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009d90 tx=0x7f90ac01c450 comp rx=0 tx=0).stop 2024-05-28T14:06:04.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 -- 172.21.15.42:0/428848206 shutdown_connections 2024-05-28T14:06:04.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 --2- 172.21.15.42:0/428848206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b40d06f0 0x7f90b40d0b80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 --2- 172.21.15.42:0/428848206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b415ed00 0x7f90b415f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 -- 172.21.15.42:0/428848206 >> 172.21.15.42:0/428848206 conn(0x7f90b406bdf0 msgr2=0x7f90b406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:04.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 -- 172.21.15.42:0/428848206 shutdown_connections 2024-05-28T14:06:04.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 -- 172.21.15.42:0/428848206 wait complete. 2024-05-28T14:06:04.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 Processor -- start 2024-05-28T14:06:04.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 -- start start 2024-05-28T14:06:04.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b406e5f0 0x7f90b41502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b40d06f0 0x7f90b414fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b415ed00 0x7f90b415b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f90b416c1f0 con 0x7f90b406e5f0 2024-05-28T14:06:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90b416c070 con 0x7f90b40d06f0 2024-05-28T14:06:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90ba200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90b416c370 con 0x7f90b415ed00 2024-05-28T14:06:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90b3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b406e5f0 0x7f90b41502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90b3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b406e5f0 0x7f90b41502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54348/0 (socket says 172.21.15.42:54348) 2024-05-28T14:06:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90b3400640 1 -- 172.21.15.42:0/2767929198 learned_addr learned my addr 172.21.15.42:0/2767929198 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90abe00640 1 --2- 172.21.15.42:0/2767929198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b40d06f0 0x7f90b414fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90b3e00640 1 --2- 172.21.15.42:0/2767929198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b415ed00 0x7f90b415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90b3400640 1 -- 172.21.15.42:0/2767929198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b415ed00 msgr2=0x7f90b415b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90b3400640 1 --2- 172.21.15.42:0/2767929198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b415ed00 0x7f90b415b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90b3400640 1 -- 172.21.15.42:0/2767929198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b40d06f0 msgr2=0x7f90b414fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:04.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90b3400640 1 --2- 172.21.15.42:0/2767929198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b40d06f0 0x7f90b414fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.134+0000 7f90b3400640 1 -- 172.21.15.42:0/2767929198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b4150880 con 0x7f90b406e5f0 2024-05-28T14:06:04.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.138+0000 7f90b3400640 1 --2- 172.21.15.42:0/2767929198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b406e5f0 0x7f90b41502a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f90a400c780 tx=0x7f90a400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.138+0000 7f90ab400640 1 -- 172.21.15.42:0/2767929198 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4018070 con 0x7f90b406e5f0 2024-05-28T14:06:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.138+0000 7f90ab400640 1 -- 172.21.15.42:0/2767929198 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a40072d0 con 0x7f90b406e5f0 2024-05-28T14:06:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.138+0000 7f90ab400640 1 -- 172.21.15.42:0/2767929198 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a40047f0 con 0x7f90b406e5f0 2024-05-28T14:06:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.138+0000 7f90ba200640 1 -- 172.21.15.42:0/2767929198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b4150b60 con 0x7f90b406e5f0 2024-05-28T14:06:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.138+0000 7f90ba200640 1 -- 172.21.15.42:0/2767929198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b4151120 con 0x7f90b406e5f0 2024-05-28T14:06:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.138+0000 7f90ba200640 1 -- 172.21.15.42:0/2767929198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b4068a90 con 0x7f90b406e5f0 2024-05-28T14:06:04.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.138+0000 7f90ab400640 1 -- 172.21.15.42:0/2767929198 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f90a4004990 con 0x7f90b406e5f0 2024-05-28T14:06:04.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.138+0000 7f90ab400640 1 --2- 172.21.15.42:0/2767929198 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9090069070 0x7f909006b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:04.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.138+0000 7f90ab400640 1 -- 172.21.15.42:0/2767929198 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5793+0+0 (secure 0 0 0) 0x7f90a4084830 con 0x7f90b406e5f0 2024-05-28T14:06:04.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.138+0000 7f90abe00640 1 --2- 172.21.15.42:0/2767929198 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9090069070 0x7f909006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:04.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.142+0000 7f90abe00640 1 --2- 172.21.15.42:0/2767929198 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9090069070 0x7f909006b530 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f90b4169610 tx=0x7f909c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:04.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.146+0000 7f90ab400640 1 -- 172.21.15.42:0/2767929198 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f90a4056270 con 0x7f90b406e5f0 2024-05-28T14:06:04.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.334+0000 7f90ba200640 1 -- 172.21.15.42:0/2767929198 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone28225", "target": ["mon-mgr", ""]}) -- 0x7f90b40b5430 con 0x7f9090069070 2024-05-28T14:06:04.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.350+0000 7f90ab400640 1 -- 172.21.15.42:0/2767929198 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f90b40b5430 con 0x7f9090069070 2024-05-28T14:06:04.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.354+0000 7f90ba200640 1 -- 172.21.15.42:0/2767929198 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9090069070 msgr2=0x7f909006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.354+0000 7f90ba200640 1 --2- 172.21.15.42:0/2767929198 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9090069070 0x7f909006b530 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f90b4169610 tx=0x7f909c009190 comp rx=0 tx=0).stop 2024-05-28T14:06:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.354+0000 7f90ba200640 1 -- 172.21.15.42:0/2767929198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b406e5f0 msgr2=0x7f90b41502a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.354+0000 7f90ba200640 1 --2- 172.21.15.42:0/2767929198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b406e5f0 0x7f90b41502a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f90a400c780 tx=0x7f90a400cc40 comp rx=0 tx=0).stop 2024-05-28T14:06:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.354+0000 7f90ba200640 1 -- 172.21.15.42:0/2767929198 shutdown_connections 2024-05-28T14:06:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.354+0000 7f90ba200640 1 --2- 172.21.15.42:0/2767929198 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9090069070 0x7f909006b530 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.354+0000 7f90ba200640 1 --2- 172.21.15.42:0/2767929198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b415ed00 0x7f90b415b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.354+0000 7f90ba200640 1 --2- 172.21.15.42:0/2767929198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b40d06f0 0x7f90b414fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.354+0000 7f90ba200640 1 --2- 172.21.15.42:0/2767929198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b406e5f0 0x7f90b41502a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.354+0000 7f90ba200640 1 -- 172.21.15.42:0/2767929198 >> 172.21.15.42:0/2767929198 conn(0x7f90b406bdf0 msgr2=0x7f90b40749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.354+0000 7f90ba200640 1 -- 172.21.15.42:0/2767929198 shutdown_connections 2024-05-28T14:06:04.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.354+0000 7f90ba200640 1 -- 172.21.15.42:0/2767929198 wait complete. 2024-05-28T14:06:04.362 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone28224 2024-05-28T14:06:04.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.518+0000 7ffa98600640 1 Processor -- start 2024-05-28T14:06:04.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.518+0000 7ffa98600640 1 -- start start 2024-05-28T14:06:04.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.522+0000 7ffa98600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa9015b350 0x7ffa9006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:04.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.522+0000 7ffa98600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa900ade40 con 0x7ffa90072d40 2024-05-28T14:06:04.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.522+0000 7ffa98600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa900ae0c0 con 0x7ffa900d0e00 2024-05-28T14:06:04.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.522+0000 7ffa98600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa900ae9f0 con 0x7ffa9015b350 2024-05-28T14:06:04.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.522+0000 7ffa95200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ffa90072d40 0x7ffa900d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:45806/0 (socket says 172.21.15.42:45806) 2024-05-28T14:06:04.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.522+0000 7ffa95200640 1 -- 172.21.15.42:0/4212657704 learned_addr learned my addr 172.21.15.42:0/4212657704 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:04.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.522+0000 7ffa95c00640 1 --2- 172.21.15.42:0/4212657704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa9015b350 0x7ffa9006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:04.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.522+0000 7ffa8fe00640 1 -- 172.21.15.42:0/4212657704 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1782715191 0 0) 0x7ffa7c001120 con 0x7ffa90072d40 2024-05-28T14:06:04.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.522+0000 7ffa8fe00640 1 -- 172.21.15.42:0/4212657704 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa78003530 con 0x7ffa90072d40 2024-05-28T14:06:04.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.522+0000 7ffa8fe00640 1 -- 172.21.15.42:0/4212657704 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3633459833 0 0) 0x7ffa900ae0c0 con 0x7ffa900d0e00 2024-05-28T14:06:04.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.522+0000 7ffa8fe00640 1 -- 172.21.15.42:0/4212657704 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa7c001120 con 0x7ffa900d0e00 2024-05-28T14:06:04.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.522+0000 7ffa95c00640 1 -- 172.21.15.42:0/4212657704 >> v1:172.21.15.63:6789/0 conn(0x7ffa900d0e00 legacy=0x7ffa900aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:04.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.522+0000 7ffa95c00640 1 -- 172.21.15.42:0/4212657704 >> v1:172.21.15.42:6789/0 conn(0x7ffa90072d40 legacy=0x7ffa900d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:04.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.522+0000 7ffa95c00640 1 -- 172.21.15.42:0/4212657704 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa900ae3a0 con 0x7ffa9015b350 2024-05-28T14:06:04.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.526+0000 7ffa95c00640 1 --2- 172.21.15.42:0/4212657704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa9015b350 0x7ffa9006dd40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa900aeb70 tx=0x7ffa8401c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1a9d46a22cc57a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:04.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.526+0000 7ffa8fe00640 1 -- 172.21.15.42:0/4212657704 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa840b0070 con 0x7ffa9015b350 2024-05-28T14:06:04.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.526+0000 7ffa8fe00640 1 -- 172.21.15.42:0/4212657704 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa840ab600 con 0x7ffa9015b350 2024-05-28T14:06:04.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.526+0000 7ffa8fe00640 1 -- 172.21.15.42:0/4212657704 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa840ab9d0 con 0x7ffa9015b350 2024-05-28T14:06:04.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.526+0000 7ffa98600640 1 -- 172.21.15.42:0/4212657704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa9015b350 msgr2=0x7ffa9006dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:04.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.526+0000 7ffa98600640 1 --2- 172.21.15.42:0/4212657704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa9015b350 0x7ffa9006dd40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa900aeb70 tx=0x7ffa8401c430 comp rx=0 tx=0).stop 2024-05-28T14:06:04.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.526+0000 7ffa98600640 1 -- 172.21.15.42:0/4212657704 shutdown_connections 2024-05-28T14:06:04.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.526+0000 7ffa98600640 1 --2- 172.21.15.42:0/4212657704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa9015b350 0x7ffa9006dd40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.526+0000 7ffa98600640 1 -- 172.21.15.42:0/4212657704 >> 172.21.15.42:0/4212657704 conn(0x7ffa9006bdf0 msgr2=0x7ffa9006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:04.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.526+0000 7ffa98600640 1 -- 172.21.15.42:0/4212657704 shutdown_connections 2024-05-28T14:06:04.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.526+0000 7ffa98600640 1 -- 172.21.15.42:0/4212657704 wait complete. 2024-05-28T14:06:04.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.526+0000 7ffa98600640 1 Processor -- start 2024-05-28T14:06:04.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.526+0000 7ffa98600640 1 -- start start 2024-05-28T14:06:04.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa98600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa90072d40 0x7ffa900aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:04.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa98600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa900d0e00 0x7ffa9015c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:04.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa98600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa9015b350 0x7ffa9014de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:04.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa98600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffa900afb20 con 0x7ffa90072d40 2024-05-28T14:06:04.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa98600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa900af9a0 con 0x7ffa9015b350 2024-05-28T14:06:04.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa98600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa900b3af0 con 0x7ffa900d0e00 2024-05-28T14:06:04.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa95c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa90072d40 0x7ffa900aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:04.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa95c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa90072d40 0x7ffa900aff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54354/0 (socket says 172.21.15.42:54354) 2024-05-28T14:06:04.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa95c00640 1 -- 172.21.15.42:0/779843442 learned_addr learned my addr 172.21.15.42:0/779843442 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:04.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa95200640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa900d0e00 0x7ffa9015c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:04.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa96600640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa9015b350 0x7ffa9014de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:04.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa95c00640 1 -- 172.21.15.42:0/779843442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa900d0e00 msgr2=0x7ffa9015c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:04.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa95c00640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa900d0e00 0x7ffa9015c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa95c00640 1 -- 172.21.15.42:0/779843442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa9015b350 msgr2=0x7ffa9014de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:04.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa95c00640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa9015b350 0x7ffa9014de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa95c00640 1 -- 172.21.15.42:0/779843442 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9009e000 con 0x7ffa90072d40 2024-05-28T14:06:04.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa95200640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa900d0e00 0x7ffa9015c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:04.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa96600640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa9015b350 0x7ffa9014de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:04.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa95c00640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa90072d40 0x7ffa900aff90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ffa8401a060 tx=0x7ffa840191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:04.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa8d600640 1 -- 172.21.15.42:0/779843442 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa840bb070 con 0x7ffa90072d40 2024-05-28T14:06:04.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa8d600640 1 -- 172.21.15.42:0/779843442 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa840b7460 con 0x7ffa90072d40 2024-05-28T14:06:04.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa8d600640 1 -- 172.21.15.42:0/779843442 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa840b0040 con 0x7ffa90072d40 2024-05-28T14:06:04.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa98600640 1 -- 172.21.15.42:0/779843442 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa9015b9d0 con 0x7ffa90072d40 2024-05-28T14:06:04.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa98600640 1 -- 172.21.15.42:0/779843442 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa9015bc70 con 0x7ffa90072d40 2024-05-28T14:06:04.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa98600640 1 -- 172.21.15.42:0/779843442 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa900d0c10 con 0x7ffa90072d40 2024-05-28T14:06:04.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.530+0000 7ffa8d600640 1 -- 172.21.15.42:0/779843442 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7ffa840be070 con 0x7ffa90072d40 2024-05-28T14:06:04.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.534+0000 7ffa8d600640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa700690a0 0x7ffa7006b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:04.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.534+0000 7ffa8d600640 1 -- 172.21.15.42:0/779843442 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5793+0+0 (secure 0 0 0) 0x7ffa8411f3d0 con 0x7ffa90072d40 2024-05-28T14:06:04.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.534+0000 7ffa95200640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa700690a0 0x7ffa7006b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:04.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.534+0000 7ffa95200640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa700690a0 0x7ffa7006b560 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ffa901f6df0 tx=0x7ffa7c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:04.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.538+0000 7ffa8d600640 1 -- 172.21.15.42:0/779843442 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffa840c5070 con 0x7ffa90072d40 2024-05-28T14:06:04.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.782+0000 7ffa98600640 1 -- 172.21.15.42:0/779843442 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone28224", "target": ["mon-mgr", ""]}) -- 0x7ffa90068a90 con 0x7ffa700690a0 2024-05-28T14:06:04.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.818+0000 7ffa8d600640 1 -- 172.21.15.42:0/779843442 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffa90068a90 con 0x7ffa700690a0 2024-05-28T14:06:04.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.822+0000 7ffa98600640 1 -- 172.21.15.42:0/779843442 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa700690a0 msgr2=0x7ffa7006b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:04.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.822+0000 7ffa98600640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa700690a0 0x7ffa7006b560 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ffa901f6df0 tx=0x7ffa7c009040 comp rx=0 tx=0).stop 2024-05-28T14:06:04.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.822+0000 7ffa98600640 1 -- 172.21.15.42:0/779843442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa90072d40 msgr2=0x7ffa900aff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:04.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.822+0000 7ffa98600640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa90072d40 0x7ffa900aff90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ffa8401a060 tx=0x7ffa840191d0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.822+0000 7ffa98600640 1 -- 172.21.15.42:0/779843442 shutdown_connections 2024-05-28T14:06:04.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.822+0000 7ffa98600640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa700690a0 0x7ffa7006b560 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.822+0000 7ffa98600640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa9015b350 0x7ffa9014de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.822+0000 7ffa98600640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa900d0e00 0x7ffa9015c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.822+0000 7ffa98600640 1 --2- 172.21.15.42:0/779843442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa90072d40 0x7ffa900aff90 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:04.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.822+0000 7ffa98600640 1 -- 172.21.15.42:0/779843442 >> 172.21.15.42:0/779843442 conn(0x7ffa9006bdf0 msgr2=0x7ffa900b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:04.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.822+0000 7ffa98600640 1 -- 172.21.15.42:0/779843442 shutdown_connections 2024-05-28T14:06:04.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04.822+0000 7ffa98600640 1 -- 172.21.15.42:0/779843442 wait complete. 2024-05-28T14:06:04.855 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-28T14:06:04.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-28T14:06:04.957 INFO:teuthology.orchestra.run.smithi042.stdout:2 2024-05-28T14:06:04.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:06:04.972 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T14:06:04.972 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:06:05.103 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:06:05.213 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T14:06:05.213 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:06:05.213 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:06:05.213 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:06:05.228 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:06:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.526+0000 7f7e27400640 1 Processor -- start 2024-05-28T14:06:05.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.526+0000 7f7e27400640 1 -- start start 2024-05-28T14:06:05.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.530+0000 7f7e27400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7e2015c920 0x7f7e200cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.530+0000 7f7e27400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7e2015d780 0x7f7e201602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.530+0000 7f7e27400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e2006be20 con 0x7f7e20160830 2024-05-28T14:06:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.530+0000 7f7e27400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f7e20160c60 con 0x7f7e2015d780 2024-05-28T14:06:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.530+0000 7f7e27400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7e2006c0c0 con 0x7f7e2015c920 2024-05-28T14:06:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.530+0000 7f7e1fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7e2015d780 0x7f7e201602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.530+0000 7f7e1fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7e2015d780 0x7f7e201602f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54364/0 (socket says 172.21.15.42:54364) 2024-05-28T14:06:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.530+0000 7f7e25400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f7e20160830 0x7f7e200b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:45812/0 (socket says 172.21.15.42:45812) 2024-05-28T14:06:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.530+0000 7f7e1fe00640 1 -- 172.21.15.42:0/3580562428 learned_addr learned my addr 172.21.15.42:0/3580562428 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.530+0000 7f7e24a00640 1 --2- 172.21.15.42:0/3580562428 >> v2:172.21.15.63:3300/0 conn(0x7f7e2015c920 0x7f7e200cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:05.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.530+0000 7f7e24a00640 1 -- 172.21.15.42:0/3580562428 >> v2:172.21.15.42:3300/0 conn(0x7f7e2015d780 msgr2=0x7f7e201602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:05.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.530+0000 7f7e24a00640 1 --2- 172.21.15.42:0/3580562428 >> v2:172.21.15.42:3300/0 conn(0x7f7e2015d780 0x7f7e201602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:05.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.530+0000 7f7e24a00640 1 -- 172.21.15.42:0/3580562428 >> v1:172.21.15.42:6789/0 conn(0x7f7e20160830 legacy=0x7f7e200b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:05.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.530+0000 7f7e24a00640 1 -- 172.21.15.42:0/3580562428 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2006c3a0 con 0x7f7e2015c920 2024-05-28T14:06:05.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.534+0000 7f7e24a00640 1 --2- 172.21.15.42:0/3580562428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e2015c920 0x7f7e200cdc10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7e200d2480 tx=0x7f7e1001c320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=874132257583826 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:05.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.538+0000 7f7e1f400640 1 -- 172.21.15.42:0/3580562428 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e100b0070 con 0x7f7e2015c920 2024-05-28T14:06:05.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.538+0000 7f7e1f400640 1 -- 172.21.15.42:0/3580562428 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e100ab400 con 0x7f7e2015c920 2024-05-28T14:06:05.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.538+0000 7f7e1f400640 1 -- 172.21.15.42:0/3580562428 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e100ab7d0 con 0x7f7e2015c920 2024-05-28T14:06:05.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.538+0000 7f7e27400640 1 -- 172.21.15.42:0/3580562428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e2015c920 msgr2=0x7f7e200cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:05.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.538+0000 7f7e27400640 1 --2- 172.21.15.42:0/3580562428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e2015c920 0x7f7e200cdc10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7e200d2480 tx=0x7f7e1001c320 comp rx=0 tx=0).stop 2024-05-28T14:06:05.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.538+0000 7f7e27400640 1 -- 172.21.15.42:0/3580562428 shutdown_connections 2024-05-28T14:06:05.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.538+0000 7f7e27400640 1 --2- 172.21.15.42:0/3580562428 >> v2:172.21.15.42:3300/0 conn(0x7f7e2015d780 0x7f7e201602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:05.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.538+0000 7f7e27400640 1 --2- 172.21.15.42:0/3580562428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e2015c920 0x7f7e200cdc10 secure :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7e200d2480 tx=0x7f7e1001c320 comp rx=0 tx=0).stop 2024-05-28T14:06:05.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.538+0000 7f7e27400640 1 -- 172.21.15.42:0/3580562428 >> 172.21.15.42:0/3580562428 conn(0x7f7e200b68f0 msgr2=0x7f7e200b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:05.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.538+0000 7f7e27400640 1 -- 172.21.15.42:0/3580562428 shutdown_connections 2024-05-28T14:06:05.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.538+0000 7f7e27400640 1 -- 172.21.15.42:0/3580562428 wait complete. 2024-05-28T14:06:05.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.538+0000 7f7e27400640 1 Processor -- start 2024-05-28T14:06:05.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.542+0000 7f7e27400640 1 -- start start 2024-05-28T14:06:05.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.542+0000 7f7e27400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2015d780 0x7f7e2015af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:05.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.542+0000 7f7e27400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e20160830 0x7f7e2013fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:05.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.542+0000 7f7e27400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e200cfa60 0x7f7e20158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:05.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.542+0000 7f7e27400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2006bb70 con 0x7f7e20160830 2024-05-28T14:06:05.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.542+0000 7f7e27400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2006b9f0 con 0x7f7e200cfa60 2024-05-28T14:06:05.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.542+0000 7f7e27400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2016ba90 con 0x7f7e2015d780 2024-05-28T14:06:05.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.542+0000 7f7e25400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e200cfa60 0x7f7e20158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:05.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.542+0000 7f7e25400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e200cfa60 0x7f7e20158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:49056/0 (socket says 172.21.15.42:49056) 2024-05-28T14:06:05.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.542+0000 7f7e25400640 1 -- 172.21.15.42:0/334707886 learned_addr learned my addr 172.21.15.42:0/334707886 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:05.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.542+0000 7f7e24a00640 1 --2- 172.21.15.42:0/334707886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2015d780 0x7f7e2015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:05.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.542+0000 7f7e1fe00640 1 --2- 172.21.15.42:0/334707886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e20160830 0x7f7e2013fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:05.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.546+0000 7f7e1fe00640 1 -- 172.21.15.42:0/334707886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2015d780 msgr2=0x7f7e2015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:05.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.546+0000 7f7e1fe00640 1 --2- 172.21.15.42:0/334707886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2015d780 0x7f7e2015af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:05.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.546+0000 7f7e1fe00640 1 -- 172.21.15.42:0/334707886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e200cfa60 msgr2=0x7f7e20158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:05.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.546+0000 7f7e1fe00640 1 --2- 172.21.15.42:0/334707886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e200cfa60 0x7f7e20158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:05.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.546+0000 7f7e1fe00640 1 -- 172.21.15.42:0/334707886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2015b7b0 con 0x7f7e20160830 2024-05-28T14:06:05.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.546+0000 7f7e1fe00640 1 --2- 172.21.15.42:0/334707886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e20160830 0x7f7e2013fff0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400e8a0 tx=0x7f7e1400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:05.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.546+0000 7f7e1cc00640 1 -- 172.21.15.42:0/334707886 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14017070 con 0x7f7e20160830 2024-05-28T14:06:05.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.546+0000 7f7e1cc00640 1 -- 172.21.15.42:0/334707886 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e1400c940 con 0x7f7e20160830 2024-05-28T14:06:05.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.546+0000 7f7e1cc00640 1 -- 172.21.15.42:0/334707886 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1400cd10 con 0x7f7e20160830 2024-05-28T14:06:05.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.546+0000 7f7e27400640 1 -- 172.21.15.42:0/334707886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2015ba90 con 0x7f7e20160830 2024-05-28T14:06:05.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.546+0000 7f7e27400640 1 -- 172.21.15.42:0/334707886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e20140a60 con 0x7f7e20160830 2024-05-28T14:06:05.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.550+0000 7f7e27400640 1 -- 172.21.15.42:0/334707886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e200691a0 con 0x7f7e20160830 2024-05-28T14:06:05.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.550+0000 7f7e1cc00640 1 -- 172.21.15.42:0/334707886 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f7e1401a070 con 0x7f7e20160830 2024-05-28T14:06:05.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.554+0000 7f7e1cc00640 1 --2- 172.21.15.42:0/334707886 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e04069140 0x7f7e0406b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:05.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.554+0000 7f7e1cc00640 1 -- 172.21.15.42:0/334707886 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5793+0+0 (secure 0 0 0) 0x7f7e140854a0 con 0x7f7e20160830 2024-05-28T14:06:05.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.554+0000 7f7e24a00640 1 --2- 172.21.15.42:0/334707886 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e04069140 0x7f7e0406b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:05.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.554+0000 7f7e24a00640 1 --2- 172.21.15.42:0/334707886 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e04069140 0x7f7e0406b600 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7e100ad000 tx=0x7f7e10008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:05.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.562+0000 7f7e1cc00640 1 -- 172.21.15.42:0/334707886 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7e14056e60 con 0x7f7e20160830 2024-05-28T14:06:05.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.882+0000 7f7e27400640 1 -- 172.21.15.42:0/334707886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e200ce5c0 con 0x7f7e20160830 2024-05-28T14:06:05.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.886+0000 7f7e1cc00640 1 -- 172.21.15.42:0/334707886 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16757 (secure 0 0 0) 0x7f7e1405b160 con 0x7f7e20160830 2024-05-28T14:06:05.872 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:05.872 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":62,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:06:03.536070+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:04:20.721059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-28T14:04:21.922127+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":62,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:06:05.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.886+0000 7f7e27400640 1 -- 172.21.15.42:0/334707886 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e04069140 msgr2=0x7f7e0406b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.886+0000 7f7e27400640 1 --2- 172.21.15.42:0/334707886 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e04069140 0x7f7e0406b600 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7e100ad000 tx=0x7f7e10008040 comp rx=0 tx=0).stop 2024-05-28T14:06:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.886+0000 7f7e27400640 1 -- 172.21.15.42:0/334707886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e20160830 msgr2=0x7f7e2013fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.886+0000 7f7e27400640 1 --2- 172.21.15.42:0/334707886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e20160830 0x7f7e2013fff0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400e8a0 tx=0x7f7e1400ed60 comp rx=0 tx=0).stop 2024-05-28T14:06:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.890+0000 7f7e27400640 1 -- 172.21.15.42:0/334707886 shutdown_connections 2024-05-28T14:06:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.890+0000 7f7e27400640 1 --2- 172.21.15.42:0/334707886 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e04069140 0x7f7e0406b600 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.890+0000 7f7e27400640 1 --2- 172.21.15.42:0/334707886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e200cfa60 0x7f7e20158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.890+0000 7f7e27400640 1 --2- 172.21.15.42:0/334707886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e20160830 0x7f7e2013fff0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.890+0000 7f7e27400640 1 --2- 172.21.15.42:0/334707886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2015d780 0x7f7e2015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.890+0000 7f7e27400640 1 -- 172.21.15.42:0/334707886 >> 172.21.15.42:0/334707886 conn(0x7f7e200b68f0 msgr2=0x7f7e201600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:05.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.890+0000 7f7e27400640 1 -- 172.21.15.42:0/334707886 shutdown_connections 2024-05-28T14:06:05.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:05.890+0000 7f7e27400640 1 -- 172.21.15.42:0/334707886 wait complete. 2024-05-28T14:06:05.913 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T14:04:20.721059+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T14:04:21.922127+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 62, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:06:05.913 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:06:06.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.170+0000 7f215c600640 1 Processor -- start 2024-05-28T14:06:06.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.170+0000 7f215c600640 1 -- start start 2024-05-28T14:06:06.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.174+0000 7f215c600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21540724f0 0x7f215406f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:06.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.174+0000 7f215c600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f21540cfb40 0x7f21540b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:06.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.174+0000 7f215c600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2154166800 con 0x7f2154162d10 2024-05-28T14:06:06.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.174+0000 7f215c600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f21541630f0 con 0x7f21540cfb40 2024-05-28T14:06:06.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.174+0000 7f215c600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f215406f8c0 con 0x7f21540724f0 2024-05-28T14:06:06.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.174+0000 7f2159c00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f2154162d10 0x7f2154071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:43424/0 (socket says 172.21.15.42:43424) 2024-05-28T14:06:06.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.174+0000 7f2159c00640 1 -- 172.21.15.42:0/1369476275 learned_addr learned my addr 172.21.15.42:0/1369476275 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:06.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.174+0000 7f2159200640 1 --2- 172.21.15.42:0/1369476275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21540724f0 0x7f215406f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:06.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.174+0000 7f215a600640 1 --2- 172.21.15.42:0/1369476275 >> v2:172.21.15.63:3300/0 conn(0x7f21540cfb40 0x7f21540b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:06.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.178+0000 7f2159200640 1 -- 172.21.15.42:0/1369476275 >> v2:172.21.15.63:3300/0 conn(0x7f21540cfb40 msgr2=0x7f21540b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:06.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.178+0000 7f2159200640 1 --2- 172.21.15.42:0/1369476275 >> v2:172.21.15.63:3300/0 conn(0x7f21540cfb40 0x7f21540b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.178+0000 7f2159200640 1 -- 172.21.15.42:0/1369476275 >> v1:172.21.15.63:6789/0 conn(0x7f2154162d10 legacy=0x7f2154071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:06.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.178+0000 7f2159200640 1 -- 172.21.15.42:0/1369476275 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2154166ac0 con 0x7f21540724f0 2024-05-28T14:06:06.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.178+0000 7f2153e00640 1 -- 172.21.15.42:0/1369476275 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1303314127 0 0) 0x7f2154166800 con 0x7f2154162d10 2024-05-28T14:06:06.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.182+0000 7f2159200640 1 --2- 172.21.15.42:0/1369476275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21540724f0 0x7f215406f2e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f21400097c0 tx=0x7f214001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1509924bfaa18533 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:06.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.182+0000 7f2153e00640 1 -- 172.21.15.42:0/1369476275 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21400b2070 con 0x7f21540724f0 2024-05-28T14:06:06.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.182+0000 7f2153e00640 1 -- 172.21.15.42:0/1369476275 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21400ad640 con 0x7f21540724f0 2024-05-28T14:06:06.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.186+0000 7f2153e00640 1 -- 172.21.15.42:0/1369476275 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21400ada10 con 0x7f21540724f0 2024-05-28T14:06:06.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.186+0000 7f215c600640 1 -- 172.21.15.42:0/1369476275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21540724f0 msgr2=0x7f215406f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:06.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.186+0000 7f215c600640 1 --2- 172.21.15.42:0/1369476275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21540724f0 0x7f215406f2e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f21400097c0 tx=0x7f214001c4a0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.186+0000 7f215c600640 1 -- 172.21.15.42:0/1369476275 shutdown_connections 2024-05-28T14:06:06.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.186+0000 7f215c600640 1 --2- 172.21.15.42:0/1369476275 >> v2:172.21.15.63:3300/0 conn(0x7f21540cfb40 0x7f21540b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.186+0000 7f215c600640 1 --2- 172.21.15.42:0/1369476275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21540724f0 0x7f215406f2e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.186+0000 7f215c600640 1 -- 172.21.15.42:0/1369476275 >> 172.21.15.42:0/1369476275 conn(0x7f21540b21f0 msgr2=0x7f21540b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:06.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.186+0000 7f215c600640 1 -- 172.21.15.42:0/1369476275 shutdown_connections 2024-05-28T14:06:06.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.186+0000 7f215c600640 1 -- 172.21.15.42:0/1369476275 wait complete. 2024-05-28T14:06:06.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.190+0000 7f215c600640 1 Processor -- start 2024-05-28T14:06:06.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.190+0000 7f215c600640 1 -- start start 2024-05-28T14:06:06.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.190+0000 7f215c600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21540724f0 0x7f2154164ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:06.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.190+0000 7f215c600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21540cfb40 0x7f21541424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:06.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.190+0000 7f215c600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2154162d10 0x7f215415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:06.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.190+0000 7f215c600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f21541675a0 con 0x7f21540724f0 2024-05-28T14:06:06.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.190+0000 7f215c600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f21540b4240 con 0x7f2154162d10 2024-05-28T14:06:06.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.190+0000 7f215c600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f21541401c0 con 0x7f21540cfb40 2024-05-28T14:06:06.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.190+0000 7f215a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2154162d10 0x7f215415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:06.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.190+0000 7f215a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2154162d10 0x7f215415bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:49072/0 (socket says 172.21.15.42:49072) 2024-05-28T14:06:06.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.190+0000 7f215a600640 1 -- 172.21.15.42:0/1433925670 learned_addr learned my addr 172.21.15.42:0/1433925670 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:06.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.190+0000 7f2159c00640 1 --2- 172.21.15.42:0/1433925670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21540724f0 0x7f2154164ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:06.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.194+0000 7f2159200640 1 --2- 172.21.15.42:0/1433925670 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21540cfb40 0x7f21541424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:06.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.194+0000 7f215a600640 1 -- 172.21.15.42:0/1433925670 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21540cfb40 msgr2=0x7f21541424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:06.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.194+0000 7f215a600640 1 --2- 172.21.15.42:0/1433925670 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21540cfb40 0x7f21541424b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.194+0000 7f215a600640 1 -- 172.21.15.42:0/1433925670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21540724f0 msgr2=0x7f2154164ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:06.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.194+0000 7f215a600640 1 --2- 172.21.15.42:0/1433925670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21540724f0 0x7f2154164ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.194+0000 7f215a600640 1 -- 172.21.15.42:0/1433925670 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2154165650 con 0x7f2154162d10 2024-05-28T14:06:06.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.194+0000 7f215a600640 1 --2- 172.21.15.42:0/1433925670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2154162d10 0x7f215415bcc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f214c007e30 tx=0x7f214c00e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:06.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.194+0000 7f2151600640 1 -- 172.21.15.42:0/1433925670 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c014620 con 0x7f2154162d10 2024-05-28T14:06:06.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.194+0000 7f2151600640 1 -- 172.21.15.42:0/1433925670 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214c019070 con 0x7f2154162d10 2024-05-28T14:06:06.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.194+0000 7f2151600640 1 -- 172.21.15.42:0/1433925670 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c014a00 con 0x7f2154162d10 2024-05-28T14:06:06.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.194+0000 7f215c600640 1 -- 172.21.15.42:0/1433925670 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2154165930 con 0x7f2154162d10 2024-05-28T14:06:06.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.194+0000 7f215c600640 1 -- 172.21.15.42:0/1433925670 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2154140410 con 0x7f2154162d10 2024-05-28T14:06:06.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.194+0000 7f215c600640 1 -- 172.21.15.42:0/1433925670 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21540738b0 con 0x7f2154162d10 2024-05-28T14:06:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.198+0000 7f2151600640 1 -- 172.21.15.42:0/1433925670 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f214c014ba0 con 0x7f2154162d10 2024-05-28T14:06:06.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.198+0000 7f2151600640 1 --2- 172.21.15.42:0/1433925670 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21340690c0 0x7f213406b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:06.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.198+0000 7f2151600640 1 -- 172.21.15.42:0/1433925670 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5793+0+0 (secure 0 0 0) 0x7f214c085c20 con 0x7f2154162d10 2024-05-28T14:06:06.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.198+0000 7f2159c00640 1 --2- 172.21.15.42:0/1433925670 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21340690c0 0x7f213406b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:06.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.202+0000 7f2159c00640 1 --2- 172.21.15.42:0/1433925670 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21340690c0 0x7f213406b580 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2148000f70 tx=0x7f2148007240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:06.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.210+0000 7f2151600640 1 -- 172.21.15.42:0/1433925670 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f214c057c70 con 0x7f2154162d10 2024-05-28T14:06:06.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.434+0000 7f215c600640 1 -- 172.21.15.42:0/1433925670 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f215404fdd0 con 0x7f2154162d10 2024-05-28T14:06:06.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.434+0000 7f2151600640 1 -- 172.21.15.42:0/1433925670 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) ==== 93+0+10 (secure 0 0 0) 0x7f214c057570 con 0x7f2154162d10 2024-05-28T14:06:06.420 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:06:06.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.434+0000 7f215c600640 1 -- 172.21.15.42:0/1433925670 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21340690c0 msgr2=0x7f213406b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:06.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.434+0000 7f215c600640 1 --2- 172.21.15.42:0/1433925670 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21340690c0 0x7f213406b580 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2148000f70 tx=0x7f2148007240 comp rx=0 tx=0).stop 2024-05-28T14:06:06.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.434+0000 7f215c600640 1 -- 172.21.15.42:0/1433925670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2154162d10 msgr2=0x7f215415bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:06.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.434+0000 7f215c600640 1 --2- 172.21.15.42:0/1433925670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2154162d10 0x7f215415bcc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f214c007e30 tx=0x7f214c00e4b0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.434+0000 7f215c600640 1 -- 172.21.15.42:0/1433925670 shutdown_connections 2024-05-28T14:06:06.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.434+0000 7f215c600640 1 --2- 172.21.15.42:0/1433925670 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21340690c0 0x7f213406b580 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.434+0000 7f215c600640 1 --2- 172.21.15.42:0/1433925670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2154162d10 0x7f215415bcc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.434+0000 7f215c600640 1 --2- 172.21.15.42:0/1433925670 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21540cfb40 0x7f21541424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.434+0000 7f215c600640 1 --2- 172.21.15.42:0/1433925670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21540724f0 0x7f2154164ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.434+0000 7f215c600640 1 -- 172.21.15.42:0/1433925670 >> 172.21.15.42:0/1433925670 conn(0x7f21540b21f0 msgr2=0x7f21540b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:06.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.434+0000 7f215c600640 1 -- 172.21.15.42:0/1433925670 shutdown_connections 2024-05-28T14:06:06.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.434+0000 7f215c600640 1 -- 172.21.15.42:0/1433925670 wait complete. 2024-05-28T14:06:06.442 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:06:06.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71d6200640 1 Processor -- start 2024-05-28T14:06:06.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71d6200640 1 -- start start 2024-05-28T14:06:06.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71d6200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d015b570 0x7f71d00cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:06.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71d6200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f71d015b950 0x7f71d00b2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:06.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71d6200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71d00b2f20 con 0x7f71d00cdde0 2024-05-28T14:06:06.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71d6200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f71d00b44e0 con 0x7f71d015b950 2024-05-28T14:06:06.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71d6200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f71d00b47c0 con 0x7f71d015b570 2024-05-28T14:06:06.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71c7e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f71d015b950 0x7f71d00b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:06.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71cf400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d015b570 0x7f71d00cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:06.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71cfe00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f71d00cdde0 0x7f71d00d0890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:43430/0 (socket says 172.21.15.42:43430) 2024-05-28T14:06:06.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71c7e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f71d015b950 0x7f71d00b2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:49076/0 (socket says 172.21.15.42:49076) 2024-05-28T14:06:06.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71cfe00640 1 -- 172.21.15.42:0/2228897503 learned_addr learned my addr 172.21.15.42:0/2228897503 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:06.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71cf400640 1 -- 172.21.15.42:0/2228897503 >> v2:172.21.15.63:3300/0 conn(0x7f71d015b950 msgr2=0x7f71d00b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:06.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71cf400640 1 --2- 172.21.15.42:0/2228897503 >> v2:172.21.15.63:3300/0 conn(0x7f71d015b950 0x7f71d00b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71cf400640 1 -- 172.21.15.42:0/2228897503 >> v1:172.21.15.63:6789/0 conn(0x7f71d00cdde0 legacy=0x7f71d00d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:06.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71cea00640 1 -- 172.21.15.42:0/2228897503 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4027927848 0 0) 0x7f71d00b2f20 con 0x7f71d00cdde0 2024-05-28T14:06:06.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71cf400640 1 -- 172.21.15.42:0/2228897503 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d00b4aa0 con 0x7f71d015b570 2024-05-28T14:06:06.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.602+0000 7f71c7e00640 1 --2- 172.21.15.42:0/2228897503 >> v2:172.21.15.63:3300/0 conn(0x7f71d015b950 0x7f71d00b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:06:06.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.606+0000 7f71cf400640 1 --2- 172.21.15.42:0/2228897503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d015b570 0x7f71d00cefc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f71c00097c0 tx=0x7f71c001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d12cdf8ced612ccf server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.606+0000 7f71cea00640 1 -- 172.21.15.42:0/2228897503 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00b0070 con 0x7f71d015b570 2024-05-28T14:06:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.606+0000 7f71cea00640 1 -- 172.21.15.42:0/2228897503 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c00ab600 con 0x7f71d015b570 2024-05-28T14:06:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.606+0000 7f71cea00640 1 -- 172.21.15.42:0/2228897503 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00ab9d0 con 0x7f71d015b570 2024-05-28T14:06:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.606+0000 7f71d6200640 1 -- 172.21.15.42:0/2228897503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d015b570 msgr2=0x7f71d00cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.606+0000 7f71d6200640 1 --2- 172.21.15.42:0/2228897503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d015b570 0x7f71d00cefc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f71c00097c0 tx=0x7f71c001c410 comp rx=0 tx=0).stop 2024-05-28T14:06:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.606+0000 7f71d6200640 1 -- 172.21.15.42:0/2228897503 shutdown_connections 2024-05-28T14:06:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.606+0000 7f71d6200640 1 --2- 172.21.15.42:0/2228897503 >> v2:172.21.15.63:3300/0 conn(0x7f71d015b950 0x7f71d00b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.606+0000 7f71d6200640 1 --2- 172.21.15.42:0/2228897503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d015b570 0x7f71d00cefc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.606+0000 7f71d6200640 1 -- 172.21.15.42:0/2228897503 >> 172.21.15.42:0/2228897503 conn(0x7f71d00b6960 msgr2=0x7f71d00b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.606+0000 7f71d6200640 1 -- 172.21.15.42:0/2228897503 shutdown_connections 2024-05-28T14:06:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.606+0000 7f71d6200640 1 -- 172.21.15.42:0/2228897503 wait complete. 2024-05-28T14:06:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71d6200640 1 Processor -- start 2024-05-28T14:06:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71d6200640 1 -- start start 2024-05-28T14:06:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71d6200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d00cdde0 0x7f71d0145f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71d6200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d015b570 0x7f71d014f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71d6200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71d015b950 0x7f71d014a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71d6200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f71d006b9f0 con 0x7f71d015b570 2024-05-28T14:06:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71d6200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f71d006bb70 con 0x7f71d015b950 2024-05-28T14:06:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71d6200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f71d006bcf0 con 0x7f71d00cdde0 2024-05-28T14:06:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71c7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d015b570 0x7f71d014f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71c7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d015b570 0x7f71d014f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54376/0 (socket says 172.21.15.42:54376) 2024-05-28T14:06:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71c7e00640 1 -- 172.21.15.42:0/1202462446 learned_addr learned my addr 172.21.15.42:0/1202462446 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71cf400640 1 --2- 172.21.15.42:0/1202462446 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d00cdde0 0x7f71d0145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71cfe00640 1 --2- 172.21.15.42:0/1202462446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71d015b950 0x7f71d014a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71c7e00640 1 -- 172.21.15.42:0/1202462446 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d00cdde0 msgr2=0x7f71d0145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71c7e00640 1 --2- 172.21.15.42:0/1202462446 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d00cdde0 0x7f71d0145f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71c7e00640 1 -- 172.21.15.42:0/1202462446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71d015b950 msgr2=0x7f71d014a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71c7e00640 1 --2- 172.21.15.42:0/1202462446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71d015b950 0x7f71d014a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71c7e00640 1 -- 172.21.15.42:0/1202462446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d015ebe0 con 0x7f71d015b570 2024-05-28T14:06:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71c7e00640 1 --2- 172.21.15.42:0/1202462446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d015b570 0x7f71d014f930 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f71b800e980 tx=0x7f71b800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71c7400640 1 -- 172.21.15.42:0/1202462446 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8017070 con 0x7f71d015b570 2024-05-28T14:06:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71c7400640 1 -- 172.21.15.42:0/1202462446 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b800ca00 con 0x7f71d015b570 2024-05-28T14:06:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71c7400640 1 -- 172.21.15.42:0/1202462446 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b800cdd0 con 0x7f71d015b570 2024-05-28T14:06:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71d6200640 1 -- 172.21.15.42:0/1202462446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d014fe70 con 0x7f71d015b570 2024-05-28T14:06:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.610+0000 7f71d6200640 1 -- 172.21.15.42:0/1202462446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d0149e40 con 0x7f71d015b570 2024-05-28T14:06:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.614+0000 7f71d6200640 1 -- 172.21.15.42:0/1202462446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d009f7a0 con 0x7f71d015b570 2024-05-28T14:06:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.614+0000 7f71c7400640 1 -- 172.21.15.42:0/1202462446 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f71b801a070 con 0x7f71d015b570 2024-05-28T14:06:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.614+0000 7f71c7400640 1 --2- 172.21.15.42:0/1202462446 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f71ac069070 0x7f71ac06b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.614+0000 7f71c7400640 1 -- 172.21.15.42:0/1202462446 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5793+0+0 (secure 0 0 0) 0x7f71b80851b0 con 0x7f71d015b570 2024-05-28T14:06:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.614+0000 7f71cf400640 1 --2- 172.21.15.42:0/1202462446 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f71ac069070 0x7f71ac06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.614+0000 7f71cf400640 1 --2- 172.21.15.42:0/1202462446 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f71ac069070 0x7f71ac06b530 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f71c00ad000 tx=0x7f71c00a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:06.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.618+0000 7f71c7400640 1 -- 172.21.15.42:0/1202462446 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f71b8056b70 con 0x7f71d015b570 2024-05-28T14:06:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.882+0000 7f71d6200640 1 -- 172.21.15.42:0/1202462446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f71d004fdd0 con 0x7f71d015b570 2024-05-28T14:06:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.882+0000 7f71c7400640 1 -- 172.21.15.42:0/1202462446 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) ==== 104+0+11 (secure 0 0 0) 0x7f71b805ae70 con 0x7f71d015b570 2024-05-28T14:06:06.869 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:06:06.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.886+0000 7f71d6200640 1 -- 172.21.15.42:0/1202462446 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f71ac069070 msgr2=0x7f71ac06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:06.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.886+0000 7f71d6200640 1 --2- 172.21.15.42:0/1202462446 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f71ac069070 0x7f71ac06b530 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f71c00ad000 tx=0x7f71c00a6000 comp rx=0 tx=0).stop 2024-05-28T14:06:06.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.886+0000 7f71d6200640 1 -- 172.21.15.42:0/1202462446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d015b570 msgr2=0x7f71d014f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:06.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.886+0000 7f71d6200640 1 --2- 172.21.15.42:0/1202462446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d015b570 0x7f71d014f930 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f71b800e980 tx=0x7f71b800ee40 comp rx=0 tx=0).stop 2024-05-28T14:06:06.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.886+0000 7f71d6200640 1 -- 172.21.15.42:0/1202462446 shutdown_connections 2024-05-28T14:06:06.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.886+0000 7f71d6200640 1 --2- 172.21.15.42:0/1202462446 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f71ac069070 0x7f71ac06b530 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.886+0000 7f71d6200640 1 --2- 172.21.15.42:0/1202462446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71d015b950 0x7f71d014a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.886+0000 7f71d6200640 1 --2- 172.21.15.42:0/1202462446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71d015b570 0x7f71d014f930 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.886+0000 7f71d6200640 1 --2- 172.21.15.42:0/1202462446 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71d00cdde0 0x7f71d0145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:06.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.886+0000 7f71d6200640 1 -- 172.21.15.42:0/1202462446 >> 172.21.15.42:0/1202462446 conn(0x7f71d00b6960 msgr2=0x7f71d0159eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:06.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.886+0000 7f71d6200640 1 -- 172.21.15.42:0/1202462446 shutdown_connections 2024-05-28T14:06:06.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:06.886+0000 7f71d6200640 1 -- 172.21.15.42:0/1202462446 wait complete. 2024-05-28T14:06:06.912 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:06:07.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.150+0000 7f7341400640 1 Processor -- start 2024-05-28T14:06:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.150+0000 7f7341400640 1 -- start start 2024-05-28T14:06:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.154+0000 7f7341400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f733c15ccb0 0x7f733c15d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.154+0000 7f7341400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f733c0d01b0 0x7f733c0d0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.154+0000 7f7341400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f733c15d5d0 con 0x7f733c0cd0d0 2024-05-28T14:06:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.154+0000 7f7341400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f733c0ce210 con 0x7f733c0d01b0 2024-05-28T14:06:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.154+0000 7f7341400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f733c0cd600 con 0x7f733c15ccb0 2024-05-28T14:06:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.154+0000 7f733a800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f733c15ccb0 0x7f733c15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.154+0000 7f733a800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f733c15ccb0 0x7f733c15d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:49094/0 (socket says 172.21.15.42:49094) 2024-05-28T14:06:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.154+0000 7f733a800640 1 -- 172.21.15.42:0/1034623112 learned_addr learned my addr 172.21.15.42:0/1034623112 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.154+0000 7f733b200640 1 --2- 172.21.15.42:0/1034623112 >> v2:172.21.15.42:3300/0 conn(0x7f733c0d01b0 0x7f733c0d0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.154+0000 7f733a800640 1 -- 172.21.15.42:0/1034623112 >> v2:172.21.15.42:3300/0 conn(0x7f733c0d01b0 msgr2=0x7f733c0d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.154+0000 7f733a800640 1 --2- 172.21.15.42:0/1034623112 >> v2:172.21.15.42:3300/0 conn(0x7f733c0d01b0 0x7f733c0d0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.154+0000 7f733a800640 1 -- 172.21.15.42:0/1034623112 >> v1:172.21.15.42:6789/0 conn(0x7f733c0cd0d0 legacy=0x7f733c09b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.154+0000 7f733a800640 1 -- 172.21.15.42:0/1034623112 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c0cd8e0 con 0x7f733c15ccb0 2024-05-28T14:06:07.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.162+0000 7f733a800640 1 --2- 172.21.15.42:0/1034623112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f733c15ccb0 0x7f733c15d090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f733c073900 tx=0x7f732c01c4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f18b802ab4ec95af server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:07.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.162+0000 7f7339400640 1 -- 172.21.15.42:0/1034623112 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c0b1070 con 0x7f733c15ccb0 2024-05-28T14:06:07.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.162+0000 7f7339400640 1 -- 172.21.15.42:0/1034623112 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f732c0ac630 con 0x7f733c15ccb0 2024-05-28T14:06:07.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.162+0000 7f7339400640 1 -- 172.21.15.42:0/1034623112 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c0a8050 con 0x7f733c15ccb0 2024-05-28T14:06:07.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.162+0000 7f7341400640 1 -- 172.21.15.42:0/1034623112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f733c15ccb0 msgr2=0x7f733c15d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:07.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.162+0000 7f7341400640 1 --2- 172.21.15.42:0/1034623112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f733c15ccb0 0x7f733c15d090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f733c073900 tx=0x7f732c01c4e0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.162+0000 7f7341400640 1 -- 172.21.15.42:0/1034623112 shutdown_connections 2024-05-28T14:06:07.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.162+0000 7f7341400640 1 --2- 172.21.15.42:0/1034623112 >> v2:172.21.15.42:3300/0 conn(0x7f733c0d01b0 0x7f733c0d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.162+0000 7f7341400640 1 --2- 172.21.15.42:0/1034623112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f733c15ccb0 0x7f733c15d090 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.162+0000 7f7341400640 1 -- 172.21.15.42:0/1034623112 >> 172.21.15.42:0/1034623112 conn(0x7f733c0b40d0 msgr2=0x7f733c0b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:07.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.162+0000 7f7341400640 1 -- 172.21.15.42:0/1034623112 shutdown_connections 2024-05-28T14:06:07.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.162+0000 7f7341400640 1 -- 172.21.15.42:0/1034623112 wait complete. 2024-05-28T14:06:07.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.166+0000 7f7341400640 1 Processor -- start 2024-05-28T14:06:07.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.166+0000 7f7341400640 1 -- start start 2024-05-28T14:06:07.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.166+0000 7f7341400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f733c0cd0d0 0x7f733c0c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:07.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.166+0000 7f7341400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f733c0d01b0 0x7f733c0b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:07.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f7341400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f733c15ccb0 0x7f733c0cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:07.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f7341400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f733c1695b0 con 0x7f733c0d01b0 2024-05-28T14:06:07.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f7341400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f733c15caf0 con 0x7f733c15ccb0 2024-05-28T14:06:07.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f7341400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f733c169730 con 0x7f733c0cd0d0 2024-05-28T14:06:07.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f733a800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f733c0cd0d0 0x7f733c0c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:07.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f733a800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f733c0cd0d0 0x7f733c0c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:58644/0 (socket says 172.21.15.42:58644) 2024-05-28T14:06:07.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f733a800640 1 -- 172.21.15.42:0/199157751 learned_addr learned my addr 172.21.15.42:0/199157751 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:07.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f733b200640 1 --2- 172.21.15.42:0/199157751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f733c15ccb0 0x7f733c0cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:07.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f7339e00640 1 --2- 172.21.15.42:0/199157751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f733c0d01b0 0x7f733c0b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:07.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f733a800640 1 -- 172.21.15.42:0/199157751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f733c15ccb0 msgr2=0x7f733c0cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:07.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f733a800640 1 --2- 172.21.15.42:0/199157751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f733c15ccb0 0x7f733c0cac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f733a800640 1 -- 172.21.15.42:0/199157751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f733c0d01b0 msgr2=0x7f733c0b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:07.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f733a800640 1 --2- 172.21.15.42:0/199157751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f733c0d01b0 0x7f733c0b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f733a800640 1 -- 172.21.15.42:0/199157751 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c0b6b90 con 0x7f733c0cd0d0 2024-05-28T14:06:07.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f733a800640 1 --2- 172.21.15.42:0/199157751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f733c0cd0d0 0x7f733c0c9910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f732c0190f0 tx=0x7f732c019280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:07.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f7332a00640 1 -- 172.21.15.42:0/199157751 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c0b1050 con 0x7f733c0cd0d0 2024-05-28T14:06:07.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f7332a00640 1 -- 172.21.15.42:0/199157751 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f732c0bb070 con 0x7f733c0cd0d0 2024-05-28T14:06:07.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f7332a00640 1 -- 172.21.15.42:0/199157751 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c0a8040 con 0x7f733c0cd0d0 2024-05-28T14:06:07.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f7341400640 1 -- 172.21.15.42:0/199157751 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733c0b6e70 con 0x7f733c0cd0d0 2024-05-28T14:06:07.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.170+0000 7f7341400640 1 -- 172.21.15.42:0/199157751 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f733c0c6d00 con 0x7f733c0cd0d0 2024-05-28T14:06:07.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.174+0000 7f7341400640 1 -- 172.21.15.42:0/199157751 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73140052a0 con 0x7f733c0cd0d0 2024-05-28T14:06:07.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.178+0000 7f7332a00640 1 -- 172.21.15.42:0/199157751 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f732c0ab800 con 0x7f733c0cd0d0 2024-05-28T14:06:07.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.178+0000 7f7332a00640 1 --2- 172.21.15.42:0/199157751 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f731c0690f0 0x7f731c06b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:07.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.178+0000 7f7332a00640 1 -- 172.21.15.42:0/199157751 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5793+0+0 (secure 0 0 0) 0x7f732c11f280 con 0x7f733c0cd0d0 2024-05-28T14:06:07.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.178+0000 7f7339e00640 1 --2- 172.21.15.42:0/199157751 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f731c0690f0 0x7f731c06b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:07.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.182+0000 7f7339e00640 1 --2- 172.21.15.42:0/199157751 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f731c0690f0 0x7f731c06b5b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f733c15bc70 tx=0x7f7328008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:07.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.182+0000 7f7332a00640 1 -- 172.21.15.42:0/199157751 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f732c0160d0 con 0x7f733c0cd0d0 2024-05-28T14:06:07.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.510+0000 7f7341400640 1 -- 172.21.15.42:0/199157751 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7314005860 con 0x7f733c0cd0d0 2024-05-28T14:06:07.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.514+0000 7f7332a00640 1 -- 172.21.15.42:0/199157751 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) ==== 100+0+11 (secure 0 0 0) 0x7f732c0f1350 con 0x7f733c0cd0d0 2024-05-28T14:06:07.501 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:06:07.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.518+0000 7f7341400640 1 -- 172.21.15.42:0/199157751 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f731c0690f0 msgr2=0x7f731c06b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:07.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.518+0000 7f7341400640 1 --2- 172.21.15.42:0/199157751 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f731c0690f0 0x7f731c06b5b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f733c15bc70 tx=0x7f7328008040 comp rx=0 tx=0).stop 2024-05-28T14:06:07.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.518+0000 7f7341400640 1 -- 172.21.15.42:0/199157751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f733c0cd0d0 msgr2=0x7f733c0c9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:07.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.518+0000 7f7341400640 1 --2- 172.21.15.42:0/199157751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f733c0cd0d0 0x7f733c0c9910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f732c0190f0 tx=0x7f732c019280 comp rx=0 tx=0).stop 2024-05-28T14:06:07.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.518+0000 7f7341400640 1 -- 172.21.15.42:0/199157751 shutdown_connections 2024-05-28T14:06:07.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.518+0000 7f7341400640 1 --2- 172.21.15.42:0/199157751 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f731c0690f0 0x7f731c06b5b0 secure :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f733c15bc70 tx=0x7f7328008040 comp rx=0 tx=0).stop 2024-05-28T14:06:07.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.518+0000 7f7341400640 1 --2- 172.21.15.42:0/199157751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f733c15ccb0 0x7f733c0cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.518+0000 7f7341400640 1 --2- 172.21.15.42:0/199157751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f733c0d01b0 0x7f733c0b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.518+0000 7f7341400640 1 --2- 172.21.15.42:0/199157751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f733c0cd0d0 0x7f733c0c9910 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.518+0000 7f7341400640 1 -- 172.21.15.42:0/199157751 >> 172.21.15.42:0/199157751 conn(0x7f733c0b40d0 msgr2=0x7f733c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:07.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.518+0000 7f7341400640 1 -- 172.21.15.42:0/199157751 shutdown_connections 2024-05-28T14:06:07.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.518+0000 7f7341400640 1 -- 172.21.15.42:0/199157751 wait complete. 2024-05-28T14:06:07.539 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T14:06:07.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.750+0000 7f01b7c00640 1 Processor -- start 2024-05-28T14:06:07.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01b7c00640 1 -- start start 2024-05-28T14:06:07.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01b7c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f01b015bf30 0x7f01b015c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:07.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01b7c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f01b00cf230 0x7f01b0072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:07.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01b7c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01b0072ba0 0x7f01b006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:07.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01b7c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f01b00721d0 con 0x7f01b00cf230 2024-05-28T14:06:07.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01b7c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f01b00cfc10 con 0x7f01b015bf30 2024-05-28T14:06:07.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01b7c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f01b00cfef0 con 0x7f01b0072ba0 2024-05-28T14:06:07.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01afe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f01b00cf230 0x7f01b0072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:07.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01afe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f01b00cf230 0x7f01b0072660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47526/0 (socket says 172.21.15.42:47526) 2024-05-28T14:06:07.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01afe00640 1 -- 172.21.15.42:0/1424119551 learned_addr learned my addr 172.21.15.42:0/1424119551 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:07.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01b5c00640 1 --2- 172.21.15.42:0/1424119551 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01b0072ba0 0x7f01b006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:07.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01b5200640 1 --2- 172.21.15.42:0/1424119551 >> v2:172.21.15.63:3300/0 conn(0x7f01b015bf30 0x7f01b015c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:07.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01afe00640 1 -- 172.21.15.42:0/1424119551 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01b0072ba0 msgr2=0x7f01b006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:07.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01afe00640 1 --2- 172.21.15.42:0/1424119551 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01b0072ba0 0x7f01b006ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01afe00640 1 -- 172.21.15.42:0/1424119551 >> v2:172.21.15.63:3300/0 conn(0x7f01b015bf30 msgr2=0x7f01b015c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:07.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01afe00640 1 --2- 172.21.15.42:0/1424119551 >> v2:172.21.15.63:3300/0 conn(0x7f01b015bf30 0x7f01b015c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.754+0000 7f01afe00640 1 -- 172.21.15.42:0/1424119551 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b00d01d0 con 0x7f01b00cf230 2024-05-28T14:06:07.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01afe00640 1 --2- 172.21.15.42:0/1424119551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b00cf230 0x7f01b0072660 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f01b006c990 tx=0x7f019c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d668699b6b98f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:07.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01af400640 1 -- 172.21.15.42:0/1424119551 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c0b1070 con 0x7f01b00cf230 2024-05-28T14:06:07.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01af400640 1 -- 172.21.15.42:0/1424119551 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f019c0ac630 con 0x7f01b00cf230 2024-05-28T14:06:07.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01af400640 1 -- 172.21.15.42:0/1424119551 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c0a8050 con 0x7f01b00cf230 2024-05-28T14:06:07.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01b7c00640 1 -- 172.21.15.42:0/1424119551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b00cf230 msgr2=0x7f01b0072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:07.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01b7c00640 1 --2- 172.21.15.42:0/1424119551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b00cf230 0x7f01b0072660 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f01b006c990 tx=0x7f019c01c4f0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01b7c00640 1 -- 172.21.15.42:0/1424119551 shutdown_connections 2024-05-28T14:06:07.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01b7c00640 1 --2- 172.21.15.42:0/1424119551 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01b0072ba0 0x7f01b006ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01b7c00640 1 --2- 172.21.15.42:0/1424119551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b00cf230 0x7f01b0072660 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01b7c00640 1 --2- 172.21.15.42:0/1424119551 >> v2:172.21.15.63:3300/0 conn(0x7f01b015bf30 0x7f01b015c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01b7c00640 1 -- 172.21.15.42:0/1424119551 >> 172.21.15.42:0/1424119551 conn(0x7f01b006b9f0 msgr2=0x7f01b006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:07.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01b7c00640 1 -- 172.21.15.42:0/1424119551 shutdown_connections 2024-05-28T14:06:07.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01b7c00640 1 -- 172.21.15.42:0/1424119551 wait complete. 2024-05-28T14:06:07.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01b7c00640 1 Processor -- start 2024-05-28T14:06:07.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.758+0000 7f01b7c00640 1 -- start start 2024-05-28T14:06:07.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01b7c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01b0072ba0 0x7f01b013d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:07.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01b7c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b00cf230 0x7f01b00aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:07.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01b7c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01b015bf30 0x7f01b0154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:07.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01b7c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f01b015f020 con 0x7f01b00cf230 2024-05-28T14:06:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01b7c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f01b006f5a0 con 0x7f01b0072ba0 2024-05-28T14:06:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01b7c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f01b015f1a0 con 0x7f01b015bf30 2024-05-28T14:06:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b00cf230 0x7f01b00aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b00cf230 0x7f01b00aeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47540/0 (socket says 172.21.15.42:47540) 2024-05-28T14:06:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01afe00640 1 -- 172.21.15.42:0/2017389684 learned_addr learned my addr 172.21.15.42:0/2017389684 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01b5200640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01b0072ba0 0x7f01b013d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01b5c00640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01b015bf30 0x7f01b0154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01afe00640 1 -- 172.21.15.42:0/2017389684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01b015bf30 msgr2=0x7f01b0154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01afe00640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01b015bf30 0x7f01b0154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01afe00640 1 -- 172.21.15.42:0/2017389684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01b0072ba0 msgr2=0x7f01b013d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01afe00640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01b0072ba0 0x7f01b013d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01afe00640 1 -- 172.21.15.42:0/2017389684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b0153ab0 con 0x7f01b00cf230 2024-05-28T14:06:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01b5c00640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01b015bf30 0x7f01b0154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01afe00640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b00cf230 0x7f01b00aeca0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f019c0af040 tx=0x7f019c019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:07.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01acc00640 1 -- 172.21.15.42:0/2017389684 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c0b1070 con 0x7f01b00cf230 2024-05-28T14:06:07.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01acc00640 1 -- 172.21.15.42:0/2017389684 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f019c0ab5e0 con 0x7f01b00cf230 2024-05-28T14:06:07.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01acc00640 1 -- 172.21.15.42:0/2017389684 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c0a8040 con 0x7f01b00cf230 2024-05-28T14:06:07.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01b7c00640 1 -- 172.21.15.42:0/2017389684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b0153d90 con 0x7f01b00cf230 2024-05-28T14:06:07.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01b7c00640 1 -- 172.21.15.42:0/2017389684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b0154350 con 0x7f01b00cf230 2024-05-28T14:06:07.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.762+0000 7f01b7c00640 1 -- 172.21.15.42:0/2017389684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b0073890 con 0x7f01b00cf230 2024-05-28T14:06:07.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.766+0000 7f01acc00640 1 -- 172.21.15.42:0/2017389684 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f019c0be060 con 0x7f01b00cf230 2024-05-28T14:06:07.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.766+0000 7f01acc00640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f01900690a0 0x7f019006b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:07.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.766+0000 7f01acc00640 1 -- 172.21.15.42:0/2017389684 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5765+0+0 (secure 0 0 0) 0x7f019c11efc0 con 0x7f01b00cf230 2024-05-28T14:06:07.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.766+0000 7f01b5200640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f01900690a0 0x7f019006b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:07.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.766+0000 7f01b5200640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f01900690a0 0x7f019006b560 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f01b015c430 tx=0x7f01a4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:07.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.770+0000 7f01acc00640 1 -- 172.21.15.42:0/2017389684 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f019c016110 con 0x7f01b00cf230 2024-05-28T14:06:07.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.950+0000 7f01b7c00640 1 -- 172.21.15.42:0/2017389684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f01b006ef70 con 0x7f01b00cf230 2024-05-28T14:06:07.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.954+0000 7f01acc00640 1 -- 172.21.15.42:0/2017389684 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) ==== 80+0+0 (secure 0 0 0) 0x7f019c0f0d40 con 0x7f01b00cf230 2024-05-28T14:06:07.940 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T14:06:07.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.954+0000 7f01b7c00640 1 -- 172.21.15.42:0/2017389684 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f01900690a0 msgr2=0x7f019006b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:07.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.954+0000 7f01b7c00640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f01900690a0 0x7f019006b560 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f01b015c430 tx=0x7f01a4008040 comp rx=0 tx=0).stop 2024-05-28T14:06:07.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.954+0000 7f01b7c00640 1 -- 172.21.15.42:0/2017389684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b00cf230 msgr2=0x7f01b00aeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:07.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.954+0000 7f01b7c00640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b00cf230 0x7f01b00aeca0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f019c0af040 tx=0x7f019c019290 comp rx=0 tx=0).stop 2024-05-28T14:06:07.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.954+0000 7f01b7c00640 1 -- 172.21.15.42:0/2017389684 shutdown_connections 2024-05-28T14:06:07.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.954+0000 7f01b7c00640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f01900690a0 0x7f019006b560 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.954+0000 7f01b7c00640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01b015bf30 0x7f01b0154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.954+0000 7f01b7c00640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b00cf230 0x7f01b00aeca0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.954+0000 7f01b7c00640 1 --2- 172.21.15.42:0/2017389684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01b0072ba0 0x7f01b013d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:07.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.954+0000 7f01b7c00640 1 -- 172.21.15.42:0/2017389684 >> 172.21.15.42:0/2017389684 conn(0x7f01b006b9f0 msgr2=0x7f01b0151290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:07.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.954+0000 7f01b7c00640 1 -- 172.21.15.42:0/2017389684 shutdown_connections 2024-05-28T14:06:07.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:07.954+0000 7f01b7c00640 1 -- 172.21.15.42:0/2017389684 wait complete. 2024-05-28T14:06:07.961 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-05-28T14:06:07.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:07.961 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-28T14:06:07.981 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[41448]: fuse finished with error 0 and tester_r 0 2024-05-28T14:06:07.982 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:06:07.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:07.982 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-28T14:06:08.139 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:06:08.140 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:06:08.140 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:06:08.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:06:08.149 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:06:08.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:08.149 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-28T14:06:08.204 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-28T14:06:08.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:06:08.206 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:06:08.206 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:06:08.206 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:06:08.215 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:06:08.215 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:06:08.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:08.215 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-28T14:06:08.270 INFO:teuthology.orchestra.run.smithi042.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-28T14:06:08.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:06:08.272 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:06:08.272 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:06:08.272 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:06:08.282 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:06:08.282 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:06:08.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:08.283 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-28T14:06:08.332 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-28T14:06:08.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:06:08.333 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:06:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.486+0000 7f0824800640 1 Processor -- start 2024-05-28T14:06:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.486+0000 7f0824800640 1 -- start start 2024-05-28T14:06:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.490+0000 7f0824800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f081c06f660 0x7f081c073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.490+0000 7f0824800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f081c0d0e00 0x7f081c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.490+0000 7f0824800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f081c15f280 con 0x7f081c15ba40 2024-05-28T14:06:08.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.490+0000 7f0824800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f081c15f030 con 0x7f081c0d0e00 2024-05-28T14:06:08.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.490+0000 7f0824800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f081c15f560 con 0x7f081c06f660 2024-05-28T14:06:08.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.490+0000 7f0822800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f081c0d0e00 0x7f081c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:08.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.490+0000 7f0822800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f081c0d0e00 0x7f081c06dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47556/0 (socket says 172.21.15.42:47556) 2024-05-28T14:06:08.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.490+0000 7f0822800640 1 -- 172.21.15.42:0/2366329843 learned_addr learned my addr 172.21.15.42:0/2366329843 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:08.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.490+0000 7f0821e00640 1 --2- 172.21.15.42:0/2366329843 >> v2:172.21.15.63:3300/0 conn(0x7f081c06f660 0x7f081c073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:08.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.490+0000 7f0822800640 1 -- 172.21.15.42:0/2366329843 >> v2:172.21.15.63:3300/0 conn(0x7f081c06f660 msgr2=0x7f081c073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:08.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.490+0000 7f0822800640 1 --2- 172.21.15.42:0/2366329843 >> v2:172.21.15.63:3300/0 conn(0x7f081c06f660 0x7f081c073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.490+0000 7f0822800640 1 -- 172.21.15.42:0/2366329843 >> v1:172.21.15.42:6789/0 conn(0x7f081c15ba40 legacy=0x7f081c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:08.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.490+0000 7f0822800640 1 -- 172.21.15.42:0/2366329843 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c15f840 con 0x7f081c0d0e00 2024-05-28T14:06:08.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.490+0000 7f0821e00640 1 --2- 172.21.15.42:0/2366329843 >> v2:172.21.15.63:3300/0 conn(0x7f081c06f660 0x7f081c073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.494+0000 7f0822800640 1 --2- 172.21.15.42:0/2366329843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f081c0d0e00 0x7f081c06dd60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f081c0d13a0 tx=0x7f081801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1328d9ccf1433190 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:08.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.494+0000 7f0820a00640 1 -- 172.21.15.42:0/2366329843 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08180a5740 con 0x7f081c0d0e00 2024-05-28T14:06:08.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.494+0000 7f0820a00640 1 -- 172.21.15.42:0/2366329843 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08180a58e0 con 0x7f081c0d0e00 2024-05-28T14:06:08.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.494+0000 7f0820a00640 1 -- 172.21.15.42:0/2366329843 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08180a5cb0 con 0x7f081c0d0e00 2024-05-28T14:06:08.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.494+0000 7f0824800640 1 -- 172.21.15.42:0/2366329843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f081c0d0e00 msgr2=0x7f081c06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:08.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.494+0000 7f0824800640 1 --2- 172.21.15.42:0/2366329843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f081c0d0e00 0x7f081c06dd60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f081c0d13a0 tx=0x7f081801c8a0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.494+0000 7f0824800640 1 -- 172.21.15.42:0/2366329843 shutdown_connections 2024-05-28T14:06:08.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.494+0000 7f0824800640 1 --2- 172.21.15.42:0/2366329843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f081c0d0e00 0x7f081c06dd60 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.494+0000 7f0824800640 1 --2- 172.21.15.42:0/2366329843 >> v2:172.21.15.63:3300/0 conn(0x7f081c06f660 0x7f081c073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.494+0000 7f0824800640 1 -- 172.21.15.42:0/2366329843 >> 172.21.15.42:0/2366329843 conn(0x7f081c06bdf0 msgr2=0x7f081c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:08.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.494+0000 7f0824800640 1 -- 172.21.15.42:0/2366329843 shutdown_connections 2024-05-28T14:06:08.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.494+0000 7f0824800640 1 -- 172.21.15.42:0/2366329843 wait complete. 2024-05-28T14:06:08.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.494+0000 7f0824800640 1 Processor -- start 2024-05-28T14:06:08.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.494+0000 7f0824800640 1 -- start start 2024-05-28T14:06:08.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0824800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f081c06f660 0x7f081c148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0824800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c0d0e00 0x7f081c155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0824800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f081c155680 0x7f081c151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0824800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f081c168550 con 0x7f081c155680 2024-05-28T14:06:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0824800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f081c1683d0 con 0x7f081c0d0e00 2024-05-28T14:06:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0824800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f081c1686d0 con 0x7f081c06f660 2024-05-28T14:06:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0822800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f081c155680 0x7f081c151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0822800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f081c155680 0x7f081c151ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47572/0 (socket says 172.21.15.42:47572) 2024-05-28T14:06:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0822800640 1 -- 172.21.15.42:0/279201158 learned_addr learned my addr 172.21.15.42:0/279201158 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0821e00640 1 --2- 172.21.15.42:0/279201158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f081c06f660 0x7f081c148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0821400640 1 --2- 172.21.15.42:0/279201158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c0d0e00 0x7f081c155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0822800640 1 -- 172.21.15.42:0/279201158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f081c06f660 msgr2=0x7f081c148fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:08.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0822800640 1 --2- 172.21.15.42:0/279201158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f081c06f660 0x7f081c148fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0822800640 1 -- 172.21.15.42:0/279201158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c0d0e00 msgr2=0x7f081c155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:08.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0822800640 1 --2- 172.21.15.42:0/279201158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c0d0e00 0x7f081c155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0822800640 1 -- 172.21.15.42:0/279201158 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c152b00 con 0x7f081c155680 2024-05-28T14:06:08.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0822800640 1 --2- 172.21.15.42:0/279201158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f081c155680 0x7f081c151ef0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f08180af040 tx=0x7f0818019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:08.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0816000640 1 -- 172.21.15.42:0/279201158 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08180bc030 con 0x7f081c155680 2024-05-28T14:06:08.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0816000640 1 -- 172.21.15.42:0/279201158 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08180b8470 con 0x7f081c155680 2024-05-28T14:06:08.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0816000640 1 -- 172.21.15.42:0/279201158 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08180b8840 con 0x7f081c155680 2024-05-28T14:06:08.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0824800640 1 -- 172.21.15.42:0/279201158 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081c152de0 con 0x7f081c155680 2024-05-28T14:06:08.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0824800640 1 -- 172.21.15.42:0/279201158 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f081c1533a0 con 0x7f081c155680 2024-05-28T14:06:08.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0824800640 1 -- 172.21.15.42:0/279201158 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f081c068a90 con 0x7f081c155680 2024-05-28T14:06:08.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.498+0000 7f0816000640 1 -- 172.21.15.42:0/279201158 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f08180bf070 con 0x7f081c155680 2024-05-28T14:06:08.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.502+0000 7f0816000640 1 --2- 172.21.15.42:0/279201158 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0800069070 0x7f080006b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:08.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.502+0000 7f0816000640 1 -- 172.21.15.42:0/279201158 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5765+0+0 (secure 0 0 0) 0x7f08181204d0 con 0x7f081c155680 2024-05-28T14:06:08.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.502+0000 7f0821e00640 1 --2- 172.21.15.42:0/279201158 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0800069070 0x7f080006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:08.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.502+0000 7f0821e00640 1 --2- 172.21.15.42:0/279201158 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0800069070 0x7f080006b530 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f081c155f00 tx=0x7f0810008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:08.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.506+0000 7f0816000640 1 -- 172.21.15.42:0/279201158 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f08180c6070 con 0x7f081c155680 2024-05-28T14:06:08.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.706+0000 7f0824800640 1 -- 172.21.15.42:0/279201158 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f081c072bb0 con 0x7f081c155680 2024-05-28T14:06:08.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.706+0000 7f0816000640 1 -- 172.21.15.42:0/279201158 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5178 (secure 0 0 0) 0x7f08180f8070 con 0x7f081c155680 2024-05-28T14:06:08.695 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:08.695 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1310016671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1310016671},{"type":"v1","addr":"172.21.15.63:6834","nonce":1310016671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1602237641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1602237641},{"type":"v1","addr":"172.21.15.42:6835","nonce":1602237641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/142807607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":142807607},{"type":"v1","addr":"172.21.15.42:6837","nonce":142807607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:04:22.960438+0000","modified":"2024-05-28T14:04:25.983153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3311861302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3311861302},{"type":"v1","addr":"172.21.15.63:6835","nonce":3311861302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6420,"qdb_cluster":[6420]},"id":4}]} 2024-05-28T14:06:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 37 2024-05-28T14:06:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.710+0000 7f0824800640 1 -- 172.21.15.42:0/279201158 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0800069070 msgr2=0x7f080006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.710+0000 7f0824800640 1 --2- 172.21.15.42:0/279201158 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0800069070 0x7f080006b530 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f081c155f00 tx=0x7f0810008040 comp rx=0 tx=0).stop 2024-05-28T14:06:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.710+0000 7f0824800640 1 -- 172.21.15.42:0/279201158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f081c155680 msgr2=0x7f081c151ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.710+0000 7f0824800640 1 --2- 172.21.15.42:0/279201158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f081c155680 0x7f081c151ef0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f08180af040 tx=0x7f0818019640 comp rx=0 tx=0).stop 2024-05-28T14:06:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.710+0000 7f0824800640 1 -- 172.21.15.42:0/279201158 shutdown_connections 2024-05-28T14:06:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.710+0000 7f0824800640 1 --2- 172.21.15.42:0/279201158 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0800069070 0x7f080006b530 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.710+0000 7f0824800640 1 --2- 172.21.15.42:0/279201158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f081c155680 0x7f081c151ef0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.710+0000 7f0824800640 1 --2- 172.21.15.42:0/279201158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c0d0e00 0x7f081c155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.710+0000 7f0824800640 1 --2- 172.21.15.42:0/279201158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f081c06f660 0x7f081c148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.710+0000 7f0824800640 1 -- 172.21.15.42:0/279201158 >> 172.21.15.42:0/279201158 conn(0x7f081c06bdf0 msgr2=0x7f081c14f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.710+0000 7f0824800640 1 -- 172.21.15.42:0/279201158 shutdown_connections 2024-05-28T14:06:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.710+0000 7f0824800640 1 -- 172.21.15.42:0/279201158 wait complete. 2024-05-28T14:06:08.717 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:06:08.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.870+0000 7f56d8800640 1 Processor -- start 2024-05-28T14:06:08.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.870+0000 7f56d8800640 1 -- start start 2024-05-28T14:06:08.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.870+0000 7f56d8800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d0156a70 0x7f56d006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:08.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.870+0000 7f56d8800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f56d0156e50 0x7f56d00af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:08.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.870+0000 7f56d8800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d015cb90 con 0x7f56d00af880 2024-05-28T14:06:08.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.870+0000 7f56d8800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f56d015ce50 con 0x7f56d0156e50 2024-05-28T14:06:08.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.870+0000 7f56d8800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f56d015d130 con 0x7f56d0156a70 2024-05-28T14:06:08.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.874+0000 7f56d5e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d0156a70 0x7f56d006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:08.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.874+0000 7f56d5400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f56d0156e50 0x7f56d00af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:08.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.874+0000 7f56d5e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d0156a70 0x7f56d006dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:37292/0 (socket says 172.21.15.42:37292) 2024-05-28T14:06:08.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.874+0000 7f56d6800640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f56d00af880 0x7f56d00d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:39524/0 (socket says 172.21.15.42:39524) 2024-05-28T14:06:08.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.874+0000 7f56d5e00640 1 -- 172.21.15.42:0/1224816284 learned_addr learned my addr 172.21.15.42:0/1224816284 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:08.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.874+0000 7f56d5e00640 1 -- 172.21.15.42:0/1224816284 >> v2:172.21.15.63:3300/0 conn(0x7f56d0156e50 msgr2=0x7f56d00af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:08.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.874+0000 7f56d5e00640 1 --2- 172.21.15.42:0/1224816284 >> v2:172.21.15.63:3300/0 conn(0x7f56d0156e50 0x7f56d00af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.874+0000 7f56d5e00640 1 -- 172.21.15.42:0/1224816284 >> v1:172.21.15.63:6789/0 conn(0x7f56d00af880 legacy=0x7f56d00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:08.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.874+0000 7f56d5e00640 1 -- 172.21.15.42:0/1224816284 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d015d410 con 0x7f56d0156a70 2024-05-28T14:06:08.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.874+0000 7f56d5e00640 1 --2- 172.21.15.42:0/1224816284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d0156a70 0x7f56d006dd60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009870 tx=0x7f56c401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86cb12f0e166e0d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:08.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d4a00640 1 -- 172.21.15.42:0/1224816284 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c40a7070 con 0x7f56d0156a70 2024-05-28T14:06:08.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d4a00640 1 -- 172.21.15.42:0/1224816284 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c40b0070 con 0x7f56d0156a70 2024-05-28T14:06:08.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d4a00640 1 -- 172.21.15.42:0/1224816284 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c40ab890 con 0x7f56d0156a70 2024-05-28T14:06:08.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 -- 172.21.15.42:0/1224816284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d0156a70 msgr2=0x7f56d006dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:08.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 --2- 172.21.15.42:0/1224816284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d0156a70 0x7f56d006dd60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009870 tx=0x7f56c401c560 comp rx=0 tx=0).stop 2024-05-28T14:06:08.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 -- 172.21.15.42:0/1224816284 shutdown_connections 2024-05-28T14:06:08.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 --2- 172.21.15.42:0/1224816284 >> v2:172.21.15.63:3300/0 conn(0x7f56d0156e50 0x7f56d00af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 --2- 172.21.15.42:0/1224816284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d0156a70 0x7f56d006dd60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 -- 172.21.15.42:0/1224816284 >> 172.21.15.42:0/1224816284 conn(0x7f56d006bdf0 msgr2=0x7f56d006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:08.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 -- 172.21.15.42:0/1224816284 shutdown_connections 2024-05-28T14:06:08.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 -- 172.21.15.42:0/1224816284 wait complete. 2024-05-28T14:06:08.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 Processor -- start 2024-05-28T14:06:08.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 -- start start 2024-05-28T14:06:08.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56d00af880 0x7f56d015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:08.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56d0156a70 0x7f56d015f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:08.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d0156e50 0x7f56d01fbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:08.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0160b40 con 0x7f56d0156a70 2024-05-28T14:06:08.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56d00d1010 con 0x7f56d00af880 2024-05-28T14:06:08.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d8800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f56d0160cc0 con 0x7f56d0156e50 2024-05-28T14:06:08.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56d0156a70 0x7f56d015f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:08.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56d0156a70 0x7f56d015f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47584/0 (socket says 172.21.15.42:47584) 2024-05-28T14:06:08.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.878+0000 7f56d5400640 1 -- 172.21.15.42:0/2841268643 learned_addr learned my addr 172.21.15.42:0/2841268643 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:08.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56d5e00640 1 --2- 172.21.15.42:0/2841268643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56d00af880 0x7f56d015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:08.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56d6800640 1 --2- 172.21.15.42:0/2841268643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d0156e50 0x7f56d01fbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:08.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56d5400640 1 -- 172.21.15.42:0/2841268643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d0156e50 msgr2=0x7f56d01fbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:08.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56d5400640 1 --2- 172.21.15.42:0/2841268643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d0156e50 0x7f56d01fbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56d5400640 1 -- 172.21.15.42:0/2841268643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56d00af880 msgr2=0x7f56d015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:08.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56d5400640 1 --2- 172.21.15.42:0/2841268643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56d00af880 0x7f56d015ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:08.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56d5400640 1 -- 172.21.15.42:0/2841268643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d01fc8e0 con 0x7f56d0156a70 2024-05-28T14:06:08.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56d5400640 1 --2- 172.21.15.42:0/2841268643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56d0156a70 0x7f56d015f2c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f56c000e8a0 tx=0x7f56c000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:08.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56ca000640 1 -- 172.21.15.42:0/2841268643 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0017070 con 0x7f56d0156a70 2024-05-28T14:06:08.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56ca000640 1 -- 172.21.15.42:0/2841268643 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c000c940 con 0x7f56d0156a70 2024-05-28T14:06:08.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56ca000640 1 -- 172.21.15.42:0/2841268643 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c000ccf0 con 0x7f56d0156a70 2024-05-28T14:06:08.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56d8800640 1 -- 172.21.15.42:0/2841268643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d01fcbc0 con 0x7f56d0156a70 2024-05-28T14:06:08.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56d8800640 1 -- 172.21.15.42:0/2841268643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d01fd180 con 0x7f56d0156a70 2024-05-28T14:06:08.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56d8800640 1 -- 172.21.15.42:0/2841268643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d0068a90 con 0x7f56d0156a70 2024-05-28T14:06:08.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56ca000640 1 -- 172.21.15.42:0/2841268643 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f56c001a070 con 0x7f56d0156a70 2024-05-28T14:06:08.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.882+0000 7f56ca000640 1 --2- 172.21.15.42:0/2841268643 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56b40690a0 0x7f56b406b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:08.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.886+0000 7f56ca000640 1 -- 172.21.15.42:0/2841268643 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5765+0+0 (secure 0 0 0) 0x7f56c00851c0 con 0x7f56d0156a70 2024-05-28T14:06:08.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.886+0000 7f56d5e00640 1 --2- 172.21.15.42:0/2841268643 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56b40690a0 0x7f56b406b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:08.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.886+0000 7f56d5e00640 1 --2- 172.21.15.42:0/2841268643 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56b40690a0 0x7f56b406b560 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f56c40af000 tx=0x7f56c40a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:08.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:08.886+0000 7f56ca000640 1 -- 172.21.15.42:0/2841268643 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f56c0056b80 con 0x7f56d0156a70 2024-05-28T14:06:09.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.054+0000 7f56d8800640 1 -- 172.21.15.42:0/2841268643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56d00b5160 con 0x7f56d0156a70 2024-05-28T14:06:09.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.054+0000 7f56ca000640 1 -- 172.21.15.42:0/2841268643 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16713 (secure 0 0 0) 0x7f56c005ae80 con 0x7f56d0156a70 2024-05-28T14:06:09.043 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:09.043 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":63,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:06:07.536649+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:04:20.721059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-28T14:04:21.922127+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":62,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:06:09.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.058+0000 7f56d8800640 1 -- 172.21.15.42:0/2841268643 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56b40690a0 msgr2=0x7f56b406b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.058+0000 7f56d8800640 1 --2- 172.21.15.42:0/2841268643 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56b40690a0 0x7f56b406b560 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f56c40af000 tx=0x7f56c40a7000 comp rx=0 tx=0).stop 2024-05-28T14:06:09.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.058+0000 7f56d8800640 1 -- 172.21.15.42:0/2841268643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56d0156a70 msgr2=0x7f56d015f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.058+0000 7f56d8800640 1 --2- 172.21.15.42:0/2841268643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56d0156a70 0x7f56d015f2c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f56c000e8a0 tx=0x7f56c000eca0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.058+0000 7f56d8800640 1 -- 172.21.15.42:0/2841268643 shutdown_connections 2024-05-28T14:06:09.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.058+0000 7f56d8800640 1 --2- 172.21.15.42:0/2841268643 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56b40690a0 0x7f56b406b560 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.058+0000 7f56d8800640 1 --2- 172.21.15.42:0/2841268643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d0156e50 0x7f56d01fbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.058+0000 7f56d8800640 1 --2- 172.21.15.42:0/2841268643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56d0156a70 0x7f56d015f2c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.058+0000 7f56d8800640 1 --2- 172.21.15.42:0/2841268643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56d00af880 0x7f56d015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.058+0000 7f56d8800640 1 -- 172.21.15.42:0/2841268643 >> 172.21.15.42:0/2841268643 conn(0x7f56d006bdf0 msgr2=0x7f56d00b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:09.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.058+0000 7f56d8800640 1 -- 172.21.15.42:0/2841268643 shutdown_connections 2024-05-28T14:06:09.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.058+0000 7f56d8800640 1 -- 172.21.15.42:0/2841268643 wait complete. 2024-05-28T14:06:09.065 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T14:04:20.721059+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T14:04:21.922127+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 62, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:06:09.065 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:06:09.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48db800640 1 Processor -- start 2024-05-28T14:06:09.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48db800640 1 -- start start 2024-05-28T14:06:09.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48db800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f48d409bc60 0x7f48d40af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48db800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48d40d10a0 0x7f48d40af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48db800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48d40cddd0 con 0x7f48d415fd70 2024-05-28T14:06:09.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48db800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f48d40ce0b0 con 0x7f48d409bc60 2024-05-28T14:06:09.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48db800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f48d40ce390 con 0x7f48d40d10a0 2024-05-28T14:06:09.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48d9800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48d40d10a0 0x7f48d40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48cbe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f48d409bc60 0x7f48d40af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48d8e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f48d415fd70 0x7f48d40d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:39540/0 (socket says 172.21.15.42:39540) 2024-05-28T14:06:09.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48d9800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48d40d10a0 0x7f48d40af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:37310/0 (socket says 172.21.15.42:37310) 2024-05-28T14:06:09.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48d8e00640 1 -- 172.21.15.42:0/2262817540 learned_addr learned my addr 172.21.15.42:0/2262817540 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:09.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48cbe00640 1 -- 172.21.15.42:0/2262817540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48d40d10a0 msgr2=0x7f48d40af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:09.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48cbe00640 1 --2- 172.21.15.42:0/2262817540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48d40d10a0 0x7f48d40af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48cbe00640 1 -- 172.21.15.42:0/2262817540 >> v1:172.21.15.63:6789/0 conn(0x7f48d415fd70 legacy=0x7f48d40d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.222+0000 7f48cbe00640 1 -- 172.21.15.42:0/2262817540 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d40ce670 con 0x7f48d409bc60 2024-05-28T14:06:09.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.226+0000 7f48cbe00640 1 --2- 172.21.15.42:0/2262817540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d409bc60 0x7f48d40af3d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0097c0 tx=0x7f48bc01c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c686b17bcfc4dc53 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:09.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.226+0000 7f48d3e00640 1 -- 172.21.15.42:0/2262817540 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc0b0070 con 0x7f48d409bc60 2024-05-28T14:06:09.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.226+0000 7f48d3e00640 1 -- 172.21.15.42:0/2262817540 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48bc0ab600 con 0x7f48d409bc60 2024-05-28T14:06:09.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.226+0000 7f48d3e00640 1 -- 172.21.15.42:0/2262817540 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc0ab9d0 con 0x7f48d409bc60 2024-05-28T14:06:09.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.226+0000 7f48db800640 1 -- 172.21.15.42:0/2262817540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d409bc60 msgr2=0x7f48d40af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.226+0000 7f48db800640 1 --2- 172.21.15.42:0/2262817540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d409bc60 0x7f48d40af3d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0097c0 tx=0x7f48bc01c380 comp rx=0 tx=0).stop 2024-05-28T14:06:09.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.226+0000 7f48db800640 1 -- 172.21.15.42:0/2262817540 shutdown_connections 2024-05-28T14:06:09.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.226+0000 7f48db800640 1 --2- 172.21.15.42:0/2262817540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48d40d10a0 0x7f48d40af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.226+0000 7f48db800640 1 --2- 172.21.15.42:0/2262817540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d409bc60 0x7f48d40af3d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.226+0000 7f48db800640 1 -- 172.21.15.42:0/2262817540 >> 172.21.15.42:0/2262817540 conn(0x7f48d40b4050 msgr2=0x7f48d40b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:09.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.226+0000 7f48db800640 1 -- 172.21.15.42:0/2262817540 shutdown_connections 2024-05-28T14:06:09.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.226+0000 7f48db800640 1 -- 172.21.15.42:0/2262817540 wait complete. 2024-05-28T14:06:09.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48db800640 1 Processor -- start 2024-05-28T14:06:09.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48db800640 1 -- start start 2024-05-28T14:06:09.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48db800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48d409bc60 0x7f48d41555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48db800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48d40d10a0 0x7f48d4156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48db800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d415fd70 0x7f48d4166160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48db800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41602d0 con 0x7f48d409bc60 2024-05-28T14:06:09.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48db800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4160150 con 0x7f48d415fd70 2024-05-28T14:06:09.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48db800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f48d40b1410 con 0x7f48d40d10a0 2024-05-28T14:06:09.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48d8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48d409bc60 0x7f48d41555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48d8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48d409bc60 0x7f48d41555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47600/0 (socket says 172.21.15.42:47600) 2024-05-28T14:06:09.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48d8e00640 1 -- 172.21.15.42:0/195496678 learned_addr learned my addr 172.21.15.42:0/195496678 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:09.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48cbe00640 1 --2- 172.21.15.42:0/195496678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48d40d10a0 0x7f48d4156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48d9800640 1 --2- 172.21.15.42:0/195496678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d415fd70 0x7f48d4166160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48d8e00640 1 -- 172.21.15.42:0/195496678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48d40d10a0 msgr2=0x7f48d4156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48d8e00640 1 --2- 172.21.15.42:0/195496678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48d40d10a0 0x7f48d4156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48d8e00640 1 -- 172.21.15.42:0/195496678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d415fd70 msgr2=0x7f48d4166160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48d8e00640 1 --2- 172.21.15.42:0/195496678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d415fd70 0x7f48d4166160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48d8e00640 1 -- 172.21.15.42:0/195496678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d4157480 con 0x7f48d409bc60 2024-05-28T14:06:09.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48d8e00640 1 --2- 172.21.15.42:0/195496678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48d409bc60 0x7f48d41555d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c370 tx=0x7f48c400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:09.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48d1600640 1 -- 172.21.15.42:0/195496678 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4017070 con 0x7f48d409bc60 2024-05-28T14:06:09.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48d1600640 1 -- 172.21.15.42:0/195496678 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c4005cf0 con 0x7f48d409bc60 2024-05-28T14:06:09.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48db800640 1 -- 172.21.15.42:0/195496678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d415a510 con 0x7f48d409bc60 2024-05-28T14:06:09.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48d1600640 1 -- 172.21.15.42:0/195496678 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40072f0 con 0x7f48d409bc60 2024-05-28T14:06:09.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.230+0000 7f48db800640 1 -- 172.21.15.42:0/195496678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d415aad0 con 0x7f48d409bc60 2024-05-28T14:06:09.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.234+0000 7f48db800640 1 -- 172.21.15.42:0/195496678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d40a0650 con 0x7f48d409bc60 2024-05-28T14:06:09.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.234+0000 7f48d1600640 1 -- 172.21.15.42:0/195496678 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f48c401a070 con 0x7f48d409bc60 2024-05-28T14:06:09.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.234+0000 7f48d1600640 1 --2- 172.21.15.42:0/195496678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f48b0069070 0x7f48b006b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.234+0000 7f48d1600640 1 -- 172.21.15.42:0/195496678 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5765+0+0 (secure 0 0 0) 0x7f48c4084730 con 0x7f48d409bc60 2024-05-28T14:06:09.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.234+0000 7f48cbe00640 1 --2- 172.21.15.42:0/195496678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f48b0069070 0x7f48b006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.234+0000 7f48cbe00640 1 --2- 172.21.15.42:0/195496678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f48b0069070 0x7f48b006b530 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0ad000 tx=0x7f48bc0a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:09.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.238+0000 7f48d1600640 1 -- 172.21.15.42:0/195496678 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f48d4157480 con 0x7f48d409bc60 2024-05-28T14:06:09.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.402+0000 7f48db800640 1 -- 172.21.15.42:0/195496678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f48d40d0560 con 0x7f48d409bc60 2024-05-28T14:06:09.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.402+0000 7f48d1600640 1 -- 172.21.15.42:0/195496678 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) ==== 93+0+10 (secure 0 0 0) 0x7f48c4056060 con 0x7f48d409bc60 2024-05-28T14:06:09.390 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:06:09.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.406+0000 7f48db800640 1 -- 172.21.15.42:0/195496678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f48b0069070 msgr2=0x7f48b006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.406+0000 7f48db800640 1 --2- 172.21.15.42:0/195496678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f48b0069070 0x7f48b006b530 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0ad000 tx=0x7f48bc0a6000 comp rx=0 tx=0).stop 2024-05-28T14:06:09.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.406+0000 7f48db800640 1 -- 172.21.15.42:0/195496678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48d409bc60 msgr2=0x7f48d41555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.406+0000 7f48db800640 1 --2- 172.21.15.42:0/195496678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48d409bc60 0x7f48d41555d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c370 tx=0x7f48c400c830 comp rx=0 tx=0).stop 2024-05-28T14:06:09.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.406+0000 7f48db800640 1 -- 172.21.15.42:0/195496678 shutdown_connections 2024-05-28T14:06:09.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.406+0000 7f48db800640 1 --2- 172.21.15.42:0/195496678 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f48b0069070 0x7f48b006b530 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.406+0000 7f48db800640 1 --2- 172.21.15.42:0/195496678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d415fd70 0x7f48d4166160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.406+0000 7f48db800640 1 --2- 172.21.15.42:0/195496678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48d40d10a0 0x7f48d4156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.406+0000 7f48db800640 1 --2- 172.21.15.42:0/195496678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48d409bc60 0x7f48d41555d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.406+0000 7f48db800640 1 -- 172.21.15.42:0/195496678 >> 172.21.15.42:0/195496678 conn(0x7f48d40b4050 msgr2=0x7f48d415c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:09.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.406+0000 7f48db800640 1 -- 172.21.15.42:0/195496678 shutdown_connections 2024-05-28T14:06:09.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.406+0000 7f48db800640 1 -- 172.21.15.42:0/195496678 wait complete. 2024-05-28T14:06:09.411 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:06:09.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.566+0000 7f0ea1c00640 1 Processor -- start 2024-05-28T14:06:09.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.566+0000 7f0ea1c00640 1 -- start start 2024-05-28T14:06:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.570+0000 7f0ea1c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0e9c0b2240 0x7f0e9c06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.570+0000 7f0ea1c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0e9c06bf30 0x7f0e9c0af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.570+0000 7f0ea1c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e9c0b4440 con 0x7f0e9c15e8c0 2024-05-28T14:06:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.570+0000 7f0ea1c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0e9c0ae420 con 0x7f0e9c06bf30 2024-05-28T14:06:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.570+0000 7f0ea1c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0e9c06c370 con 0x7f0e9c0b2240 2024-05-28T14:06:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.570+0000 7f0e9b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0e9c06bf30 0x7f0e9c0af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.570+0000 7f0e9b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0e9c06bf30 0x7f0e9c0af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47614/0 (socket says 172.21.15.42:47614) 2024-05-28T14:06:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.570+0000 7f0e9b400640 1 -- 172.21.15.42:0/1482248054 learned_addr learned my addr 172.21.15.42:0/1482248054 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.570+0000 7f0e92000640 1 --2- 172.21.15.42:0/1482248054 >> v2:172.21.15.63:3300/0 conn(0x7f0e9c0b2240 0x7f0e9c06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.570+0000 7f0e9b400640 1 -- 172.21.15.42:0/1482248054 >> v2:172.21.15.63:3300/0 conn(0x7f0e9c0b2240 msgr2=0x7f0e9c06b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.570+0000 7f0e9b400640 1 --2- 172.21.15.42:0/1482248054 >> v2:172.21.15.63:3300/0 conn(0x7f0e9c0b2240 0x7f0e9c06b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.570+0000 7f0e9b400640 1 -- 172.21.15.42:0/1482248054 >> v1:172.21.15.42:6789/0 conn(0x7f0e9c15e8c0 legacy=0x7f0e9c0b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.570+0000 7f0e9b400640 1 -- 172.21.15.42:0/1482248054 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c0b0f10 con 0x7f0e9c06bf30 2024-05-28T14:06:09.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.574+0000 7f0e9b400640 1 --2- 172.21.15.42:0/1482248054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e9c06bf30 0x7f0e9c0af150 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c068290 tx=0x7f0e9401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67e21286b70ab4d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.574+0000 7f0e9a000640 1 -- 172.21.15.42:0/1482248054 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e940a5620 con 0x7f0e9c06bf30 2024-05-28T14:06:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.574+0000 7f0e9a000640 1 -- 172.21.15.42:0/1482248054 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e940a57c0 con 0x7f0e9c06bf30 2024-05-28T14:06:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.574+0000 7f0e9a000640 1 -- 172.21.15.42:0/1482248054 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e940a5b90 con 0x7f0e9c06bf30 2024-05-28T14:06:09.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.574+0000 7f0ea1c00640 1 -- 172.21.15.42:0/1482248054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e9c06bf30 msgr2=0x7f0e9c0af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.574+0000 7f0ea1c00640 1 --2- 172.21.15.42:0/1482248054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e9c06bf30 0x7f0e9c0af150 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c068290 tx=0x7f0e9401c960 comp rx=0 tx=0).stop 2024-05-28T14:06:09.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.574+0000 7f0ea1c00640 1 -- 172.21.15.42:0/1482248054 shutdown_connections 2024-05-28T14:06:09.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.574+0000 7f0ea1c00640 1 --2- 172.21.15.42:0/1482248054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e9c06bf30 0x7f0e9c0af150 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.574+0000 7f0ea1c00640 1 --2- 172.21.15.42:0/1482248054 >> v2:172.21.15.63:3300/0 conn(0x7f0e9c0b2240 0x7f0e9c06b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.574+0000 7f0ea1c00640 1 -- 172.21.15.42:0/1482248054 >> 172.21.15.42:0/1482248054 conn(0x7f0e9c0b6960 msgr2=0x7f0e9c0b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:09.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.574+0000 7f0ea1c00640 1 -- 172.21.15.42:0/1482248054 shutdown_connections 2024-05-28T14:06:09.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.574+0000 7f0ea1c00640 1 -- 172.21.15.42:0/1482248054 wait complete. 2024-05-28T14:06:09.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.574+0000 7f0ea1c00640 1 Processor -- start 2024-05-28T14:06:09.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.574+0000 7f0ea1c00640 1 -- start start 2024-05-28T14:06:09.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0ea1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e9c06bf30 0x7f0e9c161b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0ea1c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e9c0b2240 0x7f0e9c155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0ea1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e9c15e8c0 0x7f0e9c14f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0ea1c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c15ee20 con 0x7f0e9c0b2240 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0ea1c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c15eca0 con 0x7f0e9c06bf30 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0ea1c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9c0afea0 con 0x7f0e9c15e8c0 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e92000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e9c0b2240 0x7f0e9c155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e92000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e9c0b2240 0x7f0e9c155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47616/0 (socket says 172.21.15.42:47616) 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e92000640 1 -- 172.21.15.42:0/3916171258 learned_addr learned my addr 172.21.15.42:0/3916171258 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e9aa00640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e9c06bf30 0x7f0e9c161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e9b400640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e9c15e8c0 0x7f0e9c14f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e92000640 1 -- 172.21.15.42:0/3916171258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e9c15e8c0 msgr2=0x7f0e9c14f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e92000640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e9c15e8c0 0x7f0e9c14f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e92000640 1 -- 172.21.15.42:0/3916171258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e9c06bf30 msgr2=0x7f0e9c161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e92000640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e9c06bf30 0x7f0e9c161b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e92000640 1 -- 172.21.15.42:0/3916171258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c158970 con 0x7f0e9c0b2240 2024-05-28T14:06:09.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e9aa00640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e9c06bf30 0x7f0e9c161b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:09.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e92000640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e9c0b2240 0x7f0e9c155250 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400e960 tx=0x7f0e8400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:09.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e93400640 1 -- 172.21.15.42:0/3916171258 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84017070 con 0x7f0e9c0b2240 2024-05-28T14:06:09.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e93400640 1 -- 172.21.15.42:0/3916171258 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e8400ca00 con 0x7f0e9c0b2240 2024-05-28T14:06:09.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e93400640 1 -- 172.21.15.42:0/3916171258 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8400cdb0 con 0x7f0e9c0b2240 2024-05-28T14:06:09.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0ea1c00640 1 -- 172.21.15.42:0/3916171258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9c155ca0 con 0x7f0e9c0b2240 2024-05-28T14:06:09.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0ea1c00640 1 -- 172.21.15.42:0/3916171258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9c158ef0 con 0x7f0e9c0b2240 2024-05-28T14:06:09.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0ea1c00640 1 -- 172.21.15.42:0/3916171258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e9c155fe0 con 0x7f0e9c0b2240 2024-05-28T14:06:09.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.578+0000 7f0e93400640 1 -- 172.21.15.42:0/3916171258 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f0e8401a070 con 0x7f0e9c0b2240 2024-05-28T14:06:09.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.582+0000 7f0e93400640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e78069070 0x7f0e7806b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.582+0000 7f0e93400640 1 -- 172.21.15.42:0/3916171258 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5765+0+0 (secure 0 0 0) 0x7f0e84085350 con 0x7f0e9c0b2240 2024-05-28T14:06:09.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.582+0000 7f0e9aa00640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e78069070 0x7f0e7806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.582+0000 7f0e9aa00640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e78069070 0x7f0e7806b530 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c001670 tx=0x7f0e8c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:09.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.586+0000 7f0e93400640 1 -- 172.21.15.42:0/3916171258 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0e84056d10 con 0x7f0e9c0b2240 2024-05-28T14:06:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.750+0000 7f0ea1c00640 1 -- 172.21.15.42:0/3916171258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0e9c09f7a0 con 0x7f0e9c0b2240 2024-05-28T14:06:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.750+0000 7f0e93400640 1 -- 172.21.15.42:0/3916171258 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) ==== 104+0+11 (secure 0 0 0) 0x7f0e8405b010 con 0x7f0e9c0b2240 2024-05-28T14:06:09.738 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:06:09.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.754+0000 7f0ea1c00640 1 -- 172.21.15.42:0/3916171258 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e78069070 msgr2=0x7f0e7806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.754+0000 7f0ea1c00640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e78069070 0x7f0e7806b530 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c001670 tx=0x7f0e8c009040 comp rx=0 tx=0).stop 2024-05-28T14:06:09.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.754+0000 7f0ea1c00640 1 -- 172.21.15.42:0/3916171258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e9c0b2240 msgr2=0x7f0e9c155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.754+0000 7f0ea1c00640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e9c0b2240 0x7f0e9c155250 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400e960 tx=0x7f0e8400ee20 comp rx=0 tx=0).stop 2024-05-28T14:06:09.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.754+0000 7f0ea1c00640 1 -- 172.21.15.42:0/3916171258 shutdown_connections 2024-05-28T14:06:09.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.754+0000 7f0ea1c00640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e78069070 0x7f0e7806b530 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.754+0000 7f0ea1c00640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e9c15e8c0 0x7f0e9c14f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.754+0000 7f0ea1c00640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e9c0b2240 0x7f0e9c155250 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.754+0000 7f0ea1c00640 1 --2- 172.21.15.42:0/3916171258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e9c06bf30 0x7f0e9c161b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.754+0000 7f0ea1c00640 1 -- 172.21.15.42:0/3916171258 >> 172.21.15.42:0/3916171258 conn(0x7f0e9c0b6960 msgr2=0x7f0e9c0b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:09.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.754+0000 7f0ea1c00640 1 -- 172.21.15.42:0/3916171258 shutdown_connections 2024-05-28T14:06:09.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.754+0000 7f0ea1c00640 1 -- 172.21.15.42:0/3916171258 wait complete. 2024-05-28T14:06:09.760 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:06:09.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.918+0000 7f999de00640 1 Processor -- start 2024-05-28T14:06:09.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.918+0000 7f999de00640 1 -- start start 2024-05-28T14:06:09.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.918+0000 7f999de00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9998144f80 0x7f9998072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.918+0000 7f999de00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99980afe40 con 0x7f999815c0c0 2024-05-28T14:06:09.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.918+0000 7f999de00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f999815c600 con 0x7f999806ee20 2024-05-28T14:06:09.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.918+0000 7f999de00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f99980724d0 con 0x7f9998144f80 2024-05-28T14:06:09.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.918+0000 7f9996a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9998144f80 0x7f9998072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.918+0000 7f9996a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9998144f80 0x7f9998072960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47630/0 (socket says 172.21.15.42:47630) 2024-05-28T14:06:09.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.918+0000 7f9996a00640 1 -- 172.21.15.42:0/736017596 learned_addr learned my addr 172.21.15.42:0/736017596 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:09.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.918+0000 7f9996a00640 1 -- 172.21.15.42:0/736017596 >> v1:172.21.15.63:6789/0 conn(0x7f999806ee20 legacy=0x7f999815b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.918+0000 7f9996a00640 1 -- 172.21.15.42:0/736017596 >> v1:172.21.15.42:6789/0 conn(0x7f999815c0c0 legacy=0x7f99980b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.918+0000 7f9996a00640 1 -- 172.21.15.42:0/736017596 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999815c8e0 con 0x7f9998144f80 2024-05-28T14:06:09.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.922+0000 7f9996a00640 1 --2- 172.21.15.42:0/736017596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9998144f80 0x7f9998072960 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9988009e50 tx=0x7f998801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d731b123a205cbfc server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.922+0000 7f9996000640 1 -- 172.21.15.42:0/736017596 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99880a5700 con 0x7f9998144f80 2024-05-28T14:06:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.922+0000 7f9996000640 1 -- 172.21.15.42:0/736017596 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99880a58a0 con 0x7f9998144f80 2024-05-28T14:06:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.922+0000 7f9996000640 1 -- 172.21.15.42:0/736017596 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99880a5c70 con 0x7f9998144f80 2024-05-28T14:06:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.922+0000 7f999de00640 1 -- 172.21.15.42:0/736017596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9998144f80 msgr2=0x7f9998072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.922+0000 7f999de00640 1 --2- 172.21.15.42:0/736017596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9998144f80 0x7f9998072960 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9988009e50 tx=0x7f998801ca30 comp rx=0 tx=0).stop 2024-05-28T14:06:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.922+0000 7f999de00640 1 -- 172.21.15.42:0/736017596 shutdown_connections 2024-05-28T14:06:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.922+0000 7f999de00640 1 --2- 172.21.15.42:0/736017596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9998144f80 0x7f9998072960 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.922+0000 7f999de00640 1 -- 172.21.15.42:0/736017596 >> 172.21.15.42:0/736017596 conn(0x7f99980b21f0 msgr2=0x7f99980b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.922+0000 7f999de00640 1 -- 172.21.15.42:0/736017596 shutdown_connections 2024-05-28T14:06:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.922+0000 7f999de00640 1 -- 172.21.15.42:0/736017596 wait complete. 2024-05-28T14:06:09.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.922+0000 7f999de00640 1 Processor -- start 2024-05-28T14:06:09.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.922+0000 7f999de00640 1 -- start start 2024-05-28T14:06:09.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f999de00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f999806ee20 0x7f9998140140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f999de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9998144f80 0x7f99980cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f999de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999815c0c0 0x7f9998159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f999de00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f999806f790 con 0x7f999815c0c0 2024-05-28T14:06:09.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f999de00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f999806f610 con 0x7f9998144f80 2024-05-28T14:06:09.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f999de00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9998169950 con 0x7f999806ee20 2024-05-28T14:06:09.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f9997400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999815c0c0 0x7f9998159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f9997400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999815c0c0 0x7f9998159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47636/0 (socket says 172.21.15.42:47636) 2024-05-28T14:06:09.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f9997400640 1 -- 172.21.15.42:0/2171510721 learned_addr learned my addr 172.21.15.42:0/2171510721 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:09.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f998e000640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9998144f80 0x7f99980cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f9996a00640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f999806ee20 0x7f9998140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f9997400640 1 -- 172.21.15.42:0/2171510721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f999806ee20 msgr2=0x7f9998140140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f9997400640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f999806ee20 0x7f9998140140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f9997400640 1 -- 172.21.15.42:0/2171510721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9998144f80 msgr2=0x7f99980cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:09.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f9997400640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9998144f80 0x7f99980cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:09.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f9997400640 1 -- 172.21.15.42:0/2171510721 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9998140720 con 0x7f999815c0c0 2024-05-28T14:06:09.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f9996a00640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f999806ee20 0x7f9998140140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:09.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f9997400640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999815c0c0 0x7f9998159c80 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f999000c370 tx=0x7f999000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:09.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f998fe00640 1 -- 172.21.15.42:0/2171510721 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990005220 con 0x7f999815c0c0 2024-05-28T14:06:09.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f998fe00640 1 -- 172.21.15.42:0/2171510721 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99900053c0 con 0x7f999815c0c0 2024-05-28T14:06:09.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f999de00640 1 -- 172.21.15.42:0/2171510721 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9998140a00 con 0x7f999815c0c0 2024-05-28T14:06:09.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f998fe00640 1 -- 172.21.15.42:0/2171510721 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990007270 con 0x7f999815c0c0 2024-05-28T14:06:09.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f999de00640 1 -- 172.21.15.42:0/2171510721 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99980a9650 con 0x7f999815c0c0 2024-05-28T14:06:09.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.926+0000 7f999de00640 1 -- 172.21.15.42:0/2171510721 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99980738b0 con 0x7f999815c0c0 2024-05-28T14:06:09.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.930+0000 7f998fe00640 1 -- 172.21.15.42:0/2171510721 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f9990019020 con 0x7f999815c0c0 2024-05-28T14:06:09.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.930+0000 7f998fe00640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99740690a0 0x7f997406b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:09.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.930+0000 7f998fe00640 1 -- 172.21.15.42:0/2171510721 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5765+0+0 (secure 0 0 0) 0x7f99900845b0 con 0x7f999815c0c0 2024-05-28T14:06:09.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.930+0000 7f9996a00640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99740690a0 0x7f997406b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:09.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.930+0000 7f9996a00640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99740690a0 0x7f997406b560 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f99880ae000 tx=0x7f99880a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:09.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:09.934+0000 7f998fe00640 1 -- 172.21.15.42:0/2171510721 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9990056620 con 0x7f999815c0c0 2024-05-28T14:06:10.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.106+0000 7f999de00640 1 -- 172.21.15.42:0/2171510721 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f99980a9820 con 0x7f999815c0c0 2024-05-28T14:06:10.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.106+0000 7f998fe00640 1 -- 172.21.15.42:0/2171510721 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) ==== 100+0+11 (secure 0 0 0) 0x7f9990055f20 con 0x7f999815c0c0 2024-05-28T14:06:10.092 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:06:10.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.106+0000 7f999de00640 1 -- 172.21.15.42:0/2171510721 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99740690a0 msgr2=0x7f997406b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.106+0000 7f999de00640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99740690a0 0x7f997406b560 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f99880ae000 tx=0x7f99880a7070 comp rx=0 tx=0).stop 2024-05-28T14:06:10.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.106+0000 7f999de00640 1 -- 172.21.15.42:0/2171510721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999815c0c0 msgr2=0x7f9998159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.106+0000 7f999de00640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999815c0c0 0x7f9998159c80 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f999000c370 tx=0x7f999000c830 comp rx=0 tx=0).stop 2024-05-28T14:06:10.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.106+0000 7f999de00640 1 -- 172.21.15.42:0/2171510721 shutdown_connections 2024-05-28T14:06:10.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.106+0000 7f999de00640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99740690a0 0x7f997406b560 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.106+0000 7f999de00640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999815c0c0 0x7f9998159c80 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.106+0000 7f999de00640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9998144f80 0x7f99980cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.106+0000 7f999de00640 1 --2- 172.21.15.42:0/2171510721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f999806ee20 0x7f9998140140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.106+0000 7f999de00640 1 -- 172.21.15.42:0/2171510721 >> 172.21.15.42:0/2171510721 conn(0x7f99980b21f0 msgr2=0x7f99980b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:10.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.106+0000 7f999de00640 1 -- 172.21.15.42:0/2171510721 shutdown_connections 2024-05-28T14:06:10.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.106+0000 7f999de00640 1 -- 172.21.15.42:0/2171510721 wait complete. 2024-05-28T14:06:10.114 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:06:10.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ec3800640 1 Processor -- start 2024-05-28T14:06:10.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ec3800640 1 -- start start 2024-05-28T14:06:10.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ec3800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7ebc0ce1b0 0x7f7ebc1672e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ec3800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ebc0cfd80 con 0x7f7ebc15a460 2024-05-28T14:06:10.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ec3800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ebc15cea0 con 0x7f7ebc06c390 2024-05-28T14:06:10.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ec3800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7ebc15cc60 con 0x7f7ebc0ce1b0 2024-05-28T14:06:10.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ec0e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f7ebc15a460 0x7f7ebc0cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47762/0 (socket says 172.21.15.42:47762) 2024-05-28T14:06:10.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ec0e00640 1 -- 172.21.15.42:0/898950747 learned_addr learned my addr 172.21.15.42:0/898950747 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:10.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ec1800640 1 --2- 172.21.15.42:0/898950747 >> v2:172.21.15.63:3300/0 conn(0x7f7ebc0ce1b0 0x7f7ebc1672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 -- 172.21.15.42:0/898950747 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4226561021 0 0) 0x7f7ebc0cfd80 con 0x7f7ebc15a460 2024-05-28T14:06:10.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 -- 172.21.15.42:0/898950747 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ea4003530 con 0x7f7ebc15a460 2024-05-28T14:06:10.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 -- 172.21.15.42:0/898950747 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 660947911 0 0) 0x7f7ea4003530 con 0x7f7ebc15a460 2024-05-28T14:06:10.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 -- 172.21.15.42:0/898950747 >> v2:172.21.15.63:3300/0 conn(0x7f7ebc0ce1b0 msgr2=0x7f7ebc1672e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:10.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 --2- 172.21.15.42:0/898950747 >> v2:172.21.15.63:3300/0 conn(0x7f7ebc0ce1b0 0x7f7ebc1672e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 -- 172.21.15.42:0/898950747 >> v1:172.21.15.63:6789/0 conn(0x7f7ebc06c390 legacy=0x7f7ebc0cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 -- 172.21.15.42:0/898950747 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc15d180 con 0x7f7ebc15a460 2024-05-28T14:06:10.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 -- 172.21.15.42:0/898950747 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f7eb0001680 con 0x7f7ebc15a460 2024-05-28T14:06:10.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 -- 172.21.15.42:0/898950747 >> v1:172.21.15.42:6789/0 conn(0x7f7ebc15a460 legacy=0x7f7ebc0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 --2- 172.21.15.42:0/898950747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40039c0 0x7f7ea4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 --2- 172.21.15.42:0/898950747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ea4004810 0x7f7ea4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 --2- 172.21.15.42:0/898950747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ea40051d0 0x7f7ea400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 -- 172.21.15.42:0/898950747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc15cc60 con 0x7f7ea4004810 2024-05-28T14:06:10.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 -- 172.21.15.42:0/898950747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea400c170 con 0x7f7ea40051d0 2024-05-28T14:06:10.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebb400640 1 -- 172.21.15.42:0/898950747 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea400c370 con 0x7f7ea40039c0 2024-05-28T14:06:10.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ebbe00640 1 --2- 172.21.15.42:0/898950747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ea4004810 0x7f7ea4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ec0e00640 1 --2- 172.21.15.42:0/898950747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40039c0 0x7f7ea4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.270+0000 7f7ec1800640 1 --2- 172.21.15.42:0/898950747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ea40051d0 0x7f7ea400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ebbe00640 1 -- 172.21.15.42:0/898950747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40039c0 msgr2=0x7f7ea4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:10.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ebbe00640 1 --2- 172.21.15.42:0/898950747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40039c0 0x7f7ea4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ebbe00640 1 -- 172.21.15.42:0/898950747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ea40051d0 msgr2=0x7f7ea400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:10.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ebbe00640 1 --2- 172.21.15.42:0/898950747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ea40051d0 0x7f7ea400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ebbe00640 1 -- 172.21.15.42:0/898950747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea4003530 con 0x7f7ea4004810 2024-05-28T14:06:10.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ec1800640 1 -- 172.21.15.42:0/898950747 reap_dead start 2024-05-28T14:06:10.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ec0e00640 1 --2- 172.21.15.42:0/898950747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40039c0 0x7f7ea4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:10.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ebbe00640 1 --2- 172.21.15.42:0/898950747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ea4004810 0x7f7ea4004c90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8001780 tx=0x7f7ea801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc697ba6f10137f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:10.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ebb400640 1 -- 172.21.15.42:0/898950747 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea80013a0 con 0x7f7ea4004810 2024-05-28T14:06:10.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ebb400640 1 -- 172.21.15.42:0/898950747 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea80052f0 con 0x7f7ea4004810 2024-05-28T14:06:10.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ebb400640 1 -- 172.21.15.42:0/898950747 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8003840 con 0x7f7ea4004810 2024-05-28T14:06:10.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ec3800640 1 -- 172.21.15.42:0/898950747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ea4004810 msgr2=0x7f7ea4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ec3800640 1 --2- 172.21.15.42:0/898950747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ea4004810 0x7f7ea4004c90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8001780 tx=0x7f7ea801c400 comp rx=0 tx=0).stop 2024-05-28T14:06:10.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ec3800640 1 -- 172.21.15.42:0/898950747 shutdown_connections 2024-05-28T14:06:10.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ec3800640 1 --2- 172.21.15.42:0/898950747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ea4004810 0x7f7ea4004c90 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ec3800640 1 -- 172.21.15.42:0/898950747 >> 172.21.15.42:0/898950747 conn(0x7f7ebc0b2f60 msgr2=0x7f7ebc0b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:10.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ec3800640 1 -- 172.21.15.42:0/898950747 shutdown_connections 2024-05-28T14:06:10.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.274+0000 7f7ec3800640 1 -- 172.21.15.42:0/898950747 wait complete. 2024-05-28T14:06:10.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec3800640 1 Processor -- start 2024-05-28T14:06:10.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec3800640 1 -- start start 2024-05-28T14:06:10.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec3800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ea4004810 0x7f7ebc1516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec3800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ebc0af3c0 0x7f7ebc0af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec3800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc0b56c0 0x7f7ebc0b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec3800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc0cfb00 con 0x7f7ebc0b56c0 2024-05-28T14:06:10.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec3800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc0cf980 con 0x7f7ea4004810 2024-05-28T14:06:10.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec3800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7ebc06c7c0 con 0x7f7ebc0af3c0 2024-05-28T14:06:10.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc0b56c0 0x7f7ebc0b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc0b56c0 0x7f7ebc0b5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47646/0 (socket says 172.21.15.42:47646) 2024-05-28T14:06:10.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec1800640 1 -- 172.21.15.42:0/147737163 learned_addr learned my addr 172.21.15.42:0/147737163 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:10.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec0e00640 1 --2- 172.21.15.42:0/147737163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ea4004810 0x7f7ebc1516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ebbe00640 1 --2- 172.21.15.42:0/147737163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ebc0af3c0 0x7f7ebc0af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec1800640 1 -- 172.21.15.42:0/147737163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ebc0af3c0 msgr2=0x7f7ebc0af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec1800640 1 --2- 172.21.15.42:0/147737163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ebc0af3c0 0x7f7ebc0af840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec1800640 1 -- 172.21.15.42:0/147737163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ea4004810 msgr2=0x7f7ebc1516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec1800640 1 --2- 172.21.15.42:0/147737163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ea4004810 0x7f7ebc1516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec1800640 1 -- 172.21.15.42:0/147737163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc06fa10 con 0x7f7ebc0b56c0 2024-05-28T14:06:10.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec1800640 1 --2- 172.21.15.42:0/147737163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc0b56c0 0x7f7ebc0b5b40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc15de00 tx=0x7f7eb400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:10.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7eb8c00640 1 -- 172.21.15.42:0/147737163 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb400fdb0 con 0x7f7ebc0b56c0 2024-05-28T14:06:10.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7eb8c00640 1 -- 172.21.15.42:0/147737163 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb401d070 con 0x7f7ebc0b56c0 2024-05-28T14:06:10.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec3800640 1 -- 172.21.15.42:0/147737163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc06fcf0 con 0x7f7ebc0b56c0 2024-05-28T14:06:10.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7eb8c00640 1 -- 172.21.15.42:0/147737163 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4015670 con 0x7f7ebc0b56c0 2024-05-28T14:06:10.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.278+0000 7f7ec3800640 1 -- 172.21.15.42:0/147737163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc0702b0 con 0x7f7ebc0b56c0 2024-05-28T14:06:10.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.282+0000 7f7ec3800640 1 -- 172.21.15.42:0/147737163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ebc068a90 con 0x7f7ebc0b56c0 2024-05-28T14:06:10.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.282+0000 7f7eb8c00640 1 -- 172.21.15.42:0/147737163 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f7eb4015810 con 0x7f7ebc0b56c0 2024-05-28T14:06:10.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.282+0000 7f7eb8c00640 1 --2- 172.21.15.42:0/147737163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e9c0690a0 0x7f7e9c06b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.282+0000 7f7eb8c00640 1 -- 172.21.15.42:0/147737163 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5765+0+0 (secure 0 0 0) 0x7f7eb40860c0 con 0x7f7ebc0b56c0 2024-05-28T14:06:10.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.282+0000 7f7ec0e00640 1 --2- 172.21.15.42:0/147737163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e9c0690a0 0x7f7e9c06b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.282+0000 7f7ec0e00640 1 --2- 172.21.15.42:0/147737163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e9c0690a0 0x7f7e9c06b560 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0001cd0 tx=0x7f7eb0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:10.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.286+0000 7f7eb8c00640 1 -- 172.21.15.42:0/147737163 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7eb4010040 con 0x7f7ebc0b56c0 2024-05-28T14:06:10.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.498+0000 7f7ec3800640 1 -- 172.21.15.42:0/147737163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ebc159d40 con 0x7f7ebc0b56c0 2024-05-28T14:06:10.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.498+0000 7f7eb8c00640 1 -- 172.21.15.42:0/147737163 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5178 (secure 0 0 0) 0x7f7eb4057ea0 con 0x7f7ebc0b56c0 2024-05-28T14:06:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:10.485 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1310016671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1310016671},{"type":"v1","addr":"172.21.15.63:6834","nonce":1310016671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1602237641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1602237641},{"type":"v1","addr":"172.21.15.42:6835","nonce":1602237641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/142807607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":142807607},{"type":"v1","addr":"172.21.15.42:6837","nonce":142807607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:04:22.960438+0000","modified":"2024-05-28T14:04:25.983153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3311861302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3311861302},{"type":"v1","addr":"172.21.15.63:6835","nonce":3311861302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6420,"qdb_cluster":[6420]},"id":4}]} 2024-05-28T14:06:10.485 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 37 2024-05-28T14:06:10.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.498+0000 7f7ec3800640 1 -- 172.21.15.42:0/147737163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e9c0690a0 msgr2=0x7f7e9c06b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.498+0000 7f7ec3800640 1 --2- 172.21.15.42:0/147737163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e9c0690a0 0x7f7e9c06b560 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0001cd0 tx=0x7f7eb0009040 comp rx=0 tx=0).stop 2024-05-28T14:06:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.498+0000 7f7ec3800640 1 -- 172.21.15.42:0/147737163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc0b56c0 msgr2=0x7f7ebc0b5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.498+0000 7f7ec3800640 1 --2- 172.21.15.42:0/147737163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc0b56c0 0x7f7ebc0b5b40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc15de00 tx=0x7f7eb400f580 comp rx=0 tx=0).stop 2024-05-28T14:06:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.498+0000 7f7ec3800640 1 -- 172.21.15.42:0/147737163 shutdown_connections 2024-05-28T14:06:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.498+0000 7f7ec3800640 1 --2- 172.21.15.42:0/147737163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7e9c0690a0 0x7f7e9c06b560 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.498+0000 7f7ec3800640 1 --2- 172.21.15.42:0/147737163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc0b56c0 0x7f7ebc0b5b40 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.498+0000 7f7ec3800640 1 --2- 172.21.15.42:0/147737163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ebc0af3c0 0x7f7ebc0af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.498+0000 7f7ec3800640 1 --2- 172.21.15.42:0/147737163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ea4004810 0x7f7ebc1516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.498+0000 7f7ec3800640 1 -- 172.21.15.42:0/147737163 >> 172.21.15.42:0/147737163 conn(0x7f7ebc0b2f60 msgr2=0x7f7ebc0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.498+0000 7f7ec3800640 1 -- 172.21.15.42:0/147737163 shutdown_connections 2024-05-28T14:06:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.498+0000 7f7ec3800640 1 -- 172.21.15.42:0/147737163 wait complete. 2024-05-28T14:06:10.506 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:06:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1ff000640 1 Processor -- start 2024-05-28T14:06:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1ff000640 1 -- start start 2024-05-28T14:06:10.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1ff000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe1f806ff10 0x7fe1f815e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1ff000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f80703a0 0x7fe1f815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1ff000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1f80d0660 con 0x7fe1f8070800 2024-05-28T14:06:10.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1ff000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe1f809ba60 con 0x7fe1f806ff10 2024-05-28T14:06:10.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1ff000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f80d0940 con 0x7fe1f80703a0 2024-05-28T14:06:10.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1fdc00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe1f8070800 0x7fe1f8070be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47778/0 (socket says 172.21.15.42:47778) 2024-05-28T14:06:10.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1fdc00640 1 -- 172.21.15.42:0/3731044877 learned_addr learned my addr 172.21.15.42:0/3731044877 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:10.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1fd200640 1 --2- 172.21.15.42:0/3731044877 >> v2:172.21.15.63:3300/0 conn(0x7fe1f806ff10 0x7fe1f815e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1fe600640 1 --2- 172.21.15.42:0/3731044877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f80703a0 0x7fe1f815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1f7e00640 1 -- 172.21.15.42:0/3731044877 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3244397740 0 0) 0x7fe1f80d0660 con 0x7fe1f8070800 2024-05-28T14:06:10.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1f7e00640 1 -- 172.21.15.42:0/3731044877 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1e0003530 con 0x7fe1f8070800 2024-05-28T14:06:10.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1fd200640 1 -- 172.21.15.42:0/3731044877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f80703a0 msgr2=0x7fe1f815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:10.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1fd200640 1 --2- 172.21.15.42:0/3731044877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f80703a0 0x7fe1f815ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1fd200640 1 -- 172.21.15.42:0/3731044877 >> v1:172.21.15.42:6789/0 conn(0x7fe1f8070800 legacy=0x7fe1f8070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.662+0000 7fe1fd200640 1 -- 172.21.15.42:0/3731044877 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f80d0c20 con 0x7fe1f806ff10 2024-05-28T14:06:10.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.666+0000 7fe1fd200640 1 --2- 172.21.15.42:0/3731044877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f806ff10 0x7fe1f815e910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40097f0 tx=0x7fe1e401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99fe79c93be3ba40 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:10.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.666+0000 7fe1f7e00640 1 -- 172.21.15.42:0/3731044877 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40b2070 con 0x7fe1f806ff10 2024-05-28T14:06:10.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.666+0000 7fe1f7e00640 1 -- 172.21.15.42:0/3731044877 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1e40ad640 con 0x7fe1f806ff10 2024-05-28T14:06:10.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.666+0000 7fe1f7e00640 1 -- 172.21.15.42:0/3731044877 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40ada10 con 0x7fe1f806ff10 2024-05-28T14:06:10.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.666+0000 7fe1ff000640 1 -- 172.21.15.42:0/3731044877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f806ff10 msgr2=0x7fe1f815e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.666+0000 7fe1ff000640 1 --2- 172.21.15.42:0/3731044877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f806ff10 0x7fe1f815e910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40097f0 tx=0x7fe1e401c4a0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.666+0000 7fe1ff000640 1 -- 172.21.15.42:0/3731044877 shutdown_connections 2024-05-28T14:06:10.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.666+0000 7fe1ff000640 1 --2- 172.21.15.42:0/3731044877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f80703a0 0x7fe1f815ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.666+0000 7fe1ff000640 1 --2- 172.21.15.42:0/3731044877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f806ff10 0x7fe1f815e910 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.666+0000 7fe1ff000640 1 -- 172.21.15.42:0/3731044877 >> 172.21.15.42:0/3731044877 conn(0x7fe1f80b3dd0 msgr2=0x7fe1f80b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.666+0000 7fe1ff000640 1 -- 172.21.15.42:0/3731044877 shutdown_connections 2024-05-28T14:06:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.666+0000 7fe1ff000640 1 -- 172.21.15.42:0/3731044877 wait complete. 2024-05-28T14:06:10.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.666+0000 7fe1ff000640 1 Processor -- start 2024-05-28T14:06:10.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1ff000640 1 -- start start 2024-05-28T14:06:10.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1ff000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f806ff10 0x7fe1f8152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1ff000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f80703a0 0x7fe1f815d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1ff000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1f8070800 0x7fe1f8159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1ff000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f80b1a10 con 0x7fe1f8070800 2024-05-28T14:06:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1ff000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f80b1890 con 0x7fe1f806ff10 2024-05-28T14:06:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1ff000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f816b5d0 con 0x7fe1f80703a0 2024-05-28T14:06:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1fe600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1f8070800 0x7fe1f8159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1fdc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f806ff10 0x7fe1f8152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1fe600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1f8070800 0x7fe1f8159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47652/0 (socket says 172.21.15.42:47652) 2024-05-28T14:06:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1fdc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f806ff10 0x7fe1f8152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38950/0 (socket says 172.21.15.42:38950) 2024-05-28T14:06:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1fe600640 1 -- 172.21.15.42:0/1785071320 learned_addr learned my addr 172.21.15.42:0/1785071320 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1fd200640 1 --2- 172.21.15.42:0/1785071320 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f80703a0 0x7fe1f815d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1fe600640 1 -- 172.21.15.42:0/1785071320 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f80703a0 msgr2=0x7fe1f815d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1fe600640 1 --2- 172.21.15.42:0/1785071320 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f80703a0 0x7fe1f815d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1fe600640 1 -- 172.21.15.42:0/1785071320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f806ff10 msgr2=0x7fe1f8152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1fe600640 1 --2- 172.21.15.42:0/1785071320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f806ff10 0x7fe1f8152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1fe600640 1 -- 172.21.15.42:0/1785071320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f80cb390 con 0x7fe1f8070800 2024-05-28T14:06:10.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1fe600640 1 --2- 172.21.15.42:0/1785071320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1f8070800 0x7fe1f8159b10 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000ef90 tx=0x7fe1f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:10.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1f5600640 1 -- 172.21.15.42:0/1785071320 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0018020 con 0x7fe1f8070800 2024-05-28T14:06:10.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1f5600640 1 -- 172.21.15.42:0/1785071320 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f0014480 con 0x7fe1f8070800 2024-05-28T14:06:10.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1f5600640 1 -- 172.21.15.42:0/1785071320 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0014870 con 0x7fe1f8070800 2024-05-28T14:06:10.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1ff000640 1 -- 172.21.15.42:0/1785071320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f80cb670 con 0x7fe1f8070800 2024-05-28T14:06:10.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1ff000640 1 -- 172.21.15.42:0/1785071320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f80cbc30 con 0x7fe1f8070800 2024-05-28T14:06:10.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.670+0000 7fe1ff000640 1 -- 172.21.15.42:0/1785071320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f804fcf0 con 0x7fe1f8070800 2024-05-28T14:06:10.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.674+0000 7fe1f5600640 1 -- 172.21.15.42:0/1785071320 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7fe1f001b020 con 0x7fe1f8070800 2024-05-28T14:06:10.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.674+0000 7fe1f5600640 1 --2- 172.21.15.42:0/1785071320 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe1d80690a0 0x7fe1d806b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.674+0000 7fe1f5600640 1 -- 172.21.15.42:0/1785071320 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5765+0+0 (secure 0 0 0) 0x7fe1f00861e0 con 0x7fe1f8070800 2024-05-28T14:06:10.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.674+0000 7fe1fdc00640 1 --2- 172.21.15.42:0/1785071320 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe1d80690a0 0x7fe1d806b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.674+0000 7fe1fdc00640 1 --2- 172.21.15.42:0/1785071320 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe1d80690a0 0x7fe1d806b560 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec007fd0 tx=0x7fe1ec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:10.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.678+0000 7fe1f5600640 1 -- 172.21.15.42:0/1785071320 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe1f0022020 con 0x7fe1f8070800 2024-05-28T14:06:10.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.842+0000 7fe1ff000640 1 -- 172.21.15.42:0/1785071320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1f80d1d50 con 0x7fe1f8070800 2024-05-28T14:06:10.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.846+0000 7fe1f5600640 1 -- 172.21.15.42:0/1785071320 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16713 (secure 0 0 0) 0x7fe1f0010040 con 0x7fe1f8070800 2024-05-28T14:06:10.832 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:10.832 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":63,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:06:07.536649+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:04:20.721059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-28T14:04:21.922127+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":62,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:06:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.846+0000 7fe1ff000640 1 -- 172.21.15.42:0/1785071320 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe1d80690a0 msgr2=0x7fe1d806b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.846+0000 7fe1ff000640 1 --2- 172.21.15.42:0/1785071320 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe1d80690a0 0x7fe1d806b560 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec007fd0 tx=0x7fe1ec009040 comp rx=0 tx=0).stop 2024-05-28T14:06:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.846+0000 7fe1ff000640 1 -- 172.21.15.42:0/1785071320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1f8070800 msgr2=0x7fe1f8159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.846+0000 7fe1ff000640 1 --2- 172.21.15.42:0/1785071320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1f8070800 0x7fe1f8159b10 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000ef90 tx=0x7fe1f000c490 comp rx=0 tx=0).stop 2024-05-28T14:06:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.846+0000 7fe1ff000640 1 -- 172.21.15.42:0/1785071320 shutdown_connections 2024-05-28T14:06:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.846+0000 7fe1ff000640 1 --2- 172.21.15.42:0/1785071320 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe1d80690a0 0x7fe1d806b560 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.846+0000 7fe1ff000640 1 --2- 172.21.15.42:0/1785071320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1f8070800 0x7fe1f8159b10 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.846+0000 7fe1ff000640 1 --2- 172.21.15.42:0/1785071320 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f80703a0 0x7fe1f815d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.846+0000 7fe1ff000640 1 --2- 172.21.15.42:0/1785071320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f806ff10 0x7fe1f8152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.846+0000 7fe1ff000640 1 -- 172.21.15.42:0/1785071320 >> 172.21.15.42:0/1785071320 conn(0x7fe1f80b3dd0 msgr2=0x7fe1f806d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.846+0000 7fe1ff000640 1 -- 172.21.15.42:0/1785071320 shutdown_connections 2024-05-28T14:06:10.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:10.846+0000 7fe1ff000640 1 -- 172.21.15.42:0/1785071320 wait complete. 2024-05-28T14:06:10.853 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-28T14:06:10.853 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:06:10.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.006+0000 7f4b6a800640 1 Processor -- start 2024-05-28T14:06:10.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.006+0000 7f4b6a800640 1 -- start start 2024-05-28T14:06:10.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.010+0000 7f4b6a800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4b640d0e00 0x7f4b641663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:10.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.010+0000 7f4b6a800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b6415f280 con 0x7f4b6406f660 2024-05-28T14:06:10.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.010+0000 7f4b6a800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b6415f560 con 0x7f4b6415ba40 2024-05-28T14:06:10.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.010+0000 7f4b6a800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f4b6415f030 con 0x7f4b640d0e00 2024-05-28T14:06:10.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.010+0000 7f4b63400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f4b6406f660 0x7f4b64073980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47790/0 (socket says 172.21.15.42:47790) 2024-05-28T14:06:10.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.010+0000 7f4b63400640 1 -- 172.21.15.42:0/531051302 learned_addr learned my addr 172.21.15.42:0/531051302 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:10.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.010+0000 7f4b63e00640 1 --2- 172.21.15.42:0/531051302 >> v2:172.21.15.42:3300/0 conn(0x7f4b640d0e00 0x7f4b641663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:10.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.010+0000 7f4b63e00640 1 -- 172.21.15.42:0/531051302 >> v1:172.21.15.63:6789/0 conn(0x7f4b6415ba40 legacy=0x7f4b640d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.010+0000 7f4b63e00640 1 -- 172.21.15.42:0/531051302 >> v1:172.21.15.42:6789/0 conn(0x7f4b6406f660 legacy=0x7f4b64073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:10.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.010+0000 7f4b63e00640 1 -- 172.21.15.42:0/531051302 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6415f840 con 0x7f4b640d0e00 2024-05-28T14:06:11.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.014+0000 7f4b63e00640 1 --2- 172.21.15.42:0/531051302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b640d0e00 0x7f4b641663c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009d90 tx=0x7f4b5c01c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362fb6af14471677 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:11.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.014+0000 7f4b62a00640 1 -- 172.21.15.42:0/531051302 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0b2070 con 0x7f4b640d0e00 2024-05-28T14:06:11.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.014+0000 7f4b62a00640 1 -- 172.21.15.42:0/531051302 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b5c0a5900 con 0x7f4b640d0e00 2024-05-28T14:06:11.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.014+0000 7f4b62a00640 1 -- 172.21.15.42:0/531051302 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0a5cd0 con 0x7f4b640d0e00 2024-05-28T14:06:11.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.014+0000 7f4b6a800640 1 -- 172.21.15.42:0/531051302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b640d0e00 msgr2=0x7f4b641663c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.014+0000 7f4b6a800640 1 --2- 172.21.15.42:0/531051302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b640d0e00 0x7f4b641663c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009d90 tx=0x7f4b5c01c820 comp rx=0 tx=0).stop 2024-05-28T14:06:11.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.014+0000 7f4b6a800640 1 -- 172.21.15.42:0/531051302 shutdown_connections 2024-05-28T14:06:11.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.014+0000 7f4b6a800640 1 --2- 172.21.15.42:0/531051302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b640d0e00 0x7f4b641663c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.014+0000 7f4b6a800640 1 -- 172.21.15.42:0/531051302 >> 172.21.15.42:0/531051302 conn(0x7f4b6406bdf0 msgr2=0x7f4b6406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:11.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.014+0000 7f4b6a800640 1 -- 172.21.15.42:0/531051302 shutdown_connections 2024-05-28T14:06:11.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.014+0000 7f4b6a800640 1 -- 172.21.15.42:0/531051302 wait complete. 2024-05-28T14:06:11.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.014+0000 7f4b6a800640 1 Processor -- start 2024-05-28T14:06:11.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.014+0000 7f4b6a800640 1 -- start start 2024-05-28T14:06:11.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b6a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b6406f660 0x7f4b64149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b6a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b640d0e00 0x7f4b640ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b6a800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b640b04c0 0x7f4b6415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b6a800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b640b3670 con 0x7f4b640d0e00 2024-05-28T14:06:11.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b6a800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b640b34f0 con 0x7f4b6406f660 2024-05-28T14:06:11.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b6a800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b640b37f0 con 0x7f4b640b04c0 2024-05-28T14:06:11.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b5be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b640d0e00 0x7f4b640ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b5be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b640d0e00 0x7f4b640ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47674/0 (socket says 172.21.15.42:47674) 2024-05-28T14:06:11.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b5be00640 1 -- 172.21.15.42:0/798381345 learned_addr learned my addr 172.21.15.42:0/798381345 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:11.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b63400640 1 --2- 172.21.15.42:0/798381345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b6406f660 0x7f4b64149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b63e00640 1 --2- 172.21.15.42:0/798381345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b640b04c0 0x7f4b6415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b5be00640 1 -- 172.21.15.42:0/798381345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b640b04c0 msgr2=0x7f4b6415d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b5be00640 1 --2- 172.21.15.42:0/798381345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b640b04c0 0x7f4b6415d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b5be00640 1 -- 172.21.15.42:0/798381345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b6406f660 msgr2=0x7f4b64149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b5be00640 1 --2- 172.21.15.42:0/798381345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b6406f660 0x7f4b64149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b5be00640 1 -- 172.21.15.42:0/798381345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b641f7ef0 con 0x7f4b640d0e00 2024-05-28T14:06:11.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b5be00640 1 --2- 172.21.15.42:0/798381345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b640d0e00 0x7f4b640ccd00 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00c370 tx=0x7f4b4c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:11.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b5b400640 1 -- 172.21.15.42:0/798381345 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0052d0 con 0x7f4b640d0e00 2024-05-28T14:06:11.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b5b400640 1 -- 172.21.15.42:0/798381345 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4c005470 con 0x7f4b640d0e00 2024-05-28T14:06:11.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b5b400640 1 -- 172.21.15.42:0/798381345 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c007260 con 0x7f4b640d0e00 2024-05-28T14:06:11.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b6a800640 1 -- 172.21.15.42:0/798381345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b640cdbc0 con 0x7f4b640d0e00 2024-05-28T14:06:11.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b6a800640 1 -- 172.21.15.42:0/798381345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b640cd8e0 con 0x7f4b640d0e00 2024-05-28T14:06:11.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b6a800640 1 -- 172.21.15.42:0/798381345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b64068a90 con 0x7f4b640d0e00 2024-05-28T14:06:11.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.018+0000 7f4b5b400640 1 -- 172.21.15.42:0/798381345 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f4b4c007400 con 0x7f4b640d0e00 2024-05-28T14:06:11.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.022+0000 7f4b5b400640 1 --2- 172.21.15.42:0/798381345 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b40069070 0x7f4b4006b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.022+0000 7f4b5b400640 1 -- 172.21.15.42:0/798381345 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5765+0+0 (secure 0 0 0) 0x7f4b4c0848b0 con 0x7f4b640d0e00 2024-05-28T14:06:11.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.022+0000 7f4b63400640 1 --2- 172.21.15.42:0/798381345 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b40069070 0x7f4b4006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.022+0000 7f4b63400640 1 --2- 172.21.15.42:0/798381345 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b40069070 0x7f4b4006b530 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4b54005890 tx=0x7f4b54009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:11.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.026+0000 7f4b5b400640 1 -- 172.21.15.42:0/798381345 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4b4c0561b0 con 0x7f4b640d0e00 2024-05-28T14:06:11.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.226+0000 7f4b6a800640 1 -- 172.21.15.42:0/798381345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b6406c270 con 0x7f4b640d0e00 2024-05-28T14:06:11.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.226+0000 7f4b5b400640 1 -- 172.21.15.42:0/798381345 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5178 (secure 0 0 0) 0x7f4b4c05a4b0 con 0x7f4b640d0e00 2024-05-28T14:06:11.213 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:11.213 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1310016671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1310016671},{"type":"v1","addr":"172.21.15.63:6834","nonce":1310016671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1602237641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1602237641},{"type":"v1","addr":"172.21.15.42:6835","nonce":1602237641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/142807607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":142807607},{"type":"v1","addr":"172.21.15.42:6837","nonce":142807607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:04:22.960438+0000","modified":"2024-05-28T14:04:25.983153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3311861302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3311861302},{"type":"v1","addr":"172.21.15.63:6835","nonce":3311861302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6420,"qdb_cluster":[6420]},"id":4}]} 2024-05-28T14:06:11.213 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 37 2024-05-28T14:06:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.226+0000 7f4b6a800640 1 -- 172.21.15.42:0/798381345 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b40069070 msgr2=0x7f4b4006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.226+0000 7f4b6a800640 1 --2- 172.21.15.42:0/798381345 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b40069070 0x7f4b4006b530 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4b54005890 tx=0x7f4b54009040 comp rx=0 tx=0).stop 2024-05-28T14:06:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.226+0000 7f4b6a800640 1 -- 172.21.15.42:0/798381345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b640d0e00 msgr2=0x7f4b640ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.226+0000 7f4b6a800640 1 --2- 172.21.15.42:0/798381345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b640d0e00 0x7f4b640ccd00 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00c370 tx=0x7f4b4c00c830 comp rx=0 tx=0).stop 2024-05-28T14:06:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.226+0000 7f4b6a800640 1 -- 172.21.15.42:0/798381345 shutdown_connections 2024-05-28T14:06:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.226+0000 7f4b6a800640 1 --2- 172.21.15.42:0/798381345 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b40069070 0x7f4b4006b530 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.226+0000 7f4b6a800640 1 --2- 172.21.15.42:0/798381345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b640b04c0 0x7f4b6415d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.226+0000 7f4b6a800640 1 --2- 172.21.15.42:0/798381345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b640d0e00 0x7f4b640ccd00 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.226+0000 7f4b6a800640 1 --2- 172.21.15.42:0/798381345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b6406f660 0x7f4b64149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.226+0000 7f4b6a800640 1 -- 172.21.15.42:0/798381345 >> 172.21.15.42:0/798381345 conn(0x7f4b6406bdf0 msgr2=0x7f4b640d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.226+0000 7f4b6a800640 1 -- 172.21.15.42:0/798381345 shutdown_connections 2024-05-28T14:06:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.226+0000 7f4b6a800640 1 -- 172.21.15.42:0/798381345 wait complete. 2024-05-28T14:06:11.235 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:06:11.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f6923200640 1 Processor -- start 2024-05-28T14:06:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f6923200640 1 -- start start 2024-05-28T14:06:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f6923200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f692409aa50 0x7f692409ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f6923200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f69240cff00 0x7f69240d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f6923200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f692415edb0 con 0x7f69240d1ae0 2024-05-28T14:06:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f6923200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6924071560 con 0x7f692409aa50 2024-05-28T14:06:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f6923200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f692409b3d0 con 0x7f69240cff00 2024-05-28T14:06:11.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f691be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f692409aa50 0x7f692409ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f691be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f692409aa50 0x7f692409ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47690/0 (socket says 172.21.15.42:47690) 2024-05-28T14:06:11.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f691be00640 1 -- 172.21.15.42:0/990670527 learned_addr learned my addr 172.21.15.42:0/990670527 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:11.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f6922800640 1 --2- 172.21.15.42:0/990670527 >> v2:172.21.15.63:3300/0 conn(0x7f69240cff00 0x7f69240d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f691be00640 1 -- 172.21.15.42:0/990670527 >> v2:172.21.15.63:3300/0 conn(0x7f69240cff00 msgr2=0x7f69240d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:11.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f691be00640 1 --2- 172.21.15.42:0/990670527 >> v2:172.21.15.63:3300/0 conn(0x7f69240cff00 0x7f69240d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f691be00640 1 -- 172.21.15.42:0/990670527 >> v1:172.21.15.63:6789/0 conn(0x7f69240d1ae0 legacy=0x7f692406d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:11.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.390+0000 7f691be00640 1 -- 172.21.15.42:0/990670527 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6924070950 con 0x7f692409aa50 2024-05-28T14:06:11.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.394+0000 7f691be00640 1 --2- 172.21.15.42:0/990670527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f692409aa50 0x7f692409ae90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6924163560 tx=0x7f690c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d654ed556c764a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:11.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.394+0000 7f6921400640 1 -- 172.21.15.42:0/990670527 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c0b5070 con 0x7f692409aa50 2024-05-28T14:06:11.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.394+0000 7f6921400640 1 -- 172.21.15.42:0/990670527 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690c0a89d0 con 0x7f692409aa50 2024-05-28T14:06:11.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.394+0000 7f6921400640 1 -- 172.21.15.42:0/990670527 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c0a8da0 con 0x7f692409aa50 2024-05-28T14:06:11.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.394+0000 7f6923200640 1 -- 172.21.15.42:0/990670527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f692409aa50 msgr2=0x7f692409ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.394+0000 7f6923200640 1 --2- 172.21.15.42:0/990670527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f692409aa50 0x7f692409ae90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6924163560 tx=0x7f690c01ecd0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.394+0000 7f6923200640 1 -- 172.21.15.42:0/990670527 shutdown_connections 2024-05-28T14:06:11.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.394+0000 7f6923200640 1 --2- 172.21.15.42:0/990670527 >> v2:172.21.15.63:3300/0 conn(0x7f69240cff00 0x7f69240d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.394+0000 7f6923200640 1 --2- 172.21.15.42:0/990670527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f692409aa50 0x7f692409ae90 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.394+0000 7f6923200640 1 -- 172.21.15.42:0/990670527 >> 172.21.15.42:0/990670527 conn(0x7f69240d2330 msgr2=0x7f69240d2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:11.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.394+0000 7f6923200640 1 -- 172.21.15.42:0/990670527 shutdown_connections 2024-05-28T14:06:11.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.394+0000 7f6923200640 1 -- 172.21.15.42:0/990670527 wait complete. 2024-05-28T14:06:11.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6923200640 1 Processor -- start 2024-05-28T14:06:11.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6923200640 1 -- start start 2024-05-28T14:06:11.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6923200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f692409aa50 0x7f692409dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6923200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69240cff00 0x7f6924154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6923200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69240d1ae0 0x7f692409fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6923200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69240711f0 con 0x7f69240d1ae0 2024-05-28T14:06:11.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6923200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6924070f10 con 0x7f69240cff00 2024-05-28T14:06:11.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6923200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f692409bc00 con 0x7f692409aa50 2024-05-28T14:06:11.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6922800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69240d1ae0 0x7f692409fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6922800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69240d1ae0 0x7f692409fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47696/0 (socket says 172.21.15.42:47696) 2024-05-28T14:06:11.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6922800640 1 -- 172.21.15.42:0/364892632 learned_addr learned my addr 172.21.15.42:0/364892632 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:11.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f691be00640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69240cff00 0x7f6924154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6921e00640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f692409aa50 0x7f692409dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6922800640 1 -- 172.21.15.42:0/364892632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f692409aa50 msgr2=0x7f692409dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6922800640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f692409aa50 0x7f692409dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6922800640 1 -- 172.21.15.42:0/364892632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69240cff00 msgr2=0x7f6924154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6922800640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69240cff00 0x7f6924154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6922800640 1 -- 172.21.15.42:0/364892632 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6924154760 con 0x7f69240d1ae0 2024-05-28T14:06:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6921e00640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f692409aa50 0x7f692409dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6922800640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69240d1ae0 0x7f692409fd70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f691c007e40 tx=0x7f691c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f691a000640 1 -- 172.21.15.42:0/364892632 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c014620 con 0x7f69240d1ae0 2024-05-28T14:06:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f691a000640 1 -- 172.21.15.42:0/364892632 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f691c019070 con 0x7f69240d1ae0 2024-05-28T14:06:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6923200640 1 -- 172.21.15.42:0/364892632 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692409e320 con 0x7f69240d1ae0 2024-05-28T14:06:11.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f691a000640 1 -- 172.21.15.42:0/364892632 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c014a20 con 0x7f69240d1ae0 2024-05-28T14:06:11.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.398+0000 7f6923200640 1 -- 172.21.15.42:0/364892632 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692409e8e0 con 0x7f69240d1ae0 2024-05-28T14:06:11.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.402+0000 7f6923200640 1 -- 172.21.15.42:0/364892632 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692404fcf0 con 0x7f69240d1ae0 2024-05-28T14:06:11.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.402+0000 7f691a000640 1 -- 172.21.15.42:0/364892632 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f691c014bc0 con 0x7f69240d1ae0 2024-05-28T14:06:11.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.402+0000 7f691a000640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6900069070 0x7f690006b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.402+0000 7f691a000640 1 -- 172.21.15.42:0/364892632 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5765+0+0 (secure 0 0 0) 0x7f691c085c60 con 0x7f69240d1ae0 2024-05-28T14:06:11.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.402+0000 7f6921e00640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6900069070 0x7f690006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.402+0000 7f6921e00640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6900069070 0x7f690006b530 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f69140098b0 tx=0x7f6914008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:11.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.406+0000 7f691a000640 1 -- 172.21.15.42:0/364892632 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f691c057c70 con 0x7f69240d1ae0 2024-05-28T14:06:11.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.606+0000 7f6923200640 1 -- 172.21.15.42:0/364892632 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6924071b40 con 0x7f69240d1ae0 2024-05-28T14:06:11.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.606+0000 7f691a000640 1 -- 172.21.15.42:0/364892632 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5178 (secure 0 0 0) 0x7f691c057570 con 0x7f69240d1ae0 2024-05-28T14:06:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:11.594 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1310016671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1310016671},{"type":"v1","addr":"172.21.15.63:6834","nonce":1310016671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1602237641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1602237641},{"type":"v1","addr":"172.21.15.42:6835","nonce":1602237641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/142807607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":142807607},{"type":"v1","addr":"172.21.15.42:6837","nonce":142807607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:04:22.960438+0000","modified":"2024-05-28T14:04:25.983153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3311861302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3311861302},{"type":"v1","addr":"172.21.15.63:6835","nonce":3311861302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6420,"qdb_cluster":[6420]},"id":4}]} 2024-05-28T14:06:11.594 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 37 2024-05-28T14:06:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.610+0000 7f6923200640 1 -- 172.21.15.42:0/364892632 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6900069070 msgr2=0x7f690006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.610+0000 7f6923200640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6900069070 0x7f690006b530 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f69140098b0 tx=0x7f6914008040 comp rx=0 tx=0).stop 2024-05-28T14:06:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.610+0000 7f6923200640 1 -- 172.21.15.42:0/364892632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69240d1ae0 msgr2=0x7f692409fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.610+0000 7f6923200640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69240d1ae0 0x7f692409fd70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f691c007e40 tx=0x7f691c00e4b0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.610+0000 7f6923200640 1 -- 172.21.15.42:0/364892632 shutdown_connections 2024-05-28T14:06:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.610+0000 7f6923200640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6900069070 0x7f690006b530 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.610+0000 7f6923200640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69240d1ae0 0x7f692409fd70 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.610+0000 7f6923200640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69240cff00 0x7f6924154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.610+0000 7f6923200640 1 --2- 172.21.15.42:0/364892632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f692409aa50 0x7f692409dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.610+0000 7f6923200640 1 -- 172.21.15.42:0/364892632 >> 172.21.15.42:0/364892632 conn(0x7f69240d2330 msgr2=0x7f69240b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.610+0000 7f6923200640 1 -- 172.21.15.42:0/364892632 shutdown_connections 2024-05-28T14:06:11.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.610+0000 7f6923200640 1 -- 172.21.15.42:0/364892632 wait complete. 2024-05-28T14:06:11.615 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:06:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59abc00640 1 Processor -- start 2024-05-28T14:06:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59abc00640 1 -- start start 2024-05-28T14:06:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59abc00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f59a415c990 0x7f59a40d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59abc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f59a409bbd0 0x7f59a409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59abc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59a40cd300 con 0x7f59a40d06a0 2024-05-28T14:06:11.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59abc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f59a415cd70 con 0x7f59a409bbd0 2024-05-28T14:06:11.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59abc00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f59a40b63f0 con 0x7f59a415c990 2024-05-28T14:06:11.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59a3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f59a409bbd0 0x7f59a409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59a3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f59a409bbd0 0x7f59a409aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47698/0 (socket says 172.21.15.42:47698) 2024-05-28T14:06:11.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59a3e00640 1 -- 172.21.15.42:0/4006854975 learned_addr learned my addr 172.21.15.42:0/4006854975 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:11.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59a9200640 1 --2- 172.21.15.42:0/4006854975 >> v2:172.21.15.63:3300/0 conn(0x7f59a415c990 0x7f59a40d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59a3e00640 1 -- 172.21.15.42:0/4006854975 >> v2:172.21.15.63:3300/0 conn(0x7f59a415c990 msgr2=0x7f59a40d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:11.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59a3e00640 1 --2- 172.21.15.42:0/4006854975 >> v2:172.21.15.63:3300/0 conn(0x7f59a415c990 0x7f59a40d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59a3e00640 1 -- 172.21.15.42:0/4006854975 >> v1:172.21.15.63:6789/0 conn(0x7f59a40d06a0 legacy=0x7f59a4160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59a3e00640 1 -- 172.21.15.42:0/4006854975 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a40cd5e0 con 0x7f59a409bbd0 2024-05-28T14:06:11.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.770+0000 7f59a9200640 1 --2- 172.21.15.42:0/4006854975 >> v2:172.21.15.63:3300/0 conn(0x7f59a415c990 0x7f59a40d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:11.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.774+0000 7f59a3e00640 1 --2- 172.21.15.42:0/4006854975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59a409bbd0 0x7f59a409aa50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f59a4073350 tx=0x7f599001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bee4a606316a00 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:11.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.774+0000 7f59a3400640 1 -- 172.21.15.42:0/4006854975 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59900ab600 con 0x7f59a409bbd0 2024-05-28T14:06:11.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.774+0000 7f59a3400640 1 -- 172.21.15.42:0/4006854975 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59900b0070 con 0x7f59a409bbd0 2024-05-28T14:06:11.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.774+0000 7f59a3400640 1 -- 172.21.15.42:0/4006854975 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59900ab9c0 con 0x7f59a409bbd0 2024-05-28T14:06:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.774+0000 7f59abc00640 1 -- 172.21.15.42:0/4006854975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59a409bbd0 msgr2=0x7f59a409aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.774+0000 7f59abc00640 1 --2- 172.21.15.42:0/4006854975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59a409bbd0 0x7f59a409aa50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f59a4073350 tx=0x7f599001c4f0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.774+0000 7f59abc00640 1 -- 172.21.15.42:0/4006854975 shutdown_connections 2024-05-28T14:06:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.774+0000 7f59abc00640 1 --2- 172.21.15.42:0/4006854975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59a409bbd0 0x7f59a409aa50 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.774+0000 7f59abc00640 1 --2- 172.21.15.42:0/4006854975 >> v2:172.21.15.63:3300/0 conn(0x7f59a415c990 0x7f59a40d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.774+0000 7f59abc00640 1 -- 172.21.15.42:0/4006854975 >> 172.21.15.42:0/4006854975 conn(0x7f59a40b4080 msgr2=0x7f59a40b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.774+0000 7f59abc00640 1 -- 172.21.15.42:0/4006854975 shutdown_connections 2024-05-28T14:06:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.774+0000 7f59abc00640 1 -- 172.21.15.42:0/4006854975 wait complete. 2024-05-28T14:06:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59abc00640 1 Processor -- start 2024-05-28T14:06:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59abc00640 1 -- start start 2024-05-28T14:06:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59abc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59a40d06a0 0x7f59a4154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59abc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a415c990 0x7f59a406db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59abc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59a41569a0 0x7f59a4156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59abc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f59a40ce3a0 con 0x7f59a41569a0 2024-05-28T14:06:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59abc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4169740 con 0x7f59a415c990 2024-05-28T14:06:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59abc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f59a4159e20 con 0x7f59a40d06a0 2024-05-28T14:06:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59a41569a0 0x7f59a4156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59a41569a0 0x7f59a4156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47702/0 (socket says 172.21.15.42:47702) 2024-05-28T14:06:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a9c00640 1 -- 172.21.15.42:0/2805436016 learned_addr learned my addr 172.21.15.42:0/2805436016 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a3e00640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a415c990 0x7f59a406db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a9200640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59a40d06a0 0x7f59a4154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a9c00640 1 -- 172.21.15.42:0/2805436016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59a40d06a0 msgr2=0x7f59a4154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a9c00640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59a40d06a0 0x7f59a4154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a9c00640 1 -- 172.21.15.42:0/2805436016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a415c990 msgr2=0x7f59a406db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a9c00640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a415c990 0x7f59a406db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a9c00640 1 -- 172.21.15.42:0/2805436016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a40c7d10 con 0x7f59a41569a0 2024-05-28T14:06:11.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a9200640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59a40d06a0 0x7f59a4154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:11.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a9c00640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59a41569a0 0x7f59a4156e50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f599c00c7d0 tx=0x7f599c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:11.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a0c00640 1 -- 172.21.15.42:0/2805436016 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c003b80 con 0x7f59a41569a0 2024-05-28T14:06:11.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a0c00640 1 -- 172.21.15.42:0/2805436016 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599c003d20 con 0x7f59a41569a0 2024-05-28T14:06:11.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59abc00640 1 -- 172.21.15.42:0/2805436016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a40c7ff0 con 0x7f59a41569a0 2024-05-28T14:06:11.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59a0c00640 1 -- 172.21.15.42:0/2805436016 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c00aa30 con 0x7f59a41569a0 2024-05-28T14:06:11.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59abc00640 1 -- 172.21.15.42:0/2805436016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a40c85b0 con 0x7f59a41569a0 2024-05-28T14:06:11.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.778+0000 7f59abc00640 1 -- 172.21.15.42:0/2805436016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a4068a90 con 0x7f59a41569a0 2024-05-28T14:06:11.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.782+0000 7f59a0c00640 1 -- 172.21.15.42:0/2805436016 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f599c00abd0 con 0x7f59a41569a0 2024-05-28T14:06:11.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.782+0000 7f59a0c00640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59840690a0 0x7f598406b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:11.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.782+0000 7f59a0c00640 1 -- 172.21.15.42:0/2805436016 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5765+0+0 (secure 0 0 0) 0x7f599c085240 con 0x7f59a41569a0 2024-05-28T14:06:11.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.782+0000 7f59a9200640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59840690a0 0x7f598406b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:11.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.782+0000 7f59a9200640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59840690a0 0x7f598406b560 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f59a40a03c0 tx=0x7f5998008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:11.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.786+0000 7f59a0c00640 1 -- 172.21.15.42:0/2805436016 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f599c057250 con 0x7f59a41569a0 2024-05-28T14:06:11.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.954+0000 7f59abc00640 1 -- 172.21.15.42:0/2805436016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59a40d02b0 con 0x7f59a41569a0 2024-05-28T14:06:11.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.954+0000 7f59a0c00640 1 -- 172.21.15.42:0/2805436016 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16713 (secure 0 0 0) 0x7f599c056b50 con 0x7f59a41569a0 2024-05-28T14:06:11.943 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:11.943 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":63,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:06:07.536649+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:04:20.721059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-28T14:04:21.922127+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":62,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:06:11.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.958+0000 7f59abc00640 1 -- 172.21.15.42:0/2805436016 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59840690a0 msgr2=0x7f598406b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.958+0000 7f59abc00640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59840690a0 0x7f598406b560 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f59a40a03c0 tx=0x7f5998008040 comp rx=0 tx=0).stop 2024-05-28T14:06:11.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.958+0000 7f59abc00640 1 -- 172.21.15.42:0/2805436016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59a41569a0 msgr2=0x7f59a4156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:11.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.958+0000 7f59abc00640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59a41569a0 0x7f59a4156e50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f599c00c7d0 tx=0x7f599c00cc90 comp rx=0 tx=0).stop 2024-05-28T14:06:11.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.958+0000 7f59abc00640 1 -- 172.21.15.42:0/2805436016 shutdown_connections 2024-05-28T14:06:11.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.958+0000 7f59abc00640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59840690a0 0x7f598406b560 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.958+0000 7f59abc00640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59a41569a0 0x7f59a4156e50 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.958+0000 7f59abc00640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59a415c990 0x7f59a406db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.958+0000 7f59abc00640 1 --2- 172.21.15.42:0/2805436016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59a40d06a0 0x7f59a4154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:11.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.958+0000 7f59abc00640 1 -- 172.21.15.42:0/2805436016 >> 172.21.15.42:0/2805436016 conn(0x7f59a40b4080 msgr2=0x7f59a4166db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:11.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.958+0000 7f59abc00640 1 -- 172.21.15.42:0/2805436016 shutdown_connections 2024-05-28T14:06:11.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:11.958+0000 7f59abc00640 1 -- 172.21.15.42:0/2805436016 wait complete. 2024-05-28T14:06:11.966 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-28T14:06:12.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.122+0000 7f6c30000640 1 Processor -- start 2024-05-28T14:06:12.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.122+0000 7f6c30000640 1 -- start start 2024-05-28T14:06:12.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.122+0000 7f6c30000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6c28155a00 0x7f6c2806ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:12.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.122+0000 7f6c30000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c2806cf40 0x7f6c2806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:12.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.122+0000 7f6c30000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c2806fc20 con 0x7f6c2806f3f0 2024-05-28T14:06:12.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.122+0000 7f6c30000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6c2806d380 con 0x7f6c28155a00 2024-05-28T14:06:12.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.122+0000 7f6c30000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2815ff90 con 0x7f6c2806cf40 2024-05-28T14:06:12.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.122+0000 7f6c2e000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f6c2806f3f0 0x7f6c281623c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47792/0 (socket says 172.21.15.42:47792) 2024-05-28T14:06:12.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.122+0000 7f6c2e000640 1 -- 172.21.15.42:0/1149114837 learned_addr learned my addr 172.21.15.42:0/1149114837 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:12.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.126+0000 7f6c2d600640 1 --2- 172.21.15.42:0/1149114837 >> v2:172.21.15.63:3300/0 conn(0x7f6c28155a00 0x7f6c2806ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:12.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.126+0000 7f6c2cc00640 1 --2- 172.21.15.42:0/1149114837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c2806cf40 0x7f6c2806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:12.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.126+0000 7f6c27e00640 1 -- 172.21.15.42:0/1149114837 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3106635602 0 0) 0x7f6c2806fc20 con 0x7f6c2806f3f0 2024-05-28T14:06:12.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.126+0000 7f6c27e00640 1 -- 172.21.15.42:0/1149114837 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c10003530 con 0x7f6c2806f3f0 2024-05-28T14:06:12.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.126+0000 7f6c2d600640 1 -- 172.21.15.42:0/1149114837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c2806cf40 msgr2=0x7f6c2806eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:12.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.126+0000 7f6c2d600640 1 --2- 172.21.15.42:0/1149114837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c2806cf40 0x7f6c2806eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.126+0000 7f6c2d600640 1 -- 172.21.15.42:0/1149114837 >> v1:172.21.15.42:6789/0 conn(0x7f6c2806f3f0 legacy=0x7f6c281623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:12.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.126+0000 7f6c2d600640 1 -- 172.21.15.42:0/1149114837 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c28160270 con 0x7f6c28155a00 2024-05-28T14:06:12.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.126+0000 7f6c2d600640 1 --2- 172.21.15.42:0/1149114837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c28155a00 0x7f6c2806ca00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0097c0 tx=0x7f6c1c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1f92cde33fffa0a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:12.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.126+0000 7f6c27e00640 1 -- 172.21.15.42:0/1149114837 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0b2070 con 0x7f6c28155a00 2024-05-28T14:06:12.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.126+0000 7f6c27e00640 1 -- 172.21.15.42:0/1149114837 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1c0ad640 con 0x7f6c28155a00 2024-05-28T14:06:12.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.126+0000 7f6c27e00640 1 -- 172.21.15.42:0/1149114837 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0ada10 con 0x7f6c28155a00 2024-05-28T14:06:12.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 -- 172.21.15.42:0/1149114837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c28155a00 msgr2=0x7f6c2806ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:12.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 --2- 172.21.15.42:0/1149114837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c28155a00 0x7f6c2806ca00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0097c0 tx=0x7f6c1c01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 -- 172.21.15.42:0/1149114837 shutdown_connections 2024-05-28T14:06:12.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 --2- 172.21.15.42:0/1149114837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c2806cf40 0x7f6c2806eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 --2- 172.21.15.42:0/1149114837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c28155a00 0x7f6c2806ca00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 -- 172.21.15.42:0/1149114837 >> 172.21.15.42:0/1149114837 conn(0x7f6c280725e0 msgr2=0x7f6c280729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:12.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 -- 172.21.15.42:0/1149114837 shutdown_connections 2024-05-28T14:06:12.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 -- 172.21.15.42:0/1149114837 wait complete. 2024-05-28T14:06:12.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 Processor -- start 2024-05-28T14:06:12.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 -- start start 2024-05-28T14:06:12.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c2806cf40 0x7f6c2814a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:12.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c2806f3f0 0x7f6c2814a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:12.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c28155a00 0x7f6c28156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:12.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2816b8e0 con 0x7f6c2806f3f0 2024-05-28T14:06:12.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2816b760 con 0x7f6c2806cf40 2024-05-28T14:06:12.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c30000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2816ba60 con 0x7f6c28155a00 2024-05-28T14:06:12.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c2cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c2806f3f0 0x7f6c2814a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c2cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c2806f3f0 0x7f6c2814a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47714/0 (socket says 172.21.15.42:47714) 2024-05-28T14:06:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c2cc00640 1 -- 172.21.15.42:0/727609434 learned_addr learned my addr 172.21.15.42:0/727609434 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c2d600640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c2806cf40 0x7f6c2814a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c2e000640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c28155a00 0x7f6c28156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c2cc00640 1 -- 172.21.15.42:0/727609434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c28155a00 msgr2=0x7f6c28156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.130+0000 7f6c2cc00640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c28155a00 0x7f6c28156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c2cc00640 1 -- 172.21.15.42:0/727609434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c2806cf40 msgr2=0x7f6c2814a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c2cc00640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c2806cf40 0x7f6c2814a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c2cc00640 1 -- 172.21.15.42:0/727609434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c28154580 con 0x7f6c2806f3f0 2024-05-28T14:06:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c2d600640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c2806cf40 0x7f6c2814a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c2cc00640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c2806f3f0 0x7f6c2814a9a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400e930 tx=0x7f6c1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:12.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c25600640 1 -- 172.21.15.42:0/727609434 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14017070 con 0x7f6c2806f3f0 2024-05-28T14:06:12.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c25600640 1 -- 172.21.15.42:0/727609434 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1400ca00 con 0x7f6c2806f3f0 2024-05-28T14:06:12.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c25600640 1 -- 172.21.15.42:0/727609434 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400cdb0 con 0x7f6c2806f3f0 2024-05-28T14:06:12.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c30000640 1 -- 172.21.15.42:0/727609434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c28154860 con 0x7f6c2806f3f0 2024-05-28T14:06:12.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c30000640 1 -- 172.21.15.42:0/727609434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2814bf30 con 0x7f6c2806f3f0 2024-05-28T14:06:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c30000640 1 -- 172.21.15.42:0/727609434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c28068310 con 0x7f6c2806f3f0 2024-05-28T14:06:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c25600640 1 -- 172.21.15.42:0/727609434 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f6c1401a070 con 0x7f6c2806f3f0 2024-05-28T14:06:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c25600640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c08069070 0x7f6c0806b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c25600640 1 -- 172.21.15.42:0/727609434 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5765+0+0 (secure 0 0 0) 0x7f6c140857c0 con 0x7f6c2806f3f0 2024-05-28T14:06:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.134+0000 7f6c2d600640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c08069070 0x7f6c0806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.138+0000 7f6c2d600640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c08069070 0x7f6c0806b530 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0af000 tx=0x7f6c1c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:12.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.138+0000 7f6c25600640 1 -- 172.21.15.42:0/727609434 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6c14057890 con 0x7f6c2806f3f0 2024-05-28T14:06:12.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.342+0000 7f6c30000640 1 -- 172.21.15.42:0/727609434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6c28073f20 con 0x7f6c2806f3f0 2024-05-28T14:06:12.330 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-05-28T14:06:12.342+0000 7f248b600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-28T14:06:12.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.346+0000 7f6c25600640 1 -- 172.21.15.42:0/727609434 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7f6c14057190 con 0x7f6c2806f3f0 2024-05-28T14:06:12.332 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-28T14:06:12.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.350+0000 7f6c30000640 1 -- 172.21.15.42:0/727609434 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c08069070 msgr2=0x7f6c0806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:12.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.350+0000 7f6c30000640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c08069070 0x7f6c0806b530 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0af000 tx=0x7f6c1c008040 comp rx=0 tx=0).stop 2024-05-28T14:06:12.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.350+0000 7f6c30000640 1 -- 172.21.15.42:0/727609434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c2806f3f0 msgr2=0x7f6c2814a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:12.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.350+0000 7f6c30000640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c2806f3f0 0x7f6c2814a9a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400e930 tx=0x7f6c1400edf0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.350+0000 7f6c30000640 1 -- 172.21.15.42:0/727609434 shutdown_connections 2024-05-28T14:06:12.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.350+0000 7f6c30000640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c08069070 0x7f6c0806b530 secure :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0af000 tx=0x7f6c1c008040 comp rx=0 tx=0).stop 2024-05-28T14:06:12.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.350+0000 7f6c30000640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c28155a00 0x7f6c28156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.350+0000 7f6c30000640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c2806f3f0 0x7f6c2814a9a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.350+0000 7f6c30000640 1 --2- 172.21.15.42:0/727609434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c2806cf40 0x7f6c2814a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.350+0000 7f6c30000640 1 -- 172.21.15.42:0/727609434 >> 172.21.15.42:0/727609434 conn(0x7f6c280725e0 msgr2=0x7f6c280b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:12.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.350+0000 7f6c30000640 1 -- 172.21.15.42:0/727609434 shutdown_connections 2024-05-28T14:06:12.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.350+0000 7f6c30000640 1 -- 172.21.15.42:0/727609434 wait complete. 2024-05-28T14:06:12.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-28T14:06:12.405 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T14:06:12.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.514+0000 7fc011400640 1 Processor -- start 2024-05-28T14:06:12.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.514+0000 7fc011400640 1 -- start start 2024-05-28T14:06:12.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.514+0000 7fc011400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00c144f80 0x7fc00c072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:12.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.514+0000 7fc011400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc00c15c0c0 0x7fc00c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:12.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.514+0000 7fc011400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc00c0afe40 con 0x7fc00c06ee20 2024-05-28T14:06:12.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.514+0000 7fc011400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc00c06f7c0 con 0x7fc00c15c0c0 2024-05-28T14:06:12.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.514+0000 7fc011400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc00c0724d0 con 0x7fc00c144f80 2024-05-28T14:06:12.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.514+0000 7fc00a800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00c144f80 0x7fc00c072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:12.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.514+0000 7fc00a800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00c144f80 0x7fc00c072960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:37432/0 (socket says 172.21.15.42:37432) 2024-05-28T14:06:12.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.514+0000 7fc001e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fc00c06ee20 0x7fc00c15b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:39576/0 (socket says 172.21.15.42:39576) 2024-05-28T14:06:12.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.514+0000 7fc00a800640 1 -- 172.21.15.42:0/1888723972 learned_addr learned my addr 172.21.15.42:0/1888723972 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:12.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.514+0000 7fc00b200640 1 --2- 172.21.15.42:0/1888723972 >> v2:172.21.15.63:3300/0 conn(0x7fc00c15c0c0 0x7fc00c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:12.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.518+0000 7fc00a800640 1 -- 172.21.15.42:0/1888723972 >> v2:172.21.15.63:3300/0 conn(0x7fc00c15c0c0 msgr2=0x7fc00c06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:12.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.518+0000 7fc00a800640 1 --2- 172.21.15.42:0/1888723972 >> v2:172.21.15.63:3300/0 conn(0x7fc00c15c0c0 0x7fc00c06f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.518+0000 7fc00a800640 1 -- 172.21.15.42:0/1888723972 >> v1:172.21.15.63:6789/0 conn(0x7fc00c06ee20 legacy=0x7fc00c15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:12.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.518+0000 7fc00a800640 1 -- 172.21.15.42:0/1888723972 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c15c600 con 0x7fc00c144f80 2024-05-28T14:06:12.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.518+0000 7fc00a800640 1 --2- 172.21.15.42:0/1888723972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00c144f80 0x7fc00c072960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009870 tx=0x7fbffc01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c898e6ea3ddfdc server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:12.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.518+0000 7fc009e00640 1 -- 172.21.15.42:0/1888723972 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0ab650 con 0x7fc00c144f80 2024-05-28T14:06:12.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.518+0000 7fc009e00640 1 -- 172.21.15.42:0/1888723972 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc0b0070 con 0x7fc00c144f80 2024-05-28T14:06:12.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.518+0000 7fc009e00640 1 -- 172.21.15.42:0/1888723972 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0aba10 con 0x7fc00c144f80 2024-05-28T14:06:12.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.518+0000 7fc011400640 1 -- 172.21.15.42:0/1888723972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00c144f80 msgr2=0x7fc00c072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:12.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.518+0000 7fc011400640 1 --2- 172.21.15.42:0/1888723972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00c144f80 0x7fc00c072960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009870 tx=0x7fbffc01c560 comp rx=0 tx=0).stop 2024-05-28T14:06:12.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc011400640 1 -- 172.21.15.42:0/1888723972 shutdown_connections 2024-05-28T14:06:12.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc011400640 1 --2- 172.21.15.42:0/1888723972 >> v2:172.21.15.63:3300/0 conn(0x7fc00c15c0c0 0x7fc00c06f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc011400640 1 --2- 172.21.15.42:0/1888723972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00c144f80 0x7fc00c072960 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc011400640 1 -- 172.21.15.42:0/1888723972 >> 172.21.15.42:0/1888723972 conn(0x7fc00c0b21f0 msgr2=0x7fc00c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:12.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc011400640 1 -- 172.21.15.42:0/1888723972 shutdown_connections 2024-05-28T14:06:12.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc011400640 1 -- 172.21.15.42:0/1888723972 wait complete. 2024-05-28T14:06:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc011400640 1 Processor -- start 2024-05-28T14:06:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc011400640 1 -- start start 2024-05-28T14:06:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc011400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc00c06ee20 0x7fc00c1fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc011400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc00c144f80 0x7fc00c15d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc011400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00c15c0c0 0x7fc00c0b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc011400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c072ea0 con 0x7fc00c06ee20 2024-05-28T14:06:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc011400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c0b4010 con 0x7fc00c144f80 2024-05-28T14:06:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc011400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc00c13d9d0 con 0x7fc00c15c0c0 2024-05-28T14:06:12.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc00a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc00c06ee20 0x7fc00c1fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:12.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc00a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc00c06ee20 0x7fc00c1fa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47718/0 (socket says 172.21.15.42:47718) 2024-05-28T14:06:12.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc00a800640 1 -- 172.21.15.42:0/3876359791 learned_addr learned my addr 172.21.15.42:0/3876359791 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:12.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc001e00640 1 --2- 172.21.15.42:0/3876359791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc00c144f80 0x7fc00c15d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:12.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc00b200640 1 --2- 172.21.15.42:0/3876359791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00c15c0c0 0x7fc00c0b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc00a800640 1 -- 172.21.15.42:0/3876359791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00c15c0c0 msgr2=0x7fc00c0b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc00a800640 1 --2- 172.21.15.42:0/3876359791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00c15c0c0 0x7fc00c0b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc00a800640 1 -- 172.21.15.42:0/3876359791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc00c144f80 msgr2=0x7fc00c15d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc00a800640 1 --2- 172.21.15.42:0/3876359791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc00c144f80 0x7fc00c15d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc00a800640 1 -- 172.21.15.42:0/3876359791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c13dcb0 con 0x7fc00c06ee20 2024-05-28T14:06:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc00a800640 1 --2- 172.21.15.42:0/3876359791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc00c06ee20 0x7fc00c1fa2b0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0ad040 tx=0x7fbffc01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.522+0000 7fc003400640 1 -- 172.21.15.42:0/3876359791 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0be5f0 con 0x7fc00c06ee20 2024-05-28T14:06:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.526+0000 7fc003400640 1 -- 172.21.15.42:0/3876359791 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc0be7b0 con 0x7fc00c06ee20 2024-05-28T14:06:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.526+0000 7fc011400640 1 -- 172.21.15.42:0/3876359791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c13df90 con 0x7fc00c06ee20 2024-05-28T14:06:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.526+0000 7fc003400640 1 -- 172.21.15.42:0/3876359791 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0beba0 con 0x7fc00c06ee20 2024-05-28T14:06:12.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.526+0000 7fc011400640 1 -- 172.21.15.42:0/3876359791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c13e550 con 0x7fc00c06ee20 2024-05-28T14:06:12.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.526+0000 7fc011400640 1 -- 172.21.15.42:0/3876359791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc00c0738b0 con 0x7fc00c06ee20 2024-05-28T14:06:12.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.526+0000 7fc003400640 1 -- 172.21.15.42:0/3876359791 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7fbffc0b43f0 con 0x7fc00c06ee20 2024-05-28T14:06:12.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.526+0000 7fc003400640 1 --2- 172.21.15.42:0/3876359791 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfe8069070 0x7fbfe806b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:12.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.526+0000 7fc003400640 1 -- 172.21.15.42:0/3876359791 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5823+0+0 (secure 0 0 0) 0x7fbffc1201e0 con 0x7fc00c06ee20 2024-05-28T14:06:12.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.526+0000 7fc001e00640 1 --2- 172.21.15.42:0/3876359791 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfe8069070 0x7fbfe806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:12.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.526+0000 7fc001e00640 1 --2- 172.21.15.42:0/3876359791 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfe8069070 0x7fbfe806b530 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc00c069410 tx=0x7fbff4007e90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:12.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.530+0000 7fc003400640 1 -- 172.21.15.42:0/3876359791 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbffc0c4060 con 0x7fc00c06ee20 2024-05-28T14:06:12.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:12.730+0000 7fc011400640 1 -- 172.21.15.42:0/3876359791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc00c1f76a0 con 0x7fc00c06ee20 2024-05-28T14:06:13.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.346+0000 7fc003400640 1 -- 172.21.15.42:0/3876359791 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7fbffc016150 con 0x7fc00c06ee20 2024-05-28T14:06:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.350+0000 7fc011400640 1 -- 172.21.15.42:0/3876359791 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfe8069070 msgr2=0x7fbfe806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.350+0000 7fc011400640 1 --2- 172.21.15.42:0/3876359791 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfe8069070 0x7fbfe806b530 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc00c069410 tx=0x7fbff4007e90 comp rx=0 tx=0).stop 2024-05-28T14:06:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.350+0000 7fc011400640 1 -- 172.21.15.42:0/3876359791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc00c06ee20 msgr2=0x7fc00c1fa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.350+0000 7fc011400640 1 --2- 172.21.15.42:0/3876359791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc00c06ee20 0x7fc00c1fa2b0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0ad040 tx=0x7fbffc01ada0 comp rx=0 tx=0).stop 2024-05-28T14:06:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.350+0000 7fc011400640 1 -- 172.21.15.42:0/3876359791 shutdown_connections 2024-05-28T14:06:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.350+0000 7fc011400640 1 --2- 172.21.15.42:0/3876359791 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfe8069070 0x7fbfe806b530 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.350+0000 7fc011400640 1 --2- 172.21.15.42:0/3876359791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00c15c0c0 0x7fc00c0b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.350+0000 7fc011400640 1 --2- 172.21.15.42:0/3876359791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc00c144f80 0x7fc00c15d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.350+0000 7fc011400640 1 --2- 172.21.15.42:0/3876359791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc00c06ee20 0x7fc00c1fa2b0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.350+0000 7fc011400640 1 -- 172.21.15.42:0/3876359791 >> 172.21.15.42:0/3876359791 conn(0x7fc00c0b21f0 msgr2=0x7fc00c06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.350+0000 7fc011400640 1 -- 172.21.15.42:0/3876359791 shutdown_connections 2024-05-28T14:06:13.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.350+0000 7fc011400640 1 -- 172.21.15.42:0/3876359791 wait complete. 2024-05-28T14:06:13.363 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-28T14:06:13.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc34c000640 1 Processor -- start 2024-05-28T14:06:13.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc34c000640 1 -- start start 2024-05-28T14:06:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc34c000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3440d0b70 0x7fc3440d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc34c000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3440d14d0 con 0x7fc34415c800 2024-05-28T14:06:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc34c000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc34409b7d0 con 0x7fc3440b2a40 2024-05-28T14:06:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc34c000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc3440b25b0 con 0x7fc3440d0b70 2024-05-28T14:06:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc349600640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc34415c800 0x7fc34409aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47808/0 (socket says 172.21.15.42:47808) 2024-05-28T14:06:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc349600640 1 -- 172.21.15.42:0/1478515682 learned_addr learned my addr 172.21.15.42:0/1478515682 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc348c00640 1 --2- 172.21.15.42:0/1478515682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3440d0b70 0x7fc3440d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc343e00640 1 -- 172.21.15.42:0/1478515682 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 103726095 0 0) 0x7fc3440d14d0 con 0x7fc34415c800 2024-05-28T14:06:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc343e00640 1 -- 172.21.15.42:0/1478515682 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc32c003530 con 0x7fc34415c800 2024-05-28T14:06:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc343e00640 1 -- 172.21.15.42:0/1478515682 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 673700378 0 0) 0x7fc34409b7d0 con 0x7fc3440b2a40 2024-05-28T14:06:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc343e00640 1 -- 172.21.15.42:0/1478515682 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3440d14d0 con 0x7fc3440b2a40 2024-05-28T14:06:13.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc343e00640 1 -- 172.21.15.42:0/1478515682 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2462029058 0 0) 0x7fc32c003530 con 0x7fc34415c800 2024-05-28T14:06:13.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc348c00640 1 -- 172.21.15.42:0/1478515682 >> v1:172.21.15.63:6789/0 conn(0x7fc3440b2a40 legacy=0x7fc3440b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:13.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc348c00640 1 -- 172.21.15.42:0/1478515682 >> v1:172.21.15.42:6789/0 conn(0x7fc34415c800 legacy=0x7fc34409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:13.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.526+0000 7fc348c00640 1 -- 172.21.15.42:0/1478515682 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34409b4f0 con 0x7fc3440d0b70 2024-05-28T14:06:13.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.530+0000 7fc348c00640 1 --2- 172.21.15.42:0/1478515682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3440d0b70 0x7fc3440d0f90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc3300097f0 tx=0x7fc33001c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3573ef8fc746420 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:13.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.530+0000 7fc343e00640 1 -- 172.21.15.42:0/1478515682 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3300b0070 con 0x7fc3440d0b70 2024-05-28T14:06:13.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.530+0000 7fc343e00640 1 -- 172.21.15.42:0/1478515682 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc330004030 con 0x7fc3440d0b70 2024-05-28T14:06:13.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.530+0000 7fc343e00640 1 -- 172.21.15.42:0/1478515682 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc338001680 con 0x7fc3440d0b70 2024-05-28T14:06:13.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.530+0000 7fc34c000640 1 -- 172.21.15.42:0/1478515682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3440d0b70 msgr2=0x7fc3440d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:13.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.530+0000 7fc34c000640 1 --2- 172.21.15.42:0/1478515682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3440d0b70 0x7fc3440d0f90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc3300097f0 tx=0x7fc33001c3d0 comp rx=0 tx=0).stop 2024-05-28T14:06:13.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.530+0000 7fc34c000640 1 -- 172.21.15.42:0/1478515682 shutdown_connections 2024-05-28T14:06:13.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.530+0000 7fc34c000640 1 --2- 172.21.15.42:0/1478515682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3440d0b70 0x7fc3440d0f90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:13.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.530+0000 7fc34c000640 1 -- 172.21.15.42:0/1478515682 >> 172.21.15.42:0/1478515682 conn(0x7fc3440b4080 msgr2=0x7fc3440b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:13.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.530+0000 7fc34c000640 1 -- 172.21.15.42:0/1478515682 shutdown_connections 2024-05-28T14:06:13.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.530+0000 7fc34c000640 1 -- 172.21.15.42:0/1478515682 wait complete. 2024-05-28T14:06:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34c000640 1 Processor -- start 2024-05-28T14:06:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34c000640 1 -- start start 2024-05-28T14:06:13.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34c000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3440b2a40 0x7fc34414a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:13.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34c000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3440d0b70 0x7fc3440b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:13.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34c000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc34415c800 0x7fc3441563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:13.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34c000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc34415cbe0 con 0x7fc34415c800 2024-05-28T14:06:13.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34c000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc34409b160 con 0x7fc3440b2a40 2024-05-28T14:06:13.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34c000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc34415cd60 con 0x7fc3440d0b70 2024-05-28T14:06:13.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc34415c800 0x7fc3441563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:13.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc34415c800 0x7fc3441563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47722/0 (socket says 172.21.15.42:47722) 2024-05-28T14:06:13.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34a000640 1 -- 172.21.15.42:0/1625234197 learned_addr learned my addr 172.21.15.42:0/1625234197 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:13.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc349600640 1 --2- 172.21.15.42:0/1625234197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3440b2a40 0x7fc34414a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:13.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc348c00640 1 --2- 172.21.15.42:0/1625234197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3440d0b70 0x7fc3440b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:13.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34a000640 1 -- 172.21.15.42:0/1625234197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3440d0b70 msgr2=0x7fc3440b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:13.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34a000640 1 --2- 172.21.15.42:0/1625234197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3440d0b70 0x7fc3440b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:13.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34a000640 1 -- 172.21.15.42:0/1625234197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3440b2a40 msgr2=0x7fc34414a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:13.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34a000640 1 --2- 172.21.15.42:0/1625234197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3440b2a40 0x7fc34414a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:13.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34a000640 1 -- 172.21.15.42:0/1625234197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3440b7e90 con 0x7fc34415c800 2024-05-28T14:06:13.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34a000640 1 --2- 172.21.15.42:0/1625234197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc34415c800 0x7fc3441563c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00c760 tx=0x7fc33c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:13.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc341600640 1 -- 172.21.15.42:0/1625234197 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c003b80 con 0x7fc34415c800 2024-05-28T14:06:13.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc341600640 1 -- 172.21.15.42:0/1625234197 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc33c003d20 con 0x7fc34415c800 2024-05-28T14:06:13.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34c000640 1 -- 172.21.15.42:0/1625234197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3440c6530 con 0x7fc34415c800 2024-05-28T14:06:13.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc341600640 1 -- 172.21.15.42:0/1625234197 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c00aa90 con 0x7fc34415c800 2024-05-28T14:06:13.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.534+0000 7fc34c000640 1 -- 172.21.15.42:0/1625234197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3440c6af0 con 0x7fc34415c800 2024-05-28T14:06:13.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.538+0000 7fc34c000640 1 -- 172.21.15.42:0/1625234197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc344068a90 con 0x7fc34415c800 2024-05-28T14:06:13.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.538+0000 7fc341600640 1 -- 172.21.15.42:0/1625234197 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7fc33c00ac30 con 0x7fc34415c800 2024-05-28T14:06:13.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.538+0000 7fc341600640 1 --2- 172.21.15.42:0/1625234197 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc324069070 0x7fc32406b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:13.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.538+0000 7fc341600640 1 -- 172.21.15.42:0/1625234197 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5823+0+0 (secure 0 0 0) 0x7fc33c085170 con 0x7fc34415c800 2024-05-28T14:06:13.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.538+0000 7fc349600640 1 --2- 172.21.15.42:0/1625234197 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc324069070 0x7fc32406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:13.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.538+0000 7fc349600640 1 --2- 172.21.15.42:0/1625234197 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc324069070 0x7fc32406b530 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc3380011f0 tx=0x7fc338001680 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:13.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.542+0000 7fc341600640 1 -- 172.21.15.42:0/1625234197 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc33c056b30 con 0x7fc34415c800 2024-05-28T14:06:13.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:13.710+0000 7fc34c000640 1 -- 172.21.15.42:0/1625234197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc3440cc9c0 con 0x7fc34415c800 2024-05-28T14:06:14.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.350+0000 7fc341600640 1 -- 172.21.15.42:0/1625234197 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v65) ==== 181+0+0 (secure 0 0 0) 0x7fc33c05ae30 con 0x7fc34415c800 2024-05-28T14:06:14.336 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-05-28T14:06:14.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.350+0000 7fc34c000640 1 -- 172.21.15.42:0/1625234197 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc324069070 msgr2=0x7fc32406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:14.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.350+0000 7fc34c000640 1 --2- 172.21.15.42:0/1625234197 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc324069070 0x7fc32406b530 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc3380011f0 tx=0x7fc338001680 comp rx=0 tx=0).stop 2024-05-28T14:06:14.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.350+0000 7fc34c000640 1 -- 172.21.15.42:0/1625234197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc34415c800 msgr2=0x7fc3441563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:14.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.350+0000 7fc34c000640 1 --2- 172.21.15.42:0/1625234197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc34415c800 0x7fc3441563c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00c760 tx=0x7fc33c00cc20 comp rx=0 tx=0).stop 2024-05-28T14:06:14.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.354+0000 7fc34c000640 1 -- 172.21.15.42:0/1625234197 shutdown_connections 2024-05-28T14:06:14.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.354+0000 7fc34c000640 1 --2- 172.21.15.42:0/1625234197 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc324069070 0x7fc32406b530 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:14.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.354+0000 7fc34c000640 1 --2- 172.21.15.42:0/1625234197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc34415c800 0x7fc3441563c0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:14.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.354+0000 7fc34c000640 1 --2- 172.21.15.42:0/1625234197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3440d0b70 0x7fc3440b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:14.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.354+0000 7fc34c000640 1 --2- 172.21.15.42:0/1625234197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3440b2a40 0x7fc34414a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:14.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.354+0000 7fc34c000640 1 -- 172.21.15.42:0/1625234197 >> 172.21.15.42:0/1625234197 conn(0x7fc3440b4080 msgr2=0x7fc344071310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:14.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.354+0000 7fc34c000640 1 -- 172.21.15.42:0/1625234197 shutdown_connections 2024-05-28T14:06:14.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.354+0000 7fc34c000640 1 -- 172.21.15.42:0/1625234197 wait complete. 2024-05-28T14:06:14.367 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-28T14:06:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.530+0000 7f436e800640 1 Processor -- start 2024-05-28T14:06:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.530+0000 7f436e800640 1 -- start start 2024-05-28T14:06:14.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.530+0000 7f436e800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f43681482d0 0x7f43680b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.530+0000 7f436e800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f43680b4760 0x7f43680d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.530+0000 7f436e800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43680ada00 con 0x7f43680d0b30 2024-05-28T14:06:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.530+0000 7f436e800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f43680d0fa0 con 0x7f43680b4760 2024-05-28T14:06:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.530+0000 7f436e800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f43680b4ba0 con 0x7f43681482d0 2024-05-28T14:06:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.530+0000 7f435ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f43680b4760 0x7f43680d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.530+0000 7f4367e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f43680d0b30 0x7f43680720c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47820/0 (socket says 172.21.15.42:47820) 2024-05-28T14:06:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.530+0000 7f4367e00640 1 -- 172.21.15.42:0/4095770824 learned_addr learned my addr 172.21.15.42:0/4095770824 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f4367400640 1 --2- 172.21.15.42:0/4095770824 >> v2:172.21.15.63:3300/0 conn(0x7f43681482d0 0x7f43680b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f435ea00640 1 -- 172.21.15.42:0/4095770824 >> v2:172.21.15.63:3300/0 conn(0x7f43681482d0 msgr2=0x7f43680b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f435ea00640 1 --2- 172.21.15.42:0/4095770824 >> v2:172.21.15.63:3300/0 conn(0x7f43681482d0 0x7f43680b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f435ea00640 1 -- 172.21.15.42:0/4095770824 >> v1:172.21.15.42:6789/0 conn(0x7f43680d0b30 legacy=0x7f43680720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f435ea00640 1 -- 172.21.15.42:0/4095770824 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436806f470 con 0x7f43680b4760 2024-05-28T14:06:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f4366a00640 1 -- 172.21.15.42:0/4095770824 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2468382262 0 0) 0x7f43600014a0 con 0x7f43680d0b30 2024-05-28T14:06:14.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f435ea00640 1 --2- 172.21.15.42:0/4095770824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43680b4760 0x7f43680d05f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f436806dae0 tx=0x7f435001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c52f4ac0330129b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:14.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f4366a00640 1 -- 172.21.15.42:0/4095770824 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43500b0070 con 0x7f43680b4760 2024-05-28T14:06:14.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f4366a00640 1 -- 172.21.15.42:0/4095770824 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43500ab600 con 0x7f43680b4760 2024-05-28T14:06:14.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f4366a00640 1 -- 172.21.15.42:0/4095770824 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43500ab9d0 con 0x7f43680b4760 2024-05-28T14:06:14.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f436e800640 1 -- 172.21.15.42:0/4095770824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43680b4760 msgr2=0x7f43680d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:14.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f436e800640 1 --2- 172.21.15.42:0/4095770824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43680b4760 0x7f43680d05f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f436806dae0 tx=0x7f435001c480 comp rx=0 tx=0).stop 2024-05-28T14:06:14.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f436e800640 1 -- 172.21.15.42:0/4095770824 shutdown_connections 2024-05-28T14:06:14.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f436e800640 1 --2- 172.21.15.42:0/4095770824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43680b4760 0x7f43680d05f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:14.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f436e800640 1 --2- 172.21.15.42:0/4095770824 >> v2:172.21.15.63:3300/0 conn(0x7f43681482d0 0x7f43680b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:14.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.534+0000 7f436e800640 1 -- 172.21.15.42:0/4095770824 >> 172.21.15.42:0/4095770824 conn(0x7f436806b9f0 msgr2=0x7f436806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:14.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f436e800640 1 -- 172.21.15.42:0/4095770824 shutdown_connections 2024-05-28T14:06:14.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f436e800640 1 -- 172.21.15.42:0/4095770824 wait complete. 2024-05-28T14:06:14.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f436e800640 1 Processor -- start 2024-05-28T14:06:14.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f436e800640 1 -- start start 2024-05-28T14:06:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f436e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43680b4760 0x7f436813e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f436e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43680d0b30 0x7f436813fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f436e800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43681482d0 0x7f4368140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f436e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f436806f160 con 0x7f43680b4760 2024-05-28T14:06:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f436e800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f43681486b0 con 0x7f43680d0b30 2024-05-28T14:06:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f436e800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f436816b460 con 0x7f43681482d0 2024-05-28T14:06:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f4367400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43680b4760 0x7f436813e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f4367400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43680b4760 0x7f436813e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47738/0 (socket says 172.21.15.42:47738) 2024-05-28T14:06:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f4367400640 1 -- 172.21.15.42:0/2181524655 learned_addr learned my addr 172.21.15.42:0/2181524655 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f435ea00640 1 --2- 172.21.15.42:0/2181524655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43680d0b30 0x7f436813fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f4367e00640 1 --2- 172.21.15.42:0/2181524655 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43681482d0 0x7f4368140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f4367400640 1 -- 172.21.15.42:0/2181524655 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43681482d0 msgr2=0x7f4368140080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f4367400640 1 --2- 172.21.15.42:0/2181524655 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43681482d0 0x7f4368140080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f4367400640 1 -- 172.21.15.42:0/2181524655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43680d0b30 msgr2=0x7f436813fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:14.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f4367400640 1 --2- 172.21.15.42:0/2181524655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43680d0b30 0x7f436813fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:14.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f4367400640 1 -- 172.21.15.42:0/2181524655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436813e890 con 0x7f43680b4760 2024-05-28T14:06:14.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f4367400640 1 --2- 172.21.15.42:0/2181524655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43680b4760 0x7f436813e350 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f435800d5f0 tx=0x7f435800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:14.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.538+0000 7f435fe00640 1 -- 172.21.15.42:0/2181524655 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43580125f0 con 0x7f43680b4760 2024-05-28T14:06:14.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.542+0000 7f435fe00640 1 -- 172.21.15.42:0/2181524655 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4358019070 con 0x7f43680b4760 2024-05-28T14:06:14.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.542+0000 7f435fe00640 1 -- 172.21.15.42:0/2181524655 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43580129b0 con 0x7f43680b4760 2024-05-28T14:06:14.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.542+0000 7f436e800640 1 -- 172.21.15.42:0/2181524655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4368145840 con 0x7f43680b4760 2024-05-28T14:06:14.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.542+0000 7f436e800640 1 -- 172.21.15.42:0/2181524655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4368145e00 con 0x7f43680b4760 2024-05-28T14:06:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.542+0000 7f436e800640 1 -- 172.21.15.42:0/2181524655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f436806c9f0 con 0x7f43680b4760 2024-05-28T14:06:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.542+0000 7f435fe00640 1 -- 172.21.15.42:0/2181524655 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f4358012b50 con 0x7f43680b4760 2024-05-28T14:06:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.542+0000 7f435fe00640 1 --2- 172.21.15.42:0/2181524655 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4344069070 0x7f434406b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.542+0000 7f435fe00640 1 -- 172.21.15.42:0/2181524655 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5385+0+0 (secure 0 0 0) 0x7f4358085a80 con 0x7f43680b4760 2024-05-28T14:06:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.542+0000 7f435ea00640 1 --2- 172.21.15.42:0/2181524655 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4344069070 0x7f434406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:14.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.542+0000 7f435ea00640 1 --2- 172.21.15.42:0/2181524655 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4344069070 0x7f434406b530 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f43500ad000 tx=0x7f4350008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:14.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.546+0000 7f435fe00640 1 -- 172.21.15.42:0/2181524655 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4358057800 con 0x7f43680b4760 2024-05-28T14:06:14.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:14.714+0000 7f436e800640 1 -- 172.21.15.42:0/2181524655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4368068a90 con 0x7f43680b4760 2024-05-28T14:06:15.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.354+0000 7f435fe00640 1 -- 172.21.15.42:0/2181524655 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v66) ==== 169+0+0 (secure 0 0 0) 0x7f4358057100 con 0x7f43680b4760 2024-05-28T14:06:15.342 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-05-28T14:06:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.358+0000 7f436e800640 1 -- 172.21.15.42:0/2181524655 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4344069070 msgr2=0x7f434406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.358+0000 7f436e800640 1 --2- 172.21.15.42:0/2181524655 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4344069070 0x7f434406b530 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f43500ad000 tx=0x7f4350008040 comp rx=0 tx=0).stop 2024-05-28T14:06:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.358+0000 7f436e800640 1 -- 172.21.15.42:0/2181524655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43680b4760 msgr2=0x7f436813e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.358+0000 7f436e800640 1 --2- 172.21.15.42:0/2181524655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43680b4760 0x7f436813e350 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f435800d5f0 tx=0x7f435800dab0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.358+0000 7f436e800640 1 -- 172.21.15.42:0/2181524655 shutdown_connections 2024-05-28T14:06:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.358+0000 7f436e800640 1 --2- 172.21.15.42:0/2181524655 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4344069070 0x7f434406b530 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.358+0000 7f436e800640 1 --2- 172.21.15.42:0/2181524655 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43681482d0 0x7f4368140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.358+0000 7f436e800640 1 --2- 172.21.15.42:0/2181524655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43680d0b30 0x7f436813fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.358+0000 7f436e800640 1 --2- 172.21.15.42:0/2181524655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43680b4760 0x7f436813e350 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.358+0000 7f436e800640 1 -- 172.21.15.42:0/2181524655 >> 172.21.15.42:0/2181524655 conn(0x7f436806b9f0 msgr2=0x7f43681418e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:15.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.362+0000 7f436e800640 1 -- 172.21.15.42:0/2181524655 shutdown_connections 2024-05-28T14:06:15.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.362+0000 7f436e800640 1 -- 172.21.15.42:0/2181524655 wait complete. 2024-05-28T14:06:15.380 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-28T14:06:15.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.542+0000 7fd841800640 1 Processor -- start 2024-05-28T14:06:15.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.542+0000 7fd841800640 1 -- start start 2024-05-28T14:06:15.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.542+0000 7fd841800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd83c15c0c0 0x7fd83c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:15.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.542+0000 7fd841800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd83c0afe40 con 0x7fd83c06ee20 2024-05-28T14:06:15.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.542+0000 7fd841800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd83c15c600 con 0x7fd83c144f80 2024-05-28T14:06:15.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.542+0000 7fd841800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd83c06f7c0 con 0x7fd83c15c0c0 2024-05-28T14:06:15.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.542+0000 7fd83b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd83c15c0c0 0x7fd83c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:15.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.542+0000 7fd83b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd83c15c0c0 0x7fd83c06f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47740/0 (socket says 172.21.15.42:47740) 2024-05-28T14:06:15.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.542+0000 7fd83b400640 1 -- 172.21.15.42:0/1417042320 learned_addr learned my addr 172.21.15.42:0/1417042320 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:15.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.542+0000 7fd833e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd83c06ee20 0x7fd83c15b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47830/0 (socket says 172.21.15.42:47830) 2024-05-28T14:06:15.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.542+0000 7fd83b400640 1 -- 172.21.15.42:0/1417042320 >> v1:172.21.15.63:6789/0 conn(0x7fd83c144f80 legacy=0x7fd83c072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:15.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.542+0000 7fd83b400640 1 -- 172.21.15.42:0/1417042320 >> v1:172.21.15.42:6789/0 conn(0x7fd83c06ee20 legacy=0x7fd83c15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:15.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.542+0000 7fd83b400640 1 -- 172.21.15.42:0/1417042320 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83c15c8e0 con 0x7fd83c15c0c0 2024-05-28T14:06:15.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.542+0000 7fd83a000640 1 -- 172.21.15.42:0/1417042320 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 409717995 0 0) 0x7fd83c0afe40 con 0x7fd83c06ee20 2024-05-28T14:06:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.546+0000 7fd83b400640 1 --2- 172.21.15.42:0/1417042320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c15c0c0 0x7fd83c06f280 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd834009e50 tx=0x7fd83401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77fc37debfbcc6e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.546+0000 7fd83a000640 1 -- 172.21.15.42:0/1417042320 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340a5700 con 0x7fd83c15c0c0 2024-05-28T14:06:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.546+0000 7fd83a000640 1 -- 172.21.15.42:0/1417042320 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8340a58a0 con 0x7fd83c15c0c0 2024-05-28T14:06:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.546+0000 7fd83a000640 1 -- 172.21.15.42:0/1417042320 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340a5c70 con 0x7fd83c15c0c0 2024-05-28T14:06:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.546+0000 7fd841800640 1 -- 172.21.15.42:0/1417042320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c15c0c0 msgr2=0x7fd83c06f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.546+0000 7fd841800640 1 --2- 172.21.15.42:0/1417042320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c15c0c0 0x7fd83c06f280 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd834009e50 tx=0x7fd83401ca30 comp rx=0 tx=0).stop 2024-05-28T14:06:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.546+0000 7fd841800640 1 -- 172.21.15.42:0/1417042320 shutdown_connections 2024-05-28T14:06:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.546+0000 7fd841800640 1 --2- 172.21.15.42:0/1417042320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c15c0c0 0x7fd83c06f280 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.546+0000 7fd841800640 1 -- 172.21.15.42:0/1417042320 >> 172.21.15.42:0/1417042320 conn(0x7fd83c0b21f0 msgr2=0x7fd83c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.546+0000 7fd841800640 1 -- 172.21.15.42:0/1417042320 shutdown_connections 2024-05-28T14:06:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.546+0000 7fd841800640 1 -- 172.21.15.42:0/1417042320 wait complete. 2024-05-28T14:06:15.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.546+0000 7fd841800640 1 Processor -- start 2024-05-28T14:06:15.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.546+0000 7fd841800640 1 -- start start 2024-05-28T14:06:15.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd841800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd83c06ee20 0x7fd83c13cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:15.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd841800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd83c144f80 0x7fd83c13e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:15.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd841800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c15c0c0 0x7fd83c1fa850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:15.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd841800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c0724d0 con 0x7fd83c15c0c0 2024-05-28T14:06:15.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd841800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c0b4260 con 0x7fd83c144f80 2024-05-28T14:06:15.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd841800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd83c169140 con 0x7fd83c06ee20 2024-05-28T14:06:15.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd83b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c15c0c0 0x7fd83c1fa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:15.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd83b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c15c0c0 0x7fd83c1fa850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47744/0 (socket says 172.21.15.42:47744) 2024-05-28T14:06:15.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd83b400640 1 -- 172.21.15.42:0/2553621920 learned_addr learned my addr 172.21.15.42:0/2553621920 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:15.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd833e00640 1 --2- 172.21.15.42:0/2553621920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd83c144f80 0x7fd83c13e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:15.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd83b400640 1 -- 172.21.15.42:0/2553621920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd83c06ee20 msgr2=0x7fd83c13cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:15.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd83b400640 1 --2- 172.21.15.42:0/2553621920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd83c06ee20 0x7fd83c13cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd83b400640 1 -- 172.21.15.42:0/2553621920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd83c144f80 msgr2=0x7fd83c13e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:15.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd83b400640 1 --2- 172.21.15.42:0/2553621920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd83c144f80 0x7fd83c13e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd83b400640 1 -- 172.21.15.42:0/2553621920 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83c1faf80 con 0x7fd83c15c0c0 2024-05-28T14:06:15.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd833e00640 1 --2- 172.21.15.42:0/2553621920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd83c144f80 0x7fd83c13e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:15.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd83b400640 1 --2- 172.21.15.42:0/2553621920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c15c0c0 0x7fd83c1fa850 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd8340af040 tx=0x7fd8340197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:15.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd832a00640 1 -- 172.21.15.42:0/2553621920 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340bc020 con 0x7fd83c15c0c0 2024-05-28T14:06:15.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd832a00640 1 -- 172.21.15.42:0/2553621920 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8340b8670 con 0x7fd83c15c0c0 2024-05-28T14:06:15.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd832a00640 1 -- 172.21.15.42:0/2553621920 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340b8a60 con 0x7fd83c15c0c0 2024-05-28T14:06:15.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd841800640 1 -- 172.21.15.42:0/2553621920 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83c1fb260 con 0x7fd83c15c0c0 2024-05-28T14:06:15.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd841800640 1 -- 172.21.15.42:0/2553621920 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd83c1586c0 con 0x7fd83c15c0c0 2024-05-28T14:06:15.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.550+0000 7fd841800640 1 -- 172.21.15.42:0/2553621920 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd83c0738b0 con 0x7fd83c15c0c0 2024-05-28T14:06:15.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.554+0000 7fd832a00640 1 -- 172.21.15.42:0/2553621920 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7fd8340bf020 con 0x7fd83c15c0c0 2024-05-28T14:06:15.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.554+0000 7fd832a00640 1 --2- 172.21.15.42:0/2553621920 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd818069070 0x7fd81806b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.554+0000 7fd832a00640 1 -- 172.21.15.42:0/2553621920 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4971+0+0 (secure 0 0 0) 0x7fd8341206a0 con 0x7fd83c15c0c0 2024-05-28T14:06:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.554+0000 7fd83aa00640 1 --2- 172.21.15.42:0/2553621920 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd818069070 0x7fd81806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.554+0000 7fd83aa00640 1 --2- 172.21.15.42:0/2553621920 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd818069070 0x7fd81806b530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd82c000f70 tx=0x7fd82c0079c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:15.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.558+0000 7fd832a00640 1 -- 172.21.15.42:0/2553621920 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd8340c6030 con 0x7fd83c15c0c0 2024-05-28T14:06:15.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.762+0000 7fd841800640 1 -- 172.21.15.42:0/2553621920 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd83c15eb60 con 0x7fd83c15c0c0 2024-05-28T14:06:15.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.766+0000 7fd832a00640 1 -- 172.21.15.42:0/2553621920 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v16) ==== 104+0+0 (secure 0 0 0) 0x7fd834016620 con 0x7fd83c15c0c0 2024-05-28T14:06:15.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.770+0000 7fd841800640 1 -- 172.21.15.42:0/2553621920 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd818069070 msgr2=0x7fd81806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:15.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.770+0000 7fd841800640 1 --2- 172.21.15.42:0/2553621920 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd818069070 0x7fd81806b530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd82c000f70 tx=0x7fd82c0079c0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.770+0000 7fd841800640 1 -- 172.21.15.42:0/2553621920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c15c0c0 msgr2=0x7fd83c1fa850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:15.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.770+0000 7fd841800640 1 --2- 172.21.15.42:0/2553621920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c15c0c0 0x7fd83c1fa850 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd8340af040 tx=0x7fd8340197d0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.770+0000 7fd841800640 1 -- 172.21.15.42:0/2553621920 shutdown_connections 2024-05-28T14:06:15.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.770+0000 7fd841800640 1 --2- 172.21.15.42:0/2553621920 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd818069070 0x7fd81806b530 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.770+0000 7fd841800640 1 --2- 172.21.15.42:0/2553621920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c15c0c0 0x7fd83c1fa850 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.770+0000 7fd841800640 1 --2- 172.21.15.42:0/2553621920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd83c144f80 0x7fd83c13e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.770+0000 7fd841800640 1 --2- 172.21.15.42:0/2553621920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd83c06ee20 0x7fd83c13cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.770+0000 7fd841800640 1 -- 172.21.15.42:0/2553621920 >> 172.21.15.42:0/2553621920 conn(0x7fd83c0b21f0 msgr2=0x7fd83c0cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:15.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.770+0000 7fd841800640 1 -- 172.21.15.42:0/2553621920 shutdown_connections 2024-05-28T14:06:15.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.770+0000 7fd841800640 1 -- 172.21.15.42:0/2553621920 wait complete. 2024-05-28T14:06:15.775 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/snapshot_clone_no_wait 2024-05-28T14:06:15.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa8f200640 1 Processor -- start 2024-05-28T14:06:15.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa8f200640 1 -- start start 2024-05-28T14:06:15.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa8f200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa900cf540 0x7ffa900cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:15.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa8f200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffa900707e0 0x7ffa900cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:15.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa8f200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa90160170 con 0x7ffa90145d50 2024-05-28T14:06:15.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa8f200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ffa90160450 con 0x7ffa900707e0 2024-05-28T14:06:15.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa8f200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa90160730 con 0x7ffa900cf540 2024-05-28T14:06:15.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa8e800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffa900707e0 0x7ffa900cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:15.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa87e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ffa90145d50 0x7ffa90071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47840/0 (socket says 172.21.15.42:47840) 2024-05-28T14:06:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa87e00640 1 -- 172.21.15.42:0/1661604416 learned_addr learned my addr 172.21.15.42:0/1661604416 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa8de00640 1 --2- 172.21.15.42:0/1661604416 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa900cf540 0x7ffa900cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa8d400640 1 -- 172.21.15.42:0/1661604416 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3548702378 0 0) 0x7ffa90160170 con 0x7ffa90145d50 2024-05-28T14:06:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa8e800640 1 -- 172.21.15.42:0/1661604416 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa900cf540 msgr2=0x7ffa900cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa8e800640 1 --2- 172.21.15.42:0/1661604416 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa900cf540 0x7ffa900cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa8e800640 1 -- 172.21.15.42:0/1661604416 >> v1:172.21.15.42:6789/0 conn(0x7ffa90145d50 legacy=0x7ffa90071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.934+0000 7ffa8e800640 1 -- 172.21.15.42:0/1661604416 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa900d0ed0 con 0x7ffa900707e0 2024-05-28T14:06:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.938+0000 7ffa8e800640 1 --2- 172.21.15.42:0/1661604416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa900707e0 0x7ffa900cce80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009d90 tx=0x7ffa8801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a35f85e31d8776 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.938+0000 7ffa8d400640 1 -- 172.21.15.42:0/1661604416 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa880a8620 con 0x7ffa900707e0 2024-05-28T14:06:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.938+0000 7ffa8d400640 1 -- 172.21.15.42:0/1661604416 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa880a87c0 con 0x7ffa900707e0 2024-05-28T14:06:15.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.938+0000 7ffa8d400640 1 -- 172.21.15.42:0/1661604416 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa880a8b90 con 0x7ffa900707e0 2024-05-28T14:06:15.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.938+0000 7ffa8f200640 1 -- 172.21.15.42:0/1661604416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa900707e0 msgr2=0x7ffa900cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:15.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.938+0000 7ffa8f200640 1 --2- 172.21.15.42:0/1661604416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa900707e0 0x7ffa900cce80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009d90 tx=0x7ffa8801c8e0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.938+0000 7ffa8f200640 1 -- 172.21.15.42:0/1661604416 shutdown_connections 2024-05-28T14:06:15.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.938+0000 7ffa8f200640 1 --2- 172.21.15.42:0/1661604416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa900707e0 0x7ffa900cce80 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.938+0000 7ffa8f200640 1 --2- 172.21.15.42:0/1661604416 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa900cf540 0x7ffa900cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.938+0000 7ffa8f200640 1 -- 172.21.15.42:0/1661604416 >> 172.21.15.42:0/1661604416 conn(0x7ffa9006fcc0 msgr2=0x7ffa900700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:15.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.938+0000 7ffa8f200640 1 -- 172.21.15.42:0/1661604416 shutdown_connections 2024-05-28T14:06:15.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.938+0000 7ffa8f200640 1 -- 172.21.15.42:0/1661604416 wait complete. 2024-05-28T14:06:15.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.938+0000 7ffa8f200640 1 Processor -- start 2024-05-28T14:06:15.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.938+0000 7ffa8f200640 1 -- start start 2024-05-28T14:06:15.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa8f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa900707e0 0x7ffa9015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:15.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa8f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa900cf540 0x7ffa9015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:15.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa8f200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa90145d50 0x7ffa90153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:15.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa8f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffa90169350 con 0x7ffa900cf540 2024-05-28T14:06:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa8f200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa900d1100 con 0x7ffa900707e0 2024-05-28T14:06:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa8f200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa901694d0 con 0x7ffa90145d50 2024-05-28T14:06:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa87e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa900cf540 0x7ffa9015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa87e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa900cf540 0x7ffa9015c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47764/0 (socket says 172.21.15.42:47764) 2024-05-28T14:06:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa87e00640 1 -- 172.21.15.42:0/2550302972 learned_addr learned my addr 172.21.15.42:0/2550302972 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa8de00640 1 --2- 172.21.15.42:0/2550302972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa900707e0 0x7ffa9015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa8e800640 1 --2- 172.21.15.42:0/2550302972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa90145d50 0x7ffa90153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa87e00640 1 -- 172.21.15.42:0/2550302972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa90145d50 msgr2=0x7ffa90153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa87e00640 1 --2- 172.21.15.42:0/2550302972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa90145d50 0x7ffa90153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa87e00640 1 -- 172.21.15.42:0/2550302972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa900707e0 msgr2=0x7ffa9015bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:15.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa87e00640 1 --2- 172.21.15.42:0/2550302972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa900707e0 0x7ffa9015bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:15.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa87e00640 1 -- 172.21.15.42:0/2550302972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9015c440 con 0x7ffa900cf540 2024-05-28T14:06:15.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa87e00640 1 --2- 172.21.15.42:0/2550302972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa900cf540 0x7ffa9015c7b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800c450 tx=0x7ffa7800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:15.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa86000640 1 -- 172.21.15.42:0/2550302972 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78018070 con 0x7ffa900cf540 2024-05-28T14:06:15.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa86000640 1 -- 172.21.15.42:0/2550302972 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa780074e0 con 0x7ffa900cf540 2024-05-28T14:06:15.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa86000640 1 -- 172.21.15.42:0/2550302972 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa780049f0 con 0x7ffa900cf540 2024-05-28T14:06:15.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa8f200640 1 -- 172.21.15.42:0/2550302972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa9015ccf0 con 0x7ffa900cf540 2024-05-28T14:06:15.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa8f200640 1 -- 172.21.15.42:0/2550302972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa9015b420 con 0x7ffa900cf540 2024-05-28T14:06:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.942+0000 7ffa8f200640 1 -- 172.21.15.42:0/2550302972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa9004fcf0 con 0x7ffa900cf540 2024-05-28T14:06:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.946+0000 7ffa86000640 1 -- 172.21.15.42:0/2550302972 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7ffa7801b070 con 0x7ffa900cf540 2024-05-28T14:06:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.946+0000 7ffa86000640 1 --2- 172.21.15.42:0/2550302972 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa6c069070 0x7ffa6c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.946+0000 7ffa86000640 1 -- 172.21.15.42:0/2550302972 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4971+0+0 (secure 0 0 0) 0x7ffa780846c0 con 0x7ffa900cf540 2024-05-28T14:06:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.946+0000 7ffa8de00640 1 --2- 172.21.15.42:0/2550302972 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa6c069070 0x7ffa6c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.946+0000 7ffa8de00640 1 --2- 172.21.15.42:0/2550302972 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa6c069070 0x7ffa6c06b530 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffa80009b60 tx=0x7ffa80009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:15.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:15.950+0000 7ffa86000640 1 -- 172.21.15.42:0/2550302972 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffa78056080 con 0x7ffa900cf540 2024-05-28T14:06:16.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.114+0000 7ffa8f200640 1 -- 172.21.15.42:0/2550302972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_no_wait"} v 0) -- 0x7ffa900702f0 con 0x7ffa900cf540 2024-05-28T14:06:16.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.118+0000 7ffa86000640 1 -- 172.21.15.42:0/2550302972 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_no_wait"}]=0 v17) ==== 117+0+0 (secure 0 0 0) 0x7ffa7805a380 con 0x7ffa900cf540 2024-05-28T14:06:16.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.122+0000 7ffa8f200640 1 -- 172.21.15.42:0/2550302972 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa6c069070 msgr2=0x7ffa6c06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:16.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.122+0000 7ffa8f200640 1 --2- 172.21.15.42:0/2550302972 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa6c069070 0x7ffa6c06b530 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffa80009b60 tx=0x7ffa80009190 comp rx=0 tx=0).stop 2024-05-28T14:06:16.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.122+0000 7ffa8f200640 1 -- 172.21.15.42:0/2550302972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa900cf540 msgr2=0x7ffa9015c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:16.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.122+0000 7ffa8f200640 1 --2- 172.21.15.42:0/2550302972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa900cf540 0x7ffa9015c7b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800c450 tx=0x7ffa7800c910 comp rx=0 tx=0).stop 2024-05-28T14:06:16.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.122+0000 7ffa8f200640 1 -- 172.21.15.42:0/2550302972 shutdown_connections 2024-05-28T14:06:16.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.122+0000 7ffa8f200640 1 --2- 172.21.15.42:0/2550302972 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa6c069070 0x7ffa6c06b530 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:16.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.122+0000 7ffa8f200640 1 --2- 172.21.15.42:0/2550302972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa90145d50 0x7ffa90153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.122+0000 7ffa8f200640 1 --2- 172.21.15.42:0/2550302972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa900cf540 0x7ffa9015c7b0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.122+0000 7ffa8f200640 1 --2- 172.21.15.42:0/2550302972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa900707e0 0x7ffa9015bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.122+0000 7ffa8f200640 1 -- 172.21.15.42:0/2550302972 >> 172.21.15.42:0/2550302972 conn(0x7ffa9006fcc0 msgr2=0x7ffa900b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.122+0000 7ffa8f200640 1 -- 172.21.15.42:0/2550302972 shutdown_connections 2024-05-28T14:06:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.122+0000 7ffa8f200640 1 -- 172.21.15.42:0/2550302972 wait complete. 2024-05-28T14:06:16.129 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones' 2024-05-28T14:06:16.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.282+0000 7fd8e2000640 1 Processor -- start 2024-05-28T14:06:16.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.282+0000 7fd8e2000640 1 -- start start 2024-05-28T14:06:16.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.286+0000 7fd8e2000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd8dc06f660 0x7fd8dc073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:16.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.286+0000 7fd8e2000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8dc0d0e00 0x7fd8dc06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:16.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.286+0000 7fd8e2000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8dc15f280 con 0x7fd8dc15ba40 2024-05-28T14:06:16.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.286+0000 7fd8e2000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd8dc15f030 con 0x7fd8dc06f660 2024-05-28T14:06:16.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.286+0000 7fd8e2000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd8dc15f560 con 0x7fd8dc0d0e00 2024-05-28T14:06:16.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.286+0000 7fd8db400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd8dc06f660 0x7fd8dc073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:16.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.286+0000 7fd8db400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd8dc06f660 0x7fd8dc073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47778/0 (socket says 172.21.15.42:47778) 2024-05-28T14:06:16.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.286+0000 7fd8db400640 1 -- 172.21.15.42:0/1196201363 learned_addr learned my addr 172.21.15.42:0/1196201363 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:16.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.286+0000 7fd8dbe00640 1 --2- 172.21.15.42:0/1196201363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8dc0d0e00 0x7fd8dc06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:16.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.286+0000 7fd8db400640 1 -- 172.21.15.42:0/1196201363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8dc0d0e00 msgr2=0x7fd8dc06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:16.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.286+0000 7fd8db400640 1 --2- 172.21.15.42:0/1196201363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8dc0d0e00 0x7fd8dc06dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:16.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.286+0000 7fd8db400640 1 -- 172.21.15.42:0/1196201363 >> v1:172.21.15.42:6789/0 conn(0x7fd8dc15ba40 legacy=0x7fd8dc0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:16.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.286+0000 7fd8db400640 1 -- 172.21.15.42:0/1196201363 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8dc15f840 con 0x7fd8dc06f660 2024-05-28T14:06:16.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.286+0000 7fd8daa00640 1 -- 172.21.15.42:0/1196201363 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 430637738 0 0) 0x7fd8dc15f280 con 0x7fd8dc15ba40 2024-05-28T14:06:16.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8db400640 1 --2- 172.21.15.42:0/1196201363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8dc06f660 0x7fd8dc073980 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009870 tx=0x7fd8cc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946c6f3be4463f22 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:16.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8daa00640 1 -- 172.21.15.42:0/1196201363 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc0ab5f0 con 0x7fd8dc06f660 2024-05-28T14:06:16.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8daa00640 1 -- 172.21.15.42:0/1196201363 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8cc0b0070 con 0x7fd8dc06f660 2024-05-28T14:06:16.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8daa00640 1 -- 172.21.15.42:0/1196201363 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc0ab9b0 con 0x7fd8dc06f660 2024-05-28T14:06:16.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8e2000640 1 -- 172.21.15.42:0/1196201363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8dc06f660 msgr2=0x7fd8dc073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:16.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8e2000640 1 --2- 172.21.15.42:0/1196201363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8dc06f660 0x7fd8dc073980 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009870 tx=0x7fd8cc01c410 comp rx=0 tx=0).stop 2024-05-28T14:06:16.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8e2000640 1 -- 172.21.15.42:0/1196201363 shutdown_connections 2024-05-28T14:06:16.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8e2000640 1 --2- 172.21.15.42:0/1196201363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8dc0d0e00 0x7fd8dc06dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:16.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8e2000640 1 --2- 172.21.15.42:0/1196201363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8dc06f660 0x7fd8dc073980 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:16.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8e2000640 1 -- 172.21.15.42:0/1196201363 >> 172.21.15.42:0/1196201363 conn(0x7fd8dc06bdf0 msgr2=0x7fd8dc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:16.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8e2000640 1 -- 172.21.15.42:0/1196201363 shutdown_connections 2024-05-28T14:06:16.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8e2000640 1 -- 172.21.15.42:0/1196201363 wait complete. 2024-05-28T14:06:16.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8e2000640 1 Processor -- start 2024-05-28T14:06:16.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8e2000640 1 -- start start 2024-05-28T14:06:16.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8e2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8dc06f660 0x7fd8dc14e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:16.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8e2000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8dc0d0e00 0x7fd8dc1557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:16.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.290+0000 7fd8e2000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8dc0b0350 0x7fd8dc0b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:16.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8e2000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc168550 con 0x7fd8dc06f660 2024-05-28T14:06:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8e2000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc1683d0 con 0x7fd8dc0d0e00 2024-05-28T14:06:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8e2000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd8dc1686d0 con 0x7fd8dc0b0350 2024-05-28T14:06:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8db400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8dc06f660 0x7fd8dc14e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8db400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8dc06f660 0x7fd8dc14e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47786/0 (socket says 172.21.15.42:47786) 2024-05-28T14:06:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8db400640 1 -- 172.21.15.42:0/1645232254 learned_addr learned my addr 172.21.15.42:0/1645232254 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8d2a00640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8dc0d0e00 0x7fd8dc1557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8dbe00640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8dc0b0350 0x7fd8dc0b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8db400640 1 -- 172.21.15.42:0/1645232254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8dc0b0350 msgr2=0x7fd8dc0b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8db400640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8dc0b0350 0x7fd8dc0b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8db400640 1 -- 172.21.15.42:0/1645232254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8dc0d0e00 msgr2=0x7fd8dc1557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8db400640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8dc0d0e00 0x7fd8dc1557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8db400640 1 -- 172.21.15.42:0/1645232254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8dc0b3910 con 0x7fd8dc06f660 2024-05-28T14:06:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8d2a00640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8dc0d0e00 0x7fd8dc1557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8db400640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8dc06f660 0x7fd8dc14e9f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc01a040 tx=0x7fd8cc01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:16.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8d3e00640 1 -- 172.21.15.42:0/1645232254 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc0ba070 con 0x7fd8dc06f660 2024-05-28T14:06:16.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8d3e00640 1 -- 172.21.15.42:0/1645232254 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8cc004230 con 0x7fd8dc06f660 2024-05-28T14:06:16.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8d3e00640 1 -- 172.21.15.42:0/1645232254 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc0ab8c0 con 0x7fd8dc06f660 2024-05-28T14:06:16.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8e2000640 1 -- 172.21.15.42:0/1645232254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8dc09b080 con 0x7fd8dc06f660 2024-05-28T14:06:16.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8e2000640 1 -- 172.21.15.42:0/1645232254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8dc09b640 con 0x7fd8dc06f660 2024-05-28T14:06:16.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8e2000640 1 -- 172.21.15.42:0/1645232254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8dc068a90 con 0x7fd8dc06f660 2024-05-28T14:06:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.294+0000 7fd8d3e00640 1 -- 172.21.15.42:0/1645232254 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7fd8cc0aba60 con 0x7fd8dc06f660 2024-05-28T14:06:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.298+0000 7fd8d3e00640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd8b8069070 0x7fd8b806b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.298+0000 7fd8d3e00640 1 -- 172.21.15.42:0/1645232254 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4971+0+0 (secure 0 0 0) 0x7fd8cc11f3b0 con 0x7fd8dc06f660 2024-05-28T14:06:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.298+0000 7fd8d2a00640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd8b8069070 0x7fd8b806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:16.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.298+0000 7fd8d2a00640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd8b8069070 0x7fd8b806b530 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc14fe80 tx=0x7fd8c4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:16.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.302+0000 7fd8d3e00640 1 -- 172.21.15.42:0/1645232254 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd8cc016150 con 0x7fd8dc06f660 2024-05-28T14:06:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:16.526+0000 7fd8e2000640 1 -- 172.21.15.42:0/1645232254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones"]} v 0) -- 0x7fd8dc072bb0 con 0x7fd8dc06f660 2024-05-28T14:06:17.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.362+0000 7fd8d3e00640 1 -- 172.21.15.42:0/1645232254 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones"]}]=0 v364) ==== 162+0+0 (secure 0 0 0) 0x7fd8cc0bc350 con 0x7fd8dc06f660 2024-05-28T14:06:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.366+0000 7fd8e2000640 1 -- 172.21.15.42:0/1645232254 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd8b8069070 msgr2=0x7fd8b806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.366+0000 7fd8e2000640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd8b8069070 0x7fd8b806b530 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc14fe80 tx=0x7fd8c4009040 comp rx=0 tx=0).stop 2024-05-28T14:06:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.366+0000 7fd8e2000640 1 -- 172.21.15.42:0/1645232254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8dc06f660 msgr2=0x7fd8dc14e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.366+0000 7fd8e2000640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8dc06f660 0x7fd8dc14e9f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc01a040 tx=0x7fd8cc01ac50 comp rx=0 tx=0).stop 2024-05-28T14:06:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.366+0000 7fd8e2000640 1 -- 172.21.15.42:0/1645232254 shutdown_connections 2024-05-28T14:06:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.366+0000 7fd8e2000640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd8b8069070 0x7fd8b806b530 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.366+0000 7fd8e2000640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8dc0b0350 0x7fd8dc0b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.366+0000 7fd8e2000640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8dc0d0e00 0x7fd8dc1557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.366+0000 7fd8e2000640 1 --2- 172.21.15.42:0/1645232254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8dc06f660 0x7fd8dc14e9f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.366+0000 7fd8e2000640 1 -- 172.21.15.42:0/1645232254 >> 172.21.15.42:0/1645232254 conn(0x7fd8dc06bdf0 msgr2=0x7fd8dc15d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:17.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.366+0000 7fd8e2000640 1 -- 172.21.15.42:0/1645232254 shutdown_connections 2024-05-28T14:06:17.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.366+0000 7fd8e2000640 1 -- 172.21.15.42:0/1645232254 wait complete. 2024-05-28T14:06:17.377 INFO:tasks.cephfs_test_runner:Test that 2 progress bars are printed when number of clone jobs is ... ok 2024-05-28T14:06:17.378 INFO:tasks.cephfs_test_runner:Starting test: test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress indicated by progress bar for ongoing+pending clones 2024-05-28T14:06:17.378 INFO:tasks.cephfs_test_runner:test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-28T14:06:17.378 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:06:17.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.534+0000 7fdbdc600640 1 Processor -- start 2024-05-28T14:06:17.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbdc600640 1 -- start start 2024-05-28T14:06:17.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbdc600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdbd415e970 0x7fdbd40d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:17.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbdc600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdbd406ff60 0x7fdbd40ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:17.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbdc600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbd4070740 con 0x7fdbd40cd040 2024-05-28T14:06:17.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbdc600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd40d0640 con 0x7fdbd415e970 2024-05-28T14:06:17.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbdc600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd40d1160 con 0x7fdbd406ff60 2024-05-28T14:06:17.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbd9c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdbd415e970 0x7fdbd40d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:17.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbd9c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdbd415e970 0x7fdbd40d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42880/0 (socket says 172.21.15.42:42880) 2024-05-28T14:06:17.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbd9c00640 1 -- 172.21.15.42:0/660352580 learned_addr learned my addr 172.21.15.42:0/660352580 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:17.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbda600640 1 --1- 172.21.15.42:0/660352580 >> v1:172.21.15.42:6789/0 conn(0x7fdbd40cd040 0x7fdbd4162500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42558/0 (socket says 172.21.15.42:42558) 2024-05-28T14:06:17.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbd9200640 1 --2- 172.21.15.42:0/660352580 >> v2:172.21.15.63:3300/0 conn(0x7fdbd406ff60 0x7fdbd40ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:17.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbd9c00640 1 -- 172.21.15.42:0/660352580 >> v2:172.21.15.63:3300/0 conn(0x7fdbd406ff60 msgr2=0x7fdbd40ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:17.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbd9c00640 1 --2- 172.21.15.42:0/660352580 >> v2:172.21.15.63:3300/0 conn(0x7fdbd406ff60 0x7fdbd40ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbd9c00640 1 -- 172.21.15.42:0/660352580 >> v1:172.21.15.42:6789/0 conn(0x7fdbd40cd040 legacy=0x7fdbd4162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:17.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.538+0000 7fdbd9c00640 1 -- 172.21.15.42:0/660352580 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd40d0870 con 0x7fdbd415e970 2024-05-28T14:06:17.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.542+0000 7fdbd9c00640 1 --2- 172.21.15.42:0/660352580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd415e970 0x7fdbd40d15f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4162460 tx=0x7fdbc801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1698dfd442bac9c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:17.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.542+0000 7fdbd3e00640 1 -- 172.21.15.42:0/660352580 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80b0070 con 0x7fdbd415e970 2024-05-28T14:06:17.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.542+0000 7fdbd3e00640 1 -- 172.21.15.42:0/660352580 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc80ab600 con 0x7fdbd415e970 2024-05-28T14:06:17.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.542+0000 7fdbd3e00640 1 -- 172.21.15.42:0/660352580 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80ab9d0 con 0x7fdbd415e970 2024-05-28T14:06:17.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.542+0000 7fdbdc600640 1 -- 172.21.15.42:0/660352580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd415e970 msgr2=0x7fdbd40d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:17.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.542+0000 7fdbdc600640 1 --2- 172.21.15.42:0/660352580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd415e970 0x7fdbd40d15f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4162460 tx=0x7fdbc801c460 comp rx=0 tx=0).stop 2024-05-28T14:06:17.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.542+0000 7fdbdc600640 1 -- 172.21.15.42:0/660352580 shutdown_connections 2024-05-28T14:06:17.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.542+0000 7fdbdc600640 1 --2- 172.21.15.42:0/660352580 >> v2:172.21.15.63:3300/0 conn(0x7fdbd406ff60 0x7fdbd40ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.542+0000 7fdbdc600640 1 --2- 172.21.15.42:0/660352580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd415e970 0x7fdbd40d15f0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.542+0000 7fdbdc600640 1 -- 172.21.15.42:0/660352580 >> 172.21.15.42:0/660352580 conn(0x7fdbd40b3fa0 msgr2=0x7fdbd40b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:17.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.542+0000 7fdbdc600640 1 -- 172.21.15.42:0/660352580 shutdown_connections 2024-05-28T14:06:17.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.542+0000 7fdbdc600640 1 -- 172.21.15.42:0/660352580 wait complete. 2024-05-28T14:06:17.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.542+0000 7fdbdc600640 1 Processor -- start 2024-05-28T14:06:17.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.542+0000 7fdbdc600640 1 -- start start 2024-05-28T14:06:17.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbdc600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbd406ff60 0x7fdbd40b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:17.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbdc600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd40cd040 0x7fdbd4158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:17.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbdc600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbd415e970 0x7fdbd40b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:17.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbdc600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd40704c0 con 0x7fdbd40cd040 2024-05-28T14:06:17.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbdc600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4070340 con 0x7fdbd415e970 2024-05-28T14:06:17.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbdc600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd416b840 con 0x7fdbd406ff60 2024-05-28T14:06:17.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbd9200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd40cd040 0x7fdbd4158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:17.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbd9200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd40cd040 0x7fdbd4158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42892/0 (socket says 172.21.15.42:42892) 2024-05-28T14:06:17.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbd9200640 1 -- 172.21.15.42:0/3616328545 learned_addr learned my addr 172.21.15.42:0/3616328545 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:17.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbd9c00640 1 --2- 172.21.15.42:0/3616328545 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbd406ff60 0x7fdbd40b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:17.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbda600640 1 --2- 172.21.15.42:0/3616328545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbd415e970 0x7fdbd40b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:17.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbd9200640 1 -- 172.21.15.42:0/3616328545 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbd406ff60 msgr2=0x7fdbd40b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:17.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbd9200640 1 --2- 172.21.15.42:0/3616328545 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbd406ff60 0x7fdbd40b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbd9200640 1 -- 172.21.15.42:0/3616328545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbd415e970 msgr2=0x7fdbd40b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:17.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbd9200640 1 --2- 172.21.15.42:0/3616328545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbd415e970 0x7fdbd40b68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbd9200640 1 -- 172.21.15.42:0/3616328545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd40b7340 con 0x7fdbd40cd040 2024-05-28T14:06:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbd9200640 1 --2- 172.21.15.42:0/3616328545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd40cd040 0x7fdbd4158750 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000d620 tx=0x7fdbc000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbd1600640 1 -- 172.21.15.42:0/3616328545 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000b620 con 0x7fdbd40cd040 2024-05-28T14:06:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbd1600640 1 -- 172.21.15.42:0/3616328545 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc0019070 con 0x7fdbd40cd040 2024-05-28T14:06:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbd1600640 1 -- 172.21.15.42:0/3616328545 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000b9e0 con 0x7fdbd40cd040 2024-05-28T14:06:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbdc600640 1 -- 172.21.15.42:0/3616328545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd40b5190 con 0x7fdbd40cd040 2024-05-28T14:06:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbdc600640 1 -- 172.21.15.42:0/3616328545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd40b5750 con 0x7fdbd40cd040 2024-05-28T14:06:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbdc600640 1 -- 172.21.15.42:0/3616328545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd4068a90 con 0x7fdbd40cd040 2024-05-28T14:06:17.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.546+0000 7fdbd1600640 1 -- 172.21.15.42:0/3616328545 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7fdbc000bb80 con 0x7fdbd40cd040 2024-05-28T14:06:17.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.550+0000 7fdbd1600640 1 --2- 172.21.15.42:0/3616328545 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdbb4069070 0x7fdbb406b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:17.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.550+0000 7fdbd1600640 1 -- 172.21.15.42:0/3616328545 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4971+0+0 (secure 0 0 0) 0x7fdbc0085960 con 0x7fdbd40cd040 2024-05-28T14:06:17.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.550+0000 7fdbd9c00640 1 --2- 172.21.15.42:0/3616328545 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdbb4069070 0x7fdbb406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:17.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.550+0000 7fdbd9c00640 1 --2- 172.21.15.42:0/3616328545 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdbb4069070 0x7fdbb406b530 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80ad000 tx=0x7fdbc80a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:17.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.554+0000 7fdbd1600640 1 -- 172.21.15.42:0/3616328545 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdbc0057090 con 0x7fdbd40cd040 2024-05-28T14:06:17.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.718+0000 7fdbdc600640 1 -- 172.21.15.42:0/3616328545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdbd40d1fd0 con 0x7fdbd40cd040 2024-05-28T14:06:17.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.718+0000 7fdbd1600640 1 -- 172.21.15.42:0/3616328545 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+13003 (secure 0 0 0) 0x7fdbc005b390 con 0x7fdbd40cd040 2024-05-28T14:06:17.705 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:17.706 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":66,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:06:15.354952+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6835/3311861302":"2024-05-29T14:06:12.348585+0000","172.21.15.63:6833/3311861302":"2024-05-29T14:06:12.348585+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:06:17.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.722+0000 7fdbdc600640 1 -- 172.21.15.42:0/3616328545 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdbb4069070 msgr2=0x7fdbb406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:17.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.722+0000 7fdbdc600640 1 --2- 172.21.15.42:0/3616328545 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdbb4069070 0x7fdbb406b530 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80ad000 tx=0x7fdbc80a6000 comp rx=0 tx=0).stop 2024-05-28T14:06:17.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.722+0000 7fdbdc600640 1 -- 172.21.15.42:0/3616328545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd40cd040 msgr2=0x7fdbd4158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:17.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.722+0000 7fdbdc600640 1 --2- 172.21.15.42:0/3616328545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd40cd040 0x7fdbd4158750 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000d620 tx=0x7fdbc000dae0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.722+0000 7fdbdc600640 1 -- 172.21.15.42:0/3616328545 shutdown_connections 2024-05-28T14:06:17.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.722+0000 7fdbdc600640 1 --2- 172.21.15.42:0/3616328545 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdbb4069070 0x7fdbb406b530 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.722+0000 7fdbdc600640 1 --2- 172.21.15.42:0/3616328545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbd415e970 0x7fdbd40b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.722+0000 7fdbdc600640 1 --2- 172.21.15.42:0/3616328545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd40cd040 0x7fdbd4158750 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.722+0000 7fdbdc600640 1 --2- 172.21.15.42:0/3616328545 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbd406ff60 0x7fdbd40b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.722+0000 7fdbdc600640 1 -- 172.21.15.42:0/3616328545 >> 172.21.15.42:0/3616328545 conn(0x7fdbd40b3fa0 msgr2=0x7fdbd4070930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:17.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.722+0000 7fdbdc600640 1 -- 172.21.15.42:0/3616328545 shutdown_connections 2024-05-28T14:06:17.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.722+0000 7fdbdc600640 1 -- 172.21.15.42:0/3616328545 wait complete. 2024-05-28T14:06:17.728 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:06:17.728 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:06:17.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.886+0000 7f2c7de00640 1 Processor -- start 2024-05-28T14:06:17.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.886+0000 7f2c7de00640 1 -- start start 2024-05-28T14:06:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.886+0000 7f2c7de00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2c780b4c10 0x7f2c78071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.886+0000 7f2c7de00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c780b3690 con 0x7f2c78072320 2024-05-28T14:06:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.886+0000 7f2c7de00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c780b3970 con 0x7f2c78148090 2024-05-28T14:06:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.886+0000 7f2c7de00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f2c78148470 con 0x7f2c780b4c10 2024-05-28T14:06:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.886+0000 7f2c6e000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2c780b4c10 0x7f2c78071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.886+0000 7f2c77400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f2c78072320 0x7f2c780aeb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42566/0 (socket says 172.21.15.42:42566) 2024-05-28T14:06:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.886+0000 7f2c6e000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2c780b4c10 0x7f2c78071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42894/0 (socket says 172.21.15.42:42894) 2024-05-28T14:06:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.886+0000 7f2c77400640 1 -- 172.21.15.42:0/2499806577 learned_addr learned my addr 172.21.15.42:0/2499806577 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.886+0000 7f2c6e000640 1 -- 172.21.15.42:0/2499806577 >> v1:172.21.15.63:6789/0 conn(0x7f2c78148090 legacy=0x7f2c780b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.890+0000 7f2c6e000640 1 -- 172.21.15.42:0/2499806577 >> v1:172.21.15.42:6789/0 conn(0x7f2c78072320 legacy=0x7f2c780aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.890+0000 7f2c6e000640 1 -- 172.21.15.42:0/2499806577 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c780b3c50 con 0x7f2c780b4c10 2024-05-28T14:06:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.890+0000 7f2c76000640 1 -- 172.21.15.42:0/2499806577 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3758166482 0 0) 0x7f2c780b3690 con 0x7f2c78072320 2024-05-28T14:06:17.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.890+0000 7f2c6e000640 1 --2- 172.21.15.42:0/2499806577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c780b4c10 0x7f2c78071de0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2c600097c0 tx=0x7f2c6001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736d56eb51176138 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:17.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.890+0000 7f2c76000640 1 -- 172.21.15.42:0/2499806577 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c600b0070 con 0x7f2c780b4c10 2024-05-28T14:06:17.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.890+0000 7f2c76000640 1 -- 172.21.15.42:0/2499806577 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c600ab600 con 0x7f2c780b4c10 2024-05-28T14:06:17.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.890+0000 7f2c76000640 1 -- 172.21.15.42:0/2499806577 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c600ab9d0 con 0x7f2c780b4c10 2024-05-28T14:06:17.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.890+0000 7f2c7de00640 1 -- 172.21.15.42:0/2499806577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c780b4c10 msgr2=0x7f2c78071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:17.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.890+0000 7f2c7de00640 1 --2- 172.21.15.42:0/2499806577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c780b4c10 0x7f2c78071de0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2c600097c0 tx=0x7f2c6001c410 comp rx=0 tx=0).stop 2024-05-28T14:06:17.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.890+0000 7f2c7de00640 1 -- 172.21.15.42:0/2499806577 shutdown_connections 2024-05-28T14:06:17.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.890+0000 7f2c7de00640 1 --2- 172.21.15.42:0/2499806577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c780b4c10 0x7f2c78071de0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.890+0000 7f2c7de00640 1 -- 172.21.15.42:0/2499806577 >> 172.21.15.42:0/2499806577 conn(0x7f2c780b21f0 msgr2=0x7f2c780b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:17.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.890+0000 7f2c7de00640 1 -- 172.21.15.42:0/2499806577 shutdown_connections 2024-05-28T14:06:17.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.890+0000 7f2c7de00640 1 -- 172.21.15.42:0/2499806577 wait complete. 2024-05-28T14:06:17.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c7de00640 1 Processor -- start 2024-05-28T14:06:17.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c7de00640 1 -- start start 2024-05-28T14:06:17.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c7de00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c78072320 0x7f2c78155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:17.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c7de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c78148090 0x7f2c78146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:17.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c7de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c78159920 0x7f2c7815da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:17.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c7de00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2c780b5830 con 0x7f2c78148090 2024-05-28T14:06:17.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c7de00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c780afcf0 con 0x7f2c78159920 2024-05-28T14:06:17.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c7de00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2c7816c550 con 0x7f2c78072320 2024-05-28T14:06:17.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c6e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c78148090 0x7f2c78146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:17.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c6e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c78148090 0x7f2c78146f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42904/0 (socket says 172.21.15.42:42904) 2024-05-28T14:06:17.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c6e000640 1 -- 172.21.15.42:0/3185351579 learned_addr learned my addr 172.21.15.42:0/3185351579 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:17.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c76a00640 1 --2- 172.21.15.42:0/3185351579 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c78072320 0x7f2c78155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:17.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c77400640 1 --2- 172.21.15.42:0/3185351579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c78159920 0x7f2c7815da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:17.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c6e000640 1 -- 172.21.15.42:0/3185351579 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c78072320 msgr2=0x7f2c78155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c6e000640 1 --2- 172.21.15.42:0/3185351579 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c78072320 0x7f2c78155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c6e000640 1 -- 172.21.15.42:0/3185351579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c78159920 msgr2=0x7f2c7815da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c6e000640 1 --2- 172.21.15.42:0/3185351579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c78159920 0x7f2c7815da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c6e000640 1 -- 172.21.15.42:0/3185351579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c78145690 con 0x7f2c78148090 2024-05-28T14:06:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c6e000640 1 --2- 172.21.15.42:0/3185351579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c78148090 0x7f2c78146f10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2c600ae040 tx=0x7f2c6001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c6fe00640 1 -- 172.21.15.42:0/3185351579 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c600b0040 con 0x7f2c78148090 2024-05-28T14:06:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c6fe00640 1 -- 172.21.15.42:0/3185351579 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c60004030 con 0x7f2c78148090 2024-05-28T14:06:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c6fe00640 1 -- 172.21.15.42:0/3185351579 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60004420 con 0x7f2c78148090 2024-05-28T14:06:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c7de00640 1 -- 172.21.15.42:0/3185351579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c781f7760 con 0x7f2c78148090 2024-05-28T14:06:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.894+0000 7f2c7de00640 1 -- 172.21.15.42:0/3185351579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c781f7a00 con 0x7f2c78148090 2024-05-28T14:06:17.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.898+0000 7f2c7de00640 1 -- 172.21.15.42:0/3185351579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c780738b0 con 0x7f2c78148090 2024-05-28T14:06:17.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.898+0000 7f2c6fe00640 1 -- 172.21.15.42:0/3185351579 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f2c60003620 con 0x7f2c78148090 2024-05-28T14:06:17.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.898+0000 7f2c6fe00640 1 --2- 172.21.15.42:0/3185351579 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c540690a0 0x7f2c5406b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:17.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.898+0000 7f2c6fe00640 1 -- 172.21.15.42:0/3185351579 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4971+0+0 (secure 0 0 0) 0x7f2c6011fb80 con 0x7f2c78148090 2024-05-28T14:06:17.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.898+0000 7f2c76a00640 1 --2- 172.21.15.42:0/3185351579 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c540690a0 0x7f2c5406b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:17.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.898+0000 7f2c76a00640 1 --2- 172.21.15.42:0/3185351579 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c540690a0 0x7f2c5406b560 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2c680012f0 tx=0x7f2c68009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:17.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:17.902+0000 7f2c6fe00640 1 -- 172.21.15.42:0/3185351579 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2c78145690 con 0x7f2c78148090 2024-05-28T14:06:18.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.066+0000 7f2c7de00640 1 -- 172.21.15.42:0/3185351579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2c7804fdd0 con 0x7f2c78148090 2024-05-28T14:06:18.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.066+0000 7f2c6fe00640 1 -- 172.21.15.42:0/3185351579 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f2c60016090 con 0x7f2c78148090 2024-05-28T14:06:18.055 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:06:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.070+0000 7f2c7de00640 1 -- 172.21.15.42:0/3185351579 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c540690a0 msgr2=0x7f2c5406b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.070+0000 7f2c7de00640 1 --2- 172.21.15.42:0/3185351579 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c540690a0 0x7f2c5406b560 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2c680012f0 tx=0x7f2c68009040 comp rx=0 tx=0).stop 2024-05-28T14:06:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.070+0000 7f2c7de00640 1 -- 172.21.15.42:0/3185351579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c78148090 msgr2=0x7f2c78146f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.070+0000 7f2c7de00640 1 --2- 172.21.15.42:0/3185351579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c78148090 0x7f2c78146f10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2c600ae040 tx=0x7f2c6001ac50 comp rx=0 tx=0).stop 2024-05-28T14:06:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.070+0000 7f2c7de00640 1 -- 172.21.15.42:0/3185351579 shutdown_connections 2024-05-28T14:06:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.070+0000 7f2c7de00640 1 --2- 172.21.15.42:0/3185351579 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c540690a0 0x7f2c5406b560 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.070+0000 7f2c7de00640 1 --2- 172.21.15.42:0/3185351579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c78159920 0x7f2c7815da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.070+0000 7f2c7de00640 1 --2- 172.21.15.42:0/3185351579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c78148090 0x7f2c78146f10 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.070+0000 7f2c7de00640 1 --2- 172.21.15.42:0/3185351579 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c78072320 0x7f2c78155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.070+0000 7f2c7de00640 1 -- 172.21.15.42:0/3185351579 >> 172.21.15.42:0/3185351579 conn(0x7f2c780b21f0 msgr2=0x7f2c7806de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.070+0000 7f2c7de00640 1 -- 172.21.15.42:0/3185351579 shutdown_connections 2024-05-28T14:06:18.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.070+0000 7f2c7de00640 1 -- 172.21.15.42:0/3185351579 wait complete. 2024-05-28T14:06:18.077 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-05-28T14:06:18.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.230+0000 7f7caf400640 1 Processor -- start 2024-05-28T14:06:18.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.230+0000 7f7caf400640 1 -- start start 2024-05-28T14:06:18.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.234+0000 7f7caf400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7ca80d0420 0x7f7ca815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:18.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.234+0000 7f7caf400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7ca8067810 0x7f7ca80b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:18.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.234+0000 7f7caf400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ca806eaa0 con 0x7f7ca80b4110 2024-05-28T14:06:18.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.234+0000 7f7caf400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f7ca80b4580 con 0x7f7ca80d0420 2024-05-28T14:06:18.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.234+0000 7f7caf400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7ca806ed80 con 0x7f7ca8067810 2024-05-28T14:06:18.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.234+0000 7f7caca00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7ca80d0420 0x7f7ca815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:18.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.234+0000 7f7caca00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7ca80d0420 0x7f7ca815ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42916/0 (socket says 172.21.15.42:42916) 2024-05-28T14:06:18.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.234+0000 7f7caca00640 1 -- 172.21.15.42:0/2339715117 learned_addr learned my addr 172.21.15.42:0/2339715117 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:18.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.234+0000 7f7caca00640 1 -- 172.21.15.42:0/2339715117 >> v2:172.21.15.63:3300/0 conn(0x7f7ca8067810 msgr2=0x7f7ca80b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:18.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.234+0000 7f7caca00640 1 --2- 172.21.15.42:0/2339715117 >> v2:172.21.15.63:3300/0 conn(0x7f7ca8067810 0x7f7ca80b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:18.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.234+0000 7f7caca00640 1 -- 172.21.15.42:0/2339715117 >> v1:172.21.15.63:6789/0 conn(0x7f7ca80b4110 legacy=0x7f7ca8162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:18.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.234+0000 7f7caca00640 1 -- 172.21.15.42:0/2339715117 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca806f060 con 0x7f7ca80d0420 2024-05-28T14:06:18.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.238+0000 7f7caca00640 1 --2- 172.21.15.42:0/2339715117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ca80d0420 0x7f7ca815ed10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80b3850 tx=0x7f7c9801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a18be9274acccbd server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:18.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.238+0000 7f7ca7e00640 1 -- 172.21.15.42:0/2339715117 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c980b0070 con 0x7f7ca80d0420 2024-05-28T14:06:18.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.238+0000 7f7ca7e00640 1 -- 172.21.15.42:0/2339715117 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c980ab600 con 0x7f7ca80d0420 2024-05-28T14:06:18.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.238+0000 7f7ca7e00640 1 -- 172.21.15.42:0/2339715117 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c980ab9d0 con 0x7f7ca80d0420 2024-05-28T14:06:18.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.238+0000 7f7caf400640 1 -- 172.21.15.42:0/2339715117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ca80d0420 msgr2=0x7f7ca815ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:18.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.238+0000 7f7caf400640 1 --2- 172.21.15.42:0/2339715117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ca80d0420 0x7f7ca815ed10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80b3850 tx=0x7f7c9801c450 comp rx=0 tx=0).stop 2024-05-28T14:06:18.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.238+0000 7f7caf400640 1 -- 172.21.15.42:0/2339715117 shutdown_connections 2024-05-28T14:06:18.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.238+0000 7f7caf400640 1 --2- 172.21.15.42:0/2339715117 >> v2:172.21.15.63:3300/0 conn(0x7f7ca8067810 0x7f7ca80b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:18.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.238+0000 7f7caf400640 1 --2- 172.21.15.42:0/2339715117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ca80d0420 0x7f7ca815ed10 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:18.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.238+0000 7f7caf400640 1 -- 172.21.15.42:0/2339715117 >> 172.21.15.42:0/2339715117 conn(0x7f7ca80d0b40 msgr2=0x7f7ca80d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:18.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.238+0000 7f7caf400640 1 -- 172.21.15.42:0/2339715117 shutdown_connections 2024-05-28T14:06:18.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.238+0000 7f7caf400640 1 -- 172.21.15.42:0/2339715117 wait complete. 2024-05-28T14:06:18.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.238+0000 7f7caf400640 1 Processor -- start 2024-05-28T14:06:18.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.238+0000 7f7caf400640 1 -- start start 2024-05-28T14:06:18.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7caf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca8067810 0x7f7ca81608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:18.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7caf400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ca80b4110 0x7f7ca8149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:18.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7caf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ca80d0420 0x7f7ca81553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:18.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7caf400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca816b4c0 con 0x7f7ca80d0420 2024-05-28T14:06:18.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7caf400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca806e730 con 0x7f7ca8067810 2024-05-28T14:06:18.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7caf400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7ca816b640 con 0x7f7ca80b4110 2024-05-28T14:06:18.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7cad400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ca80d0420 0x7f7ca81553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:18.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7cad400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ca80d0420 0x7f7ca81553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42920/0 (socket says 172.21.15.42:42920) 2024-05-28T14:06:18.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7cad400640 1 -- 172.21.15.42:0/1055048055 learned_addr learned my addr 172.21.15.42:0/1055048055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7caca00640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca8067810 0x7f7ca81608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7c9fe00640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ca80b4110 0x7f7ca8149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7cad400640 1 -- 172.21.15.42:0/1055048055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ca80b4110 msgr2=0x7f7ca8149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7cad400640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ca80b4110 0x7f7ca8149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7cad400640 1 -- 172.21.15.42:0/1055048055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca8067810 msgr2=0x7f7ca81608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7cad400640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca8067810 0x7f7ca81608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7cad400640 1 -- 172.21.15.42:0/1055048055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca8149ae0 con 0x7f7ca80d0420 2024-05-28T14:06:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7caca00640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca8067810 0x7f7ca81608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7cad400640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ca80d0420 0x7f7ca81553a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000c780 tx=0x7f7ca000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7ca5600640 1 -- 172.21.15.42:0/1055048055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0017070 con 0x7f7ca80d0420 2024-05-28T14:06:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7ca5600640 1 -- 172.21.15.42:0/1055048055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca0003db0 con 0x7f7ca80d0420 2024-05-28T14:06:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7ca5600640 1 -- 172.21.15.42:0/1055048055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca00052b0 con 0x7f7ca80d0420 2024-05-28T14:06:18.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7caf400640 1 -- 172.21.15.42:0/1055048055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca8149d80 con 0x7f7ca80d0420 2024-05-28T14:06:18.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7caf400640 1 -- 172.21.15.42:0/1055048055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca81496c0 con 0x7f7ca80d0420 2024-05-28T14:06:18.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7caf400640 1 -- 172.21.15.42:0/1055048055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ca8068a90 con 0x7f7ca80d0420 2024-05-28T14:06:18.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.242+0000 7f7ca5600640 1 -- 172.21.15.42:0/1055048055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f7ca0005450 con 0x7f7ca80d0420 2024-05-28T14:06:18.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.246+0000 7f7ca5600640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c840690a0 0x7f7c8406b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:18.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.246+0000 7f7ca5600640 1 -- 172.21.15.42:0/1055048055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4971+0+0 (secure 0 0 0) 0x7f7ca00848c0 con 0x7f7ca80d0420 2024-05-28T14:06:18.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.246+0000 7f7caca00640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c840690a0 0x7f7c8406b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:18.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.246+0000 7f7caca00640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c840690a0 0x7f7c8406b560 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7c980ad000 tx=0x7f7c980a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:18.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.250+0000 7f7ca5600640 1 -- 172.21.15.42:0/1055048055 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7ca00568d0 con 0x7f7ca80d0420 2024-05-28T14:06:18.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:18.474+0000 7f7caf400640 1 -- 172.21.15.42:0/1055048055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f7ca80b0fd0 con 0x7f7ca80d0420 2024-05-28T14:06:18.771 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:06:18.775 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:06:19.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.374+0000 7f7ca5600640 1 -- 172.21.15.42:0/1055048055 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v366) ==== 177+0+0 (secure 0 0 0) 0x7f7ca00561d0 con 0x7f7ca80d0420 2024-05-28T14:06:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.374+0000 7f7caf400640 1 -- 172.21.15.42:0/1055048055 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c840690a0 msgr2=0x7f7c8406b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.374+0000 7f7caf400640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c840690a0 0x7f7c8406b560 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7c980ad000 tx=0x7f7c980a6000 comp rx=0 tx=0).stop 2024-05-28T14:06:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.374+0000 7f7caf400640 1 -- 172.21.15.42:0/1055048055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ca80d0420 msgr2=0x7f7ca81553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.374+0000 7f7caf400640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ca80d0420 0x7f7ca81553a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000c780 tx=0x7f7ca000cc40 comp rx=0 tx=0).stop 2024-05-28T14:06:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.374+0000 7f7caf400640 1 -- 172.21.15.42:0/1055048055 shutdown_connections 2024-05-28T14:06:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.374+0000 7f7caf400640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c840690a0 0x7f7c8406b560 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.374+0000 7f7caf400640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ca80d0420 0x7f7ca81553a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.374+0000 7f7caf400640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ca80b4110 0x7f7ca8149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.374+0000 7f7caf400640 1 --2- 172.21.15.42:0/1055048055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca8067810 0x7f7ca81608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.374+0000 7f7caf400640 1 -- 172.21.15.42:0/1055048055 >> 172.21.15.42:0/1055048055 conn(0x7f7ca80d0b40 msgr2=0x7f7ca80b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.374+0000 7f7caf400640 1 -- 172.21.15.42:0/1055048055 shutdown_connections 2024-05-28T14:06:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.374+0000 7f7caf400640 1 -- 172.21.15.42:0/1055048055 wait complete. 2024-05-28T14:06:19.387 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T14:06:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12d2400640 1 Processor -- start 2024-05-28T14:06:19.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12d2400640 1 -- start start 2024-05-28T14:06:19.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12d2400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12cc0ce3b0 0x7f12cc0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:19.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12d2400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12cc160230 con 0x7f12cc071de0 2024-05-28T14:06:19.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12d2400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12cc160510 con 0x7f12cc13fd10 2024-05-28T14:06:19.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12d2400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f12cc1607f0 con 0x7f12cc0ce3b0 2024-05-28T14:06:19.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12cb400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f12cc071de0 0x7f12cc0721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42568/0 (socket says 172.21.15.42:42568) 2024-05-28T14:06:19.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12cb400640 1 -- 172.21.15.42:0/2407311202 learned_addr learned my addr 172.21.15.42:0/2407311202 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:19.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12c3e00640 1 --2- 172.21.15.42:0/2407311202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12cc0ce3b0 0x7f12cc0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:19.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 336454655 0 0) 0x7f12cc160230 con 0x7f12cc071de0 2024-05-28T14:06:19.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12b0003530 con 0x7f12cc071de0 2024-05-28T14:06:19.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 983220669 0 0) 0x7f12cc160510 con 0x7f12cc13fd10 2024-05-28T14:06:19.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12cc160230 con 0x7f12cc13fd10 2024-05-28T14:06:19.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1186899894 0 0) 0x7f12b0003530 con 0x7f12cc071de0 2024-05-28T14:06:19.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12cc0ce3b0 msgr2=0x7f12cc0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:19.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 --2- 172.21.15.42:0/2407311202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12cc0ce3b0 0x7f12cc0ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:19.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 >> v1:172.21.15.63:6789/0 conn(0x7f12cc13fd10 legacy=0x7f12cc1401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:19.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc160ad0 con 0x7f12cc071de0 2024-05-28T14:06:19.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f12bc001680 con 0x7f12cc071de0 2024-05-28T14:06:19.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 >> v1:172.21.15.42:6789/0 conn(0x7f12cc071de0 legacy=0x7f12cc0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:19.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 --2- 172.21.15.42:0/2407311202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12b0003b80 0x7f12b0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:19.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 --2- 172.21.15.42:0/2407311202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12b0004810 0x7f12b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:19.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 --2- 172.21.15.42:0/2407311202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12b00051d0 0x7f12b000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1607f0 con 0x7f12b00051d0 2024-05-28T14:06:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f12b000c100 con 0x7f12b0003b80 2024-05-28T14:06:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f12b000c300 con 0x7f12b0004810 2024-05-28T14:06:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12c3e00640 1 --2- 172.21.15.42:0/2407311202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12cc0ce3b0 0x7f12cc0ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:06:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.550+0000 7f12cbe00640 1 --2- 172.21.15.42:0/2407311202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12b00051d0 0x7f12b000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.554+0000 7f12cb400640 1 --2- 172.21.15.42:0/2407311202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12b0003b80 0x7f12b0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.554+0000 7f12c3e00640 1 --2- 172.21.15.42:0/2407311202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12b0004810 0x7f12b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.554+0000 7f12cbe00640 1 -- 172.21.15.42:0/2407311202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12b0004810 msgr2=0x7f12b0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.554+0000 7f12cbe00640 1 --2- 172.21.15.42:0/2407311202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12b0004810 0x7f12b0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:19.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.554+0000 7f12cbe00640 1 -- 172.21.15.42:0/2407311202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12b0003b80 msgr2=0x7f12b0003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:19.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.554+0000 7f12cbe00640 1 --2- 172.21.15.42:0/2407311202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12b0003b80 0x7f12b0003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:19.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.554+0000 7f12cbe00640 1 -- 172.21.15.42:0/2407311202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b0003530 con 0x7f12b00051d0 2024-05-28T14:06:19.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.554+0000 7f12cbe00640 1 --2- 172.21.15.42:0/2407311202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12b00051d0 0x7f12b000b9d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f12c4001fd0 tx=0x7f12c401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe66b2961750f1f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:19.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.554+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4001c20 con 0x7f12b00051d0 2024-05-28T14:06:19.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.554+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12c4003ba0 con 0x7f12b00051d0 2024-05-28T14:06:19.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.554+0000 7f12caa00640 1 -- 172.21.15.42:0/2407311202 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c40050d0 con 0x7f12b00051d0 2024-05-28T14:06:19.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12d2400640 1 -- 172.21.15.42:0/2407311202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12b00051d0 msgr2=0x7f12b000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:19.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12d2400640 1 --2- 172.21.15.42:0/2407311202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12b00051d0 0x7f12b000b9d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f12c4001fd0 tx=0x7f12c401c8b0 comp rx=0 tx=0).stop 2024-05-28T14:06:19.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12cbe00640 1 -- 172.21.15.42:0/2407311202 reap_dead start 2024-05-28T14:06:19.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12d2400640 1 -- 172.21.15.42:0/2407311202 shutdown_connections 2024-05-28T14:06:19.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12d2400640 1 -- 172.21.15.42:0/2407311202 >> 172.21.15.42:0/2407311202 conn(0x7f12cc0b21f0 msgr2=0x7f12cc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:19.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12d2400640 1 -- 172.21.15.42:0/2407311202 shutdown_connections 2024-05-28T14:06:19.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12d2400640 1 -- 172.21.15.42:0/2407311202 wait complete. 2024-05-28T14:06:19.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12d2400640 1 Processor -- start 2024-05-28T14:06:19.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12d2400640 1 -- start start 2024-05-28T14:06:19.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12d2400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12b00051d0 0x7f12cc155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:19.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12d2400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12cc0b18d0 0x7f12cc15b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:19.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12d2400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12cc0b4100 0x7f12cc15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:19.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12d2400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc168830 con 0x7f12cc0b4100 2024-05-28T14:06:19.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12d2400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1686b0 con 0x7f12cc0b18d0 2024-05-28T14:06:19.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12d2400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f12cc1689b0 con 0x7f12b00051d0 2024-05-28T14:06:19.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12cbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12cc0b4100 0x7f12cc15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:19.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12cb400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12b00051d0 0x7f12cc155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:19.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12cbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12cc0b4100 0x7f12cc15b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42938/0 (socket says 172.21.15.42:42938) 2024-05-28T14:06:19.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12cbe00640 1 -- 172.21.15.42:0/1787668916 learned_addr learned my addr 172.21.15.42:0/1787668916 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:19.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12c3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12cc0b18d0 0x7f12cc15b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:19.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12cbe00640 1 -- 172.21.15.42:0/1787668916 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12b00051d0 msgr2=0x7f12cc155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:19.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12cbe00640 1 --2- 172.21.15.42:0/1787668916 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12b00051d0 0x7f12cc155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:19.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12cbe00640 1 -- 172.21.15.42:0/1787668916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12cc0b18d0 msgr2=0x7f12cc15b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:19.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12cbe00640 1 --2- 172.21.15.42:0/1787668916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12cc0b18d0 0x7f12cc15b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:19.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.558+0000 7f12cbe00640 1 -- 172.21.15.42:0/1787668916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc1668f0 con 0x7f12cc0b4100 2024-05-28T14:06:19.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.562+0000 7f12cbe00640 1 --2- 172.21.15.42:0/1787668916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12cc0b4100 0x7f12cc15b6b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f12c4001fd0 tx=0x7f12b0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:19.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.562+0000 7f12c3400640 1 -- 172.21.15.42:0/1787668916 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c40bf070 con 0x7f12cc0b4100 2024-05-28T14:06:19.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.562+0000 7f12c3400640 1 -- 172.21.15.42:0/1787668916 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12c4001c00 con 0x7f12cc0b4100 2024-05-28T14:06:19.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.562+0000 7f12c3400640 1 -- 172.21.15.42:0/1787668916 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4005830 con 0x7f12cc0b4100 2024-05-28T14:06:19.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.562+0000 7f12d2400640 1 -- 172.21.15.42:0/1787668916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12cc0b1db0 con 0x7f12cc0b4100 2024-05-28T14:06:19.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.562+0000 7f12d2400640 1 -- 172.21.15.42:0/1787668916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12cc166e90 con 0x7f12cc0b4100 2024-05-28T14:06:19.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.562+0000 7f12d2400640 1 -- 172.21.15.42:0/1787668916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12cc0738b0 con 0x7f12cc0b4100 2024-05-28T14:06:19.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.562+0000 7f12c3400640 1 -- 172.21.15.42:0/1787668916 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f12c40a6040 con 0x7f12cc0b4100 2024-05-28T14:06:19.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.562+0000 7f12c3400640 1 --2- 172.21.15.42:0/1787668916 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12a8069240 0x7f12a806b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:19.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.562+0000 7f12c3400640 1 -- 172.21.15.42:0/1787668916 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4971+0+0 (secure 0 0 0) 0x7f12c411f0a0 con 0x7f12cc0b4100 2024-05-28T14:06:19.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.562+0000 7f12cb400640 1 --2- 172.21.15.42:0/1787668916 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12a8069240 0x7f12a806b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:19.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.566+0000 7f12cb400640 1 --2- 172.21.15.42:0/1787668916 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12a8069240 0x7f12a806b700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000c10 tx=0x7f12bc007040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:19.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.566+0000 7f12c3400640 1 -- 172.21.15.42:0/1787668916 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f12c40b7070 con 0x7f12cc0b4100 2024-05-28T14:06:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.734+0000 7f12d2400640 1 -- 172.21.15.42:0/1787668916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f12cc04fdd0 con 0x7f12cc0b4100 2024-05-28T14:06:19.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.734+0000 7f12c3400640 1 -- 172.21.15.42:0/1787668916 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f12c40ac020 con 0x7f12cc0b4100 2024-05-28T14:06:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.738+0000 7f12d2400640 1 -- 172.21.15.42:0/1787668916 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12a8069240 msgr2=0x7f12a806b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.738+0000 7f12d2400640 1 --2- 172.21.15.42:0/1787668916 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12a8069240 0x7f12a806b700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000c10 tx=0x7f12bc007040 comp rx=0 tx=0).stop 2024-05-28T14:06:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.738+0000 7f12d2400640 1 -- 172.21.15.42:0/1787668916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12cc0b4100 msgr2=0x7f12cc15b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.738+0000 7f12d2400640 1 --2- 172.21.15.42:0/1787668916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12cc0b4100 0x7f12cc15b6b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f12c4001fd0 tx=0x7f12b0005840 comp rx=0 tx=0).stop 2024-05-28T14:06:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.738+0000 7f12d2400640 1 -- 172.21.15.42:0/1787668916 shutdown_connections 2024-05-28T14:06:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.738+0000 7f12d2400640 1 --2- 172.21.15.42:0/1787668916 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f12a8069240 0x7f12a806b700 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:19.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.738+0000 7f12d2400640 1 --2- 172.21.15.42:0/1787668916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12cc0b4100 0x7f12cc15b6b0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:19.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.738+0000 7f12d2400640 1 --2- 172.21.15.42:0/1787668916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12cc0b18d0 0x7f12cc15b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:19.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.738+0000 7f12d2400640 1 --2- 172.21.15.42:0/1787668916 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12b00051d0 0x7f12cc155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:19.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.738+0000 7f12d2400640 1 -- 172.21.15.42:0/1787668916 >> 172.21.15.42:0/1787668916 conn(0x7f12cc0b21f0 msgr2=0x7f12cc156430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:19.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.738+0000 7f12d2400640 1 -- 172.21.15.42:0/1787668916 shutdown_connections 2024-05-28T14:06:19.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:19.738+0000 7f12d2400640 1 -- 172.21.15.42:0/1787668916 wait complete. 2024-05-28T14:06:19.744 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T14:06:19.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:06:19.829 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:06:19.912 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T14:06:19.912 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:06:19.913 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:06:19.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:06:19.922 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:06:19.922 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:06:19.931 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:06:19.931 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:06:19.931 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:06:19.972 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:06:19.972 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:06:19.972 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:06:19.985 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:06:19.986 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-28T14:06:19.986 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:06:20.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.166+0000 7fc005000640 1 Processor -- start 2024-05-28T14:06:20.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.166+0000 7fc005000640 1 -- start start 2024-05-28T14:06:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fc005000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00015e5c0 0x7fc0000b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fc005000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc0000b51b0 0x7fc0000d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fc005000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc0000d0c30 0x7fc000072250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fc005000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc0000b3720 con 0x7fc0000b51b0 2024-05-28T14:06:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fc005000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc0000b39f0 con 0x7fc0000d0c30 2024-05-28T14:06:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fc005000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc0000b3cd0 con 0x7fc00015e5c0 2024-05-28T14:06:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fbff7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc0000b51b0 0x7fc0000d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fbff7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc0000b51b0 0x7fc0000d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42944/0 (socket says 172.21.15.42:42944) 2024-05-28T14:06:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fbff7e00640 1 -- 172.21.15.42:0/254145257 learned_addr learned my addr 172.21.15.42:0/254145257 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fbffe800640 1 --2- 172.21.15.42:0/254145257 >> v2:172.21.15.63:3300/0 conn(0x7fc0000d0c30 0x7fc000072250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fbffde00640 1 --2- 172.21.15.42:0/254145257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00015e5c0 0x7fc0000b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fbff7e00640 1 -- 172.21.15.42:0/254145257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00015e5c0 msgr2=0x7fc0000b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fbff7e00640 1 --2- 172.21.15.42:0/254145257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00015e5c0 0x7fc0000b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fbff7e00640 1 -- 172.21.15.42:0/254145257 >> v2:172.21.15.63:3300/0 conn(0x7fc0000d0c30 msgr2=0x7fc000072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:20.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fbff7e00640 1 --2- 172.21.15.42:0/254145257 >> v2:172.21.15.63:3300/0 conn(0x7fc0000d0c30 0x7fc000072250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.170+0000 7fbff7e00640 1 -- 172.21.15.42:0/254145257 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0000b3fb0 con 0x7fc0000b51b0 2024-05-28T14:06:20.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fbff7e00640 1 --2- 172.21.15.42:0/254145257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0000b51b0 0x7fc0000d06f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc00006f1a0 tx=0x7fbfe801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d4e9b94d294b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fbffd400640 1 -- 172.21.15.42:0/254145257 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe80b0070 con 0x7fc0000b51b0 2024-05-28T14:06:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fbffd400640 1 -- 172.21.15.42:0/254145257 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe80ab600 con 0x7fc0000b51b0 2024-05-28T14:06:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fbffd400640 1 -- 172.21.15.42:0/254145257 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe80ab9d0 con 0x7fc0000b51b0 2024-05-28T14:06:20.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fc005000640 1 -- 172.21.15.42:0/254145257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0000b51b0 msgr2=0x7fc0000d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:20.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fc005000640 1 --2- 172.21.15.42:0/254145257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0000b51b0 0x7fc0000d06f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc00006f1a0 tx=0x7fbfe801c260 comp rx=0 tx=0).stop 2024-05-28T14:06:20.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fc005000640 1 -- 172.21.15.42:0/254145257 shutdown_connections 2024-05-28T14:06:20.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fc005000640 1 --2- 172.21.15.42:0/254145257 >> v2:172.21.15.63:3300/0 conn(0x7fc0000d0c30 0x7fc000072250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fc005000640 1 --2- 172.21.15.42:0/254145257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0000b51b0 0x7fc0000d06f0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fc005000640 1 --2- 172.21.15.42:0/254145257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc00015e5c0 0x7fc0000b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fc005000640 1 -- 172.21.15.42:0/254145257 >> 172.21.15.42:0/254145257 conn(0x7fc00006bdf0 msgr2=0x7fc00006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:20.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fc005000640 1 -- 172.21.15.42:0/254145257 shutdown_connections 2024-05-28T14:06:20.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fc005000640 1 -- 172.21.15.42:0/254145257 wait complete. 2024-05-28T14:06:20.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fc005000640 1 Processor -- start 2024-05-28T14:06:20.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.174+0000 7fc005000640 1 -- start start 2024-05-28T14:06:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fc005000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0000b51b0 0x7fc000154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fc005000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0000d0c30 0x7fc0001fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fc005000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc00015e5c0 0x7fc00009a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fc005000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc0000adbf0 con 0x7fc00015e5c0 2024-05-28T14:06:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fc005000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0000ada70 con 0x7fc0000d0c30 2024-05-28T14:06:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fc005000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc0000add70 con 0x7fc0000b51b0 2024-05-28T14:06:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbffe800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc00015e5c0 0x7fc00009a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbffe800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc00015e5c0 0x7fc00009a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42960/0 (socket says 172.21.15.42:42960) 2024-05-28T14:06:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbffe800640 1 -- 172.21.15.42:0/3292110811 learned_addr learned my addr 172.21.15.42:0/3292110811 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbffde00640 1 --2- 172.21.15.42:0/3292110811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0000b51b0 0x7fc000154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbff7e00640 1 --2- 172.21.15.42:0/3292110811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0000d0c30 0x7fc0001fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbffe800640 1 -- 172.21.15.42:0/3292110811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0000b51b0 msgr2=0x7fc000154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbffe800640 1 --2- 172.21.15.42:0/3292110811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0000b51b0 0x7fc000154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbffe800640 1 -- 172.21.15.42:0/3292110811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0000d0c30 msgr2=0x7fc0001fb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbffe800640 1 --2- 172.21.15.42:0/3292110811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0000d0c30 0x7fc0001fb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbffe800640 1 -- 172.21.15.42:0/3292110811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc000153d50 con 0x7fc00015e5c0 2024-05-28T14:06:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbffe800640 1 --2- 172.21.15.42:0/3292110811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc00015e5c0 0x7fc00009a2d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbff8007e60 tx=0x7fbff800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:20.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbff6000640 1 -- 172.21.15.42:0/3292110811 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff8014620 con 0x7fc00015e5c0 2024-05-28T14:06:20.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbff6000640 1 -- 172.21.15.42:0/3292110811 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff8019070 con 0x7fc00015e5c0 2024-05-28T14:06:20.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbff6000640 1 -- 172.21.15.42:0/3292110811 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff8014a20 con 0x7fc00015e5c0 2024-05-28T14:06:20.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fc005000640 1 -- 172.21.15.42:0/3292110811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc000154030 con 0x7fc00015e5c0 2024-05-28T14:06:20.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fc005000640 1 -- 172.21.15.42:0/3292110811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0001545f0 con 0x7fc00015e5c0 2024-05-28T14:06:20.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fc005000640 1 -- 172.21.15.42:0/3292110811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc000068a90 con 0x7fc00015e5c0 2024-05-28T14:06:20.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.178+0000 7fbff6000640 1 -- 172.21.15.42:0/3292110811 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7fbff8014bc0 con 0x7fc00015e5c0 2024-05-28T14:06:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.182+0000 7fbff6000640 1 --2- 172.21.15.42:0/3292110811 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfdc069070 0x7fbfdc06b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.182+0000 7fbff6000640 1 -- 172.21.15.42:0/3292110811 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4971+0+0 (secure 0 0 0) 0x7fbff8085b20 con 0x7fc00015e5c0 2024-05-28T14:06:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.182+0000 7fbffde00640 1 --2- 172.21.15.42:0/3292110811 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfdc069070 0x7fbfdc06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.182+0000 7fbffde00640 1 --2- 172.21.15.42:0/3292110811 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfdc069070 0x7fbfdc06b530 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbff00079c0 tx=0x7fbff0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:20.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.186+0000 7fbff6000640 1 -- 172.21.15.42:0/3292110811 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbff80574e0 con 0x7fc00015e5c0 2024-05-28T14:06:20.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.382+0000 7fc005000640 1 -- 172.21.15.42:0/3292110811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0001573e0 con 0x7fc00015e5c0 2024-05-28T14:06:20.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.382+0000 7fbff6000640 1 -- 172.21.15.42:0/3292110811 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3867 (secure 0 0 0) 0x7fbff805b7e0 con 0x7fc00015e5c0 2024-05-28T14:06:20.371 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:20.371 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1310016671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1310016671},{"type":"v1","addr":"172.21.15.63:6834","nonce":1310016671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1602237641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1602237641},{"type":"v1","addr":"172.21.15.42:6835","nonce":1602237641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/142807607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":142807607},{"type":"v1","addr":"172.21.15.42:6837","nonce":142807607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":55979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1038977426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1038977426},{"type":"v1","addr":"172.21.15.63:6835","nonce":1038977426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[]} 2024-05-28T14:06:20.371 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 39 2024-05-28T14:06:20.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.386+0000 7fc005000640 1 -- 172.21.15.42:0/3292110811 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfdc069070 msgr2=0x7fbfdc06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:20.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.386+0000 7fc005000640 1 --2- 172.21.15.42:0/3292110811 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfdc069070 0x7fbfdc06b530 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbff00079c0 tx=0x7fbff0008040 comp rx=0 tx=0).stop 2024-05-28T14:06:20.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.386+0000 7fc005000640 1 -- 172.21.15.42:0/3292110811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc00015e5c0 msgr2=0x7fc00009a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:20.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.386+0000 7fc005000640 1 --2- 172.21.15.42:0/3292110811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc00015e5c0 0x7fc00009a2d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbff8007e60 tx=0x7fbff800e4b0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.386+0000 7fc005000640 1 -- 172.21.15.42:0/3292110811 shutdown_connections 2024-05-28T14:06:20.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.386+0000 7fc005000640 1 --2- 172.21.15.42:0/3292110811 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfdc069070 0x7fbfdc06b530 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.386+0000 7fc005000640 1 --2- 172.21.15.42:0/3292110811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc00015e5c0 0x7fc00009a2d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.386+0000 7fc005000640 1 --2- 172.21.15.42:0/3292110811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0000d0c30 0x7fc0001fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.386+0000 7fc005000640 1 --2- 172.21.15.42:0/3292110811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0000b51b0 0x7fc000154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.386+0000 7fc005000640 1 -- 172.21.15.42:0/3292110811 >> 172.21.15.42:0/3292110811 conn(0x7fc00006bdf0 msgr2=0x7fc0000b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:20.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.386+0000 7fc005000640 1 -- 172.21.15.42:0/3292110811 shutdown_connections 2024-05-28T14:06:20.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.386+0000 7fc005000640 1 -- 172.21.15.42:0/3292110811 wait complete. 2024-05-28T14:06:20.393 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-28T14:06:20.393 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-28T14:06:20.393 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-28T14:06:20.393 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:06:20.393 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-28T14:06:20.393 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-28T14:06:20.394 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-28T14:06:20.394 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:06:20.394 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-28T14:06:20.394 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-28T14:06:20.394 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-28T14:06:20.394 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:06:20.394 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-28T14:06:20.395 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-28T14:06:20.395 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-28T14:06:20.395 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:06:20.409 INFO:tasks.ceph.mds.b:Stopped 2024-05-28T14:06:20.409 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-28T14:06:20.410 INFO:tasks.ceph.mds.d:Stopped 2024-05-28T14:06:20.410 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-28T14:06:20.411 INFO:tasks.ceph.mds.a:Stopped 2024-05-28T14:06:20.411 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-28T14:06:20.412 INFO:tasks.ceph.mds.c:Stopped 2024-05-28T14:06:20.412 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-28T14:06:20.456 INFO:tasks.ceph.mds.b:Started 2024-05-28T14:06:20.456 DEBUG:teuthology.parallel:result is None 2024-05-28T14:06:20.458 INFO:tasks.ceph.mds.a:Started 2024-05-28T14:06:20.458 DEBUG:teuthology.parallel:result is None 2024-05-28T14:06:20.458 INFO:tasks.ceph.mds.d:Started 2024-05-28T14:06:20.458 DEBUG:teuthology.parallel:result is None 2024-05-28T14:06:20.459 INFO:tasks.ceph.mds.c:Started 2024-05-28T14:06:20.459 DEBUG:teuthology.parallel:result is None 2024-05-28T14:06:20.460 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-28T14:06:20.548 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-05-28T14:06:20.548 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-05-28T14:06:20.549 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T14:06:20.550 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-28T14:06:20.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc151800640 1 Processor -- start 2024-05-28T14:06:20.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc151800640 1 -- start start 2024-05-28T14:06:20.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc151800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc14c09baf0 0x7fc14c0d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:20.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc151800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc14c0d0cd0 0x7fc14c0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:20.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc151800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc14c0d10e0 con 0x7fc14c15fdf0 2024-05-28T14:06:20.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc151800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc14c0ce6e0 con 0x7fc14c0d0cd0 2024-05-28T14:06:20.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc151800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc14c0ce400 con 0x7fc14c09baf0 2024-05-28T14:06:20.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc14b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc14c0d0cd0 0x7fc14c0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:20.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc14b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc14c0d0cd0 0x7fc14c0b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42978/0 (socket says 172.21.15.42:42978) 2024-05-28T14:06:20.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc14b400640 1 -- 172.21.15.42:0/3354926530 learned_addr learned my addr 172.21.15.42:0/3354926530 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:20.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc143e00640 1 --2- 172.21.15.42:0/3354926530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc14c09baf0 0x7fc14c0d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:20.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc14b400640 1 -- 172.21.15.42:0/3354926530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc14c09baf0 msgr2=0x7fc14c0d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:20.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc14b400640 1 --2- 172.21.15.42:0/3354926530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc14c09baf0 0x7fc14c0d0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc14b400640 1 -- 172.21.15.42:0/3354926530 >> v1:172.21.15.63:6789/0 conn(0x7fc14c15fdf0 legacy=0x7fc14c0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:20.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.622+0000 7fc14b400640 1 -- 172.21.15.42:0/3354926530 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14c0b1ea0 con 0x7fc14c0d0cd0 2024-05-28T14:06:20.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.626+0000 7fc14b400640 1 --2- 172.21.15.42:0/3354926530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc14c0d0cd0 0x7fc14c0b2960 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc144009e50 tx=0x7fc14401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9f5242ad29311a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:20.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.626+0000 7fc14a000640 1 -- 172.21.15.42:0/3354926530 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1440a5850 con 0x7fc14c0d0cd0 2024-05-28T14:06:20.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.626+0000 7fc14a000640 1 -- 172.21.15.42:0/3354926530 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1440a59f0 con 0x7fc14c0d0cd0 2024-05-28T14:06:20.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.626+0000 7fc14a000640 1 -- 172.21.15.42:0/3354926530 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1440a5dc0 con 0x7fc14c0d0cd0 2024-05-28T14:06:20.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.626+0000 7fc151800640 1 -- 172.21.15.42:0/3354926530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc14c0d0cd0 msgr2=0x7fc14c0b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:20.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.626+0000 7fc151800640 1 --2- 172.21.15.42:0/3354926530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc14c0d0cd0 0x7fc14c0b2960 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc144009e50 tx=0x7fc14401c970 comp rx=0 tx=0).stop 2024-05-28T14:06:20.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.626+0000 7fc151800640 1 -- 172.21.15.42:0/3354926530 shutdown_connections 2024-05-28T14:06:20.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.626+0000 7fc151800640 1 --2- 172.21.15.42:0/3354926530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc14c0d0cd0 0x7fc14c0b2960 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.626+0000 7fc151800640 1 --2- 172.21.15.42:0/3354926530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc14c09baf0 0x7fc14c0d0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.626+0000 7fc151800640 1 -- 172.21.15.42:0/3354926530 >> 172.21.15.42:0/3354926530 conn(0x7fc14c0b4050 msgr2=0x7fc14c0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:20.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.626+0000 7fc151800640 1 -- 172.21.15.42:0/3354926530 shutdown_connections 2024-05-28T14:06:20.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.626+0000 7fc151800640 1 -- 172.21.15.42:0/3354926530 wait complete. 2024-05-28T14:06:20.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc151800640 1 Processor -- start 2024-05-28T14:06:20.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc151800640 1 -- start start 2024-05-28T14:06:20.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc151800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc14c09baf0 0x7fc14c166f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:20.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc151800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc14c0d0cd0 0x7fc14c06ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:20.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc151800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc14c15fdf0 0x7fc14c14e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:20.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc151800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c0b3830 con 0x7fc14c09baf0 2024-05-28T14:06:20.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc151800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c0b36b0 con 0x7fc14c0d0cd0 2024-05-28T14:06:20.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc151800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc14c0b39b0 con 0x7fc14c15fdf0 2024-05-28T14:06:20.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc14aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc14c09baf0 0x7fc14c166f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:20.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc14aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc14c09baf0 0x7fc14c166f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42988/0 (socket says 172.21.15.42:42988) 2024-05-28T14:06:20.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc14aa00640 1 -- 172.21.15.42:0/2282432168 learned_addr learned my addr 172.21.15.42:0/2282432168 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:20.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc143e00640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc14c0d0cd0 0x7fc14c06ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:20.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc14b400640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc14c15fdf0 0x7fc14c14e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:20.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc14aa00640 1 -- 172.21.15.42:0/2282432168 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc14c15fdf0 msgr2=0x7fc14c14e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:20.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc14aa00640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc14c15fdf0 0x7fc14c14e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc14aa00640 1 -- 172.21.15.42:0/2282432168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc14c0d0cd0 msgr2=0x7fc14c06ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc14aa00640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc14c0d0cd0 0x7fc14c06ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc14aa00640 1 -- 172.21.15.42:0/2282432168 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14c06cd50 con 0x7fc14c09baf0 2024-05-28T14:06:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc143e00640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc14c0d0cd0 0x7fc14c06ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc14aa00640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc14c09baf0 0x7fc14c166f30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00c3c0 tx=0x7fc13c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc142a00640 1 -- 172.21.15.42:0/2282432168 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c017070 con 0x7fc14c09baf0 2024-05-28T14:06:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc142a00640 1 -- 172.21.15.42:0/2282432168 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc13c0052d0 con 0x7fc14c09baf0 2024-05-28T14:06:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc142a00640 1 -- 172.21.15.42:0/2282432168 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c003840 con 0x7fc14c09baf0 2024-05-28T14:06:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc151800640 1 -- 172.21.15.42:0/2282432168 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14c06d030 con 0x7fc14c09baf0 2024-05-28T14:06:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.630+0000 7fc151800640 1 -- 172.21.15.42:0/2282432168 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc14c06d5f0 con 0x7fc14c09baf0 2024-05-28T14:06:20.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.634+0000 7fc151800640 1 -- 172.21.15.42:0/2282432168 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc14c0a0650 con 0x7fc14c09baf0 2024-05-28T14:06:20.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.634+0000 7fc142a00640 1 -- 172.21.15.42:0/2282432168 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7fc13c01a070 con 0x7fc14c09baf0 2024-05-28T14:06:20.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.634+0000 7fc142a00640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc128069070 0x7fc12806b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:20.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.634+0000 7fc142a00640 1 -- 172.21.15.42:0/2282432168 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4971+0+0 (secure 0 0 0) 0x7fc13c084750 con 0x7fc14c09baf0 2024-05-28T14:06:20.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.634+0000 7fc143e00640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc128069070 0x7fc12806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:20.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.634+0000 7fc143e00640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc128069070 0x7fc12806b530 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc14c1faf80 tx=0x7fc134008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:20.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.638+0000 7fc142a00640 1 -- 172.21.15.42:0/2282432168 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc14c0d10e0 con 0x7fc14c09baf0 2024-05-28T14:06:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:20.818+0000 7fc151800640 1 -- 172.21.15.42:0/2282432168 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc14c071d60 con 0x7fc14c09baf0 2024-05-28T14:06:21.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.386+0000 7fc142a00640 1 -- 172.21.15.42:0/2282432168 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v68) ==== 97+0+0 (secure 0 0 0) 0x7fc13c005480 con 0x7fc14c09baf0 2024-05-28T14:06:21.373 INFO:teuthology.orchestra.run.smithi042.stderr: removed all blocklist entries 2024-05-28T14:06:21.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.390+0000 7fc151800640 1 -- 172.21.15.42:0/2282432168 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc128069070 msgr2=0x7fc12806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:21.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.390+0000 7fc151800640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc128069070 0x7fc12806b530 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc14c1faf80 tx=0x7fc134008040 comp rx=0 tx=0).stop 2024-05-28T14:06:21.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.390+0000 7fc151800640 1 -- 172.21.15.42:0/2282432168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc14c09baf0 msgr2=0x7fc14c166f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:21.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.390+0000 7fc151800640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc14c09baf0 0x7fc14c166f30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00c3c0 tx=0x7fc13c00c880 comp rx=0 tx=0).stop 2024-05-28T14:06:21.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.390+0000 7fc151800640 1 -- 172.21.15.42:0/2282432168 shutdown_connections 2024-05-28T14:06:21.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.390+0000 7fc151800640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc128069070 0x7fc12806b530 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.390+0000 7fc151800640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc14c15fdf0 0x7fc14c14e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.390+0000 7fc151800640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc14c0d0cd0 0x7fc14c06ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.390+0000 7fc151800640 1 --2- 172.21.15.42:0/2282432168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc14c09baf0 0x7fc14c166f30 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.390+0000 7fc151800640 1 -- 172.21.15.42:0/2282432168 >> 172.21.15.42:0/2282432168 conn(0x7fc14c0b4050 msgr2=0x7fc14c0cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:21.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.390+0000 7fc151800640 1 -- 172.21.15.42:0/2282432168 shutdown_connections 2024-05-28T14:06:21.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.390+0000 7fc151800640 1 -- 172.21.15.42:0/2282432168 wait complete. 2024-05-28T14:06:21.399 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-28T14:06:21.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.554+0000 7f5716e00640 1 Processor -- start 2024-05-28T14:06:21.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.554+0000 7f5716e00640 1 -- start start 2024-05-28T14:06:21.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f5716e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5710070c60 0x7f571015fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:21.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f5716e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5710070710 0x7f57100aec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:21.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f5716e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5710070360 con 0x7f571015ed40 2024-05-28T14:06:21.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f5716e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f57100cfcb0 con 0x7f5710070710 2024-05-28T14:06:21.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f5716e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f57100700a0 con 0x7f5710070c60 2024-05-28T14:06:21.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f570f400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5710070c60 0x7f571015fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:21.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f5714e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5710070710 0x7f57100aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:21.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f570fe00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f571015ed40 0x7f571015f4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:59732/0 (socket says 172.21.15.42:59732) 2024-05-28T14:06:21.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f570fe00640 1 -- 172.21.15.42:0/3042072723 learned_addr learned my addr 172.21.15.42:0/3042072723 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:21.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f570ea00640 1 -- 172.21.15.42:0/3042072723 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 295516379 0 0) 0x7f5710070360 con 0x7f571015ed40 2024-05-28T14:06:21.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f570ea00640 1 -- 172.21.15.42:0/3042072723 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56f4003530 con 0x7f571015ed40 2024-05-28T14:06:21.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f570f400640 1 -- 172.21.15.42:0/3042072723 >> v2:172.21.15.63:3300/0 conn(0x7f5710070710 msgr2=0x7f57100aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:21.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f570f400640 1 --2- 172.21.15.42:0/3042072723 >> v2:172.21.15.63:3300/0 conn(0x7f5710070710 0x7f57100aec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f570f400640 1 -- 172.21.15.42:0/3042072723 >> v1:172.21.15.63:6789/0 conn(0x7f571015ed40 legacy=0x7f571015f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:21.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f570f400640 1 -- 172.21.15.42:0/3042072723 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57100d0140 con 0x7f5710070c60 2024-05-28T14:06:21.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.558+0000 7f5714e00640 1 --2- 172.21.15.42:0/3042072723 >> v2:172.21.15.63:3300/0 conn(0x7f5710070710 0x7f57100aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:06:21.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.562+0000 7f570f400640 1 --2- 172.21.15.42:0/3042072723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5710070c60 0x7f571015fbc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0097c0 tx=0x7f56fc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea27e3dd9925f8d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:21.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.562+0000 7f570ea00640 1 -- 172.21.15.42:0/3042072723 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0b0070 con 0x7f5710070c60 2024-05-28T14:06:21.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.562+0000 7f570ea00640 1 -- 172.21.15.42:0/3042072723 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56fc0ab600 con 0x7f5710070c60 2024-05-28T14:06:21.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.562+0000 7f570ea00640 1 -- 172.21.15.42:0/3042072723 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0ab9d0 con 0x7f5710070c60 2024-05-28T14:06:21.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.562+0000 7f5716e00640 1 -- 172.21.15.42:0/3042072723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5710070c60 msgr2=0x7f571015fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:21.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.562+0000 7f5716e00640 1 --2- 172.21.15.42:0/3042072723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5710070c60 0x7f571015fbc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0097c0 tx=0x7f56fc01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.562+0000 7f5716e00640 1 -- 172.21.15.42:0/3042072723 shutdown_connections 2024-05-28T14:06:21.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.562+0000 7f5716e00640 1 --2- 172.21.15.42:0/3042072723 >> v2:172.21.15.63:3300/0 conn(0x7f5710070710 0x7f57100aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.562+0000 7f5716e00640 1 --2- 172.21.15.42:0/3042072723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5710070c60 0x7f571015fbc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.562+0000 7f5716e00640 1 -- 172.21.15.42:0/3042072723 >> 172.21.15.42:0/3042072723 conn(0x7f57100b2f80 msgr2=0x7f57100b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:21.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.562+0000 7f5716e00640 1 -- 172.21.15.42:0/3042072723 shutdown_connections 2024-05-28T14:06:21.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.562+0000 7f5716e00640 1 -- 172.21.15.42:0/3042072723 wait complete. 2024-05-28T14:06:21.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.562+0000 7f5716e00640 1 Processor -- start 2024-05-28T14:06:21.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.562+0000 7f5716e00640 1 -- start start 2024-05-28T14:06:21.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f5716e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5710070710 0x7f5710164840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:21.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f5716e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5710070c60 0x7f571015c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:21.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f5716e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f571015ed40 0x7f57100b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:21.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f5716e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f57100af970 con 0x7f5710070c60 2024-05-28T14:06:21.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f5716e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57100b3ec0 con 0x7f571015ed40 2024-05-28T14:06:21.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f5716e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f57100afaf0 con 0x7f5710070710 2024-05-28T14:06:21.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f570f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5710070c60 0x7f571015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:21.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f570f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5710070c60 0x7f571015c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42998/0 (socket says 172.21.15.42:42998) 2024-05-28T14:06:21.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f570f400640 1 -- 172.21.15.42:0/4102240166 learned_addr learned my addr 172.21.15.42:0/4102240166 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:21.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f570fe00640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5710070710 0x7f5710164840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:21.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f5714e00640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f571015ed40 0x7f57100b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:21.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f570f400640 1 -- 172.21.15.42:0/4102240166 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5710070710 msgr2=0x7f5710164840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:21.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f570f400640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5710070710 0x7f5710164840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f570f400640 1 -- 172.21.15.42:0/4102240166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f571015ed40 msgr2=0x7f57100b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:21.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f570f400640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f571015ed40 0x7f57100b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f570f400640 1 -- 172.21.15.42:0/4102240166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571015d0f0 con 0x7f5710070c60 2024-05-28T14:06:21.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f5714e00640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f571015ed40 0x7f57100b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:21.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f570f400640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5710070c60 0x7f571015c630 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0ae040 tx=0x7f56fc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:21.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f5707e00640 1 -- 172.21.15.42:0/4102240166 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0b0040 con 0x7f5710070c60 2024-05-28T14:06:21.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f5707e00640 1 -- 172.21.15.42:0/4102240166 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56fc0b9070 con 0x7f5710070c60 2024-05-28T14:06:21.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f5707e00640 1 -- 172.21.15.42:0/4102240166 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0b4650 con 0x7f5710070c60 2024-05-28T14:06:21.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f5716e00640 1 -- 172.21.15.42:0/4102240166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5710155960 con 0x7f5710070c60 2024-05-28T14:06:21.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f5716e00640 1 -- 172.21.15.42:0/4102240166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f571015d3d0 con 0x7f5710070c60 2024-05-28T14:06:21.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.566+0000 7f5716e00640 1 -- 172.21.15.42:0/4102240166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5710068a90 con 0x7f5710070c60 2024-05-28T14:06:21.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.570+0000 7f5707e00640 1 -- 172.21.15.42:0/4102240166 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f56fc0b47f0 con 0x7f5710070c60 2024-05-28T14:06:21.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.570+0000 7f5707e00640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56f0069070 0x7f56f006b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:21.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.570+0000 7f5707e00640 1 -- 172.21.15.42:0/4102240166 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4885+0+0 (secure 0 0 0) 0x7f56fc11f140 con 0x7f5710070c60 2024-05-28T14:06:21.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.570+0000 7f570fe00640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56f0069070 0x7f56f006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:21.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.570+0000 7f570fe00640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56f0069070 0x7f56f006b530 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5700005890 tx=0x7f5700009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:21.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.574+0000 7f5707e00640 1 -- 172.21.15.42:0/4102240166 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f56fc016090 con 0x7f5710070c60 2024-05-28T14:06:21.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.798+0000 7f5716e00640 1 -- 172.21.15.42:0/4102240166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f571014e3e0 con 0x7f5710070c60 2024-05-28T14:06:21.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.798+0000 7f5707e00640 1 -- 172.21.15.42:0/4102240166 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f56fc0f10d0 con 0x7f5710070c60 2024-05-28T14:06:21.787 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:21.787 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:06:21.787 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_dump": [ 2024-05-28T14:06:21.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.787 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.a", 2024-05-28T14:06:21.787 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmE+3RLRAADuTKGfKeVafGeLpgr9WKUg==", 2024-05-28T14:06:21.787 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.788 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:06:21.788 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:06:21.788 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:06:21.788 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.788 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.b", 2024-05-28T14:06:21.788 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVm1rmVOhAAV46KzDJnRfDZYIZanS0g2g==", 2024-05-28T14:06:21.788 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.788 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:06:21.788 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:06:21.788 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:06:21.788 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.789 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.c", 2024-05-28T14:06:21.789 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVme4aYNRAAVxJTlugnxtFTexUclF/IwA==", 2024-05-28T14:06:21.789 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.789 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:06:21.789 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:06:21.789 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:06:21.789 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.789 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.d", 2024-05-28T14:06:21.790 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmeAdBBhAAr4IVFQpjQXFVlnFTHnPV9w==", 2024-05-28T14:06:21.790 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.790 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:06:21.790 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:06:21.790 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:06:21.790 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.790 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.0", 2024-05-28T14:06:21.790 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDB4lVmGMDBJhAA3jIyR7XKLGXEgimIGNsw6A==", 2024-05-28T14:06:21.790 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.790 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:06:21.791 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:06:21.791 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.791 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.1", 2024-05-28T14:06:21.791 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDE4lVmxMisLxAAcIz7GPlkZwvW5ig+DbAVGg==", 2024-05-28T14:06:21.791 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.791 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:06:21.791 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:06:21.791 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.792 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.2", 2024-05-28T14:06:21.792 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDH4lVmSSk4OhAAmLlLysM4Im/Kew4Xbq2/OA==", 2024-05-28T14:06:21.792 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.792 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:06:21.792 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:06:21.792 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.792 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.3", 2024-05-28T14:06:21.792 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDL4lVmvzrYBhAAiAAOQXZBuTsMVYa1vE5rkg==", 2024-05-28T14:06:21.792 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.792 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:06:21.792 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:06:21.793 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.793 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.793 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.4", 2024-05-28T14:06:21.793 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDQ4lVmrqnTOhAA2pIfgBT9LJDIhp+GgFy8Gg==", 2024-05-28T14:06:21.793 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.793 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:06:21.793 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:06:21.793 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.793 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.793 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.5", 2024-05-28T14:06:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDU4lVmYpsQCxAAoTK2aMBPrHKvsUnNOTj/Dw==", 2024-05-28T14:06:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:06:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:06:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.6", 2024-05-28T14:06:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDX4lVm6TCAFxAAs3rkDyKcLhWrn/EkIo0bVA==", 2024-05-28T14:06:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:06:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:06:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.7", 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDa4lVmr5NwIRAA7Lb70VYV82dr0Frn8CCUXg==", 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.0", 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw==", 2024-05-28T14:06:21.795 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:06:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-05-28T14:06:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-28T14:06:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.1", 2024-05-28T14:06:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmcfkLFBAADNyODKTtYAlFWquZ/LeeqQ==", 2024-05-28T14:06:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:06:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:06:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:06:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:06:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.2", 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmoxx+EBAA8mLNak5AwRmWQIgkYtMOrg==", 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.3", 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmxSUsGhAAbgr1eOh/sMfrEONKKHFo0g==", 2024-05-28T14:06:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:06:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:06:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:06:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:06:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.admin", 2024-05-28T14:06:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmOSBPERAAt4x2DBal4PnqxqeWQutLDQ==", 2024-05-28T14:06:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:06:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:06:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:06:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.x", 2024-05-28T14:06:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmNO5tKRAA70hPLwoE83keovlq2rolxQ==", 2024-05-28T14:06:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:06:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T14:06:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.y", 2024-05-28T14:06:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmxTVyJRAATMLAAqKne20YvV537uXZ6g==", 2024-05-28T14:06:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:06:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:06:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T14:06:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:06:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-28T14:06:21.800 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:06:21.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.802+0000 7f5716e00640 1 -- 172.21.15.42:0/4102240166 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56f0069070 msgr2=0x7f56f006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:21.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.802+0000 7f5716e00640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56f0069070 0x7f56f006b530 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5700005890 tx=0x7f5700009040 comp rx=0 tx=0).stop 2024-05-28T14:06:21.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.802+0000 7f5716e00640 1 -- 172.21.15.42:0/4102240166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5710070c60 msgr2=0x7f571015c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:21.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.802+0000 7f5716e00640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5710070c60 0x7f571015c630 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0ae040 tx=0x7f56fc019240 comp rx=0 tx=0).stop 2024-05-28T14:06:21.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.802+0000 7f5716e00640 1 -- 172.21.15.42:0/4102240166 shutdown_connections 2024-05-28T14:06:21.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.802+0000 7f5716e00640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56f0069070 0x7f56f006b530 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.802+0000 7f5716e00640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f571015ed40 0x7f57100b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.802+0000 7f5716e00640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5710070c60 0x7f571015c630 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.802+0000 7f5716e00640 1 --2- 172.21.15.42:0/4102240166 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5710070710 0x7f5710164840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.802+0000 7f5716e00640 1 -- 172.21.15.42:0/4102240166 >> 172.21.15.42:0/4102240166 conn(0x7f57100b2f80 msgr2=0x7f571015f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:21.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.802+0000 7f5716e00640 1 -- 172.21.15.42:0/4102240166 shutdown_connections 2024-05-28T14:06:21.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.802+0000 7f5716e00640 1 -- 172.21.15.42:0/4102240166 wait complete. 2024-05-28T14:06:21.809 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:06:21.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bbea00640 1 Processor -- start 2024-05-28T14:06:21.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bbea00640 1 -- start start 2024-05-28T14:06:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bbea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9bb815c990 0x7f9bb80d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bbea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9bb80d06a0 0x7f9bb80b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bbea00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bb80cd300 con 0x7f9bb809bbd0 2024-05-28T14:06:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bbea00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9bb815cd70 con 0x7f9bb815c990 2024-05-28T14:06:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bbea00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9bb80cd5e0 con 0x7f9bb80d06a0 2024-05-28T14:06:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bb7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9bb815c990 0x7f9bb80d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bb7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9bb815c990 0x7f9bb80d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43012/0 (socket says 172.21.15.42:43012) 2024-05-28T14:06:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bb7e00640 1 -- 172.21.15.42:0/2490956615 learned_addr learned my addr 172.21.15.42:0/2490956615 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bbca00640 1 --2- 172.21.15.42:0/2490956615 >> v2:172.21.15.63:3300/0 conn(0x7f9bb80d06a0 0x7f9bb80b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bb7e00640 1 -- 172.21.15.42:0/2490956615 >> v2:172.21.15.63:3300/0 conn(0x7f9bb80d06a0 msgr2=0x7f9bb80b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bb7e00640 1 --2- 172.21.15.42:0/2490956615 >> v2:172.21.15.63:3300/0 conn(0x7f9bb80d06a0 0x7f9bb80b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bb7e00640 1 -- 172.21.15.42:0/2490956615 >> v1:172.21.15.63:6789/0 conn(0x7f9bb809bbd0 legacy=0x7f9bb809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.966+0000 7f9bb7e00640 1 -- 172.21.15.42:0/2490956615 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb80cd8c0 con 0x7f9bb815c990 2024-05-28T14:06:21.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.970+0000 7f9bb7e00640 1 --2- 172.21.15.42:0/2490956615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb815c990 0x7f9bb80d0160 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8073870 tx=0x7f9ba801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1655f2fff0319690 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:21.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.970+0000 7f9bb7400640 1 -- 172.21.15.42:0/2490956615 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba80aa650 con 0x7f9bb815c990 2024-05-28T14:06:21.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.970+0000 7f9bb7400640 1 -- 172.21.15.42:0/2490956615 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba80af070 con 0x7f9bb815c990 2024-05-28T14:06:21.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.970+0000 7f9bb7400640 1 -- 172.21.15.42:0/2490956615 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba80aaa10 con 0x7f9bb815c990 2024-05-28T14:06:21.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.970+0000 7f9bbea00640 1 -- 172.21.15.42:0/2490956615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb815c990 msgr2=0x7f9bb80d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:21.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.970+0000 7f9bbea00640 1 --2- 172.21.15.42:0/2490956615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb815c990 0x7f9bb80d0160 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8073870 tx=0x7f9ba801c4f0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.970+0000 7f9bbea00640 1 -- 172.21.15.42:0/2490956615 shutdown_connections 2024-05-28T14:06:21.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.970+0000 7f9bbea00640 1 --2- 172.21.15.42:0/2490956615 >> v2:172.21.15.63:3300/0 conn(0x7f9bb80d06a0 0x7f9bb80b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.970+0000 7f9bbea00640 1 --2- 172.21.15.42:0/2490956615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb815c990 0x7f9bb80d0160 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.970+0000 7f9bbea00640 1 -- 172.21.15.42:0/2490956615 >> 172.21.15.42:0/2490956615 conn(0x7f9bb80b4080 msgr2=0x7f9bb80b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:21.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.970+0000 7f9bbea00640 1 -- 172.21.15.42:0/2490956615 shutdown_connections 2024-05-28T14:06:21.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.970+0000 7f9bbea00640 1 -- 172.21.15.42:0/2490956615 wait complete. 2024-05-28T14:06:21.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bbea00640 1 Processor -- start 2024-05-28T14:06:21.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bbea00640 1 -- start start 2024-05-28T14:06:21.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bbea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb80d06a0 0x7f9bb8155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:21.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bbea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9bb815c990 0x7f9bb8166f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:21.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bbea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bb80c6a90 0x7f9bb8165500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:21.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bbea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb809b2e0 con 0x7f9bb80d06a0 2024-05-28T14:06:21.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bbea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb809b160 con 0x7f9bb80c6a90 2024-05-28T14:06:21.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bbea00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb80b6130 con 0x7f9bb815c990 2024-05-28T14:06:21.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb80d06a0 0x7f9bb8155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:21.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb80d06a0 0x7f9bb8155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43014/0 (socket says 172.21.15.42:43014) 2024-05-28T14:06:21.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bb7e00640 1 -- 172.21.15.42:0/251883530 learned_addr learned my addr 172.21.15.42:0/251883530 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:21.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bb7e00640 1 -- 172.21.15.42:0/251883530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9bb815c990 msgr2=0x7f9bb8166f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:21.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bb7e00640 1 --2- 172.21.15.42:0/251883530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9bb815c990 0x7f9bb8166f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bbca00640 1 --2- 172.21.15.42:0/251883530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bb80c6a90 0x7f9bb8165500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:21.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bb7e00640 1 -- 172.21.15.42:0/251883530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bb80c6a90 msgr2=0x7f9bb8165500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:21.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bb7e00640 1 --2- 172.21.15.42:0/251883530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bb80c6a90 0x7f9bb8165500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:21.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bb7e00640 1 -- 172.21.15.42:0/251883530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb81674a0 con 0x7f9bb80d06a0 2024-05-28T14:06:21.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bbca00640 1 --2- 172.21.15.42:0/251883530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bb80c6a90 0x7f9bb8165500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:21.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bb7e00640 1 --2- 172.21.15.42:0/251883530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb80d06a0 0x7f9bb8155a50 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9ba801c9d0 tx=0x7f9ba801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:21.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bb4c00640 1 -- 172.21.15.42:0/251883530 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba80bd5f0 con 0x7f9bb80d06a0 2024-05-28T14:06:21.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bb4c00640 1 -- 172.21.15.42:0/251883530 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba80a6040 con 0x7f9bb80d06a0 2024-05-28T14:06:21.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bb4c00640 1 -- 172.21.15.42:0/251883530 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba80bd9f0 con 0x7f9bb80d06a0 2024-05-28T14:06:21.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bbea00640 1 -- 172.21.15.42:0/251883530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb8164780 con 0x7f9bb80d06a0 2024-05-28T14:06:21.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bbea00640 1 -- 172.21.15.42:0/251883530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb8164d00 con 0x7f9bb80d06a0 2024-05-28T14:06:21.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.974+0000 7f9bbea00640 1 -- 172.21.15.42:0/251883530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb8068a90 con 0x7f9bb80d06a0 2024-05-28T14:06:21.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.978+0000 7f9bb4c00640 1 -- 172.21.15.42:0/251883530 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f9ba80b33f0 con 0x7f9bb80d06a0 2024-05-28T14:06:21.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.978+0000 7f9bb4c00640 1 --2- 172.21.15.42:0/251883530 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b940690a0 0x7f9b9406b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:21.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.978+0000 7f9bb4c00640 1 -- 172.21.15.42:0/251883530 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4885+0+0 (secure 0 0 0) 0x7f9ba811f720 con 0x7f9bb80d06a0 2024-05-28T14:06:21.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.978+0000 7f9bafe00640 1 --2- 172.21.15.42:0/251883530 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b940690a0 0x7f9b9406b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:21.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.978+0000 7f9bafe00640 1 --2- 172.21.15.42:0/251883530 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b940690a0 0x7f9b9406b560 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0000f70 tx=0x7f9ba00079e0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:21.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:21.982+0000 7f9bb4c00640 1 -- 172.21.15.42:0/251883530 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9ba8126070 con 0x7f9bb80d06a0 2024-05-28T14:06:22.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.150+0000 7f9bbea00640 1 -- 172.21.15.42:0/251883530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bb8156a90 con 0x7f9bb80d06a0 2024-05-28T14:06:22.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.150+0000 7f9bb4c00640 1 -- 172.21.15.42:0/251883530 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+12874 (secure 0 0 0) 0x7f9ba80c3060 con 0x7f9bb80d06a0 2024-05-28T14:06:22.136 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:22.136 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":70,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:06:21.398692+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:06:22.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.150+0000 7f9bbea00640 1 -- 172.21.15.42:0/251883530 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b940690a0 msgr2=0x7f9b9406b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.150+0000 7f9bbea00640 1 --2- 172.21.15.42:0/251883530 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b940690a0 0x7f9b9406b560 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0000f70 tx=0x7f9ba00079e0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.150+0000 7f9bbea00640 1 -- 172.21.15.42:0/251883530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb80d06a0 msgr2=0x7f9bb8155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.150+0000 7f9bbea00640 1 --2- 172.21.15.42:0/251883530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb80d06a0 0x7f9bb8155a50 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9ba801c9d0 tx=0x7f9ba801ad30 comp rx=0 tx=0).stop 2024-05-28T14:06:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.150+0000 7f9bbea00640 1 -- 172.21.15.42:0/251883530 shutdown_connections 2024-05-28T14:06:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.150+0000 7f9bbea00640 1 --2- 172.21.15.42:0/251883530 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b940690a0 0x7f9b9406b560 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.150+0000 7f9bbea00640 1 --2- 172.21.15.42:0/251883530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bb80c6a90 0x7f9bb8165500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.150+0000 7f9bbea00640 1 --2- 172.21.15.42:0/251883530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9bb815c990 0x7f9bb8166f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.150+0000 7f9bbea00640 1 --2- 172.21.15.42:0/251883530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bb80d06a0 0x7f9bb8155a50 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.150+0000 7f9bbea00640 1 -- 172.21.15.42:0/251883530 >> 172.21.15.42:0/251883530 conn(0x7f9bb80b4080 msgr2=0x7f9bb80b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.150+0000 7f9bbea00640 1 -- 172.21.15.42:0/251883530 shutdown_connections 2024-05-28T14:06:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.150+0000 7f9bbea00640 1 -- 172.21.15.42:0/251883530 wait complete. 2024-05-28T14:06:22.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:06:22.158 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:06:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.314+0000 7fd07e000640 1 Processor -- start 2024-05-28T14:06:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.314+0000 7fd07e000640 1 -- start start 2024-05-28T14:06:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.318+0000 7fd07e000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd078144f80 0x7fd078072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.318+0000 7fd07e000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd07815c0c0 0x7fd07806f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.318+0000 7fd07e000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0780afe40 con 0x7fd07806ee20 2024-05-28T14:06:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.318+0000 7fd07e000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd07806f7c0 con 0x7fd078144f80 2024-05-28T14:06:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.318+0000 7fd07e000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0780724d0 con 0x7fd07815c0c0 2024-05-28T14:06:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.318+0000 7fd077400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd078144f80 0x7fd078072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.318+0000 7fd077400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd078144f80 0x7fd078072960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43024/0 (socket says 172.21.15.42:43024) 2024-05-28T14:06:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.318+0000 7fd077400640 1 -- 172.21.15.42:0/3053246966 learned_addr learned my addr 172.21.15.42:0/3053246966 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.318+0000 7fd06fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd07806ee20 0x7fd07815b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42598/0 (socket says 172.21.15.42:42598) 2024-05-28T14:06:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.318+0000 7fd077e00640 1 --2- 172.21.15.42:0/3053246966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd07815c0c0 0x7fd07806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.318+0000 7fd077400640 1 -- 172.21.15.42:0/3053246966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd07815c0c0 msgr2=0x7fd07806f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.318+0000 7fd077400640 1 --2- 172.21.15.42:0/3053246966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd07815c0c0 0x7fd07806f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.318+0000 7fd077400640 1 -- 172.21.15.42:0/3053246966 >> v1:172.21.15.42:6789/0 conn(0x7fd07806ee20 legacy=0x7fd07815b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.318+0000 7fd077400640 1 -- 172.21.15.42:0/3053246966 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07815c600 con 0x7fd078144f80 2024-05-28T14:06:22.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.322+0000 7fd077400640 1 --2- 172.21.15.42:0/3053246966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078144f80 0x7fd078072960 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd068009870 tx=0x7fd06801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae69c1fd88d98bdf server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:22.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.322+0000 7fd076a00640 1 -- 172.21.15.42:0/3053246966 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0680ab650 con 0x7fd078144f80 2024-05-28T14:06:22.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.322+0000 7fd076a00640 1 -- 172.21.15.42:0/3053246966 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0680b0070 con 0x7fd078144f80 2024-05-28T14:06:22.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.322+0000 7fd076a00640 1 -- 172.21.15.42:0/3053246966 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0680aba10 con 0x7fd078144f80 2024-05-28T14:06:22.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.322+0000 7fd07e000640 1 -- 172.21.15.42:0/3053246966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078144f80 msgr2=0x7fd078072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:22.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.322+0000 7fd07e000640 1 --2- 172.21.15.42:0/3053246966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078144f80 0x7fd078072960 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd068009870 tx=0x7fd06801c560 comp rx=0 tx=0).stop 2024-05-28T14:06:22.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.322+0000 7fd07e000640 1 -- 172.21.15.42:0/3053246966 shutdown_connections 2024-05-28T14:06:22.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.322+0000 7fd07e000640 1 --2- 172.21.15.42:0/3053246966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd07815c0c0 0x7fd07806f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.322+0000 7fd07e000640 1 --2- 172.21.15.42:0/3053246966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078144f80 0x7fd078072960 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.322+0000 7fd07e000640 1 -- 172.21.15.42:0/3053246966 >> 172.21.15.42:0/3053246966 conn(0x7fd0780b21f0 msgr2=0x7fd0780b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:22.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.322+0000 7fd07e000640 1 -- 172.21.15.42:0/3053246966 shutdown_connections 2024-05-28T14:06:22.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.322+0000 7fd07e000640 1 -- 172.21.15.42:0/3053246966 wait complete. 2024-05-28T14:06:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.322+0000 7fd07e000640 1 Processor -- start 2024-05-28T14:06:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.322+0000 7fd07e000640 1 -- start start 2024-05-28T14:06:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd07e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07806ee20 0x7fd07815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd07e000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd078144f80 0x7fd07815d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd07e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07815c0c0 0x7fd0780b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd07e000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd078072ea0 con 0x7fd07806ee20 2024-05-28T14:06:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd07e000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0780b4010 con 0x7fd07815c0c0 2024-05-28T14:06:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd07e000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd07813f1b0 con 0x7fd078144f80 2024-05-28T14:06:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd077400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07806ee20 0x7fd07815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd077400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07806ee20 0x7fd07815cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43032/0 (socket says 172.21.15.42:43032) 2024-05-28T14:06:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd077400640 1 -- 172.21.15.42:0/2616970171 learned_addr learned my addr 172.21.15.42:0/2616970171 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd06fe00640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd078144f80 0x7fd07815d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd077e00640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07815c0c0 0x7fd0780b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd077400640 1 -- 172.21.15.42:0/2616970171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd078144f80 msgr2=0x7fd07815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd077400640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd078144f80 0x7fd07815d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd077400640 1 -- 172.21.15.42:0/2616970171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07815c0c0 msgr2=0x7fd0780b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd077400640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07815c0c0 0x7fd0780b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd077400640 1 -- 172.21.15.42:0/2616970171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07815d930 con 0x7fd07806ee20 2024-05-28T14:06:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd077e00640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07815c0c0 0x7fd0780b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd077400640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07806ee20 0x7fd07815cd30 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd0680ad040 tx=0x7fd06801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd06f400640 1 -- 172.21.15.42:0/2616970171 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0680be5f0 con 0x7fd07806ee20 2024-05-28T14:06:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd06f400640 1 -- 172.21.15.42:0/2616970171 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0680be7b0 con 0x7fd07806ee20 2024-05-28T14:06:22.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd06f400640 1 -- 172.21.15.42:0/2616970171 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0680beba0 con 0x7fd07806ee20 2024-05-28T14:06:22.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd07e000640 1 -- 172.21.15.42:0/2616970171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07813f400 con 0x7fd07806ee20 2024-05-28T14:06:22.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd07e000640 1 -- 172.21.15.42:0/2616970171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd078154d70 con 0x7fd07806ee20 2024-05-28T14:06:22.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd07e000640 1 -- 172.21.15.42:0/2616970171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0780738b0 con 0x7fd07806ee20 2024-05-28T14:06:22.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.326+0000 7fd06f400640 1 -- 172.21.15.42:0/2616970171 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7fd0680b43f0 con 0x7fd07806ee20 2024-05-28T14:06:22.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.330+0000 7fd06f400640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd054069070 0x7fd05406b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:22.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.330+0000 7fd06f400640 1 -- 172.21.15.42:0/2616970171 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4885+0+0 (secure 0 0 0) 0x7fd068120180 con 0x7fd07806ee20 2024-05-28T14:06:22.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.330+0000 7fd06fe00640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd054069070 0x7fd05406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:22.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.330+0000 7fd06fe00640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd054069070 0x7fd05406b530 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd07815fb50 tx=0x7fd060007e90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:22.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.334+0000 7fd06f400640 1 -- 172.21.15.42:0/2616970171 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd0680c4060 con 0x7fd07806ee20 2024-05-28T14:06:22.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.498+0000 7fd07e000640 1 -- 172.21.15.42:0/2616970171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd07804fdd0 con 0x7fd07806ee20 2024-05-28T14:06:22.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.498+0000 7fd06f400640 1 -- 172.21.15.42:0/2616970171 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7fd07815d930 con 0x7fd07806ee20 2024-05-28T14:06:22.487 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:06:22.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.502+0000 7fd07e000640 1 -- 172.21.15.42:0/2616970171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd054069070 msgr2=0x7fd05406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:22.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.502+0000 7fd07e000640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd054069070 0x7fd05406b530 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd07815fb50 tx=0x7fd060007e90 comp rx=0 tx=0).stop 2024-05-28T14:06:22.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.502+0000 7fd07e000640 1 -- 172.21.15.42:0/2616970171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07806ee20 msgr2=0x7fd07815cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:22.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.502+0000 7fd07e000640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07806ee20 0x7fd07815cd30 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd0680ad040 tx=0x7fd06801ada0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.502+0000 7fd07e000640 1 -- 172.21.15.42:0/2616970171 shutdown_connections 2024-05-28T14:06:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.502+0000 7fd07e000640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd054069070 0x7fd05406b530 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.502+0000 7fd07e000640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07815c0c0 0x7fd0780b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.502+0000 7fd07e000640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd078144f80 0x7fd07815d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.502+0000 7fd07e000640 1 --2- 172.21.15.42:0/2616970171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07806ee20 0x7fd07815cd30 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.502+0000 7fd07e000640 1 -- 172.21.15.42:0/2616970171 >> 172.21.15.42:0/2616970171 conn(0x7fd0780b21f0 msgr2=0x7fd0780cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.502+0000 7fd07e000640 1 -- 172.21.15.42:0/2616970171 shutdown_connections 2024-05-28T14:06:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.502+0000 7fd07e000640 1 -- 172.21.15.42:0/2616970171 wait complete. 2024-05-28T14:06:22.508 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-28T14:06:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff4800640 1 Processor -- start 2024-05-28T14:06:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff4800640 1 -- start start 2024-05-28T14:06:22.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff4800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8fec15c990 0x7f8fec0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:22.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff4800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fec0d06a0 0x7f8fec0b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:22.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff4800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fec0cd300 con 0x7f8fec09bbd0 2024-05-28T14:06:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff4800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8fec15cd70 con 0x7f8fec15c990 2024-05-28T14:06:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff4800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8fec0cd5e0 con 0x7f8fec0d06a0 2024-05-28T14:06:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff1e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8fec15c990 0x7f8fec0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff1e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8fec15c990 0x7f8fec0d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43040/0 (socket says 172.21.15.42:43040) 2024-05-28T14:06:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff1e00640 1 -- 172.21.15.42:0/2146997951 learned_addr learned my addr 172.21.15.42:0/2146997951 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff2800640 1 --2- 172.21.15.42:0/2146997951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fec0d06a0 0x7f8fec0b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff1e00640 1 -- 172.21.15.42:0/2146997951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fec0d06a0 msgr2=0x7f8fec0b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff1e00640 1 --2- 172.21.15.42:0/2146997951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fec0d06a0 0x7f8fec0b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff1e00640 1 -- 172.21.15.42:0/2146997951 >> v1:172.21.15.63:6789/0 conn(0x7f8fec09bbd0 legacy=0x7f8fec09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.694+0000 7f8ff1e00640 1 -- 172.21.15.42:0/2146997951 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fec0cd8c0 con 0x7f8fec15c990 2024-05-28T14:06:22.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.698+0000 7f8ff1e00640 1 --2- 172.21.15.42:0/2146997951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fec15c990 0x7f8fec0d0160 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009870 tx=0x7f8fe001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c47471ee60133af server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:22.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.698+0000 7f8ff0a00640 1 -- 172.21.15.42:0/2146997951 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00a95f0 con 0x7f8fec15c990 2024-05-28T14:06:22.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.698+0000 7f8ff0a00640 1 -- 172.21.15.42:0/2146997951 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe00a9790 con 0x7f8fec15c990 2024-05-28T14:06:22.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.698+0000 7f8ff0a00640 1 -- 172.21.15.42:0/2146997951 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00a9b60 con 0x7f8fec15c990 2024-05-28T14:06:22.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.698+0000 7f8ff4800640 1 -- 172.21.15.42:0/2146997951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fec15c990 msgr2=0x7f8fec0d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:22.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.698+0000 7f8ff4800640 1 --2- 172.21.15.42:0/2146997951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fec15c990 0x7f8fec0d0160 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009870 tx=0x7f8fe001c530 comp rx=0 tx=0).stop 2024-05-28T14:06:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.698+0000 7f8ff4800640 1 -- 172.21.15.42:0/2146997951 shutdown_connections 2024-05-28T14:06:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.698+0000 7f8ff4800640 1 --2- 172.21.15.42:0/2146997951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fec0d06a0 0x7f8fec0b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.698+0000 7f8ff4800640 1 --2- 172.21.15.42:0/2146997951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fec15c990 0x7f8fec0d0160 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.698+0000 7f8ff4800640 1 -- 172.21.15.42:0/2146997951 >> 172.21.15.42:0/2146997951 conn(0x7f8fec0b4080 msgr2=0x7f8fec0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.698+0000 7f8ff4800640 1 -- 172.21.15.42:0/2146997951 shutdown_connections 2024-05-28T14:06:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.698+0000 7f8ff4800640 1 -- 172.21.15.42:0/2146997951 wait complete. 2024-05-28T14:06:22.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff4800640 1 Processor -- start 2024-05-28T14:06:22.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff4800640 1 -- start start 2024-05-28T14:06:22.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff4800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fec0d06a0 0x7f8fec158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:22.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff4800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fec15c990 0x7f8fec0c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:22.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff4800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fec0c63a0 0x7f8fec0c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:22.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff4800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec09b2e0 con 0x7f8fec0d06a0 2024-05-28T14:06:22.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff4800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec09b160 con 0x7f8fec15c990 2024-05-28T14:06:22.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff4800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8fec0b6130 con 0x7f8fec0c63a0 2024-05-28T14:06:22.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fec0d06a0 0x7f8fec158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:22.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fec0d06a0 0x7f8fec158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43044/0 (socket says 172.21.15.42:43044) 2024-05-28T14:06:22.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff1e00640 1 -- 172.21.15.42:0/2589376992 learned_addr learned my addr 172.21.15.42:0/2589376992 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:22.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff2800640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fec0c63a0 0x7f8fec0c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:22.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff1400640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fec15c990 0x7f8fec0c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:22.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff1e00640 1 -- 172.21.15.42:0/2589376992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fec0c63a0 msgr2=0x7f8fec0c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:22.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff1e00640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fec0c63a0 0x7f8fec0c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff1e00640 1 -- 172.21.15.42:0/2589376992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fec15c990 msgr2=0x7f8fec0c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:22.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff1e00640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fec15c990 0x7f8fec0c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff1e00640 1 -- 172.21.15.42:0/2589376992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fec0b6c30 con 0x7f8fec0d06a0 2024-05-28T14:06:22.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff2800640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fec0c63a0 0x7f8fec0c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:22.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff1e00640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fec0d06a0 0x7f8fec158ba0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8fe001cdd0 tx=0x7f8fe00192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:22.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8fe6000640 1 -- 172.21.15.42:0/2589376992 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00a95f0 con 0x7f8fec0d06a0 2024-05-28T14:06:22.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8fe6000640 1 -- 172.21.15.42:0/2589376992 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe00b1800 con 0x7f8fec0d06a0 2024-05-28T14:06:22.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff4800640 1 -- 172.21.15.42:0/2589376992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fec0c6e00 con 0x7f8fec0d06a0 2024-05-28T14:06:22.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8fe6000640 1 -- 172.21.15.42:0/2589376992 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00b1bd0 con 0x7f8fec0d06a0 2024-05-28T14:06:22.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff4800640 1 -- 172.21.15.42:0/2589376992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fec152360 con 0x7f8fec0d06a0 2024-05-28T14:06:22.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.702+0000 7f8ff4800640 1 -- 172.21.15.42:0/2589376992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fec068a90 con 0x7f8fec0d06a0 2024-05-28T14:06:22.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.706+0000 7f8fe6000640 1 -- 172.21.15.42:0/2589376992 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f8fe00bc450 con 0x7f8fec0d06a0 2024-05-28T14:06:22.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.706+0000 7f8fe6000640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fd0069070 0x7f8fd006b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:22.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.706+0000 7f8fe6000640 1 -- 172.21.15.42:0/2589376992 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4885+0+0 (secure 0 0 0) 0x7f8fe0120020 con 0x7f8fec0d06a0 2024-05-28T14:06:22.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.706+0000 7f8ff1400640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fd0069070 0x7f8fd006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:22.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.706+0000 7f8ff1400640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fd0069070 0x7f8fd006b530 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0cce00 tx=0x7f8fdc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:22.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.710+0000 7f8fe6000640 1 -- 172.21.15.42:0/2589376992 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8fe00ad020 con 0x7f8fec0d06a0 2024-05-28T14:06:22.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.878+0000 7f8ff4800640 1 -- 172.21.15.42:0/2589376992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8fec0b4ce0 con 0x7f8fec0d06a0 2024-05-28T14:06:22.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.878+0000 7f8fe6000640 1 -- 172.21.15.42:0/2589376992 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7f8fe0016150 con 0x7f8fec0d06a0 2024-05-28T14:06:22.864 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:22.864 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:06:22.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:22.864 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-05-28T14:06:22.864 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-05-28T14:06:22.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:22.864 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:06:22.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.878+0000 7f8ff4800640 1 -- 172.21.15.42:0/2589376992 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fd0069070 msgr2=0x7f8fd006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:22.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.878+0000 7f8ff4800640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fd0069070 0x7f8fd006b530 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0cce00 tx=0x7f8fdc009040 comp rx=0 tx=0).stop 2024-05-28T14:06:22.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.878+0000 7f8ff4800640 1 -- 172.21.15.42:0/2589376992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fec0d06a0 msgr2=0x7f8fec158ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:22.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.878+0000 7f8ff4800640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fec0d06a0 0x7f8fec158ba0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8fe001cdd0 tx=0x7f8fe00192d0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.878+0000 7f8ff4800640 1 -- 172.21.15.42:0/2589376992 shutdown_connections 2024-05-28T14:06:22.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.878+0000 7f8ff4800640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8fd0069070 0x7f8fd006b530 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.878+0000 7f8ff4800640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fec0c63a0 0x7f8fec0c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.878+0000 7f8ff4800640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fec15c990 0x7f8fec0c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.878+0000 7f8ff4800640 1 --2- 172.21.15.42:0/2589376992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fec0d06a0 0x7f8fec158ba0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:22.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.878+0000 7f8ff4800640 1 -- 172.21.15.42:0/2589376992 >> 172.21.15.42:0/2589376992 conn(0x7f8fec0b4080 msgr2=0x7f8fec0b6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:22.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.878+0000 7f8ff4800640 1 -- 172.21.15.42:0/2589376992 shutdown_connections 2024-05-28T14:06:22.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:22.878+0000 7f8ff4800640 1 -- 172.21.15.42:0/2589376992 wait complete. 2024-05-28T14:06:22.885 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-28T14:06:22.885 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-28T14:06:23.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19aa600640 1 Processor -- start 2024-05-28T14:06:23.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19aa600640 1 -- start start 2024-05-28T14:06:23.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19aa600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f19a409b160 0x7f19a40ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:23.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19aa600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19a40b3e00 con 0x7f19a415ec50 2024-05-28T14:06:23.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19aa600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19a40b4ee0 con 0x7f19a415f030 2024-05-28T14:06:23.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19aa600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f19a409b810 con 0x7f19a409b160 2024-05-28T14:06:23.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19a3400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f19a415ec50 0x7f19a406e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42610/0 (socket says 172.21.15.42:42610) 2024-05-28T14:06:23.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19a3400640 1 -- 172.21.15.42:0/653082491 learned_addr learned my addr 172.21.15.42:0/653082491 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:23.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19a3e00640 1 --2- 172.21.15.42:0/653082491 >> v2:172.21.15.63:3300/0 conn(0x7f19a409b160 0x7f19a40ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:23.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19a2a00640 1 -- 172.21.15.42:0/653082491 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 333697424 0 0) 0x7f19a40b3e00 con 0x7f19a415ec50 2024-05-28T14:06:23.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19a2a00640 1 -- 172.21.15.42:0/653082491 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1988003530 con 0x7f19a415ec50 2024-05-28T14:06:23.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19a3e00640 1 -- 172.21.15.42:0/653082491 >> v1:172.21.15.63:6789/0 conn(0x7f19a415f030 legacy=0x7f19a409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:23.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19a3e00640 1 -- 172.21.15.42:0/653082491 >> v1:172.21.15.42:6789/0 conn(0x7f19a415ec50 legacy=0x7f19a406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:23.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19a2a00640 1 -- 172.21.15.42:0/653082491 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 802683775 0 0) 0x7f1988003530 con 0x7f19a415ec50 2024-05-28T14:06:23.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.046+0000 7f19a3e00640 1 -- 172.21.15.42:0/653082491 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a40b2fe0 con 0x7f19a409b160 2024-05-28T14:06:23.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.050+0000 7f19a3e00640 1 --2- 172.21.15.42:0/653082491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a409b160 0x7f19a40ccb30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f199c009d90 tx=0x7f199c01c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8245cfbae17ccc94 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:23.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.050+0000 7f19a2a00640 1 -- 172.21.15.42:0/653082491 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c0b2070 con 0x7f19a409b160 2024-05-28T14:06:23.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.050+0000 7f19a2a00640 1 -- 172.21.15.42:0/653082491 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f199c0a56a0 con 0x7f19a409b160 2024-05-28T14:06:23.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.050+0000 7f19a2a00640 1 -- 172.21.15.42:0/653082491 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994001680 con 0x7f19a409b160 2024-05-28T14:06:23.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.050+0000 7f19aa600640 1 -- 172.21.15.42:0/653082491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a409b160 msgr2=0x7f19a40ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:23.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.050+0000 7f19aa600640 1 --2- 172.21.15.42:0/653082491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a409b160 0x7f19a40ccb30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f199c009d90 tx=0x7f199c01c860 comp rx=0 tx=0).stop 2024-05-28T14:06:23.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.050+0000 7f19aa600640 1 -- 172.21.15.42:0/653082491 shutdown_connections 2024-05-28T14:06:23.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.050+0000 7f19aa600640 1 --2- 172.21.15.42:0/653082491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a409b160 0x7f19a40ccb30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.050+0000 7f19aa600640 1 -- 172.21.15.42:0/653082491 >> 172.21.15.42:0/653082491 conn(0x7f19a40b3fd0 msgr2=0x7f19a40b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:23.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.050+0000 7f19aa600640 1 -- 172.21.15.42:0/653082491 shutdown_connections 2024-05-28T14:06:23.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.050+0000 7f19aa600640 1 -- 172.21.15.42:0/653082491 wait complete. 2024-05-28T14:06:23.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.050+0000 7f19aa600640 1 Processor -- start 2024-05-28T14:06:23.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19aa600640 1 -- start start 2024-05-28T14:06:23.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19aa600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a409b160 0x7f19a41623b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:23.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19aa600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19a415ec50 0x7f19a40b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:23.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19aa600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a415f030 0x7f19a4158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:23.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19aa600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f19a40cd1f0 con 0x7f19a409b160 2024-05-28T14:06:23.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19aa600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19a40cd070 con 0x7f19a415f030 2024-05-28T14:06:23.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19aa600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f19a40cd370 con 0x7f19a415ec50 2024-05-28T14:06:23.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19a3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a409b160 0x7f19a41623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:23.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19a3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a409b160 0x7f19a41623b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43048/0 (socket says 172.21.15.42:43048) 2024-05-28T14:06:23.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19a3400640 1 -- 172.21.15.42:0/3265848563 learned_addr learned my addr 172.21.15.42:0/3265848563 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:23.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19a3e00640 1 --2- 172.21.15.42:0/3265848563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a415f030 0x7f19a4158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:23.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f199be00640 1 --2- 172.21.15.42:0/3265848563 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19a415ec50 0x7f19a40b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:23.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19a3400640 1 -- 172.21.15.42:0/3265848563 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19a415ec50 msgr2=0x7f19a40b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:23.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19a3400640 1 --2- 172.21.15.42:0/3265848563 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19a415ec50 0x7f19a40b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19a3400640 1 -- 172.21.15.42:0/3265848563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a415f030 msgr2=0x7f19a4158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:23.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19a3400640 1 --2- 172.21.15.42:0/3265848563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a415f030 0x7f19a4158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19a3400640 1 -- 172.21.15.42:0/3265848563 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a40b7820 con 0x7f19a409b160 2024-05-28T14:06:23.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19a3400640 1 --2- 172.21.15.42:0/3265848563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a409b160 0x7f19a41623b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f199400c730 tx=0x7f199400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:23.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f199b400640 1 -- 172.21.15.42:0/3265848563 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994017070 con 0x7f19a409b160 2024-05-28T14:06:23.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f199b400640 1 -- 172.21.15.42:0/3265848563 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1994005b00 con 0x7f19a409b160 2024-05-28T14:06:23.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f199b400640 1 -- 172.21.15.42:0/3265848563 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994007070 con 0x7f19a409b160 2024-05-28T14:06:23.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19aa600640 1 -- 172.21.15.42:0/3265848563 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a40b7ab0 con 0x7f19a409b160 2024-05-28T14:06:23.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19aa600640 1 -- 172.21.15.42:0/3265848563 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a40af480 con 0x7f19a409b160 2024-05-28T14:06:23.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.054+0000 7f19aa600640 1 -- 172.21.15.42:0/3265848563 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a4068a90 con 0x7f19a409b160 2024-05-28T14:06:23.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.058+0000 7f199b400640 1 -- 172.21.15.42:0/3265848563 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f199401a070 con 0x7f19a409b160 2024-05-28T14:06:23.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.058+0000 7f199b400640 1 --2- 172.21.15.42:0/3265848563 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1980069070 0x7f198006b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:23.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.058+0000 7f199b400640 1 -- 172.21.15.42:0/3265848563 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4885+0+0 (secure 0 0 0) 0x7f1994084890 con 0x7f19a409b160 2024-05-28T14:06:23.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.058+0000 7f199be00640 1 --2- 172.21.15.42:0/3265848563 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1980069070 0x7f198006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:23.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.058+0000 7f199be00640 1 --2- 172.21.15.42:0/3265848563 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1980069070 0x7f198006b530 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f19a4074b10 tx=0x7f198c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:23.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.062+0000 7f199b400640 1 -- 172.21.15.42:0/3265848563 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1994056250 con 0x7f19a409b160 2024-05-28T14:06:23.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.226+0000 7f19aa600640 1 -- 172.21.15.42:0/3265848563 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f19a40c8ef0 con 0x7f19a409b160 2024-05-28T14:06:23.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.394+0000 7f199b400640 1 -- 172.21.15.42:0/3265848563 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7f199405a550 con 0x7f19a409b160 2024-05-28T14:06:23.381 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-05-28T14:06:23.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.394+0000 7f19aa600640 1 -- 172.21.15.42:0/3265848563 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1980069070 msgr2=0x7f198006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:23.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.394+0000 7f19aa600640 1 --2- 172.21.15.42:0/3265848563 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1980069070 0x7f198006b530 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f19a4074b10 tx=0x7f198c009040 comp rx=0 tx=0).stop 2024-05-28T14:06:23.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.394+0000 7f19aa600640 1 -- 172.21.15.42:0/3265848563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a409b160 msgr2=0x7f19a41623b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:23.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.394+0000 7f19aa600640 1 --2- 172.21.15.42:0/3265848563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a409b160 0x7f19a41623b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f199400c730 tx=0x7f199400cbf0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.394+0000 7f19aa600640 1 -- 172.21.15.42:0/3265848563 shutdown_connections 2024-05-28T14:06:23.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.394+0000 7f19aa600640 1 --2- 172.21.15.42:0/3265848563 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1980069070 0x7f198006b530 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.394+0000 7f19aa600640 1 --2- 172.21.15.42:0/3265848563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a415f030 0x7f19a4158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.394+0000 7f19aa600640 1 --2- 172.21.15.42:0/3265848563 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19a415ec50 0x7f19a40b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.394+0000 7f19aa600640 1 --2- 172.21.15.42:0/3265848563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19a409b160 0x7f19a41623b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.394+0000 7f19aa600640 1 -- 172.21.15.42:0/3265848563 >> 172.21.15.42:0/3265848563 conn(0x7f19a40b3fd0 msgr2=0x7f19a409aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:23.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.398+0000 7f19aa600640 1 -- 172.21.15.42:0/3265848563 shutdown_connections 2024-05-28T14:06:23.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.398+0000 7f19aa600640 1 -- 172.21.15.42:0/3265848563 wait complete. 2024-05-28T14:06:23.408 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-28T14:06:23.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a18000640 1 Processor -- start 2024-05-28T14:06:23.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a18000640 1 -- start start 2024-05-28T14:06:23.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a18000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1a1015cb50 0x7f1a1009b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:23.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a18000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a1009ba40 con 0x7f1a1015c770 2024-05-28T14:06:23.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a18000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a100c60d0 con 0x7f1a100d11d0 2024-05-28T14:06:23.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a18000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1a100c6720 con 0x7f1a1015cb50 2024-05-28T14:06:23.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a15600640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1a1015c770 0x7f1a100d0ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42612/0 (socket says 172.21.15.42:42612) 2024-05-28T14:06:23.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a15600640 1 -- 172.21.15.42:0/1803827810 learned_addr learned my addr 172.21.15.42:0/1803827810 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:23.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a14c00640 1 --2- 172.21.15.42:0/1803827810 >> v2:172.21.15.63:3300/0 conn(0x7f1a1015cb50 0x7f1a1009b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:23.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 -- 172.21.15.42:0/1803827810 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3916849791 0 0) 0x7f1a1009ba40 con 0x7f1a1015c770 2024-05-28T14:06:23.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 -- 172.21.15.42:0/1803827810 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19f8003530 con 0x7f1a1015c770 2024-05-28T14:06:23.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 -- 172.21.15.42:0/1803827810 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2133771815 0 0) 0x7f19f8003530 con 0x7f1a1015c770 2024-05-28T14:06:23.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 -- 172.21.15.42:0/1803827810 >> v2:172.21.15.63:3300/0 conn(0x7f1a1015cb50 msgr2=0x7f1a1009b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:23.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 --2- 172.21.15.42:0/1803827810 >> v2:172.21.15.63:3300/0 conn(0x7f1a1015cb50 0x7f1a1009b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 -- 172.21.15.42:0/1803827810 >> v1:172.21.15.63:6789/0 conn(0x7f1a100d11d0 legacy=0x7f1a10166470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:23.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 -- 172.21.15.42:0/1803827810 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a100c63b0 con 0x7f1a1015c770 2024-05-28T14:06:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 -- 172.21.15.42:0/1803827810 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f1a04001680 con 0x7f1a1015c770 2024-05-28T14:06:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 -- 172.21.15.42:0/1803827810 >> v1:172.21.15.42:6789/0 conn(0x7f1a1015c770 legacy=0x7f1a100d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 --2- 172.21.15.42:0/1803827810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f80039c0 0x7f19f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 --2- 172.21.15.42:0/1803827810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f8004810 0x7f19f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 --2- 172.21.15.42:0/1803827810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19f80051d0 0x7f19f800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 -- 172.21.15.42:0/1803827810 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1a100c6720 con 0x7f19f8004810 2024-05-28T14:06:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 -- 172.21.15.42:0/1803827810 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19f800c080 con 0x7f19f80039c0 2024-05-28T14:06:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.570+0000 7f1a0fe00640 1 -- 172.21.15.42:0/1803827810 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f19f800c280 con 0x7f19f80051d0 2024-05-28T14:06:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.574+0000 7f1a14c00640 1 --2- 172.21.15.42:0/1803827810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f8004810 0x7f19f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.574+0000 7f1a15600640 1 --2- 172.21.15.42:0/1803827810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f80039c0 0x7f19f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.574+0000 7f1a16000640 1 --2- 172.21.15.42:0/1803827810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19f80051d0 0x7f19f800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.574+0000 7f1a15600640 1 -- 172.21.15.42:0/1803827810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19f80051d0 msgr2=0x7f19f800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.574+0000 7f1a15600640 1 --2- 172.21.15.42:0/1803827810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19f80051d0 0x7f19f800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.574+0000 7f1a15600640 1 -- 172.21.15.42:0/1803827810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f8004810 msgr2=0x7f19f8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:23.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.574+0000 7f1a15600640 1 --2- 172.21.15.42:0/1803827810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f8004810 0x7f19f8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.574+0000 7f1a15600640 1 -- 172.21.15.42:0/1803827810 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f8003530 con 0x7f19f80039c0 2024-05-28T14:06:23.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.574+0000 7f1a14c00640 1 --2- 172.21.15.42:0/1803827810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f8004810 0x7f19f8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:06:23.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.574+0000 7f1a16000640 1 -- 172.21.15.42:0/1803827810 reap_dead start 2024-05-28T14:06:23.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.574+0000 7f1a15600640 1 --2- 172.21.15.42:0/1803827810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f80039c0 0x7f19f8003de0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400c210 tx=0x7f1a0401cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=135b68b35f0073e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:23.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.574+0000 7f1a0fe00640 1 -- 172.21.15.42:0/1803827810 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04007d40 con 0x7f19f80039c0 2024-05-28T14:06:23.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.574+0000 7f1a0fe00640 1 -- 172.21.15.42:0/1803827810 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a040a9070 con 0x7f19f80039c0 2024-05-28T14:06:23.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.574+0000 7f1a0fe00640 1 -- 172.21.15.42:0/1803827810 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a040a5e40 con 0x7f19f80039c0 2024-05-28T14:06:23.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 -- 172.21.15.42:0/1803827810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f80039c0 msgr2=0x7f19f8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:23.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 --2- 172.21.15.42:0/1803827810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f80039c0 0x7f19f8003de0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400c210 tx=0x7f1a0401cca0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 -- 172.21.15.42:0/1803827810 shutdown_connections 2024-05-28T14:06:23.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 --2- 172.21.15.42:0/1803827810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f80039c0 0x7f19f8003de0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 -- 172.21.15.42:0/1803827810 >> 172.21.15.42:0/1803827810 conn(0x7f1a100b3fd0 msgr2=0x7f1a100b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:23.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 -- 172.21.15.42:0/1803827810 shutdown_connections 2024-05-28T14:06:23.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 -- 172.21.15.42:0/1803827810 wait complete. 2024-05-28T14:06:23.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 Processor -- start 2024-05-28T14:06:23.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 -- start start 2024-05-28T14:06:23.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f80039c0 0x7f1a100cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a101f8b90 0x7f1a1015ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a100b72b0 0x7f1a100b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1a100d02c0 con 0x7f19f80039c0 2024-05-28T14:06:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1a100ce2c0 con 0x7f1a100b72b0 2024-05-28T14:06:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a18000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1a100d0440 con 0x7f1a101f8b90 2024-05-28T14:06:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a15600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f80039c0 0x7f1a100cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a15600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f80039c0 0x7f1a100cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43074/0 (socket says 172.21.15.42:43074) 2024-05-28T14:06:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a15600640 1 -- 172.21.15.42:0/2775827088 learned_addr learned my addr 172.21.15.42:0/2775827088 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a16000640 1 --2- 172.21.15.42:0/2775827088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a100b72b0 0x7f1a100b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a14c00640 1 --2- 172.21.15.42:0/2775827088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a101f8b90 0x7f1a1015ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a15600640 1 -- 172.21.15.42:0/2775827088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a101f8b90 msgr2=0x7f1a1015ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a15600640 1 --2- 172.21.15.42:0/2775827088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a101f8b90 0x7f1a1015ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a15600640 1 -- 172.21.15.42:0/2775827088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a100b72b0 msgr2=0x7f1a100b7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:23.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a15600640 1 --2- 172.21.15.42:0/2775827088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a100b72b0 0x7f1a100b7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:23.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.578+0000 7f1a15600640 1 -- 172.21.15.42:0/2775827088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a101f4ce0 con 0x7f19f80039c0 2024-05-28T14:06:23.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.582+0000 7f1a15600640 1 --2- 172.21.15.42:0/2775827088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f80039c0 0x7f1a100cb130 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1a0401c710 tx=0x7f19f8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:23.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.582+0000 7f1a0d600640 1 -- 172.21.15.42:0/2775827088 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04007d20 con 0x7f19f80039c0 2024-05-28T14:06:23.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.582+0000 7f1a0d600640 1 -- 172.21.15.42:0/2775827088 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a040a5bb0 con 0x7f19f80039c0 2024-05-28T14:06:23.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.582+0000 7f1a0d600640 1 -- 172.21.15.42:0/2775827088 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04007140 con 0x7f19f80039c0 2024-05-28T14:06:23.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.582+0000 7f1a18000640 1 -- 172.21.15.42:0/2775827088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a101f4fc0 con 0x7f19f80039c0 2024-05-28T14:06:23.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.582+0000 7f1a18000640 1 -- 172.21.15.42:0/2775827088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a101f5580 con 0x7f19f80039c0 2024-05-28T14:06:23.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.582+0000 7f1a18000640 1 -- 172.21.15.42:0/2775827088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a100b7c70 con 0x7f19f80039c0 2024-05-28T14:06:23.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.582+0000 7f1a0d600640 1 -- 172.21.15.42:0/2775827088 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f1a040072e0 con 0x7f19f80039c0 2024-05-28T14:06:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.582+0000 7f1a0d600640 1 --2- 172.21.15.42:0/2775827088 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19f0069070 0x7f19f006b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.586+0000 7f1a0d600640 1 -- 172.21.15.42:0/2775827088 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5255+0+0 (secure 0 0 0) 0x7f1a0411ef00 con 0x7f19f80039c0 2024-05-28T14:06:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.586+0000 7f1a14c00640 1 --2- 172.21.15.42:0/2775827088 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19f0069070 0x7f19f006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.586+0000 7f1a14c00640 1 --2- 172.21.15.42:0/2775827088 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19f0069070 0x7f19f006b530 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1a10160080 tx=0x7f19fc009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:23.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.590+0000 7f1a0d600640 1 -- 172.21.15.42:0/2775827088 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1a040169b0 con 0x7f19f80039c0 2024-05-28T14:06:23.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:23.754+0000 7f1a18000640 1 -- 172.21.15.42:0/2775827088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1a100a04d0 con 0x7f19f80039c0 2024-05-28T14:06:24.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.402+0000 7f1a0d600640 1 -- 172.21.15.42:0/2775827088 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) ==== 170+0+0 (secure 0 0 0) 0x7f1a040f0f70 con 0x7f19f80039c0 2024-05-28T14:06:24.388 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-05-28T14:06:24.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.402+0000 7f1a18000640 1 -- 172.21.15.42:0/2775827088 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19f0069070 msgr2=0x7f19f006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:24.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.402+0000 7f1a18000640 1 --2- 172.21.15.42:0/2775827088 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19f0069070 0x7f19f006b530 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1a10160080 tx=0x7f19fc009190 comp rx=0 tx=0).stop 2024-05-28T14:06:24.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.402+0000 7f1a18000640 1 -- 172.21.15.42:0/2775827088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f80039c0 msgr2=0x7f1a100cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:24.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.402+0000 7f1a18000640 1 --2- 172.21.15.42:0/2775827088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f80039c0 0x7f1a100cb130 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1a0401c710 tx=0x7f19f8005840 comp rx=0 tx=0).stop 2024-05-28T14:06:24.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.406+0000 7f1a18000640 1 -- 172.21.15.42:0/2775827088 shutdown_connections 2024-05-28T14:06:24.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.406+0000 7f1a18000640 1 --2- 172.21.15.42:0/2775827088 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19f0069070 0x7f19f006b530 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:24.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.406+0000 7f1a18000640 1 --2- 172.21.15.42:0/2775827088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a100b72b0 0x7f1a100b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:24.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.406+0000 7f1a18000640 1 --2- 172.21.15.42:0/2775827088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a101f8b90 0x7f1a1015ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:24.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.406+0000 7f1a18000640 1 --2- 172.21.15.42:0/2775827088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f80039c0 0x7f1a100cb130 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:24.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.406+0000 7f1a18000640 1 -- 172.21.15.42:0/2775827088 >> 172.21.15.42:0/2775827088 conn(0x7f1a100b3fd0 msgr2=0x7f1a100cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:24.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.406+0000 7f1a18000640 1 -- 172.21.15.42:0/2775827088 shutdown_connections 2024-05-28T14:06:24.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.406+0000 7f1a18000640 1 -- 172.21.15.42:0/2775827088 wait complete. 2024-05-28T14:06:24.432 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-28T14:06:24.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.590+0000 7f8511a00640 1 Processor -- start 2024-05-28T14:06:24.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.590+0000 7f8511a00640 1 -- start start 2024-05-28T14:06:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f8511a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f850c0b0e80 0x7f850c06cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f8511a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f850c09b160 0x7f850c15e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f8511a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f850c0d0770 con 0x7f850c09b8e0 2024-05-28T14:06:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f8511a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f850c0d0a10 con 0x7f850c09b160 2024-05-28T14:06:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f8511a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f850c0d0cf0 con 0x7f850c0b0e80 2024-05-28T14:06:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f850b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f850c09b160 0x7f850c15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f850b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f850c09b160 0x7f850c15e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43086/0 (socket says 172.21.15.42:43086) 2024-05-28T14:06:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f850b400640 1 -- 172.21.15.42:0/3159575341 learned_addr learned my addr 172.21.15.42:0/3159575341 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f8502000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f850c09b8e0 0x7f850c09aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42620/0 (socket says 172.21.15.42:42620) 2024-05-28T14:06:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f850aa00640 1 --2- 172.21.15.42:0/3159575341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f850c0b0e80 0x7f850c06cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f850b400640 1 -- 172.21.15.42:0/3159575341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f850c0b0e80 msgr2=0x7f850c06cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f850b400640 1 --2- 172.21.15.42:0/3159575341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f850c0b0e80 0x7f850c06cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f850b400640 1 -- 172.21.15.42:0/3159575341 >> v1:172.21.15.42:6789/0 conn(0x7f850c09b8e0 legacy=0x7f850c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f850b400640 1 -- 172.21.15.42:0/3159575341 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c0d0fd0 con 0x7f850c09b160 2024-05-28T14:06:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.594+0000 7f850a000640 1 -- 172.21.15.42:0/3159575341 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 148855464 0 0) 0x7f850c0d0770 con 0x7f850c09b8e0 2024-05-28T14:06:24.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.598+0000 7f850b400640 1 --2- 172.21.15.42:0/3159575341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f850c09b160 0x7f850c15e110 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8504009e50 tx=0x7f850401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abaa2e7db507e31 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:24.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.598+0000 7f850a000640 1 -- 172.21.15.42:0/3159575341 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85040a5700 con 0x7f850c09b160 2024-05-28T14:06:24.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.598+0000 7f850a000640 1 -- 172.21.15.42:0/3159575341 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85040a58a0 con 0x7f850c09b160 2024-05-28T14:06:24.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.598+0000 7f850a000640 1 -- 172.21.15.42:0/3159575341 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85040a5c70 con 0x7f850c09b160 2024-05-28T14:06:24.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.598+0000 7f8511a00640 1 -- 172.21.15.42:0/3159575341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f850c09b160 msgr2=0x7f850c15e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:24.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.598+0000 7f8511a00640 1 --2- 172.21.15.42:0/3159575341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f850c09b160 0x7f850c15e110 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8504009e50 tx=0x7f850401ca30 comp rx=0 tx=0).stop 2024-05-28T14:06:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.598+0000 7f8511a00640 1 -- 172.21.15.42:0/3159575341 shutdown_connections 2024-05-28T14:06:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.598+0000 7f8511a00640 1 --2- 172.21.15.42:0/3159575341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f850c09b160 0x7f850c15e110 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.598+0000 7f8511a00640 1 --2- 172.21.15.42:0/3159575341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f850c0b0e80 0x7f850c06cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.598+0000 7f8511a00640 1 -- 172.21.15.42:0/3159575341 >> 172.21.15.42:0/3159575341 conn(0x7f850c0b3f90 msgr2=0x7f850c0b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.598+0000 7f8511a00640 1 -- 172.21.15.42:0/3159575341 shutdown_connections 2024-05-28T14:06:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.598+0000 7f8511a00640 1 -- 172.21.15.42:0/3159575341 wait complete. 2024-05-28T14:06:24.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.598+0000 7f8511a00640 1 Processor -- start 2024-05-28T14:06:24.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.598+0000 7f8511a00640 1 -- start start 2024-05-28T14:06:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f8511a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f850c09b160 0x7f850c15d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f8511a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f850c09b8e0 0x7f850c153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f8511a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f850c0b0e80 0x7f850c158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f8511a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f850c0af350 con 0x7f850c09b160 2024-05-28T14:06:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f8511a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f850c0af1d0 con 0x7f850c09b8e0 2024-05-28T14:06:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f8511a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f850c0af4d0 con 0x7f850c0b0e80 2024-05-28T14:06:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f850aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f850c09b160 0x7f850c15d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f850aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f850c09b160 0x7f850c15d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43094/0 (socket says 172.21.15.42:43094) 2024-05-28T14:06:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f850aa00640 1 -- 172.21.15.42:0/2628795803 learned_addr learned my addr 172.21.15.42:0/2628795803 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f8502000640 1 --2- 172.21.15.42:0/2628795803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f850c09b8e0 0x7f850c153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f850b400640 1 --2- 172.21.15.42:0/2628795803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f850c0b0e80 0x7f850c158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:24.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f850aa00640 1 -- 172.21.15.42:0/2628795803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f850c0b0e80 msgr2=0x7f850c158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:24.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f850aa00640 1 --2- 172.21.15.42:0/2628795803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f850c0b0e80 0x7f850c158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:24.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f850aa00640 1 -- 172.21.15.42:0/2628795803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f850c09b8e0 msgr2=0x7f850c153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:24.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f850aa00640 1 --2- 172.21.15.42:0/2628795803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f850c09b8e0 0x7f850c153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:24.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f850aa00640 1 -- 172.21.15.42:0/2628795803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c15c630 con 0x7f850c09b160 2024-05-28T14:06:24.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f850aa00640 1 --2- 172.21.15.42:0/2628795803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f850c09b160 0x7f850c15d990 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00e870 tx=0x7f84fc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:24.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f8503400640 1 -- 172.21.15.42:0/2628795803 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc017070 con 0x7f850c09b160 2024-05-28T14:06:24.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f8503400640 1 -- 172.21.15.42:0/2628795803 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84fc00c940 con 0x7f850c09b160 2024-05-28T14:06:24.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f8503400640 1 -- 172.21.15.42:0/2628795803 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc00ccf0 con 0x7f850c09b160 2024-05-28T14:06:24.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f8511a00640 1 -- 172.21.15.42:0/2628795803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850c15c950 con 0x7f850c09b160 2024-05-28T14:06:24.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f8511a00640 1 -- 172.21.15.42:0/2628795803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850c15d190 con 0x7f850c09b160 2024-05-28T14:06:24.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.602+0000 7f8511a00640 1 -- 172.21.15.42:0/2628795803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f850c0a0550 con 0x7f850c09b160 2024-05-28T14:06:24.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.606+0000 7f8503400640 1 -- 172.21.15.42:0/2628795803 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f84fc01a070 con 0x7f850c09b160 2024-05-28T14:06:24.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.606+0000 7f8503400640 1 --2- 172.21.15.42:0/2628795803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f84e8069070 0x7f84e806b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:24.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.606+0000 7f8503400640 1 -- 172.21.15.42:0/2628795803 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5637+0+0 (secure 0 0 0) 0x7f84fc085bd0 con 0x7f850c09b160 2024-05-28T14:06:24.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.606+0000 7f8502000640 1 --2- 172.21.15.42:0/2628795803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f84e8069070 0x7f84e806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:24.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.606+0000 7f8502000640 1 --2- 172.21.15.42:0/2628795803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f84e8069070 0x7f84e806b530 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f850c0b8510 tx=0x7f84f4007b60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:24.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.610+0000 7f8503400640 1 -- 172.21.15.42:0/2628795803 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f850c15c950 con 0x7f850c09b160 2024-05-28T14:06:24.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:24.810+0000 7f8511a00640 1 -- 172.21.15.42:0/2628795803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f850c04fdd0 con 0x7f850c09b160 2024-05-28T14:06:25.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.418+0000 7f8503400640 1 -- 172.21.15.42:0/2628795803 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v42) ==== 174+0+0 (secure 0 0 0) 0x7f84fc057b50 con 0x7f850c09b160 2024-05-28T14:06:25.410 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 10 and data pool 11 2024-05-28T14:06:25.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.434+0000 7f8511a00640 1 -- 172.21.15.42:0/2628795803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f84e8069070 msgr2=0x7f84e806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:25.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.434+0000 7f8511a00640 1 --2- 172.21.15.42:0/2628795803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f84e8069070 0x7f84e806b530 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f850c0b8510 tx=0x7f84f4007b60 comp rx=0 tx=0).stop 2024-05-28T14:06:25.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.434+0000 7f8511a00640 1 -- 172.21.15.42:0/2628795803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f850c09b160 msgr2=0x7f850c15d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:25.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.434+0000 7f8511a00640 1 --2- 172.21.15.42:0/2628795803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f850c09b160 0x7f850c15d990 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00e870 tx=0x7f84fc00ec70 comp rx=0 tx=0).stop 2024-05-28T14:06:25.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.434+0000 7f8511a00640 1 -- 172.21.15.42:0/2628795803 shutdown_connections 2024-05-28T14:06:25.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.434+0000 7f8511a00640 1 --2- 172.21.15.42:0/2628795803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f84e8069070 0x7f84e806b530 secure :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f850c0b8510 tx=0x7f84f4007b60 comp rx=0 tx=0).stop 2024-05-28T14:06:25.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.434+0000 7f8511a00640 1 --2- 172.21.15.42:0/2628795803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f850c0b0e80 0x7f850c158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.434+0000 7f8511a00640 1 --2- 172.21.15.42:0/2628795803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f850c09b8e0 0x7f850c153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.434+0000 7f8511a00640 1 --2- 172.21.15.42:0/2628795803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f850c09b160 0x7f850c15d990 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.434+0000 7f8511a00640 1 -- 172.21.15.42:0/2628795803 >> 172.21.15.42:0/2628795803 conn(0x7f850c0b3f90 msgr2=0x7f850c156670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:25.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.438+0000 7f8511a00640 1 -- 172.21.15.42:0/2628795803 shutdown_connections 2024-05-28T14:06:25.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.438+0000 7f8511a00640 1 -- 172.21.15.42:0/2628795803 wait complete. 2024-05-28T14:06:25.454 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:06:25.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fd1600640 1 Processor -- start 2024-05-28T14:06:25.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fd1600640 1 -- start start 2024-05-28T14:06:25.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fd1600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9fcc06f660 0x7f9fcc073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:25.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fd1600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9fcc0d0e00 0x7f9fcc06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:25.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fd1600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fcc15f280 con 0x7f9fcc15ba40 2024-05-28T14:06:25.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fd1600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9fcc15f030 con 0x7f9fcc06f660 2024-05-28T14:06:25.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fd1600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9fcc15f560 con 0x7f9fcc0d0e00 2024-05-28T14:06:25.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fcaa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9fcc06f660 0x7f9fcc073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:25.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fcaa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9fcc06f660 0x7f9fcc073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43104/0 (socket says 172.21.15.42:43104) 2024-05-28T14:06:25.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fcaa00640 1 -- 172.21.15.42:0/2922885702 learned_addr learned my addr 172.21.15.42:0/2922885702 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:25.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fcb400640 1 --2- 172.21.15.42:0/2922885702 >> v2:172.21.15.63:3300/0 conn(0x7f9fcc0d0e00 0x7f9fcc06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:25.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fcaa00640 1 -- 172.21.15.42:0/2922885702 >> v2:172.21.15.63:3300/0 conn(0x7f9fcc0d0e00 msgr2=0x7f9fcc06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:25.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fcaa00640 1 --2- 172.21.15.42:0/2922885702 >> v2:172.21.15.63:3300/0 conn(0x7f9fcc0d0e00 0x7f9fcc06dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fcaa00640 1 -- 172.21.15.42:0/2922885702 >> v1:172.21.15.63:6789/0 conn(0x7f9fcc15ba40 legacy=0x7f9fcc0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:25.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fcaa00640 1 -- 172.21.15.42:0/2922885702 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc15f840 con 0x7f9fcc06f660 2024-05-28T14:06:25.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.614+0000 7f9fcb400640 1 --2- 172.21.15.42:0/2922885702 >> v2:172.21.15.63:3300/0 conn(0x7f9fcc0d0e00 0x7f9fcc06dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:25.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.618+0000 7f9fcaa00640 1 --2- 172.21.15.42:0/2922885702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fcc06f660 0x7f9fcc073980 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc15cb20 tx=0x7f9fbc01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5996523c518a1d7b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:25.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.618+0000 7f9fca000640 1 -- 172.21.15.42:0/2922885702 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0b0070 con 0x7f9fcc06f660 2024-05-28T14:06:25.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.618+0000 7f9fca000640 1 -- 172.21.15.42:0/2922885702 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fbc0ab600 con 0x7f9fcc06f660 2024-05-28T14:06:25.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.618+0000 7f9fca000640 1 -- 172.21.15.42:0/2922885702 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0ab9d0 con 0x7f9fcc06f660 2024-05-28T14:06:25.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.618+0000 7f9fd1600640 1 -- 172.21.15.42:0/2922885702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fcc06f660 msgr2=0x7f9fcc073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:25.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.618+0000 7f9fd1600640 1 --2- 172.21.15.42:0/2922885702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fcc06f660 0x7f9fcc073980 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc15cb20 tx=0x7f9fbc01c390 comp rx=0 tx=0).stop 2024-05-28T14:06:25.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.618+0000 7f9fd1600640 1 -- 172.21.15.42:0/2922885702 shutdown_connections 2024-05-28T14:06:25.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.618+0000 7f9fd1600640 1 --2- 172.21.15.42:0/2922885702 >> v2:172.21.15.63:3300/0 conn(0x7f9fcc0d0e00 0x7f9fcc06dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.618+0000 7f9fd1600640 1 --2- 172.21.15.42:0/2922885702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fcc06f660 0x7f9fcc073980 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.618+0000 7f9fd1600640 1 -- 172.21.15.42:0/2922885702 >> 172.21.15.42:0/2922885702 conn(0x7f9fcc06bdf0 msgr2=0x7f9fcc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:25.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.618+0000 7f9fd1600640 1 -- 172.21.15.42:0/2922885702 shutdown_connections 2024-05-28T14:06:25.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.618+0000 7f9fd1600640 1 -- 172.21.15.42:0/2922885702 wait complete. 2024-05-28T14:06:25.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fd1600640 1 Processor -- start 2024-05-28T14:06:25.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fd1600640 1 -- start start 2024-05-28T14:06:25.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fd1600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fcc06f660 0x7f9fcc0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:25.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fd1600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fcc0d0e00 0x7f9fcc0cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:25.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fd1600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fcc0b1c00 0x7f9fcc0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:25.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fd1600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc168550 con 0x7f9fcc06f660 2024-05-28T14:06:25.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fd1600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1683d0 con 0x7f9fcc0d0e00 2024-05-28T14:06:25.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fd1600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9fcc1686d0 con 0x7f9fcc0b1c00 2024-05-28T14:06:25.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fcaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fcc06f660 0x7f9fcc0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:25.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fcaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fcc06f660 0x7f9fcc0cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43110/0 (socket says 172.21.15.42:43110) 2024-05-28T14:06:25.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fcaa00640 1 -- 172.21.15.42:0/3337454370 learned_addr learned my addr 172.21.15.42:0/3337454370 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:25.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fcb400640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fcc0b1c00 0x7f9fcc0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:25.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fc2000640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fcc0d0e00 0x7f9fcc0cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:25.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fcaa00640 1 -- 172.21.15.42:0/3337454370 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fcc0b1c00 msgr2=0x7f9fcc0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:25.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fcaa00640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fcc0b1c00 0x7f9fcc0aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fcaa00640 1 -- 172.21.15.42:0/3337454370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fcc0d0e00 msgr2=0x7f9fcc0cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:25.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fcaa00640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fcc0d0e00 0x7f9fcc0cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fcaa00640 1 -- 172.21.15.42:0/3337454370 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc14ed50 con 0x7f9fcc06f660 2024-05-28T14:06:25.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fcb400640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fcc0b1c00 0x7f9fcc0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:25.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fcaa00640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fcc06f660 0x7f9fcc0cd2f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0ae040 tx=0x7f9fbc019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:25.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fc3400640 1 -- 172.21.15.42:0/3337454370 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0b0040 con 0x7f9fcc06f660 2024-05-28T14:06:25.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fc3400640 1 -- 172.21.15.42:0/3337454370 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fbc0b9070 con 0x7f9fcc06f660 2024-05-28T14:06:25.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fc3400640 1 -- 172.21.15.42:0/3337454370 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0b48a0 con 0x7f9fcc06f660 2024-05-28T14:06:25.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fd1600640 1 -- 172.21.15.42:0/3337454370 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fcc0b20e0 con 0x7f9fcc06f660 2024-05-28T14:06:25.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fd1600640 1 -- 172.21.15.42:0/3337454370 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fcc0b26a0 con 0x7f9fcc06f660 2024-05-28T14:06:25.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.622+0000 7f9fd1600640 1 -- 172.21.15.42:0/3337454370 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fcc068a90 con 0x7f9fcc06f660 2024-05-28T14:06:25.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.626+0000 7f9fc3400640 1 -- 172.21.15.42:0/3337454370 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f9fbc0ab4c0 con 0x7f9fcc06f660 2024-05-28T14:06:25.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.626+0000 7f9fc3400640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fa8069070 0x7f9fa806b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:25.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.626+0000 7f9fc3400640 1 -- 172.21.15.42:0/3337454370 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5737+0+0 (secure 0 0 0) 0x7f9fbc11fdf0 con 0x7f9fcc06f660 2024-05-28T14:06:25.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.626+0000 7f9fc2000640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fa8069070 0x7f9fa806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:25.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.626+0000 7f9fc2000640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fa8069070 0x7f9fa806b530 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc152400 tx=0x7f9fb4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:25.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.630+0000 7f9fc3400640 1 -- 172.21.15.42:0/3337454370 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9fbc0160d0 con 0x7f9fcc06f660 2024-05-28T14:06:25.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.798+0000 7f9fd1600640 1 -- 172.21.15.42:0/3337454370 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fcc0b41b0 con 0x7f9fcc06f660 2024-05-28T14:06:25.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.798+0000 7f9fc3400640 1 -- 172.21.15.42:0/3337454370 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16664 (secure 0 0 0) 0x7f9fbc0f1d70 con 0x7f9fcc06f660 2024-05-28T14:06:25.785 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:25.786 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":73,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:06:24.814704+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:06:23.241667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-28T14:06:23.769074+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:06:25.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.802+0000 7f9fd1600640 1 -- 172.21.15.42:0/3337454370 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fa8069070 msgr2=0x7f9fa806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:25.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.802+0000 7f9fd1600640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fa8069070 0x7f9fa806b530 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc152400 tx=0x7f9fb4008040 comp rx=0 tx=0).stop 2024-05-28T14:06:25.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.802+0000 7f9fd1600640 1 -- 172.21.15.42:0/3337454370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fcc06f660 msgr2=0x7f9fcc0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:25.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.802+0000 7f9fd1600640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fcc06f660 0x7f9fcc0cd2f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0ae040 tx=0x7f9fbc019130 comp rx=0 tx=0).stop 2024-05-28T14:06:25.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.802+0000 7f9fd1600640 1 -- 172.21.15.42:0/3337454370 shutdown_connections 2024-05-28T14:06:25.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.802+0000 7f9fd1600640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fa8069070 0x7f9fa806b530 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.802+0000 7f9fd1600640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fcc0b1c00 0x7f9fcc0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.802+0000 7f9fd1600640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fcc0d0e00 0x7f9fcc0cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.802+0000 7f9fd1600640 1 --2- 172.21.15.42:0/3337454370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fcc06f660 0x7f9fcc0cd2f0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.802+0000 7f9fd1600640 1 -- 172.21.15.42:0/3337454370 >> 172.21.15.42:0/3337454370 conn(0x7f9fcc06bdf0 msgr2=0x7f9fcc14f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:25.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.802+0000 7f9fd1600640 1 -- 172.21.15.42:0/3337454370 shutdown_connections 2024-05-28T14:06:25.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.802+0000 7f9fd1600640 1 -- 172.21.15.42:0/3337454370 wait complete. 2024-05-28T14:06:25.807 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:06:25.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.962+0000 7f8652600640 1 Processor -- start 2024-05-28T14:06:25.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.962+0000 7f8652600640 1 -- start start 2024-05-28T14:06:25.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.962+0000 7f8652600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f864c15ba40 0x7f864c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:25.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.962+0000 7f8652600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c072dd0 0x7f864c0d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:25.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.962+0000 7f8652600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f864c15f280 con 0x7f864c06f660 2024-05-28T14:06:25.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.962+0000 7f8652600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f864c15f030 con 0x7f864c15ba40 2024-05-28T14:06:25.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.962+0000 7f8652600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f864c15f560 con 0x7f864c072dd0 2024-05-28T14:06:25.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.962+0000 7f8642a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f864c15ba40 0x7f864c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:25.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.966+0000 7f8642a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f864c15ba40 0x7f864c0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43118/0 (socket says 172.21.15.42:43118) 2024-05-28T14:06:25.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.966+0000 7f8642a00640 1 -- 172.21.15.42:0/1450060000 learned_addr learned my addr 172.21.15.42:0/1450060000 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.966+0000 7f8642a00640 1 -- 172.21.15.42:0/1450060000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c072dd0 msgr2=0x7f864c0d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.966+0000 7f864be00640 1 --2- 172.21.15.42:0/1450060000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c072dd0 0x7f864c0d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.966+0000 7f8642a00640 1 --2- 172.21.15.42:0/1450060000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c072dd0 0x7f864c0d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.966+0000 7f8642a00640 1 -- 172.21.15.42:0/1450060000 >> v1:172.21.15.63:6789/0 conn(0x7f864c06f660 legacy=0x7f864c073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.966+0000 7f8642a00640 1 -- 172.21.15.42:0/1450060000 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864c15f840 con 0x7f864c15ba40 2024-05-28T14:06:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.966+0000 7f864be00640 1 --2- 172.21.15.42:0/1450060000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c072dd0 0x7f864c0d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:25.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.966+0000 7f8642a00640 1 --2- 172.21.15.42:0/1450060000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f864c15ba40 0x7f864c0d06f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8634009870 tx=0x7f863401c0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90de2fa192c5546 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:25.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.966+0000 7f864aa00640 1 -- 172.21.15.42:0/1450060000 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86340b1070 con 0x7f864c15ba40 2024-05-28T14:06:25.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.966+0000 7f864aa00640 1 -- 172.21.15.42:0/1450060000 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86340a48b0 con 0x7f864c15ba40 2024-05-28T14:06:25.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.966+0000 7f864aa00640 1 -- 172.21.15.42:0/1450060000 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86340a4c80 con 0x7f864c15ba40 2024-05-28T14:06:25.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 -- 172.21.15.42:0/1450060000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f864c15ba40 msgr2=0x7f864c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 --2- 172.21.15.42:0/1450060000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f864c15ba40 0x7f864c0d06f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8634009870 tx=0x7f863401c0b0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 -- 172.21.15.42:0/1450060000 shutdown_connections 2024-05-28T14:06:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 --2- 172.21.15.42:0/1450060000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c072dd0 0x7f864c0d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 --2- 172.21.15.42:0/1450060000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f864c15ba40 0x7f864c0d06f0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 -- 172.21.15.42:0/1450060000 >> 172.21.15.42:0/1450060000 conn(0x7f864c06bdf0 msgr2=0x7f864c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 -- 172.21.15.42:0/1450060000 shutdown_connections 2024-05-28T14:06:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 -- 172.21.15.42:0/1450060000 wait complete. 2024-05-28T14:06:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 Processor -- start 2024-05-28T14:06:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 -- start start 2024-05-28T14:06:25.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c06f660 0x7f864c14ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:25.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f864c072dd0 0x7f864c14b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:25.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c14dd60 0x7f864c14e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:25.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f864c168f70 con 0x7f864c072dd0 2024-05-28T14:06:25.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f864c168df0 con 0x7f864c06f660 2024-05-28T14:06:25.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8652600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f864c1690f0 con 0x7f864c14dd60 2024-05-28T14:06:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8642a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f864c072dd0 0x7f864c14b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8642a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f864c072dd0 0x7f864c14b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43132/0 (socket says 172.21.15.42:43132) 2024-05-28T14:06:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8642a00640 1 -- 172.21.15.42:0/905058046 learned_addr learned my addr 172.21.15.42:0/905058046 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f864b400640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c06f660 0x7f864c14ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f864be00640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c14dd60 0x7f864c14e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8642a00640 1 -- 172.21.15.42:0/905058046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c14dd60 msgr2=0x7f864c14e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8642a00640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c14dd60 0x7f864c14e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8642a00640 1 -- 172.21.15.42:0/905058046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c06f660 msgr2=0x7f864c14ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8642a00640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c06f660 0x7f864c14ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8642a00640 1 -- 172.21.15.42:0/905058046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864c1518d0 con 0x7f864c072dd0 2024-05-28T14:06:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f864b400640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c06f660 0x7f864c14ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f864be00640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c14dd60 0x7f864c14e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.970+0000 7f8642a00640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f864c072dd0 0x7f864c14b000 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f863401c950 tx=0x7f8634019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:25.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.974+0000 7f8643e00640 1 -- 172.21.15.42:0/905058046 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86340bc070 con 0x7f864c072dd0 2024-05-28T14:06:25.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.974+0000 7f8643e00640 1 -- 172.21.15.42:0/905058046 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86340aca70 con 0x7f864c072dd0 2024-05-28T14:06:25.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.974+0000 7f8652600640 1 -- 172.21.15.42:0/905058046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864c14c220 con 0x7f864c072dd0 2024-05-28T14:06:25.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.974+0000 7f8643e00640 1 -- 172.21.15.42:0/905058046 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86340b1040 con 0x7f864c072dd0 2024-05-28T14:06:25.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.974+0000 7f8652600640 1 -- 172.21.15.42:0/905058046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864c09ad80 con 0x7f864c072dd0 2024-05-28T14:06:25.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.974+0000 7f8652600640 1 -- 172.21.15.42:0/905058046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f864c068a90 con 0x7f864c072dd0 2024-05-28T14:06:25.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.974+0000 7f8643e00640 1 -- 172.21.15.42:0/905058046 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f86340bf070 con 0x7f864c072dd0 2024-05-28T14:06:25.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.974+0000 7f8643e00640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f86280690a0 0x7f862806b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:25.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.974+0000 7f8643e00640 1 -- 172.21.15.42:0/905058046 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5737+0+0 (secure 0 0 0) 0x7f863411e560 con 0x7f864c072dd0 2024-05-28T14:06:25.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.974+0000 7f864b400640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f86280690a0 0x7f862806b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:25.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.978+0000 7f864b400640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f86280690a0 0x7f862806b560 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f863c00a850 tx=0x7f863c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:25.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:25.978+0000 7f8643e00640 1 -- 172.21.15.42:0/905058046 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8634016110 con 0x7f864c072dd0 2024-05-28T14:06:26.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.142+0000 7f8652600640 1 -- 172.21.15.42:0/905058046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f864c09af50 con 0x7f864c072dd0 2024-05-28T14:06:26.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.146+0000 7f8643e00640 1 -- 172.21.15.42:0/905058046 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16664 (secure 0 0 0) 0x7f86340f0570 con 0x7f864c072dd0 2024-05-28T14:06:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:26.132 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":73,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:06:24.814704+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:06:23.241667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-28T14:06:23.769074+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:06:26.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.146+0000 7f8652600640 1 -- 172.21.15.42:0/905058046 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f86280690a0 msgr2=0x7f862806b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:26.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.146+0000 7f8652600640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f86280690a0 0x7f862806b560 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f863c00a850 tx=0x7f863c008040 comp rx=0 tx=0).stop 2024-05-28T14:06:26.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.146+0000 7f8652600640 1 -- 172.21.15.42:0/905058046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f864c072dd0 msgr2=0x7f864c14b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:26.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.146+0000 7f8652600640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f864c072dd0 0x7f864c14b000 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f863401c950 tx=0x7f8634019170 comp rx=0 tx=0).stop 2024-05-28T14:06:26.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.146+0000 7f8652600640 1 -- 172.21.15.42:0/905058046 shutdown_connections 2024-05-28T14:06:26.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.146+0000 7f8652600640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f86280690a0 0x7f862806b560 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:26.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.146+0000 7f8652600640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c14dd60 0x7f864c14e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:26.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.146+0000 7f8652600640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f864c072dd0 0x7f864c14b000 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:26.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.146+0000 7f8652600640 1 --2- 172.21.15.42:0/905058046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c06f660 0x7f864c14ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:26.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.146+0000 7f8652600640 1 -- 172.21.15.42:0/905058046 >> 172.21.15.42:0/905058046 conn(0x7f864c06bdf0 msgr2=0x7f864c15d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:26.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.146+0000 7f8652600640 1 -- 172.21.15.42:0/905058046 shutdown_connections 2024-05-28T14:06:26.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.146+0000 7f8652600640 1 -- 172.21.15.42:0/905058046 wait complete. 2024-05-28T14:06:26.153 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-28T14:06:26.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.310+0000 7f6100400640 1 Processor -- start 2024-05-28T14:06:26.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.310+0000 7f6100400640 1 -- start start 2024-05-28T14:06:26.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.310+0000 7f6100400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f80724f0 0x7f60f8169530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:26.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.310+0000 7f6100400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60f80b3690 con 0x7f60f80b4c10 2024-05-28T14:06:26.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.310+0000 7f6100400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60f80b3970 con 0x7f60f8148090 2024-05-28T14:06:26.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.310+0000 7f6100400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60f8148470 con 0x7f60f80724f0 2024-05-28T14:06:26.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.310+0000 7f60fd000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f60f80b4c10 0x7f60f8071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42630/0 (socket says 172.21.15.42:42630) 2024-05-28T14:06:26.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.310+0000 7f60fd000640 1 -- 172.21.15.42:0/655882079 learned_addr learned my addr 172.21.15.42:0/655882079 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:26.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.310+0000 7f60fe400640 1 --2- 172.21.15.42:0/655882079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f80724f0 0x7f60f8169530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:26.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.310+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1648936545 0 0) 0x7f60f80b3690 con 0x7f60f80b4c10 2024-05-28T14:06:26.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.310+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60e0003530 con 0x7f60f80b4c10 2024-05-28T14:06:26.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3203998004 0 0) 0x7f60f80b3970 con 0x7f60f8148090 2024-05-28T14:06:26.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60f80b3690 con 0x7f60f8148090 2024-05-28T14:06:26.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2826856406 0 0) 0x7f60e0003530 con 0x7f60f80b4c10 2024-05-28T14:06:26.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f80724f0 msgr2=0x7f60f8169530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:26.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 --2- 172.21.15.42:0/655882079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f80724f0 0x7f60f8169530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:26.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 >> v1:172.21.15.63:6789/0 conn(0x7f60f8148090 legacy=0x7f60f80b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:26.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f80b3c50 con 0x7f60f80b4c10 2024-05-28T14:06:26.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f60e40016a0 con 0x7f60f80b4c10 2024-05-28T14:06:26.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60fe400640 1 --2- 172.21.15.42:0/655882079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f80724f0 0x7f60f8169530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:06:26.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 >> v1:172.21.15.42:6789/0 conn(0x7f60f80b4c10 legacy=0x7f60f8071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:26.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 --2- 172.21.15.42:0/655882079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60e0003b80 0x7f60e0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:26.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 --2- 172.21.15.42:0/655882079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e0004810 0x7f60e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:26.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 --2- 172.21.15.42:0/655882079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f80724f0 0x7f60e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:26.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8148470 con 0x7f60e0003b80 2024-05-28T14:06:26.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60e000bae0 con 0x7f60e0004810 2024-05-28T14:06:26.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60e000bce0 con 0x7f60f80724f0 2024-05-28T14:06:26.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60fda00640 1 --2- 172.21.15.42:0/655882079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60e0003b80 0x7f60e0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:26.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60fd000640 1 --2- 172.21.15.42:0/655882079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e0004810 0x7f60e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:26.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60fe400640 1 --2- 172.21.15.42:0/655882079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f80724f0 0x7f60e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:26.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60fda00640 1 -- 172.21.15.42:0/655882079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f80724f0 msgr2=0x7f60e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:26.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60fda00640 1 --2- 172.21.15.42:0/655882079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f80724f0 0x7f60e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:26.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60fda00640 1 -- 172.21.15.42:0/655882079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e0004810 msgr2=0x7f60e0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:26.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60fda00640 1 --2- 172.21.15.42:0/655882079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e0004810 0x7f60e0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:26.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60fda00640 1 -- 172.21.15.42:0/655882079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e0003530 con 0x7f60e0003b80 2024-05-28T14:06:26.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60fd000640 1 --2- 172.21.15.42:0/655882079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e0004810 0x7f60e0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60fda00640 1 --2- 172.21.15.42:0/655882079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60e0003b80 0x7f60e0003f80 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f60ec007fd0 tx=0x7f60ec01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a6ca86c9dd16f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.314+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec001520 con 0x7f60e0003b80 2024-05-28T14:06:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ec0052f0 con 0x7f60e0003b80 2024-05-28T14:06:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f60f7e00640 1 -- 172.21.15.42:0/655882079 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec003840 con 0x7f60e0003b80 2024-05-28T14:06:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f6100400640 1 -- 172.21.15.42:0/655882079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60e0003b80 msgr2=0x7f60e0003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f6100400640 1 --2- 172.21.15.42:0/655882079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60e0003b80 0x7f60e0003f80 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f60ec007fd0 tx=0x7f60ec01c540 comp rx=0 tx=0).stop 2024-05-28T14:06:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f60fe400640 1 -- 172.21.15.42:0/655882079 reap_dead start 2024-05-28T14:06:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f6100400640 1 -- 172.21.15.42:0/655882079 shutdown_connections 2024-05-28T14:06:26.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f6100400640 1 -- 172.21.15.42:0/655882079 >> 172.21.15.42:0/655882079 conn(0x7f60f80b21f0 msgr2=0x7f60f80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:26.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f6100400640 1 -- 172.21.15.42:0/655882079 shutdown_connections 2024-05-28T14:06:26.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f6100400640 1 -- 172.21.15.42:0/655882079 wait complete. 2024-05-28T14:06:26.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f6100400640 1 Processor -- start 2024-05-28T14:06:26.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f6100400640 1 -- start start 2024-05-28T14:06:26.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f6100400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e0003b80 0x7f60f815da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:26.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f6100400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f815df70 0x7f60f81fb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:26.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f6100400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f81fb7a0 0x7f60f813c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:26.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f6100400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f60f816a0d0 con 0x7f60f815df70 2024-05-28T14:06:26.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f6100400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8169f50 con 0x7f60e0003b80 2024-05-28T14:06:26.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f6100400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60f816a250 con 0x7f60f81fb7a0 2024-05-28T14:06:26.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f60fd000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f815df70 0x7f60f81fb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:26.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f60fd000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f815df70 0x7f60f81fb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43150/0 (socket says 172.21.15.42:43150) 2024-05-28T14:06:26.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f60fd000640 1 -- 172.21.15.42:0/4013943120 learned_addr learned my addr 172.21.15.42:0/4013943120 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:26.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f60fd000640 1 -- 172.21.15.42:0/4013943120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f81fb7a0 msgr2=0x7f60f813c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:26.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f60fd000640 1 --2- 172.21.15.42:0/4013943120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f81fb7a0 0x7f60f813c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:26.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f60fd000640 1 -- 172.21.15.42:0/4013943120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e0003b80 msgr2=0x7f60f815da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:26.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.318+0000 7f60fd000640 1 --2- 172.21.15.42:0/4013943120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e0003b80 0x7f60f815da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:26.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.322+0000 7f60fd000640 1 -- 172.21.15.42:0/4013943120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f8159450 con 0x7f60f815df70 2024-05-28T14:06:26.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.322+0000 7f60fd000640 1 --2- 172.21.15.42:0/4013943120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f815df70 0x7f60f81fb260 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f60e400e780 tx=0x7f60e400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:26.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.322+0000 7f60f5600640 1 -- 172.21.15.42:0/4013943120 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e400c640 con 0x7f60f815df70 2024-05-28T14:06:26.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.322+0000 7f60f5600640 1 -- 172.21.15.42:0/4013943120 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e4018070 con 0x7f60f815df70 2024-05-28T14:06:26.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.322+0000 7f6100400640 1 -- 172.21.15.42:0/4013943120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f81596f0 con 0x7f60f815df70 2024-05-28T14:06:26.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.322+0000 7f60f5600640 1 -- 172.21.15.42:0/4013943120 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e400c9e0 con 0x7f60f815df70 2024-05-28T14:06:26.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.322+0000 7f6100400640 1 -- 172.21.15.42:0/4013943120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f8159cb0 con 0x7f60f815df70 2024-05-28T14:06:26.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.322+0000 7f6100400640 1 -- 172.21.15.42:0/4013943120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f80738b0 con 0x7f60f815df70 2024-05-28T14:06:26.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.322+0000 7f60f5600640 1 -- 172.21.15.42:0/4013943120 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f60e400cb80 con 0x7f60f815df70 2024-05-28T14:06:26.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.322+0000 7f60f5600640 1 --2- 172.21.15.42:0/4013943120 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60d8069070 0x7f60d806b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:26.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.322+0000 7f60f5600640 1 -- 172.21.15.42:0/4013943120 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5737+0+0 (secure 0 0 0) 0x7f60e4085390 con 0x7f60f815df70 2024-05-28T14:06:26.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.322+0000 7f60fda00640 1 --2- 172.21.15.42:0/4013943120 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60d8069070 0x7f60d806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:26.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.326+0000 7f60fda00640 1 --2- 172.21.15.42:0/4013943120 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60d8069070 0x7f60d806b530 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0ac000 tx=0x7f60ec0012e0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:26.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.326+0000 7f60f5600640 1 -- 172.21.15.42:0/4013943120 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f60e4057110 con 0x7f60f815df70 2024-05-28T14:06:26.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:26.534+0000 7f6100400640 1 -- 172.21.15.42:0/4013943120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f60f804fdd0 con 0x7f60f815df70 2024-05-28T14:06:27.378 INFO:tasks.ceph.mds.a.smithi042.stderr:2024-05-28T14:06:27.390+0000 7fe9c9800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-28T14:06:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.430+0000 7f60f5600640 1 -- 172.21.15.42:0/4013943120 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7f60e4056a10 con 0x7f60f815df70 2024-05-28T14:06:27.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.430+0000 7f6100400640 1 -- 172.21.15.42:0/4013943120 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60d8069070 msgr2=0x7f60d806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:27.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.430+0000 7f6100400640 1 --2- 172.21.15.42:0/4013943120 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60d8069070 0x7f60d806b530 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0ac000 tx=0x7f60ec0012e0 comp rx=0 tx=0).stop 2024-05-28T14:06:27.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.430+0000 7f6100400640 1 -- 172.21.15.42:0/4013943120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f815df70 msgr2=0x7f60f81fb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:27.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.430+0000 7f6100400640 1 --2- 172.21.15.42:0/4013943120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f815df70 0x7f60f81fb260 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f60e400e780 tx=0x7f60e400ec40 comp rx=0 tx=0).stop 2024-05-28T14:06:27.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.430+0000 7f6100400640 1 -- 172.21.15.42:0/4013943120 shutdown_connections 2024-05-28T14:06:27.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.430+0000 7f6100400640 1 --2- 172.21.15.42:0/4013943120 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60d8069070 0x7f60d806b530 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:27.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.430+0000 7f6100400640 1 --2- 172.21.15.42:0/4013943120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f81fb7a0 0x7f60f813c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:27.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.430+0000 7f6100400640 1 --2- 172.21.15.42:0/4013943120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f815df70 0x7f60f81fb260 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:27.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.430+0000 7f6100400640 1 --2- 172.21.15.42:0/4013943120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e0003b80 0x7f60f815da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:27.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.430+0000 7f6100400640 1 -- 172.21.15.42:0/4013943120 >> 172.21.15.42:0/4013943120 conn(0x7f60f80b21f0 msgr2=0x7f60f806ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:27.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.430+0000 7f6100400640 1 -- 172.21.15.42:0/4013943120 shutdown_connections 2024-05-28T14:06:27.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.430+0000 7f6100400640 1 -- 172.21.15.42:0/4013943120 wait complete. 2024-05-28T14:06:27.443 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-28T14:06:27.443 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-28T14:06:27.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.606+0000 7fa103a00640 1 Processor -- start 2024-05-28T14:06:27.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.606+0000 7fa103a00640 1 -- start start 2024-05-28T14:06:27.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.606+0000 7fa103a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa0fc071de0 0x7fa0fc0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:27.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.606+0000 7fa103a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0fc072700 con 0x7fa0fc13fd10 2024-05-28T14:06:27.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.606+0000 7fa103a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0fc160230 con 0x7fa0fc0ce3b0 2024-05-28T14:06:27.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.606+0000 7fa103a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa0fc160510 con 0x7fa0fc071de0 2024-05-28T14:06:27.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.606+0000 7fa101000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa0fc071de0 0x7fa0fc0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:27.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.606+0000 7fa101000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa0fc071de0 0x7fa0fc0721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53076/0 (socket says 172.21.15.42:53076) 2024-05-28T14:06:27.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.606+0000 7fa101a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa0fc13fd10 0x7fa0fc140160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42804/0 (socket says 172.21.15.42:42804) 2024-05-28T14:06:27.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.606+0000 7fa101000640 1 -- 172.21.15.42:0/4168650192 learned_addr learned my addr 172.21.15.42:0/4168650192 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:27.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.606+0000 7fa101000640 1 -- 172.21.15.42:0/4168650192 >> v1:172.21.15.63:6789/0 conn(0x7fa0fc0ce3b0 legacy=0x7fa0fc0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:27.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.610+0000 7fa101000640 1 -- 172.21.15.42:0/4168650192 >> v1:172.21.15.42:6789/0 conn(0x7fa0fc13fd10 legacy=0x7fa0fc140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:27.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.610+0000 7fa101000640 1 -- 172.21.15.42:0/4168650192 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc1607f0 con 0x7fa0fc071de0 2024-05-28T14:06:27.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.610+0000 7fa101000640 1 --2- 172.21.15.42:0/4168650192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc071de0 0x7fa0fc0721c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009870 tx=0x7fa0f001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a204e41156344d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:27.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.610+0000 7fa0fb400640 1 -- 172.21.15.42:0/4168650192 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00a7650 con 0x7fa0fc071de0 2024-05-28T14:06:27.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.610+0000 7fa0fb400640 1 -- 172.21.15.42:0/4168650192 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f00a77f0 con 0x7fa0fc071de0 2024-05-28T14:06:27.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.610+0000 7fa0fb400640 1 -- 172.21.15.42:0/4168650192 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00a7bc0 con 0x7fa0fc071de0 2024-05-28T14:06:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.610+0000 7fa103a00640 1 -- 172.21.15.42:0/4168650192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc071de0 msgr2=0x7fa0fc0721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.610+0000 7fa103a00640 1 --2- 172.21.15.42:0/4168650192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc071de0 0x7fa0fc0721c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009870 tx=0x7fa0f001c000 comp rx=0 tx=0).stop 2024-05-28T14:06:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.610+0000 7fa103a00640 1 -- 172.21.15.42:0/4168650192 shutdown_connections 2024-05-28T14:06:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.610+0000 7fa103a00640 1 --2- 172.21.15.42:0/4168650192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc071de0 0x7fa0fc0721c0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.610+0000 7fa103a00640 1 -- 172.21.15.42:0/4168650192 >> 172.21.15.42:0/4168650192 conn(0x7fa0fc0b21f0 msgr2=0x7fa0fc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.610+0000 7fa103a00640 1 -- 172.21.15.42:0/4168650192 shutdown_connections 2024-05-28T14:06:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.610+0000 7fa103a00640 1 -- 172.21.15.42:0/4168650192 wait complete. 2024-05-28T14:06:27.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa103a00640 1 Processor -- start 2024-05-28T14:06:27.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa103a00640 1 -- start start 2024-05-28T14:06:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa103a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0fc071de0 0x7fa0fc15ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa103a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0fc0ce3b0 0x7fa0fc138f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa103a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc13fd10 0x7fa0fc0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa103a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc16b870 con 0x7fa0fc13fd10 2024-05-28T14:06:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa103a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc16b6f0 con 0x7fa0fc071de0 2024-05-28T14:06:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa103a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa0fc16b9f0 con 0x7fa0fc0ce3b0 2024-05-28T14:06:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa101a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc13fd10 0x7fa0fc0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa101000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0fc071de0 0x7fa0fc15ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa101a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc13fd10 0x7fa0fc0b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53090/0 (socket says 172.21.15.42:53090) 2024-05-28T14:06:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa101000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0fc071de0 0x7fa0fc15ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:50528/0 (socket says 172.21.15.42:50528) 2024-05-28T14:06:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa101a00640 1 -- 172.21.15.42:0/3865853139 learned_addr learned my addr 172.21.15.42:0/3865853139 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa0fbe00640 1 --2- 172.21.15.42:0/3865853139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0fc0ce3b0 0x7fa0fc138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa101a00640 1 -- 172.21.15.42:0/3865853139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0fc0ce3b0 msgr2=0x7fa0fc138f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa101a00640 1 --2- 172.21.15.42:0/3865853139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0fc0ce3b0 0x7fa0fc138f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa101a00640 1 -- 172.21.15.42:0/3865853139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0fc071de0 msgr2=0x7fa0fc15ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa101a00640 1 --2- 172.21.15.42:0/3865853139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0fc071de0 0x7fa0fc15ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa101a00640 1 -- 172.21.15.42:0/3865853139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc0b1e90 con 0x7fa0fc13fd10 2024-05-28T14:06:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa0fbe00640 1 --2- 172.21.15.42:0/3865853139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0fc0ce3b0 0x7fa0fc138f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa101a00640 1 --2- 172.21.15.42:0/3865853139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc13fd10 0x7fa0fc0b1950 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400cb20 tx=0x7fa0f4002080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa0f8c00640 1 -- 172.21.15.42:0/3865853139 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4018070 con 0x7fa0fc13fd10 2024-05-28T14:06:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa0f8c00640 1 -- 172.21.15.42:0/3865853139 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f400a700 con 0x7fa0fc13fd10 2024-05-28T14:06:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa0f8c00640 1 -- 172.21.15.42:0/3865853139 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f400aad0 con 0x7fa0fc13fd10 2024-05-28T14:06:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa103a00640 1 -- 172.21.15.42:0/3865853139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc15f100 con 0x7fa0fc13fd10 2024-05-28T14:06:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.614+0000 7fa103a00640 1 -- 172.21.15.42:0/3865853139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc15f6c0 con 0x7fa0fc13fd10 2024-05-28T14:06:27.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.618+0000 7fa103a00640 1 -- 172.21.15.42:0/3865853139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0fc0738b0 con 0x7fa0fc13fd10 2024-05-28T14:06:27.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.618+0000 7fa0f8c00640 1 -- 172.21.15.42:0/3865853139 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7fa0f400ac70 con 0x7fa0fc13fd10 2024-05-28T14:06:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.618+0000 7fa0f8c00640 1 --2- 172.21.15.42:0/3865853139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa0dc069070 0x7fa0dc06b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.618+0000 7fa0f8c00640 1 -- 172.21.15.42:0/3865853139 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7fa0f4084b00 con 0x7fa0fc13fd10 2024-05-28T14:06:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.618+0000 7fa101000640 1 --2- 172.21.15.42:0/3865853139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa0dc069070 0x7fa0dc06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.618+0000 7fa101000640 1 --2- 172.21.15.42:0/3865853139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa0dc069070 0x7fa0dc06b530 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00ae000 tx=0x7fa0f00b2040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:27.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.622+0000 7fa0f8c00640 1 -- 172.21.15.42:0/3865853139 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa0f40564c0 con 0x7fa0fc13fd10 2024-05-28T14:06:27.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:27.822+0000 7fa103a00640 1 -- 172.21.15.42:0/3865853139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa0fc04fdd0 con 0x7fa0fc13fd10 2024-05-28T14:06:28.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.430+0000 7fa0f8c00640 1 -- 172.21.15.42:0/3865853139 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7fa0f405a7c0 con 0x7fa0fc13fd10 2024-05-28T14:06:28.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.434+0000 7fa103a00640 1 -- 172.21.15.42:0/3865853139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa0dc069070 msgr2=0x7fa0dc06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:28.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.434+0000 7fa103a00640 1 --2- 172.21.15.42:0/3865853139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa0dc069070 0x7fa0dc06b530 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00ae000 tx=0x7fa0f00b2040 comp rx=0 tx=0).stop 2024-05-28T14:06:28.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.434+0000 7fa103a00640 1 -- 172.21.15.42:0/3865853139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc13fd10 msgr2=0x7fa0fc0b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:28.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.434+0000 7fa103a00640 1 --2- 172.21.15.42:0/3865853139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc13fd10 0x7fa0fc0b1950 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400cb20 tx=0x7fa0f4002080 comp rx=0 tx=0).stop 2024-05-28T14:06:28.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.434+0000 7fa103a00640 1 -- 172.21.15.42:0/3865853139 shutdown_connections 2024-05-28T14:06:28.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.434+0000 7fa103a00640 1 --2- 172.21.15.42:0/3865853139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa0dc069070 0x7fa0dc06b530 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.434+0000 7fa103a00640 1 --2- 172.21.15.42:0/3865853139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc13fd10 0x7fa0fc0b1950 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.434+0000 7fa103a00640 1 --2- 172.21.15.42:0/3865853139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0fc0ce3b0 0x7fa0fc138f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.434+0000 7fa103a00640 1 --2- 172.21.15.42:0/3865853139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0fc071de0 0x7fa0fc15ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.434+0000 7fa103a00640 1 -- 172.21.15.42:0/3865853139 >> 172.21.15.42:0/3865853139 conn(0x7fa0fc0b21f0 msgr2=0x7fa0fc0b3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:28.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.434+0000 7fa103a00640 1 -- 172.21.15.42:0/3865853139 shutdown_connections 2024-05-28T14:06:28.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.434+0000 7fa103a00640 1 -- 172.21.15.42:0/3865853139 wait complete. 2024-05-28T14:06:28.446 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-28T14:06:28.446 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:06:28.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.606+0000 7f3d74800640 1 Processor -- start 2024-05-28T14:06:28.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.606+0000 7f3d74800640 1 -- start start 2024-05-28T14:06:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.606+0000 7f3d74800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3d6c0b3aa0 0x7f3d6c0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.606+0000 7f3d74800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6c06c700 con 0x7f3d6c06c100 2024-05-28T14:06:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.606+0000 7f3d74800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6c1605a0 con 0x7f3d6c15c970 2024-05-28T14:06:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.606+0000 7f3d74800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3d6c1611b0 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.606+0000 7f3d71400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f3d6c06c100 0x7f3d6c0d03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42806/0 (socket says 172.21.15.42:42806) 2024-05-28T14:06:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.606+0000 7f3d71400640 1 -- 172.21.15.42:0/1865160400 learned_addr learned my addr 172.21.15.42:0/1865160400 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:28.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.606+0000 7f3d72800640 1 --2- 172.21.15.42:0/1865160400 >> v2:172.21.15.63:3300/0 conn(0x7f3d6c0b3aa0 0x7f3d6c0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:28.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.606+0000 7f3d70a00640 1 -- 172.21.15.42:0/1865160400 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2436383329 0 0) 0x7f3d6c06c700 con 0x7f3d6c06c100 2024-05-28T14:06:28.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.606+0000 7f3d70a00640 1 -- 172.21.15.42:0/1865160400 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d54003530 con 0x7f3d6c06c100 2024-05-28T14:06:28.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.606+0000 7f3d72800640 1 -- 172.21.15.42:0/1865160400 >> v1:172.21.15.63:6789/0 conn(0x7f3d6c15c970 legacy=0x7f3d6c06b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:28.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.606+0000 7f3d72800640 1 -- 172.21.15.42:0/1865160400 >> v1:172.21.15.42:6789/0 conn(0x7f3d6c06c100 legacy=0x7f3d6c0d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:28.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.606+0000 7f3d72800640 1 -- 172.21.15.42:0/1865160400 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c160880 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.610+0000 7f3d72800640 1 --2- 172.21.15.42:0/1865160400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d6c0b3aa0 0x7f3d6c0ccfa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009d90 tx=0x7f3d6801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=143a2fcb1f470814 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.610+0000 7f3d70a00640 1 -- 172.21.15.42:0/1865160400 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d680b2070 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.610+0000 7f3d70a00640 1 -- 172.21.15.42:0/1865160400 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d680a5800 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.610+0000 7f3d70a00640 1 -- 172.21.15.42:0/1865160400 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d680a5bd0 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.610+0000 7f3d74800640 1 -- 172.21.15.42:0/1865160400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d6c0b3aa0 msgr2=0x7f3d6c0ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:28.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.610+0000 7f3d74800640 1 --2- 172.21.15.42:0/1865160400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d6c0b3aa0 0x7f3d6c0ccfa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009d90 tx=0x7f3d6801c970 comp rx=0 tx=0).stop 2024-05-28T14:06:28.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.610+0000 7f3d74800640 1 -- 172.21.15.42:0/1865160400 shutdown_connections 2024-05-28T14:06:28.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.610+0000 7f3d74800640 1 --2- 172.21.15.42:0/1865160400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d6c0b3aa0 0x7f3d6c0ccfa0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.610+0000 7f3d74800640 1 -- 172.21.15.42:0/1865160400 >> 172.21.15.42:0/1865160400 conn(0x7f3d6c0b2f80 msgr2=0x7f3d6c0b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:28.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.610+0000 7f3d74800640 1 -- 172.21.15.42:0/1865160400 shutdown_connections 2024-05-28T14:06:28.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.610+0000 7f3d74800640 1 -- 172.21.15.42:0/1865160400 wait complete. 2024-05-28T14:06:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d74800640 1 Processor -- start 2024-05-28T14:06:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d74800640 1 -- start start 2024-05-28T14:06:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d74800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d6c06c100 0x7f3d6c167be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d74800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d6c0b3aa0 0x7f3d6c155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d74800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d6c156350 0x7f3d6c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d74800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c16a910 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d74800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c160350 con 0x7f3d6c06c100 2024-05-28T14:06:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d74800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6c16aa90 con 0x7f3d6c156350 2024-05-28T14:06:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d71400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d6c0b3aa0 0x7f3d6c155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d71400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d6c0b3aa0 0x7f3d6c155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53094/0 (socket says 172.21.15.42:53094) 2024-05-28T14:06:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d71400640 1 -- 172.21.15.42:0/878197426 learned_addr learned my addr 172.21.15.42:0/878197426 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d71e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d6c06c100 0x7f3d6c167be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d72800640 1 --2- 172.21.15.42:0/878197426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d6c156350 0x7f3d6c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d71400640 1 -- 172.21.15.42:0/878197426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d6c156350 msgr2=0x7f3d6c1567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d71400640 1 --2- 172.21.15.42:0/878197426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d6c156350 0x7f3d6c1567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d71400640 1 -- 172.21.15.42:0/878197426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d6c06c100 msgr2=0x7f3d6c167be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d71400640 1 --2- 172.21.15.42:0/878197426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d6c06c100 0x7f3d6c167be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d71400640 1 -- 172.21.15.42:0/878197426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c0c7e10 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d71400640 1 --2- 172.21.15.42:0/878197426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d6c0b3aa0 0x7f3d6c155e10 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00c420 tx=0x7f3d5c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:28.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d66000640 1 -- 172.21.15.42:0/878197426 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c017070 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d66000640 1 -- 172.21.15.42:0/878197426 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d5c005cf0 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d74800640 1 -- 172.21.15.42:0/878197426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c166d60 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d66000640 1 -- 172.21.15.42:0/878197426 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c0072d0 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.614+0000 7f3d74800640 1 -- 172.21.15.42:0/878197426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c0c7330 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.618+0000 7f3d74800640 1 -- 172.21.15.42:0/878197426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c068a90 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.618+0000 7f3d66000640 1 -- 172.21.15.42:0/878197426 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f3d5c01a070 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.618+0000 7f3d66000640 1 --2- 172.21.15.42:0/878197426 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d50069070 0x7f3d5006b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.618+0000 7f3d66000640 1 -- 172.21.15.42:0/878197426 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7f3d5c0846b0 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.618+0000 7f3d71e00640 1 --2- 172.21.15.42:0/878197426 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d50069070 0x7f3d5006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:28.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.618+0000 7f3d71e00640 1 --2- 172.21.15.42:0/878197426 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d50069070 0x7f3d5006b530 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3d60007fd0 tx=0x7f3d60009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:28.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.622+0000 7f3d66000640 1 -- 172.21.15.42:0/878197426 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3d5c056070 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.818+0000 7f3d74800640 1 -- 172.21.15.42:0/878197426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d6c0b73b0 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.822+0000 7f3d66000640 1 -- 172.21.15.42:0/878197426 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5180 (secure 0 0 0) 0x7f3d5c05a370 con 0x7f3d6c0b3aa0 2024-05-28T14:06:28.808 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:28.808 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2618011545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2618011545},{"type":"v1","addr":"172.21.15.63:6835","nonce":2618011545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":7077,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/986242955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":986242955},{"type":"v1","addr":"172.21.15.42:6836","nonce":986242955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":7080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1618433260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1618433260},{"type":"v1","addr":"172.21.15.63:6834","nonce":1618433260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:06:24.814669+0000","modified":"2024-05-28T14:06:27.829347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6837/1199082803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1199082803},{"type":"v1","addr":"172.21.15.42:6837","nonce":1199082803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7083,"qdb_cluster":[7083]},"id":5}]} 2024-05-28T14:06:28.808 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 46 2024-05-28T14:06:28.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.822+0000 7f3d74800640 1 -- 172.21.15.42:0/878197426 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d50069070 msgr2=0x7f3d5006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:28.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.822+0000 7f3d74800640 1 --2- 172.21.15.42:0/878197426 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d50069070 0x7f3d5006b530 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3d60007fd0 tx=0x7f3d60009040 comp rx=0 tx=0).stop 2024-05-28T14:06:28.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.822+0000 7f3d74800640 1 -- 172.21.15.42:0/878197426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d6c0b3aa0 msgr2=0x7f3d6c155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:28.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.822+0000 7f3d74800640 1 --2- 172.21.15.42:0/878197426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d6c0b3aa0 0x7f3d6c155e10 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00c420 tx=0x7f3d5c00c8e0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.822+0000 7f3d74800640 1 -- 172.21.15.42:0/878197426 shutdown_connections 2024-05-28T14:06:28.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.822+0000 7f3d74800640 1 --2- 172.21.15.42:0/878197426 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d50069070 0x7f3d5006b530 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.822+0000 7f3d74800640 1 --2- 172.21.15.42:0/878197426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d6c156350 0x7f3d6c1567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.822+0000 7f3d74800640 1 --2- 172.21.15.42:0/878197426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d6c0b3aa0 0x7f3d6c155e10 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.822+0000 7f3d74800640 1 --2- 172.21.15.42:0/878197426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d6c06c100 0x7f3d6c167be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.822+0000 7f3d74800640 1 -- 172.21.15.42:0/878197426 >> 172.21.15.42:0/878197426 conn(0x7f3d6c0b2f80 msgr2=0x7f3d6c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:28.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.822+0000 7f3d74800640 1 -- 172.21.15.42:0/878197426 shutdown_connections 2024-05-28T14:06:28.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.822+0000 7f3d74800640 1 -- 172.21.15.42:0/878197426 wait complete. 2024-05-28T14:06:28.830 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:06:28.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faadd000640 1 Processor -- start 2024-05-28T14:06:28.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faadd000640 1 -- start start 2024-05-28T14:06:28.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faadd000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faad806f660 0x7faad8073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:28.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faadd000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad815ba40 0x7faad80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faadd000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faad815f280 con 0x7faad8072dd0 2024-05-28T14:06:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faadd000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7faad815f030 con 0x7faad806f660 2024-05-28T14:06:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faadd000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faad815f560 con 0x7faad815ba40 2024-05-28T14:06:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faad5e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faad806f660 0x7faad8073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faad5e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faad806f660 0x7faad8073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53100/0 (socket says 172.21.15.42:53100) 2024-05-28T14:06:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faad5e00640 1 -- 172.21.15.42:0/333394825 learned_addr learned my addr 172.21.15.42:0/333394825 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faacfe00640 1 --2- 172.21.15.42:0/333394825 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad815ba40 0x7faad80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faad5e00640 1 -- 172.21.15.42:0/333394825 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad815ba40 msgr2=0x7faad80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faad5e00640 1 --2- 172.21.15.42:0/333394825 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad815ba40 0x7faad80d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faad5e00640 1 -- 172.21.15.42:0/333394825 >> v1:172.21.15.63:6789/0 conn(0x7faad8072dd0 legacy=0x7faad80b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.986+0000 7faad5e00640 1 -- 172.21.15.42:0/333394825 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad815f840 con 0x7faad806f660 2024-05-28T14:06:28.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.990+0000 7faad5e00640 1 --2- 172.21.15.42:0/333394825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faad806f660 0x7faad8073980 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7faac8009870 tx=0x7faac801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce19aab2b5e927b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:28.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.990+0000 7faad5400640 1 -- 172.21.15.42:0/333394825 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac80b0070 con 0x7faad806f660 2024-05-28T14:06:28.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.990+0000 7faad5400640 1 -- 172.21.15.42:0/333394825 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac80ab600 con 0x7faad806f660 2024-05-28T14:06:28.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.990+0000 7faad5400640 1 -- 172.21.15.42:0/333394825 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac80ab9d0 con 0x7faad806f660 2024-05-28T14:06:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.990+0000 7faadd000640 1 -- 172.21.15.42:0/333394825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faad806f660 msgr2=0x7faad8073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.990+0000 7faadd000640 1 --2- 172.21.15.42:0/333394825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faad806f660 0x7faad8073980 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7faac8009870 tx=0x7faac801c410 comp rx=0 tx=0).stop 2024-05-28T14:06:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.990+0000 7faadd000640 1 -- 172.21.15.42:0/333394825 shutdown_connections 2024-05-28T14:06:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.990+0000 7faadd000640 1 --2- 172.21.15.42:0/333394825 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad815ba40 0x7faad80d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.990+0000 7faadd000640 1 --2- 172.21.15.42:0/333394825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faad806f660 0x7faad8073980 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.990+0000 7faadd000640 1 -- 172.21.15.42:0/333394825 >> 172.21.15.42:0/333394825 conn(0x7faad806bdf0 msgr2=0x7faad806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.990+0000 7faadd000640 1 -- 172.21.15.42:0/333394825 shutdown_connections 2024-05-28T14:06:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.990+0000 7faadd000640 1 -- 172.21.15.42:0/333394825 wait complete. 2024-05-28T14:06:28.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faadd000640 1 Processor -- start 2024-05-28T14:06:28.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faadd000640 1 -- start start 2024-05-28T14:06:28.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faadd000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faad806f660 0x7faad81f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:28.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faadd000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad8072dd0 0x7faad809aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faadd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad81603a0 0x7faad814f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faadd000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faad8168550 con 0x7faad806f660 2024-05-28T14:06:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faadd000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faad81683d0 con 0x7faad81603a0 2024-05-28T14:06:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faadd000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faad81686d0 con 0x7faad8072dd0 2024-05-28T14:06:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faad5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faad806f660 0x7faad81f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faad5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faad806f660 0x7faad81f79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53116/0 (socket says 172.21.15.42:53116) 2024-05-28T14:06:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faad5e00640 1 -- 172.21.15.42:0/777032214 learned_addr learned my addr 172.21.15.42:0/777032214 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faacfe00640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad8072dd0 0x7faad809aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faad6800640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad81603a0 0x7faad814f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faad5e00640 1 -- 172.21.15.42:0/777032214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad8072dd0 msgr2=0x7faad809aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faad5e00640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad8072dd0 0x7faad809aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faad5e00640 1 -- 172.21.15.42:0/777032214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad81603a0 msgr2=0x7faad814f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:28.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faad5e00640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad81603a0 0x7faad814f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:28.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faad5e00640 1 -- 172.21.15.42:0/777032214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad81f7f20 con 0x7faad806f660 2024-05-28T14:06:28.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faacfe00640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad8072dd0 0x7faad809aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:28.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faad5e00640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faad806f660 0x7faad81f79e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7faac80ae040 tx=0x7faac801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:28.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faace000640 1 -- 172.21.15.42:0/777032214 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac80ba020 con 0x7faad806f660 2024-05-28T14:06:28.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faace000640 1 -- 172.21.15.42:0/777032214 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac80b6640 con 0x7faad806f660 2024-05-28T14:06:28.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faace000640 1 -- 172.21.15.42:0/777032214 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac80b0040 con 0x7faad806f660 2024-05-28T14:06:28.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faadd000640 1 -- 172.21.15.42:0/777032214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad809b3f0 con 0x7faad806f660 2024-05-28T14:06:28.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faadd000640 1 -- 172.21.15.42:0/777032214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad809b9b0 con 0x7faad806f660 2024-05-28T14:06:28.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.994+0000 7faadd000640 1 -- 172.21.15.42:0/777032214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad809c0d0 con 0x7faad806f660 2024-05-28T14:06:28.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.998+0000 7faace000640 1 -- 172.21.15.42:0/777032214 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7faac80bd020 con 0x7faad806f660 2024-05-28T14:06:28.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.998+0000 7faace000640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faab4069070 0x7faab406b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:28.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.998+0000 7faace000640 1 -- 172.21.15.42:0/777032214 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7faac811f5f0 con 0x7faad806f660 2024-05-28T14:06:28.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.998+0000 7faacfe00640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faab4069070 0x7faab406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:28.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:28.998+0000 7faacfe00640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faab4069070 0x7faab406b530 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faad80cedb0 tx=0x7faac0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:28.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.002+0000 7faace000640 1 -- 172.21.15.42:0/777032214 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faac80c4030 con 0x7faad806f660 2024-05-28T14:06:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.166+0000 7faadd000640 1 -- 172.21.15.42:0/777032214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faad8068a90 con 0x7faad806f660 2024-05-28T14:06:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.166+0000 7faace000640 1 -- 172.21.15.42:0/777032214 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16651 (secure 0 0 0) 0x7faac8016150 con 0x7faad806f660 2024-05-28T14:06:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:29.155 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":74,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:06:26.420970+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:06:23.241667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-28T14:06:23.769074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:06:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.170+0000 7faadd000640 1 -- 172.21.15.42:0/777032214 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faab4069070 msgr2=0x7faab406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.170+0000 7faadd000640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faab4069070 0x7faab406b530 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faad80cedb0 tx=0x7faac0008040 comp rx=0 tx=0).stop 2024-05-28T14:06:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.170+0000 7faadd000640 1 -- 172.21.15.42:0/777032214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faad806f660 msgr2=0x7faad81f79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.170+0000 7faadd000640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faad806f660 0x7faad81f79e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7faac80ae040 tx=0x7faac801ac50 comp rx=0 tx=0).stop 2024-05-28T14:06:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.170+0000 7faadd000640 1 -- 172.21.15.42:0/777032214 shutdown_connections 2024-05-28T14:06:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.170+0000 7faadd000640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faab4069070 0x7faab406b530 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.170+0000 7faadd000640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad81603a0 0x7faad814f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.170+0000 7faadd000640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad8072dd0 0x7faad809aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.170+0000 7faadd000640 1 --2- 172.21.15.42:0/777032214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faad806f660 0x7faad81f79e0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.170+0000 7faadd000640 1 -- 172.21.15.42:0/777032214 >> 172.21.15.42:0/777032214 conn(0x7faad806bdf0 msgr2=0x7faad815a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.170+0000 7faadd000640 1 -- 172.21.15.42:0/777032214 shutdown_connections 2024-05-28T14:06:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.170+0000 7faadd000640 1 -- 172.21.15.42:0/777032214 wait complete. 2024-05-28T14:06:29.177 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-28T14:06:29.177 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:06:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.330+0000 7f3725400640 1 Processor -- start 2024-05-28T14:06:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.330+0000 7f3725400640 1 -- start start 2024-05-28T14:06:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.330+0000 7f3725400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f372015e5c0 0x7f37200b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.334+0000 7f3725400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37200b3720 con 0x7f37200b51b0 2024-05-28T14:06:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.334+0000 7f3725400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37200b39f0 con 0x7f37200d0e00 2024-05-28T14:06:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.334+0000 7f3725400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f37200b3cd0 con 0x7f372015e5c0 2024-05-28T14:06:29.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.334+0000 7f371e800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f372015e5c0 0x7f37200b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:29.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.334+0000 7f3717e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f37200b51b0 0x7f37200d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42814/0 (socket says 172.21.15.42:42814) 2024-05-28T14:06:29.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.334+0000 7f3717e00640 1 -- 172.21.15.42:0/3349624906 learned_addr learned my addr 172.21.15.42:0/3349624906 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:29.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.334+0000 7f371e800640 1 -- 172.21.15.42:0/3349624906 >> v1:172.21.15.63:6789/0 conn(0x7f37200d0e00 legacy=0x7f37200ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:29.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.334+0000 7f371e800640 1 -- 172.21.15.42:0/3349624906 >> v1:172.21.15.42:6789/0 conn(0x7f37200b51b0 legacy=0x7f37200d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:29.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.334+0000 7f371e800640 1 -- 172.21.15.42:0/3349624906 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37200b3fb0 con 0x7f372015e5c0 2024-05-28T14:06:29.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.334+0000 7f371de00640 1 -- 172.21.15.42:0/3349624906 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 554206710 0 0) 0x7f37200b3720 con 0x7f37200b51b0 2024-05-28T14:06:29.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.334+0000 7f371e800640 1 --2- 172.21.15.42:0/3349624906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f372015e5c0 0x7f37200b31e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f37100097c0 tx=0x7f371001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e4c14f6b0de878 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:29.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.334+0000 7f371de00640 1 -- 172.21.15.42:0/3349624906 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37100aa5f0 con 0x7f372015e5c0 2024-05-28T14:06:29.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.334+0000 7f371de00640 1 -- 172.21.15.42:0/3349624906 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37100af070 con 0x7f372015e5c0 2024-05-28T14:06:29.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.334+0000 7f371de00640 1 -- 172.21.15.42:0/3349624906 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37100aa9b0 con 0x7f372015e5c0 2024-05-28T14:06:29.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 -- 172.21.15.42:0/3349624906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f372015e5c0 msgr2=0x7f37200b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:29.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 --2- 172.21.15.42:0/3349624906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f372015e5c0 0x7f37200b31e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f37100097c0 tx=0x7f371001c470 comp rx=0 tx=0).stop 2024-05-28T14:06:29.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 -- 172.21.15.42:0/3349624906 shutdown_connections 2024-05-28T14:06:29.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 --2- 172.21.15.42:0/3349624906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f372015e5c0 0x7f37200b31e0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 -- 172.21.15.42:0/3349624906 >> 172.21.15.42:0/3349624906 conn(0x7f372006bdf0 msgr2=0x7f372006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:29.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 -- 172.21.15.42:0/3349624906 shutdown_connections 2024-05-28T14:06:29.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 -- 172.21.15.42:0/3349624906 wait complete. 2024-05-28T14:06:29.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 Processor -- start 2024-05-28T14:06:29.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 -- start start 2024-05-28T14:06:29.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37200b51b0 0x7f3720151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:29.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37200d0e00 0x7f372015c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:29.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f372015e5c0 0x7f3720154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:29.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f37200aeb10 con 0x7f37200b51b0 2024-05-28T14:06:29.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f37200ae990 con 0x7f372015e5c0 2024-05-28T14:06:29.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3725400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f372016c2d0 con 0x7f37200d0e00 2024-05-28T14:06:29.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f371e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37200b51b0 0x7f3720151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:29.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f371e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37200b51b0 0x7f3720151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53142/0 (socket says 172.21.15.42:53142) 2024-05-28T14:06:29.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f371e800640 1 -- 172.21.15.42:0/1663407908 learned_addr learned my addr 172.21.15.42:0/1663407908 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:29.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3717e00640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37200d0e00 0x7f372015c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:29.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f371f200640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f372015e5c0 0x7f3720154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:29.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f371e800640 1 -- 172.21.15.42:0/1663407908 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37200d0e00 msgr2=0x7f372015c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:29.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f371e800640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37200d0e00 0x7f372015c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f371e800640 1 -- 172.21.15.42:0/1663407908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f372015e5c0 msgr2=0x7f3720154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:29.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f371e800640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f372015e5c0 0x7f3720154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f371e800640 1 -- 172.21.15.42:0/1663407908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372015cf60 con 0x7f37200b51b0 2024-05-28T14:06:29.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.338+0000 7f3717e00640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37200d0e00 0x7f372015c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:29.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.342+0000 7f371e800640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37200b51b0 0x7f3720151070 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f37100ac040 tx=0x7f371001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:29.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.342+0000 7f3716a00640 1 -- 172.21.15.42:0/1663407908 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37100be610 con 0x7f37200b51b0 2024-05-28T14:06:29.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.342+0000 7f3716a00640 1 -- 172.21.15.42:0/1663407908 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37100be7d0 con 0x7f37200b51b0 2024-05-28T14:06:29.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.342+0000 7f3716a00640 1 -- 172.21.15.42:0/1663407908 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37100bebc0 con 0x7f37200b51b0 2024-05-28T14:06:29.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.342+0000 7f3725400640 1 -- 172.21.15.42:0/1663407908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372015d240 con 0x7f37200b51b0 2024-05-28T14:06:29.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.342+0000 7f3725400640 1 -- 172.21.15.42:0/1663407908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372015d800 con 0x7f37200b51b0 2024-05-28T14:06:29.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.342+0000 7f3725400640 1 -- 172.21.15.42:0/1663407908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3720068a90 con 0x7f37200b51b0 2024-05-28T14:06:29.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.342+0000 7f3716a00640 1 -- 172.21.15.42:0/1663407908 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f37100bc020 con 0x7f37200b51b0 2024-05-28T14:06:29.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.342+0000 7f3716a00640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36fc069070 0x7f36fc06b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:29.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.342+0000 7f3716a00640 1 -- 172.21.15.42:0/1663407908 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7f371011fb20 con 0x7f37200b51b0 2024-05-28T14:06:29.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.342+0000 7f3717e00640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36fc069070 0x7f36fc06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:29.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.346+0000 7f3717e00640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36fc069070 0x7f36fc06b530 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f37200cf7b0 tx=0x7f3708007c80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:29.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.346+0000 7f3716a00640 1 -- 172.21.15.42:0/1663407908 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f37100c5030 con 0x7f37200b51b0 2024-05-28T14:06:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.510+0000 7f3725400640 1 -- 172.21.15.42:0/1663407908 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37201515b0 con 0x7f36fc069070 2024-05-28T14:06:29.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.518+0000 7f3716a00640 1 -- 172.21.15.42:0/1663407908 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246753 (secure 0 0 0) 0x7f37201515b0 con 0x7f36fc069070 2024-05-28T14:06:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:29.509 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:06:29.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.522+0000 7f3725400640 1 -- 172.21.15.42:0/1663407908 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36fc069070 msgr2=0x7f36fc06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:29.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.522+0000 7f3725400640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36fc069070 0x7f36fc06b530 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f37200cf7b0 tx=0x7f3708007c80 comp rx=0 tx=0).stop 2024-05-28T14:06:29.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.522+0000 7f3725400640 1 -- 172.21.15.42:0/1663407908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37200b51b0 msgr2=0x7f3720151070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.522+0000 7f3725400640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37200b51b0 0x7f3720151070 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f37100ac040 tx=0x7f371001acb0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.522+0000 7f3725400640 1 -- 172.21.15.42:0/1663407908 shutdown_connections 2024-05-28T14:06:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.522+0000 7f3725400640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36fc069070 0x7f36fc06b530 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.522+0000 7f3725400640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f372015e5c0 0x7f3720154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.522+0000 7f3725400640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37200d0e00 0x7f372015c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.522+0000 7f3725400640 1 --2- 172.21.15.42:0/1663407908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37200b51b0 0x7f3720151070 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.522+0000 7f3725400640 1 -- 172.21.15.42:0/1663407908 >> 172.21.15.42:0/1663407908 conn(0x7f372006bdf0 msgr2=0x7f37200b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.522+0000 7f3725400640 1 -- 172.21.15.42:0/1663407908 shutdown_connections 2024-05-28T14:06:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.522+0000 7f3725400640 1 -- 172.21.15.42:0/1663407908 wait complete. 2024-05-28T14:06:29.529 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-05-28T14:06:28.995245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":451740,"kb_used_data":7972,"kb_used_omap":12,"kb_used_meta":443635,"kb_avail":754522980,"statfs":{"total":773094113280,"available":772631531520,"internally_reserved":0,"allocated":8163328,"data_stored":4134275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":454282802},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.595421"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.424413+0000","last_change":"2024-05-28T14:06:24.424413+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.424413+0000","last_undegraded":"2024-05-28T14:06:24.424413+0000","last_fullsized":"2024-05-28T14:06:24.424413+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.421572+0000","last_change":"2024-05-28T14:06:24.421572+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.421572+0000","last_undegraded":"2024-05-28T14:06:24.421572+0000","last_fullsized":"2024-05-28T14:06:24.421572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415723+0000","last_change":"2024-05-28T14:06:24.415723+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415723+0000","last_undegraded":"2024-05-28T14:06:24.415723+0000","last_fullsized":"2024-05-28T14:06:24.415723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.418596+0000","last_change":"2024-05-28T14:06:24.418596+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.418596+0000","last_undegraded":"2024-05-28T14:06:24.418596+0000","last_fullsized":"2024-05-28T14:06:24.418596+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417356+0000","last_change":"2024-05-28T14:06:24.417356+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417356+0000","last_undegraded":"2024-05-28T14:06:24.417356+0000","last_fullsized":"2024-05-28T14:06:24.417356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417815+0000","last_change":"2024-05-28T14:06:24.417815+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417815+0000","last_undegraded":"2024-05-28T14:06:24.417815+0000","last_fullsized":"2024-05-28T14:06:24.417815+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416338+0000","last_change":"2024-05-28T14:06:24.416338+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416338+0000","last_undegraded":"2024-05-28T14:06:24.416338+0000","last_fullsized":"2024-05-28T14:06:24.416338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.425317+0000","last_change":"2024-05-28T14:06:24.425317+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.425317+0000","last_undegraded":"2024-05-28T14:06:24.425317+0000","last_fullsized":"2024-05-28T14:06:24.425317+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417044+0000","last_change":"2024-05-28T14:06:24.417044+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417044+0000","last_undegraded":"2024-05-28T14:06:24.417044+0000","last_fullsized":"2024-05-28T14:06:24.417044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413819+0000","last_change":"2024-05-28T14:06:24.413819+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413819+0000","last_undegraded":"2024-05-28T14:06:24.413819+0000","last_fullsized":"2024-05-28T14:06:24.413819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.425975+0000","last_change":"2024-05-28T14:06:24.425975+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.425975+0000","last_undegraded":"2024-05-28T14:06:24.425975+0000","last_fullsized":"2024-05-28T14:06:24.425975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.418211+0000","last_change":"2024-05-28T14:06:24.418211+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.418211+0000","last_undegraded":"2024-05-28T14:06:24.418211+0000","last_fullsized":"2024-05-28T14:06:24.418211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.419930+0000","last_change":"2024-05-28T14:06:24.419930+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.419930+0000","last_undegraded":"2024-05-28T14:06:24.419930+0000","last_fullsized":"2024-05-28T14:06:24.419930+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414944+0000","last_change":"2024-05-28T14:06:24.414944+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414944+0000","last_undegraded":"2024-05-28T14:06:24.414944+0000","last_fullsized":"2024-05-28T14:06:24.414944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415671+0000","last_change":"2024-05-28T14:06:24.415671+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415671+0000","last_undegraded":"2024-05-28T14:06:24.415671+0000","last_fullsized":"2024-05-28T14:06:24.415671+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412671+0000","last_change":"2024-05-28T14:06:24.412671+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412671+0000","last_undegraded":"2024-05-28T14:06:24.412671+0000","last_fullsized":"2024-05-28T14:06:24.412671+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414216+0000","last_change":"2024-05-28T14:06:24.414216+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414216+0000","last_undegraded":"2024-05-28T14:06:24.414216+0000","last_fullsized":"2024-05-28T14:06:24.414216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412534+0000","last_change":"2024-05-28T14:06:24.412534+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412534+0000","last_undegraded":"2024-05-28T14:06:24.412534+0000","last_fullsized":"2024-05-28T14:06:24.412534+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.427380+0000","last_change":"2024-05-28T14:06:24.427380+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.427380+0000","last_undegraded":"2024-05-28T14:06:24.427380+0000","last_fullsized":"2024-05-28T14:06:24.427380+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.420242+0000","last_change":"2024-05-28T14:06:24.420242+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.420242+0000","last_undegraded":"2024-05-28T14:06:24.420242+0000","last_fullsized":"2024-05-28T14:06:24.420242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413445+0000","last_change":"2024-05-28T14:06:24.413445+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413445+0000","last_undegraded":"2024-05-28T14:06:24.413445+0000","last_fullsized":"2024-05-28T14:06:24.413445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413956+0000","last_change":"2024-05-28T14:06:24.413956+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413956+0000","last_undegraded":"2024-05-28T14:06:24.413956+0000","last_fullsized":"2024-05-28T14:06:24.413956+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.422591+0000","last_change":"2024-05-28T14:06:24.422591+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.422591+0000","last_undegraded":"2024-05-28T14:06:24.422591+0000","last_fullsized":"2024-05-28T14:06:24.422591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.421618+0000","last_change":"2024-05-28T14:06:24.421618+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.421618+0000","last_undegraded":"2024-05-28T14:06:24.421618+0000","last_fullsized":"2024-05-28T14:06:24.421618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413641+0000","last_change":"2024-05-28T14:06:24.413641+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413641+0000","last_undegraded":"2024-05-28T14:06:24.413641+0000","last_fullsized":"2024-05-28T14:06:24.413641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415922+0000","last_change":"2024-05-28T14:06:24.415922+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415922+0000","last_undegraded":"2024-05-28T14:06:24.415922+0000","last_fullsized":"2024-05-28T14:06:24.415922+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415300+0000","last_change":"2024-05-28T14:06:24.415300+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415300+0000","last_undegraded":"2024-05-28T14:06:24.415300+0000","last_fullsized":"2024-05-28T14:06:24.415300+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417303+0000","last_change":"2024-05-28T14:06:24.417303+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417303+0000","last_undegraded":"2024-05-28T14:06:24.417303+0000","last_fullsized":"2024-05-28T14:06:24.417303+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413946+0000","last_change":"2024-05-28T14:06:24.413946+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413946+0000","last_undegraded":"2024-05-28T14:06:24.413946+0000","last_fullsized":"2024-05-28T14:06:24.413946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416788+0000","last_change":"2024-05-28T14:06:24.416788+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416788+0000","last_undegraded":"2024-05-28T14:06:24.416788+0000","last_fullsized":"2024-05-28T14:06:24.416788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415329+0000","last_change":"2024-05-28T14:06:24.415329+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415329+0000","last_undegraded":"2024-05-28T14:06:24.415329+0000","last_fullsized":"2024-05-28T14:06:24.415329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416575+0000","last_change":"2024-05-28T14:06:24.416575+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416575+0000","last_undegraded":"2024-05-28T14:06:24.416575+0000","last_fullsized":"2024-05-28T14:06:24.416575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.422580+0000","last_change":"2024-05-28T14:06:24.422580+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.422580+0000","last_undegraded":"2024-05-28T14:06:24.422580+0000","last_fullsized":"2024-05-28T14:06:24.422580+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.437020+0000","last_change":"2024-05-28T14:06:24.435935+0000","last_active":"2024-05-28T14:06:24.437020+0000","last_peered":"2024-05-28T14:06:24.437020+0000","last_clean":"2024-05-28T14:06:24.437020+0000","last_became_active":"2024-05-28T14:06:24.435637+0000","last_became_peered":"2024-05-28T14:06:24.435637+0000","last_unstale":"2024-05-28T14:06:24.437020+0000","last_undegraded":"2024-05-28T14:06:24.437020+0000","last_fullsized":"2024-05-28T14:06:24.437020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:32:00.614120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413739+0000","last_change":"2024-05-28T14:06:24.413739+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413739+0000","last_undegraded":"2024-05-28T14:06:24.413739+0000","last_fullsized":"2024-05-28T14:06:24.413739+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.430746+0000","last_change":"2024-05-28T14:06:24.427568+0000","last_active":"2024-05-28T14:06:24.430746+0000","last_peered":"2024-05-28T14:06:24.430746+0000","last_clean":"2024-05-28T14:06:24.430746+0000","last_became_active":"2024-05-28T14:06:24.427288+0000","last_became_peered":"2024-05-28T14:06:24.427288+0000","last_unstale":"2024-05-28T14:06:24.430746+0000","last_undegraded":"2024-05-28T14:06:24.430746+0000","last_fullsized":"2024-05-28T14:06:24.430746+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:34:47.264602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.424448+0000","last_change":"2024-05-28T14:06:24.424448+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.424448+0000","last_undegraded":"2024-05-28T14:06:24.424448+0000","last_fullsized":"2024-05-28T14:06:24.424448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.437327+0000","last_change":"2024-05-28T14:06:24.435174+0000","last_active":"2024-05-28T14:06:24.437327+0000","last_peered":"2024-05-28T14:06:24.437327+0000","last_clean":"2024-05-28T14:06:24.437327+0000","last_became_active":"2024-05-28T14:06:24.434890+0000","last_became_peered":"2024-05-28T14:06:24.434890+0000","last_unstale":"2024-05-28T14:06:24.437327+0000","last_undegraded":"2024-05-28T14:06:24.437327+0000","last_fullsized":"2024-05-28T14:06:24.437327+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:31:55.554880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.425545+0000","last_change":"2024-05-28T14:06:24.425545+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.425545+0000","last_undegraded":"2024-05-28T14:06:24.425545+0000","last_fullsized":"2024-05-28T14:06:24.425545+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.433105+0000","last_change":"2024-05-28T14:06:24.428906+0000","last_active":"2024-05-28T14:06:24.433105+0000","last_peered":"2024-05-28T14:06:24.433105+0000","last_clean":"2024-05-28T14:06:24.433105+0000","last_became_active":"2024-05-28T14:06:24.427835+0000","last_became_peered":"2024-05-28T14:06:24.427835+0000","last_unstale":"2024-05-28T14:06:24.433105+0000","last_undegraded":"2024-05-28T14:06:24.433105+0000","last_fullsized":"2024-05-28T14:06:24.433105+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:04:56.310115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416321+0000","last_change":"2024-05-28T14:06:24.416321+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416321+0000","last_undegraded":"2024-05-28T14:06:24.416321+0000","last_fullsized":"2024-05-28T14:06:24.416321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.426845+0000","last_change":"2024-05-28T14:06:24.422900+0000","last_active":"2024-05-28T14:06:24.426845+0000","last_peered":"2024-05-28T14:06:24.426845+0000","last_clean":"2024-05-28T14:06:24.426845+0000","last_became_active":"2024-05-28T14:06:24.422439+0000","last_became_peered":"2024-05-28T14:06:24.422439+0000","last_unstale":"2024-05-28T14:06:24.426845+0000","last_undegraded":"2024-05-28T14:06:24.426845+0000","last_fullsized":"2024-05-28T14:06:24.426845+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:56:12.299871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412777+0000","last_change":"2024-05-28T14:06:24.412777+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412777+0000","last_undegraded":"2024-05-28T14:06:24.412777+0000","last_fullsized":"2024-05-28T14:06:24.412777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.433127+0000","last_change":"2024-05-28T14:06:24.431434+0000","last_active":"2024-05-28T14:06:24.433127+0000","last_peered":"2024-05-28T14:06:24.433127+0000","last_clean":"2024-05-28T14:06:24.433127+0000","last_became_active":"2024-05-28T14:06:24.430297+0000","last_became_peered":"2024-05-28T14:06:24.430297+0000","last_unstale":"2024-05-28T14:06:24.433127+0000","last_undegraded":"2024-05-28T14:06:24.433127+0000","last_fullsized":"2024-05-28T14:06:24.433127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:46:10.252003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413731+0000","last_change":"2024-05-28T14:06:24.413731+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413731+0000","last_undegraded":"2024-05-28T14:06:24.413731+0000","last_fullsized":"2024-05-28T14:06:24.413731+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.429025+0000","last_change":"2024-05-28T14:06:24.423413+0000","last_active":"2024-05-28T14:06:24.429025+0000","last_peered":"2024-05-28T14:06:24.429025+0000","last_clean":"2024-05-28T14:06:24.429025+0000","last_became_active":"2024-05-28T14:06:24.423140+0000","last_became_peered":"2024-05-28T14:06:24.423140+0000","last_unstale":"2024-05-28T14:06:24.429025+0000","last_undegraded":"2024-05-28T14:06:24.429025+0000","last_fullsized":"2024-05-28T14:06:24.429025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:37:36.072281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415270+0000","last_change":"2024-05-28T14:06:24.415270+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415270+0000","last_undegraded":"2024-05-28T14:06:24.415270+0000","last_fullsized":"2024-05-28T14:06:24.415270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.429196+0000","last_change":"2024-05-28T14:06:24.425685+0000","last_active":"2024-05-28T14:06:24.429196+0000","last_peered":"2024-05-28T14:06:24.429196+0000","last_clean":"2024-05-28T14:06:24.429196+0000","last_became_active":"2024-05-28T14:06:24.425387+0000","last_became_peered":"2024-05-28T14:06:24.425387+0000","last_unstale":"2024-05-28T14:06:24.429196+0000","last_undegraded":"2024-05-28T14:06:24.429196+0000","last_fullsized":"2024-05-28T14:06:24.429196+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:53:53.347808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415360+0000","last_change":"2024-05-28T14:06:24.415360+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415360+0000","last_undegraded":"2024-05-28T14:06:24.415360+0000","last_fullsized":"2024-05-28T14:06:24.415360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.423810+0000","last_change":"2024-05-28T14:06:24.422382+0000","last_active":"2024-05-28T14:06:24.423810+0000","last_peered":"2024-05-28T14:06:24.423810+0000","last_clean":"2024-05-28T14:06:24.423810+0000","last_became_active":"2024-05-28T14:06:24.422093+0000","last_became_peered":"2024-05-28T14:06:24.422093+0000","last_unstale":"2024-05-28T14:06:24.423810+0000","last_undegraded":"2024-05-28T14:06:24.423810+0000","last_fullsized":"2024-05-28T14:06:24.423810+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:58:13.946282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.423794+0000","last_change":"2024-05-28T14:06:24.423794+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.423794+0000","last_undegraded":"2024-05-28T14:06:24.423794+0000","last_fullsized":"2024-05-28T14:06:24.423794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.438284+0000","last_change":"2024-05-28T14:06:24.436081+0000","last_active":"2024-05-28T14:06:24.438284+0000","last_peered":"2024-05-28T14:06:24.438284+0000","last_clean":"2024-05-28T14:06:24.438284+0000","last_became_active":"2024-05-28T14:06:24.435796+0000","last_became_peered":"2024-05-28T14:06:24.435796+0000","last_unstale":"2024-05-28T14:06:24.438284+0000","last_undegraded":"2024-05-28T14:06:24.438284+0000","last_fullsized":"2024-05-28T14:06:24.438284+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:37:36.072281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.423969+0000","last_change":"2024-05-28T14:06:24.423969+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.423969+0000","last_undegraded":"2024-05-28T14:06:24.423969+0000","last_fullsized":"2024-05-28T14:06:24.423969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.426613+0000","last_change":"2024-05-28T14:06:24.421989+0000","last_active":"2024-05-28T14:06:24.426613+0000","last_peered":"2024-05-28T14:06:24.426613+0000","last_clean":"2024-05-28T14:06:24.426613+0000","last_became_active":"2024-05-28T14:06:24.421727+0000","last_became_peered":"2024-05-28T14:06:24.421727+0000","last_unstale":"2024-05-28T14:06:24.426613+0000","last_undegraded":"2024-05-28T14:06:24.426613+0000","last_fullsized":"2024-05-28T14:06:24.426613+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:35:55.179829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413607+0000","last_change":"2024-05-28T14:06:24.413607+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413607+0000","last_undegraded":"2024-05-28T14:06:24.413607+0000","last_fullsized":"2024-05-28T14:06:24.413607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.423031+0000","last_change":"2024-05-28T14:06:24.421197+0000","last_active":"2024-05-28T14:06:24.423031+0000","last_peered":"2024-05-28T14:06:24.423031+0000","last_clean":"2024-05-28T14:06:24.423031+0000","last_became_active":"2024-05-28T14:06:24.420925+0000","last_became_peered":"2024-05-28T14:06:24.420925+0000","last_unstale":"2024-05-28T14:06:24.423031+0000","last_undegraded":"2024-05-28T14:06:24.423031+0000","last_fullsized":"2024-05-28T14:06:24.423031+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:04:37.401131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.423713+0000","last_change":"2024-05-28T14:06:24.423713+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.423713+0000","last_undegraded":"2024-05-28T14:06:24.423713+0000","last_fullsized":"2024-05-28T14:06:24.423713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431171+0000","last_change":"2024-05-28T14:06:24.429639+0000","last_active":"2024-05-28T14:06:24.431171+0000","last_peered":"2024-05-28T14:06:24.431171+0000","last_clean":"2024-05-28T14:06:24.431171+0000","last_became_active":"2024-05-28T14:06:24.429133+0000","last_became_peered":"2024-05-28T14:06:24.429133+0000","last_unstale":"2024-05-28T14:06:24.431171+0000","last_undegraded":"2024-05-28T14:06:24.431171+0000","last_fullsized":"2024-05-28T14:06:24.431171+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:26.316274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415570+0000","last_change":"2024-05-28T14:06:24.415570+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415570+0000","last_undegraded":"2024-05-28T14:06:24.415570+0000","last_fullsized":"2024-05-28T14:06:24.415570+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.435344+0000","last_change":"2024-05-28T14:06:24.432341+0000","last_active":"2024-05-28T14:06:24.435344+0000","last_peered":"2024-05-28T14:06:24.435344+0000","last_clean":"2024-05-28T14:06:24.435344+0000","last_became_active":"2024-05-28T14:06:24.432058+0000","last_became_peered":"2024-05-28T14:06:24.432058+0000","last_unstale":"2024-05-28T14:06:24.435344+0000","last_undegraded":"2024-05-28T14:06:24.435344+0000","last_fullsized":"2024-05-28T14:06:24.435344+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:59:39.619307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414289+0000","last_change":"2024-05-28T14:06:24.414289+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414289+0000","last_undegraded":"2024-05-28T14:06:24.414289+0000","last_fullsized":"2024-05-28T14:06:24.414289+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431568+0000","last_change":"2024-05-28T14:06:24.426720+0000","last_active":"2024-05-28T14:06:24.431568+0000","last_peered":"2024-05-28T14:06:24.431568+0000","last_clean":"2024-05-28T14:06:24.431568+0000","last_became_active":"2024-05-28T14:06:24.426399+0000","last_became_peered":"2024-05-28T14:06:24.426399+0000","last_unstale":"2024-05-28T14:06:24.431568+0000","last_undegraded":"2024-05-28T14:06:24.431568+0000","last_fullsized":"2024-05-28T14:06:24.431568+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:26:45.081785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414452+0000","last_change":"2024-05-28T14:06:24.414452+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414452+0000","last_undegraded":"2024-05-28T14:06:24.414452+0000","last_fullsized":"2024-05-28T14:06:24.414452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431522+0000","last_change":"2024-05-28T14:06:24.424772+0000","last_active":"2024-05-28T14:06:24.431522+0000","last_peered":"2024-05-28T14:06:24.431522+0000","last_clean":"2024-05-28T14:06:24.431522+0000","last_became_active":"2024-05-28T14:06:24.424480+0000","last_became_peered":"2024-05-28T14:06:24.424480+0000","last_unstale":"2024-05-28T14:06:24.431522+0000","last_undegraded":"2024-05-28T14:06:24.431522+0000","last_fullsized":"2024-05-28T14:06:24.431522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:45:23.593514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413039+0000","last_change":"2024-05-28T14:06:24.413039+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413039+0000","last_undegraded":"2024-05-28T14:06:24.413039+0000","last_fullsized":"2024-05-28T14:06:24.413039+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.437107+0000","last_change":"2024-05-28T14:06:24.434783+0000","last_active":"2024-05-28T14:06:24.437107+0000","last_peered":"2024-05-28T14:06:24.437107+0000","last_clean":"2024-05-28T14:06:24.437107+0000","last_became_active":"2024-05-28T14:06:24.434490+0000","last_became_peered":"2024-05-28T14:06:24.434490+0000","last_unstale":"2024-05-28T14:06:24.437107+0000","last_undegraded":"2024-05-28T14:06:24.437107+0000","last_fullsized":"2024-05-28T14:06:24.437107+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:46:01.970253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.423255+0000","last_change":"2024-05-28T14:06:24.423255+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.423255+0000","last_undegraded":"2024-05-28T14:06:24.423255+0000","last_fullsized":"2024-05-28T14:06:24.423255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.437420+0000","last_change":"2024-05-28T14:06:24.434447+0000","last_active":"2024-05-28T14:06:24.437420+0000","last_peered":"2024-05-28T14:06:24.437420+0000","last_clean":"2024-05-28T14:06:24.437420+0000","last_became_active":"2024-05-28T14:06:24.433002+0000","last_became_peered":"2024-05-28T14:06:24.433002+0000","last_unstale":"2024-05-28T14:06:24.437420+0000","last_undegraded":"2024-05-28T14:06:24.437420+0000","last_fullsized":"2024-05-28T14:06:24.437420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:41:19.574984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412440+0000","last_change":"2024-05-28T14:06:24.412440+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412440+0000","last_undegraded":"2024-05-28T14:06:24.412440+0000","last_fullsized":"2024-05-28T14:06:24.412440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.432802+0000","last_change":"2024-05-28T14:06:24.428701+0000","last_active":"2024-05-28T14:06:24.432802+0000","last_peered":"2024-05-28T14:06:24.432802+0000","last_clean":"2024-05-28T14:06:24.432802+0000","last_became_active":"2024-05-28T14:06:24.428477+0000","last_became_peered":"2024-05-28T14:06:24.428477+0000","last_unstale":"2024-05-28T14:06:24.432802+0000","last_undegraded":"2024-05-28T14:06:24.432802+0000","last_fullsized":"2024-05-28T14:06:24.432802+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:38:32.586189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412600+0000","last_change":"2024-05-28T14:06:24.412600+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412600+0000","last_undegraded":"2024-05-28T14:06:24.412600+0000","last_fullsized":"2024-05-28T14:06:24.412600+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.438463+0000","last_change":"2024-05-28T14:06:24.437928+0000","last_active":"2024-05-28T14:06:24.438463+0000","last_peered":"2024-05-28T14:06:24.438463+0000","last_clean":"2024-05-28T14:06:24.438463+0000","last_became_active":"2024-05-28T14:06:24.437366+0000","last_became_peered":"2024-05-28T14:06:24.437366+0000","last_unstale":"2024-05-28T14:06:24.438463+0000","last_undegraded":"2024-05-28T14:06:24.438463+0000","last_fullsized":"2024-05-28T14:06:24.438463+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:08.591204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413482+0000","last_change":"2024-05-28T14:06:24.413482+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413482+0000","last_undegraded":"2024-05-28T14:06:24.413482+0000","last_fullsized":"2024-05-28T14:06:24.413482+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.432994+0000","last_change":"2024-05-28T14:06:24.429400+0000","last_active":"2024-05-28T14:06:24.432994+0000","last_peered":"2024-05-28T14:06:24.432994+0000","last_clean":"2024-05-28T14:06:24.432994+0000","last_became_active":"2024-05-28T14:06:24.429124+0000","last_became_peered":"2024-05-28T14:06:24.429124+0000","last_unstale":"2024-05-28T14:06:24.432994+0000","last_undegraded":"2024-05-28T14:06:24.432994+0000","last_fullsized":"2024-05-28T14:06:24.432994+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:39:32.793666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414095+0000","last_change":"2024-05-28T14:06:24.414095+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414095+0000","last_undegraded":"2024-05-28T14:06:24.414095+0000","last_fullsized":"2024-05-28T14:06:24.414095+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.422911+0000","last_change":"2024-05-28T14:06:24.420613+0000","last_active":"2024-05-28T14:06:24.422911+0000","last_peered":"2024-05-28T14:06:24.422911+0000","last_clean":"2024-05-28T14:06:24.422911+0000","last_became_active":"2024-05-28T14:06:24.420171+0000","last_became_peered":"2024-05-28T14:06:24.420171+0000","last_unstale":"2024-05-28T14:06:24.422911+0000","last_undegraded":"2024-05-28T14:06:24.422911+0000","last_fullsized":"2024-05-28T14:06:24.422911+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:49.326481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414920+0000","last_change":"2024-05-28T14:06:24.414920+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414920+0000","last_undegraded":"2024-05-28T14:06:24.414920+0000","last_fullsized":"2024-05-28T14:06:24.414920+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.430690+0000","last_change":"2024-05-28T14:06:24.427464+0000","last_active":"2024-05-28T14:06:24.430690+0000","last_peered":"2024-05-28T14:06:24.430690+0000","last_clean":"2024-05-28T14:06:24.430690+0000","last_became_active":"2024-05-28T14:06:24.427145+0000","last_became_peered":"2024-05-28T14:06:24.427145+0000","last_unstale":"2024-05-28T14:06:24.430690+0000","last_undegraded":"2024-05-28T14:06:24.430690+0000","last_fullsized":"2024-05-28T14:06:24.430690+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:06:23.658146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417284+0000","last_change":"2024-05-28T14:06:24.417284+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417284+0000","last_undegraded":"2024-05-28T14:06:24.417284+0000","last_fullsized":"2024-05-28T14:06:24.417284+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.436513+0000","last_change":"2024-05-28T14:06:24.432369+0000","last_active":"2024-05-28T14:06:24.436513+0000","last_peered":"2024-05-28T14:06:24.436513+0000","last_clean":"2024-05-28T14:06:24.436513+0000","last_became_active":"2024-05-28T14:06:24.431992+0000","last_became_peered":"2024-05-28T14:06:24.431992+0000","last_unstale":"2024-05-28T14:06:24.436513+0000","last_undegraded":"2024-05-28T14:06:24.436513+0000","last_fullsized":"2024-05-28T14:06:24.436513+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:42:55.217345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416075+0000","last_change":"2024-05-28T14:06:24.416075+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416075+0000","last_undegraded":"2024-05-28T14:06:24.416075+0000","last_fullsized":"2024-05-28T14:06:24.416075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431126+0000","last_change":"2024-05-28T14:06:24.427952+0000","last_active":"2024-05-28T14:06:24.431126+0000","last_peered":"2024-05-28T14:06:24.431126+0000","last_clean":"2024-05-28T14:06:24.431126+0000","last_became_active":"2024-05-28T14:06:24.427511+0000","last_became_peered":"2024-05-28T14:06:24.427511+0000","last_unstale":"2024-05-28T14:06:24.431126+0000","last_undegraded":"2024-05-28T14:06:24.431126+0000","last_fullsized":"2024-05-28T14:06:24.431126+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:03:34.586234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.425338+0000","last_change":"2024-05-28T14:06:24.425338+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.425338+0000","last_undegraded":"2024-05-28T14:06:24.425338+0000","last_fullsized":"2024-05-28T14:06:24.425338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.430060+0000","last_change":"2024-05-28T14:06:24.427264+0000","last_active":"2024-05-28T14:06:24.430060+0000","last_peered":"2024-05-28T14:06:24.430060+0000","last_clean":"2024-05-28T14:06:24.430060+0000","last_became_active":"2024-05-28T14:06:24.426662+0000","last_became_peered":"2024-05-28T14:06:24.426662+0000","last_unstale":"2024-05-28T14:06:24.430060+0000","last_undegraded":"2024-05-28T14:06:24.430060+0000","last_fullsized":"2024-05-28T14:06:24.430060+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:28:56.343364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.424918+0000","last_change":"2024-05-28T14:06:24.424918+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.424918+0000","last_undegraded":"2024-05-28T14:06:24.424918+0000","last_fullsized":"2024-05-28T14:06:24.424918+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.432299+0000","last_change":"2024-05-28T14:06:24.431741+0000","last_active":"2024-05-28T14:06:24.432299+0000","last_peered":"2024-05-28T14:06:24.432299+0000","last_clean":"2024-05-28T14:06:24.432299+0000","last_became_active":"2024-05-28T14:06:24.431388+0000","last_became_peered":"2024-05-28T14:06:24.431388+0000","last_unstale":"2024-05-28T14:06:24.432299+0000","last_undegraded":"2024-05-28T14:06:24.432299+0000","last_fullsized":"2024-05-28T14:06:24.432299+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:11:06.256080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412880+0000","last_change":"2024-05-28T14:06:24.412880+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412880+0000","last_undegraded":"2024-05-28T14:06:24.412880+0000","last_fullsized":"2024-05-28T14:06:24.412880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.424536+0000","last_change":"2024-05-28T14:06:24.420374+0000","last_active":"2024-05-28T14:06:24.424536+0000","last_peered":"2024-05-28T14:06:24.424536+0000","last_clean":"2024-05-28T14:06:24.424536+0000","last_became_active":"2024-05-28T14:06:24.420044+0000","last_became_peered":"2024-05-28T14:06:24.420044+0000","last_unstale":"2024-05-28T14:06:24.424536+0000","last_undegraded":"2024-05-28T14:06:24.424536+0000","last_fullsized":"2024-05-28T14:06:24.424536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:29:46.118850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.422978+0000","last_change":"2024-05-28T14:06:24.422978+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.422978+0000","last_undegraded":"2024-05-28T14:06:24.422978+0000","last_fullsized":"2024-05-28T14:06:24.422978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.425967+0000","last_change":"2024-05-28T14:06:24.422804+0000","last_active":"2024-05-28T14:06:24.425967+0000","last_peered":"2024-05-28T14:06:24.425967+0000","last_clean":"2024-05-28T14:06:24.425967+0000","last_became_active":"2024-05-28T14:06:24.422468+0000","last_became_peered":"2024-05-28T14:06:24.422468+0000","last_unstale":"2024-05-28T14:06:24.425967+0000","last_undegraded":"2024-05-28T14:06:24.425967+0000","last_fullsized":"2024-05-28T14:06:24.425967+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:46:01.970253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412444+0000","last_change":"2024-05-28T14:06:24.412444+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412444+0000","last_undegraded":"2024-05-28T14:06:24.412444+0000","last_fullsized":"2024-05-28T14:06:24.412444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.433548+0000","last_change":"2024-05-28T14:06:24.429508+0000","last_active":"2024-05-28T14:06:24.433548+0000","last_peered":"2024-05-28T14:06:24.433548+0000","last_clean":"2024-05-28T14:06:24.433548+0000","last_became_active":"2024-05-28T14:06:24.429101+0000","last_became_peered":"2024-05-28T14:06:24.429101+0000","last_unstale":"2024-05-28T14:06:24.433548+0000","last_undegraded":"2024-05-28T14:06:24.433548+0000","last_fullsized":"2024-05-28T14:06:24.433548+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:15.013592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417313+0000","last_change":"2024-05-28T14:06:24.417313+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417313+0000","last_undegraded":"2024-05-28T14:06:24.417313+0000","last_fullsized":"2024-05-28T14:06:24.417313+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431063+0000","last_change":"2024-05-28T14:06:24.427387+0000","last_active":"2024-05-28T14:06:24.431063+0000","last_peered":"2024-05-28T14:06:24.431063+0000","last_clean":"2024-05-28T14:06:24.431063+0000","last_became_active":"2024-05-28T14:06:24.427106+0000","last_became_peered":"2024-05-28T14:06:24.427106+0000","last_unstale":"2024-05-28T14:06:24.431063+0000","last_undegraded":"2024-05-28T14:06:24.431063+0000","last_fullsized":"2024-05-28T14:06:24.431063+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:19:02.178759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414347+0000","last_change":"2024-05-28T14:06:24.414347+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414347+0000","last_undegraded":"2024-05-28T14:06:24.414347+0000","last_fullsized":"2024-05-28T14:06:24.414347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.436299+0000","last_change":"2024-05-28T14:06:24.433815+0000","last_active":"2024-05-28T14:06:24.436299+0000","last_peered":"2024-05-28T14:06:24.436299+0000","last_clean":"2024-05-28T14:06:24.436299+0000","last_became_active":"2024-05-28T14:06:24.433524+0000","last_became_peered":"2024-05-28T14:06:24.433524+0000","last_unstale":"2024-05-28T14:06:24.436299+0000","last_undegraded":"2024-05-28T14:06:24.436299+0000","last_fullsized":"2024-05-28T14:06:24.436299+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:04:37.401131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":264,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.429488+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:06:24.429488+0000","last_peered":"2024-05-28T14:06:24.429488+0000","last_clean":"2024-05-28T14:06:24.429488+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:06:24.429488+0000","last_undegraded":"2024-05-28T14:06:24.429488+0000","last_fullsized":"2024-05-28T14:06:24.429488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574949,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58988,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312852,"statfs":{"total":96636764160,"available":96576360448,"internally_reserved":0,"allocated":1470464,"data_stored":961924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54236,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317604,"statfs":{"total":96636764160,"available":96581226496,"internally_reserved":0,"allocated":864256,"data_stored":365253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54248,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317592,"statfs":{"total":96636764160,"available":96581214208,"internally_reserved":0,"allocated":876544,"data_stored":371556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58392,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313448,"statfs":{"total":96636764160,"available":96576970752,"internally_reserved":0,"allocated":864256,"data_stored":365253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45928,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325912,"statfs":{"total":96636764160,"available":96589733888,"internally_reserved":0,"allocated":876544,"data_stored":371556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54236,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317604,"statfs":{"total":96636764160,"available":96581226496,"internally_reserved":0,"allocated":864256,"data_stored":365253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62564,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309276,"statfs":{"total":96636764160,"available":96572698624,"internally_reserved":0,"allocated":876544,"data_stored":371556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574950,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63148,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308692,"statfs":{"total":96636764160,"available":96572100608,"internally_reserved":0,"allocated":1470464,"data_stored":961924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:06:29.534 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:06:29.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f6470800640 1 Processor -- start 2024-05-28T14:06:29.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f6470800640 1 -- start start 2024-05-28T14:06:29.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f6470800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f646815c990 0x7f64680d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:29.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f6470800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f646809bbd0 0x7f646809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:29.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f6470800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64680cd300 con 0x7f64680d06a0 2024-05-28T14:06:29.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f6470800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f646815cd70 con 0x7f646815c990 2024-05-28T14:06:29.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f6470800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f64680b63f0 con 0x7f646809bbd0 2024-05-28T14:06:29.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f646de00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f646815c990 0x7f64680d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:29.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f646d400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f646809bbd0 0x7f646809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:29.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f646de00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f646815c990 0x7f64680d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:50562/0 (socket says 172.21.15.42:50562) 2024-05-28T14:06:29.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f646de00640 1 -- 172.21.15.42:0/1476817042 learned_addr learned my addr 172.21.15.42:0/1476817042 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:29.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f646d400640 1 -- 172.21.15.42:0/1476817042 >> v2:172.21.15.63:3300/0 conn(0x7f646815c990 msgr2=0x7f64680d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:29.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f646d400640 1 --2- 172.21.15.42:0/1476817042 >> v2:172.21.15.63:3300/0 conn(0x7f646815c990 0x7f64680d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f646d400640 1 -- 172.21.15.42:0/1476817042 >> v1:172.21.15.63:6789/0 conn(0x7f64680d06a0 legacy=0x7f6468160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.730+0000 7f646d400640 1 -- 172.21.15.42:0/1476817042 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64680cd5e0 con 0x7f646809bbd0 2024-05-28T14:06:29.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.734+0000 7f646d400640 1 --2- 172.21.15.42:0/1476817042 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f646809bbd0 0x7f646809aa50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f64580097c0 tx=0x7f645801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3797b1ecb1e671d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.734+0000 7f646ca00640 1 -- 172.21.15.42:0/1476817042 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64580b2070 con 0x7f646809bbd0 2024-05-28T14:06:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.734+0000 7f646ca00640 1 -- 172.21.15.42:0/1476817042 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64580ad680 con 0x7f646809bbd0 2024-05-28T14:06:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.734+0000 7f646ca00640 1 -- 172.21.15.42:0/1476817042 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64580ada50 con 0x7f646809bbd0 2024-05-28T14:06:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.734+0000 7f6470800640 1 -- 172.21.15.42:0/1476817042 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f646809bbd0 msgr2=0x7f646809aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.734+0000 7f6470800640 1 --2- 172.21.15.42:0/1476817042 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f646809bbd0 0x7f646809aa50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f64580097c0 tx=0x7f645801c470 comp rx=0 tx=0).stop 2024-05-28T14:06:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.734+0000 7f6470800640 1 -- 172.21.15.42:0/1476817042 shutdown_connections 2024-05-28T14:06:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.734+0000 7f6470800640 1 --2- 172.21.15.42:0/1476817042 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f646809bbd0 0x7f646809aa50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.734+0000 7f6470800640 1 --2- 172.21.15.42:0/1476817042 >> v2:172.21.15.63:3300/0 conn(0x7f646815c990 0x7f64680d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.734+0000 7f6470800640 1 -- 172.21.15.42:0/1476817042 >> 172.21.15.42:0/1476817042 conn(0x7f64680b4080 msgr2=0x7f64680b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.734+0000 7f6470800640 1 -- 172.21.15.42:0/1476817042 shutdown_connections 2024-05-28T14:06:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.734+0000 7f6470800640 1 -- 172.21.15.42:0/1476817042 wait complete. 2024-05-28T14:06:29.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f6470800640 1 Processor -- start 2024-05-28T14:06:29.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f6470800640 1 -- start start 2024-05-28T14:06:29.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f6470800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64680d06a0 0x7f6468159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:29.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f6470800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f646815c990 0x7f64680c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:29.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f6470800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64680ca270 0x7f646806db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f6470800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f64680ce3a0 con 0x7f646815c990 2024-05-28T14:06:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f6470800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6468169740 con 0x7f64680d06a0 2024-05-28T14:06:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f6470800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f64680c9de0 con 0x7f64680ca270 2024-05-28T14:06:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f646d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f646815c990 0x7f64680c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f646d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f646815c990 0x7f64680c8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53148/0 (socket says 172.21.15.42:53148) 2024-05-28T14:06:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f646d400640 1 -- 172.21.15.42:0/1238475195 learned_addr learned my addr 172.21.15.42:0/1238475195 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f646de00640 1 --2- 172.21.15.42:0/1238475195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64680d06a0 0x7f6468159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f646e800640 1 --2- 172.21.15.42:0/1238475195 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64680ca270 0x7f646806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f646d400640 1 -- 172.21.15.42:0/1238475195 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64680ca270 msgr2=0x7f646806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f646d400640 1 --2- 172.21.15.42:0/1238475195 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64680ca270 0x7f646806db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f646d400640 1 -- 172.21.15.42:0/1238475195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64680d06a0 msgr2=0x7f6468159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f646d400640 1 --2- 172.21.15.42:0/1238475195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64680d06a0 0x7f6468159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f646d400640 1 -- 172.21.15.42:0/1238475195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6468071120 con 0x7f646815c990 2024-05-28T14:06:29.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f646d400640 1 --2- 172.21.15.42:0/1238475195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f646815c990 0x7f64680c8550 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f645801a0a0 tx=0x7f645801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:29.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f6462000640 1 -- 172.21.15.42:0/1238475195 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64580b2040 con 0x7f646815c990 2024-05-28T14:06:29.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f6462000640 1 -- 172.21.15.42:0/1238475195 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64580be070 con 0x7f646815c990 2024-05-28T14:06:29.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f6462000640 1 -- 172.21.15.42:0/1238475195 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64580add50 con 0x7f646815c990 2024-05-28T14:06:29.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f6470800640 1 -- 172.21.15.42:0/1238475195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646815a3b0 con 0x7f646815c990 2024-05-28T14:06:29.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f6470800640 1 -- 172.21.15.42:0/1238475195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64680c9a70 con 0x7f646815c990 2024-05-28T14:06:29.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.738+0000 7f6470800640 1 -- 172.21.15.42:0/1238475195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6468068a90 con 0x7f646815c990 2024-05-28T14:06:29.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.742+0000 7f6462000640 1 -- 172.21.15.42:0/1238475195 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f64580c3460 con 0x7f646815c990 2024-05-28T14:06:29.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.742+0000 7f6462000640 1 --2- 172.21.15.42:0/1238475195 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f644c0690a0 0x7f644c06b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:29.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.742+0000 7f6462000640 1 -- 172.21.15.42:0/1238475195 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7f645811f830 con 0x7f646815c990 2024-05-28T14:06:29.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.742+0000 7f646de00640 1 --2- 172.21.15.42:0/1238475195 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f644c0690a0 0x7f644c06b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:29.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.742+0000 7f646de00640 1 --2- 172.21.15.42:0/1238475195 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f644c0690a0 0x7f644c06b560 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f645c009ab0 tx=0x7f645c009240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.746+0000 7f6462000640 1 -- 172.21.15.42:0/1238475195 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6458016090 con 0x7f646815c990 2024-05-28T14:06:29.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.906+0000 7f6470800640 1 -- 172.21.15.42:0/1238475195 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64680b4ce0 con 0x7f644c0690a0 2024-05-28T14:06:29.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.914+0000 7f6462000640 1 -- 172.21.15.42:0/1238475195 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246753 (secure 0 0 0) 0x7f64680b4ce0 con 0x7f644c0690a0 2024-05-28T14:06:29.904 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:06:29.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.918+0000 7f6470800640 1 -- 172.21.15.42:0/1238475195 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f644c0690a0 msgr2=0x7f644c06b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.918+0000 7f6470800640 1 --2- 172.21.15.42:0/1238475195 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f644c0690a0 0x7f644c06b560 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f645c009ab0 tx=0x7f645c009240 comp rx=0 tx=0).stop 2024-05-28T14:06:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.918+0000 7f6470800640 1 -- 172.21.15.42:0/1238475195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f646815c990 msgr2=0x7f64680c8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.918+0000 7f6470800640 1 --2- 172.21.15.42:0/1238475195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f646815c990 0x7f64680c8550 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f645801a0a0 tx=0x7f645801acb0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.918+0000 7f6470800640 1 -- 172.21.15.42:0/1238475195 shutdown_connections 2024-05-28T14:06:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.918+0000 7f6470800640 1 --2- 172.21.15.42:0/1238475195 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f644c0690a0 0x7f644c06b560 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.918+0000 7f6470800640 1 --2- 172.21.15.42:0/1238475195 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64680ca270 0x7f646806db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.918+0000 7f6470800640 1 --2- 172.21.15.42:0/1238475195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f646815c990 0x7f64680c8550 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.918+0000 7f6470800640 1 --2- 172.21.15.42:0/1238475195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64680d06a0 0x7f6468159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.918+0000 7f6470800640 1 -- 172.21.15.42:0/1238475195 >> 172.21.15.42:0/1238475195 conn(0x7f64680b4080 msgr2=0x7f64680b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.918+0000 7f6470800640 1 -- 172.21.15.42:0/1238475195 shutdown_connections 2024-05-28T14:06:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:29.918+0000 7f6470800640 1 -- 172.21.15.42:0/1238475195 wait complete. 2024-05-28T14:06:29.925 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-05-28T14:06:28.995245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":451740,"kb_used_data":7972,"kb_used_omap":12,"kb_used_meta":443635,"kb_avail":754522980,"statfs":{"total":773094113280,"available":772631531520,"internally_reserved":0,"allocated":8163328,"data_stored":4134275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":454282802},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.595421"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.424413+0000","last_change":"2024-05-28T14:06:24.424413+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.424413+0000","last_undegraded":"2024-05-28T14:06:24.424413+0000","last_fullsized":"2024-05-28T14:06:24.424413+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.421572+0000","last_change":"2024-05-28T14:06:24.421572+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.421572+0000","last_undegraded":"2024-05-28T14:06:24.421572+0000","last_fullsized":"2024-05-28T14:06:24.421572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415723+0000","last_change":"2024-05-28T14:06:24.415723+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415723+0000","last_undegraded":"2024-05-28T14:06:24.415723+0000","last_fullsized":"2024-05-28T14:06:24.415723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.418596+0000","last_change":"2024-05-28T14:06:24.418596+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.418596+0000","last_undegraded":"2024-05-28T14:06:24.418596+0000","last_fullsized":"2024-05-28T14:06:24.418596+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417356+0000","last_change":"2024-05-28T14:06:24.417356+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417356+0000","last_undegraded":"2024-05-28T14:06:24.417356+0000","last_fullsized":"2024-05-28T14:06:24.417356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417815+0000","last_change":"2024-05-28T14:06:24.417815+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417815+0000","last_undegraded":"2024-05-28T14:06:24.417815+0000","last_fullsized":"2024-05-28T14:06:24.417815+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416338+0000","last_change":"2024-05-28T14:06:24.416338+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416338+0000","last_undegraded":"2024-05-28T14:06:24.416338+0000","last_fullsized":"2024-05-28T14:06:24.416338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.425317+0000","last_change":"2024-05-28T14:06:24.425317+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.425317+0000","last_undegraded":"2024-05-28T14:06:24.425317+0000","last_fullsized":"2024-05-28T14:06:24.425317+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417044+0000","last_change":"2024-05-28T14:06:24.417044+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417044+0000","last_undegraded":"2024-05-28T14:06:24.417044+0000","last_fullsized":"2024-05-28T14:06:24.417044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413819+0000","last_change":"2024-05-28T14:06:24.413819+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413819+0000","last_undegraded":"2024-05-28T14:06:24.413819+0000","last_fullsized":"2024-05-28T14:06:24.413819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.425975+0000","last_change":"2024-05-28T14:06:24.425975+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.425975+0000","last_undegraded":"2024-05-28T14:06:24.425975+0000","last_fullsized":"2024-05-28T14:06:24.425975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.418211+0000","last_change":"2024-05-28T14:06:24.418211+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.418211+0000","last_undegraded":"2024-05-28T14:06:24.418211+0000","last_fullsized":"2024-05-28T14:06:24.418211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.419930+0000","last_change":"2024-05-28T14:06:24.419930+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.419930+0000","last_undegraded":"2024-05-28T14:06:24.419930+0000","last_fullsized":"2024-05-28T14:06:24.419930+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414944+0000","last_change":"2024-05-28T14:06:24.414944+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414944+0000","last_undegraded":"2024-05-28T14:06:24.414944+0000","last_fullsized":"2024-05-28T14:06:24.414944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415671+0000","last_change":"2024-05-28T14:06:24.415671+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415671+0000","last_undegraded":"2024-05-28T14:06:24.415671+0000","last_fullsized":"2024-05-28T14:06:24.415671+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412671+0000","last_change":"2024-05-28T14:06:24.412671+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412671+0000","last_undegraded":"2024-05-28T14:06:24.412671+0000","last_fullsized":"2024-05-28T14:06:24.412671+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414216+0000","last_change":"2024-05-28T14:06:24.414216+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414216+0000","last_undegraded":"2024-05-28T14:06:24.414216+0000","last_fullsized":"2024-05-28T14:06:24.414216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412534+0000","last_change":"2024-05-28T14:06:24.412534+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412534+0000","last_undegraded":"2024-05-28T14:06:24.412534+0000","last_fullsized":"2024-05-28T14:06:24.412534+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.427380+0000","last_change":"2024-05-28T14:06:24.427380+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.427380+0000","last_undegraded":"2024-05-28T14:06:24.427380+0000","last_fullsized":"2024-05-28T14:06:24.427380+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.420242+0000","last_change":"2024-05-28T14:06:24.420242+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.420242+0000","last_undegraded":"2024-05-28T14:06:24.420242+0000","last_fullsized":"2024-05-28T14:06:24.420242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413445+0000","last_change":"2024-05-28T14:06:24.413445+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413445+0000","last_undegraded":"2024-05-28T14:06:24.413445+0000","last_fullsized":"2024-05-28T14:06:24.413445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413956+0000","last_change":"2024-05-28T14:06:24.413956+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413956+0000","last_undegraded":"2024-05-28T14:06:24.413956+0000","last_fullsized":"2024-05-28T14:06:24.413956+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.422591+0000","last_change":"2024-05-28T14:06:24.422591+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.422591+0000","last_undegraded":"2024-05-28T14:06:24.422591+0000","last_fullsized":"2024-05-28T14:06:24.422591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.421618+0000","last_change":"2024-05-28T14:06:24.421618+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.421618+0000","last_undegraded":"2024-05-28T14:06:24.421618+0000","last_fullsized":"2024-05-28T14:06:24.421618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413641+0000","last_change":"2024-05-28T14:06:24.413641+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413641+0000","last_undegraded":"2024-05-28T14:06:24.413641+0000","last_fullsized":"2024-05-28T14:06:24.413641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415922+0000","last_change":"2024-05-28T14:06:24.415922+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415922+0000","last_undegraded":"2024-05-28T14:06:24.415922+0000","last_fullsized":"2024-05-28T14:06:24.415922+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415300+0000","last_change":"2024-05-28T14:06:24.415300+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415300+0000","last_undegraded":"2024-05-28T14:06:24.415300+0000","last_fullsized":"2024-05-28T14:06:24.415300+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417303+0000","last_change":"2024-05-28T14:06:24.417303+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417303+0000","last_undegraded":"2024-05-28T14:06:24.417303+0000","last_fullsized":"2024-05-28T14:06:24.417303+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413946+0000","last_change":"2024-05-28T14:06:24.413946+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413946+0000","last_undegraded":"2024-05-28T14:06:24.413946+0000","last_fullsized":"2024-05-28T14:06:24.413946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416788+0000","last_change":"2024-05-28T14:06:24.416788+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416788+0000","last_undegraded":"2024-05-28T14:06:24.416788+0000","last_fullsized":"2024-05-28T14:06:24.416788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415329+0000","last_change":"2024-05-28T14:06:24.415329+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415329+0000","last_undegraded":"2024-05-28T14:06:24.415329+0000","last_fullsized":"2024-05-28T14:06:24.415329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416575+0000","last_change":"2024-05-28T14:06:24.416575+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416575+0000","last_undegraded":"2024-05-28T14:06:24.416575+0000","last_fullsized":"2024-05-28T14:06:24.416575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.422580+0000","last_change":"2024-05-28T14:06:24.422580+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.422580+0000","last_undegraded":"2024-05-28T14:06:24.422580+0000","last_fullsized":"2024-05-28T14:06:24.422580+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.437020+0000","last_change":"2024-05-28T14:06:24.435935+0000","last_active":"2024-05-28T14:06:24.437020+0000","last_peered":"2024-05-28T14:06:24.437020+0000","last_clean":"2024-05-28T14:06:24.437020+0000","last_became_active":"2024-05-28T14:06:24.435637+0000","last_became_peered":"2024-05-28T14:06:24.435637+0000","last_unstale":"2024-05-28T14:06:24.437020+0000","last_undegraded":"2024-05-28T14:06:24.437020+0000","last_fullsized":"2024-05-28T14:06:24.437020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:32:00.614120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413739+0000","last_change":"2024-05-28T14:06:24.413739+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413739+0000","last_undegraded":"2024-05-28T14:06:24.413739+0000","last_fullsized":"2024-05-28T14:06:24.413739+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.430746+0000","last_change":"2024-05-28T14:06:24.427568+0000","last_active":"2024-05-28T14:06:24.430746+0000","last_peered":"2024-05-28T14:06:24.430746+0000","last_clean":"2024-05-28T14:06:24.430746+0000","last_became_active":"2024-05-28T14:06:24.427288+0000","last_became_peered":"2024-05-28T14:06:24.427288+0000","last_unstale":"2024-05-28T14:06:24.430746+0000","last_undegraded":"2024-05-28T14:06:24.430746+0000","last_fullsized":"2024-05-28T14:06:24.430746+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:34:47.264602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.424448+0000","last_change":"2024-05-28T14:06:24.424448+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.424448+0000","last_undegraded":"2024-05-28T14:06:24.424448+0000","last_fullsized":"2024-05-28T14:06:24.424448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.437327+0000","last_change":"2024-05-28T14:06:24.435174+0000","last_active":"2024-05-28T14:06:24.437327+0000","last_peered":"2024-05-28T14:06:24.437327+0000","last_clean":"2024-05-28T14:06:24.437327+0000","last_became_active":"2024-05-28T14:06:24.434890+0000","last_became_peered":"2024-05-28T14:06:24.434890+0000","last_unstale":"2024-05-28T14:06:24.437327+0000","last_undegraded":"2024-05-28T14:06:24.437327+0000","last_fullsized":"2024-05-28T14:06:24.437327+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:31:55.554880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.425545+0000","last_change":"2024-05-28T14:06:24.425545+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.425545+0000","last_undegraded":"2024-05-28T14:06:24.425545+0000","last_fullsized":"2024-05-28T14:06:24.425545+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.433105+0000","last_change":"2024-05-28T14:06:24.428906+0000","last_active":"2024-05-28T14:06:24.433105+0000","last_peered":"2024-05-28T14:06:24.433105+0000","last_clean":"2024-05-28T14:06:24.433105+0000","last_became_active":"2024-05-28T14:06:24.427835+0000","last_became_peered":"2024-05-28T14:06:24.427835+0000","last_unstale":"2024-05-28T14:06:24.433105+0000","last_undegraded":"2024-05-28T14:06:24.433105+0000","last_fullsized":"2024-05-28T14:06:24.433105+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:04:56.310115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416321+0000","last_change":"2024-05-28T14:06:24.416321+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416321+0000","last_undegraded":"2024-05-28T14:06:24.416321+0000","last_fullsized":"2024-05-28T14:06:24.416321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.426845+0000","last_change":"2024-05-28T14:06:24.422900+0000","last_active":"2024-05-28T14:06:24.426845+0000","last_peered":"2024-05-28T14:06:24.426845+0000","last_clean":"2024-05-28T14:06:24.426845+0000","last_became_active":"2024-05-28T14:06:24.422439+0000","last_became_peered":"2024-05-28T14:06:24.422439+0000","last_unstale":"2024-05-28T14:06:24.426845+0000","last_undegraded":"2024-05-28T14:06:24.426845+0000","last_fullsized":"2024-05-28T14:06:24.426845+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:56:12.299871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412777+0000","last_change":"2024-05-28T14:06:24.412777+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412777+0000","last_undegraded":"2024-05-28T14:06:24.412777+0000","last_fullsized":"2024-05-28T14:06:24.412777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.433127+0000","last_change":"2024-05-28T14:06:24.431434+0000","last_active":"2024-05-28T14:06:24.433127+0000","last_peered":"2024-05-28T14:06:24.433127+0000","last_clean":"2024-05-28T14:06:24.433127+0000","last_became_active":"2024-05-28T14:06:24.430297+0000","last_became_peered":"2024-05-28T14:06:24.430297+0000","last_unstale":"2024-05-28T14:06:24.433127+0000","last_undegraded":"2024-05-28T14:06:24.433127+0000","last_fullsized":"2024-05-28T14:06:24.433127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:46:10.252003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413731+0000","last_change":"2024-05-28T14:06:24.413731+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413731+0000","last_undegraded":"2024-05-28T14:06:24.413731+0000","last_fullsized":"2024-05-28T14:06:24.413731+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.429025+0000","last_change":"2024-05-28T14:06:24.423413+0000","last_active":"2024-05-28T14:06:24.429025+0000","last_peered":"2024-05-28T14:06:24.429025+0000","last_clean":"2024-05-28T14:06:24.429025+0000","last_became_active":"2024-05-28T14:06:24.423140+0000","last_became_peered":"2024-05-28T14:06:24.423140+0000","last_unstale":"2024-05-28T14:06:24.429025+0000","last_undegraded":"2024-05-28T14:06:24.429025+0000","last_fullsized":"2024-05-28T14:06:24.429025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:37:36.072281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415270+0000","last_change":"2024-05-28T14:06:24.415270+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415270+0000","last_undegraded":"2024-05-28T14:06:24.415270+0000","last_fullsized":"2024-05-28T14:06:24.415270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.429196+0000","last_change":"2024-05-28T14:06:24.425685+0000","last_active":"2024-05-28T14:06:24.429196+0000","last_peered":"2024-05-28T14:06:24.429196+0000","last_clean":"2024-05-28T14:06:24.429196+0000","last_became_active":"2024-05-28T14:06:24.425387+0000","last_became_peered":"2024-05-28T14:06:24.425387+0000","last_unstale":"2024-05-28T14:06:24.429196+0000","last_undegraded":"2024-05-28T14:06:24.429196+0000","last_fullsized":"2024-05-28T14:06:24.429196+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:53:53.347808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415360+0000","last_change":"2024-05-28T14:06:24.415360+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415360+0000","last_undegraded":"2024-05-28T14:06:24.415360+0000","last_fullsized":"2024-05-28T14:06:24.415360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.423810+0000","last_change":"2024-05-28T14:06:24.422382+0000","last_active":"2024-05-28T14:06:24.423810+0000","last_peered":"2024-05-28T14:06:24.423810+0000","last_clean":"2024-05-28T14:06:24.423810+0000","last_became_active":"2024-05-28T14:06:24.422093+0000","last_became_peered":"2024-05-28T14:06:24.422093+0000","last_unstale":"2024-05-28T14:06:24.423810+0000","last_undegraded":"2024-05-28T14:06:24.423810+0000","last_fullsized":"2024-05-28T14:06:24.423810+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:58:13.946282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.423794+0000","last_change":"2024-05-28T14:06:24.423794+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.423794+0000","last_undegraded":"2024-05-28T14:06:24.423794+0000","last_fullsized":"2024-05-28T14:06:24.423794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.438284+0000","last_change":"2024-05-28T14:06:24.436081+0000","last_active":"2024-05-28T14:06:24.438284+0000","last_peered":"2024-05-28T14:06:24.438284+0000","last_clean":"2024-05-28T14:06:24.438284+0000","last_became_active":"2024-05-28T14:06:24.435796+0000","last_became_peered":"2024-05-28T14:06:24.435796+0000","last_unstale":"2024-05-28T14:06:24.438284+0000","last_undegraded":"2024-05-28T14:06:24.438284+0000","last_fullsized":"2024-05-28T14:06:24.438284+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:37:36.072281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.423969+0000","last_change":"2024-05-28T14:06:24.423969+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.423969+0000","last_undegraded":"2024-05-28T14:06:24.423969+0000","last_fullsized":"2024-05-28T14:06:24.423969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.426613+0000","last_change":"2024-05-28T14:06:24.421989+0000","last_active":"2024-05-28T14:06:24.426613+0000","last_peered":"2024-05-28T14:06:24.426613+0000","last_clean":"2024-05-28T14:06:24.426613+0000","last_became_active":"2024-05-28T14:06:24.421727+0000","last_became_peered":"2024-05-28T14:06:24.421727+0000","last_unstale":"2024-05-28T14:06:24.426613+0000","last_undegraded":"2024-05-28T14:06:24.426613+0000","last_fullsized":"2024-05-28T14:06:24.426613+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:35:55.179829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413607+0000","last_change":"2024-05-28T14:06:24.413607+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413607+0000","last_undegraded":"2024-05-28T14:06:24.413607+0000","last_fullsized":"2024-05-28T14:06:24.413607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.423031+0000","last_change":"2024-05-28T14:06:24.421197+0000","last_active":"2024-05-28T14:06:24.423031+0000","last_peered":"2024-05-28T14:06:24.423031+0000","last_clean":"2024-05-28T14:06:24.423031+0000","last_became_active":"2024-05-28T14:06:24.420925+0000","last_became_peered":"2024-05-28T14:06:24.420925+0000","last_unstale":"2024-05-28T14:06:24.423031+0000","last_undegraded":"2024-05-28T14:06:24.423031+0000","last_fullsized":"2024-05-28T14:06:24.423031+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:04:37.401131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.423713+0000","last_change":"2024-05-28T14:06:24.423713+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.423713+0000","last_undegraded":"2024-05-28T14:06:24.423713+0000","last_fullsized":"2024-05-28T14:06:24.423713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431171+0000","last_change":"2024-05-28T14:06:24.429639+0000","last_active":"2024-05-28T14:06:24.431171+0000","last_peered":"2024-05-28T14:06:24.431171+0000","last_clean":"2024-05-28T14:06:24.431171+0000","last_became_active":"2024-05-28T14:06:24.429133+0000","last_became_peered":"2024-05-28T14:06:24.429133+0000","last_unstale":"2024-05-28T14:06:24.431171+0000","last_undegraded":"2024-05-28T14:06:24.431171+0000","last_fullsized":"2024-05-28T14:06:24.431171+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:26.316274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415570+0000","last_change":"2024-05-28T14:06:24.415570+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415570+0000","last_undegraded":"2024-05-28T14:06:24.415570+0000","last_fullsized":"2024-05-28T14:06:24.415570+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.435344+0000","last_change":"2024-05-28T14:06:24.432341+0000","last_active":"2024-05-28T14:06:24.435344+0000","last_peered":"2024-05-28T14:06:24.435344+0000","last_clean":"2024-05-28T14:06:24.435344+0000","last_became_active":"2024-05-28T14:06:24.432058+0000","last_became_peered":"2024-05-28T14:06:24.432058+0000","last_unstale":"2024-05-28T14:06:24.435344+0000","last_undegraded":"2024-05-28T14:06:24.435344+0000","last_fullsized":"2024-05-28T14:06:24.435344+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:59:39.619307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414289+0000","last_change":"2024-05-28T14:06:24.414289+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414289+0000","last_undegraded":"2024-05-28T14:06:24.414289+0000","last_fullsized":"2024-05-28T14:06:24.414289+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431568+0000","last_change":"2024-05-28T14:06:24.426720+0000","last_active":"2024-05-28T14:06:24.431568+0000","last_peered":"2024-05-28T14:06:24.431568+0000","last_clean":"2024-05-28T14:06:24.431568+0000","last_became_active":"2024-05-28T14:06:24.426399+0000","last_became_peered":"2024-05-28T14:06:24.426399+0000","last_unstale":"2024-05-28T14:06:24.431568+0000","last_undegraded":"2024-05-28T14:06:24.431568+0000","last_fullsized":"2024-05-28T14:06:24.431568+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:26:45.081785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414452+0000","last_change":"2024-05-28T14:06:24.414452+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414452+0000","last_undegraded":"2024-05-28T14:06:24.414452+0000","last_fullsized":"2024-05-28T14:06:24.414452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431522+0000","last_change":"2024-05-28T14:06:24.424772+0000","last_active":"2024-05-28T14:06:24.431522+0000","last_peered":"2024-05-28T14:06:24.431522+0000","last_clean":"2024-05-28T14:06:24.431522+0000","last_became_active":"2024-05-28T14:06:24.424480+0000","last_became_peered":"2024-05-28T14:06:24.424480+0000","last_unstale":"2024-05-28T14:06:24.431522+0000","last_undegraded":"2024-05-28T14:06:24.431522+0000","last_fullsized":"2024-05-28T14:06:24.431522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:45:23.593514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413039+0000","last_change":"2024-05-28T14:06:24.413039+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413039+0000","last_undegraded":"2024-05-28T14:06:24.413039+0000","last_fullsized":"2024-05-28T14:06:24.413039+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.437107+0000","last_change":"2024-05-28T14:06:24.434783+0000","last_active":"2024-05-28T14:06:24.437107+0000","last_peered":"2024-05-28T14:06:24.437107+0000","last_clean":"2024-05-28T14:06:24.437107+0000","last_became_active":"2024-05-28T14:06:24.434490+0000","last_became_peered":"2024-05-28T14:06:24.434490+0000","last_unstale":"2024-05-28T14:06:24.437107+0000","last_undegraded":"2024-05-28T14:06:24.437107+0000","last_fullsized":"2024-05-28T14:06:24.437107+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:46:01.970253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.423255+0000","last_change":"2024-05-28T14:06:24.423255+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.423255+0000","last_undegraded":"2024-05-28T14:06:24.423255+0000","last_fullsized":"2024-05-28T14:06:24.423255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.437420+0000","last_change":"2024-05-28T14:06:24.434447+0000","last_active":"2024-05-28T14:06:24.437420+0000","last_peered":"2024-05-28T14:06:24.437420+0000","last_clean":"2024-05-28T14:06:24.437420+0000","last_became_active":"2024-05-28T14:06:24.433002+0000","last_became_peered":"2024-05-28T14:06:24.433002+0000","last_unstale":"2024-05-28T14:06:24.437420+0000","last_undegraded":"2024-05-28T14:06:24.437420+0000","last_fullsized":"2024-05-28T14:06:24.437420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:41:19.574984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412440+0000","last_change":"2024-05-28T14:06:24.412440+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412440+0000","last_undegraded":"2024-05-28T14:06:24.412440+0000","last_fullsized":"2024-05-28T14:06:24.412440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.432802+0000","last_change":"2024-05-28T14:06:24.428701+0000","last_active":"2024-05-28T14:06:24.432802+0000","last_peered":"2024-05-28T14:06:24.432802+0000","last_clean":"2024-05-28T14:06:24.432802+0000","last_became_active":"2024-05-28T14:06:24.428477+0000","last_became_peered":"2024-05-28T14:06:24.428477+0000","last_unstale":"2024-05-28T14:06:24.432802+0000","last_undegraded":"2024-05-28T14:06:24.432802+0000","last_fullsized":"2024-05-28T14:06:24.432802+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:38:32.586189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412600+0000","last_change":"2024-05-28T14:06:24.412600+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412600+0000","last_undegraded":"2024-05-28T14:06:24.412600+0000","last_fullsized":"2024-05-28T14:06:24.412600+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.438463+0000","last_change":"2024-05-28T14:06:24.437928+0000","last_active":"2024-05-28T14:06:24.438463+0000","last_peered":"2024-05-28T14:06:24.438463+0000","last_clean":"2024-05-28T14:06:24.438463+0000","last_became_active":"2024-05-28T14:06:24.437366+0000","last_became_peered":"2024-05-28T14:06:24.437366+0000","last_unstale":"2024-05-28T14:06:24.438463+0000","last_undegraded":"2024-05-28T14:06:24.438463+0000","last_fullsized":"2024-05-28T14:06:24.438463+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:08.591204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413482+0000","last_change":"2024-05-28T14:06:24.413482+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413482+0000","last_undegraded":"2024-05-28T14:06:24.413482+0000","last_fullsized":"2024-05-28T14:06:24.413482+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.432994+0000","last_change":"2024-05-28T14:06:24.429400+0000","last_active":"2024-05-28T14:06:24.432994+0000","last_peered":"2024-05-28T14:06:24.432994+0000","last_clean":"2024-05-28T14:06:24.432994+0000","last_became_active":"2024-05-28T14:06:24.429124+0000","last_became_peered":"2024-05-28T14:06:24.429124+0000","last_unstale":"2024-05-28T14:06:24.432994+0000","last_undegraded":"2024-05-28T14:06:24.432994+0000","last_fullsized":"2024-05-28T14:06:24.432994+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:39:32.793666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414095+0000","last_change":"2024-05-28T14:06:24.414095+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414095+0000","last_undegraded":"2024-05-28T14:06:24.414095+0000","last_fullsized":"2024-05-28T14:06:24.414095+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.422911+0000","last_change":"2024-05-28T14:06:24.420613+0000","last_active":"2024-05-28T14:06:24.422911+0000","last_peered":"2024-05-28T14:06:24.422911+0000","last_clean":"2024-05-28T14:06:24.422911+0000","last_became_active":"2024-05-28T14:06:24.420171+0000","last_became_peered":"2024-05-28T14:06:24.420171+0000","last_unstale":"2024-05-28T14:06:24.422911+0000","last_undegraded":"2024-05-28T14:06:24.422911+0000","last_fullsized":"2024-05-28T14:06:24.422911+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:49.326481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414920+0000","last_change":"2024-05-28T14:06:24.414920+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414920+0000","last_undegraded":"2024-05-28T14:06:24.414920+0000","last_fullsized":"2024-05-28T14:06:24.414920+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.430690+0000","last_change":"2024-05-28T14:06:24.427464+0000","last_active":"2024-05-28T14:06:24.430690+0000","last_peered":"2024-05-28T14:06:24.430690+0000","last_clean":"2024-05-28T14:06:24.430690+0000","last_became_active":"2024-05-28T14:06:24.427145+0000","last_became_peered":"2024-05-28T14:06:24.427145+0000","last_unstale":"2024-05-28T14:06:24.430690+0000","last_undegraded":"2024-05-28T14:06:24.430690+0000","last_fullsized":"2024-05-28T14:06:24.430690+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:06:23.658146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417284+0000","last_change":"2024-05-28T14:06:24.417284+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417284+0000","last_undegraded":"2024-05-28T14:06:24.417284+0000","last_fullsized":"2024-05-28T14:06:24.417284+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.436513+0000","last_change":"2024-05-28T14:06:24.432369+0000","last_active":"2024-05-28T14:06:24.436513+0000","last_peered":"2024-05-28T14:06:24.436513+0000","last_clean":"2024-05-28T14:06:24.436513+0000","last_became_active":"2024-05-28T14:06:24.431992+0000","last_became_peered":"2024-05-28T14:06:24.431992+0000","last_unstale":"2024-05-28T14:06:24.436513+0000","last_undegraded":"2024-05-28T14:06:24.436513+0000","last_fullsized":"2024-05-28T14:06:24.436513+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:42:55.217345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416075+0000","last_change":"2024-05-28T14:06:24.416075+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416075+0000","last_undegraded":"2024-05-28T14:06:24.416075+0000","last_fullsized":"2024-05-28T14:06:24.416075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431126+0000","last_change":"2024-05-28T14:06:24.427952+0000","last_active":"2024-05-28T14:06:24.431126+0000","last_peered":"2024-05-28T14:06:24.431126+0000","last_clean":"2024-05-28T14:06:24.431126+0000","last_became_active":"2024-05-28T14:06:24.427511+0000","last_became_peered":"2024-05-28T14:06:24.427511+0000","last_unstale":"2024-05-28T14:06:24.431126+0000","last_undegraded":"2024-05-28T14:06:24.431126+0000","last_fullsized":"2024-05-28T14:06:24.431126+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:03:34.586234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.425338+0000","last_change":"2024-05-28T14:06:24.425338+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.425338+0000","last_undegraded":"2024-05-28T14:06:24.425338+0000","last_fullsized":"2024-05-28T14:06:24.425338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.430060+0000","last_change":"2024-05-28T14:06:24.427264+0000","last_active":"2024-05-28T14:06:24.430060+0000","last_peered":"2024-05-28T14:06:24.430060+0000","last_clean":"2024-05-28T14:06:24.430060+0000","last_became_active":"2024-05-28T14:06:24.426662+0000","last_became_peered":"2024-05-28T14:06:24.426662+0000","last_unstale":"2024-05-28T14:06:24.430060+0000","last_undegraded":"2024-05-28T14:06:24.430060+0000","last_fullsized":"2024-05-28T14:06:24.430060+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:28:56.343364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.424918+0000","last_change":"2024-05-28T14:06:24.424918+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.424918+0000","last_undegraded":"2024-05-28T14:06:24.424918+0000","last_fullsized":"2024-05-28T14:06:24.424918+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.432299+0000","last_change":"2024-05-28T14:06:24.431741+0000","last_active":"2024-05-28T14:06:24.432299+0000","last_peered":"2024-05-28T14:06:24.432299+0000","last_clean":"2024-05-28T14:06:24.432299+0000","last_became_active":"2024-05-28T14:06:24.431388+0000","last_became_peered":"2024-05-28T14:06:24.431388+0000","last_unstale":"2024-05-28T14:06:24.432299+0000","last_undegraded":"2024-05-28T14:06:24.432299+0000","last_fullsized":"2024-05-28T14:06:24.432299+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:11:06.256080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412880+0000","last_change":"2024-05-28T14:06:24.412880+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412880+0000","last_undegraded":"2024-05-28T14:06:24.412880+0000","last_fullsized":"2024-05-28T14:06:24.412880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.424536+0000","last_change":"2024-05-28T14:06:24.420374+0000","last_active":"2024-05-28T14:06:24.424536+0000","last_peered":"2024-05-28T14:06:24.424536+0000","last_clean":"2024-05-28T14:06:24.424536+0000","last_became_active":"2024-05-28T14:06:24.420044+0000","last_became_peered":"2024-05-28T14:06:24.420044+0000","last_unstale":"2024-05-28T14:06:24.424536+0000","last_undegraded":"2024-05-28T14:06:24.424536+0000","last_fullsized":"2024-05-28T14:06:24.424536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:29:46.118850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.422978+0000","last_change":"2024-05-28T14:06:24.422978+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.422978+0000","last_undegraded":"2024-05-28T14:06:24.422978+0000","last_fullsized":"2024-05-28T14:06:24.422978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.425967+0000","last_change":"2024-05-28T14:06:24.422804+0000","last_active":"2024-05-28T14:06:24.425967+0000","last_peered":"2024-05-28T14:06:24.425967+0000","last_clean":"2024-05-28T14:06:24.425967+0000","last_became_active":"2024-05-28T14:06:24.422468+0000","last_became_peered":"2024-05-28T14:06:24.422468+0000","last_unstale":"2024-05-28T14:06:24.425967+0000","last_undegraded":"2024-05-28T14:06:24.425967+0000","last_fullsized":"2024-05-28T14:06:24.425967+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:46:01.970253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412444+0000","last_change":"2024-05-28T14:06:24.412444+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412444+0000","last_undegraded":"2024-05-28T14:06:24.412444+0000","last_fullsized":"2024-05-28T14:06:24.412444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.433548+0000","last_change":"2024-05-28T14:06:24.429508+0000","last_active":"2024-05-28T14:06:24.433548+0000","last_peered":"2024-05-28T14:06:24.433548+0000","last_clean":"2024-05-28T14:06:24.433548+0000","last_became_active":"2024-05-28T14:06:24.429101+0000","last_became_peered":"2024-05-28T14:06:24.429101+0000","last_unstale":"2024-05-28T14:06:24.433548+0000","last_undegraded":"2024-05-28T14:06:24.433548+0000","last_fullsized":"2024-05-28T14:06:24.433548+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:15.013592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417313+0000","last_change":"2024-05-28T14:06:24.417313+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417313+0000","last_undegraded":"2024-05-28T14:06:24.417313+0000","last_fullsized":"2024-05-28T14:06:24.417313+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431063+0000","last_change":"2024-05-28T14:06:24.427387+0000","last_active":"2024-05-28T14:06:24.431063+0000","last_peered":"2024-05-28T14:06:24.431063+0000","last_clean":"2024-05-28T14:06:24.431063+0000","last_became_active":"2024-05-28T14:06:24.427106+0000","last_became_peered":"2024-05-28T14:06:24.427106+0000","last_unstale":"2024-05-28T14:06:24.431063+0000","last_undegraded":"2024-05-28T14:06:24.431063+0000","last_fullsized":"2024-05-28T14:06:24.431063+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:19:02.178759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414347+0000","last_change":"2024-05-28T14:06:24.414347+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414347+0000","last_undegraded":"2024-05-28T14:06:24.414347+0000","last_fullsized":"2024-05-28T14:06:24.414347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.436299+0000","last_change":"2024-05-28T14:06:24.433815+0000","last_active":"2024-05-28T14:06:24.436299+0000","last_peered":"2024-05-28T14:06:24.436299+0000","last_clean":"2024-05-28T14:06:24.436299+0000","last_became_active":"2024-05-28T14:06:24.433524+0000","last_became_peered":"2024-05-28T14:06:24.433524+0000","last_unstale":"2024-05-28T14:06:24.436299+0000","last_undegraded":"2024-05-28T14:06:24.436299+0000","last_fullsized":"2024-05-28T14:06:24.436299+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:04:37.401131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":264,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.429488+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:06:24.429488+0000","last_peered":"2024-05-28T14:06:24.429488+0000","last_clean":"2024-05-28T14:06:24.429488+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:06:24.429488+0000","last_undegraded":"2024-05-28T14:06:24.429488+0000","last_fullsized":"2024-05-28T14:06:24.429488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574949,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58988,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312852,"statfs":{"total":96636764160,"available":96576360448,"internally_reserved":0,"allocated":1470464,"data_stored":961924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54236,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317604,"statfs":{"total":96636764160,"available":96581226496,"internally_reserved":0,"allocated":864256,"data_stored":365253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54248,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317592,"statfs":{"total":96636764160,"available":96581214208,"internally_reserved":0,"allocated":876544,"data_stored":371556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58392,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313448,"statfs":{"total":96636764160,"available":96576970752,"internally_reserved":0,"allocated":864256,"data_stored":365253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45928,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325912,"statfs":{"total":96636764160,"available":96589733888,"internally_reserved":0,"allocated":876544,"data_stored":371556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54236,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317604,"statfs":{"total":96636764160,"available":96581226496,"internally_reserved":0,"allocated":864256,"data_stored":365253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62564,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309276,"statfs":{"total":96636764160,"available":96572698624,"internally_reserved":0,"allocated":876544,"data_stored":371556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574950,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63148,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308692,"statfs":{"total":96636764160,"available":96572100608,"internally_reserved":0,"allocated":1470464,"data_stored":961924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:06:29.929 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:06:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.118+0000 7f1a25600640 1 Processor -- start 2024-05-28T14:06:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.118+0000 7f1a25600640 1 -- start start 2024-05-28T14:06:30.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.122+0000 7f1a25600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1a2006fa20 0x7f1a200d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.122+0000 7f1a25600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a20067760 con 0x7f1a2015c740 2024-05-28T14:06:30.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.122+0000 7f1a25600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a20067a00 con 0x7f1a2006ca00 2024-05-28T14:06:30.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.122+0000 7f1a25600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1a200cda60 con 0x7f1a2006fa20 2024-05-28T14:06:30.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.122+0000 7f1a1ea00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1a2015c740 0x7f1a200cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42830/0 (socket says 172.21.15.42:42830) 2024-05-28T14:06:30.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.122+0000 7f1a1ea00640 1 -- 172.21.15.42:0/3214338676 learned_addr learned my addr 172.21.15.42:0/3214338676 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:30.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.122+0000 7f1a16000640 1 --2- 172.21.15.42:0/3214338676 >> v2:172.21.15.42:3300/0 conn(0x7f1a2006fa20 0x7f1a200d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.122+0000 7f1a16000640 1 -- 172.21.15.42:0/3214338676 >> v1:172.21.15.63:6789/0 conn(0x7f1a2006ca00 legacy=0x7f1a2015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:30.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.122+0000 7f1a16000640 1 -- 172.21.15.42:0/3214338676 >> v1:172.21.15.42:6789/0 conn(0x7f1a2015c740 legacy=0x7f1a200cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.122+0000 7f1a16000640 1 -- 172.21.15.42:0/3214338676 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a20067ce0 con 0x7f1a2006fa20 2024-05-28T14:06:30.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a16000640 1 --2- 172.21.15.42:0/3214338676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a2006fa20 0x7f1a200d0590 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009d90 tx=0x7f1a0801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b97cd58a80b0a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a1e000640 1 -- 172.21.15.42:0/3214338676 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a080b2070 con 0x7f1a2006fa20 2024-05-28T14:06:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a1e000640 1 -- 172.21.15.42:0/3214338676 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a080a5870 con 0x7f1a2006fa20 2024-05-28T14:06:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a1e000640 1 -- 172.21.15.42:0/3214338676 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a080a5c40 con 0x7f1a2006fa20 2024-05-28T14:06:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a25600640 1 -- 172.21.15.42:0/3214338676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a2006fa20 msgr2=0x7f1a200d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a25600640 1 --2- 172.21.15.42:0/3214338676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a2006fa20 0x7f1a200d0590 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009d90 tx=0x7f1a0801c8a0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a25600640 1 -- 172.21.15.42:0/3214338676 shutdown_connections 2024-05-28T14:06:30.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a25600640 1 --2- 172.21.15.42:0/3214338676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a2006fa20 0x7f1a200d0590 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a25600640 1 -- 172.21.15.42:0/3214338676 >> 172.21.15.42:0/3214338676 conn(0x7f1a20072450 msgr2=0x7f1a20072860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:30.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a25600640 1 -- 172.21.15.42:0/3214338676 shutdown_connections 2024-05-28T14:06:30.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a25600640 1 -- 172.21.15.42:0/3214338676 wait complete. 2024-05-28T14:06:30.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a25600640 1 Processor -- start 2024-05-28T14:06:30.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a25600640 1 -- start start 2024-05-28T14:06:30.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a25600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a2006ca00 0x7f1a20148700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.126+0000 7f1a25600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a2006fa20 0x7f1a200b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a25600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a2015c740 0x7f1a200b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a25600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2016ab40 con 0x7f1a2015c740 2024-05-28T14:06:30.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a25600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2015d890 con 0x7f1a2006fa20 2024-05-28T14:06:30.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a25600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1a2016acc0 con 0x7f1a2006ca00 2024-05-28T14:06:30.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a1f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a2015c740 0x7f1a200b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a1f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a2015c740 0x7f1a200b11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53170/0 (socket says 172.21.15.42:53170) 2024-05-28T14:06:30.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a1f400640 1 -- 172.21.15.42:0/1409008168 learned_addr learned my addr 172.21.15.42:0/1409008168 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:30.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a16000640 1 --2- 172.21.15.42:0/1409008168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a2006fa20 0x7f1a200b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a1ea00640 1 --2- 172.21.15.42:0/1409008168 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a2006ca00 0x7f1a20148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a1f400640 1 -- 172.21.15.42:0/1409008168 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a2006ca00 msgr2=0x7f1a20148700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a1f400640 1 --2- 172.21.15.42:0/1409008168 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a2006ca00 0x7f1a20148700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a1f400640 1 -- 172.21.15.42:0/1409008168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a2006fa20 msgr2=0x7f1a200b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a1f400640 1 --2- 172.21.15.42:0/1409008168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a2006fa20 0x7f1a200b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a1f400640 1 -- 172.21.15.42:0/1409008168 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2015c2b0 con 0x7f1a2015c740 2024-05-28T14:06:30.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a1f400640 1 --2- 172.21.15.42:0/1409008168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a2015c740 0x7f1a200b11e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800c3c0 tx=0x7f1a1800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:30.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a17400640 1 -- 172.21.15.42:0/1409008168 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18011790 con 0x7f1a2015c740 2024-05-28T14:06:30.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a17400640 1 -- 172.21.15.42:0/1409008168 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a18017070 con 0x7f1a2015c740 2024-05-28T14:06:30.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a17400640 1 -- 172.21.15.42:0/1409008168 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18011b70 con 0x7f1a2015c740 2024-05-28T14:06:30.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a25600640 1 -- 172.21.15.42:0/1409008168 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a20148d30 con 0x7f1a2015c740 2024-05-28T14:06:30.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a25600640 1 -- 172.21.15.42:0/1409008168 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a200b4700 con 0x7f1a2015c740 2024-05-28T14:06:30.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a25600640 1 -- 172.21.15.42:0/1409008168 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a20068a90 con 0x7f1a2015c740 2024-05-28T14:06:30.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.130+0000 7f1a17400640 1 -- 172.21.15.42:0/1409008168 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f1a18011d10 con 0x7f1a2015c740 2024-05-28T14:06:30.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.134+0000 7f1a17400640 1 --2- 172.21.15.42:0/1409008168 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19fc069070 0x7f19fc06b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.134+0000 7f1a17400640 1 -- 172.21.15.42:0/1409008168 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7f1a18085070 con 0x7f1a2015c740 2024-05-28T14:06:30.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.134+0000 7f1a1ea00640 1 --2- 172.21.15.42:0/1409008168 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19fc069070 0x7f19fc06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.134+0000 7f1a1ea00640 1 --2- 172.21.15.42:0/1409008168 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19fc069070 0x7f19fc06b530 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1a10000f70 tx=0x7f1a10007e90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:30.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.138+0000 7f1a17400640 1 -- 172.21.15.42:0/1409008168 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1a20148d30 con 0x7f1a2015c740 2024-05-28T14:06:30.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.298+0000 7f1a25600640 1 -- 172.21.15.42:0/1409008168 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a20151d20 con 0x7f19fc069070 2024-05-28T14:06:30.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.306+0000 7f1a17400640 1 -- 172.21.15.42:0/1409008168 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246753 (secure 0 0 0) 0x7f1a20151d20 con 0x7f19fc069070 2024-05-28T14:06:30.294 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:30.296 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:06:30.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.310+0000 7f1a25600640 1 -- 172.21.15.42:0/1409008168 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19fc069070 msgr2=0x7f19fc06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.310+0000 7f1a25600640 1 --2- 172.21.15.42:0/1409008168 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19fc069070 0x7f19fc06b530 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1a10000f70 tx=0x7f1a10007e90 comp rx=0 tx=0).stop 2024-05-28T14:06:30.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.310+0000 7f1a25600640 1 -- 172.21.15.42:0/1409008168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a2015c740 msgr2=0x7f1a200b11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.310+0000 7f1a25600640 1 --2- 172.21.15.42:0/1409008168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a2015c740 0x7f1a200b11e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800c3c0 tx=0x7f1a1800c880 comp rx=0 tx=0).stop 2024-05-28T14:06:30.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.310+0000 7f1a25600640 1 -- 172.21.15.42:0/1409008168 shutdown_connections 2024-05-28T14:06:30.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.310+0000 7f1a25600640 1 --2- 172.21.15.42:0/1409008168 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f19fc069070 0x7f19fc06b530 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.310+0000 7f1a25600640 1 --2- 172.21.15.42:0/1409008168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a2015c740 0x7f1a200b11e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.310+0000 7f1a25600640 1 --2- 172.21.15.42:0/1409008168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a2006fa20 0x7f1a200b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.310+0000 7f1a25600640 1 --2- 172.21.15.42:0/1409008168 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a2006ca00 0x7f1a20148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.310+0000 7f1a25600640 1 -- 172.21.15.42:0/1409008168 >> 172.21.15.42:0/1409008168 conn(0x7f1a20072450 msgr2=0x7f1a20152520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:30.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.310+0000 7f1a25600640 1 -- 172.21.15.42:0/1409008168 shutdown_connections 2024-05-28T14:06:30.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.310+0000 7f1a25600640 1 -- 172.21.15.42:0/1409008168 wait complete. 2024-05-28T14:06:30.316 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-05-28T14:06:28.995245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":451740,"kb_used_data":7972,"kb_used_omap":12,"kb_used_meta":443635,"kb_avail":754522980,"statfs":{"total":773094113280,"available":772631531520,"internally_reserved":0,"allocated":8163328,"data_stored":4134275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":454282802},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.595421"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.424413+0000","last_change":"2024-05-28T14:06:24.424413+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.424413+0000","last_undegraded":"2024-05-28T14:06:24.424413+0000","last_fullsized":"2024-05-28T14:06:24.424413+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.421572+0000","last_change":"2024-05-28T14:06:24.421572+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.421572+0000","last_undegraded":"2024-05-28T14:06:24.421572+0000","last_fullsized":"2024-05-28T14:06:24.421572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415723+0000","last_change":"2024-05-28T14:06:24.415723+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415723+0000","last_undegraded":"2024-05-28T14:06:24.415723+0000","last_fullsized":"2024-05-28T14:06:24.415723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.418596+0000","last_change":"2024-05-28T14:06:24.418596+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.418596+0000","last_undegraded":"2024-05-28T14:06:24.418596+0000","last_fullsized":"2024-05-28T14:06:24.418596+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417356+0000","last_change":"2024-05-28T14:06:24.417356+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417356+0000","last_undegraded":"2024-05-28T14:06:24.417356+0000","last_fullsized":"2024-05-28T14:06:24.417356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417815+0000","last_change":"2024-05-28T14:06:24.417815+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417815+0000","last_undegraded":"2024-05-28T14:06:24.417815+0000","last_fullsized":"2024-05-28T14:06:24.417815+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416338+0000","last_change":"2024-05-28T14:06:24.416338+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416338+0000","last_undegraded":"2024-05-28T14:06:24.416338+0000","last_fullsized":"2024-05-28T14:06:24.416338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.425317+0000","last_change":"2024-05-28T14:06:24.425317+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.425317+0000","last_undegraded":"2024-05-28T14:06:24.425317+0000","last_fullsized":"2024-05-28T14:06:24.425317+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417044+0000","last_change":"2024-05-28T14:06:24.417044+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417044+0000","last_undegraded":"2024-05-28T14:06:24.417044+0000","last_fullsized":"2024-05-28T14:06:24.417044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413819+0000","last_change":"2024-05-28T14:06:24.413819+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413819+0000","last_undegraded":"2024-05-28T14:06:24.413819+0000","last_fullsized":"2024-05-28T14:06:24.413819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.425975+0000","last_change":"2024-05-28T14:06:24.425975+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.425975+0000","last_undegraded":"2024-05-28T14:06:24.425975+0000","last_fullsized":"2024-05-28T14:06:24.425975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.418211+0000","last_change":"2024-05-28T14:06:24.418211+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.418211+0000","last_undegraded":"2024-05-28T14:06:24.418211+0000","last_fullsized":"2024-05-28T14:06:24.418211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.419930+0000","last_change":"2024-05-28T14:06:24.419930+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.419930+0000","last_undegraded":"2024-05-28T14:06:24.419930+0000","last_fullsized":"2024-05-28T14:06:24.419930+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414944+0000","last_change":"2024-05-28T14:06:24.414944+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414944+0000","last_undegraded":"2024-05-28T14:06:24.414944+0000","last_fullsized":"2024-05-28T14:06:24.414944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415671+0000","last_change":"2024-05-28T14:06:24.415671+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415671+0000","last_undegraded":"2024-05-28T14:06:24.415671+0000","last_fullsized":"2024-05-28T14:06:24.415671+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412671+0000","last_change":"2024-05-28T14:06:24.412671+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412671+0000","last_undegraded":"2024-05-28T14:06:24.412671+0000","last_fullsized":"2024-05-28T14:06:24.412671+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414216+0000","last_change":"2024-05-28T14:06:24.414216+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414216+0000","last_undegraded":"2024-05-28T14:06:24.414216+0000","last_fullsized":"2024-05-28T14:06:24.414216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412534+0000","last_change":"2024-05-28T14:06:24.412534+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412534+0000","last_undegraded":"2024-05-28T14:06:24.412534+0000","last_fullsized":"2024-05-28T14:06:24.412534+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.427380+0000","last_change":"2024-05-28T14:06:24.427380+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.427380+0000","last_undegraded":"2024-05-28T14:06:24.427380+0000","last_fullsized":"2024-05-28T14:06:24.427380+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.420242+0000","last_change":"2024-05-28T14:06:24.420242+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.420242+0000","last_undegraded":"2024-05-28T14:06:24.420242+0000","last_fullsized":"2024-05-28T14:06:24.420242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413445+0000","last_change":"2024-05-28T14:06:24.413445+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413445+0000","last_undegraded":"2024-05-28T14:06:24.413445+0000","last_fullsized":"2024-05-28T14:06:24.413445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413956+0000","last_change":"2024-05-28T14:06:24.413956+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413956+0000","last_undegraded":"2024-05-28T14:06:24.413956+0000","last_fullsized":"2024-05-28T14:06:24.413956+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.422591+0000","last_change":"2024-05-28T14:06:24.422591+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.422591+0000","last_undegraded":"2024-05-28T14:06:24.422591+0000","last_fullsized":"2024-05-28T14:06:24.422591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.421618+0000","last_change":"2024-05-28T14:06:24.421618+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.421618+0000","last_undegraded":"2024-05-28T14:06:24.421618+0000","last_fullsized":"2024-05-28T14:06:24.421618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413641+0000","last_change":"2024-05-28T14:06:24.413641+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413641+0000","last_undegraded":"2024-05-28T14:06:24.413641+0000","last_fullsized":"2024-05-28T14:06:24.413641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415922+0000","last_change":"2024-05-28T14:06:24.415922+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415922+0000","last_undegraded":"2024-05-28T14:06:24.415922+0000","last_fullsized":"2024-05-28T14:06:24.415922+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415300+0000","last_change":"2024-05-28T14:06:24.415300+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415300+0000","last_undegraded":"2024-05-28T14:06:24.415300+0000","last_fullsized":"2024-05-28T14:06:24.415300+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417303+0000","last_change":"2024-05-28T14:06:24.417303+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417303+0000","last_undegraded":"2024-05-28T14:06:24.417303+0000","last_fullsized":"2024-05-28T14:06:24.417303+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413946+0000","last_change":"2024-05-28T14:06:24.413946+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413946+0000","last_undegraded":"2024-05-28T14:06:24.413946+0000","last_fullsized":"2024-05-28T14:06:24.413946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416788+0000","last_change":"2024-05-28T14:06:24.416788+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416788+0000","last_undegraded":"2024-05-28T14:06:24.416788+0000","last_fullsized":"2024-05-28T14:06:24.416788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415329+0000","last_change":"2024-05-28T14:06:24.415329+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415329+0000","last_undegraded":"2024-05-28T14:06:24.415329+0000","last_fullsized":"2024-05-28T14:06:24.415329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416575+0000","last_change":"2024-05-28T14:06:24.416575+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416575+0000","last_undegraded":"2024-05-28T14:06:24.416575+0000","last_fullsized":"2024-05-28T14:06:24.416575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.422580+0000","last_change":"2024-05-28T14:06:24.422580+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.422580+0000","last_undegraded":"2024-05-28T14:06:24.422580+0000","last_fullsized":"2024-05-28T14:06:24.422580+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.437020+0000","last_change":"2024-05-28T14:06:24.435935+0000","last_active":"2024-05-28T14:06:24.437020+0000","last_peered":"2024-05-28T14:06:24.437020+0000","last_clean":"2024-05-28T14:06:24.437020+0000","last_became_active":"2024-05-28T14:06:24.435637+0000","last_became_peered":"2024-05-28T14:06:24.435637+0000","last_unstale":"2024-05-28T14:06:24.437020+0000","last_undegraded":"2024-05-28T14:06:24.437020+0000","last_fullsized":"2024-05-28T14:06:24.437020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:32:00.614120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413739+0000","last_change":"2024-05-28T14:06:24.413739+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413739+0000","last_undegraded":"2024-05-28T14:06:24.413739+0000","last_fullsized":"2024-05-28T14:06:24.413739+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.430746+0000","last_change":"2024-05-28T14:06:24.427568+0000","last_active":"2024-05-28T14:06:24.430746+0000","last_peered":"2024-05-28T14:06:24.430746+0000","last_clean":"2024-05-28T14:06:24.430746+0000","last_became_active":"2024-05-28T14:06:24.427288+0000","last_became_peered":"2024-05-28T14:06:24.427288+0000","last_unstale":"2024-05-28T14:06:24.430746+0000","last_undegraded":"2024-05-28T14:06:24.430746+0000","last_fullsized":"2024-05-28T14:06:24.430746+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:34:47.264602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.424448+0000","last_change":"2024-05-28T14:06:24.424448+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.424448+0000","last_undegraded":"2024-05-28T14:06:24.424448+0000","last_fullsized":"2024-05-28T14:06:24.424448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.437327+0000","last_change":"2024-05-28T14:06:24.435174+0000","last_active":"2024-05-28T14:06:24.437327+0000","last_peered":"2024-05-28T14:06:24.437327+0000","last_clean":"2024-05-28T14:06:24.437327+0000","last_became_active":"2024-05-28T14:06:24.434890+0000","last_became_peered":"2024-05-28T14:06:24.434890+0000","last_unstale":"2024-05-28T14:06:24.437327+0000","last_undegraded":"2024-05-28T14:06:24.437327+0000","last_fullsized":"2024-05-28T14:06:24.437327+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:31:55.554880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.425545+0000","last_change":"2024-05-28T14:06:24.425545+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.425545+0000","last_undegraded":"2024-05-28T14:06:24.425545+0000","last_fullsized":"2024-05-28T14:06:24.425545+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.433105+0000","last_change":"2024-05-28T14:06:24.428906+0000","last_active":"2024-05-28T14:06:24.433105+0000","last_peered":"2024-05-28T14:06:24.433105+0000","last_clean":"2024-05-28T14:06:24.433105+0000","last_became_active":"2024-05-28T14:06:24.427835+0000","last_became_peered":"2024-05-28T14:06:24.427835+0000","last_unstale":"2024-05-28T14:06:24.433105+0000","last_undegraded":"2024-05-28T14:06:24.433105+0000","last_fullsized":"2024-05-28T14:06:24.433105+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:04:56.310115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416321+0000","last_change":"2024-05-28T14:06:24.416321+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416321+0000","last_undegraded":"2024-05-28T14:06:24.416321+0000","last_fullsized":"2024-05-28T14:06:24.416321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.426845+0000","last_change":"2024-05-28T14:06:24.422900+0000","last_active":"2024-05-28T14:06:24.426845+0000","last_peered":"2024-05-28T14:06:24.426845+0000","last_clean":"2024-05-28T14:06:24.426845+0000","last_became_active":"2024-05-28T14:06:24.422439+0000","last_became_peered":"2024-05-28T14:06:24.422439+0000","last_unstale":"2024-05-28T14:06:24.426845+0000","last_undegraded":"2024-05-28T14:06:24.426845+0000","last_fullsized":"2024-05-28T14:06:24.426845+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:56:12.299871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412777+0000","last_change":"2024-05-28T14:06:24.412777+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412777+0000","last_undegraded":"2024-05-28T14:06:24.412777+0000","last_fullsized":"2024-05-28T14:06:24.412777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.433127+0000","last_change":"2024-05-28T14:06:24.431434+0000","last_active":"2024-05-28T14:06:24.433127+0000","last_peered":"2024-05-28T14:06:24.433127+0000","last_clean":"2024-05-28T14:06:24.433127+0000","last_became_active":"2024-05-28T14:06:24.430297+0000","last_became_peered":"2024-05-28T14:06:24.430297+0000","last_unstale":"2024-05-28T14:06:24.433127+0000","last_undegraded":"2024-05-28T14:06:24.433127+0000","last_fullsized":"2024-05-28T14:06:24.433127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:46:10.252003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413731+0000","last_change":"2024-05-28T14:06:24.413731+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413731+0000","last_undegraded":"2024-05-28T14:06:24.413731+0000","last_fullsized":"2024-05-28T14:06:24.413731+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.429025+0000","last_change":"2024-05-28T14:06:24.423413+0000","last_active":"2024-05-28T14:06:24.429025+0000","last_peered":"2024-05-28T14:06:24.429025+0000","last_clean":"2024-05-28T14:06:24.429025+0000","last_became_active":"2024-05-28T14:06:24.423140+0000","last_became_peered":"2024-05-28T14:06:24.423140+0000","last_unstale":"2024-05-28T14:06:24.429025+0000","last_undegraded":"2024-05-28T14:06:24.429025+0000","last_fullsized":"2024-05-28T14:06:24.429025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:37:36.072281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415270+0000","last_change":"2024-05-28T14:06:24.415270+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415270+0000","last_undegraded":"2024-05-28T14:06:24.415270+0000","last_fullsized":"2024-05-28T14:06:24.415270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.429196+0000","last_change":"2024-05-28T14:06:24.425685+0000","last_active":"2024-05-28T14:06:24.429196+0000","last_peered":"2024-05-28T14:06:24.429196+0000","last_clean":"2024-05-28T14:06:24.429196+0000","last_became_active":"2024-05-28T14:06:24.425387+0000","last_became_peered":"2024-05-28T14:06:24.425387+0000","last_unstale":"2024-05-28T14:06:24.429196+0000","last_undegraded":"2024-05-28T14:06:24.429196+0000","last_fullsized":"2024-05-28T14:06:24.429196+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:53:53.347808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415360+0000","last_change":"2024-05-28T14:06:24.415360+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415360+0000","last_undegraded":"2024-05-28T14:06:24.415360+0000","last_fullsized":"2024-05-28T14:06:24.415360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.423810+0000","last_change":"2024-05-28T14:06:24.422382+0000","last_active":"2024-05-28T14:06:24.423810+0000","last_peered":"2024-05-28T14:06:24.423810+0000","last_clean":"2024-05-28T14:06:24.423810+0000","last_became_active":"2024-05-28T14:06:24.422093+0000","last_became_peered":"2024-05-28T14:06:24.422093+0000","last_unstale":"2024-05-28T14:06:24.423810+0000","last_undegraded":"2024-05-28T14:06:24.423810+0000","last_fullsized":"2024-05-28T14:06:24.423810+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:58:13.946282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.423794+0000","last_change":"2024-05-28T14:06:24.423794+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.423794+0000","last_undegraded":"2024-05-28T14:06:24.423794+0000","last_fullsized":"2024-05-28T14:06:24.423794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.438284+0000","last_change":"2024-05-28T14:06:24.436081+0000","last_active":"2024-05-28T14:06:24.438284+0000","last_peered":"2024-05-28T14:06:24.438284+0000","last_clean":"2024-05-28T14:06:24.438284+0000","last_became_active":"2024-05-28T14:06:24.435796+0000","last_became_peered":"2024-05-28T14:06:24.435796+0000","last_unstale":"2024-05-28T14:06:24.438284+0000","last_undegraded":"2024-05-28T14:06:24.438284+0000","last_fullsized":"2024-05-28T14:06:24.438284+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:37:36.072281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.423969+0000","last_change":"2024-05-28T14:06:24.423969+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.423969+0000","last_undegraded":"2024-05-28T14:06:24.423969+0000","last_fullsized":"2024-05-28T14:06:24.423969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.426613+0000","last_change":"2024-05-28T14:06:24.421989+0000","last_active":"2024-05-28T14:06:24.426613+0000","last_peered":"2024-05-28T14:06:24.426613+0000","last_clean":"2024-05-28T14:06:24.426613+0000","last_became_active":"2024-05-28T14:06:24.421727+0000","last_became_peered":"2024-05-28T14:06:24.421727+0000","last_unstale":"2024-05-28T14:06:24.426613+0000","last_undegraded":"2024-05-28T14:06:24.426613+0000","last_fullsized":"2024-05-28T14:06:24.426613+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:35:55.179829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413607+0000","last_change":"2024-05-28T14:06:24.413607+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413607+0000","last_undegraded":"2024-05-28T14:06:24.413607+0000","last_fullsized":"2024-05-28T14:06:24.413607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.423031+0000","last_change":"2024-05-28T14:06:24.421197+0000","last_active":"2024-05-28T14:06:24.423031+0000","last_peered":"2024-05-28T14:06:24.423031+0000","last_clean":"2024-05-28T14:06:24.423031+0000","last_became_active":"2024-05-28T14:06:24.420925+0000","last_became_peered":"2024-05-28T14:06:24.420925+0000","last_unstale":"2024-05-28T14:06:24.423031+0000","last_undegraded":"2024-05-28T14:06:24.423031+0000","last_fullsized":"2024-05-28T14:06:24.423031+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:04:37.401131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.423713+0000","last_change":"2024-05-28T14:06:24.423713+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.423713+0000","last_undegraded":"2024-05-28T14:06:24.423713+0000","last_fullsized":"2024-05-28T14:06:24.423713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431171+0000","last_change":"2024-05-28T14:06:24.429639+0000","last_active":"2024-05-28T14:06:24.431171+0000","last_peered":"2024-05-28T14:06:24.431171+0000","last_clean":"2024-05-28T14:06:24.431171+0000","last_became_active":"2024-05-28T14:06:24.429133+0000","last_became_peered":"2024-05-28T14:06:24.429133+0000","last_unstale":"2024-05-28T14:06:24.431171+0000","last_undegraded":"2024-05-28T14:06:24.431171+0000","last_fullsized":"2024-05-28T14:06:24.431171+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:26.316274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.415570+0000","last_change":"2024-05-28T14:06:24.415570+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.415570+0000","last_undegraded":"2024-05-28T14:06:24.415570+0000","last_fullsized":"2024-05-28T14:06:24.415570+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.435344+0000","last_change":"2024-05-28T14:06:24.432341+0000","last_active":"2024-05-28T14:06:24.435344+0000","last_peered":"2024-05-28T14:06:24.435344+0000","last_clean":"2024-05-28T14:06:24.435344+0000","last_became_active":"2024-05-28T14:06:24.432058+0000","last_became_peered":"2024-05-28T14:06:24.432058+0000","last_unstale":"2024-05-28T14:06:24.435344+0000","last_undegraded":"2024-05-28T14:06:24.435344+0000","last_fullsized":"2024-05-28T14:06:24.435344+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:59:39.619307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414289+0000","last_change":"2024-05-28T14:06:24.414289+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414289+0000","last_undegraded":"2024-05-28T14:06:24.414289+0000","last_fullsized":"2024-05-28T14:06:24.414289+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431568+0000","last_change":"2024-05-28T14:06:24.426720+0000","last_active":"2024-05-28T14:06:24.431568+0000","last_peered":"2024-05-28T14:06:24.431568+0000","last_clean":"2024-05-28T14:06:24.431568+0000","last_became_active":"2024-05-28T14:06:24.426399+0000","last_became_peered":"2024-05-28T14:06:24.426399+0000","last_unstale":"2024-05-28T14:06:24.431568+0000","last_undegraded":"2024-05-28T14:06:24.431568+0000","last_fullsized":"2024-05-28T14:06:24.431568+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:26:45.081785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414452+0000","last_change":"2024-05-28T14:06:24.414452+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414452+0000","last_undegraded":"2024-05-28T14:06:24.414452+0000","last_fullsized":"2024-05-28T14:06:24.414452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431522+0000","last_change":"2024-05-28T14:06:24.424772+0000","last_active":"2024-05-28T14:06:24.431522+0000","last_peered":"2024-05-28T14:06:24.431522+0000","last_clean":"2024-05-28T14:06:24.431522+0000","last_became_active":"2024-05-28T14:06:24.424480+0000","last_became_peered":"2024-05-28T14:06:24.424480+0000","last_unstale":"2024-05-28T14:06:24.431522+0000","last_undegraded":"2024-05-28T14:06:24.431522+0000","last_fullsized":"2024-05-28T14:06:24.431522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:45:23.593514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413039+0000","last_change":"2024-05-28T14:06:24.413039+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413039+0000","last_undegraded":"2024-05-28T14:06:24.413039+0000","last_fullsized":"2024-05-28T14:06:24.413039+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.437107+0000","last_change":"2024-05-28T14:06:24.434783+0000","last_active":"2024-05-28T14:06:24.437107+0000","last_peered":"2024-05-28T14:06:24.437107+0000","last_clean":"2024-05-28T14:06:24.437107+0000","last_became_active":"2024-05-28T14:06:24.434490+0000","last_became_peered":"2024-05-28T14:06:24.434490+0000","last_unstale":"2024-05-28T14:06:24.437107+0000","last_undegraded":"2024-05-28T14:06:24.437107+0000","last_fullsized":"2024-05-28T14:06:24.437107+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:46:01.970253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.423255+0000","last_change":"2024-05-28T14:06:24.423255+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.423255+0000","last_undegraded":"2024-05-28T14:06:24.423255+0000","last_fullsized":"2024-05-28T14:06:24.423255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.437420+0000","last_change":"2024-05-28T14:06:24.434447+0000","last_active":"2024-05-28T14:06:24.437420+0000","last_peered":"2024-05-28T14:06:24.437420+0000","last_clean":"2024-05-28T14:06:24.437420+0000","last_became_active":"2024-05-28T14:06:24.433002+0000","last_became_peered":"2024-05-28T14:06:24.433002+0000","last_unstale":"2024-05-28T14:06:24.437420+0000","last_undegraded":"2024-05-28T14:06:24.437420+0000","last_fullsized":"2024-05-28T14:06:24.437420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:41:19.574984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412440+0000","last_change":"2024-05-28T14:06:24.412440+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412440+0000","last_undegraded":"2024-05-28T14:06:24.412440+0000","last_fullsized":"2024-05-28T14:06:24.412440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.432802+0000","last_change":"2024-05-28T14:06:24.428701+0000","last_active":"2024-05-28T14:06:24.432802+0000","last_peered":"2024-05-28T14:06:24.432802+0000","last_clean":"2024-05-28T14:06:24.432802+0000","last_became_active":"2024-05-28T14:06:24.428477+0000","last_became_peered":"2024-05-28T14:06:24.428477+0000","last_unstale":"2024-05-28T14:06:24.432802+0000","last_undegraded":"2024-05-28T14:06:24.432802+0000","last_fullsized":"2024-05-28T14:06:24.432802+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:38:32.586189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412600+0000","last_change":"2024-05-28T14:06:24.412600+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412600+0000","last_undegraded":"2024-05-28T14:06:24.412600+0000","last_fullsized":"2024-05-28T14:06:24.412600+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.438463+0000","last_change":"2024-05-28T14:06:24.437928+0000","last_active":"2024-05-28T14:06:24.438463+0000","last_peered":"2024-05-28T14:06:24.438463+0000","last_clean":"2024-05-28T14:06:24.438463+0000","last_became_active":"2024-05-28T14:06:24.437366+0000","last_became_peered":"2024-05-28T14:06:24.437366+0000","last_unstale":"2024-05-28T14:06:24.438463+0000","last_undegraded":"2024-05-28T14:06:24.438463+0000","last_fullsized":"2024-05-28T14:06:24.438463+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:08.591204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.413482+0000","last_change":"2024-05-28T14:06:24.413482+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.413482+0000","last_undegraded":"2024-05-28T14:06:24.413482+0000","last_fullsized":"2024-05-28T14:06:24.413482+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.432994+0000","last_change":"2024-05-28T14:06:24.429400+0000","last_active":"2024-05-28T14:06:24.432994+0000","last_peered":"2024-05-28T14:06:24.432994+0000","last_clean":"2024-05-28T14:06:24.432994+0000","last_became_active":"2024-05-28T14:06:24.429124+0000","last_became_peered":"2024-05-28T14:06:24.429124+0000","last_unstale":"2024-05-28T14:06:24.432994+0000","last_undegraded":"2024-05-28T14:06:24.432994+0000","last_fullsized":"2024-05-28T14:06:24.432994+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:39:32.793666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414095+0000","last_change":"2024-05-28T14:06:24.414095+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414095+0000","last_undegraded":"2024-05-28T14:06:24.414095+0000","last_fullsized":"2024-05-28T14:06:24.414095+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.422911+0000","last_change":"2024-05-28T14:06:24.420613+0000","last_active":"2024-05-28T14:06:24.422911+0000","last_peered":"2024-05-28T14:06:24.422911+0000","last_clean":"2024-05-28T14:06:24.422911+0000","last_became_active":"2024-05-28T14:06:24.420171+0000","last_became_peered":"2024-05-28T14:06:24.420171+0000","last_unstale":"2024-05-28T14:06:24.422911+0000","last_undegraded":"2024-05-28T14:06:24.422911+0000","last_fullsized":"2024-05-28T14:06:24.422911+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:49.326481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414920+0000","last_change":"2024-05-28T14:06:24.414920+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414920+0000","last_undegraded":"2024-05-28T14:06:24.414920+0000","last_fullsized":"2024-05-28T14:06:24.414920+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.430690+0000","last_change":"2024-05-28T14:06:24.427464+0000","last_active":"2024-05-28T14:06:24.430690+0000","last_peered":"2024-05-28T14:06:24.430690+0000","last_clean":"2024-05-28T14:06:24.430690+0000","last_became_active":"2024-05-28T14:06:24.427145+0000","last_became_peered":"2024-05-28T14:06:24.427145+0000","last_unstale":"2024-05-28T14:06:24.430690+0000","last_undegraded":"2024-05-28T14:06:24.430690+0000","last_fullsized":"2024-05-28T14:06:24.430690+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:06:23.658146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417284+0000","last_change":"2024-05-28T14:06:24.417284+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417284+0000","last_undegraded":"2024-05-28T14:06:24.417284+0000","last_fullsized":"2024-05-28T14:06:24.417284+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.436513+0000","last_change":"2024-05-28T14:06:24.432369+0000","last_active":"2024-05-28T14:06:24.436513+0000","last_peered":"2024-05-28T14:06:24.436513+0000","last_clean":"2024-05-28T14:06:24.436513+0000","last_became_active":"2024-05-28T14:06:24.431992+0000","last_became_peered":"2024-05-28T14:06:24.431992+0000","last_unstale":"2024-05-28T14:06:24.436513+0000","last_undegraded":"2024-05-28T14:06:24.436513+0000","last_fullsized":"2024-05-28T14:06:24.436513+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:42:55.217345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.416075+0000","last_change":"2024-05-28T14:06:24.416075+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.416075+0000","last_undegraded":"2024-05-28T14:06:24.416075+0000","last_fullsized":"2024-05-28T14:06:24.416075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431126+0000","last_change":"2024-05-28T14:06:24.427952+0000","last_active":"2024-05-28T14:06:24.431126+0000","last_peered":"2024-05-28T14:06:24.431126+0000","last_clean":"2024-05-28T14:06:24.431126+0000","last_became_active":"2024-05-28T14:06:24.427511+0000","last_became_peered":"2024-05-28T14:06:24.427511+0000","last_unstale":"2024-05-28T14:06:24.431126+0000","last_undegraded":"2024-05-28T14:06:24.431126+0000","last_fullsized":"2024-05-28T14:06:24.431126+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:03:34.586234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.425338+0000","last_change":"2024-05-28T14:06:24.425338+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.425338+0000","last_undegraded":"2024-05-28T14:06:24.425338+0000","last_fullsized":"2024-05-28T14:06:24.425338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.430060+0000","last_change":"2024-05-28T14:06:24.427264+0000","last_active":"2024-05-28T14:06:24.430060+0000","last_peered":"2024-05-28T14:06:24.430060+0000","last_clean":"2024-05-28T14:06:24.430060+0000","last_became_active":"2024-05-28T14:06:24.426662+0000","last_became_peered":"2024-05-28T14:06:24.426662+0000","last_unstale":"2024-05-28T14:06:24.430060+0000","last_undegraded":"2024-05-28T14:06:24.430060+0000","last_fullsized":"2024-05-28T14:06:24.430060+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:28:56.343364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.424918+0000","last_change":"2024-05-28T14:06:24.424918+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.424918+0000","last_undegraded":"2024-05-28T14:06:24.424918+0000","last_fullsized":"2024-05-28T14:06:24.424918+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.432299+0000","last_change":"2024-05-28T14:06:24.431741+0000","last_active":"2024-05-28T14:06:24.432299+0000","last_peered":"2024-05-28T14:06:24.432299+0000","last_clean":"2024-05-28T14:06:24.432299+0000","last_became_active":"2024-05-28T14:06:24.431388+0000","last_became_peered":"2024-05-28T14:06:24.431388+0000","last_unstale":"2024-05-28T14:06:24.432299+0000","last_undegraded":"2024-05-28T14:06:24.432299+0000","last_fullsized":"2024-05-28T14:06:24.432299+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:11:06.256080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412880+0000","last_change":"2024-05-28T14:06:24.412880+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412880+0000","last_undegraded":"2024-05-28T14:06:24.412880+0000","last_fullsized":"2024-05-28T14:06:24.412880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.424536+0000","last_change":"2024-05-28T14:06:24.420374+0000","last_active":"2024-05-28T14:06:24.424536+0000","last_peered":"2024-05-28T14:06:24.424536+0000","last_clean":"2024-05-28T14:06:24.424536+0000","last_became_active":"2024-05-28T14:06:24.420044+0000","last_became_peered":"2024-05-28T14:06:24.420044+0000","last_unstale":"2024-05-28T14:06:24.424536+0000","last_undegraded":"2024-05-28T14:06:24.424536+0000","last_fullsized":"2024-05-28T14:06:24.424536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:29:46.118850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.422978+0000","last_change":"2024-05-28T14:06:24.422978+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.422978+0000","last_undegraded":"2024-05-28T14:06:24.422978+0000","last_fullsized":"2024-05-28T14:06:24.422978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.425967+0000","last_change":"2024-05-28T14:06:24.422804+0000","last_active":"2024-05-28T14:06:24.425967+0000","last_peered":"2024-05-28T14:06:24.425967+0000","last_clean":"2024-05-28T14:06:24.425967+0000","last_became_active":"2024-05-28T14:06:24.422468+0000","last_became_peered":"2024-05-28T14:06:24.422468+0000","last_unstale":"2024-05-28T14:06:24.425967+0000","last_undegraded":"2024-05-28T14:06:24.425967+0000","last_fullsized":"2024-05-28T14:06:24.425967+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:46:01.970253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.412444+0000","last_change":"2024-05-28T14:06:24.412444+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.412444+0000","last_undegraded":"2024-05-28T14:06:24.412444+0000","last_fullsized":"2024-05-28T14:06:24.412444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.433548+0000","last_change":"2024-05-28T14:06:24.429508+0000","last_active":"2024-05-28T14:06:24.433548+0000","last_peered":"2024-05-28T14:06:24.433548+0000","last_clean":"2024-05-28T14:06:24.433548+0000","last_became_active":"2024-05-28T14:06:24.429101+0000","last_became_peered":"2024-05-28T14:06:24.429101+0000","last_unstale":"2024-05-28T14:06:24.433548+0000","last_undegraded":"2024-05-28T14:06:24.433548+0000","last_fullsized":"2024-05-28T14:06:24.433548+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:15.013592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.417313+0000","last_change":"2024-05-28T14:06:24.417313+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.417313+0000","last_undegraded":"2024-05-28T14:06:24.417313+0000","last_fullsized":"2024-05-28T14:06:24.417313+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.431063+0000","last_change":"2024-05-28T14:06:24.427387+0000","last_active":"2024-05-28T14:06:24.431063+0000","last_peered":"2024-05-28T14:06:24.431063+0000","last_clean":"2024-05-28T14:06:24.431063+0000","last_became_active":"2024-05-28T14:06:24.427106+0000","last_became_peered":"2024-05-28T14:06:24.427106+0000","last_unstale":"2024-05-28T14:06:24.431063+0000","last_undegraded":"2024-05-28T14:06:24.431063+0000","last_fullsized":"2024-05-28T14:06:24.431063+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:19:02.178759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-28T14:06:24.414347+0000","last_change":"2024-05-28T14:06:24.414347+0000","last_active":"2024-05-28T14:06:24.399936+0000","last_peered":"2024-05-28T14:06:24.399936+0000","last_clean":"2024-05-28T14:06:24.399936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:06:24.414347+0000","last_undegraded":"2024-05-28T14:06:24.414347+0000","last_fullsized":"2024-05-28T14:06:24.414347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.436299+0000","last_change":"2024-05-28T14:06:24.433815+0000","last_active":"2024-05-28T14:06:24.436299+0000","last_peered":"2024-05-28T14:06:24.436299+0000","last_clean":"2024-05-28T14:06:24.436299+0000","last_became_active":"2024-05-28T14:06:24.433524+0000","last_became_peered":"2024-05-28T14:06:24.433524+0000","last_unstale":"2024-05-28T14:06:24.436299+0000","last_undegraded":"2024-05-28T14:06:24.436299+0000","last_fullsized":"2024-05-28T14:06:24.436299+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:04:37.401131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":264,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-28T14:06:24.429488+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:06:24.429488+0000","last_peered":"2024-05-28T14:06:24.429488+0000","last_clean":"2024-05-28T14:06:24.429488+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:06:24.429488+0000","last_undegraded":"2024-05-28T14:06:24.429488+0000","last_fullsized":"2024-05-28T14:06:24.429488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574949,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58988,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312852,"statfs":{"total":96636764160,"available":96576360448,"internally_reserved":0,"allocated":1470464,"data_stored":961924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54236,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317604,"statfs":{"total":96636764160,"available":96581226496,"internally_reserved":0,"allocated":864256,"data_stored":365253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54248,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317592,"statfs":{"total":96636764160,"available":96581214208,"internally_reserved":0,"allocated":876544,"data_stored":371556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58392,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313448,"statfs":{"total":96636764160,"available":96576970752,"internally_reserved":0,"allocated":864256,"data_stored":365253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45928,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325912,"statfs":{"total":96636764160,"available":96589733888,"internally_reserved":0,"allocated":876544,"data_stored":371556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54236,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317604,"statfs":{"total":96636764160,"available":96581226496,"internally_reserved":0,"allocated":864256,"data_stored":365253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62564,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309276,"statfs":{"total":96636764160,"available":96572698624,"internally_reserved":0,"allocated":876544,"data_stored":371556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574950,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63148,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308692,"statfs":{"total":96636764160,"available":96572100608,"internally_reserved":0,"allocated":1470464,"data_stored":961924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:06:30.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2024-05-28T14:06:30.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.424413+0000', 'last_change': '2024-05-28T14:06:24.424413+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.424413+0000', 'last_undegraded': '2024-05-28T14:06:24.424413+0000', 'last_fullsized': '2024-05-28T14:06:24.424413+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:06:30.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2024-05-28T14:06:30.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.421572+0000', 'last_change': '2024-05-28T14:06:24.421572+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.421572+0000', 'last_undegraded': '2024-05-28T14:06:24.421572+0000', 'last_fullsized': '2024-05-28T14:06:24.421572+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:06:30.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2024-05-28T14:06:30.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.415723+0000', 'last_change': '2024-05-28T14:06:24.415723+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.415723+0000', 'last_undegraded': '2024-05-28T14:06:24.415723+0000', 'last_fullsized': '2024-05-28T14:06:24.415723+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:06:30.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2024-05-28T14:06:30.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.418596+0000', 'last_change': '2024-05-28T14:06:24.418596+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.418596+0000', 'last_undegraded': '2024-05-28T14:06:24.418596+0000', 'last_fullsized': '2024-05-28T14:06:24.418596+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:06:30.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2024-05-28T14:06:30.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.417356+0000', 'last_change': '2024-05-28T14:06:24.417356+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.417356+0000', 'last_undegraded': '2024-05-28T14:06:24.417356+0000', 'last_fullsized': '2024-05-28T14:06:24.417356+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:06:30.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2024-05-28T14:06:30.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.417815+0000', 'last_change': '2024-05-28T14:06:24.417815+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.417815+0000', 'last_undegraded': '2024-05-28T14:06:24.417815+0000', 'last_fullsized': '2024-05-28T14:06:24.417815+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:06:30.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2024-05-28T14:06:30.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.416338+0000', 'last_change': '2024-05-28T14:06:24.416338+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.416338+0000', 'last_undegraded': '2024-05-28T14:06:24.416338+0000', 'last_fullsized': '2024-05-28T14:06:24.416338+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:06:30.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2024-05-28T14:06:30.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.425317+0000', 'last_change': '2024-05-28T14:06:24.425317+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.425317+0000', 'last_undegraded': '2024-05-28T14:06:24.425317+0000', 'last_fullsized': '2024-05-28T14:06:24.425317+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:06:30.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2024-05-28T14:06:30.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.417044+0000', 'last_change': '2024-05-28T14:06:24.417044+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.417044+0000', 'last_undegraded': '2024-05-28T14:06:24.417044+0000', 'last_fullsized': '2024-05-28T14:06:24.417044+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:06:30.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2024-05-28T14:06:30.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.413819+0000', 'last_change': '2024-05-28T14:06:24.413819+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.413819+0000', 'last_undegraded': '2024-05-28T14:06:24.413819+0000', 'last_fullsized': '2024-05-28T14:06:24.413819+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:06:30.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2024-05-28T14:06:30.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.425975+0000', 'last_change': '2024-05-28T14:06:24.425975+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.425975+0000', 'last_undegraded': '2024-05-28T14:06:24.425975+0000', 'last_fullsized': '2024-05-28T14:06:24.425975+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:06:30.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2024-05-28T14:06:30.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.418211+0000', 'last_change': '2024-05-28T14:06:24.418211+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.418211+0000', 'last_undegraded': '2024-05-28T14:06:24.418211+0000', 'last_fullsized': '2024-05-28T14:06:24.418211+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:06:30.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2024-05-28T14:06:30.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.419930+0000', 'last_change': '2024-05-28T14:06:24.419930+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.419930+0000', 'last_undegraded': '2024-05-28T14:06:24.419930+0000', 'last_fullsized': '2024-05-28T14:06:24.419930+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:06:30.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2024-05-28T14:06:30.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.414944+0000', 'last_change': '2024-05-28T14:06:24.414944+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.414944+0000', 'last_undegraded': '2024-05-28T14:06:24.414944+0000', 'last_fullsized': '2024-05-28T14:06:24.414944+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:06:30.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2024-05-28T14:06:30.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.415671+0000', 'last_change': '2024-05-28T14:06:24.415671+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.415671+0000', 'last_undegraded': '2024-05-28T14:06:24.415671+0000', 'last_fullsized': '2024-05-28T14:06:24.415671+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:06:30.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2024-05-28T14:06:30.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.412671+0000', 'last_change': '2024-05-28T14:06:24.412671+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.412671+0000', 'last_undegraded': '2024-05-28T14:06:24.412671+0000', 'last_fullsized': '2024-05-28T14:06:24.412671+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:06:30.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2024-05-28T14:06:30.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.414216+0000', 'last_change': '2024-05-28T14:06:24.414216+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.414216+0000', 'last_undegraded': '2024-05-28T14:06:24.414216+0000', 'last_fullsized': '2024-05-28T14:06:24.414216+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:06:30.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2024-05-28T14:06:30.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.412534+0000', 'last_change': '2024-05-28T14:06:24.412534+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.412534+0000', 'last_undegraded': '2024-05-28T14:06:24.412534+0000', 'last_fullsized': '2024-05-28T14:06:24.412534+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:06:30.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2024-05-28T14:06:30.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.427380+0000', 'last_change': '2024-05-28T14:06:24.427380+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.427380+0000', 'last_undegraded': '2024-05-28T14:06:24.427380+0000', 'last_fullsized': '2024-05-28T14:06:24.427380+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:06:30.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2024-05-28T14:06:30.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.420242+0000', 'last_change': '2024-05-28T14:06:24.420242+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.420242+0000', 'last_undegraded': '2024-05-28T14:06:24.420242+0000', 'last_fullsized': '2024-05-28T14:06:24.420242+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:06:30.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2024-05-28T14:06:30.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.413445+0000', 'last_change': '2024-05-28T14:06:24.413445+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.413445+0000', 'last_undegraded': '2024-05-28T14:06:24.413445+0000', 'last_fullsized': '2024-05-28T14:06:24.413445+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:06:30.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2024-05-28T14:06:30.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.413956+0000', 'last_change': '2024-05-28T14:06:24.413956+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.413956+0000', 'last_undegraded': '2024-05-28T14:06:24.413956+0000', 'last_fullsized': '2024-05-28T14:06:24.413956+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:06:30.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2024-05-28T14:06:30.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.422591+0000', 'last_change': '2024-05-28T14:06:24.422591+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.422591+0000', 'last_undegraded': '2024-05-28T14:06:24.422591+0000', 'last_fullsized': '2024-05-28T14:06:24.422591+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:06:30.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2024-05-28T14:06:30.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.421618+0000', 'last_change': '2024-05-28T14:06:24.421618+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.421618+0000', 'last_undegraded': '2024-05-28T14:06:24.421618+0000', 'last_fullsized': '2024-05-28T14:06:24.421618+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:06:30.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2024-05-28T14:06:30.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.413641+0000', 'last_change': '2024-05-28T14:06:24.413641+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.413641+0000', 'last_undegraded': '2024-05-28T14:06:24.413641+0000', 'last_fullsized': '2024-05-28T14:06:24.413641+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:06:30.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2024-05-28T14:06:30.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.415922+0000', 'last_change': '2024-05-28T14:06:24.415922+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.415922+0000', 'last_undegraded': '2024-05-28T14:06:24.415922+0000', 'last_fullsized': '2024-05-28T14:06:24.415922+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:06:30.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2024-05-28T14:06:30.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.415300+0000', 'last_change': '2024-05-28T14:06:24.415300+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.415300+0000', 'last_undegraded': '2024-05-28T14:06:24.415300+0000', 'last_fullsized': '2024-05-28T14:06:24.415300+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:06:30.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2024-05-28T14:06:30.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.417303+0000', 'last_change': '2024-05-28T14:06:24.417303+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.417303+0000', 'last_undegraded': '2024-05-28T14:06:24.417303+0000', 'last_fullsized': '2024-05-28T14:06:24.417303+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:06:30.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2024-05-28T14:06:30.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.413946+0000', 'last_change': '2024-05-28T14:06:24.413946+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.413946+0000', 'last_undegraded': '2024-05-28T14:06:24.413946+0000', 'last_fullsized': '2024-05-28T14:06:24.413946+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:06:30.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2024-05-28T14:06:30.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.416788+0000', 'last_change': '2024-05-28T14:06:24.416788+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.416788+0000', 'last_undegraded': '2024-05-28T14:06:24.416788+0000', 'last_fullsized': '2024-05-28T14:06:24.416788+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:06:30.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2024-05-28T14:06:30.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.415329+0000', 'last_change': '2024-05-28T14:06:24.415329+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.415329+0000', 'last_undegraded': '2024-05-28T14:06:24.415329+0000', 'last_fullsized': '2024-05-28T14:06:24.415329+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:06:30.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2024-05-28T14:06:30.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.416575+0000', 'last_change': '2024-05-28T14:06:24.416575+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.416575+0000', 'last_undegraded': '2024-05-28T14:06:24.416575+0000', 'last_fullsized': '2024-05-28T14:06:24.416575+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:06:30.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2024-05-28T14:06:30.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.422580+0000', 'last_change': '2024-05-28T14:06:24.422580+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.422580+0000', 'last_undegraded': '2024-05-28T14:06:24.422580+0000', 'last_fullsized': '2024-05-28T14:06:24.422580+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:06:30.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2024-05-28T14:06:30.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.413739+0000', 'last_change': '2024-05-28T14:06:24.413739+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.413739+0000', 'last_undegraded': '2024-05-28T14:06:24.413739+0000', 'last_fullsized': '2024-05-28T14:06:24.413739+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:06:30.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2024-05-28T14:06:30.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.424448+0000', 'last_change': '2024-05-28T14:06:24.424448+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.424448+0000', 'last_undegraded': '2024-05-28T14:06:24.424448+0000', 'last_fullsized': '2024-05-28T14:06:24.424448+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:06:30.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2024-05-28T14:06:30.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.425545+0000', 'last_change': '2024-05-28T14:06:24.425545+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.425545+0000', 'last_undegraded': '2024-05-28T14:06:24.425545+0000', 'last_fullsized': '2024-05-28T14:06:24.425545+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:06:30.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2024-05-28T14:06:30.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.416321+0000', 'last_change': '2024-05-28T14:06:24.416321+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.416321+0000', 'last_undegraded': '2024-05-28T14:06:24.416321+0000', 'last_fullsized': '2024-05-28T14:06:24.416321+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:06:30.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2024-05-28T14:06:30.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.412777+0000', 'last_change': '2024-05-28T14:06:24.412777+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.412777+0000', 'last_undegraded': '2024-05-28T14:06:24.412777+0000', 'last_fullsized': '2024-05-28T14:06:24.412777+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:06:30.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2024-05-28T14:06:30.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.413731+0000', 'last_change': '2024-05-28T14:06:24.413731+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.413731+0000', 'last_undegraded': '2024-05-28T14:06:24.413731+0000', 'last_fullsized': '2024-05-28T14:06:24.413731+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:06:30.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2024-05-28T14:06:30.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.415270+0000', 'last_change': '2024-05-28T14:06:24.415270+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.415270+0000', 'last_undegraded': '2024-05-28T14:06:24.415270+0000', 'last_fullsized': '2024-05-28T14:06:24.415270+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:06:30.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2024-05-28T14:06:30.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.415360+0000', 'last_change': '2024-05-28T14:06:24.415360+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.415360+0000', 'last_undegraded': '2024-05-28T14:06:24.415360+0000', 'last_fullsized': '2024-05-28T14:06:24.415360+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:06:30.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2024-05-28T14:06:30.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.423794+0000', 'last_change': '2024-05-28T14:06:24.423794+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.423794+0000', 'last_undegraded': '2024-05-28T14:06:24.423794+0000', 'last_fullsized': '2024-05-28T14:06:24.423794+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:06:30.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2024-05-28T14:06:30.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.423969+0000', 'last_change': '2024-05-28T14:06:24.423969+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.423969+0000', 'last_undegraded': '2024-05-28T14:06:24.423969+0000', 'last_fullsized': '2024-05-28T14:06:24.423969+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:06:30.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2024-05-28T14:06:30.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.413607+0000', 'last_change': '2024-05-28T14:06:24.413607+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.413607+0000', 'last_undegraded': '2024-05-28T14:06:24.413607+0000', 'last_fullsized': '2024-05-28T14:06:24.413607+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:06:30.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2024-05-28T14:06:30.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.423713+0000', 'last_change': '2024-05-28T14:06:24.423713+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.423713+0000', 'last_undegraded': '2024-05-28T14:06:24.423713+0000', 'last_fullsized': '2024-05-28T14:06:24.423713+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:06:30.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2024-05-28T14:06:30.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.415570+0000', 'last_change': '2024-05-28T14:06:24.415570+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.415570+0000', 'last_undegraded': '2024-05-28T14:06:24.415570+0000', 'last_fullsized': '2024-05-28T14:06:24.415570+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:06:30.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2024-05-28T14:06:30.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.414289+0000', 'last_change': '2024-05-28T14:06:24.414289+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.414289+0000', 'last_undegraded': '2024-05-28T14:06:24.414289+0000', 'last_fullsized': '2024-05-28T14:06:24.414289+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:06:30.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2024-05-28T14:06:30.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.414452+0000', 'last_change': '2024-05-28T14:06:24.414452+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.414452+0000', 'last_undegraded': '2024-05-28T14:06:24.414452+0000', 'last_fullsized': '2024-05-28T14:06:24.414452+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:06:30.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2024-05-28T14:06:30.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.413039+0000', 'last_change': '2024-05-28T14:06:24.413039+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.413039+0000', 'last_undegraded': '2024-05-28T14:06:24.413039+0000', 'last_fullsized': '2024-05-28T14:06:24.413039+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:06:30.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2024-05-28T14:06:30.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.423255+0000', 'last_change': '2024-05-28T14:06:24.423255+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.423255+0000', 'last_undegraded': '2024-05-28T14:06:24.423255+0000', 'last_fullsized': '2024-05-28T14:06:24.423255+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:06:30.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2024-05-28T14:06:30.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.412440+0000', 'last_change': '2024-05-28T14:06:24.412440+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.412440+0000', 'last_undegraded': '2024-05-28T14:06:24.412440+0000', 'last_fullsized': '2024-05-28T14:06:24.412440+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:06:30.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2024-05-28T14:06:30.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.412600+0000', 'last_change': '2024-05-28T14:06:24.412600+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.412600+0000', 'last_undegraded': '2024-05-28T14:06:24.412600+0000', 'last_fullsized': '2024-05-28T14:06:24.412600+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:06:30.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2024-05-28T14:06:30.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.413482+0000', 'last_change': '2024-05-28T14:06:24.413482+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.413482+0000', 'last_undegraded': '2024-05-28T14:06:24.413482+0000', 'last_fullsized': '2024-05-28T14:06:24.413482+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:06:30.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2024-05-28T14:06:30.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.414095+0000', 'last_change': '2024-05-28T14:06:24.414095+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.414095+0000', 'last_undegraded': '2024-05-28T14:06:24.414095+0000', 'last_fullsized': '2024-05-28T14:06:24.414095+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:06:30.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2024-05-28T14:06:30.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.414920+0000', 'last_change': '2024-05-28T14:06:24.414920+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.414920+0000', 'last_undegraded': '2024-05-28T14:06:24.414920+0000', 'last_fullsized': '2024-05-28T14:06:24.414920+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:06:30.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2024-05-28T14:06:30.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.417284+0000', 'last_change': '2024-05-28T14:06:24.417284+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.417284+0000', 'last_undegraded': '2024-05-28T14:06:24.417284+0000', 'last_fullsized': '2024-05-28T14:06:24.417284+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:06:30.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2024-05-28T14:06:30.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.416075+0000', 'last_change': '2024-05-28T14:06:24.416075+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.416075+0000', 'last_undegraded': '2024-05-28T14:06:24.416075+0000', 'last_fullsized': '2024-05-28T14:06:24.416075+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:06:30.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2024-05-28T14:06:30.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.425338+0000', 'last_change': '2024-05-28T14:06:24.425338+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.425338+0000', 'last_undegraded': '2024-05-28T14:06:24.425338+0000', 'last_fullsized': '2024-05-28T14:06:24.425338+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:06:30.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2024-05-28T14:06:30.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.424918+0000', 'last_change': '2024-05-28T14:06:24.424918+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.424918+0000', 'last_undegraded': '2024-05-28T14:06:24.424918+0000', 'last_fullsized': '2024-05-28T14:06:24.424918+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:06:30.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2024-05-28T14:06:30.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.412880+0000', 'last_change': '2024-05-28T14:06:24.412880+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.412880+0000', 'last_undegraded': '2024-05-28T14:06:24.412880+0000', 'last_fullsized': '2024-05-28T14:06:24.412880+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:06:30.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2024-05-28T14:06:30.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.422978+0000', 'last_change': '2024-05-28T14:06:24.422978+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.422978+0000', 'last_undegraded': '2024-05-28T14:06:24.422978+0000', 'last_fullsized': '2024-05-28T14:06:24.422978+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:06:30.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2024-05-28T14:06:30.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.412444+0000', 'last_change': '2024-05-28T14:06:24.412444+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.412444+0000', 'last_undegraded': '2024-05-28T14:06:24.412444+0000', 'last_fullsized': '2024-05-28T14:06:24.412444+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:06:30.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2024-05-28T14:06:30.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.417313+0000', 'last_change': '2024-05-28T14:06:24.417313+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.417313+0000', 'last_undegraded': '2024-05-28T14:06:24.417313+0000', 'last_fullsized': '2024-05-28T14:06:24.417313+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:06:30.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2024-05-28T14:06:30.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:06:24.414347+0000', 'last_change': '2024-05-28T14:06:24.414347+0000', 'last_active': '2024-05-28T14:06:24.399936+0000', 'last_peered': '2024-05-28T14:06:24.399936+0000', 'last_clean': '2024-05-28T14:06:24.399936+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:06:24.414347+0000', 'last_undegraded': '2024-05-28T14:06:24.414347+0000', 'last_fullsized': '2024-05-28T14:06:24.414347+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'last_clean_scrub_stamp': '2024-05-28T14:06:24.399936+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:06:30.332 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-28T14:06:30.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.510+0000 7fb788200640 1 Processor -- start 2024-05-28T14:06:30.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.510+0000 7fb788200640 1 -- start start 2024-05-28T14:06:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb788200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7800cd090 0x7fb7800cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb788200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb7800b5760 0x7fb7800b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb788200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7800d0760 con 0x7fb78015fd00 2024-05-28T14:06:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb788200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb7800b52d0 con 0x7fb7800b5760 2024-05-28T14:06:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb788200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb7800d0a30 con 0x7fb7800cd090 2024-05-28T14:06:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb785800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb78015fd00 0x7fb7800d0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42838/0 (socket says 172.21.15.42:42838) 2024-05-28T14:06:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb785800640 1 -- 172.21.15.42:0/819513098 learned_addr learned my addr 172.21.15.42:0/819513098 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb786200640 1 --2- 172.21.15.42:0/819513098 >> v2:172.21.15.63:3300/0 conn(0x7fb7800b5760 0x7fb7800b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb784e00640 1 --2- 172.21.15.42:0/819513098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7800cd090 0x7fb7800cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb77fe00640 1 -- 172.21.15.42:0/819513098 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 745958673 0 0) 0x7fb7800d0760 con 0x7fb78015fd00 2024-05-28T14:06:30.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb77fe00640 1 -- 172.21.15.42:0/819513098 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb768003530 con 0x7fb78015fd00 2024-05-28T14:06:30.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb786200640 1 -- 172.21.15.42:0/819513098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7800cd090 msgr2=0x7fb7800cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:30.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb786200640 1 --2- 172.21.15.42:0/819513098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7800cd090 0x7fb7800cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb786200640 1 -- 172.21.15.42:0/819513098 >> v1:172.21.15.42:6789/0 conn(0x7fb78015fd00 legacy=0x7fb7800d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb786200640 1 -- 172.21.15.42:0/819513098 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7800d0d10 con 0x7fb7800b5760 2024-05-28T14:06:30.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.514+0000 7fb784e00640 1 --2- 172.21.15.42:0/819513098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7800cd090 0x7fb7800cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:06:30.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb786200640 1 --2- 172.21.15.42:0/819513098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7800b5760 0x7fb7800b5bf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb778009d90 tx=0x7fb77801c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a25495c7b51b15de server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:30.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb77fe00640 1 -- 172.21.15.42:0/819513098 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7780b2070 con 0x7fb7800b5760 2024-05-28T14:06:30.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb77fe00640 1 -- 172.21.15.42:0/819513098 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7780a5a00 con 0x7fb7800b5760 2024-05-28T14:06:30.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb77fe00640 1 -- 172.21.15.42:0/819513098 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7780a5dd0 con 0x7fb7800b5760 2024-05-28T14:06:30.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 -- 172.21.15.42:0/819513098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7800b5760 msgr2=0x7fb7800b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 --2- 172.21.15.42:0/819513098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7800b5760 0x7fb7800b5bf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb778009d90 tx=0x7fb77801c940 comp rx=0 tx=0).stop 2024-05-28T14:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 -- 172.21.15.42:0/819513098 shutdown_connections 2024-05-28T14:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 --2- 172.21.15.42:0/819513098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7800b5760 0x7fb7800b5bf0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 --2- 172.21.15.42:0/819513098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7800cd090 0x7fb7800cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 -- 172.21.15.42:0/819513098 >> 172.21.15.42:0/819513098 conn(0x7fb7800b3f70 msgr2=0x7fb7800b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 -- 172.21.15.42:0/819513098 shutdown_connections 2024-05-28T14:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 -- 172.21.15.42:0/819513098 wait complete. 2024-05-28T14:06:30.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 Processor -- start 2024-05-28T14:06:30.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 -- start start 2024-05-28T14:06:30.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7800b5760 0x7fb780151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7800cd090 0x7fb78014ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78015fd00 0x7fb7800c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb780160260 con 0x7fb78015fd00 2024-05-28T14:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7801600e0 con 0x7fb7800cd090 2024-05-28T14:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.518+0000 7fb788200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb7800b34d0 con 0x7fb7800b5760 2024-05-28T14:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb786200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78015fd00 0x7fb7800c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb786200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78015fd00 0x7fb7800c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53182/0 (socket says 172.21.15.42:53182) 2024-05-28T14:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb786200640 1 -- 172.21.15.42:0/4034832578 learned_addr learned my addr 172.21.15.42:0/4034832578 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb785800640 1 --2- 172.21.15.42:0/4034832578 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7800b5760 0x7fb780151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb784e00640 1 --2- 172.21.15.42:0/4034832578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7800cd090 0x7fb78014ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb786200640 1 -- 172.21.15.42:0/4034832578 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7800b5760 msgr2=0x7fb780151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb786200640 1 --2- 172.21.15.42:0/4034832578 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7800b5760 0x7fb780151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb786200640 1 -- 172.21.15.42:0/4034832578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7800cd090 msgr2=0x7fb78014ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb786200640 1 --2- 172.21.15.42:0/4034832578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7800cd090 0x7fb78014ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb786200640 1 -- 172.21.15.42:0/4034832578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb780151c30 con 0x7fb78015fd00 2024-05-28T14:06:30.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb786200640 1 --2- 172.21.15.42:0/4034832578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78015fd00 0x7fb7800c9c10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb7780b0040 tx=0x7fb7780196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:30.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb77d600640 1 -- 172.21.15.42:0/4034832578 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7780b2050 con 0x7fb78015fd00 2024-05-28T14:06:30.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb77d600640 1 -- 172.21.15.42:0/4034832578 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7780be070 con 0x7fb78015fd00 2024-05-28T14:06:30.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb77d600640 1 -- 172.21.15.42:0/4034832578 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7780ada90 con 0x7fb78015fd00 2024-05-28T14:06:30.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb788200640 1 -- 172.21.15.42:0/4034832578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb780151f10 con 0x7fb78015fd00 2024-05-28T14:06:30.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb788200640 1 -- 172.21.15.42:0/4034832578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7800c96f0 con 0x7fb78015fd00 2024-05-28T14:06:30.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb788200640 1 -- 172.21.15.42:0/4034832578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb780152430 con 0x7fb78015fd00 2024-05-28T14:06:30.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.522+0000 7fb77d600640 1 -- 172.21.15.42:0/4034832578 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7fb7800b6940 con 0x7fb78015fd00 2024-05-28T14:06:30.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.526+0000 7fb77d600640 1 --2- 172.21.15.42:0/4034832578 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb760069070 0x7fb76006b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.526+0000 7fb77d600640 1 -- 172.21.15.42:0/4034832578 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7fb778120440 con 0x7fb78015fd00 2024-05-28T14:06:30.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.526+0000 7fb785800640 1 --2- 172.21.15.42:0/4034832578 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb760069070 0x7fb76006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.526+0000 7fb785800640 1 --2- 172.21.15.42:0/4034832578 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb760069070 0x7fb76006b530 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb774005890 tx=0x7fb774009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:30.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.530+0000 7fb77d600640 1 -- 172.21.15.42:0/4034832578 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb780152430 con 0x7fb78015fd00 2024-05-28T14:06:30.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.750+0000 7fb788200640 1 -- 172.21.15.42:0/4034832578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb780068a90 con 0x7fb78015fd00 2024-05-28T14:06:30.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.750+0000 7fb77d600640 1 -- 172.21.15.42:0/4034832578 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1035 (secure 0 0 0) 0x7fb778016560 con 0x7fb78015fd00 2024-05-28T14:06:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:30.738 INFO:teuthology.orchestra.run.smithi042.stdout:{"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":512,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":74,"num_osds":8,"num_up_osds":8,"osd_up_since":1716904689,"num_in_osds":8,"osd_in_since":1716904680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":462581760,"bytes_avail":772631531520,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":46,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":7083}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-05-28T14:05:28.979186+0000","services":{}},"progress_events":{}} 2024-05-28T14:06:30.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.754+0000 7fb788200640 1 -- 172.21.15.42:0/4034832578 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb760069070 msgr2=0x7fb76006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.754+0000 7fb788200640 1 --2- 172.21.15.42:0/4034832578 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb760069070 0x7fb76006b530 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb774005890 tx=0x7fb774009040 comp rx=0 tx=0).stop 2024-05-28T14:06:30.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.754+0000 7fb788200640 1 -- 172.21.15.42:0/4034832578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78015fd00 msgr2=0x7fb7800c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.754+0000 7fb788200640 1 --2- 172.21.15.42:0/4034832578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78015fd00 0x7fb7800c9c10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb7780b0040 tx=0x7fb7780196e0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.754+0000 7fb788200640 1 -- 172.21.15.42:0/4034832578 shutdown_connections 2024-05-28T14:06:30.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.754+0000 7fb788200640 1 --2- 172.21.15.42:0/4034832578 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb760069070 0x7fb76006b530 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.754+0000 7fb788200640 1 --2- 172.21.15.42:0/4034832578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78015fd00 0x7fb7800c9c10 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.754+0000 7fb788200640 1 --2- 172.21.15.42:0/4034832578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7800cd090 0x7fb78014ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.754+0000 7fb788200640 1 --2- 172.21.15.42:0/4034832578 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7800b5760 0x7fb780151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.754+0000 7fb788200640 1 -- 172.21.15.42:0/4034832578 >> 172.21.15.42:0/4034832578 conn(0x7fb7800b3f70 msgr2=0x7fb7800b7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:30.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.754+0000 7fb788200640 1 -- 172.21.15.42:0/4034832578 shutdown_connections 2024-05-28T14:06:30.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.754+0000 7fb788200640 1 -- 172.21.15.42:0/4034832578 wait complete. 2024-05-28T14:06:30.760 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-28T14:06:30.760 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:06:30.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.914+0000 7eff64e00640 1 Processor -- start 2024-05-28T14:06:30.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.914+0000 7eff64e00640 1 -- start start 2024-05-28T14:06:30.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff64e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7eff600d0420 0x7eff6015ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff64e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff60067810 0x7eff600b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff64e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7eff600b4110 0x7eff60162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff64e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7eff6006eaa0 con 0x7eff600d0420 2024-05-28T14:06:30.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff64e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7eff6006ed80 con 0x7eff600b4110 2024-05-28T14:06:30.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff64e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7eff6006f060 con 0x7eff60067810 2024-05-28T14:06:30.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff5de00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7eff600d0420 0x7eff6015ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff5de00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7eff600d0420 0x7eff6015ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53186/0 (socket says 172.21.15.42:53186) 2024-05-28T14:06:30.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff5de00640 1 -- 172.21.15.42:0/305400487 learned_addr learned my addr 172.21.15.42:0/305400487 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:30.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff55400640 1 --2- 172.21.15.42:0/305400487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff60067810 0x7eff600b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff5e800640 1 --2- 172.21.15.42:0/305400487 >> v2:172.21.15.63:3300/0 conn(0x7eff600b4110 0x7eff60162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff5de00640 1 -- 172.21.15.42:0/305400487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff60067810 msgr2=0x7eff600b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:30.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff5de00640 1 --2- 172.21.15.42:0/305400487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff60067810 0x7eff600b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff5de00640 1 -- 172.21.15.42:0/305400487 >> v2:172.21.15.63:3300/0 conn(0x7eff600b4110 msgr2=0x7eff60162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:30.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff5de00640 1 --2- 172.21.15.42:0/305400487 >> v2:172.21.15.63:3300/0 conn(0x7eff600b4110 0x7eff60162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff5de00640 1 -- 172.21.15.42:0/305400487 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6006f340 con 0x7eff600d0420 2024-05-28T14:06:30.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff5e800640 1 --2- 172.21.15.42:0/305400487 >> v2:172.21.15.63:3300/0 conn(0x7eff600b4110 0x7eff60162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:30.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.918+0000 7eff55400640 1 --2- 172.21.15.42:0/305400487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff60067810 0x7eff600b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:30.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff5de00640 1 --2- 172.21.15.42:0/305400487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff600d0420 0x7eff6015ed10 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7eff6006ef90 tx=0x7eff5001c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3812efa9e620928 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:30.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff5d400640 1 -- 172.21.15.42:0/305400487 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff500b0070 con 0x7eff600d0420 2024-05-28T14:06:30.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff5d400640 1 -- 172.21.15.42:0/305400487 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff500ab600 con 0x7eff600d0420 2024-05-28T14:06:30.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff5d400640 1 -- 172.21.15.42:0/305400487 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff500ab9d0 con 0x7eff600d0420 2024-05-28T14:06:30.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 -- 172.21.15.42:0/305400487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff600d0420 msgr2=0x7eff6015ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 --2- 172.21.15.42:0/305400487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff600d0420 0x7eff6015ed10 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7eff6006ef90 tx=0x7eff5001c1d0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 -- 172.21.15.42:0/305400487 shutdown_connections 2024-05-28T14:06:30.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 --2- 172.21.15.42:0/305400487 >> v2:172.21.15.63:3300/0 conn(0x7eff600b4110 0x7eff60162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 --2- 172.21.15.42:0/305400487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff60067810 0x7eff600b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 --2- 172.21.15.42:0/305400487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff600d0420 0x7eff6015ed10 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 -- 172.21.15.42:0/305400487 >> 172.21.15.42:0/305400487 conn(0x7eff600d0b40 msgr2=0x7eff600d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:30.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 -- 172.21.15.42:0/305400487 shutdown_connections 2024-05-28T14:06:30.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 -- 172.21.15.42:0/305400487 wait complete. 2024-05-28T14:06:30.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 Processor -- start 2024-05-28T14:06:30.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 -- start start 2024-05-28T14:06:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff60067810 0x7eff6014e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff600b4110 0x7eff6014fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff600d0420 0x7eff60150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7eff6006be10 con 0x7eff600d0420 2024-05-28T14:06:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff6006bc90 con 0x7eff600b4110 2024-05-28T14:06:30.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff64e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7eff6006bf90 con 0x7eff60067810 2024-05-28T14:06:30.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff5e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff600d0420 0x7eff60150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff5e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff600d0420 0x7eff60150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53188/0 (socket says 172.21.15.42:53188) 2024-05-28T14:06:30.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff5e800640 1 -- 172.21.15.42:0/1798189024 learned_addr learned my addr 172.21.15.42:0/1798189024 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:30.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.922+0000 7eff5de00640 1 --2- 172.21.15.42:0/1798189024 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff60067810 0x7eff6014e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff55400640 1 --2- 172.21.15.42:0/1798189024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff600b4110 0x7eff6014fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff5e800640 1 -- 172.21.15.42:0/1798189024 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff60067810 msgr2=0x7eff6014e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff5e800640 1 --2- 172.21.15.42:0/1798189024 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff60067810 0x7eff6014e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff5e800640 1 -- 172.21.15.42:0/1798189024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff600b4110 msgr2=0x7eff6014fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:30.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff5e800640 1 --2- 172.21.15.42:0/1798189024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff600b4110 0x7eff6014fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:30.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff5e800640 1 -- 172.21.15.42:0/1798189024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff60151ee0 con 0x7eff600d0420 2024-05-28T14:06:30.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff5e800640 1 --2- 172.21.15.42:0/1798189024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff600d0420 0x7eff60150370 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7eff5800cef0 tx=0x7eff5800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:30.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff56a00640 1 -- 172.21.15.42:0/1798189024 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5801a070 con 0x7eff600d0420 2024-05-28T14:06:30.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff56a00640 1 -- 172.21.15.42:0/1798189024 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff5800d860 con 0x7eff600d0420 2024-05-28T14:06:30.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff56a00640 1 -- 172.21.15.42:0/1798189024 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5800dc50 con 0x7eff600d0420 2024-05-28T14:06:30.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff64e00640 1 -- 172.21.15.42:0/1798189024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6014f5b0 con 0x7eff600d0420 2024-05-28T14:06:30.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff64e00640 1 -- 172.21.15.42:0/1798189024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6014fb50 con 0x7eff600d0420 2024-05-28T14:06:30.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff64e00640 1 -- 172.21.15.42:0/1798189024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff60068a90 con 0x7eff600d0420 2024-05-28T14:06:30.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff56a00640 1 -- 172.21.15.42:0/1798189024 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7eff5801d070 con 0x7eff600d0420 2024-05-28T14:06:30.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.926+0000 7eff56a00640 1 --2- 172.21.15.42:0/1798189024 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7eff3c069070 0x7eff3c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:30.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.930+0000 7eff56a00640 1 -- 172.21.15.42:0/1798189024 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7eff58085c10 con 0x7eff600d0420 2024-05-28T14:06:30.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.930+0000 7eff5de00640 1 --2- 172.21.15.42:0/1798189024 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7eff3c069070 0x7eff3c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:30.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.930+0000 7eff5de00640 1 --2- 172.21.15.42:0/1798189024 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7eff3c069070 0x7eff3c06b530 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7eff500ad000 tx=0x7eff50008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:30.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:30.934+0000 7eff56a00640 1 -- 172.21.15.42:0/1798189024 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7eff58057690 con 0x7eff600d0420 2024-05-28T14:06:31.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:31.094+0000 7eff64e00640 1 -- 172.21.15.42:0/1798189024 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff600b0060 con 0x7eff3c069070 2024-05-28T14:06:31.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:31.102+0000 7eff56a00640 1 -- 172.21.15.42:0/1798189024 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7eff600b0060 con 0x7eff3c069070 2024-05-28T14:06:31.089 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:31.091 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:06:31.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:31.102+0000 7eff64e00640 1 -- 172.21.15.42:0/1798189024 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7eff3c069070 msgr2=0x7eff3c06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:31.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:31.102+0000 7eff64e00640 1 --2- 172.21.15.42:0/1798189024 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7eff3c069070 0x7eff3c06b530 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7eff500ad000 tx=0x7eff50008040 comp rx=0 tx=0).stop 2024-05-28T14:06:31.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:31.102+0000 7eff64e00640 1 -- 172.21.15.42:0/1798189024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff600d0420 msgr2=0x7eff60150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:31.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:31.102+0000 7eff64e00640 1 --2- 172.21.15.42:0/1798189024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff600d0420 0x7eff60150370 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7eff5800cef0 tx=0x7eff5800a710 comp rx=0 tx=0).stop 2024-05-28T14:06:31.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:31.102+0000 7eff64e00640 1 -- 172.21.15.42:0/1798189024 shutdown_connections 2024-05-28T14:06:31.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:31.102+0000 7eff64e00640 1 --2- 172.21.15.42:0/1798189024 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7eff3c069070 0x7eff3c06b530 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:31.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:31.102+0000 7eff64e00640 1 --2- 172.21.15.42:0/1798189024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff600d0420 0x7eff60150370 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:31.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:31.102+0000 7eff64e00640 1 --2- 172.21.15.42:0/1798189024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff600b4110 0x7eff6014fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:31.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:31.102+0000 7eff64e00640 1 --2- 172.21.15.42:0/1798189024 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff60067810 0x7eff6014e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:31.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:31.102+0000 7eff64e00640 1 -- 172.21.15.42:0/1798189024 >> 172.21.15.42:0/1798189024 conn(0x7eff600d0b40 msgr2=0x7eff600b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:31.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:31.102+0000 7eff64e00640 1 -- 172.21.15.42:0/1798189024 shutdown_connections 2024-05-28T14:06:31.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:31.106+0000 7eff64e00640 1 -- 172.21.15.42:0/1798189024 wait complete. 2024-05-28T14:06:31.110 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-05-28T14:06:30.995602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":435392,"kb_used_data":8264,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754539328,"statfs":{"total":773094113280,"available":772648271872,"internally_reserved":0,"allocated":8462336,"data_stored":4263052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":437243442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.588001"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815319+0000","last_change":"2024-05-28T14:06:25.436476+0000","last_active":"2024-05-28T14:06:26.815319+0000","last_peered":"2024-05-28T14:06:26.815319+0000","last_clean":"2024-05-28T14:06:26.815319+0000","last_became_active":"2024-05-28T14:06:25.436147+0000","last_became_peered":"2024-05-28T14:06:25.436147+0000","last_unstale":"2024-05-28T14:06:26.815319+0000","last_undegraded":"2024-05-28T14:06:26.815319+0000","last_fullsized":"2024-05-28T14:06:26.815319+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:04:20.056562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816331+0000","last_change":"2024-05-28T14:06:25.435200+0000","last_active":"2024-05-28T14:06:26.816331+0000","last_peered":"2024-05-28T14:06:26.816331+0000","last_clean":"2024-05-28T14:06:26.816331+0000","last_became_active":"2024-05-28T14:06:25.429892+0000","last_became_peered":"2024-05-28T14:06:25.429892+0000","last_unstale":"2024-05-28T14:06:26.816331+0000","last_undegraded":"2024-05-28T14:06:26.816331+0000","last_fullsized":"2024-05-28T14:06:26.816331+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:03:35.592277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814690+0000","last_change":"2024-05-28T14:06:25.435750+0000","last_active":"2024-05-28T14:06:26.814690+0000","last_peered":"2024-05-28T14:06:26.814690+0000","last_clean":"2024-05-28T14:06:26.814690+0000","last_became_active":"2024-05-28T14:06:25.433833+0000","last_became_peered":"2024-05-28T14:06:25.433833+0000","last_unstale":"2024-05-28T14:06:26.814690+0000","last_undegraded":"2024-05-28T14:06:26.814690+0000","last_fullsized":"2024-05-28T14:06:26.814690+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:01:25.637892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814302+0000","last_change":"2024-05-28T14:06:25.432371+0000","last_active":"2024-05-28T14:06:26.814302+0000","last_peered":"2024-05-28T14:06:26.814302+0000","last_clean":"2024-05-28T14:06:26.814302+0000","last_became_active":"2024-05-28T14:06:25.432078+0000","last_became_peered":"2024-05-28T14:06:25.432078+0000","last_unstale":"2024-05-28T14:06:26.814302+0000","last_undegraded":"2024-05-28T14:06:26.814302+0000","last_fullsized":"2024-05-28T14:06:26.814302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:34:50.335279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812230+0000","last_change":"2024-05-28T14:06:25.430672+0000","last_active":"2024-05-28T14:06:26.812230+0000","last_peered":"2024-05-28T14:06:26.812230+0000","last_clean":"2024-05-28T14:06:26.812230+0000","last_became_active":"2024-05-28T14:06:25.430390+0000","last_became_peered":"2024-05-28T14:06:25.430390+0000","last_unstale":"2024-05-28T14:06:26.812230+0000","last_undegraded":"2024-05-28T14:06:26.812230+0000","last_fullsized":"2024-05-28T14:06:26.812230+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:04:38.407175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815845+0000","last_change":"2024-05-28T14:06:25.440954+0000","last_active":"2024-05-28T14:06:26.815845+0000","last_peered":"2024-05-28T14:06:26.815845+0000","last_clean":"2024-05-28T14:06:26.815845+0000","last_became_active":"2024-05-28T14:06:25.440634+0000","last_became_peered":"2024-05-28T14:06:25.440634+0000","last_unstale":"2024-05-28T14:06:26.815845+0000","last_undegraded":"2024-05-28T14:06:26.815845+0000","last_fullsized":"2024-05-28T14:06:26.815845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:33:29.862414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.430604+0000","last_change":"2024-05-28T14:06:25.428279+0000","last_active":"2024-05-28T14:06:26.430604+0000","last_peered":"2024-05-28T14:06:26.430604+0000","last_clean":"2024-05-28T14:06:26.430604+0000","last_became_active":"2024-05-28T14:06:25.426405+0000","last_became_peered":"2024-05-28T14:06:25.426405+0000","last_unstale":"2024-05-28T14:06:26.430604+0000","last_undegraded":"2024-05-28T14:06:26.430604+0000","last_fullsized":"2024-05-28T14:06:26.430604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:19:41.907890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815090+0000","last_change":"2024-05-28T14:06:25.433150+0000","last_active":"2024-05-28T14:06:26.815090+0000","last_peered":"2024-05-28T14:06:26.815090+0000","last_clean":"2024-05-28T14:06:26.815090+0000","last_became_active":"2024-05-28T14:06:25.432858+0000","last_became_peered":"2024-05-28T14:06:25.432858+0000","last_unstale":"2024-05-28T14:06:26.815090+0000","last_undegraded":"2024-05-28T14:06:26.815090+0000","last_fullsized":"2024-05-28T14:06:26.815090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:34:56.946212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812514+0000","last_change":"2024-05-28T14:06:25.435943+0000","last_active":"2024-05-28T14:06:26.812514+0000","last_peered":"2024-05-28T14:06:26.812514+0000","last_clean":"2024-05-28T14:06:26.812514+0000","last_became_active":"2024-05-28T14:06:25.435445+0000","last_became_peered":"2024-05-28T14:06:25.435445+0000","last_unstale":"2024-05-28T14:06:26.812514+0000","last_undegraded":"2024-05-28T14:06:26.812514+0000","last_fullsized":"2024-05-28T14:06:26.812514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:58:09.166023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431778+0000","last_change":"2024-05-28T14:06:25.429227+0000","last_active":"2024-05-28T14:06:26.431778+0000","last_peered":"2024-05-28T14:06:26.431778+0000","last_clean":"2024-05-28T14:06:26.431778+0000","last_became_active":"2024-05-28T14:06:25.428962+0000","last_became_peered":"2024-05-28T14:06:25.428962+0000","last_unstale":"2024-05-28T14:06:26.431778+0000","last_undegraded":"2024-05-28T14:06:26.431778+0000","last_fullsized":"2024-05-28T14:06:26.431778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:06:24.664189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815222+0000","last_change":"2024-05-28T14:06:25.439943+0000","last_active":"2024-05-28T14:06:26.815222+0000","last_peered":"2024-05-28T14:06:26.815222+0000","last_clean":"2024-05-28T14:06:26.815222+0000","last_became_active":"2024-05-28T14:06:25.439316+0000","last_became_peered":"2024-05-28T14:06:25.439316+0000","last_unstale":"2024-05-28T14:06:26.815222+0000","last_undegraded":"2024-05-28T14:06:26.815222+0000","last_fullsized":"2024-05-28T14:06:26.815222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:02:25.085918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814370+0000","last_change":"2024-05-28T14:06:25.423591+0000","last_active":"2024-05-28T14:06:26.814370+0000","last_peered":"2024-05-28T14:06:26.814370+0000","last_clean":"2024-05-28T14:06:26.814370+0000","last_became_active":"2024-05-28T14:06:25.423051+0000","last_became_peered":"2024-05-28T14:06:25.423051+0000","last_unstale":"2024-05-28T14:06:26.814370+0000","last_undegraded":"2024-05-28T14:06:26.814370+0000","last_fullsized":"2024-05-28T14:06:26.814370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:42:02.140613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428441+0000","last_change":"2024-05-28T14:06:25.433443+0000","last_active":"2024-05-28T14:06:26.428441+0000","last_peered":"2024-05-28T14:06:26.428441+0000","last_clean":"2024-05-28T14:06:26.428441+0000","last_became_active":"2024-05-28T14:06:25.433165+0000","last_became_peered":"2024-05-28T14:06:25.433165+0000","last_unstale":"2024-05-28T14:06:26.428441+0000","last_undegraded":"2024-05-28T14:06:26.428441+0000","last_fullsized":"2024-05-28T14:06:26.428441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:20:06.730622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812530+0000","last_change":"2024-05-28T14:06:25.423146+0000","last_active":"2024-05-28T14:06:26.812530+0000","last_peered":"2024-05-28T14:06:26.812530+0000","last_clean":"2024-05-28T14:06:26.812530+0000","last_became_active":"2024-05-28T14:06:25.422848+0000","last_became_peered":"2024-05-28T14:06:25.422848+0000","last_unstale":"2024-05-28T14:06:26.812530+0000","last_undegraded":"2024-05-28T14:06:26.812530+0000","last_fullsized":"2024-05-28T14:06:26.812530+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:31.303741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428426+0000","last_change":"2024-05-28T14:06:25.429616+0000","last_active":"2024-05-28T14:06:26.428426+0000","last_peered":"2024-05-28T14:06:26.428426+0000","last_clean":"2024-05-28T14:06:26.428426+0000","last_became_active":"2024-05-28T14:06:25.428642+0000","last_became_peered":"2024-05-28T14:06:25.428642+0000","last_unstale":"2024-05-28T14:06:26.428426+0000","last_undegraded":"2024-05-28T14:06:26.428426+0000","last_fullsized":"2024-05-28T14:06:26.428426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:42:08.042876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.430964+0000","last_change":"2024-05-28T14:06:25.431909+0000","last_active":"2024-05-28T14:06:26.430964+0000","last_peered":"2024-05-28T14:06:26.430964+0000","last_clean":"2024-05-28T14:06:26.430964+0000","last_became_active":"2024-05-28T14:06:25.431610+0000","last_became_peered":"2024-05-28T14:06:25.431610+0000","last_unstale":"2024-05-28T14:06:26.430964+0000","last_undegraded":"2024-05-28T14:06:26.430964+0000","last_fullsized":"2024-05-28T14:06:26.430964+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:27:03.768824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812676+0000","last_change":"2024-05-28T14:06:25.435839+0000","last_active":"2024-05-28T14:06:26.812676+0000","last_peered":"2024-05-28T14:06:26.812676+0000","last_clean":"2024-05-28T14:06:26.812676+0000","last_became_active":"2024-05-28T14:06:25.435378+0000","last_became_peered":"2024-05-28T14:06:25.435378+0000","last_unstale":"2024-05-28T14:06:26.812676+0000","last_undegraded":"2024-05-28T14:06:26.812676+0000","last_fullsized":"2024-05-28T14:06:26.812676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:53:18.602592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814169+0000","last_change":"2024-05-28T14:06:25.432059+0000","last_active":"2024-05-28T14:06:26.814169+0000","last_peered":"2024-05-28T14:06:26.814169+0000","last_clean":"2024-05-28T14:06:26.814169+0000","last_became_active":"2024-05-28T14:06:25.431758+0000","last_became_peered":"2024-05-28T14:06:25.431758+0000","last_unstale":"2024-05-28T14:06:26.814169+0000","last_undegraded":"2024-05-28T14:06:26.814169+0000","last_fullsized":"2024-05-28T14:06:26.814169+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:34:23.759315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816090+0000","last_change":"2024-05-28T14:06:25.434141+0000","last_active":"2024-05-28T14:06:26.816090+0000","last_peered":"2024-05-28T14:06:26.816090+0000","last_clean":"2024-05-28T14:06:26.816090+0000","last_became_active":"2024-05-28T14:06:25.433659+0000","last_became_peered":"2024-05-28T14:06:25.433659+0000","last_unstale":"2024-05-28T14:06:26.816090+0000","last_undegraded":"2024-05-28T14:06:26.816090+0000","last_fullsized":"2024-05-28T14:06:26.816090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:50.332524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428393+0000","last_change":"2024-05-28T14:06:25.428098+0000","last_active":"2024-05-28T14:06:26.428393+0000","last_peered":"2024-05-28T14:06:26.428393+0000","last_clean":"2024-05-28T14:06:26.428393+0000","last_became_active":"2024-05-28T14:06:25.427805+0000","last_became_peered":"2024-05-28T14:06:25.427805+0000","last_unstale":"2024-05-28T14:06:26.428393+0000","last_undegraded":"2024-05-28T14:06:26.428393+0000","last_fullsized":"2024-05-28T14:06:26.428393+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:09.597247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431052+0000","last_change":"2024-05-28T14:06:25.430100+0000","last_active":"2024-05-28T14:06:26.431052+0000","last_peered":"2024-05-28T14:06:26.431052+0000","last_clean":"2024-05-28T14:06:26.431052+0000","last_became_active":"2024-05-28T14:06:25.429842+0000","last_became_peered":"2024-05-28T14:06:25.429842+0000","last_unstale":"2024-05-28T14:06:26.431052+0000","last_undegraded":"2024-05-28T14:06:26.431052+0000","last_fullsized":"2024-05-28T14:06:26.431052+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:50.332524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431178+0000","last_change":"2024-05-28T14:06:25.432563+0000","last_active":"2024-05-28T14:06:26.431178+0000","last_peered":"2024-05-28T14:06:26.431178+0000","last_clean":"2024-05-28T14:06:26.431178+0000","last_became_active":"2024-05-28T14:06:25.432252+0000","last_became_peered":"2024-05-28T14:06:25.432252+0000","last_unstale":"2024-05-28T14:06:26.431178+0000","last_undegraded":"2024-05-28T14:06:26.431178+0000","last_fullsized":"2024-05-28T14:06:26.431178+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:26:46.087828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814797+0000","last_change":"2024-05-28T14:06:25.437622+0000","last_active":"2024-05-28T14:06:26.814797+0000","last_peered":"2024-05-28T14:06:26.814797+0000","last_clean":"2024-05-28T14:06:26.814797+0000","last_became_active":"2024-05-28T14:06:25.437270+0000","last_became_peered":"2024-05-28T14:06:25.437270+0000","last_unstale":"2024-05-28T14:06:26.814797+0000","last_undegraded":"2024-05-28T14:06:26.814797+0000","last_fullsized":"2024-05-28T14:06:26.814797+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:07:33.366108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815240+0000","last_change":"2024-05-28T14:06:25.445645+0000","last_active":"2024-05-28T14:06:26.815240+0000","last_peered":"2024-05-28T14:06:26.815240+0000","last_clean":"2024-05-28T14:06:26.815240+0000","last_became_active":"2024-05-28T14:06:25.445274+0000","last_became_peered":"2024-05-28T14:06:25.445274+0000","last_unstale":"2024-05-28T14:06:26.815240+0000","last_undegraded":"2024-05-28T14:06:26.815240+0000","last_fullsized":"2024-05-28T14:06:26.815240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:56:13.305914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431535+0000","last_change":"2024-05-28T14:06:25.437597+0000","last_active":"2024-05-28T14:06:26.431535+0000","last_peered":"2024-05-28T14:06:26.431535+0000","last_clean":"2024-05-28T14:06:26.431535+0000","last_became_active":"2024-05-28T14:06:25.437267+0000","last_became_peered":"2024-05-28T14:06:25.437267+0000","last_unstale":"2024-05-28T14:06:26.431535+0000","last_undegraded":"2024-05-28T14:06:26.431535+0000","last_fullsized":"2024-05-28T14:06:26.431535+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:55:04.339742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.432101+0000","last_change":"2024-05-28T14:06:25.436883+0000","last_active":"2024-05-28T14:06:26.432101+0000","last_peered":"2024-05-28T14:06:26.432101+0000","last_clean":"2024-05-28T14:06:26.432101+0000","last_became_active":"2024-05-28T14:06:25.436484+0000","last_became_peered":"2024-05-28T14:06:25.436484+0000","last_unstale":"2024-05-28T14:06:26.432101+0000","last_undegraded":"2024-05-28T14:06:26.432101+0000","last_fullsized":"2024-05-28T14:06:26.432101+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:10:33.978239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814687+0000","last_change":"2024-05-28T14:06:25.434166+0000","last_active":"2024-05-28T14:06:26.814687+0000","last_peered":"2024-05-28T14:06:26.814687+0000","last_clean":"2024-05-28T14:06:26.814687+0000","last_became_active":"2024-05-28T14:06:25.433541+0000","last_became_peered":"2024-05-28T14:06:25.433541+0000","last_unstale":"2024-05-28T14:06:26.814687+0000","last_undegraded":"2024-05-28T14:06:26.814687+0000","last_fullsized":"2024-05-28T14:06:26.814687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:57:49.955869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.432075+0000","last_change":"2024-05-28T14:06:25.432535+0000","last_active":"2024-05-28T14:06:26.432075+0000","last_peered":"2024-05-28T14:06:26.432075+0000","last_clean":"2024-05-28T14:06:26.432075+0000","last_became_active":"2024-05-28T14:06:25.432243+0000","last_became_peered":"2024-05-28T14:06:25.432243+0000","last_unstale":"2024-05-28T14:06:26.432075+0000","last_undegraded":"2024-05-28T14:06:26.432075+0000","last_fullsized":"2024-05-28T14:06:26.432075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:11:07.262123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.813755+0000","last_change":"2024-05-28T14:06:25.434512+0000","last_active":"2024-05-28T14:06:26.813755+0000","last_peered":"2024-05-28T14:06:26.813755+0000","last_clean":"2024-05-28T14:06:26.813755+0000","last_became_active":"2024-05-28T14:06:25.434154+0000","last_became_peered":"2024-05-28T14:06:25.434154+0000","last_unstale":"2024-05-28T14:06:26.813755+0000","last_undegraded":"2024-05-28T14:06:26.813755+0000","last_fullsized":"2024-05-28T14:06:26.813755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:47:43.307227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431355+0000","last_change":"2024-05-28T14:06:25.431700+0000","last_active":"2024-05-28T14:06:26.431355+0000","last_peered":"2024-05-28T14:06:26.431355+0000","last_clean":"2024-05-28T14:06:26.431355+0000","last_became_active":"2024-05-28T14:06:25.430896+0000","last_became_peered":"2024-05-28T14:06:25.430896+0000","last_unstale":"2024-05-28T14:06:26.431355+0000","last_undegraded":"2024-05-28T14:06:26.431355+0000","last_fullsized":"2024-05-28T14:06:26.431355+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:32:01.620163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431818+0000","last_change":"2024-05-28T14:06:25.426047+0000","last_active":"2024-05-28T14:06:26.431818+0000","last_peered":"2024-05-28T14:06:26.431818+0000","last_clean":"2024-05-28T14:06:26.431818+0000","last_became_active":"2024-05-28T14:06:25.425752+0000","last_became_peered":"2024-05-28T14:06:25.425752+0000","last_unstale":"2024-05-28T14:06:26.431818+0000","last_undegraded":"2024-05-28T14:06:26.431818+0000","last_fullsized":"2024-05-28T14:06:26.431818+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:20:06.730622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815244+0000","last_change":"2024-05-28T14:06:25.440590+0000","last_active":"2024-05-28T14:06:26.815244+0000","last_peered":"2024-05-28T14:06:26.815244+0000","last_clean":"2024-05-28T14:06:26.815244+0000","last_became_active":"2024-05-28T14:06:25.439210+0000","last_became_peered":"2024-05-28T14:06:25.439210+0000","last_unstale":"2024-05-28T14:06:26.815244+0000","last_undegraded":"2024-05-28T14:06:26.815244+0000","last_fullsized":"2024-05-28T14:06:26.815244+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:53:54.369051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815182+0000","last_change":"2024-05-28T14:06:25.445738+0000","last_active":"2024-05-28T14:06:26.815182+0000","last_peered":"2024-05-28T14:06:26.815182+0000","last_clean":"2024-05-28T14:06:26.815182+0000","last_became_active":"2024-05-28T14:06:25.445392+0000","last_became_peered":"2024-05-28T14:06:25.445392+0000","last_unstale":"2024-05-28T14:06:26.815182+0000","last_undegraded":"2024-05-28T14:06:26.815182+0000","last_fullsized":"2024-05-28T14:06:26.815182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:35:56.185872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815557+0000","last_change":"2024-05-28T14:06:24.435935+0000","last_active":"2024-05-28T14:06:26.815557+0000","last_peered":"2024-05-28T14:06:26.815557+0000","last_clean":"2024-05-28T14:06:26.815557+0000","last_became_active":"2024-05-28T14:06:24.435637+0000","last_became_peered":"2024-05-28T14:06:24.435637+0000","last_unstale":"2024-05-28T14:06:26.815557+0000","last_undegraded":"2024-05-28T14:06:26.815557+0000","last_fullsized":"2024-05-28T14:06:26.815557+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:42:07.950563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812744+0000","last_change":"2024-05-28T14:06:25.435699+0000","last_active":"2024-05-28T14:06:26.812744+0000","last_peered":"2024-05-28T14:06:26.812744+0000","last_clean":"2024-05-28T14:06:26.812744+0000","last_became_active":"2024-05-28T14:06:25.434919+0000","last_became_peered":"2024-05-28T14:06:25.434919+0000","last_unstale":"2024-05-28T14:06:26.812744+0000","last_undegraded":"2024-05-28T14:06:26.812744+0000","last_fullsized":"2024-05-28T14:06:26.812744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:58:14.952326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428734+0000","last_change":"2024-05-28T14:06:24.427568+0000","last_active":"2024-05-28T14:06:26.428734+0000","last_peered":"2024-05-28T14:06:26.428734+0000","last_clean":"2024-05-28T14:06:26.428734+0000","last_became_active":"2024-05-28T14:06:24.427288+0000","last_became_peered":"2024-05-28T14:06:24.427288+0000","last_unstale":"2024-05-28T14:06:26.428734+0000","last_undegraded":"2024-05-28T14:06:26.428734+0000","last_fullsized":"2024-05-28T14:06:26.428734+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:54.168407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816219+0000","last_change":"2024-05-28T14:06:25.443335+0000","last_active":"2024-05-28T14:06:26.816219+0000","last_peered":"2024-05-28T14:06:26.816219+0000","last_clean":"2024-05-28T14:06:26.816219+0000","last_became_active":"2024-05-28T14:06:25.441253+0000","last_became_peered":"2024-05-28T14:06:25.441253+0000","last_unstale":"2024-05-28T14:06:26.816219+0000","last_undegraded":"2024-05-28T14:06:26.816219+0000","last_fullsized":"2024-05-28T14:06:26.816219+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:26:46.087828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431643+0000","last_change":"2024-05-28T14:06:24.435174+0000","last_active":"2024-05-28T14:06:26.431643+0000","last_peered":"2024-05-28T14:06:26.431643+0000","last_clean":"2024-05-28T14:06:26.431643+0000","last_became_active":"2024-05-28T14:06:24.434890+0000","last_became_peered":"2024-05-28T14:06:24.434890+0000","last_unstale":"2024-05-28T14:06:26.431643+0000","last_undegraded":"2024-05-28T14:06:26.431643+0000","last_fullsized":"2024-05-28T14:06:26.431643+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:54:28.504352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815382+0000","last_change":"2024-05-28T14:06:25.438263+0000","last_active":"2024-05-28T14:06:26.815382+0000","last_peered":"2024-05-28T14:06:26.815382+0000","last_clean":"2024-05-28T14:06:26.815382+0000","last_became_active":"2024-05-28T14:06:25.437316+0000","last_became_peered":"2024-05-28T14:06:25.437316+0000","last_unstale":"2024-05-28T14:06:26.815382+0000","last_undegraded":"2024-05-28T14:06:26.815382+0000","last_fullsized":"2024-05-28T14:06:26.815382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:00:56.168101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816377+0000","last_change":"2024-05-28T14:06:24.428906+0000","last_active":"2024-05-28T14:06:26.816377+0000","last_peered":"2024-05-28T14:06:26.816377+0000","last_clean":"2024-05-28T14:06:26.816377+0000","last_became_active":"2024-05-28T14:06:24.427835+0000","last_became_peered":"2024-05-28T14:06:24.427835+0000","last_unstale":"2024-05-28T14:06:26.816377+0000","last_undegraded":"2024-05-28T14:06:26.816377+0000","last_fullsized":"2024-05-28T14:06:26.816377+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:41:19.574984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431511+0000","last_change":"2024-05-28T14:06:25.435910+0000","last_active":"2024-05-28T14:06:26.431511+0000","last_peered":"2024-05-28T14:06:26.431511+0000","last_clean":"2024-05-28T14:06:26.431511+0000","last_became_active":"2024-05-28T14:06:25.435620+0000","last_became_peered":"2024-05-28T14:06:25.435620+0000","last_unstale":"2024-05-28T14:06:26.431511+0000","last_undegraded":"2024-05-28T14:06:26.431511+0000","last_fullsized":"2024-05-28T14:06:26.431511+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:09.597247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814170+0000","last_change":"2024-05-28T14:06:24.422900+0000","last_active":"2024-05-28T14:06:26.814170+0000","last_peered":"2024-05-28T14:06:26.814170+0000","last_clean":"2024-05-28T14:06:26.814170+0000","last_became_active":"2024-05-28T14:06:24.422439+0000","last_became_peered":"2024-05-28T14:06:24.422439+0000","last_unstale":"2024-05-28T14:06:26.814170+0000","last_undegraded":"2024-05-28T14:06:26.814170+0000","last_fullsized":"2024-05-28T14:06:26.814170+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:02:24.079875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431360+0000","last_change":"2024-05-28T14:06:25.437152+0000","last_active":"2024-05-28T14:06:26.431360+0000","last_peered":"2024-05-28T14:06:26.431360+0000","last_clean":"2024-05-28T14:06:26.431360+0000","last_became_active":"2024-05-28T14:06:25.435921+0000","last_became_peered":"2024-05-28T14:06:25.435921+0000","last_unstale":"2024-05-28T14:06:26.431360+0000","last_undegraded":"2024-05-28T14:06:26.431360+0000","last_fullsized":"2024-05-28T14:06:26.431360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:03:35.592277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815823+0000","last_change":"2024-05-28T14:06:24.431434+0000","last_active":"2024-05-28T14:06:26.815823+0000","last_peered":"2024-05-28T14:06:26.815823+0000","last_clean":"2024-05-28T14:06:26.815823+0000","last_became_active":"2024-05-28T14:06:24.430297+0000","last_became_peered":"2024-05-28T14:06:24.430297+0000","last_unstale":"2024-05-28T14:06:26.815823+0000","last_undegraded":"2024-05-28T14:06:26.815823+0000","last_fullsized":"2024-05-28T14:06:26.815823+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:26.316274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812074+0000","last_change":"2024-05-28T14:06:25.421716+0000","last_active":"2024-05-28T14:06:26.812074+0000","last_peered":"2024-05-28T14:06:26.812074+0000","last_clean":"2024-05-28T14:06:26.812074+0000","last_became_active":"2024-05-28T14:06:25.421417+0000","last_became_peered":"2024-05-28T14:06:25.421417+0000","last_unstale":"2024-05-28T14:06:26.812074+0000","last_undegraded":"2024-05-28T14:06:26.812074+0000","last_fullsized":"2024-05-28T14:06:26.812074+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:42:56.223388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.813869+0000","last_change":"2024-05-28T14:06:24.423413+0000","last_active":"2024-05-28T14:06:26.813869+0000","last_peered":"2024-05-28T14:06:26.813869+0000","last_clean":"2024-05-28T14:06:26.813869+0000","last_became_active":"2024-05-28T14:06:24.423140+0000","last_became_peered":"2024-05-28T14:06:24.423140+0000","last_unstale":"2024-05-28T14:06:26.813869+0000","last_undegraded":"2024-05-28T14:06:26.813869+0000","last_fullsized":"2024-05-28T14:06:26.813869+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:30.297697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428516+0000","last_change":"2024-05-28T14:06:25.426700+0000","last_active":"2024-05-28T14:06:26.428516+0000","last_peered":"2024-05-28T14:06:26.428516+0000","last_clean":"2024-05-28T14:06:26.428516+0000","last_became_active":"2024-05-28T14:06:25.426302+0000","last_became_peered":"2024-05-28T14:06:25.426302+0000","last_unstale":"2024-05-28T14:06:26.428516+0000","last_undegraded":"2024-05-28T14:06:26.428516+0000","last_fullsized":"2024-05-28T14:06:26.428516+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:42:08.956606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812445+0000","last_change":"2024-05-28T14:06:24.425685+0000","last_active":"2024-05-28T14:06:26.812445+0000","last_peered":"2024-05-28T14:06:26.812445+0000","last_clean":"2024-05-28T14:06:26.812445+0000","last_became_active":"2024-05-28T14:06:24.425387+0000","last_became_peered":"2024-05-28T14:06:24.425387+0000","last_unstale":"2024-05-28T14:06:26.812445+0000","last_undegraded":"2024-05-28T14:06:26.812445+0000","last_fullsized":"2024-05-28T14:06:26.812445+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:08:08.408635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.432175+0000","last_change":"2024-05-28T14:06:25.428573+0000","last_active":"2024-05-28T14:06:26.432175+0000","last_peered":"2024-05-28T14:06:26.432175+0000","last_clean":"2024-05-28T14:06:26.432175+0000","last_became_active":"2024-05-28T14:06:25.428209+0000","last_became_peered":"2024-05-28T14:06:25.428209+0000","last_unstale":"2024-05-28T14:06:26.432175+0000","last_undegraded":"2024-05-28T14:06:26.432175+0000","last_fullsized":"2024-05-28T14:06:26.432175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:44.471299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431661+0000","last_change":"2024-05-28T14:06:24.422382+0000","last_active":"2024-05-28T14:06:26.431661+0000","last_peered":"2024-05-28T14:06:26.431661+0000","last_clean":"2024-05-28T14:06:26.431661+0000","last_became_active":"2024-05-28T14:06:24.422093+0000","last_became_peered":"2024-05-28T14:06:24.422093+0000","last_unstale":"2024-05-28T14:06:26.431661+0000","last_undegraded":"2024-05-28T14:06:26.431661+0000","last_fullsized":"2024-05-28T14:06:26.431661+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:42:01.134570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815444+0000","last_change":"2024-05-28T14:06:25.444460+0000","last_active":"2024-05-28T14:06:26.815444+0000","last_peered":"2024-05-28T14:06:26.815444+0000","last_clean":"2024-05-28T14:06:26.815444+0000","last_became_active":"2024-05-28T14:06:25.443586+0000","last_became_peered":"2024-05-28T14:06:25.443586+0000","last_unstale":"2024-05-28T14:06:26.815444+0000","last_undegraded":"2024-05-28T14:06:26.815444+0000","last_fullsized":"2024-05-28T14:06:26.815444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:54:10.116238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816384+0000","last_change":"2024-05-28T14:06:24.436081+0000","last_active":"2024-05-28T14:06:26.816384+0000","last_peered":"2024-05-28T14:06:26.816384+0000","last_clean":"2024-05-28T14:06:26.816384+0000","last_became_active":"2024-05-28T14:06:24.435796+0000","last_became_peered":"2024-05-28T14:06:24.435796+0000","last_unstale":"2024-05-28T14:06:26.816384+0000","last_undegraded":"2024-05-28T14:06:26.816384+0000","last_fullsized":"2024-05-28T14:06:26.816384+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:31:55.554880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816342+0000","last_change":"2024-05-28T14:06:25.439274+0000","last_active":"2024-05-28T14:06:26.816342+0000","last_peered":"2024-05-28T14:06:26.816342+0000","last_clean":"2024-05-28T14:06:26.816342+0000","last_became_active":"2024-05-28T14:06:25.438618+0000","last_became_peered":"2024-05-28T14:06:25.438618+0000","last_unstale":"2024-05-28T14:06:26.816342+0000","last_undegraded":"2024-05-28T14:06:26.816342+0000","last_fullsized":"2024-05-28T14:06:26.816342+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:55:04.339742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814796+0000","last_change":"2024-05-28T14:06:24.421989+0000","last_active":"2024-05-28T14:06:26.814796+0000","last_peered":"2024-05-28T14:06:26.814796+0000","last_clean":"2024-05-28T14:06:26.814796+0000","last_became_active":"2024-05-28T14:06:24.421727+0000","last_became_peered":"2024-05-28T14:06:24.421727+0000","last_unstale":"2024-05-28T14:06:26.814796+0000","last_undegraded":"2024-05-28T14:06:26.814796+0000","last_fullsized":"2024-05-28T14:06:26.814796+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:00:55.162058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812268+0000","last_change":"2024-05-28T14:06:25.425553+0000","last_active":"2024-05-28T14:06:26.812268+0000","last_peered":"2024-05-28T14:06:26.812268+0000","last_clean":"2024-05-28T14:06:26.812268+0000","last_became_active":"2024-05-28T14:06:25.425258+0000","last_became_peered":"2024-05-28T14:06:25.425258+0000","last_unstale":"2024-05-28T14:06:26.812268+0000","last_undegraded":"2024-05-28T14:06:26.812268+0000","last_fullsized":"2024-05-28T14:06:26.812268+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:46:02.976297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814468+0000","last_change":"2024-05-28T14:06:24.421197+0000","last_active":"2024-05-28T14:06:26.814468+0000","last_peered":"2024-05-28T14:06:26.814468+0000","last_clean":"2024-05-28T14:06:26.814468+0000","last_became_active":"2024-05-28T14:06:24.420925+0000","last_became_peered":"2024-05-28T14:06:24.420925+0000","last_unstale":"2024-05-28T14:06:26.814468+0000","last_undegraded":"2024-05-28T14:06:26.814468+0000","last_fullsized":"2024-05-28T14:06:26.814468+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:58:13.946282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814794+0000","last_change":"2024-05-28T14:06:25.438975+0000","last_active":"2024-05-28T14:06:26.814794+0000","last_peered":"2024-05-28T14:06:26.814794+0000","last_clean":"2024-05-28T14:06:26.814794+0000","last_became_active":"2024-05-28T14:06:25.437155+0000","last_became_peered":"2024-05-28T14:06:25.437155+0000","last_unstale":"2024-05-28T14:06:26.814794+0000","last_undegraded":"2024-05-28T14:06:26.814794+0000","last_fullsized":"2024-05-28T14:06:26.814794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:02:24.821686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814010+0000","last_change":"2024-05-28T14:06:24.429639+0000","last_active":"2024-05-28T14:06:26.814010+0000","last_peered":"2024-05-28T14:06:26.814010+0000","last_clean":"2024-05-28T14:06:26.814010+0000","last_became_active":"2024-05-28T14:06:24.429133+0000","last_became_peered":"2024-05-28T14:06:24.429133+0000","last_unstale":"2024-05-28T14:06:26.814010+0000","last_undegraded":"2024-05-28T14:06:26.814010+0000","last_fullsized":"2024-05-28T14:06:26.814010+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:22:45.391365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431983+0000","last_change":"2024-05-28T14:06:25.422321+0000","last_active":"2024-05-28T14:06:26.431983+0000","last_peered":"2024-05-28T14:06:26.431983+0000","last_clean":"2024-05-28T14:06:26.431983+0000","last_became_active":"2024-05-28T14:06:25.422020+0000","last_became_peered":"2024-05-28T14:06:25.422020+0000","last_unstale":"2024-05-28T14:06:26.431983+0000","last_undegraded":"2024-05-28T14:06:26.431983+0000","last_fullsized":"2024-05-28T14:06:26.431983+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:42:08.956606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815957+0000","last_change":"2024-05-28T14:06:24.432341+0000","last_active":"2024-05-28T14:06:26.815957+0000","last_peered":"2024-05-28T14:06:26.815957+0000","last_clean":"2024-05-28T14:06:26.815957+0000","last_became_active":"2024-05-28T14:06:24.432058+0000","last_became_peered":"2024-05-28T14:06:24.432058+0000","last_unstale":"2024-05-28T14:06:26.815957+0000","last_undegraded":"2024-05-28T14:06:26.815957+0000","last_fullsized":"2024-05-28T14:06:26.815957+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:30.297697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428551+0000","last_change":"2024-05-28T14:06:25.433584+0000","last_active":"2024-05-28T14:06:26.428551+0000","last_peered":"2024-05-28T14:06:26.428551+0000","last_clean":"2024-05-28T14:06:26.428551+0000","last_became_active":"2024-05-28T14:06:25.432726+0000","last_became_peered":"2024-05-28T14:06:25.432726+0000","last_unstale":"2024-05-28T14:06:26.428551+0000","last_undegraded":"2024-05-28T14:06:26.428551+0000","last_fullsized":"2024-05-28T14:06:26.428551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:10:33.978239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.430645+0000","last_change":"2024-05-28T14:06:24.426720+0000","last_active":"2024-05-28T14:06:26.430645+0000","last_peered":"2024-05-28T14:06:26.430645+0000","last_clean":"2024-05-28T14:06:26.430645+0000","last_became_active":"2024-05-28T14:06:24.426399+0000","last_became_peered":"2024-05-28T14:06:24.426399+0000","last_unstale":"2024-05-28T14:06:26.430645+0000","last_undegraded":"2024-05-28T14:06:26.430645+0000","last_fullsized":"2024-05-28T14:06:26.430645+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:55:03.333699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812310+0000","last_change":"2024-05-28T14:06:25.440554+0000","last_active":"2024-05-28T14:06:26.812310+0000","last_peered":"2024-05-28T14:06:26.812310+0000","last_clean":"2024-05-28T14:06:26.812310+0000","last_became_active":"2024-05-28T14:06:25.440273+0000","last_became_peered":"2024-05-28T14:06:25.440273+0000","last_unstale":"2024-05-28T14:06:26.812310+0000","last_undegraded":"2024-05-28T14:06:26.812310+0000","last_fullsized":"2024-05-28T14:06:26.812310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:37:37.078325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431020+0000","last_change":"2024-05-28T14:06:24.424772+0000","last_active":"2024-05-28T14:06:26.431020+0000","last_peered":"2024-05-28T14:06:26.431020+0000","last_clean":"2024-05-28T14:06:26.431020+0000","last_became_active":"2024-05-28T14:06:24.424480+0000","last_became_peered":"2024-05-28T14:06:24.424480+0000","last_unstale":"2024-05-28T14:06:26.431020+0000","last_undegraded":"2024-05-28T14:06:26.431020+0000","last_fullsized":"2024-05-28T14:06:26.431020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:34:49.329236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431827+0000","last_change":"2024-05-28T14:06:25.434340+0000","last_active":"2024-05-28T14:06:26.431827+0000","last_peered":"2024-05-28T14:06:26.431827+0000","last_clean":"2024-05-28T14:06:26.431827+0000","last_became_active":"2024-05-28T14:06:25.434049+0000","last_became_peered":"2024-05-28T14:06:25.434049+0000","last_unstale":"2024-05-28T14:06:26.431827+0000","last_undegraded":"2024-05-28T14:06:26.431827+0000","last_fullsized":"2024-05-28T14:06:26.431827+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:34:48.270645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816490+0000","last_change":"2024-05-28T14:06:24.434783+0000","last_active":"2024-05-28T14:06:26.816490+0000","last_peered":"2024-05-28T14:06:26.816490+0000","last_clean":"2024-05-28T14:06:26.816490+0000","last_became_active":"2024-05-28T14:06:24.434490+0000","last_became_peered":"2024-05-28T14:06:24.434490+0000","last_unstale":"2024-05-28T14:06:26.816490+0000","last_undegraded":"2024-05-28T14:06:26.816490+0000","last_fullsized":"2024-05-28T14:06:26.816490+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:34:49.329236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815517+0000","last_change":"2024-05-28T14:06:25.440884+0000","last_active":"2024-05-28T14:06:26.815517+0000","last_peered":"2024-05-28T14:06:26.815517+0000","last_clean":"2024-05-28T14:06:26.815517+0000","last_became_active":"2024-05-28T14:06:25.440471+0000","last_became_peered":"2024-05-28T14:06:25.440471+0000","last_unstale":"2024-05-28T14:06:26.815517+0000","last_undegraded":"2024-05-28T14:06:26.815517+0000","last_fullsized":"2024-05-28T14:06:26.815517+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:22:01.027740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431898+0000","last_change":"2024-05-28T14:06:24.434447+0000","last_active":"2024-05-28T14:06:26.431898+0000","last_peered":"2024-05-28T14:06:26.431898+0000","last_clean":"2024-05-28T14:06:26.431898+0000","last_became_active":"2024-05-28T14:06:24.433002+0000","last_became_peered":"2024-05-28T14:06:24.433002+0000","last_unstale":"2024-05-28T14:06:26.431898+0000","last_undegraded":"2024-05-28T14:06:26.431898+0000","last_fullsized":"2024-05-28T14:06:26.431898+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:34:22.753272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814461+0000","last_change":"2024-05-28T14:06:25.423337+0000","last_active":"2024-05-28T14:06:26.814461+0000","last_peered":"2024-05-28T14:06:26.814461+0000","last_clean":"2024-05-28T14:06:26.814461+0000","last_became_active":"2024-05-28T14:06:25.422687+0000","last_became_peered":"2024-05-28T14:06:25.422687+0000","last_unstale":"2024-05-28T14:06:26.814461+0000","last_undegraded":"2024-05-28T14:06:26.814461+0000","last_fullsized":"2024-05-28T14:06:26.814461+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:52:13.920211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816190+0000","last_change":"2024-05-28T14:06:24.428701+0000","last_active":"2024-05-28T14:06:26.816190+0000","last_peered":"2024-05-28T14:06:26.816190+0000","last_clean":"2024-05-28T14:06:26.816190+0000","last_became_active":"2024-05-28T14:06:24.428477+0000","last_became_peered":"2024-05-28T14:06:24.428477+0000","last_unstale":"2024-05-28T14:06:26.816190+0000","last_undegraded":"2024-05-28T14:06:26.816190+0000","last_fullsized":"2024-05-28T14:06:26.816190+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:54:28.504352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812539+0000","last_change":"2024-05-28T14:06:25.425033+0000","last_active":"2024-05-28T14:06:26.812539+0000","last_peered":"2024-05-28T14:06:26.812539+0000","last_clean":"2024-05-28T14:06:26.812539+0000","last_became_active":"2024-05-28T14:06:25.424735+0000","last_became_peered":"2024-05-28T14:06:25.424735+0000","last_unstale":"2024-05-28T14:06:26.812539+0000","last_undegraded":"2024-05-28T14:06:26.812539+0000","last_fullsized":"2024-05-28T14:06:26.812539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:22:46.397408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815605+0000","last_change":"2024-05-28T14:06:24.437928+0000","last_active":"2024-05-28T14:06:26.815605+0000","last_peered":"2024-05-28T14:06:26.815605+0000","last_clean":"2024-05-28T14:06:26.815605+0000","last_became_active":"2024-05-28T14:06:24.437366+0000","last_became_peered":"2024-05-28T14:06:24.437366+0000","last_unstale":"2024-05-28T14:06:26.815605+0000","last_undegraded":"2024-05-28T14:06:26.815605+0000","last_fullsized":"2024-05-28T14:06:26.815605+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:20:05.724578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814078+0000","last_change":"2024-05-28T14:06:25.432485+0000","last_active":"2024-05-28T14:06:26.814078+0000","last_peered":"2024-05-28T14:06:26.814078+0000","last_clean":"2024-05-28T14:06:26.814078+0000","last_became_active":"2024-05-28T14:06:25.432222+0000","last_became_peered":"2024-05-28T14:06:25.432222+0000","last_unstale":"2024-05-28T14:06:26.814078+0000","last_undegraded":"2024-05-28T14:06:26.814078+0000","last_fullsized":"2024-05-28T14:06:26.814078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:18:13.308854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815959+0000","last_change":"2024-05-28T14:06:24.429400+0000","last_active":"2024-05-28T14:06:26.815959+0000","last_peered":"2024-05-28T14:06:26.815959+0000","last_clean":"2024-05-28T14:06:26.815959+0000","last_became_active":"2024-05-28T14:06:24.429124+0000","last_became_peered":"2024-05-28T14:06:24.429124+0000","last_unstale":"2024-05-28T14:06:26.815959+0000","last_undegraded":"2024-05-28T14:06:26.815959+0000","last_fullsized":"2024-05-28T14:06:26.815959+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:22:45.391365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.813660+0000","last_change":"2024-05-28T14:06:25.436652+0000","last_active":"2024-05-28T14:06:26.813660+0000","last_peered":"2024-05-28T14:06:26.813660+0000","last_clean":"2024-05-28T14:06:26.813660+0000","last_became_active":"2024-05-28T14:06:25.436330+0000","last_became_peered":"2024-05-28T14:06:25.436330+0000","last_unstale":"2024-05-28T14:06:26.813660+0000","last_undegraded":"2024-05-28T14:06:26.813660+0000","last_fullsized":"2024-05-28T14:06:26.813660+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:14:20.201888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431315+0000","last_change":"2024-05-28T14:06:24.420613+0000","last_active":"2024-05-28T14:06:26.431315+0000","last_peered":"2024-05-28T14:06:26.431315+0000","last_clean":"2024-05-28T14:06:26.431315+0000","last_became_active":"2024-05-28T14:06:24.420171+0000","last_became_peered":"2024-05-28T14:06:24.420171+0000","last_unstale":"2024-05-28T14:06:26.431315+0000","last_undegraded":"2024-05-28T14:06:26.431315+0000","last_fullsized":"2024-05-28T14:06:26.431315+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:06:23.658146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.813678+0000","last_change":"2024-05-28T14:06:25.436950+0000","last_active":"2024-05-28T14:06:26.813678+0000","last_peered":"2024-05-28T14:06:26.813678+0000","last_clean":"2024-05-28T14:06:26.813678+0000","last_became_active":"2024-05-28T14:06:25.436591+0000","last_became_peered":"2024-05-28T14:06:25.436591+0000","last_unstale":"2024-05-28T14:06:26.813678+0000","last_undegraded":"2024-05-28T14:06:26.813678+0000","last_fullsized":"2024-05-28T14:06:26.813678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:42:29.665913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428218+0000","last_change":"2024-05-28T14:06:24.427464+0000","last_active":"2024-05-28T14:06:26.428218+0000","last_peered":"2024-05-28T14:06:26.428218+0000","last_clean":"2024-05-28T14:06:26.428218+0000","last_became_active":"2024-05-28T14:06:24.427145+0000","last_became_peered":"2024-05-28T14:06:24.427145+0000","last_unstale":"2024-05-28T14:06:26.428218+0000","last_undegraded":"2024-05-28T14:06:26.428218+0000","last_fullsized":"2024-05-28T14:06:26.428218+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:58:08.424233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428742+0000","last_change":"2024-05-28T14:06:25.423080+0000","last_active":"2024-05-28T14:06:26.428742+0000","last_peered":"2024-05-28T14:06:26.428742+0000","last_clean":"2024-05-28T14:06:26.428742+0000","last_became_active":"2024-05-28T14:06:25.422305+0000","last_became_peered":"2024-05-28T14:06:25.422305+0000","last_unstale":"2024-05-28T14:06:26.428742+0000","last_undegraded":"2024-05-28T14:06:26.428742+0000","last_fullsized":"2024-05-28T14:06:26.428742+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:44.471299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816497+0000","last_change":"2024-05-28T14:06:24.432369+0000","last_active":"2024-05-28T14:06:26.816497+0000","last_peered":"2024-05-28T14:06:26.816497+0000","last_clean":"2024-05-28T14:06:26.816497+0000","last_became_active":"2024-05-28T14:06:24.431992+0000","last_became_peered":"2024-05-28T14:06:24.431992+0000","last_unstale":"2024-05-28T14:06:26.816497+0000","last_undegraded":"2024-05-28T14:06:26.816497+0000","last_fullsized":"2024-05-28T14:06:26.816497+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:42:01.134570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431538+0000","last_change":"2024-05-28T14:06:25.427193+0000","last_active":"2024-05-28T14:06:26.431538+0000","last_peered":"2024-05-28T14:06:26.431538+0000","last_clean":"2024-05-28T14:06:26.431538+0000","last_became_active":"2024-05-28T14:06:25.426889+0000","last_became_peered":"2024-05-28T14:06:25.426889+0000","last_unstale":"2024-05-28T14:06:26.431538+0000","last_undegraded":"2024-05-28T14:06:26.431538+0000","last_fullsized":"2024-05-28T14:06:26.431538+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:42:08.042876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.432269+0000","last_change":"2024-05-28T14:06:24.427952+0000","last_active":"2024-05-28T14:06:26.432269+0000","last_peered":"2024-05-28T14:06:26.432269+0000","last_clean":"2024-05-28T14:06:26.432269+0000","last_became_active":"2024-05-28T14:06:24.427511+0000","last_became_peered":"2024-05-28T14:06:24.427511+0000","last_unstale":"2024-05-28T14:06:26.432269+0000","last_undegraded":"2024-05-28T14:06:26.432269+0000","last_fullsized":"2024-05-28T14:06:26.432269+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:34:47.264602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815624+0000","last_change":"2024-05-28T14:06:25.445031+0000","last_active":"2024-05-28T14:06:26.815624+0000","last_peered":"2024-05-28T14:06:26.815624+0000","last_clean":"2024-05-28T14:06:26.815624+0000","last_became_active":"2024-05-28T14:06:25.443895+0000","last_became_peered":"2024-05-28T14:06:25.443895+0000","last_unstale":"2024-05-28T14:06:26.815624+0000","last_undegraded":"2024-05-28T14:06:26.815624+0000","last_fullsized":"2024-05-28T14:06:26.815624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:32:49.349214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812576+0000","last_change":"2024-05-28T14:06:24.427264+0000","last_active":"2024-05-28T14:06:26.812576+0000","last_peered":"2024-05-28T14:06:26.812576+0000","last_clean":"2024-05-28T14:06:26.812576+0000","last_became_active":"2024-05-28T14:06:24.426662+0000","last_became_peered":"2024-05-28T14:06:24.426662+0000","last_unstale":"2024-05-28T14:06:26.812576+0000","last_undegraded":"2024-05-28T14:06:26.812576+0000","last_fullsized":"2024-05-28T14:06:26.812576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:53:56.024111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815464+0000","last_change":"2024-05-28T14:06:25.439712+0000","last_active":"2024-05-28T14:06:26.815464+0000","last_peered":"2024-05-28T14:06:26.815464+0000","last_clean":"2024-05-28T14:06:26.815464+0000","last_became_active":"2024-05-28T14:06:25.438826+0000","last_became_peered":"2024-05-28T14:06:25.438826+0000","last_unstale":"2024-05-28T14:06:26.815464+0000","last_undegraded":"2024-05-28T14:06:26.815464+0000","last_fullsized":"2024-05-28T14:06:26.815464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:50:50.831055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428504+0000","last_change":"2024-05-28T14:06:24.431741+0000","last_active":"2024-05-28T14:06:26.428504+0000","last_peered":"2024-05-28T14:06:26.428504+0000","last_clean":"2024-05-28T14:06:26.428504+0000","last_became_active":"2024-05-28T14:06:24.431388+0000","last_became_peered":"2024-05-28T14:06:24.431388+0000","last_unstale":"2024-05-28T14:06:26.428504+0000","last_undegraded":"2024-05-28T14:06:26.428504+0000","last_fullsized":"2024-05-28T14:06:26.428504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:27:28.253532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812448+0000","last_change":"2024-05-28T14:06:25.433206+0000","last_active":"2024-05-28T14:06:26.812448+0000","last_peered":"2024-05-28T14:06:26.812448+0000","last_clean":"2024-05-28T14:06:26.812448+0000","last_became_active":"2024-05-28T14:06:25.432909+0000","last_became_peered":"2024-05-28T14:06:25.432909+0000","last_unstale":"2024-05-28T14:06:26.812448+0000","last_undegraded":"2024-05-28T14:06:26.812448+0000","last_fullsized":"2024-05-28T14:06:26.812448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:27.322317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812351+0000","last_change":"2024-05-28T14:06:24.420374+0000","last_active":"2024-05-28T14:06:26.812351+0000","last_peered":"2024-05-28T14:06:26.812351+0000","last_clean":"2024-05-28T14:06:26.812351+0000","last_became_active":"2024-05-28T14:06:24.420044+0000","last_became_peered":"2024-05-28T14:06:24.420044+0000","last_unstale":"2024-05-28T14:06:26.812351+0000","last_undegraded":"2024-05-28T14:06:26.812351+0000","last_fullsized":"2024-05-28T14:06:26.812351+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:12:23.194270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815942+0000","last_change":"2024-05-28T14:06:25.442435+0000","last_active":"2024-05-28T14:06:26.815942+0000","last_peered":"2024-05-28T14:06:26.815942+0000","last_clean":"2024-05-28T14:06:26.815942+0000","last_became_active":"2024-05-28T14:06:25.442182+0000","last_became_peered":"2024-05-28T14:06:25.442182+0000","last_unstale":"2024-05-28T14:06:26.815942+0000","last_undegraded":"2024-05-28T14:06:26.815942+0000","last_fullsized":"2024-05-28T14:06:26.815942+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:27:03.768824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.813888+0000","last_change":"2024-05-28T14:06:24.422804+0000","last_active":"2024-05-28T14:06:26.813888+0000","last_peered":"2024-05-28T14:06:26.813888+0000","last_clean":"2024-05-28T14:06:26.813888+0000","last_became_active":"2024-05-28T14:06:24.422468+0000","last_became_peered":"2024-05-28T14:06:24.422468+0000","last_unstale":"2024-05-28T14:06:26.813888+0000","last_undegraded":"2024-05-28T14:06:26.813888+0000","last_fullsized":"2024-05-28T14:06:26.813888+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:53:17.596549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.813960+0000","last_change":"2024-05-28T14:06:25.428347+0000","last_active":"2024-05-28T14:06:26.813960+0000","last_peered":"2024-05-28T14:06:26.813960+0000","last_clean":"2024-05-28T14:06:26.813960+0000","last_became_active":"2024-05-28T14:06:25.428035+0000","last_became_peered":"2024-05-28T14:06:25.428035+0000","last_unstale":"2024-05-28T14:06:26.813960+0000","last_undegraded":"2024-05-28T14:06:26.813960+0000","last_fullsized":"2024-05-28T14:06:26.813960+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:54:29.510395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.430867+0000","last_change":"2024-05-28T14:06:24.429508+0000","last_active":"2024-05-28T14:06:26.430867+0000","last_peered":"2024-05-28T14:06:26.430867+0000","last_clean":"2024-05-28T14:06:26.430867+0000","last_became_active":"2024-05-28T14:06:24.429101+0000","last_became_peered":"2024-05-28T14:06:24.429101+0000","last_unstale":"2024-05-28T14:06:26.430867+0000","last_undegraded":"2024-05-28T14:06:26.430867+0000","last_fullsized":"2024-05-28T14:06:26.430867+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:28:49.528879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815702+0000","last_change":"2024-05-28T14:06:25.444941+0000","last_active":"2024-05-28T14:06:26.815702+0000","last_peered":"2024-05-28T14:06:26.815702+0000","last_clean":"2024-05-28T14:06:26.815702+0000","last_became_active":"2024-05-28T14:06:25.444505+0000","last_became_peered":"2024-05-28T14:06:25.444505+0000","last_unstale":"2024-05-28T14:06:26.815702+0000","last_undegraded":"2024-05-28T14:06:26.815702+0000","last_fullsized":"2024-05-28T14:06:26.815702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:17:03.474414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815343+0000","last_change":"2024-05-28T14:06:24.427387+0000","last_active":"2024-05-28T14:06:26.815343+0000","last_peered":"2024-05-28T14:06:26.815343+0000","last_clean":"2024-05-28T14:06:26.815343+0000","last_became_active":"2024-05-28T14:06:24.427106+0000","last_became_peered":"2024-05-28T14:06:24.427106+0000","last_unstale":"2024-05-28T14:06:26.815343+0000","last_undegraded":"2024-05-28T14:06:26.815343+0000","last_fullsized":"2024-05-28T14:06:26.815343+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:58:08.159980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.813779+0000","last_change":"2024-05-28T14:06:25.431355+0000","last_active":"2024-05-28T14:06:26.813779+0000","last_peered":"2024-05-28T14:06:26.813779+0000","last_clean":"2024-05-28T14:06:26.813779+0000","last_became_active":"2024-05-28T14:06:25.430927+0000","last_became_peered":"2024-05-28T14:06:25.430927+0000","last_unstale":"2024-05-28T14:06:26.813779+0000","last_undegraded":"2024-05-28T14:06:26.813779+0000","last_fullsized":"2024-05-28T14:06:26.813779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:22:05.399179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815957+0000","last_change":"2024-05-28T14:06:24.433815+0000","last_active":"2024-05-28T14:06:26.815957+0000","last_peered":"2024-05-28T14:06:26.815957+0000","last_clean":"2024-05-28T14:06:26.815957+0000","last_became_active":"2024-05-28T14:06:24.433524+0000","last_became_peered":"2024-05-28T14:06:24.433524+0000","last_unstale":"2024-05-28T14:06:26.815957+0000","last_undegraded":"2024-05-28T14:06:26.815957+0000","last_fullsized":"2024-05-28T14:06:26.815957+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:58:13.946282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":268,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814838+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:06:26.814838+0000","last_peered":"2024-05-28T14:06:26.814838+0000","last_clean":"2024-05-28T14:06:26.814838+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:06:26.814838+0000","last_undegraded":"2024-05-28T14:06:26.814838+0000","last_fullsized":"2024-05-28T14:06:26.814838+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59016,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312824,"statfs":{"total":96636764160,"available":96576331776,"internally_reserved":0,"allocated":1503232,"data_stored":975671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574950,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54280,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317560,"statfs":{"total":96636764160,"available":96581181440,"internally_reserved":0,"allocated":909312,"data_stored":384779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54276,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317564,"statfs":{"total":96636764160,"available":96581185536,"internally_reserved":0,"allocated":905216,"data_stored":384777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58432,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313408,"statfs":{"total":96636764160,"available":96576929792,"internally_reserved":0,"allocated":905216,"data_stored":384757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574950,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45956,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325884,"statfs":{"total":96636764160,"available":96589705216,"internally_reserved":0,"allocated":909312,"data_stored":384799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574951,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54284,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317556,"statfs":{"total":96636764160,"available":96581177344,"internally_reserved":0,"allocated":913408,"data_stored":386299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62616,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309224,"statfs":{"total":96636764160,"available":96572645376,"internally_reserved":0,"allocated":921600,"data_stored":386915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46532,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325308,"statfs":{"total":96636764160,"available":96589115392,"internally_reserved":0,"allocated":1495040,"data_stored":975055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:06:34.115 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:06:34.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.282+0000 7f5c6d400640 1 Processor -- start 2024-05-28T14:06:34.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.282+0000 7f5c6d400640 1 -- start start 2024-05-28T14:06:34.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.282+0000 7f5c6d400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c6815ba40 0x7f5c680d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:34.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.282+0000 7f5c6d400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5c68072dd0 0x7f5c680d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:34.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.282+0000 7f5c6d400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c6815f280 con 0x7f5c6806f660 2024-05-28T14:06:34.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.282+0000 7f5c6d400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5c6815f030 con 0x7f5c68072dd0 2024-05-28T14:06:34.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.282+0000 7f5c6d400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6815f560 con 0x7f5c6815ba40 2024-05-28T14:06:34.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.282+0000 7f5c5fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c6815ba40 0x7f5c680d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:34.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.282+0000 7f5c67200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5c68072dd0 0x7f5c680d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:34.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.282+0000 7f5c5fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c6815ba40 0x7f5c680d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:58648/0 (socket says 172.21.15.42:58648) 2024-05-28T14:06:34.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.282+0000 7f5c66800640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f5c6806f660 0x7f5c68073980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:44860/0 (socket says 172.21.15.42:44860) 2024-05-28T14:06:34.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.282+0000 7f5c67200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5c68072dd0 0x7f5c680d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:50628/0 (socket says 172.21.15.42:50628) 2024-05-28T14:06:34.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.282+0000 7f5c5fe00640 1 -- 172.21.15.42:0/3279425598 learned_addr learned my addr 172.21.15.42:0/3279425598 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:34.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c67200640 1 -- 172.21.15.42:0/3279425598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c6815ba40 msgr2=0x7f5c680d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:34.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c67200640 1 --2- 172.21.15.42:0/3279425598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c6815ba40 0x7f5c680d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c67200640 1 -- 172.21.15.42:0/3279425598 >> v1:172.21.15.63:6789/0 conn(0x7f5c6806f660 legacy=0x7f5c68073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:34.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c67200640 1 -- 172.21.15.42:0/3279425598 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6815f840 con 0x7f5c68072dd0 2024-05-28T14:06:34.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c5fe00640 1 --2- 172.21.15.42:0/3279425598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c6815ba40 0x7f5c680d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:06:34.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c65e00640 1 -- 172.21.15.42:0/3279425598 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4096578115 0 0) 0x7f5c6815f280 con 0x7f5c6806f660 2024-05-28T14:06:34.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c67200640 1 --2- 172.21.15.42:0/3279425598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c68072dd0 0x7f5c680d0c30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009d90 tx=0x7f5c6001c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=856464bb6e936512 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:34.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c65e00640 1 -- 172.21.15.42:0/3279425598 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c600b1070 con 0x7f5c68072dd0 2024-05-28T14:06:34.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c65e00640 1 -- 172.21.15.42:0/3279425598 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c600a4e50 con 0x7f5c68072dd0 2024-05-28T14:06:34.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c65e00640 1 -- 172.21.15.42:0/3279425598 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c600ac660 con 0x7f5c68072dd0 2024-05-28T14:06:34.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c6d400640 1 -- 172.21.15.42:0/3279425598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c68072dd0 msgr2=0x7f5c680d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:34.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c6d400640 1 --2- 172.21.15.42:0/3279425598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c68072dd0 0x7f5c680d0c30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009d90 tx=0x7f5c6001c500 comp rx=0 tx=0).stop 2024-05-28T14:06:34.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c6d400640 1 -- 172.21.15.42:0/3279425598 shutdown_connections 2024-05-28T14:06:34.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c6d400640 1 --2- 172.21.15.42:0/3279425598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c68072dd0 0x7f5c680d0c30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c6d400640 1 --2- 172.21.15.42:0/3279425598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c6815ba40 0x7f5c680d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.286+0000 7f5c6d400640 1 -- 172.21.15.42:0/3279425598 >> 172.21.15.42:0/3279425598 conn(0x7f5c6806bdf0 msgr2=0x7f5c6806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:34.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c6d400640 1 -- 172.21.15.42:0/3279425598 shutdown_connections 2024-05-28T14:06:34.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c6d400640 1 -- 172.21.15.42:0/3279425598 wait complete. 2024-05-28T14:06:34.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c6d400640 1 Processor -- start 2024-05-28T14:06:34.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c6d400640 1 -- start start 2024-05-28T14:06:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c6d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6806f660 0x7f5c68150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c6d400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c68072dd0 0x7f5c680ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c6d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c681515a0 0x7f5c680ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c6d400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5c68168f70 con 0x7f5c6806f660 2024-05-28T14:06:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c6d400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c68168df0 con 0x7f5c681515a0 2024-05-28T14:06:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c6d400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5c681690f0 con 0x7f5c68072dd0 2024-05-28T14:06:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c66800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6806f660 0x7f5c68150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c66800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6806f660 0x7f5c68150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53200/0 (socket says 172.21.15.42:53200) 2024-05-28T14:06:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c66800640 1 -- 172.21.15.42:0/2796436586 learned_addr learned my addr 172.21.15.42:0/2796436586 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c67200640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c681515a0 0x7f5c680ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c5fe00640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c68072dd0 0x7f5c680ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c66800640 1 -- 172.21.15.42:0/2796436586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c68072dd0 msgr2=0x7f5c680ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c66800640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c68072dd0 0x7f5c680ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c66800640 1 -- 172.21.15.42:0/2796436586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c681515a0 msgr2=0x7f5c680ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c66800640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c681515a0 0x7f5c680ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c66800640 1 -- 172.21.15.42:0/2796436586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c68151210 con 0x7f5c6806f660 2024-05-28T14:06:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c67200640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c681515a0 0x7f5c680ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.290+0000 7f5c66800640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6806f660 0x7f5c68150cd0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5c580011f0 tx=0x7f5c5800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.294+0000 7f5c5ea00640 1 -- 172.21.15.42:0/2796436586 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58018070 con 0x7f5c6806f660 2024-05-28T14:06:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.294+0000 7f5c5ea00640 1 -- 172.21.15.42:0/2796436586 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c580074e0 con 0x7f5c6806f660 2024-05-28T14:06:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.294+0000 7f5c6d400640 1 -- 172.21.15.42:0/2796436586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c68159ae0 con 0x7f5c6806f660 2024-05-28T14:06:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.294+0000 7f5c5ea00640 1 -- 172.21.15.42:0/2796436586 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58004a30 con 0x7f5c6806f660 2024-05-28T14:06:34.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.294+0000 7f5c6d400640 1 -- 172.21.15.42:0/2796436586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c6815a0a0 con 0x7f5c6806f660 2024-05-28T14:06:34.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.294+0000 7f5c6d400640 1 -- 172.21.15.42:0/2796436586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c68068a90 con 0x7f5c6806f660 2024-05-28T14:06:34.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.294+0000 7f5c5ea00640 1 -- 172.21.15.42:0/2796436586 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f5c58004bd0 con 0x7f5c6806f660 2024-05-28T14:06:34.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.294+0000 7f5c5ea00640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5c44069070 0x7f5c4406b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:34.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.294+0000 7f5c5ea00640 1 -- 172.21.15.42:0/2796436586 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7f5c58084990 con 0x7f5c6806f660 2024-05-28T14:06:34.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.294+0000 7f5c5fe00640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5c44069070 0x7f5c4406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:34.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.298+0000 7f5c5fe00640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5c44069070 0x7f5c4406b530 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5c6815c860 tx=0x7f5c50008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:34.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.302+0000 7f5c5ea00640 1 -- 172.21.15.42:0/2796436586 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5c580567e0 con 0x7f5c6806f660 2024-05-28T14:06:34.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.462+0000 7f5c6d400640 1 -- 172.21.15.42:0/2796436586 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5c6815ad80 con 0x7f5c44069070 2024-05-28T14:06:34.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.466+0000 7f5c5ea00640 1 -- 172.21.15.42:0/2796436586 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7f5c6815ad80 con 0x7f5c44069070 2024-05-28T14:06:34.457 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:34.458 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:06:34.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.470+0000 7f5c6d400640 1 -- 172.21.15.42:0/2796436586 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5c44069070 msgr2=0x7f5c4406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:34.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.470+0000 7f5c6d400640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5c44069070 0x7f5c4406b530 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5c6815c860 tx=0x7f5c50008040 comp rx=0 tx=0).stop 2024-05-28T14:06:34.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.470+0000 7f5c6d400640 1 -- 172.21.15.42:0/2796436586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6806f660 msgr2=0x7f5c68150cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:34.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.470+0000 7f5c6d400640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6806f660 0x7f5c68150cd0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5c580011f0 tx=0x7f5c5800c9b0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.470+0000 7f5c6d400640 1 -- 172.21.15.42:0/2796436586 shutdown_connections 2024-05-28T14:06:34.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.470+0000 7f5c6d400640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5c44069070 0x7f5c4406b530 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.470+0000 7f5c6d400640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c681515a0 0x7f5c680ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.470+0000 7f5c6d400640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c68072dd0 0x7f5c680ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.470+0000 7f5c6d400640 1 --2- 172.21.15.42:0/2796436586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6806f660 0x7f5c68150cd0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.470+0000 7f5c6d400640 1 -- 172.21.15.42:0/2796436586 >> 172.21.15.42:0/2796436586 conn(0x7f5c6806bdf0 msgr2=0x7f5c6815d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:34.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.470+0000 7f5c6d400640 1 -- 172.21.15.42:0/2796436586 shutdown_connections 2024-05-28T14:06:34.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.470+0000 7f5c6d400640 1 -- 172.21.15.42:0/2796436586 wait complete. 2024-05-28T14:06:34.477 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-05-28T14:06:32.996184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":435392,"kb_used_data":8264,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754539328,"statfs":{"total":773094113280,"available":772648271872,"internally_reserved":0,"allocated":8462336,"data_stored":4263052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":437243442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001846"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815319+0000","last_change":"2024-05-28T14:06:25.436476+0000","last_active":"2024-05-28T14:06:26.815319+0000","last_peered":"2024-05-28T14:06:26.815319+0000","last_clean":"2024-05-28T14:06:26.815319+0000","last_became_active":"2024-05-28T14:06:25.436147+0000","last_became_peered":"2024-05-28T14:06:25.436147+0000","last_unstale":"2024-05-28T14:06:26.815319+0000","last_undegraded":"2024-05-28T14:06:26.815319+0000","last_fullsized":"2024-05-28T14:06:26.815319+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:04:20.056562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816331+0000","last_change":"2024-05-28T14:06:25.435200+0000","last_active":"2024-05-28T14:06:26.816331+0000","last_peered":"2024-05-28T14:06:26.816331+0000","last_clean":"2024-05-28T14:06:26.816331+0000","last_became_active":"2024-05-28T14:06:25.429892+0000","last_became_peered":"2024-05-28T14:06:25.429892+0000","last_unstale":"2024-05-28T14:06:26.816331+0000","last_undegraded":"2024-05-28T14:06:26.816331+0000","last_fullsized":"2024-05-28T14:06:26.816331+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:03:35.592277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814690+0000","last_change":"2024-05-28T14:06:25.435750+0000","last_active":"2024-05-28T14:06:26.814690+0000","last_peered":"2024-05-28T14:06:26.814690+0000","last_clean":"2024-05-28T14:06:26.814690+0000","last_became_active":"2024-05-28T14:06:25.433833+0000","last_became_peered":"2024-05-28T14:06:25.433833+0000","last_unstale":"2024-05-28T14:06:26.814690+0000","last_undegraded":"2024-05-28T14:06:26.814690+0000","last_fullsized":"2024-05-28T14:06:26.814690+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:01:25.637892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814302+0000","last_change":"2024-05-28T14:06:25.432371+0000","last_active":"2024-05-28T14:06:26.814302+0000","last_peered":"2024-05-28T14:06:26.814302+0000","last_clean":"2024-05-28T14:06:26.814302+0000","last_became_active":"2024-05-28T14:06:25.432078+0000","last_became_peered":"2024-05-28T14:06:25.432078+0000","last_unstale":"2024-05-28T14:06:26.814302+0000","last_undegraded":"2024-05-28T14:06:26.814302+0000","last_fullsized":"2024-05-28T14:06:26.814302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:34:50.335279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812230+0000","last_change":"2024-05-28T14:06:25.430672+0000","last_active":"2024-05-28T14:06:26.812230+0000","last_peered":"2024-05-28T14:06:26.812230+0000","last_clean":"2024-05-28T14:06:26.812230+0000","last_became_active":"2024-05-28T14:06:25.430390+0000","last_became_peered":"2024-05-28T14:06:25.430390+0000","last_unstale":"2024-05-28T14:06:26.812230+0000","last_undegraded":"2024-05-28T14:06:26.812230+0000","last_fullsized":"2024-05-28T14:06:26.812230+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:04:38.407175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815845+0000","last_change":"2024-05-28T14:06:25.440954+0000","last_active":"2024-05-28T14:06:26.815845+0000","last_peered":"2024-05-28T14:06:26.815845+0000","last_clean":"2024-05-28T14:06:26.815845+0000","last_became_active":"2024-05-28T14:06:25.440634+0000","last_became_peered":"2024-05-28T14:06:25.440634+0000","last_unstale":"2024-05-28T14:06:26.815845+0000","last_undegraded":"2024-05-28T14:06:26.815845+0000","last_fullsized":"2024-05-28T14:06:26.815845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:33:29.862414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.430604+0000","last_change":"2024-05-28T14:06:25.428279+0000","last_active":"2024-05-28T14:06:26.430604+0000","last_peered":"2024-05-28T14:06:26.430604+0000","last_clean":"2024-05-28T14:06:26.430604+0000","last_became_active":"2024-05-28T14:06:25.426405+0000","last_became_peered":"2024-05-28T14:06:25.426405+0000","last_unstale":"2024-05-28T14:06:26.430604+0000","last_undegraded":"2024-05-28T14:06:26.430604+0000","last_fullsized":"2024-05-28T14:06:26.430604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:19:41.907890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815090+0000","last_change":"2024-05-28T14:06:25.433150+0000","last_active":"2024-05-28T14:06:26.815090+0000","last_peered":"2024-05-28T14:06:26.815090+0000","last_clean":"2024-05-28T14:06:26.815090+0000","last_became_active":"2024-05-28T14:06:25.432858+0000","last_became_peered":"2024-05-28T14:06:25.432858+0000","last_unstale":"2024-05-28T14:06:26.815090+0000","last_undegraded":"2024-05-28T14:06:26.815090+0000","last_fullsized":"2024-05-28T14:06:26.815090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:34:56.946212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812514+0000","last_change":"2024-05-28T14:06:25.435943+0000","last_active":"2024-05-28T14:06:26.812514+0000","last_peered":"2024-05-28T14:06:26.812514+0000","last_clean":"2024-05-28T14:06:26.812514+0000","last_became_active":"2024-05-28T14:06:25.435445+0000","last_became_peered":"2024-05-28T14:06:25.435445+0000","last_unstale":"2024-05-28T14:06:26.812514+0000","last_undegraded":"2024-05-28T14:06:26.812514+0000","last_fullsized":"2024-05-28T14:06:26.812514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:58:09.166023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431778+0000","last_change":"2024-05-28T14:06:25.429227+0000","last_active":"2024-05-28T14:06:26.431778+0000","last_peered":"2024-05-28T14:06:26.431778+0000","last_clean":"2024-05-28T14:06:26.431778+0000","last_became_active":"2024-05-28T14:06:25.428962+0000","last_became_peered":"2024-05-28T14:06:25.428962+0000","last_unstale":"2024-05-28T14:06:26.431778+0000","last_undegraded":"2024-05-28T14:06:26.431778+0000","last_fullsized":"2024-05-28T14:06:26.431778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:06:24.664189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815222+0000","last_change":"2024-05-28T14:06:25.439943+0000","last_active":"2024-05-28T14:06:26.815222+0000","last_peered":"2024-05-28T14:06:26.815222+0000","last_clean":"2024-05-28T14:06:26.815222+0000","last_became_active":"2024-05-28T14:06:25.439316+0000","last_became_peered":"2024-05-28T14:06:25.439316+0000","last_unstale":"2024-05-28T14:06:26.815222+0000","last_undegraded":"2024-05-28T14:06:26.815222+0000","last_fullsized":"2024-05-28T14:06:26.815222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:02:25.085918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814370+0000","last_change":"2024-05-28T14:06:25.423591+0000","last_active":"2024-05-28T14:06:26.814370+0000","last_peered":"2024-05-28T14:06:26.814370+0000","last_clean":"2024-05-28T14:06:26.814370+0000","last_became_active":"2024-05-28T14:06:25.423051+0000","last_became_peered":"2024-05-28T14:06:25.423051+0000","last_unstale":"2024-05-28T14:06:26.814370+0000","last_undegraded":"2024-05-28T14:06:26.814370+0000","last_fullsized":"2024-05-28T14:06:26.814370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:42:02.140613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428441+0000","last_change":"2024-05-28T14:06:25.433443+0000","last_active":"2024-05-28T14:06:26.428441+0000","last_peered":"2024-05-28T14:06:26.428441+0000","last_clean":"2024-05-28T14:06:26.428441+0000","last_became_active":"2024-05-28T14:06:25.433165+0000","last_became_peered":"2024-05-28T14:06:25.433165+0000","last_unstale":"2024-05-28T14:06:26.428441+0000","last_undegraded":"2024-05-28T14:06:26.428441+0000","last_fullsized":"2024-05-28T14:06:26.428441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:20:06.730622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812530+0000","last_change":"2024-05-28T14:06:25.423146+0000","last_active":"2024-05-28T14:06:26.812530+0000","last_peered":"2024-05-28T14:06:26.812530+0000","last_clean":"2024-05-28T14:06:26.812530+0000","last_became_active":"2024-05-28T14:06:25.422848+0000","last_became_peered":"2024-05-28T14:06:25.422848+0000","last_unstale":"2024-05-28T14:06:26.812530+0000","last_undegraded":"2024-05-28T14:06:26.812530+0000","last_fullsized":"2024-05-28T14:06:26.812530+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:31.303741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428426+0000","last_change":"2024-05-28T14:06:25.429616+0000","last_active":"2024-05-28T14:06:26.428426+0000","last_peered":"2024-05-28T14:06:26.428426+0000","last_clean":"2024-05-28T14:06:26.428426+0000","last_became_active":"2024-05-28T14:06:25.428642+0000","last_became_peered":"2024-05-28T14:06:25.428642+0000","last_unstale":"2024-05-28T14:06:26.428426+0000","last_undegraded":"2024-05-28T14:06:26.428426+0000","last_fullsized":"2024-05-28T14:06:26.428426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:42:08.042876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.430964+0000","last_change":"2024-05-28T14:06:25.431909+0000","last_active":"2024-05-28T14:06:26.430964+0000","last_peered":"2024-05-28T14:06:26.430964+0000","last_clean":"2024-05-28T14:06:26.430964+0000","last_became_active":"2024-05-28T14:06:25.431610+0000","last_became_peered":"2024-05-28T14:06:25.431610+0000","last_unstale":"2024-05-28T14:06:26.430964+0000","last_undegraded":"2024-05-28T14:06:26.430964+0000","last_fullsized":"2024-05-28T14:06:26.430964+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:27:03.768824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812676+0000","last_change":"2024-05-28T14:06:25.435839+0000","last_active":"2024-05-28T14:06:26.812676+0000","last_peered":"2024-05-28T14:06:26.812676+0000","last_clean":"2024-05-28T14:06:26.812676+0000","last_became_active":"2024-05-28T14:06:25.435378+0000","last_became_peered":"2024-05-28T14:06:25.435378+0000","last_unstale":"2024-05-28T14:06:26.812676+0000","last_undegraded":"2024-05-28T14:06:26.812676+0000","last_fullsized":"2024-05-28T14:06:26.812676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:53:18.602592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814169+0000","last_change":"2024-05-28T14:06:25.432059+0000","last_active":"2024-05-28T14:06:26.814169+0000","last_peered":"2024-05-28T14:06:26.814169+0000","last_clean":"2024-05-28T14:06:26.814169+0000","last_became_active":"2024-05-28T14:06:25.431758+0000","last_became_peered":"2024-05-28T14:06:25.431758+0000","last_unstale":"2024-05-28T14:06:26.814169+0000","last_undegraded":"2024-05-28T14:06:26.814169+0000","last_fullsized":"2024-05-28T14:06:26.814169+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:34:23.759315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816090+0000","last_change":"2024-05-28T14:06:25.434141+0000","last_active":"2024-05-28T14:06:26.816090+0000","last_peered":"2024-05-28T14:06:26.816090+0000","last_clean":"2024-05-28T14:06:26.816090+0000","last_became_active":"2024-05-28T14:06:25.433659+0000","last_became_peered":"2024-05-28T14:06:25.433659+0000","last_unstale":"2024-05-28T14:06:26.816090+0000","last_undegraded":"2024-05-28T14:06:26.816090+0000","last_fullsized":"2024-05-28T14:06:26.816090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:50.332524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428393+0000","last_change":"2024-05-28T14:06:25.428098+0000","last_active":"2024-05-28T14:06:26.428393+0000","last_peered":"2024-05-28T14:06:26.428393+0000","last_clean":"2024-05-28T14:06:26.428393+0000","last_became_active":"2024-05-28T14:06:25.427805+0000","last_became_peered":"2024-05-28T14:06:25.427805+0000","last_unstale":"2024-05-28T14:06:26.428393+0000","last_undegraded":"2024-05-28T14:06:26.428393+0000","last_fullsized":"2024-05-28T14:06:26.428393+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:09.597247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431052+0000","last_change":"2024-05-28T14:06:25.430100+0000","last_active":"2024-05-28T14:06:26.431052+0000","last_peered":"2024-05-28T14:06:26.431052+0000","last_clean":"2024-05-28T14:06:26.431052+0000","last_became_active":"2024-05-28T14:06:25.429842+0000","last_became_peered":"2024-05-28T14:06:25.429842+0000","last_unstale":"2024-05-28T14:06:26.431052+0000","last_undegraded":"2024-05-28T14:06:26.431052+0000","last_fullsized":"2024-05-28T14:06:26.431052+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:50.332524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431178+0000","last_change":"2024-05-28T14:06:25.432563+0000","last_active":"2024-05-28T14:06:26.431178+0000","last_peered":"2024-05-28T14:06:26.431178+0000","last_clean":"2024-05-28T14:06:26.431178+0000","last_became_active":"2024-05-28T14:06:25.432252+0000","last_became_peered":"2024-05-28T14:06:25.432252+0000","last_unstale":"2024-05-28T14:06:26.431178+0000","last_undegraded":"2024-05-28T14:06:26.431178+0000","last_fullsized":"2024-05-28T14:06:26.431178+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:26:46.087828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814797+0000","last_change":"2024-05-28T14:06:25.437622+0000","last_active":"2024-05-28T14:06:26.814797+0000","last_peered":"2024-05-28T14:06:26.814797+0000","last_clean":"2024-05-28T14:06:26.814797+0000","last_became_active":"2024-05-28T14:06:25.437270+0000","last_became_peered":"2024-05-28T14:06:25.437270+0000","last_unstale":"2024-05-28T14:06:26.814797+0000","last_undegraded":"2024-05-28T14:06:26.814797+0000","last_fullsized":"2024-05-28T14:06:26.814797+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:07:33.366108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815240+0000","last_change":"2024-05-28T14:06:25.445645+0000","last_active":"2024-05-28T14:06:26.815240+0000","last_peered":"2024-05-28T14:06:26.815240+0000","last_clean":"2024-05-28T14:06:26.815240+0000","last_became_active":"2024-05-28T14:06:25.445274+0000","last_became_peered":"2024-05-28T14:06:25.445274+0000","last_unstale":"2024-05-28T14:06:26.815240+0000","last_undegraded":"2024-05-28T14:06:26.815240+0000","last_fullsized":"2024-05-28T14:06:26.815240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:56:13.305914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431535+0000","last_change":"2024-05-28T14:06:25.437597+0000","last_active":"2024-05-28T14:06:26.431535+0000","last_peered":"2024-05-28T14:06:26.431535+0000","last_clean":"2024-05-28T14:06:26.431535+0000","last_became_active":"2024-05-28T14:06:25.437267+0000","last_became_peered":"2024-05-28T14:06:25.437267+0000","last_unstale":"2024-05-28T14:06:26.431535+0000","last_undegraded":"2024-05-28T14:06:26.431535+0000","last_fullsized":"2024-05-28T14:06:26.431535+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:55:04.339742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.432101+0000","last_change":"2024-05-28T14:06:25.436883+0000","last_active":"2024-05-28T14:06:26.432101+0000","last_peered":"2024-05-28T14:06:26.432101+0000","last_clean":"2024-05-28T14:06:26.432101+0000","last_became_active":"2024-05-28T14:06:25.436484+0000","last_became_peered":"2024-05-28T14:06:25.436484+0000","last_unstale":"2024-05-28T14:06:26.432101+0000","last_undegraded":"2024-05-28T14:06:26.432101+0000","last_fullsized":"2024-05-28T14:06:26.432101+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:10:33.978239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814687+0000","last_change":"2024-05-28T14:06:25.434166+0000","last_active":"2024-05-28T14:06:26.814687+0000","last_peered":"2024-05-28T14:06:26.814687+0000","last_clean":"2024-05-28T14:06:26.814687+0000","last_became_active":"2024-05-28T14:06:25.433541+0000","last_became_peered":"2024-05-28T14:06:25.433541+0000","last_unstale":"2024-05-28T14:06:26.814687+0000","last_undegraded":"2024-05-28T14:06:26.814687+0000","last_fullsized":"2024-05-28T14:06:26.814687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:57:49.955869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.432075+0000","last_change":"2024-05-28T14:06:25.432535+0000","last_active":"2024-05-28T14:06:26.432075+0000","last_peered":"2024-05-28T14:06:26.432075+0000","last_clean":"2024-05-28T14:06:26.432075+0000","last_became_active":"2024-05-28T14:06:25.432243+0000","last_became_peered":"2024-05-28T14:06:25.432243+0000","last_unstale":"2024-05-28T14:06:26.432075+0000","last_undegraded":"2024-05-28T14:06:26.432075+0000","last_fullsized":"2024-05-28T14:06:26.432075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:11:07.262123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.813755+0000","last_change":"2024-05-28T14:06:25.434512+0000","last_active":"2024-05-28T14:06:26.813755+0000","last_peered":"2024-05-28T14:06:26.813755+0000","last_clean":"2024-05-28T14:06:26.813755+0000","last_became_active":"2024-05-28T14:06:25.434154+0000","last_became_peered":"2024-05-28T14:06:25.434154+0000","last_unstale":"2024-05-28T14:06:26.813755+0000","last_undegraded":"2024-05-28T14:06:26.813755+0000","last_fullsized":"2024-05-28T14:06:26.813755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:47:43.307227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431355+0000","last_change":"2024-05-28T14:06:25.431700+0000","last_active":"2024-05-28T14:06:26.431355+0000","last_peered":"2024-05-28T14:06:26.431355+0000","last_clean":"2024-05-28T14:06:26.431355+0000","last_became_active":"2024-05-28T14:06:25.430896+0000","last_became_peered":"2024-05-28T14:06:25.430896+0000","last_unstale":"2024-05-28T14:06:26.431355+0000","last_undegraded":"2024-05-28T14:06:26.431355+0000","last_fullsized":"2024-05-28T14:06:26.431355+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:32:01.620163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431818+0000","last_change":"2024-05-28T14:06:25.426047+0000","last_active":"2024-05-28T14:06:26.431818+0000","last_peered":"2024-05-28T14:06:26.431818+0000","last_clean":"2024-05-28T14:06:26.431818+0000","last_became_active":"2024-05-28T14:06:25.425752+0000","last_became_peered":"2024-05-28T14:06:25.425752+0000","last_unstale":"2024-05-28T14:06:26.431818+0000","last_undegraded":"2024-05-28T14:06:26.431818+0000","last_fullsized":"2024-05-28T14:06:26.431818+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:20:06.730622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815244+0000","last_change":"2024-05-28T14:06:25.440590+0000","last_active":"2024-05-28T14:06:26.815244+0000","last_peered":"2024-05-28T14:06:26.815244+0000","last_clean":"2024-05-28T14:06:26.815244+0000","last_became_active":"2024-05-28T14:06:25.439210+0000","last_became_peered":"2024-05-28T14:06:25.439210+0000","last_unstale":"2024-05-28T14:06:26.815244+0000","last_undegraded":"2024-05-28T14:06:26.815244+0000","last_fullsized":"2024-05-28T14:06:26.815244+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:53:54.369051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815182+0000","last_change":"2024-05-28T14:06:25.445738+0000","last_active":"2024-05-28T14:06:26.815182+0000","last_peered":"2024-05-28T14:06:26.815182+0000","last_clean":"2024-05-28T14:06:26.815182+0000","last_became_active":"2024-05-28T14:06:25.445392+0000","last_became_peered":"2024-05-28T14:06:25.445392+0000","last_unstale":"2024-05-28T14:06:26.815182+0000","last_undegraded":"2024-05-28T14:06:26.815182+0000","last_fullsized":"2024-05-28T14:06:26.815182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:35:56.185872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815557+0000","last_change":"2024-05-28T14:06:24.435935+0000","last_active":"2024-05-28T14:06:26.815557+0000","last_peered":"2024-05-28T14:06:26.815557+0000","last_clean":"2024-05-28T14:06:26.815557+0000","last_became_active":"2024-05-28T14:06:24.435637+0000","last_became_peered":"2024-05-28T14:06:24.435637+0000","last_unstale":"2024-05-28T14:06:26.815557+0000","last_undegraded":"2024-05-28T14:06:26.815557+0000","last_fullsized":"2024-05-28T14:06:26.815557+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:42:07.950563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812744+0000","last_change":"2024-05-28T14:06:25.435699+0000","last_active":"2024-05-28T14:06:26.812744+0000","last_peered":"2024-05-28T14:06:26.812744+0000","last_clean":"2024-05-28T14:06:26.812744+0000","last_became_active":"2024-05-28T14:06:25.434919+0000","last_became_peered":"2024-05-28T14:06:25.434919+0000","last_unstale":"2024-05-28T14:06:26.812744+0000","last_undegraded":"2024-05-28T14:06:26.812744+0000","last_fullsized":"2024-05-28T14:06:26.812744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:58:14.952326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428734+0000","last_change":"2024-05-28T14:06:24.427568+0000","last_active":"2024-05-28T14:06:26.428734+0000","last_peered":"2024-05-28T14:06:26.428734+0000","last_clean":"2024-05-28T14:06:26.428734+0000","last_became_active":"2024-05-28T14:06:24.427288+0000","last_became_peered":"2024-05-28T14:06:24.427288+0000","last_unstale":"2024-05-28T14:06:26.428734+0000","last_undegraded":"2024-05-28T14:06:26.428734+0000","last_fullsized":"2024-05-28T14:06:26.428734+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:54.168407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816219+0000","last_change":"2024-05-28T14:06:25.443335+0000","last_active":"2024-05-28T14:06:26.816219+0000","last_peered":"2024-05-28T14:06:26.816219+0000","last_clean":"2024-05-28T14:06:26.816219+0000","last_became_active":"2024-05-28T14:06:25.441253+0000","last_became_peered":"2024-05-28T14:06:25.441253+0000","last_unstale":"2024-05-28T14:06:26.816219+0000","last_undegraded":"2024-05-28T14:06:26.816219+0000","last_fullsized":"2024-05-28T14:06:26.816219+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:26:46.087828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431643+0000","last_change":"2024-05-28T14:06:24.435174+0000","last_active":"2024-05-28T14:06:26.431643+0000","last_peered":"2024-05-28T14:06:26.431643+0000","last_clean":"2024-05-28T14:06:26.431643+0000","last_became_active":"2024-05-28T14:06:24.434890+0000","last_became_peered":"2024-05-28T14:06:24.434890+0000","last_unstale":"2024-05-28T14:06:26.431643+0000","last_undegraded":"2024-05-28T14:06:26.431643+0000","last_fullsized":"2024-05-28T14:06:26.431643+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:54:28.504352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815382+0000","last_change":"2024-05-28T14:06:25.438263+0000","last_active":"2024-05-28T14:06:26.815382+0000","last_peered":"2024-05-28T14:06:26.815382+0000","last_clean":"2024-05-28T14:06:26.815382+0000","last_became_active":"2024-05-28T14:06:25.437316+0000","last_became_peered":"2024-05-28T14:06:25.437316+0000","last_unstale":"2024-05-28T14:06:26.815382+0000","last_undegraded":"2024-05-28T14:06:26.815382+0000","last_fullsized":"2024-05-28T14:06:26.815382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:00:56.168101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816377+0000","last_change":"2024-05-28T14:06:24.428906+0000","last_active":"2024-05-28T14:06:26.816377+0000","last_peered":"2024-05-28T14:06:26.816377+0000","last_clean":"2024-05-28T14:06:26.816377+0000","last_became_active":"2024-05-28T14:06:24.427835+0000","last_became_peered":"2024-05-28T14:06:24.427835+0000","last_unstale":"2024-05-28T14:06:26.816377+0000","last_undegraded":"2024-05-28T14:06:26.816377+0000","last_fullsized":"2024-05-28T14:06:26.816377+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:41:19.574984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431511+0000","last_change":"2024-05-28T14:06:25.435910+0000","last_active":"2024-05-28T14:06:26.431511+0000","last_peered":"2024-05-28T14:06:26.431511+0000","last_clean":"2024-05-28T14:06:26.431511+0000","last_became_active":"2024-05-28T14:06:25.435620+0000","last_became_peered":"2024-05-28T14:06:25.435620+0000","last_unstale":"2024-05-28T14:06:26.431511+0000","last_undegraded":"2024-05-28T14:06:26.431511+0000","last_fullsized":"2024-05-28T14:06:26.431511+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:09.597247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814170+0000","last_change":"2024-05-28T14:06:24.422900+0000","last_active":"2024-05-28T14:06:26.814170+0000","last_peered":"2024-05-28T14:06:26.814170+0000","last_clean":"2024-05-28T14:06:26.814170+0000","last_became_active":"2024-05-28T14:06:24.422439+0000","last_became_peered":"2024-05-28T14:06:24.422439+0000","last_unstale":"2024-05-28T14:06:26.814170+0000","last_undegraded":"2024-05-28T14:06:26.814170+0000","last_fullsized":"2024-05-28T14:06:26.814170+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:02:24.079875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431360+0000","last_change":"2024-05-28T14:06:25.437152+0000","last_active":"2024-05-28T14:06:26.431360+0000","last_peered":"2024-05-28T14:06:26.431360+0000","last_clean":"2024-05-28T14:06:26.431360+0000","last_became_active":"2024-05-28T14:06:25.435921+0000","last_became_peered":"2024-05-28T14:06:25.435921+0000","last_unstale":"2024-05-28T14:06:26.431360+0000","last_undegraded":"2024-05-28T14:06:26.431360+0000","last_fullsized":"2024-05-28T14:06:26.431360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:03:35.592277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815823+0000","last_change":"2024-05-28T14:06:24.431434+0000","last_active":"2024-05-28T14:06:26.815823+0000","last_peered":"2024-05-28T14:06:26.815823+0000","last_clean":"2024-05-28T14:06:26.815823+0000","last_became_active":"2024-05-28T14:06:24.430297+0000","last_became_peered":"2024-05-28T14:06:24.430297+0000","last_unstale":"2024-05-28T14:06:26.815823+0000","last_undegraded":"2024-05-28T14:06:26.815823+0000","last_fullsized":"2024-05-28T14:06:26.815823+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:26.316274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812074+0000","last_change":"2024-05-28T14:06:25.421716+0000","last_active":"2024-05-28T14:06:26.812074+0000","last_peered":"2024-05-28T14:06:26.812074+0000","last_clean":"2024-05-28T14:06:26.812074+0000","last_became_active":"2024-05-28T14:06:25.421417+0000","last_became_peered":"2024-05-28T14:06:25.421417+0000","last_unstale":"2024-05-28T14:06:26.812074+0000","last_undegraded":"2024-05-28T14:06:26.812074+0000","last_fullsized":"2024-05-28T14:06:26.812074+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:42:56.223388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.813869+0000","last_change":"2024-05-28T14:06:24.423413+0000","last_active":"2024-05-28T14:06:26.813869+0000","last_peered":"2024-05-28T14:06:26.813869+0000","last_clean":"2024-05-28T14:06:26.813869+0000","last_became_active":"2024-05-28T14:06:24.423140+0000","last_became_peered":"2024-05-28T14:06:24.423140+0000","last_unstale":"2024-05-28T14:06:26.813869+0000","last_undegraded":"2024-05-28T14:06:26.813869+0000","last_fullsized":"2024-05-28T14:06:26.813869+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:30.297697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428516+0000","last_change":"2024-05-28T14:06:25.426700+0000","last_active":"2024-05-28T14:06:26.428516+0000","last_peered":"2024-05-28T14:06:26.428516+0000","last_clean":"2024-05-28T14:06:26.428516+0000","last_became_active":"2024-05-28T14:06:25.426302+0000","last_became_peered":"2024-05-28T14:06:25.426302+0000","last_unstale":"2024-05-28T14:06:26.428516+0000","last_undegraded":"2024-05-28T14:06:26.428516+0000","last_fullsized":"2024-05-28T14:06:26.428516+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:42:08.956606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812445+0000","last_change":"2024-05-28T14:06:24.425685+0000","last_active":"2024-05-28T14:06:26.812445+0000","last_peered":"2024-05-28T14:06:26.812445+0000","last_clean":"2024-05-28T14:06:26.812445+0000","last_became_active":"2024-05-28T14:06:24.425387+0000","last_became_peered":"2024-05-28T14:06:24.425387+0000","last_unstale":"2024-05-28T14:06:26.812445+0000","last_undegraded":"2024-05-28T14:06:26.812445+0000","last_fullsized":"2024-05-28T14:06:26.812445+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:08:08.408635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.432175+0000","last_change":"2024-05-28T14:06:25.428573+0000","last_active":"2024-05-28T14:06:26.432175+0000","last_peered":"2024-05-28T14:06:26.432175+0000","last_clean":"2024-05-28T14:06:26.432175+0000","last_became_active":"2024-05-28T14:06:25.428209+0000","last_became_peered":"2024-05-28T14:06:25.428209+0000","last_unstale":"2024-05-28T14:06:26.432175+0000","last_undegraded":"2024-05-28T14:06:26.432175+0000","last_fullsized":"2024-05-28T14:06:26.432175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:44.471299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431661+0000","last_change":"2024-05-28T14:06:24.422382+0000","last_active":"2024-05-28T14:06:26.431661+0000","last_peered":"2024-05-28T14:06:26.431661+0000","last_clean":"2024-05-28T14:06:26.431661+0000","last_became_active":"2024-05-28T14:06:24.422093+0000","last_became_peered":"2024-05-28T14:06:24.422093+0000","last_unstale":"2024-05-28T14:06:26.431661+0000","last_undegraded":"2024-05-28T14:06:26.431661+0000","last_fullsized":"2024-05-28T14:06:26.431661+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:42:01.134570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815444+0000","last_change":"2024-05-28T14:06:25.444460+0000","last_active":"2024-05-28T14:06:26.815444+0000","last_peered":"2024-05-28T14:06:26.815444+0000","last_clean":"2024-05-28T14:06:26.815444+0000","last_became_active":"2024-05-28T14:06:25.443586+0000","last_became_peered":"2024-05-28T14:06:25.443586+0000","last_unstale":"2024-05-28T14:06:26.815444+0000","last_undegraded":"2024-05-28T14:06:26.815444+0000","last_fullsized":"2024-05-28T14:06:26.815444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:54:10.116238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816384+0000","last_change":"2024-05-28T14:06:24.436081+0000","last_active":"2024-05-28T14:06:26.816384+0000","last_peered":"2024-05-28T14:06:26.816384+0000","last_clean":"2024-05-28T14:06:26.816384+0000","last_became_active":"2024-05-28T14:06:24.435796+0000","last_became_peered":"2024-05-28T14:06:24.435796+0000","last_unstale":"2024-05-28T14:06:26.816384+0000","last_undegraded":"2024-05-28T14:06:26.816384+0000","last_fullsized":"2024-05-28T14:06:26.816384+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:31:55.554880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816342+0000","last_change":"2024-05-28T14:06:25.439274+0000","last_active":"2024-05-28T14:06:26.816342+0000","last_peered":"2024-05-28T14:06:26.816342+0000","last_clean":"2024-05-28T14:06:26.816342+0000","last_became_active":"2024-05-28T14:06:25.438618+0000","last_became_peered":"2024-05-28T14:06:25.438618+0000","last_unstale":"2024-05-28T14:06:26.816342+0000","last_undegraded":"2024-05-28T14:06:26.816342+0000","last_fullsized":"2024-05-28T14:06:26.816342+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:55:04.339742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814796+0000","last_change":"2024-05-28T14:06:24.421989+0000","last_active":"2024-05-28T14:06:26.814796+0000","last_peered":"2024-05-28T14:06:26.814796+0000","last_clean":"2024-05-28T14:06:26.814796+0000","last_became_active":"2024-05-28T14:06:24.421727+0000","last_became_peered":"2024-05-28T14:06:24.421727+0000","last_unstale":"2024-05-28T14:06:26.814796+0000","last_undegraded":"2024-05-28T14:06:26.814796+0000","last_fullsized":"2024-05-28T14:06:26.814796+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:00:55.162058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812268+0000","last_change":"2024-05-28T14:06:25.425553+0000","last_active":"2024-05-28T14:06:26.812268+0000","last_peered":"2024-05-28T14:06:26.812268+0000","last_clean":"2024-05-28T14:06:26.812268+0000","last_became_active":"2024-05-28T14:06:25.425258+0000","last_became_peered":"2024-05-28T14:06:25.425258+0000","last_unstale":"2024-05-28T14:06:26.812268+0000","last_undegraded":"2024-05-28T14:06:26.812268+0000","last_fullsized":"2024-05-28T14:06:26.812268+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:46:02.976297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814468+0000","last_change":"2024-05-28T14:06:24.421197+0000","last_active":"2024-05-28T14:06:26.814468+0000","last_peered":"2024-05-28T14:06:26.814468+0000","last_clean":"2024-05-28T14:06:26.814468+0000","last_became_active":"2024-05-28T14:06:24.420925+0000","last_became_peered":"2024-05-28T14:06:24.420925+0000","last_unstale":"2024-05-28T14:06:26.814468+0000","last_undegraded":"2024-05-28T14:06:26.814468+0000","last_fullsized":"2024-05-28T14:06:26.814468+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:58:13.946282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814794+0000","last_change":"2024-05-28T14:06:25.438975+0000","last_active":"2024-05-28T14:06:26.814794+0000","last_peered":"2024-05-28T14:06:26.814794+0000","last_clean":"2024-05-28T14:06:26.814794+0000","last_became_active":"2024-05-28T14:06:25.437155+0000","last_became_peered":"2024-05-28T14:06:25.437155+0000","last_unstale":"2024-05-28T14:06:26.814794+0000","last_undegraded":"2024-05-28T14:06:26.814794+0000","last_fullsized":"2024-05-28T14:06:26.814794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:02:24.821686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814010+0000","last_change":"2024-05-28T14:06:24.429639+0000","last_active":"2024-05-28T14:06:26.814010+0000","last_peered":"2024-05-28T14:06:26.814010+0000","last_clean":"2024-05-28T14:06:26.814010+0000","last_became_active":"2024-05-28T14:06:24.429133+0000","last_became_peered":"2024-05-28T14:06:24.429133+0000","last_unstale":"2024-05-28T14:06:26.814010+0000","last_undegraded":"2024-05-28T14:06:26.814010+0000","last_fullsized":"2024-05-28T14:06:26.814010+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:22:45.391365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431983+0000","last_change":"2024-05-28T14:06:25.422321+0000","last_active":"2024-05-28T14:06:26.431983+0000","last_peered":"2024-05-28T14:06:26.431983+0000","last_clean":"2024-05-28T14:06:26.431983+0000","last_became_active":"2024-05-28T14:06:25.422020+0000","last_became_peered":"2024-05-28T14:06:25.422020+0000","last_unstale":"2024-05-28T14:06:26.431983+0000","last_undegraded":"2024-05-28T14:06:26.431983+0000","last_fullsized":"2024-05-28T14:06:26.431983+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:42:08.956606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815957+0000","last_change":"2024-05-28T14:06:24.432341+0000","last_active":"2024-05-28T14:06:26.815957+0000","last_peered":"2024-05-28T14:06:26.815957+0000","last_clean":"2024-05-28T14:06:26.815957+0000","last_became_active":"2024-05-28T14:06:24.432058+0000","last_became_peered":"2024-05-28T14:06:24.432058+0000","last_unstale":"2024-05-28T14:06:26.815957+0000","last_undegraded":"2024-05-28T14:06:26.815957+0000","last_fullsized":"2024-05-28T14:06:26.815957+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:30.297697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428551+0000","last_change":"2024-05-28T14:06:25.433584+0000","last_active":"2024-05-28T14:06:26.428551+0000","last_peered":"2024-05-28T14:06:26.428551+0000","last_clean":"2024-05-28T14:06:26.428551+0000","last_became_active":"2024-05-28T14:06:25.432726+0000","last_became_peered":"2024-05-28T14:06:25.432726+0000","last_unstale":"2024-05-28T14:06:26.428551+0000","last_undegraded":"2024-05-28T14:06:26.428551+0000","last_fullsized":"2024-05-28T14:06:26.428551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:10:33.978239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.430645+0000","last_change":"2024-05-28T14:06:24.426720+0000","last_active":"2024-05-28T14:06:26.430645+0000","last_peered":"2024-05-28T14:06:26.430645+0000","last_clean":"2024-05-28T14:06:26.430645+0000","last_became_active":"2024-05-28T14:06:24.426399+0000","last_became_peered":"2024-05-28T14:06:24.426399+0000","last_unstale":"2024-05-28T14:06:26.430645+0000","last_undegraded":"2024-05-28T14:06:26.430645+0000","last_fullsized":"2024-05-28T14:06:26.430645+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:55:03.333699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812310+0000","last_change":"2024-05-28T14:06:25.440554+0000","last_active":"2024-05-28T14:06:26.812310+0000","last_peered":"2024-05-28T14:06:26.812310+0000","last_clean":"2024-05-28T14:06:26.812310+0000","last_became_active":"2024-05-28T14:06:25.440273+0000","last_became_peered":"2024-05-28T14:06:25.440273+0000","last_unstale":"2024-05-28T14:06:26.812310+0000","last_undegraded":"2024-05-28T14:06:26.812310+0000","last_fullsized":"2024-05-28T14:06:26.812310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:37:37.078325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431020+0000","last_change":"2024-05-28T14:06:24.424772+0000","last_active":"2024-05-28T14:06:26.431020+0000","last_peered":"2024-05-28T14:06:26.431020+0000","last_clean":"2024-05-28T14:06:26.431020+0000","last_became_active":"2024-05-28T14:06:24.424480+0000","last_became_peered":"2024-05-28T14:06:24.424480+0000","last_unstale":"2024-05-28T14:06:26.431020+0000","last_undegraded":"2024-05-28T14:06:26.431020+0000","last_fullsized":"2024-05-28T14:06:26.431020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:34:49.329236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431827+0000","last_change":"2024-05-28T14:06:25.434340+0000","last_active":"2024-05-28T14:06:26.431827+0000","last_peered":"2024-05-28T14:06:26.431827+0000","last_clean":"2024-05-28T14:06:26.431827+0000","last_became_active":"2024-05-28T14:06:25.434049+0000","last_became_peered":"2024-05-28T14:06:25.434049+0000","last_unstale":"2024-05-28T14:06:26.431827+0000","last_undegraded":"2024-05-28T14:06:26.431827+0000","last_fullsized":"2024-05-28T14:06:26.431827+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:34:48.270645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816490+0000","last_change":"2024-05-28T14:06:24.434783+0000","last_active":"2024-05-28T14:06:26.816490+0000","last_peered":"2024-05-28T14:06:26.816490+0000","last_clean":"2024-05-28T14:06:26.816490+0000","last_became_active":"2024-05-28T14:06:24.434490+0000","last_became_peered":"2024-05-28T14:06:24.434490+0000","last_unstale":"2024-05-28T14:06:26.816490+0000","last_undegraded":"2024-05-28T14:06:26.816490+0000","last_fullsized":"2024-05-28T14:06:26.816490+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:34:49.329236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815517+0000","last_change":"2024-05-28T14:06:25.440884+0000","last_active":"2024-05-28T14:06:26.815517+0000","last_peered":"2024-05-28T14:06:26.815517+0000","last_clean":"2024-05-28T14:06:26.815517+0000","last_became_active":"2024-05-28T14:06:25.440471+0000","last_became_peered":"2024-05-28T14:06:25.440471+0000","last_unstale":"2024-05-28T14:06:26.815517+0000","last_undegraded":"2024-05-28T14:06:26.815517+0000","last_fullsized":"2024-05-28T14:06:26.815517+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:22:01.027740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431898+0000","last_change":"2024-05-28T14:06:24.434447+0000","last_active":"2024-05-28T14:06:26.431898+0000","last_peered":"2024-05-28T14:06:26.431898+0000","last_clean":"2024-05-28T14:06:26.431898+0000","last_became_active":"2024-05-28T14:06:24.433002+0000","last_became_peered":"2024-05-28T14:06:24.433002+0000","last_unstale":"2024-05-28T14:06:26.431898+0000","last_undegraded":"2024-05-28T14:06:26.431898+0000","last_fullsized":"2024-05-28T14:06:26.431898+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:34:22.753272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814461+0000","last_change":"2024-05-28T14:06:25.423337+0000","last_active":"2024-05-28T14:06:26.814461+0000","last_peered":"2024-05-28T14:06:26.814461+0000","last_clean":"2024-05-28T14:06:26.814461+0000","last_became_active":"2024-05-28T14:06:25.422687+0000","last_became_peered":"2024-05-28T14:06:25.422687+0000","last_unstale":"2024-05-28T14:06:26.814461+0000","last_undegraded":"2024-05-28T14:06:26.814461+0000","last_fullsized":"2024-05-28T14:06:26.814461+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:52:13.920211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816190+0000","last_change":"2024-05-28T14:06:24.428701+0000","last_active":"2024-05-28T14:06:26.816190+0000","last_peered":"2024-05-28T14:06:26.816190+0000","last_clean":"2024-05-28T14:06:26.816190+0000","last_became_active":"2024-05-28T14:06:24.428477+0000","last_became_peered":"2024-05-28T14:06:24.428477+0000","last_unstale":"2024-05-28T14:06:26.816190+0000","last_undegraded":"2024-05-28T14:06:26.816190+0000","last_fullsized":"2024-05-28T14:06:26.816190+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:54:28.504352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812539+0000","last_change":"2024-05-28T14:06:25.425033+0000","last_active":"2024-05-28T14:06:26.812539+0000","last_peered":"2024-05-28T14:06:26.812539+0000","last_clean":"2024-05-28T14:06:26.812539+0000","last_became_active":"2024-05-28T14:06:25.424735+0000","last_became_peered":"2024-05-28T14:06:25.424735+0000","last_unstale":"2024-05-28T14:06:26.812539+0000","last_undegraded":"2024-05-28T14:06:26.812539+0000","last_fullsized":"2024-05-28T14:06:26.812539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:22:46.397408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815605+0000","last_change":"2024-05-28T14:06:24.437928+0000","last_active":"2024-05-28T14:06:26.815605+0000","last_peered":"2024-05-28T14:06:26.815605+0000","last_clean":"2024-05-28T14:06:26.815605+0000","last_became_active":"2024-05-28T14:06:24.437366+0000","last_became_peered":"2024-05-28T14:06:24.437366+0000","last_unstale":"2024-05-28T14:06:26.815605+0000","last_undegraded":"2024-05-28T14:06:26.815605+0000","last_fullsized":"2024-05-28T14:06:26.815605+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:20:05.724578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814078+0000","last_change":"2024-05-28T14:06:25.432485+0000","last_active":"2024-05-28T14:06:26.814078+0000","last_peered":"2024-05-28T14:06:26.814078+0000","last_clean":"2024-05-28T14:06:26.814078+0000","last_became_active":"2024-05-28T14:06:25.432222+0000","last_became_peered":"2024-05-28T14:06:25.432222+0000","last_unstale":"2024-05-28T14:06:26.814078+0000","last_undegraded":"2024-05-28T14:06:26.814078+0000","last_fullsized":"2024-05-28T14:06:26.814078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:18:13.308854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815959+0000","last_change":"2024-05-28T14:06:24.429400+0000","last_active":"2024-05-28T14:06:26.815959+0000","last_peered":"2024-05-28T14:06:26.815959+0000","last_clean":"2024-05-28T14:06:26.815959+0000","last_became_active":"2024-05-28T14:06:24.429124+0000","last_became_peered":"2024-05-28T14:06:24.429124+0000","last_unstale":"2024-05-28T14:06:26.815959+0000","last_undegraded":"2024-05-28T14:06:26.815959+0000","last_fullsized":"2024-05-28T14:06:26.815959+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:22:45.391365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.813660+0000","last_change":"2024-05-28T14:06:25.436652+0000","last_active":"2024-05-28T14:06:26.813660+0000","last_peered":"2024-05-28T14:06:26.813660+0000","last_clean":"2024-05-28T14:06:26.813660+0000","last_became_active":"2024-05-28T14:06:25.436330+0000","last_became_peered":"2024-05-28T14:06:25.436330+0000","last_unstale":"2024-05-28T14:06:26.813660+0000","last_undegraded":"2024-05-28T14:06:26.813660+0000","last_fullsized":"2024-05-28T14:06:26.813660+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:14:20.201888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431315+0000","last_change":"2024-05-28T14:06:24.420613+0000","last_active":"2024-05-28T14:06:26.431315+0000","last_peered":"2024-05-28T14:06:26.431315+0000","last_clean":"2024-05-28T14:06:26.431315+0000","last_became_active":"2024-05-28T14:06:24.420171+0000","last_became_peered":"2024-05-28T14:06:24.420171+0000","last_unstale":"2024-05-28T14:06:26.431315+0000","last_undegraded":"2024-05-28T14:06:26.431315+0000","last_fullsized":"2024-05-28T14:06:26.431315+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:06:23.658146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.813678+0000","last_change":"2024-05-28T14:06:25.436950+0000","last_active":"2024-05-28T14:06:26.813678+0000","last_peered":"2024-05-28T14:06:26.813678+0000","last_clean":"2024-05-28T14:06:26.813678+0000","last_became_active":"2024-05-28T14:06:25.436591+0000","last_became_peered":"2024-05-28T14:06:25.436591+0000","last_unstale":"2024-05-28T14:06:26.813678+0000","last_undegraded":"2024-05-28T14:06:26.813678+0000","last_fullsized":"2024-05-28T14:06:26.813678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:42:29.665913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428218+0000","last_change":"2024-05-28T14:06:24.427464+0000","last_active":"2024-05-28T14:06:26.428218+0000","last_peered":"2024-05-28T14:06:26.428218+0000","last_clean":"2024-05-28T14:06:26.428218+0000","last_became_active":"2024-05-28T14:06:24.427145+0000","last_became_peered":"2024-05-28T14:06:24.427145+0000","last_unstale":"2024-05-28T14:06:26.428218+0000","last_undegraded":"2024-05-28T14:06:26.428218+0000","last_fullsized":"2024-05-28T14:06:26.428218+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:58:08.424233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428742+0000","last_change":"2024-05-28T14:06:25.423080+0000","last_active":"2024-05-28T14:06:26.428742+0000","last_peered":"2024-05-28T14:06:26.428742+0000","last_clean":"2024-05-28T14:06:26.428742+0000","last_became_active":"2024-05-28T14:06:25.422305+0000","last_became_peered":"2024-05-28T14:06:25.422305+0000","last_unstale":"2024-05-28T14:06:26.428742+0000","last_undegraded":"2024-05-28T14:06:26.428742+0000","last_fullsized":"2024-05-28T14:06:26.428742+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:44.471299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.816497+0000","last_change":"2024-05-28T14:06:24.432369+0000","last_active":"2024-05-28T14:06:26.816497+0000","last_peered":"2024-05-28T14:06:26.816497+0000","last_clean":"2024-05-28T14:06:26.816497+0000","last_became_active":"2024-05-28T14:06:24.431992+0000","last_became_peered":"2024-05-28T14:06:24.431992+0000","last_unstale":"2024-05-28T14:06:26.816497+0000","last_undegraded":"2024-05-28T14:06:26.816497+0000","last_fullsized":"2024-05-28T14:06:26.816497+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:42:01.134570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.431538+0000","last_change":"2024-05-28T14:06:25.427193+0000","last_active":"2024-05-28T14:06:26.431538+0000","last_peered":"2024-05-28T14:06:26.431538+0000","last_clean":"2024-05-28T14:06:26.431538+0000","last_became_active":"2024-05-28T14:06:25.426889+0000","last_became_peered":"2024-05-28T14:06:25.426889+0000","last_unstale":"2024-05-28T14:06:26.431538+0000","last_undegraded":"2024-05-28T14:06:26.431538+0000","last_fullsized":"2024-05-28T14:06:26.431538+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:42:08.042876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.432269+0000","last_change":"2024-05-28T14:06:24.427952+0000","last_active":"2024-05-28T14:06:26.432269+0000","last_peered":"2024-05-28T14:06:26.432269+0000","last_clean":"2024-05-28T14:06:26.432269+0000","last_became_active":"2024-05-28T14:06:24.427511+0000","last_became_peered":"2024-05-28T14:06:24.427511+0000","last_unstale":"2024-05-28T14:06:26.432269+0000","last_undegraded":"2024-05-28T14:06:26.432269+0000","last_fullsized":"2024-05-28T14:06:26.432269+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:34:47.264602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815624+0000","last_change":"2024-05-28T14:06:25.445031+0000","last_active":"2024-05-28T14:06:26.815624+0000","last_peered":"2024-05-28T14:06:26.815624+0000","last_clean":"2024-05-28T14:06:26.815624+0000","last_became_active":"2024-05-28T14:06:25.443895+0000","last_became_peered":"2024-05-28T14:06:25.443895+0000","last_unstale":"2024-05-28T14:06:26.815624+0000","last_undegraded":"2024-05-28T14:06:26.815624+0000","last_fullsized":"2024-05-28T14:06:26.815624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:32:49.349214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812576+0000","last_change":"2024-05-28T14:06:24.427264+0000","last_active":"2024-05-28T14:06:26.812576+0000","last_peered":"2024-05-28T14:06:26.812576+0000","last_clean":"2024-05-28T14:06:26.812576+0000","last_became_active":"2024-05-28T14:06:24.426662+0000","last_became_peered":"2024-05-28T14:06:24.426662+0000","last_unstale":"2024-05-28T14:06:26.812576+0000","last_undegraded":"2024-05-28T14:06:26.812576+0000","last_fullsized":"2024-05-28T14:06:26.812576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:53:56.024111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815464+0000","last_change":"2024-05-28T14:06:25.439712+0000","last_active":"2024-05-28T14:06:26.815464+0000","last_peered":"2024-05-28T14:06:26.815464+0000","last_clean":"2024-05-28T14:06:26.815464+0000","last_became_active":"2024-05-28T14:06:25.438826+0000","last_became_peered":"2024-05-28T14:06:25.438826+0000","last_unstale":"2024-05-28T14:06:26.815464+0000","last_undegraded":"2024-05-28T14:06:26.815464+0000","last_fullsized":"2024-05-28T14:06:26.815464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:50:50.831055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.428504+0000","last_change":"2024-05-28T14:06:24.431741+0000","last_active":"2024-05-28T14:06:26.428504+0000","last_peered":"2024-05-28T14:06:26.428504+0000","last_clean":"2024-05-28T14:06:26.428504+0000","last_became_active":"2024-05-28T14:06:24.431388+0000","last_became_peered":"2024-05-28T14:06:24.431388+0000","last_unstale":"2024-05-28T14:06:26.428504+0000","last_undegraded":"2024-05-28T14:06:26.428504+0000","last_fullsized":"2024-05-28T14:06:26.428504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:27:28.253532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812448+0000","last_change":"2024-05-28T14:06:25.433206+0000","last_active":"2024-05-28T14:06:26.812448+0000","last_peered":"2024-05-28T14:06:26.812448+0000","last_clean":"2024-05-28T14:06:26.812448+0000","last_became_active":"2024-05-28T14:06:25.432909+0000","last_became_peered":"2024-05-28T14:06:25.432909+0000","last_unstale":"2024-05-28T14:06:26.812448+0000","last_undegraded":"2024-05-28T14:06:26.812448+0000","last_fullsized":"2024-05-28T14:06:26.812448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:27.322317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.812351+0000","last_change":"2024-05-28T14:06:24.420374+0000","last_active":"2024-05-28T14:06:26.812351+0000","last_peered":"2024-05-28T14:06:26.812351+0000","last_clean":"2024-05-28T14:06:26.812351+0000","last_became_active":"2024-05-28T14:06:24.420044+0000","last_became_peered":"2024-05-28T14:06:24.420044+0000","last_unstale":"2024-05-28T14:06:26.812351+0000","last_undegraded":"2024-05-28T14:06:26.812351+0000","last_fullsized":"2024-05-28T14:06:26.812351+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:12:23.194270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815942+0000","last_change":"2024-05-28T14:06:25.442435+0000","last_active":"2024-05-28T14:06:26.815942+0000","last_peered":"2024-05-28T14:06:26.815942+0000","last_clean":"2024-05-28T14:06:26.815942+0000","last_became_active":"2024-05-28T14:06:25.442182+0000","last_became_peered":"2024-05-28T14:06:25.442182+0000","last_unstale":"2024-05-28T14:06:26.815942+0000","last_undegraded":"2024-05-28T14:06:26.815942+0000","last_fullsized":"2024-05-28T14:06:26.815942+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:27:03.768824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.813888+0000","last_change":"2024-05-28T14:06:24.422804+0000","last_active":"2024-05-28T14:06:26.813888+0000","last_peered":"2024-05-28T14:06:26.813888+0000","last_clean":"2024-05-28T14:06:26.813888+0000","last_became_active":"2024-05-28T14:06:24.422468+0000","last_became_peered":"2024-05-28T14:06:24.422468+0000","last_unstale":"2024-05-28T14:06:26.813888+0000","last_undegraded":"2024-05-28T14:06:26.813888+0000","last_fullsized":"2024-05-28T14:06:26.813888+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:53:17.596549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.813960+0000","last_change":"2024-05-28T14:06:25.428347+0000","last_active":"2024-05-28T14:06:26.813960+0000","last_peered":"2024-05-28T14:06:26.813960+0000","last_clean":"2024-05-28T14:06:26.813960+0000","last_became_active":"2024-05-28T14:06:25.428035+0000","last_became_peered":"2024-05-28T14:06:25.428035+0000","last_unstale":"2024-05-28T14:06:26.813960+0000","last_undegraded":"2024-05-28T14:06:26.813960+0000","last_fullsized":"2024-05-28T14:06:26.813960+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:54:29.510395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.430867+0000","last_change":"2024-05-28T14:06:24.429508+0000","last_active":"2024-05-28T14:06:26.430867+0000","last_peered":"2024-05-28T14:06:26.430867+0000","last_clean":"2024-05-28T14:06:26.430867+0000","last_became_active":"2024-05-28T14:06:24.429101+0000","last_became_peered":"2024-05-28T14:06:24.429101+0000","last_unstale":"2024-05-28T14:06:26.430867+0000","last_undegraded":"2024-05-28T14:06:26.430867+0000","last_fullsized":"2024-05-28T14:06:26.430867+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:28:49.528879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815702+0000","last_change":"2024-05-28T14:06:25.444941+0000","last_active":"2024-05-28T14:06:26.815702+0000","last_peered":"2024-05-28T14:06:26.815702+0000","last_clean":"2024-05-28T14:06:26.815702+0000","last_became_active":"2024-05-28T14:06:25.444505+0000","last_became_peered":"2024-05-28T14:06:25.444505+0000","last_unstale":"2024-05-28T14:06:26.815702+0000","last_undegraded":"2024-05-28T14:06:26.815702+0000","last_fullsized":"2024-05-28T14:06:26.815702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:17:03.474414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815343+0000","last_change":"2024-05-28T14:06:24.427387+0000","last_active":"2024-05-28T14:06:26.815343+0000","last_peered":"2024-05-28T14:06:26.815343+0000","last_clean":"2024-05-28T14:06:26.815343+0000","last_became_active":"2024-05-28T14:06:24.427106+0000","last_became_peered":"2024-05-28T14:06:24.427106+0000","last_unstale":"2024-05-28T14:06:26.815343+0000","last_undegraded":"2024-05-28T14:06:26.815343+0000","last_fullsized":"2024-05-28T14:06:26.815343+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:58:08.159980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.813779+0000","last_change":"2024-05-28T14:06:25.431355+0000","last_active":"2024-05-28T14:06:26.813779+0000","last_peered":"2024-05-28T14:06:26.813779+0000","last_clean":"2024-05-28T14:06:26.813779+0000","last_became_active":"2024-05-28T14:06:25.430927+0000","last_became_peered":"2024-05-28T14:06:25.430927+0000","last_unstale":"2024-05-28T14:06:26.813779+0000","last_undegraded":"2024-05-28T14:06:26.813779+0000","last_fullsized":"2024-05-28T14:06:26.813779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:24.399936+0000","last_clean_scrub_stamp":"2024-05-28T14:06:24.399936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:22:05.399179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.815957+0000","last_change":"2024-05-28T14:06:24.433815+0000","last_active":"2024-05-28T14:06:26.815957+0000","last_peered":"2024-05-28T14:06:26.815957+0000","last_clean":"2024-05-28T14:06:26.815957+0000","last_became_active":"2024-05-28T14:06:24.433524+0000","last_became_peered":"2024-05-28T14:06:24.433524+0000","last_unstale":"2024-05-28T14:06:26.815957+0000","last_undegraded":"2024-05-28T14:06:26.815957+0000","last_fullsized":"2024-05-28T14:06:26.815957+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:06:23.393893+0000","last_clean_scrub_stamp":"2024-05-28T14:06:23.393893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:58:13.946282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":268,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-28T14:06:26.814838+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:06:26.814838+0000","last_peered":"2024-05-28T14:06:26.814838+0000","last_clean":"2024-05-28T14:06:26.814838+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:06:26.814838+0000","last_undegraded":"2024-05-28T14:06:26.814838+0000","last_fullsized":"2024-05-28T14:06:26.814838+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59016,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312824,"statfs":{"total":96636764160,"available":96576331776,"internally_reserved":0,"allocated":1503232,"data_stored":975671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574950,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54280,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317560,"statfs":{"total":96636764160,"available":96581181440,"internally_reserved":0,"allocated":909312,"data_stored":384779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54276,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317564,"statfs":{"total":96636764160,"available":96581185536,"internally_reserved":0,"allocated":905216,"data_stored":384777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58432,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313408,"statfs":{"total":96636764160,"available":96576929792,"internally_reserved":0,"allocated":905216,"data_stored":384757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574950,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45956,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325884,"statfs":{"total":96636764160,"available":96589705216,"internally_reserved":0,"allocated":909312,"data_stored":384799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574951,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54284,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317556,"statfs":{"total":96636764160,"available":96581177344,"internally_reserved":0,"allocated":913408,"data_stored":386299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62616,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309224,"statfs":{"total":96636764160,"available":96572645376,"internally_reserved":0,"allocated":921600,"data_stored":386915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46532,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325308,"statfs":{"total":96636764160,"available":96589115392,"internally_reserved":0,"allocated":1495040,"data_stored":975055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:06:34.481 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-28T14:06:34.481 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-28T14:06:34.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.682+0000 7f1fbfe00640 1 Processor -- start 2024-05-28T14:06:34.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.682+0000 7f1fbfe00640 1 -- start start 2024-05-28T14:06:34.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.686+0000 7f1fbfe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1fc015a160 0x7f1fc0071730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:34.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.686+0000 7f1fbfe00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fc0071c70 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.686+0000 7f1fbfe00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fc015cad0 con 0x7f1fc00cc920 2024-05-28T14:06:34.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.686+0000 7f1fbfe00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1fc015a540 con 0x7f1fc015a160 2024-05-28T14:06:34.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.686+0000 7f1fb7e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1fc00d1ab0 0x7f1fc015d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42846/0 (socket says 172.21.15.42:42846) 2024-05-28T14:06:34.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.686+0000 7f1fb7e00640 1 -- 172.21.15.42:0/698872229 learned_addr learned my addr 172.21.15.42:0/698872229 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:34.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.686+0000 7f1fbea00640 1 --2- 172.21.15.42:0/698872229 >> v2:172.21.15.63:3300/0 conn(0x7f1fc015a160 0x7f1fc0071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:34.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.686+0000 7f1fbe000640 1 -- 172.21.15.42:0/698872229 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1099470594 0 0) 0x7f1fc0071c70 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.686+0000 7f1fbe000640 1 -- 172.21.15.42:0/698872229 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fa4003530 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.686+0000 7f1fbe000640 1 -- 172.21.15.42:0/698872229 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 738801611 0 0) 0x7f1fa4003530 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.686+0000 7f1fbea00640 1 -- 172.21.15.42:0/698872229 >> v1:172.21.15.63:6789/0 conn(0x7f1fc00cc920 legacy=0x7f1fc01600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:34.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.686+0000 7f1fbea00640 1 -- 172.21.15.42:0/698872229 >> v1:172.21.15.42:6789/0 conn(0x7f1fc00d1ab0 legacy=0x7f1fc015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:34.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.686+0000 7f1fbea00640 1 -- 172.21.15.42:0/698872229 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc015d9c0 con 0x7f1fc015a160 2024-05-28T14:06:34.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.690+0000 7f1fbea00640 1 --2- 172.21.15.42:0/698872229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fc015a160 0x7f1fc0071730 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1fc015a6c0 tx=0x7f1fb001c2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f9b1d4b82d4be0b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:34.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.690+0000 7f1fbe000640 1 -- 172.21.15.42:0/698872229 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb00b0070 con 0x7f1fc015a160 2024-05-28T14:06:34.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.690+0000 7f1fbe000640 1 -- 172.21.15.42:0/698872229 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb00ab400 con 0x7f1fc015a160 2024-05-28T14:06:34.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.690+0000 7f1fbe000640 1 -- 172.21.15.42:0/698872229 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8001680 con 0x7f1fc015a160 2024-05-28T14:06:34.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.690+0000 7f1fbfe00640 1 -- 172.21.15.42:0/698872229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fc015a160 msgr2=0x7f1fc0071730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:34.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.690+0000 7f1fbfe00640 1 --2- 172.21.15.42:0/698872229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fc015a160 0x7f1fc0071730 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1fc015a6c0 tx=0x7f1fb001c2a0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.690+0000 7f1fbfe00640 1 -- 172.21.15.42:0/698872229 shutdown_connections 2024-05-28T14:06:34.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.690+0000 7f1fbfe00640 1 --2- 172.21.15.42:0/698872229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fc015a160 0x7f1fc0071730 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.690+0000 7f1fbfe00640 1 -- 172.21.15.42:0/698872229 >> 172.21.15.42:0/698872229 conn(0x7f1fc00b3ec0 msgr2=0x7f1fc00b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:34.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.690+0000 7f1fbfe00640 1 -- 172.21.15.42:0/698872229 shutdown_connections 2024-05-28T14:06:34.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.690+0000 7f1fbfe00640 1 -- 172.21.15.42:0/698872229 wait complete. 2024-05-28T14:06:34.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.690+0000 7f1fbfe00640 1 Processor -- start 2024-05-28T14:06:34.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.690+0000 7f1fbfe00640 1 -- start start 2024-05-28T14:06:34.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fbfe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1fc00cc920 0x7f1fc0159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:34.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fbfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fc00d1ab0 0x7f1fc006cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:34.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fbfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fc015a160 0x7f1fc0150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:34.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fbfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc00cd140 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fbfe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc016ac80 con 0x7f1fc015a160 2024-05-28T14:06:34.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fbfe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc0150d80 con 0x7f1fc00cc920 2024-05-28T14:06:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fc00d1ab0 0x7f1fc006cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fc00d1ab0 0x7f1fc006cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53202/0 (socket says 172.21.15.42:53202) 2024-05-28T14:06:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fb7e00640 1 -- 172.21.15.42:0/622069260 learned_addr learned my addr 172.21.15.42:0/622069260 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fbea00640 1 --2- 172.21.15.42:0/622069260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1fc00cc920 0x7f1fc0159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fbf400640 1 --2- 172.21.15.42:0/622069260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fc015a160 0x7f1fc0150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fb7e00640 1 -- 172.21.15.42:0/622069260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1fc00cc920 msgr2=0x7f1fc0159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fb7e00640 1 --2- 172.21.15.42:0/622069260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1fc00cc920 0x7f1fc0159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fb7e00640 1 -- 172.21.15.42:0/622069260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fc015a160 msgr2=0x7f1fc0150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fb7e00640 1 --2- 172.21.15.42:0/622069260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fc015a160 0x7f1fc0150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fb7e00640 1 -- 172.21.15.42:0/622069260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc006d250 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fb7e00640 1 --2- 172.21.15.42:0/622069260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fc00d1ab0 0x7f1fc006cb00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800c530 tx=0x7f1fa800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:34.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fb6a00640 1 -- 172.21.15.42:0/622069260 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8017070 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fb6a00640 1 -- 172.21.15.42:0/622069260 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa8005cf0 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fb6a00640 1 -- 172.21.15.42:0/622069260 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80072b0 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fbfe00640 1 -- 172.21.15.42:0/622069260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc006d530 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fbfe00640 1 -- 172.21.15.42:0/622069260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc01523c0 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.694+0000 7f1fbfe00640 1 -- 172.21.15.42:0/622069260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc004fcf0 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.698+0000 7f1fb6a00640 1 -- 172.21.15.42:0/622069260 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f1fa801a070 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.698+0000 7f1fb6a00640 1 --2- 172.21.15.42:0/622069260 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1f9c069070 0x7f1f9c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.698+0000 7f1fb6a00640 1 -- 172.21.15.42:0/622069260 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7f1fa8084870 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.698+0000 7f1fbea00640 1 --2- 172.21.15.42:0/622069260 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1f9c069070 0x7f1f9c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.698+0000 7f1fbea00640 1 --2- 172.21.15.42:0/622069260 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1f9c069070 0x7f1f9c06b530 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00ad000 tx=0x7f1fb0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:34.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.702+0000 7f1fb6a00640 1 -- 172.21.15.42:0/622069260 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1fa8056170 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.922+0000 7f1fbfe00640 1 -- 172.21.15.42:0/622069260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1fc0071050 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.926+0000 7f1fb6a00640 1 -- 172.21.15.42:0/622069260 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f1fa805a470 con 0x7f1fc00d1ab0 2024-05-28T14:06:34.912 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-05-28T14:06:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw== 2024-05-28T14:06:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow" 2024-05-28T14:06:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-05-28T14:06:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-28T14:06:34.915 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.0 2024-05-28T14:06:34.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.926+0000 7f1fbfe00640 1 -- 172.21.15.42:0/622069260 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1f9c069070 msgr2=0x7f1f9c06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:34.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.926+0000 7f1fbfe00640 1 --2- 172.21.15.42:0/622069260 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1f9c069070 0x7f1f9c06b530 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00ad000 tx=0x7f1fb0008040 comp rx=0 tx=0).stop 2024-05-28T14:06:34.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.926+0000 7f1fbfe00640 1 -- 172.21.15.42:0/622069260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fc00d1ab0 msgr2=0x7f1fc006cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:34.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.926+0000 7f1fbfe00640 1 --2- 172.21.15.42:0/622069260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fc00d1ab0 0x7f1fc006cb00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800c530 tx=0x7f1fa800c9f0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.926+0000 7f1fbfe00640 1 -- 172.21.15.42:0/622069260 shutdown_connections 2024-05-28T14:06:34.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.926+0000 7f1fbfe00640 1 --2- 172.21.15.42:0/622069260 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1f9c069070 0x7f1f9c06b530 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.926+0000 7f1fbfe00640 1 --2- 172.21.15.42:0/622069260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fc015a160 0x7f1fc0150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.926+0000 7f1fbfe00640 1 --2- 172.21.15.42:0/622069260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fc00d1ab0 0x7f1fc006cb00 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.926+0000 7f1fbfe00640 1 --2- 172.21.15.42:0/622069260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1fc00cc920 0x7f1fc0159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:34.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.926+0000 7f1fbfe00640 1 -- 172.21.15.42:0/622069260 >> 172.21.15.42:0/622069260 conn(0x7f1fc00b3ec0 msgr2=0x7f1fc015bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:34.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.926+0000 7f1fbfe00640 1 -- 172.21.15.42:0/622069260 shutdown_connections 2024-05-28T14:06:34.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:34.926+0000 7f1fbfe00640 1 -- 172.21.15.42:0/622069260 wait complete. 2024-05-28T14:06:34.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:06:35.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f733ea00640 1 Processor -- start 2024-05-28T14:06:35.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f733ea00640 1 -- start start 2024-05-28T14:06:35.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f733ea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f733815ea80 0x7f73380d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f733ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f73380d1820 0x7f733809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f733ea00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73380ccd00 con 0x7f733809af90 2024-05-28T14:06:35.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f733ea00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f733815ee60 con 0x7f73380d1820 2024-05-28T14:06:35.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f733ea00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f73380ccfe0 con 0x7f733815ea80 2024-05-28T14:06:35.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f732fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f73380d1820 0x7f733809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f733ca00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f733809af90 0x7f733806cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42860/0 (socket says 172.21.15.42:42860) 2024-05-28T14:06:35.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f733ca00640 1 -- 172.21.15.42:0/4069511202 learned_addr learned my addr 172.21.15.42:0/4069511202 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:35.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f7337e00640 1 --2- 172.21.15.42:0/4069511202 >> v2:172.21.15.63:3300/0 conn(0x7f733815ea80 0x7f73380d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f732fe00640 1 -- 172.21.15.42:0/4069511202 >> v2:172.21.15.63:3300/0 conn(0x7f733815ea80 msgr2=0x7f73380d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:35.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f732fe00640 1 --2- 172.21.15.42:0/4069511202 >> v2:172.21.15.63:3300/0 conn(0x7f733815ea80 0x7f73380d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f732fe00640 1 -- 172.21.15.42:0/4069511202 >> v1:172.21.15.42:6789/0 conn(0x7f733809af90 legacy=0x7f733806cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f7337400640 1 -- 172.21.15.42:0/4069511202 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1684471534 0 0) 0x7f73380ccd00 con 0x7f733809af90 2024-05-28T14:06:35.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.090+0000 7f732fe00640 1 -- 172.21.15.42:0/4069511202 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73380cd2c0 con 0x7f73380d1820 2024-05-28T14:06:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.094+0000 7f732fe00640 1 --2- 172.21.15.42:0/4069511202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73380d1820 0x7f733809aa50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f73380b4d20 tx=0x7f732001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211149f2e7d71cd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.094+0000 7f7337400640 1 -- 172.21.15.42:0/4069511202 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200b0070 con 0x7f73380d1820 2024-05-28T14:06:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.094+0000 7f7337400640 1 -- 172.21.15.42:0/4069511202 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73200ab600 con 0x7f73380d1820 2024-05-28T14:06:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.094+0000 7f7337400640 1 -- 172.21.15.42:0/4069511202 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200ab9d0 con 0x7f73380d1820 2024-05-28T14:06:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.094+0000 7f733ea00640 1 -- 172.21.15.42:0/4069511202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73380d1820 msgr2=0x7f733809aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.094+0000 7f733ea00640 1 --2- 172.21.15.42:0/4069511202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73380d1820 0x7f733809aa50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f73380b4d20 tx=0x7f732001c480 comp rx=0 tx=0).stop 2024-05-28T14:06:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.094+0000 7f733ea00640 1 -- 172.21.15.42:0/4069511202 shutdown_connections 2024-05-28T14:06:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.094+0000 7f733ea00640 1 --2- 172.21.15.42:0/4069511202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73380d1820 0x7f733809aa50 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.094+0000 7f733ea00640 1 --2- 172.21.15.42:0/4069511202 >> v2:172.21.15.63:3300/0 conn(0x7f733815ea80 0x7f73380d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.094+0000 7f733ea00640 1 -- 172.21.15.42:0/4069511202 >> 172.21.15.42:0/4069511202 conn(0x7f73380b3f90 msgr2=0x7f73380b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.094+0000 7f733ea00640 1 -- 172.21.15.42:0/4069511202 shutdown_connections 2024-05-28T14:06:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.094+0000 7f733ea00640 1 -- 172.21.15.42:0/4069511202 wait complete. 2024-05-28T14:06:35.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ea00640 1 Processor -- start 2024-05-28T14:06:35.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ea00640 1 -- start start 2024-05-28T14:06:35.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f733809af90 0x7f73380cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73380d1820 0x7f73381568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f733815ea80 0x7f73380c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f733816b540 con 0x7f733815ea80 2024-05-28T14:06:35.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f73380b1c00 con 0x7f73380d1820 2024-05-28T14:06:35.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ea00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f733816b6c0 con 0x7f733809af90 2024-05-28T14:06:35.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f733815ea80 0x7f73380c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f733815ea80 0x7f73380c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53208/0 (socket says 172.21.15.42:53208) 2024-05-28T14:06:35.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ca00640 1 -- 172.21.15.42:0/4168449723 learned_addr learned my addr 172.21.15.42:0/4168449723 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:35.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f732fe00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73380d1820 0x7f73381568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ca00640 1 -- 172.21.15.42:0/4168449723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f733809af90 msgr2=0x7f73380cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f7337e00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f733809af90 0x7f73380cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ca00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f733809af90 0x7f73380cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ca00640 1 -- 172.21.15.42:0/4168449723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73380d1820 msgr2=0x7f73381568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ca00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73380d1820 0x7f73381568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ca00640 1 -- 172.21.15.42:0/4168449723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73380c9c60 con 0x7f733815ea80 2024-05-28T14:06:35.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f732fe00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73380d1820 0x7f73381568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:35.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f7337e00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f733809af90 0x7f73380cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:35.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ca00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f733815ea80 0x7f73380c9e60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f733000c700 tx=0x7f733000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:35.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f7334c00640 1 -- 172.21.15.42:0/4168449723 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330018070 con 0x7f733815ea80 2024-05-28T14:06:35.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f7334c00640 1 -- 172.21.15.42:0/4168449723 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733000ce10 con 0x7f733815ea80 2024-05-28T14:06:35.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f7334c00640 1 -- 172.21.15.42:0/4168449723 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733000a860 con 0x7f733815ea80 2024-05-28T14:06:35.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ea00640 1 -- 172.21.15.42:0/4168449723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73380b0710 con 0x7f733815ea80 2024-05-28T14:06:35.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.098+0000 7f733ea00640 1 -- 172.21.15.42:0/4168449723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73380b0cd0 con 0x7f733815ea80 2024-05-28T14:06:35.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.102+0000 7f733ea00640 1 -- 172.21.15.42:0/4168449723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7338068a90 con 0x7f733815ea80 2024-05-28T14:06:35.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.102+0000 7f7334c00640 1 -- 172.21.15.42:0/4168449723 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f733000aa00 con 0x7f733815ea80 2024-05-28T14:06:35.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.102+0000 7f7334c00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7314069070 0x7f731406b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.102+0000 7f7334c00640 1 -- 172.21.15.42:0/4168449723 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7f73300849a0 con 0x7f733815ea80 2024-05-28T14:06:35.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.102+0000 7f7337e00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7314069070 0x7f731406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.102+0000 7f7337e00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7314069070 0x7f731406b530 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7328009ae0 tx=0x7f7328008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:35.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.106+0000 7f7334c00640 1 -- 172.21.15.42:0/4168449723 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f73380c9c60 con 0x7f733815ea80 2024-05-28T14:06:35.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.306+0000 7f733ea00640 1 -- 172.21.15.42:0/4168449723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73380d13f0 con 0x7f733815ea80 2024-05-28T14:06:35.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.306+0000 7f7334c00640 1 -- 172.21.15.42:0/4168449723 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5180 (secure 0 0 0) 0x7f7330056a10 con 0x7f733815ea80 2024-05-28T14:06:35.293 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:06:35.293 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2618011545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2618011545},{"type":"v1","addr":"172.21.15.63:6835","nonce":2618011545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":7077,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/986242955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":986242955},{"type":"v1","addr":"172.21.15.42:6836","nonce":986242955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":7080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1618433260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1618433260},{"type":"v1","addr":"172.21.15.63:6834","nonce":1618433260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:06:24.814669+0000","modified":"2024-05-28T14:06:27.829347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6837/1199082803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1199082803},{"type":"v1","addr":"172.21.15.42:6837","nonce":1199082803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7083,"qdb_cluster":[7083]},"id":5}]} 2024-05-28T14:06:35.294 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 46 2024-05-28T14:06:35.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.310+0000 7f733ea00640 1 -- 172.21.15.42:0/4168449723 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7314069070 msgr2=0x7f731406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.310+0000 7f733ea00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7314069070 0x7f731406b530 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7328009ae0 tx=0x7f7328008040 comp rx=0 tx=0).stop 2024-05-28T14:06:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.310+0000 7f733ea00640 1 -- 172.21.15.42:0/4168449723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f733815ea80 msgr2=0x7f73380c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.310+0000 7f733ea00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f733815ea80 0x7f73380c9e60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f733000c700 tx=0x7f733000cbc0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.310+0000 7f733ea00640 1 -- 172.21.15.42:0/4168449723 shutdown_connections 2024-05-28T14:06:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.310+0000 7f733ea00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7314069070 0x7f731406b530 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.310+0000 7f733ea00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f733815ea80 0x7f73380c9e60 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.310+0000 7f733ea00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73380d1820 0x7f73381568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.310+0000 7f733ea00640 1 --2- 172.21.15.42:0/4168449723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f733809af90 0x7f73380cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.310+0000 7f733ea00640 1 -- 172.21.15.42:0/4168449723 >> 172.21.15.42:0/4168449723 conn(0x7f73380b3f90 msgr2=0x7f733815dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.310+0000 7f733ea00640 1 -- 172.21.15.42:0/4168449723 shutdown_connections 2024-05-28T14:06:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.310+0000 7f733ea00640 1 -- 172.21.15.42:0/4168449723 wait complete. 2024-05-28T14:06:35.718 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-28T14:06:24.814669+0000', 'modified': '2024-05-28T14:06:27.829347+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7083}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7083': {'gid': 7083, 'name': 'a', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.42:6837/1199082803', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6835', 'nonce': 1199082803}, {'type': 'v1', 'addr': '172.21.15.42:6837', 'nonce': 1199082803}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7083, 'qdb_cluster': [7083]} 2024-05-28T14:06:35.718 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-28T14:06:35.718 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-28T14:06:35.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.874+0000 7fbb60e00640 1 Processor -- start 2024-05-28T14:06:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.874+0000 7fbb60e00640 1 -- start start 2024-05-28T14:06:35.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.878+0000 7fbb60e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbb5c15bf30 0x7fbb5c15c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.878+0000 7fbb60e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbb5c0cf230 0x7fbb5c072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.878+0000 7fbb60e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb5c0cfc10 con 0x7fbb5c072ba0 2024-05-28T14:06:35.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.878+0000 7fbb60e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fbb5c0721d0 con 0x7fbb5c15bf30 2024-05-28T14:06:35.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.878+0000 7fbb60e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbb5c0cfef0 con 0x7fbb5c0cf230 2024-05-28T14:06:35.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.878+0000 7fbb59e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbb5c15bf30 0x7fbb5c15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.878+0000 7fbb59e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbb5c15bf30 0x7fbb5c15c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53214/0 (socket says 172.21.15.42:53214) 2024-05-28T14:06:35.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.878+0000 7fbb59e00640 1 -- 172.21.15.42:0/3735740496 learned_addr learned my addr 172.21.15.42:0/3735740496 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:35.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.878+0000 7fbb53e00640 1 --2- 172.21.15.42:0/3735740496 >> v2:172.21.15.63:3300/0 conn(0x7fbb5c0cf230 0x7fbb5c072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.878+0000 7fbb59e00640 1 -- 172.21.15.42:0/3735740496 >> v2:172.21.15.63:3300/0 conn(0x7fbb5c0cf230 msgr2=0x7fbb5c072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:35.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.878+0000 7fbb59e00640 1 --2- 172.21.15.42:0/3735740496 >> v2:172.21.15.63:3300/0 conn(0x7fbb5c0cf230 0x7fbb5c072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.878+0000 7fbb59e00640 1 -- 172.21.15.42:0/3735740496 >> v1:172.21.15.63:6789/0 conn(0x7fbb5c072ba0 legacy=0x7fbb5c06ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.878+0000 7fbb59e00640 1 -- 172.21.15.42:0/3735740496 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c0d01d0 con 0x7fbb5c15bf30 2024-05-28T14:06:35.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.882+0000 7fbb59e00640 1 --2- 172.21.15.42:0/3735740496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb5c15bf30 0x7fbb5c15c310 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0b0130 tx=0x7fbb4c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390d783ce26cb674 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:35.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.882+0000 7fbb59400640 1 -- 172.21.15.42:0/3735740496 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0ab610 con 0x7fbb5c15bf30 2024-05-28T14:06:35.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.882+0000 7fbb59400640 1 -- 172.21.15.42:0/3735740496 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb4c0b0070 con 0x7fbb5c15bf30 2024-05-28T14:06:35.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.882+0000 7fbb59400640 1 -- 172.21.15.42:0/3735740496 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0ab9d0 con 0x7fbb5c15bf30 2024-05-28T14:06:35.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.882+0000 7fbb60e00640 1 -- 172.21.15.42:0/3735740496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb5c15bf30 msgr2=0x7fbb5c15c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.882+0000 7fbb60e00640 1 --2- 172.21.15.42:0/3735740496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb5c15bf30 0x7fbb5c15c310 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0b0130 tx=0x7fbb4c01c4f0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.882+0000 7fbb60e00640 1 -- 172.21.15.42:0/3735740496 shutdown_connections 2024-05-28T14:06:35.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.882+0000 7fbb60e00640 1 --2- 172.21.15.42:0/3735740496 >> v2:172.21.15.63:3300/0 conn(0x7fbb5c0cf230 0x7fbb5c072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.882+0000 7fbb60e00640 1 --2- 172.21.15.42:0/3735740496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb5c15bf30 0x7fbb5c15c310 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.882+0000 7fbb60e00640 1 -- 172.21.15.42:0/3735740496 >> 172.21.15.42:0/3735740496 conn(0x7fbb5c06b9f0 msgr2=0x7fbb5c06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:35.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.882+0000 7fbb60e00640 1 -- 172.21.15.42:0/3735740496 shutdown_connections 2024-05-28T14:06:35.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.882+0000 7fbb60e00640 1 -- 172.21.15.42:0/3735740496 wait complete. 2024-05-28T14:06:35.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.882+0000 7fbb60e00640 1 Processor -- start 2024-05-28T14:06:35.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.882+0000 7fbb60e00640 1 -- start start 2024-05-28T14:06:35.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb60e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb5c072ba0 0x7fbb5c14b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb60e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb5c0cf230 0x7fbb5c1512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb60e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5c15bf30 0x7fbb5c1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb60e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c06f4e0 con 0x7fbb5c072ba0 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb60e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c06f360 con 0x7fbb5c0cf230 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb60e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5c0cf610 con 0x7fbb5c15bf30 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb59e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb5c072ba0 0x7fbb5c14b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb59e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb5c072ba0 0x7fbb5c14b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53228/0 (socket says 172.21.15.42:53228) 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb59e00640 1 -- 172.21.15.42:0/2302683717 learned_addr learned my addr 172.21.15.42:0/2302683717 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb53e00640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb5c0cf230 0x7fbb5c1512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb5a800640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5c15bf30 0x7fbb5c1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb59e00640 1 -- 172.21.15.42:0/2302683717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5c15bf30 msgr2=0x7fbb5c1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb59e00640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5c15bf30 0x7fbb5c1549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb59e00640 1 -- 172.21.15.42:0/2302683717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb5c0cf230 msgr2=0x7fbb5c1512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb59e00640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb5c0cf230 0x7fbb5c1512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb59e00640 1 -- 172.21.15.42:0/2302683717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c150230 con 0x7fbb5c072ba0 2024-05-28T14:06:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb53e00640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb5c0cf230 0x7fbb5c1512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:35.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb59e00640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb5c072ba0 0x7fbb5c14b530 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0ad040 tx=0x7fbb4c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:35.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb52000640 1 -- 172.21.15.42:0/2302683717 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0be5f0 con 0x7fbb5c072ba0 2024-05-28T14:06:35.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb52000640 1 -- 172.21.15.42:0/2302683717 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb4c0be7b0 con 0x7fbb5c072ba0 2024-05-28T14:06:35.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb52000640 1 -- 172.21.15.42:0/2302683717 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0beba0 con 0x7fbb5c072ba0 2024-05-28T14:06:35.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb60e00640 1 -- 172.21.15.42:0/2302683717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5c150510 con 0x7fbb5c072ba0 2024-05-28T14:06:35.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb60e00640 1 -- 172.21.15.42:0/2302683717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb5c150ad0 con 0x7fbb5c072ba0 2024-05-28T14:06:35.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.886+0000 7fbb52000640 1 -- 172.21.15.42:0/2302683717 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7fbb4c0b43f0 con 0x7fbb5c072ba0 2024-05-28T14:06:35.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb52000640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb38069070 0x7fbb3806b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb52000640 1 -- 172.21.15.42:0/2302683717 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7fbb4c11f790 con 0x7fbb5c072ba0 2024-05-28T14:06:35.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 Processor -- start 2024-05-28T14:06:35.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb53e00640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb38069070 0x7fbb3806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 -- start start 2024-05-28T14:06:35.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb53e00640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb38069070 0x7fbb3806b530 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0a9700 tx=0x7fbb44008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:35.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b293500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b293a40 0x56508b246110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b246650 0x56508b246a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56508b097420 con 0x56508b293a40 2024-05-28T14:06:35.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56508b031bf0 con 0x56508b293120 2024-05-28T14:06:35.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56508b0933e0 con 0x56508b246650 2024-05-28T14:06:35.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b293a40 0x56508b246110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b293a40 0x56508b246110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53244/0 (socket says 172.21.15.42:53244) 2024-05-28T14:06:35.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb53e00640 1 -- 172.21.15.42:0/2969943581 learned_addr learned my addr 172.21.15.42:0/2969943581 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:35.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb59e00640 1 --2- 172.21.15.42:0/2969943581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b293500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb5a800640 1 --2- 172.21.15.42:0/2969943581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b246650 0x56508b246a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb53e00640 1 -- 172.21.15.42:0/2969943581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b246650 msgr2=0x56508b246a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:35.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb53e00640 1 --2- 172.21.15.42:0/2969943581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b246650 0x56508b246a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb53e00640 1 -- 172.21.15.42:0/2969943581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 msgr2=0x56508b293500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:35.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb53e00640 1 --2- 172.21.15.42:0/2969943581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b293500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb53e00640 1 -- 172.21.15.42:0/2969943581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56508b13f450 con 0x56508b293a40 2024-05-28T14:06:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb5a800640 1 --2- 172.21.15.42:0/2969943581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b246650 0x56508b246a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb53e00640 1 --2- 172.21.15.42:0/2969943581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b293a40 0x56508b246110 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400fab0 tx=0x7fbb44012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ad6986d720fd4c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb4aa00640 1 -- 172.21.15.42:0/2969943581 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4401b050 con 0x56508b293a40 2024-05-28T14:06:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb4aa00640 1 -- 172.21.15.42:0/2969943581 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb44025070 con 0x56508b293a40 2024-05-28T14:06:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb4aa00640 1 -- 172.21.15.42:0/2969943581 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44020440 con 0x56508b293a40 2024-05-28T14:06:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 -- 172.21.15.42:0/2969943581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b293a40 msgr2=0x56508b246110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 --2- 172.21.15.42:0/2969943581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b293a40 0x56508b246110 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400fab0 tx=0x7fbb44012000 comp rx=0 tx=0).stop 2024-05-28T14:06:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 -- 172.21.15.42:0/2969943581 shutdown_connections 2024-05-28T14:06:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 --2- 172.21.15.42:0/2969943581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b246650 0x56508b246a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 --2- 172.21.15.42:0/2969943581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b293a40 0x56508b246110 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 --2- 172.21.15.42:0/2969943581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b293500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.890+0000 7fbb64202000 1 -- 172.21.15.42:0/2969943581 >> 172.21.15.42:0/2969943581 conn(0x56508b273ec0 msgr2=0x56508b2762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 1 -- 172.21.15.42:0/2969943581 shutdown_connections 2024-05-28T14:06:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 1 -- 172.21.15.42:0/2969943581 wait complete. 2024-05-28T14:06:35.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 1 -- start start 2024-05-28T14:06:35.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 1 Processor -- start 2024-05-28T14:06:35.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b24bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b24c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b293a40 0x56508b24c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56508b037ec0 con 0x56508b246650 2024-05-28T14:06:35.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56508b118120 con 0x56508b293120 2024-05-28T14:06:35.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56508b0a2350 con 0x56508b293a40 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 10 client.0 _ll_register_callbacks cb 0x56508b0744d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb59e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b24bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb59e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b24bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53260/0 (socket says 172.21.15.42:53260) 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb59e00640 1 -- 172.21.15.42:0/2433825926 learned_addr learned my addr 172.21.15.42:0/2433825926 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb53e00640 1 --2- 172.21.15.42:0/2433825926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b24c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb5a800640 1 --2- 172.21.15.42:0/2433825926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b293a40 0x56508b24c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb59e00640 1 -- 172.21.15.42:0/2433825926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b293a40 msgr2=0x56508b24c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb59e00640 1 --2- 172.21.15.42:0/2433825926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b293a40 0x56508b24c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb59e00640 1 -- 172.21.15.42:0/2433825926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 msgr2=0x56508b24c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb59e00640 1 --2- 172.21.15.42:0/2433825926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b24c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb59e00640 1 -- 172.21.15.42:0/2433825926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56508b0dcf80 con 0x56508b246650 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb53e00640 1 --2- 172.21.15.42:0/2433825926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b24c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb59e00640 1 --2- 172.21.15.42:0/2433825926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b24bb40 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c097fe0 tx=0x7fbb4c0f1540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e221d69eb1f96f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb3f400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T14:06:35.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb3f400640 1 -- 172.21.15.42:0/2433825926 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c11d060 con 0x56508b246650 2024-05-28T14:06:35.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb3f400640 1 -- 172.21.15.42:0/2433825926 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7fbb4c0f03d0 con 0x56508b246650 2024-05-28T14:06:35.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 1 -- 172.21.15.42:0/2433825926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb44025070 con 0x56508b246650 2024-05-28T14:06:35.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb3f400640 1 -- 172.21.15.42:0/2433825926 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fbb4c10e050 con 0x56508b246650 2024-05-28T14:06:35.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 10 client.7239 fetch_fsmap learned FSMap version 46 2024-05-28T14:06:35.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 1 -- 172.21.15.42:0/2433825926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x56508b070f00 con 0x56508b246650 2024-05-28T14:06:35.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb3f400640 1 -- 172.21.15.42:0/2433825926 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb44025070 con 0x56508b246650 2024-05-28T14:06:35.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb3f400640 1 -- 172.21.15.42:0/2433825926 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7fbb4c0ecad0 con 0x56508b246650 2024-05-28T14:06:35.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 10 client.7239 fetch_fsmap finished waiting for FSMap version 46 2024-05-28T14:06:35.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 10 client.7239 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-28T14:06:35.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 20 client.7239 populate_metadata read hostname 'smithi042' 2024-05-28T14:06:35.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 1 --2- 172.21.15.42:0/2433825926 >> [v2:172.21.15.42:6835/1199082803,v1:172.21.15.42:6837/1199082803] conn(0x56508b2a23c0 0x56508b2a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 4 client.7239 mds_command: new command op to 7083 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-28T14:06:35.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb64202000 1 -- 172.21.15.42:0/2433825926 --> [v2:172.21.15.42:6835/1199082803,v1:172.21.15.42:6837/1199082803] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56508b030bc0 con 0x56508b2a23c0 2024-05-28T14:06:35.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.894+0000 7fbb53e00640 1 --2- 172.21.15.42:0/2433825926 >> [v2:172.21.15.42:6835/1199082803,v1:172.21.15.42:6837/1199082803] conn(0x56508b2a23c0 0x56508b2a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb3f400640 10 client.7239 ms_handle_connect on v2:172.21.15.42:6835/1199082803 2024-05-28T14:06:35.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb53e00640 1 --2- 172.21.15.42:0/2433825926 >> [v2:172.21.15.42:6835/1199082803,v1:172.21.15.42:6837/1199082803] conn(0x56508b2a23c0 0x56508b2a47b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3578ae73b62883ae server_cookie=ae64ef4d0fa45355 in_seq=0 out_seq=0 2024-05-28T14:06:35.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb3f400640 1 -- 172.21.15.42:0/2433825926 <== mds.0 v2:172.21.15.42:6835/1199082803 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56508b030bc0 con 0x56508b2a23c0 2024-05-28T14:06:35.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb3f400640 10 client.7239 handle_command_reply: tid=0 2024-05-28T14:06:35.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb64202000 1 client.7239 shutdown 2024-05-28T14:06:35.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb64202000 1 -- 172.21.15.42:0/2433825926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 msgr2=0x56508b24bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb64202000 1 --2- 172.21.15.42:0/2433825926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b24bb40 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c097fe0 tx=0x7fbb4c0f1540 comp rx=0 tx=0).stop 2024-05-28T14:06:35.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb64202000 1 -- 172.21.15.42:0/2433825926 shutdown_connections 2024-05-28T14:06:35.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb64202000 1 --2- 172.21.15.42:0/2433825926 >> [v2:172.21.15.42:6835/1199082803,v1:172.21.15.42:6837/1199082803] conn(0x56508b2a23c0 0x56508b2a47b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb64202000 1 --2- 172.21.15.42:0/2433825926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b293a40 0x56508b24c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb64202000 1 --2- 172.21.15.42:0/2433825926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b24c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb3f400640 0 client.7239 ms_handle_reset on v2:172.21.15.42:6835/1199082803 2024-05-28T14:06:35.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb64202000 1 --2- 172.21.15.42:0/2433825926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b24bb40 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb64202000 1 -- 172.21.15.42:0/2433825926 >> 172.21.15.42:0/2433825926 conn(0x56508b273ec0 msgr2=0x56508b2762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:35.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb64202000 1 -- 172.21.15.42:0/2433825926 shutdown_connections 2024-05-28T14:06:35.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb64202000 1 -- 172.21.15.42:0/2433825926 wait complete. 2024-05-28T14:06:35.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.898+0000 7fbb64202000 20 client.7239 trim_cache size 0 max 16384 2024-05-28T14:06:35.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.914+0000 7fbb64202000 1 Processor -- start 2024-05-28T14:06:35.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.914+0000 7fbb64202000 1 -- start start 2024-05-28T14:06:35.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b273ec0 0x56508b245770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b245cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b2a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c2028e0 con 0x56508b246650 2024-05-28T14:06:35.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c0aff10 con 0x56508b293120 2024-05-28T14:06:35.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5c0b0560 con 0x56508b273ec0 2024-05-28T14:06:35.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b245cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b245cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53262/0 (socket says 172.21.15.42:53262) 2024-05-28T14:06:35.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb53e00640 1 -- 172.21.15.42:0/3013736866 learned_addr learned my addr 172.21.15.42:0/3013736866 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:35.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb5a800640 1 --2- 172.21.15.42:0/3013736866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b2a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb59e00640 1 --2- 172.21.15.42:0/3013736866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b273ec0 0x56508b245770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb53e00640 1 -- 172.21.15.42:0/3013736866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b273ec0 msgr2=0x56508b245770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb53e00640 1 --2- 172.21.15.42:0/3013736866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b273ec0 0x56508b245770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb53e00640 1 -- 172.21.15.42:0/3013736866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 msgr2=0x56508b2a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb53e00640 1 --2- 172.21.15.42:0/3013736866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b2a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb53e00640 1 -- 172.21.15.42:0/3013736866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56508b293870 con 0x56508b246650 2024-05-28T14:06:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb5a800640 1 --2- 172.21.15.42:0/3013736866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b2a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb53e00640 1 --2- 172.21.15.42:0/3013736866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b245cb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbb44021000 tx=0x7fbb44014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394b309d899d6418 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb4a000640 1 -- 172.21.15.42:0/3013736866 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4401b070 con 0x56508b246650 2024-05-28T14:06:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb4a000640 1 -- 172.21.15.42:0/3013736866 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb44025040 con 0x56508b246650 2024-05-28T14:06:35.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb4a000640 1 -- 172.21.15.42:0/3013736866 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44034d50 con 0x56508b246650 2024-05-28T14:06:35.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 -- 172.21.15.42:0/3013736866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 msgr2=0x56508b245cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 --2- 172.21.15.42:0/3013736866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b245cb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbb44021000 tx=0x7fbb44014000 comp rx=0 tx=0).stop 2024-05-28T14:06:35.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 -- 172.21.15.42:0/3013736866 shutdown_connections 2024-05-28T14:06:35.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 --2- 172.21.15.42:0/3013736866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b2a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 --2- 172.21.15.42:0/3013736866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b245cb0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 --2- 172.21.15.42:0/3013736866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b273ec0 0x56508b245770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 -- 172.21.15.42:0/3013736866 >> 172.21.15.42:0/3013736866 conn(0x56508b24e540 msgr2=0x56508b2a7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:35.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 -- 172.21.15.42:0/3013736866 shutdown_connections 2024-05-28T14:06:35.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 -- 172.21.15.42:0/3013736866 wait complete. 2024-05-28T14:06:35.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 -- start start 2024-05-28T14:06:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.918+0000 7fbb64202000 1 Processor -- start 2024-05-28T14:06:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b2a7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b273ec0 0x56508b2be340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b2be880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56508b096730 con 0x56508b246650 2024-05-28T14:06:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56508b0639e0 con 0x56508b293120 2024-05-28T14:06:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56508b09ba50 con 0x56508b273ec0 2024-05-28T14:06:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 10 client.0 _ll_register_callbacks cb 0x7fbb4c0abf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T14:06:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb59e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b2a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb59e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b2a7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53272/0 (socket says 172.21.15.42:53272) 2024-05-28T14:06:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb59e00640 1 -- 172.21.15.42:0/763951047 learned_addr learned my addr 172.21.15.42:0/763951047 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb5a800640 1 --2- 172.21.15.42:0/763951047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b2be880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb53e00640 1 --2- 172.21.15.42:0/763951047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b273ec0 0x56508b2be340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb59e00640 1 -- 172.21.15.42:0/763951047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b273ec0 msgr2=0x56508b2be340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb59e00640 1 --2- 172.21.15.42:0/763951047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b273ec0 0x56508b2be340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb59e00640 1 -- 172.21.15.42:0/763951047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 msgr2=0x56508b2be880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb59e00640 1 --2- 172.21.15.42:0/763951047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b2be880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb59e00640 1 -- 172.21.15.42:0/763951047 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56508b01de80 con 0x56508b246650 2024-05-28T14:06:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb53e00640 1 --2- 172.21.15.42:0/763951047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b273ec0 0x56508b2be340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb59e00640 1 --2- 172.21.15.42:0/763951047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b2a7970 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c01c320 tx=0x7fbb4c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e9657b04660127 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb3f400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T14:06:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb3f400640 1 -- 172.21.15.42:0/763951047 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c1093f0 con 0x56508b246650 2024-05-28T14:06:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb3f400640 1 -- 172.21.15.42:0/763951047 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7fbb4c10acc0 con 0x56508b246650 2024-05-28T14:06:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 -- 172.21.15.42:0/763951047 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb44025040 con 0x56508b246650 2024-05-28T14:06:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb3f400640 1 -- 172.21.15.42:0/763951047 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fbb4c11d020 con 0x56508b246650 2024-05-28T14:06:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 10 client.7245 fetch_fsmap learned FSMap version 46 2024-05-28T14:06:35.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 -- 172.21.15.42:0/763951047 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x56508b05fb30 con 0x56508b246650 2024-05-28T14:06:35.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb3f400640 1 -- 172.21.15.42:0/763951047 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb44025040 con 0x56508b246650 2024-05-28T14:06:35.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb3f400640 1 -- 172.21.15.42:0/763951047 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7fbb4c0f62c0 con 0x56508b246650 2024-05-28T14:06:35.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 10 client.7245 fetch_fsmap finished waiting for FSMap version 46 2024-05-28T14:06:35.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 10 client.7245 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-28T14:06:35.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 20 client.7245 populate_metadata read hostname 'smithi042' 2024-05-28T14:06:35.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 --2- 172.21.15.42:0/763951047 >> [v2:172.21.15.42:6835/1199082803,v1:172.21.15.42:6837/1199082803] conn(0x56508b293a40 0x56508b2bedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:35.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 4 client.7245 mds_command: new command op to 7083 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-28T14:06:35.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 -- 172.21.15.42:0/763951047 --> [v2:172.21.15.42:6835/1199082803,v1:172.21.15.42:6837/1199082803] -- command(tid 0: {"prefix": "status"}) -- 0x56508b0a4e40 con 0x56508b293a40 2024-05-28T14:06:35.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb53e00640 1 --2- 172.21.15.42:0/763951047 >> [v2:172.21.15.42:6835/1199082803,v1:172.21.15.42:6837/1199082803] conn(0x56508b293a40 0x56508b2bedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:35.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb3f400640 10 client.7245 ms_handle_connect on v2:172.21.15.42:6835/1199082803 2024-05-28T14:06:35.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb53e00640 1 --2- 172.21.15.42:0/763951047 >> [v2:172.21.15.42:6835/1199082803,v1:172.21.15.42:6837/1199082803] conn(0x56508b293a40 0x56508b2bedc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71fecfabd7eec8e5 server_cookie=7cb2281bb607b43f in_seq=0 out_seq=0 2024-05-28T14:06:35.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb3f400640 1 -- 172.21.15.42:0/763951047 <== mds.0 v2:172.21.15.42:6835/1199082803 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56508b0a4e40 con 0x56508b293a40 2024-05-28T14:06:35.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb3f400640 10 client.7245 handle_command_reply: tid=0 2024-05-28T14:06:35.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 client.7245 shutdown 2024-05-28T14:06:35.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 -- 172.21.15.42:0/763951047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 msgr2=0x56508b2a7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 --2- 172.21.15.42:0/763951047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b2a7970 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c01c320 tx=0x7fbb4c000bd0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 -- 172.21.15.42:0/763951047 shutdown_connections 2024-05-28T14:06:35.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 --2- 172.21.15.42:0/763951047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56508b293120 0x56508b2be880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 --2- 172.21.15.42:0/763951047 >> [v2:172.21.15.42:6835/1199082803,v1:172.21.15.42:6837/1199082803] conn(0x56508b293a40 0x56508b2bedc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 --2- 172.21.15.42:0/763951047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56508b273ec0 0x56508b2be340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 --2- 172.21.15.42:0/763951047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56508b246650 0x56508b2a7970 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb3f400640 0 client.7245 ms_handle_reset on v2:172.21.15.42:6835/1199082803 2024-05-28T14:06:35.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 -- 172.21.15.42:0/763951047 >> 172.21.15.42:0/763951047 conn(0x56508b24e540 msgr2=0x56508b2911d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:35.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 -- 172.21.15.42:0/763951047 shutdown_connections 2024-05-28T14:06:35.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.922+0000 7fbb64202000 1 -- 172.21.15.42:0/763951047 wait complete. 2024-05-28T14:06:35.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.926+0000 7fbb64202000 20 client.7245 trim_cache size 0 max 16384 2024-05-28T14:06:35.915 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:06:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:06:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-05-28T14:06:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 7083, 2024-05-28T14:06:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-28T14:06:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-28T14:06:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-28T14:06:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 10.504011240000001, 2024-05-28T14:06:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 46, 2024-05-28T14:06:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 74, 2024-05-28T14:06:35.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 74, 2024-05-28T14:06:35.916 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 15.356555008999999 2024-05-28T14:06:35.916 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:06:35.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.926+0000 7fbb60e00640 1 -- 172.21.15.42:0/2302683717 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb38069070 msgr2=0x7fbb3806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.926+0000 7fbb60e00640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb38069070 0x7fbb3806b530 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0a9700 tx=0x7fbb44008040 comp rx=0 tx=0).stop 2024-05-28T14:06:35.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.926+0000 7fbb60e00640 1 -- 172.21.15.42:0/2302683717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb5c072ba0 msgr2=0x7fbb5c14b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:35.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.926+0000 7fbb60e00640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb5c072ba0 0x7fbb5c14b530 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0ad040 tx=0x7fbb4c01ad30 comp rx=0 tx=0).stop 2024-05-28T14:06:35.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.926+0000 7fbb60e00640 1 -- 172.21.15.42:0/2302683717 shutdown_connections 2024-05-28T14:06:35.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.926+0000 7fbb60e00640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb38069070 0x7fbb3806b530 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.926+0000 7fbb60e00640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5c15bf30 0x7fbb5c1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.926+0000 7fbb60e00640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb5c0cf230 0x7fbb5c1512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.926+0000 7fbb60e00640 1 --2- 172.21.15.42:0/2302683717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb5c072ba0 0x7fbb5c14b530 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:35.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.926+0000 7fbb60e00640 1 -- 172.21.15.42:0/2302683717 >> 172.21.15.42:0/2302683717 conn(0x7fbb5c06b9f0 msgr2=0x7fbb5c15a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:35.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.926+0000 7fbb60e00640 1 -- 172.21.15.42:0/2302683717 shutdown_connections 2024-05-28T14:06:35.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:35.926+0000 7fbb60e00640 1 -- 172.21.15.42:0/2302683717 wait complete. 2024-05-28T14:06:35.931 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-28T14:06:36.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62fc600640 1 Processor -- start 2024-05-28T14:06:36.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62fc600640 1 -- start start 2024-05-28T14:06:36.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62fc600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f40d0420 0x7f62f415ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:36.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62fc600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f62f4067810 0x7f62f40b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:36.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62fc600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62f406eaa0 con 0x7f62f40b4110 2024-05-28T14:06:36.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62fc600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f62f406ed80 con 0x7f62f4067810 2024-05-28T14:06:36.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62fc600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f62f406f060 con 0x7f62f40d0420 2024-05-28T14:06:36.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62fa600640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f62f40b4110 0x7f62f4162350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42874/0 (socket says 172.21.15.42:42874) 2024-05-28T14:06:36.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62fa600640 1 -- 172.21.15.42:0/4203585774 learned_addr learned my addr 172.21.15.42:0/4203585774 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:36.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62f9200640 1 --2- 172.21.15.42:0/4203585774 >> v2:172.21.15.42:3300/0 conn(0x7f62f4067810 0x7f62f40b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:36.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62f9c00640 1 --2- 172.21.15.42:0/4203585774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f40d0420 0x7f62f415ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:36.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62f9200640 1 -- 172.21.15.42:0/4203585774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f40d0420 msgr2=0x7f62f415ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:36.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62f9200640 1 --2- 172.21.15.42:0/4203585774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f40d0420 0x7f62f415ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:36.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62f9200640 1 -- 172.21.15.42:0/4203585774 >> v1:172.21.15.42:6789/0 conn(0x7f62f40b4110 legacy=0x7f62f4162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:36.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.086+0000 7f62f9200640 1 -- 172.21.15.42:0/4203585774 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f406f340 con 0x7f62f4067810 2024-05-28T14:06:36.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.090+0000 7f62f9200640 1 --2- 172.21.15.42:0/4203585774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4067810 0x7f62f40b3bd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f62e00097c0 tx=0x7f62e001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6b5aa03c6cbbb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:36.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.090+0000 7f62f3e00640 1 -- 172.21.15.42:0/4203585774 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e00b0070 con 0x7f62f4067810 2024-05-28T14:06:36.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.090+0000 7f62f3e00640 1 -- 172.21.15.42:0/4203585774 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e00ab600 con 0x7f62f4067810 2024-05-28T14:06:36.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.090+0000 7f62f3e00640 1 -- 172.21.15.42:0/4203585774 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e00ab9d0 con 0x7f62f4067810 2024-05-28T14:06:36.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.090+0000 7f62fc600640 1 -- 172.21.15.42:0/4203585774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4067810 msgr2=0x7f62f40b3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:36.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.090+0000 7f62fc600640 1 --2- 172.21.15.42:0/4203585774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4067810 0x7f62f40b3bd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f62e00097c0 tx=0x7f62e001c410 comp rx=0 tx=0).stop 2024-05-28T14:06:36.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.090+0000 7f62fc600640 1 -- 172.21.15.42:0/4203585774 shutdown_connections 2024-05-28T14:06:36.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.090+0000 7f62fc600640 1 --2- 172.21.15.42:0/4203585774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4067810 0x7f62f40b3bd0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:36.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.090+0000 7f62fc600640 1 --2- 172.21.15.42:0/4203585774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f40d0420 0x7f62f415ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:36.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.090+0000 7f62fc600640 1 -- 172.21.15.42:0/4203585774 >> 172.21.15.42:0/4203585774 conn(0x7f62f40d0b40 msgr2=0x7f62f40d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:36.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.090+0000 7f62fc600640 1 -- 172.21.15.42:0/4203585774 shutdown_connections 2024-05-28T14:06:36.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.090+0000 7f62fc600640 1 -- 172.21.15.42:0/4203585774 wait complete. 2024-05-28T14:06:36.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62fc600640 1 Processor -- start 2024-05-28T14:06:36.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62fc600640 1 -- start start 2024-05-28T14:06:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62fc600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4067810 0x7f62f41537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62fc600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f40b4110 0x7f62f4151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62fc600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f40d0420 0x7f62f4151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62fc600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f62f416b640 con 0x7f62f4067810 2024-05-28T14:06:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62fc600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62f416b4c0 con 0x7f62f40b4110 2024-05-28T14:06:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62fc600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f62f416b7c0 con 0x7f62f40d0420 2024-05-28T14:06:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62f9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4067810 0x7f62f41537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62f9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4067810 0x7f62f41537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53284/0 (socket says 172.21.15.42:53284) 2024-05-28T14:06:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62f9c00640 1 -- 172.21.15.42:0/1822120282 learned_addr learned my addr 172.21.15.42:0/1822120282 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62f9200640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f40b4110 0x7f62f4151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62fa600640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f40d0420 0x7f62f4151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62f9c00640 1 -- 172.21.15.42:0/1822120282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f40d0420 msgr2=0x7f62f4151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62f9c00640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f40d0420 0x7f62f4151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62f9c00640 1 -- 172.21.15.42:0/1822120282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f40b4110 msgr2=0x7f62f4151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62f9c00640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f40b4110 0x7f62f4151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62f9c00640 1 -- 172.21.15.42:0/1822120282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f4152290 con 0x7f62f4067810 2024-05-28T14:06:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62fa600640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f40d0420 0x7f62f4151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62f9c00640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4067810 0x7f62f41537c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f62e800e870 tx=0x7f62e800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62f1600640 1 -- 172.21.15.42:0/1822120282 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8017070 con 0x7f62f4067810 2024-05-28T14:06:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62f1600640 1 -- 172.21.15.42:0/1822120282 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e800c940 con 0x7f62f4067810 2024-05-28T14:06:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62f1600640 1 -- 172.21.15.42:0/1822120282 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e800ccf0 con 0x7f62f4067810 2024-05-28T14:06:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62fc600640 1 -- 172.21.15.42:0/1822120282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f4152800 con 0x7f62f4067810 2024-05-28T14:06:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62fc600640 1 -- 172.21.15.42:0/1822120282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f4152dc0 con 0x7f62f4067810 2024-05-28T14:06:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.094+0000 7f62fc600640 1 -- 172.21.15.42:0/1822120282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f4068a90 con 0x7f62f4067810 2024-05-28T14:06:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.098+0000 7f62f1600640 1 -- 172.21.15.42:0/1822120282 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f62e801a070 con 0x7f62f4067810 2024-05-28T14:06:36.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.098+0000 7f62f1600640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f62d4069070 0x7f62d406b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:36.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.098+0000 7f62f1600640 1 -- 172.21.15.42:0/1822120282 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7f62e8085220 con 0x7f62f4067810 2024-05-28T14:06:36.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.098+0000 7f62f9200640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f62d4069070 0x7f62d406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:36.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.098+0000 7f62f9200640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f62d4069070 0x7f62d406b530 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f62e00ad000 tx=0x7f62e0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:36.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.102+0000 7f62f1600640 1 -- 172.21.15.42:0/1822120282 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f62e8056be0 con 0x7f62f4067810 2024-05-28T14:06:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.302+0000 7f62fc600640 1 -- 172.21.15.42:0/1822120282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f62f4152520 con 0x7f62f4067810 2024-05-28T14:06:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.302+0000 7f62f1600640 1 -- 172.21.15.42:0/1822120282 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7f62e805aee0 con 0x7f62f4067810 2024-05-28T14:06:36.289 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-28T14:06:36.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.302+0000 7f62fc600640 1 -- 172.21.15.42:0/1822120282 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f62d4069070 msgr2=0x7f62d406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:36.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.302+0000 7f62fc600640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f62d4069070 0x7f62d406b530 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f62e00ad000 tx=0x7f62e0008040 comp rx=0 tx=0).stop 2024-05-28T14:06:36.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.302+0000 7f62fc600640 1 -- 172.21.15.42:0/1822120282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4067810 msgr2=0x7f62f41537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:36.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.302+0000 7f62fc600640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4067810 0x7f62f41537c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f62e800e870 tx=0x7f62e800ec70 comp rx=0 tx=0).stop 2024-05-28T14:06:36.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.302+0000 7f62fc600640 1 -- 172.21.15.42:0/1822120282 shutdown_connections 2024-05-28T14:06:36.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.302+0000 7f62fc600640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f62d4069070 0x7f62d406b530 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:36.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.302+0000 7f62fc600640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f40d0420 0x7f62f4151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:36.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.302+0000 7f62fc600640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f40b4110 0x7f62f4151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:36.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.302+0000 7f62fc600640 1 --2- 172.21.15.42:0/1822120282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4067810 0x7f62f41537c0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:36.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.302+0000 7f62fc600640 1 -- 172.21.15.42:0/1822120282 >> 172.21.15.42:0/1822120282 conn(0x7f62f40d0b40 msgr2=0x7f62f4168c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:36.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.302+0000 7f62fc600640 1 -- 172.21.15.42:0/1822120282 shutdown_connections 2024-05-28T14:06:36.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36.302+0000 7f62fc600640 1 -- 172.21.15.42:0/1822120282 wait complete. 2024-05-28T14:06:36.311 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-28T14:06:36.311 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-28T14:06:36.311 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-05-28T14:06:36.311 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-28T14:06:36.311 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-28T14:06:36.311 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-28T14:06:36.311 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-28T14:06:36.311 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-28T14:06:36.311 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:06:36.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:36.312 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-05-28T14:06:36.321 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T14:06:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T14:06:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T14:06:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:06:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-05-28T14:06:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:06:36.321 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:06:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-05-28T14:06:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-05-28T14:06:36.322 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:06:36.322 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-05-28T14:06:36.322 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-05-28T14:06:36.322 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-28T14:06:36.322 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:06:36.322 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-05-28T14:06:36.322 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-28T14:06:36.322 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 41951sec preferred_lft 41951sec 2024-05-28T14:06:36.322 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2024-05-28T14:06:36.322 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:06:36.322 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:06:36.322 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:06:36.322 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-05-28T14:06:36.323 INFO:teuthology.orchestra.run.smithi042.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-28T14:06:36.323 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 02:42:65:2b:63:73 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:06:36.323 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-28T14:06:36.323 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:06:36.323 INFO:teuthology.orchestra.run.smithi042.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-28T14:06:36.323 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-28T14:06:36.323 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-28T14:06:36.323 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:06:36.323 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-28T14:06:36.323 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:06:36.323 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T14:06:36.323 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:06:36.323 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-28T14:06:36.323 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:06:36.324 INFO:teuthology.orchestra.run.smithi042.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T14:06:36.324 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:06:36.324 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-28T14:06:36.324 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:06:36.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:36.325 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-28T14:06:36.385 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-05-28T14:06:36.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:36.387 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-05-28T14:06:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-05-28T14:06:36.440 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-28T14:06:36.441 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-05-28T14:06:36.441 INFO:teuthology.orchestra.run.smithi042.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-28T14:06:36.441 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-05-28T14:06:36.441 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-28T14:06:36.441 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T14:06:36.441 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T14:06:36.441 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-28T14:06:36.441 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-28T14:06:36.441 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-28T14:06:36.441 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T14:06:36.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T14:06:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:06:36.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:36.580 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-28T14:06:36.590 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-28T14:06:36.590 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-28T14:06:36.590 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:06:36.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:36.591 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-05-28T14:06:36.654 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-28T14:06:36.655 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T14:06:36.655 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-28T14:06:36.707 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:06:36.707 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T14:06:36.707 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-28T14:06:36.758 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:06:36.819 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:06:36.872 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:06:36.872 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:06:36.872 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:06:36.872 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:06:36.872 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:06:36.872 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:06:36.872 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:06:36.873 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:06:36.873 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:06:36.873 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:06:36.873 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:06:36.873 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:06:36.873 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:06:36.873 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:06:36.873 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:06:36.873 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:06:36.873 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:06:36.873 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:06:36.873 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:06:36.873 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:06:36.874 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:06:36.874 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:06:36.874 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:06:36.874 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:06:36.874 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:06:36.874 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:06:36.874 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:06:36.874 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:06:36.874 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:06:36.874 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:06:36.874 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:06:36.874 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:06:36.874 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:06:36.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:36.875 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:06:36.931 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-28T14:06:36.931 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-28T14:06:36.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:06:37.002 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:06:37.052 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:06:37.052 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:06:37.052 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:06:37.052 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:06:37.052 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:06:37.053 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:06:37.053 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:06:37.053 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:06:37.053 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:06:37.053 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:06:37.053 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:06:37.053 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:06:37.053 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:06:37.053 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:06:37.053 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:06:37.053 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:06:37.053 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:06:37.053 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:06:37.054 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:06:37.054 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:06:37.054 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:06:37.054 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:06:37.054 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:06:37.054 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:06:37.054 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:06:37.054 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:06:37.054 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:06:37.054 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:06:37.054 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:06:37.054 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:06:37.054 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:06:37.055 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:06:37.055 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:06:37.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:37.056 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:06:37.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-05-28T14:06:37.090+0000 7f111b865580 -1 init, newargv = 0x55c85351aa60 newargc=15 2024-05-28T14:06:37.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[44218]: starting ceph client 2024-05-28T14:06:37.085 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[44218]: starting fuse 2024-05-28T14:06:37.087 INFO:teuthology.orchestra.run.smithi042.stdout:44 2024-05-28T14:06:37.088 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-28T14:06:37.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-05-28T14:06:37.088 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-05-28T14:06:37.088 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-05-28T14:06:37.088 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-05-28T14:06:37.088 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-05-28T14:06:37.088 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:06:37.088 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-05-28T14:06:37.088 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-28T14:06:37.088 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-05-28T14:06:37.088 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-28T14:06:37.088 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:06:37.088 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-05-28T14:06:37.089 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-05-28T14:06:37.089 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-05-28T14:06:37.089 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-05-28T14:06:37.089 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:06:37.089 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-05-28T14:06:37.089 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-28T14:06:37.089 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-28T14:06:37.089 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-28T14:06:37.089 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-05-28T14:06:37.089 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-05-28T14:06:37.089 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-05-28T14:06:37.090 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-28T14:06:37.090 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:06:37.090 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-05-28T14:06:37.090 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-05-28T14:06:37.213 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.44218.asok 2024-05-28T14:06:37.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:06:37.225 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44218.asok 2024-05-28T14:06:37.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:37.225 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44218.asok status 2024-05-28T14:06:37.402 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:06:37.402 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-05-28T14:06:37.402 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-28T14:06:37.402 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-28T14:06:37.403 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-05-28T14:06:37.403 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-05-28T14:06:37.403 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-28T14:06:37.403 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "44218", 2024-05-28T14:06:37.403 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-05-28T14:06:37.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:37.403 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-05-28T14:06:37.403 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-05-28T14:06:37.403 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 7260, 2024-05-28T14:06:37.403 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-05-28T14:06:37.403 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-05-28T14:06:37.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-05-28T14:06:37.403 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 7260 2024-05-28T14:06:37.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:37.404 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T14:06:37.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T14:06:37.404 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T14:06:37.404 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3168018286 2024-05-28T14:06:37.404 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:37.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:37.404 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T14:06:37.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T14:06:37.404 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T14:06:37.404 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3168018286 2024-05-28T14:06:37.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:06:37.404 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.7260 192.168.0.1:0/3168018286", 2024-05-28T14:06:37.404 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/3168018286", 2024-05-28T14:06:37.404 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-05-28T14:06:37.405 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 46, 2024-05-28T14:06:37.405 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 74, 2024-05-28T14:06:37.405 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-05-28T14:06:37.405 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-05-28T14:06:37.405 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-05-28T14:06:37.405 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:06:37.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:37.418 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-05-28T14:06:37.418 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-28T14:06:37.427 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-05-28T14:06:37.427 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-28T14:06:37.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:37.427 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-28T14:06:37.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-28T14:06:37.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.690+0000 7fb9d7400640 1 Processor -- start 2024-05-28T14:06:37.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.690+0000 7fb9d7400640 1 -- start start 2024-05-28T14:06:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9d7400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb9d015b350 0x7fb9d006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9d0072d40 0x7fb9d00d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9d7400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d00ade40 con 0x7fb9d00737c0 2024-05-28T14:06:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9d7400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb9d00ae9f0 con 0x7fb9d015b350 2024-05-28T14:06:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9d7400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb9d00ae0c0 con 0x7fb9d0072d40 2024-05-28T14:06:37.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9d4a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb9d015b350 0x7fb9d006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:37.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9d0072d40 0x7fb9d00d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:37.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9d5400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fb9d00737c0 0x7fb9d00aee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:33224/0 (socket says 172.21.15.42:33224) 2024-05-28T14:06:37.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9d0072d40 0x7fb9d00d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:47654/0 (socket says 172.21.15.42:47654) 2024-05-28T14:06:37.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9d5400640 1 -- 172.21.15.42:0/2679520577 learned_addr learned my addr 172.21.15.42:0/2679520577 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:37.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9d4a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb9d015b350 0x7fb9d006dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:47908/0 (socket says 172.21.15.42:47908) 2024-05-28T14:06:37.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9d4a00640 1 -- 172.21.15.42:0/2679520577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9d0072d40 msgr2=0x7fb9d00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:37.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9d4a00640 1 --2- 172.21.15.42:0/2679520577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9d0072d40 0x7fb9d00d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:37.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9d4a00640 1 -- 172.21.15.42:0/2679520577 >> v1:172.21.15.63:6789/0 conn(0x7fb9d00737c0 legacy=0x7fb9d00aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:37.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9d4a00640 1 -- 172.21.15.42:0/2679520577 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d00ae3a0 con 0x7fb9d015b350 2024-05-28T14:06:37.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.694+0000 7fb9c7e00640 1 --2- 172.21.15.42:0/2679520577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9d0072d40 0x7fb9d00d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:06:37.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.698+0000 7fb9d4a00640 1 --2- 172.21.15.42:0/2679520577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9d015b350 0x7fb9d006dd40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009d90 tx=0x7fb9c001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=987a66a2b5e97c11 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:37.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.698+0000 7fb9cfe00640 1 -- 172.21.15.42:0/2679520577 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00b2070 con 0x7fb9d015b350 2024-05-28T14:06:37.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.698+0000 7fb9cfe00640 1 -- 172.21.15.42:0/2679520577 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9c00a57a0 con 0x7fb9d015b350 2024-05-28T14:06:37.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.698+0000 7fb9cfe00640 1 -- 172.21.15.42:0/2679520577 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00a5b70 con 0x7fb9d015b350 2024-05-28T14:06:37.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.698+0000 7fb9d7400640 1 -- 172.21.15.42:0/2679520577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9d015b350 msgr2=0x7fb9d006dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:37.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.698+0000 7fb9d7400640 1 --2- 172.21.15.42:0/2679520577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9d015b350 0x7fb9d006dd40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009d90 tx=0x7fb9c001c8e0 comp rx=0 tx=0).stop 2024-05-28T14:06:37.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.698+0000 7fb9d7400640 1 -- 172.21.15.42:0/2679520577 shutdown_connections 2024-05-28T14:06:37.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.698+0000 7fb9d7400640 1 --2- 172.21.15.42:0/2679520577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9d0072d40 0x7fb9d00d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:37.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.698+0000 7fb9d7400640 1 --2- 172.21.15.42:0/2679520577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9d015b350 0x7fb9d006dd40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.698+0000 7fb9d7400640 1 -- 172.21.15.42:0/2679520577 >> 172.21.15.42:0/2679520577 conn(0x7fb9d006bdf0 msgr2=0x7fb9d006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.698+0000 7fb9d7400640 1 -- 172.21.15.42:0/2679520577 shutdown_connections 2024-05-28T14:06:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.698+0000 7fb9d7400640 1 -- 172.21.15.42:0/2679520577 wait complete. 2024-05-28T14:06:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.698+0000 7fb9d7400640 1 Processor -- start 2024-05-28T14:06:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.698+0000 7fb9d7400640 1 -- start start 2024-05-28T14:06:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9d0072d40 0x7fb9d0147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9d00737c0 0x7fb9d00b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9d015b350 0x7fb9d0150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d7400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d00afb20 con 0x7fb9d0072d40 2024-05-28T14:06:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d7400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d00af9a0 con 0x7fb9d015b350 2024-05-28T14:06:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d7400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb9d0166af0 con 0x7fb9d00737c0 2024-05-28T14:06:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d4a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9d0072d40 0x7fb9d0147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:37.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d4a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9d0072d40 0x7fb9d0147890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39274/0 (socket says 172.21.15.42:39274) 2024-05-28T14:06:37.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d4a00640 1 -- 172.21.15.42:0/1338030683 learned_addr learned my addr 172.21.15.42:0/1338030683 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:37.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d5400640 1 --2- 172.21.15.42:0/1338030683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9d015b350 0x7fb9d0150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:37.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9c7e00640 1 --2- 172.21.15.42:0/1338030683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9d00737c0 0x7fb9d00b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:37.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d4a00640 1 -- 172.21.15.42:0/1338030683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9d00737c0 msgr2=0x7fb9d00b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:37.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d4a00640 1 --2- 172.21.15.42:0/1338030683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9d00737c0 0x7fb9d00b41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:37.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d4a00640 1 -- 172.21.15.42:0/1338030683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9d015b350 msgr2=0x7fb9d0150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:37.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d4a00640 1 --2- 172.21.15.42:0/1338030683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9d015b350 0x7fb9d0150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:37.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d4a00640 1 -- 172.21.15.42:0/1338030683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d00b48e0 con 0x7fb9d0072d40 2024-05-28T14:06:37.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d4a00640 1 --2- 172.21.15.42:0/1338030683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9d0072d40 0x7fb9d0147890 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb9c001a510 tx=0x7fb9c0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:37.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9cd600640 1 -- 172.21.15.42:0/1338030683 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00bd020 con 0x7fb9d0072d40 2024-05-28T14:06:37.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9cd600640 1 -- 172.21.15.42:0/1338030683 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9c00b8470 con 0x7fb9d0072d40 2024-05-28T14:06:37.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9cd600640 1 -- 172.21.15.42:0/1338030683 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00b2040 con 0x7fb9d0072d40 2024-05-28T14:06:37.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d7400640 1 -- 172.21.15.42:0/1338030683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d00b2060 con 0x7fb9d0072d40 2024-05-28T14:06:37.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d7400640 1 -- 172.21.15.42:0/1338030683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d00b2620 con 0x7fb9d0072d40 2024-05-28T14:06:37.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.702+0000 7fb9d7400640 1 -- 172.21.15.42:0/1338030683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9d0068a90 con 0x7fb9d0072d40 2024-05-28T14:06:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.706+0000 7fb9cd600640 1 -- 172.21.15.42:0/1338030683 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7fb9c00c0020 con 0x7fb9d0072d40 2024-05-28T14:06:37.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.706+0000 7fb9cd600640 1 --2- 172.21.15.42:0/1338030683 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb9ac069070 0x7fb9ac06b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:37.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.706+0000 7fb9cd600640 1 -- 172.21.15.42:0/1338030683 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7fb9c0120060 con 0x7fb9d0072d40 2024-05-28T14:06:37.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.706+0000 7fb9c7e00640 1 --2- 172.21.15.42:0/1338030683 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb9ac069070 0x7fb9ac06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:37.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.706+0000 7fb9c7e00640 1 --2- 172.21.15.42:0/1338030683 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb9ac069070 0x7fb9ac06b530 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009580 tx=0x7fb9b8009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:37.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.710+0000 7fb9cd600640 1 -- 172.21.15.42:0/1338030683 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb9c00c7030 con 0x7fb9d0072d40 2024-05-28T14:06:37.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.910+0000 7fb9d7400640 1 -- 172.21.15.42:0/1338030683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fb9d0072b20 con 0x7fb9d0072d40 2024-05-28T14:06:37.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.934+0000 7fb9cd600640 1 -- 172.21.15.42:0/1338030683 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) ==== 136+0+0 (secure 0 0 0) 0x7fb9c0016560 con 0x7fb9d0072d40 2024-05-28T14:06:37.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.938+0000 7fb9d7400640 1 -- 172.21.15.42:0/1338030683 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb9ac069070 msgr2=0x7fb9ac06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:37.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.938+0000 7fb9d7400640 1 --2- 172.21.15.42:0/1338030683 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb9ac069070 0x7fb9ac06b530 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009580 tx=0x7fb9b8009190 comp rx=0 tx=0).stop 2024-05-28T14:06:37.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.938+0000 7fb9d7400640 1 -- 172.21.15.42:0/1338030683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9d0072d40 msgr2=0x7fb9d0147890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:37.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.938+0000 7fb9d7400640 1 --2- 172.21.15.42:0/1338030683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9d0072d40 0x7fb9d0147890 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb9c001a510 tx=0x7fb9c0019680 comp rx=0 tx=0).stop 2024-05-28T14:06:37.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.938+0000 7fb9d7400640 1 -- 172.21.15.42:0/1338030683 shutdown_connections 2024-05-28T14:06:37.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.938+0000 7fb9d7400640 1 --2- 172.21.15.42:0/1338030683 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb9ac069070 0x7fb9ac06b530 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:37.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.938+0000 7fb9d7400640 1 --2- 172.21.15.42:0/1338030683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9d015b350 0x7fb9d0150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:37.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.938+0000 7fb9d7400640 1 --2- 172.21.15.42:0/1338030683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9d00737c0 0x7fb9d00b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:37.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.938+0000 7fb9d7400640 1 --2- 172.21.15.42:0/1338030683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9d0072d40 0x7fb9d0147890 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:37.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.938+0000 7fb9d7400640 1 -- 172.21.15.42:0/1338030683 >> 172.21.15.42:0/1338030683 conn(0x7fb9d006bdf0 msgr2=0x7fb9d00b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:37.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.938+0000 7fb9d7400640 1 -- 172.21.15.42:0/1338030683 shutdown_connections 2024-05-28T14:06:37.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:37.938+0000 7fb9d7400640 1 -- 172.21.15.42:0/1338030683 wait complete. 2024-05-28T14:06:37.943 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:06:38.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.098+0000 7f6328e00640 1 Processor -- start 2024-05-28T14:06:38.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.098+0000 7f6328e00640 1 -- start start 2024-05-28T14:06:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.098+0000 7f6328e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f632415bf30 0x7f632415c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.098+0000 7f6328e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f63240cf690 0x7f632406ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.098+0000 7f6328e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63240cfc10 con 0x7f63240cf230 2024-05-28T14:06:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.098+0000 7f6328e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f63240721d0 con 0x7f632415bf30 2024-05-28T14:06:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.098+0000 7f6328e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f63240cfef0 con 0x7f63240cf690 2024-05-28T14:06:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.102+0000 7f6321e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f632415bf30 0x7f632415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.102+0000 7f631be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f63240cf230 0x7f6324072660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55274/0 (socket says 172.21.15.42:55274) 2024-05-28T14:06:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.102+0000 7f631be00640 1 -- 172.21.15.42:0/3675472084 learned_addr learned my addr 172.21.15.42:0/3675472084 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:38.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.102+0000 7f6322800640 1 --2- 172.21.15.42:0/3675472084 >> v2:172.21.15.63:3300/0 conn(0x7f63240cf690 0x7f632406ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.102+0000 7f6321e00640 1 -- 172.21.15.42:0/3675472084 >> v2:172.21.15.63:3300/0 conn(0x7f63240cf690 msgr2=0x7f632406ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:38.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.102+0000 7f6321e00640 1 --2- 172.21.15.42:0/3675472084 >> v2:172.21.15.63:3300/0 conn(0x7f63240cf690 0x7f632406ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.102+0000 7f6321e00640 1 -- 172.21.15.42:0/3675472084 >> v1:172.21.15.42:6789/0 conn(0x7f63240cf230 legacy=0x7f6324072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.102+0000 7f6321e00640 1 -- 172.21.15.42:0/3675472084 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63240d01d0 con 0x7f632415bf30 2024-05-28T14:06:38.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.102+0000 7f6322800640 1 --2- 172.21.15.42:0/3675472084 >> v2:172.21.15.63:3300/0 conn(0x7f63240cf690 0x7f632406ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:38.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.102+0000 7f6321400640 1 -- 172.21.15.42:0/3675472084 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3442792022 0 0) 0x7f63240cfc10 con 0x7f63240cf230 2024-05-28T14:06:38.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.102+0000 7f6321e00640 1 --2- 172.21.15.42:0/3675472084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f632415bf30 0x7f632415c310 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f632415ef80 tx=0x7f631401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea42b36a591b1eb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:38.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.102+0000 7f6321400640 1 -- 172.21.15.42:0/3675472084 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63140b0070 con 0x7f632415bf30 2024-05-28T14:06:38.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.102+0000 7f6321400640 1 -- 172.21.15.42:0/3675472084 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63140ab600 con 0x7f632415bf30 2024-05-28T14:06:38.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.102+0000 7f6321400640 1 -- 172.21.15.42:0/3675472084 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63140ab9d0 con 0x7f632415bf30 2024-05-28T14:06:38.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 -- 172.21.15.42:0/3675472084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f632415bf30 msgr2=0x7f632415c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 --2- 172.21.15.42:0/3675472084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f632415bf30 0x7f632415c310 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f632415ef80 tx=0x7f631401c3c0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 -- 172.21.15.42:0/3675472084 shutdown_connections 2024-05-28T14:06:38.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 --2- 172.21.15.42:0/3675472084 >> v2:172.21.15.63:3300/0 conn(0x7f63240cf690 0x7f632406ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 --2- 172.21.15.42:0/3675472084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f632415bf30 0x7f632415c310 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 -- 172.21.15.42:0/3675472084 >> 172.21.15.42:0/3675472084 conn(0x7f632406b9f0 msgr2=0x7f632406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:38.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 -- 172.21.15.42:0/3675472084 shutdown_connections 2024-05-28T14:06:38.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 -- 172.21.15.42:0/3675472084 wait complete. 2024-05-28T14:06:38.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 Processor -- start 2024-05-28T14:06:38.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 -- start start 2024-05-28T14:06:38.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63240cf230 0x7f6324153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63240cf690 0x7f6324148020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f632415bf30 0x7f6324156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6324168330 con 0x7f63240cf230 2024-05-28T14:06:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f63241681b0 con 0x7f63240cf690 2024-05-28T14:06:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6328e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f63241684b0 con 0x7f632415bf30 2024-05-28T14:06:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6321e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63240cf230 0x7f6324153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6321e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63240cf230 0x7f6324153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39294/0 (socket says 172.21.15.42:39294) 2024-05-28T14:06:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6321e00640 1 -- 172.21.15.42:0/1966215411 learned_addr learned my addr 172.21.15.42:0/1966215411 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f631be00640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63240cf690 0x7f6324148020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6321e00640 1 -- 172.21.15.42:0/1966215411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f632415bf30 msgr2=0x7f6324156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6322800640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f632415bf30 0x7f6324156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6321e00640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f632415bf30 0x7f6324156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6321e00640 1 -- 172.21.15.42:0/1966215411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63240cf690 msgr2=0x7f6324148020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6321e00640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63240cf690 0x7f6324148020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6321e00640 1 -- 172.21.15.42:0/1966215411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63241544d0 con 0x7f63240cf230 2024-05-28T14:06:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f631be00640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63240cf690 0x7f6324148020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.106+0000 7f6322800640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f632415bf30 0x7f6324156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.110+0000 7f6321e00640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63240cf230 0x7f6324153f90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f63140ae040 tx=0x7f6314019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.110+0000 7f631a000640 1 -- 172.21.15.42:0/1966215411 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63140b8070 con 0x7f63240cf230 2024-05-28T14:06:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.110+0000 7f631a000640 1 -- 172.21.15.42:0/1966215411 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63140b4410 con 0x7f63240cf230 2024-05-28T14:06:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.110+0000 7f631a000640 1 -- 172.21.15.42:0/1966215411 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63140b0040 con 0x7f63240cf230 2024-05-28T14:06:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.110+0000 7f6328e00640 1 -- 172.21.15.42:0/1966215411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632415a3e0 con 0x7f63240cf230 2024-05-28T14:06:38.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.110+0000 7f6328e00640 1 -- 172.21.15.42:0/1966215411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f632415a9a0 con 0x7f63240cf230 2024-05-28T14:06:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.110+0000 7f6328e00640 1 -- 172.21.15.42:0/1966215411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f632415af90 con 0x7f63240cf230 2024-05-28T14:06:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.110+0000 7f631a000640 1 -- 172.21.15.42:0/1966215411 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f63140bb070 con 0x7f63240cf230 2024-05-28T14:06:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.110+0000 7f631a000640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6300069070 0x7f630006b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.110+0000 7f631a000640 1 -- 172.21.15.42:0/1966215411 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7f631411f1e0 con 0x7f63240cf230 2024-05-28T14:06:38.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.110+0000 7f631be00640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6300069070 0x7f630006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.114+0000 7f631be00640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6300069070 0x7f630006b530 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f632414c350 tx=0x7f630c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:38.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.114+0000 7f631a000640 1 -- 172.21.15.42:0/1966215411 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f63241544d0 con 0x7f63240cf230 2024-05-28T14:06:38.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.302+0000 7f6328e00640 1 -- 172.21.15.42:0/1966215411 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f6324073890 con 0x7f6300069070 2024-05-28T14:06:38.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.302+0000 7f631a000640 1 -- 172.21.15.42:0/1966215411 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f6324073890 con 0x7f6300069070 2024-05-28T14:06:38.290 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:06:38.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:06:38.290 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:06:38.290 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:06:38.290 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:06:38.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.306+0000 7f6328e00640 1 -- 172.21.15.42:0/1966215411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6300069070 msgr2=0x7f630006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.306+0000 7f6328e00640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6300069070 0x7f630006b530 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f632414c350 tx=0x7f630c009040 comp rx=0 tx=0).stop 2024-05-28T14:06:38.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.306+0000 7f6328e00640 1 -- 172.21.15.42:0/1966215411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63240cf230 msgr2=0x7f6324153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.306+0000 7f6328e00640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63240cf230 0x7f6324153f90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f63140ae040 tx=0x7f6314019160 comp rx=0 tx=0).stop 2024-05-28T14:06:38.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.306+0000 7f6328e00640 1 -- 172.21.15.42:0/1966215411 shutdown_connections 2024-05-28T14:06:38.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.306+0000 7f6328e00640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6300069070 0x7f630006b530 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.306+0000 7f6328e00640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f632415bf30 0x7f6324156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.306+0000 7f6328e00640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63240cf690 0x7f6324148020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.306+0000 7f6328e00640 1 --2- 172.21.15.42:0/1966215411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63240cf230 0x7f6324153f90 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.306+0000 7f6328e00640 1 -- 172.21.15.42:0/1966215411 >> 172.21.15.42:0/1966215411 conn(0x7f632406b9f0 msgr2=0x7f632414f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:38.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.306+0000 7f6328e00640 1 -- 172.21.15.42:0/1966215411 shutdown_connections 2024-05-28T14:06:38.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.306+0000 7f6328e00640 1 -- 172.21.15.42:0/1966215411 wait complete. 2024-05-28T14:06:38.312 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T14:06:38.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c3a600640 1 Processor -- start 2024-05-28T14:06:38.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c3a600640 1 -- start start 2024-05-28T14:06:38.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c3a600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7c3409b160 0x7f7c3406cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c3a600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c3409b8e0 0x7f7c3415e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c3a600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c340d0770 con 0x7f7c340b0e80 2024-05-28T14:06:38.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c3a600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f7c340d0a10 con 0x7f7c3409b160 2024-05-28T14:06:38.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c3a600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7c340d0cf0 con 0x7f7c3409b8e0 2024-05-28T14:06:38.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c32a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7c3409b160 0x7f7c3406cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c33400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f7c340b0e80 0x7f7c3409aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55276/0 (socket says 172.21.15.42:55276) 2024-05-28T14:06:38.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c33400640 1 -- 172.21.15.42:0/4125084612 learned_addr learned my addr 172.21.15.42:0/4125084612 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:38.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c33e00640 1 --2- 172.21.15.42:0/4125084612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c3409b8e0 0x7f7c3415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c32a00640 1 -- 172.21.15.42:0/4125084612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c3409b8e0 msgr2=0x7f7c3415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:38.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c32a00640 1 --2- 172.21.15.42:0/4125084612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c3409b8e0 0x7f7c3415e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c32a00640 1 -- 172.21.15.42:0/4125084612 >> v1:172.21.15.42:6789/0 conn(0x7f7c340b0e80 legacy=0x7f7c3409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c32a00640 1 -- 172.21.15.42:0/4125084612 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c340d0fd0 con 0x7f7c3409b160 2024-05-28T14:06:38.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.470+0000 7f7c32000640 1 -- 172.21.15.42:0/4125084612 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2848264543 0 0) 0x7f7c340d0770 con 0x7f7c340b0e80 2024-05-28T14:06:38.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.474+0000 7f7c32a00640 1 --2- 172.21.15.42:0/4125084612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3409b160 0x7f7c3406cb00 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7c200097c0 tx=0x7f7c2001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da0ac8aa8be15c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:38.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.474+0000 7f7c32000640 1 -- 172.21.15.42:0/4125084612 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c200b0070 con 0x7f7c3409b160 2024-05-28T14:06:38.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.474+0000 7f7c32000640 1 -- 172.21.15.42:0/4125084612 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c200ab600 con 0x7f7c3409b160 2024-05-28T14:06:38.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.474+0000 7f7c32000640 1 -- 172.21.15.42:0/4125084612 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c200ab9d0 con 0x7f7c3409b160 2024-05-28T14:06:38.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.474+0000 7f7c3a600640 1 -- 172.21.15.42:0/4125084612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3409b160 msgr2=0x7f7c3406cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.474+0000 7f7c3a600640 1 --2- 172.21.15.42:0/4125084612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3409b160 0x7f7c3406cb00 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7c200097c0 tx=0x7f7c2001c4a0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.474+0000 7f7c3a600640 1 -- 172.21.15.42:0/4125084612 shutdown_connections 2024-05-28T14:06:38.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.474+0000 7f7c3a600640 1 --2- 172.21.15.42:0/4125084612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c3409b8e0 0x7f7c3415e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.474+0000 7f7c3a600640 1 --2- 172.21.15.42:0/4125084612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3409b160 0x7f7c3406cb00 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.474+0000 7f7c3a600640 1 -- 172.21.15.42:0/4125084612 >> 172.21.15.42:0/4125084612 conn(0x7f7c340b3f90 msgr2=0x7f7c340b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:38.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.474+0000 7f7c3a600640 1 -- 172.21.15.42:0/4125084612 shutdown_connections 2024-05-28T14:06:38.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.474+0000 7f7c3a600640 1 -- 172.21.15.42:0/4125084612 wait complete. 2024-05-28T14:06:38.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c3a600640 1 Processor -- start 2024-05-28T14:06:38.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c3a600640 1 -- start start 2024-05-28T14:06:38.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c3a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c3409b160 0x7f7c34148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c3a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3409b8e0 0x7f7c34160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c3a600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c340b0e80 0x7f7c340b6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c3a600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7c340af350 con 0x7f7c3409b8e0 2024-05-28T14:06:38.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c3a600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7c340af1d0 con 0x7f7c3409b160 2024-05-28T14:06:38.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c3a600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7c340af4d0 con 0x7f7c340b0e80 2024-05-28T14:06:38.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c32a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3409b8e0 0x7f7c34160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c32a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3409b8e0 0x7f7c34160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39312/0 (socket says 172.21.15.42:39312) 2024-05-28T14:06:38.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c32a00640 1 -- 172.21.15.42:0/1212366307 learned_addr learned my addr 172.21.15.42:0/1212366307 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c33400640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c3409b160 0x7f7c34148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c33e00640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c340b0e80 0x7f7c340b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c32a00640 1 -- 172.21.15.42:0/1212366307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c340b0e80 msgr2=0x7f7c340b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c32a00640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c340b0e80 0x7f7c340b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c32a00640 1 -- 172.21.15.42:0/1212366307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c3409b160 msgr2=0x7f7c34148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c32a00640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c3409b160 0x7f7c34148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c32a00640 1 -- 172.21.15.42:0/1212366307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3415c180 con 0x7f7c3409b8e0 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c33e00640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c340b0e80 0x7f7c340b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c33400640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c3409b160 0x7f7c34148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c32a00640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3409b8e0 0x7f7c34160ce0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7c200097c0 tx=0x7f7c2001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c2b400640 1 -- 172.21.15.42:0/1212366307 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c200b0050 con 0x7f7c3409b8e0 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c2b400640 1 -- 172.21.15.42:0/1212366307 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c200b1040 con 0x7f7c3409b8e0 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c3a600640 1 -- 172.21.15.42:0/1212366307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3415c460 con 0x7f7c3409b8e0 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c2b400640 1 -- 172.21.15.42:0/1212366307 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20006e20 con 0x7f7c3409b8e0 2024-05-28T14:06:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c3a600640 1 -- 172.21.15.42:0/1212366307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c3415ca20 con 0x7f7c3409b8e0 2024-05-28T14:06:38.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.478+0000 7f7c3a600640 1 -- 172.21.15.42:0/1212366307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c340a0550 con 0x7f7c3409b8e0 2024-05-28T14:06:38.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.482+0000 7f7c2b400640 1 -- 172.21.15.42:0/1212366307 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7f7c200be060 con 0x7f7c3409b8e0 2024-05-28T14:06:38.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.482+0000 7f7c2b400640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c140690a0 0x7f7c1406b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.482+0000 7f7c2b400640 1 -- 172.21.15.42:0/1212366307 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7f7c2011ecc0 con 0x7f7c3409b8e0 2024-05-28T14:06:38.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.482+0000 7f7c33400640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c140690a0 0x7f7c1406b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.482+0000 7f7c33400640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c140690a0 0x7f7c1406b560 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7c24007fd0 tx=0x7f7c24009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:38.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.486+0000 7f7c2b400640 1 -- 172.21.15.42:0/1212366307 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7c20016090 con 0x7f7c3409b8e0 2024-05-28T14:06:38.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.650+0000 7f7c3a600640 1 -- 172.21.15.42:0/1212366307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7c340b16a0 con 0x7f7c3409b8e0 2024-05-28T14:06:38.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.650+0000 7f7c2b400640 1 -- 172.21.15.42:0/1212366307 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f7c200f0cd0 con 0x7f7c3409b8e0 2024-05-28T14:06:38.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.654+0000 7f7c3a600640 1 -- 172.21.15.42:0/1212366307 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c140690a0 msgr2=0x7f7c1406b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.654+0000 7f7c3a600640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c140690a0 0x7f7c1406b560 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7c24007fd0 tx=0x7f7c24009040 comp rx=0 tx=0).stop 2024-05-28T14:06:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.654+0000 7f7c3a600640 1 -- 172.21.15.42:0/1212366307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3409b8e0 msgr2=0x7f7c34160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.654+0000 7f7c3a600640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3409b8e0 0x7f7c34160ce0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7c200097c0 tx=0x7f7c2001ace0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.654+0000 7f7c3a600640 1 -- 172.21.15.42:0/1212366307 shutdown_connections 2024-05-28T14:06:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.654+0000 7f7c3a600640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7c140690a0 0x7f7c1406b560 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.654+0000 7f7c3a600640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c340b0e80 0x7f7c340b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.654+0000 7f7c3a600640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3409b8e0 0x7f7c34160ce0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.654+0000 7f7c3a600640 1 --2- 172.21.15.42:0/1212366307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c3409b160 0x7f7c34148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.654+0000 7f7c3a600640 1 -- 172.21.15.42:0/1212366307 >> 172.21.15.42:0/1212366307 conn(0x7f7c340b3f90 msgr2=0x7f7c3406ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.654+0000 7f7c3a600640 1 -- 172.21.15.42:0/1212366307 shutdown_connections 2024-05-28T14:06:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.654+0000 7f7c3a600640 1 -- 172.21.15.42:0/1212366307 wait complete. 2024-05-28T14:06:38.659 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-05-28T14:06:38.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb97200640 1 Processor -- start 2024-05-28T14:06:38.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb97200640 1 -- start start 2024-05-28T14:06:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb97200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffb9015e8c0 0x7ffb900b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb97200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffb900b0650 0x7ffb9006b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb97200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb9006bf30 0x7ffb900cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb97200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ffb900ae420 con 0x7ffb900b0650 2024-05-28T14:06:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb97200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffb900b4440 con 0x7ffb9015e8c0 2024-05-28T14:06:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb97200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffb900b0f10 con 0x7ffb9006bf30 2024-05-28T14:06:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb8f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffb900b0650 0x7ffb9006b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb8f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffb900b0650 0x7ffb9006b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39320/0 (socket says 172.21.15.42:39320) 2024-05-28T14:06:38.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb8f400640 1 -- 172.21.15.42:0/4158079970 learned_addr learned my addr 172.21.15.42:0/4158079970 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:38.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb8fe00640 1 --2- 172.21.15.42:0/4158079970 >> v2:172.21.15.63:3300/0 conn(0x7ffb9015e8c0 0x7ffb900b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb95200640 1 --2- 172.21.15.42:0/4158079970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb9006bf30 0x7ffb900cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb8f400640 1 -- 172.21.15.42:0/4158079970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb9006bf30 msgr2=0x7ffb900cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:38.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb8f400640 1 --2- 172.21.15.42:0/4158079970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb9006bf30 0x7ffb900cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb8f400640 1 -- 172.21.15.42:0/4158079970 >> v2:172.21.15.63:3300/0 conn(0x7ffb9015e8c0 msgr2=0x7ffb900b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:38.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb8f400640 1 --2- 172.21.15.42:0/4158079970 >> v2:172.21.15.63:3300/0 conn(0x7ffb9015e8c0 0x7ffb900b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.818+0000 7ffb8f400640 1 -- 172.21.15.42:0/4158079970 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb900ae670 con 0x7ffb900b0650 2024-05-28T14:06:38.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.822+0000 7ffb8f400640 1 --2- 172.21.15.42:0/4158079970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb900b0650 0x7ffb9006b9f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ffb900b41b0 tx=0x7ffb7c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9520af9e555343d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:38.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.822+0000 7ffb8ea00640 1 -- 172.21.15.42:0/4158079970 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0b0070 con 0x7ffb900b0650 2024-05-28T14:06:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.822+0000 7ffb8ea00640 1 -- 172.21.15.42:0/4158079970 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c0ab600 con 0x7ffb900b0650 2024-05-28T14:06:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.822+0000 7ffb8ea00640 1 -- 172.21.15.42:0/4158079970 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0ab9d0 con 0x7ffb900b0650 2024-05-28T14:06:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.822+0000 7ffb97200640 1 -- 172.21.15.42:0/4158079970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb900b0650 msgr2=0x7ffb9006b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.822+0000 7ffb97200640 1 --2- 172.21.15.42:0/4158079970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb900b0650 0x7ffb9006b9f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ffb900b41b0 tx=0x7ffb7c01c490 comp rx=0 tx=0).stop 2024-05-28T14:06:38.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.822+0000 7ffb97200640 1 -- 172.21.15.42:0/4158079970 shutdown_connections 2024-05-28T14:06:38.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.822+0000 7ffb97200640 1 --2- 172.21.15.42:0/4158079970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb9006bf30 0x7ffb900cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.822+0000 7ffb97200640 1 --2- 172.21.15.42:0/4158079970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb900b0650 0x7ffb9006b9f0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.822+0000 7ffb97200640 1 --2- 172.21.15.42:0/4158079970 >> v2:172.21.15.63:3300/0 conn(0x7ffb9015e8c0 0x7ffb900b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.822+0000 7ffb97200640 1 -- 172.21.15.42:0/4158079970 >> 172.21.15.42:0/4158079970 conn(0x7ffb900b6960 msgr2=0x7ffb900b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:38.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.822+0000 7ffb97200640 1 -- 172.21.15.42:0/4158079970 shutdown_connections 2024-05-28T14:06:38.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.822+0000 7ffb97200640 1 -- 172.21.15.42:0/4158079970 wait complete. 2024-05-28T14:06:38.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb97200640 1 Processor -- start 2024-05-28T14:06:38.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb97200640 1 -- start start 2024-05-28T14:06:38.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb97200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb9006bf30 0x7ffb901574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb97200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb900b0650 0x7ffb90147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb97200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9015e8c0 0x7ffb9006ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb97200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffb900b0c40 con 0x7ffb9006bf30 2024-05-28T14:06:38.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb97200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffb900b0ac0 con 0x7ffb9015e8c0 2024-05-28T14:06:38.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb97200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffb900af150 con 0x7ffb900b0650 2024-05-28T14:06:38.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb8fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb9006bf30 0x7ffb901574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb8fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb9006bf30 0x7ffb901574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39334/0 (socket says 172.21.15.42:39334) 2024-05-28T14:06:38.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb8fe00640 1 -- 172.21.15.42:0/2778065539 learned_addr learned my addr 172.21.15.42:0/2778065539 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:38.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb95200640 1 --2- 172.21.15.42:0/2778065539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9015e8c0 0x7ffb9006ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb8f400640 1 --2- 172.21.15.42:0/2778065539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb900b0650 0x7ffb90147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb8fe00640 1 -- 172.21.15.42:0/2778065539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb900b0650 msgr2=0x7ffb90147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb8fe00640 1 --2- 172.21.15.42:0/2778065539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb900b0650 0x7ffb90147e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb8fe00640 1 -- 172.21.15.42:0/2778065539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9015e8c0 msgr2=0x7ffb9006ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb8fe00640 1 --2- 172.21.15.42:0/2778065539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9015e8c0 0x7ffb9006ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb8fe00640 1 -- 172.21.15.42:0/2778065539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb90159730 con 0x7ffb9006bf30 2024-05-28T14:06:38.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb8fe00640 1 --2- 172.21.15.42:0/2778065539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb9006bf30 0x7ffb901574e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000d5f0 tx=0x7ffb8000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:38.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb87e00640 1 -- 172.21.15.42:0/2778065539 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb800125f0 con 0x7ffb9006bf30 2024-05-28T14:06:38.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb87e00640 1 -- 172.21.15.42:0/2778065539 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb80019070 con 0x7ffb9006bf30 2024-05-28T14:06:38.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb97200640 1 -- 172.21.15.42:0/2778065539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb90157c30 con 0x7ffb9006bf30 2024-05-28T14:06:38.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb87e00640 1 -- 172.21.15.42:0/2778065539 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb800129b0 con 0x7ffb9006bf30 2024-05-28T14:06:38.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb97200640 1 -- 172.21.15.42:0/2778065539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb901581f0 con 0x7ffb9006bf30 2024-05-28T14:06:38.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.826+0000 7ffb97200640 1 -- 172.21.15.42:0/2778065539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb9009f7a0 con 0x7ffb9006bf30 2024-05-28T14:06:38.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.830+0000 7ffb87e00640 1 -- 172.21.15.42:0/2778065539 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7ffb80012b50 con 0x7ffb9006bf30 2024-05-28T14:06:38.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.830+0000 7ffb87e00640 1 --2- 172.21.15.42:0/2778065539 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb700690a0 0x7ffb7006b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.830+0000 7ffb87e00640 1 -- 172.21.15.42:0/2778065539 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7ffb80085940 con 0x7ffb9006bf30 2024-05-28T14:06:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.830+0000 7ffb8f400640 1 --2- 172.21.15.42:0/2778065539 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb700690a0 0x7ffb7006b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.830+0000 7ffb8f400640 1 --2- 172.21.15.42:0/2778065539 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb700690a0 0x7ffb7006b560 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ffb90068290 tx=0x7ffb7c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:38.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.834+0000 7ffb87e00640 1 -- 172.21.15.42:0/2778065539 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffb80011340 con 0x7ffb9006bf30 2024-05-28T14:06:38.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:38.998+0000 7ffb97200640 1 -- 172.21.15.42:0/2778065539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7ffb9004fdd0 con 0x7ffb9006bf30 2024-05-28T14:06:38.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.002+0000 7ffb87e00640 1 -- 172.21.15.42:0/2778065539 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v19)=0 v19) ==== 136+0+0 (secure 0 0 0) 0x7ffb80057720 con 0x7ffb9006bf30 2024-05-28T14:06:38.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.006+0000 7ffb97200640 1 -- 172.21.15.42:0/2778065539 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb700690a0 msgr2=0x7ffb7006b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.006+0000 7ffb97200640 1 --2- 172.21.15.42:0/2778065539 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb700690a0 0x7ffb7006b560 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ffb90068290 tx=0x7ffb7c008040 comp rx=0 tx=0).stop 2024-05-28T14:06:38.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.006+0000 7ffb97200640 1 -- 172.21.15.42:0/2778065539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb9006bf30 msgr2=0x7ffb901574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:38.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.006+0000 7ffb97200640 1 --2- 172.21.15.42:0/2778065539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb9006bf30 0x7ffb901574e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000d5f0 tx=0x7ffb8000dab0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.006+0000 7ffb97200640 1 -- 172.21.15.42:0/2778065539 shutdown_connections 2024-05-28T14:06:38.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.006+0000 7ffb97200640 1 --2- 172.21.15.42:0/2778065539 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb700690a0 0x7ffb7006b560 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.006+0000 7ffb97200640 1 --2- 172.21.15.42:0/2778065539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9015e8c0 0x7ffb9006ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.006+0000 7ffb97200640 1 --2- 172.21.15.42:0/2778065539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb900b0650 0x7ffb90147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.006+0000 7ffb97200640 1 --2- 172.21.15.42:0/2778065539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb9006bf30 0x7ffb901574e0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:38.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.006+0000 7ffb97200640 1 -- 172.21.15.42:0/2778065539 >> 172.21.15.42:0/2778065539 conn(0x7ffb900b6960 msgr2=0x7ffb9015d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:38.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.006+0000 7ffb97200640 1 -- 172.21.15.42:0/2778065539 shutdown_connections 2024-05-28T14:06:38.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.006+0000 7ffb97200640 1 -- 172.21.15.42:0/2778065539 wait complete. 2024-05-28T14:06:39.003 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:39.018+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:39.003 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:39.018+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:39.003 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:39.018+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:39.003 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:39.018+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:39.004 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:39.018+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:39.015 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-28T14:06:39.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7dc800640 1 Processor -- start 2024-05-28T14:06:39.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7dc800640 1 -- start start 2024-05-28T14:06:39.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7dc800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7d40d0e00 0x7ff7d406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:39.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7dc800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff7d4072d40 0x7ff7d40aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:39.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7dc800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7d40ade40 con 0x7ff7d415b350 2024-05-28T14:06:39.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7dc800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff7d40ae9f0 con 0x7ff7d4072d40 2024-05-28T14:06:39.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7dc800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d40ae0c0 con 0x7ff7d40d0e00 2024-05-28T14:06:39.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7d9e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff7d415b350 0x7ff7d40d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55290/0 (socket says 172.21.15.42:55290) 2024-05-28T14:06:39.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7d9e00640 1 -- 172.21.15.42:0/2577388565 learned_addr learned my addr 172.21.15.42:0/2577388565 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:06:39.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7da800640 1 --2- 172.21.15.42:0/2577388565 >> v2:172.21.15.42:3300/0 conn(0x7ff7d4072d40 0x7ff7d40aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:39.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7d9400640 1 --2- 172.21.15.42:0/2577388565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7d40d0e00 0x7ff7d406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:39.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7da800640 1 -- 172.21.15.42:0/2577388565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7d40d0e00 msgr2=0x7ff7d406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:39.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7da800640 1 --2- 172.21.15.42:0/2577388565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7d40d0e00 0x7ff7d406dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7da800640 1 -- 172.21.15.42:0/2577388565 >> v1:172.21.15.42:6789/0 conn(0x7ff7d415b350 legacy=0x7ff7d40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:39.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7da800640 1 -- 172.21.15.42:0/2577388565 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d40ae3a0 con 0x7ff7d4072d40 2024-05-28T14:06:39.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.174+0000 7ff7d9400640 1 --2- 172.21.15.42:0/2577388565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7d40d0e00 0x7ff7d406dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.178+0000 7ff7da800640 1 --2- 172.21.15.42:0/2577388565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7d4072d40 0x7ff7d40aee80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0009d90 tx=0x7ff7d001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d240a623dbd82a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.178+0000 7ff7d8a00640 1 -- 172.21.15.42:0/2577388565 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d00b2070 con 0x7ff7d4072d40 2024-05-28T14:06:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.178+0000 7ff7d8a00640 1 -- 172.21.15.42:0/2577388565 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7d00a5820 con 0x7ff7d4072d40 2024-05-28T14:06:39.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.178+0000 7ff7d8a00640 1 -- 172.21.15.42:0/2577388565 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d00a5bf0 con 0x7ff7d4072d40 2024-05-28T14:06:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.178+0000 7ff7dc800640 1 -- 172.21.15.42:0/2577388565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7d4072d40 msgr2=0x7ff7d40aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.178+0000 7ff7dc800640 1 --2- 172.21.15.42:0/2577388565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7d4072d40 0x7ff7d40aee80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0009d90 tx=0x7ff7d001c8e0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.178+0000 7ff7dc800640 1 -- 172.21.15.42:0/2577388565 shutdown_connections 2024-05-28T14:06:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.178+0000 7ff7dc800640 1 --2- 172.21.15.42:0/2577388565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7d4072d40 0x7ff7d40aee80 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.178+0000 7ff7dc800640 1 --2- 172.21.15.42:0/2577388565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7d40d0e00 0x7ff7d406dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.178+0000 7ff7dc800640 1 -- 172.21.15.42:0/2577388565 >> 172.21.15.42:0/2577388565 conn(0x7ff7d406bdf0 msgr2=0x7ff7d406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.178+0000 7ff7dc800640 1 -- 172.21.15.42:0/2577388565 shutdown_connections 2024-05-28T14:06:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.178+0000 7ff7dc800640 1 -- 172.21.15.42:0/2577388565 wait complete. 2024-05-28T14:06:39.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.178+0000 7ff7dc800640 1 Processor -- start 2024-05-28T14:06:39.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7dc800640 1 -- start start 2024-05-28T14:06:39.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7dc800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d4072d40 0x7ff7d4151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:39.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7dc800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7d40d0e00 0x7ff7d4147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:39.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7dc800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7d415b350 0x7ff7d414d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:39.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7dc800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d4166460 con 0x7ff7d415b350 2024-05-28T14:06:39.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7dc800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d41662e0 con 0x7ff7d4072d40 2024-05-28T14:06:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7dc800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d41665e0 con 0x7ff7d40d0e00 2024-05-28T14:06:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7da800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7d415b350 0x7ff7d414d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7da800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7d415b350 0x7ff7d414d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39368/0 (socket says 172.21.15.42:39368) 2024-05-28T14:06:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7da800640 1 -- 172.21.15.42:0/1119389996 learned_addr learned my addr 172.21.15.42:0/1119389996 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7d9e00640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d4072d40 0x7ff7d4151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7d9400640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7d40d0e00 0x7ff7d4147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7da800640 1 -- 172.21.15.42:0/1119389996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7d40d0e00 msgr2=0x7ff7d4147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7da800640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7d40d0e00 0x7ff7d4147cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7da800640 1 -- 172.21.15.42:0/1119389996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d4072d40 msgr2=0x7ff7d4151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7da800640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d4072d40 0x7ff7d4151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7da800640 1 -- 172.21.15.42:0/1119389996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d4148200 con 0x7ff7d415b350 2024-05-28T14:06:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7d9e00640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d4072d40 0x7ff7d4151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:06:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7da800640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7d415b350 0x7ff7d414d480 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff7d001af90 tx=0x7ff7d0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7ce000640 1 -- 172.21.15.42:0/1119389996 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d00b2050 con 0x7ff7d415b350 2024-05-28T14:06:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7ce000640 1 -- 172.21.15.42:0/1119389996 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7d00be070 con 0x7ff7d415b350 2024-05-28T14:06:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7dc800640 1 -- 172.21.15.42:0/1119389996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d414e790 con 0x7ff7d415b350 2024-05-28T14:06:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7ce000640 1 -- 172.21.15.42:0/1119389996 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d00ada50 con 0x7ff7d415b350 2024-05-28T14:06:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7dc800640 1 -- 172.21.15.42:0/1119389996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d414cc80 con 0x7ff7d415b350 2024-05-28T14:06:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.182+0000 7ff7dc800640 1 -- 172.21.15.42:0/1119389996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d4068a90 con 0x7ff7d415b350 2024-05-28T14:06:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.186+0000 7ff7ce000640 1 -- 172.21.15.42:0/1119389996 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7ff7d00a56a0 con 0x7ff7d415b350 2024-05-28T14:06:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.186+0000 7ff7ce000640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff7b8069070 0x7ff7b806b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.186+0000 7ff7ce000640 1 -- 172.21.15.42:0/1119389996 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7ff7d01207c0 con 0x7ff7d415b350 2024-05-28T14:06:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.186+0000 7ff7d9e00640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff7b8069070 0x7ff7b806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:39.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.186+0000 7ff7d9e00640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff7b8069070 0x7ff7b806b530 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8000f70 tx=0x7ff7c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:39.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.190+0000 7ff7ce000640 1 -- 172.21.15.42:0/1119389996 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff7d0016560 con 0x7ff7d415b350 2024-05-28T14:06:39.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.378+0000 7ff7dc800640 1 -- 172.21.15.42:0/1119389996 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7ff7d414ea70 con 0x7ff7b8069070 2024-05-28T14:06:39.370 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:39.382+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:39.370 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:39.382+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:39.370 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:39.382+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:39.370 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:39.382+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:39.370 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:06:39.382+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:06:39.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.418+0000 7ff7ce000640 1 -- 172.21.15.42:0/1119389996 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff7d414ea70 con 0x7ff7b8069070 2024-05-28T14:06:39.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.422+0000 7ff7dc800640 1 -- 172.21.15.42:0/1119389996 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff7b8069070 msgr2=0x7ff7b806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:39.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.422+0000 7ff7dc800640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff7b8069070 0x7ff7b806b530 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8000f70 tx=0x7ff7c8009040 comp rx=0 tx=0).stop 2024-05-28T14:06:39.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.422+0000 7ff7dc800640 1 -- 172.21.15.42:0/1119389996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7d415b350 msgr2=0x7ff7d414d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:39.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.422+0000 7ff7dc800640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7d415b350 0x7ff7d414d480 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff7d001af90 tx=0x7ff7d0019680 comp rx=0 tx=0).stop 2024-05-28T14:06:39.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.422+0000 7ff7dc800640 1 -- 172.21.15.42:0/1119389996 shutdown_connections 2024-05-28T14:06:39.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.422+0000 7ff7dc800640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff7b8069070 0x7ff7b806b530 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.422+0000 7ff7dc800640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7d415b350 0x7ff7d414d480 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.422+0000 7ff7dc800640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7d40d0e00 0x7ff7d4147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.422+0000 7ff7dc800640 1 --2- 172.21.15.42:0/1119389996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d4072d40 0x7ff7d4151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.422+0000 7ff7dc800640 1 -- 172.21.15.42:0/1119389996 >> 172.21.15.42:0/1119389996 conn(0x7ff7d406bdf0 msgr2=0x7ff7d40b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:39.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.422+0000 7ff7dc800640 1 -- 172.21.15.42:0/1119389996 shutdown_connections 2024-05-28T14:06:39.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.422+0000 7ff7dc800640 1 -- 172.21.15.42:0/1119389996 wait complete. 2024-05-28T14:06:39.427 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:06:39.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf39800640 1 Processor -- start 2024-05-28T14:06:39.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf39800640 1 -- start start 2024-05-28T14:06:39.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf39800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcf3415e8c0 0x7fcf340b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:39.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf39800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf340b0650 0x7fcf3406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:39.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf39800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcf340b0a90 0x7fcf3406bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:39.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf39800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fcf340ae420 con 0x7fcf340b0a90 2024-05-28T14:06:39.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf39800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcf340b4440 con 0x7fcf3415e8c0 2024-05-28T14:06:39.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf39800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcf340b0f10 con 0x7fcf340b0650 2024-05-28T14:06:39.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf33400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcf340b0a90 0x7fcf3406bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:39.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf33400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcf340b0a90 0x7fcf3406bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39404/0 (socket says 172.21.15.42:39404) 2024-05-28T14:06:39.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf33400640 1 -- 172.21.15.42:0/3413295576 learned_addr learned my addr 172.21.15.42:0/3413295576 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf2a000640 1 --2- 172.21.15.42:0/3413295576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf340b0650 0x7fcf3406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf32a00640 1 --2- 172.21.15.42:0/3413295576 >> v2:172.21.15.63:3300/0 conn(0x7fcf3415e8c0 0x7fcf340b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf33400640 1 -- 172.21.15.42:0/3413295576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf340b0650 msgr2=0x7fcf3406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf33400640 1 --2- 172.21.15.42:0/3413295576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf340b0650 0x7fcf3406b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf33400640 1 -- 172.21.15.42:0/3413295576 >> v2:172.21.15.63:3300/0 conn(0x7fcf3415e8c0 msgr2=0x7fcf340b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:06:39.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf33400640 1 --2- 172.21.15.42:0/3413295576 >> v2:172.21.15.63:3300/0 conn(0x7fcf3415e8c0 0x7fcf340b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.586+0000 7fcf33400640 1 -- 172.21.15.42:0/3413295576 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf340ae670 con 0x7fcf340b0a90 2024-05-28T14:06:39.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.590+0000 7fcf33400640 1 --2- 172.21.15.42:0/3413295576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf340b0a90 0x7fcf3406bf30 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009870 tx=0x7fcf2c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b802e790aefe00 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:39.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.590+0000 7fcf32000640 1 -- 172.21.15.42:0/3413295576 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c0ab600 con 0x7fcf340b0a90 2024-05-28T14:06:39.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.590+0000 7fcf32000640 1 -- 172.21.15.42:0/3413295576 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2c0b0070 con 0x7fcf340b0a90 2024-05-28T14:06:39.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.590+0000 7fcf32000640 1 -- 172.21.15.42:0/3413295576 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c0ab9c0 con 0x7fcf340b0a90 2024-05-28T14:06:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.590+0000 7fcf39800640 1 -- 172.21.15.42:0/3413295576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf340b0a90 msgr2=0x7fcf3406bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.590+0000 7fcf39800640 1 --2- 172.21.15.42:0/3413295576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf340b0a90 0x7fcf3406bf30 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009870 tx=0x7fcf2c01c440 comp rx=0 tx=0).stop 2024-05-28T14:06:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.590+0000 7fcf39800640 1 -- 172.21.15.42:0/3413295576 shutdown_connections 2024-05-28T14:06:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.590+0000 7fcf39800640 1 --2- 172.21.15.42:0/3413295576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf340b0a90 0x7fcf3406bf30 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.590+0000 7fcf39800640 1 --2- 172.21.15.42:0/3413295576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf340b0650 0x7fcf3406b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.590+0000 7fcf39800640 1 --2- 172.21.15.42:0/3413295576 >> v2:172.21.15.63:3300/0 conn(0x7fcf3415e8c0 0x7fcf340b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.590+0000 7fcf39800640 1 -- 172.21.15.42:0/3413295576 >> 172.21.15.42:0/3413295576 conn(0x7fcf340b6960 msgr2=0x7fcf340b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.590+0000 7fcf39800640 1 -- 172.21.15.42:0/3413295576 shutdown_connections 2024-05-28T14:06:39.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.590+0000 7fcf39800640 1 -- 172.21.15.42:0/3413295576 wait complete. 2024-05-28T14:06:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf39800640 1 Processor -- start 2024-05-28T14:06:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf39800640 1 -- start start 2024-05-28T14:06:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf39800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf340b0650 0x7fcf341517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf39800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf340b0a90 0x7fcf34159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf39800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf3415e8c0 0x7fcf34152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf39800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcf340ce760 con 0x7fcf340b0a90 2024-05-28T14:06:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf39800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf340ce5e0 con 0x7fcf3415e8c0 2024-05-28T14:06:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf39800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcf340ce8e0 con 0x7fcf340b0650 2024-05-28T14:06:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf2a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf340b0a90 0x7fcf34159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:39.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf2a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf340b0a90 0x7fcf34159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39410/0 (socket says 172.21.15.42:39410) 2024-05-28T14:06:39.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf2a000640 1 -- 172.21.15.42:0/1366356408 learned_addr learned my addr 172.21.15.42:0/1366356408 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:06:39.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf32a00640 1 --2- 172.21.15.42:0/1366356408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf340b0650 0x7fcf341517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:39.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf33400640 1 --2- 172.21.15.42:0/1366356408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf3415e8c0 0x7fcf34152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:39.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf2a000640 1 -- 172.21.15.42:0/1366356408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf340b0650 msgr2=0x7fcf341517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:39.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf2a000640 1 --2- 172.21.15.42:0/1366356408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf340b0650 0x7fcf341517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf2a000640 1 -- 172.21.15.42:0/1366356408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf3415e8c0 msgr2=0x7fcf34152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:39.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf2a000640 1 --2- 172.21.15.42:0/1366356408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf3415e8c0 0x7fcf34152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf2a000640 1 -- 172.21.15.42:0/1366356408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf340d0060 con 0x7fcf340b0a90 2024-05-28T14:06:39.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf2a000640 1 --2- 172.21.15.42:0/1366356408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf340b0a90 0x7fcf34159950 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00d800 tx=0x7fcf1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:39.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf2b400640 1 -- 172.21.15.42:0/1366356408 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c00b6c0 con 0x7fcf340b0a90 2024-05-28T14:06:39.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf2b400640 1 -- 172.21.15.42:0/1366356408 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf1c019070 con 0x7fcf340b0a90 2024-05-28T14:06:39.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf2b400640 1 -- 172.21.15.42:0/1366356408 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c00ba60 con 0x7fcf340b0a90 2024-05-28T14:06:39.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf39800640 1 -- 172.21.15.42:0/1366356408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf340d0340 con 0x7fcf340b0a90 2024-05-28T14:06:39.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.594+0000 7fcf39800640 1 -- 172.21.15.42:0/1366356408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf340d0900 con 0x7fcf340b0a90 2024-05-28T14:06:39.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.598+0000 7fcf39800640 1 -- 172.21.15.42:0/1366356408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3409f7a0 con 0x7fcf340b0a90 2024-05-28T14:06:39.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.598+0000 7fcf2b400640 1 -- 172.21.15.42:0/1366356408 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 89160+0+0 (secure 0 0 0) 0x7fcf1c00bc00 con 0x7fcf340b0a90 2024-05-28T14:06:39.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.598+0000 7fcf2b400640 1 --2- 172.21.15.42:0/1366356408 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf10069070 0x7fcf1006b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:06:39.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.598+0000 7fcf2b400640 1 -- 172.21.15.42:0/1366356408 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7fcf1c085a50 con 0x7fcf340b0a90 2024-05-28T14:06:39.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.598+0000 7fcf32a00640 1 --2- 172.21.15.42:0/1366356408 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf10069070 0x7fcf1006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:06:39.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.598+0000 7fcf32a00640 1 --2- 172.21.15.42:0/1366356408 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf10069070 0x7fcf1006b530 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcf240039f0 tx=0x7fcf24009830 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:06:39.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.602+0000 7fcf2b400640 1 -- 172.21.15.42:0/1366356408 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcf1c057180 con 0x7fcf340b0a90 2024-05-28T14:06:39.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.794+0000 7fcf39800640 1 -- 172.21.15.42:0/1366356408 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fcf3404fdd0 con 0x7fcf10069070 2024-05-28T14:06:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.798+0000 7fcf2b400640 1 -- 172.21.15.42:0/1366356408 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fcf3404fdd0 con 0x7fcf10069070 2024-05-28T14:06:39.786 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/9cc37a41-e306-43e2-89a9-4518e569c510 2024-05-28T14:06:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.798+0000 7fcf39800640 1 -- 172.21.15.42:0/1366356408 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf10069070 msgr2=0x7fcf1006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.798+0000 7fcf39800640 1 --2- 172.21.15.42:0/1366356408 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf10069070 0x7fcf1006b530 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcf240039f0 tx=0x7fcf24009830 comp rx=0 tx=0).stop 2024-05-28T14:06:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.798+0000 7fcf39800640 1 -- 172.21.15.42:0/1366356408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf340b0a90 msgr2=0x7fcf34159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:06:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.798+0000 7fcf39800640 1 --2- 172.21.15.42:0/1366356408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf340b0a90 0x7fcf34159950 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00d800 tx=0x7fcf1c00dcc0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.798+0000 7fcf39800640 1 -- 172.21.15.42:0/1366356408 shutdown_connections 2024-05-28T14:06:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.798+0000 7fcf39800640 1 --2- 172.21.15.42:0/1366356408 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf10069070 0x7fcf1006b530 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.798+0000 7fcf39800640 1 --2- 172.21.15.42:0/1366356408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf3415e8c0 0x7fcf34152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.798+0000 7fcf39800640 1 --2- 172.21.15.42:0/1366356408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf340b0a90 0x7fcf34159950 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.798+0000 7fcf39800640 1 --2- 172.21.15.42:0/1366356408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf340b0650 0x7fcf341517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:06:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.798+0000 7fcf39800640 1 -- 172.21.15.42:0/1366356408 >> 172.21.15.42:0/1366356408 conn(0x7fcf340b6960 msgr2=0x7fcf3406f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:06:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.798+0000 7fcf39800640 1 -- 172.21.15.42:0/1366356408 shutdown_connections 2024-05-28T14:06:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:06:39.798+0000 7fcf39800640 1 -- 172.21.15.42:0/1366356408 wait complete. 2024-05-28T14:06:39.807 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/9cc37a41-e306-43e2-89a9-4518e569c510 2024-05-28T14:06:39.807 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:06:39.807 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:06:39.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:39.816 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9cc37a41-e306-43e2-89a9-4518e569c510/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:06:46.899 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:06:46.899 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:06:46.899 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.01954 s, 153 MB/s 2024-05-28T14:06:46.900 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:06:46.900 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:06:46.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:46.909 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9cc37a41-e306-43e2-89a9-4518e569c510/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:06:48.835 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:06:48.840 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:06:53.814 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:06:53.815 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:06:53.815 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.84831 s, 157 MB/s 2024-05-28T14:06:53.816 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:06:53.816 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:06:53.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:06:53.825 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9cc37a41-e306-43e2-89a9-4518e569c510/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:07:00.790 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:07:00.791 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:07:00.791 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.91195 s, 155 MB/s 2024-05-28T14:07:00.792 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-28T14:07:00.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.950+0000 7fd800a00640 1 Processor -- start 2024-05-28T14:07:00.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.950+0000 7fd800a00640 1 -- start start 2024-05-28T14:07:00.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd800a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7fc15b350 0x7fd7fc06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:00.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd800a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd7fc072d40 0x7fd7fc0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd800a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd7fc0737c0 0x7fd7fc0d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd800a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd7fc0ae9f0 con 0x7fd7fc0737c0 2024-05-28T14:07:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd800a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd7fc0ade40 con 0x7fd7fc072d40 2024-05-28T14:07:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd800a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd7fc0ae0c0 con 0x7fd7fc15b350 2024-05-28T14:07:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd7fa800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd7fc0737c0 0x7fd7fc0d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd7fa800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd7fc0737c0 0x7fd7fc0d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46686/0 (socket says 172.21.15.42:46686) 2024-05-28T14:07:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd7fa800640 1 -- 172.21.15.42:0/2595613363 learned_addr learned my addr 172.21.15.42:0/2595613363 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd7f9e00640 1 --2- 172.21.15.42:0/2595613363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7fc15b350 0x7fd7fc06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd7f3e00640 1 --2- 172.21.15.42:0/2595613363 >> v2:172.21.15.63:3300/0 conn(0x7fd7fc072d40 0x7fd7fc0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd7fa800640 1 -- 172.21.15.42:0/2595613363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7fc15b350 msgr2=0x7fd7fc06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd7fa800640 1 --2- 172.21.15.42:0/2595613363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7fc15b350 0x7fd7fc06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd7fa800640 1 -- 172.21.15.42:0/2595613363 >> v2:172.21.15.63:3300/0 conn(0x7fd7fc072d40 msgr2=0x7fd7fc0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd7fa800640 1 --2- 172.21.15.42:0/2595613363 >> v2:172.21.15.63:3300/0 conn(0x7fd7fc072d40 0x7fd7fc0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.954+0000 7fd7fa800640 1 -- 172.21.15.42:0/2595613363 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7fc0ae3a0 con 0x7fd7fc0737c0 2024-05-28T14:07:00.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd7fa800640 1 --2- 172.21.15.42:0/2595613363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7fc0737c0 0x7fd7fc0d0c30 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0d12d0 tx=0x7fd7f401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbd4de4ab9525d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:00.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd7f9400640 1 -- 172.21.15.42:0/2595613363 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f40a5660 con 0x7fd7fc0737c0 2024-05-28T14:07:00.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd7f9400640 1 -- 172.21.15.42:0/2595613363 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f40a5800 con 0x7fd7fc0737c0 2024-05-28T14:07:00.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd7f9400640 1 -- 172.21.15.42:0/2595613363 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f40a5bd0 con 0x7fd7fc0737c0 2024-05-28T14:07:00.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd800a00640 1 -- 172.21.15.42:0/2595613363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7fc0737c0 msgr2=0x7fd7fc0d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:00.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd800a00640 1 --2- 172.21.15.42:0/2595613363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7fc0737c0 0x7fd7fc0d0c30 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0d12d0 tx=0x7fd7f401c960 comp rx=0 tx=0).stop 2024-05-28T14:07:00.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd800a00640 1 -- 172.21.15.42:0/2595613363 shutdown_connections 2024-05-28T14:07:00.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd800a00640 1 --2- 172.21.15.42:0/2595613363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7fc0737c0 0x7fd7fc0d0c30 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:00.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd800a00640 1 --2- 172.21.15.42:0/2595613363 >> v2:172.21.15.63:3300/0 conn(0x7fd7fc072d40 0x7fd7fc0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:00.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd800a00640 1 --2- 172.21.15.42:0/2595613363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7fc15b350 0x7fd7fc06dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:00.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd800a00640 1 -- 172.21.15.42:0/2595613363 >> 172.21.15.42:0/2595613363 conn(0x7fd7fc06bdf0 msgr2=0x7fd7fc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:00.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd800a00640 1 -- 172.21.15.42:0/2595613363 shutdown_connections 2024-05-28T14:07:00.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd800a00640 1 -- 172.21.15.42:0/2595613363 wait complete. 2024-05-28T14:07:00.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd800a00640 1 Processor -- start 2024-05-28T14:07:00.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.958+0000 7fd800a00640 1 -- start start 2024-05-28T14:07:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd800a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7fc072d40 0x7fd7fc099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd800a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7fc0737c0 0x7fd7fc14f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd800a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7fc15b350 0x7fd7fc14f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd800a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc0af7f0 con 0x7fd7fc0737c0 2024-05-28T14:07:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd800a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc0af670 con 0x7fd7fc072d40 2024-05-28T14:07:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd800a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd7fc0af970 con 0x7fd7fc15b350 2024-05-28T14:07:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7fc0737c0 0x7fd7fc14f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7fc0737c0 0x7fd7fc14f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46688/0 (socket says 172.21.15.42:46688) 2024-05-28T14:07:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7f3e00640 1 -- 172.21.15.42:0/2925716305 learned_addr learned my addr 172.21.15.42:0/2925716305 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7f9e00640 1 --2- 172.21.15.42:0/2925716305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7fc072d40 0x7fd7fc099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7fa800640 1 --2- 172.21.15.42:0/2925716305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7fc15b350 0x7fd7fc14f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7f3e00640 1 -- 172.21.15.42:0/2925716305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7fc15b350 msgr2=0x7fd7fc14f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7f3e00640 1 --2- 172.21.15.42:0/2925716305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7fc15b350 0x7fd7fc14f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7f3e00640 1 -- 172.21.15.42:0/2925716305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7fc072d40 msgr2=0x7fd7fc099920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7f3e00640 1 --2- 172.21.15.42:0/2925716305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7fc072d40 0x7fd7fc099920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7f3e00640 1 -- 172.21.15.42:0/2925716305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7fc152cd0 con 0x7fd7fc0737c0 2024-05-28T14:07:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7f3e00640 1 --2- 172.21.15.42:0/2925716305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7fc0737c0 0x7fd7fc14f3d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400d800 tx=0x7fd7e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7f2000640 1 -- 172.21.15.42:0/2925716305 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4018020 con 0x7fd7fc0737c0 2024-05-28T14:07:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7f2000640 1 -- 172.21.15.42:0/2925716305 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e40096a0 con 0x7fd7fc0737c0 2024-05-28T14:07:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7f2000640 1 -- 172.21.15.42:0/2925716305 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4009a90 con 0x7fd7fc0737c0 2024-05-28T14:07:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd800a00640 1 -- 172.21.15.42:0/2925716305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7fc152fb0 con 0x7fd7fc0737c0 2024-05-28T14:07:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd800a00640 1 -- 172.21.15.42:0/2925716305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7fc153570 con 0x7fd7fc0737c0 2024-05-28T14:07:00.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd800a00640 1 -- 172.21.15.42:0/2925716305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7fc068a90 con 0x7fd7fc0737c0 2024-05-28T14:07:00.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.962+0000 7fd7f2000640 1 -- 172.21.15.42:0/2925716305 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 89262+0+0 (secure 0 0 0) 0x7fd7e401b020 con 0x7fd7fc0737c0 2024-05-28T14:07:00.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.966+0000 7fd7f2000640 1 --2- 172.21.15.42:0/2925716305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7d8069190 0x7fd7d806b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.966+0000 7fd7f2000640 1 -- 172.21.15.42:0/2925716305 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7fd7e4085bc0 con 0x7fd7fc0737c0 2024-05-28T14:07:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.966+0000 7fd7f9e00640 1 --2- 172.21.15.42:0/2925716305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7d8069190 0x7fd7d806b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.966+0000 7fd7f9e00640 1 --2- 172.21.15.42:0/2925716305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7d8069190 0x7fd7d806b650 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec007a60 tx=0x7fd7ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:00.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:00.970+0000 7fd7f2000640 1 -- 172.21.15.42:0/2925716305 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd7e4022020 con 0x7fd7fc0737c0 2024-05-28T14:07:01.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.166+0000 7fd800a00640 1 -- 172.21.15.42:0/2925716305 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fd7fc072b20 con 0x7fd7d8069190 2024-05-28T14:07:01.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.178+0000 7fd7f2000640 1 -- 172.21.15.42:0/2925716305 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd7fc072b20 con 0x7fd7d8069190 2024-05-28T14:07:01.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.178+0000 7fd800a00640 1 -- 172.21.15.42:0/2925716305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7d8069190 msgr2=0x7fd7d806b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:01.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.178+0000 7fd800a00640 1 --2- 172.21.15.42:0/2925716305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7d8069190 0x7fd7d806b650 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec007a60 tx=0x7fd7ec008040 comp rx=0 tx=0).stop 2024-05-28T14:07:01.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.178+0000 7fd800a00640 1 -- 172.21.15.42:0/2925716305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7fc0737c0 msgr2=0x7fd7fc14f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:01.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.178+0000 7fd800a00640 1 --2- 172.21.15.42:0/2925716305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7fc0737c0 0x7fd7fc14f3d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400d800 tx=0x7fd7e400dcc0 comp rx=0 tx=0).stop 2024-05-28T14:07:01.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.178+0000 7fd800a00640 1 -- 172.21.15.42:0/2925716305 shutdown_connections 2024-05-28T14:07:01.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.178+0000 7fd800a00640 1 --2- 172.21.15.42:0/2925716305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7d8069190 0x7fd7d806b650 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:01.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.178+0000 7fd800a00640 1 --2- 172.21.15.42:0/2925716305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7fc15b350 0x7fd7fc14f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:01.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.178+0000 7fd800a00640 1 --2- 172.21.15.42:0/2925716305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7fc0737c0 0x7fd7fc14f3d0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:01.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.178+0000 7fd800a00640 1 --2- 172.21.15.42:0/2925716305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7fc072d40 0x7fd7fc099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:01.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.178+0000 7fd800a00640 1 -- 172.21.15.42:0/2925716305 >> 172.21.15.42:0/2925716305 conn(0x7fd7fc06bdf0 msgr2=0x7fd7fc15f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:01.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.178+0000 7fd800a00640 1 -- 172.21.15.42:0/2925716305 shutdown_connections 2024-05-28T14:07:01.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.178+0000 7fd800a00640 1 -- 172.21.15.42:0/2925716305 wait complete. 2024-05-28T14:07:01.186 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:07:01.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.342+0000 7fc8d2c00640 1 Processor -- start 2024-05-28T14:07:01.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.342+0000 7fc8d2c00640 1 -- start start 2024-05-28T14:07:01.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.346+0000 7fc8d2c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc8cc0cce90 0x7fc8cc15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:01.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.346+0000 7fc8d2c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8cc0b34f0 con 0x7fc8cc0b3cd0 2024-05-28T14:07:01.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.346+0000 7fc8d2c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8cc15ba90 con 0x7fc8cc072670 2024-05-28T14:07:01.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.346+0000 7fc8d2c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc8cc0b5870 con 0x7fc8cc0cce90 2024-05-28T14:07:01.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.346+0000 7fc8c3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc8cc0cce90 0x7fc8cc15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:01.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.346+0000 7fc8d0c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc8cc0b3cd0 0x7fc8cc06dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:48874/0 (socket says 172.21.15.42:48874) 2024-05-28T14:07:01.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.346+0000 7fc8d0c00640 1 -- 172.21.15.42:0/1687688408 learned_addr learned my addr 172.21.15.42:0/1687688408 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:07:01.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.346+0000 7fc8c3e00640 1 -- 172.21.15.42:0/1687688408 >> v1:172.21.15.63:6789/0 conn(0x7fc8cc072670 legacy=0x7fc8cc072a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:01.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.346+0000 7fc8c3e00640 1 -- 172.21.15.42:0/1687688408 >> v1:172.21.15.42:6789/0 conn(0x7fc8cc0b3cd0 legacy=0x7fc8cc06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:01.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.346+0000 7fc8c3e00640 1 -- 172.21.15.42:0/1687688408 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8cc15bd70 con 0x7fc8cc0cce90 2024-05-28T14:07:01.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.346+0000 7fc8cb400640 1 -- 172.21.15.42:0/1687688408 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 226636640 0 0) 0x7fc8cc15ba90 con 0x7fc8cc072670 2024-05-28T14:07:01.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8c3e00640 1 --2- 172.21.15.42:0/1687688408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8cc0cce90 0x7fc8cc15b1f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40097c0 tx=0x7fc8b401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959f75d15964a814 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:01.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8cb400640 1 -- 172.21.15.42:0/1687688408 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b40b0070 con 0x7fc8cc0cce90 2024-05-28T14:07:01.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8cb400640 1 -- 172.21.15.42:0/1687688408 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b40ab600 con 0x7fc8cc0cce90 2024-05-28T14:07:01.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8cb400640 1 -- 172.21.15.42:0/1687688408 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b40ab9d0 con 0x7fc8cc0cce90 2024-05-28T14:07:01.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8d2c00640 1 -- 172.21.15.42:0/1687688408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8cc0cce90 msgr2=0x7fc8cc15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:01.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8d2c00640 1 --2- 172.21.15.42:0/1687688408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8cc0cce90 0x7fc8cc15b1f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40097c0 tx=0x7fc8b401c4a0 comp rx=0 tx=0).stop 2024-05-28T14:07:01.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8d2c00640 1 -- 172.21.15.42:0/1687688408 shutdown_connections 2024-05-28T14:07:01.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8d2c00640 1 --2- 172.21.15.42:0/1687688408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8cc0cce90 0x7fc8cc15b1f0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:01.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8d2c00640 1 -- 172.21.15.42:0/1687688408 >> 172.21.15.42:0/1687688408 conn(0x7fc8cc0b21f0 msgr2=0x7fc8cc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:01.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8d2c00640 1 -- 172.21.15.42:0/1687688408 shutdown_connections 2024-05-28T14:07:01.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8d2c00640 1 -- 172.21.15.42:0/1687688408 wait complete. 2024-05-28T14:07:01.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8d2c00640 1 Processor -- start 2024-05-28T14:07:01.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8d2c00640 1 -- start start 2024-05-28T14:07:01.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8d2c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8cc072670 0x7fc8cc146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:01.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8d2c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8cc0b3cd0 0x7fc8cc157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.350+0000 7fc8d2c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8cc0cce90 0x7fc8cc155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d2c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc169570 con 0x7fc8cc0cce90 2024-05-28T14:07:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d2c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc1693f0 con 0x7fc8cc0b3cd0 2024-05-28T14:07:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d2c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc8cc1696f0 con 0x7fc8cc072670 2024-05-28T14:07:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8cc0cce90 0x7fc8cc155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8cc0cce90 0x7fc8cc155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46712/0 (socket says 172.21.15.42:46712) 2024-05-28T14:07:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d0c00640 1 -- 172.21.15.42:0/4203877195 learned_addr learned my addr 172.21.15.42:0/4203877195 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8c3e00640 1 --2- 172.21.15.42:0/4203877195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8cc0b3cd0 0x7fc8cc157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8cbe00640 1 --2- 172.21.15.42:0/4203877195 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8cc072670 0x7fc8cc146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d0c00640 1 -- 172.21.15.42:0/4203877195 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8cc072670 msgr2=0x7fc8cc146a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d0c00640 1 --2- 172.21.15.42:0/4203877195 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8cc072670 0x7fc8cc146a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d0c00640 1 -- 172.21.15.42:0/4203877195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8cc0b3cd0 msgr2=0x7fc8cc157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d0c00640 1 --2- 172.21.15.42:0/4203877195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8cc0b3cd0 0x7fc8cc157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:01.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d0c00640 1 -- 172.21.15.42:0/4203877195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8cc1562c0 con 0x7fc8cc0cce90 2024-05-28T14:07:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d0c00640 1 --2- 172.21.15.42:0/4203877195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8cc0cce90 0x7fc8cc155870 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400cbc0 tx=0x7fc8c400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8c8c00640 1 -- 172.21.15.42:0/4203877195 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4017070 con 0x7fc8cc0cce90 2024-05-28T14:07:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8c8c00640 1 -- 172.21.15.42:0/4203877195 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c400ace0 con 0x7fc8cc0cce90 2024-05-28T14:07:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d2c00640 1 -- 172.21.15.42:0/4203877195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8cc1565a0 con 0x7fc8cc0cce90 2024-05-28T14:07:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8c8c00640 1 -- 172.21.15.42:0/4203877195 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4012600 con 0x7fc8cc0cce90 2024-05-28T14:07:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d2c00640 1 -- 172.21.15.42:0/4203877195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8cc156b60 con 0x7fc8cc0cce90 2024-05-28T14:07:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8d2c00640 1 -- 172.21.15.42:0/4203877195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8cc157030 con 0x7fc8cc0cce90 2024-05-28T14:07:01.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.354+0000 7fc8c8c00640 1 -- 172.21.15.42:0/4203877195 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 89262+0+0 (secure 0 0 0) 0x7fc8c401a070 con 0x7fc8cc0cce90 2024-05-28T14:07:01.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.358+0000 7fc8c8c00640 1 --2- 172.21.15.42:0/4203877195 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc8a8069190 0x7fc8a806b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:01.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.358+0000 7fc8c8c00640 1 -- 172.21.15.42:0/4203877195 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5737+0+0 (secure 0 0 0) 0x7fc8c4085380 con 0x7fc8cc0cce90 2024-05-28T14:07:01.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.358+0000 7fc8cbe00640 1 --2- 172.21.15.42:0/4203877195 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc8a8069190 0x7fc8a806b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:01.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.358+0000 7fc8cbe00640 1 --2- 172.21.15.42:0/4203877195 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc8a8069190 0x7fc8a806b650 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0cc610 tx=0x7fc8bc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:01.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.362+0000 7fc8c8c00640 1 -- 172.21.15.42:0/4203877195 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc8c4056a80 con 0x7fc8cc0cce90 2024-05-28T14:07:01.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.558+0000 7fc8d2c00640 1 -- 172.21.15.42:0/4203877195 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fc8cc04fdd0 con 0x7fc8a8069190 2024-05-28T14:07:01.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.562+0000 7fc8c8c00640 1 -- 172.21.15.42:0/4203877195 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fc8cc04fdd0 con 0x7fc8a8069190 2024-05-28T14:07:01.549 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/9cc37a41-e306-43e2-89a9-4518e569c510 2024-05-28T14:07:01.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.562+0000 7fc8d2c00640 1 -- 172.21.15.42:0/4203877195 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc8a8069190 msgr2=0x7fc8a806b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:01.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.562+0000 7fc8d2c00640 1 --2- 172.21.15.42:0/4203877195 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc8a8069190 0x7fc8a806b650 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0cc610 tx=0x7fc8bc009040 comp rx=0 tx=0).stop 2024-05-28T14:07:01.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.562+0000 7fc8d2c00640 1 -- 172.21.15.42:0/4203877195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8cc0cce90 msgr2=0x7fc8cc155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:01.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.562+0000 7fc8d2c00640 1 --2- 172.21.15.42:0/4203877195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8cc0cce90 0x7fc8cc155870 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400cbc0 tx=0x7fc8c400a430 comp rx=0 tx=0).stop 2024-05-28T14:07:01.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.562+0000 7fc8d2c00640 1 -- 172.21.15.42:0/4203877195 shutdown_connections 2024-05-28T14:07:01.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.562+0000 7fc8d2c00640 1 --2- 172.21.15.42:0/4203877195 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc8a8069190 0x7fc8a806b650 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:01.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.562+0000 7fc8d2c00640 1 --2- 172.21.15.42:0/4203877195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8cc0cce90 0x7fc8cc155870 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:01.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.562+0000 7fc8d2c00640 1 --2- 172.21.15.42:0/4203877195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8cc0b3cd0 0x7fc8cc157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:01.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.562+0000 7fc8d2c00640 1 --2- 172.21.15.42:0/4203877195 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8cc072670 0x7fc8cc146a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:01.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.562+0000 7fc8d2c00640 1 -- 172.21.15.42:0/4203877195 >> 172.21.15.42:0/4203877195 conn(0x7fc8cc0b21f0 msgr2=0x7fc8cc1596e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:01.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.566+0000 7fc8d2c00640 1 -- 172.21.15.42:0/4203877195 shutdown_connections 2024-05-28T14:07:01.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:01.566+0000 7fc8d2c00640 1 -- 172.21.15.42:0/4203877195 wait complete. 2024-05-28T14:07:01.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:07:01.570 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9cc37a41-e306-43e2-89a9-4518e569c510 2024-05-28T14:07:01.570 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:07:01.583 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/9cc37a41-e306-43e2-89a9-4518e569c510 2024-05-28T14:07:01.583 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3138256896" 2024-05-28T14:07:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:07:01.584 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-28T14:07:03.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:07:03.585 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9cc37a41-e306-43e2-89a9-4518e569c510 2024-05-28T14:07:03.585 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:07:03.600 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/9cc37a41-e306-43e2-89a9-4518e569c510 2024-05-28T14:07:03.600 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3138256896" 2024-05-28T14:07:03.600 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:07:03.601 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-28T14:07:05.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:07:05.602 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9cc37a41-e306-43e2-89a9-4518e569c510 2024-05-28T14:07:05.602 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:07:05.615 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/9cc37a41-e306-43e2-89a9-4518e569c510 2024-05-28T14:07:05.616 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3221225472" 2024-05-28T14:07:05.616 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:07:05.616 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-28T14:07:05.617 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone8111 2024-05-28T14:07:05.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f7567800640 1 Processor -- start 2024-05-28T14:07:05.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f7567800640 1 -- start start 2024-05-28T14:07:05.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f7567800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7560071de0 0x7f75600721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:05.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f7567800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f75600ce3b0 0x7f75600ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:05.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f7567800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7560072700 con 0x7f7560141d10 2024-05-28T14:07:05.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f7567800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f7560160230 con 0x7f7560071de0 2024-05-28T14:07:05.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f7567800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7560160510 con 0x7f75600ce3b0 2024-05-28T14:07:05.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f7564e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7560071de0 0x7f75600721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:05.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f7564e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7560071de0 0x7f75600721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46726/0 (socket says 172.21.15.42:46726) 2024-05-28T14:07:05.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f7564e00640 1 -- 172.21.15.42:0/1201913684 learned_addr learned my addr 172.21.15.42:0/1201913684 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:05.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f755fe00640 1 --2- 172.21.15.42:0/1201913684 >> v2:172.21.15.63:3300/0 conn(0x7f75600ce3b0 0x7f75600ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:05.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f7564e00640 1 -- 172.21.15.42:0/1201913684 >> v2:172.21.15.63:3300/0 conn(0x7f75600ce3b0 msgr2=0x7f75600ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:05.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f7564e00640 1 --2- 172.21.15.42:0/1201913684 >> v2:172.21.15.63:3300/0 conn(0x7f75600ce3b0 0x7f75600ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:05.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f7564e00640 1 -- 172.21.15.42:0/1201913684 >> v1:172.21.15.42:6789/0 conn(0x7f7560141d10 legacy=0x7f7560142160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:05.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f7564e00640 1 -- 172.21.15.42:0/1201913684 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75601607f0 con 0x7f7560071de0 2024-05-28T14:07:05.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.826+0000 7f755fe00640 1 --2- 172.21.15.42:0/1201913684 >> v2:172.21.15.63:3300/0 conn(0x7f75600ce3b0 0x7f75600ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:07:05.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.830+0000 7f7564e00640 1 --2- 172.21.15.42:0/1201913684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7560071de0 0x7f75600721c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7560073280 tx=0x7f755401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82b431f9fde357a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:05.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.830+0000 7f755f400640 1 -- 172.21.15.42:0/1201913684 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75540b0070 con 0x7f7560071de0 2024-05-28T14:07:05.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.830+0000 7f755f400640 1 -- 172.21.15.42:0/1201913684 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75540ab600 con 0x7f7560071de0 2024-05-28T14:07:05.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.830+0000 7f755f400640 1 -- 172.21.15.42:0/1201913684 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75540ab9d0 con 0x7f7560071de0 2024-05-28T14:07:05.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.830+0000 7f7567800640 1 -- 172.21.15.42:0/1201913684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7560071de0 msgr2=0x7f75600721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:05.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.830+0000 7f7567800640 1 --2- 172.21.15.42:0/1201913684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7560071de0 0x7f75600721c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7560073280 tx=0x7f755401c400 comp rx=0 tx=0).stop 2024-05-28T14:07:05.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.830+0000 7f7567800640 1 -- 172.21.15.42:0/1201913684 shutdown_connections 2024-05-28T14:07:05.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.830+0000 7f7567800640 1 --2- 172.21.15.42:0/1201913684 >> v2:172.21.15.63:3300/0 conn(0x7f75600ce3b0 0x7f75600ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:05.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.830+0000 7f7567800640 1 --2- 172.21.15.42:0/1201913684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7560071de0 0x7f75600721c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:05.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.830+0000 7f7567800640 1 -- 172.21.15.42:0/1201913684 >> 172.21.15.42:0/1201913684 conn(0x7f75600b21f0 msgr2=0x7f75600b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:05.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.830+0000 7f7567800640 1 -- 172.21.15.42:0/1201913684 shutdown_connections 2024-05-28T14:07:05.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.830+0000 7f7567800640 1 -- 172.21.15.42:0/1201913684 wait complete. 2024-05-28T14:07:05.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.830+0000 7f7567800640 1 Processor -- start 2024-05-28T14:07:05.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.830+0000 7f7567800640 1 -- start start 2024-05-28T14:07:05.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7567800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7560071de0 0x7f7560156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7567800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75600ce3b0 0x7f756013d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7567800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7560141d10 0x7f756013de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7567800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7560169e30 con 0x7f7560071de0 2024-05-28T14:07:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7567800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7560169cb0 con 0x7f75600ce3b0 2024-05-28T14:07:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7567800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7560169fb0 con 0x7f7560141d10 2024-05-28T14:07:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7564e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7560071de0 0x7f7560156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7564e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7560071de0 0x7f7560156400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46742/0 (socket says 172.21.15.42:46742) 2024-05-28T14:07:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7564e00640 1 -- 172.21.15.42:0/2395216344 learned_addr learned my addr 172.21.15.42:0/2395216344 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f755fe00640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75600ce3b0 0x7f756013d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7565800640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7560141d10 0x7f756013de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7564e00640 1 -- 172.21.15.42:0/2395216344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7560141d10 msgr2=0x7f756013de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7564e00640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7560141d10 0x7f756013de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7564e00640 1 -- 172.21.15.42:0/2395216344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75600ce3b0 msgr2=0x7f756013d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:05.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7564e00640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75600ce3b0 0x7f756013d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:05.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7564e00640 1 -- 172.21.15.42:0/2395216344 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75600b4650 con 0x7f7560071de0 2024-05-28T14:07:05.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7565800640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7560141d10 0x7f756013de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:07:05.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7564e00640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7560071de0 0x7f7560156400 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f75540ae040 tx=0x7f755401ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:05.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f755cc00640 1 -- 172.21.15.42:0/2395216344 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75540ba020 con 0x7f7560071de0 2024-05-28T14:07:05.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f755cc00640 1 -- 172.21.15.42:0/2395216344 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75540b6640 con 0x7f7560071de0 2024-05-28T14:07:05.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7567800640 1 -- 172.21.15.42:0/2395216344 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75600b4930 con 0x7f7560071de0 2024-05-28T14:07:05.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f755cc00640 1 -- 172.21.15.42:0/2395216344 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75540b0040 con 0x7f7560071de0 2024-05-28T14:07:05.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7567800640 1 -- 172.21.15.42:0/2395216344 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75600b4ef0 con 0x7f7560071de0 2024-05-28T14:07:05.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.834+0000 7f7567800640 1 -- 172.21.15.42:0/2395216344 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75600738b0 con 0x7f7560071de0 2024-05-28T14:07:05.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.838+0000 7f755cc00640 1 -- 172.21.15.42:0/2395216344 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 89262+0+0 (secure 0 0 0) 0x7f75540bd020 con 0x7f7560071de0 2024-05-28T14:07:05.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.838+0000 7f755cc00640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7540069190 0x7f754006b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:05.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.838+0000 7f755cc00640 1 -- 172.21.15.42:0/2395216344 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f755411f700 con 0x7f7560071de0 2024-05-28T14:07:05.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.838+0000 7f755fe00640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7540069190 0x7f754006b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:05.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.838+0000 7f755fe00640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7540069190 0x7f754006b650 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f75600b40f0 tx=0x7f754c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:05.842+0000 7f755cc00640 1 -- 172.21.15.42:0/2395216344 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f75540c4030 con 0x7f7560071de0 2024-05-28T14:07:06.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.038+0000 7f7567800640 1 -- 172.21.15.42:0/2395216344 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone8111", "target": ["mon-mgr", ""]}) -- 0x7f756004fdd0 con 0x7f7540069190 2024-05-28T14:07:06.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.090+0000 7f755cc00640 1 -- 172.21.15.42:0/2395216344 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f756004fdd0 con 0x7f7540069190 2024-05-28T14:07:06.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.090+0000 7f7567800640 1 -- 172.21.15.42:0/2395216344 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7540069190 msgr2=0x7f754006b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:06.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.090+0000 7f7567800640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7540069190 0x7f754006b650 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f75600b40f0 tx=0x7f754c008040 comp rx=0 tx=0).stop 2024-05-28T14:07:06.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.090+0000 7f7567800640 1 -- 172.21.15.42:0/2395216344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7560071de0 msgr2=0x7f7560156400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:06.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.090+0000 7f7567800640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7560071de0 0x7f7560156400 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f75540ae040 tx=0x7f755401ac40 comp rx=0 tx=0).stop 2024-05-28T14:07:06.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.090+0000 7f7567800640 1 -- 172.21.15.42:0/2395216344 shutdown_connections 2024-05-28T14:07:06.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.090+0000 7f7567800640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7540069190 0x7f754006b650 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:06.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.090+0000 7f7567800640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7560141d10 0x7f756013de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:06.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.090+0000 7f7567800640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75600ce3b0 0x7f756013d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:06.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.090+0000 7f7567800640 1 --2- 172.21.15.42:0/2395216344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7560071de0 0x7f7560156400 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:06.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.090+0000 7f7567800640 1 -- 172.21.15.42:0/2395216344 >> 172.21.15.42:0/2395216344 conn(0x7f75600b21f0 msgr2=0x7f75601405b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:06.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.090+0000 7f7567800640 1 -- 172.21.15.42:0/2395216344 shutdown_connections 2024-05-28T14:07:06.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.090+0000 7f7567800640 1 -- 172.21.15.42:0/2395216344 wait complete. 2024-05-28T14:07:06.080 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:06.094+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:06.080 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:06.094+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:06.080 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:06.094+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:06.080 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:06.094+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:06.080 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:06.094+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:06.097 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:06.110+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:06.097 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:06.110+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:06.097 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:06.110+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:06.097 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:06.110+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:06.098 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:06.110+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:06.103 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone8112 2024-05-28T14:07:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.270+0000 7ffb4dc00640 1 Processor -- start 2024-05-28T14:07:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.274+0000 7ffb4dc00640 1 -- start start 2024-05-28T14:07:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.274+0000 7ffb4dc00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffb480b7d70 0x7ffb480cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.274+0000 7ffb4dc00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb480d1a80 con 0x7ffb4815e870 2024-05-28T14:07:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.274+0000 7ffb4dc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb480ccbd0 con 0x7ffb4815ec50 2024-05-28T14:07:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.274+0000 7ffb4dc00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffb480cd500 con 0x7ffb480b7d70 2024-05-28T14:07:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.274+0000 7ffb46a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ffb4815e870 0x7ffb480d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:48896/0 (socket says 172.21.15.42:48896) 2024-05-28T14:07:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.274+0000 7ffb46a00640 1 -- 172.21.15.42:0/1919255992 learned_addr learned my addr 172.21.15.42:0/1919255992 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:07:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.274+0000 7ffb47400640 1 --2- 172.21.15.42:0/1919255992 >> v2:172.21.15.63:3300/0 conn(0x7ffb480b7d70 0x7ffb480cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.274+0000 7ffb46000640 1 -- 172.21.15.42:0/1919255992 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1391426592 0 0) 0x7ffb480d1a80 con 0x7ffb4815e870 2024-05-28T14:07:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.274+0000 7ffb46000640 1 -- 172.21.15.42:0/1919255992 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb2c003530 con 0x7ffb4815e870 2024-05-28T14:07:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.274+0000 7ffb47400640 1 -- 172.21.15.42:0/1919255992 >> v1:172.21.15.63:6789/0 conn(0x7ffb4815ec50 legacy=0x7ffb480b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.274+0000 7ffb47400640 1 -- 172.21.15.42:0/1919255992 >> v1:172.21.15.42:6789/0 conn(0x7ffb4815e870 legacy=0x7ffb480d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.274+0000 7ffb47400640 1 -- 172.21.15.42:0/1919255992 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb480cceb0 con 0x7ffb480b7d70 2024-05-28T14:07:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.278+0000 7ffb47400640 1 --2- 172.21.15.42:0/1919255992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb480b7d70 0x7ffb480cd990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009d90 tx=0x7ffb4001f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e658fa5e1df3178 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.278+0000 7ffb46000640 1 -- 172.21.15.42:0/1919255992 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb400b4070 con 0x7ffb480b7d70 2024-05-28T14:07:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.278+0000 7ffb46000640 1 -- 172.21.15.42:0/1919255992 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb400af600 con 0x7ffb480b7d70 2024-05-28T14:07:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.278+0000 7ffb46000640 1 -- 172.21.15.42:0/1919255992 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb400af9d0 con 0x7ffb480b7d70 2024-05-28T14:07:06.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.278+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1919255992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb480b7d70 msgr2=0x7ffb480cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:06.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.278+0000 7ffb4dc00640 1 --2- 172.21.15.42:0/1919255992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb480b7d70 0x7ffb480cd990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009d90 tx=0x7ffb4001f150 comp rx=0 tx=0).stop 2024-05-28T14:07:06.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.278+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1919255992 shutdown_connections 2024-05-28T14:07:06.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.278+0000 7ffb4dc00640 1 --2- 172.21.15.42:0/1919255992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb480b7d70 0x7ffb480cd990 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:06.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.278+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1919255992 >> 172.21.15.42:0/1919255992 conn(0x7ffb480b3fd0 msgr2=0x7ffb480b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:06.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.278+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1919255992 shutdown_connections 2024-05-28T14:07:06.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.278+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1919255992 wait complete. 2024-05-28T14:07:06.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.278+0000 7ffb4dc00640 1 Processor -- start 2024-05-28T14:07:06.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.278+0000 7ffb4dc00640 1 -- start start 2024-05-28T14:07:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb4dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb480b7d70 0x7ffb4815a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb4dc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb4815e870 0x7ffb480c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb4dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb4815ec50 0x7ffb4815fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb4dc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffb480b5330 con 0x7ffb4815ec50 2024-05-28T14:07:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb4dc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffb480b81a0 con 0x7ffb480b7d70 2024-05-28T14:07:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb4dc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffb480c84b0 con 0x7ffb4815e870 2024-05-28T14:07:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb47400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb4815ec50 0x7ffb4815fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb47400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb4815ec50 0x7ffb4815fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46786/0 (socket says 172.21.15.42:46786) 2024-05-28T14:07:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb47400640 1 -- 172.21.15.42:0/1355993343 learned_addr learned my addr 172.21.15.42:0/1355993343 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:06.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb46a00640 1 --2- 172.21.15.42:0/1355993343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb480b7d70 0x7ffb4815a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:06.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb3fe00640 1 --2- 172.21.15.42:0/1355993343 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb4815e870 0x7ffb480c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:06.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb47400640 1 -- 172.21.15.42:0/1355993343 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb4815e870 msgr2=0x7ffb480c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:06.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb47400640 1 --2- 172.21.15.42:0/1355993343 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb4815e870 0x7ffb480c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:06.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb47400640 1 -- 172.21.15.42:0/1355993343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb480b7d70 msgr2=0x7ffb4815a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:06.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb47400640 1 --2- 172.21.15.42:0/1355993343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb480b7d70 0x7ffb4815a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:06.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb47400640 1 -- 172.21.15.42:0/1355993343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb481f5450 con 0x7ffb4815ec50 2024-05-28T14:07:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb47400640 1 --2- 172.21.15.42:0/1355993343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb4815ec50 0x7ffb4815fec0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ffb400b2040 tx=0x7ffb4001bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb3ea00640 1 -- 172.21.15.42:0/1355993343 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb400b4040 con 0x7ffb4815ec50 2024-05-28T14:07:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb3ea00640 1 -- 172.21.15.42:0/1355993343 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb400c1070 con 0x7ffb4815ec50 2024-05-28T14:07:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb3ea00640 1 -- 172.21.15.42:0/1355993343 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40009450 con 0x7ffb4815ec50 2024-05-28T14:07:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1355993343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb4815aed0 con 0x7ffb4815ec50 2024-05-28T14:07:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1355993343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb4815f3e0 con 0x7ffb4815ec50 2024-05-28T14:07:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1355993343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb4804fdd0 con 0x7ffb4815ec50 2024-05-28T14:07:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.282+0000 7ffb3ea00640 1 -- 172.21.15.42:0/1355993343 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 89262+0+0 (secure 0 0 0) 0x7ffb400ba6b0 con 0x7ffb4815ec50 2024-05-28T14:07:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.286+0000 7ffb3ea00640 1 --2- 172.21.15.42:0/1355993343 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb24069190 0x7ffb2406b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.286+0000 7ffb3ea00640 1 -- 172.21.15.42:0/1355993343 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7ffb40123d50 con 0x7ffb4815ec50 2024-05-28T14:07:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.286+0000 7ffb46a00640 1 --2- 172.21.15.42:0/1355993343 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb24069190 0x7ffb2406b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.286+0000 7ffb46a00640 1 --2- 172.21.15.42:0/1355993343 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb24069190 0x7ffb2406b650 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffb380011f0 tx=0x7ffb38009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:06.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.290+0000 7ffb3ea00640 1 -- 172.21.15.42:0/1355993343 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffb40018cb0 con 0x7ffb4815ec50 2024-05-28T14:07:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:06.486+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1355993343 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone8112", "target": ["mon-mgr", ""]}) -- 0x7ffb48068da0 con 0x7ffb24069190 2024-05-28T14:07:07.081 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:07.094+0000 7f0735e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:07.081 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:07.094+0000 7f0735e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:07.082 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:07.094+0000 7f0735e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:07.082 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:07.094+0000 7f0735e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:07.082 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:07.094+0000 7f0735e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:07.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.190+0000 7ffb3ea00640 1 -- 172.21.15.42:0/1355993343 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffb48068da0 con 0x7ffb24069190 2024-05-28T14:07:07.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.194+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1355993343 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb24069190 msgr2=0x7ffb2406b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:07.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.194+0000 7ffb4dc00640 1 --2- 172.21.15.42:0/1355993343 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb24069190 0x7ffb2406b650 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffb380011f0 tx=0x7ffb38009040 comp rx=0 tx=0).stop 2024-05-28T14:07:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.194+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1355993343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb4815ec50 msgr2=0x7ffb4815fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.194+0000 7ffb4dc00640 1 --2- 172.21.15.42:0/1355993343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb4815ec50 0x7ffb4815fec0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ffb400b2040 tx=0x7ffb4001bef0 comp rx=0 tx=0).stop 2024-05-28T14:07:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.198+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1355993343 shutdown_connections 2024-05-28T14:07:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.198+0000 7ffb4dc00640 1 --2- 172.21.15.42:0/1355993343 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffb24069190 0x7ffb2406b650 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.198+0000 7ffb4dc00640 1 --2- 172.21.15.42:0/1355993343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb4815ec50 0x7ffb4815fec0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.198+0000 7ffb4dc00640 1 --2- 172.21.15.42:0/1355993343 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb4815e870 0x7ffb480c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.198+0000 7ffb4dc00640 1 --2- 172.21.15.42:0/1355993343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb480b7d70 0x7ffb4815a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.198+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1355993343 >> 172.21.15.42:0/1355993343 conn(0x7ffb480b3fd0 msgr2=0x7ffb480716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.198+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1355993343 shutdown_connections 2024-05-28T14:07:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.198+0000 7ffb4dc00640 1 -- 172.21.15.42:0/1355993343 wait complete. 2024-05-28T14:07:07.202 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:07.214+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:07.202 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:07.214+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:07.202 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:07.214+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:07.203 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:07.214+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:07.203 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:07.214+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:07.232 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone8113 2024-05-28T14:07:07.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.546+0000 7f9aa3200640 1 Processor -- start 2024-05-28T14:07:07.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.546+0000 7f9aa3200640 1 -- start start 2024-05-28T14:07:07.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.546+0000 7f9aa3200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9a9c15c670 0x7f9a9c072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:07.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.546+0000 7f9aa3200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a9c0cded0 con 0x7f9a9c0cd960 2024-05-28T14:07:07.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.546+0000 7f9aa3200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a9c0ce1b0 con 0x7f9a9c0cef70 2024-05-28T14:07:07.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.546+0000 7f9aa3200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9a9c0ceae0 con 0x7f9a9c15c670 2024-05-28T14:07:07.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.546+0000 7f9aa1200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f9a9c0cd960 0x7f9a9c15ca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51530/0 (socket says 172.21.15.42:51530) 2024-05-28T14:07:07.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.546+0000 7f9aa1200640 1 -- 172.21.15.42:0/1477622317 learned_addr learned my addr 172.21.15.42:0/1477622317 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:07:07.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a93e00640 1 --2- 172.21.15.42:0/1477622317 >> v2:172.21.15.63:3300/0 conn(0x7f9a9c15c670 0x7f9a9c072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:07.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3970352411 0 0) 0x7f9a9c0cded0 con 0x7f9a9c0cd960 2024-05-28T14:07:07.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a78003530 con 0x7f9a9c0cd960 2024-05-28T14:07:07.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2112980475 0 0) 0x7f9a9c0ce1b0 con 0x7f9a9c0cef70 2024-05-28T14:07:07.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a9c0cded0 con 0x7f9a9c0cef70 2024-05-28T14:07:07.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2015821092 0 0) 0x7f9a78003530 con 0x7f9a9c0cd960 2024-05-28T14:07:07.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 >> v2:172.21.15.63:3300/0 conn(0x7f9a9c15c670 msgr2=0x7f9a9c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:07.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 --2- 172.21.15.42:0/1477622317 >> v2:172.21.15.63:3300/0 conn(0x7f9a9c15c670 0x7f9a9c072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:07.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 >> v1:172.21.15.63:6789/0 conn(0x7f9a9c0cef70 legacy=0x7f9a9c0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:07.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9c0ce490 con 0x7f9a9c0cd960 2024-05-28T14:07:07.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f9a94001a00 con 0x7f9a9c0cd960 2024-05-28T14:07:07.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 >> v1:172.21.15.42:6789/0 conn(0x7f9a9c0cd960 legacy=0x7f9a9c15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:07.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 --2- 172.21.15.42:0/1477622317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a78003b50 0x7f9a78003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:07.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 --2- 172.21.15.42:0/1477622317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a78004810 0x7f9a78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:07.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 --2- 172.21.15.42:0/1477622317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a780099c0 0x7f9a78005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:07.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c0ceae0 con 0x7f9a78003b50 2024-05-28T14:07:07.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9a78014900 con 0x7f9a780099c0 2024-05-28T14:07:07.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9a78014b00 con 0x7f9a78004810 2024-05-28T14:07:07.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a93e00640 1 --2- 172.21.15.42:0/1477622317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a78003b50 0x7f9a78003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:07.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9aa1200640 1 --2- 172.21.15.42:0/1477622317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a780099c0 0x7f9a78005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:07.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.550+0000 7f9a9be00640 1 --2- 172.21.15.42:0/1477622317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a78004810 0x7f9a78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:07.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.554+0000 7f9a93e00640 1 -- 172.21.15.42:0/1477622317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a78004810 msgr2=0x7f9a78004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:07.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.554+0000 7f9a93e00640 1 --2- 172.21.15.42:0/1477622317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a78004810 0x7f9a78004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:07.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.554+0000 7f9a93e00640 1 -- 172.21.15.42:0/1477622317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a780099c0 msgr2=0x7f9a78005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:07.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.554+0000 7f9a93e00640 1 --2- 172.21.15.42:0/1477622317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a780099c0 0x7f9a78005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:07.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.554+0000 7f9a93e00640 1 -- 172.21.15.42:0/1477622317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a78003530 con 0x7f9a78003b50 2024-05-28T14:07:07.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.554+0000 7f9aa1200640 1 -- 172.21.15.42:0/1477622317 reap_dead start 2024-05-28T14:07:07.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.558+0000 7f9a93e00640 1 --2- 172.21.15.42:0/1477622317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a78003b50 0x7f9a78003f50 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009ac0 tx=0x7f9a8001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4d1c592fd0443 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:07.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.558+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80009690 con 0x7f9a78003b50 2024-05-28T14:07:07.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.558+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a800a6430 con 0x7f9a78003b50 2024-05-28T14:07:07.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.558+0000 7f9a9b400640 1 -- 172.21.15.42:0/1477622317 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a800a6800 con 0x7f9a78003b50 2024-05-28T14:07:07.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.558+0000 7f9aa3200640 1 -- 172.21.15.42:0/1477622317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a78003b50 msgr2=0x7f9a78003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:07.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.558+0000 7f9aa3200640 1 --2- 172.21.15.42:0/1477622317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a78003b50 0x7f9a78003f50 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009ac0 tx=0x7f9a8001ca50 comp rx=0 tx=0).stop 2024-05-28T14:07:07.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.562+0000 7f9aa3200640 1 -- 172.21.15.42:0/1477622317 shutdown_connections 2024-05-28T14:07:07.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.562+0000 7f9aa3200640 1 --2- 172.21.15.42:0/1477622317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a78003b50 0x7f9a78003f50 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:07.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.562+0000 7f9aa3200640 1 -- 172.21.15.42:0/1477622317 >> 172.21.15.42:0/1477622317 conn(0x7f9a9c06b9f0 msgr2=0x7f9a9c06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:07.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.562+0000 7f9aa3200640 1 -- 172.21.15.42:0/1477622317 shutdown_connections 2024-05-28T14:07:07.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.562+0000 7f9aa3200640 1 -- 172.21.15.42:0/1477622317 wait complete. 2024-05-28T14:07:07.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.566+0000 7f9aa3200640 1 Processor -- start 2024-05-28T14:07:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.566+0000 7f9aa3200640 1 -- start start 2024-05-28T14:07:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.570+0000 7f9aa3200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a78003b50 0x7f9a9c15b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.570+0000 7f9aa3200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c204a70 0x7f9a9c158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.570+0000 7f9aa3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a9c204ef0 0x7f9a9c158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.570+0000 7f9aa3200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c06e540 con 0x7f9a9c204a70 2024-05-28T14:07:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.570+0000 7f9aa3200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c06e3c0 con 0x7f9a9c204ef0 2024-05-28T14:07:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.570+0000 7f9aa3200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9a9c06e6c0 con 0x7f9a78003b50 2024-05-28T14:07:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.570+0000 7f9a9be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c204a70 0x7f9a9c158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:07.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.570+0000 7f9a9be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c204a70 0x7f9a9c158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56066/0 (socket says 172.21.15.42:56066) 2024-05-28T14:07:07.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.570+0000 7f9a9be00640 1 -- 172.21.15.42:0/2975820899 learned_addr learned my addr 172.21.15.42:0/2975820899 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:07.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.570+0000 7f9aa1200640 1 --2- 172.21.15.42:0/2975820899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a9c204ef0 0x7f9a9c158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:07.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.574+0000 7f9a9be00640 1 -- 172.21.15.42:0/2975820899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a78003b50 msgr2=0x7f9a9c15b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:07.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.574+0000 7f9a9be00640 1 --2- 172.21.15.42:0/2975820899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a78003b50 0x7f9a9c15b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:07.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.574+0000 7f9a9be00640 1 -- 172.21.15.42:0/2975820899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a9c204ef0 msgr2=0x7f9a9c158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:07.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.574+0000 7f9a9be00640 1 --2- 172.21.15.42:0/2975820899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a9c204ef0 0x7f9a9c158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:07.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.574+0000 7f9a9be00640 1 -- 172.21.15.42:0/2975820899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9c159870 con 0x7f9a9c204a70 2024-05-28T14:07:07.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.574+0000 7f9a9be00640 1 --2- 172.21.15.42:0/2975820899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c204a70 0x7f9a9c158970 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800f7a0 tx=0x7f9a8800fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:07.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.574+0000 7f9a98c00640 1 -- 172.21.15.42:0/2975820899 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8801a070 con 0x7f9a9c204a70 2024-05-28T14:07:07.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.574+0000 7f9aa3200640 1 -- 172.21.15.42:0/2975820899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9c205490 con 0x7f9a9c204a70 2024-05-28T14:07:07.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.574+0000 7f9a98c00640 1 -- 172.21.15.42:0/2975820899 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a88005110 con 0x7f9a9c204a70 2024-05-28T14:07:07.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.574+0000 7f9aa3200640 1 -- 172.21.15.42:0/2975820899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a9c205a50 con 0x7f9a9c204a70 2024-05-28T14:07:07.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.574+0000 7f9a98c00640 1 -- 172.21.15.42:0/2975820899 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a880067e0 con 0x7f9a9c204a70 2024-05-28T14:07:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.578+0000 7f9a91600640 1 -- 172.21.15.42:0/2975820899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a680052a0 con 0x7f9a9c204a70 2024-05-28T14:07:07.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.578+0000 7f9a98c00640 1 -- 172.21.15.42:0/2975820899 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 89262+0+0 (secure 0 0 0) 0x7f9a8801d070 con 0x7f9a9c204a70 2024-05-28T14:07:07.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.582+0000 7f9a98c00640 1 --2- 172.21.15.42:0/2975820899 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9a70069190 0x7f9a7006b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:07.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.582+0000 7f9a98c00640 1 -- 172.21.15.42:0/2975820899 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f9a88084e70 con 0x7f9a9c204a70 2024-05-28T14:07:07.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.590+0000 7f9a98c00640 1 -- 172.21.15.42:0/2975820899 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9a88056c20 con 0x7f9a9c204a70 2024-05-28T14:07:07.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.594+0000 7f9a93e00640 1 --2- 172.21.15.42:0/2975820899 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9a70069190 0x7f9a7006b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.622+0000 7f9a98c00640 1 -- 172.21.15.42:0/2975820899 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 26) ==== 89364+0+0 (secure 0 0 0) 0x7f9a88052b90 con 0x7f9a9c204a70 2024-05-28T14:07:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:07.622+0000 7f9a93e00640 1 --2- 172.21.15.42:0/2975820899 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9a70069190 0x7f9a7006b650 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9a800ad000 tx=0x7f9a80004740 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:08.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:08.022+0000 7f9a91600640 1 -- 172.21.15.42:0/2975820899 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone8113", "target": ["mon-mgr", ""]}) -- 0x7f9a68000c90 con 0x7f9a70069190 2024-05-28T14:07:08.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:08.586+0000 7f9a98c00640 1 -- 172.21.15.42:0/2975820899 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9a68000c90 con 0x7f9a70069190 2024-05-28T14:07:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:08.594+0000 7f9a91600640 1 -- 172.21.15.42:0/2975820899 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9a70069190 msgr2=0x7f9a7006b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:08.594+0000 7f9a91600640 1 --2- 172.21.15.42:0/2975820899 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9a70069190 0x7f9a7006b650 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9a800ad000 tx=0x7f9a80004740 comp rx=0 tx=0).stop 2024-05-28T14:07:08.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:08.594+0000 7f9a91600640 1 -- 172.21.15.42:0/2975820899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c204a70 msgr2=0x7f9a9c158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:08.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:08.594+0000 7f9a91600640 1 --2- 172.21.15.42:0/2975820899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c204a70 0x7f9a9c158970 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800f7a0 tx=0x7f9a8800fba0 comp rx=0 tx=0).stop 2024-05-28T14:07:08.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:08.594+0000 7f9a91600640 1 -- 172.21.15.42:0/2975820899 shutdown_connections 2024-05-28T14:07:08.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:08.594+0000 7f9a91600640 1 --2- 172.21.15.42:0/2975820899 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9a70069190 0x7f9a7006b650 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:08.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:08.594+0000 7f9a91600640 1 --2- 172.21.15.42:0/2975820899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a9c204ef0 0x7f9a9c158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:08.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:08.594+0000 7f9a91600640 1 --2- 172.21.15.42:0/2975820899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c204a70 0x7f9a9c158970 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:08.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:08.594+0000 7f9a91600640 1 --2- 172.21.15.42:0/2975820899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a78003b50 0x7f9a9c15b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:08.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:08.594+0000 7f9a91600640 1 -- 172.21.15.42:0/2975820899 >> 172.21.15.42:0/2975820899 conn(0x7f9a9c06b9f0 msgr2=0x7f9a9c0af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:08.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:08.594+0000 7f9a91600640 1 -- 172.21.15.42:0/2975820899 shutdown_connections 2024-05-28T14:07:08.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:08.594+0000 7f9a91600640 1 -- 172.21.15.42:0/2975820899 wait complete. 2024-05-28T14:07:08.634 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:08.646+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:08.634 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:08.646+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:08.634 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:08.646+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:08.635 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:08.646+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:08.635 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:08.646+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:08.644 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone8114 2024-05-28T14:07:09.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.046+0000 7fd423200640 1 Processor -- start 2024-05-28T14:07:09.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.046+0000 7fd423200640 1 -- start start 2024-05-28T14:07:09.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd423200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd41c04f9b0 0x7fd41c050660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:09.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd423200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd41c0344f0 0x7fd41c04fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:09.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd423200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd41c0062a0 0x7fd41c0541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:09.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd423200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd41c0e0380 con 0x7fd41c0344f0 2024-05-28T14:07:09.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd423200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd41c0539e0 con 0x7fd41c0062a0 2024-05-28T14:07:09.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd423200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd41c0502d0 con 0x7fd41c04f9b0 2024-05-28T14:07:09.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd421400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd41c0344f0 0x7fd41c04fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:09.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd421400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd41c0344f0 0x7fd41c04fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56104/0 (socket says 172.21.15.42:56104) 2024-05-28T14:07:09.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd421400640 1 -- 172.21.15.42:0/2580784262 learned_addr learned my addr 172.21.15.42:0/2580784262 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:09.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd421e00640 1 --2- 172.21.15.42:0/2580784262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd41c04f9b0 0x7fd41c050660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd422800640 1 --2- 172.21.15.42:0/2580784262 >> v2:172.21.15.63:3300/0 conn(0x7fd41c0062a0 0x7fd41c0541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd421400640 1 -- 172.21.15.42:0/2580784262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd41c04f9b0 msgr2=0x7fd41c050660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd421400640 1 --2- 172.21.15.42:0/2580784262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd41c04f9b0 0x7fd41c050660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:09.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd421400640 1 -- 172.21.15.42:0/2580784262 >> v2:172.21.15.63:3300/0 conn(0x7fd41c0062a0 msgr2=0x7fd41c0541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:09.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd421400640 1 --2- 172.21.15.42:0/2580784262 >> v2:172.21.15.63:3300/0 conn(0x7fd41c0062a0 0x7fd41c0541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:09.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd421400640 1 -- 172.21.15.42:0/2580784262 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41c050c90 con 0x7fd41c0344f0 2024-05-28T14:07:09.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.050+0000 7fd421e00640 1 --2- 172.21.15.42:0/2580784262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd41c04f9b0 0x7fd41c050660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:07:09.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.054+0000 7fd421400640 1 --2- 172.21.15.42:0/2580784262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd41c0344f0 0x7fd41c04fd90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd41c048e30 tx=0x7fd41001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3eff8cc4fc6e0e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:09.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.062+0000 7fd420a00640 1 -- 172.21.15.42:0/2580784262 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4100b1070 con 0x7fd41c0344f0 2024-05-28T14:07:09.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.062+0000 7fd420a00640 1 -- 172.21.15.42:0/2580784262 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4100ac620 con 0x7fd41c0344f0 2024-05-28T14:07:09.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.062+0000 7fd420a00640 1 -- 172.21.15.42:0/2580784262 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4100ac9f0 con 0x7fd41c0344f0 2024-05-28T14:07:09.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.062+0000 7fd423200640 1 -- 172.21.15.42:0/2580784262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd41c0344f0 msgr2=0x7fd41c04fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:09.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.062+0000 7fd423200640 1 --2- 172.21.15.42:0/2580784262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd41c0344f0 0x7fd41c04fd90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd41c048e30 tx=0x7fd41001c4f0 comp rx=0 tx=0).stop 2024-05-28T14:07:09.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.062+0000 7fd423200640 1 -- 172.21.15.42:0/2580784262 shutdown_connections 2024-05-28T14:07:09.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.062+0000 7fd423200640 1 --2- 172.21.15.42:0/2580784262 >> v2:172.21.15.63:3300/0 conn(0x7fd41c0062a0 0x7fd41c0541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:09.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.062+0000 7fd423200640 1 --2- 172.21.15.42:0/2580784262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd41c0344f0 0x7fd41c04fd90 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:09.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.062+0000 7fd423200640 1 --2- 172.21.15.42:0/2580784262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd41c04f9b0 0x7fd41c050660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:09.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.062+0000 7fd423200640 1 -- 172.21.15.42:0/2580784262 >> 172.21.15.42:0/2580784262 conn(0x7fd41c055360 msgr2=0x7fd41c055770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:09.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.066+0000 7fd423200640 1 -- 172.21.15.42:0/2580784262 shutdown_connections 2024-05-28T14:07:09.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.066+0000 7fd423200640 1 -- 172.21.15.42:0/2580784262 wait complete. 2024-05-28T14:07:09.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.070+0000 7fd423200640 1 Processor -- start 2024-05-28T14:07:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.070+0000 7fd423200640 1 -- start start 2024-05-28T14:07:09.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.074+0000 7fd423200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c0062a0 0x7fd41c0d8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:09.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.074+0000 7fd423200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd41c04f9b0 0x7fd41c04c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:09.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.074+0000 7fd423200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd41c1801b0 0x7fd41c0e26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:09.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.074+0000 7fd423200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c0084b0 con 0x7fd41c04f9b0 2024-05-28T14:07:09.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.074+0000 7fd423200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c008330 con 0x7fd41c0062a0 2024-05-28T14:07:09.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.074+0000 7fd423200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd41c0546e0 con 0x7fd41c1801b0 2024-05-28T14:07:09.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.074+0000 7fd421e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c0062a0 0x7fd41c0d8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:09.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.074+0000 7fd421400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd41c04f9b0 0x7fd41c04c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:09.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.074+0000 7fd421400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd41c04f9b0 0x7fd41c04c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56110/0 (socket says 172.21.15.42:56110) 2024-05-28T14:07:09.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.074+0000 7fd421400640 1 -- 172.21.15.42:0/1106262879 learned_addr learned my addr 172.21.15.42:0/1106262879 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:09.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.078+0000 7fd421400640 1 -- 172.21.15.42:0/1106262879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd41c1801b0 msgr2=0x7fd41c0e26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:09.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.078+0000 7fd422800640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd41c1801b0 0x7fd41c0e26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:09.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.078+0000 7fd421400640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd41c1801b0 0x7fd41c0e26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:09.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.078+0000 7fd421400640 1 -- 172.21.15.42:0/1106262879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c0062a0 msgr2=0x7fd41c0d8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:09.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.082+0000 7fd421400640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c0062a0 0x7fd41c0d8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:09.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.082+0000 7fd421400640 1 -- 172.21.15.42:0/1106262879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41c0de760 con 0x7fd41c04f9b0 2024-05-28T14:07:09.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.082+0000 7fd422800640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd41c1801b0 0x7fd41c0e26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:07:09.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.082+0000 7fd421e00640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c0062a0 0x7fd41c0d8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:07:09.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.082+0000 7fd421400640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd41c04f9b0 0x7fd41c04c430 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd410000c10 tx=0x7fd410019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:09.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.086+0000 7fd41a000640 1 -- 172.21.15.42:0/1106262879 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4100b1040 con 0x7fd41c04f9b0 2024-05-28T14:07:09.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.086+0000 7fd41a000640 1 -- 172.21.15.42:0/1106262879 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4100bb070 con 0x7fd41c04f9b0 2024-05-28T14:07:09.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.086+0000 7fd41a000640 1 -- 172.21.15.42:0/1106262879 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4100b1040 con 0x7fd41c04f9b0 2024-05-28T14:07:09.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.086+0000 7fd423200640 1 -- 172.21.15.42:0/1106262879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41c04ced0 con 0x7fd41c04f9b0 2024-05-28T14:07:09.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.086+0000 7fd423200640 1 -- 172.21.15.42:0/1106262879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41c04d490 con 0x7fd41c04f9b0 2024-05-28T14:07:09.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.090+0000 7fd41a000640 1 -- 172.21.15.42:0/1106262879 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 89364+0+0 (secure 0 0 0) 0x7fd4100c0460 con 0x7fd41c04f9b0 2024-05-28T14:07:09.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.090+0000 7fd423200640 1 -- 172.21.15.42:0/1106262879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd41c005720 con 0x7fd41c04f9b0 2024-05-28T14:07:09.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.090+0000 7fd41a000640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd4040694b0 0x7fd40406b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:09.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.090+0000 7fd41a000640 1 -- 172.21.15.42:0/1106262879 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fd41011f050 con 0x7fd41c04f9b0 2024-05-28T14:07:09.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.102+0000 7fd421e00640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd4040694b0 0x7fd40406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:09.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.106+0000 7fd41a000640 1 -- 172.21.15.42:0/1106262879 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd410128080 con 0x7fd41c04f9b0 2024-05-28T14:07:09.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.106+0000 7fd421e00640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd4040694b0 0x7fd40406b970 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd414009b10 tx=0x7fd414008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:09.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.574+0000 7fd423200640 1 -- 172.21.15.42:0/1106262879 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone8114", "target": ["mon-mgr", ""]}) -- 0x7fd41c038040 con 0x7fd4040694b0 2024-05-28T14:07:09.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:09.718+0000 7fd41a000640 1 -- 172.21.15.42:0/1106262879 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7fd41011ec10 con 0x7fd41c04f9b0 2024-05-28T14:07:10.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.034+0000 7fd41a000640 1 -- 172.21.15.42:0/1106262879 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd41c038040 con 0x7fd4040694b0 2024-05-28T14:07:10.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.042+0000 7fd423200640 1 -- 172.21.15.42:0/1106262879 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd4040694b0 msgr2=0x7fd40406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:10.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.042+0000 7fd423200640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd4040694b0 0x7fd40406b970 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd414009b10 tx=0x7fd414008040 comp rx=0 tx=0).stop 2024-05-28T14:07:10.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.042+0000 7fd423200640 1 -- 172.21.15.42:0/1106262879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd41c04f9b0 msgr2=0x7fd41c04c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:10.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.042+0000 7fd423200640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd41c04f9b0 0x7fd41c04c430 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd410000c10 tx=0x7fd410019290 comp rx=0 tx=0).stop 2024-05-28T14:07:10.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.042+0000 7fd423200640 1 -- 172.21.15.42:0/1106262879 shutdown_connections 2024-05-28T14:07:10.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.042+0000 7fd423200640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd4040694b0 0x7fd40406b970 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:10.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.042+0000 7fd423200640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd41c1801b0 0x7fd41c0e26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:10.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.042+0000 7fd423200640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd41c04f9b0 0x7fd41c04c430 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:10.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.042+0000 7fd423200640 1 --2- 172.21.15.42:0/1106262879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c0062a0 0x7fd41c0d8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:10.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.042+0000 7fd423200640 1 -- 172.21.15.42:0/1106262879 >> 172.21.15.42:0/1106262879 conn(0x7fd41c055360 msgr2=0x7fd41c052f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:10.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.050+0000 7fd423200640 1 -- 172.21.15.42:0/1106262879 shutdown_connections 2024-05-28T14:07:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.054+0000 7fd423200640 1 -- 172.21.15.42:0/1106262879 wait complete. 2024-05-28T14:07:10.138 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone8115 2024-05-28T14:07:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.506+0000 7fedeaa00640 1 Processor -- start 2024-05-28T14:07:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.506+0000 7fedeaa00640 1 -- start start 2024-05-28T14:07:10.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.506+0000 7fedeaa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede40d3200 0x7fede40d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:10.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.506+0000 7fedeaa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fede40d0bb0 0x7fede406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.506+0000 7fedeaa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fede40d35e0 0x7fede406d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.506+0000 7fedeaa00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fede40cf8b0 con 0x7fede40d0bb0 2024-05-28T14:07:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.506+0000 7fedeaa00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fede40cfb90 con 0x7fede40d35e0 2024-05-28T14:07:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.506+0000 7fedeaa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fede40cfe70 con 0x7fede40d3200 2024-05-28T14:07:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.506+0000 7fede3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fede40d0bb0 0x7fede406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.506+0000 7fede3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fede40d0bb0 0x7fede406cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56120/0 (socket says 172.21.15.42:56120) 2024-05-28T14:07:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.506+0000 7fede3400640 1 -- 172.21.15.42:0/3288251462 learned_addr learned my addr 172.21.15.42:0/3288251462 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.506+0000 7fede8a00640 1 --2- 172.21.15.42:0/3288251462 >> v2:172.21.15.63:3300/0 conn(0x7fede40d35e0 0x7fede406d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.506+0000 7fede3e00640 1 --2- 172.21.15.42:0/3288251462 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede40d3200 0x7fede40d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.506+0000 7fede8a00640 1 -- 172.21.15.42:0/3288251462 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede40d3200 msgr2=0x7fede40d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.510+0000 7fede8a00640 1 --2- 172.21.15.42:0/3288251462 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede40d3200 0x7fede40d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.510+0000 7fede8a00640 1 -- 172.21.15.42:0/3288251462 >> v2:172.21.15.42:3300/0 conn(0x7fede40d0bb0 msgr2=0x7fede406cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:10.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.510+0000 7fede8a00640 1 --2- 172.21.15.42:0/3288251462 >> v2:172.21.15.42:3300/0 conn(0x7fede40d0bb0 0x7fede406cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:10.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.510+0000 7fede8a00640 1 -- 172.21.15.42:0/3288251462 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede40d0150 con 0x7fede40d35e0 2024-05-28T14:07:10.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.510+0000 7fede3400640 1 --2- 172.21.15.42:0/3288251462 >> v2:172.21.15.42:3300/0 conn(0x7fede40d0bb0 0x7fede406cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:07:10.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.510+0000 7fede3e00640 1 --2- 172.21.15.42:0/3288251462 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede40d3200 0x7fede40d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:07:10.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.514+0000 7fede8a00640 1 --2- 172.21.15.42:0/3288251462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede40d35e0 0x7fede406d0a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fede40d4660 tx=0x7fedd001c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e40c4115b0dc8cc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:10.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.514+0000 7fede2a00640 1 -- 172.21.15.42:0/3288251462 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd00b1070 con 0x7fede40d35e0 2024-05-28T14:07:10.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.514+0000 7fede2a00640 1 -- 172.21.15.42:0/3288251462 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd00ac630 con 0x7fede40d35e0 2024-05-28T14:07:10.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.518+0000 7fede2a00640 1 -- 172.21.15.42:0/3288251462 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd00a8050 con 0x7fede40d35e0 2024-05-28T14:07:10.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.518+0000 7fedeaa00640 1 -- 172.21.15.42:0/3288251462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede40d35e0 msgr2=0x7fede406d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:10.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.518+0000 7fedeaa00640 1 --2- 172.21.15.42:0/3288251462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede40d35e0 0x7fede406d0a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fede40d4660 tx=0x7fedd001c3f0 comp rx=0 tx=0).stop 2024-05-28T14:07:10.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.518+0000 7fedeaa00640 1 -- 172.21.15.42:0/3288251462 shutdown_connections 2024-05-28T14:07:10.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.518+0000 7fedeaa00640 1 --2- 172.21.15.42:0/3288251462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede40d35e0 0x7fede406d0a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:10.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.518+0000 7fedeaa00640 1 --2- 172.21.15.42:0/3288251462 >> v2:172.21.15.42:3300/0 conn(0x7fede40d0bb0 0x7fede406cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:10.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.518+0000 7fedeaa00640 1 --2- 172.21.15.42:0/3288251462 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede40d3200 0x7fede40d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:10.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.518+0000 7fedeaa00640 1 -- 172.21.15.42:0/3288251462 >> 172.21.15.42:0/3288251462 conn(0x7fede408bfe0 msgr2=0x7fede40d4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:10.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.518+0000 7fedeaa00640 1 -- 172.21.15.42:0/3288251462 shutdown_connections 2024-05-28T14:07:10.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.518+0000 7fedeaa00640 1 -- 172.21.15.42:0/3288251462 wait complete. 2024-05-28T14:07:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.522+0000 7fedeaa00640 1 Processor -- start 2024-05-28T14:07:10.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.522+0000 7fedeaa00640 1 -- start start 2024-05-28T14:07:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.522+0000 7fedeaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fede40d0bb0 0x7fede4150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.522+0000 7fedeaa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede40d3200 0x7fede41613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.522+0000 7fedeaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede40d35e0 0x7fede41ff630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.522+0000 7fedeaa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fede415e630 con 0x7fede40d0bb0 2024-05-28T14:07:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.522+0000 7fedeaa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fede40d1b30 con 0x7fede40d35e0 2024-05-28T14:07:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.522+0000 7fedeaa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fede415e7b0 con 0x7fede40d3200 2024-05-28T14:07:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.526+0000 7fede3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fede40d0bb0 0x7fede4150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.526+0000 7fede3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fede40d0bb0 0x7fede4150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56134/0 (socket says 172.21.15.42:56134) 2024-05-28T14:07:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.526+0000 7fede3e00640 1 -- 172.21.15.42:0/243872121 learned_addr learned my addr 172.21.15.42:0/243872121 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.526+0000 7fede8a00640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede40d35e0 0x7fede41ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.526+0000 7fede3400640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede40d3200 0x7fede41613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.526+0000 7fede3e00640 1 -- 172.21.15.42:0/243872121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede40d3200 msgr2=0x7fede41613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.526+0000 7fede3e00640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede40d3200 0x7fede41613f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.526+0000 7fede3e00640 1 -- 172.21.15.42:0/243872121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede40d35e0 msgr2=0x7fede41ff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.526+0000 7fede3e00640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede40d35e0 0x7fede41ff630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.526+0000 7fede3e00640 1 -- 172.21.15.42:0/243872121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede415d150 con 0x7fede40d0bb0 2024-05-28T14:07:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.526+0000 7fede8a00640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede40d35e0 0x7fede41ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:07:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.526+0000 7fede3e00640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fede40d0bb0 0x7fede4150c50 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7feddc00cf90 tx=0x7feddc00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.526+0000 7feddbe00640 1 -- 172.21.15.42:0/243872121 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc01a070 con 0x7fede40d0bb0 2024-05-28T14:07:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.530+0000 7feddbe00640 1 -- 172.21.15.42:0/243872121 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feddc00d840 con 0x7fede40d0bb0 2024-05-28T14:07:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.530+0000 7feddbe00640 1 -- 172.21.15.42:0/243872121 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc00dc10 con 0x7fede40d0bb0 2024-05-28T14:07:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.530+0000 7fedeaa00640 1 -- 172.21.15.42:0/243872121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede415d430 con 0x7fede40d0bb0 2024-05-28T14:07:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.530+0000 7fedeaa00640 1 -- 172.21.15.42:0/243872121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede415d9f0 con 0x7fede40d0bb0 2024-05-28T14:07:10.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.530+0000 7fedeaa00640 1 -- 172.21.15.42:0/243872121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede415dbf0 con 0x7fede40d0bb0 2024-05-28T14:07:10.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.538+0000 7feddbe00640 1 -- 172.21.15.42:0/243872121 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7feddc01d070 con 0x7fede40d0bb0 2024-05-28T14:07:10.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.538+0000 7feddbe00640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fedc40694b0 0x7fedc406b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:10.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.538+0000 7feddbe00640 1 -- 172.21.15.42:0/243872121 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7feddc086040 con 0x7fede40d0bb0 2024-05-28T14:07:10.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.554+0000 7feddbe00640 1 -- 172.21.15.42:0/243872121 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feddc057700 con 0x7fede40d0bb0 2024-05-28T14:07:10.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.558+0000 7fede3400640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fedc40694b0 0x7fedc406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:10.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:10.562+0000 7fede3400640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fedc40694b0 0x7fedc406b970 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fede4162310 tx=0x7fedd4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:11.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:11.042+0000 7fedeaa00640 1 -- 172.21.15.42:0/243872121 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone8115", "target": ["mon-mgr", ""]}) -- 0x7fede40a25c0 con 0x7fedc40694b0 2024-05-28T14:07:11.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:11.670+0000 7feddbe00640 1 -- 172.21.15.42:0/243872121 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fede40a25c0 con 0x7fedc40694b0 2024-05-28T14:07:11.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:11.678+0000 7fedeaa00640 1 -- 172.21.15.42:0/243872121 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fedc40694b0 msgr2=0x7fedc406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:11.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:11.678+0000 7fedeaa00640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fedc40694b0 0x7fedc406b970 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fede4162310 tx=0x7fedd4008040 comp rx=0 tx=0).stop 2024-05-28T14:07:11.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:11.678+0000 7fedeaa00640 1 -- 172.21.15.42:0/243872121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fede40d0bb0 msgr2=0x7fede4150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:11.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:11.678+0000 7fedeaa00640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fede40d0bb0 0x7fede4150c50 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7feddc00cf90 tx=0x7feddc00a7d0 comp rx=0 tx=0).stop 2024-05-28T14:07:11.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:11.678+0000 7fedeaa00640 1 -- 172.21.15.42:0/243872121 shutdown_connections 2024-05-28T14:07:11.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:11.678+0000 7fedeaa00640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fedc40694b0 0x7fedc406b970 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:11.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:11.678+0000 7fedeaa00640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede40d35e0 0x7fede41ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:11.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:11.678+0000 7fedeaa00640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede40d3200 0x7fede41613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:11.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:11.678+0000 7fedeaa00640 1 --2- 172.21.15.42:0/243872121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fede40d0bb0 0x7fede4150c50 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:11.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:11.678+0000 7fedeaa00640 1 -- 172.21.15.42:0/243872121 >> 172.21.15.42:0/243872121 conn(0x7fede408bfe0 msgr2=0x7fede415a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:11.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:11.678+0000 7fedeaa00640 1 -- 172.21.15.42:0/243872121 shutdown_connections 2024-05-28T14:07:11.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:11.678+0000 7fedeaa00640 1 -- 172.21.15.42:0/243872121 wait complete. 2024-05-28T14:07:11.735 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:07:12.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.154+0000 7f4153e00640 1 Processor -- start 2024-05-28T14:07:12.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.158+0000 7f4153e00640 1 -- start start 2024-05-28T14:07:12.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.162+0000 7f4153e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f41540b3b80 0x7f415415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:12.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.162+0000 7f4153e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f415415b820 0x7f41540d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:12.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.162+0000 7f4153e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41540d0ec0 con 0x7f415415a810 2024-05-28T14:07:12.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.162+0000 7f4153e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f415415abf0 con 0x7f415415b820 2024-05-28T14:07:12.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.162+0000 7f4153e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f415415de10 con 0x7f41540b3b80 2024-05-28T14:07:12.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.162+0000 7f4153400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f415415b820 0x7f41540d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:12.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.162+0000 7f4153400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f415415b820 0x7f41540d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56146/0 (socket says 172.21.15.42:56146) 2024-05-28T14:07:12.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.162+0000 7f4153400640 1 -- 172.21.15.42:0/333598792 learned_addr learned my addr 172.21.15.42:0/333598792 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:12.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.162+0000 7f4151600640 1 -- 172.21.15.42:0/333598792 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3569767879 0 0) 0x7f41540d0ec0 con 0x7f415415a810 2024-05-28T14:07:12.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.162+0000 7f4153400640 1 -- 172.21.15.42:0/333598792 >> v2:172.21.15.63:3300/0 conn(0x7f41540b3b80 msgr2=0x7f415415cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:12.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.162+0000 7f4152000640 1 --2- 172.21.15.42:0/333598792 >> v2:172.21.15.63:3300/0 conn(0x7f41540b3b80 0x7f415415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:12.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.162+0000 7f4153400640 1 --2- 172.21.15.42:0/333598792 >> v2:172.21.15.63:3300/0 conn(0x7f41540b3b80 0x7f415415cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:12.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.162+0000 7f4153400640 1 -- 172.21.15.42:0/333598792 >> v1:172.21.15.63:6789/0 conn(0x7f415415a810 legacy=0x7f41540b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:12.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.162+0000 7f4153400640 1 -- 172.21.15.42:0/333598792 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415415d200 con 0x7f415415b820 2024-05-28T14:07:12.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.170+0000 7f4153400640 1 --2- 172.21.15.42:0/333598792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f415415b820 0x7f41540d00f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f415406d9d0 tx=0x7f414c0202f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08433388c707044 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:12.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.170+0000 7f4151600640 1 -- 172.21.15.42:0/333598792 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c0b5070 con 0x7f415415b820 2024-05-28T14:07:12.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.170+0000 7f4151600640 1 -- 172.21.15.42:0/333598792 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f414c00a2d0 con 0x7f415415b820 2024-05-28T14:07:12.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.170+0000 7f4151600640 1 -- 172.21.15.42:0/333598792 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c008720 con 0x7f415415b820 2024-05-28T14:07:12.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.174+0000 7f4153e00640 1 -- 172.21.15.42:0/333598792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f415415b820 msgr2=0x7f41540d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:12.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.174+0000 7f4153e00640 1 --2- 172.21.15.42:0/333598792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f415415b820 0x7f41540d00f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f415406d9d0 tx=0x7f414c0202f0 comp rx=0 tx=0).stop 2024-05-28T14:07:12.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.174+0000 7f4153e00640 1 -- 172.21.15.42:0/333598792 shutdown_connections 2024-05-28T14:07:12.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.174+0000 7f4153e00640 1 --2- 172.21.15.42:0/333598792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f415415b820 0x7f41540d00f0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.174+0000 7f4153e00640 1 --2- 172.21.15.42:0/333598792 >> v2:172.21.15.63:3300/0 conn(0x7f41540b3b80 0x7f415415cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:12.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.174+0000 7f4153e00640 1 -- 172.21.15.42:0/333598792 >> 172.21.15.42:0/333598792 conn(0x7f4154072120 msgr2=0x7f41540b55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:12.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.174+0000 7f4153e00640 1 -- 172.21.15.42:0/333598792 shutdown_connections 2024-05-28T14:07:12.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.174+0000 7f4153e00640 1 -- 172.21.15.42:0/333598792 wait complete. 2024-05-28T14:07:12.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.178+0000 7f4153e00640 1 Processor -- start 2024-05-28T14:07:12.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.178+0000 7f4153e00640 1 -- start start 2024-05-28T14:07:12.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41540b3b80 0x7f4154099130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:12.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f415415a810 0x7f4154096290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:12.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f415415b820 0x7f41540967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:12.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f415415daa0 con 0x7f415415b820 2024-05-28T14:07:12.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f415415d7c0 con 0x7f415415a810 2024-05-28T14:07:12.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f415415bc40 con 0x7f41540b3b80 2024-05-28T14:07:12.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f415415b820 0x7f41540967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:12.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f415415b820 0x7f41540967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56156/0 (socket says 172.21.15.42:56156) 2024-05-28T14:07:12.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153400640 1 -- 172.21.15.42:0/2484597030 learned_addr learned my addr 172.21.15.42:0/2484597030 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4152000640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f415415a810 0x7f4154096290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4152a00640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41540b3b80 0x7f4154099130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153400640 1 -- 172.21.15.42:0/2484597030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41540b3b80 msgr2=0x7f4154099130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153400640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41540b3b80 0x7f4154099130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153400640 1 -- 172.21.15.42:0/2484597030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f415415a810 msgr2=0x7f4154096290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153400640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f415415a810 0x7f4154096290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153400640 1 -- 172.21.15.42:0/2484597030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41541fd1e0 con 0x7f415415b820 2024-05-28T14:07:12.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4152a00640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41540b3b80 0x7f4154099130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:07:12.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.182+0000 7f4153400640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f415415b820 0x7f41540967d0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f414c0b3040 tx=0x7f414c01d090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:12.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.186+0000 7f414aa00640 1 -- 172.21.15.42:0/2484597030 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c0b5040 con 0x7f415415b820 2024-05-28T14:07:12.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.186+0000 7f414aa00640 1 -- 172.21.15.42:0/2484597030 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f414c00a0d0 con 0x7f415415b820 2024-05-28T14:07:12.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.186+0000 7f414aa00640 1 -- 172.21.15.42:0/2484597030 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c00a620 con 0x7f415415b820 2024-05-28T14:07:12.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.186+0000 7f4153e00640 1 -- 172.21.15.42:0/2484597030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41541f8df0 con 0x7f415415b820 2024-05-28T14:07:12.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.186+0000 7f4153e00640 1 -- 172.21.15.42:0/2484597030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41541f93b0 con 0x7f415415b820 2024-05-28T14:07:12.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.186+0000 7f413fe00640 1 -- 172.21.15.42:0/2484597030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41280052a0 con 0x7f415415b820 2024-05-28T14:07:12.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.190+0000 7f414aa00640 1 -- 172.21.15.42:0/2484597030 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f414c0c2030 con 0x7f415415b820 2024-05-28T14:07:12.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.194+0000 7f414aa00640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4134069460 0x7f413406b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:12.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.194+0000 7f414aa00640 1 -- 172.21.15.42:0/2484597030 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f414c123a00 con 0x7f415415b820 2024-05-28T14:07:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.202+0000 7f414aa00640 1 -- 172.21.15.42:0/2484597030 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f414c0c9060 con 0x7f415415b820 2024-05-28T14:07:12.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.202+0000 7f4152a00640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4134069460 0x7f413406b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:12.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.234+0000 7f4152a00640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4134069460 0x7f413406b920 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f415415d0f0 tx=0x7f4144009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:12.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.866+0000 7f413fe00640 1 -- 172.21.15.42:0/2484597030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4128005860 con 0x7f415415b820 2024-05-28T14:07:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.870+0000 7f414aa00640 1 -- 172.21.15.42:0/2484597030 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2589 (secure 0 0 0) 0x7f414c019e50 con 0x7f415415b820 2024-05-28T14:07:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:07:12.856 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:07:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:07:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:07:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:07:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:07:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:07:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:07:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:07:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:07:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:07:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:07:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:07:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:07:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:07:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:07:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 554, 2024-05-28T14:07:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:07:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:07:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:07:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:07:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:07:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 76, 2024-05-28T14:07:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:07:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:07:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:07:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:07:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:07:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:07:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:07:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:07:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:07:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:07:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:07:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:07:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:07:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1188, 2024-05-28T14:07:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 4845444629, 2024-05-28T14:07:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 9397317632, 2024-05-28T14:07:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 763696795648, 2024-05-28T14:07:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:07:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_objects_per_sec": 2, 2024-05-28T14:07:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_bytes_per_sec": 11004249, 2024-05-28T14:07:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_keys_per_sec": 0, 2024-05-28T14:07:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects_recovered": 21, 2024-05-28T14:07:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "num_bytes_recovered": 88080384, 2024-05-28T14:07:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "num_keys_recovered": 0, 2024-05-28T14:07:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 216418835, 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 225888021, 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 53, 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 65 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 47, 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5, 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 5, 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:07:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "a", 2024-05-28T14:07:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:07:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 7083 2024-05-28T14:07:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:07:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:07:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:07:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:07:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:07:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:07:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:07:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:07:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:07:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:07:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 12, 2024-05-28T14:07:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:05:28.979186+0000", 2024-05-28T14:07:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:07:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:07:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "a1b82310-2935-40e8-bab3-b505395fb2e3": { 2024-05-28T14:07:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Global Recovery Event (0s)\n [............................] ", 2024-05-28T14:07:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0, 2024-05-28T14:07:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:07:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "d3383476-f722-479f-9c40-e404f8ef3423": { 2024-05-28T14:07:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "4 ongoing clones; Avg progress is 18.489% (3s)\n [=====.......................] (remaining: 16s)", 2024-05-28T14:07:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.18489000201225281, 2024-05-28T14:07:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:07:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:12.863 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:07:12.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.878+0000 7f413fe00640 1 -- 172.21.15.42:0/2484597030 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4134069460 msgr2=0x7f413406b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:12.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.878+0000 7f413fe00640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4134069460 0x7f413406b920 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f415415d0f0 tx=0x7f4144009040 comp rx=0 tx=0).stop 2024-05-28T14:07:12.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.878+0000 7f413fe00640 1 -- 172.21.15.42:0/2484597030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f415415b820 msgr2=0x7f41540967d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:12.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.878+0000 7f413fe00640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f415415b820 0x7f41540967d0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f414c0b3040 tx=0x7f414c01d090 comp rx=0 tx=0).stop 2024-05-28T14:07:12.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.878+0000 7f413fe00640 1 -- 172.21.15.42:0/2484597030 shutdown_connections 2024-05-28T14:07:12.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.878+0000 7f413fe00640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4134069460 0x7f413406b920 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:12.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.878+0000 7f413fe00640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f415415b820 0x7f41540967d0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:12.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.878+0000 7f413fe00640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f415415a810 0x7f4154096290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:12.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.878+0000 7f413fe00640 1 --2- 172.21.15.42:0/2484597030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41540b3b80 0x7f4154099130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:12.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.878+0000 7f413fe00640 1 -- 172.21.15.42:0/2484597030 >> 172.21.15.42:0/2484597030 conn(0x7f4154072120 msgr2=0x7f415406f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:12.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.882+0000 7f413fe00640 1 -- 172.21.15.42:0/2484597030 shutdown_connections 2024-05-28T14:07:12.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:12.882+0000 7f413fe00640 1 -- 172.21.15.42:0/2484597030 wait complete. 2024-05-28T14:07:14.014 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:07:14.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.374+0000 7f5b77200640 1 Processor -- start 2024-05-28T14:07:14.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.378+0000 7f5b77200640 1 -- start start 2024-05-28T14:07:14.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.378+0000 7f5b77200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b7003a5c0 0x7f5b700e5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:14.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.378+0000 7f5b77200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5b70034120 0x7f5b70006920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:14.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.378+0000 7f5b77200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b70006160 con 0x7f5b700e0fc0 2024-05-28T14:07:14.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.378+0000 7f5b77200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5b7004f500 con 0x7f5b70034120 2024-05-28T14:07:14.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.378+0000 7f5b77200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5b7004ecd0 con 0x7f5b7003a5c0 2024-05-28T14:07:14.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.382+0000 7f5b75400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b7003a5c0 0x7f5b700e5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:14.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.382+0000 7f5b7d000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f5b700e0fc0 0x7f5b70039eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51544/0 (socket says 172.21.15.42:51544) 2024-05-28T14:07:14.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.382+0000 7f5b7d000640 1 -- 172.21.15.42:0/366021485 learned_addr learned my addr 172.21.15.42:0/366021485 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:07:14.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.382+0000 7f5b74a00640 1 -- 172.21.15.42:0/366021485 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3088444704 0 0) 0x7f5b70006160 con 0x7f5b700e0fc0 2024-05-28T14:07:14.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.382+0000 7f5b74a00640 1 -- 172.21.15.42:0/366021485 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b5c003530 con 0x7f5b700e0fc0 2024-05-28T14:07:14.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.382+0000 7f5b75400640 1 -- 172.21.15.42:0/366021485 >> v2:172.21.15.42:3300/0 conn(0x7f5b70034120 msgr2=0x7f5b70006920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:14.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.382+0000 7f5b75400640 1 --2- 172.21.15.42:0/366021485 >> v2:172.21.15.42:3300/0 conn(0x7f5b70034120 0x7f5b70006920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:14.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.382+0000 7f5b74a00640 1 -- 172.21.15.42:0/366021485 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3769303360 0 0) 0x7f5b5c003530 con 0x7f5b700e0fc0 2024-05-28T14:07:14.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.382+0000 7f5b75400640 1 -- 172.21.15.42:0/366021485 >> v1:172.21.15.42:6789/0 conn(0x7f5b700e0fc0 legacy=0x7f5b70039eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:14.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.382+0000 7f5b75400640 1 -- 172.21.15.42:0/366021485 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b70006400 con 0x7f5b7003a5c0 2024-05-28T14:07:14.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.394+0000 7f5b75400640 1 --2- 172.21.15.42:0/366021485 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b7003a5c0 0x7f5b700e5620 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5b640097c0 tx=0x7f5b6401ce20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c333f08a01294a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:14.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.394+0000 7f5b74a00640 1 -- 172.21.15.42:0/366021485 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b640b0070 con 0x7f5b7003a5c0 2024-05-28T14:07:14.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.394+0000 7f5b74a00640 1 -- 172.21.15.42:0/366021485 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b640ab400 con 0x7f5b7003a5c0 2024-05-28T14:07:14.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.394+0000 7f5b74a00640 1 -- 172.21.15.42:0/366021485 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b640b0070 con 0x7f5b7003a5c0 2024-05-28T14:07:14.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.394+0000 7f5b77200640 1 -- 172.21.15.42:0/366021485 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b7003a5c0 msgr2=0x7f5b700e5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:14.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.394+0000 7f5b77200640 1 --2- 172.21.15.42:0/366021485 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b7003a5c0 0x7f5b700e5620 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5b640097c0 tx=0x7f5b6401ce20 comp rx=0 tx=0).stop 2024-05-28T14:07:14.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.398+0000 7f5b77200640 1 -- 172.21.15.42:0/366021485 shutdown_connections 2024-05-28T14:07:14.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.398+0000 7f5b77200640 1 --2- 172.21.15.42:0/366021485 >> v2:172.21.15.42:3300/0 conn(0x7f5b70034120 0x7f5b70006920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:14.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.398+0000 7f5b77200640 1 --2- 172.21.15.42:0/366021485 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b7003a5c0 0x7f5b700e5620 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:14.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.398+0000 7f5b77200640 1 -- 172.21.15.42:0/366021485 >> 172.21.15.42:0/366021485 conn(0x7f5b70054ca0 msgr2=0x7f5b700550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:14.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.398+0000 7f5b77200640 1 -- 172.21.15.42:0/366021485 shutdown_connections 2024-05-28T14:07:14.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.402+0000 7f5b77200640 1 -- 172.21.15.42:0/366021485 wait complete. 2024-05-28T14:07:14.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.402+0000 7f5b77200640 1 Processor -- start 2024-05-28T14:07:14.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.406+0000 7f5b77200640 1 -- start start 2024-05-28T14:07:14.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b77200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b70034120 0x7f5b700dcb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:14.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b77200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b7003a5c0 0x7f5b700e29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:14.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b77200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b700e0fc0 0x7f5b70053260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:14.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b77200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5b700077f0 con 0x7f5b70034120 2024-05-28T14:07:14.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b77200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5b70007670 con 0x7f5b7003a5c0 2024-05-28T14:07:14.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b7d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b70034120 0x7f5b700dcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:14.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b7d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b70034120 0x7f5b700dcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56178/0 (socket says 172.21.15.42:56178) 2024-05-28T14:07:14.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b7d000640 1 -- 172.21.15.42:0/2859192502 learned_addr learned my addr 172.21.15.42:0/2859192502 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:14.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b77200640 1 -- 172.21.15.42:0/2859192502 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5b70007970 con 0x7f5b700e0fc0 2024-05-28T14:07:14.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b75e00640 1 --2- 172.21.15.42:0/2859192502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b700e0fc0 0x7f5b70053260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b75e00640 1 -- 172.21.15.42:0/2859192502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b7003a5c0 msgr2=0x7f5b700e29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b75e00640 1 --2- 172.21.15.42:0/2859192502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b7003a5c0 0x7f5b700e29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b75e00640 1 -- 172.21.15.42:0/2859192502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b70034120 msgr2=0x7f5b700dcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b75e00640 1 --2- 172.21.15.42:0/2859192502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b70034120 0x7f5b700dcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.414+0000 7f5b75e00640 1 -- 172.21.15.42:0/2859192502 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b700e32b0 con 0x7f5b700e0fc0 2024-05-28T14:07:14.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.418+0000 7f5b75e00640 1 --2- 172.21.15.42:0/2859192502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b700e0fc0 0x7f5b70053260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5b78051000 tx=0x7f5b78069570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:14.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.422+0000 7f5b6e000640 1 -- 172.21.15.42:0/2859192502 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78050260 con 0x7f5b700e0fc0 2024-05-28T14:07:14.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.422+0000 7f5b6e000640 1 -- 172.21.15.42:0/2859192502 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b78071840 con 0x7f5b700e0fc0 2024-05-28T14:07:14.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.422+0000 7f5b6e000640 1 -- 172.21.15.42:0/2859192502 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78071c70 con 0x7f5b700e0fc0 2024-05-28T14:07:14.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.422+0000 7f5b77200640 1 -- 172.21.15.42:0/2859192502 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b700e3590 con 0x7f5b700e0fc0 2024-05-28T14:07:14.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.422+0000 7f5b77200640 1 -- 172.21.15.42:0/2859192502 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b700e3b50 con 0x7f5b700e0fc0 2024-05-28T14:07:14.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.426+0000 7f5b77200640 1 -- 172.21.15.42:0/2859192502 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b500052a0 con 0x7f5b700e0fc0 2024-05-28T14:07:14.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.426+0000 7f5b6e000640 1 -- 172.21.15.42:0/2859192502 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f5b78094430 con 0x7f5b700e0fc0 2024-05-28T14:07:14.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.430+0000 7f5b6e000640 1 --2- 172.21.15.42:0/2859192502 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b58069530 0x7f5b5806b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:14.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.430+0000 7f5b6e000640 1 -- 172.21.15.42:0/2859192502 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f5b78100990 con 0x7f5b700e0fc0 2024-05-28T14:07:14.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.434+0000 7f5b7d000640 1 --2- 172.21.15.42:0/2859192502 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b58069530 0x7f5b5806b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:14.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.442+0000 7f5b6e000640 1 -- 172.21.15.42:0/2859192502 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b780d26a0 con 0x7f5b700e0fc0 2024-05-28T14:07:14.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:14.458+0000 7f5b7d000640 1 --2- 172.21.15.42:0/2859192502 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b58069530 0x7f5b5806b9f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5b70048520 tx=0x7f5b68000ee0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:15.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:15.106+0000 7f5b77200640 1 -- 172.21.15.42:0/2859192502 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5b50005860 con 0x7f5b700e0fc0 2024-05-28T14:07:15.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:15.110+0000 7f5b6e000640 1 -- 172.21.15.42:0/2859192502 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2374 (secure 0 0 0) 0x7f5b780d1fa0 con 0x7f5b700e0fc0 2024-05-28T14:07:15.096 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:07:15.096 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:07:15.096 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:07:15.096 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:07:15.096 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:07:15.096 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:07:15.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:07:15.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 556, 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:07:15.097 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 76, 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:07:15.098 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1188, 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 4845444629, 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 9397317632, 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 763696795648, 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_objects_per_sec": 2, 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_bytes_per_sec": 9347376, 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_keys_per_sec": 0, 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects_recovered": 21, 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: "num_bytes_recovered": 88080384, 2024-05-28T14:07:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: "num_keys_recovered": 0, 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 183833372, 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 191876815, 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 45, 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 55 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 47, 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5, 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 5, 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:07:15.100 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "a", 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 7083 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:15.101 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:07:15.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:15.102 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:07:15.102 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 12, 2024-05-28T14:07:15.102 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:05:28.979186+0000", 2024-05-28T14:07:15.102 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:07:15.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:15.102 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:07:15.102 INFO:teuthology.orchestra.run.smithi042.stdout: "d3383476-f722-479f-9c40-e404f8ef3423": { 2024-05-28T14:07:15.102 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "4 ongoing clones; Avg progress is 16.602% (5s)\n [====........................] (remaining: 29s)", 2024-05-28T14:07:15.102 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.16602000594139099, 2024-05-28T14:07:15.102 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:07:15.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:15.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:15.103 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:07:15.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:15.114+0000 7f5b77200640 1 -- 172.21.15.42:0/2859192502 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b58069530 msgr2=0x7f5b5806b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:15.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:15.114+0000 7f5b77200640 1 --2- 172.21.15.42:0/2859192502 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b58069530 0x7f5b5806b9f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5b70048520 tx=0x7f5b68000ee0 comp rx=0 tx=0).stop 2024-05-28T14:07:15.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:15.114+0000 7f5b77200640 1 -- 172.21.15.42:0/2859192502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b700e0fc0 msgr2=0x7f5b70053260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:15.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:15.114+0000 7f5b77200640 1 --2- 172.21.15.42:0/2859192502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b700e0fc0 0x7f5b70053260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5b78051000 tx=0x7f5b78069570 comp rx=0 tx=0).stop 2024-05-28T14:07:15.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:15.118+0000 7f5b77200640 1 -- 172.21.15.42:0/2859192502 shutdown_connections 2024-05-28T14:07:15.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:15.118+0000 7f5b77200640 1 --2- 172.21.15.42:0/2859192502 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b58069530 0x7f5b5806b9f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:15.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:15.118+0000 7f5b77200640 1 --2- 172.21.15.42:0/2859192502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b700e0fc0 0x7f5b70053260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:15.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:15.118+0000 7f5b77200640 1 --2- 172.21.15.42:0/2859192502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b7003a5c0 0x7f5b700e29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:15.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:15.118+0000 7f5b77200640 1 --2- 172.21.15.42:0/2859192502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b70034120 0x7f5b700dcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:15.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:15.118+0000 7f5b77200640 1 -- 172.21.15.42:0/2859192502 >> 172.21.15.42:0/2859192502 conn(0x7f5b70054ca0 msgr2=0x7f5b7004d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:15.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:15.118+0000 7f5b77200640 1 -- 172.21.15.42:0/2859192502 shutdown_connections 2024-05-28T14:07:15.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:15.118+0000 7f5b77200640 1 -- 172.21.15.42:0/2859192502 wait complete. 2024-05-28T14:07:16.177 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:07:16.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.606+0000 7f2cd7400640 1 Processor -- start 2024-05-28T14:07:16.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.606+0000 7f2cd7400640 1 -- start start 2024-05-28T14:07:16.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.610+0000 7f2cd7400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cd00b5e10 0x7f2cd015bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:16.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.610+0000 7f2cd7400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2cd00730d0 0x7f2cd0073540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:16.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.610+0000 7f2cd7400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cd00b4270 con 0x7f2cd01519b0 2024-05-28T14:07:16.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.610+0000 7f2cd7400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2cd0073a80 con 0x7f2cd00730d0 2024-05-28T14:07:16.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.610+0000 7f2cd7400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd00afe20 con 0x7f2cd00b5e10 2024-05-28T14:07:16.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.610+0000 7f2cd4a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cd00b5e10 0x7f2cd015bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:16.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.610+0000 7f2cd5400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2cd00730d0 0x7f2cd0073540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:16.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.610+0000 7f2cd4a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cd00b5e10 0x7f2cd015bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42642/0 (socket says 172.21.15.42:42642) 2024-05-28T14:07:16.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.610+0000 7f2cd4a00640 1 -- 172.21.15.42:0/3104607703 learned_addr learned my addr 172.21.15.42:0/3104607703 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:16.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.610+0000 7f2cd4a00640 1 -- 172.21.15.42:0/3104607703 >> v2:172.21.15.63:3300/0 conn(0x7f2cd00730d0 msgr2=0x7f2cd0073540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:16.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.610+0000 7f2cd4a00640 1 --2- 172.21.15.42:0/3104607703 >> v2:172.21.15.63:3300/0 conn(0x7f2cd00730d0 0x7f2cd0073540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:16.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.610+0000 7f2cd4a00640 1 -- 172.21.15.42:0/3104607703 >> v1:172.21.15.63:6789/0 conn(0x7f2cd01519b0 legacy=0x7f2cd00b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:16.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.610+0000 7f2cd4a00640 1 -- 172.21.15.42:0/3104607703 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd00b0100 con 0x7f2cd00b5e10 2024-05-28T14:07:16.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.618+0000 7f2cd4a00640 1 --2- 172.21.15.42:0/3104607703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cd00b5e10 0x7f2cd015bf60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009e50 tx=0x7f2cc001c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f32e15bbe07547ec server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:16.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.618+0000 7f2ccf400640 1 -- 172.21.15.42:0/3104607703 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00a55e0 con 0x7f2cd00b5e10 2024-05-28T14:07:16.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.618+0000 7f2ccf400640 1 -- 172.21.15.42:0/3104607703 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc00a5780 con 0x7f2cd00b5e10 2024-05-28T14:07:16.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.618+0000 7f2ccf400640 1 -- 172.21.15.42:0/3104607703 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00a5b50 con 0x7f2cd00b5e10 2024-05-28T14:07:16.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.622+0000 7f2cd7400640 1 -- 172.21.15.42:0/3104607703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cd00b5e10 msgr2=0x7f2cd015bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:16.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.622+0000 7f2cd7400640 1 --2- 172.21.15.42:0/3104607703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cd00b5e10 0x7f2cd015bf60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009e50 tx=0x7f2cc001c960 comp rx=0 tx=0).stop 2024-05-28T14:07:16.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.622+0000 7f2cd7400640 1 -- 172.21.15.42:0/3104607703 shutdown_connections 2024-05-28T14:07:16.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.622+0000 7f2cd7400640 1 --2- 172.21.15.42:0/3104607703 >> v2:172.21.15.63:3300/0 conn(0x7f2cd00730d0 0x7f2cd0073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:16.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.622+0000 7f2cd7400640 1 --2- 172.21.15.42:0/3104607703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cd00b5e10 0x7f2cd015bf60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:16.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.622+0000 7f2cd7400640 1 -- 172.21.15.42:0/3104607703 >> 172.21.15.42:0/3104607703 conn(0x7f2cd0072220 msgr2=0x7f2cd00b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:16.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.622+0000 7f2cd7400640 1 -- 172.21.15.42:0/3104607703 shutdown_connections 2024-05-28T14:07:16.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.622+0000 7f2cd7400640 1 -- 172.21.15.42:0/3104607703 wait complete. 2024-05-28T14:07:16.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2cd7400640 1 Processor -- start 2024-05-28T14:07:16.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2cd7400640 1 -- start start 2024-05-28T14:07:16.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2cd7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cd00730d0 0x7f2cd00adef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:16.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2cd7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cd00b5e10 0x7f2cd0096710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:16.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2cd7400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cd01519b0 0x7f2cd015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:16.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2cd7400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd00b3510 con 0x7f2cd00730d0 2024-05-28T14:07:16.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2cd7400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd00b3390 con 0x7f2cd00b5e10 2024-05-28T14:07:16.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2cd7400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd00b16d0 con 0x7f2cd01519b0 2024-05-28T14:07:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2cd4a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cd00730d0 0x7f2cd00adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2cd5400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cd01519b0 0x7f2cd015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2ccfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cd00b5e10 0x7f2cd0096710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2cd5400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cd01519b0 0x7f2cd015c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:42644/0 (socket says 172.21.15.42:42644) 2024-05-28T14:07:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2cd5400640 1 -- 172.21.15.42:0/2487739136 learned_addr learned my addr 172.21.15.42:0/2487739136 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2ccfe00640 1 -- 172.21.15.42:0/2487739136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cd01519b0 msgr2=0x7f2cd015c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2ccfe00640 1 --2- 172.21.15.42:0/2487739136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cd01519b0 0x7f2cd015c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2ccfe00640 1 -- 172.21.15.42:0/2487739136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cd00730d0 msgr2=0x7f2cd00adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2ccfe00640 1 --2- 172.21.15.42:0/2487739136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cd00730d0 0x7f2cd00adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:16.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2ccfe00640 1 -- 172.21.15.42:0/2487739136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd015cc10 con 0x7f2cd00b5e10 2024-05-28T14:07:16.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.626+0000 7f2ccfe00640 1 --2- 172.21.15.42:0/2487739136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cd00b5e10 0x7f2cd0096710 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00b410 tx=0x7f2cbc00b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:16.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.630+0000 7f2cccc00640 1 -- 172.21.15.42:0/2487739136 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc015020 con 0x7f2cd00b5e10 2024-05-28T14:07:16.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.630+0000 7f2cccc00640 1 -- 172.21.15.42:0/2487739136 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cbc011830 con 0x7f2cd00b5e10 2024-05-28T14:07:16.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.630+0000 7f2cccc00640 1 -- 172.21.15.42:0/2487739136 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc011c20 con 0x7f2cd00b5e10 2024-05-28T14:07:16.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.630+0000 7f2cd7400640 1 -- 172.21.15.42:0/2487739136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd015f5f0 con 0x7f2cd00b5e10 2024-05-28T14:07:16.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.630+0000 7f2cd7400640 1 -- 172.21.15.42:0/2487739136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd015f890 con 0x7f2cd00b5e10 2024-05-28T14:07:16.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.630+0000 7f2cd7400640 1 -- 172.21.15.42:0/2487739136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd0068a90 con 0x7f2cd00b5e10 2024-05-28T14:07:16.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.630+0000 7f2cccc00640 1 -- 172.21.15.42:0/2487739136 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f2cbc018020 con 0x7f2cd00b5e10 2024-05-28T14:07:16.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.634+0000 7f2cccc00640 1 --2- 172.21.15.42:0/2487739136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2cb0069460 0x7f2cb006b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:16.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.634+0000 7f2cccc00640 1 -- 172.21.15.42:0/2487739136 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f2cbc085de0 con 0x7f2cd00b5e10 2024-05-28T14:07:16.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.638+0000 7f2cccc00640 1 -- 172.21.15.42:0/2487739136 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2cbc01f020 con 0x7f2cd00b5e10 2024-05-28T14:07:16.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.670+0000 7f2cd4a00640 1 --2- 172.21.15.42:0/2487739136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2cb0069460 0x7f2cb006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:16.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:16.674+0000 7f2cd4a00640 1 --2- 172.21.15.42:0/2487739136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2cb0069460 0x7f2cb006b920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00ae000 tx=0x7f2cc00a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:17.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.214+0000 7f2cd7400640 1 -- 172.21.15.42:0/2487739136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2cd00d1080 con 0x7f2cd00b5e10 2024-05-28T14:07:17.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.218+0000 7f2cccc00640 1 -- 172.21.15.42:0/2487739136 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2652 (secure 0 0 0) 0x7f2cbc057df0 con 0x7f2cd00b5e10 2024-05-28T14:07:17.207 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:07:17.207 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:07:17.207 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:07:17.207 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:07:17.207 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:07:17.207 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:07:17.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:07:17.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:17.207 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:07:17.207 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:07:17.208 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:07:17.208 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:07:17.208 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:07:17.208 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:17.208 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:07:17.208 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:07:17.208 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:07:17.208 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:07:17.208 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:17.208 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 558, 2024-05-28T14:07:17.208 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:07:17.208 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:07:17.208 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:07:17.209 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:07:17.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:17.209 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:07:17.209 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 76, 2024-05-28T14:07:17.209 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:07:17.209 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:07:17.209 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:07:17.209 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:07:17.209 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:07:17.209 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:07:17.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:17.209 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:07:17.209 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:07:17.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:07:17.210 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:07:17.210 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:07:17.210 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:17.210 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:17.210 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:07:17.210 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:07:17.210 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 2043, 2024-05-28T14:07:17.210 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 8431786560, 2024-05-28T14:07:17.210 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 16362024960, 2024-05-28T14:07:17.210 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 756732088320, 2024-05-28T14:07:17.210 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:07:17.210 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_objects_per_sec": 2, 2024-05-28T14:07:17.210 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_bytes_per_sec": 8450188, 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_keys_per_sec": 0, 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects_recovered": 21, 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: "num_bytes_recovered": 88080384, 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: "num_keys_recovered": 0, 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 514257007, 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 517527211, 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 125, 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 140 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 47, 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5, 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:07:17.211 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:07:17.212 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:07:17.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:07:17.212 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 5, 2024-05-28T14:07:17.212 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:07:17.212 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "a", 2024-05-28T14:07:17.212 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:07:17.212 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 7083 2024-05-28T14:07:17.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:17.212 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:17.212 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:07:17.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:17.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:07:17.212 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:07:17.212 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:07:17.213 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:07:17.213 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:07:17.213 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:07:17.213 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:07:17.213 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:17.213 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:07:17.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:17.213 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:07:17.213 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 12, 2024-05-28T14:07:17.213 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:05:28.979186+0000", 2024-05-28T14:07:17.213 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:07:17.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:17.213 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:07:17.213 INFO:teuthology.orchestra.run.smithi042.stdout: "346e2288-a00b-4e81-8626-13f7a50e5c02": { 2024-05-28T14:07:17.214 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "5 ongoing+pending clones; Avg progress is 28.594% (1s)\n [========....................] (remaining: 4s)", 2024-05-28T14:07:17.214 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.28593999147415161, 2024-05-28T14:07:17.214 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:07:17.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:17.214 INFO:teuthology.orchestra.run.smithi042.stdout: "d3383476-f722-479f-9c40-e404f8ef3423": { 2024-05-28T14:07:17.214 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "4 ongoing clones; Avg progress is 22.266% (7s)\n [======......................] (remaining: 26s)", 2024-05-28T14:07:17.214 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.22266000509262085, 2024-05-28T14:07:17.214 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:07:17.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:17.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:17.214 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:07:17.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.226+0000 7f2cd7400640 1 -- 172.21.15.42:0/2487739136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2cb0069460 msgr2=0x7f2cb006b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:17.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.226+0000 7f2cd7400640 1 --2- 172.21.15.42:0/2487739136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2cb0069460 0x7f2cb006b920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00ae000 tx=0x7f2cc00a7070 comp rx=0 tx=0).stop 2024-05-28T14:07:17.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.226+0000 7f2cd7400640 1 -- 172.21.15.42:0/2487739136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cd00b5e10 msgr2=0x7f2cd0096710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:17.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.226+0000 7f2cd7400640 1 --2- 172.21.15.42:0/2487739136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cd00b5e10 0x7f2cd0096710 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00b410 tx=0x7f2cbc00b8d0 comp rx=0 tx=0).stop 2024-05-28T14:07:17.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.226+0000 7f2cd7400640 1 -- 172.21.15.42:0/2487739136 shutdown_connections 2024-05-28T14:07:17.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.226+0000 7f2cd7400640 1 --2- 172.21.15.42:0/2487739136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2cb0069460 0x7f2cb006b920 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:17.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.226+0000 7f2cd7400640 1 --2- 172.21.15.42:0/2487739136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cd01519b0 0x7f2cd015c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:17.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.226+0000 7f2cd7400640 1 --2- 172.21.15.42:0/2487739136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cd00b5e10 0x7f2cd0096710 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:17.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.226+0000 7f2cd7400640 1 --2- 172.21.15.42:0/2487739136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cd00730d0 0x7f2cd00adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:17.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.226+0000 7f2cd7400640 1 -- 172.21.15.42:0/2487739136 >> 172.21.15.42:0/2487739136 conn(0x7f2cd0072220 msgr2=0x7f2cd006ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:17.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.226+0000 7f2cd7400640 1 -- 172.21.15.42:0/2487739136 shutdown_connections 2024-05-28T14:07:17.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.226+0000 7f2cd7400640 1 -- 172.21.15.42:0/2487739136 wait complete. 2024-05-28T14:07:17.288 INFO:tasks.cephfs.test_volumes:pevs - {'346e2288-a00b-4e81-8626-13f7a50e5c02': {'message': '5 ongoing+pending clones; Avg progress is 28.594% (1s)\n [========....................] (remaining: 4s)', 'progress': 0.2859399914741516, 'add_to_ceph_s': True}, 'd3383476-f722-479f-9c40-e404f8ef3423': {'message': '4 ongoing clones; Avg progress is 22.266% (7s)\n [======......................] (remaining: 26s)', 'progress': 0.22266000509262085, 'add_to_ceph_s': True}} 2024-05-28T14:07:17.288 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone8116 2024-05-28T14:07:17.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.722+0000 7fccbc200640 1 Processor -- start 2024-05-28T14:07:17.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.722+0000 7fccbc200640 1 -- start start 2024-05-28T14:07:17.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.722+0000 7fccbc200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fccb4162b80 0x7fccb40b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:17.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.722+0000 7fccbc200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fccb406dd10 0x7fccb4071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:17.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.722+0000 7fccbc200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccb40b3d40 con 0x7fccb406e1a0 2024-05-28T14:07:17.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.722+0000 7fccbc200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fccb40ce1c0 con 0x7fccb4162b80 2024-05-28T14:07:17.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.722+0000 7fccbc200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fccb406e760 con 0x7fccb406dd10 2024-05-28T14:07:17.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.726+0000 7fccba200640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fccb406e1a0 0x7fccb40cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:46160/0 (socket says 172.21.15.42:46160) 2024-05-28T14:07:17.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.726+0000 7fccba200640 1 -- 172.21.15.42:0/1451096887 learned_addr learned my addr 172.21.15.42:0/1451096887 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:07:17.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.726+0000 7fccb9800640 1 --2- 172.21.15.42:0/1451096887 >> v2:172.21.15.42:3300/0 conn(0x7fccb4162b80 0x7fccb40b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:17.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.726+0000 7fccb8e00640 1 --2- 172.21.15.42:0/1451096887 >> v2:172.21.15.63:3300/0 conn(0x7fccb406dd10 0x7fccb4071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:17.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.726+0000 7fccb9800640 1 -- 172.21.15.42:0/1451096887 >> v2:172.21.15.63:3300/0 conn(0x7fccb406dd10 msgr2=0x7fccb4071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:17.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.726+0000 7fccb9800640 1 --2- 172.21.15.42:0/1451096887 >> v2:172.21.15.63:3300/0 conn(0x7fccb406dd10 0x7fccb4071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:17.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.726+0000 7fccb9800640 1 -- 172.21.15.42:0/1451096887 >> v1:172.21.15.63:6789/0 conn(0x7fccb406e1a0 legacy=0x7fccb40cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:17.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.726+0000 7fccb9800640 1 -- 172.21.15.42:0/1451096887 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb406ea40 con 0x7fccb4162b80 2024-05-28T14:07:17.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.738+0000 7fccb9800640 1 --2- 172.21.15.42:0/1451096887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb4162b80 0x7fccb40b3800 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fccb40b2180 tx=0x7fcca001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34a6e1149d93899 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:17.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.738+0000 7fccb3e00640 1 -- 172.21.15.42:0/1451096887 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca00b0070 con 0x7fccb4162b80 2024-05-28T14:07:17.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.738+0000 7fccb3e00640 1 -- 172.21.15.42:0/1451096887 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca00ab600 con 0x7fccb4162b80 2024-05-28T14:07:17.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.738+0000 7fccb3e00640 1 -- 172.21.15.42:0/1451096887 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca00ab9d0 con 0x7fccb4162b80 2024-05-28T14:07:17.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.746+0000 7fccbc200640 1 -- 172.21.15.42:0/1451096887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb4162b80 msgr2=0x7fccb40b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:17.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.746+0000 7fccbc200640 1 --2- 172.21.15.42:0/1451096887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb4162b80 0x7fccb40b3800 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fccb40b2180 tx=0x7fcca001c480 comp rx=0 tx=0).stop 2024-05-28T14:07:17.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.750+0000 7fccbc200640 1 -- 172.21.15.42:0/1451096887 shutdown_connections 2024-05-28T14:07:17.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.750+0000 7fccbc200640 1 --2- 172.21.15.42:0/1451096887 >> v2:172.21.15.63:3300/0 conn(0x7fccb406dd10 0x7fccb4071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.750+0000 7fccbc200640 1 --2- 172.21.15.42:0/1451096887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb4162b80 0x7fccb40b3800 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.750+0000 7fccbc200640 1 -- 172.21.15.42:0/1451096887 >> 172.21.15.42:0/1451096887 conn(0x7fccb40b21f0 msgr2=0x7fccb40b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.754+0000 7fccbc200640 1 -- 172.21.15.42:0/1451096887 shutdown_connections 2024-05-28T14:07:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.754+0000 7fccbc200640 1 -- 172.21.15.42:0/1451096887 wait complete. 2024-05-28T14:07:17.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.758+0000 7fccbc200640 1 Processor -- start 2024-05-28T14:07:17.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.762+0000 7fccbc200640 1 -- start start 2024-05-28T14:07:17.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.766+0000 7fccbc200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb406dd10 0x7fccb41415b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:17.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.766+0000 7fccbc200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccb406e1a0 0x7fccb41586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:17.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.766+0000 7fccbc200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb4162b80 0x7fccb406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:17.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.766+0000 7fccb9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb406dd10 0x7fccb41415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:17.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.766+0000 7fccb9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb406dd10 0x7fccb41415b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52812/0 (socket says 172.21.15.42:52812) 2024-05-28T14:07:17.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.766+0000 7fccb9800640 1 -- 172.21.15.42:0/556993295 learned_addr learned my addr 172.21.15.42:0/556993295 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:17.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.766+0000 7fccbc200640 1 -- 172.21.15.42:0/556993295 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fccb40cc920 con 0x7fccb406dd10 2024-05-28T14:07:17.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.766+0000 7fccbc200640 1 -- 172.21.15.42:0/556993295 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fccb4072730 con 0x7fccb406e1a0 2024-05-28T14:07:17.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.766+0000 7fccbc200640 1 -- 172.21.15.42:0/556993295 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fccb415b8f0 con 0x7fccb4162b80 2024-05-28T14:07:17.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.766+0000 7fccb9800640 1 -- 172.21.15.42:0/556993295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb4162b80 msgr2=0x7fccb406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.766+0000 7fccb9800640 1 --2- 172.21.15.42:0/556993295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb4162b80 0x7fccb406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.766+0000 7fccb9800640 1 -- 172.21.15.42:0/556993295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccb406e1a0 msgr2=0x7fccb41586e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-28T14:07:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.766+0000 7fccb9800640 1 --2- 172.21.15.42:0/556993295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccb406e1a0 0x7fccb41586e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.766+0000 7fccb9800640 1 -- 172.21.15.42:0/556993295 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb40b3870 con 0x7fccb406dd10 2024-05-28T14:07:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.770+0000 7fccb9800640 1 --2- 172.21.15.42:0/556993295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb406dd10 0x7fccb41415b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fcca00ae040 tx=0x7fcca0019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:17.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.770+0000 7fccb1600640 1 -- 172.21.15.42:0/556993295 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca00b0040 con 0x7fccb406dd10 2024-05-28T14:07:17.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.770+0000 7fccbc200640 1 -- 172.21.15.42:0/556993295 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb415bb90 con 0x7fccb406dd10 2024-05-28T14:07:17.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.770+0000 7fccbc200640 1 -- 172.21.15.42:0/556993295 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb4159160 con 0x7fccb406dd10 2024-05-28T14:07:17.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.778+0000 7fccb1600640 1 -- 172.21.15.42:0/556993295 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca00b9070 con 0x7fccb406dd10 2024-05-28T14:07:17.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.778+0000 7fccb1600640 1 -- 172.21.15.42:0/556993295 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca00be660 con 0x7fccb406dd10 2024-05-28T14:07:17.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.778+0000 7fccb1600640 1 -- 172.21.15.42:0/556993295 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7fcca00be880 con 0x7fccb406dd10 2024-05-28T14:07:17.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.778+0000 7fccb1600640 1 --2- 172.21.15.42:0/556993295 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcc94069490 0x7fcc9406b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:17.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.778+0000 7fccb1600640 1 -- 172.21.15.42:0/556993295 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fcca011f420 con 0x7fccb406dd10 2024-05-28T14:07:17.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.778+0000 7fccbc200640 1 -- 172.21.15.42:0/556993295 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc900052a0 con 0x7fccb406dd10 2024-05-28T14:07:17.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.794+0000 7fccb1600640 1 -- 172.21.15.42:0/556993295 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcca0016070 con 0x7fccb406dd10 2024-05-28T14:07:17.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.802+0000 7fccb8e00640 1 --2- 172.21.15.42:0/556993295 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcc94069490 0x7fcc9406b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:17.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:17.802+0000 7fccb8e00640 1 --2- 172.21.15.42:0/556993295 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcc94069490 0x7fcc9406b950 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fccb4154090 tx=0x7fcca8009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:18.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:18.378+0000 7fccbc200640 1 -- 172.21.15.42:0/556993295 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone8116", "target": ["mon-mgr", ""]}) -- 0x7fcc90000c90 con 0x7fcc94069490 2024-05-28T14:07:18.926 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:07:18.939 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:07:19.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.098+0000 7fccb1600640 1 -- 172.21.15.42:0/556993295 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcc90000c90 con 0x7fcc94069490 2024-05-28T14:07:19.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.106+0000 7fccbc200640 1 -- 172.21.15.42:0/556993295 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcc94069490 msgr2=0x7fcc9406b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:19.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.106+0000 7fccbc200640 1 --2- 172.21.15.42:0/556993295 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcc94069490 0x7fcc9406b950 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fccb4154090 tx=0x7fcca8009190 comp rx=0 tx=0).stop 2024-05-28T14:07:19.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.106+0000 7fccbc200640 1 -- 172.21.15.42:0/556993295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb406dd10 msgr2=0x7fccb41415b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:19.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.106+0000 7fccbc200640 1 --2- 172.21.15.42:0/556993295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb406dd10 0x7fccb41415b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fcca00ae040 tx=0x7fcca0019220 comp rx=0 tx=0).stop 2024-05-28T14:07:19.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.106+0000 7fccbc200640 1 -- 172.21.15.42:0/556993295 shutdown_connections 2024-05-28T14:07:19.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.106+0000 7fccbc200640 1 --2- 172.21.15.42:0/556993295 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcc94069490 0x7fcc9406b950 secure :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fccb4154090 tx=0x7fcca8009190 comp rx=0 tx=0).stop 2024-05-28T14:07:19.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.106+0000 7fccbc200640 1 --2- 172.21.15.42:0/556993295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccb4162b80 0x7fccb406b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:19.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.106+0000 7fccbc200640 1 --2- 172.21.15.42:0/556993295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccb406e1a0 0x7fccb41586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:19.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.106+0000 7fccbc200640 1 --2- 172.21.15.42:0/556993295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccb406dd10 0x7fccb41415b0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:19.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.106+0000 7fccbc200640 1 -- 172.21.15.42:0/556993295 >> 172.21.15.42:0/556993295 conn(0x7fccb40b21f0 msgr2=0x7fccb4074a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.126+0000 7fccbc200640 1 -- 172.21.15.42:0/556993295 shutdown_connections 2024-05-28T14:07:19.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.130+0000 7fccbc200640 1 -- 172.21.15.42:0/556993295 wait complete. 2024-05-28T14:07:19.224 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone8117 2024-05-28T14:07:19.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.734+0000 7f59d7e00640 1 Processor -- start 2024-05-28T14:07:19.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.738+0000 7f59d7e00640 1 -- start start 2024-05-28T14:07:19.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.738+0000 7f59d7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f59d0148090 0x7f59d00b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:19.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.738+0000 7f59d7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59d00b4c10 0x7f59d0071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:19.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.738+0000 7f59d7e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f59d0072320 0x7f59d01623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:19.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.738+0000 7f59d7e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f59d0148470 con 0x7f59d0148090 2024-05-28T14:07:19.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.738+0000 7f59d7e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f59d00b4730 con 0x7f59d0072320 2024-05-28T14:07:19.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.738+0000 7f59d7e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f59d00b3690 con 0x7f59d00b4c10 2024-05-28T14:07:19.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.738+0000 7f59d5400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f59d0148090 0x7f59d00b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:19.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.738+0000 7f59d5e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f59d0072320 0x7f59d01623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:19.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.738+0000 7f59d5e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f59d0072320 0x7f59d01623b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:58488/0 (socket says 172.21.15.42:58488) 2024-05-28T14:07:19.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.738+0000 7f59d5e00640 1 -- 172.21.15.42:0/2290508294 learned_addr learned my addr 172.21.15.42:0/2290508294 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:19.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.742+0000 7f59d4a00640 1 --2- 172.21.15.42:0/2290508294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59d00b4c10 0x7f59d0071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:19.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.742+0000 7f59d5e00640 1 -- 172.21.15.42:0/2290508294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59d00b4c10 msgr2=0x7f59d0071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:19.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.742+0000 7f59d5e00640 1 --2- 172.21.15.42:0/2290508294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59d00b4c10 0x7f59d0071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:19.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.742+0000 7f59d5e00640 1 -- 172.21.15.42:0/2290508294 >> v2:172.21.15.42:3300/0 conn(0x7f59d0148090 msgr2=0x7f59d00b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:19.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.742+0000 7f59d5e00640 1 --2- 172.21.15.42:0/2290508294 >> v2:172.21.15.42:3300/0 conn(0x7f59d0148090 0x7f59d00b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:19.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.742+0000 7f59d5e00640 1 -- 172.21.15.42:0/2290508294 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d00b3970 con 0x7f59d0072320 2024-05-28T14:07:19.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.746+0000 7f59d5e00640 1 --2- 172.21.15.42:0/2290508294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59d0072320 0x7f59d01623b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f59d0159430 tx=0x7f59c801c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd129bc5ad1b2ef6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:19.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.750+0000 7f59cfe00640 1 -- 172.21.15.42:0/2290508294 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80ac020 con 0x7f59d0072320 2024-05-28T14:07:19.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.750+0000 7f59cfe00640 1 -- 172.21.15.42:0/2290508294 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c80a6420 con 0x7f59d0072320 2024-05-28T14:07:19.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.750+0000 7f59cfe00640 1 -- 172.21.15.42:0/2290508294 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80a67f0 con 0x7f59d0072320 2024-05-28T14:07:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.750+0000 7f59d7e00640 1 -- 172.21.15.42:0/2290508294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59d0072320 msgr2=0x7f59d01623b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.750+0000 7f59d7e00640 1 --2- 172.21.15.42:0/2290508294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59d0072320 0x7f59d01623b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f59d0159430 tx=0x7f59c801c6a0 comp rx=0 tx=0).stop 2024-05-28T14:07:19.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.750+0000 7f59d7e00640 1 -- 172.21.15.42:0/2290508294 shutdown_connections 2024-05-28T14:07:19.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.750+0000 7f59d7e00640 1 --2- 172.21.15.42:0/2290508294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59d0072320 0x7f59d01623b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:19.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.750+0000 7f59d7e00640 1 --2- 172.21.15.42:0/2290508294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59d00b4c10 0x7f59d0071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:19.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.750+0000 7f59d7e00640 1 --2- 172.21.15.42:0/2290508294 >> v2:172.21.15.42:3300/0 conn(0x7f59d0148090 0x7f59d00b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:19.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.750+0000 7f59d7e00640 1 -- 172.21.15.42:0/2290508294 >> 172.21.15.42:0/2290508294 conn(0x7f59d00b21f0 msgr2=0x7f59d00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:19.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.750+0000 7f59d7e00640 1 -- 172.21.15.42:0/2290508294 shutdown_connections 2024-05-28T14:07:19.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.750+0000 7f59d7e00640 1 -- 172.21.15.42:0/2290508294 wait complete. 2024-05-28T14:07:19.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.754+0000 7f59d7e00640 1 Processor -- start 2024-05-28T14:07:19.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.754+0000 7f59d7e00640 1 -- start start 2024-05-28T14:07:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.754+0000 7f59d7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59d0072320 0x7f59d01428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.754+0000 7f59d7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59d0148090 0x7f59d015ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.754+0000 7f59d7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59d015f370 0x7f59d0145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:19.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.754+0000 7f59d7e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f59d00b57e0 con 0x7f59d0148090 2024-05-28T14:07:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.754+0000 7f59d7e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59d00b5660 con 0x7f59d015f370 2024-05-28T14:07:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.754+0000 7f59d7e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f59d00aeb40 con 0x7f59d0072320 2024-05-28T14:07:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59d5400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59d0072320 0x7f59d01428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59d5400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59d0072320 0x7f59d01428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:48550/0 (socket says 172.21.15.42:48550) 2024-05-28T14:07:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59d5400640 1 -- 172.21.15.42:0/1932287292 learned_addr learned my addr 172.21.15.42:0/1932287292 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59d5e00640 1 --2- 172.21.15.42:0/1932287292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59d015f370 0x7f59d0145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:19.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59d4a00640 1 --2- 172.21.15.42:0/1932287292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59d0148090 0x7f59d015ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59d5e00640 1 -- 172.21.15.42:0/1932287292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59d0072320 msgr2=0x7f59d01428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59d5e00640 1 --2- 172.21.15.42:0/1932287292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59d0072320 0x7f59d01428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59d5e00640 1 -- 172.21.15.42:0/1932287292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59d0148090 msgr2=0x7f59d015ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59d5e00640 1 --2- 172.21.15.42:0/1932287292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59d0148090 0x7f59d015ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:19.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59d5e00640 1 -- 172.21.15.42:0/1932287292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d006eef0 con 0x7f59d015f370 2024-05-28T14:07:19.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59d5e00640 1 --2- 172.21.15.42:0/1932287292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59d015f370 0x7f59d0145fc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f59c80aa000 tx=0x7f59c8019440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59cd600640 1 -- 172.21.15.42:0/1932287292 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80b1420 con 0x7f59d015f370 2024-05-28T14:07:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59cd600640 1 -- 172.21.15.42:0/1932287292 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c80b15e0 con 0x7f59d015f370 2024-05-28T14:07:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59cd600640 1 -- 172.21.15.42:0/1932287292 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80b1a10 con 0x7f59d015f370 2024-05-28T14:07:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59d7e00640 1 -- 172.21.15.42:0/1932287292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d00ab560 con 0x7f59d015f370 2024-05-28T14:07:19.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.758+0000 7f59d7e00640 1 -- 172.21.15.42:0/1932287292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d00abb20 con 0x7f59d015f370 2024-05-28T14:07:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.762+0000 7f59d7e00640 1 -- 172.21.15.42:0/1932287292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d00738b0 con 0x7f59d015f370 2024-05-28T14:07:19.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.766+0000 7f59cd600640 1 -- 172.21.15.42:0/1932287292 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f59c80af020 con 0x7f59d015f370 2024-05-28T14:07:19.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.766+0000 7f59cd600640 1 --2- 172.21.15.42:0/1932287292 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59b0069560 0x7f59b006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:19.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.766+0000 7f59cd600640 1 -- 172.21.15.42:0/1932287292 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f59c811f050 con 0x7f59d015f370 2024-05-28T14:07:19.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.770+0000 7f59d5400640 1 --2- 172.21.15.42:0/1932287292 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59b0069560 0x7f59b006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:19.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.770+0000 7f59d5400640 1 --2- 172.21.15.42:0/1932287292 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59b0069560 0x7f59b006ba20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f59d0162410 tx=0x7f59c4009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:19.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:19.774+0000 7f59cd600640 1 -- 172.21.15.42:0/1932287292 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f59c80c0060 con 0x7f59d015f370 2024-05-28T14:07:20.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:20.386+0000 7f59d7e00640 1 -- 172.21.15.42:0/1932287292 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone8117", "target": ["mon-mgr", ""]}) -- 0x7f59d004fdd0 con 0x7f59b0069560 2024-05-28T14:07:22.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.070+0000 7f59cd600640 1 -- 172.21.15.42:0/1932287292 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f59d004fdd0 con 0x7f59b0069560 2024-05-28T14:07:22.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.078+0000 7f59d7e00640 1 -- 172.21.15.42:0/1932287292 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59b0069560 msgr2=0x7f59b006ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:22.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.078+0000 7f59d7e00640 1 --2- 172.21.15.42:0/1932287292 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59b0069560 0x7f59b006ba20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f59d0162410 tx=0x7f59c4009190 comp rx=0 tx=0).stop 2024-05-28T14:07:22.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.078+0000 7f59d7e00640 1 -- 172.21.15.42:0/1932287292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59d015f370 msgr2=0x7f59d0145fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:22.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.078+0000 7f59d7e00640 1 --2- 172.21.15.42:0/1932287292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59d015f370 0x7f59d0145fc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f59c80aa000 tx=0x7f59c8019440 comp rx=0 tx=0).stop 2024-05-28T14:07:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.078+0000 7f59d7e00640 1 -- 172.21.15.42:0/1932287292 shutdown_connections 2024-05-28T14:07:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.078+0000 7f59d7e00640 1 --2- 172.21.15.42:0/1932287292 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59b0069560 0x7f59b006ba20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.078+0000 7f59d7e00640 1 --2- 172.21.15.42:0/1932287292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59d015f370 0x7f59d0145fc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.078+0000 7f59d7e00640 1 --2- 172.21.15.42:0/1932287292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59d0148090 0x7f59d015ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.078+0000 7f59d7e00640 1 --2- 172.21.15.42:0/1932287292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59d0072320 0x7f59d01428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.078+0000 7f59d7e00640 1 -- 172.21.15.42:0/1932287292 >> 172.21.15.42:0/1932287292 conn(0x7f59d00b21f0 msgr2=0x7f59d006f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:22.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.086+0000 7f59d7e00640 1 -- 172.21.15.42:0/1932287292 shutdown_connections 2024-05-28T14:07:22.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.086+0000 7f59d7e00640 1 -- 172.21.15.42:0/1932287292 wait complete. 2024-05-28T14:07:22.146 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone8118 2024-05-28T14:07:22.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.566+0000 7f09b7e00640 1 Processor -- start 2024-05-28T14:07:22.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.570+0000 7f09b7e00640 1 -- start start 2024-05-28T14:07:22.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.574+0000 7f09b7e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f09b0053970 0x7f09b004f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:22.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.574+0000 7f09b7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09b004fd50 0x7f09b00063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:22.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.574+0000 7f09b7e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09b0053e70 con 0x7f09b00e30d0 2024-05-28T14:07:22.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.574+0000 7f09b7e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f09b0037bc0 con 0x7f09b0053970 2024-05-28T14:07:22.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.574+0000 7f09b7e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f09b0054150 con 0x7f09b004fd50 2024-05-28T14:07:22.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.574+0000 7f09b6000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f09b0053970 0x7f09b004f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:22.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.574+0000 7f09b6000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f09b0053970 0x7f09b004f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:58496/0 (socket says 172.21.15.42:58496) 2024-05-28T14:07:22.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.574+0000 7f09b6000640 1 -- 172.21.15.42:0/1714414836 learned_addr learned my addr 172.21.15.42:0/1714414836 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:22.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.574+0000 7f09b6000640 1 -- 172.21.15.42:0/1714414836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09b004fd50 msgr2=0x7f09b00063d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-28T14:07:22.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.574+0000 7f09b6000640 1 --2- 172.21.15.42:0/1714414836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09b004fd50 0x7f09b00063d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:22.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.574+0000 7f09b5600640 1 -- 172.21.15.42:0/1714414836 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3215839107 0 0) 0x7f09b0053e70 con 0x7f09b00e30d0 2024-05-28T14:07:22.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.574+0000 7f09b6000640 1 -- 172.21.15.42:0/1714414836 >> v1:172.21.15.63:6789/0 conn(0x7f09b00e30d0 legacy=0x7f09b003a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:22.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.574+0000 7f09b6000640 1 -- 172.21.15.42:0/1714414836 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b0054430 con 0x7f09b0053970 2024-05-28T14:07:22.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.582+0000 7f09b6000640 1 --2- 172.21.15.42:0/1714414836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09b0053970 0x7f09b004f810 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009870 tx=0x7f09a4024d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c97824cdb7932b0c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:22.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.582+0000 7f09b5600640 1 -- 172.21.15.42:0/1714414836 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a40ba070 con 0x7f09b0053970 2024-05-28T14:07:22.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.582+0000 7f09b5600640 1 -- 172.21.15.42:0/1714414836 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a40adc60 con 0x7f09b0053970 2024-05-28T14:07:22.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.582+0000 7f09b5600640 1 -- 172.21.15.42:0/1714414836 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a40b5660 con 0x7f09b0053970 2024-05-28T14:07:22.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.586+0000 7f09b7e00640 1 -- 172.21.15.42:0/1714414836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09b0053970 msgr2=0x7f09b004f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:22.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.586+0000 7f09b7e00640 1 --2- 172.21.15.42:0/1714414836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09b0053970 0x7f09b004f810 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009870 tx=0x7f09a4024d90 comp rx=0 tx=0).stop 2024-05-28T14:07:22.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.586+0000 7f09b7e00640 1 -- 172.21.15.42:0/1714414836 shutdown_connections 2024-05-28T14:07:22.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.586+0000 7f09b7e00640 1 --2- 172.21.15.42:0/1714414836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09b004fd50 0x7f09b00063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:22.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.586+0000 7f09b7e00640 1 --2- 172.21.15.42:0/1714414836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09b0053970 0x7f09b004f810 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:22.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.586+0000 7f09b7e00640 1 -- 172.21.15.42:0/1714414836 >> 172.21.15.42:0/1714414836 conn(0x7f09b0009fc0 msgr2=0x7f09b000a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:22.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.586+0000 7f09b7e00640 1 -- 172.21.15.42:0/1714414836 shutdown_connections 2024-05-28T14:07:22.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.586+0000 7f09b7e00640 1 -- 172.21.15.42:0/1714414836 wait complete. 2024-05-28T14:07:22.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.590+0000 7f09b7e00640 1 Processor -- start 2024-05-28T14:07:22.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.590+0000 7f09b7e00640 1 -- start start 2024-05-28T14:07:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.594+0000 7f09b7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09b004fd50 0x7f09b004e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.594+0000 7f09b7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09b0053970 0x7f09b017b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.594+0000 7f09b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09b00e30d0 0x7f09b0038be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.594+0000 7f09b7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09b00e30d0 0x7f09b0038be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.594+0000 7f09b7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09b00e30d0 0x7f09b0038be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52842/0 (socket says 172.21.15.42:52842) 2024-05-28T14:07:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.594+0000 7f09b6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09b0053970 0x7f09b017b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.598+0000 7f09b6a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09b004fd50 0x7f09b004e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.598+0000 7f09b6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09b0053970 0x7f09b017b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:58500/0 (socket says 172.21.15.42:58500) 2024-05-28T14:07:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.598+0000 7f09b7400640 1 -- 172.21.15.42:0/2620072884 learned_addr learned my addr 172.21.15.42:0/2620072884 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:22.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.598+0000 7f09b7e00640 1 -- 172.21.15.42:0/2620072884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f09b0007530 con 0x7f09b00e30d0 2024-05-28T14:07:22.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.598+0000 7f09b7e00640 1 -- 172.21.15.42:0/2620072884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f09b00073b0 con 0x7f09b0053970 2024-05-28T14:07:22.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.598+0000 7f09b7e00640 1 -- 172.21.15.42:0/2620072884 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f09b00076b0 con 0x7f09b004fd50 2024-05-28T14:07:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.598+0000 7f09b6000640 1 -- 172.21.15.42:0/2620072884 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09b004fd50 msgr2=0x7f09b004e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.598+0000 7f09b6000640 1 --2- 172.21.15.42:0/2620072884 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09b004fd50 0x7f09b004e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.598+0000 7f09b6000640 1 -- 172.21.15.42:0/2620072884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09b00e30d0 msgr2=0x7f09b0038be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.598+0000 7f09b6000640 1 --2- 172.21.15.42:0/2620072884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09b00e30d0 0x7f09b0038be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.598+0000 7f09b6000640 1 -- 172.21.15.42:0/2620072884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b004d190 con 0x7f09b0053970 2024-05-28T14:07:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.602+0000 7f09b6000640 1 --2- 172.21.15.42:0/2620072884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09b0053970 0x7f09b017b290 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f09a40b4000 tx=0x7f09a4021b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.602+0000 7f09aea00640 1 -- 172.21.15.42:0/2620072884 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a40ba030 con 0x7f09b0053970 2024-05-28T14:07:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.602+0000 7f09aea00640 1 -- 172.21.15.42:0/2620072884 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a40140d0 con 0x7f09b0053970 2024-05-28T14:07:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.602+0000 7f09aea00640 1 -- 172.21.15.42:0/2620072884 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a40142e0 con 0x7f09b0053970 2024-05-28T14:07:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.602+0000 7f09b7e00640 1 -- 172.21.15.42:0/2620072884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b004d470 con 0x7f09b0053970 2024-05-28T14:07:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.602+0000 7f09b7e00640 1 -- 172.21.15.42:0/2620072884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b004da30 con 0x7f09b0053970 2024-05-28T14:07:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.606+0000 7f09a3e00640 1 -- 172.21.15.42:0/2620072884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f098c0052a0 con 0x7f09b0053970 2024-05-28T14:07:22.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.614+0000 7f09aea00640 1 -- 172.21.15.42:0/2620072884 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f09a40bd070 con 0x7f09b0053970 2024-05-28T14:07:22.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.622+0000 7f09aea00640 1 --2- 172.21.15.42:0/2620072884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0998069460 0x7f099806b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:22.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.622+0000 7f09aea00640 1 -- 172.21.15.42:0/2620072884 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f09a411f290 con 0x7f09b0053970 2024-05-28T14:07:22.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.626+0000 7f09b6a00640 1 --2- 172.21.15.42:0/2620072884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0998069460 0x7f099806b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:22.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.626+0000 7f09aea00640 1 -- 172.21.15.42:0/2620072884 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f09a40c4020 con 0x7f09b0053970 2024-05-28T14:07:22.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:22.626+0000 7f09b6a00640 1 --2- 172.21.15.42:0/2620072884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0998069460 0x7f099806b920 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f09a8000f70 tx=0x7f09a8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:23.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:23.174+0000 7f09a3e00640 1 -- 172.21.15.42:0/2620072884 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone8118", "target": ["mon-mgr", ""]}) -- 0x7f098c000c90 con 0x7f0998069460 2024-05-28T14:07:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:23.586+0000 7f09aea00640 1 -- 172.21.15.42:0/2620072884 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f098c000c90 con 0x7f0998069460 2024-05-28T14:07:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:23.594+0000 7f09a3e00640 1 -- 172.21.15.42:0/2620072884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0998069460 msgr2=0x7f099806b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:23.594+0000 7f09a3e00640 1 --2- 172.21.15.42:0/2620072884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0998069460 0x7f099806b920 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f09a8000f70 tx=0x7f09a8009040 comp rx=0 tx=0).stop 2024-05-28T14:07:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:23.594+0000 7f09a3e00640 1 -- 172.21.15.42:0/2620072884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09b0053970 msgr2=0x7f09b017b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:23.594+0000 7f09a3e00640 1 --2- 172.21.15.42:0/2620072884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09b0053970 0x7f09b017b290 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f09a40b4000 tx=0x7f09a4021b30 comp rx=0 tx=0).stop 2024-05-28T14:07:23.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:23.598+0000 7f09a3e00640 1 -- 172.21.15.42:0/2620072884 shutdown_connections 2024-05-28T14:07:23.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:23.598+0000 7f09a3e00640 1 --2- 172.21.15.42:0/2620072884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0998069460 0x7f099806b920 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:23.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:23.598+0000 7f09a3e00640 1 --2- 172.21.15.42:0/2620072884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09b00e30d0 0x7f09b0038be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:23.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:23.598+0000 7f09a3e00640 1 --2- 172.21.15.42:0/2620072884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09b0053970 0x7f09b017b290 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:23.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:23.598+0000 7f09a3e00640 1 --2- 172.21.15.42:0/2620072884 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09b004fd50 0x7f09b004e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:23.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:23.598+0000 7f09a3e00640 1 -- 172.21.15.42:0/2620072884 >> 172.21.15.42:0/2620072884 conn(0x7f09b0009fc0 msgr2=0x7f09b004ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:23.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:23.598+0000 7f09a3e00640 1 -- 172.21.15.42:0/2620072884 shutdown_connections 2024-05-28T14:07:23.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:23.598+0000 7f09a3e00640 1 -- 172.21.15.42:0/2620072884 wait complete. 2024-05-28T14:07:23.643 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone8119 2024-05-28T14:07:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.014+0000 7fb5af200640 1 Processor -- start 2024-05-28T14:07:24.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.014+0000 7fb5af200640 1 -- start start 2024-05-28T14:07:24.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.014+0000 7fb5af200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb5b0073cd0 0x7fb5b00b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:24.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.014+0000 7fb5af200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5b0098e00 0x7fb5b0099270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:24.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.014+0000 7fb5af200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5b00cd5b0 con 0x7fb5b00731e0 2024-05-28T14:07:24.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.014+0000 7fb5af200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb5b00ad040 con 0x7fb5b0073cd0 2024-05-28T14:07:24.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.014+0000 7fb5af200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b00997b0 con 0x7fb5b0098e00 2024-05-28T14:07:24.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.018+0000 7fb5ade00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fb5b00731e0 0x7fb5b00735c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:46176/0 (socket says 172.21.15.42:46176) 2024-05-28T14:07:24.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.018+0000 7fb5ade00640 1 -- 172.21.15.42:0/2161599986 learned_addr learned my addr 172.21.15.42:0/2161599986 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:07:24.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.018+0000 7fb5ae800640 1 --2- 172.21.15.42:0/2161599986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5b0098e00 0x7fb5b0099270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:24.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.018+0000 7fb5ad400640 1 --2- 172.21.15.42:0/2161599986 >> v2:172.21.15.63:3300/0 conn(0x7fb5b0073cd0 0x7fb5b00b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:24.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.018+0000 7fb5aca00640 1 -- 172.21.15.42:0/2161599986 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2824607004 0 0) 0x7fb5b00cd5b0 con 0x7fb5b00731e0 2024-05-28T14:07:24.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.046+0000 7fb5aca00640 1 -- 172.21.15.42:0/2161599986 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb594003530 con 0x7fb5b00731e0 2024-05-28T14:07:24.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.046+0000 7fb5ae800640 1 -- 172.21.15.42:0/2161599986 >> v2:172.21.15.63:3300/0 conn(0x7fb5b0073cd0 msgr2=0x7fb5b00b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:24.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.046+0000 7fb5ae800640 1 --2- 172.21.15.42:0/2161599986 >> v2:172.21.15.63:3300/0 conn(0x7fb5b0073cd0 0x7fb5b00b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:24.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.046+0000 7fb5ae800640 1 -- 172.21.15.42:0/2161599986 >> v1:172.21.15.63:6789/0 conn(0x7fb5b00731e0 legacy=0x7fb5b00735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:24.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.046+0000 7fb5ae800640 1 -- 172.21.15.42:0/2161599986 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b00cd890 con 0x7fb5b0098e00 2024-05-28T14:07:24.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.054+0000 7fb5ae800640 1 --2- 172.21.15.42:0/2161599986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5b0098e00 0x7fb5b0099270 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800bfa0 tx=0x7fb5a801e980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d84d41ec0f2f6f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:24.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.054+0000 7fb5aca00640 1 -- 172.21.15.42:0/2161599986 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0001680 con 0x7fb5b0098e00 2024-05-28T14:07:24.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.054+0000 7fb5aca00640 1 -- 172.21.15.42:0/2161599986 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a800b0d0 con 0x7fb5b0098e00 2024-05-28T14:07:24.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.054+0000 7fb5aca00640 1 -- 172.21.15.42:0/2161599986 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80a77d0 con 0x7fb5b0098e00 2024-05-28T14:07:24.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.058+0000 7fb5af200640 1 -- 172.21.15.42:0/2161599986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5b0098e00 msgr2=0x7fb5b0099270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:24.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.058+0000 7fb5af200640 1 --2- 172.21.15.42:0/2161599986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5b0098e00 0x7fb5b0099270 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800bfa0 tx=0x7fb5a801e980 comp rx=0 tx=0).stop 2024-05-28T14:07:24.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.058+0000 7fb5af200640 1 -- 172.21.15.42:0/2161599986 shutdown_connections 2024-05-28T14:07:24.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.058+0000 7fb5af200640 1 --2- 172.21.15.42:0/2161599986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5b0098e00 0x7fb5b0099270 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:24.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.058+0000 7fb5af200640 1 --2- 172.21.15.42:0/2161599986 >> v2:172.21.15.63:3300/0 conn(0x7fb5b0073cd0 0x7fb5b00b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:24.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.058+0000 7fb5af200640 1 -- 172.21.15.42:0/2161599986 >> 172.21.15.42:0/2161599986 conn(0x7fb5b00b1d60 msgr2=0x7fb5b00b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:24.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.058+0000 7fb5af200640 1 -- 172.21.15.42:0/2161599986 shutdown_connections 2024-05-28T14:07:24.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.058+0000 7fb5af200640 1 -- 172.21.15.42:0/2161599986 wait complete. 2024-05-28T14:07:24.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.062+0000 7fb5af200640 1 Processor -- start 2024-05-28T14:07:24.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.062+0000 7fb5af200640 1 -- start start 2024-05-28T14:07:24.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.062+0000 7fb5af200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5b00731e0 0x7fb5b009dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:24.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.062+0000 7fb5af200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5b0073cd0 0x7fb5b008a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:24.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.062+0000 7fb5af200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5b0098e00 0x7fb5b009ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:24.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.062+0000 7fb5af200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b00ae350 con 0x7fb5b00731e0 2024-05-28T14:07:24.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.062+0000 7fb5af200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b00ae1d0 con 0x7fb5b0073cd0 2024-05-28T14:07:24.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.062+0000 7fb5af200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b00ae4d0 con 0x7fb5b0098e00 2024-05-28T14:07:24.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.062+0000 7fb5ade00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5b00731e0 0x7fb5b009dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:24.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.062+0000 7fb5ad400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5b0073cd0 0x7fb5b008a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:24.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.062+0000 7fb5ade00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5b00731e0 0x7fb5b009dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52852/0 (socket says 172.21.15.42:52852) 2024-05-28T14:07:24.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.062+0000 7fb5ade00640 1 -- 172.21.15.42:0/1296503080 learned_addr learned my addr 172.21.15.42:0/1296503080 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.062+0000 7fb5ad400640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5b0073cd0 0x7fb5b008a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:58516/0 (socket says 172.21.15.42:58516) 2024-05-28T14:07:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.062+0000 7fb5ae800640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5b0098e00 0x7fb5b009ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.066+0000 7fb5ad400640 1 -- 172.21.15.42:0/1296503080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5b0098e00 msgr2=0x7fb5b009ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.066+0000 7fb5ad400640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5b0098e00 0x7fb5b009ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.066+0000 7fb5ad400640 1 -- 172.21.15.42:0/1296503080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5b00731e0 msgr2=0x7fb5b009dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.066+0000 7fb5ad400640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5b00731e0 0x7fb5b009dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.066+0000 7fb5ad400640 1 -- 172.21.15.42:0/1296503080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b009cc80 con 0x7fb5b0073cd0 2024-05-28T14:07:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.066+0000 7fb5ae800640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5b0098e00 0x7fb5b009ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:07:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.066+0000 7fb5ad400640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5b0073cd0 0x7fb5b008a870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00da60 tx=0x7fb59c00de60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.066+0000 7fb5a6000640 1 -- 172.21.15.42:0/1296503080 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c00cce0 con 0x7fb5b0073cd0 2024-05-28T14:07:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.066+0000 7fb5a6000640 1 -- 172.21.15.42:0/1296503080 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59c00a470 con 0x7fb5b0073cd0 2024-05-28T14:07:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.066+0000 7fb5a6000640 1 -- 172.21.15.42:0/1296503080 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c00a820 con 0x7fb5b0073cd0 2024-05-28T14:07:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.066+0000 7fb5af200640 1 -- 172.21.15.42:0/1296503080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b009cf60 con 0x7fb5b0073cd0 2024-05-28T14:07:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.066+0000 7fb5af200640 1 -- 172.21.15.42:0/1296503080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b009d520 con 0x7fb5b0073cd0 2024-05-28T14:07:24.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.070+0000 7fb5af200640 1 -- 172.21.15.42:0/1296503080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b004fcf0 con 0x7fb5b0073cd0 2024-05-28T14:07:24.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.070+0000 7fb5a6000640 1 -- 172.21.15.42:0/1296503080 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7fb59c008040 con 0x7fb5b0073cd0 2024-05-28T14:07:24.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.074+0000 7fb5a6000640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb590069460 0x7fb59006b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:24.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.074+0000 7fb5a6000640 1 -- 172.21.15.42:0/1296503080 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fb59c086340 con 0x7fb5b0073cd0 2024-05-28T14:07:24.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.078+0000 7fb5a6000640 1 -- 172.21.15.42:0/1296503080 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb59c08f020 con 0x7fb5b0073cd0 2024-05-28T14:07:24.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.082+0000 7fb5ade00640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb590069460 0x7fb59006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:24.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.086+0000 7fb5ade00640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb590069460 0x7fb59006b920 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0001ec0 tx=0x7fb5a0001bb0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:24.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:24.554+0000 7fb5af200640 1 -- 172.21.15.42:0/1296503080 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone8119", "target": ["mon-mgr", ""]}) -- 0x7fb5b00ce2e0 con 0x7fb590069460 2024-05-28T14:07:25.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.266+0000 7fb5a6000640 1 -- 172.21.15.42:0/1296503080 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb5b00ce2e0 con 0x7fb590069460 2024-05-28T14:07:25.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.278+0000 7fb5af200640 1 -- 172.21.15.42:0/1296503080 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb590069460 msgr2=0x7fb59006b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:25.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.278+0000 7fb5af200640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb590069460 0x7fb59006b920 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0001ec0 tx=0x7fb5a0001bb0 comp rx=0 tx=0).stop 2024-05-28T14:07:25.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.278+0000 7fb5af200640 1 -- 172.21.15.42:0/1296503080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5b0073cd0 msgr2=0x7fb5b008a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:25.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.278+0000 7fb5af200640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5b0073cd0 0x7fb5b008a870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00da60 tx=0x7fb59c00de60 comp rx=0 tx=0).stop 2024-05-28T14:07:25.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.278+0000 7fb5af200640 1 -- 172.21.15.42:0/1296503080 shutdown_connections 2024-05-28T14:07:25.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.282+0000 7fb5af200640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb590069460 0x7fb59006b920 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:25.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.282+0000 7fb5af200640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5b0098e00 0x7fb5b009ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:25.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.282+0000 7fb5af200640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5b0073cd0 0x7fb5b008a870 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:25.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.282+0000 7fb5af200640 1 --2- 172.21.15.42:0/1296503080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5b00731e0 0x7fb5b009dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:25.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.282+0000 7fb5af200640 1 -- 172.21.15.42:0/1296503080 >> 172.21.15.42:0/1296503080 conn(0x7fb5b00b1d60 msgr2=0x7fb5b00b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:25.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.282+0000 7fb5af200640 1 -- 172.21.15.42:0/1296503080 shutdown_connections 2024-05-28T14:07:25.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.282+0000 7fb5af200640 1 -- 172.21.15.42:0/1296503080 wait complete. 2024-05-28T14:07:25.328 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone81110 2024-05-28T14:07:25.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.734+0000 7f277b000640 1 Processor -- start 2024-05-28T14:07:25.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.734+0000 7f277b000640 1 -- start start 2024-05-28T14:07:25.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.738+0000 7f277b000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2774148090 0x7f27740b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.738+0000 7f277b000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f27740724f0 0x7f27740aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.738+0000 7f277b000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27740b3690 con 0x7f27740b4c10 2024-05-28T14:07:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.738+0000 7f277b000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f2774148470 con 0x7f27740724f0 2024-05-28T14:07:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.738+0000 7f277b000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f27740b4730 con 0x7f2774148090 2024-05-28T14:07:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.738+0000 7f2779000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f27740724f0 0x7f27740aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.738+0000 7f2779000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f27740724f0 0x7f27740aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52864/0 (socket says 172.21.15.42:52864) 2024-05-28T14:07:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.738+0000 7f2779000640 1 -- 172.21.15.42:0/1059429548 learned_addr learned my addr 172.21.15.42:0/1059429548 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.738+0000 7f2773e00640 1 --2- 172.21.15.42:0/1059429548 >> v2:172.21.15.63:3300/0 conn(0x7f2774148090 0x7f27740b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.738+0000 7f2773e00640 1 -- 172.21.15.42:0/1059429548 >> v2:172.21.15.42:3300/0 conn(0x7f27740724f0 msgr2=0x7f27740aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.738+0000 7f2773e00640 1 --2- 172.21.15.42:0/1059429548 >> v2:172.21.15.42:3300/0 conn(0x7f27740724f0 0x7f27740aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:25.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.738+0000 7f2773e00640 1 -- 172.21.15.42:0/1059429548 >> v1:172.21.15.42:6789/0 conn(0x7f27740b4c10 legacy=0x7f2774071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:25.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.738+0000 7f2773e00640 1 -- 172.21.15.42:0/1059429548 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27740b3970 con 0x7f2774148090 2024-05-28T14:07:25.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.746+0000 7f2773e00640 1 --2- 172.21.15.42:0/1059429548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2774148090 0x7f27740b5120 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2774071730 tx=0x7f276401c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e95ffb959389b9e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:25.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.750+0000 7f2773400640 1 -- 172.21.15.42:0/1059429548 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27640b2070 con 0x7f2774148090 2024-05-28T14:07:25.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.750+0000 7f2773400640 1 -- 172.21.15.42:0/1059429548 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27640ad680 con 0x7f2774148090 2024-05-28T14:07:25.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.750+0000 7f2773400640 1 -- 172.21.15.42:0/1059429548 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27640ada50 con 0x7f2774148090 2024-05-28T14:07:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.754+0000 7f277b000640 1 -- 172.21.15.42:0/1059429548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2774148090 msgr2=0x7f27740b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.754+0000 7f277b000640 1 --2- 172.21.15.42:0/1059429548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2774148090 0x7f27740b5120 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2774071730 tx=0x7f276401c380 comp rx=0 tx=0).stop 2024-05-28T14:07:25.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.754+0000 7f277b000640 1 -- 172.21.15.42:0/1059429548 shutdown_connections 2024-05-28T14:07:25.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.754+0000 7f277b000640 1 --2- 172.21.15.42:0/1059429548 >> v2:172.21.15.42:3300/0 conn(0x7f27740724f0 0x7f27740aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:25.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.754+0000 7f277b000640 1 --2- 172.21.15.42:0/1059429548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2774148090 0x7f27740b5120 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:25.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.754+0000 7f277b000640 1 -- 172.21.15.42:0/1059429548 >> 172.21.15.42:0/1059429548 conn(0x7f27740b21f0 msgr2=0x7f27740b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:25.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.754+0000 7f277b000640 1 -- 172.21.15.42:0/1059429548 shutdown_connections 2024-05-28T14:07:25.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.754+0000 7f277b000640 1 -- 172.21.15.42:0/1059429548 wait complete. 2024-05-28T14:07:25.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.758+0000 7f277b000640 1 Processor -- start 2024-05-28T14:07:25.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.758+0000 7f277b000640 1 -- start start 2024-05-28T14:07:25.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.762+0000 7f277b000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27740724f0 0x7f27741584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:25.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.762+0000 7f277b000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2774148090 0x7f27740cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:25.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.762+0000 7f277b000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27741402b0 0x7f27740cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:25.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.762+0000 7f277b000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f27740b57e0 con 0x7f27740724f0 2024-05-28T14:07:25.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.762+0000 7f277b000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f27740b5660 con 0x7f2774148090 2024-05-28T14:07:25.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.762+0000 7f277b000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f27740af890 con 0x7f27741402b0 2024-05-28T14:07:25.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.762+0000 7f2779000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27741402b0 0x7f27740cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:25.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.762+0000 7f2779000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27741402b0 0x7f27740cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:48594/0 (socket says 172.21.15.42:48594) 2024-05-28T14:07:25.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.762+0000 7f2779000640 1 -- 172.21.15.42:0/2585297792 learned_addr learned my addr 172.21.15.42:0/2585297792 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:25.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.766+0000 7f276be00640 1 --2- 172.21.15.42:0/2585297792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2774148090 0x7f27740cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:25.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.766+0000 7f2779000640 1 -- 172.21.15.42:0/2585297792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2774148090 msgr2=0x7f27740cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:25.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.766+0000 7f2779000640 1 --2- 172.21.15.42:0/2585297792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2774148090 0x7f27740cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:25.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.766+0000 7f2779000640 1 -- 172.21.15.42:0/2585297792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27740724f0 msgr2=0x7f27741584a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-28T14:07:25.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.766+0000 7f2779000640 1 --2- 172.21.15.42:0/2585297792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27740724f0 0x7f27741584a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:25.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.766+0000 7f2779000640 1 -- 172.21.15.42:0/2585297792 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27740cf3b0 con 0x7f27741402b0 2024-05-28T14:07:25.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.766+0000 7f276be00640 1 --2- 172.21.15.42:0/2585297792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2774148090 0x7f27740cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:07:25.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.766+0000 7f2779000640 1 --2- 172.21.15.42:0/2585297792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27741402b0 0x7f27740cdc70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f276c007b20 tx=0x7f276c00d0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:25.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.766+0000 7f2770c00640 1 -- 172.21.15.42:0/2585297792 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c00db10 con 0x7f27741402b0 2024-05-28T14:07:25.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.766+0000 7f2770c00640 1 -- 172.21.15.42:0/2585297792 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f276c00dcb0 con 0x7f27741402b0 2024-05-28T14:07:25.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.766+0000 7f277b000640 1 -- 172.21.15.42:0/2585297792 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277413f7d0 con 0x7f27741402b0 2024-05-28T14:07:25.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.766+0000 7f2770c00640 1 -- 172.21.15.42:0/2585297792 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c00de60 con 0x7f27741402b0 2024-05-28T14:07:25.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.766+0000 7f277b000640 1 -- 172.21.15.42:0/2585297792 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f277413fd90 con 0x7f27741402b0 2024-05-28T14:07:25.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.770+0000 7f277b000640 1 -- 172.21.15.42:0/2585297792 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27740738b0 con 0x7f27741402b0 2024-05-28T14:07:25.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.774+0000 7f2770c00640 1 -- 172.21.15.42:0/2585297792 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f276c01a020 con 0x7f27741402b0 2024-05-28T14:07:25.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.774+0000 7f2770c00640 1 --2- 172.21.15.42:0/2585297792 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f27500694b0 0x7f275006b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:25.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.774+0000 7f2770c00640 1 -- 172.21.15.42:0/2585297792 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f276c085300 con 0x7f27741402b0 2024-05-28T14:07:25.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.782+0000 7f2773e00640 1 --2- 172.21.15.42:0/2585297792 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f27500694b0 0x7f275006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:25.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.782+0000 7f2773e00640 1 --2- 172.21.15.42:0/2585297792 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f27500694b0 0x7f275006b970 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f27640af000 tx=0x7f2764008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:25.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:25.786+0000 7f2770c00640 1 -- 172.21.15.42:0/2585297792 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f276c010070 con 0x7f27741402b0 2024-05-28T14:07:26.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:26.274+0000 7f277b000640 1 -- 172.21.15.42:0/2585297792 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone81110", "target": ["mon-mgr", ""]}) -- 0x7f2774161f80 con 0x7f27500694b0 2024-05-28T14:07:27.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.058+0000 7f2770c00640 1 -- 172.21.15.42:0/2585297792 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2774161f80 con 0x7f27500694b0 2024-05-28T14:07:27.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.070+0000 7f277b000640 1 -- 172.21.15.42:0/2585297792 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f27500694b0 msgr2=0x7f275006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:27.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.070+0000 7f277b000640 1 --2- 172.21.15.42:0/2585297792 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f27500694b0 0x7f275006b970 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f27640af000 tx=0x7f2764008040 comp rx=0 tx=0).stop 2024-05-28T14:07:27.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.070+0000 7f277b000640 1 -- 172.21.15.42:0/2585297792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27741402b0 msgr2=0x7f27740cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:27.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.070+0000 7f277b000640 1 --2- 172.21.15.42:0/2585297792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27741402b0 0x7f27740cdc70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f276c007b20 tx=0x7f276c00d0d0 comp rx=0 tx=0).stop 2024-05-28T14:07:27.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.070+0000 7f277b000640 1 -- 172.21.15.42:0/2585297792 shutdown_connections 2024-05-28T14:07:27.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.070+0000 7f277b000640 1 --2- 172.21.15.42:0/2585297792 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f27500694b0 0x7f275006b970 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:27.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.070+0000 7f277b000640 1 --2- 172.21.15.42:0/2585297792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27741402b0 0x7f27740cdc70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:27.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.070+0000 7f277b000640 1 --2- 172.21.15.42:0/2585297792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2774148090 0x7f27740cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:27.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.070+0000 7f277b000640 1 --2- 172.21.15.42:0/2585297792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27740724f0 0x7f27741584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:27.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.070+0000 7f277b000640 1 -- 172.21.15.42:0/2585297792 >> 172.21.15.42:0/2585297792 conn(0x7f27740b21f0 msgr2=0x7f27740b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:27.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.070+0000 7f277b000640 1 -- 172.21.15.42:0/2585297792 shutdown_connections 2024-05-28T14:07:27.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.070+0000 7f277b000640 1 -- 172.21.15.42:0/2585297792 wait complete. 2024-05-28T14:07:27.144 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone81111 2024-05-28T14:07:27.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.558+0000 7f7fdf600640 1 Processor -- start 2024-05-28T14:07:27.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.558+0000 7f7fdf600640 1 -- start start 2024-05-28T14:07:27.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.562+0000 7f7fdf600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7fd815ec50 0x7f7fd8070060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:27.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.562+0000 7f7fdf600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd80cfda0 0x7f7fd80cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:27.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.562+0000 7f7fdf600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd806f820 con 0x7f7fd815f030 2024-05-28T14:07:27.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.562+0000 7f7fdf600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f7fd80705a0 con 0x7f7fd815ec50 2024-05-28T14:07:27.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.562+0000 7f7fdf600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd806faa0 con 0x7f7fd80cfda0 2024-05-28T14:07:27.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.562+0000 7f7fdd600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd80cfda0 0x7f7fd80cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:27.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.562+0000 7f7fdd600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd80cfda0 0x7f7fd80cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:45570/0 (socket says 172.21.15.42:45570) 2024-05-28T14:07:27.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.562+0000 7f7fdd600640 1 -- 172.21.15.42:0/1887446924 learned_addr learned my addr 172.21.15.42:0/1887446924 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:27.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.566+0000 7f7fd7400640 1 -- 172.21.15.42:0/1887446924 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1947314427 0 0) 0x7f7fc4001120 con 0x7f7fd815f030 2024-05-28T14:07:27.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.566+0000 7f7fd7400640 1 -- 172.21.15.42:0/1887446924 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fc0003530 con 0x7f7fd815f030 2024-05-28T14:07:27.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.566+0000 7f7fdd600640 1 -- 172.21.15.42:0/1887446924 >> v2:172.21.15.42:3300/0 conn(0x7f7fd815ec50 msgr2=0x7f7fd8070060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T14:07:27.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.566+0000 7f7fdd600640 1 --2- 172.21.15.42:0/1887446924 >> v2:172.21.15.42:3300/0 conn(0x7f7fd815ec50 0x7f7fd8070060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:27.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.566+0000 7f7fdd600640 1 -- 172.21.15.42:0/1887446924 >> v1:172.21.15.63:6789/0 conn(0x7f7fd815f030 legacy=0x7f7fd80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:27.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.566+0000 7f7fdd600640 1 -- 172.21.15.42:0/1887446924 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd806fd80 con 0x7f7fd80cfda0 2024-05-28T14:07:27.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.570+0000 7f7fdd600640 1 --2- 172.21.15.42:0/1887446924 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd80cfda0 0x7f7fd80cc4c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000d150 tx=0x7f7fd001ebb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f34549eec6f8d46 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:27.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.582+0000 7f7fd7400640 1 -- 172.21.15.42:0/1887446924 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd000b2d0 con 0x7f7fd80cfda0 2024-05-28T14:07:27.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.582+0000 7f7fd7400640 1 -- 172.21.15.42:0/1887446924 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fd000b470 con 0x7f7fd80cfda0 2024-05-28T14:07:27.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.582+0000 7f7fd7400640 1 -- 172.21.15.42:0/1887446924 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd00a7ca0 con 0x7f7fd80cfda0 2024-05-28T14:07:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.586+0000 7f7fdf600640 1 -- 172.21.15.42:0/1887446924 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd80cfda0 msgr2=0x7f7fd80cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.586+0000 7f7fdf600640 1 --2- 172.21.15.42:0/1887446924 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd80cfda0 0x7f7fd80cc4c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000d150 tx=0x7f7fd001ebb0 comp rx=0 tx=0).stop 2024-05-28T14:07:27.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.586+0000 7f7fdf600640 1 -- 172.21.15.42:0/1887446924 shutdown_connections 2024-05-28T14:07:27.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.586+0000 7f7fdf600640 1 --2- 172.21.15.42:0/1887446924 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd80cfda0 0x7f7fd80cc4c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:27.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.586+0000 7f7fdf600640 1 --2- 172.21.15.42:0/1887446924 >> v2:172.21.15.42:3300/0 conn(0x7f7fd815ec50 0x7f7fd8070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:27.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.586+0000 7f7fdf600640 1 -- 172.21.15.42:0/1887446924 >> 172.21.15.42:0/1887446924 conn(0x7f7fd80b2f60 msgr2=0x7f7fd80b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:27.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.594+0000 7f7fdf600640 1 -- 172.21.15.42:0/1887446924 shutdown_connections 2024-05-28T14:07:27.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.598+0000 7f7fdf600640 1 -- 172.21.15.42:0/1887446924 wait complete. 2024-05-28T14:07:27.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.602+0000 7f7fdf600640 1 Processor -- start 2024-05-28T14:07:27.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.602+0000 7f7fdf600640 1 -- start start 2024-05-28T14:07:27.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.610+0000 7f7fdf600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd80cfda0 0x7f7fd81527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:27.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.610+0000 7f7fdf600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd815ec50 0x7f7fd80b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:27.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.610+0000 7f7fdf600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fd815f030 0x7f7fd80b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:27.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.610+0000 7f7fdf600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd80cd5a0 con 0x7f7fd815f030 2024-05-28T14:07:27.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.610+0000 7f7fdf600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd80cd420 con 0x7f7fd815ec50 2024-05-28T14:07:27.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.610+0000 7f7fdf600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd80cd720 con 0x7f7fd80cfda0 2024-05-28T14:07:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.610+0000 7f7fdcc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd80cfda0 0x7f7fd81527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.610+0000 7f7fdcc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd80cfda0 0x7f7fd81527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:45582/0 (socket says 172.21.15.42:45582) 2024-05-28T14:07:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.610+0000 7f7fdcc00640 1 -- 172.21.15.42:0/3390857726 learned_addr learned my addr 172.21.15.42:0/3390857726 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.610+0000 7f7fd7e00640 1 --2- 172.21.15.42:0/3390857726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd815ec50 0x7f7fd80b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.610+0000 7f7fdcc00640 1 -- 172.21.15.42:0/3390857726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd815ec50 msgr2=0x7f7fd80b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.610+0000 7f7fdcc00640 1 --2- 172.21.15.42:0/3390857726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd815ec50 0x7f7fd80b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.610+0000 7f7fdcc00640 1 -- 172.21.15.42:0/3390857726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fd815f030 msgr2=0x7f7fd80b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.610+0000 7f7fdcc00640 1 --2- 172.21.15.42:0/3390857726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fd815f030 0x7f7fd80b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:27.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.614+0000 7f7fdcc00640 1 -- 172.21.15.42:0/3390857726 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd80b6130 con 0x7f7fd80cfda0 2024-05-28T14:07:27.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.618+0000 7f7fdcc00640 1 --2- 172.21.15.42:0/3390857726 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd80cfda0 0x7f7fd81527e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00b4a0 tx=0x7f7fcc00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:27.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.618+0000 7f7fd4c00640 1 -- 172.21.15.42:0/3390857726 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc011830 con 0x7f7fd80cfda0 2024-05-28T14:07:27.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.618+0000 7f7fd4c00640 1 -- 172.21.15.42:0/3390857726 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fcc016070 con 0x7f7fd80cfda0 2024-05-28T14:07:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.618+0000 7f7fdf600640 1 -- 172.21.15.42:0/3390857726 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd80b6410 con 0x7f7fd80cfda0 2024-05-28T14:07:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.618+0000 7f7fd4c00640 1 -- 172.21.15.42:0/3390857726 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc011c50 con 0x7f7fd80cfda0 2024-05-28T14:07:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.618+0000 7f7fdf600640 1 -- 172.21.15.42:0/3390857726 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd8154960 con 0x7f7fd80cfda0 2024-05-28T14:07:27.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.626+0000 7f7fdf600640 1 -- 172.21.15.42:0/3390857726 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd809f5c0 con 0x7f7fd80cfda0 2024-05-28T14:07:27.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.634+0000 7f7fd4c00640 1 -- 172.21.15.42:0/3390857726 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f7fcc01b460 con 0x7f7fd80cfda0 2024-05-28T14:07:27.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.638+0000 7f7fd4c00640 1 --2- 172.21.15.42:0/3390857726 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7fb8069460 0x7f7fb806b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:27.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.638+0000 7f7fd4c00640 1 -- 172.21.15.42:0/3390857726 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f7fcc0852e0 con 0x7f7fd80cfda0 2024-05-28T14:07:27.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.638+0000 7f7fd4c00640 1 -- 172.21.15.42:0/3390857726 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7fcc053260 con 0x7f7fd80cfda0 2024-05-28T14:07:27.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.646+0000 7f7fd7e00640 1 --2- 172.21.15.42:0/3390857726 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7fb8069460 0x7f7fb806b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:27.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:27.646+0000 7f7fd7e00640 1 --2- 172.21.15.42:0/3390857726 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7fb8069460 0x7f7fb806b920 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8153850 tx=0x7f7fc4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:28.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.122+0000 7f7fdf600640 1 -- 172.21.15.42:0/3390857726 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone81111", "target": ["mon-mgr", ""]}) -- 0x7f7fd804fdd0 con 0x7f7fb8069460 2024-05-28T14:07:28.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.546+0000 7f7fd4c00640 1 -- 172.21.15.42:0/3390857726 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7fd804fdd0 con 0x7f7fb8069460 2024-05-28T14:07:28.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.550+0000 7f7fdf600640 1 -- 172.21.15.42:0/3390857726 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7fb8069460 msgr2=0x7f7fb806b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:28.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.550+0000 7f7fdf600640 1 --2- 172.21.15.42:0/3390857726 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7fb8069460 0x7f7fb806b920 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8153850 tx=0x7f7fc4009040 comp rx=0 tx=0).stop 2024-05-28T14:07:28.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.550+0000 7f7fdf600640 1 -- 172.21.15.42:0/3390857726 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd80cfda0 msgr2=0x7f7fd81527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:28.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.550+0000 7f7fdf600640 1 --2- 172.21.15.42:0/3390857726 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd80cfda0 0x7f7fd81527e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00b4a0 tx=0x7f7fcc00b960 comp rx=0 tx=0).stop 2024-05-28T14:07:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.550+0000 7f7fdf600640 1 -- 172.21.15.42:0/3390857726 shutdown_connections 2024-05-28T14:07:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.550+0000 7f7fdf600640 1 --2- 172.21.15.42:0/3390857726 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7fb8069460 0x7f7fb806b920 secure :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8153850 tx=0x7f7fc4009040 comp rx=0 tx=0).stop 2024-05-28T14:07:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.550+0000 7f7fdf600640 1 --2- 172.21.15.42:0/3390857726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fd815f030 0x7f7fd80b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.550+0000 7f7fdf600640 1 --2- 172.21.15.42:0/3390857726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd815ec50 0x7f7fd80b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:28.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.550+0000 7f7fdf600640 1 --2- 172.21.15.42:0/3390857726 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd80cfda0 0x7f7fd81527e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:28.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.550+0000 7f7fdf600640 1 -- 172.21.15.42:0/3390857726 >> 172.21.15.42:0/3390857726 conn(0x7f7fd80b2f60 msgr2=0x7f7fd80b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:28.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.550+0000 7f7fdf600640 1 -- 172.21.15.42:0/3390857726 shutdown_connections 2024-05-28T14:07:28.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.550+0000 7f7fdf600640 1 -- 172.21.15.42:0/3390857726 wait complete. 2024-05-28T14:07:28.569 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone81112 2024-05-28T14:07:28.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.966+0000 7f1971a00640 1 Processor -- start 2024-05-28T14:07:28.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.974+0000 7f1971a00640 1 -- start start 2024-05-28T14:07:28.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.978+0000 7f1971a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1964050e10 0x7f196403a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:28.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.978+0000 7f196be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f19640062e0 0x7f1964050700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:57224/0 (socket says 172.21.15.42:57224) 2024-05-28T14:07:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.986+0000 7f196be00640 1 -- 172.21.15.42:0/1368106862 learned_addr learned my addr 172.21.15.42:0/1368106862 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:07:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.986+0000 7f1971a00640 1 --2- 172.21.15.42:0/1368106862 >> v2:172.21.15.42:3300/0 conn(0x7f1964052610 0x7f19640e33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.986+0000 7f1971a00640 1 -- 172.21.15.42:0/1368106862 --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f196404fdf0 con 0x7f19640062e0 2024-05-28T14:07:28.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.986+0000 7f1971a00640 1 -- 172.21.15.42:0/1368106862 --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f19640e3910 con 0x7f1964052610 2024-05-28T14:07:28.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.986+0000 7f1971a00640 1 -- 172.21.15.42:0/1368106862 --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f19640500d0 con 0x7f1964050e10 2024-05-28T14:07:28.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.990+0000 7f196b400640 1 --2- 172.21.15.42:0/1368106862 >> v2:172.21.15.63:3300/0 conn(0x7f1964050e10 0x7f196403a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:28.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.990+0000 7f196b400640 1 -- 172.21.15.42:0/1368106862 >> v2:172.21.15.42:3300/0 conn(0x7f1964052610 msgr2=0x7f19640e33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:28.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.990+0000 7f196b400640 1 --2- 172.21.15.42:0/1368106862 >> v2:172.21.15.42:3300/0 conn(0x7f1964052610 0x7f19640e33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:28.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.990+0000 7f196b400640 1 -- 172.21.15.42:0/1368106862 >> v1:172.21.15.42:6789/0 conn(0x7f19640062e0 legacy=0x7f1964050700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:28.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:28.990+0000 7f196b400640 1 -- 172.21.15.42:0/1368106862 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1964038ec0 con 0x7f1964050e10 2024-05-28T14:07:29.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.018+0000 7f196b400640 1 --2- 172.21.15.42:0/1368106862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1964050e10 0x7f196403a640 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1964048490 tx=0x7f195801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f77c277c121badd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:29.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.018+0000 7f196aa00640 1 -- 172.21.15.42:0/1368106862 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19580aa5f0 con 0x7f1964050e10 2024-05-28T14:07:29.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.018+0000 7f196aa00640 1 -- 172.21.15.42:0/1368106862 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19580af070 con 0x7f1964050e10 2024-05-28T14:07:29.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.018+0000 7f196aa00640 1 -- 172.21.15.42:0/1368106862 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19580aa9b0 con 0x7f1964050e10 2024-05-28T14:07:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.030+0000 7f1971a00640 1 -- 172.21.15.42:0/1368106862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1964050e10 msgr2=0x7f196403a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.030+0000 7f1971a00640 1 --2- 172.21.15.42:0/1368106862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1964050e10 0x7f196403a640 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1964048490 tx=0x7f195801c470 comp rx=0 tx=0).stop 2024-05-28T14:07:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.030+0000 7f1971a00640 1 -- 172.21.15.42:0/1368106862 shutdown_connections 2024-05-28T14:07:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.030+0000 7f1971a00640 1 --2- 172.21.15.42:0/1368106862 >> v2:172.21.15.42:3300/0 conn(0x7f1964052610 0x7f19640e33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.030+0000 7f1971a00640 1 --2- 172.21.15.42:0/1368106862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1964050e10 0x7f196403a640 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.030+0000 7f1971a00640 1 -- 172.21.15.42:0/1368106862 >> 172.21.15.42:0/1368106862 conn(0x7f1964009fc0 msgr2=0x7f196400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:29.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.030+0000 7f1971a00640 1 -- 172.21.15.42:0/1368106862 shutdown_connections 2024-05-28T14:07:29.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.030+0000 7f1971a00640 1 -- 172.21.15.42:0/1368106862 wait complete. 2024-05-28T14:07:29.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.034+0000 7f1971a00640 1 Processor -- start 2024-05-28T14:07:29.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.034+0000 7f1971a00640 1 -- start start 2024-05-28T14:07:29.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.038+0000 7f1971a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19640062e0 0x7f196404bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:29.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.038+0000 7f1971a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1964050e10 0x7f19640d5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:29.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f196b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1964050e10 0x7f19640d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:29.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f196b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1964050e10 0x7f19640d5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:50578/0 (socket says 172.21.15.42:50578) 2024-05-28T14:07:29.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f196b400640 1 -- 172.21.15.42:0/1985705179 learned_addr learned my addr 172.21.15.42:0/1985705179 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f196be00640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19640062e0 0x7f196404bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f1971a00640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1964052610 0x7f19640eba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:29.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f1971a00640 1 -- 172.21.15.42:0/1985705179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1964039ed0 con 0x7f19640062e0 2024-05-28T14:07:29.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f1971a00640 1 -- 172.21.15.42:0/1985705179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19640503b0 con 0x7f1964050e10 2024-05-28T14:07:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f1971a00640 1 -- 172.21.15.42:0/1985705179 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f196403a050 con 0x7f1964052610 2024-05-28T14:07:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f1971000640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1964052610 0x7f19640eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f196b400640 1 -- 172.21.15.42:0/1985705179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1964052610 msgr2=0x7f19640eba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f196b400640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1964052610 0x7f19640eba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f196b400640 1 -- 172.21.15.42:0/1985705179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19640062e0 msgr2=0x7f196404bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f196b400640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19640062e0 0x7f196404bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:29.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f196b400640 1 -- 172.21.15.42:0/1985705179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19640db520 con 0x7f1964050e10 2024-05-28T14:07:29.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.046+0000 7f196b400640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1964050e10 0x7f19640d5180 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f19580ac040 tx=0x7f195801acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:29.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.050+0000 7f195fe00640 1 -- 172.21.15.42:0/1985705179 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19580b4610 con 0x7f1964050e10 2024-05-28T14:07:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.050+0000 7f195fe00640 1 -- 172.21.15.42:0/1985705179 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19580b47d0 con 0x7f1964050e10 2024-05-28T14:07:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.050+0000 7f195fe00640 1 -- 172.21.15.42:0/1985705179 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19580b4bc0 con 0x7f1964050e10 2024-05-28T14:07:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.054+0000 7f1971a00640 1 -- 172.21.15.42:0/1985705179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19640d8060 con 0x7f1964050e10 2024-05-28T14:07:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.054+0000 7f1971a00640 1 -- 172.21.15.42:0/1985705179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19640d56c0 con 0x7f1964050e10 2024-05-28T14:07:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.062+0000 7f1971a00640 1 -- 172.21.15.42:0/1985705179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19640052d0 con 0x7f1964050e10 2024-05-28T14:07:29.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.062+0000 7f195fe00640 1 -- 172.21.15.42:0/1985705179 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f19580bc020 con 0x7f1964050e10 2024-05-28T14:07:29.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.062+0000 7f195fe00640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f194c069530 0x7f194c06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:29.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.062+0000 7f195fe00640 1 -- 172.21.15.42:0/1985705179 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f195811fd60 con 0x7f1964050e10 2024-05-28T14:07:29.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.078+0000 7f195fe00640 1 -- 172.21.15.42:0/1985705179 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1958016090 con 0x7f1964050e10 2024-05-28T14:07:29.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.082+0000 7f196be00640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f194c069530 0x7f194c06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.102+0000 7f196be00640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f194c069530 0x7f194c06b9f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1960000f70 tx=0x7f1960009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:29.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:29.602+0000 7f1971a00640 1 -- 172.21.15.42:0/1985705179 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone81112", "target": ["mon-mgr", ""]}) -- 0x7f1964005fc0 con 0x7f194c069530 2024-05-28T14:07:30.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.470+0000 7f195fe00640 1 -- 172.21.15.42:0/1985705179 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1964005fc0 con 0x7f194c069530 2024-05-28T14:07:30.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.478+0000 7f1971a00640 1 -- 172.21.15.42:0/1985705179 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f194c069530 msgr2=0x7f194c06b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:30.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.478+0000 7f1971a00640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f194c069530 0x7f194c06b9f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1960000f70 tx=0x7f1960009040 comp rx=0 tx=0).stop 2024-05-28T14:07:30.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.478+0000 7f1971a00640 1 -- 172.21.15.42:0/1985705179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1964050e10 msgr2=0x7f19640d5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:30.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.478+0000 7f1971a00640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1964050e10 0x7f19640d5180 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f19580ac040 tx=0x7f195801acb0 comp rx=0 tx=0).stop 2024-05-28T14:07:30.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.478+0000 7f1971a00640 1 -- 172.21.15.42:0/1985705179 shutdown_connections 2024-05-28T14:07:30.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.478+0000 7f1971a00640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f194c069530 0x7f194c06b9f0 secure :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1960000f70 tx=0x7f1960009040 comp rx=0 tx=0).stop 2024-05-28T14:07:30.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.478+0000 7f1971a00640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1964052610 0x7f19640eba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:30.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.478+0000 7f1971a00640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1964050e10 0x7f19640d5180 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:30.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.478+0000 7f1971a00640 1 --2- 172.21.15.42:0/1985705179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19640062e0 0x7f196404bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:30.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.478+0000 7f1971a00640 1 -- 172.21.15.42:0/1985705179 >> 172.21.15.42:0/1985705179 conn(0x7f1964009fc0 msgr2=0x7f19640e1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:30.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.478+0000 7f1971a00640 1 -- 172.21.15.42:0/1985705179 shutdown_connections 2024-05-28T14:07:30.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.478+0000 7f1971a00640 1 -- 172.21.15.42:0/1985705179 wait complete. 2024-05-28T14:07:30.566 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone81113 2024-05-28T14:07:30.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.930+0000 7f2b5a600640 1 Processor -- start 2024-05-28T14:07:30.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.930+0000 7f2b5a600640 1 -- start start 2024-05-28T14:07:30.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.934+0000 7f2b5a600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2b54162bb0 0x7f2b5406f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:30.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.934+0000 7f2b5a600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b54072150 0x7f2b540725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:30.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.934+0000 7f2b5a600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2b54162f90 0x7f2b541633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:30.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.934+0000 7f2b5a600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f2b54072af0 con 0x7f2b54162f90 2024-05-28T14:07:30.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.934+0000 7f2b5a600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2b5406eed0 con 0x7f2b54162bb0 2024-05-28T14:07:30.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.934+0000 7f2b5a600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5406e2c0 con 0x7f2b54072150 2024-05-28T14:07:30.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.938+0000 7f2b53e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2b54162f90 0x7f2b541633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:30.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.938+0000 7f2b53e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2b54162f90 0x7f2b541633e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59842/0 (socket says 172.21.15.42:59842) 2024-05-28T14:07:30.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.938+0000 7f2b53e00640 1 -- 172.21.15.42:0/965188292 learned_addr learned my addr 172.21.15.42:0/965188292 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:30.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.938+0000 7f2b52a00640 1 --2- 172.21.15.42:0/965188292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b54072150 0x7f2b540725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:30.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.942+0000 7f2b53e00640 1 -- 172.21.15.42:0/965188292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b54072150 msgr2=0x7f2b540725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:30.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.942+0000 7f2b53e00640 1 --2- 172.21.15.42:0/965188292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b54072150 0x7f2b540725b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:30.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.942+0000 7f2b53e00640 1 -- 172.21.15.42:0/965188292 >> v2:172.21.15.63:3300/0 conn(0x7f2b54162bb0 msgr2=0x7f2b5406f940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-28T14:07:30.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.942+0000 7f2b53e00640 1 --2- 172.21.15.42:0/965188292 >> v2:172.21.15.63:3300/0 conn(0x7f2b54162bb0 0x7f2b5406f940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:30.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.942+0000 7f2b53e00640 1 -- 172.21.15.42:0/965188292 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5406e5a0 con 0x7f2b54162f90 2024-05-28T14:07:30.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.946+0000 7f2b53e00640 1 --2- 172.21.15.42:0/965188292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b54162f90 0x7f2b541633e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2b54068d00 tx=0x7f2b4001c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb60f3b646c0aa3e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:30.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.946+0000 7f2b52000640 1 -- 172.21.15.42:0/965188292 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b400a8650 con 0x7f2b54162f90 2024-05-28T14:07:30.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.946+0000 7f2b52000640 1 -- 172.21.15.42:0/965188292 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b400a87f0 con 0x7f2b54162f90 2024-05-28T14:07:30.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.946+0000 7f2b52000640 1 -- 172.21.15.42:0/965188292 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b400a8bc0 con 0x7f2b54162f90 2024-05-28T14:07:30.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.950+0000 7f2b5a600640 1 -- 172.21.15.42:0/965188292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b54162f90 msgr2=0x7f2b541633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:30.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.950+0000 7f2b5a600640 1 --2- 172.21.15.42:0/965188292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b54162f90 0x7f2b541633e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2b54068d00 tx=0x7f2b4001c690 comp rx=0 tx=0).stop 2024-05-28T14:07:30.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.950+0000 7f2b5a600640 1 -- 172.21.15.42:0/965188292 shutdown_connections 2024-05-28T14:07:30.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.950+0000 7f2b5a600640 1 --2- 172.21.15.42:0/965188292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b54162f90 0x7f2b541633e0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:30.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.950+0000 7f2b5a600640 1 --2- 172.21.15.42:0/965188292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b54072150 0x7f2b540725b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:30.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.950+0000 7f2b5a600640 1 --2- 172.21.15.42:0/965188292 >> v2:172.21.15.63:3300/0 conn(0x7f2b54162bb0 0x7f2b5406f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:30.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.950+0000 7f2b5a600640 1 -- 172.21.15.42:0/965188292 >> 172.21.15.42:0/965188292 conn(0x7f2b5406b9f0 msgr2=0x7f2b5406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:30.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.954+0000 7f2b5a600640 1 -- 172.21.15.42:0/965188292 shutdown_connections 2024-05-28T14:07:30.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.954+0000 7f2b5a600640 1 -- 172.21.15.42:0/965188292 wait complete. 2024-05-28T14:07:30.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.962+0000 7f2b5a600640 1 Processor -- start 2024-05-28T14:07:30.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.962+0000 7f2b5a600640 1 -- start start 2024-05-28T14:07:30.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.962+0000 7f2b5a600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b54072150 0x7f2b5415a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:30.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.962+0000 7f2b5a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b54162bb0 0x7f2b541445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:30.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.962+0000 7f2b5a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b54162f90 0x7f2b5414ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:30.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.962+0000 7f2b5a600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2b540d04b0 con 0x7f2b54162bb0 2024-05-28T14:07:30.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.962+0000 7f2b5a600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b540d0330 con 0x7f2b54162f90 2024-05-28T14:07:30.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.962+0000 7f2b5a600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5406fe80 con 0x7f2b54072150 2024-05-28T14:07:30.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.966+0000 7f2b53e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b54162f90 0x7f2b5414ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:30.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.966+0000 7f2b53e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b54162f90 0x7f2b5414ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:50586/0 (socket says 172.21.15.42:50586) 2024-05-28T14:07:30.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.966+0000 7f2b53e00640 1 -- 172.21.15.42:0/2289854431 learned_addr learned my addr 172.21.15.42:0/2289854431 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:30.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.966+0000 7f2b53400640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b54072150 0x7f2b5415a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.966+0000 7f2b52a00640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b54162bb0 0x7f2b541445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.966+0000 7f2b53e00640 1 -- 172.21.15.42:0/2289854431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b54072150 msgr2=0x7f2b5415a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.966+0000 7f2b53e00640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b54072150 0x7f2b5415a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.966+0000 7f2b53e00640 1 -- 172.21.15.42:0/2289854431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b54162bb0 msgr2=0x7f2b541445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.966+0000 7f2b53e00640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b54162bb0 0x7f2b541445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.966+0000 7f2b53e00640 1 -- 172.21.15.42:0/2289854431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b541615a0 con 0x7f2b54162f90 2024-05-28T14:07:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.966+0000 7f2b52a00640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b54162bb0 0x7f2b541445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:07:30.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.966+0000 7f2b53e00640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b54162f90 0x7f2b5414ed90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2b4001a2c0 tx=0x7f2b40019430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:30.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.970+0000 7f2b4b400640 1 -- 172.21.15.42:0/2289854431 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b400be070 con 0x7f2b54162f90 2024-05-28T14:07:30.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.970+0000 7f2b4b400640 1 -- 172.21.15.42:0/2289854431 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b400b0400 con 0x7f2b54162f90 2024-05-28T14:07:30.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.970+0000 7f2b4b400640 1 -- 172.21.15.42:0/2289854431 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b400b07f0 con 0x7f2b54162f90 2024-05-28T14:07:30.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.970+0000 7f2b5a600640 1 -- 172.21.15.42:0/2289854431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b54161880 con 0x7f2b54162f90 2024-05-28T14:07:30.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.970+0000 7f2b5a600640 1 -- 172.21.15.42:0/2289854431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b54161e40 con 0x7f2b54162f90 2024-05-28T14:07:30.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.974+0000 7f2b5a600640 1 -- 172.21.15.42:0/2289854431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b54073b90 con 0x7f2b54162f90 2024-05-28T14:07:30.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.978+0000 7f2b4b400640 1 -- 172.21.15.42:0/2289854431 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f2b400a6040 con 0x7f2b54162f90 2024-05-28T14:07:30.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.978+0000 7f2b4b400640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b34069530 0x7f2b3406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:30.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.978+0000 7f2b4b400640 1 -- 172.21.15.42:0/2289854431 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f2b4011f400 con 0x7f2b54162f90 2024-05-28T14:07:30.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.986+0000 7f2b4b400640 1 -- 172.21.15.42:0/2289854431 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2b400ac030 con 0x7f2b54162f90 2024-05-28T14:07:30.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:30.994+0000 7f2b53400640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b34069530 0x7f2b3406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:31.002+0000 7f2b53400640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b34069530 0x7f2b3406b9f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2b54146170 tx=0x7f2b4c00d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:31.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:31.594+0000 7f2b5a600640 1 -- 172.21.15.42:0/2289854431 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone81113", "target": ["mon-mgr", ""]}) -- 0x7f2b5415ac00 con 0x7f2b34069530 2024-05-28T14:07:32.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.386+0000 7f2b4b400640 1 -- 172.21.15.42:0/2289854431 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2b5415ac00 con 0x7f2b34069530 2024-05-28T14:07:32.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.406+0000 7f2b5a600640 1 -- 172.21.15.42:0/2289854431 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b34069530 msgr2=0x7f2b3406b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:32.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.406+0000 7f2b5a600640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b34069530 0x7f2b3406b9f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2b54146170 tx=0x7f2b4c00d040 comp rx=0 tx=0).stop 2024-05-28T14:07:32.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.406+0000 7f2b5a600640 1 -- 172.21.15.42:0/2289854431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b54162f90 msgr2=0x7f2b5414ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:32.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.406+0000 7f2b5a600640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b54162f90 0x7f2b5414ed90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2b4001a2c0 tx=0x7f2b40019430 comp rx=0 tx=0).stop 2024-05-28T14:07:32.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.406+0000 7f2b5a600640 1 -- 172.21.15.42:0/2289854431 shutdown_connections 2024-05-28T14:07:32.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.406+0000 7f2b5a600640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b34069530 0x7f2b3406b9f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:32.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.406+0000 7f2b5a600640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b54162f90 0x7f2b5414ed90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:32.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.406+0000 7f2b5a600640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b54162bb0 0x7f2b541445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:32.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.406+0000 7f2b5a600640 1 --2- 172.21.15.42:0/2289854431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b54072150 0x7f2b5415a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:32.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.406+0000 7f2b5a600640 1 -- 172.21.15.42:0/2289854431 >> 172.21.15.42:0/2289854431 conn(0x7f2b5406b9f0 msgr2=0x7f2b54153df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:32.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.410+0000 7f2b5a600640 1 -- 172.21.15.42:0/2289854431 shutdown_connections 2024-05-28T14:07:32.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.410+0000 7f2b5a600640 1 -- 172.21.15.42:0/2289854431 wait complete. 2024-05-28T14:07:32.470 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone81114 2024-05-28T14:07:32.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.974+0000 7fa1fe400640 1 Processor -- start 2024-05-28T14:07:32.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.978+0000 7fa1fe400640 1 -- start start 2024-05-28T14:07:32.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1fe400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa1f0047c20 0x7fa1f00eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:32.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1fe400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1f0005ef0 0x7fa1f0006320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1fe400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f0034220 con 0x7fa1f0033e40 2024-05-28T14:07:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1fe400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f0050210 con 0x7fa1f0047c20 2024-05-28T14:07:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1fe400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f0037bc0 con 0x7fa1f0005ef0 2024-05-28T14:07:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1fd000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa1f0033e40 0x7fa1f00eb180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:57228/0 (socket says 172.21.15.42:57228) 2024-05-28T14:07:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1fd000640 1 -- 172.21.15.42:0/3789748461 learned_addr learned my addr 172.21.15.42:0/3789748461 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:07:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1f7400640 1 -- 172.21.15.42:0/3789748461 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 183534413 0 0) 0x7fa1f0034220 con 0x7fa1f0033e40 2024-05-28T14:07:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1f7400640 1 -- 172.21.15.42:0/3789748461 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1e0003530 con 0x7fa1f0033e40 2024-05-28T14:07:32.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1fda00640 1 --2- 172.21.15.42:0/3789748461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1f0005ef0 0x7fa1f0006320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:32.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1f7400640 1 -- 172.21.15.42:0/3789748461 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1036980816 0 0) 0x7fa1e0003530 con 0x7fa1f0033e40 2024-05-28T14:07:32.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1f7400640 1 -- 172.21.15.42:0/3789748461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1f0005ef0 msgr2=0x7fa1f0006320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:32.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1f7400640 1 --2- 172.21.15.42:0/3789748461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1f0005ef0 0x7fa1f0006320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:32.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1f7400640 1 -- 172.21.15.42:0/3789748461 >> v2:172.21.15.63:3300/0 conn(0x7fa1f0047c20 msgr2=0x7fa1f00eb890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-28T14:07:32.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1f7400640 1 --2- 172.21.15.42:0/3789748461 >> v2:172.21.15.63:3300/0 conn(0x7fa1f0047c20 0x7fa1f00eb890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:32.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1f7400640 1 -- 172.21.15.42:0/3789748461 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f004fa30 con 0x7fa1f0033e40 2024-05-28T14:07:32.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.982+0000 7fa1f7400640 1 -- 172.21.15.42:0/3789748461 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fa1ec001680 con 0x7fa1f0033e40 2024-05-28T14:07:32.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.986+0000 7fa1f7400640 1 -- 172.21.15.42:0/3789748461 >> v1:172.21.15.42:6789/0 conn(0x7fa1f0033e40 legacy=0x7fa1f00eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:32.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.986+0000 7fa1f7400640 1 --2- 172.21.15.42:0/3789748461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e00039c0 0x7fa1e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:32.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.986+0000 7fa1f7400640 1 --2- 172.21.15.42:0/3789748461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1e0004810 0x7fa1e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:32.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.986+0000 7fa1f7400640 1 --2- 172.21.15.42:0/3789748461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00051d0 0x7fa1e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:32.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.986+0000 7fa1f7400640 1 -- 172.21.15.42:0/3789748461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0050210 con 0x7fa1e00039c0 2024-05-28T14:07:32.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.986+0000 7fa1f7400640 1 -- 172.21.15.42:0/3789748461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e000c000 con 0x7fa1e00051d0 2024-05-28T14:07:32.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.986+0000 7fa1f7400640 1 -- 172.21.15.42:0/3789748461 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e000c200 con 0x7fa1e0004810 2024-05-28T14:07:32.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.994+0000 7fa1fda00640 1 --2- 172.21.15.42:0/3789748461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00051d0 0x7fa1e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:32.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.994+0000 7fa1fd000640 1 --2- 172.21.15.42:0/3789748461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e00039c0 0x7fa1e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:32.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.994+0000 7fa1f7e00640 1 --2- 172.21.15.42:0/3789748461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1e0004810 0x7fa1e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:32.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.994+0000 7fa1fda00640 1 -- 172.21.15.42:0/3789748461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1e0004810 msgr2=0x7fa1e0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:32.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.994+0000 7fa1fda00640 1 --2- 172.21.15.42:0/3789748461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1e0004810 0x7fa1e0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:32.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.994+0000 7fa1fda00640 1 -- 172.21.15.42:0/3789748461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e00039c0 msgr2=0x7fa1e0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:32.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.994+0000 7fa1fda00640 1 --2- 172.21.15.42:0/3789748461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e00039c0 0x7fa1e0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:32.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:32.994+0000 7fa1fda00640 1 -- 172.21.15.42:0/3789748461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e0003530 con 0x7fa1e00051d0 2024-05-28T14:07:32.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.002+0000 7fa1fd000640 1 --2- 172.21.15.42:0/3789748461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e00039c0 0x7fa1e0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:07:32.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.006+0000 7fa1fda00640 1 --2- 172.21.15.42:0/3789748461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00051d0 0x7fa1e000b9d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8050b40 tx=0x7fa1f8098170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5407cf02496589b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:32.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.010+0000 7fa1f7400640 1 -- 172.21.15.42:0/3789748461 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8050c80 con 0x7fa1e00051d0 2024-05-28T14:07:32.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.010+0000 7fa1f7400640 1 -- 172.21.15.42:0/3789748461 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f8120d90 con 0x7fa1e00051d0 2024-05-28T14:07:32.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.010+0000 7fa1f7400640 1 -- 172.21.15.42:0/3789748461 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f81285f0 con 0x7fa1e00051d0 2024-05-28T14:07:32.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.010+0000 7fa1fe400640 1 -- 172.21.15.42:0/3789748461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00051d0 msgr2=0x7fa1e000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:32.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.010+0000 7fa1fe400640 1 --2- 172.21.15.42:0/3789748461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00051d0 0x7fa1e000b9d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8050b40 tx=0x7fa1f8098170 comp rx=0 tx=0).stop 2024-05-28T14:07:32.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.014+0000 7fa1fda00640 1 -- 172.21.15.42:0/3789748461 reap_dead start 2024-05-28T14:07:33.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.014+0000 7fa1fe400640 1 -- 172.21.15.42:0/3789748461 shutdown_connections 2024-05-28T14:07:33.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.014+0000 7fa1fe400640 1 -- 172.21.15.42:0/3789748461 >> 172.21.15.42:0/3789748461 conn(0x7fa1f0009fc0 msgr2=0x7fa1f000a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:33.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.022+0000 7fa1fe400640 1 -- 172.21.15.42:0/3789748461 shutdown_connections 2024-05-28T14:07:33.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.026+0000 7fa1fe400640 1 -- 172.21.15.42:0/3789748461 wait complete. 2024-05-28T14:07:33.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fe400640 1 Processor -- start 2024-05-28T14:07:33.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fe400640 1 -- start start 2024-05-28T14:07:33.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fe400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00051d0 0x7fa1f0035380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:33.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fe400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1f0047150 0x7fa1f0182ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:33.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fe400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1f0186450 0x7fa1f0048af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:33.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fe400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f004fd80 con 0x7fa1f0047150 2024-05-28T14:07:33.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fe400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f004f090 con 0x7fa1e00051d0 2024-05-28T14:07:33.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fe400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f004f5c0 con 0x7fa1f0186450 2024-05-28T14:07:33.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00051d0 0x7fa1f0035380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:33.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00051d0 0x7fa1f0035380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:50604/0 (socket says 172.21.15.42:50604) 2024-05-28T14:07:33.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fd000640 1 -- 172.21.15.42:0/1910433721 learned_addr learned my addr 172.21.15.42:0/1910433721 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:33.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fda00640 1 --2- 172.21.15.42:0/1910433721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1f0186450 0x7fa1f0048af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:33.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fd000640 1 -- 172.21.15.42:0/1910433721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1f0186450 msgr2=0x7fa1f0048af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:33.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fd000640 1 --2- 172.21.15.42:0/1910433721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1f0186450 0x7fa1f0048af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:33.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fd000640 1 -- 172.21.15.42:0/1910433721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1f0047150 msgr2=0x7fa1f0182ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:33.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fd000640 1 --2- 172.21.15.42:0/1910433721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1f0047150 0x7fa1f0182ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:33.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.034+0000 7fa1fd000640 1 -- 172.21.15.42:0/1910433721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f0183400 con 0x7fa1e00051d0 2024-05-28T14:07:33.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.038+0000 7fa1fd000640 1 --2- 172.21.15.42:0/1910433721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00051d0 0x7fa1f0035380 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec001b10 tx=0x7fa1ec007790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:33.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.038+0000 7fa1f4c00640 1 -- 172.21.15.42:0/1910433721 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec019020 con 0x7fa1e00051d0 2024-05-28T14:07:33.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.038+0000 7fa1f4c00640 1 -- 172.21.15.42:0/1910433721 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1ec010950 con 0x7fa1e00051d0 2024-05-28T14:07:33.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.038+0000 7fa1f4c00640 1 -- 172.21.15.42:0/1910433721 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec011040 con 0x7fa1e00051d0 2024-05-28T14:07:33.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.038+0000 7fa1fe400640 1 -- 172.21.15.42:0/1910433721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f00476f0 con 0x7fa1e00051d0 2024-05-28T14:07:33.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.038+0000 7fa1fe400640 1 -- 172.21.15.42:0/1910433721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f0038090 con 0x7fa1e00051d0 2024-05-28T14:07:33.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.042+0000 7fa1fe400640 1 -- 172.21.15.42:0/1910433721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f00052d0 con 0x7fa1e00051d0 2024-05-28T14:07:33.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.046+0000 7fa1f4c00640 1 -- 172.21.15.42:0/1910433721 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7fa1ec01c020 con 0x7fa1e00051d0 2024-05-28T14:07:33.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.050+0000 7fa1f4c00640 1 --2- 172.21.15.42:0/1910433721 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa1d80694b0 0x7fa1d806b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:33.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.050+0000 7fa1f4c00640 1 -- 172.21.15.42:0/1910433721 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fa1ec088d50 con 0x7fa1e00051d0 2024-05-28T14:07:33.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.050+0000 7fa1f4c00640 1 -- 172.21.15.42:0/1910433721 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa1ec023020 con 0x7fa1e00051d0 2024-05-28T14:07:33.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.074+0000 7fa1f7e00640 1 --2- 172.21.15.42:0/1910433721 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa1d80694b0 0x7fa1d806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:33.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.090+0000 7fa1f7e00640 1 --2- 172.21.15.42:0/1910433721 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa1d80694b0 0x7fa1d806b970 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0034af0 tx=0x7fa1e4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:33.614+0000 7fa1fe400640 1 -- 172.21.15.42:0/1910433721 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone81114", "target": ["mon-mgr", ""]}) -- 0x7fa1f0052830 con 0x7fa1d80694b0 2024-05-28T14:07:34.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.314+0000 7fa1f4c00640 1 -- 172.21.15.42:0/1910433721 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa1f0052830 con 0x7fa1d80694b0 2024-05-28T14:07:34.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.322+0000 7fa1fe400640 1 -- 172.21.15.42:0/1910433721 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa1d80694b0 msgr2=0x7fa1d806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:34.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.322+0000 7fa1fe400640 1 --2- 172.21.15.42:0/1910433721 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa1d80694b0 0x7fa1d806b970 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0034af0 tx=0x7fa1e4008040 comp rx=0 tx=0).stop 2024-05-28T14:07:34.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.322+0000 7fa1fe400640 1 -- 172.21.15.42:0/1910433721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00051d0 msgr2=0x7fa1f0035380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:34.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.322+0000 7fa1fe400640 1 --2- 172.21.15.42:0/1910433721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00051d0 0x7fa1f0035380 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec001b10 tx=0x7fa1ec007790 comp rx=0 tx=0).stop 2024-05-28T14:07:34.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.322+0000 7fa1fe400640 1 -- 172.21.15.42:0/1910433721 shutdown_connections 2024-05-28T14:07:34.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.322+0000 7fa1fe400640 1 --2- 172.21.15.42:0/1910433721 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa1d80694b0 0x7fa1d806b970 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:34.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.322+0000 7fa1fe400640 1 --2- 172.21.15.42:0/1910433721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1f0186450 0x7fa1f0048af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:34.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.322+0000 7fa1fe400640 1 --2- 172.21.15.42:0/1910433721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1f0047150 0x7fa1f0182ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:34.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.322+0000 7fa1fe400640 1 --2- 172.21.15.42:0/1910433721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00051d0 0x7fa1f0035380 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:34.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.322+0000 7fa1fe400640 1 -- 172.21.15.42:0/1910433721 >> 172.21.15.42:0/1910433721 conn(0x7fa1f0009fc0 msgr2=0x7fa1f0006300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:34.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.322+0000 7fa1fe400640 1 -- 172.21.15.42:0/1910433721 shutdown_connections 2024-05-28T14:07:34.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.322+0000 7fa1fe400640 1 -- 172.21.15.42:0/1910433721 wait complete. 2024-05-28T14:07:34.383 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone81115 2024-05-28T14:07:34.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.730+0000 7f8eebe00640 1 Processor -- start 2024-05-28T14:07:34.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.730+0000 7f8eebe00640 1 -- start start 2024-05-28T14:07:34.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.734+0000 7f8eebe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8eec0b2880 0x7f8eec0cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:34.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.734+0000 7f8eebe00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eec0cfef0 con 0x7f8eec072f80 2024-05-28T14:07:34.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.734+0000 7f8eebe00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eec0cee50 con 0x7f8eec0cc8a0 2024-05-28T14:07:34.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.734+0000 7f8eebe00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8eec0cec20 con 0x7f8eec0b2880 2024-05-28T14:07:34.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.734+0000 7f8eeaa00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8eec072f80 0x7f8eec073360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:57230/0 (socket says 172.21.15.42:57230) 2024-05-28T14:07:34.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.734+0000 7f8eeaa00640 1 -- 172.21.15.42:0/1230552283 learned_addr learned my addr 172.21.15.42:0/1230552283 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:07:34.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.734+0000 7f8eea000640 1 --2- 172.21.15.42:0/1230552283 >> v2:172.21.15.63:3300/0 conn(0x7f8eec0b2880 0x7f8eec0cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:34.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.734+0000 7f8ee9600640 1 -- 172.21.15.42:0/1230552283 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1732287139 0 0) 0x7f8eec0cfef0 con 0x7f8eec072f80 2024-05-28T14:07:34.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.734+0000 7f8ee9600640 1 -- 172.21.15.42:0/1230552283 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ed0003530 con 0x7f8eec072f80 2024-05-28T14:07:34.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.734+0000 7f8eea000640 1 -- 172.21.15.42:0/1230552283 >> v1:172.21.15.63:6789/0 conn(0x7f8eec0cc8a0 legacy=0x7f8eec0cccd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T14:07:34.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.734+0000 7f8eea000640 1 -- 172.21.15.42:0/1230552283 >> v1:172.21.15.42:6789/0 conn(0x7f8eec072f80 legacy=0x7f8eec073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:34.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.734+0000 7f8eea000640 1 -- 172.21.15.42:0/1230552283 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eec0cf130 con 0x7f8eec0b2880 2024-05-28T14:07:34.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.738+0000 7f8eea000640 1 --2- 172.21.15.42:0/1230552283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eec0b2880 0x7f8eec0cc250 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80097c0 tx=0x7f8ed801c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b922443c0cfb2c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:34.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.738+0000 7f8ee9600640 1 -- 172.21.15.42:0/1230552283 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80b0070 con 0x7f8eec0b2880 2024-05-28T14:07:34.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.738+0000 7f8ee9600640 1 -- 172.21.15.42:0/1230552283 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed8004230 con 0x7f8eec0b2880 2024-05-28T14:07:34.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.738+0000 7f8ee9600640 1 -- 172.21.15.42:0/1230552283 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8003620 con 0x7f8eec0b2880 2024-05-28T14:07:34.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.738+0000 7f8eebe00640 1 -- 172.21.15.42:0/1230552283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eec0b2880 msgr2=0x7f8eec0cc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:34.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.738+0000 7f8eebe00640 1 --2- 172.21.15.42:0/1230552283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eec0b2880 0x7f8eec0cc250 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80097c0 tx=0x7f8ed801c460 comp rx=0 tx=0).stop 2024-05-28T14:07:34.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.742+0000 7f8eebe00640 1 -- 172.21.15.42:0/1230552283 shutdown_connections 2024-05-28T14:07:34.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.742+0000 7f8eebe00640 1 --2- 172.21.15.42:0/1230552283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eec0b2880 0x7f8eec0cc250 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:34.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.742+0000 7f8eebe00640 1 -- 172.21.15.42:0/1230552283 >> 172.21.15.42:0/1230552283 conn(0x7f8eec0b1d60 msgr2=0x7f8eec0b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:34.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.742+0000 7f8eebe00640 1 -- 172.21.15.42:0/1230552283 shutdown_connections 2024-05-28T14:07:34.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.742+0000 7f8eebe00640 1 -- 172.21.15.42:0/1230552283 wait complete. 2024-05-28T14:07:34.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.746+0000 7f8eebe00640 1 Processor -- start 2024-05-28T14:07:34.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.746+0000 7f8eebe00640 1 -- start start 2024-05-28T14:07:34.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.746+0000 7f8eebe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8eec072f80 0x7f8eec097780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:34.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.746+0000 7f8eebe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8eec0b2880 0x7f8eec097cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:34.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.750+0000 7f8eebe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eec0cc8a0 0x7f8eec0b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:34.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.750+0000 7f8eebe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec0cd970 con 0x7f8eec0b2880 2024-05-28T14:07:34.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.750+0000 7f8eebe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec0cd7f0 con 0x7f8eec0cc8a0 2024-05-28T14:07:34.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.750+0000 7f8eebe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8eec15ca40 con 0x7f8eec072f80 2024-05-28T14:07:34.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.750+0000 7f8eeb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eec0cc8a0 0x7f8eec0b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:34.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.750+0000 7f8eeb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eec0cc8a0 0x7f8eec0b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:50618/0 (socket says 172.21.15.42:50618) 2024-05-28T14:07:34.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.750+0000 7f8eeb400640 1 -- 172.21.15.42:0/3665424173 learned_addr learned my addr 172.21.15.42:0/3665424173 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:34.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.750+0000 7f8eea000640 1 --2- 172.21.15.42:0/3665424173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8eec0b2880 0x7f8eec097cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:34.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.750+0000 7f8eea000640 1 -- 172.21.15.42:0/3665424173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8eec072f80 msgr2=0x7f8eec097780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:34.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.750+0000 7f8eea000640 1 --2- 172.21.15.42:0/3665424173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8eec072f80 0x7f8eec097780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:34.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.750+0000 7f8eea000640 1 -- 172.21.15.42:0/3665424173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eec0cc8a0 msgr2=0x7f8eec0b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:34.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.750+0000 7f8eea000640 1 --2- 172.21.15.42:0/3665424173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eec0cc8a0 0x7f8eec0b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:34.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.750+0000 7f8eea000640 1 -- 172.21.15.42:0/3665424173 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eec099140 con 0x7f8eec0b2880 2024-05-28T14:07:34.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.766+0000 7f8eea000640 1 --2- 172.21.15.42:0/3665424173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8eec0b2880 0x7f8eec097cc0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8004000 tx=0x7f8ed8019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:34.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.770+0000 7f8ee3400640 1 -- 172.21.15.42:0/3665424173 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80b9070 con 0x7f8eec0b2880 2024-05-28T14:07:34.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.770+0000 7f8eebe00640 1 -- 172.21.15.42:0/3665424173 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eec099420 con 0x7f8eec0b2880 2024-05-28T14:07:34.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.774+0000 7f8ee3400640 1 -- 172.21.15.42:0/3665424173 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed8000e50 con 0x7f8eec0b2880 2024-05-28T14:07:34.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.778+0000 7f8ee3400640 1 -- 172.21.15.42:0/3665424173 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80b0040 con 0x7f8eec0b2880 2024-05-28T14:07:34.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.782+0000 7f8eebe00640 1 -- 172.21.15.42:0/3665424173 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eec0999e0 con 0x7f8eec0b2880 2024-05-28T14:07:34.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.782+0000 7f8ee3400640 1 -- 172.21.15.42:0/3665424173 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f8ed80bc070 con 0x7f8eec0b2880 2024-05-28T14:07:34.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.782+0000 7f8eebe00640 1 -- 172.21.15.42:0/3665424173 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eec04fcf0 con 0x7f8eec0b2880 2024-05-28T14:07:34.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.786+0000 7f8ee3400640 1 --2- 172.21.15.42:0/3665424173 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ecc069660 0x7f8ecc06bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:34.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.790+0000 7f8ee3400640 1 -- 172.21.15.42:0/3665424173 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f8ed80043e0 con 0x7f8eec0b2880 2024-05-28T14:07:34.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.822+0000 7f8ee3400640 1 -- 172.21.15.42:0/3665424173 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ed8016050 con 0x7f8eec0b2880 2024-05-28T14:07:34.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.846+0000 7f8eeaa00640 1 --2- 172.21.15.42:0/3665424173 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ecc069660 0x7f8ecc06bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:34.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:34.846+0000 7f8eeaa00640 1 --2- 172.21.15.42:0/3665424173 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ecc069660 0x7f8ecc06bb20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0012f0 tx=0x7f8edc0015c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:35.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:35.374+0000 7f8eebe00640 1 -- 172.21.15.42:0/3665424173 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone81115", "target": ["mon-mgr", ""]}) -- 0x7f8eec0ae380 con 0x7f8ecc069660 2024-05-28T14:07:36.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:36.638+0000 7f8ee3400640 1 -- 172.21.15.42:0/3665424173 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8eec0ae380 con 0x7f8ecc069660 2024-05-28T14:07:36.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:36.662+0000 7f8eebe00640 1 -- 172.21.15.42:0/3665424173 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ecc069660 msgr2=0x7f8ecc06bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:36.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:36.662+0000 7f8eebe00640 1 --2- 172.21.15.42:0/3665424173 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ecc069660 0x7f8ecc06bb20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0012f0 tx=0x7f8edc0015c0 comp rx=0 tx=0).stop 2024-05-28T14:07:36.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:36.662+0000 7f8eebe00640 1 -- 172.21.15.42:0/3665424173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8eec0b2880 msgr2=0x7f8eec097cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:36.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:36.662+0000 7f8eebe00640 1 --2- 172.21.15.42:0/3665424173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8eec0b2880 0x7f8eec097cc0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8004000 tx=0x7f8ed8019200 comp rx=0 tx=0).stop 2024-05-28T14:07:36.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:36.662+0000 7f8eebe00640 1 -- 172.21.15.42:0/3665424173 shutdown_connections 2024-05-28T14:07:36.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:36.662+0000 7f8eebe00640 1 --2- 172.21.15.42:0/3665424173 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ecc069660 0x7f8ecc06bb20 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:36.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:36.662+0000 7f8eebe00640 1 --2- 172.21.15.42:0/3665424173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eec0cc8a0 0x7f8eec0b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:36.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:36.662+0000 7f8eebe00640 1 --2- 172.21.15.42:0/3665424173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8eec0b2880 0x7f8eec097cc0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:36.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:36.662+0000 7f8eebe00640 1 --2- 172.21.15.42:0/3665424173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8eec072f80 0x7f8eec097780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:36.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:36.662+0000 7f8eebe00640 1 -- 172.21.15.42:0/3665424173 >> 172.21.15.42:0/3665424173 conn(0x7f8eec0b1d60 msgr2=0x7f8eec15b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:36.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:36.662+0000 7f8eebe00640 1 -- 172.21.15.42:0/3665424173 shutdown_connections 2024-05-28T14:07:36.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:36.662+0000 7f8eebe00640 1 -- 172.21.15.42:0/3665424173 wait complete. 2024-05-28T14:07:36.715 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone81116 2024-05-28T14:07:37.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.142+0000 7f977b200640 1 Processor -- start 2024-05-28T14:07:37.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.142+0000 7f977b200640 1 -- start start 2024-05-28T14:07:37.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.142+0000 7f977b200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9774162ac0 0x7f977406e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:37.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.142+0000 7f977b200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f97741670c0 0x7f977406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:37.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.142+0000 7f977b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97740b4eb0 0x7f97740ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:37.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.146+0000 7f977b200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f97740b4160 con 0x7f9774162ac0 2024-05-28T14:07:37.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.146+0000 7f977b200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f97740b43d0 con 0x7f97741670c0 2024-05-28T14:07:37.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.146+0000 7f977b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f97740b46b0 con 0x7f97740b4eb0 2024-05-28T14:07:37.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.146+0000 7f9779e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9774162ac0 0x7f977406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:37.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.146+0000 7f9779e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9774162ac0 0x7f977406e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59888/0 (socket says 172.21.15.42:59888) 2024-05-28T14:07:37.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.146+0000 7f9779e00640 1 -- 172.21.15.42:0/2590227053 learned_addr learned my addr 172.21.15.42:0/2590227053 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:37.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.146+0000 7f9779e00640 1 -- 172.21.15.42:0/2590227053 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97740b4eb0 msgr2=0x7f97740ccb00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-28T14:07:37.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.146+0000 7f9779e00640 1 --2- 172.21.15.42:0/2590227053 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97740b4eb0 0x7f97740ccb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:37.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.146+0000 7f9779e00640 1 -- 172.21.15.42:0/2590227053 >> v2:172.21.15.63:3300/0 conn(0x7f97741670c0 msgr2=0x7f977406eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.146+0000 7f9779400640 1 --2- 172.21.15.42:0/2590227053 >> v2:172.21.15.63:3300/0 conn(0x7f97741670c0 0x7f977406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.146+0000 7f9779e00640 1 --2- 172.21.15.42:0/2590227053 >> v2:172.21.15.63:3300/0 conn(0x7f97741670c0 0x7f977406eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.146+0000 7f9779e00640 1 -- 172.21.15.42:0/2590227053 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97740b4990 con 0x7f9774162ac0 2024-05-28T14:07:37.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.150+0000 7f9779e00640 1 --2- 172.21.15.42:0/2590227053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9774162ac0 0x7f977406e970 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f97740cdec0 tx=0x7f976801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ed3600c73578e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:37.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.150+0000 7f9778a00640 1 -- 172.21.15.42:0/2590227053 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97680ab660 con 0x7f9774162ac0 2024-05-28T14:07:37.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.150+0000 7f9778a00640 1 -- 172.21.15.42:0/2590227053 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97680b0070 con 0x7f9774162ac0 2024-05-28T14:07:37.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.150+0000 7f9778a00640 1 -- 172.21.15.42:0/2590227053 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97680aba20 con 0x7f9774162ac0 2024-05-28T14:07:37.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.154+0000 7f977b200640 1 -- 172.21.15.42:0/2590227053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9774162ac0 msgr2=0x7f977406e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:37.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.154+0000 7f977b200640 1 --2- 172.21.15.42:0/2590227053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9774162ac0 0x7f977406e970 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f97740cdec0 tx=0x7f976801c540 comp rx=0 tx=0).stop 2024-05-28T14:07:37.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.154+0000 7f977b200640 1 -- 172.21.15.42:0/2590227053 shutdown_connections 2024-05-28T14:07:37.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.154+0000 7f977b200640 1 --2- 172.21.15.42:0/2590227053 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97740b4eb0 0x7f97740ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:37.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.154+0000 7f977b200640 1 --2- 172.21.15.42:0/2590227053 >> v2:172.21.15.63:3300/0 conn(0x7f97741670c0 0x7f977406eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:37.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.154+0000 7f977b200640 1 --2- 172.21.15.42:0/2590227053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9774162ac0 0x7f977406e970 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:37.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.154+0000 7f977b200640 1 -- 172.21.15.42:0/2590227053 >> 172.21.15.42:0/2590227053 conn(0x7f977406bbc0 msgr2=0x7f977406bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:37.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.154+0000 7f977b200640 1 -- 172.21.15.42:0/2590227053 shutdown_connections 2024-05-28T14:07:37.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.154+0000 7f977b200640 1 -- 172.21.15.42:0/2590227053 wait complete. 2024-05-28T14:07:37.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.158+0000 7f977b200640 1 Processor -- start 2024-05-28T14:07:37.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.158+0000 7f977b200640 1 -- start start 2024-05-28T14:07:37.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.158+0000 7f977b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97740b4eb0 0x7f9774165560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:37.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.158+0000 7f977b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9774162ac0 0x7f97740b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:37.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.158+0000 7f977b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9774165aa0 0x7f97740b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:37.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.158+0000 7f977b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f97740ad040 con 0x7f9774162ac0 2024-05-28T14:07:37.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.158+0000 7f977b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f977406e740 con 0x7f9774165aa0 2024-05-28T14:07:37.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.158+0000 7f977b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f97740ad1c0 con 0x7f97740b4eb0 2024-05-28T14:07:37.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.162+0000 7f9779e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97740b4eb0 0x7f9774165560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:37.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.162+0000 7f9779e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97740b4eb0 0x7f9774165560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:45688/0 (socket says 172.21.15.42:45688) 2024-05-28T14:07:37.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.162+0000 7f9779e00640 1 -- 172.21.15.42:0/497228261 learned_addr learned my addr 172.21.15.42:0/497228261 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:37.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.162+0000 7f977a800640 1 --2- 172.21.15.42:0/497228261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9774165aa0 0x7f97740b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:37.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.162+0000 7f9779400640 1 --2- 172.21.15.42:0/497228261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9774162ac0 0x7f97740b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:37.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.162+0000 7f9779e00640 1 -- 172.21.15.42:0/497228261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9774165aa0 msgr2=0x7f97740b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:37.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.162+0000 7f9779e00640 1 --2- 172.21.15.42:0/497228261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9774165aa0 0x7f97740b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:37.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.162+0000 7f9779e00640 1 -- 172.21.15.42:0/497228261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9774162ac0 msgr2=0x7f97740b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:37.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.162+0000 7f9779e00640 1 --2- 172.21.15.42:0/497228261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9774162ac0 0x7f97740b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:37.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.162+0000 7f9779e00640 1 -- 172.21.15.42:0/497228261 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97741fc2e0 con 0x7f97740b4eb0 2024-05-28T14:07:37.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.162+0000 7f9779e00640 1 --2- 172.21.15.42:0/497228261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97740b4eb0 0x7f9774165560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f97680ad040 tx=0x7f976801ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:37.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.166+0000 7f976e000640 1 -- 172.21.15.42:0/497228261 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97680be5f0 con 0x7f97740b4eb0 2024-05-28T14:07:37.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.166+0000 7f976e000640 1 -- 172.21.15.42:0/497228261 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97680be790 con 0x7f97740b4eb0 2024-05-28T14:07:37.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.166+0000 7f976e000640 1 -- 172.21.15.42:0/497228261 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97680beb80 con 0x7f97740b4eb0 2024-05-28T14:07:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.170+0000 7f977b200640 1 -- 172.21.15.42:0/497228261 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97741fc5c0 con 0x7f97740b4eb0 2024-05-28T14:07:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.170+0000 7f977b200640 1 -- 172.21.15.42:0/497228261 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f97741fbf50 con 0x7f97740b4eb0 2024-05-28T14:07:37.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.170+0000 7f977b200640 1 -- 172.21.15.42:0/497228261 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97741fcbd0 con 0x7f97740b4eb0 2024-05-28T14:07:37.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.174+0000 7f976e000640 1 -- 172.21.15.42:0/497228261 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f97680b43f0 con 0x7f97740b4eb0 2024-05-28T14:07:37.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.178+0000 7f976e000640 1 --2- 172.21.15.42:0/497228261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f97540694e0 0x7f975406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:37.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.178+0000 7f976e000640 1 -- 172.21.15.42:0/497228261 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f9768120420 con 0x7f97740b4eb0 2024-05-28T14:07:37.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.178+0000 7f976e000640 1 -- 172.21.15.42:0/497228261 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9768016130 con 0x7f97740b4eb0 2024-05-28T14:07:37.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.206+0000 7f9779400640 1 --2- 172.21.15.42:0/497228261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f97540694e0 0x7f975406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:37.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.218+0000 7f9779400640 1 --2- 172.21.15.42:0/497228261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f97540694e0 0x7f975406b9a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9774163900 tx=0x7f9760008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:37.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:37.766+0000 7f977b200640 1 -- 172.21.15.42:0/497228261 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone81116", "target": ["mon-mgr", ""]}) -- 0x7f977404fcf0 con 0x7f97540694e0 2024-05-28T14:07:38.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:38.530+0000 7f976e000640 1 -- 172.21.15.42:0/497228261 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f977404fcf0 con 0x7f97540694e0 2024-05-28T14:07:38.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:38.538+0000 7f977b200640 1 -- 172.21.15.42:0/497228261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f97540694e0 msgr2=0x7f975406b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:38.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:38.538+0000 7f977b200640 1 --2- 172.21.15.42:0/497228261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f97540694e0 0x7f975406b9a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9774163900 tx=0x7f9760008040 comp rx=0 tx=0).stop 2024-05-28T14:07:38.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:38.538+0000 7f977b200640 1 -- 172.21.15.42:0/497228261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97740b4eb0 msgr2=0x7f9774165560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:38.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:38.538+0000 7f977b200640 1 --2- 172.21.15.42:0/497228261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97740b4eb0 0x7f9774165560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f97680ad040 tx=0x7f976801ad80 comp rx=0 tx=0).stop 2024-05-28T14:07:38.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:38.538+0000 7f977b200640 1 -- 172.21.15.42:0/497228261 shutdown_connections 2024-05-28T14:07:38.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:38.538+0000 7f977b200640 1 --2- 172.21.15.42:0/497228261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f97540694e0 0x7f975406b9a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:38.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:38.538+0000 7f977b200640 1 --2- 172.21.15.42:0/497228261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9774165aa0 0x7f97740b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:38.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:38.538+0000 7f977b200640 1 --2- 172.21.15.42:0/497228261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9774162ac0 0x7f97740b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:38.538+0000 7f977b200640 1 --2- 172.21.15.42:0/497228261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97740b4eb0 0x7f9774165560 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:38.538+0000 7f977b200640 1 -- 172.21.15.42:0/497228261 >> 172.21.15.42:0/497228261 conn(0x7f977406bbc0 msgr2=0x7f97740cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:38.538+0000 7f977b200640 1 -- 172.21.15.42:0/497228261 shutdown_connections 2024-05-28T14:07:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:38.542+0000 7f977b200640 1 -- 172.21.15.42:0/497228261 wait complete. 2024-05-28T14:07:38.585 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone81117 2024-05-28T14:07:39.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.053+0000 7fdd33200640 1 Processor -- start 2024-05-28T14:07:39.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.053+0000 7fdd33200640 1 -- start start 2024-05-28T14:07:39.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.057+0000 7fdd33200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdd2c0f04a0 0x7fdd2c0f2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:39.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.057+0000 7fdd33200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd2c034220 con 0x7fdd2c033e40 2024-05-28T14:07:39.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.057+0000 7fdd33200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd2c04fa30 con 0x7fdd2c047c20 2024-05-28T14:07:39.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.057+0000 7fdd33200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fdd2c050210 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.061+0000 7fdd32800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdd2c0f04a0 0x7fdd2c0f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:39.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.061+0000 7fdd32800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdd2c0f04a0 0x7fdd2c0f2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51252/0 (socket says 172.21.15.42:51252) 2024-05-28T14:07:39.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.061+0000 7fdd32800640 1 -- 172.21.15.42:0/219437163 learned_addr learned my addr 172.21.15.42:0/219437163 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:39.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.061+0000 7fdd32800640 1 -- 172.21.15.42:0/219437163 >> v1:172.21.15.63:6789/0 conn(0x7fdd2c047c20 legacy=0x7fdd2c0eb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:39.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.061+0000 7fdd32800640 1 -- 172.21.15.42:0/219437163 >> v1:172.21.15.42:6789/0 conn(0x7fdd2c033e40 legacy=0x7fdd2c0eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:39.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.061+0000 7fdd32800640 1 -- 172.21.15.42:0/219437163 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c04f2e0 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.069+0000 7fdd32800640 1 --2- 172.21.15.42:0/219437163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd2c0f04a0 0x7fdd2c0f2890 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fdd34065050 tx=0x7fdd3409a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb5d3f97809aa60 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:39.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.069+0000 7fdd30a00640 1 -- 172.21.15.42:0/219437163 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34128740 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.069+0000 7fdd30a00640 1 -- 172.21.15.42:0/219437163 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd341288e0 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.069+0000 7fdd30a00640 1 -- 172.21.15.42:0/219437163 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34128d70 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.073+0000 7fdd33200640 1 -- 172.21.15.42:0/219437163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd2c0f04a0 msgr2=0x7fdd2c0f2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:39.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.073+0000 7fdd33200640 1 --2- 172.21.15.42:0/219437163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd2c0f04a0 0x7fdd2c0f2890 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fdd34065050 tx=0x7fdd3409a880 comp rx=0 tx=0).stop 2024-05-28T14:07:39.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.077+0000 7fdd33200640 1 -- 172.21.15.42:0/219437163 shutdown_connections 2024-05-28T14:07:39.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.077+0000 7fdd33200640 1 --2- 172.21.15.42:0/219437163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd2c0f04a0 0x7fdd2c0f2890 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:39.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.077+0000 7fdd33200640 1 -- 172.21.15.42:0/219437163 >> 172.21.15.42:0/219437163 conn(0x7fdd2c009fc0 msgr2=0x7fdd2c00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:39.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.081+0000 7fdd33200640 1 -- 172.21.15.42:0/219437163 shutdown_connections 2024-05-28T14:07:39.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.081+0000 7fdd33200640 1 -- 172.21.15.42:0/219437163 wait complete. 2024-05-28T14:07:39.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.093+0000 7fdd33200640 1 Processor -- start 2024-05-28T14:07:39.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.093+0000 7fdd33200640 1 -- start start 2024-05-28T14:07:39.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.097+0000 7fdd33200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd2c033e40 0x7fdd2c0d5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:39.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.097+0000 7fdd33200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd2c047c20 0x7fdd2c039310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:39.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.097+0000 7fdd33200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd2c0f04a0 0x7fdd2c039850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:39.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.097+0000 7fdd31400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd2c047c20 0x7fdd2c039310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:39.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.101+0000 7fdd31400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd2c047c20 0x7fdd2c039310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:49520/0 (socket says 172.21.15.42:49520) 2024-05-28T14:07:39.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.101+0000 7fdd32800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd2c0f04a0 0x7fdd2c039850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:39.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.101+0000 7fdd32800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd2c0f04a0 0x7fdd2c039850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51264/0 (socket says 172.21.15.42:51264) 2024-05-28T14:07:39.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.101+0000 7fdd31400640 1 -- 172.21.15.42:0/3113273884 learned_addr learned my addr 172.21.15.42:0/3113273884 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:39.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.101+0000 7fdd33200640 1 -- 172.21.15.42:0/3113273884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c04f090 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.101+0000 7fdd33200640 1 -- 172.21.15.42:0/3113273884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c04fd80 con 0x7fdd2c033e40 2024-05-28T14:07:39.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.101+0000 7fdd33200640 1 -- 172.21.15.42:0/3113273884 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2c0ebfa0 con 0x7fdd2c047c20 2024-05-28T14:07:39.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.101+0000 7fdd31e00640 1 --2- 172.21.15.42:0/3113273884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd2c033e40 0x7fdd2c0d5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:39.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.105+0000 7fdd32800640 1 -- 172.21.15.42:0/3113273884 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd2c047c20 msgr2=0x7fdd2c039310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:39.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.105+0000 7fdd32800640 1 --2- 172.21.15.42:0/3113273884 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd2c047c20 0x7fdd2c039310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:39.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.105+0000 7fdd32800640 1 -- 172.21.15.42:0/3113273884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd2c033e40 msgr2=0x7fdd2c0d5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:39.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.105+0000 7fdd32800640 1 --2- 172.21.15.42:0/3113273884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd2c033e40 0x7fdd2c0d5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:39.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.105+0000 7fdd32800640 1 -- 172.21.15.42:0/3113273884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c0353d0 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.105+0000 7fdd32800640 1 --2- 172.21.15.42:0/3113273884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd2c0f04a0 0x7fdd2c039850 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fdd34065520 tx=0x7fdd340744b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:39.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.105+0000 7fdd2a000640 1 -- 172.21.15.42:0/3113273884 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34068390 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.105+0000 7fdd33200640 1 -- 172.21.15.42:0/3113273884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2c0356b0 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.109+0000 7fdd33200640 1 -- 172.21.15.42:0/3113273884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd2c035c70 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.109+0000 7fdd2a000640 1 -- 172.21.15.42:0/3113273884 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd34128880 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.109+0000 7fdd2a000640 1 -- 172.21.15.42:0/3113273884 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34067760 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.113+0000 7fdd2a000640 1 -- 172.21.15.42:0/3113273884 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7fdd34138030 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.113+0000 7fdd2a000640 1 --2- 172.21.15.42:0/3113273884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd14069460 0x7fdd1406b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:39.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.117+0000 7fdd33200640 1 -- 172.21.15.42:0/3113273884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0c0052a0 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.121+0000 7fdd2a000640 1 -- 172.21.15.42:0/3113273884 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fdd3419df70 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.129+0000 7fdd2a000640 1 -- 172.21.15.42:0/3113273884 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdd3413f020 con 0x7fdd2c0f04a0 2024-05-28T14:07:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.169+0000 7fdd31e00640 1 --2- 172.21.15.42:0/3113273884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd14069460 0x7fdd1406b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.177+0000 7fdd31e00640 1 --2- 172.21.15.42:0/3113273884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd14069460 0x7fdd1406b920 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdd20003d60 tx=0x7fdd20009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:39.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:39.741+0000 7fdd33200640 1 -- 172.21.15.42:0/3113273884 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone81117", "target": ["mon-mgr", ""]}) -- 0x7fdd0c000c90 con 0x7fdd14069460 2024-05-28T14:07:40.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.369+0000 7fdd2a000640 1 -- 172.21.15.42:0/3113273884 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdd0c000c90 con 0x7fdd14069460 2024-05-28T14:07:40.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.373+0000 7fdd33200640 1 -- 172.21.15.42:0/3113273884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd14069460 msgr2=0x7fdd1406b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:40.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.373+0000 7fdd33200640 1 --2- 172.21.15.42:0/3113273884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd14069460 0x7fdd1406b920 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdd20003d60 tx=0x7fdd20009040 comp rx=0 tx=0).stop 2024-05-28T14:07:40.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.373+0000 7fdd33200640 1 -- 172.21.15.42:0/3113273884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd2c0f04a0 msgr2=0x7fdd2c039850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:40.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.373+0000 7fdd33200640 1 --2- 172.21.15.42:0/3113273884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd2c0f04a0 0x7fdd2c039850 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fdd34065520 tx=0x7fdd340744b0 comp rx=0 tx=0).stop 2024-05-28T14:07:40.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.377+0000 7fdd33200640 1 -- 172.21.15.42:0/3113273884 shutdown_connections 2024-05-28T14:07:40.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.377+0000 7fdd33200640 1 --2- 172.21.15.42:0/3113273884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd14069460 0x7fdd1406b920 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:40.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.377+0000 7fdd33200640 1 --2- 172.21.15.42:0/3113273884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd2c0f04a0 0x7fdd2c039850 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:40.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.377+0000 7fdd33200640 1 --2- 172.21.15.42:0/3113273884 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd2c047c20 0x7fdd2c039310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:40.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.377+0000 7fdd33200640 1 --2- 172.21.15.42:0/3113273884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd2c033e40 0x7fdd2c0d5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:40.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.377+0000 7fdd33200640 1 -- 172.21.15.42:0/3113273884 >> 172.21.15.42:0/3113273884 conn(0x7fdd2c009fc0 msgr2=0x7fdd2c037d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:40.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.377+0000 7fdd33200640 1 -- 172.21.15.42:0/3113273884 shutdown_connections 2024-05-28T14:07:40.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.377+0000 7fdd33200640 1 -- 172.21.15.42:0/3113273884 wait complete. 2024-05-28T14:07:40.435 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone81118 2024-05-28T14:07:40.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.809+0000 7f8570200640 1 Processor -- start 2024-05-28T14:07:40.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.813+0000 7f8570200640 1 -- start start 2024-05-28T14:07:40.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.813+0000 7f8570200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8568162d10 0x7f856806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:40.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.813+0000 7f8570200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f85680cfb40 0x7f8568071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:40.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.813+0000 7f8570200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8568166800 con 0x7f8568072320 2024-05-28T14:07:40.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.813+0000 7f8570200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f85681630f0 con 0x7f85680cfb40 2024-05-28T14:07:40.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.813+0000 7f8570200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f85680b4500 con 0x7f8568162d10 2024-05-28T14:07:40.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.813+0000 7f856ce00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f85680cfb40 0x7f8568071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:40.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.813+0000 7f856ce00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f85680cfb40 0x7f8568071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51266/0 (socket says 172.21.15.42:51266) 2024-05-28T14:07:40.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.813+0000 7f856ce00640 1 -- 172.21.15.42:0/2816536743 learned_addr learned my addr 172.21.15.42:0/2816536743 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:40.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.817+0000 7f856d800640 1 --2- 172.21.15.42:0/2816536743 >> v2:172.21.15.63:3300/0 conn(0x7f8568162d10 0x7f856806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:40.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.817+0000 7f8567e00640 1 -- 172.21.15.42:0/2816536743 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2675654184 0 0) 0x7f8568166800 con 0x7f8568072320 2024-05-28T14:07:40.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.817+0000 7f8567e00640 1 -- 172.21.15.42:0/2816536743 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8550003530 con 0x7f8568072320 2024-05-28T14:07:40.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.817+0000 7f856ce00640 1 -- 172.21.15.42:0/2816536743 >> v2:172.21.15.63:3300/0 conn(0x7f8568162d10 msgr2=0x7f856806f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:40.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.817+0000 7f856ce00640 1 --2- 172.21.15.42:0/2816536743 >> v2:172.21.15.63:3300/0 conn(0x7f8568162d10 0x7f856806f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:40.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.817+0000 7f856ce00640 1 -- 172.21.15.42:0/2816536743 >> v1:172.21.15.63:6789/0 conn(0x7f8568072320 legacy=0x7f856806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:40.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.817+0000 7f856ce00640 1 -- 172.21.15.42:0/2816536743 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856806f8c0 con 0x7f85680cfb40 2024-05-28T14:07:40.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.821+0000 7f856ce00640 1 --2- 172.21.15.42:0/2816536743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85680cfb40 0x7f8568071de0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8568073280 tx=0x7f855401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce01fc6490d8b37 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:40.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.821+0000 7f8567e00640 1 -- 172.21.15.42:0/2816536743 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85540b0070 con 0x7f85680cfb40 2024-05-28T14:07:40.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.821+0000 7f8567e00640 1 -- 172.21.15.42:0/2816536743 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85540ab600 con 0x7f85680cfb40 2024-05-28T14:07:40.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.821+0000 7f8567e00640 1 -- 172.21.15.42:0/2816536743 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85540ab9d0 con 0x7f85680cfb40 2024-05-28T14:07:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.829+0000 7f8570200640 1 -- 172.21.15.42:0/2816536743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85680cfb40 msgr2=0x7f8568071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.829+0000 7f8570200640 1 --2- 172.21.15.42:0/2816536743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85680cfb40 0x7f8568071de0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8568073280 tx=0x7f855401c540 comp rx=0 tx=0).stop 2024-05-28T14:07:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.829+0000 7f8570200640 1 -- 172.21.15.42:0/2816536743 shutdown_connections 2024-05-28T14:07:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.829+0000 7f8570200640 1 --2- 172.21.15.42:0/2816536743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85680cfb40 0x7f8568071de0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.829+0000 7f8570200640 1 --2- 172.21.15.42:0/2816536743 >> v2:172.21.15.63:3300/0 conn(0x7f8568162d10 0x7f856806f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.829+0000 7f8570200640 1 -- 172.21.15.42:0/2816536743 >> 172.21.15.42:0/2816536743 conn(0x7f85680b21f0 msgr2=0x7f85680b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.829+0000 7f8570200640 1 -- 172.21.15.42:0/2816536743 shutdown_connections 2024-05-28T14:07:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.829+0000 7f8570200640 1 -- 172.21.15.42:0/2816536743 wait complete. 2024-05-28T14:07:40.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.833+0000 7f8570200640 1 Processor -- start 2024-05-28T14:07:40.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.833+0000 7f8570200640 1 -- start start 2024-05-28T14:07:40.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.837+0000 7f8570200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8568072320 0x7f856814c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:40.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.837+0000 7f8570200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85680cfb40 0x7f856814cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:40.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.837+0000 7f8570200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8568162d10 0x7f85680ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:40.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.837+0000 7f856ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85680cfb40 0x7f856814cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:40.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.837+0000 7f856ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85680cfb40 0x7f856814cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:47222/0 (socket says 172.21.15.42:47222) 2024-05-28T14:07:40.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.837+0000 7f856d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8568072320 0x7f856814c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:40.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.837+0000 7f8570200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f85681675a0 con 0x7f8568072320 2024-05-28T14:07:40.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.837+0000 7f856ce00640 1 -- 172.21.15.42:0/1267423345 learned_addr learned my addr 172.21.15.42:0/1267423345 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:40.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.837+0000 7f8570200640 1 -- 172.21.15.42:0/1267423345 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85680afec0 con 0x7f85680cfb40 2024-05-28T14:07:40.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.837+0000 7f8570200640 1 -- 172.21.15.42:0/1267423345 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f856814d2c0 con 0x7f8568162d10 2024-05-28T14:07:40.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.841+0000 7f856ce00640 1 -- 172.21.15.42:0/1267423345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8568162d10 msgr2=0x7f85680ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:40.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.841+0000 7f856e200640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8568162d10 0x7f85680ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:40.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.841+0000 7f856ce00640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8568162d10 0x7f85680ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:40.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.841+0000 7f856ce00640 1 -- 172.21.15.42:0/1267423345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8568072320 msgr2=0x7f856814c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:40.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.841+0000 7f856ce00640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8568072320 0x7f856814c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:40.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.841+0000 7f856ce00640 1 -- 172.21.15.42:0/1267423345 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85680a9ec0 con 0x7f85680cfb40 2024-05-28T14:07:40.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.841+0000 7f856d800640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8568072320 0x7f856814c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:07:40.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.841+0000 7f856e200640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8568162d10 0x7f85680ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:07:40.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.841+0000 7f856ce00640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85680cfb40 0x7f856814cd80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f855401a170 tx=0x7f85540192e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:40.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.845+0000 7f8565600640 1 -- 172.21.15.42:0/1267423345 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85540bb070 con 0x7f85680cfb40 2024-05-28T14:07:40.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.845+0000 7f8565600640 1 -- 172.21.15.42:0/1267423345 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85540b1040 con 0x7f85680cfb40 2024-05-28T14:07:40.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.845+0000 7f8565600640 1 -- 172.21.15.42:0/1267423345 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85540b0070 con 0x7f85680cfb40 2024-05-28T14:07:40.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.845+0000 7f8570200640 1 -- 172.21.15.42:0/1267423345 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85680aa1a0 con 0x7f85680cfb40 2024-05-28T14:07:40.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.845+0000 7f8570200640 1 -- 172.21.15.42:0/1267423345 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85680aa760 con 0x7f85680cfb40 2024-05-28T14:07:40.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.853+0000 7f8570200640 1 -- 172.21.15.42:0/1267423345 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85440052a0 con 0x7f85680cfb40 2024-05-28T14:07:40.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.853+0000 7f8565600640 1 -- 172.21.15.42:0/1267423345 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f85540aa7b0 con 0x7f85680cfb40 2024-05-28T14:07:40.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.853+0000 7f8565600640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85480694b0 0x7f854806b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:40.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.853+0000 7f8565600640 1 -- 172.21.15.42:0/1267423345 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f855411eb00 con 0x7f85680cfb40 2024-05-28T14:07:40.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.857+0000 7f856d800640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85480694b0 0x7f854806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:40.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.861+0000 7f8565600640 1 -- 172.21.15.42:0/1267423345 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8554016130 con 0x7f85680cfb40 2024-05-28T14:07:40.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:40.865+0000 7f856d800640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85480694b0 0x7f854806b970 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f855c00b9f0 tx=0x7f855c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:41.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:41.353+0000 7f8570200640 1 -- 172.21.15.42:0/1267423345 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone81118", "target": ["mon-mgr", ""]}) -- 0x7f8544000c90 con 0x7f85480694b0 2024-05-28T14:07:41.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:41.985+0000 7f8565600640 1 -- 172.21.15.42:0/1267423345 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8544000c90 con 0x7f85480694b0 2024-05-28T14:07:41.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:41.989+0000 7f8570200640 1 -- 172.21.15.42:0/1267423345 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85480694b0 msgr2=0x7f854806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:41.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:41.989+0000 7f8570200640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85480694b0 0x7f854806b970 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f855c00b9f0 tx=0x7f855c008040 comp rx=0 tx=0).stop 2024-05-28T14:07:41.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:41.989+0000 7f8570200640 1 -- 172.21.15.42:0/1267423345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85680cfb40 msgr2=0x7f856814cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:41.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:41.989+0000 7f8570200640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85680cfb40 0x7f856814cd80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f855401a170 tx=0x7f85540192e0 comp rx=0 tx=0).stop 2024-05-28T14:07:41.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:41.989+0000 7f8570200640 1 -- 172.21.15.42:0/1267423345 shutdown_connections 2024-05-28T14:07:41.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:41.989+0000 7f8570200640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85480694b0 0x7f854806b970 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:41.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:41.989+0000 7f8570200640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8568162d10 0x7f85680ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:41.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:41.989+0000 7f8570200640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85680cfb40 0x7f856814cd80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:41.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:41.989+0000 7f8570200640 1 --2- 172.21.15.42:0/1267423345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8568072320 0x7f856814c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:41.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:41.989+0000 7f8570200640 1 -- 172.21.15.42:0/1267423345 >> 172.21.15.42:0/1267423345 conn(0x7f85680b21f0 msgr2=0x7f85680b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:41.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:41.989+0000 7f8570200640 1 -- 172.21.15.42:0/1267423345 shutdown_connections 2024-05-28T14:07:41.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:41.989+0000 7f8570200640 1 -- 172.21.15.42:0/1267423345 wait complete. 2024-05-28T14:07:42.057 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone81119 2024-05-28T14:07:42.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.449+0000 7fe6c9a00640 1 Processor -- start 2024-05-28T14:07:42.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.449+0000 7fe6c9a00640 1 -- start start 2024-05-28T14:07:42.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.453+0000 7fe6c9a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6c4071de0 0x7fe6c40721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:42.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.453+0000 7fe6c9a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe6c40ce3b0 0x7fe6c40ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:42.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.453+0000 7fe6c9a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6c4072700 con 0x7fe6c4141d10 2024-05-28T14:07:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.453+0000 7fe6c9a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fe6c4160230 con 0x7fe6c40ce3b0 2024-05-28T14:07:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.453+0000 7fe6c9a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c4160510 con 0x7fe6c4071de0 2024-05-28T14:07:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.453+0000 7fe6ba000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe6c40ce3b0 0x7fe6c40ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.453+0000 7fe6ba000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe6c40ce3b0 0x7fe6c40ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51276/0 (socket says 172.21.15.42:51276) 2024-05-28T14:07:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.453+0000 7fe6ba000640 1 -- 172.21.15.42:0/434734515 learned_addr learned my addr 172.21.15.42:0/434734515 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.453+0000 7fe6c2a00640 1 --2- 172.21.15.42:0/434734515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6c4071de0 0x7fe6c40721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:42.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.453+0000 7fe6ba000640 1 -- 172.21.15.42:0/434734515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6c4071de0 msgr2=0x7fe6c40721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:42.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.453+0000 7fe6ba000640 1 --2- 172.21.15.42:0/434734515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6c4071de0 0x7fe6c40721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:42.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.453+0000 7fe6ba000640 1 -- 172.21.15.42:0/434734515 >> v1:172.21.15.63:6789/0 conn(0x7fe6c4141d10 legacy=0x7fe6c41421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:42.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.453+0000 7fe6ba000640 1 -- 172.21.15.42:0/434734515 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c41607f0 con 0x7fe6c40ce3b0 2024-05-28T14:07:42.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.457+0000 7fe6c2a00640 1 --2- 172.21.15.42:0/434734515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6c4071de0 0x7fe6c40721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:07:42.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.461+0000 7fe6ba000640 1 --2- 172.21.15.42:0/434734515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6c40ce3b0 0x7fe6c40ce810 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009870 tx=0x7fe6ac01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cd9dd60d367ac1 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:42.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.465+0000 7fe6c2000640 1 -- 172.21.15.42:0/434734515 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0b0070 con 0x7fe6c40ce3b0 2024-05-28T14:07:42.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.465+0000 7fe6c2000640 1 -- 172.21.15.42:0/434734515 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6ac0a4e50 con 0x7fe6c40ce3b0 2024-05-28T14:07:42.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.465+0000 7fe6c2000640 1 -- 172.21.15.42:0/434734515 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0ab820 con 0x7fe6c40ce3b0 2024-05-28T14:07:42.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.469+0000 7fe6c9a00640 1 -- 172.21.15.42:0/434734515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6c40ce3b0 msgr2=0x7fe6c40ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:42.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.469+0000 7fe6c9a00640 1 --2- 172.21.15.42:0/434734515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6c40ce3b0 0x7fe6c40ce810 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009870 tx=0x7fe6ac01c400 comp rx=0 tx=0).stop 2024-05-28T14:07:42.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.469+0000 7fe6c9a00640 1 -- 172.21.15.42:0/434734515 shutdown_connections 2024-05-28T14:07:42.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.469+0000 7fe6c9a00640 1 --2- 172.21.15.42:0/434734515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6c40ce3b0 0x7fe6c40ce810 secure :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009870 tx=0x7fe6ac01c400 comp rx=0 tx=0).stop 2024-05-28T14:07:42.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.469+0000 7fe6c9a00640 1 --2- 172.21.15.42:0/434734515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6c4071de0 0x7fe6c40721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:42.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.469+0000 7fe6c9a00640 1 -- 172.21.15.42:0/434734515 >> 172.21.15.42:0/434734515 conn(0x7fe6c40b21f0 msgr2=0x7fe6c40b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:42.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.469+0000 7fe6c9a00640 1 -- 172.21.15.42:0/434734515 shutdown_connections 2024-05-28T14:07:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.469+0000 7fe6c9a00640 1 -- 172.21.15.42:0/434734515 wait complete. 2024-05-28T14:07:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.477+0000 7fe6c9a00640 1 Processor -- start 2024-05-28T14:07:42.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.477+0000 7fe6c9a00640 1 -- start start 2024-05-28T14:07:42.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.477+0000 7fe6c9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6c4071de0 0x7fe6c42007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:42.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.477+0000 7fe6c9a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6c4141d10 0x7fe6c41316e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:42.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.477+0000 7fe6c9a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c40b1380 0x7fe6c40b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:42.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.477+0000 7fe6c9a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4169e30 con 0x7fe6c4071de0 2024-05-28T14:07:42.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.477+0000 7fe6c9a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4169cb0 con 0x7fe6c40b1380 2024-05-28T14:07:42.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.477+0000 7fe6c9a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c4169fb0 con 0x7fe6c4141d10 2024-05-28T14:07:42.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.477+0000 7fe6c3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c40b1380 0x7fe6c40b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:42.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.477+0000 7fe6ba000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6c4141d10 0x7fe6c41316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:42.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.477+0000 7fe6c3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c40b1380 0x7fe6c40b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:47224/0 (socket says 172.21.15.42:47224) 2024-05-28T14:07:42.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.477+0000 7fe6c3400640 1 -- 172.21.15.42:0/1593958082 learned_addr learned my addr 172.21.15.42:0/1593958082 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:42.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.477+0000 7fe6c2a00640 1 --2- 172.21.15.42:0/1593958082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6c4071de0 0x7fe6c42007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:42.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.481+0000 7fe6ba000640 1 -- 172.21.15.42:0/1593958082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c40b1380 msgr2=0x7fe6c40b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:42.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.481+0000 7fe6ba000640 1 --2- 172.21.15.42:0/1593958082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c40b1380 0x7fe6c40b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:42.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.481+0000 7fe6ba000640 1 -- 172.21.15.42:0/1593958082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6c4071de0 msgr2=0x7fe6c42007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:42.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.481+0000 7fe6ba000640 1 --2- 172.21.15.42:0/1593958082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6c4071de0 0x7fe6c42007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:42.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.481+0000 7fe6ba000640 1 -- 172.21.15.42:0/1593958082 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c413a4a0 con 0x7fe6c4141d10 2024-05-28T14:07:42.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.485+0000 7fe6c2a00640 1 --2- 172.21.15.42:0/1593958082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6c4071de0 0x7fe6c42007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:07:42.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.485+0000 7fe6ba000640 1 --2- 172.21.15.42:0/1593958082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6c4141d10 0x7fe6c41316e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe6c406ffc0 tx=0x7fe6ac0191a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:42.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.489+0000 7fe6bb400640 1 -- 172.21.15.42:0/1593958082 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0b0050 con 0x7fe6c4141d10 2024-05-28T14:07:42.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.489+0000 7fe6bb400640 1 -- 172.21.15.42:0/1593958082 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6ac0b1040 con 0x7fe6c4141d10 2024-05-28T14:07:42.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.489+0000 7fe6bb400640 1 -- 172.21.15.42:0/1593958082 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0b5440 con 0x7fe6c4141d10 2024-05-28T14:07:42.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.489+0000 7fe6c9a00640 1 -- 172.21.15.42:0/1593958082 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c4130c00 con 0x7fe6c4141d10 2024-05-28T14:07:42.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.489+0000 7fe6c9a00640 1 -- 172.21.15.42:0/1593958082 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c41311c0 con 0x7fe6c4141d10 2024-05-28T14:07:42.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.493+0000 7fe6c9a00640 1 -- 172.21.15.42:0/1593958082 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c41313c0 con 0x7fe6c4141d10 2024-05-28T14:07:42.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.497+0000 7fe6bb400640 1 -- 172.21.15.42:0/1593958082 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7fe6ac0bc030 con 0x7fe6c4141d10 2024-05-28T14:07:42.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.501+0000 7fe6bb400640 1 --2- 172.21.15.42:0/1593958082 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6a0069560 0x7fe6a006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:42.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.501+0000 7fe6bb400640 1 -- 172.21.15.42:0/1593958082 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fe6ac11ecd0 con 0x7fe6c4141d10 2024-05-28T14:07:42.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.505+0000 7fe6bb400640 1 -- 172.21.15.42:0/1593958082 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe6c41313c0 con 0x7fe6c4141d10 2024-05-28T14:07:42.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.513+0000 7fe6c2a00640 1 --2- 172.21.15.42:0/1593958082 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6a0069560 0x7fe6a006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:42.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:42.517+0000 7fe6c2a00640 1 --2- 172.21.15.42:0/1593958082 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6a0069560 0x7fe6a006ba20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400a9b0 tx=0x7fe6b4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:42.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:43.009+0000 7fe6c9a00640 1 -- 172.21.15.42:0/1593958082 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone81119", "target": ["mon-mgr", ""]}) -- 0x7fe6c413a780 con 0x7fe6a0069560 2024-05-28T14:07:43.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:43.913+0000 7fe6bb400640 1 -- 172.21.15.42:0/1593958082 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe6c413a780 con 0x7fe6a0069560 2024-05-28T14:07:43.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:43.929+0000 7fe6c9a00640 1 -- 172.21.15.42:0/1593958082 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6a0069560 msgr2=0x7fe6a006ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:43.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:43.929+0000 7fe6c9a00640 1 --2- 172.21.15.42:0/1593958082 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6a0069560 0x7fe6a006ba20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400a9b0 tx=0x7fe6b4008040 comp rx=0 tx=0).stop 2024-05-28T14:07:43.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:43.929+0000 7fe6c9a00640 1 -- 172.21.15.42:0/1593958082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6c4141d10 msgr2=0x7fe6c41316e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:43.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:43.929+0000 7fe6c9a00640 1 --2- 172.21.15.42:0/1593958082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6c4141d10 0x7fe6c41316e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe6c406ffc0 tx=0x7fe6ac0191a0 comp rx=0 tx=0).stop 2024-05-28T14:07:43.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:43.929+0000 7fe6c9a00640 1 -- 172.21.15.42:0/1593958082 shutdown_connections 2024-05-28T14:07:43.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:43.929+0000 7fe6c9a00640 1 --2- 172.21.15.42:0/1593958082 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6a0069560 0x7fe6a006ba20 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:43.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:43.929+0000 7fe6c9a00640 1 --2- 172.21.15.42:0/1593958082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6c40b1380 0x7fe6c40b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:43.929+0000 7fe6c9a00640 1 --2- 172.21.15.42:0/1593958082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6c4141d10 0x7fe6c41316e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:43.929+0000 7fe6c9a00640 1 --2- 172.21.15.42:0/1593958082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6c4071de0 0x7fe6c42007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:43.929+0000 7fe6c9a00640 1 -- 172.21.15.42:0/1593958082 >> 172.21.15.42:0/1593958082 conn(0x7fe6c40b21f0 msgr2=0x7fe6c40b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:43.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:43.929+0000 7fe6c9a00640 1 -- 172.21.15.42:0/1593958082 shutdown_connections 2024-05-28T14:07:43.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:43.929+0000 7fe6c9a00640 1 -- 172.21.15.42:0/1593958082 wait complete. 2024-05-28T14:07:44.004 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone81120 2024-05-28T14:07:44.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.533+0000 7f7a4b200640 1 Processor -- start 2024-05-28T14:07:44.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.533+0000 7f7a4b200640 1 -- start start 2024-05-28T14:07:44.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.537+0000 7f7a4b200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7a4c099490 0x7f7a4c0b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:44.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.537+0000 7f7a4b200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7a4c0cee50 0x7f7a4c0cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:44.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.537+0000 7f7a4b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4c0b08f0 0x7f7a4c0cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:44.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.537+0000 7f7a4b200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f7a4c0cd740 con 0x7f7a4c0cee50 2024-05-28T14:07:44.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.537+0000 7f7a4b200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7a4c0cfd30 con 0x7f7a4c099490 2024-05-28T14:07:44.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.537+0000 7f7a4b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4c0ccb30 con 0x7f7a4c0b08f0 2024-05-28T14:07:44.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.537+0000 7f7a49e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7a4c099490 0x7f7a4c0b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:44.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.537+0000 7f7a49400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7a4c0cee50 0x7f7a4c0cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:44.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.541+0000 7f7a49e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7a4c099490 0x7f7a4c0b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:47234/0 (socket says 172.21.15.42:47234) 2024-05-28T14:07:44.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.541+0000 7f7a49e00640 1 -- 172.21.15.42:0/2840356428 learned_addr learned my addr 172.21.15.42:0/2840356428 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:44.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.541+0000 7f7a49400640 1 -- 172.21.15.42:0/2840356428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4c0b08f0 msgr2=0x7f7a4c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:44.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.541+0000 7f7a49400640 1 --2- 172.21.15.42:0/2840356428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4c0b08f0 0x7f7a4c0cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:44.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.541+0000 7f7a49400640 1 -- 172.21.15.42:0/2840356428 >> v2:172.21.15.63:3300/0 conn(0x7f7a4c099490 msgr2=0x7f7a4c0b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:44.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.541+0000 7f7a49400640 1 --2- 172.21.15.42:0/2840356428 >> v2:172.21.15.63:3300/0 conn(0x7f7a4c099490 0x7f7a4c0b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:44.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.541+0000 7f7a49400640 1 -- 172.21.15.42:0/2840356428 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c0cce10 con 0x7f7a4c0cee50 2024-05-28T14:07:44.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.545+0000 7f7a49400640 1 --2- 172.21.15.42:0/2840356428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4c0cee50 0x7f7a4c0cf2b0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c068760 tx=0x7f7a3801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3116d453df4111ef server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:44.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.549+0000 7f7a48a00640 1 -- 172.21.15.42:0/2840356428 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a380b0070 con 0x7f7a4c0cee50 2024-05-28T14:07:44.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.549+0000 7f7a48a00640 1 -- 172.21.15.42:0/2840356428 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a380ab600 con 0x7f7a4c0cee50 2024-05-28T14:07:44.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.549+0000 7f7a48a00640 1 -- 172.21.15.42:0/2840356428 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a380ab9d0 con 0x7f7a4c0cee50 2024-05-28T14:07:44.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.549+0000 7f7a4b200640 1 -- 172.21.15.42:0/2840356428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4c0cee50 msgr2=0x7f7a4c0cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:44.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.549+0000 7f7a4b200640 1 --2- 172.21.15.42:0/2840356428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4c0cee50 0x7f7a4c0cf2b0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c068760 tx=0x7f7a3801c330 comp rx=0 tx=0).stop 2024-05-28T14:07:44.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.557+0000 7f7a4b200640 1 -- 172.21.15.42:0/2840356428 shutdown_connections 2024-05-28T14:07:44.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.557+0000 7f7a4b200640 1 --2- 172.21.15.42:0/2840356428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4c0b08f0 0x7f7a4c0cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.557+0000 7f7a4b200640 1 --2- 172.21.15.42:0/2840356428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4c0cee50 0x7f7a4c0cf2b0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.557+0000 7f7a4b200640 1 --2- 172.21.15.42:0/2840356428 >> v2:172.21.15.63:3300/0 conn(0x7f7a4c099490 0x7f7a4c0b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.557+0000 7f7a4b200640 1 -- 172.21.15.42:0/2840356428 >> 172.21.15.42:0/2840356428 conn(0x7f7a4c0b1d60 msgr2=0x7f7a4c0b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:44.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.557+0000 7f7a4b200640 1 -- 172.21.15.42:0/2840356428 shutdown_connections 2024-05-28T14:07:44.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.557+0000 7f7a4b200640 1 -- 172.21.15.42:0/2840356428 wait complete. 2024-05-28T14:07:44.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.565+0000 7f7a4b200640 1 Processor -- start 2024-05-28T14:07:44.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.569+0000 7f7a4b200640 1 -- start start 2024-05-28T14:07:44.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.573+0000 7f7a4b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4c099490 0x7f7a4c1699e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:44.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.581+0000 7f7a4b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a4c0b08f0 0x7f7a4c16c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:44.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.581+0000 7f7a4b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4c0cee50 0x7f7a4c16a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:44.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.581+0000 7f7a4b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c073910 con 0x7f7a4c099490 2024-05-28T14:07:44.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.581+0000 7f7a4b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c073790 con 0x7f7a4c0b08f0 2024-05-28T14:07:44.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.581+0000 7f7a4b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4c073a90 con 0x7f7a4c0cee50 2024-05-28T14:07:44.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.581+0000 7f7a49e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4c099490 0x7f7a4c1699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.581+0000 7f7a49e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4c099490 0x7f7a4c1699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51312/0 (socket says 172.21.15.42:51312) 2024-05-28T14:07:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.581+0000 7f7a49e00640 1 -- 172.21.15.42:0/3422216769 learned_addr learned my addr 172.21.15.42:0/3422216769 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.581+0000 7f7a4a800640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4c0cee50 0x7f7a4c16a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.585+0000 7f7a49400640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a4c0b08f0 0x7f7a4c16c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.585+0000 7f7a4a800640 1 -- 172.21.15.42:0/3422216769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a4c0b08f0 msgr2=0x7f7a4c16c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.585+0000 7f7a4a800640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a4c0b08f0 0x7f7a4c16c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.585+0000 7f7a4a800640 1 -- 172.21.15.42:0/3422216769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4c099490 msgr2=0x7f7a4c1699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.585+0000 7f7a4a800640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4c099490 0x7f7a4c1699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:44.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.585+0000 7f7a4a800640 1 -- 172.21.15.42:0/3422216769 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c16c030 con 0x7f7a4c0cee50 2024-05-28T14:07:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.585+0000 7f7a49400640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a4c0b08f0 0x7f7a4c16c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:07:44.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.585+0000 7f7a4a800640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4c0cee50 0x7f7a4c16a210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400bdf0 tx=0x7f7a4400bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.593+0000 7f7a42000640 1 -- 172.21.15.42:0/3422216769 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44019070 con 0x7f7a4c0cee50 2024-05-28T14:07:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.593+0000 7f7a42000640 1 -- 172.21.15.42:0/3422216769 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a44014440 con 0x7f7a4c0cee50 2024-05-28T14:07:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.593+0000 7f7a42000640 1 -- 172.21.15.42:0/3422216769 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44014830 con 0x7f7a4c0cee50 2024-05-28T14:07:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.593+0000 7f7a4b200640 1 -- 172.21.15.42:0/3422216769 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4c0ad290 con 0x7f7a4c0cee50 2024-05-28T14:07:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.597+0000 7f7a4b200640 1 -- 172.21.15.42:0/3422216769 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4c0ad850 con 0x7f7a4c0cee50 2024-05-28T14:07:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.601+0000 7f7a4b200640 1 -- 172.21.15.42:0/3422216769 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a240052a0 con 0x7f7a4c0cee50 2024-05-28T14:07:44.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.605+0000 7f7a42000640 1 -- 172.21.15.42:0/3422216769 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f7a4401c070 con 0x7f7a4c0cee50 2024-05-28T14:07:44.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.605+0000 7f7a42000640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a2c0694b0 0x7f7a2c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:44.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.613+0000 7f7a42000640 1 -- 172.21.15.42:0/3422216769 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f7a44054e80 con 0x7f7a4c0cee50 2024-05-28T14:07:44.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.625+0000 7f7a49e00640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a2c0694b0 0x7f7a2c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:44.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.625+0000 7f7a42000640 1 -- 172.21.15.42:0/3422216769 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7a44056de0 con 0x7f7a4c0cee50 2024-05-28T14:07:44.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:44.661+0000 7f7a49e00640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a2c0694b0 0x7f7a2c06b970 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c086610 tx=0x7f7a3c016040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:45.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:45.125+0000 7f7a4b200640 1 -- 172.21.15.42:0/3422216769 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone81120", "target": ["mon-mgr", ""]}) -- 0x7f7a24000c90 con 0x7f7a2c0694b0 2024-05-28T14:07:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:45.877+0000 7f7a42000640 1 -- 172.21.15.42:0/3422216769 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7a24000c90 con 0x7f7a2c0694b0 2024-05-28T14:07:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:45.893+0000 7f7a4b200640 1 -- 172.21.15.42:0/3422216769 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a2c0694b0 msgr2=0x7f7a2c06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:45.893+0000 7f7a4b200640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a2c0694b0 0x7f7a2c06b970 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c086610 tx=0x7f7a3c016040 comp rx=0 tx=0).stop 2024-05-28T14:07:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:45.893+0000 7f7a4b200640 1 -- 172.21.15.42:0/3422216769 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4c0cee50 msgr2=0x7f7a4c16a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:45.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:45.893+0000 7f7a4b200640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4c0cee50 0x7f7a4c16a210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400bdf0 tx=0x7f7a4400bee0 comp rx=0 tx=0).stop 2024-05-28T14:07:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:45.893+0000 7f7a4b200640 1 -- 172.21.15.42:0/3422216769 shutdown_connections 2024-05-28T14:07:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:45.893+0000 7f7a4b200640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a2c0694b0 0x7f7a2c06b970 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:45.893+0000 7f7a4b200640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4c0cee50 0x7f7a4c16a210 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:45.893+0000 7f7a4b200640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a4c0b08f0 0x7f7a4c16c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:45.893+0000 7f7a4b200640 1 --2- 172.21.15.42:0/3422216769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4c099490 0x7f7a4c1699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:45.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:45.893+0000 7f7a4b200640 1 -- 172.21.15.42:0/3422216769 >> 172.21.15.42:0/3422216769 conn(0x7f7a4c0b1d60 msgr2=0x7f7a4c0cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:45.893+0000 7f7a4b200640 1 -- 172.21.15.42:0/3422216769 shutdown_connections 2024-05-28T14:07:45.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:45.893+0000 7f7a4b200640 1 -- 172.21.15.42:0/3422216769 wait complete. 2024-05-28T14:07:47.945 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:07:48.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.309+0000 7fc9b9600640 1 Processor -- start 2024-05-28T14:07:48.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.313+0000 7fc9b9600640 1 -- start start 2024-05-28T14:07:48.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.313+0000 7fc9b9600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc9b406c100 0x7fc9b416da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:48.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.313+0000 7fc9b9600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9b4167650 con 0x7fc9b4162cd0 2024-05-28T14:07:48.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.313+0000 7fc9b9600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9b40d0180 con 0x7fc9b4167100 2024-05-28T14:07:48.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.313+0000 7fc9b9600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc9b40b4310 con 0x7fc9b406c100 2024-05-28T14:07:48.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.313+0000 7fc9b2a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc9b4162cd0 0x7fc9b40b3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60196/0 (socket says 172.21.15.42:60196) 2024-05-28T14:07:48.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.313+0000 7fc9b2a00640 1 -- 172.21.15.42:0/906672701 learned_addr learned my addr 172.21.15.42:0/906672701 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:07:48.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.313+0000 7fc9b1600640 1 -- 172.21.15.42:0/906672701 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1367673561 0 0) 0x7fc9b40d0180 con 0x7fc9b4167100 2024-05-28T14:07:48.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.313+0000 7fc9b1600640 1 -- 172.21.15.42:0/906672701 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc998003530 con 0x7fc9b4167100 2024-05-28T14:07:48.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b1600640 1 -- 172.21.15.42:0/906672701 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2531905682 0 0) 0x7fc998003530 con 0x7fc9b4167100 2024-05-28T14:07:48.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b3400640 1 --2- 172.21.15.42:0/906672701 >> v2:172.21.15.42:3300/0 conn(0x7fc9b406c100 0x7fc9b416da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:48.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b1600640 1 -- 172.21.15.42:0/906672701 >> v2:172.21.15.42:3300/0 conn(0x7fc9b406c100 msgr2=0x7fc9b416da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:48.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b1600640 1 --2- 172.21.15.42:0/906672701 >> v2:172.21.15.42:3300/0 conn(0x7fc9b406c100 0x7fc9b416da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:48.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b1600640 1 -- 172.21.15.42:0/906672701 >> v1:172.21.15.42:6789/0 conn(0x7fc9b4162cd0 legacy=0x7fc9b40b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:48.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b1600640 1 -- 172.21.15.42:0/906672701 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b40d0460 con 0x7fc9b4167100 2024-05-28T14:07:48.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b1600640 1 -- 172.21.15.42:0/906672701 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fc9a00015b0 con 0x7fc9b4167100 2024-05-28T14:07:48.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b1600640 1 -- 172.21.15.42:0/906672701 >> v1:172.21.15.63:6789/0 conn(0x7fc9b4167100 legacy=0x7fc9b406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:48.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b1600640 1 --2- 172.21.15.42:0/906672701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc998003a10 0x7fc998003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:48.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b1600640 1 --2- 172.21.15.42:0/906672701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc998004810 0x7fc998004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:48.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b1600640 1 --2- 172.21.15.42:0/906672701 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9980053d0 0x7fc99800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:48.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b1600640 1 -- 172.21.15.42:0/906672701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b40b4310 con 0x7fc998003a10 2024-05-28T14:07:48.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b1600640 1 -- 172.21.15.42:0/906672701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc99800c240 con 0x7fc998004810 2024-05-28T14:07:48.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b1600640 1 -- 172.21.15.42:0/906672701 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc99800c440 con 0x7fc9980053d0 2024-05-28T14:07:48.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b3400640 1 --2- 172.21.15.42:0/906672701 >> v2:172.21.15.42:3300/0 conn(0x7fc9b406c100 0x7fc9b416da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:07:48.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b2a00640 1 --2- 172.21.15.42:0/906672701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc998003a10 0x7fc998003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:48.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b2000640 1 --2- 172.21.15.42:0/906672701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc998004810 0x7fc998004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:48.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b3400640 1 --2- 172.21.15.42:0/906672701 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9980053d0 0x7fc99800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:48.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b2a00640 1 -- 172.21.15.42:0/906672701 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9980053d0 msgr2=0x7fc99800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:48.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b2a00640 1 --2- 172.21.15.42:0/906672701 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9980053d0 0x7fc99800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:48.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b2a00640 1 -- 172.21.15.42:0/906672701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc998004810 msgr2=0x7fc998004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:48.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b2a00640 1 --2- 172.21.15.42:0/906672701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc998004810 0x7fc998004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:48.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b2a00640 1 -- 172.21.15.42:0/906672701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc998003530 con 0x7fc998003a10 2024-05-28T14:07:48.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.317+0000 7fc9b3400640 1 -- 172.21.15.42:0/906672701 reap_dead start 2024-05-28T14:07:48.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.325+0000 7fc9b2a00640 1 --2- 172.21.15.42:0/906672701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc998003a10 0x7fc998003e30 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4001830 tx=0x7fc9a401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d925690b6994af0d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:48.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.325+0000 7fc9b1600640 1 -- 172.21.15.42:0/906672701 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4001450 con 0x7fc998003a10 2024-05-28T14:07:48.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.325+0000 7fc9b1600640 1 -- 172.21.15.42:0/906672701 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9a4004ab0 con 0x7fc998003a10 2024-05-28T14:07:48.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.325+0000 7fc9b1600640 1 -- 172.21.15.42:0/906672701 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4007070 con 0x7fc998003a10 2024-05-28T14:07:48.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.325+0000 7fc9b9600640 1 -- 172.21.15.42:0/906672701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc998003a10 msgr2=0x7fc998003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:48.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.325+0000 7fc9b9600640 1 --2- 172.21.15.42:0/906672701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc998003a10 0x7fc998003e30 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4001830 tx=0x7fc9a401c4a0 comp rx=0 tx=0).stop 2024-05-28T14:07:48.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.325+0000 7fc9b9600640 1 -- 172.21.15.42:0/906672701 shutdown_connections 2024-05-28T14:07:48.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.325+0000 7fc9b9600640 1 --2- 172.21.15.42:0/906672701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc998003a10 0x7fc998003e30 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:48.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.325+0000 7fc9b9600640 1 -- 172.21.15.42:0/906672701 >> 172.21.15.42:0/906672701 conn(0x7fc9b40d07c0 msgr2=0x7fc9b40d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:48.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.325+0000 7fc9b9600640 1 -- 172.21.15.42:0/906672701 shutdown_connections 2024-05-28T14:07:48.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.325+0000 7fc9b9600640 1 -- 172.21.15.42:0/906672701 wait complete. 2024-05-28T14:07:48.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.329+0000 7fc9b9600640 1 Processor -- start 2024-05-28T14:07:48.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.329+0000 7fc9b9600640 1 -- start start 2024-05-28T14:07:48.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.333+0000 7fc9b9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b4074c30 0x7fc9b415dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:48.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.333+0000 7fc9b9600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b415e1b0 0x7fc9b41566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:48.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.333+0000 7fc9b9600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9b4075010 0x7fc9b4156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:48.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.333+0000 7fc9b9600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b40b4a60 con 0x7fc9b4074c30 2024-05-28T14:07:48.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.333+0000 7fc9b9600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b40b45f0 con 0x7fc9b415e1b0 2024-05-28T14:07:48.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.333+0000 7fc9b9600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b40729d0 con 0x7fc9b4075010 2024-05-28T14:07:48.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.333+0000 7fc9b2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b4074c30 0x7fc9b415dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:48.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.333+0000 7fc9b2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b4074c30 0x7fc9b415dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44910/0 (socket says 172.21.15.42:44910) 2024-05-28T14:07:48.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.333+0000 7fc9b2a00640 1 -- 172.21.15.42:0/1784374426 learned_addr learned my addr 172.21.15.42:0/1784374426 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:48.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.337+0000 7fc9b2a00640 1 -- 172.21.15.42:0/1784374426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9b4075010 msgr2=0x7fc9b4156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:48.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.337+0000 7fc9b2a00640 1 --2- 172.21.15.42:0/1784374426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9b4075010 0x7fc9b4156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:48.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.337+0000 7fc9b2a00640 1 -- 172.21.15.42:0/1784374426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b415e1b0 msgr2=0x7fc9b41566f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T14:07:48.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.337+0000 7fc9b2a00640 1 --2- 172.21.15.42:0/1784374426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b415e1b0 0x7fc9b41566f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:48.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.337+0000 7fc9b2a00640 1 -- 172.21.15.42:0/1784374426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b41fd500 con 0x7fc9b4074c30 2024-05-28T14:07:48.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.337+0000 7fc9b2a00640 1 --2- 172.21.15.42:0/1784374426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b4074c30 0x7fc9b415dc70 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fc9a401c2d0 tx=0x7fc998005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:48.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.337+0000 7fc9aaa00640 1 -- 172.21.15.42:0/1784374426 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40048f0 con 0x7fc9b4074c30 2024-05-28T14:07:48.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.337+0000 7fc9aaa00640 1 -- 172.21.15.42:0/1784374426 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9a4004c60 con 0x7fc9b4074c30 2024-05-28T14:07:48.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.337+0000 7fc9aaa00640 1 -- 172.21.15.42:0/1784374426 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4007320 con 0x7fc9b4074c30 2024-05-28T14:07:48.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.337+0000 7fc9b9600640 1 -- 172.21.15.42:0/1784374426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b4159450 con 0x7fc9b4074c30 2024-05-28T14:07:48.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.337+0000 7fc9b9600640 1 -- 172.21.15.42:0/1784374426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b4159a10 con 0x7fc9b4074c30 2024-05-28T14:07:48.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.341+0000 7fc9b9600640 1 -- 172.21.15.42:0/1784374426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b40691a0 con 0x7fc9b4074c30 2024-05-28T14:07:48.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.349+0000 7fc9aaa00640 1 -- 172.21.15.42:0/1784374426 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7fc9a40074c0 con 0x7fc9b4074c30 2024-05-28T14:07:48.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.349+0000 7fc9aaa00640 1 --2- 172.21.15.42:0/1784374426 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc994069530 0x7fc99406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:48.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.349+0000 7fc9aaa00640 1 -- 172.21.15.42:0/1784374426 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fc9a411ecd0 con 0x7fc9b4074c30 2024-05-28T14:07:48.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.357+0000 7fc9b2000640 1 --2- 172.21.15.42:0/1784374426 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc994069530 0x7fc99406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:48.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.357+0000 7fc9aaa00640 1 -- 172.21.15.42:0/1784374426 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc9a40161b0 con 0x7fc9b4074c30 2024-05-28T14:07:48.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:48.365+0000 7fc9b2000640 1 --2- 172.21.15.42:0/1784374426 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc994069530 0x7fc99406b9f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0000c10 tx=0x7fc9a0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:48.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.005+0000 7fc9b9600640 1 -- 172.21.15.42:0/1784374426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc9b406c7c0 con 0x7fc9b4074c30 2024-05-28T14:07:49.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.017+0000 7fc9aaa00640 1 -- 172.21.15.42:0/1784374426 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2420 (secure 0 0 0) 0x7fc9a40f0910 con 0x7fc9b4074c30 2024-05-28T14:07:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:07:49.001 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:07:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:07:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:07:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:07:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:07:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:07:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:07:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:07:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:07:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:07:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:07:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:07:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:07:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:07:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:07:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 590, 2024-05-28T14:07:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:07:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:07:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 76, 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:07:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 7065, 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 29430221080, 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 58641145856, 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 714452967424, 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 575857335, 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 574862124, 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 139, 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 153 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:07:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 47, 2024-05-28T14:07:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5, 2024-05-28T14:07:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:07:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:07:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:07:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:07:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:07:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 5, 2024-05-28T14:07:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:07:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "a", 2024-05-28T14:07:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:07:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 7083 2024-05-28T14:07:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:07:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:07:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:07:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:07:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:07:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:07:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:07:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:07:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:07:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:07:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 14, 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:07:33.038180+0000", 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "346e2288-a00b-4e81-8626-13f7a50e5c02": { 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "12 ongoing+pending clones; Avg progress is 18.924% (32s)\n [=====.......................] (remaining: 2m)", 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.1892399936914444, 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "d3383476-f722-479f-9c40-e404f8ef3423": { 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "4 ongoing clones; Avg progress is 3.971% (38s)\n [=...........................] (remaining: 15m)", 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.039710000157356262, 2024-05-28T14:07:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:07:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:07:49.008 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:07:49.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.021+0000 7fc9b9600640 1 -- 172.21.15.42:0/1784374426 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc994069530 msgr2=0x7fc99406b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:49.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.021+0000 7fc9b9600640 1 --2- 172.21.15.42:0/1784374426 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc994069530 0x7fc99406b9f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0000c10 tx=0x7fc9a0009040 comp rx=0 tx=0).stop 2024-05-28T14:07:49.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.021+0000 7fc9b9600640 1 -- 172.21.15.42:0/1784374426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b4074c30 msgr2=0x7fc9b415dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:49.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.021+0000 7fc9b9600640 1 --2- 172.21.15.42:0/1784374426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b4074c30 0x7fc9b415dc70 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fc9a401c2d0 tx=0x7fc998005840 comp rx=0 tx=0).stop 2024-05-28T14:07:49.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.021+0000 7fc9b9600640 1 -- 172.21.15.42:0/1784374426 shutdown_connections 2024-05-28T14:07:49.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.021+0000 7fc9b9600640 1 --2- 172.21.15.42:0/1784374426 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc994069530 0x7fc99406b9f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:49.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.021+0000 7fc9b9600640 1 --2- 172.21.15.42:0/1784374426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9b4075010 0x7fc9b4156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:49.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.021+0000 7fc9b9600640 1 --2- 172.21.15.42:0/1784374426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b415e1b0 0x7fc9b41566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:49.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.021+0000 7fc9b9600640 1 --2- 172.21.15.42:0/1784374426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9b4074c30 0x7fc9b415dc70 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:49.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.033+0000 7fc9b9600640 1 -- 172.21.15.42:0/1784374426 >> 172.21.15.42:0/1784374426 conn(0x7fc9b40d07c0 msgr2=0x7fc9b40ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:49.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.033+0000 7fc9b9600640 1 -- 172.21.15.42:0/1784374426 shutdown_connections 2024-05-28T14:07:49.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.033+0000 7fc9b9600640 1 -- 172.21.15.42:0/1784374426 wait complete. 2024-05-28T14:07:49.026 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:07:49.035 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:07:49.113 INFO:tasks.cephfs.test_volumes:pevs - {'346e2288-a00b-4e81-8626-13f7a50e5c02': {'message': '12 ongoing+pending clones; Avg progress is 18.924% (32s)\n [=====.......................] (remaining: 2m)', 'progress': 0.1892399936914444, 'add_to_ceph_s': True}, 'd3383476-f722-479f-9c40-e404f8ef3423': {'message': '4 ongoing clones; Avg progress is 3.971% (38s)\n [=...........................] (remaining: 15m)', 'progress': 0.03971000015735626, 'add_to_ceph_s': True}} 2024-05-28T14:07:49.113 INFO:tasks.cephfs.test_volumes:new_onpen_p is less than onpen_p. 2024-05-28T14:07:49.113 INFO:tasks.cephfs.test_volumes:new_onpen_p = 0.1892399936914444; onpen_p = 0.2859399914741516 2024-05-28T14:07:49.113 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone8111 2024-05-28T14:07:49.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.605+0000 7f8981000640 1 Processor -- start 2024-05-28T14:07:49.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.609+0000 7f8981000640 1 -- start start 2024-05-28T14:07:49.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.617+0000 7f8981000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f897c15ba70 0x7f897c06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:49.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.617+0000 7f8979400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f897c15ba70 0x7f897c06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:49.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.617+0000 7f8979400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f897c15ba70 0x7f897c06f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44918/0 (socket says 172.21.15.42:44918) 2024-05-28T14:07:49.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.621+0000 7f8981000640 1 -- 172.21.15.42:0/387912602 --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897c072710 con 0x7f897c06f800 2024-05-28T14:07:49.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.621+0000 7f8981000640 1 -- 172.21.15.42:0/387912602 --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897c15e330 con 0x7f897c154dd0 2024-05-28T14:07:49.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.621+0000 7f8981000640 1 -- 172.21.15.42:0/387912602 --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f897c0724f0 con 0x7f897c15ba70 2024-05-28T14:07:49.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.617+0000 7f8979e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f897c154dd0 0x7f897c071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:49488/0 (socket says 172.21.15.42:49488) 2024-05-28T14:07:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.617+0000 7f8979400640 1 -- 172.21.15.42:0/387912602 learned_addr learned my addr 172.21.15.42:0/387912602 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.621+0000 7f8979400640 1 -- 172.21.15.42:0/387912602 >> v1:172.21.15.63:6789/0 conn(0x7f897c154dd0 legacy=0x7f897c071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.621+0000 7f8979400640 1 -- 172.21.15.42:0/387912602 >> v1:172.21.15.42:6789/0 conn(0x7f897c06f800 legacy=0x7f897c0ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.621+0000 7f8979400640 1 -- 172.21.15.42:0/387912602 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c15e610 con 0x7f897c15ba70 2024-05-28T14:07:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.637+0000 7f8979400640 1 --2- 172.21.15.42:0/387912602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c15ba70 0x7f897c06f2c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f896800b210 tx=0x7f896801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ddf5d14ef60ee7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:49.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.641+0000 7f8978a00640 1 -- 172.21.15.42:0/387912602 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89680a5d50 con 0x7f897c15ba70 2024-05-28T14:07:49.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.641+0000 7f8978a00640 1 -- 172.21.15.42:0/387912602 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89680078d0 con 0x7f897c15ba70 2024-05-28T14:07:49.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.641+0000 7f8978a00640 1 -- 172.21.15.42:0/387912602 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968007ce0 con 0x7f897c15ba70 2024-05-28T14:07:49.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.649+0000 7f8981000640 1 -- 172.21.15.42:0/387912602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c15ba70 msgr2=0x7f897c06f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:49.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.649+0000 7f8981000640 1 --2- 172.21.15.42:0/387912602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c15ba70 0x7f897c06f2c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f896800b210 tx=0x7f896801cc80 comp rx=0 tx=0).stop 2024-05-28T14:07:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.649+0000 7f8981000640 1 -- 172.21.15.42:0/387912602 shutdown_connections 2024-05-28T14:07:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.649+0000 7f8981000640 1 --2- 172.21.15.42:0/387912602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c15ba70 0x7f897c06f2c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.649+0000 7f8981000640 1 -- 172.21.15.42:0/387912602 >> 172.21.15.42:0/387912602 conn(0x7f897c0b21f0 msgr2=0x7f897c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.649+0000 7f8981000640 1 -- 172.21.15.42:0/387912602 shutdown_connections 2024-05-28T14:07:49.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.653+0000 7f8981000640 1 -- 172.21.15.42:0/387912602 wait complete. 2024-05-28T14:07:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.661+0000 7f8981000640 1 Processor -- start 2024-05-28T14:07:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.665+0000 7f8981000640 1 -- start start 2024-05-28T14:07:49.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.673+0000 7f8981000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f897c06f800 0x7f897c203e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:49.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.681+0000 7f8981000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f897c154dd0 0x7f897c0ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:49.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.681+0000 7f8981000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c15ba70 0x7f897c0aa160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:49.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.681+0000 7f8979e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f897c06f800 0x7f897c203e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:49.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.681+0000 7f8979e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f897c06f800 0x7f897c203e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:60726/0 (socket says 172.21.15.42:60726) 2024-05-28T14:07:49.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.681+0000 7f897a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c15ba70 0x7f897c0aa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.681+0000 7f897a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c15ba70 0x7f897c0aa160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44934/0 (socket says 172.21.15.42:44934) 2024-05-28T14:07:49.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.689+0000 7f8979400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f897c154dd0 0x7f897c0ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:49.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.689+0000 7f8979400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f897c154dd0 0x7f897c0ac150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:47102/0 (socket says 172.21.15.42:47102) 2024-05-28T14:07:49.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.697+0000 7f8981000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f897c15f0f0 con 0x7f897c15ba70 2024-05-28T14:07:49.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.697+0000 7f8981000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f897c06fc70 con 0x7f897c06f800 2024-05-28T14:07:49.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.697+0000 7f8981000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f897c1587b0 con 0x7f897c154dd0 2024-05-28T14:07:49.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.697+0000 7f8979e00640 1 -- 172.21.15.42:0/1706937059 learned_addr learned my addr 172.21.15.42:0/1706937059 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:49.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.697+0000 7f8979e00640 1 -- 172.21.15.42:0/1706937059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f897c154dd0 msgr2=0x7f897c0ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:49.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.697+0000 7f8979e00640 1 --2- 172.21.15.42:0/1706937059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f897c154dd0 0x7f897c0ac150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:49.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.697+0000 7f8979e00640 1 -- 172.21.15.42:0/1706937059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c15ba70 msgr2=0x7f897c0aa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:49.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.697+0000 7f8979e00640 1 --2- 172.21.15.42:0/1706937059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c15ba70 0x7f897c0aa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:49.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.697+0000 7f8979e00640 1 -- 172.21.15.42:0/1706937059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c158a00 con 0x7f897c06f800 2024-05-28T14:07:49.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.701+0000 7f8979e00640 1 --2- 172.21.15.42:0/1706937059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f897c06f800 0x7f897c203e40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f897400c7c0 tx=0x7f897400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:49.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.701+0000 7f8979400640 1 --2- 172.21.15.42:0/1706937059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f897c154dd0 0x7f897c0ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-28T14:07:49.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.705+0000 7f8972000640 1 -- 172.21.15.42:0/1706937059 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974017070 con 0x7f897c06f800 2024-05-28T14:07:49.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.705+0000 7f8972000640 1 -- 172.21.15.42:0/1706937059 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f897400a640 con 0x7f897c06f800 2024-05-28T14:07:49.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.705+0000 7f8972000640 1 -- 172.21.15.42:0/1706937059 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897400a9f0 con 0x7f897c06f800 2024-05-28T14:07:49.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.709+0000 7f8981000640 1 -- 172.21.15.42:0/1706937059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897c158ce0 con 0x7f897c06f800 2024-05-28T14:07:49.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.709+0000 7f8981000640 1 -- 172.21.15.42:0/1706937059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897c0ac690 con 0x7f897c06f800 2024-05-28T14:07:49.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.717+0000 7f8981000640 1 -- 172.21.15.42:0/1706937059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897c0738b0 con 0x7f897c06f800 2024-05-28T14:07:49.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.721+0000 7f8972000640 1 -- 172.21.15.42:0/1706937059 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f897401a070 con 0x7f897c06f800 2024-05-28T14:07:49.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.721+0000 7f8972000640 1 --2- 172.21.15.42:0/1706937059 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f895c069530 0x7f895c06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:49.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.721+0000 7f8972000640 1 -- 172.21.15.42:0/1706937059 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f8974084e70 con 0x7f897c06f800 2024-05-28T14:07:49.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.737+0000 7f8972000640 1 -- 172.21.15.42:0/1706937059 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f89740564f0 con 0x7f897c06f800 2024-05-28T14:07:49.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.797+0000 7f8979400640 1 --2- 172.21.15.42:0/1706937059 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f895c069530 0x7f895c06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:49.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:49.805+0000 7f8979400640 1 --2- 172.21.15.42:0/1706937059 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f895c069530 0x7f895c06b9f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f89680ae000 tx=0x7f8968008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:50.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:50.237+0000 7f8981000640 1 -- 172.21.15.42:0/1706937059 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone8111", "target": ["mon-mgr", ""]}) -- 0x7f897c04fdd0 con 0x7f895c069530 2024-05-28T14:07:50.364 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:50.381+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:50.364 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:50.381+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:50.364 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:50.381+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:50.364 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:50.381+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:50.364 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:50.381+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:07:50.975 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:50.989+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:07:50.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:50.989+0000 7f8972000640 1 -- 172.21.15.42:0/1706937059 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f897c04fdd0 con 0x7f895c069530 2024-05-28T14:07:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:07:51.037 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:07:51.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone8112 2024-05-28T14:07:51.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.501+0000 7f603ec00640 1 Processor -- start 2024-05-28T14:07:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.501+0000 7f603ec00640 1 -- start start 2024-05-28T14:07:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.505+0000 7f603ec00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f60380cfe50 0x7f603806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.505+0000 7f603ec00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f603815b850 con 0x7f60380724f0 2024-05-28T14:07:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.505+0000 7f603ec00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f603815bb30 con 0x7f603815add0 2024-05-28T14:07:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.505+0000 7f603ec00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f603806df20 con 0x7f60380cfe50 2024-05-28T14:07:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.509+0000 7f6037400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f603815add0 0x7f6038071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:49504/0 (socket says 172.21.15.42:49504) 2024-05-28T14:07:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.509+0000 7f6037400640 1 -- 172.21.15.42:0/2190355578 learned_addr learned my addr 172.21.15.42:0/2190355578 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:07:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.509+0000 7f6037e00640 1 --2- 172.21.15.42:0/2190355578 >> v2:172.21.15.42:3300/0 conn(0x7f60380cfe50 0x7f603806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.509+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1006688228 0 0) 0x7f603815bb30 con 0x7f603815add0 2024-05-28T14:07:51.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.509+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f601c003530 con 0x7f603815add0 2024-05-28T14:07:51.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.513+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1015213261 0 0) 0x7f603815b850 con 0x7f60380724f0 2024-05-28T14:07:51.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.513+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f603815bb30 con 0x7f60380724f0 2024-05-28T14:07:51.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.513+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1063668597 0 0) 0x7f601c003530 con 0x7f603815add0 2024-05-28T14:07:51.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.513+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 >> v2:172.21.15.42:3300/0 conn(0x7f60380cfe50 msgr2=0x7f603806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:51.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.513+0000 7f6036a00640 1 --2- 172.21.15.42:0/2190355578 >> v2:172.21.15.42:3300/0 conn(0x7f60380cfe50 0x7f603806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:51.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.513+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 >> v1:172.21.15.42:6789/0 conn(0x7f60380724f0 legacy=0x7f60380b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:51.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.513+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603806e200 con 0x7f603815add0 2024-05-28T14:07:51.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.513+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f6024001530 con 0x7f603815add0 2024-05-28T14:07:51.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.517+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 >> v1:172.21.15.63:6789/0 conn(0x7f603815add0 legacy=0x7f6038071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:51.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.517+0000 7f6036a00640 1 --2- 172.21.15.42:0/2190355578 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f601c0037b0 0x7f601c003b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:51.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.517+0000 7f6036a00640 1 --2- 172.21.15.42:0/2190355578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f601c004190 0x7f601c00b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:51.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.517+0000 7f6036a00640 1 --2- 172.21.15.42:0/2190355578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f601c004e40 0x7f601c0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.517+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f603806df20 con 0x7f601c004190 2024-05-28T14:07:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.517+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f601c005bb0 con 0x7f601c004e40 2024-05-28T14:07:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.517+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f601c005db0 con 0x7f601c0037b0 2024-05-28T14:07:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.517+0000 7f6037e00640 1 --2- 172.21.15.42:0/2190355578 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f601c0037b0 0x7f601c003b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.517+0000 7f603cc00640 1 --2- 172.21.15.42:0/2190355578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f601c004e40 0x7f601c0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.517+0000 7f6037e00640 1 -- 172.21.15.42:0/2190355578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f601c004e40 msgr2=0x7f601c0052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.517+0000 7f6037e00640 1 --2- 172.21.15.42:0/2190355578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f601c004e40 0x7f601c0052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.517+0000 7f6037e00640 1 -- 172.21.15.42:0/2190355578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f601c004190 msgr2=0x7f601c00b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.517+0000 7f6037e00640 1 --2- 172.21.15.42:0/2190355578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f601c004190 0x7f601c00b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.517+0000 7f6037e00640 1 -- 172.21.15.42:0/2190355578 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c003530 con 0x7f601c0037b0 2024-05-28T14:07:51.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.525+0000 7f603cc00640 1 -- 172.21.15.42:0/2190355578 reap_dead start 2024-05-28T14:07:51.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.541+0000 7f6037e00640 1 --2- 172.21.15.42:0/2190355578 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f601c0037b0 0x7f601c003b90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f602800b890 tx=0x7f602802d590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab9b9ba1bba15e66 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:51.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.541+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280c3070 con 0x7f601c0037b0 2024-05-28T14:07:51.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.541+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60280be640 con 0x7f601c0037b0 2024-05-28T14:07:51.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.541+0000 7f6036a00640 1 -- 172.21.15.42:0/2190355578 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280c3070 con 0x7f601c0037b0 2024-05-28T14:07:51.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.541+0000 7f603ec00640 1 -- 172.21.15.42:0/2190355578 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f601c0037b0 msgr2=0x7f601c003b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:51.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.541+0000 7f603ec00640 1 --2- 172.21.15.42:0/2190355578 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f601c0037b0 0x7f601c003b90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f602800b890 tx=0x7f602802d590 comp rx=0 tx=0).stop 2024-05-28T14:07:51.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.545+0000 7f603ec00640 1 -- 172.21.15.42:0/2190355578 shutdown_connections 2024-05-28T14:07:51.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.545+0000 7f603ec00640 1 --2- 172.21.15.42:0/2190355578 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f601c0037b0 0x7f601c003b90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:51.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.545+0000 7f603ec00640 1 -- 172.21.15.42:0/2190355578 >> 172.21.15.42:0/2190355578 conn(0x7f60380b21f0 msgr2=0x7f60380b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:51.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.561+0000 7f603ec00640 1 -- 172.21.15.42:0/2190355578 shutdown_connections 2024-05-28T14:07:51.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.561+0000 7f603ec00640 1 -- 172.21.15.42:0/2190355578 wait complete. 2024-05-28T14:07:51.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.573+0000 7f603ec00640 1 Processor -- start 2024-05-28T14:07:51.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.573+0000 7f603ec00640 1 -- start start 2024-05-28T14:07:51.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.573+0000 7f603ec00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f601c0037b0 0x7f6038153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:51.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.573+0000 7f603ec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60380ac270 0x7f60380a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:51.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.573+0000 7f603ec00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60380a99f0 0x7f60380b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:51.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.577+0000 7f6037400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60380ac270 0x7f60380a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:51.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.577+0000 7f6037e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f601c0037b0 0x7f6038153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:51.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.577+0000 7f6037e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f601c0037b0 0x7f6038153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:47138/0 (socket says 172.21.15.42:47138) 2024-05-28T14:07:51.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.577+0000 7f6037400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60380ac270 0x7f60380a94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44970/0 (socket says 172.21.15.42:44970) 2024-05-28T14:07:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.577+0000 7f603ec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6038070050 con 0x7f60380ac270 2024-05-28T14:07:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.577+0000 7f603ec00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f603806fed0 con 0x7f60380a99f0 2024-05-28T14:07:51.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.577+0000 7f603ec00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f603815b250 con 0x7f601c0037b0 2024-05-28T14:07:51.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.577+0000 7f6037e00640 1 -- 172.21.15.42:0/4181453416 learned_addr learned my addr 172.21.15.42:0/4181453416 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:51.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.577+0000 7f603cc00640 1 --2- 172.21.15.42:0/4181453416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60380a99f0 0x7f60380b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:51.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.577+0000 7f6037e00640 1 -- 172.21.15.42:0/4181453416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60380a99f0 msgr2=0x7f60380b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:51.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.577+0000 7f6037e00640 1 --2- 172.21.15.42:0/4181453416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60380a99f0 0x7f60380b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:51.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.577+0000 7f6037e00640 1 -- 172.21.15.42:0/4181453416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60380ac270 msgr2=0x7f60380a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:51.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.577+0000 7f6037e00640 1 --2- 172.21.15.42:0/4181453416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60380ac270 0x7f60380a94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:51.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.577+0000 7f6037e00640 1 -- 172.21.15.42:0/4181453416 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60380a9f50 con 0x7f601c0037b0 2024-05-28T14:07:51.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.581+0000 7f6037e00640 1 --2- 172.21.15.42:0/4181453416 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f601c0037b0 0x7f6038153fe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f602800b890 tx=0x7f601c004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:51.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.581+0000 7f602fe00640 1 -- 172.21.15.42:0/4181453416 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280cd070 con 0x7f601c0037b0 2024-05-28T14:07:51.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.581+0000 7f602fe00640 1 -- 172.21.15.42:0/4181453416 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6028004020 con 0x7f601c0037b0 2024-05-28T14:07:51.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.581+0000 7f602fe00640 1 -- 172.21.15.42:0/4181453416 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280c3070 con 0x7f601c0037b0 2024-05-28T14:07:51.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.581+0000 7f603ec00640 1 -- 172.21.15.42:0/4181453416 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60381552c0 con 0x7f601c0037b0 2024-05-28T14:07:51.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.581+0000 7f603ec00640 1 -- 172.21.15.42:0/4181453416 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6038155880 con 0x7f601c0037b0 2024-05-28T14:07:51.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.593+0000 7f603ec00640 1 -- 172.21.15.42:0/4181453416 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60100052a0 con 0x7f601c0037b0 2024-05-28T14:07:51.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.597+0000 7f602fe00640 1 -- 172.21.15.42:0/4181453416 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 27) ==== 89517+0+0 (secure 0 0 0) 0x7f60280bd7b0 con 0x7f601c0037b0 2024-05-28T14:07:51.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.605+0000 7f602fe00640 1 --2- 172.21.15.42:0/4181453416 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6018069560 0x7f601806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:51.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.605+0000 7f602fe00640 1 -- 172.21.15.42:0/4181453416 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f6028131110 con 0x7f601c0037b0 2024-05-28T14:07:51.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.613+0000 7f602fe00640 1 -- 172.21.15.42:0/4181453416 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f60280272a0 con 0x7f601c0037b0 2024-05-28T14:07:51.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.613+0000 7f6037400640 1 --2- 172.21.15.42:0/4181453416 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6018069560 0x7f601806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:51.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.757+0000 7f6037400640 1 --2- 172.21.15.42:0/4181453416 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6018069560 0x7f601806ba20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f603815a910 tx=0x7f6024001620 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:51.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:51.793+0000 7f602fe00640 1 -- 172.21.15.42:0/4181453416 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mgrmap(e 28) ==== 89568+0+0 (secure 0 0 0) 0x7f60280ff140 con 0x7f601c0037b0 2024-05-28T14:07:52.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:52.113+0000 7f603ec00640 1 -- 172.21.15.42:0/4181453416 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone8112", "target": ["mon-mgr", ""]}) -- 0x7f6010000bd0 con 0x7f6018069560 2024-05-28T14:07:52.711 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:52.725+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:07:52.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:52.729+0000 7f602fe00640 1 -- 172.21.15.42:0/4181453416 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f6010000bd0 con 0x7f6018069560 2024-05-28T14:07:52.715 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:07:52.746 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:07:52.747 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone8113 2024-05-28T14:07:53.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.241+0000 7f89b0200640 1 Processor -- start 2024-05-28T14:07:53.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.241+0000 7f89b0200640 1 -- start start 2024-05-28T14:07:53.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.245+0000 7f89b0200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f89a815ba70 0x7f89a806f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:53.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.245+0000 7f89b0200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f89a806f800 0x7f89a80ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:53.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.245+0000 7f89b0200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89a8072710 con 0x7f89a8154dd0 2024-05-28T14:07:53.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.245+0000 7f89b0200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f89a80724f0 con 0x7f89a806f800 2024-05-28T14:07:53.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.245+0000 7f89b0200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f89a806fc70 con 0x7f89a815ba70 2024-05-28T14:07:53.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.245+0000 7f89ae200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f89a806f800 0x7f89a80ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:53.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.245+0000 7f89ae200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f89a806f800 0x7f89a80ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44978/0 (socket says 172.21.15.42:44978) 2024-05-28T14:07:53.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.245+0000 7f89ae200640 1 -- 172.21.15.42:0/2352411014 learned_addr learned my addr 172.21.15.42:0/2352411014 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:53.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.245+0000 7f89ace00640 1 --2- 172.21.15.42:0/2352411014 >> v2:172.21.15.63:3300/0 conn(0x7f89a815ba70 0x7f89a806f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:53.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.245+0000 7f89ae200640 1 -- 172.21.15.42:0/2352411014 >> v2:172.21.15.63:3300/0 conn(0x7f89a815ba70 msgr2=0x7f89a806f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:53.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.245+0000 7f89ae200640 1 --2- 172.21.15.42:0/2352411014 >> v2:172.21.15.63:3300/0 conn(0x7f89a815ba70 0x7f89a806f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:53.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.245+0000 7f89ae200640 1 -- 172.21.15.42:0/2352411014 >> v1:172.21.15.63:6789/0 conn(0x7f89a8154dd0 legacy=0x7f89a8071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:53.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.245+0000 7f89ae200640 1 -- 172.21.15.42:0/2352411014 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a815e330 con 0x7f89a806f800 2024-05-28T14:07:53.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.245+0000 7f89a7e00640 1 -- 172.21.15.42:0/2352411014 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2205854587 0 0) 0x7f89a8072710 con 0x7f89a8154dd0 2024-05-28T14:07:53.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.253+0000 7f89ae200640 1 --2- 172.21.15.42:0/2352411014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a806f800 0x7f89a80ce590 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f899c009870 tx=0x7f899c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948fc63f0403ef24 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:53.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.253+0000 7f89a7e00640 1 -- 172.21.15.42:0/2352411014 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c0b1070 con 0x7f89a806f800 2024-05-28T14:07:53.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.253+0000 7f89a7e00640 1 -- 172.21.15.42:0/2352411014 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f899c0a8050 con 0x7f89a806f800 2024-05-28T14:07:53.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.253+0000 7f89a7e00640 1 -- 172.21.15.42:0/2352411014 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c0ac860 con 0x7f89a806f800 2024-05-28T14:07:53.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.253+0000 7f89b0200640 1 -- 172.21.15.42:0/2352411014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a806f800 msgr2=0x7f89a80ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:53.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.253+0000 7f89b0200640 1 --2- 172.21.15.42:0/2352411014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a806f800 0x7f89a80ce590 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f899c009870 tx=0x7f899c01c510 comp rx=0 tx=0).stop 2024-05-28T14:07:53.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.269+0000 7f89b0200640 1 -- 172.21.15.42:0/2352411014 shutdown_connections 2024-05-28T14:07:53.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.269+0000 7f89b0200640 1 --2- 172.21.15.42:0/2352411014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a806f800 0x7f89a80ce590 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:53.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.269+0000 7f89b0200640 1 --2- 172.21.15.42:0/2352411014 >> v2:172.21.15.63:3300/0 conn(0x7f89a815ba70 0x7f89a806f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:53.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.269+0000 7f89b0200640 1 -- 172.21.15.42:0/2352411014 >> 172.21.15.42:0/2352411014 conn(0x7f89a80b21f0 msgr2=0x7f89a80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:53.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.277+0000 7f89b0200640 1 -- 172.21.15.42:0/2352411014 shutdown_connections 2024-05-28T14:07:53.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.277+0000 7f89b0200640 1 -- 172.21.15.42:0/2352411014 wait complete. 2024-05-28T14:07:53.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.289+0000 7f89b0200640 1 Processor -- start 2024-05-28T14:07:53.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.289+0000 7f89b0200640 1 -- start start 2024-05-28T14:07:53.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.289+0000 7f89b0200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a806f800 0x7f89a8207260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:53.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.289+0000 7f89b0200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a8154dd0 0x7f89a80b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:53.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.289+0000 7f89b0200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a815ba70 0x7f89a80b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:53.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.289+0000 7f89b0200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f89a815f0f0 con 0x7f89a806f800 2024-05-28T14:07:53.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.289+0000 7f89b0200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f89a81551b0 con 0x7f89a8154dd0 2024-05-28T14:07:53.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.289+0000 7f89b0200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f89a8130930 con 0x7f89a815ba70 2024-05-28T14:07:53.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.293+0000 7f89ad800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a806f800 0x7f89a8207260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:53.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.293+0000 7f89ad800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a806f800 0x7f89a8207260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44994/0 (socket says 172.21.15.42:44994) 2024-05-28T14:07:53.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.293+0000 7f89ad800640 1 -- 172.21.15.42:0/4084081945 learned_addr learned my addr 172.21.15.42:0/4084081945 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:53.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.297+0000 7f89ad800640 1 -- 172.21.15.42:0/4084081945 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a815ba70 msgr2=0x7f89a80b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:53.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.297+0000 7f89ad800640 1 --2- 172.21.15.42:0/4084081945 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a815ba70 0x7f89a80b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:53.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.297+0000 7f89ad800640 1 -- 172.21.15.42:0/4084081945 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a8154dd0 msgr2=0x7f89a80b3a80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T14:07:53.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.297+0000 7f89ad800640 1 --2- 172.21.15.42:0/4084081945 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a8154dd0 0x7f89a80b3a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:53.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.297+0000 7f89ad800640 1 -- 172.21.15.42:0/4084081945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a8168990 con 0x7f89a806f800 2024-05-28T14:07:53.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.301+0000 7f89ad800640 1 --2- 172.21.15.42:0/4084081945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89a806f800 0x7f89a8207260 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f89a000c810 tx=0x7f89a000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:53.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.301+0000 7f89a5600640 1 -- 172.21.15.42:0/4084081945 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0003db0 con 0x7f89a806f800 2024-05-28T14:07:53.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.305+0000 7f89a5600640 1 -- 172.21.15.42:0/4084081945 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a0005070 con 0x7f89a806f800 2024-05-28T14:07:53.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.305+0000 7f89a5600640 1 -- 172.21.15.42:0/4084081945 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0005460 con 0x7f89a806f800 2024-05-28T14:07:53.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.305+0000 7f89b0200640 1 -- 172.21.15.42:0/4084081945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a80b4850 con 0x7f89a806f800 2024-05-28T14:07:53.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.305+0000 7f89b0200640 1 -- 172.21.15.42:0/4084081945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a80b4500 con 0x7f89a806f800 2024-05-28T14:07:53.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.313+0000 7f89b0200640 1 -- 172.21.15.42:0/4084081945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a80738b0 con 0x7f89a806f800 2024-05-28T14:07:53.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.321+0000 7f89a5600640 1 -- 172.21.15.42:0/4084081945 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 28) ==== 89568+0+0 (secure 0 0 0) 0x7f89a0005600 con 0x7f89a806f800 2024-05-28T14:07:53.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.321+0000 7f89a5600640 1 --2- 172.21.15.42:0/4084081945 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89880694f0 0x7f898806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:53.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.321+0000 7f89a5600640 1 -- 172.21.15.42:0/4084081945 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f89a00858b0 con 0x7f89a806f800 2024-05-28T14:07:53.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.333+0000 7f89a5600640 1 -- 172.21.15.42:0/4084081945 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f89a0057570 con 0x7f89a806f800 2024-05-28T14:07:53.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.365+0000 7f89ace00640 1 --2- 172.21.15.42:0/4084081945 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89880694f0 0x7f898806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:53.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:53.381+0000 7f89ace00640 1 --2- 172.21.15.42:0/4084081945 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f89880694f0 0x7f898806b9b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f89a8072300 tx=0x7f8994008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:53.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:54.005+0000 7f89b0200640 1 -- 172.21.15.42:0/4084081945 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone8113", "target": ["mon-mgr", ""]}) -- 0x7f89a815f270 con 0x7f89880694f0 2024-05-28T14:07:54.480 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:54.497+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:07:54.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:54.497+0000 7f89a5600640 1 -- 172.21.15.42:0/4084081945 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f89a815f270 con 0x7f89880694f0 2024-05-28T14:07:54.482 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:07:54.516 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:07:54.517 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone8114 2024-05-28T14:07:55.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.013+0000 7f710f200640 1 Processor -- start 2024-05-28T14:07:55.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.017+0000 7f710f200640 1 -- start start 2024-05-28T14:07:55.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.017+0000 7f710f200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7108162b80 0x7f71080b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:55.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.017+0000 7f710f200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f710806dd10 0x7f710806e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:55.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.017+0000 7f710f200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7108071de0 0x7f7108072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:55.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.017+0000 7f710f200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f71080b3d40 con 0x7f710806dd10 2024-05-28T14:07:55.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.017+0000 7f710f200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f71080ce1c0 con 0x7f7108071de0 2024-05-28T14:07:55.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.017+0000 7f710f200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f710806e760 con 0x7f7108162b80 2024-05-28T14:07:55.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.017+0000 7f710d200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7108071de0 0x7f7108072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:55.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.017+0000 7f710d200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7108071de0 0x7f7108072270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:60796/0 (socket says 172.21.15.42:60796) 2024-05-28T14:07:55.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.017+0000 7f710d200640 1 -- 172.21.15.42:0/723410384 learned_addr learned my addr 172.21.15.42:0/723410384 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:55.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.021+0000 7f7107e00640 1 --2- 172.21.15.42:0/723410384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7108162b80 0x7f71080b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:55.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.021+0000 7f7107400640 1 --2- 172.21.15.42:0/723410384 >> v2:172.21.15.42:3300/0 conn(0x7f710806dd10 0x7f710806e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.021+0000 7f7107e00640 1 -- 172.21.15.42:0/723410384 >> v2:172.21.15.63:3300/0 conn(0x7f7108071de0 msgr2=0x7f7108072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.025+0000 7f7107e00640 1 --2- 172.21.15.42:0/723410384 >> v2:172.21.15.63:3300/0 conn(0x7f7108071de0 0x7f7108072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.025+0000 7f7107e00640 1 -- 172.21.15.42:0/723410384 >> v2:172.21.15.42:3300/0 conn(0x7f710806dd10 msgr2=0x7f710806e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.025+0000 7f7107e00640 1 --2- 172.21.15.42:0/723410384 >> v2:172.21.15.42:3300/0 conn(0x7f710806dd10 0x7f710806e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.025+0000 7f7107e00640 1 -- 172.21.15.42:0/723410384 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710806ea40 con 0x7f7108162b80 2024-05-28T14:07:55.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.025+0000 7f7107400640 1 --2- 172.21.15.42:0/723410384 >> v2:172.21.15.42:3300/0 conn(0x7f710806dd10 0x7f710806e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:07:55.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.025+0000 7f710d200640 1 --2- 172.21.15.42:0/723410384 >> v2:172.21.15.63:3300/0 conn(0x7f7108071de0 0x7f7108072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:07:55.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.037+0000 7f7107e00640 1 --2- 172.21.15.42:0/723410384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7108162b80 0x7f71080b3800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7108072ac0 tx=0x7f710001c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c6ea025f3ee6ddc server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:55.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.041+0000 7f7106a00640 1 -- 172.21.15.42:0/723410384 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71000b2070 con 0x7f7108162b80 2024-05-28T14:07:55.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.041+0000 7f7106a00640 1 -- 172.21.15.42:0/723410384 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71000a57b0 con 0x7f7108162b80 2024-05-28T14:07:55.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.041+0000 7f7106a00640 1 -- 172.21.15.42:0/723410384 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71000a5b80 con 0x7f7108162b80 2024-05-28T14:07:55.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.045+0000 7f710f200640 1 -- 172.21.15.42:0/723410384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7108162b80 msgr2=0x7f71080b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:55.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.045+0000 7f710f200640 1 --2- 172.21.15.42:0/723410384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7108162b80 0x7f71080b3800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7108072ac0 tx=0x7f710001c930 comp rx=0 tx=0).stop 2024-05-28T14:07:55.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.045+0000 7f710f200640 1 -- 172.21.15.42:0/723410384 shutdown_connections 2024-05-28T14:07:55.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.045+0000 7f710f200640 1 --2- 172.21.15.42:0/723410384 >> v2:172.21.15.63:3300/0 conn(0x7f7108071de0 0x7f7108072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:55.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.045+0000 7f710f200640 1 --2- 172.21.15.42:0/723410384 >> v2:172.21.15.42:3300/0 conn(0x7f710806dd10 0x7f710806e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:55.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.045+0000 7f710f200640 1 --2- 172.21.15.42:0/723410384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7108162b80 0x7f71080b3800 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:55.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.045+0000 7f710f200640 1 -- 172.21.15.42:0/723410384 >> 172.21.15.42:0/723410384 conn(0x7f71080b21f0 msgr2=0x7f71080b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:55.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.049+0000 7f710f200640 1 -- 172.21.15.42:0/723410384 shutdown_connections 2024-05-28T14:07:55.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.049+0000 7f710f200640 1 -- 172.21.15.42:0/723410384 wait complete. 2024-05-28T14:07:55.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.057+0000 7f710f200640 1 Processor -- start 2024-05-28T14:07:55.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.057+0000 7f710f200640 1 -- start start 2024-05-28T14:07:55.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.057+0000 7f710f200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f710806dd10 0x7f71081569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:55.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.057+0000 7f710f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7108071de0 0x7f710815d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:55.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.057+0000 7f710f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7108162b80 0x7f7108147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:55.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.057+0000 7f710f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f71081630e0 con 0x7f7108162b80 2024-05-28T14:07:55.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.057+0000 7f710f200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7108162f60 con 0x7f7108071de0 2024-05-28T14:07:55.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.057+0000 7f710f200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f71080cf470 con 0x7f710806dd10 2024-05-28T14:07:55.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.061+0000 7f7107e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f710806dd10 0x7f71081569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:55.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.061+0000 7f7107e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f710806dd10 0x7f71081569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:47170/0 (socket says 172.21.15.42:47170) 2024-05-28T14:07:55.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.061+0000 7f7107e00640 1 -- 172.21.15.42:0/149434456 learned_addr learned my addr 172.21.15.42:0/149434456 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:55.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.061+0000 7f7107400640 1 --2- 172.21.15.42:0/149434456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7108071de0 0x7f710815d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:55.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.061+0000 7f7107e00640 1 -- 172.21.15.42:0/149434456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7108071de0 msgr2=0x7f710815d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:55.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.061+0000 7f7107e00640 1 --2- 172.21.15.42:0/149434456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7108071de0 0x7f710815d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:55.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.061+0000 7f7107e00640 1 -- 172.21.15.42:0/149434456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7108162b80 msgr2=0x7f7108147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:55.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.061+0000 7f7107e00640 1 --2- 172.21.15.42:0/149434456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7108162b80 0x7f7108147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:55.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.061+0000 7f7107e00640 1 -- 172.21.15.42:0/149434456 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7108147a50 con 0x7f710806dd10 2024-05-28T14:07:55.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.065+0000 7f7107e00640 1 --2- 172.21.15.42:0/149434456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f710806dd10 0x7f71081569a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f710001a560 tx=0x7f71000196d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:55.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.065+0000 7f70ffe00640 1 -- 172.21.15.42:0/149434456 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71000bd020 con 0x7f710806dd10 2024-05-28T14:07:55.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.065+0000 7f70ffe00640 1 -- 172.21.15.42:0/149434456 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71000b3040 con 0x7f710806dd10 2024-05-28T14:07:55.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.065+0000 7f70ffe00640 1 -- 172.21.15.42:0/149434456 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71000b2070 con 0x7f710806dd10 2024-05-28T14:07:55.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.073+0000 7f710f200640 1 -- 172.21.15.42:0/149434456 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710815c680 con 0x7f710806dd10 2024-05-28T14:07:55.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.073+0000 7f710f200640 1 -- 172.21.15.42:0/149434456 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f710815cc40 con 0x7f710806dd10 2024-05-28T14:07:55.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.077+0000 7f70ffe00640 1 -- 172.21.15.42:0/149434456 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 28) ==== 89568+0+0 (secure 0 0 0) 0x7f71000c0020 con 0x7f710806dd10 2024-05-28T14:07:55.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.081+0000 7f70ffe00640 1 --2- 172.21.15.42:0/149434456 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70e8069540 0x7f70e806ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:55.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.081+0000 7f70ffe00640 1 -- 172.21.15.42:0/149434456 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f7100120850 con 0x7f710806dd10 2024-05-28T14:07:55.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.081+0000 7f710f200640 1 -- 172.21.15.42:0/149434456 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71080738b0 con 0x7f710806dd10 2024-05-28T14:07:55.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.093+0000 7f70ffe00640 1 -- 172.21.15.42:0/149434456 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7100016520 con 0x7f710806dd10 2024-05-28T14:07:55.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.113+0000 7f7107400640 1 --2- 172.21.15.42:0/149434456 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70e8069540 0x7f70e806ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:55.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.117+0000 7f7107400640 1 --2- 172.21.15.42:0/149434456 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f70e8069540 0x7f70e806ba00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f710806e250 tx=0x7f70f8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:55.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:55.633+0000 7f710f200640 1 -- 172.21.15.42:0/149434456 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone8114", "target": ["mon-mgr", ""]}) -- 0x7f710806e280 con 0x7f70e8069540 2024-05-28T14:07:55.991 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:56.009+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:07:55.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.009+0000 7f70ffe00640 1 -- 172.21.15.42:0/149434456 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f710806e280 con 0x7f70e8069540 2024-05-28T14:07:55.993 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:07:56.020 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:07:56.021 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone8115 2024-05-28T14:07:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.473+0000 7f21e4400640 1 Processor -- start 2024-05-28T14:07:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.473+0000 7f21e4400640 1 -- start start 2024-05-28T14:07:56.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.477+0000 7f21e4400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f21dc0af650 0x7f21dc0b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:56.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.477+0000 7f21e4400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f21dc06c3d0 0x7f21dc168340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:56.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.477+0000 7f21e4400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21dc06c970 con 0x7f21dc067760 2024-05-28T14:07:56.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.477+0000 7f21e4400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f21dc06d4f0 con 0x7f21dc06c3d0 2024-05-28T14:07:56.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.477+0000 7f21e4400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f21dc06cc50 con 0x7f21dc0af650 2024-05-28T14:07:56.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.477+0000 7f21e2400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f21dc06c3d0 0x7f21dc168340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:56.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.477+0000 7f21e1000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f21dc0af650 0x7f21dc0b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:56.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.477+0000 7f21e1000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f21dc0af650 0x7f21dc0b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:60812/0 (socket says 172.21.15.42:60812) 2024-05-28T14:07:56.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.477+0000 7f21e1000640 1 -- 172.21.15.42:0/146636786 learned_addr learned my addr 172.21.15.42:0/146636786 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:56.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.481+0000 7f21e2400640 1 -- 172.21.15.42:0/146636786 >> v2:172.21.15.63:3300/0 conn(0x7f21dc0af650 msgr2=0x7f21dc0b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:56.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.481+0000 7f21e2400640 1 --2- 172.21.15.42:0/146636786 >> v2:172.21.15.63:3300/0 conn(0x7f21dc0af650 0x7f21dc0b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:56.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.481+0000 7f21e2400640 1 -- 172.21.15.42:0/146636786 >> v1:172.21.15.63:6789/0 conn(0x7f21dc067760 legacy=0x7f21dc067b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:56.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.481+0000 7f21e2400640 1 -- 172.21.15.42:0/146636786 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc06cf30 con 0x7f21dc06c3d0 2024-05-28T14:07:56.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.489+0000 7f21e2400640 1 --2- 172.21.15.42:0/146636786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc06c3d0 0x7f21dc168340 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f21dc148680 tx=0x7f21d401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce3c75748db8035 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:56.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.489+0000 7f21dbe00640 1 -- 172.21.15.42:0/146636786 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d40b2070 con 0x7f21dc06c3d0 2024-05-28T14:07:56.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.489+0000 7f21dbe00640 1 -- 172.21.15.42:0/146636786 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d40a5910 con 0x7f21dc06c3d0 2024-05-28T14:07:56.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.489+0000 7f21dbe00640 1 -- 172.21.15.42:0/146636786 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d40a5ce0 con 0x7f21dc06c3d0 2024-05-28T14:07:56.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.493+0000 7f21e4400640 1 -- 172.21.15.42:0/146636786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc06c3d0 msgr2=0x7f21dc168340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:56.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.493+0000 7f21e4400640 1 --2- 172.21.15.42:0/146636786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc06c3d0 0x7f21dc168340 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f21dc148680 tx=0x7f21d401c870 comp rx=0 tx=0).stop 2024-05-28T14:07:56.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.493+0000 7f21e4400640 1 -- 172.21.15.42:0/146636786 shutdown_connections 2024-05-28T14:07:56.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.493+0000 7f21e4400640 1 --2- 172.21.15.42:0/146636786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc06c3d0 0x7f21dc168340 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:56.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.493+0000 7f21e4400640 1 --2- 172.21.15.42:0/146636786 >> v2:172.21.15.63:3300/0 conn(0x7f21dc0af650 0x7f21dc0b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:56.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.493+0000 7f21e4400640 1 -- 172.21.15.42:0/146636786 >> 172.21.15.42:0/146636786 conn(0x7f21dc073250 msgr2=0x7f21dc0b6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:56.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.493+0000 7f21e4400640 1 -- 172.21.15.42:0/146636786 shutdown_connections 2024-05-28T14:07:56.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.493+0000 7f21e4400640 1 -- 172.21.15.42:0/146636786 wait complete. 2024-05-28T14:07:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.497+0000 7f21e4400640 1 Processor -- start 2024-05-28T14:07:56.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.497+0000 7f21e4400640 1 -- start start 2024-05-28T14:07:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.501+0000 7f21e4400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21dc067760 0x7f21dc15a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.501+0000 7f21e4400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc06c3d0 0x7f21dc157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.501+0000 7f21e4400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21dc0af650 0x7f21dc0ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.501+0000 7f21e4400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc16b220 con 0x7f21dc06c3d0 2024-05-28T14:07:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.501+0000 7f21e4400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc16b0a0 con 0x7f21dc067760 2024-05-28T14:07:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.501+0000 7f21e4400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f21dc16b3a0 con 0x7f21dc0af650 2024-05-28T14:07:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.505+0000 7f21e1a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21dc067760 0x7f21dc15a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.505+0000 7f21e1a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21dc067760 0x7f21dc15a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:60824/0 (socket says 172.21.15.42:60824) 2024-05-28T14:07:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.505+0000 7f21e1a00640 1 -- 172.21.15.42:0/3685905003 learned_addr learned my addr 172.21.15.42:0/3685905003 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.505+0000 7f21e1a00640 1 -- 172.21.15.42:0/3685905003 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21dc0af650 msgr2=0x7f21dc0ca830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T14:07:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.505+0000 7f21e1a00640 1 --2- 172.21.15.42:0/3685905003 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21dc0af650 0x7f21dc0ca830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.505+0000 7f21e1a00640 1 -- 172.21.15.42:0/3685905003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc06c3d0 msgr2=0x7f21dc157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:56.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.505+0000 7f21e1a00640 1 --2- 172.21.15.42:0/3685905003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21dc06c3d0 0x7f21dc157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:56.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.505+0000 7f21e1a00640 1 -- 172.21.15.42:0/3685905003 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc157c60 con 0x7f21dc067760 2024-05-28T14:07:56.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.505+0000 7f21e1a00640 1 --2- 172.21.15.42:0/3685905003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21dc067760 0x7f21dc15a6a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f21d0007fd0 tx=0x7f21d0014420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:56.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.509+0000 7f21d9600640 1 -- 172.21.15.42:0/3685905003 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0014b30 con 0x7f21dc067760 2024-05-28T14:07:56.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.509+0000 7f21e4400640 1 -- 172.21.15.42:0/3685905003 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21dc0cad70 con 0x7f21dc067760 2024-05-28T14:07:56.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.509+0000 7f21e4400640 1 -- 172.21.15.42:0/3685905003 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21dc13f920 con 0x7f21dc067760 2024-05-28T14:07:56.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.509+0000 7f21d9600640 1 -- 172.21.15.42:0/3685905003 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d0014cd0 con 0x7f21dc067760 2024-05-28T14:07:56.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.509+0000 7f21d9600640 1 -- 172.21.15.42:0/3685905003 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d00052f0 con 0x7f21dc067760 2024-05-28T14:07:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.509+0000 7f21e4400640 1 -- 172.21.15.42:0/3685905003 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b80052a0 con 0x7f21dc067760 2024-05-28T14:07:56.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.513+0000 7f21d9600640 1 -- 172.21.15.42:0/3685905003 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 28) ==== 89568+0+0 (secure 0 0 0) 0x7f21d00120d0 con 0x7f21dc067760 2024-05-28T14:07:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.517+0000 7f21d9600640 1 --2- 172.21.15.42:0/3685905003 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21bc0695f0 0x7f21bc06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.521+0000 7f21d9600640 1 -- 172.21.15.42:0/3685905003 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f21d008dc30 con 0x7f21dc067760 2024-05-28T14:07:56.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.537+0000 7f21d9600640 1 -- 172.21.15.42:0/3685905003 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f21dc0cad70 con 0x7f21dc067760 2024-05-28T14:07:56.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.537+0000 7f21e1000640 1 --2- 172.21.15.42:0/3685905003 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21bc0695f0 0x7f21bc06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:56.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:56.553+0000 7f21e1000640 1 --2- 172.21.15.42:0/3685905003 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21bc0695f0 0x7f21bc06bab0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f21dc144920 tx=0x7f21c8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.129+0000 7f21e4400640 1 -- 172.21.15.42:0/3685905003 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone8115", "target": ["mon-mgr", ""]}) -- 0x7f21b8000bd0 con 0x7f21bc0695f0 2024-05-28T14:07:57.296 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:57.313+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:07:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.313+0000 7f21d9600640 1 -- 172.21.15.42:0/3685905003 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f21b8000bd0 con 0x7f21bc0695f0 2024-05-28T14:07:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:07:57.319 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:07:57.320 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone8116 2024-05-28T14:07:57.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.685+0000 7f7943a00640 1 Processor -- start 2024-05-28T14:07:57.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.689+0000 7f7943a00640 1 -- start start 2024-05-28T14:07:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.709+0000 7f7943a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c06f9d0 0x7f793c154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.709+0000 7f7943a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f793c072710 con 0x7f793c15ba70 2024-05-28T14:07:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.709+0000 7f7943a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f793c15e330 con 0x7f793c0b2d10 2024-05-28T14:07:57.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.709+0000 7f7943a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f793c0724f0 con 0x7f793c06f9d0 2024-05-28T14:07:57.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.713+0000 7f7941a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c06f9d0 0x7f793c154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:57.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.713+0000 7f7941a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c06f9d0 0x7f793c154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:55166/0 (socket says 172.21.15.42:55166) 2024-05-28T14:07:57.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.713+0000 7f7941a00640 1 -- 172.21.15.42:0/490322350 learned_addr learned my addr 172.21.15.42:0/490322350 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:57.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.713+0000 7f7941a00640 1 -- 172.21.15.42:0/490322350 >> v1:172.21.15.63:6789/0 conn(0x7f793c0b2d10 legacy=0x7f793c071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:57.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.713+0000 7f7941a00640 1 -- 172.21.15.42:0/490322350 >> v1:172.21.15.42:6789/0 conn(0x7f793c15ba70 legacy=0x7f793c06f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:57.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.713+0000 7f7941a00640 1 -- 172.21.15.42:0/490322350 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c15e610 con 0x7f793c06f9d0 2024-05-28T14:07:57.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.729+0000 7f7941a00640 1 --2- 172.21.15.42:0/490322350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c06f9d0 0x7f793c154dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f792c00bb70 tx=0x7f792c01eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4db1ea5a808f72a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:57.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.733+0000 7f793b400640 1 -- 172.21.15.42:0/490322350 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c0b4070 con 0x7f793c06f9d0 2024-05-28T14:07:57.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.733+0000 7f793b400640 1 -- 172.21.15.42:0/490322350 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f792c0a7c30 con 0x7f793c06f9d0 2024-05-28T14:07:57.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.733+0000 7f793b400640 1 -- 172.21.15.42:0/490322350 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c0af440 con 0x7f793c06f9d0 2024-05-28T14:07:57.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.737+0000 7f7943a00640 1 -- 172.21.15.42:0/490322350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c06f9d0 msgr2=0x7f793c154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:57.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.737+0000 7f7943a00640 1 --2- 172.21.15.42:0/490322350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c06f9d0 0x7f793c154dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f792c00bb70 tx=0x7f792c01eba0 comp rx=0 tx=0).stop 2024-05-28T14:07:57.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.741+0000 7f7943a00640 1 -- 172.21.15.42:0/490322350 shutdown_connections 2024-05-28T14:07:57.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.741+0000 7f7943a00640 1 --2- 172.21.15.42:0/490322350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c06f9d0 0x7f793c154dd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:57.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.741+0000 7f7943a00640 1 -- 172.21.15.42:0/490322350 >> 172.21.15.42:0/490322350 conn(0x7f793c0b21f0 msgr2=0x7f793c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:57.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.741+0000 7f7943a00640 1 -- 172.21.15.42:0/490322350 shutdown_connections 2024-05-28T14:07:57.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.741+0000 7f7943a00640 1 -- 172.21.15.42:0/490322350 wait complete. 2024-05-28T14:07:57.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.745+0000 7f7943a00640 1 Processor -- start 2024-05-28T14:07:57.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.745+0000 7f7943a00640 1 -- start start 2024-05-28T14:07:57.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.749+0000 7f7943a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f793c06f9d0 0x7f793c153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:57.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.749+0000 7f7941000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f793c06f9d0 0x7f793c153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:57.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.749+0000 7f7941000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f793c06f9d0 0x7f793c153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59338/0 (socket says 172.21.15.42:59338) 2024-05-28T14:07:57.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.749+0000 7f7941000640 1 -- 172.21.15.42:0/967930910 learned_addr learned my addr 172.21.15.42:0/967930910 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:57.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.749+0000 7f7943a00640 1 --2- 172.21.15.42:0/967930910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c0b2d10 0x7f793c158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:57.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.749+0000 7f7943a00640 1 --2- 172.21.15.42:0/967930910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f793c15ba70 0x7f793c15ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:57.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.749+0000 7f793be00640 1 --2- 172.21.15.42:0/967930910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c0b2d10 0x7f793c158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:57.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.749+0000 7f7943a00640 1 -- 172.21.15.42:0/967930910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f793c0cef20 con 0x7f793c06f9d0 2024-05-28T14:07:57.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.753+0000 7f7943a00640 1 -- 172.21.15.42:0/967930910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f793c0ceda0 con 0x7f793c15ba70 2024-05-28T14:07:57.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.753+0000 7f7943a00640 1 -- 172.21.15.42:0/967930910 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f793c0cf0a0 con 0x7f793c0b2d10 2024-05-28T14:07:57.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.753+0000 7f7941000640 1 -- 172.21.15.42:0/967930910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c0b2d10 msgr2=0x7f793c158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:57.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.753+0000 7f7941000640 1 --2- 172.21.15.42:0/967930910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c0b2d10 0x7f793c158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:57.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.753+0000 7f7941000640 1 -- 172.21.15.42:0/967930910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f793c15ba70 msgr2=0x7f793c15ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:57.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.753+0000 7f7941000640 1 --2- 172.21.15.42:0/967930910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f793c15ba70 0x7f793c15ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:57.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.753+0000 7f7941000640 1 -- 172.21.15.42:0/967930910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c0b54f0 con 0x7f793c06f9d0 2024-05-28T14:07:57.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.757+0000 7f7941000640 1 --2- 172.21.15.42:0/967930910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f793c06f9d0 0x7f793c153540 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f792800c450 tx=0x7f792800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:57.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.757+0000 7f7938c00640 1 -- 172.21.15.42:0/967930910 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928011790 con 0x7f793c06f9d0 2024-05-28T14:07:57.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.757+0000 7f7938c00640 1 -- 172.21.15.42:0/967930910 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7928017070 con 0x7f793c06f9d0 2024-05-28T14:07:57.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.761+0000 7f7943a00640 1 -- 172.21.15.42:0/967930910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793c205120 con 0x7f793c06f9d0 2024-05-28T14:07:57.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.761+0000 7f7943a00640 1 -- 172.21.15.42:0/967930910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f793c0a9910 con 0x7f793c06f9d0 2024-05-28T14:07:57.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.765+0000 7f7943a00640 1 -- 172.21.15.42:0/967930910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f793c0738b0 con 0x7f793c06f9d0 2024-05-28T14:07:57.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.761+0000 7f7938c00640 1 -- 172.21.15.42:0/967930910 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928011b90 con 0x7f793c06f9d0 2024-05-28T14:07:57.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.773+0000 7f7938c00640 1 -- 172.21.15.42:0/967930910 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 28) ==== 89568+0+0 (secure 0 0 0) 0x7f7928011d30 con 0x7f793c06f9d0 2024-05-28T14:07:57.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.777+0000 7f7938c00640 1 --2- 172.21.15.42:0/967930910 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f791c069570 0x7f791c06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:57.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.777+0000 7f7938c00640 1 -- 172.21.15.42:0/967930910 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f79280860f0 con 0x7f793c06f9d0 2024-05-28T14:07:57.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.793+0000 7f793be00640 1 --2- 172.21.15.42:0/967930910 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f791c069570 0x7f791c06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:57.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.797+0000 7f7938c00640 1 -- 172.21.15.42:0/967930910 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7928057df0 con 0x7f793c06f9d0 2024-05-28T14:07:57.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:57.805+0000 7f793be00640 1 --2- 172.21.15.42:0/967930910 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f791c069570 0x7f791c06ba30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f793c06c460 tx=0x7f793400d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:58.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:58.313+0000 7f7943a00640 1 -- 172.21.15.42:0/967930910 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone8116", "target": ["mon-mgr", ""]}) -- 0x7f793c04fdd0 con 0x7f791c069570 2024-05-28T14:07:58.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:58.829+0000 7f7938c00640 1 -- 172.21.15.42:0/967930910 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f793c04fdd0 con 0x7f791c069570 2024-05-28T14:07:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:07:58.815 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:07:58.829+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:07:58.837 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:07:58.838 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone8117 2024-05-28T14:07:59.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.221+0000 7f1ee3000640 1 Processor -- start 2024-05-28T14:07:59.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.221+0000 7f1ee3000640 1 -- start start 2024-05-28T14:07:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.225+0000 7f1ee3000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1edc06dd10 0x7f1edc071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.225+0000 7f1ee3000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1edc0ce1c0 con 0x7f1edc06e1a0 2024-05-28T14:07:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.225+0000 7f1ee3000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1edc06e760 con 0x7f1edc162b80 2024-05-28T14:07:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.225+0000 7f1ee3000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1edc06ea40 con 0x7f1edc06dd10 2024-05-28T14:07:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.225+0000 7f1ee1000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1edc06e1a0 0x7f1edc0cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36464/0 (socket says 172.21.15.42:36464) 2024-05-28T14:07:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.225+0000 7f1ee1000640 1 -- 172.21.15.42:0/1538972065 learned_addr learned my addr 172.21.15.42:0/1538972065 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:07:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.225+0000 7f1edb400640 1 --2- 172.21.15.42:0/1538972065 >> v2:172.21.15.63:3300/0 conn(0x7f1edc06dd10 0x7f1edc071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:59.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.225+0000 7f1edb400640 1 -- 172.21.15.42:0/1538972065 >> v1:172.21.15.63:6789/0 conn(0x7f1edc162b80 legacy=0x7f1edc0b3800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-28T14:07:59.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.225+0000 7f1edb400640 1 -- 172.21.15.42:0/1538972065 >> v1:172.21.15.42:6789/0 conn(0x7f1edc06e1a0 legacy=0x7f1edc0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:07:59.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.225+0000 7f1edb400640 1 -- 172.21.15.42:0/1538972065 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1edc0ce460 con 0x7f1edc06dd10 2024-05-28T14:07:59.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.233+0000 7f1edb400640 1 --2- 172.21.15.42:0/1538972065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1edc06dd10 0x7f1edc071de0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80097c0 tx=0x7f1ec801c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0e479c18c5bf912 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:59.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.237+0000 7f1edaa00640 1 -- 172.21.15.42:0/1538972065 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec80b0070 con 0x7f1edc06dd10 2024-05-28T14:07:59.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.237+0000 7f1edaa00640 1 -- 172.21.15.42:0/1538972065 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec80ab600 con 0x7f1edc06dd10 2024-05-28T14:07:59.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.237+0000 7f1edaa00640 1 -- 172.21.15.42:0/1538972065 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec80ab9d0 con 0x7f1edc06dd10 2024-05-28T14:07:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.241+0000 7f1ee3000640 1 -- 172.21.15.42:0/1538972065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1edc06dd10 msgr2=0x7f1edc071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.241+0000 7f1ee3000640 1 --2- 172.21.15.42:0/1538972065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1edc06dd10 0x7f1edc071de0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80097c0 tx=0x7f1ec801c420 comp rx=0 tx=0).stop 2024-05-28T14:07:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.241+0000 7f1ee3000640 1 -- 172.21.15.42:0/1538972065 shutdown_connections 2024-05-28T14:07:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.241+0000 7f1ee3000640 1 --2- 172.21.15.42:0/1538972065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1edc06dd10 0x7f1edc071de0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.241+0000 7f1ee3000640 1 -- 172.21.15.42:0/1538972065 >> 172.21.15.42:0/1538972065 conn(0x7f1edc0b21f0 msgr2=0x7f1edc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:07:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.241+0000 7f1ee3000640 1 -- 172.21.15.42:0/1538972065 shutdown_connections 2024-05-28T14:07:59.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.241+0000 7f1ee3000640 1 -- 172.21.15.42:0/1538972065 wait complete. 2024-05-28T14:07:59.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.245+0000 7f1ee3000640 1 Processor -- start 2024-05-28T14:07:59.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.245+0000 7f1ee3000640 1 -- start start 2024-05-28T14:07:59.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.245+0000 7f1ee3000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1edc06dd10 0x7f1edc204a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:59.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.245+0000 7f1ee3000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1edc06e1a0 0x7f1edc15e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:59.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.245+0000 7f1ee3000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1edc162b80 0x7f1edc161800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:59.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1edbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1edc06dd10 0x7f1edc204a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:59.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1edbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1edc06dd10 0x7f1edc204a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59344/0 (socket says 172.21.15.42:59344) 2024-05-28T14:07:59.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1ee3000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc162f60 con 0x7f1edc06dd10 2024-05-28T14:07:59.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1ee3000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc072730 con 0x7f1edc06e1a0 2024-05-28T14:07:59.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1ee3000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1edc1630e0 con 0x7f1edc162b80 2024-05-28T14:07:59.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1edbe00640 1 -- 172.21.15.42:0/1707637466 learned_addr learned my addr 172.21.15.42:0/1707637466 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:07:59.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1ee1000640 1 --2- 172.21.15.42:0/1707637466 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1edc162b80 0x7f1edc161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:59.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1edb400640 1 --2- 172.21.15.42:0/1707637466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1edc06e1a0 0x7f1edc15e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:59.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1edbe00640 1 -- 172.21.15.42:0/1707637466 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1edc162b80 msgr2=0x7f1edc161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:59.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1edbe00640 1 --2- 172.21.15.42:0/1707637466 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1edc162b80 0x7f1edc161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:59.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1edbe00640 1 -- 172.21.15.42:0/1707637466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1edc06e1a0 msgr2=0x7f1edc15e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:07:59.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1edbe00640 1 --2- 172.21.15.42:0/1707637466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1edc06e1a0 0x7f1edc15e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:07:59.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1edbe00640 1 -- 172.21.15.42:0/1707637466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1edc16d310 con 0x7f1edc06dd10 2024-05-28T14:07:59.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1edb400640 1 --2- 172.21.15.42:0/1707637466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1edc06e1a0 0x7f1edc15e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:07:59.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1edbe00640 1 --2- 172.21.15.42:0/1707637466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1edc06dd10 0x7f1edc204a20 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00b3e0 tx=0x7f1ecc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:59.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1ed3e00640 1 -- 172.21.15.42:0/1707637466 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc015020 con 0x7f1edc06dd10 2024-05-28T14:07:59.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1ed3e00640 1 -- 172.21.15.42:0/1707637466 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ecc011850 con 0x7f1edc06dd10 2024-05-28T14:07:59.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1ed3e00640 1 -- 172.21.15.42:0/1707637466 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc011c40 con 0x7f1edc06dd10 2024-05-28T14:07:59.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1ee3000640 1 -- 172.21.15.42:0/1707637466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1edc16d5f0 con 0x7f1edc06dd10 2024-05-28T14:07:59.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.249+0000 7f1ee3000640 1 -- 172.21.15.42:0/1707637466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1edc16a660 con 0x7f1edc06dd10 2024-05-28T14:07:59.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.253+0000 7f1ee3000640 1 -- 172.21.15.42:0/1707637466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1edc0738b0 con 0x7f1edc06dd10 2024-05-28T14:07:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.257+0000 7f1ed3e00640 1 -- 172.21.15.42:0/1707637466 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 28) ==== 89568+0+0 (secure 0 0 0) 0x7f1ecc018020 con 0x7f1edc06dd10 2024-05-28T14:07:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.257+0000 7f1ed3e00640 1 --2- 172.21.15.42:0/1707637466 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ebc0694f0 0x7f1ebc06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:07:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.257+0000 7f1ed3e00640 1 -- 172.21.15.42:0/1707637466 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f1ecc0856a0 con 0x7f1edc06dd10 2024-05-28T14:07:59.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.265+0000 7f1edb400640 1 --2- 172.21.15.42:0/1707637466 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ebc0694f0 0x7f1ebc06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:07:59.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.269+0000 7f1edb400640 1 --2- 172.21.15.42:0/1707637466 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ebc0694f0 0x7f1ebc06b9b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80ad000 tx=0x7f1ec8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:07:59.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.269+0000 7f1ed3e00640 1 -- 172.21.15.42:0/1707637466 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ecc01f020 con 0x7f1edc06dd10 2024-05-28T14:07:59.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:07:59.781+0000 7f1ee3000640 1 -- 172.21.15.42:0/1707637466 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone8117", "target": ["mon-mgr", ""]}) -- 0x7f1edc04fdd0 con 0x7f1ebc0694f0 2024-05-28T14:08:00.139 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:00.153+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:08:00.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.157+0000 7f1ed3e00640 1 -- 172.21.15.42:0/1707637466 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f1edc04fdd0 con 0x7f1ebc0694f0 2024-05-28T14:08:00.140 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:08:00.163 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:08:00.164 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone8118 2024-05-28T14:08:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.549+0000 7f0442e00640 1 Processor -- start 2024-05-28T14:08:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.553+0000 7f0442e00640 1 -- start start 2024-05-28T14:08:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.553+0000 7f0442e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0434047d60 0x7f0434037d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.553+0000 7f0442e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04340482b0 0x7f0434038270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.553+0000 7f0442e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0434006b70 con 0x7f043404f500 2024-05-28T14:08:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.553+0000 7f0442e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0434006e50 con 0x7f0434047d60 2024-05-28T14:08:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.553+0000 7f0442e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0434007130 con 0x7f04340482b0 2024-05-28T14:08:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f0441a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f043404f500 0x7f043404f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36480/0 (socket says 172.21.15.42:36480) 2024-05-28T14:08:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f0441a00640 1 -- 172.21.15.42:0/888616809 learned_addr learned my addr 172.21.15.42:0/888616809 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:08:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f0444400640 1 --2- 172.21.15.42:0/888616809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04340482b0 0x7f0434038270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:00.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f043be00640 1 -- 172.21.15.42:0/888616809 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2001307955 0 0) 0x7f0434006b70 con 0x7f043404f500 2024-05-28T14:08:00.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f043be00640 1 -- 172.21.15.42:0/888616809 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0424003530 con 0x7f043404f500 2024-05-28T14:08:00.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f0441000640 1 --2- 172.21.15.42:0/888616809 >> v2:172.21.15.42:3300/0 conn(0x7f0434047d60 0x7f0434037d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:00.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f043be00640 1 -- 172.21.15.42:0/888616809 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2968649448 0 0) 0x7f0424003530 con 0x7f043404f500 2024-05-28T14:08:00.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f043be00640 1 -- 172.21.15.42:0/888616809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04340482b0 msgr2=0x7f0434038270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:00.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f043be00640 1 --2- 172.21.15.42:0/888616809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04340482b0 0x7f0434038270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:00.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f043be00640 1 -- 172.21.15.42:0/888616809 >> v2:172.21.15.42:3300/0 conn(0x7f0434047d60 msgr2=0x7f0434037d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:00.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f043be00640 1 --2- 172.21.15.42:0/888616809 >> v2:172.21.15.42:3300/0 conn(0x7f0434047d60 0x7f0434037d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:00.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f043be00640 1 -- 172.21.15.42:0/888616809 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0434007410 con 0x7f043404f500 2024-05-28T14:08:00.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f0441000640 1 --2- 172.21.15.42:0/888616809 >> v2:172.21.15.42:3300/0 conn(0x7f0434047d60 0x7f0434037d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:08:00.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f043be00640 1 -- 172.21.15.42:0/888616809 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f0430001680 con 0x7f043404f500 2024-05-28T14:08:00.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.557+0000 7f043be00640 1 -- 172.21.15.42:0/888616809 >> v1:172.21.15.42:6789/0 conn(0x7f043404f500 legacy=0x7f043404f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:00.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.561+0000 7f043be00640 1 --2- 172.21.15.42:0/888616809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04240039c0 0x7f0424003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:00.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.561+0000 7f043be00640 1 --2- 172.21.15.42:0/888616809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0424004810 0x7f0424004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:00.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.561+0000 7f043be00640 1 --2- 172.21.15.42:0/888616809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04240051d0 0x7f042400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:00.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.561+0000 7f043be00640 1 -- 172.21.15.42:0/888616809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0434006e50 con 0x7f04240039c0 2024-05-28T14:08:00.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.561+0000 7f043be00640 1 -- 172.21.15.42:0/888616809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0434007130 con 0x7f0424004810 2024-05-28T14:08:00.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.561+0000 7f043be00640 1 -- 172.21.15.42:0/888616809 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f042400c040 con 0x7f04240051d0 2024-05-28T14:08:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.561+0000 7f0444400640 1 --2- 172.21.15.42:0/888616809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04340482b0 0x7f0434038270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:08:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.561+0000 7f0441000640 1 --2- 172.21.15.42:0/888616809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0424004810 0x7f0424004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.561+0000 7f0441a00640 1 --2- 172.21.15.42:0/888616809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04240039c0 0x7f0424003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.561+0000 7f0441a00640 1 -- 172.21.15.42:0/888616809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04240051d0 msgr2=0x7f042400b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-28T14:08:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.561+0000 7f0441a00640 1 --2- 172.21.15.42:0/888616809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04240051d0 0x7f042400b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.561+0000 7f0441a00640 1 -- 172.21.15.42:0/888616809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0424004810 msgr2=0x7f0424004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.561+0000 7f0441a00640 1 --2- 172.21.15.42:0/888616809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0424004810 0x7f0424004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.561+0000 7f0441a00640 1 -- 172.21.15.42:0/888616809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0424003530 con 0x7f04240039c0 2024-05-28T14:08:00.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.565+0000 7f0441a00640 1 --2- 172.21.15.42:0/888616809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04240039c0 0x7f0424003de0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f043000c240 tx=0x7f043001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6863fb0d59033 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:00.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.569+0000 7f043be00640 1 -- 172.21.15.42:0/888616809 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430007d80 con 0x7f04240039c0 2024-05-28T14:08:00.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.569+0000 7f043be00640 1 -- 172.21.15.42:0/888616809 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04300a9020 con 0x7f04240039c0 2024-05-28T14:08:00.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.569+0000 7f043be00640 1 -- 172.21.15.42:0/888616809 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430005ae0 con 0x7f04240039c0 2024-05-28T14:08:00.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.573+0000 7f0442e00640 1 -- 172.21.15.42:0/888616809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04240039c0 msgr2=0x7f0424003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:00.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.573+0000 7f0442e00640 1 --2- 172.21.15.42:0/888616809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04240039c0 0x7f0424003de0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f043000c240 tx=0x7f043001ccc0 comp rx=0 tx=0).stop 2024-05-28T14:08:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.573+0000 7f0444400640 1 -- 172.21.15.42:0/888616809 reap_dead start 2024-05-28T14:08:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.577+0000 7f0442e00640 1 -- 172.21.15.42:0/888616809 shutdown_connections 2024-05-28T14:08:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.577+0000 7f0442e00640 1 -- 172.21.15.42:0/888616809 >> 172.21.15.42:0/888616809 conn(0x7f0434054ca0 msgr2=0x7f04340550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.577+0000 7f0442e00640 1 -- 172.21.15.42:0/888616809 shutdown_connections 2024-05-28T14:08:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.577+0000 7f0442e00640 1 -- 172.21.15.42:0/888616809 wait complete. 2024-05-28T14:08:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.585+0000 7f0442e00640 1 Processor -- start 2024-05-28T14:08:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.585+0000 7f0442e00640 1 -- start start 2024-05-28T14:08:00.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.589+0000 7f0442e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04240039c0 0x7f04340e2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:00.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.589+0000 7f0442e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04340ef550 0x7f0434181f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:00.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.589+0000 7f0442e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04340ef9d0 0x7f04341824a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:00.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.589+0000 7f0442e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0434038e90 con 0x7f04340ef9d0 2024-05-28T14:08:00.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.589+0000 7f0442e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0434038d10 con 0x7f04340ef550 2024-05-28T14:08:00.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.589+0000 7f0442e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0434039010 con 0x7f04240039c0 2024-05-28T14:08:00.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.593+0000 7f0444400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04340ef9d0 0x7f04341824a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:00.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.593+0000 7f0444400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04340ef9d0 0x7f04341824a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59370/0 (socket says 172.21.15.42:59370) 2024-05-28T14:08:00.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.593+0000 7f0444400640 1 -- 172.21.15.42:0/3637607080 learned_addr learned my addr 172.21.15.42:0/3637607080 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:00.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.593+0000 7f0441a00640 1 --2- 172.21.15.42:0/3637607080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04240039c0 0x7f04340e2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:00.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.593+0000 7f0441000640 1 --2- 172.21.15.42:0/3637607080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04340ef550 0x7f0434181f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:00.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.597+0000 7f0444400640 1 -- 172.21.15.42:0/3637607080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04240039c0 msgr2=0x7f04340e2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:00.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.597+0000 7f0444400640 1 --2- 172.21.15.42:0/3637607080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04240039c0 0x7f04340e2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:00.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.597+0000 7f0444400640 1 -- 172.21.15.42:0/3637607080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04340ef550 msgr2=0x7f0434181f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:00.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.597+0000 7f0444400640 1 --2- 172.21.15.42:0/3637607080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04340ef550 0x7f0434181f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:00.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.597+0000 7f0444400640 1 -- 172.21.15.42:0/3637607080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04341829e0 con 0x7f04340ef9d0 2024-05-28T14:08:00.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.597+0000 7f0444400640 1 --2- 172.21.15.42:0/3637607080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04340ef9d0 0x7f04341824a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f043c069140 tx=0x7f043c069570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:00.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.597+0000 7f0439600640 1 -- 172.21.15.42:0/3637607080 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c098070 con 0x7f04340ef9d0 2024-05-28T14:08:00.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.601+0000 7f0442e00640 1 -- 172.21.15.42:0/3637607080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0434189290 con 0x7f04340ef9d0 2024-05-28T14:08:00.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.601+0000 7f0442e00640 1 -- 172.21.15.42:0/3637607080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0434048c50 con 0x7f04340ef9d0 2024-05-28T14:08:00.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.601+0000 7f0439600640 1 -- 172.21.15.42:0/3637607080 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f043c071810 con 0x7f04340ef9d0 2024-05-28T14:08:00.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.601+0000 7f0439600640 1 -- 172.21.15.42:0/3637607080 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c098070 con 0x7f04340ef9d0 2024-05-28T14:08:00.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.609+0000 7f042ea00640 1 -- 172.21.15.42:0/3637607080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04340054a0 con 0x7f04340ef9d0 2024-05-28T14:08:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.621+0000 7f0439600640 1 -- 172.21.15.42:0/3637607080 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 28) ==== 89568+0+0 (secure 0 0 0) 0x7f043c09b070 con 0x7f04340ef9d0 2024-05-28T14:08:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.625+0000 7f0439600640 1 --2- 172.21.15.42:0/3637607080 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f041c0696f0 0x7f041c06bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:00.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.625+0000 7f0439600640 1 -- 172.21.15.42:0/3637607080 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f043c100360 con 0x7f04340ef9d0 2024-05-28T14:08:00.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.629+0000 7f0439600640 1 -- 172.21.15.42:0/3637607080 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f043c0d2020 con 0x7f04340ef9d0 2024-05-28T14:08:00.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.657+0000 7f0441a00640 1 --2- 172.21.15.42:0/3637607080 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f041c0696f0 0x7f041c06bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:00.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:00.661+0000 7f0441a00640 1 --2- 172.21.15.42:0/3637607080 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f041c0696f0 0x7f041c06bbb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f04300aa000 tx=0x7f0430001ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:01.113+0000 7f042ea00640 1 -- 172.21.15.42:0/3637607080 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone8118", "target": ["mon-mgr", ""]}) -- 0x7f0434004f60 con 0x7f041c0696f0 2024-05-28T14:08:01.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:01.709+0000 7f0439600640 1 -- 172.21.15.42:0/3637607080 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f0434004f60 con 0x7f041c0696f0 2024-05-28T14:08:01.691 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:01.705+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:08:01.692 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:08:01.731 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:08:01.732 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone8119 2024-05-28T14:08:02.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.097+0000 7f23fe200640 1 Processor -- start 2024-05-28T14:08:02.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.097+0000 7f23fe200640 1 -- start start 2024-05-28T14:08:02.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.101+0000 7f23fe200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f23f815ea80 0x7f23f8073990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:02.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.101+0000 7f23fe200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f23f80b1090 0x7f23f80cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:02.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.101+0000 7f23fe200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23f80b2bb0 con 0x7f23f80cee50 2024-05-28T14:08:02.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.101+0000 7f23fe200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f23f80cff40 con 0x7f23f80b1090 2024-05-28T14:08:02.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.101+0000 7f23fe200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f23f8073ed0 con 0x7f23f815ea80 2024-05-28T14:08:02.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.101+0000 7f23fd800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f23f80b1090 0x7f23f80cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:02.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.101+0000 7f23fd800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f23f80b1090 0x7f23f80cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59376/0 (socket says 172.21.15.42:59376) 2024-05-28T14:08:02.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.101+0000 7f23fd800640 1 -- 172.21.15.42:0/113783741 learned_addr learned my addr 172.21.15.42:0/113783741 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:02.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.101+0000 7f23fce00640 1 --2- 172.21.15.42:0/113783741 >> v2:172.21.15.63:3300/0 conn(0x7f23f815ea80 0x7f23f8073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:02.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.101+0000 7f23fd800640 1 -- 172.21.15.42:0/113783741 >> v2:172.21.15.63:3300/0 conn(0x7f23f815ea80 msgr2=0x7f23f8073990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:02.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.105+0000 7f23fd800640 1 --2- 172.21.15.42:0/113783741 >> v2:172.21.15.63:3300/0 conn(0x7f23f815ea80 0x7f23f8073990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:02.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.105+0000 7f23fd800640 1 -- 172.21.15.42:0/113783741 >> v1:172.21.15.63:6789/0 conn(0x7f23f80cee50 legacy=0x7f23f80b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:02.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.105+0000 7f23fd800640 1 -- 172.21.15.42:0/113783741 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f80cf3c0 con 0x7f23f80b1090 2024-05-28T14:08:02.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.109+0000 7f23fd800640 1 --2- 172.21.15.42:0/113783741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23f80b1090 0x7f23f80cccf0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009e50 tx=0x7f23f0024f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1649f735cae27020 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:02.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.109+0000 7f23f7e00640 1 -- 172.21.15.42:0/113783741 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f00b9070 con 0x7f23f80b1090 2024-05-28T14:08:02.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.109+0000 7f23f7e00640 1 -- 172.21.15.42:0/113783741 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f00b4400 con 0x7f23f80b1090 2024-05-28T14:08:02.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.113+0000 7f23f7e00640 1 -- 172.21.15.42:0/113783741 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f00b47d0 con 0x7f23f80b1090 2024-05-28T14:08:02.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.113+0000 7f23fe200640 1 -- 172.21.15.42:0/113783741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23f80b1090 msgr2=0x7f23f80cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.113+0000 7f23fe200640 1 --2- 172.21.15.42:0/113783741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23f80b1090 0x7f23f80cccf0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009e50 tx=0x7f23f0024f20 comp rx=0 tx=0).stop 2024-05-28T14:08:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.113+0000 7f23fe200640 1 -- 172.21.15.42:0/113783741 shutdown_connections 2024-05-28T14:08:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.113+0000 7f23fe200640 1 --2- 172.21.15.42:0/113783741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23f80b1090 0x7f23f80cccf0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.113+0000 7f23fe200640 1 --2- 172.21.15.42:0/113783741 >> v2:172.21.15.63:3300/0 conn(0x7f23f815ea80 0x7f23f8073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.113+0000 7f23fe200640 1 -- 172.21.15.42:0/113783741 >> 172.21.15.42:0/113783741 conn(0x7f23f80b1d60 msgr2=0x7f23f80b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:02.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.117+0000 7f23fe200640 1 -- 172.21.15.42:0/113783741 shutdown_connections 2024-05-28T14:08:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.117+0000 7f23fe200640 1 -- 172.21.15.42:0/113783741 wait complete. 2024-05-28T14:08:02.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.121+0000 7f23fe200640 1 Processor -- start 2024-05-28T14:08:02.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.121+0000 7f23fe200640 1 -- start start 2024-05-28T14:08:02.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.121+0000 7f23fe200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23f80b1090 0x7f23f815a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.121+0000 7f23fe200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23f80cee50 0x7f23f814f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.121+0000 7f23fe200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23f815ea80 0x7f23f814ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.121+0000 7f23fe200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f23f80ad040 con 0x7f23f815ea80 2024-05-28T14:08:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.121+0000 7f23fe200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f23f815ee60 con 0x7f23f80b1090 2024-05-28T14:08:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.121+0000 7f23fe200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f23f80cda40 con 0x7f23f80cee50 2024-05-28T14:08:02.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.129+0000 7f23efe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23f80cee50 0x7f23f814f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:02.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.129+0000 7f23fce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23f80b1090 0x7f23f815a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:02.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.129+0000 7f23efe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23f80cee50 0x7f23f814f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:55228/0 (socket says 172.21.15.42:55228) 2024-05-28T14:08:02.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.129+0000 7f23efe00640 1 -- 172.21.15.42:0/1504069003 learned_addr learned my addr 172.21.15.42:0/1504069003 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:02.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.129+0000 7f23fd800640 1 --2- 172.21.15.42:0/1504069003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23f815ea80 0x7f23f814ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:02.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.129+0000 7f23efe00640 1 -- 172.21.15.42:0/1504069003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23f80b1090 msgr2=0x7f23f815a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:02.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.129+0000 7f23efe00640 1 --2- 172.21.15.42:0/1504069003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23f80b1090 0x7f23f815a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:02.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.129+0000 7f23efe00640 1 -- 172.21.15.42:0/1504069003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23f815ea80 msgr2=0x7f23f814ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:02.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.129+0000 7f23efe00640 1 --2- 172.21.15.42:0/1504069003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23f815ea80 0x7f23f814ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:02.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.129+0000 7f23efe00640 1 -- 172.21.15.42:0/1504069003 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f814f490 con 0x7f23f80cee50 2024-05-28T14:08:02.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.129+0000 7f23efe00640 1 --2- 172.21.15.42:0/1504069003 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23f80cee50 0x7f23f814f9f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f23e000c4c0 tx=0x7f23e0001e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:02.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.141+0000 7f23f5600640 1 -- 172.21.15.42:0/1504069003 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0017070 con 0x7f23f80cee50 2024-05-28T14:08:02.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.141+0000 7f23f5600640 1 -- 172.21.15.42:0/1504069003 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e0003600 con 0x7f23f80cee50 2024-05-28T14:08:02.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.141+0000 7f23f5600640 1 -- 172.21.15.42:0/1504069003 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e00039f0 con 0x7f23f80cee50 2024-05-28T14:08:02.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.149+0000 7f23fe200640 1 -- 172.21.15.42:0/1504069003 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f814c5a0 con 0x7f23f80cee50 2024-05-28T14:08:02.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.149+0000 7f23fe200640 1 -- 172.21.15.42:0/1504069003 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f814cb60 con 0x7f23f80cee50 2024-05-28T14:08:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.153+0000 7f23fe200640 1 -- 172.21.15.42:0/1504069003 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f804fcf0 con 0x7f23f80cee50 2024-05-28T14:08:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.153+0000 7f23f5600640 1 -- 172.21.15.42:0/1504069003 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 28) ==== 89568+0+0 (secure 0 0 0) 0x7f23e001a070 con 0x7f23f80cee50 2024-05-28T14:08:02.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.157+0000 7f23f5600640 1 --2- 172.21.15.42:0/1504069003 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23d40695c0 0x7f23d406ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:02.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.157+0000 7f23f5600640 1 -- 172.21.15.42:0/1504069003 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f23e0084c50 con 0x7f23f80cee50 2024-05-28T14:08:02.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.161+0000 7f23f5600640 1 -- 172.21.15.42:0/1504069003 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f23e0056910 con 0x7f23f80cee50 2024-05-28T14:08:02.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.189+0000 7f23fce00640 1 --2- 172.21.15.42:0/1504069003 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23d40695c0 0x7f23d406ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:02.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.205+0000 7f23fce00640 1 --2- 172.21.15.42:0/1504069003 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23d40695c0 0x7f23d406ba80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f23f8152aa0 tx=0x7f23e8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:02.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:02.749+0000 7f23fe200640 1 -- 172.21.15.42:0/1504069003 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone8119", "target": ["mon-mgr", ""]}) -- 0x7f23f80cc370 con 0x7f23d40695c0 2024-05-28T14:08:03.038 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:03.053+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:08:03.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.053+0000 7f23f5600640 1 -- 172.21.15.42:0/1504069003 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f23f80cc370 con 0x7f23d40695c0 2024-05-28T14:08:03.039 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:08:03.100 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:08:03.101 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone81110 2024-05-28T14:08:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.609+0000 7ffa70a00640 1 Processor -- start 2024-05-28T14:08:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.609+0000 7ffa70a00640 1 -- start start 2024-05-28T14:08:03.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.609+0000 7ffa70a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa640e30d0 0x7ffa6403a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:03.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.609+0000 7ffa70a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffa64053970 0x7ffa6404f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:03.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.609+0000 7ffa70a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffa6404fd50 0x7ffa640e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:03.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.609+0000 7ffa70a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ffa64037bc0 con 0x7ffa6404fd50 2024-05-28T14:08:03.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.609+0000 7ffa70a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffa640501b0 con 0x7ffa64053970 2024-05-28T14:08:03.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.609+0000 7ffa70a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa64053e70 con 0x7ffa640e30d0 2024-05-28T14:08:03.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.613+0000 7ffa6b400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa640e30d0 0x7ffa6403a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:03.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.613+0000 7ffa6b400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa640e30d0 0x7ffa6403a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:55246/0 (socket says 172.21.15.42:55246) 2024-05-28T14:08:03.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.613+0000 7ffa6b400640 1 -- 172.21.15.42:0/788392733 learned_addr learned my addr 172.21.15.42:0/788392733 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:03.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.613+0000 7ffa6be00640 1 --2- 172.21.15.42:0/788392733 >> v2:172.21.15.42:3300/0 conn(0x7ffa6404fd50 0x7ffa640e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:03.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.613+0000 7ffa6aa00640 1 --2- 172.21.15.42:0/788392733 >> v2:172.21.15.63:3300/0 conn(0x7ffa64053970 0x7ffa6404f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:03.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.613+0000 7ffa6be00640 1 -- 172.21.15.42:0/788392733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa640e30d0 msgr2=0x7ffa6403a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:03.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.613+0000 7ffa6be00640 1 --2- 172.21.15.42:0/788392733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa640e30d0 0x7ffa6403a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:03.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.613+0000 7ffa6be00640 1 -- 172.21.15.42:0/788392733 >> v2:172.21.15.63:3300/0 conn(0x7ffa64053970 msgr2=0x7ffa6404f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:03.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.613+0000 7ffa6be00640 1 --2- 172.21.15.42:0/788392733 >> v2:172.21.15.63:3300/0 conn(0x7ffa64053970 0x7ffa6404f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:03.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.613+0000 7ffa6be00640 1 -- 172.21.15.42:0/788392733 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa64054150 con 0x7ffa6404fd50 2024-05-28T14:08:03.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.617+0000 7ffa6be00640 1 --2- 172.21.15.42:0/788392733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa6404fd50 0x7ffa640e6cb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c06af90 tx=0x7ffa6c097ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5d4cb7e53bac2b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:03.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.621+0000 7ffa6a000640 1 -- 172.21.15.42:0/788392733 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c120a40 con 0x7ffa6404fd50 2024-05-28T14:08:03.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.621+0000 7ffa6a000640 1 -- 172.21.15.42:0/788392733 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c120be0 con 0x7ffa6404fd50 2024-05-28T14:08:03.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.625+0000 7ffa70a00640 1 -- 172.21.15.42:0/788392733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa6404fd50 msgr2=0x7ffa640e6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:03.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.625+0000 7ffa70a00640 1 --2- 172.21.15.42:0/788392733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa6404fd50 0x7ffa640e6cb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c06af90 tx=0x7ffa6c097ce0 comp rx=0 tx=0).stop 2024-05-28T14:08:03.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.629+0000 7ffa70a00640 1 -- 172.21.15.42:0/788392733 shutdown_connections 2024-05-28T14:08:03.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.629+0000 7ffa70a00640 1 --2- 172.21.15.42:0/788392733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa6404fd50 0x7ffa640e6cb0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:03.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.629+0000 7ffa70a00640 1 --2- 172.21.15.42:0/788392733 >> v2:172.21.15.63:3300/0 conn(0x7ffa64053970 0x7ffa6404f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:03.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.629+0000 7ffa70a00640 1 --2- 172.21.15.42:0/788392733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa640e30d0 0x7ffa6403a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:03.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.629+0000 7ffa70a00640 1 -- 172.21.15.42:0/788392733 >> 172.21.15.42:0/788392733 conn(0x7ffa64009fc0 msgr2=0x7ffa6400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:03.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.629+0000 7ffa70a00640 1 -- 172.21.15.42:0/788392733 shutdown_connections 2024-05-28T14:08:03.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.633+0000 7ffa70a00640 1 -- 172.21.15.42:0/788392733 wait complete. 2024-05-28T14:08:03.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.633+0000 7ffa70a00640 1 Processor -- start 2024-05-28T14:08:03.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.633+0000 7ffa70a00640 1 -- start start 2024-05-28T14:08:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.637+0000 7ffa70a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa6404fd50 0x7ffa6404be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.637+0000 7ffa70a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa64053970 0x7ffa640e3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.637+0000 7ffa70a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa640e30d0 0x7ffa640d9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.637+0000 7ffa70a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffa640e3630 con 0x7ffa6404fd50 2024-05-28T14:08:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.637+0000 7ffa70a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa640e34b0 con 0x7ffa64053970 2024-05-28T14:08:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.637+0000 7ffa70a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa640063b0 con 0x7ffa640e30d0 2024-05-28T14:08:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.637+0000 7ffa6b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa6404fd50 0x7ffa6404be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.637+0000 7ffa6b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa6404fd50 0x7ffa6404be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59418/0 (socket says 172.21.15.42:59418) 2024-05-28T14:08:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.637+0000 7ffa6b400640 1 -- 172.21.15.42:0/3175343589 learned_addr learned my addr 172.21.15.42:0/3175343589 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.637+0000 7ffa6aa00640 1 --2- 172.21.15.42:0/3175343589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa64053970 0x7ffa640e3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.637+0000 7ffa6be00640 1 --2- 172.21.15.42:0/3175343589 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa640e30d0 0x7ffa640d9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.641+0000 7ffa6b400640 1 -- 172.21.15.42:0/3175343589 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa640e30d0 msgr2=0x7ffa640d9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.641+0000 7ffa6b400640 1 --2- 172.21.15.42:0/3175343589 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa640e30d0 0x7ffa640d9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.641+0000 7ffa6b400640 1 -- 172.21.15.42:0/3175343589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa64053970 msgr2=0x7ffa640e3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.641+0000 7ffa6b400640 1 --2- 172.21.15.42:0/3175343589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa64053970 0x7ffa640e3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.641+0000 7ffa6b400640 1 -- 172.21.15.42:0/3175343589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa640e6ca0 con 0x7ffa6404fd50 2024-05-28T14:08:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.641+0000 7ffa6b400640 1 --2- 172.21.15.42:0/3175343589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa6404fd50 0x7ffa6404be40 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000d800 tx=0x7ffa6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:03.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.641+0000 7ffa5f400640 1 -- 172.21.15.42:0/3175343589 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa600096a0 con 0x7ffa6404fd50 2024-05-28T14:08:03.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.641+0000 7ffa5f400640 1 -- 172.21.15.42:0/3175343589 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa60019070 con 0x7ffa6404fd50 2024-05-28T14:08:03.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.641+0000 7ffa5f400640 1 -- 172.21.15.42:0/3175343589 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60009a40 con 0x7ffa6404fd50 2024-05-28T14:08:03.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.641+0000 7ffa70a00640 1 -- 172.21.15.42:0/3175343589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa640dd1b0 con 0x7ffa6404fd50 2024-05-28T14:08:03.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.641+0000 7ffa70a00640 1 -- 172.21.15.42:0/3175343589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa640dd770 con 0x7ffa6404fd50 2024-05-28T14:08:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.649+0000 7ffa5f400640 1 -- 172.21.15.42:0/3175343589 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 28) ==== 89568+0+0 (secure 0 0 0) 0x7ffa60009be0 con 0x7ffa6404fd50 2024-05-28T14:08:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.649+0000 7ffa5cc00640 1 -- 172.21.15.42:0/3175343589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa640dd970 con 0x7ffa6404fd50 2024-05-28T14:08:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.649+0000 7ffa5f400640 1 --2- 172.21.15.42:0/3175343589 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa4c0695c0 0x7ffa4c06ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.649+0000 7ffa5f400640 1 -- 172.21.15.42:0/3175343589 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7ffa60085c20 con 0x7ffa6404fd50 2024-05-28T14:08:03.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.669+0000 7ffa5f400640 1 -- 172.21.15.42:0/3175343589 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffa640dd970 con 0x7ffa6404fd50 2024-05-28T14:08:03.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.713+0000 7ffa6aa00640 1 --2- 172.21.15.42:0/3175343589 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa4c0695c0 0x7ffa4c06ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:03.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:03.749+0000 7ffa6aa00640 1 --2- 172.21.15.42:0/3175343589 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa4c0695c0 0x7ffa4c06ba80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ffa640d9810 tx=0x7ffa58008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:04.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:04.181+0000 7ffa5f400640 1 -- 172.21.15.42:0/3175343589 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7ffa60053850 con 0x7ffa6404fd50 2024-05-28T14:08:04.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:04.289+0000 7ffa5cc00640 1 -- 172.21.15.42:0/3175343589 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone81110", "target": ["mon-mgr", ""]}) -- 0x7ffa64052830 con 0x7ffa4c0695c0 2024-05-28T14:08:04.792 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:04.805+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:08:04.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:04.809+0000 7ffa5f400640 1 -- 172.21.15.42:0/3175343589 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7ffa64052830 con 0x7ffa4c0695c0 2024-05-28T14:08:04.798 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:08:04.840 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:08:04.841 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone81111 2024-05-28T14:08:05.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.265+0000 7f8654600640 1 Processor -- start 2024-05-28T14:08:05.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.265+0000 7f8654600640 1 -- start start 2024-05-28T14:08:05.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.269+0000 7f8654600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f864c15c8a0 0x7f864c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:05.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.269+0000 7f8654600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f864c0ce090 0x7f864c09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:05.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.269+0000 7f8654600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c09af90 0x7f864c160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:05.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.269+0000 7f8654600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f864c15cc80 con 0x7f864c15c8a0 2024-05-28T14:08:05.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.269+0000 7f8654600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f864c0d0280 con 0x7f864c0ce090 2024-05-28T14:08:05.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.269+0000 7f8654600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f864c0d0560 con 0x7f864c09af90 2024-05-28T14:08:05.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.269+0000 7f8651200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f864c0ce090 0x7f864c09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:05.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.269+0000 7f8652600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c09af90 0x7f864c160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:05.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.273+0000 7f8652600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c09af90 0x7f864c160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:55260/0 (socket says 172.21.15.42:55260) 2024-05-28T14:08:05.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.273+0000 7f8652600640 1 -- 172.21.15.42:0/2975066587 learned_addr learned my addr 172.21.15.42:0/2975066587 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:05.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.273+0000 7f8652600640 1 -- 172.21.15.42:0/2975066587 >> v2:172.21.15.63:3300/0 conn(0x7f864c0ce090 msgr2=0x7f864c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:05.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.273+0000 7f8652600640 1 --2- 172.21.15.42:0/2975066587 >> v2:172.21.15.63:3300/0 conn(0x7f864c0ce090 0x7f864c09aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:05.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.273+0000 7f8652600640 1 -- 172.21.15.42:0/2975066587 >> v2:172.21.15.42:3300/0 conn(0x7f864c15c8a0 msgr2=0x7f864c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:05.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.273+0000 7f8652600640 1 --2- 172.21.15.42:0/2975066587 >> v2:172.21.15.42:3300/0 conn(0x7f864c15c8a0 0x7f864c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:05.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.273+0000 7f8652600640 1 -- 172.21.15.42:0/2975066587 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864c0d0840 con 0x7f864c09af90 2024-05-28T14:08:05.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.281+0000 7f8652600640 1 --2- 172.21.15.42:0/2975066587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c09af90 0x7f864c160380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f864c06f8a0 tx=0x7f864401c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ecda4a022b22655 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:05.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.281+0000 7f864be00640 1 -- 172.21.15.42:0/2975066587 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86440ac070 con 0x7f864c09af90 2024-05-28T14:08:05.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.281+0000 7f864be00640 1 -- 172.21.15.42:0/2975066587 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86440a8420 con 0x7f864c09af90 2024-05-28T14:08:05.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.281+0000 7f864be00640 1 -- 172.21.15.42:0/2975066587 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86440a87f0 con 0x7f864c09af90 2024-05-28T14:08:05.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.281+0000 7f8654600640 1 -- 172.21.15.42:0/2975066587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c09af90 msgr2=0x7f864c160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:05.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.281+0000 7f8654600640 1 --2- 172.21.15.42:0/2975066587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c09af90 0x7f864c160380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f864c06f8a0 tx=0x7f864401c750 comp rx=0 tx=0).stop 2024-05-28T14:08:05.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.285+0000 7f8654600640 1 -- 172.21.15.42:0/2975066587 shutdown_connections 2024-05-28T14:08:05.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.285+0000 7f8654600640 1 --2- 172.21.15.42:0/2975066587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c09af90 0x7f864c160380 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:05.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.285+0000 7f8654600640 1 --2- 172.21.15.42:0/2975066587 >> v2:172.21.15.63:3300/0 conn(0x7f864c0ce090 0x7f864c09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:05.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.285+0000 7f8654600640 1 --2- 172.21.15.42:0/2975066587 >> v2:172.21.15.42:3300/0 conn(0x7f864c15c8a0 0x7f864c0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:05.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.285+0000 7f8654600640 1 -- 172.21.15.42:0/2975066587 >> 172.21.15.42:0/2975066587 conn(0x7f864c0b3f90 msgr2=0x7f864c0b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:05.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.289+0000 7f8654600640 1 -- 172.21.15.42:0/2975066587 shutdown_connections 2024-05-28T14:08:05.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.293+0000 7f8654600640 1 -- 172.21.15.42:0/2975066587 wait complete. 2024-05-28T14:08:05.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.301+0000 7f8654600640 1 Processor -- start 2024-05-28T14:08:05.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.301+0000 7f8654600640 1 -- start start 2024-05-28T14:08:05.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.301+0000 7f8654600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c09af90 0x7f864c15a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:05.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.305+0000 7f8654600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f864c0ce090 0x7f864c0ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:05.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.305+0000 7f8654600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c15c8a0 0x7f864c14fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:05.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.305+0000 7f8654600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f864c0ce470 con 0x7f864c0ce090 2024-05-28T14:08:05.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.305+0000 7f8654600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f864c0d1570 con 0x7f864c15c8a0 2024-05-28T14:08:05.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.305+0000 7f8654600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f864c0b52f0 con 0x7f864c09af90 2024-05-28T14:08:05.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.305+0000 7f8652600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c15c8a0 0x7f864c14fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:05.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.305+0000 7f8652600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c15c8a0 0x7f864c14fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:35830/0 (socket says 172.21.15.42:35830) 2024-05-28T14:08:05.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.305+0000 7f8652600640 1 -- 172.21.15.42:0/1573194123 learned_addr learned my addr 172.21.15.42:0/1573194123 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:05.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.305+0000 7f8652600640 1 -- 172.21.15.42:0/1573194123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c09af90 msgr2=0x7f864c15a890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T14:08:05.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.305+0000 7f8652600640 1 --2- 172.21.15.42:0/1573194123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f864c09af90 0x7f864c15a890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:05.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.309+0000 7f8652600640 1 -- 172.21.15.42:0/1573194123 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f864c0ce090 msgr2=0x7f864c0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:05.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.309+0000 7f8652600640 1 --2- 172.21.15.42:0/1573194123 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f864c0ce090 0x7f864c0ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:05.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.309+0000 7f8652600640 1 -- 172.21.15.42:0/1573194123 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864c0cabd0 con 0x7f864c15c8a0 2024-05-28T14:08:05.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.309+0000 7f8652600640 1 --2- 172.21.15.42:0/1573194123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f864c15c8a0 0x7f864c14fa00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f864401ae00 tx=0x7f86440194f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:05.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.325+0000 7f8649600640 1 -- 172.21.15.42:0/1573194123 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86440ac040 con 0x7f864c15c8a0 2024-05-28T14:08:05.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.325+0000 7f8649600640 1 -- 172.21.15.42:0/1573194123 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86440bd670 con 0x7f864c15c8a0 2024-05-28T14:08:05.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.325+0000 7f8649600640 1 -- 172.21.15.42:0/1573194123 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644009510 con 0x7f864c15c8a0 2024-05-28T14:08:05.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.325+0000 7f8654600640 1 -- 172.21.15.42:0/1573194123 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864c0caeb0 con 0x7f864c15c8a0 2024-05-28T14:08:05.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.325+0000 7f8654600640 1 -- 172.21.15.42:0/1573194123 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864c0cb470 con 0x7f864c15c8a0 2024-05-28T14:08:05.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.333+0000 7f8654600640 1 -- 172.21.15.42:0/1573194123 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86280052a0 con 0x7f864c15c8a0 2024-05-28T14:08:05.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.333+0000 7f8649600640 1 -- 172.21.15.42:0/1573194123 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f86440af070 con 0x7f864c15c8a0 2024-05-28T14:08:05.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.337+0000 7f8649600640 1 --2- 172.21.15.42:0/1573194123 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f863006d8c0 0x7f863006fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:05.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.337+0000 7f8649600640 1 -- 172.21.15.42:0/1573194123 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f8644120210 con 0x7f864c15c8a0 2024-05-28T14:08:05.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.345+0000 7f8651c00640 1 --2- 172.21.15.42:0/1573194123 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f863006d8c0 0x7f863006fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:05.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.349+0000 7f8649600640 1 -- 172.21.15.42:0/1573194123 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8644016600 con 0x7f864c15c8a0 2024-05-28T14:08:05.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.381+0000 7f8651c00640 1 --2- 172.21.15.42:0/1573194123 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f863006d8c0 0x7f863006fd80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f863c009670 tx=0x7f863c006b90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:05.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:05.817+0000 7f8654600640 1 -- 172.21.15.42:0/1573194123 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone81111", "target": ["mon-mgr", ""]}) -- 0x7f8628000bd0 con 0x7f863006d8c0 2024-05-28T14:08:06.153 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:06.169+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:08:06.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.169+0000 7f8649600640 1 -- 172.21.15.42:0/1573194123 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f8628000bd0 con 0x7f863006d8c0 2024-05-28T14:08:06.154 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:08:06.174 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:08:06.175 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone81112 2024-05-28T14:08:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.525+0000 7f76bf600640 1 Processor -- start 2024-05-28T14:08:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.529+0000 7f76bf600640 1 -- start start 2024-05-28T14:08:06.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.529+0000 7f76bf600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f76b8140d20 0x7f76b8167490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:06.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.529+0000 7f76bf600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b8160230 con 0x7f76b8071de0 2024-05-28T14:08:06.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.529+0000 7f76bf600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b8160510 con 0x7f76b80ce3b0 2024-05-28T14:08:06.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.529+0000 7f76bf600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f76b81607f0 con 0x7f76b8140d20 2024-05-28T14:08:06.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.529+0000 7f76bcc00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f76b8071de0 0x7f76b80721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36492/0 (socket says 172.21.15.42:36492) 2024-05-28T14:08:06.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.529+0000 7f76bcc00640 1 -- 172.21.15.42:0/3245317969 learned_addr learned my addr 172.21.15.42:0/3245317969 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:08:06.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.533+0000 7f76bd600640 1 --2- 172.21.15.42:0/3245317969 >> v2:172.21.15.63:3300/0 conn(0x7f76b8140d20 0x7f76b8167490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:06.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.533+0000 7f76b7400640 1 -- 172.21.15.42:0/3245317969 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 842126332 0 0) 0x7f76b8160230 con 0x7f76b8071de0 2024-05-28T14:08:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.533+0000 7f76b7400640 1 -- 172.21.15.42:0/3245317969 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f769c003530 con 0x7f76b8071de0 2024-05-28T14:08:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.533+0000 7f76bd600640 1 -- 172.21.15.42:0/3245317969 >> v1:172.21.15.63:6789/0 conn(0x7f76b80ce3b0 legacy=0x7f76b80ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.533+0000 7f76bd600640 1 -- 172.21.15.42:0/3245317969 >> v1:172.21.15.42:6789/0 conn(0x7f76b8071de0 legacy=0x7f76b80721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.533+0000 7f76bd600640 1 -- 172.21.15.42:0/3245317969 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b8160ad0 con 0x7f76b8140d20 2024-05-28T14:08:06.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.537+0000 7f76bd600640 1 --2- 172.21.15.42:0/3245317969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8140d20 0x7f76b8167490 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f76a00097c0 tx=0x7f76a001c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e88255e7e7c96f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:06.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.537+0000 7f76b7400640 1 -- 172.21.15.42:0/3245317969 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00b0070 con 0x7f76b8140d20 2024-05-28T14:08:06.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.537+0000 7f76b7400640 1 -- 172.21.15.42:0/3245317969 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a00ab600 con 0x7f76b8140d20 2024-05-28T14:08:06.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.537+0000 7f76b7400640 1 -- 172.21.15.42:0/3245317969 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00ab9d0 con 0x7f76b8140d20 2024-05-28T14:08:06.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.541+0000 7f76bf600640 1 -- 172.21.15.42:0/3245317969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8140d20 msgr2=0x7f76b8167490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:06.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.541+0000 7f76bf600640 1 --2- 172.21.15.42:0/3245317969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8140d20 0x7f76b8167490 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f76a00097c0 tx=0x7f76a001c340 comp rx=0 tx=0).stop 2024-05-28T14:08:06.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.541+0000 7f76bf600640 1 -- 172.21.15.42:0/3245317969 shutdown_connections 2024-05-28T14:08:06.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.541+0000 7f76bf600640 1 --2- 172.21.15.42:0/3245317969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8140d20 0x7f76b8167490 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:06.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.541+0000 7f76bf600640 1 -- 172.21.15.42:0/3245317969 >> 172.21.15.42:0/3245317969 conn(0x7f76b80b21f0 msgr2=0x7f76b80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:06.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.541+0000 7f76bf600640 1 -- 172.21.15.42:0/3245317969 shutdown_connections 2024-05-28T14:08:06.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.541+0000 7f76bf600640 1 -- 172.21.15.42:0/3245317969 wait complete. 2024-05-28T14:08:06.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.541+0000 7f76bf600640 1 Processor -- start 2024-05-28T14:08:06.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.541+0000 7f76bf600640 1 -- start start 2024-05-28T14:08:06.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bf600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b8071de0 0x7f76b81458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:06.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bf600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76b80ce3b0 0x7f76b806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:06.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bf600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8140d20 0x7f76b806f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:06.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bf600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f76b816a990 con 0x7f76b80ce3b0 2024-05-28T14:08:06.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bf600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76b816a810 con 0x7f76b8140d20 2024-05-28T14:08:06.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bf600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76b816ab10 con 0x7f76b8071de0 2024-05-28T14:08:06.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76b80ce3b0 0x7f76b806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:06.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76b80ce3b0 0x7f76b806f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59454/0 (socket says 172.21.15.42:59454) 2024-05-28T14:08:06.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76b7e00640 1 -- 172.21.15.42:0/2844900206 learned_addr learned my addr 172.21.15.42:0/2844900206 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:06.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bd600640 1 --2- 172.21.15.42:0/2844900206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8140d20 0x7f76b806f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:06.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bcc00640 1 --2- 172.21.15.42:0/2844900206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b8071de0 0x7f76b81458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:06.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bd600640 1 -- 172.21.15.42:0/2844900206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b8071de0 msgr2=0x7f76b81458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:06.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bd600640 1 --2- 172.21.15.42:0/2844900206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b8071de0 0x7f76b81458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:06.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bd600640 1 -- 172.21.15.42:0/2844900206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76b80ce3b0 msgr2=0x7f76b806f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:06.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bd600640 1 --2- 172.21.15.42:0/2844900206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76b80ce3b0 0x7f76b806f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:06.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bd600640 1 -- 172.21.15.42:0/2844900206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b815eb60 con 0x7f76b8140d20 2024-05-28T14:08:06.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bd600640 1 --2- 172.21.15.42:0/2844900206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b8140d20 0x7f76b806f610 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f76a00ab000 tx=0x7f76a00190e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:06.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76b4c00640 1 -- 172.21.15.42:0/2844900206 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00b9070 con 0x7f76b8140d20 2024-05-28T14:08:06.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76b4c00640 1 -- 172.21.15.42:0/2844900206 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a00b5690 con 0x7f76b8140d20 2024-05-28T14:08:06.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76b4c00640 1 -- 172.21.15.42:0/2844900206 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00b0040 con 0x7f76b8140d20 2024-05-28T14:08:06.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bf600640 1 -- 172.21.15.42:0/2844900206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b815ee40 con 0x7f76b8140d20 2024-05-28T14:08:06.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.545+0000 7f76bf600640 1 -- 172.21.15.42:0/2844900206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b815f400 con 0x7f76b8140d20 2024-05-28T14:08:06.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.549+0000 7f76bf600640 1 -- 172.21.15.42:0/2844900206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b80738b0 con 0x7f76b8140d20 2024-05-28T14:08:06.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.549+0000 7f76b4c00640 1 -- 172.21.15.42:0/2844900206 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f76a00bc070 con 0x7f76b8140d20 2024-05-28T14:08:06.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.549+0000 7f76b4c00640 1 --2- 172.21.15.42:0/2844900206 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f76940694b0 0x7f769406b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:06.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.549+0000 7f76b4c00640 1 -- 172.21.15.42:0/2844900206 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f76a011fb70 con 0x7f76b8140d20 2024-05-28T14:08:06.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.553+0000 7f76bcc00640 1 --2- 172.21.15.42:0/2844900206 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f76940694b0 0x7f769406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.553+0000 7f76bcc00640 1 --2- 172.21.15.42:0/2844900206 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f76940694b0 0x7f769406b970 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f76b0005fd0 tx=0x7f76b0005b80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:06.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:06.565+0000 7f76b4c00640 1 -- 172.21.15.42:0/2844900206 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f76a00c3030 con 0x7f76b8140d20 2024-05-28T14:08:07.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.085+0000 7f76bf600640 1 -- 172.21.15.42:0/2844900206 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone81112", "target": ["mon-mgr", ""]}) -- 0x7f76b804fdd0 con 0x7f76940694b0 2024-05-28T14:08:07.139 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:07.153+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:08:07.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.169+0000 7f76b4c00640 1 -- 172.21.15.42:0/2844900206 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f76b804fdd0 con 0x7f76940694b0 2024-05-28T14:08:07.153 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:08:07.178 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:08:07.178 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone81113 2024-05-28T14:08:07.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.641+0000 7f12b1e00640 1 Processor -- start 2024-05-28T14:08:07.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.641+0000 7f12b1e00640 1 -- start start 2024-05-28T14:08:07.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.641+0000 7f12b1e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f12ac0cf020 0x7f12ac06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:07.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.641+0000 7f12b1e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12ac0b25c0 con 0x7f12ac162fe0 2024-05-28T14:08:07.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.641+0000 7f12b1e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12ac0b28a0 con 0x7f12ac06bf30 2024-05-28T14:08:07.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.641+0000 7f12b1e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f12ac0b2e60 con 0x7f12ac0cf020 2024-05-28T14:08:07.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.645+0000 7f12aa000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f12ac0cf020 0x7f12ac06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:07.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.645+0000 7f12aa000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f12ac0cf020 0x7f12ac06b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:49928/0 (socket says 172.21.15.42:49928) 2024-05-28T14:08:07.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.645+0000 7f12aa000640 1 -- 172.21.15.42:0/1247539698 learned_addr learned my addr 172.21.15.42:0/1247539698 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:07.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.645+0000 7f12aa000640 1 -- 172.21.15.42:0/1247539698 >> v1:172.21.15.63:6789/0 conn(0x7f12ac06bf30 legacy=0x7f12ac0b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:07.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.645+0000 7f12aa000640 1 -- 172.21.15.42:0/1247539698 >> v1:172.21.15.42:6789/0 conn(0x7f12ac162fe0 legacy=0x7f12ac0b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:07.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.645+0000 7f12aa000640 1 -- 172.21.15.42:0/1247539698 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac0b2b80 con 0x7f12ac0cf020 2024-05-28T14:08:07.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.649+0000 7f12aa000640 1 --2- 172.21.15.42:0/1247539698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12ac0cf020 0x7f12ac06b9f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f12980097f0 tx=0x7f129801bfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c9ddd568de5df28 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:07.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.649+0000 7f12a9600640 1 -- 172.21.15.42:0/1247539698 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12980a7640 con 0x7f12ac0cf020 2024-05-28T14:08:07.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.649+0000 7f12a9600640 1 -- 172.21.15.42:0/1247539698 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12980a77e0 con 0x7f12ac0cf020 2024-05-28T14:08:07.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.649+0000 7f12a9600640 1 -- 172.21.15.42:0/1247539698 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12980a7bb0 con 0x7f12ac0cf020 2024-05-28T14:08:07.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.653+0000 7f12b1e00640 1 -- 172.21.15.42:0/1247539698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12ac0cf020 msgr2=0x7f12ac06b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:07.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.653+0000 7f12b1e00640 1 --2- 172.21.15.42:0/1247539698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12ac0cf020 0x7f12ac06b9f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f12980097f0 tx=0x7f129801bfb0 comp rx=0 tx=0).stop 2024-05-28T14:08:07.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.653+0000 7f12b1e00640 1 -- 172.21.15.42:0/1247539698 shutdown_connections 2024-05-28T14:08:07.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.653+0000 7f12b1e00640 1 --2- 172.21.15.42:0/1247539698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12ac0cf020 0x7f12ac06b9f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:07.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.653+0000 7f12b1e00640 1 -- 172.21.15.42:0/1247539698 >> 172.21.15.42:0/1247539698 conn(0x7f12ac09aa50 msgr2=0x7f12ac0b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:07.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.653+0000 7f12b1e00640 1 -- 172.21.15.42:0/1247539698 shutdown_connections 2024-05-28T14:08:07.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.653+0000 7f12b1e00640 1 -- 172.21.15.42:0/1247539698 wait complete. 2024-05-28T14:08:07.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.657+0000 7f12b1e00640 1 Processor -- start 2024-05-28T14:08:07.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.657+0000 7f12b1e00640 1 -- start start 2024-05-28T14:08:07.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12b1e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12ac06bf30 0x7f12ac0cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:07.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12b1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12ac0cf020 0x7f12ac15d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:07.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12b1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac162fe0 0x7f12ac155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:07.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12b1e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac171b30 con 0x7f12ac162fe0 2024-05-28T14:08:07.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12b1e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac1719b0 con 0x7f12ac0cf020 2024-05-28T14:08:07.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12b1e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f12ac171cb0 con 0x7f12ac06bf30 2024-05-28T14:08:07.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12ab400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac162fe0 0x7f12ac155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:07.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12ab400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac162fe0 0x7f12ac155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56056/0 (socket says 172.21.15.42:56056) 2024-05-28T14:08:07.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12ab400640 1 -- 172.21.15.42:0/2034040259 learned_addr learned my addr 172.21.15.42:0/2034040259 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:07.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12ab400640 1 -- 172.21.15.42:0/2034040259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12ac06bf30 msgr2=0x7f12ac0cb050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T14:08:07.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12ab400640 1 --2- 172.21.15.42:0/2034040259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12ac06bf30 0x7f12ac0cb050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:07.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12ab400640 1 -- 172.21.15.42:0/2034040259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12ac0cf020 msgr2=0x7f12ac15d580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T14:08:07.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12ab400640 1 --2- 172.21.15.42:0/2034040259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12ac0cf020 0x7f12ac15d580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:07.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12ab400640 1 -- 172.21.15.42:0/2034040259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac0b5d10 con 0x7f12ac162fe0 2024-05-28T14:08:07.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.661+0000 7f12ab400640 1 --2- 172.21.15.42:0/2034040259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac162fe0 0x7f12ac155bb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f12a400c8e0 tx=0x7f12a400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:07.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.665+0000 7f12a3400640 1 -- 172.21.15.42:0/2034040259 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4017070 con 0x7f12ac162fe0 2024-05-28T14:08:07.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.665+0000 7f12a3400640 1 -- 172.21.15.42:0/2034040259 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12a4005070 con 0x7f12ac162fe0 2024-05-28T14:08:07.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.665+0000 7f12a3400640 1 -- 172.21.15.42:0/2034040259 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4005420 con 0x7f12ac162fe0 2024-05-28T14:08:07.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.665+0000 7f12b1e00640 1 -- 172.21.15.42:0/2034040259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ac15dc40 con 0x7f12ac162fe0 2024-05-28T14:08:07.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.665+0000 7f12b1e00640 1 -- 172.21.15.42:0/2034040259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ac0c95a0 con 0x7f12ac162fe0 2024-05-28T14:08:07.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.665+0000 7f12a3400640 1 -- 172.21.15.42:0/2034040259 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f12a401a070 con 0x7f12ac162fe0 2024-05-28T14:08:07.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.669+0000 7f12a3400640 1 --2- 172.21.15.42:0/2034040259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f128c0694e0 0x7f128c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:07.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.669+0000 7f12a3400640 1 -- 172.21.15.42:0/2034040259 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f12a4085020 con 0x7f12ac162fe0 2024-05-28T14:08:07.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.669+0000 7f12b1e00640 1 -- 172.21.15.42:0/2034040259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12ac0b5b40 con 0x7f12ac162fe0 2024-05-28T14:08:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.681+0000 7f12a3400640 1 -- 172.21.15.42:0/2034040259 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f12a4056af0 con 0x7f12ac162fe0 2024-05-28T14:08:07.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.689+0000 7f12aaa00640 1 --2- 172.21.15.42:0/2034040259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f128c0694e0 0x7f128c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:07.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:07.697+0000 7f12aaa00640 1 --2- 172.21.15.42:0/2034040259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f128c0694e0 0x7f128c06b9a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f12ac1585b0 tx=0x7f129c0079e0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:08.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:08.225+0000 7f12b1e00640 1 -- 172.21.15.42:0/2034040259 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone81113", "target": ["mon-mgr", ""]}) -- 0x7f12ac0b5ee0 con 0x7f128c0694e0 2024-05-28T14:08:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:09.761+0000 7f12a3400640 1 -- 172.21.15.42:0/2034040259 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f12ac0b5ee0 con 0x7f128c0694e0 2024-05-28T14:08:09.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:09.761+0000 7f12a0c00640 1 -- 172.21.15.42:0/2034040259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f128c0694e0 msgr2=0x7f128c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:09.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:09.761+0000 7f12a0c00640 1 --2- 172.21.15.42:0/2034040259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f128c0694e0 0x7f128c06b9a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f12ac1585b0 tx=0x7f129c0079e0 comp rx=0 tx=0).stop 2024-05-28T14:08:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:09.761+0000 7f12a0c00640 1 -- 172.21.15.42:0/2034040259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac162fe0 msgr2=0x7f12ac155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:09.761+0000 7f12a0c00640 1 --2- 172.21.15.42:0/2034040259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac162fe0 0x7f12ac155bb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f12a400c8e0 tx=0x7f12a400cda0 comp rx=0 tx=0).stop 2024-05-28T14:08:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:09.765+0000 7f12a0c00640 1 -- 172.21.15.42:0/2034040259 shutdown_connections 2024-05-28T14:08:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:09.765+0000 7f12a0c00640 1 --2- 172.21.15.42:0/2034040259 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f128c0694e0 0x7f128c06b9a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:09.765+0000 7f12a0c00640 1 --2- 172.21.15.42:0/2034040259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12ac162fe0 0x7f12ac155bb0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:09.765+0000 7f12a0c00640 1 --2- 172.21.15.42:0/2034040259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12ac0cf020 0x7f12ac15d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:09.765+0000 7f12a0c00640 1 --2- 172.21.15.42:0/2034040259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12ac06bf30 0x7f12ac0cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:09.765+0000 7f12a0c00640 1 -- 172.21.15.42:0/2034040259 >> 172.21.15.42:0/2034040259 conn(0x7f12ac09aa50 msgr2=0x7f12ac0b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:09.765+0000 7f12a0c00640 1 -- 172.21.15.42:0/2034040259 shutdown_connections 2024-05-28T14:08:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:09.765+0000 7f12a0c00640 1 -- 172.21.15.42:0/2034040259 wait complete. 2024-05-28T14:08:09.817 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone81114 2024-05-28T14:08:10.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.225+0000 7efff9600640 1 Processor -- start 2024-05-28T14:08:10.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.225+0000 7efff9600640 1 -- start start 2024-05-28T14:08:10.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.229+0000 7efff9600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efff415c8a0 0x7efff40cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:10.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.229+0000 7efff9600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efff40ce090 0x7efff409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:10.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.229+0000 7efff9600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efff409af90 0x7efff4160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:10.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.229+0000 7efff9600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7efff415cc80 con 0x7efff409af90 2024-05-28T14:08:10.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.229+0000 7efff9600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efff40d0280 con 0x7efff40ce090 2024-05-28T14:08:10.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.229+0000 7efff9600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efff40d0560 con 0x7efff415c8a0 2024-05-28T14:08:10.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.229+0000 7efff3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efff409af90 0x7efff4160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:10.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.229+0000 7efff3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efff409af90 0x7efff4160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56058/0 (socket says 172.21.15.42:56058) 2024-05-28T14:08:10.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.229+0000 7efff3400640 1 -- 172.21.15.42:0/1651879981 learned_addr learned my addr 172.21.15.42:0/1651879981 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:10.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.229+0000 7efff3400640 1 -- 172.21.15.42:0/1651879981 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efff415c8a0 msgr2=0x7efff40cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:10.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.229+0000 7efff3400640 1 --2- 172.21.15.42:0/1651879981 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efff415c8a0 0x7efff40cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:10.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.229+0000 7efff3400640 1 -- 172.21.15.42:0/1651879981 >> v2:172.21.15.63:3300/0 conn(0x7efff40ce090 msgr2=0x7efff409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:10.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.229+0000 7efff3400640 1 --2- 172.21.15.42:0/1651879981 >> v2:172.21.15.63:3300/0 conn(0x7efff40ce090 0x7efff409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:10.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.229+0000 7efff3400640 1 -- 172.21.15.42:0/1651879981 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff40d0840 con 0x7efff409af90 2024-05-28T14:08:10.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.233+0000 7efff3400640 1 --2- 172.21.15.42:0/1651879981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff409af90 0x7efff4160380 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7efff4073940 tx=0x7effec020b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4761bcbda9cd92 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:10.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.233+0000 7efff2000640 1 -- 172.21.15.42:0/1651879981 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0a98e0 con 0x7efff409af90 2024-05-28T14:08:10.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.233+0000 7efff2000640 1 -- 172.21.15.42:0/1651879981 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effec0a9a80 con 0x7efff409af90 2024-05-28T14:08:10.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.233+0000 7efff2000640 1 -- 172.21.15.42:0/1651879981 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0a9e50 con 0x7efff409af90 2024-05-28T14:08:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.237+0000 7efff9600640 1 -- 172.21.15.42:0/1651879981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff409af90 msgr2=0x7efff4160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.237+0000 7efff9600640 1 --2- 172.21.15.42:0/1651879981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff409af90 0x7efff4160380 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7efff4073940 tx=0x7effec020b40 comp rx=0 tx=0).stop 2024-05-28T14:08:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.237+0000 7efff9600640 1 -- 172.21.15.42:0/1651879981 shutdown_connections 2024-05-28T14:08:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.237+0000 7efff9600640 1 --2- 172.21.15.42:0/1651879981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff409af90 0x7efff4160380 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.237+0000 7efff9600640 1 --2- 172.21.15.42:0/1651879981 >> v2:172.21.15.63:3300/0 conn(0x7efff40ce090 0x7efff409aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.237+0000 7efff9600640 1 --2- 172.21.15.42:0/1651879981 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efff415c8a0 0x7efff40cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.237+0000 7efff9600640 1 -- 172.21.15.42:0/1651879981 >> 172.21.15.42:0/1651879981 conn(0x7efff40b3f90 msgr2=0x7efff40b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.237+0000 7efff9600640 1 -- 172.21.15.42:0/1651879981 shutdown_connections 2024-05-28T14:08:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.237+0000 7efff9600640 1 -- 172.21.15.42:0/1651879981 wait complete. 2024-05-28T14:08:10.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.241+0000 7efff9600640 1 Processor -- start 2024-05-28T14:08:10.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.241+0000 7efff9600640 1 -- start start 2024-05-28T14:08:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.241+0000 7efff9600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efff409af90 0x7efff40c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.241+0000 7efff9600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efff40ce090 0x7efff41f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.241+0000 7efff9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff415c8a0 0x7efff414f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.241+0000 7efff9600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efff406eb10 con 0x7efff415c8a0 2024-05-28T14:08:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.241+0000 7efff9600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efff40d1530 con 0x7efff40ce090 2024-05-28T14:08:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.241+0000 7efff9600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efff40b52f0 con 0x7efff409af90 2024-05-28T14:08:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.245+0000 7efff3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff415c8a0 0x7efff414f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.245+0000 7efff3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff415c8a0 0x7efff414f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56060/0 (socket says 172.21.15.42:56060) 2024-05-28T14:08:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.245+0000 7efff3400640 1 -- 172.21.15.42:0/2758369569 learned_addr learned my addr 172.21.15.42:0/2758369569 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:10.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.245+0000 7efff2a00640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efff409af90 0x7efff40c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:10.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.245+0000 7effea000640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efff40ce090 0x7efff41f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:10.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.245+0000 7efff3400640 1 -- 172.21.15.42:0/2758369569 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efff409af90 msgr2=0x7efff40c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:10.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.245+0000 7efff3400640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efff409af90 0x7efff40c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:10.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.245+0000 7efff3400640 1 -- 172.21.15.42:0/2758369569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efff40ce090 msgr2=0x7efff41f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:10.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.245+0000 7efff3400640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efff40ce090 0x7efff41f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:10.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.245+0000 7efff3400640 1 -- 172.21.15.42:0/2758369569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff40ca9c0 con 0x7efff415c8a0 2024-05-28T14:08:10.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.245+0000 7efff3400640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff415c8a0 0x7efff414f7d0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7effec01f1f0 tx=0x7effec01d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:10.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.245+0000 7efff2a00640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efff409af90 0x7efff40c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:08:10.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.249+0000 7effeb400640 1 -- 172.21.15.42:0/2758369569 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0c0020 con 0x7efff415c8a0 2024-05-28T14:08:10.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.249+0000 7effeb400640 1 -- 172.21.15.42:0/2758369569 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effec0bc470 con 0x7efff415c8a0 2024-05-28T14:08:10.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.249+0000 7effeb400640 1 -- 172.21.15.42:0/2758369569 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0a98e0 con 0x7efff415c8a0 2024-05-28T14:08:10.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.249+0000 7efff9600640 1 -- 172.21.15.42:0/2758369569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff40c8d10 con 0x7efff415c8a0 2024-05-28T14:08:10.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.249+0000 7efff9600640 1 -- 172.21.15.42:0/2758369569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff40cb180 con 0x7efff415c8a0 2024-05-28T14:08:10.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.253+0000 7efff9600640 1 -- 172.21.15.42:0/2758369569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff40c8a70 con 0x7efff415c8a0 2024-05-28T14:08:10.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.253+0000 7effeb400640 1 -- 172.21.15.42:0/2758369569 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7effec0c3020 con 0x7efff415c8a0 2024-05-28T14:08:10.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.257+0000 7effeb400640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7effd00694b0 0x7effd006b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:10.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.261+0000 7effeb400640 1 -- 172.21.15.42:0/2758369569 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7effec0efa10 con 0x7efff415c8a0 2024-05-28T14:08:10.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.273+0000 7effeb400640 1 -- 172.21.15.42:0/2758369569 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7effec01a9f0 con 0x7efff415c8a0 2024-05-28T14:08:10.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.281+0000 7efff2a00640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7effd00694b0 0x7effd006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:10.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.313+0000 7efff2a00640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7effd00694b0 0x7effd006b970 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7effdc00a870 tx=0x7effdc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:10.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:10.749+0000 7efff9600640 1 -- 172.21.15.42:0/2758369569 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone81114", "target": ["mon-mgr", ""]}) -- 0x7efff404fdd0 con 0x7effd00694b0 2024-05-28T14:08:11.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:11.761+0000 7effeb400640 1 -- 172.21.15.42:0/2758369569 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efff404fdd0 con 0x7effd00694b0 2024-05-28T14:08:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:11.769+0000 7efff9600640 1 -- 172.21.15.42:0/2758369569 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7effd00694b0 msgr2=0x7effd006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:11.769+0000 7efff9600640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7effd00694b0 0x7effd006b970 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7effdc00a870 tx=0x7effdc008040 comp rx=0 tx=0).stop 2024-05-28T14:08:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:11.769+0000 7efff9600640 1 -- 172.21.15.42:0/2758369569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff415c8a0 msgr2=0x7efff414f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:11.769+0000 7efff9600640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff415c8a0 0x7efff414f7d0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7effec01f1f0 tx=0x7effec01d8e0 comp rx=0 tx=0).stop 2024-05-28T14:08:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:11.769+0000 7efff9600640 1 -- 172.21.15.42:0/2758369569 shutdown_connections 2024-05-28T14:08:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:11.769+0000 7efff9600640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7effd00694b0 0x7effd006b970 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:11.769+0000 7efff9600640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff415c8a0 0x7efff414f7d0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:11.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:11.769+0000 7efff9600640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efff40ce090 0x7efff41f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:11.769+0000 7efff9600640 1 --2- 172.21.15.42:0/2758369569 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efff409af90 0x7efff40c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:11.769+0000 7efff9600640 1 -- 172.21.15.42:0/2758369569 >> 172.21.15.42:0/2758369569 conn(0x7efff40b3f90 msgr2=0x7efff40b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:11.773+0000 7efff9600640 1 -- 172.21.15.42:0/2758369569 shutdown_connections 2024-05-28T14:08:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:11.773+0000 7efff9600640 1 -- 172.21.15.42:0/2758369569 wait complete. 2024-05-28T14:08:11.849 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone81115 2024-05-28T14:08:12.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.273+0000 7fe243400640 1 Processor -- start 2024-05-28T14:08:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.277+0000 7fe243400640 1 -- start start 2024-05-28T14:08:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.277+0000 7fe243400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe23c154dd0 0x7fe23c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.277+0000 7fe243400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe23c15ba70 0x7fe23c06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.277+0000 7fe243400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe23c072710 con 0x7fe23c06f800 2024-05-28T14:08:12.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.277+0000 7fe243400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe23c15e330 con 0x7fe23c15ba70 2024-05-28T14:08:12.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.277+0000 7fe243400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe23c15e610 con 0x7fe23c154dd0 2024-05-28T14:08:12.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.281+0000 7fe240a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe23c154dd0 0x7fe23c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:12.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.281+0000 7fe240a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe23c154dd0 0x7fe23c071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:39838/0 (socket says 172.21.15.42:39838) 2024-05-28T14:08:12.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.281+0000 7fe240a00640 1 -- 172.21.15.42:0/1103441289 learned_addr learned my addr 172.21.15.42:0/1103441289 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:12.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.281+0000 7fe23be00640 1 --2- 172.21.15.42:0/1103441289 >> v2:172.21.15.63:3300/0 conn(0x7fe23c15ba70 0x7fe23c06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:12.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.285+0000 7fe240a00640 1 -- 172.21.15.42:0/1103441289 >> v2:172.21.15.63:3300/0 conn(0x7fe23c15ba70 msgr2=0x7fe23c06f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:12.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.285+0000 7fe240a00640 1 --2- 172.21.15.42:0/1103441289 >> v2:172.21.15.63:3300/0 conn(0x7fe23c15ba70 0x7fe23c06f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:12.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.285+0000 7fe240a00640 1 -- 172.21.15.42:0/1103441289 >> v1:172.21.15.42:6789/0 conn(0x7fe23c06f800 legacy=0x7fe23c06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:12.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.285+0000 7fe240a00640 1 -- 172.21.15.42:0/1103441289 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23c15e8f0 con 0x7fe23c154dd0 2024-05-28T14:08:12.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.297+0000 7fe240a00640 1 --2- 172.21.15.42:0/1103441289 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe23c154dd0 0x7fe23c071de0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c110 tx=0x7fe23401cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5021207c9d347247 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:12.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.297+0000 7fe23b400640 1 -- 172.21.15.42:0/1103441289 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2340ab070 con 0x7fe23c154dd0 2024-05-28T14:08:12.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.297+0000 7fe23b400640 1 -- 172.21.15.42:0/1103441289 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2340a5b60 con 0x7fe23c154dd0 2024-05-28T14:08:12.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.297+0000 7fe23b400640 1 -- 172.21.15.42:0/1103441289 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23400be60 con 0x7fe23c154dd0 2024-05-28T14:08:12.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.297+0000 7fe243400640 1 -- 172.21.15.42:0/1103441289 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe23c154dd0 msgr2=0x7fe23c071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:12.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.297+0000 7fe243400640 1 --2- 172.21.15.42:0/1103441289 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe23c154dd0 0x7fe23c071de0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c110 tx=0x7fe23401cc00 comp rx=0 tx=0).stop 2024-05-28T14:08:12.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.301+0000 7fe243400640 1 -- 172.21.15.42:0/1103441289 shutdown_connections 2024-05-28T14:08:12.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.301+0000 7fe243400640 1 --2- 172.21.15.42:0/1103441289 >> v2:172.21.15.63:3300/0 conn(0x7fe23c15ba70 0x7fe23c06f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:12.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.301+0000 7fe243400640 1 --2- 172.21.15.42:0/1103441289 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe23c154dd0 0x7fe23c071de0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:12.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.301+0000 7fe243400640 1 -- 172.21.15.42:0/1103441289 >> 172.21.15.42:0/1103441289 conn(0x7fe23c0b21f0 msgr2=0x7fe23c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:12.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.301+0000 7fe243400640 1 -- 172.21.15.42:0/1103441289 shutdown_connections 2024-05-28T14:08:12.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.301+0000 7fe243400640 1 -- 172.21.15.42:0/1103441289 wait complete. 2024-05-28T14:08:12.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.313+0000 7fe243400640 1 Processor -- start 2024-05-28T14:08:12.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.313+0000 7fe243400640 1 -- start start 2024-05-28T14:08:12.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.321+0000 7fe243400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe23c06f800 0x7fe23c0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:12.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.325+0000 7fe243400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe23c154dd0 0x7fe23c0b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:12.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.325+0000 7fe243400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe23c15ba70 0x7fe23c156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:12.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.325+0000 7fe243400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c15f0f0 con 0x7fe23c06f800 2024-05-28T14:08:12.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.325+0000 7fe243400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c072320 con 0x7fe23c15ba70 2024-05-28T14:08:12.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.325+0000 7fe243400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe23c0b3da0 con 0x7fe23c154dd0 2024-05-28T14:08:12.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.325+0000 7fe240a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe23c06f800 0x7fe23c0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:12.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.325+0000 7fe240a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe23c06f800 0x7fe23c0b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56068/0 (socket says 172.21.15.42:56068) 2024-05-28T14:08:12.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.325+0000 7fe240a00640 1 -- 172.21.15.42:0/829979301 learned_addr learned my addr 172.21.15.42:0/829979301 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:12.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.325+0000 7fe23be00640 1 --2- 172.21.15.42:0/829979301 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe23c154dd0 0x7fe23c0b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:12.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.329+0000 7fe23be00640 1 -- 172.21.15.42:0/829979301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe23c15ba70 msgr2=0x7fe23c156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:12.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.329+0000 7fe23be00640 1 --2- 172.21.15.42:0/829979301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe23c15ba70 0x7fe23c156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:12.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.329+0000 7fe23be00640 1 -- 172.21.15.42:0/829979301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe23c06f800 msgr2=0x7fe23c0b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:12.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.329+0000 7fe23be00640 1 --2- 172.21.15.42:0/829979301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe23c06f800 0x7fe23c0b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:12.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.329+0000 7fe23be00640 1 -- 172.21.15.42:0/829979301 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23c0b4080 con 0x7fe23c154dd0 2024-05-28T14:08:12.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.333+0000 7fe23be00640 1 --2- 172.21.15.42:0/829979301 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe23c154dd0 0x7fe23c0b32e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00baf0 tx=0x7fe22c00df80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:12.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.333+0000 7fe240a00640 1 --2- 172.21.15.42:0/829979301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe23c06f800 0x7fe23c0b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:08:12.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.333+0000 7fe238c00640 1 -- 172.21.15.42:0/829979301 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c01b070 con 0x7fe23c154dd0 2024-05-28T14:08:12.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.333+0000 7fe238c00640 1 -- 172.21.15.42:0/829979301 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe22c00f950 con 0x7fe23c154dd0 2024-05-28T14:08:12.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.333+0000 7fe238c00640 1 -- 172.21.15.42:0/829979301 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c00fd00 con 0x7fe23c154dd0 2024-05-28T14:08:12.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.333+0000 7fe243400640 1 -- 172.21.15.42:0/829979301 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23c0b4360 con 0x7fe23c154dd0 2024-05-28T14:08:12.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.333+0000 7fe243400640 1 -- 172.21.15.42:0/829979301 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23c15cf90 con 0x7fe23c154dd0 2024-05-28T14:08:12.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.337+0000 7fe232000640 1 -- 172.21.15.42:0/829979301 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe20c0052a0 con 0x7fe23c154dd0 2024-05-28T14:08:12.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.337+0000 7fe238c00640 1 -- 172.21.15.42:0/829979301 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7fe22c01e070 con 0x7fe23c154dd0 2024-05-28T14:08:12.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.341+0000 7fe238c00640 1 --2- 172.21.15.42:0/829979301 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe218069460 0x7fe21806b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:12.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.341+0000 7fe238c00640 1 -- 172.21.15.42:0/829979301 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fe22c085cc0 con 0x7fe23c154dd0 2024-05-28T14:08:12.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.345+0000 7fe240a00640 1 --2- 172.21.15.42:0/829979301 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe218069460 0x7fe21806b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:12.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.345+0000 7fe240a00640 1 --2- 172.21.15.42:0/829979301 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe218069460 0x7fe21806b920 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c110 tx=0x7fe234004ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:12.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.349+0000 7fe238c00640 1 -- 172.21.15.42:0/829979301 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe22c057340 con 0x7fe23c154dd0 2024-05-28T14:08:12.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:12.889+0000 7fe232000640 1 -- 172.21.15.42:0/829979301 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone81115", "target": ["mon-mgr", ""]}) -- 0x7fe20c000bd0 con 0x7fe218069460 2024-05-28T14:08:14.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:14.709+0000 7fe238c00640 1 -- 172.21.15.42:0/829979301 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe20c000bd0 con 0x7fe218069460 2024-05-28T14:08:14.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:14.717+0000 7fe232000640 1 -- 172.21.15.42:0/829979301 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe218069460 msgr2=0x7fe21806b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:14.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:14.717+0000 7fe232000640 1 --2- 172.21.15.42:0/829979301 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe218069460 0x7fe21806b920 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c110 tx=0x7fe234004ac0 comp rx=0 tx=0).stop 2024-05-28T14:08:14.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:14.717+0000 7fe232000640 1 -- 172.21.15.42:0/829979301 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe23c154dd0 msgr2=0x7fe23c0b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:14.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:14.717+0000 7fe232000640 1 --2- 172.21.15.42:0/829979301 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe23c154dd0 0x7fe23c0b32e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00baf0 tx=0x7fe22c00df80 comp rx=0 tx=0).stop 2024-05-28T14:08:14.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:14.717+0000 7fe232000640 1 -- 172.21.15.42:0/829979301 shutdown_connections 2024-05-28T14:08:14.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:14.717+0000 7fe232000640 1 --2- 172.21.15.42:0/829979301 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe218069460 0x7fe21806b920 secure :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c110 tx=0x7fe234004ac0 comp rx=0 tx=0).stop 2024-05-28T14:08:14.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:14.717+0000 7fe232000640 1 --2- 172.21.15.42:0/829979301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe23c15ba70 0x7fe23c156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:14.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:14.717+0000 7fe232000640 1 --2- 172.21.15.42:0/829979301 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe23c154dd0 0x7fe23c0b32e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:14.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:14.717+0000 7fe232000640 1 --2- 172.21.15.42:0/829979301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe23c06f800 0x7fe23c0b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:14.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:14.717+0000 7fe232000640 1 -- 172.21.15.42:0/829979301 >> 172.21.15.42:0/829979301 conn(0x7fe23c0b21f0 msgr2=0x7fe23c14dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:14.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:14.717+0000 7fe232000640 1 -- 172.21.15.42:0/829979301 shutdown_connections 2024-05-28T14:08:14.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:14.721+0000 7fe232000640 1 -- 172.21.15.42:0/829979301 wait complete. 2024-05-28T14:08:14.760 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone81116 2024-05-28T14:08:15.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.161+0000 7fcdb3e00640 1 Processor -- start 2024-05-28T14:08:15.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb3e00640 1 -- start start 2024-05-28T14:08:15.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcdb415e9d0 0x7fcdb40ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:15.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdb40b3860 0x7fcdb40ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:15.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb3e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcdb40aead0 0x7fcdb40afa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:15.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb3e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fcdb4070000 con 0x7fcdb415e9d0 2024-05-28T14:08:15.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb3e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcdb40cf7c0 con 0x7fcdb40aead0 2024-05-28T14:08:15.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb3e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb4070270 con 0x7fcdb40b3860 2024-05-28T14:08:15.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb2000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdb40b3860 0x7fcdb40ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:15.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb2000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdb40b3860 0x7fcdb40ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:39858/0 (socket says 172.21.15.42:39858) 2024-05-28T14:08:15.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb2000640 1 -- 172.21.15.42:0/358781541 learned_addr learned my addr 172.21.15.42:0/358781541 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:15.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb3400640 1 --2- 172.21.15.42:0/358781541 >> v2:172.21.15.63:3300/0 conn(0x7fcdb40aead0 0x7fcdb40afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:15.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb2000640 1 -- 172.21.15.42:0/358781541 >> v2:172.21.15.63:3300/0 conn(0x7fcdb40aead0 msgr2=0x7fcdb40afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:15.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb2000640 1 --2- 172.21.15.42:0/358781541 >> v2:172.21.15.63:3300/0 conn(0x7fcdb40aead0 0x7fcdb40afa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:15.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb2000640 1 -- 172.21.15.42:0/358781541 >> v2:172.21.15.42:3300/0 conn(0x7fcdb415e9d0 msgr2=0x7fcdb40ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:15.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb2000640 1 --2- 172.21.15.42:0/358781541 >> v2:172.21.15.42:3300/0 conn(0x7fcdb415e9d0 0x7fcdb40ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:15.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.165+0000 7fcdb2000640 1 -- 172.21.15.42:0/358781541 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb4070550 con 0x7fcdb40b3860 2024-05-28T14:08:15.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.169+0000 7fcdb3400640 1 --2- 172.21.15.42:0/358781541 >> v2:172.21.15.63:3300/0 conn(0x7fcdb40aead0 0x7fcdb40afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:08:15.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.169+0000 7fcdb2000640 1 --2- 172.21.15.42:0/358781541 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdb40b3860 0x7fcdb40ae590 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40b7330 tx=0x7fcda001c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83464b72eaf0bf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:15.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.173+0000 7fcdb1600640 1 -- 172.21.15.42:0/358781541 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda00b0070 con 0x7fcdb40b3860 2024-05-28T14:08:15.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.173+0000 7fcdb1600640 1 -- 172.21.15.42:0/358781541 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda00ab600 con 0x7fcdb40b3860 2024-05-28T14:08:15.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.173+0000 7fcdb1600640 1 -- 172.21.15.42:0/358781541 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda00ab9d0 con 0x7fcdb40b3860 2024-05-28T14:08:15.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.173+0000 7fcdb3e00640 1 -- 172.21.15.42:0/358781541 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdb40b3860 msgr2=0x7fcdb40ae590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:15.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.173+0000 7fcdb3e00640 1 --2- 172.21.15.42:0/358781541 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdb40b3860 0x7fcdb40ae590 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40b7330 tx=0x7fcda001c490 comp rx=0 tx=0).stop 2024-05-28T14:08:15.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.173+0000 7fcdb3e00640 1 -- 172.21.15.42:0/358781541 shutdown_connections 2024-05-28T14:08:15.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.173+0000 7fcdb3e00640 1 --2- 172.21.15.42:0/358781541 >> v2:172.21.15.63:3300/0 conn(0x7fcdb40aead0 0x7fcdb40afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:15.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.173+0000 7fcdb3e00640 1 --2- 172.21.15.42:0/358781541 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdb40b3860 0x7fcdb40ae590 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:15.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.173+0000 7fcdb3e00640 1 --2- 172.21.15.42:0/358781541 >> v2:172.21.15.42:3300/0 conn(0x7fcdb415e9d0 0x7fcdb40ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:15.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.173+0000 7fcdb3e00640 1 -- 172.21.15.42:0/358781541 >> 172.21.15.42:0/358781541 conn(0x7fcdb40b2d70 msgr2=0x7fcdb40b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:15.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.173+0000 7fcdb3e00640 1 -- 172.21.15.42:0/358781541 shutdown_connections 2024-05-28T14:08:15.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.173+0000 7fcdb3e00640 1 -- 172.21.15.42:0/358781541 wait complete. 2024-05-28T14:08:15.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.177+0000 7fcdb3e00640 1 Processor -- start 2024-05-28T14:08:15.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.177+0000 7fcdb3e00640 1 -- start start 2024-05-28T14:08:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.177+0000 7fcdb3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdb40aead0 0x7fcdb40c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.177+0000 7fcdb3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdb40b3860 0x7fcdb4156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.177+0000 7fcdb3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb415e9d0 0x7fcdb40c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.177+0000 7fcdb3e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb4168240 con 0x7fcdb415e9d0 2024-05-28T14:08:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.177+0000 7fcdb3e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb415efc0 con 0x7fcdb40aead0 2024-05-28T14:08:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.177+0000 7fcdb3e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb40b07c0 con 0x7fcdb40b3860 2024-05-28T14:08:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.181+0000 7fcdb2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdb40aead0 0x7fcdb40c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.181+0000 7fcdb2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdb40aead0 0x7fcdb40c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:50002/0 (socket says 172.21.15.42:50002) 2024-05-28T14:08:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.181+0000 7fcdb2a00640 1 -- 172.21.15.42:0/1651876085 learned_addr learned my addr 172.21.15.42:0/1651876085 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.181+0000 7fcdb2000640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdb40b3860 0x7fcdb4156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.181+0000 7fcdb3400640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb415e9d0 0x7fcdb40c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.181+0000 7fcdb2a00640 1 -- 172.21.15.42:0/1651876085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdb40b3860 msgr2=0x7fcdb4156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.181+0000 7fcdb2a00640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdb40b3860 0x7fcdb4156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.181+0000 7fcdb2a00640 1 -- 172.21.15.42:0/1651876085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb415e9d0 msgr2=0x7fcdb40c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.181+0000 7fcdb2a00640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb415e9d0 0x7fcdb40c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:15.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.181+0000 7fcdb2a00640 1 -- 172.21.15.42:0/1651876085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb40c5910 con 0x7fcdb40aead0 2024-05-28T14:08:15.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.181+0000 7fcdb2000640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdb40b3860 0x7fcdb4156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:08:15.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.181+0000 7fcdb2a00640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdb40aead0 0x7fcdb40c53d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcda400b440 tx=0x7fcda400b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:15.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.185+0000 7fcdaaa00640 1 -- 172.21.15.42:0/1651876085 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4015020 con 0x7fcdb40aead0 2024-05-28T14:08:15.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.185+0000 7fcdaaa00640 1 -- 172.21.15.42:0/1651876085 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda4011850 con 0x7fcdb40aead0 2024-05-28T14:08:15.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.185+0000 7fcdaaa00640 1 -- 172.21.15.42:0/1651876085 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4011c40 con 0x7fcdb40aead0 2024-05-28T14:08:15.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.185+0000 7fcdb3e00640 1 -- 172.21.15.42:0/1651876085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb40c49a0 con 0x7fcdb40aead0 2024-05-28T14:08:15.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.185+0000 7fcdb3e00640 1 -- 172.21.15.42:0/1651876085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb40c4f60 con 0x7fcdb40aead0 2024-05-28T14:08:15.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.189+0000 7fcdb3e00640 1 -- 172.21.15.42:0/1651876085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb404fcf0 con 0x7fcdb40aead0 2024-05-28T14:08:15.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.201+0000 7fcdaaa00640 1 -- 172.21.15.42:0/1651876085 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7fcda4018020 con 0x7fcdb40aead0 2024-05-28T14:08:15.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.201+0000 7fcdaaa00640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcd94069530 0x7fcd9406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:15.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.201+0000 7fcdaaa00640 1 -- 172.21.15.42:0/1651876085 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fcda40853a0 con 0x7fcdb40aead0 2024-05-28T14:08:15.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.209+0000 7fcdaaa00640 1 -- 172.21.15.42:0/1651876085 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcda401f020 con 0x7fcdb40aead0 2024-05-28T14:08:15.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.253+0000 7fcdb2000640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcd94069530 0x7fcd9406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:15.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.317+0000 7fcdb2000640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcd94069530 0x7fcd9406b9f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fcda00ad000 tx=0x7fcda00a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:15.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:15.705+0000 7fcdb3e00640 1 -- 172.21.15.42:0/1651876085 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone81116", "target": ["mon-mgr", ""]}) -- 0x7fcdb4070830 con 0x7fcd94069530 2024-05-28T14:08:17.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.257+0000 7fcdaaa00640 1 -- 172.21.15.42:0/1651876085 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcdb4070830 con 0x7fcd94069530 2024-05-28T14:08:17.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.261+0000 7fcdb3e00640 1 -- 172.21.15.42:0/1651876085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcd94069530 msgr2=0x7fcd9406b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:17.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.261+0000 7fcdb3e00640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcd94069530 0x7fcd9406b9f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fcda00ad000 tx=0x7fcda00a6000 comp rx=0 tx=0).stop 2024-05-28T14:08:17.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.261+0000 7fcdb3e00640 1 -- 172.21.15.42:0/1651876085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdb40aead0 msgr2=0x7fcdb40c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:17.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.261+0000 7fcdb3e00640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdb40aead0 0x7fcdb40c53d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcda400b440 tx=0x7fcda400b900 comp rx=0 tx=0).stop 2024-05-28T14:08:17.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.261+0000 7fcdb3e00640 1 -- 172.21.15.42:0/1651876085 shutdown_connections 2024-05-28T14:08:17.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.261+0000 7fcdb3e00640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcd94069530 0x7fcd9406b9f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:17.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.261+0000 7fcdb3e00640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb415e9d0 0x7fcdb40c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:17.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.261+0000 7fcdb3e00640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdb40b3860 0x7fcdb4156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:17.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.261+0000 7fcdb3e00640 1 --2- 172.21.15.42:0/1651876085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdb40aead0 0x7fcdb40c53d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:17.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.261+0000 7fcdb3e00640 1 -- 172.21.15.42:0/1651876085 >> 172.21.15.42:0/1651876085 conn(0x7fcdb40b2d70 msgr2=0x7fcdb40b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:17.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.261+0000 7fcdb3e00640 1 -- 172.21.15.42:0/1651876085 shutdown_connections 2024-05-28T14:08:17.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.261+0000 7fcdb3e00640 1 -- 172.21.15.42:0/1651876085 wait complete. 2024-05-28T14:08:17.268 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone81117 2024-05-28T14:08:17.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.693+0000 7f6e0ec00640 1 Processor -- start 2024-05-28T14:08:17.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.693+0000 7f6e0ec00640 1 -- start start 2024-05-28T14:08:17.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.693+0000 7f6e0ec00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6e080b3990 0x7f6e080b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:17.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.693+0000 7f6e0ec00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e080d2c20 con 0x7f6e080b6540 2024-05-28T14:08:17.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.693+0000 7f6e0ec00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e080b78e0 con 0x7f6e080b6160 2024-05-28T14:08:17.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.693+0000 7f6e0ec00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6e080b7c10 con 0x7f6e080b3990 2024-05-28T14:08:17.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.697+0000 7f6e0e200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f6e080b6540 0x7f6e080b85e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47520/0 (socket says 172.21.15.42:47520) 2024-05-28T14:08:17.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.697+0000 7f6e0e200640 1 -- 172.21.15.42:0/1339521174 learned_addr learned my addr 172.21.15.42:0/1339521174 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:08:17.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.697+0000 7f6e0ce00640 1 --2- 172.21.15.42:0/1339521174 >> v2:172.21.15.42:3300/0 conn(0x7f6e080b3990 0x7f6e080b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:17.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.701+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2464373071 0 0) 0x7f6e080b78e0 con 0x7f6e080b6160 2024-05-28T14:08:17.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.701+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6df0003530 con 0x7f6e080b6160 2024-05-28T14:08:17.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.705+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1982710555 0 0) 0x7f6e080d2c20 con 0x7f6e080b6540 2024-05-28T14:08:17.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.705+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e080b78e0 con 0x7f6e080b6540 2024-05-28T14:08:17.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.705+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2742914707 0 0) 0x7f6df0003530 con 0x7f6e080b6160 2024-05-28T14:08:17.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.705+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 >> v2:172.21.15.42:3300/0 conn(0x7f6e080b3990 msgr2=0x7f6e080b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:17.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.705+0000 7f6e07e00640 1 --2- 172.21.15.42:0/1339521174 >> v2:172.21.15.42:3300/0 conn(0x7f6e080b3990 0x7f6e080b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:17.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.705+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 >> v1:172.21.15.42:6789/0 conn(0x7f6e080b6540 legacy=0x7f6e080b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:17.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.705+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e080d2ec0 con 0x7f6e080b6160 2024-05-28T14:08:17.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.705+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f6dfc001680 con 0x7f6e080b6160 2024-05-28T14:08:17.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.705+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 >> v1:172.21.15.63:6789/0 conn(0x7f6e080b6160 legacy=0x7f6e080cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:17.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.705+0000 7f6e07e00640 1 --2- 172.21.15.42:0/1339521174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6df0003b50 0x7f6df0003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:17.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.705+0000 7f6e07e00640 1 --2- 172.21.15.42:0/1339521174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6df0004810 0x7f6df0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:17.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.709+0000 7f6e07e00640 1 --2- 172.21.15.42:0/1339521174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6df00051d0 0x7f6df000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:17.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.709+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6df000c020 con 0x7f6df0003b50 2024-05-28T14:08:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.709+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6df000c1d0 con 0x7f6df0004810 2024-05-28T14:08:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.709+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6df000c3d0 con 0x7f6df00051d0 2024-05-28T14:08:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.709+0000 7f6e0e200640 1 --2- 172.21.15.42:0/1339521174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6df00051d0 0x7f6df000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.713+0000 7f6e0d800640 1 --2- 172.21.15.42:0/1339521174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6df0003b50 0x7f6df0003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:17.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.713+0000 7f6e0ce00640 1 --2- 172.21.15.42:0/1339521174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6df0004810 0x7f6df0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:17.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.713+0000 7f6e0e200640 1 -- 172.21.15.42:0/1339521174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6df0004810 msgr2=0x7f6df0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:17.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.713+0000 7f6e0e200640 1 --2- 172.21.15.42:0/1339521174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6df0004810 0x7f6df0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:17.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.713+0000 7f6e0e200640 1 -- 172.21.15.42:0/1339521174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6df0003b50 msgr2=0x7f6df0003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:17.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.713+0000 7f6e0e200640 1 --2- 172.21.15.42:0/1339521174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6df0003b50 0x7f6df0003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:17.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.713+0000 7f6e0e200640 1 -- 172.21.15.42:0/1339521174 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e080d2ec0 con 0x7f6df00051d0 2024-05-28T14:08:17.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.729+0000 7f6e0e200640 1 -- 172.21.15.42:0/1339521174 reap_dead start 2024-05-28T14:08:17.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.737+0000 7f6e0e200640 1 --2- 172.21.15.42:0/1339521174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6df00051d0 0x7f6df000b9d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000c320 tx=0x7f6e0001cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ddea584711102de server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:17.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.741+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00002060 con 0x7f6df00051d0 2024-05-28T14:08:17.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.741+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e000a9070 con 0x7f6df00051d0 2024-05-28T14:08:17.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.741+0000 7f6e07e00640 1 -- 172.21.15.42:0/1339521174 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e000a5c40 con 0x7f6df00051d0 2024-05-28T14:08:17.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.745+0000 7f6e0ec00640 1 -- 172.21.15.42:0/1339521174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6df00051d0 msgr2=0x7f6df000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:17.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.745+0000 7f6e0ec00640 1 --2- 172.21.15.42:0/1339521174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6df00051d0 0x7f6df000b9d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000c320 tx=0x7f6e0001cd80 comp rx=0 tx=0).stop 2024-05-28T14:08:17.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.749+0000 7f6e0ec00640 1 -- 172.21.15.42:0/1339521174 shutdown_connections 2024-05-28T14:08:17.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.749+0000 7f6e0ec00640 1 --2- 172.21.15.42:0/1339521174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6df00051d0 0x7f6df000b9d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:17.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.749+0000 7f6e0ec00640 1 -- 172.21.15.42:0/1339521174 >> 172.21.15.42:0/1339521174 conn(0x7f6e080d0f00 msgr2=0x7f6e080d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:17.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.749+0000 7f6e0ec00640 1 -- 172.21.15.42:0/1339521174 shutdown_connections 2024-05-28T14:08:17.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.749+0000 7f6e0ec00640 1 -- 172.21.15.42:0/1339521174 wait complete. 2024-05-28T14:08:17.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.765+0000 7f6e0ec00640 1 Processor -- start 2024-05-28T14:08:17.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.769+0000 7f6e0ec00640 1 -- start start 2024-05-28T14:08:17.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.777+0000 7f6e0ec00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6df00051d0 0x7f6e080cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:17.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.785+0000 7f6e0ec00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e080ccbd0 0x7f6e0815d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:17.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.785+0000 7f6e0ec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e0815dd70 0x7f6e080b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:17.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.785+0000 7f6e0ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e080ccbd0 0x7f6e0815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:17.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.785+0000 7f6e0d800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6df00051d0 0x7f6e080cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:17.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.785+0000 7f6e0d800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6df00051d0 0x7f6e080cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:58770/0 (socket says 172.21.15.42:58770) 2024-05-28T14:08:17.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.793+0000 7f6e0ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e080ccbd0 0x7f6e0815d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:60962/0 (socket says 172.21.15.42:60962) 2024-05-28T14:08:17.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.793+0000 7f6e0ec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6e080cf1f0 con 0x7f6e0815dd70 2024-05-28T14:08:17.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.793+0000 7f6e0ec00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6e080cf070 con 0x7f6e080ccbd0 2024-05-28T14:08:17.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.793+0000 7f6e0ec00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6e080cf370 con 0x7f6df00051d0 2024-05-28T14:08:17.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.797+0000 7f6e0d800640 1 -- 172.21.15.42:0/3626809239 learned_addr learned my addr 172.21.15.42:0/3626809239 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:17.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.797+0000 7f6e0e200640 1 --2- 172.21.15.42:0/3626809239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e0815dd70 0x7f6e080b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:17.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.797+0000 7f6e0d800640 1 -- 172.21.15.42:0/3626809239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e080ccbd0 msgr2=0x7f6e0815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:17.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.797+0000 7f6e0d800640 1 --2- 172.21.15.42:0/3626809239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e080ccbd0 0x7f6e0815d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:17.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.797+0000 7f6e0d800640 1 -- 172.21.15.42:0/3626809239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e0815dd70 msgr2=0x7f6e080b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:17.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.797+0000 7f6e0d800640 1 --2- 172.21.15.42:0/3626809239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e0815dd70 0x7f6e080b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:17.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.797+0000 7f6e0d800640 1 -- 172.21.15.42:0/3626809239 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e080cd170 con 0x7f6df00051d0 2024-05-28T14:08:17.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.797+0000 7f6e0ce00640 1 --2- 172.21.15.42:0/3626809239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e080ccbd0 0x7f6e0815d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:08:17.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.797+0000 7f6e0d800640 1 --2- 172.21.15.42:0/3626809239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6df00051d0 0x7f6e080cc690 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00e7b0 tx=0x7f6dfc00ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:17.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.801+0000 7f6e05600640 1 -- 172.21.15.42:0/3626809239 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc0052e0 con 0x7f6df00051d0 2024-05-28T14:08:17.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.801+0000 7f6e0ec00640 1 -- 172.21.15.42:0/3626809239 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e080cd450 con 0x7f6df00051d0 2024-05-28T14:08:17.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.801+0000 7f6e0ec00640 1 -- 172.21.15.42:0/3626809239 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e080cda10 con 0x7f6df00051d0 2024-05-28T14:08:17.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.805+0000 7f6e05600640 1 -- 172.21.15.42:0/3626809239 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dfc018070 con 0x7f6df00051d0 2024-05-28T14:08:17.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.805+0000 7f6e05600640 1 -- 172.21.15.42:0/3626809239 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc00c900 con 0x7f6df00051d0 2024-05-28T14:08:17.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.809+0000 7f6e0ec00640 1 -- 172.21.15.42:0/3626809239 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de40052a0 con 0x7f6df00051d0 2024-05-28T14:08:17.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.809+0000 7f6e05600640 1 -- 172.21.15.42:0/3626809239 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f6dfc00cb20 con 0x7f6df00051d0 2024-05-28T14:08:17.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.809+0000 7f6e05600640 1 --2- 172.21.15.42:0/3626809239 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6de80694b0 0x7f6de806b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:17.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.813+0000 7f6e05600640 1 -- 172.21.15.42:0/3626809239 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f6dfc00ce10 con 0x7f6df00051d0 2024-05-28T14:08:17.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.821+0000 7f6e05600640 1 -- 172.21.15.42:0/3626809239 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6dfc057490 con 0x7f6df00051d0 2024-05-28T14:08:17.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.833+0000 7f6e0ce00640 1 --2- 172.21.15.42:0/3626809239 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6de80694b0 0x7f6de806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:17.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:17.845+0000 7f6e0ce00640 1 --2- 172.21.15.42:0/3626809239 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6de80694b0 0x7f6de806b970 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6e080b34a0 tx=0x7f6df4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:18.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:18.301+0000 7f6e0ec00640 1 -- 172.21.15.42:0/3626809239 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone81117", "target": ["mon-mgr", ""]}) -- 0x7f6de4000bd0 con 0x7f6de80694b0 2024-05-28T14:08:19.141 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:08:19.147 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:08:19.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:19.689+0000 7f6e05600640 1 -- 172.21.15.42:0/3626809239 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6de4000bd0 con 0x7f6de80694b0 2024-05-28T14:08:19.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:19.697+0000 7f6e0ec00640 1 -- 172.21.15.42:0/3626809239 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6de80694b0 msgr2=0x7f6de806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:19.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:19.697+0000 7f6e0ec00640 1 --2- 172.21.15.42:0/3626809239 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6de80694b0 0x7f6de806b970 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6e080b34a0 tx=0x7f6df4008040 comp rx=0 tx=0).stop 2024-05-28T14:08:19.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:19.697+0000 7f6e0ec00640 1 -- 172.21.15.42:0/3626809239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6df00051d0 msgr2=0x7f6e080cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:19.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:19.697+0000 7f6e0ec00640 1 --2- 172.21.15.42:0/3626809239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6df00051d0 0x7f6e080cc690 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00e7b0 tx=0x7f6dfc00ec70 comp rx=0 tx=0).stop 2024-05-28T14:08:19.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:19.697+0000 7f6e0ec00640 1 -- 172.21.15.42:0/3626809239 shutdown_connections 2024-05-28T14:08:19.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:19.697+0000 7f6e0ec00640 1 --2- 172.21.15.42:0/3626809239 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6de80694b0 0x7f6de806b970 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:19.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:19.697+0000 7f6e0ec00640 1 --2- 172.21.15.42:0/3626809239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e0815dd70 0x7f6e080b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:19.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:19.697+0000 7f6e0ec00640 1 --2- 172.21.15.42:0/3626809239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e080ccbd0 0x7f6e0815d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:19.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:19.697+0000 7f6e0ec00640 1 --2- 172.21.15.42:0/3626809239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6df00051d0 0x7f6e080cc690 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00e7b0 tx=0x7f6dfc00ec70 comp rx=0 tx=0).stop 2024-05-28T14:08:19.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:19.697+0000 7f6e0ec00640 1 -- 172.21.15.42:0/3626809239 >> 172.21.15.42:0/3626809239 conn(0x7f6e080d0f00 msgr2=0x7f6e080b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:19.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:19.701+0000 7f6e0ec00640 1 -- 172.21.15.42:0/3626809239 shutdown_connections 2024-05-28T14:08:19.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:19.701+0000 7f6e0ec00640 1 -- 172.21.15.42:0/3626809239 wait complete. 2024-05-28T14:08:19.795 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone81118 2024-05-28T14:08:20.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.169+0000 7f9b2d200640 1 Processor -- start 2024-05-28T14:08:20.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.169+0000 7f9b2d200640 1 -- start start 2024-05-28T14:08:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.173+0000 7f9b2d200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b28162d10 0x7f9b2806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:20.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.173+0000 7f9b2d200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b28166800 con 0x7f9b280cfb40 2024-05-28T14:08:20.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.173+0000 7f9b2d200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b2806f8c0 con 0x7f9b280724f0 2024-05-28T14:08:20.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.173+0000 7f9b2d200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9b281630f0 con 0x7f9b28162d10 2024-05-28T14:08:20.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.173+0000 7f9b25e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b28162d10 0x7f9b2806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:20.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.173+0000 7f9b25e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b28162d10 0x7f9b2806f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:58786/0 (socket says 172.21.15.42:58786) 2024-05-28T14:08:20.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.173+0000 7f9b25e00640 1 -- 172.21.15.42:0/597005848 learned_addr learned my addr 172.21.15.42:0/597005848 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:20.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.181+0000 7f9b25e00640 1 -- 172.21.15.42:0/597005848 >> v1:172.21.15.63:6789/0 conn(0x7f9b280724f0 legacy=0x7f9b280b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:20.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.181+0000 7f9b25e00640 1 -- 172.21.15.42:0/597005848 >> v1:172.21.15.42:6789/0 conn(0x7f9b280cfb40 legacy=0x7f9b28071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.181+0000 7f9b25e00640 1 -- 172.21.15.42:0/597005848 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b28166ac0 con 0x7f9b28162d10 2024-05-28T14:08:20.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.185+0000 7f9b25e00640 1 --2- 172.21.15.42:0/597005848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b28162d10 0x7f9b2806f2e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b180098a0 tx=0x7f9b1801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e894fee17998e03 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:20.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.189+0000 7f9b25400640 1 -- 172.21.15.42:0/597005848 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b180ab430 con 0x7f9b28162d10 2024-05-28T14:08:20.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.189+0000 7f9b25400640 1 -- 172.21.15.42:0/597005848 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b180b0070 con 0x7f9b28162d10 2024-05-28T14:08:20.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.189+0000 7f9b25400640 1 -- 172.21.15.42:0/597005848 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b180ab7f0 con 0x7f9b28162d10 2024-05-28T14:08:20.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.189+0000 7f9b2d200640 1 -- 172.21.15.42:0/597005848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b28162d10 msgr2=0x7f9b2806f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:20.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.189+0000 7f9b2d200640 1 --2- 172.21.15.42:0/597005848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b28162d10 0x7f9b2806f2e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b180098a0 tx=0x7f9b1801c540 comp rx=0 tx=0).stop 2024-05-28T14:08:20.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.189+0000 7f9b2d200640 1 -- 172.21.15.42:0/597005848 shutdown_connections 2024-05-28T14:08:20.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.189+0000 7f9b2d200640 1 --2- 172.21.15.42:0/597005848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b28162d10 0x7f9b2806f2e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:20.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.189+0000 7f9b2d200640 1 -- 172.21.15.42:0/597005848 >> 172.21.15.42:0/597005848 conn(0x7f9b280b21f0 msgr2=0x7f9b280b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:20.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.189+0000 7f9b2d200640 1 -- 172.21.15.42:0/597005848 shutdown_connections 2024-05-28T14:08:20.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.189+0000 7f9b2d200640 1 -- 172.21.15.42:0/597005848 wait complete. 2024-05-28T14:08:20.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.193+0000 7f9b2d200640 1 Processor -- start 2024-05-28T14:08:20.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.193+0000 7f9b2d200640 1 -- start start 2024-05-28T14:08:20.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.193+0000 7f9b2d200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b280724f0 0x7f9b28151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:20.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.193+0000 7f9b2d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b280cfb40 0x7f9b28163280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:20.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.193+0000 7f9b2d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b28162d10 0x7f9b281637c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:20.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.193+0000 7f9b2d200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9b281675a0 con 0x7f9b280cfb40 2024-05-28T14:08:20.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.193+0000 7f9b2d200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9b281709a0 con 0x7f9b28162d10 2024-05-28T14:08:20.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.193+0000 7f9b2d200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9b281640d0 con 0x7f9b280724f0 2024-05-28T14:08:20.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b1fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b280cfb40 0x7f9b28163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:20.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b1fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b280cfb40 0x7f9b28163280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45244/0 (socket says 172.21.15.42:45244) 2024-05-28T14:08:20.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b1fe00640 1 -- 172.21.15.42:0/3637908653 learned_addr learned my addr 172.21.15.42:0/3637908653 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:20.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b26800640 1 --2- 172.21.15.42:0/3637908653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b28162d10 0x7f9b281637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:20.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b1fe00640 1 -- 172.21.15.42:0/3637908653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b280724f0 msgr2=0x7f9b28151ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T14:08:20.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b1fe00640 1 --2- 172.21.15.42:0/3637908653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b280724f0 0x7f9b28151ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:20.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b1fe00640 1 -- 172.21.15.42:0/3637908653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b28162d10 msgr2=0x7f9b281637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:20.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b1fe00640 1 --2- 172.21.15.42:0/3637908653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b28162d10 0x7f9b281637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:20.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b1fe00640 1 -- 172.21.15.42:0/3637908653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2813dfe0 con 0x7f9b280cfb40 2024-05-28T14:08:20.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b1fe00640 1 --2- 172.21.15.42:0/3637908653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b280cfb40 0x7f9b28163280 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000c490 tx=0x7f9b10001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:20.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b1e000640 1 -- 172.21.15.42:0/3637908653 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b10017070 con 0x7f9b280cfb40 2024-05-28T14:08:20.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b1e000640 1 -- 172.21.15.42:0/3637908653 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b10003600 con 0x7f9b280cfb40 2024-05-28T14:08:20.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b1e000640 1 -- 172.21.15.42:0/3637908653 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b100039f0 con 0x7f9b280cfb40 2024-05-28T14:08:20.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b2d200640 1 -- 172.21.15.42:0/3637908653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b28152910 con 0x7f9b280cfb40 2024-05-28T14:08:20.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.197+0000 7f9b2d200640 1 -- 172.21.15.42:0/3637908653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b2813e2b0 con 0x7f9b280cfb40 2024-05-28T14:08:20.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.201+0000 7f9b2d200640 1 -- 172.21.15.42:0/3637908653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b000052a0 con 0x7f9b280cfb40 2024-05-28T14:08:20.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.205+0000 7f9b1e000640 1 -- 172.21.15.42:0/3637908653 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f9b1001a070 con 0x7f9b280cfb40 2024-05-28T14:08:20.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.205+0000 7f9b1e000640 1 --2- 172.21.15.42:0/3637908653 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b040694b0 0x7f9b0406b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:20.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.209+0000 7f9b1e000640 1 -- 172.21.15.42:0/3637908653 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f9b10085070 con 0x7f9b280cfb40 2024-05-28T14:08:20.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.213+0000 7f9b25e00640 1 --2- 172.21.15.42:0/3637908653 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b040694b0 0x7f9b0406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:20.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.217+0000 7f9b1e000640 1 -- 172.21.15.42:0/3637908653 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9b10021020 con 0x7f9b280cfb40 2024-05-28T14:08:20.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.221+0000 7f9b25e00640 1 --2- 172.21.15.42:0/3637908653 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b040694b0 0x7f9b0406b970 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9b180af000 tx=0x7f9b180a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:20.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:20.713+0000 7f9b2d200640 1 -- 172.21.15.42:0/3637908653 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone81118", "target": ["mon-mgr", ""]}) -- 0x7f9b00000bd0 con 0x7f9b040694b0 2024-05-28T14:08:21.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.497+0000 7f9b1e000640 1 -- 172.21.15.42:0/3637908653 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9b00000bd0 con 0x7f9b040694b0 2024-05-28T14:08:21.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.509+0000 7f9b2d200640 1 -- 172.21.15.42:0/3637908653 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b040694b0 msgr2=0x7f9b0406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:21.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.509+0000 7f9b2d200640 1 --2- 172.21.15.42:0/3637908653 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b040694b0 0x7f9b0406b970 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9b180af000 tx=0x7f9b180a7000 comp rx=0 tx=0).stop 2024-05-28T14:08:21.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.509+0000 7f9b2d200640 1 -- 172.21.15.42:0/3637908653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b280cfb40 msgr2=0x7f9b28163280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:21.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.509+0000 7f9b2d200640 1 --2- 172.21.15.42:0/3637908653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b280cfb40 0x7f9b28163280 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000c490 tx=0x7f9b10001e80 comp rx=0 tx=0).stop 2024-05-28T14:08:21.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.509+0000 7f9b2d200640 1 -- 172.21.15.42:0/3637908653 shutdown_connections 2024-05-28T14:08:21.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.509+0000 7f9b2d200640 1 --2- 172.21.15.42:0/3637908653 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9b040694b0 0x7f9b0406b970 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:21.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.509+0000 7f9b2d200640 1 --2- 172.21.15.42:0/3637908653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b28162d10 0x7f9b281637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:21.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.509+0000 7f9b2d200640 1 --2- 172.21.15.42:0/3637908653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b280cfb40 0x7f9b28163280 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:21.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.509+0000 7f9b2d200640 1 --2- 172.21.15.42:0/3637908653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b280724f0 0x7f9b28151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:21.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.509+0000 7f9b2d200640 1 -- 172.21.15.42:0/3637908653 >> 172.21.15.42:0/3637908653 conn(0x7f9b280b21f0 msgr2=0x7f9b280cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:21.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.509+0000 7f9b2d200640 1 -- 172.21.15.42:0/3637908653 shutdown_connections 2024-05-28T14:08:21.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.509+0000 7f9b2d200640 1 -- 172.21.15.42:0/3637908653 wait complete. 2024-05-28T14:08:21.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone81119 2024-05-28T14:08:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.913+0000 7f5127200640 1 Processor -- start 2024-05-28T14:08:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.913+0000 7f5127200640 1 -- start start 2024-05-28T14:08:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.917+0000 7f5127200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f51200b3fc0 0x7f512006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:21.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.917+0000 7f5127200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5120162f60 con 0x7f5120167070 2024-05-28T14:08:21.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.917+0000 7f5127200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512006f5e0 con 0x7f51200d0320 2024-05-28T14:08:21.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.917+0000 7f5127200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f51200b4630 con 0x7f51200b3fc0 2024-05-28T14:08:21.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.917+0000 7f5126800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f5120167070 0x7f51200cf260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47536/0 (socket says 172.21.15.42:47536) 2024-05-28T14:08:21.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.917+0000 7f5126800640 1 -- 172.21.15.42:0/2255944736 learned_addr learned my addr 172.21.15.42:0/2255944736 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:08:21.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.917+0000 7f5125400640 1 --2- 172.21.15.42:0/2255944736 >> v2:172.21.15.63:3300/0 conn(0x7f51200b3fc0 0x7f512006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:21.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.921+0000 7f5124a00640 1 -- 172.21.15.42:0/2255944736 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 626523734 0 0) 0x7f5120162f60 con 0x7f5120167070 2024-05-28T14:08:21.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.921+0000 7f5124a00640 1 -- 172.21.15.42:0/2255944736 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5108003530 con 0x7f5120167070 2024-05-28T14:08:21.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.921+0000 7f5124a00640 1 -- 172.21.15.42:0/2255944736 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3797402496 0 0) 0x7f512006f5e0 con 0x7f51200d0320 2024-05-28T14:08:21.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.921+0000 7f5124a00640 1 -- 172.21.15.42:0/2255944736 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5120162f60 con 0x7f51200d0320 2024-05-28T14:08:21.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.921+0000 7f5124a00640 1 -- 172.21.15.42:0/2255944736 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 55946084 0 0) 0x7f5108003530 con 0x7f5120167070 2024-05-28T14:08:21.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.921+0000 7f5125400640 1 -- 172.21.15.42:0/2255944736 >> v1:172.21.15.63:6789/0 conn(0x7f51200d0320 legacy=0x7f51200d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:21.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.921+0000 7f5125400640 1 -- 172.21.15.42:0/2255944736 >> v1:172.21.15.42:6789/0 conn(0x7f5120167070 legacy=0x7f51200cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:21.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.921+0000 7f5125400640 1 -- 172.21.15.42:0/2255944736 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51200b4910 con 0x7f51200b3fc0 2024-05-28T14:08:21.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.929+0000 7f5125400640 1 --2- 172.21.15.42:0/2255944736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51200b3fc0 0x7f512006e4a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5114009740 tx=0x7f511401c270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9162bd160d4f4b08 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:21.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.929+0000 7f5124a00640 1 -- 172.21.15.42:0/2255944736 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51140b0070 con 0x7f51200b3fc0 2024-05-28T14:08:21.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.929+0000 7f5124a00640 1 -- 172.21.15.42:0/2255944736 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51140ab400 con 0x7f51200b3fc0 2024-05-28T14:08:21.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.929+0000 7f5124a00640 1 -- 172.21.15.42:0/2255944736 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c001a00 con 0x7f51200b3fc0 2024-05-28T14:08:21.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.933+0000 7f5127200640 1 -- 172.21.15.42:0/2255944736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51200b3fc0 msgr2=0x7f512006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:21.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.933+0000 7f5127200640 1 --2- 172.21.15.42:0/2255944736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51200b3fc0 0x7f512006e4a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5114009740 tx=0x7f511401c270 comp rx=0 tx=0).stop 2024-05-28T14:08:21.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.933+0000 7f5127200640 1 -- 172.21.15.42:0/2255944736 shutdown_connections 2024-05-28T14:08:21.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.933+0000 7f5127200640 1 --2- 172.21.15.42:0/2255944736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51200b3fc0 0x7f512006e4a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:21.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.933+0000 7f5127200640 1 -- 172.21.15.42:0/2255944736 >> 172.21.15.42:0/2255944736 conn(0x7f512006bbc0 msgr2=0x7f512006bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:21.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.933+0000 7f5127200640 1 -- 172.21.15.42:0/2255944736 shutdown_connections 2024-05-28T14:08:21.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.933+0000 7f5127200640 1 -- 172.21.15.42:0/2255944736 wait complete. 2024-05-28T14:08:21.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.937+0000 7f5127200640 1 Processor -- start 2024-05-28T14:08:21.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.937+0000 7f5127200640 1 -- start start 2024-05-28T14:08:21.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.937+0000 7f5127200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51200d0320 0x7f5120163130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:21.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.937+0000 7f5127200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5120167070 0x7f51201626b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:21.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.937+0000 7f5127200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512015a110 0x7f5120156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:21.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.937+0000 7f5127200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f51200cff00 con 0x7f51200d0320 2024-05-28T14:08:21.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.937+0000 7f5127200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f51200cfd80 con 0x7f512015a110 2024-05-28T14:08:21.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.937+0000 7f5127200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5120171680 con 0x7f5120167070 2024-05-28T14:08:21.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.941+0000 7f5125400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5120167070 0x7f51201626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:21.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.941+0000 7f5126800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512015a110 0x7f5120156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:21.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.941+0000 7f5126800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512015a110 0x7f5120156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:60998/0 (socket says 172.21.15.42:60998) 2024-05-28T14:08:21.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.941+0000 7f5126800640 1 -- 172.21.15.42:0/3266311174 learned_addr learned my addr 172.21.15.42:0/3266311174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:21.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.941+0000 7f5125e00640 1 --2- 172.21.15.42:0/3266311174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51200d0320 0x7f5120163130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:21.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.945+0000 7f5125e00640 1 -- 172.21.15.42:0/3266311174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5120167070 msgr2=0x7f51201626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:21.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.945+0000 7f5125e00640 1 --2- 172.21.15.42:0/3266311174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5120167070 0x7f51201626b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:21.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.945+0000 7f5125e00640 1 -- 172.21.15.42:0/3266311174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512015a110 msgr2=0x7f5120156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:21.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.945+0000 7f5125e00640 1 --2- 172.21.15.42:0/3266311174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512015a110 0x7f5120156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:21.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.945+0000 7f5125e00640 1 -- 172.21.15.42:0/3266311174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51200d0c40 con 0x7f51200d0320 2024-05-28T14:08:21.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.945+0000 7f5125e00640 1 --2- 172.21.15.42:0/3266311174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51200d0320 0x7f5120163130 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f511000c7b0 tx=0x7f511000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:21.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.945+0000 7f511a000640 1 -- 172.21.15.42:0/3266311174 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110005480 con 0x7f51200d0320 2024-05-28T14:08:21.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.945+0000 7f511a000640 1 -- 172.21.15.42:0/3266311174 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511000a640 con 0x7f51200d0320 2024-05-28T14:08:21.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.945+0000 7f511a000640 1 -- 172.21.15.42:0/3266311174 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511000aa30 con 0x7f51200d0320 2024-05-28T14:08:21.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.945+0000 7f5127200640 1 -- 172.21.15.42:0/3266311174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512015d810 con 0x7f51200d0320 2024-05-28T14:08:21.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.949+0000 7f5127200640 1 -- 172.21.15.42:0/3266311174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512015ddd0 con 0x7f51200d0320 2024-05-28T14:08:21.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.949+0000 7f5127200640 1 -- 172.21.15.42:0/3266311174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f512004fcf0 con 0x7f51200d0320 2024-05-28T14:08:21.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.953+0000 7f511a000640 1 -- 172.21.15.42:0/3266311174 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f511000abd0 con 0x7f51200d0320 2024-05-28T14:08:21.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.953+0000 7f511a000640 1 --2- 172.21.15.42:0/3266311174 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f51040694b0 0x7f510406b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:21.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.953+0000 7f511a000640 1 -- 172.21.15.42:0/3266311174 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f51100857b0 con 0x7f51200d0320 2024-05-28T14:08:21.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.957+0000 7f5125400640 1 --2- 172.21.15.42:0/3266311174 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f51040694b0 0x7f510406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.961+0000 7f5125400640 1 --2- 172.21.15.42:0/3266311174 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f51040694b0 0x7f510406b970 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f51140ad000 tx=0x7f51140a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:21.961+0000 7f511a000640 1 -- 172.21.15.42:0/3266311174 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5110057480 con 0x7f51200d0320 2024-05-28T14:08:22.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:22.397+0000 7f5127200640 1 -- 172.21.15.42:0/3266311174 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone81119", "target": ["mon-mgr", ""]}) -- 0x7f51200b1060 con 0x7f51040694b0 2024-05-28T14:08:22.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:22.909+0000 7f511a000640 1 -- 172.21.15.42:0/3266311174 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f51200b1060 con 0x7f51040694b0 2024-05-28T14:08:22.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:22.913+0000 7f5127200640 1 -- 172.21.15.42:0/3266311174 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f51040694b0 msgr2=0x7f510406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:22.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:22.913+0000 7f5127200640 1 --2- 172.21.15.42:0/3266311174 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f51040694b0 0x7f510406b970 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f51140ad000 tx=0x7f51140a6000 comp rx=0 tx=0).stop 2024-05-28T14:08:22.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:22.913+0000 7f5127200640 1 -- 172.21.15.42:0/3266311174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51200d0320 msgr2=0x7f5120163130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:22.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:22.913+0000 7f5127200640 1 --2- 172.21.15.42:0/3266311174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51200d0320 0x7f5120163130 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f511000c7b0 tx=0x7f511000cc70 comp rx=0 tx=0).stop 2024-05-28T14:08:22.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:22.913+0000 7f5127200640 1 -- 172.21.15.42:0/3266311174 shutdown_connections 2024-05-28T14:08:22.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:22.913+0000 7f5127200640 1 --2- 172.21.15.42:0/3266311174 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f51040694b0 0x7f510406b970 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:22.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:22.913+0000 7f5127200640 1 --2- 172.21.15.42:0/3266311174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512015a110 0x7f5120156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:22.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:22.913+0000 7f5127200640 1 --2- 172.21.15.42:0/3266311174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5120167070 0x7f51201626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:22.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:22.913+0000 7f5127200640 1 --2- 172.21.15.42:0/3266311174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51200d0320 0x7f5120163130 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:22.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:22.913+0000 7f5127200640 1 -- 172.21.15.42:0/3266311174 >> 172.21.15.42:0/3266311174 conn(0x7f512006bbc0 msgr2=0x7f512006bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:22.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:22.913+0000 7f5127200640 1 -- 172.21.15.42:0/3266311174 shutdown_connections 2024-05-28T14:08:22.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:22.913+0000 7f5127200640 1 -- 172.21.15.42:0/3266311174 wait complete. 2024-05-28T14:08:22.927 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone81120 2024-05-28T14:08:23.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.225+0000 7f8db6600640 1 Processor -- start 2024-05-28T14:08:23.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.225+0000 7f8db6600640 1 -- start start 2024-05-28T14:08:23.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.229+0000 7f8db6600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8db015ba70 0x7f8db006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:23.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.229+0000 7f8db6600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8db0072710 con 0x7f8db0154dd0 2024-05-28T14:08:23.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.229+0000 7f8db6600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8db015e330 con 0x7f8db006f800 2024-05-28T14:08:23.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.229+0000 7f8db6600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8db00724f0 con 0x7f8db015ba70 2024-05-28T14:08:23.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.229+0000 7f8daf400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8db0154dd0 0x7f8db0071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47550/0 (socket says 172.21.15.42:47550) 2024-05-28T14:08:23.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.229+0000 7f8daf400640 1 -- 172.21.15.42:0/3852027901 learned_addr learned my addr 172.21.15.42:0/3852027901 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:08:23.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.229+0000 7f8da7e00640 1 --2- 172.21.15.42:0/3852027901 >> v2:172.21.15.63:3300/0 conn(0x7f8db015ba70 0x7f8db006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:23.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.229+0000 7f8daea00640 1 -- 172.21.15.42:0/3852027901 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3515315749 0 0) 0x7f8db015e330 con 0x7f8db006f800 2024-05-28T14:08:23.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.229+0000 7f8daea00640 1 -- 172.21.15.42:0/3852027901 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d94003530 con 0x7f8db006f800 2024-05-28T14:08:23.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.233+0000 7f8da7e00640 1 -- 172.21.15.42:0/3852027901 >> v1:172.21.15.63:6789/0 conn(0x7f8db006f800 legacy=0x7f8db00ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:23.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.233+0000 7f8da7e00640 1 -- 172.21.15.42:0/3852027901 >> v1:172.21.15.42:6789/0 conn(0x7f8db0154dd0 legacy=0x7f8db0071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:23.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.233+0000 7f8da7e00640 1 -- 172.21.15.42:0/3852027901 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db015e610 con 0x7f8db015ba70 2024-05-28T14:08:23.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.237+0000 7f8da7e00640 1 --2- 172.21.15.42:0/3852027901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8db015ba70 0x7f8db006f2c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800ba80 tx=0x7f8d9801eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a270219155999ab9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:23.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.237+0000 7f8daea00640 1 -- 172.21.15.42:0/3852027901 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d980ad070 con 0x7f8db015ba70 2024-05-28T14:08:23.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.237+0000 7f8daea00640 1 -- 172.21.15.42:0/3852027901 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d980a7bc0 con 0x7f8db015ba70 2024-05-28T14:08:23.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.237+0000 7f8daea00640 1 -- 172.21.15.42:0/3852027901 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d980b1400 con 0x7f8db015ba70 2024-05-28T14:08:23.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.237+0000 7f8db6600640 1 -- 172.21.15.42:0/3852027901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8db015ba70 msgr2=0x7f8db006f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:23.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.237+0000 7f8db6600640 1 --2- 172.21.15.42:0/3852027901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8db015ba70 0x7f8db006f2c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800ba80 tx=0x7f8d9801eaf0 comp rx=0 tx=0).stop 2024-05-28T14:08:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.241+0000 7f8db6600640 1 -- 172.21.15.42:0/3852027901 shutdown_connections 2024-05-28T14:08:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.241+0000 7f8db6600640 1 --2- 172.21.15.42:0/3852027901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8db015ba70 0x7f8db006f2c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.241+0000 7f8db6600640 1 -- 172.21.15.42:0/3852027901 >> 172.21.15.42:0/3852027901 conn(0x7f8db00b21f0 msgr2=0x7f8db00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.241+0000 7f8db6600640 1 -- 172.21.15.42:0/3852027901 shutdown_connections 2024-05-28T14:08:23.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.245+0000 7f8db6600640 1 -- 172.21.15.42:0/3852027901 wait complete. 2024-05-28T14:08:23.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.253+0000 7f8db6600640 1 Processor -- start 2024-05-28T14:08:23.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8db6600640 1 -- start start 2024-05-28T14:08:23.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8db6600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8db006f800 0x7f8db00b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:23.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8db6600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8db0154dd0 0x7f8db00b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:23.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8db6600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8db015ba70 0x7f8db00b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:23.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8db6600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8db015f0f0 con 0x7f8db0154dd0 2024-05-28T14:08:23.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8db6600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8db006fc70 con 0x7f8db015ba70 2024-05-28T14:08:23.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8db6600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8db00b4260 con 0x7f8db006f800 2024-05-28T14:08:23.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8da7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8db0154dd0 0x7f8db00b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:23.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8da7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8db0154dd0 0x7f8db00b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45258/0 (socket says 172.21.15.42:45258) 2024-05-28T14:08:23.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8da7e00640 1 -- 172.21.15.42:0/4236011556 learned_addr learned my addr 172.21.15.42:0/4236011556 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:23.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8dafe00640 1 --2- 172.21.15.42:0/4236011556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8db015ba70 0x7f8db00b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:23.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8da7e00640 1 -- 172.21.15.42:0/4236011556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8db006f800 msgr2=0x7f8db00b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:23.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8da7e00640 1 --2- 172.21.15.42:0/4236011556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8db006f800 0x7f8db00b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:23.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8da7e00640 1 -- 172.21.15.42:0/4236011556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8db015ba70 msgr2=0x7f8db00b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:23.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8da7e00640 1 --2- 172.21.15.42:0/4236011556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8db015ba70 0x7f8db00b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:23.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.257+0000 7f8da7e00640 1 -- 172.21.15.42:0/4236011556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db01473b0 con 0x7f8db0154dd0 2024-05-28T14:08:23.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.261+0000 7f8da7e00640 1 --2- 172.21.15.42:0/4236011556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8db0154dd0 0x7f8db00b0060 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800ba80 tx=0x7f8d9801b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:23.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.261+0000 7f8da7400640 1 -- 172.21.15.42:0/4236011556 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d980c0070 con 0x7f8db0154dd0 2024-05-28T14:08:23.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.261+0000 7f8da7400640 1 -- 172.21.15.42:0/4236011556 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d980b1e10 con 0x7f8db0154dd0 2024-05-28T14:08:23.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.261+0000 7f8da7400640 1 -- 172.21.15.42:0/4236011556 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d980ad040 con 0x7f8db0154dd0 2024-05-28T14:08:23.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.261+0000 7f8db6600640 1 -- 172.21.15.42:0/4236011556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db014a930 con 0x7f8db0154dd0 2024-05-28T14:08:23.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.261+0000 7f8db6600640 1 -- 172.21.15.42:0/4236011556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db014aef0 con 0x7f8db0154dd0 2024-05-28T14:08:23.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.261+0000 7f8db6600640 1 -- 172.21.15.42:0/4236011556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db00738b0 con 0x7f8db0154dd0 2024-05-28T14:08:23.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.269+0000 7f8da7400640 1 -- 172.21.15.42:0/4236011556 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f8d980c3070 con 0x7f8db0154dd0 2024-05-28T14:08:23.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.269+0000 7f8da7400640 1 --2- 172.21.15.42:0/4236011556 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d8c069530 0x7f8d8c06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:23.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.269+0000 7f8da7400640 1 -- 172.21.15.42:0/4236011556 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f8d9811fdc0 con 0x7f8db0154dd0 2024-05-28T14:08:23.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.269+0000 7f8daf400640 1 --2- 172.21.15.42:0/4236011556 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d8c069530 0x7f8d8c06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:23.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.273+0000 7f8daf400640 1 --2- 172.21.15.42:0/4236011556 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d8c069530 0x7f8d8c06b9f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8da0006120 tx=0x7f8da0005cc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:23.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.273+0000 7f8da7400640 1 -- 172.21.15.42:0/4236011556 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8d980ca020 con 0x7f8db0154dd0 2024-05-28T14:08:23.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.629+0000 7f8db6600640 1 -- 172.21.15.42:0/4236011556 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone81120", "target": ["mon-mgr", ""]}) -- 0x7f8db004fdd0 con 0x7f8d8c069530 2024-05-28T14:08:23.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.837+0000 7f8da7400640 1 -- 172.21.15.42:0/4236011556 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8db004fdd0 con 0x7f8d8c069530 2024-05-28T14:08:23.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.841+0000 7f8db6600640 1 -- 172.21.15.42:0/4236011556 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d8c069530 msgr2=0x7f8d8c06b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.841+0000 7f8db6600640 1 --2- 172.21.15.42:0/4236011556 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d8c069530 0x7f8d8c06b9f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8da0006120 tx=0x7f8da0005cc0 comp rx=0 tx=0).stop 2024-05-28T14:08:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.841+0000 7f8db6600640 1 -- 172.21.15.42:0/4236011556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8db0154dd0 msgr2=0x7f8db00b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.841+0000 7f8db6600640 1 --2- 172.21.15.42:0/4236011556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8db0154dd0 0x7f8db00b0060 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800ba80 tx=0x7f8d9801b890 comp rx=0 tx=0).stop 2024-05-28T14:08:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.841+0000 7f8db6600640 1 -- 172.21.15.42:0/4236011556 shutdown_connections 2024-05-28T14:08:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.841+0000 7f8db6600640 1 --2- 172.21.15.42:0/4236011556 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d8c069530 0x7f8d8c06b9f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.841+0000 7f8db6600640 1 --2- 172.21.15.42:0/4236011556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8db015ba70 0x7f8db00b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.841+0000 7f8db6600640 1 --2- 172.21.15.42:0/4236011556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8db0154dd0 0x7f8db00b0060 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.841+0000 7f8db6600640 1 --2- 172.21.15.42:0/4236011556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8db006f800 0x7f8db00b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.841+0000 7f8db6600640 1 -- 172.21.15.42:0/4236011556 >> 172.21.15.42:0/4236011556 conn(0x7f8db00b21f0 msgr2=0x7f8db00ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.841+0000 7f8db6600640 1 -- 172.21.15.42:0/4236011556 shutdown_connections 2024-05-28T14:08:23.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:23.841+0000 7f8db6600640 1 -- 172.21.15.42:0/4236011556 wait complete. 2024-05-28T14:08:23.844 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:08:23.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.005+0000 7f9d78600640 1 Processor -- start 2024-05-28T14:08:23.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.005+0000 7f9d78600640 1 -- start start 2024-05-28T14:08:23.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.005+0000 7f9d78600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d7006f030 0x7f9d7006db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:23.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.005+0000 7f9d78600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d7006f480 con 0x7f9d700724d0 2024-05-28T14:08:23.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.005+0000 7f9d78600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d7006e240 con 0x7f9d70145920 2024-05-28T14:08:23.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.005+0000 7f9d78600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9d7006e520 con 0x7f9d7006f030 2024-05-28T14:08:23.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.005+0000 7f9d75200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f9d700724d0 0x7f9d7015d120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47566/0 (socket says 172.21.15.42:47566) 2024-05-28T14:08:23.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.005+0000 7f9d75200640 1 -- 172.21.15.42:0/181020120 learned_addr learned my addr 172.21.15.42:0/181020120 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:08:23.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.005+0000 7f9d76600640 1 --2- 172.21.15.42:0/181020120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d7006f030 0x7f9d7006db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.009+0000 7f9d6fe00640 1 -- 172.21.15.42:0/181020120 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2618442903 0 0) 0x7f9d7006f480 con 0x7f9d700724d0 2024-05-28T14:08:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.009+0000 7f9d6fe00640 1 -- 172.21.15.42:0/181020120 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d58003530 con 0x7f9d700724d0 2024-05-28T14:08:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.009+0000 7f9d6fe00640 1 -- 172.21.15.42:0/181020120 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 568609108 0 0) 0x7f9d7006e240 con 0x7f9d70145920 2024-05-28T14:08:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.009+0000 7f9d6fe00640 1 -- 172.21.15.42:0/181020120 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d7006f480 con 0x7f9d70145920 2024-05-28T14:08:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.009+0000 7f9d76600640 1 -- 172.21.15.42:0/181020120 >> v1:172.21.15.63:6789/0 conn(0x7f9d70145920 legacy=0x7f9d70071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.009+0000 7f9d76600640 1 -- 172.21.15.42:0/181020120 >> v1:172.21.15.42:6789/0 conn(0x7f9d700724d0 legacy=0x7f9d7015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.009+0000 7f9d76600640 1 -- 172.21.15.42:0/181020120 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7006e800 con 0x7f9d7006f030 2024-05-28T14:08:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.009+0000 7f9d76600640 1 --2- 172.21.15.42:0/181020120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d7006f030 0x7f9d7006db40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009dc0 tx=0x7f9d6801c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c49de3e22143df7f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.009+0000 7f9d6fe00640 1 -- 172.21.15.42:0/181020120 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d680b2070 con 0x7f9d7006f030 2024-05-28T14:08:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.009+0000 7f9d6fe00640 1 -- 172.21.15.42:0/181020120 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d680a58d0 con 0x7f9d7006f030 2024-05-28T14:08:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.009+0000 7f9d6fe00640 1 -- 172.21.15.42:0/181020120 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d680a5ca0 con 0x7f9d7006f030 2024-05-28T14:08:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 -- 172.21.15.42:0/181020120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d7006f030 msgr2=0x7f9d7006db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 --2- 172.21.15.42:0/181020120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d7006f030 0x7f9d7006db40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009dc0 tx=0x7f9d6801c900 comp rx=0 tx=0).stop 2024-05-28T14:08:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 -- 172.21.15.42:0/181020120 shutdown_connections 2024-05-28T14:08:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 --2- 172.21.15.42:0/181020120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d7006f030 0x7f9d7006db40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 -- 172.21.15.42:0/181020120 >> 172.21.15.42:0/181020120 conn(0x7f9d7006b9f0 msgr2=0x7f9d7006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 -- 172.21.15.42:0/181020120 shutdown_connections 2024-05-28T14:08:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 -- 172.21.15.42:0/181020120 wait complete. 2024-05-28T14:08:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 Processor -- start 2024-05-28T14:08:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 -- start start 2024-05-28T14:08:23.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d7006f030 0x7f9d7013dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:23.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d700724d0 0x7f9d70159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:23.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d70145920 0x7f9d70159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:23.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7015e3d0 con 0x7f9d7006f030 2024-05-28T14:08:23.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7015e250 con 0x7f9d700724d0 2024-05-28T14:08:23.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d78600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9d7016a260 con 0x7f9d70145920 2024-05-28T14:08:23.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d75c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d7006f030 0x7f9d7013dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:23.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d75c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d7006f030 0x7f9d7013dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45266/0 (socket says 172.21.15.42:45266) 2024-05-28T14:08:23.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d75c00640 1 -- 172.21.15.42:0/1977289106 learned_addr learned my addr 172.21.15.42:0/1977289106 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:23.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d76600640 1 --2- 172.21.15.42:0/1977289106 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d70145920 0x7f9d70159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:23.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d75200640 1 --2- 172.21.15.42:0/1977289106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d700724d0 0x7f9d70159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:23.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d75c00640 1 -- 172.21.15.42:0/1977289106 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d70145920 msgr2=0x7f9d70159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:23.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d75c00640 1 --2- 172.21.15.42:0/1977289106 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d70145920 0x7f9d70159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:23.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d75c00640 1 -- 172.21.15.42:0/1977289106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d700724d0 msgr2=0x7f9d70159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:23.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d75c00640 1 --2- 172.21.15.42:0/1977289106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d700724d0 0x7f9d70159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:23.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d75c00640 1 -- 172.21.15.42:0/1977289106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d700ae280 con 0x7f9d7006f030 2024-05-28T14:08:24.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.013+0000 7f9d75c00640 1 --2- 172.21.15.42:0/1977289106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d7006f030 0x7f9d7013dd30 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400c500 tx=0x7f9d6400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:24.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.017+0000 7f9d6d600640 1 -- 172.21.15.42:0/1977289106 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64017070 con 0x7f9d7006f030 2024-05-28T14:08:24.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.017+0000 7f9d6d600640 1 -- 172.21.15.42:0/1977289106 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d64005310 con 0x7f9d7006f030 2024-05-28T14:08:24.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.017+0000 7f9d78600640 1 -- 172.21.15.42:0/1977289106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d700ae520 con 0x7f9d7006f030 2024-05-28T14:08:24.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.017+0000 7f9d6d600640 1 -- 172.21.15.42:0/1977289106 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d640054c0 con 0x7f9d7006f030 2024-05-28T14:08:24.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.017+0000 7f9d78600640 1 -- 172.21.15.42:0/1977289106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d700b4a30 con 0x7f9d7006f030 2024-05-28T14:08:24.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.017+0000 7f9d78600640 1 -- 172.21.15.42:0/1977289106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d70073890 con 0x7f9d7006f030 2024-05-28T14:08:24.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.017+0000 7f9d6d600640 1 -- 172.21.15.42:0/1977289106 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f9d6401a070 con 0x7f9d7006f030 2024-05-28T14:08:24.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.017+0000 7f9d6d600640 1 --2- 172.21.15.42:0/1977289106 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9d50069460 0x7f9d5006b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:24.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.017+0000 7f9d6d600640 1 -- 172.21.15.42:0/1977289106 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f9d64084b80 con 0x7f9d7006f030 2024-05-28T14:08:24.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.017+0000 7f9d75200640 1 --2- 172.21.15.42:0/1977289106 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9d50069460 0x7f9d5006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:24.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.017+0000 7f9d75200640 1 --2- 172.21.15.42:0/1977289106 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9d50069460 0x7f9d5006b920 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9d7015c310 tx=0x7f9d5c007d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:24.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.021+0000 7f9d6d600640 1 -- 172.21.15.42:0/1977289106 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9d64056180 con 0x7f9d7006f030 2024-05-28T14:08:24.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.209+0000 7f9d78600640 1 -- 172.21.15.42:0/1977289106 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f9d700aff40 con 0x7f9d50069460 2024-05-28T14:08:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.209+0000 7f9d6d600640 1 -- 172.21.15.42:0/1977289106 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f9d700aff40 con 0x7f9d50069460 2024-05-28T14:08:24.194 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:08:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:08:24.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:24.194 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:08:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.213+0000 7f9d78600640 1 -- 172.21.15.42:0/1977289106 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9d50069460 msgr2=0x7f9d5006b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.213+0000 7f9d78600640 1 --2- 172.21.15.42:0/1977289106 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9d50069460 0x7f9d5006b920 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9d7015c310 tx=0x7f9d5c007d40 comp rx=0 tx=0).stop 2024-05-28T14:08:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.213+0000 7f9d78600640 1 -- 172.21.15.42:0/1977289106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d7006f030 msgr2=0x7f9d7013dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.213+0000 7f9d78600640 1 --2- 172.21.15.42:0/1977289106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d7006f030 0x7f9d7013dd30 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400c500 tx=0x7f9d6400c9c0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.213+0000 7f9d78600640 1 -- 172.21.15.42:0/1977289106 shutdown_connections 2024-05-28T14:08:24.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.213+0000 7f9d78600640 1 --2- 172.21.15.42:0/1977289106 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9d50069460 0x7f9d5006b920 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.213+0000 7f9d78600640 1 --2- 172.21.15.42:0/1977289106 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d70145920 0x7f9d70159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.213+0000 7f9d78600640 1 --2- 172.21.15.42:0/1977289106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d700724d0 0x7f9d70159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.213+0000 7f9d78600640 1 --2- 172.21.15.42:0/1977289106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d7006f030 0x7f9d7013dd30 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.213+0000 7f9d78600640 1 -- 172.21.15.42:0/1977289106 >> 172.21.15.42:0/1977289106 conn(0x7f9d7006b9f0 msgr2=0x7f9d70071dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:24.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.213+0000 7f9d78600640 1 -- 172.21.15.42:0/1977289106 shutdown_connections 2024-05-28T14:08:24.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.213+0000 7f9d78600640 1 -- 172.21.15.42:0/1977289106 wait complete. 2024-05-28T14:08:24.215 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:08:24.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd98c00640 1 Processor -- start 2024-05-28T14:08:24.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd98c00640 1 -- start start 2024-05-28T14:08:24.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd98c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbd940d0d50 0x7fbd941621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:24.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd98c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbd940cf320 0x7fbd940cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:24.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd98c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd94163120 con 0x7fbd940d03e0 2024-05-28T14:08:24.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd98c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fbd940d1e00 con 0x7fbd940cf320 2024-05-28T14:08:24.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd98c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbd94162880 con 0x7fbd940d0d50 2024-05-28T14:08:24.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd8be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbd940cf320 0x7fbd940cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:24.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd8be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbd940cf320 0x7fbd940cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45274/0 (socket says 172.21.15.42:45274) 2024-05-28T14:08:24.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd8be00640 1 -- 172.21.15.42:0/4042398507 learned_addr learned my addr 172.21.15.42:0/4042398507 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:24.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd93400640 1 --2- 172.21.15.42:0/4042398507 >> v2:172.21.15.63:3300/0 conn(0x7fbd940d0d50 0x7fbd941621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:24.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd8be00640 1 -- 172.21.15.42:0/4042398507 >> v2:172.21.15.63:3300/0 conn(0x7fbd940d0d50 msgr2=0x7fbd941621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:24.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd8be00640 1 --2- 172.21.15.42:0/4042398507 >> v2:172.21.15.63:3300/0 conn(0x7fbd940d0d50 0x7fbd941621c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd8be00640 1 -- 172.21.15.42:0/4042398507 >> v1:172.21.15.63:6789/0 conn(0x7fbd940d03e0 legacy=0x7fbd9415f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:24.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd8be00640 1 -- 172.21.15.42:0/4042398507 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd94162b60 con 0x7fbd940cf320 2024-05-28T14:08:24.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.373+0000 7fbd93400640 1 --2- 172.21.15.42:0/4042398507 >> v2:172.21.15.63:3300/0 conn(0x7fbd940d0d50 0x7fbd941621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:08:24.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.377+0000 7fbd8be00640 1 --2- 172.21.15.42:0/4042398507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd940cf320 0x7fbd940cfea0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fbd94050e20 tx=0x7fbd7c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32341b632d3d523 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:24.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.377+0000 7fbd92a00640 1 -- 172.21.15.42:0/4042398507 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0aa650 con 0x7fbd940cf320 2024-05-28T14:08:24.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.377+0000 7fbd92a00640 1 -- 172.21.15.42:0/4042398507 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7c0af070 con 0x7fbd940cf320 2024-05-28T14:08:24.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.377+0000 7fbd92a00640 1 -- 172.21.15.42:0/4042398507 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0aaa10 con 0x7fbd940cf320 2024-05-28T14:08:24.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.377+0000 7fbd98c00640 1 -- 172.21.15.42:0/4042398507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd940cf320 msgr2=0x7fbd940cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:24.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.377+0000 7fbd98c00640 1 --2- 172.21.15.42:0/4042398507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd940cf320 0x7fbd940cfea0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fbd94050e20 tx=0x7fbd7c01c4f0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.377+0000 7fbd98c00640 1 -- 172.21.15.42:0/4042398507 shutdown_connections 2024-05-28T14:08:24.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.377+0000 7fbd98c00640 1 --2- 172.21.15.42:0/4042398507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd940cf320 0x7fbd940cfea0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.377+0000 7fbd98c00640 1 --2- 172.21.15.42:0/4042398507 >> v2:172.21.15.63:3300/0 conn(0x7fbd940d0d50 0x7fbd941621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.377+0000 7fbd98c00640 1 -- 172.21.15.42:0/4042398507 >> 172.21.15.42:0/4042398507 conn(0x7fbd940b3ec0 msgr2=0x7fbd940b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:24.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.377+0000 7fbd98c00640 1 -- 172.21.15.42:0/4042398507 shutdown_connections 2024-05-28T14:08:24.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.377+0000 7fbd98c00640 1 -- 172.21.15.42:0/4042398507 wait complete. 2024-05-28T14:08:24.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.377+0000 7fbd98c00640 1 Processor -- start 2024-05-28T14:08:24.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.377+0000 7fbd98c00640 1 -- start start 2024-05-28T14:08:24.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd98c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd940cf320 0x7fbd9415e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:24.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd98c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd940d03e0 0x7fbd94149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:24.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd98c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd940d0d50 0x7fbd941faff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:24.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd98c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9415f980 con 0x7fbd940cf320 2024-05-28T14:08:24.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd98c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd940d1130 con 0x7fbd940d03e0 2024-05-28T14:08:24.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd98c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9415fb00 con 0x7fbd940d0d50 2024-05-28T14:08:24.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd93400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd940cf320 0x7fbd9415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:24.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd93400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd940cf320 0x7fbd9415e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45282/0 (socket says 172.21.15.42:45282) 2024-05-28T14:08:24.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd93400640 1 -- 172.21.15.42:0/2170671620 learned_addr learned my addr 172.21.15.42:0/2170671620 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:24.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd8be00640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd940d03e0 0x7fbd94149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:24.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd93e00640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd940d0d50 0x7fbd941faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:24.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd93400640 1 -- 172.21.15.42:0/2170671620 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd940d0d50 msgr2=0x7fbd941faff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:24.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd93400640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd940d0d50 0x7fbd941faff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd93400640 1 -- 172.21.15.42:0/2170671620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd940d03e0 msgr2=0x7fbd94149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:24.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd93400640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd940d03e0 0x7fbd94149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd93400640 1 -- 172.21.15.42:0/2170671620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd941497c0 con 0x7fbd940cf320 2024-05-28T14:08:24.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd93e00640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd940d0d50 0x7fbd941faff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:08:24.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd93400640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd940cf320 0x7fbd9415e430 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400d800 tx=0x7fbd8400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:24.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd8b400640 1 -- 172.21.15.42:0/2170671620 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84018020 con 0x7fbd940cf320 2024-05-28T14:08:24.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd8b400640 1 -- 172.21.15.42:0/2170671620 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd840096c0 con 0x7fbd940cf320 2024-05-28T14:08:24.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd8b400640 1 -- 172.21.15.42:0/2170671620 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84009ab0 con 0x7fbd940cf320 2024-05-28T14:08:24.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd98c00640 1 -- 172.21.15.42:0/2170671620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9415d670 con 0x7fbd940cf320 2024-05-28T14:08:24.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd98c00640 1 -- 172.21.15.42:0/2170671620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9415dc30 con 0x7fbd940cf320 2024-05-28T14:08:24.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.381+0000 7fbd98c00640 1 -- 172.21.15.42:0/2170671620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd94068a90 con 0x7fbd940cf320 2024-05-28T14:08:24.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.385+0000 7fbd8b400640 1 -- 172.21.15.42:0/2170671620 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7fbd8401b020 con 0x7fbd940cf320 2024-05-28T14:08:24.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.385+0000 7fbd8b400640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd70069490 0x7fbd7006b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:24.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.385+0000 7fbd8b400640 1 -- 172.21.15.42:0/2170671620 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fbd840859b0 con 0x7fbd940cf320 2024-05-28T14:08:24.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.385+0000 7fbd8be00640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd70069490 0x7fbd7006b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:24.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.385+0000 7fbd8be00640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd70069490 0x7fbd7006b950 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0ae000 tx=0x7fbd7c0a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:24.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.389+0000 7fbd8b400640 1 -- 172.21.15.42:0/2170671620 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbd84022020 con 0x7fbd940cf320 2024-05-28T14:08:24.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.613+0000 7fbd98c00640 1 -- 172.21.15.42:0/2170671620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbd940b2580 con 0x7fbd940cf320 2024-05-28T14:08:24.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.613+0000 7fbd8b400640 1 -- 172.21.15.42:0/2170671620 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2421 (secure 0 0 0) 0x7fbd840579c0 con 0x7fbd940cf320 2024-05-28T14:08:24.596 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:08:24.597 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:08:24.597 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:08:24.597 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:08:24.597 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:08:24.597 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:08:24.597 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:08:24.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:24.597 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:08:24.597 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:08:24.597 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:08:24.597 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:08:24.597 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:08:24.597 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:24.598 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:08:24.598 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:08:24.598 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:08:24.598 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:08:24.598 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:24.598 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 625, 2024-05-28T14:08:24.598 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:08:24.598 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:08:24.598 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:08:24.598 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:08:24.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:24.598 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:08:24.598 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 76, 2024-05-28T14:08:24.599 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:08:24.599 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:08:24.599 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:08:24.599 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:08:24.599 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:08:24.599 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:08:24.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:24.599 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:08:24.599 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:08:24.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:24.599 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:08:24.599 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:08:24.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:24.599 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 12687, 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 52995537556, 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 106222800896, 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 666871312384, 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 566563265, 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 558925978, 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 138, 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 149 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 47, 2024-05-28T14:08:24.600 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5, 2024-05-28T14:08:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:08:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:08:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:08:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:08:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 5, 2024-05-28T14:08:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:08:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "a", 2024-05-28T14:08:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:08:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 7083 2024-05-28T14:08:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:08:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:24.602 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:08:24.602 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:08:24.602 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:08:24.602 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:08:24.602 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:08:24.602 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:08:24.602 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:08:24.602 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:24.602 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:08:24.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:24.602 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:08:24.602 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 14, 2024-05-28T14:08:24.602 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:07:33.038180+0000", 2024-05-28T14:08:24.602 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:08:24.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:24.603 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:08:24.603 INFO:teuthology.orchestra.run.smithi042.stdout: "346e2288-a00b-4e81-8626-13f7a50e5c02": { 2024-05-28T14:08:24.603 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "2 ongoing+pending clones; Avg progress is 29.037% (69s)\n [========....................] (remaining: 2m)", 2024-05-28T14:08:24.603 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.29036998748779297, 2024-05-28T14:08:24.603 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:08:24.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:24.603 INFO:teuthology.orchestra.run.smithi042.stdout: "d3383476-f722-479f-9c40-e404f8ef3423": { 2024-05-28T14:08:24.603 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "2 ongoing clones; Avg progress is 29.037% (75s)\n [========....................] (remaining: 3m)", 2024-05-28T14:08:24.603 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.29036998748779297, 2024-05-28T14:08:24.603 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:08:24.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:24.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:24.603 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:08:24.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.613+0000 7fbd98c00640 1 -- 172.21.15.42:0/2170671620 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd70069490 msgr2=0x7fbd7006b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:24.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.613+0000 7fbd98c00640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd70069490 0x7fbd7006b950 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0ae000 tx=0x7fbd7c0a7000 comp rx=0 tx=0).stop 2024-05-28T14:08:24.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.613+0000 7fbd98c00640 1 -- 172.21.15.42:0/2170671620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd940cf320 msgr2=0x7fbd9415e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:24.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.613+0000 7fbd98c00640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd940cf320 0x7fbd9415e430 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400d800 tx=0x7fbd8400dcc0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.613+0000 7fbd98c00640 1 -- 172.21.15.42:0/2170671620 shutdown_connections 2024-05-28T14:08:24.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.613+0000 7fbd98c00640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbd70069490 0x7fbd7006b950 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.613+0000 7fbd98c00640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd940d0d50 0x7fbd941faff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.613+0000 7fbd98c00640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd940d03e0 0x7fbd94149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.613+0000 7fbd98c00640 1 --2- 172.21.15.42:0/2170671620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd940cf320 0x7fbd9415e430 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:24.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.613+0000 7fbd98c00640 1 -- 172.21.15.42:0/2170671620 >> 172.21.15.42:0/2170671620 conn(0x7fbd940b3ec0 msgr2=0x7fbd94161290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:24.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.613+0000 7fbd98c00640 1 -- 172.21.15.42:0/2170671620 shutdown_connections 2024-05-28T14:08:24.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:24.613+0000 7fbd98c00640 1 -- 172.21.15.42:0/2170671620 wait complete. 2024-05-28T14:08:29.618 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:08:29.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28b5400640 1 Processor -- start 2024-05-28T14:08:29.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28b5400640 1 -- start start 2024-05-28T14:08:29.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28b5400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f28b00b52d0 0x7f28b009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:29.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28b5400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f28b009af90 0x7f28b00b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:29.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28b5400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28b00ccce0 con 0x7f28b015e9d0 2024-05-28T14:08:29.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28b5400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f28b00ccfc0 con 0x7f28b00b52d0 2024-05-28T14:08:29.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28b5400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f28b00cd2a0 con 0x7f28b009af90 2024-05-28T14:08:29.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28ade00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f28b00b52d0 0x7f28b009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:29.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28ade00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f28b00b52d0 0x7f28b009aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37200/0 (socket says 172.21.15.42:37200) 2024-05-28T14:08:29.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28ade00640 1 -- 172.21.15.42:0/2950244414 learned_addr learned my addr 172.21.15.42:0/2950244414 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:29.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28af200640 1 --2- 172.21.15.42:0/2950244414 >> v2:172.21.15.63:3300/0 conn(0x7f28b009af90 0x7f28b00b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:29.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28ade00640 1 -- 172.21.15.42:0/2950244414 >> v2:172.21.15.63:3300/0 conn(0x7f28b009af90 msgr2=0x7f28b00b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:29.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28ade00640 1 --2- 172.21.15.42:0/2950244414 >> v2:172.21.15.63:3300/0 conn(0x7f28b009af90 0x7f28b00b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:29.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28ade00640 1 -- 172.21.15.42:0/2950244414 >> v1:172.21.15.42:6789/0 conn(0x7f28b015e9d0 legacy=0x7f28b00d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:29.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.789+0000 7f28ade00640 1 -- 172.21.15.42:0/2950244414 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b00cd580 con 0x7f28b00b52d0 2024-05-28T14:08:29.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.793+0000 7f28ade00640 1 --2- 172.21.15.42:0/2950244414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b00b52d0 0x7f28b009aa50 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f28b00738a0 tx=0x7f289c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2586c2004e3abdca server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:29.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.793+0000 7f28ad400640 1 -- 172.21.15.42:0/2950244414 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c0b0070 con 0x7f28b00b52d0 2024-05-28T14:08:29.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.793+0000 7f28ad400640 1 -- 172.21.15.42:0/2950244414 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f289c0ab600 con 0x7f28b00b52d0 2024-05-28T14:08:29.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.793+0000 7f28ad400640 1 -- 172.21.15.42:0/2950244414 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c0ab9d0 con 0x7f28b00b52d0 2024-05-28T14:08:29.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.793+0000 7f28b5400640 1 -- 172.21.15.42:0/2950244414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b00b52d0 msgr2=0x7f28b009aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:29.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.793+0000 7f28b5400640 1 --2- 172.21.15.42:0/2950244414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b00b52d0 0x7f28b009aa50 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f28b00738a0 tx=0x7f289c01c520 comp rx=0 tx=0).stop 2024-05-28T14:08:29.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.793+0000 7f28b5400640 1 -- 172.21.15.42:0/2950244414 shutdown_connections 2024-05-28T14:08:29.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.793+0000 7f28b5400640 1 --2- 172.21.15.42:0/2950244414 >> v2:172.21.15.63:3300/0 conn(0x7f28b009af90 0x7f28b00b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:29.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.793+0000 7f28b5400640 1 --2- 172.21.15.42:0/2950244414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b00b52d0 0x7f28b009aa50 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:29.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.793+0000 7f28b5400640 1 -- 172.21.15.42:0/2950244414 >> 172.21.15.42:0/2950244414 conn(0x7f28b00b3f70 msgr2=0x7f28b00b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:29.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.793+0000 7f28b5400640 1 -- 172.21.15.42:0/2950244414 shutdown_connections 2024-05-28T14:08:29.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.793+0000 7f28b5400640 1 -- 172.21.15.42:0/2950244414 wait complete. 2024-05-28T14:08:29.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.793+0000 7f28b5400640 1 Processor -- start 2024-05-28T14:08:29.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.793+0000 7f28b5400640 1 -- start start 2024-05-28T14:08:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28b5400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28b009af90 0x7f28b0155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28b5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b00b52d0 0x7f28b0159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28b5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b015e9d0 0x7f28b015a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28b5400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f28b00cdf00 con 0x7f28b015e9d0 2024-05-28T14:08:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28b5400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28b00cdd80 con 0x7f28b00b52d0 2024-05-28T14:08:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28b5400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f28b00ce080 con 0x7f28b009af90 2024-05-28T14:08:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28af200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b015e9d0 0x7f28b015a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28af200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b015e9d0 0x7f28b015a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37208/0 (socket says 172.21.15.42:37208) 2024-05-28T14:08:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28af200640 1 -- 172.21.15.42:0/4090702612 learned_addr learned my addr 172.21.15.42:0/4090702612 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28ade00640 1 --2- 172.21.15.42:0/4090702612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b00b52d0 0x7f28b0159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:29.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28ae800640 1 --2- 172.21.15.42:0/4090702612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28b009af90 0x7f28b0155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:29.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28af200640 1 -- 172.21.15.42:0/4090702612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28b009af90 msgr2=0x7f28b0155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:29.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28af200640 1 --2- 172.21.15.42:0/4090702612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28b009af90 0x7f28b0155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:29.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28af200640 1 -- 172.21.15.42:0/4090702612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b00b52d0 msgr2=0x7f28b0159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:29.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28af200640 1 --2- 172.21.15.42:0/4090702612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b00b52d0 0x7f28b0159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:29.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28af200640 1 -- 172.21.15.42:0/4090702612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b01545d0 con 0x7f28b015e9d0 2024-05-28T14:08:29.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28af200640 1 --2- 172.21.15.42:0/4090702612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b015e9d0 0x7f28b015a0a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f28a8007e40 tx=0x7f28a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:29.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28a6a00640 1 -- 172.21.15.42:0/4090702612 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8014620 con 0x7f28b015e9d0 2024-05-28T14:08:29.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28a6a00640 1 -- 172.21.15.42:0/4090702612 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a8019070 con 0x7f28b015e9d0 2024-05-28T14:08:29.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28a6a00640 1 -- 172.21.15.42:0/4090702612 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8014a20 con 0x7f28b015e9d0 2024-05-28T14:08:29.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28b5400640 1 -- 172.21.15.42:0/4090702612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b0154870 con 0x7f28b015e9d0 2024-05-28T14:08:29.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28b5400640 1 -- 172.21.15.42:0/4090702612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b0154e30 con 0x7f28b015e9d0 2024-05-28T14:08:29.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.797+0000 7f28b5400640 1 -- 172.21.15.42:0/4090702612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b01f75a0 con 0x7f28b015e9d0 2024-05-28T14:08:29.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.801+0000 7f28a6a00640 1 -- 172.21.15.42:0/4090702612 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f28a8014bc0 con 0x7f28b015e9d0 2024-05-28T14:08:29.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.801+0000 7f28a6a00640 1 --2- 172.21.15.42:0/4090702612 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2890069460 0x7f289006b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:29.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.801+0000 7f28a6a00640 1 -- 172.21.15.42:0/4090702612 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f28a8085d10 con 0x7f28b015e9d0 2024-05-28T14:08:29.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.801+0000 7f28ae800640 1 --2- 172.21.15.42:0/4090702612 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2890069460 0x7f289006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:29.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.801+0000 7f28ae800640 1 --2- 172.21.15.42:0/4090702612 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2890069460 0x7f289006b920 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f28a0001770 tx=0x7f28a0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:29.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:29.805+0000 7f28a6a00640 1 -- 172.21.15.42:0/4090702612 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f28a80576d0 con 0x7f28b015e9d0 2024-05-28T14:08:30.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:30.029+0000 7f28b5400640 1 -- 172.21.15.42:0/4090702612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f28b00b4690 con 0x7f28b015e9d0 2024-05-28T14:08:30.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:30.029+0000 7f28a6a00640 1 -- 172.21.15.42:0/4090702612 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2381 (secure 0 0 0) 0x7f28a805b9d0 con 0x7f28b015e9d0 2024-05-28T14:08:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:08:30.014 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:08:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:08:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:08:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:08:30.014 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:08:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:08:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:08:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:08:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:08:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:08:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:08:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:08:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:08:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:08:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:08:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:30.015 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 631, 2024-05-28T14:08:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:08:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:08:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:08:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:08:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:08:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 76, 2024-05-28T14:08:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:08:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:08:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:08:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:08:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:08:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:08:30.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:30.017 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:08:30.017 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:08:30.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:30.017 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:08:30.017 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:08:30.017 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:30.017 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:30.017 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:08:30.017 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:08:30.017 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 13276, 2024-05-28T14:08:30.017 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 55470419995, 2024-05-28T14:08:30.017 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 111754944512, 2024-05-28T14:08:30.017 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 661339168768, 2024-05-28T14:08:30.017 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:08:30.018 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 480717974, 2024-05-28T14:08:30.018 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 486004461, 2024-05-28T14:08:30.018 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 118, 2024-05-28T14:08:30.018 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 132 2024-05-28T14:08:30.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:30.018 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:08:30.018 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 47, 2024-05-28T14:08:30.018 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5, 2024-05-28T14:08:30.018 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:08:30.018 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:08:30.018 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:08:30.018 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:08:30.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:30.019 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 5, 2024-05-28T14:08:30.019 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:08:30.019 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "a", 2024-05-28T14:08:30.019 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:08:30.019 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 7083 2024-05-28T14:08:30.019 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:30.019 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:30.019 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:08:30.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:30.019 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:08:30.019 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:08:30.019 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:08:30.019 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:08:30.019 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:08:30.020 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:08:30.020 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:08:30.020 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:30.020 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:08:30.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:30.020 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:08:30.020 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 14, 2024-05-28T14:08:30.020 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:07:33.038180+0000", 2024-05-28T14:08:30.020 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:08:30.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:30.020 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:08:30.020 INFO:teuthology.orchestra.run.smithi042.stdout: "346e2288-a00b-4e81-8626-13f7a50e5c02": { 2024-05-28T14:08:30.020 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 26.0 seconds before exiting (75s)\n [============================] ", 2024-05-28T14:08:30.020 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:08:30.021 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:08:30.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:30.021 INFO:teuthology.orchestra.run.smithi042.stdout: "d3383476-f722-479f-9c40-e404f8ef3423": { 2024-05-28T14:08:30.021 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 26.0 seconds before exiting (81s)\n [============================] ", 2024-05-28T14:08:30.021 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:08:30.021 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:08:30.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:30.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:30.021 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:08:30.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:30.033+0000 7f28b5400640 1 -- 172.21.15.42:0/4090702612 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2890069460 msgr2=0x7f289006b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:30.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:30.033+0000 7f28b5400640 1 --2- 172.21.15.42:0/4090702612 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2890069460 0x7f289006b920 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f28a0001770 tx=0x7f28a0009040 comp rx=0 tx=0).stop 2024-05-28T14:08:30.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:30.033+0000 7f28b5400640 1 -- 172.21.15.42:0/4090702612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b015e9d0 msgr2=0x7f28b015a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:30.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:30.033+0000 7f28b5400640 1 --2- 172.21.15.42:0/4090702612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b015e9d0 0x7f28b015a0a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f28a8007e40 tx=0x7f28a800e4b0 comp rx=0 tx=0).stop 2024-05-28T14:08:30.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:30.033+0000 7f28b5400640 1 -- 172.21.15.42:0/4090702612 shutdown_connections 2024-05-28T14:08:30.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:30.033+0000 7f28b5400640 1 --2- 172.21.15.42:0/4090702612 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2890069460 0x7f289006b920 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:30.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:30.033+0000 7f28b5400640 1 --2- 172.21.15.42:0/4090702612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b015e9d0 0x7f28b015a0a0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:30.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:30.033+0000 7f28b5400640 1 --2- 172.21.15.42:0/4090702612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b00b52d0 0x7f28b0159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:30.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:30.033+0000 7f28b5400640 1 --2- 172.21.15.42:0/4090702612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28b009af90 0x7f28b0155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:30.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:30.033+0000 7f28b5400640 1 -- 172.21.15.42:0/4090702612 >> 172.21.15.42:0/4090702612 conn(0x7f28b00b3f70 msgr2=0x7f28b009b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:30.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:30.033+0000 7f28b5400640 1 -- 172.21.15.42:0/4090702612 shutdown_connections 2024-05-28T14:08:30.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:30.033+0000 7f28b5400640 1 -- 172.21.15.42:0/4090702612 wait complete. 2024-05-28T14:08:35.037 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:08:35.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.205+0000 7fa0b3e00640 1 Processor -- start 2024-05-28T14:08:35.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.205+0000 7fa0b3e00640 1 -- start start 2024-05-28T14:08:35.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.209+0000 7fa0b3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa0b415a160 0x7fa0b415ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:35.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.209+0000 7fa0b3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0b40cef90 0x7fa0b40cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:35.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.209+0000 7fa0b3e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b40cf8e0 con 0x7fa0b41609d0 2024-05-28T14:08:35.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.209+0000 7fa0b3e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b415da10 con 0x7fa0b415a160 2024-05-28T14:08:35.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.209+0000 7fa0b3e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b4160e80 con 0x7fa0b40cef90 2024-05-28T14:08:35.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.209+0000 7fa0b2a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa0b415a160 0x7fa0b415ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:35.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.209+0000 7fa0b2a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa0b415a160 0x7fa0b415ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37212/0 (socket says 172.21.15.42:37212) 2024-05-28T14:08:35.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.209+0000 7fa0b2a00640 1 -- 172.21.15.42:0/1443032059 learned_addr learned my addr 172.21.15.42:0/1443032059 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:35.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.209+0000 7fa0b3400640 1 --2- 172.21.15.42:0/1443032059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0b40cef90 0x7fa0b40cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:35.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.209+0000 7fa0b2a00640 1 -- 172.21.15.42:0/1443032059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0b40cef90 msgr2=0x7fa0b40cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:35.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.209+0000 7fa0b2a00640 1 --2- 172.21.15.42:0/1443032059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0b40cef90 0x7fa0b40cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:35.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.209+0000 7fa0b2a00640 1 -- 172.21.15.42:0/1443032059 >> v1:172.21.15.63:6789/0 conn(0x7fa0b41609d0 legacy=0x7fa0b40cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:35.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.209+0000 7fa0b2a00640 1 -- 172.21.15.42:0/1443032059 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b415d170 con 0x7fa0b415a160 2024-05-28T14:08:35.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.213+0000 7fa0b2a00640 1 --2- 172.21.15.42:0/1443032059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b415a160 0x7fa0b415ca70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009870 tx=0x7fa0a401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6c6546432ac7ed server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:35.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.213+0000 7fa0b2000640 1 -- 172.21.15.42:0/1443032059 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a40a7070 con 0x7fa0b415a160 2024-05-28T14:08:35.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.213+0000 7fa0b2000640 1 -- 172.21.15.42:0/1443032059 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a40b0070 con 0x7fa0b415a160 2024-05-28T14:08:35.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.213+0000 7fa0b2000640 1 -- 172.21.15.42:0/1443032059 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a40ab890 con 0x7fa0b415a160 2024-05-28T14:08:35.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.213+0000 7fa0b3e00640 1 -- 172.21.15.42:0/1443032059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b415a160 msgr2=0x7fa0b415ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:35.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.213+0000 7fa0b3e00640 1 --2- 172.21.15.42:0/1443032059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b415a160 0x7fa0b415ca70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009870 tx=0x7fa0a401c560 comp rx=0 tx=0).stop 2024-05-28T14:08:35.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.213+0000 7fa0b3e00640 1 -- 172.21.15.42:0/1443032059 shutdown_connections 2024-05-28T14:08:35.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.213+0000 7fa0b3e00640 1 --2- 172.21.15.42:0/1443032059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0b40cef90 0x7fa0b40cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:35.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.213+0000 7fa0b3e00640 1 --2- 172.21.15.42:0/1443032059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b415a160 0x7fa0b415ca70 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:35.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.213+0000 7fa0b3e00640 1 -- 172.21.15.42:0/1443032059 >> 172.21.15.42:0/1443032059 conn(0x7fa0b40b3ec0 msgr2=0x7fa0b40b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:35.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.213+0000 7fa0b3e00640 1 -- 172.21.15.42:0/1443032059 shutdown_connections 2024-05-28T14:08:35.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.213+0000 7fa0b3e00640 1 -- 172.21.15.42:0/1443032059 wait complete. 2024-05-28T14:08:35.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.213+0000 7fa0b3e00640 1 Processor -- start 2024-05-28T14:08:35.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.213+0000 7fa0b3e00640 1 -- start start 2024-05-28T14:08:35.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b40cef90 0x7fa0b41541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:35.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0b3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b415a160 0x7fa0b415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:35.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0b3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0b41609d0 0x7fa0b415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:35.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0b3e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4169390 con 0x7fa0b415a160 2024-05-28T14:08:35.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0b3e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b40d05b0 con 0x7fa0b40cef90 2024-05-28T14:08:35.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0b3e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b4169510 con 0x7fa0b41609d0 2024-05-28T14:08:35.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b415a160 0x7fa0b415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:35.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b415a160 0x7fa0b415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37222/0 (socket says 172.21.15.42:37222) 2024-05-28T14:08:35.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0abe00640 1 -- 172.21.15.42:0/692357276 learned_addr learned my addr 172.21.15.42:0/692357276 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:35.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0b2a00640 1 --2- 172.21.15.42:0/692357276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b40cef90 0x7fa0b41541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:35.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0b3400640 1 --2- 172.21.15.42:0/692357276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0b41609d0 0x7fa0b415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:35.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0abe00640 1 -- 172.21.15.42:0/692357276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0b41609d0 msgr2=0x7fa0b415afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:35.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0abe00640 1 --2- 172.21.15.42:0/692357276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0b41609d0 0x7fa0b415afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:35.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0abe00640 1 -- 172.21.15.42:0/692357276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b40cef90 msgr2=0x7fa0b41541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:35.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0abe00640 1 --2- 172.21.15.42:0/692357276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b40cef90 0x7fa0b41541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:35.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0abe00640 1 -- 172.21.15.42:0/692357276 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b415b5f0 con 0x7fa0b415a160 2024-05-28T14:08:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0abe00640 1 --2- 172.21.15.42:0/692357276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b415a160 0x7fa0b415aa80 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00c530 tx=0x7fa09c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0aaa00640 1 -- 172.21.15.42:0/692357276 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c016020 con 0x7fa0b415a160 2024-05-28T14:08:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0aaa00640 1 -- 172.21.15.42:0/692357276 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa09c005220 con 0x7fa0b415a160 2024-05-28T14:08:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0aaa00640 1 -- 172.21.15.42:0/692357276 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c007030 con 0x7fa0b415a160 2024-05-28T14:08:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0b3e00640 1 -- 172.21.15.42:0/692357276 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b415b8d0 con 0x7fa0b415a160 2024-05-28T14:08:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0b3e00640 1 -- 172.21.15.42:0/692357276 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b415be90 con 0x7fa0b415a160 2024-05-28T14:08:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0b3e00640 1 -- 172.21.15.42:0/692357276 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b4068a90 con 0x7fa0b415a160 2024-05-28T14:08:35.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.217+0000 7fa0aaa00640 1 -- 172.21.15.42:0/692357276 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7fa09c0071d0 con 0x7fa0b415a160 2024-05-28T14:08:35.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.221+0000 7fa0aaa00640 1 --2- 172.21.15.42:0/692357276 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa090069460 0x7fa09006b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:35.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.221+0000 7fa0aaa00640 1 -- 172.21.15.42:0/692357276 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fa09c084e40 con 0x7fa0b415a160 2024-05-28T14:08:35.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.221+0000 7fa0b2a00640 1 --2- 172.21.15.42:0/692357276 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa090069460 0x7fa09006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:35.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.221+0000 7fa0b2a00640 1 --2- 172.21.15.42:0/692357276 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa090069460 0x7fa09006b920 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40af000 tx=0x7fa0a40a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.225+0000 7fa0aaa00640 1 -- 172.21.15.42:0/692357276 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa09c020020 con 0x7fa0b415a160 2024-05-28T14:08:35.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.445+0000 7fa0b3e00640 1 -- 172.21.15.42:0/692357276 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa0b4155ce0 con 0x7fa0b415a160 2024-05-28T14:08:35.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.445+0000 7fa0aaa00640 1 -- 172.21.15.42:0/692357276 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2379 (secure 0 0 0) 0x7fa0b40cf8e0 con 0x7fa0b415a160 2024-05-28T14:08:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:08:35.429 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:08:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:08:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:08:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:08:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:08:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:08:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:08:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:08:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:08:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:08:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:08:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:08:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:08:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:08:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:08:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 636, 2024-05-28T14:08:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:08:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:08:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:08:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:08:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:08:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 76, 2024-05-28T14:08:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:08:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:08:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:08:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:08:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:08:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:08:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:08:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:08:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:08:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:08:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:08:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:08:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 13273, 2024-05-28T14:08:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 55470457918, 2024-05-28T14:08:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 111754977280, 2024-05-28T14:08:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 661339136000, 2024-05-28T14:08:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:08:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 201943908, 2024-05-28T14:08:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 206162569, 2024-05-28T14:08:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 50, 2024-05-28T14:08:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 59 2024-05-28T14:08:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:08:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 47, 2024-05-28T14:08:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5, 2024-05-28T14:08:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:08:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:08:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:08:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:08:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 5, 2024-05-28T14:08:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:08:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "a", 2024-05-28T14:08:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:08:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 7083 2024-05-28T14:08:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:08:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:08:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:08:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:08:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:08:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:08:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:08:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:08:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:08:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:08:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 14, 2024-05-28T14:08:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:07:33.038180+0000", 2024-05-28T14:08:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:08:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:08:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "346e2288-a00b-4e81-8626-13f7a50e5c02": { 2024-05-28T14:08:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 22.0 seconds before exiting (79s)\n [============================] ", 2024-05-28T14:08:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:08:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:08:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "d3383476-f722-479f-9c40-e404f8ef3423": { 2024-05-28T14:08:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 22.0 seconds before exiting (85s)\n [============================] ", 2024-05-28T14:08:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:08:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:08:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:35.436 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:08:35.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.445+0000 7fa0b3e00640 1 -- 172.21.15.42:0/692357276 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa090069460 msgr2=0x7fa09006b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:35.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.445+0000 7fa0b3e00640 1 --2- 172.21.15.42:0/692357276 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa090069460 0x7fa09006b920 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40af000 tx=0x7fa0a40a7000 comp rx=0 tx=0).stop 2024-05-28T14:08:35.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.445+0000 7fa0b3e00640 1 -- 172.21.15.42:0/692357276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b415a160 msgr2=0x7fa0b415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:35.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.445+0000 7fa0b3e00640 1 --2- 172.21.15.42:0/692357276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b415a160 0x7fa0b415aa80 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00c530 tx=0x7fa09c00c9f0 comp rx=0 tx=0).stop 2024-05-28T14:08:35.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.445+0000 7fa0b3e00640 1 -- 172.21.15.42:0/692357276 shutdown_connections 2024-05-28T14:08:35.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.445+0000 7fa0b3e00640 1 --2- 172.21.15.42:0/692357276 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa090069460 0x7fa09006b920 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:35.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.445+0000 7fa0b3e00640 1 --2- 172.21.15.42:0/692357276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0b41609d0 0x7fa0b415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:35.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.445+0000 7fa0b3e00640 1 --2- 172.21.15.42:0/692357276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b415a160 0x7fa0b415aa80 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:35.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.445+0000 7fa0b3e00640 1 --2- 172.21.15.42:0/692357276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b40cef90 0x7fa0b41541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:35.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.445+0000 7fa0b3e00640 1 -- 172.21.15.42:0/692357276 >> 172.21.15.42:0/692357276 conn(0x7fa0b40b3ec0 msgr2=0x7fa0b40b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:35.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.449+0000 7fa0b3e00640 1 -- 172.21.15.42:0/692357276 shutdown_connections 2024-05-28T14:08:35.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:35.449+0000 7fa0b3e00640 1 -- 172.21.15.42:0/692357276 wait complete. 2024-05-28T14:08:40.451 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:08:40.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.621+0000 7f6cb7a00640 1 Processor -- start 2024-05-28T14:08:40.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.621+0000 7f6cb7a00640 1 -- start start 2024-05-28T14:08:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.621+0000 7f6cb7a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cb009b160 0x7f6cb00b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.621+0000 7f6cb7a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cb00b5450 con 0x7f6cb00d02f0 2024-05-28T14:08:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.621+0000 7f6cb7a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cb00b5730 con 0x7f6cb015c710 2024-05-28T14:08:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.621+0000 7f6cb7a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb00b6060 con 0x7f6cb009b160 2024-05-28T14:08:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.625+0000 7f6cafe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f6cb00d02f0 0x7f6cb009aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51048/0 (socket says 172.21.15.42:51048) 2024-05-28T14:08:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.625+0000 7f6cafe00640 1 -- 172.21.15.42:0/4265116874 learned_addr learned my addr 172.21.15.42:0/4265116874 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:08:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.625+0000 7f6cb5a00640 1 --2- 172.21.15.42:0/4265116874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cb009b160 0x7f6cb00b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.625+0000 7f6caf400640 1 -- 172.21.15.42:0/4265116874 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3633508534 0 0) 0x7f6cb00b5450 con 0x7f6cb00d02f0 2024-05-28T14:08:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.625+0000 7f6caf400640 1 -- 172.21.15.42:0/4265116874 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c98003530 con 0x7f6cb00d02f0 2024-05-28T14:08:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.625+0000 7f6caf400640 1 -- 172.21.15.42:0/4265116874 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1882319427 0 0) 0x7f6cb00b5730 con 0x7f6cb015c710 2024-05-28T14:08:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.625+0000 7f6caf400640 1 -- 172.21.15.42:0/4265116874 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cb00b5450 con 0x7f6cb015c710 2024-05-28T14:08:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.625+0000 7f6cb5a00640 1 -- 172.21.15.42:0/4265116874 >> v1:172.21.15.63:6789/0 conn(0x7f6cb015c710 legacy=0x7f6cb009b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.625+0000 7f6cb5a00640 1 -- 172.21.15.42:0/4265116874 >> v1:172.21.15.42:6789/0 conn(0x7f6cb00d02f0 legacy=0x7f6cb009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.625+0000 7f6cb5a00640 1 -- 172.21.15.42:0/4265116874 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb00b5a10 con 0x7f6cb009b160 2024-05-28T14:08:40.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.625+0000 7f6cb5a00640 1 --2- 172.21.15.42:0/4265116874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cb009b160 0x7f6cb00b3090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009dc0 tx=0x7f6ca801f170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81f41158756f5c47 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6caf400640 1 -- 172.21.15.42:0/4265116874 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80b4070 con 0x7f6cb009b160 2024-05-28T14:08:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6caf400640 1 -- 172.21.15.42:0/4265116874 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca80af600 con 0x7f6cb009b160 2024-05-28T14:08:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6caf400640 1 -- 172.21.15.42:0/4265116874 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80af9d0 con 0x7f6cb009b160 2024-05-28T14:08:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 -- 172.21.15.42:0/4265116874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cb009b160 msgr2=0x7f6cb00b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 --2- 172.21.15.42:0/4265116874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cb009b160 0x7f6cb00b3090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009dc0 tx=0x7f6ca801f170 comp rx=0 tx=0).stop 2024-05-28T14:08:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 -- 172.21.15.42:0/4265116874 shutdown_connections 2024-05-28T14:08:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 --2- 172.21.15.42:0/4265116874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cb009b160 0x7f6cb00b3090 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 -- 172.21.15.42:0/4265116874 >> 172.21.15.42:0/4265116874 conn(0x7f6cb00b3f70 msgr2=0x7f6cb00b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 -- 172.21.15.42:0/4265116874 shutdown_connections 2024-05-28T14:08:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 -- 172.21.15.42:0/4265116874 wait complete. 2024-05-28T14:08:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 Processor -- start 2024-05-28T14:08:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 -- start start 2024-05-28T14:08:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb009b160 0x7f6cb015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cb00d02f0 0x7f6cb00c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cb015c710 0x7f6cb00b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb00b7680 con 0x7f6cb009b160 2024-05-28T14:08:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb00b7500 con 0x7f6cb00d02f0 2024-05-28T14:08:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb7a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb00b7800 con 0x7f6cb015c710 2024-05-28T14:08:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb009b160 0x7f6cb015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb009b160 0x7f6cb015baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42796/0 (socket says 172.21.15.42:42796) 2024-05-28T14:08:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb5000640 1 -- 172.21.15.42:0/1098281069 learned_addr learned my addr 172.21.15.42:0/1098281069 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:40.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cb5a00640 1 --2- 172.21.15.42:0/1098281069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cb015c710 0x7f6cb00b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.629+0000 7f6cafe00640 1 --2- 172.21.15.42:0/1098281069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cb00d02f0 0x7f6cb00c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cb5000640 1 -- 172.21.15.42:0/1098281069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cb015c710 msgr2=0x7f6cb00b7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cb5000640 1 --2- 172.21.15.42:0/1098281069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cb015c710 0x7f6cb00b7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cb5000640 1 -- 172.21.15.42:0/1098281069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cb00d02f0 msgr2=0x7f6cb00c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cb5000640 1 --2- 172.21.15.42:0/1098281069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cb00d02f0 0x7f6cb00c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cb5000640 1 -- 172.21.15.42:0/1098281069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb00c6a10 con 0x7f6cb009b160 2024-05-28T14:08:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cb5000640 1 --2- 172.21.15.42:0/1098281069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb009b160 0x7f6cb015baa0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400c430 tx=0x7f6ca400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cacc00640 1 -- 172.21.15.42:0/1098281069 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4017070 con 0x7f6cb009b160 2024-05-28T14:08:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cacc00640 1 -- 172.21.15.42:0/1098281069 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca40052f0 con 0x7f6cb009b160 2024-05-28T14:08:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cb7a00640 1 -- 172.21.15.42:0/1098281069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb00c6cf0 con 0x7f6cb009b160 2024-05-28T14:08:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cacc00640 1 -- 172.21.15.42:0/1098281069 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4003840 con 0x7f6cb009b160 2024-05-28T14:08:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cb7a00640 1 -- 172.21.15.42:0/1098281069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb00c8830 con 0x7f6cb009b160 2024-05-28T14:08:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cb7a00640 1 -- 172.21.15.42:0/1098281069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb0068a90 con 0x7f6cb009b160 2024-05-28T14:08:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cacc00640 1 -- 172.21.15.42:0/1098281069 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f6ca401a070 con 0x7f6cb009b160 2024-05-28T14:08:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cacc00640 1 --2- 172.21.15.42:0/1098281069 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c90069460 0x7f6c9006b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.633+0000 7f6cacc00640 1 -- 172.21.15.42:0/1098281069 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f6ca4084b60 con 0x7f6cb009b160 2024-05-28T14:08:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.637+0000 7f6cafe00640 1 --2- 172.21.15.42:0/1098281069 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c90069460 0x7f6c9006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:40.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.637+0000 7f6cafe00640 1 --2- 172.21.15.42:0/1098281069 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c90069460 0x7f6c9006b920 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00c85b0 tx=0x7f6c9c007d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:40.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.641+0000 7f6cacc00640 1 -- 172.21.15.42:0/1098281069 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6ca40561e0 con 0x7f6cb009b160 2024-05-28T14:08:40.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.861+0000 7f6cb7a00640 1 -- 172.21.15.42:0/1098281069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6cb009b950 con 0x7f6cb009b160 2024-05-28T14:08:40.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.861+0000 7f6cacc00640 1 -- 172.21.15.42:0/1098281069 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2366 (secure 0 0 0) 0x7f6ca405a4e0 con 0x7f6cb009b160 2024-05-28T14:08:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:08:40.847 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:08:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:08:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:08:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:08:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:08:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:08:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:08:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:08:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:08:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:08:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:08:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:08:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:08:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:08:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:08:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 642, 2024-05-28T14:08:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:08:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:08:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:08:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:08:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:08:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 76, 2024-05-28T14:08:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:08:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:08:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:08:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:08:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:08:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:08:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:08:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:08:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:08:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:08:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:08:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:08:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 13273, 2024-05-28T14:08:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 55470497969, 2024-05-28T14:08:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 111755059200, 2024-05-28T14:08:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 661339054080, 2024-05-28T14:08:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:08:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 511, 2024-05-28T14:08:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 7933, 2024-05-28T14:08:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:08:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 2 2024-05-28T14:08:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:08:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 47, 2024-05-28T14:08:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5, 2024-05-28T14:08:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:08:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:08:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:08:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:08:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 5, 2024-05-28T14:08:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:08:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "a", 2024-05-28T14:08:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:08:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 7083 2024-05-28T14:08:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:08:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:08:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:08:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:08:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:08:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:08:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:08:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:08:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:08:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:08:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 14, 2024-05-28T14:08:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:07:33.038180+0000", 2024-05-28T14:08:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:08:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:08:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "346e2288-a00b-4e81-8626-13f7a50e5c02": { 2024-05-28T14:08:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 16.0 seconds before exiting (85s)\n [============================] ", 2024-05-28T14:08:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:08:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:08:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "d3383476-f722-479f-9c40-e404f8ef3423": { 2024-05-28T14:08:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 16.0 seconds before exiting (91s)\n [============================] ", 2024-05-28T14:08:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:08:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:08:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:40.861 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:08:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.865+0000 7f6cb7a00640 1 -- 172.21.15.42:0/1098281069 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c90069460 msgr2=0x7f6c9006b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.865+0000 7f6cb7a00640 1 --2- 172.21.15.42:0/1098281069 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c90069460 0x7f6c9006b920 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00c85b0 tx=0x7f6c9c007d40 comp rx=0 tx=0).stop 2024-05-28T14:08:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.865+0000 7f6cb7a00640 1 -- 172.21.15.42:0/1098281069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb009b160 msgr2=0x7f6cb015baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.865+0000 7f6cb7a00640 1 --2- 172.21.15.42:0/1098281069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb009b160 0x7f6cb015baa0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400c430 tx=0x7f6ca400c8f0 comp rx=0 tx=0).stop 2024-05-28T14:08:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.865+0000 7f6cb7a00640 1 -- 172.21.15.42:0/1098281069 shutdown_connections 2024-05-28T14:08:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.865+0000 7f6cb7a00640 1 --2- 172.21.15.42:0/1098281069 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6c90069460 0x7f6c9006b920 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.865+0000 7f6cb7a00640 1 --2- 172.21.15.42:0/1098281069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cb015c710 0x7f6cb00b7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:40.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.865+0000 7f6cb7a00640 1 --2- 172.21.15.42:0/1098281069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cb00d02f0 0x7f6cb00c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:40.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.865+0000 7f6cb7a00640 1 --2- 172.21.15.42:0/1098281069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb009b160 0x7f6cb015baa0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:40.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.865+0000 7f6cb7a00640 1 -- 172.21.15.42:0/1098281069 >> 172.21.15.42:0/1098281069 conn(0x7f6cb00b3f70 msgr2=0x7f6cb0070720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:40.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.865+0000 7f6cb7a00640 1 -- 172.21.15.42:0/1098281069 shutdown_connections 2024-05-28T14:08:40.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:40.865+0000 7f6cb7a00640 1 -- 172.21.15.42:0/1098281069 wait complete. 2024-05-28T14:08:45.870 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:08:46.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8ca2800640 1 Processor -- start 2024-05-28T14:08:46.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8ca2800640 1 -- start start 2024-05-28T14:08:46.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8ca2800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c9c09b160 0x7f8c9c06cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:46.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8ca2800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8c9c0b1b10 0x7f8c9c0d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:46.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8ca2800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c9c06d5b0 con 0x7f8c9c0b3ad0 2024-05-28T14:08:46.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8ca2800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8c9c0cd8f0 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8ca2800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8c9c06d890 con 0x7f8c9c09b160 2024-05-28T14:08:46.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8c9be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8c9c0b1b10 0x7f8c9c0d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:46.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8c9be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8c9c0b1b10 0x7f8c9c0d1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42798/0 (socket says 172.21.15.42:42798) 2024-05-28T14:08:46.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8c9be00640 1 -- 172.21.15.42:0/3384688055 learned_addr learned my addr 172.21.15.42:0/3384688055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:46.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8c92a00640 1 --2- 172.21.15.42:0/3384688055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c9c09b160 0x7f8c9c06cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:46.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8c9be00640 1 -- 172.21.15.42:0/3384688055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c9c09b160 msgr2=0x7f8c9c06cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:46.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8c9be00640 1 --2- 172.21.15.42:0/3384688055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c9c09b160 0x7f8c9c06cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:46.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8c9be00640 1 -- 172.21.15.42:0/3384688055 >> v1:172.21.15.63:6789/0 conn(0x7f8c9c0b3ad0 legacy=0x7f8c9c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:46.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.041+0000 7f8c9be00640 1 -- 172.21.15.42:0/3384688055 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9c0cd580 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.045+0000 7f8c9be00640 1 --2- 172.21.15.42:0/3384688055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c9c0b1b10 0x7f8c9c0d1760 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009e50 tx=0x7f8c9401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824c9c7f5e3c8ae8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:46.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.045+0000 7f8c9aa00640 1 -- 172.21.15.42:0/3384688055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c940a5700 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.045+0000 7f8c9aa00640 1 -- 172.21.15.42:0/3384688055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c940a58a0 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.045+0000 7f8c9aa00640 1 -- 172.21.15.42:0/3384688055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c940a5c70 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.045+0000 7f8ca2800640 1 -- 172.21.15.42:0/3384688055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c9c0b1b10 msgr2=0x7f8c9c0d1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:46.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.045+0000 7f8ca2800640 1 --2- 172.21.15.42:0/3384688055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c9c0b1b10 0x7f8c9c0d1760 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009e50 tx=0x7f8c9401ca30 comp rx=0 tx=0).stop 2024-05-28T14:08:46.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.045+0000 7f8ca2800640 1 -- 172.21.15.42:0/3384688055 shutdown_connections 2024-05-28T14:08:46.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.045+0000 7f8ca2800640 1 --2- 172.21.15.42:0/3384688055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c9c0b1b10 0x7f8c9c0d1760 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:46.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.045+0000 7f8ca2800640 1 --2- 172.21.15.42:0/3384688055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c9c09b160 0x7f8c9c06cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:46.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.045+0000 7f8ca2800640 1 -- 172.21.15.42:0/3384688055 >> 172.21.15.42:0/3384688055 conn(0x7f8c9c0b3f70 msgr2=0x7f8c9c0b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:46.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.045+0000 7f8ca2800640 1 -- 172.21.15.42:0/3384688055 shutdown_connections 2024-05-28T14:08:46.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.045+0000 7f8ca2800640 1 -- 172.21.15.42:0/3384688055 wait complete. 2024-05-28T14:08:46.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8ca2800640 1 Processor -- start 2024-05-28T14:08:46.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8ca2800640 1 -- start start 2024-05-28T14:08:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8ca2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c9c09b160 0x7f8c9c159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8ca2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c9c0b1b10 0x7f8c9c15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8ca2800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c9c0b3ad0 0x7f8c9c0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8ca2800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c0ce900 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8ca2800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c0ce780 con 0x7f8c9c09b160 2024-05-28T14:08:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8ca2800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8c9c159b90 con 0x7f8c9c0b3ad0 2024-05-28T14:08:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8c92a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c9c0b1b10 0x7f8c9c15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8c92a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c9c0b1b10 0x7f8c9c15bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42808/0 (socket says 172.21.15.42:42808) 2024-05-28T14:08:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8c92a00640 1 -- 172.21.15.42:0/1609497847 learned_addr learned my addr 172.21.15.42:0/1609497847 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8c9b400640 1 --2- 172.21.15.42:0/1609497847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c9c09b160 0x7f8c9c159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8c9be00640 1 --2- 172.21.15.42:0/1609497847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c9c0b3ad0 0x7f8c9c0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8c92a00640 1 -- 172.21.15.42:0/1609497847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c9c0b3ad0 msgr2=0x7f8c9c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8c92a00640 1 --2- 172.21.15.42:0/1609497847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c9c0b3ad0 0x7f8c9c0cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8c92a00640 1 -- 172.21.15.42:0/1609497847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c9c09b160 msgr2=0x7f8c9c159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:46.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8c92a00640 1 --2- 172.21.15.42:0/1609497847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c9c09b160 0x7f8c9c159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:46.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8c92a00640 1 -- 172.21.15.42:0/1609497847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9c15c760 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8c92a00640 1 --2- 172.21.15.42:0/1609497847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c9c0b1b10 0x7f8c9c15bd80 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400d5f0 tx=0x7f8c8400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:46.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8c93e00640 1 -- 172.21.15.42:0/1609497847 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c840125f0 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8c93e00640 1 -- 172.21.15.42:0/1609497847 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c84019070 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8c93e00640 1 -- 172.21.15.42:0/1609497847 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c840129b0 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8ca2800640 1 -- 172.21.15.42:0/1609497847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9c15ca40 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8ca2800640 1 -- 172.21.15.42:0/1609497847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c9c15d000 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.049+0000 7f8ca2800640 1 -- 172.21.15.42:0/1609497847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c9c068a90 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.053+0000 7f8c93e00640 1 -- 172.21.15.42:0/1609497847 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f8c84012b50 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.053+0000 7f8c93e00640 1 --2- 172.21.15.42:0/1609497847 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c78069460 0x7f8c7806b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:46.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.053+0000 7f8c93e00640 1 -- 172.21.15.42:0/1609497847 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f8c840858d0 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.053+0000 7f8c9b400640 1 --2- 172.21.15.42:0/1609497847 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c78069460 0x7f8c7806b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:46.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.053+0000 7f8c9b400640 1 --2- 172.21.15.42:0/1609497847 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c78069460 0x7f8c7806b920 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c001810 tx=0x7f8c8c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.057+0000 7f8c93e00640 1 -- 172.21.15.42:0/1609497847 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8c840578e0 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.281+0000 7f8ca2800640 1 -- 172.21.15.42:0/1609497847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8c9c06fef0 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.281+0000 7f8c93e00640 1 -- 172.21.15.42:0/1609497847 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2335 (secure 0 0 0) 0x7f8c840571e0 con 0x7f8c9c0b1b10 2024-05-28T14:08:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 647, 2024-05-28T14:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:08:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 76, 2024-05-28T14:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 13273, 2024-05-28T14:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 55470505398, 2024-05-28T14:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 111755075584, 2024-05-28T14:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 661339037696, 2024-05-28T14:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 4265, 2024-05-28T14:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 47, 2024-05-28T14:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5, 2024-05-28T14:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 5, 2024-05-28T14:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "a", 2024-05-28T14:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 7083 2024-05-28T14:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 14, 2024-05-28T14:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:07:33.038180+0000", 2024-05-28T14:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "346e2288-a00b-4e81-8626-13f7a50e5c02": { 2024-05-28T14:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 10.0 seconds before exiting (91s)\n [============================] ", 2024-05-28T14:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "d3383476-f722-479f-9c40-e404f8ef3423": { 2024-05-28T14:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 10.0 seconds before exiting (97s)\n [============================] ", 2024-05-28T14:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:08:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:08:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:46.276 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:08:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.285+0000 7f8ca2800640 1 -- 172.21.15.42:0/1609497847 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c78069460 msgr2=0x7f8c7806b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.285+0000 7f8ca2800640 1 --2- 172.21.15.42:0/1609497847 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c78069460 0x7f8c7806b920 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c001810 tx=0x7f8c8c009040 comp rx=0 tx=0).stop 2024-05-28T14:08:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.285+0000 7f8ca2800640 1 -- 172.21.15.42:0/1609497847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c9c0b1b10 msgr2=0x7f8c9c15bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.285+0000 7f8ca2800640 1 --2- 172.21.15.42:0/1609497847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c9c0b1b10 0x7f8c9c15bd80 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400d5f0 tx=0x7f8c8400dab0 comp rx=0 tx=0).stop 2024-05-28T14:08:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.285+0000 7f8ca2800640 1 -- 172.21.15.42:0/1609497847 shutdown_connections 2024-05-28T14:08:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.285+0000 7f8ca2800640 1 --2- 172.21.15.42:0/1609497847 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c78069460 0x7f8c7806b920 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.285+0000 7f8ca2800640 1 --2- 172.21.15.42:0/1609497847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c9c0b3ad0 0x7f8c9c0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.285+0000 7f8ca2800640 1 --2- 172.21.15.42:0/1609497847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c9c0b1b10 0x7f8c9c15bd80 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:46.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.285+0000 7f8ca2800640 1 --2- 172.21.15.42:0/1609497847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c9c09b160 0x7f8c9c159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.285+0000 7f8ca2800640 1 -- 172.21.15.42:0/1609497847 >> 172.21.15.42:0/1609497847 conn(0x7f8c9c0b3f70 msgr2=0x7f8c9c06cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.285+0000 7f8ca2800640 1 -- 172.21.15.42:0/1609497847 shutdown_connections 2024-05-28T14:08:46.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:46.285+0000 7f8ca2800640 1 -- 172.21.15.42:0/1609497847 wait complete. 2024-05-28T14:08:49.252 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:08:49.255 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:08:51.289 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:08:51.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.457+0000 7f9fe7600640 1 Processor -- start 2024-05-28T14:08:51.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.457+0000 7f9fe7600640 1 -- start start 2024-05-28T14:08:51.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.461+0000 7f9fe7600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9fe015b760 0x7f9fe00cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:51.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.461+0000 7f9fe7600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9fe00cf970 0x7f9fe00c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:51.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.461+0000 7f9fe7600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fe00c5020 con 0x7f9fe00ce390 2024-05-28T14:08:51.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.461+0000 7f9fe7600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9fe00cff90 con 0x7f9fe015b760 2024-05-28T14:08:51.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.461+0000 7f9fe7600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9fe00d0270 con 0x7f9fe00cf970 2024-05-28T14:08:51.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.461+0000 7f9fe4c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9fe015b760 0x7f9fe00cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:51.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.461+0000 7f9fe4c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9fe015b760 0x7f9fe00cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50708/0 (socket says 172.21.15.42:50708) 2024-05-28T14:08:51.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.461+0000 7f9fe4c00640 1 -- 172.21.15.42:0/3280104341 learned_addr learned my addr 172.21.15.42:0/3280104341 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:51.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.461+0000 7f9fe5600640 1 --2- 172.21.15.42:0/3280104341 >> v2:172.21.15.63:3300/0 conn(0x7f9fe00cf970 0x7f9fe00c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:51.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.461+0000 7f9fe4c00640 1 -- 172.21.15.42:0/3280104341 >> v2:172.21.15.63:3300/0 conn(0x7f9fe00cf970 msgr2=0x7f9fe00c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:51.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.461+0000 7f9fe4c00640 1 --2- 172.21.15.42:0/3280104341 >> v2:172.21.15.63:3300/0 conn(0x7f9fe00cf970 0x7f9fe00c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:51.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.461+0000 7f9fe4c00640 1 -- 172.21.15.42:0/3280104341 >> v1:172.21.15.63:6789/0 conn(0x7f9fe00ce390 legacy=0x7f9fe00cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:51.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.461+0000 7f9fe4c00640 1 -- 172.21.15.42:0/3280104341 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe00c52c0 con 0x7f9fe015b760 2024-05-28T14:08:51.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.461+0000 7f9fe4c00640 1 --2- 172.21.15.42:0/3280104341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fe015b760 0x7f9fe00cde50 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0072a30 tx=0x7f9fd001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e171dcf86faf24 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:51.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fdfe00640 1 -- 172.21.15.42:0/3280104341 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd00b0070 con 0x7f9fe015b760 2024-05-28T14:08:51.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fdfe00640 1 -- 172.21.15.42:0/3280104341 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd00ab600 con 0x7f9fe015b760 2024-05-28T14:08:51.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fdfe00640 1 -- 172.21.15.42:0/3280104341 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd00ab9d0 con 0x7f9fe015b760 2024-05-28T14:08:51.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 -- 172.21.15.42:0/3280104341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fe015b760 msgr2=0x7f9fe00cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:51.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 --2- 172.21.15.42:0/3280104341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fe015b760 0x7f9fe00cde50 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0072a30 tx=0x7f9fd001c4e0 comp rx=0 tx=0).stop 2024-05-28T14:08:51.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 -- 172.21.15.42:0/3280104341 shutdown_connections 2024-05-28T14:08:51.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 --2- 172.21.15.42:0/3280104341 >> v2:172.21.15.63:3300/0 conn(0x7f9fe00cf970 0x7f9fe00c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:51.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 --2- 172.21.15.42:0/3280104341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fe015b760 0x7f9fe00cde50 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:51.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 -- 172.21.15.42:0/3280104341 >> 172.21.15.42:0/3280104341 conn(0x7f9fe009aa50 msgr2=0x7f9fe00b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:51.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 -- 172.21.15.42:0/3280104341 shutdown_connections 2024-05-28T14:08:51.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 -- 172.21.15.42:0/3280104341 wait complete. 2024-05-28T14:08:51.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 Processor -- start 2024-05-28T14:08:51.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 -- start start 2024-05-28T14:08:51.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe00ce390 0x7f9fe014cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fe00cf970 0x7f9fe014bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fe015b760 0x7f9fe014f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe00c6ba0 con 0x7f9fe00cf970 2024-05-28T14:08:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe00c6a20 con 0x7f9fe00ce390 2024-05-28T14:08:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe7600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe00c6d20 con 0x7f9fe015b760 2024-05-28T14:08:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fd7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fe00cf970 0x7f9fe014bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fd7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fe00cf970 0x7f9fe014bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50712/0 (socket says 172.21.15.42:50712) 2024-05-28T14:08:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fd7e00640 1 -- 172.21.15.42:0/269283463 learned_addr learned my addr 172.21.15.42:0/269283463 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.465+0000 7f9fe4c00640 1 --2- 172.21.15.42:0/269283463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe00ce390 0x7f9fe014cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fe5600640 1 --2- 172.21.15.42:0/269283463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fe015b760 0x7f9fe014f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:51.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fd7e00640 1 -- 172.21.15.42:0/269283463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fe015b760 msgr2=0x7f9fe014f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:51.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fd7e00640 1 --2- 172.21.15.42:0/269283463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fe015b760 0x7f9fe014f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:51.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fd7e00640 1 -- 172.21.15.42:0/269283463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe00ce390 msgr2=0x7f9fe014cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:51.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fd7e00640 1 --2- 172.21.15.42:0/269283463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe00ce390 0x7f9fe014cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:51.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fd7e00640 1 -- 172.21.15.42:0/269283463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe014d6e0 con 0x7f9fe00cf970 2024-05-28T14:08:51.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fd7e00640 1 --2- 172.21.15.42:0/269283463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fe00cf970 0x7f9fe014bde0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800c480 tx=0x7f9fc800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:51.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fdd600640 1 -- 172.21.15.42:0/269283463 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8012830 con 0x7f9fe00cf970 2024-05-28T14:08:51.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fdd600640 1 -- 172.21.15.42:0/269283463 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fc8017070 con 0x7f9fe00cf970 2024-05-28T14:08:51.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fdd600640 1 -- 172.21.15.42:0/269283463 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8012c30 con 0x7f9fe00cf970 2024-05-28T14:08:51.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fe7600640 1 -- 172.21.15.42:0/269283463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe015c100 con 0x7f9fe00cf970 2024-05-28T14:08:51.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fe7600640 1 -- 172.21.15.42:0/269283463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe015c6c0 con 0x7f9fe00cf970 2024-05-28T14:08:51.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fe7600640 1 -- 172.21.15.42:0/269283463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe0068a90 con 0x7f9fe00cf970 2024-05-28T14:08:51.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fdd600640 1 -- 172.21.15.42:0/269283463 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f9fc801c460 con 0x7f9fe00cf970 2024-05-28T14:08:51.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fdd600640 1 --2- 172.21.15.42:0/269283463 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fbc069460 0x7f9fbc06b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:51.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fdd600640 1 -- 172.21.15.42:0/269283463 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f9fc80852e0 con 0x7f9fe00cf970 2024-05-28T14:08:51.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.469+0000 7f9fe4c00640 1 --2- 172.21.15.42:0/269283463 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fbc069460 0x7f9fbc06b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:51.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.473+0000 7f9fe4c00640 1 --2- 172.21.15.42:0/269283463 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fbc069460 0x7f9fbc06b920 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00ad000 tx=0x7f9fd0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:51.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.477+0000 7f9fdd600640 1 -- 172.21.15.42:0/269283463 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9fc80572f0 con 0x7f9fe00cf970 2024-05-28T14:08:51.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.705+0000 7f9fe7600640 1 -- 172.21.15.42:0/269283463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9fe014c320 con 0x7f9fe00cf970 2024-05-28T14:08:51.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.705+0000 7f9fdd600640 1 -- 172.21.15.42:0/269283463 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2333 (secure 0 0 0) 0x7f9fc8056bf0 con 0x7f9fe00cf970 2024-05-28T14:08:51.688 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:08:51.688 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:08:51.689 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:08:51.689 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:08:51.689 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:08:51.689 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:08:51.689 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:08:51.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:51.689 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:08:51.690 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:08:51.690 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:08:51.690 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:08:51.690 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:08:51.690 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:51.690 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:08:51.691 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:08:51.691 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:08:51.691 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:08:51.691 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:51.691 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 653, 2024-05-28T14:08:51.691 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:08:51.691 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:08:51.692 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:08:51.692 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:08:51.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:51.692 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:08:51.692 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 76, 2024-05-28T14:08:51.692 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:08:51.692 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:08:51.692 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:08:51.692 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:08:51.693 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:08:51.693 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:08:51.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:51.693 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:08:51.693 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:08:51.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:51.693 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:08:51.693 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:08:51.693 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:51.693 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:51.693 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:08:51.693 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:08:51.693 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 13273, 2024-05-28T14:08:51.694 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 55470505398, 2024-05-28T14:08:51.694 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 111755075584, 2024-05-28T14:08:51.694 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 661339037696, 2024-05-28T14:08:51.694 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:08:51.694 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 767, 2024-05-28T14:08:51.694 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:08:51.694 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:08:51.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:51.694 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:08:51.694 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 47, 2024-05-28T14:08:51.694 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5, 2024-05-28T14:08:51.694 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:08:51.694 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 5, 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "a", 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 7083 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:08:51.695 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:08:51.696 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:08:51.696 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:08:51.696 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:08:51.696 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:08:51.696 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:51.696 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:08:51.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:51.696 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:08:51.696 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 14, 2024-05-28T14:08:51.696 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:07:33.038180+0000", 2024-05-28T14:08:51.696 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:08:51.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:51.696 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:08:51.696 INFO:teuthology.orchestra.run.smithi042.stdout: "346e2288-a00b-4e81-8626-13f7a50e5c02": { 2024-05-28T14:08:51.697 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 4.0 seconds before exiting (97s)\n [============================] ", 2024-05-28T14:08:51.697 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:08:51.697 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:08:51.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:51.697 INFO:teuthology.orchestra.run.smithi042.stdout: "d3383476-f722-479f-9c40-e404f8ef3423": { 2024-05-28T14:08:51.697 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 4.0 seconds before exiting (103s)\n [============================] ", 2024-05-28T14:08:51.697 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:08:51.697 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:08:51.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:51.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:51.697 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:08:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.705+0000 7f9fe7600640 1 -- 172.21.15.42:0/269283463 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fbc069460 msgr2=0x7f9fbc06b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.705+0000 7f9fe7600640 1 --2- 172.21.15.42:0/269283463 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fbc069460 0x7f9fbc06b920 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00ad000 tx=0x7f9fd0008040 comp rx=0 tx=0).stop 2024-05-28T14:08:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.705+0000 7f9fe7600640 1 -- 172.21.15.42:0/269283463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fe00cf970 msgr2=0x7f9fe014bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.705+0000 7f9fe7600640 1 --2- 172.21.15.42:0/269283463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fe00cf970 0x7f9fe014bde0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800c480 tx=0x7f9fc800c940 comp rx=0 tx=0).stop 2024-05-28T14:08:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.705+0000 7f9fe7600640 1 -- 172.21.15.42:0/269283463 shutdown_connections 2024-05-28T14:08:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.705+0000 7f9fe7600640 1 --2- 172.21.15.42:0/269283463 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fbc069460 0x7f9fbc06b920 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.705+0000 7f9fe7600640 1 --2- 172.21.15.42:0/269283463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fe015b760 0x7f9fe014f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.705+0000 7f9fe7600640 1 --2- 172.21.15.42:0/269283463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fe00cf970 0x7f9fe014bde0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.705+0000 7f9fe7600640 1 --2- 172.21.15.42:0/269283463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe00ce390 0x7f9fe014cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.705+0000 7f9fe7600640 1 -- 172.21.15.42:0/269283463 >> 172.21.15.42:0/269283463 conn(0x7f9fe009aa50 msgr2=0x7f9fe0157460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.705+0000 7f9fe7600640 1 -- 172.21.15.42:0/269283463 shutdown_connections 2024-05-28T14:08:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:51.705+0000 7f9fe7600640 1 -- 172.21.15.42:0/269283463 wait complete. 2024-05-28T14:08:56.711 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:08:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faeaa000640 1 Processor -- start 2024-05-28T14:08:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faeaa000640 1 -- start start 2024-05-28T14:08:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faeaa000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faea4163a10 0x7faea4163120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faeaa000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faea4163df0 0x7faea406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faeaa000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faea4072aa0 con 0x7faea4071ce0 2024-05-28T14:08:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faeaa000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faea406e240 con 0x7faea4163a10 2024-05-28T14:08:56.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faeaa000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faea406e470 con 0x7faea4163df0 2024-05-28T14:08:56.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faea2a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faea4163df0 0x7faea406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:56.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faea3400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faea4163a10 0x7faea4163120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:56.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faea2a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faea4163df0 0x7faea406dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:45538/0 (socket says 172.21.15.42:45538) 2024-05-28T14:08:56.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faea2a00640 1 -- 172.21.15.42:0/553549402 learned_addr learned my addr 172.21.15.42:0/553549402 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:56.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faea3e00640 1 --1- 172.21.15.42:0/553549402 >> v1:172.21.15.63:6789/0 conn(0x7faea4071ce0 0x7faea4166710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:47944/0 (socket says 172.21.15.42:47944) 2024-05-28T14:08:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faea3400640 1 -- 172.21.15.42:0/553549402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faea4163df0 msgr2=0x7faea406dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faea3400640 1 --2- 172.21.15.42:0/553549402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faea4163df0 0x7faea406dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faea3400640 1 -- 172.21.15.42:0/553549402 >> v1:172.21.15.63:6789/0 conn(0x7faea4071ce0 legacy=0x7faea4166710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faea3400640 1 -- 172.21.15.42:0/553549402 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea406e750 con 0x7faea4163a10 2024-05-28T14:08:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.881+0000 7faea2a00640 1 --2- 172.21.15.42:0/553549402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faea4163df0 0x7faea406dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:08:56.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.885+0000 7faea3400640 1 --2- 172.21.15.42:0/553549402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faea4163a10 0x7faea4163120 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fae940097c0 tx=0x7fae9401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae903d3ec50b6fd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:56.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.885+0000 7faea2000640 1 -- 172.21.15.42:0/553549402 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae940b2070 con 0x7faea4163a10 2024-05-28T14:08:56.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.885+0000 7faea2000640 1 -- 172.21.15.42:0/553549402 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae940ad640 con 0x7faea4163a10 2024-05-28T14:08:56.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.885+0000 7faea2000640 1 -- 172.21.15.42:0/553549402 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae940ada10 con 0x7faea4163a10 2024-05-28T14:08:56.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.885+0000 7faeaa000640 1 -- 172.21.15.42:0/553549402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faea4163a10 msgr2=0x7faea4163120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:56.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.885+0000 7faeaa000640 1 --2- 172.21.15.42:0/553549402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faea4163a10 0x7faea4163120 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fae940097c0 tx=0x7fae9401c4a0 comp rx=0 tx=0).stop 2024-05-28T14:08:56.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.885+0000 7faeaa000640 1 -- 172.21.15.42:0/553549402 shutdown_connections 2024-05-28T14:08:56.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.885+0000 7faeaa000640 1 --2- 172.21.15.42:0/553549402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faea4163df0 0x7faea406dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:56.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.885+0000 7faeaa000640 1 --2- 172.21.15.42:0/553549402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faea4163a10 0x7faea4163120 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:56.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.885+0000 7faeaa000640 1 -- 172.21.15.42:0/553549402 >> 172.21.15.42:0/553549402 conn(0x7faea406bff0 msgr2=0x7faea40b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:56.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.885+0000 7faeaa000640 1 -- 172.21.15.42:0/553549402 shutdown_connections 2024-05-28T14:08:56.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.885+0000 7faeaa000640 1 -- 172.21.15.42:0/553549402 wait complete. 2024-05-28T14:08:56.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faeaa000640 1 Processor -- start 2024-05-28T14:08:56.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faeaa000640 1 -- start start 2024-05-28T14:08:56.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faeaa000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faea4071ce0 0x7faea415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:56.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faeaa000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faea4163a10 0x7faea414c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:56.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faeaa000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faea4163df0 0x7faea415b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:56.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faeaa000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faea406f510 con 0x7faea4163df0 2024-05-28T14:08:56.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faeaa000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faea4072520 con 0x7faea4071ce0 2024-05-28T14:08:56.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faeaa000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faea415bbc0 con 0x7faea4163a10 2024-05-28T14:08:56.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faea3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faea4163df0 0x7faea415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:56.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faea3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faea4163df0 0x7faea415b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50726/0 (socket says 172.21.15.42:50726) 2024-05-28T14:08:56.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faea3e00640 1 -- 172.21.15.42:0/23326222 learned_addr learned my addr 172.21.15.42:0/23326222 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:56.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faea3400640 1 --2- 172.21.15.42:0/23326222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faea4071ce0 0x7faea415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:56.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faea2a00640 1 --2- 172.21.15.42:0/23326222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faea4163a10 0x7faea414c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:56.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faea3e00640 1 -- 172.21.15.42:0/23326222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faea4163a10 msgr2=0x7faea414c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:56.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faea3e00640 1 --2- 172.21.15.42:0/23326222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faea4163a10 0x7faea414c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:56.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faea3e00640 1 -- 172.21.15.42:0/23326222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faea4071ce0 msgr2=0x7faea415a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:56.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faea3e00640 1 --2- 172.21.15.42:0/23326222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faea4071ce0 0x7faea415a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:56.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faea3e00640 1 -- 172.21.15.42:0/23326222 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea409ded0 con 0x7faea4163df0 2024-05-28T14:08:56.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faea3e00640 1 --2- 172.21.15.42:0/23326222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faea4163df0 0x7faea415b050 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00c7a0 tx=0x7fae9c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:56.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7fae9b400640 1 -- 172.21.15.42:0/23326222 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c017070 con 0x7faea4163df0 2024-05-28T14:08:56.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7fae9b400640 1 -- 172.21.15.42:0/23326222 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae9c00a5e0 con 0x7faea4163df0 2024-05-28T14:08:56.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7fae9b400640 1 -- 172.21.15.42:0/23326222 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00a990 con 0x7faea4163df0 2024-05-28T14:08:56.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faeaa000640 1 -- 172.21.15.42:0/23326222 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea4159b20 con 0x7faea4163df0 2024-05-28T14:08:56.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.889+0000 7faeaa000640 1 -- 172.21.15.42:0/23326222 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea415a0e0 con 0x7faea4163df0 2024-05-28T14:08:56.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.893+0000 7faeaa000640 1 -- 172.21.15.42:0/23326222 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faea4068a90 con 0x7faea4163df0 2024-05-28T14:08:56.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.893+0000 7fae9b400640 1 -- 172.21.15.42:0/23326222 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7fae9c01a070 con 0x7faea4163df0 2024-05-28T14:08:56.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.893+0000 7fae9b400640 1 --2- 172.21.15.42:0/23326222 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fae84069460 0x7fae8406b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:56.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.893+0000 7fae9b400640 1 -- 172.21.15.42:0/23326222 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fae9c084e10 con 0x7faea4163df0 2024-05-28T14:08:56.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.893+0000 7faea3400640 1 --2- 172.21.15.42:0/23326222 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fae84069460 0x7fae8406b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:56.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.893+0000 7faea3400640 1 --2- 172.21.15.42:0/23326222 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fae84069460 0x7fae8406b920 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fae940af000 tx=0x7fae94008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:56.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:56.897+0000 7fae9b400640 1 -- 172.21.15.42:0/23326222 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fae9c056490 con 0x7faea4163df0 2024-05-28T14:08:57.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.125+0000 7faeaa000640 1 -- 172.21.15.42:0/23326222 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7faea40b2cb0 con 0x7faea4163df0 2024-05-28T14:08:57.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.125+0000 7fae9b400640 1 -- 172.21.15.42:0/23326222 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1733 (secure 0 0 0) 0x7fae9c05a790 con 0x7faea4163df0 2024-05-28T14:08:57.111 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:08:57.111 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:08:57.111 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:08:57.111 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:08:57.111 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:08:57.111 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:08:57.111 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:08:57.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.111 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:08:57.111 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:08:57.111 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:08:57.111 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:08:57.112 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:08:57.112 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:57.112 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:08:57.112 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:08:57.112 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:08:57.112 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:08:57.112 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:57.112 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 658, 2024-05-28T14:08:57.112 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:08:57.112 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:08:57.112 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:08:57.112 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:08:57.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:08:57.113 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 76, 2024-05-28T14:08:57.113 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:08:57.113 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:08:57.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:08:57.113 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:08:57.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:08:57.113 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:08:57.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.113 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:08:57.113 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:08:57.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.113 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:08:57.113 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:08:57.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:57.114 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:57.114 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:08:57.114 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:08:57.114 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 13273, 2024-05-28T14:08:57.114 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 55470505398, 2024-05-28T14:08:57.114 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 111755075584, 2024-05-28T14:08:57.114 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 661339037696, 2024-05-28T14:08:57.114 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280 2024-05-28T14:08:57.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.114 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:08:57.114 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 47, 2024-05-28T14:08:57.114 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5, 2024-05-28T14:08:57.115 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:08:57.115 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:08:57.115 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:08:57.115 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:08:57.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.115 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 5, 2024-05-28T14:08:57.115 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:08:57.115 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "a", 2024-05-28T14:08:57.115 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:08:57.115 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 7083 2024-05-28T14:08:57.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:57.115 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:57.115 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:08:57.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.116 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:08:57.116 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:08:57.116 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:08:57.116 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:08:57.116 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:08:57.116 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:08:57.116 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:08:57.116 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:08:57.116 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:08:57.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.116 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:08:57.116 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 14, 2024-05-28T14:08:57.116 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:07:33.038180+0000", 2024-05-28T14:08:57.117 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:08:57.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.117 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": {} 2024-05-28T14:08:57.117 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:08:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.129+0000 7faeaa000640 1 -- 172.21.15.42:0/23326222 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fae84069460 msgr2=0x7fae8406b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.129+0000 7faeaa000640 1 --2- 172.21.15.42:0/23326222 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fae84069460 0x7fae8406b920 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fae940af000 tx=0x7fae94008040 comp rx=0 tx=0).stop 2024-05-28T14:08:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.129+0000 7faeaa000640 1 -- 172.21.15.42:0/23326222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faea4163df0 msgr2=0x7faea415b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.129+0000 7faeaa000640 1 --2- 172.21.15.42:0/23326222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faea4163df0 0x7faea415b050 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00c7a0 tx=0x7fae9c00cc60 comp rx=0 tx=0).stop 2024-05-28T14:08:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.129+0000 7faeaa000640 1 -- 172.21.15.42:0/23326222 shutdown_connections 2024-05-28T14:08:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.129+0000 7faeaa000640 1 --2- 172.21.15.42:0/23326222 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fae84069460 0x7fae8406b920 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.129+0000 7faeaa000640 1 --2- 172.21.15.42:0/23326222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faea4163df0 0x7faea415b050 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.129+0000 7faeaa000640 1 --2- 172.21.15.42:0/23326222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faea4163a10 0x7faea414c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.129+0000 7faeaa000640 1 --2- 172.21.15.42:0/23326222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faea4071ce0 0x7faea415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.129+0000 7faeaa000640 1 -- 172.21.15.42:0/23326222 >> 172.21.15.42:0/23326222 conn(0x7faea406bff0 msgr2=0x7faea40b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:57.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.129+0000 7faeaa000640 1 -- 172.21.15.42:0/23326222 shutdown_connections 2024-05-28T14:08:57.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.129+0000 7faeaa000640 1 -- 172.21.15.42:0/23326222 wait complete. 2024-05-28T14:08:57.132 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-28T14:08:57.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e48400640 1 Processor -- start 2024-05-28T14:08:57.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e48400640 1 -- start start 2024-05-28T14:08:57.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e48400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e4015cb50 0x7f8e4009b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:57.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e48400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e4009ba40 con 0x7f8e400d11d0 2024-05-28T14:08:57.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e48400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e400c60d0 con 0x7f8e4015c770 2024-05-28T14:08:57.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e48400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8e400c6720 con 0x7f8e4015cb50 2024-05-28T14:08:57.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e46400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8e400d11d0 0x7f8e40166470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38060/0 (socket says 172.21.15.42:38060) 2024-05-28T14:08:57.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e46400640 1 -- 172.21.15.42:0/1189184659 learned_addr learned my addr 172.21.15.42:0/1189184659 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:08:57.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e45000640 1 --2- 172.21.15.42:0/1189184659 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e4015cb50 0x7f8e4009b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:57.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1641081615 0 0) 0x7f8e4009ba40 con 0x7f8e400d11d0 2024-05-28T14:08:57.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e28003530 con 0x7f8e400d11d0 2024-05-28T14:08:57.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2000127705 0 0) 0x7f8e400c60d0 con 0x7f8e4015c770 2024-05-28T14:08:57.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e4009ba40 con 0x7f8e4015c770 2024-05-28T14:08:57.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3114539674 0 0) 0x7f8e28003530 con 0x7f8e400d11d0 2024-05-28T14:08:57.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e4015cb50 msgr2=0x7f8e4009b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:57.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 --2- 172.21.15.42:0/1189184659 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e4015cb50 0x7f8e4009b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 >> v1:172.21.15.63:6789/0 conn(0x7f8e4015c770 legacy=0x7f8e400d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e400c63b0 con 0x7f8e400d11d0 2024-05-28T14:08:57.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f8e38001a00 con 0x7f8e400d11d0 2024-05-28T14:08:57.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 >> v1:172.21.15.42:6789/0 conn(0x7f8e400d11d0 legacy=0x7f8e40166470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 --2- 172.21.15.42:0/1189184659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e28003b80 0x7f8e28003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:57.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 --2- 172.21.15.42:0/1189184659 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e4015cb50 0x7f8e28004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:57.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 --2- 172.21.15.42:0/1189184659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e28004d50 0x7f8e280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:57.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8e400c6720 con 0x7f8e28004d50 2024-05-28T14:08:57.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2800ba80 con 0x7f8e28003b80 2024-05-28T14:08:57.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.293+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8e2800bc30 con 0x7f8e4015cb50 2024-05-28T14:08:57.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.297+0000 7f8e46400640 1 --2- 172.21.15.42:0/1189184659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e28004d50 0x7f8e280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:57.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.297+0000 7f8e45a00640 1 --2- 172.21.15.42:0/1189184659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e28003b80 0x7f8e28003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:57.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.297+0000 7f8e45000640 1 --2- 172.21.15.42:0/1189184659 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e4015cb50 0x7f8e28004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:57.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.297+0000 7f8e46400640 1 -- 172.21.15.42:0/1189184659 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e4015cb50 msgr2=0x7f8e28004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:57.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.297+0000 7f8e46400640 1 --2- 172.21.15.42:0/1189184659 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e4015cb50 0x7f8e28004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.297+0000 7f8e46400640 1 -- 172.21.15.42:0/1189184659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e28003b80 msgr2=0x7f8e28003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:57.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.297+0000 7f8e46400640 1 --2- 172.21.15.42:0/1189184659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e28003b80 0x7f8e28003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.297+0000 7f8e46400640 1 -- 172.21.15.42:0/1189184659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e28003530 con 0x7f8e28004d50 2024-05-28T14:08:57.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.297+0000 7f8e46400640 1 --2- 172.21.15.42:0/1189184659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e28004d50 0x7f8e280051d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800a7d0 tx=0x7f8e3801d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9201036339094692 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:57.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.297+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3800a430 con 0x7f8e28004d50 2024-05-28T14:08:57.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.297+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e380ad650 con 0x7f8e28004d50 2024-05-28T14:08:57.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.297+0000 7f8e3fe00640 1 -- 172.21.15.42:0/1189184659 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e380ada40 con 0x7f8e28004d50 2024-05-28T14:08:57.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e48400640 1 -- 172.21.15.42:0/1189184659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e28004d50 msgr2=0x7f8e280051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e48400640 1 --2- 172.21.15.42:0/1189184659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e28004d50 0x7f8e280051d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800a7d0 tx=0x7f8e3801d2e0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e46400640 1 -- 172.21.15.42:0/1189184659 reap_dead start 2024-05-28T14:08:57.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e48400640 1 -- 172.21.15.42:0/1189184659 shutdown_connections 2024-05-28T14:08:57.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e48400640 1 -- 172.21.15.42:0/1189184659 >> 172.21.15.42:0/1189184659 conn(0x7f8e400b3fd0 msgr2=0x7f8e400b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:57.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e48400640 1 -- 172.21.15.42:0/1189184659 shutdown_connections 2024-05-28T14:08:57.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e48400640 1 -- 172.21.15.42:0/1189184659 wait complete. 2024-05-28T14:08:57.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e48400640 1 Processor -- start 2024-05-28T14:08:57.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e48400640 1 -- start start 2024-05-28T14:08:57.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e48400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e28004d50 0x7f8e4015aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:57.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e48400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e400ca690 0x7f8e400cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:57.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e48400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e400b72b0 0x7f8e400b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:57.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e48400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8e400c8bc0 con 0x7f8e28004d50 2024-05-28T14:08:57.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e48400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8e400ce2c0 con 0x7f8e400ca690 2024-05-28T14:08:57.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e48400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8e400c8d40 con 0x7f8e400b72b0 2024-05-28T14:08:57.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e45a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e28004d50 0x7f8e4015aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:57.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e45a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e28004d50 0x7f8e4015aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50748/0 (socket says 172.21.15.42:50748) 2024-05-28T14:08:57.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e45a00640 1 -- 172.21.15.42:0/1888612627 learned_addr learned my addr 172.21.15.42:0/1888612627 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:57.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e45000640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e400ca690 0x7f8e400cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:57.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e46400640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e400b72b0 0x7f8e400b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:57.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e45a00640 1 -- 172.21.15.42:0/1888612627 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e400b72b0 msgr2=0x7f8e400b7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e45a00640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e400b72b0 0x7f8e400b7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e45a00640 1 -- 172.21.15.42:0/1888612627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e400ca690 msgr2=0x7f8e400cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e45a00640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e400ca690 0x7f8e400cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e45a00640 1 -- 172.21.15.42:0/1888612627 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4015b030 con 0x7f8e28004d50 2024-05-28T14:08:57.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.301+0000 7f8e45000640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e400ca690 0x7f8e400cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:08:57.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.305+0000 7f8e45a00640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e28004d50 0x7f8e4015aaf0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400f730 tx=0x7f8e3400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:57.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.305+0000 7f8e3d600640 1 -- 172.21.15.42:0/1888612627 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e340050d0 con 0x7f8e28004d50 2024-05-28T14:08:57.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.305+0000 7f8e3d600640 1 -- 172.21.15.42:0/1888612627 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e34018070 con 0x7f8e28004d50 2024-05-28T14:08:57.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.305+0000 7f8e3d600640 1 -- 172.21.15.42:0/1888612627 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400d6d0 con 0x7f8e28004d50 2024-05-28T14:08:57.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.305+0000 7f8e48400640 1 -- 172.21.15.42:0/1888612627 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e401f5a50 con 0x7f8e28004d50 2024-05-28T14:08:57.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.305+0000 7f8e48400640 1 -- 172.21.15.42:0/1888612627 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e400c99c0 con 0x7f8e28004d50 2024-05-28T14:08:57.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.305+0000 7f8e48400640 1 -- 172.21.15.42:0/1888612627 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e400b7c70 con 0x7f8e28004d50 2024-05-28T14:08:57.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.305+0000 7f8e3d600640 1 -- 172.21.15.42:0/1888612627 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f8e3400d870 con 0x7f8e28004d50 2024-05-28T14:08:57.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.305+0000 7f8e3d600640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e20069460 0x7f8e2006b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:57.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.305+0000 7f8e3d600640 1 -- 172.21.15.42:0/1888612627 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f8e34085340 con 0x7f8e28004d50 2024-05-28T14:08:57.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.305+0000 7f8e45000640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e20069460 0x7f8e2006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:57.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.309+0000 7f8e45000640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e20069460 0x7f8e2006b920 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8e400c7560 tx=0x7f8e2c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:57.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.309+0000 7f8e3d600640 1 -- 172.21.15.42:0/1888612627 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8e34057350 con 0x7f8e28004d50 2024-05-28T14:08:57.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.505+0000 7f8e48400640 1 -- 172.21.15.42:0/1888612627 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8e4004fdd0 con 0x7f8e20069460 2024-05-28T14:08:57.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.517+0000 7f8e3d600640 1 -- 172.21.15.42:0/1888612627 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1009 (secure 0 0 0) 0x7f8e4004fdd0 con 0x7f8e20069460 2024-05-28T14:08:57.501 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:08:57.501 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:08:57.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.501 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone8117" 2024-05-28T14:08:57.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.502 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone8116" 2024-05-28T14:08:57.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.502 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone81113" 2024-05-28T14:08:57.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.502 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone8111" 2024-05-28T14:08:57.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.502 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sv1" 2024-05-28T14:08:57.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.503 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone81115" 2024-05-28T14:08:57.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.503 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone81114" 2024-05-28T14:08:57.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.503 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone81112" 2024-05-28T14:08:57.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.503 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone8115" 2024-05-28T14:08:57.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.504 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone81118" 2024-05-28T14:08:57.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.504 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone81117" 2024-05-28T14:08:57.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.504 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone81111" 2024-05-28T14:08:57.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.504 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone8112" 2024-05-28T14:08:57.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.504 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone81116" 2024-05-28T14:08:57.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.505 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone81119" 2024-05-28T14:08:57.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.505 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone81110" 2024-05-28T14:08:57.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.505 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone8119" 2024-05-28T14:08:57.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.505 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone8114" 2024-05-28T14:08:57.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.506 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone8113" 2024-05-28T14:08:57.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.506 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone81120" 2024-05-28T14:08:57.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:08:57.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:57.506 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone8118" 2024-05-28T14:08:57.506 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:57.506 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:08:57.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.521+0000 7f8e48400640 1 -- 172.21.15.42:0/1888612627 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e20069460 msgr2=0x7f8e2006b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.521+0000 7f8e48400640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e20069460 0x7f8e2006b920 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8e400c7560 tx=0x7f8e2c009190 comp rx=0 tx=0).stop 2024-05-28T14:08:57.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.521+0000 7f8e48400640 1 -- 172.21.15.42:0/1888612627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e28004d50 msgr2=0x7f8e4015aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.521+0000 7f8e48400640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e28004d50 0x7f8e4015aaf0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400f730 tx=0x7f8e3400fb30 comp rx=0 tx=0).stop 2024-05-28T14:08:57.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.521+0000 7f8e48400640 1 -- 172.21.15.42:0/1888612627 shutdown_connections 2024-05-28T14:08:57.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.521+0000 7f8e48400640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e20069460 0x7f8e2006b920 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.521+0000 7f8e48400640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e400b72b0 0x7f8e400b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.521+0000 7f8e48400640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e400ca690 0x7f8e400cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.521+0000 7f8e48400640 1 --2- 172.21.15.42:0/1888612627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e28004d50 0x7f8e4015aaf0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.521+0000 7f8e48400640 1 -- 172.21.15.42:0/1888612627 >> 172.21.15.42:0/1888612627 conn(0x7f8e400b3fd0 msgr2=0x7f8e400cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:57.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.521+0000 7f8e48400640 1 -- 172.21.15.42:0/1888612627 shutdown_connections 2024-05-28T14:08:57.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.521+0000 7f8e48400640 1 -- 172.21.15.42:0/1888612627 wait complete. 2024-05-28T14:08:57.523 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone8117 2024-05-28T14:08:57.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.685+0000 7fc5ca400640 1 Processor -- start 2024-05-28T14:08:57.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.685+0000 7fc5ca400640 1 -- start start 2024-05-28T14:08:57.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.689+0000 7fc5ca400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc5c40d0e00 0x7fc5c406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:57.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.689+0000 7fc5ca400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c41570a0 con 0x7fc5c406ee80 2024-05-28T14:08:57.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.689+0000 7fc5ca400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c415cf00 con 0x7fc5c40afa70 2024-05-28T14:08:57.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.689+0000 7fc5ca400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c415d1e0 con 0x7fc5c40d0e00 2024-05-28T14:08:57.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.689+0000 7fc5c3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc5c40d0e00 0x7fc5c406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:57.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.689+0000 7fc5c3400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc5c406ee80 0x7fc5c40af360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38658/0 (socket says 172.21.15.42:38658) 2024-05-28T14:08:57.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.689+0000 7fc5c3400640 1 -- 172.21.15.42:0/851236380 learned_addr learned my addr 172.21.15.42:0/851236380 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:08:57.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.689+0000 7fc5c3e00640 1 -- 172.21.15.42:0/851236380 >> v1:172.21.15.63:6789/0 conn(0x7fc5c40afa70 legacy=0x7fc5c40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.689+0000 7fc5c3e00640 1 -- 172.21.15.42:0/851236380 >> v1:172.21.15.42:6789/0 conn(0x7fc5c406ee80 legacy=0x7fc5c40af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.689+0000 7fc5c3e00640 1 -- 172.21.15.42:0/851236380 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c415d4c0 con 0x7fc5c40d0e00 2024-05-28T14:08:57.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.689+0000 7fc5c2a00640 1 -- 172.21.15.42:0/851236380 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 368280285 0 0) 0x7fc5c41570a0 con 0x7fc5c406ee80 2024-05-28T14:08:57.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.689+0000 7fc5c3e00640 1 --2- 172.21.15.42:0/851236380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5c40d0e00 0x7fc5c406dd40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0097c0 tx=0x7fc5bc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1dcb4d5da588b7e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:57.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5c2a00640 1 -- 172.21.15.42:0/851236380 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0b0070 con 0x7fc5c40d0e00 2024-05-28T14:08:57.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5c2a00640 1 -- 172.21.15.42:0/851236380 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5bc0ab600 con 0x7fc5c40d0e00 2024-05-28T14:08:57.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5c2a00640 1 -- 172.21.15.42:0/851236380 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0ab9d0 con 0x7fc5c40d0e00 2024-05-28T14:08:57.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 -- 172.21.15.42:0/851236380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5c40d0e00 msgr2=0x7fc5c406dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 --2- 172.21.15.42:0/851236380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5c40d0e00 0x7fc5c406dd40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0097c0 tx=0x7fc5bc01c3e0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 -- 172.21.15.42:0/851236380 shutdown_connections 2024-05-28T14:08:57.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 --2- 172.21.15.42:0/851236380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5c40d0e00 0x7fc5c406dd40 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 -- 172.21.15.42:0/851236380 >> 172.21.15.42:0/851236380 conn(0x7fc5c406bdf0 msgr2=0x7fc5c406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:57.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 -- 172.21.15.42:0/851236380 shutdown_connections 2024-05-28T14:08:57.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 -- 172.21.15.42:0/851236380 wait complete. 2024-05-28T14:08:57.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 Processor -- start 2024-05-28T14:08:57.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 -- start start 2024-05-28T14:08:57.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5c406ee80 0x7fc5c4150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:57.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5c40afa70 0x7fc5c4152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5c40d0e00 0x7fc5c4149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c4160e10 con 0x7fc5c406ee80 2024-05-28T14:08:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c4160c90 con 0x7fc5c40d0e00 2024-05-28T14:08:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5ca400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c4160f90 con 0x7fc5c40afa70 2024-05-28T14:08:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5c3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5c406ee80 0x7fc5c4150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5c3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5c406ee80 0x7fc5c4150e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48528/0 (socket says 172.21.15.42:48528) 2024-05-28T14:08:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5c3400640 1 -- 172.21.15.42:0/1423401757 learned_addr learned my addr 172.21.15.42:0/1423401757 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5c3e00640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5c40d0e00 0x7fc5c4149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.693+0000 7fc5baa00640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5c40afa70 0x7fc5c4152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5c3400640 1 -- 172.21.15.42:0/1423401757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5c40afa70 msgr2=0x7fc5c4152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5c3400640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5c40afa70 0x7fc5c4152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5c3400640 1 -- 172.21.15.42:0/1423401757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5c40d0e00 msgr2=0x7fc5c4149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5c3400640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5c40d0e00 0x7fc5c4149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5c3400640 1 -- 172.21.15.42:0/1423401757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c4157140 con 0x7fc5c406ee80 2024-05-28T14:08:57.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5baa00640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5c40afa70 0x7fc5c4152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:08:57.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5c3e00640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5c40d0e00 0x7fc5c4149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:08:57.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5c3400640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5c406ee80 0x7fc5c4150e30 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400c8c0 tx=0x7fc5b400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:57.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5bbe00640 1 -- 172.21.15.42:0/1423401757 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4018070 con 0x7fc5c406ee80 2024-05-28T14:08:57.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5bbe00640 1 -- 172.21.15.42:0/1423401757 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b400a640 con 0x7fc5c406ee80 2024-05-28T14:08:57.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5bbe00640 1 -- 172.21.15.42:0/1423401757 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400aa30 con 0x7fc5c406ee80 2024-05-28T14:08:57.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5ca400640 1 -- 172.21.15.42:0/1423401757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c41573e0 con 0x7fc5c406ee80 2024-05-28T14:08:57.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5ca400640 1 -- 172.21.15.42:0/1423401757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c41579a0 con 0x7fc5c406ee80 2024-05-28T14:08:57.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5ca400640 1 -- 172.21.15.42:0/1423401757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c4068a90 con 0x7fc5c406ee80 2024-05-28T14:08:57.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5bbe00640 1 -- 172.21.15.42:0/1423401757 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7fc5b400abd0 con 0x7fc5c406ee80 2024-05-28T14:08:57.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5bbe00640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc5a0069460 0x7fc5a006b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:57.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5bbe00640 1 -- 172.21.15.42:0/1423401757 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fc5b4084c30 con 0x7fc5c406ee80 2024-05-28T14:08:57.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.697+0000 7fc5baa00640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc5a0069460 0x7fc5a006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:57.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.701+0000 7fc5baa00640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc5a0069460 0x7fc5a006b920 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc5c415e960 tx=0x7fc5ac009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:57.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.705+0000 7fc5bbe00640 1 -- 172.21.15.42:0/1423401757 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc5b4056c40 con 0x7fc5c406ee80 2024-05-28T14:08:57.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.893+0000 7fc5ca400640 1 -- 172.21.15.42:0/1423401757 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone8117", "target": ["mon-mgr", ""]}) -- 0x7fc5c4157be0 con 0x7fc5a0069460 2024-05-28T14:08:57.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.909+0000 7fc5bbe00640 1 -- 172.21.15.42:0/1423401757 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc5c4157be0 con 0x7fc5a0069460 2024-05-28T14:08:57.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.913+0000 7fc5ca400640 1 -- 172.21.15.42:0/1423401757 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc5a0069460 msgr2=0x7fc5a006b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.913+0000 7fc5ca400640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc5a0069460 0x7fc5a006b920 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc5c415e960 tx=0x7fc5ac009040 comp rx=0 tx=0).stop 2024-05-28T14:08:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.913+0000 7fc5ca400640 1 -- 172.21.15.42:0/1423401757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5c406ee80 msgr2=0x7fc5c4150e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.913+0000 7fc5ca400640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5c406ee80 0x7fc5c4150e30 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400c8c0 tx=0x7fc5b400cd80 comp rx=0 tx=0).stop 2024-05-28T14:08:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.913+0000 7fc5ca400640 1 -- 172.21.15.42:0/1423401757 shutdown_connections 2024-05-28T14:08:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.913+0000 7fc5ca400640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc5a0069460 0x7fc5a006b920 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.913+0000 7fc5ca400640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5c40d0e00 0x7fc5c4149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.913+0000 7fc5ca400640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5c40afa70 0x7fc5c4152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.913+0000 7fc5ca400640 1 --2- 172.21.15.42:0/1423401757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5c406ee80 0x7fc5c4150e30 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.913+0000 7fc5ca400640 1 -- 172.21.15.42:0/1423401757 >> 172.21.15.42:0/1423401757 conn(0x7fc5c406bdf0 msgr2=0x7fc5c40aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.913+0000 7fc5ca400640 1 -- 172.21.15.42:0/1423401757 shutdown_connections 2024-05-28T14:08:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:57.913+0000 7fc5ca400640 1 -- 172.21.15.42:0/1423401757 wait complete. 2024-05-28T14:08:57.899 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:57.917+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:08:57.899 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:57.917+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:08:57.899 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:57.917+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:08:57.899 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:57.917+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:08:57.899 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:57.917+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:08:57.916 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:57.933+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:08:57.916 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:57.933+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:08:57.916 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:57.933+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:08:57.916 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:57.933+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:08:57.916 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:08:57.933+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:08:57.924 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone8116 2024-05-28T14:08:58.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.089+0000 7fbfde000640 1 Processor -- start 2024-05-28T14:08:58.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfde000640 1 -- start start 2024-05-28T14:08:58.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfde000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbfd806ee80 0x7fbfd806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfde000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfd81570a0 con 0x7fbfd80d0e00 2024-05-28T14:08:58.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfde000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfd815cf00 con 0x7fbfd80af360 2024-05-28T14:08:58.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfde000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbfd815d1e0 con 0x7fbfd806ee80 2024-05-28T14:08:58.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfd7e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fbfd80d0e00 0x7fbfd8160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38670/0 (socket says 172.21.15.42:38670) 2024-05-28T14:08:58.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfd7e00640 1 -- 172.21.15.42:0/3834699584 learned_addr learned my addr 172.21.15.42:0/3834699584 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:08:58.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfd7400640 1 --2- 172.21.15.42:0/3834699584 >> v2:172.21.15.63:3300/0 conn(0x7fbfd806ee80 0x7fbfd806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfd6a00640 1 -- 172.21.15.42:0/3834699584 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4034574701 0 0) 0x7fbfd81570a0 con 0x7fbfd80d0e00 2024-05-28T14:08:58.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfd6a00640 1 -- 172.21.15.42:0/3834699584 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfbc003530 con 0x7fbfd80d0e00 2024-05-28T14:08:58.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfd6a00640 1 -- 172.21.15.42:0/3834699584 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 698252767 0 0) 0x7fbfd815cf00 con 0x7fbfd80af360 2024-05-28T14:08:58.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfd6a00640 1 -- 172.21.15.42:0/3834699584 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfd81570a0 con 0x7fbfd80af360 2024-05-28T14:08:58.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfd7400640 1 -- 172.21.15.42:0/3834699584 >> v1:172.21.15.63:6789/0 conn(0x7fbfd80af360 legacy=0x7fbfd80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfd7400640 1 -- 172.21.15.42:0/3834699584 >> v1:172.21.15.42:6789/0 conn(0x7fbfd80d0e00 legacy=0x7fbfd8160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.093+0000 7fbfd7400640 1 -- 172.21.15.42:0/3834699584 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd815d4c0 con 0x7fbfd806ee80 2024-05-28T14:08:58.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.097+0000 7fbfd7400640 1 --2- 172.21.15.42:0/3834699584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd806ee80 0x7fbfd806dd40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009d90 tx=0x7fbfc801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96f4c02ee11a00f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:58.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.097+0000 7fbfd6a00640 1 -- 172.21.15.42:0/3834699584 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80b2070 con 0x7fbfd806ee80 2024-05-28T14:08:58.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.097+0000 7fbfd6a00640 1 -- 172.21.15.42:0/3834699584 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc80a5990 con 0x7fbfd806ee80 2024-05-28T14:08:58.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.097+0000 7fbfd6a00640 1 -- 172.21.15.42:0/3834699584 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80a5d60 con 0x7fbfd806ee80 2024-05-28T14:08:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.097+0000 7fbfde000640 1 -- 172.21.15.42:0/3834699584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd806ee80 msgr2=0x7fbfd806dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.097+0000 7fbfde000640 1 --2- 172.21.15.42:0/3834699584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd806ee80 0x7fbfd806dd40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009d90 tx=0x7fbfc801c8b0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.097+0000 7fbfde000640 1 -- 172.21.15.42:0/3834699584 shutdown_connections 2024-05-28T14:08:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.097+0000 7fbfde000640 1 --2- 172.21.15.42:0/3834699584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd806ee80 0x7fbfd806dd40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.097+0000 7fbfde000640 1 -- 172.21.15.42:0/3834699584 >> 172.21.15.42:0/3834699584 conn(0x7fbfd806bdf0 msgr2=0x7fbfd806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.097+0000 7fbfde000640 1 -- 172.21.15.42:0/3834699584 shutdown_connections 2024-05-28T14:08:58.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.097+0000 7fbfde000640 1 -- 172.21.15.42:0/3834699584 wait complete. 2024-05-28T14:08:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.097+0000 7fbfde000640 1 Processor -- start 2024-05-28T14:08:58.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.097+0000 7fbfde000640 1 -- start start 2024-05-28T14:08:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfde000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbfd806ee80 0x7fbfd8152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfde000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd80af360 0x7fbfd8159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfde000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd80d0e00 0x7fbfd8159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfde000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd815cce0 con 0x7fbfd80d0e00 2024-05-28T14:08:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfde000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd815cb60 con 0x7fbfd80af360 2024-05-28T14:08:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfde000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd8160fa0 con 0x7fbfd806ee80 2024-05-28T14:08:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfd7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd80d0e00 0x7fbfd8159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfd7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd80d0e00 0x7fbfd8159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48574/0 (socket says 172.21.15.42:48574) 2024-05-28T14:08:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfd7e00640 1 -- 172.21.15.42:0/1164377655 learned_addr learned my addr 172.21.15.42:0/1164377655 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfd7400640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbfd806ee80 0x7fbfd8152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfcea00640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd80af360 0x7fbfd8159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfd7e00640 1 -- 172.21.15.42:0/1164377655 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbfd806ee80 msgr2=0x7fbfd8152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfd7e00640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbfd806ee80 0x7fbfd8152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfd7e00640 1 -- 172.21.15.42:0/1164377655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd80af360 msgr2=0x7fbfd8159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfd7e00640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd80af360 0x7fbfd8159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfd7e00640 1 -- 172.21.15.42:0/1164377655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd8166ce0 con 0x7fbfd80d0e00 2024-05-28T14:08:58.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfd7400640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbfd806ee80 0x7fbfd8152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:08:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfd7e00640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd80d0e00 0x7fbfd8159980 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000c3c0 tx=0x7fbfd000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfcfe00640 1 -- 172.21.15.42:0/1164377655 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0017070 con 0x7fbfd80d0e00 2024-05-28T14:08:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfcfe00640 1 -- 172.21.15.42:0/1164377655 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd0005db0 con 0x7fbfd80d0e00 2024-05-28T14:08:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfcfe00640 1 -- 172.21.15.42:0/1164377655 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0007310 con 0x7fbfd80d0e00 2024-05-28T14:08:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfde000640 1 -- 172.21.15.42:0/1164377655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd8166fc0 con 0x7fbfd80d0e00 2024-05-28T14:08:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfde000640 1 -- 172.21.15.42:0/1164377655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd80cd710 con 0x7fbfd80d0e00 2024-05-28T14:08:58.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.101+0000 7fbfde000640 1 -- 172.21.15.42:0/1164377655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd81674e0 con 0x7fbfd80d0e00 2024-05-28T14:08:58.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.105+0000 7fbfcfe00640 1 -- 172.21.15.42:0/1164377655 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7fbfd001a070 con 0x7fbfd80d0e00 2024-05-28T14:08:58.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.105+0000 7fbfcfe00640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfb4069490 0x7fbfb406b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.105+0000 7fbfcfe00640 1 -- 172.21.15.42:0/1164377655 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fbfd0084c30 con 0x7fbfd80d0e00 2024-05-28T14:08:58.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.105+0000 7fbfd7400640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfb4069490 0x7fbfb406b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.105+0000 7fbfd7400640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfb4069490 0x7fbfb406b950 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80af000 tx=0x7fbfc80a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:58.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.109+0000 7fbfcfe00640 1 -- 172.21.15.42:0/1164377655 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbfd0056900 con 0x7fbfd80d0e00 2024-05-28T14:08:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.301+0000 7fbfde000640 1 -- 172.21.15.42:0/1164377655 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone8116", "target": ["mon-mgr", ""]}) -- 0x7fbfd8068a90 con 0x7fbfb4069490 2024-05-28T14:08:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.317+0000 7fbfcfe00640 1 -- 172.21.15.42:0/1164377655 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbfd8068a90 con 0x7fbfb4069490 2024-05-28T14:08:58.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.321+0000 7fbfde000640 1 -- 172.21.15.42:0/1164377655 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfb4069490 msgr2=0x7fbfb406b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.321+0000 7fbfde000640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfb4069490 0x7fbfb406b950 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80af000 tx=0x7fbfc80a7070 comp rx=0 tx=0).stop 2024-05-28T14:08:58.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.321+0000 7fbfde000640 1 -- 172.21.15.42:0/1164377655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd80d0e00 msgr2=0x7fbfd8159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.321+0000 7fbfde000640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd80d0e00 0x7fbfd8159980 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000c3c0 tx=0x7fbfd000c880 comp rx=0 tx=0).stop 2024-05-28T14:08:58.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.321+0000 7fbfde000640 1 -- 172.21.15.42:0/1164377655 shutdown_connections 2024-05-28T14:08:58.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.321+0000 7fbfde000640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbfb4069490 0x7fbfb406b950 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.321+0000 7fbfde000640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd80d0e00 0x7fbfd8159980 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.321+0000 7fbfde000640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd80af360 0x7fbfd8159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.321+0000 7fbfde000640 1 --2- 172.21.15.42:0/1164377655 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbfd806ee80 0x7fbfd8152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.321+0000 7fbfde000640 1 -- 172.21.15.42:0/1164377655 >> 172.21.15.42:0/1164377655 conn(0x7fbfd806bdf0 msgr2=0x7fbfd80d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:58.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.321+0000 7fbfde000640 1 -- 172.21.15.42:0/1164377655 shutdown_connections 2024-05-28T14:08:58.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.321+0000 7fbfde000640 1 -- 172.21.15.42:0/1164377655 wait complete. 2024-05-28T14:08:58.328 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone81113 2024-05-28T14:08:58.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.489+0000 7f39da800640 1 Processor -- start 2024-05-28T14:08:58.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.489+0000 7f39da800640 1 -- start start 2024-05-28T14:08:58.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.493+0000 7f39da800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f39d406ee80 0x7f39d406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.493+0000 7f39da800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f39d40d0e00 0x7f39d40af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.493+0000 7f39da800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d41570a0 con 0x7f39d40af360 2024-05-28T14:08:58.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.493+0000 7f39da800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f39d415cf00 con 0x7f39d406ee80 2024-05-28T14:08:58.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.493+0000 7f39da800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f39d415d1e0 con 0x7f39d40d0e00 2024-05-28T14:08:58.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.493+0000 7f39d3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f39d406ee80 0x7f39d406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.493+0000 7f39d3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f39d406ee80 0x7f39d406dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48586/0 (socket says 172.21.15.42:48586) 2024-05-28T14:08:58.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.493+0000 7f39cbe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f39d40af360 0x7f39d40d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38686/0 (socket says 172.21.15.42:38686) 2024-05-28T14:08:58.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.493+0000 7f39d3400640 1 -- 172.21.15.42:0/3989544909 learned_addr learned my addr 172.21.15.42:0/3989544909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:58.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.493+0000 7f39d3e00640 1 --2- 172.21.15.42:0/3989544909 >> v2:172.21.15.63:3300/0 conn(0x7f39d40d0e00 0x7f39d40af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.493+0000 7f39d3400640 1 -- 172.21.15.42:0/3989544909 >> v2:172.21.15.63:3300/0 conn(0x7f39d40d0e00 msgr2=0x7f39d40af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:58.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.493+0000 7f39d3400640 1 --2- 172.21.15.42:0/3989544909 >> v2:172.21.15.63:3300/0 conn(0x7f39d40d0e00 0x7f39d40af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.493+0000 7f39d3400640 1 -- 172.21.15.42:0/3989544909 >> v1:172.21.15.42:6789/0 conn(0x7f39d40af360 legacy=0x7f39d40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.493+0000 7f39d3400640 1 -- 172.21.15.42:0/3989544909 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d415d4c0 con 0x7f39d406ee80 2024-05-28T14:08:58.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.497+0000 7f39d3400640 1 --2- 172.21.15.42:0/3989544909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39d406ee80 0x7f39d406dd40 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f39d409e4e0 tx=0x7f39c401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31654ce11dca206 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:58.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.497+0000 7f39d2a00640 1 -- 172.21.15.42:0/3989544909 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c40b1070 con 0x7f39d406ee80 2024-05-28T14:08:58.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.497+0000 7f39d2a00640 1 -- 172.21.15.42:0/3989544909 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c40ac620 con 0x7f39d406ee80 2024-05-28T14:08:58.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.497+0000 7f39d2a00640 1 -- 172.21.15.42:0/3989544909 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c40ac9f0 con 0x7f39d406ee80 2024-05-28T14:08:58.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.497+0000 7f39da800640 1 -- 172.21.15.42:0/3989544909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39d406ee80 msgr2=0x7f39d406dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.497+0000 7f39da800640 1 --2- 172.21.15.42:0/3989544909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39d406ee80 0x7f39d406dd40 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f39d409e4e0 tx=0x7f39c401c460 comp rx=0 tx=0).stop 2024-05-28T14:08:58.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.497+0000 7f39da800640 1 -- 172.21.15.42:0/3989544909 shutdown_connections 2024-05-28T14:08:58.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.497+0000 7f39da800640 1 --2- 172.21.15.42:0/3989544909 >> v2:172.21.15.63:3300/0 conn(0x7f39d40d0e00 0x7f39d40af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.497+0000 7f39da800640 1 --2- 172.21.15.42:0/3989544909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39d406ee80 0x7f39d406dd40 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.497+0000 7f39da800640 1 -- 172.21.15.42:0/3989544909 >> 172.21.15.42:0/3989544909 conn(0x7f39d406bdf0 msgr2=0x7f39d406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:58.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.497+0000 7f39da800640 1 -- 172.21.15.42:0/3989544909 shutdown_connections 2024-05-28T14:08:58.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.497+0000 7f39da800640 1 -- 172.21.15.42:0/3989544909 wait complete. 2024-05-28T14:08:58.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.497+0000 7f39da800640 1 Processor -- start 2024-05-28T14:08:58.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.497+0000 7f39da800640 1 -- start start 2024-05-28T14:08:58.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39da800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39d406ee80 0x7f39d413e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39da800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39d40af360 0x7f39d4071d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39da800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39d40d0e00 0x7f39d4073080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39da800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f39d4160e10 con 0x7f39d40d0e00 2024-05-28T14:08:58.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39da800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f39d4160c90 con 0x7f39d406ee80 2024-05-28T14:08:58.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39da800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f39d4160f90 con 0x7f39d40af360 2024-05-28T14:08:58.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39d3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39d40d0e00 0x7f39d4073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39d3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39d40d0e00 0x7f39d4073080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48592/0 (socket says 172.21.15.42:48592) 2024-05-28T14:08:58.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39d3e00640 1 -- 172.21.15.42:0/2811562163 learned_addr learned my addr 172.21.15.42:0/2811562163 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:58.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39cbe00640 1 --2- 172.21.15.42:0/2811562163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39d40af360 0x7f39d4071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39d3400640 1 --2- 172.21.15.42:0/2811562163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39d406ee80 0x7f39d413e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39d3e00640 1 -- 172.21.15.42:0/2811562163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39d40af360 msgr2=0x7f39d4071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39d3e00640 1 --2- 172.21.15.42:0/2811562163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39d40af360 0x7f39d4071d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39d3e00640 1 -- 172.21.15.42:0/2811562163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39d406ee80 msgr2=0x7f39d413e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39d3e00640 1 --2- 172.21.15.42:0/2811562163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39d406ee80 0x7f39d413e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39d3e00640 1 -- 172.21.15.42:0/2811562163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d40723b0 con 0x7f39d40d0e00 2024-05-28T14:08:58.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39d3e00640 1 --2- 172.21.15.42:0/2811562163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39d40d0e00 0x7f39d4073080 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00efc0 tx=0x7f39cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:58.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39cb400640 1 -- 172.21.15.42:0/2811562163 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc018020 con 0x7f39d40d0e00 2024-05-28T14:08:58.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39cb400640 1 -- 172.21.15.42:0/2811562163 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39cc014480 con 0x7f39d40d0e00 2024-05-28T14:08:58.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39da800640 1 -- 172.21.15.42:0/2811562163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d4072690 con 0x7f39d40d0e00 2024-05-28T14:08:58.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39cb400640 1 -- 172.21.15.42:0/2811562163 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc014870 con 0x7f39d40d0e00 2024-05-28T14:08:58.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39da800640 1 -- 172.21.15.42:0/2811562163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d4072c50 con 0x7f39d40d0e00 2024-05-28T14:08:58.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39da800640 1 -- 172.21.15.42:0/2811562163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d4068a90 con 0x7f39d40d0e00 2024-05-28T14:08:58.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.501+0000 7f39cb400640 1 -- 172.21.15.42:0/2811562163 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f39cc01b020 con 0x7f39d40d0e00 2024-05-28T14:08:58.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.505+0000 7f39cb400640 1 --2- 172.21.15.42:0/2811562163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f39b0069460 0x7f39b006b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.505+0000 7f39cb400640 1 -- 172.21.15.42:0/2811562163 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f39cc0860a0 con 0x7f39d40d0e00 2024-05-28T14:08:58.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.505+0000 7f39d3400640 1 --2- 172.21.15.42:0/2811562163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f39b0069460 0x7f39b006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.505+0000 7f39d3400640 1 --2- 172.21.15.42:0/2811562163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f39b0069460 0x7f39b006b920 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f39c40ae000 tx=0x7f39c4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:58.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.509+0000 7f39cb400640 1 -- 172.21.15.42:0/2811562163 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f39cc022070 con 0x7f39d40d0e00 2024-05-28T14:08:58.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.697+0000 7f39da800640 1 -- 172.21.15.42:0/2811562163 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone81113", "target": ["mon-mgr", ""]}) -- 0x7f39d40b5160 con 0x7f39b0069460 2024-05-28T14:08:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.717+0000 7f39cb400640 1 -- 172.21.15.42:0/2811562163 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f39d40b5160 con 0x7f39b0069460 2024-05-28T14:08:58.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.717+0000 7f39da800640 1 -- 172.21.15.42:0/2811562163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f39b0069460 msgr2=0x7f39b006b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.717+0000 7f39da800640 1 --2- 172.21.15.42:0/2811562163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f39b0069460 0x7f39b006b920 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f39c40ae000 tx=0x7f39c4008040 comp rx=0 tx=0).stop 2024-05-28T14:08:58.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.717+0000 7f39da800640 1 -- 172.21.15.42:0/2811562163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39d40d0e00 msgr2=0x7f39d4073080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.717+0000 7f39da800640 1 --2- 172.21.15.42:0/2811562163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39d40d0e00 0x7f39d4073080 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00efc0 tx=0x7f39cc00c490 comp rx=0 tx=0).stop 2024-05-28T14:08:58.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.717+0000 7f39da800640 1 -- 172.21.15.42:0/2811562163 shutdown_connections 2024-05-28T14:08:58.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.717+0000 7f39da800640 1 --2- 172.21.15.42:0/2811562163 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f39b0069460 0x7f39b006b920 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.717+0000 7f39da800640 1 --2- 172.21.15.42:0/2811562163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39d40d0e00 0x7f39d4073080 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.717+0000 7f39da800640 1 --2- 172.21.15.42:0/2811562163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39d40af360 0x7f39d4071d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.717+0000 7f39da800640 1 --2- 172.21.15.42:0/2811562163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39d406ee80 0x7f39d413e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.717+0000 7f39da800640 1 -- 172.21.15.42:0/2811562163 >> 172.21.15.42:0/2811562163 conn(0x7f39d406bdf0 msgr2=0x7f39d406f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:58.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.717+0000 7f39da800640 1 -- 172.21.15.42:0/2811562163 shutdown_connections 2024-05-28T14:08:58.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.717+0000 7f39da800640 1 -- 172.21.15.42:0/2811562163 wait complete. 2024-05-28T14:08:58.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone8111 2024-05-28T14:08:58.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.885+0000 7fba0d400640 1 Processor -- start 2024-05-28T14:08:58.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.885+0000 7fba0d400640 1 -- start start 2024-05-28T14:08:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.885+0000 7fba0d400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba080ae240 0x7fba0806c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.885+0000 7fba0d400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fba0806c570 0x7fba080ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.885+0000 7fba0d400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba080cd0c0 con 0x7fba0806bc50 2024-05-28T14:08:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.885+0000 7fba0d400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fba080cdcd0 con 0x7fba0806c570 2024-05-28T14:08:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.885+0000 7fba0d400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fba080cd3a0 con 0x7fba080ae240 2024-05-28T14:08:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.889+0000 7fba07200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fba0806c570 0x7fba080ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.889+0000 7fba06800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fba0806bc50 0x7fba080d0b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38698/0 (socket says 172.21.15.42:38698) 2024-05-28T14:08:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.889+0000 7fba06800640 1 -- 172.21.15.42:0/3834683713 learned_addr learned my addr 172.21.15.42:0/3834683713 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:08:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.889+0000 7fba07200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fba0806c570 0x7fba080ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48600/0 (socket says 172.21.15.42:48600) 2024-05-28T14:08:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.889+0000 7fb9fde00640 1 --2- 172.21.15.42:0/3834683713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba080ae240 0x7fba0806c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.889+0000 7fba07200640 1 -- 172.21.15.42:0/3834683713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba080ae240 msgr2=0x7fba0806c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.889+0000 7fba07200640 1 --2- 172.21.15.42:0/3834683713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba080ae240 0x7fba0806c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.889+0000 7fba07200640 1 -- 172.21.15.42:0/3834683713 >> v1:172.21.15.42:6789/0 conn(0x7fba0806bc50 legacy=0x7fba080d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.889+0000 7fba05e00640 1 -- 172.21.15.42:0/3834683713 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4160508493 0 0) 0x7fba080cd0c0 con 0x7fba0806bc50 2024-05-28T14:08:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.889+0000 7fba07200640 1 -- 172.21.15.42:0/3834683713 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba080cd680 con 0x7fba0806c570 2024-05-28T14:08:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.889+0000 7fba07200640 1 --2- 172.21.15.42:0/3834683713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba0806c570 0x7fba080ce160 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fba00009d90 tx=0x7fba0001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8076d415f0e29e07 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.889+0000 7fba05e00640 1 -- 172.21.15.42:0/3834683713 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba000b2070 con 0x7fba0806c570 2024-05-28T14:08:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.889+0000 7fba05e00640 1 -- 172.21.15.42:0/3834683713 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba000a5a50 con 0x7fba0806c570 2024-05-28T14:08:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.889+0000 7fba05e00640 1 -- 172.21.15.42:0/3834683713 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba000a5e20 con 0x7fba0806c570 2024-05-28T14:08:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 -- 172.21.15.42:0/3834683713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba0806c570 msgr2=0x7fba080ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 --2- 172.21.15.42:0/3834683713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba0806c570 0x7fba080ce160 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fba00009d90 tx=0x7fba0001c970 comp rx=0 tx=0).stop 2024-05-28T14:08:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 -- 172.21.15.42:0/3834683713 shutdown_connections 2024-05-28T14:08:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 --2- 172.21.15.42:0/3834683713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba0806c570 0x7fba080ce160 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 --2- 172.21.15.42:0/3834683713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba080ae240 0x7fba0806c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 -- 172.21.15.42:0/3834683713 >> 172.21.15.42:0/3834683713 conn(0x7fba080b2f60 msgr2=0x7fba080b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 -- 172.21.15.42:0/3834683713 shutdown_connections 2024-05-28T14:08:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 -- 172.21.15.42:0/3834683713 wait complete. 2024-05-28T14:08:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 Processor -- start 2024-05-28T14:08:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 -- start start 2024-05-28T14:08:58.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba0806bc50 0x7fba080c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba0806c570 0x7fba080c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba080ae240 0x7fba080c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fba08161280 con 0x7fba080ae240 2024-05-28T14:08:58.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba08161100 con 0x7fba0806c570 2024-05-28T14:08:58.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba0d400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fba08161400 con 0x7fba0806bc50 2024-05-28T14:08:58.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba07200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba080ae240 0x7fba080c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba07200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba080ae240 0x7fba080c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48604/0 (socket says 172.21.15.42:48604) 2024-05-28T14:08:58.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba07200640 1 -- 172.21.15.42:0/2432972309 learned_addr learned my addr 172.21.15.42:0/2432972309 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:58.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba06800640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba0806bc50 0x7fba080c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fb9fde00640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba0806c570 0x7fba080c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba07200640 1 -- 172.21.15.42:0/2432972309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba0806bc50 msgr2=0x7fba080c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba07200640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba0806bc50 0x7fba080c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba07200640 1 -- 172.21.15.42:0/2432972309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba0806c570 msgr2=0x7fba080c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:58.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba07200640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba0806c570 0x7fba080c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:58.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba07200640 1 -- 172.21.15.42:0/2432972309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba080c7de0 con 0x7fba080ae240 2024-05-28T14:08:58.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba06800640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba0806bc50 0x7fba080c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:08:58.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fb9fde00640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba0806c570 0x7fba080c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:08:58.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.893+0000 7fba07200640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba080ae240 0x7fba080c6760 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fba000b0040 tx=0x7fba00019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:58.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.897+0000 7fb9ff400640 1 -- 172.21.15.42:0/2432972309 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba000b2040 con 0x7fba080ae240 2024-05-28T14:08:58.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.897+0000 7fb9ff400640 1 -- 172.21.15.42:0/2432972309 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba000bf070 con 0x7fba080ae240 2024-05-28T14:08:58.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.897+0000 7fb9ff400640 1 -- 172.21.15.42:0/2432972309 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00009450 con 0x7fba080ae240 2024-05-28T14:08:58.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.897+0000 7fba0d400640 1 -- 172.21.15.42:0/2432972309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba08155a50 con 0x7fba080ae240 2024-05-28T14:08:58.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.897+0000 7fba0d400640 1 -- 172.21.15.42:0/2432972309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba080c5f60 con 0x7fba080ae240 2024-05-28T14:08:58.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.897+0000 7fba0d400640 1 -- 172.21.15.42:0/2432972309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba080c5a70 con 0x7fba080ae240 2024-05-28T14:08:58.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.897+0000 7fb9ff400640 1 -- 172.21.15.42:0/2432972309 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7fba000b86b0 con 0x7fba080ae240 2024-05-28T14:08:58.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.897+0000 7fb9ff400640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb9e4069490 0x7fb9e406b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:58.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.897+0000 7fb9ff400640 1 -- 172.21.15.42:0/2432972309 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fba00121480 con 0x7fba080ae240 2024-05-28T14:08:58.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.897+0000 7fba06800640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb9e4069490 0x7fb9e406b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:58.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.901+0000 7fba06800640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb9e4069490 0x7fb9e406b950 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0000f70 tx=0x7fb9f0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:58.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:58.901+0000 7fb9ff400640 1 -- 172.21.15.42:0/2432972309 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fba00016560 con 0x7fba080ae240 2024-05-28T14:08:59.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.097+0000 7fba0d400640 1 -- 172.21.15.42:0/2432972309 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone8111", "target": ["mon-mgr", ""]}) -- 0x7fba080af2d0 con 0x7fb9e4069490 2024-05-28T14:08:59.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.117+0000 7fb9ff400640 1 -- 172.21.15.42:0/2432972309 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fba080af2d0 con 0x7fb9e4069490 2024-05-28T14:08:59.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.121+0000 7fba0d400640 1 -- 172.21.15.42:0/2432972309 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb9e4069490 msgr2=0x7fb9e406b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:59.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.121+0000 7fba0d400640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb9e4069490 0x7fb9e406b950 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0000f70 tx=0x7fb9f0009040 comp rx=0 tx=0).stop 2024-05-28T14:08:59.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.121+0000 7fba0d400640 1 -- 172.21.15.42:0/2432972309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba080ae240 msgr2=0x7fba080c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.121+0000 7fba0d400640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba080ae240 0x7fba080c6760 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fba000b0040 tx=0x7fba00019710 comp rx=0 tx=0).stop 2024-05-28T14:08:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.121+0000 7fba0d400640 1 -- 172.21.15.42:0/2432972309 shutdown_connections 2024-05-28T14:08:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.121+0000 7fba0d400640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb9e4069490 0x7fb9e406b950 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.121+0000 7fba0d400640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba080ae240 0x7fba080c6760 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.121+0000 7fba0d400640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba0806c570 0x7fba080c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.121+0000 7fba0d400640 1 --2- 172.21.15.42:0/2432972309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba0806bc50 0x7fba080c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.121+0000 7fba0d400640 1 -- 172.21.15.42:0/2432972309 >> 172.21.15.42:0/2432972309 conn(0x7fba080b2f60 msgr2=0x7fba080afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.121+0000 7fba0d400640 1 -- 172.21.15.42:0/2432972309 shutdown_connections 2024-05-28T14:08:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.121+0000 7fba0d400640 1 -- 172.21.15.42:0/2432972309 wait complete. 2024-05-28T14:08:59.126 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-28T14:08:59.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f9990800640 1 Processor -- start 2024-05-28T14:08:59.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f9990800640 1 -- start start 2024-05-28T14:08:59.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f9990800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f998806ee80 0x7f998806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:59.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f9990800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f99880af360 0x7f99880af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:59.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f9990800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99881570a0 con 0x7f99880d06f0 2024-05-28T14:08:59.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f9990800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f998815cf00 con 0x7f99880af360 2024-05-28T14:08:59.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f9990800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f998815d1e0 con 0x7f998806ee80 2024-05-28T14:08:59.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f998e800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f99880d06f0 0x7f9988160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38712/0 (socket says 172.21.15.42:38712) 2024-05-28T14:08:59.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f998e800640 1 -- 172.21.15.42:0/135869455 learned_addr learned my addr 172.21.15.42:0/135869455 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:08:59.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f998de00640 1 --2- 172.21.15.42:0/135869455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f998806ee80 0x7f998806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:59.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f998d400640 1 --2- 172.21.15.42:0/135869455 >> v2:172.21.15.63:3300/0 conn(0x7f99880af360 0x7f99880af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:59.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f998ca00640 1 -- 172.21.15.42:0/135869455 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3765543783 0 0) 0x7f99881570a0 con 0x7f99880d06f0 2024-05-28T14:08:59.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f998ca00640 1 -- 172.21.15.42:0/135869455 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9970003530 con 0x7f99880d06f0 2024-05-28T14:08:59.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f998de00640 1 -- 172.21.15.42:0/135869455 >> v2:172.21.15.63:3300/0 conn(0x7f99880af360 msgr2=0x7f99880af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:59.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f998de00640 1 --2- 172.21.15.42:0/135869455 >> v2:172.21.15.63:3300/0 conn(0x7f99880af360 0x7f99880af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f998de00640 1 -- 172.21.15.42:0/135869455 >> v1:172.21.15.42:6789/0 conn(0x7f99880d06f0 legacy=0x7f9988160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:59.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f998de00640 1 -- 172.21.15.42:0/135869455 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998815d4c0 con 0x7f998806ee80 2024-05-28T14:08:59.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.289+0000 7f998d400640 1 --2- 172.21.15.42:0/135869455 >> v2:172.21.15.63:3300/0 conn(0x7f99880af360 0x7f99880af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:08:59.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.293+0000 7f998de00640 1 --2- 172.21.15.42:0/135869455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f998806ee80 0x7f998806dd40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f997c0097c0 tx=0x7f997c01c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73326d889aa2415d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:59.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.293+0000 7f998ca00640 1 -- 172.21.15.42:0/135869455 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c0b0070 con 0x7f998806ee80 2024-05-28T14:08:59.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.293+0000 7f998ca00640 1 -- 172.21.15.42:0/135869455 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f997c0ab600 con 0x7f998806ee80 2024-05-28T14:08:59.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.293+0000 7f998ca00640 1 -- 172.21.15.42:0/135869455 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c0a4e60 con 0x7f998806ee80 2024-05-28T14:08:59.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.293+0000 7f9990800640 1 -- 172.21.15.42:0/135869455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f998806ee80 msgr2=0x7f998806dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:59.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.293+0000 7f9990800640 1 --2- 172.21.15.42:0/135869455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f998806ee80 0x7f998806dd40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f997c0097c0 tx=0x7f997c01c380 comp rx=0 tx=0).stop 2024-05-28T14:08:59.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.293+0000 7f9990800640 1 -- 172.21.15.42:0/135869455 shutdown_connections 2024-05-28T14:08:59.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.293+0000 7f9990800640 1 --2- 172.21.15.42:0/135869455 >> v2:172.21.15.63:3300/0 conn(0x7f99880af360 0x7f99880af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.293+0000 7f9990800640 1 --2- 172.21.15.42:0/135869455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f998806ee80 0x7f998806dd40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.293+0000 7f9990800640 1 -- 172.21.15.42:0/135869455 >> 172.21.15.42:0/135869455 conn(0x7f998806bdf0 msgr2=0x7f998806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:59.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.293+0000 7f9990800640 1 -- 172.21.15.42:0/135869455 shutdown_connections 2024-05-28T14:08:59.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.293+0000 7f9990800640 1 -- 172.21.15.42:0/135869455 wait complete. 2024-05-28T14:08:59.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f9990800640 1 Processor -- start 2024-05-28T14:08:59.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f9990800640 1 -- start start 2024-05-28T14:08:59.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f9990800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f998806ee80 0x7f998814c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:59.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f9990800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99880af360 0x7f9988152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:59.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f9990800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99880d06f0 0x7f998814a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:59.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f9990800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f99880d0f10 con 0x7f998806ee80 2024-05-28T14:08:59.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f9990800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9988160fa0 con 0x7f99880af360 2024-05-28T14:08:59.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f9990800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9988071d20 con 0x7f99880d06f0 2024-05-28T14:08:59.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f998de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f998806ee80 0x7f998814c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:59.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f998de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f998806ee80 0x7f998814c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48606/0 (socket says 172.21.15.42:48606) 2024-05-28T14:08:59.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f998de00640 1 -- 172.21.15.42:0/3743631063 learned_addr learned my addr 172.21.15.42:0/3743631063 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:59.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f998d400640 1 --2- 172.21.15.42:0/3743631063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99880af360 0x7f9988152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:59.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f998e800640 1 --2- 172.21.15.42:0/3743631063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99880d06f0 0x7f998814a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:59.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f998de00640 1 -- 172.21.15.42:0/3743631063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99880d06f0 msgr2=0x7f998814a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:59.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f998de00640 1 --2- 172.21.15.42:0/3743631063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99880d06f0 0x7f998814a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f998de00640 1 -- 172.21.15.42:0/3743631063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99880af360 msgr2=0x7f9988152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:59.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f998de00640 1 --2- 172.21.15.42:0/3743631063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99880af360 0x7f9988152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f998de00640 1 -- 172.21.15.42:0/3743631063 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998815c250 con 0x7f998806ee80 2024-05-28T14:08:59.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f998de00640 1 --2- 172.21.15.42:0/3743631063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f998806ee80 0x7f998814c060 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f997c019fb0 tx=0x7f997c01abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:59.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f9982000640 1 -- 172.21.15.42:0/3743631063 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c0b0050 con 0x7f998806ee80 2024-05-28T14:08:59.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f9982000640 1 -- 172.21.15.42:0/3743631063 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f997c0bc070 con 0x7f998806ee80 2024-05-28T14:08:59.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f9982000640 1 -- 172.21.15.42:0/3743631063 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c006e20 con 0x7f998806ee80 2024-05-28T14:08:59.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f9990800640 1 -- 172.21.15.42:0/3743631063 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998815c530 con 0x7f998806ee80 2024-05-28T14:08:59.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.297+0000 7f9990800640 1 -- 172.21.15.42:0/3743631063 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9988072250 con 0x7f998806ee80 2024-05-28T14:08:59.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.301+0000 7f9990800640 1 -- 172.21.15.42:0/3743631063 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9988068a90 con 0x7f998806ee80 2024-05-28T14:08:59.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.301+0000 7f9982000640 1 -- 172.21.15.42:0/3743631063 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f997c0b5830 con 0x7f998806ee80 2024-05-28T14:08:59.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.301+0000 7f9982000640 1 --2- 172.21.15.42:0/3743631063 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f996c069460 0x7f996c06b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:59.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.301+0000 7f9982000640 1 -- 172.21.15.42:0/3743631063 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f997c120420 con 0x7f998806ee80 2024-05-28T14:08:59.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.301+0000 7f998d400640 1 --2- 172.21.15.42:0/3743631063 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f996c069460 0x7f996c06b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:59.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.305+0000 7f998d400640 1 --2- 172.21.15.42:0/3743631063 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f996c069460 0x7f996c06b920 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9978009580 tx=0x7f9978009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:59.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.305+0000 7f9982000640 1 -- 172.21.15.42:0/3743631063 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f997c016090 con 0x7f998806ee80 2024-05-28T14:08:59.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.501+0000 7f9990800640 1 -- 172.21.15.42:0/3743631063 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f998815a0d0 con 0x7f996c069460 2024-05-28T14:08:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.513+0000 7f9982000640 1 -- 172.21.15.42:0/3743631063 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f998815a0d0 con 0x7f996c069460 2024-05-28T14:08:59.496 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:08:59.496 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:08:59.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:08:59.496 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ss1" 2024-05-28T14:08:59.496 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:08:59.496 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:08:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.513+0000 7f9990800640 1 -- 172.21.15.42:0/3743631063 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f996c069460 msgr2=0x7f996c06b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.513+0000 7f9990800640 1 --2- 172.21.15.42:0/3743631063 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f996c069460 0x7f996c06b920 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9978009580 tx=0x7f9978009190 comp rx=0 tx=0).stop 2024-05-28T14:08:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.513+0000 7f9990800640 1 -- 172.21.15.42:0/3743631063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f998806ee80 msgr2=0x7f998814c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.513+0000 7f9990800640 1 --2- 172.21.15.42:0/3743631063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f998806ee80 0x7f998814c060 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f997c019fb0 tx=0x7f997c01abc0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.513+0000 7f9990800640 1 -- 172.21.15.42:0/3743631063 shutdown_connections 2024-05-28T14:08:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.513+0000 7f9990800640 1 --2- 172.21.15.42:0/3743631063 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f996c069460 0x7f996c06b920 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.513+0000 7f9990800640 1 --2- 172.21.15.42:0/3743631063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99880d06f0 0x7f998814a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.513+0000 7f9990800640 1 --2- 172.21.15.42:0/3743631063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99880af360 0x7f9988152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.513+0000 7f9990800640 1 --2- 172.21.15.42:0/3743631063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f998806ee80 0x7f998814c060 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.513+0000 7f9990800640 1 -- 172.21.15.42:0/3743631063 >> 172.21.15.42:0/3743631063 conn(0x7f998806bdf0 msgr2=0x7f998806f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.513+0000 7f9990800640 1 -- 172.21.15.42:0/3743631063 shutdown_connections 2024-05-28T14:08:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.513+0000 7f9990800640 1 -- 172.21.15.42:0/3743631063 wait complete. 2024-05-28T14:08:59.517 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:08:59.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f7391400640 1 Processor -- start 2024-05-28T14:08:59.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f7391400640 1 -- start start 2024-05-28T14:08:59.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f7391400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f738c071de0 0x7f738c0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:59.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f7391400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f738c0ce3b0 0x7f738c0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:59.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f7391400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f738c141d10 0x7f738c1421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:59.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f7391400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f738c072700 con 0x7f738c141d10 2024-05-28T14:08:59.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f7391400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f738c160230 con 0x7f738c071de0 2024-05-28T14:08:59.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f7391400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f738c160510 con 0x7f738c0ce3b0 2024-05-28T14:08:59.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f738b200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f738c141d10 0x7f738c1421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:59.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f738b200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f738c141d10 0x7f738c1421a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48616/0 (socket says 172.21.15.42:48616) 2024-05-28T14:08:59.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f738b200640 1 -- 172.21.15.42:0/4294566201 learned_addr learned my addr 172.21.15.42:0/4294566201 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:59.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f738a800640 1 --2- 172.21.15.42:0/4294566201 >> v2:172.21.15.63:3300/0 conn(0x7f738c071de0 0x7f738c0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:59.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f7381e00640 1 --2- 172.21.15.42:0/4294566201 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f738c0ce3b0 0x7f738c0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:59.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f738b200640 1 -- 172.21.15.42:0/4294566201 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f738c0ce3b0 msgr2=0x7f738c0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:59.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f738b200640 1 --2- 172.21.15.42:0/4294566201 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f738c0ce3b0 0x7f738c0ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f738b200640 1 -- 172.21.15.42:0/4294566201 >> v2:172.21.15.63:3300/0 conn(0x7f738c071de0 msgr2=0x7f738c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:08:59.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f738b200640 1 --2- 172.21.15.42:0/4294566201 >> v2:172.21.15.63:3300/0 conn(0x7f738c071de0 0x7f738c0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f738b200640 1 -- 172.21.15.42:0/4294566201 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c1607f0 con 0x7f738c141d10 2024-05-28T14:08:59.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f738a800640 1 --2- 172.21.15.42:0/4294566201 >> v2:172.21.15.63:3300/0 conn(0x7f738c071de0 0x7f738c0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:08:59.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.681+0000 7f7381e00640 1 --2- 172.21.15.42:0/4294566201 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f738c0ce3b0 0x7f738c0ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:08:59.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.685+0000 7f738b200640 1 --2- 172.21.15.42:0/4294566201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f738c141d10 0x7f738c1421a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f738c06cbf0 tx=0x7f738401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc42bebe92eab51b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:59.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.685+0000 7f7389e00640 1 -- 172.21.15.42:0/4294566201 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73840a5550 con 0x7f738c141d10 2024-05-28T14:08:59.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.685+0000 7f7389e00640 1 -- 172.21.15.42:0/4294566201 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73840a56f0 con 0x7f738c141d10 2024-05-28T14:08:59.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.685+0000 7f7389e00640 1 -- 172.21.15.42:0/4294566201 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73840a5ac0 con 0x7f738c141d10 2024-05-28T14:08:59.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.685+0000 7f7391400640 1 -- 172.21.15.42:0/4294566201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f738c141d10 msgr2=0x7f738c1421a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:59.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.685+0000 7f7391400640 1 --2- 172.21.15.42:0/4294566201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f738c141d10 0x7f738c1421a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f738c06cbf0 tx=0x7f738401c8d0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.685+0000 7f7391400640 1 -- 172.21.15.42:0/4294566201 shutdown_connections 2024-05-28T14:08:59.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.685+0000 7f7391400640 1 --2- 172.21.15.42:0/4294566201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f738c141d10 0x7f738c1421a0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.685+0000 7f7391400640 1 --2- 172.21.15.42:0/4294566201 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f738c0ce3b0 0x7f738c0ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.685+0000 7f7391400640 1 --2- 172.21.15.42:0/4294566201 >> v2:172.21.15.63:3300/0 conn(0x7f738c071de0 0x7f738c0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.685+0000 7f7391400640 1 -- 172.21.15.42:0/4294566201 >> 172.21.15.42:0/4294566201 conn(0x7f738c0b21f0 msgr2=0x7f738c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:59.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.685+0000 7f7391400640 1 -- 172.21.15.42:0/4294566201 shutdown_connections 2024-05-28T14:08:59.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.685+0000 7f7391400640 1 -- 172.21.15.42:0/4294566201 wait complete. 2024-05-28T14:08:59.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f7391400640 1 Processor -- start 2024-05-28T14:08:59.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f7391400640 1 -- start start 2024-05-28T14:08:59.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f7391400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f738c071de0 0x7f738c135d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:59.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f7391400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f738c0ce3b0 0x7f738c13b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:59.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f7391400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f738c141d10 0x7f738c13eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:59.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f7391400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f738c140520 con 0x7f738c071de0 2024-05-28T14:08:59.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f7391400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f738c13f8c0 con 0x7f738c0ce3b0 2024-05-28T14:08:59.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f7391400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f738c1406a0 con 0x7f738c141d10 2024-05-28T14:08:59.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f738a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f738c071de0 0x7f738c135d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:59.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f738a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f738c071de0 0x7f738c135d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48628/0 (socket says 172.21.15.42:48628) 2024-05-28T14:08:59.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f738a800640 1 -- 172.21.15.42:0/3613884880 learned_addr learned my addr 172.21.15.42:0/3613884880 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:08:59.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f738b200640 1 --2- 172.21.15.42:0/3613884880 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f738c141d10 0x7f738c13eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:59.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f7381e00640 1 --2- 172.21.15.42:0/3613884880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f738c0ce3b0 0x7f738c13b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:59.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f738a800640 1 -- 172.21.15.42:0/3613884880 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f738c141d10 msgr2=0x7f738c13eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:59.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f738a800640 1 --2- 172.21.15.42:0/3613884880 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f738c141d10 0x7f738c13eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f738a800640 1 -- 172.21.15.42:0/3613884880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f738c0ce3b0 msgr2=0x7f738c13b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:59.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f738a800640 1 --2- 172.21.15.42:0/3613884880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f738c0ce3b0 0x7f738c13b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f738a800640 1 -- 172.21.15.42:0/3613884880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c13b990 con 0x7f738c071de0 2024-05-28T14:08:59.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f738a800640 1 --2- 172.21.15.42:0/3613884880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f738c071de0 0x7f738c135d70 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f737c00e840 tx=0x7f737c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:59.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f7383400640 1 -- 172.21.15.42:0/3613884880 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c00f040 con 0x7f738c071de0 2024-05-28T14:08:59.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f7383400640 1 -- 172.21.15.42:0/3613884880 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f737c004030 con 0x7f738c071de0 2024-05-28T14:08:59.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f7391400640 1 -- 172.21.15.42:0/3613884880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f738c13bc10 con 0x7f738c071de0 2024-05-28T14:08:59.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f7383400640 1 -- 172.21.15.42:0/3613884880 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c004400 con 0x7f738c071de0 2024-05-28T14:08:59.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.689+0000 7f7391400640 1 -- 172.21.15.42:0/3613884880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f738c151af0 con 0x7f738c071de0 2024-05-28T14:08:59.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.693+0000 7f7391400640 1 -- 172.21.15.42:0/3613884880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f738c0738b0 con 0x7f738c071de0 2024-05-28T14:08:59.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.693+0000 7f7383400640 1 -- 172.21.15.42:0/3613884880 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 89517+0+0 (secure 0 0 0) 0x7f737c00c6d0 con 0x7f738c071de0 2024-05-28T14:08:59.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.693+0000 7f7383400640 1 --2- 172.21.15.42:0/3613884880 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7368069460 0x7f736806b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:08:59.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.693+0000 7f7383400640 1 -- 172.21.15.42:0/3613884880 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f737c085480 con 0x7f738c071de0 2024-05-28T14:08:59.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.693+0000 7f7381e00640 1 --2- 172.21.15.42:0/3613884880 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7368069460 0x7f736806b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:08:59.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.693+0000 7f7381e00640 1 --2- 172.21.15.42:0/3613884880 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7368069460 0x7f736806b920 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f738c15a040 tx=0x7f7374008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:08:59.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.697+0000 7f7383400640 1 -- 172.21.15.42:0/3613884880 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f738c13b990 con 0x7f738c071de0 2024-05-28T14:08:59.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.897+0000 7f7391400640 1 -- 172.21.15.42:0/3613884880 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f738c04fdd0 con 0x7f7368069460 2024-05-28T14:08:59.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.921+0000 7f7383400640 1 -- 172.21.15.42:0/3613884880 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f738c04fdd0 con 0x7f7368069460 2024-05-28T14:08:59.905 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:08:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.925+0000 7f7391400640 1 -- 172.21.15.42:0/3613884880 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7368069460 msgr2=0x7f736806b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.925+0000 7f7391400640 1 --2- 172.21.15.42:0/3613884880 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7368069460 0x7f736806b920 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f738c15a040 tx=0x7f7374008040 comp rx=0 tx=0).stop 2024-05-28T14:08:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.925+0000 7f7391400640 1 -- 172.21.15.42:0/3613884880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f738c071de0 msgr2=0x7f738c135d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:08:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.925+0000 7f7391400640 1 --2- 172.21.15.42:0/3613884880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f738c071de0 0x7f738c135d70 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f737c00e840 tx=0x7f737c00ed00 comp rx=0 tx=0).stop 2024-05-28T14:08:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.925+0000 7f7391400640 1 -- 172.21.15.42:0/3613884880 shutdown_connections 2024-05-28T14:08:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.925+0000 7f7391400640 1 --2- 172.21.15.42:0/3613884880 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7368069460 0x7f736806b920 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.925+0000 7f7391400640 1 --2- 172.21.15.42:0/3613884880 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f738c141d10 0x7f738c13eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.925+0000 7f7391400640 1 --2- 172.21.15.42:0/3613884880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f738c0ce3b0 0x7f738c13b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.925+0000 7f7391400640 1 --2- 172.21.15.42:0/3613884880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f738c071de0 0x7f738c135d70 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:08:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.925+0000 7f7391400640 1 -- 172.21.15.42:0/3613884880 >> 172.21.15.42:0/3613884880 conn(0x7f738c0b21f0 msgr2=0x7f738c0cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:08:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.925+0000 7f7391400640 1 -- 172.21.15.42:0/3613884880 shutdown_connections 2024-05-28T14:08:59.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:08:59.925+0000 7f7391400640 1 -- 172.21.15.42:0/3613884880 wait complete. 2024-05-28T14:08:59.927 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-28T14:09:00.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb997600640 1 Processor -- start 2024-05-28T14:09:00.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb997600640 1 -- start start 2024-05-28T14:09:00.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb997600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb99015ec50 0x7fb9900b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb997600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9900d1a80 con 0x7fb99015e870 2024-05-28T14:09:00.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb997600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9900ccbd0 con 0x7fb9900b7ba0 2024-05-28T14:09:00.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb997600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb9900cd500 con 0x7fb99015ec50 2024-05-28T14:09:00.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb994c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb99015e870 0x7fb9900d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38718/0 (socket says 172.21.15.42:38718) 2024-05-28T14:09:00.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb994c00640 1 -- 172.21.15.42:0/3530515573 learned_addr learned my addr 172.21.15.42:0/3530515573 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:00.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98fe00640 1 --2- 172.21.15.42:0/3530515573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb99015ec50 0x7fb9900b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2404706640 0 0) 0x7fb9900d1a80 con 0x7fb99015e870 2024-05-28T14:09:00.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb978003530 con 0x7fb99015e870 2024-05-28T14:09:00.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2310743307 0 0) 0x7fb9900ccbd0 con 0x7fb9900b7ba0 2024-05-28T14:09:00.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9900d1a80 con 0x7fb9900b7ba0 2024-05-28T14:09:00.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 16720788 0 0) 0x7fb978003530 con 0x7fb99015e870 2024-05-28T14:09:00.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb99015ec50 msgr2=0x7fb9900b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:00.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 --2- 172.21.15.42:0/3530515573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb99015ec50 0x7fb9900b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 >> v1:172.21.15.63:6789/0 conn(0x7fb9900b7ba0 legacy=0x7fb99009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9900cceb0 con 0x7fb99015e870 2024-05-28T14:09:00.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fb984001680 con 0x7fb99015e870 2024-05-28T14:09:00.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 >> v1:172.21.15.42:6789/0 conn(0x7fb99015e870 legacy=0x7fb9900d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 --2- 172.21.15.42:0/3530515573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb99015ec50 0x7fb978003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 --2- 172.21.15.42:0/3530515573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9780040e0 0x7fb978004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 --2- 172.21.15.42:0/3530515573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb978004d50 0x7fb9780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9900cd500 con 0x7fb9780040e0 2024-05-28T14:09:00.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb97800bab0 con 0x7fb978004d50 2024-05-28T14:09:00.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb97800bcb0 con 0x7fb99015ec50 2024-05-28T14:09:00.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98fe00640 1 --2- 172.21.15.42:0/3530515573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9780040e0 0x7fb978004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb994c00640 1 --2- 172.21.15.42:0/3530515573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb99015ec50 0x7fb978003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb995600640 1 --2- 172.21.15.42:0/3530515573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb978004d50 0x7fb9780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98fe00640 1 -- 172.21.15.42:0/3530515573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb99015ec50 msgr2=0x7fb978003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:00.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98fe00640 1 --2- 172.21.15.42:0/3530515573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb99015ec50 0x7fb978003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98fe00640 1 -- 172.21.15.42:0/3530515573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb978004d50 msgr2=0x7fb9780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:00.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98fe00640 1 --2- 172.21.15.42:0/3530515573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb978004d50 0x7fb9780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb98fe00640 1 -- 172.21.15.42:0/3530515573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb978003530 con 0x7fb9780040e0 2024-05-28T14:09:00.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.093+0000 7fb995600640 1 --2- 172.21.15.42:0/3530515573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb978004d50 0x7fb9780051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:00.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.097+0000 7fb98fe00640 1 --2- 172.21.15.42:0/3530515573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9780040e0 0x7fb978004810 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009ab0 tx=0x7fb97c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9abcb3794a8f02f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:00.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.097+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c0b2070 con 0x7fb9780040e0 2024-05-28T14:09:00.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.097+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c0a5640 con 0x7fb9780040e0 2024-05-28T14:09:00.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.097+0000 7fb98f400640 1 -- 172.21.15.42:0/3530515573 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c0096b0 con 0x7fb9780040e0 2024-05-28T14:09:00.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.097+0000 7fb997600640 1 -- 172.21.15.42:0/3530515573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9780040e0 msgr2=0x7fb978004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.097+0000 7fb997600640 1 --2- 172.21.15.42:0/3530515573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9780040e0 0x7fb978004810 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009ab0 tx=0x7fb97c01ca50 comp rx=0 tx=0).stop 2024-05-28T14:09:00.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.097+0000 7fb995600640 1 -- 172.21.15.42:0/3530515573 reap_dead start 2024-05-28T14:09:00.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.097+0000 7fb997600640 1 -- 172.21.15.42:0/3530515573 shutdown_connections 2024-05-28T14:09:00.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.097+0000 7fb997600640 1 -- 172.21.15.42:0/3530515573 >> 172.21.15.42:0/3530515573 conn(0x7fb9900b3fd0 msgr2=0x7fb9900b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:00.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.097+0000 7fb997600640 1 -- 172.21.15.42:0/3530515573 shutdown_connections 2024-05-28T14:09:00.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.097+0000 7fb997600640 1 -- 172.21.15.42:0/3530515573 wait complete. 2024-05-28T14:09:00.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb997600640 1 Processor -- start 2024-05-28T14:09:00.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb997600640 1 -- start start 2024-05-28T14:09:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb997600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9780040e0 0x7fb990161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb997600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9900c8620 0x7fb9900c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb997600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9900c8a80 0x7fb9900c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb997600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9900b5330 con 0x7fb9900c8620 2024-05-28T14:09:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb997600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb9900ce9a0 con 0x7fb9900c8a80 2024-05-28T14:09:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb997600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb99015fe00 con 0x7fb9780040e0 2024-05-28T14:09:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb98fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9900c8620 0x7fb9900c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb98fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9900c8620 0x7fb9900c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48650/0 (socket says 172.21.15.42:48650) 2024-05-28T14:09:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb98fe00640 1 -- 172.21.15.42:0/1588254879 learned_addr learned my addr 172.21.15.42:0/1588254879 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb994c00640 1 --2- 172.21.15.42:0/1588254879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9780040e0 0x7fb990161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb995600640 1 --2- 172.21.15.42:0/1588254879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9900c8a80 0x7fb9900c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb98fe00640 1 -- 172.21.15.42:0/1588254879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9780040e0 msgr2=0x7fb990161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb98fe00640 1 --2- 172.21.15.42:0/1588254879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9780040e0 0x7fb990161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb98fe00640 1 -- 172.21.15.42:0/1588254879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9900c8a80 msgr2=0x7fb9900c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb98fe00640 1 --2- 172.21.15.42:0/1588254879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9900c8a80 0x7fb9900c6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb98fe00640 1 -- 172.21.15.42:0/1588254879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb990160030 con 0x7fb9900c8620 2024-05-28T14:09:00.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb98fe00640 1 --2- 172.21.15.42:0/1588254879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9900c8620 0x7fb9900c5fd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb97c01c880 tx=0x7fb978005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:00.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb98cc00640 1 -- 172.21.15.42:0/1588254879 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c0b2040 con 0x7fb9900c8620 2024-05-28T14:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb98cc00640 1 -- 172.21.15.42:0/1588254879 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c00f000 con 0x7fb9900c8620 2024-05-28T14:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb98cc00640 1 -- 172.21.15.42:0/1588254879 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c004220 con 0x7fb9900c8620 2024-05-28T14:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb997600640 1 -- 172.21.15.42:0/1588254879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9901602d0 con 0x7fb9900c8620 2024-05-28T14:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.101+0000 7fb997600640 1 -- 172.21.15.42:0/1588254879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb990160890 con 0x7fb9900c8620 2024-05-28T14:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.105+0000 7fb997600640 1 -- 172.21.15.42:0/1588254879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb990160d80 con 0x7fb9900c8620 2024-05-28T14:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.105+0000 7fb98cc00640 1 -- 172.21.15.42:0/1588254879 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 30) ==== 89619+0+0 (secure 0 0 0) 0x7fb97c003620 con 0x7fb9900c8620 2024-05-28T14:09:00.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.105+0000 7fb98cc00640 1 --2- 172.21.15.42:0/1588254879 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb970069580 0x7fb97006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.105+0000 7fb98cc00640 1 -- 172.21.15.42:0/1588254879 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fb97c11f150 con 0x7fb9900c8620 2024-05-28T14:09:00.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.105+0000 7fb994c00640 1 --2- 172.21.15.42:0/1588254879 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb970069580 0x7fb97006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.105+0000 7fb994c00640 1 --2- 172.21.15.42:0/1588254879 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb970069580 0x7fb97006ba40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb984001c90 tx=0x7fb9840014a0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.109+0000 7fb98cc00640 1 -- 172.21.15.42:0/1588254879 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb97c016760 con 0x7fb9900c8620 2024-05-28T14:09:00.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.301+0000 7fb997600640 1 -- 172.21.15.42:0/1588254879 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fb9900c6ab0 con 0x7fb970069580 2024-05-28T14:09:00.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.317+0000 7fb98cc00640 1 -- 172.21.15.42:0/1588254879 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb9900c6ab0 con 0x7fb970069580 2024-05-28T14:09:00.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.321+0000 7fb997600640 1 -- 172.21.15.42:0/1588254879 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb970069580 msgr2=0x7fb97006ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.321+0000 7fb997600640 1 --2- 172.21.15.42:0/1588254879 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb970069580 0x7fb97006ba40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb984001c90 tx=0x7fb9840014a0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.321+0000 7fb997600640 1 -- 172.21.15.42:0/1588254879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9900c8620 msgr2=0x7fb9900c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.321+0000 7fb997600640 1 --2- 172.21.15.42:0/1588254879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9900c8620 0x7fb9900c5fd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb97c01c880 tx=0x7fb978005840 comp rx=0 tx=0).stop 2024-05-28T14:09:00.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.321+0000 7fb997600640 1 -- 172.21.15.42:0/1588254879 shutdown_connections 2024-05-28T14:09:00.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.321+0000 7fb997600640 1 --2- 172.21.15.42:0/1588254879 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb970069580 0x7fb97006ba40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.321+0000 7fb997600640 1 --2- 172.21.15.42:0/1588254879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9900c8a80 0x7fb9900c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.321+0000 7fb997600640 1 --2- 172.21.15.42:0/1588254879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9900c8620 0x7fb9900c5fd0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.321+0000 7fb997600640 1 --2- 172.21.15.42:0/1588254879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9780040e0 0x7fb990161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.321+0000 7fb997600640 1 -- 172.21.15.42:0/1588254879 >> 172.21.15.42:0/1588254879 conn(0x7fb9900b3fd0 msgr2=0x7fb99009aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:00.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.321+0000 7fb997600640 1 -- 172.21.15.42:0/1588254879 shutdown_connections 2024-05-28T14:09:00.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.321+0000 7fb997600640 1 -- 172.21.15.42:0/1588254879 wait complete. 2024-05-28T14:09:00.332 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone81115 2024-05-28T14:09:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.497+0000 7fba6a200640 1 Processor -- start 2024-05-28T14:09:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.501+0000 7fba6a200640 1 -- start start 2024-05-28T14:09:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.501+0000 7fba6a200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fba640d2950 0x7fba6406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.501+0000 7fba6a200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fba640d1520 0x7fba6406d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.501+0000 7fba6a200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba640ced30 con 0x7fba6415e460 2024-05-28T14:09:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.501+0000 7fba6a200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fba64162190 con 0x7fba640d1520 2024-05-28T14:09:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.501+0000 7fba6a200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fba64162470 con 0x7fba640d2950 2024-05-28T14:09:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.501+0000 7fba69800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fba640d1520 0x7fba6406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.501+0000 7fba68e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fba6415e460 0x7fba640d2240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38730/0 (socket says 172.21.15.42:38730) 2024-05-28T14:09:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.501+0000 7fba68e00640 1 -- 172.21.15.42:0/3917436902 learned_addr learned my addr 172.21.15.42:0/3917436902 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.501+0000 7fba63e00640 1 --2- 172.21.15.42:0/3917436902 >> v2:172.21.15.63:3300/0 conn(0x7fba640d2950 0x7fba6406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.501+0000 7fba69800640 1 -- 172.21.15.42:0/3917436902 >> v2:172.21.15.63:3300/0 conn(0x7fba640d2950 msgr2=0x7fba6406d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:00.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.501+0000 7fba69800640 1 --2- 172.21.15.42:0/3917436902 >> v2:172.21.15.63:3300/0 conn(0x7fba640d2950 0x7fba6406d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.501+0000 7fba69800640 1 -- 172.21.15.42:0/3917436902 >> v1:172.21.15.42:6789/0 conn(0x7fba6415e460 legacy=0x7fba640d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.501+0000 7fba69800640 1 -- 172.21.15.42:0/3917436902 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba64162750 con 0x7fba640d1520 2024-05-28T14:09:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.505+0000 7fba69800640 1 --2- 172.21.15.42:0/3917436902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba640d1520 0x7fba6406d700 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fba6406f570 tx=0x7fba5c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5149a979b1058399 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.505+0000 7fba63400640 1 -- 172.21.15.42:0/3917436902 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0b2070 con 0x7fba640d1520 2024-05-28T14:09:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.505+0000 7fba63400640 1 -- 172.21.15.42:0/3917436902 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba5c0a5850 con 0x7fba640d1520 2024-05-28T14:09:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.505+0000 7fba63400640 1 -- 172.21.15.42:0/3917436902 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0a5c20 con 0x7fba640d1520 2024-05-28T14:09:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.505+0000 7fba6a200640 1 -- 172.21.15.42:0/3917436902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba640d1520 msgr2=0x7fba6406d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.505+0000 7fba6a200640 1 --2- 172.21.15.42:0/3917436902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba640d1520 0x7fba6406d700 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fba6406f570 tx=0x7fba5c01c930 comp rx=0 tx=0).stop 2024-05-28T14:09:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.505+0000 7fba6a200640 1 -- 172.21.15.42:0/3917436902 shutdown_connections 2024-05-28T14:09:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.505+0000 7fba6a200640 1 --2- 172.21.15.42:0/3917436902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba640d1520 0x7fba6406d700 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.505+0000 7fba6a200640 1 --2- 172.21.15.42:0/3917436902 >> v2:172.21.15.63:3300/0 conn(0x7fba640d2950 0x7fba6406d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.505+0000 7fba6a200640 1 -- 172.21.15.42:0/3917436902 >> 172.21.15.42:0/3917436902 conn(0x7fba6406fd70 msgr2=0x7fba64070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.505+0000 7fba6a200640 1 -- 172.21.15.42:0/3917436902 shutdown_connections 2024-05-28T14:09:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.505+0000 7fba6a200640 1 -- 172.21.15.42:0/3917436902 wait complete. 2024-05-28T14:09:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.505+0000 7fba6a200640 1 Processor -- start 2024-05-28T14:09:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba6a200640 1 -- start start 2024-05-28T14:09:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba6a200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba640d1520 0x7fba6414fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba6a200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba640d2950 0x7fba64150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba6a200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6415e460 0x7fba64150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba6a200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fba6415e9c0 con 0x7fba640d1520 2024-05-28T14:09:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba6a200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba6415e840 con 0x7fba640d2950 2024-05-28T14:09:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba6a200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fba6416b530 con 0x7fba6415e460 2024-05-28T14:09:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba68e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba640d1520 0x7fba6414fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba68e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba640d1520 0x7fba6414fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48660/0 (socket says 172.21.15.42:48660) 2024-05-28T14:09:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba68e00640 1 -- 172.21.15.42:0/435432949 learned_addr learned my addr 172.21.15.42:0/435432949 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba63e00640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba640d2950 0x7fba64150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba69800640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6415e460 0x7fba64150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba63e00640 1 -- 172.21.15.42:0/435432949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6415e460 msgr2=0x7fba64150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba63e00640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6415e460 0x7fba64150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba63e00640 1 -- 172.21.15.42:0/435432949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba640d1520 msgr2=0x7fba6414fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba63e00640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba640d1520 0x7fba6414fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba63e00640 1 -- 172.21.15.42:0/435432949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba640ce2d0 con 0x7fba640d2950 2024-05-28T14:09:00.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba68e00640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba640d1520 0x7fba6414fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:09:00.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba63e00640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba640d2950 0x7fba64150420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fba5000e840 tx=0x7fba5000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:00.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba60c00640 1 -- 172.21.15.42:0/435432949 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50017070 con 0x7fba640d2950 2024-05-28T14:09:00.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba60c00640 1 -- 172.21.15.42:0/435432949 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba5000c940 con 0x7fba640d2950 2024-05-28T14:09:00.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba6a200640 1 -- 172.21.15.42:0/435432949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba640ce5b0 con 0x7fba640d2950 2024-05-28T14:09:00.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba60c00640 1 -- 172.21.15.42:0/435432949 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5000ccf0 con 0x7fba640d2950 2024-05-28T14:09:00.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba6a200640 1 -- 172.21.15.42:0/435432949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba64158a70 con 0x7fba640d2950 2024-05-28T14:09:00.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.509+0000 7fba6a200640 1 -- 172.21.15.42:0/435432949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba64068a90 con 0x7fba640d2950 2024-05-28T14:09:00.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.513+0000 7fba60c00640 1 -- 172.21.15.42:0/435432949 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 30) ==== 89619+0+0 (secure 0 0 0) 0x7fba5001a070 con 0x7fba640d2950 2024-05-28T14:09:00.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.513+0000 7fba60c00640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba44069580 0x7fba4406ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.513+0000 7fba60c00640 1 -- 172.21.15.42:0/435432949 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7fba50085bb0 con 0x7fba640d2950 2024-05-28T14:09:00.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.513+0000 7fba68e00640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba44069580 0x7fba4406ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.513+0000 7fba68e00640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba44069580 0x7fba4406ba40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fba58007fd0 tx=0x7fba58007f20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:00.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.517+0000 7fba60c00640 1 -- 172.21.15.42:0/435432949 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fba500571e0 con 0x7fba640d2950 2024-05-28T14:09:00.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.709+0000 7fba6a200640 1 -- 172.21.15.42:0/435432949 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone81115", "target": ["mon-mgr", ""]}) -- 0x7fba640b1530 con 0x7fba44069580 2024-05-28T14:09:00.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.729+0000 7fba60c00640 1 -- 172.21.15.42:0/435432949 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fba640b1530 con 0x7fba44069580 2024-05-28T14:09:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.733+0000 7fba6a200640 1 -- 172.21.15.42:0/435432949 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba44069580 msgr2=0x7fba4406ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.733+0000 7fba6a200640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba44069580 0x7fba4406ba40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fba58007fd0 tx=0x7fba58007f20 comp rx=0 tx=0).stop 2024-05-28T14:09:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.733+0000 7fba6a200640 1 -- 172.21.15.42:0/435432949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba640d2950 msgr2=0x7fba64150420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.733+0000 7fba6a200640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba640d2950 0x7fba64150420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fba5000e840 tx=0x7fba5000ed00 comp rx=0 tx=0).stop 2024-05-28T14:09:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.733+0000 7fba6a200640 1 -- 172.21.15.42:0/435432949 shutdown_connections 2024-05-28T14:09:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.733+0000 7fba6a200640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba44069580 0x7fba4406ba40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.733+0000 7fba6a200640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fba6415e460 0x7fba64150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.733+0000 7fba6a200640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba640d2950 0x7fba64150420 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.733+0000 7fba6a200640 1 --2- 172.21.15.42:0/435432949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba640d1520 0x7fba6414fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.733+0000 7fba6a200640 1 -- 172.21.15.42:0/435432949 >> 172.21.15.42:0/435432949 conn(0x7fba6406fd70 msgr2=0x7fba6415c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.733+0000 7fba6a200640 1 -- 172.21.15.42:0/435432949 shutdown_connections 2024-05-28T14:09:00.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.733+0000 7fba6a200640 1 -- 172.21.15.42:0/435432949 wait complete. 2024-05-28T14:09:00.739 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone81114 2024-05-28T14:09:00.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.969+0000 7f9eb5400640 1 Processor -- start 2024-05-28T14:09:00.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.969+0000 7f9eb5400640 1 -- start start 2024-05-28T14:09:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.969+0000 7f9eb5400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9eb006ee80 0x7f9eb006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.969+0000 7f9eb5400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9eb00af360 0x7f9eb00af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.969+0000 7f9eb5400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9eb00d06f0 0x7f9eb00d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.969+0000 7f9eb5400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb00d10a0 con 0x7f9eb00af360 2024-05-28T14:09:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.969+0000 7f9eb5400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb01570a0 con 0x7f9eb00d06f0 2024-05-28T14:09:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.969+0000 7f9eb5400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb015cf00 con 0x7f9eb006ee80 2024-05-28T14:09:00.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.969+0000 7f9eaf200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9eb00d06f0 0x7f9eb00d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.969+0000 7f9eaf200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9eb00d06f0 0x7f9eb00d0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:46824/0 (socket says 172.21.15.42:46824) 2024-05-28T14:09:00.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.969+0000 7f9eaf200640 1 -- 172.21.15.42:0/1765749428 learned_addr learned my addr 172.21.15.42:0/1765749428 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:00.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.969+0000 7f9eade00640 1 --2- 172.21.15.42:0/1765749428 >> v2:172.21.15.42:3300/0 conn(0x7f9eb00af360 0x7f9eb00af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.973+0000 7f9eae800640 1 --2- 172.21.15.42:0/1765749428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9eb006ee80 0x7f9eb006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.973+0000 7f9eade00640 1 -- 172.21.15.42:0/1765749428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9eb006ee80 msgr2=0x7f9eb006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.973+0000 7f9eade00640 1 --2- 172.21.15.42:0/1765749428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9eb006ee80 0x7f9eb006dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.973+0000 7f9eade00640 1 -- 172.21.15.42:0/1765749428 >> v2:172.21.15.63:3300/0 conn(0x7f9eb00d06f0 msgr2=0x7f9eb00d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.973+0000 7f9eade00640 1 --2- 172.21.15.42:0/1765749428 >> v2:172.21.15.63:3300/0 conn(0x7f9eb00d06f0 0x7f9eb00d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.973+0000 7f9eade00640 1 -- 172.21.15.42:0/1765749428 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb015d1e0 con 0x7f9eb00af360 2024-05-28T14:09:00.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.977+0000 7f9eae800640 1 --2- 172.21.15.42:0/1765749428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9eb006ee80 0x7f9eb006dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:00.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.977+0000 7f9eade00640 1 --2- 172.21.15.42:0/1765749428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb00af360 0x7f9eb00af7e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9eb015d110 tx=0x7f9e9c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed51a9ffe39ce991 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:00.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.977+0000 7f9ead400640 1 -- 172.21.15.42:0/1765749428 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0b0070 con 0x7f9eb00af360 2024-05-28T14:09:00.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.977+0000 7f9ead400640 1 -- 172.21.15.42:0/1765749428 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e9c0ab600 con 0x7f9eb00af360 2024-05-28T14:09:00.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.977+0000 7f9ead400640 1 -- 172.21.15.42:0/1765749428 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0ab9d0 con 0x7f9eb00af360 2024-05-28T14:09:00.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.977+0000 7f9eb5400640 1 -- 172.21.15.42:0/1765749428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb00af360 msgr2=0x7f9eb00af7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.977+0000 7f9eb5400640 1 --2- 172.21.15.42:0/1765749428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb00af360 0x7f9eb00af7e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9eb015d110 tx=0x7f9e9c01c380 comp rx=0 tx=0).stop 2024-05-28T14:09:00.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.977+0000 7f9eb5400640 1 -- 172.21.15.42:0/1765749428 shutdown_connections 2024-05-28T14:09:00.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.977+0000 7f9eb5400640 1 --2- 172.21.15.42:0/1765749428 >> v2:172.21.15.63:3300/0 conn(0x7f9eb00d06f0 0x7f9eb00d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.977+0000 7f9eb5400640 1 --2- 172.21.15.42:0/1765749428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb00af360 0x7f9eb00af7e0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.977+0000 7f9eb5400640 1 --2- 172.21.15.42:0/1765749428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9eb006ee80 0x7f9eb006dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.977+0000 7f9eb5400640 1 -- 172.21.15.42:0/1765749428 >> 172.21.15.42:0/1765749428 conn(0x7f9eb006bdf0 msgr2=0x7f9eb006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:00.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.977+0000 7f9eb5400640 1 -- 172.21.15.42:0/1765749428 shutdown_connections 2024-05-28T14:09:00.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.977+0000 7f9eb5400640 1 -- 172.21.15.42:0/1765749428 wait complete. 2024-05-28T14:09:00.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eb5400640 1 Processor -- start 2024-05-28T14:09:00.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eb5400640 1 -- start start 2024-05-28T14:09:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eb5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eb006ee80 0x7f9eb00b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eb5400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9eb00af360 0x7f9eb0146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eb5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb00d06f0 0x7f9eb0072d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eb5400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb015cce0 con 0x7f9eb00d06f0 2024-05-28T14:09:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eb5400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb015cb60 con 0x7f9eb006ee80 2024-05-28T14:09:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eb5400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb0160c90 con 0x7f9eb00af360 2024-05-28T14:09:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eae800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eb006ee80 0x7f9eb00b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eae800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eb006ee80 0x7f9eb00b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:46826/0 (socket says 172.21.15.42:46826) 2024-05-28T14:09:00.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eae800640 1 -- 172.21.15.42:0/317778372 learned_addr learned my addr 172.21.15.42:0/317778372 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:00.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eade00640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9eb00af360 0x7f9eb0146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eaf200640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb00d06f0 0x7f9eb0072d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eaf200640 1 -- 172.21.15.42:0/317778372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9eb00af360 msgr2=0x7f9eb0146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eaf200640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9eb00af360 0x7f9eb0146200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eaf200640 1 -- 172.21.15.42:0/317778372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eb006ee80 msgr2=0x7f9eb00b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:00.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eaf200640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eb006ee80 0x7f9eb00b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:00.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eaf200640 1 -- 172.21.15.42:0/317778372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb0073ab0 con 0x7f9eb00d06f0 2024-05-28T14:09:00.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eae800640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eb006ee80 0x7f9eb00b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:09:00.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.981+0000 7f9eaf200640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb00d06f0 0x7f9eb0072d70 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8007a90 tx=0x7f9ea8007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:00.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.985+0000 7f9ea6a00640 1 -- 172.21.15.42:0/317778372 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea801a070 con 0x7f9eb00d06f0 2024-05-28T14:09:00.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.985+0000 7f9ea6a00640 1 -- 172.21.15.42:0/317778372 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea800dbe0 con 0x7f9eb00d06f0 2024-05-28T14:09:00.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.985+0000 7f9eb5400640 1 -- 172.21.15.42:0/317778372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb0073d90 con 0x7f9eb00d06f0 2024-05-28T14:09:00.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.985+0000 7f9ea6a00640 1 -- 172.21.15.42:0/317778372 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8015400 con 0x7f9eb00d06f0 2024-05-28T14:09:00.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.985+0000 7f9eb5400640 1 -- 172.21.15.42:0/317778372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb0146a50 con 0x7f9eb00d06f0 2024-05-28T14:09:00.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.985+0000 7f9eb5400640 1 -- 172.21.15.42:0/317778372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb0068a90 con 0x7f9eb00d06f0 2024-05-28T14:09:00.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.985+0000 7f9ea6a00640 1 -- 172.21.15.42:0/317778372 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 30) ==== 89619+0+0 (secure 0 0 0) 0x7f9ea801d070 con 0x7f9eb00d06f0 2024-05-28T14:09:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.989+0000 7f9ea6a00640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9e90069580 0x7f9e9006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.989+0000 7f9ea6a00640 1 -- 172.21.15.42:0/317778372 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5795+0+0 (secure 0 0 0) 0x7f9ea8085690 con 0x7f9eb00d06f0 2024-05-28T14:09:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.989+0000 7f9eae800640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9e90069580 0x7f9e9006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:00.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.989+0000 7f9eae800640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9e90069580 0x7f9e9006ba40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0006b20 tx=0x7f9ea0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:00.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:00.993+0000 7f9ea6a00640 1 -- 172.21.15.42:0/317778372 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9ea8010070 con 0x7f9eb00d06f0 2024-05-28T14:09:01.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:01.357+0000 7f9eb5400640 1 -- 172.21.15.42:0/317778372 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone81114", "target": ["mon-mgr", ""]}) -- 0x7f9eb00751b0 con 0x7f9e90069580 2024-05-28T14:09:02.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.005+0000 7f9ea6a00640 1 -- 172.21.15.42:0/317778372 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ea000fe20 con 0x7f9e90069580 2024-05-28T14:09:03.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.021+0000 7f9eb5400640 1 -- 172.21.15.42:0/317778372 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9e90069580 msgr2=0x7f9e9006ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:03.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.021+0000 7f9eb5400640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9e90069580 0x7f9e9006ba40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0006b20 tx=0x7f9ea0008040 comp rx=0 tx=0).stop 2024-05-28T14:09:03.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.021+0000 7f9eb5400640 1 -- 172.21.15.42:0/317778372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb00d06f0 msgr2=0x7f9eb0072d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:03.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.021+0000 7f9eb5400640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb00d06f0 0x7f9eb0072d70 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8007a90 tx=0x7f9ea8007e90 comp rx=0 tx=0).stop 2024-05-28T14:09:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.025+0000 7f9eb5400640 1 -- 172.21.15.42:0/317778372 shutdown_connections 2024-05-28T14:09:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.025+0000 7f9eb5400640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9e90069580 0x7f9e9006ba40 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.025+0000 7f9eb5400640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb00d06f0 0x7f9eb0072d70 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.025+0000 7f9eb5400640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9eb00af360 0x7f9eb0146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.025+0000 7f9eb5400640 1 --2- 172.21.15.42:0/317778372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9eb006ee80 0x7f9eb00b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.025+0000 7f9eb5400640 1 -- 172.21.15.42:0/317778372 >> 172.21.15.42:0/317778372 conn(0x7f9eb006bdf0 msgr2=0x7f9eb00b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.029+0000 7f9eb5400640 1 -- 172.21.15.42:0/317778372 shutdown_connections 2024-05-28T14:09:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.029+0000 7f9eb5400640 1 -- 172.21.15.42:0/317778372 wait complete. 2024-05-28T14:09:03.059 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone81112 2024-05-28T14:09:03.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.305+0000 7faf7fe00640 1 Processor -- start 2024-05-28T14:09:03.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.305+0000 7faf7fe00640 1 -- start start 2024-05-28T14:09:03.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.309+0000 7faf7fe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faf78070720 0x7faf7806c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:03.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.309+0000 7faf7fe00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf78070d00 con 0x7faf7814c7f0 2024-05-28T14:09:03.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.309+0000 7faf7fe00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf7806ccb0 con 0x7faf780b6f70 2024-05-28T14:09:03.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.309+0000 7faf7fe00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faf7806f240 con 0x7faf78070720 2024-05-28T14:09:03.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.309+0000 7faf7de00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faf78070720 0x7faf7806c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:03.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.309+0000 7faf7de00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faf78070720 0x7faf7806c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:46832/0 (socket says 172.21.15.42:46832) 2024-05-28T14:09:03.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.309+0000 7faf7de00640 1 -- 172.21.15.42:0/2292884069 learned_addr learned my addr 172.21.15.42:0/2292884069 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:03.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.309+0000 7faf7de00640 1 -- 172.21.15.42:0/2292884069 >> v1:172.21.15.63:6789/0 conn(0x7faf780b6f70 legacy=0x7faf780ae600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-28T14:09:03.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.309+0000 7faf7de00640 1 -- 172.21.15.42:0/2292884069 >> v1:172.21.15.42:6789/0 conn(0x7faf7814c7f0 legacy=0x7faf780b6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:03.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.309+0000 7faf7de00640 1 -- 172.21.15.42:0/2292884069 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7806cf90 con 0x7faf78070720 2024-05-28T14:09:03.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.313+0000 7faf7de00640 1 --2- 172.21.15.42:0/2292884069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf78070720 0x7faf7806c400 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faf70009870 tx=0x7faf7001c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a35cfdcfcec5413d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:03.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.313+0000 7faf77e00640 1 -- 172.21.15.42:0/2292884069 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700b0070 con 0x7faf78070720 2024-05-28T14:09:03.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.313+0000 7faf77e00640 1 -- 172.21.15.42:0/2292884069 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf700ab600 con 0x7faf78070720 2024-05-28T14:09:03.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.313+0000 7faf77e00640 1 -- 172.21.15.42:0/2292884069 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700ab9d0 con 0x7faf78070720 2024-05-28T14:09:03.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.313+0000 7faf7fe00640 1 -- 172.21.15.42:0/2292884069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf78070720 msgr2=0x7faf7806c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:03.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.313+0000 7faf7fe00640 1 --2- 172.21.15.42:0/2292884069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf78070720 0x7faf7806c400 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faf70009870 tx=0x7faf7001c490 comp rx=0 tx=0).stop 2024-05-28T14:09:03.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.313+0000 7faf7fe00640 1 -- 172.21.15.42:0/2292884069 shutdown_connections 2024-05-28T14:09:03.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.313+0000 7faf7fe00640 1 --2- 172.21.15.42:0/2292884069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf78070720 0x7faf7806c400 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.313+0000 7faf7fe00640 1 -- 172.21.15.42:0/2292884069 >> 172.21.15.42:0/2292884069 conn(0x7faf780b6150 msgr2=0x7faf7809aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:03.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.313+0000 7faf7fe00640 1 -- 172.21.15.42:0/2292884069 shutdown_connections 2024-05-28T14:09:03.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.313+0000 7faf7fe00640 1 -- 172.21.15.42:0/2292884069 wait complete. 2024-05-28T14:09:03.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.317+0000 7faf7fe00640 1 Processor -- start 2024-05-28T14:09:03.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.317+0000 7faf7fe00640 1 -- start start 2024-05-28T14:09:03.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.317+0000 7faf7fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf78070720 0x7faf78163030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:03.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.317+0000 7faf7fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf780b6f70 0x7faf78161fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:03.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.317+0000 7faf7fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7814c7f0 0x7faf78146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:03.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.317+0000 7faf7fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faf780af930 con 0x7faf78070720 2024-05-28T14:09:03.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.317+0000 7faf7fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf780af7b0 con 0x7faf7814c7f0 2024-05-28T14:09:03.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.317+0000 7faf7fe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faf78070fe0 con 0x7faf780b6f70 2024-05-28T14:09:03.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.317+0000 7faf7de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7814c7f0 0x7faf78146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.317+0000 7faf7de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7814c7f0 0x7faf78146910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:46836/0 (socket says 172.21.15.42:46836) 2024-05-28T14:09:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.317+0000 7faf7de00640 1 -- 172.21.15.42:0/288040456 learned_addr learned my addr 172.21.15.42:0/288040456 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.317+0000 7faf7ca00640 1 --2- 172.21.15.42:0/288040456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf780b6f70 0x7faf78161fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.317+0000 7faf7d400640 1 --2- 172.21.15.42:0/288040456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf78070720 0x7faf78163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.321+0000 7faf7de00640 1 -- 172.21.15.42:0/288040456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf780b6f70 msgr2=0x7faf78161fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.321+0000 7faf7de00640 1 --2- 172.21.15.42:0/288040456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf780b6f70 0x7faf78161fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.321+0000 7faf7de00640 1 -- 172.21.15.42:0/288040456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf78070720 msgr2=0x7faf78163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.321+0000 7faf7de00640 1 --2- 172.21.15.42:0/288040456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf78070720 0x7faf78163030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.321+0000 7faf7de00640 1 -- 172.21.15.42:0/288040456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf78148bc0 con 0x7faf7814c7f0 2024-05-28T14:09:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.321+0000 7faf7de00640 1 --2- 172.21.15.42:0/288040456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7814c7f0 0x7faf78146910 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7faf700190a0 tx=0x7faf70019230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:03.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.321+0000 7faf75600640 1 -- 172.21.15.42:0/288040456 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700b0040 con 0x7faf7814c7f0 2024-05-28T14:09:03.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.321+0000 7faf75600640 1 -- 172.21.15.42:0/288040456 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf700b9070 con 0x7faf7814c7f0 2024-05-28T14:09:03.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.321+0000 7faf75600640 1 -- 172.21.15.42:0/288040456 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700b4840 con 0x7faf7814c7f0 2024-05-28T14:09:03.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.321+0000 7faf7fe00640 1 -- 172.21.15.42:0/288040456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf78148ea0 con 0x7faf7814c7f0 2024-05-28T14:09:03.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.321+0000 7faf7fe00640 1 -- 172.21.15.42:0/288040456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf78149460 con 0x7faf7814c7f0 2024-05-28T14:09:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.321+0000 7faf7fe00640 1 -- 172.21.15.42:0/288040456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf78068a90 con 0x7faf7814c7f0 2024-05-28T14:09:03.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.325+0000 7faf75600640 1 -- 172.21.15.42:0/288040456 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 30) ==== 89619+0+0 (secure 0 0 0) 0x7faf700b49e0 con 0x7faf7814c7f0 2024-05-28T14:09:03.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.325+0000 7faf75600640 1 --2- 172.21.15.42:0/288040456 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf58069580 0x7faf5806ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:03.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.325+0000 7faf75600640 1 -- 172.21.15.42:0/288040456 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5851+0+0 (secure 0 0 0) 0x7faf7011f790 con 0x7faf7814c7f0 2024-05-28T14:09:03.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.325+0000 7faf7d400640 1 --2- 172.21.15.42:0/288040456 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf58069580 0x7faf5806ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:03.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.325+0000 7faf7d400640 1 --2- 172.21.15.42:0/288040456 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf58069580 0x7faf5806ba40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faf680012f0 tx=0x7faf68009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.341+0000 7faf75600640 1 -- 172.21.15.42:0/288040456 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faf70016110 con 0x7faf7814c7f0 2024-05-28T14:09:03.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.645+0000 7faf7fe00640 1 -- 172.21.15.42:0/288040456 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone81112", "target": ["mon-mgr", ""]}) -- 0x7faf78163570 con 0x7faf58069580 2024-05-28T14:09:03.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.661+0000 7faf75600640 1 -- 172.21.15.42:0/288040456 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faf78163570 con 0x7faf58069580 2024-05-28T14:09:03.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.665+0000 7faf7fe00640 1 -- 172.21.15.42:0/288040456 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf58069580 msgr2=0x7faf5806ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:03.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.665+0000 7faf7fe00640 1 --2- 172.21.15.42:0/288040456 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf58069580 0x7faf5806ba40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faf680012f0 tx=0x7faf68009040 comp rx=0 tx=0).stop 2024-05-28T14:09:03.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.665+0000 7faf7fe00640 1 -- 172.21.15.42:0/288040456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7814c7f0 msgr2=0x7faf78146910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:03.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.665+0000 7faf7fe00640 1 --2- 172.21.15.42:0/288040456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7814c7f0 0x7faf78146910 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7faf700190a0 tx=0x7faf70019230 comp rx=0 tx=0).stop 2024-05-28T14:09:03.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.665+0000 7faf7fe00640 1 -- 172.21.15.42:0/288040456 shutdown_connections 2024-05-28T14:09:03.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.665+0000 7faf7fe00640 1 --2- 172.21.15.42:0/288040456 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf58069580 0x7faf5806ba40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.665+0000 7faf7fe00640 1 --2- 172.21.15.42:0/288040456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7814c7f0 0x7faf78146910 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.665+0000 7faf7fe00640 1 --2- 172.21.15.42:0/288040456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf780b6f70 0x7faf78161fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.665+0000 7faf7fe00640 1 --2- 172.21.15.42:0/288040456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf78070720 0x7faf78163030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.665+0000 7faf7fe00640 1 -- 172.21.15.42:0/288040456 >> 172.21.15.42:0/288040456 conn(0x7faf780b6150 msgr2=0x7faf78067760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:03.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.665+0000 7faf7fe00640 1 -- 172.21.15.42:0/288040456 shutdown_connections 2024-05-28T14:09:03.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.665+0000 7faf7fe00640 1 -- 172.21.15.42:0/288040456 wait complete. 2024-05-28T14:09:03.671 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone8115 2024-05-28T14:09:03.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92e1800640 1 Processor -- start 2024-05-28T14:09:03.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92e1800640 1 -- start start 2024-05-28T14:09:03.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92e1800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92dc06e740 0x7f92dc0cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:03.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92e1800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92dc0ce110 0x7f92dc09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:03.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92e1800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92dc06ff10 con 0x7f92dc06c940 2024-05-28T14:09:03.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92e1800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f92dc06eb20 con 0x7f92dc0ce110 2024-05-28T14:09:03.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92e1800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f92dc0701b0 con 0x7f92dc06e740 2024-05-28T14:09:03.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92d3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92dc0ce110 0x7f92dc09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:03.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92d3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92dc0ce110 0x7f92dc09aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48694/0 (socket says 172.21.15.42:48694) 2024-05-28T14:09:03.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92d3e00640 1 -- 172.21.15.42:0/207174893 learned_addr learned my addr 172.21.15.42:0/207174893 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:03.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92daa00640 1 --2- 172.21.15.42:0/207174893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92dc06e740 0x7f92dc0cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:03.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92d3e00640 1 -- 172.21.15.42:0/207174893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92dc06e740 msgr2=0x7f92dc0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:03.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92d3e00640 1 --2- 172.21.15.42:0/207174893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92dc06e740 0x7f92dc0cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92d3e00640 1 -- 172.21.15.42:0/207174893 >> v1:172.21.15.63:6789/0 conn(0x7f92dc06c940 legacy=0x7f92dc070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:03.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.837+0000 7f92d3e00640 1 -- 172.21.15.42:0/207174893 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc070490 con 0x7f92dc0ce110 2024-05-28T14:09:03.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.841+0000 7f92d3e00640 1 --2- 172.21.15.42:0/207174893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc0ce110 0x7f92dc09aa50 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009870 tx=0x7f92c401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a4197ea87e7028 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:03.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.841+0000 7f92da000640 1 -- 172.21.15.42:0/207174893 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c40ab640 con 0x7f92dc0ce110 2024-05-28T14:09:03.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.841+0000 7f92da000640 1 -- 172.21.15.42:0/207174893 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c40b0070 con 0x7f92dc0ce110 2024-05-28T14:09:03.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.841+0000 7f92da000640 1 -- 172.21.15.42:0/207174893 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c40aba00 con 0x7f92dc0ce110 2024-05-28T14:09:03.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.841+0000 7f92e1800640 1 -- 172.21.15.42:0/207174893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc0ce110 msgr2=0x7f92dc09aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.841+0000 7f92e1800640 1 --2- 172.21.15.42:0/207174893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc0ce110 0x7f92dc09aa50 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009870 tx=0x7f92c401c530 comp rx=0 tx=0).stop 2024-05-28T14:09:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.841+0000 7f92e1800640 1 -- 172.21.15.42:0/207174893 shutdown_connections 2024-05-28T14:09:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.841+0000 7f92e1800640 1 --2- 172.21.15.42:0/207174893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc0ce110 0x7f92dc09aa50 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.841+0000 7f92e1800640 1 --2- 172.21.15.42:0/207174893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92dc06e740 0x7f92dc0cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.841+0000 7f92e1800640 1 -- 172.21.15.42:0/207174893 >> 172.21.15.42:0/207174893 conn(0x7f92dc0b3fa0 msgr2=0x7f92dc0b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.841+0000 7f92e1800640 1 -- 172.21.15.42:0/207174893 shutdown_connections 2024-05-28T14:09:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.841+0000 7f92e1800640 1 -- 172.21.15.42:0/207174893 wait complete. 2024-05-28T14:09:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.841+0000 7f92e1800640 1 Processor -- start 2024-05-28T14:09:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.841+0000 7f92e1800640 1 -- start start 2024-05-28T14:09:03.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92e1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92dc06c940 0x7f92dc15e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:03.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92e1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc06e740 0x7f92dc160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:03.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92e1800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92dc0ce110 0x7f92dc161020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:03.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92e1800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc15e2a0 con 0x7f92dc06e740 2024-05-28T14:09:03.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92e1800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc15e120 con 0x7f92dc06c940 2024-05-28T14:09:03.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92e1800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f92dc15e420 con 0x7f92dc0ce110 2024-05-28T14:09:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92d3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc06e740 0x7f92dc160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92d3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc06e740 0x7f92dc160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48702/0 (socket says 172.21.15.42:48702) 2024-05-28T14:09:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92d3e00640 1 -- 172.21.15.42:0/2094953266 learned_addr learned my addr 172.21.15.42:0/2094953266 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92daa00640 1 --2- 172.21.15.42:0/2094953266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92dc06c940 0x7f92dc15e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92db400640 1 --2- 172.21.15.42:0/2094953266 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92dc0ce110 0x7f92dc161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92d3e00640 1 -- 172.21.15.42:0/2094953266 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92dc0ce110 msgr2=0x7f92dc161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92d3e00640 1 --2- 172.21.15.42:0/2094953266 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92dc0ce110 0x7f92dc161020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92d3e00640 1 -- 172.21.15.42:0/2094953266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92dc06c940 msgr2=0x7f92dc15e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92d3e00640 1 --2- 172.21.15.42:0/2094953266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92dc06c940 0x7f92dc15e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92d3e00640 1 -- 172.21.15.42:0/2094953266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc15eea0 con 0x7f92dc06e740 2024-05-28T14:09:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92d3e00640 1 --2- 172.21.15.42:0/2094953266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc06e740 0x7f92dc160ae0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f92c40ad040 tx=0x7f92c401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:03.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92d2a00640 1 -- 172.21.15.42:0/2094953266 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c40be3f0 con 0x7f92dc06e740 2024-05-28T14:09:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92d2a00640 1 -- 172.21.15.42:0/2094953266 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c40be5b0 con 0x7f92dc06e740 2024-05-28T14:09:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92d2a00640 1 -- 172.21.15.42:0/2094953266 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c40be9a0 con 0x7f92dc06e740 2024-05-28T14:09:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92e1800640 1 -- 172.21.15.42:0/2094953266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92dc15f180 con 0x7f92dc06e740 2024-05-28T14:09:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92e1800640 1 -- 172.21.15.42:0/2094953266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92dc0b1fd0 con 0x7f92dc06e740 2024-05-28T14:09:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.845+0000 7f92e1800640 1 -- 172.21.15.42:0/2094953266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92dc068a90 con 0x7f92dc06e740 2024-05-28T14:09:03.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.849+0000 7f92d2a00640 1 -- 172.21.15.42:0/2094953266 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 30) ==== 89619+0+0 (secure 0 0 0) 0x7f92c40b43f0 con 0x7f92dc06e740 2024-05-28T14:09:03.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.849+0000 7f92d2a00640 1 --2- 172.21.15.42:0/2094953266 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92b8069580 0x7f92b806ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:03.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.849+0000 7f92d2a00640 1 -- 172.21.15.42:0/2094953266 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5851+0+0 (secure 0 0 0) 0x7f92c411f730 con 0x7f92dc06e740 2024-05-28T14:09:03.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.849+0000 7f92daa00640 1 --2- 172.21.15.42:0/2094953266 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92b8069580 0x7f92b806ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:03.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.849+0000 7f92daa00640 1 --2- 172.21.15.42:0/2094953266 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92b8069580 0x7f92b806ba40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f92dc15a440 tx=0x7f92cc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:03.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:03.853+0000 7f92d2a00640 1 -- 172.21.15.42:0/2094953266 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f92c4016150 con 0x7f92dc06e740 2024-05-28T14:09:04.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.041+0000 7f92e1800640 1 -- 172.21.15.42:0/2094953266 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone8115", "target": ["mon-mgr", ""]}) -- 0x7f92dc161560 con 0x7f92b8069580 2024-05-28T14:09:04.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.049+0000 7f92d2a00640 1 -- 172.21.15.42:0/2094953266 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f92c40ed760 con 0x7f92dc06e740 2024-05-28T14:09:04.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.061+0000 7f92d2a00640 1 -- 172.21.15.42:0/2094953266 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f92dc161560 con 0x7f92b8069580 2024-05-28T14:09:04.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.065+0000 7f92e1800640 1 -- 172.21.15.42:0/2094953266 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92b8069580 msgr2=0x7f92b806ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:04.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.065+0000 7f92e1800640 1 --2- 172.21.15.42:0/2094953266 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92b8069580 0x7f92b806ba40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f92dc15a440 tx=0x7f92cc008040 comp rx=0 tx=0).stop 2024-05-28T14:09:04.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.065+0000 7f92e1800640 1 -- 172.21.15.42:0/2094953266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc06e740 msgr2=0x7f92dc160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:04.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.065+0000 7f92e1800640 1 --2- 172.21.15.42:0/2094953266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc06e740 0x7f92dc160ae0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f92c40ad040 tx=0x7f92c401ad70 comp rx=0 tx=0).stop 2024-05-28T14:09:04.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.065+0000 7f92e1800640 1 -- 172.21.15.42:0/2094953266 shutdown_connections 2024-05-28T14:09:04.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.065+0000 7f92e1800640 1 --2- 172.21.15.42:0/2094953266 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92b8069580 0x7f92b806ba40 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.065+0000 7f92e1800640 1 --2- 172.21.15.42:0/2094953266 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92dc0ce110 0x7f92dc161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.065+0000 7f92e1800640 1 --2- 172.21.15.42:0/2094953266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc06e740 0x7f92dc160ae0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.065+0000 7f92e1800640 1 --2- 172.21.15.42:0/2094953266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92dc06c940 0x7f92dc15e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.065+0000 7f92e1800640 1 -- 172.21.15.42:0/2094953266 >> 172.21.15.42:0/2094953266 conn(0x7f92dc0b3fa0 msgr2=0x7f92dc0d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:04.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.065+0000 7f92e1800640 1 -- 172.21.15.42:0/2094953266 shutdown_connections 2024-05-28T14:09:04.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.065+0000 7f92e1800640 1 -- 172.21.15.42:0/2094953266 wait complete. 2024-05-28T14:09:04.075 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone81118 2024-05-28T14:09:04.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.237+0000 7f86b9600640 1 Processor -- start 2024-05-28T14:09:04.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.237+0000 7f86b9600640 1 -- start start 2024-05-28T14:09:04.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.237+0000 7f86b9600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f86b415b350 0x7f86b406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:04.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.237+0000 7f86b9600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86b4072d40 0x7f86b40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:04.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86b9600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f86b40737c0 0x7f86b40d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:04.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86b9600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f86b40ae9f0 con 0x7f86b415b350 2024-05-28T14:09:04.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86b9600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f86b40ade40 con 0x7f86b40737c0 2024-05-28T14:09:04.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86b9600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f86b40ae0c0 con 0x7f86b4072d40 2024-05-28T14:09:04.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86b2a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f86b415b350 0x7f86b406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:04.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86b2a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f86b415b350 0x7f86b406dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48712/0 (socket says 172.21.15.42:48712) 2024-05-28T14:09:04.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86b2a00640 1 -- 172.21.15.42:0/2133611359 learned_addr learned my addr 172.21.15.42:0/2133611359 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:04.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86b3400640 1 --2- 172.21.15.42:0/2133611359 >> v2:172.21.15.63:3300/0 conn(0x7f86b40737c0 0x7f86b40d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:04.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86aa000640 1 --2- 172.21.15.42:0/2133611359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86b4072d40 0x7f86b40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:04.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86b2a00640 1 -- 172.21.15.42:0/2133611359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86b4072d40 msgr2=0x7f86b40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:04.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86b2a00640 1 --2- 172.21.15.42:0/2133611359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86b4072d40 0x7f86b40d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86b2a00640 1 -- 172.21.15.42:0/2133611359 >> v2:172.21.15.63:3300/0 conn(0x7f86b40737c0 msgr2=0x7f86b40d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:04.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86b2a00640 1 --2- 172.21.15.42:0/2133611359 >> v2:172.21.15.63:3300/0 conn(0x7f86b40737c0 0x7f86b40d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86b2a00640 1 -- 172.21.15.42:0/2133611359 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b40ae3a0 con 0x7f86b415b350 2024-05-28T14:09:04.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.241+0000 7f86b2a00640 1 --2- 172.21.15.42:0/2133611359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86b415b350 0x7f86b406dd40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f86b40ae960 tx=0x7f86a401eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622748377950f378 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:04.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b2000640 1 -- 172.21.15.42:0/2133611359 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a40b4070 con 0x7f86b415b350 2024-05-28T14:09:04.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b2000640 1 -- 172.21.15.42:0/2133611359 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a40a7960 con 0x7f86b415b350 2024-05-28T14:09:04.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b2000640 1 -- 172.21.15.42:0/2133611359 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a40a7d30 con 0x7f86b415b350 2024-05-28T14:09:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 -- 172.21.15.42:0/2133611359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86b415b350 msgr2=0x7f86b406dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 --2- 172.21.15.42:0/2133611359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86b415b350 0x7f86b406dd40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f86b40ae960 tx=0x7f86a401eb60 comp rx=0 tx=0).stop 2024-05-28T14:09:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 -- 172.21.15.42:0/2133611359 shutdown_connections 2024-05-28T14:09:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 --2- 172.21.15.42:0/2133611359 >> v2:172.21.15.63:3300/0 conn(0x7f86b40737c0 0x7f86b40d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 --2- 172.21.15.42:0/2133611359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86b4072d40 0x7f86b40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 --2- 172.21.15.42:0/2133611359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86b415b350 0x7f86b406dd40 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 -- 172.21.15.42:0/2133611359 >> 172.21.15.42:0/2133611359 conn(0x7f86b406bdf0 msgr2=0x7f86b406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 -- 172.21.15.42:0/2133611359 shutdown_connections 2024-05-28T14:09:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 -- 172.21.15.42:0/2133611359 wait complete. 2024-05-28T14:09:04.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 Processor -- start 2024-05-28T14:09:04.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 -- start start 2024-05-28T14:09:04.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86b4072d40 0x7f86b414ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:04.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86b40737c0 0x7f86b414be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:04.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86b415b350 0x7f86b40c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:04.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f86b40af810 con 0x7f86b40737c0 2024-05-28T14:09:04.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86b40af690 con 0x7f86b415b350 2024-05-28T14:09:04.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b9600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f86b40af990 con 0x7f86b4072d40 2024-05-28T14:09:04.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b2a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86b4072d40 0x7f86b414ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:04.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b2a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86b4072d40 0x7f86b414ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:34828/0 (socket says 172.21.15.42:34828) 2024-05-28T14:09:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86b2a00640 1 -- 172.21.15.42:0/1766403605 learned_addr learned my addr 172.21.15.42:0/1766403605 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.245+0000 7f86aa000640 1 --2- 172.21.15.42:0/1766403605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86b40737c0 0x7f86b414be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86b3400640 1 --2- 172.21.15.42:0/1766403605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86b415b350 0x7f86b40c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86b2a00640 1 -- 172.21.15.42:0/1766403605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86b415b350 msgr2=0x7f86b40c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86b2a00640 1 --2- 172.21.15.42:0/1766403605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86b415b350 0x7f86b40c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86b2a00640 1 -- 172.21.15.42:0/1766403605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86b40737c0 msgr2=0x7f86b414be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86b2a00640 1 --2- 172.21.15.42:0/1766403605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86b40737c0 0x7f86b414be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86b2a00640 1 -- 172.21.15.42:0/1766403605 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b414b140 con 0x7f86b4072d40 2024-05-28T14:09:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86b2a00640 1 --2- 172.21.15.42:0/1766403605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86b4072d40 0x7f86b414ac00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86a401f400 tx=0x7f86a401d3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:04.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86ab400640 1 -- 172.21.15.42:0/1766403605 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a40bf070 con 0x7f86b4072d40 2024-05-28T14:09:04.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86ab400640 1 -- 172.21.15.42:0/1766403605 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a40af930 con 0x7f86b4072d40 2024-05-28T14:09:04.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86ab400640 1 -- 172.21.15.42:0/1766403605 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a40b4070 con 0x7f86b4072d40 2024-05-28T14:09:04.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86b9600640 1 -- 172.21.15.42:0/1766403605 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b414c7d0 con 0x7f86b4072d40 2024-05-28T14:09:04.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86b9600640 1 -- 172.21.15.42:0/1766403605 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b414cd90 con 0x7f86b4072d40 2024-05-28T14:09:04.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86b9600640 1 -- 172.21.15.42:0/1766403605 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b414e940 con 0x7f86b4072d40 2024-05-28T14:09:04.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86ab400640 1 -- 172.21.15.42:0/1766403605 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f86a40c2070 con 0x7f86b4072d40 2024-05-28T14:09:04.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86ab400640 1 --2- 172.21.15.42:0/1766403605 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f86900694f0 0x7f869006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:04.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.249+0000 7f86ab400640 1 -- 172.21.15.42:0/1766403605 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(77..77 src has 1..77) ==== 5851+0+0 (secure 0 0 0) 0x7f86a411f8e0 con 0x7f86b4072d40 2024-05-28T14:09:04.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.253+0000 7f86aa000640 1 --2- 172.21.15.42:0/1766403605 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f86900694f0 0x7f869006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:04.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.253+0000 7f86aa000640 1 --2- 172.21.15.42:0/1766403605 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f86900694f0 0x7f869006b9b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f86b41519f0 tx=0x7f869c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:04.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.257+0000 7f86ab400640 1 -- 172.21.15.42:0/1766403605 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f86a40187e0 con 0x7f86b4072d40 2024-05-28T14:09:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.449+0000 7f86b9600640 1 -- 172.21.15.42:0/1766403605 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone81118", "target": ["mon-mgr", ""]}) -- 0x7f86b4068a90 con 0x7f86900694f0 2024-05-28T14:09:04.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.465+0000 7f86ab400640 1 -- 172.21.15.42:0/1766403605 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f86b4068a90 con 0x7f86900694f0 2024-05-28T14:09:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.469+0000 7f86b9600640 1 -- 172.21.15.42:0/1766403605 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f86900694f0 msgr2=0x7f869006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.469+0000 7f86b9600640 1 --2- 172.21.15.42:0/1766403605 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f86900694f0 0x7f869006b9b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f86b41519f0 tx=0x7f869c009190 comp rx=0 tx=0).stop 2024-05-28T14:09:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.469+0000 7f86b9600640 1 -- 172.21.15.42:0/1766403605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86b4072d40 msgr2=0x7f86b414ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.469+0000 7f86b9600640 1 --2- 172.21.15.42:0/1766403605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86b4072d40 0x7f86b414ac00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86a401f400 tx=0x7f86a401d3a0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.469+0000 7f86b9600640 1 -- 172.21.15.42:0/1766403605 shutdown_connections 2024-05-28T14:09:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.469+0000 7f86b9600640 1 --2- 172.21.15.42:0/1766403605 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f86900694f0 0x7f869006b9b0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.469+0000 7f86b9600640 1 --2- 172.21.15.42:0/1766403605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86b415b350 0x7f86b40c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.469+0000 7f86b9600640 1 --2- 172.21.15.42:0/1766403605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86b40737c0 0x7f86b414be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.469+0000 7f86b9600640 1 --2- 172.21.15.42:0/1766403605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86b4072d40 0x7f86b414ac00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.469+0000 7f86b9600640 1 -- 172.21.15.42:0/1766403605 >> 172.21.15.42:0/1766403605 conn(0x7f86b406bdf0 msgr2=0x7f86b406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:04.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.469+0000 7f86b9600640 1 -- 172.21.15.42:0/1766403605 shutdown_connections 2024-05-28T14:09:04.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.469+0000 7f86b9600640 1 -- 172.21.15.42:0/1766403605 wait complete. 2024-05-28T14:09:04.483 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone81117 2024-05-28T14:09:04.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.645+0000 7fe04f200640 1 Processor -- start 2024-05-28T14:09:04.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.645+0000 7fe04f200640 1 -- start start 2024-05-28T14:09:04.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.645+0000 7fe04f200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0480d0e00 0x7fe04806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:04.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.645+0000 7fe04f200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe048072d40 0x7fe0480aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:04.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.645+0000 7fe04f200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0480ade40 con 0x7fe04815b350 2024-05-28T14:09:04.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.645+0000 7fe04f200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe0480ae9f0 con 0x7fe048072d40 2024-05-28T14:09:04.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.645+0000 7fe04f200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0480ae0c0 con 0x7fe0480d0e00 2024-05-28T14:09:04.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.649+0000 7fe04d200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe048072d40 0x7fe0480aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:04.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.649+0000 7fe03f400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0480d0e00 0x7fe04806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:04.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.649+0000 7fe047e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fe04815b350 0x7fe0480d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:38306/0 (socket says 172.21.15.42:38306) 2024-05-28T14:09:04.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.649+0000 7fe04d200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe048072d40 0x7fe0480aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:46862/0 (socket says 172.21.15.42:46862) 2024-05-28T14:09:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.649+0000 7fe047e00640 1 -- 172.21.15.42:0/1457437621 learned_addr learned my addr 172.21.15.42:0/1457437621 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.649+0000 7fe04d200640 1 -- 172.21.15.42:0/1457437621 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0480d0e00 msgr2=0x7fe04806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.649+0000 7fe04d200640 1 --2- 172.21.15.42:0/1457437621 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0480d0e00 0x7fe04806dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.649+0000 7fe04d200640 1 -- 172.21.15.42:0/1457437621 >> v1:172.21.15.63:6789/0 conn(0x7fe04815b350 legacy=0x7fe0480d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.649+0000 7fe04d200640 1 -- 172.21.15.42:0/1457437621 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0480ae3a0 con 0x7fe048072d40 2024-05-28T14:09:04.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.649+0000 7fe047400640 1 -- 172.21.15.42:0/1457437621 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1798234875 0 0) 0x7fe0380014a0 con 0x7fe04815b350 2024-05-28T14:09:04.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.649+0000 7fe04d200640 1 --2- 172.21.15.42:0/1457437621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe048072d40 0x7fe0480aee80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe0400097c0 tx=0x7fe04001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cb74ea99a402770 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:04.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.649+0000 7fe047400640 1 -- 172.21.15.42:0/1457437621 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0400b0070 con 0x7fe048072d40 2024-05-28T14:09:04.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.649+0000 7fe047400640 1 -- 172.21.15.42:0/1457437621 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0400ab600 con 0x7fe048072d40 2024-05-28T14:09:04.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.649+0000 7fe047400640 1 -- 172.21.15.42:0/1457437621 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0400ab9d0 con 0x7fe048072d40 2024-05-28T14:09:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 -- 172.21.15.42:0/1457437621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe048072d40 msgr2=0x7fe0480aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 --2- 172.21.15.42:0/1457437621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe048072d40 0x7fe0480aee80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe0400097c0 tx=0x7fe04001c410 comp rx=0 tx=0).stop 2024-05-28T14:09:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 -- 172.21.15.42:0/1457437621 shutdown_connections 2024-05-28T14:09:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 --2- 172.21.15.42:0/1457437621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe048072d40 0x7fe0480aee80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 --2- 172.21.15.42:0/1457437621 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0480d0e00 0x7fe04806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 -- 172.21.15.42:0/1457437621 >> 172.21.15.42:0/1457437621 conn(0x7fe04806bdf0 msgr2=0x7fe04806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 -- 172.21.15.42:0/1457437621 shutdown_connections 2024-05-28T14:09:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 -- 172.21.15.42:0/1457437621 wait complete. 2024-05-28T14:09:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 Processor -- start 2024-05-28T14:09:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 -- start start 2024-05-28T14:09:04.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe048072d40 0x7fe04815e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:04.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0480d0e00 0x7fe048150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:04.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe04815b350 0x7fe0480b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:04.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe048166460 con 0x7fe048072d40 2024-05-28T14:09:04.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe0481662e0 con 0x7fe04815b350 2024-05-28T14:09:04.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04f200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0481665e0 con 0x7fe0480d0e00 2024-05-28T14:09:04.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe047e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe048072d40 0x7fe04815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:04.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe047e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe048072d40 0x7fe04815e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48730/0 (socket says 172.21.15.42:48730) 2024-05-28T14:09:04.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe04d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe04815b350 0x7fe0480b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:04.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe047e00640 1 -- 172.21.15.42:0/4044160935 learned_addr learned my addr 172.21.15.42:0/4044160935 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:04.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe03f400640 1 --2- 172.21.15.42:0/4044160935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0480d0e00 0x7fe048150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:04.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe047e00640 1 -- 172.21.15.42:0/4044160935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0480d0e00 msgr2=0x7fe048150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe047e00640 1 --2- 172.21.15.42:0/4044160935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0480d0e00 0x7fe048150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe047e00640 1 -- 172.21.15.42:0/4044160935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe04815b350 msgr2=0x7fe0480b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe047e00640 1 --2- 172.21.15.42:0/4044160935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe04815b350 0x7fe0480b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.653+0000 7fe047e00640 1 -- 172.21.15.42:0/4044160935 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0481fb2a0 con 0x7fe048072d40 2024-05-28T14:09:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.657+0000 7fe047e00640 1 --2- 172.21.15.42:0/4044160935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe048072d40 0x7fe04815e170 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe03800c7c0 tx=0x7fe03800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.657+0000 7fe044c00640 1 -- 172.21.15.42:0/4044160935 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038018070 con 0x7fe048072d40 2024-05-28T14:09:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.657+0000 7fe044c00640 1 -- 172.21.15.42:0/4044160935 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe03800a640 con 0x7fe048072d40 2024-05-28T14:09:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.657+0000 7fe044c00640 1 -- 172.21.15.42:0/4044160935 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03800aa30 con 0x7fe048072d40 2024-05-28T14:09:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.657+0000 7fe04f200640 1 -- 172.21.15.42:0/4044160935 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0481fb580 con 0x7fe048072d40 2024-05-28T14:09:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.657+0000 7fe04f200640 1 -- 172.21.15.42:0/4044160935 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0481fbb40 con 0x7fe048072d40 2024-05-28T14:09:04.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.657+0000 7fe04f200640 1 -- 172.21.15.42:0/4044160935 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe048068a90 con 0x7fe048072d40 2024-05-28T14:09:04.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.657+0000 7fe044c00640 1 -- 172.21.15.42:0/4044160935 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fe03800abd0 con 0x7fe048072d40 2024-05-28T14:09:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.657+0000 7fe044c00640 1 --2- 172.21.15.42:0/4044160935 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe024069520 0x7fe02406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.657+0000 7fe044c00640 1 -- 172.21.15.42:0/4044160935 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5851+0+0 (secure 0 0 0) 0x7fe038084da0 con 0x7fe048072d40 2024-05-28T14:09:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.657+0000 7fe03f400640 1 --2- 172.21.15.42:0/4044160935 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe024069520 0x7fe02406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.661+0000 7fe03f400640 1 --2- 172.21.15.42:0/4044160935 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe024069520 0x7fe02406b9e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe030009b60 tx=0x7fe030009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:04.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.661+0000 7fe044c00640 1 -- 172.21.15.42:0/4044160935 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe0480ade40 con 0x7fe048072d40 2024-05-28T14:09:04.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.853+0000 7fe04f200640 1 -- 172.21.15.42:0/4044160935 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone81117", "target": ["mon-mgr", ""]}) -- 0x7fe048072b20 con 0x7fe024069520 2024-05-28T14:09:04.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.865+0000 7fe044c00640 1 -- 172.21.15.42:0/4044160935 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe048072b20 con 0x7fe024069520 2024-05-28T14:09:04.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.869+0000 7fe04f200640 1 -- 172.21.15.42:0/4044160935 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe024069520 msgr2=0x7fe02406b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:04.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.869+0000 7fe04f200640 1 --2- 172.21.15.42:0/4044160935 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe024069520 0x7fe02406b9e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe030009b60 tx=0x7fe030009190 comp rx=0 tx=0).stop 2024-05-28T14:09:04.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.869+0000 7fe04f200640 1 -- 172.21.15.42:0/4044160935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe048072d40 msgr2=0x7fe04815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:04.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.869+0000 7fe04f200640 1 --2- 172.21.15.42:0/4044160935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe048072d40 0x7fe04815e170 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe03800c7c0 tx=0x7fe03800cc80 comp rx=0 tx=0).stop 2024-05-28T14:09:04.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.869+0000 7fe04f200640 1 -- 172.21.15.42:0/4044160935 shutdown_connections 2024-05-28T14:09:04.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.869+0000 7fe04f200640 1 --2- 172.21.15.42:0/4044160935 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe024069520 0x7fe02406b9e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.869+0000 7fe04f200640 1 --2- 172.21.15.42:0/4044160935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe04815b350 0x7fe0480b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.869+0000 7fe04f200640 1 --2- 172.21.15.42:0/4044160935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0480d0e00 0x7fe048150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.869+0000 7fe04f200640 1 --2- 172.21.15.42:0/4044160935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe048072d40 0x7fe04815e170 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:04.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.869+0000 7fe04f200640 1 -- 172.21.15.42:0/4044160935 >> 172.21.15.42:0/4044160935 conn(0x7fe04806bdf0 msgr2=0x7fe0480b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:04.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.869+0000 7fe04f200640 1 -- 172.21.15.42:0/4044160935 shutdown_connections 2024-05-28T14:09:04.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:04.869+0000 7fe04f200640 1 -- 172.21.15.42:0/4044160935 wait complete. 2024-05-28T14:09:04.880 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone81111 2024-05-28T14:09:05.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.041+0000 7f3d8ca00640 1 Processor -- start 2024-05-28T14:09:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.041+0000 7f3d8ca00640 1 -- start start 2024-05-28T14:09:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.041+0000 7f3d8ca00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3d8806e740 0x7f3d880cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.041+0000 7f3d8ca00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3d8809b160 0x7f3d88070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.041+0000 7f3d8ca00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d8806ff10 con 0x7f3d880ce110 2024-05-28T14:09:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.041+0000 7f3d8ca00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f3d8806eb20 con 0x7f3d8809b160 2024-05-28T14:09:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.041+0000 7f3d8ca00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3d880701b0 con 0x7f3d8806e740 2024-05-28T14:09:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.045+0000 7f3d7d400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f3d880ce110 0x7f3d8809aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38748/0 (socket says 172.21.15.42:38748) 2024-05-28T14:09:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.045+0000 7f3d7d400640 1 -- 172.21.15.42:0/213320172 learned_addr learned my addr 172.21.15.42:0/213320172 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.045+0000 7f3d86800640 1 --2- 172.21.15.42:0/213320172 >> v2:172.21.15.42:3300/0 conn(0x7f3d8809b160 0x7f3d88070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.045+0000 7f3d85e00640 1 --2- 172.21.15.42:0/213320172 >> v2:172.21.15.63:3300/0 conn(0x7f3d8806e740 0x7f3d880cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.045+0000 7f3d86800640 1 -- 172.21.15.42:0/213320172 >> v2:172.21.15.63:3300/0 conn(0x7f3d8806e740 msgr2=0x7f3d880cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.045+0000 7f3d86800640 1 --2- 172.21.15.42:0/213320172 >> v2:172.21.15.63:3300/0 conn(0x7f3d8806e740 0x7f3d880cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.045+0000 7f3d86800640 1 -- 172.21.15.42:0/213320172 >> v1:172.21.15.42:6789/0 conn(0x7f3d880ce110 legacy=0x7f3d8809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.045+0000 7f3d86800640 1 -- 172.21.15.42:0/213320172 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d88070490 con 0x7f3d8809b160 2024-05-28T14:09:05.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.045+0000 7f3d86800640 1 --2- 172.21.15.42:0/213320172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8809b160 0x7f3d88070f70 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f3d8806f990 tx=0x7f3d8001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76046d646143cdda server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.045+0000 7f3d85400640 1 -- 172.21.15.42:0/213320172 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d800b2070 con 0x7f3d8809b160 2024-05-28T14:09:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.045+0000 7f3d85400640 1 -- 172.21.15.42:0/213320172 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d800a59c0 con 0x7f3d8809b160 2024-05-28T14:09:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.045+0000 7f3d85400640 1 -- 172.21.15.42:0/213320172 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d800a5d90 con 0x7f3d8809b160 2024-05-28T14:09:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 -- 172.21.15.42:0/213320172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8809b160 msgr2=0x7f3d88070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 --2- 172.21.15.42:0/213320172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8809b160 0x7f3d88070f70 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f3d8806f990 tx=0x7f3d8001c900 comp rx=0 tx=0).stop 2024-05-28T14:09:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 -- 172.21.15.42:0/213320172 shutdown_connections 2024-05-28T14:09:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 --2- 172.21.15.42:0/213320172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8809b160 0x7f3d88070f70 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 --2- 172.21.15.42:0/213320172 >> v2:172.21.15.63:3300/0 conn(0x7f3d8806e740 0x7f3d880cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 -- 172.21.15.42:0/213320172 >> 172.21.15.42:0/213320172 conn(0x7f3d880b3fa0 msgr2=0x7f3d880b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:05.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 -- 172.21.15.42:0/213320172 shutdown_connections 2024-05-28T14:09:05.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 -- 172.21.15.42:0/213320172 wait complete. 2024-05-28T14:09:05.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 Processor -- start 2024-05-28T14:09:05.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 -- start start 2024-05-28T14:09:05.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d8806e740 0x7f3d881477f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8809b160 0x7f3d8815f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d880ce110 0x7f3d880cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8815e2a0 con 0x7f3d8809b160 2024-05-28T14:09:05.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8815e120 con 0x7f3d880ce110 2024-05-28T14:09:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d8ca00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8815e420 con 0x7f3d8806e740 2024-05-28T14:09:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d7d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8809b160 0x7f3d8815f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d7d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8809b160 0x7f3d8815f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48746/0 (socket says 172.21.15.42:48746) 2024-05-28T14:09:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d7d400640 1 -- 172.21.15.42:0/3685694821 learned_addr learned my addr 172.21.15.42:0/3685694821 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d86800640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d880ce110 0x7f3d880cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d85e00640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d8806e740 0x7f3d881477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d7d400640 1 -- 172.21.15.42:0/3685694821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d8806e740 msgr2=0x7f3d881477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d7d400640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d8806e740 0x7f3d881477f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d7d400640 1 -- 172.21.15.42:0/3685694821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d880ce110 msgr2=0x7f3d880cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d7d400640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d880ce110 0x7f3d880cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d7d400640 1 -- 172.21.15.42:0/3685694821 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8815fa70 con 0x7f3d8809b160 2024-05-28T14:09:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.049+0000 7f3d85e00640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d8806e740 0x7f3d881477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.053+0000 7f3d7d400640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8809b160 0x7f3d8815f220 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800c370 tx=0x7f3d7800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:05.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.053+0000 7f3d7ea00640 1 -- 172.21.15.42:0/3685694821 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78017070 con 0x7f3d8809b160 2024-05-28T14:09:05.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.053+0000 7f3d7ea00640 1 -- 172.21.15.42:0/3685694821 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d78005cf0 con 0x7f3d8809b160 2024-05-28T14:09:05.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.053+0000 7f3d7ea00640 1 -- 172.21.15.42:0/3685694821 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780072f0 con 0x7f3d8809b160 2024-05-28T14:09:05.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.053+0000 7f3d8ca00640 1 -- 172.21.15.42:0/3685694821 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8815fd50 con 0x7f3d8809b160 2024-05-28T14:09:05.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.053+0000 7f3d8ca00640 1 -- 172.21.15.42:0/3685694821 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8815f760 con 0x7f3d8809b160 2024-05-28T14:09:05.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.053+0000 7f3d8ca00640 1 -- 172.21.15.42:0/3685694821 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d88068a90 con 0x7f3d8809b160 2024-05-28T14:09:05.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.053+0000 7f3d7ea00640 1 -- 172.21.15.42:0/3685694821 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f3d7801a070 con 0x7f3d8809b160 2024-05-28T14:09:05.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.053+0000 7f3d7ea00640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d640694f0 0x7f3d6406b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.053+0000 7f3d7ea00640 1 -- 172.21.15.42:0/3685694821 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5851+0+0 (secure 0 0 0) 0x7f3d78084c90 con 0x7f3d8809b160 2024-05-28T14:09:05.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.053+0000 7f3d85e00640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d640694f0 0x7f3d6406b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.057+0000 7f3d85e00640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d640694f0 0x7f3d6406b9b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009ae0 tx=0x7f3d70008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:05.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.057+0000 7f3d7ea00640 1 -- 172.21.15.42:0/3685694821 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3d780568c0 con 0x7f3d8809b160 2024-05-28T14:09:05.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.249+0000 7f3d8ca00640 1 -- 172.21.15.42:0/3685694821 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone81111", "target": ["mon-mgr", ""]}) -- 0x7f3d880ce9a0 con 0x7f3d640694f0 2024-05-28T14:09:05.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.265+0000 7f3d7ea00640 1 -- 172.21.15.42:0/3685694821 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3d880ce9a0 con 0x7f3d640694f0 2024-05-28T14:09:05.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.265+0000 7f3d8ca00640 1 -- 172.21.15.42:0/3685694821 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d640694f0 msgr2=0x7f3d6406b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.265+0000 7f3d8ca00640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d640694f0 0x7f3d6406b9b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009ae0 tx=0x7f3d70008040 comp rx=0 tx=0).stop 2024-05-28T14:09:05.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.265+0000 7f3d8ca00640 1 -- 172.21.15.42:0/3685694821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8809b160 msgr2=0x7f3d8815f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.265+0000 7f3d8ca00640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8809b160 0x7f3d8815f220 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800c370 tx=0x7f3d7800c830 comp rx=0 tx=0).stop 2024-05-28T14:09:05.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.265+0000 7f3d8ca00640 1 -- 172.21.15.42:0/3685694821 shutdown_connections 2024-05-28T14:09:05.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.265+0000 7f3d8ca00640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3d640694f0 0x7f3d6406b9b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.265+0000 7f3d8ca00640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d880ce110 0x7f3d880cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.265+0000 7f3d8ca00640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8809b160 0x7f3d8815f220 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.265+0000 7f3d8ca00640 1 --2- 172.21.15.42:0/3685694821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d8806e740 0x7f3d881477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.265+0000 7f3d8ca00640 1 -- 172.21.15.42:0/3685694821 >> 172.21.15.42:0/3685694821 conn(0x7f3d880b3fa0 msgr2=0x7f3d8809aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:05.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.269+0000 7f3d8ca00640 1 -- 172.21.15.42:0/3685694821 shutdown_connections 2024-05-28T14:09:05.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.269+0000 7f3d8ca00640 1 -- 172.21.15.42:0/3685694821 wait complete. 2024-05-28T14:09:05.274 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone8112 2024-05-28T14:09:05.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.437+0000 7f0ea5e00640 1 Processor -- start 2024-05-28T14:09:05.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.437+0000 7f0ea5e00640 1 -- start start 2024-05-28T14:09:05.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.437+0000 7f0ea5e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0ea009b160 0x7f0ea0070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.437+0000 7f0ea5e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea006ff10 con 0x7f0ea006e740 2024-05-28T14:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.437+0000 7f0ea5e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea00701b0 con 0x7f0ea006c940 2024-05-28T14:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.437+0000 7f0ea5e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0ea006eb20 con 0x7f0ea009b160 2024-05-28T14:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.437+0000 7f0e9ea00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0ea006e740 0x7f0ea00cdbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38758/0 (socket says 172.21.15.42:38758) 2024-05-28T14:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.437+0000 7f0e9ea00640 1 -- 172.21.15.42:0/1089642033 learned_addr learned my addr 172.21.15.42:0/1089642033 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.437+0000 7f0e9f400640 1 --2- 172.21.15.42:0/1089642033 >> v2:172.21.15.63:3300/0 conn(0x7f0ea009b160 0x7f0ea0070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.441+0000 7f0e9d600640 1 -- 172.21.15.42:0/1089642033 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2366756648 0 0) 0x7f0ea006ff10 con 0x7f0ea006e740 2024-05-28T14:09:05.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.441+0000 7f0e9d600640 1 -- 172.21.15.42:0/1089642033 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e84003530 con 0x7f0ea006e740 2024-05-28T14:09:05.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.441+0000 7f0e9d600640 1 -- 172.21.15.42:0/1089642033 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 635367837 0 0) 0x7f0ea00701b0 con 0x7f0ea006c940 2024-05-28T14:09:05.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.441+0000 7f0e9d600640 1 -- 172.21.15.42:0/1089642033 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ea006ff10 con 0x7f0ea006c940 2024-05-28T14:09:05.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.441+0000 7f0e9f400640 1 -- 172.21.15.42:0/1089642033 >> v1:172.21.15.63:6789/0 conn(0x7f0ea006c940 legacy=0x7f0ea009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.441+0000 7f0e9f400640 1 -- 172.21.15.42:0/1089642033 >> v1:172.21.15.42:6789/0 conn(0x7f0ea006e740 legacy=0x7f0ea00cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.441+0000 7f0e9f400640 1 -- 172.21.15.42:0/1089642033 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea0070490 con 0x7f0ea009b160 2024-05-28T14:09:05.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.441+0000 7f0e9f400640 1 --2- 172.21.15.42:0/1089642033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ea009b160 0x7f0ea0070f70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009d90 tx=0x7f0e9801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b14c9389efa506f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:05.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.441+0000 7f0e9d600640 1 -- 172.21.15.42:0/1089642033 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e980b2070 con 0x7f0ea009b160 2024-05-28T14:09:05.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.441+0000 7f0e9d600640 1 -- 172.21.15.42:0/1089642033 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e980a5970 con 0x7f0ea009b160 2024-05-28T14:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.441+0000 7f0e9d600640 1 -- 172.21.15.42:0/1089642033 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e980a5d40 con 0x7f0ea009b160 2024-05-28T14:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 -- 172.21.15.42:0/1089642033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ea009b160 msgr2=0x7f0ea0070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 --2- 172.21.15.42:0/1089642033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ea009b160 0x7f0ea0070f70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009d90 tx=0x7f0e9801c8b0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 -- 172.21.15.42:0/1089642033 shutdown_connections 2024-05-28T14:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 --2- 172.21.15.42:0/1089642033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ea009b160 0x7f0ea0070f70 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 -- 172.21.15.42:0/1089642033 >> 172.21.15.42:0/1089642033 conn(0x7f0ea00b3fa0 msgr2=0x7f0ea00b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 -- 172.21.15.42:0/1089642033 shutdown_connections 2024-05-28T14:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 -- 172.21.15.42:0/1089642033 wait complete. 2024-05-28T14:09:05.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 Processor -- start 2024-05-28T14:09:05.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 -- start start 2024-05-28T14:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ea006c940 0x7f0ea00d1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ea006e740 0x7f0ea00c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ea009b160 0x7f0ea00b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea015e120 con 0x7f0ea006c940 2024-05-28T14:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea009baa0 con 0x7f0ea009b160 2024-05-28T14:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0ea5e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea015e2a0 con 0x7f0ea006e740 2024-05-28T14:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0e9ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ea006c940 0x7f0ea00d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0e9ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ea006c940 0x7f0ea00d1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48752/0 (socket says 172.21.15.42:48752) 2024-05-28T14:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0e9ea00640 1 -- 172.21.15.42:0/777211613 learned_addr learned my addr 172.21.15.42:0/777211613 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0e9e000640 1 --2- 172.21.15.42:0/777211613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ea006e740 0x7f0ea00c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.445+0000 7f0e9f400640 1 --2- 172.21.15.42:0/777211613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ea009b160 0x7f0ea00b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.449+0000 7f0e9ea00640 1 -- 172.21.15.42:0/777211613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ea006e740 msgr2=0x7f0ea00c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.449+0000 7f0e9ea00640 1 --2- 172.21.15.42:0/777211613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ea006e740 0x7f0ea00c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.449+0000 7f0e9ea00640 1 -- 172.21.15.42:0/777211613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ea009b160 msgr2=0x7f0ea00b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.449+0000 7f0e9ea00640 1 --2- 172.21.15.42:0/777211613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ea009b160 0x7f0ea00b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.449+0000 7f0e9ea00640 1 -- 172.21.15.42:0/777211613 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea0154440 con 0x7f0ea006c940 2024-05-28T14:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.449+0000 7f0e9ea00640 1 --2- 172.21.15.42:0/777211613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ea006c940 0x7f0ea00d1990 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000c4d0 tx=0x7f0e9000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:05.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.449+0000 7f0e97400640 1 -- 172.21.15.42:0/777211613 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90017070 con 0x7f0ea006c940 2024-05-28T14:09:05.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.449+0000 7f0e97400640 1 -- 172.21.15.42:0/777211613 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e90005cf0 con 0x7f0ea006c940 2024-05-28T14:09:05.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.449+0000 7f0ea5e00640 1 -- 172.21.15.42:0/777211613 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea01546e0 con 0x7f0ea006c940 2024-05-28T14:09:05.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.449+0000 7f0e97400640 1 -- 172.21.15.42:0/777211613 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90007270 con 0x7f0ea006c940 2024-05-28T14:09:05.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.449+0000 7f0ea5e00640 1 -- 172.21.15.42:0/777211613 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea0154f80 con 0x7f0ea006c940 2024-05-28T14:09:05.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.449+0000 7f0ea5e00640 1 -- 172.21.15.42:0/777211613 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea0154ad0 con 0x7f0ea006c940 2024-05-28T14:09:05.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.449+0000 7f0e97400640 1 -- 172.21.15.42:0/777211613 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f0e9001a070 con 0x7f0ea006c940 2024-05-28T14:09:05.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.453+0000 7f0e97400640 1 --2- 172.21.15.42:0/777211613 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e800694f0 0x7f0e8006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.453+0000 7f0e97400640 1 -- 172.21.15.42:0/777211613 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5851+0+0 (secure 0 0 0) 0x7f0e90084ba0 con 0x7f0ea006c940 2024-05-28T14:09:05.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.453+0000 7f0e9e000640 1 --2- 172.21.15.42:0/777211613 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e800694f0 0x7f0e8006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.453+0000 7f0e9e000640 1 --2- 172.21.15.42:0/777211613 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e800694f0 0x7f0e8006b9b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0ea009ada0 tx=0x7f0e8c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:05.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.457+0000 7f0e97400640 1 -- 172.21.15.42:0/777211613 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0e900561e0 con 0x7f0ea006c940 2024-05-28T14:09:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.653+0000 7f0ea5e00640 1 -- 172.21.15.42:0/777211613 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone8112", "target": ["mon-mgr", ""]}) -- 0x7f0ea0068a90 con 0x7f0e800694f0 2024-05-28T14:09:05.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.669+0000 7f0e97400640 1 -- 172.21.15.42:0/777211613 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0ea0068a90 con 0x7f0e800694f0 2024-05-28T14:09:05.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.669+0000 7f0ea5e00640 1 -- 172.21.15.42:0/777211613 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e800694f0 msgr2=0x7f0e8006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.669+0000 7f0ea5e00640 1 --2- 172.21.15.42:0/777211613 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e800694f0 0x7f0e8006b9b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0ea009ada0 tx=0x7f0e8c009040 comp rx=0 tx=0).stop 2024-05-28T14:09:05.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.669+0000 7f0ea5e00640 1 -- 172.21.15.42:0/777211613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ea006c940 msgr2=0x7f0ea00d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.669+0000 7f0ea5e00640 1 --2- 172.21.15.42:0/777211613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ea006c940 0x7f0ea00d1990 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000c4d0 tx=0x7f0e9000c990 comp rx=0 tx=0).stop 2024-05-28T14:09:05.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.669+0000 7f0ea5e00640 1 -- 172.21.15.42:0/777211613 shutdown_connections 2024-05-28T14:09:05.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.669+0000 7f0ea5e00640 1 --2- 172.21.15.42:0/777211613 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e800694f0 0x7f0e8006b9b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.669+0000 7f0ea5e00640 1 --2- 172.21.15.42:0/777211613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ea009b160 0x7f0ea00b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.669+0000 7f0ea5e00640 1 --2- 172.21.15.42:0/777211613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ea006e740 0x7f0ea00c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.669+0000 7f0ea5e00640 1 --2- 172.21.15.42:0/777211613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ea006c940 0x7f0ea00d1990 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.669+0000 7f0ea5e00640 1 -- 172.21.15.42:0/777211613 >> 172.21.15.42:0/777211613 conn(0x7f0ea00b3fa0 msgr2=0x7f0ea00b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:05.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.673+0000 7f0ea5e00640 1 -- 172.21.15.42:0/777211613 shutdown_connections 2024-05-28T14:09:05.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.673+0000 7f0ea5e00640 1 -- 172.21.15.42:0/777211613 wait complete. 2024-05-28T14:09:05.676 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone81116 2024-05-28T14:09:05.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.841+0000 7f2cd4800640 1 Processor -- start 2024-05-28T14:09:05.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.841+0000 7f2cd4800640 1 -- start start 2024-05-28T14:09:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.845+0000 7f2cd4800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ccc0ccfc0 0x7f2ccc0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.845+0000 7f2cd4800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ccc0d1250 con 0x7f2ccc15fcc0 2024-05-28T14:09:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.845+0000 7f2cd4800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ccc0b5510 con 0x7f2ccc0cd960 2024-05-28T14:09:05.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.845+0000 7f2cd4800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2ccc0b57f0 con 0x7f2ccc0ccfc0 2024-05-28T14:09:05.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.845+0000 7f2cd1e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f2ccc15fcc0 0x7f2ccc0d0a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:38774/0 (socket says 172.21.15.42:38774) 2024-05-28T14:09:05.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.845+0000 7f2cd1e00640 1 -- 172.21.15.42:0/3455550918 learned_addr learned my addr 172.21.15.42:0/3455550918 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:05.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.845+0000 7f2cd1400640 1 --2- 172.21.15.42:0/3455550918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ccc0ccfc0 0x7f2ccc0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.845+0000 7f2cd0a00640 1 -- 172.21.15.42:0/3455550918 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3204879005 0 0) 0x7f2ccc0b5510 con 0x7f2ccc0cd960 2024-05-28T14:09:05.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.845+0000 7f2cd0a00640 1 -- 172.21.15.42:0/3455550918 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cb4007940 con 0x7f2ccc0cd960 2024-05-28T14:09:05.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.845+0000 7f2cd0a00640 1 -- 172.21.15.42:0/3455550918 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1145657734 0 0) 0x7f2ccc0d1250 con 0x7f2ccc15fcc0 2024-05-28T14:09:05.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.845+0000 7f2cd0a00640 1 -- 172.21.15.42:0/3455550918 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ccc0b5510 con 0x7f2ccc15fcc0 2024-05-28T14:09:05.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.845+0000 7f2cd1400640 1 -- 172.21.15.42:0/3455550918 >> v1:172.21.15.63:6789/0 conn(0x7f2ccc0cd960 legacy=0x7f2ccc0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.845+0000 7f2cd1400640 1 -- 172.21.15.42:0/3455550918 >> v1:172.21.15.42:6789/0 conn(0x7f2ccc15fcc0 legacy=0x7f2ccc0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.845+0000 7f2cd1400640 1 -- 172.21.15.42:0/3455550918 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc0b5ad0 con 0x7f2ccc0ccfc0 2024-05-28T14:09:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.853+0000 7f2cd1400640 1 --2- 172.21.15.42:0/3455550918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ccc0ccfc0 0x7f2ccc0cd420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc013de0 tx=0x7f2cbc024bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee739d721700e16 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.853+0000 7f2cd0a00640 1 -- 172.21.15.42:0/3455550918 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc0ba070 con 0x7f2ccc0ccfc0 2024-05-28T14:09:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.853+0000 7f2cd0a00640 1 -- 172.21.15.42:0/3455550918 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cbc0ad8a0 con 0x7f2ccc0ccfc0 2024-05-28T14:09:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.853+0000 7f2cd0a00640 1 -- 172.21.15.42:0/3455550918 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc0adc70 con 0x7f2ccc0ccfc0 2024-05-28T14:09:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.853+0000 7f2cd4800640 1 -- 172.21.15.42:0/3455550918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ccc0ccfc0 msgr2=0x7f2ccc0cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.853+0000 7f2cd4800640 1 --2- 172.21.15.42:0/3455550918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ccc0ccfc0 0x7f2ccc0cd420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc013de0 tx=0x7f2cbc024bd0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.853+0000 7f2cd4800640 1 -- 172.21.15.42:0/3455550918 shutdown_connections 2024-05-28T14:09:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.853+0000 7f2cd4800640 1 --2- 172.21.15.42:0/3455550918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ccc0ccfc0 0x7f2ccc0cd420 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc013de0 tx=0x7f2cbc024bd0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.853+0000 7f2cd4800640 1 -- 172.21.15.42:0/3455550918 >> 172.21.15.42:0/3455550918 conn(0x7f2ccc0b3fa0 msgr2=0x7f2ccc0b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.857+0000 7f2cd4800640 1 -- 172.21.15.42:0/3455550918 shutdown_connections 2024-05-28T14:09:05.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.857+0000 7f2cd4800640 1 -- 172.21.15.42:0/3455550918 wait complete. 2024-05-28T14:09:05.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.857+0000 7f2cd4800640 1 Processor -- start 2024-05-28T14:09:05.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd4800640 1 -- start start 2024-05-28T14:09:05.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd4800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ccc0cd960 0x7f2ccc0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd4800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ccc15fcc0 0x7f2ccc0cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd4800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ccc155cf0 0x7f2ccc156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd4800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc0b72a0 con 0x7f2ccc15fcc0 2024-05-28T14:09:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd4800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc0b7120 con 0x7f2ccc0cd960 2024-05-28T14:09:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd4800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2ccc168270 con 0x7f2ccc155cf0 2024-05-28T14:09:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ccc15fcc0 0x7f2ccc0cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ccc15fcc0 0x7f2ccc0cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48756/0 (socket says 172.21.15.42:48756) 2024-05-28T14:09:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd1400640 1 -- 172.21.15.42:0/405608867 learned_addr learned my addr 172.21.15.42:0/405608867 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:05.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd2800640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ccc155cf0 0x7f2ccc156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd1e00640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ccc0cd960 0x7f2ccc0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd1400640 1 -- 172.21.15.42:0/405608867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ccc155cf0 msgr2=0x7f2ccc156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd1400640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ccc155cf0 0x7f2ccc156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd1400640 1 -- 172.21.15.42:0/405608867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ccc0cd960 msgr2=0x7f2ccc0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:05.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.861+0000 7f2cd1400640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ccc0cd960 0x7f2ccc0cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:05.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.865+0000 7f2cd1400640 1 -- 172.21.15.42:0/405608867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc159140 con 0x7f2ccc15fcc0 2024-05-28T14:09:05.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.865+0000 7f2cd2800640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ccc155cf0 0x7f2ccc156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:05.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.865+0000 7f2cd1400640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ccc15fcc0 0x7f2ccc0cb9b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc013d30 tx=0x7f2cbc021970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:05.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.865+0000 7f2cc6000640 1 -- 172.21.15.42:0/405608867 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc0c4070 con 0x7f2ccc15fcc0 2024-05-28T14:09:05.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.865+0000 7f2cd4800640 1 -- 172.21.15.42:0/405608867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc157c90 con 0x7f2ccc15fcc0 2024-05-28T14:09:05.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.865+0000 7f2cd4800640 1 -- 172.21.15.42:0/405608867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc158250 con 0x7f2ccc15fcc0 2024-05-28T14:09:05.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.865+0000 7f2cc6000640 1 -- 172.21.15.42:0/405608867 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cbc0b5a90 con 0x7f2ccc15fcc0 2024-05-28T14:09:05.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.865+0000 7f2cc6000640 1 -- 172.21.15.42:0/405608867 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc0ba040 con 0x7f2ccc15fcc0 2024-05-28T14:09:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.865+0000 7f2cd4800640 1 -- 172.21.15.42:0/405608867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ccc068a90 con 0x7f2ccc15fcc0 2024-05-28T14:09:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.869+0000 7f2cc6000640 1 -- 172.21.15.42:0/405608867 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f2cbc0c7070 con 0x7f2ccc15fcc0 2024-05-28T14:09:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.869+0000 7f2cc6000640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2cb0069570 0x7f2cb006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.869+0000 7f2cc6000640 1 -- 172.21.15.42:0/405608867 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5851+0+0 (secure 0 0 0) 0x7f2cbc1276c0 con 0x7f2ccc15fcc0 2024-05-28T14:09:05.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.869+0000 7f2cd1e00640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2cb0069570 0x7f2cb006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:05.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.873+0000 7f2cd1e00640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2cb0069570 0x7f2cb006ba30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00011f0 tx=0x7f2cc0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:05.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:05.873+0000 7f2cc6000640 1 -- 172.21.15.42:0/405608867 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2cbc0ce080 con 0x7f2ccc15fcc0 2024-05-28T14:09:06.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.241+0000 7f2cd4800640 1 -- 172.21.15.42:0/405608867 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone81116", "target": ["mon-mgr", ""]}) -- 0x7f2ccc0d2290 con 0x7f2cb0069570 2024-05-28T14:09:06.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.273+0000 7f2cc6000640 1 -- 172.21.15.42:0/405608867 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2ccc0d2290 con 0x7f2cb0069570 2024-05-28T14:09:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.277+0000 7f2cd4800640 1 -- 172.21.15.42:0/405608867 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2cb0069570 msgr2=0x7f2cb006ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.277+0000 7f2cd4800640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2cb0069570 0x7f2cb006ba30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00011f0 tx=0x7f2cc0009040 comp rx=0 tx=0).stop 2024-05-28T14:09:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.277+0000 7f2cd4800640 1 -- 172.21.15.42:0/405608867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ccc15fcc0 msgr2=0x7f2ccc0cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.277+0000 7f2cd4800640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ccc15fcc0 0x7f2ccc0cb9b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc013d30 tx=0x7f2cbc021970 comp rx=0 tx=0).stop 2024-05-28T14:09:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.277+0000 7f2cd4800640 1 -- 172.21.15.42:0/405608867 shutdown_connections 2024-05-28T14:09:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.277+0000 7f2cd4800640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2cb0069570 0x7f2cb006ba30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.277+0000 7f2cd4800640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ccc155cf0 0x7f2ccc156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.277+0000 7f2cd4800640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ccc15fcc0 0x7f2ccc0cb9b0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.277+0000 7f2cd4800640 1 --2- 172.21.15.42:0/405608867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ccc0cd960 0x7f2ccc0cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:06.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.277+0000 7f2cd4800640 1 -- 172.21.15.42:0/405608867 >> 172.21.15.42:0/405608867 conn(0x7f2ccc0b3fa0 msgr2=0x7f2ccc0b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:06.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.281+0000 7f2cd4800640 1 -- 172.21.15.42:0/405608867 shutdown_connections 2024-05-28T14:09:06.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.281+0000 7f2cd4800640 1 -- 172.21.15.42:0/405608867 wait complete. 2024-05-28T14:09:06.302 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone81119 2024-05-28T14:09:06.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.549+0000 7ff234600640 1 Processor -- start 2024-05-28T14:09:06.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.549+0000 7ff234600640 1 -- start start 2024-05-28T14:09:06.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.549+0000 7ff234600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff22c0afa70 0x7ff22c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:06.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.549+0000 7ff234600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff22c1570a0 con 0x7ff22c0d06f0 2024-05-28T14:09:06.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.549+0000 7ff234600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff22c15cf00 con 0x7ff22c06ee80 2024-05-28T14:09:06.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.549+0000 7ff234600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff22c15d1e0 con 0x7ff22c0afa70 2024-05-28T14:09:06.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff231c00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7ff22c06ee80 0x7ff22c0af360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:38348/0 (socket says 172.21.15.42:38348) 2024-05-28T14:09:06.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff231c00640 1 -- 172.21.15.42:0/2123781378 learned_addr learned my addr 172.21.15.42:0/2123781378 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:06.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff231200640 1 --2- 172.21.15.42:0/2123781378 >> v2:172.21.15.42:3300/0 conn(0x7ff22c0afa70 0x7ff22c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:06.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 -- 172.21.15.42:0/2123781378 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3056471075 0 0) 0x7ff22c15cf00 con 0x7ff22c06ee80 2024-05-28T14:09:06.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 -- 172.21.15.42:0/2123781378 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff214003530 con 0x7ff22c06ee80 2024-05-28T14:09:06.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 -- 172.21.15.42:0/2123781378 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1056019623 0 0) 0x7ff214003530 con 0x7ff22c06ee80 2024-05-28T14:09:06.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 -- 172.21.15.42:0/2123781378 >> v2:172.21.15.42:3300/0 conn(0x7ff22c0afa70 msgr2=0x7ff22c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 --2- 172.21.15.42:0/2123781378 >> v2:172.21.15.42:3300/0 conn(0x7ff22c0afa70 0x7ff22c06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 -- 172.21.15.42:0/2123781378 >> v1:172.21.15.42:6789/0 conn(0x7ff22c0d06f0 legacy=0x7ff22c160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 -- 172.21.15.42:0/2123781378 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c15d4c0 con 0x7ff22c06ee80 2024-05-28T14:09:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 -- 172.21.15.42:0/2123781378 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7ff21c001680 con 0x7ff22c06ee80 2024-05-28T14:09:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 -- 172.21.15.42:0/2123781378 >> v1:172.21.15.63:6789/0 conn(0x7ff22c06ee80 legacy=0x7ff22c0af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 --2- 172.21.15.42:0/2123781378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2140039e0 0x7ff214003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 --2- 172.21.15.42:0/2123781378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff214004810 0x7ff214004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 --2- 172.21.15.42:0/2123781378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2140052f0 0x7ff21400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 -- 172.21.15.42:0/2123781378 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c15d1e0 con 0x7ff2140052f0 2024-05-28T14:09:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 -- 172.21.15.42:0/2123781378 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff21400c1f0 con 0x7ff2140039e0 2024-05-28T14:09:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff22be00640 1 -- 172.21.15.42:0/2123781378 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff21400c3f0 con 0x7ff214004810 2024-05-28T14:09:06.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff231200640 1 --2- 172.21.15.42:0/2123781378 >> v2:172.21.15.42:3300/0 conn(0x7ff22c0afa70 0x7ff22c06dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:06.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff231c00640 1 --2- 172.21.15.42:0/2123781378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2140039e0 0x7ff214003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:06.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff232600640 1 --2- 172.21.15.42:0/2123781378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2140052f0 0x7ff21400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:06.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff231c00640 1 -- 172.21.15.42:0/2123781378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff214004810 msgr2=0x7ff214004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:06.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff231200640 1 --2- 172.21.15.42:0/2123781378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff214004810 0x7ff214004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:06.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff231c00640 1 --2- 172.21.15.42:0/2123781378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff214004810 0x7ff214004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:06.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff231c00640 1 -- 172.21.15.42:0/2123781378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2140052f0 msgr2=0x7ff21400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:06.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff231c00640 1 --2- 172.21.15.42:0/2123781378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2140052f0 0x7ff21400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:06.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff231c00640 1 -- 172.21.15.42:0/2123781378 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c15cf00 con 0x7ff2140039e0 2024-05-28T14:09:06.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff232600640 1 -- 172.21.15.42:0/2123781378 reap_dead start 2024-05-28T14:09:06.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.553+0000 7ff231200640 1 --2- 172.21.15.42:0/2123781378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff214004810 0x7ff214004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:06.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.557+0000 7ff231c00640 1 --2- 172.21.15.42:0/2123781378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2140039e0 0x7ff214003e20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00c1c0 tx=0x7ff21c01f390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd560f3684acbcc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:06.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.561+0000 7ff22be00640 1 -- 172.21.15.42:0/2123781378 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c007cf0 con 0x7ff2140039e0 2024-05-28T14:09:06.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.561+0000 7ff22be00640 1 -- 172.21.15.42:0/2123781378 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff21c004ad0 con 0x7ff2140039e0 2024-05-28T14:09:06.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.561+0000 7ff22be00640 1 -- 172.21.15.42:0/2123781378 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c007110 con 0x7ff2140039e0 2024-05-28T14:09:06.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.561+0000 7ff234600640 1 -- 172.21.15.42:0/2123781378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2140039e0 msgr2=0x7ff214003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:06.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.561+0000 7ff234600640 1 --2- 172.21.15.42:0/2123781378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2140039e0 0x7ff214003e20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00c1c0 tx=0x7ff21c01f390 comp rx=0 tx=0).stop 2024-05-28T14:09:06.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.561+0000 7ff234600640 1 -- 172.21.15.42:0/2123781378 shutdown_connections 2024-05-28T14:09:06.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.561+0000 7ff234600640 1 --2- 172.21.15.42:0/2123781378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2140039e0 0x7ff214003e20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:06.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.561+0000 7ff234600640 1 -- 172.21.15.42:0/2123781378 >> 172.21.15.42:0/2123781378 conn(0x7ff22c06bdf0 msgr2=0x7ff22c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:06.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.561+0000 7ff234600640 1 -- 172.21.15.42:0/2123781378 shutdown_connections 2024-05-28T14:09:06.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.561+0000 7ff234600640 1 -- 172.21.15.42:0/2123781378 wait complete. 2024-05-28T14:09:06.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.561+0000 7ff234600640 1 Processor -- start 2024-05-28T14:09:06.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff234600640 1 -- start start 2024-05-28T14:09:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff234600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2140039e0 0x7ff22c099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff234600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff22c0ce110 0x7ff22c0ce570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff234600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff22c15f0e0 0x7ff22c15f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff234600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c15cb60 con 0x7ff22c0ce110 2024-05-28T14:09:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff234600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c15da80 con 0x7ff2140039e0 2024-05-28T14:09:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff234600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff22c15cce0 con 0x7ff22c15f0e0 2024-05-28T14:09:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff231200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff22c0ce110 0x7ff22c0ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff231c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2140039e0 0x7ff22c099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff231200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff22c0ce110 0x7ff22c0ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48776/0 (socket says 172.21.15.42:48776) 2024-05-28T14:09:06.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff231200640 1 -- 172.21.15.42:0/131902362 learned_addr learned my addr 172.21.15.42:0/131902362 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:06.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff232600640 1 --2- 172.21.15.42:0/131902362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff22c15f0e0 0x7ff22c15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:06.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff231200640 1 -- 172.21.15.42:0/131902362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff22c15f0e0 msgr2=0x7ff22c15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff231200640 1 --2- 172.21.15.42:0/131902362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff22c15f0e0 0x7ff22c15f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff231200640 1 -- 172.21.15.42:0/131902362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2140039e0 msgr2=0x7ff22c099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff231200640 1 --2- 172.21.15.42:0/131902362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2140039e0 0x7ff22c099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff231200640 1 -- 172.21.15.42:0/131902362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c0cd070 con 0x7ff22c0ce110 2024-05-28T14:09:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff231200640 1 --2- 172.21.15.42:0/131902362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff22c0ce110 0x7ff22c0ce570 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7ff22000f520 tx=0x7ff22000f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff229600640 1 -- 172.21.15.42:0/131902362 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220004230 con 0x7ff22c0ce110 2024-05-28T14:09:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff229600640 1 -- 172.21.15.42:0/131902362 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2200043d0 con 0x7ff22c0ce110 2024-05-28T14:09:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff234600640 1 -- 172.21.15.42:0/131902362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22c0cd350 con 0x7ff22c0ce110 2024-05-28T14:09:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff229600640 1 -- 172.21.15.42:0/131902362 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2200148b0 con 0x7ff22c0ce110 2024-05-28T14:09:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff234600640 1 -- 172.21.15.42:0/131902362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22c0cd910 con 0x7ff22c0ce110 2024-05-28T14:09:06.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.565+0000 7ff234600640 1 -- 172.21.15.42:0/131902362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff22c068a90 con 0x7ff22c0ce110 2024-05-28T14:09:06.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.569+0000 7ff229600640 1 -- 172.21.15.42:0/131902362 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7ff220012070 con 0x7ff22c0ce110 2024-05-28T14:09:06.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.569+0000 7ff229600640 1 --2- 172.21.15.42:0/131902362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff2100694f0 0x7ff21006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:06.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.569+0000 7ff229600640 1 -- 172.21.15.42:0/131902362 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5851+0+0 (secure 0 0 0) 0x7ff2200854c0 con 0x7ff22c0ce110 2024-05-28T14:09:06.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.569+0000 7ff231c00640 1 --2- 172.21.15.42:0/131902362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff2100694f0 0x7ff21006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:06.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.569+0000 7ff231c00640 1 --2- 172.21.15.42:0/131902362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff2100694f0 0x7ff21006b9b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0ad000 tx=0x7ff21c007040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:06.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.573+0000 7ff229600640 1 -- 172.21.15.42:0/131902362 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff220018020 con 0x7ff22c0ce110 2024-05-28T14:09:06.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.765+0000 7ff234600640 1 -- 172.21.15.42:0/131902362 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone81119", "target": ["mon-mgr", ""]}) -- 0x7ff22c0b5160 con 0x7ff2100694f0 2024-05-28T14:09:06.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.781+0000 7ff229600640 1 -- 172.21.15.42:0/131902362 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff22c0b5160 con 0x7ff2100694f0 2024-05-28T14:09:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.781+0000 7ff234600640 1 -- 172.21.15.42:0/131902362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff2100694f0 msgr2=0x7ff21006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.781+0000 7ff234600640 1 --2- 172.21.15.42:0/131902362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff2100694f0 0x7ff21006b9b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0ad000 tx=0x7ff21c007040 comp rx=0 tx=0).stop 2024-05-28T14:09:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.781+0000 7ff234600640 1 -- 172.21.15.42:0/131902362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff22c0ce110 msgr2=0x7ff22c0ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.781+0000 7ff234600640 1 --2- 172.21.15.42:0/131902362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff22c0ce110 0x7ff22c0ce570 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7ff22000f520 tx=0x7ff22000f920 comp rx=0 tx=0).stop 2024-05-28T14:09:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.781+0000 7ff234600640 1 -- 172.21.15.42:0/131902362 shutdown_connections 2024-05-28T14:09:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.781+0000 7ff234600640 1 --2- 172.21.15.42:0/131902362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff2100694f0 0x7ff21006b9b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.781+0000 7ff234600640 1 --2- 172.21.15.42:0/131902362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff22c15f0e0 0x7ff22c15f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.781+0000 7ff234600640 1 --2- 172.21.15.42:0/131902362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff22c0ce110 0x7ff22c0ce570 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.781+0000 7ff234600640 1 --2- 172.21.15.42:0/131902362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2140039e0 0x7ff22c099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.781+0000 7ff234600640 1 -- 172.21.15.42:0/131902362 >> 172.21.15.42:0/131902362 conn(0x7ff22c06bdf0 msgr2=0x7ff22c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.781+0000 7ff234600640 1 -- 172.21.15.42:0/131902362 shutdown_connections 2024-05-28T14:09:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:06.785+0000 7ff234600640 1 -- 172.21.15.42:0/131902362 wait complete. 2024-05-28T14:09:06.795 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone81110 2024-05-28T14:09:07.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.041+0000 7f88d4e00640 1 Processor -- start 2024-05-28T14:09:07.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.041+0000 7f88d4e00640 1 -- start start 2024-05-28T14:09:07.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.045+0000 7f88d4e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88d015d1d0 0x7f88d006e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:07.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.045+0000 7f88d4e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f88d00d0e00 0x7f88d0071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:07.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.045+0000 7f88d4e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88d015d5b0 con 0x7f88d006de20 2024-05-28T14:09:07.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.045+0000 7f88d4e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f88d0167360 con 0x7f88d00d0e00 2024-05-28T14:09:07.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.045+0000 7f88d4e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f88d0166750 con 0x7f88d015d1d0 2024-05-28T14:09:07.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.045+0000 7f88cde00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88d015d1d0 0x7f88d006e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:07.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.045+0000 7f88cde00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88d015d1d0 0x7f88d006e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:34892/0 (socket says 172.21.15.42:34892) 2024-05-28T14:09:07.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.045+0000 7f88cde00640 1 -- 172.21.15.42:0/2365154407 learned_addr learned my addr 172.21.15.42:0/2365154407 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:07.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.045+0000 7f88ce800640 1 --2- 172.21.15.42:0/2365154407 >> v2:172.21.15.63:3300/0 conn(0x7f88d00d0e00 0x7f88d0071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:07.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.045+0000 7f88cde00640 1 -- 172.21.15.42:0/2365154407 >> v2:172.21.15.63:3300/0 conn(0x7f88d00d0e00 msgr2=0x7f88d0071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:07.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.045+0000 7f88cde00640 1 --2- 172.21.15.42:0/2365154407 >> v2:172.21.15.63:3300/0 conn(0x7f88d00d0e00 0x7f88d0071f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.045+0000 7f88cde00640 1 -- 172.21.15.42:0/2365154407 >> v1:172.21.15.42:6789/0 conn(0x7f88d006de20 legacy=0x7f88d00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:07.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.045+0000 7f88cde00640 1 -- 172.21.15.42:0/2365154407 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d0166a30 con 0x7f88d015d1d0 2024-05-28T14:09:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.049+0000 7f88cde00640 1 --2- 172.21.15.42:0/2365154407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88d015d1d0 0x7f88d006e2c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009870 tx=0x7f88bc01c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ca3374a2c7b3531 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:07.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.053+0000 7f88cca00640 1 -- 172.21.15.42:0/2365154407 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0aa610 con 0x7f88d015d1d0 2024-05-28T14:09:07.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.053+0000 7f88cca00640 1 -- 172.21.15.42:0/2365154407 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88bc0af070 con 0x7f88d015d1d0 2024-05-28T14:09:07.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.053+0000 7f88cca00640 1 -- 172.21.15.42:0/2365154407 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0aa9d0 con 0x7f88d015d1d0 2024-05-28T14:09:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.053+0000 7f88d4e00640 1 -- 172.21.15.42:0/2365154407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88d015d1d0 msgr2=0x7f88d006e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.053+0000 7f88d4e00640 1 --2- 172.21.15.42:0/2365154407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88d015d1d0 0x7f88d006e2c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009870 tx=0x7f88bc01c4f0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.053+0000 7f88d4e00640 1 -- 172.21.15.42:0/2365154407 shutdown_connections 2024-05-28T14:09:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.053+0000 7f88d4e00640 1 --2- 172.21.15.42:0/2365154407 >> v2:172.21.15.63:3300/0 conn(0x7f88d00d0e00 0x7f88d0071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.053+0000 7f88d4e00640 1 --2- 172.21.15.42:0/2365154407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88d015d1d0 0x7f88d006e2c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.053+0000 7f88d4e00640 1 -- 172.21.15.42:0/2365154407 >> 172.21.15.42:0/2365154407 conn(0x7f88d006bdf0 msgr2=0x7f88d006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.053+0000 7f88d4e00640 1 -- 172.21.15.42:0/2365154407 shutdown_connections 2024-05-28T14:09:07.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.053+0000 7f88d4e00640 1 -- 172.21.15.42:0/2365154407 wait complete. 2024-05-28T14:09:07.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.057+0000 7f88d4e00640 1 Processor -- start 2024-05-28T14:09:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.057+0000 7f88d4e00640 1 -- start start 2024-05-28T14:09:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.057+0000 7f88d4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88d006de20 0x7f88d009dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.057+0000 7f88d4e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88d00d0e00 0x7f88d0155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.057+0000 7f88d4e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88d015d1d0 0x7f88d015b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.057+0000 7f88d4e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0073010 con 0x7f88d006de20 2024-05-28T14:09:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.057+0000 7f88d4e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0072e90 con 0x7f88d015d1d0 2024-05-28T14:09:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.057+0000 7f88d4e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f88d0073190 con 0x7f88d00d0e00 2024-05-28T14:09:07.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.057+0000 7f88cde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88d006de20 0x7f88d009dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:07.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.057+0000 7f88ce800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88d015d1d0 0x7f88d015b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:07.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.057+0000 7f88ce800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88d015d1d0 0x7f88d015b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:46934/0 (socket says 172.21.15.42:46934) 2024-05-28T14:09:07.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.057+0000 7f88ce800640 1 -- 172.21.15.42:0/3548829690 learned_addr learned my addr 172.21.15.42:0/3548829690 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:07.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.057+0000 7f88cd400640 1 --2- 172.21.15.42:0/3548829690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88d00d0e00 0x7f88d0155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:07.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.061+0000 7f88cde00640 1 -- 172.21.15.42:0/3548829690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88d00d0e00 msgr2=0x7f88d0155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:07.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.061+0000 7f88cde00640 1 --2- 172.21.15.42:0/3548829690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88d00d0e00 0x7f88d0155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.061+0000 7f88cde00640 1 -- 172.21.15.42:0/3548829690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88d015d1d0 msgr2=0x7f88d015b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:07.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.061+0000 7f88cde00640 1 --2- 172.21.15.42:0/3548829690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88d015d1d0 0x7f88d015b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.061+0000 7f88cde00640 1 -- 172.21.15.42:0/3548829690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d0159fc0 con 0x7f88d006de20 2024-05-28T14:09:07.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.061+0000 7f88cde00640 1 --2- 172.21.15.42:0/3548829690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88d006de20 0x7f88d009dcd0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0ab000 tx=0x7f88bc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:07.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.061+0000 7f88c6000640 1 -- 172.21.15.42:0/3548829690 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0bd5f0 con 0x7f88d006de20 2024-05-28T14:09:07.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.061+0000 7f88c6000640 1 -- 172.21.15.42:0/3548829690 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88bc0bd790 con 0x7f88d006de20 2024-05-28T14:09:07.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.061+0000 7f88c6000640 1 -- 172.21.15.42:0/3548829690 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0bdb60 con 0x7f88d006de20 2024-05-28T14:09:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.061+0000 7f88d4e00640 1 -- 172.21.15.42:0/3548829690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d015a2a0 con 0x7f88d006de20 2024-05-28T14:09:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.061+0000 7f88d4e00640 1 -- 172.21.15.42:0/3548829690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d015a860 con 0x7f88d006de20 2024-05-28T14:09:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.065+0000 7f88d4e00640 1 -- 172.21.15.42:0/3548829690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d0068a90 con 0x7f88d006de20 2024-05-28T14:09:07.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.065+0000 7f88c6000640 1 -- 172.21.15.42:0/3548829690 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f88bc0b33f0 con 0x7f88d006de20 2024-05-28T14:09:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.069+0000 7f88c6000640 1 --2- 172.21.15.42:0/3548829690 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f88b00695c0 0x7f88b006ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.069+0000 7f88c6000640 1 -- 172.21.15.42:0/3548829690 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5851+0+0 (secure 0 0 0) 0x7f88bc120240 con 0x7f88d006de20 2024-05-28T14:09:07.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.069+0000 7f88cd400640 1 --2- 172.21.15.42:0/3548829690 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f88b00695c0 0x7f88b006ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:07.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.069+0000 7f88cd400640 1 --2- 172.21.15.42:0/3548829690 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f88b00695c0 0x7f88b006ba80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f88d01ff040 tx=0x7f88c0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:07.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.077+0000 7f88c6000640 1 -- 172.21.15.42:0/3548829690 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f88bc0a7070 con 0x7f88d006de20 2024-05-28T14:09:07.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.437+0000 7f88d4e00640 1 -- 172.21.15.42:0/3548829690 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone81110", "target": ["mon-mgr", ""]}) -- 0x7f88d0068a90 con 0x7f88b00695c0 2024-05-28T14:09:07.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.465+0000 7f88c6000640 1 -- 172.21.15.42:0/3548829690 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f88d0068a90 con 0x7f88b00695c0 2024-05-28T14:09:07.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.469+0000 7f88d4e00640 1 -- 172.21.15.42:0/3548829690 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f88b00695c0 msgr2=0x7f88b006ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:07.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.469+0000 7f88d4e00640 1 --2- 172.21.15.42:0/3548829690 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f88b00695c0 0x7f88b006ba80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f88d01ff040 tx=0x7f88c0009040 comp rx=0 tx=0).stop 2024-05-28T14:09:07.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.469+0000 7f88d4e00640 1 -- 172.21.15.42:0/3548829690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88d006de20 msgr2=0x7f88d009dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:07.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.469+0000 7f88d4e00640 1 --2- 172.21.15.42:0/3548829690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88d006de20 0x7f88d009dcd0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0ab000 tx=0x7f88bc01ad30 comp rx=0 tx=0).stop 2024-05-28T14:09:07.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.469+0000 7f88d4e00640 1 -- 172.21.15.42:0/3548829690 shutdown_connections 2024-05-28T14:09:07.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.469+0000 7f88d4e00640 1 --2- 172.21.15.42:0/3548829690 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f88b00695c0 0x7f88b006ba80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.469+0000 7f88d4e00640 1 --2- 172.21.15.42:0/3548829690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88d015d1d0 0x7f88d015b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.469+0000 7f88d4e00640 1 --2- 172.21.15.42:0/3548829690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88d00d0e00 0x7f88d0155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.469+0000 7f88d4e00640 1 --2- 172.21.15.42:0/3548829690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88d006de20 0x7f88d009dcd0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.469+0000 7f88d4e00640 1 -- 172.21.15.42:0/3548829690 >> 172.21.15.42:0/3548829690 conn(0x7f88d006bdf0 msgr2=0x7f88d01642f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:07.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.473+0000 7f88d4e00640 1 -- 172.21.15.42:0/3548829690 shutdown_connections 2024-05-28T14:09:07.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.473+0000 7f88d4e00640 1 -- 172.21.15.42:0/3548829690 wait complete. 2024-05-28T14:09:07.493 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone8119 2024-05-28T14:09:07.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21d1c00640 1 Processor -- start 2024-05-28T14:09:07.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21d1c00640 1 -- start start 2024-05-28T14:09:07.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21d1c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f21cc0b01c0 0x7f21cc0730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:07.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21d1c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f21cc073610 0x7f21cc06ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:07.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21d1c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21cc15ece0 con 0x7f21cc06cf40 2024-05-28T14:09:07.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21d1c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f21cc06d3b0 con 0x7f21cc073610 2024-05-28T14:09:07.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21d1c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f21cc15ef60 con 0x7f21cc0b01c0 2024-05-28T14:09:07.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21c3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f21cc073610 0x7f21cc06ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:07.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21c3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f21cc073610 0x7f21cc06ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43436/0 (socket says 172.21.15.42:43436) 2024-05-28T14:09:07.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21c3e00640 1 -- 172.21.15.42:0/1433867196 learned_addr learned my addr 172.21.15.42:0/1433867196 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:07.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21caa00640 1 --2- 172.21.15.42:0/1433867196 >> v2:172.21.15.63:3300/0 conn(0x7f21cc0b01c0 0x7f21cc0730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:07.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21c3e00640 1 -- 172.21.15.42:0/1433867196 >> v2:172.21.15.63:3300/0 conn(0x7f21cc0b01c0 msgr2=0x7f21cc0730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:07.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21c3e00640 1 --2- 172.21.15.42:0/1433867196 >> v2:172.21.15.63:3300/0 conn(0x7f21cc0b01c0 0x7f21cc0730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21c3e00640 1 -- 172.21.15.42:0/1433867196 >> v1:172.21.15.63:6789/0 conn(0x7f21cc06cf40 legacy=0x7f21cc154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:07.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.741+0000 7f21c3e00640 1 -- 172.21.15.42:0/1433867196 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc15f240 con 0x7f21cc073610 2024-05-28T14:09:07.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.745+0000 7f21c3e00640 1 --2- 172.21.15.42:0/1433867196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21cc073610 0x7f21cc06ca00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f21cc073e60 tx=0x7f21b401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d736d642c581a59d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:07.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.745+0000 7f21ca000640 1 -- 172.21.15.42:0/1433867196 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b40a7070 con 0x7f21cc073610 2024-05-28T14:09:07.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.745+0000 7f21ca000640 1 -- 172.21.15.42:0/1433867196 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b40b0070 con 0x7f21cc073610 2024-05-28T14:09:07.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.745+0000 7f21ca000640 1 -- 172.21.15.42:0/1433867196 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b40ab890 con 0x7f21cc073610 2024-05-28T14:09:07.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.745+0000 7f21d1c00640 1 -- 172.21.15.42:0/1433867196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21cc073610 msgr2=0x7f21cc06ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:07.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.745+0000 7f21d1c00640 1 --2- 172.21.15.42:0/1433867196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21cc073610 0x7f21cc06ca00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f21cc073e60 tx=0x7f21b401c4c0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.745+0000 7f21d1c00640 1 -- 172.21.15.42:0/1433867196 shutdown_connections 2024-05-28T14:09:07.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.745+0000 7f21d1c00640 1 --2- 172.21.15.42:0/1433867196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21cc073610 0x7f21cc06ca00 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.745+0000 7f21d1c00640 1 --2- 172.21.15.42:0/1433867196 >> v2:172.21.15.63:3300/0 conn(0x7f21cc0b01c0 0x7f21cc0730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.745+0000 7f21d1c00640 1 -- 172.21.15.42:0/1433867196 >> 172.21.15.42:0/1433867196 conn(0x7f21cc0725b0 msgr2=0x7f21cc0729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:07.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.745+0000 7f21d1c00640 1 -- 172.21.15.42:0/1433867196 shutdown_connections 2024-05-28T14:09:07.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.745+0000 7f21d1c00640 1 -- 172.21.15.42:0/1433867196 wait complete. 2024-05-28T14:09:07.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21d1c00640 1 Processor -- start 2024-05-28T14:09:07.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21d1c00640 1 -- start start 2024-05-28T14:09:07.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21d1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21cc06cf40 0x7f21cc149d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:07.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21d1c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21cc073610 0x7f21cc158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:07.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21d1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21cc0b01c0 0x7f21cc158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:07.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21d1c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc0b05a0 con 0x7f21cc073610 2024-05-28T14:09:07.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21d1c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc0b4e00 con 0x7f21cc0b01c0 2024-05-28T14:09:07.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21d1c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f21cc06e850 con 0x7f21cc06cf40 2024-05-28T14:09:07.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21c3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21cc073610 0x7f21cc158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:07.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21c3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21cc073610 0x7f21cc158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43450/0 (socket says 172.21.15.42:43450) 2024-05-28T14:09:07.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21c3e00640 1 -- 172.21.15.42:0/2233370858 learned_addr learned my addr 172.21.15.42:0/2233370858 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:07.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21caa00640 1 --2- 172.21.15.42:0/2233370858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21cc06cf40 0x7f21cc149d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:07.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21cb400640 1 --2- 172.21.15.42:0/2233370858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21cc0b01c0 0x7f21cc158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:07.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21c3e00640 1 -- 172.21.15.42:0/2233370858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21cc06cf40 msgr2=0x7f21cc149d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:07.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21c3e00640 1 --2- 172.21.15.42:0/2233370858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21cc06cf40 0x7f21cc149d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21c3e00640 1 -- 172.21.15.42:0/2233370858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21cc0b01c0 msgr2=0x7f21cc158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:07.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21c3e00640 1 --2- 172.21.15.42:0/2233370858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21cc0b01c0 0x7f21cc158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:07.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21c3e00640 1 -- 172.21.15.42:0/2233370858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc06cd10 con 0x7f21cc073610 2024-05-28T14:09:07.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21c3e00640 1 --2- 172.21.15.42:0/2233370858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21cc073610 0x7f21cc158400 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f21b40190d0 tx=0x7f21b401ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:07.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21c2a00640 1 -- 172.21.15.42:0/2233370858 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b40be5f0 con 0x7f21cc073610 2024-05-28T14:09:07.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21c2a00640 1 -- 172.21.15.42:0/2233370858 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b40be790 con 0x7f21cc073610 2024-05-28T14:09:07.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21c2a00640 1 -- 172.21.15.42:0/2233370858 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b40beb60 con 0x7f21cc073610 2024-05-28T14:09:07.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21d1c00640 1 -- 172.21.15.42:0/2233370858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21cc0ce320 con 0x7f21cc073610 2024-05-28T14:09:07.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21d1c00640 1 -- 172.21.15.42:0/2233370858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21cc0ce8e0 con 0x7f21cc073610 2024-05-28T14:09:07.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.749+0000 7f21d1c00640 1 -- 172.21.15.42:0/2233370858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21cc068a90 con 0x7f21cc073610 2024-05-28T14:09:07.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.753+0000 7f21c2a00640 1 -- 172.21.15.42:0/2233370858 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f21b40b43f0 con 0x7f21cc073610 2024-05-28T14:09:07.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.753+0000 7f21c2a00640 1 --2- 172.21.15.42:0/2233370858 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21a80694f0 0x7f21a806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:07.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.753+0000 7f21c2a00640 1 -- 172.21.15.42:0/2233370858 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7f21b411fae0 con 0x7f21cc073610 2024-05-28T14:09:07.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.753+0000 7f21caa00640 1 --2- 172.21.15.42:0/2233370858 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21a80694f0 0x7f21a806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:07.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.753+0000 7f21caa00640 1 --2- 172.21.15.42:0/2233370858 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21a80694f0 0x7f21a806b9b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009ac0 tx=0x7f21bc009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:07.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:07.757+0000 7f21c2a00640 1 -- 172.21.15.42:0/2233370858 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f21b4016110 con 0x7f21cc073610 2024-05-28T14:09:08.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.057+0000 7f21d1c00640 1 -- 172.21.15.42:0/2233370858 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone8119", "target": ["mon-mgr", ""]}) -- 0x7f21cc067910 con 0x7f21a80694f0 2024-05-28T14:09:08.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.093+0000 7f21c2a00640 1 -- 172.21.15.42:0/2233370858 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f21cc067910 con 0x7f21a80694f0 2024-05-28T14:09:08.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.097+0000 7f21d1c00640 1 -- 172.21.15.42:0/2233370858 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21a80694f0 msgr2=0x7f21a806b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:08.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.097+0000 7f21d1c00640 1 --2- 172.21.15.42:0/2233370858 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21a80694f0 0x7f21a806b9b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009ac0 tx=0x7f21bc009190 comp rx=0 tx=0).stop 2024-05-28T14:09:08.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.097+0000 7f21d1c00640 1 -- 172.21.15.42:0/2233370858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21cc073610 msgr2=0x7f21cc158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:08.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.097+0000 7f21d1c00640 1 --2- 172.21.15.42:0/2233370858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21cc073610 0x7f21cc158400 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f21b40190d0 tx=0x7f21b401ad00 comp rx=0 tx=0).stop 2024-05-28T14:09:08.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.097+0000 7f21d1c00640 1 -- 172.21.15.42:0/2233370858 shutdown_connections 2024-05-28T14:09:08.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.097+0000 7f21d1c00640 1 --2- 172.21.15.42:0/2233370858 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21a80694f0 0x7f21a806b9b0 secure :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009ac0 tx=0x7f21bc009190 comp rx=0 tx=0).stop 2024-05-28T14:09:08.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.097+0000 7f21d1c00640 1 --2- 172.21.15.42:0/2233370858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21cc0b01c0 0x7f21cc158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.097+0000 7f21d1c00640 1 --2- 172.21.15.42:0/2233370858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21cc073610 0x7f21cc158400 secure :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f21b40190d0 tx=0x7f21b401ad00 comp rx=0 tx=0).stop 2024-05-28T14:09:08.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.097+0000 7f21d1c00640 1 --2- 172.21.15.42:0/2233370858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21cc06cf40 0x7f21cc149d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.097+0000 7f21d1c00640 1 -- 172.21.15.42:0/2233370858 >> 172.21.15.42:0/2233370858 conn(0x7f21cc0725b0 msgr2=0x7f21cc0b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:08.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.097+0000 7f21d1c00640 1 -- 172.21.15.42:0/2233370858 shutdown_connections 2024-05-28T14:09:08.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.097+0000 7f21d1c00640 1 -- 172.21.15.42:0/2233370858 wait complete. 2024-05-28T14:09:08.123 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone8114 2024-05-28T14:09:08.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.365+0000 7f2be3a00640 1 Processor -- start 2024-05-28T14:09:08.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.369+0000 7f2be3a00640 1 -- start start 2024-05-28T14:09:08.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.369+0000 7f2be3a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2bdc06de20 0x7f2bdc0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:08.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.369+0000 7f2be3a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bdc15d5b0 con 0x7f2bdc15d1d0 2024-05-28T14:09:08.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.369+0000 7f2be3a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bdc166750 con 0x7f2bdc0d0c30 2024-05-28T14:09:08.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.369+0000 7f2be3a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2bdc0d1080 con 0x7f2bdc06de20 2024-05-28T14:09:08.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.369+0000 7f2bdbe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2bdc06de20 0x7f2bdc0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:08.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.369+0000 7f2bdbe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2bdc06de20 0x7f2bdc0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:54200/0 (socket says 172.21.15.42:54200) 2024-05-28T14:09:08.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.369+0000 7f2bdbe00640 1 -- 172.21.15.42:0/3760587464 learned_addr learned my addr 172.21.15.42:0/3760587464 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:08.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.373+0000 7f2bdbe00640 1 -- 172.21.15.42:0/3760587464 >> v1:172.21.15.63:6789/0 conn(0x7f2bdc0d0c30 legacy=0x7f2bdc071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:08.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.373+0000 7f2bdbe00640 1 -- 172.21.15.42:0/3760587464 >> v1:172.21.15.42:6789/0 conn(0x7f2bdc15d1d0 legacy=0x7f2bdc06e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:08.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.373+0000 7f2bdbe00640 1 -- 172.21.15.42:0/3760587464 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc166a30 con 0x7f2bdc06de20 2024-05-28T14:09:08.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.377+0000 7f2bdbe00640 1 --2- 172.21.15.42:0/3760587464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bdc06de20 0x7f2bdc0d06f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009870 tx=0x7f2bc801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=445dbab1b9a21888 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:08.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.377+0000 7f2bdb400640 1 -- 172.21.15.42:0/3760587464 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80ab650 con 0x7f2bdc06de20 2024-05-28T14:09:08.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.377+0000 7f2bdb400640 1 -- 172.21.15.42:0/3760587464 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc80b0070 con 0x7f2bdc06de20 2024-05-28T14:09:08.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.377+0000 7f2bdb400640 1 -- 172.21.15.42:0/3760587464 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80aba10 con 0x7f2bdc06de20 2024-05-28T14:09:08.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.381+0000 7f2be3a00640 1 -- 172.21.15.42:0/3760587464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bdc06de20 msgr2=0x7f2bdc0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:08.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.381+0000 7f2be3a00640 1 --2- 172.21.15.42:0/3760587464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bdc06de20 0x7f2bdc0d06f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009870 tx=0x7f2bc801c560 comp rx=0 tx=0).stop 2024-05-28T14:09:08.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.381+0000 7f2be3a00640 1 -- 172.21.15.42:0/3760587464 shutdown_connections 2024-05-28T14:09:08.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.381+0000 7f2be3a00640 1 --2- 172.21.15.42:0/3760587464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bdc06de20 0x7f2bdc0d06f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.381+0000 7f2be3a00640 1 -- 172.21.15.42:0/3760587464 >> 172.21.15.42:0/3760587464 conn(0x7f2bdc06bdf0 msgr2=0x7f2bdc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:08.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.381+0000 7f2be3a00640 1 -- 172.21.15.42:0/3760587464 shutdown_connections 2024-05-28T14:09:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.381+0000 7f2be3a00640 1 -- 172.21.15.42:0/3760587464 wait complete. 2024-05-28T14:09:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.385+0000 7f2be3a00640 1 Processor -- start 2024-05-28T14:09:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.385+0000 7f2be3a00640 1 -- start start 2024-05-28T14:09:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2be3a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bdc06de20 0x7f2bdc09ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2be3a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bdc0d0c30 0x7f2bdc157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2be3a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bdc15d1d0 0x7f2bdc158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2be3a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc072fd0 con 0x7f2bdc06de20 2024-05-28T14:09:08.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2be3a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc072e50 con 0x7f2bdc15d1d0 2024-05-28T14:09:08.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2be3a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2bdc073150 con 0x7f2bdc0d0c30 2024-05-28T14:09:08.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2bdbe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bdc0d0c30 0x7f2bdc157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:08.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2bdbe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bdc0d0c30 0x7f2bdc157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:55560/0 (socket says 172.21.15.42:55560) 2024-05-28T14:09:08.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2bdbe00640 1 -- 172.21.15.42:0/3913294383 learned_addr learned my addr 172.21.15.42:0/3913294383 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:08.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2be1a00640 1 --2- 172.21.15.42:0/3913294383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bdc15d1d0 0x7f2bdc158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:08.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2bdbe00640 1 -- 172.21.15.42:0/3913294383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bdc15d1d0 msgr2=0x7f2bdc158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:08.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2bdbe00640 1 --2- 172.21.15.42:0/3913294383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bdc15d1d0 0x7f2bdc158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2bdbe00640 1 -- 172.21.15.42:0/3913294383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bdc06de20 msgr2=0x7f2bdc09ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:08.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2bdbe00640 1 --2- 172.21.15.42:0/3913294383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bdc06de20 0x7f2bdc09ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2bdbe00640 1 -- 172.21.15.42:0/3913294383 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc09c560 con 0x7f2bdc0d0c30 2024-05-28T14:09:08.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2bdbe00640 1 --2- 172.21.15.42:0/3913294383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bdc0d0c30 0x7f2bdc157d50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80ad040 tx=0x7f2bc801ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:08.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2bd8c00640 1 -- 172.21.15.42:0/3913294383 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80abe60 con 0x7f2bdc0d0c30 2024-05-28T14:09:08.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2bd8c00640 1 -- 172.21.15.42:0/3913294383 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc80b4820 con 0x7f2bdc0d0c30 2024-05-28T14:09:08.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.389+0000 7f2bd8c00640 1 -- 172.21.15.42:0/3913294383 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80b4c10 con 0x7f2bdc0d0c30 2024-05-28T14:09:08.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.393+0000 7f2be3a00640 1 -- 172.21.15.42:0/3913294383 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc158b20 con 0x7f2bdc0d0c30 2024-05-28T14:09:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.393+0000 7f2be3a00640 1 -- 172.21.15.42:0/3913294383 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc158560 con 0x7f2bdc0d0c30 2024-05-28T14:09:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.393+0000 7f2be3a00640 1 -- 172.21.15.42:0/3913294383 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bdc068a90 con 0x7f2bdc0d0c30 2024-05-28T14:09:08.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.397+0000 7f2bd8c00640 1 -- 172.21.15.42:0/3913294383 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f2bc80b3380 con 0x7f2bdc0d0c30 2024-05-28T14:09:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.397+0000 7f2bd8c00640 1 --2- 172.21.15.42:0/3913294383 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2bbc0695c0 0x7f2bbc06ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:08.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.397+0000 7f2bd8c00640 1 -- 172.21.15.42:0/3913294383 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7f2bc811fd80 con 0x7f2bdc0d0c30 2024-05-28T14:09:08.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.397+0000 7f2be1000640 1 --2- 172.21.15.42:0/3913294383 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2bbc0695c0 0x7f2bbc06ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:08.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.401+0000 7f2be1000640 1 --2- 172.21.15.42:0/3913294383 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2bbc0695c0 0x7f2bbc06ba80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0000f70 tx=0x7f2bd00079c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:08.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.401+0000 7f2bd8c00640 1 -- 172.21.15.42:0/3913294383 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2bdc09c560 con 0x7f2bdc0d0c30 2024-05-28T14:09:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.737+0000 7f2be3a00640 1 -- 172.21.15.42:0/3913294383 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone8114", "target": ["mon-mgr", ""]}) -- 0x7f2bdc0af4a0 con 0x7f2bbc0695c0 2024-05-28T14:09:08.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.749+0000 7f2bd8c00640 1 -- 172.21.15.42:0/3913294383 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2bdc0af4a0 con 0x7f2bbc0695c0 2024-05-28T14:09:08.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.753+0000 7f2be3a00640 1 -- 172.21.15.42:0/3913294383 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2bbc0695c0 msgr2=0x7f2bbc06ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:08.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.753+0000 7f2be3a00640 1 --2- 172.21.15.42:0/3913294383 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2bbc0695c0 0x7f2bbc06ba80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0000f70 tx=0x7f2bd00079c0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.753+0000 7f2be3a00640 1 -- 172.21.15.42:0/3913294383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bdc0d0c30 msgr2=0x7f2bdc157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:08.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.753+0000 7f2be3a00640 1 --2- 172.21.15.42:0/3913294383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bdc0d0c30 0x7f2bdc157d50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80ad040 tx=0x7f2bc801ada0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.753+0000 7f2be3a00640 1 -- 172.21.15.42:0/3913294383 shutdown_connections 2024-05-28T14:09:08.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.753+0000 7f2be3a00640 1 --2- 172.21.15.42:0/3913294383 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2bbc0695c0 0x7f2bbc06ba80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.753+0000 7f2be3a00640 1 --2- 172.21.15.42:0/3913294383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bdc15d1d0 0x7f2bdc158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.753+0000 7f2be3a00640 1 --2- 172.21.15.42:0/3913294383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bdc0d0c30 0x7f2bdc157d50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.753+0000 7f2be3a00640 1 --2- 172.21.15.42:0/3913294383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bdc06de20 0x7f2bdc09ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.753+0000 7f2be3a00640 1 -- 172.21.15.42:0/3913294383 >> 172.21.15.42:0/3913294383 conn(0x7f2bdc06bdf0 msgr2=0x7f2bdc165870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:08.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.753+0000 7f2be3a00640 1 -- 172.21.15.42:0/3913294383 shutdown_connections 2024-05-28T14:09:08.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.753+0000 7f2be3a00640 1 -- 172.21.15.42:0/3913294383 wait complete. 2024-05-28T14:09:08.760 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone8113 2024-05-28T14:09:08.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.973+0000 7f9593800640 1 Processor -- start 2024-05-28T14:09:08.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.973+0000 7f9593800640 1 -- start start 2024-05-28T14:09:08.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.977+0000 7f9593800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f958c15d1d0 0x7f958c06e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:08.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.977+0000 7f9593800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f958c0d0e00 0x7f958c071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:08.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.977+0000 7f9593800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f958c15d5b0 con 0x7f958c06de20 2024-05-28T14:09:08.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.977+0000 7f9593800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f958c167360 con 0x7f958c15d1d0 2024-05-28T14:09:08.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.977+0000 7f9593800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f958c166750 con 0x7f958c0d0e00 2024-05-28T14:09:08.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.977+0000 7f9591800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f958c0d0e00 0x7f958c071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:08.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.977+0000 7f9591800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f958c0d0e00 0x7f958c071f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:55564/0 (socket says 172.21.15.42:55564) 2024-05-28T14:09:08.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.977+0000 7f9591800640 1 -- 172.21.15.42:0/4262475782 learned_addr learned my addr 172.21.15.42:0/4262475782 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:08.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.977+0000 7f9590e00640 1 --2- 172.21.15.42:0/4262475782 >> v2:172.21.15.63:3300/0 conn(0x7f958c15d1d0 0x7f958c06e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:08.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.977+0000 7f9591800640 1 -- 172.21.15.42:0/4262475782 >> v2:172.21.15.63:3300/0 conn(0x7f958c15d1d0 msgr2=0x7f958c06e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:08.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.977+0000 7f9591800640 1 --2- 172.21.15.42:0/4262475782 >> v2:172.21.15.63:3300/0 conn(0x7f958c15d1d0 0x7f958c06e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.977+0000 7f9591800640 1 -- 172.21.15.42:0/4262475782 >> v1:172.21.15.63:6789/0 conn(0x7f958c06de20 legacy=0x7f958c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:08.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.977+0000 7f9591800640 1 -- 172.21.15.42:0/4262475782 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c166a30 con 0x7f958c0d0e00 2024-05-28T14:09:08.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.977+0000 7f9590e00640 1 --2- 172.21.15.42:0/4262475782 >> v2:172.21.15.63:3300/0 conn(0x7f958c15d1d0 0x7f958c06e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:08.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.981+0000 7f9591800640 1 --2- 172.21.15.42:0/4262475782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f958c0d0e00 0x7f958c071f30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9584009e50 tx=0x7f958401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2c82672f98aaa68 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:08.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.985+0000 7f958b400640 1 -- 172.21.15.42:0/4262475782 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95840a58d0 con 0x7f958c0d0e00 2024-05-28T14:09:08.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.985+0000 7f958b400640 1 -- 172.21.15.42:0/4262475782 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95840a5a70 con 0x7f958c0d0e00 2024-05-28T14:09:08.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.985+0000 7f958b400640 1 -- 172.21.15.42:0/4262475782 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95840a5e40 con 0x7f958c0d0e00 2024-05-28T14:09:08.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.985+0000 7f9593800640 1 -- 172.21.15.42:0/4262475782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f958c0d0e00 msgr2=0x7f958c071f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:08.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.985+0000 7f9593800640 1 --2- 172.21.15.42:0/4262475782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f958c0d0e00 0x7f958c071f30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9584009e50 tx=0x7f958401ca00 comp rx=0 tx=0).stop 2024-05-28T14:09:08.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.985+0000 7f9593800640 1 -- 172.21.15.42:0/4262475782 shutdown_connections 2024-05-28T14:09:08.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.985+0000 7f9593800640 1 --2- 172.21.15.42:0/4262475782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f958c0d0e00 0x7f958c071f30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.985+0000 7f9593800640 1 --2- 172.21.15.42:0/4262475782 >> v2:172.21.15.63:3300/0 conn(0x7f958c15d1d0 0x7f958c06e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.985+0000 7f9593800640 1 -- 172.21.15.42:0/4262475782 >> 172.21.15.42:0/4262475782 conn(0x7f958c06bdf0 msgr2=0x7f958c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:08.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.985+0000 7f9593800640 1 -- 172.21.15.42:0/4262475782 shutdown_connections 2024-05-28T14:09:08.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.985+0000 7f9593800640 1 -- 172.21.15.42:0/4262475782 wait complete. 2024-05-28T14:09:08.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9593800640 1 Processor -- start 2024-05-28T14:09:08.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9593800640 1 -- start start 2024-05-28T14:09:08.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9593800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f958c06de20 0x7f958c157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:08.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9593800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f958c0d0e00 0x7f958c154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:08.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9593800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f958c15d1d0 0x7f958c0b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:08.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9593800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f958c073050 con 0x7f958c06de20 2024-05-28T14:09:08.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9593800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f958c072ed0 con 0x7f958c0d0e00 2024-05-28T14:09:08.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9593800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f958c0731d0 con 0x7f958c15d1d0 2024-05-28T14:09:08.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9590e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f958c06de20 0x7f958c157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:08.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9590e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f958c06de20 0x7f958c157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43464/0 (socket says 172.21.15.42:43464) 2024-05-28T14:09:08.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9590e00640 1 -- 172.21.15.42:0/633459720 learned_addr learned my addr 172.21.15.42:0/633459720 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:08.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9591800640 1 --2- 172.21.15.42:0/633459720 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f958c15d1d0 0x7f958c0b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f958be00640 1 --2- 172.21.15.42:0/633459720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f958c0d0e00 0x7f958c154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9591800640 1 -- 172.21.15.42:0/633459720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f958c0d0e00 msgr2=0x7f958c154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9591800640 1 --2- 172.21.15.42:0/633459720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f958c0d0e00 0x7f958c154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9591800640 1 -- 172.21.15.42:0/633459720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f958c06de20 msgr2=0x7f958c157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9591800640 1 --2- 172.21.15.42:0/633459720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f958c06de20 0x7f958c157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9591800640 1 -- 172.21.15.42:0/633459720 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c09e0f0 con 0x7f958c15d1d0 2024-05-28T14:09:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.989+0000 7f9591800640 1 --2- 172.21.15.42:0/633459720 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f958c15d1d0 0x7f958c0b0640 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f95840ab000 tx=0x7f95840197a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:08.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.993+0000 7f9588c00640 1 -- 172.21.15.42:0/633459720 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95840bc030 con 0x7f958c15d1d0 2024-05-28T14:09:08.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.993+0000 7f9588c00640 1 -- 172.21.15.42:0/633459720 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9584007c60 con 0x7f958c15d1d0 2024-05-28T14:09:08.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.993+0000 7f9593800640 1 -- 172.21.15.42:0/633459720 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958c157460 con 0x7f958c15d1d0 2024-05-28T14:09:08.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.993+0000 7f9588c00640 1 -- 172.21.15.42:0/633459720 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95840a58d0 con 0x7f958c15d1d0 2024-05-28T14:09:08.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.993+0000 7f9593800640 1 -- 172.21.15.42:0/633459720 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f958c157a20 con 0x7f958c15d1d0 2024-05-28T14:09:08.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.993+0000 7f9593800640 1 -- 172.21.15.42:0/633459720 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f958c068a90 con 0x7f958c15d1d0 2024-05-28T14:09:08.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.993+0000 7f9588c00640 1 -- 172.21.15.42:0/633459720 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f95840bf070 con 0x7f958c15d1d0 2024-05-28T14:09:08.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.997+0000 7f9588c00640 1 --2- 172.21.15.42:0/633459720 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f956c069570 0x7f956c06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:08.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.997+0000 7f9588c00640 1 -- 172.21.15.42:0/633459720 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7f9584120840 con 0x7f958c15d1d0 2024-05-28T14:09:08.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.997+0000 7f9590e00640 1 --2- 172.21.15.42:0/633459720 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f956c069570 0x7f956c06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:08.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:08.997+0000 7f9590e00640 1 --2- 172.21.15.42:0/633459720 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f956c069570 0x7f956c06ba30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f958c1627a0 tx=0x7f9580008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:08.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.001+0000 7f9588c00640 1 -- 172.21.15.42:0/633459720 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f95840c7020 con 0x7f958c15d1d0 2024-05-28T14:09:09.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.321+0000 7f9593800640 1 -- 172.21.15.42:0/633459720 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone8113", "target": ["mon-mgr", ""]}) -- 0x7f958c157210 con 0x7f956c069570 2024-05-28T14:09:09.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.337+0000 7f9588c00640 1 -- 172.21.15.42:0/633459720 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f958c157210 con 0x7f956c069570 2024-05-28T14:09:09.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.337+0000 7f9593800640 1 -- 172.21.15.42:0/633459720 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f956c069570 msgr2=0x7f956c06ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:09.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.337+0000 7f9593800640 1 --2- 172.21.15.42:0/633459720 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f956c069570 0x7f956c06ba30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f958c1627a0 tx=0x7f9580008040 comp rx=0 tx=0).stop 2024-05-28T14:09:09.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.337+0000 7f9593800640 1 -- 172.21.15.42:0/633459720 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f958c15d1d0 msgr2=0x7f958c0b0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:09.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.337+0000 7f9593800640 1 --2- 172.21.15.42:0/633459720 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f958c15d1d0 0x7f958c0b0640 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f95840ab000 tx=0x7f95840197a0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.337+0000 7f9593800640 1 -- 172.21.15.42:0/633459720 shutdown_connections 2024-05-28T14:09:09.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.337+0000 7f9593800640 1 --2- 172.21.15.42:0/633459720 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f956c069570 0x7f956c06ba30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.337+0000 7f9593800640 1 --2- 172.21.15.42:0/633459720 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f958c15d1d0 0x7f958c0b0640 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.337+0000 7f9593800640 1 --2- 172.21.15.42:0/633459720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f958c0d0e00 0x7f958c154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.337+0000 7f9593800640 1 --2- 172.21.15.42:0/633459720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f958c06de20 0x7f958c157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.337+0000 7f9593800640 1 -- 172.21.15.42:0/633459720 >> 172.21.15.42:0/633459720 conn(0x7f958c06bdf0 msgr2=0x7f958c0af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:09.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.337+0000 7f9593800640 1 -- 172.21.15.42:0/633459720 shutdown_connections 2024-05-28T14:09:09.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.337+0000 7f9593800640 1 -- 172.21.15.42:0/633459720 wait complete. 2024-05-28T14:09:09.345 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone81120 2024-05-28T14:09:09.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.505+0000 7fcf3d000640 1 Processor -- start 2024-05-28T14:09:09.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.505+0000 7fcf3d000640 1 -- start start 2024-05-28T14:09:09.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf3d000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3806ee80 0x7fcf3806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:09.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf3d000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcf380d0e00 0x7fcf380af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:09.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf3d000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf381570a0 con 0x7fcf380af360 2024-05-28T14:09:09.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf3d000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcf3815cf00 con 0x7fcf380d0e00 2024-05-28T14:09:09.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf3d000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3815d1e0 con 0x7fcf3806ee80 2024-05-28T14:09:09.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf36800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcf380d0e00 0x7fcf380af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:09.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf35e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3806ee80 0x7fcf3806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:09.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf36800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcf380d0e00 0x7fcf380af7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:54244/0 (socket says 172.21.15.42:54244) 2024-05-28T14:09:09.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf36800640 1 -- 172.21.15.42:0/293321240 learned_addr learned my addr 172.21.15.42:0/293321240 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:09.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf35e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3806ee80 0x7fcf3806dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:55576/0 (socket says 172.21.15.42:55576) 2024-05-28T14:09:09.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf35e00640 1 -- 172.21.15.42:0/293321240 >> v2:172.21.15.63:3300/0 conn(0x7fcf380d0e00 msgr2=0x7fcf380af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:09.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf35e00640 1 --2- 172.21.15.42:0/293321240 >> v2:172.21.15.63:3300/0 conn(0x7fcf380d0e00 0x7fcf380af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf35e00640 1 -- 172.21.15.42:0/293321240 >> v1:172.21.15.63:6789/0 conn(0x7fcf380af360 legacy=0x7fcf380d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:09.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf35e00640 1 -- 172.21.15.42:0/293321240 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3815d4c0 con 0x7fcf3806ee80 2024-05-28T14:09:09.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf36800640 1 --2- 172.21.15.42:0/293321240 >> v2:172.21.15.63:3300/0 conn(0x7fcf380d0e00 0x7fcf380af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:09:09.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.509+0000 7fcf35400640 1 -- 172.21.15.42:0/293321240 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2061904968 0 0) 0x7fcf381570a0 con 0x7fcf380af360 2024-05-28T14:09:09.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.513+0000 7fcf35e00640 1 --2- 172.21.15.42:0/293321240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3806ee80 0x7fcf3806dd40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcf280097c0 tx=0x7fcf2801c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73b733c83278e0f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:09.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.513+0000 7fcf35400640 1 -- 172.21.15.42:0/293321240 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf280b0070 con 0x7fcf3806ee80 2024-05-28T14:09:09.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.513+0000 7fcf35400640 1 -- 172.21.15.42:0/293321240 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf280ab600 con 0x7fcf3806ee80 2024-05-28T14:09:09.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.513+0000 7fcf35400640 1 -- 172.21.15.42:0/293321240 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf280a4e60 con 0x7fcf3806ee80 2024-05-28T14:09:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.513+0000 7fcf3d000640 1 -- 172.21.15.42:0/293321240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3806ee80 msgr2=0x7fcf3806dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.513+0000 7fcf3d000640 1 --2- 172.21.15.42:0/293321240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3806ee80 0x7fcf3806dd40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcf280097c0 tx=0x7fcf2801c380 comp rx=0 tx=0).stop 2024-05-28T14:09:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.513+0000 7fcf3d000640 1 -- 172.21.15.42:0/293321240 shutdown_connections 2024-05-28T14:09:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.513+0000 7fcf3d000640 1 --2- 172.21.15.42:0/293321240 >> v2:172.21.15.63:3300/0 conn(0x7fcf380d0e00 0x7fcf380af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.513+0000 7fcf3d000640 1 --2- 172.21.15.42:0/293321240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3806ee80 0x7fcf3806dd40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.513+0000 7fcf3d000640 1 -- 172.21.15.42:0/293321240 >> 172.21.15.42:0/293321240 conn(0x7fcf3806bdf0 msgr2=0x7fcf3806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.513+0000 7fcf3d000640 1 -- 172.21.15.42:0/293321240 shutdown_connections 2024-05-28T14:09:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.513+0000 7fcf3d000640 1 -- 172.21.15.42:0/293321240 wait complete. 2024-05-28T14:09:09.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.513+0000 7fcf3d000640 1 Processor -- start 2024-05-28T14:09:09.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.513+0000 7fcf3d000640 1 -- start start 2024-05-28T14:09:09.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf3d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3806ee80 0x7fcf3814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:09.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf3d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf380af360 0x7fcf3814b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf3d000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf380d0e00 0x7fcf3815b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf3d000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcf38160e10 con 0x7fcf3806ee80 2024-05-28T14:09:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf3d000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf38160c90 con 0x7fcf380af360 2024-05-28T14:09:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf3d000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcf38160f90 con 0x7fcf380d0e00 2024-05-28T14:09:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf35e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3806ee80 0x7fcf3814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf35e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3806ee80 0x7fcf3814ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43472/0 (socket says 172.21.15.42:43472) 2024-05-28T14:09:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf35e00640 1 -- 172.21.15.42:0/646910873 learned_addr learned my addr 172.21.15.42:0/646910873 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf2d400640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf380af360 0x7fcf3814b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf36800640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf380d0e00 0x7fcf3815b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf35e00640 1 -- 172.21.15.42:0/646910873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf380d0e00 msgr2=0x7fcf3815b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf35e00640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf380d0e00 0x7fcf3815b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf35e00640 1 -- 172.21.15.42:0/646910873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf380af360 msgr2=0x7fcf3814b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf35e00640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf380af360 0x7fcf3814b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf35e00640 1 -- 172.21.15.42:0/646910873 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3814bad0 con 0x7fcf3806ee80 2024-05-28T14:09:09.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf2d400640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf380af360 0x7fcf3814b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:09.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf36800640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf380d0e00 0x7fcf3815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:09.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf35e00640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3806ee80 0x7fcf3814ff50 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fcf280ae040 tx=0x7fcf2801abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:09.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf2ea00640 1 -- 172.21.15.42:0/646910873 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf280b0050 con 0x7fcf3806ee80 2024-05-28T14:09:09.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf2ea00640 1 -- 172.21.15.42:0/646910873 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf280bc070 con 0x7fcf3806ee80 2024-05-28T14:09:09.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf2ea00640 1 -- 172.21.15.42:0/646910873 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf280b5610 con 0x7fcf3806ee80 2024-05-28T14:09:09.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf3d000640 1 -- 172.21.15.42:0/646910873 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf381505f0 con 0x7fcf3806ee80 2024-05-28T14:09:09.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf3d000640 1 -- 172.21.15.42:0/646910873 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3815ab50 con 0x7fcf3806ee80 2024-05-28T14:09:09.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.517+0000 7fcf3d000640 1 -- 172.21.15.42:0/646910873 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf38068a90 con 0x7fcf3806ee80 2024-05-28T14:09:09.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.521+0000 7fcf2ea00640 1 -- 172.21.15.42:0/646910873 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fcf280b57b0 con 0x7fcf3806ee80 2024-05-28T14:09:09.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.521+0000 7fcf2ea00640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf140694f0 0x7fcf1406b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:09.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.521+0000 7fcf2ea00640 1 -- 172.21.15.42:0/646910873 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7fcf2811fea0 con 0x7fcf3806ee80 2024-05-28T14:09:09.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.521+0000 7fcf2d400640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf140694f0 0x7fcf1406b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:09.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.521+0000 7fcf2d400640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf140694f0 0x7fcf1406b9b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fcf3815c870 tx=0x7fcf20009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:09.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.525+0000 7fcf2ea00640 1 -- 172.21.15.42:0/646910873 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcf3814bad0 con 0x7fcf3806ee80 2024-05-28T14:09:09.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.713+0000 7fcf3d000640 1 -- 172.21.15.42:0/646910873 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone81120", "target": ["mon-mgr", ""]}) -- 0x7fcf380b5160 con 0x7fcf140694f0 2024-05-28T14:09:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.733+0000 7fcf2ea00640 1 -- 172.21.15.42:0/646910873 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcf380b5160 con 0x7fcf140694f0 2024-05-28T14:09:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.733+0000 7fcf3d000640 1 -- 172.21.15.42:0/646910873 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf140694f0 msgr2=0x7fcf1406b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.733+0000 7fcf3d000640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf140694f0 0x7fcf1406b9b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fcf3815c870 tx=0x7fcf20009040 comp rx=0 tx=0).stop 2024-05-28T14:09:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.733+0000 7fcf3d000640 1 -- 172.21.15.42:0/646910873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3806ee80 msgr2=0x7fcf3814ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.733+0000 7fcf3d000640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3806ee80 0x7fcf3814ff50 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fcf280ae040 tx=0x7fcf2801abc0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.733+0000 7fcf3d000640 1 -- 172.21.15.42:0/646910873 shutdown_connections 2024-05-28T14:09:09.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.733+0000 7fcf3d000640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf140694f0 0x7fcf1406b9b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.733+0000 7fcf3d000640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf380d0e00 0x7fcf3815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.733+0000 7fcf3d000640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf380af360 0x7fcf3814b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.733+0000 7fcf3d000640 1 --2- 172.21.15.42:0/646910873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3806ee80 0x7fcf3814ff50 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.733+0000 7fcf3d000640 1 -- 172.21.15.42:0/646910873 >> 172.21.15.42:0/646910873 conn(0x7fcf3806bdf0 msgr2=0x7fcf380ad670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:09.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.733+0000 7fcf3d000640 1 -- 172.21.15.42:0/646910873 shutdown_connections 2024-05-28T14:09:09.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.733+0000 7fcf3d000640 1 -- 172.21.15.42:0/646910873 wait complete. 2024-05-28T14:09:09.742 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone8118 2024-05-28T14:09:09.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.953+0000 7f688c400640 1 Processor -- start 2024-05-28T14:09:09.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.957+0000 7f688c400640 1 -- start start 2024-05-28T14:09:09.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.957+0000 7f688c400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f688406e5f0 0x7f6884162cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:09.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.957+0000 7f688c400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6884163200 con 0x7f688415ed00 2024-05-28T14:09:09.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.957+0000 7f688c400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f688415f7d0 con 0x7f68840d0e00 2024-05-28T14:09:09.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.957+0000 7f688c400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f688415fab0 con 0x7f688406e5f0 2024-05-28T14:09:09.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.957+0000 7f6889a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f688406e5f0 0x7f6884162cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:09.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.957+0000 7f6889a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f688406e5f0 0x7f6884162cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43486/0 (socket says 172.21.15.42:43486) 2024-05-28T14:09:09.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.957+0000 7f6889a00640 1 -- 172.21.15.42:0/2676847945 learned_addr learned my addr 172.21.15.42:0/2676847945 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:09.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.957+0000 7f6889a00640 1 -- 172.21.15.42:0/2676847945 >> v1:172.21.15.63:6789/0 conn(0x7f68840d0e00 legacy=0x7f6884071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:09.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.957+0000 7f6889a00640 1 -- 172.21.15.42:0/2676847945 >> v1:172.21.15.42:6789/0 conn(0x7f688415ed00 legacy=0x7f68840d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:09.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.957+0000 7f6883e00640 1 -- 172.21.15.42:0/2676847945 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 584100489 0 0) 0x7f6884163200 con 0x7f688415ed00 2024-05-28T14:09:09.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.957+0000 7f6889a00640 1 -- 172.21.15.42:0/2676847945 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68841624c0 con 0x7f688406e5f0 2024-05-28T14:09:09.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.961+0000 7f6889a00640 1 --2- 172.21.15.42:0/2676847945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f688406e5f0 0x7f6884162cc0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6870009870 tx=0x7f687001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3d38477b82a2b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:09.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.965+0000 7f6883e00640 1 -- 172.21.15.42:0/2676847945 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68700b0070 con 0x7f688406e5f0 2024-05-28T14:09:09.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.965+0000 7f6883e00640 1 -- 172.21.15.42:0/2676847945 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68700ab600 con 0x7f688406e5f0 2024-05-28T14:09:09.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.965+0000 7f6883e00640 1 -- 172.21.15.42:0/2676847945 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68700ab9d0 con 0x7f688406e5f0 2024-05-28T14:09:09.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.965+0000 7f688c400640 1 -- 172.21.15.42:0/2676847945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f688406e5f0 msgr2=0x7f6884162cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:09.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.965+0000 7f688c400640 1 --2- 172.21.15.42:0/2676847945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f688406e5f0 0x7f6884162cc0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6870009870 tx=0x7f687001c440 comp rx=0 tx=0).stop 2024-05-28T14:09:09.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.965+0000 7f688c400640 1 -- 172.21.15.42:0/2676847945 shutdown_connections 2024-05-28T14:09:09.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.965+0000 7f688c400640 1 --2- 172.21.15.42:0/2676847945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f688406e5f0 0x7f6884162cc0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.965+0000 7f688c400640 1 -- 172.21.15.42:0/2676847945 >> 172.21.15.42:0/2676847945 conn(0x7f688406bdf0 msgr2=0x7f688406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:09.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.965+0000 7f688c400640 1 -- 172.21.15.42:0/2676847945 shutdown_connections 2024-05-28T14:09:09.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.965+0000 7f688c400640 1 -- 172.21.15.42:0/2676847945 wait complete. 2024-05-28T14:09:09.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f688c400640 1 Processor -- start 2024-05-28T14:09:09.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f688c400640 1 -- start start 2024-05-28T14:09:09.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f688c400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f688406e5f0 0x7f688414a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f688c400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68840d0e00 0x7f68841505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f688c400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f688415ed00 0x7f688414ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f688c400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6884073cf0 con 0x7f688406e5f0 2024-05-28T14:09:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f688c400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f688415f530 con 0x7f68840d0e00 2024-05-28T14:09:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f688c400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f688416ce70 con 0x7f688415ed00 2024-05-28T14:09:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f6889a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f688406e5f0 0x7f688414a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f6889a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f688406e5f0 0x7f688414a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43492/0 (socket says 172.21.15.42:43492) 2024-05-28T14:09:09.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f6889a00640 1 -- 172.21.15.42:0/3087280356 learned_addr learned my addr 172.21.15.42:0/3087280356 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f6889000640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68840d0e00 0x7f68841505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f688a400640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f688415ed00 0x7f688414ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f6889a00640 1 -- 172.21.15.42:0/3087280356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f688415ed00 msgr2=0x7f688414ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f6889a00640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f688415ed00 0x7f688414ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f6889a00640 1 -- 172.21.15.42:0/3087280356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68840d0e00 msgr2=0x7f68841505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f6889a00640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68840d0e00 0x7f68841505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f6889a00640 1 -- 172.21.15.42:0/3087280356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688414eb20 con 0x7f688406e5f0 2024-05-28T14:09:09.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.969+0000 7f6889000640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68840d0e00 0x7f68841505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:09.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.973+0000 7f6889a00640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f688406e5f0 0x7f688414a8f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f68700ae040 tx=0x7f687001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:09.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.973+0000 7f6881600640 1 -- 172.21.15.42:0/3087280356 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68700ba020 con 0x7f688406e5f0 2024-05-28T14:09:09.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.973+0000 7f688c400640 1 -- 172.21.15.42:0/3087280356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688414ee00 con 0x7f688406e5f0 2024-05-28T14:09:09.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.973+0000 7f688c400640 1 -- 172.21.15.42:0/3087280356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f688414f3c0 con 0x7f688406e5f0 2024-05-28T14:09:09.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.973+0000 7f6881600640 1 -- 172.21.15.42:0/3087280356 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68700ab560 con 0x7f688406e5f0 2024-05-28T14:09:09.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.973+0000 7f6881600640 1 -- 172.21.15.42:0/3087280356 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68700b0040 con 0x7f688406e5f0 2024-05-28T14:09:09.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.973+0000 7f688c400640 1 -- 172.21.15.42:0/3087280356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68600052a0 con 0x7f688406e5f0 2024-05-28T14:09:09.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.973+0000 7f6881600640 1 -- 172.21.15.42:0/3087280356 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f68700bd020 con 0x7f688406e5f0 2024-05-28T14:09:09.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.977+0000 7f6881600640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f68640694f0 0x7f686406b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:09.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.977+0000 7f6881600640 1 -- 172.21.15.42:0/3087280356 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7f687011f910 con 0x7f688406e5f0 2024-05-28T14:09:09.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.977+0000 7f6889000640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f68640694f0 0x7f686406b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:09.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.977+0000 7f6889000640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f68640694f0 0x7f686406b9b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6884161ed0 tx=0x7f6878009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:09.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:09.981+0000 7f6881600640 1 -- 172.21.15.42:0/3087280356 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f68700c4030 con 0x7f688406e5f0 2024-05-28T14:09:10.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.301+0000 7f688c400640 1 -- 172.21.15.42:0/3087280356 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone8118", "target": ["mon-mgr", ""]}) -- 0x7f6860000bd0 con 0x7f68640694f0 2024-05-28T14:09:10.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.313+0000 7f6881600640 1 -- 172.21.15.42:0/3087280356 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6860000bd0 con 0x7f68640694f0 2024-05-28T14:09:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.317+0000 7f688c400640 1 -- 172.21.15.42:0/3087280356 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f68640694f0 msgr2=0x7f686406b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.317+0000 7f688c400640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f68640694f0 0x7f686406b9b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6884161ed0 tx=0x7f6878009040 comp rx=0 tx=0).stop 2024-05-28T14:09:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.317+0000 7f688c400640 1 -- 172.21.15.42:0/3087280356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f688406e5f0 msgr2=0x7f688414a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.317+0000 7f688c400640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f688406e5f0 0x7f688414a8f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f68700ae040 tx=0x7f687001ac80 comp rx=0 tx=0).stop 2024-05-28T14:09:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.317+0000 7f688c400640 1 -- 172.21.15.42:0/3087280356 shutdown_connections 2024-05-28T14:09:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.317+0000 7f688c400640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f68640694f0 0x7f686406b9b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.317+0000 7f688c400640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f688415ed00 0x7f688414ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:10.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.317+0000 7f688c400640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68840d0e00 0x7f68841505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:10.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.317+0000 7f688c400640 1 --2- 172.21.15.42:0/3087280356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f688406e5f0 0x7f688414a8f0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:10.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.317+0000 7f688c400640 1 -- 172.21.15.42:0/3087280356 >> 172.21.15.42:0/3087280356 conn(0x7f688406bdf0 msgr2=0x7f68840749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:10.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.317+0000 7f688c400640 1 -- 172.21.15.42:0/3087280356 shutdown_connections 2024-05-28T14:09:10.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.317+0000 7f688c400640 1 -- 172.21.15.42:0/3087280356 wait complete. 2024-05-28T14:09:10.327 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-28T14:09:10.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-28T14:09:10.384 INFO:teuthology.orchestra.run.smithi042.stdout:2 2024-05-28T14:09:10.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:09:10.391 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T14:09:10.391 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:09:10.505 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:09:10.596 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T14:09:10.596 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:09:10.596 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:09:10.596 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:09:10.605 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:09:10.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcc5e00640 1 Processor -- start 2024-05-28T14:09:10.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcc5e00640 1 -- start start 2024-05-28T14:09:10.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcc5e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdcc015c990 0x7fdcc00d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:10.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcc5e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc00cd300 con 0x7fdcc00d06a0 2024-05-28T14:09:10.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcc5e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc00cd5e0 con 0x7fdcc009bbd0 2024-05-28T14:09:10.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcc5e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc015cd70 con 0x7fdcc015c990 2024-05-28T14:09:10.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbf400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fdcc00d06a0 0x7fdcc00b53e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:57036/0 (socket says 172.21.15.42:57036) 2024-05-28T14:09:10.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbf400640 1 -- 172.21.15.42:0/250716742 learned_addr learned my addr 172.21.15.42:0/250716742 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:10.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbea00640 1 --2- 172.21.15.42:0/250716742 >> v2:172.21.15.63:3300/0 conn(0x7fdcc015c990 0x7fdcc00d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:10.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 -- 172.21.15.42:0/250716742 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1478903036 0 0) 0x7fdcc00cd300 con 0x7fdcc00d06a0 2024-05-28T14:09:10.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 -- 172.21.15.42:0/250716742 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdca4003530 con 0x7fdcc00d06a0 2024-05-28T14:09:10.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 -- 172.21.15.42:0/250716742 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3577027285 0 0) 0x7fdca4003530 con 0x7fdcc00d06a0 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 -- 172.21.15.42:0/250716742 >> v2:172.21.15.63:3300/0 conn(0x7fdcc015c990 msgr2=0x7fdcc00d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 --2- 172.21.15.42:0/250716742 >> v2:172.21.15.63:3300/0 conn(0x7fdcc015c990 0x7fdcc00d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 -- 172.21.15.42:0/250716742 >> v1:172.21.15.63:6789/0 conn(0x7fdcc009bbd0 legacy=0x7fdcc009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 -- 172.21.15.42:0/250716742 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc00cd8c0 con 0x7fdcc00d06a0 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 -- 172.21.15.42:0/250716742 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fdcb8001a00 con 0x7fdcc00d06a0 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 -- 172.21.15.42:0/250716742 >> v1:172.21.15.42:6789/0 conn(0x7fdcc00d06a0 legacy=0x7fdcc00b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 --2- 172.21.15.42:0/250716742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdca40039c0 0x7fdca4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 --2- 172.21.15.42:0/250716742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca4004810 0x7fdca4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 --2- 172.21.15.42:0/250716742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdca40051d0 0x7fdca400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 -- 172.21.15.42:0/250716742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdca400c040 con 0x7fdca40039c0 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 -- 172.21.15.42:0/250716742 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdca400c1f0 con 0x7fdca4004810 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbe000640 1 -- 172.21.15.42:0/250716742 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdca400c3f0 con 0x7fdca40051d0 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbea00640 1 --2- 172.21.15.42:0/250716742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdca40039c0 0x7fdca4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcb7e00640 1 --2- 172.21.15.42:0/250716742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca4004810 0x7fdca4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:10.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbf400640 1 --2- 172.21.15.42:0/250716742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdca40051d0 0x7fdca400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:10.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbea00640 1 -- 172.21.15.42:0/250716742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdca40051d0 msgr2=0x7fdca400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:10.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbea00640 1 --2- 172.21.15.42:0/250716742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdca40051d0 0x7fdca400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:10.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbea00640 1 -- 172.21.15.42:0/250716742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca4004810 msgr2=0x7fdca4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:10.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbea00640 1 --2- 172.21.15.42:0/250716742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca4004810 0x7fdca4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:10.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbea00640 1 -- 172.21.15.42:0/250716742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca4003530 con 0x7fdca40039c0 2024-05-28T14:09:10.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.805+0000 7fdcbf400640 1 -- 172.21.15.42:0/250716742 reap_dead start 2024-05-28T14:09:10.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.809+0000 7fdcbea00640 1 --2- 172.21.15.42:0/250716742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdca40039c0 0x7fdca4003de0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009ae0 tx=0x7fdcb001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c4830c81dfb062 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:10.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.809+0000 7fdcbe000640 1 -- 172.21.15.42:0/250716742 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00096b0 con 0x7fdca40039c0 2024-05-28T14:09:10.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.809+0000 7fdcbe000640 1 -- 172.21.15.42:0/250716742 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb00a6500 con 0x7fdca40039c0 2024-05-28T14:09:10.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.809+0000 7fdcbe000640 1 -- 172.21.15.42:0/250716742 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00a68f0 con 0x7fdca40039c0 2024-05-28T14:09:10.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.809+0000 7fdcc5e00640 1 -- 172.21.15.42:0/250716742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdca40039c0 msgr2=0x7fdca4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:10.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.809+0000 7fdcc5e00640 1 --2- 172.21.15.42:0/250716742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdca40039c0 0x7fdca4003de0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009ae0 tx=0x7fdcb001ca40 comp rx=0 tx=0).stop 2024-05-28T14:09:10.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.809+0000 7fdcc5e00640 1 -- 172.21.15.42:0/250716742 shutdown_connections 2024-05-28T14:09:10.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.809+0000 7fdcc5e00640 1 --2- 172.21.15.42:0/250716742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdca40039c0 0x7fdca4003de0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:10.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.809+0000 7fdcc5e00640 1 -- 172.21.15.42:0/250716742 >> 172.21.15.42:0/250716742 conn(0x7fdcc00b4080 msgr2=0x7fdcc00b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:10.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.809+0000 7fdcc5e00640 1 -- 172.21.15.42:0/250716742 shutdown_connections 2024-05-28T14:09:10.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.809+0000 7fdcc5e00640 1 -- 172.21.15.42:0/250716742 wait complete. 2024-05-28T14:09:10.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcc5e00640 1 Processor -- start 2024-05-28T14:09:10.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcc5e00640 1 -- start start 2024-05-28T14:09:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcc5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca40039c0 0x7fdcc01f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcc5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcc0158a60 0x7fdcc00b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcc5e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcc00b7680 0x7fdcc00b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcc5e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0160580 con 0x7fdcc0158a60 2024-05-28T14:09:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcc5e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0160400 con 0x7fdca40039c0 2024-05-28T14:09:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcc5e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc0160700 con 0x7fdcc00b7680 2024-05-28T14:09:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcc0158a60 0x7fdcc00b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcc0158a60 0x7fdcc00b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43516/0 (socket says 172.21.15.42:43516) 2024-05-28T14:09:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcb7e00640 1 -- 172.21.15.42:0/2605787677 learned_addr learned my addr 172.21.15.42:0/2605787677 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcbea00640 1 --2- 172.21.15.42:0/2605787677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca40039c0 0x7fdcc01f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcbf400640 1 --2- 172.21.15.42:0/2605787677 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcc00b7680 0x7fdcc00b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcb7e00640 1 -- 172.21.15.42:0/2605787677 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcc00b7680 msgr2=0x7fdcc00b7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcb7e00640 1 --2- 172.21.15.42:0/2605787677 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcc00b7680 0x7fdcc00b7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcb7e00640 1 -- 172.21.15.42:0/2605787677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca40039c0 msgr2=0x7fdcc01f8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcb7e00640 1 --2- 172.21.15.42:0/2605787677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca40039c0 0x7fdcc01f8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcb7e00640 1 -- 172.21.15.42:0/2605787677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc00b6530 con 0x7fdcc0158a60 2024-05-28T14:09:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcb7e00640 1 --2- 172.21.15.42:0/2605787677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcc0158a60 0x7fdcc00b7140 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fdca800f820 tx=0x7fdca800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:10.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcb7400640 1 -- 172.21.15.42:0/2605787677 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8001cf0 con 0x7fdcc0158a60 2024-05-28T14:09:10.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcb7400640 1 -- 172.21.15.42:0/2605787677 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdca8018070 con 0x7fdcc0158a60 2024-05-28T14:09:10.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcb7400640 1 -- 172.21.15.42:0/2605787677 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca800d730 con 0x7fdcc0158a60 2024-05-28T14:09:10.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcc5e00640 1 -- 172.21.15.42:0/2605787677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc00b51b0 con 0x7fdcc0158a60 2024-05-28T14:09:10.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.813+0000 7fdcc5e00640 1 -- 172.21.15.42:0/2605787677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc00b5770 con 0x7fdcc0158a60 2024-05-28T14:09:10.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.817+0000 7fdcc5e00640 1 -- 172.21.15.42:0/2605787677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc0068a90 con 0x7fdcc0158a60 2024-05-28T14:09:10.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.817+0000 7fdcb7400640 1 -- 172.21.15.42:0/2605787677 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fdca800d8d0 con 0x7fdcc0158a60 2024-05-28T14:09:10.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.817+0000 7fdcb7400640 1 --2- 172.21.15.42:0/2605787677 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc9c069520 0x7fdc9c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:10.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.817+0000 7fdcb7400640 1 -- 172.21.15.42:0/2605787677 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7fdca8085560 con 0x7fdcc0158a60 2024-05-28T14:09:10.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.817+0000 7fdcbea00640 1 --2- 172.21.15.42:0/2605787677 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc9c069520 0x7fdc9c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:10.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.817+0000 7fdcbea00640 1 --2- 172.21.15.42:0/2605787677 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc9c069520 0x7fdc9c06b9e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00af000 tx=0x7fdcb00046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:10.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:10.821+0000 7fdcb7400640 1 -- 172.21.15.42:0/2605787677 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdca8056bd0 con 0x7fdcc0158a60 2024-05-28T14:09:11.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.117+0000 7fdcc5e00640 1 -- 172.21.15.42:0/2605787677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdcc00b4ce0 con 0x7fdcc0158a60 2024-05-28T14:09:11.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.121+0000 7fdcb7400640 1 -- 172.21.15.42:0/2605787677 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16814 (secure 0 0 0) 0x7fdca805aed0 con 0x7fdcc0158a60 2024-05-28T14:09:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:11.105 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":78,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:09:07.614954+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:06:23.241667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-28T14:06:23.769074+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:09:11.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.125+0000 7fdcc5e00640 1 -- 172.21.15.42:0/2605787677 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc9c069520 msgr2=0x7fdc9c06b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:11.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.125+0000 7fdcc5e00640 1 --2- 172.21.15.42:0/2605787677 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc9c069520 0x7fdc9c06b9e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00af000 tx=0x7fdcb00046d0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.125+0000 7fdcc5e00640 1 -- 172.21.15.42:0/2605787677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcc0158a60 msgr2=0x7fdcc00b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:11.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.125+0000 7fdcc5e00640 1 --2- 172.21.15.42:0/2605787677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcc0158a60 0x7fdcc00b7140 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fdca800f820 tx=0x7fdca800fce0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.125+0000 7fdcc5e00640 1 -- 172.21.15.42:0/2605787677 shutdown_connections 2024-05-28T14:09:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.125+0000 7fdcc5e00640 1 --2- 172.21.15.42:0/2605787677 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdc9c069520 0x7fdc9c06b9e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.125+0000 7fdcc5e00640 1 --2- 172.21.15.42:0/2605787677 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcc00b7680 0x7fdcc00b7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.125+0000 7fdcc5e00640 1 --2- 172.21.15.42:0/2605787677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcc0158a60 0x7fdcc00b7140 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.125+0000 7fdcc5e00640 1 --2- 172.21.15.42:0/2605787677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdca40039c0 0x7fdcc01f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.125+0000 7fdcc5e00640 1 -- 172.21.15.42:0/2605787677 >> 172.21.15.42:0/2605787677 conn(0x7fdcc00b4080 msgr2=0x7fdcc015fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.125+0000 7fdcc5e00640 1 -- 172.21.15.42:0/2605787677 shutdown_connections 2024-05-28T14:09:11.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.125+0000 7fdcc5e00640 1 -- 172.21.15.42:0/2605787677 wait complete. 2024-05-28T14:09:11.141 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T14:06:23.241667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T14:06:23.769074+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 77, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:09:11.141 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:09:11.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f823be00640 1 Processor -- start 2024-05-28T14:09:11.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f823be00640 1 -- start start 2024-05-28T14:09:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f823be00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f823406e640 0x7f823406ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f823be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8234070510 0x7f823409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f823be00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8234167390 con 0x7f82340c54b0 2024-05-28T14:09:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f823be00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f82340b1880 con 0x7f823406e640 2024-05-28T14:09:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f823be00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8234162fc0 con 0x7f8234070510 2024-05-28T14:09:11.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f8238a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f823406e640 0x7f823406ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f8239e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8234070510 0x7f823409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f8238a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f823406e640 0x7f823406ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:54298/0 (socket says 172.21.15.42:54298) 2024-05-28T14:09:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f8238a00640 1 -- 172.21.15.42:0/525170792 learned_addr learned my addr 172.21.15.42:0/525170792 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f8238a00640 1 -- 172.21.15.42:0/525170792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8234070510 msgr2=0x7f823409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f8238a00640 1 --2- 172.21.15.42:0/525170792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8234070510 0x7f823409aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f8238a00640 1 -- 172.21.15.42:0/525170792 >> v1:172.21.15.42:6789/0 conn(0x7f82340c54b0 legacy=0x7f82340c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.393+0000 7f8238a00640 1 -- 172.21.15.42:0/525170792 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8234167670 con 0x7f823406e640 2024-05-28T14:09:11.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.401+0000 7f8238a00640 1 --2- 172.21.15.42:0/525170792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f823406e640 0x7f823406ffd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8228009e50 tx=0x7f822801c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db9d654e8659e5f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:11.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.401+0000 7f8233e00640 1 -- 172.21.15.42:0/525170792 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82280a5800 con 0x7f823406e640 2024-05-28T14:09:11.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.401+0000 7f8233e00640 1 -- 172.21.15.42:0/525170792 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82280a59a0 con 0x7f823406e640 2024-05-28T14:09:11.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.401+0000 7f8233e00640 1 -- 172.21.15.42:0/525170792 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82280a5d70 con 0x7f823406e640 2024-05-28T14:09:11.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.401+0000 7f823be00640 1 -- 172.21.15.42:0/525170792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f823406e640 msgr2=0x7f823406ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:11.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.401+0000 7f823be00640 1 --2- 172.21.15.42:0/525170792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f823406e640 0x7f823406ffd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8228009e50 tx=0x7f822801c980 comp rx=0 tx=0).stop 2024-05-28T14:09:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.401+0000 7f823be00640 1 -- 172.21.15.42:0/525170792 shutdown_connections 2024-05-28T14:09:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.401+0000 7f823be00640 1 --2- 172.21.15.42:0/525170792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8234070510 0x7f823409aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.401+0000 7f823be00640 1 --2- 172.21.15.42:0/525170792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f823406e640 0x7f823406ffd0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.401+0000 7f823be00640 1 -- 172.21.15.42:0/525170792 >> 172.21.15.42:0/525170792 conn(0x7f82340b3fd0 msgr2=0x7f82340b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.401+0000 7f823be00640 1 -- 172.21.15.42:0/525170792 shutdown_connections 2024-05-28T14:09:11.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.405+0000 7f823be00640 1 -- 172.21.15.42:0/525170792 wait complete. 2024-05-28T14:09:11.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.405+0000 7f823be00640 1 Processor -- start 2024-05-28T14:09:11.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.405+0000 7f823be00640 1 -- start start 2024-05-28T14:09:11.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f823be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f823406e640 0x7f8234158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:11.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f823be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8234070510 0x7f82340c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:11.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f823be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82340c54b0 0x7f823415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:11.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f823be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f82341705a0 con 0x7f823406e640 2024-05-28T14:09:11.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f823be00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8234170720 con 0x7f8234070510 2024-05-28T14:09:11.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f823be00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82341708a0 con 0x7f82340c54b0 2024-05-28T14:09:11.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f8238a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8234070510 0x7f82340c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:11.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f8238a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8234070510 0x7f82340c7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:54304/0 (socket says 172.21.15.42:54304) 2024-05-28T14:09:11.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f8238a00640 1 -- 172.21.15.42:0/4263549904 learned_addr learned my addr 172.21.15.42:0/4263549904 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:11.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f8239e00640 1 --2- 172.21.15.42:0/4263549904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82340c54b0 0x7f823415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:11.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f8239400640 1 --2- 172.21.15.42:0/4263549904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f823406e640 0x7f8234158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:11.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f8238a00640 1 -- 172.21.15.42:0/4263549904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82340c54b0 msgr2=0x7f823415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:11.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f8238a00640 1 --2- 172.21.15.42:0/4263549904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82340c54b0 0x7f823415d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f8238a00640 1 -- 172.21.15.42:0/4263549904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f823406e640 msgr2=0x7f8234158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:11.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f8238a00640 1 --2- 172.21.15.42:0/4263549904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f823406e640 0x7f8234158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f8238a00640 1 -- 172.21.15.42:0/4263549904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82340c83e0 con 0x7f8234070510 2024-05-28T14:09:11.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f8238a00640 1 --2- 172.21.15.42:0/4263549904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8234070510 0x7f82340c7900 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f82280af040 tx=0x7f8228019720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:11.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f8231600640 1 -- 172.21.15.42:0/4263549904 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82280bc020 con 0x7f8234070510 2024-05-28T14:09:11.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f8231600640 1 -- 172.21.15.42:0/4263549904 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82280b8880 con 0x7f8234070510 2024-05-28T14:09:11.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f823be00640 1 -- 172.21.15.42:0/4263549904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82340c86c0 con 0x7f8234070510 2024-05-28T14:09:11.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f8231600640 1 -- 172.21.15.42:0/4263549904 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228009510 con 0x7f8234070510 2024-05-28T14:09:11.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.409+0000 7f823be00640 1 -- 172.21.15.42:0/4263549904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82340c8c80 con 0x7f8234070510 2024-05-28T14:09:11.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.413+0000 7f823be00640 1 -- 172.21.15.42:0/4263549904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82340a0570 con 0x7f8234070510 2024-05-28T14:09:11.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.413+0000 7f8231600640 1 -- 172.21.15.42:0/4263549904 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f82280bf020 con 0x7f8234070510 2024-05-28T14:09:11.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.417+0000 7f8231600640 1 --2- 172.21.15.42:0/4263549904 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8214069540 0x7f821406ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:11.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.417+0000 7f8231600640 1 -- 172.21.15.42:0/4263549904 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7f8228120880 con 0x7f8234070510 2024-05-28T14:09:11.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.417+0000 7f8239400640 1 --2- 172.21.15.42:0/4263549904 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8214069540 0x7f821406ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:11.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.417+0000 7f8239400640 1 --2- 172.21.15.42:0/4263549904 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8214069540 0x7f821406ba00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f822c004e30 tx=0x7f822c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:11.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.425+0000 7f8231600640 1 -- 172.21.15.42:0/4263549904 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8228129020 con 0x7f8234070510 2024-05-28T14:09:11.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.713+0000 7f823be00640 1 -- 172.21.15.42:0/4263549904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8234166320 con 0x7f8234070510 2024-05-28T14:09:11.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.717+0000 7f8231600640 1 -- 172.21.15.42:0/4263549904 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f82280c6030 con 0x7f8234070510 2024-05-28T14:09:11.699 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:09:11.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.717+0000 7f823be00640 1 -- 172.21.15.42:0/4263549904 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8214069540 msgr2=0x7f821406ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:11.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.717+0000 7f823be00640 1 --2- 172.21.15.42:0/4263549904 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8214069540 0x7f821406ba00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f822c004e30 tx=0x7f822c009040 comp rx=0 tx=0).stop 2024-05-28T14:09:11.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.717+0000 7f823be00640 1 -- 172.21.15.42:0/4263549904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8234070510 msgr2=0x7f82340c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:11.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.717+0000 7f823be00640 1 --2- 172.21.15.42:0/4263549904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8234070510 0x7f82340c7900 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f82280af040 tx=0x7f8228019720 comp rx=0 tx=0).stop 2024-05-28T14:09:11.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.717+0000 7f823be00640 1 -- 172.21.15.42:0/4263549904 shutdown_connections 2024-05-28T14:09:11.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.717+0000 7f823be00640 1 --2- 172.21.15.42:0/4263549904 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8214069540 0x7f821406ba00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.717+0000 7f823be00640 1 --2- 172.21.15.42:0/4263549904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82340c54b0 0x7f823415d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.717+0000 7f823be00640 1 --2- 172.21.15.42:0/4263549904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8234070510 0x7f82340c7900 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.717+0000 7f823be00640 1 --2- 172.21.15.42:0/4263549904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f823406e640 0x7f8234158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.717+0000 7f823be00640 1 -- 172.21.15.42:0/4263549904 >> 172.21.15.42:0/4263549904 conn(0x7f82340b3fd0 msgr2=0x7f8234160c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:11.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.717+0000 7f823be00640 1 -- 172.21.15.42:0/4263549904 shutdown_connections 2024-05-28T14:09:11.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.717+0000 7f823be00640 1 -- 172.21.15.42:0/4263549904 wait complete. 2024-05-28T14:09:11.721 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:09:11.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.909+0000 7f47a5000640 1 Processor -- start 2024-05-28T14:09:11.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.913+0000 7f47a5000640 1 -- start start 2024-05-28T14:09:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.913+0000 7f47a5000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f47a0072860 0x7f47a00cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.913+0000 7f47a5000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47a00cded0 con 0x7f47a015c670 2024-05-28T14:09:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.913+0000 7f47a5000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47a00ce1b0 con 0x7f47a00cef70 2024-05-28T14:09:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.913+0000 7f47a5000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f47a00ceae0 con 0x7f47a0072860 2024-05-28T14:09:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.913+0000 7f479de00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f47a015c670 0x7f47a0072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:57062/0 (socket says 172.21.15.42:57062) 2024-05-28T14:09:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.913+0000 7f479de00640 1 -- 172.21.15.42:0/15437329 learned_addr learned my addr 172.21.15.42:0/15437329 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.913+0000 7f479d400640 1 --2- 172.21.15.42:0/15437329 >> v2:172.21.15.63:3300/0 conn(0x7f47a0072860 0x7f47a00cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.913+0000 7f479ca00640 1 -- 172.21.15.42:0/15437329 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1332364075 0 0) 0x7f47a00cded0 con 0x7f47a015c670 2024-05-28T14:09:11.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.913+0000 7f479ca00640 1 -- 172.21.15.42:0/15437329 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4784003530 con 0x7f47a015c670 2024-05-28T14:09:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.913+0000 7f479d400640 1 -- 172.21.15.42:0/15437329 >> v1:172.21.15.63:6789/0 conn(0x7f47a00cef70 legacy=0x7f47a015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.913+0000 7f479d400640 1 -- 172.21.15.42:0/15437329 >> v1:172.21.15.42:6789/0 conn(0x7f47a015c670 legacy=0x7f47a0072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:11.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.913+0000 7f479d400640 1 -- 172.21.15.42:0/15437329 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a00ce490 con 0x7f47a0072860 2024-05-28T14:09:11.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.917+0000 7f479d400640 1 --2- 172.21.15.42:0/15437329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47a0072860 0x7f47a00cd250 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f47900097c0 tx=0x7f479001c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e31a07cf77892b91 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:11.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.921+0000 7f479ca00640 1 -- 172.21.15.42:0/15437329 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47900b0070 con 0x7f47a0072860 2024-05-28T14:09:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.921+0000 7f479ca00640 1 -- 172.21.15.42:0/15437329 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47900ab600 con 0x7f47a0072860 2024-05-28T14:09:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.921+0000 7f479ca00640 1 -- 172.21.15.42:0/15437329 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47900ab9d0 con 0x7f47a0072860 2024-05-28T14:09:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.921+0000 7f47a5000640 1 -- 172.21.15.42:0/15437329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47a0072860 msgr2=0x7f47a00cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:11.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.921+0000 7f47a5000640 1 --2- 172.21.15.42:0/15437329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47a0072860 0x7f47a00cd250 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f47900097c0 tx=0x7f479001c420 comp rx=0 tx=0).stop 2024-05-28T14:09:11.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.921+0000 7f47a5000640 1 -- 172.21.15.42:0/15437329 shutdown_connections 2024-05-28T14:09:11.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.921+0000 7f47a5000640 1 --2- 172.21.15.42:0/15437329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47a0072860 0x7f47a00cd250 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.921+0000 7f47a5000640 1 -- 172.21.15.42:0/15437329 >> 172.21.15.42:0/15437329 conn(0x7f47a006b9f0 msgr2=0x7f47a006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:11.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.921+0000 7f47a5000640 1 -- 172.21.15.42:0/15437329 shutdown_connections 2024-05-28T14:09:11.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.921+0000 7f47a5000640 1 -- 172.21.15.42:0/15437329 wait complete. 2024-05-28T14:09:11.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f47a5000640 1 Processor -- start 2024-05-28T14:09:11.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f47a5000640 1 -- start start 2024-05-28T14:09:11.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f47a5000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47a0072860 0x7f47a0154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:11.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f47a5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47a00cef70 0x7f47a0141110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:11.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f47a5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a015c670 0x7f47a00b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:11.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f47a5000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f47a016a920 con 0x7f47a015c670 2024-05-28T14:09:11.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f47a5000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f47a016a7a0 con 0x7f47a00cef70 2024-05-28T14:09:11.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f47a5000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f47a016aaa0 con 0x7f47a0072860 2024-05-28T14:09:11.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f479e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a015c670 0x7f47a00b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:11.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f479e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a015c670 0x7f47a00b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43538/0 (socket says 172.21.15.42:43538) 2024-05-28T14:09:11.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f479e800640 1 -- 172.21.15.42:0/948468560 learned_addr learned my addr 172.21.15.42:0/948468560 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:11.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f479de00640 1 --2- 172.21.15.42:0/948468560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47a0072860 0x7f47a0154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:11.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f479e800640 1 -- 172.21.15.42:0/948468560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47a0072860 msgr2=0x7f47a0154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:11.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f479e800640 1 --2- 172.21.15.42:0/948468560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47a0072860 0x7f47a0154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f479e800640 1 -- 172.21.15.42:0/948468560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47a00cef70 msgr2=0x7f47a0141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:11.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f479e800640 1 --2- 172.21.15.42:0/948468560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47a00cef70 0x7f47a0141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:11.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f479e800640 1 -- 172.21.15.42:0/948468560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a014fb50 con 0x7f47a015c670 2024-05-28T14:09:11.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f479e800640 1 --2- 172.21.15.42:0/948468560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a015c670 0x7f47a00b3970 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f478800c3c0 tx=0x7f478800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:11.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f4796000640 1 -- 172.21.15.42:0/948468560 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788011790 con 0x7f47a015c670 2024-05-28T14:09:11.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f4796000640 1 -- 172.21.15.42:0/948468560 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4788017070 con 0x7f47a015c670 2024-05-28T14:09:11.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f4796000640 1 -- 172.21.15.42:0/948468560 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788011b50 con 0x7f47a015c670 2024-05-28T14:09:11.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f47a5000640 1 -- 172.21.15.42:0/948468560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a0155d30 con 0x7f47a015c670 2024-05-28T14:09:11.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.925+0000 7f47a5000640 1 -- 172.21.15.42:0/948468560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a01551c0 con 0x7f47a015c670 2024-05-28T14:09:11.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.929+0000 7f47a5000640 1 -- 172.21.15.42:0/948468560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a0073b90 con 0x7f47a015c670 2024-05-28T14:09:11.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.929+0000 7f4796000640 1 -- 172.21.15.42:0/948468560 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f4788011cf0 con 0x7f47a015c670 2024-05-28T14:09:11.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.933+0000 7f4796000640 1 --2- 172.21.15.42:0/948468560 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f477c069540 0x7f477c06ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:11.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.933+0000 7f4796000640 1 -- 172.21.15.42:0/948468560 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7f4788086120 con 0x7f47a015c670 2024-05-28T14:09:11.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.933+0000 7f479de00640 1 --2- 172.21.15.42:0/948468560 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f477c069540 0x7f477c06ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:11.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.937+0000 7f479de00640 1 --2- 172.21.15.42:0/948468560 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f477c069540 0x7f477c06ba00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f47980018b0 tx=0x7f4798001840 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:11.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:11.937+0000 7f4796000640 1 -- 172.21.15.42:0/948468560 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4788057e20 con 0x7f47a015c670 2024-05-28T14:09:12.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.241+0000 7f47a5000640 1 -- 172.21.15.42:0/948468560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f47a0155400 con 0x7f47a015c670 2024-05-28T14:09:12.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.241+0000 7f4796000640 1 -- 172.21.15.42:0/948468560 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f4788057720 con 0x7f47a015c670 2024-05-28T14:09:12.226 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:09:12.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.245+0000 7f47a5000640 1 -- 172.21.15.42:0/948468560 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f477c069540 msgr2=0x7f477c06ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:12.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.245+0000 7f47a5000640 1 --2- 172.21.15.42:0/948468560 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f477c069540 0x7f477c06ba00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f47980018b0 tx=0x7f4798001840 comp rx=0 tx=0).stop 2024-05-28T14:09:12.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.245+0000 7f47a5000640 1 -- 172.21.15.42:0/948468560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a015c670 msgr2=0x7f47a00b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:12.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.245+0000 7f47a5000640 1 --2- 172.21.15.42:0/948468560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a015c670 0x7f47a00b3970 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f478800c3c0 tx=0x7f478800c880 comp rx=0 tx=0).stop 2024-05-28T14:09:12.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.245+0000 7f47a5000640 1 -- 172.21.15.42:0/948468560 shutdown_connections 2024-05-28T14:09:12.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.245+0000 7f47a5000640 1 --2- 172.21.15.42:0/948468560 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f477c069540 0x7f477c06ba00 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.245+0000 7f47a5000640 1 --2- 172.21.15.42:0/948468560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47a015c670 0x7f47a00b3970 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.245+0000 7f47a5000640 1 --2- 172.21.15.42:0/948468560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47a00cef70 0x7f47a0141110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.245+0000 7f47a5000640 1 --2- 172.21.15.42:0/948468560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47a0072860 0x7f47a0154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.245+0000 7f47a5000640 1 -- 172.21.15.42:0/948468560 >> 172.21.15.42:0/948468560 conn(0x7f47a006b9f0 msgr2=0x7f47a0156170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:12.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.249+0000 7f47a5000640 1 -- 172.21.15.42:0/948468560 shutdown_connections 2024-05-28T14:09:12.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.249+0000 7f47a5000640 1 -- 172.21.15.42:0/948468560 wait complete. 2024-05-28T14:09:12.264 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:09:12.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.501+0000 7f5534c00640 1 Processor -- start 2024-05-28T14:09:12.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.501+0000 7f5534c00640 1 -- start start 2024-05-28T14:09:12.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.505+0000 7f5534c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f55300cd450 0x7f55300d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:12.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.505+0000 7f5534c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55300d0b30 con 0x7f553006f680 2024-05-28T14:09:12.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.505+0000 7f5534c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55300d0e10 con 0x7f553006b9f0 2024-05-28T14:09:12.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.505+0000 7f5534c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f553006ec30 con 0x7f55300cd450 2024-05-28T14:09:12.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.505+0000 7f552fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f553006f680 0x7f55300cb830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:57064/0 (socket says 172.21.15.42:57064) 2024-05-28T14:09:12.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.505+0000 7f552fe00640 1 -- 172.21.15.42:0/2500137308 learned_addr learned my addr 172.21.15.42:0/2500137308 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:12.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.505+0000 7f5526a00640 1 --2- 172.21.15.42:0/2500137308 >> v2:172.21.15.63:3300/0 conn(0x7f55300cd450 0x7f55300d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:12.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.505+0000 7f552ea00640 1 -- 172.21.15.42:0/2500137308 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2391367150 0 0) 0x7f55300d0b30 con 0x7f553006f680 2024-05-28T14:09:12.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.505+0000 7f5526a00640 1 -- 172.21.15.42:0/2500137308 >> v1:172.21.15.63:6789/0 conn(0x7f553006b9f0 legacy=0x7f553006bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:12.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.505+0000 7f5526a00640 1 -- 172.21.15.42:0/2500137308 >> v1:172.21.15.42:6789/0 conn(0x7f553006f680 legacy=0x7f55300cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:12.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.505+0000 7f5526a00640 1 -- 172.21.15.42:0/2500137308 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553006ee80 con 0x7f55300cd450 2024-05-28T14:09:12.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5526a00640 1 --2- 172.21.15.42:0/2500137308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55300cd450 0x7f55300d0450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5518009740 tx=0x7f551801c240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53eb4dbf2ace92e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:12.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f552ea00640 1 -- 172.21.15.42:0/2500137308 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55180b0070 con 0x7f55300cd450 2024-05-28T14:09:12.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f552ea00640 1 -- 172.21.15.42:0/2500137308 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55180ab600 con 0x7f55300cd450 2024-05-28T14:09:12.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f552ea00640 1 -- 172.21.15.42:0/2500137308 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55180ab9d0 con 0x7f55300cd450 2024-05-28T14:09:12.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 -- 172.21.15.42:0/2500137308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55300cd450 msgr2=0x7f55300d0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:12.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 --2- 172.21.15.42:0/2500137308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55300cd450 0x7f55300d0450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5518009740 tx=0x7f551801c240 comp rx=0 tx=0).stop 2024-05-28T14:09:12.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 -- 172.21.15.42:0/2500137308 shutdown_connections 2024-05-28T14:09:12.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 --2- 172.21.15.42:0/2500137308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55300cd450 0x7f55300d0450 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 -- 172.21.15.42:0/2500137308 >> 172.21.15.42:0/2500137308 conn(0x7f55300b2d70 msgr2=0x7f55300b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:12.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 -- 172.21.15.42:0/2500137308 shutdown_connections 2024-05-28T14:09:12.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 -- 172.21.15.42:0/2500137308 wait complete. 2024-05-28T14:09:12.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 Processor -- start 2024-05-28T14:09:12.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 -- start start 2024-05-28T14:09:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f553006b9f0 0x7f553015e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f553006f680 0x7f553015e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55300cd450 0x7f55300c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f553016cbf0 con 0x7f553006f680 2024-05-28T14:09:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f553016ca70 con 0x7f55300cd450 2024-05-28T14:09:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5534c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f553016cd70 con 0x7f553006b9f0 2024-05-28T14:09:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5526a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f553006f680 0x7f553015e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f552f400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f553006b9f0 0x7f553015e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.509+0000 7f5526a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f553006f680 0x7f553015e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43544/0 (socket says 172.21.15.42:43544) 2024-05-28T14:09:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f5526a00640 1 -- 172.21.15.42:0/3884976572 learned_addr learned my addr 172.21.15.42:0/3884976572 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f552fe00640 1 --2- 172.21.15.42:0/3884976572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55300cd450 0x7f55300c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f5526a00640 1 -- 172.21.15.42:0/3884976572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f553006b9f0 msgr2=0x7f553015e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f5526a00640 1 --2- 172.21.15.42:0/3884976572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f553006b9f0 0x7f553015e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f5526a00640 1 -- 172.21.15.42:0/3884976572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55300cd450 msgr2=0x7f55300c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f5526a00640 1 --2- 172.21.15.42:0/3884976572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55300cd450 0x7f55300c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f5526a00640 1 -- 172.21.15.42:0/3884976572 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55300c4ce0 con 0x7f553006f680 2024-05-28T14:09:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f552fe00640 1 --2- 172.21.15.42:0/3884976572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55300cd450 0x7f55300c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:09:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f5526a00640 1 --2- 172.21.15.42:0/3884976572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f553006f680 0x7f553015e8e0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f55180ae040 tx=0x7f5518018fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f5527e00640 1 -- 172.21.15.42:0/3884976572 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55180b0040 con 0x7f553006f680 2024-05-28T14:09:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f5527e00640 1 -- 172.21.15.42:0/3884976572 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5518004030 con 0x7f553006f680 2024-05-28T14:09:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f5527e00640 1 -- 172.21.15.42:0/3884976572 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518004400 con 0x7f553006f680 2024-05-28T14:09:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f5534c00640 1 -- 172.21.15.42:0/3884976572 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55300c4fc0 con 0x7f553006f680 2024-05-28T14:09:12.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f5534c00640 1 -- 172.21.15.42:0/3884976572 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55300c7530 con 0x7f553006f680 2024-05-28T14:09:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f5534c00640 1 -- 172.21.15.42:0/3884976572 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553004fcf0 con 0x7f553006f680 2024-05-28T14:09:12.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.513+0000 7f5527e00640 1 -- 172.21.15.42:0/3884976572 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f55180b3040 con 0x7f553006f680 2024-05-28T14:09:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.517+0000 7f5527e00640 1 --2- 172.21.15.42:0/3884976572 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f550c0694f0 0x7f550c06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.517+0000 7f5527e00640 1 -- 172.21.15.42:0/3884976572 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7f551811fd10 con 0x7f553006f680 2024-05-28T14:09:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.517+0000 7f552f400640 1 --2- 172.21.15.42:0/3884976572 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f550c0694f0 0x7f550c06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:12.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.517+0000 7f552f400640 1 --2- 172.21.15.42:0/3884976572 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f550c0694f0 0x7f550c06b9b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5520007fd0 tx=0x7f5520009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:12.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.521+0000 7f5527e00640 1 -- 172.21.15.42:0/3884976572 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f55180bb020 con 0x7f553006f680 2024-05-28T14:09:12.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.685+0000 7f5534c00640 1 -- 172.21.15.42:0/3884976572 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f553006e680 con 0x7f553006f680 2024-05-28T14:09:12.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.685+0000 7f5527e00640 1 -- 172.21.15.42:0/3884976572 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f5518015f50 con 0x7f553006f680 2024-05-28T14:09:12.669 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:09:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.685+0000 7f5534c00640 1 -- 172.21.15.42:0/3884976572 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f550c0694f0 msgr2=0x7f550c06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.685+0000 7f5534c00640 1 --2- 172.21.15.42:0/3884976572 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f550c0694f0 0x7f550c06b9b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5520007fd0 tx=0x7f5520009040 comp rx=0 tx=0).stop 2024-05-28T14:09:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.685+0000 7f5534c00640 1 -- 172.21.15.42:0/3884976572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f553006f680 msgr2=0x7f553015e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.685+0000 7f5534c00640 1 --2- 172.21.15.42:0/3884976572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f553006f680 0x7f553015e8e0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f55180ae040 tx=0x7f5518018fe0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.689+0000 7f5534c00640 1 -- 172.21.15.42:0/3884976572 shutdown_connections 2024-05-28T14:09:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.689+0000 7f5534c00640 1 --2- 172.21.15.42:0/3884976572 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f550c0694f0 0x7f550c06b9b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.689+0000 7f5534c00640 1 --2- 172.21.15.42:0/3884976572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55300cd450 0x7f55300c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.689+0000 7f5534c00640 1 --2- 172.21.15.42:0/3884976572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f553006f680 0x7f553015e8e0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.689+0000 7f5534c00640 1 --2- 172.21.15.42:0/3884976572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f553006b9f0 0x7f553015e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.689+0000 7f5534c00640 1 -- 172.21.15.42:0/3884976572 >> 172.21.15.42:0/3884976572 conn(0x7f55300b2d70 msgr2=0x7f55300b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.689+0000 7f5534c00640 1 -- 172.21.15.42:0/3884976572 shutdown_connections 2024-05-28T14:09:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.689+0000 7f5534c00640 1 -- 172.21.15.42:0/3884976572 wait complete. 2024-05-28T14:09:12.691 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T14:09:12.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.857+0000 7fa69f200640 1 Processor -- start 2024-05-28T14:09:12.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.857+0000 7fa69f200640 1 -- start start 2024-05-28T14:09:12.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.861+0000 7fa69f200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa6a00cc9c0 0x7fa6a01413d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:12.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.861+0000 7fa69f200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a00cce30 con 0x7fa6a0141910 2024-05-28T14:09:12.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.861+0000 7fa69f200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a006bb90 con 0x7fa6a01601c0 2024-05-28T14:09:12.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.861+0000 7fa69f200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a006c950 con 0x7fa6a00cc9c0 2024-05-28T14:09:12.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.861+0000 7fa697e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa6a00cc9c0 0x7fa6a01413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:12.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.861+0000 7fa697e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa6a00cc9c0 0x7fa6a01413d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43554/0 (socket says 172.21.15.42:43554) 2024-05-28T14:09:12.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.861+0000 7fa697e00640 1 -- 172.21.15.42:0/580753002 learned_addr learned my addr 172.21.15.42:0/580753002 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:12.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.861+0000 7fa697e00640 1 -- 172.21.15.42:0/580753002 >> v1:172.21.15.63:6789/0 conn(0x7fa6a01601c0 legacy=0x7fa6a01605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:12.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.861+0000 7fa697e00640 1 -- 172.21.15.42:0/580753002 >> v1:172.21.15.42:6789/0 conn(0x7fa6a0141910 legacy=0x7fa6a0141d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:12.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.861+0000 7fa697e00640 1 -- 172.21.15.42:0/580753002 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a006be70 con 0x7fa6a00cc9c0 2024-05-28T14:09:12.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.865+0000 7fa697e00640 1 --2- 172.21.15.42:0/580753002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a00cc9c0 0x7fa6a01413d0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa688009870 tx=0x7fa68801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf7d22a062ff522 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:12.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.865+0000 7fa69d400640 1 -- 172.21.15.42:0/580753002 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6880b0070 con 0x7fa6a00cc9c0 2024-05-28T14:09:12.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.865+0000 7fa69d400640 1 -- 172.21.15.42:0/580753002 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6880ab600 con 0x7fa6a00cc9c0 2024-05-28T14:09:12.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.865+0000 7fa69d400640 1 -- 172.21.15.42:0/580753002 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6880ab9d0 con 0x7fa6a00cc9c0 2024-05-28T14:09:12.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.865+0000 7fa69f200640 1 -- 172.21.15.42:0/580753002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a00cc9c0 msgr2=0x7fa6a01413d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:12.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.865+0000 7fa69f200640 1 --2- 172.21.15.42:0/580753002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a00cc9c0 0x7fa6a01413d0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa688009870 tx=0x7fa68801c4d0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.865+0000 7fa69f200640 1 -- 172.21.15.42:0/580753002 shutdown_connections 2024-05-28T14:09:12.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.865+0000 7fa69f200640 1 --2- 172.21.15.42:0/580753002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a00cc9c0 0x7fa6a01413d0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.865+0000 7fa69f200640 1 -- 172.21.15.42:0/580753002 >> 172.21.15.42:0/580753002 conn(0x7fa6a00d0590 msgr2=0x7fa6a00d09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:12.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.869+0000 7fa69f200640 1 -- 172.21.15.42:0/580753002 shutdown_connections 2024-05-28T14:09:12.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.869+0000 7fa69f200640 1 -- 172.21.15.42:0/580753002 wait complete. 2024-05-28T14:09:12.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.869+0000 7fa69f200640 1 Processor -- start 2024-05-28T14:09:12.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.869+0000 7fa69f200640 1 -- start start 2024-05-28T14:09:12.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.869+0000 7fa69f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a00cc9c0 0x7fa6a00c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:12.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.869+0000 7fa69f200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6a0141910 0x7fa6a013d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa69f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6a01601c0 0x7fa6a00ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa69f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a0160cb0 con 0x7fa6a00cc9c0 2024-05-28T14:09:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa69f200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a006c4c0 con 0x7fa6a01601c0 2024-05-28T14:09:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa69f200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a016ab20 con 0x7fa6a0141910 2024-05-28T14:09:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa69de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a00cc9c0 0x7fa6a00c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa69de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a00cc9c0 0x7fa6a00c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43566/0 (socket says 172.21.15.42:43566) 2024-05-28T14:09:12.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa69de00640 1 -- 172.21.15.42:0/1220608016 learned_addr learned my addr 172.21.15.42:0/1220608016 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:12.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa697e00640 1 --2- 172.21.15.42:0/1220608016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6a0141910 0x7fa6a013d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:12.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa697e00640 1 -- 172.21.15.42:0/1220608016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6a01601c0 msgr2=0x7fa6a00ce160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T14:09:12.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa697e00640 1 --2- 172.21.15.42:0/1220608016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6a01601c0 0x7fa6a00ce160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa697e00640 1 -- 172.21.15.42:0/1220608016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a00cc9c0 msgr2=0x7fa6a00c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:12.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa697e00640 1 --2- 172.21.15.42:0/1220608016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a00cc9c0 0x7fa6a00c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:12.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa697e00640 1 -- 172.21.15.42:0/1220608016 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a0136bf0 con 0x7fa6a0141910 2024-05-28T14:09:12.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa69de00640 1 --2- 172.21.15.42:0/1220608016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a00cc9c0 0x7fa6a00c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:09:12.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa697e00640 1 --2- 172.21.15.42:0/1220608016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6a0141910 0x7fa6a013d310 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa6880ab000 tx=0x7fa688019270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:12.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa696000640 1 -- 172.21.15.42:0/1220608016 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6880b0040 con 0x7fa6a0141910 2024-05-28T14:09:12.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa696000640 1 -- 172.21.15.42:0/1220608016 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6880b4440 con 0x7fa6a0141910 2024-05-28T14:09:12.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa696000640 1 -- 172.21.15.42:0/1220608016 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6880b47f0 con 0x7fa6a0141910 2024-05-28T14:09:12.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa69f200640 1 -- 172.21.15.42:0/1220608016 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a0136ed0 con 0x7fa6a0141910 2024-05-28T14:09:12.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.873+0000 7fa69f200640 1 -- 172.21.15.42:0/1220608016 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a0137770 con 0x7fa6a0141910 2024-05-28T14:09:12.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.877+0000 7fa69f200640 1 -- 172.21.15.42:0/1220608016 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a01370a0 con 0x7fa6a0141910 2024-05-28T14:09:12.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.877+0000 7fa696000640 1 -- 172.21.15.42:0/1220608016 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fa6880bb020 con 0x7fa6a0141910 2024-05-28T14:09:12.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.881+0000 7fa696000640 1 --2- 172.21.15.42:0/1220608016 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa67c0694f0 0x7fa67c06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:12.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.881+0000 7fa696000640 1 -- 172.21.15.42:0/1220608016 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7fa6880dd020 con 0x7fa6a0141910 2024-05-28T14:09:12.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.881+0000 7fa69de00640 1 --2- 172.21.15.42:0/1220608016 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa67c0694f0 0x7fa67c06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:12.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.881+0000 7fa69de00640 1 --2- 172.21.15.42:0/1220608016 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa67c0694f0 0x7fa67c06b9b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa690007fd0 tx=0x7fa690009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:12.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:12.885+0000 7fa696000640 1 -- 172.21.15.42:0/1220608016 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa688016170 con 0x7fa6a0141910 2024-05-28T14:09:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.213+0000 7fa69f200640 1 -- 172.21.15.42:0/1220608016 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa6a004fcf0 con 0x7fa6a0141910 2024-05-28T14:09:13.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.213+0000 7fa696000640 1 -- 172.21.15.42:0/1220608016 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) ==== 80+0+0 (secure 0 0 0) 0x7fa6880f1310 con 0x7fa6a0141910 2024-05-28T14:09:13.198 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T14:09:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.221+0000 7fa69f200640 1 -- 172.21.15.42:0/1220608016 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa67c0694f0 msgr2=0x7fa67c06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.221+0000 7fa69f200640 1 --2- 172.21.15.42:0/1220608016 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa67c0694f0 0x7fa67c06b9b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa690007fd0 tx=0x7fa690009040 comp rx=0 tx=0).stop 2024-05-28T14:09:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.221+0000 7fa69f200640 1 -- 172.21.15.42:0/1220608016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6a0141910 msgr2=0x7fa6a013d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.221+0000 7fa69f200640 1 --2- 172.21.15.42:0/1220608016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6a0141910 0x7fa6a013d310 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa6880ab000 tx=0x7fa688019270 comp rx=0 tx=0).stop 2024-05-28T14:09:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.221+0000 7fa69f200640 1 -- 172.21.15.42:0/1220608016 shutdown_connections 2024-05-28T14:09:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.221+0000 7fa69f200640 1 --2- 172.21.15.42:0/1220608016 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa67c0694f0 0x7fa67c06b9b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.221+0000 7fa69f200640 1 --2- 172.21.15.42:0/1220608016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6a01601c0 0x7fa6a00ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.221+0000 7fa69f200640 1 --2- 172.21.15.42:0/1220608016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6a0141910 0x7fa6a013d310 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.221+0000 7fa69f200640 1 --2- 172.21.15.42:0/1220608016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a00cc9c0 0x7fa6a00c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.221+0000 7fa69f200640 1 -- 172.21.15.42:0/1220608016 >> 172.21.15.42:0/1220608016 conn(0x7fa6a00d0590 msgr2=0x7fa6a006ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.221+0000 7fa69f200640 1 -- 172.21.15.42:0/1220608016 shutdown_connections 2024-05-28T14:09:13.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.221+0000 7fa69f200640 1 -- 172.21.15.42:0/1220608016 wait complete. 2024-05-28T14:09:13.238 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-05-28T14:09:13.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:13.239 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-28T14:09:13.263 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:09:13.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:13.264 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-28T14:09:13.273 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[44218]: fuse finished with error 0 and tester_r 0 2024-05-28T14:09:13.399 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:09:13.399 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:09:13.400 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:09:13.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:09:13.409 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:09:13.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:13.409 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-28T14:09:13.461 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-28T14:09:13.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:09:13.462 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:09:13.462 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:09:13.462 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:09:13.471 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:09:13.471 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:09:13.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:13.471 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-28T14:09:13.522 INFO:teuthology.orchestra.run.smithi042.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-28T14:09:13.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:09:13.523 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:09:13.523 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:09:13.523 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:09:13.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:09:13.532 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:09:13.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:13.532 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-28T14:09:13.580 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-28T14:09:13.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:09:13.581 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:09:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f5570200640 1 Processor -- start 2024-05-28T14:09:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f5570200640 1 -- start start 2024-05-28T14:09:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f5570200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f556815c990 0x7f55680d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f5570200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f55680d06a0 0x7f55680b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f5570200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55680cd300 con 0x7f556809bbd0 2024-05-28T14:09:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f5570200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f556815cd70 con 0x7f55680d06a0 2024-05-28T14:09:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f5570200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f55680cd5e0 con 0x7f556815c990 2024-05-28T14:09:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f556e200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f55680d06a0 0x7f55680b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f556e200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f55680d06a0 0x7f55680b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43582/0 (socket says 172.21.15.42:43582) 2024-05-28T14:09:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f556e200640 1 -- 172.21.15.42:0/977945329 learned_addr learned my addr 172.21.15.42:0/977945329 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:13.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f556d800640 1 --2- 172.21.15.42:0/977945329 >> v2:172.21.15.63:3300/0 conn(0x7f556815c990 0x7f55680d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:13.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f556e200640 1 -- 172.21.15.42:0/977945329 >> v2:172.21.15.63:3300/0 conn(0x7f556815c990 msgr2=0x7f55680d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:13.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f556e200640 1 --2- 172.21.15.42:0/977945329 >> v2:172.21.15.63:3300/0 conn(0x7f556815c990 0x7f55680d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:13.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f556e200640 1 -- 172.21.15.42:0/977945329 >> v1:172.21.15.63:6789/0 conn(0x7f556809bbd0 legacy=0x7f556809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:13.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.741+0000 7f556e200640 1 -- 172.21.15.42:0/977945329 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55680cd8c0 con 0x7f55680d06a0 2024-05-28T14:09:13.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.745+0000 7f556e200640 1 --2- 172.21.15.42:0/977945329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55680d06a0 0x7f55680b53e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5568073350 tx=0x7f556001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7ef0670fb6d67 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:13.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.745+0000 7f5567e00640 1 -- 172.21.15.42:0/977945329 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55600a5880 con 0x7f55680d06a0 2024-05-28T14:09:13.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.745+0000 7f5567e00640 1 -- 172.21.15.42:0/977945329 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55600a5a20 con 0x7f55680d06a0 2024-05-28T14:09:13.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.745+0000 7f5567e00640 1 -- 172.21.15.42:0/977945329 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55600a5df0 con 0x7f55680d06a0 2024-05-28T14:09:13.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 -- 172.21.15.42:0/977945329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55680d06a0 msgr2=0x7f55680b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:13.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 --2- 172.21.15.42:0/977945329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55680d06a0 0x7f55680b53e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5568073350 tx=0x7f556001c9c0 comp rx=0 tx=0).stop 2024-05-28T14:09:13.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 -- 172.21.15.42:0/977945329 shutdown_connections 2024-05-28T14:09:13.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 --2- 172.21.15.42:0/977945329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55680d06a0 0x7f55680b53e0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:13.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 --2- 172.21.15.42:0/977945329 >> v2:172.21.15.63:3300/0 conn(0x7f556815c990 0x7f55680d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:13.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 -- 172.21.15.42:0/977945329 >> 172.21.15.42:0/977945329 conn(0x7f55680b4080 msgr2=0x7f55680b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:13.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 -- 172.21.15.42:0/977945329 shutdown_connections 2024-05-28T14:09:13.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 -- 172.21.15.42:0/977945329 wait complete. 2024-05-28T14:09:13.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 Processor -- start 2024-05-28T14:09:13.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 -- start start 2024-05-28T14:09:13.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55680d06a0 0x7f5568156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:13.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f556815c990 0x7f55681594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:13.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5568159a30 0x7f556815d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f556809b2e0 con 0x7f556815c990 2024-05-28T14:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f556809b160 con 0x7f55680d06a0 2024-05-28T14:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f5570200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f55680b6130 con 0x7f5568159a30 2024-05-28T14:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f556ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f556815c990 0x7f55681594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f556d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55680d06a0 0x7f5568156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f556e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5568159a30 0x7f556815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f556ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f556815c990 0x7f55681594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43588/0 (socket says 172.21.15.42:43588) 2024-05-28T14:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f556e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5568159a30 0x7f556815d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:55694/0 (socket says 172.21.15.42:55694) 2024-05-28T14:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f556e200640 1 -- 172.21.15.42:0/741935954 learned_addr learned my addr 172.21.15.42:0/741935954 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f556d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55680d06a0 0x7f5568156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:54372/0 (socket says 172.21.15.42:54372) 2024-05-28T14:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f556d800640 1 -- 172.21.15.42:0/741935954 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5568159a30 msgr2=0x7f556815d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f556d800640 1 --2- 172.21.15.42:0/741935954 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5568159a30 0x7f556815d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f556d800640 1 -- 172.21.15.42:0/741935954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f556815c990 msgr2=0x7f55681594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f556d800640 1 --2- 172.21.15.42:0/741935954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f556815c990 0x7f55681594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.749+0000 7f556d800640 1 -- 172.21.15.42:0/741935954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5568159ed0 con 0x7f55680d06a0 2024-05-28T14:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.753+0000 7f556ce00640 1 --2- 172.21.15.42:0/741935954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f556815c990 0x7f55681594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.753+0000 7f556d800640 1 --2- 172.21.15.42:0/741935954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55680d06a0 0x7f5568156a60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f555c00e8a0 tx=0x7f555c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.753+0000 7f5565600640 1 -- 172.21.15.42:0/741935954 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c017070 con 0x7f55680d06a0 2024-05-28T14:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.753+0000 7f5565600640 1 -- 172.21.15.42:0/741935954 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f555c00c940 con 0x7f55680d06a0 2024-05-28T14:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.753+0000 7f5570200640 1 -- 172.21.15.42:0/741935954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556815a1b0 con 0x7f55680d06a0 2024-05-28T14:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.753+0000 7f5565600640 1 -- 172.21.15.42:0/741935954 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c00cd10 con 0x7f55680d06a0 2024-05-28T14:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.753+0000 7f5570200640 1 -- 172.21.15.42:0/741935954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556815a770 con 0x7f55680d06a0 2024-05-28T14:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.753+0000 7f5570200640 1 -- 172.21.15.42:0/741935954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5568068a90 con 0x7f55680d06a0 2024-05-28T14:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.753+0000 7f5565600640 1 -- 172.21.15.42:0/741935954 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f555c01a070 con 0x7f55680d06a0 2024-05-28T14:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.753+0000 7f5565600640 1 --2- 172.21.15.42:0/741935954 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55480694f0 0x7f554806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.753+0000 7f5565600640 1 -- 172.21.15.42:0/741935954 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7f555c0859f0 con 0x7f55680d06a0 2024-05-28T14:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.753+0000 7f556ce00640 1 --2- 172.21.15.42:0/741935954 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55480694f0 0x7f554806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.757+0000 7f556ce00640 1 --2- 172.21.15.42:0/741935954 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55480694f0 0x7f554806b9b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f556806ddb0 tx=0x7f5554009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:13.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.757+0000 7f5565600640 1 -- 172.21.15.42:0/741935954 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f555c057060 con 0x7f55680d06a0 2024-05-28T14:09:13.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.961+0000 7f5570200640 1 -- 172.21.15.42:0/741935954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f556815b620 con 0x7f55680d06a0 2024-05-28T14:09:13.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.961+0000 7f5565600640 1 -- 172.21.15.42:0/741935954 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5180 (secure 0 0 0) 0x7f555c05b360 con 0x7f55680d06a0 2024-05-28T14:09:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:13.946 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2618011545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2618011545},{"type":"v1","addr":"172.21.15.63:6835","nonce":2618011545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":7077,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/986242955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":986242955},{"type":"v1","addr":"172.21.15.42:6836","nonce":986242955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":7080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1618433260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1618433260},{"type":"v1","addr":"172.21.15.63:6834","nonce":1618433260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:06:24.814669+0000","modified":"2024-05-28T14:06:27.829347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6837/1199082803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1199082803},{"type":"v1","addr":"172.21.15.42:6837","nonce":1199082803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7083,"qdb_cluster":[7083]},"id":5}]} 2024-05-28T14:09:13.946 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 47 2024-05-28T14:09:13.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.965+0000 7f5570200640 1 -- 172.21.15.42:0/741935954 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55480694f0 msgr2=0x7f554806b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:13.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.965+0000 7f5570200640 1 --2- 172.21.15.42:0/741935954 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55480694f0 0x7f554806b9b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f556806ddb0 tx=0x7f5554009040 comp rx=0 tx=0).stop 2024-05-28T14:09:13.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.965+0000 7f5570200640 1 -- 172.21.15.42:0/741935954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55680d06a0 msgr2=0x7f5568156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:13.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.965+0000 7f5570200640 1 --2- 172.21.15.42:0/741935954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55680d06a0 0x7f5568156a60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f555c00e8a0 tx=0x7f555c00ed60 comp rx=0 tx=0).stop 2024-05-28T14:09:13.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.965+0000 7f5570200640 1 -- 172.21.15.42:0/741935954 shutdown_connections 2024-05-28T14:09:13.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.965+0000 7f5570200640 1 --2- 172.21.15.42:0/741935954 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55480694f0 0x7f554806b9b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:13.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.965+0000 7f5570200640 1 --2- 172.21.15.42:0/741935954 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5568159a30 0x7f556815d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:13.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.965+0000 7f5570200640 1 --2- 172.21.15.42:0/741935954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f556815c990 0x7f55681594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:13.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.965+0000 7f5570200640 1 --2- 172.21.15.42:0/741935954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55680d06a0 0x7f5568156a60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:13.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.965+0000 7f5570200640 1 -- 172.21.15.42:0/741935954 >> 172.21.15.42:0/741935954 conn(0x7f55680b4080 msgr2=0x7f556809aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:13.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.965+0000 7f5570200640 1 -- 172.21.15.42:0/741935954 shutdown_connections 2024-05-28T14:09:13.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:13.965+0000 7f5570200640 1 -- 172.21.15.42:0/741935954 wait complete. 2024-05-28T14:09:13.968 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:09:14.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.125+0000 7fb0bb600640 1 Processor -- start 2024-05-28T14:09:14.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.125+0000 7fb0bb600640 1 -- start start 2024-05-28T14:09:14.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.125+0000 7fb0bb600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb0b415e970 0x7fb0b40d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.125+0000 7fb0bb600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb0b40cd210 0x7fb0b409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.125+0000 7fb0bb600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0b4070740 con 0x7fb0b406ff60 2024-05-28T14:09:14.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.125+0000 7fb0bb600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb0b40d0640 con 0x7fb0b40cd210 2024-05-28T14:09:14.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.125+0000 7fb0bb600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb0b40d1160 con 0x7fb0b415e970 2024-05-28T14:09:14.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.125+0000 7fb0b9600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb0b40cd210 0x7fb0b409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.125+0000 7fb0b9600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb0b40cd210 0x7fb0b409aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43602/0 (socket says 172.21.15.42:43602) 2024-05-28T14:09:14.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.125+0000 7fb0b9600640 1 -- 172.21.15.42:0/1111165168 learned_addr learned my addr 172.21.15.42:0/1111165168 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:14.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.125+0000 7fb0b8c00640 1 --2- 172.21.15.42:0/1111165168 >> v2:172.21.15.63:3300/0 conn(0x7fb0b415e970 0x7fb0b40d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.129+0000 7fb0b9600640 1 -- 172.21.15.42:0/1111165168 >> v2:172.21.15.63:3300/0 conn(0x7fb0b415e970 msgr2=0x7fb0b40d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:14.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.129+0000 7fb0b9600640 1 --2- 172.21.15.42:0/1111165168 >> v2:172.21.15.63:3300/0 conn(0x7fb0b415e970 0x7fb0b40d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.129+0000 7fb0b9600640 1 -- 172.21.15.42:0/1111165168 >> v1:172.21.15.63:6789/0 conn(0x7fb0b406ff60 legacy=0x7fb0b40ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.129+0000 7fb0b9600640 1 -- 172.21.15.42:0/1111165168 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b40d0870 con 0x7fb0b40cd210 2024-05-28T14:09:14.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.129+0000 7fb0b8c00640 1 --2- 172.21.15.42:0/1111165168 >> v2:172.21.15.63:3300/0 conn(0x7fb0b415e970 0x7fb0b40d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:14.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.129+0000 7fb0b9600640 1 --2- 172.21.15.42:0/1111165168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b40cd210 0x7fb0b409aa50 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40732d0 tx=0x7fb0ac01c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bd9986ee5f2009 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:14.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.129+0000 7fb0b3e00640 1 -- 172.21.15.42:0/1111165168 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac0a5530 con 0x7fb0b40cd210 2024-05-28T14:09:14.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.129+0000 7fb0b3e00640 1 -- 172.21.15.42:0/1111165168 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ac0a56d0 con 0x7fb0b40cd210 2024-05-28T14:09:14.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.129+0000 7fb0b3e00640 1 -- 172.21.15.42:0/1111165168 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac0a5aa0 con 0x7fb0b40cd210 2024-05-28T14:09:14.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.129+0000 7fb0bb600640 1 -- 172.21.15.42:0/1111165168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b40cd210 msgr2=0x7fb0b409aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.129+0000 7fb0bb600640 1 --2- 172.21.15.42:0/1111165168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b40cd210 0x7fb0b409aa50 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40732d0 tx=0x7fb0ac01c670 comp rx=0 tx=0).stop 2024-05-28T14:09:14.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.129+0000 7fb0bb600640 1 -- 172.21.15.42:0/1111165168 shutdown_connections 2024-05-28T14:09:14.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.129+0000 7fb0bb600640 1 --2- 172.21.15.42:0/1111165168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b40cd210 0x7fb0b409aa50 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0bb600640 1 --2- 172.21.15.42:0/1111165168 >> v2:172.21.15.63:3300/0 conn(0x7fb0b415e970 0x7fb0b40d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0bb600640 1 -- 172.21.15.42:0/1111165168 >> 172.21.15.42:0/1111165168 conn(0x7fb0b40b3fa0 msgr2=0x7fb0b40b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:14.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0bb600640 1 -- 172.21.15.42:0/1111165168 shutdown_connections 2024-05-28T14:09:14.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0bb600640 1 -- 172.21.15.42:0/1111165168 wait complete. 2024-05-28T14:09:14.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0bb600640 1 Processor -- start 2024-05-28T14:09:14.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0bb600640 1 -- start start 2024-05-28T14:09:14.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0bb600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0b406ff60 0x7fb0b4149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0bb600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b40cd210 0x7fb0b40c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0bb600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0b415e970 0x7fb0b40c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0bb600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b40cd6a0 con 0x7fb0b40cd210 2024-05-28T14:09:14.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0bb600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b4070340 con 0x7fb0b415e970 2024-05-28T14:09:14.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0bb600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0b40c8710 con 0x7fb0b406ff60 2024-05-28T14:09:14.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b40cd210 0x7fb0b40c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b40cd210 0x7fb0b40c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43618/0 (socket says 172.21.15.42:43618) 2024-05-28T14:09:14.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0abe00640 1 -- 172.21.15.42:0/3150100141 learned_addr learned my addr 172.21.15.42:0/3150100141 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:14.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0b9600640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0b415e970 0x7fb0b40c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0b8c00640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0b406ff60 0x7fb0b4149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0abe00640 1 -- 172.21.15.42:0/3150100141 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0b406ff60 msgr2=0x7fb0b4149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0abe00640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0b406ff60 0x7fb0b4149230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0abe00640 1 -- 172.21.15.42:0/3150100141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0b415e970 msgr2=0x7fb0b40c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0abe00640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0b415e970 0x7fb0b40c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0abe00640 1 -- 172.21.15.42:0/3150100141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b414da40 con 0x7fb0b40cd210 2024-05-28T14:09:14.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0b9600640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0b415e970 0x7fb0b40c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:14.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0b8c00640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0b406ff60 0x7fb0b4149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:14.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0abe00640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b40cd210 0x7fb0b40c81d0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00c370 tx=0x7fb09c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:14.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0b1600640 1 -- 172.21.15.42:0/3150100141 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0052d0 con 0x7fb0b40cd210 2024-05-28T14:09:14.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0b1600640 1 -- 172.21.15.42:0/3150100141 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb09c005470 con 0x7fb0b40cd210 2024-05-28T14:09:14.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0bb600640 1 -- 172.21.15.42:0/3150100141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b414dd20 con 0x7fb0b40cd210 2024-05-28T14:09:14.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.133+0000 7fb0b1600640 1 -- 172.21.15.42:0/3150100141 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c007260 con 0x7fb0b40cd210 2024-05-28T14:09:14.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.137+0000 7fb0bb600640 1 -- 172.21.15.42:0/3150100141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b40ca8d0 con 0x7fb0b40cd210 2024-05-28T14:09:14.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.137+0000 7fb0bb600640 1 -- 172.21.15.42:0/3150100141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b4068a90 con 0x7fb0b40cd210 2024-05-28T14:09:14.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.137+0000 7fb0b1600640 1 -- 172.21.15.42:0/3150100141 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fb09c007400 con 0x7fb0b40cd210 2024-05-28T14:09:14.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.137+0000 7fb0b1600640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb0900694f0 0x7fb09006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.137+0000 7fb0b1600640 1 -- 172.21.15.42:0/3150100141 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7fb09c084d40 con 0x7fb0b40cd210 2024-05-28T14:09:14.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.137+0000 7fb0b8c00640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb0900694f0 0x7fb09006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.137+0000 7fb0b8c00640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb0900694f0 0x7fb09006b9b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009980 tx=0x7fb0a4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:14.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.141+0000 7fb0b1600640 1 -- 172.21.15.42:0/3150100141 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb09c056a00 con 0x7fb0b40cd210 2024-05-28T14:09:14.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.309+0000 7fb0bb600640 1 -- 172.21.15.42:0/3150100141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0b40b3d50 con 0x7fb0b40cd210 2024-05-28T14:09:14.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.309+0000 7fb0b1600640 1 -- 172.21.15.42:0/3150100141 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16814 (secure 0 0 0) 0x7fb09c056300 con 0x7fb0b40cd210 2024-05-28T14:09:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:14.294 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":78,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:09:07.614954+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:06:23.241667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-28T14:06:23.769074+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:09:14.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.313+0000 7fb0bb600640 1 -- 172.21.15.42:0/3150100141 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb0900694f0 msgr2=0x7fb09006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.313+0000 7fb0bb600640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb0900694f0 0x7fb09006b9b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009980 tx=0x7fb0a4008040 comp rx=0 tx=0).stop 2024-05-28T14:09:14.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.313+0000 7fb0bb600640 1 -- 172.21.15.42:0/3150100141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b40cd210 msgr2=0x7fb0b40c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.313+0000 7fb0bb600640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b40cd210 0x7fb0b40c81d0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00c370 tx=0x7fb09c00c830 comp rx=0 tx=0).stop 2024-05-28T14:09:14.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.313+0000 7fb0bb600640 1 -- 172.21.15.42:0/3150100141 shutdown_connections 2024-05-28T14:09:14.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.313+0000 7fb0bb600640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb0900694f0 0x7fb09006b9b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.313+0000 7fb0bb600640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0b415e970 0x7fb0b40c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.313+0000 7fb0bb600640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0b40cd210 0x7fb0b40c81d0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.313+0000 7fb0bb600640 1 --2- 172.21.15.42:0/3150100141 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0b406ff60 0x7fb0b4149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.313+0000 7fb0bb600640 1 -- 172.21.15.42:0/3150100141 >> 172.21.15.42:0/3150100141 conn(0x7fb0b40b3fa0 msgr2=0x7fb0b406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:14.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.313+0000 7fb0bb600640 1 -- 172.21.15.42:0/3150100141 shutdown_connections 2024-05-28T14:09:14.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.313+0000 7fb0bb600640 1 -- 172.21.15.42:0/3150100141 wait complete. 2024-05-28T14:09:14.315 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T14:06:23.241667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T14:06:23.769074+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 77, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:09:14.315 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:09:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea47400640 1 Processor -- start 2024-05-28T14:09:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea47400640 1 -- start start 2024-05-28T14:09:14.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea47400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fea4006ee20 0x7fea4006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea47400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fea4015b9b0 0x7fea4015be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea47400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea400afe40 con 0x7fea40144f80 2024-05-28T14:09:14.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea47400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fea4006f7c0 con 0x7fea4015b9b0 2024-05-28T14:09:14.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea47400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fea400724d0 con 0x7fea4006ee20 2024-05-28T14:09:14.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea45400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fea4015b9b0 0x7fea4015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea45400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fea4015b9b0 0x7fea4015be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43620/0 (socket says 172.21.15.42:43620) 2024-05-28T14:09:14.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea45400640 1 -- 172.21.15.42:0/3238682516 learned_addr learned my addr 172.21.15.42:0/3238682516 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:14.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea37e00640 1 --2- 172.21.15.42:0/3238682516 >> v2:172.21.15.63:3300/0 conn(0x7fea4006ee20 0x7fea4006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea45400640 1 -- 172.21.15.42:0/3238682516 >> v2:172.21.15.63:3300/0 conn(0x7fea4006ee20 msgr2=0x7fea4006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:14.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea45400640 1 --2- 172.21.15.42:0/3238682516 >> v2:172.21.15.63:3300/0 conn(0x7fea4006ee20 0x7fea4006f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea45400640 1 -- 172.21.15.42:0/3238682516 >> v1:172.21.15.63:6789/0 conn(0x7fea40144f80 legacy=0x7fea40072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea45400640 1 -- 172.21.15.42:0/3238682516 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4015c340 con 0x7fea4015b9b0 2024-05-28T14:09:14.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.477+0000 7fea37e00640 1 --2- 172.21.15.42:0/3238682516 >> v2:172.21.15.63:3300/0 conn(0x7fea4006ee20 0x7fea4006f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:14.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.481+0000 7fea45400640 1 --2- 172.21.15.42:0/3238682516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4015b9b0 0x7fea4015be00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fea40075500 tx=0x7fea3801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4db8797de15595 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:14.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.481+0000 7fea3fe00640 1 -- 172.21.15.42:0/3238682516 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea380a56c0 con 0x7fea4015b9b0 2024-05-28T14:09:14.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.481+0000 7fea3fe00640 1 -- 172.21.15.42:0/3238682516 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea380a5860 con 0x7fea4015b9b0 2024-05-28T14:09:14.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.481+0000 7fea3fe00640 1 -- 172.21.15.42:0/3238682516 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea380a5c30 con 0x7fea4015b9b0 2024-05-28T14:09:14.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.481+0000 7fea47400640 1 -- 172.21.15.42:0/3238682516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4015b9b0 msgr2=0x7fea4015be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.481+0000 7fea47400640 1 --2- 172.21.15.42:0/3238682516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4015b9b0 0x7fea4015be00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fea40075500 tx=0x7fea3801c9f0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.481+0000 7fea47400640 1 -- 172.21.15.42:0/3238682516 shutdown_connections 2024-05-28T14:09:14.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.481+0000 7fea47400640 1 --2- 172.21.15.42:0/3238682516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4015b9b0 0x7fea4015be00 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.481+0000 7fea47400640 1 --2- 172.21.15.42:0/3238682516 >> v2:172.21.15.63:3300/0 conn(0x7fea4006ee20 0x7fea4006f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.481+0000 7fea47400640 1 -- 172.21.15.42:0/3238682516 >> 172.21.15.42:0/3238682516 conn(0x7fea400b21f0 msgr2=0x7fea400b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:14.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.481+0000 7fea47400640 1 -- 172.21.15.42:0/3238682516 shutdown_connections 2024-05-28T14:09:14.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.481+0000 7fea47400640 1 -- 172.21.15.42:0/3238682516 wait complete. 2024-05-28T14:09:14.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.481+0000 7fea47400640 1 Processor -- start 2024-05-28T14:09:14.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.481+0000 7fea47400640 1 -- start start 2024-05-28T14:09:14.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea47400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4006ee20 0x7fea401363b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea47400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea40144f80 0x7fea40141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea47400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea4015b9b0 0x7fea4015aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea47400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea400b4010 con 0x7fea4006ee20 2024-05-28T14:09:14.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea47400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea40072160 con 0x7fea40144f80 2024-05-28T14:09:14.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea47400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea400b4190 con 0x7fea4015b9b0 2024-05-28T14:09:14.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea44a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4006ee20 0x7fea401363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea44a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4006ee20 0x7fea401363b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43626/0 (socket says 172.21.15.42:43626) 2024-05-28T14:09:14.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea44a00640 1 -- 172.21.15.42:0/1552168742 learned_addr learned my addr 172.21.15.42:0/1552168742 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:14.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea37e00640 1 --2- 172.21.15.42:0/1552168742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea40144f80 0x7fea40141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea45400640 1 --2- 172.21.15.42:0/1552168742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea4015b9b0 0x7fea4015aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea44a00640 1 -- 172.21.15.42:0/1552168742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea4015b9b0 msgr2=0x7fea4015aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea44a00640 1 --2- 172.21.15.42:0/1552168742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea4015b9b0 0x7fea4015aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea44a00640 1 -- 172.21.15.42:0/1552168742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea40144f80 msgr2=0x7fea40141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea44a00640 1 --2- 172.21.15.42:0/1552168742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea40144f80 0x7fea40141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea44a00640 1 -- 172.21.15.42:0/1552168742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea401422f0 con 0x7fea4006ee20 2024-05-28T14:09:14.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea44a00640 1 --2- 172.21.15.42:0/1552168742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4006ee20 0x7fea401363b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fea3000c420 tx=0x7fea3000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:14.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea3d600640 1 -- 172.21.15.42:0/1552168742 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea300052d0 con 0x7fea4006ee20 2024-05-28T14:09:14.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea3d600640 1 -- 172.21.15.42:0/1552168742 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea30005470 con 0x7fea4006ee20 2024-05-28T14:09:14.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea3d600640 1 -- 172.21.15.42:0/1552168742 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30007260 con 0x7fea4006ee20 2024-05-28T14:09:14.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea47400640 1 -- 172.21.15.42:0/1552168742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea401369e0 con 0x7fea4006ee20 2024-05-28T14:09:14.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea47400640 1 -- 172.21.15.42:0/1552168742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea4015ce50 con 0x7fea4006ee20 2024-05-28T14:09:14.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.485+0000 7fea47400640 1 -- 172.21.15.42:0/1552168742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea400738b0 con 0x7fea4006ee20 2024-05-28T14:09:14.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.489+0000 7fea3d600640 1 -- 172.21.15.42:0/1552168742 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fea30007400 con 0x7fea4006ee20 2024-05-28T14:09:14.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.489+0000 7fea3d600640 1 --2- 172.21.15.42:0/1552168742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea1c0694f0 0x7fea1c06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.489+0000 7fea3d600640 1 -- 172.21.15.42:0/1552168742 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7fea30084cc0 con 0x7fea4006ee20 2024-05-28T14:09:14.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.489+0000 7fea37e00640 1 --2- 172.21.15.42:0/1552168742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea1c0694f0 0x7fea1c06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.489+0000 7fea37e00640 1 --2- 172.21.15.42:0/1552168742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea1c0694f0 0x7fea1c06b9b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fea400cdee0 tx=0x7fea28008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:14.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.493+0000 7fea3d600640 1 -- 172.21.15.42:0/1552168742 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fea30056390 con 0x7fea4006ee20 2024-05-28T14:09:14.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.657+0000 7fea47400640 1 -- 172.21.15.42:0/1552168742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fea4004fdd0 con 0x7fea4006ee20 2024-05-28T14:09:14.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.657+0000 7fea3d600640 1 -- 172.21.15.42:0/1552168742 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7fea3005a690 con 0x7fea4006ee20 2024-05-28T14:09:14.642 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:09:14.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.661+0000 7fea47400640 1 -- 172.21.15.42:0/1552168742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea1c0694f0 msgr2=0x7fea1c06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.661+0000 7fea47400640 1 --2- 172.21.15.42:0/1552168742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea1c0694f0 0x7fea1c06b9b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fea400cdee0 tx=0x7fea28008040 comp rx=0 tx=0).stop 2024-05-28T14:09:14.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.661+0000 7fea47400640 1 -- 172.21.15.42:0/1552168742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4006ee20 msgr2=0x7fea401363b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.661+0000 7fea47400640 1 --2- 172.21.15.42:0/1552168742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4006ee20 0x7fea401363b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fea3000c420 tx=0x7fea3000c8e0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.661+0000 7fea47400640 1 -- 172.21.15.42:0/1552168742 shutdown_connections 2024-05-28T14:09:14.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.661+0000 7fea47400640 1 --2- 172.21.15.42:0/1552168742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea1c0694f0 0x7fea1c06b9b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.661+0000 7fea47400640 1 --2- 172.21.15.42:0/1552168742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea4015b9b0 0x7fea4015aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.661+0000 7fea47400640 1 --2- 172.21.15.42:0/1552168742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea40144f80 0x7fea40141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.661+0000 7fea47400640 1 --2- 172.21.15.42:0/1552168742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4006ee20 0x7fea401363b0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.661+0000 7fea47400640 1 -- 172.21.15.42:0/1552168742 >> 172.21.15.42:0/1552168742 conn(0x7fea400b21f0 msgr2=0x7fea400b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:14.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.661+0000 7fea47400640 1 -- 172.21.15.42:0/1552168742 shutdown_connections 2024-05-28T14:09:14.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.661+0000 7fea47400640 1 -- 172.21.15.42:0/1552168742 wait complete. 2024-05-28T14:09:14.664 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:09:14.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.825+0000 7f164ca00640 1 Processor -- start 2024-05-28T14:09:14.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.825+0000 7f164ca00640 1 -- start start 2024-05-28T14:09:14.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.825+0000 7f164ca00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1648071de0 0x7f16480721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.825+0000 7f164ca00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16480ce3b0 0x7f16480ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.825+0000 7f164ca00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1648072700 con 0x7f1648141d10 2024-05-28T14:09:14.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.825+0000 7f164ca00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1648160230 con 0x7f1648071de0 2024-05-28T14:09:14.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.825+0000 7f164ca00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1648160510 con 0x7f16480ce3b0 2024-05-28T14:09:14.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.825+0000 7f1645e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1648071de0 0x7f16480721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.825+0000 7f1645e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1648071de0 0x7f16480721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43640/0 (socket says 172.21.15.42:43640) 2024-05-28T14:09:14.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.825+0000 7f1646800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1648141d10 0x7f16481421a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:57088/0 (socket says 172.21.15.42:57088) 2024-05-28T14:09:14.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.825+0000 7f1645e00640 1 -- 172.21.15.42:0/1314970813 learned_addr learned my addr 172.21.15.42:0/1314970813 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:14.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.825+0000 7f163fe00640 1 --2- 172.21.15.42:0/1314970813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16480ce3b0 0x7f16480ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f1645e00640 1 -- 172.21.15.42:0/1314970813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16480ce3b0 msgr2=0x7f16480ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:14.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f1645e00640 1 --2- 172.21.15.42:0/1314970813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16480ce3b0 0x7f16480ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f1645e00640 1 -- 172.21.15.42:0/1314970813 >> v1:172.21.15.42:6789/0 conn(0x7f1648141d10 legacy=0x7f16481421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f1645400640 1 -- 172.21.15.42:0/1314970813 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1451509753 0 0) 0x7f1648072700 con 0x7f1648141d10 2024-05-28T14:09:14.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f1645e00640 1 -- 172.21.15.42:0/1314970813 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16481607f0 con 0x7f1648071de0 2024-05-28T14:09:14.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f1645e00640 1 --2- 172.21.15.42:0/1314970813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1648071de0 0x7f16480721c0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f1638009870 tx=0x7f163801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512cb55057b5f4a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:14.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f1645400640 1 -- 172.21.15.42:0/1314970813 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16380b0070 con 0x7f1648071de0 2024-05-28T14:09:14.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f1645400640 1 -- 172.21.15.42:0/1314970813 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16380ab600 con 0x7f1648071de0 2024-05-28T14:09:14.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f1645400640 1 -- 172.21.15.42:0/1314970813 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16380ab9d0 con 0x7f1648071de0 2024-05-28T14:09:14.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f164ca00640 1 -- 172.21.15.42:0/1314970813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1648071de0 msgr2=0x7f16480721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f164ca00640 1 --2- 172.21.15.42:0/1314970813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1648071de0 0x7f16480721c0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f1638009870 tx=0x7f163801c440 comp rx=0 tx=0).stop 2024-05-28T14:09:14.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f164ca00640 1 -- 172.21.15.42:0/1314970813 shutdown_connections 2024-05-28T14:09:14.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f164ca00640 1 --2- 172.21.15.42:0/1314970813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16480ce3b0 0x7f16480ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f164ca00640 1 --2- 172.21.15.42:0/1314970813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1648071de0 0x7f16480721c0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.829+0000 7f164ca00640 1 -- 172.21.15.42:0/1314970813 >> 172.21.15.42:0/1314970813 conn(0x7f16480b21f0 msgr2=0x7f16480b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:14.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f164ca00640 1 -- 172.21.15.42:0/1314970813 shutdown_connections 2024-05-28T14:09:14.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f164ca00640 1 -- 172.21.15.42:0/1314970813 wait complete. 2024-05-28T14:09:14.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f164ca00640 1 Processor -- start 2024-05-28T14:09:14.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f164ca00640 1 -- start start 2024-05-28T14:09:14.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f164ca00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1648071de0 0x7f164813aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f164ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480ce3b0 0x7f1648159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f164ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1648141d10 0x7f164813dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f164ca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1648169e30 con 0x7f16480ce3b0 2024-05-28T14:09:14.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f164ca00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1648169cb0 con 0x7f1648141d10 2024-05-28T14:09:14.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f164ca00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1648169fb0 con 0x7f1648071de0 2024-05-28T14:09:14.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f163fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480ce3b0 0x7f1648159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f163fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480ce3b0 0x7f1648159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43646/0 (socket says 172.21.15.42:43646) 2024-05-28T14:09:14.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f163fe00640 1 -- 172.21.15.42:0/2792368389 learned_addr learned my addr 172.21.15.42:0/2792368389 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:14.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f1645e00640 1 --2- 172.21.15.42:0/2792368389 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1648071de0 0x7f164813aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f1646800640 1 --2- 172.21.15.42:0/2792368389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1648141d10 0x7f164813dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f163fe00640 1 -- 172.21.15.42:0/2792368389 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1648071de0 msgr2=0x7f164813aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f163fe00640 1 --2- 172.21.15.42:0/2792368389 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1648071de0 0x7f164813aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f163fe00640 1 -- 172.21.15.42:0/2792368389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1648141d10 msgr2=0x7f164813dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f163fe00640 1 --2- 172.21.15.42:0/2792368389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1648141d10 0x7f164813dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f163fe00640 1 -- 172.21.15.42:0/2792368389 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164815a1d0 con 0x7f16480ce3b0 2024-05-28T14:09:14.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f163fe00640 1 --2- 172.21.15.42:0/2792368389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480ce3b0 0x7f1648159730 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f163000d5f0 tx=0x7f163000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:14.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f163e000640 1 -- 172.21.15.42:0/2792368389 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16300125f0 con 0x7f16480ce3b0 2024-05-28T14:09:14.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f163e000640 1 -- 172.21.15.42:0/2792368389 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1630019070 con 0x7f16480ce3b0 2024-05-28T14:09:14.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f163e000640 1 -- 172.21.15.42:0/2792368389 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16300129b0 con 0x7f16480ce3b0 2024-05-28T14:09:14.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.833+0000 7f164ca00640 1 -- 172.21.15.42:0/2792368389 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f164815e2e0 con 0x7f16480ce3b0 2024-05-28T14:09:14.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.837+0000 7f164ca00640 1 -- 172.21.15.42:0/2792368389 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f164815e8a0 con 0x7f16480ce3b0 2024-05-28T14:09:14.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.837+0000 7f164ca00640 1 -- 172.21.15.42:0/2792368389 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16480738b0 con 0x7f16480ce3b0 2024-05-28T14:09:14.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.837+0000 7f163e000640 1 -- 172.21.15.42:0/2792368389 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f1630012b50 con 0x7f16480ce3b0 2024-05-28T14:09:14.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.837+0000 7f163e000640 1 --2- 172.21.15.42:0/2792368389 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16240694f0 0x7f162406b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:14.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.837+0000 7f163e000640 1 -- 172.21.15.42:0/2792368389 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7f1630085b80 con 0x7f16480ce3b0 2024-05-28T14:09:14.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.837+0000 7f1645e00640 1 --2- 172.21.15.42:0/2792368389 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16240694f0 0x7f162406b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:14.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.837+0000 7f1645e00640 1 --2- 172.21.15.42:0/2792368389 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16240694f0 0x7f162406b9b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f16380ad000 tx=0x7f16380a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:14.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:14.841+0000 7f163e000640 1 -- 172.21.15.42:0/2792368389 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f16300571f0 con 0x7f16480ce3b0 2024-05-28T14:09:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.005+0000 7f164ca00640 1 -- 172.21.15.42:0/2792368389 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1648072ed0 con 0x7f16480ce3b0 2024-05-28T14:09:14.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.009+0000 7f163e000640 1 -- 172.21.15.42:0/2792368389 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f163005b4f0 con 0x7f16480ce3b0 2024-05-28T14:09:14.991 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:09:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.009+0000 7f164ca00640 1 -- 172.21.15.42:0/2792368389 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16240694f0 msgr2=0x7f162406b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.009+0000 7f164ca00640 1 --2- 172.21.15.42:0/2792368389 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16240694f0 0x7f162406b9b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f16380ad000 tx=0x7f16380a6000 comp rx=0 tx=0).stop 2024-05-28T14:09:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.009+0000 7f164ca00640 1 -- 172.21.15.42:0/2792368389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480ce3b0 msgr2=0x7f1648159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.009+0000 7f164ca00640 1 --2- 172.21.15.42:0/2792368389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480ce3b0 0x7f1648159730 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f163000d5f0 tx=0x7f163000dab0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.009+0000 7f164ca00640 1 -- 172.21.15.42:0/2792368389 shutdown_connections 2024-05-28T14:09:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.009+0000 7f164ca00640 1 --2- 172.21.15.42:0/2792368389 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16240694f0 0x7f162406b9b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.009+0000 7f164ca00640 1 --2- 172.21.15.42:0/2792368389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1648141d10 0x7f164813dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.009+0000 7f164ca00640 1 --2- 172.21.15.42:0/2792368389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480ce3b0 0x7f1648159730 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.009+0000 7f164ca00640 1 --2- 172.21.15.42:0/2792368389 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1648071de0 0x7f164813aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.009+0000 7f164ca00640 1 -- 172.21.15.42:0/2792368389 >> 172.21.15.42:0/2792368389 conn(0x7f16480b21f0 msgr2=0x7f164815a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.009+0000 7f164ca00640 1 -- 172.21.15.42:0/2792368389 shutdown_connections 2024-05-28T14:09:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.009+0000 7f164ca00640 1 -- 172.21.15.42:0/2792368389 wait complete. 2024-05-28T14:09:15.013 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:09:15.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.173+0000 7fd7dce00640 1 Processor -- start 2024-05-28T14:09:15.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.173+0000 7fd7dce00640 1 -- start start 2024-05-28T14:09:15.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7dce00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7d8071de0 0x7fd7d80721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7dce00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd7d80ce3b0 0x7fd7d80ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7dce00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7d8072700 con 0x7fd7d8141d10 2024-05-28T14:09:15.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7dce00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd7d8160230 con 0x7fd7d80ce3b0 2024-05-28T14:09:15.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7dce00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d8160510 con 0x7fd7d8071de0 2024-05-28T14:09:15.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d6800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd7d8141d10 0x7fd7d81421a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:57098/0 (socket says 172.21.15.42:57098) 2024-05-28T14:09:15.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d6800640 1 -- 172.21.15.42:0/1704956264 learned_addr learned my addr 172.21.15.42:0/1704956264 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:15.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5e00640 1 --2- 172.21.15.42:0/1704956264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7d8071de0 0x7fd7d80721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7cd400640 1 --2- 172.21.15.42:0/1704956264 >> v2:172.21.15.63:3300/0 conn(0x7fd7d80ce3b0 0x7fd7d80ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 -- 172.21.15.42:0/1704956264 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3358762115 0 0) 0x7fd7d8072700 con 0x7fd7d8141d10 2024-05-28T14:09:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 -- 172.21.15.42:0/1704956264 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7bc003530 con 0x7fd7d8141d10 2024-05-28T14:09:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 -- 172.21.15.42:0/1704956264 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2400812794 0 0) 0x7fd7bc003530 con 0x7fd7d8141d10 2024-05-28T14:09:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 -- 172.21.15.42:0/1704956264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7d8071de0 msgr2=0x7fd7d80721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 --2- 172.21.15.42:0/1704956264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7d8071de0 0x7fd7d80721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 -- 172.21.15.42:0/1704956264 >> v2:172.21.15.63:3300/0 conn(0x7fd7d80ce3b0 msgr2=0x7fd7d80ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 --2- 172.21.15.42:0/1704956264 >> v2:172.21.15.63:3300/0 conn(0x7fd7d80ce3b0 0x7fd7d80ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 -- 172.21.15.42:0/1704956264 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d81607f0 con 0x7fd7d8141d10 2024-05-28T14:09:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 -- 172.21.15.42:0/1704956264 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fd7d0001a00 con 0x7fd7d8141d10 2024-05-28T14:09:15.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 -- 172.21.15.42:0/1704956264 >> v1:172.21.15.42:6789/0 conn(0x7fd7d8141d10 legacy=0x7fd7d81421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 --2- 172.21.15.42:0/1704956264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7bc0039c0 0x7fd7bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 --2- 172.21.15.42:0/1704956264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7d8071de0 0x7fd7bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 --2- 172.21.15.42:0/1704956264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7bc004d50 0x7fd7bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 -- 172.21.15.42:0/1704956264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8160230 con 0x7fd7bc0039c0 2024-05-28T14:09:15.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 -- 172.21.15.42:0/1704956264 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8160510 con 0x7fd7bc004d50 2024-05-28T14:09:15.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5400640 1 -- 172.21.15.42:0/1704956264 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd7bc00ba50 con 0x7fd7d8071de0 2024-05-28T14:09:15.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5e00640 1 --2- 172.21.15.42:0/1704956264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7bc0039c0 0x7fd7bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d6800640 1 --2- 172.21.15.42:0/1704956264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7bc004d50 0x7fd7bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7cd400640 1 --2- 172.21.15.42:0/1704956264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7d8071de0 0x7fd7bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5e00640 1 -- 172.21.15.42:0/1704956264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7d8071de0 msgr2=0x7fd7bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5e00640 1 --2- 172.21.15.42:0/1704956264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7d8071de0 0x7fd7bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5e00640 1 -- 172.21.15.42:0/1704956264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7bc004d50 msgr2=0x7fd7bc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5e00640 1 --2- 172.21.15.42:0/1704956264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7bc004d50 0x7fd7bc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d5e00640 1 -- 172.21.15.42:0/1704956264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7bc003530 con 0x7fd7bc0039c0 2024-05-28T14:09:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.177+0000 7fd7d6800640 1 --2- 172.21.15.42:0/1704956264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7bc004d50 0x7fd7bc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.181+0000 7fd7d5e00640 1 --2- 172.21.15.42:0/1704956264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7bc0039c0 0x7fd7bc003de0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009ae0 tx=0x7fd7c801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5457f56d667b6a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.181+0000 7fd7d5400640 1 -- 172.21.15.42:0/1704956264 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80096b0 con 0x7fd7bc0039c0 2024-05-28T14:09:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.181+0000 7fd7d5400640 1 -- 172.21.15.42:0/1704956264 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c80a6450 con 0x7fd7bc0039c0 2024-05-28T14:09:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.181+0000 7fd7d5400640 1 -- 172.21.15.42:0/1704956264 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80a6840 con 0x7fd7bc0039c0 2024-05-28T14:09:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.181+0000 7fd7dce00640 1 -- 172.21.15.42:0/1704956264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7bc0039c0 msgr2=0x7fd7bc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.181+0000 7fd7dce00640 1 --2- 172.21.15.42:0/1704956264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7bc0039c0 0x7fd7bc003de0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009ae0 tx=0x7fd7c801ca70 comp rx=0 tx=0).stop 2024-05-28T14:09:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.181+0000 7fd7d6800640 1 -- 172.21.15.42:0/1704956264 reap_dead start 2024-05-28T14:09:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.181+0000 7fd7dce00640 1 -- 172.21.15.42:0/1704956264 shutdown_connections 2024-05-28T14:09:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.181+0000 7fd7dce00640 1 -- 172.21.15.42:0/1704956264 >> 172.21.15.42:0/1704956264 conn(0x7fd7d80b21f0 msgr2=0x7fd7d80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.181+0000 7fd7dce00640 1 -- 172.21.15.42:0/1704956264 shutdown_connections 2024-05-28T14:09:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.181+0000 7fd7dce00640 1 -- 172.21.15.42:0/1704956264 wait complete. 2024-05-28T14:09:15.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.181+0000 7fd7dce00640 1 Processor -- start 2024-05-28T14:09:15.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.181+0000 7fd7dce00640 1 -- start start 2024-05-28T14:09:15.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7dce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7bc0039c0 0x7fd7d81fb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7dce00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7d80b35f0 0x7fd7d8139110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7dce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7d80b4d90 0x7fd7d8163470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7dce00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8072340 con 0x7fd7bc0039c0 2024-05-28T14:09:15.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7dce00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d80721c0 con 0x7fd7d80b4d90 2024-05-28T14:09:15.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7dce00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d80724c0 con 0x7fd7d80b35f0 2024-05-28T14:09:15.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7d5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7bc0039c0 0x7fd7d81fb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7d5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7bc0039c0 0x7fd7d81fb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43670/0 (socket says 172.21.15.42:43670) 2024-05-28T14:09:15.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7d5e00640 1 -- 172.21.15.42:0/4126055176 learned_addr learned my addr 172.21.15.42:0/4126055176 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:15.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7cd400640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7d80b35f0 0x7fd7d8139110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7d6800640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7d80b4d90 0x7fd7d8163470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7d5e00640 1 -- 172.21.15.42:0/4126055176 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7d80b35f0 msgr2=0x7fd7d8139110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7d5e00640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7d80b35f0 0x7fd7d8139110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7d5e00640 1 -- 172.21.15.42:0/4126055176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7d80b4d90 msgr2=0x7fd7d8163470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7d5e00640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7d80b4d90 0x7fd7d8163470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7d5e00640 1 -- 172.21.15.42:0/4126055176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d80b3a70 con 0x7fd7bc0039c0 2024-05-28T14:09:15.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7cd400640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7d80b35f0 0x7fd7d8139110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:15.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7d5e00640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7bc0039c0 0x7fd7d81fb140 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fd7c801cf50 tx=0x7fd7bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:15.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7cea00640 1 -- 172.21.15.42:0/4126055176 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80be070 con 0x7fd7bc0039c0 2024-05-28T14:09:15.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7cea00640 1 -- 172.21.15.42:0/4126055176 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c800f000 con 0x7fd7bc0039c0 2024-05-28T14:09:15.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7dce00640 1 -- 172.21.15.42:0/4126055176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d80b3cf0 con 0x7fd7bc0039c0 2024-05-28T14:09:15.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7cea00640 1 -- 172.21.15.42:0/4126055176 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80096b0 con 0x7fd7bc0039c0 2024-05-28T14:09:15.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7dce00640 1 -- 172.21.15.42:0/4126055176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d80b42b0 con 0x7fd7bc0039c0 2024-05-28T14:09:15.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7dce00640 1 -- 172.21.15.42:0/4126055176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d80738b0 con 0x7fd7bc0039c0 2024-05-28T14:09:15.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.185+0000 7fd7cea00640 1 -- 172.21.15.42:0/4126055176 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fd7c80c1070 con 0x7fd7bc0039c0 2024-05-28T14:09:15.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.189+0000 7fd7cea00640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7b4069520 0x7fd7b406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.189+0000 7fd7cea00640 1 -- 172.21.15.42:0/4126055176 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7fd7c811ee20 con 0x7fd7bc0039c0 2024-05-28T14:09:15.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.189+0000 7fd7cd400640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7b4069520 0x7fd7b406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.189+0000 7fd7cd400640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7b4069520 0x7fd7b406b9e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd7d806fde0 tx=0x7fd7c0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:15.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.193+0000 7fd7cea00640 1 -- 172.21.15.42:0/4126055176 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd7d80b3a70 con 0x7fd7bc0039c0 2024-05-28T14:09:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.353+0000 7fd7dce00640 1 -- 172.21.15.42:0/4126055176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd7d804fdd0 con 0x7fd7bc0039c0 2024-05-28T14:09:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.357+0000 7fd7cea00640 1 -- 172.21.15.42:0/4126055176 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7fd7c80ac020 con 0x7fd7bc0039c0 2024-05-28T14:09:15.339 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:09:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.357+0000 7fd7dce00640 1 -- 172.21.15.42:0/4126055176 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7b4069520 msgr2=0x7fd7b406b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.357+0000 7fd7dce00640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7b4069520 0x7fd7b406b9e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd7d806fde0 tx=0x7fd7c0008040 comp rx=0 tx=0).stop 2024-05-28T14:09:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.357+0000 7fd7dce00640 1 -- 172.21.15.42:0/4126055176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7bc0039c0 msgr2=0x7fd7d81fb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.357+0000 7fd7dce00640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7bc0039c0 0x7fd7d81fb140 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fd7c801cf50 tx=0x7fd7bc005840 comp rx=0 tx=0).stop 2024-05-28T14:09:15.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.357+0000 7fd7dce00640 1 -- 172.21.15.42:0/4126055176 shutdown_connections 2024-05-28T14:09:15.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.357+0000 7fd7dce00640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd7b4069520 0x7fd7b406b9e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.357+0000 7fd7dce00640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7d80b4d90 0x7fd7d8163470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.357+0000 7fd7dce00640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7d80b35f0 0x7fd7d8139110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.357+0000 7fd7dce00640 1 --2- 172.21.15.42:0/4126055176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7bc0039c0 0x7fd7d81fb140 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.357+0000 7fd7dce00640 1 -- 172.21.15.42:0/4126055176 >> 172.21.15.42:0/4126055176 conn(0x7fd7d80b21f0 msgr2=0x7fd7d8141130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:15.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.357+0000 7fd7dce00640 1 -- 172.21.15.42:0/4126055176 shutdown_connections 2024-05-28T14:09:15.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.357+0000 7fd7dce00640 1 -- 172.21.15.42:0/4126055176 wait complete. 2024-05-28T14:09:15.362 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:09:15.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.517+0000 7fa56c600640 1 Processor -- start 2024-05-28T14:09:15.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.517+0000 7fa56c600640 1 -- start start 2024-05-28T14:09:15.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.521+0000 7fa56c600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56406f660 0x7fa564073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.521+0000 7fa56c600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa56415ba40 0x7fa5640d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.521+0000 7fa56c600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa56415f280 con 0x7fa564072dd0 2024-05-28T14:09:15.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.521+0000 7fa56c600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa56415f030 con 0x7fa56415ba40 2024-05-28T14:09:15.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.521+0000 7fa56c600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa56415f560 con 0x7fa56406f660 2024-05-28T14:09:15.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.521+0000 7fa569200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa56415ba40 0x7fa5640d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.521+0000 7fa569200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa56415ba40 0x7fa5640d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43676/0 (socket says 172.21.15.42:43676) 2024-05-28T14:09:15.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.521+0000 7fa569200640 1 -- 172.21.15.42:0/2110943587 learned_addr learned my addr 172.21.15.42:0/2110943587 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:15.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.521+0000 7fa569c00640 1 --2- 172.21.15.42:0/2110943587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56406f660 0x7fa564073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.521+0000 7fa569200640 1 -- 172.21.15.42:0/2110943587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56406f660 msgr2=0x7fa564073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:15.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.521+0000 7fa569200640 1 --2- 172.21.15.42:0/2110943587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56406f660 0x7fa564073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.521+0000 7fa569200640 1 -- 172.21.15.42:0/2110943587 >> v1:172.21.15.63:6789/0 conn(0x7fa564072dd0 legacy=0x7fa5640b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.521+0000 7fa569200640 1 -- 172.21.15.42:0/2110943587 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56415f840 con 0x7fa56415ba40 2024-05-28T14:09:15.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.525+0000 7fa569200640 1 --2- 172.21.15.42:0/2110943587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa56415ba40 0x7fa5640d06f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa55000bb10 tx=0x7fa55001eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cb9437a1b3e97d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.525+0000 7fa563e00640 1 -- 172.21.15.42:0/2110943587 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5500b4070 con 0x7fa56415ba40 2024-05-28T14:09:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.525+0000 7fa563e00640 1 -- 172.21.15.42:0/2110943587 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5500a7ac0 con 0x7fa56415ba40 2024-05-28T14:09:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.525+0000 7fa563e00640 1 -- 172.21.15.42:0/2110943587 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5500af440 con 0x7fa56415ba40 2024-05-28T14:09:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.525+0000 7fa56c600640 1 -- 172.21.15.42:0/2110943587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa56415ba40 msgr2=0x7fa5640d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.525+0000 7fa56c600640 1 --2- 172.21.15.42:0/2110943587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa56415ba40 0x7fa5640d06f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa55000bb10 tx=0x7fa55001eae0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.525+0000 7fa56c600640 1 -- 172.21.15.42:0/2110943587 shutdown_connections 2024-05-28T14:09:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.525+0000 7fa56c600640 1 --2- 172.21.15.42:0/2110943587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa56415ba40 0x7fa5640d06f0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.525+0000 7fa56c600640 1 --2- 172.21.15.42:0/2110943587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56406f660 0x7fa564073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.525+0000 7fa56c600640 1 -- 172.21.15.42:0/2110943587 >> 172.21.15.42:0/2110943587 conn(0x7fa56406bdf0 msgr2=0x7fa56406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.525+0000 7fa56c600640 1 -- 172.21.15.42:0/2110943587 shutdown_connections 2024-05-28T14:09:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.525+0000 7fa56c600640 1 -- 172.21.15.42:0/2110943587 wait complete. 2024-05-28T14:09:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.525+0000 7fa56c600640 1 Processor -- start 2024-05-28T14:09:15.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.525+0000 7fa56c600640 1 -- start start 2024-05-28T14:09:15.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56c600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56406f660 0x7fa56415a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56c600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa564072dd0 0x7fa5641490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56c600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa56414f0f0 0x7fa564149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56c600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa5640b3460 con 0x7fa56414f0f0 2024-05-28T14:09:15.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56c600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa5640b32e0 con 0x7fa56406f660 2024-05-28T14:09:15.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56c600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa5640b35e0 con 0x7fa564072dd0 2024-05-28T14:09:15.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa56414f0f0 0x7fa564149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa569c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56406f660 0x7fa56415a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa569200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa564072dd0 0x7fa5641490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa56414f0f0 0x7fa564149630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43686/0 (socket says 172.21.15.42:43686) 2024-05-28T14:09:15.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56a600640 1 -- 172.21.15.42:0/2109519719 learned_addr learned my addr 172.21.15.42:0/2109519719 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:15.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56a600640 1 -- 172.21.15.42:0/2109519719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa564072dd0 msgr2=0x7fa5641490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56a600640 1 --2- 172.21.15.42:0/2109519719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa564072dd0 0x7fa5641490f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56a600640 1 -- 172.21.15.42:0/2109519719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56406f660 msgr2=0x7fa56415a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56a600640 1 --2- 172.21.15.42:0/2109519719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56406f660 0x7fa56415a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56a600640 1 -- 172.21.15.42:0/2109519719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56414e5b0 con 0x7fa56414f0f0 2024-05-28T14:09:15.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56a600640 1 --2- 172.21.15.42:0/2109519719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa56414f0f0 0x7fa564149630 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00c770 tx=0x7fa55c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:15.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa561600640 1 -- 172.21.15.42:0/2109519719 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c003b80 con 0x7fa56414f0f0 2024-05-28T14:09:15.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa561600640 1 -- 172.21.15.42:0/2109519719 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c003d20 con 0x7fa56414f0f0 2024-05-28T14:09:15.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa561600640 1 -- 172.21.15.42:0/2109519719 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00aa90 con 0x7fa56414f0f0 2024-05-28T14:09:15.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56c600640 1 -- 172.21.15.42:0/2109519719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56414e890 con 0x7fa56414f0f0 2024-05-28T14:09:15.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56c600640 1 -- 172.21.15.42:0/2109519719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5640cd080 con 0x7fa56414f0f0 2024-05-28T14:09:15.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa56c600640 1 -- 172.21.15.42:0/2109519719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa564068a90 con 0x7fa56414f0f0 2024-05-28T14:09:15.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.529+0000 7fa561600640 1 -- 172.21.15.42:0/2109519719 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fa55c00ac30 con 0x7fa56414f0f0 2024-05-28T14:09:15.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.533+0000 7fa561600640 1 --2- 172.21.15.42:0/2109519719 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5440694f0 0x7fa54406b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.533+0000 7fa561600640 1 -- 172.21.15.42:0/2109519719 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7fa55c085660 con 0x7fa56414f0f0 2024-05-28T14:09:15.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.533+0000 7fa569c00640 1 --2- 172.21.15.42:0/2109519719 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5440694f0 0x7fa54406b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.533+0000 7fa569c00640 1 --2- 172.21.15.42:0/2109519719 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5440694f0 0x7fa54406b9b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa558009b60 tx=0x7fa558009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:15.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.537+0000 7fa561600640 1 -- 172.21.15.42:0/2109519719 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa55c056cd0 con 0x7fa56414f0f0 2024-05-28T14:09:15.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.737+0000 7fa56c600640 1 -- 172.21.15.42:0/2109519719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa564072bb0 con 0x7fa56414f0f0 2024-05-28T14:09:15.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.741+0000 7fa561600640 1 -- 172.21.15.42:0/2109519719 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5180 (secure 0 0 0) 0x7fa55c05afd0 con 0x7fa56414f0f0 2024-05-28T14:09:15.723 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:15.723 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2618011545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2618011545},{"type":"v1","addr":"172.21.15.63:6835","nonce":2618011545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":7077,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/986242955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":986242955},{"type":"v1","addr":"172.21.15.42:6836","nonce":986242955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":7080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1618433260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1618433260},{"type":"v1","addr":"172.21.15.63:6834","nonce":1618433260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:06:24.814669+0000","modified":"2024-05-28T14:06:27.829347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6837/1199082803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1199082803},{"type":"v1","addr":"172.21.15.42:6837","nonce":1199082803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7083,"qdb_cluster":[7083]},"id":5}]} 2024-05-28T14:09:15.723 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 47 2024-05-28T14:09:15.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.741+0000 7fa56c600640 1 -- 172.21.15.42:0/2109519719 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5440694f0 msgr2=0x7fa54406b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.741+0000 7fa56c600640 1 --2- 172.21.15.42:0/2109519719 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5440694f0 0x7fa54406b9b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa558009b60 tx=0x7fa558009190 comp rx=0 tx=0).stop 2024-05-28T14:09:15.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.741+0000 7fa56c600640 1 -- 172.21.15.42:0/2109519719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa56414f0f0 msgr2=0x7fa564149630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.741+0000 7fa56c600640 1 --2- 172.21.15.42:0/2109519719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa56414f0f0 0x7fa564149630 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00c770 tx=0x7fa55c00cc30 comp rx=0 tx=0).stop 2024-05-28T14:09:15.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.741+0000 7fa56c600640 1 -- 172.21.15.42:0/2109519719 shutdown_connections 2024-05-28T14:09:15.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.741+0000 7fa56c600640 1 --2- 172.21.15.42:0/2109519719 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5440694f0 0x7fa54406b9b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.741+0000 7fa56c600640 1 --2- 172.21.15.42:0/2109519719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa56414f0f0 0x7fa564149630 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.741+0000 7fa56c600640 1 --2- 172.21.15.42:0/2109519719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa564072dd0 0x7fa5641490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.741+0000 7fa56c600640 1 --2- 172.21.15.42:0/2109519719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56406f660 0x7fa56415a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.741+0000 7fa56c600640 1 -- 172.21.15.42:0/2109519719 >> 172.21.15.42:0/2109519719 conn(0x7fa56406bdf0 msgr2=0x7fa564152fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:15.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.741+0000 7fa56c600640 1 -- 172.21.15.42:0/2109519719 shutdown_connections 2024-05-28T14:09:15.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.741+0000 7fa56c600640 1 -- 172.21.15.42:0/2109519719 wait complete. 2024-05-28T14:09:15.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:09:15.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.901+0000 7f802a000640 1 Processor -- start 2024-05-28T14:09:15.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.901+0000 7f802a000640 1 -- start start 2024-05-28T14:09:15.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.905+0000 7f802a000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8024153320 0x7f80241609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.905+0000 7f802a000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f80240b4b30 0x7f80240aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.905+0000 7f802a000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80240af7f0 con 0x7f80240af2e0 2024-05-28T14:09:15.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.905+0000 7f802a000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f802415ffa0 con 0x7f8024153320 2024-05-28T14:09:15.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.905+0000 7f802a000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f80240afad0 con 0x7f80240b4b30 2024-05-28T14:09:15.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.905+0000 7f8028c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8024153320 0x7f80241609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.905+0000 7f8028c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8024153320 0x7f80241609f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43700/0 (socket says 172.21.15.42:43700) 2024-05-28T14:09:15.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.905+0000 7f8028c00640 1 -- 172.21.15.42:0/2383503809 learned_addr learned my addr 172.21.15.42:0/2383503809 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:15.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.905+0000 7f801be00640 1 --2- 172.21.15.42:0/2383503809 >> v2:172.21.15.63:3300/0 conn(0x7f80240b4b30 0x7f80240aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.905+0000 7f8028c00640 1 -- 172.21.15.42:0/2383503809 >> v2:172.21.15.63:3300/0 conn(0x7f80240b4b30 msgr2=0x7f80240aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:15.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.905+0000 7f8028c00640 1 --2- 172.21.15.42:0/2383503809 >> v2:172.21.15.63:3300/0 conn(0x7f80240b4b30 0x7f80240aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.905+0000 7f8028c00640 1 -- 172.21.15.42:0/2383503809 >> v1:172.21.15.63:6789/0 conn(0x7f80240af2e0 legacy=0x7f80240d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:15.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.905+0000 7f8028c00640 1 -- 172.21.15.42:0/2383503809 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80241601f0 con 0x7f8024153320 2024-05-28T14:09:15.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.905+0000 7f801be00640 1 --2- 172.21.15.42:0/2383503809 >> v2:172.21.15.63:3300/0 conn(0x7f80240b4b30 0x7f80240aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:15.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.909+0000 7f8028c00640 1 --2- 172.21.15.42:0/2383503809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8024153320 0x7f80241609f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f802406ff30 tx=0x7f801401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d5d58d5a17e100 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:15.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.909+0000 7f8023e00640 1 -- 172.21.15.42:0/2383503809 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140aa430 con 0x7f8024153320 2024-05-28T14:09:15.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.909+0000 7f8023e00640 1 -- 172.21.15.42:0/2383503809 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80140af070 con 0x7f8024153320 2024-05-28T14:09:15.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.909+0000 7f8023e00640 1 -- 172.21.15.42:0/2383503809 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140aa7f0 con 0x7f8024153320 2024-05-28T14:09:15.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.909+0000 7f802a000640 1 -- 172.21.15.42:0/2383503809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8024153320 msgr2=0x7f80241609f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.909+0000 7f802a000640 1 --2- 172.21.15.42:0/2383503809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8024153320 0x7f80241609f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f802406ff30 tx=0x7f801401c4b0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.909+0000 7f802a000640 1 -- 172.21.15.42:0/2383503809 shutdown_connections 2024-05-28T14:09:15.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.909+0000 7f802a000640 1 --2- 172.21.15.42:0/2383503809 >> v2:172.21.15.63:3300/0 conn(0x7f80240b4b30 0x7f80240aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.909+0000 7f802a000640 1 --2- 172.21.15.42:0/2383503809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8024153320 0x7f80241609f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.909+0000 7f802a000640 1 -- 172.21.15.42:0/2383503809 >> 172.21.15.42:0/2383503809 conn(0x7f802406bbc0 msgr2=0x7f802406bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:15.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.909+0000 7f802a000640 1 -- 172.21.15.42:0/2383503809 shutdown_connections 2024-05-28T14:09:15.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.909+0000 7f802a000640 1 -- 172.21.15.42:0/2383503809 wait complete. 2024-05-28T14:09:15.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.909+0000 7f802a000640 1 Processor -- start 2024-05-28T14:09:15.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.909+0000 7f802a000640 1 -- start start 2024-05-28T14:09:15.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f802a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80240af2e0 0x7f802414f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f802a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80240b4b30 0x7f802406f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f802a000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8024153320 0x7f8024151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f802a000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f802415d1c0 con 0x7f80240b4b30 2024-05-28T14:09:15.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f802a000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80240d0c80 con 0x7f80240af2e0 2024-05-28T14:09:15.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f802a000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f802415d340 con 0x7f8024153320 2024-05-28T14:09:15.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f801be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80240b4b30 0x7f802406f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f801be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80240b4b30 0x7f802406f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43714/0 (socket says 172.21.15.42:43714) 2024-05-28T14:09:15.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f801be00640 1 -- 172.21.15.42:0/4239260349 learned_addr learned my addr 172.21.15.42:0/4239260349 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:15.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f8028c00640 1 --2- 172.21.15.42:0/4239260349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80240af2e0 0x7f802414f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f8029600640 1 --2- 172.21.15.42:0/4239260349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8024153320 0x7f8024151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f801be00640 1 -- 172.21.15.42:0/4239260349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8024153320 msgr2=0x7f8024151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f801be00640 1 --2- 172.21.15.42:0/4239260349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8024153320 0x7f8024151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f801be00640 1 -- 172.21.15.42:0/4239260349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80240af2e0 msgr2=0x7f802414f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:15.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f801be00640 1 --2- 172.21.15.42:0/4239260349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80240af2e0 0x7f802414f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:15.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f801be00640 1 -- 172.21.15.42:0/4239260349 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80241527d0 con 0x7f80240b4b30 2024-05-28T14:09:15.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f801be00640 1 --2- 172.21.15.42:0/4239260349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80240b4b30 0x7f802406f0f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f800c00e840 tx=0x7f800c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:15.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f8021600640 1 -- 172.21.15.42:0/4239260349 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c017070 con 0x7f80240b4b30 2024-05-28T14:09:15.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f8021600640 1 -- 172.21.15.42:0/4239260349 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c00c8d0 con 0x7f80240b4b30 2024-05-28T14:09:15.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f8021600640 1 -- 172.21.15.42:0/4239260349 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c00f040 con 0x7f80240b4b30 2024-05-28T14:09:15.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f802a000640 1 -- 172.21.15.42:0/4239260349 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8024152ab0 con 0x7f80240b4b30 2024-05-28T14:09:15.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f802a000640 1 -- 172.21.15.42:0/4239260349 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f802414fcc0 con 0x7f80240b4b30 2024-05-28T14:09:15.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f802a000640 1 -- 172.21.15.42:0/4239260349 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f802404fcf0 con 0x7f80240b4b30 2024-05-28T14:09:15.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.913+0000 7f8021600640 1 -- 172.21.15.42:0/4239260349 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f800c01a070 con 0x7f80240b4b30 2024-05-28T14:09:15.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.917+0000 7f8021600640 1 --2- 172.21.15.42:0/4239260349 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80000694f0 0x7f800006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:15.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.917+0000 7f8021600640 1 -- 172.21.15.42:0/4239260349 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7f800c085410 con 0x7f80240b4b30 2024-05-28T14:09:15.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.917+0000 7f8028c00640 1 --2- 172.21.15.42:0/4239260349 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80000694f0 0x7f800006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:15.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.917+0000 7f8028c00640 1 --2- 172.21.15.42:0/4239260349 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80000694f0 0x7f800006b9b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f80140ae000 tx=0x7f80140a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:15.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:15.921+0000 7f8021600640 1 -- 172.21.15.42:0/4239260349 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f800c056ae0 con 0x7f80240b4b30 2024-05-28T14:09:16.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.085+0000 7f802a000640 1 -- 172.21.15.42:0/4239260349 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f802406cc70 con 0x7f80240b4b30 2024-05-28T14:09:16.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.089+0000 7f8021600640 1 -- 172.21.15.42:0/4239260349 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16814 (secure 0 0 0) 0x7f800c08b1c0 con 0x7f80240b4b30 2024-05-28T14:09:16.071 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:16.071 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":78,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:09:07.614954+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:06:23.241667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-28T14:06:23.769074+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:09:16.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.089+0000 7f802a000640 1 -- 172.21.15.42:0/4239260349 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80000694f0 msgr2=0x7f800006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:16.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.089+0000 7f802a000640 1 --2- 172.21.15.42:0/4239260349 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80000694f0 0x7f800006b9b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f80140ae000 tx=0x7f80140a7000 comp rx=0 tx=0).stop 2024-05-28T14:09:16.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.089+0000 7f802a000640 1 -- 172.21.15.42:0/4239260349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80240b4b30 msgr2=0x7f802406f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:16.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.089+0000 7f802a000640 1 --2- 172.21.15.42:0/4239260349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80240b4b30 0x7f802406f0f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f800c00e840 tx=0x7f800c00ed00 comp rx=0 tx=0).stop 2024-05-28T14:09:16.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.089+0000 7f802a000640 1 -- 172.21.15.42:0/4239260349 shutdown_connections 2024-05-28T14:09:16.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.089+0000 7f802a000640 1 --2- 172.21.15.42:0/4239260349 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80000694f0 0x7f800006b9b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.089+0000 7f802a000640 1 --2- 172.21.15.42:0/4239260349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8024153320 0x7f8024151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.089+0000 7f802a000640 1 --2- 172.21.15.42:0/4239260349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80240b4b30 0x7f802406f0f0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.089+0000 7f802a000640 1 --2- 172.21.15.42:0/4239260349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80240af2e0 0x7f802414f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.089+0000 7f802a000640 1 -- 172.21.15.42:0/4239260349 >> 172.21.15.42:0/4239260349 conn(0x7f802406bbc0 msgr2=0x7f802415e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:16.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.089+0000 7f802a000640 1 -- 172.21.15.42:0/4239260349 shutdown_connections 2024-05-28T14:09:16.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.089+0000 7f802a000640 1 -- 172.21.15.42:0/4239260349 wait complete. 2024-05-28T14:09:16.093 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-28T14:09:16.093 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:09:16.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.249+0000 7f7756400640 1 Processor -- start 2024-05-28T14:09:16.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.249+0000 7f7756400640 1 -- start start 2024-05-28T14:09:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f7756400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f775015ba40 0x7f77500d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f7756400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f775015f280 con 0x7f775006f660 2024-05-28T14:09:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f7756400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f775015f560 con 0x7f7750072dd0 2024-05-28T14:09:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f7756400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f775015f030 con 0x7f775015ba40 2024-05-28T14:09:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774f400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f775006f660 0x7f7750073980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:57112/0 (socket says 172.21.15.42:57112) 2024-05-28T14:09:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774f400640 1 -- 172.21.15.42:0/3751482619 learned_addr learned my addr 172.21.15.42:0/3751482619 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:16.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f7746a00640 1 --2- 172.21.15.42:0/3751482619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f775015ba40 0x7f77500d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 -- 172.21.15.42:0/3751482619 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 20428637 0 0) 0x7f775015f280 con 0x7f775006f660 2024-05-28T14:09:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 -- 172.21.15.42:0/3751482619 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7734003530 con 0x7f775006f660 2024-05-28T14:09:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 -- 172.21.15.42:0/3751482619 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 994613896 0 0) 0x7f7734003530 con 0x7f775006f660 2024-05-28T14:09:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 -- 172.21.15.42:0/3751482619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f775015ba40 msgr2=0x7f77500d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 --2- 172.21.15.42:0/3751482619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f775015ba40 0x7f77500d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 -- 172.21.15.42:0/3751482619 >> v1:172.21.15.63:6789/0 conn(0x7f7750072dd0 legacy=0x7f775006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 -- 172.21.15.42:0/3751482619 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775015f840 con 0x7f775006f660 2024-05-28T14:09:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 -- 172.21.15.42:0/3751482619 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f7740001a00 con 0x7f775006f660 2024-05-28T14:09:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 -- 172.21.15.42:0/3751482619 >> v1:172.21.15.42:6789/0 conn(0x7f775006f660 legacy=0x7f7750073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 --2- 172.21.15.42:0/3751482619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77340039f0 0x7f7734003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 --2- 172.21.15.42:0/3751482619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7734004810 0x7f7734004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 --2- 172.21.15.42:0/3751482619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7734005240 0x7f773400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 -- 172.21.15.42:0/3751482619 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f775015f030 con 0x7f77340039f0 2024-05-28T14:09:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 -- 172.21.15.42:0/3751482619 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f773400bfc0 con 0x7f7734004810 2024-05-28T14:09:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774ea00640 1 -- 172.21.15.42:0/3751482619 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f773400c170 con 0x7f7734005240 2024-05-28T14:09:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774f400640 1 --2- 172.21.15.42:0/3751482619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77340039f0 0x7f7734003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774fe00640 1 --2- 172.21.15.42:0/3751482619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7734005240 0x7f773400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f7746a00640 1 --2- 172.21.15.42:0/3751482619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7734004810 0x7f7734004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774f400640 1 -- 172.21.15.42:0/3751482619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7734005240 msgr2=0x7f773400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774f400640 1 --2- 172.21.15.42:0/3751482619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7734005240 0x7f773400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774f400640 1 -- 172.21.15.42:0/3751482619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7734004810 msgr2=0x7f7734004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774f400640 1 --2- 172.21.15.42:0/3751482619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7734004810 0x7f7734004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.253+0000 7f774f400640 1 -- 172.21.15.42:0/3751482619 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7734003530 con 0x7f77340039f0 2024-05-28T14:09:16.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.257+0000 7f774f400640 1 --2- 172.21.15.42:0/3751482619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77340039f0 0x7f7734003e10 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f774000a840 tx=0x7f774001d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2315874b287523f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.257+0000 7f774ea00640 1 -- 172.21.15.42:0/3751482619 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774000a460 con 0x7f77340039f0 2024-05-28T14:09:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.257+0000 7f774ea00640 1 -- 172.21.15.42:0/3751482619 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77400ad650 con 0x7f77340039f0 2024-05-28T14:09:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.257+0000 7f774ea00640 1 -- 172.21.15.42:0/3751482619 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77400ada40 con 0x7f77340039f0 2024-05-28T14:09:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.257+0000 7f7756400640 1 -- 172.21.15.42:0/3751482619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77340039f0 msgr2=0x7f7734003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.257+0000 7f7756400640 1 --2- 172.21.15.42:0/3751482619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77340039f0 0x7f7734003e10 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f774000a840 tx=0x7f774001d330 comp rx=0 tx=0).stop 2024-05-28T14:09:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.257+0000 7f774fe00640 1 -- 172.21.15.42:0/3751482619 reap_dead start 2024-05-28T14:09:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.257+0000 7f7756400640 1 -- 172.21.15.42:0/3751482619 shutdown_connections 2024-05-28T14:09:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.257+0000 7f7756400640 1 -- 172.21.15.42:0/3751482619 >> 172.21.15.42:0/3751482619 conn(0x7f775006bdf0 msgr2=0x7f775006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.257+0000 7f7756400640 1 -- 172.21.15.42:0/3751482619 shutdown_connections 2024-05-28T14:09:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.257+0000 7f7756400640 1 -- 172.21.15.42:0/3751482619 wait complete. 2024-05-28T14:09:16.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7756400640 1 Processor -- start 2024-05-28T14:09:16.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7756400640 1 -- start start 2024-05-28T14:09:16.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7756400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77340039f0 0x7f7750161380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7756400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77500b1380 0x7f77500b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7756400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f775014bdb0 0x7f775014c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7756400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f77500d0a80 con 0x7f775014bdb0 2024-05-28T14:09:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7756400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77500d0900 con 0x7f77340039f0 2024-05-28T14:09:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7756400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f77500d0c00 con 0x7f77500b1380 2024-05-28T14:09:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f774fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f775014bdb0 0x7f775014c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f774fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f775014bdb0 0x7f775014c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43734/0 (socket says 172.21.15.42:43734) 2024-05-28T14:09:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f774fe00640 1 -- 172.21.15.42:0/4233330693 learned_addr learned my addr 172.21.15.42:0/4233330693 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f774f400640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77340039f0 0x7f7750161380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7746a00640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77500b1380 0x7f77500b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f774fe00640 1 -- 172.21.15.42:0/4233330693 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77500b1380 msgr2=0x7f77500b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f774fe00640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77500b1380 0x7f77500b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f774fe00640 1 -- 172.21.15.42:0/4233330693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77340039f0 msgr2=0x7f7750161380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f774fe00640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77340039f0 0x7f7750161380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f774fe00640 1 -- 172.21.15.42:0/4233330693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77500afc70 con 0x7f775014bdb0 2024-05-28T14:09:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f774f400640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77340039f0 0x7f7750161380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:16.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f774fe00640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f775014bdb0 0x7f775014c230 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f77500ccb80 tx=0x7f774800f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:16.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7747e00640 1 -- 172.21.15.42:0/4233330693 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77480052e0 con 0x7f775014bdb0 2024-05-28T14:09:16.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7747e00640 1 -- 172.21.15.42:0/4233330693 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7748018070 con 0x7f775014bdb0 2024-05-28T14:09:16.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7747e00640 1 -- 172.21.15.42:0/4233330693 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748003840 con 0x7f775014bdb0 2024-05-28T14:09:16.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7756400640 1 -- 172.21.15.42:0/4233330693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77500aff50 con 0x7f775014bdb0 2024-05-28T14:09:16.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7756400640 1 -- 172.21.15.42:0/4233330693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77500b0510 con 0x7f775014bdb0 2024-05-28T14:09:16.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.261+0000 7f7756400640 1 -- 172.21.15.42:0/4233330693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7750068a90 con 0x7f775014bdb0 2024-05-28T14:09:16.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.265+0000 7f7747e00640 1 -- 172.21.15.42:0/4233330693 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f77480039e0 con 0x7f775014bdb0 2024-05-28T14:09:16.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.265+0000 7f7747e00640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f772c069520 0x7f772c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.265+0000 7f7747e00640 1 -- 172.21.15.42:0/4233330693 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7f7748085340 con 0x7f775014bdb0 2024-05-28T14:09:16.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.265+0000 7f774f400640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f772c069520 0x7f772c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.265+0000 7f774f400640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f772c069520 0x7f772c06b9e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f77400ad000 tx=0x7f775015f450 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:16.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.269+0000 7f7747e00640 1 -- 172.21.15.42:0/4233330693 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7748057000 con 0x7f775014bdb0 2024-05-28T14:09:16.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.469+0000 7f7756400640 1 -- 172.21.15.42:0/4233330693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77500ae1c0 con 0x7f775014bdb0 2024-05-28T14:09:16.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.469+0000 7f7747e00640 1 -- 172.21.15.42:0/4233330693 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5180 (secure 0 0 0) 0x7f7748056900 con 0x7f775014bdb0 2024-05-28T14:09:16.453 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:16.453 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2618011545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2618011545},{"type":"v1","addr":"172.21.15.63:6835","nonce":2618011545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":7077,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/986242955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":986242955},{"type":"v1","addr":"172.21.15.42:6836","nonce":986242955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":7080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1618433260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1618433260},{"type":"v1","addr":"172.21.15.63:6834","nonce":1618433260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:06:24.814669+0000","modified":"2024-05-28T14:06:27.829347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6837/1199082803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1199082803},{"type":"v1","addr":"172.21.15.42:6837","nonce":1199082803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7083,"qdb_cluster":[7083]},"id":5}]} 2024-05-28T14:09:16.453 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 47 2024-05-28T14:09:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.469+0000 7f7756400640 1 -- 172.21.15.42:0/4233330693 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f772c069520 msgr2=0x7f772c06b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.469+0000 7f7756400640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f772c069520 0x7f772c06b9e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f77400ad000 tx=0x7f775015f450 comp rx=0 tx=0).stop 2024-05-28T14:09:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.469+0000 7f7756400640 1 -- 172.21.15.42:0/4233330693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f775014bdb0 msgr2=0x7f775014c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.469+0000 7f7756400640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f775014bdb0 0x7f775014c230 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f77500ccb80 tx=0x7f774800f8a0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.469+0000 7f7756400640 1 -- 172.21.15.42:0/4233330693 shutdown_connections 2024-05-28T14:09:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.469+0000 7f7756400640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f772c069520 0x7f772c06b9e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.469+0000 7f7756400640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f775014bdb0 0x7f775014c230 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.469+0000 7f7756400640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77500b1380 0x7f77500b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.469+0000 7f7756400640 1 --2- 172.21.15.42:0/4233330693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77340039f0 0x7f7750161380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.469+0000 7f7756400640 1 -- 172.21.15.42:0/4233330693 >> 172.21.15.42:0/4233330693 conn(0x7f775006bdf0 msgr2=0x7f7750155390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:16.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.473+0000 7f7756400640 1 -- 172.21.15.42:0/4233330693 shutdown_connections 2024-05-28T14:09:16.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.473+0000 7f7756400640 1 -- 172.21.15.42:0/4233330693 wait complete. 2024-05-28T14:09:16.475 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:09:16.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16ba200640 1 Processor -- start 2024-05-28T14:09:16.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16ba200640 1 -- start start 2024-05-28T14:09:16.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16ba200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f16b415eb00 0x7f16b40cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16ba200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f16b40cfe00 0x7f16b406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16ba200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16b406bf30 0x7f16b40b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16ba200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f16b406c3a0 con 0x7f16b40cfe00 2024-05-28T14:09:16.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16ba200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f16b415eee0 con 0x7f16b415eb00 2024-05-28T14:09:16.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16ba200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f16b40ceb20 con 0x7f16b406bf30 2024-05-28T14:09:16.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16aaa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f16b40cfe00 0x7f16b406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16aaa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f16b40cfe00 0x7f16b406b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43750/0 (socket says 172.21.15.42:43750) 2024-05-28T14:09:16.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16aaa00640 1 -- 172.21.15.42:0/3698679009 learned_addr learned my addr 172.21.15.42:0/3698679009 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:16.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16b3e00640 1 --2- 172.21.15.42:0/3698679009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16b406bf30 0x7f16b40b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16b3400640 1 --2- 172.21.15.42:0/3698679009 >> v2:172.21.15.63:3300/0 conn(0x7f16b415eb00 0x7f16b40cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16aaa00640 1 -- 172.21.15.42:0/3698679009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16b406bf30 msgr2=0x7f16b40b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:16.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16aaa00640 1 --2- 172.21.15.42:0/3698679009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16b406bf30 0x7f16b40b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16aaa00640 1 -- 172.21.15.42:0/3698679009 >> v2:172.21.15.63:3300/0 conn(0x7f16b415eb00 msgr2=0x7f16b40cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:16.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16aaa00640 1 --2- 172.21.15.42:0/3698679009 >> v2:172.21.15.63:3300/0 conn(0x7f16b415eb00 0x7f16b40cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16aaa00640 1 -- 172.21.15.42:0/3698679009 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b40cee00 con 0x7f16b40cfe00 2024-05-28T14:09:16.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.633+0000 7f16b3400640 1 --2- 172.21.15.42:0/3698679009 >> v2:172.21.15.63:3300/0 conn(0x7f16b415eb00 0x7f16b40cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:16.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.637+0000 7f16aaa00640 1 --2- 172.21.15.42:0/3698679009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b40cfe00 0x7f16b406b9f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f16b4071870 tx=0x7f169c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee45e17ca2be977 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:16.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.637+0000 7f16b2a00640 1 -- 172.21.15.42:0/3698679009 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c0b0070 con 0x7f16b40cfe00 2024-05-28T14:09:16.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.637+0000 7f16b2a00640 1 -- 172.21.15.42:0/3698679009 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169c0ab600 con 0x7f16b40cfe00 2024-05-28T14:09:16.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.637+0000 7f16b2a00640 1 -- 172.21.15.42:0/3698679009 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c0ab9d0 con 0x7f16b40cfe00 2024-05-28T14:09:16.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.637+0000 7f16ba200640 1 -- 172.21.15.42:0/3698679009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b40cfe00 msgr2=0x7f16b406b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:16.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.637+0000 7f16ba200640 1 --2- 172.21.15.42:0/3698679009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b40cfe00 0x7f16b406b9f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f16b4071870 tx=0x7f169c01c520 comp rx=0 tx=0).stop 2024-05-28T14:09:16.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.637+0000 7f16ba200640 1 -- 172.21.15.42:0/3698679009 shutdown_connections 2024-05-28T14:09:16.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.637+0000 7f16ba200640 1 --2- 172.21.15.42:0/3698679009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16b406bf30 0x7f16b40b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.637+0000 7f16ba200640 1 --2- 172.21.15.42:0/3698679009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b40cfe00 0x7f16b406b9f0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.637+0000 7f16ba200640 1 --2- 172.21.15.42:0/3698679009 >> v2:172.21.15.63:3300/0 conn(0x7f16b415eb00 0x7f16b40cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.637+0000 7f16ba200640 1 -- 172.21.15.42:0/3698679009 >> 172.21.15.42:0/3698679009 conn(0x7f16b40b68f0 msgr2=0x7f16b40b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:16.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.637+0000 7f16ba200640 1 -- 172.21.15.42:0/3698679009 shutdown_connections 2024-05-28T14:09:16.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.637+0000 7f16ba200640 1 -- 172.21.15.42:0/3698679009 wait complete. 2024-05-28T14:09:16.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16ba200640 1 Processor -- start 2024-05-28T14:09:16.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16ba200640 1 -- start start 2024-05-28T14:09:16.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16ba200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16b406bf30 0x7f16b41429f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16ba200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16b40cfe00 0x7f16b40cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16ba200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b415eb00 0x7f16b41571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16ba200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f16b40b66d0 con 0x7f16b415eb00 2024-05-28T14:09:16.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16ba200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f16b40ce890 con 0x7f16b40cfe00 2024-05-28T14:09:16.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16ba200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f16b40b43d0 con 0x7f16b406bf30 2024-05-28T14:09:16.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16b3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b415eb00 0x7f16b41571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16b3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b415eb00 0x7f16b41571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43756/0 (socket says 172.21.15.42:43756) 2024-05-28T14:09:16.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16b3e00640 1 -- 172.21.15.42:0/4754427 learned_addr learned my addr 172.21.15.42:0/4754427 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:16.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16b3400640 1 --2- 172.21.15.42:0/4754427 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16b406bf30 0x7f16b41429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16aaa00640 1 --2- 172.21.15.42:0/4754427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16b40cfe00 0x7f16b40cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16b3e00640 1 -- 172.21.15.42:0/4754427 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16b406bf30 msgr2=0x7f16b41429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:16.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16b3e00640 1 --2- 172.21.15.42:0/4754427 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16b406bf30 0x7f16b41429f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16b3e00640 1 -- 172.21.15.42:0/4754427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16b40cfe00 msgr2=0x7f16b40cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:16.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16b3e00640 1 --2- 172.21.15.42:0/4754427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16b40cfe00 0x7f16b40cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16b3e00640 1 -- 172.21.15.42:0/4754427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b40ccf50 con 0x7f16b415eb00 2024-05-28T14:09:16.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16b3e00640 1 --2- 172.21.15.42:0/4754427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b415eb00 0x7f16b41571c0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00d800 tx=0x7f16ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:16.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16abe00640 1 -- 172.21.15.42:0/4754427 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac018020 con 0x7f16b415eb00 2024-05-28T14:09:16.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16abe00640 1 -- 172.21.15.42:0/4754427 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16ac0096c0 con 0x7f16b415eb00 2024-05-28T14:09:16.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16abe00640 1 -- 172.21.15.42:0/4754427 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac009ab0 con 0x7f16b415eb00 2024-05-28T14:09:16.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16ba200640 1 -- 172.21.15.42:0/4754427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b40cd230 con 0x7f16b415eb00 2024-05-28T14:09:16.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.641+0000 7f16ba200640 1 -- 172.21.15.42:0/4754427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b4157700 con 0x7f16b415eb00 2024-05-28T14:09:16.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.645+0000 7f16ba200640 1 -- 172.21.15.42:0/4754427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b40ae650 con 0x7f16b415eb00 2024-05-28T14:09:16.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.645+0000 7f16abe00640 1 -- 172.21.15.42:0/4754427 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f16ac01b020 con 0x7f16b415eb00 2024-05-28T14:09:16.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.645+0000 7f16abe00640 1 --2- 172.21.15.42:0/4754427 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16900694f0 0x7f169006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:16.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.645+0000 7f16abe00640 1 -- 172.21.15.42:0/4754427 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7f16ac085c90 con 0x7f16b415eb00 2024-05-28T14:09:16.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.645+0000 7f16b3400640 1 --2- 172.21.15.42:0/4754427 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16900694f0 0x7f169006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:16.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.645+0000 7f16b3400640 1 --2- 172.21.15.42:0/4754427 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16900694f0 0x7f169006b9b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f16b41fada0 tx=0x7f16a4007770 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:16.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.649+0000 7f16abe00640 1 -- 172.21.15.42:0/4754427 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f16ac022020 con 0x7f16b415eb00 2024-05-28T14:09:16.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.853+0000 7f16ba200640 1 -- 172.21.15.42:0/4754427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16b4068a90 con 0x7f16b415eb00 2024-05-28T14:09:16.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.853+0000 7f16abe00640 1 -- 172.21.15.42:0/4754427 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5180 (secure 0 0 0) 0x7f16ac057300 con 0x7f16b415eb00 2024-05-28T14:09:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:16.837 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2618011545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2618011545},{"type":"v1","addr":"172.21.15.63:6835","nonce":2618011545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":7077,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/986242955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":986242955},{"type":"v1","addr":"172.21.15.42:6836","nonce":986242955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":7080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1618433260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1618433260},{"type":"v1","addr":"172.21.15.63:6834","nonce":1618433260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:06:24.814669+0000","modified":"2024-05-28T14:06:27.829347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6837/1199082803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":1199082803},{"type":"v1","addr":"172.21.15.42:6837","nonce":1199082803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7083,"qdb_cluster":[7083]},"id":5}]} 2024-05-28T14:09:16.837 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 47 2024-05-28T14:09:16.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.853+0000 7f16ba200640 1 -- 172.21.15.42:0/4754427 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16900694f0 msgr2=0x7f169006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:16.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.853+0000 7f16ba200640 1 --2- 172.21.15.42:0/4754427 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16900694f0 0x7f169006b9b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f16b41fada0 tx=0x7f16a4007770 comp rx=0 tx=0).stop 2024-05-28T14:09:16.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.853+0000 7f16ba200640 1 -- 172.21.15.42:0/4754427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b415eb00 msgr2=0x7f16b41571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:16.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.853+0000 7f16ba200640 1 --2- 172.21.15.42:0/4754427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b415eb00 0x7f16b41571c0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00d800 tx=0x7f16ac00dcc0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.853+0000 7f16ba200640 1 -- 172.21.15.42:0/4754427 shutdown_connections 2024-05-28T14:09:16.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.853+0000 7f16ba200640 1 --2- 172.21.15.42:0/4754427 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f16900694f0 0x7f169006b9b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.853+0000 7f16ba200640 1 --2- 172.21.15.42:0/4754427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b415eb00 0x7f16b41571c0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.853+0000 7f16ba200640 1 --2- 172.21.15.42:0/4754427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16b40cfe00 0x7f16b40cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.853+0000 7f16ba200640 1 --2- 172.21.15.42:0/4754427 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16b406bf30 0x7f16b41429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:16.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.853+0000 7f16ba200640 1 -- 172.21.15.42:0/4754427 >> 172.21.15.42:0/4754427 conn(0x7f16b40b68f0 msgr2=0x7f16b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:16.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.853+0000 7f16ba200640 1 -- 172.21.15.42:0/4754427 shutdown_connections 2024-05-28T14:09:16.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:16.853+0000 7f16ba200640 1 -- 172.21.15.42:0/4754427 wait complete. 2024-05-28T14:09:16.859 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:09:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5929c00640 1 Processor -- start 2024-05-28T14:09:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5929c00640 1 -- start start 2024-05-28T14:09:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5929c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f592415c970 0x7f59240ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5929c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f592406c100 0x7f59240d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5929c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f592406c700 con 0x7f59240b3aa0 2024-05-28T14:09:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5929c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f59241611b0 con 0x7f592415c970 2024-05-28T14:09:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5929c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5924160350 con 0x7f592406c100 2024-05-28T14:09:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5922a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f592415c970 0x7f59240ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5922a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f592415c970 0x7f59240ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43770/0 (socket says 172.21.15.42:43770) 2024-05-28T14:09:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5922a00640 1 -- 172.21.15.42:0/108345727 learned_addr learned my addr 172.21.15.42:0/108345727 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5923400640 1 --2- 172.21.15.42:0/108345727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f592406c100 0x7f59240d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5922a00640 1 -- 172.21.15.42:0/108345727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f592406c100 msgr2=0x7f59240d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5922a00640 1 --2- 172.21.15.42:0/108345727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f592406c100 0x7f59240d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5922a00640 1 -- 172.21.15.42:0/108345727 >> v1:172.21.15.42:6789/0 conn(0x7f59240b3aa0 legacy=0x7f592406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.017+0000 7f5922a00640 1 -- 172.21.15.42:0/108345727 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59241605a0 con 0x7f592415c970 2024-05-28T14:09:17.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.021+0000 7f5922a00640 1 --2- 172.21.15.42:0/108345727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592415c970 0x7f59240ccfa0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5914009870 tx=0x7f591401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6a45f4368727b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:17.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.021+0000 7f5922000640 1 -- 172.21.15.42:0/108345727 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140b2070 con 0x7f592415c970 2024-05-28T14:09:17.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.021+0000 7f5922000640 1 -- 172.21.15.42:0/108345727 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59140a9070 con 0x7f592415c970 2024-05-28T14:09:17.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.021+0000 7f5922000640 1 -- 172.21.15.42:0/108345727 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140ad680 con 0x7f592415c970 2024-05-28T14:09:17.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.021+0000 7f5929c00640 1 -- 172.21.15.42:0/108345727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592415c970 msgr2=0x7f59240ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.021+0000 7f5929c00640 1 --2- 172.21.15.42:0/108345727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592415c970 0x7f59240ccfa0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5914009870 tx=0x7f591401c560 comp rx=0 tx=0).stop 2024-05-28T14:09:17.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.021+0000 7f5929c00640 1 -- 172.21.15.42:0/108345727 shutdown_connections 2024-05-28T14:09:17.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.021+0000 7f5929c00640 1 --2- 172.21.15.42:0/108345727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f592406c100 0x7f59240d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.021+0000 7f5929c00640 1 --2- 172.21.15.42:0/108345727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592415c970 0x7f59240ccfa0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.021+0000 7f5929c00640 1 -- 172.21.15.42:0/108345727 >> 172.21.15.42:0/108345727 conn(0x7f59240b2f80 msgr2=0x7f59240b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:17.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.021+0000 7f5929c00640 1 -- 172.21.15.42:0/108345727 shutdown_connections 2024-05-28T14:09:17.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.021+0000 7f5929c00640 1 -- 172.21.15.42:0/108345727 wait complete. 2024-05-28T14:09:17.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5929c00640 1 Processor -- start 2024-05-28T14:09:17.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5929c00640 1 -- start start 2024-05-28T14:09:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5929c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592406c100 0x7f592414b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5929c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59240b3aa0 0x7f59240c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5929c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59240c54a0 0x7f59240aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5929c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5924169870 con 0x7f592406c100 2024-05-28T14:09:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5929c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59241696f0 con 0x7f59240b3aa0 2024-05-28T14:09:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5929c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f59241699f0 con 0x7f59240c54a0 2024-05-28T14:09:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5922a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592406c100 0x7f592414b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5922a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592406c100 0x7f592414b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43784/0 (socket says 172.21.15.42:43784) 2024-05-28T14:09:17.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5922a00640 1 -- 172.21.15.42:0/3059430155 learned_addr learned my addr 172.21.15.42:0/3059430155 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:17.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5923400640 1 --2- 172.21.15.42:0/3059430155 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59240c54a0 0x7f59240aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f591a000640 1 --2- 172.21.15.42:0/3059430155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59240b3aa0 0x7f59240c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5922a00640 1 -- 172.21.15.42:0/3059430155 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59240c54a0 msgr2=0x7f59240aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5922a00640 1 --2- 172.21.15.42:0/3059430155 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59240c54a0 0x7f59240aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5922a00640 1 -- 172.21.15.42:0/3059430155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59240b3aa0 msgr2=0x7f59240c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5922a00640 1 --2- 172.21.15.42:0/3059430155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59240b3aa0 0x7f59240c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5922a00640 1 -- 172.21.15.42:0/3059430155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59240afed0 con 0x7f592406c100 2024-05-28T14:09:17.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5922a00640 1 --2- 172.21.15.42:0/3059430155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592406c100 0x7f592414b7f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f59140097c0 tx=0x7f591401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:17.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f591b400640 1 -- 172.21.15.42:0/3059430155 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140bc020 con 0x7f592406c100 2024-05-28T14:09:17.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f591b400640 1 -- 172.21.15.42:0/3059430155 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59140ade50 con 0x7f592406c100 2024-05-28T14:09:17.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f591b400640 1 -- 172.21.15.42:0/3059430155 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140b2040 con 0x7f592406c100 2024-05-28T14:09:17.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5929c00640 1 -- 172.21.15.42:0/3059430155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59240afbf0 con 0x7f592406c100 2024-05-28T14:09:17.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.025+0000 7f5929c00640 1 -- 172.21.15.42:0/3059430155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59240b4d70 con 0x7f592406c100 2024-05-28T14:09:17.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.029+0000 7f5929c00640 1 -- 172.21.15.42:0/3059430155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59240c58b0 con 0x7f592406c100 2024-05-28T14:09:17.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.029+0000 7f591b400640 1 -- 172.21.15.42:0/3059430155 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f59140bf020 con 0x7f592406c100 2024-05-28T14:09:17.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.029+0000 7f591b400640 1 --2- 172.21.15.42:0/3059430155 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59000694f0 0x7f590006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.029+0000 7f591b400640 1 -- 172.21.15.42:0/3059430155 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7f591411faf0 con 0x7f592406c100 2024-05-28T14:09:17.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.029+0000 7f591a000640 1 --2- 172.21.15.42:0/3059430155 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59000694f0 0x7f590006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.029+0000 7f591a000640 1 --2- 172.21.15.42:0/3059430155 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59000694f0 0x7f590006b9b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f592406bf80 tx=0x7f590c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:17.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.033+0000 7f591b400640 1 -- 172.21.15.42:0/3059430155 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f59240c58b0 con 0x7f592406c100 2024-05-28T14:09:17.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.197+0000 7f5929c00640 1 -- 172.21.15.42:0/3059430155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5924068a90 con 0x7f592406c100 2024-05-28T14:09:17.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.201+0000 7f591b400640 1 -- 172.21.15.42:0/3059430155 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16814 (secure 0 0 0) 0x7f59140c6030 con 0x7f592406c100 2024-05-28T14:09:17.184 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:17.184 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":78,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:09:07.614954+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:06:23.241667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-28T14:06:23.769074+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:09:17.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.201+0000 7f5929c00640 1 -- 172.21.15.42:0/3059430155 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59000694f0 msgr2=0x7f590006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.201+0000 7f5929c00640 1 --2- 172.21.15.42:0/3059430155 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59000694f0 0x7f590006b9b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f592406bf80 tx=0x7f590c009040 comp rx=0 tx=0).stop 2024-05-28T14:09:17.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.201+0000 7f5929c00640 1 -- 172.21.15.42:0/3059430155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592406c100 msgr2=0x7f592414b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.201+0000 7f5929c00640 1 --2- 172.21.15.42:0/3059430155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592406c100 0x7f592414b7f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f59140097c0 tx=0x7f591401ada0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.201+0000 7f5929c00640 1 -- 172.21.15.42:0/3059430155 shutdown_connections 2024-05-28T14:09:17.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.201+0000 7f5929c00640 1 --2- 172.21.15.42:0/3059430155 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59000694f0 0x7f590006b9b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.201+0000 7f5929c00640 1 --2- 172.21.15.42:0/3059430155 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59240c54a0 0x7f59240aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.201+0000 7f5929c00640 1 --2- 172.21.15.42:0/3059430155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59240b3aa0 0x7f59240c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.201+0000 7f5929c00640 1 --2- 172.21.15.42:0/3059430155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592406c100 0x7f592414b7f0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.201+0000 7f5929c00640 1 -- 172.21.15.42:0/3059430155 >> 172.21.15.42:0/3059430155 conn(0x7f59240b2f80 msgr2=0x7f59240cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:17.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.201+0000 7f5929c00640 1 -- 172.21.15.42:0/3059430155 shutdown_connections 2024-05-28T14:09:17.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.201+0000 7f5929c00640 1 -- 172.21.15.42:0/3059430155 wait complete. 2024-05-28T14:09:17.206 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-28T14:09:17.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4450a00640 1 Processor -- start 2024-05-28T14:09:17.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4450a00640 1 -- start start 2024-05-28T14:09:17.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4450a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f444c15bf30 0x7f444c15c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4450a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f444c0cf230 0x7f444c072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4450a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f444c0cfc10 con 0x7f444c072ba0 2024-05-28T14:09:17.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4450a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f444c0721d0 con 0x7f444c0cf230 2024-05-28T14:09:17.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4450a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f444c0cfef0 con 0x7f444c15bf30 2024-05-28T14:09:17.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f444a800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f444c072ba0 0x7f444c06ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:57124/0 (socket says 172.21.15.42:57124) 2024-05-28T14:09:17.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f444a800640 1 -- 172.21.15.42:0/1034238292 learned_addr learned my addr 172.21.15.42:0/1034238292 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:17.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449e00640 1 --2- 172.21.15.42:0/1034238292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f444c15bf30 0x7f444c15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4443e00640 1 --2- 172.21.15.42:0/1034238292 >> v2:172.21.15.63:3300/0 conn(0x7f444c0cf230 0x7f444c072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 -- 172.21.15.42:0/1034238292 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 406097257 0 0) 0x7f444c0cfc10 con 0x7f444c072ba0 2024-05-28T14:09:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 -- 172.21.15.42:0/1034238292 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4430005740 con 0x7f444c072ba0 2024-05-28T14:09:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 -- 172.21.15.42:0/1034238292 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 631389037 0 0) 0x7f4430005740 con 0x7f444c072ba0 2024-05-28T14:09:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 -- 172.21.15.42:0/1034238292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f444c15bf30 msgr2=0x7f444c15c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 --2- 172.21.15.42:0/1034238292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f444c15bf30 0x7f444c15c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 -- 172.21.15.42:0/1034238292 >> v2:172.21.15.63:3300/0 conn(0x7f444c0cf230 msgr2=0x7f444c072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 --2- 172.21.15.42:0/1034238292 >> v2:172.21.15.63:3300/0 conn(0x7f444c0cf230 0x7f444c072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 -- 172.21.15.42:0/1034238292 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444c0d01d0 con 0x7f444c072ba0 2024-05-28T14:09:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 -- 172.21.15.42:0/1034238292 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f4444001a00 con 0x7f444c072ba0 2024-05-28T14:09:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 -- 172.21.15.42:0/1034238292 >> v1:172.21.15.42:6789/0 conn(0x7f444c072ba0 legacy=0x7f444c06ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 --2- 172.21.15.42:0/1034238292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4430005bd0 0x7f4430005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 --2- 172.21.15.42:0/1034238292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4430006a20 0x7f4430006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 --2- 172.21.15.42:0/1034238292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f444c15bf30 0x7f44300073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 -- 172.21.15.42:0/1034238292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f444c0721d0 con 0x7f4430005bd0 2024-05-28T14:09:17.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 -- 172.21.15.42:0/1034238292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f444c0cfef0 con 0x7f4430006a20 2024-05-28T14:09:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.365+0000 7f4449400640 1 -- 172.21.15.42:0/1034238292 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f443000dbe0 con 0x7f444c15bf30 2024-05-28T14:09:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.369+0000 7f4449e00640 1 --2- 172.21.15.42:0/1034238292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4430005bd0 0x7f4430005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.369+0000 7f4443e00640 1 --2- 172.21.15.42:0/1034238292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4430006a20 0x7f4430006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.369+0000 7f444a800640 1 --2- 172.21.15.42:0/1034238292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f444c15bf30 0x7f44300073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.369+0000 7f4449e00640 1 -- 172.21.15.42:0/1034238292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f444c15bf30 msgr2=0x7f44300073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.369+0000 7f4449e00640 1 --2- 172.21.15.42:0/1034238292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f444c15bf30 0x7f44300073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.369+0000 7f4449e00640 1 -- 172.21.15.42:0/1034238292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4430006a20 msgr2=0x7f4430006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.369+0000 7f4449e00640 1 --2- 172.21.15.42:0/1034238292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4430006a20 0x7f4430006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.369+0000 7f4449e00640 1 -- 172.21.15.42:0/1034238292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4430005740 con 0x7f4430005bd0 2024-05-28T14:09:17.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.369+0000 7f4449e00640 1 --2- 172.21.15.42:0/1034238292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4430005bd0 0x7f4430005ff0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f443c009ae0 tx=0x7f443c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370409c5f82be0f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:17.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.369+0000 7f4449400640 1 -- 172.21.15.42:0/1034238292 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0b2070 con 0x7f4430005bd0 2024-05-28T14:09:17.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.369+0000 7f4449400640 1 -- 172.21.15.42:0/1034238292 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f443c0a5550 con 0x7f4430005bd0 2024-05-28T14:09:17.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.369+0000 7f4449400640 1 -- 172.21.15.42:0/1034238292 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0096b0 con 0x7f4430005bd0 2024-05-28T14:09:17.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.369+0000 7f4450a00640 1 -- 172.21.15.42:0/1034238292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4430005bd0 msgr2=0x7f4430005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.369+0000 7f4450a00640 1 --2- 172.21.15.42:0/1034238292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4430005bd0 0x7f4430005ff0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f443c009ae0 tx=0x7f443c01ca40 comp rx=0 tx=0).stop 2024-05-28T14:09:17.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f444a800640 1 -- 172.21.15.42:0/1034238292 reap_dead start 2024-05-28T14:09:17.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4450a00640 1 -- 172.21.15.42:0/1034238292 shutdown_connections 2024-05-28T14:09:17.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4450a00640 1 -- 172.21.15.42:0/1034238292 >> 172.21.15.42:0/1034238292 conn(0x7f444c06b9f0 msgr2=0x7f444c06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:17.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4450a00640 1 -- 172.21.15.42:0/1034238292 shutdown_connections 2024-05-28T14:09:17.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4450a00640 1 -- 172.21.15.42:0/1034238292 wait complete. 2024-05-28T14:09:17.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4450a00640 1 Processor -- start 2024-05-28T14:09:17.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4450a00640 1 -- start start 2024-05-28T14:09:17.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4450a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4430005bd0 0x7f444c13d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4450a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f444c15e780 0x7f444c15ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4450a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f444c0a94b0 0x7f444c0a9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4450a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f444c0b4070 con 0x7f444c0a94b0 2024-05-28T14:09:17.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4450a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f444c0b3ef0 con 0x7f4430005bd0 2024-05-28T14:09:17.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4450a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f444c0b41f0 con 0x7f444c15e780 2024-05-28T14:09:17.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f444a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f444c0a94b0 0x7f444c0a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f444a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f444c0a94b0 0x7f444c0a9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49332/0 (socket says 172.21.15.42:49332) 2024-05-28T14:09:17.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f444a800640 1 -- 172.21.15.42:0/2257047145 learned_addr learned my addr 172.21.15.42:0/2257047145 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:17.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4443e00640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f444c15e780 0x7f444c15ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4449e00640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4430005bd0 0x7f444c13d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f444a800640 1 -- 172.21.15.42:0/2257047145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f444c15e780 msgr2=0x7f444c15ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f444a800640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f444c15e780 0x7f444c15ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f444a800640 1 -- 172.21.15.42:0/2257047145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4430005bd0 msgr2=0x7f444c13d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f444a800640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4430005bd0 0x7f444c13d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f444a800640 1 -- 172.21.15.42:0/2257047145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444c0a9e70 con 0x7f444c0a94b0 2024-05-28T14:09:17.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f4449e00640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4430005bd0 0x7f444c13d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:17.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.373+0000 7f444a800640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f444c0a94b0 0x7f444c0a9930 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f4444007600 tx=0x7f4444007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:17.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.377+0000 7f4442000640 1 -- 172.21.15.42:0/2257047145 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444400aad0 con 0x7f444c0a94b0 2024-05-28T14:09:17.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.377+0000 7f4442000640 1 -- 172.21.15.42:0/2257047145 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4444018070 con 0x7f444c0a94b0 2024-05-28T14:09:17.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.377+0000 7f4442000640 1 -- 172.21.15.42:0/2257047145 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44440133f0 con 0x7f444c0a94b0 2024-05-28T14:09:17.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.377+0000 7f4450a00640 1 -- 172.21.15.42:0/2257047145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444c0aa130 con 0x7f444c0a94b0 2024-05-28T14:09:17.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.377+0000 7f4450a00640 1 -- 172.21.15.42:0/2257047145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f444c13d990 con 0x7f444c0a94b0 2024-05-28T14:09:17.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.377+0000 7f4450a00640 1 -- 172.21.15.42:0/2257047145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f444c073890 con 0x7f444c0a94b0 2024-05-28T14:09:17.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.377+0000 7f4442000640 1 -- 172.21.15.42:0/2257047145 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f4444013590 con 0x7f444c0a94b0 2024-05-28T14:09:17.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.377+0000 7f4442000640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4428069520 0x7f442806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.377+0000 7f4442000640 1 -- 172.21.15.42:0/2257047145 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5823+0+0 (secure 0 0 0) 0x7f4444085cb0 con 0x7f444c0a94b0 2024-05-28T14:09:17.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.377+0000 7f4449e00640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4428069520 0x7f442806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.381+0000 7f4449e00640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4428069520 0x7f442806b9e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f443c0af000 tx=0x7f443c0046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:17.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.381+0000 7f4442000640 1 -- 172.21.15.42:0/2257047145 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4444057380 con 0x7f444c0a94b0 2024-05-28T14:09:17.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.585+0000 7f4450a00640 1 -- 172.21.15.42:0/2257047145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f444c04fdd0 con 0x7f444c0a94b0 2024-05-28T14:09:17.570 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-05-28T14:09:17.585+0000 7f248b600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-28T14:09:17.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.589+0000 7f4442000640 1 -- 172.21.15.42:0/2257047145 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7f444405b680 con 0x7f444c0a94b0 2024-05-28T14:09:17.573 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-28T14:09:17.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.593+0000 7f4450a00640 1 -- 172.21.15.42:0/2257047145 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4428069520 msgr2=0x7f442806b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.593+0000 7f4450a00640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4428069520 0x7f442806b9e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f443c0af000 tx=0x7f443c0046d0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.593+0000 7f4450a00640 1 -- 172.21.15.42:0/2257047145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f444c0a94b0 msgr2=0x7f444c0a9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.593+0000 7f4450a00640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f444c0a94b0 0x7f444c0a9930 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f4444007600 tx=0x7f4444007a00 comp rx=0 tx=0).stop 2024-05-28T14:09:17.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.593+0000 7f4450a00640 1 -- 172.21.15.42:0/2257047145 shutdown_connections 2024-05-28T14:09:17.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.593+0000 7f4450a00640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4428069520 0x7f442806b9e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.593+0000 7f4450a00640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f444c0a94b0 0x7f444c0a9930 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.593+0000 7f4450a00640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f444c15e780 0x7f444c15ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.593+0000 7f4450a00640 1 --2- 172.21.15.42:0/2257047145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4430005bd0 0x7f444c13d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.593+0000 7f4450a00640 1 -- 172.21.15.42:0/2257047145 >> 172.21.15.42:0/2257047145 conn(0x7f444c06b9f0 msgr2=0x7f444c152080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:17.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.593+0000 7f4450a00640 1 -- 172.21.15.42:0/2257047145 shutdown_connections 2024-05-28T14:09:17.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.593+0000 7f4450a00640 1 -- 172.21.15.42:0/2257047145 wait complete. 2024-05-28T14:09:17.599 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-28T14:09:17.644 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-05-28T14:09:17.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f214a800640 1 Processor -- start 2024-05-28T14:09:17.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f214a800640 1 -- start start 2024-05-28T14:09:17.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f214a800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2144144f80 0x7f2144072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f214a800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f214406ee20 0x7f214406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f214a800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21440afe40 con 0x7f214415b9b0 2024-05-28T14:09:17.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f214a800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f214406f7c0 con 0x7f2144144f80 2024-05-28T14:09:17.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f214a800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f21440724d0 con 0x7f214406ee20 2024-05-28T14:09:17.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f2143400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2144144f80 0x7f2144072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f2143400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2144144f80 0x7f2144072960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49362/0 (socket says 172.21.15.42:49362) 2024-05-28T14:09:17.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f2143400640 1 -- 172.21.15.42:0/2770102115 learned_addr learned my addr 172.21.15.42:0/2770102115 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:17.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f2142a00640 1 --2- 172.21.15.42:0/2770102115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f214406ee20 0x7f214406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f2143400640 1 -- 172.21.15.42:0/2770102115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f214406ee20 msgr2=0x7f214406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:17.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f2143400640 1 --2- 172.21.15.42:0/2770102115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f214406ee20 0x7f214406f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f2143400640 1 -- 172.21.15.42:0/2770102115 >> v1:172.21.15.63:6789/0 conn(0x7f214415b9b0 legacy=0x7f214415be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.765+0000 7f2143400640 1 -- 172.21.15.42:0/2770102115 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214415c600 con 0x7f2144144f80 2024-05-28T14:09:17.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.769+0000 7f2143400640 1 --2- 172.21.15.42:0/2770102115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2144144f80 0x7f2144072960 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2134009870 tx=0x7f213401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258575d6a3e17d4a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:17.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.769+0000 7f2142000640 1 -- 172.21.15.42:0/2770102115 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340ab650 con 0x7f2144144f80 2024-05-28T14:09:17.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.769+0000 7f2142000640 1 -- 172.21.15.42:0/2770102115 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21340b0070 con 0x7f2144144f80 2024-05-28T14:09:17.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.769+0000 7f2142000640 1 -- 172.21.15.42:0/2770102115 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340aba10 con 0x7f2144144f80 2024-05-28T14:09:17.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.769+0000 7f214a800640 1 -- 172.21.15.42:0/2770102115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2144144f80 msgr2=0x7f2144072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.769+0000 7f214a800640 1 --2- 172.21.15.42:0/2770102115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2144144f80 0x7f2144072960 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2134009870 tx=0x7f213401c560 comp rx=0 tx=0).stop 2024-05-28T14:09:17.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.769+0000 7f214a800640 1 -- 172.21.15.42:0/2770102115 shutdown_connections 2024-05-28T14:09:17.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.769+0000 7f214a800640 1 --2- 172.21.15.42:0/2770102115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f214406ee20 0x7f214406f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.769+0000 7f214a800640 1 --2- 172.21.15.42:0/2770102115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2144144f80 0x7f2144072960 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.769+0000 7f214a800640 1 -- 172.21.15.42:0/2770102115 >> 172.21.15.42:0/2770102115 conn(0x7f21440b21f0 msgr2=0x7f21440b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:17.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.769+0000 7f214a800640 1 -- 172.21.15.42:0/2770102115 shutdown_connections 2024-05-28T14:09:17.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.769+0000 7f214a800640 1 -- 172.21.15.42:0/2770102115 wait complete. 2024-05-28T14:09:17.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.769+0000 7f214a800640 1 Processor -- start 2024-05-28T14:09:17.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.769+0000 7f214a800640 1 -- start start 2024-05-28T14:09:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f214a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f214406ee20 0x7f2144141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f214a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2144144f80 0x7f21441404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f214a800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f214415b9b0 0x7f21440aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f214a800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2144072ea0 con 0x7f2144144f80 2024-05-28T14:09:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f214a800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f21440b4010 con 0x7f214406ee20 2024-05-28T14:09:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f214a800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2144142330 con 0x7f214415b9b0 2024-05-28T14:09:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f2142a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2144144f80 0x7f21441404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f2142a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2144144f80 0x7f21441404a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49372/0 (socket says 172.21.15.42:49372) 2024-05-28T14:09:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f2142a00640 1 -- 172.21.15.42:0/3068295494 learned_addr learned my addr 172.21.15.42:0/3068295494 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:17.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f2143e00640 1 --2- 172.21.15.42:0/3068295494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f214415b9b0 0x7f21440aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f2143400640 1 --2- 172.21.15.42:0/3068295494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f214406ee20 0x7f2144141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f2142a00640 1 -- 172.21.15.42:0/3068295494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f214415b9b0 msgr2=0x7f21440aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f2142a00640 1 --2- 172.21.15.42:0/3068295494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f214415b9b0 0x7f21440aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f2142a00640 1 -- 172.21.15.42:0/3068295494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f214406ee20 msgr2=0x7f2144141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:17.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f2142a00640 1 --2- 172.21.15.42:0/3068295494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f214406ee20 0x7f2144141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:17.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f2142a00640 1 -- 172.21.15.42:0/3068295494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2144142610 con 0x7f2144144f80 2024-05-28T14:09:17.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f2142a00640 1 --2- 172.21.15.42:0/3068295494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2144144f80 0x7f21441404a0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f213000d800 tx=0x7f213000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:17.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f213b400640 1 -- 172.21.15.42:0/3068295494 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213000b6c0 con 0x7f2144144f80 2024-05-28T14:09:17.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f213b400640 1 -- 172.21.15.42:0/3068295494 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2130019070 con 0x7f2144144f80 2024-05-28T14:09:17.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f213b400640 1 -- 172.21.15.42:0/3068295494 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213000ba60 con 0x7f2144144f80 2024-05-28T14:09:17.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f214a800640 1 -- 172.21.15.42:0/3068295494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21441428f0 con 0x7f2144144f80 2024-05-28T14:09:17.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f214a800640 1 -- 172.21.15.42:0/3068295494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2144142eb0 con 0x7f2144144f80 2024-05-28T14:09:17.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.773+0000 7f214a800640 1 -- 172.21.15.42:0/3068295494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21440738b0 con 0x7f2144144f80 2024-05-28T14:09:17.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.777+0000 7f213b400640 1 -- 172.21.15.42:0/3068295494 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f213000bc00 con 0x7f2144144f80 2024-05-28T14:09:17.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.777+0000 7f213b400640 1 --2- 172.21.15.42:0/3068295494 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21240694f0 0x7f212406b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:17.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.777+0000 7f213b400640 1 -- 172.21.15.42:0/3068295494 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5881+0+0 (secure 0 0 0) 0x7f2130085c10 con 0x7f2144144f80 2024-05-28T14:09:17.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.777+0000 7f2143400640 1 --2- 172.21.15.42:0/3068295494 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21240694f0 0x7f212406b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:17.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.777+0000 7f2143400640 1 --2- 172.21.15.42:0/3068295494 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21240694f0 0x7f212406b9b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f21340af000 tx=0x7f21340a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:17.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.781+0000 7f213b400640 1 -- 172.21.15.42:0/3068295494 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2130057280 con 0x7f2144144f80 2024-05-28T14:09:17.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:17.981+0000 7f214a800640 1 -- 172.21.15.42:0/3068295494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f214404fdd0 con 0x7f2144144f80 2024-05-28T14:09:18.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.669+0000 7f213b400640 1 -- 172.21.15.42:0/3068295494 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7f213005b580 con 0x7f2144144f80 2024-05-28T14:09:18.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.673+0000 7f214a800640 1 -- 172.21.15.42:0/3068295494 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21240694f0 msgr2=0x7f212406b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:18.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.673+0000 7f214a800640 1 --2- 172.21.15.42:0/3068295494 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21240694f0 0x7f212406b9b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f21340af000 tx=0x7f21340a7000 comp rx=0 tx=0).stop 2024-05-28T14:09:18.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.673+0000 7f214a800640 1 -- 172.21.15.42:0/3068295494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2144144f80 msgr2=0x7f21441404a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:18.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.673+0000 7f214a800640 1 --2- 172.21.15.42:0/3068295494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2144144f80 0x7f21441404a0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f213000d800 tx=0x7f213000dcc0 comp rx=0 tx=0).stop 2024-05-28T14:09:18.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.673+0000 7f214a800640 1 -- 172.21.15.42:0/3068295494 shutdown_connections 2024-05-28T14:09:18.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.673+0000 7f214a800640 1 --2- 172.21.15.42:0/3068295494 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21240694f0 0x7f212406b9b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:18.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.673+0000 7f214a800640 1 --2- 172.21.15.42:0/3068295494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f214415b9b0 0x7f21440aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:18.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.673+0000 7f214a800640 1 --2- 172.21.15.42:0/3068295494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2144144f80 0x7f21441404a0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:18.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.673+0000 7f214a800640 1 --2- 172.21.15.42:0/3068295494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f214406ee20 0x7f2144141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:18.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.673+0000 7f214a800640 1 -- 172.21.15.42:0/3068295494 >> 172.21.15.42:0/3068295494 conn(0x7f21440b21f0 msgr2=0x7f2144155810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:18.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.673+0000 7f214a800640 1 -- 172.21.15.42:0/3068295494 shutdown_connections 2024-05-28T14:09:18.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.673+0000 7f214a800640 1 -- 172.21.15.42:0/3068295494 wait complete. 2024-05-28T14:09:18.683 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-28T14:09:18.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.893+0000 7f9d03e00640 1 Processor -- start 2024-05-28T14:09:18.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.893+0000 7f9d03e00640 1 -- start start 2024-05-28T14:09:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.893+0000 7f9d03e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cfc15b350 0x7f9cfc06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.893+0000 7f9d03e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9cfc072d40 0x7f9cfc0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.893+0000 7f9d03e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9cfc0737c0 0x7f9cfc0d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.893+0000 7f9d03e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9cfc0ae9f0 con 0x7f9cfc0737c0 2024-05-28T14:09:18.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.893+0000 7f9d03e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9cfc0ade40 con 0x7f9cfc072d40 2024-05-28T14:09:18.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.893+0000 7f9d03e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9cfc0ae0c0 con 0x7f9cfc15b350 2024-05-28T14:09:18.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.893+0000 7f9d01e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9cfc0737c0 0x7f9cfc0d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:18.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.893+0000 7f9d01e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9cfc0737c0 0x7f9cfc0d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49380/0 (socket says 172.21.15.42:49380) 2024-05-28T14:09:18.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.893+0000 7f9d01e00640 1 -- 172.21.15.42:0/4284695939 learned_addr learned my addr 172.21.15.42:0/4284695939 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:18.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.893+0000 7f9d01400640 1 --2- 172.21.15.42:0/4284695939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cfc15b350 0x7f9cfc06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:18.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.893+0000 7f9d00a00640 1 --2- 172.21.15.42:0/4284695939 >> v2:172.21.15.63:3300/0 conn(0x7f9cfc072d40 0x7f9cfc0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:18.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.897+0000 7f9d01e00640 1 -- 172.21.15.42:0/4284695939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cfc15b350 msgr2=0x7f9cfc06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:18.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.897+0000 7f9d01e00640 1 --2- 172.21.15.42:0/4284695939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cfc15b350 0x7f9cfc06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:18.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.897+0000 7f9d01e00640 1 -- 172.21.15.42:0/4284695939 >> v2:172.21.15.63:3300/0 conn(0x7f9cfc072d40 msgr2=0x7f9cfc0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:18.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.897+0000 7f9d01e00640 1 --2- 172.21.15.42:0/4284695939 >> v2:172.21.15.63:3300/0 conn(0x7f9cfc072d40 0x7f9cfc0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:18.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.897+0000 7f9d01e00640 1 -- 172.21.15.42:0/4284695939 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cfc0ae3a0 con 0x7f9cfc0737c0 2024-05-28T14:09:18.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.897+0000 7f9d01e00640 1 --2- 172.21.15.42:0/4284695939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cfc0737c0 0x7f9cfc0d0c30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0d12d0 tx=0x7f9cf401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34702a1c8e2bffa server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:18.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.897+0000 7f9cfbe00640 1 -- 172.21.15.42:0/4284695939 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf40a5660 con 0x7f9cfc0737c0 2024-05-28T14:09:18.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.897+0000 7f9cfbe00640 1 -- 172.21.15.42:0/4284695939 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf40a5800 con 0x7f9cfc0737c0 2024-05-28T14:09:18.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.897+0000 7f9cfbe00640 1 -- 172.21.15.42:0/4284695939 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf40a5bd0 con 0x7f9cfc0737c0 2024-05-28T14:09:18.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.897+0000 7f9d03e00640 1 -- 172.21.15.42:0/4284695939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cfc0737c0 msgr2=0x7f9cfc0d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 --2- 172.21.15.42:0/4284695939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cfc0737c0 0x7f9cfc0d0c30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0d12d0 tx=0x7f9cf401c960 comp rx=0 tx=0).stop 2024-05-28T14:09:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 -- 172.21.15.42:0/4284695939 shutdown_connections 2024-05-28T14:09:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 --2- 172.21.15.42:0/4284695939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cfc0737c0 0x7f9cfc0d0c30 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 --2- 172.21.15.42:0/4284695939 >> v2:172.21.15.63:3300/0 conn(0x7f9cfc072d40 0x7f9cfc0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 --2- 172.21.15.42:0/4284695939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cfc15b350 0x7f9cfc06dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 -- 172.21.15.42:0/4284695939 >> 172.21.15.42:0/4284695939 conn(0x7f9cfc06bdf0 msgr2=0x7f9cfc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 -- 172.21.15.42:0/4284695939 shutdown_connections 2024-05-28T14:09:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 -- 172.21.15.42:0/4284695939 wait complete. 2024-05-28T14:09:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 Processor -- start 2024-05-28T14:09:18.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 -- start start 2024-05-28T14:09:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cfc072d40 0x7f9cfc15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cfc0737c0 0x7f9cfc14da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cfc15b350 0x7f9cfc152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc0af7f0 con 0x7f9cfc072d40 2024-05-28T14:09:18.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc0af670 con 0x7f9cfc15b350 2024-05-28T14:09:18.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d03e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9cfc0af970 con 0x7f9cfc0737c0 2024-05-28T14:09:18.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d01400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cfc072d40 0x7f9cfc15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:18.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d01400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cfc072d40 0x7f9cfc15b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49390/0 (socket says 172.21.15.42:49390) 2024-05-28T14:09:18.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d01400640 1 -- 172.21.15.42:0/872222565 learned_addr learned my addr 172.21.15.42:0/872222565 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:18.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d01e00640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cfc15b350 0x7f9cfc152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:18.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d00a00640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cfc0737c0 0x7f9cfc14da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:18.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d01400640 1 -- 172.21.15.42:0/872222565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cfc0737c0 msgr2=0x7f9cfc14da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:18.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d01400640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cfc0737c0 0x7f9cfc14da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:18.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d01400640 1 -- 172.21.15.42:0/872222565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cfc15b350 msgr2=0x7f9cfc152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:18.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d01400640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cfc15b350 0x7f9cfc152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:18.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d01400640 1 -- 172.21.15.42:0/872222565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cfc15c0d0 con 0x7f9cfc072d40 2024-05-28T14:09:18.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d01e00640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cfc15b350 0x7f9cfc152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:18.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d00a00640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cfc0737c0 0x7f9cfc14da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:18.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.901+0000 7f9d01400640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cfc072d40 0x7f9cfc15b9d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000e930 tx=0x7f9cf000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:18.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.905+0000 7f9cf9600640 1 -- 172.21.15.42:0/872222565 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0017070 con 0x7f9cfc072d40 2024-05-28T14:09:18.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.905+0000 7f9cf9600640 1 -- 172.21.15.42:0/872222565 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf000ca00 con 0x7f9cfc072d40 2024-05-28T14:09:18.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.905+0000 7f9cf9600640 1 -- 172.21.15.42:0/872222565 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000cdb0 con 0x7f9cfc072d40 2024-05-28T14:09:18.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.905+0000 7f9d03e00640 1 -- 172.21.15.42:0/872222565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cfc15c3b0 con 0x7f9cfc072d40 2024-05-28T14:09:18.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.905+0000 7f9d03e00640 1 -- 172.21.15.42:0/872222565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cfc15c970 con 0x7f9cfc072d40 2024-05-28T14:09:18.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.905+0000 7f9d03e00640 1 -- 172.21.15.42:0/872222565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cfc14f650 con 0x7f9cfc072d40 2024-05-28T14:09:18.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.905+0000 7f9cf9600640 1 -- 172.21.15.42:0/872222565 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f9cf001a070 con 0x7f9cfc072d40 2024-05-28T14:09:18.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.905+0000 7f9cf9600640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9cdc0694f0 0x7f9cdc06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:18.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.905+0000 7f9cf9600640 1 -- 172.21.15.42:0/872222565 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5881+0+0 (secure 0 0 0) 0x7f9cf0085890 con 0x7f9cfc072d40 2024-05-28T14:09:18.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.905+0000 7f9d00a00640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9cdc0694f0 0x7f9cdc06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:18.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.909+0000 7f9d00a00640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9cdc0694f0 0x7f9cdc06b9b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0b3a70 tx=0x7f9ce8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:18.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:18.909+0000 7f9cf9600640 1 -- 172.21.15.42:0/872222565 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9cfc14f650 con 0x7f9cfc072d40 2024-05-28T14:09:19.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.077+0000 7f9d03e00640 1 -- 172.21.15.42:0/872222565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9cfc068a90 con 0x7f9cfc072d40 2024-05-28T14:09:19.310 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:09:19.313 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:09:19.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.677+0000 7f9cf9600640 1 -- 172.21.15.42:0/872222565 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) ==== 181+0+0 (secure 0 0 0) 0x7f9cf0057670 con 0x7f9cfc072d40 2024-05-28T14:09:19.659 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-05-28T14:09:19.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.681+0000 7f9d03e00640 1 -- 172.21.15.42:0/872222565 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9cdc0694f0 msgr2=0x7f9cdc06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:19.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.681+0000 7f9d03e00640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9cdc0694f0 0x7f9cdc06b9b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0b3a70 tx=0x7f9ce8008040 comp rx=0 tx=0).stop 2024-05-28T14:09:19.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.681+0000 7f9d03e00640 1 -- 172.21.15.42:0/872222565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cfc072d40 msgr2=0x7f9cfc15b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:19.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.681+0000 7f9d03e00640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cfc072d40 0x7f9cfc15b9d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000e930 tx=0x7f9cf000edf0 comp rx=0 tx=0).stop 2024-05-28T14:09:19.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.681+0000 7f9d03e00640 1 -- 172.21.15.42:0/872222565 shutdown_connections 2024-05-28T14:09:19.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.681+0000 7f9d03e00640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9cdc0694f0 0x7f9cdc06b9b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:19.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.681+0000 7f9d03e00640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cfc15b350 0x7f9cfc152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:19.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.681+0000 7f9d03e00640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cfc0737c0 0x7f9cfc14da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:19.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.681+0000 7f9d03e00640 1 --2- 172.21.15.42:0/872222565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9cfc072d40 0x7f9cfc15b9d0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:19.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.681+0000 7f9d03e00640 1 -- 172.21.15.42:0/872222565 >> 172.21.15.42:0/872222565 conn(0x7f9cfc06bdf0 msgr2=0x7f9cfc165a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:19.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.681+0000 7f9d03e00640 1 -- 172.21.15.42:0/872222565 shutdown_connections 2024-05-28T14:09:19.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.681+0000 7f9d03e00640 1 -- 172.21.15.42:0/872222565 wait complete. 2024-05-28T14:09:19.691 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-28T14:09:20.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.857+0000 7f7d26600640 1 Processor -- start 2024-05-28T14:09:20.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.857+0000 7f7d26600640 1 -- start start 2024-05-28T14:09:20.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.861+0000 7f7d26600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7d200735e0 0x7f7d200b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:20.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.861+0000 7f7d26600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d200adf70 con 0x7f7d200aeb70 2024-05-28T14:09:20.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.861+0000 7f7d26600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d20099930 con 0x7f7d20073200 2024-05-28T14:09:20.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.861+0000 7f7d26600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f7d20073e20 con 0x7f7d200735e0 2024-05-28T14:09:20.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.861+0000 7f7d17e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7d200735e0 0x7f7d200b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:20.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.861+0000 7f7d17e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7d200735e0 0x7f7d200b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49406/0 (socket says 172.21.15.42:49406) 2024-05-28T14:09:20.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.861+0000 7f7d17e00640 1 -- 172.21.15.42:0/4004341662 learned_addr learned my addr 172.21.15.42:0/4004341662 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:20.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.861+0000 7f7d25c00640 1 --1- 172.21.15.42:0/4004341662 >> v1:172.21.15.42:6789/0 conn(0x7f7d200aeb70 0x7f7d200ad860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56386/0 (socket says 172.21.15.42:56386) 2024-05-28T14:09:20.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:19.861+0000 7f7d1fe00640 1 -- 172.21.15.42:0/4004341662 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2834387925 0 0) 0x7f7d20099930 con 0x7f7d20073200 2024-05-28T14:09:20.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.025+0000 7f7d1fe00640 1 -- 172.21.15.42:0/4004341662 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d04003510 con 0x7f7d20073200 2024-05-28T14:09:20.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.025+0000 7f7d17e00640 1 -- 172.21.15.42:0/4004341662 >> v1:172.21.15.63:6789/0 conn(0x7f7d20073200 legacy=0x7f7d200ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:20.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.025+0000 7f7d17e00640 1 -- 172.21.15.42:0/4004341662 >> v1:172.21.15.42:6789/0 conn(0x7f7d200aeb70 legacy=0x7f7d200ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:20.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.025+0000 7f7d17e00640 1 -- 172.21.15.42:0/4004341662 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d20099bd0 con 0x7f7d200735e0 2024-05-28T14:09:20.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.025+0000 7f7d17e00640 1 --2- 172.21.15.42:0/4004341662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d200735e0 0x7f7d200b2990 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f7d20099b00 tx=0x7f7d0801ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5397dc60e173a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:20.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.025+0000 7f7d1fe00640 1 -- 172.21.15.42:0/4004341662 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d08007ad0 con 0x7f7d200735e0 2024-05-28T14:09:20.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.025+0000 7f7d1fe00640 1 -- 172.21.15.42:0/4004341662 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d08007c70 con 0x7f7d200735e0 2024-05-28T14:09:20.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.025+0000 7f7d1fe00640 1 -- 172.21.15.42:0/4004341662 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d080a8a70 con 0x7f7d200735e0 2024-05-28T14:09:20.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.029+0000 7f7d26600640 1 -- 172.21.15.42:0/4004341662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d200735e0 msgr2=0x7f7d200b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:20.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.029+0000 7f7d26600640 1 --2- 172.21.15.42:0/4004341662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d200735e0 0x7f7d200b2990 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f7d20099b00 tx=0x7f7d0801ec40 comp rx=0 tx=0).stop 2024-05-28T14:09:20.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.029+0000 7f7d26600640 1 -- 172.21.15.42:0/4004341662 shutdown_connections 2024-05-28T14:09:20.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.029+0000 7f7d26600640 1 --2- 172.21.15.42:0/4004341662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d200735e0 0x7f7d200b2990 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:20.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.029+0000 7f7d26600640 1 -- 172.21.15.42:0/4004341662 >> 172.21.15.42:0/4004341662 conn(0x7f7d200b1e70 msgr2=0x7f7d200b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:20.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.029+0000 7f7d26600640 1 -- 172.21.15.42:0/4004341662 shutdown_connections 2024-05-28T14:09:20.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.029+0000 7f7d26600640 1 -- 172.21.15.42:0/4004341662 wait complete. 2024-05-28T14:09:20.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.029+0000 7f7d26600640 1 Processor -- start 2024-05-28T14:09:20.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.029+0000 7f7d26600640 1 -- start start 2024-05-28T14:09:20.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d26600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d20073200 0x7f7d20200bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:20.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d26600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d200735e0 0x7f7d200c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:20.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d26600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d200aeb70 0x7f7d20096990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:20.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d26600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20099eb0 con 0x7f7d200735e0 2024-05-28T14:09:20.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d26600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7d200af130 con 0x7f7d20073200 2024-05-28T14:09:20.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d26600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d200d0490 con 0x7f7d200aeb70 2024-05-28T14:09:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d17e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d200735e0 0x7f7d200c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d17e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d200735e0 0x7f7d200c6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49422/0 (socket says 172.21.15.42:49422) 2024-05-28T14:09:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d17e00640 1 -- 172.21.15.42:0/1395139096 learned_addr learned my addr 172.21.15.42:0/1395139096 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d25200640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d20073200 0x7f7d20200bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d25c00640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d200aeb70 0x7f7d20096990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d17e00640 1 -- 172.21.15.42:0/1395139096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d200aeb70 msgr2=0x7f7d20096990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d17e00640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d200aeb70 0x7f7d20096990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d17e00640 1 -- 172.21.15.42:0/1395139096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d20073200 msgr2=0x7f7d20200bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:20.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d17e00640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d20073200 0x7f7d20200bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:20.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d17e00640 1 -- 172.21.15.42:0/1395139096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2009a370 con 0x7f7d200735e0 2024-05-28T14:09:20.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d25c00640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d200aeb70 0x7f7d20096990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:20.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d17e00640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d200735e0 0x7f7d200c6260 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f7d080b2040 tx=0x7f7d0801b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:20.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d1d600640 1 -- 172.21.15.42:0/1395139096 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d080090d0 con 0x7f7d200735e0 2024-05-28T14:09:20.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d1d600640 1 -- 172.21.15.42:0/1395139096 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d08009270 con 0x7f7d200735e0 2024-05-28T14:09:20.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d1d600640 1 -- 172.21.15.42:0/1395139096 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d08009640 con 0x7f7d200735e0 2024-05-28T14:09:20.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d26600640 1 -- 172.21.15.42:0/1395139096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2009a650 con 0x7f7d200735e0 2024-05-28T14:09:20.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d26600640 1 -- 172.21.15.42:0/1395139096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2009ac10 con 0x7f7d200735e0 2024-05-28T14:09:20.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.033+0000 7f7d26600640 1 -- 172.21.15.42:0/1395139096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d20068a90 con 0x7f7d200735e0 2024-05-28T14:09:20.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.037+0000 7f7d1d600640 1 -- 172.21.15.42:0/1395139096 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f7d080c3020 con 0x7f7d200735e0 2024-05-28T14:09:20.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.037+0000 7f7d1d600640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7cfc0694f0 0x7f7cfc06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:20.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.037+0000 7f7d1d600640 1 -- 172.21.15.42:0/1395139096 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5443+0+0 (secure 0 0 0) 0x7f7d08123390 con 0x7f7d200735e0 2024-05-28T14:09:20.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.037+0000 7f7d25200640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7cfc0694f0 0x7f7cfc06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:20.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.037+0000 7f7d25200640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7cfc0694f0 0x7f7cfc06b9b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7d100011f0 tx=0x7f7d10009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:20.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.041+0000 7f7d1d600640 1 -- 172.21.15.42:0/1395139096 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7d080ca020 con 0x7f7d200735e0 2024-05-28T14:09:20.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.213+0000 7f7d26600640 1 -- 172.21.15.42:0/1395139096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7d200b3a50 con 0x7f7d200735e0 2024-05-28T14:09:20.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.677+0000 7f7d1d600640 1 -- 172.21.15.42:0/1395139096 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) ==== 169+0+0 (secure 0 0 0) 0x7f7d08007e20 con 0x7f7d200735e0 2024-05-28T14:09:20.661 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-05-28T14:09:20.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.681+0000 7f7d26600640 1 -- 172.21.15.42:0/1395139096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7cfc0694f0 msgr2=0x7f7cfc06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:20.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.681+0000 7f7d26600640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7cfc0694f0 0x7f7cfc06b9b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7d100011f0 tx=0x7f7d10009040 comp rx=0 tx=0).stop 2024-05-28T14:09:20.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.681+0000 7f7d26600640 1 -- 172.21.15.42:0/1395139096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d200735e0 msgr2=0x7f7d200c6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:20.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.681+0000 7f7d26600640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d200735e0 0x7f7d200c6260 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f7d080b2040 tx=0x7f7d0801b9e0 comp rx=0 tx=0).stop 2024-05-28T14:09:20.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.681+0000 7f7d26600640 1 -- 172.21.15.42:0/1395139096 shutdown_connections 2024-05-28T14:09:20.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.681+0000 7f7d26600640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7cfc0694f0 0x7f7cfc06b9b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:20.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.681+0000 7f7d26600640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d200aeb70 0x7f7d20096990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:20.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.681+0000 7f7d26600640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d200735e0 0x7f7d200c6260 secure :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f7d080b2040 tx=0x7f7d0801b9e0 comp rx=0 tx=0).stop 2024-05-28T14:09:20.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.681+0000 7f7d26600640 1 --2- 172.21.15.42:0/1395139096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d20073200 0x7f7d20200bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:20.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.681+0000 7f7d26600640 1 -- 172.21.15.42:0/1395139096 >> 172.21.15.42:0/1395139096 conn(0x7f7d200b1e70 msgr2=0x7f7d200b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:20.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.681+0000 7f7d26600640 1 -- 172.21.15.42:0/1395139096 shutdown_connections 2024-05-28T14:09:20.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.681+0000 7f7d26600640 1 -- 172.21.15.42:0/1395139096 wait complete. 2024-05-28T14:09:20.697 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-28T14:09:20.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.861+0000 7f46f6e00640 1 Processor -- start 2024-05-28T14:09:20.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46f6e00640 1 -- start start 2024-05-28T14:09:20.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46f6e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f46f00c6fc0 0x7f46f00c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:20.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46f6e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f46f00ce850 0x7f46f006b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:20.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46f6e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f006f160 con 0x7f46f00b3af0 2024-05-28T14:09:20.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46f6e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f46f00705b0 con 0x7f46f00ce850 2024-05-28T14:09:20.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46f6e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f46f0070850 con 0x7f46f00c6fc0 2024-05-28T14:09:20.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46f4e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f46f00ce850 0x7f46f006b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:20.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46ef400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f46f00b3af0 0x7f46f00ce140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56390/0 (socket says 172.21.15.42:56390) 2024-05-28T14:09:20.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46ef400640 1 -- 172.21.15.42:0/2083330521 learned_addr learned my addr 172.21.15.42:0/2083330521 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:20.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46efe00640 1 --2- 172.21.15.42:0/2083330521 >> v2:172.21.15.63:3300/0 conn(0x7f46f00c6fc0 0x7f46f00c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:20.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46eea00640 1 -- 172.21.15.42:0/2083330521 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 596241147 0 0) 0x7f46f006f160 con 0x7f46f00b3af0 2024-05-28T14:09:20.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46eea00640 1 -- 172.21.15.42:0/2083330521 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46d4003530 con 0x7f46f00b3af0 2024-05-28T14:09:20.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46f4e00640 1 -- 172.21.15.42:0/2083330521 >> v2:172.21.15.63:3300/0 conn(0x7f46f00c6fc0 msgr2=0x7f46f00c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:20.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46f4e00640 1 --2- 172.21.15.42:0/2083330521 >> v2:172.21.15.63:3300/0 conn(0x7f46f00c6fc0 0x7f46f00c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:20.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46f4e00640 1 -- 172.21.15.42:0/2083330521 >> v1:172.21.15.42:6789/0 conn(0x7f46f00b3af0 legacy=0x7f46f00ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:20.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46f4e00640 1 -- 172.21.15.42:0/2083330521 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f0070b30 con 0x7f46f00ce850 2024-05-28T14:09:20.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.865+0000 7f46efe00640 1 --2- 172.21.15.42:0/2083330521 >> v2:172.21.15.63:3300/0 conn(0x7f46f00c6fc0 0x7f46f00c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:20.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.869+0000 7f46f4e00640 1 --2- 172.21.15.42:0/2083330521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46f00ce850 0x7f46f006b9f0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009d90 tx=0x7f46e801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c7128c280ab430 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.869+0000 7f46eea00640 1 -- 172.21.15.42:0/2083330521 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e80b2070 con 0x7f46f00ce850 2024-05-28T14:09:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.869+0000 7f46eea00640 1 -- 172.21.15.42:0/2083330521 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e80a5750 con 0x7f46f00ce850 2024-05-28T14:09:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.869+0000 7f46eea00640 1 -- 172.21.15.42:0/2083330521 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e80a5b00 con 0x7f46f00ce850 2024-05-28T14:09:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.869+0000 7f46f6e00640 1 -- 172.21.15.42:0/2083330521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46f00ce850 msgr2=0x7f46f006b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.869+0000 7f46f6e00640 1 --2- 172.21.15.42:0/2083330521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46f00ce850 0x7f46f006b9f0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009d90 tx=0x7f46e801c8a0 comp rx=0 tx=0).stop 2024-05-28T14:09:20.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.869+0000 7f46f6e00640 1 -- 172.21.15.42:0/2083330521 shutdown_connections 2024-05-28T14:09:20.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.869+0000 7f46f6e00640 1 --2- 172.21.15.42:0/2083330521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46f00ce850 0x7f46f006b9f0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:20.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.869+0000 7f46f6e00640 1 --2- 172.21.15.42:0/2083330521 >> v2:172.21.15.63:3300/0 conn(0x7f46f00c6fc0 0x7f46f00c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:20.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.869+0000 7f46f6e00640 1 -- 172.21.15.42:0/2083330521 >> 172.21.15.42:0/2083330521 conn(0x7f46f00b2fd0 msgr2=0x7f46f00b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:20.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.869+0000 7f46f6e00640 1 -- 172.21.15.42:0/2083330521 shutdown_connections 2024-05-28T14:09:20.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.869+0000 7f46f6e00640 1 -- 172.21.15.42:0/2083330521 wait complete. 2024-05-28T14:09:20.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.869+0000 7f46f6e00640 1 Processor -- start 2024-05-28T14:09:20.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.869+0000 7f46f6e00640 1 -- start start 2024-05-28T14:09:20.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46f6e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46f00b3af0 0x7f46f00af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:20.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46f6e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46f00c6fc0 0x7f46f014c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:20.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46f6e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46f00ce850 0x7f46f01567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:20.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46f6e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46f00d1000 con 0x7f46f00b3af0 2024-05-28T14:09:20.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46f6e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46f00c6780 con 0x7f46f00c6fc0 2024-05-28T14:09:20.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46f6e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46f00d1180 con 0x7f46f00ce850 2024-05-28T14:09:20.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46efe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46f00b3af0 0x7f46f00af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:20.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46efe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46f00b3af0 0x7f46f00af6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49436/0 (socket says 172.21.15.42:49436) 2024-05-28T14:09:20.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46efe00640 1 -- 172.21.15.42:0/2153707518 learned_addr learned my addr 172.21.15.42:0/2153707518 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:20.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46ef400640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46f00c6fc0 0x7f46f014c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:20.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46f4e00640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46f00ce850 0x7f46f01567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:20.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46efe00640 1 -- 172.21.15.42:0/2153707518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46f00ce850 msgr2=0x7f46f01567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:20.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46efe00640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46f00ce850 0x7f46f01567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:20.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46efe00640 1 -- 172.21.15.42:0/2153707518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46f00c6fc0 msgr2=0x7f46f014c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:20.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46efe00640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46f00c6fc0 0x7f46f014c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:20.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46efe00640 1 -- 172.21.15.42:0/2153707518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f015d2a0 con 0x7f46f00b3af0 2024-05-28T14:09:20.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46ef400640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46f00c6fc0 0x7f46f014c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:20.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46f4e00640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46f00ce850 0x7f46f01567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:20.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46efe00640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46f00b3af0 0x7f46f00af6b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f46e000e840 tx=0x7f46e000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:20.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46e7e00640 1 -- 172.21.15.42:0/2153707518 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0017070 con 0x7f46f00b3af0 2024-05-28T14:09:20.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46e7e00640 1 -- 172.21.15.42:0/2153707518 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e000c8d0 con 0x7f46f00b3af0 2024-05-28T14:09:20.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46e7e00640 1 -- 172.21.15.42:0/2153707518 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e000f040 con 0x7f46f00b3af0 2024-05-28T14:09:20.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46f6e00640 1 -- 172.21.15.42:0/2153707518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f015c4e0 con 0x7f46f00b3af0 2024-05-28T14:09:20.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46f6e00640 1 -- 172.21.15.42:0/2153707518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f015caa0 con 0x7f46f00b3af0 2024-05-28T14:09:20.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46f6e00640 1 -- 172.21.15.42:0/2153707518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f009f630 con 0x7f46f00b3af0 2024-05-28T14:09:20.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.873+0000 7f46e7e00640 1 -- 172.21.15.42:0/2153707518 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f46e001a070 con 0x7f46f00b3af0 2024-05-28T14:09:20.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.877+0000 7f46e7e00640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f46d00694f0 0x7f46d006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:20.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.877+0000 7f46e7e00640 1 -- 172.21.15.42:0/2153707518 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4971+0+0 (secure 0 0 0) 0x7f46e0085470 con 0x7f46f00b3af0 2024-05-28T14:09:20.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.877+0000 7f46ef400640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f46d00694f0 0x7f46d006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:20.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.877+0000 7f46ef400640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f46d00694f0 0x7f46d006b9b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f46f0158150 tx=0x7f46dc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:20.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:20.881+0000 7f46e7e00640 1 -- 172.21.15.42:0/2153707518 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f46e0057190 con 0x7f46f00b3af0 2024-05-28T14:09:21.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.049+0000 7f46f6e00640 1 -- 172.21.15.42:0/2153707518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f46f00afe20 con 0x7f46f00b3af0 2024-05-28T14:09:21.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.049+0000 7f46e7e00640 1 -- 172.21.15.42:0/2153707518 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f46e0056a90 con 0x7f46f00b3af0 2024-05-28T14:09:21.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.053+0000 7f46f6e00640 1 -- 172.21.15.42:0/2153707518 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f46d00694f0 msgr2=0x7f46d006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:21.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.053+0000 7f46f6e00640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f46d00694f0 0x7f46d006b9b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f46f0158150 tx=0x7f46dc009040 comp rx=0 tx=0).stop 2024-05-28T14:09:21.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.053+0000 7f46f6e00640 1 -- 172.21.15.42:0/2153707518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46f00b3af0 msgr2=0x7f46f00af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.053+0000 7f46f6e00640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46f00b3af0 0x7f46f00af6b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f46e000e840 tx=0x7f46e000ed00 comp rx=0 tx=0).stop 2024-05-28T14:09:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.053+0000 7f46f6e00640 1 -- 172.21.15.42:0/2153707518 shutdown_connections 2024-05-28T14:09:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.053+0000 7f46f6e00640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f46d00694f0 0x7f46d006b9b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.053+0000 7f46f6e00640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46f00ce850 0x7f46f01567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.053+0000 7f46f6e00640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46f00c6fc0 0x7f46f014c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.053+0000 7f46f6e00640 1 --2- 172.21.15.42:0/2153707518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46f00b3af0 0x7f46f00af6b0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.053+0000 7f46f6e00640 1 -- 172.21.15.42:0/2153707518 >> 172.21.15.42:0/2153707518 conn(0x7f46f00b2fd0 msgr2=0x7f46f00b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.053+0000 7f46f6e00640 1 -- 172.21.15.42:0/2153707518 shutdown_connections 2024-05-28T14:09:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.053+0000 7f46f6e00640 1 -- 172.21.15.42:0/2153707518 wait complete. 2024-05-28T14:09:21.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/snapshot_clone_no_wait 2024-05-28T14:09:21.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6fa5000640 1 Processor -- start 2024-05-28T14:09:21.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6fa5000640 1 -- start start 2024-05-28T14:09:21.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6fa5000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6fa0144f80 0x7f6fa0072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:21.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6fa5000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6fa006ee20 0x7f6fa006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:21.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6fa5000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fa00afe40 con 0x7f6fa015b9b0 2024-05-28T14:09:21.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6fa5000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa006f7c0 con 0x7f6fa006ee20 2024-05-28T14:09:21.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6fa5000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa00724d0 con 0x7f6fa0144f80 2024-05-28T14:09:21.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6f97e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6fa006ee20 0x7f6fa006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:21.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6f97e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6fa006ee20 0x7f6fa006f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49448/0 (socket says 172.21.15.42:49448) 2024-05-28T14:09:21.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6f9de00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6fa0144f80 0x7f6fa0072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:21.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6f97e00640 1 -- 172.21.15.42:0/1460545200 learned_addr learned my addr 172.21.15.42:0/1460545200 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:21.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6f97e00640 1 -- 172.21.15.42:0/1460545200 >> v2:172.21.15.63:3300/0 conn(0x7f6fa0144f80 msgr2=0x7f6fa0072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:21.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6f97e00640 1 --2- 172.21.15.42:0/1460545200 >> v2:172.21.15.63:3300/0 conn(0x7f6fa0144f80 0x7f6fa0072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6f97e00640 1 -- 172.21.15.42:0/1460545200 >> v1:172.21.15.63:6789/0 conn(0x7f6fa015b9b0 legacy=0x7f6fa015be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:21.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.217+0000 7f6f97e00640 1 -- 172.21.15.42:0/1460545200 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa015c600 con 0x7f6fa006ee20 2024-05-28T14:09:21.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.221+0000 7f6f97e00640 1 --2- 172.21.15.42:0/1460545200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fa006ee20 0x7f6fa006f280 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0071760 tx=0x7f6f8801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb836d5cf4fada3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:21.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.221+0000 7f6f9d400640 1 -- 172.21.15.42:0/1460545200 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f880ab610 con 0x7f6fa006ee20 2024-05-28T14:09:21.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.221+0000 7f6f9d400640 1 -- 172.21.15.42:0/1460545200 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f880b0070 con 0x7f6fa006ee20 2024-05-28T14:09:21.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.221+0000 7f6f9d400640 1 -- 172.21.15.42:0/1460545200 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f880ab9d0 con 0x7f6fa006ee20 2024-05-28T14:09:21.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.221+0000 7f6fa5000640 1 -- 172.21.15.42:0/1460545200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fa006ee20 msgr2=0x7f6fa006f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:21.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.221+0000 7f6fa5000640 1 --2- 172.21.15.42:0/1460545200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fa006ee20 0x7f6fa006f280 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0071760 tx=0x7f6f8801c520 comp rx=0 tx=0).stop 2024-05-28T14:09:21.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.221+0000 7f6fa5000640 1 -- 172.21.15.42:0/1460545200 shutdown_connections 2024-05-28T14:09:21.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.221+0000 7f6fa5000640 1 --2- 172.21.15.42:0/1460545200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fa006ee20 0x7f6fa006f280 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.221+0000 7f6fa5000640 1 --2- 172.21.15.42:0/1460545200 >> v2:172.21.15.63:3300/0 conn(0x7f6fa0144f80 0x7f6fa0072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.221+0000 7f6fa5000640 1 -- 172.21.15.42:0/1460545200 >> 172.21.15.42:0/1460545200 conn(0x7f6fa00b21f0 msgr2=0x7f6fa00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:21.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.221+0000 7f6fa5000640 1 -- 172.21.15.42:0/1460545200 shutdown_connections 2024-05-28T14:09:21.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.221+0000 7f6fa5000640 1 -- 172.21.15.42:0/1460545200 wait complete. 2024-05-28T14:09:21.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.221+0000 7f6fa5000640 1 Processor -- start 2024-05-28T14:09:21.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6fa5000640 1 -- start start 2024-05-28T14:09:21.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6fa5000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fa006ee20 0x7f6fa01510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:21.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6fa5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fa0144f80 0x7f6fa013e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:21.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6fa5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fa015b9b0 0x7f6fa0134f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:21.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6fa5000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa0072ea0 con 0x7f6fa0144f80 2024-05-28T14:09:21.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6fa5000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa00b4010 con 0x7f6fa015b9b0 2024-05-28T14:09:21.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6fa5000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa0151610 con 0x7f6fa006ee20 2024-05-28T14:09:21.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6f97e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fa0144f80 0x7f6fa013e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:21.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6f9de00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fa006ee20 0x7f6fa01510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:21.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6f9de00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fa006ee20 0x7f6fa01510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:57988/0 (socket says 172.21.15.42:57988) 2024-05-28T14:09:21.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6f9de00640 1 -- 172.21.15.42:0/2393908912 learned_addr learned my addr 172.21.15.42:0/2393908912 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:21.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6f9e800640 1 --2- 172.21.15.42:0/2393908912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fa015b9b0 0x7f6fa0134f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:21.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6f97e00640 1 -- 172.21.15.42:0/2393908912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fa006ee20 msgr2=0x7f6fa01510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:21.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6f97e00640 1 --2- 172.21.15.42:0/2393908912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fa006ee20 0x7f6fa01510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6f97e00640 1 -- 172.21.15.42:0/2393908912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fa015b9b0 msgr2=0x7f6fa0134f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:21.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6f97e00640 1 --2- 172.21.15.42:0/2393908912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fa015b9b0 0x7f6fa0134f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6f97e00640 1 -- 172.21.15.42:0/2393908912 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa0133ed0 con 0x7f6fa0144f80 2024-05-28T14:09:21.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6f97e00640 1 --2- 172.21.15.42:0/2393908912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fa0144f80 0x7f6fa013e820 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6f880ac000 tx=0x7f6f8801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:21.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6f96000640 1 -- 172.21.15.42:0/2393908912 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f880be5f0 con 0x7f6fa0144f80 2024-05-28T14:09:21.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6f96000640 1 -- 172.21.15.42:0/2393908912 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f880be7b0 con 0x7f6fa0144f80 2024-05-28T14:09:21.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6f96000640 1 -- 172.21.15.42:0/2393908912 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f880beba0 con 0x7f6fa0144f80 2024-05-28T14:09:21.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6fa5000640 1 -- 172.21.15.42:0/2393908912 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa01341b0 con 0x7f6fa0144f80 2024-05-28T14:09:21.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6fa5000640 1 -- 172.21.15.42:0/2393908912 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa0134770 con 0x7f6fa0144f80 2024-05-28T14:09:21.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.225+0000 7f6fa5000640 1 -- 172.21.15.42:0/2393908912 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa00738b0 con 0x7f6fa0144f80 2024-05-28T14:09:21.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.229+0000 7f6f96000640 1 -- 172.21.15.42:0/2393908912 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f6f880b43f0 con 0x7f6fa0144f80 2024-05-28T14:09:21.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.229+0000 7f6f96000640 1 --2- 172.21.15.42:0/2393908912 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f7c0694f0 0x7f6f7c06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:21.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.229+0000 7f6f96000640 1 -- 172.21.15.42:0/2393908912 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4971+0+0 (secure 0 0 0) 0x7f6f8811f930 con 0x7f6fa0144f80 2024-05-28T14:09:21.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.229+0000 7f6f9de00640 1 --2- 172.21.15.42:0/2393908912 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f7c0694f0 0x7f6f7c06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:21.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.229+0000 7f6f9de00640 1 --2- 172.21.15.42:0/2393908912 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f7c0694f0 0x7f6f7c06b9b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009c60 tx=0x7f6f90009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:21.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.233+0000 7f6f96000640 1 -- 172.21.15.42:0/2393908912 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6f880a8050 con 0x7f6fa0144f80 2024-05-28T14:09:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.445+0000 7f6fa5000640 1 -- 172.21.15.42:0/2393908912 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_no_wait"} v 0) -- 0x7f6fa015ec20 con 0x7f6fa0144f80 2024-05-28T14:09:21.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.449+0000 7f6f96000640 1 -- 172.21.15.42:0/2393908912 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_no_wait"}]=0 v21) ==== 117+0+0 (secure 0 0 0) 0x7f6f88016110 con 0x7f6fa0144f80 2024-05-28T14:09:21.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.449+0000 7f6fa5000640 1 -- 172.21.15.42:0/2393908912 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f7c0694f0 msgr2=0x7f6f7c06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:21.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.449+0000 7f6fa5000640 1 --2- 172.21.15.42:0/2393908912 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f7c0694f0 0x7f6f7c06b9b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009c60 tx=0x7f6f90009190 comp rx=0 tx=0).stop 2024-05-28T14:09:21.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.449+0000 7f6fa5000640 1 -- 172.21.15.42:0/2393908912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fa0144f80 msgr2=0x7f6fa013e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:21.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.449+0000 7f6fa5000640 1 --2- 172.21.15.42:0/2393908912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fa0144f80 0x7f6fa013e820 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6f880ac000 tx=0x7f6f8801ad60 comp rx=0 tx=0).stop 2024-05-28T14:09:21.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.449+0000 7f6fa5000640 1 -- 172.21.15.42:0/2393908912 shutdown_connections 2024-05-28T14:09:21.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.449+0000 7f6fa5000640 1 --2- 172.21.15.42:0/2393908912 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f7c0694f0 0x7f6f7c06b9b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.449+0000 7f6fa5000640 1 --2- 172.21.15.42:0/2393908912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fa015b9b0 0x7f6fa0134f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.449+0000 7f6fa5000640 1 --2- 172.21.15.42:0/2393908912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fa0144f80 0x7f6fa013e820 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.449+0000 7f6fa5000640 1 --2- 172.21.15.42:0/2393908912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fa006ee20 0x7f6fa01510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.449+0000 7f6fa5000640 1 -- 172.21.15.42:0/2393908912 >> 172.21.15.42:0/2393908912 conn(0x7f6fa00b21f0 msgr2=0x7f6fa00b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:21.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.449+0000 7f6fa5000640 1 -- 172.21.15.42:0/2393908912 shutdown_connections 2024-05-28T14:09:21.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.449+0000 7f6fa5000640 1 -- 172.21.15.42:0/2393908912 wait complete. 2024-05-28T14:09:21.458 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-05-28T14:09:21.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fbf200640 1 Processor -- start 2024-05-28T14:09:21.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fbf200640 1 -- start start 2024-05-28T14:09:21.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fbf200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fb815c970 0x7f4fb80ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:21.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fbf200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4fb806c100 0x7f4fb80d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:21.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fbf200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fb806c700 con 0x7f4fb80b3aa0 2024-05-28T14:09:21.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fbf200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4fb81611b0 con 0x7f4fb806c100 2024-05-28T14:09:21.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fbf200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb8160350 con 0x7f4fb815c970 2024-05-28T14:09:21.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fb7400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f4fb80b3aa0 0x7f4fb806b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56394/0 (socket says 172.21.15.42:56394) 2024-05-28T14:09:21.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fb7400640 1 -- 172.21.15.42:0/2549677841 learned_addr learned my addr 172.21.15.42:0/2549677841 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:21.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fbd200640 1 --2- 172.21.15.42:0/2549677841 >> v2:172.21.15.63:3300/0 conn(0x7f4fb806c100 0x7f4fb80d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:21.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fb7e00640 1 --2- 172.21.15.42:0/2549677841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fb815c970 0x7f4fb80ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:21.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fb6a00640 1 -- 172.21.15.42:0/2549677841 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2574759470 0 0) 0x7f4fb806c700 con 0x7f4fb80b3aa0 2024-05-28T14:09:21.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fb6a00640 1 -- 172.21.15.42:0/2549677841 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f9c003530 con 0x7f4fb80b3aa0 2024-05-28T14:09:21.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fb7e00640 1 -- 172.21.15.42:0/2549677841 >> v2:172.21.15.63:3300/0 conn(0x7f4fb806c100 msgr2=0x7f4fb80d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:21.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fb7e00640 1 --2- 172.21.15.42:0/2549677841 >> v2:172.21.15.63:3300/0 conn(0x7f4fb806c100 0x7f4fb80d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fb7e00640 1 -- 172.21.15.42:0/2549677841 >> v1:172.21.15.42:6789/0 conn(0x7f4fb80b3aa0 legacy=0x7f4fb806b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:21.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.617+0000 7f4fb7e00640 1 -- 172.21.15.42:0/2549677841 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb81605a0 con 0x7f4fb815c970 2024-05-28T14:09:21.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.621+0000 7f4fb7e00640 1 --2- 172.21.15.42:0/2549677841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fb815c970 0x7f4fb80ccfa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80097c0 tx=0x7f4fa801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c00c8a81ee83a36a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:21.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.621+0000 7f4fb6a00640 1 -- 172.21.15.42:0/2549677841 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa80b0070 con 0x7f4fb815c970 2024-05-28T14:09:21.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.621+0000 7f4fb6a00640 1 -- 172.21.15.42:0/2549677841 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa80ab400 con 0x7f4fb815c970 2024-05-28T14:09:21.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.621+0000 7f4fb6a00640 1 -- 172.21.15.42:0/2549677841 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa80ab7d0 con 0x7f4fb815c970 2024-05-28T14:09:21.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.621+0000 7f4fbf200640 1 -- 172.21.15.42:0/2549677841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fb815c970 msgr2=0x7f4fb80ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:21.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.621+0000 7f4fbf200640 1 --2- 172.21.15.42:0/2549677841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fb815c970 0x7f4fb80ccfa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80097c0 tx=0x7f4fa801c4a0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.621+0000 7f4fbf200640 1 -- 172.21.15.42:0/2549677841 shutdown_connections 2024-05-28T14:09:21.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.621+0000 7f4fbf200640 1 --2- 172.21.15.42:0/2549677841 >> v2:172.21.15.63:3300/0 conn(0x7f4fb806c100 0x7f4fb80d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.621+0000 7f4fbf200640 1 --2- 172.21.15.42:0/2549677841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fb815c970 0x7f4fb80ccfa0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.621+0000 7f4fbf200640 1 -- 172.21.15.42:0/2549677841 >> 172.21.15.42:0/2549677841 conn(0x7f4fb80b2f80 msgr2=0x7f4fb80b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:21.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.621+0000 7f4fbf200640 1 -- 172.21.15.42:0/2549677841 shutdown_connections 2024-05-28T14:09:21.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.621+0000 7f4fbf200640 1 -- 172.21.15.42:0/2549677841 wait complete. 2024-05-28T14:09:21.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fbf200640 1 Processor -- start 2024-05-28T14:09:21.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fbf200640 1 -- start start 2024-05-28T14:09:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fbf200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fb806c100 0x7f4fb81574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fbf200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fb80b3aa0 0x7f4fb80b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fbf200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fb81569a0 0x7f4fb80c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fbf200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8169870 con 0x7f4fb806c100 2024-05-28T14:09:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fbf200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81696f0 con 0x7f4fb80b3aa0 2024-05-28T14:09:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fbf200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb81699f0 con 0x7f4fb81569a0 2024-05-28T14:09:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fb806c100 0x7f4fb81574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fb806c100 0x7f4fb81574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49468/0 (socket says 172.21.15.42:49468) 2024-05-28T14:09:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fb7e00640 1 -- 172.21.15.42:0/90822095 learned_addr learned my addr 172.21.15.42:0/90822095 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fbd200640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fb81569a0 0x7f4fb80c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:21.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fb7400640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fb80b3aa0 0x7f4fb80b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fb7e00640 1 -- 172.21.15.42:0/90822095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fb81569a0 msgr2=0x7f4fb80c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fb7e00640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fb81569a0 0x7f4fb80c6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fb7e00640 1 -- 172.21.15.42:0/90822095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fb80b3aa0 msgr2=0x7f4fb80b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fb7e00640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fb80b3aa0 0x7f4fb80b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fb7e00640 1 -- 172.21.15.42:0/90822095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb8068290 con 0x7f4fb806c100 2024-05-28T14:09:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fb7400640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fb80b3aa0 0x7f4fb80b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fb7e00640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fb806c100 0x7f4fb81574e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80ae040 tx=0x7f4fa801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fafe00640 1 -- 172.21.15.42:0/90822095 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa80b0050 con 0x7f4fb806c100 2024-05-28T14:09:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fafe00640 1 -- 172.21.15.42:0/90822095 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa80bc070 con 0x7f4fb806c100 2024-05-28T14:09:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fbf200640 1 -- 172.21.15.42:0/90822095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb80b5220 con 0x7f4fb806c100 2024-05-28T14:09:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fafe00640 1 -- 172.21.15.42:0/90822095 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa80b7890 con 0x7f4fb806c100 2024-05-28T14:09:21.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.625+0000 7f4fbf200640 1 -- 172.21.15.42:0/90822095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb80c6650 con 0x7f4fb806c100 2024-05-28T14:09:21.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.629+0000 7f4fbf200640 1 -- 172.21.15.42:0/90822095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb8068a90 con 0x7f4fb806c100 2024-05-28T14:09:21.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.629+0000 7f4fafe00640 1 -- 172.21.15.42:0/90822095 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f4fa80b7a30 con 0x7f4fb806c100 2024-05-28T14:09:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.629+0000 7f4fafe00640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4f98069520 0x7f4f9806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.629+0000 7f4fafe00640 1 -- 172.21.15.42:0/90822095 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4971+0+0 (secure 0 0 0) 0x7f4fa811fe30 con 0x7f4fb806c100 2024-05-28T14:09:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.629+0000 7f4fb7400640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4f98069520 0x7f4f9806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:21.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.629+0000 7f4fb7400640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4f98069520 0x7f4f9806b9e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8159d70 tx=0x7f4fa4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:21.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.633+0000 7f4fafe00640 1 -- 172.21.15.42:0/90822095 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4fa8016090 con 0x7f4fb806c100 2024-05-28T14:09:21.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:21.897+0000 7f4fbf200640 1 -- 172.21.15.42:0/90822095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f4fb80b73b0 con 0x7f4fb806c100 2024-05-28T14:09:22.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.697+0000 7f4fafe00640 1 -- 172.21.15.42:0/90822095 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v508) ==== 174+0+0 (secure 0 0 0) 0x7f4fa80f1b50 con 0x7f4fb806c100 2024-05-28T14:09:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.701+0000 7f4fbf200640 1 -- 172.21.15.42:0/90822095 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4f98069520 msgr2=0x7f4f9806b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.701+0000 7f4fbf200640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4f98069520 0x7f4f9806b9e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8159d70 tx=0x7f4fa4009040 comp rx=0 tx=0).stop 2024-05-28T14:09:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.701+0000 7f4fbf200640 1 -- 172.21.15.42:0/90822095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fb806c100 msgr2=0x7f4fb81574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.701+0000 7f4fbf200640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fb806c100 0x7f4fb81574e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80ae040 tx=0x7f4fa801ace0 comp rx=0 tx=0).stop 2024-05-28T14:09:22.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.701+0000 7f4fbf200640 1 -- 172.21.15.42:0/90822095 shutdown_connections 2024-05-28T14:09:22.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.701+0000 7f4fbf200640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4f98069520 0x7f4f9806b9e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:22.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.701+0000 7f4fbf200640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fb81569a0 0x7f4fb80c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:22.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.701+0000 7f4fbf200640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fb80b3aa0 0x7f4fb80b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:22.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.701+0000 7f4fbf200640 1 --2- 172.21.15.42:0/90822095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fb806c100 0x7f4fb81574e0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:22.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.701+0000 7f4fbf200640 1 -- 172.21.15.42:0/90822095 >> 172.21.15.42:0/90822095 conn(0x7f4fb80b2f80 msgr2=0x7f4fb815f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:22.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.701+0000 7f4fbf200640 1 -- 172.21.15.42:0/90822095 shutdown_connections 2024-05-28T14:09:22.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.701+0000 7f4fbf200640 1 -- 172.21.15.42:0/90822095 wait complete. 2024-05-28T14:09:22.710 INFO:tasks.cephfs_test_runner:Test that progress indicated by progress bar for ongoing+pending clones ... ok 2024-05-28T14:09:22.710 INFO:tasks.cephfs_test_runner:Starting test: test_stats_are_printed (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the command "ceph fs clone status" prints progress stats 2024-05-28T14:09:22.710 INFO:tasks.cephfs_test_runner:test_stats_are_printed (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-28T14:09:22.711 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:09:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0ff200640 1 Processor -- start 2024-05-28T14:09:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0ff200640 1 -- start start 2024-05-28T14:09:22.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0ff200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe10006c100 0x7fe1000b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:22.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0ff200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe100147b40 0x7fe100145d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:22.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0ff200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1000b5bd0 con 0x7fe1000b1b00 2024-05-28T14:09:22.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0ff200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fe100145590 con 0x7fe10006c100 2024-05-28T14:09:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0ff200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe100145870 con 0x7fe100147b40 2024-05-28T14:09:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0f7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe10006c100 0x7fe1000b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0f7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe10006c100 0x7fe1000b4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49482/0 (socket says 172.21.15.42:49482) 2024-05-28T14:09:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0f7e00640 1 -- 172.21.15.42:0/2879592033 learned_addr learned my addr 172.21.15.42:0/2879592033 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0fe800640 1 --2- 172.21.15.42:0/2879592033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe100147b40 0x7fe100145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:22.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0f7e00640 1 -- 172.21.15.42:0/2879592033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe100147b40 msgr2=0x7fe100145d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:22.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0f7e00640 1 --2- 172.21.15.42:0/2879592033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe100147b40 0x7fe100145d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:22.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0f7e00640 1 -- 172.21.15.42:0/2879592033 >> v1:172.21.15.63:6789/0 conn(0x7fe1000b1b00 legacy=0x7fe10006b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:22.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0f7e00640 1 -- 172.21.15.42:0/2879592033 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1000b5e70 con 0x7fe10006c100 2024-05-28T14:09:22.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.877+0000 7fe0fe800640 1 --2- 172.21.15.42:0/2879592033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe100147b40 0x7fe100145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:22.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.881+0000 7fe0f7e00640 1 --2- 172.21.15.42:0/2879592033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe10006c100 0x7fe1000b4280 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe1001622d0 tx=0x7fe0e801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc77899ba5fc7033 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:22.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.881+0000 7fe0fd400640 1 -- 172.21.15.42:0/2879592033 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e80b1070 con 0x7fe10006c100 2024-05-28T14:09:22.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.881+0000 7fe0fd400640 1 -- 172.21.15.42:0/2879592033 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e80a8050 con 0x7fe10006c100 2024-05-28T14:09:22.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.881+0000 7fe0fd400640 1 -- 172.21.15.42:0/2879592033 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e80ac860 con 0x7fe10006c100 2024-05-28T14:09:22.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.881+0000 7fe0ff200640 1 -- 172.21.15.42:0/2879592033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe10006c100 msgr2=0x7fe1000b4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:22.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.881+0000 7fe0ff200640 1 --2- 172.21.15.42:0/2879592033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe10006c100 0x7fe1000b4280 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe1001622d0 tx=0x7fe0e801c420 comp rx=0 tx=0).stop 2024-05-28T14:09:22.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.881+0000 7fe0ff200640 1 -- 172.21.15.42:0/2879592033 shutdown_connections 2024-05-28T14:09:22.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.881+0000 7fe0ff200640 1 --2- 172.21.15.42:0/2879592033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe100147b40 0x7fe100145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:22.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.881+0000 7fe0ff200640 1 --2- 172.21.15.42:0/2879592033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe10006c100 0x7fe1000b4280 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:22.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.881+0000 7fe0ff200640 1 -- 172.21.15.42:0/2879592033 >> 172.21.15.42:0/2879592033 conn(0x7fe1000b67a0 msgr2=0x7fe1000b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:22.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.881+0000 7fe0ff200640 1 -- 172.21.15.42:0/2879592033 shutdown_connections 2024-05-28T14:09:22.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.881+0000 7fe0ff200640 1 -- 172.21.15.42:0/2879592033 wait complete. 2024-05-28T14:09:22.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.881+0000 7fe0ff200640 1 Processor -- start 2024-05-28T14:09:22.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.881+0000 7fe0ff200640 1 -- start start 2024-05-28T14:09:22.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0ff200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe10006c100 0x7fe1001449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:22.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0ff200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1000b1b00 0x7fe10009d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:22.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0ff200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe100147b40 0x7fe10009bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:22.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0ff200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe100146bd0 con 0x7fe1000b1b00 2024-05-28T14:09:22.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0ff200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe100146a50 con 0x7fe10006c100 2024-05-28T14:09:22.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0ff200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe100146d50 con 0x7fe100147b40 2024-05-28T14:09:22.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0f7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1000b1b00 0x7fe10009d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:22.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0f7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1000b1b00 0x7fe10009d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49486/0 (socket says 172.21.15.42:49486) 2024-05-28T14:09:22.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0f7e00640 1 -- 172.21.15.42:0/934289429 learned_addr learned my addr 172.21.15.42:0/934289429 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:22.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0fde00640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe10006c100 0x7fe1001449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:22.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0fe800640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe100147b40 0x7fe10009bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:22.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0f7e00640 1 -- 172.21.15.42:0/934289429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe100147b40 msgr2=0x7fe10009bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:22.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0f7e00640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe100147b40 0x7fe10009bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:22.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0f7e00640 1 -- 172.21.15.42:0/934289429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe10006c100 msgr2=0x7fe1001449b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:22.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0f7e00640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe10006c100 0x7fe1001449b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:22.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0f7e00640 1 -- 172.21.15.42:0/934289429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10009c330 con 0x7fe1000b1b00 2024-05-28T14:09:22.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0fe800640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe100147b40 0x7fe10009bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:22.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0f7e00640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1000b1b00 0x7fe10009d670 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80af040 tx=0x7fe0e801ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:22.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0f6000640 1 -- 172.21.15.42:0/934289429 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e80b1040 con 0x7fe1000b1b00 2024-05-28T14:09:22.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0f6000640 1 -- 172.21.15.42:0/934289429 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e80bd070 con 0x7fe1000b1b00 2024-05-28T14:09:22.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0ff200640 1 -- 172.21.15.42:0/934289429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10009c5d0 con 0x7fe1000b1b00 2024-05-28T14:09:22.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0f6000640 1 -- 172.21.15.42:0/934289429 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e80b7660 con 0x7fe1000b1b00 2024-05-28T14:09:22.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0ff200640 1 -- 172.21.15.42:0/934289429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe10009cb90 con 0x7fe1000b1b00 2024-05-28T14:09:22.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.885+0000 7fe0ff200640 1 -- 172.21.15.42:0/934289429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10004fcf0 con 0x7fe1000b1b00 2024-05-28T14:09:22.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.889+0000 7fe0f6000640 1 -- 172.21.15.42:0/934289429 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fe0e80b7800 con 0x7fe1000b1b00 2024-05-28T14:09:22.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.889+0000 7fe0f6000640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe0dc0694f0 0x7fe0dc06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:22.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.889+0000 7fe0f6000640 1 -- 172.21.15.42:0/934289429 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4971+0+0 (secure 0 0 0) 0x7fe0e811fda0 con 0x7fe1000b1b00 2024-05-28T14:09:22.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.889+0000 7fe0fde00640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe0dc0694f0 0x7fe0dc06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:22.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.889+0000 7fe0fde00640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe0dc0694f0 0x7fe0dc06b9b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe10015ea50 tx=0x7fe0f0007e40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:22.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:22.893+0000 7fe0f6000640 1 -- 172.21.15.42:0/934289429 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe0e80160d0 con 0x7fe1000b1b00 2024-05-28T14:09:23.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.061+0000 7fe0ff200640 1 -- 172.21.15.42:0/934289429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1000b6d00 con 0x7fe1000b1b00 2024-05-28T14:09:23.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.061+0000 7fe0f6000640 1 -- 172.21.15.42:0/934289429 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+13004 (secure 0 0 0) 0x7fe0e80f1ac0 con 0x7fe1000b1b00 2024-05-28T14:09:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:23.045 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:09:20.678008+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6837/1199082803":"2024-05-29T14:09:17.589433+0000","172.21.15.42:6835/1199082803":"2024-05-29T14:09:17.589433+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:09:23.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.061+0000 7fe0ff200640 1 -- 172.21.15.42:0/934289429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe0dc0694f0 msgr2=0x7fe0dc06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.061+0000 7fe0ff200640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe0dc0694f0 0x7fe0dc06b9b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe10015ea50 tx=0x7fe0f0007e40 comp rx=0 tx=0).stop 2024-05-28T14:09:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.061+0000 7fe0ff200640 1 -- 172.21.15.42:0/934289429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1000b1b00 msgr2=0x7fe10009d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.061+0000 7fe0ff200640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1000b1b00 0x7fe10009d670 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80af040 tx=0x7fe0e801ac60 comp rx=0 tx=0).stop 2024-05-28T14:09:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.061+0000 7fe0ff200640 1 -- 172.21.15.42:0/934289429 shutdown_connections 2024-05-28T14:09:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.061+0000 7fe0ff200640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe0dc0694f0 0x7fe0dc06b9b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.061+0000 7fe0ff200640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe100147b40 0x7fe10009bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.061+0000 7fe0ff200640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1000b1b00 0x7fe10009d670 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.061+0000 7fe0ff200640 1 --2- 172.21.15.42:0/934289429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe10006c100 0x7fe1001449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.061+0000 7fe0ff200640 1 -- 172.21.15.42:0/934289429 >> 172.21.15.42:0/934289429 conn(0x7fe1000b67a0 msgr2=0x7fe1000aef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.061+0000 7fe0ff200640 1 -- 172.21.15.42:0/934289429 shutdown_connections 2024-05-28T14:09:23.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.061+0000 7fe0ff200640 1 -- 172.21.15.42:0/934289429 wait complete. 2024-05-28T14:09:23.066 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:09:23.067 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:09:23.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe931000640 1 Processor -- start 2024-05-28T14:09:23.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe931000640 1 -- start start 2024-05-28T14:09:23.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe931000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe92c071de0 0x7fe92c0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe931000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92c13fd10 0x7fe92c1401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe931000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92c072700 con 0x7fe92c0ce3b0 2024-05-28T14:09:23.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe931000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe92c160230 con 0x7fe92c071de0 2024-05-28T14:09:23.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe931000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe92c160510 con 0x7fe92c13fd10 2024-05-28T14:09:23.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe923e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe92c0ce3b0 0x7fe92c0ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56404/0 (socket says 172.21.15.42:56404) 2024-05-28T14:09:23.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe923e00640 1 -- 172.21.15.42:0/2283996982 learned_addr learned my addr 172.21.15.42:0/2283996982 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:23.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe92a800640 1 --2- 172.21.15.42:0/2283996982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92c13fd10 0x7fe92c1401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929e00640 1 --2- 172.21.15.42:0/2283996982 >> v2:172.21.15.63:3300/0 conn(0x7fe92c071de0 0x7fe92c0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 -- 172.21.15.42:0/2283996982 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 233052508 0 0) 0x7fe92c072700 con 0x7fe92c0ce3b0 2024-05-28T14:09:23.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 -- 172.21.15.42:0/2283996982 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe910003530 con 0x7fe92c0ce3b0 2024-05-28T14:09:23.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 -- 172.21.15.42:0/2283996982 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3296898204 0 0) 0x7fe910003530 con 0x7fe92c0ce3b0 2024-05-28T14:09:23.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 -- 172.21.15.42:0/2283996982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92c13fd10 msgr2=0x7fe92c1401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:23.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 --2- 172.21.15.42:0/2283996982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92c13fd10 0x7fe92c1401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 -- 172.21.15.42:0/2283996982 >> v2:172.21.15.63:3300/0 conn(0x7fe92c071de0 msgr2=0x7fe92c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:23.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 --2- 172.21.15.42:0/2283996982 >> v2:172.21.15.63:3300/0 conn(0x7fe92c071de0 0x7fe92c0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 -- 172.21.15.42:0/2283996982 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c1607f0 con 0x7fe92c0ce3b0 2024-05-28T14:09:23.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 -- 172.21.15.42:0/2283996982 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fe914001680 con 0x7fe92c0ce3b0 2024-05-28T14:09:23.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 -- 172.21.15.42:0/2283996982 >> v1:172.21.15.42:6789/0 conn(0x7fe92c0ce3b0 legacy=0x7fe92c0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:23.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 --2- 172.21.15.42:0/2283996982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9100039c0 0x7fe910003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 --2- 172.21.15.42:0/2283996982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe910004810 0x7fe910004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 --2- 172.21.15.42:0/2283996982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92c13fd10 0x7fe9100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 -- 172.21.15.42:0/2283996982 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c160230 con 0x7fe9100039c0 2024-05-28T14:09:23.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 -- 172.21.15.42:0/2283996982 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c160510 con 0x7fe910004810 2024-05-28T14:09:23.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929400640 1 -- 172.21.15.42:0/2283996982 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe91000ba50 con 0x7fe92c13fd10 2024-05-28T14:09:23.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929e00640 1 --2- 172.21.15.42:0/2283996982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9100039c0 0x7fe910003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe923e00640 1 --2- 172.21.15.42:0/2283996982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe910004810 0x7fe910004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe92a800640 1 --2- 172.21.15.42:0/2283996982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92c13fd10 0x7fe9100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929e00640 1 -- 172.21.15.42:0/2283996982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92c13fd10 msgr2=0x7fe9100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:23.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929e00640 1 --2- 172.21.15.42:0/2283996982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92c13fd10 0x7fe9100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929e00640 1 -- 172.21.15.42:0/2283996982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe910004810 msgr2=0x7fe910004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:23.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929e00640 1 --2- 172.21.15.42:0/2283996982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe910004810 0x7fe910004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe929e00640 1 -- 172.21.15.42:0/2283996982 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe910003530 con 0x7fe9100039c0 2024-05-28T14:09:23.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.229+0000 7fe92a800640 1 --2- 172.21.15.42:0/2283996982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92c13fd10 0x7fe9100051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:23.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.233+0000 7fe929e00640 1 --2- 172.21.15.42:0/2283996982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9100039c0 0x7fe910003de0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009ae0 tx=0x7fe91c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7159267762b1800 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:23.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.233+0000 7fe929400640 1 -- 172.21.15.42:0/2283996982 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c0096b0 con 0x7fe9100039c0 2024-05-28T14:09:23.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.233+0000 7fe929400640 1 -- 172.21.15.42:0/2283996982 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe91c0a6490 con 0x7fe9100039c0 2024-05-28T14:09:23.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.233+0000 7fe929400640 1 -- 172.21.15.42:0/2283996982 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c0a6880 con 0x7fe9100039c0 2024-05-28T14:09:23.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.233+0000 7fe931000640 1 -- 172.21.15.42:0/2283996982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9100039c0 msgr2=0x7fe910003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:23.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.233+0000 7fe931000640 1 --2- 172.21.15.42:0/2283996982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9100039c0 0x7fe910003de0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009ae0 tx=0x7fe91c01cab0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.233+0000 7fe92a800640 1 -- 172.21.15.42:0/2283996982 reap_dead start 2024-05-28T14:09:23.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.233+0000 7fe931000640 1 -- 172.21.15.42:0/2283996982 shutdown_connections 2024-05-28T14:09:23.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.233+0000 7fe931000640 1 -- 172.21.15.42:0/2283996982 >> 172.21.15.42:0/2283996982 conn(0x7fe92c0b21f0 msgr2=0x7fe92c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:23.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.233+0000 7fe931000640 1 -- 172.21.15.42:0/2283996982 shutdown_connections 2024-05-28T14:09:23.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.233+0000 7fe931000640 1 -- 172.21.15.42:0/2283996982 wait complete. 2024-05-28T14:09:23.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe931000640 1 Processor -- start 2024-05-28T14:09:23.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe931000640 1 -- start start 2024-05-28T14:09:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe931000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9100039c0 0x7fe92c155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe931000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe92c133bc0 0x7fe92c136240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe931000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92c136780 0x7fe92c06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe931000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c140d40 con 0x7fe92c133bc0 2024-05-28T14:09:23.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe931000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c140bc0 con 0x7fe9100039c0 2024-05-28T14:09:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe931000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe92c140ec0 con 0x7fe92c136780 2024-05-28T14:09:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe923e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe92c133bc0 0x7fe92c136240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe923e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe92c133bc0 0x7fe92c136240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49498/0 (socket says 172.21.15.42:49498) 2024-05-28T14:09:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe923e00640 1 -- 172.21.15.42:0/2453529432 learned_addr learned my addr 172.21.15.42:0/2453529432 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe92a800640 1 --2- 172.21.15.42:0/2453529432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92c136780 0x7fe92c06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe929e00640 1 --2- 172.21.15.42:0/2453529432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9100039c0 0x7fe92c155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe923e00640 1 -- 172.21.15.42:0/2453529432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92c136780 msgr2=0x7fe92c06e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe923e00640 1 --2- 172.21.15.42:0/2453529432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92c136780 0x7fe92c06e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe923e00640 1 -- 172.21.15.42:0/2453529432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9100039c0 msgr2=0x7fe92c155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe923e00640 1 --2- 172.21.15.42:0/2453529432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9100039c0 0x7fe92c155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe923e00640 1 -- 172.21.15.42:0/2453529432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c156050 con 0x7fe92c133bc0 2024-05-28T14:09:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe923e00640 1 --2- 172.21.15.42:0/2453529432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe92c133bc0 0x7fe92c136240 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe914004e60 tx=0x7fe914007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:23.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe922000640 1 -- 172.21.15.42:0/2453529432 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91400f7b0 con 0x7fe92c133bc0 2024-05-28T14:09:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe922000640 1 -- 172.21.15.42:0/2453529432 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe914018070 con 0x7fe92c133bc0 2024-05-28T14:09:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe931000640 1 -- 172.21.15.42:0/2453529432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92c1f7610 con 0x7fe92c133bc0 2024-05-28T14:09:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe922000640 1 -- 172.21.15.42:0/2453529432 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91400fb70 con 0x7fe92c133bc0 2024-05-28T14:09:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.237+0000 7fe931000640 1 -- 172.21.15.42:0/2453529432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c15dbe0 con 0x7fe92c133bc0 2024-05-28T14:09:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.241+0000 7fe931000640 1 -- 172.21.15.42:0/2453529432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe92c0738b0 con 0x7fe92c133bc0 2024-05-28T14:09:23.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.241+0000 7fe922000640 1 -- 172.21.15.42:0/2453529432 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fe91400fd10 con 0x7fe92c133bc0 2024-05-28T14:09:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.241+0000 7fe922000640 1 --2- 172.21.15.42:0/2453529432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe9080694f0 0x7fe90806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.241+0000 7fe922000640 1 -- 172.21.15.42:0/2453529432 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4971+0+0 (secure 0 0 0) 0x7fe914085840 con 0x7fe92c133bc0 2024-05-28T14:09:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.241+0000 7fe929e00640 1 --2- 172.21.15.42:0/2453529432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe9080694f0 0x7fe90806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.241+0000 7fe929e00640 1 --2- 172.21.15.42:0/2453529432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe9080694f0 0x7fe90806b9b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0ad000 tx=0x7fe91c0b2040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:23.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.245+0000 7fe922000640 1 -- 172.21.15.42:0/2453529432 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe914057500 con 0x7fe92c133bc0 2024-05-28T14:09:23.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.413+0000 7fe931000640 1 -- 172.21.15.42:0/2453529432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe92c04fdd0 con 0x7fe92c133bc0 2024-05-28T14:09:23.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.413+0000 7fe922000640 1 -- 172.21.15.42:0/2453529432 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fe914056e00 con 0x7fe92c133bc0 2024-05-28T14:09:23.398 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:09:23.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.417+0000 7fe931000640 1 -- 172.21.15.42:0/2453529432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe9080694f0 msgr2=0x7fe90806b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:23.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.417+0000 7fe931000640 1 --2- 172.21.15.42:0/2453529432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe9080694f0 0x7fe90806b9b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0ad000 tx=0x7fe91c0b2040 comp rx=0 tx=0).stop 2024-05-28T14:09:23.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.417+0000 7fe931000640 1 -- 172.21.15.42:0/2453529432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe92c133bc0 msgr2=0x7fe92c136240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:23.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.417+0000 7fe931000640 1 --2- 172.21.15.42:0/2453529432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe92c133bc0 0x7fe92c136240 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe914004e60 tx=0x7fe914007430 comp rx=0 tx=0).stop 2024-05-28T14:09:23.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.417+0000 7fe931000640 1 -- 172.21.15.42:0/2453529432 shutdown_connections 2024-05-28T14:09:23.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.417+0000 7fe931000640 1 --2- 172.21.15.42:0/2453529432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe9080694f0 0x7fe90806b9b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.417+0000 7fe931000640 1 --2- 172.21.15.42:0/2453529432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92c136780 0x7fe92c06e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.417+0000 7fe931000640 1 --2- 172.21.15.42:0/2453529432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe92c133bc0 0x7fe92c136240 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.417+0000 7fe931000640 1 --2- 172.21.15.42:0/2453529432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9100039c0 0x7fe92c155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.417+0000 7fe931000640 1 -- 172.21.15.42:0/2453529432 >> 172.21.15.42:0/2453529432 conn(0x7fe92c0b21f0 msgr2=0x7fe92c0721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:23.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.417+0000 7fe931000640 1 -- 172.21.15.42:0/2453529432 shutdown_connections 2024-05-28T14:09:23.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.417+0000 7fe931000640 1 -- 172.21.15.42:0/2453529432 wait complete. 2024-05-28T14:09:23.420 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed' 2024-05-28T14:09:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.577+0000 7f31b9400640 1 Processor -- start 2024-05-28T14:09:23.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.577+0000 7f31b9400640 1 -- start start 2024-05-28T14:09:23.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.577+0000 7f31b9400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31b40b42e0 0x7f31b415ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.577+0000 7f31b9400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b406eaa0 con 0x7f31b4067810 2024-05-28T14:09:23.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.577+0000 7f31b9400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b406ed80 con 0x7f31b40d0420 2024-05-28T14:09:23.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.577+0000 7f31b9400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f31b406f060 con 0x7f31b40b42e0 2024-05-28T14:09:23.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.577+0000 7f31b3200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f31b4067810 0x7f31b406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56414/0 (socket says 172.21.15.42:56414) 2024-05-28T14:09:23.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.577+0000 7f31b3200640 1 -- 172.21.15.42:0/2361863768 learned_addr learned my addr 172.21.15.42:0/2361863768 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:23.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.577+0000 7f31a9e00640 1 --2- 172.21.15.42:0/2361863768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31b40b42e0 0x7f31b415ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 425958517 0 0) 0x7f31b406eaa0 con 0x7f31b4067810 2024-05-28T14:09:23.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3198003530 con 0x7f31b4067810 2024-05-28T14:09:23.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3487041554 0 0) 0x7f31b406ed80 con 0x7f31b40d0420 2024-05-28T14:09:23.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31b406eaa0 con 0x7f31b40d0420 2024-05-28T14:09:23.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1011896082 0 0) 0x7f3198003530 con 0x7f31b4067810 2024-05-28T14:09:23.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31b40b42e0 msgr2=0x7f31b415ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:23.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 --2- 172.21.15.42:0/2361863768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31b40b42e0 0x7f31b415ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 >> v1:172.21.15.63:6789/0 conn(0x7f31b40d0420 legacy=0x7f31b40b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:23.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b406f340 con 0x7f31b4067810 2024-05-28T14:09:23.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f31ac001a00 con 0x7f31b4067810 2024-05-28T14:09:23.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 >> v1:172.21.15.42:6789/0 conn(0x7f31b4067810 legacy=0x7f31b406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:23.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 --2- 172.21.15.42:0/2361863768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3198003b80 0x7f3198003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 --2- 172.21.15.42:0/2361863768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3198004810 0x7f3198004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 --2- 172.21.15.42:0/2361863768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31b40b42e0 0x7f31980051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f31b406f060 con 0x7f3198004810 2024-05-28T14:09:23.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f319800bae0 con 0x7f3198003b80 2024-05-28T14:09:23.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f319800bce0 con 0x7f31b40b42e0 2024-05-28T14:09:23.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31a9e00640 1 --2- 172.21.15.42:0/2361863768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3198004810 0x7f3198004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b2800640 1 --2- 172.21.15.42:0/2361863768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3198003b80 0x7f3198003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31b3200640 1 --2- 172.21.15.42:0/2361863768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31b40b42e0 0x7f31980051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31a9e00640 1 -- 172.21.15.42:0/2361863768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31b40b42e0 msgr2=0x7f31980051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:23.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31a9e00640 1 --2- 172.21.15.42:0/2361863768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31b40b42e0 0x7f31980051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31a9e00640 1 -- 172.21.15.42:0/2361863768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3198003b80 msgr2=0x7f3198003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:23.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31a9e00640 1 --2- 172.21.15.42:0/2361863768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3198003b80 0x7f3198003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.581+0000 7f31a9e00640 1 -- 172.21.15.42:0/2361863768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3198003530 con 0x7f3198004810 2024-05-28T14:09:23.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31a9e00640 1 --2- 172.21.15.42:0/2361863768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3198004810 0x7f3198004c90 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009ae0 tx=0x7f31a401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9157e6bdfe7e4d8d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:23.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a40096b0 con 0x7f3198004810 2024-05-28T14:09:23.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a40a6450 con 0x7f3198004810 2024-05-28T14:09:23.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b1e00640 1 -- 172.21.15.42:0/2361863768 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a40a6840 con 0x7f3198004810 2024-05-28T14:09:23.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b9400640 1 -- 172.21.15.42:0/2361863768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3198004810 msgr2=0x7f3198004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:23.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b9400640 1 --2- 172.21.15.42:0/2361863768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3198004810 0x7f3198004c90 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009ae0 tx=0x7f31a401ca70 comp rx=0 tx=0).stop 2024-05-28T14:09:23.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b3200640 1 -- 172.21.15.42:0/2361863768 reap_dead start 2024-05-28T14:09:23.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b9400640 1 -- 172.21.15.42:0/2361863768 shutdown_connections 2024-05-28T14:09:23.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b9400640 1 -- 172.21.15.42:0/2361863768 >> 172.21.15.42:0/2361863768 conn(0x7f31b40d0b40 msgr2=0x7f31b40d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:23.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b9400640 1 -- 172.21.15.42:0/2361863768 shutdown_connections 2024-05-28T14:09:23.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b9400640 1 -- 172.21.15.42:0/2361863768 wait complete. 2024-05-28T14:09:23.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b9400640 1 Processor -- start 2024-05-28T14:09:23.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b9400640 1 -- start start 2024-05-28T14:09:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b9400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3198004810 0x7f31b414ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b4150e00 0x7f31b4151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b40ae9f0 0x7f31b40aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b9400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f31b406cd30 con 0x7f31b4150e00 2024-05-28T14:09:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b9400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f31b406cbb0 con 0x7f31b40ae9f0 2024-05-28T14:09:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31b9400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f31b406ceb0 con 0x7f3198004810 2024-05-28T14:09:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.585+0000 7f31a9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b4150e00 0x7f31b4151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31a9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b4150e00 0x7f31b4151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49504/0 (socket says 172.21.15.42:49504) 2024-05-28T14:09:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31a9e00640 1 -- 172.21.15.42:0/2338087136 learned_addr learned my addr 172.21.15.42:0/2338087136 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:23.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31b2800640 1 --2- 172.21.15.42:0/2338087136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3198004810 0x7f31b414ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31b3200640 1 --2- 172.21.15.42:0/2338087136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b40ae9f0 0x7f31b40aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31a9e00640 1 -- 172.21.15.42:0/2338087136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3198004810 msgr2=0x7f31b414ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31a9e00640 1 --2- 172.21.15.42:0/2338087136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3198004810 0x7f31b414ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31a9e00640 1 -- 172.21.15.42:0/2338087136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b40ae9f0 msgr2=0x7f31b40aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31a9e00640 1 --2- 172.21.15.42:0/2338087136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b40ae9f0 0x7f31b40aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31a9e00640 1 -- 172.21.15.42:0/2338087136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b40af3b0 con 0x7f31b4150e00 2024-05-28T14:09:23.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31a9e00640 1 --2- 172.21.15.42:0/2338087136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b4150e00 0x7f31b4151280 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f31a401c8a0 tx=0x7f3198005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:23.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31ab400640 1 -- 172.21.15.42:0/2338087136 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a40bf070 con 0x7f31b4150e00 2024-05-28T14:09:23.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31ab400640 1 -- 172.21.15.42:0/2338087136 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a400f000 con 0x7f31b4150e00 2024-05-28T14:09:23.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31ab400640 1 -- 172.21.15.42:0/2338087136 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a40a6e20 con 0x7f31b4150e00 2024-05-28T14:09:23.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31b9400640 1 -- 172.21.15.42:0/2338087136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b4151ce0 con 0x7f31b4150e00 2024-05-28T14:09:23.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31b9400640 1 -- 172.21.15.42:0/2338087136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b40af920 con 0x7f31b4150e00 2024-05-28T14:09:23.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31b9400640 1 -- 172.21.15.42:0/2338087136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b4068a90 con 0x7f31b4150e00 2024-05-28T14:09:23.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.589+0000 7f31ab400640 1 -- 172.21.15.42:0/2338087136 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f31a40a7430 con 0x7f31b4150e00 2024-05-28T14:09:23.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.593+0000 7f31ab400640 1 --2- 172.21.15.42:0/2338087136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f31900694f0 0x7f319006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:23.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.593+0000 7f31ab400640 1 -- 172.21.15.42:0/2338087136 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4971+0+0 (secure 0 0 0) 0x7f31a411edb0 con 0x7f31b4150e00 2024-05-28T14:09:23.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.593+0000 7f31b2800640 1 --2- 172.21.15.42:0/2338087136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f31900694f0 0x7f319006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:23.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.593+0000 7f31b2800640 1 --2- 172.21.15.42:0/2338087136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f31900694f0 0x7f319006b9b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f319c007fd0 tx=0x7f319c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:23.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.597+0000 7f31ab400640 1 -- 172.21.15.42:0/2338087136 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f31a40ac020 con 0x7f31b4150e00 2024-05-28T14:09:23.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:23.821+0000 7f31b9400640 1 -- 172.21.15.42:0/2338087136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed"]} v 0) -- 0x7f31b406bd30 con 0x7f31b4150e00 2024-05-28T14:09:24.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.701+0000 7f31ab400640 1 -- 172.21.15.42:0/2338087136 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed"]}]=0 v510) ==== 156+0+0 (secure 0 0 0) 0x7f31a4016780 con 0x7f31b4150e00 2024-05-28T14:09:24.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.705+0000 7f31b9400640 1 -- 172.21.15.42:0/2338087136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f31900694f0 msgr2=0x7f319006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:24.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.705+0000 7f31b9400640 1 --2- 172.21.15.42:0/2338087136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f31900694f0 0x7f319006b9b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f319c007fd0 tx=0x7f319c009040 comp rx=0 tx=0).stop 2024-05-28T14:09:24.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.705+0000 7f31b9400640 1 -- 172.21.15.42:0/2338087136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b4150e00 msgr2=0x7f31b4151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:24.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.705+0000 7f31b9400640 1 --2- 172.21.15.42:0/2338087136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b4150e00 0x7f31b4151280 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f31a401c8a0 tx=0x7f3198005840 comp rx=0 tx=0).stop 2024-05-28T14:09:24.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.705+0000 7f31b9400640 1 -- 172.21.15.42:0/2338087136 shutdown_connections 2024-05-28T14:09:24.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.705+0000 7f31b9400640 1 --2- 172.21.15.42:0/2338087136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f31900694f0 0x7f319006b9b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:24.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.705+0000 7f31b9400640 1 --2- 172.21.15.42:0/2338087136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31b40ae9f0 0x7f31b40aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:24.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.705+0000 7f31b9400640 1 --2- 172.21.15.42:0/2338087136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b4150e00 0x7f31b4151280 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:24.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.705+0000 7f31b9400640 1 --2- 172.21.15.42:0/2338087136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3198004810 0x7f31b414ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:24.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.705+0000 7f31b9400640 1 -- 172.21.15.42:0/2338087136 >> 172.21.15.42:0/2338087136 conn(0x7f31b40d0b40 msgr2=0x7f31b41647c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:24.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.705+0000 7f31b9400640 1 -- 172.21.15.42:0/2338087136 shutdown_connections 2024-05-28T14:09:24.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.705+0000 7f31b9400640 1 -- 172.21.15.42:0/2338087136 wait complete. 2024-05-28T14:09:24.713 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T14:09:24.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f1587400640 1 Processor -- start 2024-05-28T14:09:24.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f1587400640 1 -- start start 2024-05-28T14:09:24.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f1587400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f158009aa50 0x7f15800d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:24.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f1587400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f158009ae30 0x7f158006d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:24.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f1587400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f158006dfd0 con 0x7f158006dbc0 2024-05-28T14:09:24.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f1587400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f15800d0f50 con 0x7f158009aa50 2024-05-28T14:09:24.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f1587400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f15800d1230 con 0x7f158009ae30 2024-05-28T14:09:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f1584a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f158009aa50 0x7f15800d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f1585400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f158006dbc0 0x7f15801624d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56422/0 (socket says 172.21.15.42:56422) 2024-05-28T14:09:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f1585400640 1 -- 172.21.15.42:0/3795589514 learned_addr learned my addr 172.21.15.42:0/3795589514 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f157fe00640 1 --2- 172.21.15.42:0/3795589514 >> v2:172.21.15.63:3300/0 conn(0x7f158009ae30 0x7f158006d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f1584a00640 1 -- 172.21.15.42:0/3795589514 >> v2:172.21.15.63:3300/0 conn(0x7f158009ae30 msgr2=0x7f158006d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f1584a00640 1 --2- 172.21.15.42:0/3795589514 >> v2:172.21.15.63:3300/0 conn(0x7f158009ae30 0x7f158006d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f1584a00640 1 -- 172.21.15.42:0/3795589514 >> v1:172.21.15.42:6789/0 conn(0x7f158006dbc0 legacy=0x7f15801624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f1584a00640 1 -- 172.21.15.42:0/3795589514 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15800d1510 con 0x7f158009aa50 2024-05-28T14:09:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.881+0000 7f157fe00640 1 --2- 172.21.15.42:0/3795589514 >> v2:172.21.15.63:3300/0 conn(0x7f158009ae30 0x7f158006d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.885+0000 7f1584a00640 1 --2- 172.21.15.42:0/3795589514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f158009aa50 0x7f15800d0820 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f1580073370 tx=0x7f157401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967bafc94ab752a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.885+0000 7f157f400640 1 -- 172.21.15.42:0/3795589514 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15740b0070 con 0x7f158009aa50 2024-05-28T14:09:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.885+0000 7f157f400640 1 -- 172.21.15.42:0/3795589514 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15740ab600 con 0x7f158009aa50 2024-05-28T14:09:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.885+0000 7f157f400640 1 -- 172.21.15.42:0/3795589514 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15740ab9d0 con 0x7f158009aa50 2024-05-28T14:09:24.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.885+0000 7f1587400640 1 -- 172.21.15.42:0/3795589514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f158009aa50 msgr2=0x7f15800d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:24.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.885+0000 7f1587400640 1 --2- 172.21.15.42:0/3795589514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f158009aa50 0x7f15800d0820 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f1580073370 tx=0x7f157401c480 comp rx=0 tx=0).stop 2024-05-28T14:09:24.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.885+0000 7f1587400640 1 -- 172.21.15.42:0/3795589514 shutdown_connections 2024-05-28T14:09:24.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.885+0000 7f1587400640 1 --2- 172.21.15.42:0/3795589514 >> v2:172.21.15.63:3300/0 conn(0x7f158009ae30 0x7f158006d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:24.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.885+0000 7f1587400640 1 --2- 172.21.15.42:0/3795589514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f158009aa50 0x7f15800d0820 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:24.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.885+0000 7f1587400640 1 -- 172.21.15.42:0/3795589514 >> 172.21.15.42:0/3795589514 conn(0x7f15800b3fe0 msgr2=0x7f15800b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:24.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.885+0000 7f1587400640 1 -- 172.21.15.42:0/3795589514 shutdown_connections 2024-05-28T14:09:24.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.885+0000 7f1587400640 1 -- 172.21.15.42:0/3795589514 wait complete. 2024-05-28T14:09:24.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f1587400640 1 Processor -- start 2024-05-28T14:09:24.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f1587400640 1 -- start start 2024-05-28T14:09:24.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f1587400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f158006dbc0 0x7f15801544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:24.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f1587400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f158009aa50 0x7f1580155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:24.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f1587400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f158009ae30 0x7f15801561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:24.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f1587400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f158016b990 con 0x7f158009aa50 2024-05-28T14:09:24.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f1587400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f158016b810 con 0x7f158006dbc0 2024-05-28T14:09:24.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f1587400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f158016bb10 con 0x7f158009ae30 2024-05-28T14:09:24.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f157fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f158009aa50 0x7f1580155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:24.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f157fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f158009aa50 0x7f1580155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49530/0 (socket says 172.21.15.42:49530) 2024-05-28T14:09:24.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f157fe00640 1 -- 172.21.15.42:0/669971407 learned_addr learned my addr 172.21.15.42:0/669971407 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:24.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f1584a00640 1 --2- 172.21.15.42:0/669971407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f158006dbc0 0x7f15801544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:24.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f157fe00640 1 -- 172.21.15.42:0/669971407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f158009ae30 msgr2=0x7f15801561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:24.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f157fe00640 1 --2- 172.21.15.42:0/669971407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f158009ae30 0x7f15801561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:24.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f157fe00640 1 -- 172.21.15.42:0/669971407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f158006dbc0 msgr2=0x7f15801544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:24.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f157fe00640 1 --2- 172.21.15.42:0/669971407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f158006dbc0 0x7f15801544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:24.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f157fe00640 1 -- 172.21.15.42:0/669971407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158015b2c0 con 0x7f158009aa50 2024-05-28T14:09:24.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f1584a00640 1 --2- 172.21.15.42:0/669971407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f158006dbc0 0x7f15801544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:24.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f157fe00640 1 --2- 172.21.15.42:0/669971407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f158009aa50 0x7f1580155c80 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f156c00e840 tx=0x7f156c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:24.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f157cc00640 1 -- 172.21.15.42:0/669971407 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c017070 con 0x7f158009aa50 2024-05-28T14:09:24.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f157cc00640 1 -- 172.21.15.42:0/669971407 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f156c00c8d0 con 0x7f158009aa50 2024-05-28T14:09:24.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f1587400640 1 -- 172.21.15.42:0/669971407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15801609e0 con 0x7f158009aa50 2024-05-28T14:09:24.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f157cc00640 1 -- 172.21.15.42:0/669971407 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c00f040 con 0x7f158009aa50 2024-05-28T14:09:24.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.889+0000 7f1587400640 1 -- 172.21.15.42:0/669971407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1580154a00 con 0x7f158009aa50 2024-05-28T14:09:24.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.893+0000 7f1587400640 1 -- 172.21.15.42:0/669971407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15800a0540 con 0x7f158009aa50 2024-05-28T14:09:24.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.893+0000 7f157cc00640 1 -- 172.21.15.42:0/669971407 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f156c01a070 con 0x7f158009aa50 2024-05-28T14:09:24.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.893+0000 7f157cc00640 1 --2- 172.21.15.42:0/669971407 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f15600694f0 0x7f156006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:24.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.893+0000 7f157cc00640 1 -- 172.21.15.42:0/669971407 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4971+0+0 (secure 0 0 0) 0x7f156c085500 con 0x7f158009aa50 2024-05-28T14:09:24.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.893+0000 7f1584a00640 1 --2- 172.21.15.42:0/669971407 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f15600694f0 0x7f156006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.893+0000 7f1584a00640 1 --2- 172.21.15.42:0/669971407 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f15600694f0 0x7f156006b9b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f15740ad000 tx=0x7f1574008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:24.897+0000 7f157cc00640 1 -- 172.21.15.42:0/669971407 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f156c0571c0 con 0x7f158009aa50 2024-05-28T14:09:25.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.065+0000 7f1587400640 1 -- 172.21.15.42:0/669971407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f15800cc3d0 con 0x7f158009aa50 2024-05-28T14:09:25.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.069+0000 7f157cc00640 1 -- 172.21.15.42:0/669971407 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f156c056ac0 con 0x7f158009aa50 2024-05-28T14:09:25.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.069+0000 7f1587400640 1 -- 172.21.15.42:0/669971407 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f15600694f0 msgr2=0x7f156006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:25.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.069+0000 7f1587400640 1 --2- 172.21.15.42:0/669971407 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f15600694f0 0x7f156006b9b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f15740ad000 tx=0x7f1574008040 comp rx=0 tx=0).stop 2024-05-28T14:09:25.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.069+0000 7f1587400640 1 -- 172.21.15.42:0/669971407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f158009aa50 msgr2=0x7f1580155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:25.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.069+0000 7f1587400640 1 --2- 172.21.15.42:0/669971407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f158009aa50 0x7f1580155c80 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f156c00e840 tx=0x7f156c00ed00 comp rx=0 tx=0).stop 2024-05-28T14:09:25.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.069+0000 7f1587400640 1 -- 172.21.15.42:0/669971407 shutdown_connections 2024-05-28T14:09:25.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.069+0000 7f1587400640 1 --2- 172.21.15.42:0/669971407 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f15600694f0 0x7f156006b9b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.069+0000 7f1587400640 1 --2- 172.21.15.42:0/669971407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f158009ae30 0x7f15801561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.069+0000 7f1587400640 1 --2- 172.21.15.42:0/669971407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f158009aa50 0x7f1580155c80 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.069+0000 7f1587400640 1 --2- 172.21.15.42:0/669971407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f158006dbc0 0x7f15801544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.069+0000 7f1587400640 1 -- 172.21.15.42:0/669971407 >> 172.21.15.42:0/669971407 conn(0x7f15800b3fe0 msgr2=0x7f1580158770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:25.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.069+0000 7f1587400640 1 -- 172.21.15.42:0/669971407 shutdown_connections 2024-05-28T14:09:25.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.069+0000 7f1587400640 1 -- 172.21.15.42:0/669971407 wait complete. 2024-05-28T14:09:25.073 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T14:09:25.073 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:09:25.169 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:09:25.251 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T14:09:25.252 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:09:25.252 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:09:25.260 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:09:25.260 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:09:25.260 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:09:25.269 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:09:25.270 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:09:25.270 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:09:25.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:09:25.310 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:09:25.310 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:09:25.321 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:09:25.321 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-28T14:09:25.321 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:09:25.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed90e00640 1 Processor -- start 2024-05-28T14:09:25.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed90e00640 1 -- start start 2024-05-28T14:09:25.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed90e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fed8c0b1b00 0x7fed8c0b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:25.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed90e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed8c147b40 0x7fed8c06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:25.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed90e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed8c0b5bd0 con 0x7fed8c06bf30 2024-05-28T14:09:25.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed90e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fed8c145590 con 0x7fed8c0b1b00 2024-05-28T14:09:25.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed90e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fed8c145870 con 0x7fed8c147b40 2024-05-28T14:09:25.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed8b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fed8c0b1b00 0x7fed8c0b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:25.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed8be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fed8c06bf30 0x7fed8c162370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56424/0 (socket says 172.21.15.42:56424) 2024-05-28T14:09:25.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed8be00640 1 -- 172.21.15.42:0/4080763340 learned_addr learned my addr 172.21.15.42:0/4080763340 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:25.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed82a00640 1 --2- 172.21.15.42:0/4080763340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed8c147b40 0x7fed8c06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:25.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed8b400640 1 -- 172.21.15.42:0/4080763340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed8c147b40 msgr2=0x7fed8c06b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:25.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed8b400640 1 --2- 172.21.15.42:0/4080763340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed8c147b40 0x7fed8c06b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed8b400640 1 -- 172.21.15.42:0/4080763340 >> v1:172.21.15.42:6789/0 conn(0x7fed8c06bf30 legacy=0x7fed8c162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:25.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed8b400640 1 -- 172.21.15.42:0/4080763340 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c0b5e70 con 0x7fed8c0b1b00 2024-05-28T14:09:25.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.513+0000 7fed82a00640 1 --2- 172.21.15.42:0/4080763340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed8c147b40 0x7fed8c06b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:25.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.517+0000 7fed8b400640 1 --2- 172.21.15.42:0/4080763340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c0b1b00 0x7fed8c0b4280 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0097c0 tx=0x7fed7c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90296eb8dfd91a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:25.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.517+0000 7fed8aa00640 1 -- 172.21.15.42:0/4080763340 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0b1070 con 0x7fed8c0b1b00 2024-05-28T14:09:25.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.517+0000 7fed8aa00640 1 -- 172.21.15.42:0/4080763340 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7c0ac410 con 0x7fed8c0b1b00 2024-05-28T14:09:25.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.517+0000 7fed8aa00640 1 -- 172.21.15.42:0/4080763340 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0ac7e0 con 0x7fed8c0b1b00 2024-05-28T14:09:25.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.517+0000 7fed90e00640 1 -- 172.21.15.42:0/4080763340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c0b1b00 msgr2=0x7fed8c0b4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:25.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.517+0000 7fed90e00640 1 --2- 172.21.15.42:0/4080763340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c0b1b00 0x7fed8c0b4280 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0097c0 tx=0x7fed7c01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.517+0000 7fed90e00640 1 -- 172.21.15.42:0/4080763340 shutdown_connections 2024-05-28T14:09:25.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.517+0000 7fed90e00640 1 --2- 172.21.15.42:0/4080763340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed8c147b40 0x7fed8c06b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.517+0000 7fed90e00640 1 --2- 172.21.15.42:0/4080763340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c0b1b00 0x7fed8c0b4280 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.517+0000 7fed90e00640 1 -- 172.21.15.42:0/4080763340 >> 172.21.15.42:0/4080763340 conn(0x7fed8c0b67a0 msgr2=0x7fed8c0b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:25.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.517+0000 7fed90e00640 1 -- 172.21.15.42:0/4080763340 shutdown_connections 2024-05-28T14:09:25.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.517+0000 7fed90e00640 1 -- 172.21.15.42:0/4080763340 wait complete. 2024-05-28T14:09:25.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.517+0000 7fed90e00640 1 Processor -- start 2024-05-28T14:09:25.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.517+0000 7fed90e00640 1 -- start start 2024-05-28T14:09:25.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed90e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed8c06bf30 0x7fed8c13eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:25.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed90e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed8c0b1b00 0x7fed8c15ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:25.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed90e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c147b40 0x7fed8c15cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:25.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed90e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c16b660 con 0x7fed8c147b40 2024-05-28T14:09:25.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed90e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c16b4e0 con 0x7fed8c06bf30 2024-05-28T14:09:25.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed90e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fed8c16b7e0 con 0x7fed8c0b1b00 2024-05-28T14:09:25.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed8be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c147b40 0x7fed8c15cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:25.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed8be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c147b40 0x7fed8c15cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49554/0 (socket says 172.21.15.42:49554) 2024-05-28T14:09:25.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed8be00640 1 -- 172.21.15.42:0/2170507641 learned_addr learned my addr 172.21.15.42:0/2170507641 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:25.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed8b400640 1 --2- 172.21.15.42:0/2170507641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed8c06bf30 0x7fed8c13eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:25.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed82a00640 1 --2- 172.21.15.42:0/2170507641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed8c0b1b00 0x7fed8c15ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:25.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed8be00640 1 -- 172.21.15.42:0/2170507641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed8c0b1b00 msgr2=0x7fed8c15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:25.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed8be00640 1 --2- 172.21.15.42:0/2170507641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed8c0b1b00 0x7fed8c15ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed8be00640 1 -- 172.21.15.42:0/2170507641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed8c06bf30 msgr2=0x7fed8c13eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:25.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed8be00640 1 --2- 172.21.15.42:0/2170507641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed8c06bf30 0x7fed8c13eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed8be00640 1 -- 172.21.15.42:0/2170507641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c09c040 con 0x7fed8c147b40 2024-05-28T14:09:25.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed8be00640 1 --2- 172.21.15.42:0/2170507641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c147b40 0x7fed8c15cf40 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fed8400c400 tx=0x7fed8400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:25.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed83e00640 1 -- 172.21.15.42:0/2170507641 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84017070 con 0x7fed8c147b40 2024-05-28T14:09:25.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed83e00640 1 -- 172.21.15.42:0/2170507641 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed84005cf0 con 0x7fed8c147b40 2024-05-28T14:09:25.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed83e00640 1 -- 172.21.15.42:0/2170507641 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed840072d0 con 0x7fed8c147b40 2024-05-28T14:09:25.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed90e00640 1 -- 172.21.15.42:0/2170507641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8c09c320 con 0x7fed8c147b40 2024-05-28T14:09:25.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed90e00640 1 -- 172.21.15.42:0/2170507641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8c09c600 con 0x7fed8c147b40 2024-05-28T14:09:25.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.521+0000 7fed90e00640 1 -- 172.21.15.42:0/2170507641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed8c04fcf0 con 0x7fed8c147b40 2024-05-28T14:09:25.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.525+0000 7fed83e00640 1 -- 172.21.15.42:0/2170507641 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fed8401a070 con 0x7fed8c147b40 2024-05-28T14:09:25.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.525+0000 7fed83e00640 1 --2- 172.21.15.42:0/2170507641 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fed680694f0 0x7fed6806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:25.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.525+0000 7fed83e00640 1 -- 172.21.15.42:0/2170507641 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4971+0+0 (secure 0 0 0) 0x7fed84084a90 con 0x7fed8c147b40 2024-05-28T14:09:25.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.525+0000 7fed8b400640 1 --2- 172.21.15.42:0/2170507641 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fed680694f0 0x7fed6806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:25.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.525+0000 7fed8b400640 1 --2- 172.21.15.42:0/2170507641 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fed680694f0 0x7fed6806b9b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0ae000 tx=0x7fed7c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:25.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.529+0000 7fed83e00640 1 -- 172.21.15.42:0/2170507641 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fed84056160 con 0x7fed8c147b40 2024-05-28T14:09:25.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.725+0000 7fed90e00640 1 -- 172.21.15.42:0/2170507641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed8c0b54f0 con 0x7fed8c147b40 2024-05-28T14:09:25.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.725+0000 7fed83e00640 1 -- 172.21.15.42:0/2170507641 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+3866 (secure 0 0 0) 0x7fed8405a460 con 0x7fed8c147b40 2024-05-28T14:09:25.711 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:25.711 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2618011545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2618011545},{"type":"v1","addr":"172.21.15.63:6835","nonce":2618011545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":7077,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/986242955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":986242955},{"type":"v1","addr":"172.21.15.42:6836","nonce":986242955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":7080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1618433260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1618433260},{"type":"v1","addr":"172.21.15.63:6834","nonce":1618433260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":7899,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3143854734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":3143854734},{"type":"v1","addr":"172.21.15.42:6837","nonce":3143854734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[]} 2024-05-28T14:09:25.711 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 49 2024-05-28T14:09:25.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.729+0000 7fed90e00640 1 -- 172.21.15.42:0/2170507641 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fed680694f0 msgr2=0x7fed6806b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:25.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.729+0000 7fed90e00640 1 --2- 172.21.15.42:0/2170507641 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fed680694f0 0x7fed6806b9b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0ae000 tx=0x7fed7c008040 comp rx=0 tx=0).stop 2024-05-28T14:09:25.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.729+0000 7fed90e00640 1 -- 172.21.15.42:0/2170507641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c147b40 msgr2=0x7fed8c15cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:25.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.729+0000 7fed90e00640 1 --2- 172.21.15.42:0/2170507641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c147b40 0x7fed8c15cf40 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fed8400c400 tx=0x7fed8400c8c0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.729+0000 7fed90e00640 1 -- 172.21.15.42:0/2170507641 shutdown_connections 2024-05-28T14:09:25.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.729+0000 7fed90e00640 1 --2- 172.21.15.42:0/2170507641 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fed680694f0 0x7fed6806b9b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.729+0000 7fed90e00640 1 --2- 172.21.15.42:0/2170507641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c147b40 0x7fed8c15cf40 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.729+0000 7fed90e00640 1 --2- 172.21.15.42:0/2170507641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed8c0b1b00 0x7fed8c15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.729+0000 7fed90e00640 1 --2- 172.21.15.42:0/2170507641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed8c06bf30 0x7fed8c13eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.729+0000 7fed90e00640 1 -- 172.21.15.42:0/2170507641 >> 172.21.15.42:0/2170507641 conn(0x7fed8c0b67a0 msgr2=0x7fed8c0ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:25.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.729+0000 7fed90e00640 1 -- 172.21.15.42:0/2170507641 shutdown_connections 2024-05-28T14:09:25.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.729+0000 7fed90e00640 1 -- 172.21.15.42:0/2170507641 wait complete. 2024-05-28T14:09:25.733 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-28T14:09:25.734 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-28T14:09:25.734 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-28T14:09:25.734 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:09:25.734 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-28T14:09:25.734 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-28T14:09:25.734 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-28T14:09:25.735 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:09:25.735 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-28T14:09:25.735 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-28T14:09:25.735 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-28T14:09:25.735 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:09:25.735 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-28T14:09:25.735 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-28T14:09:25.736 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-28T14:09:25.736 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:09:25.747 INFO:tasks.ceph.mds.a:Stopped 2024-05-28T14:09:25.747 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-28T14:09:25.748 INFO:tasks.ceph.mds.c:Stopped 2024-05-28T14:09:25.748 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-28T14:09:25.750 INFO:tasks.ceph.mds.b:Stopped 2024-05-28T14:09:25.750 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-28T14:09:25.751 INFO:tasks.ceph.mds.d:Stopped 2024-05-28T14:09:25.751 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-28T14:09:25.793 INFO:tasks.ceph.mds.a:Started 2024-05-28T14:09:25.793 DEBUG:teuthology.parallel:result is None 2024-05-28T14:09:25.794 INFO:tasks.ceph.mds.c:Started 2024-05-28T14:09:25.795 DEBUG:teuthology.parallel:result is None 2024-05-28T14:09:25.795 INFO:tasks.ceph.mds.b:Started 2024-05-28T14:09:25.796 DEBUG:teuthology.parallel:result is None 2024-05-28T14:09:25.797 INFO:tasks.ceph.mds.d:Started 2024-05-28T14:09:25.798 DEBUG:teuthology.parallel:result is None 2024-05-28T14:09:25.798 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-28T14:09:25.880 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-05-28T14:09:25.882 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-05-28T14:09:25.888 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T14:09:25.889 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-28T14:09:25.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.961+0000 7f0490200640 1 Processor -- start 2024-05-28T14:09:25.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.961+0000 7f0490200640 1 -- start start 2024-05-28T14:09:25.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.965+0000 7f0490200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04880d0f80 0x7f04880b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:25.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.965+0000 7f0490200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f04880cddc0 0x7f048809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:25.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.965+0000 7f0490200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f048806dc50 con 0x7f0488162130 2024-05-28T14:09:25.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.965+0000 7f0490200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f048806df30 con 0x7f04880cddc0 2024-05-28T14:09:25.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.965+0000 7f0490200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f048806e210 con 0x7f04880d0f80 2024-05-28T14:09:25.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.965+0000 7f048ce00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04880d0f80 0x7f04880b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:25.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.965+0000 7f048e200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f04880cddc0 0x7f048809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:25.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.965+0000 7f048ce00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04880d0f80 0x7f04880b1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:58150/0 (socket says 172.21.15.42:58150) 2024-05-28T14:09:25.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.965+0000 7f048ce00640 1 -- 172.21.15.42:0/2430811815 learned_addr learned my addr 172.21.15.42:0/2430811815 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:25.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.965+0000 7f048ce00640 1 -- 172.21.15.42:0/2430811815 >> v2:172.21.15.63:3300/0 conn(0x7f04880cddc0 msgr2=0x7f048809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:25.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.965+0000 7f048ce00640 1 --2- 172.21.15.42:0/2430811815 >> v2:172.21.15.63:3300/0 conn(0x7f04880cddc0 0x7f048809aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.965+0000 7f048ce00640 1 -- 172.21.15.42:0/2430811815 >> v1:172.21.15.63:6789/0 conn(0x7f0488162130 legacy=0x7f04880d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:25.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.965+0000 7f048ce00640 1 -- 172.21.15.42:0/2430811815 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048806e4f0 con 0x7f04880d0f80 2024-05-28T14:09:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.965+0000 7f048ce00640 1 --2- 172.21.15.42:0/2430811815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04880d0f80 0x7f04880b1410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0474009870 tx=0x7f047401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0e644bad7bf063c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0487e00640 1 -- 172.21.15.42:0/2430811815 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04740a7070 con 0x7f04880d0f80 2024-05-28T14:09:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0487e00640 1 -- 172.21.15.42:0/2430811815 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04740b0070 con 0x7f04880d0f80 2024-05-28T14:09:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0487e00640 1 -- 172.21.15.42:0/2430811815 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04740ab890 con 0x7f04880d0f80 2024-05-28T14:09:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 -- 172.21.15.42:0/2430811815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04880d0f80 msgr2=0x7f04880b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 --2- 172.21.15.42:0/2430811815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04880d0f80 0x7f04880b1410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0474009870 tx=0x7f047401c560 comp rx=0 tx=0).stop 2024-05-28T14:09:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 -- 172.21.15.42:0/2430811815 shutdown_connections 2024-05-28T14:09:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 --2- 172.21.15.42:0/2430811815 >> v2:172.21.15.63:3300/0 conn(0x7f04880cddc0 0x7f048809aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 --2- 172.21.15.42:0/2430811815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04880d0f80 0x7f04880b1410 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 -- 172.21.15.42:0/2430811815 >> 172.21.15.42:0/2430811815 conn(0x7f04880b4050 msgr2=0x7f04880b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 -- 172.21.15.42:0/2430811815 shutdown_connections 2024-05-28T14:09:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 -- 172.21.15.42:0/2430811815 wait complete. 2024-05-28T14:09:25.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 Processor -- start 2024-05-28T14:09:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 -- start start 2024-05-28T14:09:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04880cddc0 0x7f048814eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04880d0f80 0x7f04881573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0488162130 0x7f04881578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f048809bb30 con 0x7f04880cddc0 2024-05-28T14:09:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f048809b9b0 con 0x7f04880d0f80 2024-05-28T14:09:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f0490200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f048809bcb0 con 0x7f0488162130 2024-05-28T14:09:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f048d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04880cddc0 0x7f048814eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f048d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04880cddc0 0x7f048814eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49586/0 (socket says 172.21.15.42:49586) 2024-05-28T14:09:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f048d800640 1 -- 172.21.15.42:0/718308959 learned_addr learned my addr 172.21.15.42:0/718308959 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f048ce00640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04880d0f80 0x7f04881573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.969+0000 7f048e200640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0488162130 0x7f04881578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:25.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f048d800640 1 -- 172.21.15.42:0/718308959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0488162130 msgr2=0x7f04881578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f048d800640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0488162130 0x7f04881578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f048d800640 1 -- 172.21.15.42:0/718308959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04880d0f80 msgr2=0x7f04881573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f048d800640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04880d0f80 0x7f04881573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f048d800640 1 -- 172.21.15.42:0/718308959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0488069330 con 0x7f04880cddc0 2024-05-28T14:09:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f048ce00640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04880d0f80 0x7f04881573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:09:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f048d800640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04880cddc0 0x7f048814eb20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f047c00c370 tx=0x7f047c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f0485600640 1 -- 172.21.15.42:0/718308959 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c0052d0 con 0x7f04880cddc0 2024-05-28T14:09:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f0485600640 1 -- 172.21.15.42:0/718308959 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f047c005470 con 0x7f04880cddc0 2024-05-28T14:09:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f0485600640 1 -- 172.21.15.42:0/718308959 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c007260 con 0x7f04880cddc0 2024-05-28T14:09:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f0490200640 1 -- 172.21.15.42:0/718308959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0488160320 con 0x7f04880cddc0 2024-05-28T14:09:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f0490200640 1 -- 172.21.15.42:0/718308959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04880b73b0 con 0x7f04880cddc0 2024-05-28T14:09:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f0490200640 1 -- 172.21.15.42:0/718308959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04880a0650 con 0x7f04880cddc0 2024-05-28T14:09:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f0485600640 1 -- 172.21.15.42:0/718308959 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f047c007400 con 0x7f04880cddc0 2024-05-28T14:09:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f0485600640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f04680694f0 0x7f046806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f0485600640 1 -- 172.21.15.42:0/718308959 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4971+0+0 (secure 0 0 0) 0x7f047c084ca0 con 0x7f04880cddc0 2024-05-28T14:09:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.973+0000 7f048ce00640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f04680694f0 0x7f046806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:25.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.977+0000 7f048ce00640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f04680694f0 0x7f046806b9b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f04740af000 tx=0x7f04740a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:25.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:25.977+0000 7f0485600640 1 -- 172.21.15.42:0/718308959 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f047c056310 con 0x7f04880cddc0 2024-05-28T14:09:26.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:26.157+0000 7f0490200640 1 -- 172.21.15.42:0/718308959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f048804fdd0 con 0x7f04880cddc0 2024-05-28T14:09:27.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.081+0000 7f0485600640 1 -- 172.21.15.42:0/718308959 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) ==== 97+0+0 (secure 0 0 0) 0x7f047c05a610 con 0x7f04880cddc0 2024-05-28T14:09:27.066 INFO:teuthology.orchestra.run.smithi042.stderr: removed all blocklist entries 2024-05-28T14:09:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.085+0000 7f0490200640 1 -- 172.21.15.42:0/718308959 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f04680694f0 msgr2=0x7f046806b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.085+0000 7f0490200640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f04680694f0 0x7f046806b9b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f04740af000 tx=0x7f04740a7000 comp rx=0 tx=0).stop 2024-05-28T14:09:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.085+0000 7f0490200640 1 -- 172.21.15.42:0/718308959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04880cddc0 msgr2=0x7f048814eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.085+0000 7f0490200640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04880cddc0 0x7f048814eb20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f047c00c370 tx=0x7f047c00c830 comp rx=0 tx=0).stop 2024-05-28T14:09:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.085+0000 7f0490200640 1 -- 172.21.15.42:0/718308959 shutdown_connections 2024-05-28T14:09:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.085+0000 7f0490200640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f04680694f0 0x7f046806b9b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.085+0000 7f0490200640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0488162130 0x7f04881578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.085+0000 7f0490200640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04880d0f80 0x7f04881573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.085+0000 7f0490200640 1 --2- 172.21.15.42:0/718308959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04880cddc0 0x7f048814eb20 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.085+0000 7f0490200640 1 -- 172.21.15.42:0/718308959 >> 172.21.15.42:0/718308959 conn(0x7f04880b4050 msgr2=0x7f04881590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.085+0000 7f0490200640 1 -- 172.21.15.42:0/718308959 shutdown_connections 2024-05-28T14:09:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.085+0000 7f0490200640 1 -- 172.21.15.42:0/718308959 wait complete. 2024-05-28T14:09:27.095 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-28T14:09:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.257+0000 7f0cd1800640 1 Processor -- start 2024-05-28T14:09:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.257+0000 7f0cd1800640 1 -- start start 2024-05-28T14:09:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.257+0000 7f0cd1800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ccc0d0e00 0x7f0ccc1663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.257+0000 7f0cd1800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ccc15f280 con 0x7f0ccc15ba40 2024-05-28T14:09:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.257+0000 7f0cd1800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ccc15f560 con 0x7f0ccc06f660 2024-05-28T14:09:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.257+0000 7f0cd1800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0ccc15f030 con 0x7f0ccc0d0e00 2024-05-28T14:09:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.257+0000 7f0cc2000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0ccc15ba40 0x7f0ccc0d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56446/0 (socket says 172.21.15.42:56446) 2024-05-28T14:09:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.257+0000 7f0cc2000640 1 -- 172.21.15.42:0/2729211890 learned_addr learned my addr 172.21.15.42:0/2729211890 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.257+0000 7f0ccb400640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ccc0d0e00 0x7f0ccc1663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:27.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.257+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965396718 0 0) 0x7f0ccc15f280 con 0x7f0ccc15ba40 2024-05-28T14:09:27.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.257+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cb0005740 con 0x7f0ccc15ba40 2024-05-28T14:09:27.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.257+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3462333199 0 0) 0x7f0ccc15f560 con 0x7f0ccc06f660 2024-05-28T14:09:27.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.257+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ccc15f280 con 0x7f0ccc06f660 2024-05-28T14:09:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3888642182 0 0) 0x7f0cb0005740 con 0x7f0ccc15ba40 2024-05-28T14:09:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ccc0d0e00 msgr2=0x7f0ccc1663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ccc0d0e00 0x7f0ccc1663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 >> v1:172.21.15.63:6789/0 conn(0x7f0ccc06f660 legacy=0x7f0ccc073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc15f840 con 0x7f0ccc15ba40 2024-05-28T14:09:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f0cb4001680 con 0x7f0ccc15ba40 2024-05-28T14:09:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0ccb400640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ccc0d0e00 0x7f0ccc1663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:09:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 >> v1:172.21.15.42:6789/0 conn(0x7f0ccc15ba40 legacy=0x7f0ccc0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb0005d90 0x7f0cb0006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cb0006a20 0x7f0cb0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ccc0d0e00 0x7f0cb0007450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc15f030 con 0x7f0cb0006a20 2024-05-28T14:09:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb000dda0 con 0x7f0cb0005d90 2024-05-28T14:09:27.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb000dfa0 con 0x7f0ccc0d0e00 2024-05-28T14:09:27.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cc2000640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cb0006a20 0x7f0cb0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:27.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0ccaa00640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb0005d90 0x7f0cb0006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:27.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0ccb400640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ccc0d0e00 0x7f0cb0007450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:27.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cc2000640 1 -- 172.21.15.42:0/2729211890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ccc0d0e00 msgr2=0x7f0cb0007450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:27.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cc2000640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ccc0d0e00 0x7f0cb0007450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cc2000640 1 -- 172.21.15.42:0/2729211890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb0005d90 msgr2=0x7f0cb0006190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:27.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cc2000640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb0005d90 0x7f0cb0006190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cc2000640 1 -- 172.21.15.42:0/2729211890 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb0005740 con 0x7f0cb0006a20 2024-05-28T14:09:27.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0ccaa00640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb0005d90 0x7f0cb0006190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:27.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0ccb400640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ccc0d0e00 0x7f0cb0007450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:27.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cc2000640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cb0006a20 0x7f0cb0006ea0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400c210 tx=0x7f0cb401cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd0f790f9172119 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:27.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4007cf0 con 0x7f0cb0006a20 2024-05-28T14:09:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb40a9020 con 0x7f0cb0006a20 2024-05-28T14:09:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.261+0000 7f0cca000640 1 -- 172.21.15.42:0/2729211890 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4005b20 con 0x7f0cb0006a20 2024-05-28T14:09:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cd1800640 1 -- 172.21.15.42:0/2729211890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cb0006a20 msgr2=0x7f0cb0006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cd1800640 1 --2- 172.21.15.42:0/2729211890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cb0006a20 0x7f0cb0006ea0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400c210 tx=0x7f0cb401cbd0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0ccb400640 1 -- 172.21.15.42:0/2729211890 reap_dead start 2024-05-28T14:09:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cd1800640 1 -- 172.21.15.42:0/2729211890 shutdown_connections 2024-05-28T14:09:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cd1800640 1 -- 172.21.15.42:0/2729211890 >> 172.21.15.42:0/2729211890 conn(0x7f0ccc06bdf0 msgr2=0x7f0ccc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cd1800640 1 -- 172.21.15.42:0/2729211890 shutdown_connections 2024-05-28T14:09:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cd1800640 1 -- 172.21.15.42:0/2729211890 wait complete. 2024-05-28T14:09:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cd1800640 1 Processor -- start 2024-05-28T14:09:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cd1800640 1 -- start start 2024-05-28T14:09:27.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cd1800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb0006a20 0x7f0ccc099a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cd1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ccc0afcf0 0x7f0ccc0b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cd1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ccc15a440 0x7f0ccc15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cd1800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc0b3e80 con 0x7f0ccc0afcf0 2024-05-28T14:09:27.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cd1800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc0b3d00 con 0x7f0ccc15a440 2024-05-28T14:09:27.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cd1800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0ccc0b4000 con 0x7f0cb0006a20 2024-05-28T14:09:27.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cc2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ccc0afcf0 0x7f0ccc0b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:27.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cc2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ccc0afcf0 0x7f0ccc0b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49604/0 (socket says 172.21.15.42:49604) 2024-05-28T14:09:27.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cc2000640 1 -- 172.21.15.42:0/462030288 learned_addr learned my addr 172.21.15.42:0/462030288 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:27.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cc2000640 1 -- 172.21.15.42:0/462030288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb0006a20 msgr2=0x7f0ccc099a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cc2000640 1 --2- 172.21.15.42:0/462030288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb0006a20 0x7f0ccc099a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cc2000640 1 -- 172.21.15.42:0/462030288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ccc15a440 msgr2=0x7f0ccc15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cc2000640 1 --2- 172.21.15.42:0/462030288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ccc15a440 0x7f0ccc15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.265+0000 7f0cc2000640 1 -- 172.21.15.42:0/462030288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc0b2830 con 0x7f0ccc0afcf0 2024-05-28T14:09:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.269+0000 7f0cc2000640 1 --2- 172.21.15.42:0/462030288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ccc0afcf0 0x7f0ccc0b0170 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f0cb401ca00 tx=0x7f0cb0007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.269+0000 7f0cc3400640 1 -- 172.21.15.42:0/462030288 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4001480 con 0x7f0ccc0afcf0 2024-05-28T14:09:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.269+0000 7f0cc3400640 1 -- 172.21.15.42:0/462030288 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb4001620 con 0x7f0ccc0afcf0 2024-05-28T14:09:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.269+0000 7f0cd1800640 1 -- 172.21.15.42:0/462030288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc0b2b10 con 0x7f0ccc0afcf0 2024-05-28T14:09:27.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.269+0000 7f0cc3400640 1 -- 172.21.15.42:0/462030288 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb40049f0 con 0x7f0ccc0afcf0 2024-05-28T14:09:27.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.269+0000 7f0cd1800640 1 -- 172.21.15.42:0/462030288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc0cd550 con 0x7f0ccc0afcf0 2024-05-28T14:09:27.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.269+0000 7f0cd1800640 1 -- 172.21.15.42:0/462030288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ccc068a90 con 0x7f0ccc0afcf0 2024-05-28T14:09:27.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.269+0000 7f0cc3400640 1 -- 172.21.15.42:0/462030288 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f0cb4004b90 con 0x7f0ccc0afcf0 2024-05-28T14:09:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.269+0000 7f0cc3400640 1 --2- 172.21.15.42:0/462030288 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ca80694f0 0x7f0ca806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.269+0000 7f0cc3400640 1 -- 172.21.15.42:0/462030288 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4885+0+0 (secure 0 0 0) 0x7f0cb411ec10 con 0x7f0ccc0afcf0 2024-05-28T14:09:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.269+0000 7f0ccaa00640 1 --2- 172.21.15.42:0/462030288 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ca80694f0 0x7f0ca806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.273+0000 7f0ccaa00640 1 --2- 172.21.15.42:0/462030288 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ca80694f0 0x7f0ca806b9b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0015a0 tx=0x7f0cbc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.273+0000 7f0cc3400640 1 -- 172.21.15.42:0/462030288 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0cb4016910 con 0x7f0ccc0afcf0 2024-05-28T14:09:27.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.493+0000 7f0cd1800640 1 -- 172.21.15.42:0/462030288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0ccc0b2ce0 con 0x7f0ccc0afcf0 2024-05-28T14:09:27.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.493+0000 7f0cc3400640 1 -- 172.21.15.42:0/462030288 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f0cb40f0c80 con 0x7f0ccc0afcf0 2024-05-28T14:09:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:27.478 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:09:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_dump": [ 2024-05-28T14:09:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.a", 2024-05-28T14:09:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmE+3RLRAADuTKGfKeVafGeLpgr9WKUg==", 2024-05-28T14:09:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:09:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:09:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:09:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.b", 2024-05-28T14:09:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVm1rmVOhAAV46KzDJnRfDZYIZanS0g2g==", 2024-05-28T14:09:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:09:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:09:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:09:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.c", 2024-05-28T14:09:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVme4aYNRAAVxJTlugnxtFTexUclF/IwA==", 2024-05-28T14:09:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:09:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:09:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:09:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.d", 2024-05-28T14:09:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmeAdBBhAAr4IVFQpjQXFVlnFTHnPV9w==", 2024-05-28T14:09:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:09:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:09:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:09:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.0", 2024-05-28T14:09:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDB4lVmGMDBJhAA3jIyR7XKLGXEgimIGNsw6A==", 2024-05-28T14:09:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:09:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:09:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.1", 2024-05-28T14:09:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDE4lVmxMisLxAAcIz7GPlkZwvW5ig+DbAVGg==", 2024-05-28T14:09:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:09:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:09:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.2", 2024-05-28T14:09:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDH4lVmSSk4OhAAmLlLysM4Im/Kew4Xbq2/OA==", 2024-05-28T14:09:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:09:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:09:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.3", 2024-05-28T14:09:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDL4lVmvzrYBhAAiAAOQXZBuTsMVYa1vE5rkg==", 2024-05-28T14:09:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:09:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:09:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.4", 2024-05-28T14:09:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDQ4lVmrqnTOhAA2pIfgBT9LJDIhp+GgFy8Gg==", 2024-05-28T14:09:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:09:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:09:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.5", 2024-05-28T14:09:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDU4lVmYpsQCxAAoTK2aMBPrHKvsUnNOTj/Dw==", 2024-05-28T14:09:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:09:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:09:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.6", 2024-05-28T14:09:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDX4lVm6TCAFxAAs3rkDyKcLhWrn/EkIo0bVA==", 2024-05-28T14:09:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:09:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:09:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.7", 2024-05-28T14:09:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDa4lVmr5NwIRAA7Lb70VYV82dr0Frn8CCUXg==", 2024-05-28T14:09:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:09:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:09:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.0", 2024-05-28T14:09:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw==", 2024-05-28T14:09:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:09:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-05-28T14:09:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-28T14:09:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.1", 2024-05-28T14:09:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmcfkLFBAADNyODKTtYAlFWquZ/LeeqQ==", 2024-05-28T14:09:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:09:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:09:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:09:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:09:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.2", 2024-05-28T14:09:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmoxx+EBAA8mLNak5AwRmWQIgkYtMOrg==", 2024-05-28T14:09:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:09:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:09:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:09:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:09:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.3", 2024-05-28T14:09:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmxSUsGhAAbgr1eOh/sMfrEONKKHFo0g==", 2024-05-28T14:09:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:09:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:09:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:09:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:09:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.admin", 2024-05-28T14:09:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmOSBPERAAt4x2DBal4PnqxqeWQutLDQ==", 2024-05-28T14:09:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:09:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:09:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:09:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.x", 2024-05-28T14:09:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmNO5tKRAA70hPLwoE83keovlq2rolxQ==", 2024-05-28T14:09:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:09:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T14:09:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.y", 2024-05-28T14:09:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmxTVyJRAATMLAAqKne20YvV537uXZ6g==", 2024-05-28T14:09:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:09:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:09:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T14:09:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:09:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-28T14:09:27.492 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:09:27.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.497+0000 7f0cd1800640 1 -- 172.21.15.42:0/462030288 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ca80694f0 msgr2=0x7f0ca806b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.497+0000 7f0cd1800640 1 --2- 172.21.15.42:0/462030288 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ca80694f0 0x7f0ca806b9b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0015a0 tx=0x7f0cbc009040 comp rx=0 tx=0).stop 2024-05-28T14:09:27.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.497+0000 7f0cd1800640 1 -- 172.21.15.42:0/462030288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ccc0afcf0 msgr2=0x7f0ccc0b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.497+0000 7f0cd1800640 1 --2- 172.21.15.42:0/462030288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ccc0afcf0 0x7f0ccc0b0170 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f0cb401ca00 tx=0x7f0cb0007a50 comp rx=0 tx=0).stop 2024-05-28T14:09:27.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.497+0000 7f0cd1800640 1 -- 172.21.15.42:0/462030288 shutdown_connections 2024-05-28T14:09:27.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.497+0000 7f0cd1800640 1 --2- 172.21.15.42:0/462030288 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ca80694f0 0x7f0ca806b9b0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.497+0000 7f0cd1800640 1 --2- 172.21.15.42:0/462030288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ccc15a440 0x7f0ccc15a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.497+0000 7f0cd1800640 1 --2- 172.21.15.42:0/462030288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ccc0afcf0 0x7f0ccc0b0170 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.497+0000 7f0cd1800640 1 --2- 172.21.15.42:0/462030288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb0006a20 0x7f0ccc099a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.497+0000 7f0cd1800640 1 -- 172.21.15.42:0/462030288 >> 172.21.15.42:0/462030288 conn(0x7f0ccc06bdf0 msgr2=0x7f0ccc0d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:27.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.497+0000 7f0cd1800640 1 -- 172.21.15.42:0/462030288 shutdown_connections 2024-05-28T14:09:27.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.497+0000 7f0cd1800640 1 -- 172.21.15.42:0/462030288 wait complete. 2024-05-28T14:09:27.502 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:09:27.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f4652400640 1 Processor -- start 2024-05-28T14:09:27.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f4652400640 1 -- start start 2024-05-28T14:09:27.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f4652400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f464c0ce860 0x7f464c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f4652400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f464c162c80 0x7f464c15e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f4652400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f464c0d0de0 con 0x7f464c0d0900 2024-05-28T14:09:27.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f4652400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f464c0d19f0 con 0x7f464c162c80 2024-05-28T14:09:27.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f4652400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f464c0d10c0 con 0x7f464c0ce860 2024-05-28T14:09:27.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f464be00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f464c162c80 0x7f464c15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:27.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f464b400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f464c0ce860 0x7f464c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:27.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f464be00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f464c162c80 0x7f464c15e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:55570/0 (socket says 172.21.15.42:55570) 2024-05-28T14:09:27.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f464aa00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f464c0d0900 0x7f464c162570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:43802/0 (socket says 172.21.15.42:43802) 2024-05-28T14:09:27.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f464b400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f464c0ce860 0x7f464c06d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:54088/0 (socket says 172.21.15.42:54088) 2024-05-28T14:09:27.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f464be00640 1 -- 172.21.15.42:0/1087204037 learned_addr learned my addr 172.21.15.42:0/1087204037 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:27.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f464be00640 1 -- 172.21.15.42:0/1087204037 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f464c0ce860 msgr2=0x7f464c06d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:27.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f464be00640 1 --2- 172.21.15.42:0/1087204037 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f464c0ce860 0x7f464c06d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f464be00640 1 -- 172.21.15.42:0/1087204037 >> v1:172.21.15.63:6789/0 conn(0x7f464c0d0900 legacy=0x7f464c162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.661+0000 7f464be00640 1 -- 172.21.15.42:0/1087204037 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c0d13a0 con 0x7f464c162c80 2024-05-28T14:09:27.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.665+0000 7f464be00640 1 --2- 172.21.15.42:0/1087204037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f464c162c80 0x7f464c15e200 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4644009e50 tx=0x7f464401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec0a8aaba031349 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:27.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.665+0000 7f464a000640 1 -- 172.21.15.42:0/1087204037 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46440a5850 con 0x7f464c162c80 2024-05-28T14:09:27.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.665+0000 7f464a000640 1 -- 172.21.15.42:0/1087204037 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46440a59f0 con 0x7f464c162c80 2024-05-28T14:09:27.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.665+0000 7f464a000640 1 -- 172.21.15.42:0/1087204037 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46440a5dc0 con 0x7f464c162c80 2024-05-28T14:09:27.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.665+0000 7f4652400640 1 -- 172.21.15.42:0/1087204037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f464c162c80 msgr2=0x7f464c15e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.665+0000 7f4652400640 1 --2- 172.21.15.42:0/1087204037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f464c162c80 0x7f464c15e200 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4644009e50 tx=0x7f464401c970 comp rx=0 tx=0).stop 2024-05-28T14:09:27.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.665+0000 7f4652400640 1 -- 172.21.15.42:0/1087204037 shutdown_connections 2024-05-28T14:09:27.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.665+0000 7f4652400640 1 --2- 172.21.15.42:0/1087204037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f464c162c80 0x7f464c15e200 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.665+0000 7f4652400640 1 --2- 172.21.15.42:0/1087204037 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f464c0ce860 0x7f464c06d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.665+0000 7f4652400640 1 -- 172.21.15.42:0/1087204037 >> 172.21.15.42:0/1087204037 conn(0x7f464c0b4080 msgr2=0x7f464c0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:27.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.665+0000 7f4652400640 1 -- 172.21.15.42:0/1087204037 shutdown_connections 2024-05-28T14:09:27.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.665+0000 7f4652400640 1 -- 172.21.15.42:0/1087204037 wait complete. 2024-05-28T14:09:27.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f4652400640 1 Processor -- start 2024-05-28T14:09:27.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f4652400640 1 -- start start 2024-05-28T14:09:27.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f4652400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f464c0ce860 0x7f464c0cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f4652400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f464c0d0900 0x7f464c0cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f4652400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f464c162c80 0x7f464c0c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f4652400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f464c0ccf70 con 0x7f464c0ce860 2024-05-28T14:09:27.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f4652400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f464c0ccdf0 con 0x7f464c162c80 2024-05-28T14:09:27.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f4652400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f464c0cd0f0 con 0x7f464c0d0900 2024-05-28T14:09:27.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f464b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f464c0ce860 0x7f464c0cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:27.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f464b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f464c0ce860 0x7f464c0cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56226/0 (socket says 172.21.15.42:56226) 2024-05-28T14:09:27.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f464b400640 1 -- 172.21.15.42:0/1797672504 learned_addr learned my addr 172.21.15.42:0/1797672504 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:27.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f464aa00640 1 --2- 172.21.15.42:0/1797672504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f464c0d0900 0x7f464c0cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:27.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f464be00640 1 --2- 172.21.15.42:0/1797672504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f464c162c80 0x7f464c0c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:27.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f464b400640 1 -- 172.21.15.42:0/1797672504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f464c0d0900 msgr2=0x7f464c0cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f464b400640 1 --2- 172.21.15.42:0/1797672504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f464c0d0900 0x7f464c0cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f464b400640 1 -- 172.21.15.42:0/1797672504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f464c162c80 msgr2=0x7f464c0c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f464b400640 1 --2- 172.21.15.42:0/1797672504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f464c162c80 0x7f464c0c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f464b400640 1 -- 172.21.15.42:0/1797672504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c0ca4e0 con 0x7f464c0ce860 2024-05-28T14:09:27.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f464b400640 1 --2- 172.21.15.42:0/1797672504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f464c0ce860 0x7f464c0cb800 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f463c00e930 tx=0x7f463c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:27.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f4643400640 1 -- 172.21.15.42:0/1797672504 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c017070 con 0x7f464c0ce860 2024-05-28T14:09:27.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f4643400640 1 -- 172.21.15.42:0/1797672504 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f463c00ca00 con 0x7f464c0ce860 2024-05-28T14:09:27.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f4652400640 1 -- 172.21.15.42:0/1797672504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464c0ca7c0 con 0x7f464c0ce860 2024-05-28T14:09:27.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f4643400640 1 -- 172.21.15.42:0/1797672504 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c00cdb0 con 0x7f464c0ce860 2024-05-28T14:09:27.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f4652400640 1 -- 172.21.15.42:0/1797672504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464c0cad80 con 0x7f464c0ce860 2024-05-28T14:09:27.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.669+0000 7f4652400640 1 -- 172.21.15.42:0/1797672504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f464c0cb160 con 0x7f464c0ce860 2024-05-28T14:09:27.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.673+0000 7f4643400640 1 -- 172.21.15.42:0/1797672504 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f463c01a070 con 0x7f464c0ce860 2024-05-28T14:09:27.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.673+0000 7f4643400640 1 --2- 172.21.15.42:0/1797672504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f462c0694f0 0x7f462c06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.673+0000 7f4643400640 1 -- 172.21.15.42:0/1797672504 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4885+0+0 (secure 0 0 0) 0x7f463c085b30 con 0x7f464c0ce860 2024-05-28T14:09:27.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.673+0000 7f464aa00640 1 --2- 172.21.15.42:0/1797672504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f462c0694f0 0x7f462c06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:27.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.673+0000 7f464aa00640 1 --2- 172.21.15.42:0/1797672504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f462c0694f0 0x7f462c06b9b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f464c0cc5d0 tx=0x7f4638007e90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:27.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.677+0000 7f4643400640 1 -- 172.21.15.42:0/1797672504 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f464c0cb160 con 0x7f464c0ce860 2024-05-28T14:09:27.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.845+0000 7f4652400640 1 -- 172.21.15.42:0/1797672504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f464c155500 con 0x7f464c0ce860 2024-05-28T14:09:27.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.845+0000 7f4643400640 1 -- 172.21.15.42:0/1797672504 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+12875 (secure 0 0 0) 0x7f463c0572c0 con 0x7f464c0ce860 2024-05-28T14:09:27.828 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:27.829 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":86,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:09:27.083134+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:09:27.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.849+0000 7f4652400640 1 -- 172.21.15.42:0/1797672504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f462c0694f0 msgr2=0x7f462c06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.849+0000 7f4652400640 1 --2- 172.21.15.42:0/1797672504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f462c0694f0 0x7f462c06b9b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f464c0cc5d0 tx=0x7f4638007e90 comp rx=0 tx=0).stop 2024-05-28T14:09:27.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.849+0000 7f4652400640 1 -- 172.21.15.42:0/1797672504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f464c0ce860 msgr2=0x7f464c0cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:27.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.849+0000 7f4652400640 1 --2- 172.21.15.42:0/1797672504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f464c0ce860 0x7f464c0cb800 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f463c00e930 tx=0x7f463c00edf0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.849+0000 7f4652400640 1 -- 172.21.15.42:0/1797672504 shutdown_connections 2024-05-28T14:09:27.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.849+0000 7f4652400640 1 --2- 172.21.15.42:0/1797672504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f462c0694f0 0x7f462c06b9b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.849+0000 7f4652400640 1 --2- 172.21.15.42:0/1797672504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f464c162c80 0x7f464c0c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.849+0000 7f4652400640 1 --2- 172.21.15.42:0/1797672504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f464c0d0900 0x7f464c0cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.849+0000 7f4652400640 1 --2- 172.21.15.42:0/1797672504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f464c0ce860 0x7f464c0cb800 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:27.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.849+0000 7f4652400640 1 -- 172.21.15.42:0/1797672504 >> 172.21.15.42:0/1797672504 conn(0x7f464c0b4080 msgr2=0x7f464c071660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:27.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.849+0000 7f4652400640 1 -- 172.21.15.42:0/1797672504 shutdown_connections 2024-05-28T14:09:27.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:27.849+0000 7f4652400640 1 -- 172.21.15.42:0/1797672504 wait complete. 2024-05-28T14:09:27.850 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:09:27.851 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:09:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5754600640 1 Processor -- start 2024-05-28T14:09:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5754600640 1 -- start start 2024-05-28T14:09:27.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5754600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f574c15c0c0 0x7f574c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:27.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5754600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f574c0afe40 con 0x7f574c06ee20 2024-05-28T14:09:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5754600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f574c15c600 con 0x7f574c144f80 2024-05-28T14:09:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5754600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f574c06f7c0 con 0x7f574c15c0c0 2024-05-28T14:09:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5751200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f574c06ee20 0x7f574c15b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47274/0 (socket says 172.21.15.42:47274) 2024-05-28T14:09:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5751200640 1 -- 172.21.15.42:0/3101349903 learned_addr learned my addr 172.21.15.42:0/3101349903 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5752600640 1 --2- 172.21.15.42:0/3101349903 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f574c15c0c0 0x7f574c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955715996 0 0) 0x7f574c0afe40 con 0x7f574c06ee20 2024-05-28T14:09:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5734005740 con 0x7f574c06ee20 2024-05-28T14:09:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 74756266 0 0) 0x7f574c15c600 con 0x7f574c144f80 2024-05-28T14:09:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f574c0afe40 con 0x7f574c144f80 2024-05-28T14:09:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3685006017 0 0) 0x7f5734005740 con 0x7f574c06ee20 2024-05-28T14:09:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f574c15c0c0 msgr2=0x7f574c06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 --2- 172.21.15.42:0/3101349903 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f574c15c0c0 0x7f574c06f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 >> v1:172.21.15.63:6789/0 conn(0x7f574c144f80 legacy=0x7f574c072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c15c8e0 con 0x7f574c06ee20 2024-05-28T14:09:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f5738001680 con 0x7f574c06ee20 2024-05-28T14:09:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5752600640 1 --2- 172.21.15.42:0/3101349903 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f574c15c0c0 0x7f574c06f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:09:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 >> v1:172.21.15.42:6789/0 conn(0x7f574c06ee20 legacy=0x7f574c15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 --2- 172.21.15.42:0/3101349903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734005d90 0x7f5734006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 --2- 172.21.15.42:0/3101349903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5734006a20 0x7f5734006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 --2- 172.21.15.42:0/3101349903 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f574c15c0c0 0x7f57340073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f573400dbe0 con 0x7f5734005d90 2024-05-28T14:09:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f573400dd90 con 0x7f5734006a20 2024-05-28T14:09:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f573400df90 con 0x7f574c15c0c0 2024-05-28T14:09:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5751c00640 1 --2- 172.21.15.42:0/3101349903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734005d90 0x7f5734006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5751200640 1 --2- 172.21.15.42:0/3101349903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5734006a20 0x7f5734006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5752600640 1 --2- 172.21.15.42:0/3101349903 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f574c15c0c0 0x7f57340073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5751c00640 1 -- 172.21.15.42:0/3101349903 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f574c15c0c0 msgr2=0x7f57340073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5751c00640 1 --2- 172.21.15.42:0/3101349903 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f574c15c0c0 0x7f57340073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5751c00640 1 -- 172.21.15.42:0/3101349903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5734006a20 msgr2=0x7f5734006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5751c00640 1 --2- 172.21.15.42:0/3101349903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5734006a20 0x7f5734006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.013+0000 7f5751c00640 1 -- 172.21.15.42:0/3101349903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5734005740 con 0x7f5734005d90 2024-05-28T14:09:28.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.017+0000 7f5751c00640 1 --2- 172.21.15.42:0/3101349903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734005d90 0x7f5734006190 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5740007fd0 tx=0x7f574001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6c6dd7ff92328 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:28.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.017+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740001520 con 0x7f5734005d90 2024-05-28T14:09:28.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.017+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57400052f0 con 0x7f5734005d90 2024-05-28T14:09:28.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.017+0000 7f574be00640 1 -- 172.21.15.42:0/3101349903 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740003840 con 0x7f5734005d90 2024-05-28T14:09:28.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.017+0000 7f5754600640 1 -- 172.21.15.42:0/3101349903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734005d90 msgr2=0x7f5734006190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.017+0000 7f5754600640 1 --2- 172.21.15.42:0/3101349903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734005d90 0x7f5734006190 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5740007fd0 tx=0x7f574001c540 comp rx=0 tx=0).stop 2024-05-28T14:09:28.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.017+0000 7f5752600640 1 -- 172.21.15.42:0/3101349903 reap_dead start 2024-05-28T14:09:28.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.017+0000 7f5754600640 1 -- 172.21.15.42:0/3101349903 shutdown_connections 2024-05-28T14:09:28.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.017+0000 7f5754600640 1 -- 172.21.15.42:0/3101349903 >> 172.21.15.42:0/3101349903 conn(0x7f574c0b21f0 msgr2=0x7f574c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:28.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.017+0000 7f5754600640 1 -- 172.21.15.42:0/3101349903 shutdown_connections 2024-05-28T14:09:28.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.017+0000 7f5754600640 1 -- 172.21.15.42:0/3101349903 wait complete. 2024-05-28T14:09:28.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5754600640 1 Processor -- start 2024-05-28T14:09:28.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5754600640 1 -- start start 2024-05-28T14:09:28.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5754600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5734005d90 0x7f574c134380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5754600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f574c15e370 0x7f574c1644c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5754600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f574c15e7d0 0x7f574c164a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5754600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f574c0724d0 con 0x7f574c15e7d0 2024-05-28T14:09:28.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5754600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f574c0b42a0 con 0x7f5734005d90 2024-05-28T14:09:28.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5754600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f574c06f610 con 0x7f574c15e370 2024-05-28T14:09:28.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5752600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f574c15e7d0 0x7f574c164a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5752600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f574c15e7d0 0x7f574c164a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56248/0 (socket says 172.21.15.42:56248) 2024-05-28T14:09:28.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5752600640 1 -- 172.21.15.42:0/3020677982 learned_addr learned my addr 172.21.15.42:0/3020677982 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:28.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5751c00640 1 --2- 172.21.15.42:0/3020677982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5734005d90 0x7f574c134380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5751200640 1 --2- 172.21.15.42:0/3020677982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f574c15e370 0x7f574c1644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5752600640 1 -- 172.21.15.42:0/3020677982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f574c15e370 msgr2=0x7f574c1644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5752600640 1 --2- 172.21.15.42:0/3020677982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f574c15e370 0x7f574c1644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5752600640 1 -- 172.21.15.42:0/3020677982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5734005d90 msgr2=0x7f574c134380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5752600640 1 --2- 172.21.15.42:0/3020677982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5734005d90 0x7f574c134380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5752600640 1 -- 172.21.15.42:0/3020677982 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c13ca50 con 0x7f574c15e7d0 2024-05-28T14:09:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5752600640 1 --2- 172.21.15.42:0/3020677982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f574c15e7d0 0x7f574c164a00 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5744009ed0 tx=0x7f574400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5749600640 1 -- 172.21.15.42:0/3020677982 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574400fe20 con 0x7f574c15e7d0 2024-05-28T14:09:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5749600640 1 -- 172.21.15.42:0/3020677982 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57440039a0 con 0x7f574c15e7d0 2024-05-28T14:09:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5749600640 1 -- 172.21.15.42:0/3020677982 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5744003d90 con 0x7f574c15e7d0 2024-05-28T14:09:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5754600640 1 -- 172.21.15.42:0/3020677982 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574c13cd30 con 0x7f574c15e7d0 2024-05-28T14:09:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.021+0000 7f5754600640 1 -- 172.21.15.42:0/3020677982 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f574c13d2f0 con 0x7f574c15e7d0 2024-05-28T14:09:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.025+0000 7f5754600640 1 -- 172.21.15.42:0/3020677982 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f574c0738b0 con 0x7f574c15e7d0 2024-05-28T14:09:28.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.025+0000 7f5749600640 1 -- 172.21.15.42:0/3020677982 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f574401f020 con 0x7f574c15e7d0 2024-05-28T14:09:28.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.025+0000 7f5749600640 1 --2- 172.21.15.42:0/3020677982 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f572c0694f0 0x7f572c06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.025+0000 7f5749600640 1 -- 172.21.15.42:0/3020677982 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4885+0+0 (secure 0 0 0) 0x7f5744085af0 con 0x7f574c15e7d0 2024-05-28T14:09:28.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.025+0000 7f5751c00640 1 --2- 172.21.15.42:0/3020677982 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f572c0694f0 0x7f572c06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.025+0000 7f5751c00640 1 --2- 172.21.15.42:0/3020677982 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f572c0694f0 0x7f572c06b9b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f57400ac000 tx=0x7f57400a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:28.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.029+0000 7f5749600640 1 -- 172.21.15.42:0/3020677982 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5744010040 con 0x7f574c15e7d0 2024-05-28T14:09:28.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.197+0000 7f5754600640 1 -- 172.21.15.42:0/3020677982 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f574c04fdd0 con 0x7f574c15e7d0 2024-05-28T14:09:28.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.197+0000 7f5749600640 1 -- 172.21.15.42:0/3020677982 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f5744057160 con 0x7f574c15e7d0 2024-05-28T14:09:28.182 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:09:28.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.201+0000 7f5754600640 1 -- 172.21.15.42:0/3020677982 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f572c0694f0 msgr2=0x7f572c06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.201+0000 7f5754600640 1 --2- 172.21.15.42:0/3020677982 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f572c0694f0 0x7f572c06b9b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f57400ac000 tx=0x7f57400a6000 comp rx=0 tx=0).stop 2024-05-28T14:09:28.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.201+0000 7f5754600640 1 -- 172.21.15.42:0/3020677982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f574c15e7d0 msgr2=0x7f574c164a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.201+0000 7f5754600640 1 --2- 172.21.15.42:0/3020677982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f574c15e7d0 0x7f574c164a00 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5744009ed0 tx=0x7f574400f580 comp rx=0 tx=0).stop 2024-05-28T14:09:28.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.201+0000 7f5754600640 1 -- 172.21.15.42:0/3020677982 shutdown_connections 2024-05-28T14:09:28.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.201+0000 7f5754600640 1 --2- 172.21.15.42:0/3020677982 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f572c0694f0 0x7f572c06b9b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.201+0000 7f5754600640 1 --2- 172.21.15.42:0/3020677982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f574c15e7d0 0x7f574c164a00 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.201+0000 7f5754600640 1 --2- 172.21.15.42:0/3020677982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f574c15e370 0x7f574c1644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.201+0000 7f5754600640 1 --2- 172.21.15.42:0/3020677982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5734005d90 0x7f574c134380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.201+0000 7f5754600640 1 -- 172.21.15.42:0/3020677982 >> 172.21.15.42:0/3020677982 conn(0x7f574c0b21f0 msgr2=0x7f574c072960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.201+0000 7f5754600640 1 -- 172.21.15.42:0/3020677982 shutdown_connections 2024-05-28T14:09:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.201+0000 7f5754600640 1 -- 172.21.15.42:0/3020677982 wait complete. 2024-05-28T14:09:28.206 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-28T14:09:28.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1f5400640 1 Processor -- start 2024-05-28T14:09:28.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1f5400640 1 -- start start 2024-05-28T14:09:28.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1f5400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff1f00ce860 0x7ff1f006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1f5400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff1f00d0900 0x7ff1f0162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1f5400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1f006dcf0 0x7ff1f0162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1f5400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f00d19f0 con 0x7ff1f00ce860 2024-05-28T14:09:28.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1f5400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f00d0de0 con 0x7ff1f00d0900 2024-05-28T14:09:28.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1f5400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f00d10c0 con 0x7ff1f006dcf0 2024-05-28T14:09:28.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1ee800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff1f00ce860 0x7ff1f006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1ee800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff1f00ce860 0x7ff1f006d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56250/0 (socket says 172.21.15.42:56250) 2024-05-28T14:09:28.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1ee800640 1 -- 172.21.15.42:0/4046784870 learned_addr learned my addr 172.21.15.42:0/4046784870 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:28.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1ef200640 1 --2- 172.21.15.42:0/4046784870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1f006dcf0 0x7ff1f0162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1e5e00640 1 --2- 172.21.15.42:0/4046784870 >> v2:172.21.15.63:3300/0 conn(0x7ff1f00d0900 0x7ff1f0162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1ee800640 1 -- 172.21.15.42:0/4046784870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1f006dcf0 msgr2=0x7ff1f0162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:28.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1ee800640 1 --2- 172.21.15.42:0/4046784870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1f006dcf0 0x7ff1f0162ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1ee800640 1 -- 172.21.15.42:0/4046784870 >> v2:172.21.15.63:3300/0 conn(0x7ff1f00d0900 msgr2=0x7ff1f0162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:28.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1ee800640 1 --2- 172.21.15.42:0/4046784870 >> v2:172.21.15.63:3300/0 conn(0x7ff1f00d0900 0x7ff1f0162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1ee800640 1 -- 172.21.15.42:0/4046784870 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f00d13a0 con 0x7ff1f00ce860 2024-05-28T14:09:28.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.365+0000 7ff1e5e00640 1 --2- 172.21.15.42:0/4046784870 >> v2:172.21.15.63:3300/0 conn(0x7ff1f00d0900 0x7ff1f0162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:28.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.369+0000 7ff1ee800640 1 --2- 172.21.15.42:0/4046784870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f00ce860 0x7ff1f006d7b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00d0670 tx=0x7ff1e001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84551ae3cd5e838b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:28.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.369+0000 7ff1ede00640 1 -- 172.21.15.42:0/4046784870 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00ab640 con 0x7ff1f00ce860 2024-05-28T14:09:28.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.369+0000 7ff1ede00640 1 -- 172.21.15.42:0/4046784870 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e00b0070 con 0x7ff1f00ce860 2024-05-28T14:09:28.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.369+0000 7ff1ede00640 1 -- 172.21.15.42:0/4046784870 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00aba00 con 0x7ff1f00ce860 2024-05-28T14:09:28.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.369+0000 7ff1f5400640 1 -- 172.21.15.42:0/4046784870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f00ce860 msgr2=0x7ff1f006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.369+0000 7ff1f5400640 1 --2- 172.21.15.42:0/4046784870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f00ce860 0x7ff1f006d7b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00d0670 tx=0x7ff1e001c460 comp rx=0 tx=0).stop 2024-05-28T14:09:28.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.369+0000 7ff1f5400640 1 -- 172.21.15.42:0/4046784870 shutdown_connections 2024-05-28T14:09:28.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.369+0000 7ff1f5400640 1 --2- 172.21.15.42:0/4046784870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1f006dcf0 0x7ff1f0162ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.369+0000 7ff1f5400640 1 --2- 172.21.15.42:0/4046784870 >> v2:172.21.15.63:3300/0 conn(0x7ff1f00d0900 0x7ff1f0162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.369+0000 7ff1f5400640 1 --2- 172.21.15.42:0/4046784870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f00ce860 0x7ff1f006d7b0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.369+0000 7ff1f5400640 1 -- 172.21.15.42:0/4046784870 >> 172.21.15.42:0/4046784870 conn(0x7ff1f00b4080 msgr2=0x7ff1f00b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:28.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.369+0000 7ff1f5400640 1 -- 172.21.15.42:0/4046784870 shutdown_connections 2024-05-28T14:09:28.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.369+0000 7ff1f5400640 1 -- 172.21.15.42:0/4046784870 wait complete. 2024-05-28T14:09:28.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1f5400640 1 Processor -- start 2024-05-28T14:09:28.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1f5400640 1 -- start start 2024-05-28T14:09:28.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1f5400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1f006dcf0 0x7ff1f00caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1f5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f00ce860 0x7ff1f00cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1f5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1f00d0900 0x7ff1f014f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1f5400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f015eda0 con 0x7ff1f00ce860 2024-05-28T14:09:28.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1f5400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f015ec20 con 0x7ff1f00d0900 2024-05-28T14:09:28.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1f5400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f009aa50 con 0x7ff1f006dcf0 2024-05-28T14:09:28.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1e5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f00ce860 0x7ff1f00cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1e5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f00ce860 0x7ff1f00cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56256/0 (socket says 172.21.15.42:56256) 2024-05-28T14:09:28.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1e5e00640 1 -- 172.21.15.42:0/1133687473 learned_addr learned my addr 172.21.15.42:0/1133687473 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:28.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1ee800640 1 --2- 172.21.15.42:0/1133687473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1f006dcf0 0x7ff1f00caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1ef200640 1 --2- 172.21.15.42:0/1133687473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1f00d0900 0x7ff1f014f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1e5e00640 1 -- 172.21.15.42:0/1133687473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1f006dcf0 msgr2=0x7ff1f00caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1e5e00640 1 --2- 172.21.15.42:0/1133687473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1f006dcf0 0x7ff1f00caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1e5e00640 1 -- 172.21.15.42:0/1133687473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1f00d0900 msgr2=0x7ff1f014f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1e5e00640 1 --2- 172.21.15.42:0/1133687473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1f00d0900 0x7ff1f014f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1e5e00640 1 -- 172.21.15.42:0/1133687473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f00cb070 con 0x7ff1f00ce860 2024-05-28T14:09:28.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1e5e00640 1 --2- 172.21.15.42:0/1133687473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f00ce860 0x7ff1f00cc110 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800d830 tx=0x7ff1d800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:28.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1e7400640 1 -- 172.21.15.42:0/1133687473 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8018020 con 0x7ff1f00ce860 2024-05-28T14:09:28.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1e7400640 1 -- 172.21.15.42:0/1133687473 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1d80096c0 con 0x7ff1f00ce860 2024-05-28T14:09:28.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1e7400640 1 -- 172.21.15.42:0/1133687473 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8009ab0 con 0x7ff1f00ce860 2024-05-28T14:09:28.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1f5400640 1 -- 172.21.15.42:0/1133687473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f00cb350 con 0x7ff1f00ce860 2024-05-28T14:09:28.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.373+0000 7ff1f5400640 1 -- 172.21.15.42:0/1133687473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f00cb910 con 0x7ff1f00ce860 2024-05-28T14:09:28.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.377+0000 7ff1f5400640 1 -- 172.21.15.42:0/1133687473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f0068a90 con 0x7ff1f00ce860 2024-05-28T14:09:28.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.377+0000 7ff1e7400640 1 -- 172.21.15.42:0/1133687473 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7ff1d801b020 con 0x7ff1f00ce860 2024-05-28T14:09:28.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.377+0000 7ff1e7400640 1 --2- 172.21.15.42:0/1133687473 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1cc0694f0 0x7ff1cc06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.377+0000 7ff1e7400640 1 -- 172.21.15.42:0/1133687473 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4885+0+0 (secure 0 0 0) 0x7ff1d8085ed0 con 0x7ff1f00ce860 2024-05-28T14:09:28.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.377+0000 7ff1ee800640 1 --2- 172.21.15.42:0/1133687473 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1cc0694f0 0x7ff1cc06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.377+0000 7ff1ee800640 1 --2- 172.21.15.42:0/1133687473 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1cc0694f0 0x7ff1cc06b9b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00af000 tx=0x7ff1e00a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:28.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.381+0000 7ff1e7400640 1 -- 172.21.15.42:0/1133687473 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff1d8022020 con 0x7ff1f00ce860 2024-05-28T14:09:28.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.549+0000 7ff1f5400640 1 -- 172.21.15.42:0/1133687473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff1f00b4ce0 con 0x7ff1f00ce860 2024-05-28T14:09:28.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.549+0000 7ff1e7400640 1 -- 172.21.15.42:0/1133687473 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) ==== 84+0+65 (secure 0 0 0) 0x7ff1d8057540 con 0x7ff1f00ce860 2024-05-28T14:09:28.534 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:28.534 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:09:28.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:28.534 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-05-28T14:09:28.534 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-05-28T14:09:28.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:28.535 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:09:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.553+0000 7ff1f5400640 1 -- 172.21.15.42:0/1133687473 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1cc0694f0 msgr2=0x7ff1cc06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.553+0000 7ff1f5400640 1 --2- 172.21.15.42:0/1133687473 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1cc0694f0 0x7ff1cc06b9b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00af000 tx=0x7ff1e00a7000 comp rx=0 tx=0).stop 2024-05-28T14:09:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.553+0000 7ff1f5400640 1 -- 172.21.15.42:0/1133687473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f00ce860 msgr2=0x7ff1f00cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.553+0000 7ff1f5400640 1 --2- 172.21.15.42:0/1133687473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f00ce860 0x7ff1f00cc110 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800d830 tx=0x7ff1d800dcf0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.553+0000 7ff1f5400640 1 -- 172.21.15.42:0/1133687473 shutdown_connections 2024-05-28T14:09:28.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.553+0000 7ff1f5400640 1 --2- 172.21.15.42:0/1133687473 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1cc0694f0 0x7ff1cc06b9b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.553+0000 7ff1f5400640 1 --2- 172.21.15.42:0/1133687473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1f00d0900 0x7ff1f014f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.553+0000 7ff1f5400640 1 --2- 172.21.15.42:0/1133687473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f00ce860 0x7ff1f00cc110 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.553+0000 7ff1f5400640 1 --2- 172.21.15.42:0/1133687473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1f006dcf0 0x7ff1f00caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.553+0000 7ff1f5400640 1 -- 172.21.15.42:0/1133687473 >> 172.21.15.42:0/1133687473 conn(0x7ff1f00b4080 msgr2=0x7ff1f0156870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:28.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.553+0000 7ff1f5400640 1 -- 172.21.15.42:0/1133687473 shutdown_connections 2024-05-28T14:09:28.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.553+0000 7ff1f5400640 1 -- 172.21.15.42:0/1133687473 wait complete. 2024-05-28T14:09:28.556 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-28T14:09:28.556 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-28T14:09:28.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.717+0000 7f5e44800640 1 Processor -- start 2024-05-28T14:09:28.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.717+0000 7f5e44800640 1 -- start start 2024-05-28T14:09:28.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.717+0000 7f5e44800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e3c15fcc0 0x7f5e3c0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.717+0000 7f5e44800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5e3c09b5c0 0x7f5e3c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.717+0000 7f5e44800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e3c0d0fd0 con 0x7f5e3c09aa50 2024-05-28T14:09:28.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.717+0000 7f5e44800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5e3c0d1250 con 0x7f5e3c15fcc0 2024-05-28T14:09:28.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.717+0000 7f5e44800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5e3c0b5510 con 0x7f5e3c09b5c0 2024-05-28T14:09:28.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.721+0000 7f5e41e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e3c15fcc0 0x7f5e3c0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.721+0000 7f5e41e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e3c15fcc0 0x7f5e3c0d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56258/0 (socket says 172.21.15.42:56258) 2024-05-28T14:09:28.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.721+0000 7f5e41e00640 1 -- 172.21.15.42:0/3192574466 learned_addr learned my addr 172.21.15.42:0/3192574466 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:28.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.721+0000 7f5e42800640 1 --2- 172.21.15.42:0/3192574466 >> v2:172.21.15.63:3300/0 conn(0x7f5e3c09b5c0 0x7f5e3c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.721+0000 7f5e41e00640 1 -- 172.21.15.42:0/3192574466 >> v2:172.21.15.63:3300/0 conn(0x7f5e3c09b5c0 msgr2=0x7f5e3c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:28.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.721+0000 7f5e41e00640 1 --2- 172.21.15.42:0/3192574466 >> v2:172.21.15.63:3300/0 conn(0x7f5e3c09b5c0 0x7f5e3c0b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.721+0000 7f5e41e00640 1 -- 172.21.15.42:0/3192574466 >> v1:172.21.15.63:6789/0 conn(0x7f5e3c09aa50 legacy=0x7f5e3c09aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.721+0000 7f5e41e00640 1 -- 172.21.15.42:0/3192574466 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c0b57f0 con 0x7f5e3c15fcc0 2024-05-28T14:09:28.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.721+0000 7f5e41e00640 1 --2- 172.21.15.42:0/3192574466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e3c15fcc0 0x7f5e3c0d0a90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c073d20 tx=0x7f5e3001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2ee63efb628166 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:28.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.721+0000 7f5e40a00640 1 -- 172.21.15.42:0/3192574466 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e300b0070 con 0x7f5e3c15fcc0 2024-05-28T14:09:28.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.721+0000 7f5e40a00640 1 -- 172.21.15.42:0/3192574466 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e300ab600 con 0x7f5e3c15fcc0 2024-05-28T14:09:28.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.721+0000 7f5e40a00640 1 -- 172.21.15.42:0/3192574466 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e300ab9d0 con 0x7f5e3c15fcc0 2024-05-28T14:09:28.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 -- 172.21.15.42:0/3192574466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e3c15fcc0 msgr2=0x7f5e3c0d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 --2- 172.21.15.42:0/3192574466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e3c15fcc0 0x7f5e3c0d0a90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c073d20 tx=0x7f5e3001c520 comp rx=0 tx=0).stop 2024-05-28T14:09:28.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 -- 172.21.15.42:0/3192574466 shutdown_connections 2024-05-28T14:09:28.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 --2- 172.21.15.42:0/3192574466 >> v2:172.21.15.63:3300/0 conn(0x7f5e3c09b5c0 0x7f5e3c0b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 --2- 172.21.15.42:0/3192574466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e3c15fcc0 0x7f5e3c0d0a90 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 -- 172.21.15.42:0/3192574466 >> 172.21.15.42:0/3192574466 conn(0x7f5e3c0b3fa0 msgr2=0x7f5e3c0b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:28.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 -- 172.21.15.42:0/3192574466 shutdown_connections 2024-05-28T14:09:28.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 -- 172.21.15.42:0/3192574466 wait complete. 2024-05-28T14:09:28.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 Processor -- start 2024-05-28T14:09:28.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 -- start start 2024-05-28T14:09:28.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c09aa50 0x7f5e3c0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e3c09b5c0 0x7f5e3c06d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e3c15fcc0 0x7f5e3c0c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c0b7150 con 0x7f5e3c09b5c0 2024-05-28T14:09:28.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c0b6fd0 con 0x7f5e3c09aa50 2024-05-28T14:09:28.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e44800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5e3c0b72d0 con 0x7f5e3c15fcc0 2024-05-28T14:09:28.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e41400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e3c09b5c0 0x7f5e3c06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e41e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c09aa50 0x7f5e3c0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e41e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c09aa50 0x7f5e3c0cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:55650/0 (socket says 172.21.15.42:55650) 2024-05-28T14:09:28.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e41e00640 1 -- 172.21.15.42:0/3751117469 learned_addr learned my addr 172.21.15.42:0/3751117469 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:28.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e42800640 1 --2- 172.21.15.42:0/3751117469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e3c15fcc0 0x7f5e3c0c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e41e00640 1 -- 172.21.15.42:0/3751117469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e3c15fcc0 msgr2=0x7f5e3c0c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e41e00640 1 --2- 172.21.15.42:0/3751117469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e3c15fcc0 0x7f5e3c0c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e41e00640 1 -- 172.21.15.42:0/3751117469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e3c09b5c0 msgr2=0x7f5e3c06d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:28.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e41e00640 1 --2- 172.21.15.42:0/3751117469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e3c09b5c0 0x7f5e3c06d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:28.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e41e00640 1 -- 172.21.15.42:0/3751117469 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c0c7c30 con 0x7f5e3c09aa50 2024-05-28T14:09:28.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.725+0000 7f5e41400640 1 --2- 172.21.15.42:0/3751117469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e3c09b5c0 0x7f5e3c06d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:09:28.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.729+0000 7f5e41e00640 1 --2- 172.21.15.42:0/3751117469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c09aa50 0x7f5e3c0cbb30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5e3001ca00 tx=0x7f5e300192c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:28.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.729+0000 7f5e36000640 1 -- 172.21.15.42:0/3751117469 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e300bb070 con 0x7f5e3c09aa50 2024-05-28T14:09:28.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.729+0000 7f5e36000640 1 -- 172.21.15.42:0/3751117469 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e300b4610 con 0x7f5e3c09aa50 2024-05-28T14:09:28.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.729+0000 7f5e36000640 1 -- 172.21.15.42:0/3751117469 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e300b0040 con 0x7f5e3c09aa50 2024-05-28T14:09:28.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.729+0000 7f5e44800640 1 -- 172.21.15.42:0/3751117469 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3c06cd50 con 0x7f5e3c09aa50 2024-05-28T14:09:28.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.729+0000 7f5e44800640 1 -- 172.21.15.42:0/3751117469 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e3c06dd70 con 0x7f5e3c09aa50 2024-05-28T14:09:28.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.729+0000 7f5e44800640 1 -- 172.21.15.42:0/3751117469 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e3c068a90 con 0x7f5e3c09aa50 2024-05-28T14:09:28.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.729+0000 7f5e36000640 1 -- 172.21.15.42:0/3751117469 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f5e300b4a00 con 0x7f5e3c09aa50 2024-05-28T14:09:28.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.729+0000 7f5e36000640 1 --2- 172.21.15.42:0/3751117469 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e20069520 0x7f5e2006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:28.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.729+0000 7f5e36000640 1 -- 172.21.15.42:0/3751117469 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4885+0+0 (secure 0 0 0) 0x7f5e3011ea40 con 0x7f5e3c09aa50 2024-05-28T14:09:28.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.729+0000 7f5e41400640 1 --2- 172.21.15.42:0/3751117469 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e20069520 0x7f5e2006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:28.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.733+0000 7f5e41400640 1 --2- 172.21.15.42:0/3751117469 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e20069520 0x7f5e2006b9e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0d0db0 tx=0x7f5e2c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:28.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.733+0000 7f5e36000640 1 -- 172.21.15.42:0/3751117469 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5e30016110 con 0x7f5e3c09aa50 2024-05-28T14:09:28.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:28.901+0000 7f5e44800640 1 -- 172.21.15.42:0/3751117469 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5e3c0d2290 con 0x7f5e3c09aa50 2024-05-28T14:09:29.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.729+0000 7f5e36000640 1 -- 172.21.15.42:0/3751117469 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) ==== 138+0+0 (secure 0 0 0) 0x7f5e300f00b0 con 0x7f5e3c09aa50 2024-05-28T14:09:29.714 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-05-28T14:09:29.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.733+0000 7f5e44800640 1 -- 172.21.15.42:0/3751117469 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e20069520 msgr2=0x7f5e2006b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:29.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.733+0000 7f5e44800640 1 --2- 172.21.15.42:0/3751117469 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e20069520 0x7f5e2006b9e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0d0db0 tx=0x7f5e2c009040 comp rx=0 tx=0).stop 2024-05-28T14:09:29.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.733+0000 7f5e44800640 1 -- 172.21.15.42:0/3751117469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c09aa50 msgr2=0x7f5e3c0cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:29.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.733+0000 7f5e44800640 1 --2- 172.21.15.42:0/3751117469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c09aa50 0x7f5e3c0cbb30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5e3001ca00 tx=0x7f5e300192c0 comp rx=0 tx=0).stop 2024-05-28T14:09:29.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.733+0000 7f5e44800640 1 -- 172.21.15.42:0/3751117469 shutdown_connections 2024-05-28T14:09:29.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.733+0000 7f5e44800640 1 --2- 172.21.15.42:0/3751117469 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e20069520 0x7f5e2006b9e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:29.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.733+0000 7f5e44800640 1 --2- 172.21.15.42:0/3751117469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e3c15fcc0 0x7f5e3c0c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:29.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.733+0000 7f5e44800640 1 --2- 172.21.15.42:0/3751117469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e3c09b5c0 0x7f5e3c06d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:29.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.733+0000 7f5e44800640 1 --2- 172.21.15.42:0/3751117469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c09aa50 0x7f5e3c0cbb30 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.733+0000 7f5e44800640 1 -- 172.21.15.42:0/3751117469 >> 172.21.15.42:0/3751117469 conn(0x7f5e3c0b3fa0 msgr2=0x7f5e3c0cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.733+0000 7f5e44800640 1 -- 172.21.15.42:0/3751117469 shutdown_connections 2024-05-28T14:09:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.733+0000 7f5e44800640 1 -- 172.21.15.42:0/3751117469 wait complete. 2024-05-28T14:09:29.742 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-28T14:09:29.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d31200640 1 Processor -- start 2024-05-28T14:09:29.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d31200640 1 -- start start 2024-05-28T14:09:29.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d31200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5d2c144f80 0x7f5d2c072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:29.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d31200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d2c06ee20 0x7f5d2c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:29.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d31200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5d2c15b9b0 0x7f5d2c15be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:29.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d31200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5d2c06f7c0 con 0x7f5d2c144f80 2024-05-28T14:09:29.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d31200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5d2c0724d0 con 0x7f5d2c15b9b0 2024-05-28T14:09:29.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d31200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5d2c0afe40 con 0x7f5d2c06ee20 2024-05-28T14:09:29.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d29e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5d2c144f80 0x7f5d2c072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:29.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d29e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5d2c144f80 0x7f5d2c072960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56280/0 (socket says 172.21.15.42:56280) 2024-05-28T14:09:29.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d29e00640 1 -- 172.21.15.42:0/1184753256 learned_addr learned my addr 172.21.15.42:0/1184753256 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:29.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d21400640 1 --2- 172.21.15.42:0/1184753256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d2c06ee20 0x7f5d2c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:29.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d2a800640 1 --2- 172.21.15.42:0/1184753256 >> v2:172.21.15.63:3300/0 conn(0x7f5d2c15b9b0 0x7f5d2c15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:29.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d29e00640 1 -- 172.21.15.42:0/1184753256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d2c06ee20 msgr2=0x7f5d2c06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:29.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d29e00640 1 --2- 172.21.15.42:0/1184753256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d2c06ee20 0x7f5d2c06f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:29.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d29e00640 1 -- 172.21.15.42:0/1184753256 >> v2:172.21.15.63:3300/0 conn(0x7f5d2c15b9b0 msgr2=0x7f5d2c15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:29.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d29e00640 1 --2- 172.21.15.42:0/1184753256 >> v2:172.21.15.63:3300/0 conn(0x7f5d2c15b9b0 0x7f5d2c15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:29.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.913+0000 7f5d29e00640 1 -- 172.21.15.42:0/1184753256 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2c15c380 con 0x7f5d2c144f80 2024-05-28T14:09:29.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.917+0000 7f5d29e00640 1 --2- 172.21.15.42:0/1184753256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d2c144f80 0x7f5d2c072960 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0ce080 tx=0x7f5d1c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10957468c5c5b404 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:29.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.917+0000 7f5d29400640 1 -- 172.21.15.42:0/1184753256 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0b1070 con 0x7f5d2c144f80 2024-05-28T14:09:29.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.917+0000 7f5d29400640 1 -- 172.21.15.42:0/1184753256 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d1c0ac630 con 0x7f5d2c144f80 2024-05-28T14:09:29.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.917+0000 7f5d29400640 1 -- 172.21.15.42:0/1184753256 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0a8050 con 0x7f5d2c144f80 2024-05-28T14:09:29.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.917+0000 7f5d31200640 1 -- 172.21.15.42:0/1184753256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d2c144f80 msgr2=0x7f5d2c072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:29.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.917+0000 7f5d31200640 1 --2- 172.21.15.42:0/1184753256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d2c144f80 0x7f5d2c072960 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0ce080 tx=0x7f5d1c01c520 comp rx=0 tx=0).stop 2024-05-28T14:09:29.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.917+0000 7f5d31200640 1 -- 172.21.15.42:0/1184753256 shutdown_connections 2024-05-28T14:09:29.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.917+0000 7f5d31200640 1 --2- 172.21.15.42:0/1184753256 >> v2:172.21.15.63:3300/0 conn(0x7f5d2c15b9b0 0x7f5d2c15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:29.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.917+0000 7f5d31200640 1 --2- 172.21.15.42:0/1184753256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d2c06ee20 0x7f5d2c06f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:29.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.917+0000 7f5d31200640 1 --2- 172.21.15.42:0/1184753256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d2c144f80 0x7f5d2c072960 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:29.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.917+0000 7f5d31200640 1 -- 172.21.15.42:0/1184753256 >> 172.21.15.42:0/1184753256 conn(0x7f5d2c0b21f0 msgr2=0x7f5d2c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:29.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.917+0000 7f5d31200640 1 -- 172.21.15.42:0/1184753256 shutdown_connections 2024-05-28T14:09:29.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.917+0000 7f5d31200640 1 -- 172.21.15.42:0/1184753256 wait complete. 2024-05-28T14:09:29.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d31200640 1 Processor -- start 2024-05-28T14:09:29.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d31200640 1 -- start start 2024-05-28T14:09:29.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d31200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d2c06ee20 0x7f5d2c13a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:29.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d31200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d2c144f80 0x7f5d2c13f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d31200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d2c15b9b0 0x7f5d2c15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d31200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c072ea0 con 0x7f5d2c06ee20 2024-05-28T14:09:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d31200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c072160 con 0x7f5d2c15b9b0 2024-05-28T14:09:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d31200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5d2c13ecd0 con 0x7f5d2c144f80 2024-05-28T14:09:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d29e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d2c06ee20 0x7f5d2c13a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d29e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d2c06ee20 0x7f5d2c13a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56282/0 (socket says 172.21.15.42:56282) 2024-05-28T14:09:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d29e00640 1 -- 172.21.15.42:0/880116408 learned_addr learned my addr 172.21.15.42:0/880116408 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d2a800640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d2c15b9b0 0x7f5d2c15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d29e00640 1 -- 172.21.15.42:0/880116408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d2c144f80 msgr2=0x7f5d2c13f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d21400640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d2c144f80 0x7f5d2c13f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d29e00640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d2c144f80 0x7f5d2c13f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d29e00640 1 -- 172.21.15.42:0/880116408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d2c15b9b0 msgr2=0x7f5d2c15a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:29.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d29e00640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d2c15b9b0 0x7f5d2c15a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:29.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d29e00640 1 -- 172.21.15.42:0/880116408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2c13de20 con 0x7f5d2c06ee20 2024-05-28T14:09:29.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d2a800640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d2c15b9b0 0x7f5d2c15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:29.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d29e00640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d2c06ee20 0x7f5d2c13a290 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c01cdc0 tx=0x7f5d1c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:29.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d22a00640 1 -- 172.21.15.42:0/880116408 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0bc070 con 0x7f5d2c06ee20 2024-05-28T14:09:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d22a00640 1 -- 172.21.15.42:0/880116408 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d1c0ab5e0 con 0x7f5d2c06ee20 2024-05-28T14:09:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d22a00640 1 -- 172.21.15.42:0/880116408 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0b1070 con 0x7f5d2c06ee20 2024-05-28T14:09:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d31200640 1 -- 172.21.15.42:0/880116408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2c13e0c0 con 0x7f5d2c06ee20 2024-05-28T14:09:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d31200640 1 -- 172.21.15.42:0/880116408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2c13e680 con 0x7f5d2c06ee20 2024-05-28T14:09:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.921+0000 7f5d31200640 1 -- 172.21.15.42:0/880116408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d2c0738b0 con 0x7f5d2c06ee20 2024-05-28T14:09:29.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.925+0000 7f5d22a00640 1 -- 172.21.15.42:0/880116408 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f5d1c0ab9d0 con 0x7f5d2c06ee20 2024-05-28T14:09:29.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.925+0000 7f5d22a00640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5d08069520 0x7f5d0806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:29.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.925+0000 7f5d22a00640 1 -- 172.21.15.42:0/880116408 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5255+0+0 (secure 0 0 0) 0x7f5d1c11f7d0 con 0x7f5d2c06ee20 2024-05-28T14:09:29.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.925+0000 7f5d21400640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5d08069520 0x7f5d0806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:29.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.925+0000 7f5d21400640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5d08069520 0x7f5d0806b9e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0cddf0 tx=0x7f5d14008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:29.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:29.929+0000 7f5d22a00640 1 -- 172.21.15.42:0/880116408 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5d1c0c5020 con 0x7f5d2c06ee20 2024-05-28T14:09:30.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.093+0000 7f5d31200640 1 -- 172.21.15.42:0/880116408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5d2c159fd0 con 0x7f5d2c06ee20 2024-05-28T14:09:30.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.729+0000 7f5d22a00640 1 -- 172.21.15.42:0/880116408 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) ==== 170+0+0 (secure 0 0 0) 0x7f5d2c15c380 con 0x7f5d2c06ee20 2024-05-28T14:09:30.713 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-05-28T14:09:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.733+0000 7f5d31200640 1 -- 172.21.15.42:0/880116408 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5d08069520 msgr2=0x7f5d0806b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.733+0000 7f5d31200640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5d08069520 0x7f5d0806b9e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0cddf0 tx=0x7f5d14008040 comp rx=0 tx=0).stop 2024-05-28T14:09:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.733+0000 7f5d31200640 1 -- 172.21.15.42:0/880116408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d2c06ee20 msgr2=0x7f5d2c13a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.733+0000 7f5d31200640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d2c06ee20 0x7f5d2c13a290 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c01cdc0 tx=0x7f5d1c0192c0 comp rx=0 tx=0).stop 2024-05-28T14:09:30.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.733+0000 7f5d31200640 1 -- 172.21.15.42:0/880116408 shutdown_connections 2024-05-28T14:09:30.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.733+0000 7f5d31200640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5d08069520 0x7f5d0806b9e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:30.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.733+0000 7f5d31200640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d2c15b9b0 0x7f5d2c15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:30.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.733+0000 7f5d31200640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5d2c144f80 0x7f5d2c13f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:30.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.733+0000 7f5d31200640 1 --2- 172.21.15.42:0/880116408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d2c06ee20 0x7f5d2c13a290 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:30.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.733+0000 7f5d31200640 1 -- 172.21.15.42:0/880116408 >> 172.21.15.42:0/880116408 conn(0x7f5d2c0b21f0 msgr2=0x7f5d2c0b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:30.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.737+0000 7f5d31200640 1 -- 172.21.15.42:0/880116408 shutdown_connections 2024-05-28T14:09:30.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.737+0000 7f5d31200640 1 -- 172.21.15.42:0/880116408 wait complete. 2024-05-28T14:09:30.760 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-28T14:09:30.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ed5c00640 1 Processor -- start 2024-05-28T14:09:30.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ed5c00640 1 -- start start 2024-05-28T14:09:30.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ed5c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ed009b160 0x7f4ed006cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:30.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ed5c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4ed009b8e0 0x7f4ed015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:30.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ed5c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ed00d0770 con 0x7f4ed00b0e80 2024-05-28T14:09:30.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ed5c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f4ed00d0a10 con 0x7f4ed009b8e0 2024-05-28T14:09:30.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ed5c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed00d0cf0 con 0x7f4ed009b160 2024-05-28T14:09:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ecea00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f4ed00b0e80 0x7f4ed009aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47290/0 (socket says 172.21.15.42:47290) 2024-05-28T14:09:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ecea00640 1 -- 172.21.15.42:0/2905772704 learned_addr learned my addr 172.21.15.42:0/2905772704 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ecf400640 1 --2- 172.21.15.42:0/2905772704 >> v2:172.21.15.42:3300/0 conn(0x7f4ed009b8e0 0x7f4ed015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ece000640 1 --2- 172.21.15.42:0/2905772704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ed009b160 0x7f4ed006cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ecd600640 1 -- 172.21.15.42:0/2905772704 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3201373597 0 0) 0x7f4ed00d0770 con 0x7f4ed00b0e80 2024-05-28T14:09:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ecd600640 1 -- 172.21.15.42:0/2905772704 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4eb4003530 con 0x7f4ed00b0e80 2024-05-28T14:09:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ecf400640 1 -- 172.21.15.42:0/2905772704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ed009b160 msgr2=0x7f4ed006cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ecf400640 1 --2- 172.21.15.42:0/2905772704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ed009b160 0x7f4ed006cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ecf400640 1 -- 172.21.15.42:0/2905772704 >> v1:172.21.15.42:6789/0 conn(0x7f4ed00b0e80 legacy=0x7f4ed009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:30.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.925+0000 7f4ecf400640 1 -- 172.21.15.42:0/2905772704 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed00d0fd0 con 0x7f4ed009b8e0 2024-05-28T14:09:30.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.929+0000 7f4ecf400640 1 --2- 172.21.15.42:0/2905772704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed009b8e0 0x7f4ed015e110 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009d90 tx=0x7f4ec801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519432d46adac934 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:30.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.929+0000 7f4ecd600640 1 -- 172.21.15.42:0/2905772704 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80b2070 con 0x7f4ed009b8e0 2024-05-28T14:09:30.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.929+0000 7f4ecd600640 1 -- 172.21.15.42:0/2905772704 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec80a5830 con 0x7f4ed009b8e0 2024-05-28T14:09:30.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.929+0000 7f4ecd600640 1 -- 172.21.15.42:0/2905772704 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80a5c00 con 0x7f4ed009b8e0 2024-05-28T14:09:30.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.929+0000 7f4ed5c00640 1 -- 172.21.15.42:0/2905772704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed009b8e0 msgr2=0x7f4ed015e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:30.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.929+0000 7f4ed5c00640 1 --2- 172.21.15.42:0/2905772704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed009b8e0 0x7f4ed015e110 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009d90 tx=0x7f4ec801c970 comp rx=0 tx=0).stop 2024-05-28T14:09:30.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.929+0000 7f4ed5c00640 1 -- 172.21.15.42:0/2905772704 shutdown_connections 2024-05-28T14:09:30.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.929+0000 7f4ed5c00640 1 --2- 172.21.15.42:0/2905772704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed009b8e0 0x7f4ed015e110 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:30.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.929+0000 7f4ed5c00640 1 --2- 172.21.15.42:0/2905772704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ed009b160 0x7f4ed006cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:30.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.929+0000 7f4ed5c00640 1 -- 172.21.15.42:0/2905772704 >> 172.21.15.42:0/2905772704 conn(0x7f4ed00b3f90 msgr2=0x7f4ed00b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:30.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.929+0000 7f4ed5c00640 1 -- 172.21.15.42:0/2905772704 shutdown_connections 2024-05-28T14:09:30.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.929+0000 7f4ed5c00640 1 -- 172.21.15.42:0/2905772704 wait complete. 2024-05-28T14:09:30.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ed5c00640 1 Processor -- start 2024-05-28T14:09:30.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ed5c00640 1 -- start start 2024-05-28T14:09:30.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ed5c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed009b160 0x7f4ed0155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:30.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ed5c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ed009b8e0 0x7f4ed00b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:30.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ed5c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ed00b0e80 0x7f4ed00c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:30.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ed5c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed00af350 con 0x7f4ed009b160 2024-05-28T14:09:30.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ed5c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed00af1d0 con 0x7f4ed009b8e0 2024-05-28T14:09:30.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ed5c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed00af4d0 con 0x7f4ed00b0e80 2024-05-28T14:09:30.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ecea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed009b160 0x7f4ed0155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:30.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ecea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed009b160 0x7f4ed0155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56294/0 (socket says 172.21.15.42:56294) 2024-05-28T14:09:30.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ecea00640 1 -- 172.21.15.42:0/4122727027 learned_addr learned my addr 172.21.15.42:0/4122727027 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:30.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ecf400640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ed00b0e80 0x7f4ed00c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:30.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ece000640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ed009b8e0 0x7f4ed00b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:30.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ecea00640 1 -- 172.21.15.42:0/4122727027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ed00b0e80 msgr2=0x7f4ed00c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:30.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ecea00640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ed00b0e80 0x7f4ed00c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:30.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ecea00640 1 -- 172.21.15.42:0/4122727027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ed009b8e0 msgr2=0x7f4ed00b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:30.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ecea00640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ed009b8e0 0x7f4ed00b7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:30.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ecea00640 1 -- 172.21.15.42:0/4122727027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed00b6950 con 0x7f4ed009b160 2024-05-28T14:09:30.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ece000640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ed009b8e0 0x7f4ed00b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:30.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ecea00640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed009b160 0x7f4ed0155510 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000c3c0 tx=0x7f4ec000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:30.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ec6a00640 1 -- 172.21.15.42:0/4122727027 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0017070 con 0x7f4ed009b160 2024-05-28T14:09:30.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ec6a00640 1 -- 172.21.15.42:0/4122727027 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec0005cf0 con 0x7f4ed009b160 2024-05-28T14:09:30.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ec6a00640 1 -- 172.21.15.42:0/4122727027 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec00072f0 con 0x7f4ed009b160 2024-05-28T14:09:30.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ed5c00640 1 -- 172.21.15.42:0/4122727027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed00b6c30 con 0x7f4ed009b160 2024-05-28T14:09:30.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.933+0000 7f4ed5c00640 1 -- 172.21.15.42:0/4122727027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed00c7b00 con 0x7f4ed009b160 2024-05-28T14:09:30.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.937+0000 7f4ed5c00640 1 -- 172.21.15.42:0/4122727027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed00a0550 con 0x7f4ed009b160 2024-05-28T14:09:30.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.937+0000 7f4ec6a00640 1 -- 172.21.15.42:0/4122727027 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f4ec001a070 con 0x7f4ed009b160 2024-05-28T14:09:30.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.937+0000 7f4ec6a00640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4eb00694f0 0x7f4eb006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:30.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.937+0000 7f4ec6a00640 1 -- 172.21.15.42:0/4122727027 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5637+0+0 (secure 0 0 0) 0x7f4ec00850e0 con 0x7f4ed009b160 2024-05-28T14:09:30.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.937+0000 7f4ece000640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4eb00694f0 0x7f4eb006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:30.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.941+0000 7f4ece000640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4eb00694f0 0x7f4eb006b9b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0161680 tx=0x7f4ebc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:30.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:30.941+0000 7f4ec6a00640 1 -- 172.21.15.42:0/4122727027 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4ec0021020 con 0x7f4ed009b160 2024-05-28T14:09:31.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.145+0000 7f4ed5c00640 1 -- 172.21.15.42:0/4122727027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4ed00d1ca0 con 0x7f4ed009b160 2024-05-28T14:09:31.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.741+0000 7f4ec6a00640 1 -- 172.21.15.42:0/4122727027 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v51) ==== 174+0+0 (secure 0 0 0) 0x7f4ec0056de0 con 0x7f4ed009b160 2024-05-28T14:09:31.728 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 12 and data pool 13 2024-05-28T14:09:31.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.749+0000 7f4ed5c00640 1 -- 172.21.15.42:0/4122727027 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4eb00694f0 msgr2=0x7f4eb006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:31.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.749+0000 7f4ed5c00640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4eb00694f0 0x7f4eb006b9b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0161680 tx=0x7f4ebc008040 comp rx=0 tx=0).stop 2024-05-28T14:09:31.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.749+0000 7f4ed5c00640 1 -- 172.21.15.42:0/4122727027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed009b160 msgr2=0x7f4ed0155510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:31.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.749+0000 7f4ed5c00640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed009b160 0x7f4ed0155510 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000c3c0 tx=0x7f4ec000c880 comp rx=0 tx=0).stop 2024-05-28T14:09:31.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.749+0000 7f4ed5c00640 1 -- 172.21.15.42:0/4122727027 shutdown_connections 2024-05-28T14:09:31.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.749+0000 7f4ed5c00640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4eb00694f0 0x7f4eb006b9b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:31.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.749+0000 7f4ed5c00640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ed00b0e80 0x7f4ed00c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:31.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.749+0000 7f4ed5c00640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ed009b8e0 0x7f4ed00b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:31.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.749+0000 7f4ed5c00640 1 --2- 172.21.15.42:0/4122727027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed009b160 0x7f4ed0155510 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:31.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.749+0000 7f4ed5c00640 1 -- 172.21.15.42:0/4122727027 >> 172.21.15.42:0/4122727027 conn(0x7f4ed00b3f90 msgr2=0x7f4ed006ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:31.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.753+0000 7f4ed5c00640 1 -- 172.21.15.42:0/4122727027 shutdown_connections 2024-05-28T14:09:31.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.753+0000 7f4ed5c00640 1 -- 172.21.15.42:0/4122727027 wait complete. 2024-05-28T14:09:31.772 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:09:31.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc18e400640 1 Processor -- start 2024-05-28T14:09:31.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc18e400640 1 -- start start 2024-05-28T14:09:31.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc18e400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc18815c990 0x7fc1880d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:31.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc18e400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc1880d06a0 0x7fc1880b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:31.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc18e400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1880cd300 con 0x7fc18809bbd0 2024-05-28T14:09:31.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc18e400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc18815cd70 con 0x7fc18815c990 2024-05-28T14:09:31.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc18e400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc1880cd5e0 con 0x7fc1880d06a0 2024-05-28T14:09:31.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc187400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc18815c990 0x7fc1880d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:31.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc187400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc18815c990 0x7fc1880d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56296/0 (socket says 172.21.15.42:56296) 2024-05-28T14:09:31.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc187400640 1 -- 172.21.15.42:0/2603089148 learned_addr learned my addr 172.21.15.42:0/2603089148 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:31.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc187e00640 1 --2- 172.21.15.42:0/2603089148 >> v2:172.21.15.63:3300/0 conn(0x7fc1880d06a0 0x7fc1880b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:31.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc187400640 1 -- 172.21.15.42:0/2603089148 >> v2:172.21.15.63:3300/0 conn(0x7fc1880d06a0 msgr2=0x7fc1880b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:31.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc187400640 1 --2- 172.21.15.42:0/2603089148 >> v2:172.21.15.63:3300/0 conn(0x7fc1880d06a0 0x7fc1880b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:31.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc187400640 1 -- 172.21.15.42:0/2603089148 >> v1:172.21.15.42:6789/0 conn(0x7fc18809bbd0 legacy=0x7fc18809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:31.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.933+0000 7fc187400640 1 -- 172.21.15.42:0/2603089148 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1880cd8c0 con 0x7fc18815c990 2024-05-28T14:09:31.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.937+0000 7fc187400640 1 --2- 172.21.15.42:0/2603089148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc18815c990 0x7fc1880d0160 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fc188073870 tx=0x7fc17801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87405e630917286 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:31.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.937+0000 7fc186a00640 1 -- 172.21.15.42:0/2603089148 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1780aa650 con 0x7fc18815c990 2024-05-28T14:09:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.937+0000 7fc186a00640 1 -- 172.21.15.42:0/2603089148 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1780af070 con 0x7fc18815c990 2024-05-28T14:09:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.937+0000 7fc186a00640 1 -- 172.21.15.42:0/2603089148 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1780aaa10 con 0x7fc18815c990 2024-05-28T14:09:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.937+0000 7fc18e400640 1 -- 172.21.15.42:0/2603089148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc18815c990 msgr2=0x7fc1880d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.937+0000 7fc18e400640 1 --2- 172.21.15.42:0/2603089148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc18815c990 0x7fc1880d0160 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fc188073870 tx=0x7fc17801c4f0 comp rx=0 tx=0).stop 2024-05-28T14:09:31.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.937+0000 7fc18e400640 1 -- 172.21.15.42:0/2603089148 shutdown_connections 2024-05-28T14:09:31.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.937+0000 7fc18e400640 1 --2- 172.21.15.42:0/2603089148 >> v2:172.21.15.63:3300/0 conn(0x7fc1880d06a0 0x7fc1880b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:31.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.937+0000 7fc18e400640 1 --2- 172.21.15.42:0/2603089148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc18815c990 0x7fc1880d0160 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:31.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.937+0000 7fc18e400640 1 -- 172.21.15.42:0/2603089148 >> 172.21.15.42:0/2603089148 conn(0x7fc1880b4080 msgr2=0x7fc1880b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:31.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.937+0000 7fc18e400640 1 -- 172.21.15.42:0/2603089148 shutdown_connections 2024-05-28T14:09:31.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.937+0000 7fc18e400640 1 -- 172.21.15.42:0/2603089148 wait complete. 2024-05-28T14:09:31.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc18e400640 1 Processor -- start 2024-05-28T14:09:31.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc18e400640 1 -- start start 2024-05-28T14:09:31.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc18e400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1880d06a0 0x7fc188155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:31.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc18e400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc18815c990 0x7fc188166f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:31.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc18e400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1880c6a90 0x7fc188165500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:31.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc18e400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc18809b2e0 con 0x7fc1880c6a90 2024-05-28T14:09:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc18e400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc18809b160 con 0x7fc18815c990 2024-05-28T14:09:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc18e400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc1880b6130 con 0x7fc1880d06a0 2024-05-28T14:09:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc187e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1880c6a90 0x7fc188165500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc187e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1880c6a90 0x7fc188165500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56310/0 (socket says 172.21.15.42:56310) 2024-05-28T14:09:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc187e00640 1 -- 172.21.15.42:0/2335812124 learned_addr learned my addr 172.21.15.42:0/2335812124 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc17ea00640 1 --2- 172.21.15.42:0/2335812124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc18815c990 0x7fc188166f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc187400640 1 --2- 172.21.15.42:0/2335812124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1880d06a0 0x7fc188155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc187e00640 1 -- 172.21.15.42:0/2335812124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1880d06a0 msgr2=0x7fc188155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc187e00640 1 --2- 172.21.15.42:0/2335812124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1880d06a0 0x7fc188155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc187e00640 1 -- 172.21.15.42:0/2335812124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc18815c990 msgr2=0x7fc188166f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc187e00640 1 --2- 172.21.15.42:0/2335812124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc18815c990 0x7fc188166f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc187e00640 1 -- 172.21.15.42:0/2335812124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1881674a0 con 0x7fc1880c6a90 2024-05-28T14:09:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc187e00640 1 --2- 172.21.15.42:0/2335812124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1880c6a90 0x7fc188165500 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fc180007e60 tx=0x7fc18000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:31.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc17fe00640 1 -- 172.21.15.42:0/2335812124 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180014620 con 0x7fc1880c6a90 2024-05-28T14:09:31.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc17fe00640 1 -- 172.21.15.42:0/2335812124 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc180019070 con 0x7fc1880c6a90 2024-05-28T14:09:31.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc18e400640 1 -- 172.21.15.42:0/2335812124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc188164780 con 0x7fc1880c6a90 2024-05-28T14:09:31.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc17fe00640 1 -- 172.21.15.42:0/2335812124 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180014a00 con 0x7fc1880c6a90 2024-05-28T14:09:31.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc18e400640 1 -- 172.21.15.42:0/2335812124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc188164d00 con 0x7fc1880c6a90 2024-05-28T14:09:31.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.941+0000 7fc18e400640 1 -- 172.21.15.42:0/2335812124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc188068a90 con 0x7fc1880c6a90 2024-05-28T14:09:31.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.945+0000 7fc17fe00640 1 -- 172.21.15.42:0/2335812124 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fc180014ba0 con 0x7fc1880c6a90 2024-05-28T14:09:31.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.945+0000 7fc17fe00640 1 --2- 172.21.15.42:0/2335812124 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc164069520 0x7fc16406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:31.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.945+0000 7fc17fe00640 1 -- 172.21.15.42:0/2335812124 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5737+0+0 (secure 0 0 0) 0x7fc180086050 con 0x7fc1880c6a90 2024-05-28T14:09:31.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.945+0000 7fc187400640 1 --2- 172.21.15.42:0/2335812124 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc164069520 0x7fc16406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:31.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.945+0000 7fc187400640 1 --2- 172.21.15.42:0/2335812124 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc164069520 0x7fc16406b9e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc1780ae000 tx=0x7fc1780a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:31.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:31.949+0000 7fc17fe00640 1 -- 172.21.15.42:0/2335812124 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc1881674a0 con 0x7fc1880c6a90 2024-05-28T14:09:32.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.121+0000 7fc18e400640 1 -- 172.21.15.42:0/2335812124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc188156a90 con 0x7fc1880c6a90 2024-05-28T14:09:32.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.121+0000 7fc17fe00640 1 -- 172.21.15.42:0/2335812124 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16656 (secure 0 0 0) 0x7fc1800576c0 con 0x7fc1880c6a90 2024-05-28T14:09:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:32.104 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:09:31.146620+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:09:28.913704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-28T14:09:30.108327+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:09:32.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.121+0000 7fc18e400640 1 -- 172.21.15.42:0/2335812124 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc164069520 msgr2=0x7fc16406b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:32.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.121+0000 7fc18e400640 1 --2- 172.21.15.42:0/2335812124 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc164069520 0x7fc16406b9e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc1780ae000 tx=0x7fc1780a7000 comp rx=0 tx=0).stop 2024-05-28T14:09:32.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.121+0000 7fc18e400640 1 -- 172.21.15.42:0/2335812124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1880c6a90 msgr2=0x7fc188165500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:32.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.121+0000 7fc18e400640 1 --2- 172.21.15.42:0/2335812124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1880c6a90 0x7fc188165500 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fc180007e60 tx=0x7fc18000e4b0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.121+0000 7fc18e400640 1 -- 172.21.15.42:0/2335812124 shutdown_connections 2024-05-28T14:09:32.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.121+0000 7fc18e400640 1 --2- 172.21.15.42:0/2335812124 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc164069520 0x7fc16406b9e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.121+0000 7fc18e400640 1 --2- 172.21.15.42:0/2335812124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1880c6a90 0x7fc188165500 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.121+0000 7fc18e400640 1 --2- 172.21.15.42:0/2335812124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc18815c990 0x7fc188166f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.121+0000 7fc18e400640 1 --2- 172.21.15.42:0/2335812124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1880d06a0 0x7fc188155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.121+0000 7fc18e400640 1 -- 172.21.15.42:0/2335812124 >> 172.21.15.42:0/2335812124 conn(0x7fc1880b4080 msgr2=0x7fc1880b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:32.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.121+0000 7fc18e400640 1 -- 172.21.15.42:0/2335812124 shutdown_connections 2024-05-28T14:09:32.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.121+0000 7fc18e400640 1 -- 172.21.15.42:0/2335812124 wait complete. 2024-05-28T14:09:32.127 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:09:32.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.285+0000 7f84a5000640 1 Processor -- start 2024-05-28T14:09:32.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.285+0000 7f84a5000640 1 -- start start 2024-05-28T14:09:32.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.285+0000 7f84a5000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f84a00afa50 0x7f84a006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:32.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.285+0000 7f84a5000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84a0156e50 0x7f84a00d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:32.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.285+0000 7f84a5000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84a015cb90 con 0x7f84a0156a70 2024-05-28T14:09:32.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.285+0000 7f84a5000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f84a015ce50 con 0x7f84a00afa50 2024-05-28T14:09:32.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.285+0000 7f84a5000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f84a015d130 con 0x7f84a0156e50 2024-05-28T14:09:32.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.285+0000 7f8495400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f84a00afa50 0x7f84a006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:32.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.285+0000 7f8495400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f84a00afa50 0x7f84a006dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56324/0 (socket says 172.21.15.42:56324) 2024-05-28T14:09:32.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.285+0000 7f8495400640 1 -- 172.21.15.42:0/592504272 learned_addr learned my addr 172.21.15.42:0/592504272 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:32.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.285+0000 7f849e800640 1 --2- 172.21.15.42:0/592504272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84a0156e50 0x7f84a00d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:32.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.285+0000 7f8495400640 1 -- 172.21.15.42:0/592504272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84a0156e50 msgr2=0x7f84a00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:32.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.285+0000 7f8495400640 1 --2- 172.21.15.42:0/592504272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84a0156e50 0x7f84a00d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.285+0000 7f8495400640 1 -- 172.21.15.42:0/592504272 >> v1:172.21.15.63:6789/0 conn(0x7f84a0156a70 legacy=0x7f84a00af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:32.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.289+0000 7f8495400640 1 -- 172.21.15.42:0/592504272 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a015d410 con 0x7f84a00afa50 2024-05-28T14:09:32.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.289+0000 7f849e800640 1 --2- 172.21.15.42:0/592504272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84a0156e50 0x7f84a00d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:32.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.289+0000 7f8495400640 1 --2- 172.21.15.42:0/592504272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a00afa50 0x7f84a006dd60 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f8488009870 tx=0x7f848801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690dd3bbf15a1c59 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:32.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.289+0000 7f849d400640 1 -- 172.21.15.42:0/592504272 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84880a7070 con 0x7f84a00afa50 2024-05-28T14:09:32.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.289+0000 7f849d400640 1 -- 172.21.15.42:0/592504272 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84880b0070 con 0x7f84a00afa50 2024-05-28T14:09:32.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.289+0000 7f849d400640 1 -- 172.21.15.42:0/592504272 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84880ab890 con 0x7f84a00afa50 2024-05-28T14:09:32.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.289+0000 7f84a5000640 1 -- 172.21.15.42:0/592504272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a00afa50 msgr2=0x7f84a006dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.289+0000 7f84a5000640 1 --2- 172.21.15.42:0/592504272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a00afa50 0x7f84a006dd60 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f8488009870 tx=0x7f848801c560 comp rx=0 tx=0).stop 2024-05-28T14:09:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.289+0000 7f84a5000640 1 -- 172.21.15.42:0/592504272 shutdown_connections 2024-05-28T14:09:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.289+0000 7f84a5000640 1 --2- 172.21.15.42:0/592504272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84a0156e50 0x7f84a00d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.289+0000 7f84a5000640 1 --2- 172.21.15.42:0/592504272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a00afa50 0x7f84a006dd60 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.289+0000 7f84a5000640 1 -- 172.21.15.42:0/592504272 >> 172.21.15.42:0/592504272 conn(0x7f84a006bdf0 msgr2=0x7f84a006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.289+0000 7f84a5000640 1 -- 172.21.15.42:0/592504272 shutdown_connections 2024-05-28T14:09:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.289+0000 7f84a5000640 1 -- 172.21.15.42:0/592504272 wait complete. 2024-05-28T14:09:32.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f84a5000640 1 Processor -- start 2024-05-28T14:09:32.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f84a5000640 1 -- start start 2024-05-28T14:09:32.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f84a5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a00afa50 0x7f84a00ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:32.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f84a5000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84a0156a70 0x7f84a01595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:32.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f84a5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84a0156e50 0x7f84a0159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:32.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f84a5000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0160f50 con 0x7f84a00afa50 2024-05-28T14:09:32.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f84a5000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0160dd0 con 0x7f84a0156e50 2024-05-28T14:09:32.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f84a5000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f84a01610d0 con 0x7f84a0156a70 2024-05-28T14:09:32.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f849de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a00afa50 0x7f84a00ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:32.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f849de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a00afa50 0x7f84a00ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56330/0 (socket says 172.21.15.42:56330) 2024-05-28T14:09:32.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f849de00640 1 -- 172.21.15.42:0/3941240147 learned_addr learned my addr 172.21.15.42:0/3941240147 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:32.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f8495400640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84a0156a70 0x7f84a01595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:32.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f849e800640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84a0156e50 0x7f84a0159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:32.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f849de00640 1 -- 172.21.15.42:0/3941240147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84a0156a70 msgr2=0x7f84a01595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:32.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f849de00640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84a0156a70 0x7f84a01595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f849de00640 1 -- 172.21.15.42:0/3941240147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84a0156e50 msgr2=0x7f84a0159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:32.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f849de00640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84a0156e50 0x7f84a0159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f849de00640 1 -- 172.21.15.42:0/3941240147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a015ddd0 con 0x7f84a00afa50 2024-05-28T14:09:32.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f849e800640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84a0156e50 0x7f84a0159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:32.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f849de00640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a00afa50 0x7f84a00ccf60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f849000c370 tx=0x7f849000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:32.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f8496a00640 1 -- 172.21.15.42:0/3941240147 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84900052d0 con 0x7f84a00afa50 2024-05-28T14:09:32.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f8496a00640 1 -- 172.21.15.42:0/3941240147 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8490005470 con 0x7f84a00afa50 2024-05-28T14:09:32.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f8496a00640 1 -- 172.21.15.42:0/3941240147 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490007260 con 0x7f84a00afa50 2024-05-28T14:09:32.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.293+0000 7f84a5000640 1 -- 172.21.15.42:0/3941240147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a0073a40 con 0x7f84a00afa50 2024-05-28T14:09:32.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.297+0000 7f84a5000640 1 -- 172.21.15.42:0/3941240147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a015e320 con 0x7f84a00afa50 2024-05-28T14:09:32.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.297+0000 7f84a5000640 1 -- 172.21.15.42:0/3941240147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a0068a90 con 0x7f84a00afa50 2024-05-28T14:09:32.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.297+0000 7f8496a00640 1 -- 172.21.15.42:0/3941240147 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f8490007400 con 0x7f84a00afa50 2024-05-28T14:09:32.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.297+0000 7f8496a00640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f847c0694f0 0x7f847c06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:32.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.297+0000 7f8496a00640 1 -- 172.21.15.42:0/3941240147 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5737+0+0 (secure 0 0 0) 0x7f8490084ce0 con 0x7f84a00afa50 2024-05-28T14:09:32.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.297+0000 7f8495400640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f847c0694f0 0x7f847c06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:32.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.301+0000 7f8495400640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f847c0694f0 0x7f847c06b9b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f84880af000 tx=0x7f84880a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:32.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.301+0000 7f8496a00640 1 -- 172.21.15.42:0/3941240147 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8490056a00 con 0x7f84a00afa50 2024-05-28T14:09:32.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.465+0000 7f84a5000640 1 -- 172.21.15.42:0/3941240147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84a00b5160 con 0x7f84a00afa50 2024-05-28T14:09:32.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.469+0000 7f8496a00640 1 -- 172.21.15.42:0/3941240147 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16656 (secure 0 0 0) 0x7f8490056300 con 0x7f84a00afa50 2024-05-28T14:09:32.452 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:32.452 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:09:31.146620+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:09:28.913704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-28T14:09:30.108327+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:09:32.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.469+0000 7f84a5000640 1 -- 172.21.15.42:0/3941240147 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f847c0694f0 msgr2=0x7f847c06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:32.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.469+0000 7f84a5000640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f847c0694f0 0x7f847c06b9b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f84880af000 tx=0x7f84880a7000 comp rx=0 tx=0).stop 2024-05-28T14:09:32.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.469+0000 7f84a5000640 1 -- 172.21.15.42:0/3941240147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a00afa50 msgr2=0x7f84a00ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:32.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.469+0000 7f84a5000640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a00afa50 0x7f84a00ccf60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f849000c370 tx=0x7f849000c830 comp rx=0 tx=0).stop 2024-05-28T14:09:32.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.469+0000 7f84a5000640 1 -- 172.21.15.42:0/3941240147 shutdown_connections 2024-05-28T14:09:32.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.469+0000 7f84a5000640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f847c0694f0 0x7f847c06b9b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.469+0000 7f84a5000640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84a0156e50 0x7f84a0159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.469+0000 7f84a5000640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84a0156a70 0x7f84a01595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.469+0000 7f84a5000640 1 --2- 172.21.15.42:0/3941240147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a00afa50 0x7f84a00ccf60 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.469+0000 7f84a5000640 1 -- 172.21.15.42:0/3941240147 >> 172.21.15.42:0/3941240147 conn(0x7f84a006bdf0 msgr2=0x7f84a0153400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:32.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.469+0000 7f84a5000640 1 -- 172.21.15.42:0/3941240147 shutdown_connections 2024-05-28T14:09:32.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.469+0000 7f84a5000640 1 -- 172.21.15.42:0/3941240147 wait complete. 2024-05-28T14:09:32.474 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-28T14:09:32.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.633+0000 7fc23b400640 1 Processor -- start 2024-05-28T14:09:32.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.633+0000 7fc23b400640 1 -- start start 2024-05-28T14:09:32.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.637+0000 7fc23b400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc234144f80 0x7fc234072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:32.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.637+0000 7fc23b400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc23406ee20 0x7fc23406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:32.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.637+0000 7fc23b400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2340afe40 con 0x7fc23415b9b0 2024-05-28T14:09:32.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.637+0000 7fc23b400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc23406f7c0 con 0x7fc23406ee20 2024-05-28T14:09:32.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.637+0000 7fc23b400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc2340724d0 con 0x7fc234144f80 2024-05-28T14:09:32.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.637+0000 7fc22be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc23406ee20 0x7fc23406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:32.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.637+0000 7fc22be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc23406ee20 0x7fc23406f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56340/0 (socket says 172.21.15.42:56340) 2024-05-28T14:09:32.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.637+0000 7fc22be00640 1 -- 172.21.15.42:0/3165508676 learned_addr learned my addr 172.21.15.42:0/3165508676 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:32.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.637+0000 7fc238a00640 1 --2- 172.21.15.42:0/3165508676 >> v2:172.21.15.63:3300/0 conn(0x7fc234144f80 0x7fc234072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:32.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.637+0000 7fc22be00640 1 -- 172.21.15.42:0/3165508676 >> v2:172.21.15.63:3300/0 conn(0x7fc234144f80 msgr2=0x7fc234072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:32.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.637+0000 7fc22be00640 1 --2- 172.21.15.42:0/3165508676 >> v2:172.21.15.63:3300/0 conn(0x7fc234144f80 0x7fc234072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.637+0000 7fc22be00640 1 -- 172.21.15.42:0/3165508676 >> v1:172.21.15.63:6789/0 conn(0x7fc23415b9b0 legacy=0x7fc23415be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:32.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.637+0000 7fc22be00640 1 -- 172.21.15.42:0/3165508676 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23415c600 con 0x7fc23406ee20 2024-05-28T14:09:32.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.641+0000 7fc22be00640 1 --2- 172.21.15.42:0/3165508676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23406ee20 0x7fc23406f280 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fc234071760 tx=0x7fc21c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53450afae694f991 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:32.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.641+0000 7fc233e00640 1 -- 172.21.15.42:0/3165508676 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0ab610 con 0x7fc23406ee20 2024-05-28T14:09:32.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.641+0000 7fc233e00640 1 -- 172.21.15.42:0/3165508676 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21c0b0070 con 0x7fc23406ee20 2024-05-28T14:09:32.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.641+0000 7fc233e00640 1 -- 172.21.15.42:0/3165508676 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0ab9d0 con 0x7fc23406ee20 2024-05-28T14:09:32.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.641+0000 7fc23b400640 1 -- 172.21.15.42:0/3165508676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23406ee20 msgr2=0x7fc23406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:32.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.641+0000 7fc23b400640 1 --2- 172.21.15.42:0/3165508676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23406ee20 0x7fc23406f280 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fc234071760 tx=0x7fc21c01c520 comp rx=0 tx=0).stop 2024-05-28T14:09:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.641+0000 7fc23b400640 1 -- 172.21.15.42:0/3165508676 shutdown_connections 2024-05-28T14:09:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.641+0000 7fc23b400640 1 --2- 172.21.15.42:0/3165508676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23406ee20 0x7fc23406f280 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.641+0000 7fc23b400640 1 --2- 172.21.15.42:0/3165508676 >> v2:172.21.15.63:3300/0 conn(0x7fc234144f80 0x7fc234072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.641+0000 7fc23b400640 1 -- 172.21.15.42:0/3165508676 >> 172.21.15.42:0/3165508676 conn(0x7fc2340b21f0 msgr2=0x7fc2340b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.641+0000 7fc23b400640 1 -- 172.21.15.42:0/3165508676 shutdown_connections 2024-05-28T14:09:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.641+0000 7fc23b400640 1 -- 172.21.15.42:0/3165508676 wait complete. 2024-05-28T14:09:32.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.641+0000 7fc23b400640 1 Processor -- start 2024-05-28T14:09:32.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.641+0000 7fc23b400640 1 -- start start 2024-05-28T14:09:32.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc23b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23406ee20 0x7fc23415cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:32.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc23b400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc234144f80 0x7fc23415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:32.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc23b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23415b9b0 0x7fc23413fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:32.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc23b400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc234072ea0 con 0x7fc23415b9b0 2024-05-28T14:09:32.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc23b400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2340b4010 con 0x7fc23406ee20 2024-05-28T14:09:32.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc23b400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc234169120 con 0x7fc234144f80 2024-05-28T14:09:32.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc239400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23415b9b0 0x7fc23413fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:32.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc239400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23415b9b0 0x7fc23413fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56344/0 (socket says 172.21.15.42:56344) 2024-05-28T14:09:32.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc239400640 1 -- 172.21.15.42:0/3808212384 learned_addr learned my addr 172.21.15.42:0/3808212384 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:32.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc238a00640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23406ee20 0x7fc23415cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:32.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc22be00640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc234144f80 0x7fc23415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:32.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc239400640 1 -- 172.21.15.42:0/3808212384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc234144f80 msgr2=0x7fc23415a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:32.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc239400640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc234144f80 0x7fc23415a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc239400640 1 -- 172.21.15.42:0/3808212384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23406ee20 msgr2=0x7fc23415cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc239400640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23406ee20 0x7fc23415cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc239400640 1 -- 172.21.15.42:0/3808212384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23415d0b0 con 0x7fc23415b9b0 2024-05-28T14:09:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc238a00640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23406ee20 0x7fc23415cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:09:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc239400640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23415b9b0 0x7fc23413fe40 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00c730 tx=0x7fc22c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:32.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc231600640 1 -- 172.21.15.42:0/3808212384 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c003b80 con 0x7fc23415b9b0 2024-05-28T14:09:32.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc231600640 1 -- 172.21.15.42:0/3808212384 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22c003d20 con 0x7fc23415b9b0 2024-05-28T14:09:32.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc23b400640 1 -- 172.21.15.42:0/3808212384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc234168510 con 0x7fc23415b9b0 2024-05-28T14:09:32.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc231600640 1 -- 172.21.15.42:0/3808212384 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c00aa30 con 0x7fc23415b9b0 2024-05-28T14:09:32.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc23b400640 1 -- 172.21.15.42:0/3808212384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc234168ad0 con 0x7fc23415b9b0 2024-05-28T14:09:32.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.645+0000 7fc23b400640 1 -- 172.21.15.42:0/3808212384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2340738b0 con 0x7fc23415b9b0 2024-05-28T14:09:32.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.649+0000 7fc231600640 1 -- 172.21.15.42:0/3808212384 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fc22c00abd0 con 0x7fc23415b9b0 2024-05-28T14:09:32.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.649+0000 7fc231600640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc210069520 0x7fc21006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:32.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.649+0000 7fc231600640 1 -- 172.21.15.42:0/3808212384 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5737+0+0 (secure 0 0 0) 0x7fc22c085570 con 0x7fc23415b9b0 2024-05-28T14:09:32.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.649+0000 7fc238a00640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc210069520 0x7fc21006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:32.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.649+0000 7fc238a00640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc210069520 0x7fc21006b9e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc224009f50 tx=0x7fc224009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:32.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.653+0000 7fc231600640 1 -- 172.21.15.42:0/3808212384 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc22c056c40 con 0x7fc23415b9b0 2024-05-28T14:09:32.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:32.857+0000 7fc23b400640 1 -- 172.21.15.42:0/3808212384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc23404fdd0 con 0x7fc23415b9b0 2024-05-28T14:09:33.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.753+0000 7fc231600640 1 -- 172.21.15.42:0/3808212384 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v53) ==== 118+0+0 (secure 0 0 0) 0x7fc22c05af40 con 0x7fc23415b9b0 2024-05-28T14:09:33.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.757+0000 7fc23b400640 1 -- 172.21.15.42:0/3808212384 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc210069520 msgr2=0x7fc21006b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:33.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.757+0000 7fc23b400640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc210069520 0x7fc21006b9e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc224009f50 tx=0x7fc224009190 comp rx=0 tx=0).stop 2024-05-28T14:09:33.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.757+0000 7fc23b400640 1 -- 172.21.15.42:0/3808212384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23415b9b0 msgr2=0x7fc23413fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:33.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.757+0000 7fc23b400640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23415b9b0 0x7fc23413fe40 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00c730 tx=0x7fc22c00cbf0 comp rx=0 tx=0).stop 2024-05-28T14:09:33.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.757+0000 7fc23b400640 1 -- 172.21.15.42:0/3808212384 shutdown_connections 2024-05-28T14:09:33.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.757+0000 7fc23b400640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc210069520 0x7fc21006b9e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:33.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.757+0000 7fc23b400640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23415b9b0 0x7fc23413fe40 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:33.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.757+0000 7fc23b400640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc234144f80 0x7fc23415a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:33.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.757+0000 7fc23b400640 1 --2- 172.21.15.42:0/3808212384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23406ee20 0x7fc23415cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:33.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.757+0000 7fc23b400640 1 -- 172.21.15.42:0/3808212384 >> 172.21.15.42:0/3808212384 conn(0x7fc2340b21f0 msgr2=0x7fc2340cd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:33.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.757+0000 7fc23b400640 1 -- 172.21.15.42:0/3808212384 shutdown_connections 2024-05-28T14:09:33.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.757+0000 7fc23b400640 1 -- 172.21.15.42:0/3808212384 wait complete. 2024-05-28T14:09:33.766 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-28T14:09:33.766 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-28T14:09:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8204400640 1 Processor -- start 2024-05-28T14:09:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8204400640 1 -- start start 2024-05-28T14:09:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8204400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f81fc0ce3b0 0x7f81fc0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8204400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc13fd10 0x7f81fc1401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8204400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81fc160230 con 0x7f81fc071de0 2024-05-28T14:09:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8204400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f81fc160510 con 0x7f81fc0ce3b0 2024-05-28T14:09:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8204400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f81fc1607f0 con 0x7f81fc13fd10 2024-05-28T14:09:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8201000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f81fc0ce3b0 0x7f81fc0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8201000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f81fc0ce3b0 0x7f81fc0ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56346/0 (socket says 172.21.15.42:56346) 2024-05-28T14:09:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8201000640 1 -- 172.21.15.42:0/2842026192 learned_addr learned my addr 172.21.15.42:0/2842026192 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8202400640 1 --2- 172.21.15.42:0/2842026192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc13fd10 0x7f81fc1401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8201000640 1 -- 172.21.15.42:0/2842026192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc13fd10 msgr2=0x7f81fc1401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8201000640 1 --2- 172.21.15.42:0/2842026192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc13fd10 0x7f81fc1401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8201000640 1 -- 172.21.15.42:0/2842026192 >> v1:172.21.15.63:6789/0 conn(0x7f81fc071de0 legacy=0x7f81fc0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8201000640 1 -- 172.21.15.42:0/2842026192 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc160ad0 con 0x7f81fc0ce3b0 2024-05-28T14:09:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.933+0000 7f8202400640 1 --2- 172.21.15.42:0/2842026192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc13fd10 0x7f81fc1401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.937+0000 7f8201000640 1 --2- 172.21.15.42:0/2842026192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81fc0ce3b0 0x7f81fc0ce810 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009870 tx=0x7f81e801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8755d9ae0d62a34 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.937+0000 7f81fbe00640 1 -- 172.21.15.42:0/2842026192 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e80b0070 con 0x7f81fc0ce3b0 2024-05-28T14:09:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.937+0000 7f81fbe00640 1 -- 172.21.15.42:0/2842026192 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e80ab600 con 0x7f81fc0ce3b0 2024-05-28T14:09:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.937+0000 7f81fbe00640 1 -- 172.21.15.42:0/2842026192 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e80ab9d0 con 0x7f81fc0ce3b0 2024-05-28T14:09:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.937+0000 7f8204400640 1 -- 172.21.15.42:0/2842026192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81fc0ce3b0 msgr2=0x7f81fc0ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.937+0000 7f8204400640 1 --2- 172.21.15.42:0/2842026192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81fc0ce3b0 0x7f81fc0ce810 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009870 tx=0x7f81e801c440 comp rx=0 tx=0).stop 2024-05-28T14:09:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.937+0000 7f8204400640 1 -- 172.21.15.42:0/2842026192 shutdown_connections 2024-05-28T14:09:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.937+0000 7f8204400640 1 --2- 172.21.15.42:0/2842026192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc13fd10 0x7f81fc1401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.937+0000 7f8204400640 1 --2- 172.21.15.42:0/2842026192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81fc0ce3b0 0x7f81fc0ce810 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.937+0000 7f8204400640 1 -- 172.21.15.42:0/2842026192 >> 172.21.15.42:0/2842026192 conn(0x7f81fc0b21f0 msgr2=0x7f81fc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.937+0000 7f8204400640 1 -- 172.21.15.42:0/2842026192 shutdown_connections 2024-05-28T14:09:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.937+0000 7f8204400640 1 -- 172.21.15.42:0/2842026192 wait complete. 2024-05-28T14:09:33.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8204400640 1 Processor -- start 2024-05-28T14:09:33.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8204400640 1 -- start start 2024-05-28T14:09:33.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8204400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81fc071de0 0x7f81fc0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8204400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc0ce3b0 0x7f81fc152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8204400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81fc13fd10 0x7f81fc153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8204400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc169830 con 0x7f81fc071de0 2024-05-28T14:09:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8204400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc1696b0 con 0x7f81fc13fd10 2024-05-28T14:09:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8204400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f81fc1699b0 con 0x7f81fc0ce3b0 2024-05-28T14:09:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8201a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81fc071de0 0x7f81fc0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8201a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81fc071de0 0x7f81fc0b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56348/0 (socket says 172.21.15.42:56348) 2024-05-28T14:09:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8201a00640 1 -- 172.21.15.42:0/2108126123 learned_addr learned my addr 172.21.15.42:0/2108126123 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8201000640 1 --2- 172.21.15.42:0/2108126123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc0ce3b0 0x7f81fc152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8202400640 1 --2- 172.21.15.42:0/2108126123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81fc13fd10 0x7f81fc153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8201a00640 1 -- 172.21.15.42:0/2108126123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc0ce3b0 msgr2=0x7f81fc152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8201a00640 1 --2- 172.21.15.42:0/2108126123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc0ce3b0 0x7f81fc152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8201a00640 1 -- 172.21.15.42:0/2108126123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81fc13fd10 msgr2=0x7f81fc153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:33.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8201a00640 1 --2- 172.21.15.42:0/2108126123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81fc13fd10 0x7f81fc153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:33.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8201a00640 1 -- 172.21.15.42:0/2108126123 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc155be0 con 0x7f81fc071de0 2024-05-28T14:09:33.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8201a00640 1 --2- 172.21.15.42:0/2108126123 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81fc071de0 0x7f81fc0b4e40 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f81f000c370 tx=0x7f81f000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:33.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f81f9600640 1 -- 172.21.15.42:0/2108126123 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f00052d0 con 0x7f81fc071de0 2024-05-28T14:09:33.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f81f9600640 1 -- 172.21.15.42:0/2108126123 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81f0005470 con 0x7f81fc071de0 2024-05-28T14:09:33.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8204400640 1 -- 172.21.15.42:0/2108126123 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81fc155ec0 con 0x7f81fc071de0 2024-05-28T14:09:33.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f81f9600640 1 -- 172.21.15.42:0/2108126123 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0007260 con 0x7f81fc071de0 2024-05-28T14:09:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8204400640 1 -- 172.21.15.42:0/2108126123 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81fc156480 con 0x7f81fc071de0 2024-05-28T14:09:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.941+0000 7f8204400640 1 -- 172.21.15.42:0/2108126123 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81fc0738b0 con 0x7f81fc071de0 2024-05-28T14:09:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.945+0000 7f81f9600640 1 -- 172.21.15.42:0/2108126123 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f81f0007400 con 0x7f81fc071de0 2024-05-28T14:09:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.945+0000 7f81f9600640 1 --2- 172.21.15.42:0/2108126123 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f81dc0694f0 0x7f81dc06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.945+0000 7f81f9600640 1 -- 172.21.15.42:0/2108126123 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7f81f0084c90 con 0x7f81fc071de0 2024-05-28T14:09:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.945+0000 7f8201000640 1 --2- 172.21.15.42:0/2108126123 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f81dc0694f0 0x7f81dc06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.945+0000 7f8201000640 1 --2- 172.21.15.42:0/2108126123 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f81dc0694f0 0x7f81dc06b9b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f81e80ad000 tx=0x7f81e8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:33.949+0000 7f81f9600640 1 -- 172.21.15.42:0/2108126123 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f81f0056300 con 0x7f81fc071de0 2024-05-28T14:09:34.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.149+0000 7f8204400640 1 -- 172.21.15.42:0/2108126123 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f81fc04fdd0 con 0x7f81fc071de0 2024-05-28T14:09:34.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.757+0000 7f81f9600640 1 -- 172.21.15.42:0/2108126123 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7f81f005a600 con 0x7f81fc071de0 2024-05-28T14:09:34.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.757+0000 7f8204400640 1 -- 172.21.15.42:0/2108126123 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f81dc0694f0 msgr2=0x7f81dc06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:34.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.757+0000 7f8204400640 1 --2- 172.21.15.42:0/2108126123 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f81dc0694f0 0x7f81dc06b9b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f81e80ad000 tx=0x7f81e8008040 comp rx=0 tx=0).stop 2024-05-28T14:09:34.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.757+0000 7f8204400640 1 -- 172.21.15.42:0/2108126123 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81fc071de0 msgr2=0x7f81fc0b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:34.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.757+0000 7f8204400640 1 --2- 172.21.15.42:0/2108126123 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81fc071de0 0x7f81fc0b4e40 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f81f000c370 tx=0x7f81f000c830 comp rx=0 tx=0).stop 2024-05-28T14:09:34.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.757+0000 7f8204400640 1 -- 172.21.15.42:0/2108126123 shutdown_connections 2024-05-28T14:09:34.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.757+0000 7f8204400640 1 --2- 172.21.15.42:0/2108126123 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f81dc0694f0 0x7f81dc06b9b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:34.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.757+0000 7f8204400640 1 --2- 172.21.15.42:0/2108126123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81fc13fd10 0x7f81fc153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:34.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.757+0000 7f8204400640 1 --2- 172.21.15.42:0/2108126123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc0ce3b0 0x7f81fc152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:34.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.757+0000 7f8204400640 1 --2- 172.21.15.42:0/2108126123 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81fc071de0 0x7f81fc0b4e40 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:34.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.757+0000 7f8204400640 1 -- 172.21.15.42:0/2108126123 >> 172.21.15.42:0/2108126123 conn(0x7f81fc0b21f0 msgr2=0x7f81fc1455f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:34.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.761+0000 7f8204400640 1 -- 172.21.15.42:0/2108126123 shutdown_connections 2024-05-28T14:09:34.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.761+0000 7f8204400640 1 -- 172.21.15.42:0/2108126123 wait complete. 2024-05-28T14:09:34.767 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-28T14:09:34.767 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:09:34.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.929+0000 7f3e18e00640 1 Processor -- start 2024-05-28T14:09:34.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.929+0000 7f3e18e00640 1 -- start start 2024-05-28T14:09:34.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.929+0000 7f3e18e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3e1415ea80 0x7f3e140d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:34.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.929+0000 7f3e18e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e140ccd00 con 0x7f3e140d1820 2024-05-28T14:09:34.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.929+0000 7f3e18e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e140ccfe0 con 0x7f3e1409b160 2024-05-28T14:09:34.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.929+0000 7f3e18e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f3e1415ee60 con 0x7f3e1415ea80 2024-05-28T14:09:34.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.929+0000 7f3e11e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3e1415ea80 0x7f3e140d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:34.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.933+0000 7f3e11e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3e1415ea80 0x7f3e140d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56362/0 (socket says 172.21.15.42:56362) 2024-05-28T14:09:34.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.933+0000 7f3e11e00640 1 -- 172.21.15.42:0/1137895322 learned_addr learned my addr 172.21.15.42:0/1137895322 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:34.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.933+0000 7f3e0be00640 1 --1- 172.21.15.42:0/1137895322 >> v1:172.21.15.42:6789/0 conn(0x7f3e140d1820 0x7f3e1409aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47312/0 (socket says 172.21.15.42:47312) 2024-05-28T14:09:34.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.933+0000 7f3e11e00640 1 -- 172.21.15.42:0/1137895322 >> v1:172.21.15.63:6789/0 conn(0x7f3e1409b160 legacy=0x7f3e140cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:34.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.933+0000 7f3e11e00640 1 -- 172.21.15.42:0/1137895322 >> v1:172.21.15.42:6789/0 conn(0x7f3e140d1820 legacy=0x7f3e1409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:34.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.933+0000 7f3e11e00640 1 -- 172.21.15.42:0/1137895322 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e140cd2c0 con 0x7f3e1415ea80 2024-05-28T14:09:34.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.933+0000 7f3e11400640 1 -- 172.21.15.42:0/1137895322 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1554419781 0 0) 0x7f3e140ccd00 con 0x7f3e140d1820 2024-05-28T14:09:34.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.933+0000 7f3e11e00640 1 --2- 172.21.15.42:0/1137895322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1415ea80 0x7f3e140d12e0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009870 tx=0x7f3e0401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6f59d37328f985 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:34.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.933+0000 7f3e11400640 1 -- 172.21.15.42:0/1137895322 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e040b2070 con 0x7f3e1415ea80 2024-05-28T14:09:34.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.933+0000 7f3e11400640 1 -- 172.21.15.42:0/1137895322 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e040a9070 con 0x7f3e1415ea80 2024-05-28T14:09:34.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.933+0000 7f3e11400640 1 -- 172.21.15.42:0/1137895322 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e040ad880 con 0x7f3e1415ea80 2024-05-28T14:09:34.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 -- 172.21.15.42:0/1137895322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1415ea80 msgr2=0x7f3e140d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:34.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 --2- 172.21.15.42:0/1137895322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1415ea80 0x7f3e140d12e0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009870 tx=0x7f3e0401c560 comp rx=0 tx=0).stop 2024-05-28T14:09:34.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 -- 172.21.15.42:0/1137895322 shutdown_connections 2024-05-28T14:09:34.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 --2- 172.21.15.42:0/1137895322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1415ea80 0x7f3e140d12e0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:34.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 -- 172.21.15.42:0/1137895322 >> 172.21.15.42:0/1137895322 conn(0x7f3e140b3f90 msgr2=0x7f3e140b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:34.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 -- 172.21.15.42:0/1137895322 shutdown_connections 2024-05-28T14:09:34.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 -- 172.21.15.42:0/1137895322 wait complete. 2024-05-28T14:09:34.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 Processor -- start 2024-05-28T14:09:34.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 -- start start 2024-05-28T14:09:34.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e1409b160 0x7f3e1415eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:34.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e140d1820 0x7f3e14160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:34.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1415ea80 0x7f3e14152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:34.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1416c6d0 con 0x7f3e1415ea80 2024-05-28T14:09:34.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1416c550 con 0x7f3e1409b160 2024-05-28T14:09:34.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3e1416c850 con 0x7f3e140d1820 2024-05-28T14:09:34.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e12800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1415ea80 0x7f3e14152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:34.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e12800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1415ea80 0x7f3e14152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56378/0 (socket says 172.21.15.42:56378) 2024-05-28T14:09:34.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e12800640 1 -- 172.21.15.42:0/1869167947 learned_addr learned my addr 172.21.15.42:0/1869167947 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:34.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e0be00640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e140d1820 0x7f3e14160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:34.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e11e00640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e1409b160 0x7f3e1415eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:34.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e12800640 1 -- 172.21.15.42:0/1869167947 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e140d1820 msgr2=0x7f3e14160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:34.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e12800640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e140d1820 0x7f3e14160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:34.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e12800640 1 -- 172.21.15.42:0/1869167947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e1409b160 msgr2=0x7f3e1415eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:34.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e12800640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e1409b160 0x7f3e1415eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:34.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e12800640 1 -- 172.21.15.42:0/1869167947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e140cafa0 con 0x7f3e1415ea80 2024-05-28T14:09:34.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e0be00640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e140d1820 0x7f3e14160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:34.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e11e00640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e1409b160 0x7f3e1415eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:34.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e12800640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1415ea80 0x7f3e14152510 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00c730 tx=0x7f3e0c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:34.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e0a000640 1 -- 172.21.15.42:0/1869167947 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c003b80 con 0x7f3e1415ea80 2024-05-28T14:09:34.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e0a000640 1 -- 172.21.15.42:0/1869167947 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0c003d20 con 0x7f3e1415ea80 2024-05-28T14:09:34.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e0a000640 1 -- 172.21.15.42:0/1869167947 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00aa90 con 0x7f3e1415ea80 2024-05-28T14:09:34.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.937+0000 7f3e18e00640 1 -- 172.21.15.42:0/1869167947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e14161870 con 0x7f3e1415ea80 2024-05-28T14:09:34.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.941+0000 7f3e18e00640 1 -- 172.21.15.42:0/1869167947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e14161b10 con 0x7f3e1415ea80 2024-05-28T14:09:34.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.941+0000 7f3e18e00640 1 -- 172.21.15.42:0/1869167947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e14068a90 con 0x7f3e1415ea80 2024-05-28T14:09:34.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.941+0000 7f3e0a000640 1 -- 172.21.15.42:0/1869167947 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f3e0c00ac30 con 0x7f3e1415ea80 2024-05-28T14:09:34.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.941+0000 7f3e0a000640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3df00694f0 0x7f3df006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:34.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.941+0000 7f3e0a000640 1 -- 172.21.15.42:0/1869167947 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7f3e0c085700 con 0x7f3e1415ea80 2024-05-28T14:09:34.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.941+0000 7f3e11e00640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3df00694f0 0x7f3df006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:34.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.941+0000 7f3e11e00640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3df00694f0 0x7f3df006b9b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3e040af000 tx=0x7f3e040a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:34.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:34.945+0000 7f3e0a000640 1 -- 172.21.15.42:0/1869167947 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3e0c057420 con 0x7f3e1415ea80 2024-05-28T14:09:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.145+0000 7f3e18e00640 1 -- 172.21.15.42:0/1869167947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e14070380 con 0x7f3e1415ea80 2024-05-28T14:09:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.145+0000 7f3e0a000640 1 -- 172.21.15.42:0/1869167947 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5182 (secure 0 0 0) 0x7f3e0c056d20 con 0x7f3e1415ea80 2024-05-28T14:09:35.133 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:35.134 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7657,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/402076081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":402076081},{"type":"v1","addr":"172.21.15.63:6835","nonce":402076081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7998,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3779622485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3779622485},{"type":"v1","addr":"172.21.15.42:6835","nonce":3779622485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2961573665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2961573665},{"type":"v1","addr":"172.21.15.42:6837","nonce":2961573665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:09:31.146587+0000","modified":"2024-05-28T14:09:34.152047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":56621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56621":{"gid":56621,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/615628755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":615628755},{"type":"v1","addr":"172.21.15.63:6834","nonce":615628755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":56621,"qdb_cluster":[56621]},"id":6}]} 2024-05-28T14:09:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 54 2024-05-28T14:09:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.149+0000 7f3e18e00640 1 -- 172.21.15.42:0/1869167947 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3df00694f0 msgr2=0x7f3df006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.149+0000 7f3e18e00640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3df00694f0 0x7f3df006b9b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3e040af000 tx=0x7f3e040a7000 comp rx=0 tx=0).stop 2024-05-28T14:09:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.149+0000 7f3e18e00640 1 -- 172.21.15.42:0/1869167947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1415ea80 msgr2=0x7f3e14152510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.149+0000 7f3e18e00640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1415ea80 0x7f3e14152510 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00c730 tx=0x7f3e0c00cbf0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.149+0000 7f3e18e00640 1 -- 172.21.15.42:0/1869167947 shutdown_connections 2024-05-28T14:09:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.149+0000 7f3e18e00640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3df00694f0 0x7f3df006b9b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.149+0000 7f3e18e00640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1415ea80 0x7f3e14152510 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.149+0000 7f3e18e00640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e140d1820 0x7f3e14160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.149+0000 7f3e18e00640 1 --2- 172.21.15.42:0/1869167947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e1409b160 0x7f3e1415eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.149+0000 7f3e18e00640 1 -- 172.21.15.42:0/1869167947 >> 172.21.15.42:0/1869167947 conn(0x7f3e140b3f90 msgr2=0x7f3e140b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.149+0000 7f3e18e00640 1 -- 172.21.15.42:0/1869167947 shutdown_connections 2024-05-28T14:09:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.149+0000 7f3e18e00640 1 -- 172.21.15.42:0/1869167947 wait complete. 2024-05-28T14:09:35.153 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:09:35.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.309+0000 7f6d75800640 1 Processor -- start 2024-05-28T14:09:35.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d75800640 1 -- start start 2024-05-28T14:09:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d75800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d70156a70 0x7f6d7006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d75800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d7015cb90 con 0x7f6d70156e50 2024-05-28T14:09:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d75800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d7015ce50 con 0x7f6d700afa50 2024-05-28T14:09:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d75800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6d7015d130 con 0x7f6d70156a70 2024-05-28T14:09:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d66000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f6d70156e50 0x7f6d700af340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47314/0 (socket says 172.21.15.42:47314) 2024-05-28T14:09:35.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d66000640 1 -- 172.21.15.42:0/2373653073 learned_addr learned my addr 172.21.15.42:0/2373653073 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:35.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6ea00640 1 --2- 172.21.15.42:0/2373653073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d70156a70 0x7f6d7006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:35.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 -- 172.21.15.42:0/2373653073 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 752488729 0 0) 0x7f6d7015cb90 con 0x7f6d70156e50 2024-05-28T14:09:35.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 -- 172.21.15.42:0/2373653073 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d54003530 con 0x7f6d70156e50 2024-05-28T14:09:35.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 -- 172.21.15.42:0/2373653073 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4039091691 0 0) 0x7f6d54003530 con 0x7f6d70156e50 2024-05-28T14:09:35.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 -- 172.21.15.42:0/2373653073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d70156a70 msgr2=0x7f6d7006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:35.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 --2- 172.21.15.42:0/2373653073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d70156a70 0x7f6d7006dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 -- 172.21.15.42:0/2373653073 >> v1:172.21.15.63:6789/0 conn(0x7f6d700afa50 legacy=0x7f6d700d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 -- 172.21.15.42:0/2373653073 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7015d410 con 0x7f6d70156e50 2024-05-28T14:09:35.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 -- 172.21.15.42:0/2373653073 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f6d58001680 con 0x7f6d70156e50 2024-05-28T14:09:35.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 -- 172.21.15.42:0/2373653073 >> v1:172.21.15.42:6789/0 conn(0x7f6d70156e50 legacy=0x7f6d700af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 --2- 172.21.15.42:0/2373653073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540039f0 0x7f6d54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:35.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 --2- 172.21.15.42:0/2373653073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d70156a70 0x7f6d54004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:35.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 --2- 172.21.15.42:0/2373653073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d54004d50 0x7f6d540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:35.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 -- 172.21.15.42:0/2373653073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7015d130 con 0x7f6d540039f0 2024-05-28T14:09:35.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 -- 172.21.15.42:0/2373653073 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5400b9d0 con 0x7f6d54004d50 2024-05-28T14:09:35.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6e000640 1 -- 172.21.15.42:0/2373653073 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5400bbd0 con 0x7f6d70156a70 2024-05-28T14:09:35.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6ea00640 1 --2- 172.21.15.42:0/2373653073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540039f0 0x7f6d54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:35.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d66000640 1 --2- 172.21.15.42:0/2373653073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d70156a70 0x7f6d54004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:35.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6f400640 1 --2- 172.21.15.42:0/2373653073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d54004d50 0x7f6d540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:35.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6ea00640 1 -- 172.21.15.42:0/2373653073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d70156a70 msgr2=0x7f6d54004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:35.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6ea00640 1 --2- 172.21.15.42:0/2373653073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d70156a70 0x7f6d54004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6ea00640 1 -- 172.21.15.42:0/2373653073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d54004d50 msgr2=0x7f6d540051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:35.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6ea00640 1 --2- 172.21.15.42:0/2373653073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d54004d50 0x7f6d540051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.313+0000 7f6d6ea00640 1 -- 172.21.15.42:0/2373653073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d54003530 con 0x7f6d540039f0 2024-05-28T14:09:35.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.317+0000 7f6d6ea00640 1 --2- 172.21.15.42:0/2373653073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540039f0 0x7f6d54003dd0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009fc0 tx=0x7f6d6001cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb0faf2af94ad71 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:35.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.317+0000 7f6d6e000640 1 -- 172.21.15.42:0/2373653073 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d600b2070 con 0x7f6d540039f0 2024-05-28T14:09:35.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.317+0000 7f6d6e000640 1 -- 172.21.15.42:0/2373653073 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d600a5bd0 con 0x7f6d540039f0 2024-05-28T14:09:35.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.317+0000 7f6d6e000640 1 -- 172.21.15.42:0/2373653073 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60009c10 con 0x7f6d540039f0 2024-05-28T14:09:35.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.317+0000 7f6d75800640 1 -- 172.21.15.42:0/2373653073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540039f0 msgr2=0x7f6d54003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.317+0000 7f6d75800640 1 --2- 172.21.15.42:0/2373653073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540039f0 0x7f6d54003dd0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009fc0 tx=0x7f6d6001cbc0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.317+0000 7f6d6f400640 1 -- 172.21.15.42:0/2373653073 reap_dead start 2024-05-28T14:09:35.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.317+0000 7f6d75800640 1 -- 172.21.15.42:0/2373653073 shutdown_connections 2024-05-28T14:09:35.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.317+0000 7f6d75800640 1 -- 172.21.15.42:0/2373653073 >> 172.21.15.42:0/2373653073 conn(0x7f6d7006bdf0 msgr2=0x7f6d7006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:35.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.317+0000 7f6d75800640 1 -- 172.21.15.42:0/2373653073 shutdown_connections 2024-05-28T14:09:35.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.317+0000 7f6d75800640 1 -- 172.21.15.42:0/2373653073 wait complete. 2024-05-28T14:09:35.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d75800640 1 Processor -- start 2024-05-28T14:09:35.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d75800640 1 -- start start 2024-05-28T14:09:35.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d75800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540039f0 0x7f6d7014d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:35.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d75800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d7015f780 0x7f6d70152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:35.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d75800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d7014dc30 0x7f6d701530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:35.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d75800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6d700d0f80 con 0x7f6d540039f0 2024-05-28T14:09:35.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d75800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6d700d0e00 con 0x7f6d7014dc30 2024-05-28T14:09:35.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d75800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6d7006dd60 con 0x7f6d7015f780 2024-05-28T14:09:35.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d6ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540039f0 0x7f6d7014d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:35.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d6ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540039f0 0x7f6d7014d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56394/0 (socket says 172.21.15.42:56394) 2024-05-28T14:09:35.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d6ea00640 1 -- 172.21.15.42:0/3108144364 learned_addr learned my addr 172.21.15.42:0/3108144364 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:35.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d66000640 1 --2- 172.21.15.42:0/3108144364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d7015f780 0x7f6d70152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:35.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d6f400640 1 --2- 172.21.15.42:0/3108144364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d7014dc30 0x7f6d701530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:35.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d6ea00640 1 -- 172.21.15.42:0/3108144364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d7015f780 msgr2=0x7f6d70152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d6ea00640 1 --2- 172.21.15.42:0/3108144364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d7015f780 0x7f6d70152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d6ea00640 1 -- 172.21.15.42:0/3108144364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d7014dc30 msgr2=0x7f6d701530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d6ea00640 1 --2- 172.21.15.42:0/3108144364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d7014dc30 0x7f6d701530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d6ea00640 1 -- 172.21.15.42:0/3108144364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7015fc10 con 0x7f6d540039f0 2024-05-28T14:09:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d6ea00640 1 --2- 172.21.15.42:0/3108144364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540039f0 0x7f6d7014d590 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6d600b0040 tx=0x7f6d60009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d67400640 1 -- 172.21.15.42:0/3108144364 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d600ba070 con 0x7f6d540039f0 2024-05-28T14:09:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d67400640 1 -- 172.21.15.42:0/3108144364 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d600ad9c0 con 0x7f6d540039f0 2024-05-28T14:09:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d67400640 1 -- 172.21.15.42:0/3108144364 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d600b2040 con 0x7f6d540039f0 2024-05-28T14:09:35.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d75800640 1 -- 172.21.15.42:0/3108144364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d700ccea0 con 0x7f6d540039f0 2024-05-28T14:09:35.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d75800640 1 -- 172.21.15.42:0/3108144364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d700cd980 con 0x7f6d540039f0 2024-05-28T14:09:35.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.321+0000 7f6d75800640 1 -- 172.21.15.42:0/3108144364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d70068a90 con 0x7f6d540039f0 2024-05-28T14:09:35.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.325+0000 7f6d67400640 1 -- 172.21.15.42:0/3108144364 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f6d600b6410 con 0x7f6d540039f0 2024-05-28T14:09:35.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.325+0000 7f6d67400640 1 --2- 172.21.15.42:0/3108144364 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d4c0694f0 0x7f6d4c06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:35.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.325+0000 7f6d67400640 1 -- 172.21.15.42:0/3108144364 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7f6d6011ff50 con 0x7f6d540039f0 2024-05-28T14:09:35.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.325+0000 7f6d66000640 1 --2- 172.21.15.42:0/3108144364 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d4c0694f0 0x7f6d4c06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:35.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.325+0000 7f6d66000640 1 --2- 172.21.15.42:0/3108144364 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d4c0694f0 0x7f6d4c06b9b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6d70153820 tx=0x7f6d58009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:35.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.329+0000 7f6d67400640 1 -- 172.21.15.42:0/3108144364 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6d600168d0 con 0x7f6d540039f0 2024-05-28T14:09:35.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.493+0000 7f6d75800640 1 -- 172.21.15.42:0/3108144364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d700b5160 con 0x7f6d540039f0 2024-05-28T14:09:35.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.497+0000 7f6d67400640 1 -- 172.21.15.42:0/3108144364 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16643 (secure 0 0 0) 0x7f6d600f1c10 con 0x7f6d540039f0 2024-05-28T14:09:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:35.480 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":90,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:09:32.743921+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:09:28.913704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-28T14:09:30.108327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:09:35.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.497+0000 7f6d75800640 1 -- 172.21.15.42:0/3108144364 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d4c0694f0 msgr2=0x7f6d4c06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.497+0000 7f6d75800640 1 --2- 172.21.15.42:0/3108144364 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d4c0694f0 0x7f6d4c06b9b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6d70153820 tx=0x7f6d58009040 comp rx=0 tx=0).stop 2024-05-28T14:09:35.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.497+0000 7f6d75800640 1 -- 172.21.15.42:0/3108144364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540039f0 msgr2=0x7f6d7014d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.497+0000 7f6d75800640 1 --2- 172.21.15.42:0/3108144364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540039f0 0x7f6d7014d590 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6d600b0040 tx=0x7f6d60009750 comp rx=0 tx=0).stop 2024-05-28T14:09:35.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.497+0000 7f6d75800640 1 -- 172.21.15.42:0/3108144364 shutdown_connections 2024-05-28T14:09:35.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.497+0000 7f6d75800640 1 --2- 172.21.15.42:0/3108144364 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d4c0694f0 0x7f6d4c06b9b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.497+0000 7f6d75800640 1 --2- 172.21.15.42:0/3108144364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d7014dc30 0x7f6d701530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.497+0000 7f6d75800640 1 --2- 172.21.15.42:0/3108144364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d7015f780 0x7f6d70152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.497+0000 7f6d75800640 1 --2- 172.21.15.42:0/3108144364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540039f0 0x7f6d7014d590 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.497+0000 7f6d75800640 1 -- 172.21.15.42:0/3108144364 >> 172.21.15.42:0/3108144364 conn(0x7f6d7006bdf0 msgr2=0x7f6d7006e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:35.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.497+0000 7f6d75800640 1 -- 172.21.15.42:0/3108144364 shutdown_connections 2024-05-28T14:09:35.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.497+0000 7f6d75800640 1 -- 172.21.15.42:0/3108144364 wait complete. 2024-05-28T14:09:35.502 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-28T14:09:35.502 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:09:35.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.657+0000 7f21bc400640 1 Processor -- start 2024-05-28T14:09:35.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.657+0000 7f21bc400640 1 -- start start 2024-05-28T14:09:35.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.661+0000 7f21bc400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f21b406f660 0x7f21b4073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:35.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.661+0000 7f21bc400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21b415ba40 0x7f21b40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:35.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.661+0000 7f21bc400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21b415f280 con 0x7f21b4072dd0 2024-05-28T14:09:35.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.661+0000 7f21bc400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f21b415f030 con 0x7f21b406f660 2024-05-28T14:09:35.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.661+0000 7f21bc400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f21b415f560 con 0x7f21b415ba40 2024-05-28T14:09:35.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.661+0000 7f21b9a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f21b406f660 0x7f21b4073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:35.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.661+0000 7f21b9a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f21b406f660 0x7f21b4073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56398/0 (socket says 172.21.15.42:56398) 2024-05-28T14:09:35.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.661+0000 7f21b9a00640 1 -- 172.21.15.42:0/3418974664 learned_addr learned my addr 172.21.15.42:0/3418974664 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:35.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.661+0000 7f21b9000640 1 --2- 172.21.15.42:0/3418974664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21b415ba40 0x7f21b40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:35.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.661+0000 7f21b9a00640 1 -- 172.21.15.42:0/3418974664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21b415ba40 msgr2=0x7f21b40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:35.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.661+0000 7f21b9a00640 1 --2- 172.21.15.42:0/3418974664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21b415ba40 0x7f21b40d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.661+0000 7f21b9a00640 1 -- 172.21.15.42:0/3418974664 >> v1:172.21.15.63:6789/0 conn(0x7f21b4072dd0 legacy=0x7f21b40b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.661+0000 7f21b9a00640 1 -- 172.21.15.42:0/3418974664 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b415f840 con 0x7f21b406f660 2024-05-28T14:09:35.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.665+0000 7f21b9a00640 1 --2- 172.21.15.42:0/3418974664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b406f660 0x7f21b4073980 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009870 tx=0x7f21a801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6123dc17edff512d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:35.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.665+0000 7f21b3e00640 1 -- 172.21.15.42:0/3418974664 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a80b0070 con 0x7f21b406f660 2024-05-28T14:09:35.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.665+0000 7f21b3e00640 1 -- 172.21.15.42:0/3418974664 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a80ab600 con 0x7f21b406f660 2024-05-28T14:09:35.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.665+0000 7f21b3e00640 1 -- 172.21.15.42:0/3418974664 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a80ab9d0 con 0x7f21b406f660 2024-05-28T14:09:35.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.665+0000 7f21bc400640 1 -- 172.21.15.42:0/3418974664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b406f660 msgr2=0x7f21b4073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.665+0000 7f21bc400640 1 --2- 172.21.15.42:0/3418974664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b406f660 0x7f21b4073980 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009870 tx=0x7f21a801c410 comp rx=0 tx=0).stop 2024-05-28T14:09:35.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.665+0000 7f21bc400640 1 -- 172.21.15.42:0/3418974664 shutdown_connections 2024-05-28T14:09:35.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.665+0000 7f21bc400640 1 --2- 172.21.15.42:0/3418974664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21b415ba40 0x7f21b40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.665+0000 7f21bc400640 1 --2- 172.21.15.42:0/3418974664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b406f660 0x7f21b4073980 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.665+0000 7f21bc400640 1 -- 172.21.15.42:0/3418974664 >> 172.21.15.42:0/3418974664 conn(0x7f21b406bdf0 msgr2=0x7f21b406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:35.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.665+0000 7f21bc400640 1 -- 172.21.15.42:0/3418974664 shutdown_connections 2024-05-28T14:09:35.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.665+0000 7f21bc400640 1 -- 172.21.15.42:0/3418974664 wait complete. 2024-05-28T14:09:35.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.665+0000 7f21bc400640 1 Processor -- start 2024-05-28T14:09:35.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.665+0000 7f21bc400640 1 -- start start 2024-05-28T14:09:35.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21bc400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21b406f660 0x7f21b41484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:35.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21bc400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21b4072dd0 0x7f21b40adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:35.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21bc400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b4150860 0x7f21b414b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:35.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21bc400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f21b40b3460 con 0x7f21b4150860 2024-05-28T14:09:35.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21bc400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f21b40b32e0 con 0x7f21b406f660 2024-05-28T14:09:35.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21bc400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f21b40b35e0 con 0x7f21b4072dd0 2024-05-28T14:09:35.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21ba400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b4150860 0x7f21b414b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:35.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21ba400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b4150860 0x7f21b414b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56402/0 (socket says 172.21.15.42:56402) 2024-05-28T14:09:35.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21ba400640 1 -- 172.21.15.42:0/3271761988 learned_addr learned my addr 172.21.15.42:0/3271761988 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:35.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21b9000640 1 --2- 172.21.15.42:0/3271761988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21b4072dd0 0x7f21b40adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:35.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21b9a00640 1 --2- 172.21.15.42:0/3271761988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21b406f660 0x7f21b41484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:35.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21ba400640 1 -- 172.21.15.42:0/3271761988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21b4072dd0 msgr2=0x7f21b40adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21ba400640 1 --2- 172.21.15.42:0/3271761988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21b4072dd0 0x7f21b40adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21ba400640 1 -- 172.21.15.42:0/3271761988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21b406f660 msgr2=0x7f21b41484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21ba400640 1 --2- 172.21.15.42:0/3271761988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21b406f660 0x7f21b41484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21ba400640 1 -- 172.21.15.42:0/3271761988 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b4150cf0 con 0x7f21b4150860 2024-05-28T14:09:35.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21ba400640 1 --2- 172.21.15.42:0/3271761988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b4150860 0x7f21b414b2e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00c770 tx=0x7f21ac00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:35.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21b1600640 1 -- 172.21.15.42:0/3271761988 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac003b80 con 0x7f21b4150860 2024-05-28T14:09:35.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21b1600640 1 -- 172.21.15.42:0/3271761988 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ac003d20 con 0x7f21b4150860 2024-05-28T14:09:35.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21bc400640 1 -- 172.21.15.42:0/3271761988 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b4150fb0 con 0x7f21b4150860 2024-05-28T14:09:35.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21b1600640 1 -- 172.21.15.42:0/3271761988 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac00aa30 con 0x7f21b4150860 2024-05-28T14:09:35.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21bc400640 1 -- 172.21.15.42:0/3271761988 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b4151570 con 0x7f21b4150860 2024-05-28T14:09:35.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21bc400640 1 -- 172.21.15.42:0/3271761988 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b4068a90 con 0x7f21b4150860 2024-05-28T14:09:35.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.669+0000 7f21b1600640 1 -- 172.21.15.42:0/3271761988 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f21ac00abd0 con 0x7f21b4150860 2024-05-28T14:09:35.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.673+0000 7f21b1600640 1 --2- 172.21.15.42:0/3271761988 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21940694f0 0x7f219406b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:35.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.673+0000 7f21b1600640 1 -- 172.21.15.42:0/3271761988 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7f21ac0855b0 con 0x7f21b4150860 2024-05-28T14:09:35.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.673+0000 7f21b9a00640 1 --2- 172.21.15.42:0/3271761988 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21940694f0 0x7f219406b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:35.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.673+0000 7f21b9a00640 1 --2- 172.21.15.42:0/3271761988 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21940694f0 0x7f219406b9b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f21a80ad000 tx=0x7f21a80a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:35.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.677+0000 7f21b1600640 1 -- 172.21.15.42:0/3271761988 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f21ac056c80 con 0x7f21b4150860 2024-05-28T14:09:35.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.841+0000 7f21bc400640 1 -- 172.21.15.42:0/3271761988 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21b4072bb0 con 0x7f21940694f0 2024-05-28T14:09:35.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.849+0000 7f21b1600640 1 -- 172.21.15.42:0/3271761988 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250670 (secure 0 0 0) 0x7f21b4072bb0 con 0x7f21940694f0 2024-05-28T14:09:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:35.837 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:09:35.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.853+0000 7f21bc400640 1 -- 172.21.15.42:0/3271761988 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21940694f0 msgr2=0x7f219406b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.853+0000 7f21bc400640 1 --2- 172.21.15.42:0/3271761988 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21940694f0 0x7f219406b9b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f21a80ad000 tx=0x7f21a80a6000 comp rx=0 tx=0).stop 2024-05-28T14:09:35.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.853+0000 7f21bc400640 1 -- 172.21.15.42:0/3271761988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b4150860 msgr2=0x7f21b414b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:35.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.853+0000 7f21bc400640 1 --2- 172.21.15.42:0/3271761988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b4150860 0x7f21b414b2e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00c770 tx=0x7f21ac00cc30 comp rx=0 tx=0).stop 2024-05-28T14:09:35.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.853+0000 7f21bc400640 1 -- 172.21.15.42:0/3271761988 shutdown_connections 2024-05-28T14:09:35.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.853+0000 7f21bc400640 1 --2- 172.21.15.42:0/3271761988 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f21940694f0 0x7f219406b9b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.853+0000 7f21bc400640 1 --2- 172.21.15.42:0/3271761988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21b4150860 0x7f21b414b2e0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.853+0000 7f21bc400640 1 --2- 172.21.15.42:0/3271761988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21b4072dd0 0x7f21b40adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.853+0000 7f21bc400640 1 --2- 172.21.15.42:0/3271761988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21b406f660 0x7f21b41484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:35.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.853+0000 7f21bc400640 1 -- 172.21.15.42:0/3271761988 >> 172.21.15.42:0/3271761988 conn(0x7f21b406bdf0 msgr2=0x7f21b414ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:35.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.853+0000 7f21bc400640 1 -- 172.21.15.42:0/3271761988 shutdown_connections 2024-05-28T14:09:35.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:35.853+0000 7f21bc400640 1 -- 172.21.15.42:0/3271761988 wait complete. 2024-05-28T14:09:35.856 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-05-28T14:09:35.112730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":852952,"kb_used_data":9812,"kb_used_omap":12,"kb_used_meta":842995,"kb_avail":754121768,"statfs":{"total":773094113280,"available":772220690432,"internally_reserved":0,"allocated":10047488,"data_stored":4985879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":863227442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.387621"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151546+0000","last_change":"2024-05-28T14:09:31.763443+0000","last_active":"2024-05-28T14:09:33.151546+0000","last_peered":"2024-05-28T14:09:33.151546+0000","last_clean":"2024-05-28T14:09:33.151546+0000","last_became_active":"2024-05-28T14:09:31.763080+0000","last_became_peered":"2024-05-28T14:09:31.763080+0000","last_unstale":"2024-05-28T14:09:33.151546+0000","last_undegraded":"2024-05-28T14:09:33.151546+0000","last_fullsized":"2024-05-28T14:09:33.151546+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762829+0000","last_change":"2024-05-28T14:09:31.768406+0000","last_active":"2024-05-28T14:09:32.762829+0000","last_peered":"2024-05-28T14:09:32.762829+0000","last_clean":"2024-05-28T14:09:32.762829+0000","last_became_active":"2024-05-28T14:09:31.768150+0000","last_became_peered":"2024-05-28T14:09:31.768150+0000","last_unstale":"2024-05-28T14:09:32.762829+0000","last_undegraded":"2024-05-28T14:09:32.762829+0000","last_fullsized":"2024-05-28T14:09:32.762829+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:38.614174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150788+0000","last_change":"2024-05-28T14:09:31.770809+0000","last_active":"2024-05-28T14:09:33.150788+0000","last_peered":"2024-05-28T14:09:33.150788+0000","last_clean":"2024-05-28T14:09:33.150788+0000","last_became_active":"2024-05-28T14:09:31.770547+0000","last_became_peered":"2024-05-28T14:09:31.770547+0000","last_unstale":"2024-05-28T14:09:33.150788+0000","last_undegraded":"2024-05-28T14:09:33.150788+0000","last_fullsized":"2024-05-28T14:09:33.150788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:34:22.994257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751824+0000","last_change":"2024-05-28T14:09:31.756701+0000","last_active":"2024-05-28T14:09:32.751824+0000","last_peered":"2024-05-28T14:09:32.751824+0000","last_clean":"2024-05-28T14:09:32.751824+0000","last_became_active":"2024-05-28T14:09:31.756425+0000","last_became_peered":"2024-05-28T14:09:31.756425+0000","last_unstale":"2024-05-28T14:09:32.751824+0000","last_undegraded":"2024-05-28T14:09:32.751824+0000","last_fullsized":"2024-05-28T14:09:32.751824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:43.749597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750865+0000","last_change":"2024-05-28T14:09:31.756691+0000","last_active":"2024-05-28T14:09:32.750865+0000","last_peered":"2024-05-28T14:09:32.750865+0000","last_clean":"2024-05-28T14:09:32.750865+0000","last_became_active":"2024-05-28T14:09:31.756434+0000","last_became_peered":"2024-05-28T14:09:31.756434+0000","last_unstale":"2024-05-28T14:09:32.750865+0000","last_undegraded":"2024-05-28T14:09:32.750865+0000","last_fullsized":"2024-05-28T14:09:32.750865+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751086+0000","last_change":"2024-05-28T14:09:31.761009+0000","last_active":"2024-05-28T14:09:32.751086+0000","last_peered":"2024-05-28T14:09:32.751086+0000","last_clean":"2024-05-28T14:09:32.751086+0000","last_became_active":"2024-05-28T14:09:31.760693+0000","last_became_peered":"2024-05-28T14:09:31.760693+0000","last_unstale":"2024-05-28T14:09:32.751086+0000","last_undegraded":"2024-05-28T14:09:32.751086+0000","last_fullsized":"2024-05-28T14:09:32.751086+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751130+0000","last_change":"2024-05-28T14:09:31.748781+0000","last_active":"2024-05-28T14:09:32.751130+0000","last_peered":"2024-05-28T14:09:32.751130+0000","last_clean":"2024-05-28T14:09:32.751130+0000","last_became_active":"2024-05-28T14:09:31.748446+0000","last_became_peered":"2024-05-28T14:09:31.748446+0000","last_unstale":"2024-05-28T14:09:32.751130+0000","last_undegraded":"2024-05-28T14:09:32.751130+0000","last_fullsized":"2024-05-28T14:09:32.751130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151416+0000","last_change":"2024-05-28T14:09:31.755098+0000","last_active":"2024-05-28T14:09:33.151416+0000","last_peered":"2024-05-28T14:09:33.151416+0000","last_clean":"2024-05-28T14:09:33.151416+0000","last_became_active":"2024-05-28T14:09:31.754790+0000","last_became_peered":"2024-05-28T14:09:31.754790+0000","last_unstale":"2024-05-28T14:09:33.151416+0000","last_undegraded":"2024-05-28T14:09:33.151416+0000","last_fullsized":"2024-05-28T14:09:33.151416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762851+0000","last_change":"2024-05-28T14:09:31.765027+0000","last_active":"2024-05-28T14:09:32.762851+0000","last_peered":"2024-05-28T14:09:32.762851+0000","last_clean":"2024-05-28T14:09:32.762851+0000","last_became_active":"2024-05-28T14:09:31.764660+0000","last_became_peered":"2024-05-28T14:09:31.764660+0000","last_unstale":"2024-05-28T14:09:32.762851+0000","last_undegraded":"2024-05-28T14:09:32.762851+0000","last_fullsized":"2024-05-28T14:09:32.762851+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:59.297622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150769+0000","last_change":"2024-05-28T14:09:31.756022+0000","last_active":"2024-05-28T14:09:33.150769+0000","last_peered":"2024-05-28T14:09:33.150769+0000","last_clean":"2024-05-28T14:09:33.150769+0000","last_became_active":"2024-05-28T14:09:31.755772+0000","last_became_peered":"2024-05-28T14:09:31.755772+0000","last_unstale":"2024-05-28T14:09:33.150769+0000","last_undegraded":"2024-05-28T14:09:33.150769+0000","last_fullsized":"2024-05-28T14:09:33.150769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:35.172304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150463+0000","last_change":"2024-05-28T14:09:31.764389+0000","last_active":"2024-05-28T14:09:33.150463+0000","last_peered":"2024-05-28T14:09:33.150463+0000","last_clean":"2024-05-28T14:09:33.150463+0000","last_became_active":"2024-05-28T14:09:31.764102+0000","last_became_peered":"2024-05-28T14:09:31.764102+0000","last_unstale":"2024-05-28T14:09:33.150463+0000","last_undegraded":"2024-05-28T14:09:33.150463+0000","last_fullsized":"2024-05-28T14:09:33.150463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:37:41.765233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747849+0000","last_change":"2024-05-28T14:09:31.766893+0000","last_active":"2024-05-28T14:09:32.747849+0000","last_peered":"2024-05-28T14:09:32.747849+0000","last_clean":"2024-05-28T14:09:32.747849+0000","last_became_active":"2024-05-28T14:09:31.766457+0000","last_became_peered":"2024-05-28T14:09:31.766457+0000","last_unstale":"2024-05-28T14:09:32.747849+0000","last_undegraded":"2024-05-28T14:09:32.747849+0000","last_fullsized":"2024-05-28T14:09:32.747849+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:34:22.994257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751531+0000","last_change":"2024-05-28T14:09:31.753479+0000","last_active":"2024-05-28T14:09:32.751531+0000","last_peered":"2024-05-28T14:09:32.751531+0000","last_clean":"2024-05-28T14:09:32.751531+0000","last_became_active":"2024-05-28T14:09:31.753094+0000","last_became_peered":"2024-05-28T14:09:31.753094+0000","last_unstale":"2024-05-28T14:09:32.751531+0000","last_undegraded":"2024-05-28T14:09:32.751531+0000","last_fullsized":"2024-05-28T14:09:32.751531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:49:42.278616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752189+0000","last_change":"2024-05-28T14:09:31.759451+0000","last_active":"2024-05-28T14:09:32.752189+0000","last_peered":"2024-05-28T14:09:32.752189+0000","last_clean":"2024-05-28T14:09:32.752189+0000","last_became_active":"2024-05-28T14:09:31.759197+0000","last_became_peered":"2024-05-28T14:09:31.759197+0000","last_unstale":"2024-05-28T14:09:32.752189+0000","last_undegraded":"2024-05-28T14:09:32.752189+0000","last_fullsized":"2024-05-28T14:09:32.752189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765154+0000","last_change":"2024-05-28T14:09:31.753754+0000","last_active":"2024-05-28T14:09:32.765154+0000","last_peered":"2024-05-28T14:09:32.765154+0000","last_clean":"2024-05-28T14:09:32.765154+0000","last_became_active":"2024-05-28T14:09:31.753211+0000","last_became_peered":"2024-05-28T14:09:31.753211+0000","last_unstale":"2024-05-28T14:09:32.765154+0000","last_undegraded":"2024-05-28T14:09:32.765154+0000","last_fullsized":"2024-05-28T14:09:32.765154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762453+0000","last_change":"2024-05-28T14:09:31.755223+0000","last_active":"2024-05-28T14:09:32.762453+0000","last_peered":"2024-05-28T14:09:32.762453+0000","last_clean":"2024-05-28T14:09:32.762453+0000","last_became_active":"2024-05-28T14:09:31.754880+0000","last_became_peered":"2024-05-28T14:09:31.754880+0000","last_unstale":"2024-05-28T14:09:32.762453+0000","last_undegraded":"2024-05-28T14:09:32.762453+0000","last_fullsized":"2024-05-28T14:09:32.762453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:25.450855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150682+0000","last_change":"2024-05-28T14:09:31.752700+0000","last_active":"2024-05-28T14:09:33.150682+0000","last_peered":"2024-05-28T14:09:33.150682+0000","last_clean":"2024-05-28T14:09:33.150682+0000","last_became_active":"2024-05-28T14:09:31.752367+0000","last_became_peered":"2024-05-28T14:09:31.752367+0000","last_unstale":"2024-05-28T14:09:33.150682+0000","last_undegraded":"2024-05-28T14:09:33.150682+0000","last_fullsized":"2024-05-28T14:09:33.150682+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:43.749597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762838+0000","last_change":"2024-05-28T14:09:31.756171+0000","last_active":"2024-05-28T14:09:32.762838+0000","last_peered":"2024-05-28T14:09:32.762838+0000","last_clean":"2024-05-28T14:09:32.762838+0000","last_became_active":"2024-05-28T14:09:31.754405+0000","last_became_peered":"2024-05-28T14:09:31.754405+0000","last_unstale":"2024-05-28T14:09:32.762838+0000","last_undegraded":"2024-05-28T14:09:32.762838+0000","last_fullsized":"2024-05-28T14:09:32.762838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:19.325065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150852+0000","last_change":"2024-05-28T14:09:31.760741+0000","last_active":"2024-05-28T14:09:33.150852+0000","last_peered":"2024-05-28T14:09:33.150852+0000","last_clean":"2024-05-28T14:09:33.150852+0000","last_became_active":"2024-05-28T14:09:31.760404+0000","last_became_peered":"2024-05-28T14:09:31.760404+0000","last_unstale":"2024-05-28T14:09:33.150852+0000","last_undegraded":"2024-05-28T14:09:33.150852+0000","last_fullsized":"2024-05-28T14:09:33.150852+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:24.707344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751996+0000","last_change":"2024-05-28T14:09:31.754702+0000","last_active":"2024-05-28T14:09:32.751996+0000","last_peered":"2024-05-28T14:09:32.751996+0000","last_clean":"2024-05-28T14:09:32.751996+0000","last_became_active":"2024-05-28T14:09:31.754215+0000","last_became_peered":"2024-05-28T14:09:31.754215+0000","last_unstale":"2024-05-28T14:09:32.751996+0000","last_undegraded":"2024-05-28T14:09:32.751996+0000","last_fullsized":"2024-05-28T14:09:32.751996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:36:07.311954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151011+0000","last_change":"2024-05-28T14:09:31.765289+0000","last_active":"2024-05-28T14:09:33.151011+0000","last_peered":"2024-05-28T14:09:33.151011+0000","last_clean":"2024-05-28T14:09:33.151011+0000","last_became_active":"2024-05-28T14:09:31.763980+0000","last_became_peered":"2024-05-28T14:09:31.763980+0000","last_unstale":"2024-05-28T14:09:33.151011+0000","last_undegraded":"2024-05-28T14:09:33.151011+0000","last_fullsized":"2024-05-28T14:09:33.151011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762598+0000","last_change":"2024-05-28T14:09:31.758017+0000","last_active":"2024-05-28T14:09:32.762598+0000","last_peered":"2024-05-28T14:09:32.762598+0000","last_clean":"2024-05-28T14:09:32.762598+0000","last_became_active":"2024-05-28T14:09:31.757674+0000","last_became_peered":"2024-05-28T14:09:31.757674+0000","last_unstale":"2024-05-28T14:09:32.762598+0000","last_undegraded":"2024-05-28T14:09:32.762598+0000","last_fullsized":"2024-05-28T14:09:32.762598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:46.760174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762477+0000","last_change":"2024-05-28T14:09:31.762403+0000","last_active":"2024-05-28T14:09:32.762477+0000","last_peered":"2024-05-28T14:09:32.762477+0000","last_clean":"2024-05-28T14:09:32.762477+0000","last_became_active":"2024-05-28T14:09:31.762048+0000","last_became_peered":"2024-05-28T14:09:31.762048+0000","last_unstale":"2024-05-28T14:09:32.762477+0000","last_undegraded":"2024-05-28T14:09:32.762477+0000","last_fullsized":"2024-05-28T14:09:32.762477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:56.072759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751599+0000","last_change":"2024-05-28T14:09:31.747630+0000","last_active":"2024-05-28T14:09:32.751599+0000","last_peered":"2024-05-28T14:09:32.751599+0000","last_clean":"2024-05-28T14:09:32.751599+0000","last_became_active":"2024-05-28T14:09:31.747183+0000","last_became_peered":"2024-05-28T14:09:31.747183+0000","last_unstale":"2024-05-28T14:09:32.751599+0000","last_undegraded":"2024-05-28T14:09:32.751599+0000","last_fullsized":"2024-05-28T14:09:32.751599+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:49:44.321180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151402+0000","last_change":"2024-05-28T14:09:31.768533+0000","last_active":"2024-05-28T14:09:33.151402+0000","last_peered":"2024-05-28T14:09:33.151402+0000","last_clean":"2024-05-28T14:09:33.151402+0000","last_became_active":"2024-05-28T14:09:31.768237+0000","last_became_peered":"2024-05-28T14:09:31.768237+0000","last_unstale":"2024-05-28T14:09:33.151402+0000","last_undegraded":"2024-05-28T14:09:33.151402+0000","last_fullsized":"2024-05-28T14:09:33.151402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752209+0000","last_change":"2024-05-28T14:09:31.752686+0000","last_active":"2024-05-28T14:09:32.752209+0000","last_peered":"2024-05-28T14:09:32.752209+0000","last_clean":"2024-05-28T14:09:32.752209+0000","last_became_active":"2024-05-28T14:09:31.752411+0000","last_became_peered":"2024-05-28T14:09:31.752411+0000","last_unstale":"2024-05-28T14:09:32.752209+0000","last_undegraded":"2024-05-28T14:09:32.752209+0000","last_fullsized":"2024-05-28T14:09:32.752209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151595+0000","last_change":"2024-05-28T14:09:30.759880+0000","last_active":"2024-05-28T14:09:33.151595+0000","last_peered":"2024-05-28T14:09:33.151595+0000","last_clean":"2024-05-28T14:09:33.151595+0000","last_became_active":"2024-05-28T14:09:30.759575+0000","last_became_peered":"2024-05-28T14:09:30.759575+0000","last_unstale":"2024-05-28T14:09:33.151595+0000","last_undegraded":"2024-05-28T14:09:33.151595+0000","last_fullsized":"2024-05-28T14:09:33.151595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:55.065487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151493+0000","last_change":"2024-05-28T14:09:31.752593+0000","last_active":"2024-05-28T14:09:33.151493+0000","last_peered":"2024-05-28T14:09:33.151493+0000","last_clean":"2024-05-28T14:09:33.151493+0000","last_became_active":"2024-05-28T14:09:31.752332+0000","last_became_peered":"2024-05-28T14:09:31.752332+0000","last_unstale":"2024-05-28T14:09:33.151493+0000","last_undegraded":"2024-05-28T14:09:33.151493+0000","last_fullsized":"2024-05-28T14:09:33.151493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752466+0000","last_change":"2024-05-28T14:09:30.755821+0000","last_active":"2024-05-28T14:09:32.752466+0000","last_peered":"2024-05-28T14:09:32.752466+0000","last_clean":"2024-05-28T14:09:32.752466+0000","last_became_active":"2024-05-28T14:09:30.755452+0000","last_became_peered":"2024-05-28T14:09:30.755452+0000","last_unstale":"2024-05-28T14:09:32.752466+0000","last_undegraded":"2024-05-28T14:09:32.752466+0000","last_fullsized":"2024-05-28T14:09:32.752466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:18.317793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151220+0000","last_change":"2024-05-28T14:09:31.765522+0000","last_active":"2024-05-28T14:09:33.151220+0000","last_peered":"2024-05-28T14:09:33.151220+0000","last_clean":"2024-05-28T14:09:33.151220+0000","last_became_active":"2024-05-28T14:09:31.764349+0000","last_became_peered":"2024-05-28T14:09:31.764349+0000","last_unstale":"2024-05-28T14:09:33.151220+0000","last_undegraded":"2024-05-28T14:09:33.151220+0000","last_fullsized":"2024-05-28T14:09:33.151220+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765049+0000","last_change":"2024-05-28T14:09:30.752545+0000","last_active":"2024-05-28T14:09:32.765049+0000","last_peered":"2024-05-28T14:09:32.765049+0000","last_clean":"2024-05-28T14:09:32.765049+0000","last_became_active":"2024-05-28T14:09:30.752219+0000","last_became_peered":"2024-05-28T14:09:30.752219+0000","last_unstale":"2024-05-28T14:09:32.765049+0000","last_undegraded":"2024-05-28T14:09:32.765049+0000","last_fullsized":"2024-05-28T14:09:32.765049+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:49:43.313909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151148+0000","last_change":"2024-05-28T14:09:31.765767+0000","last_active":"2024-05-28T14:09:33.151148+0000","last_peered":"2024-05-28T14:09:33.151148+0000","last_clean":"2024-05-28T14:09:33.151148+0000","last_became_active":"2024-05-28T14:09:31.765407+0000","last_became_peered":"2024-05-28T14:09:31.765407+0000","last_unstale":"2024-05-28T14:09:33.151148+0000","last_undegraded":"2024-05-28T14:09:33.151148+0000","last_fullsized":"2024-05-28T14:09:33.151148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:27:09.475191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764918+0000","last_change":"2024-05-28T14:09:30.754683+0000","last_active":"2024-05-28T14:09:32.764918+0000","last_peered":"2024-05-28T14:09:32.764918+0000","last_clean":"2024-05-28T14:09:32.764918+0000","last_became_active":"2024-05-28T14:09:30.754405+0000","last_became_peered":"2024-05-28T14:09:30.754405+0000","last_unstale":"2024-05-28T14:09:32.764918+0000","last_undegraded":"2024-05-28T14:09:32.764918+0000","last_fullsized":"2024-05-28T14:09:32.764918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:29:45.272413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762565+0000","last_change":"2024-05-28T14:09:31.762430+0000","last_active":"2024-05-28T14:09:32.762565+0000","last_peered":"2024-05-28T14:09:32.762565+0000","last_clean":"2024-05-28T14:09:32.762565+0000","last_became_active":"2024-05-28T14:09:31.762101+0000","last_became_peered":"2024-05-28T14:09:31.762101+0000","last_unstale":"2024-05-28T14:09:32.762565+0000","last_undegraded":"2024-05-28T14:09:32.762565+0000","last_fullsized":"2024-05-28T14:09:32.762565+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:40:39.401890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752066+0000","last_change":"2024-05-28T14:09:30.747550+0000","last_active":"2024-05-28T14:09:32.752066+0000","last_peered":"2024-05-28T14:09:32.752066+0000","last_clean":"2024-05-28T14:09:32.752066+0000","last_became_active":"2024-05-28T14:09:30.747222+0000","last_became_peered":"2024-05-28T14:09:30.747222+0000","last_unstale":"2024-05-28T14:09:32.752066+0000","last_undegraded":"2024-05-28T14:09:32.752066+0000","last_fullsized":"2024-05-28T14:09:32.752066+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:58.290350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751952+0000","last_change":"2024-05-28T14:09:31.759538+0000","last_active":"2024-05-28T14:09:32.751952+0000","last_peered":"2024-05-28T14:09:32.751952+0000","last_clean":"2024-05-28T14:09:32.751952+0000","last_became_active":"2024-05-28T14:09:31.759258+0000","last_became_peered":"2024-05-28T14:09:31.759258+0000","last_unstale":"2024-05-28T14:09:32.751952+0000","last_undegraded":"2024-05-28T14:09:32.751952+0000","last_fullsized":"2024-05-28T14:09:32.751952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:35.172304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751683+0000","last_change":"2024-05-28T14:09:30.759103+0000","last_active":"2024-05-28T14:09:32.751683+0000","last_peered":"2024-05-28T14:09:32.751683+0000","last_clean":"2024-05-28T14:09:32.751683+0000","last_became_active":"2024-05-28T14:09:30.758628+0000","last_became_peered":"2024-05-28T14:09:30.758628+0000","last_unstale":"2024-05-28T14:09:32.751683+0000","last_undegraded":"2024-05-28T14:09:32.751683+0000","last_fullsized":"2024-05-28T14:09:32.751683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:54.698345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150467+0000","last_change":"2024-05-28T14:09:31.761382+0000","last_active":"2024-05-28T14:09:33.150467+0000","last_peered":"2024-05-28T14:09:33.150467+0000","last_clean":"2024-05-28T14:09:33.150467+0000","last_became_active":"2024-05-28T14:09:31.761095+0000","last_became_peered":"2024-05-28T14:09:31.761095+0000","last_unstale":"2024-05-28T14:09:33.150467+0000","last_undegraded":"2024-05-28T14:09:33.150467+0000","last_fullsized":"2024-05-28T14:09:33.150467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:31:10.786309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764699+0000","last_change":"2024-05-28T14:09:30.762673+0000","last_active":"2024-05-28T14:09:32.764699+0000","last_peered":"2024-05-28T14:09:32.764699+0000","last_clean":"2024-05-28T14:09:32.764699+0000","last_became_active":"2024-05-28T14:09:30.761793+0000","last_became_peered":"2024-05-28T14:09:30.761793+0000","last_unstale":"2024-05-28T14:09:32.764699+0000","last_undegraded":"2024-05-28T14:09:32.764699+0000","last_fullsized":"2024-05-28T14:09:32.764699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:40:25.005440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747896+0000","last_change":"2024-05-28T14:09:31.766958+0000","last_active":"2024-05-28T14:09:32.747896+0000","last_peered":"2024-05-28T14:09:32.747896+0000","last_clean":"2024-05-28T14:09:32.747896+0000","last_became_active":"2024-05-28T14:09:31.766301+0000","last_became_peered":"2024-05-28T14:09:31.766301+0000","last_unstale":"2024-05-28T14:09:32.747896+0000","last_undegraded":"2024-05-28T14:09:32.747896+0000","last_fullsized":"2024-05-28T14:09:32.747896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:24.707344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762264+0000","last_change":"2024-05-28T14:09:30.746473+0000","last_active":"2024-05-28T14:09:32.762264+0000","last_peered":"2024-05-28T14:09:32.762264+0000","last_clean":"2024-05-28T14:09:32.762264+0000","last_became_active":"2024-05-28T14:09:30.746177+0000","last_became_peered":"2024-05-28T14:09:30.746177+0000","last_unstale":"2024-05-28T14:09:32.762264+0000","last_undegraded":"2024-05-28T14:09:32.762264+0000","last_fullsized":"2024-05-28T14:09:32.762264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:41.356836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.748332+0000","last_change":"2024-05-28T14:09:31.756713+0000","last_active":"2024-05-28T14:09:32.748332+0000","last_peered":"2024-05-28T14:09:32.748332+0000","last_clean":"2024-05-28T14:09:32.748332+0000","last_became_active":"2024-05-28T14:09:31.755716+0000","last_became_peered":"2024-05-28T14:09:31.755716+0000","last_unstale":"2024-05-28T14:09:32.748332+0000","last_undegraded":"2024-05-28T14:09:32.748332+0000","last_fullsized":"2024-05-28T14:09:32.748332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:02:06.074104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151668+0000","last_change":"2024-05-28T14:09:30.758732+0000","last_active":"2024-05-28T14:09:33.151668+0000","last_peered":"2024-05-28T14:09:33.151668+0000","last_clean":"2024-05-28T14:09:33.151668+0000","last_became_active":"2024-05-28T14:09:30.758400+0000","last_became_peered":"2024-05-28T14:09:30.758400+0000","last_unstale":"2024-05-28T14:09:33.151668+0000","last_undegraded":"2024-05-28T14:09:33.151668+0000","last_fullsized":"2024-05-28T14:09:33.151668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:45.752902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750903+0000","last_change":"2024-05-28T14:09:31.762311+0000","last_active":"2024-05-28T14:09:32.750903+0000","last_peered":"2024-05-28T14:09:32.750903+0000","last_clean":"2024-05-28T14:09:32.750903+0000","last_became_active":"2024-05-28T14:09:31.761990+0000","last_became_peered":"2024-05-28T14:09:31.761990+0000","last_unstale":"2024-05-28T14:09:32.750903+0000","last_undegraded":"2024-05-28T14:09:32.750903+0000","last_fullsized":"2024-05-28T14:09:32.750903+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151408+0000","last_change":"2024-05-28T14:09:30.757972+0000","last_active":"2024-05-28T14:09:33.151408+0000","last_peered":"2024-05-28T14:09:33.151408+0000","last_clean":"2024-05-28T14:09:33.151408+0000","last_became_active":"2024-05-28T14:09:30.757716+0000","last_became_peered":"2024-05-28T14:09:30.757716+0000","last_unstale":"2024-05-28T14:09:33.151408+0000","last_undegraded":"2024-05-28T14:09:33.151408+0000","last_fullsized":"2024-05-28T14:09:33.151408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:40:38.394618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764978+0000","last_change":"2024-05-28T14:09:31.754712+0000","last_active":"2024-05-28T14:09:32.764978+0000","last_peered":"2024-05-28T14:09:32.764978+0000","last_clean":"2024-05-28T14:09:32.764978+0000","last_became_active":"2024-05-28T14:09:31.754427+0000","last_became_peered":"2024-05-28T14:09:31.754427+0000","last_unstale":"2024-05-28T14:09:32.764978+0000","last_undegraded":"2024-05-28T14:09:32.764978+0000","last_fullsized":"2024-05-28T14:09:32.764978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151098+0000","last_change":"2024-05-28T14:09:30.756257+0000","last_active":"2024-05-28T14:09:33.151098+0000","last_peered":"2024-05-28T14:09:33.151098+0000","last_clean":"2024-05-28T14:09:33.151098+0000","last_became_active":"2024-05-28T14:09:30.755923+0000","last_became_peered":"2024-05-28T14:09:30.755923+0000","last_unstale":"2024-05-28T14:09:33.151098+0000","last_undegraded":"2024-05-28T14:09:33.151098+0000","last_fullsized":"2024-05-28T14:09:33.151098+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:54.698345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752090+0000","last_change":"2024-05-28T14:09:31.748217+0000","last_active":"2024-05-28T14:09:32.752090+0000","last_peered":"2024-05-28T14:09:32.752090+0000","last_clean":"2024-05-28T14:09:32.752090+0000","last_became_active":"2024-05-28T14:09:31.747336+0000","last_became_peered":"2024-05-28T14:09:31.747336+0000","last_unstale":"2024-05-28T14:09:32.752090+0000","last_undegraded":"2024-05-28T14:09:32.752090+0000","last_fullsized":"2024-05-28T14:09:32.752090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"89'6","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750773+0000","last_change":"2024-05-28T14:09:30.760654+0000","last_active":"2024-05-28T14:09:32.750773+0000","last_peered":"2024-05-28T14:09:32.750773+0000","last_clean":"2024-05-28T14:09:32.750773+0000","last_became_active":"2024-05-28T14:09:30.760328+0000","last_became_peered":"2024-05-28T14:09:30.760328+0000","last_unstale":"2024-05-28T14:09:32.750773+0000","last_undegraded":"2024-05-28T14:09:32.750773+0000","last_fullsized":"2024-05-28T14:09:32.750773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:54:12.659735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764801+0000","last_change":"2024-05-28T14:09:31.742984+0000","last_active":"2024-05-28T14:09:32.764801+0000","last_peered":"2024-05-28T14:09:32.764801+0000","last_clean":"2024-05-28T14:09:32.764801+0000","last_became_active":"2024-05-28T14:09:31.742692+0000","last_became_peered":"2024-05-28T14:09:31.742692+0000","last_unstale":"2024-05-28T14:09:32.764801+0000","last_undegraded":"2024-05-28T14:09:32.764801+0000","last_fullsized":"2024-05-28T14:09:32.764801+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762506+0000","last_change":"2024-05-28T14:09:30.751922+0000","last_active":"2024-05-28T14:09:32.762506+0000","last_peered":"2024-05-28T14:09:32.762506+0000","last_clean":"2024-05-28T14:09:32.762506+0000","last_became_active":"2024-05-28T14:09:30.751646+0000","last_became_peered":"2024-05-28T14:09:30.751646+0000","last_unstale":"2024-05-28T14:09:32.762506+0000","last_undegraded":"2024-05-28T14:09:32.762506+0000","last_fullsized":"2024-05-28T14:09:32.762506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:24:14.215364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150460+0000","last_change":"2024-05-28T14:09:31.763288+0000","last_active":"2024-05-28T14:09:33.150460+0000","last_peered":"2024-05-28T14:09:33.150460+0000","last_clean":"2024-05-28T14:09:33.150460+0000","last_became_active":"2024-05-28T14:09:31.762800+0000","last_became_peered":"2024-05-28T14:09:31.762800+0000","last_unstale":"2024-05-28T14:09:33.150460+0000","last_undegraded":"2024-05-28T14:09:33.150460+0000","last_fullsized":"2024-05-28T14:09:33.150460+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:46:19.764662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747904+0000","last_change":"2024-05-28T14:09:30.753559+0000","last_active":"2024-05-28T14:09:32.747904+0000","last_peered":"2024-05-28T14:09:32.747904+0000","last_clean":"2024-05-28T14:09:32.747904+0000","last_became_active":"2024-05-28T14:09:30.753253+0000","last_became_peered":"2024-05-28T14:09:30.753253+0000","last_unstale":"2024-05-28T14:09:32.747904+0000","last_undegraded":"2024-05-28T14:09:32.747904+0000","last_fullsized":"2024-05-28T14:09:32.747904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:15:03.984981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762339+0000","last_change":"2024-05-28T14:09:31.765598+0000","last_active":"2024-05-28T14:09:32.762339+0000","last_peered":"2024-05-28T14:09:32.762339+0000","last_clean":"2024-05-28T14:09:32.762339+0000","last_became_active":"2024-05-28T14:09:31.760967+0000","last_became_peered":"2024-05-28T14:09:31.760967+0000","last_unstale":"2024-05-28T14:09:32.762339+0000","last_undegraded":"2024-05-28T14:09:32.762339+0000","last_fullsized":"2024-05-28T14:09:32.762339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:24:40.914669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750623+0000","last_change":"2024-05-28T14:09:30.758094+0000","last_active":"2024-05-28T14:09:32.750623+0000","last_peered":"2024-05-28T14:09:32.750623+0000","last_clean":"2024-05-28T14:09:32.750623+0000","last_became_active":"2024-05-28T14:09:30.757801+0000","last_became_peered":"2024-05-28T14:09:30.757801+0000","last_unstale":"2024-05-28T14:09:32.750623+0000","last_undegraded":"2024-05-28T14:09:32.750623+0000","last_fullsized":"2024-05-28T14:09:32.750623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:40:25.005440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762291+0000","last_change":"2024-05-28T14:09:31.754734+0000","last_active":"2024-05-28T14:09:32.762291+0000","last_peered":"2024-05-28T14:09:32.762291+0000","last_clean":"2024-05-28T14:09:32.762291+0000","last_became_active":"2024-05-28T14:09:31.754459+0000","last_became_peered":"2024-05-28T14:09:31.754459+0000","last_unstale":"2024-05-28T14:09:32.762291+0000","last_undegraded":"2024-05-28T14:09:32.762291+0000","last_fullsized":"2024-05-28T14:09:32.762291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:19:30.516845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"89'8","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764575+0000","last_change":"2024-05-28T14:09:30.759773+0000","last_active":"2024-05-28T14:09:32.764575+0000","last_peered":"2024-05-28T14:09:32.764575+0000","last_clean":"2024-05-28T14:09:32.764575+0000","last_became_active":"2024-05-28T14:09:30.759218+0000","last_became_peered":"2024-05-28T14:09:30.759218+0000","last_unstale":"2024-05-28T14:09:32.764575+0000","last_undegraded":"2024-05-28T14:09:32.764575+0000","last_fullsized":"2024-05-28T14:09:32.764575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:14:14.485471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151031+0000","last_change":"2024-05-28T14:09:31.766044+0000","last_active":"2024-05-28T14:09:33.151031+0000","last_peered":"2024-05-28T14:09:33.151031+0000","last_clean":"2024-05-28T14:09:33.151031+0000","last_became_active":"2024-05-28T14:09:31.763364+0000","last_became_peered":"2024-05-28T14:09:31.763364+0000","last_unstale":"2024-05-28T14:09:33.151031+0000","last_undegraded":"2024-05-28T14:09:33.151031+0000","last_fullsized":"2024-05-28T14:09:33.151031+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:54:38.549065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151738+0000","last_change":"2024-05-28T14:09:30.763452+0000","last_active":"2024-05-28T14:09:33.151738+0000","last_peered":"2024-05-28T14:09:33.151738+0000","last_clean":"2024-05-28T14:09:33.151738+0000","last_became_active":"2024-05-28T14:09:30.763075+0000","last_became_peered":"2024-05-28T14:09:30.763075+0000","last_unstale":"2024-05-28T14:09:33.151738+0000","last_undegraded":"2024-05-28T14:09:33.151738+0000","last_fullsized":"2024-05-28T14:09:33.151738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:37.606903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151668+0000","last_change":"2024-05-28T14:09:31.769821+0000","last_active":"2024-05-28T14:09:33.151668+0000","last_peered":"2024-05-28T14:09:33.151668+0000","last_clean":"2024-05-28T14:09:33.151668+0000","last_became_active":"2024-05-28T14:09:31.769534+0000","last_became_peered":"2024-05-28T14:09:31.769534+0000","last_unstale":"2024-05-28T14:09:33.151668+0000","last_undegraded":"2024-05-28T14:09:33.151668+0000","last_fullsized":"2024-05-28T14:09:33.151668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765169+0000","last_change":"2024-05-28T14:09:30.752703+0000","last_active":"2024-05-28T14:09:32.765169+0000","last_peered":"2024-05-28T14:09:32.765169+0000","last_clean":"2024-05-28T14:09:32.765169+0000","last_became_active":"2024-05-28T14:09:30.752399+0000","last_became_peered":"2024-05-28T14:09:30.752399+0000","last_unstale":"2024-05-28T14:09:32.765169+0000","last_undegraded":"2024-05-28T14:09:32.765169+0000","last_fullsized":"2024-05-28T14:09:32.765169+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:42.742326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752236+0000","last_change":"2024-05-28T14:09:31.757626+0000","last_active":"2024-05-28T14:09:32.752236+0000","last_peered":"2024-05-28T14:09:32.752236+0000","last_clean":"2024-05-28T14:09:32.752236+0000","last_became_active":"2024-05-28T14:09:31.756947+0000","last_became_peered":"2024-05-28T14:09:31.756947+0000","last_unstale":"2024-05-28T14:09:32.752236+0000","last_undegraded":"2024-05-28T14:09:32.752236+0000","last_fullsized":"2024-05-28T14:09:32.752236+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765042+0000","last_change":"2024-05-28T14:09:30.760474+0000","last_active":"2024-05-28T14:09:32.765042+0000","last_peered":"2024-05-28T14:09:32.765042+0000","last_clean":"2024-05-28T14:09:32.765042+0000","last_became_active":"2024-05-28T14:09:30.759783+0000","last_became_peered":"2024-05-28T14:09:30.759783+0000","last_unstale":"2024-05-28T14:09:32.765042+0000","last_undegraded":"2024-05-28T14:09:32.765042+0000","last_fullsized":"2024-05-28T14:09:32.765042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:49:41.271344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.748033+0000","last_change":"2024-05-28T14:09:30.754286+0000","last_active":"2024-05-28T14:09:32.748033+0000","last_peered":"2024-05-28T14:09:32.748033+0000","last_clean":"2024-05-28T14:09:32.748033+0000","last_became_active":"2024-05-28T14:09:30.753997+0000","last_became_peered":"2024-05-28T14:09:30.753997+0000","last_unstale":"2024-05-28T14:09:32.748033+0000","last_undegraded":"2024-05-28T14:09:32.748033+0000","last_fullsized":"2024-05-28T14:09:32.748033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:34:27.167006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751282+0000","last_change":"2024-05-28T14:09:31.760545+0000","last_active":"2024-05-28T14:09:32.751282+0000","last_peered":"2024-05-28T14:09:32.751282+0000","last_clean":"2024-05-28T14:09:32.751282+0000","last_became_active":"2024-05-28T14:09:31.760024+0000","last_became_peered":"2024-05-28T14:09:31.760024+0000","last_unstale":"2024-05-28T14:09:32.751282+0000","last_undegraded":"2024-05-28T14:09:32.751282+0000","last_fullsized":"2024-05-28T14:09:32.751282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-05-28T14:09:29.732310+0000","last_change":"2024-05-28T14:09:29.732310+0000","last_active":"2024-05-28T14:09:29.717625+0000","last_peered":"2024-05-28T14:09:29.717625+0000","last_clean":"2024-05-28T14:09:29.717625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:29.732310+0000","last_undegraded":"2024-05-28T14:09:29.732310+0000","last_fullsized":"2024-05-28T14:09:29.732310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-05-28T14:09:29.731567+0000","last_change":"2024-05-28T14:09:29.731567+0000","last_active":"2024-05-28T14:09:29.717625+0000","last_peered":"2024-05-28T14:09:29.717625+0000","last_clean":"2024-05-28T14:09:29.717625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:29.731567+0000","last_undegraded":"2024-05-28T14:09:29.731567+0000","last_fullsized":"2024-05-28T14:09:29.731567+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751081+0000","last_change":"2024-05-28T14:09:31.748847+0000","last_active":"2024-05-28T14:09:32.751081+0000","last_peered":"2024-05-28T14:09:32.751081+0000","last_clean":"2024-05-28T14:09:32.751081+0000","last_became_active":"2024-05-28T14:09:31.748479+0000","last_became_peered":"2024-05-28T14:09:31.748479+0000","last_unstale":"2024-05-28T14:09:32.751081+0000","last_undegraded":"2024-05-28T14:09:32.751081+0000","last_fullsized":"2024-05-28T14:09:32.751081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747718+0000","last_change":"2024-05-28T14:09:30.755244+0000","last_active":"2024-05-28T14:09:32.747718+0000","last_peered":"2024-05-28T14:09:32.747718+0000","last_clean":"2024-05-28T14:09:32.747718+0000","last_became_active":"2024-05-28T14:09:30.754984+0000","last_became_peered":"2024-05-28T14:09:30.754984+0000","last_unstale":"2024-05-28T14:09:32.747718+0000","last_undegraded":"2024-05-28T14:09:32.747718+0000","last_fullsized":"2024-05-28T14:09:32.747718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:14:58.988961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752367+0000","last_change":"2024-05-28T14:09:31.745138+0000","last_active":"2024-05-28T14:09:32.752367+0000","last_peered":"2024-05-28T14:09:32.752367+0000","last_clean":"2024-05-28T14:09:32.752367+0000","last_became_active":"2024-05-28T14:09:31.744819+0000","last_became_peered":"2024-05-28T14:09:31.744819+0000","last_unstale":"2024-05-28T14:09:32.752367+0000","last_undegraded":"2024-05-28T14:09:32.752367+0000","last_fullsized":"2024-05-28T14:09:32.752367+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762790+0000","last_change":"2024-05-28T14:09:30.760787+0000","last_active":"2024-05-28T14:09:32.762790+0000","last_peered":"2024-05-28T14:09:32.762790+0000","last_clean":"2024-05-28T14:09:32.762790+0000","last_became_active":"2024-05-28T14:09:30.760502+0000","last_became_peered":"2024-05-28T14:09:30.760502+0000","last_unstale":"2024-05-28T14:09:32.762790+0000","last_undegraded":"2024-05-28T14:09:32.762790+0000","last_fullsized":"2024-05-28T14:09:32.762790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:31:35.886734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150913+0000","last_change":"2024-05-28T14:09:31.771112+0000","last_active":"2024-05-28T14:09:33.150913+0000","last_peered":"2024-05-28T14:09:33.150913+0000","last_clean":"2024-05-28T14:09:33.150913+0000","last_became_active":"2024-05-28T14:09:31.770660+0000","last_became_peered":"2024-05-28T14:09:31.770660+0000","last_unstale":"2024-05-28T14:09:33.150913+0000","last_undegraded":"2024-05-28T14:09:33.150913+0000","last_fullsized":"2024-05-28T14:09:33.150913+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:02:06.074104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762843+0000","last_change":"2024-05-28T14:09:30.751801+0000","last_active":"2024-05-28T14:09:32.762843+0000","last_peered":"2024-05-28T14:09:32.762843+0000","last_clean":"2024-05-28T14:09:32.762843+0000","last_became_active":"2024-05-28T14:09:30.751504+0000","last_became_peered":"2024-05-28T14:09:30.751504+0000","last_unstale":"2024-05-28T14:09:32.762843+0000","last_undegraded":"2024-05-28T14:09:32.762843+0000","last_fullsized":"2024-05-28T14:09:32.762843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:42.042818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151673+0000","last_change":"2024-05-28T14:09:31.765811+0000","last_active":"2024-05-28T14:09:33.151673+0000","last_peered":"2024-05-28T14:09:33.151673+0000","last_clean":"2024-05-28T14:09:33.151673+0000","last_became_active":"2024-05-28T14:09:31.764710+0000","last_became_peered":"2024-05-28T14:09:31.764710+0000","last_unstale":"2024-05-28T14:09:33.151673+0000","last_undegraded":"2024-05-28T14:09:33.151673+0000","last_fullsized":"2024-05-28T14:09:33.151673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:57.967039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151377+0000","last_change":"2024-05-28T14:09:30.760473+0000","last_active":"2024-05-28T14:09:33.151377+0000","last_peered":"2024-05-28T14:09:33.151377+0000","last_clean":"2024-05-28T14:09:33.151377+0000","last_became_active":"2024-05-28T14:09:30.760129+0000","last_became_peered":"2024-05-28T14:09:30.760129+0000","last_unstale":"2024-05-28T14:09:33.151377+0000","last_undegraded":"2024-05-28T14:09:33.151377+0000","last_fullsized":"2024-05-28T14:09:33.151377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:18.317793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752265+0000","last_change":"2024-05-28T14:09:31.748833+0000","last_active":"2024-05-28T14:09:32.752265+0000","last_peered":"2024-05-28T14:09:32.752265+0000","last_clean":"2024-05-28T14:09:32.752265+0000","last_became_active":"2024-05-28T14:09:31.747731+0000","last_became_peered":"2024-05-28T14:09:31.747731+0000","last_unstale":"2024-05-28T14:09:32.752265+0000","last_undegraded":"2024-05-28T14:09:32.752265+0000","last_fullsized":"2024-05-28T14:09:32.752265+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151485+0000","last_change":"2024-05-28T14:09:30.758617+0000","last_active":"2024-05-28T14:09:33.151485+0000","last_peered":"2024-05-28T14:09:33.151485+0000","last_clean":"2024-05-28T14:09:33.151485+0000","last_became_active":"2024-05-28T14:09:30.758317+0000","last_became_peered":"2024-05-28T14:09:30.758317+0000","last_unstale":"2024-05-28T14:09:33.151485+0000","last_undegraded":"2024-05-28T14:09:33.151485+0000","last_fullsized":"2024-05-28T14:09:33.151485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:10.090547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751157+0000","last_change":"2024-05-28T14:09:31.760268+0000","last_active":"2024-05-28T14:09:32.751157+0000","last_peered":"2024-05-28T14:09:32.751157+0000","last_clean":"2024-05-28T14:09:32.751157+0000","last_became_active":"2024-05-28T14:09:31.759889+0000","last_became_peered":"2024-05-28T14:09:31.759889+0000","last_unstale":"2024-05-28T14:09:32.751157+0000","last_undegraded":"2024-05-28T14:09:32.751157+0000","last_fullsized":"2024-05-28T14:09:32.751157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764892+0000","last_change":"2024-05-28T14:09:30.754916+0000","last_active":"2024-05-28T14:09:32.764892+0000","last_peered":"2024-05-28T14:09:32.764892+0000","last_clean":"2024-05-28T14:09:32.764892+0000","last_became_active":"2024-05-28T14:09:30.754019+0000","last_became_peered":"2024-05-28T14:09:30.754019+0000","last_unstale":"2024-05-28T14:09:32.764892+0000","last_undegraded":"2024-05-28T14:09:32.764892+0000","last_fullsized":"2024-05-28T14:09:32.764892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:34.165033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764999+0000","last_change":"2024-05-28T14:09:31.748964+0000","last_active":"2024-05-28T14:09:32.764999+0000","last_peered":"2024-05-28T14:09:32.764999+0000","last_clean":"2024-05-28T14:09:32.764999+0000","last_became_active":"2024-05-28T14:09:31.748393+0000","last_became_peered":"2024-05-28T14:09:31.748393+0000","last_unstale":"2024-05-28T14:09:32.764999+0000","last_undegraded":"2024-05-28T14:09:32.764999+0000","last_fullsized":"2024-05-28T14:09:32.764999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751724+0000","last_change":"2024-05-28T14:09:30.748735+0000","last_active":"2024-05-28T14:09:32.751724+0000","last_peered":"2024-05-28T14:09:32.751724+0000","last_clean":"2024-05-28T14:09:32.751724+0000","last_became_active":"2024-05-28T14:09:30.748404+0000","last_became_peered":"2024-05-28T14:09:30.748404+0000","last_unstale":"2024-05-28T14:09:32.751724+0000","last_undegraded":"2024-05-28T14:09:32.751724+0000","last_fullsized":"2024-05-28T14:09:32.751724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:37.606903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150429+0000","last_change":"2024-05-28T14:09:31.764716+0000","last_active":"2024-05-28T14:09:33.150429+0000","last_peered":"2024-05-28T14:09:33.150429+0000","last_clean":"2024-05-28T14:09:33.150429+0000","last_became_active":"2024-05-28T14:09:31.764407+0000","last_became_peered":"2024-05-28T14:09:31.764407+0000","last_unstale":"2024-05-28T14:09:33.150429+0000","last_undegraded":"2024-05-28T14:09:33.150429+0000","last_fullsized":"2024-05-28T14:09:33.150429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:14:59.996233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151098+0000","last_change":"2024-05-28T14:09:30.755558+0000","last_active":"2024-05-28T14:09:33.151098+0000","last_peered":"2024-05-28T14:09:33.151098+0000","last_clean":"2024-05-28T14:09:33.151098+0000","last_became_active":"2024-05-28T14:09:30.755257+0000","last_became_peered":"2024-05-28T14:09:30.755257+0000","last_unstale":"2024-05-28T14:09:33.151098+0000","last_undegraded":"2024-05-28T14:09:33.151098+0000","last_fullsized":"2024-05-28T14:09:33.151098+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:58.290350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150859+0000","last_change":"2024-05-28T14:09:31.758209+0000","last_active":"2024-05-28T14:09:33.150859+0000","last_peered":"2024-05-28T14:09:33.150859+0000","last_clean":"2024-05-28T14:09:33.150859+0000","last_became_active":"2024-05-28T14:09:31.757910+0000","last_became_peered":"2024-05-28T14:09:31.757910+0000","last_unstale":"2024-05-28T14:09:33.150859+0000","last_undegraded":"2024-05-28T14:09:33.150859+0000","last_fullsized":"2024-05-28T14:09:33.150859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:36:07.311954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751556+0000","last_change":"2024-05-28T14:09:30.754615+0000","last_active":"2024-05-28T14:09:32.751556+0000","last_peered":"2024-05-28T14:09:32.751556+0000","last_clean":"2024-05-28T14:09:32.751556+0000","last_became_active":"2024-05-28T14:09:30.754325+0000","last_became_peered":"2024-05-28T14:09:30.754325+0000","last_unstale":"2024-05-28T14:09:32.751556+0000","last_undegraded":"2024-05-28T14:09:32.751556+0000","last_fullsized":"2024-05-28T14:09:32.751556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:45.752902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762719+0000","last_change":"2024-05-28T14:09:31.760657+0000","last_active":"2024-05-28T14:09:32.762719+0000","last_peered":"2024-05-28T14:09:32.762719+0000","last_clean":"2024-05-28T14:09:32.762719+0000","last_became_active":"2024-05-28T14:09:31.759615+0000","last_became_peered":"2024-05-28T14:09:31.759615+0000","last_unstale":"2024-05-28T14:09:32.762719+0000","last_undegraded":"2024-05-28T14:09:32.762719+0000","last_fullsized":"2024-05-28T14:09:32.762719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:55.705617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151176+0000","last_change":"2024-05-28T14:09:30.762060+0000","last_active":"2024-05-28T14:09:33.151176+0000","last_peered":"2024-05-28T14:09:33.151176+0000","last_clean":"2024-05-28T14:09:33.151176+0000","last_became_active":"2024-05-28T14:09:30.761778+0000","last_became_peered":"2024-05-28T14:09:30.761778+0000","last_unstale":"2024-05-28T14:09:33.151176+0000","last_undegraded":"2024-05-28T14:09:33.151176+0000","last_fullsized":"2024-05-28T14:09:33.151176+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:56.959767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":300,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150107+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:09:33.150107+0000","last_peered":"2024-05-28T14:09:33.150107+0000","last_clean":"2024-05-28T14:09:33.150107+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:09:33.150107+0000","last_undegraded":"2024-05-28T14:09:33.150107+0000","last_fullsized":"2024-05-28T14:09:33.150107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":7},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123028,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248812,"statfs":{"total":96636764160,"available":96510783488,"internally_reserved":0,"allocated":1703936,"data_stored":1068711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122448,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249392,"statfs":{"total":96636764160,"available":96511377408,"internally_reserved":0,"allocated":1118208,"data_stored":478967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":124254662},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76628,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295212,"statfs":{"total":96636764160,"available":96558297088,"internally_reserved":0,"allocated":1122304,"data_stored":479907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82184,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289656,"statfs":{"total":96636764160,"available":96552607744,"internally_reserved":0,"allocated":1105920,"data_stored":478273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574987,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130772,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241068,"statfs":{"total":96636764160,"available":96502853632,"internally_reserved":0,"allocated":1118208,"data_stored":479885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":132774344},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144604,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227236,"statfs":{"total":96636764160,"available":96488689664,"internally_reserved":0,"allocated":1056768,"data_stored":452994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108564,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94263276,"statfs":{"total":96636764160,"available":96525594624,"internally_reserved":0,"allocated":1118208,"data_stored":478467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64724,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307116,"statfs":{"total":96636764160,"available":96570486784,"internally_reserved":0,"allocated":1703936,"data_stored":1068675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:09:35.861 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:09:36.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.057+0000 7fb3c3c00640 1 Processor -- start 2024-05-28T14:09:36.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.057+0000 7fb3c3c00640 1 -- start start 2024-05-28T14:09:36.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.057+0000 7fb3c3c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb3bc162c80 0x7fb3bc15e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.057+0000 7fb3c3c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3bc0d0de0 con 0x7fb3bc0d0900 2024-05-28T14:09:36.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.057+0000 7fb3c3c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3bc0d10c0 con 0x7fb3bc0ce860 2024-05-28T14:09:36.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.057+0000 7fb3c3c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb3bc0d19f0 con 0x7fb3bc162c80 2024-05-28T14:09:36.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.057+0000 7fb3c1c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb3bc162c80 0x7fb3bc15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.057+0000 7fb3b3e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb3bc0d0900 0x7fb3bc162570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47316/0 (socket says 172.21.15.42:47316) 2024-05-28T14:09:36.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.057+0000 7fb3b3e00640 1 -- 172.21.15.42:0/1505899910 learned_addr learned my addr 172.21.15.42:0/1505899910 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:36.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.057+0000 7fb3c1c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb3bc162c80 0x7fb3bc15e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56418/0 (socket says 172.21.15.42:56418) 2024-05-28T14:09:36.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.057+0000 7fb3c1c00640 1 -- 172.21.15.42:0/1505899910 >> v1:172.21.15.63:6789/0 conn(0x7fb3bc0ce860 legacy=0x7fb3bc06d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.057+0000 7fb3c1c00640 1 -- 172.21.15.42:0/1505899910 >> v1:172.21.15.42:6789/0 conn(0x7fb3bc0d0900 legacy=0x7fb3bc162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.057+0000 7fb3c1c00640 1 -- 172.21.15.42:0/1505899910 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc0d13a0 con 0x7fb3bc162c80 2024-05-28T14:09:36.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.057+0000 7fb3bbe00640 1 -- 172.21.15.42:0/1505899910 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4043788883 0 0) 0x7fb3bc0d0de0 con 0x7fb3bc0d0900 2024-05-28T14:09:36.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.061+0000 7fb3c1c00640 1 --2- 172.21.15.42:0/1505899910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc162c80 0x7fb3bc15e200 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009d90 tx=0x7fb3b401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafd3f4e449c9740 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:36.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.061+0000 7fb3bbe00640 1 -- 172.21.15.42:0/1505899910 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40b2070 con 0x7fb3bc162c80 2024-05-28T14:09:36.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.061+0000 7fb3bbe00640 1 -- 172.21.15.42:0/1505899910 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b40a5890 con 0x7fb3bc162c80 2024-05-28T14:09:36.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.061+0000 7fb3bbe00640 1 -- 172.21.15.42:0/1505899910 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40a5c60 con 0x7fb3bc162c80 2024-05-28T14:09:36.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.061+0000 7fb3c3c00640 1 -- 172.21.15.42:0/1505899910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc162c80 msgr2=0x7fb3bc15e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.061+0000 7fb3c3c00640 1 --2- 172.21.15.42:0/1505899910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc162c80 0x7fb3bc15e200 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009d90 tx=0x7fb3b401c8b0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.061+0000 7fb3c3c00640 1 -- 172.21.15.42:0/1505899910 shutdown_connections 2024-05-28T14:09:36.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.061+0000 7fb3c3c00640 1 --2- 172.21.15.42:0/1505899910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc162c80 0x7fb3bc15e200 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.061+0000 7fb3c3c00640 1 -- 172.21.15.42:0/1505899910 >> 172.21.15.42:0/1505899910 conn(0x7fb3bc0b4080 msgr2=0x7fb3bc0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:36.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.061+0000 7fb3c3c00640 1 -- 172.21.15.42:0/1505899910 shutdown_connections 2024-05-28T14:09:36.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.061+0000 7fb3c3c00640 1 -- 172.21.15.42:0/1505899910 wait complete. 2024-05-28T14:09:36.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c3c00640 1 Processor -- start 2024-05-28T14:09:36.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c3c00640 1 -- start start 2024-05-28T14:09:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c3c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc0ce860 0x7fb3bc0b8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c3c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3bc0d0900 0x7fb3bc0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c3c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3bc162c80 0x7fb3bc15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c3c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc16ca40 con 0x7fb3bc0ce860 2024-05-28T14:09:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c3c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc16c8c0 con 0x7fb3bc0d0900 2024-05-28T14:09:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c3c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb3bc16cbc0 con 0x7fb3bc162c80 2024-05-28T14:09:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc0ce860 0x7fb3bc0b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc0ce860 0x7fb3bc0b8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56420/0 (socket says 172.21.15.42:56420) 2024-05-28T14:09:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c1200640 1 -- 172.21.15.42:0/3874919039 learned_addr learned my addr 172.21.15.42:0/3874919039 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:36.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3b3e00640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3bc0d0900 0x7fb3bc0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c1200640 1 -- 172.21.15.42:0/3874919039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3bc162c80 msgr2=0x7fb3bc15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c1c00640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3bc162c80 0x7fb3bc15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c1200640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3bc162c80 0x7fb3bc15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c1200640 1 -- 172.21.15.42:0/3874919039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3bc0d0900 msgr2=0x7fb3bc0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c1200640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3bc0d0900 0x7fb3bc0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c1200640 1 -- 172.21.15.42:0/3874919039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc0b5920 con 0x7fb3bc0ce860 2024-05-28T14:09:36.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3b3e00640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3bc0d0900 0x7fb3bc0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:36.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c1c00640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3bc162c80 0x7fb3bc15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:36.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c1200640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc0ce860 0x7fb3bc0b8040 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00c500 tx=0x7fb3ac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:36.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3b9600640 1 -- 172.21.15.42:0/3874919039 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac016020 con 0x7fb3bc0ce860 2024-05-28T14:09:36.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3b9600640 1 -- 172.21.15.42:0/3874919039 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ac005220 con 0x7fb3bc0ce860 2024-05-28T14:09:36.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3b9600640 1 -- 172.21.15.42:0/3874919039 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac007030 con 0x7fb3bc0ce860 2024-05-28T14:09:36.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c3c00640 1 -- 172.21.15.42:0/3874919039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3bc0b5c00 con 0x7fb3bc0ce860 2024-05-28T14:09:36.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c3c00640 1 -- 172.21.15.42:0/3874919039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3bc0b61c0 con 0x7fb3bc0ce860 2024-05-28T14:09:36.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.065+0000 7fb3c3c00640 1 -- 172.21.15.42:0/3874919039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3bc068a90 con 0x7fb3bc0ce860 2024-05-28T14:09:36.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.069+0000 7fb3b9600640 1 -- 172.21.15.42:0/3874919039 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fb3ac0071f0 con 0x7fb3bc0ce860 2024-05-28T14:09:36.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.069+0000 7fb3b9600640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb398069520 0x7fb39806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.069+0000 7fb3b9600640 1 -- 172.21.15.42:0/3874919039 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7fb3ac085320 con 0x7fb3bc0ce860 2024-05-28T14:09:36.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.069+0000 7fb3b3e00640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb398069520 0x7fb39806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.069+0000 7fb3b3e00640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb398069520 0x7fb39806b9e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0b6f00 tx=0x7fb3a4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:36.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.073+0000 7fb3b9600640 1 -- 172.21.15.42:0/3874919039 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb3ac020020 con 0x7fb3bc0ce860 2024-05-28T14:09:36.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.233+0000 7fb3c3c00640 1 -- 172.21.15.42:0/3874919039 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb3bc069230 con 0x7fb398069520 2024-05-28T14:09:36.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.241+0000 7fb3b9600640 1 -- 172.21.15.42:0/3874919039 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250670 (secure 0 0 0) 0x7fb3bc069230 con 0x7fb398069520 2024-05-28T14:09:36.227 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:36.229 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:09:36.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.245+0000 7fb3c3c00640 1 -- 172.21.15.42:0/3874919039 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb398069520 msgr2=0x7fb39806b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.245+0000 7fb3c3c00640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb398069520 0x7fb39806b9e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0b6f00 tx=0x7fb3a4009040 comp rx=0 tx=0).stop 2024-05-28T14:09:36.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.245+0000 7fb3c3c00640 1 -- 172.21.15.42:0/3874919039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc0ce860 msgr2=0x7fb3bc0b8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.245+0000 7fb3c3c00640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc0ce860 0x7fb3bc0b8040 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00c500 tx=0x7fb3ac00c9c0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.245+0000 7fb3c3c00640 1 -- 172.21.15.42:0/3874919039 shutdown_connections 2024-05-28T14:09:36.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.245+0000 7fb3c3c00640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb398069520 0x7fb39806b9e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.245+0000 7fb3c3c00640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3bc162c80 0x7fb3bc15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.245+0000 7fb3c3c00640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3bc0d0900 0x7fb3bc0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.245+0000 7fb3c3c00640 1 --2- 172.21.15.42:0/3874919039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc0ce860 0x7fb3bc0b8040 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.245+0000 7fb3c3c00640 1 -- 172.21.15.42:0/3874919039 >> 172.21.15.42:0/3874919039 conn(0x7fb3bc0b4080 msgr2=0x7fb3bc070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:36.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.245+0000 7fb3c3c00640 1 -- 172.21.15.42:0/3874919039 shutdown_connections 2024-05-28T14:09:36.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.245+0000 7fb3c3c00640 1 -- 172.21.15.42:0/3874919039 wait complete. 2024-05-28T14:09:36.249 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-05-28T14:09:35.112730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":852952,"kb_used_data":9812,"kb_used_omap":12,"kb_used_meta":842995,"kb_avail":754121768,"statfs":{"total":773094113280,"available":772220690432,"internally_reserved":0,"allocated":10047488,"data_stored":4985879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":863227442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.387621"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151546+0000","last_change":"2024-05-28T14:09:31.763443+0000","last_active":"2024-05-28T14:09:33.151546+0000","last_peered":"2024-05-28T14:09:33.151546+0000","last_clean":"2024-05-28T14:09:33.151546+0000","last_became_active":"2024-05-28T14:09:31.763080+0000","last_became_peered":"2024-05-28T14:09:31.763080+0000","last_unstale":"2024-05-28T14:09:33.151546+0000","last_undegraded":"2024-05-28T14:09:33.151546+0000","last_fullsized":"2024-05-28T14:09:33.151546+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762829+0000","last_change":"2024-05-28T14:09:31.768406+0000","last_active":"2024-05-28T14:09:32.762829+0000","last_peered":"2024-05-28T14:09:32.762829+0000","last_clean":"2024-05-28T14:09:32.762829+0000","last_became_active":"2024-05-28T14:09:31.768150+0000","last_became_peered":"2024-05-28T14:09:31.768150+0000","last_unstale":"2024-05-28T14:09:32.762829+0000","last_undegraded":"2024-05-28T14:09:32.762829+0000","last_fullsized":"2024-05-28T14:09:32.762829+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:38.614174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150788+0000","last_change":"2024-05-28T14:09:31.770809+0000","last_active":"2024-05-28T14:09:33.150788+0000","last_peered":"2024-05-28T14:09:33.150788+0000","last_clean":"2024-05-28T14:09:33.150788+0000","last_became_active":"2024-05-28T14:09:31.770547+0000","last_became_peered":"2024-05-28T14:09:31.770547+0000","last_unstale":"2024-05-28T14:09:33.150788+0000","last_undegraded":"2024-05-28T14:09:33.150788+0000","last_fullsized":"2024-05-28T14:09:33.150788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:34:22.994257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751824+0000","last_change":"2024-05-28T14:09:31.756701+0000","last_active":"2024-05-28T14:09:32.751824+0000","last_peered":"2024-05-28T14:09:32.751824+0000","last_clean":"2024-05-28T14:09:32.751824+0000","last_became_active":"2024-05-28T14:09:31.756425+0000","last_became_peered":"2024-05-28T14:09:31.756425+0000","last_unstale":"2024-05-28T14:09:32.751824+0000","last_undegraded":"2024-05-28T14:09:32.751824+0000","last_fullsized":"2024-05-28T14:09:32.751824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:43.749597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750865+0000","last_change":"2024-05-28T14:09:31.756691+0000","last_active":"2024-05-28T14:09:32.750865+0000","last_peered":"2024-05-28T14:09:32.750865+0000","last_clean":"2024-05-28T14:09:32.750865+0000","last_became_active":"2024-05-28T14:09:31.756434+0000","last_became_peered":"2024-05-28T14:09:31.756434+0000","last_unstale":"2024-05-28T14:09:32.750865+0000","last_undegraded":"2024-05-28T14:09:32.750865+0000","last_fullsized":"2024-05-28T14:09:32.750865+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751086+0000","last_change":"2024-05-28T14:09:31.761009+0000","last_active":"2024-05-28T14:09:32.751086+0000","last_peered":"2024-05-28T14:09:32.751086+0000","last_clean":"2024-05-28T14:09:32.751086+0000","last_became_active":"2024-05-28T14:09:31.760693+0000","last_became_peered":"2024-05-28T14:09:31.760693+0000","last_unstale":"2024-05-28T14:09:32.751086+0000","last_undegraded":"2024-05-28T14:09:32.751086+0000","last_fullsized":"2024-05-28T14:09:32.751086+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751130+0000","last_change":"2024-05-28T14:09:31.748781+0000","last_active":"2024-05-28T14:09:32.751130+0000","last_peered":"2024-05-28T14:09:32.751130+0000","last_clean":"2024-05-28T14:09:32.751130+0000","last_became_active":"2024-05-28T14:09:31.748446+0000","last_became_peered":"2024-05-28T14:09:31.748446+0000","last_unstale":"2024-05-28T14:09:32.751130+0000","last_undegraded":"2024-05-28T14:09:32.751130+0000","last_fullsized":"2024-05-28T14:09:32.751130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151416+0000","last_change":"2024-05-28T14:09:31.755098+0000","last_active":"2024-05-28T14:09:33.151416+0000","last_peered":"2024-05-28T14:09:33.151416+0000","last_clean":"2024-05-28T14:09:33.151416+0000","last_became_active":"2024-05-28T14:09:31.754790+0000","last_became_peered":"2024-05-28T14:09:31.754790+0000","last_unstale":"2024-05-28T14:09:33.151416+0000","last_undegraded":"2024-05-28T14:09:33.151416+0000","last_fullsized":"2024-05-28T14:09:33.151416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762851+0000","last_change":"2024-05-28T14:09:31.765027+0000","last_active":"2024-05-28T14:09:32.762851+0000","last_peered":"2024-05-28T14:09:32.762851+0000","last_clean":"2024-05-28T14:09:32.762851+0000","last_became_active":"2024-05-28T14:09:31.764660+0000","last_became_peered":"2024-05-28T14:09:31.764660+0000","last_unstale":"2024-05-28T14:09:32.762851+0000","last_undegraded":"2024-05-28T14:09:32.762851+0000","last_fullsized":"2024-05-28T14:09:32.762851+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:59.297622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150769+0000","last_change":"2024-05-28T14:09:31.756022+0000","last_active":"2024-05-28T14:09:33.150769+0000","last_peered":"2024-05-28T14:09:33.150769+0000","last_clean":"2024-05-28T14:09:33.150769+0000","last_became_active":"2024-05-28T14:09:31.755772+0000","last_became_peered":"2024-05-28T14:09:31.755772+0000","last_unstale":"2024-05-28T14:09:33.150769+0000","last_undegraded":"2024-05-28T14:09:33.150769+0000","last_fullsized":"2024-05-28T14:09:33.150769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:35.172304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150463+0000","last_change":"2024-05-28T14:09:31.764389+0000","last_active":"2024-05-28T14:09:33.150463+0000","last_peered":"2024-05-28T14:09:33.150463+0000","last_clean":"2024-05-28T14:09:33.150463+0000","last_became_active":"2024-05-28T14:09:31.764102+0000","last_became_peered":"2024-05-28T14:09:31.764102+0000","last_unstale":"2024-05-28T14:09:33.150463+0000","last_undegraded":"2024-05-28T14:09:33.150463+0000","last_fullsized":"2024-05-28T14:09:33.150463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:37:41.765233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747849+0000","last_change":"2024-05-28T14:09:31.766893+0000","last_active":"2024-05-28T14:09:32.747849+0000","last_peered":"2024-05-28T14:09:32.747849+0000","last_clean":"2024-05-28T14:09:32.747849+0000","last_became_active":"2024-05-28T14:09:31.766457+0000","last_became_peered":"2024-05-28T14:09:31.766457+0000","last_unstale":"2024-05-28T14:09:32.747849+0000","last_undegraded":"2024-05-28T14:09:32.747849+0000","last_fullsized":"2024-05-28T14:09:32.747849+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:34:22.994257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751531+0000","last_change":"2024-05-28T14:09:31.753479+0000","last_active":"2024-05-28T14:09:32.751531+0000","last_peered":"2024-05-28T14:09:32.751531+0000","last_clean":"2024-05-28T14:09:32.751531+0000","last_became_active":"2024-05-28T14:09:31.753094+0000","last_became_peered":"2024-05-28T14:09:31.753094+0000","last_unstale":"2024-05-28T14:09:32.751531+0000","last_undegraded":"2024-05-28T14:09:32.751531+0000","last_fullsized":"2024-05-28T14:09:32.751531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:49:42.278616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752189+0000","last_change":"2024-05-28T14:09:31.759451+0000","last_active":"2024-05-28T14:09:32.752189+0000","last_peered":"2024-05-28T14:09:32.752189+0000","last_clean":"2024-05-28T14:09:32.752189+0000","last_became_active":"2024-05-28T14:09:31.759197+0000","last_became_peered":"2024-05-28T14:09:31.759197+0000","last_unstale":"2024-05-28T14:09:32.752189+0000","last_undegraded":"2024-05-28T14:09:32.752189+0000","last_fullsized":"2024-05-28T14:09:32.752189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765154+0000","last_change":"2024-05-28T14:09:31.753754+0000","last_active":"2024-05-28T14:09:32.765154+0000","last_peered":"2024-05-28T14:09:32.765154+0000","last_clean":"2024-05-28T14:09:32.765154+0000","last_became_active":"2024-05-28T14:09:31.753211+0000","last_became_peered":"2024-05-28T14:09:31.753211+0000","last_unstale":"2024-05-28T14:09:32.765154+0000","last_undegraded":"2024-05-28T14:09:32.765154+0000","last_fullsized":"2024-05-28T14:09:32.765154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762453+0000","last_change":"2024-05-28T14:09:31.755223+0000","last_active":"2024-05-28T14:09:32.762453+0000","last_peered":"2024-05-28T14:09:32.762453+0000","last_clean":"2024-05-28T14:09:32.762453+0000","last_became_active":"2024-05-28T14:09:31.754880+0000","last_became_peered":"2024-05-28T14:09:31.754880+0000","last_unstale":"2024-05-28T14:09:32.762453+0000","last_undegraded":"2024-05-28T14:09:32.762453+0000","last_fullsized":"2024-05-28T14:09:32.762453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:25.450855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150682+0000","last_change":"2024-05-28T14:09:31.752700+0000","last_active":"2024-05-28T14:09:33.150682+0000","last_peered":"2024-05-28T14:09:33.150682+0000","last_clean":"2024-05-28T14:09:33.150682+0000","last_became_active":"2024-05-28T14:09:31.752367+0000","last_became_peered":"2024-05-28T14:09:31.752367+0000","last_unstale":"2024-05-28T14:09:33.150682+0000","last_undegraded":"2024-05-28T14:09:33.150682+0000","last_fullsized":"2024-05-28T14:09:33.150682+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:43.749597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762838+0000","last_change":"2024-05-28T14:09:31.756171+0000","last_active":"2024-05-28T14:09:32.762838+0000","last_peered":"2024-05-28T14:09:32.762838+0000","last_clean":"2024-05-28T14:09:32.762838+0000","last_became_active":"2024-05-28T14:09:31.754405+0000","last_became_peered":"2024-05-28T14:09:31.754405+0000","last_unstale":"2024-05-28T14:09:32.762838+0000","last_undegraded":"2024-05-28T14:09:32.762838+0000","last_fullsized":"2024-05-28T14:09:32.762838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:19.325065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150852+0000","last_change":"2024-05-28T14:09:31.760741+0000","last_active":"2024-05-28T14:09:33.150852+0000","last_peered":"2024-05-28T14:09:33.150852+0000","last_clean":"2024-05-28T14:09:33.150852+0000","last_became_active":"2024-05-28T14:09:31.760404+0000","last_became_peered":"2024-05-28T14:09:31.760404+0000","last_unstale":"2024-05-28T14:09:33.150852+0000","last_undegraded":"2024-05-28T14:09:33.150852+0000","last_fullsized":"2024-05-28T14:09:33.150852+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:24.707344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751996+0000","last_change":"2024-05-28T14:09:31.754702+0000","last_active":"2024-05-28T14:09:32.751996+0000","last_peered":"2024-05-28T14:09:32.751996+0000","last_clean":"2024-05-28T14:09:32.751996+0000","last_became_active":"2024-05-28T14:09:31.754215+0000","last_became_peered":"2024-05-28T14:09:31.754215+0000","last_unstale":"2024-05-28T14:09:32.751996+0000","last_undegraded":"2024-05-28T14:09:32.751996+0000","last_fullsized":"2024-05-28T14:09:32.751996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:36:07.311954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151011+0000","last_change":"2024-05-28T14:09:31.765289+0000","last_active":"2024-05-28T14:09:33.151011+0000","last_peered":"2024-05-28T14:09:33.151011+0000","last_clean":"2024-05-28T14:09:33.151011+0000","last_became_active":"2024-05-28T14:09:31.763980+0000","last_became_peered":"2024-05-28T14:09:31.763980+0000","last_unstale":"2024-05-28T14:09:33.151011+0000","last_undegraded":"2024-05-28T14:09:33.151011+0000","last_fullsized":"2024-05-28T14:09:33.151011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762598+0000","last_change":"2024-05-28T14:09:31.758017+0000","last_active":"2024-05-28T14:09:32.762598+0000","last_peered":"2024-05-28T14:09:32.762598+0000","last_clean":"2024-05-28T14:09:32.762598+0000","last_became_active":"2024-05-28T14:09:31.757674+0000","last_became_peered":"2024-05-28T14:09:31.757674+0000","last_unstale":"2024-05-28T14:09:32.762598+0000","last_undegraded":"2024-05-28T14:09:32.762598+0000","last_fullsized":"2024-05-28T14:09:32.762598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:46.760174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762477+0000","last_change":"2024-05-28T14:09:31.762403+0000","last_active":"2024-05-28T14:09:32.762477+0000","last_peered":"2024-05-28T14:09:32.762477+0000","last_clean":"2024-05-28T14:09:32.762477+0000","last_became_active":"2024-05-28T14:09:31.762048+0000","last_became_peered":"2024-05-28T14:09:31.762048+0000","last_unstale":"2024-05-28T14:09:32.762477+0000","last_undegraded":"2024-05-28T14:09:32.762477+0000","last_fullsized":"2024-05-28T14:09:32.762477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:56.072759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751599+0000","last_change":"2024-05-28T14:09:31.747630+0000","last_active":"2024-05-28T14:09:32.751599+0000","last_peered":"2024-05-28T14:09:32.751599+0000","last_clean":"2024-05-28T14:09:32.751599+0000","last_became_active":"2024-05-28T14:09:31.747183+0000","last_became_peered":"2024-05-28T14:09:31.747183+0000","last_unstale":"2024-05-28T14:09:32.751599+0000","last_undegraded":"2024-05-28T14:09:32.751599+0000","last_fullsized":"2024-05-28T14:09:32.751599+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:49:44.321180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151402+0000","last_change":"2024-05-28T14:09:31.768533+0000","last_active":"2024-05-28T14:09:33.151402+0000","last_peered":"2024-05-28T14:09:33.151402+0000","last_clean":"2024-05-28T14:09:33.151402+0000","last_became_active":"2024-05-28T14:09:31.768237+0000","last_became_peered":"2024-05-28T14:09:31.768237+0000","last_unstale":"2024-05-28T14:09:33.151402+0000","last_undegraded":"2024-05-28T14:09:33.151402+0000","last_fullsized":"2024-05-28T14:09:33.151402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752209+0000","last_change":"2024-05-28T14:09:31.752686+0000","last_active":"2024-05-28T14:09:32.752209+0000","last_peered":"2024-05-28T14:09:32.752209+0000","last_clean":"2024-05-28T14:09:32.752209+0000","last_became_active":"2024-05-28T14:09:31.752411+0000","last_became_peered":"2024-05-28T14:09:31.752411+0000","last_unstale":"2024-05-28T14:09:32.752209+0000","last_undegraded":"2024-05-28T14:09:32.752209+0000","last_fullsized":"2024-05-28T14:09:32.752209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151595+0000","last_change":"2024-05-28T14:09:30.759880+0000","last_active":"2024-05-28T14:09:33.151595+0000","last_peered":"2024-05-28T14:09:33.151595+0000","last_clean":"2024-05-28T14:09:33.151595+0000","last_became_active":"2024-05-28T14:09:30.759575+0000","last_became_peered":"2024-05-28T14:09:30.759575+0000","last_unstale":"2024-05-28T14:09:33.151595+0000","last_undegraded":"2024-05-28T14:09:33.151595+0000","last_fullsized":"2024-05-28T14:09:33.151595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:55.065487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151493+0000","last_change":"2024-05-28T14:09:31.752593+0000","last_active":"2024-05-28T14:09:33.151493+0000","last_peered":"2024-05-28T14:09:33.151493+0000","last_clean":"2024-05-28T14:09:33.151493+0000","last_became_active":"2024-05-28T14:09:31.752332+0000","last_became_peered":"2024-05-28T14:09:31.752332+0000","last_unstale":"2024-05-28T14:09:33.151493+0000","last_undegraded":"2024-05-28T14:09:33.151493+0000","last_fullsized":"2024-05-28T14:09:33.151493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752466+0000","last_change":"2024-05-28T14:09:30.755821+0000","last_active":"2024-05-28T14:09:32.752466+0000","last_peered":"2024-05-28T14:09:32.752466+0000","last_clean":"2024-05-28T14:09:32.752466+0000","last_became_active":"2024-05-28T14:09:30.755452+0000","last_became_peered":"2024-05-28T14:09:30.755452+0000","last_unstale":"2024-05-28T14:09:32.752466+0000","last_undegraded":"2024-05-28T14:09:32.752466+0000","last_fullsized":"2024-05-28T14:09:32.752466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:18.317793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151220+0000","last_change":"2024-05-28T14:09:31.765522+0000","last_active":"2024-05-28T14:09:33.151220+0000","last_peered":"2024-05-28T14:09:33.151220+0000","last_clean":"2024-05-28T14:09:33.151220+0000","last_became_active":"2024-05-28T14:09:31.764349+0000","last_became_peered":"2024-05-28T14:09:31.764349+0000","last_unstale":"2024-05-28T14:09:33.151220+0000","last_undegraded":"2024-05-28T14:09:33.151220+0000","last_fullsized":"2024-05-28T14:09:33.151220+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765049+0000","last_change":"2024-05-28T14:09:30.752545+0000","last_active":"2024-05-28T14:09:32.765049+0000","last_peered":"2024-05-28T14:09:32.765049+0000","last_clean":"2024-05-28T14:09:32.765049+0000","last_became_active":"2024-05-28T14:09:30.752219+0000","last_became_peered":"2024-05-28T14:09:30.752219+0000","last_unstale":"2024-05-28T14:09:32.765049+0000","last_undegraded":"2024-05-28T14:09:32.765049+0000","last_fullsized":"2024-05-28T14:09:32.765049+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:49:43.313909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151148+0000","last_change":"2024-05-28T14:09:31.765767+0000","last_active":"2024-05-28T14:09:33.151148+0000","last_peered":"2024-05-28T14:09:33.151148+0000","last_clean":"2024-05-28T14:09:33.151148+0000","last_became_active":"2024-05-28T14:09:31.765407+0000","last_became_peered":"2024-05-28T14:09:31.765407+0000","last_unstale":"2024-05-28T14:09:33.151148+0000","last_undegraded":"2024-05-28T14:09:33.151148+0000","last_fullsized":"2024-05-28T14:09:33.151148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:27:09.475191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764918+0000","last_change":"2024-05-28T14:09:30.754683+0000","last_active":"2024-05-28T14:09:32.764918+0000","last_peered":"2024-05-28T14:09:32.764918+0000","last_clean":"2024-05-28T14:09:32.764918+0000","last_became_active":"2024-05-28T14:09:30.754405+0000","last_became_peered":"2024-05-28T14:09:30.754405+0000","last_unstale":"2024-05-28T14:09:32.764918+0000","last_undegraded":"2024-05-28T14:09:32.764918+0000","last_fullsized":"2024-05-28T14:09:32.764918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:29:45.272413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762565+0000","last_change":"2024-05-28T14:09:31.762430+0000","last_active":"2024-05-28T14:09:32.762565+0000","last_peered":"2024-05-28T14:09:32.762565+0000","last_clean":"2024-05-28T14:09:32.762565+0000","last_became_active":"2024-05-28T14:09:31.762101+0000","last_became_peered":"2024-05-28T14:09:31.762101+0000","last_unstale":"2024-05-28T14:09:32.762565+0000","last_undegraded":"2024-05-28T14:09:32.762565+0000","last_fullsized":"2024-05-28T14:09:32.762565+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:40:39.401890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752066+0000","last_change":"2024-05-28T14:09:30.747550+0000","last_active":"2024-05-28T14:09:32.752066+0000","last_peered":"2024-05-28T14:09:32.752066+0000","last_clean":"2024-05-28T14:09:32.752066+0000","last_became_active":"2024-05-28T14:09:30.747222+0000","last_became_peered":"2024-05-28T14:09:30.747222+0000","last_unstale":"2024-05-28T14:09:32.752066+0000","last_undegraded":"2024-05-28T14:09:32.752066+0000","last_fullsized":"2024-05-28T14:09:32.752066+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:58.290350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751952+0000","last_change":"2024-05-28T14:09:31.759538+0000","last_active":"2024-05-28T14:09:32.751952+0000","last_peered":"2024-05-28T14:09:32.751952+0000","last_clean":"2024-05-28T14:09:32.751952+0000","last_became_active":"2024-05-28T14:09:31.759258+0000","last_became_peered":"2024-05-28T14:09:31.759258+0000","last_unstale":"2024-05-28T14:09:32.751952+0000","last_undegraded":"2024-05-28T14:09:32.751952+0000","last_fullsized":"2024-05-28T14:09:32.751952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:35.172304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751683+0000","last_change":"2024-05-28T14:09:30.759103+0000","last_active":"2024-05-28T14:09:32.751683+0000","last_peered":"2024-05-28T14:09:32.751683+0000","last_clean":"2024-05-28T14:09:32.751683+0000","last_became_active":"2024-05-28T14:09:30.758628+0000","last_became_peered":"2024-05-28T14:09:30.758628+0000","last_unstale":"2024-05-28T14:09:32.751683+0000","last_undegraded":"2024-05-28T14:09:32.751683+0000","last_fullsized":"2024-05-28T14:09:32.751683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:54.698345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150467+0000","last_change":"2024-05-28T14:09:31.761382+0000","last_active":"2024-05-28T14:09:33.150467+0000","last_peered":"2024-05-28T14:09:33.150467+0000","last_clean":"2024-05-28T14:09:33.150467+0000","last_became_active":"2024-05-28T14:09:31.761095+0000","last_became_peered":"2024-05-28T14:09:31.761095+0000","last_unstale":"2024-05-28T14:09:33.150467+0000","last_undegraded":"2024-05-28T14:09:33.150467+0000","last_fullsized":"2024-05-28T14:09:33.150467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:31:10.786309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764699+0000","last_change":"2024-05-28T14:09:30.762673+0000","last_active":"2024-05-28T14:09:32.764699+0000","last_peered":"2024-05-28T14:09:32.764699+0000","last_clean":"2024-05-28T14:09:32.764699+0000","last_became_active":"2024-05-28T14:09:30.761793+0000","last_became_peered":"2024-05-28T14:09:30.761793+0000","last_unstale":"2024-05-28T14:09:32.764699+0000","last_undegraded":"2024-05-28T14:09:32.764699+0000","last_fullsized":"2024-05-28T14:09:32.764699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:40:25.005440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747896+0000","last_change":"2024-05-28T14:09:31.766958+0000","last_active":"2024-05-28T14:09:32.747896+0000","last_peered":"2024-05-28T14:09:32.747896+0000","last_clean":"2024-05-28T14:09:32.747896+0000","last_became_active":"2024-05-28T14:09:31.766301+0000","last_became_peered":"2024-05-28T14:09:31.766301+0000","last_unstale":"2024-05-28T14:09:32.747896+0000","last_undegraded":"2024-05-28T14:09:32.747896+0000","last_fullsized":"2024-05-28T14:09:32.747896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:24.707344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762264+0000","last_change":"2024-05-28T14:09:30.746473+0000","last_active":"2024-05-28T14:09:32.762264+0000","last_peered":"2024-05-28T14:09:32.762264+0000","last_clean":"2024-05-28T14:09:32.762264+0000","last_became_active":"2024-05-28T14:09:30.746177+0000","last_became_peered":"2024-05-28T14:09:30.746177+0000","last_unstale":"2024-05-28T14:09:32.762264+0000","last_undegraded":"2024-05-28T14:09:32.762264+0000","last_fullsized":"2024-05-28T14:09:32.762264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:41.356836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.748332+0000","last_change":"2024-05-28T14:09:31.756713+0000","last_active":"2024-05-28T14:09:32.748332+0000","last_peered":"2024-05-28T14:09:32.748332+0000","last_clean":"2024-05-28T14:09:32.748332+0000","last_became_active":"2024-05-28T14:09:31.755716+0000","last_became_peered":"2024-05-28T14:09:31.755716+0000","last_unstale":"2024-05-28T14:09:32.748332+0000","last_undegraded":"2024-05-28T14:09:32.748332+0000","last_fullsized":"2024-05-28T14:09:32.748332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:02:06.074104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151668+0000","last_change":"2024-05-28T14:09:30.758732+0000","last_active":"2024-05-28T14:09:33.151668+0000","last_peered":"2024-05-28T14:09:33.151668+0000","last_clean":"2024-05-28T14:09:33.151668+0000","last_became_active":"2024-05-28T14:09:30.758400+0000","last_became_peered":"2024-05-28T14:09:30.758400+0000","last_unstale":"2024-05-28T14:09:33.151668+0000","last_undegraded":"2024-05-28T14:09:33.151668+0000","last_fullsized":"2024-05-28T14:09:33.151668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:45.752902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750903+0000","last_change":"2024-05-28T14:09:31.762311+0000","last_active":"2024-05-28T14:09:32.750903+0000","last_peered":"2024-05-28T14:09:32.750903+0000","last_clean":"2024-05-28T14:09:32.750903+0000","last_became_active":"2024-05-28T14:09:31.761990+0000","last_became_peered":"2024-05-28T14:09:31.761990+0000","last_unstale":"2024-05-28T14:09:32.750903+0000","last_undegraded":"2024-05-28T14:09:32.750903+0000","last_fullsized":"2024-05-28T14:09:32.750903+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151408+0000","last_change":"2024-05-28T14:09:30.757972+0000","last_active":"2024-05-28T14:09:33.151408+0000","last_peered":"2024-05-28T14:09:33.151408+0000","last_clean":"2024-05-28T14:09:33.151408+0000","last_became_active":"2024-05-28T14:09:30.757716+0000","last_became_peered":"2024-05-28T14:09:30.757716+0000","last_unstale":"2024-05-28T14:09:33.151408+0000","last_undegraded":"2024-05-28T14:09:33.151408+0000","last_fullsized":"2024-05-28T14:09:33.151408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:40:38.394618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764978+0000","last_change":"2024-05-28T14:09:31.754712+0000","last_active":"2024-05-28T14:09:32.764978+0000","last_peered":"2024-05-28T14:09:32.764978+0000","last_clean":"2024-05-28T14:09:32.764978+0000","last_became_active":"2024-05-28T14:09:31.754427+0000","last_became_peered":"2024-05-28T14:09:31.754427+0000","last_unstale":"2024-05-28T14:09:32.764978+0000","last_undegraded":"2024-05-28T14:09:32.764978+0000","last_fullsized":"2024-05-28T14:09:32.764978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151098+0000","last_change":"2024-05-28T14:09:30.756257+0000","last_active":"2024-05-28T14:09:33.151098+0000","last_peered":"2024-05-28T14:09:33.151098+0000","last_clean":"2024-05-28T14:09:33.151098+0000","last_became_active":"2024-05-28T14:09:30.755923+0000","last_became_peered":"2024-05-28T14:09:30.755923+0000","last_unstale":"2024-05-28T14:09:33.151098+0000","last_undegraded":"2024-05-28T14:09:33.151098+0000","last_fullsized":"2024-05-28T14:09:33.151098+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:54.698345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752090+0000","last_change":"2024-05-28T14:09:31.748217+0000","last_active":"2024-05-28T14:09:32.752090+0000","last_peered":"2024-05-28T14:09:32.752090+0000","last_clean":"2024-05-28T14:09:32.752090+0000","last_became_active":"2024-05-28T14:09:31.747336+0000","last_became_peered":"2024-05-28T14:09:31.747336+0000","last_unstale":"2024-05-28T14:09:32.752090+0000","last_undegraded":"2024-05-28T14:09:32.752090+0000","last_fullsized":"2024-05-28T14:09:32.752090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"89'6","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750773+0000","last_change":"2024-05-28T14:09:30.760654+0000","last_active":"2024-05-28T14:09:32.750773+0000","last_peered":"2024-05-28T14:09:32.750773+0000","last_clean":"2024-05-28T14:09:32.750773+0000","last_became_active":"2024-05-28T14:09:30.760328+0000","last_became_peered":"2024-05-28T14:09:30.760328+0000","last_unstale":"2024-05-28T14:09:32.750773+0000","last_undegraded":"2024-05-28T14:09:32.750773+0000","last_fullsized":"2024-05-28T14:09:32.750773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:54:12.659735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764801+0000","last_change":"2024-05-28T14:09:31.742984+0000","last_active":"2024-05-28T14:09:32.764801+0000","last_peered":"2024-05-28T14:09:32.764801+0000","last_clean":"2024-05-28T14:09:32.764801+0000","last_became_active":"2024-05-28T14:09:31.742692+0000","last_became_peered":"2024-05-28T14:09:31.742692+0000","last_unstale":"2024-05-28T14:09:32.764801+0000","last_undegraded":"2024-05-28T14:09:32.764801+0000","last_fullsized":"2024-05-28T14:09:32.764801+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762506+0000","last_change":"2024-05-28T14:09:30.751922+0000","last_active":"2024-05-28T14:09:32.762506+0000","last_peered":"2024-05-28T14:09:32.762506+0000","last_clean":"2024-05-28T14:09:32.762506+0000","last_became_active":"2024-05-28T14:09:30.751646+0000","last_became_peered":"2024-05-28T14:09:30.751646+0000","last_unstale":"2024-05-28T14:09:32.762506+0000","last_undegraded":"2024-05-28T14:09:32.762506+0000","last_fullsized":"2024-05-28T14:09:32.762506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:24:14.215364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150460+0000","last_change":"2024-05-28T14:09:31.763288+0000","last_active":"2024-05-28T14:09:33.150460+0000","last_peered":"2024-05-28T14:09:33.150460+0000","last_clean":"2024-05-28T14:09:33.150460+0000","last_became_active":"2024-05-28T14:09:31.762800+0000","last_became_peered":"2024-05-28T14:09:31.762800+0000","last_unstale":"2024-05-28T14:09:33.150460+0000","last_undegraded":"2024-05-28T14:09:33.150460+0000","last_fullsized":"2024-05-28T14:09:33.150460+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:46:19.764662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747904+0000","last_change":"2024-05-28T14:09:30.753559+0000","last_active":"2024-05-28T14:09:32.747904+0000","last_peered":"2024-05-28T14:09:32.747904+0000","last_clean":"2024-05-28T14:09:32.747904+0000","last_became_active":"2024-05-28T14:09:30.753253+0000","last_became_peered":"2024-05-28T14:09:30.753253+0000","last_unstale":"2024-05-28T14:09:32.747904+0000","last_undegraded":"2024-05-28T14:09:32.747904+0000","last_fullsized":"2024-05-28T14:09:32.747904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:15:03.984981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762339+0000","last_change":"2024-05-28T14:09:31.765598+0000","last_active":"2024-05-28T14:09:32.762339+0000","last_peered":"2024-05-28T14:09:32.762339+0000","last_clean":"2024-05-28T14:09:32.762339+0000","last_became_active":"2024-05-28T14:09:31.760967+0000","last_became_peered":"2024-05-28T14:09:31.760967+0000","last_unstale":"2024-05-28T14:09:32.762339+0000","last_undegraded":"2024-05-28T14:09:32.762339+0000","last_fullsized":"2024-05-28T14:09:32.762339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:24:40.914669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750623+0000","last_change":"2024-05-28T14:09:30.758094+0000","last_active":"2024-05-28T14:09:32.750623+0000","last_peered":"2024-05-28T14:09:32.750623+0000","last_clean":"2024-05-28T14:09:32.750623+0000","last_became_active":"2024-05-28T14:09:30.757801+0000","last_became_peered":"2024-05-28T14:09:30.757801+0000","last_unstale":"2024-05-28T14:09:32.750623+0000","last_undegraded":"2024-05-28T14:09:32.750623+0000","last_fullsized":"2024-05-28T14:09:32.750623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:40:25.005440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762291+0000","last_change":"2024-05-28T14:09:31.754734+0000","last_active":"2024-05-28T14:09:32.762291+0000","last_peered":"2024-05-28T14:09:32.762291+0000","last_clean":"2024-05-28T14:09:32.762291+0000","last_became_active":"2024-05-28T14:09:31.754459+0000","last_became_peered":"2024-05-28T14:09:31.754459+0000","last_unstale":"2024-05-28T14:09:32.762291+0000","last_undegraded":"2024-05-28T14:09:32.762291+0000","last_fullsized":"2024-05-28T14:09:32.762291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:19:30.516845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"89'8","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764575+0000","last_change":"2024-05-28T14:09:30.759773+0000","last_active":"2024-05-28T14:09:32.764575+0000","last_peered":"2024-05-28T14:09:32.764575+0000","last_clean":"2024-05-28T14:09:32.764575+0000","last_became_active":"2024-05-28T14:09:30.759218+0000","last_became_peered":"2024-05-28T14:09:30.759218+0000","last_unstale":"2024-05-28T14:09:32.764575+0000","last_undegraded":"2024-05-28T14:09:32.764575+0000","last_fullsized":"2024-05-28T14:09:32.764575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:14:14.485471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151031+0000","last_change":"2024-05-28T14:09:31.766044+0000","last_active":"2024-05-28T14:09:33.151031+0000","last_peered":"2024-05-28T14:09:33.151031+0000","last_clean":"2024-05-28T14:09:33.151031+0000","last_became_active":"2024-05-28T14:09:31.763364+0000","last_became_peered":"2024-05-28T14:09:31.763364+0000","last_unstale":"2024-05-28T14:09:33.151031+0000","last_undegraded":"2024-05-28T14:09:33.151031+0000","last_fullsized":"2024-05-28T14:09:33.151031+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:54:38.549065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151738+0000","last_change":"2024-05-28T14:09:30.763452+0000","last_active":"2024-05-28T14:09:33.151738+0000","last_peered":"2024-05-28T14:09:33.151738+0000","last_clean":"2024-05-28T14:09:33.151738+0000","last_became_active":"2024-05-28T14:09:30.763075+0000","last_became_peered":"2024-05-28T14:09:30.763075+0000","last_unstale":"2024-05-28T14:09:33.151738+0000","last_undegraded":"2024-05-28T14:09:33.151738+0000","last_fullsized":"2024-05-28T14:09:33.151738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:37.606903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151668+0000","last_change":"2024-05-28T14:09:31.769821+0000","last_active":"2024-05-28T14:09:33.151668+0000","last_peered":"2024-05-28T14:09:33.151668+0000","last_clean":"2024-05-28T14:09:33.151668+0000","last_became_active":"2024-05-28T14:09:31.769534+0000","last_became_peered":"2024-05-28T14:09:31.769534+0000","last_unstale":"2024-05-28T14:09:33.151668+0000","last_undegraded":"2024-05-28T14:09:33.151668+0000","last_fullsized":"2024-05-28T14:09:33.151668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765169+0000","last_change":"2024-05-28T14:09:30.752703+0000","last_active":"2024-05-28T14:09:32.765169+0000","last_peered":"2024-05-28T14:09:32.765169+0000","last_clean":"2024-05-28T14:09:32.765169+0000","last_became_active":"2024-05-28T14:09:30.752399+0000","last_became_peered":"2024-05-28T14:09:30.752399+0000","last_unstale":"2024-05-28T14:09:32.765169+0000","last_undegraded":"2024-05-28T14:09:32.765169+0000","last_fullsized":"2024-05-28T14:09:32.765169+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:42.742326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752236+0000","last_change":"2024-05-28T14:09:31.757626+0000","last_active":"2024-05-28T14:09:32.752236+0000","last_peered":"2024-05-28T14:09:32.752236+0000","last_clean":"2024-05-28T14:09:32.752236+0000","last_became_active":"2024-05-28T14:09:31.756947+0000","last_became_peered":"2024-05-28T14:09:31.756947+0000","last_unstale":"2024-05-28T14:09:32.752236+0000","last_undegraded":"2024-05-28T14:09:32.752236+0000","last_fullsized":"2024-05-28T14:09:32.752236+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765042+0000","last_change":"2024-05-28T14:09:30.760474+0000","last_active":"2024-05-28T14:09:32.765042+0000","last_peered":"2024-05-28T14:09:32.765042+0000","last_clean":"2024-05-28T14:09:32.765042+0000","last_became_active":"2024-05-28T14:09:30.759783+0000","last_became_peered":"2024-05-28T14:09:30.759783+0000","last_unstale":"2024-05-28T14:09:32.765042+0000","last_undegraded":"2024-05-28T14:09:32.765042+0000","last_fullsized":"2024-05-28T14:09:32.765042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:49:41.271344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.748033+0000","last_change":"2024-05-28T14:09:30.754286+0000","last_active":"2024-05-28T14:09:32.748033+0000","last_peered":"2024-05-28T14:09:32.748033+0000","last_clean":"2024-05-28T14:09:32.748033+0000","last_became_active":"2024-05-28T14:09:30.753997+0000","last_became_peered":"2024-05-28T14:09:30.753997+0000","last_unstale":"2024-05-28T14:09:32.748033+0000","last_undegraded":"2024-05-28T14:09:32.748033+0000","last_fullsized":"2024-05-28T14:09:32.748033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:34:27.167006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751282+0000","last_change":"2024-05-28T14:09:31.760545+0000","last_active":"2024-05-28T14:09:32.751282+0000","last_peered":"2024-05-28T14:09:32.751282+0000","last_clean":"2024-05-28T14:09:32.751282+0000","last_became_active":"2024-05-28T14:09:31.760024+0000","last_became_peered":"2024-05-28T14:09:31.760024+0000","last_unstale":"2024-05-28T14:09:32.751282+0000","last_undegraded":"2024-05-28T14:09:32.751282+0000","last_fullsized":"2024-05-28T14:09:32.751282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-05-28T14:09:29.732310+0000","last_change":"2024-05-28T14:09:29.732310+0000","last_active":"2024-05-28T14:09:29.717625+0000","last_peered":"2024-05-28T14:09:29.717625+0000","last_clean":"2024-05-28T14:09:29.717625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:29.732310+0000","last_undegraded":"2024-05-28T14:09:29.732310+0000","last_fullsized":"2024-05-28T14:09:29.732310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-05-28T14:09:29.731567+0000","last_change":"2024-05-28T14:09:29.731567+0000","last_active":"2024-05-28T14:09:29.717625+0000","last_peered":"2024-05-28T14:09:29.717625+0000","last_clean":"2024-05-28T14:09:29.717625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:29.731567+0000","last_undegraded":"2024-05-28T14:09:29.731567+0000","last_fullsized":"2024-05-28T14:09:29.731567+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751081+0000","last_change":"2024-05-28T14:09:31.748847+0000","last_active":"2024-05-28T14:09:32.751081+0000","last_peered":"2024-05-28T14:09:32.751081+0000","last_clean":"2024-05-28T14:09:32.751081+0000","last_became_active":"2024-05-28T14:09:31.748479+0000","last_became_peered":"2024-05-28T14:09:31.748479+0000","last_unstale":"2024-05-28T14:09:32.751081+0000","last_undegraded":"2024-05-28T14:09:32.751081+0000","last_fullsized":"2024-05-28T14:09:32.751081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747718+0000","last_change":"2024-05-28T14:09:30.755244+0000","last_active":"2024-05-28T14:09:32.747718+0000","last_peered":"2024-05-28T14:09:32.747718+0000","last_clean":"2024-05-28T14:09:32.747718+0000","last_became_active":"2024-05-28T14:09:30.754984+0000","last_became_peered":"2024-05-28T14:09:30.754984+0000","last_unstale":"2024-05-28T14:09:32.747718+0000","last_undegraded":"2024-05-28T14:09:32.747718+0000","last_fullsized":"2024-05-28T14:09:32.747718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:14:58.988961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752367+0000","last_change":"2024-05-28T14:09:31.745138+0000","last_active":"2024-05-28T14:09:32.752367+0000","last_peered":"2024-05-28T14:09:32.752367+0000","last_clean":"2024-05-28T14:09:32.752367+0000","last_became_active":"2024-05-28T14:09:31.744819+0000","last_became_peered":"2024-05-28T14:09:31.744819+0000","last_unstale":"2024-05-28T14:09:32.752367+0000","last_undegraded":"2024-05-28T14:09:32.752367+0000","last_fullsized":"2024-05-28T14:09:32.752367+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762790+0000","last_change":"2024-05-28T14:09:30.760787+0000","last_active":"2024-05-28T14:09:32.762790+0000","last_peered":"2024-05-28T14:09:32.762790+0000","last_clean":"2024-05-28T14:09:32.762790+0000","last_became_active":"2024-05-28T14:09:30.760502+0000","last_became_peered":"2024-05-28T14:09:30.760502+0000","last_unstale":"2024-05-28T14:09:32.762790+0000","last_undegraded":"2024-05-28T14:09:32.762790+0000","last_fullsized":"2024-05-28T14:09:32.762790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:31:35.886734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150913+0000","last_change":"2024-05-28T14:09:31.771112+0000","last_active":"2024-05-28T14:09:33.150913+0000","last_peered":"2024-05-28T14:09:33.150913+0000","last_clean":"2024-05-28T14:09:33.150913+0000","last_became_active":"2024-05-28T14:09:31.770660+0000","last_became_peered":"2024-05-28T14:09:31.770660+0000","last_unstale":"2024-05-28T14:09:33.150913+0000","last_undegraded":"2024-05-28T14:09:33.150913+0000","last_fullsized":"2024-05-28T14:09:33.150913+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:02:06.074104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762843+0000","last_change":"2024-05-28T14:09:30.751801+0000","last_active":"2024-05-28T14:09:32.762843+0000","last_peered":"2024-05-28T14:09:32.762843+0000","last_clean":"2024-05-28T14:09:32.762843+0000","last_became_active":"2024-05-28T14:09:30.751504+0000","last_became_peered":"2024-05-28T14:09:30.751504+0000","last_unstale":"2024-05-28T14:09:32.762843+0000","last_undegraded":"2024-05-28T14:09:32.762843+0000","last_fullsized":"2024-05-28T14:09:32.762843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:42.042818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151673+0000","last_change":"2024-05-28T14:09:31.765811+0000","last_active":"2024-05-28T14:09:33.151673+0000","last_peered":"2024-05-28T14:09:33.151673+0000","last_clean":"2024-05-28T14:09:33.151673+0000","last_became_active":"2024-05-28T14:09:31.764710+0000","last_became_peered":"2024-05-28T14:09:31.764710+0000","last_unstale":"2024-05-28T14:09:33.151673+0000","last_undegraded":"2024-05-28T14:09:33.151673+0000","last_fullsized":"2024-05-28T14:09:33.151673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:57.967039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151377+0000","last_change":"2024-05-28T14:09:30.760473+0000","last_active":"2024-05-28T14:09:33.151377+0000","last_peered":"2024-05-28T14:09:33.151377+0000","last_clean":"2024-05-28T14:09:33.151377+0000","last_became_active":"2024-05-28T14:09:30.760129+0000","last_became_peered":"2024-05-28T14:09:30.760129+0000","last_unstale":"2024-05-28T14:09:33.151377+0000","last_undegraded":"2024-05-28T14:09:33.151377+0000","last_fullsized":"2024-05-28T14:09:33.151377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:18.317793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752265+0000","last_change":"2024-05-28T14:09:31.748833+0000","last_active":"2024-05-28T14:09:32.752265+0000","last_peered":"2024-05-28T14:09:32.752265+0000","last_clean":"2024-05-28T14:09:32.752265+0000","last_became_active":"2024-05-28T14:09:31.747731+0000","last_became_peered":"2024-05-28T14:09:31.747731+0000","last_unstale":"2024-05-28T14:09:32.752265+0000","last_undegraded":"2024-05-28T14:09:32.752265+0000","last_fullsized":"2024-05-28T14:09:32.752265+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151485+0000","last_change":"2024-05-28T14:09:30.758617+0000","last_active":"2024-05-28T14:09:33.151485+0000","last_peered":"2024-05-28T14:09:33.151485+0000","last_clean":"2024-05-28T14:09:33.151485+0000","last_became_active":"2024-05-28T14:09:30.758317+0000","last_became_peered":"2024-05-28T14:09:30.758317+0000","last_unstale":"2024-05-28T14:09:33.151485+0000","last_undegraded":"2024-05-28T14:09:33.151485+0000","last_fullsized":"2024-05-28T14:09:33.151485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:10.090547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751157+0000","last_change":"2024-05-28T14:09:31.760268+0000","last_active":"2024-05-28T14:09:32.751157+0000","last_peered":"2024-05-28T14:09:32.751157+0000","last_clean":"2024-05-28T14:09:32.751157+0000","last_became_active":"2024-05-28T14:09:31.759889+0000","last_became_peered":"2024-05-28T14:09:31.759889+0000","last_unstale":"2024-05-28T14:09:32.751157+0000","last_undegraded":"2024-05-28T14:09:32.751157+0000","last_fullsized":"2024-05-28T14:09:32.751157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764892+0000","last_change":"2024-05-28T14:09:30.754916+0000","last_active":"2024-05-28T14:09:32.764892+0000","last_peered":"2024-05-28T14:09:32.764892+0000","last_clean":"2024-05-28T14:09:32.764892+0000","last_became_active":"2024-05-28T14:09:30.754019+0000","last_became_peered":"2024-05-28T14:09:30.754019+0000","last_unstale":"2024-05-28T14:09:32.764892+0000","last_undegraded":"2024-05-28T14:09:32.764892+0000","last_fullsized":"2024-05-28T14:09:32.764892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:34.165033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764999+0000","last_change":"2024-05-28T14:09:31.748964+0000","last_active":"2024-05-28T14:09:32.764999+0000","last_peered":"2024-05-28T14:09:32.764999+0000","last_clean":"2024-05-28T14:09:32.764999+0000","last_became_active":"2024-05-28T14:09:31.748393+0000","last_became_peered":"2024-05-28T14:09:31.748393+0000","last_unstale":"2024-05-28T14:09:32.764999+0000","last_undegraded":"2024-05-28T14:09:32.764999+0000","last_fullsized":"2024-05-28T14:09:32.764999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751724+0000","last_change":"2024-05-28T14:09:30.748735+0000","last_active":"2024-05-28T14:09:32.751724+0000","last_peered":"2024-05-28T14:09:32.751724+0000","last_clean":"2024-05-28T14:09:32.751724+0000","last_became_active":"2024-05-28T14:09:30.748404+0000","last_became_peered":"2024-05-28T14:09:30.748404+0000","last_unstale":"2024-05-28T14:09:32.751724+0000","last_undegraded":"2024-05-28T14:09:32.751724+0000","last_fullsized":"2024-05-28T14:09:32.751724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:37.606903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150429+0000","last_change":"2024-05-28T14:09:31.764716+0000","last_active":"2024-05-28T14:09:33.150429+0000","last_peered":"2024-05-28T14:09:33.150429+0000","last_clean":"2024-05-28T14:09:33.150429+0000","last_became_active":"2024-05-28T14:09:31.764407+0000","last_became_peered":"2024-05-28T14:09:31.764407+0000","last_unstale":"2024-05-28T14:09:33.150429+0000","last_undegraded":"2024-05-28T14:09:33.150429+0000","last_fullsized":"2024-05-28T14:09:33.150429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:14:59.996233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151098+0000","last_change":"2024-05-28T14:09:30.755558+0000","last_active":"2024-05-28T14:09:33.151098+0000","last_peered":"2024-05-28T14:09:33.151098+0000","last_clean":"2024-05-28T14:09:33.151098+0000","last_became_active":"2024-05-28T14:09:30.755257+0000","last_became_peered":"2024-05-28T14:09:30.755257+0000","last_unstale":"2024-05-28T14:09:33.151098+0000","last_undegraded":"2024-05-28T14:09:33.151098+0000","last_fullsized":"2024-05-28T14:09:33.151098+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:58.290350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150859+0000","last_change":"2024-05-28T14:09:31.758209+0000","last_active":"2024-05-28T14:09:33.150859+0000","last_peered":"2024-05-28T14:09:33.150859+0000","last_clean":"2024-05-28T14:09:33.150859+0000","last_became_active":"2024-05-28T14:09:31.757910+0000","last_became_peered":"2024-05-28T14:09:31.757910+0000","last_unstale":"2024-05-28T14:09:33.150859+0000","last_undegraded":"2024-05-28T14:09:33.150859+0000","last_fullsized":"2024-05-28T14:09:33.150859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:36:07.311954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751556+0000","last_change":"2024-05-28T14:09:30.754615+0000","last_active":"2024-05-28T14:09:32.751556+0000","last_peered":"2024-05-28T14:09:32.751556+0000","last_clean":"2024-05-28T14:09:32.751556+0000","last_became_active":"2024-05-28T14:09:30.754325+0000","last_became_peered":"2024-05-28T14:09:30.754325+0000","last_unstale":"2024-05-28T14:09:32.751556+0000","last_undegraded":"2024-05-28T14:09:32.751556+0000","last_fullsized":"2024-05-28T14:09:32.751556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:45.752902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762719+0000","last_change":"2024-05-28T14:09:31.760657+0000","last_active":"2024-05-28T14:09:32.762719+0000","last_peered":"2024-05-28T14:09:32.762719+0000","last_clean":"2024-05-28T14:09:32.762719+0000","last_became_active":"2024-05-28T14:09:31.759615+0000","last_became_peered":"2024-05-28T14:09:31.759615+0000","last_unstale":"2024-05-28T14:09:32.762719+0000","last_undegraded":"2024-05-28T14:09:32.762719+0000","last_fullsized":"2024-05-28T14:09:32.762719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:55.705617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151176+0000","last_change":"2024-05-28T14:09:30.762060+0000","last_active":"2024-05-28T14:09:33.151176+0000","last_peered":"2024-05-28T14:09:33.151176+0000","last_clean":"2024-05-28T14:09:33.151176+0000","last_became_active":"2024-05-28T14:09:30.761778+0000","last_became_peered":"2024-05-28T14:09:30.761778+0000","last_unstale":"2024-05-28T14:09:33.151176+0000","last_undegraded":"2024-05-28T14:09:33.151176+0000","last_fullsized":"2024-05-28T14:09:33.151176+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:56.959767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":300,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150107+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:09:33.150107+0000","last_peered":"2024-05-28T14:09:33.150107+0000","last_clean":"2024-05-28T14:09:33.150107+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:09:33.150107+0000","last_undegraded":"2024-05-28T14:09:33.150107+0000","last_fullsized":"2024-05-28T14:09:33.150107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":7},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123028,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248812,"statfs":{"total":96636764160,"available":96510783488,"internally_reserved":0,"allocated":1703936,"data_stored":1068711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122448,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249392,"statfs":{"total":96636764160,"available":96511377408,"internally_reserved":0,"allocated":1118208,"data_stored":478967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":124254662},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76628,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295212,"statfs":{"total":96636764160,"available":96558297088,"internally_reserved":0,"allocated":1122304,"data_stored":479907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82184,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289656,"statfs":{"total":96636764160,"available":96552607744,"internally_reserved":0,"allocated":1105920,"data_stored":478273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574987,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130772,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241068,"statfs":{"total":96636764160,"available":96502853632,"internally_reserved":0,"allocated":1118208,"data_stored":479885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":132774344},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144604,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227236,"statfs":{"total":96636764160,"available":96488689664,"internally_reserved":0,"allocated":1056768,"data_stored":452994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108564,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94263276,"statfs":{"total":96636764160,"available":96525594624,"internally_reserved":0,"allocated":1118208,"data_stored":478467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64724,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307116,"statfs":{"total":96636764160,"available":96570486784,"internally_reserved":0,"allocated":1703936,"data_stored":1068675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:09:36.253 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:09:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806fc00640 1 Processor -- start 2024-05-28T14:09:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806fc00640 1 -- start start 2024-05-28T14:09:36.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806fc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f80680ce860 0x7f806806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806fc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8068162c80 0x7f806815e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806fc00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80680d0de0 con 0x7f80680d0900 2024-05-28T14:09:36.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806fc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f80680d19f0 con 0x7f80680ce860 2024-05-28T14:09:36.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806fc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80680d10c0 con 0x7f8068162c80 2024-05-28T14:09:36.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806d200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f80680ce860 0x7f806806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806d200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f80680ce860 0x7f806806d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56422/0 (socket says 172.21.15.42:56422) 2024-05-28T14:09:36.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f8067e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f80680d0900 0x7f8068162570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47324/0 (socket says 172.21.15.42:47324) 2024-05-28T14:09:36.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806d200640 1 -- 172.21.15.42:0/339498340 learned_addr learned my addr 172.21.15.42:0/339498340 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:36.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806dc00640 1 --2- 172.21.15.42:0/339498340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8068162c80 0x7f806815e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806d200640 1 -- 172.21.15.42:0/339498340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8068162c80 msgr2=0x7f806815e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:36.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806d200640 1 --2- 172.21.15.42:0/339498340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8068162c80 0x7f806815e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806d200640 1 -- 172.21.15.42:0/339498340 >> v1:172.21.15.42:6789/0 conn(0x7f80680d0900 legacy=0x7f8068162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f806d200640 1 -- 172.21.15.42:0/339498340 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80680d13a0 con 0x7f80680ce860 2024-05-28T14:09:36.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.449+0000 7f8067400640 1 -- 172.21.15.42:0/339498340 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4152521081 0 0) 0x7f80680d0de0 con 0x7f80680d0900 2024-05-28T14:09:36.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.453+0000 7f806d200640 1 --2- 172.21.15.42:0/339498340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80680ce860 0x7f806806d7b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f805c009870 tx=0x7f805c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62625b80c56c6ce server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:36.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.453+0000 7f8067400640 1 -- 172.21.15.42:0/339498340 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c0aa620 con 0x7f80680ce860 2024-05-28T14:09:36.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.453+0000 7f8067400640 1 -- 172.21.15.42:0/339498340 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f805c0af070 con 0x7f80680ce860 2024-05-28T14:09:36.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.453+0000 7f8067400640 1 -- 172.21.15.42:0/339498340 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c0aa9e0 con 0x7f80680ce860 2024-05-28T14:09:36.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.453+0000 7f806fc00640 1 -- 172.21.15.42:0/339498340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80680ce860 msgr2=0x7f806806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.453+0000 7f806fc00640 1 --2- 172.21.15.42:0/339498340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80680ce860 0x7f806806d7b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f805c009870 tx=0x7f805c01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.453+0000 7f806fc00640 1 -- 172.21.15.42:0/339498340 shutdown_connections 2024-05-28T14:09:36.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.453+0000 7f806fc00640 1 --2- 172.21.15.42:0/339498340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8068162c80 0x7f806815e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.453+0000 7f806fc00640 1 --2- 172.21.15.42:0/339498340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80680ce860 0x7f806806d7b0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.453+0000 7f806fc00640 1 -- 172.21.15.42:0/339498340 >> 172.21.15.42:0/339498340 conn(0x7f80680b4080 msgr2=0x7f80680b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:36.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.453+0000 7f806fc00640 1 -- 172.21.15.42:0/339498340 shutdown_connections 2024-05-28T14:09:36.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.453+0000 7f806fc00640 1 -- 172.21.15.42:0/339498340 wait complete. 2024-05-28T14:09:36.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.453+0000 7f806fc00640 1 Processor -- start 2024-05-28T14:09:36.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.453+0000 7f806fc00640 1 -- start start 2024-05-28T14:09:36.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f806fc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80680ce860 0x7f80680b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f806fc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80680d0900 0x7f8068151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f806fc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8068162c80 0x7f8068157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f806fc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f80680ccf70 con 0x7f80680d0900 2024-05-28T14:09:36.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f806fc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80680ccdf0 con 0x7f80680ce860 2024-05-28T14:09:36.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f806fc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80680cd0f0 con 0x7f8068162c80 2024-05-28T14:09:36.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f8067e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80680d0900 0x7f8068151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f8067e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80680d0900 0x7f8068151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56436/0 (socket says 172.21.15.42:56436) 2024-05-28T14:09:36.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f8067e00640 1 -- 172.21.15.42:0/2151573382 learned_addr learned my addr 172.21.15.42:0/2151573382 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:36.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f806d200640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80680ce860 0x7f80680b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f806dc00640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8068162c80 0x7f8068157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f8067e00640 1 -- 172.21.15.42:0/2151573382 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8068162c80 msgr2=0x7f8068157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f8067e00640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8068162c80 0x7f8068157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f8067e00640 1 -- 172.21.15.42:0/2151573382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80680ce860 msgr2=0x7f80680b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f8067e00640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80680ce860 0x7f80680b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f8067e00640 1 -- 172.21.15.42:0/2151573382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8068152370 con 0x7f80680d0900 2024-05-28T14:09:36.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f806d200640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80680ce860 0x7f80680b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:36.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f8067e00640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80680d0900 0x7f8068151e30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f80540011f0 tx=0x7f805400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:36.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f8064c00640 1 -- 172.21.15.42:0/2151573382 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054018070 con 0x7f80680d0900 2024-05-28T14:09:36.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f8064c00640 1 -- 172.21.15.42:0/2151573382 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80540074e0 con 0x7f80680d0900 2024-05-28T14:09:36.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f8064c00640 1 -- 172.21.15.42:0/2151573382 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054004a10 con 0x7f80680d0900 2024-05-28T14:09:36.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f806fc00640 1 -- 172.21.15.42:0/2151573382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80680b7420 con 0x7f80680d0900 2024-05-28T14:09:36.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f806fc00640 1 -- 172.21.15.42:0/2151573382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80680b6660 con 0x7f80680d0900 2024-05-28T14:09:36.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.457+0000 7f806fc00640 1 -- 172.21.15.42:0/2151573382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8068068a90 con 0x7f80680d0900 2024-05-28T14:09:36.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.461+0000 7f8064c00640 1 -- 172.21.15.42:0/2151573382 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f8054004bb0 con 0x7f80680d0900 2024-05-28T14:09:36.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.461+0000 7f8064c00640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80480694f0 0x7f804806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.461+0000 7f8064c00640 1 -- 172.21.15.42:0/2151573382 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7f8054084c50 con 0x7f80680d0900 2024-05-28T14:09:36.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.461+0000 7f806d200640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80480694f0 0x7f804806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.461+0000 7f806d200640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80480694f0 0x7f804806b9b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f805c0ae000 tx=0x7f805c0a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:36.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.465+0000 7f8064c00640 1 -- 172.21.15.42:0/2151573382 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8054056970 con 0x7f80680d0900 2024-05-28T14:09:36.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.625+0000 7f806fc00640 1 -- 172.21.15.42:0/2151573382 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80680b4ce0 con 0x7f80480694f0 2024-05-28T14:09:36.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.629+0000 7f8064c00640 1 -- 172.21.15.42:0/2151573382 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250670 (secure 0 0 0) 0x7f80680b4ce0 con 0x7f80480694f0 2024-05-28T14:09:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:36.618 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:09:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.633+0000 7f806fc00640 1 -- 172.21.15.42:0/2151573382 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80480694f0 msgr2=0x7f804806b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.633+0000 7f806fc00640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80480694f0 0x7f804806b9b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f805c0ae000 tx=0x7f805c0a7000 comp rx=0 tx=0).stop 2024-05-28T14:09:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.633+0000 7f806fc00640 1 -- 172.21.15.42:0/2151573382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80680d0900 msgr2=0x7f8068151e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.633+0000 7f806fc00640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80680d0900 0x7f8068151e30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f80540011f0 tx=0x7f805400c9b0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.633+0000 7f806fc00640 1 -- 172.21.15.42:0/2151573382 shutdown_connections 2024-05-28T14:09:36.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.633+0000 7f806fc00640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f80480694f0 0x7f804806b9b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.633+0000 7f806fc00640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8068162c80 0x7f8068157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.633+0000 7f806fc00640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80680d0900 0x7f8068151e30 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.633+0000 7f806fc00640 1 --2- 172.21.15.42:0/2151573382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80680ce860 0x7f80680b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.633+0000 7f806fc00640 1 -- 172.21.15.42:0/2151573382 >> 172.21.15.42:0/2151573382 conn(0x7f80680b4080 msgr2=0x7f806815f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:36.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.633+0000 7f806fc00640 1 -- 172.21.15.42:0/2151573382 shutdown_connections 2024-05-28T14:09:36.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.633+0000 7f806fc00640 1 -- 172.21.15.42:0/2151573382 wait complete. 2024-05-28T14:09:36.638 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-05-28T14:09:35.112730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":852952,"kb_used_data":9812,"kb_used_omap":12,"kb_used_meta":842995,"kb_avail":754121768,"statfs":{"total":773094113280,"available":772220690432,"internally_reserved":0,"allocated":10047488,"data_stored":4985879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":863227442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.387621"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151546+0000","last_change":"2024-05-28T14:09:31.763443+0000","last_active":"2024-05-28T14:09:33.151546+0000","last_peered":"2024-05-28T14:09:33.151546+0000","last_clean":"2024-05-28T14:09:33.151546+0000","last_became_active":"2024-05-28T14:09:31.763080+0000","last_became_peered":"2024-05-28T14:09:31.763080+0000","last_unstale":"2024-05-28T14:09:33.151546+0000","last_undegraded":"2024-05-28T14:09:33.151546+0000","last_fullsized":"2024-05-28T14:09:33.151546+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762829+0000","last_change":"2024-05-28T14:09:31.768406+0000","last_active":"2024-05-28T14:09:32.762829+0000","last_peered":"2024-05-28T14:09:32.762829+0000","last_clean":"2024-05-28T14:09:32.762829+0000","last_became_active":"2024-05-28T14:09:31.768150+0000","last_became_peered":"2024-05-28T14:09:31.768150+0000","last_unstale":"2024-05-28T14:09:32.762829+0000","last_undegraded":"2024-05-28T14:09:32.762829+0000","last_fullsized":"2024-05-28T14:09:32.762829+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:38.614174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150788+0000","last_change":"2024-05-28T14:09:31.770809+0000","last_active":"2024-05-28T14:09:33.150788+0000","last_peered":"2024-05-28T14:09:33.150788+0000","last_clean":"2024-05-28T14:09:33.150788+0000","last_became_active":"2024-05-28T14:09:31.770547+0000","last_became_peered":"2024-05-28T14:09:31.770547+0000","last_unstale":"2024-05-28T14:09:33.150788+0000","last_undegraded":"2024-05-28T14:09:33.150788+0000","last_fullsized":"2024-05-28T14:09:33.150788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:34:22.994257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751824+0000","last_change":"2024-05-28T14:09:31.756701+0000","last_active":"2024-05-28T14:09:32.751824+0000","last_peered":"2024-05-28T14:09:32.751824+0000","last_clean":"2024-05-28T14:09:32.751824+0000","last_became_active":"2024-05-28T14:09:31.756425+0000","last_became_peered":"2024-05-28T14:09:31.756425+0000","last_unstale":"2024-05-28T14:09:32.751824+0000","last_undegraded":"2024-05-28T14:09:32.751824+0000","last_fullsized":"2024-05-28T14:09:32.751824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:43.749597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750865+0000","last_change":"2024-05-28T14:09:31.756691+0000","last_active":"2024-05-28T14:09:32.750865+0000","last_peered":"2024-05-28T14:09:32.750865+0000","last_clean":"2024-05-28T14:09:32.750865+0000","last_became_active":"2024-05-28T14:09:31.756434+0000","last_became_peered":"2024-05-28T14:09:31.756434+0000","last_unstale":"2024-05-28T14:09:32.750865+0000","last_undegraded":"2024-05-28T14:09:32.750865+0000","last_fullsized":"2024-05-28T14:09:32.750865+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751086+0000","last_change":"2024-05-28T14:09:31.761009+0000","last_active":"2024-05-28T14:09:32.751086+0000","last_peered":"2024-05-28T14:09:32.751086+0000","last_clean":"2024-05-28T14:09:32.751086+0000","last_became_active":"2024-05-28T14:09:31.760693+0000","last_became_peered":"2024-05-28T14:09:31.760693+0000","last_unstale":"2024-05-28T14:09:32.751086+0000","last_undegraded":"2024-05-28T14:09:32.751086+0000","last_fullsized":"2024-05-28T14:09:32.751086+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751130+0000","last_change":"2024-05-28T14:09:31.748781+0000","last_active":"2024-05-28T14:09:32.751130+0000","last_peered":"2024-05-28T14:09:32.751130+0000","last_clean":"2024-05-28T14:09:32.751130+0000","last_became_active":"2024-05-28T14:09:31.748446+0000","last_became_peered":"2024-05-28T14:09:31.748446+0000","last_unstale":"2024-05-28T14:09:32.751130+0000","last_undegraded":"2024-05-28T14:09:32.751130+0000","last_fullsized":"2024-05-28T14:09:32.751130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151416+0000","last_change":"2024-05-28T14:09:31.755098+0000","last_active":"2024-05-28T14:09:33.151416+0000","last_peered":"2024-05-28T14:09:33.151416+0000","last_clean":"2024-05-28T14:09:33.151416+0000","last_became_active":"2024-05-28T14:09:31.754790+0000","last_became_peered":"2024-05-28T14:09:31.754790+0000","last_unstale":"2024-05-28T14:09:33.151416+0000","last_undegraded":"2024-05-28T14:09:33.151416+0000","last_fullsized":"2024-05-28T14:09:33.151416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762851+0000","last_change":"2024-05-28T14:09:31.765027+0000","last_active":"2024-05-28T14:09:32.762851+0000","last_peered":"2024-05-28T14:09:32.762851+0000","last_clean":"2024-05-28T14:09:32.762851+0000","last_became_active":"2024-05-28T14:09:31.764660+0000","last_became_peered":"2024-05-28T14:09:31.764660+0000","last_unstale":"2024-05-28T14:09:32.762851+0000","last_undegraded":"2024-05-28T14:09:32.762851+0000","last_fullsized":"2024-05-28T14:09:32.762851+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:59.297622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150769+0000","last_change":"2024-05-28T14:09:31.756022+0000","last_active":"2024-05-28T14:09:33.150769+0000","last_peered":"2024-05-28T14:09:33.150769+0000","last_clean":"2024-05-28T14:09:33.150769+0000","last_became_active":"2024-05-28T14:09:31.755772+0000","last_became_peered":"2024-05-28T14:09:31.755772+0000","last_unstale":"2024-05-28T14:09:33.150769+0000","last_undegraded":"2024-05-28T14:09:33.150769+0000","last_fullsized":"2024-05-28T14:09:33.150769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:35.172304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150463+0000","last_change":"2024-05-28T14:09:31.764389+0000","last_active":"2024-05-28T14:09:33.150463+0000","last_peered":"2024-05-28T14:09:33.150463+0000","last_clean":"2024-05-28T14:09:33.150463+0000","last_became_active":"2024-05-28T14:09:31.764102+0000","last_became_peered":"2024-05-28T14:09:31.764102+0000","last_unstale":"2024-05-28T14:09:33.150463+0000","last_undegraded":"2024-05-28T14:09:33.150463+0000","last_fullsized":"2024-05-28T14:09:33.150463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:37:41.765233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747849+0000","last_change":"2024-05-28T14:09:31.766893+0000","last_active":"2024-05-28T14:09:32.747849+0000","last_peered":"2024-05-28T14:09:32.747849+0000","last_clean":"2024-05-28T14:09:32.747849+0000","last_became_active":"2024-05-28T14:09:31.766457+0000","last_became_peered":"2024-05-28T14:09:31.766457+0000","last_unstale":"2024-05-28T14:09:32.747849+0000","last_undegraded":"2024-05-28T14:09:32.747849+0000","last_fullsized":"2024-05-28T14:09:32.747849+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:34:22.994257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751531+0000","last_change":"2024-05-28T14:09:31.753479+0000","last_active":"2024-05-28T14:09:32.751531+0000","last_peered":"2024-05-28T14:09:32.751531+0000","last_clean":"2024-05-28T14:09:32.751531+0000","last_became_active":"2024-05-28T14:09:31.753094+0000","last_became_peered":"2024-05-28T14:09:31.753094+0000","last_unstale":"2024-05-28T14:09:32.751531+0000","last_undegraded":"2024-05-28T14:09:32.751531+0000","last_fullsized":"2024-05-28T14:09:32.751531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:49:42.278616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752189+0000","last_change":"2024-05-28T14:09:31.759451+0000","last_active":"2024-05-28T14:09:32.752189+0000","last_peered":"2024-05-28T14:09:32.752189+0000","last_clean":"2024-05-28T14:09:32.752189+0000","last_became_active":"2024-05-28T14:09:31.759197+0000","last_became_peered":"2024-05-28T14:09:31.759197+0000","last_unstale":"2024-05-28T14:09:32.752189+0000","last_undegraded":"2024-05-28T14:09:32.752189+0000","last_fullsized":"2024-05-28T14:09:32.752189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765154+0000","last_change":"2024-05-28T14:09:31.753754+0000","last_active":"2024-05-28T14:09:32.765154+0000","last_peered":"2024-05-28T14:09:32.765154+0000","last_clean":"2024-05-28T14:09:32.765154+0000","last_became_active":"2024-05-28T14:09:31.753211+0000","last_became_peered":"2024-05-28T14:09:31.753211+0000","last_unstale":"2024-05-28T14:09:32.765154+0000","last_undegraded":"2024-05-28T14:09:32.765154+0000","last_fullsized":"2024-05-28T14:09:32.765154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762453+0000","last_change":"2024-05-28T14:09:31.755223+0000","last_active":"2024-05-28T14:09:32.762453+0000","last_peered":"2024-05-28T14:09:32.762453+0000","last_clean":"2024-05-28T14:09:32.762453+0000","last_became_active":"2024-05-28T14:09:31.754880+0000","last_became_peered":"2024-05-28T14:09:31.754880+0000","last_unstale":"2024-05-28T14:09:32.762453+0000","last_undegraded":"2024-05-28T14:09:32.762453+0000","last_fullsized":"2024-05-28T14:09:32.762453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:25.450855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150682+0000","last_change":"2024-05-28T14:09:31.752700+0000","last_active":"2024-05-28T14:09:33.150682+0000","last_peered":"2024-05-28T14:09:33.150682+0000","last_clean":"2024-05-28T14:09:33.150682+0000","last_became_active":"2024-05-28T14:09:31.752367+0000","last_became_peered":"2024-05-28T14:09:31.752367+0000","last_unstale":"2024-05-28T14:09:33.150682+0000","last_undegraded":"2024-05-28T14:09:33.150682+0000","last_fullsized":"2024-05-28T14:09:33.150682+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:43.749597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762838+0000","last_change":"2024-05-28T14:09:31.756171+0000","last_active":"2024-05-28T14:09:32.762838+0000","last_peered":"2024-05-28T14:09:32.762838+0000","last_clean":"2024-05-28T14:09:32.762838+0000","last_became_active":"2024-05-28T14:09:31.754405+0000","last_became_peered":"2024-05-28T14:09:31.754405+0000","last_unstale":"2024-05-28T14:09:32.762838+0000","last_undegraded":"2024-05-28T14:09:32.762838+0000","last_fullsized":"2024-05-28T14:09:32.762838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:19.325065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150852+0000","last_change":"2024-05-28T14:09:31.760741+0000","last_active":"2024-05-28T14:09:33.150852+0000","last_peered":"2024-05-28T14:09:33.150852+0000","last_clean":"2024-05-28T14:09:33.150852+0000","last_became_active":"2024-05-28T14:09:31.760404+0000","last_became_peered":"2024-05-28T14:09:31.760404+0000","last_unstale":"2024-05-28T14:09:33.150852+0000","last_undegraded":"2024-05-28T14:09:33.150852+0000","last_fullsized":"2024-05-28T14:09:33.150852+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:24.707344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751996+0000","last_change":"2024-05-28T14:09:31.754702+0000","last_active":"2024-05-28T14:09:32.751996+0000","last_peered":"2024-05-28T14:09:32.751996+0000","last_clean":"2024-05-28T14:09:32.751996+0000","last_became_active":"2024-05-28T14:09:31.754215+0000","last_became_peered":"2024-05-28T14:09:31.754215+0000","last_unstale":"2024-05-28T14:09:32.751996+0000","last_undegraded":"2024-05-28T14:09:32.751996+0000","last_fullsized":"2024-05-28T14:09:32.751996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:36:07.311954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151011+0000","last_change":"2024-05-28T14:09:31.765289+0000","last_active":"2024-05-28T14:09:33.151011+0000","last_peered":"2024-05-28T14:09:33.151011+0000","last_clean":"2024-05-28T14:09:33.151011+0000","last_became_active":"2024-05-28T14:09:31.763980+0000","last_became_peered":"2024-05-28T14:09:31.763980+0000","last_unstale":"2024-05-28T14:09:33.151011+0000","last_undegraded":"2024-05-28T14:09:33.151011+0000","last_fullsized":"2024-05-28T14:09:33.151011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762598+0000","last_change":"2024-05-28T14:09:31.758017+0000","last_active":"2024-05-28T14:09:32.762598+0000","last_peered":"2024-05-28T14:09:32.762598+0000","last_clean":"2024-05-28T14:09:32.762598+0000","last_became_active":"2024-05-28T14:09:31.757674+0000","last_became_peered":"2024-05-28T14:09:31.757674+0000","last_unstale":"2024-05-28T14:09:32.762598+0000","last_undegraded":"2024-05-28T14:09:32.762598+0000","last_fullsized":"2024-05-28T14:09:32.762598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:46.760174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762477+0000","last_change":"2024-05-28T14:09:31.762403+0000","last_active":"2024-05-28T14:09:32.762477+0000","last_peered":"2024-05-28T14:09:32.762477+0000","last_clean":"2024-05-28T14:09:32.762477+0000","last_became_active":"2024-05-28T14:09:31.762048+0000","last_became_peered":"2024-05-28T14:09:31.762048+0000","last_unstale":"2024-05-28T14:09:32.762477+0000","last_undegraded":"2024-05-28T14:09:32.762477+0000","last_fullsized":"2024-05-28T14:09:32.762477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:56.072759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751599+0000","last_change":"2024-05-28T14:09:31.747630+0000","last_active":"2024-05-28T14:09:32.751599+0000","last_peered":"2024-05-28T14:09:32.751599+0000","last_clean":"2024-05-28T14:09:32.751599+0000","last_became_active":"2024-05-28T14:09:31.747183+0000","last_became_peered":"2024-05-28T14:09:31.747183+0000","last_unstale":"2024-05-28T14:09:32.751599+0000","last_undegraded":"2024-05-28T14:09:32.751599+0000","last_fullsized":"2024-05-28T14:09:32.751599+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:49:44.321180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151402+0000","last_change":"2024-05-28T14:09:31.768533+0000","last_active":"2024-05-28T14:09:33.151402+0000","last_peered":"2024-05-28T14:09:33.151402+0000","last_clean":"2024-05-28T14:09:33.151402+0000","last_became_active":"2024-05-28T14:09:31.768237+0000","last_became_peered":"2024-05-28T14:09:31.768237+0000","last_unstale":"2024-05-28T14:09:33.151402+0000","last_undegraded":"2024-05-28T14:09:33.151402+0000","last_fullsized":"2024-05-28T14:09:33.151402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752209+0000","last_change":"2024-05-28T14:09:31.752686+0000","last_active":"2024-05-28T14:09:32.752209+0000","last_peered":"2024-05-28T14:09:32.752209+0000","last_clean":"2024-05-28T14:09:32.752209+0000","last_became_active":"2024-05-28T14:09:31.752411+0000","last_became_peered":"2024-05-28T14:09:31.752411+0000","last_unstale":"2024-05-28T14:09:32.752209+0000","last_undegraded":"2024-05-28T14:09:32.752209+0000","last_fullsized":"2024-05-28T14:09:32.752209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151595+0000","last_change":"2024-05-28T14:09:30.759880+0000","last_active":"2024-05-28T14:09:33.151595+0000","last_peered":"2024-05-28T14:09:33.151595+0000","last_clean":"2024-05-28T14:09:33.151595+0000","last_became_active":"2024-05-28T14:09:30.759575+0000","last_became_peered":"2024-05-28T14:09:30.759575+0000","last_unstale":"2024-05-28T14:09:33.151595+0000","last_undegraded":"2024-05-28T14:09:33.151595+0000","last_fullsized":"2024-05-28T14:09:33.151595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:55.065487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151493+0000","last_change":"2024-05-28T14:09:31.752593+0000","last_active":"2024-05-28T14:09:33.151493+0000","last_peered":"2024-05-28T14:09:33.151493+0000","last_clean":"2024-05-28T14:09:33.151493+0000","last_became_active":"2024-05-28T14:09:31.752332+0000","last_became_peered":"2024-05-28T14:09:31.752332+0000","last_unstale":"2024-05-28T14:09:33.151493+0000","last_undegraded":"2024-05-28T14:09:33.151493+0000","last_fullsized":"2024-05-28T14:09:33.151493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752466+0000","last_change":"2024-05-28T14:09:30.755821+0000","last_active":"2024-05-28T14:09:32.752466+0000","last_peered":"2024-05-28T14:09:32.752466+0000","last_clean":"2024-05-28T14:09:32.752466+0000","last_became_active":"2024-05-28T14:09:30.755452+0000","last_became_peered":"2024-05-28T14:09:30.755452+0000","last_unstale":"2024-05-28T14:09:32.752466+0000","last_undegraded":"2024-05-28T14:09:32.752466+0000","last_fullsized":"2024-05-28T14:09:32.752466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:18.317793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151220+0000","last_change":"2024-05-28T14:09:31.765522+0000","last_active":"2024-05-28T14:09:33.151220+0000","last_peered":"2024-05-28T14:09:33.151220+0000","last_clean":"2024-05-28T14:09:33.151220+0000","last_became_active":"2024-05-28T14:09:31.764349+0000","last_became_peered":"2024-05-28T14:09:31.764349+0000","last_unstale":"2024-05-28T14:09:33.151220+0000","last_undegraded":"2024-05-28T14:09:33.151220+0000","last_fullsized":"2024-05-28T14:09:33.151220+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765049+0000","last_change":"2024-05-28T14:09:30.752545+0000","last_active":"2024-05-28T14:09:32.765049+0000","last_peered":"2024-05-28T14:09:32.765049+0000","last_clean":"2024-05-28T14:09:32.765049+0000","last_became_active":"2024-05-28T14:09:30.752219+0000","last_became_peered":"2024-05-28T14:09:30.752219+0000","last_unstale":"2024-05-28T14:09:32.765049+0000","last_undegraded":"2024-05-28T14:09:32.765049+0000","last_fullsized":"2024-05-28T14:09:32.765049+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:49:43.313909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151148+0000","last_change":"2024-05-28T14:09:31.765767+0000","last_active":"2024-05-28T14:09:33.151148+0000","last_peered":"2024-05-28T14:09:33.151148+0000","last_clean":"2024-05-28T14:09:33.151148+0000","last_became_active":"2024-05-28T14:09:31.765407+0000","last_became_peered":"2024-05-28T14:09:31.765407+0000","last_unstale":"2024-05-28T14:09:33.151148+0000","last_undegraded":"2024-05-28T14:09:33.151148+0000","last_fullsized":"2024-05-28T14:09:33.151148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:27:09.475191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764918+0000","last_change":"2024-05-28T14:09:30.754683+0000","last_active":"2024-05-28T14:09:32.764918+0000","last_peered":"2024-05-28T14:09:32.764918+0000","last_clean":"2024-05-28T14:09:32.764918+0000","last_became_active":"2024-05-28T14:09:30.754405+0000","last_became_peered":"2024-05-28T14:09:30.754405+0000","last_unstale":"2024-05-28T14:09:32.764918+0000","last_undegraded":"2024-05-28T14:09:32.764918+0000","last_fullsized":"2024-05-28T14:09:32.764918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:29:45.272413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762565+0000","last_change":"2024-05-28T14:09:31.762430+0000","last_active":"2024-05-28T14:09:32.762565+0000","last_peered":"2024-05-28T14:09:32.762565+0000","last_clean":"2024-05-28T14:09:32.762565+0000","last_became_active":"2024-05-28T14:09:31.762101+0000","last_became_peered":"2024-05-28T14:09:31.762101+0000","last_unstale":"2024-05-28T14:09:32.762565+0000","last_undegraded":"2024-05-28T14:09:32.762565+0000","last_fullsized":"2024-05-28T14:09:32.762565+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:40:39.401890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752066+0000","last_change":"2024-05-28T14:09:30.747550+0000","last_active":"2024-05-28T14:09:32.752066+0000","last_peered":"2024-05-28T14:09:32.752066+0000","last_clean":"2024-05-28T14:09:32.752066+0000","last_became_active":"2024-05-28T14:09:30.747222+0000","last_became_peered":"2024-05-28T14:09:30.747222+0000","last_unstale":"2024-05-28T14:09:32.752066+0000","last_undegraded":"2024-05-28T14:09:32.752066+0000","last_fullsized":"2024-05-28T14:09:32.752066+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:58.290350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751952+0000","last_change":"2024-05-28T14:09:31.759538+0000","last_active":"2024-05-28T14:09:32.751952+0000","last_peered":"2024-05-28T14:09:32.751952+0000","last_clean":"2024-05-28T14:09:32.751952+0000","last_became_active":"2024-05-28T14:09:31.759258+0000","last_became_peered":"2024-05-28T14:09:31.759258+0000","last_unstale":"2024-05-28T14:09:32.751952+0000","last_undegraded":"2024-05-28T14:09:32.751952+0000","last_fullsized":"2024-05-28T14:09:32.751952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:35.172304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751683+0000","last_change":"2024-05-28T14:09:30.759103+0000","last_active":"2024-05-28T14:09:32.751683+0000","last_peered":"2024-05-28T14:09:32.751683+0000","last_clean":"2024-05-28T14:09:32.751683+0000","last_became_active":"2024-05-28T14:09:30.758628+0000","last_became_peered":"2024-05-28T14:09:30.758628+0000","last_unstale":"2024-05-28T14:09:32.751683+0000","last_undegraded":"2024-05-28T14:09:32.751683+0000","last_fullsized":"2024-05-28T14:09:32.751683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:54.698345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150467+0000","last_change":"2024-05-28T14:09:31.761382+0000","last_active":"2024-05-28T14:09:33.150467+0000","last_peered":"2024-05-28T14:09:33.150467+0000","last_clean":"2024-05-28T14:09:33.150467+0000","last_became_active":"2024-05-28T14:09:31.761095+0000","last_became_peered":"2024-05-28T14:09:31.761095+0000","last_unstale":"2024-05-28T14:09:33.150467+0000","last_undegraded":"2024-05-28T14:09:33.150467+0000","last_fullsized":"2024-05-28T14:09:33.150467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:31:10.786309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764699+0000","last_change":"2024-05-28T14:09:30.762673+0000","last_active":"2024-05-28T14:09:32.764699+0000","last_peered":"2024-05-28T14:09:32.764699+0000","last_clean":"2024-05-28T14:09:32.764699+0000","last_became_active":"2024-05-28T14:09:30.761793+0000","last_became_peered":"2024-05-28T14:09:30.761793+0000","last_unstale":"2024-05-28T14:09:32.764699+0000","last_undegraded":"2024-05-28T14:09:32.764699+0000","last_fullsized":"2024-05-28T14:09:32.764699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:40:25.005440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747896+0000","last_change":"2024-05-28T14:09:31.766958+0000","last_active":"2024-05-28T14:09:32.747896+0000","last_peered":"2024-05-28T14:09:32.747896+0000","last_clean":"2024-05-28T14:09:32.747896+0000","last_became_active":"2024-05-28T14:09:31.766301+0000","last_became_peered":"2024-05-28T14:09:31.766301+0000","last_unstale":"2024-05-28T14:09:32.747896+0000","last_undegraded":"2024-05-28T14:09:32.747896+0000","last_fullsized":"2024-05-28T14:09:32.747896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:24.707344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762264+0000","last_change":"2024-05-28T14:09:30.746473+0000","last_active":"2024-05-28T14:09:32.762264+0000","last_peered":"2024-05-28T14:09:32.762264+0000","last_clean":"2024-05-28T14:09:32.762264+0000","last_became_active":"2024-05-28T14:09:30.746177+0000","last_became_peered":"2024-05-28T14:09:30.746177+0000","last_unstale":"2024-05-28T14:09:32.762264+0000","last_undegraded":"2024-05-28T14:09:32.762264+0000","last_fullsized":"2024-05-28T14:09:32.762264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:41.356836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.748332+0000","last_change":"2024-05-28T14:09:31.756713+0000","last_active":"2024-05-28T14:09:32.748332+0000","last_peered":"2024-05-28T14:09:32.748332+0000","last_clean":"2024-05-28T14:09:32.748332+0000","last_became_active":"2024-05-28T14:09:31.755716+0000","last_became_peered":"2024-05-28T14:09:31.755716+0000","last_unstale":"2024-05-28T14:09:32.748332+0000","last_undegraded":"2024-05-28T14:09:32.748332+0000","last_fullsized":"2024-05-28T14:09:32.748332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:02:06.074104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151668+0000","last_change":"2024-05-28T14:09:30.758732+0000","last_active":"2024-05-28T14:09:33.151668+0000","last_peered":"2024-05-28T14:09:33.151668+0000","last_clean":"2024-05-28T14:09:33.151668+0000","last_became_active":"2024-05-28T14:09:30.758400+0000","last_became_peered":"2024-05-28T14:09:30.758400+0000","last_unstale":"2024-05-28T14:09:33.151668+0000","last_undegraded":"2024-05-28T14:09:33.151668+0000","last_fullsized":"2024-05-28T14:09:33.151668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:45.752902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750903+0000","last_change":"2024-05-28T14:09:31.762311+0000","last_active":"2024-05-28T14:09:32.750903+0000","last_peered":"2024-05-28T14:09:32.750903+0000","last_clean":"2024-05-28T14:09:32.750903+0000","last_became_active":"2024-05-28T14:09:31.761990+0000","last_became_peered":"2024-05-28T14:09:31.761990+0000","last_unstale":"2024-05-28T14:09:32.750903+0000","last_undegraded":"2024-05-28T14:09:32.750903+0000","last_fullsized":"2024-05-28T14:09:32.750903+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151408+0000","last_change":"2024-05-28T14:09:30.757972+0000","last_active":"2024-05-28T14:09:33.151408+0000","last_peered":"2024-05-28T14:09:33.151408+0000","last_clean":"2024-05-28T14:09:33.151408+0000","last_became_active":"2024-05-28T14:09:30.757716+0000","last_became_peered":"2024-05-28T14:09:30.757716+0000","last_unstale":"2024-05-28T14:09:33.151408+0000","last_undegraded":"2024-05-28T14:09:33.151408+0000","last_fullsized":"2024-05-28T14:09:33.151408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:40:38.394618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764978+0000","last_change":"2024-05-28T14:09:31.754712+0000","last_active":"2024-05-28T14:09:32.764978+0000","last_peered":"2024-05-28T14:09:32.764978+0000","last_clean":"2024-05-28T14:09:32.764978+0000","last_became_active":"2024-05-28T14:09:31.754427+0000","last_became_peered":"2024-05-28T14:09:31.754427+0000","last_unstale":"2024-05-28T14:09:32.764978+0000","last_undegraded":"2024-05-28T14:09:32.764978+0000","last_fullsized":"2024-05-28T14:09:32.764978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151098+0000","last_change":"2024-05-28T14:09:30.756257+0000","last_active":"2024-05-28T14:09:33.151098+0000","last_peered":"2024-05-28T14:09:33.151098+0000","last_clean":"2024-05-28T14:09:33.151098+0000","last_became_active":"2024-05-28T14:09:30.755923+0000","last_became_peered":"2024-05-28T14:09:30.755923+0000","last_unstale":"2024-05-28T14:09:33.151098+0000","last_undegraded":"2024-05-28T14:09:33.151098+0000","last_fullsized":"2024-05-28T14:09:33.151098+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:54.698345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752090+0000","last_change":"2024-05-28T14:09:31.748217+0000","last_active":"2024-05-28T14:09:32.752090+0000","last_peered":"2024-05-28T14:09:32.752090+0000","last_clean":"2024-05-28T14:09:32.752090+0000","last_became_active":"2024-05-28T14:09:31.747336+0000","last_became_peered":"2024-05-28T14:09:31.747336+0000","last_unstale":"2024-05-28T14:09:32.752090+0000","last_undegraded":"2024-05-28T14:09:32.752090+0000","last_fullsized":"2024-05-28T14:09:32.752090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"89'6","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750773+0000","last_change":"2024-05-28T14:09:30.760654+0000","last_active":"2024-05-28T14:09:32.750773+0000","last_peered":"2024-05-28T14:09:32.750773+0000","last_clean":"2024-05-28T14:09:32.750773+0000","last_became_active":"2024-05-28T14:09:30.760328+0000","last_became_peered":"2024-05-28T14:09:30.760328+0000","last_unstale":"2024-05-28T14:09:32.750773+0000","last_undegraded":"2024-05-28T14:09:32.750773+0000","last_fullsized":"2024-05-28T14:09:32.750773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:54:12.659735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764801+0000","last_change":"2024-05-28T14:09:31.742984+0000","last_active":"2024-05-28T14:09:32.764801+0000","last_peered":"2024-05-28T14:09:32.764801+0000","last_clean":"2024-05-28T14:09:32.764801+0000","last_became_active":"2024-05-28T14:09:31.742692+0000","last_became_peered":"2024-05-28T14:09:31.742692+0000","last_unstale":"2024-05-28T14:09:32.764801+0000","last_undegraded":"2024-05-28T14:09:32.764801+0000","last_fullsized":"2024-05-28T14:09:32.764801+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762506+0000","last_change":"2024-05-28T14:09:30.751922+0000","last_active":"2024-05-28T14:09:32.762506+0000","last_peered":"2024-05-28T14:09:32.762506+0000","last_clean":"2024-05-28T14:09:32.762506+0000","last_became_active":"2024-05-28T14:09:30.751646+0000","last_became_peered":"2024-05-28T14:09:30.751646+0000","last_unstale":"2024-05-28T14:09:32.762506+0000","last_undegraded":"2024-05-28T14:09:32.762506+0000","last_fullsized":"2024-05-28T14:09:32.762506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:24:14.215364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150460+0000","last_change":"2024-05-28T14:09:31.763288+0000","last_active":"2024-05-28T14:09:33.150460+0000","last_peered":"2024-05-28T14:09:33.150460+0000","last_clean":"2024-05-28T14:09:33.150460+0000","last_became_active":"2024-05-28T14:09:31.762800+0000","last_became_peered":"2024-05-28T14:09:31.762800+0000","last_unstale":"2024-05-28T14:09:33.150460+0000","last_undegraded":"2024-05-28T14:09:33.150460+0000","last_fullsized":"2024-05-28T14:09:33.150460+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:46:19.764662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747904+0000","last_change":"2024-05-28T14:09:30.753559+0000","last_active":"2024-05-28T14:09:32.747904+0000","last_peered":"2024-05-28T14:09:32.747904+0000","last_clean":"2024-05-28T14:09:32.747904+0000","last_became_active":"2024-05-28T14:09:30.753253+0000","last_became_peered":"2024-05-28T14:09:30.753253+0000","last_unstale":"2024-05-28T14:09:32.747904+0000","last_undegraded":"2024-05-28T14:09:32.747904+0000","last_fullsized":"2024-05-28T14:09:32.747904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:15:03.984981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762339+0000","last_change":"2024-05-28T14:09:31.765598+0000","last_active":"2024-05-28T14:09:32.762339+0000","last_peered":"2024-05-28T14:09:32.762339+0000","last_clean":"2024-05-28T14:09:32.762339+0000","last_became_active":"2024-05-28T14:09:31.760967+0000","last_became_peered":"2024-05-28T14:09:31.760967+0000","last_unstale":"2024-05-28T14:09:32.762339+0000","last_undegraded":"2024-05-28T14:09:32.762339+0000","last_fullsized":"2024-05-28T14:09:32.762339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:24:40.914669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750623+0000","last_change":"2024-05-28T14:09:30.758094+0000","last_active":"2024-05-28T14:09:32.750623+0000","last_peered":"2024-05-28T14:09:32.750623+0000","last_clean":"2024-05-28T14:09:32.750623+0000","last_became_active":"2024-05-28T14:09:30.757801+0000","last_became_peered":"2024-05-28T14:09:30.757801+0000","last_unstale":"2024-05-28T14:09:32.750623+0000","last_undegraded":"2024-05-28T14:09:32.750623+0000","last_fullsized":"2024-05-28T14:09:32.750623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:40:25.005440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762291+0000","last_change":"2024-05-28T14:09:31.754734+0000","last_active":"2024-05-28T14:09:32.762291+0000","last_peered":"2024-05-28T14:09:32.762291+0000","last_clean":"2024-05-28T14:09:32.762291+0000","last_became_active":"2024-05-28T14:09:31.754459+0000","last_became_peered":"2024-05-28T14:09:31.754459+0000","last_unstale":"2024-05-28T14:09:32.762291+0000","last_undegraded":"2024-05-28T14:09:32.762291+0000","last_fullsized":"2024-05-28T14:09:32.762291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:19:30.516845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"89'8","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764575+0000","last_change":"2024-05-28T14:09:30.759773+0000","last_active":"2024-05-28T14:09:32.764575+0000","last_peered":"2024-05-28T14:09:32.764575+0000","last_clean":"2024-05-28T14:09:32.764575+0000","last_became_active":"2024-05-28T14:09:30.759218+0000","last_became_peered":"2024-05-28T14:09:30.759218+0000","last_unstale":"2024-05-28T14:09:32.764575+0000","last_undegraded":"2024-05-28T14:09:32.764575+0000","last_fullsized":"2024-05-28T14:09:32.764575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:14:14.485471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151031+0000","last_change":"2024-05-28T14:09:31.766044+0000","last_active":"2024-05-28T14:09:33.151031+0000","last_peered":"2024-05-28T14:09:33.151031+0000","last_clean":"2024-05-28T14:09:33.151031+0000","last_became_active":"2024-05-28T14:09:31.763364+0000","last_became_peered":"2024-05-28T14:09:31.763364+0000","last_unstale":"2024-05-28T14:09:33.151031+0000","last_undegraded":"2024-05-28T14:09:33.151031+0000","last_fullsized":"2024-05-28T14:09:33.151031+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:54:38.549065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151738+0000","last_change":"2024-05-28T14:09:30.763452+0000","last_active":"2024-05-28T14:09:33.151738+0000","last_peered":"2024-05-28T14:09:33.151738+0000","last_clean":"2024-05-28T14:09:33.151738+0000","last_became_active":"2024-05-28T14:09:30.763075+0000","last_became_peered":"2024-05-28T14:09:30.763075+0000","last_unstale":"2024-05-28T14:09:33.151738+0000","last_undegraded":"2024-05-28T14:09:33.151738+0000","last_fullsized":"2024-05-28T14:09:33.151738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:37.606903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151668+0000","last_change":"2024-05-28T14:09:31.769821+0000","last_active":"2024-05-28T14:09:33.151668+0000","last_peered":"2024-05-28T14:09:33.151668+0000","last_clean":"2024-05-28T14:09:33.151668+0000","last_became_active":"2024-05-28T14:09:31.769534+0000","last_became_peered":"2024-05-28T14:09:31.769534+0000","last_unstale":"2024-05-28T14:09:33.151668+0000","last_undegraded":"2024-05-28T14:09:33.151668+0000","last_fullsized":"2024-05-28T14:09:33.151668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765169+0000","last_change":"2024-05-28T14:09:30.752703+0000","last_active":"2024-05-28T14:09:32.765169+0000","last_peered":"2024-05-28T14:09:32.765169+0000","last_clean":"2024-05-28T14:09:32.765169+0000","last_became_active":"2024-05-28T14:09:30.752399+0000","last_became_peered":"2024-05-28T14:09:30.752399+0000","last_unstale":"2024-05-28T14:09:32.765169+0000","last_undegraded":"2024-05-28T14:09:32.765169+0000","last_fullsized":"2024-05-28T14:09:32.765169+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:42.742326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752236+0000","last_change":"2024-05-28T14:09:31.757626+0000","last_active":"2024-05-28T14:09:32.752236+0000","last_peered":"2024-05-28T14:09:32.752236+0000","last_clean":"2024-05-28T14:09:32.752236+0000","last_became_active":"2024-05-28T14:09:31.756947+0000","last_became_peered":"2024-05-28T14:09:31.756947+0000","last_unstale":"2024-05-28T14:09:32.752236+0000","last_undegraded":"2024-05-28T14:09:32.752236+0000","last_fullsized":"2024-05-28T14:09:32.752236+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765042+0000","last_change":"2024-05-28T14:09:30.760474+0000","last_active":"2024-05-28T14:09:32.765042+0000","last_peered":"2024-05-28T14:09:32.765042+0000","last_clean":"2024-05-28T14:09:32.765042+0000","last_became_active":"2024-05-28T14:09:30.759783+0000","last_became_peered":"2024-05-28T14:09:30.759783+0000","last_unstale":"2024-05-28T14:09:32.765042+0000","last_undegraded":"2024-05-28T14:09:32.765042+0000","last_fullsized":"2024-05-28T14:09:32.765042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:49:41.271344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.748033+0000","last_change":"2024-05-28T14:09:30.754286+0000","last_active":"2024-05-28T14:09:32.748033+0000","last_peered":"2024-05-28T14:09:32.748033+0000","last_clean":"2024-05-28T14:09:32.748033+0000","last_became_active":"2024-05-28T14:09:30.753997+0000","last_became_peered":"2024-05-28T14:09:30.753997+0000","last_unstale":"2024-05-28T14:09:32.748033+0000","last_undegraded":"2024-05-28T14:09:32.748033+0000","last_fullsized":"2024-05-28T14:09:32.748033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:34:27.167006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751282+0000","last_change":"2024-05-28T14:09:31.760545+0000","last_active":"2024-05-28T14:09:32.751282+0000","last_peered":"2024-05-28T14:09:32.751282+0000","last_clean":"2024-05-28T14:09:32.751282+0000","last_became_active":"2024-05-28T14:09:31.760024+0000","last_became_peered":"2024-05-28T14:09:31.760024+0000","last_unstale":"2024-05-28T14:09:32.751282+0000","last_undegraded":"2024-05-28T14:09:32.751282+0000","last_fullsized":"2024-05-28T14:09:32.751282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-05-28T14:09:29.732310+0000","last_change":"2024-05-28T14:09:29.732310+0000","last_active":"2024-05-28T14:09:29.717625+0000","last_peered":"2024-05-28T14:09:29.717625+0000","last_clean":"2024-05-28T14:09:29.717625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:29.732310+0000","last_undegraded":"2024-05-28T14:09:29.732310+0000","last_fullsized":"2024-05-28T14:09:29.732310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-28T14:09:30.724896+0000","last_change":"2024-05-28T14:09:30.724896+0000","last_active":"2024-05-28T14:09:30.724896+0000","last_peered":"2024-05-28T14:09:30.724896+0000","last_clean":"2024-05-28T14:09:30.724896+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:30.724896+0000","last_undegraded":"2024-05-28T14:09:30.724896+0000","last_fullsized":"2024-05-28T14:09:30.724896+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-05-28T14:09:29.731567+0000","last_change":"2024-05-28T14:09:29.731567+0000","last_active":"2024-05-28T14:09:29.717625+0000","last_peered":"2024-05-28T14:09:29.717625+0000","last_clean":"2024-05-28T14:09:29.717625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:09:29.731567+0000","last_undegraded":"2024-05-28T14:09:29.731567+0000","last_fullsized":"2024-05-28T14:09:29.731567+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751081+0000","last_change":"2024-05-28T14:09:31.748847+0000","last_active":"2024-05-28T14:09:32.751081+0000","last_peered":"2024-05-28T14:09:32.751081+0000","last_clean":"2024-05-28T14:09:32.751081+0000","last_became_active":"2024-05-28T14:09:31.748479+0000","last_became_peered":"2024-05-28T14:09:31.748479+0000","last_unstale":"2024-05-28T14:09:32.751081+0000","last_undegraded":"2024-05-28T14:09:32.751081+0000","last_fullsized":"2024-05-28T14:09:32.751081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747718+0000","last_change":"2024-05-28T14:09:30.755244+0000","last_active":"2024-05-28T14:09:32.747718+0000","last_peered":"2024-05-28T14:09:32.747718+0000","last_clean":"2024-05-28T14:09:32.747718+0000","last_became_active":"2024-05-28T14:09:30.754984+0000","last_became_peered":"2024-05-28T14:09:30.754984+0000","last_unstale":"2024-05-28T14:09:32.747718+0000","last_undegraded":"2024-05-28T14:09:32.747718+0000","last_fullsized":"2024-05-28T14:09:32.747718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:14:58.988961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752367+0000","last_change":"2024-05-28T14:09:31.745138+0000","last_active":"2024-05-28T14:09:32.752367+0000","last_peered":"2024-05-28T14:09:32.752367+0000","last_clean":"2024-05-28T14:09:32.752367+0000","last_became_active":"2024-05-28T14:09:31.744819+0000","last_became_peered":"2024-05-28T14:09:31.744819+0000","last_unstale":"2024-05-28T14:09:32.752367+0000","last_undegraded":"2024-05-28T14:09:32.752367+0000","last_fullsized":"2024-05-28T14:09:32.752367+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762790+0000","last_change":"2024-05-28T14:09:30.760787+0000","last_active":"2024-05-28T14:09:32.762790+0000","last_peered":"2024-05-28T14:09:32.762790+0000","last_clean":"2024-05-28T14:09:32.762790+0000","last_became_active":"2024-05-28T14:09:30.760502+0000","last_became_peered":"2024-05-28T14:09:30.760502+0000","last_unstale":"2024-05-28T14:09:32.762790+0000","last_undegraded":"2024-05-28T14:09:32.762790+0000","last_fullsized":"2024-05-28T14:09:32.762790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:31:35.886734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150913+0000","last_change":"2024-05-28T14:09:31.771112+0000","last_active":"2024-05-28T14:09:33.150913+0000","last_peered":"2024-05-28T14:09:33.150913+0000","last_clean":"2024-05-28T14:09:33.150913+0000","last_became_active":"2024-05-28T14:09:31.770660+0000","last_became_peered":"2024-05-28T14:09:31.770660+0000","last_unstale":"2024-05-28T14:09:33.150913+0000","last_undegraded":"2024-05-28T14:09:33.150913+0000","last_fullsized":"2024-05-28T14:09:33.150913+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:02:06.074104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762843+0000","last_change":"2024-05-28T14:09:30.751801+0000","last_active":"2024-05-28T14:09:32.762843+0000","last_peered":"2024-05-28T14:09:32.762843+0000","last_clean":"2024-05-28T14:09:32.762843+0000","last_became_active":"2024-05-28T14:09:30.751504+0000","last_became_peered":"2024-05-28T14:09:30.751504+0000","last_unstale":"2024-05-28T14:09:32.762843+0000","last_undegraded":"2024-05-28T14:09:32.762843+0000","last_fullsized":"2024-05-28T14:09:32.762843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:42.042818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151673+0000","last_change":"2024-05-28T14:09:31.765811+0000","last_active":"2024-05-28T14:09:33.151673+0000","last_peered":"2024-05-28T14:09:33.151673+0000","last_clean":"2024-05-28T14:09:33.151673+0000","last_became_active":"2024-05-28T14:09:31.764710+0000","last_became_peered":"2024-05-28T14:09:31.764710+0000","last_unstale":"2024-05-28T14:09:33.151673+0000","last_undegraded":"2024-05-28T14:09:33.151673+0000","last_fullsized":"2024-05-28T14:09:33.151673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:57.967039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151377+0000","last_change":"2024-05-28T14:09:30.760473+0000","last_active":"2024-05-28T14:09:33.151377+0000","last_peered":"2024-05-28T14:09:33.151377+0000","last_clean":"2024-05-28T14:09:33.151377+0000","last_became_active":"2024-05-28T14:09:30.760129+0000","last_became_peered":"2024-05-28T14:09:30.760129+0000","last_unstale":"2024-05-28T14:09:33.151377+0000","last_undegraded":"2024-05-28T14:09:33.151377+0000","last_fullsized":"2024-05-28T14:09:33.151377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:18.317793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752265+0000","last_change":"2024-05-28T14:09:31.748833+0000","last_active":"2024-05-28T14:09:32.752265+0000","last_peered":"2024-05-28T14:09:32.752265+0000","last_clean":"2024-05-28T14:09:32.752265+0000","last_became_active":"2024-05-28T14:09:31.747731+0000","last_became_peered":"2024-05-28T14:09:31.747731+0000","last_unstale":"2024-05-28T14:09:32.752265+0000","last_undegraded":"2024-05-28T14:09:32.752265+0000","last_fullsized":"2024-05-28T14:09:32.752265+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151485+0000","last_change":"2024-05-28T14:09:30.758617+0000","last_active":"2024-05-28T14:09:33.151485+0000","last_peered":"2024-05-28T14:09:33.151485+0000","last_clean":"2024-05-28T14:09:33.151485+0000","last_became_active":"2024-05-28T14:09:30.758317+0000","last_became_peered":"2024-05-28T14:09:30.758317+0000","last_unstale":"2024-05-28T14:09:33.151485+0000","last_undegraded":"2024-05-28T14:09:33.151485+0000","last_fullsized":"2024-05-28T14:09:33.151485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:10.090547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751157+0000","last_change":"2024-05-28T14:09:31.760268+0000","last_active":"2024-05-28T14:09:32.751157+0000","last_peered":"2024-05-28T14:09:32.751157+0000","last_clean":"2024-05-28T14:09:32.751157+0000","last_became_active":"2024-05-28T14:09:31.759889+0000","last_became_peered":"2024-05-28T14:09:31.759889+0000","last_unstale":"2024-05-28T14:09:32.751157+0000","last_undegraded":"2024-05-28T14:09:32.751157+0000","last_fullsized":"2024-05-28T14:09:32.751157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764892+0000","last_change":"2024-05-28T14:09:30.754916+0000","last_active":"2024-05-28T14:09:32.764892+0000","last_peered":"2024-05-28T14:09:32.764892+0000","last_clean":"2024-05-28T14:09:32.764892+0000","last_became_active":"2024-05-28T14:09:30.754019+0000","last_became_peered":"2024-05-28T14:09:30.754019+0000","last_unstale":"2024-05-28T14:09:32.764892+0000","last_undegraded":"2024-05-28T14:09:32.764892+0000","last_fullsized":"2024-05-28T14:09:32.764892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:34.165033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764999+0000","last_change":"2024-05-28T14:09:31.748964+0000","last_active":"2024-05-28T14:09:32.764999+0000","last_peered":"2024-05-28T14:09:32.764999+0000","last_clean":"2024-05-28T14:09:32.764999+0000","last_became_active":"2024-05-28T14:09:31.748393+0000","last_became_peered":"2024-05-28T14:09:31.748393+0000","last_unstale":"2024-05-28T14:09:32.764999+0000","last_undegraded":"2024-05-28T14:09:32.764999+0000","last_fullsized":"2024-05-28T14:09:32.764999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751724+0000","last_change":"2024-05-28T14:09:30.748735+0000","last_active":"2024-05-28T14:09:32.751724+0000","last_peered":"2024-05-28T14:09:32.751724+0000","last_clean":"2024-05-28T14:09:32.751724+0000","last_became_active":"2024-05-28T14:09:30.748404+0000","last_became_peered":"2024-05-28T14:09:30.748404+0000","last_unstale":"2024-05-28T14:09:32.751724+0000","last_undegraded":"2024-05-28T14:09:32.751724+0000","last_fullsized":"2024-05-28T14:09:32.751724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:37.606903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150429+0000","last_change":"2024-05-28T14:09:31.764716+0000","last_active":"2024-05-28T14:09:33.150429+0000","last_peered":"2024-05-28T14:09:33.150429+0000","last_clean":"2024-05-28T14:09:33.150429+0000","last_became_active":"2024-05-28T14:09:31.764407+0000","last_became_peered":"2024-05-28T14:09:31.764407+0000","last_unstale":"2024-05-28T14:09:33.150429+0000","last_undegraded":"2024-05-28T14:09:33.150429+0000","last_fullsized":"2024-05-28T14:09:33.150429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:14:59.996233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151098+0000","last_change":"2024-05-28T14:09:30.755558+0000","last_active":"2024-05-28T14:09:33.151098+0000","last_peered":"2024-05-28T14:09:33.151098+0000","last_clean":"2024-05-28T14:09:33.151098+0000","last_became_active":"2024-05-28T14:09:30.755257+0000","last_became_peered":"2024-05-28T14:09:30.755257+0000","last_unstale":"2024-05-28T14:09:33.151098+0000","last_undegraded":"2024-05-28T14:09:33.151098+0000","last_fullsized":"2024-05-28T14:09:33.151098+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:58.290350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150859+0000","last_change":"2024-05-28T14:09:31.758209+0000","last_active":"2024-05-28T14:09:33.150859+0000","last_peered":"2024-05-28T14:09:33.150859+0000","last_clean":"2024-05-28T14:09:33.150859+0000","last_became_active":"2024-05-28T14:09:31.757910+0000","last_became_peered":"2024-05-28T14:09:31.757910+0000","last_unstale":"2024-05-28T14:09:33.150859+0000","last_undegraded":"2024-05-28T14:09:33.150859+0000","last_fullsized":"2024-05-28T14:09:33.150859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:36:07.311954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751556+0000","last_change":"2024-05-28T14:09:30.754615+0000","last_active":"2024-05-28T14:09:32.751556+0000","last_peered":"2024-05-28T14:09:32.751556+0000","last_clean":"2024-05-28T14:09:32.751556+0000","last_became_active":"2024-05-28T14:09:30.754325+0000","last_became_peered":"2024-05-28T14:09:30.754325+0000","last_unstale":"2024-05-28T14:09:32.751556+0000","last_undegraded":"2024-05-28T14:09:32.751556+0000","last_fullsized":"2024-05-28T14:09:32.751556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:45.752902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762719+0000","last_change":"2024-05-28T14:09:31.760657+0000","last_active":"2024-05-28T14:09:32.762719+0000","last_peered":"2024-05-28T14:09:32.762719+0000","last_clean":"2024-05-28T14:09:32.762719+0000","last_became_active":"2024-05-28T14:09:31.759615+0000","last_became_peered":"2024-05-28T14:09:31.759615+0000","last_unstale":"2024-05-28T14:09:32.762719+0000","last_undegraded":"2024-05-28T14:09:32.762719+0000","last_fullsized":"2024-05-28T14:09:32.762719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:55.705617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151176+0000","last_change":"2024-05-28T14:09:30.762060+0000","last_active":"2024-05-28T14:09:33.151176+0000","last_peered":"2024-05-28T14:09:33.151176+0000","last_clean":"2024-05-28T14:09:33.151176+0000","last_became_active":"2024-05-28T14:09:30.761778+0000","last_became_peered":"2024-05-28T14:09:30.761778+0000","last_unstale":"2024-05-28T14:09:33.151176+0000","last_undegraded":"2024-05-28T14:09:33.151176+0000","last_fullsized":"2024-05-28T14:09:33.151176+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:56.959767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":300,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150107+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:09:33.150107+0000","last_peered":"2024-05-28T14:09:33.150107+0000","last_clean":"2024-05-28T14:09:33.150107+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:09:33.150107+0000","last_undegraded":"2024-05-28T14:09:33.150107+0000","last_fullsized":"2024-05-28T14:09:33.150107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":7},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123028,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248812,"statfs":{"total":96636764160,"available":96510783488,"internally_reserved":0,"allocated":1703936,"data_stored":1068711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122448,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249392,"statfs":{"total":96636764160,"available":96511377408,"internally_reserved":0,"allocated":1118208,"data_stored":478967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":124254662},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76628,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295212,"statfs":{"total":96636764160,"available":96558297088,"internally_reserved":0,"allocated":1122304,"data_stored":479907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82184,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289656,"statfs":{"total":96636764160,"available":96552607744,"internally_reserved":0,"allocated":1105920,"data_stored":478273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574987,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130772,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241068,"statfs":{"total":96636764160,"available":96502853632,"internally_reserved":0,"allocated":1118208,"data_stored":479885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":132774344},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144604,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227236,"statfs":{"total":96636764160,"available":96488689664,"internally_reserved":0,"allocated":1056768,"data_stored":452994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108564,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94263276,"statfs":{"total":96636764160,"available":96525594624,"internally_reserved":0,"allocated":1118208,"data_stored":478467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64724,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307116,"statfs":{"total":96636764160,"available":96570486784,"internally_reserved":0,"allocated":1703936,"data_stored":1068675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:09:36.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2024-05-28T14:09:36.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-28T14:09:30.724896+0000', 'last_change': '2024-05-28T14:09:30.724896+0000', 'last_active': '2024-05-28T14:09:30.724896+0000', 'last_peered': '2024-05-28T14:09:30.724896+0000', 'last_clean': '2024-05-28T14:09:30.724896+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:09:30.724896+0000', 'last_undegraded': '2024-05-28T14:09:30.724896+0000', 'last_fullsized': '2024-05-28T14:09:30.724896+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_clean_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-28T14:09:36.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.39 is not active+clean 2024-05-28T14:09:36.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-28T14:09:30.724896+0000', 'last_change': '2024-05-28T14:09:30.724896+0000', 'last_active': '2024-05-28T14:09:30.724896+0000', 'last_peered': '2024-05-28T14:09:30.724896+0000', 'last_clean': '2024-05-28T14:09:30.724896+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:09:30.724896+0000', 'last_undegraded': '2024-05-28T14:09:30.724896+0000', 'last_fullsized': '2024-05-28T14:09:30.724896+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_clean_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-28T14:09:36.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.32 is not active+clean 2024-05-28T14:09:36.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-28T14:09:30.724896+0000', 'last_change': '2024-05-28T14:09:30.724896+0000', 'last_active': '2024-05-28T14:09:30.724896+0000', 'last_peered': '2024-05-28T14:09:30.724896+0000', 'last_clean': '2024-05-28T14:09:30.724896+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:09:30.724896+0000', 'last_undegraded': '2024-05-28T14:09:30.724896+0000', 'last_fullsized': '2024-05-28T14:09:30.724896+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_clean_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-28T14:09:36.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2e is not active+clean 2024-05-28T14:09:36.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-28T14:09:30.724896+0000', 'last_change': '2024-05-28T14:09:30.724896+0000', 'last_active': '2024-05-28T14:09:30.724896+0000', 'last_peered': '2024-05-28T14:09:30.724896+0000', 'last_clean': '2024-05-28T14:09:30.724896+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:09:30.724896+0000', 'last_undegraded': '2024-05-28T14:09:30.724896+0000', 'last_fullsized': '2024-05-28T14:09:30.724896+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_clean_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-28T14:09:36.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2c is not active+clean 2024-05-28T14:09:36.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-28T14:09:30.724896+0000', 'last_change': '2024-05-28T14:09:30.724896+0000', 'last_active': '2024-05-28T14:09:30.724896+0000', 'last_peered': '2024-05-28T14:09:30.724896+0000', 'last_clean': '2024-05-28T14:09:30.724896+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:09:30.724896+0000', 'last_undegraded': '2024-05-28T14:09:30.724896+0000', 'last_fullsized': '2024-05-28T14:09:30.724896+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_clean_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-28T14:09:36.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2024-05-28T14:09:36.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-28T14:09:30.724896+0000', 'last_change': '2024-05-28T14:09:30.724896+0000', 'last_active': '2024-05-28T14:09:30.724896+0000', 'last_peered': '2024-05-28T14:09:30.724896+0000', 'last_clean': '2024-05-28T14:09:30.724896+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:09:30.724896+0000', 'last_undegraded': '2024-05-28T14:09:30.724896+0000', 'last_fullsized': '2024-05-28T14:09:30.724896+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_clean_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-28T14:09:36.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.28 is not active+clean 2024-05-28T14:09:36.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-28T14:09:30.724896+0000', 'last_change': '2024-05-28T14:09:30.724896+0000', 'last_active': '2024-05-28T14:09:30.724896+0000', 'last_peered': '2024-05-28T14:09:30.724896+0000', 'last_clean': '2024-05-28T14:09:30.724896+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:09:30.724896+0000', 'last_undegraded': '2024-05-28T14:09:30.724896+0000', 'last_fullsized': '2024-05-28T14:09:30.724896+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_clean_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-28T14:09:36.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2024-05-28T14:09:36.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-28T14:09:30.724896+0000', 'last_change': '2024-05-28T14:09:30.724896+0000', 'last_active': '2024-05-28T14:09:30.724896+0000', 'last_peered': '2024-05-28T14:09:30.724896+0000', 'last_clean': '2024-05-28T14:09:30.724896+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:09:30.724896+0000', 'last_undegraded': '2024-05-28T14:09:30.724896+0000', 'last_fullsized': '2024-05-28T14:09:30.724896+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_clean_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-28T14:09:36.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.a is not active+clean 2024-05-28T14:09:36.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:09:29.732310+0000', 'last_change': '2024-05-28T14:09:29.732310+0000', 'last_active': '2024-05-28T14:09:29.717625+0000', 'last_peered': '2024-05-28T14:09:29.717625+0000', 'last_clean': '2024-05-28T14:09:29.717625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:09:29.732310+0000', 'last_undegraded': '2024-05-28T14:09:29.732310+0000', 'last_fullsized': '2024-05-28T14:09:29.732310+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:09:29.717625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:09:29.717625+0000', 'last_clean_scrub_stamp': '2024-05-28T14:09:29.717625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:09:36.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2024-05-28T14:09:36.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-28T14:09:30.724896+0000', 'last_change': '2024-05-28T14:09:30.724896+0000', 'last_active': '2024-05-28T14:09:30.724896+0000', 'last_peered': '2024-05-28T14:09:30.724896+0000', 'last_clean': '2024-05-28T14:09:30.724896+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:09:30.724896+0000', 'last_undegraded': '2024-05-28T14:09:30.724896+0000', 'last_fullsized': '2024-05-28T14:09:30.724896+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'last_clean_scrub_stamp': '2024-05-28T14:09:30.724896+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-28T14:09:36.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2024-05-28T14:09:36.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:09:29.731567+0000', 'last_change': '2024-05-28T14:09:29.731567+0000', 'last_active': '2024-05-28T14:09:29.717625+0000', 'last_peered': '2024-05-28T14:09:29.717625+0000', 'last_clean': '2024-05-28T14:09:29.717625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:09:29.731567+0000', 'last_undegraded': '2024-05-28T14:09:29.731567+0000', 'last_fullsized': '2024-05-28T14:09:29.731567+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:09:29.717625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:09:29.717625+0000', 'last_clean_scrub_stamp': '2024-05-28T14:09:29.717625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:09:36.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-28T14:09:36.704 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-28T14:09:36.718+0000 7f5c73600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-28T14:09:36.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.841+0000 7f08b3200640 1 Processor -- start 2024-05-28T14:09:36.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.841+0000 7f08b3200640 1 -- start start 2024-05-28T14:09:36.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08b3200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f08ac15e880 0x7f08ac0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08b3200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f08ac0d1780 0x7f08ac0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08b3200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08ac0b1290 con 0x7f08ac0d5360 2024-05-28T14:09:36.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08b3200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f08ac0b1ba0 con 0x7f08ac15e880 2024-05-28T14:09:36.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08b3200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f08ac0d0610 con 0x7f08ac0d1780 2024-05-28T14:09:36.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08abe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f08ac15e880 0x7f08ac0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08abe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f08ac15e880 0x7f08ac0cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56444/0 (socket says 172.21.15.42:56444) 2024-05-28T14:09:36.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08abe00640 1 -- 172.21.15.42:0/3050782231 learned_addr learned my addr 172.21.15.42:0/3050782231 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:36.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08a3400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f08ac0d5360 0x7f08ac0d1070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:47340/0 (socket says 172.21.15.42:47340) 2024-05-28T14:09:36.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08b1200640 1 --2- 172.21.15.42:0/3050782231 >> v2:172.21.15.63:3300/0 conn(0x7f08ac0d1780 0x7f08ac0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08abe00640 1 -- 172.21.15.42:0/3050782231 >> v2:172.21.15.63:3300/0 conn(0x7f08ac0d1780 msgr2=0x7f08ac0b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:36.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08abe00640 1 --2- 172.21.15.42:0/3050782231 >> v2:172.21.15.63:3300/0 conn(0x7f08ac0d1780 0x7f08ac0b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08abe00640 1 -- 172.21.15.42:0/3050782231 >> v1:172.21.15.42:6789/0 conn(0x7f08ac0d5360 legacy=0x7f08ac0d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08abe00640 1 -- 172.21.15.42:0/3050782231 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac0b1550 con 0x7f08ac15e880 2024-05-28T14:09:36.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08ab400640 1 -- 172.21.15.42:0/3050782231 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2184441508 0 0) 0x7f08ac0b1290 con 0x7f08ac0d5360 2024-05-28T14:09:36.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.845+0000 7f08b1200640 1 --2- 172.21.15.42:0/3050782231 >> v2:172.21.15.63:3300/0 conn(0x7f08ac0d1780 0x7f08ac0b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:36.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08abe00640 1 --2- 172.21.15.42:0/3050782231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac15e880 0x7f08ac0cfd30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f08ac08c3f0 tx=0x7f089c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20207667e5f9d4ab server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:36.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08ab400640 1 -- 172.21.15.42:0/3050782231 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c0b5070 con 0x7f08ac15e880 2024-05-28T14:09:36.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08ab400640 1 -- 172.21.15.42:0/3050782231 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f089c0a8a10 con 0x7f08ac15e880 2024-05-28T14:09:36.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08ab400640 1 -- 172.21.15.42:0/3050782231 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c0a8de0 con 0x7f08ac15e880 2024-05-28T14:09:36.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 -- 172.21.15.42:0/3050782231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac15e880 msgr2=0x7f08ac0cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 --2- 172.21.15.42:0/3050782231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac15e880 0x7f08ac0cfd30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f08ac08c3f0 tx=0x7f089c01ed10 comp rx=0 tx=0).stop 2024-05-28T14:09:36.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 -- 172.21.15.42:0/3050782231 shutdown_connections 2024-05-28T14:09:36.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 --2- 172.21.15.42:0/3050782231 >> v2:172.21.15.63:3300/0 conn(0x7f08ac0d1780 0x7f08ac0b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 --2- 172.21.15.42:0/3050782231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac15e880 0x7f08ac0cfd30 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 -- 172.21.15.42:0/3050782231 >> 172.21.15.42:0/3050782231 conn(0x7f08ac08bfe0 msgr2=0x7f08ac0d4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:36.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 -- 172.21.15.42:0/3050782231 shutdown_connections 2024-05-28T14:09:36.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 -- 172.21.15.42:0/3050782231 wait complete. 2024-05-28T14:09:36.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 Processor -- start 2024-05-28T14:09:36.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 -- start start 2024-05-28T14:09:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08ac0d1780 0x7f08ac1f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0d5360 0x7f08ac0cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac15e880 0x7f08ac0cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac0d03f0 con 0x7f08ac0d5360 2024-05-28T14:09:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac0d0270 con 0x7f08ac0d1780 2024-05-28T14:09:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.849+0000 7f08b3200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f08ac15ec60 con 0x7f08ac15e880 2024-05-28T14:09:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08a3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0d5360 0x7f08ac0cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08a3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0d5360 0x7f08ac0cbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56452/0 (socket says 172.21.15.42:56452) 2024-05-28T14:09:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08a3400640 1 -- 172.21.15.42:0/634720305 learned_addr learned my addr 172.21.15.42:0/634720305 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08abe00640 1 --2- 172.21.15.42:0/634720305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08ac0d1780 0x7f08ac1f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08b1200640 1 --2- 172.21.15.42:0/634720305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac15e880 0x7f08ac0cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08a3400640 1 -- 172.21.15.42:0/634720305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac15e880 msgr2=0x7f08ac0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08a3400640 1 --2- 172.21.15.42:0/634720305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac15e880 0x7f08ac0cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08a3400640 1 -- 172.21.15.42:0/634720305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08ac0d1780 msgr2=0x7f08ac1f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08a3400640 1 --2- 172.21.15.42:0/634720305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08ac0d1780 0x7f08ac1f7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08a3400640 1 -- 172.21.15.42:0/634720305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac0cc550 con 0x7f08ac0d5360 2024-05-28T14:09:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08a3400640 1 --2- 172.21.15.42:0/634720305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0d5360 0x7f08ac0cbad0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f089400c520 tx=0x7f089400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08a8c00640 1 -- 172.21.15.42:0/634720305 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894018070 con 0x7f08ac0d5360 2024-05-28T14:09:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08a8c00640 1 -- 172.21.15.42:0/634720305 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08940074e0 con 0x7f08ac0d5360 2024-05-28T14:09:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08a8c00640 1 -- 172.21.15.42:0/634720305 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08940049f0 con 0x7f08ac0d5360 2024-05-28T14:09:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08b3200640 1 -- 172.21.15.42:0/634720305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ac0cc800 con 0x7f08ac0d5360 2024-05-28T14:09:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08b3200640 1 -- 172.21.15.42:0/634720305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ac0ccdc0 con 0x7f08ac0d5360 2024-05-28T14:09:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08b3200640 1 -- 172.21.15.42:0/634720305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08ac068a90 con 0x7f08ac0d5360 2024-05-28T14:09:36.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.853+0000 7f08a8c00640 1 -- 172.21.15.42:0/634720305 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f0894004b90 con 0x7f08ac0d5360 2024-05-28T14:09:36.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.857+0000 7f08a8c00640 1 --2- 172.21.15.42:0/634720305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f08880694f0 0x7f088806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:36.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.857+0000 7f08a8c00640 1 -- 172.21.15.42:0/634720305 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7f0894084ae0 con 0x7f08ac0d5360 2024-05-28T14:09:36.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.857+0000 7f08abe00640 1 --2- 172.21.15.42:0/634720305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f08880694f0 0x7f088806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:36.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.857+0000 7f08abe00640 1 --2- 172.21.15.42:0/634720305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f08880694f0 0x7f088806b9b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f089c0b2000 tx=0x7f089c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:36.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:36.861+0000 7f08a8c00640 1 -- 172.21.15.42:0/634720305 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0894056150 con 0x7f08ac0d5360 2024-05-28T14:09:37.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.081+0000 7f08b3200640 1 -- 172.21.15.42:0/634720305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f08ac06d600 con 0x7f08ac0d5360 2024-05-28T14:09:37.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.081+0000 7f08a8c00640 1 -- 172.21.15.42:0/634720305 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1177 (secure 0 0 0) 0x7f089405a450 con 0x7f08ac0d5360 2024-05-28T14:09:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:37.065 INFO:teuthology.orchestra.run.smithi042.stdout:{"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":698,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1716904689,"num_in_osds":8,"osd_in_since":1716904680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":86},{"state_name":"unknown","count":9},{"state_name":"creating+peering","count":2}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":873422848,"bytes_avail":772220690432,"bytes_total":773094113280,"unknown_pgs_ratio":0.092783503234386444,"inactive_pgs_ratio":0.020618556067347527,"write_bytes_sec":2660,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":54,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":56621}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-05-28T14:07:33.038180+0000","services":{}},"progress_events":{}} 2024-05-28T14:09:37.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.085+0000 7f08b3200640 1 -- 172.21.15.42:0/634720305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f08880694f0 msgr2=0x7f088806b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:37.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.085+0000 7f08b3200640 1 --2- 172.21.15.42:0/634720305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f08880694f0 0x7f088806b9b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f089c0b2000 tx=0x7f089c009040 comp rx=0 tx=0).stop 2024-05-28T14:09:37.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.085+0000 7f08b3200640 1 -- 172.21.15.42:0/634720305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0d5360 msgr2=0x7f08ac0cbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:37.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.085+0000 7f08b3200640 1 --2- 172.21.15.42:0/634720305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0d5360 0x7f08ac0cbad0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f089400c520 tx=0x7f089400c9e0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.085+0000 7f08b3200640 1 -- 172.21.15.42:0/634720305 shutdown_connections 2024-05-28T14:09:37.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.085+0000 7f08b3200640 1 --2- 172.21.15.42:0/634720305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f08880694f0 0x7f088806b9b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.085+0000 7f08b3200640 1 --2- 172.21.15.42:0/634720305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac15e880 0x7f08ac0cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.085+0000 7f08b3200640 1 --2- 172.21.15.42:0/634720305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0d5360 0x7f08ac0cbad0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.085+0000 7f08b3200640 1 --2- 172.21.15.42:0/634720305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08ac0d1780 0x7f08ac1f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.085+0000 7f08b3200640 1 -- 172.21.15.42:0/634720305 >> 172.21.15.42:0/634720305 conn(0x7f08ac08bfe0 msgr2=0x7f08ac15d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:37.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.085+0000 7f08b3200640 1 -- 172.21.15.42:0/634720305 shutdown_connections 2024-05-28T14:09:37.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.085+0000 7f08b3200640 1 -- 172.21.15.42:0/634720305 wait complete. 2024-05-28T14:09:37.087 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-28T14:09:37.087 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:09:37.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.245+0000 7f7463600640 1 Processor -- start 2024-05-28T14:09:37.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.245+0000 7f7463600640 1 -- start start 2024-05-28T14:09:37.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.245+0000 7f7463600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f745c0d0420 0x7f745c15ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:37.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.245+0000 7f7463600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f745c067810 0x7f745c0b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:37.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.245+0000 7f7463600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f745c06eaa0 con 0x7f745c0b4110 2024-05-28T14:09:37.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.245+0000 7f7463600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f745c06ed80 con 0x7f745c067810 2024-05-28T14:09:37.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.245+0000 7f7463600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f745c06f060 con 0x7f745c0d0420 2024-05-28T14:09:37.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.245+0000 7f7453e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f745c067810 0x7f745c0b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:37.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.245+0000 7f7453e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f745c067810 0x7f745c0b3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56454/0 (socket says 172.21.15.42:56454) 2024-05-28T14:09:37.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.245+0000 7f7453e00640 1 -- 172.21.15.42:0/4077507968 learned_addr learned my addr 172.21.15.42:0/4077507968 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:37.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.245+0000 7f7460c00640 1 --2- 172.21.15.42:0/4077507968 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f745c0d0420 0x7f745c15ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:37.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.245+0000 7f7453e00640 1 -- 172.21.15.42:0/4077507968 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f745c0d0420 msgr2=0x7f745c15ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:37.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.245+0000 7f7453e00640 1 --2- 172.21.15.42:0/4077507968 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f745c0d0420 0x7f745c15ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.245+0000 7f7453e00640 1 -- 172.21.15.42:0/4077507968 >> v1:172.21.15.63:6789/0 conn(0x7f745c0b4110 legacy=0x7f745c162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:37.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.249+0000 7f7453e00640 1 -- 172.21.15.42:0/4077507968 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c06f340 con 0x7f745c067810 2024-05-28T14:09:37.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.249+0000 7f7453e00640 1 --2- 172.21.15.42:0/4077507968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f745c067810 0x7f745c0b3bd0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f7444009870 tx=0x7f744401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f9b2823e892c27 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:37.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.249+0000 7f745be00640 1 -- 172.21.15.42:0/4077507968 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74440b0070 con 0x7f745c067810 2024-05-28T14:09:37.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.249+0000 7f745be00640 1 -- 172.21.15.42:0/4077507968 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74440ab600 con 0x7f745c067810 2024-05-28T14:09:37.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.249+0000 7f745be00640 1 -- 172.21.15.42:0/4077507968 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74440ab9d0 con 0x7f745c067810 2024-05-28T14:09:37.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.249+0000 7f7463600640 1 -- 172.21.15.42:0/4077507968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f745c067810 msgr2=0x7f745c0b3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:37.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.249+0000 7f7463600640 1 --2- 172.21.15.42:0/4077507968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f745c067810 0x7f745c0b3bd0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f7444009870 tx=0x7f744401c490 comp rx=0 tx=0).stop 2024-05-28T14:09:37.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.249+0000 7f7463600640 1 -- 172.21.15.42:0/4077507968 shutdown_connections 2024-05-28T14:09:37.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.249+0000 7f7463600640 1 --2- 172.21.15.42:0/4077507968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f745c067810 0x7f745c0b3bd0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.249+0000 7f7463600640 1 --2- 172.21.15.42:0/4077507968 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f745c0d0420 0x7f745c15ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.249+0000 7f7463600640 1 -- 172.21.15.42:0/4077507968 >> 172.21.15.42:0/4077507968 conn(0x7f745c0d0b40 msgr2=0x7f745c0d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:37.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.249+0000 7f7463600640 1 -- 172.21.15.42:0/4077507968 shutdown_connections 2024-05-28T14:09:37.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.249+0000 7f7463600640 1 -- 172.21.15.42:0/4077507968 wait complete. 2024-05-28T14:09:37.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7463600640 1 Processor -- start 2024-05-28T14:09:37.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7463600640 1 -- start start 2024-05-28T14:09:37.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7463600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f745c067810 0x7f745c155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:37.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7463600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f745c0b4110 0x7f745c152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:37.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7463600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f745c0d0420 0x7f745c150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:37.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7463600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f745c16b640 con 0x7f745c067810 2024-05-28T14:09:37.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7463600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f745c16b4c0 con 0x7f745c0d0420 2024-05-28T14:09:37.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7463600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f745c16b7c0 con 0x7f745c0b4110 2024-05-28T14:09:37.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7460c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f745c067810 0x7f745c155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:37.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7460c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f745c067810 0x7f745c155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56464/0 (socket says 172.21.15.42:56464) 2024-05-28T14:09:37.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7460c00640 1 -- 172.21.15.42:0/1913480837 learned_addr learned my addr 172.21.15.42:0/1913480837 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:37.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7453e00640 1 --2- 172.21.15.42:0/1913480837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f745c0b4110 0x7f745c152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:37.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7461600640 1 --2- 172.21.15.42:0/1913480837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f745c0d0420 0x7f745c150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:37.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7460c00640 1 -- 172.21.15.42:0/1913480837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f745c0b4110 msgr2=0x7f745c152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:37.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7460c00640 1 --2- 172.21.15.42:0/1913480837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f745c0b4110 0x7f745c152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7460c00640 1 -- 172.21.15.42:0/1913480837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f745c0d0420 msgr2=0x7f745c150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:37.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7460c00640 1 --2- 172.21.15.42:0/1913480837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f745c0d0420 0x7f745c150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7460c00640 1 -- 172.21.15.42:0/1913480837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c152ca0 con 0x7f745c067810 2024-05-28T14:09:37.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7460c00640 1 --2- 172.21.15.42:0/1913480837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f745c067810 0x7f745c155110 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f744c00d620 tx=0x7f744c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:37.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7459600640 1 -- 172.21.15.42:0/1913480837 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c00b620 con 0x7f745c067810 2024-05-28T14:09:37.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7459600640 1 -- 172.21.15.42:0/1913480837 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f744c019070 con 0x7f745c067810 2024-05-28T14:09:37.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7459600640 1 -- 172.21.15.42:0/1913480837 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c00b9e0 con 0x7f745c067810 2024-05-28T14:09:37.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7463600640 1 -- 172.21.15.42:0/1913480837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c152f80 con 0x7f745c067810 2024-05-28T14:09:37.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.253+0000 7f7463600640 1 -- 172.21.15.42:0/1913480837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c153540 con 0x7f745c067810 2024-05-28T14:09:37.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.257+0000 7f7463600640 1 -- 172.21.15.42:0/1913480837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f745c068a90 con 0x7f745c067810 2024-05-28T14:09:37.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.257+0000 7f7459600640 1 -- 172.21.15.42:0/1913480837 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f744c00bb80 con 0x7f745c067810 2024-05-28T14:09:37.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.257+0000 7f7459600640 1 --2- 172.21.15.42:0/1913480837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f74380694f0 0x7f743806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:37.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.257+0000 7f7459600640 1 -- 172.21.15.42:0/1913480837 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7f744c085b50 con 0x7f745c067810 2024-05-28T14:09:37.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.257+0000 7f7453e00640 1 --2- 172.21.15.42:0/1913480837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f74380694f0 0x7f743806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:37.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.257+0000 7f7453e00640 1 --2- 172.21.15.42:0/1913480837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f74380694f0 0x7f743806b9b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f74440ad000 tx=0x7f7444008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:37.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.261+0000 7f7459600640 1 -- 172.21.15.42:0/1913480837 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f744c0571c0 con 0x7f745c067810 2024-05-28T14:09:37.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.421+0000 7f7463600640 1 -- 172.21.15.42:0/1913480837 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f745c06d5e0 con 0x7f74380694f0 2024-05-28T14:09:37.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.429+0000 7f7459600640 1 -- 172.21.15.42:0/1913480837 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252102 (secure 0 0 0) 0x7f745c06d5e0 con 0x7f74380694f0 2024-05-28T14:09:37.415 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:37.416 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:09:37.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.433+0000 7f7463600640 1 -- 172.21.15.42:0/1913480837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f74380694f0 msgr2=0x7f743806b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.433+0000 7f7463600640 1 --2- 172.21.15.42:0/1913480837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f74380694f0 0x7f743806b9b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f74440ad000 tx=0x7f7444008040 comp rx=0 tx=0).stop 2024-05-28T14:09:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.433+0000 7f7463600640 1 -- 172.21.15.42:0/1913480837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f745c067810 msgr2=0x7f745c155110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.433+0000 7f7463600640 1 --2- 172.21.15.42:0/1913480837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f745c067810 0x7f745c155110 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f744c00d620 tx=0x7f744c00dae0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.433+0000 7f7463600640 1 -- 172.21.15.42:0/1913480837 shutdown_connections 2024-05-28T14:09:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.433+0000 7f7463600640 1 --2- 172.21.15.42:0/1913480837 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f74380694f0 0x7f743806b9b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.433+0000 7f7463600640 1 --2- 172.21.15.42:0/1913480837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f745c0d0420 0x7f745c150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.433+0000 7f7463600640 1 --2- 172.21.15.42:0/1913480837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f745c0b4110 0x7f745c152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.433+0000 7f7463600640 1 --2- 172.21.15.42:0/1913480837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f745c067810 0x7f745c155110 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.433+0000 7f7463600640 1 -- 172.21.15.42:0/1913480837 >> 172.21.15.42:0/1913480837 conn(0x7f745c0d0b40 msgr2=0x7f745c072600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.433+0000 7f7463600640 1 -- 172.21.15.42:0/1913480837 shutdown_connections 2024-05-28T14:09:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:37.433+0000 7f7463600640 1 -- 172.21.15.42:0/1913480837 wait complete. 2024-05-28T14:09:37.435 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-05-28T14:09:37.113208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":853000,"kb_used_data":9864,"kb_used_omap":12,"kb_used_meta":842995,"kb_avail":754121720,"statfs":{"total":773094113280,"available":772220641280,"internally_reserved":0,"allocated":10100736,"data_stored":5011740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":863227442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.380272"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151546+0000","last_change":"2024-05-28T14:09:31.763443+0000","last_active":"2024-05-28T14:09:33.151546+0000","last_peered":"2024-05-28T14:09:33.151546+0000","last_clean":"2024-05-28T14:09:33.151546+0000","last_became_active":"2024-05-28T14:09:31.763080+0000","last_became_peered":"2024-05-28T14:09:31.763080+0000","last_unstale":"2024-05-28T14:09:33.151546+0000","last_undegraded":"2024-05-28T14:09:33.151546+0000","last_fullsized":"2024-05-28T14:09:33.151546+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762829+0000","last_change":"2024-05-28T14:09:31.768406+0000","last_active":"2024-05-28T14:09:32.762829+0000","last_peered":"2024-05-28T14:09:32.762829+0000","last_clean":"2024-05-28T14:09:32.762829+0000","last_became_active":"2024-05-28T14:09:31.768150+0000","last_became_peered":"2024-05-28T14:09:31.768150+0000","last_unstale":"2024-05-28T14:09:32.762829+0000","last_undegraded":"2024-05-28T14:09:32.762829+0000","last_fullsized":"2024-05-28T14:09:32.762829+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:38.614174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150788+0000","last_change":"2024-05-28T14:09:31.770809+0000","last_active":"2024-05-28T14:09:33.150788+0000","last_peered":"2024-05-28T14:09:33.150788+0000","last_clean":"2024-05-28T14:09:33.150788+0000","last_became_active":"2024-05-28T14:09:31.770547+0000","last_became_peered":"2024-05-28T14:09:31.770547+0000","last_unstale":"2024-05-28T14:09:33.150788+0000","last_undegraded":"2024-05-28T14:09:33.150788+0000","last_fullsized":"2024-05-28T14:09:33.150788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:34:22.994257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149345+0000","last_change":"2024-05-28T14:09:31.761634+0000","last_active":"2024-05-28T14:09:33.149345+0000","last_peered":"2024-05-28T14:09:33.149345+0000","last_clean":"2024-05-28T14:09:33.149345+0000","last_became_active":"2024-05-28T14:09:31.761270+0000","last_became_peered":"2024-05-28T14:09:31.761270+0000","last_unstale":"2024-05-28T14:09:33.149345+0000","last_undegraded":"2024-05-28T14:09:33.149345+0000","last_fullsized":"2024-05-28T14:09:33.149345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:43.050090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751824+0000","last_change":"2024-05-28T14:09:31.756701+0000","last_active":"2024-05-28T14:09:32.751824+0000","last_peered":"2024-05-28T14:09:32.751824+0000","last_clean":"2024-05-28T14:09:32.751824+0000","last_became_active":"2024-05-28T14:09:31.756425+0000","last_became_peered":"2024-05-28T14:09:31.756425+0000","last_unstale":"2024-05-28T14:09:32.751824+0000","last_undegraded":"2024-05-28T14:09:32.751824+0000","last_fullsized":"2024-05-28T14:09:32.751824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:43.749597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750865+0000","last_change":"2024-05-28T14:09:31.756691+0000","last_active":"2024-05-28T14:09:32.750865+0000","last_peered":"2024-05-28T14:09:32.750865+0000","last_clean":"2024-05-28T14:09:32.750865+0000","last_became_active":"2024-05-28T14:09:31.756434+0000","last_became_peered":"2024-05-28T14:09:31.756434+0000","last_unstale":"2024-05-28T14:09:32.750865+0000","last_undegraded":"2024-05-28T14:09:32.750865+0000","last_fullsized":"2024-05-28T14:09:32.750865+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149073+0000","last_change":"2024-05-28T14:09:31.757041+0000","last_active":"2024-05-28T14:09:33.149073+0000","last_peered":"2024-05-28T14:09:33.149073+0000","last_clean":"2024-05-28T14:09:33.149073+0000","last_became_active":"2024-05-28T14:09:31.756717+0000","last_became_peered":"2024-05-28T14:09:31.756717+0000","last_unstale":"2024-05-28T14:09:33.149073+0000","last_undegraded":"2024-05-28T14:09:33.149073+0000","last_fullsized":"2024-05-28T14:09:33.149073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:34:26.316657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751086+0000","last_change":"2024-05-28T14:09:31.761009+0000","last_active":"2024-05-28T14:09:32.751086+0000","last_peered":"2024-05-28T14:09:32.751086+0000","last_clean":"2024-05-28T14:09:32.751086+0000","last_became_active":"2024-05-28T14:09:31.760693+0000","last_became_peered":"2024-05-28T14:09:31.760693+0000","last_unstale":"2024-05-28T14:09:32.751086+0000","last_undegraded":"2024-05-28T14:09:32.751086+0000","last_fullsized":"2024-05-28T14:09:32.751086+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751130+0000","last_change":"2024-05-28T14:09:31.748781+0000","last_active":"2024-05-28T14:09:32.751130+0000","last_peered":"2024-05-28T14:09:32.751130+0000","last_clean":"2024-05-28T14:09:32.751130+0000","last_became_active":"2024-05-28T14:09:31.748446+0000","last_became_peered":"2024-05-28T14:09:31.748446+0000","last_unstale":"2024-05-28T14:09:32.751130+0000","last_undegraded":"2024-05-28T14:09:32.751130+0000","last_fullsized":"2024-05-28T14:09:32.751130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151416+0000","last_change":"2024-05-28T14:09:31.755098+0000","last_active":"2024-05-28T14:09:33.151416+0000","last_peered":"2024-05-28T14:09:33.151416+0000","last_clean":"2024-05-28T14:09:33.151416+0000","last_became_active":"2024-05-28T14:09:31.754790+0000","last_became_peered":"2024-05-28T14:09:31.754790+0000","last_unstale":"2024-05-28T14:09:33.151416+0000","last_undegraded":"2024-05-28T14:09:33.151416+0000","last_fullsized":"2024-05-28T14:09:33.151416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762851+0000","last_change":"2024-05-28T14:09:31.765027+0000","last_active":"2024-05-28T14:09:32.762851+0000","last_peered":"2024-05-28T14:09:32.762851+0000","last_clean":"2024-05-28T14:09:32.762851+0000","last_became_active":"2024-05-28T14:09:31.764660+0000","last_became_peered":"2024-05-28T14:09:31.764660+0000","last_unstale":"2024-05-28T14:09:32.762851+0000","last_undegraded":"2024-05-28T14:09:32.762851+0000","last_fullsized":"2024-05-28T14:09:32.762851+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:59.297622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150769+0000","last_change":"2024-05-28T14:09:31.756022+0000","last_active":"2024-05-28T14:09:33.150769+0000","last_peered":"2024-05-28T14:09:33.150769+0000","last_clean":"2024-05-28T14:09:33.150769+0000","last_became_active":"2024-05-28T14:09:31.755772+0000","last_became_peered":"2024-05-28T14:09:31.755772+0000","last_unstale":"2024-05-28T14:09:33.150769+0000","last_undegraded":"2024-05-28T14:09:33.150769+0000","last_fullsized":"2024-05-28T14:09:33.150769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:35.172304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150463+0000","last_change":"2024-05-28T14:09:31.764389+0000","last_active":"2024-05-28T14:09:33.150463+0000","last_peered":"2024-05-28T14:09:33.150463+0000","last_clean":"2024-05-28T14:09:33.150463+0000","last_became_active":"2024-05-28T14:09:31.764102+0000","last_became_peered":"2024-05-28T14:09:31.764102+0000","last_unstale":"2024-05-28T14:09:33.150463+0000","last_undegraded":"2024-05-28T14:09:33.150463+0000","last_fullsized":"2024-05-28T14:09:33.150463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:37:41.765233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149157+0000","last_change":"2024-05-28T14:09:31.756927+0000","last_active":"2024-05-28T14:09:33.149157+0000","last_peered":"2024-05-28T14:09:33.149157+0000","last_clean":"2024-05-28T14:09:33.149157+0000","last_became_active":"2024-05-28T14:09:31.756570+0000","last_became_peered":"2024-05-28T14:09:31.756570+0000","last_unstale":"2024-05-28T14:09:33.149157+0000","last_undegraded":"2024-05-28T14:09:33.149157+0000","last_fullsized":"2024-05-28T14:09:33.149157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:31:36.894006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747849+0000","last_change":"2024-05-28T14:09:31.766893+0000","last_active":"2024-05-28T14:09:32.747849+0000","last_peered":"2024-05-28T14:09:32.747849+0000","last_clean":"2024-05-28T14:09:32.747849+0000","last_became_active":"2024-05-28T14:09:31.766457+0000","last_became_peered":"2024-05-28T14:09:31.766457+0000","last_unstale":"2024-05-28T14:09:32.747849+0000","last_undegraded":"2024-05-28T14:09:32.747849+0000","last_fullsized":"2024-05-28T14:09:32.747849+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:34:22.994257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751531+0000","last_change":"2024-05-28T14:09:31.753479+0000","last_active":"2024-05-28T14:09:32.751531+0000","last_peered":"2024-05-28T14:09:32.751531+0000","last_clean":"2024-05-28T14:09:32.751531+0000","last_became_active":"2024-05-28T14:09:31.753094+0000","last_became_peered":"2024-05-28T14:09:31.753094+0000","last_unstale":"2024-05-28T14:09:32.751531+0000","last_undegraded":"2024-05-28T14:09:32.751531+0000","last_fullsized":"2024-05-28T14:09:32.751531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:49:42.278616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752189+0000","last_change":"2024-05-28T14:09:31.759451+0000","last_active":"2024-05-28T14:09:32.752189+0000","last_peered":"2024-05-28T14:09:32.752189+0000","last_clean":"2024-05-28T14:09:32.752189+0000","last_became_active":"2024-05-28T14:09:31.759197+0000","last_became_peered":"2024-05-28T14:09:31.759197+0000","last_unstale":"2024-05-28T14:09:32.752189+0000","last_undegraded":"2024-05-28T14:09:32.752189+0000","last_fullsized":"2024-05-28T14:09:32.752189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149434+0000","last_change":"2024-05-28T14:09:31.758504+0000","last_active":"2024-05-28T14:09:33.149434+0000","last_peered":"2024-05-28T14:09:33.149434+0000","last_clean":"2024-05-28T14:09:33.149434+0000","last_became_active":"2024-05-28T14:09:31.757438+0000","last_became_peered":"2024-05-28T14:09:31.757438+0000","last_unstale":"2024-05-28T14:09:33.149434+0000","last_undegraded":"2024-05-28T14:09:33.149434+0000","last_fullsized":"2024-05-28T14:09:33.149434+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:18:46.990801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765154+0000","last_change":"2024-05-28T14:09:31.753754+0000","last_active":"2024-05-28T14:09:32.765154+0000","last_peered":"2024-05-28T14:09:32.765154+0000","last_clean":"2024-05-28T14:09:32.765154+0000","last_became_active":"2024-05-28T14:09:31.753211+0000","last_became_peered":"2024-05-28T14:09:31.753211+0000","last_unstale":"2024-05-28T14:09:32.765154+0000","last_undegraded":"2024-05-28T14:09:32.765154+0000","last_fullsized":"2024-05-28T14:09:32.765154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149445+0000","last_change":"2024-05-28T14:09:31.754140+0000","last_active":"2024-05-28T14:09:33.149445+0000","last_peered":"2024-05-28T14:09:33.149445+0000","last_clean":"2024-05-28T14:09:33.149445+0000","last_became_active":"2024-05-28T14:09:31.753838+0000","last_became_peered":"2024-05-28T14:09:31.753838+0000","last_unstale":"2024-05-28T14:09:33.149445+0000","last_undegraded":"2024-05-28T14:09:33.149445+0000","last_fullsized":"2024-05-28T14:09:33.149445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:29:28.766392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149223+0000","last_change":"2024-05-28T14:09:31.751182+0000","last_active":"2024-05-28T14:09:33.149223+0000","last_peered":"2024-05-28T14:09:33.149223+0000","last_clean":"2024-05-28T14:09:33.149223+0000","last_became_active":"2024-05-28T14:09:31.750664+0000","last_became_peered":"2024-05-28T14:09:31.750664+0000","last_unstale":"2024-05-28T14:09:33.149223+0000","last_undegraded":"2024-05-28T14:09:33.149223+0000","last_fullsized":"2024-05-28T14:09:33.149223+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:24:15.222635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762453+0000","last_change":"2024-05-28T14:09:31.755223+0000","last_active":"2024-05-28T14:09:32.762453+0000","last_peered":"2024-05-28T14:09:32.762453+0000","last_clean":"2024-05-28T14:09:32.762453+0000","last_became_active":"2024-05-28T14:09:31.754880+0000","last_became_peered":"2024-05-28T14:09:31.754880+0000","last_unstale":"2024-05-28T14:09:32.762453+0000","last_undegraded":"2024-05-28T14:09:32.762453+0000","last_fullsized":"2024-05-28T14:09:32.762453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:25.450855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150682+0000","last_change":"2024-05-28T14:09:31.752700+0000","last_active":"2024-05-28T14:09:33.150682+0000","last_peered":"2024-05-28T14:09:33.150682+0000","last_clean":"2024-05-28T14:09:33.150682+0000","last_became_active":"2024-05-28T14:09:31.752367+0000","last_became_peered":"2024-05-28T14:09:31.752367+0000","last_unstale":"2024-05-28T14:09:33.150682+0000","last_undegraded":"2024-05-28T14:09:33.150682+0000","last_fullsized":"2024-05-28T14:09:33.150682+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:43.749597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149066+0000","last_change":"2024-05-28T14:09:31.751477+0000","last_active":"2024-05-28T14:09:33.149066+0000","last_peered":"2024-05-28T14:09:33.149066+0000","last_clean":"2024-05-28T14:09:33.149066+0000","last_became_active":"2024-05-28T14:09:31.751198+0000","last_became_peered":"2024-05-28T14:09:31.751198+0000","last_unstale":"2024-05-28T14:09:33.149066+0000","last_undegraded":"2024-05-28T14:09:33.149066+0000","last_fullsized":"2024-05-28T14:09:33.149066+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:42.364107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762838+0000","last_change":"2024-05-28T14:09:31.756171+0000","last_active":"2024-05-28T14:09:32.762838+0000","last_peered":"2024-05-28T14:09:32.762838+0000","last_clean":"2024-05-28T14:09:32.762838+0000","last_became_active":"2024-05-28T14:09:31.754405+0000","last_became_peered":"2024-05-28T14:09:31.754405+0000","last_unstale":"2024-05-28T14:09:32.762838+0000","last_undegraded":"2024-05-28T14:09:32.762838+0000","last_fullsized":"2024-05-28T14:09:32.762838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:19.325065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150852+0000","last_change":"2024-05-28T14:09:31.760741+0000","last_active":"2024-05-28T14:09:33.150852+0000","last_peered":"2024-05-28T14:09:33.150852+0000","last_clean":"2024-05-28T14:09:33.150852+0000","last_became_active":"2024-05-28T14:09:31.760404+0000","last_became_peered":"2024-05-28T14:09:31.760404+0000","last_unstale":"2024-05-28T14:09:33.150852+0000","last_undegraded":"2024-05-28T14:09:33.150852+0000","last_fullsized":"2024-05-28T14:09:33.150852+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:24.707344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751996+0000","last_change":"2024-05-28T14:09:31.754702+0000","last_active":"2024-05-28T14:09:32.751996+0000","last_peered":"2024-05-28T14:09:32.751996+0000","last_clean":"2024-05-28T14:09:32.751996+0000","last_became_active":"2024-05-28T14:09:31.754215+0000","last_became_peered":"2024-05-28T14:09:31.754215+0000","last_unstale":"2024-05-28T14:09:32.751996+0000","last_undegraded":"2024-05-28T14:09:32.751996+0000","last_fullsized":"2024-05-28T14:09:32.751996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:36:07.311954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151011+0000","last_change":"2024-05-28T14:09:31.765289+0000","last_active":"2024-05-28T14:09:33.151011+0000","last_peered":"2024-05-28T14:09:33.151011+0000","last_clean":"2024-05-28T14:09:33.151011+0000","last_became_active":"2024-05-28T14:09:31.763980+0000","last_became_peered":"2024-05-28T14:09:31.763980+0000","last_unstale":"2024-05-28T14:09:33.151011+0000","last_undegraded":"2024-05-28T14:09:33.151011+0000","last_fullsized":"2024-05-28T14:09:33.151011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762598+0000","last_change":"2024-05-28T14:09:31.758017+0000","last_active":"2024-05-28T14:09:32.762598+0000","last_peered":"2024-05-28T14:09:32.762598+0000","last_clean":"2024-05-28T14:09:32.762598+0000","last_became_active":"2024-05-28T14:09:31.757674+0000","last_became_peered":"2024-05-28T14:09:31.757674+0000","last_unstale":"2024-05-28T14:09:32.762598+0000","last_undegraded":"2024-05-28T14:09:32.762598+0000","last_fullsized":"2024-05-28T14:09:32.762598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:46.760174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762477+0000","last_change":"2024-05-28T14:09:31.762403+0000","last_active":"2024-05-28T14:09:32.762477+0000","last_peered":"2024-05-28T14:09:32.762477+0000","last_clean":"2024-05-28T14:09:32.762477+0000","last_became_active":"2024-05-28T14:09:31.762048+0000","last_became_peered":"2024-05-28T14:09:31.762048+0000","last_unstale":"2024-05-28T14:09:32.762477+0000","last_undegraded":"2024-05-28T14:09:32.762477+0000","last_fullsized":"2024-05-28T14:09:32.762477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:56.072759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751599+0000","last_change":"2024-05-28T14:09:31.747630+0000","last_active":"2024-05-28T14:09:32.751599+0000","last_peered":"2024-05-28T14:09:32.751599+0000","last_clean":"2024-05-28T14:09:32.751599+0000","last_became_active":"2024-05-28T14:09:31.747183+0000","last_became_peered":"2024-05-28T14:09:31.747183+0000","last_unstale":"2024-05-28T14:09:32.751599+0000","last_undegraded":"2024-05-28T14:09:32.751599+0000","last_fullsized":"2024-05-28T14:09:32.751599+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:49:44.321180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151402+0000","last_change":"2024-05-28T14:09:31.768533+0000","last_active":"2024-05-28T14:09:33.151402+0000","last_peered":"2024-05-28T14:09:33.151402+0000","last_clean":"2024-05-28T14:09:33.151402+0000","last_became_active":"2024-05-28T14:09:31.768237+0000","last_became_peered":"2024-05-28T14:09:31.768237+0000","last_unstale":"2024-05-28T14:09:33.151402+0000","last_undegraded":"2024-05-28T14:09:33.151402+0000","last_fullsized":"2024-05-28T14:09:33.151402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752209+0000","last_change":"2024-05-28T14:09:31.752686+0000","last_active":"2024-05-28T14:09:32.752209+0000","last_peered":"2024-05-28T14:09:32.752209+0000","last_clean":"2024-05-28T14:09:32.752209+0000","last_became_active":"2024-05-28T14:09:31.752411+0000","last_became_peered":"2024-05-28T14:09:31.752411+0000","last_unstale":"2024-05-28T14:09:32.752209+0000","last_undegraded":"2024-05-28T14:09:32.752209+0000","last_fullsized":"2024-05-28T14:09:32.752209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151595+0000","last_change":"2024-05-28T14:09:30.759880+0000","last_active":"2024-05-28T14:09:33.151595+0000","last_peered":"2024-05-28T14:09:33.151595+0000","last_clean":"2024-05-28T14:09:33.151595+0000","last_became_active":"2024-05-28T14:09:30.759575+0000","last_became_peered":"2024-05-28T14:09:30.759575+0000","last_unstale":"2024-05-28T14:09:33.151595+0000","last_undegraded":"2024-05-28T14:09:33.151595+0000","last_fullsized":"2024-05-28T14:09:33.151595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:55.065487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151493+0000","last_change":"2024-05-28T14:09:31.752593+0000","last_active":"2024-05-28T14:09:33.151493+0000","last_peered":"2024-05-28T14:09:33.151493+0000","last_clean":"2024-05-28T14:09:33.151493+0000","last_became_active":"2024-05-28T14:09:31.752332+0000","last_became_peered":"2024-05-28T14:09:31.752332+0000","last_unstale":"2024-05-28T14:09:33.151493+0000","last_undegraded":"2024-05-28T14:09:33.151493+0000","last_fullsized":"2024-05-28T14:09:33.151493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752466+0000","last_change":"2024-05-28T14:09:30.755821+0000","last_active":"2024-05-28T14:09:32.752466+0000","last_peered":"2024-05-28T14:09:32.752466+0000","last_clean":"2024-05-28T14:09:32.752466+0000","last_became_active":"2024-05-28T14:09:30.755452+0000","last_became_peered":"2024-05-28T14:09:30.755452+0000","last_unstale":"2024-05-28T14:09:32.752466+0000","last_undegraded":"2024-05-28T14:09:32.752466+0000","last_fullsized":"2024-05-28T14:09:32.752466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:18.317793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151220+0000","last_change":"2024-05-28T14:09:31.765522+0000","last_active":"2024-05-28T14:09:33.151220+0000","last_peered":"2024-05-28T14:09:33.151220+0000","last_clean":"2024-05-28T14:09:33.151220+0000","last_became_active":"2024-05-28T14:09:31.764349+0000","last_became_peered":"2024-05-28T14:09:31.764349+0000","last_unstale":"2024-05-28T14:09:33.151220+0000","last_undegraded":"2024-05-28T14:09:33.151220+0000","last_fullsized":"2024-05-28T14:09:33.151220+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765049+0000","last_change":"2024-05-28T14:09:30.752545+0000","last_active":"2024-05-28T14:09:32.765049+0000","last_peered":"2024-05-28T14:09:32.765049+0000","last_clean":"2024-05-28T14:09:32.765049+0000","last_became_active":"2024-05-28T14:09:30.752219+0000","last_became_peered":"2024-05-28T14:09:30.752219+0000","last_unstale":"2024-05-28T14:09:32.765049+0000","last_undegraded":"2024-05-28T14:09:32.765049+0000","last_fullsized":"2024-05-28T14:09:32.765049+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:49:43.313909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151148+0000","last_change":"2024-05-28T14:09:31.765767+0000","last_active":"2024-05-28T14:09:33.151148+0000","last_peered":"2024-05-28T14:09:33.151148+0000","last_clean":"2024-05-28T14:09:33.151148+0000","last_became_active":"2024-05-28T14:09:31.765407+0000","last_became_peered":"2024-05-28T14:09:31.765407+0000","last_unstale":"2024-05-28T14:09:33.151148+0000","last_undegraded":"2024-05-28T14:09:33.151148+0000","last_fullsized":"2024-05-28T14:09:33.151148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:27:09.475191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764918+0000","last_change":"2024-05-28T14:09:30.754683+0000","last_active":"2024-05-28T14:09:32.764918+0000","last_peered":"2024-05-28T14:09:32.764918+0000","last_clean":"2024-05-28T14:09:32.764918+0000","last_became_active":"2024-05-28T14:09:30.754405+0000","last_became_peered":"2024-05-28T14:09:30.754405+0000","last_unstale":"2024-05-28T14:09:32.764918+0000","last_undegraded":"2024-05-28T14:09:32.764918+0000","last_fullsized":"2024-05-28T14:09:32.764918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:29:45.272413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762565+0000","last_change":"2024-05-28T14:09:31.762430+0000","last_active":"2024-05-28T14:09:32.762565+0000","last_peered":"2024-05-28T14:09:32.762565+0000","last_clean":"2024-05-28T14:09:32.762565+0000","last_became_active":"2024-05-28T14:09:31.762101+0000","last_became_peered":"2024-05-28T14:09:31.762101+0000","last_unstale":"2024-05-28T14:09:32.762565+0000","last_undegraded":"2024-05-28T14:09:32.762565+0000","last_fullsized":"2024-05-28T14:09:32.762565+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:40:39.401890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752066+0000","last_change":"2024-05-28T14:09:30.747550+0000","last_active":"2024-05-28T14:09:32.752066+0000","last_peered":"2024-05-28T14:09:32.752066+0000","last_clean":"2024-05-28T14:09:32.752066+0000","last_became_active":"2024-05-28T14:09:30.747222+0000","last_became_peered":"2024-05-28T14:09:30.747222+0000","last_unstale":"2024-05-28T14:09:32.752066+0000","last_undegraded":"2024-05-28T14:09:32.752066+0000","last_fullsized":"2024-05-28T14:09:32.752066+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:58.290350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751952+0000","last_change":"2024-05-28T14:09:31.759538+0000","last_active":"2024-05-28T14:09:32.751952+0000","last_peered":"2024-05-28T14:09:32.751952+0000","last_clean":"2024-05-28T14:09:32.751952+0000","last_became_active":"2024-05-28T14:09:31.759258+0000","last_became_peered":"2024-05-28T14:09:31.759258+0000","last_unstale":"2024-05-28T14:09:32.751952+0000","last_undegraded":"2024-05-28T14:09:32.751952+0000","last_fullsized":"2024-05-28T14:09:32.751952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:35.172304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751683+0000","last_change":"2024-05-28T14:09:30.759103+0000","last_active":"2024-05-28T14:09:32.751683+0000","last_peered":"2024-05-28T14:09:32.751683+0000","last_clean":"2024-05-28T14:09:32.751683+0000","last_became_active":"2024-05-28T14:09:30.758628+0000","last_became_peered":"2024-05-28T14:09:30.758628+0000","last_unstale":"2024-05-28T14:09:32.751683+0000","last_undegraded":"2024-05-28T14:09:32.751683+0000","last_fullsized":"2024-05-28T14:09:32.751683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:54.698345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150467+0000","last_change":"2024-05-28T14:09:31.761382+0000","last_active":"2024-05-28T14:09:33.150467+0000","last_peered":"2024-05-28T14:09:33.150467+0000","last_clean":"2024-05-28T14:09:33.150467+0000","last_became_active":"2024-05-28T14:09:31.761095+0000","last_became_peered":"2024-05-28T14:09:31.761095+0000","last_unstale":"2024-05-28T14:09:33.150467+0000","last_undegraded":"2024-05-28T14:09:33.150467+0000","last_fullsized":"2024-05-28T14:09:33.150467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:31:10.786309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764699+0000","last_change":"2024-05-28T14:09:30.762673+0000","last_active":"2024-05-28T14:09:32.764699+0000","last_peered":"2024-05-28T14:09:32.764699+0000","last_clean":"2024-05-28T14:09:32.764699+0000","last_became_active":"2024-05-28T14:09:30.761793+0000","last_became_peered":"2024-05-28T14:09:30.761793+0000","last_unstale":"2024-05-28T14:09:32.764699+0000","last_undegraded":"2024-05-28T14:09:32.764699+0000","last_fullsized":"2024-05-28T14:09:32.764699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:40:25.005440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747896+0000","last_change":"2024-05-28T14:09:31.766958+0000","last_active":"2024-05-28T14:09:32.747896+0000","last_peered":"2024-05-28T14:09:32.747896+0000","last_clean":"2024-05-28T14:09:32.747896+0000","last_became_active":"2024-05-28T14:09:31.766301+0000","last_became_peered":"2024-05-28T14:09:31.766301+0000","last_unstale":"2024-05-28T14:09:32.747896+0000","last_undegraded":"2024-05-28T14:09:32.747896+0000","last_fullsized":"2024-05-28T14:09:32.747896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:24.707344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762264+0000","last_change":"2024-05-28T14:09:30.746473+0000","last_active":"2024-05-28T14:09:32.762264+0000","last_peered":"2024-05-28T14:09:32.762264+0000","last_clean":"2024-05-28T14:09:32.762264+0000","last_became_active":"2024-05-28T14:09:30.746177+0000","last_became_peered":"2024-05-28T14:09:30.746177+0000","last_unstale":"2024-05-28T14:09:32.762264+0000","last_undegraded":"2024-05-28T14:09:32.762264+0000","last_fullsized":"2024-05-28T14:09:32.762264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:41.356836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.748332+0000","last_change":"2024-05-28T14:09:31.756713+0000","last_active":"2024-05-28T14:09:32.748332+0000","last_peered":"2024-05-28T14:09:32.748332+0000","last_clean":"2024-05-28T14:09:32.748332+0000","last_became_active":"2024-05-28T14:09:31.755716+0000","last_became_peered":"2024-05-28T14:09:31.755716+0000","last_unstale":"2024-05-28T14:09:32.748332+0000","last_undegraded":"2024-05-28T14:09:32.748332+0000","last_fullsized":"2024-05-28T14:09:32.748332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:02:06.074104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151668+0000","last_change":"2024-05-28T14:09:30.758732+0000","last_active":"2024-05-28T14:09:33.151668+0000","last_peered":"2024-05-28T14:09:33.151668+0000","last_clean":"2024-05-28T14:09:33.151668+0000","last_became_active":"2024-05-28T14:09:30.758400+0000","last_became_peered":"2024-05-28T14:09:30.758400+0000","last_unstale":"2024-05-28T14:09:33.151668+0000","last_undegraded":"2024-05-28T14:09:33.151668+0000","last_fullsized":"2024-05-28T14:09:33.151668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:45.752902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750903+0000","last_change":"2024-05-28T14:09:31.762311+0000","last_active":"2024-05-28T14:09:32.750903+0000","last_peered":"2024-05-28T14:09:32.750903+0000","last_clean":"2024-05-28T14:09:32.750903+0000","last_became_active":"2024-05-28T14:09:31.761990+0000","last_became_peered":"2024-05-28T14:09:31.761990+0000","last_unstale":"2024-05-28T14:09:32.750903+0000","last_undegraded":"2024-05-28T14:09:32.750903+0000","last_fullsized":"2024-05-28T14:09:32.750903+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151408+0000","last_change":"2024-05-28T14:09:30.757972+0000","last_active":"2024-05-28T14:09:33.151408+0000","last_peered":"2024-05-28T14:09:33.151408+0000","last_clean":"2024-05-28T14:09:33.151408+0000","last_became_active":"2024-05-28T14:09:30.757716+0000","last_became_peered":"2024-05-28T14:09:30.757716+0000","last_unstale":"2024-05-28T14:09:33.151408+0000","last_undegraded":"2024-05-28T14:09:33.151408+0000","last_fullsized":"2024-05-28T14:09:33.151408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:40:38.394618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764978+0000","last_change":"2024-05-28T14:09:31.754712+0000","last_active":"2024-05-28T14:09:32.764978+0000","last_peered":"2024-05-28T14:09:32.764978+0000","last_clean":"2024-05-28T14:09:32.764978+0000","last_became_active":"2024-05-28T14:09:31.754427+0000","last_became_peered":"2024-05-28T14:09:31.754427+0000","last_unstale":"2024-05-28T14:09:32.764978+0000","last_undegraded":"2024-05-28T14:09:32.764978+0000","last_fullsized":"2024-05-28T14:09:32.764978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151098+0000","last_change":"2024-05-28T14:09:30.756257+0000","last_active":"2024-05-28T14:09:33.151098+0000","last_peered":"2024-05-28T14:09:33.151098+0000","last_clean":"2024-05-28T14:09:33.151098+0000","last_became_active":"2024-05-28T14:09:30.755923+0000","last_became_peered":"2024-05-28T14:09:30.755923+0000","last_unstale":"2024-05-28T14:09:33.151098+0000","last_undegraded":"2024-05-28T14:09:33.151098+0000","last_fullsized":"2024-05-28T14:09:33.151098+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:54.698345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752090+0000","last_change":"2024-05-28T14:09:31.748217+0000","last_active":"2024-05-28T14:09:32.752090+0000","last_peered":"2024-05-28T14:09:32.752090+0000","last_clean":"2024-05-28T14:09:32.752090+0000","last_became_active":"2024-05-28T14:09:31.747336+0000","last_became_peered":"2024-05-28T14:09:31.747336+0000","last_unstale":"2024-05-28T14:09:32.752090+0000","last_undegraded":"2024-05-28T14:09:32.752090+0000","last_fullsized":"2024-05-28T14:09:32.752090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"89'6","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750773+0000","last_change":"2024-05-28T14:09:30.760654+0000","last_active":"2024-05-28T14:09:32.750773+0000","last_peered":"2024-05-28T14:09:32.750773+0000","last_clean":"2024-05-28T14:09:32.750773+0000","last_became_active":"2024-05-28T14:09:30.760328+0000","last_became_peered":"2024-05-28T14:09:30.760328+0000","last_unstale":"2024-05-28T14:09:32.750773+0000","last_undegraded":"2024-05-28T14:09:32.750773+0000","last_fullsized":"2024-05-28T14:09:32.750773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:54:12.659735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764801+0000","last_change":"2024-05-28T14:09:31.742984+0000","last_active":"2024-05-28T14:09:32.764801+0000","last_peered":"2024-05-28T14:09:32.764801+0000","last_clean":"2024-05-28T14:09:32.764801+0000","last_became_active":"2024-05-28T14:09:31.742692+0000","last_became_peered":"2024-05-28T14:09:31.742692+0000","last_unstale":"2024-05-28T14:09:32.764801+0000","last_undegraded":"2024-05-28T14:09:32.764801+0000","last_fullsized":"2024-05-28T14:09:32.764801+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762506+0000","last_change":"2024-05-28T14:09:30.751922+0000","last_active":"2024-05-28T14:09:32.762506+0000","last_peered":"2024-05-28T14:09:32.762506+0000","last_clean":"2024-05-28T14:09:32.762506+0000","last_became_active":"2024-05-28T14:09:30.751646+0000","last_became_peered":"2024-05-28T14:09:30.751646+0000","last_unstale":"2024-05-28T14:09:32.762506+0000","last_undegraded":"2024-05-28T14:09:32.762506+0000","last_fullsized":"2024-05-28T14:09:32.762506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:24:14.215364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150460+0000","last_change":"2024-05-28T14:09:31.763288+0000","last_active":"2024-05-28T14:09:33.150460+0000","last_peered":"2024-05-28T14:09:33.150460+0000","last_clean":"2024-05-28T14:09:33.150460+0000","last_became_active":"2024-05-28T14:09:31.762800+0000","last_became_peered":"2024-05-28T14:09:31.762800+0000","last_unstale":"2024-05-28T14:09:33.150460+0000","last_undegraded":"2024-05-28T14:09:33.150460+0000","last_fullsized":"2024-05-28T14:09:33.150460+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:46:19.764662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747904+0000","last_change":"2024-05-28T14:09:30.753559+0000","last_active":"2024-05-28T14:09:32.747904+0000","last_peered":"2024-05-28T14:09:32.747904+0000","last_clean":"2024-05-28T14:09:32.747904+0000","last_became_active":"2024-05-28T14:09:30.753253+0000","last_became_peered":"2024-05-28T14:09:30.753253+0000","last_unstale":"2024-05-28T14:09:32.747904+0000","last_undegraded":"2024-05-28T14:09:32.747904+0000","last_fullsized":"2024-05-28T14:09:32.747904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:15:03.984981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762339+0000","last_change":"2024-05-28T14:09:31.765598+0000","last_active":"2024-05-28T14:09:32.762339+0000","last_peered":"2024-05-28T14:09:32.762339+0000","last_clean":"2024-05-28T14:09:32.762339+0000","last_became_active":"2024-05-28T14:09:31.760967+0000","last_became_peered":"2024-05-28T14:09:31.760967+0000","last_unstale":"2024-05-28T14:09:32.762339+0000","last_undegraded":"2024-05-28T14:09:32.762339+0000","last_fullsized":"2024-05-28T14:09:32.762339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:24:40.914669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750623+0000","last_change":"2024-05-28T14:09:30.758094+0000","last_active":"2024-05-28T14:09:32.750623+0000","last_peered":"2024-05-28T14:09:32.750623+0000","last_clean":"2024-05-28T14:09:32.750623+0000","last_became_active":"2024-05-28T14:09:30.757801+0000","last_became_peered":"2024-05-28T14:09:30.757801+0000","last_unstale":"2024-05-28T14:09:32.750623+0000","last_undegraded":"2024-05-28T14:09:32.750623+0000","last_fullsized":"2024-05-28T14:09:32.750623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:40:25.005440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762291+0000","last_change":"2024-05-28T14:09:31.754734+0000","last_active":"2024-05-28T14:09:32.762291+0000","last_peered":"2024-05-28T14:09:32.762291+0000","last_clean":"2024-05-28T14:09:32.762291+0000","last_became_active":"2024-05-28T14:09:31.754459+0000","last_became_peered":"2024-05-28T14:09:31.754459+0000","last_unstale":"2024-05-28T14:09:32.762291+0000","last_undegraded":"2024-05-28T14:09:32.762291+0000","last_fullsized":"2024-05-28T14:09:32.762291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:19:30.516845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"89'8","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764575+0000","last_change":"2024-05-28T14:09:30.759773+0000","last_active":"2024-05-28T14:09:32.764575+0000","last_peered":"2024-05-28T14:09:32.764575+0000","last_clean":"2024-05-28T14:09:32.764575+0000","last_became_active":"2024-05-28T14:09:30.759218+0000","last_became_peered":"2024-05-28T14:09:30.759218+0000","last_unstale":"2024-05-28T14:09:32.764575+0000","last_undegraded":"2024-05-28T14:09:32.764575+0000","last_fullsized":"2024-05-28T14:09:32.764575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:14:14.485471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151031+0000","last_change":"2024-05-28T14:09:31.766044+0000","last_active":"2024-05-28T14:09:33.151031+0000","last_peered":"2024-05-28T14:09:33.151031+0000","last_clean":"2024-05-28T14:09:33.151031+0000","last_became_active":"2024-05-28T14:09:31.763364+0000","last_became_peered":"2024-05-28T14:09:31.763364+0000","last_unstale":"2024-05-28T14:09:33.151031+0000","last_undegraded":"2024-05-28T14:09:33.151031+0000","last_fullsized":"2024-05-28T14:09:33.151031+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:54:38.549065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151738+0000","last_change":"2024-05-28T14:09:30.763452+0000","last_active":"2024-05-28T14:09:33.151738+0000","last_peered":"2024-05-28T14:09:33.151738+0000","last_clean":"2024-05-28T14:09:33.151738+0000","last_became_active":"2024-05-28T14:09:30.763075+0000","last_became_peered":"2024-05-28T14:09:30.763075+0000","last_unstale":"2024-05-28T14:09:33.151738+0000","last_undegraded":"2024-05-28T14:09:33.151738+0000","last_fullsized":"2024-05-28T14:09:33.151738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:37.606903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151668+0000","last_change":"2024-05-28T14:09:31.769821+0000","last_active":"2024-05-28T14:09:33.151668+0000","last_peered":"2024-05-28T14:09:33.151668+0000","last_clean":"2024-05-28T14:09:33.151668+0000","last_became_active":"2024-05-28T14:09:31.769534+0000","last_became_peered":"2024-05-28T14:09:31.769534+0000","last_unstale":"2024-05-28T14:09:33.151668+0000","last_undegraded":"2024-05-28T14:09:33.151668+0000","last_fullsized":"2024-05-28T14:09:33.151668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765169+0000","last_change":"2024-05-28T14:09:30.752703+0000","last_active":"2024-05-28T14:09:32.765169+0000","last_peered":"2024-05-28T14:09:32.765169+0000","last_clean":"2024-05-28T14:09:32.765169+0000","last_became_active":"2024-05-28T14:09:30.752399+0000","last_became_peered":"2024-05-28T14:09:30.752399+0000","last_unstale":"2024-05-28T14:09:32.765169+0000","last_undegraded":"2024-05-28T14:09:32.765169+0000","last_fullsized":"2024-05-28T14:09:32.765169+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:42.742326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752236+0000","last_change":"2024-05-28T14:09:31.757626+0000","last_active":"2024-05-28T14:09:32.752236+0000","last_peered":"2024-05-28T14:09:32.752236+0000","last_clean":"2024-05-28T14:09:32.752236+0000","last_became_active":"2024-05-28T14:09:31.756947+0000","last_became_peered":"2024-05-28T14:09:31.756947+0000","last_unstale":"2024-05-28T14:09:32.752236+0000","last_undegraded":"2024-05-28T14:09:32.752236+0000","last_fullsized":"2024-05-28T14:09:32.752236+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765042+0000","last_change":"2024-05-28T14:09:30.760474+0000","last_active":"2024-05-28T14:09:32.765042+0000","last_peered":"2024-05-28T14:09:32.765042+0000","last_clean":"2024-05-28T14:09:32.765042+0000","last_became_active":"2024-05-28T14:09:30.759783+0000","last_became_peered":"2024-05-28T14:09:30.759783+0000","last_unstale":"2024-05-28T14:09:32.765042+0000","last_undegraded":"2024-05-28T14:09:32.765042+0000","last_fullsized":"2024-05-28T14:09:32.765042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:49:41.271344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149495+0000","last_change":"2024-05-28T14:09:31.761651+0000","last_active":"2024-05-28T14:09:33.149495+0000","last_peered":"2024-05-28T14:09:33.149495+0000","last_clean":"2024-05-28T14:09:33.149495+0000","last_became_active":"2024-05-28T14:09:31.761350+0000","last_became_peered":"2024-05-28T14:09:31.761350+0000","last_unstale":"2024-05-28T14:09:33.149495+0000","last_undegraded":"2024-05-28T14:09:33.149495+0000","last_fullsized":"2024-05-28T14:09:33.149495+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:13:58.981771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.748033+0000","last_change":"2024-05-28T14:09:30.754286+0000","last_active":"2024-05-28T14:09:32.748033+0000","last_peered":"2024-05-28T14:09:32.748033+0000","last_clean":"2024-05-28T14:09:32.748033+0000","last_became_active":"2024-05-28T14:09:30.753997+0000","last_became_peered":"2024-05-28T14:09:30.753997+0000","last_unstale":"2024-05-28T14:09:32.748033+0000","last_undegraded":"2024-05-28T14:09:32.748033+0000","last_fullsized":"2024-05-28T14:09:32.748033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:34:27.167006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751282+0000","last_change":"2024-05-28T14:09:31.760545+0000","last_active":"2024-05-28T14:09:32.751282+0000","last_peered":"2024-05-28T14:09:32.751282+0000","last_clean":"2024-05-28T14:09:32.751282+0000","last_became_active":"2024-05-28T14:09:31.760024+0000","last_became_peered":"2024-05-28T14:09:31.760024+0000","last_unstale":"2024-05-28T14:09:32.751282+0000","last_undegraded":"2024-05-28T14:09:32.751282+0000","last_fullsized":"2024-05-28T14:09:32.751282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149448+0000","last_change":"2024-05-28T14:09:30.749340+0000","last_active":"2024-05-28T14:09:33.149448+0000","last_peered":"2024-05-28T14:09:33.149448+0000","last_clean":"2024-05-28T14:09:33.149448+0000","last_became_active":"2024-05-28T14:09:30.749082+0000","last_became_peered":"2024-05-28T14:09:30.749082+0000","last_unstale":"2024-05-28T14:09:33.149448+0000","last_undegraded":"2024-05-28T14:09:33.149448+0000","last_fullsized":"2024-05-28T14:09:33.149448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:53:51.026943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149352+0000","last_change":"2024-05-28T14:09:31.755784+0000","last_active":"2024-05-28T14:09:33.149352+0000","last_peered":"2024-05-28T14:09:33.149352+0000","last_clean":"2024-05-28T14:09:33.149352+0000","last_became_active":"2024-05-28T14:09:31.755418+0000","last_became_peered":"2024-05-28T14:09:31.755418+0000","last_unstale":"2024-05-28T14:09:33.149352+0000","last_undegraded":"2024-05-28T14:09:33.149352+0000","last_fullsized":"2024-05-28T14:09:33.149352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:29:32.550652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149017+0000","last_change":"2024-05-28T14:09:30.757283+0000","last_active":"2024-05-28T14:09:33.149017+0000","last_peered":"2024-05-28T14:09:33.149017+0000","last_clean":"2024-05-28T14:09:33.149017+0000","last_became_active":"2024-05-28T14:09:30.756776+0000","last_became_peered":"2024-05-28T14:09:30.756776+0000","last_unstale":"2024-05-28T14:09:33.149017+0000","last_undegraded":"2024-05-28T14:09:33.149017+0000","last_fullsized":"2024-05-28T14:09:33.149017+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:25:28.204845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751081+0000","last_change":"2024-05-28T14:09:31.748847+0000","last_active":"2024-05-28T14:09:32.751081+0000","last_peered":"2024-05-28T14:09:32.751081+0000","last_clean":"2024-05-28T14:09:32.751081+0000","last_became_active":"2024-05-28T14:09:31.748479+0000","last_became_peered":"2024-05-28T14:09:31.748479+0000","last_unstale":"2024-05-28T14:09:32.751081+0000","last_undegraded":"2024-05-28T14:09:32.751081+0000","last_fullsized":"2024-05-28T14:09:32.751081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747718+0000","last_change":"2024-05-28T14:09:30.755244+0000","last_active":"2024-05-28T14:09:32.747718+0000","last_peered":"2024-05-28T14:09:32.747718+0000","last_clean":"2024-05-28T14:09:32.747718+0000","last_became_active":"2024-05-28T14:09:30.754984+0000","last_became_peered":"2024-05-28T14:09:30.754984+0000","last_unstale":"2024-05-28T14:09:32.747718+0000","last_undegraded":"2024-05-28T14:09:32.747718+0000","last_fullsized":"2024-05-28T14:09:32.747718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:14:58.988961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752367+0000","last_change":"2024-05-28T14:09:31.745138+0000","last_active":"2024-05-28T14:09:32.752367+0000","last_peered":"2024-05-28T14:09:32.752367+0000","last_clean":"2024-05-28T14:09:32.752367+0000","last_became_active":"2024-05-28T14:09:31.744819+0000","last_became_peered":"2024-05-28T14:09:31.744819+0000","last_unstale":"2024-05-28T14:09:32.752367+0000","last_undegraded":"2024-05-28T14:09:32.752367+0000","last_fullsized":"2024-05-28T14:09:32.752367+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762790+0000","last_change":"2024-05-28T14:09:30.760787+0000","last_active":"2024-05-28T14:09:32.762790+0000","last_peered":"2024-05-28T14:09:32.762790+0000","last_clean":"2024-05-28T14:09:32.762790+0000","last_became_active":"2024-05-28T14:09:30.760502+0000","last_became_peered":"2024-05-28T14:09:30.760502+0000","last_unstale":"2024-05-28T14:09:32.762790+0000","last_undegraded":"2024-05-28T14:09:32.762790+0000","last_fullsized":"2024-05-28T14:09:32.762790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:31:35.886734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150913+0000","last_change":"2024-05-28T14:09:31.771112+0000","last_active":"2024-05-28T14:09:33.150913+0000","last_peered":"2024-05-28T14:09:33.150913+0000","last_clean":"2024-05-28T14:09:33.150913+0000","last_became_active":"2024-05-28T14:09:31.770660+0000","last_became_peered":"2024-05-28T14:09:31.770660+0000","last_unstale":"2024-05-28T14:09:33.150913+0000","last_undegraded":"2024-05-28T14:09:33.150913+0000","last_fullsized":"2024-05-28T14:09:33.150913+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:02:06.074104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762843+0000","last_change":"2024-05-28T14:09:30.751801+0000","last_active":"2024-05-28T14:09:32.762843+0000","last_peered":"2024-05-28T14:09:32.762843+0000","last_clean":"2024-05-28T14:09:32.762843+0000","last_became_active":"2024-05-28T14:09:30.751504+0000","last_became_peered":"2024-05-28T14:09:30.751504+0000","last_unstale":"2024-05-28T14:09:32.762843+0000","last_undegraded":"2024-05-28T14:09:32.762843+0000","last_fullsized":"2024-05-28T14:09:32.762843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:42.042818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151673+0000","last_change":"2024-05-28T14:09:31.765811+0000","last_active":"2024-05-28T14:09:33.151673+0000","last_peered":"2024-05-28T14:09:33.151673+0000","last_clean":"2024-05-28T14:09:33.151673+0000","last_became_active":"2024-05-28T14:09:31.764710+0000","last_became_peered":"2024-05-28T14:09:31.764710+0000","last_unstale":"2024-05-28T14:09:33.151673+0000","last_undegraded":"2024-05-28T14:09:33.151673+0000","last_fullsized":"2024-05-28T14:09:33.151673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:57.967039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151377+0000","last_change":"2024-05-28T14:09:30.760473+0000","last_active":"2024-05-28T14:09:33.151377+0000","last_peered":"2024-05-28T14:09:33.151377+0000","last_clean":"2024-05-28T14:09:33.151377+0000","last_became_active":"2024-05-28T14:09:30.760129+0000","last_became_peered":"2024-05-28T14:09:30.760129+0000","last_unstale":"2024-05-28T14:09:33.151377+0000","last_undegraded":"2024-05-28T14:09:33.151377+0000","last_fullsized":"2024-05-28T14:09:33.151377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:18.317793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752265+0000","last_change":"2024-05-28T14:09:31.748833+0000","last_active":"2024-05-28T14:09:32.752265+0000","last_peered":"2024-05-28T14:09:32.752265+0000","last_clean":"2024-05-28T14:09:32.752265+0000","last_became_active":"2024-05-28T14:09:31.747731+0000","last_became_peered":"2024-05-28T14:09:31.747731+0000","last_unstale":"2024-05-28T14:09:32.752265+0000","last_undegraded":"2024-05-28T14:09:32.752265+0000","last_fullsized":"2024-05-28T14:09:32.752265+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151485+0000","last_change":"2024-05-28T14:09:30.758617+0000","last_active":"2024-05-28T14:09:33.151485+0000","last_peered":"2024-05-28T14:09:33.151485+0000","last_clean":"2024-05-28T14:09:33.151485+0000","last_became_active":"2024-05-28T14:09:30.758317+0000","last_became_peered":"2024-05-28T14:09:30.758317+0000","last_unstale":"2024-05-28T14:09:33.151485+0000","last_undegraded":"2024-05-28T14:09:33.151485+0000","last_fullsized":"2024-05-28T14:09:33.151485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:10.090547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751157+0000","last_change":"2024-05-28T14:09:31.760268+0000","last_active":"2024-05-28T14:09:32.751157+0000","last_peered":"2024-05-28T14:09:32.751157+0000","last_clean":"2024-05-28T14:09:32.751157+0000","last_became_active":"2024-05-28T14:09:31.759889+0000","last_became_peered":"2024-05-28T14:09:31.759889+0000","last_unstale":"2024-05-28T14:09:32.751157+0000","last_undegraded":"2024-05-28T14:09:32.751157+0000","last_fullsized":"2024-05-28T14:09:32.751157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764892+0000","last_change":"2024-05-28T14:09:30.754916+0000","last_active":"2024-05-28T14:09:32.764892+0000","last_peered":"2024-05-28T14:09:32.764892+0000","last_clean":"2024-05-28T14:09:32.764892+0000","last_became_active":"2024-05-28T14:09:30.754019+0000","last_became_peered":"2024-05-28T14:09:30.754019+0000","last_unstale":"2024-05-28T14:09:32.764892+0000","last_undegraded":"2024-05-28T14:09:32.764892+0000","last_fullsized":"2024-05-28T14:09:32.764892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:34.165033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764999+0000","last_change":"2024-05-28T14:09:31.748964+0000","last_active":"2024-05-28T14:09:32.764999+0000","last_peered":"2024-05-28T14:09:32.764999+0000","last_clean":"2024-05-28T14:09:32.764999+0000","last_became_active":"2024-05-28T14:09:31.748393+0000","last_became_peered":"2024-05-28T14:09:31.748393+0000","last_unstale":"2024-05-28T14:09:32.764999+0000","last_undegraded":"2024-05-28T14:09:32.764999+0000","last_fullsized":"2024-05-28T14:09:32.764999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751724+0000","last_change":"2024-05-28T14:09:30.748735+0000","last_active":"2024-05-28T14:09:32.751724+0000","last_peered":"2024-05-28T14:09:32.751724+0000","last_clean":"2024-05-28T14:09:32.751724+0000","last_became_active":"2024-05-28T14:09:30.748404+0000","last_became_peered":"2024-05-28T14:09:30.748404+0000","last_unstale":"2024-05-28T14:09:32.751724+0000","last_undegraded":"2024-05-28T14:09:32.751724+0000","last_fullsized":"2024-05-28T14:09:32.751724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:37.606903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150429+0000","last_change":"2024-05-28T14:09:31.764716+0000","last_active":"2024-05-28T14:09:33.150429+0000","last_peered":"2024-05-28T14:09:33.150429+0000","last_clean":"2024-05-28T14:09:33.150429+0000","last_became_active":"2024-05-28T14:09:31.764407+0000","last_became_peered":"2024-05-28T14:09:31.764407+0000","last_unstale":"2024-05-28T14:09:33.150429+0000","last_undegraded":"2024-05-28T14:09:33.150429+0000","last_fullsized":"2024-05-28T14:09:33.150429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:14:59.996233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151098+0000","last_change":"2024-05-28T14:09:30.755558+0000","last_active":"2024-05-28T14:09:33.151098+0000","last_peered":"2024-05-28T14:09:33.151098+0000","last_clean":"2024-05-28T14:09:33.151098+0000","last_became_active":"2024-05-28T14:09:30.755257+0000","last_became_peered":"2024-05-28T14:09:30.755257+0000","last_unstale":"2024-05-28T14:09:33.151098+0000","last_undegraded":"2024-05-28T14:09:33.151098+0000","last_fullsized":"2024-05-28T14:09:33.151098+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:58.290350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150859+0000","last_change":"2024-05-28T14:09:31.758209+0000","last_active":"2024-05-28T14:09:33.150859+0000","last_peered":"2024-05-28T14:09:33.150859+0000","last_clean":"2024-05-28T14:09:33.150859+0000","last_became_active":"2024-05-28T14:09:31.757910+0000","last_became_peered":"2024-05-28T14:09:31.757910+0000","last_unstale":"2024-05-28T14:09:33.150859+0000","last_undegraded":"2024-05-28T14:09:33.150859+0000","last_fullsized":"2024-05-28T14:09:33.150859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:36:07.311954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751556+0000","last_change":"2024-05-28T14:09:30.754615+0000","last_active":"2024-05-28T14:09:32.751556+0000","last_peered":"2024-05-28T14:09:32.751556+0000","last_clean":"2024-05-28T14:09:32.751556+0000","last_became_active":"2024-05-28T14:09:30.754325+0000","last_became_peered":"2024-05-28T14:09:30.754325+0000","last_unstale":"2024-05-28T14:09:32.751556+0000","last_undegraded":"2024-05-28T14:09:32.751556+0000","last_fullsized":"2024-05-28T14:09:32.751556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:45.752902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762719+0000","last_change":"2024-05-28T14:09:31.760657+0000","last_active":"2024-05-28T14:09:32.762719+0000","last_peered":"2024-05-28T14:09:32.762719+0000","last_clean":"2024-05-28T14:09:32.762719+0000","last_became_active":"2024-05-28T14:09:31.759615+0000","last_became_peered":"2024-05-28T14:09:31.759615+0000","last_unstale":"2024-05-28T14:09:32.762719+0000","last_undegraded":"2024-05-28T14:09:32.762719+0000","last_fullsized":"2024-05-28T14:09:32.762719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:55.705617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151176+0000","last_change":"2024-05-28T14:09:30.762060+0000","last_active":"2024-05-28T14:09:33.151176+0000","last_peered":"2024-05-28T14:09:33.151176+0000","last_clean":"2024-05-28T14:09:33.151176+0000","last_became_active":"2024-05-28T14:09:30.761778+0000","last_became_peered":"2024-05-28T14:09:30.761778+0000","last_unstale":"2024-05-28T14:09:33.151176+0000","last_undegraded":"2024-05-28T14:09:33.151176+0000","last_fullsized":"2024-05-28T14:09:33.151176+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:56.959767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":300,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150107+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:09:33.150107+0000","last_peered":"2024-05-28T14:09:33.150107+0000","last_clean":"2024-05-28T14:09:33.150107+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:09:33.150107+0000","last_undegraded":"2024-05-28T14:09:33.150107+0000","last_fullsized":"2024-05-28T14:09:33.150107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123028,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248812,"statfs":{"total":96636764160,"available":96510783488,"internally_reserved":0,"allocated":1703936,"data_stored":1068711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122448,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249392,"statfs":{"total":96636764160,"available":96511377408,"internally_reserved":0,"allocated":1118208,"data_stored":478967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":124254662},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76628,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295212,"statfs":{"total":96636764160,"available":96558297088,"internally_reserved":0,"allocated":1122304,"data_stored":479907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82184,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289656,"statfs":{"total":96636764160,"available":96552607744,"internally_reserved":0,"allocated":1105920,"data_stored":478273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574987,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130772,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241068,"statfs":{"total":96636764160,"available":96502853632,"internally_reserved":0,"allocated":1118208,"data_stored":479885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":132774344},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144652,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227188,"statfs":{"total":96636764160,"available":96488640512,"internally_reserved":0,"allocated":1110016,"data_stored":478855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108564,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94263276,"statfs":{"total":96636764160,"available":96525594624,"internally_reserved":0,"allocated":1118208,"data_stored":478467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64724,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307116,"statfs":{"total":96636764160,"available":96570486784,"internally_reserved":0,"allocated":1703936,"data_stored":1068675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:09:40.440 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:09:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.609+0000 7fdb18200640 1 Processor -- start 2024-05-28T14:09:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.609+0000 7fdb18200640 1 -- start start 2024-05-28T14:09:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.613+0000 7fdb18200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdb1015ba40 0x7fdb100d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.613+0000 7fdb18200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb10072dd0 0x7fdb100d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.613+0000 7fdb18200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb1015f280 con 0x7fdb1006f660 2024-05-28T14:09:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.613+0000 7fdb18200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdb1015f030 con 0x7fdb1015ba40 2024-05-28T14:09:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.613+0000 7fdb18200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1015f560 con 0x7fdb10072dd0 2024-05-28T14:09:40.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.613+0000 7fdb16200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb10072dd0 0x7fdb100d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.613+0000 7fdb16200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb10072dd0 0x7fdb100d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:60650/0 (socket says 172.21.15.42:60650) 2024-05-28T14:09:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.613+0000 7fdb16200640 1 -- 172.21.15.42:0/3398159970 learned_addr learned my addr 172.21.15.42:0/3398159970 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.613+0000 7fdb14e00640 1 --2- 172.21.15.42:0/3398159970 >> v2:172.21.15.63:3300/0 conn(0x7fdb1015ba40 0x7fdb100d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.613+0000 7fdb14e00640 1 -- 172.21.15.42:0/3398159970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb10072dd0 msgr2=0x7fdb100d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.613+0000 7fdb14e00640 1 --2- 172.21.15.42:0/3398159970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb10072dd0 0x7fdb100d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.613+0000 7fdb14e00640 1 -- 172.21.15.42:0/3398159970 >> v1:172.21.15.63:6789/0 conn(0x7fdb1006f660 legacy=0x7fdb10073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.613+0000 7fdb14e00640 1 -- 172.21.15.42:0/3398159970 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1015f840 con 0x7fdb1015ba40 2024-05-28T14:09:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.613+0000 7fdb16200640 1 --2- 172.21.15.42:0/3398159970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb10072dd0 0x7fdb100d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:09:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb14e00640 1 --2- 172.21.15.42:0/3398159970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb1015ba40 0x7fdb100d06f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0097c0 tx=0x7fdafc01c030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e76fb5651020ddba server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:40.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb0fe00640 1 -- 172.21.15.42:0/3398159970 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0b1070 con 0x7fdb1015ba40 2024-05-28T14:09:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb0fe00640 1 -- 172.21.15.42:0/3398159970 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdafc0a4870 con 0x7fdb1015ba40 2024-05-28T14:09:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb0fe00640 1 -- 172.21.15.42:0/3398159970 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0a4c40 con 0x7fdb1015ba40 2024-05-28T14:09:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 -- 172.21.15.42:0/3398159970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb1015ba40 msgr2=0x7fdb100d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 --2- 172.21.15.42:0/3398159970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb1015ba40 0x7fdb100d06f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0097c0 tx=0x7fdafc01c030 comp rx=0 tx=0).stop 2024-05-28T14:09:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 -- 172.21.15.42:0/3398159970 shutdown_connections 2024-05-28T14:09:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 --2- 172.21.15.42:0/3398159970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb10072dd0 0x7fdb100d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 --2- 172.21.15.42:0/3398159970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb1015ba40 0x7fdb100d06f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 -- 172.21.15.42:0/3398159970 >> 172.21.15.42:0/3398159970 conn(0x7fdb1006bdf0 msgr2=0x7fdb1006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 -- 172.21.15.42:0/3398159970 shutdown_connections 2024-05-28T14:09:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 -- 172.21.15.42:0/3398159970 wait complete. 2024-05-28T14:09:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 Processor -- start 2024-05-28T14:09:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 -- start start 2024-05-28T14:09:40.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb1006f660 0x7fdb10099900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:40.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb10072dd0 0x7fdb10154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:40.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb1009a2d0 0x7fdb10155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:40.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdb10168f70 con 0x7fdb10072dd0 2024-05-28T14:09:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdb10168df0 con 0x7fdb1006f660 2024-05-28T14:09:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.617+0000 7fdb18200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdb101690f0 con 0x7fdb1009a2d0 2024-05-28T14:09:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb14e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb10072dd0 0x7fdb10154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb14e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb10072dd0 0x7fdb10154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38884/0 (socket says 172.21.15.42:38884) 2024-05-28T14:09:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb14e00640 1 -- 172.21.15.42:0/1579041460 learned_addr learned my addr 172.21.15.42:0/1579041460 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb16200640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb1009a2d0 0x7fdb10155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb15800640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb1006f660 0x7fdb10099900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb14e00640 1 -- 172.21.15.42:0/1579041460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb1009a2d0 msgr2=0x7fdb10155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb14e00640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb1009a2d0 0x7fdb10155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb14e00640 1 -- 172.21.15.42:0/1579041460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb1006f660 msgr2=0x7fdb10099900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb14e00640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb1006f660 0x7fdb10099900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb14e00640 1 -- 172.21.15.42:0/1579041460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb100cdb30 con 0x7fdb10072dd0 2024-05-28T14:09:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb16200640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb1009a2d0 0x7fdb10155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:40.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb15800640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb1006f660 0x7fdb10099900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:40.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb14e00640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb10072dd0 0x7fdb10154e00 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fdafc019e70 tx=0x7fdafc0190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:40.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb0d600640 1 -- 172.21.15.42:0/1579041460 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0b1070 con 0x7fdb10072dd0 2024-05-28T14:09:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb0d600640 1 -- 172.21.15.42:0/1579041460 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdafc0bb070 con 0x7fdb10072dd0 2024-05-28T14:09:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb18200640 1 -- 172.21.15.42:0/1579041460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb100cde10 con 0x7fdb10072dd0 2024-05-28T14:09:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb0d600640 1 -- 172.21.15.42:0/1579041460 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0acda0 con 0x7fdb10072dd0 2024-05-28T14:09:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb18200640 1 -- 172.21.15.42:0/1579041460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb100ce3d0 con 0x7fdb10072dd0 2024-05-28T14:09:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb18200640 1 -- 172.21.15.42:0/1579041460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb10068a90 con 0x7fdb10072dd0 2024-05-28T14:09:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.621+0000 7fdb0d600640 1 -- 172.21.15.42:0/1579041460 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fdafc0a4730 con 0x7fdb10072dd0 2024-05-28T14:09:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.625+0000 7fdb0d600640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdaf00694f0 0x7fdaf006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.625+0000 7fdb0d600640 1 -- 172.21.15.42:0/1579041460 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7fdafc11efd0 con 0x7fdb10072dd0 2024-05-28T14:09:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.625+0000 7fdb15800640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdaf00694f0 0x7fdaf006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.625+0000 7fdb15800640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdaf00694f0 0x7fdaf006b9b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdb1009a960 tx=0x7fdb04009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.629+0000 7fdb0d600640 1 -- 172.21.15.42:0/1579041460 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdafc016090 con 0x7fdb10072dd0 2024-05-28T14:09:40.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.789+0000 7fdb18200640 1 -- 172.21.15.42:0/1579041460 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb10153390 con 0x7fdaf00694f0 2024-05-28T14:09:40.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.797+0000 7fdb0d600640 1 -- 172.21.15.42:0/1579041460 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252102 (secure 0 0 0) 0x7fdb10153390 con 0x7fdaf00694f0 2024-05-28T14:09:40.782 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:40.784 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:09:40.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.801+0000 7fdb18200640 1 -- 172.21.15.42:0/1579041460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdaf00694f0 msgr2=0x7fdaf006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:40.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.801+0000 7fdb18200640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdaf00694f0 0x7fdaf006b9b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdb1009a960 tx=0x7fdb04009040 comp rx=0 tx=0).stop 2024-05-28T14:09:40.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.801+0000 7fdb18200640 1 -- 172.21.15.42:0/1579041460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb10072dd0 msgr2=0x7fdb10154e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:40.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.801+0000 7fdb18200640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb10072dd0 0x7fdb10154e00 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fdafc019e70 tx=0x7fdafc0190f0 comp rx=0 tx=0).stop 2024-05-28T14:09:40.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.801+0000 7fdb18200640 1 -- 172.21.15.42:0/1579041460 shutdown_connections 2024-05-28T14:09:40.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.801+0000 7fdb18200640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdaf00694f0 0x7fdaf006b9b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:40.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.801+0000 7fdb18200640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb1009a2d0 0x7fdb10155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:40.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.801+0000 7fdb18200640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb10072dd0 0x7fdb10154e00 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:40.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.801+0000 7fdb18200640 1 --2- 172.21.15.42:0/1579041460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb1006f660 0x7fdb10099900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:40.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.801+0000 7fdb18200640 1 -- 172.21.15.42:0/1579041460 >> 172.21.15.42:0/1579041460 conn(0x7fdb1006bdf0 msgr2=0x7fdb1006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:40.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.801+0000 7fdb18200640 1 -- 172.21.15.42:0/1579041460 shutdown_connections 2024-05-28T14:09:40.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:40.801+0000 7fdb18200640 1 -- 172.21.15.42:0/1579041460 wait complete. 2024-05-28T14:09:40.804 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-05-28T14:09:39.113628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":853000,"kb_used_data":9864,"kb_used_omap":12,"kb_used_meta":842995,"kb_avail":754121720,"statfs":{"total":773094113280,"available":772220641280,"internally_reserved":0,"allocated":10100736,"data_stored":5011740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":863227442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001762"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151546+0000","last_change":"2024-05-28T14:09:31.763443+0000","last_active":"2024-05-28T14:09:33.151546+0000","last_peered":"2024-05-28T14:09:33.151546+0000","last_clean":"2024-05-28T14:09:33.151546+0000","last_became_active":"2024-05-28T14:09:31.763080+0000","last_became_peered":"2024-05-28T14:09:31.763080+0000","last_unstale":"2024-05-28T14:09:33.151546+0000","last_undegraded":"2024-05-28T14:09:33.151546+0000","last_fullsized":"2024-05-28T14:09:33.151546+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762829+0000","last_change":"2024-05-28T14:09:31.768406+0000","last_active":"2024-05-28T14:09:32.762829+0000","last_peered":"2024-05-28T14:09:32.762829+0000","last_clean":"2024-05-28T14:09:32.762829+0000","last_became_active":"2024-05-28T14:09:31.768150+0000","last_became_peered":"2024-05-28T14:09:31.768150+0000","last_unstale":"2024-05-28T14:09:32.762829+0000","last_undegraded":"2024-05-28T14:09:32.762829+0000","last_fullsized":"2024-05-28T14:09:32.762829+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:38.614174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150788+0000","last_change":"2024-05-28T14:09:31.770809+0000","last_active":"2024-05-28T14:09:33.150788+0000","last_peered":"2024-05-28T14:09:33.150788+0000","last_clean":"2024-05-28T14:09:33.150788+0000","last_became_active":"2024-05-28T14:09:31.770547+0000","last_became_peered":"2024-05-28T14:09:31.770547+0000","last_unstale":"2024-05-28T14:09:33.150788+0000","last_undegraded":"2024-05-28T14:09:33.150788+0000","last_fullsized":"2024-05-28T14:09:33.150788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:34:22.994257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149345+0000","last_change":"2024-05-28T14:09:31.761634+0000","last_active":"2024-05-28T14:09:33.149345+0000","last_peered":"2024-05-28T14:09:33.149345+0000","last_clean":"2024-05-28T14:09:33.149345+0000","last_became_active":"2024-05-28T14:09:31.761270+0000","last_became_peered":"2024-05-28T14:09:31.761270+0000","last_unstale":"2024-05-28T14:09:33.149345+0000","last_undegraded":"2024-05-28T14:09:33.149345+0000","last_fullsized":"2024-05-28T14:09:33.149345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:43.050090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751824+0000","last_change":"2024-05-28T14:09:31.756701+0000","last_active":"2024-05-28T14:09:32.751824+0000","last_peered":"2024-05-28T14:09:32.751824+0000","last_clean":"2024-05-28T14:09:32.751824+0000","last_became_active":"2024-05-28T14:09:31.756425+0000","last_became_peered":"2024-05-28T14:09:31.756425+0000","last_unstale":"2024-05-28T14:09:32.751824+0000","last_undegraded":"2024-05-28T14:09:32.751824+0000","last_fullsized":"2024-05-28T14:09:32.751824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:43.749597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750865+0000","last_change":"2024-05-28T14:09:31.756691+0000","last_active":"2024-05-28T14:09:32.750865+0000","last_peered":"2024-05-28T14:09:32.750865+0000","last_clean":"2024-05-28T14:09:32.750865+0000","last_became_active":"2024-05-28T14:09:31.756434+0000","last_became_peered":"2024-05-28T14:09:31.756434+0000","last_unstale":"2024-05-28T14:09:32.750865+0000","last_undegraded":"2024-05-28T14:09:32.750865+0000","last_fullsized":"2024-05-28T14:09:32.750865+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149073+0000","last_change":"2024-05-28T14:09:31.757041+0000","last_active":"2024-05-28T14:09:33.149073+0000","last_peered":"2024-05-28T14:09:33.149073+0000","last_clean":"2024-05-28T14:09:33.149073+0000","last_became_active":"2024-05-28T14:09:31.756717+0000","last_became_peered":"2024-05-28T14:09:31.756717+0000","last_unstale":"2024-05-28T14:09:33.149073+0000","last_undegraded":"2024-05-28T14:09:33.149073+0000","last_fullsized":"2024-05-28T14:09:33.149073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:34:26.316657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751086+0000","last_change":"2024-05-28T14:09:31.761009+0000","last_active":"2024-05-28T14:09:32.751086+0000","last_peered":"2024-05-28T14:09:32.751086+0000","last_clean":"2024-05-28T14:09:32.751086+0000","last_became_active":"2024-05-28T14:09:31.760693+0000","last_became_peered":"2024-05-28T14:09:31.760693+0000","last_unstale":"2024-05-28T14:09:32.751086+0000","last_undegraded":"2024-05-28T14:09:32.751086+0000","last_fullsized":"2024-05-28T14:09:32.751086+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751130+0000","last_change":"2024-05-28T14:09:31.748781+0000","last_active":"2024-05-28T14:09:32.751130+0000","last_peered":"2024-05-28T14:09:32.751130+0000","last_clean":"2024-05-28T14:09:32.751130+0000","last_became_active":"2024-05-28T14:09:31.748446+0000","last_became_peered":"2024-05-28T14:09:31.748446+0000","last_unstale":"2024-05-28T14:09:32.751130+0000","last_undegraded":"2024-05-28T14:09:32.751130+0000","last_fullsized":"2024-05-28T14:09:32.751130+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151416+0000","last_change":"2024-05-28T14:09:31.755098+0000","last_active":"2024-05-28T14:09:33.151416+0000","last_peered":"2024-05-28T14:09:33.151416+0000","last_clean":"2024-05-28T14:09:33.151416+0000","last_became_active":"2024-05-28T14:09:31.754790+0000","last_became_peered":"2024-05-28T14:09:31.754790+0000","last_unstale":"2024-05-28T14:09:33.151416+0000","last_undegraded":"2024-05-28T14:09:33.151416+0000","last_fullsized":"2024-05-28T14:09:33.151416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762851+0000","last_change":"2024-05-28T14:09:31.765027+0000","last_active":"2024-05-28T14:09:32.762851+0000","last_peered":"2024-05-28T14:09:32.762851+0000","last_clean":"2024-05-28T14:09:32.762851+0000","last_became_active":"2024-05-28T14:09:31.764660+0000","last_became_peered":"2024-05-28T14:09:31.764660+0000","last_unstale":"2024-05-28T14:09:32.762851+0000","last_undegraded":"2024-05-28T14:09:32.762851+0000","last_fullsized":"2024-05-28T14:09:32.762851+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:59.297622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150769+0000","last_change":"2024-05-28T14:09:31.756022+0000","last_active":"2024-05-28T14:09:33.150769+0000","last_peered":"2024-05-28T14:09:33.150769+0000","last_clean":"2024-05-28T14:09:33.150769+0000","last_became_active":"2024-05-28T14:09:31.755772+0000","last_became_peered":"2024-05-28T14:09:31.755772+0000","last_unstale":"2024-05-28T14:09:33.150769+0000","last_undegraded":"2024-05-28T14:09:33.150769+0000","last_fullsized":"2024-05-28T14:09:33.150769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:35.172304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150463+0000","last_change":"2024-05-28T14:09:31.764389+0000","last_active":"2024-05-28T14:09:33.150463+0000","last_peered":"2024-05-28T14:09:33.150463+0000","last_clean":"2024-05-28T14:09:33.150463+0000","last_became_active":"2024-05-28T14:09:31.764102+0000","last_became_peered":"2024-05-28T14:09:31.764102+0000","last_unstale":"2024-05-28T14:09:33.150463+0000","last_undegraded":"2024-05-28T14:09:33.150463+0000","last_fullsized":"2024-05-28T14:09:33.150463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:37:41.765233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149157+0000","last_change":"2024-05-28T14:09:31.756927+0000","last_active":"2024-05-28T14:09:33.149157+0000","last_peered":"2024-05-28T14:09:33.149157+0000","last_clean":"2024-05-28T14:09:33.149157+0000","last_became_active":"2024-05-28T14:09:31.756570+0000","last_became_peered":"2024-05-28T14:09:31.756570+0000","last_unstale":"2024-05-28T14:09:33.149157+0000","last_undegraded":"2024-05-28T14:09:33.149157+0000","last_fullsized":"2024-05-28T14:09:33.149157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:31:36.894006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747849+0000","last_change":"2024-05-28T14:09:31.766893+0000","last_active":"2024-05-28T14:09:32.747849+0000","last_peered":"2024-05-28T14:09:32.747849+0000","last_clean":"2024-05-28T14:09:32.747849+0000","last_became_active":"2024-05-28T14:09:31.766457+0000","last_became_peered":"2024-05-28T14:09:31.766457+0000","last_unstale":"2024-05-28T14:09:32.747849+0000","last_undegraded":"2024-05-28T14:09:32.747849+0000","last_fullsized":"2024-05-28T14:09:32.747849+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:34:22.994257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751531+0000","last_change":"2024-05-28T14:09:31.753479+0000","last_active":"2024-05-28T14:09:32.751531+0000","last_peered":"2024-05-28T14:09:32.751531+0000","last_clean":"2024-05-28T14:09:32.751531+0000","last_became_active":"2024-05-28T14:09:31.753094+0000","last_became_peered":"2024-05-28T14:09:31.753094+0000","last_unstale":"2024-05-28T14:09:32.751531+0000","last_undegraded":"2024-05-28T14:09:32.751531+0000","last_fullsized":"2024-05-28T14:09:32.751531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:49:42.278616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752189+0000","last_change":"2024-05-28T14:09:31.759451+0000","last_active":"2024-05-28T14:09:32.752189+0000","last_peered":"2024-05-28T14:09:32.752189+0000","last_clean":"2024-05-28T14:09:32.752189+0000","last_became_active":"2024-05-28T14:09:31.759197+0000","last_became_peered":"2024-05-28T14:09:31.759197+0000","last_unstale":"2024-05-28T14:09:32.752189+0000","last_undegraded":"2024-05-28T14:09:32.752189+0000","last_fullsized":"2024-05-28T14:09:32.752189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149434+0000","last_change":"2024-05-28T14:09:31.758504+0000","last_active":"2024-05-28T14:09:33.149434+0000","last_peered":"2024-05-28T14:09:33.149434+0000","last_clean":"2024-05-28T14:09:33.149434+0000","last_became_active":"2024-05-28T14:09:31.757438+0000","last_became_peered":"2024-05-28T14:09:31.757438+0000","last_unstale":"2024-05-28T14:09:33.149434+0000","last_undegraded":"2024-05-28T14:09:33.149434+0000","last_fullsized":"2024-05-28T14:09:33.149434+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:18:46.990801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765154+0000","last_change":"2024-05-28T14:09:31.753754+0000","last_active":"2024-05-28T14:09:32.765154+0000","last_peered":"2024-05-28T14:09:32.765154+0000","last_clean":"2024-05-28T14:09:32.765154+0000","last_became_active":"2024-05-28T14:09:31.753211+0000","last_became_peered":"2024-05-28T14:09:31.753211+0000","last_unstale":"2024-05-28T14:09:32.765154+0000","last_undegraded":"2024-05-28T14:09:32.765154+0000","last_fullsized":"2024-05-28T14:09:32.765154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149445+0000","last_change":"2024-05-28T14:09:31.754140+0000","last_active":"2024-05-28T14:09:33.149445+0000","last_peered":"2024-05-28T14:09:33.149445+0000","last_clean":"2024-05-28T14:09:33.149445+0000","last_became_active":"2024-05-28T14:09:31.753838+0000","last_became_peered":"2024-05-28T14:09:31.753838+0000","last_unstale":"2024-05-28T14:09:33.149445+0000","last_undegraded":"2024-05-28T14:09:33.149445+0000","last_fullsized":"2024-05-28T14:09:33.149445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:29:28.766392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149223+0000","last_change":"2024-05-28T14:09:31.751182+0000","last_active":"2024-05-28T14:09:33.149223+0000","last_peered":"2024-05-28T14:09:33.149223+0000","last_clean":"2024-05-28T14:09:33.149223+0000","last_became_active":"2024-05-28T14:09:31.750664+0000","last_became_peered":"2024-05-28T14:09:31.750664+0000","last_unstale":"2024-05-28T14:09:33.149223+0000","last_undegraded":"2024-05-28T14:09:33.149223+0000","last_fullsized":"2024-05-28T14:09:33.149223+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:24:15.222635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762453+0000","last_change":"2024-05-28T14:09:31.755223+0000","last_active":"2024-05-28T14:09:32.762453+0000","last_peered":"2024-05-28T14:09:32.762453+0000","last_clean":"2024-05-28T14:09:32.762453+0000","last_became_active":"2024-05-28T14:09:31.754880+0000","last_became_peered":"2024-05-28T14:09:31.754880+0000","last_unstale":"2024-05-28T14:09:32.762453+0000","last_undegraded":"2024-05-28T14:09:32.762453+0000","last_fullsized":"2024-05-28T14:09:32.762453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:26:25.450855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150682+0000","last_change":"2024-05-28T14:09:31.752700+0000","last_active":"2024-05-28T14:09:33.150682+0000","last_peered":"2024-05-28T14:09:33.150682+0000","last_clean":"2024-05-28T14:09:33.150682+0000","last_became_active":"2024-05-28T14:09:31.752367+0000","last_became_peered":"2024-05-28T14:09:31.752367+0000","last_unstale":"2024-05-28T14:09:33.150682+0000","last_undegraded":"2024-05-28T14:09:33.150682+0000","last_fullsized":"2024-05-28T14:09:33.150682+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:43.749597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149066+0000","last_change":"2024-05-28T14:09:31.751477+0000","last_active":"2024-05-28T14:09:33.149066+0000","last_peered":"2024-05-28T14:09:33.149066+0000","last_clean":"2024-05-28T14:09:33.149066+0000","last_became_active":"2024-05-28T14:09:31.751198+0000","last_became_peered":"2024-05-28T14:09:31.751198+0000","last_unstale":"2024-05-28T14:09:33.149066+0000","last_undegraded":"2024-05-28T14:09:33.149066+0000","last_fullsized":"2024-05-28T14:09:33.149066+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:42.364107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762838+0000","last_change":"2024-05-28T14:09:31.756171+0000","last_active":"2024-05-28T14:09:32.762838+0000","last_peered":"2024-05-28T14:09:32.762838+0000","last_clean":"2024-05-28T14:09:32.762838+0000","last_became_active":"2024-05-28T14:09:31.754405+0000","last_became_peered":"2024-05-28T14:09:31.754405+0000","last_unstale":"2024-05-28T14:09:32.762838+0000","last_undegraded":"2024-05-28T14:09:32.762838+0000","last_fullsized":"2024-05-28T14:09:32.762838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:19.325065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150852+0000","last_change":"2024-05-28T14:09:31.760741+0000","last_active":"2024-05-28T14:09:33.150852+0000","last_peered":"2024-05-28T14:09:33.150852+0000","last_clean":"2024-05-28T14:09:33.150852+0000","last_became_active":"2024-05-28T14:09:31.760404+0000","last_became_peered":"2024-05-28T14:09:31.760404+0000","last_unstale":"2024-05-28T14:09:33.150852+0000","last_undegraded":"2024-05-28T14:09:33.150852+0000","last_fullsized":"2024-05-28T14:09:33.150852+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:24.707344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751996+0000","last_change":"2024-05-28T14:09:31.754702+0000","last_active":"2024-05-28T14:09:32.751996+0000","last_peered":"2024-05-28T14:09:32.751996+0000","last_clean":"2024-05-28T14:09:32.751996+0000","last_became_active":"2024-05-28T14:09:31.754215+0000","last_became_peered":"2024-05-28T14:09:31.754215+0000","last_unstale":"2024-05-28T14:09:32.751996+0000","last_undegraded":"2024-05-28T14:09:32.751996+0000","last_fullsized":"2024-05-28T14:09:32.751996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:36:07.311954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151011+0000","last_change":"2024-05-28T14:09:31.765289+0000","last_active":"2024-05-28T14:09:33.151011+0000","last_peered":"2024-05-28T14:09:33.151011+0000","last_clean":"2024-05-28T14:09:33.151011+0000","last_became_active":"2024-05-28T14:09:31.763980+0000","last_became_peered":"2024-05-28T14:09:31.763980+0000","last_unstale":"2024-05-28T14:09:33.151011+0000","last_undegraded":"2024-05-28T14:09:33.151011+0000","last_fullsized":"2024-05-28T14:09:33.151011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762598+0000","last_change":"2024-05-28T14:09:31.758017+0000","last_active":"2024-05-28T14:09:32.762598+0000","last_peered":"2024-05-28T14:09:32.762598+0000","last_clean":"2024-05-28T14:09:32.762598+0000","last_became_active":"2024-05-28T14:09:31.757674+0000","last_became_peered":"2024-05-28T14:09:31.757674+0000","last_unstale":"2024-05-28T14:09:32.762598+0000","last_undegraded":"2024-05-28T14:09:32.762598+0000","last_fullsized":"2024-05-28T14:09:32.762598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:46.760174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762477+0000","last_change":"2024-05-28T14:09:31.762403+0000","last_active":"2024-05-28T14:09:32.762477+0000","last_peered":"2024-05-28T14:09:32.762477+0000","last_clean":"2024-05-28T14:09:32.762477+0000","last_became_active":"2024-05-28T14:09:31.762048+0000","last_became_peered":"2024-05-28T14:09:31.762048+0000","last_unstale":"2024-05-28T14:09:32.762477+0000","last_undegraded":"2024-05-28T14:09:32.762477+0000","last_fullsized":"2024-05-28T14:09:32.762477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:56.072759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751599+0000","last_change":"2024-05-28T14:09:31.747630+0000","last_active":"2024-05-28T14:09:32.751599+0000","last_peered":"2024-05-28T14:09:32.751599+0000","last_clean":"2024-05-28T14:09:32.751599+0000","last_became_active":"2024-05-28T14:09:31.747183+0000","last_became_peered":"2024-05-28T14:09:31.747183+0000","last_unstale":"2024-05-28T14:09:32.751599+0000","last_undegraded":"2024-05-28T14:09:32.751599+0000","last_fullsized":"2024-05-28T14:09:32.751599+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:49:44.321180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151402+0000","last_change":"2024-05-28T14:09:31.768533+0000","last_active":"2024-05-28T14:09:33.151402+0000","last_peered":"2024-05-28T14:09:33.151402+0000","last_clean":"2024-05-28T14:09:33.151402+0000","last_became_active":"2024-05-28T14:09:31.768237+0000","last_became_peered":"2024-05-28T14:09:31.768237+0000","last_unstale":"2024-05-28T14:09:33.151402+0000","last_undegraded":"2024-05-28T14:09:33.151402+0000","last_fullsized":"2024-05-28T14:09:33.151402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752209+0000","last_change":"2024-05-28T14:09:31.752686+0000","last_active":"2024-05-28T14:09:32.752209+0000","last_peered":"2024-05-28T14:09:32.752209+0000","last_clean":"2024-05-28T14:09:32.752209+0000","last_became_active":"2024-05-28T14:09:31.752411+0000","last_became_peered":"2024-05-28T14:09:31.752411+0000","last_unstale":"2024-05-28T14:09:32.752209+0000","last_undegraded":"2024-05-28T14:09:32.752209+0000","last_fullsized":"2024-05-28T14:09:32.752209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:30:26.104660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151595+0000","last_change":"2024-05-28T14:09:30.759880+0000","last_active":"2024-05-28T14:09:33.151595+0000","last_peered":"2024-05-28T14:09:33.151595+0000","last_clean":"2024-05-28T14:09:33.151595+0000","last_became_active":"2024-05-28T14:09:30.759575+0000","last_became_peered":"2024-05-28T14:09:30.759575+0000","last_unstale":"2024-05-28T14:09:33.151595+0000","last_undegraded":"2024-05-28T14:09:33.151595+0000","last_fullsized":"2024-05-28T14:09:33.151595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:55.065487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151493+0000","last_change":"2024-05-28T14:09:31.752593+0000","last_active":"2024-05-28T14:09:33.151493+0000","last_peered":"2024-05-28T14:09:33.151493+0000","last_clean":"2024-05-28T14:09:33.151493+0000","last_became_active":"2024-05-28T14:09:31.752332+0000","last_became_peered":"2024-05-28T14:09:31.752332+0000","last_unstale":"2024-05-28T14:09:33.151493+0000","last_undegraded":"2024-05-28T14:09:33.151493+0000","last_fullsized":"2024-05-28T14:09:33.151493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752466+0000","last_change":"2024-05-28T14:09:30.755821+0000","last_active":"2024-05-28T14:09:32.752466+0000","last_peered":"2024-05-28T14:09:32.752466+0000","last_clean":"2024-05-28T14:09:32.752466+0000","last_became_active":"2024-05-28T14:09:30.755452+0000","last_became_peered":"2024-05-28T14:09:30.755452+0000","last_unstale":"2024-05-28T14:09:32.752466+0000","last_undegraded":"2024-05-28T14:09:32.752466+0000","last_fullsized":"2024-05-28T14:09:32.752466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:18.317793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151220+0000","last_change":"2024-05-28T14:09:31.765522+0000","last_active":"2024-05-28T14:09:33.151220+0000","last_peered":"2024-05-28T14:09:33.151220+0000","last_clean":"2024-05-28T14:09:33.151220+0000","last_became_active":"2024-05-28T14:09:31.764349+0000","last_became_peered":"2024-05-28T14:09:31.764349+0000","last_unstale":"2024-05-28T14:09:33.151220+0000","last_undegraded":"2024-05-28T14:09:33.151220+0000","last_fullsized":"2024-05-28T14:09:33.151220+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765049+0000","last_change":"2024-05-28T14:09:30.752545+0000","last_active":"2024-05-28T14:09:32.765049+0000","last_peered":"2024-05-28T14:09:32.765049+0000","last_clean":"2024-05-28T14:09:32.765049+0000","last_became_active":"2024-05-28T14:09:30.752219+0000","last_became_peered":"2024-05-28T14:09:30.752219+0000","last_unstale":"2024-05-28T14:09:32.765049+0000","last_undegraded":"2024-05-28T14:09:32.765049+0000","last_fullsized":"2024-05-28T14:09:32.765049+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:49:43.313909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151148+0000","last_change":"2024-05-28T14:09:31.765767+0000","last_active":"2024-05-28T14:09:33.151148+0000","last_peered":"2024-05-28T14:09:33.151148+0000","last_clean":"2024-05-28T14:09:33.151148+0000","last_became_active":"2024-05-28T14:09:31.765407+0000","last_became_peered":"2024-05-28T14:09:31.765407+0000","last_unstale":"2024-05-28T14:09:33.151148+0000","last_undegraded":"2024-05-28T14:09:33.151148+0000","last_fullsized":"2024-05-28T14:09:33.151148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:27:09.475191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764918+0000","last_change":"2024-05-28T14:09:30.754683+0000","last_active":"2024-05-28T14:09:32.764918+0000","last_peered":"2024-05-28T14:09:32.764918+0000","last_clean":"2024-05-28T14:09:32.764918+0000","last_became_active":"2024-05-28T14:09:30.754405+0000","last_became_peered":"2024-05-28T14:09:30.754405+0000","last_unstale":"2024-05-28T14:09:32.764918+0000","last_undegraded":"2024-05-28T14:09:32.764918+0000","last_fullsized":"2024-05-28T14:09:32.764918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:29:45.272413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762565+0000","last_change":"2024-05-28T14:09:31.762430+0000","last_active":"2024-05-28T14:09:32.762565+0000","last_peered":"2024-05-28T14:09:32.762565+0000","last_clean":"2024-05-28T14:09:32.762565+0000","last_became_active":"2024-05-28T14:09:31.762101+0000","last_became_peered":"2024-05-28T14:09:31.762101+0000","last_unstale":"2024-05-28T14:09:32.762565+0000","last_undegraded":"2024-05-28T14:09:32.762565+0000","last_fullsized":"2024-05-28T14:09:32.762565+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:40:39.401890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752066+0000","last_change":"2024-05-28T14:09:30.747550+0000","last_active":"2024-05-28T14:09:32.752066+0000","last_peered":"2024-05-28T14:09:32.752066+0000","last_clean":"2024-05-28T14:09:32.752066+0000","last_became_active":"2024-05-28T14:09:30.747222+0000","last_became_peered":"2024-05-28T14:09:30.747222+0000","last_unstale":"2024-05-28T14:09:32.752066+0000","last_undegraded":"2024-05-28T14:09:32.752066+0000","last_fullsized":"2024-05-28T14:09:32.752066+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:58.290350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751952+0000","last_change":"2024-05-28T14:09:31.759538+0000","last_active":"2024-05-28T14:09:32.751952+0000","last_peered":"2024-05-28T14:09:32.751952+0000","last_clean":"2024-05-28T14:09:32.751952+0000","last_became_active":"2024-05-28T14:09:31.759258+0000","last_became_peered":"2024-05-28T14:09:31.759258+0000","last_unstale":"2024-05-28T14:09:32.751952+0000","last_undegraded":"2024-05-28T14:09:32.751952+0000","last_fullsized":"2024-05-28T14:09:32.751952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:35.172304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751683+0000","last_change":"2024-05-28T14:09:30.759103+0000","last_active":"2024-05-28T14:09:32.751683+0000","last_peered":"2024-05-28T14:09:32.751683+0000","last_clean":"2024-05-28T14:09:32.751683+0000","last_became_active":"2024-05-28T14:09:30.758628+0000","last_became_peered":"2024-05-28T14:09:30.758628+0000","last_unstale":"2024-05-28T14:09:32.751683+0000","last_undegraded":"2024-05-28T14:09:32.751683+0000","last_fullsized":"2024-05-28T14:09:32.751683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:54.698345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150467+0000","last_change":"2024-05-28T14:09:31.761382+0000","last_active":"2024-05-28T14:09:33.150467+0000","last_peered":"2024-05-28T14:09:33.150467+0000","last_clean":"2024-05-28T14:09:33.150467+0000","last_became_active":"2024-05-28T14:09:31.761095+0000","last_became_peered":"2024-05-28T14:09:31.761095+0000","last_unstale":"2024-05-28T14:09:33.150467+0000","last_undegraded":"2024-05-28T14:09:33.150467+0000","last_fullsized":"2024-05-28T14:09:33.150467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:31:10.786309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764699+0000","last_change":"2024-05-28T14:09:30.762673+0000","last_active":"2024-05-28T14:09:32.764699+0000","last_peered":"2024-05-28T14:09:32.764699+0000","last_clean":"2024-05-28T14:09:32.764699+0000","last_became_active":"2024-05-28T14:09:30.761793+0000","last_became_peered":"2024-05-28T14:09:30.761793+0000","last_unstale":"2024-05-28T14:09:32.764699+0000","last_undegraded":"2024-05-28T14:09:32.764699+0000","last_fullsized":"2024-05-28T14:09:32.764699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:40:25.005440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747896+0000","last_change":"2024-05-28T14:09:31.766958+0000","last_active":"2024-05-28T14:09:32.747896+0000","last_peered":"2024-05-28T14:09:32.747896+0000","last_clean":"2024-05-28T14:09:32.747896+0000","last_became_active":"2024-05-28T14:09:31.766301+0000","last_became_peered":"2024-05-28T14:09:31.766301+0000","last_unstale":"2024-05-28T14:09:32.747896+0000","last_undegraded":"2024-05-28T14:09:32.747896+0000","last_fullsized":"2024-05-28T14:09:32.747896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:24.707344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762264+0000","last_change":"2024-05-28T14:09:30.746473+0000","last_active":"2024-05-28T14:09:32.762264+0000","last_peered":"2024-05-28T14:09:32.762264+0000","last_clean":"2024-05-28T14:09:32.762264+0000","last_became_active":"2024-05-28T14:09:30.746177+0000","last_became_peered":"2024-05-28T14:09:30.746177+0000","last_unstale":"2024-05-28T14:09:32.762264+0000","last_undegraded":"2024-05-28T14:09:32.762264+0000","last_fullsized":"2024-05-28T14:09:32.762264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:41.356836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.748332+0000","last_change":"2024-05-28T14:09:31.756713+0000","last_active":"2024-05-28T14:09:32.748332+0000","last_peered":"2024-05-28T14:09:32.748332+0000","last_clean":"2024-05-28T14:09:32.748332+0000","last_became_active":"2024-05-28T14:09:31.755716+0000","last_became_peered":"2024-05-28T14:09:31.755716+0000","last_unstale":"2024-05-28T14:09:32.748332+0000","last_undegraded":"2024-05-28T14:09:32.748332+0000","last_fullsized":"2024-05-28T14:09:32.748332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:02:06.074104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151668+0000","last_change":"2024-05-28T14:09:30.758732+0000","last_active":"2024-05-28T14:09:33.151668+0000","last_peered":"2024-05-28T14:09:33.151668+0000","last_clean":"2024-05-28T14:09:33.151668+0000","last_became_active":"2024-05-28T14:09:30.758400+0000","last_became_peered":"2024-05-28T14:09:30.758400+0000","last_unstale":"2024-05-28T14:09:33.151668+0000","last_undegraded":"2024-05-28T14:09:33.151668+0000","last_fullsized":"2024-05-28T14:09:33.151668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:45.752902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750903+0000","last_change":"2024-05-28T14:09:31.762311+0000","last_active":"2024-05-28T14:09:32.750903+0000","last_peered":"2024-05-28T14:09:32.750903+0000","last_clean":"2024-05-28T14:09:32.750903+0000","last_became_active":"2024-05-28T14:09:31.761990+0000","last_became_peered":"2024-05-28T14:09:31.761990+0000","last_unstale":"2024-05-28T14:09:32.750903+0000","last_undegraded":"2024-05-28T14:09:32.750903+0000","last_fullsized":"2024-05-28T14:09:32.750903+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:55:09.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151408+0000","last_change":"2024-05-28T14:09:30.757972+0000","last_active":"2024-05-28T14:09:33.151408+0000","last_peered":"2024-05-28T14:09:33.151408+0000","last_clean":"2024-05-28T14:09:33.151408+0000","last_became_active":"2024-05-28T14:09:30.757716+0000","last_became_peered":"2024-05-28T14:09:30.757716+0000","last_unstale":"2024-05-28T14:09:33.151408+0000","last_undegraded":"2024-05-28T14:09:33.151408+0000","last_fullsized":"2024-05-28T14:09:33.151408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:40:38.394618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764978+0000","last_change":"2024-05-28T14:09:31.754712+0000","last_active":"2024-05-28T14:09:32.764978+0000","last_peered":"2024-05-28T14:09:32.764978+0000","last_clean":"2024-05-28T14:09:32.764978+0000","last_became_active":"2024-05-28T14:09:31.754427+0000","last_became_peered":"2024-05-28T14:09:31.754427+0000","last_unstale":"2024-05-28T14:09:32.764978+0000","last_undegraded":"2024-05-28T14:09:32.764978+0000","last_fullsized":"2024-05-28T14:09:32.764978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151098+0000","last_change":"2024-05-28T14:09:30.756257+0000","last_active":"2024-05-28T14:09:33.151098+0000","last_peered":"2024-05-28T14:09:33.151098+0000","last_clean":"2024-05-28T14:09:33.151098+0000","last_became_active":"2024-05-28T14:09:30.755923+0000","last_became_peered":"2024-05-28T14:09:30.755923+0000","last_unstale":"2024-05-28T14:09:33.151098+0000","last_undegraded":"2024-05-28T14:09:33.151098+0000","last_fullsized":"2024-05-28T14:09:33.151098+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:54.698345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752090+0000","last_change":"2024-05-28T14:09:31.748217+0000","last_active":"2024-05-28T14:09:32.752090+0000","last_peered":"2024-05-28T14:09:32.752090+0000","last_clean":"2024-05-28T14:09:32.752090+0000","last_became_active":"2024-05-28T14:09:31.747336+0000","last_became_peered":"2024-05-28T14:09:31.747336+0000","last_unstale":"2024-05-28T14:09:32.752090+0000","last_undegraded":"2024-05-28T14:09:32.752090+0000","last_fullsized":"2024-05-28T14:09:32.752090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:45:59.764496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"89'6","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750773+0000","last_change":"2024-05-28T14:09:30.760654+0000","last_active":"2024-05-28T14:09:32.750773+0000","last_peered":"2024-05-28T14:09:32.750773+0000","last_clean":"2024-05-28T14:09:32.750773+0000","last_became_active":"2024-05-28T14:09:30.760328+0000","last_became_peered":"2024-05-28T14:09:30.760328+0000","last_unstale":"2024-05-28T14:09:32.750773+0000","last_undegraded":"2024-05-28T14:09:32.750773+0000","last_fullsized":"2024-05-28T14:09:32.750773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:54:12.659735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764801+0000","last_change":"2024-05-28T14:09:31.742984+0000","last_active":"2024-05-28T14:09:32.764801+0000","last_peered":"2024-05-28T14:09:32.764801+0000","last_clean":"2024-05-28T14:09:32.764801+0000","last_became_active":"2024-05-28T14:09:31.742692+0000","last_became_peered":"2024-05-28T14:09:31.742692+0000","last_unstale":"2024-05-28T14:09:32.764801+0000","last_undegraded":"2024-05-28T14:09:32.764801+0000","last_fullsized":"2024-05-28T14:09:32.764801+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762506+0000","last_change":"2024-05-28T14:09:30.751922+0000","last_active":"2024-05-28T14:09:32.762506+0000","last_peered":"2024-05-28T14:09:32.762506+0000","last_clean":"2024-05-28T14:09:32.762506+0000","last_became_active":"2024-05-28T14:09:30.751646+0000","last_became_peered":"2024-05-28T14:09:30.751646+0000","last_unstale":"2024-05-28T14:09:32.762506+0000","last_undegraded":"2024-05-28T14:09:32.762506+0000","last_fullsized":"2024-05-28T14:09:32.762506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:24:14.215364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150460+0000","last_change":"2024-05-28T14:09:31.763288+0000","last_active":"2024-05-28T14:09:33.150460+0000","last_peered":"2024-05-28T14:09:33.150460+0000","last_clean":"2024-05-28T14:09:33.150460+0000","last_became_active":"2024-05-28T14:09:31.762800+0000","last_became_peered":"2024-05-28T14:09:31.762800+0000","last_unstale":"2024-05-28T14:09:33.150460+0000","last_undegraded":"2024-05-28T14:09:33.150460+0000","last_fullsized":"2024-05-28T14:09:33.150460+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:46:19.764662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747904+0000","last_change":"2024-05-28T14:09:30.753559+0000","last_active":"2024-05-28T14:09:32.747904+0000","last_peered":"2024-05-28T14:09:32.747904+0000","last_clean":"2024-05-28T14:09:32.747904+0000","last_became_active":"2024-05-28T14:09:30.753253+0000","last_became_peered":"2024-05-28T14:09:30.753253+0000","last_unstale":"2024-05-28T14:09:32.747904+0000","last_undegraded":"2024-05-28T14:09:32.747904+0000","last_fullsized":"2024-05-28T14:09:32.747904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:15:03.984981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762339+0000","last_change":"2024-05-28T14:09:31.765598+0000","last_active":"2024-05-28T14:09:32.762339+0000","last_peered":"2024-05-28T14:09:32.762339+0000","last_clean":"2024-05-28T14:09:32.762339+0000","last_became_active":"2024-05-28T14:09:31.760967+0000","last_became_peered":"2024-05-28T14:09:31.760967+0000","last_unstale":"2024-05-28T14:09:32.762339+0000","last_undegraded":"2024-05-28T14:09:32.762339+0000","last_fullsized":"2024-05-28T14:09:32.762339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:24:40.914669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.750623+0000","last_change":"2024-05-28T14:09:30.758094+0000","last_active":"2024-05-28T14:09:32.750623+0000","last_peered":"2024-05-28T14:09:32.750623+0000","last_clean":"2024-05-28T14:09:32.750623+0000","last_became_active":"2024-05-28T14:09:30.757801+0000","last_became_peered":"2024-05-28T14:09:30.757801+0000","last_unstale":"2024-05-28T14:09:32.750623+0000","last_undegraded":"2024-05-28T14:09:32.750623+0000","last_fullsized":"2024-05-28T14:09:32.750623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:40:25.005440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762291+0000","last_change":"2024-05-28T14:09:31.754734+0000","last_active":"2024-05-28T14:09:32.762291+0000","last_peered":"2024-05-28T14:09:32.762291+0000","last_clean":"2024-05-28T14:09:32.762291+0000","last_became_active":"2024-05-28T14:09:31.754459+0000","last_became_peered":"2024-05-28T14:09:31.754459+0000","last_unstale":"2024-05-28T14:09:32.762291+0000","last_undegraded":"2024-05-28T14:09:32.762291+0000","last_fullsized":"2024-05-28T14:09:32.762291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:19:30.516845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"89'8","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764575+0000","last_change":"2024-05-28T14:09:30.759773+0000","last_active":"2024-05-28T14:09:32.764575+0000","last_peered":"2024-05-28T14:09:32.764575+0000","last_clean":"2024-05-28T14:09:32.764575+0000","last_became_active":"2024-05-28T14:09:30.759218+0000","last_became_peered":"2024-05-28T14:09:30.759218+0000","last_unstale":"2024-05-28T14:09:32.764575+0000","last_undegraded":"2024-05-28T14:09:32.764575+0000","last_fullsized":"2024-05-28T14:09:32.764575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:14:14.485471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151031+0000","last_change":"2024-05-28T14:09:31.766044+0000","last_active":"2024-05-28T14:09:33.151031+0000","last_peered":"2024-05-28T14:09:33.151031+0000","last_clean":"2024-05-28T14:09:33.151031+0000","last_became_active":"2024-05-28T14:09:31.763364+0000","last_became_peered":"2024-05-28T14:09:31.763364+0000","last_unstale":"2024-05-28T14:09:33.151031+0000","last_undegraded":"2024-05-28T14:09:33.151031+0000","last_fullsized":"2024-05-28T14:09:33.151031+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:54:38.549065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151738+0000","last_change":"2024-05-28T14:09:30.763452+0000","last_active":"2024-05-28T14:09:33.151738+0000","last_peered":"2024-05-28T14:09:33.151738+0000","last_clean":"2024-05-28T14:09:33.151738+0000","last_became_active":"2024-05-28T14:09:30.763075+0000","last_became_peered":"2024-05-28T14:09:30.763075+0000","last_unstale":"2024-05-28T14:09:33.151738+0000","last_undegraded":"2024-05-28T14:09:33.151738+0000","last_fullsized":"2024-05-28T14:09:33.151738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:37.606903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151668+0000","last_change":"2024-05-28T14:09:31.769821+0000","last_active":"2024-05-28T14:09:33.151668+0000","last_peered":"2024-05-28T14:09:33.151668+0000","last_clean":"2024-05-28T14:09:33.151668+0000","last_became_active":"2024-05-28T14:09:31.769534+0000","last_became_peered":"2024-05-28T14:09:31.769534+0000","last_unstale":"2024-05-28T14:09:33.151668+0000","last_undegraded":"2024-05-28T14:09:33.151668+0000","last_fullsized":"2024-05-28T14:09:33.151668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765169+0000","last_change":"2024-05-28T14:09:30.752703+0000","last_active":"2024-05-28T14:09:32.765169+0000","last_peered":"2024-05-28T14:09:32.765169+0000","last_clean":"2024-05-28T14:09:32.765169+0000","last_became_active":"2024-05-28T14:09:30.752399+0000","last_became_peered":"2024-05-28T14:09:30.752399+0000","last_unstale":"2024-05-28T14:09:32.765169+0000","last_undegraded":"2024-05-28T14:09:32.765169+0000","last_fullsized":"2024-05-28T14:09:32.765169+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:42.742326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752236+0000","last_change":"2024-05-28T14:09:31.757626+0000","last_active":"2024-05-28T14:09:32.752236+0000","last_peered":"2024-05-28T14:09:32.752236+0000","last_clean":"2024-05-28T14:09:32.752236+0000","last_became_active":"2024-05-28T14:09:31.756947+0000","last_became_peered":"2024-05-28T14:09:31.756947+0000","last_unstale":"2024-05-28T14:09:32.752236+0000","last_undegraded":"2024-05-28T14:09:32.752236+0000","last_fullsized":"2024-05-28T14:09:32.752236+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.765042+0000","last_change":"2024-05-28T14:09:30.760474+0000","last_active":"2024-05-28T14:09:32.765042+0000","last_peered":"2024-05-28T14:09:32.765042+0000","last_clean":"2024-05-28T14:09:32.765042+0000","last_became_active":"2024-05-28T14:09:30.759783+0000","last_became_peered":"2024-05-28T14:09:30.759783+0000","last_unstale":"2024-05-28T14:09:32.765042+0000","last_undegraded":"2024-05-28T14:09:32.765042+0000","last_fullsized":"2024-05-28T14:09:32.765042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:49:41.271344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149495+0000","last_change":"2024-05-28T14:09:31.761651+0000","last_active":"2024-05-28T14:09:33.149495+0000","last_peered":"2024-05-28T14:09:33.149495+0000","last_clean":"2024-05-28T14:09:33.149495+0000","last_became_active":"2024-05-28T14:09:31.761350+0000","last_became_peered":"2024-05-28T14:09:31.761350+0000","last_unstale":"2024-05-28T14:09:33.149495+0000","last_undegraded":"2024-05-28T14:09:33.149495+0000","last_fullsized":"2024-05-28T14:09:33.149495+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:13:58.981771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.748033+0000","last_change":"2024-05-28T14:09:30.754286+0000","last_active":"2024-05-28T14:09:32.748033+0000","last_peered":"2024-05-28T14:09:32.748033+0000","last_clean":"2024-05-28T14:09:32.748033+0000","last_became_active":"2024-05-28T14:09:30.753997+0000","last_became_peered":"2024-05-28T14:09:30.753997+0000","last_unstale":"2024-05-28T14:09:32.748033+0000","last_undegraded":"2024-05-28T14:09:32.748033+0000","last_fullsized":"2024-05-28T14:09:32.748033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:34:27.167006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751282+0000","last_change":"2024-05-28T14:09:31.760545+0000","last_active":"2024-05-28T14:09:32.751282+0000","last_peered":"2024-05-28T14:09:32.751282+0000","last_clean":"2024-05-28T14:09:32.751282+0000","last_became_active":"2024-05-28T14:09:31.760024+0000","last_became_peered":"2024-05-28T14:09:31.760024+0000","last_unstale":"2024-05-28T14:09:32.751282+0000","last_undegraded":"2024-05-28T14:09:32.751282+0000","last_fullsized":"2024-05-28T14:09:32.751282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:04:31.235124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149448+0000","last_change":"2024-05-28T14:09:30.749340+0000","last_active":"2024-05-28T14:09:33.149448+0000","last_peered":"2024-05-28T14:09:33.149448+0000","last_clean":"2024-05-28T14:09:33.149448+0000","last_became_active":"2024-05-28T14:09:30.749082+0000","last_became_peered":"2024-05-28T14:09:30.749082+0000","last_unstale":"2024-05-28T14:09:33.149448+0000","last_undegraded":"2024-05-28T14:09:33.149448+0000","last_fullsized":"2024-05-28T14:09:33.149448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:53:51.026943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149352+0000","last_change":"2024-05-28T14:09:31.755784+0000","last_active":"2024-05-28T14:09:33.149352+0000","last_peered":"2024-05-28T14:09:33.149352+0000","last_clean":"2024-05-28T14:09:33.149352+0000","last_became_active":"2024-05-28T14:09:31.755418+0000","last_became_peered":"2024-05-28T14:09:31.755418+0000","last_unstale":"2024-05-28T14:09:33.149352+0000","last_undegraded":"2024-05-28T14:09:33.149352+0000","last_fullsized":"2024-05-28T14:09:33.149352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:29:32.550652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.149017+0000","last_change":"2024-05-28T14:09:30.757283+0000","last_active":"2024-05-28T14:09:33.149017+0000","last_peered":"2024-05-28T14:09:33.149017+0000","last_clean":"2024-05-28T14:09:33.149017+0000","last_became_active":"2024-05-28T14:09:30.756776+0000","last_became_peered":"2024-05-28T14:09:30.756776+0000","last_unstale":"2024-05-28T14:09:33.149017+0000","last_undegraded":"2024-05-28T14:09:33.149017+0000","last_fullsized":"2024-05-28T14:09:33.149017+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:25:28.204845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751081+0000","last_change":"2024-05-28T14:09:31.748847+0000","last_active":"2024-05-28T14:09:32.751081+0000","last_peered":"2024-05-28T14:09:32.751081+0000","last_clean":"2024-05-28T14:09:32.751081+0000","last_became_active":"2024-05-28T14:09:31.748479+0000","last_became_peered":"2024-05-28T14:09:31.748479+0000","last_unstale":"2024-05-28T14:09:32.751081+0000","last_undegraded":"2024-05-28T14:09:32.751081+0000","last_fullsized":"2024-05-28T14:09:32.751081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:49:21.045351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.747718+0000","last_change":"2024-05-28T14:09:30.755244+0000","last_active":"2024-05-28T14:09:32.747718+0000","last_peered":"2024-05-28T14:09:32.747718+0000","last_clean":"2024-05-28T14:09:32.747718+0000","last_became_active":"2024-05-28T14:09:30.754984+0000","last_became_peered":"2024-05-28T14:09:30.754984+0000","last_unstale":"2024-05-28T14:09:32.747718+0000","last_undegraded":"2024-05-28T14:09:32.747718+0000","last_fullsized":"2024-05-28T14:09:32.747718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:14:58.988961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752367+0000","last_change":"2024-05-28T14:09:31.745138+0000","last_active":"2024-05-28T14:09:32.752367+0000","last_peered":"2024-05-28T14:09:32.752367+0000","last_clean":"2024-05-28T14:09:32.752367+0000","last_became_active":"2024-05-28T14:09:31.744819+0000","last_became_peered":"2024-05-28T14:09:31.744819+0000","last_unstale":"2024-05-28T14:09:32.752367+0000","last_undegraded":"2024-05-28T14:09:32.752367+0000","last_fullsized":"2024-05-28T14:09:32.752367+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762790+0000","last_change":"2024-05-28T14:09:30.760787+0000","last_active":"2024-05-28T14:09:32.762790+0000","last_peered":"2024-05-28T14:09:32.762790+0000","last_clean":"2024-05-28T14:09:32.762790+0000","last_became_active":"2024-05-28T14:09:30.760502+0000","last_became_peered":"2024-05-28T14:09:30.760502+0000","last_unstale":"2024-05-28T14:09:32.762790+0000","last_undegraded":"2024-05-28T14:09:32.762790+0000","last_fullsized":"2024-05-28T14:09:32.762790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:31:35.886734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150913+0000","last_change":"2024-05-28T14:09:31.771112+0000","last_active":"2024-05-28T14:09:33.150913+0000","last_peered":"2024-05-28T14:09:33.150913+0000","last_clean":"2024-05-28T14:09:33.150913+0000","last_became_active":"2024-05-28T14:09:31.770660+0000","last_became_peered":"2024-05-28T14:09:31.770660+0000","last_unstale":"2024-05-28T14:09:33.150913+0000","last_undegraded":"2024-05-28T14:09:33.150913+0000","last_fullsized":"2024-05-28T14:09:33.150913+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:02:06.074104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762843+0000","last_change":"2024-05-28T14:09:30.751801+0000","last_active":"2024-05-28T14:09:32.762843+0000","last_peered":"2024-05-28T14:09:32.762843+0000","last_clean":"2024-05-28T14:09:32.762843+0000","last_became_active":"2024-05-28T14:09:30.751504+0000","last_became_peered":"2024-05-28T14:09:30.751504+0000","last_unstale":"2024-05-28T14:09:32.762843+0000","last_undegraded":"2024-05-28T14:09:32.762843+0000","last_fullsized":"2024-05-28T14:09:32.762843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:42.042818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151673+0000","last_change":"2024-05-28T14:09:31.765811+0000","last_active":"2024-05-28T14:09:33.151673+0000","last_peered":"2024-05-28T14:09:33.151673+0000","last_clean":"2024-05-28T14:09:33.151673+0000","last_became_active":"2024-05-28T14:09:31.764710+0000","last_became_peered":"2024-05-28T14:09:31.764710+0000","last_unstale":"2024-05-28T14:09:33.151673+0000","last_undegraded":"2024-05-28T14:09:33.151673+0000","last_fullsized":"2024-05-28T14:09:33.151673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:57.967039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151377+0000","last_change":"2024-05-28T14:09:30.760473+0000","last_active":"2024-05-28T14:09:33.151377+0000","last_peered":"2024-05-28T14:09:33.151377+0000","last_clean":"2024-05-28T14:09:33.151377+0000","last_became_active":"2024-05-28T14:09:30.760129+0000","last_became_peered":"2024-05-28T14:09:30.760129+0000","last_unstale":"2024-05-28T14:09:33.151377+0000","last_undegraded":"2024-05-28T14:09:33.151377+0000","last_fullsized":"2024-05-28T14:09:33.151377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:19:18.317793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.752265+0000","last_change":"2024-05-28T14:09:31.748833+0000","last_active":"2024-05-28T14:09:32.752265+0000","last_peered":"2024-05-28T14:09:32.752265+0000","last_clean":"2024-05-28T14:09:32.752265+0000","last_became_active":"2024-05-28T14:09:31.747731+0000","last_became_peered":"2024-05-28T14:09:31.747731+0000","last_unstale":"2024-05-28T14:09:32.752265+0000","last_undegraded":"2024-05-28T14:09:32.752265+0000","last_fullsized":"2024-05-28T14:09:32.752265+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:17:03.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151485+0000","last_change":"2024-05-28T14:09:30.758617+0000","last_active":"2024-05-28T14:09:33.151485+0000","last_peered":"2024-05-28T14:09:33.151485+0000","last_clean":"2024-05-28T14:09:33.151485+0000","last_became_active":"2024-05-28T14:09:30.758317+0000","last_became_peered":"2024-05-28T14:09:30.758317+0000","last_unstale":"2024-05-28T14:09:33.151485+0000","last_undegraded":"2024-05-28T14:09:33.151485+0000","last_fullsized":"2024-05-28T14:09:33.151485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:10.090547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751157+0000","last_change":"2024-05-28T14:09:31.760268+0000","last_active":"2024-05-28T14:09:32.751157+0000","last_peered":"2024-05-28T14:09:32.751157+0000","last_clean":"2024-05-28T14:09:32.751157+0000","last_became_active":"2024-05-28T14:09:31.759889+0000","last_became_peered":"2024-05-28T14:09:31.759889+0000","last_unstale":"2024-05-28T14:09:32.751157+0000","last_undegraded":"2024-05-28T14:09:32.751157+0000","last_fullsized":"2024-05-28T14:09:32.751157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764892+0000","last_change":"2024-05-28T14:09:30.754916+0000","last_active":"2024-05-28T14:09:32.764892+0000","last_peered":"2024-05-28T14:09:32.764892+0000","last_clean":"2024-05-28T14:09:32.764892+0000","last_became_active":"2024-05-28T14:09:30.754019+0000","last_became_peered":"2024-05-28T14:09:30.754019+0000","last_unstale":"2024-05-28T14:09:32.764892+0000","last_undegraded":"2024-05-28T14:09:32.764892+0000","last_fullsized":"2024-05-28T14:09:32.764892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:34.165033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.764999+0000","last_change":"2024-05-28T14:09:31.748964+0000","last_active":"2024-05-28T14:09:32.764999+0000","last_peered":"2024-05-28T14:09:32.764999+0000","last_clean":"2024-05-28T14:09:32.764999+0000","last_became_active":"2024-05-28T14:09:31.748393+0000","last_became_peered":"2024-05-28T14:09:31.748393+0000","last_unstale":"2024-05-28T14:09:32.764999+0000","last_undegraded":"2024-05-28T14:09:32.764999+0000","last_fullsized":"2024-05-28T14:09:32.764999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:40:25.896589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751724+0000","last_change":"2024-05-28T14:09:30.748735+0000","last_active":"2024-05-28T14:09:32.751724+0000","last_peered":"2024-05-28T14:09:32.751724+0000","last_clean":"2024-05-28T14:09:32.751724+0000","last_became_active":"2024-05-28T14:09:30.748404+0000","last_became_peered":"2024-05-28T14:09:30.748404+0000","last_unstale":"2024-05-28T14:09:32.751724+0000","last_undegraded":"2024-05-28T14:09:32.751724+0000","last_fullsized":"2024-05-28T14:09:32.751724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:37.606903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150429+0000","last_change":"2024-05-28T14:09:31.764716+0000","last_active":"2024-05-28T14:09:33.150429+0000","last_peered":"2024-05-28T14:09:33.150429+0000","last_clean":"2024-05-28T14:09:33.150429+0000","last_became_active":"2024-05-28T14:09:31.764407+0000","last_became_peered":"2024-05-28T14:09:31.764407+0000","last_unstale":"2024-05-28T14:09:33.150429+0000","last_undegraded":"2024-05-28T14:09:33.150429+0000","last_fullsized":"2024-05-28T14:09:33.150429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:14:59.996233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151098+0000","last_change":"2024-05-28T14:09:30.755558+0000","last_active":"2024-05-28T14:09:33.151098+0000","last_peered":"2024-05-28T14:09:33.151098+0000","last_clean":"2024-05-28T14:09:33.151098+0000","last_became_active":"2024-05-28T14:09:30.755257+0000","last_became_peered":"2024-05-28T14:09:30.755257+0000","last_unstale":"2024-05-28T14:09:33.151098+0000","last_undegraded":"2024-05-28T14:09:33.151098+0000","last_fullsized":"2024-05-28T14:09:33.151098+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:58.290350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150859+0000","last_change":"2024-05-28T14:09:31.758209+0000","last_active":"2024-05-28T14:09:33.150859+0000","last_peered":"2024-05-28T14:09:33.150859+0000","last_clean":"2024-05-28T14:09:33.150859+0000","last_became_active":"2024-05-28T14:09:31.757910+0000","last_became_peered":"2024-05-28T14:09:31.757910+0000","last_unstale":"2024-05-28T14:09:33.150859+0000","last_undegraded":"2024-05-28T14:09:33.150859+0000","last_fullsized":"2024-05-28T14:09:33.150859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:36:07.311954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.751556+0000","last_change":"2024-05-28T14:09:30.754615+0000","last_active":"2024-05-28T14:09:32.751556+0000","last_peered":"2024-05-28T14:09:32.751556+0000","last_clean":"2024-05-28T14:09:32.751556+0000","last_became_active":"2024-05-28T14:09:30.754325+0000","last_became_peered":"2024-05-28T14:09:30.754325+0000","last_unstale":"2024-05-28T14:09:32.751556+0000","last_undegraded":"2024-05-28T14:09:32.751556+0000","last_fullsized":"2024-05-28T14:09:32.751556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:10:45.752902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:32.762719+0000","last_change":"2024-05-28T14:09:31.760657+0000","last_active":"2024-05-28T14:09:32.762719+0000","last_peered":"2024-05-28T14:09:32.762719+0000","last_clean":"2024-05-28T14:09:32.762719+0000","last_became_active":"2024-05-28T14:09:31.759615+0000","last_became_peered":"2024-05-28T14:09:31.759615+0000","last_unstale":"2024-05-28T14:09:32.762719+0000","last_undegraded":"2024-05-28T14:09:32.762719+0000","last_fullsized":"2024-05-28T14:09:32.762719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:30.724896+0000","last_clean_scrub_stamp":"2024-05-28T14:09:30.724896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:23:55.705617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.151176+0000","last_change":"2024-05-28T14:09:30.762060+0000","last_active":"2024-05-28T14:09:33.151176+0000","last_peered":"2024-05-28T14:09:33.151176+0000","last_clean":"2024-05-28T14:09:33.151176+0000","last_became_active":"2024-05-28T14:09:30.761778+0000","last_became_peered":"2024-05-28T14:09:30.761778+0000","last_unstale":"2024-05-28T14:09:33.151176+0000","last_undegraded":"2024-05-28T14:09:33.151176+0000","last_fullsized":"2024-05-28T14:09:33.151176+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:09:29.717625+0000","last_clean_scrub_stamp":"2024-05-28T14:09:29.717625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:56.959767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":300,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-28T14:09:33.150107+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:09:33.150107+0000","last_peered":"2024-05-28T14:09:33.150107+0000","last_clean":"2024-05-28T14:09:33.150107+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:09:33.150107+0000","last_undegraded":"2024-05-28T14:09:33.150107+0000","last_fullsized":"2024-05-28T14:09:33.150107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123028,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248812,"statfs":{"total":96636764160,"available":96510783488,"internally_reserved":0,"allocated":1703936,"data_stored":1068711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122448,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249392,"statfs":{"total":96636764160,"available":96511377408,"internally_reserved":0,"allocated":1118208,"data_stored":478967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":124254662},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76628,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295212,"statfs":{"total":96636764160,"available":96558297088,"internally_reserved":0,"allocated":1122304,"data_stored":479907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82184,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289656,"statfs":{"total":96636764160,"available":96552607744,"internally_reserved":0,"allocated":1105920,"data_stored":478273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574987,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130772,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241068,"statfs":{"total":96636764160,"available":96502853632,"internally_reserved":0,"allocated":1118208,"data_stored":479885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":132774344},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144652,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227188,"statfs":{"total":96636764160,"available":96488640512,"internally_reserved":0,"allocated":1110016,"data_stored":478855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108564,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94263276,"statfs":{"total":96636764160,"available":96525594624,"internally_reserved":0,"allocated":1118208,"data_stored":478467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64724,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307116,"statfs":{"total":96636764160,"available":96570486784,"internally_reserved":0,"allocated":1703936,"data_stored":1068675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:09:40.808 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-28T14:09:40.808 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-28T14:09:40.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.009+0000 7f32a7600640 1 Processor -- start 2024-05-28T14:09:40.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.009+0000 7f32a7600640 1 -- start start 2024-05-28T14:09:40.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.013+0000 7f32a7600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f32a0071de0 0x7f32a00721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:40.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.013+0000 7f32a7600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32a00ce3b0 0x7f32a00ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:40.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.013+0000 7f32a7600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32a0072700 con 0x7f32a0141d10 2024-05-28T14:09:40.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.013+0000 7f32a7600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f32a0160230 con 0x7f32a0071de0 2024-05-28T14:09:40.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.013+0000 7f32a7600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f32a0160510 con 0x7f32a00ce3b0 2024-05-28T14:09:40.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.013+0000 7f3297e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32a00ce3b0 0x7f32a00ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:40.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.013+0000 7f32a4c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f32a0071de0 0x7f32a00721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:40.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.013+0000 7f32a4c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f32a0071de0 0x7f32a00721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:59406/0 (socket says 172.21.15.42:59406) 2024-05-28T14:09:40.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.013+0000 7f32a4c00640 1 -- 172.21.15.42:0/3337946476 learned_addr learned my addr 172.21.15.42:0/3337946476 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:40.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.013+0000 7f3297e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32a00ce3b0 0x7f32a00ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:60676/0 (socket says 172.21.15.42:60676) 2024-05-28T14:09:40.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.013+0000 7f3297e00640 1 -- 172.21.15.42:0/3337946476 >> v2:172.21.15.63:3300/0 conn(0x7f32a0071de0 msgr2=0x7f32a00721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:40.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.013+0000 7f3297e00640 1 --2- 172.21.15.42:0/3337946476 >> v2:172.21.15.63:3300/0 conn(0x7f32a0071de0 0x7f32a00721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:40.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.013+0000 7f3297e00640 1 -- 172.21.15.42:0/3337946476 >> v1:172.21.15.63:6789/0 conn(0x7f32a0141d10 legacy=0x7f32a01421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:40.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.013+0000 7f3297e00640 1 -- 172.21.15.42:0/3337946476 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a01607f0 con 0x7f32a00ce3b0 2024-05-28T14:09:40.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.017+0000 7f3297e00640 1 --2- 172.21.15.42:0/3337946476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32a00ce3b0 0x7f32a00ce810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f32880097c0 tx=0x7f328801c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c27a754be16ee24c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:41.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.017+0000 7f329fe00640 1 -- 172.21.15.42:0/3337946476 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32880b0070 con 0x7f32a00ce3b0 2024-05-28T14:09:41.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.017+0000 7f329fe00640 1 -- 172.21.15.42:0/3337946476 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32880ab600 con 0x7f32a00ce3b0 2024-05-28T14:09:41.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.017+0000 7f329fe00640 1 -- 172.21.15.42:0/3337946476 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32880ab9d0 con 0x7f32a00ce3b0 2024-05-28T14:09:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.017+0000 7f32a7600640 1 -- 172.21.15.42:0/3337946476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32a00ce3b0 msgr2=0x7f32a00ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.017+0000 7f32a7600640 1 --2- 172.21.15.42:0/3337946476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32a00ce3b0 0x7f32a00ce810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f32880097c0 tx=0x7f328801c380 comp rx=0 tx=0).stop 2024-05-28T14:09:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.017+0000 7f32a7600640 1 -- 172.21.15.42:0/3337946476 shutdown_connections 2024-05-28T14:09:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.017+0000 7f32a7600640 1 --2- 172.21.15.42:0/3337946476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32a00ce3b0 0x7f32a00ce810 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.017+0000 7f32a7600640 1 --2- 172.21.15.42:0/3337946476 >> v2:172.21.15.63:3300/0 conn(0x7f32a0071de0 0x7f32a00721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.017+0000 7f32a7600640 1 -- 172.21.15.42:0/3337946476 >> 172.21.15.42:0/3337946476 conn(0x7f32a00b21f0 msgr2=0x7f32a00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.017+0000 7f32a7600640 1 -- 172.21.15.42:0/3337946476 shutdown_connections 2024-05-28T14:09:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.017+0000 7f32a7600640 1 -- 172.21.15.42:0/3337946476 wait complete. 2024-05-28T14:09:41.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.017+0000 7f32a7600640 1 Processor -- start 2024-05-28T14:09:41.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.017+0000 7f32a7600640 1 -- start start 2024-05-28T14:09:41.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a7600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32a0071de0 0x7f32a015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:41.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a7600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32a00ce3b0 0x7f32a0139510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:41.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a7600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0141d10 0x7f32a00b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:41.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a7600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0169e30 con 0x7f32a0071de0 2024-05-28T14:09:41.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a7600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0169cb0 con 0x7f32a0141d10 2024-05-28T14:09:41.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a7600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f32a0169fb0 con 0x7f32a00ce3b0 2024-05-28T14:09:41.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a4c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32a0071de0 0x7f32a015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:41.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a4c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32a0071de0 0x7f32a015dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38898/0 (socket says 172.21.15.42:38898) 2024-05-28T14:09:41.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a4c00640 1 -- 172.21.15.42:0/2578998543 learned_addr learned my addr 172.21.15.42:0/2578998543 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:41.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f3297e00640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32a00ce3b0 0x7f32a0139510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:41.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a5600640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0141d10 0x7f32a00b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:41.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a4c00640 1 -- 172.21.15.42:0/2578998543 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32a00ce3b0 msgr2=0x7f32a0139510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:41.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a4c00640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32a00ce3b0 0x7f32a0139510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a4c00640 1 -- 172.21.15.42:0/2578998543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0141d10 msgr2=0x7f32a00b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:41.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a4c00640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0141d10 0x7f32a00b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a4c00640 1 -- 172.21.15.42:0/2578998543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a0139a50 con 0x7f32a0071de0 2024-05-28T14:09:41.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a5600640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0141d10 0x7f32a00b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:41.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a4c00640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32a0071de0 0x7f32a015dcf0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f329000e7f0 tx=0x7f329000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:41.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f329d600640 1 -- 172.21.15.42:0/2578998543 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290017070 con 0x7f32a0071de0 2024-05-28T14:09:41.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f329d600640 1 -- 172.21.15.42:0/2578998543 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329000c940 con 0x7f32a0071de0 2024-05-28T14:09:41.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f329d600640 1 -- 172.21.15.42:0/2578998543 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329000cd30 con 0x7f32a0071de0 2024-05-28T14:09:41.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a7600640 1 -- 172.21.15.42:0/2578998543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a00b3fb0 con 0x7f32a0071de0 2024-05-28T14:09:41.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a7600640 1 -- 172.21.15.42:0/2578998543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a00b4570 con 0x7f32a0071de0 2024-05-28T14:09:41.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.021+0000 7f32a7600640 1 -- 172.21.15.42:0/2578998543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a00738b0 con 0x7f32a0071de0 2024-05-28T14:09:41.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.025+0000 7f329d600640 1 -- 172.21.15.42:0/2578998543 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f329001a070 con 0x7f32a0071de0 2024-05-28T14:09:41.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.025+0000 7f329d600640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f327c0694f0 0x7f327c06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:41.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.025+0000 7f329d600640 1 -- 172.21.15.42:0/2578998543 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7f3290085750 con 0x7f32a0071de0 2024-05-28T14:09:41.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.025+0000 7f3297e00640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f327c0694f0 0x7f327c06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:41.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.025+0000 7f3297e00640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f327c0694f0 0x7f327c06b9b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f32880ad000 tx=0x7f32880a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:41.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.029+0000 7f329d600640 1 -- 172.21.15.42:0/2578998543 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3290056d40 con 0x7f32a0071de0 2024-05-28T14:09:41.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.249+0000 7f32a7600640 1 -- 172.21.15.42:0/2578998543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f32a004fdd0 con 0x7f32a0071de0 2024-05-28T14:09:41.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.253+0000 7f329d600640 1 -- 172.21.15.42:0/2578998543 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f329005b040 con 0x7f32a0071de0 2024-05-28T14:09:41.238 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-05-28T14:09:41.238 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw== 2024-05-28T14:09:41.238 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow" 2024-05-28T14:09:41.238 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-05-28T14:09:41.238 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-28T14:09:41.238 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.0 2024-05-28T14:09:41.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.257+0000 7f32a7600640 1 -- 172.21.15.42:0/2578998543 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f327c0694f0 msgr2=0x7f327c06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:41.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.257+0000 7f32a7600640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f327c0694f0 0x7f327c06b9b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f32880ad000 tx=0x7f32880a6000 comp rx=0 tx=0).stop 2024-05-28T14:09:41.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.257+0000 7f32a7600640 1 -- 172.21.15.42:0/2578998543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32a0071de0 msgr2=0x7f32a015dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:41.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.257+0000 7f32a7600640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32a0071de0 0x7f32a015dcf0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f329000e7f0 tx=0x7f329000ecb0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.257+0000 7f32a7600640 1 -- 172.21.15.42:0/2578998543 shutdown_connections 2024-05-28T14:09:41.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.257+0000 7f32a7600640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f327c0694f0 0x7f327c06b9b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.257+0000 7f32a7600640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0141d10 0x7f32a00b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.257+0000 7f32a7600640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32a00ce3b0 0x7f32a0139510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.257+0000 7f32a7600640 1 --2- 172.21.15.42:0/2578998543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32a0071de0 0x7f32a015dcf0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.257+0000 7f32a7600640 1 -- 172.21.15.42:0/2578998543 >> 172.21.15.42:0/2578998543 conn(0x7f32a00b21f0 msgr2=0x7f32a015b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:41.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.257+0000 7f32a7600640 1 -- 172.21.15.42:0/2578998543 shutdown_connections 2024-05-28T14:09:41.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.257+0000 7f32a7600640 1 -- 172.21.15.42:0/2578998543 wait complete. 2024-05-28T14:09:41.260 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:09:41.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b28200640 1 Processor -- start 2024-05-28T14:09:41.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b28200640 1 -- start start 2024-05-28T14:09:41.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b28200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5b20156a70 0x7f5b2006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:41.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b28200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5b20156e50 0x7f5b200af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:41.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b28200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b200afc80 con 0x7f5b200af880 2024-05-28T14:09:41.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b28200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5b2015cb90 con 0x7f5b20156a70 2024-05-28T14:09:41.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b28200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5b2015ce50 con 0x7f5b20156e50 2024-05-28T14:09:41.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b25800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5b20156a70 0x7f5b2006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:41.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b25800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5b20156a70 0x7f5b2006dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38908/0 (socket says 172.21.15.42:38908) 2024-05-28T14:09:41.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b25800640 1 -- 172.21.15.42:0/1751204976 learned_addr learned my addr 172.21.15.42:0/1751204976 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:41.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b24e00640 1 --2- 172.21.15.42:0/1751204976 >> v2:172.21.15.63:3300/0 conn(0x7f5b20156e50 0x7f5b200af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:41.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b25800640 1 -- 172.21.15.42:0/1751204976 >> v2:172.21.15.63:3300/0 conn(0x7f5b20156e50 msgr2=0x7f5b200af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:41.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b25800640 1 --2- 172.21.15.42:0/1751204976 >> v2:172.21.15.63:3300/0 conn(0x7f5b20156e50 0x7f5b200af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b25800640 1 -- 172.21.15.42:0/1751204976 >> v1:172.21.15.63:6789/0 conn(0x7f5b200af880 legacy=0x7f5b200d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:41.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b25800640 1 -- 172.21.15.42:0/1751204976 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2015d130 con 0x7f5b20156a70 2024-05-28T14:09:41.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.417+0000 7f5b24e00640 1 --2- 172.21.15.42:0/1751204976 >> v2:172.21.15.63:3300/0 conn(0x7f5b20156e50 0x7f5b200af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:41.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.421+0000 7f5b25800640 1 --2- 172.21.15.42:0/1751204976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b20156a70 0x7f5b2006dd60 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5b200701f0 tx=0x7f5b1401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcba849d3fdea33 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:41.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.421+0000 7f5b1fe00640 1 -- 172.21.15.42:0/1751204976 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b140a6040 con 0x7f5b20156a70 2024-05-28T14:09:41.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.421+0000 7f5b1fe00640 1 -- 172.21.15.42:0/1751204976 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b140af070 con 0x7f5b20156a70 2024-05-28T14:09:41.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.421+0000 7f5b1fe00640 1 -- 172.21.15.42:0/1751204976 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b140aa860 con 0x7f5b20156a70 2024-05-28T14:09:41.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.421+0000 7f5b28200640 1 -- 172.21.15.42:0/1751204976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b20156a70 msgr2=0x7f5b2006dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:41.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.421+0000 7f5b28200640 1 --2- 172.21.15.42:0/1751204976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b20156a70 0x7f5b2006dd60 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5b200701f0 tx=0x7f5b1401c4e0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.421+0000 7f5b28200640 1 -- 172.21.15.42:0/1751204976 shutdown_connections 2024-05-28T14:09:41.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.421+0000 7f5b28200640 1 --2- 172.21.15.42:0/1751204976 >> v2:172.21.15.63:3300/0 conn(0x7f5b20156e50 0x7f5b200af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.421+0000 7f5b28200640 1 --2- 172.21.15.42:0/1751204976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b20156a70 0x7f5b2006dd60 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.421+0000 7f5b28200640 1 -- 172.21.15.42:0/1751204976 >> 172.21.15.42:0/1751204976 conn(0x7f5b2006bdf0 msgr2=0x7f5b2006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:41.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.421+0000 7f5b28200640 1 -- 172.21.15.42:0/1751204976 shutdown_connections 2024-05-28T14:09:41.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.421+0000 7f5b28200640 1 -- 172.21.15.42:0/1751204976 wait complete. 2024-05-28T14:09:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b28200640 1 Processor -- start 2024-05-28T14:09:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b28200640 1 -- start start 2024-05-28T14:09:41.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b28200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b200af880 0x7f5b2014c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:41.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b28200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b20156a70 0x7f5b200cf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:41.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b28200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b20156e50 0x7f5b2006ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:41.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b28200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5b20160b40 con 0x7f5b200af880 2024-05-28T14:09:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b28200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5b200d1010 con 0x7f5b20156a70 2024-05-28T14:09:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b28200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5b20160cc0 con 0x7f5b20156e50 2024-05-28T14:09:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b25800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b200af880 0x7f5b2014c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b25800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b200af880 0x7f5b2014c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38916/0 (socket says 172.21.15.42:38916) 2024-05-28T14:09:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b25800640 1 -- 172.21.15.42:0/460539776 learned_addr learned my addr 172.21.15.42:0/460539776 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b24e00640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b20156a70 0x7f5b200cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b26200640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b20156e50 0x7f5b2006ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b25800640 1 -- 172.21.15.42:0/460539776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b20156e50 msgr2=0x7f5b2006ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b25800640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b20156e50 0x7f5b2006ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b25800640 1 -- 172.21.15.42:0/460539776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b20156a70 msgr2=0x7f5b200cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b25800640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b20156a70 0x7f5b200cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b25800640 1 -- 172.21.15.42:0/460539776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b200b1620 con 0x7f5b200af880 2024-05-28T14:09:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b24e00640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b20156a70 0x7f5b200cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b25800640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b200af880 0x7f5b2014c330 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5b1401a110 tx=0x7f5b1401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b1d600640 1 -- 172.21.15.42:0/460539776 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b140bd5f0 con 0x7f5b200af880 2024-05-28T14:09:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b1d600640 1 -- 172.21.15.42:0/460539776 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b140bd790 con 0x7f5b200af880 2024-05-28T14:09:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b1d600640 1 -- 172.21.15.42:0/460539776 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b140bdb40 con 0x7f5b200af880 2024-05-28T14:09:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.425+0000 7f5b28200640 1 -- 172.21.15.42:0/460539776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2014b570 con 0x7f5b200af880 2024-05-28T14:09:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.429+0000 7f5b28200640 1 -- 172.21.15.42:0/460539776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2014bb30 con 0x7f5b200af880 2024-05-28T14:09:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.429+0000 7f5b28200640 1 -- 172.21.15.42:0/460539776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b20068a90 con 0x7f5b200af880 2024-05-28T14:09:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.429+0000 7f5b1d600640 1 -- 172.21.15.42:0/460539776 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f5b140b33f0 con 0x7f5b200af880 2024-05-28T14:09:41.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.429+0000 7f5b1d600640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b000694f0 0x7f5b0006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:41.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.429+0000 7f5b1d600640 1 -- 172.21.15.42:0/460539776 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7f5b141203b0 con 0x7f5b200af880 2024-05-28T14:09:41.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.429+0000 7f5b24e00640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b000694f0 0x7f5b0006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:41.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.433+0000 7f5b24e00640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b000694f0 0x7f5b0006b9b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5b2015faf0 tx=0x7f5b0c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:41.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.433+0000 7f5b1d600640 1 -- 172.21.15.42:0/460539776 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b140c3060 con 0x7f5b200af880 2024-05-28T14:09:41.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.633+0000 7f5b28200640 1 -- 172.21.15.42:0/460539776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b200b5160 con 0x7f5b200af880 2024-05-28T14:09:41.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.633+0000 7f5b1d600640 1 -- 172.21.15.42:0/460539776 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5182 (secure 0 0 0) 0x7f5b140160d0 con 0x7f5b200af880 2024-05-28T14:09:41.617 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:09:41.617 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7657,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/402076081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":402076081},{"type":"v1","addr":"172.21.15.63:6835","nonce":402076081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7998,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3779622485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3779622485},{"type":"v1","addr":"172.21.15.42:6835","nonce":3779622485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2961573665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2961573665},{"type":"v1","addr":"172.21.15.42:6837","nonce":2961573665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:09:31.146587+0000","modified":"2024-05-28T14:09:34.152047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":56621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56621":{"gid":56621,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/615628755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":615628755},{"type":"v1","addr":"172.21.15.63:6834","nonce":615628755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":56621,"qdb_cluster":[56621]},"id":6}]} 2024-05-28T14:09:41.617 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 54 2024-05-28T14:09:41.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.637+0000 7f5b28200640 1 -- 172.21.15.42:0/460539776 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b000694f0 msgr2=0x7f5b0006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:41.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.637+0000 7f5b28200640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b000694f0 0x7f5b0006b9b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5b2015faf0 tx=0x7f5b0c008040 comp rx=0 tx=0).stop 2024-05-28T14:09:41.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.637+0000 7f5b28200640 1 -- 172.21.15.42:0/460539776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b200af880 msgr2=0x7f5b2014c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:41.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.637+0000 7f5b28200640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b200af880 0x7f5b2014c330 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5b1401a110 tx=0x7f5b1401ad20 comp rx=0 tx=0).stop 2024-05-28T14:09:41.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.637+0000 7f5b28200640 1 -- 172.21.15.42:0/460539776 shutdown_connections 2024-05-28T14:09:41.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.637+0000 7f5b28200640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b000694f0 0x7f5b0006b9b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.637+0000 7f5b28200640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b20156e50 0x7f5b2006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.637+0000 7f5b28200640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b20156a70 0x7f5b200cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.637+0000 7f5b28200640 1 --2- 172.21.15.42:0/460539776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b200af880 0x7f5b2014c330 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:41.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.637+0000 7f5b28200640 1 -- 172.21.15.42:0/460539776 >> 172.21.15.42:0/460539776 conn(0x7f5b2006bdf0 msgr2=0x7f5b2015ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:41.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.637+0000 7f5b28200640 1 -- 172.21.15.42:0/460539776 shutdown_connections 2024-05-28T14:09:41.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:41.637+0000 7f5b28200640 1 -- 172.21.15.42:0/460539776 wait complete. 2024-05-28T14:09:42.040 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-28T14:09:31.146587+0000', 'modified': '2024-05-28T14:09:34.152047+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56621}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56621': {'gid': 56621, 'name': 'b', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6834/615628755', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6832', 'nonce': 615628755}, {'type': 'v1', 'addr': '172.21.15.63:6834', 'nonce': 615628755}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 56621, 'qdb_cluster': [56621]} 2024-05-28T14:09:42.040 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-28T14:09:42.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-28T14:09:42.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.201+0000 7ff1cd800640 1 Processor -- start 2024-05-28T14:09:42.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.201+0000 7ff1cd800640 1 -- start start 2024-05-28T14:09:42.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.201+0000 7ff1cd800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1c815bf30 0x7ff1c815c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.201+0000 7ff1cd800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c80cfc10 con 0x7ff1c80cf690 2024-05-28T14:09:42.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.201+0000 7ff1cd800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c80cfef0 con 0x7ff1c80cf230 2024-05-28T14:09:42.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.201+0000 7ff1cd800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c80721d0 con 0x7ff1c815bf30 2024-05-28T14:09:42.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.201+0000 7ff1c7400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff1c80cf690 0x7ff1c806ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:45114/0 (socket says 172.21.15.42:45114) 2024-05-28T14:09:42.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.201+0000 7ff1c7400640 1 -- 172.21.15.42:0/3610590305 learned_addr learned my addr 172.21.15.42:0/3610590305 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:42.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.201+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/3610590305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1c815bf30 0x7ff1c815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4286886180 0 0) 0x7ff1c80cfc10 con 0x7ff1c80cf690 2024-05-28T14:09:42.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1ac005740 con 0x7ff1c80cf690 2024-05-28T14:09:42.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4113839258 0 0) 0x7ff1c80cfef0 con 0x7ff1c80cf230 2024-05-28T14:09:42.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1c80cfc10 con 0x7ff1c80cf230 2024-05-28T14:09:42.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3804623006 0 0) 0x7ff1ac005740 con 0x7ff1c80cf690 2024-05-28T14:09:42.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1c815bf30 msgr2=0x7ff1c815c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:42.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 --2- 172.21.15.42:0/3610590305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1c815bf30 0x7ff1c815c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 >> v1:172.21.15.63:6789/0 conn(0x7ff1c80cf230 legacy=0x7ff1c8072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c80d01d0 con 0x7ff1c80cf690 2024-05-28T14:09:42.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7ff1c0001a00 con 0x7ff1c80cf690 2024-05-28T14:09:42.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/3610590305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1c815bf30 0x7ff1c815c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:09:42.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 >> v1:172.21.15.42:6789/0 conn(0x7ff1c80cf690 legacy=0x7ff1c806ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 --2- 172.21.15.42:0/3610590305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1ac005d90 0x7ff1ac006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 --2- 172.21.15.42:0/3610590305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1ac006a20 0x7ff1ac006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 --2- 172.21.15.42:0/3610590305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac0073e0 0x7ff1ac00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c80721d0 con 0x7ff1ac0073e0 2024-05-28T14:09:42.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac00e1d0 con 0x7ff1ac006a20 2024-05-28T14:09:42.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ac00e3d0 con 0x7ff1ac005d90 2024-05-28T14:09:42.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c7400640 1 --2- 172.21.15.42:0/3610590305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac0073e0 0x7ff1ac00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/3610590305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1ac005d90 0x7ff1ac006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1be000640 1 --2- 172.21.15.42:0/3610590305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1ac006a20 0x7ff1ac006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c7400640 1 -- 172.21.15.42:0/3610590305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1ac005d90 msgr2=0x7ff1ac006190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:42.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c7400640 1 --2- 172.21.15.42:0/3610590305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1ac005d90 0x7ff1ac006190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c7400640 1 -- 172.21.15.42:0/3610590305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1ac006a20 msgr2=0x7ff1ac006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:42.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c7400640 1 --2- 172.21.15.42:0/3610590305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1ac006a20 0x7ff1ac006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1c7400640 1 -- 172.21.15.42:0/3610590305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac005740 con 0x7ff1ac0073e0 2024-05-28T14:09:42.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.205+0000 7ff1be000640 1 --2- 172.21.15.42:0/3610590305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1ac006a20 0x7ff1ac006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1c7400640 1 --2- 172.21.15.42:0/3610590305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac0073e0 0x7ff1ac00dbe0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000a800 tx=0x7ff1c001fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17da43c895c78b93 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c000a450 con 0x7ff1ac0073e0 2024-05-28T14:09:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c00aa670 con 0x7ff1ac0073e0 2024-05-28T14:09:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1c6000640 1 -- 172.21.15.42:0/3610590305 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c00aaae0 con 0x7ff1ac0073e0 2024-05-28T14:09:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1cd800640 1 -- 172.21.15.42:0/3610590305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac0073e0 msgr2=0x7ff1ac00dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1cd800640 1 --2- 172.21.15.42:0/3610590305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac0073e0 0x7ff1ac00dbe0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000a800 tx=0x7ff1c001fa40 comp rx=0 tx=0).stop 2024-05-28T14:09:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1c7400640 1 -- 172.21.15.42:0/3610590305 reap_dead start 2024-05-28T14:09:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1cd800640 1 -- 172.21.15.42:0/3610590305 shutdown_connections 2024-05-28T14:09:42.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1cd800640 1 -- 172.21.15.42:0/3610590305 >> 172.21.15.42:0/3610590305 conn(0x7ff1c806b9f0 msgr2=0x7ff1c806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:42.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1cd800640 1 -- 172.21.15.42:0/3610590305 shutdown_connections 2024-05-28T14:09:42.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1cd800640 1 -- 172.21.15.42:0/3610590305 wait complete. 2024-05-28T14:09:42.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1cd800640 1 Processor -- start 2024-05-28T14:09:42.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1cd800640 1 -- start start 2024-05-28T14:09:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1cd800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1ac0073e0 0x7ff1c813ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1cd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1c815a6e0 0x7ff1c815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1cd800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1c8148e90 0x7ff1c8149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1cd800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c806f360 con 0x7ff1c815a6e0 2024-05-28T14:09:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1cd800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8072d70 con 0x7ff1ac0073e0 2024-05-28T14:09:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1cd800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c806f4e0 con 0x7ff1c8148e90 2024-05-28T14:09:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1be000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1c815a6e0 0x7ff1c815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1be000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1c815a6e0 0x7ff1c815ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38928/0 (socket says 172.21.15.42:38928) 2024-05-28T14:09:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1be000640 1 -- 172.21.15.42:0/933191252 learned_addr learned my addr 172.21.15.42:0/933191252 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/933191252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1ac0073e0 0x7ff1c813ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.209+0000 7ff1c7400640 1 --2- 172.21.15.42:0/933191252 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1c8148e90 0x7ff1c8149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.213+0000 7ff1be000640 1 -- 172.21.15.42:0/933191252 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1c8148e90 msgr2=0x7ff1c8149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.213+0000 7ff1be000640 1 --2- 172.21.15.42:0/933191252 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1c8148e90 0x7ff1c8149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.213+0000 7ff1be000640 1 -- 172.21.15.42:0/933191252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1ac0073e0 msgr2=0x7ff1c813ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.213+0000 7ff1be000640 1 --2- 172.21.15.42:0/933191252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1ac0073e0 0x7ff1c813ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.213+0000 7ff1be000640 1 -- 172.21.15.42:0/933191252 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c8149850 con 0x7ff1c815a6e0 2024-05-28T14:09:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.213+0000 7ff1be000640 1 --2- 172.21.15.42:0/933191252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1c815a6e0 0x7ff1c815ab60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000e7b0 tx=0x7ff1b000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.213+0000 7ff1bf400640 1 -- 172.21.15.42:0/933191252 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b000ee60 con 0x7ff1c815a6e0 2024-05-28T14:09:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.213+0000 7ff1bf400640 1 -- 172.21.15.42:0/933191252 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b00052e0 con 0x7ff1c815a6e0 2024-05-28T14:09:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.213+0000 7ff1bf400640 1 -- 172.21.15.42:0/933191252 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0005490 con 0x7ff1c815a6e0 2024-05-28T14:09:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.213+0000 7ff1cd800640 1 -- 172.21.15.42:0/933191252 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c8147040 con 0x7ff1c815a6e0 2024-05-28T14:09:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.213+0000 7ff1cd800640 1 -- 172.21.15.42:0/933191252 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c8149dc0 con 0x7ff1c815a6e0 2024-05-28T14:09:42.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.213+0000 7ff1bf400640 1 -- 172.21.15.42:0/933191252 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7ff1b001a020 con 0x7ff1c815a6e0 2024-05-28T14:09:42.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.213+0000 7ff1bf400640 1 --2- 172.21.15.42:0/933191252 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1a40696c0 0x7ff1a406bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1bf400640 1 -- 172.21.15.42:0/933191252 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7ff1b0084fc0 con 0x7ff1c815a6e0 2024-05-28T14:09:42.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/933191252 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1a40696c0 0x7ff1a406bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 Processor -- start 2024-05-28T14:09:42.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 -- start start 2024-05-28T14:09:42.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/933191252 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1a40696c0 0x7ff1a406bb80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff1c806c990 tx=0x7ff1b8009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:42.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89120 0x564d54a89500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a89a40 0x564d54a3c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a3c650 0x564d54a3ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564d5488d420 con 0x564d54a89120 2024-05-28T14:09:42.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564d54827bf0 con 0x564d54a3c650 2024-05-28T14:09:42.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564d548893e0 con 0x564d54a89a40 2024-05-28T14:09:42.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89120 0x564d54a89500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89120 0x564d54a89500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38932/0 (socket says 172.21.15.42:38932) 2024-05-28T14:09:42.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1c6a00640 1 -- 172.21.15.42:0/3825810629 learned_addr learned my addr 172.21.15.42:0/3825810629 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:42.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1be000640 1 --2- 172.21.15.42:0/3825810629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a89a40 0x564d54a3c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1c7400640 1 --2- 172.21.15.42:0/3825810629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a3c650 0x564d54a3ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1c6a00640 1 -- 172.21.15.42:0/3825810629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a89a40 msgr2=0x564d54a3c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:42.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/3825810629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a89a40 0x564d54a3c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1c6a00640 1 -- 172.21.15.42:0/3825810629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a3c650 msgr2=0x564d54a3ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:42.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/3825810629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a3c650 0x564d54a3ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1c6a00640 1 -- 172.21.15.42:0/3825810629 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d54935450 con 0x564d54a89120 2024-05-28T14:09:42.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1be000640 1 --2- 172.21.15.42:0/3825810629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a89a40 0x564d54a3c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:42.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1c7400640 1 --2- 172.21.15.42:0/3825810629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a3c650 0x564d54a3ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:42.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/3825810629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89120 0x564d54a89500 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009e30 tx=0x7ff1b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a2455e1e6af751 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:42.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1b7400640 1 -- 172.21.15.42:0/3825810629 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b801b070 con 0x564d54a89120 2024-05-28T14:09:42.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1b7400640 1 -- 172.21.15.42:0/3825810629 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b8019070 con 0x564d54a89120 2024-05-28T14:09:42.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1b7400640 1 -- 172.21.15.42:0/3825810629 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80044c0 con 0x564d54a89120 2024-05-28T14:09:42.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 -- 172.21.15.42:0/3825810629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89120 msgr2=0x564d54a89500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/3825810629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89120 0x564d54a89500 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009e30 tx=0x7ff1b8017000 comp rx=0 tx=0).stop 2024-05-28T14:09:42.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 -- 172.21.15.42:0/3825810629 shutdown_connections 2024-05-28T14:09:42.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/3825810629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a3c650 0x564d54a3ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/3825810629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a89a40 0x564d54a3c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/3825810629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89120 0x564d54a89500 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 -- 172.21.15.42:0/3825810629 >> 172.21.15.42:0/3825810629 conn(0x564d54a69ec0 msgr2=0x564d54a6c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:42.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 -- 172.21.15.42:0/3825810629 shutdown_connections 2024-05-28T14:09:42.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.217+0000 7ff1d0c19000 1 -- 172.21.15.42:0/3825810629 wait complete. 2024-05-28T14:09:42.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 1 -- start start 2024-05-28T14:09:42.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 1 Processor -- start 2024-05-28T14:09:42.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a3c650 0x564d54a41b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 0x564d54a42080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89a40 0x564d54a425c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564d5482dec0 con 0x564d54a89a40 2024-05-28T14:09:42.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564d5490e120 con 0x564d54a89120 2024-05-28T14:09:42.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564d54898350 con 0x564d54a3c650 2024-05-28T14:09:42.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 10 client.0 _ll_register_callbacks cb 0x564d5486a4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T14:09:42.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1c7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89a40 0x564d54a425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1c7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89a40 0x564d54a425c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38944/0 (socket says 172.21.15.42:38944) 2024-05-28T14:09:42.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1c7400640 1 -- 172.21.15.42:0/1307622085 learned_addr learned my addr 172.21.15.42:0/1307622085 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:42.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/1307622085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a3c650 0x564d54a41b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1be000640 1 --2- 172.21.15.42:0/1307622085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 0x564d54a42080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1c7400640 1 -- 172.21.15.42:0/1307622085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a3c650 msgr2=0x564d54a41b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:42.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1c7400640 1 --2- 172.21.15.42:0/1307622085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a3c650 0x564d54a41b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1c7400640 1 -- 172.21.15.42:0/1307622085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 msgr2=0x564d54a42080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:42.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1c7400640 1 --2- 172.21.15.42:0/1307622085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 0x564d54a42080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1c7400640 1 -- 172.21.15.42:0/1307622085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d548d2f80 con 0x564d54a89a40 2024-05-28T14:09:42.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1c7400640 1 --2- 172.21.15.42:0/1307622085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89a40 0x564d54a425c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000a800 tx=0x7ff1c001c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e909b4d9547989e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:42.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1abe00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T14:09:42.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1abe00640 1 -- 172.21.15.42:0/1307622085 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c00c3060 con 0x564d54a89a40 2024-05-28T14:09:42.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1307622085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b8019070 con 0x564d54a89a40 2024-05-28T14:09:42.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1abe00640 1 -- 172.21.15.42:0/1307622085 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7ff1c00ab450 con 0x564d54a89a40 2024-05-28T14:09:42.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1abe00640 1 -- 172.21.15.42:0/1307622085 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ff1c00c9050 con 0x564d54a89a40 2024-05-28T14:09:42.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 10 client.8157 fetch_fsmap learned FSMap version 54 2024-05-28T14:09:42.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1307622085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x564d54866f00 con 0x564d54a89a40 2024-05-28T14:09:42.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1abe00640 1 -- 172.21.15.42:0/1307622085 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b8019070 con 0x564d54a89a40 2024-05-28T14:09:42.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1abe00640 1 -- 172.21.15.42:0/1307622085 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 54) ==== 3494+0+0 (secure 0 0 0) 0x7ff1c00cc020 con 0x564d54a89a40 2024-05-28T14:09:42.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 10 client.8157 fetch_fsmap finished waiting for FSMap version 54 2024-05-28T14:09:42.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 10 client.8157 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-28T14:09:42.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 20 client.8157 populate_metadata read hostname 'smithi042' 2024-05-28T14:09:42.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/1307622085 >> [v2:172.21.15.63:6832/615628755,v1:172.21.15.63:6834/615628755] conn(0x564d54a983c0 0x564d54a9a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 4 client.8157 mds_command: new command op to 56621 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-28T14:09:42.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1307622085 --> [v2:172.21.15.63:6832/615628755,v1:172.21.15.63:6834/615628755] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d54826bc0 con 0x564d54a983c0 2024-05-28T14:09:42.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.221+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/1307622085 >> [v2:172.21.15.63:6832/615628755,v1:172.21.15.63:6834/615628755] conn(0x564d54a983c0 0x564d54a9a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/1307622085 >> [v2:172.21.15.63:6832/615628755,v1:172.21.15.63:6834/615628755] conn(0x564d54a983c0 0x564d54a9a7b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce8efa775f82f169 server_cookie=707247eafbd90ed in_seq=0 out_seq=0 2024-05-28T14:09:42.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1abe00640 10 client.8157 ms_handle_connect on v2:172.21.15.63:6832/615628755 2024-05-28T14:09:42.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1abe00640 1 -- 172.21.15.42:0/1307622085 <== mds.0 v2:172.21.15.63:6832/615628755 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d54826bc0 con 0x564d54a983c0 2024-05-28T14:09:42.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1abe00640 10 client.8157 handle_command_reply: tid=0 2024-05-28T14:09:42.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1d0c19000 1 client.8157 shutdown 2024-05-28T14:09:42.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1307622085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89a40 msgr2=0x564d54a425c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/1307622085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89a40 0x564d54a425c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000a800 tx=0x7ff1c001c7e0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1307622085 shutdown_connections 2024-05-28T14:09:42.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/1307622085 >> [v2:172.21.15.63:6832/615628755,v1:172.21.15.63:6834/615628755] conn(0x564d54a983c0 0x564d54a9a7b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/1307622085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89a40 0x564d54a425c0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/1307622085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 0x564d54a42080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/1307622085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a3c650 0x564d54a41b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1307622085 >> 172.21.15.42:0/1307622085 conn(0x564d54a69ec0 msgr2=0x564d54a6c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:42.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1abe00640 0 client.8157 ms_handle_reset on v2:172.21.15.63:6832/615628755 2024-05-28T14:09:42.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1307622085 shutdown_connections 2024-05-28T14:09:42.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1307622085 wait complete. 2024-05-28T14:09:42.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.225+0000 7ff1d0c19000 20 client.8157 trim_cache size 0 max 16384 2024-05-28T14:09:42.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.241+0000 7ff1d0c19000 1 Processor -- start 2024-05-28T14:09:42.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.241+0000 7ff1d0c19000 1 -- start start 2024-05-28T14:09:42.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a3c650 0x564d54a3b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 0x564d54a3bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89a40 0x564d54a9a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8200490 con 0x564d54a89a40 2024-05-28T14:09:42.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c80b1340 con 0x564d54a89120 2024-05-28T14:09:42.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c80b1990 con 0x564d54a3c650 2024-05-28T14:09:42.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1c7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89a40 0x564d54a9a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1c7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89a40 0x564d54a9a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38948/0 (socket says 172.21.15.42:38948) 2024-05-28T14:09:42.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1c7400640 1 -- 172.21.15.42:0/2731220148 learned_addr learned my addr 172.21.15.42:0/2731220148 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:42.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/2731220148 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a3c650 0x564d54a3b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1be000640 1 --2- 172.21.15.42:0/2731220148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 0x564d54a3bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1c7400640 1 -- 172.21.15.42:0/2731220148 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a3c650 msgr2=0x564d54a3b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:42.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1c7400640 1 --2- 172.21.15.42:0/2731220148 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a3c650 0x564d54a3b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1c7400640 1 -- 172.21.15.42:0/2731220148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 msgr2=0x564d54a3bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:42.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1c7400640 1 --2- 172.21.15.42:0/2731220148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 0x564d54a3bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1c7400640 1 -- 172.21.15.42:0/2731220148 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d54a89870 con 0x564d54a89a40 2024-05-28T14:09:42.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1be000640 1 --2- 172.21.15.42:0/2731220148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 0x564d54a3bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:42.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1c7400640 1 --2- 172.21.15.42:0/2731220148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89a40 0x564d54a9a7b0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ff1c001fa80 tx=0x7ff1c00bc000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d54cf8980bbe4d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:42.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1b6a00640 1 -- 172.21.15.42:0/2731220148 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c00d2070 con 0x564d54a89a40 2024-05-28T14:09:42.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1b6a00640 1 -- 172.21.15.42:0/2731220148 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c00ab450 con 0x564d54a89a40 2024-05-28T14:09:42.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1b6a00640 1 -- 172.21.15.42:0/2731220148 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c00abdd0 con 0x564d54a89a40 2024-05-28T14:09:42.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 -- 172.21.15.42:0/2731220148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89a40 msgr2=0x564d54a9a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/2731220148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89a40 0x564d54a9a7b0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ff1c001fa80 tx=0x7ff1c00bc000 comp rx=0 tx=0).stop 2024-05-28T14:09:42.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 -- 172.21.15.42:0/2731220148 shutdown_connections 2024-05-28T14:09:42.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/2731220148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a89a40 0x564d54a9a7b0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/2731220148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 0x564d54a3bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/2731220148 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a3c650 0x564d54a3b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 -- 172.21.15.42:0/2731220148 >> 172.21.15.42:0/2731220148 conn(0x564d54a69ec0 msgr2=0x564d54a3b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:42.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 -- 172.21.15.42:0/2731220148 shutdown_connections 2024-05-28T14:09:42.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 -- 172.21.15.42:0/2731220148 wait complete. 2024-05-28T14:09:42.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 -- start start 2024-05-28T14:09:42.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.245+0000 7ff1d0c19000 1 Processor -- start 2024-05-28T14:09:42.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a3c650 0x564d54a9d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 0x564d54ab4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a89a40 0x564d54ab4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564d5488c730 con 0x564d54a3c650 2024-05-28T14:09:42.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564d548599e0 con 0x564d54a89120 2024-05-28T14:09:42.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564d54891a50 con 0x564d54a89a40 2024-05-28T14:09:42.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 10 client.0 _ll_register_callbacks cb 0x7ff1b8009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T14:09:42.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a3c650 0x564d54a9d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a3c650 0x564d54a9d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38956/0 (socket says 172.21.15.42:38956) 2024-05-28T14:09:42.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1c6a00640 1 -- 172.21.15.42:0/1772578497 learned_addr learned my addr 172.21.15.42:0/1772578497 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:42.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1be000640 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 0x564d54ab4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1c7400640 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a89a40 0x564d54ab4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1c6a00640 1 -- 172.21.15.42:0/1772578497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a89a40 msgr2=0x564d54ab4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:42.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a89a40 0x564d54ab4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1c6a00640 1 -- 172.21.15.42:0/1772578497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 msgr2=0x564d54ab4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:42.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 0x564d54ab4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1c6a00640 1 -- 172.21.15.42:0/1772578497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d54813e80 con 0x564d54a3c650 2024-05-28T14:09:42.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1be000640 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 0x564d54ab4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:42.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1c7400640 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a89a40 0x564d54ab4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:09:42.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1c6a00640 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a3c650 0x564d54a9d1f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8015010 tx=0x7ff1b80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b270cd134d8f481 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:42.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1abe00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T14:09:42.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1abe00640 1 -- 172.21.15.42:0/1772578497 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8037280 con 0x564d54a3c650 2024-05-28T14:09:42.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1abe00640 1 -- 172.21.15.42:0/1772578497 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7ff1b8037420 con 0x564d54a3c650 2024-05-28T14:09:42.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1772578497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1c00ab450 con 0x564d54a3c650 2024-05-28T14:09:42.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1abe00640 1 -- 172.21.15.42:0/1772578497 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ff1b8045050 con 0x564d54a3c650 2024-05-28T14:09:42.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 10 client.8163 fetch_fsmap learned FSMap version 54 2024-05-28T14:09:42.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1772578497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x564d54855b30 con 0x564d54a3c650 2024-05-28T14:09:42.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1abe00640 1 -- 172.21.15.42:0/1772578497 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c00ab450 con 0x564d54a3c650 2024-05-28T14:09:42.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1abe00640 1 -- 172.21.15.42:0/1772578497 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 54) ==== 3494+0+0 (secure 0 0 0) 0x7ff1b8048020 con 0x564d54a3c650 2024-05-28T14:09:42.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 10 client.8163 fetch_fsmap finished waiting for FSMap version 54 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 10 client.8163 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 20 client.8163 populate_metadata read hostname 'smithi042' 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.63:6832/615628755,v1:172.21.15.63:6834/615628755] conn(0x564d54ab4dc0 0x564d54af7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 4 client.8163 mds_command: new command op to 56621 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1772578497 --> [v2:172.21.15.63:6832/615628755,v1:172.21.15.63:6834/615628755] -- command(tid 0: {"prefix": "status"}) -- 0x564d5489ae40 con 0x564d54ab4dc0 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1be000640 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.63:6832/615628755,v1:172.21.15.63:6834/615628755] conn(0x564d54ab4dc0 0x564d54af7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1be000640 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.63:6832/615628755,v1:172.21.15.63:6834/615628755] conn(0x564d54ab4dc0 0x564d54af7fb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd5e7f2cb867eb82 server_cookie=fdabb4b2311d6970 in_seq=0 out_seq=0 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1abe00640 10 client.8163 ms_handle_connect on v2:172.21.15.63:6832/615628755 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1abe00640 1 -- 172.21.15.42:0/1772578497 <== mds.0 v2:172.21.15.63:6832/615628755 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x564d5489ae40 con 0x564d54ab4dc0 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1abe00640 10 client.8163 handle_command_reply: tid=0 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 client.8163 shutdown 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1772578497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a3c650 msgr2=0x564d54a9d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a3c650 0x564d54a9d1f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8015010 tx=0x7ff1b80046d0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1772578497 shutdown_connections 2024-05-28T14:09:42.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d54a89a40 0x564d54ab4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d54a89120 0x564d54ab4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.63:6832/615628755,v1:172.21.15.63:6834/615628755] conn(0x564d54ab4dc0 0x564d54af7fb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 --2- 172.21.15.42:0/1772578497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d54a3c650 0x564d54a9d1f0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1abe00640 0 client.8163 ms_handle_reset on v2:172.21.15.63:6832/615628755 2024-05-28T14:09:42.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1772578497 >> 172.21.15.42:0/1772578497 conn(0x564d54a69ec0 msgr2=0x564d54a3b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:42.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1772578497 shutdown_connections 2024-05-28T14:09:42.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 1 -- 172.21.15.42:0/1772578497 wait complete. 2024-05-28T14:09:42.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.249+0000 7ff1d0c19000 20 client.8163 trim_cache size 0 max 16384 2024-05-28T14:09:42.238 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:09:42.238 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:09:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-05-28T14:09:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 56621, 2024-05-28T14:09:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-28T14:09:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-28T14:09:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-28T14:09:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 10.501664591000001, 2024-05-28T14:09:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 54, 2024-05-28T14:09:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 90, 2024-05-28T14:09:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 90, 2024-05-28T14:09:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 16.342522748 2024-05-28T14:09:42.239 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:09:42.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.253+0000 7ff1cd800640 1 -- 172.21.15.42:0/933191252 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1a40696c0 msgr2=0x7ff1a406bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.253+0000 7ff1cd800640 1 --2- 172.21.15.42:0/933191252 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1a40696c0 0x7ff1a406bb80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff1c806c990 tx=0x7ff1b8009190 comp rx=0 tx=0).stop 2024-05-28T14:09:42.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.253+0000 7ff1cd800640 1 -- 172.21.15.42:0/933191252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1c815a6e0 msgr2=0x7ff1c815ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.253+0000 7ff1cd800640 1 --2- 172.21.15.42:0/933191252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1c815a6e0 0x7ff1c815ab60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000e7b0 tx=0x7ff1b000ec70 comp rx=0 tx=0).stop 2024-05-28T14:09:42.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.253+0000 7ff1cd800640 1 -- 172.21.15.42:0/933191252 shutdown_connections 2024-05-28T14:09:42.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.253+0000 7ff1cd800640 1 --2- 172.21.15.42:0/933191252 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff1a40696c0 0x7ff1a406bb80 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.253+0000 7ff1cd800640 1 --2- 172.21.15.42:0/933191252 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1c8148e90 0x7ff1c8149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.253+0000 7ff1cd800640 1 --2- 172.21.15.42:0/933191252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1c815a6e0 0x7ff1c815ab60 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.253+0000 7ff1cd800640 1 --2- 172.21.15.42:0/933191252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1ac0073e0 0x7ff1c813ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.253+0000 7ff1cd800640 1 -- 172.21.15.42:0/933191252 >> 172.21.15.42:0/933191252 conn(0x7ff1c806b9f0 msgr2=0x7ff1c806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:42.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.253+0000 7ff1cd800640 1 -- 172.21.15.42:0/933191252 shutdown_connections 2024-05-28T14:09:42.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.253+0000 7ff1cd800640 1 -- 172.21.15.42:0/933191252 wait complete. 2024-05-28T14:09:42.254 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-28T14:09:42.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.409+0000 7f2007200640 1 Processor -- start 2024-05-28T14:09:42.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.413+0000 7f2007200640 1 -- start start 2024-05-28T14:09:42.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.413+0000 7f2007200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f20000d0900 0x7f2000162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.413+0000 7f2007200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20000d0de0 con 0x7f20000ce860 2024-05-28T14:09:42.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.413+0000 7f2007200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20000d10c0 con 0x7f2000162ab0 2024-05-28T14:09:42.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.413+0000 7f2007200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f20000d19f0 con 0x7f20000d0900 2024-05-28T14:09:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.413+0000 7f1fffe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f20000ce860 0x7f200006d7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:45118/0 (socket says 172.21.15.42:45118) 2024-05-28T14:09:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.413+0000 7f1fffe00640 1 -- 172.21.15.42:0/2555167773 learned_addr learned my addr 172.21.15.42:0/2555167773 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.413+0000 7f1ff7400640 1 --2- 172.21.15.42:0/2555167773 >> v2:172.21.15.63:3300/0 conn(0x7f20000d0900 0x7f2000162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.413+0000 7f1fff400640 1 -- 172.21.15.42:0/2555167773 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2121025122 0 0) 0x7f20000d0de0 con 0x7f20000ce860 2024-05-28T14:09:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.413+0000 7f1fff400640 1 -- 172.21.15.42:0/2555167773 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fe4003530 con 0x7f20000ce860 2024-05-28T14:09:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.413+0000 7f1ff7400640 1 -- 172.21.15.42:0/2555167773 >> v1:172.21.15.63:6789/0 conn(0x7f2000162ab0 legacy=0x7f200015e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.413+0000 7f1ff7400640 1 -- 172.21.15.42:0/2555167773 >> v1:172.21.15.42:6789/0 conn(0x7f20000ce860 legacy=0x7f200006d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.413+0000 7f1ff7400640 1 -- 172.21.15.42:0/2555167773 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20000d13a0 con 0x7f20000d0900 2024-05-28T14:09:42.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.417+0000 7f1ff7400640 1 --2- 172.21.15.42:0/2555167773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20000d0900 0x7f2000162570 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009d90 tx=0x7f1fe801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c965cfd150c9647c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:42.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.417+0000 7f1fff400640 1 -- 172.21.15.42:0/2555167773 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80b2070 con 0x7f20000d0900 2024-05-28T14:09:42.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.417+0000 7f1fff400640 1 -- 172.21.15.42:0/2555167773 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe80a57b0 con 0x7f20000d0900 2024-05-28T14:09:42.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.417+0000 7f1fff400640 1 -- 172.21.15.42:0/2555167773 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80a5b80 con 0x7f20000d0900 2024-05-28T14:09:42.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.417+0000 7f2007200640 1 -- 172.21.15.42:0/2555167773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20000d0900 msgr2=0x7f2000162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.417+0000 7f2007200640 1 --2- 172.21.15.42:0/2555167773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20000d0900 0x7f2000162570 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009d90 tx=0x7f1fe801c8e0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.417+0000 7f2007200640 1 -- 172.21.15.42:0/2555167773 shutdown_connections 2024-05-28T14:09:42.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.417+0000 7f2007200640 1 --2- 172.21.15.42:0/2555167773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20000d0900 0x7f2000162570 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.417+0000 7f2007200640 1 -- 172.21.15.42:0/2555167773 >> 172.21.15.42:0/2555167773 conn(0x7f20000b4080 msgr2=0x7f20000b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:42.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.417+0000 7f2007200640 1 -- 172.21.15.42:0/2555167773 shutdown_connections 2024-05-28T14:09:42.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.417+0000 7f2007200640 1 -- 172.21.15.42:0/2555167773 wait complete. 2024-05-28T14:09:42.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2007200640 1 Processor -- start 2024-05-28T14:09:42.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2007200640 1 -- start start 2024-05-28T14:09:42.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2007200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20000ce860 0x7f2000153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2007200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f20000d0900 0x7f20001552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2007200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2000162ab0 0x7f2000155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2007200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f200006e040 con 0x7f2000162ab0 2024-05-28T14:09:42.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2007200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f200006dec0 con 0x7f20000ce860 2024-05-28T14:09:42.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2007200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f200016d0d0 con 0x7f20000d0900 2024-05-28T14:09:42.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2005200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2000162ab0 0x7f2000155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2005200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2000162ab0 0x7f2000155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38972/0 (socket says 172.21.15.42:38972) 2024-05-28T14:09:42.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2005200640 1 -- 172.21.15.42:0/2390610069 learned_addr learned my addr 172.21.15.42:0/2390610069 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:42.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f1fffe00640 1 --2- 172.21.15.42:0/2390610069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20000ce860 0x7f2000153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f1ff7400640 1 --2- 172.21.15.42:0/2390610069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f20000d0900 0x7f20001552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2005200640 1 -- 172.21.15.42:0/2390610069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f20000d0900 msgr2=0x7f20001552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2005200640 1 --2- 172.21.15.42:0/2390610069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f20000d0900 0x7f20001552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2005200640 1 -- 172.21.15.42:0/2390610069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20000ce860 msgr2=0x7f2000153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2005200640 1 --2- 172.21.15.42:0/2390610069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20000ce860 0x7f2000153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2005200640 1 -- 172.21.15.42:0/2390610069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200015f0e0 con 0x7f2000162ab0 2024-05-28T14:09:42.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2005200640 1 --2- 172.21.15.42:0/2390610069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2000162ab0 0x7f2000155810 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800c430 tx=0x7f1ff800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:42.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2390610069 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8005220 con 0x7f2000162ab0 2024-05-28T14:09:42.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2390610069 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff80053c0 con 0x7f2000162ab0 2024-05-28T14:09:42.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2007200640 1 -- 172.21.15.42:0/2390610069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200015f3c0 con 0x7f2000162ab0 2024-05-28T14:09:42.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2390610069 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8007270 con 0x7f2000162ab0 2024-05-28T14:09:42.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2007200640 1 -- 172.21.15.42:0/2390610069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200015f980 con 0x7f2000162ab0 2024-05-28T14:09:42.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.421+0000 7f2007200640 1 -- 172.21.15.42:0/2390610069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2000068a90 con 0x7f2000162ab0 2024-05-28T14:09:42.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.425+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2390610069 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f1ff8019020 con 0x7f2000162ab0 2024-05-28T14:09:42.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.425+0000 7f1ffcc00640 1 --2- 172.21.15.42:0/2390610069 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1fdc069520 0x7f1fdc06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:42.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.425+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2390610069 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7f1ff8084990 con 0x7f2000162ab0 2024-05-28T14:09:42.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.425+0000 7f1fffe00640 1 --2- 172.21.15.42:0/2390610069 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1fdc069520 0x7f1fdc06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:42.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.425+0000 7f1fffe00640 1 --2- 172.21.15.42:0/2390610069 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1fdc069520 0x7f1fdc06b9e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00011f0 tx=0x7f1ff0007d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:42.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.429+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2390610069 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ff8056000 con 0x7f2000162ab0 2024-05-28T14:09:42.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.629+0000 7f2007200640 1 -- 172.21.15.42:0/2390610069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f20000b4ce0 con 0x7f2000162ab0 2024-05-28T14:09:42.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.629+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2390610069 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) ==== 53+0+73 (secure 0 0 0) 0x7f1ff805a300 con 0x7f2000162ab0 2024-05-28T14:09:42.614 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-28T14:09:42.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.633+0000 7f2007200640 1 -- 172.21.15.42:0/2390610069 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1fdc069520 msgr2=0x7f1fdc06b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.633+0000 7f2007200640 1 --2- 172.21.15.42:0/2390610069 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1fdc069520 0x7f1fdc06b9e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00011f0 tx=0x7f1ff0007d40 comp rx=0 tx=0).stop 2024-05-28T14:09:42.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.633+0000 7f2007200640 1 -- 172.21.15.42:0/2390610069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2000162ab0 msgr2=0x7f2000155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:42.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.633+0000 7f2007200640 1 --2- 172.21.15.42:0/2390610069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2000162ab0 0x7f2000155810 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800c430 tx=0x7f1ff800c8f0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.633+0000 7f2007200640 1 -- 172.21.15.42:0/2390610069 shutdown_connections 2024-05-28T14:09:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.633+0000 7f2007200640 1 --2- 172.21.15.42:0/2390610069 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1fdc069520 0x7f1fdc06b9e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.633+0000 7f2007200640 1 --2- 172.21.15.42:0/2390610069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2000162ab0 0x7f2000155810 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.633+0000 7f2007200640 1 --2- 172.21.15.42:0/2390610069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f20000d0900 0x7f20001552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.633+0000 7f2007200640 1 --2- 172.21.15.42:0/2390610069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20000ce860 0x7f2000153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.633+0000 7f2007200640 1 -- 172.21.15.42:0/2390610069 >> 172.21.15.42:0/2390610069 conn(0x7f20000b4080 msgr2=0x7f200009aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.633+0000 7f2007200640 1 -- 172.21.15.42:0/2390610069 shutdown_connections 2024-05-28T14:09:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42.633+0000 7f2007200640 1 -- 172.21.15.42:0/2390610069 wait complete. 2024-05-28T14:09:42.636 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-28T14:09:42.636 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-28T14:09:42.636 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-05-28T14:09:42.636 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-28T14:09:42.636 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-28T14:09:42.637 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-28T14:09:42.637 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-28T14:09:42.637 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-28T14:09:42.637 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:09:42.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:42.637 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-05-28T14:09:42.646 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T14:09:42.647 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T14:09:42.647 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T14:09:42.647 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:09:42.647 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-05-28T14:09:42.647 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:09:42.647 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:09:42.647 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-05-28T14:09:42.647 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-05-28T14:09:42.647 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:09:42.647 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-05-28T14:09:42.647 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-05-28T14:09:42.647 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-28T14:09:42.647 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:09:42.647 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 41765sec preferred_lft 41765sec 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 02:42:65:2b:63:73 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-28T14:09:42.648 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:09:42.649 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-28T14:09:42.649 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:09:42.649 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T14:09:42.649 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:09:42.649 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-28T14:09:42.649 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:09:42.649 INFO:teuthology.orchestra.run.smithi042.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T14:09:42.649 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:09:42.649 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-28T14:09:42.649 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:09:42.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:42.649 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-28T14:09:42.713 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-05-28T14:09:42.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:42.715 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-05-28T14:09:42.769 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-05-28T14:09:42.769 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-28T14:09:42.769 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-05-28T14:09:42.769 INFO:teuthology.orchestra.run.smithi042.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-28T14:09:42.769 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-05-28T14:09:42.769 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-28T14:09:42.769 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T14:09:42.770 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T14:09:42.770 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-28T14:09:42.770 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-28T14:09:42.770 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-28T14:09:42.770 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T14:09:42.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T14:09:42.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:09:42.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:42.908 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-28T14:09:42.917 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-28T14:09:42.917 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-28T14:09:42.917 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:09:42.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:42.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-05-28T14:09:42.978 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-28T14:09:42.978 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T14:09:42.979 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-28T14:09:43.031 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:09:43.031 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T14:09:43.031 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-28T14:09:43.086 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:09:43.147 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:09:43.199 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:09:43.200 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:09:43.200 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:09:43.200 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:09:43.200 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:09:43.200 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:09:43.200 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:09:43.200 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:09:43.200 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:09:43.200 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:09:43.200 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:09:43.200 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:09:43.201 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:09:43.201 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:09:43.201 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:09:43.201 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:09:43.201 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:09:43.201 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:09:43.201 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:09:43.201 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:09:43.201 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:09:43.201 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:09:43.201 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:09:43.201 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:09:43.201 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:09:43.201 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:09:43.202 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:09:43.202 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:09:43.202 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:09:43.202 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:09:43.202 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:09:43.202 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:09:43.202 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:09:43.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:43.203 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:09:43.258 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-28T14:09:43.259 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-28T14:09:43.309 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:09:43.330 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:09:43.389 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:09:43.389 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:09:43.389 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:09:43.389 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:09:43.389 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:09:43.389 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:09:43.389 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:09:43.389 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:09:43.390 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:09:43.390 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:09:43.390 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:09:43.390 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:09:43.390 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:09:43.390 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:09:43.390 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:09:43.390 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:09:43.390 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:09:43.390 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:09:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:09:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:09:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:09:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:09:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:09:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:09:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:09:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:09:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:09:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:09:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:09:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:09:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:09:43.392 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:09:43.392 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:09:43.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:43.392 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:09:43.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-05-28T14:09:43.425+0000 7f054bd6a580 -1 init, newargv = 0x55b0c4236ab0 newargc=15 2024-05-28T14:09:43.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[48091]: starting ceph client 2024-05-28T14:09:43.415 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[48091]: starting fuse 2024-05-28T14:09:43.416 INFO:teuthology.orchestra.run.smithi042.stdout:44 2024-05-28T14:09:43.417 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-28T14:09:43.417 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-05-28T14:09:43.417 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-05-28T14:09:43.417 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-05-28T14:09:43.417 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-05-28T14:09:43.417 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-05-28T14:09:43.417 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:09:43.417 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-05-28T14:09:43.417 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-28T14:09:43.417 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-05-28T14:09:43.417 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-28T14:09:43.418 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:09:43.418 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-05-28T14:09:43.418 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-05-28T14:09:43.418 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-05-28T14:09:43.418 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-05-28T14:09:43.418 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:09:43.418 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-05-28T14:09:43.418 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-28T14:09:43.418 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-28T14:09:43.418 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-28T14:09:43.418 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-05-28T14:09:43.419 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-05-28T14:09:43.419 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-05-28T14:09:43.419 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-28T14:09:43.419 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:09:43.419 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-05-28T14:09:43.419 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-05-28T14:09:43.541 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.48091.asok 2024-05-28T14:09:43.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:09:43.553 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48091.asok 2024-05-28T14:09:43.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:43.553 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48091.asok status 2024-05-28T14:09:43.727 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:09:43.727 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-05-28T14:09:43.727 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-28T14:09:43.727 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-28T14:09:43.727 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-05-28T14:09:43.727 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-05-28T14:09:43.727 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-28T14:09:43.727 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "48091", 2024-05-28T14:09:43.727 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-05-28T14:09:43.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:43.727 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-05-28T14:09:43.727 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 8172, 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 8172 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 895080631 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T14:09:43.728 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T14:09:43.729 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 895080631 2024-05-28T14:09:43.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:09:43.729 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.8172 192.168.0.1:0/895080631", 2024-05-28T14:09:43.729 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/895080631", 2024-05-28T14:09:43.729 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-05-28T14:09:43.729 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 54, 2024-05-28T14:09:43.729 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 90, 2024-05-28T14:09:43.729 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-05-28T14:09:43.729 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-05-28T14:09:43.729 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-05-28T14:09:43.729 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:09:43.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:43.742 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-05-28T14:09:43.742 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-28T14:09:43.751 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-05-28T14:09:43.751 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-28T14:09:43.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:43.752 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-28T14:09:43.815 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-28T14:09:44.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.021+0000 7ff929200640 1 Processor -- start 2024-05-28T14:09:44.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.021+0000 7ff929200640 1 -- start start 2024-05-28T14:09:44.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.025+0000 7ff929200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff92415e840 0x7ff9240d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:44.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.025+0000 7ff929200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9240d1970 con 0x7ff92406cd10 2024-05-28T14:09:44.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.025+0000 7ff929200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9240ccca0 con 0x7ff92406dc20 2024-05-28T14:09:44.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.025+0000 7ff929200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff9240b5300 con 0x7ff92415e840 2024-05-28T14:09:44.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.025+0000 7ff921e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff92415e840 0x7ff9240d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:44.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.025+0000 7ff921e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff92415e840 0x7ff9240d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38980/0 (socket says 172.21.15.42:38980) 2024-05-28T14:09:44.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.025+0000 7ff921e00640 1 -- 172.21.15.42:0/163297578 learned_addr learned my addr 172.21.15.42:0/163297578 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:44.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.025+0000 7ff921e00640 1 -- 172.21.15.42:0/163297578 >> v1:172.21.15.63:6789/0 conn(0x7ff92406dc20 legacy=0x7ff9240cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:44.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.025+0000 7ff921e00640 1 -- 172.21.15.42:0/163297578 >> v1:172.21.15.42:6789/0 conn(0x7ff92406cd10 legacy=0x7ff92406d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:44.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.025+0000 7ff921e00640 1 -- 172.21.15.42:0/163297578 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9240ccf80 con 0x7ff92415e840 2024-05-28T14:09:44.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.029+0000 7ff921e00640 1 --2- 172.21.15.42:0/163297578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff92415e840 0x7ff9240d1200 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ff914009870 tx=0x7ff91401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe97f4755a08a1c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:44.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.029+0000 7ff921400640 1 -- 172.21.15.42:0/163297578 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140b5070 con 0x7ff92415e840 2024-05-28T14:09:44.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.029+0000 7ff921400640 1 -- 172.21.15.42:0/163297578 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9140a8a10 con 0x7ff92415e840 2024-05-28T14:09:44.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.029+0000 7ff921400640 1 -- 172.21.15.42:0/163297578 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140a8de0 con 0x7ff92415e840 2024-05-28T14:09:44.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.029+0000 7ff929200640 1 -- 172.21.15.42:0/163297578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff92415e840 msgr2=0x7ff9240d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:44.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.029+0000 7ff929200640 1 --2- 172.21.15.42:0/163297578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff92415e840 0x7ff9240d1200 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ff914009870 tx=0x7ff91401ed10 comp rx=0 tx=0).stop 2024-05-28T14:09:44.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.029+0000 7ff929200640 1 -- 172.21.15.42:0/163297578 shutdown_connections 2024-05-28T14:09:44.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.029+0000 7ff929200640 1 --2- 172.21.15.42:0/163297578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff92415e840 0x7ff9240d1200 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:44.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.029+0000 7ff929200640 1 -- 172.21.15.42:0/163297578 >> 172.21.15.42:0/163297578 conn(0x7ff9240b3fa0 msgr2=0x7ff9240b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:44.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.029+0000 7ff929200640 1 -- 172.21.15.42:0/163297578 shutdown_connections 2024-05-28T14:09:44.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.029+0000 7ff929200640 1 -- 172.21.15.42:0/163297578 wait complete. 2024-05-28T14:09:44.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.029+0000 7ff929200640 1 Processor -- start 2024-05-28T14:09:44.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.029+0000 7ff929200640 1 -- start start 2024-05-28T14:09:44.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff929200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff92406cd10 0x7ff924155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:44.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff929200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff92406dc20 0x7ff92415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:44.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff929200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff92415e840 0x7ff92415f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:44.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff929200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff92415ec20 con 0x7ff92406cd10 2024-05-28T14:09:44.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff929200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff92415eda0 con 0x7ff92415e840 2024-05-28T14:09:44.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff929200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9240cde90 con 0x7ff92406dc20 2024-05-28T14:09:44.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff921e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff92406cd10 0x7ff924155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:44.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff921e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff92406cd10 0x7ff924155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38996/0 (socket says 172.21.15.42:38996) 2024-05-28T14:09:44.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff921e00640 1 -- 172.21.15.42:0/510909279 learned_addr learned my addr 172.21.15.42:0/510909279 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:44.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff921e00640 1 -- 172.21.15.42:0/510909279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff92406dc20 msgr2=0x7ff92415ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:44.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff921e00640 1 --2- 172.21.15.42:0/510909279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff92406dc20 0x7ff92415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:44.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff921e00640 1 -- 172.21.15.42:0/510909279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff92415e840 msgr2=0x7ff92415f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:44.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff921e00640 1 --2- 172.21.15.42:0/510909279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff92415e840 0x7ff92415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:44.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff921e00640 1 -- 172.21.15.42:0/510909279 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9241f4b30 con 0x7ff92406cd10 2024-05-28T14:09:44.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff921e00640 1 --2- 172.21.15.42:0/510909279 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff92406cd10 0x7ff924155520 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7ff9140b3040 tx=0x7ff91401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:44.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff91a000640 1 -- 172.21.15.42:0/510909279 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140c0070 con 0x7ff92406cd10 2024-05-28T14:09:44.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff91a000640 1 -- 172.21.15.42:0/510909279 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9140b0c20 con 0x7ff92406cd10 2024-05-28T14:09:44.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff91a000640 1 -- 172.21.15.42:0/510909279 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140b5070 con 0x7ff92406cd10 2024-05-28T14:09:44.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff929200640 1 -- 172.21.15.42:0/510909279 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9241f4e10 con 0x7ff92406cd10 2024-05-28T14:09:44.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff929200640 1 -- 172.21.15.42:0/510909279 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9241f53d0 con 0x7ff92406cd10 2024-05-28T14:09:44.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff929200640 1 -- 172.21.15.42:0/510909279 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff924068a90 con 0x7ff92406cd10 2024-05-28T14:09:44.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.033+0000 7ff91a000640 1 -- 172.21.15.42:0/510909279 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7ff9140bb440 con 0x7ff92406cd10 2024-05-28T14:09:44.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.037+0000 7ff91a000640 1 --2- 172.21.15.42:0/510909279 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9000694f0 0x7ff90006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:44.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.037+0000 7ff91a000640 1 -- 172.21.15.42:0/510909279 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7ff914122b40 con 0x7ff92406cd10 2024-05-28T14:09:44.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.037+0000 7ff91be00640 1 --2- 172.21.15.42:0/510909279 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9000694f0 0x7ff90006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:44.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.037+0000 7ff91be00640 1 --2- 172.21.15.42:0/510909279 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9000694f0 0x7ff90006b9b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff90c000f70 tx=0x7ff90c0079e0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:44.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.041+0000 7ff91a000640 1 -- 172.21.15.42:0/510909279 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff914018870 con 0x7ff92406cd10 2024-05-28T14:09:44.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.245+0000 7ff929200640 1 -- 172.21.15.42:0/510909279 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7ff9241f5610 con 0x7ff92406cd10 2024-05-28T14:09:44.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.793+0000 7ff91a000640 1 -- 172.21.15.42:0/510909279 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v55) ==== 136+0+0 (secure 0 0 0) 0x7ff9241f5610 con 0x7ff92406cd10 2024-05-28T14:09:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.797+0000 7ff929200640 1 -- 172.21.15.42:0/510909279 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9000694f0 msgr2=0x7ff90006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.797+0000 7ff929200640 1 --2- 172.21.15.42:0/510909279 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9000694f0 0x7ff90006b9b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff90c000f70 tx=0x7ff90c0079e0 comp rx=0 tx=0).stop 2024-05-28T14:09:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.797+0000 7ff929200640 1 -- 172.21.15.42:0/510909279 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff92406cd10 msgr2=0x7ff924155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.797+0000 7ff929200640 1 --2- 172.21.15.42:0/510909279 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff92406cd10 0x7ff924155520 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7ff9140b3040 tx=0x7ff91401bab0 comp rx=0 tx=0).stop 2024-05-28T14:09:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.797+0000 7ff929200640 1 -- 172.21.15.42:0/510909279 shutdown_connections 2024-05-28T14:09:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.797+0000 7ff929200640 1 --2- 172.21.15.42:0/510909279 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9000694f0 0x7ff90006b9b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.797+0000 7ff929200640 1 --2- 172.21.15.42:0/510909279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff92415e840 0x7ff92415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.797+0000 7ff929200640 1 --2- 172.21.15.42:0/510909279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff92406dc20 0x7ff92415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.797+0000 7ff929200640 1 --2- 172.21.15.42:0/510909279 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff92406cd10 0x7ff924155520 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.797+0000 7ff929200640 1 -- 172.21.15.42:0/510909279 >> 172.21.15.42:0/510909279 conn(0x7ff9240b3fa0 msgr2=0x7ff9240d1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:44.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.797+0000 7ff929200640 1 -- 172.21.15.42:0/510909279 shutdown_connections 2024-05-28T14:09:44.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.797+0000 7ff929200640 1 -- 172.21.15.42:0/510909279 wait complete. 2024-05-28T14:09:44.804 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:09:44.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.969+0000 7fb6c6e00640 1 Processor -- start 2024-05-28T14:09:44.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.969+0000 7fb6c6e00640 1 -- start start 2024-05-28T14:09:44.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.969+0000 7fb6c6e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb6c009b160 0x7fb6c00b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:44.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.969+0000 7fb6c6e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6c00b1ac0 con 0x7fb6c00b33f0 2024-05-28T14:09:44.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.969+0000 7fb6c6e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6c00d0c80 con 0x7fb6c015f9c0 2024-05-28T14:09:44.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.969+0000 7fb6c6e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb6c00d3260 con 0x7fb6c009b160 2024-05-28T14:09:44.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.969+0000 7fb6c5000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb6c009b160 0x7fb6c00b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:44.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.969+0000 7fb6c5000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb6c009b160 0x7fb6c00b3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39000/0 (socket says 172.21.15.42:39000) 2024-05-28T14:09:44.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.969+0000 7fb6c5000640 1 -- 172.21.15.42:0/2452752546 learned_addr learned my addr 172.21.15.42:0/2452752546 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:44.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.969+0000 7fb6c6400640 1 --1- 172.21.15.42:0/2452752546 >> v1:172.21.15.42:6789/0 conn(0x7fb6c00b33f0 0x7fb6c00b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:45126/0 (socket says 172.21.15.42:45126) 2024-05-28T14:09:44.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.973+0000 7fb6c5000640 1 -- 172.21.15.42:0/2452752546 >> v1:172.21.15.63:6789/0 conn(0x7fb6c015f9c0 legacy=0x7fb6c009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:44.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.973+0000 7fb6c5000640 1 -- 172.21.15.42:0/2452752546 >> v1:172.21.15.42:6789/0 conn(0x7fb6c00b33f0 legacy=0x7fb6c00b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:44.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.973+0000 7fb6c5000640 1 -- 172.21.15.42:0/2452752546 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c00d0f60 con 0x7fb6c009b160 2024-05-28T14:09:44.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.973+0000 7fb6c5000640 1 --2- 172.21.15.42:0/2452752546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6c009b160 0x7fb6c00b3870 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009870 tx=0x7fb6ac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc68e092e13f9611 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:44.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.973+0000 7fb6bfe00640 1 -- 172.21.15.42:0/2452752546 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0b0070 con 0x7fb6c009b160 2024-05-28T14:09:44.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.973+0000 7fb6bfe00640 1 -- 172.21.15.42:0/2452752546 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ac0ab600 con 0x7fb6c009b160 2024-05-28T14:09:44.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.973+0000 7fb6bfe00640 1 -- 172.21.15.42:0/2452752546 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0ab9d0 con 0x7fb6c009b160 2024-05-28T14:09:44.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.973+0000 7fb6c6e00640 1 -- 172.21.15.42:0/2452752546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6c009b160 msgr2=0x7fb6c00b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:44.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.973+0000 7fb6c6e00640 1 --2- 172.21.15.42:0/2452752546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6c009b160 0x7fb6c00b3870 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009870 tx=0x7fb6ac01c520 comp rx=0 tx=0).stop 2024-05-28T14:09:44.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6e00640 1 -- 172.21.15.42:0/2452752546 shutdown_connections 2024-05-28T14:09:44.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6e00640 1 --2- 172.21.15.42:0/2452752546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6c009b160 0x7fb6c00b3870 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:44.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6e00640 1 -- 172.21.15.42:0/2452752546 >> 172.21.15.42:0/2452752546 conn(0x7fb6c00d2360 msgr2=0x7fb6c00d2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:44.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6e00640 1 -- 172.21.15.42:0/2452752546 shutdown_connections 2024-05-28T14:09:44.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6e00640 1 -- 172.21.15.42:0/2452752546 wait complete. 2024-05-28T14:09:44.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6e00640 1 Processor -- start 2024-05-28T14:09:44.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6e00640 1 -- start start 2024-05-28T14:09:44.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6c009b160 0x7fb6c00c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:44.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6c00b33f0 0x7fb6c00b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:44.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6c015f9c0 0x7fb6c015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:44.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c00b1df0 con 0x7fb6c00b33f0 2024-05-28T14:09:44.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c00b31a0 con 0x7fb6c009b160 2024-05-28T14:09:44.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c016a7d0 con 0x7fb6c015f9c0 2024-05-28T14:09:44.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6c00b33f0 0x7fb6c00b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:44.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6c00b33f0 0x7fb6c00b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39012/0 (socket says 172.21.15.42:39012) 2024-05-28T14:09:44.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c5000640 1 -- 172.21.15.42:0/3199760272 learned_addr learned my addr 172.21.15.42:0/3199760272 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:44.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c5a00640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6c009b160 0x7fb6c00c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:44.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c5000640 1 -- 172.21.15.42:0/3199760272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6c015f9c0 msgr2=0x7fb6c015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:44.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6400640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6c015f9c0 0x7fb6c015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:44.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c5000640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6c015f9c0 0x7fb6c015afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:44.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c5000640 1 -- 172.21.15.42:0/3199760272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6c009b160 msgr2=0x7fb6c00c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:44.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c5000640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6c009b160 0x7fb6c00c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:44.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c5000640 1 -- 172.21.15.42:0/3199760272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c00b0190 con 0x7fb6c00b33f0 2024-05-28T14:09:44.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c5a00640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6c009b160 0x7fb6c00c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:44.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c6400640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6c015f9c0 0x7fb6c015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:44.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.977+0000 7fb6c5000640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6c00b33f0 0x7fb6c00b55d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0ae040 tx=0x7fb6ac0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:44.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.981+0000 7fb6bd600640 1 -- 172.21.15.42:0/3199760272 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0bb070 con 0x7fb6c00b33f0 2024-05-28T14:09:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.981+0000 7fb6bd600640 1 -- 172.21.15.42:0/3199760272 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ac0abe10 con 0x7fb6c00b33f0 2024-05-28T14:09:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.981+0000 7fb6bd600640 1 -- 172.21.15.42:0/3199760272 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0b0040 con 0x7fb6c00b33f0 2024-05-28T14:09:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.981+0000 7fb6c6e00640 1 -- 172.21.15.42:0/3199760272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c00b5b10 con 0x7fb6c00b33f0 2024-05-28T14:09:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.981+0000 7fb6c6e00640 1 -- 172.21.15.42:0/3199760272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c015ea50 con 0x7fb6c00b33f0 2024-05-28T14:09:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.981+0000 7fb6c6e00640 1 -- 172.21.15.42:0/3199760272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c004fcf0 con 0x7fb6c00b33f0 2024-05-28T14:09:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.981+0000 7fb6bd600640 1 -- 172.21.15.42:0/3199760272 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fb6ac0aa970 con 0x7fb6c00b33f0 2024-05-28T14:09:44.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.981+0000 7fb6bd600640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb6a00694f0 0x7fb6a006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:44.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.981+0000 7fb6bd600640 1 -- 172.21.15.42:0/3199760272 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7fb6ac11f2d0 con 0x7fb6c00b33f0 2024-05-28T14:09:44.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.981+0000 7fb6c5a00640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb6a00694f0 0x7fb6a006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:44.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.985+0000 7fb6c5a00640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb6a00694f0 0x7fb6a006b9b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400a850 tx=0x7fb6b4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:44.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:44.985+0000 7fb6bd600640 1 -- 172.21.15.42:0/3199760272 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb6ac016110 con 0x7fb6c00b33f0 2024-05-28T14:09:45.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.173+0000 7fb6c6e00640 1 -- 172.21.15.42:0/3199760272 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fb6c00b7f40 con 0x7fb6a00694f0 2024-05-28T14:09:45.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.173+0000 7fb6bd600640 1 -- 172.21.15.42:0/3199760272 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fb6c00b7f40 con 0x7fb6a00694f0 2024-05-28T14:09:45.157 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:09:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:09:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:09:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:09:45.158 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:09:45.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.177+0000 7fb6c6e00640 1 -- 172.21.15.42:0/3199760272 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb6a00694f0 msgr2=0x7fb6a006b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.177+0000 7fb6c6e00640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb6a00694f0 0x7fb6a006b9b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400a850 tx=0x7fb6b4008040 comp rx=0 tx=0).stop 2024-05-28T14:09:45.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.177+0000 7fb6c6e00640 1 -- 172.21.15.42:0/3199760272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6c00b33f0 msgr2=0x7fb6c00b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.177+0000 7fb6c6e00640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6c00b33f0 0x7fb6c00b55d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0ae040 tx=0x7fb6ac0192c0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.177+0000 7fb6c6e00640 1 -- 172.21.15.42:0/3199760272 shutdown_connections 2024-05-28T14:09:45.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.177+0000 7fb6c6e00640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb6a00694f0 0x7fb6a006b9b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.177+0000 7fb6c6e00640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6c015f9c0 0x7fb6c015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.177+0000 7fb6c6e00640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6c00b33f0 0x7fb6c00b55d0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.177+0000 7fb6c6e00640 1 --2- 172.21.15.42:0/3199760272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6c009b160 0x7fb6c00c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.177+0000 7fb6c6e00640 1 -- 172.21.15.42:0/3199760272 >> 172.21.15.42:0/3199760272 conn(0x7fb6c00d2360 msgr2=0x7fb6c009aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:45.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.177+0000 7fb6c6e00640 1 -- 172.21.15.42:0/3199760272 shutdown_connections 2024-05-28T14:09:45.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.177+0000 7fb6c6e00640 1 -- 172.21.15.42:0/3199760272 wait complete. 2024-05-28T14:09:45.180 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T14:09:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.341+0000 7fb212600640 1 Processor -- start 2024-05-28T14:09:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.341+0000 7fb212600640 1 -- start start 2024-05-28T14:09:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.341+0000 7fb212600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb20c144f80 0x7fb20c072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.341+0000 7fb212600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb20c0afe40 con 0x7fb20c15c0c0 2024-05-28T14:09:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.341+0000 7fb212600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb20c15c600 con 0x7fb20c06ee20 2024-05-28T14:09:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.341+0000 7fb212600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb20c0724d0 con 0x7fb20c144f80 2024-05-28T14:09:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.341+0000 7fb20be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb20c15c0c0 0x7fb20c0b3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:45128/0 (socket says 172.21.15.42:45128) 2024-05-28T14:09:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.341+0000 7fb20be00640 1 -- 172.21.15.42:0/3339771186 learned_addr learned my addr 172.21.15.42:0/3339771186 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.341+0000 7fb20b400640 1 --2- 172.21.15.42:0/3339771186 >> v2:172.21.15.63:3300/0 conn(0x7fb20c144f80 0x7fb20c072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.341+0000 7fb20aa00640 1 -- 172.21.15.42:0/3339771186 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2285914898 0 0) 0x7fb20c0afe40 con 0x7fb20c15c0c0 2024-05-28T14:09:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.341+0000 7fb20aa00640 1 -- 172.21.15.42:0/3339771186 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1f0003530 con 0x7fb20c15c0c0 2024-05-28T14:09:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.341+0000 7fb20b400640 1 -- 172.21.15.42:0/3339771186 >> v1:172.21.15.63:6789/0 conn(0x7fb20c06ee20 legacy=0x7fb20c15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.341+0000 7fb20b400640 1 -- 172.21.15.42:0/3339771186 >> v1:172.21.15.42:6789/0 conn(0x7fb20c15c0c0 legacy=0x7fb20c0b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.341+0000 7fb20b400640 1 -- 172.21.15.42:0/3339771186 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c15c8e0 con 0x7fb20c144f80 2024-05-28T14:09:45.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.345+0000 7fb20b400640 1 --2- 172.21.15.42:0/3339771186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c144f80 0x7fb20c072960 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0097c0 tx=0x7fb1fc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec239175f5bb3277 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:45.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.345+0000 7fb20aa00640 1 -- 172.21.15.42:0/3339771186 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc0b0070 con 0x7fb20c144f80 2024-05-28T14:09:45.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.345+0000 7fb20aa00640 1 -- 172.21.15.42:0/3339771186 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1fc0ab600 con 0x7fb20c144f80 2024-05-28T14:09:45.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.345+0000 7fb20aa00640 1 -- 172.21.15.42:0/3339771186 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc0ab9d0 con 0x7fb20c144f80 2024-05-28T14:09:45.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.345+0000 7fb212600640 1 -- 172.21.15.42:0/3339771186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c144f80 msgr2=0x7fb20c072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.345+0000 7fb212600640 1 --2- 172.21.15.42:0/3339771186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c144f80 0x7fb20c072960 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0097c0 tx=0x7fb1fc01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.345+0000 7fb212600640 1 -- 172.21.15.42:0/3339771186 shutdown_connections 2024-05-28T14:09:45.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.345+0000 7fb212600640 1 --2- 172.21.15.42:0/3339771186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c144f80 0x7fb20c072960 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.345+0000 7fb212600640 1 -- 172.21.15.42:0/3339771186 >> 172.21.15.42:0/3339771186 conn(0x7fb20c0b21f0 msgr2=0x7fb20c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:45.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.345+0000 7fb212600640 1 -- 172.21.15.42:0/3339771186 shutdown_connections 2024-05-28T14:09:45.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.345+0000 7fb212600640 1 -- 172.21.15.42:0/3339771186 wait complete. 2024-05-28T14:09:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb212600640 1 Processor -- start 2024-05-28T14:09:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb212600640 1 -- start start 2024-05-28T14:09:45.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb212600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb20c06ee20 0x7fb20c1fab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:45.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb212600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb20c144f80 0x7fb20c15f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:45.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb212600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c15c0c0 0x7fb20c06e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:45.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb212600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c06f790 con 0x7fb20c144f80 2024-05-28T14:09:45.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb212600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c06f610 con 0x7fb20c15c0c0 2024-05-28T14:09:45.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb212600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb20c169950 con 0x7fb20c06ee20 2024-05-28T14:09:45.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb202a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb20c144f80 0x7fb20c15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:45.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb202a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb20c144f80 0x7fb20c15f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39022/0 (socket says 172.21.15.42:39022) 2024-05-28T14:09:45.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb20be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c15c0c0 0x7fb20c06e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:45.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb202a00640 1 -- 172.21.15.42:0/2415991623 learned_addr learned my addr 172.21.15.42:0/2415991623 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:45.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb20b400640 1 --2- 172.21.15.42:0/2415991623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb20c06ee20 0x7fb20c1fab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb202a00640 1 -- 172.21.15.42:0/2415991623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb20c06ee20 msgr2=0x7fb20c1fab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb202a00640 1 --2- 172.21.15.42:0/2415991623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb20c06ee20 0x7fb20c1fab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb202a00640 1 -- 172.21.15.42:0/2415991623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c15c0c0 msgr2=0x7fb20c06e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb202a00640 1 --2- 172.21.15.42:0/2415991623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c15c0c0 0x7fb20c06e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb202a00640 1 -- 172.21.15.42:0/2415991623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c15daf0 con 0x7fb20c144f80 2024-05-28T14:09:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb202a00640 1 --2- 172.21.15.42:0/2415991623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb20c144f80 0x7fb20c15f360 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400c500 tx=0x7fb1f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb203e00640 1 -- 172.21.15.42:0/2415991623 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4016020 con 0x7fb20c144f80 2024-05-28T14:09:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb203e00640 1 -- 172.21.15.42:0/2415991623 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f4005220 con 0x7fb20c144f80 2024-05-28T14:09:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb203e00640 1 -- 172.21.15.42:0/2415991623 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4007030 con 0x7fb20c144f80 2024-05-28T14:09:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb212600640 1 -- 172.21.15.42:0/2415991623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20c15ddd0 con 0x7fb20c144f80 2024-05-28T14:09:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.349+0000 7fb212600640 1 -- 172.21.15.42:0/2415991623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20c15e390 con 0x7fb20c144f80 2024-05-28T14:09:45.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.353+0000 7fb212600640 1 -- 172.21.15.42:0/2415991623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20c0738b0 con 0x7fb20c144f80 2024-05-28T14:09:45.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.353+0000 7fb203e00640 1 -- 172.21.15.42:0/2415991623 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fb1f40071f0 con 0x7fb20c144f80 2024-05-28T14:09:45.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.353+0000 7fb203e00640 1 --2- 172.21.15.42:0/2415991623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb1e80694f0 0x7fb1e806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:45.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.353+0000 7fb203e00640 1 -- 172.21.15.42:0/2415991623 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7fb1f40851f0 con 0x7fb20c144f80 2024-05-28T14:09:45.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.353+0000 7fb20b400640 1 --2- 172.21.15.42:0/2415991623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb1e80694f0 0x7fb1e806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:45.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.353+0000 7fb20b400640 1 --2- 172.21.15.42:0/2415991623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb1e80694f0 0x7fb1e806b9b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0ad000 tx=0x7fb1fc0a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:45.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.357+0000 7fb203e00640 1 -- 172.21.15.42:0/2415991623 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb1f4020020 con 0x7fb20c144f80 2024-05-28T14:09:45.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.525+0000 7fb212600640 1 -- 172.21.15.42:0/2415991623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb20c13fc00 con 0x7fb20c144f80 2024-05-28T14:09:45.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.525+0000 7fb203e00640 1 -- 172.21.15.42:0/2415991623 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7fb1f40567e0 con 0x7fb20c144f80 2024-05-28T14:09:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.525+0000 7fb212600640 1 -- 172.21.15.42:0/2415991623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb1e80694f0 msgr2=0x7fb1e806b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.525+0000 7fb212600640 1 --2- 172.21.15.42:0/2415991623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb1e80694f0 0x7fb1e806b9b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0ad000 tx=0x7fb1fc0a6000 comp rx=0 tx=0).stop 2024-05-28T14:09:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.525+0000 7fb212600640 1 -- 172.21.15.42:0/2415991623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb20c144f80 msgr2=0x7fb20c15f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.525+0000 7fb212600640 1 --2- 172.21.15.42:0/2415991623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb20c144f80 0x7fb20c15f360 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400c500 tx=0x7fb1f400c9c0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.529+0000 7fb212600640 1 -- 172.21.15.42:0/2415991623 shutdown_connections 2024-05-28T14:09:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.529+0000 7fb212600640 1 --2- 172.21.15.42:0/2415991623 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb1e80694f0 0x7fb1e806b9b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.529+0000 7fb212600640 1 --2- 172.21.15.42:0/2415991623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c15c0c0 0x7fb20c06e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.529+0000 7fb212600640 1 --2- 172.21.15.42:0/2415991623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb20c144f80 0x7fb20c15f360 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.529+0000 7fb212600640 1 --2- 172.21.15.42:0/2415991623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb20c06ee20 0x7fb20c1fab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.529+0000 7fb212600640 1 -- 172.21.15.42:0/2415991623 >> 172.21.15.42:0/2415991623 conn(0x7fb20c0b21f0 msgr2=0x7fb20c140120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.529+0000 7fb212600640 1 -- 172.21.15.42:0/2415991623 shutdown_connections 2024-05-28T14:09:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.529+0000 7fb212600640 1 -- 172.21.15.42:0/2415991623 wait complete. 2024-05-28T14:09:45.530 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-28T14:09:45.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56dc800640 1 Processor -- start 2024-05-28T14:09:45.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56dc800640 1 -- start start 2024-05-28T14:09:45.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56dc800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d40b01c0 0x7f56d40730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:45.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56dc800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d415ece0 con 0x7f56d406d110 2024-05-28T14:09:45.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56dc800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d415ef60 con 0x7f56d4073610 2024-05-28T14:09:45.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56dc800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f56d415f240 con 0x7f56d40b01c0 2024-05-28T14:09:45.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56da800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f56d406d110 0x7f56d4154e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:45144/0 (socket says 172.21.15.42:45144) 2024-05-28T14:09:45.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56da800640 1 -- 172.21.15.42:0/1040245251 learned_addr learned my addr 172.21.15.42:0/1040245251 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:09:45.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d9e00640 1 --2- 172.21.15.42:0/1040245251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d40b01c0 0x7f56d40730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:45.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 -- 172.21.15.42:0/1040245251 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3458552390 0 0) 0x7f56d00014a0 con 0x7f56d406d110 2024-05-28T14:09:45.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 -- 172.21.15.42:0/1040245251 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56bc003530 con 0x7f56d406d110 2024-05-28T14:09:45.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 -- 172.21.15.42:0/1040245251 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 917507346 0 0) 0x7f56bc003530 con 0x7f56d406d110 2024-05-28T14:09:45.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 -- 172.21.15.42:0/1040245251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d40b01c0 msgr2=0x7f56d40730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:45.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 --2- 172.21.15.42:0/1040245251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d40b01c0 0x7f56d40730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 -- 172.21.15.42:0/1040245251 >> v1:172.21.15.63:6789/0 conn(0x7f56d4073610 legacy=0x7f56d406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 -- 172.21.15.42:0/1040245251 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d415f520 con 0x7f56d406d110 2024-05-28T14:09:45.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 -- 172.21.15.42:0/1040245251 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f56d0001bd0 con 0x7f56d406d110 2024-05-28T14:09:45.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 -- 172.21.15.42:0/1040245251 >> v1:172.21.15.42:6789/0 conn(0x7f56d406d110 legacy=0x7f56d4154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 --2- 172.21.15.42:0/1040245251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56bc0039f0 0x7f56bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:45.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 --2- 172.21.15.42:0/1040245251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56bc0043a0 0x7f56bc004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:45.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 --2- 172.21.15.42:0/1040245251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d40b01c0 0x7f56bc0050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 -- 172.21.15.42:0/1040245251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f56d415f240 con 0x7f56bc0039f0 2024-05-28T14:09:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 -- 172.21.15.42:0/1040245251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc00b9d0 con 0x7f56bc0043a0 2024-05-28T14:09:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d8a00640 1 -- 172.21.15.42:0/1040245251 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f56bc00bbd0 con 0x7f56d40b01c0 2024-05-28T14:09:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d9e00640 1 --2- 172.21.15.42:0/1040245251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56bc0039f0 0x7f56bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d9400640 1 --2- 172.21.15.42:0/1040245251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56bc0043a0 0x7f56bc004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56da800640 1 --2- 172.21.15.42:0/1040245251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d40b01c0 0x7f56bc0050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d9e00640 1 -- 172.21.15.42:0/1040245251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d40b01c0 msgr2=0x7f56bc0050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d9e00640 1 --2- 172.21.15.42:0/1040245251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d40b01c0 0x7f56bc0050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d9e00640 1 -- 172.21.15.42:0/1040245251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56bc0043a0 msgr2=0x7f56bc004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d9e00640 1 --2- 172.21.15.42:0/1040245251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56bc0043a0 0x7f56bc004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.693+0000 7f56d9e00640 1 -- 172.21.15.42:0/1040245251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc003530 con 0x7f56bc0039f0 2024-05-28T14:09:45.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.697+0000 7f56d9e00640 1 --2- 172.21.15.42:0/1040245251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56bc0039f0 0x7f56bc003dd0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009ae0 tx=0x7f56c801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc26e9684ce6ce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:45.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.697+0000 7f56d8a00640 1 -- 172.21.15.42:0/1040245251 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c80b2070 con 0x7f56bc0039f0 2024-05-28T14:09:45.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.697+0000 7f56d8a00640 1 -- 172.21.15.42:0/1040245251 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c80a5660 con 0x7f56bc0039f0 2024-05-28T14:09:45.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.697+0000 7f56d8a00640 1 -- 172.21.15.42:0/1040245251 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c80096b0 con 0x7f56bc0039f0 2024-05-28T14:09:45.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.697+0000 7f56dc800640 1 -- 172.21.15.42:0/1040245251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56bc0039f0 msgr2=0x7f56bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.697+0000 7f56dc800640 1 --2- 172.21.15.42:0/1040245251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56bc0039f0 0x7f56bc003dd0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009ae0 tx=0x7f56c801ca70 comp rx=0 tx=0).stop 2024-05-28T14:09:45.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.697+0000 7f56da800640 1 -- 172.21.15.42:0/1040245251 reap_dead start 2024-05-28T14:09:45.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.697+0000 7f56dc800640 1 -- 172.21.15.42:0/1040245251 shutdown_connections 2024-05-28T14:09:45.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.697+0000 7f56dc800640 1 -- 172.21.15.42:0/1040245251 >> 172.21.15.42:0/1040245251 conn(0x7f56d40725b0 msgr2=0x7f56d40729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:45.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.697+0000 7f56dc800640 1 -- 172.21.15.42:0/1040245251 shutdown_connections 2024-05-28T14:09:45.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.697+0000 7f56dc800640 1 -- 172.21.15.42:0/1040245251 wait complete. 2024-05-28T14:09:45.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56dc800640 1 Processor -- start 2024-05-28T14:09:45.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56dc800640 1 -- start start 2024-05-28T14:09:45.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56dc800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56bc0039f0 0x7f56d414f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:45.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56dc800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d4160e60 0x7f56d41612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:45.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56dc800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56d4161820 0x7f56d41f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:45.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56dc800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f56d40b4e00 con 0x7f56d4161820 2024-05-28T14:09:45.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56dc800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56d415fae0 con 0x7f56bc0039f0 2024-05-28T14:09:45.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56dc800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f56d40b05a0 con 0x7f56d4160e60 2024-05-28T14:09:45.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56da800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56d4161820 0x7f56d41f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56da800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56d4161820 0x7f56d41f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39042/0 (socket says 172.21.15.42:39042) 2024-05-28T14:09:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56da800640 1 -- 172.21.15.42:0/3308980432 learned_addr learned my addr 172.21.15.42:0/3308980432 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56d9e00640 1 --2- 172.21.15.42:0/3308980432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56bc0039f0 0x7f56d414f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56d9400640 1 --2- 172.21.15.42:0/3308980432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d4160e60 0x7f56d41612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56da800640 1 -- 172.21.15.42:0/3308980432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d4160e60 msgr2=0x7f56d41612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56da800640 1 --2- 172.21.15.42:0/3308980432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d4160e60 0x7f56d41612e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56da800640 1 -- 172.21.15.42:0/3308980432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56bc0039f0 msgr2=0x7f56d414f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56da800640 1 --2- 172.21.15.42:0/3308980432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56bc0039f0 0x7f56d414f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56da800640 1 -- 172.21.15.42:0/3308980432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d4161ca0 con 0x7f56d4161820 2024-05-28T14:09:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56da800640 1 --2- 172.21.15.42:0/3308980432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56d4161820 0x7f56d41f8000 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f56d40d0b70 tx=0x7f56d000aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56ce000640 1 -- 172.21.15.42:0/3308980432 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d00078c0 con 0x7f56d4161820 2024-05-28T14:09:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56ce000640 1 -- 172.21.15.42:0/3308980432 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56d0018070 con 0x7f56d4161820 2024-05-28T14:09:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56ce000640 1 -- 172.21.15.42:0/3308980432 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0013630 con 0x7f56d4161820 2024-05-28T14:09:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56dc800640 1 -- 172.21.15.42:0/3308980432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d40d0780 con 0x7f56d4161820 2024-05-28T14:09:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.701+0000 7f56dc800640 1 -- 172.21.15.42:0/3308980432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d40d01c0 con 0x7f56d4161820 2024-05-28T14:09:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.705+0000 7f56dc800640 1 -- 172.21.15.42:0/3308980432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d4068a90 con 0x7f56d4161820 2024-05-28T14:09:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.705+0000 7f56ce000640 1 -- 172.21.15.42:0/3308980432 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7f56d00137d0 con 0x7f56d4161820 2024-05-28T14:09:45.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.705+0000 7f56ce000640 1 --2- 172.21.15.42:0/3308980432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56b80694f0 0x7f56b806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:45.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.705+0000 7f56ce000640 1 -- 172.21.15.42:0/3308980432 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7f56d0085e90 con 0x7f56d4161820 2024-05-28T14:09:45.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.705+0000 7f56d9e00640 1 --2- 172.21.15.42:0/3308980432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56b80694f0 0x7f56b806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:45.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.705+0000 7f56d9e00640 1 --2- 172.21.15.42:0/3308980432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56b80694f0 0x7f56b806b9b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009ae0 tx=0x7f56c80046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:45.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.709+0000 7f56ce000640 1 -- 172.21.15.42:0/3308980432 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f56d415ece0 con 0x7f56d4161820 2024-05-28T14:09:45.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.901+0000 7f56dc800640 1 -- 172.21.15.42:0/3308980432 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f56d4067910 con 0x7f56b80694f0 2024-05-28T14:09:45.891 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:09:45.909+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:09:45.891 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:09:45.909+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:09:45.891 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:09:45.909+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:09:45.891 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:09:45.909+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:09:45.892 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:09:45.909+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:09:45.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.945+0000 7f56ce000640 1 -- 172.21.15.42:0/3308980432 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f56d4067910 con 0x7f56b80694f0 2024-05-28T14:09:45.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.945+0000 7f56dc800640 1 -- 172.21.15.42:0/3308980432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56b80694f0 msgr2=0x7f56b806b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.945+0000 7f56dc800640 1 --2- 172.21.15.42:0/3308980432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56b80694f0 0x7f56b806b9b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009ae0 tx=0x7f56c80046d0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.945+0000 7f56dc800640 1 -- 172.21.15.42:0/3308980432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56d4161820 msgr2=0x7f56d41f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:45.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.945+0000 7f56dc800640 1 --2- 172.21.15.42:0/3308980432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56d4161820 0x7f56d41f8000 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f56d40d0b70 tx=0x7f56d000aee0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.945+0000 7f56dc800640 1 -- 172.21.15.42:0/3308980432 shutdown_connections 2024-05-28T14:09:45.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.945+0000 7f56dc800640 1 --2- 172.21.15.42:0/3308980432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f56b80694f0 0x7f56b806b9b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.945+0000 7f56dc800640 1 --2- 172.21.15.42:0/3308980432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56d4161820 0x7f56d41f8000 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.945+0000 7f56dc800640 1 --2- 172.21.15.42:0/3308980432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56d4160e60 0x7f56d41612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.945+0000 7f56dc800640 1 --2- 172.21.15.42:0/3308980432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56bc0039f0 0x7f56d414f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:45.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.945+0000 7f56dc800640 1 -- 172.21.15.42:0/3308980432 >> 172.21.15.42:0/3308980432 conn(0x7f56d40725b0 msgr2=0x7f56d40ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:45.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.945+0000 7f56dc800640 1 -- 172.21.15.42:0/3308980432 shutdown_connections 2024-05-28T14:09:45.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:45.945+0000 7f56dc800640 1 -- 172.21.15.42:0/3308980432 wait complete. 2024-05-28T14:09:45.949 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:09:46.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.109+0000 7fe670a00640 1 Processor -- start 2024-05-28T14:09:46.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.109+0000 7fe670a00640 1 -- start start 2024-05-28T14:09:46.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.109+0000 7fe670a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe66c0b0e80 0x7fe66c06cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:46.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.109+0000 7fe670a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66c09b160 0x7fe66c15e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:46.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.109+0000 7fe670a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe66c0d0770 con 0x7fe66c09b8e0 2024-05-28T14:09:46.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe670a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fe66c0d0a10 con 0x7fe66c0b0e80 2024-05-28T14:09:46.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe670a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c0d0cf0 con 0x7fe66c09b160 2024-05-28T14:09:46.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe669e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe66c0b0e80 0x7fe66c06cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:46.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe669e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe66c0b0e80 0x7fe66c06cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39072/0 (socket says 172.21.15.42:39072) 2024-05-28T14:09:46.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe669e00640 1 -- 172.21.15.42:0/2346305893 learned_addr learned my addr 172.21.15.42:0/2346305893 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:46.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe66a800640 1 --2- 172.21.15.42:0/2346305893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66c09b160 0x7fe66c15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:46.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe669e00640 1 -- 172.21.15.42:0/2346305893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66c09b160 msgr2=0x7fe66c15e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:46.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe669e00640 1 --2- 172.21.15.42:0/2346305893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66c09b160 0x7fe66c15e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:46.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe669e00640 1 -- 172.21.15.42:0/2346305893 >> v1:172.21.15.63:6789/0 conn(0x7fe66c09b8e0 legacy=0x7fe66c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:09:46.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe669e00640 1 -- 172.21.15.42:0/2346305893 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c0d0fd0 con 0x7fe66c0b0e80 2024-05-28T14:09:46.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe66a800640 1 --2- 172.21.15.42:0/2346305893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66c09b160 0x7fe66c15e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:09:46.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe669e00640 1 --2- 172.21.15.42:0/2346305893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66c0b0e80 0x7fe66c06cb00 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009870 tx=0x7fe65c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622539086af3413a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:46.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe669400640 1 -- 172.21.15.42:0/2346305893 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0b0070 con 0x7fe66c0b0e80 2024-05-28T14:09:46.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe669400640 1 -- 172.21.15.42:0/2346305893 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65c0ab600 con 0x7fe66c0b0e80 2024-05-28T14:09:46.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.113+0000 7fe669400640 1 -- 172.21.15.42:0/2346305893 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0ab9d0 con 0x7fe66c0b0e80 2024-05-28T14:09:46.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 -- 172.21.15.42:0/2346305893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66c0b0e80 msgr2=0x7fe66c06cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:46.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 --2- 172.21.15.42:0/2346305893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66c0b0e80 0x7fe66c06cb00 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009870 tx=0x7fe65c01c520 comp rx=0 tx=0).stop 2024-05-28T14:09:46.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 -- 172.21.15.42:0/2346305893 shutdown_connections 2024-05-28T14:09:46.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 --2- 172.21.15.42:0/2346305893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66c09b160 0x7fe66c15e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:46.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 --2- 172.21.15.42:0/2346305893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66c0b0e80 0x7fe66c06cb00 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:46.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 -- 172.21.15.42:0/2346305893 >> 172.21.15.42:0/2346305893 conn(0x7fe66c0b3f90 msgr2=0x7fe66c0b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:46.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 -- 172.21.15.42:0/2346305893 shutdown_connections 2024-05-28T14:09:46.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 -- 172.21.15.42:0/2346305893 wait complete. 2024-05-28T14:09:46.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 Processor -- start 2024-05-28T14:09:46.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 -- start start 2024-05-28T14:09:46.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66c09b160 0x7fe66c0b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66c09b8e0 0x7fe66c158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe66c0b0e80 0x7fe66c161800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c0af350 con 0x7fe66c09b8e0 2024-05-28T14:09:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c0af1d0 con 0x7fe66c0b0e80 2024-05-28T14:09:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe670a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c0af4d0 con 0x7fe66c09b160 2024-05-28T14:09:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe663e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66c09b8e0 0x7fe66c158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe663e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66c09b8e0 0x7fe66c158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39088/0 (socket says 172.21.15.42:39088) 2024-05-28T14:09:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe663e00640 1 -- 172.21.15.42:0/4110907171 learned_addr learned my addr 172.21.15.42:0/4110907171 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:09:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe66a800640 1 --2- 172.21.15.42:0/4110907171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe66c0b0e80 0x7fe66c161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe669e00640 1 --2- 172.21.15.42:0/4110907171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66c09b160 0x7fe66c0b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe663e00640 1 -- 172.21.15.42:0/4110907171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66c09b160 msgr2=0x7fe66c0b79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.117+0000 7fe663e00640 1 --2- 172.21.15.42:0/4110907171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66c09b160 0x7fe66c0b79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:46.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.121+0000 7fe663e00640 1 -- 172.21.15.42:0/4110907171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe66c0b0e80 msgr2=0x7fe66c161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:46.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.121+0000 7fe663e00640 1 --2- 172.21.15.42:0/4110907171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe66c0b0e80 0x7fe66c161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:46.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.121+0000 7fe663e00640 1 -- 172.21.15.42:0/4110907171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c1594e0 con 0x7fe66c09b8e0 2024-05-28T14:09:46.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.121+0000 7fe663e00640 1 --2- 172.21.15.42:0/4110907171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66c09b8e0 0x7fe66c158fa0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe65400c450 tx=0x7fe65400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:46.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.121+0000 7fe662000640 1 -- 172.21.15.42:0/4110907171 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe654012830 con 0x7fe66c09b8e0 2024-05-28T14:09:46.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.121+0000 7fe662000640 1 -- 172.21.15.42:0/4110907171 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe654017070 con 0x7fe66c09b8e0 2024-05-28T14:09:46.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.121+0000 7fe670a00640 1 -- 172.21.15.42:0/4110907171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c15c870 con 0x7fe66c09b8e0 2024-05-28T14:09:46.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.121+0000 7fe662000640 1 -- 172.21.15.42:0/4110907171 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe654012c30 con 0x7fe66c09b8e0 2024-05-28T14:09:46.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.121+0000 7fe670a00640 1 -- 172.21.15.42:0/4110907171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c15ce30 con 0x7fe66c09b8e0 2024-05-28T14:09:46.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.121+0000 7fe670a00640 1 -- 172.21.15.42:0/4110907171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe66c0a0550 con 0x7fe66c09b8e0 2024-05-28T14:09:46.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.121+0000 7fe662000640 1 -- 172.21.15.42:0/4110907171 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 89568+0+0 (secure 0 0 0) 0x7fe65401c460 con 0x7fe66c09b8e0 2024-05-28T14:09:46.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.121+0000 7fe662000640 1 --2- 172.21.15.42:0/4110907171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6480694f0 0x7fe64806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:09:46.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.121+0000 7fe662000640 1 -- 172.21.15.42:0/4110907171 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5737+0+0 (secure 0 0 0) 0x7fe654085490 con 0x7fe66c09b8e0 2024-05-28T14:09:46.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.121+0000 7fe669e00640 1 --2- 172.21.15.42:0/4110907171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6480694f0 0x7fe64806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:09:46.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.125+0000 7fe669e00640 1 --2- 172.21.15.42:0/4110907171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6480694f0 0x7fe64806b9b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0ad000 tx=0x7fe65c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:09:46.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.129+0000 7fe662000640 1 -- 172.21.15.42:0/4110907171 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe654056b00 con 0x7fe66c09b8e0 2024-05-28T14:09:46.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.321+0000 7fe670a00640 1 -- 172.21.15.42:0/4110907171 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fe66c04fdd0 con 0x7fe6480694f0 2024-05-28T14:09:46.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.325+0000 7fe662000640 1 -- 172.21.15.42:0/4110907171 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fe66c04fdd0 con 0x7fe6480694f0 2024-05-28T14:09:46.308 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/9c6ae837-11be-4379-99b1-7bf85d2f78ec 2024-05-28T14:09:46.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.325+0000 7fe670a00640 1 -- 172.21.15.42:0/4110907171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6480694f0 msgr2=0x7fe64806b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:46.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.325+0000 7fe670a00640 1 --2- 172.21.15.42:0/4110907171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6480694f0 0x7fe64806b9b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0ad000 tx=0x7fe65c008040 comp rx=0 tx=0).stop 2024-05-28T14:09:46.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.325+0000 7fe670a00640 1 -- 172.21.15.42:0/4110907171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66c09b8e0 msgr2=0x7fe66c158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:09:46.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.325+0000 7fe670a00640 1 --2- 172.21.15.42:0/4110907171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66c09b8e0 0x7fe66c158fa0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe65400c450 tx=0x7fe65400c910 comp rx=0 tx=0).stop 2024-05-28T14:09:46.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.325+0000 7fe670a00640 1 -- 172.21.15.42:0/4110907171 shutdown_connections 2024-05-28T14:09:46.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.325+0000 7fe670a00640 1 --2- 172.21.15.42:0/4110907171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6480694f0 0x7fe64806b9b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:46.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.325+0000 7fe670a00640 1 --2- 172.21.15.42:0/4110907171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe66c0b0e80 0x7fe66c161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:46.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.325+0000 7fe670a00640 1 --2- 172.21.15.42:0/4110907171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe66c09b8e0 0x7fe66c158fa0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:46.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.325+0000 7fe670a00640 1 --2- 172.21.15.42:0/4110907171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe66c09b160 0x7fe66c0b79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:09:46.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.325+0000 7fe670a00640 1 -- 172.21.15.42:0/4110907171 >> 172.21.15.42:0/4110907171 conn(0x7fe66c0b3f90 msgr2=0x7fe66c06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:09:46.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.325+0000 7fe670a00640 1 -- 172.21.15.42:0/4110907171 shutdown_connections 2024-05-28T14:09:46.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:09:46.325+0000 7fe670a00640 1 -- 172.21.15.42:0/4110907171 wait complete. 2024-05-28T14:09:46.331 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/9c6ae837-11be-4379-99b1-7bf85d2f78ec 2024-05-28T14:09:46.331 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:09:46.331 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:09:46.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:46.340 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9c6ae837-11be-4379-99b1-7bf85d2f78ec/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:09:50.409 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:09:50.414 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:09:53.534 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:09:53.535 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:09:53.535 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.12883 s, 151 MB/s 2024-05-28T14:09:53.536 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:09:53.536 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:09:53.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:09:53.545 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9c6ae837-11be-4379-99b1-7bf85d2f78ec/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:10:00.439 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:10:00.439 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:10:00.439 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.8369 s, 157 MB/s 2024-05-28T14:10:00.440 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:10:00.441 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:10:00.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:10:00.449 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9c6ae837-11be-4379-99b1-7bf85d2f78ec/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:10:07.366 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:10:07.367 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:10:07.367 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.8595 s, 157 MB/s 2024-05-28T14:10:07.368 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-28T14:10:07.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7ff007c00640 1 Processor -- start 2024-05-28T14:10:07.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7ff007c00640 1 -- start start 2024-05-28T14:10:07.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7ff007c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff00015b350 0x7ff00006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:07.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7ff007c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff000072d40 0x7ff0000d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:07.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7ff007c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0000ade40 con 0x7ff0000737c0 2024-05-28T14:10:07.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7ff007c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff0000ae9f0 con 0x7ff000072d40 2024-05-28T14:10:07.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7ff007c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff0000ae0c0 con 0x7ff00015b350 2024-05-28T14:10:07.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7feff7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff000072d40 0x7ff0000d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:07.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7ff005200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff00015b350 0x7ff00006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:07.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7ff005200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff00015b350 0x7ff00006dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:40322/0 (socket says 172.21.15.42:40322) 2024-05-28T14:10:07.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7ff005200640 1 -- 172.21.15.42:0/92483462 learned_addr learned my addr 172.21.15.42:0/92483462 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:07.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7feff7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff000072d40 0x7ff0000d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34576/0 (socket says 172.21.15.42:34576) 2024-05-28T14:10:07.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7ff005c00640 1 --1- 172.21.15.42:0/92483462 >> v1:172.21.15.63:6789/0 conn(0x7ff0000737c0 0x7ff0000aee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:46638/0 (socket says 172.21.15.42:46638) 2024-05-28T14:10:07.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7feff7e00640 1 -- 172.21.15.42:0/92483462 >> v2:172.21.15.63:3300/0 conn(0x7ff00015b350 msgr2=0x7ff00006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:07.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7feff7e00640 1 --2- 172.21.15.42:0/92483462 >> v2:172.21.15.63:3300/0 conn(0x7ff00015b350 0x7ff00006dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7feff7e00640 1 -- 172.21.15.42:0/92483462 >> v1:172.21.15.63:6789/0 conn(0x7ff0000737c0 legacy=0x7ff0000aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:07.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7feff7e00640 1 -- 172.21.15.42:0/92483462 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0000ae3a0 con 0x7ff000072d40 2024-05-28T14:10:07.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7ff005200640 1 --2- 172.21.15.42:0/92483462 >> v2:172.21.15.63:3300/0 conn(0x7ff00015b350 0x7ff00006dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:10:07.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.532+0000 7fefffe00640 1 -- 172.21.15.42:0/92483462 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3077908753 0 0) 0x7feff80014a0 con 0x7ff0000737c0 2024-05-28T14:10:07.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.536+0000 7feff7e00640 1 --2- 172.21.15.42:0/92483462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff000072d40 0x7ff0000d06f0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7ff0000d13c0 tx=0x7fefe801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68af02f6d11cb260 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:07.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.536+0000 7fefffe00640 1 -- 172.21.15.42:0/92483462 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe80b0070 con 0x7ff000072d40 2024-05-28T14:10:07.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.536+0000 7fefffe00640 1 -- 172.21.15.42:0/92483462 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe80ab600 con 0x7ff000072d40 2024-05-28T14:10:07.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.536+0000 7fefffe00640 1 -- 172.21.15.42:0/92483462 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe80ab9d0 con 0x7ff000072d40 2024-05-28T14:10:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.536+0000 7ff007c00640 1 -- 172.21.15.42:0/92483462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff000072d40 msgr2=0x7ff0000d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.536+0000 7ff007c00640 1 --2- 172.21.15.42:0/92483462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff000072d40 0x7ff0000d06f0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7ff0000d13c0 tx=0x7fefe801c3f0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.536+0000 7ff007c00640 1 -- 172.21.15.42:0/92483462 shutdown_connections 2024-05-28T14:10:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.536+0000 7ff007c00640 1 --2- 172.21.15.42:0/92483462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff000072d40 0x7ff0000d06f0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.536+0000 7ff007c00640 1 --2- 172.21.15.42:0/92483462 >> v2:172.21.15.63:3300/0 conn(0x7ff00015b350 0x7ff00006dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.536+0000 7ff007c00640 1 -- 172.21.15.42:0/92483462 >> 172.21.15.42:0/92483462 conn(0x7ff00006bdf0 msgr2=0x7ff00006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:07.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.536+0000 7ff007c00640 1 -- 172.21.15.42:0/92483462 shutdown_connections 2024-05-28T14:10:07.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.536+0000 7ff007c00640 1 -- 172.21.15.42:0/92483462 wait complete. 2024-05-28T14:10:07.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7ff007c00640 1 Processor -- start 2024-05-28T14:10:07.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7ff007c00640 1 -- start start 2024-05-28T14:10:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7ff007c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff000072d40 0x7ff00009de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7ff007c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0000737c0 0x7ff00006f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7ff007c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff00015b350 0x7ff00014b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7ff007c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff0000afb20 con 0x7ff0000737c0 2024-05-28T14:10:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7ff007c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff0000af9a0 con 0x7ff00015b350 2024-05-28T14:10:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7ff007c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff000168b00 con 0x7ff000072d40 2024-05-28T14:10:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7feff7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0000737c0 0x7ff00006f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7feff7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0000737c0 0x7ff00006f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34588/0 (socket says 172.21.15.42:34588) 2024-05-28T14:10:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7feff7e00640 1 -- 172.21.15.42:0/1124739411 learned_addr learned my addr 172.21.15.42:0/1124739411 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7ff005200640 1 --2- 172.21.15.42:0/1124739411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff000072d40 0x7ff00009de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7ff005c00640 1 --2- 172.21.15.42:0/1124739411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff00015b350 0x7ff00014b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7feff7e00640 1 -- 172.21.15.42:0/1124739411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff000072d40 msgr2=0x7ff00009de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7feff7e00640 1 --2- 172.21.15.42:0/1124739411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff000072d40 0x7ff00009de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7feff7e00640 1 -- 172.21.15.42:0/1124739411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff00015b350 msgr2=0x7ff00014b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7feff7e00640 1 --2- 172.21.15.42:0/1124739411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff00015b350 0x7ff00014b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7feff7e00640 1 -- 172.21.15.42:0/1124739411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00015c3b0 con 0x7ff0000737c0 2024-05-28T14:10:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7feff7e00640 1 --2- 172.21.15.42:0/1124739411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0000737c0 0x7ff00006f0a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fefe801a020 tx=0x7fefe8019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:07.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7feffd600640 1 -- 172.21.15.42:0/1124739411 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe80b0040 con 0x7ff0000737c0 2024-05-28T14:10:07.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7feffd600640 1 -- 172.21.15.42:0/1124739411 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe80b9070 con 0x7ff0000737c0 2024-05-28T14:10:07.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7ff007c00640 1 -- 172.21.15.42:0/1124739411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00015c630 con 0x7ff0000737c0 2024-05-28T14:10:07.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7feffd600640 1 -- 172.21.15.42:0/1124739411 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe80b4650 con 0x7ff0000737c0 2024-05-28T14:10:07.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7ff007c00640 1 -- 172.21.15.42:0/1124739411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00015cbf0 con 0x7ff0000737c0 2024-05-28T14:10:07.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.540+0000 7ff007c00640 1 -- 172.21.15.42:0/1124739411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff000068a90 con 0x7ff0000737c0 2024-05-28T14:10:07.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.544+0000 7feffd600640 1 -- 172.21.15.42:0/1124739411 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 89619+0+0 (secure 0 0 0) 0x7fefe80b47f0 con 0x7ff0000737c0 2024-05-28T14:10:07.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.544+0000 7feffd600640 1 --2- 172.21.15.42:0/1124739411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fefdc069580 0x7fefdc06ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:07.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.544+0000 7feffd600640 1 -- 172.21.15.42:0/1124739411 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7fefe811f660 con 0x7ff0000737c0 2024-05-28T14:10:07.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.544+0000 7ff005200640 1 --2- 172.21.15.42:0/1124739411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fefdc069580 0x7fefdc06ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:07.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.544+0000 7ff005200640 1 --2- 172.21.15.42:0/1124739411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fefdc069580 0x7fefdc06ba40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7feff0006cb0 tx=0x7feff0009240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:07.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.548+0000 7feffd600640 1 -- 172.21.15.42:0/1124739411 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff00015c3b0 con 0x7ff0000737c0 2024-05-28T14:10:07.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.748+0000 7ff007c00640 1 -- 172.21.15.42:0/1124739411 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7ff000072b20 con 0x7fefdc069580 2024-05-28T14:10:07.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.760+0000 7feffd600640 1 -- 172.21.15.42:0/1124739411 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff000072b20 con 0x7fefdc069580 2024-05-28T14:10:07.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.764+0000 7ff007c00640 1 -- 172.21.15.42:0/1124739411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fefdc069580 msgr2=0x7fefdc06ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:07.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.764+0000 7ff007c00640 1 --2- 172.21.15.42:0/1124739411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fefdc069580 0x7fefdc06ba40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7feff0006cb0 tx=0x7feff0009240 comp rx=0 tx=0).stop 2024-05-28T14:10:07.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.764+0000 7ff007c00640 1 -- 172.21.15.42:0/1124739411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0000737c0 msgr2=0x7ff00006f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:07.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.764+0000 7ff007c00640 1 --2- 172.21.15.42:0/1124739411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0000737c0 0x7ff00006f0a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fefe801a020 tx=0x7fefe8019190 comp rx=0 tx=0).stop 2024-05-28T14:10:07.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.764+0000 7ff007c00640 1 -- 172.21.15.42:0/1124739411 shutdown_connections 2024-05-28T14:10:07.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.764+0000 7ff007c00640 1 --2- 172.21.15.42:0/1124739411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fefdc069580 0x7fefdc06ba40 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.764+0000 7ff007c00640 1 --2- 172.21.15.42:0/1124739411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff00015b350 0x7ff00014b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.764+0000 7ff007c00640 1 --2- 172.21.15.42:0/1124739411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0000737c0 0x7ff00006f0a0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.764+0000 7ff007c00640 1 --2- 172.21.15.42:0/1124739411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff000072d40 0x7ff00009de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.764+0000 7ff007c00640 1 -- 172.21.15.42:0/1124739411 >> 172.21.15.42:0/1124739411 conn(0x7ff00006bdf0 msgr2=0x7ff0000afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.764+0000 7ff007c00640 1 -- 172.21.15.42:0/1124739411 shutdown_connections 2024-05-28T14:10:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.764+0000 7ff007c00640 1 -- 172.21.15.42:0/1124739411 wait complete. 2024-05-28T14:10:07.767 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:10:07.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8479c00640 1 Processor -- start 2024-05-28T14:10:07.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8479c00640 1 -- start start 2024-05-28T14:10:07.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8479c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f847406c5f0 0x7f84740cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:07.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8479c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f847406b9f0 0x7f84740d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:07.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8479c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84740d0c80 0x7f847415cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:07.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8479c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f84740ce100 con 0x7f847406c5f0 2024-05-28T14:10:07.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8479c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f84740cffd0 con 0x7f847406b9f0 2024-05-28T14:10:07.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8479c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f84740ce340 con 0x7f84740d0c80 2024-05-28T14:10:07.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8473e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f847406c5f0 0x7f84740cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:07.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8473e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f847406c5f0 0x7f84740cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34592/0 (socket says 172.21.15.42:34592) 2024-05-28T14:10:07.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8473e00640 1 -- 172.21.15.42:0/1499155311 learned_addr learned my addr 172.21.15.42:0/1499155311 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:07.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8479200640 1 --2- 172.21.15.42:0/1499155311 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84740d0c80 0x7f847415cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:07.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f846b400640 1 --2- 172.21.15.42:0/1499155311 >> v2:172.21.15.63:3300/0 conn(0x7f847406b9f0 0x7f84740d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:07.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8473e00640 1 -- 172.21.15.42:0/1499155311 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84740d0c80 msgr2=0x7f847415cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:07.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8473e00640 1 --2- 172.21.15.42:0/1499155311 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84740d0c80 0x7f847415cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8473e00640 1 -- 172.21.15.42:0/1499155311 >> v2:172.21.15.63:3300/0 conn(0x7f847406b9f0 msgr2=0x7f84740d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:07.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8473e00640 1 --2- 172.21.15.42:0/1499155311 >> v2:172.21.15.63:3300/0 conn(0x7f847406b9f0 0x7f84740d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8473e00640 1 -- 172.21.15.42:0/1499155311 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84740ce620 con 0x7f847406c5f0 2024-05-28T14:10:07.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.928+0000 7f8479200640 1 --2- 172.21.15.42:0/1499155311 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84740d0c80 0x7f847415cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:10:07.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.932+0000 7f8473e00640 1 --2- 172.21.15.42:0/1499155311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f847406c5f0 0x7f84740cdbc0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f84740721a0 tx=0x7f846401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5524134e15e8fe server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:07.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.932+0000 7f8473400640 1 -- 172.21.15.42:0/1499155311 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84640b0070 con 0x7f847406c5f0 2024-05-28T14:10:07.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.932+0000 7f8473400640 1 -- 172.21.15.42:0/1499155311 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84640a4e50 con 0x7f847406c5f0 2024-05-28T14:10:07.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.932+0000 7f8473400640 1 -- 172.21.15.42:0/1499155311 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84640ab820 con 0x7f847406c5f0 2024-05-28T14:10:07.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.932+0000 7f8479c00640 1 -- 172.21.15.42:0/1499155311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f847406c5f0 msgr2=0x7f84740cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:07.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.932+0000 7f8479c00640 1 --2- 172.21.15.42:0/1499155311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f847406c5f0 0x7f84740cdbc0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f84740721a0 tx=0x7f846401c400 comp rx=0 tx=0).stop 2024-05-28T14:10:07.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.932+0000 7f8479c00640 1 -- 172.21.15.42:0/1499155311 shutdown_connections 2024-05-28T14:10:07.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.932+0000 7f8479c00640 1 --2- 172.21.15.42:0/1499155311 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84740d0c80 0x7f847415cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.932+0000 7f8479c00640 1 --2- 172.21.15.42:0/1499155311 >> v2:172.21.15.63:3300/0 conn(0x7f847406b9f0 0x7f84740d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.932+0000 7f8479c00640 1 --2- 172.21.15.42:0/1499155311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f847406c5f0 0x7f84740cdbc0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.932+0000 7f8479c00640 1 -- 172.21.15.42:0/1499155311 >> 172.21.15.42:0/1499155311 conn(0x7f84740b2d70 msgr2=0x7f84740b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:07.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.932+0000 7f8479c00640 1 -- 172.21.15.42:0/1499155311 shutdown_connections 2024-05-28T14:10:07.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.932+0000 7f8479c00640 1 -- 172.21.15.42:0/1499155311 wait complete. 2024-05-28T14:10:07.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.932+0000 7f8479c00640 1 Processor -- start 2024-05-28T14:10:07.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8479c00640 1 -- start start 2024-05-28T14:10:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8479c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f847406b9f0 0x7f84741551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8479c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f847406c5f0 0x7f847414b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8479c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f847406ec30 0x7f8474155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8479c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f84740cd690 con 0x7f847406b9f0 2024-05-28T14:10:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8479c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f84740cd510 con 0x7f847406ec30 2024-05-28T14:10:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8479c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f84740cedb0 con 0x7f847406c5f0 2024-05-28T14:10:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8473e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f847406b9f0 0x7f84741551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8473e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f847406b9f0 0x7f84741551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34604/0 (socket says 172.21.15.42:34604) 2024-05-28T14:10:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8473e00640 1 -- 172.21.15.42:0/975967771 learned_addr learned my addr 172.21.15.42:0/975967771 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f846b400640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f847406c5f0 0x7f847414b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8479200640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f847406ec30 0x7f8474155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8473e00640 1 -- 172.21.15.42:0/975967771 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f847406c5f0 msgr2=0x7f847414b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:07.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8473e00640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f847406c5f0 0x7f847414b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8473e00640 1 -- 172.21.15.42:0/975967771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f847406ec30 msgr2=0x7f8474155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:07.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8473e00640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f847406ec30 0x7f8474155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:07.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8473e00640 1 -- 172.21.15.42:0/975967771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84740b0250 con 0x7f847406b9f0 2024-05-28T14:10:07.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8479200640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f847406ec30 0x7f8474155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:10:07.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8473e00640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f847406b9f0 0x7f84741551b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f84640ae040 tx=0x7f84640191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:07.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8470c00640 1 -- 172.21.15.42:0/975967771 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84640b0040 con 0x7f847406b9f0 2024-05-28T14:10:07.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8470c00640 1 -- 172.21.15.42:0/975967771 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84640b9070 con 0x7f847406b9f0 2024-05-28T14:10:07.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8470c00640 1 -- 172.21.15.42:0/975967771 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84640b4440 con 0x7f847406b9f0 2024-05-28T14:10:07.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8479c00640 1 -- 172.21.15.42:0/975967771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847414c940 con 0x7f847406b9f0 2024-05-28T14:10:07.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8479c00640 1 -- 172.21.15.42:0/975967771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f847414cf00 con 0x7f847406b9f0 2024-05-28T14:10:07.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.936+0000 7f8479c00640 1 -- 172.21.15.42:0/975967771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f847404fcf0 con 0x7f847406b9f0 2024-05-28T14:10:07.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.940+0000 7f8470c00640 1 -- 172.21.15.42:0/975967771 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 89619+0+0 (secure 0 0 0) 0x7f84640b45e0 con 0x7f847406b9f0 2024-05-28T14:10:07.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.940+0000 7f8470c00640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f84500695b0 0x7f845006ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:07.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.940+0000 7f8470c00640 1 -- 172.21.15.42:0/975967771 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7f846411f540 con 0x7f847406b9f0 2024-05-28T14:10:07.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.940+0000 7f846b400640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f84500695b0 0x7f845006ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:07.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.940+0000 7f846b400640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f84500695b0 0x7f845006ba70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f84740cc4f0 tx=0x7f845c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:07.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:07.944+0000 7f8470c00640 1 -- 172.21.15.42:0/975967771 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8464016110 con 0x7f847406b9f0 2024-05-28T14:10:08.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:08.132+0000 7f8479c00640 1 -- 172.21.15.42:0/975967771 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f84740b1e40 con 0x7f84500695b0 2024-05-28T14:10:08.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:08.136+0000 7f8470c00640 1 -- 172.21.15.42:0/975967771 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f84740b1e40 con 0x7f84500695b0 2024-05-28T14:10:08.122 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/9c6ae837-11be-4379-99b1-7bf85d2f78ec 2024-05-28T14:10:08.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:08.140+0000 7f8479c00640 1 -- 172.21.15.42:0/975967771 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f84500695b0 msgr2=0x7f845006ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:08.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:08.140+0000 7f8479c00640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f84500695b0 0x7f845006ba70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f84740cc4f0 tx=0x7f845c008040 comp rx=0 tx=0).stop 2024-05-28T14:10:08.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:08.140+0000 7f8479c00640 1 -- 172.21.15.42:0/975967771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f847406b9f0 msgr2=0x7f84741551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:08.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:08.140+0000 7f8479c00640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f847406b9f0 0x7f84741551b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f84640ae040 tx=0x7f84640191a0 comp rx=0 tx=0).stop 2024-05-28T14:10:08.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:08.140+0000 7f8479c00640 1 -- 172.21.15.42:0/975967771 shutdown_connections 2024-05-28T14:10:08.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:08.140+0000 7f8479c00640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f84500695b0 0x7f845006ba70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:08.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:08.140+0000 7f8479c00640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f847406ec30 0x7f8474155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:08.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:08.140+0000 7f8479c00640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f847406c5f0 0x7f847414b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:08.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:08.140+0000 7f8479c00640 1 --2- 172.21.15.42:0/975967771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f847406b9f0 0x7f84741551b0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:08.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:08.140+0000 7f8479c00640 1 -- 172.21.15.42:0/975967771 >> 172.21.15.42:0/975967771 conn(0x7f84740b2d70 msgr2=0x7f847415a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:08.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:08.140+0000 7f8479c00640 1 -- 172.21.15.42:0/975967771 shutdown_connections 2024-05-28T14:10:08.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:08.140+0000 7f8479c00640 1 -- 172.21.15.42:0/975967771 wait complete. 2024-05-28T14:10:08.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:10:08.144 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9c6ae837-11be-4379-99b1-7bf85d2f78ec 2024-05-28T14:10:08.144 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:10:08.157 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/9c6ae837-11be-4379-99b1-7bf85d2f78ec 2024-05-28T14:10:08.157 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3138256896" 2024-05-28T14:10:08.158 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:08.158 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-28T14:10:10.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:10:10.160 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9c6ae837-11be-4379-99b1-7bf85d2f78ec 2024-05-28T14:10:10.160 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:10:10.175 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/9c6ae837-11be-4379-99b1-7bf85d2f78ec 2024-05-28T14:10:10.175 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3138256896" 2024-05-28T14:10:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:10.176 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-28T14:10:12.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:10:12.177 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9c6ae837-11be-4379-99b1-7bf85d2f78ec 2024-05-28T14:10:12.177 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:10:12.192 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/9c6ae837-11be-4379-99b1-7bf85d2f78ec 2024-05-28T14:10:12.192 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3221225472" 2024-05-28T14:10:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:12.193 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-28T14:10:12.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-05-28T14:10:12.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.408+0000 7fd471e00640 1 Processor -- start 2024-05-28T14:10:12.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.408+0000 7fd471e00640 1 -- start start 2024-05-28T14:10:12.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.408+0000 7fd471e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd46c06ee20 0x7fd46c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:12.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.408+0000 7fd471e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd46c0afe40 con 0x7fd46c15b9b0 2024-05-28T14:10:12.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd471e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd46c15c600 con 0x7fd46c144f80 2024-05-28T14:10:12.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd471e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd46c06f7c0 con 0x7fd46c06ee20 2024-05-28T14:10:12.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46b400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd46c15b9b0 0x7fd46c15be40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34158/0 (socket says 172.21.15.42:34158) 2024-05-28T14:10:12.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46b400640 1 -- 172.21.15.42:0/3051710803 learned_addr learned my addr 172.21.15.42:0/3051710803 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:12.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd462000640 1 --2- 172.21.15.42:0/3051710803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd46c06ee20 0x7fd46c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:12.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 -- 172.21.15.42:0/3051710803 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 699986831 0 0) 0x7fd46c0afe40 con 0x7fd46c15b9b0 2024-05-28T14:10:12.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 -- 172.21.15.42:0/3051710803 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd450003530 con 0x7fd46c15b9b0 2024-05-28T14:10:12.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 -- 172.21.15.42:0/3051710803 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3572869730 0 0) 0x7fd450003530 con 0x7fd46c15b9b0 2024-05-28T14:10:12.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 -- 172.21.15.42:0/3051710803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd46c06ee20 msgr2=0x7fd46c06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:12.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 --2- 172.21.15.42:0/3051710803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd46c06ee20 0x7fd46c06f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 -- 172.21.15.42:0/3051710803 >> v1:172.21.15.63:6789/0 conn(0x7fd46c144f80 legacy=0x7fd46c072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:12.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 -- 172.21.15.42:0/3051710803 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46c15c8e0 con 0x7fd46c15b9b0 2024-05-28T14:10:12.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 -- 172.21.15.42:0/3051710803 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fd464001a00 con 0x7fd46c15b9b0 2024-05-28T14:10:12.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 -- 172.21.15.42:0/3051710803 >> v1:172.21.15.42:6789/0 conn(0x7fd46c15b9b0 legacy=0x7fd46c15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:12.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 --2- 172.21.15.42:0/3051710803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd46c06ee20 0x7fd4500039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:12.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 --2- 172.21.15.42:0/3051710803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd450003f30 0x7fd450004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:12.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 --2- 172.21.15.42:0/3051710803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd450004d50 0x7fd4500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:12.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 -- 172.21.15.42:0/3051710803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd45000b9d0 con 0x7fd450004d50 2024-05-28T14:10:12.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 -- 172.21.15.42:0/3051710803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd45000bb50 con 0x7fd450003f30 2024-05-28T14:10:12.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46a000640 1 -- 172.21.15.42:0/3051710803 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd45000bd50 con 0x7fd46c06ee20 2024-05-28T14:10:12.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46b400640 1 --2- 172.21.15.42:0/3051710803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd450004d50 0x7fd4500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:12.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46aa00640 1 --2- 172.21.15.42:0/3051710803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd46c06ee20 0x7fd4500039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:12.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd462000640 1 --2- 172.21.15.42:0/3051710803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd450003f30 0x7fd450004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:12.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46aa00640 1 -- 172.21.15.42:0/3051710803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd450003f30 msgr2=0x7fd450004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:12.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46aa00640 1 --2- 172.21.15.42:0/3051710803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd450003f30 0x7fd450004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46aa00640 1 -- 172.21.15.42:0/3051710803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd450004d50 msgr2=0x7fd4500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:12.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46aa00640 1 --2- 172.21.15.42:0/3051710803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd450004d50 0x7fd4500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46aa00640 1 -- 172.21.15.42:0/3051710803 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46c15c8e0 con 0x7fd46c06ee20 2024-05-28T14:10:12.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.412+0000 7fd46b400640 1 --2- 172.21.15.42:0/3051710803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd450004d50 0x7fd4500051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:10:12.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.416+0000 7fd46aa00640 1 --2- 172.21.15.42:0/3051710803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd46c06ee20 0x7fd4500039f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd45c001910 tx=0x7fd45c01c580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39772a37970d58c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:12.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.416+0000 7fd46a000640 1 -- 172.21.15.42:0/3051710803 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c001510 con 0x7fd46c06ee20 2024-05-28T14:10:12.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.416+0000 7fd46a000640 1 -- 172.21.15.42:0/3051710803 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd45c0af070 con 0x7fd46c06ee20 2024-05-28T14:10:12.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.416+0000 7fd46a000640 1 -- 172.21.15.42:0/3051710803 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0aa880 con 0x7fd46c06ee20 2024-05-28T14:10:12.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.416+0000 7fd471e00640 1 -- 172.21.15.42:0/3051710803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd46c06ee20 msgr2=0x7fd4500039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:12.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.416+0000 7fd471e00640 1 --2- 172.21.15.42:0/3051710803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd46c06ee20 0x7fd4500039f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd45c001910 tx=0x7fd45c01c580 comp rx=0 tx=0).stop 2024-05-28T14:10:12.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.416+0000 7fd46b400640 1 -- 172.21.15.42:0/3051710803 reap_dead start 2024-05-28T14:10:12.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.416+0000 7fd471e00640 1 -- 172.21.15.42:0/3051710803 shutdown_connections 2024-05-28T14:10:12.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.416+0000 7fd471e00640 1 -- 172.21.15.42:0/3051710803 >> 172.21.15.42:0/3051710803 conn(0x7fd46c0b21f0 msgr2=0x7fd46c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:12.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.416+0000 7fd471e00640 1 -- 172.21.15.42:0/3051710803 shutdown_connections 2024-05-28T14:10:12.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.416+0000 7fd471e00640 1 -- 172.21.15.42:0/3051710803 wait complete. 2024-05-28T14:10:12.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.416+0000 7fd471e00640 1 Processor -- start 2024-05-28T14:10:12.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.416+0000 7fd471e00640 1 -- start start 2024-05-28T14:10:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd471e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd46c06ee20 0x7fd46c15b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd471e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd46c15b5a0 0x7fd46c158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd471e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd46c13afc0 0x7fd46c159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd471e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c072160 con 0x7fd46c15b5a0 2024-05-28T14:10:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd471e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c0724d0 con 0x7fd46c06ee20 2024-05-28T14:10:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd471e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd46c06f280 con 0x7fd46c13afc0 2024-05-28T14:10:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd462000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd46c15b5a0 0x7fd46c158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd462000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd46c15b5a0 0x7fd46c158e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34620/0 (socket says 172.21.15.42:34620) 2024-05-28T14:10:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd462000640 1 -- 172.21.15.42:0/2503562027 learned_addr learned my addr 172.21.15.42:0/2503562027 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd46b400640 1 --2- 172.21.15.42:0/2503562027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd46c13afc0 0x7fd46c159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd46aa00640 1 --2- 172.21.15.42:0/2503562027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd46c06ee20 0x7fd46c15b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd462000640 1 -- 172.21.15.42:0/2503562027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd46c13afc0 msgr2=0x7fd46c159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:12.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd462000640 1 --2- 172.21.15.42:0/2503562027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd46c13afc0 0x7fd46c159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd462000640 1 -- 172.21.15.42:0/2503562027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd46c06ee20 msgr2=0x7fd46c15b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:12.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd462000640 1 --2- 172.21.15.42:0/2503562027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd46c06ee20 0x7fd46c15b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd462000640 1 -- 172.21.15.42:0/2503562027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46c0ae760 con 0x7fd46c15b5a0 2024-05-28T14:10:12.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd462000640 1 --2- 172.21.15.42:0/2503562027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd46c15b5a0 0x7fd46c158e50 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fd454011960 tx=0x7fd454011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:12.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd463e00640 1 -- 172.21.15.42:0/2503562027 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45400faf0 con 0x7fd46c15b5a0 2024-05-28T14:10:12.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd463e00640 1 -- 172.21.15.42:0/2503562027 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd45400fc90 con 0x7fd46c15b5a0 2024-05-28T14:10:12.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd471e00640 1 -- 172.21.15.42:0/2503562027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46c0aea40 con 0x7fd46c15b5a0 2024-05-28T14:10:12.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd463e00640 1 -- 172.21.15.42:0/2503562027 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454014440 con 0x7fd46c15b5a0 2024-05-28T14:10:12.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd471e00640 1 -- 172.21.15.42:0/2503562027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd46c0af000 con 0x7fd46c15b5a0 2024-05-28T14:10:12.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.420+0000 7fd471e00640 1 -- 172.21.15.42:0/2503562027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd46c0738b0 con 0x7fd46c15b5a0 2024-05-28T14:10:12.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.424+0000 7fd463e00640 1 -- 172.21.15.42:0/2503562027 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 89619+0+0 (secure 0 0 0) 0x7fd454012040 con 0x7fd46c15b5a0 2024-05-28T14:10:12.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.424+0000 7fd463e00640 1 --2- 172.21.15.42:0/2503562027 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd4480695b0 0x7fd44806ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:12.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.424+0000 7fd463e00640 1 -- 172.21.15.42:0/2503562027 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7fd454085f10 con 0x7fd46c15b5a0 2024-05-28T14:10:12.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.424+0000 7fd46aa00640 1 --2- 172.21.15.42:0/2503562027 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd4480695b0 0x7fd44806ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:12.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.424+0000 7fd46aa00640 1 --2- 172.21.15.42:0/2503562027 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd4480695b0 0x7fd44806ba70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0ae000 tx=0x7fd45c005040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:12.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.428+0000 7fd463e00640 1 -- 172.21.15.42:0/2503562027 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd454019030 con 0x7fd46c15b5a0 2024-05-28T14:10:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.628+0000 7fd471e00640 1 -- 172.21.15.42:0/2503562027 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fd46c0ceb80 con 0x7fd4480695b0 2024-05-28T14:10:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.688+0000 7fd463e00640 1 -- 172.21.15.42:0/2503562027 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd46c0ceb80 con 0x7fd4480695b0 2024-05-28T14:10:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.692+0000 7fd471e00640 1 -- 172.21.15.42:0/2503562027 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd4480695b0 msgr2=0x7fd44806ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.692+0000 7fd471e00640 1 --2- 172.21.15.42:0/2503562027 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd4480695b0 0x7fd44806ba70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0ae000 tx=0x7fd45c005040 comp rx=0 tx=0).stop 2024-05-28T14:10:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.692+0000 7fd471e00640 1 -- 172.21.15.42:0/2503562027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd46c15b5a0 msgr2=0x7fd46c158e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.692+0000 7fd471e00640 1 --2- 172.21.15.42:0/2503562027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd46c15b5a0 0x7fd46c158e50 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fd454011960 tx=0x7fd454011e20 comp rx=0 tx=0).stop 2024-05-28T14:10:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.692+0000 7fd471e00640 1 -- 172.21.15.42:0/2503562027 shutdown_connections 2024-05-28T14:10:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.692+0000 7fd471e00640 1 --2- 172.21.15.42:0/2503562027 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd4480695b0 0x7fd44806ba70 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.692+0000 7fd471e00640 1 --2- 172.21.15.42:0/2503562027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd46c13afc0 0x7fd46c159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.692+0000 7fd471e00640 1 --2- 172.21.15.42:0/2503562027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd46c15b5a0 0x7fd46c158e50 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.692+0000 7fd471e00640 1 --2- 172.21.15.42:0/2503562027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd46c06ee20 0x7fd46c15b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.692+0000 7fd471e00640 1 -- 172.21.15.42:0/2503562027 >> 172.21.15.42:0/2503562027 conn(0x7fd46c0b21f0 msgr2=0x7fd46c0b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.692+0000 7fd471e00640 1 -- 172.21.15.42:0/2503562027 shutdown_connections 2024-05-28T14:10:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.692+0000 7fd471e00640 1 -- 172.21.15.42:0/2503562027 wait complete. 2024-05-28T14:10:12.677 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:12.692+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:12.677 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:12.692+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:12.677 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:12.692+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:12.677 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:12.692+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:12.677 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:12.692+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:12.693 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:12.708+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:12.693 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:12.708+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:12.693 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:12.708+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:12.694 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:12.708+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:12.694 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:12.708+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:12.699 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-05-28T14:10:12.699 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-28T14:10:12.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd28da00640 1 Processor -- start 2024-05-28T14:10:12.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd28da00640 1 -- start start 2024-05-28T14:10:12.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd28da00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28815b780 0x7fd288073090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:12.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd28da00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd28806d110 0x7fd2880cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:12.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd28da00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd28815bb60 con 0x7fd2880d04b0 2024-05-28T14:10:12.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd28da00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd2880ce940 con 0x7fd28806d110 2024-05-28T14:10:12.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd28da00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd288073720 con 0x7fd28815b780 2024-05-28T14:10:12.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd286000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd2880d04b0 0x7fd28806ca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34168/0 (socket says 172.21.15.42:34168) 2024-05-28T14:10:12.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd286000640 1 -- 172.21.15.42:0/2249252755 learned_addr learned my addr 172.21.15.42:0/2249252755 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:12.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd286a00640 1 --2- 172.21.15.42:0/2249252755 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28815b780 0x7fd288073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:12.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd287400640 1 --2- 172.21.15.42:0/2249252755 >> v2:172.21.15.63:3300/0 conn(0x7fd28806d110 0x7fd2880cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd285600640 1 -- 172.21.15.42:0/2249252755 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2073518661 0 0) 0x7fd28815bb60 con 0x7fd2880d04b0 2024-05-28T14:10:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd285600640 1 -- 172.21.15.42:0/2249252755 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd26c003530 con 0x7fd2880d04b0 2024-05-28T14:10:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd286a00640 1 -- 172.21.15.42:0/2249252755 >> v2:172.21.15.63:3300/0 conn(0x7fd28806d110 msgr2=0x7fd2880cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd286a00640 1 --2- 172.21.15.42:0/2249252755 >> v2:172.21.15.63:3300/0 conn(0x7fd28806d110 0x7fd2880cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd286a00640 1 -- 172.21.15.42:0/2249252755 >> v1:172.21.15.42:6789/0 conn(0x7fd2880d04b0 legacy=0x7fd28806ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd286a00640 1 -- 172.21.15.42:0/2249252755 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2880b5730 con 0x7fd28815b780 2024-05-28T14:10:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.868+0000 7fd287400640 1 --2- 172.21.15.42:0/2249252755 >> v2:172.21.15.63:3300/0 conn(0x7fd28806d110 0x7fd2880cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:10:12.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.872+0000 7fd286a00640 1 --2- 172.21.15.42:0/2249252755 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28815b780 0x7fd288073090 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd2780097c0 tx=0x7fd27801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae85e114f4e38f47 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:12.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.872+0000 7fd285600640 1 -- 172.21.15.42:0/2249252755 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2780b2070 con 0x7fd28815b780 2024-05-28T14:10:12.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.872+0000 7fd285600640 1 -- 172.21.15.42:0/2249252755 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2780ad640 con 0x7fd28815b780 2024-05-28T14:10:12.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.872+0000 7fd285600640 1 -- 172.21.15.42:0/2249252755 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2780ada10 con 0x7fd28815b780 2024-05-28T14:10:12.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.872+0000 7fd28da00640 1 -- 172.21.15.42:0/2249252755 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28815b780 msgr2=0x7fd288073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:12.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.872+0000 7fd28da00640 1 --2- 172.21.15.42:0/2249252755 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28815b780 0x7fd288073090 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd2780097c0 tx=0x7fd27801c4a0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.872+0000 7fd28da00640 1 -- 172.21.15.42:0/2249252755 shutdown_connections 2024-05-28T14:10:12.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.872+0000 7fd28da00640 1 --2- 172.21.15.42:0/2249252755 >> v2:172.21.15.63:3300/0 conn(0x7fd28806d110 0x7fd2880cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.872+0000 7fd28da00640 1 --2- 172.21.15.42:0/2249252755 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28815b780 0x7fd288073090 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.872+0000 7fd28da00640 1 -- 172.21.15.42:0/2249252755 >> 172.21.15.42:0/2249252755 conn(0x7fd288072570 msgr2=0x7fd288072980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:12.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.872+0000 7fd28da00640 1 -- 172.21.15.42:0/2249252755 shutdown_connections 2024-05-28T14:10:12.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.872+0000 7fd28da00640 1 -- 172.21.15.42:0/2249252755 wait complete. 2024-05-28T14:10:12.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd28da00640 1 Processor -- start 2024-05-28T14:10:12.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd28da00640 1 -- start start 2024-05-28T14:10:12.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd28da00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28806d110 0x7fd288141220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:12.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd28da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2880d04b0 0x7fd288150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:12.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd28da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd28815b780 0x7fd2881514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:12.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd28da00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd2880b3d90 con 0x7fd2880d04b0 2024-05-28T14:10:12.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd28da00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2880b3c10 con 0x7fd28815b780 2024-05-28T14:10:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd28da00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd288166560 con 0x7fd28806d110 2024-05-28T14:10:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd286000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2880d04b0 0x7fd288150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd286000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2880d04b0 0x7fd288150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34682/0 (socket says 172.21.15.42:34682) 2024-05-28T14:10:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd286000640 1 -- 172.21.15.42:0/2852250395 learned_addr learned my addr 172.21.15.42:0/2852250395 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd286a00640 1 --2- 172.21.15.42:0/2852250395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28806d110 0x7fd288141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd287400640 1 --2- 172.21.15.42:0/2852250395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd28815b780 0x7fd2881514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd286000640 1 -- 172.21.15.42:0/2852250395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28806d110 msgr2=0x7fd288141220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd286000640 1 --2- 172.21.15.42:0/2852250395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28806d110 0x7fd288141220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd286000640 1 -- 172.21.15.42:0/2852250395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd28815b780 msgr2=0x7fd2881514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd286000640 1 --2- 172.21.15.42:0/2852250395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd28815b780 0x7fd2881514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd286000640 1 -- 172.21.15.42:0/2852250395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd288141760 con 0x7fd2880d04b0 2024-05-28T14:10:12.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd286000640 1 --2- 172.21.15.42:0/2852250395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2880d04b0 0x7fd288150f90 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd27400c450 tx=0x7fd27400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:12.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd27ea00640 1 -- 172.21.15.42:0/2852250395 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274017070 con 0x7fd2880d04b0 2024-05-28T14:10:12.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd27ea00640 1 -- 172.21.15.42:0/2852250395 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd274005cf0 con 0x7fd2880d04b0 2024-05-28T14:10:12.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd27ea00640 1 -- 172.21.15.42:0/2852250395 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2740072f0 con 0x7fd2880d04b0 2024-05-28T14:10:12.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd28da00640 1 -- 172.21.15.42:0/2852250395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd288158640 con 0x7fd2880d04b0 2024-05-28T14:10:12.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.876+0000 7fd28da00640 1 -- 172.21.15.42:0/2852250395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd288158c00 con 0x7fd2880d04b0 2024-05-28T14:10:12.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.880+0000 7fd28da00640 1 -- 172.21.15.42:0/2852250395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd288148c30 con 0x7fd2880d04b0 2024-05-28T14:10:12.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.880+0000 7fd27ea00640 1 -- 172.21.15.42:0/2852250395 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 89619+0+0 (secure 0 0 0) 0x7fd27401a070 con 0x7fd2880d04b0 2024-05-28T14:10:12.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.880+0000 7fd27ea00640 1 --2- 172.21.15.42:0/2852250395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd268069580 0x7fd26806ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:12.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.880+0000 7fd27ea00640 1 -- 172.21.15.42:0/2852250395 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7fd274084d30 con 0x7fd2880d04b0 2024-05-28T14:10:12.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.880+0000 7fd286a00640 1 --2- 172.21.15.42:0/2852250395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd268069580 0x7fd26806ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:12.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.880+0000 7fd286a00640 1 --2- 172.21.15.42:0/2852250395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd268069580 0x7fd26806ba40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd2780af000 tx=0x7fd278008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:12.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:12.884+0000 7fd27ea00640 1 -- 172.21.15.42:0/2852250395 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd2740562f0 con 0x7fd2880d04b0 2024-05-28T14:10:13.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.088+0000 7fd28da00640 1 -- 172.21.15.42:0/2852250395 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fd2880d0010 con 0x7fd268069580 2024-05-28T14:10:13.678 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:13.692+0000 7f0743a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:13.678 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:13.692+0000 7f0743a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:13.678 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:13.692+0000 7f0743a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:13.678 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:13.692+0000 7f0743a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:13.678 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:13.692+0000 7f0743a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:13.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.720+0000 7fd27ea00640 1 -- 172.21.15.42:0/2852250395 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+275 (secure 0 0 0) 0x7fd2880d0010 con 0x7fd268069580 2024-05-28T14:10:13.705 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "status": { 2024-05-28T14:10:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "in-progress", 2024-05-28T14:10:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "source": { 2024-05-28T14:10:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "volume": "cephfs", 2024-05-28T14:10:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "subvolume": "sv1", 2024-05-28T14:10:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot": "ss1" 2024-05-28T14:10:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_report": { 2024-05-28T14:10:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "percentage cloned": 0.0, 2024-05-28T14:10:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "amount cloned": "184/3.0G", 2024-05-28T14:10:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "files cloned": "3/6" 2024-05-28T14:10:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:13.707 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:13.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.724+0000 7fd28da00640 1 -- 172.21.15.42:0/2852250395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd268069580 msgr2=0x7fd26806ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:13.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.724+0000 7fd28da00640 1 --2- 172.21.15.42:0/2852250395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd268069580 0x7fd26806ba40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd2780af000 tx=0x7fd278008040 comp rx=0 tx=0).stop 2024-05-28T14:10:13.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.724+0000 7fd28da00640 1 -- 172.21.15.42:0/2852250395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2880d04b0 msgr2=0x7fd288150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:13.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.724+0000 7fd28da00640 1 --2- 172.21.15.42:0/2852250395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2880d04b0 0x7fd288150f90 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd27400c450 tx=0x7fd27400c850 comp rx=0 tx=0).stop 2024-05-28T14:10:13.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.724+0000 7fd28da00640 1 -- 172.21.15.42:0/2852250395 shutdown_connections 2024-05-28T14:10:13.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.724+0000 7fd28da00640 1 --2- 172.21.15.42:0/2852250395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd268069580 0x7fd26806ba40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.724+0000 7fd28da00640 1 --2- 172.21.15.42:0/2852250395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd28815b780 0x7fd2881514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.724+0000 7fd28da00640 1 --2- 172.21.15.42:0/2852250395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2880d04b0 0x7fd288150f90 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.724+0000 7fd28da00640 1 --2- 172.21.15.42:0/2852250395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd28806d110 0x7fd288141220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.724+0000 7fd28da00640 1 -- 172.21.15.42:0/2852250395 >> 172.21.15.42:0/2852250395 conn(0x7fd288072570 msgr2=0x7fd288156be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.724+0000 7fd28da00640 1 -- 172.21.15.42:0/2852250395 shutdown_connections 2024-05-28T14:10:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.724+0000 7fd28da00640 1 -- 172.21.15.42:0/2852250395 wait complete. 2024-05-28T14:10:13.739 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-28T14:10:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.976+0000 7f2afd000640 1 Processor -- start 2024-05-28T14:10:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.976+0000 7f2afd000640 1 -- start start 2024-05-28T14:10:13.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2afd000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2af815bef0 0x7f2af80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2afd000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2af806f910 0x7f2af8160220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2afd000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af80d19c0 con 0x7f2af80ce960 2024-05-28T14:10:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2afd000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2af80b3af0 con 0x7f2af815bef0 2024-05-28T14:10:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2afd000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2af80b4390 con 0x7f2af806f910 2024-05-28T14:10:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2af7400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2af815bef0 0x7f2af80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2aeea00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f2af80ce960 0x7f2af815c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:46666/0 (socket says 172.21.15.42:46666) 2024-05-28T14:10:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2aeea00640 1 -- 172.21.15.42:0/2030571467 learned_addr learned my addr 172.21.15.42:0/2030571467 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2af7e00640 1 --2- 172.21.15.42:0/2030571467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2af806f910 0x7f2af8160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2af6a00640 1 -- 172.21.15.42:0/2030571467 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2231497272 0 0) 0x7f2af80d19c0 con 0x7f2af80ce960 2024-05-28T14:10:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2af6a00640 1 -- 172.21.15.42:0/2030571467 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2adc003530 con 0x7f2af80ce960 2024-05-28T14:10:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2af7400640 1 -- 172.21.15.42:0/2030571467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2af806f910 msgr2=0x7f2af8160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2af7400640 1 --2- 172.21.15.42:0/2030571467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2af806f910 0x7f2af8160220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2af7400640 1 -- 172.21.15.42:0/2030571467 >> v1:172.21.15.63:6789/0 conn(0x7f2af80ce960 legacy=0x7f2af815c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.980+0000 7f2af7400640 1 -- 172.21.15.42:0/2030571467 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af8070080 con 0x7f2af815bef0 2024-05-28T14:10:13.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.988+0000 7f2af7400640 1 --2- 172.21.15.42:0/2030571467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2af815bef0 0x7f2af80706c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80097c0 tx=0x7f2ae801c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=756c7793f28d0d79 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:13.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.988+0000 7f2af6a00640 1 -- 172.21.15.42:0/2030571467 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80b0070 con 0x7f2af815bef0 2024-05-28T14:10:13.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.988+0000 7f2af6a00640 1 -- 172.21.15.42:0/2030571467 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae80ab600 con 0x7f2af815bef0 2024-05-28T14:10:13.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.988+0000 7f2af6a00640 1 -- 172.21.15.42:0/2030571467 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80ab9d0 con 0x7f2af815bef0 2024-05-28T14:10:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.992+0000 7f2afd000640 1 -- 172.21.15.42:0/2030571467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2af815bef0 msgr2=0x7f2af80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.992+0000 7f2afd000640 1 --2- 172.21.15.42:0/2030571467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2af815bef0 0x7f2af80706c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80097c0 tx=0x7f2ae801c460 comp rx=0 tx=0).stop 2024-05-28T14:10:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.992+0000 7f2afd000640 1 -- 172.21.15.42:0/2030571467 shutdown_connections 2024-05-28T14:10:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.992+0000 7f2afd000640 1 --2- 172.21.15.42:0/2030571467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2af806f910 0x7f2af8160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.992+0000 7f2afd000640 1 --2- 172.21.15.42:0/2030571467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2af815bef0 0x7f2af80706c0 secure :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80097c0 tx=0x7f2ae801c460 comp rx=0 tx=0).stop 2024-05-28T14:10:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.992+0000 7f2afd000640 1 -- 172.21.15.42:0/2030571467 >> 172.21.15.42:0/2030571467 conn(0x7f2af80cd540 msgr2=0x7f2af80cd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:13.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.992+0000 7f2afd000640 1 -- 172.21.15.42:0/2030571467 shutdown_connections 2024-05-28T14:10:13.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.992+0000 7f2afd000640 1 -- 172.21.15.42:0/2030571467 wait complete. 2024-05-28T14:10:13.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.996+0000 7f2afd000640 1 Processor -- start 2024-05-28T14:10:13.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:13.996+0000 7f2afd000640 1 -- start start 2024-05-28T14:10:13.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2afd000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2af806f910 0x7f2af8158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:13.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2afd000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2af80ce960 0x7f2af815d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:13.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2afd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2af815dd00 0x7f2af81576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:13.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2afd000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2af806fd20 con 0x7f2af806f910 2024-05-28T14:10:13.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2afd000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2af80cedd0 con 0x7f2af815dd00 2024-05-28T14:10:13.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2afd000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2af80b5030 con 0x7f2af80ce960 2024-05-28T14:10:13.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2aeea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2af80ce960 0x7f2af815d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:13.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2aeea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2af80ce960 0x7f2af815d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:47144/0 (socket says 172.21.15.42:47144) 2024-05-28T14:10:13.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2aeea00640 1 -- 172.21.15.42:0/538841741 learned_addr learned my addr 172.21.15.42:0/538841741 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:13.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2af7e00640 1 --2- 172.21.15.42:0/538841741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2af815dd00 0x7f2af81576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:13.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2aeea00640 1 -- 172.21.15.42:0/538841741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2af815dd00 msgr2=0x7f2af81576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:13.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2aeea00640 1 --2- 172.21.15.42:0/538841741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2af815dd00 0x7f2af81576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:13.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2aeea00640 1 -- 172.21.15.42:0/538841741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2af806f910 msgr2=0x7f2af8158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:13.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2aeea00640 1 --2- 172.21.15.42:0/538841741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2af806f910 0x7f2af8158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:13.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2aeea00640 1 -- 172.21.15.42:0/538841741 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af81492c0 con 0x7f2af80ce960 2024-05-28T14:10:13.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2aeea00640 1 --2- 172.21.15.42:0/538841741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2af80ce960 0x7f2af815d7c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000c4a0 tx=0x7f2ae000c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:13.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2aefe00640 1 -- 172.21.15.42:0/538841741 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0018070 con 0x7f2af80ce960 2024-05-28T14:10:13.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2aefe00640 1 -- 172.21.15.42:0/538841741 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae00074e0 con 0x7f2af80ce960 2024-05-28T14:10:13.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2aefe00640 1 -- 172.21.15.42:0/538841741 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae00049f0 con 0x7f2af80ce960 2024-05-28T14:10:13.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2afd000640 1 -- 172.21.15.42:0/538841741 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af81495a0 con 0x7f2af80ce960 2024-05-28T14:10:13.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.000+0000 7f2afd000640 1 -- 172.21.15.42:0/538841741 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af8158260 con 0x7f2af80ce960 2024-05-28T14:10:13.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.004+0000 7f2afd000640 1 -- 172.21.15.42:0/538841741 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af8158460 con 0x7f2af80ce960 2024-05-28T14:10:13.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.004+0000 7f2aefe00640 1 -- 172.21.15.42:0/538841741 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 33) ==== 89721+0+0 (secure 0 0 0) 0x7f2ae0004b90 con 0x7f2af80ce960 2024-05-28T14:10:13.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.008+0000 7f2aefe00640 1 --2- 172.21.15.42:0/538841741 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2ad40696d0 0x7f2ad406bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:13.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.008+0000 7f2aefe00640 1 -- 172.21.15.42:0/538841741 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7f2ae0084c00 con 0x7f2af80ce960 2024-05-28T14:10:13.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.012+0000 7f2aefe00640 1 -- 172.21.15.42:0/538841741 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2af8158460 con 0x7f2af80ce960 2024-05-28T14:10:13.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.012+0000 7f2af7400640 1 --2- 172.21.15.42:0/538841741 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2ad40696d0 0x7f2ad406bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:13.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.016+0000 7f2af7400640 1 --2- 172.21.15.42:0/538841741 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2ad40696d0 0x7f2ad406bb90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80ad000 tx=0x7f2ae8004740 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:14.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.360+0000 7f2afd000640 1 -- 172.21.15.42:0/538841741 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f2af8070d10 con 0x7f2ad40696d0 2024-05-28T14:10:14.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.396+0000 7f2aefe00640 1 -- 172.21.15.42:0/538841741 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+275 (secure 0 0 0) 0x7f2af8070d10 con 0x7f2ad40696d0 2024-05-28T14:10:14.382 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:14.382 INFO:teuthology.orchestra.run.smithi042.stdout: "status": { 2024-05-28T14:10:14.382 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "in-progress", 2024-05-28T14:10:14.382 INFO:teuthology.orchestra.run.smithi042.stdout: "source": { 2024-05-28T14:10:14.382 INFO:teuthology.orchestra.run.smithi042.stdout: "volume": "cephfs", 2024-05-28T14:10:14.382 INFO:teuthology.orchestra.run.smithi042.stdout: "subvolume": "sv1", 2024-05-28T14:10:14.383 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot": "ss1" 2024-05-28T14:10:14.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:14.383 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_report": { 2024-05-28T14:10:14.383 INFO:teuthology.orchestra.run.smithi042.stdout: "percentage cloned": 0.0, 2024-05-28T14:10:14.383 INFO:teuthology.orchestra.run.smithi042.stdout: "amount cloned": "184/3.0G", 2024-05-28T14:10:14.383 INFO:teuthology.orchestra.run.smithi042.stdout: "files cloned": "4/6" 2024-05-28T14:10:14.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:14.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:14.383 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:14.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.400+0000 7f2afd000640 1 -- 172.21.15.42:0/538841741 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2ad40696d0 msgr2=0x7f2ad406bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:14.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.400+0000 7f2afd000640 1 --2- 172.21.15.42:0/538841741 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2ad40696d0 0x7f2ad406bb90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80ad000 tx=0x7f2ae8004740 comp rx=0 tx=0).stop 2024-05-28T14:10:14.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.400+0000 7f2afd000640 1 -- 172.21.15.42:0/538841741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2af80ce960 msgr2=0x7f2af815d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:14.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.400+0000 7f2afd000640 1 --2- 172.21.15.42:0/538841741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2af80ce960 0x7f2af815d7c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000c4a0 tx=0x7f2ae000c960 comp rx=0 tx=0).stop 2024-05-28T14:10:14.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.400+0000 7f2afd000640 1 -- 172.21.15.42:0/538841741 shutdown_connections 2024-05-28T14:10:14.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.400+0000 7f2afd000640 1 --2- 172.21.15.42:0/538841741 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2ad40696d0 0x7f2ad406bb90 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:14.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.400+0000 7f2afd000640 1 --2- 172.21.15.42:0/538841741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2af815dd00 0x7f2af81576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:14.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.400+0000 7f2afd000640 1 --2- 172.21.15.42:0/538841741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2af80ce960 0x7f2af815d7c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:14.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.400+0000 7f2afd000640 1 --2- 172.21.15.42:0/538841741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2af806f910 0x7f2af8158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:14.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.400+0000 7f2afd000640 1 -- 172.21.15.42:0/538841741 >> 172.21.15.42:0/538841741 conn(0x7f2af80cd540 msgr2=0x7f2af806cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:14.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.400+0000 7f2afd000640 1 -- 172.21.15.42:0/538841741 shutdown_connections 2024-05-28T14:10:14.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.400+0000 7f2afd000640 1 -- 172.21.15.42:0/538841741 wait complete. 2024-05-28T14:10:14.429 INFO:tasks.cephfs.test_volumes:percentage cloned = 0.0 2024-05-28T14:10:14.429 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-28T14:10:14.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fda02000640 1 Processor -- start 2024-05-28T14:10:14.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fda02000640 1 -- start start 2024-05-28T14:10:14.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fda02000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd9fc071de0 0x7fd9fc0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:14.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fda02000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd9fc0ce3b0 0x7fd9fc0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:14.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fda02000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9fc072700 con 0x7fd9fc141d10 2024-05-28T14:10:14.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fda02000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd9fc160230 con 0x7fd9fc0ce3b0 2024-05-28T14:10:14.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fda02000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd9fc160510 con 0x7fd9fc071de0 2024-05-28T14:10:14.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fd9faa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd9fc0ce3b0 0x7fd9fc0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:14.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fd9fbe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd9fc141d10 0x7fd9fc142160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34182/0 (socket says 172.21.15.42:34182) 2024-05-28T14:10:14.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fd9fbe00640 1 -- 172.21.15.42:0/1949823576 learned_addr learned my addr 172.21.15.42:0/1949823576 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:14.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fd9fb400640 1 --2- 172.21.15.42:0/1949823576 >> v2:172.21.15.63:3300/0 conn(0x7fd9fc071de0 0x7fd9fc0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:14.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fd9fb400640 1 -- 172.21.15.42:0/1949823576 >> v2:172.21.15.42:3300/0 conn(0x7fd9fc0ce3b0 msgr2=0x7fd9fc0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:14.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fd9fb400640 1 --2- 172.21.15.42:0/1949823576 >> v2:172.21.15.42:3300/0 conn(0x7fd9fc0ce3b0 0x7fd9fc0ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:14.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fd9fb400640 1 -- 172.21.15.42:0/1949823576 >> v1:172.21.15.42:6789/0 conn(0x7fd9fc141d10 legacy=0x7fd9fc142160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:14.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fd9fb400640 1 -- 172.21.15.42:0/1949823576 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9fc1607f0 con 0x7fd9fc071de0 2024-05-28T14:10:14.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fd9faa00640 1 --2- 172.21.15.42:0/1949823576 >> v2:172.21.15.42:3300/0 conn(0x7fd9fc0ce3b0 0x7fd9fc0ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:10:14.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.620+0000 7fd9fa000640 1 -- 172.21.15.42:0/1949823576 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2485624208 0 0) 0x7fd9fc072700 con 0x7fd9fc141d10 2024-05-28T14:10:14.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.624+0000 7fd9fb400640 1 --2- 172.21.15.42:0/1949823576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9fc071de0 0x7fd9fc0721c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc06cbf0 tx=0x7fd9ec01c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=354c6645806c9f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:14.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.624+0000 7fd9fa000640 1 -- 172.21.15.42:0/1949823576 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0a66e0 con 0x7fd9fc071de0 2024-05-28T14:10:14.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.624+0000 7fd9fa000640 1 -- 172.21.15.42:0/1949823576 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9ec0a6880 con 0x7fd9fc071de0 2024-05-28T14:10:14.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.624+0000 7fd9fa000640 1 -- 172.21.15.42:0/1949823576 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0a6c50 con 0x7fd9fc071de0 2024-05-28T14:10:14.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.624+0000 7fda02000640 1 -- 172.21.15.42:0/1949823576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9fc071de0 msgr2=0x7fd9fc0721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:14.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.624+0000 7fda02000640 1 --2- 172.21.15.42:0/1949823576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9fc071de0 0x7fd9fc0721c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc06cbf0 tx=0x7fd9ec01c810 comp rx=0 tx=0).stop 2024-05-28T14:10:14.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.624+0000 7fda02000640 1 -- 172.21.15.42:0/1949823576 shutdown_connections 2024-05-28T14:10:14.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.624+0000 7fda02000640 1 --2- 172.21.15.42:0/1949823576 >> v2:172.21.15.42:3300/0 conn(0x7fd9fc0ce3b0 0x7fd9fc0ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:14.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.624+0000 7fda02000640 1 --2- 172.21.15.42:0/1949823576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9fc071de0 0x7fd9fc0721c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:14.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.624+0000 7fda02000640 1 -- 172.21.15.42:0/1949823576 >> 172.21.15.42:0/1949823576 conn(0x7fd9fc0b21f0 msgr2=0x7fd9fc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:14.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.624+0000 7fda02000640 1 -- 172.21.15.42:0/1949823576 shutdown_connections 2024-05-28T14:10:14.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.624+0000 7fda02000640 1 -- 172.21.15.42:0/1949823576 wait complete. 2024-05-28T14:10:14.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fda02000640 1 Processor -- start 2024-05-28T14:10:14.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fda02000640 1 -- start start 2024-05-28T14:10:14.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fda02000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9fc071de0 0x7fd9fc144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:14.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fda02000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9fc0ce3b0 0x7fd9fc13b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:14.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fda02000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9fc141d10 0x7fd9fc13ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:14.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fda02000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc169e30 con 0x7fd9fc071de0 2024-05-28T14:10:14.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fda02000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc169cb0 con 0x7fd9fc0ce3b0 2024-05-28T14:10:14.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fda02000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd9fc169fb0 con 0x7fd9fc141d10 2024-05-28T14:10:14.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fd9fb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9fc071de0 0x7fd9fc144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:14.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fd9fb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9fc071de0 0x7fd9fc144be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34726/0 (socket says 172.21.15.42:34726) 2024-05-28T14:10:14.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fd9fb400640 1 -- 172.21.15.42:0/2222920923 learned_addr learned my addr 172.21.15.42:0/2222920923 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:14.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fd9faa00640 1 --2- 172.21.15.42:0/2222920923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9fc0ce3b0 0x7fd9fc13b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:14.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fd9fbe00640 1 --2- 172.21.15.42:0/2222920923 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9fc141d10 0x7fd9fc13ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:14.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fd9fb400640 1 -- 172.21.15.42:0/2222920923 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9fc141d10 msgr2=0x7fd9fc13ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:14.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fd9fb400640 1 --2- 172.21.15.42:0/2222920923 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9fc141d10 0x7fd9fc13ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:14.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fd9fb400640 1 -- 172.21.15.42:0/2222920923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9fc0ce3b0 msgr2=0x7fd9fc13b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:14.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fd9fb400640 1 --2- 172.21.15.42:0/2222920923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9fc0ce3b0 0x7fd9fc13b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:14.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fd9fb400640 1 -- 172.21.15.42:0/2222920923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9fc146100 con 0x7fd9fc071de0 2024-05-28T14:10:14.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fd9fb400640 1 --2- 172.21.15.42:0/2222920923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9fc071de0 0x7fd9fc144be0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0b0040 tx=0x7fd9ec0195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:14.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fd9f3400640 1 -- 172.21.15.42:0/2222920923 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0be070 con 0x7fd9fc071de0 2024-05-28T14:10:14.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fd9f3400640 1 -- 172.21.15.42:0/2222920923 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9ec0b8860 con 0x7fd9fc071de0 2024-05-28T14:10:14.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fda02000640 1 -- 172.21.15.42:0/2222920923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9fc1433e0 con 0x7fd9fc071de0 2024-05-28T14:10:14.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fd9f3400640 1 -- 172.21.15.42:0/2222920923 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0a66e0 con 0x7fd9fc071de0 2024-05-28T14:10:14.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fda02000640 1 -- 172.21.15.42:0/2222920923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9fc1458f0 con 0x7fd9fc071de0 2024-05-28T14:10:14.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.628+0000 7fda02000640 1 -- 172.21.15.42:0/2222920923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9fc0738b0 con 0x7fd9fc071de0 2024-05-28T14:10:14.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.632+0000 7fd9f3400640 1 -- 172.21.15.42:0/2222920923 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 33) ==== 89721+0+0 (secure 0 0 0) 0x7fd9ec0c1070 con 0x7fd9fc071de0 2024-05-28T14:10:14.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.632+0000 7fd9f3400640 1 --2- 172.21.15.42:0/2222920923 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd9dc0696a0 0x7fd9dc06bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:14.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.632+0000 7fd9f3400640 1 -- 172.21.15.42:0/2222920923 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7fd9ec120b50 con 0x7fd9fc071de0 2024-05-28T14:10:14.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.632+0000 7fd9faa00640 1 --2- 172.21.15.42:0/2222920923 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd9dc0696a0 0x7fd9dc06bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:14.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.632+0000 7fd9faa00640 1 --2- 172.21.15.42:0/2222920923 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd9dc0696a0 0x7fd9dc06bb60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8009580 tx=0x7fd9e8009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:14.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.636+0000 7fd9f3400640 1 -- 172.21.15.42:0/2222920923 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd9ec0c8030 con 0x7fd9fc071de0 2024-05-28T14:10:14.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:14.972+0000 7fda02000640 1 -- 172.21.15.42:0/2222920923 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fd9fc072ed0 con 0x7fd9dc0696a0 2024-05-28T14:10:14.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.004+0000 7fd9f3400640 1 -- 172.21.15.42:0/2222920923 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7fd9fc072ed0 con 0x7fd9dc0696a0 2024-05-28T14:10:14.990 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:14.990 INFO:teuthology.orchestra.run.smithi042.stdout: "status": { 2024-05-28T14:10:14.990 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "in-progress", 2024-05-28T14:10:14.990 INFO:teuthology.orchestra.run.smithi042.stdout: "source": { 2024-05-28T14:10:14.990 INFO:teuthology.orchestra.run.smithi042.stdout: "volume": "cephfs", 2024-05-28T14:10:14.990 INFO:teuthology.orchestra.run.smithi042.stdout: "subvolume": "sv1", 2024-05-28T14:10:14.990 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot": "ss1" 2024-05-28T14:10:14.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:14.990 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_report": { 2024-05-28T14:10:14.990 INFO:teuthology.orchestra.run.smithi042.stdout: "percentage cloned": 13.542, 2024-05-28T14:10:14.990 INFO:teuthology.orchestra.run.smithi042.stdout: "amount cloned": "416M/3.0G", 2024-05-28T14:10:14.991 INFO:teuthology.orchestra.run.smithi042.stdout: "files cloned": "4/6" 2024-05-28T14:10:14.991 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:14.991 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:14.991 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.008+0000 7fda02000640 1 -- 172.21.15.42:0/2222920923 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd9dc0696a0 msgr2=0x7fd9dc06bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.008+0000 7fda02000640 1 --2- 172.21.15.42:0/2222920923 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd9dc0696a0 0x7fd9dc06bb60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8009580 tx=0x7fd9e8009190 comp rx=0 tx=0).stop 2024-05-28T14:10:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.008+0000 7fda02000640 1 -- 172.21.15.42:0/2222920923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9fc071de0 msgr2=0x7fd9fc144be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.008+0000 7fda02000640 1 --2- 172.21.15.42:0/2222920923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9fc071de0 0x7fd9fc144be0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0b0040 tx=0x7fd9ec0195b0 comp rx=0 tx=0).stop 2024-05-28T14:10:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.008+0000 7fda02000640 1 -- 172.21.15.42:0/2222920923 shutdown_connections 2024-05-28T14:10:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.008+0000 7fda02000640 1 --2- 172.21.15.42:0/2222920923 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd9dc0696a0 0x7fd9dc06bb60 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.008+0000 7fda02000640 1 --2- 172.21.15.42:0/2222920923 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9fc141d10 0x7fd9fc13ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.008+0000 7fda02000640 1 --2- 172.21.15.42:0/2222920923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9fc0ce3b0 0x7fd9fc13b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.008+0000 7fda02000640 1 --2- 172.21.15.42:0/2222920923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9fc071de0 0x7fd9fc144be0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.008+0000 7fda02000640 1 -- 172.21.15.42:0/2222920923 >> 172.21.15.42:0/2222920923 conn(0x7fd9fc0b21f0 msgr2=0x7fd9fc0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.008+0000 7fda02000640 1 -- 172.21.15.42:0/2222920923 shutdown_connections 2024-05-28T14:10:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.008+0000 7fda02000640 1 -- 172.21.15.42:0/2222920923 wait complete. 2024-05-28T14:10:15.021 INFO:tasks.cephfs.test_volumes:percentage cloned = 13.542 2024-05-28T14:10:15.022 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-28T14:10:15.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.288+0000 7f0ccf200640 1 Processor -- start 2024-05-28T14:10:15.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.288+0000 7f0ccf200640 1 -- start start 2024-05-28T14:10:15.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.292+0000 7f0ccf200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cc80d0ae0 0x7f0cc80d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:15.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.292+0000 7f0ccf200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0cc80d00d0 0x7f0cc8160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.292+0000 7f0ccf200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cc815cef0 con 0x7f0cc80d0530 2024-05-28T14:10:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.292+0000 7f0ccf200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0cc815dae0 con 0x7f0cc80d00d0 2024-05-28T14:10:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.292+0000 7f0ccf200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc815d1b0 con 0x7f0cc80d0ae0 2024-05-28T14:10:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.292+0000 7f0cbf400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0cc80d00d0 0x7f0cc8160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.292+0000 7f0cbf400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0cc80d00d0 0x7f0cc8160270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:40470/0 (socket says 172.21.15.42:40470) 2024-05-28T14:10:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.292+0000 7f0cbf400640 1 -- 172.21.15.42:0/3670893245 learned_addr learned my addr 172.21.15.42:0/3670893245 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:15.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.292+0000 7f0cbf400640 1 -- 172.21.15.42:0/3670893245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cc80d0ae0 msgr2=0x7f0cc80d0ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T14:10:15.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.292+0000 7f0cbf400640 1 --2- 172.21.15.42:0/3670893245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cc80d0ae0 0x7f0cc80d0ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.292+0000 7f0cbf400640 1 -- 172.21.15.42:0/3670893245 >> v1:172.21.15.63:6789/0 conn(0x7f0cc80d0530 legacy=0x7f0cc81607b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T14:10:15.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.292+0000 7f0cbf400640 1 -- 172.21.15.42:0/3670893245 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc815d490 con 0x7f0cc80d00d0 2024-05-28T14:10:15.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.296+0000 7f0cbf400640 1 --2- 172.21.15.42:0/3670893245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cc80d00d0 0x7f0cc8160270 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000dd10 tx=0x7f0cb0024cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=986fcf23475372ab server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:15.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.304+0000 7f0cc7400640 1 -- 172.21.15.42:0/3670893245 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb000d0d0 con 0x7f0cc80d00d0 2024-05-28T14:10:15.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.304+0000 7f0cc7400640 1 -- 172.21.15.42:0/3670893245 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb000d270 con 0x7f0cc80d00d0 2024-05-28T14:10:15.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.304+0000 7f0cc7400640 1 -- 172.21.15.42:0/3670893245 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb00adc70 con 0x7f0cc80d00d0 2024-05-28T14:10:15.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.304+0000 7f0ccf200640 1 -- 172.21.15.42:0/3670893245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cc80d00d0 msgr2=0x7f0cc8160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:15.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.304+0000 7f0ccf200640 1 --2- 172.21.15.42:0/3670893245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cc80d00d0 0x7f0cc8160270 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000dd10 tx=0x7f0cb0024cb0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.304+0000 7f0ccf200640 1 -- 172.21.15.42:0/3670893245 shutdown_connections 2024-05-28T14:10:15.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.304+0000 7f0ccf200640 1 --2- 172.21.15.42:0/3670893245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cc80d00d0 0x7f0cc8160270 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.304+0000 7f0ccf200640 1 --2- 172.21.15.42:0/3670893245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cc80d0ae0 0x7f0cc80d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.304+0000 7f0ccf200640 1 -- 172.21.15.42:0/3670893245 >> 172.21.15.42:0/3670893245 conn(0x7f0cc80b3ff0 msgr2=0x7f0cc80b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:15.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.304+0000 7f0ccf200640 1 -- 172.21.15.42:0/3670893245 shutdown_connections 2024-05-28T14:10:15.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.304+0000 7f0ccf200640 1 -- 172.21.15.42:0/3670893245 wait complete. 2024-05-28T14:10:15.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.308+0000 7f0ccf200640 1 Processor -- start 2024-05-28T14:10:15.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.308+0000 7f0ccf200640 1 -- start start 2024-05-28T14:10:15.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.308+0000 7f0ccf200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cc80d00d0 0x7f0cc815b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:15.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.308+0000 7f0ccf200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cc80d0530 0x7f0cc81596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:15.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.308+0000 7f0ccf200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cc80d0ae0 0x7f0cc8154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:15.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0cbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cc80d0530 0x7f0cc81596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:15.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0cc7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cc80d00d0 0x7f0cc815b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:15.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0cbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cc80d0530 0x7f0cc81596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34740/0 (socket says 172.21.15.42:34740) 2024-05-28T14:10:15.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0cc7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cc80d00d0 0x7f0cc815b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:47162/0 (socket says 172.21.15.42:47162) 2024-05-28T14:10:15.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0cbf400640 1 -- 172.21.15.42:0/3335732606 learned_addr learned my addr 172.21.15.42:0/3335732606 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:15.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0ccf200640 1 -- 172.21.15.42:0/3335732606 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc815ccb0 con 0x7f0cc80d0530 2024-05-28T14:10:15.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0ccf200640 1 -- 172.21.15.42:0/3335732606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc815cb30 con 0x7f0cc80d0ae0 2024-05-28T14:10:15.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0ccf200640 1 -- 172.21.15.42:0/3335732606 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc8160ec0 con 0x7f0cc80d00d0 2024-05-28T14:10:15.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0ccd200640 1 --2- 172.21.15.42:0/3335732606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cc80d0ae0 0x7f0cc8154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:15.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0cc7e00640 1 -- 172.21.15.42:0/3335732606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cc80d0ae0 msgr2=0x7f0cc8154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:15.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0cc7e00640 1 --2- 172.21.15.42:0/3335732606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cc80d0ae0 0x7f0cc8154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0cc7e00640 1 -- 172.21.15.42:0/3335732606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cc80d0530 msgr2=0x7f0cc81596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:15.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0cc7e00640 1 --2- 172.21.15.42:0/3335732606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cc80d0530 0x7f0cc81596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0cc7e00640 1 -- 172.21.15.42:0/3335732606 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc8158e00 con 0x7f0cc80d00d0 2024-05-28T14:10:15.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0cc7e00640 1 --2- 172.21.15.42:0/3335732606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cc80d00d0 0x7f0cc815b8b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800b3f0 tx=0x7f0cb800b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:15.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0cc4c00640 1 -- 172.21.15.42:0/3335732606 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8015020 con 0x7f0cc80d00d0 2024-05-28T14:10:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0cc4c00640 1 -- 172.21.15.42:0/3335732606 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb8011830 con 0x7f0cc80d00d0 2024-05-28T14:10:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.312+0000 7f0cc4c00640 1 -- 172.21.15.42:0/3335732606 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8011c20 con 0x7f0cc80d00d0 2024-05-28T14:10:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.316+0000 7f0ccf200640 1 -- 172.21.15.42:0/3335732606 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc81590e0 con 0x7f0cc80d00d0 2024-05-28T14:10:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.316+0000 7f0ccf200640 1 -- 172.21.15.42:0/3335732606 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc814f870 con 0x7f0cc80d00d0 2024-05-28T14:10:15.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.316+0000 7f0ccf200640 1 -- 172.21.15.42:0/3335732606 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc80a0650 con 0x7f0cc80d00d0 2024-05-28T14:10:15.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.316+0000 7f0cc4c00640 1 -- 172.21.15.42:0/3335732606 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 33) ==== 89721+0+0 (secure 0 0 0) 0x7f0cb8018020 con 0x7f0cc80d00d0 2024-05-28T14:10:15.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.320+0000 7f0cc4c00640 1 --2- 172.21.15.42:0/3335732606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ca40696f0 0x7f0ca406bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:15.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.320+0000 7f0cc4c00640 1 -- 172.21.15.42:0/3335732606 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7f0cb80857c0 con 0x7f0cc80d00d0 2024-05-28T14:10:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.320+0000 7f0cbf400640 1 --2- 172.21.15.42:0/3335732606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ca40696f0 0x7f0ca406bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:15.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.320+0000 7f0cbf400640 1 --2- 172.21.15.42:0/3335732606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ca40696f0 0x7f0ca406bbb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0025550 tx=0x7f0cb0004740 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:15.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.324+0000 7f0cc4c00640 1 -- 172.21.15.42:0/3335732606 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0cb801f020 con 0x7f0cc80d00d0 2024-05-28T14:10:15.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.660+0000 7f0ccf200640 1 -- 172.21.15.42:0/3335732606 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f0cc804fdd0 con 0x7f0ca40696f0 2024-05-28T14:10:15.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.680+0000 7f0cc4c00640 1 -- 172.21.15.42:0/3335732606 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f0cc804fdd0 con 0x7f0ca40696f0 2024-05-28T14:10:15.664 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "status": { 2024-05-28T14:10:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "in-progress", 2024-05-28T14:10:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "source": { 2024-05-28T14:10:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "volume": "cephfs", 2024-05-28T14:10:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "subvolume": "sv1", 2024-05-28T14:10:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot": "ss1" 2024-05-28T14:10:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_report": { 2024-05-28T14:10:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "percentage cloned": 13.542, 2024-05-28T14:10:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "amount cloned": "416M/3.0G", 2024-05-28T14:10:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "files cloned": "4/6" 2024-05-28T14:10:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:15.665 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:15.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.684+0000 7f0ccf200640 1 -- 172.21.15.42:0/3335732606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ca40696f0 msgr2=0x7f0ca406bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:15.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.684+0000 7f0ccf200640 1 --2- 172.21.15.42:0/3335732606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ca40696f0 0x7f0ca406bbb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0025550 tx=0x7f0cb0004740 comp rx=0 tx=0).stop 2024-05-28T14:10:15.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.684+0000 7f0ccf200640 1 -- 172.21.15.42:0/3335732606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cc80d00d0 msgr2=0x7f0cc815b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:15.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.684+0000 7f0ccf200640 1 --2- 172.21.15.42:0/3335732606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cc80d00d0 0x7f0cc815b8b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800b3f0 tx=0x7f0cb800b8b0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.684+0000 7f0ccf200640 1 -- 172.21.15.42:0/3335732606 shutdown_connections 2024-05-28T14:10:15.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.684+0000 7f0ccf200640 1 --2- 172.21.15.42:0/3335732606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ca40696f0 0x7f0ca406bbb0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.684+0000 7f0ccf200640 1 --2- 172.21.15.42:0/3335732606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cc80d0ae0 0x7f0cc8154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.684+0000 7f0ccf200640 1 --2- 172.21.15.42:0/3335732606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cc80d0530 0x7f0cc81596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.684+0000 7f0ccf200640 1 --2- 172.21.15.42:0/3335732606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cc80d00d0 0x7f0cc815b8b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.684+0000 7f0ccf200640 1 -- 172.21.15.42:0/3335732606 >> 172.21.15.42:0/3335732606 conn(0x7f0cc80b3ff0 msgr2=0x7f0cc80cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:15.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.684+0000 7f0ccf200640 1 -- 172.21.15.42:0/3335732606 shutdown_connections 2024-05-28T14:10:15.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.684+0000 7f0ccf200640 1 -- 172.21.15.42:0/3335732606 wait complete. 2024-05-28T14:10:15.703 INFO:tasks.cephfs.test_volumes:percentage cloned = 13.542 2024-05-28T14:10:15.703 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-28T14:10:15.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b12a00640 1 Processor -- start 2024-05-28T14:10:15.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b12a00640 1 -- start start 2024-05-28T14:10:15.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b12a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b0c15e870 0x7f5b0c0d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:15.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b12a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5b0c15ec50 0x7f5b0c0b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:15.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b12a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b0c0d1a80 con 0x7f5b0c0b7ba0 2024-05-28T14:10:15.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b12a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5b0c0cd500 con 0x7f5b0c15ec50 2024-05-28T14:10:15.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b12a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5b0c0ce9a0 con 0x7f5b0c15e870 2024-05-28T14:10:15.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b0b400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5b0c15ec50 0x7f5b0c0b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:15.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b0b400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5b0c15ec50 0x7f5b0c0b7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:40496/0 (socket says 172.21.15.42:40496) 2024-05-28T14:10:15.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b0b400640 1 -- 172.21.15.42:0/767811943 learned_addr learned my addr 172.21.15.42:0/767811943 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:15.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b0be00640 1 --2- 172.21.15.42:0/767811943 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b0c15e870 0x7f5b0c0d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:15.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b0b400640 1 -- 172.21.15.42:0/767811943 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b0c15e870 msgr2=0x7f5b0c0d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:15.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b0b400640 1 --2- 172.21.15.42:0/767811943 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b0c15e870 0x7f5b0c0d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b0b400640 1 -- 172.21.15.42:0/767811943 >> v1:172.21.15.63:6789/0 conn(0x7f5b0c0b7ba0 legacy=0x7f5b0c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:15.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b0b400640 1 -- 172.21.15.42:0/767811943 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0c0ccbd0 con 0x7f5b0c15ec50 2024-05-28T14:10:15.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b0aa00640 1 -- 172.21.15.42:0/767811943 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2948112510 0 0) 0x7f5b0c0d1a80 con 0x7f5b0c0b7ba0 2024-05-28T14:10:15.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:15.996+0000 7f5b0be00640 1 --2- 172.21.15.42:0/767811943 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b0c15e870 0x7f5b0c0d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:10:15.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.000+0000 7f5b0b400640 1 --2- 172.21.15.42:0/767811943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c15ec50 0x7f5b0c0b7660 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009e50 tx=0x7f5b0401f210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86515035ef8f26ee server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:15.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.000+0000 7f5b0aa00640 1 -- 172.21.15.42:0/767811943 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b040b4070 con 0x7f5b0c15ec50 2024-05-28T14:10:15.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.000+0000 7f5b0aa00640 1 -- 172.21.15.42:0/767811943 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b040af400 con 0x7f5b0c15ec50 2024-05-28T14:10:15.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.000+0000 7f5b0aa00640 1 -- 172.21.15.42:0/767811943 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b040af7d0 con 0x7f5b0c15ec50 2024-05-28T14:10:15.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 -- 172.21.15.42:0/767811943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c15ec50 msgr2=0x7f5b0c0b7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:15.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 --2- 172.21.15.42:0/767811943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c15ec50 0x7f5b0c0b7660 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009e50 tx=0x7f5b0401f210 comp rx=0 tx=0).stop 2024-05-28T14:10:15.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 -- 172.21.15.42:0/767811943 shutdown_connections 2024-05-28T14:10:15.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 --2- 172.21.15.42:0/767811943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c15ec50 0x7f5b0c0b7660 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 --2- 172.21.15.42:0/767811943 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b0c15e870 0x7f5b0c0d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 -- 172.21.15.42:0/767811943 >> 172.21.15.42:0/767811943 conn(0x7f5b0c0b3fd0 msgr2=0x7f5b0c0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:15.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 -- 172.21.15.42:0/767811943 shutdown_connections 2024-05-28T14:10:15.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 -- 172.21.15.42:0/767811943 wait complete. 2024-05-28T14:10:15.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 Processor -- start 2024-05-28T14:10:15.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 -- start start 2024-05-28T14:10:15.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c0b7ba0 0x7f5b0c0c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:15.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b0c15e870 0x7f5b0c0c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:15.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b0c161850 0x7f5b0c0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c0d1770 con 0x7f5b0c15e870 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c0cd190 con 0x7f5b0c0b7ba0 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.004+0000 7f5b12a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5b0c0b5330 con 0x7f5b0c161850 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b0b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b0c15e870 0x7f5b0c0c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b0b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b0c15e870 0x7f5b0c0c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34748/0 (socket says 172.21.15.42:34748) 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b0b400640 1 -- 172.21.15.42:0/3280445604 learned_addr learned my addr 172.21.15.42:0/3280445604 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b0be00640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c0b7ba0 0x7f5b0c0c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b0b400640 1 -- 172.21.15.42:0/3280445604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b0c161850 msgr2=0x7f5b0c0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b10a00640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b0c161850 0x7f5b0c0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b0b400640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b0c161850 0x7f5b0c0caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b0b400640 1 -- 172.21.15.42:0/3280445604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c0b7ba0 msgr2=0x7f5b0c0c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b0b400640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c0b7ba0 0x7f5b0c0c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b0b400640 1 -- 172.21.15.42:0/3280445604 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0c0c6750 con 0x7f5b0c15e870 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b10a00640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b0c161850 0x7f5b0c0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:10:15.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b0be00640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c0b7ba0 0x7f5b0c0c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:10:15.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b0b400640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b0c15e870 0x7f5b0c0c5fd0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009e50 tx=0x7f5b0401bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:15.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b03e00640 1 -- 172.21.15.42:0/3280445604 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b040b4040 con 0x7f5b0c15e870 2024-05-28T14:10:15.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b03e00640 1 -- 172.21.15.42:0/3280445604 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b040090d0 con 0x7f5b0c15e870 2024-05-28T14:10:15.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b12a00640 1 -- 172.21.15.42:0/3280445604 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0c150fb0 con 0x7f5b0c15e870 2024-05-28T14:10:15.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b03e00640 1 -- 172.21.15.42:0/3280445604 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04009510 con 0x7f5b0c15e870 2024-05-28T14:10:15.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b12a00640 1 -- 172.21.15.42:0/3280445604 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b0c1504d0 con 0x7f5b0c15e870 2024-05-28T14:10:15.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b12a00640 1 -- 172.21.15.42:0/3280445604 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b0c09bda0 con 0x7f5b0c15e870 2024-05-28T14:10:15.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.008+0000 7f5b03e00640 1 -- 172.21.15.42:0/3280445604 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7f5b040c3060 con 0x7f5b0c15e870 2024-05-28T14:10:15.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.012+0000 7f5b03e00640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ae8069730 0x7f5ae806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:15.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.012+0000 7f5b03e00640 1 -- 172.21.15.42:0/3280445604 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7f5b04122bc0 con 0x7f5b0c15e870 2024-05-28T14:10:15.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.012+0000 7f5b0be00640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ae8069730 0x7f5ae806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:15.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.012+0000 7f5b0be00640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ae8069730 0x7f5ae806bbf0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009a40 tx=0x7f5af4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:16.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.016+0000 7f5b03e00640 1 -- 172.21.15.42:0/3280445604 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b04018d70 con 0x7f5b0c15e870 2024-05-28T14:10:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.364+0000 7f5b12a00640 1 -- 172.21.15.42:0/3280445604 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f5b0c04fdd0 con 0x7f5ae8069730 2024-05-28T14:10:16.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.416+0000 7f5b03e00640 1 -- 172.21.15.42:0/3280445604 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7f5b0c04fdd0 con 0x7f5ae8069730 2024-05-28T14:10:16.402 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:16.402 INFO:teuthology.orchestra.run.smithi042.stdout: "status": { 2024-05-28T14:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "in-progress", 2024-05-28T14:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout: "source": { 2024-05-28T14:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout: "volume": "cephfs", 2024-05-28T14:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout: "subvolume": "sv1", 2024-05-28T14:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot": "ss1" 2024-05-28T14:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_report": { 2024-05-28T14:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout: "percentage cloned": 30.99, 2024-05-28T14:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout: "amount cloned": "952M/3.0G", 2024-05-28T14:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout: "files cloned": "4/6" 2024-05-28T14:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:16.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.424+0000 7f5b12a00640 1 -- 172.21.15.42:0/3280445604 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ae8069730 msgr2=0x7f5ae806bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:16.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.424+0000 7f5b12a00640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ae8069730 0x7f5ae806bbf0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009a40 tx=0x7f5af4008040 comp rx=0 tx=0).stop 2024-05-28T14:10:16.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.424+0000 7f5b12a00640 1 -- 172.21.15.42:0/3280445604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b0c15e870 msgr2=0x7f5b0c0c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:16.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.424+0000 7f5b12a00640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b0c15e870 0x7f5b0c0c5fd0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009e50 tx=0x7f5b0401bfb0 comp rx=0 tx=0).stop 2024-05-28T14:10:16.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.424+0000 7f5b12a00640 1 -- 172.21.15.42:0/3280445604 shutdown_connections 2024-05-28T14:10:16.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.424+0000 7f5b12a00640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ae8069730 0x7f5ae806bbf0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:16.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.424+0000 7f5b12a00640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b0c161850 0x7f5b0c0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:16.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.424+0000 7f5b12a00640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b0c15e870 0x7f5b0c0c5fd0 secure :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009e50 tx=0x7f5b0401bfb0 comp rx=0 tx=0).stop 2024-05-28T14:10:16.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.424+0000 7f5b12a00640 1 --2- 172.21.15.42:0/3280445604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c0b7ba0 0x7f5b0c0c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:16.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.424+0000 7f5b12a00640 1 -- 172.21.15.42:0/3280445604 >> 172.21.15.42:0/3280445604 conn(0x7f5b0c0b3fd0 msgr2=0x7f5b0c0b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:16.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.424+0000 7f5b12a00640 1 -- 172.21.15.42:0/3280445604 shutdown_connections 2024-05-28T14:10:16.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.424+0000 7f5b12a00640 1 -- 172.21.15.42:0/3280445604 wait complete. 2024-05-28T14:10:16.454 INFO:tasks.cephfs.test_volumes:percentage cloned = 30.99 2024-05-28T14:10:16.454 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-28T14:10:16.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.716+0000 7efd51400640 1 Processor -- start 2024-05-28T14:10:16.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.716+0000 7efd51400640 1 -- start start 2024-05-28T14:10:16.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.720+0000 7efd51400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efd4c06e740 0x7efd4c070090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:16.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.720+0000 7efd51400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd4c0d08c0 con 0x7efd4c162ab0 2024-05-28T14:10:16.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.720+0000 7efd51400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd4c0d0ba0 con 0x7efd4c0d0400 2024-05-28T14:10:16.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.720+0000 7efd51400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efd4c0d14d0 con 0x7efd4c06e740 2024-05-28T14:10:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.720+0000 7efd50a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7efd4c162ab0 0x7efd4c15df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34192/0 (socket says 172.21.15.42:34192) 2024-05-28T14:10:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.720+0000 7efd50a00640 1 -- 172.21.15.42:0/599940073 learned_addr learned my addr 172.21.15.42:0/599940073 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.720+0000 7efd4be00640 1 --2- 172.21.15.42:0/599940073 >> v2:172.21.15.63:3300/0 conn(0x7efd4c06e740 0x7efd4c070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.720+0000 7efd4aa00640 1 -- 172.21.15.42:0/599940073 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 414618883 0 0) 0x7efd4c0d08c0 con 0x7efd4c162ab0 2024-05-28T14:10:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.720+0000 7efd4aa00640 1 -- 172.21.15.42:0/599940073 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd30003530 con 0x7efd4c162ab0 2024-05-28T14:10:16.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.720+0000 7efd4be00640 1 -- 172.21.15.42:0/599940073 >> v1:172.21.15.63:6789/0 conn(0x7efd4c0d0400 legacy=0x7efd4c1623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:16.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.720+0000 7efd4be00640 1 -- 172.21.15.42:0/599940073 >> v1:172.21.15.42:6789/0 conn(0x7efd4c162ab0 legacy=0x7efd4c15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:16.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.720+0000 7efd4be00640 1 -- 172.21.15.42:0/599940073 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4c0d0e80 con 0x7efd4c06e740 2024-05-28T14:10:16.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.724+0000 7efd4be00640 1 --2- 172.21.15.42:0/599940073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd4c06e740 0x7efd4c070090 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0097c0 tx=0x7efd3c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a40862e1b50f186 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.724+0000 7efd4aa00640 1 -- 172.21.15.42:0/599940073 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c0b2070 con 0x7efd4c06e740 2024-05-28T14:10:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.724+0000 7efd4aa00640 1 -- 172.21.15.42:0/599940073 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd3c0ad680 con 0x7efd4c06e740 2024-05-28T14:10:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.724+0000 7efd4aa00640 1 -- 172.21.15.42:0/599940073 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c0ada30 con 0x7efd4c06e740 2024-05-28T14:10:16.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.728+0000 7efd51400640 1 -- 172.21.15.42:0/599940073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd4c06e740 msgr2=0x7efd4c070090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:16.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.728+0000 7efd51400640 1 --2- 172.21.15.42:0/599940073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd4c06e740 0x7efd4c070090 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0097c0 tx=0x7efd3c01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:10:16.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.728+0000 7efd51400640 1 -- 172.21.15.42:0/599940073 shutdown_connections 2024-05-28T14:10:16.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.728+0000 7efd51400640 1 --2- 172.21.15.42:0/599940073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd4c06e740 0x7efd4c070090 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:16.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.728+0000 7efd51400640 1 -- 172.21.15.42:0/599940073 >> 172.21.15.42:0/599940073 conn(0x7efd4c0b3dd0 msgr2=0x7efd4c0b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:16.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.728+0000 7efd51400640 1 -- 172.21.15.42:0/599940073 shutdown_connections 2024-05-28T14:10:16.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.728+0000 7efd51400640 1 -- 172.21.15.42:0/599940073 wait complete. 2024-05-28T14:10:16.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.728+0000 7efd51400640 1 Processor -- start 2024-05-28T14:10:16.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd51400640 1 -- start start 2024-05-28T14:10:16.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd51400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4c06e740 0x7efd4c153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:16.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd51400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd4c0d0400 0x7efd4c0b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:16.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd51400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c162ab0 0x7efd4c1609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:16.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd51400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c16ca80 con 0x7efd4c06e740 2024-05-28T14:10:16.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd51400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c16c900 con 0x7efd4c0d0400 2024-05-28T14:10:16.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd51400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd4c16cc00 con 0x7efd4c162ab0 2024-05-28T14:10:16.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd4be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4c06e740 0x7efd4c153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:16.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd4be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4c06e740 0x7efd4c153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34758/0 (socket says 172.21.15.42:34758) 2024-05-28T14:10:16.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd4be00640 1 -- 172.21.15.42:0/3389593010 learned_addr learned my addr 172.21.15.42:0/3389593010 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:16.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd50a00640 1 --2- 172.21.15.42:0/3389593010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c162ab0 0x7efd4c1609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:16.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd4b400640 1 --2- 172.21.15.42:0/3389593010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd4c0d0400 0x7efd4c0b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:16.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd4be00640 1 -- 172.21.15.42:0/3389593010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c162ab0 msgr2=0x7efd4c1609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:16.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd4be00640 1 --2- 172.21.15.42:0/3389593010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c162ab0 0x7efd4c1609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:16.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd4be00640 1 -- 172.21.15.42:0/3389593010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd4c0d0400 msgr2=0x7efd4c0b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:16.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd4be00640 1 --2- 172.21.15.42:0/3389593010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd4c0d0400 0x7efd4c0b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:16.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd4be00640 1 -- 172.21.15.42:0/3389593010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4c0c6cc0 con 0x7efd4c06e740 2024-05-28T14:10:16.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd4be00640 1 --2- 172.21.15.42:0/3389593010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4c06e740 0x7efd4c153450 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0b0040 tx=0x7efd3c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:16.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd43e00640 1 -- 172.21.15.42:0/3389593010 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c0b2050 con 0x7efd4c06e740 2024-05-28T14:10:16.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd43e00640 1 -- 172.21.15.42:0/3389593010 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd3c0b3040 con 0x7efd4c06e740 2024-05-28T14:10:16.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd43e00640 1 -- 172.21.15.42:0/3389593010 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c004020 con 0x7efd4c06e740 2024-05-28T14:10:16.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd51400640 1 -- 172.21.15.42:0/3389593010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4c0c6fa0 con 0x7efd4c06e740 2024-05-28T14:10:16.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.732+0000 7efd51400640 1 -- 172.21.15.42:0/3389593010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd4c0c7560 con 0x7efd4c06e740 2024-05-28T14:10:16.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.736+0000 7efd51400640 1 -- 172.21.15.42:0/3389593010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd4c04fcf0 con 0x7efd4c06e740 2024-05-28T14:10:16.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.736+0000 7efd43e00640 1 -- 172.21.15.42:0/3389593010 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7efd3c0c0060 con 0x7efd4c06e740 2024-05-28T14:10:16.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.740+0000 7efd43e00640 1 --2- 172.21.15.42:0/3389593010 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efd2c069730 0x7efd2c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:16.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.740+0000 7efd43e00640 1 -- 172.21.15.42:0/3389593010 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7efd3c11f290 con 0x7efd4c06e740 2024-05-28T14:10:16.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.740+0000 7efd4b400640 1 --2- 172.21.15.42:0/3389593010 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efd2c069730 0x7efd2c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:16.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.740+0000 7efd4b400640 1 --2- 172.21.15.42:0/3389593010 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efd2c069730 0x7efd2c06bbf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7efd4c1fb540 tx=0x7efd38009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:16.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:16.744+0000 7efd43e00640 1 -- 172.21.15.42:0/3389593010 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efd3c016090 con 0x7efd4c06e740 2024-05-28T14:10:17.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.036+0000 7efd51400640 1 -- 172.21.15.42:0/3389593010 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7efd4c070670 con 0x7efd2c069730 2024-05-28T14:10:17.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.064+0000 7efd43e00640 1 -- 172.21.15.42:0/3389593010 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7efd4c070670 con 0x7efd2c069730 2024-05-28T14:10:17.049 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "status": { 2024-05-28T14:10:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "in-progress", 2024-05-28T14:10:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "source": { 2024-05-28T14:10:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "volume": "cephfs", 2024-05-28T14:10:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "subvolume": "sv1", 2024-05-28T14:10:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot": "ss1" 2024-05-28T14:10:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_report": { 2024-05-28T14:10:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "percentage cloned": 49.219, 2024-05-28T14:10:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "amount cloned": "1.4G/3.0G", 2024-05-28T14:10:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "files cloned": "5/6" 2024-05-28T14:10:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:17.050 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:17.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.068+0000 7efd51400640 1 -- 172.21.15.42:0/3389593010 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efd2c069730 msgr2=0x7efd2c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:17.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.068+0000 7efd51400640 1 --2- 172.21.15.42:0/3389593010 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efd2c069730 0x7efd2c06bbf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7efd4c1fb540 tx=0x7efd38009040 comp rx=0 tx=0).stop 2024-05-28T14:10:17.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.068+0000 7efd51400640 1 -- 172.21.15.42:0/3389593010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4c06e740 msgr2=0x7efd4c153450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:17.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.068+0000 7efd51400640 1 --2- 172.21.15.42:0/3389593010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4c06e740 0x7efd4c153450 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0b0040 tx=0x7efd3c01ace0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.068+0000 7efd51400640 1 -- 172.21.15.42:0/3389593010 shutdown_connections 2024-05-28T14:10:17.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.068+0000 7efd51400640 1 --2- 172.21.15.42:0/3389593010 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efd2c069730 0x7efd2c06bbf0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.068+0000 7efd51400640 1 --2- 172.21.15.42:0/3389593010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c162ab0 0x7efd4c1609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.068+0000 7efd51400640 1 --2- 172.21.15.42:0/3389593010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd4c0d0400 0x7efd4c0b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.068+0000 7efd51400640 1 --2- 172.21.15.42:0/3389593010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4c06e740 0x7efd4c153450 secure :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0b0040 tx=0x7efd3c01ace0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.068+0000 7efd51400640 1 -- 172.21.15.42:0/3389593010 >> 172.21.15.42:0/3389593010 conn(0x7efd4c0b3dd0 msgr2=0x7efd4c0af130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:17.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.068+0000 7efd51400640 1 -- 172.21.15.42:0/3389593010 shutdown_connections 2024-05-28T14:10:17.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.068+0000 7efd51400640 1 -- 172.21.15.42:0/3389593010 wait complete. 2024-05-28T14:10:17.111 INFO:tasks.cephfs.test_volumes:percentage cloned = 49.219 2024-05-28T14:10:17.111 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-28T14:10:17.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f8678000640 1 Processor -- start 2024-05-28T14:10:17.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f8678000640 1 -- start start 2024-05-28T14:10:17.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f8678000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f86700b4670 0x7f867006b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:17.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f8678000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8670070d60 con 0x7f8670144fa0 2024-05-28T14:10:17.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f8678000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86700ae590 con 0x7f867006bf30 2024-05-28T14:10:17.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f8678000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f867006c3a0 con 0x7f86700b4670 2024-05-28T14:10:17.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f8675600640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8670144fa0 0x7f8670070650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:49444/0 (socket says 172.21.15.42:49444) 2024-05-28T14:10:17.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f8675600640 1 -- 172.21.15.42:0/1271548748 learned_addr learned my addr 172.21.15.42:0/1271548748 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:17.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f866fe00640 1 -- 172.21.15.42:0/1271548748 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2144676703 0 0) 0x7f8670070d60 con 0x7f8670144fa0 2024-05-28T14:10:17.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f8674c00640 1 --2- 172.21.15.42:0/1271548748 >> v2:172.21.15.63:3300/0 conn(0x7f86700b4670 0x7f867006b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:17.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f866fe00640 1 -- 172.21.15.42:0/1271548748 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8658003530 con 0x7f8670144fa0 2024-05-28T14:10:17.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f866fe00640 1 -- 172.21.15.42:0/1271548748 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3493260803 0 0) 0x7f8658003530 con 0x7f8670144fa0 2024-05-28T14:10:17.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f866fe00640 1 -- 172.21.15.42:0/1271548748 >> v2:172.21.15.63:3300/0 conn(0x7f86700b4670 msgr2=0x7f867006b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:17.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f866fe00640 1 --2- 172.21.15.42:0/1271548748 >> v2:172.21.15.63:3300/0 conn(0x7f86700b4670 0x7f867006b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f866fe00640 1 -- 172.21.15.42:0/1271548748 >> v1:172.21.15.63:6789/0 conn(0x7f867006bf30 legacy=0x7f86700af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:17.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f866fe00640 1 -- 172.21.15.42:0/1271548748 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86700ae870 con 0x7f8670144fa0 2024-05-28T14:10:17.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f866fe00640 1 -- 172.21.15.42:0/1271548748 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f865c001680 con 0x7f8670144fa0 2024-05-28T14:10:17.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f866fe00640 1 -- 172.21.15.42:0/1271548748 >> v1:172.21.15.42:6789/0 conn(0x7f8670144fa0 legacy=0x7f8670070650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:17.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f866fe00640 1 --2- 172.21.15.42:0/1271548748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86580039f0 0x7f8658003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:17.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.384+0000 7f866fe00640 1 --2- 172.21.15.42:0/1271548748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8658004810 0x7f8658004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.388+0000 7f866fe00640 1 --2- 172.21.15.42:0/1271548748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8658005220 0x7f865800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.388+0000 7f866fe00640 1 -- 172.21.15.42:0/1271548748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f867006c3a0 con 0x7f8658005220 2024-05-28T14:10:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.388+0000 7f866fe00640 1 -- 172.21.15.42:0/1271548748 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f865800c150 con 0x7f8658004810 2024-05-28T14:10:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.388+0000 7f866fe00640 1 -- 172.21.15.42:0/1271548748 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f865800c350 con 0x7f86580039f0 2024-05-28T14:10:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.388+0000 7f8676000640 1 --2- 172.21.15.42:0/1271548748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8658005220 0x7f865800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.388+0000 7f8675600640 1 --2- 172.21.15.42:0/1271548748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86580039f0 0x7f8658003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.388+0000 7f8676000640 1 -- 172.21.15.42:0/1271548748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86580039f0 msgr2=0x7f8658003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.388+0000 7f8676000640 1 --2- 172.21.15.42:0/1271548748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86580039f0 0x7f8658003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.388+0000 7f8676000640 1 -- 172.21.15.42:0/1271548748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8658004810 msgr2=0x7f8658004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:17.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.388+0000 7f8676000640 1 --2- 172.21.15.42:0/1271548748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8658004810 0x7f8658004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.388+0000 7f8676000640 1 -- 172.21.15.42:0/1271548748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8658003530 con 0x7f8658005220 2024-05-28T14:10:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.392+0000 7f8676000640 1 -- 172.21.15.42:0/1271548748 reap_dead start 2024-05-28T14:10:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.392+0000 7f8676000640 1 --2- 172.21.15.42:0/1271548748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8658005220 0x7f865800b9d0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8668001d50 tx=0x7f866801c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272c21d16810a68a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.392+0000 7f866fe00640 1 -- 172.21.15.42:0/1271548748 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668001950 con 0x7f8658005220 2024-05-28T14:10:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.392+0000 7f866fe00640 1 -- 172.21.15.42:0/1271548748 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8668003b80 con 0x7f8658005220 2024-05-28T14:10:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.392+0000 7f866fe00640 1 -- 172.21.15.42:0/1271548748 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668005070 con 0x7f8658005220 2024-05-28T14:10:17.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.392+0000 7f8678000640 1 -- 172.21.15.42:0/1271548748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8658005220 msgr2=0x7f865800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:17.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.392+0000 7f8678000640 1 --2- 172.21.15.42:0/1271548748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8658005220 0x7f865800b9d0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8668001d50 tx=0x7f866801c680 comp rx=0 tx=0).stop 2024-05-28T14:10:17.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.392+0000 7f8678000640 1 -- 172.21.15.42:0/1271548748 shutdown_connections 2024-05-28T14:10:17.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.392+0000 7f8678000640 1 --2- 172.21.15.42:0/1271548748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8658005220 0x7f865800b9d0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.392+0000 7f8678000640 1 -- 172.21.15.42:0/1271548748 >> 172.21.15.42:0/1271548748 conn(0x7f86700b6880 msgr2=0x7f86700b6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:17.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.392+0000 7f8678000640 1 -- 172.21.15.42:0/1271548748 shutdown_connections 2024-05-28T14:10:17.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.392+0000 7f8678000640 1 -- 172.21.15.42:0/1271548748 wait complete. 2024-05-28T14:10:17.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8678000640 1 Processor -- start 2024-05-28T14:10:17.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8678000640 1 -- start start 2024-05-28T14:10:17.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8678000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8658005220 0x7f8670141760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:17.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8678000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86700ca6e0 0x7f86701602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:17.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8678000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8670160810 0x7f867015e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:17.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8678000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f86700ae340 con 0x7f8658005220 2024-05-28T14:10:17.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8678000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86700aeec0 con 0x7f86700ca6e0 2024-05-28T14:10:17.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8678000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8670145380 con 0x7f8670160810 2024-05-28T14:10:17.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8675600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8658005220 0x7f8670141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:17.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8676000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8670160810 0x7f867015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:17.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8676000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8670160810 0x7f867015e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:37942/0 (socket says 172.21.15.42:37942) 2024-05-28T14:10:17.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8675600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8658005220 0x7f8670141760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48422/0 (socket says 172.21.15.42:48422) 2024-05-28T14:10:17.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8676000640 1 -- 172.21.15.42:0/3428161128 learned_addr learned my addr 172.21.15.42:0/3428161128 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:17.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8674c00640 1 --2- 172.21.15.42:0/3428161128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86700ca6e0 0x7f86701602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:17.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8676000640 1 -- 172.21.15.42:0/3428161128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86700ca6e0 msgr2=0x7f86701602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:17.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8676000640 1 --2- 172.21.15.42:0/3428161128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86700ca6e0 0x7f86701602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8676000640 1 -- 172.21.15.42:0/3428161128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8658005220 msgr2=0x7f8670141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:17.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8676000640 1 --2- 172.21.15.42:0/3428161128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8658005220 0x7f8670141760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.396+0000 7f8676000640 1 -- 172.21.15.42:0/3428161128 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86701421e0 con 0x7f8670160810 2024-05-28T14:10:17.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.400+0000 7f8676000640 1 --2- 172.21.15.42:0/3428161128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8670160810 0x7f867015e400 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f866801c4b0 tx=0x7f8658005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:17.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.400+0000 7f866d600640 1 -- 172.21.15.42:0/3428161128 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668007de0 con 0x7f8670160810 2024-05-28T14:10:17.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.400+0000 7f866d600640 1 -- 172.21.15.42:0/3428161128 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8668001930 con 0x7f8670160810 2024-05-28T14:10:17.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.400+0000 7f866d600640 1 -- 172.21.15.42:0/3428161128 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668006b40 con 0x7f8670160810 2024-05-28T14:10:17.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.400+0000 7f8678000640 1 -- 172.21.15.42:0/3428161128 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86701424c0 con 0x7f8670160810 2024-05-28T14:10:17.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.400+0000 7f8678000640 1 -- 172.21.15.42:0/3428161128 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8670142a80 con 0x7f8670160810 2024-05-28T14:10:17.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.400+0000 7f8678000640 1 -- 172.21.15.42:0/3428161128 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867009f6c0 con 0x7f8670160810 2024-05-28T14:10:17.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.404+0000 7f866d600640 1 -- 172.21.15.42:0/3428161128 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7f86680a7070 con 0x7f8670160810 2024-05-28T14:10:17.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.404+0000 7f866d600640 1 --2- 172.21.15.42:0/3428161128 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8650069730 0x7f865006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:17.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.404+0000 7f866d600640 1 -- 172.21.15.42:0/3428161128 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7f866811ef70 con 0x7f8670160810 2024-05-28T14:10:17.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.404+0000 7f8675600640 1 --2- 172.21.15.42:0/3428161128 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8650069730 0x7f865006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:17.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.408+0000 7f8675600640 1 --2- 172.21.15.42:0/3428161128 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8650069730 0x7f865006bbf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f865c000c10 tx=0x7f865c007040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:17.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.416+0000 7f866d600640 1 -- 172.21.15.42:0/3428161128 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f86680ba030 con 0x7f8670160810 2024-05-28T14:10:17.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.764+0000 7f8678000640 1 -- 172.21.15.42:0/3428161128 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f867004fdd0 con 0x7f8650069730 2024-05-28T14:10:17.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.784+0000 7f866d600640 1 -- 172.21.15.42:0/3428161128 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f867004fdd0 con 0x7f8650069730 2024-05-28T14:10:17.767 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "status": { 2024-05-28T14:10:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "in-progress", 2024-05-28T14:10:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "source": { 2024-05-28T14:10:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "volume": "cephfs", 2024-05-28T14:10:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "subvolume": "sv1", 2024-05-28T14:10:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot": "ss1" 2024-05-28T14:10:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_report": { 2024-05-28T14:10:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: "percentage cloned": 49.219, 2024-05-28T14:10:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: "amount cloned": "1.4G/3.0G", 2024-05-28T14:10:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: "files cloned": "5/6" 2024-05-28T14:10:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:17.768 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:17.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.784+0000 7f8678000640 1 -- 172.21.15.42:0/3428161128 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8650069730 msgr2=0x7f865006bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:17.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.784+0000 7f8678000640 1 --2- 172.21.15.42:0/3428161128 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8650069730 0x7f865006bbf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f865c000c10 tx=0x7f865c007040 comp rx=0 tx=0).stop 2024-05-28T14:10:17.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.784+0000 7f8678000640 1 -- 172.21.15.42:0/3428161128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8670160810 msgr2=0x7f867015e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:17.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.784+0000 7f8678000640 1 --2- 172.21.15.42:0/3428161128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8670160810 0x7f867015e400 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f866801c4b0 tx=0x7f8658005840 comp rx=0 tx=0).stop 2024-05-28T14:10:17.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.788+0000 7f8678000640 1 -- 172.21.15.42:0/3428161128 shutdown_connections 2024-05-28T14:10:17.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.788+0000 7f8678000640 1 --2- 172.21.15.42:0/3428161128 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8650069730 0x7f865006bbf0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.788+0000 7f8678000640 1 --2- 172.21.15.42:0/3428161128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8670160810 0x7f867015e400 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.788+0000 7f8678000640 1 --2- 172.21.15.42:0/3428161128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86700ca6e0 0x7f86701602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.788+0000 7f8678000640 1 --2- 172.21.15.42:0/3428161128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8658005220 0x7f8670141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:17.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.788+0000 7f8678000640 1 -- 172.21.15.42:0/3428161128 >> 172.21.15.42:0/3428161128 conn(0x7f86700b6880 msgr2=0x7f86700b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:17.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.788+0000 7f8678000640 1 -- 172.21.15.42:0/3428161128 shutdown_connections 2024-05-28T14:10:17.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:17.788+0000 7f8678000640 1 -- 172.21.15.42:0/3428161128 wait complete. 2024-05-28T14:10:17.819 INFO:tasks.cephfs.test_volumes:percentage cloned = 49.219 2024-05-28T14:10:17.819 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-28T14:10:18.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe246800640 1 Processor -- start 2024-05-28T14:10:18.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe246800640 1 -- start start 2024-05-28T14:10:18.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe246800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe240140d20 0x7fe240167490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe246800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe240160230 con 0x7fe2400ce3b0 2024-05-28T14:10:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe246800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe240160510 con 0x7fe240071de0 2024-05-28T14:10:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe246800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe2401607f0 con 0x7fe240140d20 2024-05-28T14:10:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe236a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe2400ce3b0 0x7fe2400ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:49452/0 (socket says 172.21.15.42:49452) 2024-05-28T14:10:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe236a00640 1 -- 172.21.15.42:0/570930107 learned_addr learned my addr 172.21.15.42:0/570930107 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe23fe00640 1 --2- 172.21.15.42:0/570930107 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe240140d20 0x7fe240167490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe23ea00640 1 -- 172.21.15.42:0/570930107 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1879768381 0 0) 0x7fe240160230 con 0x7fe2400ce3b0 2024-05-28T14:10:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe23ea00640 1 -- 172.21.15.42:0/570930107 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe224003530 con 0x7fe2400ce3b0 2024-05-28T14:10:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe23fe00640 1 -- 172.21.15.42:0/570930107 >> v1:172.21.15.63:6789/0 conn(0x7fe240071de0 legacy=0x7fe2400721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe23ea00640 1 -- 172.21.15.42:0/570930107 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 345561086 0 0) 0x7fe240160510 con 0x7fe240071de0 2024-05-28T14:10:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe23fe00640 1 -- 172.21.15.42:0/570930107 >> v1:172.21.15.42:6789/0 conn(0x7fe2400ce3b0 legacy=0x7fe2400ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.072+0000 7fe23fe00640 1 -- 172.21.15.42:0/570930107 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe240160ad0 con 0x7fe240140d20 2024-05-28T14:10:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.076+0000 7fe23fe00640 1 --2- 172.21.15.42:0/570930107 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe240140d20 0x7fe240167490 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe238009dc0 tx=0x7fe23801c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a480fb638e44b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.076+0000 7fe23ea00640 1 -- 172.21.15.42:0/570930107 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2380b2070 con 0x7fe240140d20 2024-05-28T14:10:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.076+0000 7fe23ea00640 1 -- 172.21.15.42:0/570930107 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2380a5730 con 0x7fe240140d20 2024-05-28T14:10:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.076+0000 7fe23ea00640 1 -- 172.21.15.42:0/570930107 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2380a5b00 con 0x7fe240140d20 2024-05-28T14:10:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.076+0000 7fe246800640 1 -- 172.21.15.42:0/570930107 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe240140d20 msgr2=0x7fe240167490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.076+0000 7fe246800640 1 --2- 172.21.15.42:0/570930107 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe240140d20 0x7fe240167490 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe238009dc0 tx=0x7fe23801c870 comp rx=0 tx=0).stop 2024-05-28T14:10:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.076+0000 7fe246800640 1 -- 172.21.15.42:0/570930107 shutdown_connections 2024-05-28T14:10:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.076+0000 7fe246800640 1 --2- 172.21.15.42:0/570930107 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe240140d20 0x7fe240167490 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.076+0000 7fe246800640 1 -- 172.21.15.42:0/570930107 >> 172.21.15.42:0/570930107 conn(0x7fe2400b21f0 msgr2=0x7fe2400b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.076+0000 7fe246800640 1 -- 172.21.15.42:0/570930107 shutdown_connections 2024-05-28T14:10:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.076+0000 7fe246800640 1 -- 172.21.15.42:0/570930107 wait complete. 2024-05-28T14:10:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe246800640 1 Processor -- start 2024-05-28T14:10:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe246800640 1 -- start start 2024-05-28T14:10:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe246800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe240071de0 0x7fe240155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe246800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2400ce3b0 0x7fe2400b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe246800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe240140d20 0x7fe24015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe246800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe24016a990 con 0x7fe2400ce3b0 2024-05-28T14:10:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe246800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe24016a810 con 0x7fe240071de0 2024-05-28T14:10:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe246800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe24016ab10 con 0x7fe240140d20 2024-05-28T14:10:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe236a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2400ce3b0 0x7fe2400b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe236a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2400ce3b0 0x7fe2400b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48434/0 (socket says 172.21.15.42:48434) 2024-05-28T14:10:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe236a00640 1 -- 172.21.15.42:0/3857502770 learned_addr learned my addr 172.21.15.42:0/3857502770 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe23f400640 1 --2- 172.21.15.42:0/3857502770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe240071de0 0x7fe240155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe23fe00640 1 --2- 172.21.15.42:0/3857502770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe240140d20 0x7fe24015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe236a00640 1 -- 172.21.15.42:0/3857502770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe240140d20 msgr2=0x7fe24015cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe236a00640 1 --2- 172.21.15.42:0/3857502770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe240140d20 0x7fe24015cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe236a00640 1 -- 172.21.15.42:0/3857502770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe240071de0 msgr2=0x7fe240155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe236a00640 1 --2- 172.21.15.42:0/3857502770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe240071de0 0x7fe240155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe236a00640 1 -- 172.21.15.42:0/3857502770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24012e620 con 0x7fe2400ce3b0 2024-05-28T14:10:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe236a00640 1 --2- 172.21.15.42:0/3857502770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2400ce3b0 0x7fe2400b1230 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c450 tx=0x7fe22800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe237e00640 1 -- 172.21.15.42:0/3857502770 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228017070 con 0x7fe2400ce3b0 2024-05-28T14:10:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe237e00640 1 -- 172.21.15.42:0/3857502770 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe228005cf0 con 0x7fe2400ce3b0 2024-05-28T14:10:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe246800640 1 -- 172.21.15.42:0/3857502770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24013c740 con 0x7fe2400ce3b0 2024-05-28T14:10:18.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe237e00640 1 -- 172.21.15.42:0/3857502770 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228007290 con 0x7fe2400ce3b0 2024-05-28T14:10:18.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.080+0000 7fe246800640 1 -- 172.21.15.42:0/3857502770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe24013cfe0 con 0x7fe2400ce3b0 2024-05-28T14:10:18.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.084+0000 7fe246800640 1 -- 172.21.15.42:0/3857502770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe24013cb30 con 0x7fe2400ce3b0 2024-05-28T14:10:18.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.084+0000 7fe237e00640 1 -- 172.21.15.42:0/3857502770 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7fe22801a070 con 0x7fe2400ce3b0 2024-05-28T14:10:18.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.084+0000 7fe237e00640 1 --2- 172.21.15.42:0/3857502770 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe21c069730 0x7fe21c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:18.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.084+0000 7fe237e00640 1 -- 172.21.15.42:0/3857502770 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7fe228084d30 con 0x7fe2400ce3b0 2024-05-28T14:10:18.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.084+0000 7fe23f400640 1 --2- 172.21.15.42:0/3857502770 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe21c069730 0x7fe21c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:18.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.088+0000 7fe23f400640 1 --2- 172.21.15.42:0/3857502770 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe21c069730 0x7fe21c06bbf0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe230000f70 tx=0x7fe230009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:18.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.088+0000 7fe237e00640 1 -- 172.21.15.42:0/3857502770 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe24012e620 con 0x7fe2400ce3b0 2024-05-28T14:10:18.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.404+0000 7fe246800640 1 -- 172.21.15.42:0/3857502770 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fe2400738b0 con 0x7fe21c069730 2024-05-28T14:10:18.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.432+0000 7fe237e00640 1 -- 172.21.15.42:0/3857502770 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7fe2400738b0 con 0x7fe21c069730 2024-05-28T14:10:18.419 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "status": { 2024-05-28T14:10:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "in-progress", 2024-05-28T14:10:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "source": { 2024-05-28T14:10:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "volume": "cephfs", 2024-05-28T14:10:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "subvolume": "sv1", 2024-05-28T14:10:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot": "ss1" 2024-05-28T14:10:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_report": { 2024-05-28T14:10:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "percentage cloned": 66.667, 2024-05-28T14:10:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "amount cloned": "2.0G/3.0G", 2024-05-28T14:10:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: "files cloned": "5/6" 2024-05-28T14:10:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:18.420 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:18.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.440+0000 7fe246800640 1 -- 172.21.15.42:0/3857502770 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe21c069730 msgr2=0x7fe21c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:18.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.440+0000 7fe246800640 1 --2- 172.21.15.42:0/3857502770 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe21c069730 0x7fe21c06bbf0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe230000f70 tx=0x7fe230009040 comp rx=0 tx=0).stop 2024-05-28T14:10:18.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.440+0000 7fe246800640 1 -- 172.21.15.42:0/3857502770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2400ce3b0 msgr2=0x7fe2400b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:18.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.440+0000 7fe246800640 1 --2- 172.21.15.42:0/3857502770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2400ce3b0 0x7fe2400b1230 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c450 tx=0x7fe22800c910 comp rx=0 tx=0).stop 2024-05-28T14:10:18.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.440+0000 7fe246800640 1 -- 172.21.15.42:0/3857502770 shutdown_connections 2024-05-28T14:10:18.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.440+0000 7fe246800640 1 --2- 172.21.15.42:0/3857502770 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe21c069730 0x7fe21c06bbf0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:18.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.440+0000 7fe246800640 1 --2- 172.21.15.42:0/3857502770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe240140d20 0x7fe24015cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:18.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.440+0000 7fe246800640 1 --2- 172.21.15.42:0/3857502770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2400ce3b0 0x7fe2400b1230 secure :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c450 tx=0x7fe22800c910 comp rx=0 tx=0).stop 2024-05-28T14:10:18.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.440+0000 7fe246800640 1 --2- 172.21.15.42:0/3857502770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe240071de0 0x7fe240155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:18.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.440+0000 7fe246800640 1 -- 172.21.15.42:0/3857502770 >> 172.21.15.42:0/3857502770 conn(0x7fe2400b21f0 msgr2=0x7fe24013fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:18.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.444+0000 7fe246800640 1 -- 172.21.15.42:0/3857502770 shutdown_connections 2024-05-28T14:10:18.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.444+0000 7fe246800640 1 -- 172.21.15.42:0/3857502770 wait complete. 2024-05-28T14:10:18.472 INFO:tasks.cephfs.test_volumes:percentage cloned = 66.667 2024-05-28T14:10:18.472 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-28T14:10:18.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.740+0000 7f32b6000640 1 Processor -- start 2024-05-28T14:10:18.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.740+0000 7f32b6000640 1 -- start start 2024-05-28T14:10:18.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.740+0000 7f32b6000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f32b0144fa0 0x7f32b0070650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:18.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.740+0000 7f32b6000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b0070b90 0x7f32b006b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:18.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.740+0000 7f32b6000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b00ae590 con 0x7f32b00b4670 2024-05-28T14:10:18.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.740+0000 7f32b6000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f32b00ae340 con 0x7f32b0144fa0 2024-05-28T14:10:18.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.740+0000 7f32b6000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f32b00aeec0 con 0x7f32b0070b90 2024-05-28T14:10:18.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.740+0000 7f32aea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b0070b90 0x7f32b006b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:18.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.740+0000 7f32afe00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f32b00b4670 0x7f32b006eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:48764/0 (socket says 172.21.15.42:48764) 2024-05-28T14:10:18.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.740+0000 7f32afe00640 1 -- 172.21.15.42:0/2680638959 learned_addr learned my addr 172.21.15.42:0/2680638959 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:18.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.740+0000 7f32af400640 1 --2- 172.21.15.42:0/2680638959 >> v2:172.21.15.63:3300/0 conn(0x7f32b0144fa0 0x7f32b0070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:18.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.744+0000 7f32ae000640 1 -- 172.21.15.42:0/2680638959 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 537939213 0 0) 0x7f32b00ae590 con 0x7f32b00b4670 2024-05-28T14:10:18.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.744+0000 7f32ae000640 1 -- 172.21.15.42:0/2680638959 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3294003530 con 0x7f32b00b4670 2024-05-28T14:10:18.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.744+0000 7f32af400640 1 -- 172.21.15.42:0/2680638959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b0070b90 msgr2=0x7f32b006b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:18.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.744+0000 7f32af400640 1 --2- 172.21.15.42:0/2680638959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b0070b90 0x7f32b006b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:18.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.744+0000 7f32af400640 1 -- 172.21.15.42:0/2680638959 >> v1:172.21.15.63:6789/0 conn(0x7f32b00b4670 legacy=0x7f32b006eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:18.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.744+0000 7f32af400640 1 -- 172.21.15.42:0/2680638959 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b00ae870 con 0x7f32b0144fa0 2024-05-28T14:10:18.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.744+0000 7f32aea00640 1 --2- 172.21.15.42:0/2680638959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b0070b90 0x7f32b006b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:10:18.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.744+0000 7f32af400640 1 --2- 172.21.15.42:0/2680638959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b0144fa0 0x7f32b0070650 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f32a00097c0 tx=0x7f32a001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=639d70fe5c56104 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:18.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.748+0000 7f32ae000640 1 -- 172.21.15.42:0/2680638959 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a00b1070 con 0x7f32b0144fa0 2024-05-28T14:10:18.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.748+0000 7f32ae000640 1 -- 172.21.15.42:0/2680638959 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a00ac410 con 0x7f32b0144fa0 2024-05-28T14:10:18.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.748+0000 7f32ae000640 1 -- 172.21.15.42:0/2680638959 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a00ac7e0 con 0x7f32b0144fa0 2024-05-28T14:10:18.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.748+0000 7f32b6000640 1 -- 172.21.15.42:0/2680638959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b0144fa0 msgr2=0x7f32b0070650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:18.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.748+0000 7f32b6000640 1 --2- 172.21.15.42:0/2680638959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b0144fa0 0x7f32b0070650 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f32a00097c0 tx=0x7f32a001c4a0 comp rx=0 tx=0).stop 2024-05-28T14:10:18.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.748+0000 7f32b6000640 1 -- 172.21.15.42:0/2680638959 shutdown_connections 2024-05-28T14:10:18.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.748+0000 7f32b6000640 1 --2- 172.21.15.42:0/2680638959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b0070b90 0x7f32b006b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:18.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.748+0000 7f32b6000640 1 --2- 172.21.15.42:0/2680638959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b0144fa0 0x7f32b0070650 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:18.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.748+0000 7f32b6000640 1 -- 172.21.15.42:0/2680638959 >> 172.21.15.42:0/2680638959 conn(0x7f32b00b6880 msgr2=0x7f32b00b6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:18.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.748+0000 7f32b6000640 1 -- 172.21.15.42:0/2680638959 shutdown_connections 2024-05-28T14:10:18.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.748+0000 7f32b6000640 1 -- 172.21.15.42:0/2680638959 wait complete. 2024-05-28T14:10:18.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.752+0000 7f32b6000640 1 Processor -- start 2024-05-28T14:10:18.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.752+0000 7f32b6000640 1 -- start start 2024-05-28T14:10:18.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32b6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b0070b90 0x7f32b0153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:18.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32b6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b00b4670 0x7f32b0154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:18.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32b6000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b0144fa0 0x7f32b0138c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:18.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32b6000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f32b016ba10 con 0x7f32b0070b90 2024-05-28T14:10:18.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32b6000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32b016b890 con 0x7f32b00b4670 2024-05-28T14:10:18.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32b6000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f32b016bb90 con 0x7f32b0144fa0 2024-05-28T14:10:18.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32af400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b0070b90 0x7f32b0153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:18.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32af400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b0070b90 0x7f32b0153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48446/0 (socket says 172.21.15.42:48446) 2024-05-28T14:10:18.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32af400640 1 -- 172.21.15.42:0/4277991281 learned_addr learned my addr 172.21.15.42:0/4277991281 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:18.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32aea00640 1 --2- 172.21.15.42:0/4277991281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b00b4670 0x7f32b0154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:18.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32af400640 1 -- 172.21.15.42:0/4277991281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b0144fa0 msgr2=0x7f32b0138c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T14:10:18.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32af400640 1 --2- 172.21.15.42:0/4277991281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b0144fa0 0x7f32b0138c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:18.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32af400640 1 -- 172.21.15.42:0/4277991281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b00b4670 msgr2=0x7f32b0154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:18.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32af400640 1 --2- 172.21.15.42:0/4277991281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b00b4670 0x7f32b0154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:18.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32af400640 1 -- 172.21.15.42:0/4277991281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b0155640 con 0x7f32b0070b90 2024-05-28T14:10:18.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.760+0000 7f32af400640 1 --2- 172.21.15.42:0/4277991281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b0070b90 0x7f32b0153e40 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f32a001ab50 tx=0x7f32a0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:18.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.764+0000 7f32a7400640 1 -- 172.21.15.42:0/4277991281 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a00b9020 con 0x7f32b0070b90 2024-05-28T14:10:18.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.764+0000 7f32a7400640 1 -- 172.21.15.42:0/4277991281 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a00accc0 con 0x7f32b0070b90 2024-05-28T14:10:18.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.764+0000 7f32a7400640 1 -- 172.21.15.42:0/4277991281 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a00b1040 con 0x7f32b0070b90 2024-05-28T14:10:18.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.764+0000 7f32b6000640 1 -- 172.21.15.42:0/4277991281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b015cbe0 con 0x7f32b0070b90 2024-05-28T14:10:18.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.764+0000 7f32b6000640 1 -- 172.21.15.42:0/4277991281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b0155350 con 0x7f32b0070b90 2024-05-28T14:10:18.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.764+0000 7f32b6000640 1 -- 172.21.15.42:0/4277991281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b004fdd0 con 0x7f32b0070b90 2024-05-28T14:10:18.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.764+0000 7f32a7400640 1 -- 172.21.15.42:0/4277991281 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7f32a00bc020 con 0x7f32b0070b90 2024-05-28T14:10:18.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.768+0000 7f32a7400640 1 --2- 172.21.15.42:0/4277991281 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3290069730 0x7f329006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:18.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.768+0000 7f32a7400640 1 -- 172.21.15.42:0/4277991281 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7f32a011efe0 con 0x7f32b0070b90 2024-05-28T14:10:18.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.768+0000 7f32aea00640 1 --2- 172.21.15.42:0/4277991281 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3290069730 0x7f329006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:18.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.768+0000 7f32aea00640 1 --2- 172.21.15.42:0/4277991281 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3290069730 0x7f329006bbf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f329c009580 tx=0x7f329c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:18.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:18.772+0000 7f32a7400640 1 -- 172.21.15.42:0/4277991281 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f32a0016090 con 0x7f32b0070b90 2024-05-28T14:10:19.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.100+0000 7f32b6000640 1 -- 172.21.15.42:0/4277991281 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f32b0155920 con 0x7f3290069730 2024-05-28T14:10:19.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.136+0000 7f32a7400640 1 -- 172.21.15.42:0/4277991281 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f32b0155920 con 0x7f3290069730 2024-05-28T14:10:19.119 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "status": { 2024-05-28T14:10:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "in-progress", 2024-05-28T14:10:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "source": { 2024-05-28T14:10:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "volume": "cephfs", 2024-05-28T14:10:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "subvolume": "sv1", 2024-05-28T14:10:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot": "ss1" 2024-05-28T14:10:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_report": { 2024-05-28T14:10:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "percentage cloned": 66.667, 2024-05-28T14:10:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "amount cloned": "2.0G/3.0G", 2024-05-28T14:10:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "files cloned": "5/6" 2024-05-28T14:10:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:19.120 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:19.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.136+0000 7f32b6000640 1 -- 172.21.15.42:0/4277991281 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3290069730 msgr2=0x7f329006bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:19.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.136+0000 7f32b6000640 1 --2- 172.21.15.42:0/4277991281 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3290069730 0x7f329006bbf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f329c009580 tx=0x7f329c008040 comp rx=0 tx=0).stop 2024-05-28T14:10:19.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.136+0000 7f32b6000640 1 -- 172.21.15.42:0/4277991281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b0070b90 msgr2=0x7f32b0153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.136+0000 7f32b6000640 1 --2- 172.21.15.42:0/4277991281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b0070b90 0x7f32b0153e40 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f32a001ab50 tx=0x7f32a0019240 comp rx=0 tx=0).stop 2024-05-28T14:10:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.136+0000 7f32b6000640 1 -- 172.21.15.42:0/4277991281 shutdown_connections 2024-05-28T14:10:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.136+0000 7f32b6000640 1 --2- 172.21.15.42:0/4277991281 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3290069730 0x7f329006bbf0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.136+0000 7f32b6000640 1 --2- 172.21.15.42:0/4277991281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b0144fa0 0x7f32b0138c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.136+0000 7f32b6000640 1 --2- 172.21.15.42:0/4277991281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b00b4670 0x7f32b0154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.136+0000 7f32b6000640 1 --2- 172.21.15.42:0/4277991281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b0070b90 0x7f32b0153e40 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.136+0000 7f32b6000640 1 -- 172.21.15.42:0/4277991281 >> 172.21.15.42:0/4277991281 conn(0x7f32b00b6880 msgr2=0x7f32b00b6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.136+0000 7f32b6000640 1 -- 172.21.15.42:0/4277991281 shutdown_connections 2024-05-28T14:10:19.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.136+0000 7f32b6000640 1 -- 172.21.15.42:0/4277991281 wait complete. 2024-05-28T14:10:19.190 INFO:tasks.cephfs.test_volumes:percentage cloned = 66.667 2024-05-28T14:10:19.191 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-28T14:10:19.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.484+0000 7f5129c00640 1 Processor -- start 2024-05-28T14:10:19.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.484+0000 7f5129c00640 1 -- start start 2024-05-28T14:10:19.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.488+0000 7f5129c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f512415cb50 0x7f512409b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:19.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.488+0000 7f5129c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512409ba40 con 0x7f512415c770 2024-05-28T14:10:19.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.488+0000 7f5129c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51240c60d0 con 0x7f51240d11d0 2024-05-28T14:10:19.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.488+0000 7f5129c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f51240c6720 con 0x7f512415cb50 2024-05-28T14:10:19.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.488+0000 7f5123400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f51240d11d0 0x7f5124166470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:48766/0 (socket says 172.21.15.42:48766) 2024-05-28T14:10:19.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.488+0000 7f5123400640 1 -- 172.21.15.42:0/3934926071 learned_addr learned my addr 172.21.15.42:0/3934926071 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:19.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.488+0000 7f5122000640 1 -- 172.21.15.42:0/3934926071 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4060440292 0 0) 0x7f51240c60d0 con 0x7f51240d11d0 2024-05-28T14:10:19.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.488+0000 7f5122000640 1 -- 172.21.15.42:0/3934926071 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5108003530 con 0x7f51240d11d0 2024-05-28T14:10:19.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.488+0000 7f511be00640 1 --2- 172.21.15.42:0/3934926071 >> v2:172.21.15.42:3300/0 conn(0x7f512415cb50 0x7f512409b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:19.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5122000640 1 -- 172.21.15.42:0/3934926071 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2069265687 0 0) 0x7f5108003530 con 0x7f51240d11d0 2024-05-28T14:10:19.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5122000640 1 -- 172.21.15.42:0/3934926071 >> v2:172.21.15.42:3300/0 conn(0x7f512415cb50 msgr2=0x7f512409b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:19.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5122000640 1 --2- 172.21.15.42:0/3934926071 >> v2:172.21.15.42:3300/0 conn(0x7f512415cb50 0x7f512409b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5122000640 1 -- 172.21.15.42:0/3934926071 >> v1:172.21.15.42:6789/0 conn(0x7f512415c770 legacy=0x7f51240d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:19.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5122000640 1 -- 172.21.15.42:0/3934926071 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51240c63b0 con 0x7f51240d11d0 2024-05-28T14:10:19.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5122000640 1 -- 172.21.15.42:0/3934926071 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f511c001a00 con 0x7f51240d11d0 2024-05-28T14:10:19.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5122000640 1 -- 172.21.15.42:0/3934926071 >> v1:172.21.15.63:6789/0 conn(0x7f51240d11d0 legacy=0x7f5124166470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:19.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5122000640 1 --2- 172.21.15.42:0/3934926071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080039c0 0x7f5108003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:19.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5122000640 1 --2- 172.21.15.42:0/3934926071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5108004810 0x7f5108004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:19.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5122000640 1 --2- 172.21.15.42:0/3934926071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51080051d0 0x7f510800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:19.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5122000640 1 -- 172.21.15.42:0/3934926071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f51240c6720 con 0x7f51080039c0 2024-05-28T14:10:19.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5122000640 1 -- 172.21.15.42:0/3934926071 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f510800c050 con 0x7f5108004810 2024-05-28T14:10:19.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5122000640 1 -- 172.21.15.42:0/3934926071 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f510800c250 con 0x7f51080051d0 2024-05-28T14:10:19.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f511be00640 1 --2- 172.21.15.42:0/3934926071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5108004810 0x7f5108004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:19.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5123400640 1 --2- 172.21.15.42:0/3934926071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51080051d0 0x7f510800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:19.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5122a00640 1 --2- 172.21.15.42:0/3934926071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080039c0 0x7f5108003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:19.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5123400640 1 -- 172.21.15.42:0/3934926071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5108004810 msgr2=0x7f5108004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:19.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5123400640 1 --2- 172.21.15.42:0/3934926071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5108004810 0x7f5108004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5123400640 1 -- 172.21.15.42:0/3934926071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080039c0 msgr2=0x7f5108003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:19.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5123400640 1 --2- 172.21.15.42:0/3934926071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080039c0 0x7f5108003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f5123400640 1 -- 172.21.15.42:0/3934926071 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51240c63b0 con 0x7f51080051d0 2024-05-28T14:10:19.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.492+0000 7f511be00640 1 --2- 172.21.15.42:0/3934926071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5108004810 0x7f5108004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:10:19.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.496+0000 7f5123400640 1 -- 172.21.15.42:0/3934926071 reap_dead start 2024-05-28T14:10:19.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.496+0000 7f5123400640 1 --2- 172.21.15.42:0/3934926071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51080051d0 0x7f510800b9d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f511c00a8c0 tx=0x7f511c01fae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f94812bc706fb1d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:19.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.500+0000 7f5122000640 1 -- 172.21.15.42:0/3934926071 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c0096b0 con 0x7f51080051d0 2024-05-28T14:10:19.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.500+0000 7f5122000640 1 -- 172.21.15.42:0/3934926071 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511c0aa510 con 0x7f51080051d0 2024-05-28T14:10:19.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.500+0000 7f5122000640 1 -- 172.21.15.42:0/3934926071 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c00a4c0 con 0x7f51080051d0 2024-05-28T14:10:19.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.500+0000 7f5129c00640 1 -- 172.21.15.42:0/3934926071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51080051d0 msgr2=0x7f510800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:19.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.500+0000 7f5129c00640 1 --2- 172.21.15.42:0/3934926071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51080051d0 0x7f510800b9d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f511c00a8c0 tx=0x7f511c01fae0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.500+0000 7f5129c00640 1 -- 172.21.15.42:0/3934926071 shutdown_connections 2024-05-28T14:10:19.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.500+0000 7f5129c00640 1 --2- 172.21.15.42:0/3934926071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51080051d0 0x7f510800b9d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.500+0000 7f5129c00640 1 -- 172.21.15.42:0/3934926071 >> 172.21.15.42:0/3934926071 conn(0x7f51240b3fd0 msgr2=0x7f51240b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:19.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.500+0000 7f5129c00640 1 -- 172.21.15.42:0/3934926071 shutdown_connections 2024-05-28T14:10:19.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.500+0000 7f5129c00640 1 -- 172.21.15.42:0/3934926071 wait complete. 2024-05-28T14:10:19.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.500+0000 7f5129c00640 1 Processor -- start 2024-05-28T14:10:19.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.500+0000 7f5129c00640 1 -- start start 2024-05-28T14:10:19.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5129c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080051d0 0x7f512414fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:19.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5129c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512415d210 0x7f512415d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:19.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5129c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51240c7aa0 0x7f51240c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5129c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f51240d02c0 con 0x7f51080051d0 2024-05-28T14:10:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5129c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f51240ce2c0 con 0x7f512415d210 2024-05-28T14:10:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5129c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f51240d0440 con 0x7f51240c7aa0 2024-05-28T14:10:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5122a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080051d0 0x7f512414fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5122a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080051d0 0x7f512414fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48470/0 (socket says 172.21.15.42:48470) 2024-05-28T14:10:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5122a00640 1 -- 172.21.15.42:0/767749808 learned_addr learned my addr 172.21.15.42:0/767749808 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5123400640 1 --2- 172.21.15.42:0/767749808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51240c7aa0 0x7f51240c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f511be00640 1 --2- 172.21.15.42:0/767749808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512415d210 0x7f512415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:19.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5122a00640 1 -- 172.21.15.42:0/767749808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51240c7aa0 msgr2=0x7f51240c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:19.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5122a00640 1 --2- 172.21.15.42:0/767749808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51240c7aa0 0x7f51240c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5122a00640 1 -- 172.21.15.42:0/767749808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512415d210 msgr2=0x7f512415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:19.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5122a00640 1 --2- 172.21.15.42:0/767749808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512415d210 0x7f512415d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5122a00640 1 -- 172.21.15.42:0/767749808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51241f5820 con 0x7f51080051d0 2024-05-28T14:10:19.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5122a00640 1 --2- 172.21.15.42:0/767749808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080051d0 0x7f512414fad0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f511400f670 tx=0x7f511400fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:19.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f511aa00640 1 -- 172.21.15.42:0/767749808 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114017020 con 0x7f51080051d0 2024-05-28T14:10:19.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f511aa00640 1 -- 172.21.15.42:0/767749808 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5114003aa0 con 0x7f51080051d0 2024-05-28T14:10:19.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5129c00640 1 -- 172.21.15.42:0/767749808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51241f5b00 con 0x7f51080051d0 2024-05-28T14:10:19.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f511aa00640 1 -- 172.21.15.42:0/767749808 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114004050 con 0x7f51080051d0 2024-05-28T14:10:19.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.504+0000 7f5129c00640 1 -- 172.21.15.42:0/767749808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5124071560 con 0x7f51080051d0 2024-05-28T14:10:19.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.508+0000 7f5129c00640 1 -- 172.21.15.42:0/767749808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51240a04d0 con 0x7f51080051d0 2024-05-28T14:10:19.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.508+0000 7f511aa00640 1 -- 172.21.15.42:0/767749808 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7f511401a020 con 0x7f51080051d0 2024-05-28T14:10:19.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.508+0000 7f511aa00640 1 --2- 172.21.15.42:0/767749808 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5100069760 0x7f510006bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:19.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.508+0000 7f511aa00640 1 -- 172.21.15.42:0/767749808 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7f5114085780 con 0x7f51080051d0 2024-05-28T14:10:19.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.508+0000 7f511be00640 1 --2- 172.21.15.42:0/767749808 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5100069760 0x7f510006bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:19.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.508+0000 7f511be00640 1 --2- 172.21.15.42:0/767749808 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5100069760 0x7f510006bc20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f51240b7a00 tx=0x7f510c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:19.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.516+0000 7f511aa00640 1 -- 172.21.15.42:0/767749808 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5114021020 con 0x7f51080051d0 2024-05-28T14:10:19.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.852+0000 7f5129c00640 1 -- 172.21.15.42:0/767749808 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f5124159380 con 0x7f5100069760 2024-05-28T14:10:19.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.876+0000 7f511aa00640 1 -- 172.21.15.42:0/767749808 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7f5124159380 con 0x7f5100069760 2024-05-28T14:10:19.862 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:19.862 INFO:teuthology.orchestra.run.smithi042.stdout: "status": { 2024-05-28T14:10:19.862 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "in-progress", 2024-05-28T14:10:19.862 INFO:teuthology.orchestra.run.smithi042.stdout: "source": { 2024-05-28T14:10:19.862 INFO:teuthology.orchestra.run.smithi042.stdout: "volume": "cephfs", 2024-05-28T14:10:19.862 INFO:teuthology.orchestra.run.smithi042.stdout: "subvolume": "sv1", 2024-05-28T14:10:19.862 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot": "ss1" 2024-05-28T14:10:19.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:19.863 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_report": { 2024-05-28T14:10:19.863 INFO:teuthology.orchestra.run.smithi042.stdout: "percentage cloned": 87.24, 2024-05-28T14:10:19.863 INFO:teuthology.orchestra.run.smithi042.stdout: "amount cloned": "2.6G/3.0G", 2024-05-28T14:10:19.863 INFO:teuthology.orchestra.run.smithi042.stdout: "files cloned": "6/6" 2024-05-28T14:10:19.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:19.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:19.863 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:19.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.888+0000 7f5129c00640 1 -- 172.21.15.42:0/767749808 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5100069760 msgr2=0x7f510006bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:19.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.888+0000 7f5129c00640 1 --2- 172.21.15.42:0/767749808 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5100069760 0x7f510006bc20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f51240b7a00 tx=0x7f510c009190 comp rx=0 tx=0).stop 2024-05-28T14:10:19.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.888+0000 7f5129c00640 1 -- 172.21.15.42:0/767749808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080051d0 msgr2=0x7f512414fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:19.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.888+0000 7f5129c00640 1 --2- 172.21.15.42:0/767749808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080051d0 0x7f512414fad0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f511400f670 tx=0x7f511400fa70 comp rx=0 tx=0).stop 2024-05-28T14:10:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.896+0000 7f5129c00640 1 -- 172.21.15.42:0/767749808 shutdown_connections 2024-05-28T14:10:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.896+0000 7f5129c00640 1 --2- 172.21.15.42:0/767749808 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5100069760 0x7f510006bc20 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.896+0000 7f5129c00640 1 --2- 172.21.15.42:0/767749808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51240c7aa0 0x7f51240c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.896+0000 7f5129c00640 1 --2- 172.21.15.42:0/767749808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512415d210 0x7f512415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.896+0000 7f5129c00640 1 --2- 172.21.15.42:0/767749808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080051d0 0x7f512414fad0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.896+0000 7f5129c00640 1 -- 172.21.15.42:0/767749808 >> 172.21.15.42:0/767749808 conn(0x7f51240b3fd0 msgr2=0x7f51240c6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:19.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.896+0000 7f5129c00640 1 -- 172.21.15.42:0/767749808 shutdown_connections 2024-05-28T14:10:19.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:19.896+0000 7f5129c00640 1 -- 172.21.15.42:0/767749808 wait complete. 2024-05-28T14:10:19.940 INFO:tasks.cephfs.test_volumes:percentage cloned = 87.24 2024-05-28T14:10:19.941 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-28T14:10:20.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.172+0000 7ff5c1400640 1 Processor -- start 2024-05-28T14:10:20.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.172+0000 7ff5c1400640 1 -- start start 2024-05-28T14:10:20.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.176+0000 7ff5c1400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff5bc06ee20 0x7ff5bc06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.176+0000 7ff5c1400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff5bc15b9b0 0x7ff5bc15be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.176+0000 7ff5c1400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5bc0afe40 con 0x7ff5bc144f80 2024-05-28T14:10:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.176+0000 7ff5c1400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff5bc06f7c0 con 0x7ff5bc15b9b0 2024-05-28T14:10:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.176+0000 7ff5c1400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff5bc0724d0 con 0x7ff5bc06ee20 2024-05-28T14:10:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.176+0000 7ff5bb200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff5bc15b9b0 0x7ff5bc15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.176+0000 7ff5bb200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff5bc15b9b0 0x7ff5bc15be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48478/0 (socket says 172.21.15.42:48478) 2024-05-28T14:10:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.176+0000 7ff5bb200640 1 -- 172.21.15.42:0/4261257375 learned_addr learned my addr 172.21.15.42:0/4261257375 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.176+0000 7ff5b9e00640 1 --2- 172.21.15.42:0/4261257375 >> v2:172.21.15.63:3300/0 conn(0x7ff5bc06ee20 0x7ff5bc06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.176+0000 7ff5bb200640 1 -- 172.21.15.42:0/4261257375 >> v2:172.21.15.63:3300/0 conn(0x7ff5bc06ee20 msgr2=0x7ff5bc06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.176+0000 7ff5bb200640 1 --2- 172.21.15.42:0/4261257375 >> v2:172.21.15.63:3300/0 conn(0x7ff5bc06ee20 0x7ff5bc06f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.176+0000 7ff5bb200640 1 -- 172.21.15.42:0/4261257375 >> v1:172.21.15.63:6789/0 conn(0x7ff5bc144f80 legacy=0x7ff5bc072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:20.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.176+0000 7ff5bb200640 1 -- 172.21.15.42:0/4261257375 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc15c340 con 0x7ff5bc15b9b0 2024-05-28T14:10:20.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5bb200640 1 --2- 172.21.15.42:0/4261257375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc15b9b0 0x7ff5bc15be00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc075500 tx=0x7ff5b401ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02e730b324ea416 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:20.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5b9400640 1 -- 172.21.15.42:0/4261257375 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400b2d0 con 0x7ff5bc15b9b0 2024-05-28T14:10:20.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5b9400640 1 -- 172.21.15.42:0/4261257375 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b400b470 con 0x7ff5bc15b9b0 2024-05-28T14:10:20.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5b9400640 1 -- 172.21.15.42:0/4261257375 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40a7aa0 con 0x7ff5bc15b9b0 2024-05-28T14:10:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5c1400640 1 -- 172.21.15.42:0/4261257375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc15b9b0 msgr2=0x7ff5bc15be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5c1400640 1 --2- 172.21.15.42:0/4261257375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc15b9b0 0x7ff5bc15be00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc075500 tx=0x7ff5b401ebb0 comp rx=0 tx=0).stop 2024-05-28T14:10:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5c1400640 1 -- 172.21.15.42:0/4261257375 shutdown_connections 2024-05-28T14:10:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5c1400640 1 --2- 172.21.15.42:0/4261257375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc15b9b0 0x7ff5bc15be00 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5c1400640 1 --2- 172.21.15.42:0/4261257375 >> v2:172.21.15.63:3300/0 conn(0x7ff5bc06ee20 0x7ff5bc06f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:20.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5c1400640 1 -- 172.21.15.42:0/4261257375 >> 172.21.15.42:0/4261257375 conn(0x7ff5bc0b21f0 msgr2=0x7ff5bc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5c1400640 1 -- 172.21.15.42:0/4261257375 shutdown_connections 2024-05-28T14:10:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5c1400640 1 -- 172.21.15.42:0/4261257375 wait complete. 2024-05-28T14:10:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5c1400640 1 Processor -- start 2024-05-28T14:10:20.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5c1400640 1 -- start start 2024-05-28T14:10:20.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5c1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5bc06ee20 0x7ff5bc1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:20.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.180+0000 7ff5c1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc144f80 0x7ff5bc1f7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:20.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5c1400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5bc15b9b0 0x7ff5bc1436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:20.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5c1400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc0b4010 con 0x7ff5bc144f80 2024-05-28T14:10:20.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5c1400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc072160 con 0x7ff5bc06ee20 2024-05-28T14:10:20.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5c1400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff5bc0b4190 con 0x7ff5bc15b9b0 2024-05-28T14:10:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5b9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc144f80 0x7ff5bc1f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5b9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc144f80 0x7ff5bc1f7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48484/0 (socket says 172.21.15.42:48484) 2024-05-28T14:10:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5b9e00640 1 -- 172.21.15.42:0/1606304782 learned_addr learned my addr 172.21.15.42:0/1606304782 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5ba800640 1 --2- 172.21.15.42:0/1606304782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5bc06ee20 0x7ff5bc1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5bb200640 1 --2- 172.21.15.42:0/1606304782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5bc15b9b0 0x7ff5bc1436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5b9e00640 1 -- 172.21.15.42:0/1606304782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5bc15b9b0 msgr2=0x7ff5bc1436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5b9e00640 1 --2- 172.21.15.42:0/1606304782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5bc15b9b0 0x7ff5bc1436b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5b9e00640 1 -- 172.21.15.42:0/1606304782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5bc06ee20 msgr2=0x7ff5bc1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5b9e00640 1 --2- 172.21.15.42:0/1606304782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5bc06ee20 0x7ff5bc1589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5b9e00640 1 -- 172.21.15.42:0/1606304782 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc158f00 con 0x7ff5bc144f80 2024-05-28T14:10:20.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5b9e00640 1 --2- 172.21.15.42:0/1606304782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc144f80 0x7ff5bc1f7380 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400d800 tx=0x7ff5a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:20.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5b2a00640 1 -- 172.21.15.42:0/1606304782 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400b6c0 con 0x7ff5bc144f80 2024-05-28T14:10:20.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5b2a00640 1 -- 172.21.15.42:0/1606304782 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5a4019070 con 0x7ff5bc144f80 2024-05-28T14:10:20.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5b2a00640 1 -- 172.21.15.42:0/1606304782 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400ba60 con 0x7ff5bc144f80 2024-05-28T14:10:20.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5c1400640 1 -- 172.21.15.42:0/1606304782 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5bc1f78c0 con 0x7ff5bc144f80 2024-05-28T14:10:20.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5c1400640 1 -- 172.21.15.42:0/1606304782 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5bc137710 con 0x7ff5bc144f80 2024-05-28T14:10:20.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5c1400640 1 -- 172.21.15.42:0/1606304782 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5bc0738b0 con 0x7ff5bc144f80 2024-05-28T14:10:20.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.184+0000 7ff5b2a00640 1 -- 172.21.15.42:0/1606304782 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7ff5a400bc00 con 0x7ff5bc144f80 2024-05-28T14:10:20.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.188+0000 7ff5b2a00640 1 --2- 172.21.15.42:0/1606304782 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff598069730 0x7ff59806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:20.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.188+0000 7ff5b2a00640 1 -- 172.21.15.42:0/1606304782 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7ff5a4085d90 con 0x7ff5bc144f80 2024-05-28T14:10:20.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.188+0000 7ff5ba800640 1 --2- 172.21.15.42:0/1606304782 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff598069730 0x7ff59806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:20.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.188+0000 7ff5ba800640 1 --2- 172.21.15.42:0/1606304782 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff598069730 0x7ff59806bbf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc144300 tx=0x7ff5ac007e40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:20.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.192+0000 7ff5b2a00640 1 -- 172.21.15.42:0/1606304782 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff5bc1f78c0 con 0x7ff5bc144f80 2024-05-28T14:10:20.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.396+0000 7ff5c1400640 1 -- 172.21.15.42:0/1606304782 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7ff5bc04fdd0 con 0x7ff598069730 2024-05-28T14:10:20.470 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:10:20.474 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:10:20.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.920+0000 7ff5b2a00640 1 -- 172.21.15.42:0/1606304782 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7ff5bc04fdd0 con 0x7ff598069730 2024-05-28T14:10:20.904 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "status": { 2024-05-28T14:10:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "complete" 2024-05-28T14:10:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:20.904 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:20.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.920+0000 7ff5c1400640 1 -- 172.21.15.42:0/1606304782 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff598069730 msgr2=0x7ff59806bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:20.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.920+0000 7ff5c1400640 1 --2- 172.21.15.42:0/1606304782 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff598069730 0x7ff59806bbf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc144300 tx=0x7ff5ac007e40 comp rx=0 tx=0).stop 2024-05-28T14:10:20.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.920+0000 7ff5c1400640 1 -- 172.21.15.42:0/1606304782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc144f80 msgr2=0x7ff5bc1f7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:20.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.920+0000 7ff5c1400640 1 --2- 172.21.15.42:0/1606304782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc144f80 0x7ff5bc1f7380 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400d800 tx=0x7ff5a400dcc0 comp rx=0 tx=0).stop 2024-05-28T14:10:20.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.920+0000 7ff5c1400640 1 -- 172.21.15.42:0/1606304782 shutdown_connections 2024-05-28T14:10:20.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.920+0000 7ff5c1400640 1 --2- 172.21.15.42:0/1606304782 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff598069730 0x7ff59806bbf0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:20.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.920+0000 7ff5c1400640 1 --2- 172.21.15.42:0/1606304782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5bc15b9b0 0x7ff5bc1436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:20.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.920+0000 7ff5c1400640 1 --2- 172.21.15.42:0/1606304782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc144f80 0x7ff5bc1f7380 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:20.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.920+0000 7ff5c1400640 1 --2- 172.21.15.42:0/1606304782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5bc06ee20 0x7ff5bc1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:20.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.920+0000 7ff5c1400640 1 -- 172.21.15.42:0/1606304782 >> 172.21.15.42:0/1606304782 conn(0x7ff5bc0b21f0 msgr2=0x7ff5bc0b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:20.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.920+0000 7ff5c1400640 1 -- 172.21.15.42:0/1606304782 shutdown_connections 2024-05-28T14:10:20.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:20.920+0000 7ff5c1400640 1 -- 172.21.15.42:0/1606304782 wait complete. 2024-05-28T14:10:20.927 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-05-28T14:10:20.928 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-28T14:10:21.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.088+0000 7fee2de00640 1 Processor -- start 2024-05-28T14:10:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.088+0000 7fee2de00640 1 -- start start 2024-05-28T14:10:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.088+0000 7fee2de00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fee28072670 0x7fee28072a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.088+0000 7fee2de00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee280b3cd0 0x7fee2806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.088+0000 7fee2de00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee28072f90 con 0x7fee280cce90 2024-05-28T14:10:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.088+0000 7fee2de00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fee280b5870 con 0x7fee28072670 2024-05-28T14:10:21.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.088+0000 7fee2de00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fee280b34f0 con 0x7fee280b3cd0 2024-05-28T14:10:21.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.092+0000 7fee27400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee280b3cd0 0x7fee2806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.092+0000 7fee26a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fee28072670 0x7fee28072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.092+0000 7fee27400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee280b3cd0 0x7fee2806dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:38012/0 (socket says 172.21.15.42:38012) 2024-05-28T14:10:21.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.092+0000 7fee26a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fee28072670 0x7fee28072a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:39142/0 (socket says 172.21.15.42:39142) 2024-05-28T14:10:21.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.092+0000 7fee27400640 1 -- 172.21.15.42:0/954001842 learned_addr learned my addr 172.21.15.42:0/954001842 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:21.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.092+0000 7fee1e000640 1 --1- 172.21.15.42:0/954001842 >> v1:172.21.15.63:6789/0 conn(0x7fee280cce90 0x7fee2815b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:48778/0 (socket says 172.21.15.42:48778) 2024-05-28T14:10:21.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.092+0000 7fee26a00640 1 -- 172.21.15.42:0/954001842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee280b3cd0 msgr2=0x7fee2806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:21.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.092+0000 7fee26a00640 1 --2- 172.21.15.42:0/954001842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee280b3cd0 0x7fee2806dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.092+0000 7fee26a00640 1 -- 172.21.15.42:0/954001842 >> v1:172.21.15.63:6789/0 conn(0x7fee280cce90 legacy=0x7fee2815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.092+0000 7fee26a00640 1 -- 172.21.15.42:0/954001842 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2815ba90 con 0x7fee28072670 2024-05-28T14:10:21.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.092+0000 7fee26000640 1 -- 172.21.15.42:0/954001842 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1581346893 0 0) 0x7fee28072f90 con 0x7fee280cce90 2024-05-28T14:10:21.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.092+0000 7fee26a00640 1 --2- 172.21.15.42:0/954001842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee28072670 0x7fee28072a50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fee180097c0 tx=0x7fee1801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f28982a3f99d7a53 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:21.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee26000640 1 -- 172.21.15.42:0/954001842 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180b0070 con 0x7fee28072670 2024-05-28T14:10:21.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee26000640 1 -- 172.21.15.42:0/954001842 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee180ab600 con 0x7fee28072670 2024-05-28T14:10:21.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee26000640 1 -- 172.21.15.42:0/954001842 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180ab9d0 con 0x7fee28072670 2024-05-28T14:10:21.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 -- 172.21.15.42:0/954001842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee28072670 msgr2=0x7fee28072a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 --2- 172.21.15.42:0/954001842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee28072670 0x7fee28072a50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fee180097c0 tx=0x7fee1801c4a0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 -- 172.21.15.42:0/954001842 shutdown_connections 2024-05-28T14:10:21.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 --2- 172.21.15.42:0/954001842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee280b3cd0 0x7fee2806dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 --2- 172.21.15.42:0/954001842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee28072670 0x7fee28072a50 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 -- 172.21.15.42:0/954001842 >> 172.21.15.42:0/954001842 conn(0x7fee280b21f0 msgr2=0x7fee280b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:21.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 -- 172.21.15.42:0/954001842 shutdown_connections 2024-05-28T14:10:21.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 -- 172.21.15.42:0/954001842 wait complete. 2024-05-28T14:10:21.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 Processor -- start 2024-05-28T14:10:21.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 -- start start 2024-05-28T14:10:21.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee28072670 0x7fee281519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee280b3cd0 0x7fee28157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee280cce90 0x7fee28158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fee28167bd0 con 0x7fee28072670 2024-05-28T14:10:21.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee2806e660 con 0x7fee280cce90 2024-05-28T14:10:21.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee2de00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fee28167d50 con 0x7fee280b3cd0 2024-05-28T14:10:21.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee26a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee28072670 0x7fee281519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee26a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee28072670 0x7fee281519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48486/0 (socket says 172.21.15.42:48486) 2024-05-28T14:10:21.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee26a00640 1 -- 172.21.15.42:0/3003507757 learned_addr learned my addr 172.21.15.42:0/3003507757 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:21.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee27400640 1 --2- 172.21.15.42:0/3003507757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee280cce90 0x7fee28158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.096+0000 7fee1e000640 1 --2- 172.21.15.42:0/3003507757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee280b3cd0 0x7fee28157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee26a00640 1 -- 172.21.15.42:0/3003507757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee280b3cd0 msgr2=0x7fee28157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee26a00640 1 --2- 172.21.15.42:0/3003507757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee280b3cd0 0x7fee28157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee26a00640 1 -- 172.21.15.42:0/3003507757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee280cce90 msgr2=0x7fee28158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee26a00640 1 --2- 172.21.15.42:0/3003507757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee280cce90 0x7fee28158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee26a00640 1 -- 172.21.15.42:0/3003507757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee28157120 con 0x7fee28072670 2024-05-28T14:10:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee26a00640 1 --2- 172.21.15.42:0/3003507757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee28072670 0x7fee281519d0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fee180ae040 tx=0x7fee1801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee1fe00640 1 -- 172.21.15.42:0/3003507757 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180b0050 con 0x7fee28072670 2024-05-28T14:10:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee1fe00640 1 -- 172.21.15.42:0/3003507757 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee180bc070 con 0x7fee28072670 2024-05-28T14:10:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee1fe00640 1 -- 172.21.15.42:0/3003507757 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180b5850 con 0x7fee28072670 2024-05-28T14:10:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee2de00640 1 -- 172.21.15.42:0/3003507757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee28156300 con 0x7fee28072670 2024-05-28T14:10:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee2de00640 1 -- 172.21.15.42:0/3003507757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee28156880 con 0x7fee28072670 2024-05-28T14:10:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee2de00640 1 -- 172.21.15.42:0/3003507757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee280738b0 con 0x7fee28072670 2024-05-28T14:10:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee1fe00640 1 -- 172.21.15.42:0/3003507757 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7fee180b59f0 con 0x7fee28072670 2024-05-28T14:10:21.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee1fe00640 1 --2- 172.21.15.42:0/3003507757 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fee04069730 0x7fee0406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee1fe00640 1 -- 172.21.15.42:0/3003507757 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7fee1811ff80 con 0x7fee28072670 2024-05-28T14:10:21.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.100+0000 7fee1e000640 1 --2- 172.21.15.42:0/3003507757 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fee04069730 0x7fee0406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.104+0000 7fee1e000640 1 --2- 172.21.15.42:0/3003507757 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fee04069730 0x7fee0406bbf0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fee280afd50 tx=0x7fee10007df0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:21.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.104+0000 7fee1fe00640 1 -- 172.21.15.42:0/3003507757 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fee18016090 con 0x7fee28072670 2024-05-28T14:10:21.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.304+0000 7fee2de00640 1 -- 172.21.15.42:0/3003507757 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fee2804fdd0 con 0x7fee04069730 2024-05-28T14:10:21.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.308+0000 7fee1fe00640 1 -- 172.21.15.42:0/3003507757 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fee2804fdd0 con 0x7fee04069730 2024-05-28T14:10:21.294 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:21.294 INFO:teuthology.orchestra.run.smithi042.stdout: "status": { 2024-05-28T14:10:21.294 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "complete" 2024-05-28T14:10:21.294 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:21.294 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.312+0000 7fee2de00640 1 -- 172.21.15.42:0/3003507757 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fee04069730 msgr2=0x7fee0406bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.312+0000 7fee2de00640 1 --2- 172.21.15.42:0/3003507757 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fee04069730 0x7fee0406bbf0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fee280afd50 tx=0x7fee10007df0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.312+0000 7fee2de00640 1 -- 172.21.15.42:0/3003507757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee28072670 msgr2=0x7fee281519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.312+0000 7fee2de00640 1 --2- 172.21.15.42:0/3003507757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee28072670 0x7fee281519d0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fee180ae040 tx=0x7fee1801ace0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.312+0000 7fee2de00640 1 -- 172.21.15.42:0/3003507757 shutdown_connections 2024-05-28T14:10:21.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.312+0000 7fee2de00640 1 --2- 172.21.15.42:0/3003507757 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fee04069730 0x7fee0406bbf0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.312+0000 7fee2de00640 1 --2- 172.21.15.42:0/3003507757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee280cce90 0x7fee28158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.312+0000 7fee2de00640 1 --2- 172.21.15.42:0/3003507757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee280b3cd0 0x7fee28157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.312+0000 7fee2de00640 1 --2- 172.21.15.42:0/3003507757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee28072670 0x7fee281519d0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.312+0000 7fee2de00640 1 -- 172.21.15.42:0/3003507757 >> 172.21.15.42:0/3003507757 conn(0x7fee280b21f0 msgr2=0x7fee280b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:21.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.312+0000 7fee2de00640 1 -- 172.21.15.42:0/3003507757 shutdown_connections 2024-05-28T14:10:21.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.312+0000 7fee2de00640 1 -- 172.21.15.42:0/3003507757 wait complete. 2024-05-28T14:10:21.316 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:10:21.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.472+0000 7fe743c00640 1 Processor -- start 2024-05-28T14:10:21.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.472+0000 7fe743c00640 1 -- start start 2024-05-28T14:10:21.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe743c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe73c15f750 0x7fe73c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe743c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73c15fb30 0x7fe73c0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe743c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe73c072210 con 0x7fe73c0b2a40 2024-05-28T14:10:21.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe743c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe73c072ad0 con 0x7fe73c15f750 2024-05-28T14:10:21.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe743c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe73c06f650 con 0x7fe73c15fb30 2024-05-28T14:10:21.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe741c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe73c0b2a40 0x7fe73c1623f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:49478/0 (socket says 172.21.15.42:49478) 2024-05-28T14:10:21.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe741c00640 1 -- 172.21.15.42:0/1205599068 learned_addr learned my addr 172.21.15.42:0/1205599068 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:21.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe73be00640 1 --2- 172.21.15.42:0/1205599068 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73c15fb30 0x7fe73c0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe741200640 1 --2- 172.21.15.42:0/1205599068 >> v2:172.21.15.63:3300/0 conn(0x7fe73c15f750 0x7fe73c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe73b400640 1 -- 172.21.15.42:0/1205599068 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1944919810 0 0) 0x7fe73c072210 con 0x7fe73c0b2a40 2024-05-28T14:10:21.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe73b400640 1 -- 172.21.15.42:0/1205599068 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe724003530 con 0x7fe73c0b2a40 2024-05-28T14:10:21.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe73be00640 1 -- 172.21.15.42:0/1205599068 >> v2:172.21.15.63:3300/0 conn(0x7fe73c15f750 msgr2=0x7fe73c0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:21.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe73be00640 1 --2- 172.21.15.42:0/1205599068 >> v2:172.21.15.63:3300/0 conn(0x7fe73c15f750 0x7fe73c0b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe73be00640 1 -- 172.21.15.42:0/1205599068 >> v1:172.21.15.42:6789/0 conn(0x7fe73c0b2a40 legacy=0x7fe73c1623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.476+0000 7fe73be00640 1 -- 172.21.15.42:0/1205599068 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c072480 con 0x7fe73c15fb30 2024-05-28T14:10:21.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe73be00640 1 --2- 172.21.15.42:0/1205599068 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73c15fb30 0x7fe73c0b4060 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe7280097c0 tx=0x7fe72801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d9385cb6a1eaf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:21.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe73b400640 1 -- 172.21.15.42:0/1205599068 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7280b2070 con 0x7fe73c15fb30 2024-05-28T14:10:21.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe73b400640 1 -- 172.21.15.42:0/1205599068 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7280ad680 con 0x7fe73c15fb30 2024-05-28T14:10:21.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe73b400640 1 -- 172.21.15.42:0/1205599068 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7280ada50 con 0x7fe73c15fb30 2024-05-28T14:10:21.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 -- 172.21.15.42:0/1205599068 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73c15fb30 msgr2=0x7fe73c0b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 --2- 172.21.15.42:0/1205599068 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73c15fb30 0x7fe73c0b4060 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe7280097c0 tx=0x7fe72801c470 comp rx=0 tx=0).stop 2024-05-28T14:10:21.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 -- 172.21.15.42:0/1205599068 shutdown_connections 2024-05-28T14:10:21.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 --2- 172.21.15.42:0/1205599068 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73c15fb30 0x7fe73c0b4060 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 --2- 172.21.15.42:0/1205599068 >> v2:172.21.15.63:3300/0 conn(0x7fe73c15f750 0x7fe73c0b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 -- 172.21.15.42:0/1205599068 >> 172.21.15.42:0/1205599068 conn(0x7fe73c0d07c0 msgr2=0x7fe73c0d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:21.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 -- 172.21.15.42:0/1205599068 shutdown_connections 2024-05-28T14:10:21.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 -- 172.21.15.42:0/1205599068 wait complete. 2024-05-28T14:10:21.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 Processor -- start 2024-05-28T14:10:21.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 -- start start 2024-05-28T14:10:21.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe73c0b2a40 0x7fe73c13d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c15f750 0x7fe73c0adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73c15fb30 0x7fe73c14fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c16b910 con 0x7fe73c15f750 2024-05-28T14:10:21.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c16b790 con 0x7fe73c0b2a40 2024-05-28T14:10:21.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.480+0000 7fe743c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe73c16ba90 con 0x7fe73c15fb30 2024-05-28T14:10:21.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe73be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c15f750 0x7fe73c0adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe73be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c15f750 0x7fe73c0adbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48500/0 (socket says 172.21.15.42:48500) 2024-05-28T14:10:21.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe73be00640 1 -- 172.21.15.42:0/1365149489 learned_addr learned my addr 172.21.15.42:0/1365149489 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:21.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe741200640 1 --2- 172.21.15.42:0/1365149489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe73c0b2a40 0x7fe73c13d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe741c00640 1 --2- 172.21.15.42:0/1365149489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73c15fb30 0x7fe73c14fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe73be00640 1 -- 172.21.15.42:0/1365149489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73c15fb30 msgr2=0x7fe73c14fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe73be00640 1 --2- 172.21.15.42:0/1365149489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73c15fb30 0x7fe73c14fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe73be00640 1 -- 172.21.15.42:0/1365149489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe73c0b2a40 msgr2=0x7fe73c13d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe73be00640 1 --2- 172.21.15.42:0/1365149489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe73c0b2a40 0x7fe73c13d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe73be00640 1 -- 172.21.15.42:0/1365149489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c0ae610 con 0x7fe73c15f750 2024-05-28T14:10:21.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe73be00640 1 --2- 172.21.15.42:0/1365149489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c15f750 0x7fe73c0adbc0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fe72801a0a0 tx=0x7fe728019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:21.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe738c00640 1 -- 172.21.15.42:0/1365149489 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7280bc070 con 0x7fe73c15f750 2024-05-28T14:10:21.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe738c00640 1 -- 172.21.15.42:0/1365149489 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7280adc70 con 0x7fe73c15f750 2024-05-28T14:10:21.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe738c00640 1 -- 172.21.15.42:0/1365149489 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7280b2040 con 0x7fe73c15f750 2024-05-28T14:10:21.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe743c00640 1 -- 172.21.15.42:0/1365149489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c0ae8f0 con 0x7fe73c15f750 2024-05-28T14:10:21.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe743c00640 1 -- 172.21.15.42:0/1365149489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c0aeeb0 con 0x7fe73c15f750 2024-05-28T14:10:21.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe743c00640 1 -- 172.21.15.42:0/1365149489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe73c073bb0 con 0x7fe73c15f750 2024-05-28T14:10:21.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe738c00640 1 -- 172.21.15.42:0/1365149489 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7fe7280b6630 con 0x7fe73c15f750 2024-05-28T14:10:21.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.484+0000 7fe738c00640 1 --2- 172.21.15.42:0/1365149489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe71c069730 0x7fe71c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.488+0000 7fe738c00640 1 -- 172.21.15.42:0/1365149489 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7fe72811eac0 con 0x7fe73c15f750 2024-05-28T14:10:21.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.488+0000 7fe741200640 1 --2- 172.21.15.42:0/1365149489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe71c069730 0x7fe71c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.488+0000 7fe741200640 1 --2- 172.21.15.42:0/1365149489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe71c069730 0x7fe71c06bbf0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe730009ac0 tx=0x7fe730009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:21.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.488+0000 7fe738c00640 1 -- 172.21.15.42:0/1365149489 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe728016090 con 0x7fe73c15f750 2024-05-28T14:10:21.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.676+0000 7fe743c00640 1 -- 172.21.15.42:0/1365149489 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fe73c0d15f0 con 0x7fe71c069730 2024-05-28T14:10:21.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.676+0000 7fe738c00640 1 -- 172.21.15.42:0/1365149489 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fe73c0d15f0 con 0x7fe71c069730 2024-05-28T14:10:21.662 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:10:21.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:21.662 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:10:21.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:21.662 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:10:21.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.680+0000 7fe743c00640 1 -- 172.21.15.42:0/1365149489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe71c069730 msgr2=0x7fe71c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.680+0000 7fe743c00640 1 --2- 172.21.15.42:0/1365149489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe71c069730 0x7fe71c06bbf0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe730009ac0 tx=0x7fe730009190 comp rx=0 tx=0).stop 2024-05-28T14:10:21.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.680+0000 7fe743c00640 1 -- 172.21.15.42:0/1365149489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c15f750 msgr2=0x7fe73c0adbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.680+0000 7fe743c00640 1 --2- 172.21.15.42:0/1365149489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c15f750 0x7fe73c0adbc0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fe72801a0a0 tx=0x7fe728019210 comp rx=0 tx=0).stop 2024-05-28T14:10:21.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.680+0000 7fe743c00640 1 -- 172.21.15.42:0/1365149489 shutdown_connections 2024-05-28T14:10:21.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.680+0000 7fe743c00640 1 --2- 172.21.15.42:0/1365149489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe71c069730 0x7fe71c06bbf0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.680+0000 7fe743c00640 1 --2- 172.21.15.42:0/1365149489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73c15fb30 0x7fe73c14fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.680+0000 7fe743c00640 1 --2- 172.21.15.42:0/1365149489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c15f750 0x7fe73c0adbc0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.680+0000 7fe743c00640 1 --2- 172.21.15.42:0/1365149489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe73c0b2a40 0x7fe73c13d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.680+0000 7fe743c00640 1 -- 172.21.15.42:0/1365149489 >> 172.21.15.42:0/1365149489 conn(0x7fe73c0d07c0 msgr2=0x7fe73c0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:21.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.680+0000 7fe743c00640 1 -- 172.21.15.42:0/1365149489 shutdown_connections 2024-05-28T14:10:21.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.680+0000 7fe743c00640 1 -- 172.21.15.42:0/1365149489 wait complete. 2024-05-28T14:10:21.684 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:10:21.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9de400640 1 Processor -- start 2024-05-28T14:10:21.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9de400640 1 -- start start 2024-05-28T14:10:21.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9de400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe9d8158f30 0x7fe9d8072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9de400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe9d80b4f00 0x7fe9d815ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9de400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9d815c3f0 con 0x7fe9d815bf50 2024-05-28T14:10:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9de400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d815c9b0 con 0x7fe9d80b4f00 2024-05-28T14:10:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9de400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d815c6d0 con 0x7fe9d8158f30 2024-05-28T14:10:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9cea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe9d80b4f00 0x7fe9d815ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9d7e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe9d815bf50 0x7fe9d806ee10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:49494/0 (socket says 172.21.15.42:49494) 2024-05-28T14:10:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9d7e00640 1 -- 172.21.15.42:0/24656249 learned_addr learned my addr 172.21.15.42:0/24656249 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9d7400640 1 --2- 172.21.15.42:0/24656249 >> v2:172.21.15.63:3300/0 conn(0x7fe9d8158f30 0x7fe9d8072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9cea00640 1 -- 172.21.15.42:0/24656249 >> v2:172.21.15.63:3300/0 conn(0x7fe9d8158f30 msgr2=0x7fe9d8072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9cea00640 1 --2- 172.21.15.42:0/24656249 >> v2:172.21.15.63:3300/0 conn(0x7fe9d8158f30 0x7fe9d8072610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9cea00640 1 -- 172.21.15.42:0/24656249 >> v1:172.21.15.42:6789/0 conn(0x7fe9d815bf50 legacy=0x7fe9d806ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.844+0000 7fe9cea00640 1 -- 172.21.15.42:0/24656249 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d8071e10 con 0x7fe9d80b4f00 2024-05-28T14:10:21.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.840+0000 7fe9d6a00640 1 -- 172.21.15.42:0/24656249 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 970363358 0 0) 0x7fe9d815c3f0 con 0x7fe9d815bf50 2024-05-28T14:10:21.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.844+0000 7fe9cea00640 1 --2- 172.21.15.42:0/24656249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9d80b4f00 0x7fe9d815ba10 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80b2710 tx=0x7fe9c001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12530ed7cc9b26e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:21.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.844+0000 7fe9d6a00640 1 -- 172.21.15.42:0/24656249 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00b0070 con 0x7fe9d80b4f00 2024-05-28T14:10:21.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.844+0000 7fe9d6a00640 1 -- 172.21.15.42:0/24656249 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c00ab600 con 0x7fe9d80b4f00 2024-05-28T14:10:21.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.844+0000 7fe9d6a00640 1 -- 172.21.15.42:0/24656249 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00ab9d0 con 0x7fe9d80b4f00 2024-05-28T14:10:21.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.844+0000 7fe9de400640 1 -- 172.21.15.42:0/24656249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9d80b4f00 msgr2=0x7fe9d815ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.844+0000 7fe9de400640 1 --2- 172.21.15.42:0/24656249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9d80b4f00 0x7fe9d815ba10 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80b2710 tx=0x7fe9c001c3c0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.844+0000 7fe9de400640 1 -- 172.21.15.42:0/24656249 shutdown_connections 2024-05-28T14:10:21.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.844+0000 7fe9de400640 1 --2- 172.21.15.42:0/24656249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9d80b4f00 0x7fe9d815ba10 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.844+0000 7fe9de400640 1 --2- 172.21.15.42:0/24656249 >> v2:172.21.15.63:3300/0 conn(0x7fe9d8158f30 0x7fe9d8072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.844+0000 7fe9de400640 1 -- 172.21.15.42:0/24656249 >> 172.21.15.42:0/24656249 conn(0x7fe9d806bff0 msgr2=0x7fe9d80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:21.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.844+0000 7fe9de400640 1 -- 172.21.15.42:0/24656249 shutdown_connections 2024-05-28T14:10:21.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.844+0000 7fe9de400640 1 -- 172.21.15.42:0/24656249 wait complete. 2024-05-28T14:10:21.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9de400640 1 Processor -- start 2024-05-28T14:10:21.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9de400640 1 -- start start 2024-05-28T14:10:21.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9de400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9d80b4f00 0x7fe9d814ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9de400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9d8158f30 0x7fe9d81f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9de400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9d815bf50 0x7fe9d81f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9de400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d806f6a0 con 0x7fe9d80b4f00 2024-05-28T14:10:21.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9de400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d806f520 con 0x7fe9d815bf50 2024-05-28T14:10:21.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9de400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d80b39f0 con 0x7fe9d8158f30 2024-05-28T14:10:21.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9d7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9d80b4f00 0x7fe9d814ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9d7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9d80b4f00 0x7fe9d814ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48510/0 (socket says 172.21.15.42:48510) 2024-05-28T14:10:21.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9d7400640 1 -- 172.21.15.42:0/984740068 learned_addr learned my addr 172.21.15.42:0/984740068 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9d7e00640 1 --2- 172.21.15.42:0/984740068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9d815bf50 0x7fe9d81f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9cea00640 1 --2- 172.21.15.42:0/984740068 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9d8158f30 0x7fe9d81f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9d7400640 1 -- 172.21.15.42:0/984740068 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9d8158f30 msgr2=0x7fe9d81f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9d7400640 1 --2- 172.21.15.42:0/984740068 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9d8158f30 0x7fe9d81f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9d7400640 1 -- 172.21.15.42:0/984740068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9d815bf50 msgr2=0x7fe9d81f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9d7400640 1 --2- 172.21.15.42:0/984740068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9d815bf50 0x7fe9d81f7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9d7400640 1 -- 172.21.15.42:0/984740068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d80cdf40 con 0x7fe9d80b4f00 2024-05-28T14:10:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9d7400640 1 --2- 172.21.15.42:0/984740068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9d80b4f00 0x7fe9d814ee40 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800d800 tx=0x7fe9c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9cfe00640 1 -- 172.21.15.42:0/984740068 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c800b6c0 con 0x7fe9d80b4f00 2024-05-28T14:10:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9cfe00640 1 -- 172.21.15.42:0/984740068 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c8019070 con 0x7fe9d80b4f00 2024-05-28T14:10:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9cfe00640 1 -- 172.21.15.42:0/984740068 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c800ba60 con 0x7fe9d80b4f00 2024-05-28T14:10:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9de400640 1 -- 172.21.15.42:0/984740068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d80ce220 con 0x7fe9d80b4f00 2024-05-28T14:10:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.848+0000 7fe9de400640 1 -- 172.21.15.42:0/984740068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d80ce7e0 con 0x7fe9d80b4f00 2024-05-28T14:10:21.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.852+0000 7fe9de400640 1 -- 172.21.15.42:0/984740068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d8068a90 con 0x7fe9d80b4f00 2024-05-28T14:10:21.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.852+0000 7fe9cfe00640 1 -- 172.21.15.42:0/984740068 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7fe9c800bc00 con 0x7fe9d80b4f00 2024-05-28T14:10:21.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.852+0000 7fe9cfe00640 1 --2- 172.21.15.42:0/984740068 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe9b4069760 0x7fe9b406bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:21.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.852+0000 7fe9cfe00640 1 -- 172.21.15.42:0/984740068 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7fe9c8085f20 con 0x7fe9d80b4f00 2024-05-28T14:10:21.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.852+0000 7fe9cea00640 1 --2- 172.21.15.42:0/984740068 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe9b4069760 0x7fe9b406bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:21.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.852+0000 7fe9cea00640 1 --2- 172.21.15.42:0/984740068 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe9b4069760 0x7fe9b406bc20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00ad000 tx=0x7fe9c0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:21.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:21.856+0000 7fe9cfe00640 1 -- 172.21.15.42:0/984740068 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe9c8057af0 con 0x7fe9d80b4f00 2024-05-28T14:10:22.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:22.084+0000 7fe9de400640 1 -- 172.21.15.42:0/984740068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe9d80af9d0 con 0x7fe9d80b4f00 2024-05-28T14:10:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:22.084+0000 7fe9cfe00640 1 -- 172.21.15.42:0/984740068 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2358 (secure 0 0 0) 0x7fe9c80573f0 con 0x7fe9d80b4f00 2024-05-28T14:10:22.067 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:22.067 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:22.067 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:10:22.067 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:10:22.068 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:10:22.068 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:10:22.068 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:10:22.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:22.068 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:10:22.068 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:10:22.068 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:10:22.068 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:10:22.068 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:10:22.068 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:22.068 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:10:22.068 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:10:22.068 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:10:22.068 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:10:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 743, 2024-05-28T14:10:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:10:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:10:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:10:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:10:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:10:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 92, 2024-05-28T14:10:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:10:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:10:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:10:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:10:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:10:22.070 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:10:22.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:22.070 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:10:22.070 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:10:22.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:22.070 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:10:22.070 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:10:22.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:22.070 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:22.070 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:10:22.070 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:10:22.070 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1532, 2024-05-28T14:10:22.070 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 6305003371, 2024-05-28T14:10:22.070 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 12960194560, 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 760133918720, 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_objects_per_sec": 1, 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_bytes_per_sec": 5940172, 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_keys_per_sec": 0, 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects_recovered": 17, 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: "num_bytes_recovered": 71303168, 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: "num_keys_recovered": 0, 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 259271584, 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 290389244, 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 62, 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 75 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:10:22.071 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 55, 2024-05-28T14:10:22.072 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 6, 2024-05-28T14:10:22.072 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:10:22.072 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:10:22.072 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:10:22.072 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:10:22.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:22.072 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 6, 2024-05-28T14:10:22.072 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:10:22.072 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:10:22.072 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:10:22.072 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 56621 2024-05-28T14:10:22.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:22.072 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:22.072 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 14, 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:07:33.038180+0000", 2024-05-28T14:10:22.073 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:10:22.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:22.074 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:10:22.074 INFO:teuthology.orchestra.run.smithi042.stdout: "bae8f9c0-5c5d-4728-8b27-e79ef1ba84dd": { 2024-05-28T14:10:22.074 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 30.0 seconds before exiting (7s)\n [============================] ", 2024-05-28T14:10:22.074 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:10:22.074 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:10:22.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:22.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:22.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:22.084+0000 7fe9de400640 1 -- 172.21.15.42:0/984740068 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe9b4069760 msgr2=0x7fe9b406bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:22.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:22.084+0000 7fe9de400640 1 --2- 172.21.15.42:0/984740068 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe9b4069760 0x7fe9b406bc20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00ad000 tx=0x7fe9c0008040 comp rx=0 tx=0).stop 2024-05-28T14:10:22.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:22.084+0000 7fe9de400640 1 -- 172.21.15.42:0/984740068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9d80b4f00 msgr2=0x7fe9d814ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:22.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:22.084+0000 7fe9de400640 1 --2- 172.21.15.42:0/984740068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9d80b4f00 0x7fe9d814ee40 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800d800 tx=0x7fe9c800dcc0 comp rx=0 tx=0).stop 2024-05-28T14:10:22.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:22.084+0000 7fe9de400640 1 -- 172.21.15.42:0/984740068 shutdown_connections 2024-05-28T14:10:22.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:22.084+0000 7fe9de400640 1 --2- 172.21.15.42:0/984740068 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe9b4069760 0x7fe9b406bc20 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:22.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:22.084+0000 7fe9de400640 1 --2- 172.21.15.42:0/984740068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9d815bf50 0x7fe9d81f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:22.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:22.084+0000 7fe9de400640 1 --2- 172.21.15.42:0/984740068 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9d8158f30 0x7fe9d81f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:22.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:22.084+0000 7fe9de400640 1 --2- 172.21.15.42:0/984740068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9d80b4f00 0x7fe9d814ee40 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:22.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:22.084+0000 7fe9de400640 1 -- 172.21.15.42:0/984740068 >> 172.21.15.42:0/984740068 conn(0x7fe9d806bff0 msgr2=0x7fe9d80b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:22.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:22.084+0000 7fe9de400640 1 -- 172.21.15.42:0/984740068 shutdown_connections 2024-05-28T14:10:22.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:22.084+0000 7fe9de400640 1 -- 172.21.15.42:0/984740068 wait complete. 2024-05-28T14:10:27.090 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:10:27.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.260+0000 7fd437200640 1 Processor -- start 2024-05-28T14:10:27.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.260+0000 7fd437200640 1 -- start start 2024-05-28T14:10:27.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.260+0000 7fd437200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd4300b6770 0x7fd430167540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:27.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.260+0000 7fd437200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4300d0760 con 0x7fd4300cd090 2024-05-28T14:10:27.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.260+0000 7fd437200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4300d0a30 con 0x7fd43015fd00 2024-05-28T14:10:27.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.260+0000 7fd437200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd4300b52d0 con 0x7fd4300b6770 2024-05-28T14:10:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.260+0000 7fd435200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd4300b6770 0x7fd430167540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.260+0000 7fd42f400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd4300cd090 0x7fd4300cd520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:49510/0 (socket says 172.21.15.42:49510) 2024-05-28T14:10:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.260+0000 7fd42f400640 1 -- 172.21.15.42:0/1413301865 learned_addr learned my addr 172.21.15.42:0/1413301865 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.260+0000 7fd435200640 1 -- 172.21.15.42:0/1413301865 >> v1:172.21.15.63:6789/0 conn(0x7fd43015fd00 legacy=0x7fd4300d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.260+0000 7fd435200640 1 -- 172.21.15.42:0/1413301865 >> v1:172.21.15.42:6789/0 conn(0x7fd4300cd090 legacy=0x7fd4300cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.260+0000 7fd435200640 1 -- 172.21.15.42:0/1413301865 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4300d0d10 con 0x7fd4300b6770 2024-05-28T14:10:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.264+0000 7fd435200640 1 --2- 172.21.15.42:0/1413301865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4300b6770 0x7fd430167540 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fd428009d90 tx=0x7fd42801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab26b3aa4756fb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.264+0000 7fd42ea00640 1 -- 172.21.15.42:0/1413301865 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4280b2070 con 0x7fd4300b6770 2024-05-28T14:10:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.264+0000 7fd42ea00640 1 -- 172.21.15.42:0/1413301865 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4280a5970 con 0x7fd4300b6770 2024-05-28T14:10:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.264+0000 7fd42ea00640 1 -- 172.21.15.42:0/1413301865 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4280a5d40 con 0x7fd4300b6770 2024-05-28T14:10:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.264+0000 7fd437200640 1 -- 172.21.15.42:0/1413301865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4300b6770 msgr2=0x7fd430167540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.264+0000 7fd437200640 1 --2- 172.21.15.42:0/1413301865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4300b6770 0x7fd430167540 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fd428009d90 tx=0x7fd42801c8b0 comp rx=0 tx=0).stop 2024-05-28T14:10:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.264+0000 7fd437200640 1 -- 172.21.15.42:0/1413301865 shutdown_connections 2024-05-28T14:10:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.264+0000 7fd437200640 1 --2- 172.21.15.42:0/1413301865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4300b6770 0x7fd430167540 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.264+0000 7fd437200640 1 -- 172.21.15.42:0/1413301865 >> 172.21.15.42:0/1413301865 conn(0x7fd4300b3f70 msgr2=0x7fd4300b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.264+0000 7fd437200640 1 -- 172.21.15.42:0/1413301865 shutdown_connections 2024-05-28T14:10:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.264+0000 7fd437200640 1 -- 172.21.15.42:0/1413301865 wait complete. 2024-05-28T14:10:27.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.264+0000 7fd437200640 1 Processor -- start 2024-05-28T14:10:27.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd437200640 1 -- start start 2024-05-28T14:10:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd437200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4300b6770 0x7fd4300b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd437200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4300cd090 0x7fd4300af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd437200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43015fd00 0x7fd4300cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd437200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd4300b3650 con 0x7fd43015fd00 2024-05-28T14:10:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd437200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd4300b34d0 con 0x7fd4300cd090 2024-05-28T14:10:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd437200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd43016a6b0 con 0x7fd4300b6770 2024-05-28T14:10:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd435200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43015fd00 0x7fd4300cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd435200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43015fd00 0x7fd4300cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48538/0 (socket says 172.21.15.42:48538) 2024-05-28T14:10:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd435200640 1 -- 172.21.15.42:0/3678101115 learned_addr learned my addr 172.21.15.42:0/3678101115 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:27.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd42fe00640 1 --2- 172.21.15.42:0/3678101115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4300b6770 0x7fd4300b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:27.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd42f400640 1 --2- 172.21.15.42:0/3678101115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4300cd090 0x7fd4300af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:27.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd435200640 1 -- 172.21.15.42:0/3678101115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4300b6770 msgr2=0x7fd4300b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:27.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd435200640 1 --2- 172.21.15.42:0/3678101115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4300b6770 0x7fd4300b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:27.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd435200640 1 -- 172.21.15.42:0/3678101115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4300cd090 msgr2=0x7fd4300af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:27.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd435200640 1 --2- 172.21.15.42:0/3678101115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4300cd090 0x7fd4300af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:27.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd435200640 1 -- 172.21.15.42:0/3678101115 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4300cc320 con 0x7fd43015fd00 2024-05-28T14:10:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd435200640 1 --2- 172.21.15.42:0/3678101115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43015fd00 0x7fd4300cab40 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fd42801a4e0 tx=0x7fd428019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd427e00640 1 -- 172.21.15.42:0/3678101115 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4280b2050 con 0x7fd43015fd00 2024-05-28T14:10:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd427e00640 1 -- 172.21.15.42:0/3678101115 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4280be070 con 0x7fd43015fd00 2024-05-28T14:10:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd427e00640 1 -- 172.21.15.42:0/3678101115 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4280adb50 con 0x7fd43015fd00 2024-05-28T14:10:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd437200640 1 -- 172.21.15.42:0/3678101115 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4300c7e70 con 0x7fd43015fd00 2024-05-28T14:10:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd437200640 1 -- 172.21.15.42:0/3678101115 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4300c8430 con 0x7fd43015fd00 2024-05-28T14:10:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.268+0000 7fd437200640 1 -- 172.21.15.42:0/3678101115 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd430068a90 con 0x7fd43015fd00 2024-05-28T14:10:27.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.272+0000 7fd427e00640 1 -- 172.21.15.42:0/3678101115 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7fd4280adcf0 con 0x7fd43015fd00 2024-05-28T14:10:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.272+0000 7fd427e00640 1 --2- 172.21.15.42:0/3678101115 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd410069730 0x7fd41006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.272+0000 7fd427e00640 1 -- 172.21.15.42:0/3678101115 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7fd428120c70 con 0x7fd43015fd00 2024-05-28T14:10:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.272+0000 7fd42fe00640 1 --2- 172.21.15.42:0/3678101115 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd410069730 0x7fd41006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.272+0000 7fd42fe00640 1 --2- 172.21.15.42:0/3678101115 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd410069730 0x7fd41006bbf0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd4300719e0 tx=0x7fd420009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:27.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.276+0000 7fd427e00640 1 -- 172.21.15.42:0/3678101115 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd428016560 con 0x7fd43015fd00 2024-05-28T14:10:27.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.500+0000 7fd437200640 1 -- 172.21.15.42:0/3678101115 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd4300c8770 con 0x7fd43015fd00 2024-05-28T14:10:27.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.500+0000 7fd427e00640 1 -- 172.21.15.42:0/3678101115 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2124 (secure 0 0 0) 0x7fd4280f2280 con 0x7fd43015fd00 2024-05-28T14:10:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:27.486 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:10:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:10:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:10:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:10:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:10:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:10:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:10:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:10:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:10:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:10:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:10:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:10:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:10:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:10:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 748, 2024-05-28T14:10:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:10:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 92, 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:10:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:10:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:10:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:10:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:10:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1565, 2024-05-28T14:10:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 6443486009, 2024-05-28T14:10:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 13727891456, 2024-05-28T14:10:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 759366221824, 2024-05-28T14:10:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:10:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 268357380, 2024-05-28T14:10:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 268381266, 2024-05-28T14:10:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 64, 2024-05-28T14:10:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 72 2024-05-28T14:10:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:10:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 55, 2024-05-28T14:10:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 6, 2024-05-28T14:10:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:10:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:10:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:10:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:10:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 6, 2024-05-28T14:10:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:10:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:10:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:10:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 56621 2024-05-28T14:10:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:10:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:10:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:10:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:10:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:10:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:10:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:10:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:10:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:10:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:10:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 14, 2024-05-28T14:10:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:07:33.038180+0000", 2024-05-28T14:10:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:10:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:10:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "bae8f9c0-5c5d-4728-8b27-e79ef1ba84dd": { 2024-05-28T14:10:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 26.0 seconds before exiting (11s)\n [============================] ", 2024-05-28T14:10:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:10:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:10:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:27.493 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:27.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.504+0000 7fd437200640 1 -- 172.21.15.42:0/3678101115 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd410069730 msgr2=0x7fd41006bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:27.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.504+0000 7fd437200640 1 --2- 172.21.15.42:0/3678101115 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd410069730 0x7fd41006bbf0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd4300719e0 tx=0x7fd420009040 comp rx=0 tx=0).stop 2024-05-28T14:10:27.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.504+0000 7fd437200640 1 -- 172.21.15.42:0/3678101115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43015fd00 msgr2=0x7fd4300cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:27.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.504+0000 7fd437200640 1 --2- 172.21.15.42:0/3678101115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43015fd00 0x7fd4300cab40 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fd42801a4e0 tx=0x7fd428019650 comp rx=0 tx=0).stop 2024-05-28T14:10:27.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.504+0000 7fd437200640 1 -- 172.21.15.42:0/3678101115 shutdown_connections 2024-05-28T14:10:27.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.504+0000 7fd437200640 1 --2- 172.21.15.42:0/3678101115 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd410069730 0x7fd41006bbf0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:27.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.504+0000 7fd437200640 1 --2- 172.21.15.42:0/3678101115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43015fd00 0x7fd4300cab40 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:27.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.504+0000 7fd437200640 1 --2- 172.21.15.42:0/3678101115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4300cd090 0x7fd4300af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:27.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.504+0000 7fd437200640 1 --2- 172.21.15.42:0/3678101115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4300b6770 0x7fd4300b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:27.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.504+0000 7fd437200640 1 -- 172.21.15.42:0/3678101115 >> 172.21.15.42:0/3678101115 conn(0x7fd4300b3f70 msgr2=0x7fd43015da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:27.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.504+0000 7fd437200640 1 -- 172.21.15.42:0/3678101115 shutdown_connections 2024-05-28T14:10:27.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:27.504+0000 7fd437200640 1 -- 172.21.15.42:0/3678101115 wait complete. 2024-05-28T14:10:32.509 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:10:32.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.676+0000 7ff51cc00640 1 Processor -- start 2024-05-28T14:10:32.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.676+0000 7ff51cc00640 1 -- start start 2024-05-28T14:10:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.676+0000 7ff51cc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff5180b4fb0 0x7ff518071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.676+0000 7ff51cc00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5181623d0 con 0x7ff51815f310 2024-05-28T14:10:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.676+0000 7ff51cc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5181626b0 con 0x7ff518072220 2024-05-28T14:10:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.676+0000 7ff51cc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff518162fe0 con 0x7ff5180b4fb0 2024-05-28T14:10:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.676+0000 7ff50d400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff5180b4fb0 0x7ff518071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.680+0000 7ff515e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff51815f310 0x7ff51815f6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53480/0 (socket says 172.21.15.42:53480) 2024-05-28T14:10:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.680+0000 7ff515e00640 1 -- 172.21.15.42:0/3383474982 learned_addr learned my addr 172.21.15.42:0/3383474982 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.680+0000 7ff50d400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff5180b4fb0 0x7ff518071ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34294/0 (socket says 172.21.15.42:34294) 2024-05-28T14:10:32.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.680+0000 7ff50d400640 1 -- 172.21.15.42:0/3383474982 >> v1:172.21.15.63:6789/0 conn(0x7ff518072220 legacy=0x7ff51806dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:32.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.680+0000 7ff50d400640 1 -- 172.21.15.42:0/3383474982 >> v1:172.21.15.42:6789/0 conn(0x7ff51815f310 legacy=0x7ff51815f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:32.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.680+0000 7ff50d400640 1 -- 172.21.15.42:0/3383474982 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff518162990 con 0x7ff5180b4fb0 2024-05-28T14:10:32.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.680+0000 7ff50d400640 1 --2- 172.21.15.42:0/3383474982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5180b4fb0 0x7ff518071ce0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff5000097c0 tx=0x7ff50001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6767c3abfebe66 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:32.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.680+0000 7ff515400640 1 -- 172.21.15.42:0/3383474982 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5000b0070 con 0x7ff5180b4fb0 2024-05-28T14:10:32.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.680+0000 7ff515400640 1 -- 172.21.15.42:0/3383474982 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5000ab600 con 0x7ff5180b4fb0 2024-05-28T14:10:32.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.680+0000 7ff515400640 1 -- 172.21.15.42:0/3383474982 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5000ab9d0 con 0x7ff5180b4fb0 2024-05-28T14:10:32.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 -- 172.21.15.42:0/3383474982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5180b4fb0 msgr2=0x7ff518071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:32.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 --2- 172.21.15.42:0/3383474982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5180b4fb0 0x7ff518071ce0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff5000097c0 tx=0x7ff50001c410 comp rx=0 tx=0).stop 2024-05-28T14:10:32.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 -- 172.21.15.42:0/3383474982 shutdown_connections 2024-05-28T14:10:32.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 --2- 172.21.15.42:0/3383474982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5180b4fb0 0x7ff518071ce0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:32.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 -- 172.21.15.42:0/3383474982 >> 172.21.15.42:0/3383474982 conn(0x7ff51806bff0 msgr2=0x7ff5180b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:32.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 -- 172.21.15.42:0/3383474982 shutdown_connections 2024-05-28T14:10:32.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 -- 172.21.15.42:0/3383474982 wait complete. 2024-05-28T14:10:32.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 Processor -- start 2024-05-28T14:10:32.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 -- start start 2024-05-28T14:10:32.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff518072220 0x7ff51814a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:32.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5180b4fb0 0x7ff51814c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:32.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff51815f310 0x7ff51814c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:32.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff51806f030 con 0x7ff5180b4fb0 2024-05-28T14:10:32.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff51806eeb0 con 0x7ff518072220 2024-05-28T14:10:32.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff51cc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff51816c6e0 con 0x7ff51815f310 2024-05-28T14:10:32.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff50d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5180b4fb0 0x7ff51814c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:32.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff50d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5180b4fb0 0x7ff51814c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34306/0 (socket says 172.21.15.42:34306) 2024-05-28T14:10:32.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff50d400640 1 -- 172.21.15.42:0/766417646 learned_addr learned my addr 172.21.15.42:0/766417646 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:32.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff515e00640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff518072220 0x7ff51814a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:32.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff516800640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff51815f310 0x7ff51814c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:32.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff50d400640 1 -- 172.21.15.42:0/766417646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff51815f310 msgr2=0x7ff51814c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:32.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff50d400640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff51815f310 0x7ff51814c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:32.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff50d400640 1 -- 172.21.15.42:0/766417646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff518072220 msgr2=0x7ff51814a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:32.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff50d400640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff518072220 0x7ff51814a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:32.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff50d400640 1 -- 172.21.15.42:0/766417646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5181496d0 con 0x7ff5180b4fb0 2024-05-28T14:10:32.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff516800640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff51815f310 0x7ff51814c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:10:32.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.684+0000 7ff50d400640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5180b4fb0 0x7ff51814c050 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff5000ae040 tx=0x7ff5000191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:32.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.688+0000 7ff50ea00640 1 -- 172.21.15.42:0/766417646 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5000b0040 con 0x7ff5180b4fb0 2024-05-28T14:10:32.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.688+0000 7ff50ea00640 1 -- 172.21.15.42:0/766417646 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5000b9070 con 0x7ff5180b4fb0 2024-05-28T14:10:32.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.688+0000 7ff51cc00640 1 -- 172.21.15.42:0/766417646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5181499b0 con 0x7ff5180b4fb0 2024-05-28T14:10:32.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.688+0000 7ff50ea00640 1 -- 172.21.15.42:0/766417646 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5000aaa30 con 0x7ff5180b4fb0 2024-05-28T14:10:32.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.688+0000 7ff51cc00640 1 -- 172.21.15.42:0/766417646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff518149f70 con 0x7ff5180b4fb0 2024-05-28T14:10:32.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.688+0000 7ff51cc00640 1 -- 172.21.15.42:0/766417646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff518068a90 con 0x7ff5180b4fb0 2024-05-28T14:10:32.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.688+0000 7ff50ea00640 1 -- 172.21.15.42:0/766417646 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7ff5000aabd0 con 0x7ff5180b4fb0 2024-05-28T14:10:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.688+0000 7ff50ea00640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff4f4069760 0x7ff4f406bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.688+0000 7ff50ea00640 1 -- 172.21.15.42:0/766417646 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7ff50011fc10 con 0x7ff5180b4fb0 2024-05-28T14:10:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.688+0000 7ff515e00640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff4f4069760 0x7ff4f406bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:32.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.692+0000 7ff515e00640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff4f4069760 0x7ff4f406bc20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff508005890 tx=0x7ff508009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:32.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.692+0000 7ff50ea00640 1 -- 172.21.15.42:0/766417646 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff500016090 con 0x7ff5180b4fb0 2024-05-28T14:10:32.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.916+0000 7ff51cc00640 1 -- 172.21.15.42:0/766417646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff5180b1c50 con 0x7ff5180b4fb0 2024-05-28T14:10:32.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.916+0000 7ff50ea00640 1 -- 172.21.15.42:0/766417646 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2577 (secure 0 0 0) 0x7ff5000f1870 con 0x7ff5180b4fb0 2024-05-28T14:10:32.901 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:32.901 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:32.901 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:10:32.901 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:10:32.901 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:10:32.901 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:10:32.902 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:10:32.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:32.902 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:10:32.902 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:10:32.902 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:10:32.902 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:10:32.902 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:10:32.902 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:32.902 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:10:32.902 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:10:32.902 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:10:32.902 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:10:32.902 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:32.903 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 754, 2024-05-28T14:10:32.903 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:10:32.903 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:10:32.903 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:10:32.903 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:10:32.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:32.903 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:10:32.903 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 92, 2024-05-28T14:10:32.903 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:10:32.903 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:10:32.903 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:10:32.903 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:10:32.903 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:10:32.904 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:10:32.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:32.904 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:10:32.904 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:10:32.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:32.904 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:10:32.904 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:10:32.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:32.904 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:32.904 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:10:32.904 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:10:32.904 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1565, 2024-05-28T14:10:32.904 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 6443511959, 2024-05-28T14:10:32.904 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 13799251968, 2024-05-28T14:10:32.905 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 759294861312, 2024-05-28T14:10:32.905 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:10:32.905 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 228175495, 2024-05-28T14:10:32.905 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 231686812, 2024-05-28T14:10:32.905 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 54, 2024-05-28T14:10:32.905 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 61 2024-05-28T14:10:32.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:32.905 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:10:32.905 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 55, 2024-05-28T14:10:32.905 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 6, 2024-05-28T14:10:32.905 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:10:32.905 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:10:32.905 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:10:32.906 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:10:32.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:32.906 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 6, 2024-05-28T14:10:32.906 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:10:32.906 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:10:32.906 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:10:32.906 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 56621 2024-05-28T14:10:32.906 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:32.906 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:32.906 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:10:32.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:32.906 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:10:32.906 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:10:32.907 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:10:32.907 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:10:32.907 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:10:32.907 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:10:32.907 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:10:32.907 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:32.907 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:10:32.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:32.907 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:10:32.907 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 15, 2024-05-28T14:10:32.907 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:10:27.125922+0000", 2024-05-28T14:10:32.907 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-05-28T14:10:32.907 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-05-28T14:10:32.908 INFO:teuthology.orchestra.run.smithi042.stdout: "daemons": { 2024-05-28T14:10:32.908 INFO:teuthology.orchestra.run.smithi042.stdout: "summary": "", 2024-05-28T14:10:32.908 INFO:teuthology.orchestra.run.smithi042.stdout: "b": { 2024-05-28T14:10:32.908 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:10:32.908 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:10:32.908 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:10:32.908 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:10:32.908 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:10:32.908 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:10:32.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:32.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:32.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:32.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:32.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:32.909 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:10:32.909 INFO:teuthology.orchestra.run.smithi042.stdout: "bae8f9c0-5c5d-4728-8b27-e79ef1ba84dd": { 2024-05-28T14:10:32.909 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 20.0 seconds before exiting (17s)\n [============================] ", 2024-05-28T14:10:32.909 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:10:32.909 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:10:32.909 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:32.909 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:32.909 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:32.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.920+0000 7ff51cc00640 1 -- 172.21.15.42:0/766417646 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff4f4069760 msgr2=0x7ff4f406bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:32.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.920+0000 7ff51cc00640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff4f4069760 0x7ff4f406bc20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff508005890 tx=0x7ff508009040 comp rx=0 tx=0).stop 2024-05-28T14:10:32.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.920+0000 7ff51cc00640 1 -- 172.21.15.42:0/766417646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5180b4fb0 msgr2=0x7ff51814c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:32.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.920+0000 7ff51cc00640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5180b4fb0 0x7ff51814c050 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff5000ae040 tx=0x7ff5000191b0 comp rx=0 tx=0).stop 2024-05-28T14:10:32.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.920+0000 7ff51cc00640 1 -- 172.21.15.42:0/766417646 shutdown_connections 2024-05-28T14:10:32.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.920+0000 7ff51cc00640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff4f4069760 0x7ff4f406bc20 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:32.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.920+0000 7ff51cc00640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff51815f310 0x7ff51814c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:32.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.920+0000 7ff51cc00640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5180b4fb0 0x7ff51814c050 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:32.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.920+0000 7ff51cc00640 1 --2- 172.21.15.42:0/766417646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff518072220 0x7ff51814a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:32.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.920+0000 7ff51cc00640 1 -- 172.21.15.42:0/766417646 >> 172.21.15.42:0/766417646 conn(0x7ff51806bff0 msgr2=0x7ff518161760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:32.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.920+0000 7ff51cc00640 1 -- 172.21.15.42:0/766417646 shutdown_connections 2024-05-28T14:10:32.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:32.920+0000 7ff51cc00640 1 -- 172.21.15.42:0/766417646 wait complete. 2024-05-28T14:10:37.924 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:10:38.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.092+0000 7f2b85c00640 1 Processor -- start 2024-05-28T14:10:38.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.092+0000 7f2b85c00640 1 -- start start 2024-05-28T14:10:38.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.096+0000 7f2b85c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2b800cf6a0 0x7f2b801609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:38.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.096+0000 7f2b85c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b8015d170 con 0x7f2b8015ca70 2024-05-28T14:10:38.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.096+0000 7f2b85c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b8015d450 con 0x7f2b8015a160 2024-05-28T14:10:38.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.096+0000 7f2b85c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f2b80160f10 con 0x7f2b800cf6a0 2024-05-28T14:10:38.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.096+0000 7f2b85200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2b800cf6a0 0x7f2b801609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:38.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.096+0000 7f2b85200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2b800cf6a0 0x7f2b801609d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37560/0 (socket says 172.21.15.42:37560) 2024-05-28T14:10:38.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.096+0000 7f2b85200640 1 -- 172.21.15.42:0/2505911233 learned_addr learned my addr 172.21.15.42:0/2505911233 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:38.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.096+0000 7f2b77400640 1 --1- 172.21.15.42:0/2505911233 >> v1:172.21.15.42:6789/0 conn(0x7f2b8015ca70 0x7f2b800cef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55438/0 (socket says 172.21.15.42:55438) 2024-05-28T14:10:38.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.096+0000 7f2b85200640 1 -- 172.21.15.42:0/2505911233 >> v1:172.21.15.63:6789/0 conn(0x7f2b8015a160 legacy=0x7f2b800cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:38.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.096+0000 7f2b85200640 1 -- 172.21.15.42:0/2505911233 >> v1:172.21.15.42:6789/0 conn(0x7f2b8015ca70 legacy=0x7f2b800cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:38.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.096+0000 7f2b85200640 1 -- 172.21.15.42:0/2505911233 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8015d730 con 0x7f2b800cf6a0 2024-05-28T14:10:38.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85200640 1 --2- 172.21.15.42:0/2505911233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b800cf6a0 0x7f2b801609d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009870 tx=0x7f2b7801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadf60c09317ea24 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:38.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b7f400640 1 -- 172.21.15.42:0/2505911233 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b780b2070 con 0x7f2b800cf6a0 2024-05-28T14:10:38.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b7f400640 1 -- 172.21.15.42:0/2505911233 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b780a9070 con 0x7f2b800cf6a0 2024-05-28T14:10:38.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b7f400640 1 -- 172.21.15.42:0/2505911233 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b780ad880 con 0x7f2b800cf6a0 2024-05-28T14:10:38.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 -- 172.21.15.42:0/2505911233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b800cf6a0 msgr2=0x7f2b801609d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:38.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 --2- 172.21.15.42:0/2505911233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b800cf6a0 0x7f2b801609d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009870 tx=0x7f2b7801c560 comp rx=0 tx=0).stop 2024-05-28T14:10:38.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 -- 172.21.15.42:0/2505911233 shutdown_connections 2024-05-28T14:10:38.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 --2- 172.21.15.42:0/2505911233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b800cf6a0 0x7f2b801609d0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:38.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 -- 172.21.15.42:0/2505911233 >> 172.21.15.42:0/2505911233 conn(0x7f2b800b3ec0 msgr2=0x7f2b800b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:38.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 -- 172.21.15.42:0/2505911233 shutdown_connections 2024-05-28T14:10:38.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 -- 172.21.15.42:0/2505911233 wait complete. 2024-05-28T14:10:38.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 Processor -- start 2024-05-28T14:10:38.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 -- start start 2024-05-28T14:10:38.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b800cf6a0 0x7f2b80158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:38.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b8015a160 0x7f2b800c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b8015ca70 0x7f2b80166620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8016a520 con 0x7f2b8015a160 2024-05-28T14:10:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8016a3a0 con 0x7f2b8015ca70 2024-05-28T14:10:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b85c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b8016a6a0 con 0x7f2b800cf6a0 2024-05-28T14:10:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b77400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b8015a160 0x7f2b800c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b77400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b8015a160 0x7f2b800c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37576/0 (socket says 172.21.15.42:37576) 2024-05-28T14:10:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.100+0000 7f2b77400640 1 -- 172.21.15.42:0/744728818 learned_addr learned my addr 172.21.15.42:0/744728818 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b85200640 1 --2- 172.21.15.42:0/744728818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b8015ca70 0x7f2b80166620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b7fe00640 1 --2- 172.21.15.42:0/744728818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b800cf6a0 0x7f2b80158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b77400640 1 -- 172.21.15.42:0/744728818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b800cf6a0 msgr2=0x7f2b80158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b77400640 1 --2- 172.21.15.42:0/744728818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b800cf6a0 0x7f2b80158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b77400640 1 -- 172.21.15.42:0/744728818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b8015ca70 msgr2=0x7f2b80166620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b77400640 1 --2- 172.21.15.42:0/744728818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b8015ca70 0x7f2b80166620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b77400640 1 -- 172.21.15.42:0/744728818 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b800c9ed0 con 0x7f2b8015a160 2024-05-28T14:10:38.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b77400640 1 --2- 172.21.15.42:0/744728818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b8015a160 0x7f2b800c94f0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800c430 tx=0x7f2b6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:38.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b7cc00640 1 -- 172.21.15.42:0/744728818 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b680074e0 con 0x7f2b8015a160 2024-05-28T14:10:38.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b7cc00640 1 -- 172.21.15.42:0/744728818 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b68007680 con 0x7f2b8015a160 2024-05-28T14:10:38.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b85c00640 1 -- 172.21.15.42:0/744728818 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b800ca1b0 con 0x7f2b8015a160 2024-05-28T14:10:38.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b7cc00640 1 -- 172.21.15.42:0/744728818 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68004a10 con 0x7f2b8015a160 2024-05-28T14:10:38.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b85c00640 1 -- 172.21.15.42:0/744728818 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b800ca770 con 0x7f2b8015a160 2024-05-28T14:10:38.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b85c00640 1 -- 172.21.15.42:0/744728818 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b80068a90 con 0x7f2b8015a160 2024-05-28T14:10:38.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b7cc00640 1 -- 172.21.15.42:0/744728818 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7f2b6801a020 con 0x7f2b8015a160 2024-05-28T14:10:38.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b7cc00640 1 --2- 172.21.15.42:0/744728818 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b5c069730 0x7f2b5c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:38.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b7cc00640 1 -- 172.21.15.42:0/744728818 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7f2b680851f0 con 0x7f2b8015a160 2024-05-28T14:10:38.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.104+0000 7f2b7fe00640 1 --2- 172.21.15.42:0/744728818 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b5c069730 0x7f2b5c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:38.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.108+0000 7f2b7fe00640 1 --2- 172.21.15.42:0/744728818 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b5c069730 0x7f2b5c06bbf0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2b801fabf0 tx=0x7f2b70009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:38.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.112+0000 7f2b7cc00640 1 -- 172.21.15.42:0/744728818 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2b6808c030 con 0x7f2b8015a160 2024-05-28T14:10:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.336+0000 7f2b85c00640 1 -- 172.21.15.42:0/744728818 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2b8015c320 con 0x7f2b8015a160 2024-05-28T14:10:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.336+0000 7f2b7cc00640 1 -- 172.21.15.42:0/744728818 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2080 (secure 0 0 0) 0x7f2b68056e50 con 0x7f2b8015a160 2024-05-28T14:10:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:38.320 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:10:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:10:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:10:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:10:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:10:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:10:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:10:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:10:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:10:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:10:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:10:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:10:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:10:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:10:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 759, 2024-05-28T14:10:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:10:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:10:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:10:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:10:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:10:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 92, 2024-05-28T14:10:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:10:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:10:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:10:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:10:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:10:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:10:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:10:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:10:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:10:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:10:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:10:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:10:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1565, 2024-05-28T14:10:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 6443511959, 2024-05-28T14:10:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 13799251968, 2024-05-28T14:10:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 759294861312, 2024-05-28T14:10:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:10:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 2388, 2024-05-28T14:10:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:10:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:10:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:10:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 55, 2024-05-28T14:10:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 6, 2024-05-28T14:10:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:10:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:10:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:10:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:10:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 6, 2024-05-28T14:10:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:10:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:10:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:10:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 56621 2024-05-28T14:10:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:10:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:10:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:10:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:10:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:10:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:10:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:10:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:10:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:10:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:10:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 16, 2024-05-28T14:10:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:10:37.128981+0000", 2024-05-28T14:10:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:10:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:10:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "bae8f9c0-5c5d-4728-8b27-e79ef1ba84dd": { 2024-05-28T14:10:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 14.0 seconds before exiting (23s)\n [============================] ", 2024-05-28T14:10:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:10:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:10:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:38.327 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:38.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.336+0000 7f2b85c00640 1 -- 172.21.15.42:0/744728818 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b5c069730 msgr2=0x7f2b5c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:38.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.336+0000 7f2b85c00640 1 --2- 172.21.15.42:0/744728818 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b5c069730 0x7f2b5c06bbf0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2b801fabf0 tx=0x7f2b70009040 comp rx=0 tx=0).stop 2024-05-28T14:10:38.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.336+0000 7f2b85c00640 1 -- 172.21.15.42:0/744728818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b8015a160 msgr2=0x7f2b800c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.336+0000 7f2b85c00640 1 --2- 172.21.15.42:0/744728818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b8015a160 0x7f2b800c94f0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800c430 tx=0x7f2b6800c8f0 comp rx=0 tx=0).stop 2024-05-28T14:10:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.336+0000 7f2b85c00640 1 -- 172.21.15.42:0/744728818 shutdown_connections 2024-05-28T14:10:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.336+0000 7f2b85c00640 1 --2- 172.21.15.42:0/744728818 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2b5c069730 0x7f2b5c06bbf0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.336+0000 7f2b85c00640 1 --2- 172.21.15.42:0/744728818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b8015ca70 0x7f2b80166620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.336+0000 7f2b85c00640 1 --2- 172.21.15.42:0/744728818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b8015a160 0x7f2b800c94f0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.336+0000 7f2b85c00640 1 --2- 172.21.15.42:0/744728818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b800cf6a0 0x7f2b80158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.336+0000 7f2b85c00640 1 -- 172.21.15.42:0/744728818 >> 172.21.15.42:0/744728818 conn(0x7f2b800b3ec0 msgr2=0x7f2b8015a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.336+0000 7f2b85c00640 1 -- 172.21.15.42:0/744728818 shutdown_connections 2024-05-28T14:10:38.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:38.336+0000 7f2b85c00640 1 -- 172.21.15.42:0/744728818 wait complete. 2024-05-28T14:10:43.342 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:10:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe550c00640 1 Processor -- start 2024-05-28T14:10:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe550c00640 1 -- start start 2024-05-28T14:10:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe550c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe54c15c430 0x7fe54c0cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe550c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe54c15ff80 0x7fe54c1603e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe550c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe54c0b0430 con 0x7fe54c160920 2024-05-28T14:10:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe550c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe54c0cf160 con 0x7fe54c15c430 2024-05-28T14:10:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe550c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe54c0ce550 con 0x7fe54c15ff80 2024-05-28T14:10:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe54c160920 0x7fe54c15c810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55448/0 (socket says 172.21.15.42:55448) 2024-05-28T14:10:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54be00640 1 -- 172.21.15.42:0/2365834122 learned_addr learned my addr 172.21.15.42:0/2365834122 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:43.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54aa00640 1 --2- 172.21.15.42:0/2365834122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe54c15ff80 0x7fe54c1603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:43.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54b400640 1 --2- 172.21.15.42:0/2365834122 >> v2:172.21.15.63:3300/0 conn(0x7fe54c15c430 0x7fe54c0cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:43.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 -- 172.21.15.42:0/2365834122 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1063881038 0 0) 0x7fe54c0b0430 con 0x7fe54c160920 2024-05-28T14:10:43.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 -- 172.21.15.42:0/2365834122 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe530003530 con 0x7fe54c160920 2024-05-28T14:10:43.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 -- 172.21.15.42:0/2365834122 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1905935271 0 0) 0x7fe530003530 con 0x7fe54c160920 2024-05-28T14:10:43.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 -- 172.21.15.42:0/2365834122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe54c15ff80 msgr2=0x7fe54c1603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:43.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 --2- 172.21.15.42:0/2365834122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe54c15ff80 0x7fe54c1603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:43.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 -- 172.21.15.42:0/2365834122 >> v2:172.21.15.63:3300/0 conn(0x7fe54c15c430 msgr2=0x7fe54c0cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:43.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 --2- 172.21.15.42:0/2365834122 >> v2:172.21.15.63:3300/0 conn(0x7fe54c15c430 0x7fe54c0cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:43.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 -- 172.21.15.42:0/2365834122 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c0ce830 con 0x7fe54c160920 2024-05-28T14:10:43.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 -- 172.21.15.42:0/2365834122 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fe544001a00 con 0x7fe54c160920 2024-05-28T14:10:43.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 -- 172.21.15.42:0/2365834122 >> v1:172.21.15.42:6789/0 conn(0x7fe54c160920 legacy=0x7fe54c15c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:43.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 --2- 172.21.15.42:0/2365834122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5300039c0 0x7fe530003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:43.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 --2- 172.21.15.42:0/2365834122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe530004810 0x7fe530004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:43.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 --2- 172.21.15.42:0/2365834122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe54c15ff80 0x7fe5300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:43.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 -- 172.21.15.42:0/2365834122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c0cf160 con 0x7fe530004810 2024-05-28T14:10:43.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 -- 172.21.15.42:0/2365834122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c0ce550 con 0x7fe5300039c0 2024-05-28T14:10:43.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54a000640 1 -- 172.21.15.42:0/2365834122 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe53000b9d0 con 0x7fe54c15ff80 2024-05-28T14:10:43.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54aa00640 1 --2- 172.21.15.42:0/2365834122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe530004810 0x7fe530004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:43.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54be00640 1 --2- 172.21.15.42:0/2365834122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe54c15ff80 0x7fe5300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:43.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54b400640 1 --2- 172.21.15.42:0/2365834122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5300039c0 0x7fe530003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:43.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54aa00640 1 -- 172.21.15.42:0/2365834122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe54c15ff80 msgr2=0x7fe5300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:43.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54aa00640 1 --2- 172.21.15.42:0/2365834122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe54c15ff80 0x7fe5300051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:43.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54aa00640 1 -- 172.21.15.42:0/2365834122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5300039c0 msgr2=0x7fe530003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:43.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54aa00640 1 --2- 172.21.15.42:0/2365834122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5300039c0 0x7fe530003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:43.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54aa00640 1 -- 172.21.15.42:0/2365834122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe530003530 con 0x7fe530004810 2024-05-28T14:10:43.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54be00640 1 --2- 172.21.15.42:0/2365834122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe54c15ff80 0x7fe5300051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:10:43.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.512+0000 7fe54b400640 1 --2- 172.21.15.42:0/2365834122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5300039c0 0x7fe530003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:10:43.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.516+0000 7fe54aa00640 1 --2- 172.21.15.42:0/2365834122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe530004810 0x7fe530004c90 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fe538009ae0 tx=0x7fe53801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3050afe441557a69 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:43.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.516+0000 7fe54a000640 1 -- 172.21.15.42:0/2365834122 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5380b2070 con 0x7fe530004810 2024-05-28T14:10:43.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.516+0000 7fe54a000640 1 -- 172.21.15.42:0/2365834122 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5380a5580 con 0x7fe530004810 2024-05-28T14:10:43.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.516+0000 7fe54a000640 1 -- 172.21.15.42:0/2365834122 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5380096b0 con 0x7fe530004810 2024-05-28T14:10:43.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.516+0000 7fe550c00640 1 -- 172.21.15.42:0/2365834122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe530004810 msgr2=0x7fe530004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:43.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.516+0000 7fe550c00640 1 --2- 172.21.15.42:0/2365834122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe530004810 0x7fe530004c90 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fe538009ae0 tx=0x7fe53801ca70 comp rx=0 tx=0).stop 2024-05-28T14:10:43.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.516+0000 7fe54be00640 1 -- 172.21.15.42:0/2365834122 reap_dead start 2024-05-28T14:10:43.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.516+0000 7fe550c00640 1 -- 172.21.15.42:0/2365834122 shutdown_connections 2024-05-28T14:10:43.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.516+0000 7fe550c00640 1 -- 172.21.15.42:0/2365834122 >> 172.21.15.42:0/2365834122 conn(0x7fe54c073280 msgr2=0x7fe54c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:43.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.516+0000 7fe550c00640 1 -- 172.21.15.42:0/2365834122 shutdown_connections 2024-05-28T14:10:43.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.516+0000 7fe550c00640 1 -- 172.21.15.42:0/2365834122 wait complete. 2024-05-28T14:10:43.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe550c00640 1 Processor -- start 2024-05-28T14:10:43.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe550c00640 1 -- start start 2024-05-28T14:10:43.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe550c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe530004810 0x7fe54c06e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:43.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe550c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54c156540 0x7fe54c06ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:43.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe550c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe54c149270 0x7fe54c157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:43.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe550c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c160db0 con 0x7fe54c156540 2024-05-28T14:10:43.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe550c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c0cedf0 con 0x7fe54c149270 2024-05-28T14:10:43.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe550c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe54c15ebd0 con 0x7fe530004810 2024-05-28T14:10:43.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe54aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54c156540 0x7fe54c06ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:43.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe54aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54c156540 0x7fe54c06ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37588/0 (socket says 172.21.15.42:37588) 2024-05-28T14:10:43.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe54aa00640 1 -- 172.21.15.42:0/3703499401 learned_addr learned my addr 172.21.15.42:0/3703499401 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:43.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe54b400640 1 --2- 172.21.15.42:0/3703499401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe530004810 0x7fe54c06e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:43.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe54be00640 1 --2- 172.21.15.42:0/3703499401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe54c149270 0x7fe54c157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:43.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe54aa00640 1 -- 172.21.15.42:0/3703499401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe530004810 msgr2=0x7fe54c06e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:43.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe54aa00640 1 --2- 172.21.15.42:0/3703499401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe530004810 0x7fe54c06e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:43.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe54aa00640 1 -- 172.21.15.42:0/3703499401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe54c149270 msgr2=0x7fe54c157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:43.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe54aa00640 1 --2- 172.21.15.42:0/3703499401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe54c149270 0x7fe54c157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:43.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe54aa00640 1 -- 172.21.15.42:0/3703499401 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c06ee60 con 0x7fe54c156540 2024-05-28T14:10:43.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe54aa00640 1 --2- 172.21.15.42:0/3703499401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54c156540 0x7fe54c06ff00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe538009ae0 tx=0x7fe530005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:43.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe543400640 1 -- 172.21.15.42:0/3703499401 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5380b2040 con 0x7fe54c156540 2024-05-28T14:10:43.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe543400640 1 -- 172.21.15.42:0/3703499401 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53800f000 con 0x7fe54c156540 2024-05-28T14:10:43.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe543400640 1 -- 172.21.15.42:0/3703499401 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5380096b0 con 0x7fe54c156540 2024-05-28T14:10:43.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe550c00640 1 -- 172.21.15.42:0/3703499401 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54c06f140 con 0x7fe54c156540 2024-05-28T14:10:43.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.520+0000 7fe550c00640 1 -- 172.21.15.42:0/3703499401 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54c06f700 con 0x7fe54c156540 2024-05-28T14:10:43.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.524+0000 7fe550c00640 1 -- 172.21.15.42:0/3703499401 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe54c0691a0 con 0x7fe54c156540 2024-05-28T14:10:43.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.524+0000 7fe543400640 1 -- 172.21.15.42:0/3703499401 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7fe5380c0030 con 0x7fe54c156540 2024-05-28T14:10:43.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.524+0000 7fe543400640 1 --2- 172.21.15.42:0/3703499401 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe52c069730 0x7fe52c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:43.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.524+0000 7fe543400640 1 -- 172.21.15.42:0/3703499401 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7fe53811f2d0 con 0x7fe54c156540 2024-05-28T14:10:43.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.524+0000 7fe54b400640 1 --2- 172.21.15.42:0/3703499401 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe52c069730 0x7fe52c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:43.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.524+0000 7fe54b400640 1 --2- 172.21.15.42:0/3703499401 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe52c069730 0x7fe52c06bbf0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009be0 tx=0x7fe53c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:43.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.528+0000 7fe543400640 1 -- 172.21.15.42:0/3703499401 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe5380c7060 con 0x7fe54c156540 2024-05-28T14:10:43.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.756+0000 7fe550c00640 1 -- 172.21.15.42:0/3703499401 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe54c15afd0 con 0x7fe54c156540 2024-05-28T14:10:43.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.756+0000 7fe543400640 1 -- 172.21.15.42:0/3703499401 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2080 (secure 0 0 0) 0x7fe538016780 con 0x7fe54c156540 2024-05-28T14:10:43.739 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:43.740 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:43.740 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:10:43.740 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:10:43.740 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:10:43.740 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:10:43.740 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:10:43.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:43.740 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:10:43.740 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:10:43.740 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:10:43.740 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:10:43.740 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:10:43.741 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:43.741 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:10:43.741 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:10:43.741 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:10:43.741 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:10:43.741 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:43.741 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 765, 2024-05-28T14:10:43.741 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:10:43.741 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:10:43.741 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:10:43.741 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:10:43.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:43.741 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:10:43.741 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 92, 2024-05-28T14:10:43.742 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:10:43.742 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:10:43.742 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:10:43.742 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:10:43.742 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:10:43.742 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:10:43.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:43.742 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:10:43.742 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:10:43.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:43.742 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:10:43.742 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:10:43.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:43.742 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:10:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:10:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1565, 2024-05-28T14:10:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 6443511959, 2024-05-28T14:10:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 13799251968, 2024-05-28T14:10:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 759294861312, 2024-05-28T14:10:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:10:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 2388, 2024-05-28T14:10:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:10:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:10:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:10:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 55, 2024-05-28T14:10:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 6, 2024-05-28T14:10:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:10:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:10:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:10:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:10:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 6, 2024-05-28T14:10:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:10:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:10:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:10:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 56621 2024-05-28T14:10:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:10:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:10:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:10:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:10:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:10:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:10:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:10:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:10:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:10:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:10:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 16, 2024-05-28T14:10:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:10:37.128981+0000", 2024-05-28T14:10:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:10:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:10:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bae8f9c0-5c5d-4728-8b27-e79ef1ba84dd": { 2024-05-28T14:10:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 10.0 seconds before exiting (27s)\n [============================] ", 2024-05-28T14:10:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:10:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:10:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:43.747 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.756+0000 7fe550c00640 1 -- 172.21.15.42:0/3703499401 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe52c069730 msgr2=0x7fe52c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.756+0000 7fe550c00640 1 --2- 172.21.15.42:0/3703499401 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe52c069730 0x7fe52c06bbf0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009be0 tx=0x7fe53c008040 comp rx=0 tx=0).stop 2024-05-28T14:10:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.756+0000 7fe550c00640 1 -- 172.21.15.42:0/3703499401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54c156540 msgr2=0x7fe54c06ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.756+0000 7fe550c00640 1 --2- 172.21.15.42:0/3703499401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54c156540 0x7fe54c06ff00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe538009ae0 tx=0x7fe530005840 comp rx=0 tx=0).stop 2024-05-28T14:10:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.756+0000 7fe550c00640 1 -- 172.21.15.42:0/3703499401 shutdown_connections 2024-05-28T14:10:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.756+0000 7fe550c00640 1 --2- 172.21.15.42:0/3703499401 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe52c069730 0x7fe52c06bbf0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.756+0000 7fe550c00640 1 --2- 172.21.15.42:0/3703499401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe54c149270 0x7fe54c157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.756+0000 7fe550c00640 1 --2- 172.21.15.42:0/3703499401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54c156540 0x7fe54c06ff00 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.756+0000 7fe550c00640 1 --2- 172.21.15.42:0/3703499401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe530004810 0x7fe54c06e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.756+0000 7fe550c00640 1 -- 172.21.15.42:0/3703499401 >> 172.21.15.42:0/3703499401 conn(0x7fe54c073280 msgr2=0x7fe54c0b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:43.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.756+0000 7fe550c00640 1 -- 172.21.15.42:0/3703499401 shutdown_connections 2024-05-28T14:10:43.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:43.756+0000 7fe550c00640 1 -- 172.21.15.42:0/3703499401 wait complete. 2024-05-28T14:10:48.764 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:10:48.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.932+0000 7f8608a00640 1 Processor -- start 2024-05-28T14:10:48.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.932+0000 7f8608a00640 1 -- start start 2024-05-28T14:10:48.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.932+0000 7f8608a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8604156c00 0x7f860406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:48.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.932+0000 7f8608a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f86040af6d0 0x7f860415dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:48.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.932+0000 7f8608a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f860406f6a0 con 0x7f860415d8d0 2024-05-28T14:10:48.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.932+0000 7f8608a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f860406e600 con 0x7f86040af6d0 2024-05-28T14:10:48.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.932+0000 7f8608a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f860406e8e0 con 0x7f8604156c00 2024-05-28T14:10:48.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.936+0000 7f8601e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8604156c00 0x7f860406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:48.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.936+0000 7f8602800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f86040af6d0 0x7f860415dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:48.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.936+0000 7f8601400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f860415d8d0 0x7f86040aefc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:58480/0 (socket says 172.21.15.42:58480) 2024-05-28T14:10:48.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.936+0000 7f8601400640 1 -- 172.21.15.42:0/1889896257 learned_addr learned my addr 172.21.15.42:0/1889896257 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:48.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.936+0000 7f8602800640 1 -- 172.21.15.42:0/1889896257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8604156c00 msgr2=0x7f860406dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:48.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.936+0000 7f8602800640 1 --2- 172.21.15.42:0/1889896257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8604156c00 0x7f860406dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:48.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.936+0000 7f8602800640 1 -- 172.21.15.42:0/1889896257 >> v1:172.21.15.63:6789/0 conn(0x7f860415d8d0 legacy=0x7f86040aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:48.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.936+0000 7f8602800640 1 -- 172.21.15.42:0/1889896257 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860406ebc0 con 0x7f86040af6d0 2024-05-28T14:10:48.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.936+0000 7f8601e00640 1 --2- 172.21.15.42:0/1889896257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8604156c00 0x7f860406dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:10:48.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.936+0000 7f8600a00640 1 -- 172.21.15.42:0/1889896257 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3279468788 0 0) 0x7f860406f6a0 con 0x7f860415d8d0 2024-05-28T14:10:48.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.936+0000 7f8602800640 1 --2- 172.21.15.42:0/1889896257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86040af6d0 0x7f860415dd70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009d90 tx=0x7f85fc01c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34d81508593a2776 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:48.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8600a00640 1 -- 172.21.15.42:0/1889896257 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc0b2070 con 0x7f86040af6d0 2024-05-28T14:10:48.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8600a00640 1 -- 172.21.15.42:0/1889896257 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85fc0a57a0 con 0x7f86040af6d0 2024-05-28T14:10:48.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8600a00640 1 -- 172.21.15.42:0/1889896257 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc0a5b70 con 0x7f86040af6d0 2024-05-28T14:10:48.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 -- 172.21.15.42:0/1889896257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86040af6d0 msgr2=0x7f860415dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:48.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 --2- 172.21.15.42:0/1889896257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86040af6d0 0x7f860415dd70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009d90 tx=0x7f85fc01c850 comp rx=0 tx=0).stop 2024-05-28T14:10:48.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 -- 172.21.15.42:0/1889896257 shutdown_connections 2024-05-28T14:10:48.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 --2- 172.21.15.42:0/1889896257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86040af6d0 0x7f860415dd70 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:48.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 --2- 172.21.15.42:0/1889896257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8604156c00 0x7f860406dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:48.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 -- 172.21.15.42:0/1889896257 >> 172.21.15.42:0/1889896257 conn(0x7f860406bff0 msgr2=0x7f86040b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:48.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 -- 172.21.15.42:0/1889896257 shutdown_connections 2024-05-28T14:10:48.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 -- 172.21.15.42:0/1889896257 wait complete. 2024-05-28T14:10:48.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 Processor -- start 2024-05-28T14:10:48.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 -- start start 2024-05-28T14:10:48.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86040af6d0 0x7f860414c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:48.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8604156c00 0x7f8604155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:48.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f860415d8d0 0x7f86041558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:48.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f860415ec40 con 0x7f8604156c00 2024-05-28T14:10:48.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f860415eac0 con 0x7f860415d8d0 2024-05-28T14:10:48.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8608a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f860415edc0 con 0x7f86040af6d0 2024-05-28T14:10:48.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.940+0000 7f8601400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8604156c00 0x7f8604155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:48.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f8601400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8604156c00 0x7f8604155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49490/0 (socket says 172.21.15.42:49490) 2024-05-28T14:10:48.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f8601400640 1 -- 172.21.15.42:0/4072563303 learned_addr learned my addr 172.21.15.42:0/4072563303 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:48.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f8601e00640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86040af6d0 0x7f860414c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:48.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f8602800640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f860415d8d0 0x7f86041558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:48.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f8601400640 1 -- 172.21.15.42:0/4072563303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86040af6d0 msgr2=0x7f860414c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:48.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f8601400640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86040af6d0 0x7f860414c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:48.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f8601400640 1 -- 172.21.15.42:0/4072563303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f860415d8d0 msgr2=0x7f86041558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:48.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f8601400640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f860415d8d0 0x7f86041558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:48.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f8601400640 1 -- 172.21.15.42:0/4072563303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860414c680 con 0x7f8604156c00 2024-05-28T14:10:48.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f8602800640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f860415d8d0 0x7f86041558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:10:48.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f8601400640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8604156c00 0x7f8604155370 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f85f000c480 tx=0x7f85f000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:48.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f85fa000640 1 -- 172.21.15.42:0/4072563303 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0018070 con 0x7f8604156c00 2024-05-28T14:10:48.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f85fa000640 1 -- 172.21.15.42:0/4072563303 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f0007500 con 0x7f8604156c00 2024-05-28T14:10:48.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f85fa000640 1 -- 172.21.15.42:0/4072563303 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0004a30 con 0x7f8604156c00 2024-05-28T14:10:48.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f8608a00640 1 -- 172.21.15.42:0/4072563303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86040b4620 con 0x7f8604156c00 2024-05-28T14:10:48.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f8608a00640 1 -- 172.21.15.42:0/4072563303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86040b4be0 con 0x7f8604156c00 2024-05-28T14:10:48.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f8608a00640 1 -- 172.21.15.42:0/4072563303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8604068a90 con 0x7f8604156c00 2024-05-28T14:10:48.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f85fa000640 1 -- 172.21.15.42:0/4072563303 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7f85f0004bd0 con 0x7f8604156c00 2024-05-28T14:10:48.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.944+0000 7f85fa000640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85e4069730 0x7f85e406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:48.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.948+0000 7f85fa000640 1 -- 172.21.15.42:0/4072563303 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7f85f0084bb0 con 0x7f8604156c00 2024-05-28T14:10:48.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.948+0000 7f8601e00640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85e4069730 0x7f85e406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:48.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.948+0000 7f8601e00640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85e4069730 0x7f85e406bbf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f86040b4410 tx=0x7f85f4009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:48.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:48.952+0000 7f85fa000640 1 -- 172.21.15.42:0/4072563303 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f85f00561c0 con 0x7f8604156c00 2024-05-28T14:10:49.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:49.176+0000 7f8608a00640 1 -- 172.21.15.42:0/4072563303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f86040b2a40 con 0x7f8604156c00 2024-05-28T14:10:49.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:49.176+0000 7f85fa000640 1 -- 172.21.15.42:0/4072563303 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1985 (secure 0 0 0) 0x7f85f005a4c0 con 0x7f8604156c00 2024-05-28T14:10:49.160 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:49.160 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:49.160 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:10:49.161 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:10:49.161 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:10:49.161 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:10:49.161 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:10:49.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:49.161 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:10:49.161 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:10:49.161 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:10:49.161 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:10:49.161 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:10:49.161 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:49.161 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:10:49.161 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:10:49.161 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:10:49.162 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:10:49.162 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:49.162 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 770, 2024-05-28T14:10:49.162 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:10:49.162 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:10:49.162 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:10:49.162 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:10:49.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:49.162 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:10:49.162 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 92, 2024-05-28T14:10:49.162 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:10:49.162 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:10:49.162 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1565, 2024-05-28T14:10:49.163 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 6443511959, 2024-05-28T14:10:49.164 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 13799251968, 2024-05-28T14:10:49.164 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 759294861312, 2024-05-28T14:10:49.164 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280 2024-05-28T14:10:49.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:49.164 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:10:49.164 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 55, 2024-05-28T14:10:49.164 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 6, 2024-05-28T14:10:49.164 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:10:49.164 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:10:49.164 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:10:49.164 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:10:49.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:49.164 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 6, 2024-05-28T14:10:49.164 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:10:49.165 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:10:49.165 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:10:49.165 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 56621 2024-05-28T14:10:49.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:49.165 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:49.165 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:10:49.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:49.165 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:10:49.165 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:10:49.165 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:10:49.165 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:10:49.165 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:10:49.165 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:10:49.166 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:10:49.166 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:49.166 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:10:49.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:49.166 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:10:49.166 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 16, 2024-05-28T14:10:49.166 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:10:37.128981+0000", 2024-05-28T14:10:49.166 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:10:49.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:49.166 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:10:49.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bae8f9c0-5c5d-4728-8b27-e79ef1ba84dd": { 2024-05-28T14:10:49.166 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 4.0 seconds before exiting (33s)\n [============================] ", 2024-05-28T14:10:49.166 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:10:49.167 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:10:49.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:49.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:49.167 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:49.176+0000 7f8608a00640 1 -- 172.21.15.42:0/4072563303 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85e4069730 msgr2=0x7f85e406bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:49.176+0000 7f8608a00640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85e4069730 0x7f85e406bbf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f86040b4410 tx=0x7f85f4009190 comp rx=0 tx=0).stop 2024-05-28T14:10:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:49.176+0000 7f8608a00640 1 -- 172.21.15.42:0/4072563303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8604156c00 msgr2=0x7f8604155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:49.176+0000 7f8608a00640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8604156c00 0x7f8604155370 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f85f000c480 tx=0x7f85f000c940 comp rx=0 tx=0).stop 2024-05-28T14:10:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:49.176+0000 7f8608a00640 1 -- 172.21.15.42:0/4072563303 shutdown_connections 2024-05-28T14:10:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:49.176+0000 7f8608a00640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85e4069730 0x7f85e406bbf0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:49.176+0000 7f8608a00640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f860415d8d0 0x7f86041558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:49.176+0000 7f8608a00640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8604156c00 0x7f8604155370 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:49.176+0000 7f8608a00640 1 --2- 172.21.15.42:0/4072563303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86040af6d0 0x7f860414c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:49.176+0000 7f8608a00640 1 -- 172.21.15.42:0/4072563303 >> 172.21.15.42:0/4072563303 conn(0x7f860406bff0 msgr2=0x7f860415f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:49.176+0000 7f8608a00640 1 -- 172.21.15.42:0/4072563303 shutdown_connections 2024-05-28T14:10:49.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:49.180+0000 7f8608a00640 1 -- 172.21.15.42:0/4072563303 wait complete. 2024-05-28T14:10:50.529 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:10:50.533 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:10:54.184 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:10:54.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.352+0000 7f1c9aa00640 1 Processor -- start 2024-05-28T14:10:54.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.352+0000 7f1c9aa00640 1 -- start start 2024-05-28T14:10:54.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.352+0000 7f1c9aa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1c940afcf0 0x7f1c94160b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:54.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.352+0000 7f1c9aa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1c940cddb0 0x7f1c94152db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:54.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.352+0000 7f1c9aa00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c9415da40 con 0x7f1c9415d590 2024-05-28T14:10:54.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.352+0000 7f1c9aa00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1c94160130 con 0x7f1c940afcf0 2024-05-28T14:10:54.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.352+0000 7f1c9aa00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1c940cd0e0 con 0x7f1c940cddb0 2024-05-28T14:10:54.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.356+0000 7f1c93e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1c940afcf0 0x7f1c94160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:54.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.356+0000 7f1c93e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1c940afcf0 0x7f1c94160b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49494/0 (socket says 172.21.15.42:49494) 2024-05-28T14:10:54.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.356+0000 7f1c8be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1c9415d590 0x7f1c940cd6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56706/0 (socket says 172.21.15.42:56706) 2024-05-28T14:10:54.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.356+0000 7f1c93e00640 1 -- 172.21.15.42:0/1770368755 learned_addr learned my addr 172.21.15.42:0/1770368755 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:54.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.356+0000 7f1c98a00640 1 --2- 172.21.15.42:0/1770368755 >> v2:172.21.15.63:3300/0 conn(0x7f1c940cddb0 0x7f1c94152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:54.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.356+0000 7f1c93e00640 1 -- 172.21.15.42:0/1770368755 >> v2:172.21.15.63:3300/0 conn(0x7f1c940cddb0 msgr2=0x7f1c94152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:54.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.356+0000 7f1c93e00640 1 --2- 172.21.15.42:0/1770368755 >> v2:172.21.15.63:3300/0 conn(0x7f1c940cddb0 0x7f1c94152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.356+0000 7f1c93e00640 1 -- 172.21.15.42:0/1770368755 >> v1:172.21.15.42:6789/0 conn(0x7f1c9415d590 legacy=0x7f1c940cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:54.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.356+0000 7f1c93e00640 1 -- 172.21.15.42:0/1770368755 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c940cd3c0 con 0x7f1c940afcf0 2024-05-28T14:10:54.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.356+0000 7f1c93400640 1 -- 172.21.15.42:0/1770368755 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2307324522 0 0) 0x7f1c9415da40 con 0x7f1c9415d590 2024-05-28T14:10:54.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.356+0000 7f1c93e00640 1 --2- 172.21.15.42:0/1770368755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c940afcf0 0x7f1c94160b80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f1c940b41d0 tx=0x7f1c8401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcafe36d0c465a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:54.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.356+0000 7f1c93400640 1 -- 172.21.15.42:0/1770368755 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c840aa650 con 0x7f1c940afcf0 2024-05-28T14:10:54.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.356+0000 7f1c93400640 1 -- 172.21.15.42:0/1770368755 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c840af070 con 0x7f1c940afcf0 2024-05-28T14:10:54.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.356+0000 7f1c93400640 1 -- 172.21.15.42:0/1770368755 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c840aaa10 con 0x7f1c940afcf0 2024-05-28T14:10:54.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 -- 172.21.15.42:0/1770368755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c940afcf0 msgr2=0x7f1c94160b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:54.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 --2- 172.21.15.42:0/1770368755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c940afcf0 0x7f1c94160b80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f1c940b41d0 tx=0x7f1c8401c4f0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 -- 172.21.15.42:0/1770368755 shutdown_connections 2024-05-28T14:10:54.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 --2- 172.21.15.42:0/1770368755 >> v2:172.21.15.63:3300/0 conn(0x7f1c940cddb0 0x7f1c94152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 --2- 172.21.15.42:0/1770368755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c940afcf0 0x7f1c94160b80 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 -- 172.21.15.42:0/1770368755 >> 172.21.15.42:0/1770368755 conn(0x7f1c940d0b70 msgr2=0x7f1c940d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:54.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 -- 172.21.15.42:0/1770368755 shutdown_connections 2024-05-28T14:10:54.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 -- 172.21.15.42:0/1770368755 wait complete. 2024-05-28T14:10:54.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 Processor -- start 2024-05-28T14:10:54.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 -- start start 2024-05-28T14:10:54.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c940afcf0 0x7f1c9414b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:54.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c940cddb0 0x7f1c94166c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:54.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c9415d590 0x7f1c9415f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:54.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1c940b5e30 con 0x7f1c9415d590 2024-05-28T14:10:54.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c940b5cb0 con 0x7f1c940afcf0 2024-05-28T14:10:54.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c9aa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c940b00d0 con 0x7f1c940cddb0 2024-05-28T14:10:54.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c98a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c9415d590 0x7f1c9415f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:54.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c98a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c9415d590 0x7f1c9415f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49508/0 (socket says 172.21.15.42:49508) 2024-05-28T14:10:54.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c98a00640 1 -- 172.21.15.42:0/2034354929 learned_addr learned my addr 172.21.15.42:0/2034354929 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:54.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c93e00640 1 --2- 172.21.15.42:0/2034354929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c940afcf0 0x7f1c9414b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:54.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c8be00640 1 --2- 172.21.15.42:0/2034354929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c940cddb0 0x7f1c94166c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:54.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c98a00640 1 -- 172.21.15.42:0/2034354929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c940cddb0 msgr2=0x7f1c94166c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:54.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c98a00640 1 --2- 172.21.15.42:0/2034354929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c940cddb0 0x7f1c94166c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c98a00640 1 -- 172.21.15.42:0/2034354929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c940afcf0 msgr2=0x7f1c9414b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:54.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c98a00640 1 --2- 172.21.15.42:0/2034354929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c940afcf0 0x7f1c9414b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.360+0000 7f1c98a00640 1 -- 172.21.15.42:0/2034354929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9414c310 con 0x7f1c9415d590 2024-05-28T14:10:54.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.364+0000 7f1c98a00640 1 --2- 172.21.15.42:0/2034354929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c9415d590 0x7f1c9415f0d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00efc0 tx=0x7f1c8c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:54.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.364+0000 7f1c90c00640 1 -- 172.21.15.42:0/2034354929 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c018020 con 0x7f1c9415d590 2024-05-28T14:10:54.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.364+0000 7f1c90c00640 1 -- 172.21.15.42:0/2034354929 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c8c014480 con 0x7f1c9415d590 2024-05-28T14:10:54.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.364+0000 7f1c90c00640 1 -- 172.21.15.42:0/2034354929 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c014870 con 0x7f1c9415d590 2024-05-28T14:10:54.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.364+0000 7f1c9aa00640 1 -- 172.21.15.42:0/2034354929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9414c5f0 con 0x7f1c9415d590 2024-05-28T14:10:54.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.364+0000 7f1c9aa00640 1 -- 172.21.15.42:0/2034354929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9414cbb0 con 0x7f1c9415d590 2024-05-28T14:10:54.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.364+0000 7f1c9aa00640 1 -- 172.21.15.42:0/2034354929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c94068a90 con 0x7f1c9415d590 2024-05-28T14:10:54.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.364+0000 7f1c90c00640 1 -- 172.21.15.42:0/2034354929 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7f1c8c01b020 con 0x7f1c9415d590 2024-05-28T14:10:54.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.364+0000 7f1c90c00640 1 --2- 172.21.15.42:0/2034354929 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c70069730 0x7f1c7006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:54.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.368+0000 7f1c90c00640 1 -- 172.21.15.42:0/2034354929 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7f1c8c0863d0 con 0x7f1c9415d590 2024-05-28T14:10:54.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.368+0000 7f1c93e00640 1 --2- 172.21.15.42:0/2034354929 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c70069730 0x7f1c7006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:54.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.368+0000 7f1c93e00640 1 --2- 172.21.15.42:0/2034354929 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c70069730 0x7f1c7006bbf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1c840ae000 tx=0x7f1c840a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:54.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.372+0000 7f1c90c00640 1 -- 172.21.15.42:0/2034354929 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1c8c022070 con 0x7f1c9415d590 2024-05-28T14:10:54.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.592+0000 7f1c9aa00640 1 -- 172.21.15.42:0/2034354929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1c94167ee0 con 0x7f1c9415d590 2024-05-28T14:10:54.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.592+0000 7f1c90c00640 1 -- 172.21.15.42:0/2034354929 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1731 (secure 0 0 0) 0x7f1c8c010040 con 0x7f1c9415d590 2024-05-28T14:10:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:54.579 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:10:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:10:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:10:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:10:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:10:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:10:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:10:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:10:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:10:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:10:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:10:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:10:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:10:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:10:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:10:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 775, 2024-05-28T14:10:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:10:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:10:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:10:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:10:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:10:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 92, 2024-05-28T14:10:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:10:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:10:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:10:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:10:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:10:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:10:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:10:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:10:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:10:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:10:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:10:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:10:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1565, 2024-05-28T14:10:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 6443511959, 2024-05-28T14:10:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 13799251968, 2024-05-28T14:10:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 759294861312, 2024-05-28T14:10:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280 2024-05-28T14:10:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:10:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 55, 2024-05-28T14:10:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 6, 2024-05-28T14:10:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:10:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:10:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:10:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:10:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 6, 2024-05-28T14:10:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:10:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:10:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:10:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 56621 2024-05-28T14:10:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:10:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:10:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:10:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:10:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:10:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:10:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:10:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:10:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:10:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:10:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:10:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 16, 2024-05-28T14:10:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:10:37.128981+0000", 2024-05-28T14:10:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:10:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": {} 2024-05-28T14:10:54.585 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:10:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.596+0000 7f1c9aa00640 1 -- 172.21.15.42:0/2034354929 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c70069730 msgr2=0x7f1c7006bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.596+0000 7f1c9aa00640 1 --2- 172.21.15.42:0/2034354929 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c70069730 0x7f1c7006bbf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1c840ae000 tx=0x7f1c840a7000 comp rx=0 tx=0).stop 2024-05-28T14:10:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.596+0000 7f1c9aa00640 1 -- 172.21.15.42:0/2034354929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c9415d590 msgr2=0x7f1c9415f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.596+0000 7f1c9aa00640 1 --2- 172.21.15.42:0/2034354929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c9415d590 0x7f1c9415f0d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00efc0 tx=0x7f1c8c00c490 comp rx=0 tx=0).stop 2024-05-28T14:10:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.596+0000 7f1c9aa00640 1 -- 172.21.15.42:0/2034354929 shutdown_connections 2024-05-28T14:10:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.596+0000 7f1c9aa00640 1 --2- 172.21.15.42:0/2034354929 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c70069730 0x7f1c7006bbf0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.596+0000 7f1c9aa00640 1 --2- 172.21.15.42:0/2034354929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c9415d590 0x7f1c9415f0d0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.596+0000 7f1c9aa00640 1 --2- 172.21.15.42:0/2034354929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c940cddb0 0x7f1c94166c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.596+0000 7f1c9aa00640 1 --2- 172.21.15.42:0/2034354929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c940afcf0 0x7f1c9414b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.596+0000 7f1c9aa00640 1 -- 172.21.15.42:0/2034354929 >> 172.21.15.42:0/2034354929 conn(0x7f1c940d0b70 msgr2=0x7f1c940ad950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.596+0000 7f1c9aa00640 1 -- 172.21.15.42:0/2034354929 shutdown_connections 2024-05-28T14:10:54.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.596+0000 7f1c9aa00640 1 -- 172.21.15.42:0/2034354929 wait complete. 2024-05-28T14:10:54.600 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-28T14:10:54.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb94ea00640 1 Processor -- start 2024-05-28T14:10:54.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb94ea00640 1 -- start start 2024-05-28T14:10:54.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb94ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb94806ee20 0x7fb94806f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:54.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb94ea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb94815b9b0 0x7fb94815be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:54.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb94ea00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9480afe40 con 0x7fb948144f80 2024-05-28T14:10:54.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb94ea00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb94806f7c0 con 0x7fb94806ee20 2024-05-28T14:10:54.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb94ea00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb9480724d0 con 0x7fb94815b9b0 2024-05-28T14:10:54.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb947400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb94806ee20 0x7fb94806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:54.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb947400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb94806ee20 0x7fb94806f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49516/0 (socket says 172.21.15.42:49516) 2024-05-28T14:10:54.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb947e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb948144f80 0x7fb948072960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56718/0 (socket says 172.21.15.42:56718) 2024-05-28T14:10:54.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb947400640 1 -- 172.21.15.42:0/3825789601 learned_addr learned my addr 172.21.15.42:0/3825789601 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:54.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb94ca00640 1 --2- 172.21.15.42:0/3825789601 >> v2:172.21.15.63:3300/0 conn(0x7fb94815b9b0 0x7fb94815be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:54.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb947400640 1 -- 172.21.15.42:0/3825789601 >> v2:172.21.15.63:3300/0 conn(0x7fb94815b9b0 msgr2=0x7fb94815be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:54.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb947400640 1 --2- 172.21.15.42:0/3825789601 >> v2:172.21.15.63:3300/0 conn(0x7fb94815b9b0 0x7fb94815be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb947400640 1 -- 172.21.15.42:0/3825789601 >> v1:172.21.15.42:6789/0 conn(0x7fb948144f80 legacy=0x7fb948072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:54.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb947400640 1 -- 172.21.15.42:0/3825789601 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94815c340 con 0x7fb94806ee20 2024-05-28T14:10:54.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.760+0000 7fb946a00640 1 -- 172.21.15.42:0/3825789601 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342171422 0 0) 0x7fb9480afe40 con 0x7fb948144f80 2024-05-28T14:10:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.764+0000 7fb947400640 1 --2- 172.21.15.42:0/3825789601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb94806ee20 0x7fb94806f280 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb948160060 tx=0x7fb93401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436ef8ddc76e944b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.764+0000 7fb946a00640 1 -- 172.21.15.42:0/3825789601 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9340b0070 con 0x7fb94806ee20 2024-05-28T14:10:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.764+0000 7fb946a00640 1 -- 172.21.15.42:0/3825789601 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9340ab600 con 0x7fb94806ee20 2024-05-28T14:10:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.764+0000 7fb946a00640 1 -- 172.21.15.42:0/3825789601 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9340ab9d0 con 0x7fb94806ee20 2024-05-28T14:10:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.764+0000 7fb94ea00640 1 -- 172.21.15.42:0/3825789601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb94806ee20 msgr2=0x7fb94806f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.764+0000 7fb94ea00640 1 --2- 172.21.15.42:0/3825789601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb94806ee20 0x7fb94806f280 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb948160060 tx=0x7fb93401c520 comp rx=0 tx=0).stop 2024-05-28T14:10:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.764+0000 7fb94ea00640 1 -- 172.21.15.42:0/3825789601 shutdown_connections 2024-05-28T14:10:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.764+0000 7fb94ea00640 1 --2- 172.21.15.42:0/3825789601 >> v2:172.21.15.63:3300/0 conn(0x7fb94815b9b0 0x7fb94815be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.764+0000 7fb94ea00640 1 --2- 172.21.15.42:0/3825789601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb94806ee20 0x7fb94806f280 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.764+0000 7fb94ea00640 1 -- 172.21.15.42:0/3825789601 >> 172.21.15.42:0/3825789601 conn(0x7fb9480b21f0 msgr2=0x7fb9480b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:54.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.764+0000 7fb94ea00640 1 -- 172.21.15.42:0/3825789601 shutdown_connections 2024-05-28T14:10:54.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.764+0000 7fb94ea00640 1 -- 172.21.15.42:0/3825789601 wait complete. 2024-05-28T14:10:54.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.764+0000 7fb94ea00640 1 Processor -- start 2024-05-28T14:10:54.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.764+0000 7fb94ea00640 1 -- start start 2024-05-28T14:10:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb94806ee20 0x7fb94813e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb948144f80 0x7fb94813b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb94815b9b0 0x7fb94813ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9480b4010 con 0x7fb94815b9b0 2024-05-28T14:10:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb948072160 con 0x7fb948144f80 2024-05-28T14:10:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ea00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb9480b4190 con 0x7fb94806ee20 2024-05-28T14:10:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb94815b9b0 0x7fb94813ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb94815b9b0 0x7fb94813ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49528/0 (socket says 172.21.15.42:49528) 2024-05-28T14:10:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ca00640 1 -- 172.21.15.42:0/148541529 learned_addr learned my addr 172.21.15.42:0/148541529 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb947400640 1 --2- 172.21.15.42:0/148541529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb948144f80 0x7fb94813b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb947e00640 1 --2- 172.21.15.42:0/148541529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb94806ee20 0x7fb94813e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ca00640 1 -- 172.21.15.42:0/148541529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb94806ee20 msgr2=0x7fb94813e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ca00640 1 --2- 172.21.15.42:0/148541529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb94806ee20 0x7fb94813e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ca00640 1 -- 172.21.15.42:0/148541529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb948144f80 msgr2=0x7fb94813b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ca00640 1 --2- 172.21.15.42:0/148541529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb948144f80 0x7fb94813b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ca00640 1 -- 172.21.15.42:0/148541529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb948159e60 con 0x7fb94815b9b0 2024-05-28T14:10:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ca00640 1 --2- 172.21.15.42:0/148541529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb94815b9b0 0x7fb94813ca50 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb940009d30 tx=0x7fb94000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:54.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb93fe00640 1 -- 172.21.15.42:0/148541529 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940018020 con 0x7fb94815b9b0 2024-05-28T14:10:54.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb93fe00640 1 -- 172.21.15.42:0/148541529 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb940014480 con 0x7fb94815b9b0 2024-05-28T14:10:54.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb93fe00640 1 -- 172.21.15.42:0/148541529 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940014870 con 0x7fb94815b9b0 2024-05-28T14:10:54.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ea00640 1 -- 172.21.15.42:0/148541529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94815a140 con 0x7fb94815b9b0 2024-05-28T14:10:54.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ea00640 1 -- 172.21.15.42:0/148541529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94815a700 con 0x7fb94815b9b0 2024-05-28T14:10:54.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb94ea00640 1 -- 172.21.15.42:0/148541529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9480738b0 con 0x7fb94815b9b0 2024-05-28T14:10:54.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.768+0000 7fb93fe00640 1 -- 172.21.15.42:0/148541529 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7fb94001b020 con 0x7fb94815b9b0 2024-05-28T14:10:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.772+0000 7fb93fe00640 1 --2- 172.21.15.42:0/148541529 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb928069730 0x7fb92806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.772+0000 7fb93fe00640 1 -- 172.21.15.42:0/148541529 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7fb940086480 con 0x7fb94815b9b0 2024-05-28T14:10:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.772+0000 7fb947e00640 1 --2- 172.21.15.42:0/148541529 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb928069730 0x7fb92806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.772+0000 7fb947e00640 1 --2- 172.21.15.42:0/148541529 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb928069730 0x7fb92806bbf0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb938000f70 tx=0x7fb938007b60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:54.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.776+0000 7fb93fe00640 1 -- 172.21.15.42:0/148541529 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb940022070 con 0x7fb94815b9b0 2024-05-28T14:10:54.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.960+0000 7fb94ea00640 1 -- 172.21.15.42:0/148541529 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb94815b1e0 con 0x7fb928069730 2024-05-28T14:10:54.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.968+0000 7fb93fe00640 1 -- 172.21.15.42:0/148541529 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7fb9380071f0 con 0x7fb928069730 2024-05-28T14:10:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:54.952 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:10:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sv1" 2024-05-28T14:10:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:10:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ss1clone1" 2024-05-28T14:10:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:54.952 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:10:54.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.968+0000 7fb94ea00640 1 -- 172.21.15.42:0/148541529 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb928069730 msgr2=0x7fb92806bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:54.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.968+0000 7fb94ea00640 1 --2- 172.21.15.42:0/148541529 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb928069730 0x7fb92806bbf0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb938000f70 tx=0x7fb938007b60 comp rx=0 tx=0).stop 2024-05-28T14:10:54.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.968+0000 7fb94ea00640 1 -- 172.21.15.42:0/148541529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb94815b9b0 msgr2=0x7fb94813ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:54.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.968+0000 7fb94ea00640 1 --2- 172.21.15.42:0/148541529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb94815b9b0 0x7fb94813ca50 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb940009d30 tx=0x7fb94000c490 comp rx=0 tx=0).stop 2024-05-28T14:10:54.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.968+0000 7fb94ea00640 1 -- 172.21.15.42:0/148541529 shutdown_connections 2024-05-28T14:10:54.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.968+0000 7fb94ea00640 1 --2- 172.21.15.42:0/148541529 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb928069730 0x7fb92806bbf0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.968+0000 7fb94ea00640 1 --2- 172.21.15.42:0/148541529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb94815b9b0 0x7fb94813ca50 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.968+0000 7fb94ea00640 1 --2- 172.21.15.42:0/148541529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb948144f80 0x7fb94813b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.968+0000 7fb94ea00640 1 --2- 172.21.15.42:0/148541529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb94806ee20 0x7fb94813e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:54.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.968+0000 7fb94ea00640 1 -- 172.21.15.42:0/148541529 >> 172.21.15.42:0/148541529 conn(0x7fb9480b21f0 msgr2=0x7fb9480cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:54.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.968+0000 7fb94ea00640 1 -- 172.21.15.42:0/148541529 shutdown_connections 2024-05-28T14:10:54.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:54.968+0000 7fb94ea00640 1 -- 172.21.15.42:0/148541529 wait complete. 2024-05-28T14:10:54.973 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-28T14:10:55.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.136+0000 7f67b6800640 1 Processor -- start 2024-05-28T14:10:55.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.136+0000 7f67b6800640 1 -- start start 2024-05-28T14:10:55.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.136+0000 7f67b6800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f67b006ee80 0x7f67b006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.136+0000 7f67b6800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67b01570a0 con 0x7f67b00af360 2024-05-28T14:10:55.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.136+0000 7f67b6800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67b015cf00 con 0x7f67b00d0e00 2024-05-28T14:10:55.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.136+0000 7f67b6800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f67b015d1e0 con 0x7f67b006ee80 2024-05-28T14:10:55.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.136+0000 7f67af400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f67b006ee80 0x7f67b006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.136+0000 7f67a6a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f67b00af360 0x7f67b00d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56732/0 (socket says 172.21.15.42:56732) 2024-05-28T14:10:55.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.136+0000 7f67a6a00640 1 -- 172.21.15.42:0/3715598390 learned_addr learned my addr 172.21.15.42:0/3715598390 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:55.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.136+0000 7f67af400640 1 -- 172.21.15.42:0/3715598390 >> v1:172.21.15.63:6789/0 conn(0x7f67b00d0e00 legacy=0x7f67b0160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.136+0000 7f67af400640 1 -- 172.21.15.42:0/3715598390 >> v1:172.21.15.42:6789/0 conn(0x7f67b00af360 legacy=0x7f67b00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.136+0000 7f67af400640 1 -- 172.21.15.42:0/3715598390 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b015d4c0 con 0x7f67b006ee80 2024-05-28T14:10:55.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.136+0000 7f67aea00640 1 -- 172.21.15.42:0/3715598390 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2624082554 0 0) 0x7f67b01570a0 con 0x7f67b00af360 2024-05-28T14:10:55.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.140+0000 7f67af400640 1 --2- 172.21.15.42:0/3715598390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b006ee80 0x7f67b006dd40 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009d90 tx=0x7f67a001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2379261f3fc40eb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:55.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.140+0000 7f67aea00640 1 -- 172.21.15.42:0/3715598390 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a00b2070 con 0x7f67b006ee80 2024-05-28T14:10:55.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.140+0000 7f67aea00640 1 -- 172.21.15.42:0/3715598390 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a00a5930 con 0x7f67b006ee80 2024-05-28T14:10:55.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.140+0000 7f67aea00640 1 -- 172.21.15.42:0/3715598390 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a00a5d00 con 0x7f67b006ee80 2024-05-28T14:10:55.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.140+0000 7f67b6800640 1 -- 172.21.15.42:0/3715598390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b006ee80 msgr2=0x7f67b006dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.140+0000 7f67b6800640 1 --2- 172.21.15.42:0/3715598390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b006ee80 0x7f67b006dd40 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009d90 tx=0x7f67a001c8b0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.140+0000 7f67b6800640 1 -- 172.21.15.42:0/3715598390 shutdown_connections 2024-05-28T14:10:55.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.140+0000 7f67b6800640 1 --2- 172.21.15.42:0/3715598390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b006ee80 0x7f67b006dd40 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.140+0000 7f67b6800640 1 -- 172.21.15.42:0/3715598390 >> 172.21.15.42:0/3715598390 conn(0x7f67b006bdf0 msgr2=0x7f67b006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:55.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.140+0000 7f67b6800640 1 -- 172.21.15.42:0/3715598390 shutdown_connections 2024-05-28T14:10:55.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.140+0000 7f67b6800640 1 -- 172.21.15.42:0/3715598390 wait complete. 2024-05-28T14:10:55.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67b6800640 1 Processor -- start 2024-05-28T14:10:55.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67b6800640 1 -- start start 2024-05-28T14:10:55.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67b6800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67b006ee80 0x7f67b0166090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67b6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67b00af360 0x7f67b0151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67b6800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b00d0e00 0x7f67b00cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67b6800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f67b015cce0 con 0x7f67b00d0e00 2024-05-28T14:10:55.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67b6800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f67b015cb60 con 0x7f67b00af360 2024-05-28T14:10:55.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67b6800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f67b0160fa0 con 0x7f67b006ee80 2024-05-28T14:10:55.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b00d0e00 0x7f67b00cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b00d0e00 0x7f67b00cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49546/0 (socket says 172.21.15.42:49546) 2024-05-28T14:10:55.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67afe00640 1 -- 172.21.15.42:0/2913020261 learned_addr learned my addr 172.21.15.42:0/2913020261 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:55.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67a6a00640 1 --2- 172.21.15.42:0/2913020261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67b00af360 0x7f67b0151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67af400640 1 --2- 172.21.15.42:0/2913020261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67b006ee80 0x7f67b0166090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67afe00640 1 -- 172.21.15.42:0/2913020261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67b006ee80 msgr2=0x7f67b0166090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67afe00640 1 --2- 172.21.15.42:0/2913020261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67b006ee80 0x7f67b0166090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67afe00640 1 -- 172.21.15.42:0/2913020261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67b00af360 msgr2=0x7f67b0151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67afe00640 1 --2- 172.21.15.42:0/2913020261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67b00af360 0x7f67b0151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67afe00640 1 -- 172.21.15.42:0/2913020261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b0167bd0 con 0x7f67b00d0e00 2024-05-28T14:10:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67afe00640 1 --2- 172.21.15.42:0/2913020261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b00d0e00 0x7f67b00cf080 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f67a800c340 tx=0x7f67a800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67a7e00640 1 -- 172.21.15.42:0/2913020261 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8005220 con 0x7f67b00d0e00 2024-05-28T14:10:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67a7e00640 1 -- 172.21.15.42:0/2913020261 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a80053c0 con 0x7f67b00d0e00 2024-05-28T14:10:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67a7e00640 1 -- 172.21.15.42:0/2913020261 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8007270 con 0x7f67b00d0e00 2024-05-28T14:10:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67b6800640 1 -- 172.21.15.42:0/2913020261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b00ce5a0 con 0x7f67b00d0e00 2024-05-28T14:10:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.144+0000 7f67b6800640 1 -- 172.21.15.42:0/2913020261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b01677f0 con 0x7f67b00d0e00 2024-05-28T14:10:55.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.148+0000 7f67b6800640 1 -- 172.21.15.42:0/2913020261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b0068a90 con 0x7f67b00d0e00 2024-05-28T14:10:55.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.148+0000 7f67a7e00640 1 -- 172.21.15.42:0/2913020261 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7f67a8019020 con 0x7f67b00d0e00 2024-05-28T14:10:55.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.148+0000 7f67a7e00640 1 --2- 172.21.15.42:0/2913020261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f678c069730 0x7f678c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.148+0000 7f67a7e00640 1 -- 172.21.15.42:0/2913020261 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7f67a80847f0 con 0x7f67b00d0e00 2024-05-28T14:10:55.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.148+0000 7f67af400640 1 --2- 172.21.15.42:0/2913020261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f678c069730 0x7f678c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.148+0000 7f67af400640 1 --2- 172.21.15.42:0/2913020261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f678c069730 0x7f678c06bbf0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f67a00af000 tx=0x7f67a00a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:55.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.152+0000 7f67a7e00640 1 -- 172.21.15.42:0/2913020261 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f67a80561b0 con 0x7f67b00d0e00 2024-05-28T14:10:55.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.348+0000 7f67b6800640 1 -- 172.21.15.42:0/2913020261 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67b00b19b0 con 0x7f678c069730 2024-05-28T14:10:55.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.360+0000 7f67a7e00640 1 -- 172.21.15.42:0/2913020261 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f67b00b19b0 con 0x7f678c069730 2024-05-28T14:10:55.346 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:55.346 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:10:55.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:10:55.346 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ss1" 2024-05-28T14:10:55.346 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:10:55.346 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:10:55.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.364+0000 7f67b6800640 1 -- 172.21.15.42:0/2913020261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f678c069730 msgr2=0x7f678c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.364+0000 7f67b6800640 1 --2- 172.21.15.42:0/2913020261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f678c069730 0x7f678c06bbf0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f67a00af000 tx=0x7f67a00a7070 comp rx=0 tx=0).stop 2024-05-28T14:10:55.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.364+0000 7f67b6800640 1 -- 172.21.15.42:0/2913020261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b00d0e00 msgr2=0x7f67b00cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.364+0000 7f67b6800640 1 --2- 172.21.15.42:0/2913020261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b00d0e00 0x7f67b00cf080 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f67a800c340 tx=0x7f67a800c800 comp rx=0 tx=0).stop 2024-05-28T14:10:55.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.364+0000 7f67b6800640 1 -- 172.21.15.42:0/2913020261 shutdown_connections 2024-05-28T14:10:55.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.364+0000 7f67b6800640 1 --2- 172.21.15.42:0/2913020261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f678c069730 0x7f678c06bbf0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.364+0000 7f67b6800640 1 --2- 172.21.15.42:0/2913020261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b00d0e00 0x7f67b00cf080 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.364+0000 7f67b6800640 1 --2- 172.21.15.42:0/2913020261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67b00af360 0x7f67b0151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.364+0000 7f67b6800640 1 --2- 172.21.15.42:0/2913020261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67b006ee80 0x7f67b0166090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.364+0000 7f67b6800640 1 -- 172.21.15.42:0/2913020261 >> 172.21.15.42:0/2913020261 conn(0x7f67b006bdf0 msgr2=0x7f67b00d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:55.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.364+0000 7f67b6800640 1 -- 172.21.15.42:0/2913020261 shutdown_connections 2024-05-28T14:10:55.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.364+0000 7f67b6800640 1 -- 172.21.15.42:0/2913020261 wait complete. 2024-05-28T14:10:55.367 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:10:55.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.528+0000 7fde4fe00640 1 Processor -- start 2024-05-28T14:10:55.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.528+0000 7fde4fe00640 1 -- start start 2024-05-28T14:10:55.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.532+0000 7fde4fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fde500cc660 0x7fde5015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.532+0000 7fde4fe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fde501415e0 0x7fde5006db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.532+0000 7fde4fe00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde500ce090 con 0x7fde500d0320 2024-05-28T14:10:55.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.532+0000 7fde4fe00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fde5015bff0 con 0x7fde500cc660 2024-05-28T14:10:55.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.532+0000 7fde4fe00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fde5015c2d0 con 0x7fde501415e0 2024-05-28T14:10:55.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.532+0000 7fde4ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fde500cc660 0x7fde5015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.532+0000 7fde4f400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fde500d0320 0x7fde500d0730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56746/0 (socket says 172.21.15.42:56746) 2024-05-28T14:10:55.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.532+0000 7fde4f400640 1 -- 172.21.15.42:0/109493664 learned_addr learned my addr 172.21.15.42:0/109493664 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:55.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.532+0000 7fde46000640 1 --2- 172.21.15.42:0/109493664 >> v2:172.21.15.63:3300/0 conn(0x7fde501415e0 0x7fde5006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.532+0000 7fde4ea00640 1 -- 172.21.15.42:0/109493664 >> v2:172.21.15.63:3300/0 conn(0x7fde501415e0 msgr2=0x7fde5006db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:55.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.532+0000 7fde4ea00640 1 --2- 172.21.15.42:0/109493664 >> v2:172.21.15.63:3300/0 conn(0x7fde501415e0 0x7fde5006db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.532+0000 7fde4ea00640 1 -- 172.21.15.42:0/109493664 >> v1:172.21.15.42:6789/0 conn(0x7fde500d0320 legacy=0x7fde500d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.532+0000 7fde4e000640 1 -- 172.21.15.42:0/109493664 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2996808219 0 0) 0x7fde500ce090 con 0x7fde500d0320 2024-05-28T14:10:55.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.532+0000 7fde4ea00640 1 -- 172.21.15.42:0/109493664 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5015c5b0 con 0x7fde500cc660 2024-05-28T14:10:55.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4ea00640 1 --2- 172.21.15.42:0/109493664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde500cc660 0x7fde5015b9a0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fde500cde40 tx=0x7fde4001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573a89112b10f3a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:55.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4e000640 1 -- 172.21.15.42:0/109493664 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde400b0070 con 0x7fde500cc660 2024-05-28T14:10:55.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4e000640 1 -- 172.21.15.42:0/109493664 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde400ab600 con 0x7fde500cc660 2024-05-28T14:10:55.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4e000640 1 -- 172.21.15.42:0/109493664 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde400ab9d0 con 0x7fde500cc660 2024-05-28T14:10:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 -- 172.21.15.42:0/109493664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde500cc660 msgr2=0x7fde5015b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 --2- 172.21.15.42:0/109493664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde500cc660 0x7fde5015b9a0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fde500cde40 tx=0x7fde4001c480 comp rx=0 tx=0).stop 2024-05-28T14:10:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 -- 172.21.15.42:0/109493664 shutdown_connections 2024-05-28T14:10:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 --2- 172.21.15.42:0/109493664 >> v2:172.21.15.63:3300/0 conn(0x7fde501415e0 0x7fde5006db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 --2- 172.21.15.42:0/109493664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde500cc660 0x7fde5015b9a0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 -- 172.21.15.42:0/109493664 >> 172.21.15.42:0/109493664 conn(0x7fde5006bbc0 msgr2=0x7fde5006bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 -- 172.21.15.42:0/109493664 shutdown_connections 2024-05-28T14:10:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 -- 172.21.15.42:0/109493664 wait complete. 2024-05-28T14:10:55.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 Processor -- start 2024-05-28T14:10:55.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 -- start start 2024-05-28T14:10:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde500cc660 0x7fde50155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde500d0320 0x7fde50165bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde501415e0 0x7fde500ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fde501684b0 con 0x7fde500cc660 2024-05-28T14:10:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde50168330 con 0x7fde500d0320 2024-05-28T14:10:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4fe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fde50168630 con 0x7fde501415e0 2024-05-28T14:10:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.536+0000 7fde4ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde500cc660 0x7fde50155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde4ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde500cc660 0x7fde50155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49568/0 (socket says 172.21.15.42:49568) 2024-05-28T14:10:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde4ea00640 1 -- 172.21.15.42:0/2806135489 learned_addr learned my addr 172.21.15.42:0/2806135489 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde4f400640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde501415e0 0x7fde500ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde46000640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde500d0320 0x7fde50165bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde4ea00640 1 -- 172.21.15.42:0/2806135489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde501415e0 msgr2=0x7fde500ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde4ea00640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde501415e0 0x7fde500ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde4ea00640 1 -- 172.21.15.42:0/2806135489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde500d0320 msgr2=0x7fde50165bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde4ea00640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde500d0320 0x7fde50165bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde4ea00640 1 -- 172.21.15.42:0/2806135489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde50073840 con 0x7fde500cc660 2024-05-28T14:10:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde46000640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde500d0320 0x7fde50165bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:10:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde4ea00640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde500cc660 0x7fde50155d60 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fde400ae040 tx=0x7fde4001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde47400640 1 -- 172.21.15.42:0/2806135489 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde400b0050 con 0x7fde500cc660 2024-05-28T14:10:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde47400640 1 -- 172.21.15.42:0/2806135489 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde400bc070 con 0x7fde500cc660 2024-05-28T14:10:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde47400640 1 -- 172.21.15.42:0/2806135489 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde400b5850 con 0x7fde500cc660 2024-05-28T14:10:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde4fe00640 1 -- 172.21.15.42:0/2806135489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde501660f0 con 0x7fde500cc660 2024-05-28T14:10:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde4fe00640 1 -- 172.21.15.42:0/2806135489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde500b1fe0 con 0x7fde500cc660 2024-05-28T14:10:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde4fe00640 1 -- 172.21.15.42:0/2806135489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde5004fcf0 con 0x7fde500cc660 2024-05-28T14:10:55.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.540+0000 7fde47400640 1 -- 172.21.15.42:0/2806135489 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7fde400b59f0 con 0x7fde500cc660 2024-05-28T14:10:55.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.544+0000 7fde47400640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fde2c069730 0x7fde2c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.544+0000 7fde47400640 1 -- 172.21.15.42:0/2806135489 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7fde40120110 con 0x7fde500cc660 2024-05-28T14:10:55.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.544+0000 7fde46000640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fde2c069730 0x7fde2c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.544+0000 7fde46000640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fde2c069730 0x7fde2c06bbf0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fde500b3cd0 tx=0x7fde38008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.548+0000 7fde47400640 1 -- 172.21.15.42:0/2806135489 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fde40016070 con 0x7fde500cc660 2024-05-28T14:10:55.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.744+0000 7fde4fe00640 1 -- 172.21.15.42:0/2806135489 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fde500b1ab0 con 0x7fde2c069730 2024-05-28T14:10:55.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.776+0000 7fde47400640 1 -- 172.21.15.42:0/2806135489 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fde500b1ab0 con 0x7fde2c069730 2024-05-28T14:10:55.758 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:55.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.776+0000 7fde4fe00640 1 -- 172.21.15.42:0/2806135489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fde2c069730 msgr2=0x7fde2c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.776+0000 7fde4fe00640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fde2c069730 0x7fde2c06bbf0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fde500b3cd0 tx=0x7fde38008040 comp rx=0 tx=0).stop 2024-05-28T14:10:55.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.776+0000 7fde4fe00640 1 -- 172.21.15.42:0/2806135489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde500cc660 msgr2=0x7fde50155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.776+0000 7fde4fe00640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde500cc660 0x7fde50155d60 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fde400ae040 tx=0x7fde4001acc0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.776+0000 7fde4fe00640 1 -- 172.21.15.42:0/2806135489 shutdown_connections 2024-05-28T14:10:55.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.776+0000 7fde4fe00640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fde2c069730 0x7fde2c06bbf0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.776+0000 7fde4fe00640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde501415e0 0x7fde500ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.776+0000 7fde4fe00640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde500d0320 0x7fde50165bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.776+0000 7fde4fe00640 1 --2- 172.21.15.42:0/2806135489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde500cc660 0x7fde50155d60 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.776+0000 7fde4fe00640 1 -- 172.21.15.42:0/2806135489 >> 172.21.15.42:0/2806135489 conn(0x7fde5006bbc0 msgr2=0x7fde5015a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:55.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.776+0000 7fde4fe00640 1 -- 172.21.15.42:0/2806135489 shutdown_connections 2024-05-28T14:10:55.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.776+0000 7fde4fe00640 1 -- 172.21.15.42:0/2806135489 wait complete. 2024-05-28T14:10:55.780 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-28T14:10:55.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.940+0000 7ffad4800640 1 Processor -- start 2024-05-28T14:10:55.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.940+0000 7ffad4800640 1 -- start start 2024-05-28T14:10:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.940+0000 7ffad4800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffacc0ce3b0 0x7ffacc0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.940+0000 7ffad4800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffacc13fd10 0x7ffacc140180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.940+0000 7ffad4800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffacc160230 con 0x7ffacc071de0 2024-05-28T14:10:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.940+0000 7ffad4800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ffacc160510 con 0x7ffacc13fd10 2024-05-28T14:10:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.940+0000 7ffad4800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffacc1607f0 con 0x7ffacc0ce3b0 2024-05-28T14:10:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.944+0000 7ffad2800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffacc13fd10 0x7ffacc140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.944+0000 7ffad1e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ffacc071de0 0x7ffacc0721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56748/0 (socket says 172.21.15.42:56748) 2024-05-28T14:10:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.944+0000 7ffad2800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffacc13fd10 0x7ffacc140180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49578/0 (socket says 172.21.15.42:49578) 2024-05-28T14:10:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.944+0000 7ffad1e00640 1 -- 172.21.15.42:0/609195612 learned_addr learned my addr 172.21.15.42:0/609195612 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.944+0000 7ffad1400640 1 --2- 172.21.15.42:0/609195612 >> v2:172.21.15.63:3300/0 conn(0x7ffacc0ce3b0 0x7ffacc0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.944+0000 7ffad2800640 1 -- 172.21.15.42:0/609195612 >> v2:172.21.15.63:3300/0 conn(0x7ffacc0ce3b0 msgr2=0x7ffacc0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:55.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.944+0000 7ffad2800640 1 --2- 172.21.15.42:0/609195612 >> v2:172.21.15.63:3300/0 conn(0x7ffacc0ce3b0 0x7ffacc0ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.944+0000 7ffad2800640 1 -- 172.21.15.42:0/609195612 >> v1:172.21.15.42:6789/0 conn(0x7ffacc071de0 legacy=0x7ffacc0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.944+0000 7ffad2800640 1 -- 172.21.15.42:0/609195612 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffacc160ad0 con 0x7ffacc13fd10 2024-05-28T14:10:55.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.944+0000 7ffad2800640 1 --2- 172.21.15.42:0/609195612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffacc13fd10 0x7ffacc140180 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ffacc073280 tx=0x7ffac801c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35808d283f654163 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:55.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.944+0000 7ffad0a00640 1 -- 172.21.15.42:0/609195612 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac80a66e0 con 0x7ffacc13fd10 2024-05-28T14:10:55.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.944+0000 7ffad0a00640 1 -- 172.21.15.42:0/609195612 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac80a6880 con 0x7ffacc13fd10 2024-05-28T14:10:55.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.944+0000 7ffad0a00640 1 -- 172.21.15.42:0/609195612 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac80a6c50 con 0x7ffacc13fd10 2024-05-28T14:10:55.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 -- 172.21.15.42:0/609195612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffacc13fd10 msgr2=0x7ffacc140180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 --2- 172.21.15.42:0/609195612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffacc13fd10 0x7ffacc140180 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ffacc073280 tx=0x7ffac801c810 comp rx=0 tx=0).stop 2024-05-28T14:10:55.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 -- 172.21.15.42:0/609195612 shutdown_connections 2024-05-28T14:10:55.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 --2- 172.21.15.42:0/609195612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffacc13fd10 0x7ffacc140180 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 --2- 172.21.15.42:0/609195612 >> v2:172.21.15.63:3300/0 conn(0x7ffacc0ce3b0 0x7ffacc0ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 -- 172.21.15.42:0/609195612 >> 172.21.15.42:0/609195612 conn(0x7ffacc0b21f0 msgr2=0x7ffacc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:55.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 -- 172.21.15.42:0/609195612 shutdown_connections 2024-05-28T14:10:55.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 -- 172.21.15.42:0/609195612 wait complete. 2024-05-28T14:10:55.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 Processor -- start 2024-05-28T14:10:55.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 -- start start 2024-05-28T14:10:55.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffacc071de0 0x7ffacc154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffacc0ce3b0 0x7ffacc159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffacc13fd10 0x7ffacc155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc169830 con 0x7ffacc13fd10 2024-05-28T14:10:55.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc1696b0 con 0x7ffacc0ce3b0 2024-05-28T14:10:55.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad4800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffacc1699b0 con 0x7ffacc071de0 2024-05-28T14:10:55.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffacc13fd10 0x7ffacc155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffacc13fd10 0x7ffacc155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49582/0 (socket says 172.21.15.42:49582) 2024-05-28T14:10:55.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad2800640 1 -- 172.21.15.42:0/2376375216 learned_addr learned my addr 172.21.15.42:0/2376375216 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:55.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad1400640 1 --2- 172.21.15.42:0/2376375216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffacc0ce3b0 0x7ffacc159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad1e00640 1 --2- 172.21.15.42:0/2376375216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffacc071de0 0x7ffacc154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad2800640 1 -- 172.21.15.42:0/2376375216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffacc071de0 msgr2=0x7ffacc154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad2800640 1 --2- 172.21.15.42:0/2376375216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffacc071de0 0x7ffacc154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad2800640 1 -- 172.21.15.42:0/2376375216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffacc0ce3b0 msgr2=0x7ffacc159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:55.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad2800640 1 --2- 172.21.15.42:0/2376375216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffacc0ce3b0 0x7ffacc159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:55.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.948+0000 7ffad2800640 1 -- 172.21.15.42:0/2376375216 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffacc155040 con 0x7ffacc13fd10 2024-05-28T14:10:55.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.952+0000 7ffad2800640 1 --2- 172.21.15.42:0/2376375216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffacc13fd10 0x7ffacc155240 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ffac80b0040 tx=0x7ffac80195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:55.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.952+0000 7ffac6000640 1 -- 172.21.15.42:0/2376375216 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac80be070 con 0x7ffacc13fd10 2024-05-28T14:10:55.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.952+0000 7ffac6000640 1 -- 172.21.15.42:0/2376375216 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac80a66d0 con 0x7ffacc13fd10 2024-05-28T14:10:55.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.952+0000 7ffac6000640 1 -- 172.21.15.42:0/2376375216 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac80b8a40 con 0x7ffacc13fd10 2024-05-28T14:10:55.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.952+0000 7ffad4800640 1 -- 172.21.15.42:0/2376375216 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffacc15ddc0 con 0x7ffacc13fd10 2024-05-28T14:10:55.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.952+0000 7ffad4800640 1 -- 172.21.15.42:0/2376375216 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffacc15e380 con 0x7ffacc13fd10 2024-05-28T14:10:55.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.952+0000 7ffad4800640 1 -- 172.21.15.42:0/2376375216 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffacc0738b0 con 0x7ffacc13fd10 2024-05-28T14:10:55.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.952+0000 7ffac6000640 1 -- 172.21.15.42:0/2376375216 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7ffac80c1070 con 0x7ffacc13fd10 2024-05-28T14:10:55.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.952+0000 7ffac6000640 1 --2- 172.21.15.42:0/2376375216 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffab0069760 0x7ffab006bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:55.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.952+0000 7ffac6000640 1 -- 172.21.15.42:0/2376375216 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5795+0+0 (secure 0 0 0) 0x7ffac8120840 con 0x7ffacc13fd10 2024-05-28T14:10:55.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.952+0000 7ffad1e00640 1 --2- 172.21.15.42:0/2376375216 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffab0069760 0x7ffab006bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:55.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.956+0000 7ffad1e00640 1 --2- 172.21.15.42:0/2376375216 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffab0069760 0x7ffab006bc20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ffac0000f70 tx=0x7ffac0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:55.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:55.956+0000 7ffac6000640 1 -- 172.21.15.42:0/2376375216 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffac80c8070 con 0x7ffacc13fd10 2024-05-28T14:10:56.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.152+0000 7ffad4800640 1 -- 172.21.15.42:0/2376375216 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7ffacc13cf30 con 0x7ffab0069760 2024-05-28T14:10:56.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.176+0000 7ffac6000640 1 -- 172.21.15.42:0/2376375216 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffac00058a0 con 0x7ffab0069760 2024-05-28T14:10:56.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.180+0000 7ffad4800640 1 -- 172.21.15.42:0/2376375216 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffab0069760 msgr2=0x7ffab006bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:56.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.180+0000 7ffad4800640 1 --2- 172.21.15.42:0/2376375216 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffab0069760 0x7ffab006bc20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ffac0000f70 tx=0x7ffac0009040 comp rx=0 tx=0).stop 2024-05-28T14:10:56.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.180+0000 7ffad4800640 1 -- 172.21.15.42:0/2376375216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffacc13fd10 msgr2=0x7ffacc155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:56.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.180+0000 7ffad4800640 1 --2- 172.21.15.42:0/2376375216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffacc13fd10 0x7ffacc155240 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ffac80b0040 tx=0x7ffac80195b0 comp rx=0 tx=0).stop 2024-05-28T14:10:56.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.180+0000 7ffad4800640 1 -- 172.21.15.42:0/2376375216 shutdown_connections 2024-05-28T14:10:56.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.180+0000 7ffad4800640 1 --2- 172.21.15.42:0/2376375216 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffab0069760 0x7ffab006bc20 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:56.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.180+0000 7ffad4800640 1 --2- 172.21.15.42:0/2376375216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffacc13fd10 0x7ffacc155240 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:56.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.180+0000 7ffad4800640 1 --2- 172.21.15.42:0/2376375216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffacc0ce3b0 0x7ffacc159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:56.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.180+0000 7ffad4800640 1 --2- 172.21.15.42:0/2376375216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffacc071de0 0x7ffacc154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:56.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.180+0000 7ffad4800640 1 -- 172.21.15.42:0/2376375216 >> 172.21.15.42:0/2376375216 conn(0x7ffacc0b21f0 msgr2=0x7ffacc15ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:56.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.180+0000 7ffad4800640 1 -- 172.21.15.42:0/2376375216 shutdown_connections 2024-05-28T14:10:56.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.180+0000 7ffad4800640 1 -- 172.21.15.42:0/2376375216 wait complete. 2024-05-28T14:10:56.164 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:56.180+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:56.165 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:56.180+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:56.165 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:56.180+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:56.165 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:56.180+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:56.165 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:56.180+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:56.182 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:56.196+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:56.182 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:56.196+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:56.182 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:56.196+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:56.182 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:56.196+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:56.182 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:10:56.196+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:10:56.185 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-05-28T14:10:56.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.348+0000 7f0dada00640 1 Processor -- start 2024-05-28T14:10:56.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.352+0000 7f0dada00640 1 -- start start 2024-05-28T14:10:56.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.352+0000 7f0dada00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0da806fd10 0x7f0da80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:56.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.352+0000 7f0dada00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da80cf6d0 con 0x7f0da81524c0 2024-05-28T14:10:56.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.352+0000 7f0dada00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da80b55e0 con 0x7f0da80b5130 2024-05-28T14:10:56.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.352+0000 7f0dada00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0da815fe10 con 0x7f0da806fd10 2024-05-28T14:10:56.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.352+0000 7f0da7e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0da81524c0 0x7f0da815c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56754/0 (socket says 172.21.15.42:56754) 2024-05-28T14:10:56.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.352+0000 7f0da7e00640 1 -- 172.21.15.42:0/2218542624 learned_addr learned my addr 172.21.15.42:0/2218542624 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:56.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.352+0000 7f0dad000640 1 --2- 172.21.15.42:0/2218542624 >> v2:172.21.15.42:3300/0 conn(0x7f0da806fd10 0x7f0da80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:56.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.352+0000 7f0da7400640 1 -- 172.21.15.42:0/2218542624 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1527554074 0 0) 0x7f0da80cf6d0 con 0x7f0da81524c0 2024-05-28T14:10:56.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.352+0000 7f0da7400640 1 -- 172.21.15.42:0/2218542624 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d8c003530 con 0x7f0da81524c0 2024-05-28T14:10:56.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.352+0000 7f0dad000640 1 -- 172.21.15.42:0/2218542624 >> v1:172.21.15.63:6789/0 conn(0x7f0da80b5130 legacy=0x7f0da806f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:56.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.352+0000 7f0dad000640 1 -- 172.21.15.42:0/2218542624 >> v1:172.21.15.42:6789/0 conn(0x7f0da81524c0 legacy=0x7f0da815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:56.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.352+0000 7f0dad000640 1 -- 172.21.15.42:0/2218542624 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da815f290 con 0x7f0da806fd10 2024-05-28T14:10:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.356+0000 7f0dad000640 1 --2- 172.21.15.42:0/2218542624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da806fd10 0x7f0da80cfb70 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009d90 tx=0x7f0da001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288a37eaf29217d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.356+0000 7f0da7400640 1 -- 172.21.15.42:0/2218542624 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da00b2070 con 0x7f0da806fd10 2024-05-28T14:10:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.356+0000 7f0da7400640 1 -- 172.21.15.42:0/2218542624 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da00a5a00 con 0x7f0da806fd10 2024-05-28T14:10:56.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.356+0000 7f0da7400640 1 -- 172.21.15.42:0/2218542624 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da00a5db0 con 0x7f0da806fd10 2024-05-28T14:10:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.356+0000 7f0dada00640 1 -- 172.21.15.42:0/2218542624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da806fd10 msgr2=0x7f0da80cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.356+0000 7f0dada00640 1 --2- 172.21.15.42:0/2218542624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da806fd10 0x7f0da80cfb70 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009d90 tx=0x7f0da001c940 comp rx=0 tx=0).stop 2024-05-28T14:10:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.356+0000 7f0dada00640 1 -- 172.21.15.42:0/2218542624 shutdown_connections 2024-05-28T14:10:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.356+0000 7f0dada00640 1 --2- 172.21.15.42:0/2218542624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da806fd10 0x7f0da80cfb70 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.356+0000 7f0dada00640 1 -- 172.21.15.42:0/2218542624 >> 172.21.15.42:0/2218542624 conn(0x7f0da8072dc0 msgr2=0x7f0da806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.356+0000 7f0dada00640 1 -- 172.21.15.42:0/2218542624 shutdown_connections 2024-05-28T14:10:56.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.356+0000 7f0dada00640 1 -- 172.21.15.42:0/2218542624 wait complete. 2024-05-28T14:10:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.356+0000 7f0dada00640 1 Processor -- start 2024-05-28T14:10:56.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0dada00640 1 -- start start 2024-05-28T14:10:56.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0dada00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da806fd10 0x7f0da8150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:56.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0dada00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80b5130 0x7f0da814f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:56.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0dada00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da81524c0 0x7f0da814fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:56.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0dada00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0da815ee40 con 0x7f0da80b5130 2024-05-28T14:10:56.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0dada00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0da80b0150 con 0x7f0da806fd10 2024-05-28T14:10:56.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0dada00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0da815efc0 con 0x7f0da81524c0 2024-05-28T14:10:56.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0d9fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80b5130 0x7f0da814f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:56.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0d9fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80b5130 0x7f0da814f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49614/0 (socket says 172.21.15.42:49614) 2024-05-28T14:10:56.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0d9fe00640 1 -- 172.21.15.42:0/1979329668 learned_addr learned my addr 172.21.15.42:0/1979329668 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:56.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0da7e00640 1 --2- 172.21.15.42:0/1979329668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da806fd10 0x7f0da8150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:56.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0dad000640 1 --2- 172.21.15.42:0/1979329668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da81524c0 0x7f0da814fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:56.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0d9fe00640 1 -- 172.21.15.42:0/1979329668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da81524c0 msgr2=0x7f0da814fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:56.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0d9fe00640 1 --2- 172.21.15.42:0/1979329668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da81524c0 0x7f0da814fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:56.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0d9fe00640 1 -- 172.21.15.42:0/1979329668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da806fd10 msgr2=0x7f0da8150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:56.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0d9fe00640 1 --2- 172.21.15.42:0/1979329668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da806fd10 0x7f0da8150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:56.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0d9fe00640 1 -- 172.21.15.42:0/1979329668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da814c6d0 con 0x7f0da80b5130 2024-05-28T14:10:56.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0d9fe00640 1 --2- 172.21.15.42:0/1979329668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80b5130 0x7f0da814f4e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c450 tx=0x7f0d9000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:56.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0da4c00640 1 -- 172.21.15.42:0/1979329668 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90016020 con 0x7f0da80b5130 2024-05-28T14:10:56.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0da4c00640 1 -- 172.21.15.42:0/1979329668 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d900052d0 con 0x7f0da80b5130 2024-05-28T14:10:56.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0dada00640 1 -- 172.21.15.42:0/1979329668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da814c3f0 con 0x7f0da80b5130 2024-05-28T14:10:56.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0da4c00640 1 -- 172.21.15.42:0/1979329668 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90007260 con 0x7f0da80b5130 2024-05-28T14:10:56.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0dada00640 1 -- 172.21.15.42:0/1979329668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da81373b0 con 0x7f0da80b5130 2024-05-28T14:10:56.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.360+0000 7f0dada00640 1 -- 172.21.15.42:0/1979329668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da80691a0 con 0x7f0da80b5130 2024-05-28T14:10:56.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.364+0000 7f0da4c00640 1 -- 172.21.15.42:0/1979329668 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7f0d90007400 con 0x7f0da80b5130 2024-05-28T14:10:56.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.364+0000 7f0da4c00640 1 --2- 172.21.15.42:0/1979329668 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d84069730 0x7f0d8406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:56.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.364+0000 7f0da4c00640 1 -- 172.21.15.42:0/1979329668 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7f0d90084e90 con 0x7f0da80b5130 2024-05-28T14:10:56.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.364+0000 7f0da7e00640 1 --2- 172.21.15.42:0/1979329668 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d84069730 0x7f0d8406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:56.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.364+0000 7f0da7e00640 1 --2- 172.21.15.42:0/1979329668 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d84069730 0x7f0d8406bbf0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0d980011f0 tx=0x7f0d98007d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:56.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.368+0000 7f0da4c00640 1 -- 172.21.15.42:0/1979329668 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0d90020020 con 0x7f0da80b5130 2024-05-28T14:10:56.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.560+0000 7f0dada00640 1 -- 172.21.15.42:0/1979329668 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f0da8070910 con 0x7f0d84069730 2024-05-28T14:10:56.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.580+0000 7f0da4c00640 1 -- 172.21.15.42:0/1979329668 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0da8070910 con 0x7f0d84069730 2024-05-28T14:10:56.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.580+0000 7f0dada00640 1 -- 172.21.15.42:0/1979329668 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d84069730 msgr2=0x7f0d8406bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:56.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.580+0000 7f0dada00640 1 --2- 172.21.15.42:0/1979329668 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d84069730 0x7f0d8406bbf0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0d980011f0 tx=0x7f0d98007d40 comp rx=0 tx=0).stop 2024-05-28T14:10:56.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.580+0000 7f0dada00640 1 -- 172.21.15.42:0/1979329668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80b5130 msgr2=0x7f0da814f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:56.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.580+0000 7f0dada00640 1 --2- 172.21.15.42:0/1979329668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80b5130 0x7f0da814f4e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c450 tx=0x7f0d9000c910 comp rx=0 tx=0).stop 2024-05-28T14:10:56.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.580+0000 7f0dada00640 1 -- 172.21.15.42:0/1979329668 shutdown_connections 2024-05-28T14:10:56.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.580+0000 7f0dada00640 1 --2- 172.21.15.42:0/1979329668 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0d84069730 0x7f0d8406bbf0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:56.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.580+0000 7f0dada00640 1 --2- 172.21.15.42:0/1979329668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0da81524c0 0x7f0da814fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:56.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.580+0000 7f0dada00640 1 --2- 172.21.15.42:0/1979329668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da80b5130 0x7f0da814f4e0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:56.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.580+0000 7f0dada00640 1 --2- 172.21.15.42:0/1979329668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da806fd10 0x7f0da8150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:56.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.580+0000 7f0dada00640 1 -- 172.21.15.42:0/1979329668 >> 172.21.15.42:0/1979329668 conn(0x7f0da8072dc0 msgr2=0x7f0da806c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:56.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.580+0000 7f0dada00640 1 -- 172.21.15.42:0/1979329668 shutdown_connections 2024-05-28T14:10:56.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56.580+0000 7f0dada00640 1 -- 172.21.15.42:0/1979329668 wait complete. 2024-05-28T14:10:56.587 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-28T14:10:56.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-28T14:10:56.645 INFO:teuthology.orchestra.run.smithi042.stdout:2 2024-05-28T14:10:56.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:10:56.652 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T14:10:56.652 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:10:56.789 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:10:56.852 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T14:10:56.852 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:10:56.852 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:10:56.852 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:10:56.861 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:10:57.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff085200640 1 Processor -- start 2024-05-28T14:10:57.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff085200640 1 -- start start 2024-05-28T14:10:57.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff085200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff08015fc00 0x7ff08009ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff085200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff080070840 0x7ff0800d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff085200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff08015ffe0 con 0x7ff08009aa50 2024-05-28T14:10:57.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff085200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff0800d1180 con 0x7ff08015fc00 2024-05-28T14:10:57.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff085200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff080070de0 con 0x7ff080070840 2024-05-28T14:10:57.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff07fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff08009aa50 0x7ff0800ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:56756/0 (socket says 172.21.15.42:56756) 2024-05-28T14:10:57.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff07f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff08015fc00 0x7ff08009ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff07fe00640 1 -- 172.21.15.42:0/865766020 learned_addr learned my addr 172.21.15.42:0/865766020 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:57.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff076a00640 1 --2- 172.21.15.42:0/865766020 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff080070840 0x7ff0800d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff07f400640 1 -- 172.21.15.42:0/865766020 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff080070840 msgr2=0x7ff0800d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:57.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff07f400640 1 --2- 172.21.15.42:0/865766020 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff080070840 0x7ff0800d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff07f400640 1 -- 172.21.15.42:0/865766020 >> v1:172.21.15.42:6789/0 conn(0x7ff08009aa50 legacy=0x7ff0800ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.064+0000 7ff07f400640 1 -- 172.21.15.42:0/865766020 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0800713c0 con 0x7ff08015fc00 2024-05-28T14:10:57.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.068+0000 7ff07f400640 1 --2- 172.21.15.42:0/865766020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08015fc00 0x7ff08009ba50 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7ff0700097c0 tx=0x7ff07001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f989891f02f0520f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:57.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.068+0000 7ff07ea00640 1 -- 172.21.15.42:0/865766020 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0700b0070 con 0x7ff08015fc00 2024-05-28T14:10:57.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.068+0000 7ff07ea00640 1 -- 172.21.15.42:0/865766020 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0700ab600 con 0x7ff08015fc00 2024-05-28T14:10:57.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.068+0000 7ff07ea00640 1 -- 172.21.15.42:0/865766020 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0700ab9d0 con 0x7ff08015fc00 2024-05-28T14:10:57.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.068+0000 7ff085200640 1 -- 172.21.15.42:0/865766020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08015fc00 msgr2=0x7ff08009ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.068+0000 7ff085200640 1 --2- 172.21.15.42:0/865766020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08015fc00 0x7ff08009ba50 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7ff0700097c0 tx=0x7ff07001c4a0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.068+0000 7ff085200640 1 -- 172.21.15.42:0/865766020 shutdown_connections 2024-05-28T14:10:57.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.068+0000 7ff085200640 1 --2- 172.21.15.42:0/865766020 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff080070840 0x7ff0800d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.068+0000 7ff085200640 1 --2- 172.21.15.42:0/865766020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08015fc00 0x7ff08009ba50 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.068+0000 7ff085200640 1 -- 172.21.15.42:0/865766020 >> 172.21.15.42:0/865766020 conn(0x7ff0800b3d70 msgr2=0x7ff0800b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:57.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.068+0000 7ff085200640 1 -- 172.21.15.42:0/865766020 shutdown_connections 2024-05-28T14:10:57.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.068+0000 7ff085200640 1 -- 172.21.15.42:0/865766020 wait complete. 2024-05-28T14:10:57.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff085200640 1 Processor -- start 2024-05-28T14:10:57.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff085200640 1 -- start start 2024-05-28T14:10:57.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff085200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff080070840 0x7ff08006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff085200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff08009aa50 0x7ff08015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff085200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08015fc00 0x7ff08015e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff085200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff080169750 con 0x7ff08015fc00 2024-05-28T14:10:57.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff085200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801695d0 con 0x7ff08009aa50 2024-05-28T14:10:57.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff085200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff0801698d0 con 0x7ff080070840 2024-05-28T14:10:57.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff07fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08015fc00 0x7ff08015e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff07fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08015fc00 0x7ff08015e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49630/0 (socket says 172.21.15.42:49630) 2024-05-28T14:10:57.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff07fe00640 1 -- 172.21.15.42:0/4099840654 learned_addr learned my addr 172.21.15.42:0/4099840654 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:57.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff07f400640 1 --2- 172.21.15.42:0/4099840654 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff080070840 0x7ff08006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff076a00640 1 --2- 172.21.15.42:0/4099840654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff08009aa50 0x7ff08015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff07fe00640 1 -- 172.21.15.42:0/4099840654 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff080070840 msgr2=0x7ff08006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff07fe00640 1 --2- 172.21.15.42:0/4099840654 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff080070840 0x7ff08006e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff07fe00640 1 -- 172.21.15.42:0/4099840654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff08009aa50 msgr2=0x7ff08015db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff07fe00640 1 --2- 172.21.15.42:0/4099840654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff08009aa50 0x7ff08015db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff07fe00640 1 -- 172.21.15.42:0/4099840654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08006d420 con 0x7ff08015fc00 2024-05-28T14:10:57.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff07fe00640 1 --2- 172.21.15.42:0/4099840654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08015fc00 0x7ff08015e060 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7ff07800c400 tx=0x7ff07800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:57.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff077e00640 1 -- 172.21.15.42:0/4099840654 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078017070 con 0x7ff08015fc00 2024-05-28T14:10:57.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff077e00640 1 -- 172.21.15.42:0/4099840654 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff078005cf0 con 0x7ff08015fc00 2024-05-28T14:10:57.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff085200640 1 -- 172.21.15.42:0/4099840654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08006d700 con 0x7ff08015fc00 2024-05-28T14:10:57.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff077e00640 1 -- 172.21.15.42:0/4099840654 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0780072f0 con 0x7ff08015fc00 2024-05-28T14:10:57.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.072+0000 7ff085200640 1 -- 172.21.15.42:0/4099840654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08006dcc0 con 0x7ff08015fc00 2024-05-28T14:10:57.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.076+0000 7ff085200640 1 -- 172.21.15.42:0/4099840654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08004fcf0 con 0x7ff08015fc00 2024-05-28T14:10:57.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.076+0000 7ff077e00640 1 -- 172.21.15.42:0/4099840654 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7ff07801a070 con 0x7ff08015fc00 2024-05-28T14:10:57.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.076+0000 7ff077e00640 1 --2- 172.21.15.42:0/4099840654 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff05c069730 0x7ff05c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.076+0000 7ff077e00640 1 -- 172.21.15.42:0/4099840654 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7ff078084c80 con 0x7ff08015fc00 2024-05-28T14:10:57.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.076+0000 7ff07f400640 1 --2- 172.21.15.42:0/4099840654 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff05c069730 0x7ff05c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.076+0000 7ff07f400640 1 --2- 172.21.15.42:0/4099840654 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff05c069730 0x7ff05c06bbf0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff0700ad000 tx=0x7ff0700a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:57.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.080+0000 7ff077e00640 1 -- 172.21.15.42:0/4099840654 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff0780562f0 con 0x7ff08015fc00 2024-05-28T14:10:57.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.248+0000 7ff085200640 1 -- 172.21.15.42:0/4099840654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0800b8200 con 0x7ff08015fc00 2024-05-28T14:10:57.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.252+0000 7ff077e00640 1 -- 172.21.15.42:0/4099840654 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16852 (secure 0 0 0) 0x7ff07805a5f0 con 0x7ff08015fc00 2024-05-28T14:10:57.235 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:57.235 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":93,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:10:56.060535+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:09:28.913704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-28T14:09:30.108327+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":93,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.3e","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:10:57.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.252+0000 7ff085200640 1 -- 172.21.15.42:0/4099840654 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff05c069730 msgr2=0x7ff05c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.252+0000 7ff085200640 1 --2- 172.21.15.42:0/4099840654 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff05c069730 0x7ff05c06bbf0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff0700ad000 tx=0x7ff0700a6000 comp rx=0 tx=0).stop 2024-05-28T14:10:57.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.252+0000 7ff085200640 1 -- 172.21.15.42:0/4099840654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08015fc00 msgr2=0x7ff08015e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.252+0000 7ff085200640 1 --2- 172.21.15.42:0/4099840654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08015fc00 0x7ff08015e060 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7ff07800c400 tx=0x7ff07800c8c0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.252+0000 7ff085200640 1 -- 172.21.15.42:0/4099840654 shutdown_connections 2024-05-28T14:10:57.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.252+0000 7ff085200640 1 --2- 172.21.15.42:0/4099840654 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff05c069730 0x7ff05c06bbf0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.252+0000 7ff085200640 1 --2- 172.21.15.42:0/4099840654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08015fc00 0x7ff08015e060 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.252+0000 7ff085200640 1 --2- 172.21.15.42:0/4099840654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff08009aa50 0x7ff08015db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.252+0000 7ff085200640 1 --2- 172.21.15.42:0/4099840654 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff080070840 0x7ff08006e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.252+0000 7ff085200640 1 -- 172.21.15.42:0/4099840654 >> 172.21.15.42:0/4099840654 conn(0x7ff0800b3d70 msgr2=0x7ff0800b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:57.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.252+0000 7ff085200640 1 -- 172.21.15.42:0/4099840654 shutdown_connections 2024-05-28T14:10:57.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.252+0000 7ff085200640 1 -- 172.21.15.42:0/4099840654 wait complete. 2024-05-28T14:10:57.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T14:09:28.913704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T14:09:30.108327+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 93, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:10:57.257 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:10:57.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.416+0000 7fa612600640 1 Processor -- start 2024-05-28T14:10:57.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.416+0000 7fa612600640 1 -- start start 2024-05-28T14:10:57.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.416+0000 7fa612600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa60c0cfec0 0x7fa60c0d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.416+0000 7fa612600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa60c072530 con 0x7fa60c143f90 2024-05-28T14:10:57.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.416+0000 7fa612600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa60c0d07e0 con 0x7fa60c06dfa0 2024-05-28T14:10:57.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.416+0000 7fa612600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa60c06cb00 con 0x7fa60c0cfec0 2024-05-28T14:10:57.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.416+0000 7fa60be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa60c143f90 0x7fa60c071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42912/0 (socket says 172.21.15.42:42912) 2024-05-28T14:10:57.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.416+0000 7fa60be00640 1 -- 172.21.15.42:0/2105929030 learned_addr learned my addr 172.21.15.42:0/2105929030 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:57.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.416+0000 7fa611200640 1 --2- 172.21.15.42:0/2105929030 >> v2:172.21.15.63:3300/0 conn(0x7fa60c0cfec0 0x7fa60c0d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.420+0000 7fa60b400640 1 -- 172.21.15.42:0/2105929030 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4289895964 0 0) 0x7fa60c072530 con 0x7fa60c143f90 2024-05-28T14:10:57.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.420+0000 7fa60b400640 1 -- 172.21.15.42:0/2105929030 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5f4003530 con 0x7fa60c143f90 2024-05-28T14:10:57.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.420+0000 7fa611200640 1 -- 172.21.15.42:0/2105929030 >> v1:172.21.15.63:6789/0 conn(0x7fa60c06dfa0 legacy=0x7fa60c071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.420+0000 7fa60b400640 1 -- 172.21.15.42:0/2105929030 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2234589325 0 0) 0x7fa60c0d07e0 con 0x7fa60c06dfa0 2024-05-28T14:10:57.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.420+0000 7fa611200640 1 -- 172.21.15.42:0/2105929030 >> v1:172.21.15.42:6789/0 conn(0x7fa60c143f90 legacy=0x7fa60c071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.420+0000 7fa611200640 1 -- 172.21.15.42:0/2105929030 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60c144540 con 0x7fa60c0cfec0 2024-05-28T14:10:57.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.420+0000 7fa611200640 1 --2- 172.21.15.42:0/2105929030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa60c0cfec0 0x7fa60c0d02a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa60c06cc80 tx=0x7fa60001c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f606eceaa9408b6e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:57.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.420+0000 7fa60b400640 1 -- 172.21.15.42:0/2105929030 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000b0070 con 0x7fa60c0cfec0 2024-05-28T14:10:57.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.420+0000 7fa60b400640 1 -- 172.21.15.42:0/2105929030 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6000ab600 con 0x7fa60c0cfec0 2024-05-28T14:10:57.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.420+0000 7fa60b400640 1 -- 172.21.15.42:0/2105929030 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000ab9b0 con 0x7fa60c0cfec0 2024-05-28T14:10:57.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 -- 172.21.15.42:0/2105929030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa60c0cfec0 msgr2=0x7fa60c0d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 --2- 172.21.15.42:0/2105929030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa60c0cfec0 0x7fa60c0d02a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa60c06cc80 tx=0x7fa60001c350 comp rx=0 tx=0).stop 2024-05-28T14:10:57.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 -- 172.21.15.42:0/2105929030 shutdown_connections 2024-05-28T14:10:57.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 --2- 172.21.15.42:0/2105929030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa60c0cfec0 0x7fa60c0d02a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 -- 172.21.15.42:0/2105929030 >> 172.21.15.42:0/2105929030 conn(0x7fa60c06fcc0 msgr2=0x7fa60c0700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:57.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 -- 172.21.15.42:0/2105929030 shutdown_connections 2024-05-28T14:10:57.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 -- 172.21.15.42:0/2105929030 wait complete. 2024-05-28T14:10:57.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 Processor -- start 2024-05-28T14:10:57.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 -- start start 2024-05-28T14:10:57.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa60c06dfa0 0x7fa60c1532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa60c0cfec0 0x7fa60c158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa60c143f90 0x7fa60c1585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c148310 con 0x7fa60c06dfa0 2024-05-28T14:10:57.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c148190 con 0x7fa60c143f90 2024-05-28T14:10:57.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa612600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa60c161e60 con 0x7fa60c0cfec0 2024-05-28T14:10:57.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa611200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa60c06dfa0 0x7fa60c1532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa611200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa60c06dfa0 0x7fa60c1532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52532/0 (socket says 172.21.15.42:52532) 2024-05-28T14:10:57.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa611200640 1 -- 172.21.15.42:0/2026766171 learned_addr learned my addr 172.21.15.42:0/2026766171 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:57.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa611c00640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa60c143f90 0x7fa60c1585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa60be00640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa60c0cfec0 0x7fa60c158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa611200640 1 -- 172.21.15.42:0/2026766171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa60c0cfec0 msgr2=0x7fa60c158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa611200640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa60c0cfec0 0x7fa60c158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa611200640 1 -- 172.21.15.42:0/2026766171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa60c143f90 msgr2=0x7fa60c1585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa611200640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa60c143f90 0x7fa60c1585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa611200640 1 -- 172.21.15.42:0/2026766171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60c0ceea0 con 0x7fa60c06dfa0 2024-05-28T14:10:57.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.424+0000 7fa60be00640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa60c0cfec0 0x7fa60c158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:10:57.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.428+0000 7fa611200640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa60c06dfa0 0x7fa60c1532d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fa600000c10 tx=0x7fa60001ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:57.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.428+0000 7fa608c00640 1 -- 172.21.15.42:0/2026766171 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000b0050 con 0x7fa60c06dfa0 2024-05-28T14:10:57.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.428+0000 7fa608c00640 1 -- 172.21.15.42:0/2026766171 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6000bc070 con 0x7fa60c06dfa0 2024-05-28T14:10:57.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.428+0000 7fa608c00640 1 -- 172.21.15.42:0/2026766171 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000b5810 con 0x7fa60c06dfa0 2024-05-28T14:10:57.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.428+0000 7fa612600640 1 -- 172.21.15.42:0/2026766171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60c0b1f90 con 0x7fa60c06dfa0 2024-05-28T14:10:57.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.428+0000 7fa612600640 1 -- 172.21.15.42:0/2026766171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60c0c4110 con 0x7fa60c06dfa0 2024-05-28T14:10:57.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.428+0000 7fa612600640 1 -- 172.21.15.42:0/2026766171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa60c04fcf0 con 0x7fa60c06dfa0 2024-05-28T14:10:57.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.428+0000 7fa608c00640 1 -- 172.21.15.42:0/2026766171 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7fa6000b59b0 con 0x7fa60c06dfa0 2024-05-28T14:10:57.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.428+0000 7fa608c00640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5ec069760 0x7fa5ec06bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.428+0000 7fa608c00640 1 -- 172.21.15.42:0/2026766171 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7fa60011fa60 con 0x7fa60c06dfa0 2024-05-28T14:10:57.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.428+0000 7fa60be00640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5ec069760 0x7fa5ec06bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.432+0000 7fa60be00640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5ec069760 0x7fa5ec06bc20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0b2a80 tx=0x7fa5f8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:57.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.432+0000 7fa608c00640 1 -- 172.21.15.42:0/2026766171 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa6000c5020 con 0x7fa60c06dfa0 2024-05-28T14:10:57.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.596+0000 7fa612600640 1 -- 172.21.15.42:0/2026766171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa60c0c0d90 con 0x7fa60c06dfa0 2024-05-28T14:10:57.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.596+0000 7fa608c00640 1 -- 172.21.15.42:0/2026766171 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fa600016090 con 0x7fa60c06dfa0 2024-05-28T14:10:57.582 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:10:57.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.600+0000 7fa612600640 1 -- 172.21.15.42:0/2026766171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5ec069760 msgr2=0x7fa5ec06bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.600+0000 7fa612600640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5ec069760 0x7fa5ec06bc20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0b2a80 tx=0x7fa5f8009040 comp rx=0 tx=0).stop 2024-05-28T14:10:57.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.600+0000 7fa612600640 1 -- 172.21.15.42:0/2026766171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa60c06dfa0 msgr2=0x7fa60c1532d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.600+0000 7fa612600640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa60c06dfa0 0x7fa60c1532d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fa600000c10 tx=0x7fa60001ab90 comp rx=0 tx=0).stop 2024-05-28T14:10:57.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.600+0000 7fa612600640 1 -- 172.21.15.42:0/2026766171 shutdown_connections 2024-05-28T14:10:57.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.600+0000 7fa612600640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa5ec069760 0x7fa5ec06bc20 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.600+0000 7fa612600640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa60c143f90 0x7fa60c1585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.600+0000 7fa612600640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa60c0cfec0 0x7fa60c158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.600+0000 7fa612600640 1 --2- 172.21.15.42:0/2026766171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa60c06dfa0 0x7fa60c1532d0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.600+0000 7fa612600640 1 -- 172.21.15.42:0/2026766171 >> 172.21.15.42:0/2026766171 conn(0x7fa60c06fcc0 msgr2=0x7fa60c06cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:57.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.600+0000 7fa612600640 1 -- 172.21.15.42:0/2026766171 shutdown_connections 2024-05-28T14:10:57.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.600+0000 7fa612600640 1 -- 172.21.15.42:0/2026766171 wait complete. 2024-05-28T14:10:57.604 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:10:57.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f80a6a00640 1 Processor -- start 2024-05-28T14:10:57.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f80a6a00640 1 -- start start 2024-05-28T14:10:57.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f80a6a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f80a0071de0 0x7f80a00721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f80a6a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a013fd10 0x7f80a01401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f80a6a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a0072700 con 0x7f80a00ce3b0 2024-05-28T14:10:57.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f80a6a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f80a0160230 con 0x7f80a0071de0 2024-05-28T14:10:57.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f80a6a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80a0160510 con 0x7f80a013fd10 2024-05-28T14:10:57.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f809fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f80a0071de0 0x7f80a00721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f809fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f80a0071de0 0x7f80a00721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52536/0 (socket says 172.21.15.42:52536) 2024-05-28T14:10:57.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f809fe00640 1 -- 172.21.15.42:0/1773119052 learned_addr learned my addr 172.21.15.42:0/1773119052 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:57.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f80a4a00640 1 --2- 172.21.15.42:0/1773119052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a013fd10 0x7f80a01401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f809fe00640 1 -- 172.21.15.42:0/1773119052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a013fd10 msgr2=0x7f80a01401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:57.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f809fe00640 1 --2- 172.21.15.42:0/1773119052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a013fd10 0x7f80a01401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f809fe00640 1 -- 172.21.15.42:0/1773119052 >> v1:172.21.15.63:6789/0 conn(0x7f80a00ce3b0 legacy=0x7f80a00ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:57.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f809fe00640 1 -- 172.21.15.42:0/1773119052 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a01607f0 con 0x7f80a0071de0 2024-05-28T14:10:57.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.764+0000 7f80a4a00640 1 --2- 172.21.15.42:0/1773119052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a013fd10 0x7f80a01401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:10:57.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.768+0000 7f809fe00640 1 --2- 172.21.15.42:0/1773119052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a0071de0 0x7f80a00721c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8090009e50 tx=0x7f809001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2212fdc991902db server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:57.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.768+0000 7f809f400640 1 -- 172.21.15.42:0/1773119052 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80900a5550 con 0x7f80a0071de0 2024-05-28T14:10:57.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.768+0000 7f809f400640 1 -- 172.21.15.42:0/1773119052 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80900a56f0 con 0x7f80a0071de0 2024-05-28T14:10:57.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.768+0000 7f809f400640 1 -- 172.21.15.42:0/1773119052 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80900a5ac0 con 0x7f80a0071de0 2024-05-28T14:10:57.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.768+0000 7f80a6a00640 1 -- 172.21.15.42:0/1773119052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a0071de0 msgr2=0x7f80a00721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.768+0000 7f80a6a00640 1 --2- 172.21.15.42:0/1773119052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a0071de0 0x7f80a00721c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8090009e50 tx=0x7f809001c8d0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.768+0000 7f80a6a00640 1 -- 172.21.15.42:0/1773119052 shutdown_connections 2024-05-28T14:10:57.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.768+0000 7f80a6a00640 1 --2- 172.21.15.42:0/1773119052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a013fd10 0x7f80a01401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.768+0000 7f80a6a00640 1 --2- 172.21.15.42:0/1773119052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a0071de0 0x7f80a00721c0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.768+0000 7f80a6a00640 1 -- 172.21.15.42:0/1773119052 >> 172.21.15.42:0/1773119052 conn(0x7f80a00b21f0 msgr2=0x7f80a00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:57.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.768+0000 7f80a6a00640 1 -- 172.21.15.42:0/1773119052 shutdown_connections 2024-05-28T14:10:57.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.768+0000 7f80a6a00640 1 -- 172.21.15.42:0/1773119052 wait complete. 2024-05-28T14:10:57.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.768+0000 7f80a6a00640 1 Processor -- start 2024-05-28T14:10:57.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.768+0000 7f80a6a00640 1 -- start start 2024-05-28T14:10:57.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80a0071de0 0x7f80a00b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a6a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a00ce3b0 0x7f80a006e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a013fd10 0x7f80a006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a6a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f80a0169830 con 0x7f80a013fd10 2024-05-28T14:10:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a6a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01696b0 con 0x7f80a0071de0 2024-05-28T14:10:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a6a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80a01699b0 con 0x7f80a00ce3b0 2024-05-28T14:10:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a4a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a013fd10 0x7f80a006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a4a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a013fd10 0x7f80a006ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52544/0 (socket says 172.21.15.42:52544) 2024-05-28T14:10:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a4a00640 1 -- 172.21.15.42:0/2239782375 learned_addr learned my addr 172.21.15.42:0/2239782375 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f8097400640 1 --2- 172.21.15.42:0/2239782375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a00ce3b0 0x7f80a006e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f809fe00640 1 --2- 172.21.15.42:0/2239782375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80a0071de0 0x7f80a00b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a4a00640 1 -- 172.21.15.42:0/2239782375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a00ce3b0 msgr2=0x7f80a006e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a4a00640 1 --2- 172.21.15.42:0/2239782375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a00ce3b0 0x7f80a006e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a4a00640 1 -- 172.21.15.42:0/2239782375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80a0071de0 msgr2=0x7f80a00b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a4a00640 1 --2- 172.21.15.42:0/2239782375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80a0071de0 0x7f80a00b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a4a00640 1 -- 172.21.15.42:0/2239782375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a00b3690 con 0x7f80a013fd10 2024-05-28T14:10:57.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a4a00640 1 --2- 172.21.15.42:0/2239782375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a013fd10 0x7f80a006ed40 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f809800c7f0 tx=0x7f809800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:57.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f809cc00640 1 -- 172.21.15.42:0/2239782375 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098015070 con 0x7f80a013fd10 2024-05-28T14:10:57.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f809cc00640 1 -- 172.21.15.42:0/2239782375 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8098007860 con 0x7f80a013fd10 2024-05-28T14:10:57.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f809cc00640 1 -- 172.21.15.42:0/2239782375 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098007c50 con 0x7f80a013fd10 2024-05-28T14:10:57.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a6a00640 1 -- 172.21.15.42:0/2239782375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a00b3930 con 0x7f80a013fd10 2024-05-28T14:10:57.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a6a00640 1 -- 172.21.15.42:0/2239782375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a0138230 con 0x7f80a013fd10 2024-05-28T14:10:57.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.772+0000 7f80a6a00640 1 -- 172.21.15.42:0/2239782375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a00b3f40 con 0x7f80a013fd10 2024-05-28T14:10:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.776+0000 7f809cc00640 1 -- 172.21.15.42:0/2239782375 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 34) ==== 89772+0+0 (secure 0 0 0) 0x7f8098018070 con 0x7f80a013fd10 2024-05-28T14:10:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.776+0000 7f809cc00640 1 --2- 172.21.15.42:0/2239782375 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f807c069730 0x7f807c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:57.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.776+0000 7f809cc00640 1 -- 172.21.15.42:0/2239782375 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7f8098085ae0 con 0x7f80a013fd10 2024-05-28T14:10:57.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.776+0000 7f809fe00640 1 --2- 172.21.15.42:0/2239782375 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f807c069730 0x7f807c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:57.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.776+0000 7f809fe00640 1 --2- 172.21.15.42:0/2239782375 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f807c069730 0x7f807c06bbf0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f80900ae000 tx=0x7f80900a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:57.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.780+0000 7f809cc00640 1 -- 172.21.15.42:0/2239782375 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f80980570f0 con 0x7f80a013fd10 2024-05-28T14:10:57.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.944+0000 7f80a6a00640 1 -- 172.21.15.42:0/2239782375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f80a00738b0 con 0x7f80a013fd10 2024-05-28T14:10:57.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.944+0000 7f809cc00640 1 -- 172.21.15.42:0/2239782375 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f809805b3f0 con 0x7f80a013fd10 2024-05-28T14:10:57.930 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:10:57.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.948+0000 7f80a6a00640 1 -- 172.21.15.42:0/2239782375 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f807c069730 msgr2=0x7f807c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.948+0000 7f80a6a00640 1 --2- 172.21.15.42:0/2239782375 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f807c069730 0x7f807c06bbf0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f80900ae000 tx=0x7f80900a7070 comp rx=0 tx=0).stop 2024-05-28T14:10:57.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.948+0000 7f80a6a00640 1 -- 172.21.15.42:0/2239782375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a013fd10 msgr2=0x7f80a006ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:57.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.948+0000 7f80a6a00640 1 --2- 172.21.15.42:0/2239782375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a013fd10 0x7f80a006ed40 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f809800c7f0 tx=0x7f809800ccb0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.948+0000 7f80a6a00640 1 -- 172.21.15.42:0/2239782375 shutdown_connections 2024-05-28T14:10:57.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.948+0000 7f80a6a00640 1 --2- 172.21.15.42:0/2239782375 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f807c069730 0x7f807c06bbf0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.948+0000 7f80a6a00640 1 --2- 172.21.15.42:0/2239782375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a013fd10 0x7f80a006ed40 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.948+0000 7f80a6a00640 1 --2- 172.21.15.42:0/2239782375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a00ce3b0 0x7f80a006e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.948+0000 7f80a6a00640 1 --2- 172.21.15.42:0/2239782375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80a0071de0 0x7f80a00b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:57.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.948+0000 7f80a6a00640 1 -- 172.21.15.42:0/2239782375 >> 172.21.15.42:0/2239782375 conn(0x7f80a00b21f0 msgr2=0x7f80a006f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:57.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.948+0000 7f80a6a00640 1 -- 172.21.15.42:0/2239782375 shutdown_connections 2024-05-28T14:10:57.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:57.948+0000 7f80a6a00640 1 -- 172.21.15.42:0/2239782375 wait complete. 2024-05-28T14:10:57.951 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:10:58.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f839ea00640 1 Processor -- start 2024-05-28T14:10:58.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f839ea00640 1 -- start start 2024-05-28T14:10:58.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f839ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83980b2240 0x7f839806b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:58.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f839ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f839806bf30 0x7f83980af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:58.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f839ea00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83980b4440 con 0x7f839815e8c0 2024-05-28T14:10:58.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f839ea00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f83980ae420 con 0x7f839806bf30 2024-05-28T14:10:58.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f839ea00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f83980b0f10 con 0x7f83980b2240 2024-05-28T14:10:58.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f839ca00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f839806bf30 0x7f83980af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:58.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f839ca00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f839806bf30 0x7f83980af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52550/0 (socket says 172.21.15.42:52550) 2024-05-28T14:10:58.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f839ca00640 1 -- 172.21.15.42:0/1647025503 learned_addr learned my addr 172.21.15.42:0/1647025503 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:58.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f838fe00640 1 --2- 172.21.15.42:0/1647025503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83980b2240 0x7f839806b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:58.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f839ca00640 1 -- 172.21.15.42:0/1647025503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83980b2240 msgr2=0x7f839806b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:58.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f839ca00640 1 --2- 172.21.15.42:0/1647025503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83980b2240 0x7f839806b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f839ca00640 1 -- 172.21.15.42:0/1647025503 >> v1:172.21.15.63:6789/0 conn(0x7f839815e8c0 legacy=0x7f83980b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:58.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.116+0000 7f839ca00640 1 -- 172.21.15.42:0/1647025503 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83980ae670 con 0x7f839806bf30 2024-05-28T14:10:58.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.120+0000 7f839ca00640 1 --2- 172.21.15.42:0/1647025503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f839806bf30 0x7f83980af150 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f8390009e50 tx=0x7f839001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec22a4347c0e1907 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:58.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.120+0000 7f8397400640 1 -- 172.21.15.42:0/1647025503 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83900a5660 con 0x7f839806bf30 2024-05-28T14:10:58.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.120+0000 7f8397400640 1 -- 172.21.15.42:0/1647025503 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83900a5800 con 0x7f839806bf30 2024-05-28T14:10:58.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.120+0000 7f8397400640 1 -- 172.21.15.42:0/1647025503 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83900a5bd0 con 0x7f839806bf30 2024-05-28T14:10:58.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.120+0000 7f839ea00640 1 -- 172.21.15.42:0/1647025503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f839806bf30 msgr2=0x7f83980af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:58.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.120+0000 7f839ea00640 1 --2- 172.21.15.42:0/1647025503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f839806bf30 0x7f83980af150 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f8390009e50 tx=0x7f839001c9a0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.120+0000 7f839ea00640 1 -- 172.21.15.42:0/1647025503 shutdown_connections 2024-05-28T14:10:58.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.120+0000 7f839ea00640 1 --2- 172.21.15.42:0/1647025503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f839806bf30 0x7f83980af150 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.120+0000 7f839ea00640 1 --2- 172.21.15.42:0/1647025503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83980b2240 0x7f839806b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.120+0000 7f839ea00640 1 -- 172.21.15.42:0/1647025503 >> 172.21.15.42:0/1647025503 conn(0x7f83980b6960 msgr2=0x7f83980b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:58.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.120+0000 7f839ea00640 1 -- 172.21.15.42:0/1647025503 shutdown_connections 2024-05-28T14:10:58.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.120+0000 7f839ea00640 1 -- 172.21.15.42:0/1647025503 wait complete. 2024-05-28T14:10:58.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.120+0000 7f839ea00640 1 Processor -- start 2024-05-28T14:10:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.120+0000 7f839ea00640 1 -- start start 2024-05-28T14:10:58.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f839ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f839806bf30 0x7f839814dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:58.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f839ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83980b2240 0x7f8398151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f839ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815e8c0 0x7f839814fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f839ea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f83980afea0 con 0x7f83980b2240 2024-05-28T14:10:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f839ea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83980b2620 con 0x7f839806bf30 2024-05-28T14:10:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f839ea00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f83980b0020 con 0x7f839815e8c0 2024-05-28T14:10:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f838fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83980b2240 0x7f8398151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f838fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83980b2240 0x7f8398151e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52558/0 (socket says 172.21.15.42:52558) 2024-05-28T14:10:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f838fe00640 1 -- 172.21.15.42:0/1124576702 learned_addr learned my addr 172.21.15.42:0/1124576702 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f8397e00640 1 --2- 172.21.15.42:0/1124576702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f839806bf30 0x7f839814dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f839ca00640 1 --2- 172.21.15.42:0/1124576702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815e8c0 0x7f839814fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f838fe00640 1 -- 172.21.15.42:0/1124576702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815e8c0 msgr2=0x7f839814fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f838fe00640 1 --2- 172.21.15.42:0/1124576702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815e8c0 0x7f839814fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f838fe00640 1 -- 172.21.15.42:0/1124576702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f839806bf30 msgr2=0x7f839814dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f838fe00640 1 --2- 172.21.15.42:0/1124576702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f839806bf30 0x7f839814dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f838fe00640 1 -- 172.21.15.42:0/1124576702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83980cfc20 con 0x7f83980b2240 2024-05-28T14:10:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f838fe00640 1 --2- 172.21.15.42:0/1124576702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83980b2240 0x7f8398151e40 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f838000e8a0 tx=0x7f838000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f8394c00640 1 -- 172.21.15.42:0/1124576702 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380017070 con 0x7f83980b2240 2024-05-28T14:10:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f8394c00640 1 -- 172.21.15.42:0/1124576702 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838000c940 con 0x7f83980b2240 2024-05-28T14:10:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f8394c00640 1 -- 172.21.15.42:0/1124576702 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838000ccf0 con 0x7f83980b2240 2024-05-28T14:10:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f839ea00640 1 -- 172.21.15.42:0/1124576702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83980cff00 con 0x7f83980b2240 2024-05-28T14:10:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f839ea00640 1 -- 172.21.15.42:0/1124576702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83980d04c0 con 0x7f83980b2240 2024-05-28T14:10:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.124+0000 7f839ea00640 1 -- 172.21.15.42:0/1124576702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839809f7a0 con 0x7f83980b2240 2024-05-28T14:10:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.128+0000 7f8394c00640 1 -- 172.21.15.42:0/1124576702 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f838001a070 con 0x7f83980b2240 2024-05-28T14:10:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.128+0000 7f8394c00640 1 --2- 172.21.15.42:0/1124576702 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8374069850 0x7f837406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:58.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.128+0000 7f8394c00640 1 -- 172.21.15.42:0/1124576702 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7f8380085930 con 0x7f83980b2240 2024-05-28T14:10:58.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.128+0000 7f8397e00640 1 --2- 172.21.15.42:0/1124576702 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8374069850 0x7f837406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:58.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.128+0000 7f8397e00640 1 --2- 172.21.15.42:0/1124576702 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8374069850 0x7f837406bd10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8388001650 tx=0x7f8388009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:58.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.132+0000 7f8394c00640 1 -- 172.21.15.42:0/1124576702 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8380057940 con 0x7f83980b2240 2024-05-28T14:10:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.296+0000 7f839ea00640 1 -- 172.21.15.42:0/1124576702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f839804fdd0 con 0x7f83980b2240 2024-05-28T14:10:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.296+0000 7f8394c00640 1 -- 172.21.15.42:0/1124576702 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f8380057240 con 0x7f83980b2240 2024-05-28T14:10:58.281 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:10:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.296+0000 7f839ea00640 1 -- 172.21.15.42:0/1124576702 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8374069850 msgr2=0x7f837406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.296+0000 7f839ea00640 1 --2- 172.21.15.42:0/1124576702 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8374069850 0x7f837406bd10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8388001650 tx=0x7f8388009040 comp rx=0 tx=0).stop 2024-05-28T14:10:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.296+0000 7f839ea00640 1 -- 172.21.15.42:0/1124576702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83980b2240 msgr2=0x7f8398151e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.296+0000 7f839ea00640 1 --2- 172.21.15.42:0/1124576702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83980b2240 0x7f8398151e40 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f838000e8a0 tx=0x7f838000eca0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.300+0000 7f839ea00640 1 -- 172.21.15.42:0/1124576702 shutdown_connections 2024-05-28T14:10:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.300+0000 7f839ea00640 1 --2- 172.21.15.42:0/1124576702 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8374069850 0x7f837406bd10 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.300+0000 7f839ea00640 1 --2- 172.21.15.42:0/1124576702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815e8c0 0x7f839814fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.300+0000 7f839ea00640 1 --2- 172.21.15.42:0/1124576702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83980b2240 0x7f8398151e40 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.300+0000 7f839ea00640 1 --2- 172.21.15.42:0/1124576702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f839806bf30 0x7f839814dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.300+0000 7f839ea00640 1 -- 172.21.15.42:0/1124576702 >> 172.21.15.42:0/1124576702 conn(0x7f83980b6960 msgr2=0x7f839806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:58.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.300+0000 7f839ea00640 1 -- 172.21.15.42:0/1124576702 shutdown_connections 2024-05-28T14:10:58.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.300+0000 7f839ea00640 1 -- 172.21.15.42:0/1124576702 wait complete. 2024-05-28T14:10:58.302 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T14:10:58.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.460+0000 7f69c6e00640 1 Processor -- start 2024-05-28T14:10:58.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.460+0000 7f69c6e00640 1 -- start start 2024-05-28T14:10:58.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.460+0000 7f69c6e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69c00cf690 0x7f69c006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:58.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.460+0000 7f69c6e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69c00cfc10 con 0x7f69c00cf230 2024-05-28T14:10:58.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.460+0000 7f69c6e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69c00cfef0 con 0x7f69c015bf30 2024-05-28T14:10:58.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.460+0000 7f69c6e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f69c00721d0 con 0x7f69c00cf690 2024-05-28T14:10:58.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.460+0000 7f69b7e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f69c00cf230 0x7f69c0072660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42922/0 (socket says 172.21.15.42:42922) 2024-05-28T14:10:58.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.460+0000 7f69b7e00640 1 -- 172.21.15.42:0/3734792461 learned_addr learned my addr 172.21.15.42:0/3734792461 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:10:58.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.460+0000 7f69c4e00640 1 --2- 172.21.15.42:0/3734792461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69c00cf690 0x7f69c006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:58.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.460+0000 7f69bf400640 1 -- 172.21.15.42:0/3734792461 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2234507437 0 0) 0x7f69c00cfc10 con 0x7f69c00cf230 2024-05-28T14:10:58.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.464+0000 7f69bf400640 1 -- 172.21.15.42:0/3734792461 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69a4003530 con 0x7f69c00cf230 2024-05-28T14:10:58.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.464+0000 7f69c4e00640 1 -- 172.21.15.42:0/3734792461 >> v1:172.21.15.63:6789/0 conn(0x7f69c015bf30 legacy=0x7f69c015c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:58.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.464+0000 7f69bf400640 1 -- 172.21.15.42:0/3734792461 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2030571698 0 0) 0x7f69c00cfef0 con 0x7f69c015bf30 2024-05-28T14:10:58.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.464+0000 7f69c4e00640 1 -- 172.21.15.42:0/3734792461 >> v1:172.21.15.42:6789/0 conn(0x7f69c00cf230 legacy=0x7f69c0072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:58.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.464+0000 7f69c4e00640 1 -- 172.21.15.42:0/3734792461 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c00d01d0 con 0x7f69c00cf690 2024-05-28T14:10:58.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.464+0000 7f69c4e00640 1 --2- 172.21.15.42:0/3734792461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69c00cf690 0x7f69c006ec50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009dc0 tx=0x7f69b801c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60c41b6252b4bbd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:58.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.464+0000 7f69bf400640 1 -- 172.21.15.42:0/3734792461 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b80b2070 con 0x7f69c00cf690 2024-05-28T14:10:58.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.464+0000 7f69bf400640 1 -- 172.21.15.42:0/3734792461 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b80a57c0 con 0x7f69c00cf690 2024-05-28T14:10:58.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.464+0000 7f69bf400640 1 -- 172.21.15.42:0/3734792461 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b80a5b70 con 0x7f69c00cf690 2024-05-28T14:10:58.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.464+0000 7f69c6e00640 1 -- 172.21.15.42:0/3734792461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69c00cf690 msgr2=0x7f69c006ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:58.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.464+0000 7f69c6e00640 1 --2- 172.21.15.42:0/3734792461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69c00cf690 0x7f69c006ec50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009dc0 tx=0x7f69b801c900 comp rx=0 tx=0).stop 2024-05-28T14:10:58.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69c6e00640 1 -- 172.21.15.42:0/3734792461 shutdown_connections 2024-05-28T14:10:58.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69c6e00640 1 --2- 172.21.15.42:0/3734792461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69c00cf690 0x7f69c006ec50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69c6e00640 1 -- 172.21.15.42:0/3734792461 >> 172.21.15.42:0/3734792461 conn(0x7f69c006b9f0 msgr2=0x7f69c006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:58.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69c6e00640 1 -- 172.21.15.42:0/3734792461 shutdown_connections 2024-05-28T14:10:58.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69c6e00640 1 -- 172.21.15.42:0/3734792461 wait complete. 2024-05-28T14:10:58.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69c6e00640 1 Processor -- start 2024-05-28T14:10:58.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69c6e00640 1 -- start start 2024-05-28T14:10:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69c6e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69c00cf230 0x7f69c01f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69c6e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c00cf690 0x7f69c014bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69c6e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69c015bf30 0x7f69c014e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69c6e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69c0169550 con 0x7f69c00cf230 2024-05-28T14:10:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69c6e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01693d0 con 0x7f69c00cf690 2024-05-28T14:10:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69c6e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f69c01696d0 con 0x7f69c015bf30 2024-05-28T14:10:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69bfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69c00cf230 0x7f69c01f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69bfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69c00cf230 0x7f69c01f7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52570/0 (socket says 172.21.15.42:52570) 2024-05-28T14:10:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69bfe00640 1 -- 172.21.15.42:0/2726086289 learned_addr learned my addr 172.21.15.42:0/2726086289 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:58.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69b7e00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c00cf690 0x7f69c014bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:58.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69c4e00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69c015bf30 0x7f69c014e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:58.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69bfe00640 1 -- 172.21.15.42:0/2726086289 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69c015bf30 msgr2=0x7f69c014e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:58.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69bfe00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69c015bf30 0x7f69c014e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69bfe00640 1 -- 172.21.15.42:0/2726086289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c00cf690 msgr2=0x7f69c014bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:58.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69bfe00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c00cf690 0x7f69c014bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69bfe00640 1 -- 172.21.15.42:0/2726086289 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c01f7d20 con 0x7f69c00cf230 2024-05-28T14:10:58.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69b7e00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c00cf690 0x7f69c014bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:10:58.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.468+0000 7f69bfe00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69c00cf230 0x7f69c01f7360 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f69b000c4d0 tx=0x7f69b000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:58.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.472+0000 7f69bcc00640 1 -- 172.21.15.42:0/2726086289 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0017070 con 0x7f69c00cf230 2024-05-28T14:10:58.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.472+0000 7f69bcc00640 1 -- 172.21.15.42:0/2726086289 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b0005480 con 0x7f69c00cf230 2024-05-28T14:10:58.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.472+0000 7f69bcc00640 1 -- 172.21.15.42:0/2726086289 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0003840 con 0x7f69c00cf230 2024-05-28T14:10:58.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.472+0000 7f69c6e00640 1 -- 172.21.15.42:0/2726086289 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c01f8000 con 0x7f69c00cf230 2024-05-28T14:10:58.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.472+0000 7f69c6e00640 1 -- 172.21.15.42:0/2726086289 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c014b5e0 con 0x7f69c00cf230 2024-05-28T14:10:58.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.472+0000 7f69c6e00640 1 -- 172.21.15.42:0/2726086289 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c0073890 con 0x7f69c00cf230 2024-05-28T14:10:58.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.472+0000 7f69bcc00640 1 -- 172.21.15.42:0/2726086289 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f69b001a070 con 0x7f69c00cf230 2024-05-28T14:10:58.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.472+0000 7f69bcc00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f699c069850 0x7f699c06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:58.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.472+0000 7f69bcc00640 1 -- 172.21.15.42:0/2726086289 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7f69b0084af0 con 0x7f69c00cf230 2024-05-28T14:10:58.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.472+0000 7f69b7e00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f699c069850 0x7f699c06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:58.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.476+0000 7f69b7e00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f699c069850 0x7f699c06bd10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f69c0150070 tx=0x7f69a8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:58.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.476+0000 7f69bcc00640 1 -- 172.21.15.42:0/2726086289 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f69b0056bc0 con 0x7f69c00cf230 2024-05-28T14:10:58.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.652+0000 7f69c6e00640 1 -- 172.21.15.42:0/2726086289 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f69c004fdd0 con 0x7f69c00cf230 2024-05-28T14:10:58.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.652+0000 7f69bcc00640 1 -- 172.21.15.42:0/2726086289 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v93) ==== 80+0+0 (secure 0 0 0) 0x7f69b00564c0 con 0x7f69c00cf230 2024-05-28T14:10:58.638 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T14:10:58.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.656+0000 7f69c6e00640 1 -- 172.21.15.42:0/2726086289 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f699c069850 msgr2=0x7f699c06bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:58.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.656+0000 7f69c6e00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f699c069850 0x7f699c06bd10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f69c0150070 tx=0x7f69a8009040 comp rx=0 tx=0).stop 2024-05-28T14:10:58.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.656+0000 7f69c6e00640 1 -- 172.21.15.42:0/2726086289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69c00cf230 msgr2=0x7f69c01f7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:58.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.656+0000 7f69c6e00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69c00cf230 0x7f69c01f7360 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f69b000c4d0 tx=0x7f69b000c990 comp rx=0 tx=0).stop 2024-05-28T14:10:58.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.656+0000 7f69c6e00640 1 -- 172.21.15.42:0/2726086289 shutdown_connections 2024-05-28T14:10:58.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.656+0000 7f69c6e00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f699c069850 0x7f699c06bd10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.656+0000 7f69c6e00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69c015bf30 0x7f69c014e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.656+0000 7f69c6e00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c00cf690 0x7f69c014bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.656+0000 7f69c6e00640 1 --2- 172.21.15.42:0/2726086289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69c00cf230 0x7f69c01f7360 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:58.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.656+0000 7f69c6e00640 1 -- 172.21.15.42:0/2726086289 >> 172.21.15.42:0/2726086289 conn(0x7f69c006b9f0 msgr2=0x7f69c006bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:58.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.656+0000 7f69c6e00640 1 -- 172.21.15.42:0/2726086289 shutdown_connections 2024-05-28T14:10:58.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:58.656+0000 7f69c6e00640 1 -- 172.21.15.42:0/2726086289 wait complete. 2024-05-28T14:10:58.660 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-05-28T14:10:58.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:10:58.660 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-28T14:10:58.678 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[48091]: fuse finished with error 0 and tester_r 0 2024-05-28T14:10:58.681 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:10:58.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:10:58.681 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-28T14:10:58.852 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:10:58.852 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:10:58.852 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:10:58.867 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:10:58.867 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:10:58.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:10:58.867 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-28T14:10:58.920 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-28T14:10:58.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:10:58.921 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:10:58.921 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:10:58.921 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:10:58.937 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:10:58.937 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:10:58.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:10:58.938 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-28T14:10:58.996 INFO:teuthology.orchestra.run.smithi042.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-28T14:10:58.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:10:58.998 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:10:58.998 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:10:58.998 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:10:59.012 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:10:59.012 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:10:59.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:10:59.013 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-28T14:10:59.069 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-28T14:10:59.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:10:59.071 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:10:59.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.328+0000 7f5e97e00640 1 Processor -- start 2024-05-28T14:10:59.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.328+0000 7f5e97e00640 1 -- start start 2024-05-28T14:10:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.328+0000 7f5e97e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e900b51b0 0x7f5e900d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.328+0000 7f5e97e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e900d0c30 0x7f5e900ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.328+0000 7f5e97e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e900b39f0 con 0x7f5e9015e5c0 2024-05-28T14:10:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.328+0000 7f5e97e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5e900b3cd0 con 0x7f5e900d0c30 2024-05-28T14:10:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.328+0000 7f5e97e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5e900b3fb0 con 0x7f5e900b51b0 2024-05-28T14:10:59.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.332+0000 7f5e95e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e900d0c30 0x7f5e900ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:59.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.332+0000 7f5e95e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e900d0c30 0x7f5e900ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52580/0 (socket says 172.21.15.42:52580) 2024-05-28T14:10:59.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.332+0000 7f5e95e00640 1 -- 172.21.15.42:0/780190502 learned_addr learned my addr 172.21.15.42:0/780190502 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:59.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.332+0000 7f5e94a00640 1 --2- 172.21.15.42:0/780190502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e900b51b0 0x7f5e900d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:59.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.332+0000 7f5e95e00640 1 -- 172.21.15.42:0/780190502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e900b51b0 msgr2=0x7f5e900d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:59.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.332+0000 7f5e95e00640 1 --2- 172.21.15.42:0/780190502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e900b51b0 0x7f5e900d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.332+0000 7f5e95e00640 1 -- 172.21.15.42:0/780190502 >> v1:172.21.15.63:6789/0 conn(0x7f5e9015e5c0 legacy=0x7f5e900b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:59.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.332+0000 7f5e95e00640 1 -- 172.21.15.42:0/780190502 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e900ad340 con 0x7f5e900d0c30 2024-05-28T14:10:59.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.336+0000 7f5e95e00640 1 --2- 172.21.15.42:0/780190502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e900d0c30 0x7f5e900ad860 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009e50 tx=0x7f5e8801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad61fcd3c2b60eac server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:59.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.336+0000 7f5e8fe00640 1 -- 172.21.15.42:0/780190502 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e880a5670 con 0x7f5e900d0c30 2024-05-28T14:10:59.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.336+0000 7f5e8fe00640 1 -- 172.21.15.42:0/780190502 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e880a5810 con 0x7f5e900d0c30 2024-05-28T14:10:59.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.336+0000 7f5e8fe00640 1 -- 172.21.15.42:0/780190502 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e880a5be0 con 0x7f5e900d0c30 2024-05-28T14:10:59.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.336+0000 7f5e97e00640 1 -- 172.21.15.42:0/780190502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e900d0c30 msgr2=0x7f5e900ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:59.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.336+0000 7f5e97e00640 1 --2- 172.21.15.42:0/780190502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e900d0c30 0x7f5e900ad860 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009e50 tx=0x7f5e8801c9f0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.336+0000 7f5e97e00640 1 -- 172.21.15.42:0/780190502 shutdown_connections 2024-05-28T14:10:59.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.336+0000 7f5e97e00640 1 --2- 172.21.15.42:0/780190502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e900d0c30 0x7f5e900ad860 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.336+0000 7f5e97e00640 1 --2- 172.21.15.42:0/780190502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e900b51b0 0x7f5e900d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.336+0000 7f5e97e00640 1 -- 172.21.15.42:0/780190502 >> 172.21.15.42:0/780190502 conn(0x7f5e9006bdf0 msgr2=0x7f5e9006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:59.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.340+0000 7f5e97e00640 1 -- 172.21.15.42:0/780190502 shutdown_connections 2024-05-28T14:10:59.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.340+0000 7f5e97e00640 1 -- 172.21.15.42:0/780190502 wait complete. 2024-05-28T14:10:59.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.340+0000 7f5e97e00640 1 Processor -- start 2024-05-28T14:10:59.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.344+0000 7f5e97e00640 1 -- start start 2024-05-28T14:10:59.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.344+0000 7f5e97e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e900b51b0 0x7f5e900c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:59.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.344+0000 7f5e97e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e900d0c30 0x7f5e9009a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:59.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.344+0000 7f5e97e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e9015e5c0 0x7f5e9009a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:59.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.344+0000 7f5e97e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e900ae9c0 con 0x7f5e900d0c30 2024-05-28T14:10:59.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.344+0000 7f5e97e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e900ae840 con 0x7f5e9015e5c0 2024-05-28T14:10:59.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.344+0000 7f5e97e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5e900aeb40 con 0x7f5e900b51b0 2024-05-28T14:10:59.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.344+0000 7f5e94a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e900d0c30 0x7f5e9009a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:59.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.344+0000 7f5e94a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e900d0c30 0x7f5e9009a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52586/0 (socket says 172.21.15.42:52586) 2024-05-28T14:10:59.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.344+0000 7f5e94a00640 1 -- 172.21.15.42:0/270591697 learned_addr learned my addr 172.21.15.42:0/270591697 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.344+0000 7f5e95400640 1 --2- 172.21.15.42:0/270591697 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e900b51b0 0x7f5e900c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.344+0000 7f5e95e00640 1 --2- 172.21.15.42:0/270591697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e9015e5c0 0x7f5e9009a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.348+0000 7f5e95400640 1 -- 172.21.15.42:0/270591697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e9015e5c0 msgr2=0x7f5e9009a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.348+0000 7f5e95400640 1 --2- 172.21.15.42:0/270591697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e9015e5c0 0x7f5e9009a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.348+0000 7f5e95400640 1 -- 172.21.15.42:0/270591697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e900d0c30 msgr2=0x7f5e9009a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.348+0000 7f5e95400640 1 --2- 172.21.15.42:0/270591697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e900d0c30 0x7f5e9009a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.348+0000 7f5e95400640 1 -- 172.21.15.42:0/270591697 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e900ca3b0 con 0x7f5e900b51b0 2024-05-28T14:10:59.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.348+0000 7f5e95400640 1 --2- 172.21.15.42:0/270591697 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e900b51b0 0x7f5e900c9560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400b440 tx=0x7f5e8400b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:59.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.348+0000 7f5e8d600640 1 -- 172.21.15.42:0/270591697 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84011830 con 0x7f5e900b51b0 2024-05-28T14:10:59.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.348+0000 7f5e8d600640 1 -- 172.21.15.42:0/270591697 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e84016070 con 0x7f5e900b51b0 2024-05-28T14:10:59.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.348+0000 7f5e8d600640 1 -- 172.21.15.42:0/270591697 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84011c10 con 0x7f5e900b51b0 2024-05-28T14:10:59.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.348+0000 7f5e97e00640 1 -- 172.21.15.42:0/270591697 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9009b380 con 0x7f5e900b51b0 2024-05-28T14:10:59.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.348+0000 7f5e97e00640 1 -- 172.21.15.42:0/270591697 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e9009bc20 con 0x7f5e900b51b0 2024-05-28T14:10:59.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.348+0000 7f5e97e00640 1 -- 172.21.15.42:0/270591697 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e90068a90 con 0x7f5e900b51b0 2024-05-28T14:10:59.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.352+0000 7f5e8d600640 1 -- 172.21.15.42:0/270591697 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f5e8401b460 con 0x7f5e900b51b0 2024-05-28T14:10:59.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.352+0000 7f5e8d600640 1 --2- 172.21.15.42:0/270591697 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e70069850 0x7f5e7006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:59.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.352+0000 7f5e8d600640 1 -- 172.21.15.42:0/270591697 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7f5e84085570 con 0x7f5e900b51b0 2024-05-28T14:10:59.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.352+0000 7f5e94a00640 1 --2- 172.21.15.42:0/270591697 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e70069850 0x7f5e7006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:59.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.356+0000 7f5e94a00640 1 --2- 172.21.15.42:0/270591697 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e70069850 0x7f5e7006bd10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5e900c9d40 tx=0x7f5e7c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:59.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.360+0000 7f5e8d600640 1 -- 172.21.15.42:0/270591697 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5e84056b50 con 0x7f5e900b51b0 2024-05-28T14:10:59.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.688+0000 7f5e97e00640 1 -- 172.21.15.42:0/270591697 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e900b3370 con 0x7f5e900b51b0 2024-05-28T14:10:59.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.692+0000 7f5e8d600640 1 -- 172.21.15.42:0/270591697 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5182 (secure 0 0 0) 0x7f5e8405ae50 con 0x7f5e900b51b0 2024-05-28T14:10:59.675 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:10:59.676 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7657,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/402076081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":402076081},{"type":"v1","addr":"172.21.15.63:6835","nonce":402076081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7998,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3779622485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3779622485},{"type":"v1","addr":"172.21.15.42:6835","nonce":3779622485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2961573665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2961573665},{"type":"v1","addr":"172.21.15.42:6837","nonce":2961573665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:09:31.146587+0000","modified":"2024-05-28T14:09:34.152047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":56621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56621":{"gid":56621,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/615628755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":615628755},{"type":"v1","addr":"172.21.15.63:6834","nonce":615628755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":56621,"qdb_cluster":[56621]},"id":6}]} 2024-05-28T14:10:59.676 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 55 2024-05-28T14:10:59.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.692+0000 7f5e97e00640 1 -- 172.21.15.42:0/270591697 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e70069850 msgr2=0x7f5e7006bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:59.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.692+0000 7f5e97e00640 1 --2- 172.21.15.42:0/270591697 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e70069850 0x7f5e7006bd10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5e900c9d40 tx=0x7f5e7c008040 comp rx=0 tx=0).stop 2024-05-28T14:10:59.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.692+0000 7f5e97e00640 1 -- 172.21.15.42:0/270591697 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e900b51b0 msgr2=0x7f5e900c9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:59.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.692+0000 7f5e97e00640 1 --2- 172.21.15.42:0/270591697 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e900b51b0 0x7f5e900c9560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400b440 tx=0x7f5e8400b900 comp rx=0 tx=0).stop 2024-05-28T14:10:59.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.692+0000 7f5e97e00640 1 -- 172.21.15.42:0/270591697 shutdown_connections 2024-05-28T14:10:59.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.692+0000 7f5e97e00640 1 --2- 172.21.15.42:0/270591697 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e70069850 0x7f5e7006bd10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.692+0000 7f5e97e00640 1 --2- 172.21.15.42:0/270591697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e9015e5c0 0x7f5e9009a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.692+0000 7f5e97e00640 1 --2- 172.21.15.42:0/270591697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e900d0c30 0x7f5e9009a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.692+0000 7f5e97e00640 1 --2- 172.21.15.42:0/270591697 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e900b51b0 0x7f5e900c9560 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.692+0000 7f5e97e00640 1 -- 172.21.15.42:0/270591697 >> 172.21.15.42:0/270591697 conn(0x7f5e9006bdf0 msgr2=0x7f5e9015ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:59.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.692+0000 7f5e97e00640 1 -- 172.21.15.42:0/270591697 shutdown_connections 2024-05-28T14:10:59.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.692+0000 7f5e97e00640 1 -- 172.21.15.42:0/270591697 wait complete. 2024-05-28T14:10:59.697 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:10:59.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.856+0000 7fa370a00640 1 Processor -- start 2024-05-28T14:10:59.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.856+0000 7fa370a00640 1 -- start start 2024-05-28T14:10:59.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.856+0000 7fa370a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa36c06f660 0x7fa36c073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:59.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.856+0000 7fa370a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa36c15ba40 0x7fa36c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:59.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.856+0000 7fa370a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa36c15f280 con 0x7fa36c072dd0 2024-05-28T14:10:59.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.856+0000 7fa370a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa36c15f030 con 0x7fa36c15ba40 2024-05-28T14:10:59.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.856+0000 7fa370a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa36c15f560 con 0x7fa36c06f660 2024-05-28T14:10:59.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.856+0000 7fa363e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa36c15ba40 0x7fa36c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:59.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.856+0000 7fa363e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa36c15ba40 0x7fa36c0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52588/0 (socket says 172.21.15.42:52588) 2024-05-28T14:10:59.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.856+0000 7fa363e00640 1 -- 172.21.15.42:0/3566099895 learned_addr learned my addr 172.21.15.42:0/3566099895 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:59.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.856+0000 7fa369e00640 1 --2- 172.21.15.42:0/3566099895 >> v2:172.21.15.63:3300/0 conn(0x7fa36c06f660 0x7fa36c073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:59.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.860+0000 7fa363e00640 1 -- 172.21.15.42:0/3566099895 >> v2:172.21.15.63:3300/0 conn(0x7fa36c06f660 msgr2=0x7fa36c073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:10:59.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.860+0000 7fa363e00640 1 --2- 172.21.15.42:0/3566099895 >> v2:172.21.15.63:3300/0 conn(0x7fa36c06f660 0x7fa36c073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.860+0000 7fa363e00640 1 -- 172.21.15.42:0/3566099895 >> v1:172.21.15.63:6789/0 conn(0x7fa36c072dd0 legacy=0x7fa36c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:59.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.860+0000 7fa363e00640 1 -- 172.21.15.42:0/3566099895 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36c15f840 con 0x7fa36c15ba40 2024-05-28T14:10:59.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.860+0000 7fa363e00640 1 --2- 172.21.15.42:0/3566099895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa36c15ba40 0x7fa36c0d06f0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0d13a0 tx=0x7fa35401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107878956887ae43 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:59.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.860+0000 7fa369400640 1 -- 172.21.15.42:0/3566099895 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3540b0070 con 0x7fa36c15ba40 2024-05-28T14:10:59.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.860+0000 7fa369400640 1 -- 172.21.15.42:0/3566099895 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3540ab600 con 0x7fa36c15ba40 2024-05-28T14:10:59.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.860+0000 7fa369400640 1 -- 172.21.15.42:0/3566099895 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3540ab9d0 con 0x7fa36c15ba40 2024-05-28T14:10:59.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.860+0000 7fa370a00640 1 -- 172.21.15.42:0/3566099895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa36c15ba40 msgr2=0x7fa36c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:59.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.860+0000 7fa370a00640 1 --2- 172.21.15.42:0/3566099895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa36c15ba40 0x7fa36c0d06f0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0d13a0 tx=0x7fa35401c3d0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.860+0000 7fa370a00640 1 -- 172.21.15.42:0/3566099895 shutdown_connections 2024-05-28T14:10:59.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.860+0000 7fa370a00640 1 --2- 172.21.15.42:0/3566099895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa36c15ba40 0x7fa36c0d06f0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.860+0000 7fa370a00640 1 --2- 172.21.15.42:0/3566099895 >> v2:172.21.15.63:3300/0 conn(0x7fa36c06f660 0x7fa36c073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.860+0000 7fa370a00640 1 -- 172.21.15.42:0/3566099895 >> 172.21.15.42:0/3566099895 conn(0x7fa36c06bdf0 msgr2=0x7fa36c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:10:59.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa370a00640 1 -- 172.21.15.42:0/3566099895 shutdown_connections 2024-05-28T14:10:59.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa370a00640 1 -- 172.21.15.42:0/3566099895 wait complete. 2024-05-28T14:10:59.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa370a00640 1 Processor -- start 2024-05-28T14:10:59.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa370a00640 1 -- start start 2024-05-28T14:10:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa370a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa36c06f660 0x7fa36c153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa370a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa36c072dd0 0x7fa36c148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa370a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa36c14d000 0x7fa36c1526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa370a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c166540 con 0x7fa36c14d000 2024-05-28T14:10:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa370a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1663c0 con 0x7fa36c06f660 2024-05-28T14:10:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa370a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa36c1666c0 con 0x7fa36c072dd0 2024-05-28T14:10:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa36a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa36c14d000 0x7fa36c1526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa36a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa36c14d000 0x7fa36c1526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52602/0 (socket says 172.21.15.42:52602) 2024-05-28T14:10:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa36a800640 1 -- 172.21.15.42:0/2354300462 learned_addr learned my addr 172.21.15.42:0/2354300462 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:10:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa369e00640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa36c06f660 0x7fa36c153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa363e00640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa36c072dd0 0x7fa36c148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa36a800640 1 -- 172.21.15.42:0/2354300462 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa36c072dd0 msgr2=0x7fa36c148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:59.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa36a800640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa36c072dd0 0x7fa36c148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa36a800640 1 -- 172.21.15.42:0/2354300462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa36c06f660 msgr2=0x7fa36c153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:10:59.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa36a800640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa36c06f660 0x7fa36c153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:10:59.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa36a800640 1 -- 172.21.15.42:0/2354300462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36c155940 con 0x7fa36c14d000 2024-05-28T14:10:59.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa369e00640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa36c06f660 0x7fa36c153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:10:59.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa363e00640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa36c072dd0 0x7fa36c148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:10:59.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.864+0000 7fa36a800640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa36c14d000 0x7fa36c1526e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fa36400c7a0 tx=0x7fa36400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:59.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.868+0000 7fa362000640 1 -- 172.21.15.42:0/2354300462 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364003b80 con 0x7fa36c14d000 2024-05-28T14:10:59.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.868+0000 7fa362000640 1 -- 172.21.15.42:0/2354300462 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa364003d20 con 0x7fa36c14d000 2024-05-28T14:10:59.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.868+0000 7fa362000640 1 -- 172.21.15.42:0/2354300462 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36400aa90 con 0x7fa36c14d000 2024-05-28T14:10:59.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.868+0000 7fa370a00640 1 -- 172.21.15.42:0/2354300462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36c155c20 con 0x7fa36c14d000 2024-05-28T14:10:59.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.868+0000 7fa370a00640 1 -- 172.21.15.42:0/2354300462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa36c14ca30 con 0x7fa36c14d000 2024-05-28T14:10:59.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.868+0000 7fa370a00640 1 -- 172.21.15.42:0/2354300462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa36c068a90 con 0x7fa36c14d000 2024-05-28T14:10:59.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.868+0000 7fa362000640 1 -- 172.21.15.42:0/2354300462 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fa36400ac30 con 0x7fa36c14d000 2024-05-28T14:10:59.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.868+0000 7fa362000640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa348069850 0x7fa34806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:10:59.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.868+0000 7fa362000640 1 -- 172.21.15.42:0/2354300462 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7fa3640859b0 con 0x7fa36c14d000 2024-05-28T14:10:59.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.868+0000 7fa369e00640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa348069850 0x7fa34806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:10:59.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.872+0000 7fa369e00640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa348069850 0x7fa34806bd10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0cc610 tx=0x7fa35c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:10:59.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:10:59.872+0000 7fa362000640 1 -- 172.21.15.42:0/2354300462 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa3640575e0 con 0x7fa36c14d000 2024-05-28T14:11:00.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.040+0000 7fa370a00640 1 -- 172.21.15.42:0/2354300462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa36c15adf0 con 0x7fa36c14d000 2024-05-28T14:11:00.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.040+0000 7fa362000640 1 -- 172.21.15.42:0/2354300462 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16852 (secure 0 0 0) 0x7fa364056ee0 con 0x7fa36c14d000 2024-05-28T14:11:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:00.026 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":93,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:10:56.060535+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:09:28.913704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-28T14:09:30.108327+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":93,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.3e","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:11:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.044+0000 7fa370a00640 1 -- 172.21.15.42:0/2354300462 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa348069850 msgr2=0x7fa34806bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.044+0000 7fa370a00640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa348069850 0x7fa34806bd10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0cc610 tx=0x7fa35c008040 comp rx=0 tx=0).stop 2024-05-28T14:11:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.044+0000 7fa370a00640 1 -- 172.21.15.42:0/2354300462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa36c14d000 msgr2=0x7fa36c1526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.044+0000 7fa370a00640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa36c14d000 0x7fa36c1526e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fa36400c7a0 tx=0x7fa36400cc60 comp rx=0 tx=0).stop 2024-05-28T14:11:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.044+0000 7fa370a00640 1 -- 172.21.15.42:0/2354300462 shutdown_connections 2024-05-28T14:11:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.044+0000 7fa370a00640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa348069850 0x7fa34806bd10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.044+0000 7fa370a00640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa36c14d000 0x7fa36c1526e0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.044+0000 7fa370a00640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa36c072dd0 0x7fa36c148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.044+0000 7fa370a00640 1 --2- 172.21.15.42:0/2354300462 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa36c06f660 0x7fa36c153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.044+0000 7fa370a00640 1 -- 172.21.15.42:0/2354300462 >> 172.21.15.42:0/2354300462 conn(0x7fa36c06bdf0 msgr2=0x7fa36c15e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.044+0000 7fa370a00640 1 -- 172.21.15.42:0/2354300462 shutdown_connections 2024-05-28T14:11:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.044+0000 7fa370a00640 1 -- 172.21.15.42:0/2354300462 wait complete. 2024-05-28T14:11:00.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T14:09:28.913704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T14:09:30.108327+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 93, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:11:00.049 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:11:00.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.208+0000 7fd1aaa00640 1 Processor -- start 2024-05-28T14:11:00.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.208+0000 7fd1aaa00640 1 -- start start 2024-05-28T14:11:00.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.212+0000 7fd1aaa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1a40ce3b0 0x7fd1a40ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.212+0000 7fd1aaa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd1a413fd10 0x7fd1a41401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.212+0000 7fd1aaa00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1a4160230 con 0x7fd1a4071de0 2024-05-28T14:11:00.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.212+0000 7fd1aaa00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd1a4160510 con 0x7fd1a413fd10 2024-05-28T14:11:00.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.212+0000 7fd1aaa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd1a41607f0 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.212+0000 7fd1a3e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd1a4071de0 0x7fd1a40721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42924/0 (socket says 172.21.15.42:42924) 2024-05-28T14:11:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.212+0000 7fd1a3e00640 1 -- 172.21.15.42:0/4116645960 learned_addr learned my addr 172.21.15.42:0/4116645960 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.212+0000 7fd1a3400640 1 --2- 172.21.15.42:0/4116645960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1a40ce3b0 0x7fd1a40ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.212+0000 7fd1a8a00640 1 --2- 172.21.15.42:0/4116645960 >> v2:172.21.15.63:3300/0 conn(0x7fd1a413fd10 0x7fd1a41401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.212+0000 7fd1a3400640 1 -- 172.21.15.42:0/4116645960 >> v2:172.21.15.63:3300/0 conn(0x7fd1a413fd10 msgr2=0x7fd1a41401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.212+0000 7fd1a3400640 1 --2- 172.21.15.42:0/4116645960 >> v2:172.21.15.63:3300/0 conn(0x7fd1a413fd10 0x7fd1a41401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.212+0000 7fd1a3400640 1 -- 172.21.15.42:0/4116645960 >> v1:172.21.15.42:6789/0 conn(0x7fd1a4071de0 legacy=0x7fd1a40721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.212+0000 7fd1a3400640 1 -- 172.21.15.42:0/4116645960 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a4160ad0 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.212+0000 7fd1a2a00640 1 -- 172.21.15.42:0/4116645960 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4050053184 0 0) 0x7fd1a4160230 con 0x7fd1a4071de0 2024-05-28T14:11:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1a3400640 1 --2- 172.21.15.42:0/4116645960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1a40ce3b0 0x7fd1a40ce810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd1900097c0 tx=0x7fd19001c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc505f76101b1df server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1a2a00640 1 -- 172.21.15.42:0/4116645960 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900b0070 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1a2a00640 1 -- 172.21.15.42:0/4116645960 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1900ab600 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1a2a00640 1 -- 172.21.15.42:0/4116645960 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900ab9d0 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 -- 172.21.15.42:0/4116645960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1a40ce3b0 msgr2=0x7fd1a40ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 --2- 172.21.15.42:0/4116645960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1a40ce3b0 0x7fd1a40ce810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd1900097c0 tx=0x7fd19001c380 comp rx=0 tx=0).stop 2024-05-28T14:11:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 -- 172.21.15.42:0/4116645960 shutdown_connections 2024-05-28T14:11:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 --2- 172.21.15.42:0/4116645960 >> v2:172.21.15.63:3300/0 conn(0x7fd1a413fd10 0x7fd1a41401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 --2- 172.21.15.42:0/4116645960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1a40ce3b0 0x7fd1a40ce810 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 -- 172.21.15.42:0/4116645960 >> 172.21.15.42:0/4116645960 conn(0x7fd1a40b21f0 msgr2=0x7fd1a40b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 -- 172.21.15.42:0/4116645960 shutdown_connections 2024-05-28T14:11:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 -- 172.21.15.42:0/4116645960 wait complete. 2024-05-28T14:11:00.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 Processor -- start 2024-05-28T14:11:00.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 -- start start 2024-05-28T14:11:00.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1a4071de0 0x7fd1a413d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1a40ce3b0 0x7fd1a415e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1a413fd10 0x7fd1a4155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a4169830 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a41696b0 con 0x7fd1a4071de0 2024-05-28T14:11:00.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1aaa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd1a41699b0 con 0x7fd1a413fd10 2024-05-28T14:11:00.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.216+0000 7fd1a3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1a40ce3b0 0x7fd1a415e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd1a3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1a40ce3b0 0x7fd1a415e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52604/0 (socket says 172.21.15.42:52604) 2024-05-28T14:11:00.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd1a3400640 1 -- 172.21.15.42:0/3081746271 learned_addr learned my addr 172.21.15.42:0/3081746271 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:00.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd1a8a00640 1 --2- 172.21.15.42:0/3081746271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1a413fd10 0x7fd1a4155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd1a3e00640 1 --2- 172.21.15.42:0/3081746271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1a4071de0 0x7fd1a413d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd1a3400640 1 -- 172.21.15.42:0/3081746271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1a413fd10 msgr2=0x7fd1a4155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd1a3400640 1 --2- 172.21.15.42:0/3081746271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1a413fd10 0x7fd1a4155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd1a3400640 1 -- 172.21.15.42:0/3081746271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1a4071de0 msgr2=0x7fd1a413d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd1a3400640 1 --2- 172.21.15.42:0/3081746271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1a4071de0 0x7fd1a413d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd1a3400640 1 -- 172.21.15.42:0/3081746271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a415efb0 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd1a3400640 1 --2- 172.21.15.42:0/3081746271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1a40ce3b0 0x7fd1a415e810 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fd190019fb0 tx=0x7fd19001abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:00.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd19be00640 1 -- 172.21.15.42:0/3081746271 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900b0050 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd19be00640 1 -- 172.21.15.42:0/3081746271 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1900bc070 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd1aaa00640 1 -- 172.21.15.42:0/3081746271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a415f290 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd19be00640 1 -- 172.21.15.42:0/3081746271 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900b5830 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd1aaa00640 1 -- 172.21.15.42:0/3081746271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a415f850 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd1aaa00640 1 -- 172.21.15.42:0/3081746271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a40738b0 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd19be00640 1 -- 172.21.15.42:0/3081746271 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fd1a4140ef0 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.220+0000 7fd19be00640 1 --2- 172.21.15.42:0/3081746271 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd184069850 0x7fd18406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.224+0000 7fd19be00640 1 -- 172.21.15.42:0/3081746271 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7fd19011fa10 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.224+0000 7fd1a3e00640 1 --2- 172.21.15.42:0/3081746271 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd184069850 0x7fd18406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.224+0000 7fd1a3e00640 1 --2- 172.21.15.42:0/3081746271 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd184069850 0x7fd18406bd10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd194000f70 tx=0x7fd194007df0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:00.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.224+0000 7fd19be00640 1 -- 172.21.15.42:0/3081746271 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd190016090 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.388+0000 7fd1aaa00640 1 -- 172.21.15.42:0/3081746271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd1a4140440 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.392+0000 7fd19be00640 1 -- 172.21.15.42:0/3081746271 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fd1900f14f0 con 0x7fd1a40ce3b0 2024-05-28T14:11:00.375 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:11:00.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.392+0000 7fd1aaa00640 1 -- 172.21.15.42:0/3081746271 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd184069850 msgr2=0x7fd18406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.392+0000 7fd1aaa00640 1 --2- 172.21.15.42:0/3081746271 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd184069850 0x7fd18406bd10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd194000f70 tx=0x7fd194007df0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.392+0000 7fd1aaa00640 1 -- 172.21.15.42:0/3081746271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1a40ce3b0 msgr2=0x7fd1a415e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.392+0000 7fd1aaa00640 1 --2- 172.21.15.42:0/3081746271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1a40ce3b0 0x7fd1a415e810 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fd190019fb0 tx=0x7fd19001abc0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.392+0000 7fd1aaa00640 1 -- 172.21.15.42:0/3081746271 shutdown_connections 2024-05-28T14:11:00.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.392+0000 7fd1aaa00640 1 --2- 172.21.15.42:0/3081746271 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd184069850 0x7fd18406bd10 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.392+0000 7fd1aaa00640 1 --2- 172.21.15.42:0/3081746271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1a413fd10 0x7fd1a4155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.392+0000 7fd1aaa00640 1 --2- 172.21.15.42:0/3081746271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1a40ce3b0 0x7fd1a415e810 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.392+0000 7fd1aaa00640 1 --2- 172.21.15.42:0/3081746271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1a4071de0 0x7fd1a413d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.392+0000 7fd1aaa00640 1 -- 172.21.15.42:0/3081746271 >> 172.21.15.42:0/3081746271 conn(0x7fd1a40b21f0 msgr2=0x7fd1a415ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:00.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.392+0000 7fd1aaa00640 1 -- 172.21.15.42:0/3081746271 shutdown_connections 2024-05-28T14:11:00.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.392+0000 7fd1aaa00640 1 -- 172.21.15.42:0/3081746271 wait complete. 2024-05-28T14:11:00.398 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:11:00.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.556+0000 7fc041e00640 1 Processor -- start 2024-05-28T14:11:00.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc041e00640 1 -- start start 2024-05-28T14:11:00.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc041e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc03c0d0320 0x7fc03c15cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc041e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0b3c30 0x7fc03c0b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc041e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc03c1601a0 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc041e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc03c0cd920 con 0x7fc03c0d0320 2024-05-28T14:11:00.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc041e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc03c15d160 con 0x7fc03c0b3c30 2024-05-28T14:11:00.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc03be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc03c0cd4a0 0x7fc03c0d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42932/0 (socket says 172.21.15.42:42932) 2024-05-28T14:11:00.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc03be00640 1 -- 172.21.15.42:0/3756426900 learned_addr learned my addr 172.21.15.42:0/3756426900 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc041400640 1 --2- 172.21.15.42:0/3756426900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0b3c30 0x7fc03c0b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc040a00640 1 --2- 172.21.15.42:0/3756426900 >> v2:172.21.15.63:3300/0 conn(0x7fc03c0d0320 0x7fc03c15cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc03b400640 1 -- 172.21.15.42:0/3756426900 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3585899744 0 0) 0x7fc03c1601a0 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc03b400640 1 -- 172.21.15.42:0/3756426900 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc024003530 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc041400640 1 -- 172.21.15.42:0/3756426900 >> v2:172.21.15.63:3300/0 conn(0x7fc03c0d0320 msgr2=0x7fc03c15cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc041400640 1 --2- 172.21.15.42:0/3756426900 >> v2:172.21.15.63:3300/0 conn(0x7fc03c0d0320 0x7fc03c15cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc041400640 1 -- 172.21.15.42:0/3756426900 >> v1:172.21.15.42:6789/0 conn(0x7fc03c0cd4a0 legacy=0x7fc03c0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.560+0000 7fc041400640 1 -- 172.21.15.42:0/3756426900 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c15d440 con 0x7fc03c0b3c30 2024-05-28T14:11:00.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.564+0000 7fc041400640 1 --2- 172.21.15.42:0/3756426900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0b3c30 0x7fc03c0b40c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc034009d90 tx=0x7fc03401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a033c3f8cc53b5f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:00.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.564+0000 7fc03b400640 1 -- 172.21.15.42:0/3756426900 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0340b2070 con 0x7fc03c0b3c30 2024-05-28T14:11:00.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.564+0000 7fc03b400640 1 -- 172.21.15.42:0/3756426900 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0340a58b0 con 0x7fc03c0b3c30 2024-05-28T14:11:00.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.564+0000 7fc03b400640 1 -- 172.21.15.42:0/3756426900 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0340a5c80 con 0x7fc03c0b3c30 2024-05-28T14:11:00.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.564+0000 7fc041e00640 1 -- 172.21.15.42:0/3756426900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0b3c30 msgr2=0x7fc03c0b40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.564+0000 7fc041e00640 1 --2- 172.21.15.42:0/3756426900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0b3c30 0x7fc03c0b40c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc034009d90 tx=0x7fc03401c970 comp rx=0 tx=0).stop 2024-05-28T14:11:00.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.564+0000 7fc041e00640 1 -- 172.21.15.42:0/3756426900 shutdown_connections 2024-05-28T14:11:00.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.564+0000 7fc041e00640 1 --2- 172.21.15.42:0/3756426900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0b3c30 0x7fc03c0b40c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.564+0000 7fc041e00640 1 --2- 172.21.15.42:0/3756426900 >> v2:172.21.15.63:3300/0 conn(0x7fc03c0d0320 0x7fc03c15cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.564+0000 7fc041e00640 1 -- 172.21.15.42:0/3756426900 >> 172.21.15.42:0/3756426900 conn(0x7fc03c06bbc0 msgr2=0x7fc03c06bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:00.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.564+0000 7fc041e00640 1 -- 172.21.15.42:0/3756426900 shutdown_connections 2024-05-28T14:11:00.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.564+0000 7fc041e00640 1 -- 172.21.15.42:0/3756426900 wait complete. 2024-05-28T14:11:00.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.564+0000 7fc041e00640 1 Processor -- start 2024-05-28T14:11:00.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.564+0000 7fc041e00640 1 -- start start 2024-05-28T14:11:00.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc041e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0b3c30 0x7fc03c0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc041e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c0cd4a0 0x7fc03c0ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc041e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0d0320 0x7fc03c0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc041e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c169620 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc041e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c1694a0 con 0x7fc03c0b3c30 2024-05-28T14:11:00.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc041e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc03c1697a0 con 0x7fc03c0d0320 2024-05-28T14:11:00.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc03be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c0cd4a0 0x7fc03c0ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc03be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c0cd4a0 0x7fc03c0ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52616/0 (socket says 172.21.15.42:52616) 2024-05-28T14:11:00.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc03be00640 1 -- 172.21.15.42:0/3886459847 learned_addr learned my addr 172.21.15.42:0/3886459847 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:00.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc041400640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0d0320 0x7fc03c0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc040a00640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0b3c30 0x7fc03c0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc03be00640 1 -- 172.21.15.42:0/3886459847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0d0320 msgr2=0x7fc03c0ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc03be00640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0d0320 0x7fc03c0ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc03be00640 1 -- 172.21.15.42:0/3886459847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0b3c30 msgr2=0x7fc03c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc03be00640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0b3c30 0x7fc03c0adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc03be00640 1 -- 172.21.15.42:0/3886459847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c0b0810 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc040a00640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0b3c30 0x7fc03c0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:00.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc03be00640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c0cd4a0 0x7fc03c0ae060 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fc02800c450 tx=0x7fc02800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:00.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc038c00640 1 -- 172.21.15.42:0/3886459847 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028017070 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc038c00640 1 -- 172.21.15.42:0/3886459847 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc028005cf0 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc038c00640 1 -- 172.21.15.42:0/3886459847 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028007290 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc041e00640 1 -- 172.21.15.42:0/3886459847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03c0b0af0 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc041e00640 1 -- 172.21.15.42:0/3886459847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc03c0b10b0 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.568+0000 7fc041e00640 1 -- 172.21.15.42:0/3886459847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03c04fcf0 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.572+0000 7fc038c00640 1 -- 172.21.15.42:0/3886459847 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fc02801a070 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.572+0000 7fc038c00640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc01c069850 0x7fc01c06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.572+0000 7fc038c00640 1 -- 172.21.15.42:0/3886459847 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7fc028084940 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.572+0000 7fc040a00640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc01c069850 0x7fc01c06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.572+0000 7fc040a00640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc01c069850 0x7fc01c06bd10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc030009b10 tx=0x7fc030008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.576+0000 7fc038c00640 1 -- 172.21.15.42:0/3886459847 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc028056360 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.740+0000 7fc041e00640 1 -- 172.21.15.42:0/3886459847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc03c0b4500 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.740+0000 7fc038c00640 1 -- 172.21.15.42:0/3886459847 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fc02805a660 con 0x7fc03c0cd4a0 2024-05-28T14:11:00.726 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:11:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.744+0000 7fc041e00640 1 -- 172.21.15.42:0/3886459847 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc01c069850 msgr2=0x7fc01c06bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.744+0000 7fc041e00640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc01c069850 0x7fc01c06bd10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc030009b10 tx=0x7fc030008040 comp rx=0 tx=0).stop 2024-05-28T14:11:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.744+0000 7fc041e00640 1 -- 172.21.15.42:0/3886459847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c0cd4a0 msgr2=0x7fc03c0ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.744+0000 7fc041e00640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c0cd4a0 0x7fc03c0ae060 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fc02800c450 tx=0x7fc02800c910 comp rx=0 tx=0).stop 2024-05-28T14:11:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.744+0000 7fc041e00640 1 -- 172.21.15.42:0/3886459847 shutdown_connections 2024-05-28T14:11:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.744+0000 7fc041e00640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc01c069850 0x7fc01c06bd10 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.744+0000 7fc041e00640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc03c0d0320 0x7fc03c0ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.744+0000 7fc041e00640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c0cd4a0 0x7fc03c0ae060 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.744+0000 7fc041e00640 1 --2- 172.21.15.42:0/3886459847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc03c0b3c30 0x7fc03c0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.744+0000 7fc041e00640 1 -- 172.21.15.42:0/3886459847 >> 172.21.15.42:0/3886459847 conn(0x7fc03c06bbc0 msgr2=0x7fc03c0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.744+0000 7fc041e00640 1 -- 172.21.15.42:0/3886459847 shutdown_connections 2024-05-28T14:11:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.744+0000 7fc041e00640 1 -- 172.21.15.42:0/3886459847 wait complete. 2024-05-28T14:11:00.747 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:11:00.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.904+0000 7f350be00640 1 Processor -- start 2024-05-28T14:11:00.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f350be00640 1 -- start start 2024-05-28T14:11:00.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f350be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3504144f80 0x7f3504072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f350be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f350415c0c0 0x7f350406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f350be00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35040afe40 con 0x7f350406ee20 2024-05-28T14:11:00.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f350be00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f350406f7c0 con 0x7f350415c0c0 2024-05-28T14:11:00.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f350be00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f35040724d0 con 0x7f3504144f80 2024-05-28T14:11:00.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f3508a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f350406ee20 0x7f350415b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42940/0 (socket says 172.21.15.42:42940) 2024-05-28T14:11:00.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f3508a00640 1 -- 172.21.15.42:0/448121274 learned_addr learned my addr 172.21.15.42:0/448121274 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:00.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f3509e00640 1 --2- 172.21.15.42:0/448121274 >> v2:172.21.15.42:3300/0 conn(0x7f350415c0c0 0x7f350406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f3509400640 1 --2- 172.21.15.42:0/448121274 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3504144f80 0x7f3504072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f3503e00640 1 -- 172.21.15.42:0/448121274 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1671365618 0 0) 0x7f35040afe40 con 0x7f350406ee20 2024-05-28T14:11:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f3503e00640 1 -- 172.21.15.42:0/448121274 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34ec003530 con 0x7f350406ee20 2024-05-28T14:11:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f3509e00640 1 -- 172.21.15.42:0/448121274 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3504144f80 msgr2=0x7f3504072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f3509e00640 1 --2- 172.21.15.42:0/448121274 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3504144f80 0x7f3504072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f3509e00640 1 -- 172.21.15.42:0/448121274 >> v1:172.21.15.42:6789/0 conn(0x7f350406ee20 legacy=0x7f350415b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.908+0000 7f3509e00640 1 -- 172.21.15.42:0/448121274 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350415c600 con 0x7f350415c0c0 2024-05-28T14:11:00.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.912+0000 7f3509e00640 1 --2- 172.21.15.42:0/448121274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f350415c0c0 0x7f350406f280 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009d90 tx=0x7f34fc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20329d1835055543 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:00.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.912+0000 7f3503e00640 1 -- 172.21.15.42:0/448121274 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0b2070 con 0x7f350415c0c0 2024-05-28T14:11:00.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.912+0000 7f3503e00640 1 -- 172.21.15.42:0/448121274 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34fc0a5920 con 0x7f350415c0c0 2024-05-28T14:11:00.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.912+0000 7f3503e00640 1 -- 172.21.15.42:0/448121274 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0a5cf0 con 0x7f350415c0c0 2024-05-28T14:11:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.912+0000 7f350be00640 1 -- 172.21.15.42:0/448121274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f350415c0c0 msgr2=0x7f350406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.912+0000 7f350be00640 1 --2- 172.21.15.42:0/448121274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f350415c0c0 0x7f350406f280 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009d90 tx=0x7f34fc01c970 comp rx=0 tx=0).stop 2024-05-28T14:11:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.912+0000 7f350be00640 1 -- 172.21.15.42:0/448121274 shutdown_connections 2024-05-28T14:11:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.912+0000 7f350be00640 1 --2- 172.21.15.42:0/448121274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f350415c0c0 0x7f350406f280 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.912+0000 7f350be00640 1 --2- 172.21.15.42:0/448121274 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3504144f80 0x7f3504072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.912+0000 7f350be00640 1 -- 172.21.15.42:0/448121274 >> 172.21.15.42:0/448121274 conn(0x7f35040b21f0 msgr2=0x7f35040b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.912+0000 7f350be00640 1 -- 172.21.15.42:0/448121274 shutdown_connections 2024-05-28T14:11:00.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.912+0000 7f350be00640 1 -- 172.21.15.42:0/448121274 wait complete. 2024-05-28T14:11:00.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.912+0000 7f350be00640 1 Processor -- start 2024-05-28T14:11:00.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f350be00640 1 -- start start 2024-05-28T14:11:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f350be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f350406ee20 0x7f3504136b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f350be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3504144f80 0x7f35041379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f350be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f350415c0c0 0x7f350413f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f350be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3504072ea0 con 0x7f3504144f80 2024-05-28T14:11:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f350be00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f35040b4010 con 0x7f350415c0c0 2024-05-28T14:11:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f350be00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f350415deb0 con 0x7f350406ee20 2024-05-28T14:11:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f3508a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3504144f80 0x7f35041379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f3508a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3504144f80 0x7f35041379b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52642/0 (socket says 172.21.15.42:52642) 2024-05-28T14:11:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f3508a00640 1 -- 172.21.15.42:0/2388019264 learned_addr learned my addr 172.21.15.42:0/2388019264 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f3509400640 1 --2- 172.21.15.42:0/2388019264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f350406ee20 0x7f3504136b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f3509e00640 1 --2- 172.21.15.42:0/2388019264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f350415c0c0 0x7f350413f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f3508a00640 1 -- 172.21.15.42:0/2388019264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f350406ee20 msgr2=0x7f3504136b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f3508a00640 1 --2- 172.21.15.42:0/2388019264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f350406ee20 0x7f3504136b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f3508a00640 1 -- 172.21.15.42:0/2388019264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f350415c0c0 msgr2=0x7f350413f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f3508a00640 1 --2- 172.21.15.42:0/2388019264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f350415c0c0 0x7f350413f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f3508a00640 1 -- 172.21.15.42:0/2388019264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3504138300 con 0x7f3504144f80 2024-05-28T14:11:00.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f3508a00640 1 --2- 172.21.15.42:0/2388019264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3504144f80 0x7f35041379b0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f34f000c450 tx=0x7f34f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:00.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f3501600640 1 -- 172.21.15.42:0/2388019264 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0017070 con 0x7f3504144f80 2024-05-28T14:11:00.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f3501600640 1 -- 172.21.15.42:0/2388019264 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f0005cf0 con 0x7f3504144f80 2024-05-28T14:11:00.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f3501600640 1 -- 172.21.15.42:0/2388019264 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0007290 con 0x7f3504144f80 2024-05-28T14:11:00.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f350be00640 1 -- 172.21.15.42:0/2388019264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350415d580 con 0x7f3504144f80 2024-05-28T14:11:00.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f350be00640 1 -- 172.21.15.42:0/2388019264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3504137090 con 0x7f3504144f80 2024-05-28T14:11:00.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.916+0000 7f350be00640 1 -- 172.21.15.42:0/2388019264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35040738b0 con 0x7f3504144f80 2024-05-28T14:11:00.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.920+0000 7f3501600640 1 -- 172.21.15.42:0/2388019264 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f34f001a070 con 0x7f3504144f80 2024-05-28T14:11:00.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.920+0000 7f3501600640 1 --2- 172.21.15.42:0/2388019264 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f34e4069850 0x7f34e406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:00.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.920+0000 7f3501600640 1 -- 172.21.15.42:0/2388019264 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7f34f0084a30 con 0x7f3504144f80 2024-05-28T14:11:00.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.920+0000 7f3509400640 1 --2- 172.21.15.42:0/2388019264 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f34e4069850 0x7f34e406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:00.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.920+0000 7f3509400640 1 --2- 172.21.15.42:0/2388019264 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f34e4069850 0x7f34e406bd10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009b60 tx=0x7f34f8009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:00.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:00.924+0000 7f3501600640 1 -- 172.21.15.42:0/2388019264 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f34f00563f0 con 0x7f3504144f80 2024-05-28T14:11:01.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.092+0000 7f350be00640 1 -- 172.21.15.42:0/2388019264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f350404fdd0 con 0x7f3504144f80 2024-05-28T14:11:01.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.092+0000 7f3501600640 1 -- 172.21.15.42:0/2388019264 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f34f005a6f0 con 0x7f3504144f80 2024-05-28T14:11:01.076 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:11:01.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.092+0000 7f350be00640 1 -- 172.21.15.42:0/2388019264 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f34e4069850 msgr2=0x7f34e406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.092+0000 7f350be00640 1 --2- 172.21.15.42:0/2388019264 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f34e4069850 0x7f34e406bd10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009b60 tx=0x7f34f8009190 comp rx=0 tx=0).stop 2024-05-28T14:11:01.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.092+0000 7f350be00640 1 -- 172.21.15.42:0/2388019264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3504144f80 msgr2=0x7f35041379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.092+0000 7f350be00640 1 --2- 172.21.15.42:0/2388019264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3504144f80 0x7f35041379b0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f34f000c450 tx=0x7f34f000c910 comp rx=0 tx=0).stop 2024-05-28T14:11:01.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.092+0000 7f350be00640 1 -- 172.21.15.42:0/2388019264 shutdown_connections 2024-05-28T14:11:01.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.092+0000 7f350be00640 1 --2- 172.21.15.42:0/2388019264 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f34e4069850 0x7f34e406bd10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.092+0000 7f350be00640 1 --2- 172.21.15.42:0/2388019264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f350415c0c0 0x7f350413f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.092+0000 7f350be00640 1 --2- 172.21.15.42:0/2388019264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3504144f80 0x7f35041379b0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.092+0000 7f350be00640 1 --2- 172.21.15.42:0/2388019264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f350406ee20 0x7f3504136b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.092+0000 7f350be00640 1 -- 172.21.15.42:0/2388019264 >> 172.21.15.42:0/2388019264 conn(0x7f35040b21f0 msgr2=0x7f35040b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:01.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.092+0000 7f350be00640 1 -- 172.21.15.42:0/2388019264 shutdown_connections 2024-05-28T14:11:01.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.092+0000 7f350be00640 1 -- 172.21.15.42:0/2388019264 wait complete. 2024-05-28T14:11:01.099 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:11:01.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.256+0000 7f7434a00640 1 Processor -- start 2024-05-28T14:11:01.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.256+0000 7f7434a00640 1 -- start start 2024-05-28T14:11:01.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.256+0000 7f7434a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7430156a70 0x7f743006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.256+0000 7f7434a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7430156e50 0x7f74300af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.256+0000 7f7434a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74300afc80 con 0x7f74300af880 2024-05-28T14:11:01.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.256+0000 7f7434a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f743015cb90 con 0x7f7430156e50 2024-05-28T14:11:01.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.256+0000 7f7434a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f743015ce50 con 0x7f7430156a70 2024-05-28T14:11:01.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.260+0000 7f7427e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7430156e50 0x7f74300af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.260+0000 7f7427e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7430156e50 0x7f74300af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52650/0 (socket says 172.21.15.42:52650) 2024-05-28T14:11:01.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.260+0000 7f7427e00640 1 -- 172.21.15.42:0/980990348 learned_addr learned my addr 172.21.15.42:0/980990348 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:01.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.260+0000 7f742de00640 1 --2- 172.21.15.42:0/980990348 >> v2:172.21.15.63:3300/0 conn(0x7f7430156a70 0x7f743006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.260+0000 7f7427e00640 1 -- 172.21.15.42:0/980990348 >> v2:172.21.15.63:3300/0 conn(0x7f7430156a70 msgr2=0x7f743006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:01.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.260+0000 7f7427e00640 1 --2- 172.21.15.42:0/980990348 >> v2:172.21.15.63:3300/0 conn(0x7f7430156a70 0x7f743006dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.260+0000 7f7427e00640 1 -- 172.21.15.42:0/980990348 >> v1:172.21.15.63:6789/0 conn(0x7f74300af880 legacy=0x7f74300d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.260+0000 7f7427e00640 1 -- 172.21.15.42:0/980990348 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743015d130 con 0x7f7430156e50 2024-05-28T14:11:01.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.260+0000 7f7427e00640 1 --2- 172.21.15.42:0/980990348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7430156e50 0x7f74300af340 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f74300d13a0 tx=0x7f741801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c4ae631c19d734 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:01.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.260+0000 7f742d400640 1 -- 172.21.15.42:0/980990348 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74180aa680 con 0x7f7430156e50 2024-05-28T14:11:01.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.260+0000 7f742d400640 1 -- 172.21.15.42:0/980990348 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74180af070 con 0x7f7430156e50 2024-05-28T14:11:01.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.260+0000 7f742d400640 1 -- 172.21.15.42:0/980990348 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74180aaa40 con 0x7f7430156e50 2024-05-28T14:11:01.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 -- 172.21.15.42:0/980990348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7430156e50 msgr2=0x7f74300af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 --2- 172.21.15.42:0/980990348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7430156e50 0x7f74300af340 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f74300d13a0 tx=0x7f741801c520 comp rx=0 tx=0).stop 2024-05-28T14:11:01.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 -- 172.21.15.42:0/980990348 shutdown_connections 2024-05-28T14:11:01.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 --2- 172.21.15.42:0/980990348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7430156e50 0x7f74300af340 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 --2- 172.21.15.42:0/980990348 >> v2:172.21.15.63:3300/0 conn(0x7f7430156a70 0x7f743006dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 -- 172.21.15.42:0/980990348 >> 172.21.15.42:0/980990348 conn(0x7f743006bdf0 msgr2=0x7f743006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:01.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 -- 172.21.15.42:0/980990348 shutdown_connections 2024-05-28T14:11:01.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 -- 172.21.15.42:0/980990348 wait complete. 2024-05-28T14:11:01.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 Processor -- start 2024-05-28T14:11:01.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 -- start start 2024-05-28T14:11:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74300af880 0x7f7430099a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7430156a70 0x7f7430073830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7430156e50 0x7f7430155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7430160b20 con 0x7f7430156e50 2024-05-28T14:11:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7430160ca0 con 0x7f74300af880 2024-05-28T14:11:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7434a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7430160e20 con 0x7f7430156a70 2024-05-28T14:11:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f742e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7430156e50 0x7f7430155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f742e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7430156e50 0x7f7430155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52652/0 (socket says 172.21.15.42:52652) 2024-05-28T14:11:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f742e800640 1 -- 172.21.15.42:0/3663582241 learned_addr learned my addr 172.21.15.42:0/3663582241 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f7427e00640 1 --2- 172.21.15.42:0/3663582241 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7430156a70 0x7f7430073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f742de00640 1 --2- 172.21.15.42:0/3663582241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74300af880 0x7f7430099a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f742e800640 1 -- 172.21.15.42:0/3663582241 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7430156a70 msgr2=0x7f7430073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f742e800640 1 --2- 172.21.15.42:0/3663582241 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7430156a70 0x7f7430073830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f742e800640 1 -- 172.21.15.42:0/3663582241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74300af880 msgr2=0x7f7430099a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f742e800640 1 --2- 172.21.15.42:0/3663582241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74300af880 0x7f7430099a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.264+0000 7f742e800640 1 -- 172.21.15.42:0/3663582241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7430072790 con 0x7f7430156e50 2024-05-28T14:11:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.268+0000 7f742e800640 1 --2- 172.21.15.42:0/3663582241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7430156e50 0x7f7430155520 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f742800c7a0 tx=0x7f742800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:01.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.268+0000 7f7426000640 1 -- 172.21.15.42:0/3663582241 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7428003b80 con 0x7f7430156e50 2024-05-28T14:11:01.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.268+0000 7f7426000640 1 -- 172.21.15.42:0/3663582241 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7428003d20 con 0x7f7430156e50 2024-05-28T14:11:01.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.268+0000 7f7426000640 1 -- 172.21.15.42:0/3663582241 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742800aa90 con 0x7f7430156e50 2024-05-28T14:11:01.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.268+0000 7f7434a00640 1 -- 172.21.15.42:0/3663582241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7430072a70 con 0x7f7430156e50 2024-05-28T14:11:01.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.268+0000 7f7434a00640 1 -- 172.21.15.42:0/3663582241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7430073030 con 0x7f7430156e50 2024-05-28T14:11:01.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.268+0000 7f7434a00640 1 -- 172.21.15.42:0/3663582241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7430068a90 con 0x7f7430156e50 2024-05-28T14:11:01.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.268+0000 7f7426000640 1 -- 172.21.15.42:0/3663582241 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f742800ac30 con 0x7f7430156e50 2024-05-28T14:11:01.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.268+0000 7f7426000640 1 --2- 172.21.15.42:0/3663582241 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f740c069850 0x7f740c06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.268+0000 7f7426000640 1 -- 172.21.15.42:0/3663582241 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7f7428085a40 con 0x7f7430156e50 2024-05-28T14:11:01.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.268+0000 7f742de00640 1 --2- 172.21.15.42:0/3663582241 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f740c069850 0x7f740c06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.272+0000 7f742de00640 1 --2- 172.21.15.42:0/3663582241 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f740c069850 0x7f740c06bd10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7420009af0 tx=0x7f7420008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:01.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.272+0000 7f7426000640 1 -- 172.21.15.42:0/3663582241 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f74280575e0 con 0x7f7430156e50 2024-05-28T14:11:01.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.472+0000 7f7434a00640 1 -- 172.21.15.42:0/3663582241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74300b5160 con 0x7f7430156e50 2024-05-28T14:11:01.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.476+0000 7f7426000640 1 -- 172.21.15.42:0/3663582241 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5182 (secure 0 0 0) 0x7f7428056ee0 con 0x7f7430156e50 2024-05-28T14:11:01.459 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:01.459 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7657,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/402076081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":402076081},{"type":"v1","addr":"172.21.15.63:6835","nonce":402076081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7998,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3779622485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3779622485},{"type":"v1","addr":"172.21.15.42:6835","nonce":3779622485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2961573665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2961573665},{"type":"v1","addr":"172.21.15.42:6837","nonce":2961573665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:09:31.146587+0000","modified":"2024-05-28T14:09:34.152047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":56621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56621":{"gid":56621,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/615628755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":615628755},{"type":"v1","addr":"172.21.15.63:6834","nonce":615628755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":56621,"qdb_cluster":[56621]},"id":6}]} 2024-05-28T14:11:01.460 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 55 2024-05-28T14:11:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.476+0000 7f7434a00640 1 -- 172.21.15.42:0/3663582241 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f740c069850 msgr2=0x7f740c06bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.476+0000 7f7434a00640 1 --2- 172.21.15.42:0/3663582241 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f740c069850 0x7f740c06bd10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7420009af0 tx=0x7f7420008040 comp rx=0 tx=0).stop 2024-05-28T14:11:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.476+0000 7f7434a00640 1 -- 172.21.15.42:0/3663582241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7430156e50 msgr2=0x7f7430155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.476+0000 7f7434a00640 1 --2- 172.21.15.42:0/3663582241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7430156e50 0x7f7430155520 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f742800c7a0 tx=0x7f742800cc60 comp rx=0 tx=0).stop 2024-05-28T14:11:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.476+0000 7f7434a00640 1 -- 172.21.15.42:0/3663582241 shutdown_connections 2024-05-28T14:11:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.476+0000 7f7434a00640 1 --2- 172.21.15.42:0/3663582241 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f740c069850 0x7f740c06bd10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.476+0000 7f7434a00640 1 --2- 172.21.15.42:0/3663582241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7430156e50 0x7f7430155520 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.476+0000 7f7434a00640 1 --2- 172.21.15.42:0/3663582241 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7430156a70 0x7f7430073830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.476+0000 7f7434a00640 1 --2- 172.21.15.42:0/3663582241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74300af880 0x7f7430099a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.476+0000 7f7434a00640 1 -- 172.21.15.42:0/3663582241 >> 172.21.15.42:0/3663582241 conn(0x7f743006bdf0 msgr2=0x7f743015b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:01.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.476+0000 7f7434a00640 1 -- 172.21.15.42:0/3663582241 shutdown_connections 2024-05-28T14:11:01.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.476+0000 7f7434a00640 1 -- 172.21.15.42:0/3663582241 wait complete. 2024-05-28T14:11:01.481 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:11:01.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.636+0000 7fb51bc00640 1 Processor -- start 2024-05-28T14:11:01.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.636+0000 7fb51bc00640 1 -- start start 2024-05-28T14:11:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.640+0000 7fb51bc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb514156a70 0x7fb51406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.640+0000 7fb51bc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb514156e50 0x7fb5140af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.640+0000 7fb51bc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb51415cb90 con 0x7fb5140af880 2024-05-28T14:11:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.640+0000 7fb51bc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb51415ce50 con 0x7fb514156e50 2024-05-28T14:11:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.640+0000 7fb51bc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb51415d130 con 0x7fb514156a70 2024-05-28T14:11:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.640+0000 7fb50be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb514156e50 0x7fb5140af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.640+0000 7fb50be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb514156e50 0x7fb5140af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52666/0 (socket says 172.21.15.42:52666) 2024-05-28T14:11:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.640+0000 7fb50be00640 1 -- 172.21.15.42:0/1869510351 learned_addr learned my addr 172.21.15.42:0/1869510351 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.640+0000 7fb519200640 1 --2- 172.21.15.42:0/1869510351 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb514156a70 0x7fb51406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.640+0000 7fb50be00640 1 -- 172.21.15.42:0/1869510351 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb514156a70 msgr2=0x7fb51406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.640+0000 7fb50be00640 1 --2- 172.21.15.42:0/1869510351 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb514156a70 0x7fb51406dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.640+0000 7fb50be00640 1 -- 172.21.15.42:0/1869510351 >> v1:172.21.15.63:6789/0 conn(0x7fb5140af880 legacy=0x7fb5140d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.640+0000 7fb50be00640 1 -- 172.21.15.42:0/1869510351 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51415d410 con 0x7fb514156e50 2024-05-28T14:11:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.640+0000 7fb519200640 1 --2- 172.21.15.42:0/1869510351 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb514156a70 0x7fb51406dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:01.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.644+0000 7fb50be00640 1 --2- 172.21.15.42:0/1869510351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb514156e50 0x7fb5140af340 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009870 tx=0x7fb4fc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b66ec82d6c92db1 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:01.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.644+0000 7fb513e00640 1 -- 172.21.15.42:0/1869510351 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0ab5f0 con 0x7fb514156e50 2024-05-28T14:11:01.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.644+0000 7fb513e00640 1 -- 172.21.15.42:0/1869510351 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4fc0b0070 con 0x7fb514156e50 2024-05-28T14:11:01.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.644+0000 7fb513e00640 1 -- 172.21.15.42:0/1869510351 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0ab9b0 con 0x7fb514156e50 2024-05-28T14:11:01.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.644+0000 7fb51bc00640 1 -- 172.21.15.42:0/1869510351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb514156e50 msgr2=0x7fb5140af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.644+0000 7fb51bc00640 1 --2- 172.21.15.42:0/1869510351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb514156e50 0x7fb5140af340 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009870 tx=0x7fb4fc01c520 comp rx=0 tx=0).stop 2024-05-28T14:11:01.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.644+0000 7fb51bc00640 1 -- 172.21.15.42:0/1869510351 shutdown_connections 2024-05-28T14:11:01.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.644+0000 7fb51bc00640 1 --2- 172.21.15.42:0/1869510351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb514156e50 0x7fb5140af340 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.644+0000 7fb51bc00640 1 --2- 172.21.15.42:0/1869510351 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb514156a70 0x7fb51406dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.644+0000 7fb51bc00640 1 -- 172.21.15.42:0/1869510351 >> 172.21.15.42:0/1869510351 conn(0x7fb51406bdf0 msgr2=0x7fb51406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:01.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.644+0000 7fb51bc00640 1 -- 172.21.15.42:0/1869510351 shutdown_connections 2024-05-28T14:11:01.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.644+0000 7fb51bc00640 1 -- 172.21.15.42:0/1869510351 wait complete. 2024-05-28T14:11:01.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.644+0000 7fb51bc00640 1 Processor -- start 2024-05-28T14:11:01.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.644+0000 7fb51bc00640 1 -- start start 2024-05-28T14:11:01.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb51bc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5140af880 0x7fb51414ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb51bc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb514156a70 0x7fb5140cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb51bc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb514156e50 0x7fb5140b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb51bc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb514160b20 con 0x7fb514156e50 2024-05-28T14:11:01.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb51bc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5140d1010 con 0x7fb514156a70 2024-05-28T14:11:01.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb51bc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb514160ca0 con 0x7fb5140af880 2024-05-28T14:11:01.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb519c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb514156e50 0x7fb5140b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb519c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb514156e50 0x7fb5140b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52676/0 (socket says 172.21.15.42:52676) 2024-05-28T14:11:01.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb519c00640 1 -- 172.21.15.42:0/1150686484 learned_addr learned my addr 172.21.15.42:0/1150686484 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:01.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb519200640 1 --2- 172.21.15.42:0/1150686484 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5140af880 0x7fb51414ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb50be00640 1 --2- 172.21.15.42:0/1150686484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb514156a70 0x7fb5140cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb519c00640 1 -- 172.21.15.42:0/1150686484 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5140af880 msgr2=0x7fb51414ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb519c00640 1 --2- 172.21.15.42:0/1150686484 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5140af880 0x7fb51414ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb519c00640 1 -- 172.21.15.42:0/1150686484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb514156a70 msgr2=0x7fb5140cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb519c00640 1 --2- 172.21.15.42:0/1150686484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb514156a70 0x7fb5140cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb519c00640 1 -- 172.21.15.42:0/1150686484 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5140cdfd0 con 0x7fb514156e50 2024-05-28T14:11:01.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb519c00640 1 --2- 172.21.15.42:0/1150686484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb514156e50 0x7fb5140b0980 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00c7b0 tx=0x7fb50c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:01.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb511600640 1 -- 172.21.15.42:0/1150686484 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c004b70 con 0x7fb514156e50 2024-05-28T14:11:01.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb511600640 1 -- 172.21.15.42:0/1150686484 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50c004d10 con 0x7fb514156e50 2024-05-28T14:11:01.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb511600640 1 -- 172.21.15.42:0/1150686484 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00aa30 con 0x7fb514156e50 2024-05-28T14:11:01.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb51bc00640 1 -- 172.21.15.42:0/1150686484 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5141546c0 con 0x7fb514156e50 2024-05-28T14:11:01.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb51bc00640 1 -- 172.21.15.42:0/1150686484 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5140ce2b0 con 0x7fb514156e50 2024-05-28T14:11:01.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb51bc00640 1 -- 172.21.15.42:0/1150686484 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb514068a90 con 0x7fb514156e50 2024-05-28T14:11:01.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.648+0000 7fb511600640 1 -- 172.21.15.42:0/1150686484 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fb50c00abd0 con 0x7fb514156e50 2024-05-28T14:11:01.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.652+0000 7fb511600640 1 --2- 172.21.15.42:0/1150686484 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb4f0069850 0x7fb4f006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.652+0000 7fb511600640 1 -- 172.21.15.42:0/1150686484 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7fb50c085b80 con 0x7fb514156e50 2024-05-28T14:11:01.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.652+0000 7fb519200640 1 --2- 172.21.15.42:0/1150686484 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb4f0069850 0x7fb4f006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.652+0000 7fb519200640 1 --2- 172.21.15.42:0/1150686484 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb4f0069850 0x7fb4f006bd10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb50400ab60 tx=0x7fb504008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:01.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.656+0000 7fb511600640 1 -- 172.21.15.42:0/1150686484 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb50c020070 con 0x7fb514156e50 2024-05-28T14:11:01.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.820+0000 7fb51bc00640 1 -- 172.21.15.42:0/1150686484 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb51406e530 con 0x7fb514156e50 2024-05-28T14:11:01.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.824+0000 7fb511600640 1 -- 172.21.15.42:0/1150686484 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16852 (secure 0 0 0) 0x7fb50c0571c0 con 0x7fb514156e50 2024-05-28T14:11:01.807 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:01.807 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":93,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:10:56.060535+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:09:28.913704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-28T14:09:30.108327+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":93,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.3e","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:11:01.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.824+0000 7fb51bc00640 1 -- 172.21.15.42:0/1150686484 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb4f0069850 msgr2=0x7fb4f006bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.824+0000 7fb51bc00640 1 --2- 172.21.15.42:0/1150686484 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb4f0069850 0x7fb4f006bd10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb50400ab60 tx=0x7fb504008040 comp rx=0 tx=0).stop 2024-05-28T14:11:01.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.824+0000 7fb51bc00640 1 -- 172.21.15.42:0/1150686484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb514156e50 msgr2=0x7fb5140b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.824+0000 7fb51bc00640 1 --2- 172.21.15.42:0/1150686484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb514156e50 0x7fb5140b0980 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00c7b0 tx=0x7fb50c00cc70 comp rx=0 tx=0).stop 2024-05-28T14:11:01.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.824+0000 7fb51bc00640 1 -- 172.21.15.42:0/1150686484 shutdown_connections 2024-05-28T14:11:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.824+0000 7fb51bc00640 1 --2- 172.21.15.42:0/1150686484 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb4f0069850 0x7fb4f006bd10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.824+0000 7fb51bc00640 1 --2- 172.21.15.42:0/1150686484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb514156e50 0x7fb5140b0980 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.824+0000 7fb51bc00640 1 --2- 172.21.15.42:0/1150686484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb514156a70 0x7fb5140cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.824+0000 7fb51bc00640 1 --2- 172.21.15.42:0/1150686484 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5140af880 0x7fb51414ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.824+0000 7fb51bc00640 1 -- 172.21.15.42:0/1150686484 >> 172.21.15.42:0/1150686484 conn(0x7fb51406bdf0 msgr2=0x7fb51406f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.824+0000 7fb51bc00640 1 -- 172.21.15.42:0/1150686484 shutdown_connections 2024-05-28T14:11:01.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.824+0000 7fb51bc00640 1 -- 172.21.15.42:0/1150686484 wait complete. 2024-05-28T14:11:01.829 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-28T14:11:01.830 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:11:01.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.988+0000 7f547be00640 1 Processor -- start 2024-05-28T14:11:01.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.988+0000 7f547be00640 1 -- start start 2024-05-28T14:11:01.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.988+0000 7f547be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f547406f660 0x7f5474073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.988+0000 7f547be00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f547415ba40 0x7f54740d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.988+0000 7f547be00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f547415f280 con 0x7f5474072dd0 2024-05-28T14:11:01.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.988+0000 7f547be00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f547415f030 con 0x7f547415ba40 2024-05-28T14:11:01.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.988+0000 7f547be00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f547415f560 con 0x7f547406f660 2024-05-28T14:11:01.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.988+0000 7f5479400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f547406f660 0x7f5474073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.988+0000 7f5478a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f547415ba40 0x7f54740d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.988+0000 7f5478a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f547415ba40 0x7f54740d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:35504/0 (socket says 172.21.15.42:35504) 2024-05-28T14:11:01.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.988+0000 7f5478a00640 1 -- 172.21.15.42:0/2725907949 learned_addr learned my addr 172.21.15.42:0/2725907949 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:01.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.992+0000 7f5478a00640 1 -- 172.21.15.42:0/2725907949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f547406f660 msgr2=0x7f5474073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:01.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.992+0000 7f5478a00640 1 --2- 172.21.15.42:0/2725907949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f547406f660 0x7f5474073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.992+0000 7f5478a00640 1 -- 172.21.15.42:0/2725907949 >> v1:172.21.15.63:6789/0 conn(0x7f5474072dd0 legacy=0x7f54740b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.992+0000 7f5478a00640 1 -- 172.21.15.42:0/2725907949 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547415f840 con 0x7f547415ba40 2024-05-28T14:11:01.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.992+0000 7f5473e00640 1 -- 172.21.15.42:0/2725907949 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099613938 0 0) 0x7f547415f280 con 0x7f5474072dd0 2024-05-28T14:11:01.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.992+0000 7f5478a00640 1 --2- 172.21.15.42:0/2725907949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547415ba40 0x7f54740d06f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5460009870 tx=0x7f546001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9cf8d268b3ce020 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.992+0000 7f5473e00640 1 -- 172.21.15.42:0/2725907949 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54600b0070 con 0x7f547415ba40 2024-05-28T14:11:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.992+0000 7f5473e00640 1 -- 172.21.15.42:0/2725907949 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54600ab600 con 0x7f547415ba40 2024-05-28T14:11:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.992+0000 7f5473e00640 1 -- 172.21.15.42:0/2725907949 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54600ab9d0 con 0x7f547415ba40 2024-05-28T14:11:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.992+0000 7f547be00640 1 -- 172.21.15.42:0/2725907949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547415ba40 msgr2=0x7f54740d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.992+0000 7f547be00640 1 --2- 172.21.15.42:0/2725907949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547415ba40 0x7f54740d06f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5460009870 tx=0x7f546001c410 comp rx=0 tx=0).stop 2024-05-28T14:11:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f547be00640 1 -- 172.21.15.42:0/2725907949 shutdown_connections 2024-05-28T14:11:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f547be00640 1 --2- 172.21.15.42:0/2725907949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547415ba40 0x7f54740d06f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f547be00640 1 --2- 172.21.15.42:0/2725907949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f547406f660 0x7f5474073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f547be00640 1 -- 172.21.15.42:0/2725907949 >> 172.21.15.42:0/2725907949 conn(0x7f547406bdf0 msgr2=0x7f547406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f547be00640 1 -- 172.21.15.42:0/2725907949 shutdown_connections 2024-05-28T14:11:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f547be00640 1 -- 172.21.15.42:0/2725907949 wait complete. 2024-05-28T14:11:01.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f547be00640 1 Processor -- start 2024-05-28T14:11:01.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f547be00640 1 -- start start 2024-05-28T14:11:01.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f547be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f547406f660 0x7f5474151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f547be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5474072dd0 0x7f5474156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f547be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547409b400 0x7f547415df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f547be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5474168550 con 0x7f547406f660 2024-05-28T14:11:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f547be00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f54741683d0 con 0x7f547409b400 2024-05-28T14:11:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f547be00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f54741686d0 con 0x7f5474072dd0 2024-05-28T14:11:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5479400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f547406f660 0x7f5474151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5479400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f547406f660 0x7f5474151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52678/0 (socket says 172.21.15.42:52678) 2024-05-28T14:11:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5479400640 1 -- 172.21.15.42:0/3889804223 learned_addr learned my addr 172.21.15.42:0/3889804223 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5478a00640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5474072dd0 0x7f5474156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5479e00640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547409b400 0x7f547415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5479400640 1 -- 172.21.15.42:0/3889804223 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5474072dd0 msgr2=0x7f5474156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5479400640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5474072dd0 0x7f5474156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5479400640 1 -- 172.21.15.42:0/3889804223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547409b400 msgr2=0x7f547415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5479400640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547409b400 0x7f547415df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5479400640 1 -- 172.21.15.42:0/3889804223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5474156ad0 con 0x7f547406f660 2024-05-28T14:11:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5479e00640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547409b400 0x7f547415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5479400640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f547406f660 0x7f5474151720 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f546800e930 tx=0x7f546800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5471600640 1 -- 172.21.15.42:0/3889804223 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468017070 con 0x7f547406f660 2024-05-28T14:11:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5471600640 1 -- 172.21.15.42:0/3889804223 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f546800ca00 con 0x7f547406f660 2024-05-28T14:11:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:01.996+0000 7f5471600640 1 -- 172.21.15.42:0/3889804223 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546800cdb0 con 0x7f547406f660 2024-05-28T14:11:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.000+0000 7f547be00640 1 -- 172.21.15.42:0/3889804223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5474156db0 con 0x7f547406f660 2024-05-28T14:11:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.000+0000 7f547be00640 1 -- 172.21.15.42:0/3889804223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547409cc80 con 0x7f547406f660 2024-05-28T14:11:01.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.000+0000 7f547be00640 1 -- 172.21.15.42:0/3889804223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5474156f80 con 0x7f547406f660 2024-05-28T14:11:01.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.000+0000 7f5471600640 1 -- 172.21.15.42:0/3889804223 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f546801a070 con 0x7f547406f660 2024-05-28T14:11:01.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.000+0000 7f5471600640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5454069850 0x7f545406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:01.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.000+0000 7f5471600640 1 -- 172.21.15.42:0/3889804223 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7f5468085a40 con 0x7f547406f660 2024-05-28T14:11:01.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.000+0000 7f5478a00640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5454069850 0x7f545406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:01.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.000+0000 7f5478a00640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5454069850 0x7f545406bd10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f54600ad000 tx=0x7f5460008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:01.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.004+0000 7f5471600640 1 -- 172.21.15.42:0/3889804223 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5468057b70 con 0x7f547406f660 2024-05-28T14:11:02.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.204+0000 7f547be00640 1 -- 172.21.15.42:0/3889804223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5474068a90 con 0x7f547406f660 2024-05-28T14:11:02.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.204+0000 7f5471600640 1 -- 172.21.15.42:0/3889804223 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5182 (secure 0 0 0) 0x7f5468057470 con 0x7f547406f660 2024-05-28T14:11:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:02.190 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7657,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/402076081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":402076081},{"type":"v1","addr":"172.21.15.63:6835","nonce":402076081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7998,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3779622485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3779622485},{"type":"v1","addr":"172.21.15.42:6835","nonce":3779622485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2961573665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2961573665},{"type":"v1","addr":"172.21.15.42:6837","nonce":2961573665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:09:31.146587+0000","modified":"2024-05-28T14:09:34.152047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":56621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56621":{"gid":56621,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/615628755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":615628755},{"type":"v1","addr":"172.21.15.63:6834","nonce":615628755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":56621,"qdb_cluster":[56621]},"id":6}]} 2024-05-28T14:11:02.191 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 55 2024-05-28T14:11:02.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.208+0000 7f547be00640 1 -- 172.21.15.42:0/3889804223 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5454069850 msgr2=0x7f545406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.208+0000 7f547be00640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5454069850 0x7f545406bd10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f54600ad000 tx=0x7f5460008040 comp rx=0 tx=0).stop 2024-05-28T14:11:02.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.208+0000 7f547be00640 1 -- 172.21.15.42:0/3889804223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f547406f660 msgr2=0x7f5474151720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.208+0000 7f547be00640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f547406f660 0x7f5474151720 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f546800e930 tx=0x7f546800edf0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.208+0000 7f547be00640 1 -- 172.21.15.42:0/3889804223 shutdown_connections 2024-05-28T14:11:02.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.208+0000 7f547be00640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5454069850 0x7f545406bd10 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.208+0000 7f547be00640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547409b400 0x7f547415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.208+0000 7f547be00640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5474072dd0 0x7f5474156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.208+0000 7f547be00640 1 --2- 172.21.15.42:0/3889804223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f547406f660 0x7f5474151720 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.208+0000 7f547be00640 1 -- 172.21.15.42:0/3889804223 >> 172.21.15.42:0/3889804223 conn(0x7f547406bdf0 msgr2=0x7f54741521f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:02.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.208+0000 7f547be00640 1 -- 172.21.15.42:0/3889804223 shutdown_connections 2024-05-28T14:11:02.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.208+0000 7f547be00640 1 -- 172.21.15.42:0/3889804223 wait complete. 2024-05-28T14:11:02.212 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:11:02.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.368+0000 7f3a82200640 1 Processor -- start 2024-05-28T14:11:02.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.368+0000 7f3a82200640 1 -- start start 2024-05-28T14:11:02.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a82200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a7c0cde30 0x7f3a7c0b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:02.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a82200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a7c0d17b0 con 0x7f3a7c0b1f30 2024-05-28T14:11:02.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a82200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a7c0d1a90 con 0x7f3a7c0b2bf0 2024-05-28T14:11:02.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a82200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3a7c0b0a90 con 0x7f3a7c0cde30 2024-05-28T14:11:02.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a7be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f3a7c0b1f30 0x7f3a7c1654f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42956/0 (socket says 172.21.15.42:42956) 2024-05-28T14:11:02.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a7be00640 1 -- 172.21.15.42:0/1080871296 learned_addr learned my addr 172.21.15.42:0/1080871296 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:02.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a7b400640 1 --2- 172.21.15.42:0/1080871296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a7c0cde30 0x7f3a7c0b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:02.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a7aa00640 1 -- 172.21.15.42:0/1080871296 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1502751161 0 0) 0x7f3a7c0d17b0 con 0x7f3a7c0b1f30 2024-05-28T14:11:02.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a7aa00640 1 -- 172.21.15.42:0/1080871296 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a60003530 con 0x7f3a7c0b1f30 2024-05-28T14:11:02.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a7aa00640 1 -- 172.21.15.42:0/1080871296 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 548946561 0 0) 0x7f3a7c0d1a90 con 0x7f3a7c0b2bf0 2024-05-28T14:11:02.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a7aa00640 1 -- 172.21.15.42:0/1080871296 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a7c0d17b0 con 0x7f3a7c0b2bf0 2024-05-28T14:11:02.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a7b400640 1 -- 172.21.15.42:0/1080871296 >> v1:172.21.15.63:6789/0 conn(0x7f3a7c0b2bf0 legacy=0x7f3a7c0ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a7b400640 1 -- 172.21.15.42:0/1080871296 >> v1:172.21.15.42:6789/0 conn(0x7f3a7c0b1f30 legacy=0x7f3a7c1654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a7aa00640 1 -- 172.21.15.42:0/1080871296 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 766913667 0 0) 0x7f3a60003530 con 0x7f3a7c0b1f30 2024-05-28T14:11:02.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.372+0000 7f3a7b400640 1 -- 172.21.15.42:0/1080871296 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c0d1d70 con 0x7f3a7c0cde30 2024-05-28T14:11:02.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.376+0000 7f3a7b400640 1 --2- 172.21.15.42:0/1080871296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a7c0cde30 0x7f3a7c0b26b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0b0c10 tx=0x7f3a6c01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57dc431d9c04dfbf server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:02.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.376+0000 7f3a7aa00640 1 -- 172.21.15.42:0/1080871296 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0b2070 con 0x7f3a7c0cde30 2024-05-28T14:11:02.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.376+0000 7f3a7aa00640 1 -- 172.21.15.42:0/1080871296 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a6c0a5510 con 0x7f3a7c0cde30 2024-05-28T14:11:02.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.376+0000 7f3a7aa00640 1 -- 172.21.15.42:0/1080871296 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74001680 con 0x7f3a7c0cde30 2024-05-28T14:11:02.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.376+0000 7f3a82200640 1 -- 172.21.15.42:0/1080871296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a7c0cde30 msgr2=0x7f3a7c0b26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.376+0000 7f3a82200640 1 --2- 172.21.15.42:0/1080871296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a7c0cde30 0x7f3a7c0b26b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0b0c10 tx=0x7f3a6c01c890 comp rx=0 tx=0).stop 2024-05-28T14:11:02.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.376+0000 7f3a82200640 1 -- 172.21.15.42:0/1080871296 shutdown_connections 2024-05-28T14:11:02.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.376+0000 7f3a82200640 1 --2- 172.21.15.42:0/1080871296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a7c0cde30 0x7f3a7c0b26b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.376+0000 7f3a82200640 1 -- 172.21.15.42:0/1080871296 >> 172.21.15.42:0/1080871296 conn(0x7f3a7c0b68f0 msgr2=0x7f3a7c0b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:02.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.376+0000 7f3a82200640 1 -- 172.21.15.42:0/1080871296 shutdown_connections 2024-05-28T14:11:02.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.376+0000 7f3a82200640 1 -- 172.21.15.42:0/1080871296 wait complete. 2024-05-28T14:11:02.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.376+0000 7f3a82200640 1 Processor -- start 2024-05-28T14:11:02.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a82200640 1 -- start start 2024-05-28T14:11:02.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a82200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a7c0b1f30 0x7f3a7c070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:02.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a82200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a7c0b2bf0 0x7f3a7c1481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:02.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a82200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a7c0cde30 0x7f3a7c14ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:02.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a82200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c06bd80 con 0x7f3a7c0cde30 2024-05-28T14:11:02.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a82200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c06bc00 con 0x7f3a7c0b1f30 2024-05-28T14:11:02.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a82200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3a7c06bf00 con 0x7f3a7c0b2bf0 2024-05-28T14:11:02.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a7be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a7c0cde30 0x7f3a7c14ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:02.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a7be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a7c0cde30 0x7f3a7c14ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52690/0 (socket says 172.21.15.42:52690) 2024-05-28T14:11:02.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a7be00640 1 -- 172.21.15.42:0/2238910502 learned_addr learned my addr 172.21.15.42:0/2238910502 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:02.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a7b400640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a7c0b1f30 0x7f3a7c070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:02.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a72a00640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a7c0b2bf0 0x7f3a7c1481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:02.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a7be00640 1 -- 172.21.15.42:0/2238910502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a7c0b2bf0 msgr2=0x7f3a7c1481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a7be00640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a7c0b2bf0 0x7f3a7c1481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a7be00640 1 -- 172.21.15.42:0/2238910502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a7c0b1f30 msgr2=0x7f3a7c070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a7be00640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a7c0b1f30 0x7f3a7c070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a7be00640 1 -- 172.21.15.42:0/2238910502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c159600 con 0x7f3a7c0cde30 2024-05-28T14:11:02.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a7b400640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a7c0b1f30 0x7f3a7c070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:11:02.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a7be00640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a7c0cde30 0x7f3a7c14ca20 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f3a74005890 tx=0x7f3a7400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:02.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a73e00640 1 -- 172.21.15.42:0/2238910502 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74017070 con 0x7f3a7c0cde30 2024-05-28T14:11:02.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a73e00640 1 -- 172.21.15.42:0/2238910502 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a74005ae0 con 0x7f3a7c0cde30 2024-05-28T14:11:02.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a82200640 1 -- 172.21.15.42:0/2238910502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7c156c90 con 0x7f3a7c0cde30 2024-05-28T14:11:02.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a73e00640 1 -- 172.21.15.42:0/2238910502 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74007070 con 0x7f3a7c0cde30 2024-05-28T14:11:02.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a82200640 1 -- 172.21.15.42:0/2238910502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7c157250 con 0x7f3a7c0cde30 2024-05-28T14:11:02.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.380+0000 7f3a82200640 1 -- 172.21.15.42:0/2238910502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a7c1597d0 con 0x7f3a7c0cde30 2024-05-28T14:11:02.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.384+0000 7f3a73e00640 1 -- 172.21.15.42:0/2238910502 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f3a7401a070 con 0x7f3a7c0cde30 2024-05-28T14:11:02.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.384+0000 7f3a73e00640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3a58069850 0x7f3a5806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:02.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.384+0000 7f3a73e00640 1 -- 172.21.15.42:0/2238910502 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7f3a74084a70 con 0x7f3a7c0cde30 2024-05-28T14:11:02.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.384+0000 7f3a7b400640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3a58069850 0x7f3a5806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:02.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.384+0000 7f3a7b400640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3a58069850 0x7f3a5806bd10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0af000 tx=0x7f3a6c0a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:02.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.388+0000 7f3a73e00640 1 -- 172.21.15.42:0/2238910502 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3a74056430 con 0x7f3a7c0cde30 2024-05-28T14:11:02.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.584+0000 7f3a82200640 1 -- 172.21.15.42:0/2238910502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a7c068a90 con 0x7f3a7c0cde30 2024-05-28T14:11:02.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.588+0000 7f3a73e00640 1 -- 172.21.15.42:0/2238910502 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5182 (secure 0 0 0) 0x7f3a7405a730 con 0x7f3a7c0cde30 2024-05-28T14:11:02.571 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:02.571 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7657,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/402076081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":402076081},{"type":"v1","addr":"172.21.15.63:6835","nonce":402076081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7998,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3779622485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3779622485},{"type":"v1","addr":"172.21.15.42:6835","nonce":3779622485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2961573665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2961573665},{"type":"v1","addr":"172.21.15.42:6837","nonce":2961573665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:09:31.146587+0000","modified":"2024-05-28T14:09:34.152047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":56621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56621":{"gid":56621,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/615628755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":615628755},{"type":"v1","addr":"172.21.15.63:6834","nonce":615628755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":56621,"qdb_cluster":[56621]},"id":6}]} 2024-05-28T14:11:02.571 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 55 2024-05-28T14:11:02.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.588+0000 7f3a82200640 1 -- 172.21.15.42:0/2238910502 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3a58069850 msgr2=0x7f3a5806bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.588+0000 7f3a82200640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3a58069850 0x7f3a5806bd10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0af000 tx=0x7f3a6c0a7070 comp rx=0 tx=0).stop 2024-05-28T14:11:02.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.588+0000 7f3a82200640 1 -- 172.21.15.42:0/2238910502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a7c0cde30 msgr2=0x7f3a7c14ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.588+0000 7f3a82200640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a7c0cde30 0x7f3a7c14ca20 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f3a74005890 tx=0x7f3a7400cbe0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.588+0000 7f3a82200640 1 -- 172.21.15.42:0/2238910502 shutdown_connections 2024-05-28T14:11:02.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.588+0000 7f3a82200640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3a58069850 0x7f3a5806bd10 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.588+0000 7f3a82200640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a7c0cde30 0x7f3a7c14ca20 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.588+0000 7f3a82200640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a7c0b2bf0 0x7f3a7c1481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.588+0000 7f3a82200640 1 --2- 172.21.15.42:0/2238910502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a7c0b1f30 0x7f3a7c070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.588+0000 7f3a82200640 1 -- 172.21.15.42:0/2238910502 >> 172.21.15.42:0/2238910502 conn(0x7f3a7c0b68f0 msgr2=0x7f3a7c0d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:02.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.588+0000 7f3a82200640 1 -- 172.21.15.42:0/2238910502 shutdown_connections 2024-05-28T14:11:02.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.588+0000 7f3a82200640 1 -- 172.21.15.42:0/2238910502 wait complete. 2024-05-28T14:11:02.592 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:11:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8eb7a00640 1 Processor -- start 2024-05-28T14:11:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8eb7a00640 1 -- start start 2024-05-28T14:11:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8eb7a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8eb006c100 0x7f8eb00ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8eb7a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8eb00b3aa0 0x7f8eb00d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8eb7a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb006c700 con 0x7f8eb015c970 2024-05-28T14:11:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8eb7a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb01611b0 con 0x7f8eb006c100 2024-05-28T14:11:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8eb7a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb0160350 con 0x7f8eb00b3aa0 2024-05-28T14:11:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8ea7e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8eb006c100 0x7f8eb00ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8ea7e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8eb006c100 0x7f8eb00ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:35526/0 (socket says 172.21.15.42:35526) 2024-05-28T14:11:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8ea7e00640 1 -- 172.21.15.42:0/2593934692 learned_addr learned my addr 172.21.15.42:0/2593934692 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8eb5a00640 1 --2- 172.21.15.42:0/2593934692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8eb00b3aa0 0x7f8eb00d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8ea7e00640 1 -- 172.21.15.42:0/2593934692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8eb00b3aa0 msgr2=0x7f8eb00d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8ea7e00640 1 --2- 172.21.15.42:0/2593934692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8eb00b3aa0 0x7f8eb00d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8ea7e00640 1 -- 172.21.15.42:0/2593934692 >> v1:172.21.15.63:6789/0 conn(0x7f8eb015c970 legacy=0x7f8eb006b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.752+0000 7f8ea7e00640 1 -- 172.21.15.42:0/2593934692 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb01605a0 con 0x7f8eb006c100 2024-05-28T14:11:02.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.756+0000 7f8ea7e00640 1 --2- 172.21.15.42:0/2593934692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eb006c100 0x7f8eb00ccfa0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009870 tx=0x7f8e9801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c1eefe7d5b76c56 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:02.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.756+0000 7f8eafe00640 1 -- 172.21.15.42:0/2593934692 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e980b2070 con 0x7f8eb006c100 2024-05-28T14:11:02.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.756+0000 7f8eafe00640 1 -- 172.21.15.42:0/2593934692 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e980a9070 con 0x7f8eb006c100 2024-05-28T14:11:02.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.756+0000 7f8eafe00640 1 -- 172.21.15.42:0/2593934692 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e980ad680 con 0x7f8eb006c100 2024-05-28T14:11:02.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.756+0000 7f8eb7a00640 1 -- 172.21.15.42:0/2593934692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eb006c100 msgr2=0x7f8eb00ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.756+0000 7f8eb7a00640 1 --2- 172.21.15.42:0/2593934692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eb006c100 0x7f8eb00ccfa0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009870 tx=0x7f8e9801c560 comp rx=0 tx=0).stop 2024-05-28T14:11:02.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.756+0000 7f8eb7a00640 1 -- 172.21.15.42:0/2593934692 shutdown_connections 2024-05-28T14:11:02.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.756+0000 7f8eb7a00640 1 --2- 172.21.15.42:0/2593934692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8eb00b3aa0 0x7f8eb00d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.756+0000 7f8eb7a00640 1 --2- 172.21.15.42:0/2593934692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eb006c100 0x7f8eb00ccfa0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.756+0000 7f8eb7a00640 1 -- 172.21.15.42:0/2593934692 >> 172.21.15.42:0/2593934692 conn(0x7f8eb00b2f80 msgr2=0x7f8eb00b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:02.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.756+0000 7f8eb7a00640 1 -- 172.21.15.42:0/2593934692 shutdown_connections 2024-05-28T14:11:02.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.756+0000 7f8eb7a00640 1 -- 172.21.15.42:0/2593934692 wait complete. 2024-05-28T14:11:02.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb7a00640 1 Processor -- start 2024-05-28T14:11:02.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb7a00640 1 -- start start 2024-05-28T14:11:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb7a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eb006c100 0x7f8eb014e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb7a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8eb00b3aa0 0x7f8eb015ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb7a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8eb015f4c0 0x7f8eb00b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb7a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb0169870 con 0x7f8eb015f4c0 2024-05-28T14:11:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb7a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb01696f0 con 0x7f8eb006c100 2024-05-28T14:11:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb7a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb01699f0 con 0x7f8eb00b3aa0 2024-05-28T14:11:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8eb015f4c0 0x7f8eb00b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:02.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8eb015f4c0 0x7f8eb00b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52692/0 (socket says 172.21.15.42:52692) 2024-05-28T14:11:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb5a00640 1 -- 172.21.15.42:0/1410234699 learned_addr learned my addr 172.21.15.42:0/1410234699 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb5000640 1 --2- 172.21.15.42:0/1410234699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eb006c100 0x7f8eb014e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8ea7e00640 1 --2- 172.21.15.42:0/1410234699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8eb00b3aa0 0x7f8eb015ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb5a00640 1 -- 172.21.15.42:0/1410234699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8eb00b3aa0 msgr2=0x7f8eb015ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb5a00640 1 --2- 172.21.15.42:0/1410234699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8eb00b3aa0 0x7f8eb015ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb5a00640 1 -- 172.21.15.42:0/1410234699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eb006c100 msgr2=0x7f8eb014e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb5a00640 1 --2- 172.21.15.42:0/1410234699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eb006c100 0x7f8eb014e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb5a00640 1 -- 172.21.15.42:0/1410234699 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb00c6380 con 0x7f8eb015f4c0 2024-05-28T14:11:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb5a00640 1 --2- 172.21.15.42:0/1410234699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8eb015f4c0 0x7f8eb00b4100 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009dd0 tx=0x7f8ea800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8ead600640 1 -- 172.21.15.42:0/1410234699 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8018020 con 0x7f8eb015f4c0 2024-05-28T14:11:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8ead600640 1 -- 172.21.15.42:0/1410234699 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea8014480 con 0x7f8eb015f4c0 2024-05-28T14:11:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb7a00640 1 -- 172.21.15.42:0/1410234699 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb014edb0 con 0x7f8eb015f4c0 2024-05-28T14:11:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8ead600640 1 -- 172.21.15.42:0/1410234699 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8014870 con 0x7f8eb015f4c0 2024-05-28T14:11:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.760+0000 7f8eb7a00640 1 -- 172.21.15.42:0/1410234699 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb00b50d0 con 0x7f8eb015f4c0 2024-05-28T14:11:02.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.764+0000 7f8eb7a00640 1 -- 172.21.15.42:0/1410234699 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb00c6550 con 0x7f8eb015f4c0 2024-05-28T14:11:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.764+0000 7f8ead600640 1 -- 172.21.15.42:0/1410234699 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f8ea801b020 con 0x7f8eb015f4c0 2024-05-28T14:11:02.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.764+0000 7f8ead600640 1 --2- 172.21.15.42:0/1410234699 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e8c069880 0x7f8e8c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:02.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.764+0000 7f8ead600640 1 -- 172.21.15.42:0/1410234699 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5823+0+0 (secure 0 0 0) 0x7f8ea8086370 con 0x7f8eb015f4c0 2024-05-28T14:11:02.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.764+0000 7f8eb5000640 1 --2- 172.21.15.42:0/1410234699 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e8c069880 0x7f8e8c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:02.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.764+0000 7f8eb5000640 1 --2- 172.21.15.42:0/1410234699 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e8c069880 0x7f8e8c06bd40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0007fd0 tx=0x7f8ea0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:02.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.768+0000 7f8ead600640 1 -- 172.21.15.42:0/1410234699 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ea8022030 con 0x7f8eb015f4c0 2024-05-28T14:11:02.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.936+0000 7f8eb7a00640 1 -- 172.21.15.42:0/1410234699 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8eb0068a90 con 0x7f8eb015f4c0 2024-05-28T14:11:02.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.940+0000 7f8ead600640 1 -- 172.21.15.42:0/1410234699 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16807 (secure 0 0 0) 0x7f8ea8010040 con 0x7f8eb015f4c0 2024-05-28T14:11:02.925 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:02.925 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":94,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:11:02.637108+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:09:28.913704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-28T14:09:30.108327+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":93,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.3e","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:11:02.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.940+0000 7f8eb7a00640 1 -- 172.21.15.42:0/1410234699 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e8c069880 msgr2=0x7f8e8c06bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.940+0000 7f8eb7a00640 1 --2- 172.21.15.42:0/1410234699 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e8c069880 0x7f8e8c06bd40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0007fd0 tx=0x7f8ea0009040 comp rx=0 tx=0).stop 2024-05-28T14:11:02.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.940+0000 7f8eb7a00640 1 -- 172.21.15.42:0/1410234699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8eb015f4c0 msgr2=0x7f8eb00b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:02.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.940+0000 7f8eb7a00640 1 --2- 172.21.15.42:0/1410234699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8eb015f4c0 0x7f8eb00b4100 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009dd0 tx=0x7f8ea800c490 comp rx=0 tx=0).stop 2024-05-28T14:11:02.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.940+0000 7f8eb7a00640 1 -- 172.21.15.42:0/1410234699 shutdown_connections 2024-05-28T14:11:02.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.940+0000 7f8eb7a00640 1 --2- 172.21.15.42:0/1410234699 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8e8c069880 0x7f8e8c06bd40 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.940+0000 7f8eb7a00640 1 --2- 172.21.15.42:0/1410234699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8eb015f4c0 0x7f8eb00b4100 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.940+0000 7f8eb7a00640 1 --2- 172.21.15.42:0/1410234699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8eb00b3aa0 0x7f8eb015ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.940+0000 7f8eb7a00640 1 --2- 172.21.15.42:0/1410234699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8eb006c100 0x7f8eb014e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:02.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.940+0000 7f8eb7a00640 1 -- 172.21.15.42:0/1410234699 >> 172.21.15.42:0/1410234699 conn(0x7f8eb00b2f80 msgr2=0x7f8eb00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:02.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.940+0000 7f8eb7a00640 1 -- 172.21.15.42:0/1410234699 shutdown_connections 2024-05-28T14:11:02.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:02.940+0000 7f8eb7a00640 1 -- 172.21.15.42:0/1410234699 wait complete. 2024-05-28T14:11:02.947 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-28T14:11:03.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.104+0000 7feecdc00640 1 Processor -- start 2024-05-28T14:11:03.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.104+0000 7feecdc00640 1 -- start start 2024-05-28T14:11:03.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.108+0000 7feecdc00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feec80724d0 0x7feec806f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:03.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.108+0000 7feecdc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feec806db40 0x7feec815d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:03.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.108+0000 7feecdc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feec806df90 con 0x7feec8145920 2024-05-28T14:11:03.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.108+0000 7feecdc00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7feec806e240 con 0x7feec80724d0 2024-05-28T14:11:03.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.108+0000 7feecdc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feec806e520 con 0x7feec806db40 2024-05-28T14:11:03.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.108+0000 7feebe000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feec80724d0 0x7feec806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:03.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.108+0000 7feec7400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feec806db40 0x7feec815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:03.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.108+0000 7feebe000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feec80724d0 0x7feec806f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:35550/0 (socket says 172.21.15.42:35550) 2024-05-28T14:11:03.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.108+0000 7feebe000640 1 -- 172.21.15.42:0/3372794493 learned_addr learned my addr 172.21.15.42:0/3372794493 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:03.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.108+0000 7feebe000640 1 -- 172.21.15.42:0/3372794493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feec806db40 msgr2=0x7feec815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:03.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.108+0000 7feebe000640 1 --2- 172.21.15.42:0/3372794493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feec806db40 0x7feec815d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:03.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.108+0000 7feebe000640 1 -- 172.21.15.42:0/3372794493 >> v1:172.21.15.63:6789/0 conn(0x7feec8145920 legacy=0x7feec8071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:03.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.108+0000 7feebe000640 1 -- 172.21.15.42:0/3372794493 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec806e800 con 0x7feec80724d0 2024-05-28T14:11:03.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.108+0000 7feec7400640 1 --2- 172.21.15.42:0/3372794493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feec806db40 0x7feec815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:11:03.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.112+0000 7feebe000640 1 --2- 172.21.15.42:0/3372794493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feec80724d0 0x7feec806f030 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009870 tx=0x7feeb001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa81bcf82d3643c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:03.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.112+0000 7feec6000640 1 -- 172.21.15.42:0/3372794493 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb00b0070 con 0x7feec80724d0 2024-05-28T14:11:03.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.112+0000 7feec6000640 1 -- 172.21.15.42:0/3372794493 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb00a4e50 con 0x7feec80724d0 2024-05-28T14:11:03.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.112+0000 7feec6000640 1 -- 172.21.15.42:0/3372794493 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb00ab820 con 0x7feec80724d0 2024-05-28T14:11:03.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.112+0000 7feecdc00640 1 -- 172.21.15.42:0/3372794493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feec80724d0 msgr2=0x7feec806f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:03.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.112+0000 7feecdc00640 1 --2- 172.21.15.42:0/3372794493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feec80724d0 0x7feec806f030 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009870 tx=0x7feeb001c4d0 comp rx=0 tx=0).stop 2024-05-28T14:11:03.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.112+0000 7feecdc00640 1 -- 172.21.15.42:0/3372794493 shutdown_connections 2024-05-28T14:11:03.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.112+0000 7feecdc00640 1 --2- 172.21.15.42:0/3372794493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feec806db40 0x7feec815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:03.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.112+0000 7feecdc00640 1 --2- 172.21.15.42:0/3372794493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feec80724d0 0x7feec806f030 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:03.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.112+0000 7feecdc00640 1 -- 172.21.15.42:0/3372794493 >> 172.21.15.42:0/3372794493 conn(0x7feec806b9f0 msgr2=0x7feec806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:03.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.112+0000 7feecdc00640 1 -- 172.21.15.42:0/3372794493 shutdown_connections 2024-05-28T14:11:03.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.112+0000 7feecdc00640 1 -- 172.21.15.42:0/3372794493 wait complete. 2024-05-28T14:11:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.112+0000 7feecdc00640 1 Processor -- start 2024-05-28T14:11:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.112+0000 7feecdc00640 1 -- start start 2024-05-28T14:11:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feecdc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec806db40 0x7feec81f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feecdc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feec80724d0 0x7feec8163bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feecdc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feec8145920 0x7feec80ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feecdc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feec815e240 con 0x7feec806db40 2024-05-28T14:11:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feecdc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feec815e0c0 con 0x7feec80724d0 2024-05-28T14:11:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feecdc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feec815e3c0 con 0x7feec8145920 2024-05-28T14:11:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feec6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec806db40 0x7feec81f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:03.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feec6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec806db40 0x7feec81f7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52694/0 (socket says 172.21.15.42:52694) 2024-05-28T14:11:03.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feec6a00640 1 -- 172.21.15.42:0/5205999 learned_addr learned my addr 172.21.15.42:0/5205999 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:03.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feebe000640 1 --2- 172.21.15.42:0/5205999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feec80724d0 0x7feec8163bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:03.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feec7400640 1 --2- 172.21.15.42:0/5205999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feec8145920 0x7feec80ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:03.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feec6a00640 1 -- 172.21.15.42:0/5205999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feec8145920 msgr2=0x7feec80ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:03.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feec6a00640 1 --2- 172.21.15.42:0/5205999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feec8145920 0x7feec80ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:03.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feec6a00640 1 -- 172.21.15.42:0/5205999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feec80724d0 msgr2=0x7feec8163bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:03.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feec6a00640 1 --2- 172.21.15.42:0/5205999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feec80724d0 0x7feec8163bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:03.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feec6a00640 1 -- 172.21.15.42:0/5205999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec81640f0 con 0x7feec806db40 2024-05-28T14:11:03.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feec6a00640 1 --2- 172.21.15.42:0/5205999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec806db40 0x7feec81f7840 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c450 tx=0x7feeb800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:03.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feebf400640 1 -- 172.21.15.42:0/5205999 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb80052d0 con 0x7feec806db40 2024-05-28T14:11:03.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feebf400640 1 -- 172.21.15.42:0/5205999 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb8005470 con 0x7feec806db40 2024-05-28T14:11:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feebf400640 1 -- 172.21.15.42:0/5205999 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8007260 con 0x7feec806db40 2024-05-28T14:11:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feecdc00640 1 -- 172.21.15.42:0/5205999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec81626d0 con 0x7feec806db40 2024-05-28T14:11:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feecdc00640 1 -- 172.21.15.42:0/5205999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec8162c90 con 0x7feec806db40 2024-05-28T14:11:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feecdc00640 1 -- 172.21.15.42:0/5205999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec8073890 con 0x7feec806db40 2024-05-28T14:11:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.116+0000 7feebf400640 1 -- 172.21.15.42:0/5205999 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7feeb8007400 con 0x7feec806db40 2024-05-28T14:11:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.120+0000 7feebf400640 1 --2- 172.21.15.42:0/5205999 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feea4069850 0x7feea406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.120+0000 7feebf400640 1 -- 172.21.15.42:0/5205999 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5823+0+0 (secure 0 0 0) 0x7feeb8084e90 con 0x7feec806db40 2024-05-28T14:11:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.120+0000 7feebe000640 1 --2- 172.21.15.42:0/5205999 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feea4069850 0x7feea406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.120+0000 7feebe000640 1 --2- 172.21.15.42:0/5205999 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feea4069850 0x7feea406bd10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7feeb00ad000 tx=0x7feeb0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.124+0000 7feebf400640 1 -- 172.21.15.42:0/5205999 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feeb8056470 con 0x7feec806db40 2024-05-28T14:11:03.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.324+0000 7feecdc00640 1 -- 172.21.15.42:0/5205999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7feec80aff40 con 0x7feec806db40 2024-05-28T14:11:03.308 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-05-28T14:11:03.324+0000 7f248b600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-28T14:11:03.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.324+0000 7feebf400640 1 -- 172.21.15.42:0/5205999 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) ==== 161+0+0 (secure 0 0 0) 0x7feeb805a770 con 0x7feec806db40 2024-05-28T14:11:03.310 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-28T14:11:03.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.328+0000 7feecdc00640 1 -- 172.21.15.42:0/5205999 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feea4069850 msgr2=0x7feea406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:03.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.328+0000 7feecdc00640 1 --2- 172.21.15.42:0/5205999 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feea4069850 0x7feea406bd10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7feeb00ad000 tx=0x7feeb0008040 comp rx=0 tx=0).stop 2024-05-28T14:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.328+0000 7feecdc00640 1 -- 172.21.15.42:0/5205999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec806db40 msgr2=0x7feec81f7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.328+0000 7feecdc00640 1 --2- 172.21.15.42:0/5205999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec806db40 0x7feec81f7840 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c450 tx=0x7feeb800c850 comp rx=0 tx=0).stop 2024-05-28T14:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.328+0000 7feecdc00640 1 -- 172.21.15.42:0/5205999 shutdown_connections 2024-05-28T14:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.328+0000 7feecdc00640 1 --2- 172.21.15.42:0/5205999 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feea4069850 0x7feea406bd10 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.328+0000 7feecdc00640 1 --2- 172.21.15.42:0/5205999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feec8145920 0x7feec80ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.328+0000 7feecdc00640 1 --2- 172.21.15.42:0/5205999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feec80724d0 0x7feec8163bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.328+0000 7feecdc00640 1 --2- 172.21.15.42:0/5205999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec806db40 0x7feec81f7840 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.328+0000 7feecdc00640 1 -- 172.21.15.42:0/5205999 >> 172.21.15.42:0/5205999 conn(0x7feec806b9f0 msgr2=0x7feec806bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:03.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.328+0000 7feecdc00640 1 -- 172.21.15.42:0/5205999 shutdown_connections 2024-05-28T14:11:03.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.328+0000 7feecdc00640 1 -- 172.21.15.42:0/5205999 wait complete. 2024-05-28T14:11:03.334 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-28T14:11:03.388 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T14:11:03.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.496+0000 7f6829600640 1 Processor -- start 2024-05-28T14:11:03.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.496+0000 7f6829600640 1 -- start start 2024-05-28T14:11:03.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.496+0000 7f6829600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68240cfec0 0x7f68240d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:03.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.496+0000 7f6829600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6824072530 con 0x7f6824143f90 2024-05-28T14:11:03.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.496+0000 7f6829600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68240d07e0 con 0x7f682406dfa0 2024-05-28T14:11:03.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.496+0000 7f6829600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f682406cb00 con 0x7f68240cfec0 2024-05-28T14:11:03.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.496+0000 7f681be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f6824143f90 0x7f6824071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42972/0 (socket says 172.21.15.42:42972) 2024-05-28T14:11:03.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.496+0000 7f681be00640 1 -- 172.21.15.42:0/1659038344 learned_addr learned my addr 172.21.15.42:0/1659038344 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:03.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.496+0000 7f6823e00640 1 --2- 172.21.15.42:0/1659038344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68240cfec0 0x7f68240d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:03.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.496+0000 7f6823400640 1 -- 172.21.15.42:0/1659038344 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 835010717 0 0) 0x7f6824072530 con 0x7f6824143f90 2024-05-28T14:11:03.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.496+0000 7f6823400640 1 -- 172.21.15.42:0/1659038344 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6808003530 con 0x7f6824143f90 2024-05-28T14:11:03.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.500+0000 7f6823400640 1 -- 172.21.15.42:0/1659038344 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2280197826 0 0) 0x7f68240d07e0 con 0x7f682406dfa0 2024-05-28T14:11:03.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.500+0000 7f6823400640 1 -- 172.21.15.42:0/1659038344 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6824072530 con 0x7f682406dfa0 2024-05-28T14:11:03.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.500+0000 7f6823e00640 1 -- 172.21.15.42:0/1659038344 >> v1:172.21.15.63:6789/0 conn(0x7f682406dfa0 legacy=0x7f6824071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:03.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.500+0000 7f6823e00640 1 -- 172.21.15.42:0/1659038344 >> v1:172.21.15.42:6789/0 conn(0x7f6824143f90 legacy=0x7f6824071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:03.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.500+0000 7f6823e00640 1 -- 172.21.15.42:0/1659038344 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6824144540 con 0x7f68240cfec0 2024-05-28T14:11:03.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.500+0000 7f6823e00640 1 --2- 172.21.15.42:0/1659038344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68240cfec0 0x7f68240d02a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f68140097f0 tx=0x7f681401c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7990efba4f1543 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:03.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.500+0000 7f6823400640 1 -- 172.21.15.42:0/1659038344 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68140b1070 con 0x7f68240cfec0 2024-05-28T14:11:03.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.500+0000 7f6823400640 1 -- 172.21.15.42:0/1659038344 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68140ac410 con 0x7f68240cfec0 2024-05-28T14:11:03.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.500+0000 7f6823400640 1 -- 172.21.15.42:0/1659038344 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68140ac7e0 con 0x7f68240cfec0 2024-05-28T14:11:03.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.500+0000 7f6829600640 1 -- 172.21.15.42:0/1659038344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68240cfec0 msgr2=0x7f68240d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:03.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.500+0000 7f6829600640 1 --2- 172.21.15.42:0/1659038344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68240cfec0 0x7f68240d02a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f68140097f0 tx=0x7f681401c390 comp rx=0 tx=0).stop 2024-05-28T14:11:03.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.500+0000 7f6829600640 1 -- 172.21.15.42:0/1659038344 shutdown_connections 2024-05-28T14:11:03.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.500+0000 7f6829600640 1 --2- 172.21.15.42:0/1659038344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68240cfec0 0x7f68240d02a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:03.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.500+0000 7f6829600640 1 -- 172.21.15.42:0/1659038344 >> 172.21.15.42:0/1659038344 conn(0x7f682406fcc0 msgr2=0x7f68240700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:03.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6829600640 1 -- 172.21.15.42:0/1659038344 shutdown_connections 2024-05-28T14:11:03.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6829600640 1 -- 172.21.15.42:0/1659038344 wait complete. 2024-05-28T14:11:03.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6829600640 1 Processor -- start 2024-05-28T14:11:03.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6829600640 1 -- start start 2024-05-28T14:11:03.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6829600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f682406dfa0 0x7f68240ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:03.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6829600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68240cfec0 0x7f68240c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6829600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6824143f90 0x7f68240c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6829600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6824148310 con 0x7f6824143f90 2024-05-28T14:11:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6829600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6824148190 con 0x7f682406dfa0 2024-05-28T14:11:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6829600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6824161e60 con 0x7f68240cfec0 2024-05-28T14:11:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6828c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6824143f90 0x7f68240c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6823e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f682406dfa0 0x7f68240ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6828c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6824143f90 0x7f68240c5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52706/0 (socket says 172.21.15.42:52706) 2024-05-28T14:11:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6828c00640 1 -- 172.21.15.42:0/2497087804 learned_addr learned my addr 172.21.15.42:0/2497087804 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f681be00640 1 --2- 172.21.15.42:0/2497087804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68240cfec0 0x7f68240c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6828c00640 1 -- 172.21.15.42:0/2497087804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68240cfec0 msgr2=0x7f68240c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6828c00640 1 --2- 172.21.15.42:0/2497087804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68240cfec0 0x7f68240c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6828c00640 1 -- 172.21.15.42:0/2497087804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f682406dfa0 msgr2=0x7f68240ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6828c00640 1 --2- 172.21.15.42:0/2497087804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f682406dfa0 0x7f68240ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:03.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6828c00640 1 -- 172.21.15.42:0/2497087804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68240b2da0 con 0x7f6824143f90 2024-05-28T14:11:03.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6823e00640 1 --2- 172.21.15.42:0/2497087804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f682406dfa0 0x7f68240ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:11:03.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6828c00640 1 --2- 172.21.15.42:0/2497087804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6824143f90 0x7f68240c5ce0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f681c00c6f0 tx=0x7f681c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:03.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6820c00640 1 -- 172.21.15.42:0/2497087804 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c003980 con 0x7f6824143f90 2024-05-28T14:11:03.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6820c00640 1 -- 172.21.15.42:0/2497087804 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f681c003b20 con 0x7f6824143f90 2024-05-28T14:11:03.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6820c00640 1 -- 172.21.15.42:0/2497087804 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c00a850 con 0x7f6824143f90 2024-05-28T14:11:03.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.504+0000 7f6829600640 1 -- 172.21.15.42:0/2497087804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68240cafd0 con 0x7f6824143f90 2024-05-28T14:11:03.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.508+0000 7f6829600640 1 -- 172.21.15.42:0/2497087804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f682415f780 con 0x7f6824143f90 2024-05-28T14:11:03.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.508+0000 7f6829600640 1 -- 172.21.15.42:0/2497087804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f682404fcf0 con 0x7f6824143f90 2024-05-28T14:11:03.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.508+0000 7f6820c00640 1 -- 172.21.15.42:0/2497087804 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f681c00a9f0 con 0x7f6824143f90 2024-05-28T14:11:03.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.508+0000 7f6820c00640 1 --2- 172.21.15.42:0/2497087804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6800069850 0x7f680006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:03.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.508+0000 7f6820c00640 1 -- 172.21.15.42:0/2497087804 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5881+0+0 (secure 0 0 0) 0x7f681c0857b0 con 0x7f6824143f90 2024-05-28T14:11:03.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.508+0000 7f6823e00640 1 --2- 172.21.15.42:0/2497087804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6800069850 0x7f680006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.508+0000 7f6823e00640 1 --2- 172.21.15.42:0/2497087804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6800069850 0x7f680006bd10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f68140ae000 tx=0x7f6814008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:03.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.512+0000 7f6820c00640 1 -- 172.21.15.42:0/2497087804 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f681c056df0 con 0x7f6824143f90 2024-05-28T14:11:03.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:03.712+0000 7f6829600640 1 -- 172.21.15.42:0/2497087804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f68240cdfa0 con 0x7f6824143f90 2024-05-28T14:11:04.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.328+0000 7f6820c00640 1 -- 172.21.15.42:0/2497087804 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) ==== 104+0+0 (secure 0 0 0) 0x7f681c05b0f0 con 0x7f6824143f90 2024-05-28T14:11:04.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.332+0000 7f6829600640 1 -- 172.21.15.42:0/2497087804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6800069850 msgr2=0x7f680006bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:04.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.332+0000 7f6829600640 1 --2- 172.21.15.42:0/2497087804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6800069850 0x7f680006bd10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f68140ae000 tx=0x7f6814008040 comp rx=0 tx=0).stop 2024-05-28T14:11:04.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.332+0000 7f6829600640 1 -- 172.21.15.42:0/2497087804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6824143f90 msgr2=0x7f68240c5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:04.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.332+0000 7f6829600640 1 --2- 172.21.15.42:0/2497087804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6824143f90 0x7f68240c5ce0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f681c00c6f0 tx=0x7f681c00cbb0 comp rx=0 tx=0).stop 2024-05-28T14:11:04.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.332+0000 7f6829600640 1 -- 172.21.15.42:0/2497087804 shutdown_connections 2024-05-28T14:11:04.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.332+0000 7f6829600640 1 --2- 172.21.15.42:0/2497087804 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6800069850 0x7f680006bd10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:04.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.332+0000 7f6829600640 1 --2- 172.21.15.42:0/2497087804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6824143f90 0x7f68240c5ce0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:04.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.332+0000 7f6829600640 1 --2- 172.21.15.42:0/2497087804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68240cfec0 0x7f68240c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:04.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.332+0000 7f6829600640 1 --2- 172.21.15.42:0/2497087804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f682406dfa0 0x7f68240ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:04.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.332+0000 7f6829600640 1 -- 172.21.15.42:0/2497087804 >> 172.21.15.42:0/2497087804 conn(0x7f682406fcc0 msgr2=0x7f68240b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:04.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.332+0000 7f6829600640 1 -- 172.21.15.42:0/2497087804 shutdown_connections 2024-05-28T14:11:04.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.332+0000 7f6829600640 1 -- 172.21.15.42:0/2497087804 wait complete. 2024-05-28T14:11:04.338 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-28T14:11:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.500+0000 7f425f000640 1 Processor -- start 2024-05-28T14:11:04.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.500+0000 7f425f000640 1 -- start start 2024-05-28T14:11:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.504+0000 7f425f000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42580afa70 0x7f425806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.504+0000 7f425f000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f42580d06f0 0x7f42580d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.504+0000 7f425f000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42581570a0 con 0x7f425806ee80 2024-05-28T14:11:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.504+0000 7f425f000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f425815cf00 con 0x7f42580d06f0 2024-05-28T14:11:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.504+0000 7f425f000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f425815d1e0 con 0x7f42580afa70 2024-05-28T14:11:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.504+0000 7f425d000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f42580d06f0 0x7f42580d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.504+0000 7f425d000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f42580d06f0 0x7f42580d0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52718/0 (socket says 172.21.15.42:52718) 2024-05-28T14:11:04.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.504+0000 7f425d000640 1 -- 172.21.15.42:0/2821262064 learned_addr learned my addr 172.21.15.42:0/2821262064 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:04.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.504+0000 7f424f400640 1 --2- 172.21.15.42:0/2821262064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42580afa70 0x7f425806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:04.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.504+0000 7f425d000640 1 -- 172.21.15.42:0/2821262064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42580afa70 msgr2=0x7f425806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:04.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.504+0000 7f425d000640 1 --2- 172.21.15.42:0/2821262064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42580afa70 0x7f425806dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:04.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.504+0000 7f425d000640 1 -- 172.21.15.42:0/2821262064 >> v1:172.21.15.63:6789/0 conn(0x7f425806ee80 legacy=0x7f42580af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:04.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.504+0000 7f425d000640 1 -- 172.21.15.42:0/2821262064 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425815d4c0 con 0x7f42580d06f0 2024-05-28T14:11:04.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425d000640 1 --2- 172.21.15.42:0/2821262064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42580d06f0 0x7f42580d0b80 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f4250009e50 tx=0x7f425001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4bd47bafa750fc server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:04.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f4257400640 1 -- 172.21.15.42:0/2821262064 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42500a5620 con 0x7f42580d06f0 2024-05-28T14:11:04.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f4257400640 1 -- 172.21.15.42:0/2821262064 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42500a57c0 con 0x7f42580d06f0 2024-05-28T14:11:04.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f4257400640 1 -- 172.21.15.42:0/2821262064 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42500a5b90 con 0x7f42580d06f0 2024-05-28T14:11:04.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 -- 172.21.15.42:0/2821262064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42580d06f0 msgr2=0x7f42580d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:04.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 --2- 172.21.15.42:0/2821262064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42580d06f0 0x7f42580d0b80 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f4250009e50 tx=0x7f425001c910 comp rx=0 tx=0).stop 2024-05-28T14:11:04.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 -- 172.21.15.42:0/2821262064 shutdown_connections 2024-05-28T14:11:04.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 --2- 172.21.15.42:0/2821262064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42580d06f0 0x7f42580d0b80 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:04.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 --2- 172.21.15.42:0/2821262064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42580afa70 0x7f425806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:04.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 -- 172.21.15.42:0/2821262064 >> 172.21.15.42:0/2821262064 conn(0x7f425806bdf0 msgr2=0x7f425806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:04.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 -- 172.21.15.42:0/2821262064 shutdown_connections 2024-05-28T14:11:04.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 -- 172.21.15.42:0/2821262064 wait complete. 2024-05-28T14:11:04.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 Processor -- start 2024-05-28T14:11:04.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 -- start start 2024-05-28T14:11:04.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f425806ee80 0x7f425814f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:04.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42580afa70 0x7f4258150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:04.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42580d06f0 0x7f42580ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:04.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4258160e10 con 0x7f42580d06f0 2024-05-28T14:11:04.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4258160c90 con 0x7f42580afa70 2024-05-28T14:11:04.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425f000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4258160f90 con 0x7f425806ee80 2024-05-28T14:11:04.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.508+0000 7f425d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42580d06f0 0x7f42580ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:04.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f4257e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f425806ee80 0x7f425814f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:04.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f425d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42580d06f0 0x7f42580ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52732/0 (socket says 172.21.15.42:52732) 2024-05-28T14:11:04.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f425d000640 1 -- 172.21.15.42:0/1810163718 learned_addr learned my addr 172.21.15.42:0/1810163718 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:04.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f424f400640 1 --2- 172.21.15.42:0/1810163718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42580afa70 0x7f4258150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:04.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f425d000640 1 -- 172.21.15.42:0/1810163718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f425806ee80 msgr2=0x7f425814f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:04.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f425d000640 1 --2- 172.21.15.42:0/1810163718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f425806ee80 0x7f425814f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:04.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f425d000640 1 -- 172.21.15.42:0/1810163718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42580afa70 msgr2=0x7f4258150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:04.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f425d000640 1 --2- 172.21.15.42:0/1810163718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42580afa70 0x7f4258150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:04.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f425d000640 1 -- 172.21.15.42:0/1810163718 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42580cf5e0 con 0x7f42580d06f0 2024-05-28T14:11:04.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f425d000640 1 --2- 172.21.15.42:0/1810163718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42580d06f0 0x7f42580ceb00 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f42500af040 tx=0x7f42500196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:04.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f4254c00640 1 -- 172.21.15.42:0/1810163718 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42500bc020 con 0x7f42580d06f0 2024-05-28T14:11:04.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f4254c00640 1 -- 172.21.15.42:0/1810163718 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42500b8670 con 0x7f42580d06f0 2024-05-28T14:11:04.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f4254c00640 1 -- 172.21.15.42:0/1810163718 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42500a5620 con 0x7f42580d06f0 2024-05-28T14:11:04.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f425f000640 1 -- 172.21.15.42:0/1810163718 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42580cf8c0 con 0x7f42580d06f0 2024-05-28T14:11:04.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f425f000640 1 -- 172.21.15.42:0/1810163718 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42580722c0 con 0x7f42580d06f0 2024-05-28T14:11:04.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f425f000640 1 -- 172.21.15.42:0/1810163718 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4258068a90 con 0x7f42580d06f0 2024-05-28T14:11:04.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.512+0000 7f4254c00640 1 -- 172.21.15.42:0/1810163718 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f42500bf020 con 0x7f42580d06f0 2024-05-28T14:11:04.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.516+0000 7f4254c00640 1 --2- 172.21.15.42:0/1810163718 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4234069850 0x7f423406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:04.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.516+0000 7f4254c00640 1 -- 172.21.15.42:0/1810163718 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5881+0+0 (secure 0 0 0) 0x7f42501203c0 con 0x7f42580d06f0 2024-05-28T14:11:04.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.516+0000 7f4257e00640 1 --2- 172.21.15.42:0/1810163718 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4234069850 0x7f423406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:04.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.516+0000 7f4257e00640 1 --2- 172.21.15.42:0/1810163718 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4234069850 0x7f423406bd10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4248001770 tx=0x7f4248009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:04.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.520+0000 7f4254c00640 1 -- 172.21.15.42:0/1810163718 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4250129020 con 0x7f42580d06f0 2024-05-28T14:11:04.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:04.684+0000 7f425f000640 1 -- 172.21.15.42:0/1810163718 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f42580b5160 con 0x7f42580d06f0 2024-05-28T14:11:05.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.332+0000 7f4254c00640 1 -- 172.21.15.42:0/1810163718 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7f42500c6030 con 0x7f42580d06f0 2024-05-28T14:11:05.315 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-05-28T14:11:05.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.336+0000 7f425f000640 1 -- 172.21.15.42:0/1810163718 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4234069850 msgr2=0x7f423406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:05.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.336+0000 7f425f000640 1 --2- 172.21.15.42:0/1810163718 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4234069850 0x7f423406bd10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4248001770 tx=0x7f4248009040 comp rx=0 tx=0).stop 2024-05-28T14:11:05.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.336+0000 7f425f000640 1 -- 172.21.15.42:0/1810163718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42580d06f0 msgr2=0x7f42580ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:05.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.336+0000 7f425f000640 1 --2- 172.21.15.42:0/1810163718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42580d06f0 0x7f42580ceb00 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f42500af040 tx=0x7f42500196b0 comp rx=0 tx=0).stop 2024-05-28T14:11:05.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.336+0000 7f425f000640 1 -- 172.21.15.42:0/1810163718 shutdown_connections 2024-05-28T14:11:05.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.336+0000 7f425f000640 1 --2- 172.21.15.42:0/1810163718 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4234069850 0x7f423406bd10 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:05.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.336+0000 7f425f000640 1 --2- 172.21.15.42:0/1810163718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42580d06f0 0x7f42580ceb00 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:05.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.336+0000 7f425f000640 1 --2- 172.21.15.42:0/1810163718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42580afa70 0x7f4258150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:05.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.336+0000 7f425f000640 1 --2- 172.21.15.42:0/1810163718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f425806ee80 0x7f425814f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:05.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.336+0000 7f425f000640 1 -- 172.21.15.42:0/1810163718 >> 172.21.15.42:0/1810163718 conn(0x7f425806bdf0 msgr2=0x7f425806f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:05.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.336+0000 7f425f000640 1 -- 172.21.15.42:0/1810163718 shutdown_connections 2024-05-28T14:11:05.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.336+0000 7f425f000640 1 -- 172.21.15.42:0/1810163718 wait complete. 2024-05-28T14:11:05.344 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-28T14:11:05.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.508+0000 7f1ad5a00640 1 Processor -- start 2024-05-28T14:11:05.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.508+0000 7f1ad5a00640 1 -- start start 2024-05-28T14:11:05.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.508+0000 7f1ad5a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1ad00afa70 0x7f1ad006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:05.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.508+0000 7f1ad5a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad00d06f0 0x7f1ad00d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:05.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.508+0000 7f1ad5a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ad01570a0 con 0x7f1ad006ee80 2024-05-28T14:11:05.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.508+0000 7f1ad5a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1ad015cf00 con 0x7f1ad00afa70 2024-05-28T14:11:05.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.508+0000 7f1ad5a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad015d1e0 con 0x7f1ad00d06f0 2024-05-28T14:11:05.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.508+0000 7f1ac7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1ad00afa70 0x7f1ad006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:05.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.508+0000 7f1ac7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1ad00afa70 0x7f1ad006dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52740/0 (socket says 172.21.15.42:52740) 2024-05-28T14:11:05.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.508+0000 7f1ac7e00640 1 -- 172.21.15.42:0/1992859991 learned_addr learned my addr 172.21.15.42:0/1992859991 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:05.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.508+0000 7f1acf400640 1 --2- 172.21.15.42:0/1992859991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad00d06f0 0x7f1ad00d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:05.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ac7e00640 1 -- 172.21.15.42:0/1992859991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad00d06f0 msgr2=0x7f1ad00d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:05.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ac7e00640 1 --2- 172.21.15.42:0/1992859991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad00d06f0 0x7f1ad00d0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:05.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ac7e00640 1 -- 172.21.15.42:0/1992859991 >> v1:172.21.15.63:6789/0 conn(0x7f1ad006ee80 legacy=0x7f1ad00af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:05.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ac7e00640 1 -- 172.21.15.42:0/1992859991 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad015d4c0 con 0x7f1ad00afa70 2024-05-28T14:11:05.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ac7e00640 1 --2- 172.21.15.42:0/1992859991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad00afa70 0x7f1ad006dd40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009870 tx=0x7f1ab801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7b6c8058357c57 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:05.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ace000640 1 -- 172.21.15.42:0/1992859991 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80b0070 con 0x7f1ad00afa70 2024-05-28T14:11:05.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ace000640 1 -- 172.21.15.42:0/1992859991 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab80ab600 con 0x7f1ad00afa70 2024-05-28T14:11:05.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ace000640 1 -- 172.21.15.42:0/1992859991 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80ab9d0 con 0x7f1ad00afa70 2024-05-28T14:11:05.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ad5a00640 1 -- 172.21.15.42:0/1992859991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad00afa70 msgr2=0x7f1ad006dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:05.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ad5a00640 1 --2- 172.21.15.42:0/1992859991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad00afa70 0x7f1ad006dd40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009870 tx=0x7f1ab801c440 comp rx=0 tx=0).stop 2024-05-28T14:11:05.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ad5a00640 1 -- 172.21.15.42:0/1992859991 shutdown_connections 2024-05-28T14:11:05.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ad5a00640 1 --2- 172.21.15.42:0/1992859991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad00d06f0 0x7f1ad00d0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:05.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ad5a00640 1 --2- 172.21.15.42:0/1992859991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad00afa70 0x7f1ad006dd40 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:05.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ad5a00640 1 -- 172.21.15.42:0/1992859991 >> 172.21.15.42:0/1992859991 conn(0x7f1ad006bdf0 msgr2=0x7f1ad006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:05.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ad5a00640 1 -- 172.21.15.42:0/1992859991 shutdown_connections 2024-05-28T14:11:05.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.512+0000 7f1ad5a00640 1 -- 172.21.15.42:0/1992859991 wait complete. 2024-05-28T14:11:05.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1ad5a00640 1 Processor -- start 2024-05-28T14:11:05.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1ad5a00640 1 -- start start 2024-05-28T14:11:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1ad5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad006ee80 0x7f1ad0150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1ad5a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad00afa70 0x7f1ad0155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1ad5a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad00d06f0 0x7f1ad009db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1ad5a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad0160e10 con 0x7f1ad006ee80 2024-05-28T14:11:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1ad5a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad0160c90 con 0x7f1ad00afa70 2024-05-28T14:11:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1ad5a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad0160f90 con 0x7f1ad00d06f0 2024-05-28T14:11:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1acea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad006ee80 0x7f1ad0150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1acea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad006ee80 0x7f1ad0150560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52754/0 (socket says 172.21.15.42:52754) 2024-05-28T14:11:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1acea00640 1 -- 172.21.15.42:0/119215602 learned_addr learned my addr 172.21.15.42:0/119215602 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1acf400640 1 --2- 172.21.15.42:0/119215602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad00d06f0 0x7f1ad009db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1ac7e00640 1 --2- 172.21.15.42:0/119215602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad00afa70 0x7f1ad0155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1acea00640 1 -- 172.21.15.42:0/119215602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad00d06f0 msgr2=0x7f1ad009db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1acea00640 1 --2- 172.21.15.42:0/119215602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad00d06f0 0x7f1ad009db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1acea00640 1 -- 172.21.15.42:0/119215602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad00afa70 msgr2=0x7f1ad0155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1acea00640 1 --2- 172.21.15.42:0/119215602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad00afa70 0x7f1ad0155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1acea00640 1 -- 172.21.15.42:0/119215602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad009cab0 con 0x7f1ad006ee80 2024-05-28T14:11:05.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1acea00640 1 --2- 172.21.15.42:0/119215602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad006ee80 0x7f1ad0150560 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c420 tx=0x7f1ac000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:05.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1ac6a00640 1 -- 172.21.15.42:0/119215602 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00052d0 con 0x7f1ad006ee80 2024-05-28T14:11:05.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1ac6a00640 1 -- 172.21.15.42:0/119215602 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac0005470 con 0x7f1ad006ee80 2024-05-28T14:11:05.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1ac6a00640 1 -- 172.21.15.42:0/119215602 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0007260 con 0x7f1ad006ee80 2024-05-28T14:11:05.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1ad5a00640 1 -- 172.21.15.42:0/119215602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad009d020 con 0x7f1ad006ee80 2024-05-28T14:11:05.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.516+0000 7f1ad5a00640 1 -- 172.21.15.42:0/119215602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad009d5e0 con 0x7f1ad006ee80 2024-05-28T14:11:05.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.520+0000 7f1ad5a00640 1 -- 172.21.15.42:0/119215602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad0068a90 con 0x7f1ad006ee80 2024-05-28T14:11:05.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.520+0000 7f1ac6a00640 1 -- 172.21.15.42:0/119215602 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f1ac0007400 con 0x7f1ad006ee80 2024-05-28T14:11:05.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.520+0000 7f1ac6a00640 1 --2- 172.21.15.42:0/119215602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1aac069850 0x7f1aac06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:05.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.520+0000 7f1ac6a00640 1 -- 172.21.15.42:0/119215602 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5443+0+0 (secure 0 0 0) 0x7f1ac0084f20 con 0x7f1ad006ee80 2024-05-28T14:11:05.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.520+0000 7f1ac7e00640 1 --2- 172.21.15.42:0/119215602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1aac069850 0x7f1aac06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:05.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.520+0000 7f1ac7e00640 1 --2- 172.21.15.42:0/119215602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1aac069850 0x7f1aac06bd10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80ad000 tx=0x7f1ab80a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:05.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.524+0000 7f1ac6a00640 1 -- 172.21.15.42:0/119215602 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ac00853c0 con 0x7f1ad006ee80 2024-05-28T14:11:05.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:05.700+0000 7f1ad5a00640 1 -- 172.21.15.42:0/119215602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1ad009d7b0 con 0x7f1ad006ee80 2024-05-28T14:11:06.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.332+0000 7f1ac6a00640 1 -- 172.21.15.42:0/119215602 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7f1ac0056500 con 0x7f1ad006ee80 2024-05-28T14:11:06.317 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-05-28T14:11:06.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.336+0000 7f1ad5a00640 1 -- 172.21.15.42:0/119215602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1aac069850 msgr2=0x7f1aac06bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.336+0000 7f1ad5a00640 1 --2- 172.21.15.42:0/119215602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1aac069850 0x7f1aac06bd10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80ad000 tx=0x7f1ab80a6000 comp rx=0 tx=0).stop 2024-05-28T14:11:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.336+0000 7f1ad5a00640 1 -- 172.21.15.42:0/119215602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad006ee80 msgr2=0x7f1ad0150560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.336+0000 7f1ad5a00640 1 --2- 172.21.15.42:0/119215602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad006ee80 0x7f1ad0150560 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c420 tx=0x7f1ac000c8e0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.336+0000 7f1ad5a00640 1 -- 172.21.15.42:0/119215602 shutdown_connections 2024-05-28T14:11:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.336+0000 7f1ad5a00640 1 --2- 172.21.15.42:0/119215602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1aac069850 0x7f1aac06bd10 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.336+0000 7f1ad5a00640 1 --2- 172.21.15.42:0/119215602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ad00d06f0 0x7f1ad009db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.336+0000 7f1ad5a00640 1 --2- 172.21.15.42:0/119215602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ad00afa70 0x7f1ad0155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.336+0000 7f1ad5a00640 1 --2- 172.21.15.42:0/119215602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad006ee80 0x7f1ad0150560 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.336+0000 7f1ad5a00640 1 -- 172.21.15.42:0/119215602 >> 172.21.15.42:0/119215602 conn(0x7f1ad006bdf0 msgr2=0x7f1ad00b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:06.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.336+0000 7f1ad5a00640 1 -- 172.21.15.42:0/119215602 shutdown_connections 2024-05-28T14:11:06.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.340+0000 7f1ad5a00640 1 -- 172.21.15.42:0/119215602 wait complete. 2024-05-28T14:11:06.353 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-28T14:11:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.516+0000 7f3993a00640 1 Processor -- start 2024-05-28T14:11:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.516+0000 7f3993a00640 1 -- start start 2024-05-28T14:11:06.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.516+0000 7f3993a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f398c06ee20 0x7f398c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:06.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.516+0000 7f3993a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f398c0afe40 con 0x7f398c15b9b0 2024-05-28T14:11:06.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.516+0000 7f3993a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f398c15c600 con 0x7f398c144f80 2024-05-28T14:11:06.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.516+0000 7f3993a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f398c06f7c0 con 0x7f398c06ee20 2024-05-28T14:11:06.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.516+0000 7f3983e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f398c06ee20 0x7f398c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:06.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.516+0000 7f3983e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f398c06ee20 0x7f398c06f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52758/0 (socket says 172.21.15.42:52758) 2024-05-28T14:11:06.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.516+0000 7f3991a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f398c15b9b0 0x7f398c15be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42976/0 (socket says 172.21.15.42:42976) 2024-05-28T14:11:06.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.516+0000 7f3983e00640 1 -- 172.21.15.42:0/3533199695 learned_addr learned my addr 172.21.15.42:0/3533199695 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:06.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.516+0000 7f3983e00640 1 -- 172.21.15.42:0/3533199695 >> v1:172.21.15.63:6789/0 conn(0x7f398c144f80 legacy=0x7f398c072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:06.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.516+0000 7f3983e00640 1 -- 172.21.15.42:0/3533199695 >> v1:172.21.15.42:6789/0 conn(0x7f398c15b9b0 legacy=0x7f398c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:06.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.516+0000 7f3983e00640 1 -- 172.21.15.42:0/3533199695 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398c15c8e0 con 0x7f398c06ee20 2024-05-28T14:11:06.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.516+0000 7f398be00640 1 -- 172.21.15.42:0/3533199695 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 30226890 0 0) 0x7f398c0afe40 con 0x7f398c15b9b0 2024-05-28T14:11:06.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.520+0000 7f3983e00640 1 --2- 172.21.15.42:0/3533199695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f398c06ee20 0x7f398c06f280 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f3974009870 tx=0x7f397401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a7ebc1e62fc983 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:06.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.520+0000 7f398be00640 1 -- 172.21.15.42:0/3533199695 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39740b2070 con 0x7f398c06ee20 2024-05-28T14:11:06.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.520+0000 7f398be00640 1 -- 172.21.15.42:0/3533199695 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39740a9070 con 0x7f398c06ee20 2024-05-28T14:11:06.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.520+0000 7f398be00640 1 -- 172.21.15.42:0/3533199695 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39740ad880 con 0x7f398c06ee20 2024-05-28T14:11:06.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.520+0000 7f3993a00640 1 -- 172.21.15.42:0/3533199695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f398c06ee20 msgr2=0x7f398c06f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:06.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.520+0000 7f3993a00640 1 --2- 172.21.15.42:0/3533199695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f398c06ee20 0x7f398c06f280 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f3974009870 tx=0x7f397401c560 comp rx=0 tx=0).stop 2024-05-28T14:11:06.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.520+0000 7f3993a00640 1 -- 172.21.15.42:0/3533199695 shutdown_connections 2024-05-28T14:11:06.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.520+0000 7f3993a00640 1 --2- 172.21.15.42:0/3533199695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f398c06ee20 0x7f398c06f280 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.520+0000 7f3993a00640 1 -- 172.21.15.42:0/3533199695 >> 172.21.15.42:0/3533199695 conn(0x7f398c0b21f0 msgr2=0x7f398c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:06.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.520+0000 7f3993a00640 1 -- 172.21.15.42:0/3533199695 shutdown_connections 2024-05-28T14:11:06.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.520+0000 7f3993a00640 1 -- 172.21.15.42:0/3533199695 wait complete. 2024-05-28T14:11:06.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3993a00640 1 Processor -- start 2024-05-28T14:11:06.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3993a00640 1 -- start start 2024-05-28T14:11:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3993a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f398c06ee20 0x7f398c1f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3993a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f398c144f80 0x7f398c15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3993a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f398c15b9b0 0x7f398c13bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3993a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f398c0724d0 con 0x7f398c06ee20 2024-05-28T14:11:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3993a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f398c0b4260 con 0x7f398c144f80 2024-05-28T14:11:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3993a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f398c169140 con 0x7f398c15b9b0 2024-05-28T14:11:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3991000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f398c06ee20 0x7f398c1f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3991000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f398c06ee20 0x7f398c1f7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52774/0 (socket says 172.21.15.42:52774) 2024-05-28T14:11:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3991000640 1 -- 172.21.15.42:0/1146896841 learned_addr learned my addr 172.21.15.42:0/1146896841 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3991a00640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f398c15b9b0 0x7f398c13bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3983e00640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f398c144f80 0x7f398c15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3991000640 1 -- 172.21.15.42:0/1146896841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f398c15b9b0 msgr2=0x7f398c13bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3991000640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f398c15b9b0 0x7f398c13bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3991000640 1 -- 172.21.15.42:0/1146896841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f398c144f80 msgr2=0x7f398c15aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3991000640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f398c144f80 0x7f398c15aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3991000640 1 -- 172.21.15.42:0/1146896841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398c13ce40 con 0x7f398c06ee20 2024-05-28T14:11:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3983e00640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f398c144f80 0x7f398c15aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3991000640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f398c06ee20 0x7f398c1f7420 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f397c00c500 tx=0x7f397c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3989600640 1 -- 172.21.15.42:0/1146896841 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c016020 con 0x7f398c06ee20 2024-05-28T14:11:06.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3989600640 1 -- 172.21.15.42:0/1146896841 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c005220 con 0x7f398c06ee20 2024-05-28T14:11:06.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3989600640 1 -- 172.21.15.42:0/1146896841 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c007030 con 0x7f398c06ee20 2024-05-28T14:11:06.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3993a00640 1 -- 172.21.15.42:0/1146896841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398c13d120 con 0x7f398c06ee20 2024-05-28T14:11:06.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.524+0000 7f3993a00640 1 -- 172.21.15.42:0/1146896841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f398c15cb70 con 0x7f398c06ee20 2024-05-28T14:11:06.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.528+0000 7f3993a00640 1 -- 172.21.15.42:0/1146896841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f398c0738b0 con 0x7f398c06ee20 2024-05-28T14:11:06.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.528+0000 7f3989600640 1 -- 172.21.15.42:0/1146896841 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f397c0071f0 con 0x7f398c06ee20 2024-05-28T14:11:06.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.528+0000 7f3989600640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3968069850 0x7f396806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:06.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.528+0000 7f3989600640 1 -- 172.21.15.42:0/1146896841 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7f397c085190 con 0x7f398c06ee20 2024-05-28T14:11:06.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.528+0000 7f3983e00640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3968069850 0x7f396806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:06.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.528+0000 7f3983e00640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3968069850 0x7f396806bd10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f39740af000 tx=0x7f39740a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:06.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.532+0000 7f3989600640 1 -- 172.21.15.42:0/1146896841 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f397c020020 con 0x7f398c06ee20 2024-05-28T14:11:06.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.700+0000 7f3993a00640 1 -- 172.21.15.42:0/1146896841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f398c04fdd0 con 0x7f398c06ee20 2024-05-28T14:11:06.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.700+0000 7f3989600640 1 -- 172.21.15.42:0/1146896841 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f397c075020 con 0x7f398c06ee20 2024-05-28T14:11:06.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.704+0000 7f3993a00640 1 -- 172.21.15.42:0/1146896841 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3968069850 msgr2=0x7f396806bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:06.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.704+0000 7f3993a00640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3968069850 0x7f396806bd10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f39740af000 tx=0x7f39740a7000 comp rx=0 tx=0).stop 2024-05-28T14:11:06.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.704+0000 7f3993a00640 1 -- 172.21.15.42:0/1146896841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f398c06ee20 msgr2=0x7f398c1f7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:06.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.704+0000 7f3993a00640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f398c06ee20 0x7f398c1f7420 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f397c00c500 tx=0x7f397c00c9c0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.704+0000 7f3993a00640 1 -- 172.21.15.42:0/1146896841 shutdown_connections 2024-05-28T14:11:06.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.704+0000 7f3993a00640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3968069850 0x7f396806bd10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.704+0000 7f3993a00640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f398c15b9b0 0x7f398c13bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.704+0000 7f3993a00640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f398c144f80 0x7f398c15aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.704+0000 7f3993a00640 1 --2- 172.21.15.42:0/1146896841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f398c06ee20 0x7f398c1f7420 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.704+0000 7f3993a00640 1 -- 172.21.15.42:0/1146896841 >> 172.21.15.42:0/1146896841 conn(0x7f398c0b21f0 msgr2=0x7f398c1417d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.704+0000 7f3993a00640 1 -- 172.21.15.42:0/1146896841 shutdown_connections 2024-05-28T14:11:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.704+0000 7f3993a00640 1 -- 172.21.15.42:0/1146896841 wait complete. 2024-05-28T14:11:06.709 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed' 2024-05-28T14:11:06.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.880+0000 7f2375800640 1 Processor -- start 2024-05-28T14:11:06.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.880+0000 7f2375800640 1 -- start start 2024-05-28T14:11:06.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.880+0000 7f2375800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f237015dfa0 0x7f23700732d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:06.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.880+0000 7f2375800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f237006edc0 0x7f23700b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:06.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.880+0000 7f2375800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2370073bb0 con 0x7f23700ad680 2024-05-28T14:11:06.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.880+0000 7f2375800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f23700b31e0 con 0x7f237015dfa0 2024-05-28T14:11:06.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.880+0000 7f2375800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2370073e90 con 0x7f237006edc0 2024-05-28T14:11:06.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.880+0000 7f2366000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f23700ad680 0x7f237006e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42986/0 (socket says 172.21.15.42:42986) 2024-05-28T14:11:06.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.880+0000 7f2366000640 1 -- 172.21.15.42:0/3411794190 learned_addr learned my addr 172.21.15.42:0/3411794190 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236f400640 1 --2- 172.21.15.42:0/3411794190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f237006edc0 0x7f23700b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236ea00640 1 --2- 172.21.15.42:0/3411794190 >> v2:172.21.15.63:3300/0 conn(0x7f237015dfa0 0x7f23700732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 -- 172.21.15.42:0/3411794190 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3743178621 0 0) 0x7f2370073bb0 con 0x7f23700ad680 2024-05-28T14:11:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 -- 172.21.15.42:0/3411794190 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2354003530 con 0x7f23700ad680 2024-05-28T14:11:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 -- 172.21.15.42:0/3411794190 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 371117381 0 0) 0x7f2354003530 con 0x7f23700ad680 2024-05-28T14:11:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 -- 172.21.15.42:0/3411794190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f237006edc0 msgr2=0x7f23700b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 --2- 172.21.15.42:0/3411794190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f237006edc0 0x7f23700b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 -- 172.21.15.42:0/3411794190 >> v2:172.21.15.63:3300/0 conn(0x7f237015dfa0 msgr2=0x7f23700732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 --2- 172.21.15.42:0/3411794190 >> v2:172.21.15.63:3300/0 conn(0x7f237015dfa0 0x7f23700732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 -- 172.21.15.42:0/3411794190 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23700b3430 con 0x7f23700ad680 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 -- 172.21.15.42:0/3411794190 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f23580016a0 con 0x7f23700ad680 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 -- 172.21.15.42:0/3411794190 >> v1:172.21.15.42:6789/0 conn(0x7f23700ad680 legacy=0x7f237006e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 --2- 172.21.15.42:0/3411794190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23540039c0 0x7f2354003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 --2- 172.21.15.42:0/3411794190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2354004810 0x7f2354004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 --2- 172.21.15.42:0/3411794190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f237006edc0 0x7f23540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 -- 172.21.15.42:0/3411794190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f23700b31e0 con 0x7f23540039c0 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 -- 172.21.15.42:0/3411794190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2370073e90 con 0x7f2354004810 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236e000640 1 -- 172.21.15.42:0/3411794190 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f235400ba00 con 0x7f237006edc0 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236ea00640 1 --2- 172.21.15.42:0/3411794190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23540039c0 0x7f2354003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f2366000640 1 --2- 172.21.15.42:0/3411794190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2354004810 0x7f2354004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:06.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236f400640 1 --2- 172.21.15.42:0/3411794190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f237006edc0 0x7f23540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236ea00640 1 -- 172.21.15.42:0/3411794190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f237006edc0 msgr2=0x7f23540051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236ea00640 1 --2- 172.21.15.42:0/3411794190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f237006edc0 0x7f23540051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236ea00640 1 -- 172.21.15.42:0/3411794190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2354004810 msgr2=0x7f2354004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236ea00640 1 --2- 172.21.15.42:0/3411794190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2354004810 0x7f2354004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236ea00640 1 -- 172.21.15.42:0/3411794190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2354003530 con 0x7f23540039c0 2024-05-28T14:11:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f236f400640 1 --2- 172.21.15.42:0/3411794190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f237006edc0 0x7f23540051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.884+0000 7f2366000640 1 --2- 172.21.15.42:0/3411794190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2354004810 0x7f2354004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:06.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f236ea00640 1 --2- 172.21.15.42:0/3411794190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23540039c0 0x7f2354003de0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f2360009ae0 tx=0x7f236001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe033f8716791ae server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:06.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f236e000640 1 -- 172.21.15.42:0/3411794190 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23600096b0 con 0x7f23540039c0 2024-05-28T14:11:06.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f236e000640 1 -- 172.21.15.42:0/3411794190 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23600a6430 con 0x7f23540039c0 2024-05-28T14:11:06.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f236e000640 1 -- 172.21.15.42:0/3411794190 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23600a6820 con 0x7f23540039c0 2024-05-28T14:11:06.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f2375800640 1 -- 172.21.15.42:0/3411794190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23540039c0 msgr2=0x7f2354003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:06.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f2375800640 1 --2- 172.21.15.42:0/3411794190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23540039c0 0x7f2354003de0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f2360009ae0 tx=0x7f236001ca20 comp rx=0 tx=0).stop 2024-05-28T14:11:06.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f236f400640 1 -- 172.21.15.42:0/3411794190 reap_dead start 2024-05-28T14:11:06.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f2375800640 1 -- 172.21.15.42:0/3411794190 shutdown_connections 2024-05-28T14:11:06.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f2375800640 1 -- 172.21.15.42:0/3411794190 >> 172.21.15.42:0/3411794190 conn(0x7f237006bdf0 msgr2=0x7f237006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:06.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f2375800640 1 -- 172.21.15.42:0/3411794190 shutdown_connections 2024-05-28T14:11:06.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f2375800640 1 -- 172.21.15.42:0/3411794190 wait complete. 2024-05-28T14:11:06.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f2375800640 1 Processor -- start 2024-05-28T14:11:06.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f2375800640 1 -- start start 2024-05-28T14:11:06.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f2375800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23540039c0 0x7f2370145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:06.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f2375800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2370146630 0x7f23701694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:06.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f2375800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2370157230 0x7f237014b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:06.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f2375800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f23700b3db0 con 0x7f2370157230 2024-05-28T14:11:06.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f2375800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f23700b3c30 con 0x7f23540039c0 2024-05-28T14:11:06.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f2375800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f23700b3f30 con 0x7f2370146630 2024-05-28T14:11:06.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f236f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2370157230 0x7f237014b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f236f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2370157230 0x7f237014b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52794/0 (socket says 172.21.15.42:52794) 2024-05-28T14:11:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.888+0000 7f236f400640 1 -- 172.21.15.42:0/4002936538 learned_addr learned my addr 172.21.15.42:0/4002936538 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.892+0000 7f236f400640 1 -- 172.21.15.42:0/4002936538 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2370146630 msgr2=0x7f23701694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.892+0000 7f236f400640 1 --2- 172.21.15.42:0/4002936538 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2370146630 0x7f23701694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.892+0000 7f236f400640 1 -- 172.21.15.42:0/4002936538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23540039c0 msgr2=0x7f2370145f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.892+0000 7f236f400640 1 --2- 172.21.15.42:0/4002936538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23540039c0 0x7f2370145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.892+0000 7f236f400640 1 -- 172.21.15.42:0/4002936538 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23700b1690 con 0x7f2370157230 2024-05-28T14:11:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.892+0000 7f236f400640 1 --2- 172.21.15.42:0/4002936538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2370157230 0x7f237014b540 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f23700d13f0 tx=0x7f2368010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.892+0000 7f2367400640 1 -- 172.21.15.42:0/4002936538 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23680039a0 con 0x7f2370157230 2024-05-28T14:11:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.892+0000 7f2367400640 1 -- 172.21.15.42:0/4002936538 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2368003b60 con 0x7f2370157230 2024-05-28T14:11:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.892+0000 7f2367400640 1 -- 172.21.15.42:0/4002936538 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368004020 con 0x7f2370157230 2024-05-28T14:11:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.892+0000 7f2375800640 1 -- 172.21.15.42:0/4002936538 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2370159fc0 con 0x7f2370157230 2024-05-28T14:11:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.892+0000 7f2375800640 1 -- 172.21.15.42:0/4002936538 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2370168b40 con 0x7f2370157230 2024-05-28T14:11:06.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.892+0000 7f2375800640 1 -- 172.21.15.42:0/4002936538 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23701685c0 con 0x7f2370157230 2024-05-28T14:11:06.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.892+0000 7f2367400640 1 -- 172.21.15.42:0/4002936538 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f236801e020 con 0x7f2370157230 2024-05-28T14:11:06.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.892+0000 7f2367400640 1 --2- 172.21.15.42:0/4002936538 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f234c069850 0x7f234c06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:06.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.896+0000 7f2367400640 1 -- 172.21.15.42:0/4002936538 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7f2368085e90 con 0x7f2370157230 2024-05-28T14:11:06.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.896+0000 7f236ea00640 1 --2- 172.21.15.42:0/4002936538 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f234c069850 0x7f234c06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:06.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.896+0000 7f236ea00640 1 --2- 172.21.15.42:0/4002936538 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f234c069850 0x7f234c06bd10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f23600ad000 tx=0x7f2360004740 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:06.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:06.896+0000 7f2367400640 1 -- 172.21.15.42:0/4002936538 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2368025020 con 0x7f2370157230 2024-05-28T14:11:07.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.128+0000 7f2375800640 1 -- 172.21.15.42:0/4002936538 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed"]} v 0) -- 0x7f2370068a90 con 0x7f2370157230 2024-05-28T14:11:07.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.340+0000 7f2367400640 1 -- 172.21.15.42:0/4002936538 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed"]}]=0 v588) ==== 153+0+0 (secure 0 0 0) 0x7f2368057ea0 con 0x7f2370157230 2024-05-28T14:11:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.344+0000 7f2375800640 1 -- 172.21.15.42:0/4002936538 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f234c069850 msgr2=0x7f234c06bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.344+0000 7f2375800640 1 --2- 172.21.15.42:0/4002936538 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f234c069850 0x7f234c06bd10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f23600ad000 tx=0x7f2360004740 comp rx=0 tx=0).stop 2024-05-28T14:11:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.344+0000 7f2375800640 1 -- 172.21.15.42:0/4002936538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2370157230 msgr2=0x7f237014b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.344+0000 7f2375800640 1 --2- 172.21.15.42:0/4002936538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2370157230 0x7f237014b540 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f23700d13f0 tx=0x7f2368010590 comp rx=0 tx=0).stop 2024-05-28T14:11:07.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.344+0000 7f2375800640 1 -- 172.21.15.42:0/4002936538 shutdown_connections 2024-05-28T14:11:07.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.344+0000 7f2375800640 1 --2- 172.21.15.42:0/4002936538 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f234c069850 0x7f234c06bd10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.344+0000 7f2375800640 1 --2- 172.21.15.42:0/4002936538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2370157230 0x7f237014b540 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.344+0000 7f2375800640 1 --2- 172.21.15.42:0/4002936538 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2370146630 0x7f23701694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.344+0000 7f2375800640 1 --2- 172.21.15.42:0/4002936538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23540039c0 0x7f2370145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.344+0000 7f2375800640 1 -- 172.21.15.42:0/4002936538 >> 172.21.15.42:0/4002936538 conn(0x7f237006bdf0 msgr2=0x7f23700b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:07.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.344+0000 7f2375800640 1 -- 172.21.15.42:0/4002936538 shutdown_connections 2024-05-28T14:11:07.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.344+0000 7f2375800640 1 -- 172.21.15.42:0/4002936538 wait complete. 2024-05-28T14:11:07.349 INFO:tasks.cephfs_test_runner:Test that the command "ceph fs clone status" prints progress stats ... ok 2024-05-28T14:11:07.349 INFO:tasks.cephfs_test_runner:Starting test: test_when_1_clone_is_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the progress bar, that is printed for 1 ongoing clone job, is 2024-05-28T14:11:07.349 INFO:tasks.cephfs_test_runner:test_when_1_clone_is_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-28T14:11:07.350 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:11:07.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.508+0000 7fd07d000640 1 Processor -- start 2024-05-28T14:11:07.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.508+0000 7fd07d000640 1 -- start start 2024-05-28T14:11:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.508+0000 7fd07d000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd07806f660 0x7fd078073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.508+0000 7fd07d000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd07815ba40 0x7fd0780d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.508+0000 7fd07d000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd07815f280 con 0x7fd078072dd0 2024-05-28T14:11:07.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.508+0000 7fd07d000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd07815f030 con 0x7fd07806f660 2024-05-28T14:11:07.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.508+0000 7fd07d000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd07815f560 con 0x7fd07815ba40 2024-05-28T14:11:07.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.508+0000 7fd075e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd07806f660 0x7fd078073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:07.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.512+0000 7fd075e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd07806f660 0x7fd078073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59444/0 (socket says 172.21.15.42:59444) 2024-05-28T14:11:07.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.512+0000 7fd075e00640 1 -- 172.21.15.42:0/2548293788 learned_addr learned my addr 172.21.15.42:0/2548293788 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:07.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.512+0000 7fd06d400640 1 --2- 172.21.15.42:0/2548293788 >> v2:172.21.15.63:3300/0 conn(0x7fd07815ba40 0x7fd0780d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:07.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.512+0000 7fd075e00640 1 -- 172.21.15.42:0/2548293788 >> v2:172.21.15.63:3300/0 conn(0x7fd07815ba40 msgr2=0x7fd0780d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:07.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.512+0000 7fd075e00640 1 --2- 172.21.15.42:0/2548293788 >> v2:172.21.15.63:3300/0 conn(0x7fd07815ba40 0x7fd0780d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.512+0000 7fd075e00640 1 -- 172.21.15.42:0/2548293788 >> v1:172.21.15.42:6789/0 conn(0x7fd078072dd0 legacy=0x7fd0780b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:07.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.512+0000 7fd075e00640 1 -- 172.21.15.42:0/2548293788 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07815f840 con 0x7fd07806f660 2024-05-28T14:11:07.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.512+0000 7fd075400640 1 -- 172.21.15.42:0/2548293788 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2676096635 0 0) 0x7fd07815f280 con 0x7fd078072dd0 2024-05-28T14:11:07.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.512+0000 7fd075e00640 1 --2- 172.21.15.42:0/2548293788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07806f660 0x7fd078073980 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fd0780b56b0 tx=0x7fd06801eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a6f3d145080f91 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:07.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.512+0000 7fd075400640 1 -- 172.21.15.42:0/2548293788 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0680b4070 con 0x7fd07806f660 2024-05-28T14:11:07.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.512+0000 7fd075400640 1 -- 172.21.15.42:0/2548293788 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0680a7a80 con 0x7fd07806f660 2024-05-28T14:11:07.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.512+0000 7fd075400640 1 -- 172.21.15.42:0/2548293788 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0680a7e50 con 0x7fd07806f660 2024-05-28T14:11:07.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 -- 172.21.15.42:0/2548293788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07806f660 msgr2=0x7fd078073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:07.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 --2- 172.21.15.42:0/2548293788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07806f660 0x7fd078073980 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fd0780b56b0 tx=0x7fd06801eaa0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 -- 172.21.15.42:0/2548293788 shutdown_connections 2024-05-28T14:11:07.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 --2- 172.21.15.42:0/2548293788 >> v2:172.21.15.63:3300/0 conn(0x7fd07815ba40 0x7fd0780d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 --2- 172.21.15.42:0/2548293788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07806f660 0x7fd078073980 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 -- 172.21.15.42:0/2548293788 >> 172.21.15.42:0/2548293788 conn(0x7fd07806bdf0 msgr2=0x7fd07806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:07.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 -- 172.21.15.42:0/2548293788 shutdown_connections 2024-05-28T14:11:07.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 -- 172.21.15.42:0/2548293788 wait complete. 2024-05-28T14:11:07.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 Processor -- start 2024-05-28T14:11:07.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 -- start start 2024-05-28T14:11:07.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07806f660 0x7fd0780b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:07.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd078072dd0 0x7fd07814c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:07.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078153140 0x7fd07806dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:07.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd0780b3460 con 0x7fd078153140 2024-05-28T14:11:07.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0780b32e0 con 0x7fd07806f660 2024-05-28T14:11:07.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd07d000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0780b35e0 con 0x7fd078072dd0 2024-05-28T14:11:07.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd076800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078153140 0x7fd07806dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:07.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd076800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078153140 0x7fd07806dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59456/0 (socket says 172.21.15.42:59456) 2024-05-28T14:11:07.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd076800640 1 -- 172.21.15.42:0/945117097 learned_addr learned my addr 172.21.15.42:0/945117097 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:07.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd075e00640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07806f660 0x7fd0780b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:07.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd06d400640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd078072dd0 0x7fd07814c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:07.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd076800640 1 -- 172.21.15.42:0/945117097 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd078072dd0 msgr2=0x7fd07814c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:07.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd076800640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd078072dd0 0x7fd07814c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd076800640 1 -- 172.21.15.42:0/945117097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07806f660 msgr2=0x7fd0780b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:07.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd076800640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07806f660 0x7fd0780b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd076800640 1 -- 172.21.15.42:0/945117097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07814cf50 con 0x7fd078153140 2024-05-28T14:11:07.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd075e00640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07806f660 0x7fd0780b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:07.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.516+0000 7fd076800640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078153140 0x7fd07806dc60 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fd07000c780 tx=0x7fd07000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:07.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.520+0000 7fd06ea00640 1 -- 172.21.15.42:0/945117097 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070003db0 con 0x7fd078153140 2024-05-28T14:11:07.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.520+0000 7fd06ea00640 1 -- 172.21.15.42:0/945117097 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd070005070 con 0x7fd078153140 2024-05-28T14:11:07.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.520+0000 7fd06ea00640 1 -- 172.21.15.42:0/945117097 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070005460 con 0x7fd078153140 2024-05-28T14:11:07.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.520+0000 7fd07d000640 1 -- 172.21.15.42:0/945117097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07814d230 con 0x7fd078153140 2024-05-28T14:11:07.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.520+0000 7fd07d000640 1 -- 172.21.15.42:0/945117097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07814d7f0 con 0x7fd078153140 2024-05-28T14:11:07.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.520+0000 7fd07d000640 1 -- 172.21.15.42:0/945117097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd078068a90 con 0x7fd078153140 2024-05-28T14:11:07.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.520+0000 7fd06ea00640 1 -- 172.21.15.42:0/945117097 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fd070005600 con 0x7fd078153140 2024-05-28T14:11:07.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.520+0000 7fd06ea00640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd054069850 0x7fd05406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:07.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.520+0000 7fd06ea00640 1 -- 172.21.15.42:0/945117097 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7fd070085a30 con 0x7fd078153140 2024-05-28T14:11:07.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.520+0000 7fd075e00640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd054069850 0x7fd05406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:07.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.520+0000 7fd075e00640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd054069850 0x7fd05406bd10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd0680b1000 tx=0x7fd068008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:07.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.524+0000 7fd06ea00640 1 -- 172.21.15.42:0/945117097 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd070057660 con 0x7fd078153140 2024-05-28T14:11:07.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.692+0000 7fd07d000640 1 -- 172.21.15.42:0/945117097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd07815dd90 con 0x7fd078153140 2024-05-28T14:11:07.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.692+0000 7fd06ea00640 1 -- 172.21.15.42:0/945117097 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+13002 (secure 0 0 0) 0x7fd070056f60 con 0x7fd078153140 2024-05-28T14:11:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:07.676 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":97,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:11:06.334216+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6834/615628755":"2024-05-29T14:11:03.327698+0000","172.21.15.63:6832/615628755":"2024-05-29T14:11:03.327698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:11:07.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.692+0000 7fd07d000640 1 -- 172.21.15.42:0/945117097 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd054069850 msgr2=0x7fd05406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:07.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.692+0000 7fd07d000640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd054069850 0x7fd05406bd10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd0680b1000 tx=0x7fd068008040 comp rx=0 tx=0).stop 2024-05-28T14:11:07.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.692+0000 7fd07d000640 1 -- 172.21.15.42:0/945117097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078153140 msgr2=0x7fd07806dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:07.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.692+0000 7fd07d000640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078153140 0x7fd07806dc60 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fd07000c780 tx=0x7fd07000cc40 comp rx=0 tx=0).stop 2024-05-28T14:11:07.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.692+0000 7fd07d000640 1 -- 172.21.15.42:0/945117097 shutdown_connections 2024-05-28T14:11:07.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.692+0000 7fd07d000640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd054069850 0x7fd05406bd10 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.692+0000 7fd07d000640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078153140 0x7fd07806dc60 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.692+0000 7fd07d000640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd078072dd0 0x7fd07814c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.692+0000 7fd07d000640 1 --2- 172.21.15.42:0/945117097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07806f660 0x7fd0780b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.692+0000 7fd07d000640 1 -- 172.21.15.42:0/945117097 >> 172.21.15.42:0/945117097 conn(0x7fd07806bdf0 msgr2=0x7fd0780ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:07.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.692+0000 7fd07d000640 1 -- 172.21.15.42:0/945117097 shutdown_connections 2024-05-28T14:11:07.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.692+0000 7fd07d000640 1 -- 172.21.15.42:0/945117097 wait complete. 2024-05-28T14:11:07.697 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:11:07.698 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:11:07.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.856+0000 7efedd600640 1 Processor -- start 2024-05-28T14:11:07.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.856+0000 7efedd600640 1 -- start start 2024-05-28T14:11:07.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.860+0000 7efedd600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efed80b4eb0 0x7efed806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:07.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.860+0000 7efedd600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed806f190 con 0x7efed806d110 2024-05-28T14:11:07.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.860+0000 7efedd600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed806f470 con 0x7efed80b02e0 2024-05-28T14:11:07.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.860+0000 7efedd600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efed806f750 con 0x7efed80b4eb0 2024-05-28T14:11:07.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.860+0000 7efece000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7efed806d110 0x7efed8154da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:44756/0 (socket says 172.21.15.42:44756) 2024-05-28T14:11:07.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.860+0000 7efece000640 1 -- 172.21.15.42:0/3515729229 learned_addr learned my addr 172.21.15.42:0/3515729229 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:07.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.860+0000 7efed7400640 1 --2- 172.21.15.42:0/3515729229 >> v2:172.21.15.63:3300/0 conn(0x7efed80b4eb0 0x7efed806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:07.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.860+0000 7efed6000640 1 -- 172.21.15.42:0/3515729229 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1973300511 0 0) 0x7efed806f190 con 0x7efed806d110 2024-05-28T14:11:07.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.860+0000 7efed6000640 1 -- 172.21.15.42:0/3515729229 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efebc003530 con 0x7efed806d110 2024-05-28T14:11:07.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.860+0000 7efed7400640 1 -- 172.21.15.42:0/3515729229 >> v1:172.21.15.63:6789/0 conn(0x7efed80b02e0 legacy=0x7efed806ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:07.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.860+0000 7efed7400640 1 -- 172.21.15.42:0/3515729229 >> v1:172.21.15.42:6789/0 conn(0x7efed806d110 legacy=0x7efed8154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:07.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.860+0000 7efed7400640 1 -- 172.21.15.42:0/3515729229 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed806fa30 con 0x7efed80b4eb0 2024-05-28T14:11:07.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.864+0000 7efed7400640 1 --2- 172.21.15.42:0/3515729229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efed80b4eb0 0x7efed806eb00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7efed0009dc0 tx=0x7efed001c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43d544fc3df43105 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:07.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.864+0000 7efed6000640 1 -- 172.21.15.42:0/3515729229 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed00b2070 con 0x7efed80b4eb0 2024-05-28T14:11:07.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.864+0000 7efed6000640 1 -- 172.21.15.42:0/3515729229 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed00a5940 con 0x7efed80b4eb0 2024-05-28T14:11:07.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.864+0000 7efed6000640 1 -- 172.21.15.42:0/3515729229 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed00a5d10 con 0x7efed80b4eb0 2024-05-28T14:11:07.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.864+0000 7efedd600640 1 -- 172.21.15.42:0/3515729229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efed80b4eb0 msgr2=0x7efed806eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:07.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.864+0000 7efedd600640 1 --2- 172.21.15.42:0/3515729229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efed80b4eb0 0x7efed806eb00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7efed0009dc0 tx=0x7efed001c8c0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.864+0000 7efedd600640 1 -- 172.21.15.42:0/3515729229 shutdown_connections 2024-05-28T14:11:07.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.864+0000 7efedd600640 1 --2- 172.21.15.42:0/3515729229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efed80b4eb0 0x7efed806eb00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.864+0000 7efedd600640 1 -- 172.21.15.42:0/3515729229 >> 172.21.15.42:0/3515729229 conn(0x7efed8072600 msgr2=0x7efed8072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:07.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.864+0000 7efedd600640 1 -- 172.21.15.42:0/3515729229 shutdown_connections 2024-05-28T14:11:07.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.864+0000 7efedd600640 1 -- 172.21.15.42:0/3515729229 wait complete. 2024-05-28T14:11:07.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.864+0000 7efedd600640 1 Processor -- start 2024-05-28T14:11:07.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.864+0000 7efedd600640 1 -- start start 2024-05-28T14:11:07.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efedd600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efed806d110 0x7efed8146140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:07.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efedd600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efed80b02e0 0x7efed814d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:07.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efedd600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efed80b4eb0 0x7efed81fb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:07.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efedd600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efed816c9c0 con 0x7efed806d110 2024-05-28T14:11:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efedd600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efed816c840 con 0x7efed80b02e0 2024-05-28T14:11:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efedd600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efed816cb40 con 0x7efed80b4eb0 2024-05-28T14:11:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efed6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efed806d110 0x7efed8146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efed6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efed806d110 0x7efed8146140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59472/0 (socket says 172.21.15.42:59472) 2024-05-28T14:11:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efed6a00640 1 -- 172.21.15.42:0/1207368542 learned_addr learned my addr 172.21.15.42:0/1207368542 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efece000640 1 --2- 172.21.15.42:0/1207368542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efed80b02e0 0x7efed814d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efed7400640 1 --2- 172.21.15.42:0/1207368542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efed80b4eb0 0x7efed81fb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efed6a00640 1 -- 172.21.15.42:0/1207368542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efed80b4eb0 msgr2=0x7efed81fb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efed6a00640 1 --2- 172.21.15.42:0/1207368542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efed80b4eb0 0x7efed81fb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efed6a00640 1 -- 172.21.15.42:0/1207368542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efed80b02e0 msgr2=0x7efed814d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efed6a00640 1 --2- 172.21.15.42:0/1207368542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efed80b02e0 0x7efed814d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efed6a00640 1 -- 172.21.15.42:0/1207368542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed81500c0 con 0x7efed806d110 2024-05-28T14:11:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efed6a00640 1 --2- 172.21.15.42:0/1207368542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efed806d110 0x7efed8146140 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7efec800c420 tx=0x7efec800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efecf400640 1 -- 172.21.15.42:0/1207368542 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80052d0 con 0x7efed806d110 2024-05-28T14:11:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efecf400640 1 -- 172.21.15.42:0/1207368542 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec8005470 con 0x7efed806d110 2024-05-28T14:11:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efecf400640 1 -- 172.21.15.42:0/1207368542 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8007260 con 0x7efed806d110 2024-05-28T14:11:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efedd600640 1 -- 172.21.15.42:0/1207368542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed81503a0 con 0x7efed806d110 2024-05-28T14:11:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efedd600640 1 -- 172.21.15.42:0/1207368542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed8150960 con 0x7efed806d110 2024-05-28T14:11:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efedd600640 1 -- 172.21.15.42:0/1207368542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed8073f40 con 0x7efed806d110 2024-05-28T14:11:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.868+0000 7efecf400640 1 -- 172.21.15.42:0/1207368542 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7efec8007400 con 0x7efed806d110 2024-05-28T14:11:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.872+0000 7efecf400640 1 --2- 172.21.15.42:0/1207368542 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efeb4069850 0x7efeb406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:07.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.872+0000 7efecf400640 1 -- 172.21.15.42:0/1207368542 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7efec8084f70 con 0x7efed806d110 2024-05-28T14:11:07.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.872+0000 7efece000640 1 --2- 172.21.15.42:0/1207368542 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efeb4069850 0x7efeb406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:07.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.872+0000 7efece000640 1 --2- 172.21.15.42:0/1207368542 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efeb4069850 0x7efeb406bd10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efed81490e0 tx=0x7efec0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:07.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:07.876+0000 7efecf400640 1 -- 172.21.15.42:0/1207368542 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efec8056c00 con 0x7efed806d110 2024-05-28T14:11:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.040+0000 7efedd600640 1 -- 172.21.15.42:0/1207368542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efed80d0900 con 0x7efed806d110 2024-05-28T14:11:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.040+0000 7efecf400640 1 -- 172.21.15.42:0/1207368542 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7efec8056500 con 0x7efed806d110 2024-05-28T14:11:08.025 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:11:08.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.044+0000 7efedd600640 1 -- 172.21.15.42:0/1207368542 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efeb4069850 msgr2=0x7efeb406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.044+0000 7efedd600640 1 --2- 172.21.15.42:0/1207368542 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efeb4069850 0x7efeb406bd10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efed81490e0 tx=0x7efec0009040 comp rx=0 tx=0).stop 2024-05-28T14:11:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.044+0000 7efedd600640 1 -- 172.21.15.42:0/1207368542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efed806d110 msgr2=0x7efed8146140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.044+0000 7efedd600640 1 --2- 172.21.15.42:0/1207368542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efed806d110 0x7efed8146140 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7efec800c420 tx=0x7efec800c8e0 comp rx=0 tx=0).stop 2024-05-28T14:11:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.044+0000 7efedd600640 1 -- 172.21.15.42:0/1207368542 shutdown_connections 2024-05-28T14:11:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.044+0000 7efedd600640 1 --2- 172.21.15.42:0/1207368542 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efeb4069850 0x7efeb406bd10 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.044+0000 7efedd600640 1 --2- 172.21.15.42:0/1207368542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efed80b4eb0 0x7efed81fb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.044+0000 7efedd600640 1 --2- 172.21.15.42:0/1207368542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efed80b02e0 0x7efed814d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.044+0000 7efedd600640 1 --2- 172.21.15.42:0/1207368542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efed806d110 0x7efed8146140 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.044+0000 7efedd600640 1 -- 172.21.15.42:0/1207368542 >> 172.21.15.42:0/1207368542 conn(0x7efed8072600 msgr2=0x7efed8162880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.044+0000 7efedd600640 1 -- 172.21.15.42:0/1207368542 shutdown_connections 2024-05-28T14:11:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.044+0000 7efedd600640 1 -- 172.21.15.42:0/1207368542 wait complete. 2024-05-28T14:11:08.047 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled' 2024-05-28T14:11:08.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.204+0000 7fc99a800640 1 Processor -- start 2024-05-28T14:11:08.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.204+0000 7fc99a800640 1 -- start start 2024-05-28T14:11:08.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.204+0000 7fc99a800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9940d15f0 0x7fc9940ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:08.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.204+0000 7fc99a800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc9940cd040 0x7fc99409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:08.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.204+0000 7fc99a800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc994070740 con 0x7fc99415e970 2024-05-28T14:11:08.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.204+0000 7fc99a800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc9940d0640 con 0x7fc9940cd040 2024-05-28T14:11:08.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.204+0000 7fc99a800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc9940d1160 con 0x7fc9940d15f0 2024-05-28T14:11:08.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.208+0000 7fc993e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc9940cd040 0x7fc99409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:08.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.208+0000 7fc993e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc9940cd040 0x7fc99409aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59482/0 (socket says 172.21.15.42:59482) 2024-05-28T14:11:08.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.208+0000 7fc993e00640 1 -- 172.21.15.42:0/4112795687 learned_addr learned my addr 172.21.15.42:0/4112795687 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:08.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.208+0000 7fc98be00640 1 --2- 172.21.15.42:0/4112795687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9940d15f0 0x7fc9940ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:08.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.208+0000 7fc993e00640 1 -- 172.21.15.42:0/4112795687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9940d15f0 msgr2=0x7fc9940ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:08.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.208+0000 7fc993e00640 1 --2- 172.21.15.42:0/4112795687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9940d15f0 0x7fc9940ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:08.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.208+0000 7fc993e00640 1 -- 172.21.15.42:0/4112795687 >> v1:172.21.15.63:6789/0 conn(0x7fc99415e970 legacy=0x7fc99406ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:08.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.208+0000 7fc993e00640 1 -- 172.21.15.42:0/4112795687 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9940d0870 con 0x7fc9940cd040 2024-05-28T14:11:08.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.208+0000 7fc993e00640 1 --2- 172.21.15.42:0/4112795687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9940cd040 0x7fc99409aa50 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009e50 tx=0x7fc98c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3389df3ff31900aa server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:08.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.208+0000 7fc992a00640 1 -- 172.21.15.42:0/4112795687 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0a5750 con 0x7fc9940cd040 2024-05-28T14:11:08.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.208+0000 7fc992a00640 1 -- 172.21.15.42:0/4112795687 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c0a58f0 con 0x7fc9940cd040 2024-05-28T14:11:08.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.208+0000 7fc992a00640 1 -- 172.21.15.42:0/4112795687 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0a5cc0 con 0x7fc9940cd040 2024-05-28T14:11:08.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 -- 172.21.15.42:0/4112795687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9940cd040 msgr2=0x7fc99409aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:08.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 --2- 172.21.15.42:0/4112795687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9940cd040 0x7fc99409aa50 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009e50 tx=0x7fc98c01c9a0 comp rx=0 tx=0).stop 2024-05-28T14:11:08.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 -- 172.21.15.42:0/4112795687 shutdown_connections 2024-05-28T14:11:08.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 --2- 172.21.15.42:0/4112795687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9940cd040 0x7fc99409aa50 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:08.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 --2- 172.21.15.42:0/4112795687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9940d15f0 0x7fc9940ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:08.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 -- 172.21.15.42:0/4112795687 >> 172.21.15.42:0/4112795687 conn(0x7fc9940b3fa0 msgr2=0x7fc9940b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:08.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 -- 172.21.15.42:0/4112795687 shutdown_connections 2024-05-28T14:11:08.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 -- 172.21.15.42:0/4112795687 wait complete. 2024-05-28T14:11:08.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 Processor -- start 2024-05-28T14:11:08.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 -- start start 2024-05-28T14:11:08.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9940cd040 0x7fc99414cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:08.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9940d15f0 0x7fc9940c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:08.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99415e970 0x7fc99414d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:08.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc99416b9c0 con 0x7fc99415e970 2024-05-28T14:11:08.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc99416b840 con 0x7fc9940d15f0 2024-05-28T14:11:08.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc99a800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc99416bb40 con 0x7fc9940cd040 2024-05-28T14:11:08.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc993e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99415e970 0x7fc99414d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:08.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc993e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99415e970 0x7fc99414d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59496/0 (socket says 172.21.15.42:59496) 2024-05-28T14:11:08.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc993e00640 1 -- 172.21.15.42:0/3269275336 learned_addr learned my addr 172.21.15.42:0/3269275336 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:08.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc993400640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9940cd040 0x7fc99414cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:08.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc98be00640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9940d15f0 0x7fc9940c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:08.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc993e00640 1 -- 172.21.15.42:0/3269275336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9940cd040 msgr2=0x7fc99414cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:08.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc993e00640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9940cd040 0x7fc99414cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:08.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc993e00640 1 -- 172.21.15.42:0/3269275336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9940d15f0 msgr2=0x7fc9940c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:08.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc993e00640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9940d15f0 0x7fc9940c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:08.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc993e00640 1 -- 172.21.15.42:0/3269275336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9940b6220 con 0x7fc99415e970 2024-05-28T14:11:08.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc993400640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9940cd040 0x7fc99414cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:08.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.212+0000 7fc98be00640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9940d15f0 0x7fc9940c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:08.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.216+0000 7fc993e00640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99415e970 0x7fc99414d6f0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0af040 tx=0x7fc98c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:08.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.216+0000 7fc98b400640 1 -- 172.21.15.42:0/3269275336 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0bc020 con 0x7fc99415e970 2024-05-28T14:11:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.216+0000 7fc98b400640 1 -- 172.21.15.42:0/3269275336 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c0b8670 con 0x7fc99415e970 2024-05-28T14:11:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.216+0000 7fc98b400640 1 -- 172.21.15.42:0/3269275336 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c009510 con 0x7fc99415e970 2024-05-28T14:11:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.216+0000 7fc99a800640 1 -- 172.21.15.42:0/3269275336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99415dbb0 con 0x7fc99415e970 2024-05-28T14:11:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.216+0000 7fc99a800640 1 -- 172.21.15.42:0/3269275336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99414d470 con 0x7fc99415e970 2024-05-28T14:11:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.216+0000 7fc99a800640 1 -- 172.21.15.42:0/3269275336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc994068a90 con 0x7fc99415e970 2024-05-28T14:11:08.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.216+0000 7fc98b400640 1 -- 172.21.15.42:0/3269275336 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fc98c0bf020 con 0x7fc99415e970 2024-05-28T14:11:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.216+0000 7fc98b400640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc970069850 0x7fc97006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.216+0000 7fc98b400640 1 -- 172.21.15.42:0/3269275336 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7fc98c120680 con 0x7fc99415e970 2024-05-28T14:11:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.216+0000 7fc993400640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc970069850 0x7fc97006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.220+0000 7fc993400640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc970069850 0x7fc97006bd10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc9940b5e70 tx=0x7fc984009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:08.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.220+0000 7fc98b400640 1 -- 172.21.15.42:0/3269275336 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc98c0c6030 con 0x7fc99415e970 2024-05-28T14:11:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:08.448+0000 7fc99a800640 1 -- 172.21.15.42:0/3269275336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled"]} v 0) -- 0x7fc9940d1fd0 con 0x7fc99415e970 2024-05-28T14:11:09.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.348+0000 7fc98b400640 1 -- 172.21.15.42:0/3269275336 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled"]}]=0 v590) ==== 164+0+0 (secure 0 0 0) 0x7fc98c016620 con 0x7fc99415e970 2024-05-28T14:11:09.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.352+0000 7fc99a800640 1 -- 172.21.15.42:0/3269275336 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc970069850 msgr2=0x7fc97006bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:09.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.352+0000 7fc99a800640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc970069850 0x7fc97006bd10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc9940b5e70 tx=0x7fc984009040 comp rx=0 tx=0).stop 2024-05-28T14:11:09.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.352+0000 7fc99a800640 1 -- 172.21.15.42:0/3269275336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99415e970 msgr2=0x7fc99414d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:09.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.352+0000 7fc99a800640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99415e970 0x7fc99414d6f0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0af040 tx=0x7fc98c019740 comp rx=0 tx=0).stop 2024-05-28T14:11:09.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.352+0000 7fc99a800640 1 -- 172.21.15.42:0/3269275336 shutdown_connections 2024-05-28T14:11:09.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.352+0000 7fc99a800640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc970069850 0x7fc97006bd10 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.352+0000 7fc99a800640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99415e970 0x7fc99414d6f0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.352+0000 7fc99a800640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9940d15f0 0x7fc9940c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.352+0000 7fc99a800640 1 --2- 172.21.15.42:0/3269275336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9940cd040 0x7fc99414cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.352+0000 7fc99a800640 1 -- 172.21.15.42:0/3269275336 >> 172.21.15.42:0/3269275336 conn(0x7fc9940b3fa0 msgr2=0x7fc9940b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:09.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.352+0000 7fc99a800640 1 -- 172.21.15.42:0/3269275336 shutdown_connections 2024-05-28T14:11:09.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.352+0000 7fc99a800640 1 -- 172.21.15.42:0/3269275336 wait complete. 2024-05-28T14:11:09.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T14:11:09.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.524+0000 7f812b200640 1 Processor -- start 2024-05-28T14:11:09.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.524+0000 7f812b200640 1 -- start start 2024-05-28T14:11:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.524+0000 7f812b200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8124144f80 0x7f8124072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.524+0000 7f812b200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f812406ee20 0x7f812406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.524+0000 7f812b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f812415b9b0 0x7f812415be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.524+0000 7f812b200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f812406f7c0 con 0x7f8124144f80 2024-05-28T14:11:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.524+0000 7f812b200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f81240724d0 con 0x7f812406ee20 2024-05-28T14:11:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.524+0000 7f812b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f81240afe40 con 0x7f812415b9b0 2024-05-28T14:11:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.528+0000 7f8123e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8124144f80 0x7f8124072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.528+0000 7f8123e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8124144f80 0x7f8124072960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59502/0 (socket says 172.21.15.42:59502) 2024-05-28T14:11:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.528+0000 7f8123e00640 1 -- 172.21.15.42:0/473410377 learned_addr learned my addr 172.21.15.42:0/473410377 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.528+0000 7f811b400640 1 --2- 172.21.15.42:0/473410377 >> v2:172.21.15.63:3300/0 conn(0x7f812406ee20 0x7f812406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.528+0000 7f8129200640 1 --2- 172.21.15.42:0/473410377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f812415b9b0 0x7f812415be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.528+0000 7f8123e00640 1 -- 172.21.15.42:0/473410377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f812415b9b0 msgr2=0x7f812415be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.528+0000 7f8123e00640 1 --2- 172.21.15.42:0/473410377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f812415b9b0 0x7f812415be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.528+0000 7f8123e00640 1 -- 172.21.15.42:0/473410377 >> v2:172.21.15.63:3300/0 conn(0x7f812406ee20 msgr2=0x7f812406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.528+0000 7f8123e00640 1 --2- 172.21.15.42:0/473410377 >> v2:172.21.15.63:3300/0 conn(0x7f812406ee20 0x7f812406f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.528+0000 7f8123e00640 1 -- 172.21.15.42:0/473410377 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812415c380 con 0x7f8124144f80 2024-05-28T14:11:09.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.528+0000 7f8123e00640 1 --2- 172.21.15.42:0/473410377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8124144f80 0x7f8124072960 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8124075500 tx=0x7f811401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d072dc850bc50e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:09.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.528+0000 7f8123400640 1 -- 172.21.15.42:0/473410377 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81140b1070 con 0x7f8124144f80 2024-05-28T14:11:09.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.528+0000 7f8123400640 1 -- 172.21.15.42:0/473410377 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81140ac630 con 0x7f8124144f80 2024-05-28T14:11:09.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.528+0000 7f8123400640 1 -- 172.21.15.42:0/473410377 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81140a8050 con 0x7f8124144f80 2024-05-28T14:11:09.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 -- 172.21.15.42:0/473410377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8124144f80 msgr2=0x7f8124072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:09.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 --2- 172.21.15.42:0/473410377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8124144f80 0x7f8124072960 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8124075500 tx=0x7f811401c520 comp rx=0 tx=0).stop 2024-05-28T14:11:09.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 -- 172.21.15.42:0/473410377 shutdown_connections 2024-05-28T14:11:09.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 --2- 172.21.15.42:0/473410377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f812415b9b0 0x7f812415be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 --2- 172.21.15.42:0/473410377 >> v2:172.21.15.63:3300/0 conn(0x7f812406ee20 0x7f812406f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 --2- 172.21.15.42:0/473410377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8124144f80 0x7f8124072960 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 -- 172.21.15.42:0/473410377 >> 172.21.15.42:0/473410377 conn(0x7f81240b21f0 msgr2=0x7f81240b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:09.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 -- 172.21.15.42:0/473410377 shutdown_connections 2024-05-28T14:11:09.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 -- 172.21.15.42:0/473410377 wait complete. 2024-05-28T14:11:09.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 Processor -- start 2024-05-28T14:11:09.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 -- start start 2024-05-28T14:11:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f812406ee20 0x7f8124131d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8124144f80 0x7f81240b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f812415b9b0 0x7f81240b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8124072ea0 con 0x7f8124144f80 2024-05-28T14:11:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8124072160 con 0x7f812406ee20 2024-05-28T14:11:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f812b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f81241436c0 con 0x7f812415b9b0 2024-05-28T14:11:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f811b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8124144f80 0x7f81240b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f811b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8124144f80 0x7f81240b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59514/0 (socket says 172.21.15.42:59514) 2024-05-28T14:11:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f811b400640 1 -- 172.21.15.42:0/283250103 learned_addr learned my addr 172.21.15.42:0/283250103 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f8123e00640 1 --2- 172.21.15.42:0/283250103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f812406ee20 0x7f8124131d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f8129200640 1 --2- 172.21.15.42:0/283250103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f812415b9b0 0x7f81240b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f811b400640 1 -- 172.21.15.42:0/283250103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f812415b9b0 msgr2=0x7f81240b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f811b400640 1 --2- 172.21.15.42:0/283250103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f812415b9b0 0x7f81240b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f811b400640 1 -- 172.21.15.42:0/283250103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f812406ee20 msgr2=0x7f8124131d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f811b400640 1 --2- 172.21.15.42:0/283250103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f812406ee20 0x7f8124131d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.532+0000 7f811b400640 1 -- 172.21.15.42:0/283250103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8124142620 con 0x7f8124144f80 2024-05-28T14:11:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.536+0000 7f811b400640 1 --2- 172.21.15.42:0/283250103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8124144f80 0x7f81240b4010 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f810c00e960 tx=0x7f810c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.536+0000 7f8120c00640 1 -- 172.21.15.42:0/283250103 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c017070 con 0x7f8124144f80 2024-05-28T14:11:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.536+0000 7f8120c00640 1 -- 172.21.15.42:0/283250103 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810c00ca00 con 0x7f8124144f80 2024-05-28T14:11:09.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.536+0000 7f8120c00640 1 -- 172.21.15.42:0/283250103 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c00cdb0 con 0x7f8124144f80 2024-05-28T14:11:09.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.536+0000 7f812b200640 1 -- 172.21.15.42:0/283250103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8124142900 con 0x7f8124144f80 2024-05-28T14:11:09.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.536+0000 7f812b200640 1 -- 172.21.15.42:0/283250103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8124142ec0 con 0x7f8124144f80 2024-05-28T14:11:09.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.536+0000 7f812b200640 1 -- 172.21.15.42:0/283250103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81240738b0 con 0x7f8124144f80 2024-05-28T14:11:09.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.536+0000 7f8120c00640 1 -- 172.21.15.42:0/283250103 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f810c01a070 con 0x7f8124144f80 2024-05-28T14:11:09.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.536+0000 7f8120c00640 1 --2- 172.21.15.42:0/283250103 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8100069850 0x7f810006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:09.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.536+0000 7f8120c00640 1 -- 172.21.15.42:0/283250103 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7f810c085a10 con 0x7f8124144f80 2024-05-28T14:11:09.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.536+0000 7f8123e00640 1 --2- 172.21.15.42:0/283250103 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8100069850 0x7f810006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:09.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.540+0000 7f8123e00640 1 --2- 172.21.15.42:0/283250103 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8100069850 0x7f810006bd10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f81140ae000 tx=0x7f8114009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:09.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.540+0000 7f8120c00640 1 -- 172.21.15.42:0/283250103 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f810c0573d0 con 0x7f8124144f80 2024-05-28T14:11:09.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.708+0000 7f812b200640 1 -- 172.21.15.42:0/283250103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f812404fdd0 con 0x7f8124144f80 2024-05-28T14:11:09.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.708+0000 7f8120c00640 1 -- 172.21.15.42:0/283250103 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f810c05b6d0 con 0x7f8124144f80 2024-05-28T14:11:09.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.712+0000 7f812b200640 1 -- 172.21.15.42:0/283250103 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8100069850 msgr2=0x7f810006bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:09.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.712+0000 7f812b200640 1 --2- 172.21.15.42:0/283250103 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8100069850 0x7f810006bd10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f81140ae000 tx=0x7f8114009040 comp rx=0 tx=0).stop 2024-05-28T14:11:09.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.712+0000 7f812b200640 1 -- 172.21.15.42:0/283250103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8124144f80 msgr2=0x7f81240b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:09.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.712+0000 7f812b200640 1 --2- 172.21.15.42:0/283250103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8124144f80 0x7f81240b4010 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f810c00e960 tx=0x7f810c00ee20 comp rx=0 tx=0).stop 2024-05-28T14:11:09.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.712+0000 7f812b200640 1 -- 172.21.15.42:0/283250103 shutdown_connections 2024-05-28T14:11:09.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.712+0000 7f812b200640 1 --2- 172.21.15.42:0/283250103 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8100069850 0x7f810006bd10 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.712+0000 7f812b200640 1 --2- 172.21.15.42:0/283250103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f812415b9b0 0x7f81240b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.712+0000 7f812b200640 1 --2- 172.21.15.42:0/283250103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8124144f80 0x7f81240b4010 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.712+0000 7f812b200640 1 --2- 172.21.15.42:0/283250103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f812406ee20 0x7f8124131d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:09.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.712+0000 7f812b200640 1 -- 172.21.15.42:0/283250103 >> 172.21.15.42:0/283250103 conn(0x7f81240b21f0 msgr2=0x7f812415a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:09.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.712+0000 7f812b200640 1 -- 172.21.15.42:0/283250103 shutdown_connections 2024-05-28T14:11:09.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:09.712+0000 7f812b200640 1 -- 172.21.15.42:0/283250103 wait complete. 2024-05-28T14:11:09.717 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T14:11:09.717 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:11:09.825 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:11:09.912 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T14:11:09.912 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:11:09.912 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:11:09.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:11:09.921 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:11:09.921 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:11:09.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:11:09.931 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:11:09.931 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:11:09.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:11:09.970 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:11:09.970 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:11:09.981 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:11:09.982 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-28T14:11:09.982 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:11:10.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f2161400640 1 Processor -- start 2024-05-28T14:11:10.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f2161400640 1 -- start start 2024-05-28T14:11:10.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f2161400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f215c15eb00 0x7f215c0cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f2161400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0cfe00 0x7f215c06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f2161400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f215c15eee0 con 0x7f215c06bf30 2024-05-28T14:11:10.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f2161400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f215c06c3a0 con 0x7f215c15eb00 2024-05-28T14:11:10.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f2161400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f215c0ceb20 con 0x7f215c0cfe00 2024-05-28T14:11:10.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f2153e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0cfe00 0x7f215c06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f215a800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f215c15eb00 0x7f215c0cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f2153e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0cfe00 0x7f215c06b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:53918/0 (socket says 172.21.15.42:53918) 2024-05-28T14:11:10.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f2153e00640 1 -- 172.21.15.42:0/1230873598 learned_addr learned my addr 172.21.15.42:0/1230873598 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:10.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f215a800640 1 -- 172.21.15.42:0/1230873598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0cfe00 msgr2=0x7f215c06b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:10.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f215a800640 1 --2- 172.21.15.42:0/1230873598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0cfe00 0x7f215c06b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f215a800640 1 -- 172.21.15.42:0/1230873598 >> v1:172.21.15.63:6789/0 conn(0x7f215c06bf30 legacy=0x7f215c0b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:10.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f215a800640 1 -- 172.21.15.42:0/1230873598 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c0cee00 con 0x7f215c15eb00 2024-05-28T14:11:10.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.172+0000 7f2159e00640 1 -- 172.21.15.42:0/1230873598 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4011297178 0 0) 0x7f21540014a0 con 0x7f215c06bf30 2024-05-28T14:11:10.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.176+0000 7f215a800640 1 --2- 172.21.15.42:0/1230873598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15eb00 0x7f215c0cf8c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f214c0097c0 tx=0x7f214c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f70cbec4ef559c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:10.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.176+0000 7f2159e00640 1 -- 172.21.15.42:0/1230873598 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c0b0070 con 0x7f215c15eb00 2024-05-28T14:11:10.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.176+0000 7f2159e00640 1 -- 172.21.15.42:0/1230873598 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214c0ab600 con 0x7f215c15eb00 2024-05-28T14:11:10.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.176+0000 7f2159e00640 1 -- 172.21.15.42:0/1230873598 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c0ab9d0 con 0x7f215c15eb00 2024-05-28T14:11:10.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.176+0000 7f2161400640 1 -- 172.21.15.42:0/1230873598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15eb00 msgr2=0x7f215c0cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:10.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.176+0000 7f2161400640 1 --2- 172.21.15.42:0/1230873598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15eb00 0x7f215c0cf8c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f214c0097c0 tx=0x7f214c01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.176+0000 7f2161400640 1 -- 172.21.15.42:0/1230873598 shutdown_connections 2024-05-28T14:11:10.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.176+0000 7f2161400640 1 --2- 172.21.15.42:0/1230873598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0cfe00 0x7f215c06b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.176+0000 7f2161400640 1 --2- 172.21.15.42:0/1230873598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15eb00 0x7f215c0cf8c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.176+0000 7f2161400640 1 -- 172.21.15.42:0/1230873598 >> 172.21.15.42:0/1230873598 conn(0x7f215c0b68f0 msgr2=0x7f215c0b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:10.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.176+0000 7f2161400640 1 -- 172.21.15.42:0/1230873598 shutdown_connections 2024-05-28T14:11:10.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.176+0000 7f2161400640 1 -- 172.21.15.42:0/1230873598 wait complete. 2024-05-28T14:11:10.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f2161400640 1 Processor -- start 2024-05-28T14:11:10.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f2161400640 1 -- start start 2024-05-28T14:11:10.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f2161400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c06bf30 0x7f215c14d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f2161400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0cfe00 0x7f215c156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f2161400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15eb00 0x7f215c148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f2161400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f215c16b610 con 0x7f215c06bf30 2024-05-28T14:11:10.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f2161400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f215c0ce850 con 0x7f215c15eb00 2024-05-28T14:11:10.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f2161400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f215c16b790 con 0x7f215c0cfe00 2024-05-28T14:11:10.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f215a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c06bf30 0x7f215c14d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f215a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c06bf30 0x7f215c14d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59528/0 (socket says 172.21.15.42:59528) 2024-05-28T14:11:10.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f215a800640 1 -- 172.21.15.42:0/186700430 learned_addr learned my addr 172.21.15.42:0/186700430 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:10.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f215b200640 1 --2- 172.21.15.42:0/186700430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15eb00 0x7f215c148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f2153e00640 1 --2- 172.21.15.42:0/186700430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0cfe00 0x7f215c156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f215a800640 1 -- 172.21.15.42:0/186700430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0cfe00 msgr2=0x7f215c156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:10.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f215a800640 1 --2- 172.21.15.42:0/186700430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0cfe00 0x7f215c156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f215a800640 1 -- 172.21.15.42:0/186700430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15eb00 msgr2=0x7f215c148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:10.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f215a800640 1 --2- 172.21.15.42:0/186700430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15eb00 0x7f215c148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f215a800640 1 -- 172.21.15.42:0/186700430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c0ae3b0 con 0x7f215c06bf30 2024-05-28T14:11:10.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f215a800640 1 --2- 172.21.15.42:0/186700430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c06bf30 0x7f215c14d440 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f214c01a0d0 tx=0x7f214c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:10.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f2152a00640 1 -- 172.21.15.42:0/186700430 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c0b0050 con 0x7f215c06bf30 2024-05-28T14:11:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f2152a00640 1 -- 172.21.15.42:0/186700430 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214c0bc070 con 0x7f215c06bf30 2024-05-28T14:11:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f2152a00640 1 -- 172.21.15.42:0/186700430 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c0b5a50 con 0x7f215c06bf30 2024-05-28T14:11:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f2161400640 1 -- 172.21.15.42:0/186700430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215c14dcd0 con 0x7f215c06bf30 2024-05-28T14:11:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.180+0000 7f2161400640 1 -- 172.21.15.42:0/186700430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f215c0ae950 con 0x7f215c06bf30 2024-05-28T14:11:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.184+0000 7f2161400640 1 -- 172.21.15.42:0/186700430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f215c068a90 con 0x7f215c06bf30 2024-05-28T14:11:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.184+0000 7f2152a00640 1 -- 172.21.15.42:0/186700430 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f214c0b5bf0 con 0x7f215c06bf30 2024-05-28T14:11:10.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.184+0000 7f2152a00640 1 --2- 172.21.15.42:0/186700430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2138069850 0x7f213806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.184+0000 7f2152a00640 1 -- 172.21.15.42:0/186700430 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7f214c11ff80 con 0x7f215c06bf30 2024-05-28T14:11:10.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.184+0000 7f2153e00640 1 --2- 172.21.15.42:0/186700430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2138069850 0x7f213806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.184+0000 7f2153e00640 1 --2- 172.21.15.42:0/186700430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2138069850 0x7f213806bd10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f215c15a850 tx=0x7f2144009240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:10.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.188+0000 7f2152a00640 1 -- 172.21.15.42:0/186700430 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f214c016090 con 0x7f215c06bf30 2024-05-28T14:11:10.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.392+0000 7f2161400640 1 -- 172.21.15.42:0/186700430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f215c070390 con 0x7f215c06bf30 2024-05-28T14:11:10.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.392+0000 7f2152a00640 1 -- 172.21.15.42:0/186700430 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+3867 (secure 0 0 0) 0x7f214c0f19a0 con 0x7f215c06bf30 2024-05-28T14:11:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:10.376 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":57,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7657,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/402076081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":402076081},{"type":"v1","addr":"172.21.15.63:6835","nonce":402076081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7998,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3779622485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3779622485},{"type":"v1","addr":"172.21.15.42:6835","nonce":3779622485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2961573665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2961573665},{"type":"v1","addr":"172.21.15.42:6837","nonce":2961573665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":56993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/4121043661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4121043661},{"type":"v1","addr":"172.21.15.63:6834","nonce":4121043661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[]} 2024-05-28T14:11:10.377 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 57 2024-05-28T14:11:10.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.392+0000 7f2161400640 1 -- 172.21.15.42:0/186700430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2138069850 msgr2=0x7f213806bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:10.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.392+0000 7f2161400640 1 --2- 172.21.15.42:0/186700430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2138069850 0x7f213806bd10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f215c15a850 tx=0x7f2144009240 comp rx=0 tx=0).stop 2024-05-28T14:11:10.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.392+0000 7f2161400640 1 -- 172.21.15.42:0/186700430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c06bf30 msgr2=0x7f215c14d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:10.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.392+0000 7f2161400640 1 --2- 172.21.15.42:0/186700430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c06bf30 0x7f215c14d440 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f214c01a0d0 tx=0x7f214c01ace0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.396+0000 7f2161400640 1 -- 172.21.15.42:0/186700430 shutdown_connections 2024-05-28T14:11:10.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.396+0000 7f2161400640 1 --2- 172.21.15.42:0/186700430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2138069850 0x7f213806bd10 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.396+0000 7f2161400640 1 --2- 172.21.15.42:0/186700430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c15eb00 0x7f215c148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.396+0000 7f2161400640 1 --2- 172.21.15.42:0/186700430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f215c0cfe00 0x7f215c156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.396+0000 7f2161400640 1 --2- 172.21.15.42:0/186700430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f215c06bf30 0x7f215c14d440 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.396+0000 7f2161400640 1 -- 172.21.15.42:0/186700430 >> 172.21.15.42:0/186700430 conn(0x7f215c0b68f0 msgr2=0x7f215c0b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:10.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.396+0000 7f2161400640 1 -- 172.21.15.42:0/186700430 shutdown_connections 2024-05-28T14:11:10.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.396+0000 7f2161400640 1 -- 172.21.15.42:0/186700430 wait complete. 2024-05-28T14:11:10.399 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-28T14:11:10.399 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-28T14:11:10.399 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-28T14:11:10.399 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:11:10.400 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-28T14:11:10.400 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-28T14:11:10.400 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-28T14:11:10.400 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:11:10.400 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-28T14:11:10.400 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-28T14:11:10.401 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-28T14:11:10.401 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:11:10.401 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-28T14:11:10.401 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-28T14:11:10.401 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-28T14:11:10.401 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:11:10.413 INFO:tasks.ceph.mds.a:Stopped 2024-05-28T14:11:10.414 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-28T14:11:10.415 INFO:tasks.ceph.mds.c:Stopped 2024-05-28T14:11:10.415 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-28T14:11:10.416 INFO:tasks.ceph.mds.d:Stopped 2024-05-28T14:11:10.416 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-28T14:11:10.417 INFO:tasks.ceph.mds.b:Stopped 2024-05-28T14:11:10.417 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-28T14:11:10.464 INFO:tasks.ceph.mds.a:Started 2024-05-28T14:11:10.464 DEBUG:teuthology.parallel:result is None 2024-05-28T14:11:10.466 INFO:tasks.ceph.mds.c:Started 2024-05-28T14:11:10.467 DEBUG:teuthology.parallel:result is None 2024-05-28T14:11:10.467 INFO:tasks.ceph.mds.d:Started 2024-05-28T14:11:10.468 DEBUG:teuthology.parallel:result is None 2024-05-28T14:11:10.469 INFO:tasks.ceph.mds.b:Started 2024-05-28T14:11:10.469 DEBUG:teuthology.parallel:result is None 2024-05-28T14:11:10.470 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-28T14:11:10.552 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-05-28T14:11:10.552 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-05-28T14:11:10.562 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-28T14:11:10.562 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T14:11:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.632+0000 7f0198000640 1 Processor -- start 2024-05-28T14:11:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.632+0000 7f0198000640 1 -- start start 2024-05-28T14:11:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.632+0000 7f0198000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0190145920 0x7f019006f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.632+0000 7f0198000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f019006db40 0x7f0190071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.632+0000 7f0198000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f019006dfa0 con 0x7f0190072300 2024-05-28T14:11:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.632+0000 7f0198000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0190072750 con 0x7f0190145920 2024-05-28T14:11:10.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.632+0000 7f0198000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f019006e240 con 0x7f019006db40 2024-05-28T14:11:10.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f0196000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0190072300 0x7f0190160100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:44760/0 (socket says 172.21.15.42:44760) 2024-05-28T14:11:10.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f0196000640 1 -- 172.21.15.42:0/1966107561 learned_addr learned my addr 172.21.15.42:0/1966107561 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:10.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f0195600640 1 --2- 172.21.15.42:0/1966107561 >> v2:172.21.15.63:3300/0 conn(0x7f0190145920 0x7f019006f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f0194c00640 1 --2- 172.21.15.42:0/1966107561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f019006db40 0x7f0190071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 -- 172.21.15.42:0/1966107561 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4037730417 0 0) 0x7f019006dfa0 con 0x7f0190072300 2024-05-28T14:11:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 -- 172.21.15.42:0/1966107561 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0178003530 con 0x7f0190072300 2024-05-28T14:11:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 -- 172.21.15.42:0/1966107561 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3664704082 0 0) 0x7f0178003530 con 0x7f0190072300 2024-05-28T14:11:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 -- 172.21.15.42:0/1966107561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f019006db40 msgr2=0x7f0190071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 --2- 172.21.15.42:0/1966107561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f019006db40 0x7f0190071dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 -- 172.21.15.42:0/1966107561 >> v2:172.21.15.63:3300/0 conn(0x7f0190145920 msgr2=0x7f019006f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 --2- 172.21.15.42:0/1966107561 >> v2:172.21.15.63:3300/0 conn(0x7f0190145920 0x7f019006f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 -- 172.21.15.42:0/1966107561 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019006e520 con 0x7f0190072300 2024-05-28T14:11:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 -- 172.21.15.42:0/1966107561 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f0188001a00 con 0x7f0190072300 2024-05-28T14:11:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 -- 172.21.15.42:0/1966107561 >> v1:172.21.15.42:6789/0 conn(0x7f0190072300 legacy=0x7f0190160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 --2- 172.21.15.42:0/1966107561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f019006db40 0x7f01780039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 --2- 172.21.15.42:0/1966107561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0178003f40 0x7f0178004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 --2- 172.21.15.42:0/1966107561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0178004d50 0x7f01780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 -- 172.21.15.42:0/1966107561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0190072750 con 0x7f0178003f40 2024-05-28T14:11:10.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 -- 172.21.15.42:0/1966107561 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f019006e240 con 0x7f0178004d50 2024-05-28T14:11:10.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f018fe00640 1 -- 172.21.15.42:0/1966107561 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f017800ba20 con 0x7f019006db40 2024-05-28T14:11:10.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f0194c00640 1 --2- 172.21.15.42:0/1966107561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0178003f40 0x7f0178004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f0195600640 1 --2- 172.21.15.42:0/1966107561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f019006db40 0x7f01780039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f0196000640 1 --2- 172.21.15.42:0/1966107561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0178004d50 0x7f01780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f0195600640 1 -- 172.21.15.42:0/1966107561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0178004d50 msgr2=0x7f01780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:10.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f0195600640 1 --2- 172.21.15.42:0/1966107561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0178004d50 0x7f01780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f0195600640 1 -- 172.21.15.42:0/1966107561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0178003f40 msgr2=0x7f0178004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:10.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f0195600640 1 --2- 172.21.15.42:0/1966107561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0178003f40 0x7f0178004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.636+0000 7f0195600640 1 -- 172.21.15.42:0/1966107561 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0178003530 con 0x7f019006db40 2024-05-28T14:11:10.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.640+0000 7f0195600640 1 --2- 172.21.15.42:0/1966107561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f019006db40 0x7f01780039c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0184009b10 tx=0x7f018401cab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=897bec2b27ae9dff server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:10.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.640+0000 7f018fe00640 1 -- 172.21.15.42:0/1966107561 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01840096b0 con 0x7f019006db40 2024-05-28T14:11:10.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.640+0000 7f018fe00640 1 -- 172.21.15.42:0/1966107561 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01840a9480 con 0x7f019006db40 2024-05-28T14:11:10.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.640+0000 7f018fe00640 1 -- 172.21.15.42:0/1966107561 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01840a9870 con 0x7f019006db40 2024-05-28T14:11:10.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.640+0000 7f0198000640 1 -- 172.21.15.42:0/1966107561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f019006db40 msgr2=0x7f01780039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:10.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.640+0000 7f0198000640 1 --2- 172.21.15.42:0/1966107561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f019006db40 0x7f01780039c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0184009b10 tx=0x7f018401cab0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.640+0000 7f0196000640 1 -- 172.21.15.42:0/1966107561 reap_dead start 2024-05-28T14:11:10.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.640+0000 7f0198000640 1 -- 172.21.15.42:0/1966107561 shutdown_connections 2024-05-28T14:11:10.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.640+0000 7f0198000640 1 -- 172.21.15.42:0/1966107561 >> 172.21.15.42:0/1966107561 conn(0x7f019006b9f0 msgr2=0x7f019006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:10.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.640+0000 7f0198000640 1 -- 172.21.15.42:0/1966107561 shutdown_connections 2024-05-28T14:11:10.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.640+0000 7f0198000640 1 -- 172.21.15.42:0/1966107561 wait complete. 2024-05-28T14:11:10.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.640+0000 7f0198000640 1 Processor -- start 2024-05-28T14:11:10.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.640+0000 7f0198000640 1 -- start start 2024-05-28T14:11:10.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0198000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f019006db40 0x7f01900ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0198000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01900c6190 0x7f0190134120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0198000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01901626a0 0x7f01900aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0198000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f01901673c0 con 0x7f01901626a0 2024-05-28T14:11:10.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0198000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0190167240 con 0x7f01900c6190 2024-05-28T14:11:10.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0198000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0190167540 con 0x7f019006db40 2024-05-28T14:11:10.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0196000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01901626a0 0x7f01900aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0195600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f019006db40 0x7f01900ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0196000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01901626a0 0x7f01900aea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59578/0 (socket says 172.21.15.42:59578) 2024-05-28T14:11:10.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0196000640 1 -- 172.21.15.42:0/1740017299 learned_addr learned my addr 172.21.15.42:0/1740017299 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:10.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0194c00640 1 --2- 172.21.15.42:0/1740017299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01900c6190 0x7f0190134120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0196000640 1 -- 172.21.15.42:0/1740017299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f019006db40 msgr2=0x7f01900ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:10.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0196000640 1 --2- 172.21.15.42:0/1740017299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f019006db40 0x7f01900ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0196000640 1 -- 172.21.15.42:0/1740017299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01900c6190 msgr2=0x7f0190134120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:10.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0196000640 1 --2- 172.21.15.42:0/1740017299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01900c6190 0x7f0190134120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:10.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0196000640 1 -- 172.21.15.42:0/1740017299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01901318b0 con 0x7f01901626a0 2024-05-28T14:11:10.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0196000640 1 --2- 172.21.15.42:0/1740017299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01901626a0 0x7f01900aea40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f019006c820 tx=0x7f0188007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:10.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f018d600640 1 -- 172.21.15.42:0/1740017299 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188017020 con 0x7f01901626a0 2024-05-28T14:11:10.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f018d600640 1 -- 172.21.15.42:0/1740017299 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f018800ab80 con 0x7f01901626a0 2024-05-28T14:11:10.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f018d600640 1 -- 172.21.15.42:0/1740017299 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01880133f0 con 0x7f01901626a0 2024-05-28T14:11:10.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0198000640 1 -- 172.21.15.42:0/1740017299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0190131b90 con 0x7f01901626a0 2024-05-28T14:11:10.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0198000640 1 -- 172.21.15.42:0/1740017299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0190132150 con 0x7f01901626a0 2024-05-28T14:11:10.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f0198000640 1 -- 172.21.15.42:0/1740017299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0190073890 con 0x7f01901626a0 2024-05-28T14:11:10.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.644+0000 7f018d600640 1 -- 172.21.15.42:0/1740017299 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f018801a020 con 0x7f01901626a0 2024-05-28T14:11:10.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.648+0000 7f018d600640 1 --2- 172.21.15.42:0/1740017299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0170069880 0x7f017006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:10.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.648+0000 7f018d600640 1 -- 172.21.15.42:0/1740017299 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7f0188085dd0 con 0x7f01901626a0 2024-05-28T14:11:10.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.648+0000 7f0195600640 1 --2- 172.21.15.42:0/1740017299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0170069880 0x7f017006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:10.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.648+0000 7f0195600640 1 --2- 172.21.15.42:0/1740017299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0170069880 0x7f017006bd40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f01840ae000 tx=0x7f01840046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:10.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.652+0000 7f018d600640 1 -- 172.21.15.42:0/1740017299 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0188021020 con 0x7f01901626a0 2024-05-28T14:11:10.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:10.828+0000 7f0198000640 1 -- 172.21.15.42:0/1740017299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f01900aff40 con 0x7f01901626a0 2024-05-28T14:11:11.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.364+0000 7f018d600640 1 -- 172.21.15.42:0/1740017299 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) ==== 97+0+0 (secure 0 0 0) 0x7f0188057de0 con 0x7f01901626a0 2024-05-28T14:11:11.347 INFO:teuthology.orchestra.run.smithi042.stderr: removed all blocklist entries 2024-05-28T14:11:11.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.364+0000 7f0198000640 1 -- 172.21.15.42:0/1740017299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0170069880 msgr2=0x7f017006bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:11.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.364+0000 7f0198000640 1 --2- 172.21.15.42:0/1740017299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0170069880 0x7f017006bd40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f01840ae000 tx=0x7f01840046d0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.364+0000 7f0198000640 1 -- 172.21.15.42:0/1740017299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01901626a0 msgr2=0x7f01900aea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:11.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.364+0000 7f0198000640 1 --2- 172.21.15.42:0/1740017299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01901626a0 0x7f01900aea40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f019006c820 tx=0x7f0188007a40 comp rx=0 tx=0).stop 2024-05-28T14:11:11.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.368+0000 7f0198000640 1 -- 172.21.15.42:0/1740017299 shutdown_connections 2024-05-28T14:11:11.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.368+0000 7f0198000640 1 --2- 172.21.15.42:0/1740017299 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0170069880 0x7f017006bd40 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.368+0000 7f0198000640 1 --2- 172.21.15.42:0/1740017299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01901626a0 0x7f01900aea40 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.368+0000 7f0198000640 1 --2- 172.21.15.42:0/1740017299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01900c6190 0x7f0190134120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.368+0000 7f0198000640 1 --2- 172.21.15.42:0/1740017299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f019006db40 0x7f01900ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.368+0000 7f0198000640 1 -- 172.21.15.42:0/1740017299 >> 172.21.15.42:0/1740017299 conn(0x7f019006b9f0 msgr2=0x7f01900b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:11.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.368+0000 7f0198000640 1 -- 172.21.15.42:0/1740017299 shutdown_connections 2024-05-28T14:11:11.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.368+0000 7f0198000640 1 -- 172.21.15.42:0/1740017299 wait complete. 2024-05-28T14:11:11.377 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-28T14:11:11.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.536+0000 7f9c2da00640 1 Processor -- start 2024-05-28T14:11:11.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.536+0000 7f9c2da00640 1 -- start start 2024-05-28T14:11:11.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.540+0000 7f9c2da00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9c28067810 0x7f9c2815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:11.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.540+0000 7f9c2da00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c2806eaa0 con 0x7f9c280b42e0 2024-05-28T14:11:11.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.540+0000 7f9c2da00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c2806ed80 con 0x7f9c280d0420 2024-05-28T14:11:11.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.540+0000 7f9c2da00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9c2806f060 con 0x7f9c28067810 2024-05-28T14:11:11.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.540+0000 7f9c1e000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f9c280b42e0 0x7f9c2806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:44772/0 (socket says 172.21.15.42:44772) 2024-05-28T14:11:11.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.540+0000 7f9c1e000640 1 -- 172.21.15.42:0/1323196678 learned_addr learned my addr 172.21.15.42:0/1323196678 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:11.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.540+0000 7f9c27400640 1 --2- 172.21.15.42:0/1323196678 >> v2:172.21.15.63:3300/0 conn(0x7f9c28067810 0x7f9c2815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:11.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.540+0000 7f9c26000640 1 -- 172.21.15.42:0/1323196678 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2570667735 0 0) 0x7f9c2806eaa0 con 0x7f9c280b42e0 2024-05-28T14:11:11.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.540+0000 7f9c26000640 1 -- 172.21.15.42:0/1323196678 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c0c003530 con 0x7f9c280b42e0 2024-05-28T14:11:11.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.540+0000 7f9c26000640 1 -- 172.21.15.42:0/1323196678 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 170134999 0 0) 0x7f9c2806ed80 con 0x7f9c280d0420 2024-05-28T14:11:11.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.540+0000 7f9c26000640 1 -- 172.21.15.42:0/1323196678 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c2806eaa0 con 0x7f9c280d0420 2024-05-28T14:11:11.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.540+0000 7f9c27400640 1 -- 172.21.15.42:0/1323196678 >> v1:172.21.15.63:6789/0 conn(0x7f9c280d0420 legacy=0x7f9c280b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:11.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.540+0000 7f9c27400640 1 -- 172.21.15.42:0/1323196678 >> v1:172.21.15.42:6789/0 conn(0x7f9c280b42e0 legacy=0x7f9c2806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:11.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.540+0000 7f9c27400640 1 -- 172.21.15.42:0/1323196678 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2806f340 con 0x7f9c28067810 2024-05-28T14:11:11.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.544+0000 7f9c27400640 1 --2- 172.21.15.42:0/1323196678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c28067810 0x7f9c2815ed10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009d90 tx=0x7f9c2001d340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0de94e4668a681b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:11.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.544+0000 7f9c26000640 1 -- 172.21.15.42:0/1323196678 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c200b2070 con 0x7f9c28067810 2024-05-28T14:11:11.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.544+0000 7f9c26000640 1 -- 172.21.15.42:0/1323196678 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c200a58b0 con 0x7f9c28067810 2024-05-28T14:11:11.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.544+0000 7f9c26000640 1 -- 172.21.15.42:0/1323196678 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c200a5c80 con 0x7f9c28067810 2024-05-28T14:11:11.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.544+0000 7f9c2da00640 1 -- 172.21.15.42:0/1323196678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c28067810 msgr2=0x7f9c2815ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:11.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.544+0000 7f9c2da00640 1 --2- 172.21.15.42:0/1323196678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c28067810 0x7f9c2815ed10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009d90 tx=0x7f9c2001d340 comp rx=0 tx=0).stop 2024-05-28T14:11:11.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.544+0000 7f9c2da00640 1 -- 172.21.15.42:0/1323196678 shutdown_connections 2024-05-28T14:11:11.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.544+0000 7f9c2da00640 1 --2- 172.21.15.42:0/1323196678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c28067810 0x7f9c2815ed10 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.544+0000 7f9c2da00640 1 -- 172.21.15.42:0/1323196678 >> 172.21.15.42:0/1323196678 conn(0x7f9c280d0b40 msgr2=0x7f9c280d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:11.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.544+0000 7f9c2da00640 1 -- 172.21.15.42:0/1323196678 shutdown_connections 2024-05-28T14:11:11.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.544+0000 7f9c2da00640 1 -- 172.21.15.42:0/1323196678 wait complete. 2024-05-28T14:11:11.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.544+0000 7f9c2da00640 1 Processor -- start 2024-05-28T14:11:11.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.544+0000 7f9c2da00640 1 -- start start 2024-05-28T14:11:11.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c2da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c28067810 0x7f9c2814c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:11.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c2da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c280b42e0 0x7f9c281614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:11.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c2da00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c280d0420 0x7f9c280af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:11.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c2da00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2806cd30 con 0x7f9c28067810 2024-05-28T14:11:11.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c2da00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2806cbb0 con 0x7f9c280b42e0 2024-05-28T14:11:11.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c2da00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9c2806ceb0 con 0x7f9c280d0420 2024-05-28T14:11:11.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c26a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c28067810 0x7f9c2814c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:11.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c26a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c28067810 0x7f9c2814c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59594/0 (socket says 172.21.15.42:59594) 2024-05-28T14:11:11.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c26a00640 1 -- 172.21.15.42:0/4119626072 learned_addr learned my addr 172.21.15.42:0/4119626072 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:11.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c27400640 1 --2- 172.21.15.42:0/4119626072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c280d0420 0x7f9c280af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:11.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c1e000640 1 --2- 172.21.15.42:0/4119626072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c280b42e0 0x7f9c281614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:11.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c26a00640 1 -- 172.21.15.42:0/4119626072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c280d0420 msgr2=0x7f9c280af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:11.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c26a00640 1 --2- 172.21.15.42:0/4119626072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c280d0420 0x7f9c280af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c26a00640 1 -- 172.21.15.42:0/4119626072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c280b42e0 msgr2=0x7f9c281614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:11.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c26a00640 1 --2- 172.21.15.42:0/4119626072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c280b42e0 0x7f9c281614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c26a00640 1 -- 172.21.15.42:0/4119626072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c28150880 con 0x7f9c28067810 2024-05-28T14:11:11.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c26a00640 1 --2- 172.21.15.42:0/4119626072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c28067810 0x7f9c2814c750 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800c530 tx=0x7f9c1800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:11.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c1f400640 1 -- 172.21.15.42:0/4119626072 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c180052d0 con 0x7f9c28067810 2024-05-28T14:11:11.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c1f400640 1 -- 172.21.15.42:0/4119626072 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c18005490 con 0x7f9c28067810 2024-05-28T14:11:11.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c1f400640 1 -- 172.21.15.42:0/4119626072 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18012850 con 0x7f9c28067810 2024-05-28T14:11:11.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c2da00640 1 -- 172.21.15.42:0/4119626072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c28150b60 con 0x7f9c28067810 2024-05-28T14:11:11.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c2da00640 1 -- 172.21.15.42:0/4119626072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c28151120 con 0x7f9c28067810 2024-05-28T14:11:11.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c2da00640 1 -- 172.21.15.42:0/4119626072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c28068a90 con 0x7f9c28067810 2024-05-28T14:11:11.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.548+0000 7f9c1f400640 1 -- 172.21.15.42:0/4119626072 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f9c180129f0 con 0x7f9c28067810 2024-05-28T14:11:11.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.552+0000 7f9c1f400640 1 --2- 172.21.15.42:0/4119626072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9c04069850 0x7f9c0406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:11.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.552+0000 7f9c1f400640 1 -- 172.21.15.42:0/4119626072 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4885+0+0 (secure 0 0 0) 0x7f9c18085790 con 0x7f9c28067810 2024-05-28T14:11:11.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.552+0000 7f9c1e000640 1 --2- 172.21.15.42:0/4119626072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9c04069850 0x7f9c0406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:11.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.552+0000 7f9c1e000640 1 --2- 172.21.15.42:0/4119626072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9c04069850 0x7f9c0406bd10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9c2815d890 tx=0x7f9c10009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:11.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.556+0000 7f9c1f400640 1 -- 172.21.15.42:0/4119626072 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9c18057340 con 0x7f9c28067810 2024-05-28T14:11:11.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.780+0000 7f9c2da00640 1 -- 172.21.15.42:0/4119626072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9c280b3f20 con 0x7f9c28067810 2024-05-28T14:11:11.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.780+0000 7f9c1f400640 1 -- 172.21.15.42:0/4119626072 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f9c18056c40 con 0x7f9c28067810 2024-05-28T14:11:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:11.766 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:11:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_dump": [ 2024-05-28T14:11:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.a", 2024-05-28T14:11:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmE+3RLRAADuTKGfKeVafGeLpgr9WKUg==", 2024-05-28T14:11:11.766 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:11:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:11:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:11:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.b", 2024-05-28T14:11:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVm1rmVOhAAV46KzDJnRfDZYIZanS0g2g==", 2024-05-28T14:11:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:11:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:11:11.767 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.c", 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVme4aYNRAAVxJTlugnxtFTexUclF/IwA==", 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.768 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.d", 2024-05-28T14:11:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmeAdBBhAAr4IVFQpjQXFVlnFTHnPV9w==", 2024-05-28T14:11:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:11:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:11:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:11:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.769 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.0", 2024-05-28T14:11:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDB4lVmGMDBJhAA3jIyR7XKLGXEgimIGNsw6A==", 2024-05-28T14:11:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:11:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:11:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.1", 2024-05-28T14:11:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDE4lVmxMisLxAAcIz7GPlkZwvW5ig+DbAVGg==", 2024-05-28T14:11:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:11:11.770 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:11:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.2", 2024-05-28T14:11:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDH4lVmSSk4OhAAmLlLysM4Im/Kew4Xbq2/OA==", 2024-05-28T14:11:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:11:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:11:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.771 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.3", 2024-05-28T14:11:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDL4lVmvzrYBhAAiAAOQXZBuTsMVYa1vE5rkg==", 2024-05-28T14:11:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:11:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:11:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.4", 2024-05-28T14:11:11.772 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDQ4lVmrqnTOhAA2pIfgBT9LJDIhp+GgFy8Gg==", 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.5", 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDU4lVmYpsQCxAAoTK2aMBPrHKvsUnNOTj/Dw==", 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.6", 2024-05-28T14:11:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDX4lVm6TCAFxAAs3rkDyKcLhWrn/EkIo0bVA==", 2024-05-28T14:11:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:11:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:11:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.7", 2024-05-28T14:11:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDa4lVmr5NwIRAA7Lb70VYV82dr0Frn8CCUXg==", 2024-05-28T14:11:11.774 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.0", 2024-05-28T14:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw==", 2024-05-28T14:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-05-28T14:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-28T14:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.1", 2024-05-28T14:11:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmcfkLFBAADNyODKTtYAlFWquZ/LeeqQ==", 2024-05-28T14:11:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:11:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:11:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:11:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:11:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.2", 2024-05-28T14:11:11.776 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmoxx+EBAA8mLNak5AwRmWQIgkYtMOrg==", 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.3", 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmxSUsGhAAbgr1eOh/sMfrEONKKHFo0g==", 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:11:11.777 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:11:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.admin", 2024-05-28T14:11:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmOSBPERAAt4x2DBal4PnqxqeWQutLDQ==", 2024-05-28T14:11:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:11:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:11:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:11:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.x", 2024-05-28T14:11:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmNO5tKRAA70hPLwoE83keovlq2rolxQ==", 2024-05-28T14:11:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:11:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T14:11:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.y", 2024-05-28T14:11:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmxTVyJRAATMLAAqKne20YvV537uXZ6g==", 2024-05-28T14:11:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:11:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:11:11.779 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T14:11:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:11:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:11.780 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-28T14:11:11.780 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:11:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.784+0000 7f9c2da00640 1 -- 172.21.15.42:0/4119626072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9c04069850 msgr2=0x7f9c0406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.784+0000 7f9c2da00640 1 --2- 172.21.15.42:0/4119626072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9c04069850 0x7f9c0406bd10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9c2815d890 tx=0x7f9c10009040 comp rx=0 tx=0).stop 2024-05-28T14:11:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.784+0000 7f9c2da00640 1 -- 172.21.15.42:0/4119626072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c28067810 msgr2=0x7f9c2814c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.784+0000 7f9c2da00640 1 --2- 172.21.15.42:0/4119626072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c28067810 0x7f9c2814c750 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800c530 tx=0x7f9c1800c9f0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.784+0000 7f9c2da00640 1 -- 172.21.15.42:0/4119626072 shutdown_connections 2024-05-28T14:11:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.784+0000 7f9c2da00640 1 --2- 172.21.15.42:0/4119626072 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9c04069850 0x7f9c0406bd10 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.784+0000 7f9c2da00640 1 --2- 172.21.15.42:0/4119626072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c280d0420 0x7f9c280af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.784+0000 7f9c2da00640 1 --2- 172.21.15.42:0/4119626072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c280b42e0 0x7f9c281614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.784+0000 7f9c2da00640 1 --2- 172.21.15.42:0/4119626072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c28067810 0x7f9c2814c750 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.784+0000 7f9c2da00640 1 -- 172.21.15.42:0/4119626072 >> 172.21.15.42:0/4119626072 conn(0x7f9c280d0b40 msgr2=0x7f9c281519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:11.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.784+0000 7f9c2da00640 1 -- 172.21.15.42:0/4119626072 shutdown_connections 2024-05-28T14:11:11.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.784+0000 7f9c2da00640 1 -- 172.21.15.42:0/4119626072 wait complete. 2024-05-28T14:11:11.787 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:11:11.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.944+0000 7f2766a00640 1 Processor -- start 2024-05-28T14:11:11.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.944+0000 7f2766a00640 1 -- start start 2024-05-28T14:11:11.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.944+0000 7f2766a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f276015ba40 0x7f27600d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:11.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.944+0000 7f2766a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f276015f280 con 0x7f276006f660 2024-05-28T14:11:11.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.944+0000 7f2766a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f276015f560 con 0x7f2760072dd0 2024-05-28T14:11:11.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.944+0000 7f2766a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f276015f030 con 0x7f276015ba40 2024-05-28T14:11:11.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.944+0000 7f275fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f276006f660 0x7f2760073980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:44786/0 (socket says 172.21.15.42:44786) 2024-05-28T14:11:11.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.944+0000 7f275fe00640 1 -- 172.21.15.42:0/1856334281 learned_addr learned my addr 172.21.15.42:0/1856334281 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:11.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f2757400640 1 --2- 172.21.15.42:0/1856334281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f276015ba40 0x7f27600d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:11.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 -- 172.21.15.42:0/1856334281 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1138628408 0 0) 0x7f276015f280 con 0x7f276006f660 2024-05-28T14:11:11.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 -- 172.21.15.42:0/1856334281 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2744003530 con 0x7f276006f660 2024-05-28T14:11:11.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 -- 172.21.15.42:0/1856334281 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2228204538 0 0) 0x7f2744003530 con 0x7f276006f660 2024-05-28T14:11:11.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 -- 172.21.15.42:0/1856334281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f276015ba40 msgr2=0x7f27600d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:11.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 --2- 172.21.15.42:0/1856334281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f276015ba40 0x7f27600d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 -- 172.21.15.42:0/1856334281 >> v1:172.21.15.63:6789/0 conn(0x7f2760072dd0 legacy=0x7f276006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:11.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 -- 172.21.15.42:0/1856334281 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276015f840 con 0x7f276006f660 2024-05-28T14:11:11.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 -- 172.21.15.42:0/1856334281 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f2750001680 con 0x7f276006f660 2024-05-28T14:11:11.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 -- 172.21.15.42:0/1856334281 >> v1:172.21.15.42:6789/0 conn(0x7f276006f660 legacy=0x7f2760073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:11.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 --2- 172.21.15.42:0/1856334281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27440039f0 0x7f2744003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:11.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 --2- 172.21.15.42:0/1856334281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2744004810 0x7f2744004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:11.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 --2- 172.21.15.42:0/1856334281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2744005190 0x7f274400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:11.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 -- 172.21.15.42:0/1856334281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f276015f030 con 0x7f27440039f0 2024-05-28T14:11:11.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 -- 172.21.15.42:0/1856334281 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f274400bfc0 con 0x7f2744005190 2024-05-28T14:11:11.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275f400640 1 -- 172.21.15.42:0/1856334281 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f274400e3a0 con 0x7f2744004810 2024-05-28T14:11:11.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275fe00640 1 --2- 172.21.15.42:0/1856334281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27440039f0 0x7f2744003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:11.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f2757400640 1 --2- 172.21.15.42:0/1856334281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2744004810 0x7f2744004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:11.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f2764a00640 1 --2- 172.21.15.42:0/1856334281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2744005190 0x7f274400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:11.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275fe00640 1 -- 172.21.15.42:0/1856334281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2744004810 msgr2=0x7f2744004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:11.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275fe00640 1 --2- 172.21.15.42:0/1856334281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2744004810 0x7f2744004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275fe00640 1 -- 172.21.15.42:0/1856334281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2744005190 msgr2=0x7f274400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:11.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275fe00640 1 --2- 172.21.15.42:0/1856334281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2744005190 0x7f274400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f275fe00640 1 -- 172.21.15.42:0/1856334281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2744003530 con 0x7f27440039f0 2024-05-28T14:11:11.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.948+0000 7f2764a00640 1 --2- 172.21.15.42:0/1856334281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2744005190 0x7f274400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:11.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f275fe00640 1 --2- 172.21.15.42:0/1856334281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27440039f0 0x7f2744003e10 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f275000c1f0 tx=0x7f275001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259b0ca9364ccdb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:11.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f275f400640 1 -- 172.21.15.42:0/1856334281 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750007d00 con 0x7f27440039f0 2024-05-28T14:11:11.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f275f400640 1 -- 172.21.15.42:0/1856334281 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27500a9020 con 0x7f27440039f0 2024-05-28T14:11:11.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f275f400640 1 -- 172.21.15.42:0/1856334281 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27500a5b30 con 0x7f27440039f0 2024-05-28T14:11:11.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2766a00640 1 -- 172.21.15.42:0/1856334281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27440039f0 msgr2=0x7f2744003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:11.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2766a00640 1 --2- 172.21.15.42:0/1856334281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27440039f0 0x7f2744003e10 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f275000c1f0 tx=0x7f275001cc30 comp rx=0 tx=0).stop 2024-05-28T14:11:11.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2764a00640 1 -- 172.21.15.42:0/1856334281 reap_dead start 2024-05-28T14:11:11.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2766a00640 1 -- 172.21.15.42:0/1856334281 shutdown_connections 2024-05-28T14:11:11.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2766a00640 1 -- 172.21.15.42:0/1856334281 >> 172.21.15.42:0/1856334281 conn(0x7f276006bdf0 msgr2=0x7f276006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:11.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2766a00640 1 -- 172.21.15.42:0/1856334281 shutdown_connections 2024-05-28T14:11:11.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2766a00640 1 -- 172.21.15.42:0/1856334281 wait complete. 2024-05-28T14:11:11.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2766a00640 1 Processor -- start 2024-05-28T14:11:11.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2766a00640 1 -- start start 2024-05-28T14:11:11.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2766a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27440039f0 0x7f276014b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:11.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2766a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f276009b580 0x7f276015b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:11.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2766a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276009ba00 0x7f276015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:11.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2766a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f27600d0a80 con 0x7f27440039f0 2024-05-28T14:11:11.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2766a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f27600d0900 con 0x7f276009ba00 2024-05-28T14:11:11.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f2766a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f27600d0c00 con 0x7f276009b580 2024-05-28T14:11:11.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f275fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27440039f0 0x7f276014b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:11.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f275fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27440039f0 0x7f276014b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59606/0 (socket says 172.21.15.42:59606) 2024-05-28T14:11:11.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.952+0000 7f275fe00640 1 -- 172.21.15.42:0/164717263 learned_addr learned my addr 172.21.15.42:0/164717263 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:11.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f2764a00640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276009ba00 0x7f276015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:11.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f2757400640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f276009b580 0x7f276015b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:11.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f275fe00640 1 -- 172.21.15.42:0/164717263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f276009b580 msgr2=0x7f276015b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:11.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f275fe00640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f276009b580 0x7f276015b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f275fe00640 1 -- 172.21.15.42:0/164717263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276009ba00 msgr2=0x7f276015bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:11.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f275fe00640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276009ba00 0x7f276015bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:11.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f275fe00640 1 -- 172.21.15.42:0/164717263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27600ccda0 con 0x7f27440039f0 2024-05-28T14:11:11.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f2757400640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f276009b580 0x7f276015b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:11.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f2764a00640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276009ba00 0x7f276015bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:11.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f275fe00640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27440039f0 0x7f276014b810 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f275000c1f0 tx=0x7f2744005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:11.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f275cc00640 1 -- 172.21.15.42:0/164717263 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27500010a0 con 0x7f27440039f0 2024-05-28T14:11:11.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f275cc00640 1 -- 172.21.15.42:0/164717263 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27500a5b30 con 0x7f27440039f0 2024-05-28T14:11:11.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f2766a00640 1 -- 172.21.15.42:0/164717263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27600cd080 con 0x7f27440039f0 2024-05-28T14:11:11.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f275cc00640 1 -- 172.21.15.42:0/164717263 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27500047f0 con 0x7f27440039f0 2024-05-28T14:11:11.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f2766a00640 1 -- 172.21.15.42:0/164717263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27600cd640 con 0x7f27440039f0 2024-05-28T14:11:11.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f2766a00640 1 -- 172.21.15.42:0/164717263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2760068a90 con 0x7f27440039f0 2024-05-28T14:11:11.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f275cc00640 1 -- 172.21.15.42:0/164717263 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f2750004990 con 0x7f27440039f0 2024-05-28T14:11:11.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f275cc00640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f273c069880 0x7f273c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:11.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.956+0000 7f275cc00640 1 -- 172.21.15.42:0/164717263 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4885+0+0 (secure 0 0 0) 0x7f275011f790 con 0x7f27440039f0 2024-05-28T14:11:11.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.960+0000 7f2757400640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f273c069880 0x7f273c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:11.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.960+0000 7f2757400640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f273c069880 0x7f273c06bd40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f276009b370 tx=0x7f2748008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:11.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:11.960+0000 7f275cc00640 1 -- 172.21.15.42:0/164717263 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2750016970 con 0x7f27440039f0 2024-05-28T14:11:12.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.132+0000 7f2766a00640 1 -- 172.21.15.42:0/164717263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f276006c2a0 con 0x7f27440039f0 2024-05-28T14:11:12.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.132+0000 7f275cc00640 1 -- 172.21.15.42:0/164717263 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+12876 (secure 0 0 0) 0x7f27500ad3c0 con 0x7f27440039f0 2024-05-28T14:11:12.117 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:12.117 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":101,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:11:11.374834+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:11:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.132+0000 7f2766a00640 1 -- 172.21.15.42:0/164717263 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f273c069880 msgr2=0x7f273c06bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.132+0000 7f2766a00640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f273c069880 0x7f273c06bd40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f276009b370 tx=0x7f2748008040 comp rx=0 tx=0).stop 2024-05-28T14:11:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.132+0000 7f2766a00640 1 -- 172.21.15.42:0/164717263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27440039f0 msgr2=0x7f276014b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.132+0000 7f2766a00640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27440039f0 0x7f276014b810 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f275000c1f0 tx=0x7f2744005840 comp rx=0 tx=0).stop 2024-05-28T14:11:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.132+0000 7f2766a00640 1 -- 172.21.15.42:0/164717263 shutdown_connections 2024-05-28T14:11:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.132+0000 7f2766a00640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f273c069880 0x7f273c06bd40 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.132+0000 7f2766a00640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276009ba00 0x7f276015bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.132+0000 7f2766a00640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f276009b580 0x7f276015b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.132+0000 7f2766a00640 1 --2- 172.21.15.42:0/164717263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27440039f0 0x7f276014b810 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.132+0000 7f2766a00640 1 -- 172.21.15.42:0/164717263 >> 172.21.15.42:0/164717263 conn(0x7f276006bdf0 msgr2=0x7f27601557d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:12.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.136+0000 7f2766a00640 1 -- 172.21.15.42:0/164717263 shutdown_connections 2024-05-28T14:11:12.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.136+0000 7f2766a00640 1 -- 172.21.15.42:0/164717263 wait complete. 2024-05-28T14:11:12.138 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:11:12.139 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:11:12.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7308600640 1 Processor -- start 2024-05-28T14:11:12.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7308600640 1 -- start start 2024-05-28T14:11:12.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7308600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f730009b160 0x7f730006cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7308600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f730009b8e0 0x7f730015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7308600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73000d0770 con 0x7f73000b0e80 2024-05-28T14:11:12.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7308600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f73000d0a10 con 0x7f730009b8e0 2024-05-28T14:11:12.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7308600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f73000d0cf0 con 0x7f730009b160 2024-05-28T14:11:12.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7305c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f73000b0e80 0x7f730009aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:44796/0 (socket says 172.21.15.42:44796) 2024-05-28T14:11:12.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7305c00640 1 -- 172.21.15.42:0/2482488984 learned_addr learned my addr 172.21.15.42:0/2482488984 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:12.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7306600640 1 --2- 172.21.15.42:0/2482488984 >> v2:172.21.15.42:3300/0 conn(0x7f730009b8e0 0x7f730015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7305200640 1 --2- 172.21.15.42:0/2482488984 >> v2:172.21.15.63:3300/0 conn(0x7f730009b160 0x7f730006cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f72ffe00640 1 -- 172.21.15.42:0/2482488984 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1044218687 0 0) 0x7f73000d0770 con 0x7f73000b0e80 2024-05-28T14:11:12.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f72ffe00640 1 -- 172.21.15.42:0/2482488984 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72e8003530 con 0x7f73000b0e80 2024-05-28T14:11:12.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7306600640 1 -- 172.21.15.42:0/2482488984 >> v2:172.21.15.63:3300/0 conn(0x7f730009b160 msgr2=0x7f730006cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:12.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7306600640 1 --2- 172.21.15.42:0/2482488984 >> v2:172.21.15.63:3300/0 conn(0x7f730009b160 0x7f730006cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7306600640 1 -- 172.21.15.42:0/2482488984 >> v1:172.21.15.42:6789/0 conn(0x7f73000b0e80 legacy=0x7f730009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.300+0000 7f7306600640 1 -- 172.21.15.42:0/2482488984 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73000d0fd0 con 0x7f730009b8e0 2024-05-28T14:11:12.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.304+0000 7f7306600640 1 --2- 172.21.15.42:0/2482488984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f730009b8e0 0x7f730015e110 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f7300073ec0 tx=0x7f72f801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c67b91469ea9c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:12.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.304+0000 7f72ffe00640 1 -- 172.21.15.42:0/2482488984 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f80b2070 con 0x7f730009b8e0 2024-05-28T14:11:12.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.304+0000 7f72ffe00640 1 -- 172.21.15.42:0/2482488984 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72f80a57f0 con 0x7f730009b8e0 2024-05-28T14:11:12.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.304+0000 7f72ffe00640 1 -- 172.21.15.42:0/2482488984 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f80a5bc0 con 0x7f730009b8e0 2024-05-28T14:11:12.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.304+0000 7f7308600640 1 -- 172.21.15.42:0/2482488984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f730009b8e0 msgr2=0x7f730015e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.304+0000 7f7308600640 1 --2- 172.21.15.42:0/2482488984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f730009b8e0 0x7f730015e110 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f7300073ec0 tx=0x7f72f801c930 comp rx=0 tx=0).stop 2024-05-28T14:11:12.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.304+0000 7f7308600640 1 -- 172.21.15.42:0/2482488984 shutdown_connections 2024-05-28T14:11:12.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.304+0000 7f7308600640 1 --2- 172.21.15.42:0/2482488984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f730009b8e0 0x7f730015e110 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.304+0000 7f7308600640 1 --2- 172.21.15.42:0/2482488984 >> v2:172.21.15.63:3300/0 conn(0x7f730009b160 0x7f730006cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.304+0000 7f7308600640 1 -- 172.21.15.42:0/2482488984 >> 172.21.15.42:0/2482488984 conn(0x7f73000b3f90 msgr2=0x7f73000b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:12.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.304+0000 7f7308600640 1 -- 172.21.15.42:0/2482488984 shutdown_connections 2024-05-28T14:11:12.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.304+0000 7f7308600640 1 -- 172.21.15.42:0/2482488984 wait complete. 2024-05-28T14:11:12.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7308600640 1 Processor -- start 2024-05-28T14:11:12.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7308600640 1 -- start start 2024-05-28T14:11:12.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7308600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f730009b160 0x7f730015b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7308600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f730009b8e0 0x7f730009c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7308600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73000b0e80 0x7f7300071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7308600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f73000af350 con 0x7f73000b0e80 2024-05-28T14:11:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7308600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f73000af1d0 con 0x7f730009b160 2024-05-28T14:11:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7308600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f73000af4d0 con 0x7f730009b8e0 2024-05-28T14:11:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7306600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73000b0e80 0x7f7300071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7306600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73000b0e80 0x7f7300071020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59618/0 (socket says 172.21.15.42:59618) 2024-05-28T14:11:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7306600640 1 -- 172.21.15.42:0/2292028103 learned_addr learned my addr 172.21.15.42:0/2292028103 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7305200640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f730009b8e0 0x7f730009c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7305c00640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f730009b160 0x7f730015b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7306600640 1 -- 172.21.15.42:0/2292028103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f730009b8e0 msgr2=0x7f730009c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7306600640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f730009b8e0 0x7f730009c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7306600640 1 -- 172.21.15.42:0/2292028103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f730009b160 msgr2=0x7f730015b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7306600640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f730009b160 0x7f730015b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7306600640 1 -- 172.21.15.42:0/2292028103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730015b6b0 con 0x7f73000b0e80 2024-05-28T14:11:12.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7305c00640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f730009b160 0x7f730015b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:12.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7306600640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73000b0e80 0x7f7300071020 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f72f8019540 tx=0x7f72f80196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:12.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f72fd600640 1 -- 172.21.15.42:0/2292028103 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f80b2050 con 0x7f73000b0e80 2024-05-28T14:11:12.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f72fd600640 1 -- 172.21.15.42:0/2292028103 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72f80be070 con 0x7f73000b0e80 2024-05-28T14:11:12.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7308600640 1 -- 172.21.15.42:0/2292028103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730015b950 con 0x7f73000b0e80 2024-05-28T14:11:12.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f72fd600640 1 -- 172.21.15.42:0/2292028103 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f80ad890 con 0x7f73000b0e80 2024-05-28T14:11:12.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7308600640 1 -- 172.21.15.42:0/2292028103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73000c71c0 con 0x7f73000b0e80 2024-05-28T14:11:12.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.308+0000 7f7308600640 1 -- 172.21.15.42:0/2292028103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73000a0550 con 0x7f73000b0e80 2024-05-28T14:11:12.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.312+0000 7f72fd600640 1 -- 172.21.15.42:0/2292028103 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f72f80ada30 con 0x7f73000b0e80 2024-05-28T14:11:12.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.312+0000 7f72fd600640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f72e0069850 0x7f72e006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.312+0000 7f72fd600640 1 -- 172.21.15.42:0/2292028103 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4885+0+0 (secure 0 0 0) 0x7f72f81206f0 con 0x7f73000b0e80 2024-05-28T14:11:12.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.312+0000 7f7305c00640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f72e0069850 0x7f72e006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.312+0000 7f7305c00640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f72e0069850 0x7f72e006bd10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f72f4005890 tx=0x7f72f4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:12.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.316+0000 7f72fd600640 1 -- 172.21.15.42:0/2292028103 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f72f8016520 con 0x7f73000b0e80 2024-05-28T14:11:12.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.480+0000 7f7308600640 1 -- 172.21.15.42:0/2292028103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73000b2300 con 0x7f73000b0e80 2024-05-28T14:11:12.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.480+0000 7f72fd600640 1 -- 172.21.15.42:0/2292028103 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f72f80f2820 con 0x7f73000b0e80 2024-05-28T14:11:12.466 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:11:12.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.484+0000 7f7308600640 1 -- 172.21.15.42:0/2292028103 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f72e0069850 msgr2=0x7f72e006bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.484+0000 7f7308600640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f72e0069850 0x7f72e006bd10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f72f4005890 tx=0x7f72f4009040 comp rx=0 tx=0).stop 2024-05-28T14:11:12.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.484+0000 7f7308600640 1 -- 172.21.15.42:0/2292028103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73000b0e80 msgr2=0x7f7300071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.484+0000 7f7308600640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73000b0e80 0x7f7300071020 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f72f8019540 tx=0x7f72f80196d0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.484+0000 7f7308600640 1 -- 172.21.15.42:0/2292028103 shutdown_connections 2024-05-28T14:11:12.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.484+0000 7f7308600640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f72e0069850 0x7f72e006bd10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.484+0000 7f7308600640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73000b0e80 0x7f7300071020 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.484+0000 7f7308600640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f730009b8e0 0x7f730009c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.484+0000 7f7308600640 1 --2- 172.21.15.42:0/2292028103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f730009b160 0x7f730015b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.484+0000 7f7308600640 1 -- 172.21.15.42:0/2292028103 >> 172.21.15.42:0/2292028103 conn(0x7f73000b3f90 msgr2=0x7f73000b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:12.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.484+0000 7f7308600640 1 -- 172.21.15.42:0/2292028103 shutdown_connections 2024-05-28T14:11:12.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.484+0000 7f7308600640 1 -- 172.21.15.42:0/2292028103 wait complete. 2024-05-28T14:11:12.488 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-28T14:11:12.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.644+0000 7faa72200640 1 Processor -- start 2024-05-28T14:11:12.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.644+0000 7faa72200640 1 -- start start 2024-05-28T14:11:12.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa72200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faa6c0ce860 0x7faa6c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa72200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faa6c0d0900 0x7faa6c162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa72200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa6c06dcf0 0x7faa6c162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa72200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7faa6c0d19f0 con 0x7faa6c0d0900 2024-05-28T14:11:12.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa72200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faa6c0d0de0 con 0x7faa6c0ce860 2024-05-28T14:11:12.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa72200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faa6c0d10c0 con 0x7faa6c06dcf0 2024-05-28T14:11:12.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa62a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faa6c0d0900 0x7faa6c162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa62a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faa6c0d0900 0x7faa6c162570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59626/0 (socket says 172.21.15.42:59626) 2024-05-28T14:11:12.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa62a00640 1 -- 172.21.15.42:0/4012154737 learned_addr learned my addr 172.21.15.42:0/4012154737 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:12.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa6b400640 1 --2- 172.21.15.42:0/4012154737 >> v2:172.21.15.63:3300/0 conn(0x7faa6c0ce860 0x7faa6c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa6be00640 1 --2- 172.21.15.42:0/4012154737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa6c06dcf0 0x7faa6c162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa62a00640 1 -- 172.21.15.42:0/4012154737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa6c06dcf0 msgr2=0x7faa6c162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:12.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa62a00640 1 --2- 172.21.15.42:0/4012154737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa6c06dcf0 0x7faa6c162ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa62a00640 1 -- 172.21.15.42:0/4012154737 >> v2:172.21.15.63:3300/0 conn(0x7faa6c0ce860 msgr2=0x7faa6c06d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:12.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa62a00640 1 --2- 172.21.15.42:0/4012154737 >> v2:172.21.15.63:3300/0 conn(0x7faa6c0ce860 0x7faa6c06d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa62a00640 1 -- 172.21.15.42:0/4012154737 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c0d13a0 con 0x7faa6c0d0900 2024-05-28T14:11:12.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa6b400640 1 --2- 172.21.15.42:0/4012154737 >> v2:172.21.15.63:3300/0 conn(0x7faa6c0ce860 0x7faa6c06d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:12.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.648+0000 7faa6be00640 1 --2- 172.21.15.42:0/4012154737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa6c06dcf0 0x7faa6c162ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:12.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa62a00640 1 --2- 172.21.15.42:0/4012154737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa6c0d0900 0x7faa6c162570 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7faa6c069710 tx=0x7faa5401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4419c74fe07297ee server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:12.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa6aa00640 1 -- 172.21.15.42:0/4012154737 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa540ab640 con 0x7faa6c0d0900 2024-05-28T14:11:12.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa6aa00640 1 -- 172.21.15.42:0/4012154737 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa540b0070 con 0x7faa6c0d0900 2024-05-28T14:11:12.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa6aa00640 1 -- 172.21.15.42:0/4012154737 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa540aba00 con 0x7faa6c0d0900 2024-05-28T14:11:12.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa72200640 1 -- 172.21.15.42:0/4012154737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa6c0d0900 msgr2=0x7faa6c162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa72200640 1 --2- 172.21.15.42:0/4012154737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa6c0d0900 0x7faa6c162570 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7faa6c069710 tx=0x7faa5401c460 comp rx=0 tx=0).stop 2024-05-28T14:11:12.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa72200640 1 -- 172.21.15.42:0/4012154737 shutdown_connections 2024-05-28T14:11:12.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa72200640 1 --2- 172.21.15.42:0/4012154737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa6c06dcf0 0x7faa6c162ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa72200640 1 --2- 172.21.15.42:0/4012154737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa6c0d0900 0x7faa6c162570 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa72200640 1 --2- 172.21.15.42:0/4012154737 >> v2:172.21.15.63:3300/0 conn(0x7faa6c0ce860 0x7faa6c06d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa72200640 1 -- 172.21.15.42:0/4012154737 >> 172.21.15.42:0/4012154737 conn(0x7faa6c0b4080 msgr2=0x7faa6c0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:12.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa72200640 1 -- 172.21.15.42:0/4012154737 shutdown_connections 2024-05-28T14:11:12.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa72200640 1 -- 172.21.15.42:0/4012154737 wait complete. 2024-05-28T14:11:12.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa72200640 1 Processor -- start 2024-05-28T14:11:12.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.652+0000 7faa72200640 1 -- start start 2024-05-28T14:11:12.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa72200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa6c06dcf0 0x7faa6c0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa72200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa6c0ce860 0x7faa6c14f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa72200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa6c0d0900 0x7faa6c14fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa72200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c15ede0 con 0x7faa6c06dcf0 2024-05-28T14:11:12.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa72200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c15ec60 con 0x7faa6c0d0900 2024-05-28T14:11:12.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa72200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faa6c09aa50 con 0x7faa6c0ce860 2024-05-28T14:11:12.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa6b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa6c06dcf0 0x7faa6c0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa6b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa6c06dcf0 0x7faa6c0c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59630/0 (socket says 172.21.15.42:59630) 2024-05-28T14:11:12.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa6b400640 1 -- 172.21.15.42:0/366295029 learned_addr learned my addr 172.21.15.42:0/366295029 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:12.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa6be00640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa6c0d0900 0x7faa6c14fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa62a00640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa6c0ce860 0x7faa6c14f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa6b400640 1 -- 172.21.15.42:0/366295029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa6c0ce860 msgr2=0x7faa6c14f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa6b400640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa6c0ce860 0x7faa6c14f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa6b400640 1 -- 172.21.15.42:0/366295029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa6c0d0900 msgr2=0x7faa6c14fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa6b400640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa6c0d0900 0x7faa6c14fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa6b400640 1 -- 172.21.15.42:0/366295029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c0cc630 con 0x7faa6c06dcf0 2024-05-28T14:11:12.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa62a00640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa6c0ce860 0x7faa6c14f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:12.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa6be00640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa6c0d0900 0x7faa6c14fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:12.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa6b400640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa6c06dcf0 0x7faa6c0c7570 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00e840 tx=0x7faa5c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:12.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa63e00640 1 -- 172.21.15.42:0/366295029 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c017070 con 0x7faa6c06dcf0 2024-05-28T14:11:12.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa63e00640 1 -- 172.21.15.42:0/366295029 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa5c00c8d0 con 0x7faa6c06dcf0 2024-05-28T14:11:12.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa63e00640 1 -- 172.21.15.42:0/366295029 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c00f040 con 0x7faa6c06dcf0 2024-05-28T14:11:12.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa72200640 1 -- 172.21.15.42:0/366295029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c0c7ca0 con 0x7faa6c06dcf0 2024-05-28T14:11:12.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa72200640 1 -- 172.21.15.42:0/366295029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c0c8260 con 0x7faa6c06dcf0 2024-05-28T14:11:12.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa72200640 1 -- 172.21.15.42:0/366295029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6c068a90 con 0x7faa6c06dcf0 2024-05-28T14:11:12.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.656+0000 7faa63e00640 1 -- 172.21.15.42:0/366295029 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7faa5c01a070 con 0x7faa6c06dcf0 2024-05-28T14:11:12.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.660+0000 7faa63e00640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa48069850 0x7faa4806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.660+0000 7faa63e00640 1 -- 172.21.15.42:0/366295029 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4885+0+0 (secure 0 0 0) 0x7faa5c0852d0 con 0x7faa6c06dcf0 2024-05-28T14:11:12.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.660+0000 7faa62a00640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa48069850 0x7faa4806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.660+0000 7faa62a00640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa48069850 0x7faa4806bd10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7faa540af000 tx=0x7faa540a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:12.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.664+0000 7faa63e00640 1 -- 172.21.15.42:0/366295029 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faa5c0572e0 con 0x7faa6c06dcf0 2024-05-28T14:11:12.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.828+0000 7faa72200640 1 -- 172.21.15.42:0/366295029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7faa6c0b4ce0 con 0x7faa6c06dcf0 2024-05-28T14:11:12.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.828+0000 7faa63e00640 1 -- 172.21.15.42:0/366295029 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v101) ==== 84+0+65 (secure 0 0 0) 0x7faa5c056be0 con 0x7faa6c06dcf0 2024-05-28T14:11:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:12.814 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:11:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-05-28T14:11:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-05-28T14:11:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:12.814 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:11:12.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.832+0000 7faa72200640 1 -- 172.21.15.42:0/366295029 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa48069850 msgr2=0x7faa4806bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.832+0000 7faa72200640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa48069850 0x7faa4806bd10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7faa540af000 tx=0x7faa540a7000 comp rx=0 tx=0).stop 2024-05-28T14:11:12.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.832+0000 7faa72200640 1 -- 172.21.15.42:0/366295029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa6c06dcf0 msgr2=0x7faa6c0c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.832+0000 7faa72200640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa6c06dcf0 0x7faa6c0c7570 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00e840 tx=0x7faa5c00ed00 comp rx=0 tx=0).stop 2024-05-28T14:11:12.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.832+0000 7faa72200640 1 -- 172.21.15.42:0/366295029 shutdown_connections 2024-05-28T14:11:12.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.832+0000 7faa72200640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa48069850 0x7faa4806bd10 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.832+0000 7faa72200640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa6c0d0900 0x7faa6c14fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.832+0000 7faa72200640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa6c0ce860 0x7faa6c14f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.832+0000 7faa72200640 1 --2- 172.21.15.42:0/366295029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa6c06dcf0 0x7faa6c0c7570 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.832+0000 7faa72200640 1 -- 172.21.15.42:0/366295029 >> 172.21.15.42:0/366295029 conn(0x7faa6c0b4080 msgr2=0x7faa6c15da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:12.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.832+0000 7faa72200640 1 -- 172.21.15.42:0/366295029 shutdown_connections 2024-05-28T14:11:12.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.832+0000 7faa72200640 1 -- 172.21.15.42:0/366295029 wait complete. 2024-05-28T14:11:12.836 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-28T14:11:12.836 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-28T14:11:12.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.996+0000 7f93caa00640 1 Processor -- start 2024-05-28T14:11:12.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.996+0000 7f93caa00640 1 -- start start 2024-05-28T14:11:12.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.996+0000 7f93caa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f93c406e5f0 0x7f93c4162cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:12.996+0000 7f93caa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f93c40d0e00 0x7f93c415f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.000+0000 7f93caa00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93c4163200 con 0x7f93c415ed00 2024-05-28T14:11:12.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.000+0000 7f93caa00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f93c415f7d0 con 0x7f93c40d0e00 2024-05-28T14:11:12.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.000+0000 7f93caa00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f93c415fab0 con 0x7f93c406e5f0 2024-05-28T14:11:12.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.000+0000 7f93c8a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f93c40d0e00 0x7f93c415f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.000+0000 7f93c8a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f93c40d0e00 0x7f93c415f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59644/0 (socket says 172.21.15.42:59644) 2024-05-28T14:11:12.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.000+0000 7f93c8a00640 1 -- 172.21.15.42:0/37399152 learned_addr learned my addr 172.21.15.42:0/37399152 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:12.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.000+0000 7f93c8a00640 1 -- 172.21.15.42:0/37399152 >> v2:172.21.15.63:3300/0 conn(0x7f93c406e5f0 msgr2=0x7f93c4162cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:12.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.000+0000 7f93c3e00640 1 --2- 172.21.15.42:0/37399152 >> v2:172.21.15.63:3300/0 conn(0x7f93c406e5f0 0x7f93c4162cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.000+0000 7f93c8a00640 1 --2- 172.21.15.42:0/37399152 >> v2:172.21.15.63:3300/0 conn(0x7f93c406e5f0 0x7f93c4162cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.000+0000 7f93c8a00640 1 -- 172.21.15.42:0/37399152 >> v1:172.21.15.63:6789/0 conn(0x7f93c415ed00 legacy=0x7f93c40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:12.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.000+0000 7f93c8a00640 1 -- 172.21.15.42:0/37399152 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c41624c0 con 0x7f93c40d0e00 2024-05-28T14:11:12.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.000+0000 7f93c3e00640 1 --2- 172.21.15.42:0/37399152 >> v2:172.21.15.63:3300/0 conn(0x7f93c406e5f0 0x7f93c4162cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:12.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.000+0000 7f93c8a00640 1 --2- 172.21.15.42:0/37399152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93c40d0e00 0x7f93c415f160 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f93c40b44f0 tx=0x7f93bc01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd5d579c00e9c94 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:12.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.000+0000 7f93c2a00640 1 -- 172.21.15.42:0/37399152 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0a55e0 con 0x7f93c40d0e00 2024-05-28T14:11:12.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93c2a00640 1 -- 172.21.15.42:0/37399152 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93bc0a5780 con 0x7f93c40d0e00 2024-05-28T14:11:12.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93c2a00640 1 -- 172.21.15.42:0/37399152 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0a5b50 con 0x7f93c40d0e00 2024-05-28T14:11:12.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 -- 172.21.15.42:0/37399152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93c40d0e00 msgr2=0x7f93c415f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 --2- 172.21.15.42:0/37399152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93c40d0e00 0x7f93c415f160 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f93c40b44f0 tx=0x7f93bc01c920 comp rx=0 tx=0).stop 2024-05-28T14:11:12.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 -- 172.21.15.42:0/37399152 shutdown_connections 2024-05-28T14:11:12.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 --2- 172.21.15.42:0/37399152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93c40d0e00 0x7f93c415f160 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 --2- 172.21.15.42:0/37399152 >> v2:172.21.15.63:3300/0 conn(0x7f93c406e5f0 0x7f93c4162cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 -- 172.21.15.42:0/37399152 >> 172.21.15.42:0/37399152 conn(0x7f93c406bdf0 msgr2=0x7f93c406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:12.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 -- 172.21.15.42:0/37399152 shutdown_connections 2024-05-28T14:11:12.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 -- 172.21.15.42:0/37399152 wait complete. 2024-05-28T14:11:12.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 Processor -- start 2024-05-28T14:11:12.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 -- start start 2024-05-28T14:11:12.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93c406e5f0 0x7f93c41579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f93c40d0e00 0x7f93c40adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93c415ed00 0x7f93c4148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f93c40728c0 con 0x7f93c406e5f0 2024-05-28T14:11:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4072740 con 0x7f93c415ed00 2024-05-28T14:11:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93caa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f93c4072a40 con 0x7f93c40d0e00 2024-05-28T14:11:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93c406e5f0 0x7f93c41579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93c406e5f0 0x7f93c41579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59650/0 (socket says 172.21.15.42:59650) 2024-05-28T14:11:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93c3e00640 1 -- 172.21.15.42:0/491261840 learned_addr learned my addr 172.21.15.42:0/491261840 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93c3400640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f93c40d0e00 0x7f93c40adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93c8a00640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93c415ed00 0x7f93c4148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93c3e00640 1 -- 172.21.15.42:0/491261840 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f93c40d0e00 msgr2=0x7f93c40adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93c3e00640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f93c40d0e00 0x7f93c40adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93c3e00640 1 -- 172.21.15.42:0/491261840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93c415ed00 msgr2=0x7f93c4148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93c3e00640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93c415ed00 0x7f93c4148690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93c3e00640 1 -- 172.21.15.42:0/491261840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c4158030 con 0x7f93c406e5f0 2024-05-28T14:11:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.004+0000 7f93c8a00640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93c415ed00 0x7f93c4148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:12.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.008+0000 7f93c3e00640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93c406e5f0 0x7f93c41579e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f93b400c7f0 tx=0x7f93b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.008+0000 7f93bbe00640 1 -- 172.21.15.42:0/491261840 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4007a30 con 0x7f93c406e5f0 2024-05-28T14:11:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.008+0000 7f93bbe00640 1 -- 172.21.15.42:0/491261840 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93b4007bd0 con 0x7f93c406e5f0 2024-05-28T14:11:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.008+0000 7f93bbe00640 1 -- 172.21.15.42:0/491261840 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b40113f0 con 0x7f93c406e5f0 2024-05-28T14:11:12.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.008+0000 7f93caa00640 1 -- 172.21.15.42:0/491261840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c4158310 con 0x7f93c406e5f0 2024-05-28T14:11:12.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.008+0000 7f93caa00640 1 -- 172.21.15.42:0/491261840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c41588d0 con 0x7f93c406e5f0 2024-05-28T14:11:12.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.008+0000 7f93caa00640 1 -- 172.21.15.42:0/491261840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c4158aa0 con 0x7f93c406e5f0 2024-05-28T14:11:12.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.008+0000 7f93bbe00640 1 -- 172.21.15.42:0/491261840 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f93b4011590 con 0x7f93c406e5f0 2024-05-28T14:11:12.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.008+0000 7f93bbe00640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f93a4069850 0x7f93a406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:12.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.012+0000 7f93bbe00640 1 -- 172.21.15.42:0/491261840 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4885+0+0 (secure 0 0 0) 0x7f93b4085c30 con 0x7f93c406e5f0 2024-05-28T14:11:12.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.012+0000 7f93c3400640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f93a4069850 0x7f93a406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:12.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.012+0000 7f93c3400640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f93a4069850 0x7f93a406bd10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f93c40b3a10 tx=0x7f93b0009060 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.016+0000 7f93bbe00640 1 -- 172.21.15.42:0/491261840 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f93c4158aa0 con 0x7f93c406e5f0 2024-05-28T14:11:13.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.180+0000 7f93caa00640 1 -- 172.21.15.42:0/491261840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f93c4068a90 con 0x7f93c406e5f0 2024-05-28T14:11:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.372+0000 7f93bbe00640 1 -- 172.21.15.42:0/491261840 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v102) ==== 138+0+0 (secure 0 0 0) 0x7f93b4015020 con 0x7f93c406e5f0 2024-05-28T14:11:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-05-28T14:11:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.372+0000 7f93caa00640 1 -- 172.21.15.42:0/491261840 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f93a4069850 msgr2=0x7f93a406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.372+0000 7f93caa00640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f93a4069850 0x7f93a406bd10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f93c40b3a10 tx=0x7f93b0009060 comp rx=0 tx=0).stop 2024-05-28T14:11:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.372+0000 7f93caa00640 1 -- 172.21.15.42:0/491261840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93c406e5f0 msgr2=0x7f93c41579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.372+0000 7f93caa00640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93c406e5f0 0x7f93c41579e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f93b400c7f0 tx=0x7f93b400ccb0 comp rx=0 tx=0).stop 2024-05-28T14:11:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.372+0000 7f93caa00640 1 -- 172.21.15.42:0/491261840 shutdown_connections 2024-05-28T14:11:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.372+0000 7f93caa00640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f93a4069850 0x7f93a406bd10 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.372+0000 7f93caa00640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93c415ed00 0x7f93c4148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.372+0000 7f93caa00640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f93c40d0e00 0x7f93c40adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.372+0000 7f93caa00640 1 --2- 172.21.15.42:0/491261840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93c406e5f0 0x7f93c41579e0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.372+0000 7f93caa00640 1 -- 172.21.15.42:0/491261840 >> 172.21.15.42:0/491261840 conn(0x7f93c406bdf0 msgr2=0x7f93c40b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.372+0000 7f93caa00640 1 -- 172.21.15.42:0/491261840 shutdown_connections 2024-05-28T14:11:13.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.372+0000 7f93caa00640 1 -- 172.21.15.42:0/491261840 wait complete. 2024-05-28T14:11:13.383 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-28T14:11:13.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb302c00640 1 Processor -- start 2024-05-28T14:11:13.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb302c00640 1 -- start start 2024-05-28T14:11:13.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb302c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb2fc0b02e0 0x7fb2fc06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:13.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb302c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2fc15f710 0x7fb2fc06ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:13.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb302c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb2fc06cf40 0x7fb2fc154da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:13.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb302c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb2fc06d390 con 0x7fb2fc06cf40 2024-05-28T14:11:13.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb302c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb2fc15fb30 con 0x7fb2fc0b02e0 2024-05-28T14:11:13.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb302c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2fc06f190 con 0x7fb2fc15f710 2024-05-28T14:11:13.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb300c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb2fc06cf40 0x7fb2fc154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:13.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb300c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb2fc06cf40 0x7fb2fc154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59666/0 (socket says 172.21.15.42:59666) 2024-05-28T14:11:13.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb300c00640 1 -- 172.21.15.42:0/712316510 learned_addr learned my addr 172.21.15.42:0/712316510 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:13.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb2f3400640 1 --2- 172.21.15.42:0/712316510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2fc15f710 0x7fb2fc06ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:13.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb2fbe00640 1 --2- 172.21.15.42:0/712316510 >> v2:172.21.15.63:3300/0 conn(0x7fb2fc0b02e0 0x7fb2fc06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:13.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb300c00640 1 -- 172.21.15.42:0/712316510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2fc15f710 msgr2=0x7fb2fc06ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:13.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb300c00640 1 --2- 172.21.15.42:0/712316510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2fc15f710 0x7fb2fc06ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:13.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb300c00640 1 -- 172.21.15.42:0/712316510 >> v2:172.21.15.63:3300/0 conn(0x7fb2fc0b02e0 msgr2=0x7fb2fc06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:13.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb300c00640 1 --2- 172.21.15.42:0/712316510 >> v2:172.21.15.63:3300/0 conn(0x7fb2fc0b02e0 0x7fb2fc06eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:13.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.548+0000 7fb300c00640 1 -- 172.21.15.42:0/712316510 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2fc06f470 con 0x7fb2fc06cf40 2024-05-28T14:11:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.552+0000 7fb300c00640 1 --2- 172.21.15.42:0/712316510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc06cf40 0x7fb2fc154da0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc072220 tx=0x7fb2f401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c13e2162a72f796 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.552+0000 7fb2fb400640 1 -- 172.21.15.42:0/712316510 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f40aa610 con 0x7fb2fc06cf40 2024-05-28T14:11:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.552+0000 7fb2fb400640 1 -- 172.21.15.42:0/712316510 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2f40af070 con 0x7fb2fc06cf40 2024-05-28T14:11:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.552+0000 7fb2fb400640 1 -- 172.21.15.42:0/712316510 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f40aa9d0 con 0x7fb2fc06cf40 2024-05-28T14:11:13.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.552+0000 7fb302c00640 1 -- 172.21.15.42:0/712316510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc06cf40 msgr2=0x7fb2fc154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:13.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.552+0000 7fb302c00640 1 --2- 172.21.15.42:0/712316510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc06cf40 0x7fb2fc154da0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc072220 tx=0x7fb2f401c4f0 comp rx=0 tx=0).stop 2024-05-28T14:11:13.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.552+0000 7fb302c00640 1 -- 172.21.15.42:0/712316510 shutdown_connections 2024-05-28T14:11:13.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.552+0000 7fb302c00640 1 --2- 172.21.15.42:0/712316510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc06cf40 0x7fb2fc154da0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:13.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.552+0000 7fb302c00640 1 --2- 172.21.15.42:0/712316510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2fc15f710 0x7fb2fc06ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:13.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.552+0000 7fb302c00640 1 --2- 172.21.15.42:0/712316510 >> v2:172.21.15.63:3300/0 conn(0x7fb2fc0b02e0 0x7fb2fc06eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:13.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.552+0000 7fb302c00640 1 -- 172.21.15.42:0/712316510 >> 172.21.15.42:0/712316510 conn(0x7fb2fc072600 msgr2=0x7fb2fc072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:13.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.552+0000 7fb302c00640 1 -- 172.21.15.42:0/712316510 shutdown_connections 2024-05-28T14:11:13.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.552+0000 7fb302c00640 1 -- 172.21.15.42:0/712316510 wait complete. 2024-05-28T14:11:13.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb302c00640 1 Processor -- start 2024-05-28T14:11:13.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb302c00640 1 -- start start 2024-05-28T14:11:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb302c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2fc06cf40 0x7fb2fc148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb302c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc0b02e0 0x7fb2fc0b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb302c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2fc15f710 0x7fb2fc150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb302c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc162400 con 0x7fb2fc0b02e0 2024-05-28T14:11:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb302c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc162280 con 0x7fb2fc15f710 2024-05-28T14:11:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb302c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2fc162580 con 0x7fb2fc06cf40 2024-05-28T14:11:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb2f3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc0b02e0 0x7fb2fc0b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb2f3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc0b02e0 0x7fb2fc0b1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59678/0 (socket says 172.21.15.42:59678) 2024-05-28T14:11:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb2f3400640 1 -- 172.21.15.42:0/3290333575 learned_addr learned my addr 172.21.15.42:0/3290333575 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb2fbe00640 1 --2- 172.21.15.42:0/3290333575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2fc06cf40 0x7fb2fc148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb300c00640 1 --2- 172.21.15.42:0/3290333575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2fc15f710 0x7fb2fc150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb2f3400640 1 -- 172.21.15.42:0/3290333575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2fc06cf40 msgr2=0x7fb2fc148860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb2f3400640 1 --2- 172.21.15.42:0/3290333575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2fc06cf40 0x7fb2fc148860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb2f3400640 1 -- 172.21.15.42:0/3290333575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2fc15f710 msgr2=0x7fb2fc150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb2f3400640 1 --2- 172.21.15.42:0/3290333575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2fc15f710 0x7fb2fc150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb2f3400640 1 -- 172.21.15.42:0/3290333575 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2fc14e100 con 0x7fb2fc0b02e0 2024-05-28T14:11:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb2f3400640 1 --2- 172.21.15.42:0/3290333575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc0b02e0 0x7fb2fc0b1000 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4007e60 tx=0x7fb2e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb2f8c00640 1 -- 172.21.15.42:0/3290333575 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4014620 con 0x7fb2fc0b02e0 2024-05-28T14:11:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb2f8c00640 1 -- 172.21.15.42:0/3290333575 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e4019070 con 0x7fb2fc0b02e0 2024-05-28T14:11:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb2f8c00640 1 -- 172.21.15.42:0/3290333575 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4014a20 con 0x7fb2fc0b02e0 2024-05-28T14:11:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb302c00640 1 -- 172.21.15.42:0/3290333575 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2fc14e3e0 con 0x7fb2fc0b02e0 2024-05-28T14:11:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb302c00640 1 -- 172.21.15.42:0/3290333575 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2fc153080 con 0x7fb2fc0b02e0 2024-05-28T14:11:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.556+0000 7fb302c00640 1 -- 172.21.15.42:0/3290333575 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2fc073f40 con 0x7fb2fc0b02e0 2024-05-28T14:11:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.560+0000 7fb2f8c00640 1 -- 172.21.15.42:0/3290333575 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fb2e4014bc0 con 0x7fb2fc0b02e0 2024-05-28T14:11:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.560+0000 7fb2f8c00640 1 --2- 172.21.15.42:0/3290333575 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb2d8069880 0x7fb2d806bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.560+0000 7fb2f8c00640 1 -- 172.21.15.42:0/3290333575 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5255+0+0 (secure 0 0 0) 0x7fb2e4086d50 con 0x7fb2fc0b02e0 2024-05-28T14:11:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.560+0000 7fb2fbe00640 1 --2- 172.21.15.42:0/3290333575 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb2d8069880 0x7fb2d806bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.560+0000 7fb2fbe00640 1 --2- 172.21.15.42:0/3290333575 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb2d8069880 0x7fb2d806bd40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0045f0 tx=0x7fb2ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.564+0000 7fb2f8c00640 1 -- 172.21.15.42:0/3290333575 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb2e4058710 con 0x7fb2fc0b02e0 2024-05-28T14:11:13.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:13.728+0000 7fb302c00640 1 -- 172.21.15.42:0/3290333575 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb2fc0d0900 con 0x7fb2fc0b02e0 2024-05-28T14:11:14.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.380+0000 7fb2f8c00640 1 -- 172.21.15.42:0/3290333575 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v103) ==== 170+0+0 (secure 0 0 0) 0x7fb2e405ca10 con 0x7fb2fc0b02e0 2024-05-28T14:11:14.390 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-05-28T14:11:14.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.384+0000 7fb302c00640 1 -- 172.21.15.42:0/3290333575 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb2d8069880 msgr2=0x7fb2d806bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:14.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.384+0000 7fb302c00640 1 --2- 172.21.15.42:0/3290333575 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb2d8069880 0x7fb2d806bd40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0045f0 tx=0x7fb2ec008040 comp rx=0 tx=0).stop 2024-05-28T14:11:14.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.384+0000 7fb302c00640 1 -- 172.21.15.42:0/3290333575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc0b02e0 msgr2=0x7fb2fc0b1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:14.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.384+0000 7fb302c00640 1 --2- 172.21.15.42:0/3290333575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc0b02e0 0x7fb2fc0b1000 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4007e60 tx=0x7fb2e400e4b0 comp rx=0 tx=0).stop 2024-05-28T14:11:14.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.384+0000 7fb302c00640 1 -- 172.21.15.42:0/3290333575 shutdown_connections 2024-05-28T14:11:14.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.384+0000 7fb302c00640 1 --2- 172.21.15.42:0/3290333575 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb2d8069880 0x7fb2d806bd40 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:14.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.384+0000 7fb302c00640 1 --2- 172.21.15.42:0/3290333575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2fc15f710 0x7fb2fc150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:14.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.384+0000 7fb302c00640 1 --2- 172.21.15.42:0/3290333575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc0b02e0 0x7fb2fc0b1000 secure :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4007e60 tx=0x7fb2e400e4b0 comp rx=0 tx=0).stop 2024-05-28T14:11:14.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.384+0000 7fb302c00640 1 --2- 172.21.15.42:0/3290333575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2fc06cf40 0x7fb2fc148860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:14.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.384+0000 7fb302c00640 1 -- 172.21.15.42:0/3290333575 >> 172.21.15.42:0/3290333575 conn(0x7fb2fc072600 msgr2=0x7fb2fc161170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:14.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.388+0000 7fb302c00640 1 -- 172.21.15.42:0/3290333575 shutdown_connections 2024-05-28T14:11:14.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.388+0000 7fb302c00640 1 -- 172.21.15.42:0/3290333575 wait complete. 2024-05-28T14:11:14.409 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-28T14:11:14.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c75e00640 1 Processor -- start 2024-05-28T14:11:14.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c75e00640 1 -- start start 2024-05-28T14:11:14.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c75e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c700cc660 0x7f3c7015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:14.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c75e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c700ce090 con 0x7f3c701415e0 2024-05-28T14:11:14.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c75e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c7015bff0 con 0x7f3c700d0320 2024-05-28T14:11:14.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c75e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3c7006e240 con 0x7f3c700cc660 2024-05-28T14:11:14.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c6fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f3c701415e0 0x7f3c7006db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:44798/0 (socket says 172.21.15.42:44798) 2024-05-28T14:11:14.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c6fe00640 1 -- 172.21.15.42:0/63914448 learned_addr learned my addr 172.21.15.42:0/63914448 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:14.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c74a00640 1 --2- 172.21.15.42:0/63914448 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c700cc660 0x7f3c7015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:14.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c6f400640 1 -- 172.21.15.42:0/63914448 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2485010577 0 0) 0x7f3c700ce090 con 0x7f3c701415e0 2024-05-28T14:11:14.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c6f400640 1 -- 172.21.15.42:0/63914448 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c58003530 con 0x7f3c701415e0 2024-05-28T14:11:14.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c6f400640 1 -- 172.21.15.42:0/63914448 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2949633236 0 0) 0x7f3c7015bff0 con 0x7f3c700d0320 2024-05-28T14:11:14.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c6f400640 1 -- 172.21.15.42:0/63914448 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c700ce090 con 0x7f3c700d0320 2024-05-28T14:11:14.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c74a00640 1 -- 172.21.15.42:0/63914448 >> v1:172.21.15.63:6789/0 conn(0x7f3c700d0320 legacy=0x7f3c700ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:14.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c74a00640 1 -- 172.21.15.42:0/63914448 >> v1:172.21.15.42:6789/0 conn(0x7f3c701415e0 legacy=0x7f3c7006db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:14.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.572+0000 7f3c74a00640 1 -- 172.21.15.42:0/63914448 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7015c2d0 con 0x7f3c700cc660 2024-05-28T14:11:14.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.576+0000 7f3c74a00640 1 --2- 172.21.15.42:0/63914448 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c700cc660 0x7f3c7015b9a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3c640097f0 tx=0x7f3c6401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b16ee07e86f06342 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:14.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.576+0000 7f3c6f400640 1 -- 172.21.15.42:0/63914448 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c640ab610 con 0x7f3c700cc660 2024-05-28T14:11:14.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.576+0000 7f3c6f400640 1 -- 172.21.15.42:0/63914448 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c640b0070 con 0x7f3c700cc660 2024-05-28T14:11:14.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.576+0000 7f3c6f400640 1 -- 172.21.15.42:0/63914448 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c640ab9d0 con 0x7f3c700cc660 2024-05-28T14:11:14.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.576+0000 7f3c75e00640 1 -- 172.21.15.42:0/63914448 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c700cc660 msgr2=0x7f3c7015b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:14.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.576+0000 7f3c75e00640 1 --2- 172.21.15.42:0/63914448 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c700cc660 0x7f3c7015b9a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3c640097f0 tx=0x7f3c6401c4b0 comp rx=0 tx=0).stop 2024-05-28T14:11:14.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.576+0000 7f3c75e00640 1 -- 172.21.15.42:0/63914448 shutdown_connections 2024-05-28T14:11:14.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.576+0000 7f3c75e00640 1 --2- 172.21.15.42:0/63914448 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c700cc660 0x7f3c7015b9a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:14.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.576+0000 7f3c75e00640 1 -- 172.21.15.42:0/63914448 >> 172.21.15.42:0/63914448 conn(0x7f3c7006bbc0 msgr2=0x7f3c7006bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:14.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.576+0000 7f3c75e00640 1 -- 172.21.15.42:0/63914448 shutdown_connections 2024-05-28T14:11:14.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.576+0000 7f3c75e00640 1 -- 172.21.15.42:0/63914448 wait complete. 2024-05-28T14:11:14.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c75e00640 1 Processor -- start 2024-05-28T14:11:14.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c75e00640 1 -- start start 2024-05-28T14:11:14.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c75e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c700cc660 0x7f3c7015f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:14.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c75e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c700d0320 0x7f3c700cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:14.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c75e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c701415e0 0x7f3c70169350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:14.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c75e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70141b60 con 0x7f3c700d0320 2024-05-28T14:11:14.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c75e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3c701419e0 con 0x7f3c701415e0 2024-05-28T14:11:14.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c75e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3c700cd250 con 0x7f3c700cc660 2024-05-28T14:11:14.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c700d0320 0x7f3c700cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:14.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c700d0320 0x7f3c700cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59688/0 (socket says 172.21.15.42:59688) 2024-05-28T14:11:14.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c6fe00640 1 -- 172.21.15.42:0/1160340744 learned_addr learned my addr 172.21.15.42:0/1160340744 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:14.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c74a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c700cc660 0x7f3c7015f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:14.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c75400640 1 --2- 172.21.15.42:0/1160340744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c701415e0 0x7f3c70169350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:14.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c6fe00640 1 -- 172.21.15.42:0/1160340744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c700cc660 msgr2=0x7f3c7015f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:14.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c6fe00640 1 --2- 172.21.15.42:0/1160340744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c700cc660 0x7f3c7015f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:14.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c6fe00640 1 -- 172.21.15.42:0/1160340744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c701415e0 msgr2=0x7f3c70169350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:14.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c6fe00640 1 --2- 172.21.15.42:0/1160340744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c701415e0 0x7f3c70169350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:14.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c6fe00640 1 -- 172.21.15.42:0/1160340744 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c700ca740 con 0x7f3c700d0320 2024-05-28T14:11:14.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c6fe00640 1 --2- 172.21.15.42:0/1160340744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c700d0320 0x7f3c700cb3e0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c420 tx=0x7f3c5c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:14.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c6cc00640 1 -- 172.21.15.42:0/1160340744 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c017070 con 0x7f3c700d0320 2024-05-28T14:11:14.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c6cc00640 1 -- 172.21.15.42:0/1160340744 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c5c005cf0 con 0x7f3c700d0320 2024-05-28T14:11:14.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c6cc00640 1 -- 172.21.15.42:0/1160340744 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c007310 con 0x7f3c700d0320 2024-05-28T14:11:14.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c75e00640 1 -- 172.21.15.42:0/1160340744 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c70072e80 con 0x7f3c700d0320 2024-05-28T14:11:14.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.580+0000 7f3c75e00640 1 -- 172.21.15.42:0/1160340744 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c70073440 con 0x7f3c700d0320 2024-05-28T14:11:14.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.584+0000 7f3c75e00640 1 -- 172.21.15.42:0/1160340744 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c7004fcf0 con 0x7f3c700d0320 2024-05-28T14:11:14.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.584+0000 7f3c6cc00640 1 -- 172.21.15.42:0/1160340744 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f3c5c01a070 con 0x7f3c700d0320 2024-05-28T14:11:14.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.584+0000 7f3c6cc00640 1 --2- 172.21.15.42:0/1160340744 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3c50069850 0x7f3c5006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:14.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.584+0000 7f3c6cc00640 1 -- 172.21.15.42:0/1160340744 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5637+0+0 (secure 0 0 0) 0x7f3c5c084ec0 con 0x7f3c700d0320 2024-05-28T14:11:14.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.584+0000 7f3c74a00640 1 --2- 172.21.15.42:0/1160340744 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3c50069850 0x7f3c5006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:14.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.584+0000 7f3c74a00640 1 --2- 172.21.15.42:0/1160340744 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3c50069850 0x7f3c5006bd10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3c640af000 tx=0x7f3c640a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:14.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.588+0000 7f3c6cc00640 1 -- 172.21.15.42:0/1160340744 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3c5c021020 con 0x7f3c700d0320 2024-05-28T14:11:14.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:14.788+0000 7f3c75e00640 1 -- 172.21.15.42:0/1160340744 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3c7006dd80 con 0x7f3c700d0320 2024-05-28T14:11:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.392+0000 7f3c6cc00640 1 -- 172.21.15.42:0/1160340744 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v60) ==== 174+0+0 (secure 0 0 0) 0x7f3c5c056880 con 0x7f3c700d0320 2024-05-28T14:11:15.382 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 14 and data pool 15 2024-05-28T14:11:15.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.408+0000 7f3c75e00640 1 -- 172.21.15.42:0/1160340744 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3c50069850 msgr2=0x7f3c5006bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:15.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.408+0000 7f3c75e00640 1 --2- 172.21.15.42:0/1160340744 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3c50069850 0x7f3c5006bd10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3c640af000 tx=0x7f3c640a7000 comp rx=0 tx=0).stop 2024-05-28T14:11:15.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.408+0000 7f3c75e00640 1 -- 172.21.15.42:0/1160340744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c700d0320 msgr2=0x7f3c700cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:15.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.408+0000 7f3c75e00640 1 --2- 172.21.15.42:0/1160340744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c700d0320 0x7f3c700cb3e0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c420 tx=0x7f3c5c00c8e0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.412+0000 7f3c75e00640 1 -- 172.21.15.42:0/1160340744 shutdown_connections 2024-05-28T14:11:15.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.412+0000 7f3c75e00640 1 --2- 172.21.15.42:0/1160340744 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3c50069850 0x7f3c5006bd10 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.412+0000 7f3c75e00640 1 --2- 172.21.15.42:0/1160340744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c701415e0 0x7f3c70169350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.412+0000 7f3c75e00640 1 --2- 172.21.15.42:0/1160340744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c700d0320 0x7f3c700cb3e0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.412+0000 7f3c75e00640 1 --2- 172.21.15.42:0/1160340744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c700cc660 0x7f3c7015f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.412+0000 7f3c75e00640 1 -- 172.21.15.42:0/1160340744 >> 172.21.15.42:0/1160340744 conn(0x7f3c7006bbc0 msgr2=0x7f3c7015a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:15.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.412+0000 7f3c75e00640 1 -- 172.21.15.42:0/1160340744 shutdown_connections 2024-05-28T14:11:15.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.412+0000 7f3c75e00640 1 -- 172.21.15.42:0/1160340744 wait complete. 2024-05-28T14:11:15.427 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:11:15.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.584+0000 7f2712400640 1 Processor -- start 2024-05-28T14:11:15.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.588+0000 7f2712400640 1 -- start start 2024-05-28T14:11:15.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.588+0000 7f2712400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f270c161d80 0x7f270c0b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:15.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.588+0000 7f2712400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f270c0cf390 0x7f270c0d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:15.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.588+0000 7f2712400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f270c1631f0 con 0x7f270c0d0240 2024-05-28T14:11:15.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.588+0000 7f2712400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f270c162f10 con 0x7f270c161d80 2024-05-28T14:11:15.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.588+0000 7f2712400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f270c0b3770 con 0x7f270c0cf390 2024-05-28T14:11:15.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.588+0000 7f2711000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f270c161d80 0x7f270c0b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:15.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.588+0000 7f2711000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f270c161d80 0x7f270c0b4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59694/0 (socket says 172.21.15.42:59694) 2024-05-28T14:11:15.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.588+0000 7f2711000640 1 -- 172.21.15.42:0/913845196 learned_addr learned my addr 172.21.15.42:0/913845196 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:15.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.588+0000 7f2711a00640 1 --2- 172.21.15.42:0/913845196 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f270c0cf390 0x7f270c0d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:15.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.588+0000 7f2711000640 1 -- 172.21.15.42:0/913845196 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f270c0cf390 msgr2=0x7f270c0d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:15.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.588+0000 7f2711000640 1 --2- 172.21.15.42:0/913845196 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f270c0cf390 0x7f270c0d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.588+0000 7f2711000640 1 -- 172.21.15.42:0/913845196 >> v1:172.21.15.63:6789/0 conn(0x7f270c0d0240 legacy=0x7f270c0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:15.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.588+0000 7f2711000640 1 -- 172.21.15.42:0/913845196 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270c0b3a50 con 0x7f270c161d80 2024-05-28T14:11:15.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.592+0000 7f2711000640 1 --2- 172.21.15.42:0/913845196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f270c161d80 0x7f270c0b4a50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f2700009870 tx=0x7f270001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0015e2073924690 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:15.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.592+0000 7f270b400640 1 -- 172.21.15.42:0/913845196 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27000a7070 con 0x7f270c161d80 2024-05-28T14:11:15.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.592+0000 7f270b400640 1 -- 172.21.15.42:0/913845196 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27000b0070 con 0x7f270c161d80 2024-05-28T14:11:15.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.592+0000 7f270b400640 1 -- 172.21.15.42:0/913845196 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27000ab890 con 0x7f270c161d80 2024-05-28T14:11:15.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.592+0000 7f2712400640 1 -- 172.21.15.42:0/913845196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f270c161d80 msgr2=0x7f270c0b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:15.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.592+0000 7f2712400640 1 --2- 172.21.15.42:0/913845196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f270c161d80 0x7f270c0b4a50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f2700009870 tx=0x7f270001c560 comp rx=0 tx=0).stop 2024-05-28T14:11:15.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.592+0000 7f2712400640 1 -- 172.21.15.42:0/913845196 shutdown_connections 2024-05-28T14:11:15.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.592+0000 7f2712400640 1 --2- 172.21.15.42:0/913845196 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f270c0cf390 0x7f270c0d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.592+0000 7f2712400640 1 --2- 172.21.15.42:0/913845196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f270c161d80 0x7f270c0b4a50 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.592+0000 7f2712400640 1 -- 172.21.15.42:0/913845196 >> 172.21.15.42:0/913845196 conn(0x7f270c0b7a30 msgr2=0x7f270c0b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:15.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.592+0000 7f2712400640 1 -- 172.21.15.42:0/913845196 shutdown_connections 2024-05-28T14:11:15.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.592+0000 7f2712400640 1 -- 172.21.15.42:0/913845196 wait complete. 2024-05-28T14:11:15.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.592+0000 7f2712400640 1 Processor -- start 2024-05-28T14:11:15.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.592+0000 7f2712400640 1 -- start start 2024-05-28T14:11:15.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2712400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f270c0cf390 0x7f270c151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:15.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2712400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f270c0d0240 0x7f270c15c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:15.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2712400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f270c161d80 0x7f270c15c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:15.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2712400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f270c09b5f0 con 0x7f270c161d80 2024-05-28T14:11:15.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2712400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f270c09b470 con 0x7f270c0cf390 2024-05-28T14:11:15.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2712400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f270c16b690 con 0x7f270c0d0240 2024-05-28T14:11:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2711a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f270c161d80 0x7f270c15c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2711a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f270c161d80 0x7f270c15c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59704/0 (socket says 172.21.15.42:59704) 2024-05-28T14:11:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2711a00640 1 -- 172.21.15.42:0/534192245 learned_addr learned my addr 172.21.15.42:0/534192245 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2711000640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f270c0cf390 0x7f270c151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f270be00640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f270c0d0240 0x7f270c15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2711a00640 1 -- 172.21.15.42:0/534192245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f270c0d0240 msgr2=0x7f270c15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2711a00640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f270c0d0240 0x7f270c15c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2711a00640 1 -- 172.21.15.42:0/534192245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f270c0cf390 msgr2=0x7f270c151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2711a00640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f270c0cf390 0x7f270c151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2711a00640 1 -- 172.21.15.42:0/534192245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270c0b0b70 con 0x7f270c161d80 2024-05-28T14:11:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f270be00640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f270c0d0240 0x7f270c15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2711a00640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f270c161d80 0x7f270c15c8d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f270400efc0 tx=0x7f270400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2708c00640 1 -- 172.21.15.42:0/534192245 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704018020 con 0x7f270c161d80 2024-05-28T14:11:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2708c00640 1 -- 172.21.15.42:0/534192245 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2704014480 con 0x7f270c161d80 2024-05-28T14:11:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2712400640 1 -- 172.21.15.42:0/534192245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270c0b0e50 con 0x7f270c161d80 2024-05-28T14:11:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2708c00640 1 -- 172.21.15.42:0/534192245 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704009e60 con 0x7f270c161d80 2024-05-28T14:11:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2712400640 1 -- 172.21.15.42:0/534192245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f270c1522f0 con 0x7f270c161d80 2024-05-28T14:11:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.596+0000 7f2712400640 1 -- 172.21.15.42:0/534192245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f270c04fcf0 con 0x7f270c161d80 2024-05-28T14:11:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.600+0000 7f2708c00640 1 -- 172.21.15.42:0/534192245 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f270401b020 con 0x7f270c161d80 2024-05-28T14:11:15.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.600+0000 7f2708c00640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f26ec069850 0x7f26ec06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:15.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.600+0000 7f2708c00640 1 -- 172.21.15.42:0/534192245 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5737+0+0 (secure 0 0 0) 0x7f2704086370 con 0x7f270c161d80 2024-05-28T14:11:15.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.600+0000 7f2711000640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f26ec069850 0x7f26ec06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:15.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.600+0000 7f2711000640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f26ec069850 0x7f26ec06bd10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f27000af000 tx=0x7f27000a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:15.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.604+0000 7f2708c00640 1 -- 172.21.15.42:0/534192245 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2704022030 con 0x7f270c161d80 2024-05-28T14:11:15.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.772+0000 7f2712400640 1 -- 172.21.15.42:0/534192245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f270c0cfb30 con 0x7f270c161d80 2024-05-28T14:11:15.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.772+0000 7f2708c00640 1 -- 172.21.15.42:0/534192245 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16615 (secure 0 0 0) 0x7f2704010040 con 0x7f270c161d80 2024-05-28T14:11:15.758 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:15.758 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":104,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:11:14.793590+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:11:13.194240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-28T14:11:13.743150+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:11:15.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.776+0000 7f2712400640 1 -- 172.21.15.42:0/534192245 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f26ec069850 msgr2=0x7f26ec06bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:15.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.776+0000 7f2712400640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f26ec069850 0x7f26ec06bd10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f27000af000 tx=0x7f27000a7000 comp rx=0 tx=0).stop 2024-05-28T14:11:15.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.776+0000 7f2712400640 1 -- 172.21.15.42:0/534192245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f270c161d80 msgr2=0x7f270c15c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:15.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.776+0000 7f2712400640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f270c161d80 0x7f270c15c8d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f270400efc0 tx=0x7f270400c490 comp rx=0 tx=0).stop 2024-05-28T14:11:15.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.776+0000 7f2712400640 1 -- 172.21.15.42:0/534192245 shutdown_connections 2024-05-28T14:11:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.776+0000 7f2712400640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f26ec069850 0x7f26ec06bd10 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.776+0000 7f2712400640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f270c161d80 0x7f270c15c8d0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.776+0000 7f2712400640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f270c0d0240 0x7f270c15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.776+0000 7f2712400640 1 --2- 172.21.15.42:0/534192245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f270c0cf390 0x7f270c151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.776+0000 7f2712400640 1 -- 172.21.15.42:0/534192245 >> 172.21.15.42:0/534192245 conn(0x7f270c0b7a30 msgr2=0x7f270c0b6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.776+0000 7f2712400640 1 -- 172.21.15.42:0/534192245 shutdown_connections 2024-05-28T14:11:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.776+0000 7f2712400640 1 -- 172.21.15.42:0/534192245 wait complete. 2024-05-28T14:11:15.780 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:11:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.936+0000 7f00f6a00640 1 Processor -- start 2024-05-28T14:11:15.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.936+0000 7f00f6a00640 1 -- start start 2024-05-28T14:11:15.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.940+0000 7f00f6a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f00f00b42e0 0x7f00f015ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.940+0000 7f00f6a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f00f0067810 0x7f00f006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.940+0000 7f00f6a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f006eaa0 con 0x7f00f00d0420 2024-05-28T14:11:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.940+0000 7f00f6a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f00f006ed80 con 0x7f00f00b42e0 2024-05-28T14:11:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.940+0000 7f00f6a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f00f006f060 con 0x7f00f0067810 2024-05-28T14:11:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.940+0000 7f00efe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f00f00d0420 0x7f00f00b3bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:44812/0 (socket says 172.21.15.42:44812) 2024-05-28T14:11:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.940+0000 7f00efe00640 1 -- 172.21.15.42:0/2115849366 learned_addr learned my addr 172.21.15.42:0/2115849366 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.940+0000 7f00e7400640 1 --2- 172.21.15.42:0/2115849366 >> v2:172.21.15.42:3300/0 conn(0x7f00f00b42e0 0x7f00f015ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:15.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.940+0000 7f00f4a00640 1 --2- 172.21.15.42:0/2115849366 >> v2:172.21.15.63:3300/0 conn(0x7f00f0067810 0x7f00f006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:15.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.940+0000 7f00e7400640 1 -- 172.21.15.42:0/2115849366 >> v2:172.21.15.63:3300/0 conn(0x7f00f0067810 msgr2=0x7f00f006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:15.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.940+0000 7f00e7400640 1 --2- 172.21.15.42:0/2115849366 >> v2:172.21.15.63:3300/0 conn(0x7f00f0067810 0x7f00f006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.940+0000 7f00e7400640 1 -- 172.21.15.42:0/2115849366 >> v1:172.21.15.42:6789/0 conn(0x7f00f00d0420 legacy=0x7f00f00b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:15.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.940+0000 7f00ef400640 1 -- 172.21.15.42:0/2115849366 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 307013811 0 0) 0x7f00f006eaa0 con 0x7f00f00d0420 2024-05-28T14:11:15.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.940+0000 7f00e7400640 1 -- 172.21.15.42:0/2115849366 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f006f340 con 0x7f00f00b42e0 2024-05-28T14:11:15.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.944+0000 7f00e7400640 1 --2- 172.21.15.42:0/2115849366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f00b42e0 0x7f00f015ed10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f00f00b5570 tx=0x7f00d801ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9864829b7c1da2b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:15.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.944+0000 7f00ef400640 1 -- 172.21.15.42:0/2115849366 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d80b0070 con 0x7f00f00b42e0 2024-05-28T14:11:15.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.944+0000 7f00ef400640 1 -- 172.21.15.42:0/2115849366 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d80ab600 con 0x7f00f00b42e0 2024-05-28T14:11:15.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.944+0000 7f00ef400640 1 -- 172.21.15.42:0/2115849366 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d80ab9d0 con 0x7f00f00b42e0 2024-05-28T14:11:15.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.944+0000 7f00f6a00640 1 -- 172.21.15.42:0/2115849366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f00b42e0 msgr2=0x7f00f015ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:15.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.944+0000 7f00f6a00640 1 --2- 172.21.15.42:0/2115849366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f00b42e0 0x7f00f015ed10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f00f00b5570 tx=0x7f00d801ce50 comp rx=0 tx=0).stop 2024-05-28T14:11:15.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.944+0000 7f00f6a00640 1 -- 172.21.15.42:0/2115849366 shutdown_connections 2024-05-28T14:11:15.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.944+0000 7f00f6a00640 1 --2- 172.21.15.42:0/2115849366 >> v2:172.21.15.63:3300/0 conn(0x7f00f0067810 0x7f00f006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.944+0000 7f00f6a00640 1 --2- 172.21.15.42:0/2115849366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f00b42e0 0x7f00f015ed10 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.944+0000 7f00f6a00640 1 -- 172.21.15.42:0/2115849366 >> 172.21.15.42:0/2115849366 conn(0x7f00f00d0b40 msgr2=0x7f00f00d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:15.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.944+0000 7f00f6a00640 1 -- 172.21.15.42:0/2115849366 shutdown_connections 2024-05-28T14:11:15.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.944+0000 7f00f6a00640 1 -- 172.21.15.42:0/2115849366 wait complete. 2024-05-28T14:11:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.944+0000 7f00f6a00640 1 Processor -- start 2024-05-28T14:11:15.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.944+0000 7f00f6a00640 1 -- start start 2024-05-28T14:11:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00f6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f0067810 0x7f00f00b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00f6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00f00b42e0 0x7f00f013d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00f6a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f00d0420 0x7f00f013d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00f6a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f00f006cb60 con 0x7f00f0067810 2024-05-28T14:11:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00f6a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f00f006c9e0 con 0x7f00f00b42e0 2024-05-28T14:11:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00f6a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f00f006cce0 con 0x7f00f00d0420 2024-05-28T14:11:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00efe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f0067810 0x7f00f00b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00efe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f0067810 0x7f00f00b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59736/0 (socket says 172.21.15.42:59736) 2024-05-28T14:11:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00efe00640 1 -- 172.21.15.42:0/3089316839 learned_addr learned my addr 172.21.15.42:0/3089316839 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00e7400640 1 --2- 172.21.15.42:0/3089316839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00f00b42e0 0x7f00f013d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00f4a00640 1 --2- 172.21.15.42:0/3089316839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f00d0420 0x7f00f013d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:15.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00efe00640 1 -- 172.21.15.42:0/3089316839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f00d0420 msgr2=0x7f00f013d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00efe00640 1 --2- 172.21.15.42:0/3089316839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f00d0420 0x7f00f013d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00efe00640 1 -- 172.21.15.42:0/3089316839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00f00b42e0 msgr2=0x7f00f013d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00efe00640 1 --2- 172.21.15.42:0/3089316839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00f00b42e0 0x7f00f013d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00efe00640 1 -- 172.21.15.42:0/3089316839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f015ee70 con 0x7f00f0067810 2024-05-28T14:11:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00efe00640 1 --2- 172.21.15.42:0/3089316839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f0067810 0x7f00f00b1d50 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f00e000c480 tx=0x7f00e000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00ecc00640 1 -- 172.21.15.42:0/3089316839 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0018070 con 0x7f00f0067810 2024-05-28T14:11:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00ecc00640 1 -- 172.21.15.42:0/3089316839 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e0007500 con 0x7f00f0067810 2024-05-28T14:11:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00ecc00640 1 -- 172.21.15.42:0/3089316839 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0004a30 con 0x7f00f0067810 2024-05-28T14:11:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00f6a00640 1 -- 172.21.15.42:0/3089316839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f01689a0 con 0x7f00f0067810 2024-05-28T14:11:15.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00f6a00640 1 -- 172.21.15.42:0/3089316839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f00b1550 con 0x7f00f0067810 2024-05-28T14:11:15.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00f6a00640 1 -- 172.21.15.42:0/3089316839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f0068a90 con 0x7f00f0067810 2024-05-28T14:11:15.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.948+0000 7f00ecc00640 1 -- 172.21.15.42:0/3089316839 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f00e001b070 con 0x7f00f0067810 2024-05-28T14:11:15.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.952+0000 7f00ecc00640 1 --2- 172.21.15.42:0/3089316839 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00cc069880 0x7f00cc06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:15.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.952+0000 7f00ecc00640 1 -- 172.21.15.42:0/3089316839 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5737+0+0 (secure 0 0 0) 0x7f00e0084970 con 0x7f00f0067810 2024-05-28T14:11:15.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.952+0000 7f00e7400640 1 --2- 172.21.15.42:0/3089316839 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00cc069880 0x7f00cc06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:15.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.952+0000 7f00e7400640 1 --2- 172.21.15.42:0/3089316839 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00cc069880 0x7f00cc06bd40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f00d80ad000 tx=0x7f00d8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:15.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:15.956+0000 7f00ecc00640 1 -- 172.21.15.42:0/3089316839 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f00e0056980 con 0x7f00f0067810 2024-05-28T14:11:16.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.124+0000 7f00f6a00640 1 -- 172.21.15.42:0/3089316839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00f0152ae0 con 0x7f00f0067810 2024-05-28T14:11:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.124+0000 7f00ecc00640 1 -- 172.21.15.42:0/3089316839 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16615 (secure 0 0 0) 0x7f00e0056280 con 0x7f00f0067810 2024-05-28T14:11:16.108 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:16.108 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":104,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:11:14.793590+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:11:13.194240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-28T14:11:13.743150+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:11:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.124+0000 7f00f6a00640 1 -- 172.21.15.42:0/3089316839 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00cc069880 msgr2=0x7f00cc06bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.124+0000 7f00f6a00640 1 --2- 172.21.15.42:0/3089316839 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00cc069880 0x7f00cc06bd40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f00d80ad000 tx=0x7f00d8008040 comp rx=0 tx=0).stop 2024-05-28T14:11:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.124+0000 7f00f6a00640 1 -- 172.21.15.42:0/3089316839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f0067810 msgr2=0x7f00f00b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.124+0000 7f00f6a00640 1 --2- 172.21.15.42:0/3089316839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f0067810 0x7f00f00b1d50 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f00e000c480 tx=0x7f00e000c940 comp rx=0 tx=0).stop 2024-05-28T14:11:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.124+0000 7f00f6a00640 1 -- 172.21.15.42:0/3089316839 shutdown_connections 2024-05-28T14:11:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.124+0000 7f00f6a00640 1 --2- 172.21.15.42:0/3089316839 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f00cc069880 0x7f00cc06bd40 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.124+0000 7f00f6a00640 1 --2- 172.21.15.42:0/3089316839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f00d0420 0x7f00f013d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.124+0000 7f00f6a00640 1 --2- 172.21.15.42:0/3089316839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00f00b42e0 0x7f00f013d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.124+0000 7f00f6a00640 1 --2- 172.21.15.42:0/3089316839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f0067810 0x7f00f00b1d50 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.124+0000 7f00f6a00640 1 -- 172.21.15.42:0/3089316839 >> 172.21.15.42:0/3089316839 conn(0x7f00f00d0b40 msgr2=0x7f00f00b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:16.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.124+0000 7f00f6a00640 1 -- 172.21.15.42:0/3089316839 shutdown_connections 2024-05-28T14:11:16.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.124+0000 7f00f6a00640 1 -- 172.21.15.42:0/3089316839 wait complete. 2024-05-28T14:11:16.130 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-28T14:11:16.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.288+0000 7f2ca7400640 1 Processor -- start 2024-05-28T14:11:16.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2ca7400640 1 -- start start 2024-05-28T14:11:16.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2ca7400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2ca0144f80 0x7f2ca0072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:16.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2ca7400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2ca015c0c0 0x7f2ca006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:16.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2ca7400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca00afe40 con 0x7f2ca006ee20 2024-05-28T14:11:16.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2ca7400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca006f7c0 con 0x7f2ca0144f80 2024-05-28T14:11:16.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2ca7400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca00724d0 con 0x7f2ca015c0c0 2024-05-28T14:11:16.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2ca4a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2ca0144f80 0x7f2ca0072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:16.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2c9fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f2ca006ee20 0x7f2ca015b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:44820/0 (socket says 172.21.15.42:44820) 2024-05-28T14:11:16.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2c9fe00640 1 -- 172.21.15.42:0/2859020991 learned_addr learned my addr 172.21.15.42:0/2859020991 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:16.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2ca5400640 1 --2- 172.21.15.42:0/2859020991 >> v2:172.21.15.63:3300/0 conn(0x7f2ca015c0c0 0x7f2ca006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:16.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2ca4a00640 1 -- 172.21.15.42:0/2859020991 >> v2:172.21.15.63:3300/0 conn(0x7f2ca015c0c0 msgr2=0x7f2ca006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:16.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2ca4a00640 1 --2- 172.21.15.42:0/2859020991 >> v2:172.21.15.63:3300/0 conn(0x7f2ca015c0c0 0x7f2ca006f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:16.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2ca4a00640 1 -- 172.21.15.42:0/2859020991 >> v1:172.21.15.42:6789/0 conn(0x7f2ca006ee20 legacy=0x7f2ca015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:16.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2ca4a00640 1 -- 172.21.15.42:0/2859020991 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca015c600 con 0x7f2ca0144f80 2024-05-28T14:11:16.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.292+0000 7f2c9f400640 1 -- 172.21.15.42:0/2859020991 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3398986334 0 0) 0x7f2ca00afe40 con 0x7f2ca006ee20 2024-05-28T14:11:16.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.296+0000 7f2ca4a00640 1 --2- 172.21.15.42:0/2859020991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ca0144f80 0x7f2ca0072960 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0160060 tx=0x7f2c9401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f3bfa6318b6b21 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:16.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.296+0000 7f2c9f400640 1 -- 172.21.15.42:0/2859020991 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c940b0070 con 0x7f2ca0144f80 2024-05-28T14:11:16.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.296+0000 7f2c9f400640 1 -- 172.21.15.42:0/2859020991 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c940ab600 con 0x7f2ca0144f80 2024-05-28T14:11:16.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.296+0000 7f2c9f400640 1 -- 172.21.15.42:0/2859020991 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c940ab9d0 con 0x7f2ca0144f80 2024-05-28T14:11:16.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.296+0000 7f2ca7400640 1 -- 172.21.15.42:0/2859020991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ca0144f80 msgr2=0x7f2ca0072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:16.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.296+0000 7f2ca7400640 1 --2- 172.21.15.42:0/2859020991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ca0144f80 0x7f2ca0072960 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0160060 tx=0x7f2c9401c480 comp rx=0 tx=0).stop 2024-05-28T14:11:16.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.296+0000 7f2ca7400640 1 -- 172.21.15.42:0/2859020991 shutdown_connections 2024-05-28T14:11:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.296+0000 7f2ca7400640 1 --2- 172.21.15.42:0/2859020991 >> v2:172.21.15.63:3300/0 conn(0x7f2ca015c0c0 0x7f2ca006f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.296+0000 7f2ca7400640 1 --2- 172.21.15.42:0/2859020991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ca0144f80 0x7f2ca0072960 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.296+0000 7f2ca7400640 1 -- 172.21.15.42:0/2859020991 >> 172.21.15.42:0/2859020991 conn(0x7f2ca00b21f0 msgr2=0x7f2ca00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.296+0000 7f2ca7400640 1 -- 172.21.15.42:0/2859020991 shutdown_connections 2024-05-28T14:11:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.296+0000 7f2ca7400640 1 -- 172.21.15.42:0/2859020991 wait complete. 2024-05-28T14:11:16.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.296+0000 7f2ca7400640 1 Processor -- start 2024-05-28T14:11:16.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.296+0000 7f2ca7400640 1 -- start start 2024-05-28T14:11:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca7400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ca006ee20 0x7f2ca015ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ca0144f80 0x7f2ca0135370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ca015c0c0 0x7f2ca0142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca7400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0072ea0 con 0x7f2ca015c0c0 2024-05-28T14:11:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca7400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca00b4010 con 0x7f2ca0144f80 2024-05-28T14:11:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca7400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca0135e10 con 0x7f2ca006ee20 2024-05-28T14:11:16.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ca015c0c0 0x7f2ca0142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:16.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca4a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ca006ee20 0x7f2ca015ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:16.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2c9fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ca0144f80 0x7f2ca0135370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:16.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ca015c0c0 0x7f2ca0142fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59766/0 (socket says 172.21.15.42:59766) 2024-05-28T14:11:16.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca5400640 1 -- 172.21.15.42:0/186753257 learned_addr learned my addr 172.21.15.42:0/186753257 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:16.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca5400640 1 -- 172.21.15.42:0/186753257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ca006ee20 msgr2=0x7f2ca015ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:16.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca5400640 1 --2- 172.21.15.42:0/186753257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ca006ee20 0x7f2ca015ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:16.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca5400640 1 -- 172.21.15.42:0/186753257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ca0144f80 msgr2=0x7f2ca0135370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:16.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca5400640 1 --2- 172.21.15.42:0/186753257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ca0144f80 0x7f2ca0135370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:16.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca5400640 1 -- 172.21.15.42:0/186753257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca01360f0 con 0x7f2ca015c0c0 2024-05-28T14:11:16.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca5400640 1 --2- 172.21.15.42:0/186753257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ca015c0c0 0x7f2ca0142fb0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f2c98007e60 tx=0x7f2c9800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:16.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2c9cc00640 1 -- 172.21.15.42:0/186753257 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98014420 con 0x7f2ca015c0c0 2024-05-28T14:11:16.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2c9cc00640 1 -- 172.21.15.42:0/186753257 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c98019070 con 0x7f2ca015c0c0 2024-05-28T14:11:16.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2c9cc00640 1 -- 172.21.15.42:0/186753257 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98014820 con 0x7f2ca015c0c0 2024-05-28T14:11:16.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca7400640 1 -- 172.21.15.42:0/186753257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca015e000 con 0x7f2ca015c0c0 2024-05-28T14:11:16.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca7400640 1 -- 172.21.15.42:0/186753257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca015e5c0 con 0x7f2ca015c0c0 2024-05-28T14:11:16.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.300+0000 7f2ca7400640 1 -- 172.21.15.42:0/186753257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca00738b0 con 0x7f2ca015c0c0 2024-05-28T14:11:16.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.304+0000 7f2c9cc00640 1 -- 172.21.15.42:0/186753257 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f2c980149c0 con 0x7f2ca015c0c0 2024-05-28T14:11:16.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.304+0000 7f2c9cc00640 1 --2- 172.21.15.42:0/186753257 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c80069850 0x7f2c8006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:16.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.304+0000 7f2c9cc00640 1 -- 172.21.15.42:0/186753257 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5737+0+0 (secure 0 0 0) 0x7f2c98086130 con 0x7f2ca015c0c0 2024-05-28T14:11:16.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.304+0000 7f2ca4a00640 1 --2- 172.21.15.42:0/186753257 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c80069850 0x7f2c8006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:16.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.304+0000 7f2ca4a00640 1 --2- 172.21.15.42:0/186753257 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c80069850 0x7f2c8006bd10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2c940ad000 tx=0x7f2c940a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:16.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.308+0000 7f2c9cc00640 1 -- 172.21.15.42:0/186753257 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2c98057770 con 0x7f2ca015c0c0 2024-05-28T14:11:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:16.516+0000 7f2ca7400640 1 -- 172.21.15.42:0/186753257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2ca004fdd0 con 0x7f2ca015c0c0 2024-05-28T14:11:17.356 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-28T14:11:17.373+0000 7ff448000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-28T14:11:17.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.408+0000 7f2c9cc00640 1 -- 172.21.15.42:0/186753257 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) ==== 118+0+0 (secure 0 0 0) 0x7f2c9805ba70 con 0x7f2ca015c0c0 2024-05-28T14:11:17.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.412+0000 7f2ca7400640 1 -- 172.21.15.42:0/186753257 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c80069850 msgr2=0x7f2c8006bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:17.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.412+0000 7f2ca7400640 1 --2- 172.21.15.42:0/186753257 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c80069850 0x7f2c8006bd10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2c940ad000 tx=0x7f2c940a6000 comp rx=0 tx=0).stop 2024-05-28T14:11:17.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.412+0000 7f2ca7400640 1 -- 172.21.15.42:0/186753257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ca015c0c0 msgr2=0x7f2ca0142fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:17.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.412+0000 7f2ca7400640 1 --2- 172.21.15.42:0/186753257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ca015c0c0 0x7f2ca0142fb0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f2c98007e60 tx=0x7f2c9800e4b0 comp rx=0 tx=0).stop 2024-05-28T14:11:17.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.412+0000 7f2ca7400640 1 -- 172.21.15.42:0/186753257 shutdown_connections 2024-05-28T14:11:17.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.412+0000 7f2ca7400640 1 --2- 172.21.15.42:0/186753257 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c80069850 0x7f2c8006bd10 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:17.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.412+0000 7f2ca7400640 1 --2- 172.21.15.42:0/186753257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ca015c0c0 0x7f2ca0142fb0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:17.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.412+0000 7f2ca7400640 1 --2- 172.21.15.42:0/186753257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ca0144f80 0x7f2ca0135370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:17.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.412+0000 7f2ca7400640 1 --2- 172.21.15.42:0/186753257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ca006ee20 0x7f2ca015ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:17.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.412+0000 7f2ca7400640 1 -- 172.21.15.42:0/186753257 >> 172.21.15.42:0/186753257 conn(0x7f2ca00b21f0 msgr2=0x7f2ca015b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:17.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.412+0000 7f2ca7400640 1 -- 172.21.15.42:0/186753257 shutdown_connections 2024-05-28T14:11:17.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.412+0000 7f2ca7400640 1 -- 172.21.15.42:0/186753257 wait complete. 2024-05-28T14:11:17.423 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-28T14:11:17.423 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-28T14:11:17.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.588+0000 7fd413000640 1 Processor -- start 2024-05-28T14:11:17.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.588+0000 7fd413000640 1 -- start start 2024-05-28T14:11:17.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.588+0000 7fd413000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd40c15e8c0 0x7fd40c0b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:17.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.588+0000 7fd413000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd40c0b4440 con 0x7fd40c0b0650 2024-05-28T14:11:17.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.588+0000 7fd413000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd40c0b0f10 con 0x7fd40c0b0a90 2024-05-28T14:11:17.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.588+0000 7fd413000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd40c0ae420 con 0x7fd40c15e8c0 2024-05-28T14:11:17.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd403400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd40c0b0650 0x7fd40c06b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39210/0 (socket says 172.21.15.42:39210) 2024-05-28T14:11:17.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd403400640 1 -- 172.21.15.42:0/2653488909 learned_addr learned my addr 172.21.15.42:0/2653488909 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:17.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 -- 172.21.15.42:0/2653488909 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2233076364 0 0) 0x7fd3f4001120 con 0x7fd40c0b0650 2024-05-28T14:11:17.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 -- 172.21.15.42:0/2653488909 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3f0003530 con 0x7fd40c0b0650 2024-05-28T14:11:17.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40be00640 1 --2- 172.21.15.42:0/2653488909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd40c15e8c0 0x7fd40c0b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:17.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 -- 172.21.15.42:0/2653488909 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4286800584 0 0) 0x7fd3f0003530 con 0x7fd40c0b0650 2024-05-28T14:11:17.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 -- 172.21.15.42:0/2653488909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd40c15e8c0 msgr2=0x7fd40c0b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:17.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 --2- 172.21.15.42:0/2653488909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd40c15e8c0 0x7fd40c0b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:17.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 -- 172.21.15.42:0/2653488909 >> v1:172.21.15.63:6789/0 conn(0x7fd40c0b0a90 legacy=0x7fd40c0af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:17.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 -- 172.21.15.42:0/2653488909 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c0ae670 con 0x7fd40c0b0650 2024-05-28T14:11:17.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 -- 172.21.15.42:0/2653488909 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fd3f4001850 con 0x7fd40c0b0650 2024-05-28T14:11:17.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 -- 172.21.15.42:0/2653488909 >> v1:172.21.15.42:6789/0 conn(0x7fd40c0b0650 legacy=0x7fd40c06b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:17.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 --2- 172.21.15.42:0/2653488909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3f00039f0 0x7fd3f0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:17.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 --2- 172.21.15.42:0/2653488909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f0004810 0x7fd3f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:17.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 --2- 172.21.15.42:0/2653488909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd40c15e8c0 0x7fd3f0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:17.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 -- 172.21.15.42:0/2653488909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c0ae420 con 0x7fd3f0004810 2024-05-28T14:11:17.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 -- 172.21.15.42:0/2653488909 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f000bb00 con 0x7fd3f00039f0 2024-05-28T14:11:17.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40b400640 1 -- 172.21.15.42:0/2653488909 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f000bd00 con 0x7fd40c15e8c0 2024-05-28T14:11:17.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd403400640 1 --2- 172.21.15.42:0/2653488909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f0004810 0x7fd3f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:17.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd40be00640 1 --2- 172.21.15.42:0/2653488909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3f00039f0 0x7fd3f0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:17.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd411000640 1 --2- 172.21.15.42:0/2653488909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd40c15e8c0 0x7fd3f0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:17.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd403400640 1 -- 172.21.15.42:0/2653488909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd40c15e8c0 msgr2=0x7fd3f0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:17.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd403400640 1 --2- 172.21.15.42:0/2653488909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd40c15e8c0 0x7fd3f0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:17.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd403400640 1 -- 172.21.15.42:0/2653488909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3f00039f0 msgr2=0x7fd3f0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:17.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd403400640 1 --2- 172.21.15.42:0/2653488909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3f00039f0 0x7fd3f0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:17.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.592+0000 7fd403400640 1 -- 172.21.15.42:0/2653488909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f0003530 con 0x7fd3f0004810 2024-05-28T14:11:17.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd403400640 1 --2- 172.21.15.42:0/2653488909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f0004810 0x7fd3f0004c90 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400c4c0 tx=0x7fd3f401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e7dff526a26b26 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:17.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd40b400640 1 -- 172.21.15.42:0/2653488909 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f400c0e0 con 0x7fd3f0004810 2024-05-28T14:11:17.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd40b400640 1 -- 172.21.15.42:0/2653488909 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f40a9020 con 0x7fd3f0004810 2024-05-28T14:11:17.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd40b400640 1 -- 172.21.15.42:0/2653488909 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f40a5b10 con 0x7fd3f0004810 2024-05-28T14:11:17.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd413000640 1 -- 172.21.15.42:0/2653488909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f0004810 msgr2=0x7fd3f0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:17.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd413000640 1 --2- 172.21.15.42:0/2653488909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f0004810 0x7fd3f0004c90 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400c4c0 tx=0x7fd3f401cc30 comp rx=0 tx=0).stop 2024-05-28T14:11:17.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd411000640 1 -- 172.21.15.42:0/2653488909 reap_dead start 2024-05-28T14:11:17.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd413000640 1 -- 172.21.15.42:0/2653488909 shutdown_connections 2024-05-28T14:11:17.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd413000640 1 -- 172.21.15.42:0/2653488909 >> 172.21.15.42:0/2653488909 conn(0x7fd40c0b6960 msgr2=0x7fd40c0b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:17.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd413000640 1 -- 172.21.15.42:0/2653488909 shutdown_connections 2024-05-28T14:11:17.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd413000640 1 -- 172.21.15.42:0/2653488909 wait complete. 2024-05-28T14:11:17.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd413000640 1 Processor -- start 2024-05-28T14:11:17.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd413000640 1 -- start start 2024-05-28T14:11:17.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd413000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3f0004810 0x7fd40c161800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:17.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd413000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c152ec0 0x7fd40c149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd413000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd40c1f7590 0x7fd40c15f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd413000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c0b01f0 con 0x7fd40c152ec0 2024-05-28T14:11:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd413000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c0b0070 con 0x7fd3f0004810 2024-05-28T14:11:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.596+0000 7fd413000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd40c06c100 con 0x7fd40c1f7590 2024-05-28T14:11:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd403400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c152ec0 0x7fd40c149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd403400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c152ec0 0x7fd40c149ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53226/0 (socket says 172.21.15.42:53226) 2024-05-28T14:11:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd403400640 1 -- 172.21.15.42:0/7453741 learned_addr learned my addr 172.21.15.42:0/7453741 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd40be00640 1 --2- 172.21.15.42:0/7453741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3f0004810 0x7fd40c161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd411000640 1 --2- 172.21.15.42:0/7453741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd40c1f7590 0x7fd40c15f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd403400640 1 -- 172.21.15.42:0/7453741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd40c1f7590 msgr2=0x7fd40c15f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd403400640 1 --2- 172.21.15.42:0/7453741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd40c1f7590 0x7fd40c15f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd403400640 1 -- 172.21.15.42:0/7453741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3f0004810 msgr2=0x7fd40c161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd403400640 1 --2- 172.21.15.42:0/7453741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3f0004810 0x7fd40c161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd403400640 1 -- 172.21.15.42:0/7453741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c1536b0 con 0x7fd40c152ec0 2024-05-28T14:11:17.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd403400640 1 --2- 172.21.15.42:0/7453741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c152ec0 0x7fd40c149ef0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fd3f40ab040 tx=0x7fd3f0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:17.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd408c00640 1 -- 172.21.15.42:0/7453741 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f40a6ba0 con 0x7fd40c152ec0 2024-05-28T14:11:17.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd408c00640 1 -- 172.21.15.42:0/7453741 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f40a6d40 con 0x7fd40c152ec0 2024-05-28T14:11:17.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd408c00640 1 -- 172.21.15.42:0/7453741 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4004a10 con 0x7fd40c152ec0 2024-05-28T14:11:17.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd413000640 1 -- 172.21.15.42:0/7453741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40c159c90 con 0x7fd40c152ec0 2024-05-28T14:11:17.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd413000640 1 -- 172.21.15.42:0/7453741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40c15a250 con 0x7fd40c152ec0 2024-05-28T14:11:17.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd413000640 1 -- 172.21.15.42:0/7453741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40c09f7a0 con 0x7fd40c152ec0 2024-05-28T14:11:17.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.600+0000 7fd408c00640 1 -- 172.21.15.42:0/7453741 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fd3f4004bb0 con 0x7fd40c152ec0 2024-05-28T14:11:17.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.604+0000 7fd408c00640 1 --2- 172.21.15.42:0/7453741 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3e8069850 0x7fd3e806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:17.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.604+0000 7fd408c00640 1 -- 172.21.15.42:0/7453741 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fd3f411eb10 con 0x7fd40c152ec0 2024-05-28T14:11:17.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.604+0000 7fd40be00640 1 --2- 172.21.15.42:0/7453741 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3e8069850 0x7fd3e806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:17.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.604+0000 7fd40be00640 1 --2- 172.21.15.42:0/7453741 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3e8069850 0x7fd3e806bd10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009b20 tx=0x7fd3fc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:17.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.608+0000 7fd408c00640 1 -- 172.21.15.42:0/7453741 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd3f40bf070 con 0x7fd40c152ec0 2024-05-28T14:11:17.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:17.808+0000 7fd413000640 1 -- 172.21.15.42:0/7453741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd40c04fdd0 con 0x7fd40c152ec0 2024-05-28T14:11:18.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.412+0000 7fd408c00640 1 -- 172.21.15.42:0/7453741 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v64) ==== 122+0+0 (secure 0 0 0) 0x7fd40c0b4440 con 0x7fd40c152ec0 2024-05-28T14:11:18.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.412+0000 7fd413000640 1 -- 172.21.15.42:0/7453741 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3e8069850 msgr2=0x7fd3e806bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:18.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.412+0000 7fd413000640 1 --2- 172.21.15.42:0/7453741 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3e8069850 0x7fd3e806bd10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009b20 tx=0x7fd3fc008040 comp rx=0 tx=0).stop 2024-05-28T14:11:18.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.412+0000 7fd413000640 1 -- 172.21.15.42:0/7453741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c152ec0 msgr2=0x7fd40c149ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:18.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.412+0000 7fd413000640 1 --2- 172.21.15.42:0/7453741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c152ec0 0x7fd40c149ef0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fd3f40ab040 tx=0x7fd3f0005840 comp rx=0 tx=0).stop 2024-05-28T14:11:18.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.412+0000 7fd413000640 1 -- 172.21.15.42:0/7453741 shutdown_connections 2024-05-28T14:11:18.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.412+0000 7fd413000640 1 --2- 172.21.15.42:0/7453741 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3e8069850 0x7fd3e806bd10 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.412+0000 7fd413000640 1 --2- 172.21.15.42:0/7453741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd40c1f7590 0x7fd40c15f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.412+0000 7fd413000640 1 --2- 172.21.15.42:0/7453741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c152ec0 0x7fd40c149ef0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.412+0000 7fd413000640 1 --2- 172.21.15.42:0/7453741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3f0004810 0x7fd40c161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.412+0000 7fd413000640 1 -- 172.21.15.42:0/7453741 >> 172.21.15.42:0/7453741 conn(0x7fd40c0b6960 msgr2=0x7fd40c15bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:18.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.412+0000 7fd413000640 1 -- 172.21.15.42:0/7453741 shutdown_connections 2024-05-28T14:11:18.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.412+0000 7fd413000640 1 -- 172.21.15.42:0/7453741 wait complete. 2024-05-28T14:11:18.422 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-28T14:11:18.422 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:11:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.580+0000 7fc0d3a00640 1 Processor -- start 2024-05-28T14:11:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0d3a00640 1 -- start start 2024-05-28T14:11:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0d3a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc0cc0ce860 0x7fc0cc06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0d3a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc0cc162c80 0x7fc0cc15e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0d3a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0cc0d0de0 con 0x7fc0cc0d0900 2024-05-28T14:11:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0d3a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc0cc0d19f0 con 0x7fc0cc162c80 2024-05-28T14:11:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0d3a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc0cc0d10c0 con 0x7fc0cc0ce860 2024-05-28T14:11:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0cbe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc0cc0d0900 0x7fc0cc162570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39224/0 (socket says 172.21.15.42:39224) 2024-05-28T14:11:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0cbe00640 1 -- 172.21.15.42:0/702103459 learned_addr learned my addr 172.21.15.42:0/702103459 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0d1a00640 1 --2- 172.21.15.42:0/702103459 >> v2:172.21.15.42:3300/0 conn(0x7fc0cc162c80 0x7fc0cc15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0d1000640 1 --2- 172.21.15.42:0/702103459 >> v2:172.21.15.63:3300/0 conn(0x7fc0cc0ce860 0x7fc0cc06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0cb400640 1 -- 172.21.15.42:0/702103459 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1317811148 0 0) 0x7fc0cc0d0de0 con 0x7fc0cc0d0900 2024-05-28T14:11:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0cb400640 1 -- 172.21.15.42:0/702103459 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0b4003530 con 0x7fc0cc0d0900 2024-05-28T14:11:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0d1a00640 1 -- 172.21.15.42:0/702103459 >> v2:172.21.15.63:3300/0 conn(0x7fc0cc0ce860 msgr2=0x7fc0cc06d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0d1a00640 1 --2- 172.21.15.42:0/702103459 >> v2:172.21.15.63:3300/0 conn(0x7fc0cc0ce860 0x7fc0cc06d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0d1a00640 1 -- 172.21.15.42:0/702103459 >> v1:172.21.15.42:6789/0 conn(0x7fc0cc0d0900 legacy=0x7fc0cc162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.584+0000 7fc0d1a00640 1 -- 172.21.15.42:0/702103459 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc0d13a0 con 0x7fc0cc162c80 2024-05-28T14:11:18.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.588+0000 7fc0d1a00640 1 --2- 172.21.15.42:0/702103459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0cc162c80 0x7fc0cc15e200 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc069710 tx=0x7fc0c401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29244081f17dd914 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:18.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.588+0000 7fc0cb400640 1 -- 172.21.15.42:0/702103459 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40b2070 con 0x7fc0cc162c80 2024-05-28T14:11:18.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.588+0000 7fc0cb400640 1 -- 172.21.15.42:0/702103459 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c40a5950 con 0x7fc0cc162c80 2024-05-28T14:11:18.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.588+0000 7fc0cb400640 1 -- 172.21.15.42:0/702103459 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40a5d20 con 0x7fc0cc162c80 2024-05-28T14:11:18.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.588+0000 7fc0d3a00640 1 -- 172.21.15.42:0/702103459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0cc162c80 msgr2=0x7fc0cc15e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:18.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.588+0000 7fc0d3a00640 1 --2- 172.21.15.42:0/702103459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0cc162c80 0x7fc0cc15e200 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc069710 tx=0x7fc0c401c870 comp rx=0 tx=0).stop 2024-05-28T14:11:18.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.588+0000 7fc0d3a00640 1 -- 172.21.15.42:0/702103459 shutdown_connections 2024-05-28T14:11:18.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.588+0000 7fc0d3a00640 1 --2- 172.21.15.42:0/702103459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0cc162c80 0x7fc0cc15e200 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.588+0000 7fc0d3a00640 1 --2- 172.21.15.42:0/702103459 >> v2:172.21.15.63:3300/0 conn(0x7fc0cc0ce860 0x7fc0cc06d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.588+0000 7fc0d3a00640 1 -- 172.21.15.42:0/702103459 >> 172.21.15.42:0/702103459 conn(0x7fc0cc0b4080 msgr2=0x7fc0cc0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:18.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.588+0000 7fc0d3a00640 1 -- 172.21.15.42:0/702103459 shutdown_connections 2024-05-28T14:11:18.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.588+0000 7fc0d3a00640 1 -- 172.21.15.42:0/702103459 wait complete. 2024-05-28T14:11:18.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.588+0000 7fc0d3a00640 1 Processor -- start 2024-05-28T14:11:18.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.588+0000 7fc0d3a00640 1 -- start start 2024-05-28T14:11:18.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0d3a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0cc0ce860 0x7fc0cc15e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:18.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0d3a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0cc0d0900 0x7fc0cc0c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:18.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0d3a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0cc162c80 0x7fc0cc1610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:18.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0d3a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc0ccf70 con 0x7fc0cc0d0900 2024-05-28T14:11:18.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0d3a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc0ccdf0 con 0x7fc0cc162c80 2024-05-28T14:11:18.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0d3a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc0cc0cd0f0 con 0x7fc0cc0ce860 2024-05-28T14:11:18.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0cbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0cc0d0900 0x7fc0cc0c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:18.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0d1000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0cc0ce860 0x7fc0cc15e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:18.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0cbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0cc0d0900 0x7fc0cc0c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53238/0 (socket says 172.21.15.42:53238) 2024-05-28T14:11:18.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0cbe00640 1 -- 172.21.15.42:0/350949831 learned_addr learned my addr 172.21.15.42:0/350949831 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:18.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0d1a00640 1 --2- 172.21.15.42:0/350949831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0cc162c80 0x7fc0cc1610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:18.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0cbe00640 1 -- 172.21.15.42:0/350949831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0cc0ce860 msgr2=0x7fc0cc15e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:18.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0cbe00640 1 --2- 172.21.15.42:0/350949831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0cc0ce860 0x7fc0cc15e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0cbe00640 1 -- 172.21.15.42:0/350949831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0cc162c80 msgr2=0x7fc0cc1610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:18.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0cbe00640 1 --2- 172.21.15.42:0/350949831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0cc162c80 0x7fc0cc1610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0cbe00640 1 -- 172.21.15.42:0/350949831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc0c7740 con 0x7fc0cc0d0900 2024-05-28T14:11:18.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0cbe00640 1 --2- 172.21.15.42:0/350949831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0cc0d0900 0x7fc0cc0c6e60 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fc0b800c450 tx=0x7fc0b800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:18.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0c8c00640 1 -- 172.21.15.42:0/350949831 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8017070 con 0x7fc0cc0d0900 2024-05-28T14:11:18.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0c8c00640 1 -- 172.21.15.42:0/350949831 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b8005cf0 con 0x7fc0cc0d0900 2024-05-28T14:11:18.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0c8c00640 1 -- 172.21.15.42:0/350949831 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8007290 con 0x7fc0cc0d0900 2024-05-28T14:11:18.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0d3a00640 1 -- 172.21.15.42:0/350949831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0cc0c7a20 con 0x7fc0cc0d0900 2024-05-28T14:11:18.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0d3a00640 1 -- 172.21.15.42:0/350949831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0cc0c82c0 con 0x7fc0cc0d0900 2024-05-28T14:11:18.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.592+0000 7fc0d3a00640 1 -- 172.21.15.42:0/350949831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0cc0c7e10 con 0x7fc0cc0d0900 2024-05-28T14:11:18.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.596+0000 7fc0c8c00640 1 -- 172.21.15.42:0/350949831 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fc0b801a070 con 0x7fc0cc0d0900 2024-05-28T14:11:18.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.596+0000 7fc0c8c00640 1 --2- 172.21.15.42:0/350949831 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc0ac069850 0x7fc0ac06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:18.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.596+0000 7fc0c8c00640 1 -- 172.21.15.42:0/350949831 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fc0b8084a20 con 0x7fc0cc0d0900 2024-05-28T14:11:18.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.596+0000 7fc0d1000640 1 --2- 172.21.15.42:0/350949831 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc0ac069850 0x7fc0ac06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:18.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.596+0000 7fc0d1000640 1 --2- 172.21.15.42:0/350949831 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc0ac069850 0x7fc0ac06bd10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009b60 tx=0x7fc0c0009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:18.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.600+0000 7fc0c8c00640 1 -- 172.21.15.42:0/350949831 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc0b80563b0 con 0x7fc0cc0d0900 2024-05-28T14:11:18.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.800+0000 7fc0d3a00640 1 -- 172.21.15.42:0/350949831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0cc068a90 con 0x7fc0cc0d0900 2024-05-28T14:11:18.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.800+0000 7fc0c8c00640 1 -- 172.21.15.42:0/350949831 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5190 (secure 0 0 0) 0x7fc0b805a6b0 con 0x7fc0cc0d0900 2024-05-28T14:11:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:18.784 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":64,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8607,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/1661021064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1661021064},{"type":"v1","addr":"172.21.15.42:6836","nonce":1661021064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":57050,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2398440572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":2398440572},{"type":"v1","addr":"172.21.15.42:6837","nonce":2398440572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":57053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/2530315729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":2530315729},{"type":"v1","addr":"172.21.15.63:6834","nonce":2530315729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:11:14.793551+0000","modified":"2024-05-28T14:11:17.812010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57056},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57056":{"gid":57056,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/1184583947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1184583947},{"type":"v1","addr":"172.21.15.63:6835","nonce":1184583947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57056,"qdb_cluster":[57056]},"id":7}]} 2024-05-28T14:11:18.784 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 64 2024-05-28T14:11:18.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.800+0000 7fc0d3a00640 1 -- 172.21.15.42:0/350949831 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc0ac069850 msgr2=0x7fc0ac06bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:18.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.800+0000 7fc0d3a00640 1 --2- 172.21.15.42:0/350949831 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc0ac069850 0x7fc0ac06bd10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009b60 tx=0x7fc0c0009190 comp rx=0 tx=0).stop 2024-05-28T14:11:18.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.800+0000 7fc0d3a00640 1 -- 172.21.15.42:0/350949831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0cc0d0900 msgr2=0x7fc0cc0c6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:18.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.800+0000 7fc0d3a00640 1 --2- 172.21.15.42:0/350949831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0cc0d0900 0x7fc0cc0c6e60 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fc0b800c450 tx=0x7fc0b800c910 comp rx=0 tx=0).stop 2024-05-28T14:11:18.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.800+0000 7fc0d3a00640 1 -- 172.21.15.42:0/350949831 shutdown_connections 2024-05-28T14:11:18.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.800+0000 7fc0d3a00640 1 --2- 172.21.15.42:0/350949831 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc0ac069850 0x7fc0ac06bd10 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.800+0000 7fc0d3a00640 1 --2- 172.21.15.42:0/350949831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0cc162c80 0x7fc0cc1610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.800+0000 7fc0d3a00640 1 --2- 172.21.15.42:0/350949831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0cc0d0900 0x7fc0cc0c6e60 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.800+0000 7fc0d3a00640 1 --2- 172.21.15.42:0/350949831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0cc0ce860 0x7fc0cc15e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.800+0000 7fc0d3a00640 1 -- 172.21.15.42:0/350949831 >> 172.21.15.42:0/350949831 conn(0x7fc0cc0b4080 msgr2=0x7fc0cc0cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:18.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.800+0000 7fc0d3a00640 1 -- 172.21.15.42:0/350949831 shutdown_connections 2024-05-28T14:11:18.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.800+0000 7fc0d3a00640 1 -- 172.21.15.42:0/350949831 wait complete. 2024-05-28T14:11:18.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:11:18.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.960+0000 7f9d2dc00640 1 Processor -- start 2024-05-28T14:11:18.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.964+0000 7f9d2dc00640 1 -- start start 2024-05-28T14:11:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.964+0000 7f9d2dc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9d28156a70 0x7f9d2806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.964+0000 7f9d2dc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d28156e50 0x7f9d280af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.964+0000 7f9d2dc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d2815cb90 con 0x7f9d280af880 2024-05-28T14:11:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.964+0000 7f9d2dc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9d2815ce50 con 0x7f9d28156a70 2024-05-28T14:11:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.964+0000 7f9d2dc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2815d130 con 0x7f9d28156e50 2024-05-28T14:11:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.964+0000 7f9d26a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9d28156a70 0x7f9d2806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.964+0000 7f9d26a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9d28156a70 0x7f9d2806dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53248/0 (socket says 172.21.15.42:53248) 2024-05-28T14:11:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.964+0000 7f9d26a00640 1 -- 172.21.15.42:0/3199753606 learned_addr learned my addr 172.21.15.42:0/3199753606 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:18.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.964+0000 7f9d1fe00640 1 --2- 172.21.15.42:0/3199753606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d28156e50 0x7f9d280af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:18.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.964+0000 7f9d26a00640 1 -- 172.21.15.42:0/3199753606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d28156e50 msgr2=0x7f9d280af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:18.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.964+0000 7f9d26a00640 1 --2- 172.21.15.42:0/3199753606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d28156e50 0x7f9d280af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.964+0000 7f9d26a00640 1 -- 172.21.15.42:0/3199753606 >> v1:172.21.15.63:6789/0 conn(0x7f9d280af880 legacy=0x7f9d280d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:18.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.964+0000 7f9d26a00640 1 -- 172.21.15.42:0/3199753606 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2815d410 con 0x7f9d28156a70 2024-05-28T14:11:18.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.968+0000 7f9d26a00640 1 --2- 172.21.15.42:0/3199753606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d28156a70 0x7f9d2806dd60 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009870 tx=0x7f9d1801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846bd66575e43d37 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:18.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.968+0000 7f9d26000640 1 -- 172.21.15.42:0/3199753606 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d180a7070 con 0x7f9d28156a70 2024-05-28T14:11:18.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.968+0000 7f9d26000640 1 -- 172.21.15.42:0/3199753606 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d180b0070 con 0x7f9d28156a70 2024-05-28T14:11:18.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.968+0000 7f9d26000640 1 -- 172.21.15.42:0/3199753606 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d180ab890 con 0x7f9d28156a70 2024-05-28T14:11:18.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.968+0000 7f9d2dc00640 1 -- 172.21.15.42:0/3199753606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d28156a70 msgr2=0x7f9d2806dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:18.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.968+0000 7f9d2dc00640 1 --2- 172.21.15.42:0/3199753606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d28156a70 0x7f9d2806dd60 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009870 tx=0x7f9d1801c560 comp rx=0 tx=0).stop 2024-05-28T14:11:18.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.968+0000 7f9d2dc00640 1 -- 172.21.15.42:0/3199753606 shutdown_connections 2024-05-28T14:11:18.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.968+0000 7f9d2dc00640 1 --2- 172.21.15.42:0/3199753606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d28156e50 0x7f9d280af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.968+0000 7f9d2dc00640 1 --2- 172.21.15.42:0/3199753606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d28156a70 0x7f9d2806dd60 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.968+0000 7f9d2dc00640 1 -- 172.21.15.42:0/3199753606 >> 172.21.15.42:0/3199753606 conn(0x7f9d2806bdf0 msgr2=0x7f9d2806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:18.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.968+0000 7f9d2dc00640 1 -- 172.21.15.42:0/3199753606 shutdown_connections 2024-05-28T14:11:18.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.968+0000 7f9d2dc00640 1 -- 172.21.15.42:0/3199753606 wait complete. 2024-05-28T14:11:18.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.968+0000 7f9d2dc00640 1 Processor -- start 2024-05-28T14:11:18.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.968+0000 7f9d2dc00640 1 -- start start 2024-05-28T14:11:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d2dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d280af880 0x7f9d280b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d2dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d28156a70 0x7f9d28073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d2dc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d28156e50 0x7f9d28072020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d2dc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9d28160b40 con 0x7f9d280af880 2024-05-28T14:11:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d2dc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9d280d1010 con 0x7f9d28156a70 2024-05-28T14:11:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d2dc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9d28160cc0 con 0x7f9d28156e50 2024-05-28T14:11:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d26a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d280af880 0x7f9d280b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d26a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d280af880 0x7f9d280b1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53258/0 (socket says 172.21.15.42:53258) 2024-05-28T14:11:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d26a00640 1 -- 172.21.15.42:0/2312277281 learned_addr learned my addr 172.21.15.42:0/2312277281 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d1fe00640 1 --2- 172.21.15.42:0/2312277281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d28156a70 0x7f9d28073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:18.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d27400640 1 --2- 172.21.15.42:0/2312277281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d28156e50 0x7f9d28072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d26a00640 1 -- 172.21.15.42:0/2312277281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d28156e50 msgr2=0x7f9d28072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d26a00640 1 --2- 172.21.15.42:0/2312277281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d28156e50 0x7f9d28072020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d26a00640 1 -- 172.21.15.42:0/2312277281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d28156a70 msgr2=0x7f9d28073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d26a00640 1 --2- 172.21.15.42:0/2312277281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d28156a70 0x7f9d28073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d26a00640 1 -- 172.21.15.42:0/2312277281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d28073580 con 0x7f9d280af880 2024-05-28T14:11:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d26a00640 1 --2- 172.21.15.42:0/2312277281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d280af880 0x7f9d280b1780 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f9d180ad040 tx=0x7f9d1801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d1ea00640 1 -- 172.21.15.42:0/2312277281 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d180b4800 con 0x7f9d280af880 2024-05-28T14:11:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d1ea00640 1 -- 172.21.15.42:0/2312277281 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d180b49c0 con 0x7f9d280af880 2024-05-28T14:11:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d1ea00640 1 -- 172.21.15.42:0/2312277281 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d180b4db0 con 0x7f9d280af880 2024-05-28T14:11:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d2dc00640 1 -- 172.21.15.42:0/2312277281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2814f6f0 con 0x7f9d280af880 2024-05-28T14:11:18.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d2dc00640 1 -- 172.21.15.42:0/2312277281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d2814fcb0 con 0x7f9d280af880 2024-05-28T14:11:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d2dc00640 1 -- 172.21.15.42:0/2312277281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d28068a90 con 0x7f9d280af880 2024-05-28T14:11:18.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.972+0000 7f9d1ea00640 1 -- 172.21.15.42:0/2312277281 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f9d180c7390 con 0x7f9d280af880 2024-05-28T14:11:18.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.976+0000 7f9d1ea00640 1 --2- 172.21.15.42:0/2312277281 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9d04069850 0x7f9d0406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:18.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.976+0000 7f9d1ea00640 1 -- 172.21.15.42:0/2312277281 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f9d18120260 con 0x7f9d280af880 2024-05-28T14:11:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.976+0000 7f9d1fe00640 1 --2- 172.21.15.42:0/2312277281 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9d04069850 0x7f9d0406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.976+0000 7f9d1fe00640 1 --2- 172.21.15.42:0/2312277281 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9d04069850 0x7f9d0406bd10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9d28073e50 tx=0x7f9d10009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:18.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:18.980+0000 7f9d1ea00640 1 -- 172.21.15.42:0/2312277281 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9d1812a080 con 0x7f9d280af880 2024-05-28T14:11:19.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.144+0000 7f9d2dc00640 1 -- 172.21.15.42:0/2312277281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d280b4a80 con 0x7f9d280af880 2024-05-28T14:11:19.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.148+0000 7f9d1ea00640 1 -- 172.21.15.42:0/2312277281 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16602 (secure 0 0 0) 0x7f9d180c4060 con 0x7f9d280af880 2024-05-28T14:11:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:19.131 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":105,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:11:16.396600+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:11:13.194240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-28T14:11:13.743150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:11:19.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.148+0000 7f9d2dc00640 1 -- 172.21.15.42:0/2312277281 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9d04069850 msgr2=0x7f9d0406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.148+0000 7f9d2dc00640 1 --2- 172.21.15.42:0/2312277281 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9d04069850 0x7f9d0406bd10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9d28073e50 tx=0x7f9d10009190 comp rx=0 tx=0).stop 2024-05-28T14:11:19.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.148+0000 7f9d2dc00640 1 -- 172.21.15.42:0/2312277281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d280af880 msgr2=0x7f9d280b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.148+0000 7f9d2dc00640 1 --2- 172.21.15.42:0/2312277281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d280af880 0x7f9d280b1780 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f9d180ad040 tx=0x7f9d1801ada0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.148+0000 7f9d2dc00640 1 -- 172.21.15.42:0/2312277281 shutdown_connections 2024-05-28T14:11:19.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.148+0000 7f9d2dc00640 1 --2- 172.21.15.42:0/2312277281 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9d04069850 0x7f9d0406bd10 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.148+0000 7f9d2dc00640 1 --2- 172.21.15.42:0/2312277281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d28156e50 0x7f9d28072020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.148+0000 7f9d2dc00640 1 --2- 172.21.15.42:0/2312277281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d28156a70 0x7f9d28073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.148+0000 7f9d2dc00640 1 --2- 172.21.15.42:0/2312277281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d280af880 0x7f9d280b1780 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.148+0000 7f9d2dc00640 1 -- 172.21.15.42:0/2312277281 >> 172.21.15.42:0/2312277281 conn(0x7f9d2806bdf0 msgr2=0x7f9d280b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:19.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.148+0000 7f9d2dc00640 1 -- 172.21.15.42:0/2312277281 shutdown_connections 2024-05-28T14:11:19.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.148+0000 7f9d2dc00640 1 -- 172.21.15.42:0/2312277281 wait complete. 2024-05-28T14:11:19.152 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-28T14:11:19.152 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:11:19.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.308+0000 7f9662c00640 1 Processor -- start 2024-05-28T14:11:19.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.308+0000 7f9662c00640 1 -- start start 2024-05-28T14:11:19.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.312+0000 7f9662c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f965c0afa50 0x7f965c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:19.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.312+0000 7f9662c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f965c15cb90 con 0x7f965c156e50 2024-05-28T14:11:19.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.312+0000 7f9662c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f965c15ce50 con 0x7f965c156a70 2024-05-28T14:11:19.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.312+0000 7f9662c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f965c15d130 con 0x7f965c0afa50 2024-05-28T14:11:19.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.312+0000 7f9653e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f965c0afa50 0x7f965c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:19.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.312+0000 7f9660c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f965c156e50 0x7f965c0d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39232/0 (socket says 172.21.15.42:39232) 2024-05-28T14:11:19.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.312+0000 7f9660c00640 1 -- 172.21.15.42:0/1494003640 learned_addr learned my addr 172.21.15.42:0/1494003640 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:19.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.312+0000 7f965b400640 1 -- 172.21.15.42:0/1494003640 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 50079013 0 0) 0x7f965c15cb90 con 0x7f965c156e50 2024-05-28T14:11:19.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.312+0000 7f965b400640 1 -- 172.21.15.42:0/1494003640 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9640003530 con 0x7f965c156e50 2024-05-28T14:11:19.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.312+0000 7f9653e00640 1 -- 172.21.15.42:0/1494003640 >> v1:172.21.15.63:6789/0 conn(0x7f965c156a70 legacy=0x7f965c0af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.312+0000 7f9653e00640 1 -- 172.21.15.42:0/1494003640 >> v1:172.21.15.42:6789/0 conn(0x7f965c156e50 legacy=0x7f965c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.312+0000 7f9653e00640 1 -- 172.21.15.42:0/1494003640 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965c15d410 con 0x7f965c0afa50 2024-05-28T14:11:19.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9653e00640 1 --2- 172.21.15.42:0/1494003640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f965c0afa50 0x7f965c06dd60 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f96440097c0 tx=0x7f964401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7949b4a646c13049 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:19.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f965b400640 1 -- 172.21.15.42:0/1494003640 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96440aa620 con 0x7f965c0afa50 2024-05-28T14:11:19.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f965b400640 1 -- 172.21.15.42:0/1494003640 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96440af070 con 0x7f965c0afa50 2024-05-28T14:11:19.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f965b400640 1 -- 172.21.15.42:0/1494003640 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96440aa9e0 con 0x7f965c0afa50 2024-05-28T14:11:19.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 -- 172.21.15.42:0/1494003640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f965c0afa50 msgr2=0x7f965c06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 --2- 172.21.15.42:0/1494003640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f965c0afa50 0x7f965c06dd60 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f96440097c0 tx=0x7f964401c4a0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 -- 172.21.15.42:0/1494003640 shutdown_connections 2024-05-28T14:11:19.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 --2- 172.21.15.42:0/1494003640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f965c0afa50 0x7f965c06dd60 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 -- 172.21.15.42:0/1494003640 >> 172.21.15.42:0/1494003640 conn(0x7f965c06bdf0 msgr2=0x7f965c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:19.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 -- 172.21.15.42:0/1494003640 shutdown_connections 2024-05-28T14:11:19.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 -- 172.21.15.42:0/1494003640 wait complete. 2024-05-28T14:11:19.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 Processor -- start 2024-05-28T14:11:19.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 -- start start 2024-05-28T14:11:19.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f965c0afa50 0x7f965c14eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:19.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f965c156a70 0x7f965c0b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:19.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f965c156e50 0x7f965c14f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:19.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f965c160f90 con 0x7f965c0afa50 2024-05-28T14:11:19.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f965c160e10 con 0x7f965c156a70 2024-05-28T14:11:19.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.316+0000 7f9662c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f965c16a530 con 0x7f965c156e50 2024-05-28T14:11:19.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f965be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f965c0afa50 0x7f965c14eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:19.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f965be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f965c0afa50 0x7f965c14eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53282/0 (socket says 172.21.15.42:53282) 2024-05-28T14:11:19.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f965be00640 1 -- 172.21.15.42:0/443883173 learned_addr learned my addr 172.21.15.42:0/443883173 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:19.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f9660c00640 1 --2- 172.21.15.42:0/443883173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f965c156e50 0x7f965c14f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:19.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f9653e00640 1 --2- 172.21.15.42:0/443883173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f965c156a70 0x7f965c0b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:19.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f965be00640 1 -- 172.21.15.42:0/443883173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f965c156e50 msgr2=0x7f965c14f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f965be00640 1 --2- 172.21.15.42:0/443883173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f965c156e50 0x7f965c14f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f965be00640 1 -- 172.21.15.42:0/443883173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f965c156a70 msgr2=0x7f965c0b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f965be00640 1 --2- 172.21.15.42:0/443883173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f965c156a70 0x7f965c0b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f965be00640 1 -- 172.21.15.42:0/443883173 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965c0b0dd0 con 0x7f965c0afa50 2024-05-28T14:11:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f965be00640 1 --2- 172.21.15.42:0/443883173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f965c0afa50 0x7f965c14eeb0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f964c00c420 tx=0x7f964c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f9658c00640 1 -- 172.21.15.42:0/443883173 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c0052d0 con 0x7f965c0afa50 2024-05-28T14:11:19.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f9658c00640 1 -- 172.21.15.42:0/443883173 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964c005470 con 0x7f965c0afa50 2024-05-28T14:11:19.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f9658c00640 1 -- 172.21.15.42:0/443883173 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c007260 con 0x7f965c0afa50 2024-05-28T14:11:19.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f9662c00640 1 -- 172.21.15.42:0/443883173 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965c0b10b0 con 0x7f965c0afa50 2024-05-28T14:11:19.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f9662c00640 1 -- 172.21.15.42:0/443883173 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f965c0b1670 con 0x7f965c0afa50 2024-05-28T14:11:19.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f9662c00640 1 -- 172.21.15.42:0/443883173 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f965c068a90 con 0x7f965c0afa50 2024-05-28T14:11:19.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.320+0000 7f9658c00640 1 -- 172.21.15.42:0/443883173 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f964c007400 con 0x7f965c0afa50 2024-05-28T14:11:19.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.324+0000 7f9658c00640 1 --2- 172.21.15.42:0/443883173 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9638069850 0x7f963806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:19.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.324+0000 7f9658c00640 1 -- 172.21.15.42:0/443883173 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f964c084fb0 con 0x7f965c0afa50 2024-05-28T14:11:19.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.324+0000 7f9653e00640 1 --2- 172.21.15.42:0/443883173 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9638069850 0x7f963806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:19.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.324+0000 7f9653e00640 1 --2- 172.21.15.42:0/443883173 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9638069850 0x7f963806bd10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f96440ae000 tx=0x7f96440a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:19.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.328+0000 7f9658c00640 1 -- 172.21.15.42:0/443883173 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f964c08e020 con 0x7f965c0afa50 2024-05-28T14:11:19.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.492+0000 7f9662c00640 1 -- 172.21.15.42:0/443883173 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f965c0b5160 con 0x7f9638069850 2024-05-28T14:11:19.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.496+0000 7f9658c00640 1 -- 172.21.15.42:0/443883173 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247105 (secure 0 0 0) 0x7f965c0b5160 con 0x7f9638069850 2024-05-28T14:11:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:19.486 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:11:19.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.500+0000 7f9662c00640 1 -- 172.21.15.42:0/443883173 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9638069850 msgr2=0x7f963806bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.500+0000 7f9662c00640 1 --2- 172.21.15.42:0/443883173 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9638069850 0x7f963806bd10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f96440ae000 tx=0x7f96440a7000 comp rx=0 tx=0).stop 2024-05-28T14:11:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.500+0000 7f9662c00640 1 -- 172.21.15.42:0/443883173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f965c0afa50 msgr2=0x7f965c14eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.500+0000 7f9662c00640 1 --2- 172.21.15.42:0/443883173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f965c0afa50 0x7f965c14eeb0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f964c00c420 tx=0x7f964c00c8e0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.500+0000 7f9662c00640 1 -- 172.21.15.42:0/443883173 shutdown_connections 2024-05-28T14:11:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.500+0000 7f9662c00640 1 --2- 172.21.15.42:0/443883173 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9638069850 0x7f963806bd10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.500+0000 7f9662c00640 1 --2- 172.21.15.42:0/443883173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f965c156e50 0x7f965c14f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.500+0000 7f9662c00640 1 --2- 172.21.15.42:0/443883173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f965c156a70 0x7f965c0b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.500+0000 7f9662c00640 1 --2- 172.21.15.42:0/443883173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f965c0afa50 0x7f965c14eeb0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.500+0000 7f9662c00640 1 -- 172.21.15.42:0/443883173 >> 172.21.15.42:0/443883173 conn(0x7f965c06bdf0 msgr2=0x7f965c15b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.500+0000 7f9662c00640 1 -- 172.21.15.42:0/443883173 shutdown_connections 2024-05-28T14:11:19.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.500+0000 7f9662c00640 1 -- 172.21.15.42:0/443883173 wait complete. 2024-05-28T14:11:19.505 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-05-28T14:11:19.139088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":890276,"kb_used_data":11052,"kb_used_omap":12,"kb_used_meta":879091,"kb_avail":754084444,"statfs":{"total":773094113280,"available":772182470656,"internally_reserved":0,"allocated":11317248,"data_stored":5576301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":900189746},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.763808"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399798+0000","last_change":"2024-05-28T14:11:14.399798+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399798+0000","last_undegraded":"2024-05-28T14:11:14.399798+0000","last_fullsized":"2024-05-28T14:11:14.399798+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396400+0000","last_change":"2024-05-28T14:11:14.396400+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396400+0000","last_undegraded":"2024-05-28T14:11:14.396400+0000","last_fullsized":"2024-05-28T14:11:14.396400+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397782+0000","last_change":"2024-05-28T14:11:14.397782+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397782+0000","last_undegraded":"2024-05-28T14:11:14.397782+0000","last_fullsized":"2024-05-28T14:11:14.397782+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396940+0000","last_change":"2024-05-28T14:11:14.396940+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396940+0000","last_undegraded":"2024-05-28T14:11:14.396940+0000","last_fullsized":"2024-05-28T14:11:14.396940+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395541+0000","last_change":"2024-05-28T14:11:14.395541+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395541+0000","last_undegraded":"2024-05-28T14:11:14.395541+0000","last_fullsized":"2024-05-28T14:11:14.395541+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393722+0000","last_change":"2024-05-28T14:11:14.393722+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393722+0000","last_undegraded":"2024-05-28T14:11:14.393722+0000","last_fullsized":"2024-05-28T14:11:14.393722+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393740+0000","last_change":"2024-05-28T14:11:14.393740+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393740+0000","last_undegraded":"2024-05-28T14:11:14.393740+0000","last_fullsized":"2024-05-28T14:11:14.393740+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394974+0000","last_change":"2024-05-28T14:11:14.394974+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394974+0000","last_undegraded":"2024-05-28T14:11:14.394974+0000","last_fullsized":"2024-05-28T14:11:14.394974+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.390899+0000","last_change":"2024-05-28T14:11:14.390899+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.390899+0000","last_undegraded":"2024-05-28T14:11:14.390899+0000","last_fullsized":"2024-05-28T14:11:14.390899+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395187+0000","last_change":"2024-05-28T14:11:14.395187+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395187+0000","last_undegraded":"2024-05-28T14:11:14.395187+0000","last_fullsized":"2024-05-28T14:11:14.395187+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399363+0000","last_change":"2024-05-28T14:11:14.399363+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399363+0000","last_undegraded":"2024-05-28T14:11:14.399363+0000","last_fullsized":"2024-05-28T14:11:14.399363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.389835+0000","last_change":"2024-05-28T14:11:14.389835+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.389835+0000","last_undegraded":"2024-05-28T14:11:14.389835+0000","last_fullsized":"2024-05-28T14:11:14.389835+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398498+0000","last_change":"2024-05-28T14:11:14.398498+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398498+0000","last_undegraded":"2024-05-28T14:11:14.398498+0000","last_fullsized":"2024-05-28T14:11:14.398498+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395077+0000","last_change":"2024-05-28T14:11:14.395077+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395077+0000","last_undegraded":"2024-05-28T14:11:14.395077+0000","last_fullsized":"2024-05-28T14:11:14.395077+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398243+0000","last_change":"2024-05-28T14:11:14.398243+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398243+0000","last_undegraded":"2024-05-28T14:11:14.398243+0000","last_fullsized":"2024-05-28T14:11:14.398243+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397185+0000","last_change":"2024-05-28T14:11:14.397185+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397185+0000","last_undegraded":"2024-05-28T14:11:14.397185+0000","last_fullsized":"2024-05-28T14:11:14.397185+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395430+0000","last_change":"2024-05-28T14:11:14.395430+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395430+0000","last_undegraded":"2024-05-28T14:11:14.395430+0000","last_fullsized":"2024-05-28T14:11:14.395430+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395911+0000","last_change":"2024-05-28T14:11:14.395911+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395911+0000","last_undegraded":"2024-05-28T14:11:14.395911+0000","last_fullsized":"2024-05-28T14:11:14.395911+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392659+0000","last_change":"2024-05-28T14:11:14.392659+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392659+0000","last_undegraded":"2024-05-28T14:11:14.392659+0000","last_fullsized":"2024-05-28T14:11:14.392659+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.389349+0000","last_change":"2024-05-28T14:11:14.389349+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.389349+0000","last_undegraded":"2024-05-28T14:11:14.389349+0000","last_fullsized":"2024-05-28T14:11:14.389349+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.400065+0000","last_change":"2024-05-28T14:11:14.400065+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.400065+0000","last_undegraded":"2024-05-28T14:11:14.400065+0000","last_fullsized":"2024-05-28T14:11:14.400065+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394850+0000","last_change":"2024-05-28T14:11:14.394850+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394850+0000","last_undegraded":"2024-05-28T14:11:14.394850+0000","last_fullsized":"2024-05-28T14:11:14.394850+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397557+0000","last_change":"2024-05-28T14:11:14.397557+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397557+0000","last_undegraded":"2024-05-28T14:11:14.397557+0000","last_fullsized":"2024-05-28T14:11:14.397557+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393602+0000","last_change":"2024-05-28T14:11:14.393602+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393602+0000","last_undegraded":"2024-05-28T14:11:14.393602+0000","last_fullsized":"2024-05-28T14:11:14.393602+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397515+0000","last_change":"2024-05-28T14:11:14.397515+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397515+0000","last_undegraded":"2024-05-28T14:11:14.397515+0000","last_fullsized":"2024-05-28T14:11:14.397515+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395934+0000","last_change":"2024-05-28T14:11:14.395934+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395934+0000","last_undegraded":"2024-05-28T14:11:14.395934+0000","last_fullsized":"2024-05-28T14:11:14.395934+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398870+0000","last_change":"2024-05-28T14:11:14.398870+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398870+0000","last_undegraded":"2024-05-28T14:11:14.398870+0000","last_fullsized":"2024-05-28T14:11:14.398870+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396110+0000","last_change":"2024-05-28T14:11:14.396110+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396110+0000","last_undegraded":"2024-05-28T14:11:14.396110+0000","last_fullsized":"2024-05-28T14:11:14.396110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394155+0000","last_change":"2024-05-28T14:11:14.394155+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394155+0000","last_undegraded":"2024-05-28T14:11:14.394155+0000","last_fullsized":"2024-05-28T14:11:14.394155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394647+0000","last_change":"2024-05-28T14:11:14.394647+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394647+0000","last_undegraded":"2024-05-28T14:11:14.394647+0000","last_fullsized":"2024-05-28T14:11:14.394647+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397797+0000","last_change":"2024-05-28T14:11:14.397797+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397797+0000","last_undegraded":"2024-05-28T14:11:14.397797+0000","last_fullsized":"2024-05-28T14:11:14.397797+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396693+0000","last_change":"2024-05-28T14:11:14.396693+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396693+0000","last_undegraded":"2024-05-28T14:11:14.396693+0000","last_fullsized":"2024-05-28T14:11:14.396693+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395974+0000","last_change":"2024-05-28T14:11:14.395974+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395974+0000","last_undegraded":"2024-05-28T14:11:14.395974+0000","last_fullsized":"2024-05-28T14:11:14.395974+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.406287+0000","last_change":"2024-05-28T14:11:14.403379+0000","last_active":"2024-05-28T14:11:14.406287+0000","last_peered":"2024-05-28T14:11:14.406287+0000","last_clean":"2024-05-28T14:11:14.406287+0000","last_became_active":"2024-05-28T14:11:14.403054+0000","last_became_peered":"2024-05-28T14:11:14.403054+0000","last_unstale":"2024-05-28T14:11:14.406287+0000","last_undegraded":"2024-05-28T14:11:14.406287+0000","last_fullsized":"2024-05-28T14:11:14.406287+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:48.727311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395375+0000","last_change":"2024-05-28T14:11:14.395375+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395375+0000","last_undegraded":"2024-05-28T14:11:14.395375+0000","last_fullsized":"2024-05-28T14:11:14.395375+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.414453+0000","last_change":"2024-05-28T14:11:14.413616+0000","last_active":"2024-05-28T14:11:14.414453+0000","last_peered":"2024-05-28T14:11:14.414453+0000","last_clean":"2024-05-28T14:11:14.414453+0000","last_became_active":"2024-05-28T14:11:14.413317+0000","last_became_peered":"2024-05-28T14:11:14.413317+0000","last_unstale":"2024-05-28T14:11:14.414453+0000","last_undegraded":"2024-05-28T14:11:14.414453+0000","last_fullsized":"2024-05-28T14:11:14.414453+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:53:42.892031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398141+0000","last_change":"2024-05-28T14:11:14.398141+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398141+0000","last_undegraded":"2024-05-28T14:11:14.398141+0000","last_fullsized":"2024-05-28T14:11:14.398141+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410331+0000","last_change":"2024-05-28T14:11:14.408679+0000","last_active":"2024-05-28T14:11:14.410331+0000","last_peered":"2024-05-28T14:11:14.410331+0000","last_clean":"2024-05-28T14:11:14.410331+0000","last_became_active":"2024-05-28T14:11:14.408384+0000","last_became_peered":"2024-05-28T14:11:14.408384+0000","last_unstale":"2024-05-28T14:11:14.410331+0000","last_undegraded":"2024-05-28T14:11:14.410331+0000","last_fullsized":"2024-05-28T14:11:14.410331+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:33:51.292167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.388993+0000","last_change":"2024-05-28T14:11:14.388993+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.388993+0000","last_undegraded":"2024-05-28T14:11:14.388993+0000","last_fullsized":"2024-05-28T14:11:14.388993+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.403137+0000","last_change":"2024-05-28T14:11:14.400435+0000","last_active":"2024-05-28T14:11:14.403137+0000","last_peered":"2024-05-28T14:11:14.403137+0000","last_clean":"2024-05-28T14:11:14.403137+0000","last_became_active":"2024-05-28T14:11:14.400157+0000","last_became_peered":"2024-05-28T14:11:14.400157+0000","last_unstale":"2024-05-28T14:11:14.403137+0000","last_undegraded":"2024-05-28T14:11:14.403137+0000","last_fullsized":"2024-05-28T14:11:14.403137+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:28:22.020830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.390994+0000","last_change":"2024-05-28T14:11:14.390994+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.390994+0000","last_undegraded":"2024-05-28T14:11:14.390994+0000","last_fullsized":"2024-05-28T14:11:14.390994+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415348+0000","last_change":"2024-05-28T14:11:14.412408+0000","last_active":"2024-05-28T14:11:14.415348+0000","last_peered":"2024-05-28T14:11:14.415348+0000","last_clean":"2024-05-28T14:11:14.415348+0000","last_became_active":"2024-05-28T14:11:14.411721+0000","last_became_peered":"2024-05-28T14:11:14.411721+0000","last_unstale":"2024-05-28T14:11:14.415348+0000","last_undegraded":"2024-05-28T14:11:14.415348+0000","last_fullsized":"2024-05-28T14:11:14.415348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:34.789464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393752+0000","last_change":"2024-05-28T14:11:14.393752+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393752+0000","last_undegraded":"2024-05-28T14:11:14.393752+0000","last_fullsized":"2024-05-28T14:11:14.393752+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415331+0000","last_change":"2024-05-28T14:11:14.414364+0000","last_active":"2024-05-28T14:11:14.415331+0000","last_peered":"2024-05-28T14:11:14.415331+0000","last_clean":"2024-05-28T14:11:14.415331+0000","last_became_active":"2024-05-28T14:11:14.413986+0000","last_became_peered":"2024-05-28T14:11:14.413986+0000","last_unstale":"2024-05-28T14:11:14.415331+0000","last_undegraded":"2024-05-28T14:11:14.415331+0000","last_fullsized":"2024-05-28T14:11:14.415331+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:30.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392990+0000","last_change":"2024-05-28T14:11:14.392990+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392990+0000","last_undegraded":"2024-05-28T14:11:14.392990+0000","last_fullsized":"2024-05-28T14:11:14.392990+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.407313+0000","last_change":"2024-05-28T14:11:14.406246+0000","last_active":"2024-05-28T14:11:14.407313+0000","last_peered":"2024-05-28T14:11:14.407313+0000","last_clean":"2024-05-28T14:11:14.407313+0000","last_became_active":"2024-05-28T14:11:14.405729+0000","last_became_peered":"2024-05-28T14:11:14.405729+0000","last_unstale":"2024-05-28T14:11:14.407313+0000","last_undegraded":"2024-05-28T14:11:14.407313+0000","last_fullsized":"2024-05-28T14:11:14.407313+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:54:37.950887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.390220+0000","last_change":"2024-05-28T14:11:14.390220+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.390220+0000","last_undegraded":"2024-05-28T14:11:14.390220+0000","last_fullsized":"2024-05-28T14:11:14.390220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415417+0000","last_change":"2024-05-28T14:11:14.413285+0000","last_active":"2024-05-28T14:11:14.415417+0000","last_peered":"2024-05-28T14:11:14.415417+0000","last_clean":"2024-05-28T14:11:14.415417+0000","last_became_active":"2024-05-28T14:11:14.413010+0000","last_became_peered":"2024-05-28T14:11:14.413010+0000","last_unstale":"2024-05-28T14:11:14.415417+0000","last_undegraded":"2024-05-28T14:11:14.415417+0000","last_fullsized":"2024-05-28T14:11:14.415417+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:44:00.271939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393542+0000","last_change":"2024-05-28T14:11:14.393542+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393542+0000","last_undegraded":"2024-05-28T14:11:14.393542+0000","last_fullsized":"2024-05-28T14:11:14.393542+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.413458+0000","last_change":"2024-05-28T14:11:14.406962+0000","last_active":"2024-05-28T14:11:14.413458+0000","last_peered":"2024-05-28T14:11:14.413458+0000","last_clean":"2024-05-28T14:11:14.413458+0000","last_became_active":"2024-05-28T14:11:14.406694+0000","last_became_peered":"2024-05-28T14:11:14.406694+0000","last_unstale":"2024-05-28T14:11:14.413458+0000","last_undegraded":"2024-05-28T14:11:14.413458+0000","last_fullsized":"2024-05-28T14:11:14.413458+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:56:21.193180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.389820+0000","last_change":"2024-05-28T14:11:14.389820+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.389820+0000","last_undegraded":"2024-05-28T14:11:14.389820+0000","last_fullsized":"2024-05-28T14:11:14.389820+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410625+0000","last_change":"2024-05-28T14:11:14.403779+0000","last_active":"2024-05-28T14:11:14.410625+0000","last_peered":"2024-05-28T14:11:14.410625+0000","last_clean":"2024-05-28T14:11:14.410625+0000","last_became_active":"2024-05-28T14:11:14.403497+0000","last_became_peered":"2024-05-28T14:11:14.403497+0000","last_unstale":"2024-05-28T14:11:14.410625+0000","last_undegraded":"2024-05-28T14:11:14.410625+0000","last_fullsized":"2024-05-28T14:11:14.410625+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:50:50.675765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395418+0000","last_change":"2024-05-28T14:11:14.395418+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395418+0000","last_undegraded":"2024-05-28T14:11:14.395418+0000","last_fullsized":"2024-05-28T14:11:14.395418+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.407898+0000","last_change":"2024-05-28T14:11:14.407272+0000","last_active":"2024-05-28T14:11:14.407898+0000","last_peered":"2024-05-28T14:11:14.407898+0000","last_clean":"2024-05-28T14:11:14.407898+0000","last_became_active":"2024-05-28T14:11:14.406878+0000","last_became_peered":"2024-05-28T14:11:14.406878+0000","last_unstale":"2024-05-28T14:11:14.407898+0000","last_undegraded":"2024-05-28T14:11:14.407898+0000","last_fullsized":"2024-05-28T14:11:14.407898+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:08:12.903518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398379+0000","last_change":"2024-05-28T14:11:14.398379+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398379+0000","last_undegraded":"2024-05-28T14:11:14.398379+0000","last_fullsized":"2024-05-28T14:11:14.398379+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.409199+0000","last_change":"2024-05-28T14:11:14.408531+0000","last_active":"2024-05-28T14:11:14.409199+0000","last_peered":"2024-05-28T14:11:14.409199+0000","last_clean":"2024-05-28T14:11:14.409199+0000","last_became_active":"2024-05-28T14:11:14.408165+0000","last_became_peered":"2024-05-28T14:11:14.408165+0000","last_unstale":"2024-05-28T14:11:14.409199+0000","last_undegraded":"2024-05-28T14:11:14.409199+0000","last_fullsized":"2024-05-28T14:11:14.409199+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:30.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392041+0000","last_change":"2024-05-28T14:11:14.392041+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392041+0000","last_undegraded":"2024-05-28T14:11:14.392041+0000","last_fullsized":"2024-05-28T14:11:14.392041+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.405337+0000","last_change":"2024-05-28T14:11:14.403933+0000","last_active":"2024-05-28T14:11:14.405337+0000","last_peered":"2024-05-28T14:11:14.405337+0000","last_clean":"2024-05-28T14:11:14.405337+0000","last_became_active":"2024-05-28T14:11:14.403689+0000","last_became_peered":"2024-05-28T14:11:14.403689+0000","last_unstale":"2024-05-28T14:11:14.405337+0000","last_undegraded":"2024-05-28T14:11:14.405337+0000","last_fullsized":"2024-05-28T14:11:14.405337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:43:33.888586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.401610+0000","last_change":"2024-05-28T14:11:14.401610+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.401610+0000","last_undegraded":"2024-05-28T14:11:14.401610+0000","last_fullsized":"2024-05-28T14:11:14.401610+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.416582+0000","last_change":"2024-05-28T14:11:14.416004+0000","last_active":"2024-05-28T14:11:14.416582+0000","last_peered":"2024-05-28T14:11:14.416582+0000","last_clean":"2024-05-28T14:11:14.416582+0000","last_became_active":"2024-05-28T14:11:14.415194+0000","last_became_peered":"2024-05-28T14:11:14.415194+0000","last_unstale":"2024-05-28T14:11:14.416582+0000","last_undegraded":"2024-05-28T14:11:14.416582+0000","last_fullsized":"2024-05-28T14:11:14.416582+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:56.646412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.388892+0000","last_change":"2024-05-28T14:11:14.388892+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.388892+0000","last_undegraded":"2024-05-28T14:11:14.388892+0000","last_fullsized":"2024-05-28T14:11:14.388892+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.403799+0000","last_change":"2024-05-28T14:11:14.401589+0000","last_active":"2024-05-28T14:11:14.403799+0000","last_peered":"2024-05-28T14:11:14.403799+0000","last_clean":"2024-05-28T14:11:14.403799+0000","last_became_active":"2024-05-28T14:11:14.401285+0000","last_became_peered":"2024-05-28T14:11:14.401285+0000","last_unstale":"2024-05-28T14:11:14.403799+0000","last_undegraded":"2024-05-28T14:11:14.403799+0000","last_fullsized":"2024-05-28T14:11:14.403799+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391686+0000","last_change":"2024-05-28T14:11:14.391686+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391686+0000","last_undegraded":"2024-05-28T14:11:14.391686+0000","last_fullsized":"2024-05-28T14:11:14.391686+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.413435+0000","last_change":"2024-05-28T14:11:14.408691+0000","last_active":"2024-05-28T14:11:14.413435+0000","last_peered":"2024-05-28T14:11:14.413435+0000","last_clean":"2024-05-28T14:11:14.413435+0000","last_became_active":"2024-05-28T14:11:14.407478+0000","last_became_peered":"2024-05-28T14:11:14.407478+0000","last_unstale":"2024-05-28T14:11:14.413435+0000","last_undegraded":"2024-05-28T14:11:14.413435+0000","last_fullsized":"2024-05-28T14:11:14.413435+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:52.119306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395810+0000","last_change":"2024-05-28T14:11:14.395810+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395810+0000","last_undegraded":"2024-05-28T14:11:14.395810+0000","last_fullsized":"2024-05-28T14:11:14.395810+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.400563+0000","last_change":"2024-05-28T14:11:14.398527+0000","last_active":"2024-05-28T14:11:14.400563+0000","last_peered":"2024-05-28T14:11:14.400563+0000","last_clean":"2024-05-28T14:11:14.400563+0000","last_became_active":"2024-05-28T14:11:14.398239+0000","last_became_peered":"2024-05-28T14:11:14.398239+0000","last_unstale":"2024-05-28T14:11:14.400563+0000","last_undegraded":"2024-05-28T14:11:14.400563+0000","last_fullsized":"2024-05-28T14:11:14.400563+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:41:43.968459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391743+0000","last_change":"2024-05-28T14:11:14.391743+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391743+0000","last_undegraded":"2024-05-28T14:11:14.391743+0000","last_fullsized":"2024-05-28T14:11:14.391743+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.406239+0000","last_change":"2024-05-28T14:11:14.404603+0000","last_active":"2024-05-28T14:11:14.406239+0000","last_peered":"2024-05-28T14:11:14.406239+0000","last_clean":"2024-05-28T14:11:14.406239+0000","last_became_active":"2024-05-28T14:11:14.403710+0000","last_became_peered":"2024-05-28T14:11:14.403710+0000","last_unstale":"2024-05-28T14:11:14.406239+0000","last_undegraded":"2024-05-28T14:11:14.406239+0000","last_fullsized":"2024-05-28T14:11:14.406239+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393759+0000","last_change":"2024-05-28T14:11:14.393759+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393759+0000","last_undegraded":"2024-05-28T14:11:14.393759+0000","last_fullsized":"2024-05-28T14:11:14.393759+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.403113+0000","last_change":"2024-05-28T14:11:14.399470+0000","last_active":"2024-05-28T14:11:14.403113+0000","last_peered":"2024-05-28T14:11:14.403113+0000","last_clean":"2024-05-28T14:11:14.403113+0000","last_became_active":"2024-05-28T14:11:14.397748+0000","last_became_peered":"2024-05-28T14:11:14.397748+0000","last_unstale":"2024-05-28T14:11:14.403113+0000","last_undegraded":"2024-05-28T14:11:14.403113+0000","last_fullsized":"2024-05-28T14:11:14.403113+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:07:58.660765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394208+0000","last_change":"2024-05-28T14:11:14.394208+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394208+0000","last_undegraded":"2024-05-28T14:11:14.394208+0000","last_fullsized":"2024-05-28T14:11:14.394208+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410291+0000","last_change":"2024-05-28T14:11:14.409172+0000","last_active":"2024-05-28T14:11:14.410291+0000","last_peered":"2024-05-28T14:11:14.410291+0000","last_clean":"2024-05-28T14:11:14.410291+0000","last_became_active":"2024-05-28T14:11:14.408866+0000","last_became_peered":"2024-05-28T14:11:14.408866+0000","last_unstale":"2024-05-28T14:11:14.410291+0000","last_undegraded":"2024-05-28T14:11:14.410291+0000","last_fullsized":"2024-05-28T14:11:14.410291+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:37.767057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399596+0000","last_change":"2024-05-28T14:11:14.399596+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399596+0000","last_undegraded":"2024-05-28T14:11:14.399596+0000","last_fullsized":"2024-05-28T14:11:14.399596+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410986+0000","last_change":"2024-05-28T14:11:14.410214+0000","last_active":"2024-05-28T14:11:14.410986+0000","last_peered":"2024-05-28T14:11:14.410986+0000","last_clean":"2024-05-28T14:11:14.410986+0000","last_became_active":"2024-05-28T14:11:14.409906+0000","last_became_peered":"2024-05-28T14:11:14.409906+0000","last_unstale":"2024-05-28T14:11:14.410986+0000","last_undegraded":"2024-05-28T14:11:14.410986+0000","last_fullsized":"2024-05-28T14:11:14.410986+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:41:43.968459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395584+0000","last_change":"2024-05-28T14:11:14.395584+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395584+0000","last_undegraded":"2024-05-28T14:11:14.395584+0000","last_fullsized":"2024-05-28T14:11:14.395584+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.411800+0000","last_change":"2024-05-28T14:11:14.411163+0000","last_active":"2024-05-28T14:11:14.411800+0000","last_peered":"2024-05-28T14:11:14.411800+0000","last_clean":"2024-05-28T14:11:14.411800+0000","last_became_active":"2024-05-28T14:11:14.410761+0000","last_became_peered":"2024-05-28T14:11:14.410761+0000","last_unstale":"2024-05-28T14:11:14.411800+0000","last_undegraded":"2024-05-28T14:11:14.411800+0000","last_fullsized":"2024-05-28T14:11:14.411800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396807+0000","last_change":"2024-05-28T14:11:14.396807+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396807+0000","last_undegraded":"2024-05-28T14:11:14.396807+0000","last_fullsized":"2024-05-28T14:11:14.396807+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.412103+0000","last_change":"2024-05-28T14:11:14.411437+0000","last_active":"2024-05-28T14:11:14.412103+0000","last_peered":"2024-05-28T14:11:14.412103+0000","last_clean":"2024-05-28T14:11:14.412103+0000","last_became_active":"2024-05-28T14:11:14.411081+0000","last_became_peered":"2024-05-28T14:11:14.411081+0000","last_unstale":"2024-05-28T14:11:14.412103+0000","last_undegraded":"2024-05-28T14:11:14.412103+0000","last_fullsized":"2024-05-28T14:11:14.412103+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391572+0000","last_change":"2024-05-28T14:11:14.391572+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391572+0000","last_undegraded":"2024-05-28T14:11:14.391572+0000","last_fullsized":"2024-05-28T14:11:14.391572+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.400032+0000","last_change":"2024-05-28T14:11:14.397391+0000","last_active":"2024-05-28T14:11:14.400032+0000","last_peered":"2024-05-28T14:11:14.400032+0000","last_clean":"2024-05-28T14:11:14.400032+0000","last_became_active":"2024-05-28T14:11:14.397118+0000","last_became_peered":"2024-05-28T14:11:14.397118+0000","last_unstale":"2024-05-28T14:11:14.400032+0000","last_undegraded":"2024-05-28T14:11:14.400032+0000","last_fullsized":"2024-05-28T14:11:14.400032+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:37.767057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399351+0000","last_change":"2024-05-28T14:11:14.399351+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399351+0000","last_undegraded":"2024-05-28T14:11:14.399351+0000","last_fullsized":"2024-05-28T14:11:14.399351+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410413+0000","last_change":"2024-05-28T14:11:14.409506+0000","last_active":"2024-05-28T14:11:14.410413+0000","last_peered":"2024-05-28T14:11:14.410413+0000","last_clean":"2024-05-28T14:11:14.410413+0000","last_became_active":"2024-05-28T14:11:14.407782+0000","last_became_peered":"2024-05-28T14:11:14.407782+0000","last_unstale":"2024-05-28T14:11:14.410413+0000","last_undegraded":"2024-05-28T14:11:14.410413+0000","last_fullsized":"2024-05-28T14:11:14.410413+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:41:43.968459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397667+0000","last_change":"2024-05-28T14:11:14.397667+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397667+0000","last_undegraded":"2024-05-28T14:11:14.397667+0000","last_fullsized":"2024-05-28T14:11:14.397667+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.401431+0000","last_change":"2024-05-28T14:11:14.396514+0000","last_active":"2024-05-28T14:11:14.401431+0000","last_peered":"2024-05-28T14:11:14.401431+0000","last_clean":"2024-05-28T14:11:14.401431+0000","last_became_active":"2024-05-28T14:11:14.396179+0000","last_became_peered":"2024-05-28T14:11:14.396179+0000","last_unstale":"2024-05-28T14:11:14.401431+0000","last_undegraded":"2024-05-28T14:11:14.401431+0000","last_fullsized":"2024-05-28T14:11:14.401431+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:08:51.277949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394850+0000","last_change":"2024-05-28T14:11:14.394850+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394850+0000","last_undegraded":"2024-05-28T14:11:14.394850+0000","last_fullsized":"2024-05-28T14:11:14.394850+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.407146+0000","last_change":"2024-05-28T14:11:14.405416+0000","last_active":"2024-05-28T14:11:14.407146+0000","last_peered":"2024-05-28T14:11:14.407146+0000","last_clean":"2024-05-28T14:11:14.407146+0000","last_became_active":"2024-05-28T14:11:14.405162+0000","last_became_peered":"2024-05-28T14:11:14.405162+0000","last_unstale":"2024-05-28T14:11:14.407146+0000","last_undegraded":"2024-05-28T14:11:14.407146+0000","last_fullsized":"2024-05-28T14:11:14.407146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:30.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393725+0000","last_change":"2024-05-28T14:11:14.393725+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393725+0000","last_undegraded":"2024-05-28T14:11:14.393725+0000","last_fullsized":"2024-05-28T14:11:14.393725+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410190+0000","last_change":"2024-05-28T14:11:14.408550+0000","last_active":"2024-05-28T14:11:14.410190+0000","last_peered":"2024-05-28T14:11:14.410190+0000","last_clean":"2024-05-28T14:11:14.410190+0000","last_became_active":"2024-05-28T14:11:14.408274+0000","last_became_peered":"2024-05-28T14:11:14.408274+0000","last_unstale":"2024-05-28T14:11:14.410190+0000","last_undegraded":"2024-05-28T14:11:14.410190+0000","last_fullsized":"2024-05-28T14:11:14.410190+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:13:32.928101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.388969+0000","last_change":"2024-05-28T14:11:14.388969+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.388969+0000","last_undegraded":"2024-05-28T14:11:14.388969+0000","last_fullsized":"2024-05-28T14:11:14.388969+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415284+0000","last_change":"2024-05-28T14:11:14.414517+0000","last_active":"2024-05-28T14:11:14.415284+0000","last_peered":"2024-05-28T14:11:14.415284+0000","last_clean":"2024-05-28T14:11:14.415284+0000","last_became_active":"2024-05-28T14:11:14.414145+0000","last_became_peered":"2024-05-28T14:11:14.414145+0000","last_unstale":"2024-05-28T14:11:14.415284+0000","last_undegraded":"2024-05-28T14:11:14.415284+0000","last_fullsized":"2024-05-28T14:11:14.415284+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:48:06.702787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391914+0000","last_change":"2024-05-28T14:11:14.391914+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391914+0000","last_undegraded":"2024-05-28T14:11:14.391914+0000","last_fullsized":"2024-05-28T14:11:14.391914+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.413905+0000","last_change":"2024-05-28T14:11:14.413007+0000","last_active":"2024-05-28T14:11:14.413905+0000","last_peered":"2024-05-28T14:11:14.413905+0000","last_clean":"2024-05-28T14:11:14.413905+0000","last_became_active":"2024-05-28T14:11:14.412715+0000","last_became_peered":"2024-05-28T14:11:14.412715+0000","last_unstale":"2024-05-28T14:11:14.413905+0000","last_undegraded":"2024-05-28T14:11:14.413905+0000","last_fullsized":"2024-05-28T14:11:14.413905+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:25:40.425802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396370+0000","last_change":"2024-05-28T14:11:14.396370+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396370+0000","last_undegraded":"2024-05-28T14:11:14.396370+0000","last_fullsized":"2024-05-28T14:11:14.396370+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.409580+0000","last_change":"2024-05-28T14:11:14.407608+0000","last_active":"2024-05-28T14:11:14.409580+0000","last_peered":"2024-05-28T14:11:14.409580+0000","last_clean":"2024-05-28T14:11:14.409580+0000","last_became_active":"2024-05-28T14:11:14.407327+0000","last_became_peered":"2024-05-28T14:11:14.407327+0000","last_unstale":"2024-05-28T14:11:14.409580+0000","last_undegraded":"2024-05-28T14:11:14.409580+0000","last_fullsized":"2024-05-28T14:11:14.409580+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:37:22.556212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392974+0000","last_change":"2024-05-28T14:11:14.392974+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392974+0000","last_undegraded":"2024-05-28T14:11:14.392974+0000","last_fullsized":"2024-05-28T14:11:14.392974+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.412741+0000","last_change":"2024-05-28T14:11:14.412233+0000","last_active":"2024-05-28T14:11:14.412741+0000","last_peered":"2024-05-28T14:11:14.412741+0000","last_clean":"2024-05-28T14:11:14.412741+0000","last_became_active":"2024-05-28T14:11:14.411900+0000","last_became_peered":"2024-05-28T14:11:14.411900+0000","last_unstale":"2024-05-28T14:11:14.412741+0000","last_undegraded":"2024-05-28T14:11:14.412741+0000","last_fullsized":"2024-05-28T14:11:14.412741+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:10:01.780076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":324,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.400029+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:11:14.400029+0000","last_peered":"2024-05-28T14:11:14.400029+0000","last_clean":"2024-05-28T14:11:14.400029+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:11:14.400029+0000","last_undegraded":"2024-05-28T14:11:14.400029+0000","last_fullsized":"2024-05-28T14:11:14.400029+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575007,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127328,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244512,"statfs":{"total":96636764160,"available":96506380288,"internally_reserved":0,"allocated":1855488,"data_stored":1137450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126748,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245092,"statfs":{"total":96636764160,"available":96506974208,"internally_reserved":0,"allocated":1261568,"data_stored":547082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":128514502},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64300,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307540,"statfs":{"total":96636764160,"available":96570920960,"internally_reserved":0,"allocated":1273856,"data_stored":553385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86508,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285332,"statfs":{"total":96636764160,"available":96548179968,"internally_reserved":0,"allocated":1261568,"data_stored":547082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118444,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253396,"statfs":{"total":96636764160,"available":96515477504,"internally_reserved":0,"allocated":1273856,"data_stored":553385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":119994824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148968,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222872,"statfs":{"total":96636764160,"available":96484220928,"internally_reserved":0,"allocated":1273856,"data_stored":553385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130908,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240932,"statfs":{"total":96636764160,"available":96502714368,"internally_reserved":0,"allocated":1261568,"data_stored":547082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87072,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284768,"statfs":{"total":96636764160,"available":96547602432,"internally_reserved":0,"allocated":1855488,"data_stored":1137450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:11:19.565 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:11:19.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe720600640 1 Processor -- start 2024-05-28T14:11:19.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe720600640 1 -- start start 2024-05-28T14:11:19.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe720600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe71806e740 0x7fe7180cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:19.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe720600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7180ce110 0x7fe71809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:19.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe720600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe71806ff10 con 0x7fe71806c940 2024-05-28T14:11:19.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe720600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fe71806eb20 con 0x7fe71806e740 2024-05-28T14:11:19.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe720600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7180701b0 con 0x7fe7180ce110 2024-05-28T14:11:19.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe71e600640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe71806c940 0x7fe718070f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39248/0 (socket says 172.21.15.42:39248) 2024-05-28T14:11:19.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe71e600640 1 -- 172.21.15.42:0/3585752967 learned_addr learned my addr 172.21.15.42:0/3585752967 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:19.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe71dc00640 1 --2- 172.21.15.42:0/3585752967 >> v2:172.21.15.42:3300/0 conn(0x7fe71806e740 0x7fe7180cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:19.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe71d200640 1 --2- 172.21.15.42:0/3585752967 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7180ce110 0x7fe71809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:19.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe71dc00640 1 -- 172.21.15.42:0/3585752967 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7180ce110 msgr2=0x7fe71809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:19.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe71dc00640 1 --2- 172.21.15.42:0/3585752967 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7180ce110 0x7fe71809aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe71dc00640 1 -- 172.21.15.42:0/3585752967 >> v1:172.21.15.42:6789/0 conn(0x7fe71806c940 legacy=0x7fe718070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe71dc00640 1 -- 172.21.15.42:0/3585752967 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe718070490 con 0x7fe71806e740 2024-05-28T14:11:19.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.724+0000 7fe717e00640 1 -- 172.21.15.42:0/3585752967 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 623275506 0 0) 0x7fe71806ff10 con 0x7fe71806c940 2024-05-28T14:11:19.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.728+0000 7fe71dc00640 1 --2- 172.21.15.42:0/3585752967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe71806e740 0x7fe7180cdbd0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0097c0 tx=0x7fe70c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ccba75a69a5a3a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:19.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.728+0000 7fe717e00640 1 -- 172.21.15.42:0/3585752967 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0b2070 con 0x7fe71806e740 2024-05-28T14:11:19.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.728+0000 7fe717e00640 1 -- 172.21.15.42:0/3585752967 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe70c0ad680 con 0x7fe71806e740 2024-05-28T14:11:19.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.728+0000 7fe717e00640 1 -- 172.21.15.42:0/3585752967 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0ada50 con 0x7fe71806e740 2024-05-28T14:11:19.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.728+0000 7fe720600640 1 -- 172.21.15.42:0/3585752967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe71806e740 msgr2=0x7fe7180cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.728+0000 7fe720600640 1 --2- 172.21.15.42:0/3585752967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe71806e740 0x7fe7180cdbd0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0097c0 tx=0x7fe70c01c470 comp rx=0 tx=0).stop 2024-05-28T14:11:19.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.728+0000 7fe720600640 1 -- 172.21.15.42:0/3585752967 shutdown_connections 2024-05-28T14:11:19.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.728+0000 7fe720600640 1 --2- 172.21.15.42:0/3585752967 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7180ce110 0x7fe71809aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.728+0000 7fe720600640 1 --2- 172.21.15.42:0/3585752967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe71806e740 0x7fe7180cdbd0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.728+0000 7fe720600640 1 -- 172.21.15.42:0/3585752967 >> 172.21.15.42:0/3585752967 conn(0x7fe7180b3fa0 msgr2=0x7fe7180b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:19.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.728+0000 7fe720600640 1 -- 172.21.15.42:0/3585752967 shutdown_connections 2024-05-28T14:11:19.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.728+0000 7fe720600640 1 -- 172.21.15.42:0/3585752967 wait complete. 2024-05-28T14:11:19.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe720600640 1 Processor -- start 2024-05-28T14:11:19.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe720600640 1 -- start start 2024-05-28T14:11:19.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe720600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe71806c940 0x7fe7181538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:19.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe720600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe71806e740 0x7fe7180b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:19.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe720600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7180ce110 0x7fe7181f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:19.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe720600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe71815e2a0 con 0x7fe71806c940 2024-05-28T14:11:19.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe720600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe71815e120 con 0x7fe7180ce110 2024-05-28T14:11:19.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe720600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe71815e420 con 0x7fe71806e740 2024-05-28T14:11:19.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe71dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe71806c940 0x7fe7181538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:19.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe71dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe71806c940 0x7fe7181538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53310/0 (socket says 172.21.15.42:53310) 2024-05-28T14:11:19.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe71dc00640 1 -- 172.21.15.42:0/3597068150 learned_addr learned my addr 172.21.15.42:0/3597068150 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:19.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe71d200640 1 --2- 172.21.15.42:0/3597068150 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe71806e740 0x7fe7180b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:19.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe71e600640 1 --2- 172.21.15.42:0/3597068150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7180ce110 0x7fe7181f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:19.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe71dc00640 1 -- 172.21.15.42:0/3597068150 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe71806e740 msgr2=0x7fe7180b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe71dc00640 1 --2- 172.21.15.42:0/3597068150 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe71806e740 0x7fe7180b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe71dc00640 1 -- 172.21.15.42:0/3597068150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7180ce110 msgr2=0x7fe7181f86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe71dc00640 1 --2- 172.21.15.42:0/3597068150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7180ce110 0x7fe7181f86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe71dc00640 1 -- 172.21.15.42:0/3597068150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7181f8be0 con 0x7fe71806c940 2024-05-28T14:11:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe71dc00640 1 --2- 172.21.15.42:0/3597068150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe71806c940 0x7fe7181538e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fe70c01ab20 tx=0x7fe70c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe715600640 1 -- 172.21.15.42:0/3597068150 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0b2040 con 0x7fe71806c940 2024-05-28T14:11:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe715600640 1 -- 172.21.15.42:0/3597068150 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe70c0be070 con 0x7fe71806c940 2024-05-28T14:11:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe720600640 1 -- 172.21.15.42:0/3597068150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7180b3c60 con 0x7fe71806c940 2024-05-28T14:11:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe715600640 1 -- 172.21.15.42:0/3597068150 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0b8420 con 0x7fe71806c940 2024-05-28T14:11:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe720600640 1 -- 172.21.15.42:0/3597068150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7181f8f50 con 0x7fe71806c940 2024-05-28T14:11:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.732+0000 7fe720600640 1 -- 172.21.15.42:0/3597068150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe718068a90 con 0x7fe71806c940 2024-05-28T14:11:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.736+0000 7fe715600640 1 -- 172.21.15.42:0/3597068150 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fe70c0b85c0 con 0x7fe71806c940 2024-05-28T14:11:19.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.736+0000 7fe715600640 1 --2- 172.21.15.42:0/3597068150 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6f8069850 0x7fe6f806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:19.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.736+0000 7fe715600640 1 -- 172.21.15.42:0/3597068150 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fe70c11fc60 con 0x7fe71806c940 2024-05-28T14:11:19.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.736+0000 7fe71d200640 1 --2- 172.21.15.42:0/3597068150 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6f8069850 0x7fe6f806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:19.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.736+0000 7fe71d200640 1 --2- 172.21.15.42:0/3597068150 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6f8069850 0x7fe6f806bd10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe718156820 tx=0x7fe704008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:19.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.740+0000 7fe715600640 1 -- 172.21.15.42:0/3597068150 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe70c016090 con 0x7fe71806c940 2024-05-28T14:11:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.904+0000 7fe720600640 1 -- 172.21.15.42:0/3597068150 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7180cdec0 con 0x7fe6f8069850 2024-05-28T14:11:19.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.908+0000 7fe715600640 1 -- 172.21.15.42:0/3597068150 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247105 (secure 0 0 0) 0x7fe7180cdec0 con 0x7fe6f8069850 2024-05-28T14:11:19.896 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:19.898 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:11:19.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.912+0000 7fe720600640 1 -- 172.21.15.42:0/3597068150 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6f8069850 msgr2=0x7fe6f806bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.912+0000 7fe720600640 1 --2- 172.21.15.42:0/3597068150 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6f8069850 0x7fe6f806bd10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe718156820 tx=0x7fe704008040 comp rx=0 tx=0).stop 2024-05-28T14:11:19.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.912+0000 7fe720600640 1 -- 172.21.15.42:0/3597068150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe71806c940 msgr2=0x7fe7181538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:19.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.912+0000 7fe720600640 1 --2- 172.21.15.42:0/3597068150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe71806c940 0x7fe7181538e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fe70c01ab20 tx=0x7fe70c01acb0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.912+0000 7fe720600640 1 -- 172.21.15.42:0/3597068150 shutdown_connections 2024-05-28T14:11:19.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.912+0000 7fe720600640 1 --2- 172.21.15.42:0/3597068150 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6f8069850 0x7fe6f806bd10 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.912+0000 7fe720600640 1 --2- 172.21.15.42:0/3597068150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7180ce110 0x7fe7181f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.912+0000 7fe720600640 1 --2- 172.21.15.42:0/3597068150 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe71806e740 0x7fe7180b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.912+0000 7fe720600640 1 --2- 172.21.15.42:0/3597068150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe71806c940 0x7fe7181538e0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:19.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.912+0000 7fe720600640 1 -- 172.21.15.42:0/3597068150 >> 172.21.15.42:0/3597068150 conn(0x7fe7180b3fa0 msgr2=0x7fe7181569b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:19.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.912+0000 7fe720600640 1 -- 172.21.15.42:0/3597068150 shutdown_connections 2024-05-28T14:11:19.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:19.912+0000 7fe720600640 1 -- 172.21.15.42:0/3597068150 wait complete. 2024-05-28T14:11:19.917 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-05-28T14:11:19.139088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":890276,"kb_used_data":11052,"kb_used_omap":12,"kb_used_meta":879091,"kb_avail":754084444,"statfs":{"total":773094113280,"available":772182470656,"internally_reserved":0,"allocated":11317248,"data_stored":5576301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":900189746},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.763808"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399798+0000","last_change":"2024-05-28T14:11:14.399798+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399798+0000","last_undegraded":"2024-05-28T14:11:14.399798+0000","last_fullsized":"2024-05-28T14:11:14.399798+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396400+0000","last_change":"2024-05-28T14:11:14.396400+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396400+0000","last_undegraded":"2024-05-28T14:11:14.396400+0000","last_fullsized":"2024-05-28T14:11:14.396400+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397782+0000","last_change":"2024-05-28T14:11:14.397782+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397782+0000","last_undegraded":"2024-05-28T14:11:14.397782+0000","last_fullsized":"2024-05-28T14:11:14.397782+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396940+0000","last_change":"2024-05-28T14:11:14.396940+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396940+0000","last_undegraded":"2024-05-28T14:11:14.396940+0000","last_fullsized":"2024-05-28T14:11:14.396940+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395541+0000","last_change":"2024-05-28T14:11:14.395541+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395541+0000","last_undegraded":"2024-05-28T14:11:14.395541+0000","last_fullsized":"2024-05-28T14:11:14.395541+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393722+0000","last_change":"2024-05-28T14:11:14.393722+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393722+0000","last_undegraded":"2024-05-28T14:11:14.393722+0000","last_fullsized":"2024-05-28T14:11:14.393722+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393740+0000","last_change":"2024-05-28T14:11:14.393740+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393740+0000","last_undegraded":"2024-05-28T14:11:14.393740+0000","last_fullsized":"2024-05-28T14:11:14.393740+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394974+0000","last_change":"2024-05-28T14:11:14.394974+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394974+0000","last_undegraded":"2024-05-28T14:11:14.394974+0000","last_fullsized":"2024-05-28T14:11:14.394974+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.390899+0000","last_change":"2024-05-28T14:11:14.390899+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.390899+0000","last_undegraded":"2024-05-28T14:11:14.390899+0000","last_fullsized":"2024-05-28T14:11:14.390899+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395187+0000","last_change":"2024-05-28T14:11:14.395187+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395187+0000","last_undegraded":"2024-05-28T14:11:14.395187+0000","last_fullsized":"2024-05-28T14:11:14.395187+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399363+0000","last_change":"2024-05-28T14:11:14.399363+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399363+0000","last_undegraded":"2024-05-28T14:11:14.399363+0000","last_fullsized":"2024-05-28T14:11:14.399363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.389835+0000","last_change":"2024-05-28T14:11:14.389835+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.389835+0000","last_undegraded":"2024-05-28T14:11:14.389835+0000","last_fullsized":"2024-05-28T14:11:14.389835+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398498+0000","last_change":"2024-05-28T14:11:14.398498+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398498+0000","last_undegraded":"2024-05-28T14:11:14.398498+0000","last_fullsized":"2024-05-28T14:11:14.398498+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395077+0000","last_change":"2024-05-28T14:11:14.395077+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395077+0000","last_undegraded":"2024-05-28T14:11:14.395077+0000","last_fullsized":"2024-05-28T14:11:14.395077+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398243+0000","last_change":"2024-05-28T14:11:14.398243+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398243+0000","last_undegraded":"2024-05-28T14:11:14.398243+0000","last_fullsized":"2024-05-28T14:11:14.398243+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397185+0000","last_change":"2024-05-28T14:11:14.397185+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397185+0000","last_undegraded":"2024-05-28T14:11:14.397185+0000","last_fullsized":"2024-05-28T14:11:14.397185+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395430+0000","last_change":"2024-05-28T14:11:14.395430+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395430+0000","last_undegraded":"2024-05-28T14:11:14.395430+0000","last_fullsized":"2024-05-28T14:11:14.395430+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395911+0000","last_change":"2024-05-28T14:11:14.395911+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395911+0000","last_undegraded":"2024-05-28T14:11:14.395911+0000","last_fullsized":"2024-05-28T14:11:14.395911+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392659+0000","last_change":"2024-05-28T14:11:14.392659+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392659+0000","last_undegraded":"2024-05-28T14:11:14.392659+0000","last_fullsized":"2024-05-28T14:11:14.392659+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.389349+0000","last_change":"2024-05-28T14:11:14.389349+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.389349+0000","last_undegraded":"2024-05-28T14:11:14.389349+0000","last_fullsized":"2024-05-28T14:11:14.389349+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.400065+0000","last_change":"2024-05-28T14:11:14.400065+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.400065+0000","last_undegraded":"2024-05-28T14:11:14.400065+0000","last_fullsized":"2024-05-28T14:11:14.400065+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394850+0000","last_change":"2024-05-28T14:11:14.394850+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394850+0000","last_undegraded":"2024-05-28T14:11:14.394850+0000","last_fullsized":"2024-05-28T14:11:14.394850+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397557+0000","last_change":"2024-05-28T14:11:14.397557+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397557+0000","last_undegraded":"2024-05-28T14:11:14.397557+0000","last_fullsized":"2024-05-28T14:11:14.397557+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393602+0000","last_change":"2024-05-28T14:11:14.393602+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393602+0000","last_undegraded":"2024-05-28T14:11:14.393602+0000","last_fullsized":"2024-05-28T14:11:14.393602+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397515+0000","last_change":"2024-05-28T14:11:14.397515+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397515+0000","last_undegraded":"2024-05-28T14:11:14.397515+0000","last_fullsized":"2024-05-28T14:11:14.397515+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395934+0000","last_change":"2024-05-28T14:11:14.395934+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395934+0000","last_undegraded":"2024-05-28T14:11:14.395934+0000","last_fullsized":"2024-05-28T14:11:14.395934+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398870+0000","last_change":"2024-05-28T14:11:14.398870+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398870+0000","last_undegraded":"2024-05-28T14:11:14.398870+0000","last_fullsized":"2024-05-28T14:11:14.398870+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396110+0000","last_change":"2024-05-28T14:11:14.396110+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396110+0000","last_undegraded":"2024-05-28T14:11:14.396110+0000","last_fullsized":"2024-05-28T14:11:14.396110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394155+0000","last_change":"2024-05-28T14:11:14.394155+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394155+0000","last_undegraded":"2024-05-28T14:11:14.394155+0000","last_fullsized":"2024-05-28T14:11:14.394155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394647+0000","last_change":"2024-05-28T14:11:14.394647+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394647+0000","last_undegraded":"2024-05-28T14:11:14.394647+0000","last_fullsized":"2024-05-28T14:11:14.394647+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397797+0000","last_change":"2024-05-28T14:11:14.397797+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397797+0000","last_undegraded":"2024-05-28T14:11:14.397797+0000","last_fullsized":"2024-05-28T14:11:14.397797+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396693+0000","last_change":"2024-05-28T14:11:14.396693+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396693+0000","last_undegraded":"2024-05-28T14:11:14.396693+0000","last_fullsized":"2024-05-28T14:11:14.396693+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395974+0000","last_change":"2024-05-28T14:11:14.395974+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395974+0000","last_undegraded":"2024-05-28T14:11:14.395974+0000","last_fullsized":"2024-05-28T14:11:14.395974+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.406287+0000","last_change":"2024-05-28T14:11:14.403379+0000","last_active":"2024-05-28T14:11:14.406287+0000","last_peered":"2024-05-28T14:11:14.406287+0000","last_clean":"2024-05-28T14:11:14.406287+0000","last_became_active":"2024-05-28T14:11:14.403054+0000","last_became_peered":"2024-05-28T14:11:14.403054+0000","last_unstale":"2024-05-28T14:11:14.406287+0000","last_undegraded":"2024-05-28T14:11:14.406287+0000","last_fullsized":"2024-05-28T14:11:14.406287+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:48.727311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395375+0000","last_change":"2024-05-28T14:11:14.395375+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395375+0000","last_undegraded":"2024-05-28T14:11:14.395375+0000","last_fullsized":"2024-05-28T14:11:14.395375+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.414453+0000","last_change":"2024-05-28T14:11:14.413616+0000","last_active":"2024-05-28T14:11:14.414453+0000","last_peered":"2024-05-28T14:11:14.414453+0000","last_clean":"2024-05-28T14:11:14.414453+0000","last_became_active":"2024-05-28T14:11:14.413317+0000","last_became_peered":"2024-05-28T14:11:14.413317+0000","last_unstale":"2024-05-28T14:11:14.414453+0000","last_undegraded":"2024-05-28T14:11:14.414453+0000","last_fullsized":"2024-05-28T14:11:14.414453+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:53:42.892031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398141+0000","last_change":"2024-05-28T14:11:14.398141+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398141+0000","last_undegraded":"2024-05-28T14:11:14.398141+0000","last_fullsized":"2024-05-28T14:11:14.398141+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410331+0000","last_change":"2024-05-28T14:11:14.408679+0000","last_active":"2024-05-28T14:11:14.410331+0000","last_peered":"2024-05-28T14:11:14.410331+0000","last_clean":"2024-05-28T14:11:14.410331+0000","last_became_active":"2024-05-28T14:11:14.408384+0000","last_became_peered":"2024-05-28T14:11:14.408384+0000","last_unstale":"2024-05-28T14:11:14.410331+0000","last_undegraded":"2024-05-28T14:11:14.410331+0000","last_fullsized":"2024-05-28T14:11:14.410331+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:33:51.292167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.388993+0000","last_change":"2024-05-28T14:11:14.388993+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.388993+0000","last_undegraded":"2024-05-28T14:11:14.388993+0000","last_fullsized":"2024-05-28T14:11:14.388993+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.403137+0000","last_change":"2024-05-28T14:11:14.400435+0000","last_active":"2024-05-28T14:11:14.403137+0000","last_peered":"2024-05-28T14:11:14.403137+0000","last_clean":"2024-05-28T14:11:14.403137+0000","last_became_active":"2024-05-28T14:11:14.400157+0000","last_became_peered":"2024-05-28T14:11:14.400157+0000","last_unstale":"2024-05-28T14:11:14.403137+0000","last_undegraded":"2024-05-28T14:11:14.403137+0000","last_fullsized":"2024-05-28T14:11:14.403137+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:28:22.020830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.390994+0000","last_change":"2024-05-28T14:11:14.390994+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.390994+0000","last_undegraded":"2024-05-28T14:11:14.390994+0000","last_fullsized":"2024-05-28T14:11:14.390994+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415348+0000","last_change":"2024-05-28T14:11:14.412408+0000","last_active":"2024-05-28T14:11:14.415348+0000","last_peered":"2024-05-28T14:11:14.415348+0000","last_clean":"2024-05-28T14:11:14.415348+0000","last_became_active":"2024-05-28T14:11:14.411721+0000","last_became_peered":"2024-05-28T14:11:14.411721+0000","last_unstale":"2024-05-28T14:11:14.415348+0000","last_undegraded":"2024-05-28T14:11:14.415348+0000","last_fullsized":"2024-05-28T14:11:14.415348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:34.789464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393752+0000","last_change":"2024-05-28T14:11:14.393752+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393752+0000","last_undegraded":"2024-05-28T14:11:14.393752+0000","last_fullsized":"2024-05-28T14:11:14.393752+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415331+0000","last_change":"2024-05-28T14:11:14.414364+0000","last_active":"2024-05-28T14:11:14.415331+0000","last_peered":"2024-05-28T14:11:14.415331+0000","last_clean":"2024-05-28T14:11:14.415331+0000","last_became_active":"2024-05-28T14:11:14.413986+0000","last_became_peered":"2024-05-28T14:11:14.413986+0000","last_unstale":"2024-05-28T14:11:14.415331+0000","last_undegraded":"2024-05-28T14:11:14.415331+0000","last_fullsized":"2024-05-28T14:11:14.415331+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:30.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392990+0000","last_change":"2024-05-28T14:11:14.392990+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392990+0000","last_undegraded":"2024-05-28T14:11:14.392990+0000","last_fullsized":"2024-05-28T14:11:14.392990+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.407313+0000","last_change":"2024-05-28T14:11:14.406246+0000","last_active":"2024-05-28T14:11:14.407313+0000","last_peered":"2024-05-28T14:11:14.407313+0000","last_clean":"2024-05-28T14:11:14.407313+0000","last_became_active":"2024-05-28T14:11:14.405729+0000","last_became_peered":"2024-05-28T14:11:14.405729+0000","last_unstale":"2024-05-28T14:11:14.407313+0000","last_undegraded":"2024-05-28T14:11:14.407313+0000","last_fullsized":"2024-05-28T14:11:14.407313+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:54:37.950887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.390220+0000","last_change":"2024-05-28T14:11:14.390220+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.390220+0000","last_undegraded":"2024-05-28T14:11:14.390220+0000","last_fullsized":"2024-05-28T14:11:14.390220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415417+0000","last_change":"2024-05-28T14:11:14.413285+0000","last_active":"2024-05-28T14:11:14.415417+0000","last_peered":"2024-05-28T14:11:14.415417+0000","last_clean":"2024-05-28T14:11:14.415417+0000","last_became_active":"2024-05-28T14:11:14.413010+0000","last_became_peered":"2024-05-28T14:11:14.413010+0000","last_unstale":"2024-05-28T14:11:14.415417+0000","last_undegraded":"2024-05-28T14:11:14.415417+0000","last_fullsized":"2024-05-28T14:11:14.415417+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:44:00.271939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393542+0000","last_change":"2024-05-28T14:11:14.393542+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393542+0000","last_undegraded":"2024-05-28T14:11:14.393542+0000","last_fullsized":"2024-05-28T14:11:14.393542+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.413458+0000","last_change":"2024-05-28T14:11:14.406962+0000","last_active":"2024-05-28T14:11:14.413458+0000","last_peered":"2024-05-28T14:11:14.413458+0000","last_clean":"2024-05-28T14:11:14.413458+0000","last_became_active":"2024-05-28T14:11:14.406694+0000","last_became_peered":"2024-05-28T14:11:14.406694+0000","last_unstale":"2024-05-28T14:11:14.413458+0000","last_undegraded":"2024-05-28T14:11:14.413458+0000","last_fullsized":"2024-05-28T14:11:14.413458+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:56:21.193180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.389820+0000","last_change":"2024-05-28T14:11:14.389820+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.389820+0000","last_undegraded":"2024-05-28T14:11:14.389820+0000","last_fullsized":"2024-05-28T14:11:14.389820+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410625+0000","last_change":"2024-05-28T14:11:14.403779+0000","last_active":"2024-05-28T14:11:14.410625+0000","last_peered":"2024-05-28T14:11:14.410625+0000","last_clean":"2024-05-28T14:11:14.410625+0000","last_became_active":"2024-05-28T14:11:14.403497+0000","last_became_peered":"2024-05-28T14:11:14.403497+0000","last_unstale":"2024-05-28T14:11:14.410625+0000","last_undegraded":"2024-05-28T14:11:14.410625+0000","last_fullsized":"2024-05-28T14:11:14.410625+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:50:50.675765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395418+0000","last_change":"2024-05-28T14:11:14.395418+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395418+0000","last_undegraded":"2024-05-28T14:11:14.395418+0000","last_fullsized":"2024-05-28T14:11:14.395418+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.407898+0000","last_change":"2024-05-28T14:11:14.407272+0000","last_active":"2024-05-28T14:11:14.407898+0000","last_peered":"2024-05-28T14:11:14.407898+0000","last_clean":"2024-05-28T14:11:14.407898+0000","last_became_active":"2024-05-28T14:11:14.406878+0000","last_became_peered":"2024-05-28T14:11:14.406878+0000","last_unstale":"2024-05-28T14:11:14.407898+0000","last_undegraded":"2024-05-28T14:11:14.407898+0000","last_fullsized":"2024-05-28T14:11:14.407898+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:08:12.903518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398379+0000","last_change":"2024-05-28T14:11:14.398379+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398379+0000","last_undegraded":"2024-05-28T14:11:14.398379+0000","last_fullsized":"2024-05-28T14:11:14.398379+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.409199+0000","last_change":"2024-05-28T14:11:14.408531+0000","last_active":"2024-05-28T14:11:14.409199+0000","last_peered":"2024-05-28T14:11:14.409199+0000","last_clean":"2024-05-28T14:11:14.409199+0000","last_became_active":"2024-05-28T14:11:14.408165+0000","last_became_peered":"2024-05-28T14:11:14.408165+0000","last_unstale":"2024-05-28T14:11:14.409199+0000","last_undegraded":"2024-05-28T14:11:14.409199+0000","last_fullsized":"2024-05-28T14:11:14.409199+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:30.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392041+0000","last_change":"2024-05-28T14:11:14.392041+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392041+0000","last_undegraded":"2024-05-28T14:11:14.392041+0000","last_fullsized":"2024-05-28T14:11:14.392041+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.405337+0000","last_change":"2024-05-28T14:11:14.403933+0000","last_active":"2024-05-28T14:11:14.405337+0000","last_peered":"2024-05-28T14:11:14.405337+0000","last_clean":"2024-05-28T14:11:14.405337+0000","last_became_active":"2024-05-28T14:11:14.403689+0000","last_became_peered":"2024-05-28T14:11:14.403689+0000","last_unstale":"2024-05-28T14:11:14.405337+0000","last_undegraded":"2024-05-28T14:11:14.405337+0000","last_fullsized":"2024-05-28T14:11:14.405337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:43:33.888586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.401610+0000","last_change":"2024-05-28T14:11:14.401610+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.401610+0000","last_undegraded":"2024-05-28T14:11:14.401610+0000","last_fullsized":"2024-05-28T14:11:14.401610+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.416582+0000","last_change":"2024-05-28T14:11:14.416004+0000","last_active":"2024-05-28T14:11:14.416582+0000","last_peered":"2024-05-28T14:11:14.416582+0000","last_clean":"2024-05-28T14:11:14.416582+0000","last_became_active":"2024-05-28T14:11:14.415194+0000","last_became_peered":"2024-05-28T14:11:14.415194+0000","last_unstale":"2024-05-28T14:11:14.416582+0000","last_undegraded":"2024-05-28T14:11:14.416582+0000","last_fullsized":"2024-05-28T14:11:14.416582+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:56.646412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.388892+0000","last_change":"2024-05-28T14:11:14.388892+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.388892+0000","last_undegraded":"2024-05-28T14:11:14.388892+0000","last_fullsized":"2024-05-28T14:11:14.388892+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.403799+0000","last_change":"2024-05-28T14:11:14.401589+0000","last_active":"2024-05-28T14:11:14.403799+0000","last_peered":"2024-05-28T14:11:14.403799+0000","last_clean":"2024-05-28T14:11:14.403799+0000","last_became_active":"2024-05-28T14:11:14.401285+0000","last_became_peered":"2024-05-28T14:11:14.401285+0000","last_unstale":"2024-05-28T14:11:14.403799+0000","last_undegraded":"2024-05-28T14:11:14.403799+0000","last_fullsized":"2024-05-28T14:11:14.403799+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391686+0000","last_change":"2024-05-28T14:11:14.391686+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391686+0000","last_undegraded":"2024-05-28T14:11:14.391686+0000","last_fullsized":"2024-05-28T14:11:14.391686+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.413435+0000","last_change":"2024-05-28T14:11:14.408691+0000","last_active":"2024-05-28T14:11:14.413435+0000","last_peered":"2024-05-28T14:11:14.413435+0000","last_clean":"2024-05-28T14:11:14.413435+0000","last_became_active":"2024-05-28T14:11:14.407478+0000","last_became_peered":"2024-05-28T14:11:14.407478+0000","last_unstale":"2024-05-28T14:11:14.413435+0000","last_undegraded":"2024-05-28T14:11:14.413435+0000","last_fullsized":"2024-05-28T14:11:14.413435+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:52.119306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395810+0000","last_change":"2024-05-28T14:11:14.395810+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395810+0000","last_undegraded":"2024-05-28T14:11:14.395810+0000","last_fullsized":"2024-05-28T14:11:14.395810+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.400563+0000","last_change":"2024-05-28T14:11:14.398527+0000","last_active":"2024-05-28T14:11:14.400563+0000","last_peered":"2024-05-28T14:11:14.400563+0000","last_clean":"2024-05-28T14:11:14.400563+0000","last_became_active":"2024-05-28T14:11:14.398239+0000","last_became_peered":"2024-05-28T14:11:14.398239+0000","last_unstale":"2024-05-28T14:11:14.400563+0000","last_undegraded":"2024-05-28T14:11:14.400563+0000","last_fullsized":"2024-05-28T14:11:14.400563+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:41:43.968459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391743+0000","last_change":"2024-05-28T14:11:14.391743+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391743+0000","last_undegraded":"2024-05-28T14:11:14.391743+0000","last_fullsized":"2024-05-28T14:11:14.391743+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.406239+0000","last_change":"2024-05-28T14:11:14.404603+0000","last_active":"2024-05-28T14:11:14.406239+0000","last_peered":"2024-05-28T14:11:14.406239+0000","last_clean":"2024-05-28T14:11:14.406239+0000","last_became_active":"2024-05-28T14:11:14.403710+0000","last_became_peered":"2024-05-28T14:11:14.403710+0000","last_unstale":"2024-05-28T14:11:14.406239+0000","last_undegraded":"2024-05-28T14:11:14.406239+0000","last_fullsized":"2024-05-28T14:11:14.406239+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393759+0000","last_change":"2024-05-28T14:11:14.393759+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393759+0000","last_undegraded":"2024-05-28T14:11:14.393759+0000","last_fullsized":"2024-05-28T14:11:14.393759+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.403113+0000","last_change":"2024-05-28T14:11:14.399470+0000","last_active":"2024-05-28T14:11:14.403113+0000","last_peered":"2024-05-28T14:11:14.403113+0000","last_clean":"2024-05-28T14:11:14.403113+0000","last_became_active":"2024-05-28T14:11:14.397748+0000","last_became_peered":"2024-05-28T14:11:14.397748+0000","last_unstale":"2024-05-28T14:11:14.403113+0000","last_undegraded":"2024-05-28T14:11:14.403113+0000","last_fullsized":"2024-05-28T14:11:14.403113+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:07:58.660765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394208+0000","last_change":"2024-05-28T14:11:14.394208+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394208+0000","last_undegraded":"2024-05-28T14:11:14.394208+0000","last_fullsized":"2024-05-28T14:11:14.394208+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410291+0000","last_change":"2024-05-28T14:11:14.409172+0000","last_active":"2024-05-28T14:11:14.410291+0000","last_peered":"2024-05-28T14:11:14.410291+0000","last_clean":"2024-05-28T14:11:14.410291+0000","last_became_active":"2024-05-28T14:11:14.408866+0000","last_became_peered":"2024-05-28T14:11:14.408866+0000","last_unstale":"2024-05-28T14:11:14.410291+0000","last_undegraded":"2024-05-28T14:11:14.410291+0000","last_fullsized":"2024-05-28T14:11:14.410291+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:37.767057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399596+0000","last_change":"2024-05-28T14:11:14.399596+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399596+0000","last_undegraded":"2024-05-28T14:11:14.399596+0000","last_fullsized":"2024-05-28T14:11:14.399596+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410986+0000","last_change":"2024-05-28T14:11:14.410214+0000","last_active":"2024-05-28T14:11:14.410986+0000","last_peered":"2024-05-28T14:11:14.410986+0000","last_clean":"2024-05-28T14:11:14.410986+0000","last_became_active":"2024-05-28T14:11:14.409906+0000","last_became_peered":"2024-05-28T14:11:14.409906+0000","last_unstale":"2024-05-28T14:11:14.410986+0000","last_undegraded":"2024-05-28T14:11:14.410986+0000","last_fullsized":"2024-05-28T14:11:14.410986+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:41:43.968459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395584+0000","last_change":"2024-05-28T14:11:14.395584+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395584+0000","last_undegraded":"2024-05-28T14:11:14.395584+0000","last_fullsized":"2024-05-28T14:11:14.395584+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.411800+0000","last_change":"2024-05-28T14:11:14.411163+0000","last_active":"2024-05-28T14:11:14.411800+0000","last_peered":"2024-05-28T14:11:14.411800+0000","last_clean":"2024-05-28T14:11:14.411800+0000","last_became_active":"2024-05-28T14:11:14.410761+0000","last_became_peered":"2024-05-28T14:11:14.410761+0000","last_unstale":"2024-05-28T14:11:14.411800+0000","last_undegraded":"2024-05-28T14:11:14.411800+0000","last_fullsized":"2024-05-28T14:11:14.411800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396807+0000","last_change":"2024-05-28T14:11:14.396807+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396807+0000","last_undegraded":"2024-05-28T14:11:14.396807+0000","last_fullsized":"2024-05-28T14:11:14.396807+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.412103+0000","last_change":"2024-05-28T14:11:14.411437+0000","last_active":"2024-05-28T14:11:14.412103+0000","last_peered":"2024-05-28T14:11:14.412103+0000","last_clean":"2024-05-28T14:11:14.412103+0000","last_became_active":"2024-05-28T14:11:14.411081+0000","last_became_peered":"2024-05-28T14:11:14.411081+0000","last_unstale":"2024-05-28T14:11:14.412103+0000","last_undegraded":"2024-05-28T14:11:14.412103+0000","last_fullsized":"2024-05-28T14:11:14.412103+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391572+0000","last_change":"2024-05-28T14:11:14.391572+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391572+0000","last_undegraded":"2024-05-28T14:11:14.391572+0000","last_fullsized":"2024-05-28T14:11:14.391572+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.400032+0000","last_change":"2024-05-28T14:11:14.397391+0000","last_active":"2024-05-28T14:11:14.400032+0000","last_peered":"2024-05-28T14:11:14.400032+0000","last_clean":"2024-05-28T14:11:14.400032+0000","last_became_active":"2024-05-28T14:11:14.397118+0000","last_became_peered":"2024-05-28T14:11:14.397118+0000","last_unstale":"2024-05-28T14:11:14.400032+0000","last_undegraded":"2024-05-28T14:11:14.400032+0000","last_fullsized":"2024-05-28T14:11:14.400032+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:37.767057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399351+0000","last_change":"2024-05-28T14:11:14.399351+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399351+0000","last_undegraded":"2024-05-28T14:11:14.399351+0000","last_fullsized":"2024-05-28T14:11:14.399351+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410413+0000","last_change":"2024-05-28T14:11:14.409506+0000","last_active":"2024-05-28T14:11:14.410413+0000","last_peered":"2024-05-28T14:11:14.410413+0000","last_clean":"2024-05-28T14:11:14.410413+0000","last_became_active":"2024-05-28T14:11:14.407782+0000","last_became_peered":"2024-05-28T14:11:14.407782+0000","last_unstale":"2024-05-28T14:11:14.410413+0000","last_undegraded":"2024-05-28T14:11:14.410413+0000","last_fullsized":"2024-05-28T14:11:14.410413+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:41:43.968459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397667+0000","last_change":"2024-05-28T14:11:14.397667+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397667+0000","last_undegraded":"2024-05-28T14:11:14.397667+0000","last_fullsized":"2024-05-28T14:11:14.397667+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.401431+0000","last_change":"2024-05-28T14:11:14.396514+0000","last_active":"2024-05-28T14:11:14.401431+0000","last_peered":"2024-05-28T14:11:14.401431+0000","last_clean":"2024-05-28T14:11:14.401431+0000","last_became_active":"2024-05-28T14:11:14.396179+0000","last_became_peered":"2024-05-28T14:11:14.396179+0000","last_unstale":"2024-05-28T14:11:14.401431+0000","last_undegraded":"2024-05-28T14:11:14.401431+0000","last_fullsized":"2024-05-28T14:11:14.401431+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:08:51.277949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394850+0000","last_change":"2024-05-28T14:11:14.394850+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394850+0000","last_undegraded":"2024-05-28T14:11:14.394850+0000","last_fullsized":"2024-05-28T14:11:14.394850+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.407146+0000","last_change":"2024-05-28T14:11:14.405416+0000","last_active":"2024-05-28T14:11:14.407146+0000","last_peered":"2024-05-28T14:11:14.407146+0000","last_clean":"2024-05-28T14:11:14.407146+0000","last_became_active":"2024-05-28T14:11:14.405162+0000","last_became_peered":"2024-05-28T14:11:14.405162+0000","last_unstale":"2024-05-28T14:11:14.407146+0000","last_undegraded":"2024-05-28T14:11:14.407146+0000","last_fullsized":"2024-05-28T14:11:14.407146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:30.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393725+0000","last_change":"2024-05-28T14:11:14.393725+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393725+0000","last_undegraded":"2024-05-28T14:11:14.393725+0000","last_fullsized":"2024-05-28T14:11:14.393725+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410190+0000","last_change":"2024-05-28T14:11:14.408550+0000","last_active":"2024-05-28T14:11:14.410190+0000","last_peered":"2024-05-28T14:11:14.410190+0000","last_clean":"2024-05-28T14:11:14.410190+0000","last_became_active":"2024-05-28T14:11:14.408274+0000","last_became_peered":"2024-05-28T14:11:14.408274+0000","last_unstale":"2024-05-28T14:11:14.410190+0000","last_undegraded":"2024-05-28T14:11:14.410190+0000","last_fullsized":"2024-05-28T14:11:14.410190+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:13:32.928101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.388969+0000","last_change":"2024-05-28T14:11:14.388969+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.388969+0000","last_undegraded":"2024-05-28T14:11:14.388969+0000","last_fullsized":"2024-05-28T14:11:14.388969+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415284+0000","last_change":"2024-05-28T14:11:14.414517+0000","last_active":"2024-05-28T14:11:14.415284+0000","last_peered":"2024-05-28T14:11:14.415284+0000","last_clean":"2024-05-28T14:11:14.415284+0000","last_became_active":"2024-05-28T14:11:14.414145+0000","last_became_peered":"2024-05-28T14:11:14.414145+0000","last_unstale":"2024-05-28T14:11:14.415284+0000","last_undegraded":"2024-05-28T14:11:14.415284+0000","last_fullsized":"2024-05-28T14:11:14.415284+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:48:06.702787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391914+0000","last_change":"2024-05-28T14:11:14.391914+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391914+0000","last_undegraded":"2024-05-28T14:11:14.391914+0000","last_fullsized":"2024-05-28T14:11:14.391914+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.413905+0000","last_change":"2024-05-28T14:11:14.413007+0000","last_active":"2024-05-28T14:11:14.413905+0000","last_peered":"2024-05-28T14:11:14.413905+0000","last_clean":"2024-05-28T14:11:14.413905+0000","last_became_active":"2024-05-28T14:11:14.412715+0000","last_became_peered":"2024-05-28T14:11:14.412715+0000","last_unstale":"2024-05-28T14:11:14.413905+0000","last_undegraded":"2024-05-28T14:11:14.413905+0000","last_fullsized":"2024-05-28T14:11:14.413905+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:25:40.425802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396370+0000","last_change":"2024-05-28T14:11:14.396370+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396370+0000","last_undegraded":"2024-05-28T14:11:14.396370+0000","last_fullsized":"2024-05-28T14:11:14.396370+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.409580+0000","last_change":"2024-05-28T14:11:14.407608+0000","last_active":"2024-05-28T14:11:14.409580+0000","last_peered":"2024-05-28T14:11:14.409580+0000","last_clean":"2024-05-28T14:11:14.409580+0000","last_became_active":"2024-05-28T14:11:14.407327+0000","last_became_peered":"2024-05-28T14:11:14.407327+0000","last_unstale":"2024-05-28T14:11:14.409580+0000","last_undegraded":"2024-05-28T14:11:14.409580+0000","last_fullsized":"2024-05-28T14:11:14.409580+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:37:22.556212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392974+0000","last_change":"2024-05-28T14:11:14.392974+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392974+0000","last_undegraded":"2024-05-28T14:11:14.392974+0000","last_fullsized":"2024-05-28T14:11:14.392974+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.412741+0000","last_change":"2024-05-28T14:11:14.412233+0000","last_active":"2024-05-28T14:11:14.412741+0000","last_peered":"2024-05-28T14:11:14.412741+0000","last_clean":"2024-05-28T14:11:14.412741+0000","last_became_active":"2024-05-28T14:11:14.411900+0000","last_became_peered":"2024-05-28T14:11:14.411900+0000","last_unstale":"2024-05-28T14:11:14.412741+0000","last_undegraded":"2024-05-28T14:11:14.412741+0000","last_fullsized":"2024-05-28T14:11:14.412741+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:10:01.780076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":324,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.400029+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:11:14.400029+0000","last_peered":"2024-05-28T14:11:14.400029+0000","last_clean":"2024-05-28T14:11:14.400029+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:11:14.400029+0000","last_undegraded":"2024-05-28T14:11:14.400029+0000","last_fullsized":"2024-05-28T14:11:14.400029+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575007,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127328,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244512,"statfs":{"total":96636764160,"available":96506380288,"internally_reserved":0,"allocated":1855488,"data_stored":1137450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126748,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245092,"statfs":{"total":96636764160,"available":96506974208,"internally_reserved":0,"allocated":1261568,"data_stored":547082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":128514502},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64300,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307540,"statfs":{"total":96636764160,"available":96570920960,"internally_reserved":0,"allocated":1273856,"data_stored":553385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86508,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285332,"statfs":{"total":96636764160,"available":96548179968,"internally_reserved":0,"allocated":1261568,"data_stored":547082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118444,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253396,"statfs":{"total":96636764160,"available":96515477504,"internally_reserved":0,"allocated":1273856,"data_stored":553385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":119994824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148968,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222872,"statfs":{"total":96636764160,"available":96484220928,"internally_reserved":0,"allocated":1273856,"data_stored":553385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130908,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240932,"statfs":{"total":96636764160,"available":96502714368,"internally_reserved":0,"allocated":1261568,"data_stored":547082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87072,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284768,"statfs":{"total":96636764160,"available":96547602432,"internally_reserved":0,"allocated":1855488,"data_stored":1137450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:11:19.921 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:11:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb17fa00640 1 Processor -- start 2024-05-28T14:11:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb17fa00640 1 -- start start 2024-05-28T14:11:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb17fa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb17815c990 0x7fb1780d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb17fa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb17809bbd0 0x7fb17809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb17fa00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1780cd300 con 0x7fb1780d06a0 2024-05-28T14:11:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb17fa00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb17815cd70 con 0x7fb17815c990 2024-05-28T14:11:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb17fa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb1780b63f0 con 0x7fb17809bbd0 2024-05-28T14:11:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb17d000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb17815c990 0x7fb1780d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb17d000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb17815c990 0x7fb1780d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53322/0 (socket says 172.21.15.42:53322) 2024-05-28T14:11:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb17d000640 1 -- 172.21.15.42:0/2006329662 learned_addr learned my addr 172.21.15.42:0/2006329662 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb16fe00640 1 --2- 172.21.15.42:0/2006329662 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb17809bbd0 0x7fb17809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb17d000640 1 -- 172.21.15.42:0/2006329662 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb17809bbd0 msgr2=0x7fb17809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb17d000640 1 --2- 172.21.15.42:0/2006329662 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb17809bbd0 0x7fb17809aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb17d000640 1 -- 172.21.15.42:0/2006329662 >> v1:172.21.15.63:6789/0 conn(0x7fb1780d06a0 legacy=0x7fb178160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb17d000640 1 -- 172.21.15.42:0/2006329662 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1780cd5e0 con 0x7fb17815c990 2024-05-28T14:11:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.120+0000 7fb16fe00640 1 --2- 172.21.15.42:0/2006329662 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb17809bbd0 0x7fb17809aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.124+0000 7fb17d000640 1 --2- 172.21.15.42:0/2006329662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb17815c990 0x7fb1780d0160 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fb168009870 tx=0x7fb16801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af6c38cb1672213 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.124+0000 7fb177e00640 1 -- 172.21.15.42:0/2006329662 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1680ab640 con 0x7fb17815c990 2024-05-28T14:11:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.124+0000 7fb177e00640 1 -- 172.21.15.42:0/2006329662 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1680b0070 con 0x7fb17815c990 2024-05-28T14:11:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.124+0000 7fb177e00640 1 -- 172.21.15.42:0/2006329662 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1680aba00 con 0x7fb17815c990 2024-05-28T14:11:20.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.124+0000 7fb17fa00640 1 -- 172.21.15.42:0/2006329662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb17815c990 msgr2=0x7fb1780d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.124+0000 7fb17fa00640 1 --2- 172.21.15.42:0/2006329662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb17815c990 0x7fb1780d0160 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fb168009870 tx=0x7fb16801c530 comp rx=0 tx=0).stop 2024-05-28T14:11:20.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.124+0000 7fb17fa00640 1 -- 172.21.15.42:0/2006329662 shutdown_connections 2024-05-28T14:11:20.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.124+0000 7fb17fa00640 1 --2- 172.21.15.42:0/2006329662 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb17809bbd0 0x7fb17809aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.124+0000 7fb17fa00640 1 --2- 172.21.15.42:0/2006329662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb17815c990 0x7fb1780d0160 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.124+0000 7fb17fa00640 1 -- 172.21.15.42:0/2006329662 >> 172.21.15.42:0/2006329662 conn(0x7fb1780b4080 msgr2=0x7fb1780b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:20.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.124+0000 7fb17fa00640 1 -- 172.21.15.42:0/2006329662 shutdown_connections 2024-05-28T14:11:20.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.124+0000 7fb17fa00640 1 -- 172.21.15.42:0/2006329662 wait complete. 2024-05-28T14:11:20.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb17fa00640 1 Processor -- start 2024-05-28T14:11:20.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb17fa00640 1 -- start start 2024-05-28T14:11:20.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb17fa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1780d06a0 0x7fb178151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb17fa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb17815c990 0x7fb178152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb17fa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1780c9fb0 0x7fb1781527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb17fa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb1780ce3a0 con 0x7fb17815c990 2024-05-28T14:11:20.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb17fa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb178169740 con 0x7fb1780c9fb0 2024-05-28T14:11:20.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb17fa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb178150990 con 0x7fb1780d06a0 2024-05-28T14:11:20.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb16fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb17815c990 0x7fb178152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb16fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb17815c990 0x7fb178152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53332/0 (socket says 172.21.15.42:53332) 2024-05-28T14:11:20.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb16fe00640 1 -- 172.21.15.42:0/1076730306 learned_addr learned my addr 172.21.15.42:0/1076730306 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:20.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb17d000640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1780d06a0 0x7fb178151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb17da00640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1780c9fb0 0x7fb1781527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb16fe00640 1 -- 172.21.15.42:0/1076730306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1780d06a0 msgr2=0x7fb178151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb16fe00640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1780d06a0 0x7fb178151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb16fe00640 1 -- 172.21.15.42:0/1076730306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1780c9fb0 msgr2=0x7fb1781527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb16fe00640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1780c9fb0 0x7fb1781527a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb16fe00640 1 -- 172.21.15.42:0/1076730306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb178150c70 con 0x7fb17815c990 2024-05-28T14:11:20.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb17d000640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1780d06a0 0x7fb178151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:20.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb16fe00640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb17815c990 0x7fb178152260 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb16000e810 tx=0x7fb16000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:20.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb175600640 1 -- 172.21.15.42:0/1076730306 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160017070 con 0x7fb17815c990 2024-05-28T14:11:20.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb175600640 1 -- 172.21.15.42:0/1076730306 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16000c8d0 con 0x7fb17815c990 2024-05-28T14:11:20.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb175600640 1 -- 172.21.15.42:0/1076730306 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16000f040 con 0x7fb17815c990 2024-05-28T14:11:20.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb17fa00640 1 -- 172.21.15.42:0/1076730306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb178150f50 con 0x7fb17815c990 2024-05-28T14:11:20.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb17fa00640 1 -- 172.21.15.42:0/1076730306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1780cb800 con 0x7fb17815c990 2024-05-28T14:11:20.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.128+0000 7fb17fa00640 1 -- 172.21.15.42:0/1076730306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb178068a90 con 0x7fb17815c990 2024-05-28T14:11:20.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.132+0000 7fb175600640 1 -- 172.21.15.42:0/1076730306 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fb16001a070 con 0x7fb17815c990 2024-05-28T14:11:20.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.132+0000 7fb175600640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb154069850 0x7fb15406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.132+0000 7fb175600640 1 -- 172.21.15.42:0/1076730306 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fb1600852d0 con 0x7fb17815c990 2024-05-28T14:11:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.132+0000 7fb17d000640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb154069850 0x7fb15406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.132+0000 7fb17d000640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb154069850 0x7fb15406bd10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb1680af000 tx=0x7fb1680a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:20.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.136+0000 7fb175600640 1 -- 172.21.15.42:0/1076730306 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb1600572e0 con 0x7fb17815c990 2024-05-28T14:11:20.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.296+0000 7fb17fa00640 1 -- 172.21.15.42:0/1076730306 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1780b4ce0 con 0x7fb154069850 2024-05-28T14:11:20.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.304+0000 7fb175600640 1 -- 172.21.15.42:0/1076730306 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247105 (secure 0 0 0) 0x7fb1780b4ce0 con 0x7fb154069850 2024-05-28T14:11:20.289 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:20.290 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:11:20.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.304+0000 7fb17fa00640 1 -- 172.21.15.42:0/1076730306 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb154069850 msgr2=0x7fb15406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.304+0000 7fb17fa00640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb154069850 0x7fb15406bd10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb1680af000 tx=0x7fb1680a7000 comp rx=0 tx=0).stop 2024-05-28T14:11:20.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.304+0000 7fb17fa00640 1 -- 172.21.15.42:0/1076730306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb17815c990 msgr2=0x7fb178152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.304+0000 7fb17fa00640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb17815c990 0x7fb178152260 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb16000e810 tx=0x7fb16000ecd0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.304+0000 7fb17fa00640 1 -- 172.21.15.42:0/1076730306 shutdown_connections 2024-05-28T14:11:20.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.304+0000 7fb17fa00640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb154069850 0x7fb15406bd10 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.304+0000 7fb17fa00640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1780c9fb0 0x7fb1781527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.304+0000 7fb17fa00640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb17815c990 0x7fb178152260 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.304+0000 7fb17fa00640 1 --2- 172.21.15.42:0/1076730306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1780d06a0 0x7fb178151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.304+0000 7fb17fa00640 1 -- 172.21.15.42:0/1076730306 >> 172.21.15.42:0/1076730306 conn(0x7fb1780b4080 msgr2=0x7fb1780b7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:20.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.304+0000 7fb17fa00640 1 -- 172.21.15.42:0/1076730306 shutdown_connections 2024-05-28T14:11:20.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.308+0000 7fb17fa00640 1 -- 172.21.15.42:0/1076730306 wait complete. 2024-05-28T14:11:20.311 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-05-28T14:11:19.139088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":890276,"kb_used_data":11052,"kb_used_omap":12,"kb_used_meta":879091,"kb_avail":754084444,"statfs":{"total":773094113280,"available":772182470656,"internally_reserved":0,"allocated":11317248,"data_stored":5576301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":900189746},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.763808"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399798+0000","last_change":"2024-05-28T14:11:14.399798+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399798+0000","last_undegraded":"2024-05-28T14:11:14.399798+0000","last_fullsized":"2024-05-28T14:11:14.399798+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396400+0000","last_change":"2024-05-28T14:11:14.396400+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396400+0000","last_undegraded":"2024-05-28T14:11:14.396400+0000","last_fullsized":"2024-05-28T14:11:14.396400+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397782+0000","last_change":"2024-05-28T14:11:14.397782+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397782+0000","last_undegraded":"2024-05-28T14:11:14.397782+0000","last_fullsized":"2024-05-28T14:11:14.397782+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396940+0000","last_change":"2024-05-28T14:11:14.396940+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396940+0000","last_undegraded":"2024-05-28T14:11:14.396940+0000","last_fullsized":"2024-05-28T14:11:14.396940+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395541+0000","last_change":"2024-05-28T14:11:14.395541+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395541+0000","last_undegraded":"2024-05-28T14:11:14.395541+0000","last_fullsized":"2024-05-28T14:11:14.395541+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393722+0000","last_change":"2024-05-28T14:11:14.393722+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393722+0000","last_undegraded":"2024-05-28T14:11:14.393722+0000","last_fullsized":"2024-05-28T14:11:14.393722+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393740+0000","last_change":"2024-05-28T14:11:14.393740+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393740+0000","last_undegraded":"2024-05-28T14:11:14.393740+0000","last_fullsized":"2024-05-28T14:11:14.393740+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394974+0000","last_change":"2024-05-28T14:11:14.394974+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394974+0000","last_undegraded":"2024-05-28T14:11:14.394974+0000","last_fullsized":"2024-05-28T14:11:14.394974+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.390899+0000","last_change":"2024-05-28T14:11:14.390899+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.390899+0000","last_undegraded":"2024-05-28T14:11:14.390899+0000","last_fullsized":"2024-05-28T14:11:14.390899+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395187+0000","last_change":"2024-05-28T14:11:14.395187+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395187+0000","last_undegraded":"2024-05-28T14:11:14.395187+0000","last_fullsized":"2024-05-28T14:11:14.395187+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399363+0000","last_change":"2024-05-28T14:11:14.399363+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399363+0000","last_undegraded":"2024-05-28T14:11:14.399363+0000","last_fullsized":"2024-05-28T14:11:14.399363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.389835+0000","last_change":"2024-05-28T14:11:14.389835+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.389835+0000","last_undegraded":"2024-05-28T14:11:14.389835+0000","last_fullsized":"2024-05-28T14:11:14.389835+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398498+0000","last_change":"2024-05-28T14:11:14.398498+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398498+0000","last_undegraded":"2024-05-28T14:11:14.398498+0000","last_fullsized":"2024-05-28T14:11:14.398498+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395077+0000","last_change":"2024-05-28T14:11:14.395077+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395077+0000","last_undegraded":"2024-05-28T14:11:14.395077+0000","last_fullsized":"2024-05-28T14:11:14.395077+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398243+0000","last_change":"2024-05-28T14:11:14.398243+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398243+0000","last_undegraded":"2024-05-28T14:11:14.398243+0000","last_fullsized":"2024-05-28T14:11:14.398243+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397185+0000","last_change":"2024-05-28T14:11:14.397185+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397185+0000","last_undegraded":"2024-05-28T14:11:14.397185+0000","last_fullsized":"2024-05-28T14:11:14.397185+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395430+0000","last_change":"2024-05-28T14:11:14.395430+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395430+0000","last_undegraded":"2024-05-28T14:11:14.395430+0000","last_fullsized":"2024-05-28T14:11:14.395430+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395911+0000","last_change":"2024-05-28T14:11:14.395911+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395911+0000","last_undegraded":"2024-05-28T14:11:14.395911+0000","last_fullsized":"2024-05-28T14:11:14.395911+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392659+0000","last_change":"2024-05-28T14:11:14.392659+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392659+0000","last_undegraded":"2024-05-28T14:11:14.392659+0000","last_fullsized":"2024-05-28T14:11:14.392659+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.389349+0000","last_change":"2024-05-28T14:11:14.389349+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.389349+0000","last_undegraded":"2024-05-28T14:11:14.389349+0000","last_fullsized":"2024-05-28T14:11:14.389349+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.400065+0000","last_change":"2024-05-28T14:11:14.400065+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.400065+0000","last_undegraded":"2024-05-28T14:11:14.400065+0000","last_fullsized":"2024-05-28T14:11:14.400065+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394850+0000","last_change":"2024-05-28T14:11:14.394850+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394850+0000","last_undegraded":"2024-05-28T14:11:14.394850+0000","last_fullsized":"2024-05-28T14:11:14.394850+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397557+0000","last_change":"2024-05-28T14:11:14.397557+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397557+0000","last_undegraded":"2024-05-28T14:11:14.397557+0000","last_fullsized":"2024-05-28T14:11:14.397557+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393602+0000","last_change":"2024-05-28T14:11:14.393602+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393602+0000","last_undegraded":"2024-05-28T14:11:14.393602+0000","last_fullsized":"2024-05-28T14:11:14.393602+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397515+0000","last_change":"2024-05-28T14:11:14.397515+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397515+0000","last_undegraded":"2024-05-28T14:11:14.397515+0000","last_fullsized":"2024-05-28T14:11:14.397515+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395934+0000","last_change":"2024-05-28T14:11:14.395934+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395934+0000","last_undegraded":"2024-05-28T14:11:14.395934+0000","last_fullsized":"2024-05-28T14:11:14.395934+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398870+0000","last_change":"2024-05-28T14:11:14.398870+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398870+0000","last_undegraded":"2024-05-28T14:11:14.398870+0000","last_fullsized":"2024-05-28T14:11:14.398870+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396110+0000","last_change":"2024-05-28T14:11:14.396110+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396110+0000","last_undegraded":"2024-05-28T14:11:14.396110+0000","last_fullsized":"2024-05-28T14:11:14.396110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394155+0000","last_change":"2024-05-28T14:11:14.394155+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394155+0000","last_undegraded":"2024-05-28T14:11:14.394155+0000","last_fullsized":"2024-05-28T14:11:14.394155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394647+0000","last_change":"2024-05-28T14:11:14.394647+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394647+0000","last_undegraded":"2024-05-28T14:11:14.394647+0000","last_fullsized":"2024-05-28T14:11:14.394647+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397797+0000","last_change":"2024-05-28T14:11:14.397797+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397797+0000","last_undegraded":"2024-05-28T14:11:14.397797+0000","last_fullsized":"2024-05-28T14:11:14.397797+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396693+0000","last_change":"2024-05-28T14:11:14.396693+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396693+0000","last_undegraded":"2024-05-28T14:11:14.396693+0000","last_fullsized":"2024-05-28T14:11:14.396693+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395974+0000","last_change":"2024-05-28T14:11:14.395974+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395974+0000","last_undegraded":"2024-05-28T14:11:14.395974+0000","last_fullsized":"2024-05-28T14:11:14.395974+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.406287+0000","last_change":"2024-05-28T14:11:14.403379+0000","last_active":"2024-05-28T14:11:14.406287+0000","last_peered":"2024-05-28T14:11:14.406287+0000","last_clean":"2024-05-28T14:11:14.406287+0000","last_became_active":"2024-05-28T14:11:14.403054+0000","last_became_peered":"2024-05-28T14:11:14.403054+0000","last_unstale":"2024-05-28T14:11:14.406287+0000","last_undegraded":"2024-05-28T14:11:14.406287+0000","last_fullsized":"2024-05-28T14:11:14.406287+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:48.727311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395375+0000","last_change":"2024-05-28T14:11:14.395375+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395375+0000","last_undegraded":"2024-05-28T14:11:14.395375+0000","last_fullsized":"2024-05-28T14:11:14.395375+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.414453+0000","last_change":"2024-05-28T14:11:14.413616+0000","last_active":"2024-05-28T14:11:14.414453+0000","last_peered":"2024-05-28T14:11:14.414453+0000","last_clean":"2024-05-28T14:11:14.414453+0000","last_became_active":"2024-05-28T14:11:14.413317+0000","last_became_peered":"2024-05-28T14:11:14.413317+0000","last_unstale":"2024-05-28T14:11:14.414453+0000","last_undegraded":"2024-05-28T14:11:14.414453+0000","last_fullsized":"2024-05-28T14:11:14.414453+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:53:42.892031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398141+0000","last_change":"2024-05-28T14:11:14.398141+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398141+0000","last_undegraded":"2024-05-28T14:11:14.398141+0000","last_fullsized":"2024-05-28T14:11:14.398141+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410331+0000","last_change":"2024-05-28T14:11:14.408679+0000","last_active":"2024-05-28T14:11:14.410331+0000","last_peered":"2024-05-28T14:11:14.410331+0000","last_clean":"2024-05-28T14:11:14.410331+0000","last_became_active":"2024-05-28T14:11:14.408384+0000","last_became_peered":"2024-05-28T14:11:14.408384+0000","last_unstale":"2024-05-28T14:11:14.410331+0000","last_undegraded":"2024-05-28T14:11:14.410331+0000","last_fullsized":"2024-05-28T14:11:14.410331+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:33:51.292167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.388993+0000","last_change":"2024-05-28T14:11:14.388993+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.388993+0000","last_undegraded":"2024-05-28T14:11:14.388993+0000","last_fullsized":"2024-05-28T14:11:14.388993+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.403137+0000","last_change":"2024-05-28T14:11:14.400435+0000","last_active":"2024-05-28T14:11:14.403137+0000","last_peered":"2024-05-28T14:11:14.403137+0000","last_clean":"2024-05-28T14:11:14.403137+0000","last_became_active":"2024-05-28T14:11:14.400157+0000","last_became_peered":"2024-05-28T14:11:14.400157+0000","last_unstale":"2024-05-28T14:11:14.403137+0000","last_undegraded":"2024-05-28T14:11:14.403137+0000","last_fullsized":"2024-05-28T14:11:14.403137+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:28:22.020830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.390994+0000","last_change":"2024-05-28T14:11:14.390994+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.390994+0000","last_undegraded":"2024-05-28T14:11:14.390994+0000","last_fullsized":"2024-05-28T14:11:14.390994+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415348+0000","last_change":"2024-05-28T14:11:14.412408+0000","last_active":"2024-05-28T14:11:14.415348+0000","last_peered":"2024-05-28T14:11:14.415348+0000","last_clean":"2024-05-28T14:11:14.415348+0000","last_became_active":"2024-05-28T14:11:14.411721+0000","last_became_peered":"2024-05-28T14:11:14.411721+0000","last_unstale":"2024-05-28T14:11:14.415348+0000","last_undegraded":"2024-05-28T14:11:14.415348+0000","last_fullsized":"2024-05-28T14:11:14.415348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:34.789464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393752+0000","last_change":"2024-05-28T14:11:14.393752+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393752+0000","last_undegraded":"2024-05-28T14:11:14.393752+0000","last_fullsized":"2024-05-28T14:11:14.393752+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415331+0000","last_change":"2024-05-28T14:11:14.414364+0000","last_active":"2024-05-28T14:11:14.415331+0000","last_peered":"2024-05-28T14:11:14.415331+0000","last_clean":"2024-05-28T14:11:14.415331+0000","last_became_active":"2024-05-28T14:11:14.413986+0000","last_became_peered":"2024-05-28T14:11:14.413986+0000","last_unstale":"2024-05-28T14:11:14.415331+0000","last_undegraded":"2024-05-28T14:11:14.415331+0000","last_fullsized":"2024-05-28T14:11:14.415331+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:30.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392990+0000","last_change":"2024-05-28T14:11:14.392990+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392990+0000","last_undegraded":"2024-05-28T14:11:14.392990+0000","last_fullsized":"2024-05-28T14:11:14.392990+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.407313+0000","last_change":"2024-05-28T14:11:14.406246+0000","last_active":"2024-05-28T14:11:14.407313+0000","last_peered":"2024-05-28T14:11:14.407313+0000","last_clean":"2024-05-28T14:11:14.407313+0000","last_became_active":"2024-05-28T14:11:14.405729+0000","last_became_peered":"2024-05-28T14:11:14.405729+0000","last_unstale":"2024-05-28T14:11:14.407313+0000","last_undegraded":"2024-05-28T14:11:14.407313+0000","last_fullsized":"2024-05-28T14:11:14.407313+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:54:37.950887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.390220+0000","last_change":"2024-05-28T14:11:14.390220+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.390220+0000","last_undegraded":"2024-05-28T14:11:14.390220+0000","last_fullsized":"2024-05-28T14:11:14.390220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415417+0000","last_change":"2024-05-28T14:11:14.413285+0000","last_active":"2024-05-28T14:11:14.415417+0000","last_peered":"2024-05-28T14:11:14.415417+0000","last_clean":"2024-05-28T14:11:14.415417+0000","last_became_active":"2024-05-28T14:11:14.413010+0000","last_became_peered":"2024-05-28T14:11:14.413010+0000","last_unstale":"2024-05-28T14:11:14.415417+0000","last_undegraded":"2024-05-28T14:11:14.415417+0000","last_fullsized":"2024-05-28T14:11:14.415417+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:44:00.271939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393542+0000","last_change":"2024-05-28T14:11:14.393542+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393542+0000","last_undegraded":"2024-05-28T14:11:14.393542+0000","last_fullsized":"2024-05-28T14:11:14.393542+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.413458+0000","last_change":"2024-05-28T14:11:14.406962+0000","last_active":"2024-05-28T14:11:14.413458+0000","last_peered":"2024-05-28T14:11:14.413458+0000","last_clean":"2024-05-28T14:11:14.413458+0000","last_became_active":"2024-05-28T14:11:14.406694+0000","last_became_peered":"2024-05-28T14:11:14.406694+0000","last_unstale":"2024-05-28T14:11:14.413458+0000","last_undegraded":"2024-05-28T14:11:14.413458+0000","last_fullsized":"2024-05-28T14:11:14.413458+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:56:21.193180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.389820+0000","last_change":"2024-05-28T14:11:14.389820+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.389820+0000","last_undegraded":"2024-05-28T14:11:14.389820+0000","last_fullsized":"2024-05-28T14:11:14.389820+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410625+0000","last_change":"2024-05-28T14:11:14.403779+0000","last_active":"2024-05-28T14:11:14.410625+0000","last_peered":"2024-05-28T14:11:14.410625+0000","last_clean":"2024-05-28T14:11:14.410625+0000","last_became_active":"2024-05-28T14:11:14.403497+0000","last_became_peered":"2024-05-28T14:11:14.403497+0000","last_unstale":"2024-05-28T14:11:14.410625+0000","last_undegraded":"2024-05-28T14:11:14.410625+0000","last_fullsized":"2024-05-28T14:11:14.410625+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:50:50.675765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395418+0000","last_change":"2024-05-28T14:11:14.395418+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395418+0000","last_undegraded":"2024-05-28T14:11:14.395418+0000","last_fullsized":"2024-05-28T14:11:14.395418+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.407898+0000","last_change":"2024-05-28T14:11:14.407272+0000","last_active":"2024-05-28T14:11:14.407898+0000","last_peered":"2024-05-28T14:11:14.407898+0000","last_clean":"2024-05-28T14:11:14.407898+0000","last_became_active":"2024-05-28T14:11:14.406878+0000","last_became_peered":"2024-05-28T14:11:14.406878+0000","last_unstale":"2024-05-28T14:11:14.407898+0000","last_undegraded":"2024-05-28T14:11:14.407898+0000","last_fullsized":"2024-05-28T14:11:14.407898+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:08:12.903518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398379+0000","last_change":"2024-05-28T14:11:14.398379+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398379+0000","last_undegraded":"2024-05-28T14:11:14.398379+0000","last_fullsized":"2024-05-28T14:11:14.398379+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.409199+0000","last_change":"2024-05-28T14:11:14.408531+0000","last_active":"2024-05-28T14:11:14.409199+0000","last_peered":"2024-05-28T14:11:14.409199+0000","last_clean":"2024-05-28T14:11:14.409199+0000","last_became_active":"2024-05-28T14:11:14.408165+0000","last_became_peered":"2024-05-28T14:11:14.408165+0000","last_unstale":"2024-05-28T14:11:14.409199+0000","last_undegraded":"2024-05-28T14:11:14.409199+0000","last_fullsized":"2024-05-28T14:11:14.409199+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:30.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392041+0000","last_change":"2024-05-28T14:11:14.392041+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392041+0000","last_undegraded":"2024-05-28T14:11:14.392041+0000","last_fullsized":"2024-05-28T14:11:14.392041+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.405337+0000","last_change":"2024-05-28T14:11:14.403933+0000","last_active":"2024-05-28T14:11:14.405337+0000","last_peered":"2024-05-28T14:11:14.405337+0000","last_clean":"2024-05-28T14:11:14.405337+0000","last_became_active":"2024-05-28T14:11:14.403689+0000","last_became_peered":"2024-05-28T14:11:14.403689+0000","last_unstale":"2024-05-28T14:11:14.405337+0000","last_undegraded":"2024-05-28T14:11:14.405337+0000","last_fullsized":"2024-05-28T14:11:14.405337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:43:33.888586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.401610+0000","last_change":"2024-05-28T14:11:14.401610+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.401610+0000","last_undegraded":"2024-05-28T14:11:14.401610+0000","last_fullsized":"2024-05-28T14:11:14.401610+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.416582+0000","last_change":"2024-05-28T14:11:14.416004+0000","last_active":"2024-05-28T14:11:14.416582+0000","last_peered":"2024-05-28T14:11:14.416582+0000","last_clean":"2024-05-28T14:11:14.416582+0000","last_became_active":"2024-05-28T14:11:14.415194+0000","last_became_peered":"2024-05-28T14:11:14.415194+0000","last_unstale":"2024-05-28T14:11:14.416582+0000","last_undegraded":"2024-05-28T14:11:14.416582+0000","last_fullsized":"2024-05-28T14:11:14.416582+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:56.646412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.388892+0000","last_change":"2024-05-28T14:11:14.388892+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.388892+0000","last_undegraded":"2024-05-28T14:11:14.388892+0000","last_fullsized":"2024-05-28T14:11:14.388892+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.403799+0000","last_change":"2024-05-28T14:11:14.401589+0000","last_active":"2024-05-28T14:11:14.403799+0000","last_peered":"2024-05-28T14:11:14.403799+0000","last_clean":"2024-05-28T14:11:14.403799+0000","last_became_active":"2024-05-28T14:11:14.401285+0000","last_became_peered":"2024-05-28T14:11:14.401285+0000","last_unstale":"2024-05-28T14:11:14.403799+0000","last_undegraded":"2024-05-28T14:11:14.403799+0000","last_fullsized":"2024-05-28T14:11:14.403799+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391686+0000","last_change":"2024-05-28T14:11:14.391686+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391686+0000","last_undegraded":"2024-05-28T14:11:14.391686+0000","last_fullsized":"2024-05-28T14:11:14.391686+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.413435+0000","last_change":"2024-05-28T14:11:14.408691+0000","last_active":"2024-05-28T14:11:14.413435+0000","last_peered":"2024-05-28T14:11:14.413435+0000","last_clean":"2024-05-28T14:11:14.413435+0000","last_became_active":"2024-05-28T14:11:14.407478+0000","last_became_peered":"2024-05-28T14:11:14.407478+0000","last_unstale":"2024-05-28T14:11:14.413435+0000","last_undegraded":"2024-05-28T14:11:14.413435+0000","last_fullsized":"2024-05-28T14:11:14.413435+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:52.119306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395810+0000","last_change":"2024-05-28T14:11:14.395810+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395810+0000","last_undegraded":"2024-05-28T14:11:14.395810+0000","last_fullsized":"2024-05-28T14:11:14.395810+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.400563+0000","last_change":"2024-05-28T14:11:14.398527+0000","last_active":"2024-05-28T14:11:14.400563+0000","last_peered":"2024-05-28T14:11:14.400563+0000","last_clean":"2024-05-28T14:11:14.400563+0000","last_became_active":"2024-05-28T14:11:14.398239+0000","last_became_peered":"2024-05-28T14:11:14.398239+0000","last_unstale":"2024-05-28T14:11:14.400563+0000","last_undegraded":"2024-05-28T14:11:14.400563+0000","last_fullsized":"2024-05-28T14:11:14.400563+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:41:43.968459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391743+0000","last_change":"2024-05-28T14:11:14.391743+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391743+0000","last_undegraded":"2024-05-28T14:11:14.391743+0000","last_fullsized":"2024-05-28T14:11:14.391743+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.406239+0000","last_change":"2024-05-28T14:11:14.404603+0000","last_active":"2024-05-28T14:11:14.406239+0000","last_peered":"2024-05-28T14:11:14.406239+0000","last_clean":"2024-05-28T14:11:14.406239+0000","last_became_active":"2024-05-28T14:11:14.403710+0000","last_became_peered":"2024-05-28T14:11:14.403710+0000","last_unstale":"2024-05-28T14:11:14.406239+0000","last_undegraded":"2024-05-28T14:11:14.406239+0000","last_fullsized":"2024-05-28T14:11:14.406239+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393759+0000","last_change":"2024-05-28T14:11:14.393759+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393759+0000","last_undegraded":"2024-05-28T14:11:14.393759+0000","last_fullsized":"2024-05-28T14:11:14.393759+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.403113+0000","last_change":"2024-05-28T14:11:14.399470+0000","last_active":"2024-05-28T14:11:14.403113+0000","last_peered":"2024-05-28T14:11:14.403113+0000","last_clean":"2024-05-28T14:11:14.403113+0000","last_became_active":"2024-05-28T14:11:14.397748+0000","last_became_peered":"2024-05-28T14:11:14.397748+0000","last_unstale":"2024-05-28T14:11:14.403113+0000","last_undegraded":"2024-05-28T14:11:14.403113+0000","last_fullsized":"2024-05-28T14:11:14.403113+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:07:58.660765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394208+0000","last_change":"2024-05-28T14:11:14.394208+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394208+0000","last_undegraded":"2024-05-28T14:11:14.394208+0000","last_fullsized":"2024-05-28T14:11:14.394208+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410291+0000","last_change":"2024-05-28T14:11:14.409172+0000","last_active":"2024-05-28T14:11:14.410291+0000","last_peered":"2024-05-28T14:11:14.410291+0000","last_clean":"2024-05-28T14:11:14.410291+0000","last_became_active":"2024-05-28T14:11:14.408866+0000","last_became_peered":"2024-05-28T14:11:14.408866+0000","last_unstale":"2024-05-28T14:11:14.410291+0000","last_undegraded":"2024-05-28T14:11:14.410291+0000","last_fullsized":"2024-05-28T14:11:14.410291+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:37.767057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399596+0000","last_change":"2024-05-28T14:11:14.399596+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399596+0000","last_undegraded":"2024-05-28T14:11:14.399596+0000","last_fullsized":"2024-05-28T14:11:14.399596+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410986+0000","last_change":"2024-05-28T14:11:14.410214+0000","last_active":"2024-05-28T14:11:14.410986+0000","last_peered":"2024-05-28T14:11:14.410986+0000","last_clean":"2024-05-28T14:11:14.410986+0000","last_became_active":"2024-05-28T14:11:14.409906+0000","last_became_peered":"2024-05-28T14:11:14.409906+0000","last_unstale":"2024-05-28T14:11:14.410986+0000","last_undegraded":"2024-05-28T14:11:14.410986+0000","last_fullsized":"2024-05-28T14:11:14.410986+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:41:43.968459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395584+0000","last_change":"2024-05-28T14:11:14.395584+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395584+0000","last_undegraded":"2024-05-28T14:11:14.395584+0000","last_fullsized":"2024-05-28T14:11:14.395584+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.411800+0000","last_change":"2024-05-28T14:11:14.411163+0000","last_active":"2024-05-28T14:11:14.411800+0000","last_peered":"2024-05-28T14:11:14.411800+0000","last_clean":"2024-05-28T14:11:14.411800+0000","last_became_active":"2024-05-28T14:11:14.410761+0000","last_became_peered":"2024-05-28T14:11:14.410761+0000","last_unstale":"2024-05-28T14:11:14.411800+0000","last_undegraded":"2024-05-28T14:11:14.411800+0000","last_fullsized":"2024-05-28T14:11:14.411800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396807+0000","last_change":"2024-05-28T14:11:14.396807+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396807+0000","last_undegraded":"2024-05-28T14:11:14.396807+0000","last_fullsized":"2024-05-28T14:11:14.396807+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.412103+0000","last_change":"2024-05-28T14:11:14.411437+0000","last_active":"2024-05-28T14:11:14.412103+0000","last_peered":"2024-05-28T14:11:14.412103+0000","last_clean":"2024-05-28T14:11:14.412103+0000","last_became_active":"2024-05-28T14:11:14.411081+0000","last_became_peered":"2024-05-28T14:11:14.411081+0000","last_unstale":"2024-05-28T14:11:14.412103+0000","last_undegraded":"2024-05-28T14:11:14.412103+0000","last_fullsized":"2024-05-28T14:11:14.412103+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391572+0000","last_change":"2024-05-28T14:11:14.391572+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391572+0000","last_undegraded":"2024-05-28T14:11:14.391572+0000","last_fullsized":"2024-05-28T14:11:14.391572+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.400032+0000","last_change":"2024-05-28T14:11:14.397391+0000","last_active":"2024-05-28T14:11:14.400032+0000","last_peered":"2024-05-28T14:11:14.400032+0000","last_clean":"2024-05-28T14:11:14.400032+0000","last_became_active":"2024-05-28T14:11:14.397118+0000","last_became_peered":"2024-05-28T14:11:14.397118+0000","last_unstale":"2024-05-28T14:11:14.400032+0000","last_undegraded":"2024-05-28T14:11:14.400032+0000","last_fullsized":"2024-05-28T14:11:14.400032+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:37.767057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399351+0000","last_change":"2024-05-28T14:11:14.399351+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399351+0000","last_undegraded":"2024-05-28T14:11:14.399351+0000","last_fullsized":"2024-05-28T14:11:14.399351+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410413+0000","last_change":"2024-05-28T14:11:14.409506+0000","last_active":"2024-05-28T14:11:14.410413+0000","last_peered":"2024-05-28T14:11:14.410413+0000","last_clean":"2024-05-28T14:11:14.410413+0000","last_became_active":"2024-05-28T14:11:14.407782+0000","last_became_peered":"2024-05-28T14:11:14.407782+0000","last_unstale":"2024-05-28T14:11:14.410413+0000","last_undegraded":"2024-05-28T14:11:14.410413+0000","last_fullsized":"2024-05-28T14:11:14.410413+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:41:43.968459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397667+0000","last_change":"2024-05-28T14:11:14.397667+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397667+0000","last_undegraded":"2024-05-28T14:11:14.397667+0000","last_fullsized":"2024-05-28T14:11:14.397667+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.401431+0000","last_change":"2024-05-28T14:11:14.396514+0000","last_active":"2024-05-28T14:11:14.401431+0000","last_peered":"2024-05-28T14:11:14.401431+0000","last_clean":"2024-05-28T14:11:14.401431+0000","last_became_active":"2024-05-28T14:11:14.396179+0000","last_became_peered":"2024-05-28T14:11:14.396179+0000","last_unstale":"2024-05-28T14:11:14.401431+0000","last_undegraded":"2024-05-28T14:11:14.401431+0000","last_fullsized":"2024-05-28T14:11:14.401431+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:08:51.277949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394850+0000","last_change":"2024-05-28T14:11:14.394850+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394850+0000","last_undegraded":"2024-05-28T14:11:14.394850+0000","last_fullsized":"2024-05-28T14:11:14.394850+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.407146+0000","last_change":"2024-05-28T14:11:14.405416+0000","last_active":"2024-05-28T14:11:14.407146+0000","last_peered":"2024-05-28T14:11:14.407146+0000","last_clean":"2024-05-28T14:11:14.407146+0000","last_became_active":"2024-05-28T14:11:14.405162+0000","last_became_peered":"2024-05-28T14:11:14.405162+0000","last_unstale":"2024-05-28T14:11:14.407146+0000","last_undegraded":"2024-05-28T14:11:14.407146+0000","last_fullsized":"2024-05-28T14:11:14.407146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:30.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393725+0000","last_change":"2024-05-28T14:11:14.393725+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393725+0000","last_undegraded":"2024-05-28T14:11:14.393725+0000","last_fullsized":"2024-05-28T14:11:14.393725+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410190+0000","last_change":"2024-05-28T14:11:14.408550+0000","last_active":"2024-05-28T14:11:14.410190+0000","last_peered":"2024-05-28T14:11:14.410190+0000","last_clean":"2024-05-28T14:11:14.410190+0000","last_became_active":"2024-05-28T14:11:14.408274+0000","last_became_peered":"2024-05-28T14:11:14.408274+0000","last_unstale":"2024-05-28T14:11:14.410190+0000","last_undegraded":"2024-05-28T14:11:14.410190+0000","last_fullsized":"2024-05-28T14:11:14.410190+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:13:32.928101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.388969+0000","last_change":"2024-05-28T14:11:14.388969+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.388969+0000","last_undegraded":"2024-05-28T14:11:14.388969+0000","last_fullsized":"2024-05-28T14:11:14.388969+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415284+0000","last_change":"2024-05-28T14:11:14.414517+0000","last_active":"2024-05-28T14:11:14.415284+0000","last_peered":"2024-05-28T14:11:14.415284+0000","last_clean":"2024-05-28T14:11:14.415284+0000","last_became_active":"2024-05-28T14:11:14.414145+0000","last_became_peered":"2024-05-28T14:11:14.414145+0000","last_unstale":"2024-05-28T14:11:14.415284+0000","last_undegraded":"2024-05-28T14:11:14.415284+0000","last_fullsized":"2024-05-28T14:11:14.415284+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:48:06.702787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391914+0000","last_change":"2024-05-28T14:11:14.391914+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391914+0000","last_undegraded":"2024-05-28T14:11:14.391914+0000","last_fullsized":"2024-05-28T14:11:14.391914+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.413905+0000","last_change":"2024-05-28T14:11:14.413007+0000","last_active":"2024-05-28T14:11:14.413905+0000","last_peered":"2024-05-28T14:11:14.413905+0000","last_clean":"2024-05-28T14:11:14.413905+0000","last_became_active":"2024-05-28T14:11:14.412715+0000","last_became_peered":"2024-05-28T14:11:14.412715+0000","last_unstale":"2024-05-28T14:11:14.413905+0000","last_undegraded":"2024-05-28T14:11:14.413905+0000","last_fullsized":"2024-05-28T14:11:14.413905+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:25:40.425802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396370+0000","last_change":"2024-05-28T14:11:14.396370+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396370+0000","last_undegraded":"2024-05-28T14:11:14.396370+0000","last_fullsized":"2024-05-28T14:11:14.396370+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.409580+0000","last_change":"2024-05-28T14:11:14.407608+0000","last_active":"2024-05-28T14:11:14.409580+0000","last_peered":"2024-05-28T14:11:14.409580+0000","last_clean":"2024-05-28T14:11:14.409580+0000","last_became_active":"2024-05-28T14:11:14.407327+0000","last_became_peered":"2024-05-28T14:11:14.407327+0000","last_unstale":"2024-05-28T14:11:14.409580+0000","last_undegraded":"2024-05-28T14:11:14.409580+0000","last_fullsized":"2024-05-28T14:11:14.409580+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:37:22.556212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392974+0000","last_change":"2024-05-28T14:11:14.392974+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392974+0000","last_undegraded":"2024-05-28T14:11:14.392974+0000","last_fullsized":"2024-05-28T14:11:14.392974+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.412741+0000","last_change":"2024-05-28T14:11:14.412233+0000","last_active":"2024-05-28T14:11:14.412741+0000","last_peered":"2024-05-28T14:11:14.412741+0000","last_clean":"2024-05-28T14:11:14.412741+0000","last_became_active":"2024-05-28T14:11:14.411900+0000","last_became_peered":"2024-05-28T14:11:14.411900+0000","last_unstale":"2024-05-28T14:11:14.412741+0000","last_undegraded":"2024-05-28T14:11:14.412741+0000","last_fullsized":"2024-05-28T14:11:14.412741+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:10:01.780076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":324,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.400029+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:11:14.400029+0000","last_peered":"2024-05-28T14:11:14.400029+0000","last_clean":"2024-05-28T14:11:14.400029+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:11:14.400029+0000","last_undegraded":"2024-05-28T14:11:14.400029+0000","last_fullsized":"2024-05-28T14:11:14.400029+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575007,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127328,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244512,"statfs":{"total":96636764160,"available":96506380288,"internally_reserved":0,"allocated":1855488,"data_stored":1137450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126748,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245092,"statfs":{"total":96636764160,"available":96506974208,"internally_reserved":0,"allocated":1261568,"data_stored":547082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":128514502},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64300,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307540,"statfs":{"total":96636764160,"available":96570920960,"internally_reserved":0,"allocated":1273856,"data_stored":553385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86508,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285332,"statfs":{"total":96636764160,"available":96548179968,"internally_reserved":0,"allocated":1261568,"data_stored":547082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118444,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253396,"statfs":{"total":96636764160,"available":96515477504,"internally_reserved":0,"allocated":1273856,"data_stored":553385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":119994824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148968,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222872,"statfs":{"total":96636764160,"available":96484220928,"internally_reserved":0,"allocated":1273856,"data_stored":553385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130908,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240932,"statfs":{"total":96636764160,"available":96502714368,"internally_reserved":0,"allocated":1261568,"data_stored":547082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87072,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284768,"statfs":{"total":96636764160,"available":96547602432,"internally_reserved":0,"allocated":1855488,"data_stored":1137450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:11:20.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2024-05-28T14:11:20.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.399798+0000', 'last_change': '2024-05-28T14:11:14.399798+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.399798+0000', 'last_undegraded': '2024-05-28T14:11:14.399798+0000', 'last_fullsized': '2024-05-28T14:11:14.399798+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:11:20.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2024-05-28T14:11:20.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.396400+0000', 'last_change': '2024-05-28T14:11:14.396400+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.396400+0000', 'last_undegraded': '2024-05-28T14:11:14.396400+0000', 'last_fullsized': '2024-05-28T14:11:14.396400+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:11:20.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2024-05-28T14:11:20.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.397782+0000', 'last_change': '2024-05-28T14:11:14.397782+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.397782+0000', 'last_undegraded': '2024-05-28T14:11:14.397782+0000', 'last_fullsized': '2024-05-28T14:11:14.397782+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:11:20.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2024-05-28T14:11:20.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.396940+0000', 'last_change': '2024-05-28T14:11:14.396940+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.396940+0000', 'last_undegraded': '2024-05-28T14:11:14.396940+0000', 'last_fullsized': '2024-05-28T14:11:14.396940+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:11:20.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2024-05-28T14:11:20.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.395541+0000', 'last_change': '2024-05-28T14:11:14.395541+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.395541+0000', 'last_undegraded': '2024-05-28T14:11:14.395541+0000', 'last_fullsized': '2024-05-28T14:11:14.395541+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:11:20.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2024-05-28T14:11:20.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.393722+0000', 'last_change': '2024-05-28T14:11:14.393722+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.393722+0000', 'last_undegraded': '2024-05-28T14:11:14.393722+0000', 'last_fullsized': '2024-05-28T14:11:14.393722+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:11:20.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2024-05-28T14:11:20.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.393740+0000', 'last_change': '2024-05-28T14:11:14.393740+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.393740+0000', 'last_undegraded': '2024-05-28T14:11:14.393740+0000', 'last_fullsized': '2024-05-28T14:11:14.393740+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:11:20.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2024-05-28T14:11:20.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.394974+0000', 'last_change': '2024-05-28T14:11:14.394974+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.394974+0000', 'last_undegraded': '2024-05-28T14:11:14.394974+0000', 'last_fullsized': '2024-05-28T14:11:14.394974+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:11:20.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2024-05-28T14:11:20.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.390899+0000', 'last_change': '2024-05-28T14:11:14.390899+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.390899+0000', 'last_undegraded': '2024-05-28T14:11:14.390899+0000', 'last_fullsized': '2024-05-28T14:11:14.390899+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:11:20.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2024-05-28T14:11:20.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.395187+0000', 'last_change': '2024-05-28T14:11:14.395187+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.395187+0000', 'last_undegraded': '2024-05-28T14:11:14.395187+0000', 'last_fullsized': '2024-05-28T14:11:14.395187+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:11:20.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2024-05-28T14:11:20.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.399363+0000', 'last_change': '2024-05-28T14:11:14.399363+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.399363+0000', 'last_undegraded': '2024-05-28T14:11:14.399363+0000', 'last_fullsized': '2024-05-28T14:11:14.399363+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:11:20.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2024-05-28T14:11:20.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.389835+0000', 'last_change': '2024-05-28T14:11:14.389835+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.389835+0000', 'last_undegraded': '2024-05-28T14:11:14.389835+0000', 'last_fullsized': '2024-05-28T14:11:14.389835+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:11:20.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2024-05-28T14:11:20.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.398498+0000', 'last_change': '2024-05-28T14:11:14.398498+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.398498+0000', 'last_undegraded': '2024-05-28T14:11:14.398498+0000', 'last_fullsized': '2024-05-28T14:11:14.398498+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:11:20.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2024-05-28T14:11:20.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.395077+0000', 'last_change': '2024-05-28T14:11:14.395077+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.395077+0000', 'last_undegraded': '2024-05-28T14:11:14.395077+0000', 'last_fullsized': '2024-05-28T14:11:14.395077+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:11:20.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2024-05-28T14:11:20.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.398243+0000', 'last_change': '2024-05-28T14:11:14.398243+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.398243+0000', 'last_undegraded': '2024-05-28T14:11:14.398243+0000', 'last_fullsized': '2024-05-28T14:11:14.398243+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:11:20.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2024-05-28T14:11:20.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.397185+0000', 'last_change': '2024-05-28T14:11:14.397185+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.397185+0000', 'last_undegraded': '2024-05-28T14:11:14.397185+0000', 'last_fullsized': '2024-05-28T14:11:14.397185+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:11:20.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2024-05-28T14:11:20.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.395430+0000', 'last_change': '2024-05-28T14:11:14.395430+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.395430+0000', 'last_undegraded': '2024-05-28T14:11:14.395430+0000', 'last_fullsized': '2024-05-28T14:11:14.395430+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:11:20.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2024-05-28T14:11:20.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.395911+0000', 'last_change': '2024-05-28T14:11:14.395911+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.395911+0000', 'last_undegraded': '2024-05-28T14:11:14.395911+0000', 'last_fullsized': '2024-05-28T14:11:14.395911+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:11:20.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2024-05-28T14:11:20.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.392659+0000', 'last_change': '2024-05-28T14:11:14.392659+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.392659+0000', 'last_undegraded': '2024-05-28T14:11:14.392659+0000', 'last_fullsized': '2024-05-28T14:11:14.392659+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:11:20.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2024-05-28T14:11:20.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.389349+0000', 'last_change': '2024-05-28T14:11:14.389349+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.389349+0000', 'last_undegraded': '2024-05-28T14:11:14.389349+0000', 'last_fullsized': '2024-05-28T14:11:14.389349+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:11:20.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2024-05-28T14:11:20.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.400065+0000', 'last_change': '2024-05-28T14:11:14.400065+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.400065+0000', 'last_undegraded': '2024-05-28T14:11:14.400065+0000', 'last_fullsized': '2024-05-28T14:11:14.400065+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:11:20.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2024-05-28T14:11:20.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.394850+0000', 'last_change': '2024-05-28T14:11:14.394850+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.394850+0000', 'last_undegraded': '2024-05-28T14:11:14.394850+0000', 'last_fullsized': '2024-05-28T14:11:14.394850+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:11:20.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2024-05-28T14:11:20.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.397557+0000', 'last_change': '2024-05-28T14:11:14.397557+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.397557+0000', 'last_undegraded': '2024-05-28T14:11:14.397557+0000', 'last_fullsized': '2024-05-28T14:11:14.397557+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:11:20.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2024-05-28T14:11:20.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.393602+0000', 'last_change': '2024-05-28T14:11:14.393602+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.393602+0000', 'last_undegraded': '2024-05-28T14:11:14.393602+0000', 'last_fullsized': '2024-05-28T14:11:14.393602+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:11:20.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2024-05-28T14:11:20.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.397515+0000', 'last_change': '2024-05-28T14:11:14.397515+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.397515+0000', 'last_undegraded': '2024-05-28T14:11:14.397515+0000', 'last_fullsized': '2024-05-28T14:11:14.397515+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:11:20.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2024-05-28T14:11:20.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.395934+0000', 'last_change': '2024-05-28T14:11:14.395934+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.395934+0000', 'last_undegraded': '2024-05-28T14:11:14.395934+0000', 'last_fullsized': '2024-05-28T14:11:14.395934+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:11:20.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2024-05-28T14:11:20.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.398870+0000', 'last_change': '2024-05-28T14:11:14.398870+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.398870+0000', 'last_undegraded': '2024-05-28T14:11:14.398870+0000', 'last_fullsized': '2024-05-28T14:11:14.398870+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:11:20.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2024-05-28T14:11:20.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.396110+0000', 'last_change': '2024-05-28T14:11:14.396110+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.396110+0000', 'last_undegraded': '2024-05-28T14:11:14.396110+0000', 'last_fullsized': '2024-05-28T14:11:14.396110+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:11:20.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2024-05-28T14:11:20.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.394155+0000', 'last_change': '2024-05-28T14:11:14.394155+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.394155+0000', 'last_undegraded': '2024-05-28T14:11:14.394155+0000', 'last_fullsized': '2024-05-28T14:11:14.394155+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:11:20.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2024-05-28T14:11:20.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.394647+0000', 'last_change': '2024-05-28T14:11:14.394647+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.394647+0000', 'last_undegraded': '2024-05-28T14:11:14.394647+0000', 'last_fullsized': '2024-05-28T14:11:14.394647+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:11:20.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2024-05-28T14:11:20.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.397797+0000', 'last_change': '2024-05-28T14:11:14.397797+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.397797+0000', 'last_undegraded': '2024-05-28T14:11:14.397797+0000', 'last_fullsized': '2024-05-28T14:11:14.397797+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:11:20.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2024-05-28T14:11:20.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.396693+0000', 'last_change': '2024-05-28T14:11:14.396693+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.396693+0000', 'last_undegraded': '2024-05-28T14:11:14.396693+0000', 'last_fullsized': '2024-05-28T14:11:14.396693+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:11:20.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2024-05-28T14:11:20.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.395974+0000', 'last_change': '2024-05-28T14:11:14.395974+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.395974+0000', 'last_undegraded': '2024-05-28T14:11:14.395974+0000', 'last_fullsized': '2024-05-28T14:11:14.395974+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:11:20.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2024-05-28T14:11:20.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.395375+0000', 'last_change': '2024-05-28T14:11:14.395375+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.395375+0000', 'last_undegraded': '2024-05-28T14:11:14.395375+0000', 'last_fullsized': '2024-05-28T14:11:14.395375+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:11:20.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2024-05-28T14:11:20.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.398141+0000', 'last_change': '2024-05-28T14:11:14.398141+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.398141+0000', 'last_undegraded': '2024-05-28T14:11:14.398141+0000', 'last_fullsized': '2024-05-28T14:11:14.398141+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:11:20.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2024-05-28T14:11:20.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.388993+0000', 'last_change': '2024-05-28T14:11:14.388993+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.388993+0000', 'last_undegraded': '2024-05-28T14:11:14.388993+0000', 'last_fullsized': '2024-05-28T14:11:14.388993+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:11:20.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2024-05-28T14:11:20.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.390994+0000', 'last_change': '2024-05-28T14:11:14.390994+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.390994+0000', 'last_undegraded': '2024-05-28T14:11:14.390994+0000', 'last_fullsized': '2024-05-28T14:11:14.390994+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:11:20.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2024-05-28T14:11:20.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.393752+0000', 'last_change': '2024-05-28T14:11:14.393752+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.393752+0000', 'last_undegraded': '2024-05-28T14:11:14.393752+0000', 'last_fullsized': '2024-05-28T14:11:14.393752+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:11:20.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2024-05-28T14:11:20.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.392990+0000', 'last_change': '2024-05-28T14:11:14.392990+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.392990+0000', 'last_undegraded': '2024-05-28T14:11:14.392990+0000', 'last_fullsized': '2024-05-28T14:11:14.392990+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:11:20.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2024-05-28T14:11:20.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.390220+0000', 'last_change': '2024-05-28T14:11:14.390220+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.390220+0000', 'last_undegraded': '2024-05-28T14:11:14.390220+0000', 'last_fullsized': '2024-05-28T14:11:14.390220+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:11:20.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2024-05-28T14:11:20.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.393542+0000', 'last_change': '2024-05-28T14:11:14.393542+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.393542+0000', 'last_undegraded': '2024-05-28T14:11:14.393542+0000', 'last_fullsized': '2024-05-28T14:11:14.393542+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:11:20.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2024-05-28T14:11:20.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.389820+0000', 'last_change': '2024-05-28T14:11:14.389820+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.389820+0000', 'last_undegraded': '2024-05-28T14:11:14.389820+0000', 'last_fullsized': '2024-05-28T14:11:14.389820+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:11:20.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2024-05-28T14:11:20.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.395418+0000', 'last_change': '2024-05-28T14:11:14.395418+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.395418+0000', 'last_undegraded': '2024-05-28T14:11:14.395418+0000', 'last_fullsized': '2024-05-28T14:11:14.395418+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:11:20.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2024-05-28T14:11:20.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.398379+0000', 'last_change': '2024-05-28T14:11:14.398379+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.398379+0000', 'last_undegraded': '2024-05-28T14:11:14.398379+0000', 'last_fullsized': '2024-05-28T14:11:14.398379+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:11:20.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2024-05-28T14:11:20.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.392041+0000', 'last_change': '2024-05-28T14:11:14.392041+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.392041+0000', 'last_undegraded': '2024-05-28T14:11:14.392041+0000', 'last_fullsized': '2024-05-28T14:11:14.392041+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:11:20.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2024-05-28T14:11:20.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.401610+0000', 'last_change': '2024-05-28T14:11:14.401610+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.401610+0000', 'last_undegraded': '2024-05-28T14:11:14.401610+0000', 'last_fullsized': '2024-05-28T14:11:14.401610+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:11:20.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2024-05-28T14:11:20.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.388892+0000', 'last_change': '2024-05-28T14:11:14.388892+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.388892+0000', 'last_undegraded': '2024-05-28T14:11:14.388892+0000', 'last_fullsized': '2024-05-28T14:11:14.388892+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:11:20.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2024-05-28T14:11:20.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.391686+0000', 'last_change': '2024-05-28T14:11:14.391686+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.391686+0000', 'last_undegraded': '2024-05-28T14:11:14.391686+0000', 'last_fullsized': '2024-05-28T14:11:14.391686+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:11:20.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2024-05-28T14:11:20.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.395810+0000', 'last_change': '2024-05-28T14:11:14.395810+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.395810+0000', 'last_undegraded': '2024-05-28T14:11:14.395810+0000', 'last_fullsized': '2024-05-28T14:11:14.395810+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:11:20.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2024-05-28T14:11:20.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.391743+0000', 'last_change': '2024-05-28T14:11:14.391743+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.391743+0000', 'last_undegraded': '2024-05-28T14:11:14.391743+0000', 'last_fullsized': '2024-05-28T14:11:14.391743+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:11:20.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2024-05-28T14:11:20.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.393759+0000', 'last_change': '2024-05-28T14:11:14.393759+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.393759+0000', 'last_undegraded': '2024-05-28T14:11:14.393759+0000', 'last_fullsized': '2024-05-28T14:11:14.393759+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:11:20.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2024-05-28T14:11:20.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.394208+0000', 'last_change': '2024-05-28T14:11:14.394208+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.394208+0000', 'last_undegraded': '2024-05-28T14:11:14.394208+0000', 'last_fullsized': '2024-05-28T14:11:14.394208+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:11:20.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2024-05-28T14:11:20.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.399596+0000', 'last_change': '2024-05-28T14:11:14.399596+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.399596+0000', 'last_undegraded': '2024-05-28T14:11:14.399596+0000', 'last_fullsized': '2024-05-28T14:11:14.399596+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:11:20.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2024-05-28T14:11:20.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.395584+0000', 'last_change': '2024-05-28T14:11:14.395584+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.395584+0000', 'last_undegraded': '2024-05-28T14:11:14.395584+0000', 'last_fullsized': '2024-05-28T14:11:14.395584+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:11:20.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2024-05-28T14:11:20.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.396807+0000', 'last_change': '2024-05-28T14:11:14.396807+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.396807+0000', 'last_undegraded': '2024-05-28T14:11:14.396807+0000', 'last_fullsized': '2024-05-28T14:11:14.396807+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:11:20.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2024-05-28T14:11:20.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.391572+0000', 'last_change': '2024-05-28T14:11:14.391572+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.391572+0000', 'last_undegraded': '2024-05-28T14:11:14.391572+0000', 'last_fullsized': '2024-05-28T14:11:14.391572+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:11:20.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2024-05-28T14:11:20.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.399351+0000', 'last_change': '2024-05-28T14:11:14.399351+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.399351+0000', 'last_undegraded': '2024-05-28T14:11:14.399351+0000', 'last_fullsized': '2024-05-28T14:11:14.399351+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:11:20.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2024-05-28T14:11:20.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.397667+0000', 'last_change': '2024-05-28T14:11:14.397667+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.397667+0000', 'last_undegraded': '2024-05-28T14:11:14.397667+0000', 'last_fullsized': '2024-05-28T14:11:14.397667+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:11:20.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2024-05-28T14:11:20.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.394850+0000', 'last_change': '2024-05-28T14:11:14.394850+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.394850+0000', 'last_undegraded': '2024-05-28T14:11:14.394850+0000', 'last_fullsized': '2024-05-28T14:11:14.394850+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:11:20.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2024-05-28T14:11:20.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.393725+0000', 'last_change': '2024-05-28T14:11:14.393725+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.393725+0000', 'last_undegraded': '2024-05-28T14:11:14.393725+0000', 'last_fullsized': '2024-05-28T14:11:14.393725+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:11:20.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2024-05-28T14:11:20.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.388969+0000', 'last_change': '2024-05-28T14:11:14.388969+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.388969+0000', 'last_undegraded': '2024-05-28T14:11:14.388969+0000', 'last_fullsized': '2024-05-28T14:11:14.388969+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:11:20.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2024-05-28T14:11:20.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.391914+0000', 'last_change': '2024-05-28T14:11:14.391914+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.391914+0000', 'last_undegraded': '2024-05-28T14:11:14.391914+0000', 'last_fullsized': '2024-05-28T14:11:14.391914+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:11:20.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2024-05-28T14:11:20.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.396370+0000', 'last_change': '2024-05-28T14:11:14.396370+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.396370+0000', 'last_undegraded': '2024-05-28T14:11:14.396370+0000', 'last_fullsized': '2024-05-28T14:11:14.396370+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:11:20.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2024-05-28T14:11:20.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:11:14.392974+0000', 'last_change': '2024-05-28T14:11:14.392974+0000', 'last_active': '2024-05-28T14:11:14.375217+0000', 'last_peered': '2024-05-28T14:11:14.375217+0000', 'last_clean': '2024-05-28T14:11:14.375217+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:11:14.392974+0000', 'last_undegraded': '2024-05-28T14:11:14.392974+0000', 'last_fullsized': '2024-05-28T14:11:14.392974+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'last_clean_scrub_stamp': '2024-05-28T14:11:14.375217+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-28T14:11:20.327 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-28T14:11:20.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6e3200640 1 Processor -- start 2024-05-28T14:11:20.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6e3200640 1 -- start start 2024-05-28T14:11:20.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6e3200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6dc0af6d0 0x7fc6dc167530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6e3200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6dc06f6a0 con 0x7fc6dc156c00 2024-05-28T14:11:20.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6e3200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6dc06e600 con 0x7fc6dc15d8d0 2024-05-28T14:11:20.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6e3200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc6dc06e8e0 con 0x7fc6dc0af6d0 2024-05-28T14:11:20.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6dbe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc6dc156c00 0x7fc6dc06dd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39252/0 (socket says 172.21.15.42:39252) 2024-05-28T14:11:20.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6dbe00640 1 -- 172.21.15.42:0/559097918 learned_addr learned my addr 172.21.15.42:0/559097918 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:20.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6e1200640 1 --2- 172.21.15.42:0/559097918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6dc0af6d0 0x7fc6dc167530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 -- 172.21.15.42:0/559097918 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2961527467 0 0) 0x7fc6dc06f6a0 con 0x7fc6dc156c00 2024-05-28T14:11:20.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 -- 172.21.15.42:0/559097918 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6c0003530 con 0x7fc6dc156c00 2024-05-28T14:11:20.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 -- 172.21.15.42:0/559097918 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1951054340 0 0) 0x7fc6c0003530 con 0x7fc6dc156c00 2024-05-28T14:11:20.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 -- 172.21.15.42:0/559097918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6dc0af6d0 msgr2=0x7fc6dc167530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:20.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 --2- 172.21.15.42:0/559097918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6dc0af6d0 0x7fc6dc167530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 -- 172.21.15.42:0/559097918 >> v1:172.21.15.63:6789/0 conn(0x7fc6dc15d8d0 legacy=0x7fc6dc0aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 -- 172.21.15.42:0/559097918 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6dc06ebc0 con 0x7fc6dc156c00 2024-05-28T14:11:20.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 -- 172.21.15.42:0/559097918 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fc6cc001680 con 0x7fc6dc156c00 2024-05-28T14:11:20.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 -- 172.21.15.42:0/559097918 >> v1:172.21.15.42:6789/0 conn(0x7fc6dc156c00 legacy=0x7fc6dc06dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 --2- 172.21.15.42:0/559097918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6dc0af6d0 0x7fc6c00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 --2- 172.21.15.42:0/559097918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0003f30 0x7fc6c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 --2- 172.21.15.42:0/559097918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6c0004d50 0x7fc6c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 -- 172.21.15.42:0/559097918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc06e8e0 con 0x7fc6c0003f30 2024-05-28T14:11:20.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 -- 172.21.15.42:0/559097918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c000bae0 con 0x7fc6c0004d50 2024-05-28T14:11:20.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6db400640 1 -- 172.21.15.42:0/559097918 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c000bce0 con 0x7fc6dc0af6d0 2024-05-28T14:11:20.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6d3400640 1 --2- 172.21.15.42:0/559097918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0003f30 0x7fc6c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6dbe00640 1 --2- 172.21.15.42:0/559097918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6dc0af6d0 0x7fc6c00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6e1200640 1 --2- 172.21.15.42:0/559097918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6c0004d50 0x7fc6c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6d3400640 1 -- 172.21.15.42:0/559097918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6dc0af6d0 msgr2=0x7fc6c00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:20.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6d3400640 1 --2- 172.21.15.42:0/559097918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6dc0af6d0 0x7fc6c00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6d3400640 1 -- 172.21.15.42:0/559097918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6c0004d50 msgr2=0x7fc6c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:20.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6d3400640 1 --2- 172.21.15.42:0/559097918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6c0004d50 0x7fc6c00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.516+0000 7fc6d3400640 1 -- 172.21.15.42:0/559097918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c0003530 con 0x7fc6c0003f30 2024-05-28T14:11:20.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.520+0000 7fc6d3400640 1 --2- 172.21.15.42:0/559097918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0003f30 0x7fc6c0004810 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc15dd20 tx=0x7fc6c401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d54ffc8f9783807 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:20.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.520+0000 7fc6db400640 1 -- 172.21.15.42:0/559097918 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c40b0070 con 0x7fc6c0003f30 2024-05-28T14:11:20.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.520+0000 7fc6db400640 1 -- 172.21.15.42:0/559097918 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6c40ab600 con 0x7fc6c0003f30 2024-05-28T14:11:20.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.520+0000 7fc6db400640 1 -- 172.21.15.42:0/559097918 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4001460 con 0x7fc6c0003f30 2024-05-28T14:11:20.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.520+0000 7fc6e3200640 1 -- 172.21.15.42:0/559097918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0003f30 msgr2=0x7fc6c0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.520+0000 7fc6e3200640 1 --2- 172.21.15.42:0/559097918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0003f30 0x7fc6c0004810 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc15dd20 tx=0x7fc6c401c4c0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.520+0000 7fc6e1200640 1 -- 172.21.15.42:0/559097918 reap_dead start 2024-05-28T14:11:20.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.520+0000 7fc6e3200640 1 -- 172.21.15.42:0/559097918 shutdown_connections 2024-05-28T14:11:20.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.520+0000 7fc6e3200640 1 -- 172.21.15.42:0/559097918 >> 172.21.15.42:0/559097918 conn(0x7fc6dc06bff0 msgr2=0x7fc6dc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:20.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.520+0000 7fc6e3200640 1 -- 172.21.15.42:0/559097918 shutdown_connections 2024-05-28T14:11:20.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.520+0000 7fc6e3200640 1 -- 172.21.15.42:0/559097918 wait complete. 2024-05-28T14:11:20.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6e3200640 1 Processor -- start 2024-05-28T14:11:20.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6e3200640 1 -- start start 2024-05-28T14:11:20.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6e3200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0003f30 0x7fc6dc1647a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6e3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6dc0b3b60 0x7fc6dc14d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6e3200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6dc0b3fe0 0x7fc6dc14d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6e3200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc167ec0 con 0x7fc6c0003f30 2024-05-28T14:11:20.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6e3200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc167d40 con 0x7fc6dc0b3b60 2024-05-28T14:11:20.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6e3200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc6dc168040 con 0x7fc6dc0b3fe0 2024-05-28T14:11:20.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6dbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0003f30 0x7fc6dc1647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6dbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0003f30 0x7fc6dc1647a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53348/0 (socket says 172.21.15.42:53348) 2024-05-28T14:11:20.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6dbe00640 1 -- 172.21.15.42:0/2188612464 learned_addr learned my addr 172.21.15.42:0/2188612464 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:20.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6e1200640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6dc0b3fe0 0x7fc6dc14d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6d3400640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6dc0b3b60 0x7fc6dc14d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6dbe00640 1 -- 172.21.15.42:0/2188612464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6dc0b3fe0 msgr2=0x7fc6dc14d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6dbe00640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6dc0b3fe0 0x7fc6dc14d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6dbe00640 1 -- 172.21.15.42:0/2188612464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6dc0b3b60 msgr2=0x7fc6dc14d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6dbe00640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6dc0b3b60 0x7fc6dc14d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6dbe00640 1 -- 172.21.15.42:0/2188612464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6dc14dfa0 con 0x7fc6c0003f30 2024-05-28T14:11:20.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6d3400640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6dc0b3b60 0x7fc6dc14d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:20.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6dbe00640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0003f30 0x7fc6dc1647a0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0017f0 tx=0x7fc6cc0077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:20.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6d8c00640 1 -- 172.21.15.42:0/2188612464 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc00f990 con 0x7fc6c0003f30 2024-05-28T14:11:20.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6d8c00640 1 -- 172.21.15.42:0/2188612464 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6cc018070 con 0x7fc6c0003f30 2024-05-28T14:11:20.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6e3200640 1 -- 172.21.15.42:0/2188612464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6dc073a60 con 0x7fc6c0003f30 2024-05-28T14:11:20.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6d8c00640 1 -- 172.21.15.42:0/2188612464 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc00fd30 con 0x7fc6c0003f30 2024-05-28T14:11:20.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.524+0000 7fc6e3200640 1 -- 172.21.15.42:0/2188612464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6dc072d50 con 0x7fc6c0003f30 2024-05-28T14:11:20.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.528+0000 7fc6e3200640 1 -- 172.21.15.42:0/2188612464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6dc068a90 con 0x7fc6c0003f30 2024-05-28T14:11:20.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.528+0000 7fc6d8c00640 1 -- 172.21.15.42:0/2188612464 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fc6cc01d460 con 0x7fc6c0003f30 2024-05-28T14:11:20.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.528+0000 7fc6d8c00640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc6b8069880 0x7fc6b806bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.528+0000 7fc6d8c00640 1 -- 172.21.15.42:0/2188612464 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fc6cc085bf0 con 0x7fc6c0003f30 2024-05-28T14:11:20.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.528+0000 7fc6d3400640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc6b8069880 0x7fc6b806bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.528+0000 7fc6d3400640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc6b8069880 0x7fc6b806bd40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40ad000 tx=0x7fc6c40a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:20.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.532+0000 7fc6d8c00640 1 -- 172.21.15.42:0/2188612464 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc6cc057820 con 0x7fc6c0003f30 2024-05-28T14:11:20.585 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:11:20.588 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:11:20.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.760+0000 7fc6e3200640 1 -- 172.21.15.42:0/2188612464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc6dc0b2a40 con 0x7fc6c0003f30 2024-05-28T14:11:20.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.760+0000 7fc6d8c00640 1 -- 172.21.15.42:0/2188612464 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1037 (secure 0 0 0) 0x7fc6cc057120 con 0x7fc6c0003f30 2024-05-28T14:11:20.743 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:20.744 INFO:teuthology.orchestra.run.smithi042.stdout:{"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":802,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":105,"num_osds":8,"num_up_osds":8,"osd_up_since":1716904689,"num_in_osds":8,"osd_in_since":1716904680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":911642624,"bytes_avail":772182470656,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":64,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":57056}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-05-28T14:10:37.128981+0000","services":{}},"progress_events":{}} 2024-05-28T14:11:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.760+0000 7fc6e3200640 1 -- 172.21.15.42:0/2188612464 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc6b8069880 msgr2=0x7fc6b806bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.760+0000 7fc6e3200640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc6b8069880 0x7fc6b806bd40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40ad000 tx=0x7fc6c40a6000 comp rx=0 tx=0).stop 2024-05-28T14:11:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.760+0000 7fc6e3200640 1 -- 172.21.15.42:0/2188612464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0003f30 msgr2=0x7fc6dc1647a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.760+0000 7fc6e3200640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0003f30 0x7fc6dc1647a0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0017f0 tx=0x7fc6cc0077a0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.760+0000 7fc6e3200640 1 -- 172.21.15.42:0/2188612464 shutdown_connections 2024-05-28T14:11:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.760+0000 7fc6e3200640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc6b8069880 0x7fc6b806bd40 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.760+0000 7fc6e3200640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6dc0b3fe0 0x7fc6dc14d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.760+0000 7fc6e3200640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6dc0b3b60 0x7fc6dc14d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.760+0000 7fc6e3200640 1 --2- 172.21.15.42:0/2188612464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c0003f30 0x7fc6dc1647a0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.760+0000 7fc6e3200640 1 -- 172.21.15.42:0/2188612464 >> 172.21.15.42:0/2188612464 conn(0x7fc6dc06bff0 msgr2=0x7fc6dc0d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.760+0000 7fc6e3200640 1 -- 172.21.15.42:0/2188612464 shutdown_connections 2024-05-28T14:11:20.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.760+0000 7fc6e3200640 1 -- 172.21.15.42:0/2188612464 wait complete. 2024-05-28T14:11:20.766 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-28T14:11:20.767 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:11:20.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f4f800640 1 Processor -- start 2024-05-28T14:11:20.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f4f800640 1 -- start start 2024-05-28T14:11:20.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f4f800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6f4806f660 0x7f6f48073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f4f800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f480d0e00 0x7f6f4806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f4f800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f4815f280 con 0x7f6f4815ba40 2024-05-28T14:11:20.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f4f800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6f4815f030 con 0x7f6f4806f660 2024-05-28T14:11:20.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f4f800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4815f560 con 0x7f6f480d0e00 2024-05-28T14:11:20.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f4ce00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6f4806f660 0x7f6f48073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f4d800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f480d0e00 0x7f6f4806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f3fe00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f6f4815ba40 0x7f6f480d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:54046/0 (socket says 172.21.15.42:54046) 2024-05-28T14:11:20.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f3fe00640 1 -- 172.21.15.42:0/2923207801 learned_addr learned my addr 172.21.15.42:0/2923207801 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:20.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f47e00640 1 -- 172.21.15.42:0/2923207801 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2847720793 0 0) 0x7f6f4815f280 con 0x7f6f4815ba40 2024-05-28T14:11:20.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f47e00640 1 -- 172.21.15.42:0/2923207801 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f2c003530 con 0x7f6f4815ba40 2024-05-28T14:11:20.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f4d800640 1 -- 172.21.15.42:0/2923207801 >> v2:172.21.15.63:3300/0 conn(0x7f6f4806f660 msgr2=0x7f6f48073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:20.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f4d800640 1 --2- 172.21.15.42:0/2923207801 >> v2:172.21.15.63:3300/0 conn(0x7f6f4806f660 0x7f6f48073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f4d800640 1 -- 172.21.15.42:0/2923207801 >> v1:172.21.15.63:6789/0 conn(0x7f6f4815ba40 legacy=0x7f6f480d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f4d800640 1 -- 172.21.15.42:0/2923207801 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4815f840 con 0x7f6f480d0e00 2024-05-28T14:11:20.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.924+0000 7f6f4ce00640 1 --2- 172.21.15.42:0/2923207801 >> v2:172.21.15.63:3300/0 conn(0x7f6f4806f660 0x7f6f48073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:11:20.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.928+0000 7f6f4d800640 1 --2- 172.21.15.42:0/2923207801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f480d0e00 0x7f6f4806dd60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009d90 tx=0x7f6f4001c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c93e4fd48ee02fd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:20.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.928+0000 7f6f47e00640 1 -- 172.21.15.42:0/2923207801 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f400b2070 con 0x7f6f480d0e00 2024-05-28T14:11:20.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.928+0000 7f6f47e00640 1 -- 172.21.15.42:0/2923207801 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f400a58c0 con 0x7f6f480d0e00 2024-05-28T14:11:20.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.928+0000 7f6f47e00640 1 -- 172.21.15.42:0/2923207801 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f400a5c90 con 0x7f6f480d0e00 2024-05-28T14:11:20.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.928+0000 7f6f4f800640 1 -- 172.21.15.42:0/2923207801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f480d0e00 msgr2=0x7f6f4806dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.928+0000 7f6f4f800640 1 --2- 172.21.15.42:0/2923207801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f480d0e00 0x7f6f4806dd60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009d90 tx=0x7f6f4001c820 comp rx=0 tx=0).stop 2024-05-28T14:11:20.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4f800640 1 -- 172.21.15.42:0/2923207801 shutdown_connections 2024-05-28T14:11:20.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4f800640 1 --2- 172.21.15.42:0/2923207801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f480d0e00 0x7f6f4806dd60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4f800640 1 --2- 172.21.15.42:0/2923207801 >> v2:172.21.15.63:3300/0 conn(0x7f6f4806f660 0x7f6f48073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4f800640 1 -- 172.21.15.42:0/2923207801 >> 172.21.15.42:0/2923207801 conn(0x7f6f4806bdf0 msgr2=0x7f6f4806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:20.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4f800640 1 -- 172.21.15.42:0/2923207801 shutdown_connections 2024-05-28T14:11:20.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4f800640 1 -- 172.21.15.42:0/2923207801 wait complete. 2024-05-28T14:11:20.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4f800640 1 Processor -- start 2024-05-28T14:11:20.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4f800640 1 -- start start 2024-05-28T14:11:20.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4f800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f4806f660 0x7f6f4814e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4f800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f480d0e00 0x7f6f48155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4f800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f48155880 0x7f6f48151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4f800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6f480b3460 con 0x7f6f48155880 2024-05-28T14:11:20.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4f800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f480b32e0 con 0x7f6f480d0e00 2024-05-28T14:11:20.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4f800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6f480b35e0 con 0x7f6f4806f660 2024-05-28T14:11:20.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f48155880 0x7f6f48151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f48155880 0x7f6f48151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53356/0 (socket says 172.21.15.42:53356) 2024-05-28T14:11:20.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4d800640 1 -- 172.21.15.42:0/585827096 learned_addr learned my addr 172.21.15.42:0/585827096 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:20.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4ce00640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f4806f660 0x7f6f4814e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f3fe00640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f480d0e00 0x7f6f48155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4d800640 1 -- 172.21.15.42:0/585827096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f4806f660 msgr2=0x7f6f4814e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4d800640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f4806f660 0x7f6f4814e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4d800640 1 -- 172.21.15.42:0/585827096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f480d0e00 msgr2=0x7f6f48155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:20.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4d800640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f480d0e00 0x7f6f48155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:20.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4d800640 1 -- 172.21.15.42:0/585827096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f48151f60 con 0x7f6f48155880 2024-05-28T14:11:20.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f3fe00640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f480d0e00 0x7f6f48155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:20.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.932+0000 7f6f4d800640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f48155880 0x7f6f48151230 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f6f4001aed0 tx=0x7f6f400195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:20.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.936+0000 7f6f45600640 1 -- 172.21.15.42:0/585827096 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f400b2050 con 0x7f6f48155880 2024-05-28T14:11:20.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.936+0000 7f6f45600640 1 -- 172.21.15.42:0/585827096 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f400be070 con 0x7f6f48155880 2024-05-28T14:11:20.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.936+0000 7f6f45600640 1 -- 172.21.15.42:0/585827096 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f400ad850 con 0x7f6f48155880 2024-05-28T14:11:20.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.936+0000 7f6f4f800640 1 -- 172.21.15.42:0/585827096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f48152240 con 0x7f6f48155880 2024-05-28T14:11:20.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.936+0000 7f6f4f800640 1 -- 172.21.15.42:0/585827096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f480b06c0 con 0x7f6f48155880 2024-05-28T14:11:20.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.936+0000 7f6f4f800640 1 -- 172.21.15.42:0/585827096 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f48068a90 con 0x7f6f48155880 2024-05-28T14:11:20.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.936+0000 7f6f45600640 1 -- 172.21.15.42:0/585827096 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f6f400ad9f0 con 0x7f6f48155880 2024-05-28T14:11:20.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.936+0000 7f6f45600640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f24069850 0x7f6f2406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:20.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.936+0000 7f6f45600640 1 -- 172.21.15.42:0/585827096 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f6f401207a0 con 0x7f6f48155880 2024-05-28T14:11:20.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.936+0000 7f6f4ce00640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f24069850 0x7f6f2406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:20.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.936+0000 7f6f4ce00640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f24069850 0x7f6f2406bd10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009580 tx=0x7f6f38008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:20.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:20.940+0000 7f6f45600640 1 -- 172.21.15.42:0/585827096 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6f40016560 con 0x7f6f48155880 2024-05-28T14:11:21.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:21.104+0000 7f6f4f800640 1 -- 172.21.15.42:0/585827096 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f48155d10 con 0x7f6f24069850 2024-05-28T14:11:21.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:21.108+0000 7f6f45600640 1 -- 172.21.15.42:0/585827096 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247105 (secure 0 0 0) 0x7f6f380119a0 con 0x7f6f24069850 2024-05-28T14:11:21.095 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:21.096 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:11:21.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:21.112+0000 7f6f4f800640 1 -- 172.21.15.42:0/585827096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f24069850 msgr2=0x7f6f2406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:21.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:21.112+0000 7f6f4f800640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f24069850 0x7f6f2406bd10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009580 tx=0x7f6f38008040 comp rx=0 tx=0).stop 2024-05-28T14:11:21.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:21.112+0000 7f6f4f800640 1 -- 172.21.15.42:0/585827096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f48155880 msgr2=0x7f6f48151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:21.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:21.112+0000 7f6f4f800640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f48155880 0x7f6f48151230 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f6f4001aed0 tx=0x7f6f400195c0 comp rx=0 tx=0).stop 2024-05-28T14:11:21.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:21.112+0000 7f6f4f800640 1 -- 172.21.15.42:0/585827096 shutdown_connections 2024-05-28T14:11:21.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:21.112+0000 7f6f4f800640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6f24069850 0x7f6f2406bd10 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:21.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:21.112+0000 7f6f4f800640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f48155880 0x7f6f48151230 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:21.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:21.112+0000 7f6f4f800640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f480d0e00 0x7f6f48155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:21.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:21.112+0000 7f6f4f800640 1 --2- 172.21.15.42:0/585827096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f4806f660 0x7f6f4814e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:21.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:21.112+0000 7f6f4f800640 1 -- 172.21.15.42:0/585827096 >> 172.21.15.42:0/585827096 conn(0x7f6f4806bdf0 msgr2=0x7f6f4815e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:21.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:21.112+0000 7f6f4f800640 1 -- 172.21.15.42:0/585827096 shutdown_connections 2024-05-28T14:11:21.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:21.112+0000 7f6f4f800640 1 -- 172.21.15.42:0/585827096 wait complete. 2024-05-28T14:11:21.116 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-05-28T14:11:19.139088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":890276,"kb_used_data":11052,"kb_used_omap":12,"kb_used_meta":879091,"kb_avail":754084444,"statfs":{"total":773094113280,"available":772182470656,"internally_reserved":0,"allocated":11317248,"data_stored":5576301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":900189746},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.763808"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399798+0000","last_change":"2024-05-28T14:11:14.399798+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399798+0000","last_undegraded":"2024-05-28T14:11:14.399798+0000","last_fullsized":"2024-05-28T14:11:14.399798+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396400+0000","last_change":"2024-05-28T14:11:14.396400+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396400+0000","last_undegraded":"2024-05-28T14:11:14.396400+0000","last_fullsized":"2024-05-28T14:11:14.396400+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397782+0000","last_change":"2024-05-28T14:11:14.397782+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397782+0000","last_undegraded":"2024-05-28T14:11:14.397782+0000","last_fullsized":"2024-05-28T14:11:14.397782+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396940+0000","last_change":"2024-05-28T14:11:14.396940+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396940+0000","last_undegraded":"2024-05-28T14:11:14.396940+0000","last_fullsized":"2024-05-28T14:11:14.396940+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395541+0000","last_change":"2024-05-28T14:11:14.395541+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395541+0000","last_undegraded":"2024-05-28T14:11:14.395541+0000","last_fullsized":"2024-05-28T14:11:14.395541+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393722+0000","last_change":"2024-05-28T14:11:14.393722+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393722+0000","last_undegraded":"2024-05-28T14:11:14.393722+0000","last_fullsized":"2024-05-28T14:11:14.393722+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393740+0000","last_change":"2024-05-28T14:11:14.393740+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393740+0000","last_undegraded":"2024-05-28T14:11:14.393740+0000","last_fullsized":"2024-05-28T14:11:14.393740+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394974+0000","last_change":"2024-05-28T14:11:14.394974+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394974+0000","last_undegraded":"2024-05-28T14:11:14.394974+0000","last_fullsized":"2024-05-28T14:11:14.394974+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.390899+0000","last_change":"2024-05-28T14:11:14.390899+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.390899+0000","last_undegraded":"2024-05-28T14:11:14.390899+0000","last_fullsized":"2024-05-28T14:11:14.390899+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395187+0000","last_change":"2024-05-28T14:11:14.395187+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395187+0000","last_undegraded":"2024-05-28T14:11:14.395187+0000","last_fullsized":"2024-05-28T14:11:14.395187+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399363+0000","last_change":"2024-05-28T14:11:14.399363+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399363+0000","last_undegraded":"2024-05-28T14:11:14.399363+0000","last_fullsized":"2024-05-28T14:11:14.399363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.389835+0000","last_change":"2024-05-28T14:11:14.389835+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.389835+0000","last_undegraded":"2024-05-28T14:11:14.389835+0000","last_fullsized":"2024-05-28T14:11:14.389835+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398498+0000","last_change":"2024-05-28T14:11:14.398498+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398498+0000","last_undegraded":"2024-05-28T14:11:14.398498+0000","last_fullsized":"2024-05-28T14:11:14.398498+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395077+0000","last_change":"2024-05-28T14:11:14.395077+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395077+0000","last_undegraded":"2024-05-28T14:11:14.395077+0000","last_fullsized":"2024-05-28T14:11:14.395077+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398243+0000","last_change":"2024-05-28T14:11:14.398243+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398243+0000","last_undegraded":"2024-05-28T14:11:14.398243+0000","last_fullsized":"2024-05-28T14:11:14.398243+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397185+0000","last_change":"2024-05-28T14:11:14.397185+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397185+0000","last_undegraded":"2024-05-28T14:11:14.397185+0000","last_fullsized":"2024-05-28T14:11:14.397185+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395430+0000","last_change":"2024-05-28T14:11:14.395430+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395430+0000","last_undegraded":"2024-05-28T14:11:14.395430+0000","last_fullsized":"2024-05-28T14:11:14.395430+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395911+0000","last_change":"2024-05-28T14:11:14.395911+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395911+0000","last_undegraded":"2024-05-28T14:11:14.395911+0000","last_fullsized":"2024-05-28T14:11:14.395911+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392659+0000","last_change":"2024-05-28T14:11:14.392659+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392659+0000","last_undegraded":"2024-05-28T14:11:14.392659+0000","last_fullsized":"2024-05-28T14:11:14.392659+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.389349+0000","last_change":"2024-05-28T14:11:14.389349+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.389349+0000","last_undegraded":"2024-05-28T14:11:14.389349+0000","last_fullsized":"2024-05-28T14:11:14.389349+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.400065+0000","last_change":"2024-05-28T14:11:14.400065+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.400065+0000","last_undegraded":"2024-05-28T14:11:14.400065+0000","last_fullsized":"2024-05-28T14:11:14.400065+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394850+0000","last_change":"2024-05-28T14:11:14.394850+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394850+0000","last_undegraded":"2024-05-28T14:11:14.394850+0000","last_fullsized":"2024-05-28T14:11:14.394850+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397557+0000","last_change":"2024-05-28T14:11:14.397557+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397557+0000","last_undegraded":"2024-05-28T14:11:14.397557+0000","last_fullsized":"2024-05-28T14:11:14.397557+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393602+0000","last_change":"2024-05-28T14:11:14.393602+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393602+0000","last_undegraded":"2024-05-28T14:11:14.393602+0000","last_fullsized":"2024-05-28T14:11:14.393602+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397515+0000","last_change":"2024-05-28T14:11:14.397515+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397515+0000","last_undegraded":"2024-05-28T14:11:14.397515+0000","last_fullsized":"2024-05-28T14:11:14.397515+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395934+0000","last_change":"2024-05-28T14:11:14.395934+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395934+0000","last_undegraded":"2024-05-28T14:11:14.395934+0000","last_fullsized":"2024-05-28T14:11:14.395934+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398870+0000","last_change":"2024-05-28T14:11:14.398870+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398870+0000","last_undegraded":"2024-05-28T14:11:14.398870+0000","last_fullsized":"2024-05-28T14:11:14.398870+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396110+0000","last_change":"2024-05-28T14:11:14.396110+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396110+0000","last_undegraded":"2024-05-28T14:11:14.396110+0000","last_fullsized":"2024-05-28T14:11:14.396110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394155+0000","last_change":"2024-05-28T14:11:14.394155+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394155+0000","last_undegraded":"2024-05-28T14:11:14.394155+0000","last_fullsized":"2024-05-28T14:11:14.394155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394647+0000","last_change":"2024-05-28T14:11:14.394647+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394647+0000","last_undegraded":"2024-05-28T14:11:14.394647+0000","last_fullsized":"2024-05-28T14:11:14.394647+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397797+0000","last_change":"2024-05-28T14:11:14.397797+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397797+0000","last_undegraded":"2024-05-28T14:11:14.397797+0000","last_fullsized":"2024-05-28T14:11:14.397797+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396693+0000","last_change":"2024-05-28T14:11:14.396693+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396693+0000","last_undegraded":"2024-05-28T14:11:14.396693+0000","last_fullsized":"2024-05-28T14:11:14.396693+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395974+0000","last_change":"2024-05-28T14:11:14.395974+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395974+0000","last_undegraded":"2024-05-28T14:11:14.395974+0000","last_fullsized":"2024-05-28T14:11:14.395974+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.406287+0000","last_change":"2024-05-28T14:11:14.403379+0000","last_active":"2024-05-28T14:11:14.406287+0000","last_peered":"2024-05-28T14:11:14.406287+0000","last_clean":"2024-05-28T14:11:14.406287+0000","last_became_active":"2024-05-28T14:11:14.403054+0000","last_became_peered":"2024-05-28T14:11:14.403054+0000","last_unstale":"2024-05-28T14:11:14.406287+0000","last_undegraded":"2024-05-28T14:11:14.406287+0000","last_fullsized":"2024-05-28T14:11:14.406287+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:48.727311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395375+0000","last_change":"2024-05-28T14:11:14.395375+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395375+0000","last_undegraded":"2024-05-28T14:11:14.395375+0000","last_fullsized":"2024-05-28T14:11:14.395375+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.414453+0000","last_change":"2024-05-28T14:11:14.413616+0000","last_active":"2024-05-28T14:11:14.414453+0000","last_peered":"2024-05-28T14:11:14.414453+0000","last_clean":"2024-05-28T14:11:14.414453+0000","last_became_active":"2024-05-28T14:11:14.413317+0000","last_became_peered":"2024-05-28T14:11:14.413317+0000","last_unstale":"2024-05-28T14:11:14.414453+0000","last_undegraded":"2024-05-28T14:11:14.414453+0000","last_fullsized":"2024-05-28T14:11:14.414453+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:53:42.892031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398141+0000","last_change":"2024-05-28T14:11:14.398141+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398141+0000","last_undegraded":"2024-05-28T14:11:14.398141+0000","last_fullsized":"2024-05-28T14:11:14.398141+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410331+0000","last_change":"2024-05-28T14:11:14.408679+0000","last_active":"2024-05-28T14:11:14.410331+0000","last_peered":"2024-05-28T14:11:14.410331+0000","last_clean":"2024-05-28T14:11:14.410331+0000","last_became_active":"2024-05-28T14:11:14.408384+0000","last_became_peered":"2024-05-28T14:11:14.408384+0000","last_unstale":"2024-05-28T14:11:14.410331+0000","last_undegraded":"2024-05-28T14:11:14.410331+0000","last_fullsized":"2024-05-28T14:11:14.410331+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:33:51.292167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.388993+0000","last_change":"2024-05-28T14:11:14.388993+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.388993+0000","last_undegraded":"2024-05-28T14:11:14.388993+0000","last_fullsized":"2024-05-28T14:11:14.388993+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.403137+0000","last_change":"2024-05-28T14:11:14.400435+0000","last_active":"2024-05-28T14:11:14.403137+0000","last_peered":"2024-05-28T14:11:14.403137+0000","last_clean":"2024-05-28T14:11:14.403137+0000","last_became_active":"2024-05-28T14:11:14.400157+0000","last_became_peered":"2024-05-28T14:11:14.400157+0000","last_unstale":"2024-05-28T14:11:14.403137+0000","last_undegraded":"2024-05-28T14:11:14.403137+0000","last_fullsized":"2024-05-28T14:11:14.403137+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:28:22.020830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.390994+0000","last_change":"2024-05-28T14:11:14.390994+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.390994+0000","last_undegraded":"2024-05-28T14:11:14.390994+0000","last_fullsized":"2024-05-28T14:11:14.390994+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415348+0000","last_change":"2024-05-28T14:11:14.412408+0000","last_active":"2024-05-28T14:11:14.415348+0000","last_peered":"2024-05-28T14:11:14.415348+0000","last_clean":"2024-05-28T14:11:14.415348+0000","last_became_active":"2024-05-28T14:11:14.411721+0000","last_became_peered":"2024-05-28T14:11:14.411721+0000","last_unstale":"2024-05-28T14:11:14.415348+0000","last_undegraded":"2024-05-28T14:11:14.415348+0000","last_fullsized":"2024-05-28T14:11:14.415348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:34.789464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393752+0000","last_change":"2024-05-28T14:11:14.393752+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393752+0000","last_undegraded":"2024-05-28T14:11:14.393752+0000","last_fullsized":"2024-05-28T14:11:14.393752+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415331+0000","last_change":"2024-05-28T14:11:14.414364+0000","last_active":"2024-05-28T14:11:14.415331+0000","last_peered":"2024-05-28T14:11:14.415331+0000","last_clean":"2024-05-28T14:11:14.415331+0000","last_became_active":"2024-05-28T14:11:14.413986+0000","last_became_peered":"2024-05-28T14:11:14.413986+0000","last_unstale":"2024-05-28T14:11:14.415331+0000","last_undegraded":"2024-05-28T14:11:14.415331+0000","last_fullsized":"2024-05-28T14:11:14.415331+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:30.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392990+0000","last_change":"2024-05-28T14:11:14.392990+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392990+0000","last_undegraded":"2024-05-28T14:11:14.392990+0000","last_fullsized":"2024-05-28T14:11:14.392990+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.407313+0000","last_change":"2024-05-28T14:11:14.406246+0000","last_active":"2024-05-28T14:11:14.407313+0000","last_peered":"2024-05-28T14:11:14.407313+0000","last_clean":"2024-05-28T14:11:14.407313+0000","last_became_active":"2024-05-28T14:11:14.405729+0000","last_became_peered":"2024-05-28T14:11:14.405729+0000","last_unstale":"2024-05-28T14:11:14.407313+0000","last_undegraded":"2024-05-28T14:11:14.407313+0000","last_fullsized":"2024-05-28T14:11:14.407313+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:54:37.950887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.390220+0000","last_change":"2024-05-28T14:11:14.390220+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.390220+0000","last_undegraded":"2024-05-28T14:11:14.390220+0000","last_fullsized":"2024-05-28T14:11:14.390220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415417+0000","last_change":"2024-05-28T14:11:14.413285+0000","last_active":"2024-05-28T14:11:14.415417+0000","last_peered":"2024-05-28T14:11:14.415417+0000","last_clean":"2024-05-28T14:11:14.415417+0000","last_became_active":"2024-05-28T14:11:14.413010+0000","last_became_peered":"2024-05-28T14:11:14.413010+0000","last_unstale":"2024-05-28T14:11:14.415417+0000","last_undegraded":"2024-05-28T14:11:14.415417+0000","last_fullsized":"2024-05-28T14:11:14.415417+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:44:00.271939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393542+0000","last_change":"2024-05-28T14:11:14.393542+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393542+0000","last_undegraded":"2024-05-28T14:11:14.393542+0000","last_fullsized":"2024-05-28T14:11:14.393542+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.413458+0000","last_change":"2024-05-28T14:11:14.406962+0000","last_active":"2024-05-28T14:11:14.413458+0000","last_peered":"2024-05-28T14:11:14.413458+0000","last_clean":"2024-05-28T14:11:14.413458+0000","last_became_active":"2024-05-28T14:11:14.406694+0000","last_became_peered":"2024-05-28T14:11:14.406694+0000","last_unstale":"2024-05-28T14:11:14.413458+0000","last_undegraded":"2024-05-28T14:11:14.413458+0000","last_fullsized":"2024-05-28T14:11:14.413458+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:56:21.193180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.389820+0000","last_change":"2024-05-28T14:11:14.389820+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.389820+0000","last_undegraded":"2024-05-28T14:11:14.389820+0000","last_fullsized":"2024-05-28T14:11:14.389820+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410625+0000","last_change":"2024-05-28T14:11:14.403779+0000","last_active":"2024-05-28T14:11:14.410625+0000","last_peered":"2024-05-28T14:11:14.410625+0000","last_clean":"2024-05-28T14:11:14.410625+0000","last_became_active":"2024-05-28T14:11:14.403497+0000","last_became_peered":"2024-05-28T14:11:14.403497+0000","last_unstale":"2024-05-28T14:11:14.410625+0000","last_undegraded":"2024-05-28T14:11:14.410625+0000","last_fullsized":"2024-05-28T14:11:14.410625+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:50:50.675765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395418+0000","last_change":"2024-05-28T14:11:14.395418+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395418+0000","last_undegraded":"2024-05-28T14:11:14.395418+0000","last_fullsized":"2024-05-28T14:11:14.395418+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.407898+0000","last_change":"2024-05-28T14:11:14.407272+0000","last_active":"2024-05-28T14:11:14.407898+0000","last_peered":"2024-05-28T14:11:14.407898+0000","last_clean":"2024-05-28T14:11:14.407898+0000","last_became_active":"2024-05-28T14:11:14.406878+0000","last_became_peered":"2024-05-28T14:11:14.406878+0000","last_unstale":"2024-05-28T14:11:14.407898+0000","last_undegraded":"2024-05-28T14:11:14.407898+0000","last_fullsized":"2024-05-28T14:11:14.407898+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:08:12.903518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.398379+0000","last_change":"2024-05-28T14:11:14.398379+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.398379+0000","last_undegraded":"2024-05-28T14:11:14.398379+0000","last_fullsized":"2024-05-28T14:11:14.398379+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.409199+0000","last_change":"2024-05-28T14:11:14.408531+0000","last_active":"2024-05-28T14:11:14.409199+0000","last_peered":"2024-05-28T14:11:14.409199+0000","last_clean":"2024-05-28T14:11:14.409199+0000","last_became_active":"2024-05-28T14:11:14.408165+0000","last_became_peered":"2024-05-28T14:11:14.408165+0000","last_unstale":"2024-05-28T14:11:14.409199+0000","last_undegraded":"2024-05-28T14:11:14.409199+0000","last_fullsized":"2024-05-28T14:11:14.409199+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:30.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392041+0000","last_change":"2024-05-28T14:11:14.392041+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392041+0000","last_undegraded":"2024-05-28T14:11:14.392041+0000","last_fullsized":"2024-05-28T14:11:14.392041+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.405337+0000","last_change":"2024-05-28T14:11:14.403933+0000","last_active":"2024-05-28T14:11:14.405337+0000","last_peered":"2024-05-28T14:11:14.405337+0000","last_clean":"2024-05-28T14:11:14.405337+0000","last_became_active":"2024-05-28T14:11:14.403689+0000","last_became_peered":"2024-05-28T14:11:14.403689+0000","last_unstale":"2024-05-28T14:11:14.405337+0000","last_undegraded":"2024-05-28T14:11:14.405337+0000","last_fullsized":"2024-05-28T14:11:14.405337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:43:33.888586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.401610+0000","last_change":"2024-05-28T14:11:14.401610+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.401610+0000","last_undegraded":"2024-05-28T14:11:14.401610+0000","last_fullsized":"2024-05-28T14:11:14.401610+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.416582+0000","last_change":"2024-05-28T14:11:14.416004+0000","last_active":"2024-05-28T14:11:14.416582+0000","last_peered":"2024-05-28T14:11:14.416582+0000","last_clean":"2024-05-28T14:11:14.416582+0000","last_became_active":"2024-05-28T14:11:14.415194+0000","last_became_peered":"2024-05-28T14:11:14.415194+0000","last_unstale":"2024-05-28T14:11:14.416582+0000","last_undegraded":"2024-05-28T14:11:14.416582+0000","last_fullsized":"2024-05-28T14:11:14.416582+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:36:56.646412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.388892+0000","last_change":"2024-05-28T14:11:14.388892+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.388892+0000","last_undegraded":"2024-05-28T14:11:14.388892+0000","last_fullsized":"2024-05-28T14:11:14.388892+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.403799+0000","last_change":"2024-05-28T14:11:14.401589+0000","last_active":"2024-05-28T14:11:14.403799+0000","last_peered":"2024-05-28T14:11:14.403799+0000","last_clean":"2024-05-28T14:11:14.403799+0000","last_became_active":"2024-05-28T14:11:14.401285+0000","last_became_peered":"2024-05-28T14:11:14.401285+0000","last_unstale":"2024-05-28T14:11:14.403799+0000","last_undegraded":"2024-05-28T14:11:14.403799+0000","last_fullsized":"2024-05-28T14:11:14.403799+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391686+0000","last_change":"2024-05-28T14:11:14.391686+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391686+0000","last_undegraded":"2024-05-28T14:11:14.391686+0000","last_fullsized":"2024-05-28T14:11:14.391686+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.413435+0000","last_change":"2024-05-28T14:11:14.408691+0000","last_active":"2024-05-28T14:11:14.413435+0000","last_peered":"2024-05-28T14:11:14.413435+0000","last_clean":"2024-05-28T14:11:14.413435+0000","last_became_active":"2024-05-28T14:11:14.407478+0000","last_became_peered":"2024-05-28T14:11:14.407478+0000","last_unstale":"2024-05-28T14:11:14.413435+0000","last_undegraded":"2024-05-28T14:11:14.413435+0000","last_fullsized":"2024-05-28T14:11:14.413435+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:52.119306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395810+0000","last_change":"2024-05-28T14:11:14.395810+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395810+0000","last_undegraded":"2024-05-28T14:11:14.395810+0000","last_fullsized":"2024-05-28T14:11:14.395810+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.400563+0000","last_change":"2024-05-28T14:11:14.398527+0000","last_active":"2024-05-28T14:11:14.400563+0000","last_peered":"2024-05-28T14:11:14.400563+0000","last_clean":"2024-05-28T14:11:14.400563+0000","last_became_active":"2024-05-28T14:11:14.398239+0000","last_became_peered":"2024-05-28T14:11:14.398239+0000","last_unstale":"2024-05-28T14:11:14.400563+0000","last_undegraded":"2024-05-28T14:11:14.400563+0000","last_fullsized":"2024-05-28T14:11:14.400563+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:41:43.968459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391743+0000","last_change":"2024-05-28T14:11:14.391743+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391743+0000","last_undegraded":"2024-05-28T14:11:14.391743+0000","last_fullsized":"2024-05-28T14:11:14.391743+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.406239+0000","last_change":"2024-05-28T14:11:14.404603+0000","last_active":"2024-05-28T14:11:14.406239+0000","last_peered":"2024-05-28T14:11:14.406239+0000","last_clean":"2024-05-28T14:11:14.406239+0000","last_became_active":"2024-05-28T14:11:14.403710+0000","last_became_peered":"2024-05-28T14:11:14.403710+0000","last_unstale":"2024-05-28T14:11:14.406239+0000","last_undegraded":"2024-05-28T14:11:14.406239+0000","last_fullsized":"2024-05-28T14:11:14.406239+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393759+0000","last_change":"2024-05-28T14:11:14.393759+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393759+0000","last_undegraded":"2024-05-28T14:11:14.393759+0000","last_fullsized":"2024-05-28T14:11:14.393759+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.403113+0000","last_change":"2024-05-28T14:11:14.399470+0000","last_active":"2024-05-28T14:11:14.403113+0000","last_peered":"2024-05-28T14:11:14.403113+0000","last_clean":"2024-05-28T14:11:14.403113+0000","last_became_active":"2024-05-28T14:11:14.397748+0000","last_became_peered":"2024-05-28T14:11:14.397748+0000","last_unstale":"2024-05-28T14:11:14.403113+0000","last_undegraded":"2024-05-28T14:11:14.403113+0000","last_fullsized":"2024-05-28T14:11:14.403113+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:07:58.660765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394208+0000","last_change":"2024-05-28T14:11:14.394208+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394208+0000","last_undegraded":"2024-05-28T14:11:14.394208+0000","last_fullsized":"2024-05-28T14:11:14.394208+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410291+0000","last_change":"2024-05-28T14:11:14.409172+0000","last_active":"2024-05-28T14:11:14.410291+0000","last_peered":"2024-05-28T14:11:14.410291+0000","last_clean":"2024-05-28T14:11:14.410291+0000","last_became_active":"2024-05-28T14:11:14.408866+0000","last_became_peered":"2024-05-28T14:11:14.408866+0000","last_unstale":"2024-05-28T14:11:14.410291+0000","last_undegraded":"2024-05-28T14:11:14.410291+0000","last_fullsized":"2024-05-28T14:11:14.410291+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:37.767057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399596+0000","last_change":"2024-05-28T14:11:14.399596+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399596+0000","last_undegraded":"2024-05-28T14:11:14.399596+0000","last_fullsized":"2024-05-28T14:11:14.399596+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410986+0000","last_change":"2024-05-28T14:11:14.410214+0000","last_active":"2024-05-28T14:11:14.410986+0000","last_peered":"2024-05-28T14:11:14.410986+0000","last_clean":"2024-05-28T14:11:14.410986+0000","last_became_active":"2024-05-28T14:11:14.409906+0000","last_became_peered":"2024-05-28T14:11:14.409906+0000","last_unstale":"2024-05-28T14:11:14.410986+0000","last_undegraded":"2024-05-28T14:11:14.410986+0000","last_fullsized":"2024-05-28T14:11:14.410986+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:41:43.968459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.395584+0000","last_change":"2024-05-28T14:11:14.395584+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.395584+0000","last_undegraded":"2024-05-28T14:11:14.395584+0000","last_fullsized":"2024-05-28T14:11:14.395584+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.411800+0000","last_change":"2024-05-28T14:11:14.411163+0000","last_active":"2024-05-28T14:11:14.411800+0000","last_peered":"2024-05-28T14:11:14.411800+0000","last_clean":"2024-05-28T14:11:14.411800+0000","last_became_active":"2024-05-28T14:11:14.410761+0000","last_became_peered":"2024-05-28T14:11:14.410761+0000","last_unstale":"2024-05-28T14:11:14.411800+0000","last_undegraded":"2024-05-28T14:11:14.411800+0000","last_fullsized":"2024-05-28T14:11:14.411800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396807+0000","last_change":"2024-05-28T14:11:14.396807+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396807+0000","last_undegraded":"2024-05-28T14:11:14.396807+0000","last_fullsized":"2024-05-28T14:11:14.396807+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.412103+0000","last_change":"2024-05-28T14:11:14.411437+0000","last_active":"2024-05-28T14:11:14.412103+0000","last_peered":"2024-05-28T14:11:14.412103+0000","last_clean":"2024-05-28T14:11:14.412103+0000","last_became_active":"2024-05-28T14:11:14.411081+0000","last_became_peered":"2024-05-28T14:11:14.411081+0000","last_unstale":"2024-05-28T14:11:14.412103+0000","last_undegraded":"2024-05-28T14:11:14.412103+0000","last_fullsized":"2024-05-28T14:11:14.412103+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:55:31.353559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391572+0000","last_change":"2024-05-28T14:11:14.391572+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391572+0000","last_undegraded":"2024-05-28T14:11:14.391572+0000","last_fullsized":"2024-05-28T14:11:14.391572+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.400032+0000","last_change":"2024-05-28T14:11:14.397391+0000","last_active":"2024-05-28T14:11:14.400032+0000","last_peered":"2024-05-28T14:11:14.400032+0000","last_clean":"2024-05-28T14:11:14.400032+0000","last_became_active":"2024-05-28T14:11:14.397118+0000","last_became_peered":"2024-05-28T14:11:14.397118+0000","last_unstale":"2024-05-28T14:11:14.400032+0000","last_undegraded":"2024-05-28T14:11:14.400032+0000","last_fullsized":"2024-05-28T14:11:14.400032+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:37.767057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.399351+0000","last_change":"2024-05-28T14:11:14.399351+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.399351+0000","last_undegraded":"2024-05-28T14:11:14.399351+0000","last_fullsized":"2024-05-28T14:11:14.399351+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410413+0000","last_change":"2024-05-28T14:11:14.409506+0000","last_active":"2024-05-28T14:11:14.410413+0000","last_peered":"2024-05-28T14:11:14.410413+0000","last_clean":"2024-05-28T14:11:14.410413+0000","last_became_active":"2024-05-28T14:11:14.407782+0000","last_became_peered":"2024-05-28T14:11:14.407782+0000","last_unstale":"2024-05-28T14:11:14.410413+0000","last_undegraded":"2024-05-28T14:11:14.410413+0000","last_fullsized":"2024-05-28T14:11:14.410413+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:41:43.968459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.397667+0000","last_change":"2024-05-28T14:11:14.397667+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.397667+0000","last_undegraded":"2024-05-28T14:11:14.397667+0000","last_fullsized":"2024-05-28T14:11:14.397667+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.401431+0000","last_change":"2024-05-28T14:11:14.396514+0000","last_active":"2024-05-28T14:11:14.401431+0000","last_peered":"2024-05-28T14:11:14.401431+0000","last_clean":"2024-05-28T14:11:14.401431+0000","last_became_active":"2024-05-28T14:11:14.396179+0000","last_became_peered":"2024-05-28T14:11:14.396179+0000","last_unstale":"2024-05-28T14:11:14.401431+0000","last_undegraded":"2024-05-28T14:11:14.401431+0000","last_fullsized":"2024-05-28T14:11:14.401431+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:08:51.277949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.394850+0000","last_change":"2024-05-28T14:11:14.394850+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.394850+0000","last_undegraded":"2024-05-28T14:11:14.394850+0000","last_fullsized":"2024-05-28T14:11:14.394850+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.407146+0000","last_change":"2024-05-28T14:11:14.405416+0000","last_active":"2024-05-28T14:11:14.407146+0000","last_peered":"2024-05-28T14:11:14.407146+0000","last_clean":"2024-05-28T14:11:14.407146+0000","last_became_active":"2024-05-28T14:11:14.405162+0000","last_became_peered":"2024-05-28T14:11:14.405162+0000","last_unstale":"2024-05-28T14:11:14.407146+0000","last_undegraded":"2024-05-28T14:11:14.407146+0000","last_fullsized":"2024-05-28T14:11:14.407146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:30.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.393725+0000","last_change":"2024-05-28T14:11:14.393725+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.393725+0000","last_undegraded":"2024-05-28T14:11:14.393725+0000","last_fullsized":"2024-05-28T14:11:14.393725+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.410190+0000","last_change":"2024-05-28T14:11:14.408550+0000","last_active":"2024-05-28T14:11:14.410190+0000","last_peered":"2024-05-28T14:11:14.410190+0000","last_clean":"2024-05-28T14:11:14.410190+0000","last_became_active":"2024-05-28T14:11:14.408274+0000","last_became_peered":"2024-05-28T14:11:14.408274+0000","last_unstale":"2024-05-28T14:11:14.410190+0000","last_undegraded":"2024-05-28T14:11:14.410190+0000","last_fullsized":"2024-05-28T14:11:14.410190+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:13:32.928101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.388969+0000","last_change":"2024-05-28T14:11:14.388969+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.388969+0000","last_undegraded":"2024-05-28T14:11:14.388969+0000","last_fullsized":"2024-05-28T14:11:14.388969+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.415284+0000","last_change":"2024-05-28T14:11:14.414517+0000","last_active":"2024-05-28T14:11:14.415284+0000","last_peered":"2024-05-28T14:11:14.415284+0000","last_clean":"2024-05-28T14:11:14.415284+0000","last_became_active":"2024-05-28T14:11:14.414145+0000","last_became_peered":"2024-05-28T14:11:14.414145+0000","last_unstale":"2024-05-28T14:11:14.415284+0000","last_undegraded":"2024-05-28T14:11:14.415284+0000","last_fullsized":"2024-05-28T14:11:14.415284+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:48:06.702787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.391914+0000","last_change":"2024-05-28T14:11:14.391914+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.391914+0000","last_undegraded":"2024-05-28T14:11:14.391914+0000","last_fullsized":"2024-05-28T14:11:14.391914+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.413905+0000","last_change":"2024-05-28T14:11:14.413007+0000","last_active":"2024-05-28T14:11:14.413905+0000","last_peered":"2024-05-28T14:11:14.413905+0000","last_clean":"2024-05-28T14:11:14.413905+0000","last_became_active":"2024-05-28T14:11:14.412715+0000","last_became_peered":"2024-05-28T14:11:14.412715+0000","last_unstale":"2024-05-28T14:11:14.413905+0000","last_undegraded":"2024-05-28T14:11:14.413905+0000","last_fullsized":"2024-05-28T14:11:14.413905+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:25:40.425802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.396370+0000","last_change":"2024-05-28T14:11:14.396370+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.396370+0000","last_undegraded":"2024-05-28T14:11:14.396370+0000","last_fullsized":"2024-05-28T14:11:14.396370+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.409580+0000","last_change":"2024-05-28T14:11:14.407608+0000","last_active":"2024-05-28T14:11:14.409580+0000","last_peered":"2024-05-28T14:11:14.409580+0000","last_clean":"2024-05-28T14:11:14.409580+0000","last_became_active":"2024-05-28T14:11:14.407327+0000","last_became_peered":"2024-05-28T14:11:14.407327+0000","last_unstale":"2024-05-28T14:11:14.409580+0000","last_undegraded":"2024-05-28T14:11:14.409580+0000","last_fullsized":"2024-05-28T14:11:14.409580+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:37:22.556212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-28T14:11:14.392974+0000","last_change":"2024-05-28T14:11:14.392974+0000","last_active":"2024-05-28T14:11:14.375217+0000","last_peered":"2024-05-28T14:11:14.375217+0000","last_clean":"2024-05-28T14:11:14.375217+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:11:14.392974+0000","last_undegraded":"2024-05-28T14:11:14.392974+0000","last_fullsized":"2024-05-28T14:11:14.392974+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.412741+0000","last_change":"2024-05-28T14:11:14.412233+0000","last_active":"2024-05-28T14:11:14.412741+0000","last_peered":"2024-05-28T14:11:14.412741+0000","last_clean":"2024-05-28T14:11:14.412741+0000","last_became_active":"2024-05-28T14:11:14.411900+0000","last_became_peered":"2024-05-28T14:11:14.411900+0000","last_unstale":"2024-05-28T14:11:14.412741+0000","last_undegraded":"2024-05-28T14:11:14.412741+0000","last_fullsized":"2024-05-28T14:11:14.412741+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:10:01.780076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":324,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-28T14:11:14.400029+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:11:14.400029+0000","last_peered":"2024-05-28T14:11:14.400029+0000","last_clean":"2024-05-28T14:11:14.400029+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:11:14.400029+0000","last_undegraded":"2024-05-28T14:11:14.400029+0000","last_fullsized":"2024-05-28T14:11:14.400029+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575007,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127328,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244512,"statfs":{"total":96636764160,"available":96506380288,"internally_reserved":0,"allocated":1855488,"data_stored":1137450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126748,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245092,"statfs":{"total":96636764160,"available":96506974208,"internally_reserved":0,"allocated":1261568,"data_stored":547082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":128514502},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64300,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307540,"statfs":{"total":96636764160,"available":96570920960,"internally_reserved":0,"allocated":1273856,"data_stored":553385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86508,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285332,"statfs":{"total":96636764160,"available":96548179968,"internally_reserved":0,"allocated":1261568,"data_stored":547082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118444,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253396,"statfs":{"total":96636764160,"available":96515477504,"internally_reserved":0,"allocated":1273856,"data_stored":553385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":119994824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148968,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222872,"statfs":{"total":96636764160,"available":96484220928,"internally_reserved":0,"allocated":1273856,"data_stored":553385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130908,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240932,"statfs":{"total":96636764160,"available":96502714368,"internally_reserved":0,"allocated":1261568,"data_stored":547082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87072,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284768,"statfs":{"total":96636764160,"available":96547602432,"internally_reserved":0,"allocated":1855488,"data_stored":1137450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:11:24.121 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:11:24.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.288+0000 7fa7b2800640 1 Processor -- start 2024-05-28T14:11:24.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.288+0000 7fa7b2800640 1 -- start start 2024-05-28T14:11:24.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7b2800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7ac0cd210 0x7fa7ac169830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7b2800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7ac070740 con 0x7fa7ac0d15f0 2024-05-28T14:11:24.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7b2800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7ac0d0870 con 0x7fa7ac15e970 2024-05-28T14:11:24.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7b2800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa7ac0d0640 con 0x7fa7ac0cd210 2024-05-28T14:11:24.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7a2a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa7ac0d15f0 0x7fa7ac0ccb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39254/0 (socket says 172.21.15.42:39254) 2024-05-28T14:11:24.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7a2a00640 1 -- 172.21.15.42:0/3688678513 learned_addr learned my addr 172.21.15.42:0/3688678513 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:24.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7abe00640 1 --2- 172.21.15.42:0/3688678513 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7ac0cd210 0x7fa7ac169830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3035262238 0 0) 0x7fa7ac070740 con 0x7fa7ac0d15f0 2024-05-28T14:11:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa790003530 con 0x7fa7ac0d15f0 2024-05-28T14:11:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2130823756 0 0) 0x7fa7ac0d0870 con 0x7fa7ac15e970 2024-05-28T14:11:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7ac070740 con 0x7fa7ac15e970 2024-05-28T14:11:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3967605384 0 0) 0x7fa790003530 con 0x7fa7ac0d15f0 2024-05-28T14:11:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7ac0cd210 msgr2=0x7fa7ac169830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 --2- 172.21.15.42:0/3688678513 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7ac0cd210 0x7fa7ac169830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 >> v1:172.21.15.63:6789/0 conn(0x7fa7ac15e970 legacy=0x7fa7ac06ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ac0d0b10 con 0x7fa7ac0d15f0 2024-05-28T14:11:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fa794001680 con 0x7fa7ac0d15f0 2024-05-28T14:11:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 >> v1:172.21.15.42:6789/0 conn(0x7fa7ac0d15f0 legacy=0x7fa7ac0ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 --2- 172.21.15.42:0/3688678513 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7ac0cd210 0x7fa790003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 --2- 172.21.15.42:0/3688678513 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa790004130 0x7fa790004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 --2- 172.21.15.42:0/3688678513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa790004d50 0x7fa7900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa79000bb60 con 0x7fa790004d50 2024-05-28T14:11:24.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa79000bd10 con 0x7fa790004130 2024-05-28T14:11:24.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa79000bf10 con 0x7fa7ac0cd210 2024-05-28T14:11:24.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7abe00640 1 --2- 172.21.15.42:0/3688678513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa790004d50 0x7fa7900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7ab400640 1 --2- 172.21.15.42:0/3688678513 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7ac0cd210 0x7fa790003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7a2a00640 1 --2- 172.21.15.42:0/3688678513 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa790004130 0x7fa790004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7abe00640 1 -- 172.21.15.42:0/3688678513 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7ac0cd210 msgr2=0x7fa790003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:24.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7abe00640 1 --2- 172.21.15.42:0/3688678513 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7ac0cd210 0x7fa790003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7abe00640 1 -- 172.21.15.42:0/3688678513 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa790004130 msgr2=0x7fa790004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:24.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7abe00640 1 --2- 172.21.15.42:0/3688678513 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa790004130 0x7fa790004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.292+0000 7fa7abe00640 1 -- 172.21.15.42:0/3688678513 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa790003530 con 0x7fa790004d50 2024-05-28T14:11:24.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.296+0000 7fa7abe00640 1 --2- 172.21.15.42:0/3688678513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa790004d50 0x7fa7900051d0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009fc0 tx=0x7fa7a401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550dc5722205c6cb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:24.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.296+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a40b2070 con 0x7fa790004d50 2024-05-28T14:11:24.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.296+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a40a5c90 con 0x7fa790004d50 2024-05-28T14:11:24.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.296+0000 7fa7aaa00640 1 -- 172.21.15.42:0/3688678513 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4009c10 con 0x7fa790004d50 2024-05-28T14:11:24.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.296+0000 7fa7b2800640 1 -- 172.21.15.42:0/3688678513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa790004d50 msgr2=0x7fa7900051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:24.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.296+0000 7fa7b2800640 1 --2- 172.21.15.42:0/3688678513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa790004d50 0x7fa7900051d0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009fc0 tx=0x7fa7a401cc80 comp rx=0 tx=0).stop 2024-05-28T14:11:24.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.296+0000 7fa7abe00640 1 -- 172.21.15.42:0/3688678513 reap_dead start 2024-05-28T14:11:24.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.296+0000 7fa7b2800640 1 -- 172.21.15.42:0/3688678513 shutdown_connections 2024-05-28T14:11:24.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.296+0000 7fa7b2800640 1 -- 172.21.15.42:0/3688678513 >> 172.21.15.42:0/3688678513 conn(0x7fa7ac0b3fa0 msgr2=0x7fa7ac0b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:24.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.296+0000 7fa7b2800640 1 -- 172.21.15.42:0/3688678513 shutdown_connections 2024-05-28T14:11:24.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.296+0000 7fa7b2800640 1 -- 172.21.15.42:0/3688678513 wait complete. 2024-05-28T14:11:24.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7b2800640 1 Processor -- start 2024-05-28T14:11:24.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7b2800640 1 -- start start 2024-05-28T14:11:24.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7b2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa790004d50 0x7fa7ac14e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7b2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7ac06d670 0x7fa7ac06dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7b2800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7ac0c7b30 0x7fa7ac0c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7b2800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac0d1a50 con 0x7fa790004d50 2024-05-28T14:11:24.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7b2800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac0d1160 con 0x7fa7ac06d670 2024-05-28T14:11:24.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7b2800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa7ac15ed50 con 0x7fa7ac0c7b30 2024-05-28T14:11:24.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7ab400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa790004d50 0x7fa7ac14e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7ab400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa790004d50 0x7fa7ac14e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53388/0 (socket says 172.21.15.42:53388) 2024-05-28T14:11:24.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7ab400640 1 -- 172.21.15.42:0/573034053 learned_addr learned my addr 172.21.15.42:0/573034053 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:24.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7a2a00640 1 --2- 172.21.15.42:0/573034053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7ac06d670 0x7fa7ac06dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7abe00640 1 --2- 172.21.15.42:0/573034053 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7ac0c7b30 0x7fa7ac0c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7ab400640 1 -- 172.21.15.42:0/573034053 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7ac0c7b30 msgr2=0x7fa7ac0c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:24.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7ab400640 1 --2- 172.21.15.42:0/573034053 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7ac0c7b30 0x7fa7ac0c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7ab400640 1 -- 172.21.15.42:0/573034053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7ac06d670 msgr2=0x7fa7ac06dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:24.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7ab400640 1 --2- 172.21.15.42:0/573034053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7ac06d670 0x7fa7ac06dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7ab400640 1 -- 172.21.15.42:0/573034053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ac14f180 con 0x7fa790004d50 2024-05-28T14:11:24.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7ab400640 1 --2- 172.21.15.42:0/573034053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa790004d50 0x7fa7ac14e7b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fa79c012830 tx=0x7fa79c012cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:24.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7a3e00640 1 -- 172.21.15.42:0/573034053 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c010bd0 con 0x7fa790004d50 2024-05-28T14:11:24.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7a3e00640 1 -- 172.21.15.42:0/573034053 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa79c01b070 con 0x7fa790004d50 2024-05-28T14:11:24.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7a3e00640 1 -- 172.21.15.42:0/573034053 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c016480 con 0x7fa790004d50 2024-05-28T14:11:24.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7b2800640 1 -- 172.21.15.42:0/573034053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ac0c87f0 con 0x7fa790004d50 2024-05-28T14:11:24.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.300+0000 7fa7b2800640 1 -- 172.21.15.42:0/573034053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ac14f420 con 0x7fa790004d50 2024-05-28T14:11:24.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.304+0000 7fa7b2800640 1 -- 172.21.15.42:0/573034053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7ac068a90 con 0x7fa790004d50 2024-05-28T14:11:24.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.304+0000 7fa7a3e00640 1 -- 172.21.15.42:0/573034053 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fa79c016620 con 0x7fa790004d50 2024-05-28T14:11:24.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.304+0000 7fa7a3e00640 1 --2- 172.21.15.42:0/573034053 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa788069850 0x7fa78806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.304+0000 7fa7a3e00640 1 -- 172.21.15.42:0/573034053 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fa79c0891f0 con 0x7fa790004d50 2024-05-28T14:11:24.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.304+0000 7fa7a2a00640 1 --2- 172.21.15.42:0/573034053 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa788069850 0x7fa78806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.304+0000 7fa7a2a00640 1 --2- 172.21.15.42:0/573034053 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa788069850 0x7fa78806bd10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac1f7ac0 tx=0x7fa794001090 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:24.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.308+0000 7fa7a3e00640 1 -- 172.21.15.42:0/573034053 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa79c05a740 con 0x7fa790004d50 2024-05-28T14:11:24.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.472+0000 7fa7b2800640 1 -- 172.21.15.42:0/573034053 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa7ac14f5f0 con 0x7fa788069850 2024-05-28T14:11:24.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.476+0000 7fa7a3e00640 1 -- 172.21.15.42:0/573034053 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252502 (secure 0 0 0) 0x7fa7ac14f5f0 con 0x7fa788069850 2024-05-28T14:11:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:24.465 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:11:24.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.480+0000 7fa7b2800640 1 -- 172.21.15.42:0/573034053 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa788069850 msgr2=0x7fa78806bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:24.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.480+0000 7fa7b2800640 1 --2- 172.21.15.42:0/573034053 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa788069850 0x7fa78806bd10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac1f7ac0 tx=0x7fa794001090 comp rx=0 tx=0).stop 2024-05-28T14:11:24.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.480+0000 7fa7b2800640 1 -- 172.21.15.42:0/573034053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa790004d50 msgr2=0x7fa7ac14e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:24.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.480+0000 7fa7b2800640 1 --2- 172.21.15.42:0/573034053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa790004d50 0x7fa7ac14e7b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fa79c012830 tx=0x7fa79c012cf0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.480+0000 7fa7b2800640 1 -- 172.21.15.42:0/573034053 shutdown_connections 2024-05-28T14:11:24.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.480+0000 7fa7b2800640 1 --2- 172.21.15.42:0/573034053 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa788069850 0x7fa78806bd10 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.480+0000 7fa7b2800640 1 --2- 172.21.15.42:0/573034053 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7ac0c7b30 0x7fa7ac0c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.480+0000 7fa7b2800640 1 --2- 172.21.15.42:0/573034053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7ac06d670 0x7fa7ac06dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.480+0000 7fa7b2800640 1 --2- 172.21.15.42:0/573034053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa790004d50 0x7fa7ac14e7b0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.480+0000 7fa7b2800640 1 -- 172.21.15.42:0/573034053 >> 172.21.15.42:0/573034053 conn(0x7fa7ac0b3fa0 msgr2=0x7fa7ac0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:24.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.480+0000 7fa7b2800640 1 -- 172.21.15.42:0/573034053 shutdown_connections 2024-05-28T14:11:24.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.480+0000 7fa7b2800640 1 -- 172.21.15.42:0/573034053 wait complete. 2024-05-28T14:11:24.486 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-05-28T14:11:23.140208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":890588,"kb_used_data":11368,"kb_used_omap":12,"kb_used_meta":879091,"kb_avail":754084132,"statfs":{"total":773094113280,"available":772182151168,"internally_reserved":0,"allocated":11640832,"data_stored":5717684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":900189746},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002033"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666847+0000","last_change":"2024-05-28T14:11:15.416788+0000","last_active":"2024-05-28T14:11:16.666847+0000","last_peered":"2024-05-28T14:11:16.666847+0000","last_clean":"2024-05-28T14:11:16.666847+0000","last_became_active":"2024-05-28T14:11:15.415531+0000","last_became_peered":"2024-05-28T14:11:15.415531+0000","last_unstale":"2024-05-28T14:11:16.666847+0000","last_undegraded":"2024-05-28T14:11:16.666847+0000","last_fullsized":"2024-05-28T14:11:16.666847+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:49.733517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.404748+0000","last_change":"2024-05-28T14:11:15.411390+0000","last_active":"2024-05-28T14:11:16.404748+0000","last_peered":"2024-05-28T14:11:16.404748+0000","last_clean":"2024-05-28T14:11:16.404748+0000","last_became_active":"2024-05-28T14:11:15.410627+0000","last_became_peered":"2024-05-28T14:11:15.410627+0000","last_unstale":"2024-05-28T14:11:16.404748+0000","last_undegraded":"2024-05-28T14:11:16.404748+0000","last_fullsized":"2024-05-28T14:11:16.404748+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:28:27.176941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666728+0000","last_change":"2024-05-28T14:11:15.413591+0000","last_active":"2024-05-28T14:11:16.666728+0000","last_peered":"2024-05-28T14:11:16.666728+0000","last_clean":"2024-05-28T14:11:16.666728+0000","last_became_active":"2024-05-28T14:11:15.413323+0000","last_became_peered":"2024-05-28T14:11:15.413323+0000","last_unstale":"2024-05-28T14:11:16.666728+0000","last_undegraded":"2024-05-28T14:11:16.666728+0000","last_fullsized":"2024-05-28T14:11:16.666728+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:07:59.666971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.667143+0000","last_change":"2024-05-28T14:11:15.405849+0000","last_active":"2024-05-28T14:11:16.667143+0000","last_peered":"2024-05-28T14:11:16.667143+0000","last_clean":"2024-05-28T14:11:16.667143+0000","last_became_active":"2024-05-28T14:11:15.405546+0000","last_became_peered":"2024-05-28T14:11:15.405546+0000","last_unstale":"2024-05-28T14:11:16.667143+0000","last_undegraded":"2024-05-28T14:11:16.667143+0000","last_fullsized":"2024-05-28T14:11:16.667143+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:17:48.457739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668191+0000","last_change":"2024-05-28T14:11:15.416834+0000","last_active":"2024-05-28T14:11:16.668191+0000","last_peered":"2024-05-28T14:11:16.668191+0000","last_clean":"2024-05-28T14:11:16.668191+0000","last_became_active":"2024-05-28T14:11:15.415385+0000","last_became_peered":"2024-05-28T14:11:15.415385+0000","last_unstale":"2024-05-28T14:11:16.668191+0000","last_undegraded":"2024-05-28T14:11:16.668191+0000","last_fullsized":"2024-05-28T14:11:16.668191+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:08:13.909724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668256+0000","last_change":"2024-05-28T14:11:15.410989+0000","last_active":"2024-05-28T14:11:16.668256+0000","last_peered":"2024-05-28T14:11:16.668256+0000","last_clean":"2024-05-28T14:11:16.668256+0000","last_became_active":"2024-05-28T14:11:15.410710+0000","last_became_peered":"2024-05-28T14:11:15.410710+0000","last_unstale":"2024-05-28T14:11:16.668256+0000","last_undegraded":"2024-05-28T14:11:16.668256+0000","last_fullsized":"2024-05-28T14:11:16.668256+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:36:18.645978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.667928+0000","last_change":"2024-05-28T14:11:15.414620+0000","last_active":"2024-05-28T14:11:16.667928+0000","last_peered":"2024-05-28T14:11:16.667928+0000","last_clean":"2024-05-28T14:11:16.667928+0000","last_became_active":"2024-05-28T14:11:15.414340+0000","last_became_peered":"2024-05-28T14:11:15.414340+0000","last_unstale":"2024-05-28T14:11:16.667928+0000","last_undegraded":"2024-05-28T14:11:16.667928+0000","last_fullsized":"2024-05-28T14:11:16.667928+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:24:06.150620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.404392+0000","last_change":"2024-05-28T14:11:15.410427+0000","last_active":"2024-05-28T14:11:16.404392+0000","last_peered":"2024-05-28T14:11:16.404392+0000","last_clean":"2024-05-28T14:11:16.404392+0000","last_became_active":"2024-05-28T14:11:15.410101+0000","last_became_peered":"2024-05-28T14:11:15.410101+0000","last_unstale":"2024-05-28T14:11:16.404392+0000","last_undegraded":"2024-05-28T14:11:16.404392+0000","last_fullsized":"2024-05-28T14:11:16.404392+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:10:52.550055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.408280+0000","last_change":"2024-05-28T14:11:15.398663+0000","last_active":"2024-05-28T14:11:16.408280+0000","last_peered":"2024-05-28T14:11:16.408280+0000","last_clean":"2024-05-28T14:11:16.408280+0000","last_became_active":"2024-05-28T14:11:15.398298+0000","last_became_peered":"2024-05-28T14:11:15.398298+0000","last_unstale":"2024-05-28T14:11:16.408280+0000","last_undegraded":"2024-05-28T14:11:16.408280+0000","last_fullsized":"2024-05-28T14:11:16.408280+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:50:49.319439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668485+0000","last_change":"2024-05-28T14:11:15.414878+0000","last_active":"2024-05-28T14:11:16.668485+0000","last_peered":"2024-05-28T14:11:16.668485+0000","last_clean":"2024-05-28T14:11:16.668485+0000","last_became_active":"2024-05-28T14:11:15.413400+0000","last_became_peered":"2024-05-28T14:11:15.413400+0000","last_unstale":"2024-05-28T14:11:16.668485+0000","last_undegraded":"2024-05-28T14:11:16.668485+0000","last_fullsized":"2024-05-28T14:11:16.668485+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:06:48.078036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668518+0000","last_change":"2024-05-28T14:11:15.408947+0000","last_active":"2024-05-28T14:11:16.668518+0000","last_peered":"2024-05-28T14:11:16.668518+0000","last_clean":"2024-05-28T14:11:16.668518+0000","last_became_active":"2024-05-28T14:11:15.408633+0000","last_became_peered":"2024-05-28T14:11:15.408633+0000","last_unstale":"2024-05-28T14:11:16.668518+0000","last_undegraded":"2024-05-28T14:11:16.668518+0000","last_fullsized":"2024-05-28T14:11:16.668518+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:51:53.615220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.407810+0000","last_change":"2024-05-28T14:11:15.410773+0000","last_active":"2024-05-28T14:11:16.407810+0000","last_peered":"2024-05-28T14:11:16.407810+0000","last_clean":"2024-05-28T14:11:16.407810+0000","last_became_active":"2024-05-28T14:11:15.410489+0000","last_became_peered":"2024-05-28T14:11:15.410489+0000","last_unstale":"2024-05-28T14:11:16.407810+0000","last_undegraded":"2024-05-28T14:11:16.407810+0000","last_fullsized":"2024-05-28T14:11:16.407810+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:34:40.240897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666776+0000","last_change":"2024-05-28T14:11:15.403413+0000","last_active":"2024-05-28T14:11:16.666776+0000","last_peered":"2024-05-28T14:11:16.666776+0000","last_clean":"2024-05-28T14:11:16.666776+0000","last_became_active":"2024-05-28T14:11:15.403060+0000","last_became_peered":"2024-05-28T14:11:15.403060+0000","last_unstale":"2024-05-28T14:11:16.666776+0000","last_undegraded":"2024-05-28T14:11:16.666776+0000","last_fullsized":"2024-05-28T14:11:16.666776+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:30.473211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.420894+0000","last_change":"2024-05-28T14:11:15.417540+0000","last_active":"2024-05-28T14:11:16.420894+0000","last_peered":"2024-05-28T14:11:16.420894+0000","last_clean":"2024-05-28T14:11:16.420894+0000","last_became_active":"2024-05-28T14:11:15.417262+0000","last_became_peered":"2024-05-28T14:11:15.417262+0000","last_unstale":"2024-05-28T14:11:16.420894+0000","last_undegraded":"2024-05-28T14:11:16.420894+0000","last_fullsized":"2024-05-28T14:11:16.420894+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:34:44.753694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666283+0000","last_change":"2024-05-28T14:11:15.408616+0000","last_active":"2024-05-28T14:11:16.666283+0000","last_peered":"2024-05-28T14:11:16.666283+0000","last_clean":"2024-05-28T14:11:16.666283+0000","last_became_active":"2024-05-28T14:11:15.407798+0000","last_became_peered":"2024-05-28T14:11:15.407798+0000","last_unstale":"2024-05-28T14:11:16.666283+0000","last_undegraded":"2024-05-28T14:11:16.666283+0000","last_fullsized":"2024-05-28T14:11:16.666283+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:28:27.176941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.667811+0000","last_change":"2024-05-28T14:11:15.417567+0000","last_active":"2024-05-28T14:11:16.667811+0000","last_peered":"2024-05-28T14:11:16.667811+0000","last_clean":"2024-05-28T14:11:16.667811+0000","last_became_active":"2024-05-28T14:11:15.417230+0000","last_became_peered":"2024-05-28T14:11:15.417230+0000","last_unstale":"2024-05-28T14:11:16.667811+0000","last_undegraded":"2024-05-28T14:11:16.667811+0000","last_fullsized":"2024-05-28T14:11:16.667811+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:01:32.733969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.405129+0000","last_change":"2024-05-28T14:11:15.398008+0000","last_active":"2024-05-28T14:11:16.405129+0000","last_peered":"2024-05-28T14:11:16.405129+0000","last_clean":"2024-05-28T14:11:16.405129+0000","last_became_active":"2024-05-28T14:11:15.397738+0000","last_became_peered":"2024-05-28T14:11:15.397738+0000","last_unstale":"2024-05-28T14:11:16.405129+0000","last_undegraded":"2024-05-28T14:11:16.405129+0000","last_fullsized":"2024-05-28T14:11:16.405129+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:27:51.574967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.667125+0000","last_change":"2024-05-28T14:11:15.415840+0000","last_active":"2024-05-28T14:11:16.667125+0000","last_peered":"2024-05-28T14:11:16.667125+0000","last_clean":"2024-05-28T14:11:16.667125+0000","last_became_active":"2024-05-28T14:11:15.415443+0000","last_became_peered":"2024-05-28T14:11:15.415443+0000","last_unstale":"2024-05-28T14:11:16.667125+0000","last_undegraded":"2024-05-28T14:11:16.667125+0000","last_fullsized":"2024-05-28T14:11:16.667125+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:39:38.236554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.420998+0000","last_change":"2024-05-28T14:11:15.408488+0000","last_active":"2024-05-28T14:11:16.420998+0000","last_peered":"2024-05-28T14:11:16.420998+0000","last_clean":"2024-05-28T14:11:16.420998+0000","last_became_active":"2024-05-28T14:11:15.408213+0000","last_became_peered":"2024-05-28T14:11:15.408213+0000","last_unstale":"2024-05-28T14:11:16.420998+0000","last_undegraded":"2024-05-28T14:11:16.420998+0000","last_fullsized":"2024-05-28T14:11:16.420998+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:42:17.269858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.407840+0000","last_change":"2024-05-28T14:11:15.397537+0000","last_active":"2024-05-28T14:11:16.407840+0000","last_peered":"2024-05-28T14:11:16.407840+0000","last_clean":"2024-05-28T14:11:16.407840+0000","last_became_active":"2024-05-28T14:11:15.397165+0000","last_became_peered":"2024-05-28T14:11:15.397165+0000","last_unstale":"2024-05-28T14:11:16.407840+0000","last_undegraded":"2024-05-28T14:11:16.407840+0000","last_fullsized":"2024-05-28T14:11:16.407840+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:03:53.077265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.420839+0000","last_change":"2024-05-28T14:11:15.418456+0000","last_active":"2024-05-28T14:11:16.420839+0000","last_peered":"2024-05-28T14:11:16.420839+0000","last_clean":"2024-05-28T14:11:16.420839+0000","last_became_active":"2024-05-28T14:11:15.416255+0000","last_became_peered":"2024-05-28T14:11:15.416255+0000","last_unstale":"2024-05-28T14:11:16.420839+0000","last_undegraded":"2024-05-28T14:11:16.420839+0000","last_fullsized":"2024-05-28T14:11:16.420839+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:24:07.762471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.404392+0000","last_change":"2024-05-28T14:11:15.411090+0000","last_active":"2024-05-28T14:11:16.404392+0000","last_peered":"2024-05-28T14:11:16.404392+0000","last_clean":"2024-05-28T14:11:16.404392+0000","last_became_active":"2024-05-28T14:11:15.410796+0000","last_became_peered":"2024-05-28T14:11:15.410796+0000","last_unstale":"2024-05-28T14:11:16.404392+0000","last_undegraded":"2024-05-28T14:11:16.404392+0000","last_fullsized":"2024-05-28T14:11:16.404392+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:06:48.078036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.404380+0000","last_change":"2024-05-28T14:11:15.412667+0000","last_active":"2024-05-28T14:11:16.404380+0000","last_peered":"2024-05-28T14:11:16.404380+0000","last_clean":"2024-05-28T14:11:16.404380+0000","last_became_active":"2024-05-28T14:11:15.410508+0000","last_became_peered":"2024-05-28T14:11:15.410508+0000","last_unstale":"2024-05-28T14:11:16.404380+0000","last_undegraded":"2024-05-28T14:11:16.404380+0000","last_fullsized":"2024-05-28T14:11:16.404380+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:05:51.818642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.667983+0000","last_change":"2024-05-28T14:11:15.414188+0000","last_active":"2024-05-28T14:11:16.667983+0000","last_peered":"2024-05-28T14:11:16.667983+0000","last_clean":"2024-05-28T14:11:16.667983+0000","last_became_active":"2024-05-28T14:11:15.413907+0000","last_became_peered":"2024-05-28T14:11:15.413907+0000","last_unstale":"2024-05-28T14:11:16.667983+0000","last_undegraded":"2024-05-28T14:11:16.667983+0000","last_fullsized":"2024-05-28T14:11:16.667983+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:04:22.028199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668926+0000","last_change":"2024-05-28T14:11:15.408752+0000","last_active":"2024-05-28T14:11:16.668926+0000","last_peered":"2024-05-28T14:11:16.668926+0000","last_clean":"2024-05-28T14:11:16.668926+0000","last_became_active":"2024-05-28T14:11:15.408079+0000","last_became_peered":"2024-05-28T14:11:15.408079+0000","last_unstale":"2024-05-28T14:11:16.668926+0000","last_undegraded":"2024-05-28T14:11:16.668926+0000","last_fullsized":"2024-05-28T14:11:16.668926+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:10:52.550055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.404940+0000","last_change":"2024-05-28T14:11:15.405904+0000","last_active":"2024-05-28T14:11:16.404940+0000","last_peered":"2024-05-28T14:11:16.404940+0000","last_clean":"2024-05-28T14:11:16.404940+0000","last_became_active":"2024-05-28T14:11:15.405607+0000","last_became_peered":"2024-05-28T14:11:15.405607+0000","last_unstale":"2024-05-28T14:11:16.404940+0000","last_undegraded":"2024-05-28T14:11:16.404940+0000","last_fullsized":"2024-05-28T14:11:16.404940+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:09:07.637126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668679+0000","last_change":"2024-05-28T14:11:15.417540+0000","last_active":"2024-05-28T14:11:16.668679+0000","last_peered":"2024-05-28T14:11:16.668679+0000","last_clean":"2024-05-28T14:11:16.668679+0000","last_became_active":"2024-05-28T14:11:15.417257+0000","last_became_peered":"2024-05-28T14:11:15.417257+0000","last_unstale":"2024-05-28T14:11:16.668679+0000","last_undegraded":"2024-05-28T14:11:16.668679+0000","last_fullsized":"2024-05-28T14:11:16.668679+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:03:02.565477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668241+0000","last_change":"2024-05-28T14:11:15.416013+0000","last_active":"2024-05-28T14:11:16.668241+0000","last_peered":"2024-05-28T14:11:16.668241+0000","last_clean":"2024-05-28T14:11:16.668241+0000","last_became_active":"2024-05-28T14:11:15.415438+0000","last_became_peered":"2024-05-28T14:11:15.415438+0000","last_unstale":"2024-05-28T14:11:16.668241+0000","last_undegraded":"2024-05-28T14:11:16.668241+0000","last_fullsized":"2024-05-28T14:11:16.668241+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:10:02.786282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.404764+0000","last_change":"2024-05-28T14:11:15.407546+0000","last_active":"2024-05-28T14:11:16.404764+0000","last_peered":"2024-05-28T14:11:16.404764+0000","last_clean":"2024-05-28T14:11:16.404764+0000","last_became_active":"2024-05-28T14:11:15.407264+0000","last_became_peered":"2024-05-28T14:11:15.407264+0000","last_unstale":"2024-05-28T14:11:16.404764+0000","last_undegraded":"2024-05-28T14:11:16.404764+0000","last_fullsized":"2024-05-28T14:11:16.404764+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:17:48.457739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.420704+0000","last_change":"2024-05-28T14:11:15.407731+0000","last_active":"2024-05-28T14:11:16.420704+0000","last_peered":"2024-05-28T14:11:16.420704+0000","last_clean":"2024-05-28T14:11:16.420704+0000","last_became_active":"2024-05-28T14:11:15.407432+0000","last_became_peered":"2024-05-28T14:11:15.407432+0000","last_unstale":"2024-05-28T14:11:16.420704+0000","last_undegraded":"2024-05-28T14:11:16.420704+0000","last_fullsized":"2024-05-28T14:11:16.420704+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:49:05.725553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666407+0000","last_change":"2024-05-28T14:11:15.410927+0000","last_active":"2024-05-28T14:11:16.666407+0000","last_peered":"2024-05-28T14:11:16.666407+0000","last_clean":"2024-05-28T14:11:16.666407+0000","last_became_active":"2024-05-28T14:11:15.410642+0000","last_became_peered":"2024-05-28T14:11:15.410642+0000","last_unstale":"2024-05-28T14:11:16.666407+0000","last_undegraded":"2024-05-28T14:11:16.666407+0000","last_fullsized":"2024-05-28T14:11:16.666407+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:09:07.637126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666110+0000","last_change":"2024-05-28T14:11:15.404927+0000","last_active":"2024-05-28T14:11:16.666110+0000","last_peered":"2024-05-28T14:11:16.666110+0000","last_clean":"2024-05-28T14:11:16.666110+0000","last_became_active":"2024-05-28T14:11:15.404480+0000","last_became_peered":"2024-05-28T14:11:15.404480+0000","last_unstale":"2024-05-28T14:11:16.666110+0000","last_undegraded":"2024-05-28T14:11:16.666110+0000","last_fullsized":"2024-05-28T14:11:16.666110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:15:35.207676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.420993+0000","last_change":"2024-05-28T14:11:15.414151+0000","last_active":"2024-05-28T14:11:16.420993+0000","last_peered":"2024-05-28T14:11:16.420993+0000","last_clean":"2024-05-28T14:11:16.420993+0000","last_became_active":"2024-05-28T14:11:15.413839+0000","last_became_peered":"2024-05-28T14:11:15.413839+0000","last_unstale":"2024-05-28T14:11:16.420993+0000","last_undegraded":"2024-05-28T14:11:16.420993+0000","last_fullsized":"2024-05-28T14:11:16.420993+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:54:25.510267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"104'4","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.405225+0000","last_change":"2024-05-28T14:11:14.403379+0000","last_active":"2024-05-28T14:11:16.405225+0000","last_peered":"2024-05-28T14:11:16.405225+0000","last_clean":"2024-05-28T14:11:16.405225+0000","last_became_active":"2024-05-28T14:11:14.403054+0000","last_became_peered":"2024-05-28T14:11:14.403054+0000","last_unstale":"2024-05-28T14:11:16.405225+0000","last_undegraded":"2024-05-28T14:11:16.405225+0000","last_fullsized":"2024-05-28T14:11:16.405225+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:37:57.288531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.667226+0000","last_change":"2024-05-28T14:11:15.410961+0000","last_active":"2024-05-28T14:11:16.667226+0000","last_peered":"2024-05-28T14:11:16.667226+0000","last_clean":"2024-05-28T14:11:16.667226+0000","last_became_active":"2024-05-28T14:11:15.410380+0000","last_became_peered":"2024-05-28T14:11:15.410380+0000","last_unstale":"2024-05-28T14:11:16.667226+0000","last_undegraded":"2024-05-28T14:11:16.667226+0000","last_fullsized":"2024-05-28T14:11:16.667226+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:08.242393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668668+0000","last_change":"2024-05-28T14:11:14.413616+0000","last_active":"2024-05-28T14:11:16.668668+0000","last_peered":"2024-05-28T14:11:16.668668+0000","last_clean":"2024-05-28T14:11:16.668668+0000","last_became_active":"2024-05-28T14:11:14.413317+0000","last_became_peered":"2024-05-28T14:11:14.413317+0000","last_unstale":"2024-05-28T14:11:16.668668+0000","last_undegraded":"2024-05-28T14:11:16.668668+0000","last_fullsized":"2024-05-28T14:11:16.668668+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:06:47.071830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666557+0000","last_change":"2024-05-28T14:11:15.405725+0000","last_active":"2024-05-28T14:11:16.666557+0000","last_peered":"2024-05-28T14:11:16.666557+0000","last_clean":"2024-05-28T14:11:16.666557+0000","last_became_active":"2024-05-28T14:11:15.405448+0000","last_became_peered":"2024-05-28T14:11:15.405448+0000","last_unstale":"2024-05-28T14:11:16.666557+0000","last_undegraded":"2024-05-28T14:11:16.666557+0000","last_fullsized":"2024-05-28T14:11:16.666557+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:08:52.284155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.404873+0000","last_change":"2024-05-28T14:11:14.408679+0000","last_active":"2024-05-28T14:11:16.404873+0000","last_peered":"2024-05-28T14:11:16.404873+0000","last_clean":"2024-05-28T14:11:16.404873+0000","last_became_active":"2024-05-28T14:11:14.408384+0000","last_became_peered":"2024-05-28T14:11:14.408384+0000","last_unstale":"2024-05-28T14:11:16.404873+0000","last_undegraded":"2024-05-28T14:11:16.404873+0000","last_fullsized":"2024-05-28T14:11:16.404873+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:20:53.605695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.408072+0000","last_change":"2024-05-28T14:11:15.409378+0000","last_active":"2024-05-28T14:11:16.408072+0000","last_peered":"2024-05-28T14:11:16.408072+0000","last_clean":"2024-05-28T14:11:16.408072+0000","last_became_active":"2024-05-28T14:11:15.408386+0000","last_became_peered":"2024-05-28T14:11:15.408386+0000","last_unstale":"2024-05-28T14:11:16.408072+0000","last_undegraded":"2024-05-28T14:11:16.408072+0000","last_fullsized":"2024-05-28T14:11:16.408072+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:48:49.372118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.404763+0000","last_change":"2024-05-28T14:11:14.400435+0000","last_active":"2024-05-28T14:11:16.404763+0000","last_peered":"2024-05-28T14:11:16.404763+0000","last_clean":"2024-05-28T14:11:16.404763+0000","last_became_active":"2024-05-28T14:11:14.400157+0000","last_became_peered":"2024-05-28T14:11:14.400157+0000","last_unstale":"2024-05-28T14:11:16.404763+0000","last_undegraded":"2024-05-28T14:11:16.404763+0000","last_fullsized":"2024-05-28T14:11:16.404763+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:12:16.701181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.407896+0000","last_change":"2024-05-28T14:11:15.405199+0000","last_active":"2024-05-28T14:11:16.407896+0000","last_peered":"2024-05-28T14:11:16.407896+0000","last_clean":"2024-05-28T14:11:16.407896+0000","last_became_active":"2024-05-28T14:11:15.404724+0000","last_became_peered":"2024-05-28T14:11:15.404724+0000","last_unstale":"2024-05-28T14:11:16.407896+0000","last_undegraded":"2024-05-28T14:11:16.407896+0000","last_fullsized":"2024-05-28T14:11:16.407896+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:39:09.482383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666406+0000","last_change":"2024-05-28T14:11:14.412408+0000","last_active":"2024-05-28T14:11:16.666406+0000","last_peered":"2024-05-28T14:11:16.666406+0000","last_clean":"2024-05-28T14:11:16.666406+0000","last_became_active":"2024-05-28T14:11:14.411721+0000","last_became_peered":"2024-05-28T14:11:14.411721+0000","last_unstale":"2024-05-28T14:11:16.666406+0000","last_undegraded":"2024-05-28T14:11:16.666406+0000","last_fullsized":"2024-05-28T14:11:16.666406+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:34:43.747488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668294+0000","last_change":"2024-05-28T14:11:15.413345+0000","last_active":"2024-05-28T14:11:16.668294+0000","last_peered":"2024-05-28T14:11:16.668294+0000","last_clean":"2024-05-28T14:11:16.668294+0000","last_became_active":"2024-05-28T14:11:15.413069+0000","last_became_peered":"2024-05-28T14:11:15.413069+0000","last_unstale":"2024-05-28T14:11:16.668294+0000","last_undegraded":"2024-05-28T14:11:16.668294+0000","last_fullsized":"2024-05-28T14:11:16.668294+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:52:01.257785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.404559+0000","last_change":"2024-05-28T14:11:14.414364+0000","last_active":"2024-05-28T14:11:16.404559+0000","last_peered":"2024-05-28T14:11:16.404559+0000","last_clean":"2024-05-28T14:11:16.404559+0000","last_became_active":"2024-05-28T14:11:14.413986+0000","last_became_peered":"2024-05-28T14:11:14.413986+0000","last_unstale":"2024-05-28T14:11:16.404559+0000","last_undegraded":"2024-05-28T14:11:16.404559+0000","last_fullsized":"2024-05-28T14:11:16.404559+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:48:06.702787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666208+0000","last_change":"2024-05-28T14:11:15.406493+0000","last_active":"2024-05-28T14:11:16.666208+0000","last_peered":"2024-05-28T14:11:16.666208+0000","last_clean":"2024-05-28T14:11:16.666208+0000","last_became_active":"2024-05-28T14:11:15.406217+0000","last_became_peered":"2024-05-28T14:11:15.406217+0000","last_unstale":"2024-05-28T14:11:16.666208+0000","last_undegraded":"2024-05-28T14:11:16.666208+0000","last_fullsized":"2024-05-28T14:11:16.666208+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:10:52.550055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.667984+0000","last_change":"2024-05-28T14:11:14.406246+0000","last_active":"2024-05-28T14:11:16.667984+0000","last_peered":"2024-05-28T14:11:16.667984+0000","last_clean":"2024-05-28T14:11:16.667984+0000","last_became_active":"2024-05-28T14:11:14.405729+0000","last_became_peered":"2024-05-28T14:11:14.405729+0000","last_unstale":"2024-05-28T14:11:16.667984+0000","last_undegraded":"2024-05-28T14:11:16.667984+0000","last_fullsized":"2024-05-28T14:11:16.667984+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:12:16.701181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.406910+0000","last_change":"2024-05-28T14:11:15.408823+0000","last_active":"2024-05-28T14:11:16.406910+0000","last_peered":"2024-05-28T14:11:16.406910+0000","last_clean":"2024-05-28T14:11:16.406910+0000","last_became_active":"2024-05-28T14:11:15.408536+0000","last_became_peered":"2024-05-28T14:11:15.408536+0000","last_unstale":"2024-05-28T14:11:16.406910+0000","last_undegraded":"2024-05-28T14:11:16.406910+0000","last_fullsized":"2024-05-28T14:11:16.406910+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:25:41.432008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666209+0000","last_change":"2024-05-28T14:11:14.413285+0000","last_active":"2024-05-28T14:11:16.666209+0000","last_peered":"2024-05-28T14:11:16.666209+0000","last_clean":"2024-05-28T14:11:16.666209+0000","last_became_active":"2024-05-28T14:11:14.413010+0000","last_became_peered":"2024-05-28T14:11:14.413010+0000","last_unstale":"2024-05-28T14:11:16.666209+0000","last_undegraded":"2024-05-28T14:11:16.666209+0000","last_fullsized":"2024-05-28T14:11:16.666209+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:49:04.719347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.405204+0000","last_change":"2024-05-28T14:11:15.396776+0000","last_active":"2024-05-28T14:11:16.405204+0000","last_peered":"2024-05-28T14:11:16.405204+0000","last_clean":"2024-05-28T14:11:16.405204+0000","last_became_active":"2024-05-28T14:11:15.396329+0000","last_became_peered":"2024-05-28T14:11:15.396329+0000","last_unstale":"2024-05-28T14:11:16.405204+0000","last_undegraded":"2024-05-28T14:11:16.405204+0000","last_fullsized":"2024-05-28T14:11:16.405204+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:39:38.236554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666875+0000","last_change":"2024-05-28T14:11:14.406962+0000","last_active":"2024-05-28T14:11:16.666875+0000","last_peered":"2024-05-28T14:11:16.666875+0000","last_clean":"2024-05-28T14:11:16.666875+0000","last_became_active":"2024-05-28T14:11:14.406694+0000","last_became_peered":"2024-05-28T14:11:14.406694+0000","last_unstale":"2024-05-28T14:11:16.666875+0000","last_undegraded":"2024-05-28T14:11:16.666875+0000","last_fullsized":"2024-05-28T14:11:16.666875+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:14:34.183606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.408321+0000","last_change":"2024-05-28T14:11:15.410349+0000","last_active":"2024-05-28T14:11:16.408321+0000","last_peered":"2024-05-28T14:11:16.408321+0000","last_clean":"2024-05-28T14:11:16.408321+0000","last_became_active":"2024-05-28T14:11:15.408446+0000","last_became_peered":"2024-05-28T14:11:15.408446+0000","last_unstale":"2024-05-28T14:11:16.408321+0000","last_undegraded":"2024-05-28T14:11:16.408321+0000","last_fullsized":"2024-05-28T14:11:16.408321+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:08:13.909724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.667095+0000","last_change":"2024-05-28T14:11:14.403779+0000","last_active":"2024-05-28T14:11:16.667095+0000","last_peered":"2024-05-28T14:11:16.667095+0000","last_clean":"2024-05-28T14:11:16.667095+0000","last_became_active":"2024-05-28T14:11:14.403497+0000","last_became_peered":"2024-05-28T14:11:14.403497+0000","last_unstale":"2024-05-28T14:11:16.667095+0000","last_undegraded":"2024-05-28T14:11:16.667095+0000","last_fullsized":"2024-05-28T14:11:16.667095+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:34:39.234692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668273+0000","last_change":"2024-05-28T14:11:15.417224+0000","last_active":"2024-05-28T14:11:16.668273+0000","last_peered":"2024-05-28T14:11:16.668273+0000","last_clean":"2024-05-28T14:11:16.668273+0000","last_became_active":"2024-05-28T14:11:15.416949+0000","last_became_peered":"2024-05-28T14:11:15.416949+0000","last_unstale":"2024-05-28T14:11:16.668273+0000","last_undegraded":"2024-05-28T14:11:16.668273+0000","last_fullsized":"2024-05-28T14:11:16.668273+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:53:43.898237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668819+0000","last_change":"2024-05-28T14:11:14.407272+0000","last_active":"2024-05-28T14:11:16.668819+0000","last_peered":"2024-05-28T14:11:16.668819+0000","last_clean":"2024-05-28T14:11:16.668819+0000","last_became_active":"2024-05-28T14:11:14.406878+0000","last_became_peered":"2024-05-28T14:11:14.406878+0000","last_unstale":"2024-05-28T14:11:16.668819+0000","last_undegraded":"2024-05-28T14:11:16.668819+0000","last_fullsized":"2024-05-28T14:11:16.668819+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:10:51.543850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668689+0000","last_change":"2024-05-28T14:11:15.411661+0000","last_active":"2024-05-28T14:11:16.668689+0000","last_peered":"2024-05-28T14:11:16.668689+0000","last_clean":"2024-05-28T14:11:16.668689+0000","last_became_active":"2024-05-28T14:11:15.411343+0000","last_became_peered":"2024-05-28T14:11:15.411343+0000","last_unstale":"2024-05-28T14:11:16.668689+0000","last_undegraded":"2024-05-28T14:11:16.668689+0000","last_fullsized":"2024-05-28T14:11:16.668689+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:23:17.179526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":20,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.407917+0000","last_change":"2024-05-28T14:11:14.408531+0000","last_active":"2024-05-28T14:11:16.407917+0000","last_peered":"2024-05-28T14:11:16.407917+0000","last_clean":"2024-05-28T14:11:16.407917+0000","last_became_active":"2024-05-28T14:11:14.408165+0000","last_became_peered":"2024-05-28T14:11:14.408165+0000","last_unstale":"2024-05-28T14:11:16.407917+0000","last_undegraded":"2024-05-28T14:11:16.407917+0000","last_fullsized":"2024-05-28T14:11:16.407917+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:48:06.702787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666781+0000","last_change":"2024-05-28T14:11:15.409403+0000","last_active":"2024-05-28T14:11:16.666781+0000","last_peered":"2024-05-28T14:11:16.666781+0000","last_clean":"2024-05-28T14:11:16.666781+0000","last_became_active":"2024-05-28T14:11:15.409151+0000","last_became_peered":"2024-05-28T14:11:15.409151+0000","last_unstale":"2024-05-28T14:11:16.666781+0000","last_undegraded":"2024-05-28T14:11:16.666781+0000","last_fullsized":"2024-05-28T14:11:16.666781+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:27:51.574967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.421105+0000","last_change":"2024-05-28T14:11:14.403933+0000","last_active":"2024-05-28T14:11:16.421105+0000","last_peered":"2024-05-28T14:11:16.421105+0000","last_clean":"2024-05-28T14:11:16.421105+0000","last_became_active":"2024-05-28T14:11:14.403689+0000","last_became_peered":"2024-05-28T14:11:14.403689+0000","last_unstale":"2024-05-28T14:11:16.421105+0000","last_undegraded":"2024-05-28T14:11:16.421105+0000","last_fullsized":"2024-05-28T14:11:16.421105+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:25:27.442033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666546+0000","last_change":"2024-05-28T14:11:15.416143+0000","last_active":"2024-05-28T14:11:16.666546+0000","last_peered":"2024-05-28T14:11:16.666546+0000","last_clean":"2024-05-28T14:11:16.666546+0000","last_became_active":"2024-05-28T14:11:15.415865+0000","last_became_peered":"2024-05-28T14:11:15.415865+0000","last_unstale":"2024-05-28T14:11:16.666546+0000","last_undegraded":"2024-05-28T14:11:16.666546+0000","last_fullsized":"2024-05-28T14:11:16.666546+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:14:08.954035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.421010+0000","last_change":"2024-05-28T14:11:14.416004+0000","last_active":"2024-05-28T14:11:16.421010+0000","last_peered":"2024-05-28T14:11:16.421010+0000","last_clean":"2024-05-28T14:11:16.421010+0000","last_became_active":"2024-05-28T14:11:14.415194+0000","last_became_peered":"2024-05-28T14:11:14.415194+0000","last_unstale":"2024-05-28T14:11:16.421010+0000","last_undegraded":"2024-05-28T14:11:16.421010+0000","last_fullsized":"2024-05-28T14:11:16.421010+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:04:30.445491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.406876+0000","last_change":"2024-05-28T14:11:15.398048+0000","last_active":"2024-05-28T14:11:16.406876+0000","last_peered":"2024-05-28T14:11:16.406876+0000","last_clean":"2024-05-28T14:11:16.406876+0000","last_became_active":"2024-05-28T14:11:15.397731+0000","last_became_peered":"2024-05-28T14:11:15.397731+0000","last_unstale":"2024-05-28T14:11:16.406876+0000","last_undegraded":"2024-05-28T14:11:16.406876+0000","last_fullsized":"2024-05-28T14:11:16.406876+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:50:51.681971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.406892+0000","last_change":"2024-05-28T14:11:14.401589+0000","last_active":"2024-05-28T14:11:16.406892+0000","last_peered":"2024-05-28T14:11:16.406892+0000","last_clean":"2024-05-28T14:11:16.406892+0000","last_became_active":"2024-05-28T14:11:14.401285+0000","last_became_peered":"2024-05-28T14:11:14.401285+0000","last_unstale":"2024-05-28T14:11:16.406892+0000","last_undegraded":"2024-05-28T14:11:16.406892+0000","last_fullsized":"2024-05-28T14:11:16.406892+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:54:37.950887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.420689+0000","last_change":"2024-05-28T14:11:15.408251+0000","last_active":"2024-05-28T14:11:16.420689+0000","last_peered":"2024-05-28T14:11:16.420689+0000","last_clean":"2024-05-28T14:11:16.420689+0000","last_became_active":"2024-05-28T14:11:15.407987+0000","last_became_peered":"2024-05-28T14:11:15.407987+0000","last_unstale":"2024-05-28T14:11:16.420689+0000","last_undegraded":"2024-05-28T14:11:16.420689+0000","last_fullsized":"2024-05-28T14:11:16.420689+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:03:12.382708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":22,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666229+0000","last_change":"2024-05-28T14:11:14.408691+0000","last_active":"2024-05-28T14:11:16.666229+0000","last_peered":"2024-05-28T14:11:16.666229+0000","last_clean":"2024-05-28T14:11:16.666229+0000","last_became_active":"2024-05-28T14:11:14.407478+0000","last_became_peered":"2024-05-28T14:11:14.407478+0000","last_unstale":"2024-05-28T14:11:16.666229+0000","last_undegraded":"2024-05-28T14:11:16.666229+0000","last_fullsized":"2024-05-28T14:11:16.666229+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:03:11.376502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.669074+0000","last_change":"2024-05-28T14:11:15.419321+0000","last_active":"2024-05-28T14:11:16.669074+0000","last_peered":"2024-05-28T14:11:16.669074+0000","last_clean":"2024-05-28T14:11:16.669074+0000","last_became_active":"2024-05-28T14:11:15.419037+0000","last_became_peered":"2024-05-28T14:11:15.419037+0000","last_unstale":"2024-05-28T14:11:16.669074+0000","last_undegraded":"2024-05-28T14:11:16.669074+0000","last_fullsized":"2024-05-28T14:11:16.669074+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:28:27.176941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.407787+0000","last_change":"2024-05-28T14:11:14.398527+0000","last_active":"2024-05-28T14:11:16.407787+0000","last_peered":"2024-05-28T14:11:16.407787+0000","last_clean":"2024-05-28T14:11:16.407787+0000","last_became_active":"2024-05-28T14:11:14.398239+0000","last_became_peered":"2024-05-28T14:11:14.398239+0000","last_unstale":"2024-05-28T14:11:16.407787+0000","last_undegraded":"2024-05-28T14:11:16.407787+0000","last_fullsized":"2024-05-28T14:11:16.407787+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:20:37.860587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.407529+0000","last_change":"2024-05-28T14:11:15.419244+0000","last_active":"2024-05-28T14:11:16.407529+0000","last_peered":"2024-05-28T14:11:16.407529+0000","last_clean":"2024-05-28T14:11:16.407529+0000","last_became_active":"2024-05-28T14:11:15.418923+0000","last_became_peered":"2024-05-28T14:11:15.418923+0000","last_unstale":"2024-05-28T14:11:16.407529+0000","last_undegraded":"2024-05-28T14:11:16.407529+0000","last_fullsized":"2024-05-28T14:11:16.407529+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:46:00.287246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668530+0000","last_change":"2024-05-28T14:11:14.404603+0000","last_active":"2024-05-28T14:11:16.668530+0000","last_peered":"2024-05-28T14:11:16.668530+0000","last_clean":"2024-05-28T14:11:16.668530+0000","last_became_active":"2024-05-28T14:11:14.403710+0000","last_became_peered":"2024-05-28T14:11:14.403710+0000","last_unstale":"2024-05-28T14:11:16.668530+0000","last_undegraded":"2024-05-28T14:11:16.668530+0000","last_fullsized":"2024-05-28T14:11:16.668530+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:45:59.281040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668894+0000","last_change":"2024-05-28T14:11:15.409641+0000","last_active":"2024-05-28T14:11:16.668894+0000","last_peered":"2024-05-28T14:11:16.668894+0000","last_clean":"2024-05-28T14:11:16.668894+0000","last_became_active":"2024-05-28T14:11:15.408940+0000","last_became_peered":"2024-05-28T14:11:15.408940+0000","last_unstale":"2024-05-28T14:11:16.668894+0000","last_undegraded":"2024-05-28T14:11:16.668894+0000","last_fullsized":"2024-05-28T14:11:16.668894+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:02:55.580697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.404816+0000","last_change":"2024-05-28T14:11:14.399470+0000","last_active":"2024-05-28T14:11:16.404816+0000","last_peered":"2024-05-28T14:11:16.404816+0000","last_clean":"2024-05-28T14:11:16.404816+0000","last_became_active":"2024-05-28T14:11:14.397748+0000","last_became_peered":"2024-05-28T14:11:14.397748+0000","last_unstale":"2024-05-28T14:11:16.404816+0000","last_undegraded":"2024-05-28T14:11:16.404816+0000","last_fullsized":"2024-05-28T14:11:16.404816+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:58:44.956031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.420894+0000","last_change":"2024-05-28T14:11:15.404828+0000","last_active":"2024-05-28T14:11:16.420894+0000","last_peered":"2024-05-28T14:11:16.420894+0000","last_clean":"2024-05-28T14:11:16.420894+0000","last_became_active":"2024-05-28T14:11:15.404526+0000","last_became_peered":"2024-05-28T14:11:15.404526+0000","last_unstale":"2024-05-28T14:11:16.420894+0000","last_undegraded":"2024-05-28T14:11:16.420894+0000","last_fullsized":"2024-05-28T14:11:16.420894+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:14:35.189812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.404703+0000","last_change":"2024-05-28T14:11:14.409172+0000","last_active":"2024-05-28T14:11:16.404703+0000","last_peered":"2024-05-28T14:11:16.404703+0000","last_clean":"2024-05-28T14:11:16.404703+0000","last_became_active":"2024-05-28T14:11:14.408866+0000","last_became_peered":"2024-05-28T14:11:14.408866+0000","last_unstale":"2024-05-28T14:11:16.404703+0000","last_undegraded":"2024-05-28T14:11:16.404703+0000","last_fullsized":"2024-05-28T14:11:16.404703+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:20:37.860587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.421200+0000","last_change":"2024-05-28T14:11:15.414978+0000","last_active":"2024-05-28T14:11:16.421200+0000","last_peered":"2024-05-28T14:11:16.421200+0000","last_clean":"2024-05-28T14:11:16.421200+0000","last_became_active":"2024-05-28T14:11:15.414692+0000","last_became_peered":"2024-05-28T14:11:15.414692+0000","last_unstale":"2024-05-28T14:11:16.421200+0000","last_undegraded":"2024-05-28T14:11:16.421200+0000","last_fullsized":"2024-05-28T14:11:16.421200+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:06:44.512001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.408001+0000","last_change":"2024-05-28T14:11:14.410214+0000","last_active":"2024-05-28T14:11:16.408001+0000","last_peered":"2024-05-28T14:11:16.408001+0000","last_clean":"2024-05-28T14:11:16.408001+0000","last_became_active":"2024-05-28T14:11:14.409906+0000","last_became_peered":"2024-05-28T14:11:14.409906+0000","last_unstale":"2024-05-28T14:11:16.408001+0000","last_undegraded":"2024-05-28T14:11:16.408001+0000","last_fullsized":"2024-05-28T14:11:16.408001+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:30.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.420664+0000","last_change":"2024-05-28T14:11:15.409511+0000","last_active":"2024-05-28T14:11:16.420664+0000","last_peered":"2024-05-28T14:11:16.420664+0000","last_clean":"2024-05-28T14:11:16.420664+0000","last_became_active":"2024-05-28T14:11:15.409241+0000","last_became_peered":"2024-05-28T14:11:15.409241+0000","last_unstale":"2024-05-28T14:11:16.420664+0000","last_undegraded":"2024-05-28T14:11:16.420664+0000","last_fullsized":"2024-05-28T14:11:16.420664+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:36:35.795670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.407316+0000","last_change":"2024-05-28T14:11:14.411163+0000","last_active":"2024-05-28T14:11:16.407316+0000","last_peered":"2024-05-28T14:11:16.407316+0000","last_clean":"2024-05-28T14:11:16.407316+0000","last_became_active":"2024-05-28T14:11:14.410761+0000","last_became_peered":"2024-05-28T14:11:14.410761+0000","last_unstale":"2024-05-28T14:11:16.407316+0000","last_undegraded":"2024-05-28T14:11:16.407316+0000","last_fullsized":"2024-05-28T14:11:16.407316+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:54:37.950887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.667772+0000","last_change":"2024-05-28T14:11:15.418537+0000","last_active":"2024-05-28T14:11:16.667772+0000","last_peered":"2024-05-28T14:11:16.667772+0000","last_clean":"2024-05-28T14:11:16.667772+0000","last_became_active":"2024-05-28T14:11:15.416807+0000","last_became_peered":"2024-05-28T14:11:15.416807+0000","last_unstale":"2024-05-28T14:11:16.667772+0000","last_undegraded":"2024-05-28T14:11:16.667772+0000","last_fullsized":"2024-05-28T14:11:16.667772+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:48:49.372118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.404333+0000","last_change":"2024-05-28T14:11:14.411437+0000","last_active":"2024-05-28T14:11:16.404333+0000","last_peered":"2024-05-28T14:11:16.404333+0000","last_clean":"2024-05-28T14:11:16.404333+0000","last_became_active":"2024-05-28T14:11:14.411081+0000","last_became_peered":"2024-05-28T14:11:14.411081+0000","last_unstale":"2024-05-28T14:11:16.404333+0000","last_undegraded":"2024-05-28T14:11:16.404333+0000","last_fullsized":"2024-05-28T14:11:16.404333+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:54:37.950887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.405094+0000","last_change":"2024-05-28T14:11:15.411452+0000","last_active":"2024-05-28T14:11:16.405094+0000","last_peered":"2024-05-28T14:11:16.405094+0000","last_clean":"2024-05-28T14:11:16.405094+0000","last_became_active":"2024-05-28T14:11:15.411170+0000","last_became_peered":"2024-05-28T14:11:15.411170+0000","last_unstale":"2024-05-28T14:11:16.405094+0000","last_undegraded":"2024-05-28T14:11:16.405094+0000","last_fullsized":"2024-05-28T14:11:16.405094+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:30.473211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.407650+0000","last_change":"2024-05-28T14:11:14.397391+0000","last_active":"2024-05-28T14:11:16.407650+0000","last_peered":"2024-05-28T14:11:16.407650+0000","last_clean":"2024-05-28T14:11:16.407650+0000","last_became_active":"2024-05-28T14:11:14.397118+0000","last_became_peered":"2024-05-28T14:11:14.397118+0000","last_unstale":"2024-05-28T14:11:16.407650+0000","last_undegraded":"2024-05-28T14:11:16.407650+0000","last_fullsized":"2024-05-28T14:11:16.407650+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:48:06.702787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666734+0000","last_change":"2024-05-28T14:11:15.407716+0000","last_active":"2024-05-28T14:11:16.666734+0000","last_peered":"2024-05-28T14:11:16.666734+0000","last_clean":"2024-05-28T14:11:16.666734+0000","last_became_active":"2024-05-28T14:11:15.407420+0000","last_became_peered":"2024-05-28T14:11:15.407420+0000","last_unstale":"2024-05-28T14:11:16.666734+0000","last_undegraded":"2024-05-28T14:11:16.666734+0000","last_fullsized":"2024-05-28T14:11:16.666734+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:28:23.027036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"105'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.405269+0000","last_change":"2024-05-28T14:11:14.409506+0000","last_active":"2024-05-28T14:11:16.405269+0000","last_peered":"2024-05-28T14:11:16.405269+0000","last_clean":"2024-05-28T14:11:16.405269+0000","last_became_active":"2024-05-28T14:11:14.407782+0000","last_became_peered":"2024-05-28T14:11:14.407782+0000","last_unstale":"2024-05-28T14:11:16.405269+0000","last_undegraded":"2024-05-28T14:11:16.405269+0000","last_fullsized":"2024-05-28T14:11:16.405269+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:42:29.290772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666516+0000","last_change":"2024-05-28T14:11:15.417974+0000","last_active":"2024-05-28T14:11:16.666516+0000","last_peered":"2024-05-28T14:11:16.666516+0000","last_clean":"2024-05-28T14:11:16.666516+0000","last_became_active":"2024-05-28T14:11:15.416515+0000","last_became_peered":"2024-05-28T14:11:15.416515+0000","last_unstale":"2024-05-28T14:11:16.666516+0000","last_undegraded":"2024-05-28T14:11:16.666516+0000","last_fullsized":"2024-05-28T14:11:16.666516+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:32:27.789381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.404889+0000","last_change":"2024-05-28T14:11:14.396514+0000","last_active":"2024-05-28T14:11:16.404889+0000","last_peered":"2024-05-28T14:11:16.404889+0000","last_clean":"2024-05-28T14:11:16.404889+0000","last_became_active":"2024-05-28T14:11:14.396179+0000","last_became_peered":"2024-05-28T14:11:14.396179+0000","last_unstale":"2024-05-28T14:11:16.404889+0000","last_undegraded":"2024-05-28T14:11:16.404889+0000","last_fullsized":"2024-05-28T14:11:16.404889+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:39:08.476177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668436+0000","last_change":"2024-05-28T14:11:15.414629+0000","last_active":"2024-05-28T14:11:16.668436+0000","last_peered":"2024-05-28T14:11:16.668436+0000","last_clean":"2024-05-28T14:11:16.668436+0000","last_became_active":"2024-05-28T14:11:15.414354+0000","last_became_peered":"2024-05-28T14:11:15.414354+0000","last_unstale":"2024-05-28T14:11:16.668436+0000","last_undegraded":"2024-05-28T14:11:16.668436+0000","last_fullsized":"2024-05-28T14:11:16.668436+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:05:51.818642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668342+0000","last_change":"2024-05-28T14:11:14.405416+0000","last_active":"2024-05-28T14:11:16.668342+0000","last_peered":"2024-05-28T14:11:16.668342+0000","last_clean":"2024-05-28T14:11:16.668342+0000","last_became_active":"2024-05-28T14:11:14.405162+0000","last_became_peered":"2024-05-28T14:11:14.405162+0000","last_unstale":"2024-05-28T14:11:16.668342+0000","last_undegraded":"2024-05-28T14:11:16.668342+0000","last_fullsized":"2024-05-28T14:11:16.668342+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:39:08.476177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.421067+0000","last_change":"2024-05-28T14:11:15.404246+0000","last_active":"2024-05-28T14:11:16.421067+0000","last_peered":"2024-05-28T14:11:16.421067+0000","last_clean":"2024-05-28T14:11:16.421067+0000","last_became_active":"2024-05-28T14:11:15.403920+0000","last_became_peered":"2024-05-28T14:11:15.403920+0000","last_unstale":"2024-05-28T14:11:16.421067+0000","last_undegraded":"2024-05-28T14:11:16.421067+0000","last_fullsized":"2024-05-28T14:11:16.421067+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:55:02.934164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.405063+0000","last_change":"2024-05-28T14:11:14.408550+0000","last_active":"2024-05-28T14:11:16.405063+0000","last_peered":"2024-05-28T14:11:16.405063+0000","last_clean":"2024-05-28T14:11:16.405063+0000","last_became_active":"2024-05-28T14:11:14.408274+0000","last_became_peered":"2024-05-28T14:11:14.408274+0000","last_unstale":"2024-05-28T14:11:16.405063+0000","last_undegraded":"2024-05-28T14:11:16.405063+0000","last_fullsized":"2024-05-28T14:11:16.405063+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:45:59.281040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.407501+0000","last_change":"2024-05-28T14:11:15.411228+0000","last_active":"2024-05-28T14:11:16.407501+0000","last_peered":"2024-05-28T14:11:16.407501+0000","last_clean":"2024-05-28T14:11:16.407501+0000","last_became_active":"2024-05-28T14:11:15.410901+0000","last_became_peered":"2024-05-28T14:11:15.410901+0000","last_unstale":"2024-05-28T14:11:16.407501+0000","last_undegraded":"2024-05-28T14:11:16.407501+0000","last_fullsized":"2024-05-28T14:11:16.407501+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:53:19.235355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.667771+0000","last_change":"2024-05-28T14:11:14.414517+0000","last_active":"2024-05-28T14:11:16.667771+0000","last_peered":"2024-05-28T14:11:16.667771+0000","last_clean":"2024-05-28T14:11:16.667771+0000","last_became_active":"2024-05-28T14:11:14.414145+0000","last_became_peered":"2024-05-28T14:11:14.414145+0000","last_unstale":"2024-05-28T14:11:16.667771+0000","last_undegraded":"2024-05-28T14:11:16.667771+0000","last_fullsized":"2024-05-28T14:11:16.667771+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:20:53.605695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.420661+0000","last_change":"2024-05-28T14:11:15.409330+0000","last_active":"2024-05-28T14:11:16.420661+0000","last_peered":"2024-05-28T14:11:16.420661+0000","last_clean":"2024-05-28T14:11:16.420661+0000","last_became_active":"2024-05-28T14:11:15.409066+0000","last_became_peered":"2024-05-28T14:11:15.409066+0000","last_unstale":"2024-05-28T14:11:16.420661+0000","last_undegraded":"2024-05-28T14:11:16.420661+0000","last_fullsized":"2024-05-28T14:11:16.420661+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:38:06.150233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.666414+0000","last_change":"2024-05-28T14:11:14.413007+0000","last_active":"2024-05-28T14:11:16.666414+0000","last_peered":"2024-05-28T14:11:16.666414+0000","last_clean":"2024-05-28T14:11:16.666414+0000","last_became_active":"2024-05-28T14:11:14.412715+0000","last_became_peered":"2024-05-28T14:11:14.412715+0000","last_unstale":"2024-05-28T14:11:16.666414+0000","last_undegraded":"2024-05-28T14:11:16.666414+0000","last_fullsized":"2024-05-28T14:11:16.666414+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:53:18.229149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.667975+0000","last_change":"2024-05-28T14:11:15.418173+0000","last_active":"2024-05-28T14:11:16.667975+0000","last_peered":"2024-05-28T14:11:16.667975+0000","last_clean":"2024-05-28T14:11:16.667975+0000","last_became_active":"2024-05-28T14:11:15.417875+0000","last_became_peered":"2024-05-28T14:11:15.417875+0000","last_unstale":"2024-05-28T14:11:16.667975+0000","last_undegraded":"2024-05-28T14:11:16.667975+0000","last_fullsized":"2024-05-28T14:11:16.667975+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:50:49.319439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.420789+0000","last_change":"2024-05-28T14:11:14.407608+0000","last_active":"2024-05-28T14:11:16.420789+0000","last_peered":"2024-05-28T14:11:16.420789+0000","last_clean":"2024-05-28T14:11:16.420789+0000","last_became_active":"2024-05-28T14:11:14.407327+0000","last_became_peered":"2024-05-28T14:11:14.407327+0000","last_unstale":"2024-05-28T14:11:16.420789+0000","last_undegraded":"2024-05-28T14:11:16.420789+0000","last_fullsized":"2024-05-28T14:11:16.420789+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:46:45.246601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.421165+0000","last_change":"2024-05-28T14:11:15.414732+0000","last_active":"2024-05-28T14:11:16.421165+0000","last_peered":"2024-05-28T14:11:16.421165+0000","last_clean":"2024-05-28T14:11:16.421165+0000","last_became_active":"2024-05-28T14:11:15.414437+0000","last_became_peered":"2024-05-28T14:11:15.414437+0000","last_unstale":"2024-05-28T14:11:16.421165+0000","last_undegraded":"2024-05-28T14:11:16.421165+0000","last_fullsized":"2024-05-28T14:11:16.421165+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:14.375217+0000","last_clean_scrub_stamp":"2024-05-28T14:11:14.375217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:49:26.020495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668704+0000","last_change":"2024-05-28T14:11:14.412233+0000","last_active":"2024-05-28T14:11:16.668704+0000","last_peered":"2024-05-28T14:11:16.668704+0000","last_clean":"2024-05-28T14:11:16.668704+0000","last_became_active":"2024-05-28T14:11:14.411900+0000","last_became_peered":"2024-05-28T14:11:14.411900+0000","last_unstale":"2024-05-28T14:11:16.668704+0000","last_undegraded":"2024-05-28T14:11:16.668704+0000","last_fullsized":"2024-05-28T14:11:16.668704+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:11:13.369011+0000","last_clean_scrub_stamp":"2024-05-28T14:11:13.369011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:05:50.812436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":328,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-28T14:11:16.668111+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:11:16.668111+0000","last_peered":"2024-05-28T14:11:16.668111+0000","last_clean":"2024-05-28T14:11:16.668111+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:11:16.668111+0000","last_undegraded":"2024-05-28T14:11:16.668111+0000","last_fullsized":"2024-05-28T14:11:16.668111+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575008,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127372,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244468,"statfs":{"total":96636764160,"available":96506335232,"internally_reserved":0,"allocated":1900544,"data_stored":1156996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575008,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126796,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245044,"statfs":{"total":96636764160,"available":96506925056,"internally_reserved":0,"allocated":1306624,"data_stored":567168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":128514502},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575010,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64324,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307516,"statfs":{"total":96636764160,"available":96570896384,"internally_reserved":0,"allocated":1302528,"data_stored":566586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86556,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285284,"statfs":{"total":96636764160,"available":96548130816,"internally_reserved":0,"allocated":1318912,"data_stored":568252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575008,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118480,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253360,"statfs":{"total":96636764160,"available":96515440640,"internally_reserved":0,"allocated":1314816,"data_stored":568150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":119994824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575009,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149004,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222836,"statfs":{"total":96636764160,"available":96484184064,"internally_reserved":0,"allocated":1302528,"data_stored":566550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130944,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240896,"statfs":{"total":96636764160,"available":96502677504,"internally_reserved":0,"allocated":1298432,"data_stored":566516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575009,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87112,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284728,"statfs":{"total":96636764160,"available":96547561472,"internally_reserved":0,"allocated":1896448,"data_stored":1157466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:11:24.490 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-28T14:11:24.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-28T14:11:24.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.696+0000 7ff43cc00640 1 Processor -- start 2024-05-28T14:11:24.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.696+0000 7ff43cc00640 1 -- start start 2024-05-28T14:11:24.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff43cc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff438144f80 0x7ff438072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff43cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff43806ee20 0x7ff43806f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff43cc00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff43815b9b0 0x7ff43815be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff43cc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff43806f7c0 con 0x7ff438144f80 2024-05-28T14:11:24.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff43cc00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff4380724d0 con 0x7ff43815b9b0 2024-05-28T14:11:24.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff43cc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff4380afe40 con 0x7ff43806ee20 2024-05-28T14:11:24.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff435e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff438144f80 0x7ff438072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff435e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff438144f80 0x7ff438072960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53390/0 (socket says 172.21.15.42:53390) 2024-05-28T14:11:24.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff435e00640 1 -- 172.21.15.42:0/3515157685 learned_addr learned my addr 172.21.15.42:0/3515157685 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:24.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff42d400640 1 --2- 172.21.15.42:0/3515157685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff43806ee20 0x7ff43806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff436800640 1 --2- 172.21.15.42:0/3515157685 >> v2:172.21.15.63:3300/0 conn(0x7ff43815b9b0 0x7ff43815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff435e00640 1 -- 172.21.15.42:0/3515157685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff43806ee20 msgr2=0x7ff43806f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:24.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff435e00640 1 --2- 172.21.15.42:0/3515157685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff43806ee20 0x7ff43806f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff435e00640 1 -- 172.21.15.42:0/3515157685 >> v2:172.21.15.63:3300/0 conn(0x7ff43815b9b0 msgr2=0x7ff43815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:24.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff435e00640 1 --2- 172.21.15.42:0/3515157685 >> v2:172.21.15.63:3300/0 conn(0x7ff43815b9b0 0x7ff43815be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.700+0000 7ff435e00640 1 -- 172.21.15.42:0/3515157685 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43815c380 con 0x7ff438144f80 2024-05-28T14:11:24.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff435e00640 1 --2- 172.21.15.42:0/3515157685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff438144f80 0x7ff438072960 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff4380ce080 tx=0x7ff42801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f0b84fc5e8fef1 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:24.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff435400640 1 -- 172.21.15.42:0/3515157685 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4280b1070 con 0x7ff438144f80 2024-05-28T14:11:24.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff435400640 1 -- 172.21.15.42:0/3515157685 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4280ac630 con 0x7ff438144f80 2024-05-28T14:11:24.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff435400640 1 -- 172.21.15.42:0/3515157685 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4280a8050 con 0x7ff438144f80 2024-05-28T14:11:24.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 -- 172.21.15.42:0/3515157685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff438144f80 msgr2=0x7ff438072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:24.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 --2- 172.21.15.42:0/3515157685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff438144f80 0x7ff438072960 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff4380ce080 tx=0x7ff42801c520 comp rx=0 tx=0).stop 2024-05-28T14:11:24.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 -- 172.21.15.42:0/3515157685 shutdown_connections 2024-05-28T14:11:24.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 --2- 172.21.15.42:0/3515157685 >> v2:172.21.15.63:3300/0 conn(0x7ff43815b9b0 0x7ff43815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 --2- 172.21.15.42:0/3515157685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff43806ee20 0x7ff43806f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 --2- 172.21.15.42:0/3515157685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff438144f80 0x7ff438072960 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 -- 172.21.15.42:0/3515157685 >> 172.21.15.42:0/3515157685 conn(0x7ff4380b21f0 msgr2=0x7ff4380b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:24.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 -- 172.21.15.42:0/3515157685 shutdown_connections 2024-05-28T14:11:24.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 -- 172.21.15.42:0/3515157685 wait complete. 2024-05-28T14:11:24.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 Processor -- start 2024-05-28T14:11:24.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 -- start start 2024-05-28T14:11:24.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff43806ee20 0x7ff43815b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff438144f80 0x7ff438141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff43815b9b0 0x7ff438154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff438072ea0 con 0x7ff438144f80 2024-05-28T14:11:24.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff438072160 con 0x7ff43806ee20 2024-05-28T14:11:24.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff43cc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff438157710 con 0x7ff43815b9b0 2024-05-28T14:11:24.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff42d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff438144f80 0x7ff438141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff42d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff438144f80 0x7ff438141050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53406/0 (socket says 172.21.15.42:53406) 2024-05-28T14:11:24.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.704+0000 7ff42d400640 1 -- 172.21.15.42:0/3247852774 learned_addr learned my addr 172.21.15.42:0/3247852774 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:24.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff435e00640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff43806ee20 0x7ff43815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff436800640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff43815b9b0 0x7ff438154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff42d400640 1 -- 172.21.15.42:0/3247852774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff43815b9b0 msgr2=0x7ff438154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:24.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff42d400640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff43815b9b0 0x7ff438154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff42d400640 1 -- 172.21.15.42:0/3247852774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff43806ee20 msgr2=0x7ff43815b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:24.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff42d400640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff43806ee20 0x7ff43815b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff42d400640 1 -- 172.21.15.42:0/3247852774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff438156b00 con 0x7ff438144f80 2024-05-28T14:11:24.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff436800640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff43815b9b0 0x7ff438154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:24.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff42d400640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff438144f80 0x7ff438141050 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ff42000d800 tx=0x7ff42000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:24.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff42ea00640 1 -- 172.21.15.42:0/3247852774 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42000b6c0 con 0x7ff438144f80 2024-05-28T14:11:24.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff42ea00640 1 -- 172.21.15.42:0/3247852774 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff420019070 con 0x7ff438144f80 2024-05-28T14:11:24.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff42ea00640 1 -- 172.21.15.42:0/3247852774 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42000ba60 con 0x7ff438144f80 2024-05-28T14:11:24.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff43cc00640 1 -- 172.21.15.42:0/3247852774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff438156de0 con 0x7ff438144f80 2024-05-28T14:11:24.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff43cc00640 1 -- 172.21.15.42:0/3247852774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4381573a0 con 0x7ff438144f80 2024-05-28T14:11:24.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff43cc00640 1 -- 172.21.15.42:0/3247852774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43813d080 con 0x7ff438144f80 2024-05-28T14:11:24.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff42ea00640 1 -- 172.21.15.42:0/3247852774 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7ff42000bc00 con 0x7ff438144f80 2024-05-28T14:11:24.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.708+0000 7ff42ea00640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff414069850 0x7ff41406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:24.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.712+0000 7ff42ea00640 1 -- 172.21.15.42:0/3247852774 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7ff420085ee0 con 0x7ff438144f80 2024-05-28T14:11:24.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.712+0000 7ff435e00640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff414069850 0x7ff41406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:24.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.712+0000 7ff435e00640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff414069850 0x7ff41406bd10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff4280ae000 tx=0x7ff428009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:24.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.716+0000 7ff42ea00640 1 -- 172.21.15.42:0/3247852774 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff420057b10 con 0x7ff438144f80 2024-05-28T14:11:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.944+0000 7ff43cc00640 1 -- 172.21.15.42:0/3247852774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff4380738b0 con 0x7ff438144f80 2024-05-28T14:11:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.944+0000 7ff42ea00640 1 -- 172.21.15.42:0/3247852774 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7ff420057410 con 0x7ff438144f80 2024-05-28T14:11:24.930 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-05-28T14:11:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw== 2024-05-28T14:11:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow" 2024-05-28T14:11:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-05-28T14:11:24.930 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-28T14:11:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.0 2024-05-28T14:11:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.948+0000 7ff43cc00640 1 -- 172.21.15.42:0/3247852774 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff414069850 msgr2=0x7ff41406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.948+0000 7ff43cc00640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff414069850 0x7ff41406bd10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff4280ae000 tx=0x7ff428009040 comp rx=0 tx=0).stop 2024-05-28T14:11:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.948+0000 7ff43cc00640 1 -- 172.21.15.42:0/3247852774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff438144f80 msgr2=0x7ff438141050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.948+0000 7ff43cc00640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff438144f80 0x7ff438141050 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ff42000d800 tx=0x7ff42000dcc0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.948+0000 7ff43cc00640 1 -- 172.21.15.42:0/3247852774 shutdown_connections 2024-05-28T14:11:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.948+0000 7ff43cc00640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff414069850 0x7ff41406bd10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.948+0000 7ff43cc00640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff43815b9b0 0x7ff438154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.948+0000 7ff43cc00640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff438144f80 0x7ff438141050 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.948+0000 7ff43cc00640 1 --2- 172.21.15.42:0/3247852774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff43806ee20 0x7ff43815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.948+0000 7ff43cc00640 1 -- 172.21.15.42:0/3247852774 >> 172.21.15.42:0/3247852774 conn(0x7ff4380b21f0 msgr2=0x7ff4380cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.948+0000 7ff43cc00640 1 -- 172.21.15.42:0/3247852774 shutdown_connections 2024-05-28T14:11:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:24.948+0000 7ff43cc00640 1 -- 172.21.15.42:0/3247852774 wait complete. 2024-05-28T14:11:24.952 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:11:25.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.108+0000 7f4cc5a00640 1 Processor -- start 2024-05-28T14:11:25.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.108+0000 7f4cc5a00640 1 -- start start 2024-05-28T14:11:25.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.108+0000 7f4cc5a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4cc015e970 0x7f4cc00d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.112+0000 7f4cc5a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cc0070740 con 0x7f4cc006ff60 2024-05-28T14:11:25.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.112+0000 7f4cc5a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cc00d0870 con 0x7f4cc00cd210 2024-05-28T14:11:25.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.112+0000 7f4cc5a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f4cc00d0640 con 0x7f4cc015e970 2024-05-28T14:11:25.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.112+0000 7f4cbe000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f4cc006ff60 0x7f4cc00ccb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39262/0 (socket says 172.21.15.42:39262) 2024-05-28T14:11:25.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.112+0000 7f4cbea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4cc015e970 0x7f4cc00d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.112+0000 7f4cbe000640 1 -- 172.21.15.42:0/62964063 learned_addr learned my addr 172.21.15.42:0/62964063 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:25.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.112+0000 7f4cbea00640 1 -- 172.21.15.42:0/62964063 >> v1:172.21.15.63:6789/0 conn(0x7f4cc00cd210 legacy=0x7f4cc009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:25.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.112+0000 7f4cbea00640 1 -- 172.21.15.42:0/62964063 >> v1:172.21.15.42:6789/0 conn(0x7f4cc006ff60 legacy=0x7f4cc00ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.112+0000 7f4cbd600640 1 -- 172.21.15.42:0/62964063 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2313915622 0 0) 0x7f4cc0070740 con 0x7f4cc006ff60 2024-05-28T14:11:25.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.112+0000 7f4cbea00640 1 -- 172.21.15.42:0/62964063 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc00d0b10 con 0x7f4cc015e970 2024-05-28T14:11:25.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.112+0000 7f4cbea00640 1 --2- 172.21.15.42:0/62964063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cc015e970 0x7f4cc00d15f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0009d90 tx=0x7f4cb001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ecbdad4f7f5951 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:25.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cbd600640 1 -- 172.21.15.42:0/62964063 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00b2070 con 0x7f4cc015e970 2024-05-28T14:11:25.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cbd600640 1 -- 172.21.15.42:0/62964063 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cb00a57d0 con 0x7f4cc015e970 2024-05-28T14:11:25.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cbd600640 1 -- 172.21.15.42:0/62964063 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00a5ba0 con 0x7f4cc015e970 2024-05-28T14:11:25.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 -- 172.21.15.42:0/62964063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cc015e970 msgr2=0x7f4cc00d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 --2- 172.21.15.42:0/62964063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cc015e970 0x7f4cc00d15f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0009d90 tx=0x7f4cb001c8e0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 -- 172.21.15.42:0/62964063 shutdown_connections 2024-05-28T14:11:25.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 --2- 172.21.15.42:0/62964063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cc015e970 0x7f4cc00d15f0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 -- 172.21.15.42:0/62964063 >> 172.21.15.42:0/62964063 conn(0x7f4cc00b3fa0 msgr2=0x7f4cc00b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:25.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 -- 172.21.15.42:0/62964063 shutdown_connections 2024-05-28T14:11:25.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 -- 172.21.15.42:0/62964063 wait complete. 2024-05-28T14:11:25.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 Processor -- start 2024-05-28T14:11:25.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 -- start start 2024-05-28T14:11:25.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cc006ff60 0x7f4cc00cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cc00cd210 0x7f4cc0151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cc015e970 0x7f4cc00b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc00cd7a0 con 0x7f4cc00cd210 2024-05-28T14:11:25.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc00cd620 con 0x7f4cc006ff60 2024-05-28T14:11:25.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.116+0000 7f4cc5a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4cc016c850 con 0x7f4cc015e970 2024-05-28T14:11:25.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cbe000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cc00cd210 0x7f4cc0151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cbe000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cc00cd210 0x7f4cc0151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53418/0 (socket says 172.21.15.42:53418) 2024-05-28T14:11:25.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cbe000640 1 -- 172.21.15.42:0/1189212352 learned_addr learned my addr 172.21.15.42:0/1189212352 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:25.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cbea00640 1 --2- 172.21.15.42:0/1189212352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cc006ff60 0x7f4cc00cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cbf400640 1 --2- 172.21.15.42:0/1189212352 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cc015e970 0x7f4cc00b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cbe000640 1 -- 172.21.15.42:0/1189212352 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cc015e970 msgr2=0x7f4cc00b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cbe000640 1 --2- 172.21.15.42:0/1189212352 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cc015e970 0x7f4cc00b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cbe000640 1 -- 172.21.15.42:0/1189212352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cc006ff60 msgr2=0x7f4cc00cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cbe000640 1 --2- 172.21.15.42:0/1189212352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cc006ff60 0x7f4cc00cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cbe000640 1 -- 172.21.15.42:0/1189212352 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc01524c0 con 0x7f4cc00cd210 2024-05-28T14:11:25.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cbe000640 1 --2- 172.21.15.42:0/1189212352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cc00cd210 0x7f4cc0151f80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00c410 tx=0x7f4cac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:25.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cb6a00640 1 -- 172.21.15.42:0/1189212352 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac0074e0 con 0x7f4cc00cd210 2024-05-28T14:11:25.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cb6a00640 1 -- 172.21.15.42:0/1189212352 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cac0047f0 con 0x7f4cc00cd210 2024-05-28T14:11:25.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cb6a00640 1 -- 172.21.15.42:0/1189212352 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac007680 con 0x7f4cc00cd210 2024-05-28T14:11:25.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cc5a00640 1 -- 172.21.15.42:0/1189212352 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc0161630 con 0x7f4cc00cd210 2024-05-28T14:11:25.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cc5a00640 1 -- 172.21.15.42:0/1189212352 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc0161bf0 con 0x7f4cc00cd210 2024-05-28T14:11:25.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cc5a00640 1 -- 172.21.15.42:0/1189212352 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc0068a90 con 0x7f4cc00cd210 2024-05-28T14:11:25.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.120+0000 7f4cb6a00640 1 -- 172.21.15.42:0/1189212352 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f4cac011070 con 0x7f4cc00cd210 2024-05-28T14:11:25.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.124+0000 7f4cb6a00640 1 --2- 172.21.15.42:0/1189212352 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4ca0069880 0x7f4ca006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.124+0000 7f4cb6a00640 1 -- 172.21.15.42:0/1189212352 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f4cac085560 con 0x7f4cc00cd210 2024-05-28T14:11:25.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.124+0000 7f4cbea00640 1 --2- 172.21.15.42:0/1189212352 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4ca0069880 0x7f4ca006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.124+0000 7f4cbea00640 1 --2- 172.21.15.42:0/1189212352 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4ca0069880 0x7f4ca006bd40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00af000 tx=0x7f4cb00a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:25.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.128+0000 7f4cb6a00640 1 -- 172.21.15.42:0/1189212352 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4cac056b40 con 0x7f4cc00cd210 2024-05-28T14:11:25.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.328+0000 7f4cc5a00640 1 -- 172.21.15.42:0/1189212352 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cc00d1fd0 con 0x7f4cc00cd210 2024-05-28T14:11:25.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.328+0000 7f4cb6a00640 1 -- 172.21.15.42:0/1189212352 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5190 (secure 0 0 0) 0x7f4cac05ae40 con 0x7f4cc00cd210 2024-05-28T14:11:25.313 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":64,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8607,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/1661021064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1661021064},{"type":"v1","addr":"172.21.15.42:6836","nonce":1661021064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":57050,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2398440572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":2398440572},{"type":"v1","addr":"172.21.15.42:6837","nonce":2398440572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":57053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/2530315729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":2530315729},{"type":"v1","addr":"172.21.15.63:6834","nonce":2530315729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:11:14.793551+0000","modified":"2024-05-28T14:11:17.812010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57056},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57056":{"gid":57056,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/1184583947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1184583947},{"type":"v1","addr":"172.21.15.63:6835","nonce":1184583947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57056,"qdb_cluster":[57056]},"id":7}]} 2024-05-28T14:11:25.313 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 64 2024-05-28T14:11:25.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.332+0000 7f4cc5a00640 1 -- 172.21.15.42:0/1189212352 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4ca0069880 msgr2=0x7f4ca006bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.332+0000 7f4cc5a00640 1 --2- 172.21.15.42:0/1189212352 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4ca0069880 0x7f4ca006bd40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00af000 tx=0x7f4cb00a7070 comp rx=0 tx=0).stop 2024-05-28T14:11:25.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.332+0000 7f4cc5a00640 1 -- 172.21.15.42:0/1189212352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cc00cd210 msgr2=0x7f4cc0151f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.332+0000 7f4cc5a00640 1 --2- 172.21.15.42:0/1189212352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cc00cd210 0x7f4cc0151f80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00c410 tx=0x7f4cac00c8d0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.332+0000 7f4cc5a00640 1 -- 172.21.15.42:0/1189212352 shutdown_connections 2024-05-28T14:11:25.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.332+0000 7f4cc5a00640 1 --2- 172.21.15.42:0/1189212352 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4ca0069880 0x7f4ca006bd40 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.332+0000 7f4cc5a00640 1 --2- 172.21.15.42:0/1189212352 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cc015e970 0x7f4cc00b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.332+0000 7f4cc5a00640 1 --2- 172.21.15.42:0/1189212352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cc00cd210 0x7f4cc0151f80 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.332+0000 7f4cc5a00640 1 --2- 172.21.15.42:0/1189212352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cc006ff60 0x7f4cc00cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.332+0000 7f4cc5a00640 1 -- 172.21.15.42:0/1189212352 >> 172.21.15.42:0/1189212352 conn(0x7f4cc00b3fa0 msgr2=0x7f4cc006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:25.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.332+0000 7f4cc5a00640 1 -- 172.21.15.42:0/1189212352 shutdown_connections 2024-05-28T14:11:25.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.332+0000 7f4cc5a00640 1 -- 172.21.15.42:0/1189212352 wait complete. 2024-05-28T14:11:25.737 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-28T14:11:14.793551+0000', 'modified': '2024-05-28T14:11:17.812010+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 57056}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_57056': {'gid': 57056, 'name': 'b', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6835/1184583947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6833', 'nonce': 1184583947}, {'type': 'v1', 'addr': '172.21.15.63:6835', 'nonce': 1184583947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 57056, 'qdb_cluster': [57056]} 2024-05-28T14:11:25.737 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-28T14:11:25.738 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-28T14:11:25.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.900+0000 7f5009400640 1 Processor -- start 2024-05-28T14:11:25.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.900+0000 7f5009400640 1 -- start start 2024-05-28T14:11:25.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.904+0000 7f5009400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f500406ff10 0x7f500415e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.904+0000 7f5009400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f50040703a0 0x7f500415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.904+0000 7f5009400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50040d0660 con 0x7f5004070800 2024-05-28T14:11:25.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.904+0000 7f5009400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f500409ba60 con 0x7f500406ff10 2024-05-28T14:11:25.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.904+0000 7f5009400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f50040d0940 con 0x7f50040703a0 2024-05-28T14:11:25.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.904+0000 7f5003400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f500406ff10 0x7f500415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.904+0000 7f5003e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f5004070800 0x7f5004070be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39272/0 (socket says 172.21.15.42:39272) 2024-05-28T14:11:25.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.904+0000 7f5003e00640 1 -- 172.21.15.42:0/2208382314 learned_addr learned my addr 172.21.15.42:0/2208382314 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:25.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.904+0000 7f5008a00640 1 --2- 172.21.15.42:0/2208382314 >> v2:172.21.15.63:3300/0 conn(0x7f50040703a0 0x7f500415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.904+0000 7f5003400640 1 -- 172.21.15.42:0/2208382314 >> v2:172.21.15.63:3300/0 conn(0x7f50040703a0 msgr2=0x7f500415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:25.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.904+0000 7f5003400640 1 --2- 172.21.15.42:0/2208382314 >> v2:172.21.15.63:3300/0 conn(0x7f50040703a0 0x7f500415ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.904+0000 7f5003400640 1 -- 172.21.15.42:0/2208382314 >> v1:172.21.15.42:6789/0 conn(0x7f5004070800 legacy=0x7f5004070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.904+0000 7f5003400640 1 -- 172.21.15.42:0/2208382314 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50040d0c20 con 0x7f500406ff10 2024-05-28T14:11:25.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.908+0000 7f5003400640 1 --2- 172.21.15.42:0/2208382314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f500406ff10 0x7f500415e910 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00097c0 tx=0x7f4ff001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a74f83be0d7c851 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:25.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.908+0000 7f5002a00640 1 -- 172.21.15.42:0/2208382314 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00b0070 con 0x7f500406ff10 2024-05-28T14:11:25.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.908+0000 7f5002a00640 1 -- 172.21.15.42:0/2208382314 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff0004230 con 0x7f500406ff10 2024-05-28T14:11:25.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.908+0000 7f5002a00640 1 -- 172.21.15.42:0/2208382314 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0003620 con 0x7f500406ff10 2024-05-28T14:11:25.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.908+0000 7f5009400640 1 -- 172.21.15.42:0/2208382314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f500406ff10 msgr2=0x7f500415e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.908+0000 7f5009400640 1 --2- 172.21.15.42:0/2208382314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f500406ff10 0x7f500415e910 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00097c0 tx=0x7f4ff001c460 comp rx=0 tx=0).stop 2024-05-28T14:11:25.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.908+0000 7f5009400640 1 -- 172.21.15.42:0/2208382314 shutdown_connections 2024-05-28T14:11:25.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.908+0000 7f5009400640 1 --2- 172.21.15.42:0/2208382314 >> v2:172.21.15.63:3300/0 conn(0x7f50040703a0 0x7f500415ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.908+0000 7f5009400640 1 --2- 172.21.15.42:0/2208382314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f500406ff10 0x7f500415e910 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.908+0000 7f5009400640 1 -- 172.21.15.42:0/2208382314 >> 172.21.15.42:0/2208382314 conn(0x7f50040b3dd0 msgr2=0x7f50040b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:25.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.908+0000 7f5009400640 1 -- 172.21.15.42:0/2208382314 shutdown_connections 2024-05-28T14:11:25.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.908+0000 7f5009400640 1 -- 172.21.15.42:0/2208382314 wait complete. 2024-05-28T14:11:25.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.908+0000 7f5009400640 1 Processor -- start 2024-05-28T14:11:25.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.908+0000 7f5009400640 1 -- start start 2024-05-28T14:11:25.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5009400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f500406ff10 0x7f5004154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5009400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50040703a0 0x7f50040cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5009400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5004070800 0x7f50041513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5009400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f50040b1a10 con 0x7f50040703a0 2024-05-28T14:11:25.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5009400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50040b1890 con 0x7f5004070800 2024-05-28T14:11:25.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5009400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f500416b5d0 con 0x7f500406ff10 2024-05-28T14:11:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5003400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50040703a0 0x7f50040cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5003400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50040703a0 0x7f50040cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53434/0 (socket says 172.21.15.42:53434) 2024-05-28T14:11:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5003400640 1 -- 172.21.15.42:0/1311137084 learned_addr learned my addr 172.21.15.42:0/1311137084 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5003400640 1 -- 172.21.15.42:0/1311137084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f500406ff10 msgr2=0x7f5004154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5003400640 1 --2- 172.21.15.42:0/1311137084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f500406ff10 0x7f5004154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5003400640 1 -- 172.21.15.42:0/1311137084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5004070800 msgr2=0x7f50041513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5003400640 1 --2- 172.21.15.42:0/1311137084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5004070800 0x7f50041513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5003400640 1 -- 172.21.15.42:0/1311137084 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50040c9fa0 con 0x7f50040703a0 2024-05-28T14:11:25.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5003400640 1 --2- 172.21.15.42:0/1311137084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50040703a0 0x7f50040cb040 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0019070 tx=0x7f4ff0019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:25.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f4ffbe00640 1 -- 172.21.15.42:0/1311137084 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00b0040 con 0x7f50040703a0 2024-05-28T14:11:25.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f4ffbe00640 1 -- 172.21.15.42:0/1311137084 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff00bb070 con 0x7f50040703a0 2024-05-28T14:11:25.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5009400640 1 -- 172.21.15.42:0/1311137084 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50040ca280 con 0x7f50040703a0 2024-05-28T14:11:25.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f4ffbe00640 1 -- 172.21.15.42:0/1311137084 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00ab440 con 0x7f50040703a0 2024-05-28T14:11:25.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f5009400640 1 -- 172.21.15.42:0/1311137084 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50040ca840 con 0x7f50040703a0 2024-05-28T14:11:25.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.912+0000 7f4ffbe00640 1 -- 172.21.15.42:0/1311137084 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f4ff00ab5e0 con 0x7f50040703a0 2024-05-28T14:11:25.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f4ffbe00640 1 --2- 172.21.15.42:0/1311137084 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4fe4069850 0x7f4fe406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f4ffbe00640 1 -- 172.21.15.42:0/1311137084 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f4ff011f800 con 0x7f50040703a0 2024-05-28T14:11:25.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f5003e00640 1 --2- 172.21.15.42:0/1311137084 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4fe4069850 0x7f4fe406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f500de05000 1 Processor -- start 2024-05-28T14:11:25.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f500de05000 1 -- start start 2024-05-28T14:11:25.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f5003e00640 1 --2- 172.21.15.42:0/1311137084 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4fe4069850 0x7f4fe406bd10 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4000f70 tx=0x7f4ff4007a90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:25.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f500de05000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222120 0x562279222500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f500de05000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562279222a40 0x5622791d5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f500de05000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622791d5650 0x5622791d5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f500de05000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562279026420 con 0x562279222120 2024-05-28T14:11:25.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f500de05000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562278fc0bf0 con 0x5622791d5650 2024-05-28T14:11:25.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f500de05000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5622790223e0 con 0x562279222a40 2024-05-28T14:11:25.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f5003e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222120 0x562279222500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f5003e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222120 0x562279222500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53448/0 (socket says 172.21.15.42:53448) 2024-05-28T14:11:25.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f5003e00640 1 -- 172.21.15.42:0/1509404071 learned_addr learned my addr 172.21.15.42:0/1509404071 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:25.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f5003400640 1 --2- 172.21.15.42:0/1509404071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562279222a40 0x5622791d5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f5008a00640 1 --2- 172.21.15.42:0/1509404071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622791d5650 0x5622791d5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f5003e00640 1 -- 172.21.15.42:0/1509404071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562279222a40 msgr2=0x5622791d5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:25.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f5003e00640 1 --2- 172.21.15.42:0/1509404071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562279222a40 0x5622791d5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f5003e00640 1 -- 172.21.15.42:0/1509404071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622791d5650 msgr2=0x5622791d5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f5003e00640 1 --2- 172.21.15.42:0/1509404071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622791d5650 0x5622791d5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f5003e00640 1 -- 172.21.15.42:0/1509404071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622790ce450 con 0x562279222120 2024-05-28T14:11:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f5003e00640 1 --2- 172.21.15.42:0/1509404071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222120 0x562279222500 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4005d10 tx=0x7f4ff400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c11ef4b4abcd1a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f4ff8c00640 1 -- 172.21.15.42:0/1509404071 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4005e00 con 0x562279222120 2024-05-28T14:11:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f4ff8c00640 1 -- 172.21.15.42:0/1509404071 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff4007560 con 0x562279222120 2024-05-28T14:11:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f4ff8c00640 1 -- 172.21.15.42:0/1509404071 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4016950 con 0x562279222120 2024-05-28T14:11:25.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f500de05000 1 -- 172.21.15.42:0/1509404071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222120 msgr2=0x562279222500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.916+0000 7f500de05000 1 --2- 172.21.15.42:0/1509404071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222120 0x562279222500 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4005d10 tx=0x7f4ff400f000 comp rx=0 tx=0).stop 2024-05-28T14:11:25.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 -- 172.21.15.42:0/1509404071 shutdown_connections 2024-05-28T14:11:25.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 --2- 172.21.15.42:0/1509404071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622791d5650 0x5622791d5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 --2- 172.21.15.42:0/1509404071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562279222a40 0x5622791d5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 --2- 172.21.15.42:0/1509404071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222120 0x562279222500 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 -- 172.21.15.42:0/1509404071 >> 172.21.15.42:0/1509404071 conn(0x562279202ec0 msgr2=0x5622792052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:25.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 -- 172.21.15.42:0/1509404071 shutdown_connections 2024-05-28T14:11:25.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 -- 172.21.15.42:0/1509404071 wait complete. 2024-05-28T14:11:25.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 -- start start 2024-05-28T14:11:25.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 Processor -- start 2024-05-28T14:11:25.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5622791d5650 0x5622792218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222120 0x5622791dab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222a40 0x5622791db0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562278fc6ec0 con 0x562279222a40 2024-05-28T14:11:25.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5622790a7120 con 0x562279222120 2024-05-28T14:11:25.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562279031350 con 0x5622791d5650 2024-05-28T14:11:25.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 10 client.0 _ll_register_callbacks cb 0x5622790034d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T14:11:25.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f5008a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222a40 0x5622791db0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f5008a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222a40 0x5622791db0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53460/0 (socket says 172.21.15.42:53460) 2024-05-28T14:11:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f5008a00640 1 -- 172.21.15.42:0/1888151731 learned_addr learned my addr 172.21.15.42:0/1888151731 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f5003e00640 1 --2- 172.21.15.42:0/1888151731 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5622791d5650 0x5622792218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f5003400640 1 --2- 172.21.15.42:0/1888151731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222120 0x5622791dab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f5008a00640 1 -- 172.21.15.42:0/1888151731 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5622791d5650 msgr2=0x5622792218b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f5008a00640 1 --2- 172.21.15.42:0/1888151731 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5622791d5650 0x5622792218b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f5008a00640 1 -- 172.21.15.42:0/1888151731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222120 msgr2=0x5622791dab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f5008a00640 1 --2- 172.21.15.42:0/1888151731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222120 0x5622791dab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f5008a00640 1 -- 172.21.15.42:0/1888151731 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56227906bf80 con 0x562279222a40 2024-05-28T14:11:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f5008a00640 1 --2- 172.21.15.42:0/1888151731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222a40 0x5622791db0c0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0079a0 tx=0x7f4ffc007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8416d4a7dda2559 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f4fed600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T14:11:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f4fed600640 1 -- 172.21.15.42:0/1888151731 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc00fc00 con 0x562279222a40 2024-05-28T14:11:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f4fed600640 1 -- 172.21.15.42:0/1888151731 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f4ffc00fda0 con 0x562279222a40 2024-05-28T14:11:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 -- 172.21.15.42:0/1888151731 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ff4007560 con 0x562279222a40 2024-05-28T14:11:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f4fed600640 1 -- 172.21.15.42:0/1888151731 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f4ffc023050 con 0x562279222a40 2024-05-28T14:11:25.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 10 client.8781 fetch_fsmap learned FSMap version 64 2024-05-28T14:11:25.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 1 -- 172.21.15.42:0/1888151731 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x562278ffff00 con 0x562279222a40 2024-05-28T14:11:25.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f4fed600640 1 -- 172.21.15.42:0/1888151731 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff4007560 con 0x562279222a40 2024-05-28T14:11:25.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f4fed600640 1 -- 172.21.15.42:0/1888151731 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f4ffc026020 con 0x562279222a40 2024-05-28T14:11:25.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 10 client.8781 fetch_fsmap finished waiting for FSMap version 64 2024-05-28T14:11:25.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.920+0000 7f500de05000 10 client.8781 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-28T14:11:25.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 20 client.8781 populate_metadata read hostname 'smithi042' 2024-05-28T14:11:25.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 1 --2- 172.21.15.42:0/1888151731 >> [v2:172.21.15.63:6833/1184583947,v1:172.21.15.63:6835/1184583947] conn(0x562279231890 0x562279233c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 4 client.8781 mds_command: new command op to 57056 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-28T14:11:25.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 1 -- 172.21.15.42:0/1888151731 --> [v2:172.21.15.63:6833/1184583947,v1:172.21.15.63:6835/1184583947] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562278fbfbc0 con 0x562279231890 2024-05-28T14:11:25.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f5003e00640 1 --2- 172.21.15.42:0/1888151731 >> [v2:172.21.15.63:6833/1184583947,v1:172.21.15.63:6835/1184583947] conn(0x562279231890 0x562279233c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f5003e00640 1 --2- 172.21.15.42:0/1888151731 >> [v2:172.21.15.63:6833/1184583947,v1:172.21.15.63:6835/1184583947] conn(0x562279231890 0x562279233c80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21dfb542446bd11f server_cookie=2811d3de34928492 in_seq=0 out_seq=0 2024-05-28T14:11:25.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f4fed600640 10 client.8781 ms_handle_connect on v2:172.21.15.63:6833/1184583947 2024-05-28T14:11:25.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f4fed600640 1 -- 172.21.15.42:0/1888151731 <== mds.0 v2:172.21.15.63:6833/1184583947 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562278fbfbc0 con 0x562279231890 2024-05-28T14:11:25.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f4fed600640 10 client.8781 handle_command_reply: tid=0 2024-05-28T14:11:25.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 1 client.8781 shutdown 2024-05-28T14:11:25.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 1 -- 172.21.15.42:0/1888151731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222a40 msgr2=0x5622791db0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 1 --2- 172.21.15.42:0/1888151731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222a40 0x5622791db0c0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0079a0 tx=0x7f4ffc007e60 comp rx=0 tx=0).stop 2024-05-28T14:11:25.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 1 -- 172.21.15.42:0/1888151731 shutdown_connections 2024-05-28T14:11:25.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 1 --2- 172.21.15.42:0/1888151731 >> [v2:172.21.15.63:6833/1184583947,v1:172.21.15.63:6835/1184583947] conn(0x562279231890 0x562279233c80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 1 --2- 172.21.15.42:0/1888151731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222a40 0x5622791db0c0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 1 --2- 172.21.15.42:0/1888151731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222120 0x5622791dab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 1 --2- 172.21.15.42:0/1888151731 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5622791d5650 0x5622792218b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f4fed600640 0 client.8781 ms_handle_reset on v2:172.21.15.63:6833/1184583947 2024-05-28T14:11:25.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 1 -- 172.21.15.42:0/1888151731 >> 172.21.15.42:0/1888151731 conn(0x562279202ec0 msgr2=0x5622792052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:25.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 1 -- 172.21.15.42:0/1888151731 shutdown_connections 2024-05-28T14:11:25.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 1 -- 172.21.15.42:0/1888151731 wait complete. 2024-05-28T14:11:25.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.924+0000 7f500de05000 20 client.8781 trim_cache size 0 max 16384 2024-05-28T14:11:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f500de05000 1 Processor -- start 2024-05-28T14:11:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f500de05000 1 -- start start 2024-05-28T14:11:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f500de05000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622791d5650 0x5622791d4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f500de05000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222120 0x5622791d4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f500de05000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562279222a40 0x562279233c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f500de05000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5004202d00 con 0x5622791d5650 2024-05-28T14:11:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f500de05000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50040b5370 con 0x562279222120 2024-05-28T14:11:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f500de05000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f50040b59c0 con 0x562279222a40 2024-05-28T14:11:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f5003e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622791d5650 0x5622791d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f5003e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622791d5650 0x5622791d4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53472/0 (socket says 172.21.15.42:53472) 2024-05-28T14:11:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f5003e00640 1 -- 172.21.15.42:0/2022013837 learned_addr learned my addr 172.21.15.42:0/2022013837 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:25.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f5008a00640 1 --2- 172.21.15.42:0/2022013837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562279222a40 0x562279233c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f5003400640 1 --2- 172.21.15.42:0/2022013837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222120 0x5622791d4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f5003e00640 1 -- 172.21.15.42:0/2022013837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562279222a40 msgr2=0x562279233c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:25.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f5003e00640 1 --2- 172.21.15.42:0/2022013837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562279222a40 0x562279233c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f5003e00640 1 -- 172.21.15.42:0/2022013837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222120 msgr2=0x5622791d4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:25.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f5003e00640 1 --2- 172.21.15.42:0/2022013837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222120 0x5622791d4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f5003e00640 1 -- 172.21.15.42:0/2022013837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562279222870 con 0x5622791d5650 2024-05-28T14:11:25.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f5003e00640 1 --2- 172.21.15.42:0/2022013837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622791d5650 0x5622791d4100 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4005990 tx=0x7f4ff4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21d6a5a49f075ad server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f4fefe00640 1 -- 172.21.15.42:0/2022013837 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4031070 con 0x5622791d5650 2024-05-28T14:11:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f4fefe00640 1 -- 172.21.15.42:0/2022013837 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff4024030 con 0x5622791d5650 2024-05-28T14:11:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f4fefe00640 1 -- 172.21.15.42:0/2022013837 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff402c630 con 0x5622791d5650 2024-05-28T14:11:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.944+0000 7f500de05000 1 -- 172.21.15.42:0/2022013837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622791d5650 msgr2=0x5622791d4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 --2- 172.21.15.42:0/2022013837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622791d5650 0x5622791d4100 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4005990 tx=0x7f4ff4011000 comp rx=0 tx=0).stop 2024-05-28T14:11:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 -- 172.21.15.42:0/2022013837 shutdown_connections 2024-05-28T14:11:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 --2- 172.21.15.42:0/2022013837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562279222a40 0x562279233c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 --2- 172.21.15.42:0/2022013837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222120 0x5622791d4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 --2- 172.21.15.42:0/2022013837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5622791d5650 0x5622791d4100 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 -- 172.21.15.42:0/2022013837 >> 172.21.15.42:0/2022013837 conn(0x562279202ec0 msgr2=0x562279220a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 -- 172.21.15.42:0/2022013837 shutdown_connections 2024-05-28T14:11:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 -- 172.21.15.42:0/2022013837 wait complete. 2024-05-28T14:11:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 -- start start 2024-05-28T14:11:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 Processor -- start 2024-05-28T14:11:25.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5622791d5650 0x562279236850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222120 0x56227924d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222a40 0x56227924dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562279025730 con 0x562279222120 2024-05-28T14:11:25.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562278ff29e0 con 0x562279222a40 2024-05-28T14:11:25.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56227902aa50 con 0x5622791d5650 2024-05-28T14:11:25.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 10 client.0 _ll_register_callbacks cb 0x7f4ffc009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T14:11:25.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f5003400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222120 0x56227924d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f5003400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222120 0x56227924d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53482/0 (socket says 172.21.15.42:53482) 2024-05-28T14:11:25.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f5003400640 1 -- 172.21.15.42:0/2648163398 learned_addr learned my addr 172.21.15.42:0/2648163398 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:25.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f5003e00640 1 --2- 172.21.15.42:0/2648163398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5622791d5650 0x562279236850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f5008a00640 1 --2- 172.21.15.42:0/2648163398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222a40 0x56227924dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f5003400640 1 -- 172.21.15.42:0/2648163398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5622791d5650 msgr2=0x562279236850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:25.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f5003400640 1 --2- 172.21.15.42:0/2648163398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5622791d5650 0x562279236850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f5003400640 1 -- 172.21.15.42:0/2648163398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222a40 msgr2=0x56227924dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:25.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f5003400640 1 --2- 172.21.15.42:0/2648163398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222a40 0x56227924dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f5003400640 1 -- 172.21.15.42:0/2648163398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562278face80 con 0x562279222120 2024-05-28T14:11:25.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f5003400640 1 --2- 172.21.15.42:0/2648163398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222120 0x56227924d810 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00f0a30 tx=0x7f4ff00f0750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2b16c1673a05 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:25.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f4fed600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T14:11:25.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f4fed600640 1 -- 172.21.15.42:0/2648163398 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00f9910 con 0x562279222120 2024-05-28T14:11:25.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f4fed600640 1 -- 172.21.15.42:0/2648163398 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f4ff00f0f90 con 0x562279222120 2024-05-28T14:11:25.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 -- 172.21.15.42:0/2648163398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ff4024030 con 0x562279222120 2024-05-28T14:11:25.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f4fed600640 1 -- 172.21.15.42:0/2648163398 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f4ff00f1190 con 0x562279222120 2024-05-28T14:11:25.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 10 client.8787 fetch_fsmap learned FSMap version 64 2024-05-28T14:11:25.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 -- 172.21.15.42:0/2648163398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x562278feeb30 con 0x562279222120 2024-05-28T14:11:25.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f4fed600640 1 -- 172.21.15.42:0/2648163398 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff4024030 con 0x562279222120 2024-05-28T14:11:25.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f4fed600640 1 -- 172.21.15.42:0/2648163398 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f4ff00f9480 con 0x562279222120 2024-05-28T14:11:25.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 10 client.8787 fetch_fsmap finished waiting for FSMap version 64 2024-05-28T14:11:25.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 10 client.8787 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-28T14:11:25.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 20 client.8787 populate_metadata read hostname 'smithi042' 2024-05-28T14:11:25.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 --2- 172.21.15.42:0/2648163398 >> [v2:172.21.15.63:6833/1184583947,v1:172.21.15.63:6835/1184583947] conn(0x562279291480 0x562279291860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:25.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 4 client.8787 mds_command: new command op to 57056 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-28T14:11:25.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.948+0000 7f500de05000 1 -- 172.21.15.42:0/2648163398 --> [v2:172.21.15.63:6833/1184583947,v1:172.21.15.63:6835/1184583947] -- command(tid 0: {"prefix": "status"}) -- 0x562279033e40 con 0x562279291480 2024-05-28T14:11:25.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f5003e00640 1 --2- 172.21.15.42:0/2648163398 >> [v2:172.21.15.63:6833/1184583947,v1:172.21.15.63:6835/1184583947] conn(0x562279291480 0x562279291860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:25.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f5003e00640 1 --2- 172.21.15.42:0/2648163398 >> [v2:172.21.15.63:6833/1184583947,v1:172.21.15.63:6835/1184583947] conn(0x562279291480 0x562279291860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3695125e0f7e2475 server_cookie=64264a840b34c521 in_seq=0 out_seq=0 2024-05-28T14:11:25.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f4fed600640 10 client.8787 ms_handle_connect on v2:172.21.15.63:6833/1184583947 2024-05-28T14:11:25.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f4fed600640 1 -- 172.21.15.42:0/2648163398 <== mds.0 v2:172.21.15.63:6833/1184583947 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x562279033e40 con 0x562279291480 2024-05-28T14:11:25.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f4fed600640 10 client.8787 handle_command_reply: tid=0 2024-05-28T14:11:25.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f500de05000 1 client.8787 shutdown 2024-05-28T14:11:25.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f500de05000 1 -- 172.21.15.42:0/2648163398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222120 msgr2=0x56227924d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f500de05000 1 --2- 172.21.15.42:0/2648163398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222120 0x56227924d810 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00f0a30 tx=0x7f4ff00f0750 comp rx=0 tx=0).stop 2024-05-28T14:11:25.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f500de05000 1 -- 172.21.15.42:0/2648163398 shutdown_connections 2024-05-28T14:11:25.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f500de05000 1 --2- 172.21.15.42:0/2648163398 >> [v2:172.21.15.63:6833/1184583947,v1:172.21.15.63:6835/1184583947] conn(0x562279291480 0x562279291860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f500de05000 1 --2- 172.21.15.42:0/2648163398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562279222a40 0x56227924dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f500de05000 1 --2- 172.21.15.42:0/2648163398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562279222120 0x56227924d810 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f4fed600640 0 client.8787 ms_handle_reset on v2:172.21.15.63:6833/1184583947 2024-05-28T14:11:25.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f500de05000 1 --2- 172.21.15.42:0/2648163398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5622791d5650 0x562279236850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f500de05000 1 -- 172.21.15.42:0/2648163398 >> 172.21.15.42:0/2648163398 conn(0x562279202ec0 msgr2=0x5622791d4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:25.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f500de05000 1 -- 172.21.15.42:0/2648163398 shutdown_connections 2024-05-28T14:11:25.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f500de05000 1 -- 172.21.15.42:0/2648163398 wait complete. 2024-05-28T14:11:25.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f500de05000 20 client.8787 trim_cache size 0 max 16384 2024-05-28T14:11:25.941 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:11:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:11:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-05-28T14:11:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 57056, 2024-05-28T14:11:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-28T14:11:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-28T14:11:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-28T14:11:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 10.550852711999999, 2024-05-28T14:11:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 64, 2024-05-28T14:11:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 105, 2024-05-28T14:11:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 105, 2024-05-28T14:11:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 15.368429434999999 2024-05-28T14:11:25.942 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:11:25.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f5009400640 1 -- 172.21.15.42:0/1311137084 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4fe4069850 msgr2=0x7f4fe406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f5009400640 1 --2- 172.21.15.42:0/1311137084 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4fe4069850 0x7f4fe406bd10 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4000f70 tx=0x7f4ff4007a90 comp rx=0 tx=0).stop 2024-05-28T14:11:25.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f5009400640 1 -- 172.21.15.42:0/1311137084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50040703a0 msgr2=0x7f50040cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:25.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f5009400640 1 --2- 172.21.15.42:0/1311137084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50040703a0 0x7f50040cb040 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0019070 tx=0x7f4ff0019200 comp rx=0 tx=0).stop 2024-05-28T14:11:25.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f5009400640 1 -- 172.21.15.42:0/1311137084 shutdown_connections 2024-05-28T14:11:25.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f5009400640 1 --2- 172.21.15.42:0/1311137084 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4fe4069850 0x7f4fe406bd10 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f5009400640 1 --2- 172.21.15.42:0/1311137084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5004070800 0x7f50041513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f5009400640 1 --2- 172.21.15.42:0/1311137084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50040703a0 0x7f50040cb040 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f5009400640 1 --2- 172.21.15.42:0/1311137084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f500406ff10 0x7f5004154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:25.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f5009400640 1 -- 172.21.15.42:0/1311137084 >> 172.21.15.42:0/1311137084 conn(0x7f50040b3dd0 msgr2=0x7f500406cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:25.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f5009400640 1 -- 172.21.15.42:0/1311137084 shutdown_connections 2024-05-28T14:11:25.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:25.952+0000 7f5009400640 1 -- 172.21.15.42:0/1311137084 wait complete. 2024-05-28T14:11:25.956 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-28T14:11:26.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.112+0000 7f6477200640 1 Processor -- start 2024-05-28T14:11:26.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.112+0000 7f6477200640 1 -- start start 2024-05-28T14:11:26.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.116+0000 7f6477200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6478144f50 0x7f64780706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:26.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.116+0000 7f6477200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f64780b6210 0x7f64780af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:26.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.116+0000 7f6477200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f647806ffc0 con 0x7f647806fb60 2024-05-28T14:11:26.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.116+0000 7f6477200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6478070db0 con 0x7f64780b6210 2024-05-28T14:11:26.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.116+0000 7f6477200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f64780ae240 con 0x7f6478144f50 2024-05-28T14:11:26.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.116+0000 7f6476800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f64780b6210 0x7f64780af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:26.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.116+0000 7f6476800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f64780b6210 0x7f64780af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53498/0 (socket says 172.21.15.42:53498) 2024-05-28T14:11:26.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.116+0000 7f6476800640 1 -- 172.21.15.42:0/1052486761 learned_addr learned my addr 172.21.15.42:0/1052486761 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:26.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.116+0000 7f6475e00640 1 --2- 172.21.15.42:0/1052486761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6478144f50 0x7f64780706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:26.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.116+0000 7f6476800640 1 -- 172.21.15.42:0/1052486761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6478144f50 msgr2=0x7f64780706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:26.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.116+0000 7f6476800640 1 --2- 172.21.15.42:0/1052486761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6478144f50 0x7f64780706b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:26.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.116+0000 7f6476800640 1 -- 172.21.15.42:0/1052486761 >> v1:172.21.15.63:6789/0 conn(0x7f647806fb60 legacy=0x7f64780b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:26.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.116+0000 7f6476800640 1 -- 172.21.15.42:0/1052486761 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64780ae520 con 0x7f64780b6210 2024-05-28T14:11:26.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.120+0000 7f6476800640 1 --2- 172.21.15.42:0/1052486761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64780b6210 0x7f64780af000 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6470009e50 tx=0x7f647001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54981c75ca89f97b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:26.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.120+0000 7f6475400640 1 -- 172.21.15.42:0/1052486761 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64700a5860 con 0x7f64780b6210 2024-05-28T14:11:26.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.120+0000 7f6475400640 1 -- 172.21.15.42:0/1052486761 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64700a5a00 con 0x7f64780b6210 2024-05-28T14:11:26.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.120+0000 7f6475400640 1 -- 172.21.15.42:0/1052486761 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64700a5dd0 con 0x7f64780b6210 2024-05-28T14:11:26.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.120+0000 7f6477200640 1 -- 172.21.15.42:0/1052486761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64780b6210 msgr2=0x7f64780af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:26.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.120+0000 7f6477200640 1 --2- 172.21.15.42:0/1052486761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64780b6210 0x7f64780af000 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6470009e50 tx=0x7f647001c9a0 comp rx=0 tx=0).stop 2024-05-28T14:11:26.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.120+0000 7f6477200640 1 -- 172.21.15.42:0/1052486761 shutdown_connections 2024-05-28T14:11:26.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.120+0000 7f6477200640 1 --2- 172.21.15.42:0/1052486761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64780b6210 0x7f64780af000 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:26.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.120+0000 7f6477200640 1 --2- 172.21.15.42:0/1052486761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6478144f50 0x7f64780706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:26.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.120+0000 7f6477200640 1 -- 172.21.15.42:0/1052486761 >> 172.21.15.42:0/1052486761 conn(0x7f64780b6780 msgr2=0x7f64780b6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:26.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.120+0000 7f6477200640 1 -- 172.21.15.42:0/1052486761 shutdown_connections 2024-05-28T14:11:26.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.120+0000 7f6477200640 1 -- 172.21.15.42:0/1052486761 wait complete. 2024-05-28T14:11:26.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.120+0000 7f6477200640 1 Processor -- start 2024-05-28T14:11:26.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.120+0000 7f6477200640 1 -- start start 2024-05-28T14:11:26.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f6477200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f647806fb60 0x7f6478140f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:26.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f6477200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64780b6210 0x7f647815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:26.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f6477200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6478144f50 0x7f647815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:26.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f6477200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f64780b0160 con 0x7f64780b6210 2024-05-28T14:11:26.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f6477200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64780affe0 con 0x7f6478144f50 2024-05-28T14:11:26.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f6477200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f64780b02e0 con 0x7f647806fb60 2024-05-28T14:11:26.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f646fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64780b6210 0x7f647815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:26.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f646fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64780b6210 0x7f647815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53506/0 (socket says 172.21.15.42:53506) 2024-05-28T14:11:26.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f646fe00640 1 -- 172.21.15.42:0/3426006748 learned_addr learned my addr 172.21.15.42:0/3426006748 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:26.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f6475e00640 1 --2- 172.21.15.42:0/3426006748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f647806fb60 0x7f6478140f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:26.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f6476800640 1 --2- 172.21.15.42:0/3426006748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6478144f50 0x7f647815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:26.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f646fe00640 1 -- 172.21.15.42:0/3426006748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f647806fb60 msgr2=0x7f6478140f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:26.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f646fe00640 1 --2- 172.21.15.42:0/3426006748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f647806fb60 0x7f6478140f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:26.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f646fe00640 1 -- 172.21.15.42:0/3426006748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6478144f50 msgr2=0x7f647815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:26.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f646fe00640 1 --2- 172.21.15.42:0/3426006748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6478144f50 0x7f647815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:26.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f646fe00640 1 -- 172.21.15.42:0/3426006748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64780b2510 con 0x7f64780b6210 2024-05-28T14:11:26.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f646fe00640 1 --2- 172.21.15.42:0/3426006748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64780b6210 0x7f647815d4a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f646000c370 tx=0x7f646000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:26.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f646e000640 1 -- 172.21.15.42:0/3426006748 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64600052d0 con 0x7f64780b6210 2024-05-28T14:11:26.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f646e000640 1 -- 172.21.15.42:0/3426006748 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6460005470 con 0x7f64780b6210 2024-05-28T14:11:26.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f646e000640 1 -- 172.21.15.42:0/3426006748 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6460007260 con 0x7f64780b6210 2024-05-28T14:11:26.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f6477200640 1 -- 172.21.15.42:0/3426006748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64780b27f0 con 0x7f64780b6210 2024-05-28T14:11:26.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f6477200640 1 -- 172.21.15.42:0/3426006748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64780b2db0 con 0x7f64780b6210 2024-05-28T14:11:26.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f6477200640 1 -- 172.21.15.42:0/3426006748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f647804fcf0 con 0x7f64780b6210 2024-05-28T14:11:26.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.124+0000 7f646e000640 1 -- 172.21.15.42:0/3426006748 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f6460007400 con 0x7f64780b6210 2024-05-28T14:11:26.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.128+0000 7f646e000640 1 --2- 172.21.15.42:0/3426006748 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6454069850 0x7f645406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:26.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.128+0000 7f646e000640 1 -- 172.21.15.42:0/3426006748 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f6460084ea0 con 0x7f64780b6210 2024-05-28T14:11:26.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.128+0000 7f6475e00640 1 --2- 172.21.15.42:0/3426006748 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6454069850 0x7f645406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:26.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.128+0000 7f6475e00640 1 --2- 172.21.15.42:0/3426006748 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6454069850 0x7f645406bd10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6468006aa0 tx=0x7f6468008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:26.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.132+0000 7f646e000640 1 -- 172.21.15.42:0/3426006748 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f64600852e0 con 0x7f64780b6210 2024-05-28T14:11:26.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.336+0000 7f6477200640 1 -- 172.21.15.42:0/3426006748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f64780b7330 con 0x7f64780b6210 2024-05-28T14:11:26.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.336+0000 7f646e000640 1 -- 172.21.15.42:0/3426006748 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) ==== 53+0+73 (secure 0 0 0) 0x7f6460056480 con 0x7f64780b6210 2024-05-28T14:11:26.320 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-28T14:11:26.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.336+0000 7f6477200640 1 -- 172.21.15.42:0/3426006748 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6454069850 msgr2=0x7f645406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:26.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.336+0000 7f6477200640 1 --2- 172.21.15.42:0/3426006748 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6454069850 0x7f645406bd10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6468006aa0 tx=0x7f6468008040 comp rx=0 tx=0).stop 2024-05-28T14:11:26.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.336+0000 7f6477200640 1 -- 172.21.15.42:0/3426006748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64780b6210 msgr2=0x7f647815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:26.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.336+0000 7f6477200640 1 --2- 172.21.15.42:0/3426006748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64780b6210 0x7f647815d4a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f646000c370 tx=0x7f646000c830 comp rx=0 tx=0).stop 2024-05-28T14:11:26.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.336+0000 7f6477200640 1 -- 172.21.15.42:0/3426006748 shutdown_connections 2024-05-28T14:11:26.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.336+0000 7f6477200640 1 --2- 172.21.15.42:0/3426006748 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6454069850 0x7f645406bd10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:26.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.336+0000 7f6477200640 1 --2- 172.21.15.42:0/3426006748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6478144f50 0x7f647815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:26.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.336+0000 7f6477200640 1 --2- 172.21.15.42:0/3426006748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64780b6210 0x7f647815d4a0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:26.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.336+0000 7f6477200640 1 --2- 172.21.15.42:0/3426006748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f647806fb60 0x7f6478140f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:26.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.336+0000 7f6477200640 1 -- 172.21.15.42:0/3426006748 >> 172.21.15.42:0/3426006748 conn(0x7f64780b6780 msgr2=0x7f647815f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:26.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.336+0000 7f6477200640 1 -- 172.21.15.42:0/3426006748 shutdown_connections 2024-05-28T14:11:26.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26.336+0000 7f6477200640 1 -- 172.21.15.42:0/3426006748 wait complete. 2024-05-28T14:11:26.342 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-28T14:11:26.342 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-28T14:11:26.342 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-05-28T14:11:26.342 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-28T14:11:26.342 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-28T14:11:26.343 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-28T14:11:26.343 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-28T14:11:26.343 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-28T14:11:26.343 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:11:26.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:26.344 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-05-28T14:11:26.356 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T14:11:26.356 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T14:11:26.356 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T14:11:26.356 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:11:26.356 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-05-28T14:11:26.356 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:11:26.356 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:11:26.357 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-05-28T14:11:26.357 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-05-28T14:11:26.357 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:11:26.357 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-05-28T14:11:26.357 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-05-28T14:11:26.357 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-28T14:11:26.357 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:11:26.357 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-05-28T14:11:26.357 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-28T14:11:26.357 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 41661sec preferred_lft 41661sec 2024-05-28T14:11:26.358 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2024-05-28T14:11:26.358 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:11:26.358 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:11:26.358 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:11:26.359 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-05-28T14:11:26.359 INFO:teuthology.orchestra.run.smithi042.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-28T14:11:26.359 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 02:42:65:2b:63:73 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:11:26.359 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-28T14:11:26.359 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:11:26.359 INFO:teuthology.orchestra.run.smithi042.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-28T14:11:26.359 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-28T14:11:26.359 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-28T14:11:26.359 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:11:26.359 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-28T14:11:26.359 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:11:26.360 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T14:11:26.360 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:11:26.360 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-28T14:11:26.360 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:11:26.360 INFO:teuthology.orchestra.run.smithi042.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T14:11:26.360 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:11:26.361 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-28T14:11:26.361 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:11:26.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:26.362 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-28T14:11:26.416 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-05-28T14:11:26.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:26.418 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-05-28T14:11:26.476 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-05-28T14:11:26.476 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-28T14:11:26.476 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-05-28T14:11:26.476 INFO:teuthology.orchestra.run.smithi042.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-28T14:11:26.476 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-05-28T14:11:26.476 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-28T14:11:26.477 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T14:11:26.477 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T14:11:26.477 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-28T14:11:26.477 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-28T14:11:26.477 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-28T14:11:26.477 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T14:11:26.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T14:11:26.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:11:26.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:26.616 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-28T14:11:26.625 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-28T14:11:26.625 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-28T14:11:26.626 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:11:26.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:26.626 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-05-28T14:11:26.690 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-28T14:11:26.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T14:11:26.690 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-28T14:11:26.742 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:11:26.743 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T14:11:26.743 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-28T14:11:26.794 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:11:26.854 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:11:26.907 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:11:26.908 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:11:26.908 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:11:26.908 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:11:26.908 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:11:26.908 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:11:26.908 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:11:26.908 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:11:26.908 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:11:26.908 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:11:26.908 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:11:26.909 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:11:26.909 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:11:26.909 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:11:26.909 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:11:26.909 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:11:26.909 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:11:26.909 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:11:26.909 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:11:26.909 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:11:26.909 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:11:26.909 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:11:26.909 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:11:26.910 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:11:26.910 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:11:26.910 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:11:26.910 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:11:26.910 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:11:26.910 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:11:26.910 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:11:26.910 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:11:26.910 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:11:26.910 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:11:26.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:26.911 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:11:26.966 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-28T14:11:26.966 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-28T14:11:27.013 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:11:27.034 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:11:27.092 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:11:27.092 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:11:27.092 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:11:27.092 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:11:27.092 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:11:27.092 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:11:27.092 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:11:27.092 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:11:27.092 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:11:27.092 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:11:27.092 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:11:27.093 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:11:27.093 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:11:27.093 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:11:27.093 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:11:27.093 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:11:27.093 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:11:27.093 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:11:27.093 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:11:27.093 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:11:27.093 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:11:27.093 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:11:27.093 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:11:27.093 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:11:27.093 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:11:27.094 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:11:27.094 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:11:27.094 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:11:27.094 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:11:27.094 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:11:27.094 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:11:27.094 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:11:27.094 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:11:27.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:27.095 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:11:27.111 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-05-28T14:11:27.128+0000 7f821af0e580 -1 init, newargv = 0x55d5dcf766a0 newargc=15 2024-05-28T14:11:27.111 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[50525]: starting ceph client 2024-05-28T14:11:27.118 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[50525]: starting fuse 2024-05-28T14:11:27.119 INFO:teuthology.orchestra.run.smithi042.stdout:44 2024-05-28T14:11:27.120 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-28T14:11:27.120 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-05-28T14:11:27.120 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-05-28T14:11:27.120 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-05-28T14:11:27.120 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-05-28T14:11:27.120 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-05-28T14:11:27.120 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:11:27.120 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-05-28T14:11:27.120 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-05-28T14:11:27.121 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-05-28T14:11:27.122 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-28T14:11:27.122 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:11:27.122 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-05-28T14:11:27.122 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-05-28T14:11:27.245 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.50525.asok 2024-05-28T14:11:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:11:27.257 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50525.asok 2024-05-28T14:11:27.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:27.257 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50525.asok status 2024-05-28T14:11:27.436 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:11:27.436 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-05-28T14:11:27.436 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-28T14:11:27.436 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-28T14:11:27.436 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-05-28T14:11:27.436 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-05-28T14:11:27.436 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-28T14:11:27.436 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "50525", 2024-05-28T14:11:27.436 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-05-28T14:11:27.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:27.437 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-05-28T14:11:27.437 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-05-28T14:11:27.437 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 8799, 2024-05-28T14:11:27.437 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-05-28T14:11:27.437 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-05-28T14:11:27.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-05-28T14:11:27.437 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 8799 2024-05-28T14:11:27.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:27.437 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T14:11:27.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T14:11:27.437 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T14:11:27.437 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2095458593 2024-05-28T14:11:27.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:27.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:27.438 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T14:11:27.438 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T14:11:27.438 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T14:11:27.438 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2095458593 2024-05-28T14:11:27.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:11:27.438 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.8799 192.168.0.1:0/2095458593", 2024-05-28T14:11:27.438 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/2095458593", 2024-05-28T14:11:27.438 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-05-28T14:11:27.438 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 64, 2024-05-28T14:11:27.438 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 105, 2024-05-28T14:11:27.438 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-05-28T14:11:27.439 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-05-28T14:11:27.439 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-05-28T14:11:27.439 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:11:27.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:27.453 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-05-28T14:11:27.453 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-28T14:11:27.462 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-05-28T14:11:27.462 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-28T14:11:27.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:27.463 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-28T14:11:27.526 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-28T14:11:27.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.728+0000 7fad1a400640 1 Processor -- start 2024-05-28T14:11:27.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.728+0000 7fad1a400640 1 -- start start 2024-05-28T14:11:27.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad1a400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fad1415eae0 0x7fad140d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:27.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad1a400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad1406c100 0x7fad140b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:27.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad1a400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad1415eec0 con 0x7fad140b38f0 2024-05-28T14:11:27.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad1a400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fad140cd3b0 con 0x7fad1415eae0 2024-05-28T14:11:27.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad1a400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fad140ccf20 con 0x7fad1406c100 2024-05-28T14:11:27.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad0be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fad140b38f0 0x7fad1406b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:35538/0 (socket says 172.21.15.42:35538) 2024-05-28T14:11:27.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad0be00640 1 -- 172.21.15.42:0/974211691 learned_addr learned my addr 172.21.15.42:0/974211691 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:27.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad13e00640 1 --2- 172.21.15.42:0/974211691 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad1406c100 0x7fad140b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:27.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad13400640 1 --2- 172.21.15.42:0/974211691 >> v2:172.21.15.63:3300/0 conn(0x7fad1415eae0 0x7fad140d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:27.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 -- 172.21.15.42:0/974211691 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4265704133 0 0) 0x7fad1415eec0 con 0x7fad140b38f0 2024-05-28T14:11:27.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 -- 172.21.15.42:0/974211691 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facf8003530 con 0x7fad140b38f0 2024-05-28T14:11:27.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 -- 172.21.15.42:0/974211691 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2418384363 0 0) 0x7facf8003530 con 0x7fad140b38f0 2024-05-28T14:11:27.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 -- 172.21.15.42:0/974211691 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad1406c100 msgr2=0x7fad140b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:27.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 --2- 172.21.15.42:0/974211691 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad1406c100 0x7fad140b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:27.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 -- 172.21.15.42:0/974211691 >> v2:172.21.15.63:3300/0 conn(0x7fad1415eae0 msgr2=0x7fad140d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:27.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 --2- 172.21.15.42:0/974211691 >> v2:172.21.15.63:3300/0 conn(0x7fad1415eae0 0x7fad140d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:27.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 -- 172.21.15.42:0/974211691 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad140cc5f0 con 0x7fad140b38f0 2024-05-28T14:11:27.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 -- 172.21.15.42:0/974211691 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7facfc001680 con 0x7fad140b38f0 2024-05-28T14:11:27.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad13400640 1 --2- 172.21.15.42:0/974211691 >> v2:172.21.15.63:3300/0 conn(0x7fad1415eae0 0x7fad140d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:11:27.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 -- 172.21.15.42:0/974211691 >> v1:172.21.15.42:6789/0 conn(0x7fad140b38f0 legacy=0x7fad1406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:27.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 --2- 172.21.15.42:0/974211691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf80039c0 0x7facf8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:27.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 --2- 172.21.15.42:0/974211691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facf8004810 0x7facf8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 --2- 172.21.15.42:0/974211691 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad1406c100 0x7facf80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 -- 172.21.15.42:0/974211691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fad140cd3b0 con 0x7facf80039c0 2024-05-28T14:11:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 -- 172.21.15.42:0/974211691 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fad140ccf20 con 0x7facf8004810 2024-05-28T14:11:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad12a00640 1 -- 172.21.15.42:0/974211691 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7facf800ba00 con 0x7fad1406c100 2024-05-28T14:11:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad13400640 1 --2- 172.21.15.42:0/974211691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf80039c0 0x7facf8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad0be00640 1 --2- 172.21.15.42:0/974211691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facf8004810 0x7facf8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad13e00640 1 --2- 172.21.15.42:0/974211691 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad1406c100 0x7facf80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad13400640 1 -- 172.21.15.42:0/974211691 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad1406c100 msgr2=0x7facf80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad13400640 1 --2- 172.21.15.42:0/974211691 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad1406c100 0x7facf80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad13400640 1 -- 172.21.15.42:0/974211691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facf8004810 msgr2=0x7facf8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad13400640 1 --2- 172.21.15.42:0/974211691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facf8004810 0x7facf8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad13400640 1 -- 172.21.15.42:0/974211691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf8003530 con 0x7facf80039c0 2024-05-28T14:11:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.732+0000 7fad0be00640 1 --2- 172.21.15.42:0/974211691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facf8004810 0x7facf8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:27.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.736+0000 7fad13400640 1 --2- 172.21.15.42:0/974211691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf80039c0 0x7facf8003de0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fad04009cb0 tx=0x7fad0401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2937f3c38bea2e97 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:27.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.736+0000 7fad12a00640 1 -- 172.21.15.42:0/974211691 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad040b2070 con 0x7facf80039c0 2024-05-28T14:11:27.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.736+0000 7fad12a00640 1 -- 172.21.15.42:0/974211691 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad040a5740 con 0x7facf80039c0 2024-05-28T14:11:27.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.736+0000 7fad12a00640 1 -- 172.21.15.42:0/974211691 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04009880 con 0x7facf80039c0 2024-05-28T14:11:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.736+0000 7fad1a400640 1 -- 172.21.15.42:0/974211691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf80039c0 msgr2=0x7facf8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.736+0000 7fad1a400640 1 --2- 172.21.15.42:0/974211691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf80039c0 0x7facf8003de0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fad04009cb0 tx=0x7fad0401cc30 comp rx=0 tx=0).stop 2024-05-28T14:11:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.736+0000 7fad13e00640 1 -- 172.21.15.42:0/974211691 reap_dead start 2024-05-28T14:11:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.736+0000 7fad1a400640 1 -- 172.21.15.42:0/974211691 shutdown_connections 2024-05-28T14:11:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.736+0000 7fad1a400640 1 -- 172.21.15.42:0/974211691 >> 172.21.15.42:0/974211691 conn(0x7fad1409aa50 msgr2=0x7fad140b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.736+0000 7fad1a400640 1 -- 172.21.15.42:0/974211691 shutdown_connections 2024-05-28T14:11:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.736+0000 7fad1a400640 1 -- 172.21.15.42:0/974211691 wait complete. 2024-05-28T14:11:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad1a400640 1 Processor -- start 2024-05-28T14:11:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad1a400640 1 -- start start 2024-05-28T14:11:27.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad1a400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf80039c0 0x7fad140c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:27.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad1a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad140b0080 0x7fad140b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:27.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad1a400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad14156640 0x7fad14156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:27.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad1a400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fad140b14d0 con 0x7fad140b0080 2024-05-28T14:11:27.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad1a400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fad140b1350 con 0x7fad14156640 2024-05-28T14:11:27.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad1a400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fad140b1650 con 0x7facf80039c0 2024-05-28T14:11:27.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad0be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad140b0080 0x7fad140b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:27.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad0be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad140b0080 0x7fad140b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45160/0 (socket says 172.21.15.42:45160) 2024-05-28T14:11:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad0be00640 1 -- 172.21.15.42:0/328591868 learned_addr learned my addr 172.21.15.42:0/328591868 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad13e00640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad14156640 0x7fad14156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad13400640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf80039c0 0x7fad140c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad0be00640 1 -- 172.21.15.42:0/328591868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf80039c0 msgr2=0x7fad140c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad0be00640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf80039c0 0x7fad140c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad0be00640 1 -- 172.21.15.42:0/328591868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad14156640 msgr2=0x7fad14156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad0be00640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad14156640 0x7fad14156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad0be00640 1 -- 172.21.15.42:0/328591868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad140af2c0 con 0x7fad140b0080 2024-05-28T14:11:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad13e00640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad14156640 0x7fad14156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad13400640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf80039c0 0x7fad140c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:27.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad0be00640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad140b0080 0x7fad140b04e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7facfc001870 tx=0x7facfc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:27.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad0b400640 1 -- 172.21.15.42:0/328591868 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc00ee70 con 0x7fad140b0080 2024-05-28T14:11:27.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad0b400640 1 -- 172.21.15.42:0/328591868 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facfc018070 con 0x7fad140b0080 2024-05-28T14:11:27.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad0b400640 1 -- 172.21.15.42:0/328591868 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc00ca70 con 0x7fad140b0080 2024-05-28T14:11:27.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad1a400640 1 -- 172.21.15.42:0/328591868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad140af5a0 con 0x7fad140b0080 2024-05-28T14:11:27.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad1a400640 1 -- 172.21.15.42:0/328591868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad140afb60 con 0x7fad140b0080 2024-05-28T14:11:27.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.740+0000 7fad1a400640 1 -- 172.21.15.42:0/328591868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad14068a90 con 0x7fad140b0080 2024-05-28T14:11:27.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.744+0000 7fad0b400640 1 -- 172.21.15.42:0/328591868 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7facfc00cc10 con 0x7fad140b0080 2024-05-28T14:11:27.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.744+0000 7fad0b400640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7facf0069880 0x7facf006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:27.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.744+0000 7fad0b400640 1 -- 172.21.15.42:0/328591868 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7facfc085940 con 0x7fad140b0080 2024-05-28T14:11:27.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.744+0000 7fad13400640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7facf0069880 0x7facf006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:27.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.744+0000 7fad13400640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7facf0069880 0x7facf006bd40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fad040af000 tx=0x7fad040046d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:27.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.748+0000 7fad0b400640 1 -- 172.21.15.42:0/328591868 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7facfc0574e0 con 0x7fad140b0080 2024-05-28T14:11:27.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.948+0000 7fad1a400640 1 -- 172.21.15.42:0/328591868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fad140afd30 con 0x7fad140b0080 2024-05-28T14:11:27.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.952+0000 7fad0b400640 1 -- 172.21.15.42:0/328591868 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v65) ==== 136+0+0 (secure 0 0 0) 0x7facfc085de0 con 0x7fad140b0080 2024-05-28T14:11:27.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.956+0000 7fad1a400640 1 -- 172.21.15.42:0/328591868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7facf0069880 msgr2=0x7facf006bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:27.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.956+0000 7fad1a400640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7facf0069880 0x7facf006bd40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fad040af000 tx=0x7fad040046d0 comp rx=0 tx=0).stop 2024-05-28T14:11:27.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.956+0000 7fad1a400640 1 -- 172.21.15.42:0/328591868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad140b0080 msgr2=0x7fad140b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:27.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.956+0000 7fad1a400640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad140b0080 0x7fad140b04e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7facfc001870 tx=0x7facfc00ec80 comp rx=0 tx=0).stop 2024-05-28T14:11:27.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.956+0000 7fad1a400640 1 -- 172.21.15.42:0/328591868 shutdown_connections 2024-05-28T14:11:27.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.956+0000 7fad1a400640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7facf0069880 0x7facf006bd40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:27.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.956+0000 7fad1a400640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad14156640 0x7fad14156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:27.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.956+0000 7fad1a400640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad140b0080 0x7fad140b04e0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:27.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.956+0000 7fad1a400640 1 --2- 172.21.15.42:0/328591868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf80039c0 0x7fad140c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:27.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.956+0000 7fad1a400640 1 -- 172.21.15.42:0/328591868 >> 172.21.15.42:0/328591868 conn(0x7fad1409aa50 msgr2=0x7fad1406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:27.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.956+0000 7fad1a400640 1 -- 172.21.15.42:0/328591868 shutdown_connections 2024-05-28T14:11:27.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:27.956+0000 7fad1a400640 1 -- 172.21.15.42:0/328591868 wait complete. 2024-05-28T14:11:27.960 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:11:28.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd8200640 1 Processor -- start 2024-05-28T14:11:28.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd8200640 1 -- start start 2024-05-28T14:11:28.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd8200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0cd015e820 0x7f0cd00b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd8200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cd00ae6a0 0x7f0cd006b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd8200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cd00b6590 con 0x7f0cd00ae240 2024-05-28T14:11:28.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd8200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0cd006c870 con 0x7f0cd015e820 2024-05-28T14:11:28.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd8200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd00b5d30 con 0x7f0cd00ae6a0 2024-05-28T14:11:28.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd5800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0cd015e820 0x7f0cd00b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd5800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0cd015e820 0x7f0cd00b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45168/0 (socket says 172.21.15.42:45168) 2024-05-28T14:11:28.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd5800640 1 -- 172.21.15.42:0/2259882510 learned_addr learned my addr 172.21.15.42:0/2259882510 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:28.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd6200640 1 --2- 172.21.15.42:0/2259882510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cd00ae6a0 0x7f0cd006b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd5800640 1 -- 172.21.15.42:0/2259882510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cd00ae6a0 msgr2=0x7f0cd006b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:28.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd5800640 1 --2- 172.21.15.42:0/2259882510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cd00ae6a0 0x7f0cd006b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd5800640 1 -- 172.21.15.42:0/2259882510 >> v1:172.21.15.42:6789/0 conn(0x7f0cd00ae240 legacy=0x7f0cd00b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd5800640 1 -- 172.21.15.42:0/2259882510 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd015ec00 con 0x7f0cd015e820 2024-05-28T14:11:28.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.120+0000 7f0cd6200640 1 --2- 172.21.15.42:0/2259882510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cd00ae6a0 0x7f0cd006b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:28.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.124+0000 7f0cd5800640 1 --2- 172.21.15.42:0/2259882510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd015e820 0x7f0cd00b4260 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009870 tx=0x7f0cc401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dff2a1d9e51a239 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:28.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.124+0000 7f0ccfe00640 1 -- 172.21.15.42:0/2259882510 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc40b2070 con 0x7f0cd015e820 2024-05-28T14:11:28.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.124+0000 7f0ccfe00640 1 -- 172.21.15.42:0/2259882510 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc40a9070 con 0x7f0cd015e820 2024-05-28T14:11:28.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.124+0000 7f0ccfe00640 1 -- 172.21.15.42:0/2259882510 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc40ad880 con 0x7f0cd015e820 2024-05-28T14:11:28.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.124+0000 7f0cd8200640 1 -- 172.21.15.42:0/2259882510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd015e820 msgr2=0x7f0cd00b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.124+0000 7f0cd8200640 1 --2- 172.21.15.42:0/2259882510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd015e820 0x7f0cd00b4260 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009870 tx=0x7f0cc401c560 comp rx=0 tx=0).stop 2024-05-28T14:11:28.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.124+0000 7f0cd8200640 1 -- 172.21.15.42:0/2259882510 shutdown_connections 2024-05-28T14:11:28.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.124+0000 7f0cd8200640 1 --2- 172.21.15.42:0/2259882510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cd00ae6a0 0x7f0cd006b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.124+0000 7f0cd8200640 1 --2- 172.21.15.42:0/2259882510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd015e820 0x7f0cd00b4260 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.124+0000 7f0cd8200640 1 -- 172.21.15.42:0/2259882510 >> 172.21.15.42:0/2259882510 conn(0x7f0cd00b6780 msgr2=0x7f0cd00b6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:28.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.124+0000 7f0cd8200640 1 -- 172.21.15.42:0/2259882510 shutdown_connections 2024-05-28T14:11:28.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.124+0000 7f0cd8200640 1 -- 172.21.15.42:0/2259882510 wait complete. 2024-05-28T14:11:28.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd8200640 1 Processor -- start 2024-05-28T14:11:28.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd8200640 1 -- start start 2024-05-28T14:11:28.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd8200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd00ae240 0x7f0cd009d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd8200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cd00ae6a0 0x7f0cd009e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd8200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cd015e820 0x7f0cd009edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd8200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd00b47a0 con 0x7f0cd00ae240 2024-05-28T14:11:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd8200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd006c340 con 0x7f0cd00ae6a0 2024-05-28T14:11:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd8200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd0158750 con 0x7f0cd015e820 2024-05-28T14:11:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd00ae240 0x7f0cd009d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd00ae240 0x7f0cd009d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45178/0 (socket says 172.21.15.42:45178) 2024-05-28T14:11:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd5800640 1 -- 172.21.15.42:0/1142838099 learned_addr learned my addr 172.21.15.42:0/1142838099 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd6200640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cd015e820 0x7f0cd009edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd4e00640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cd00ae6a0 0x7f0cd009e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd5800640 1 -- 172.21.15.42:0/1142838099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cd015e820 msgr2=0x7f0cd009edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd5800640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cd015e820 0x7f0cd009edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd5800640 1 -- 172.21.15.42:0/1142838099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cd00ae6a0 msgr2=0x7f0cd009e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd5800640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cd00ae6a0 0x7f0cd009e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd5800640 1 -- 172.21.15.42:0/1142838099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd009c410 con 0x7f0cd00ae240 2024-05-28T14:11:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd6200640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cd015e820 0x7f0cd009edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:28.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd5800640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd00ae240 0x7f0cd009d770 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40b0040 tx=0x7f0cc401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:28.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0ccd600640 1 -- 172.21.15.42:0/1142838099 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc40b2040 con 0x7f0cd00ae240 2024-05-28T14:11:28.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0ccd600640 1 -- 172.21.15.42:0/1142838099 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc40adcd0 con 0x7f0cd00ae240 2024-05-28T14:11:28.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd8200640 1 -- 172.21.15.42:0/1142838099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd009c6d0 con 0x7f0cd00ae240 2024-05-28T14:11:28.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0ccd600640 1 -- 172.21.15.42:0/1142838099 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc40b8680 con 0x7f0cd00ae240 2024-05-28T14:11:28.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.128+0000 7f0cd8200640 1 -- 172.21.15.42:0/1142838099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd009cc90 con 0x7f0cd00ae240 2024-05-28T14:11:28.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.132+0000 7f0cd8200640 1 -- 172.21.15.42:0/1142838099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd009f5c0 con 0x7f0cd00ae240 2024-05-28T14:11:28.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.132+0000 7f0ccd600640 1 -- 172.21.15.42:0/1142838099 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f0cc40c0030 con 0x7f0cd00ae240 2024-05-28T14:11:28.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.132+0000 7f0ccd600640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0cb0069850 0x7f0cb006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.132+0000 7f0ccd600640 1 -- 172.21.15.42:0/1142838099 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f0cc411f550 con 0x7f0cd00ae240 2024-05-28T14:11:28.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.132+0000 7f0cd4e00640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0cb0069850 0x7f0cb006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.132+0000 7f0cd4e00640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0cb0069850 0x7f0cb006bd10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc001650 tx=0x7f0cbc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:28.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.136+0000 7f0ccd600640 1 -- 172.21.15.42:0/1142838099 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0cc40c7060 con 0x7f0cd00ae240 2024-05-28T14:11:28.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.324+0000 7f0cd8200640 1 -- 172.21.15.42:0/1142838099 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f0cd0163240 con 0x7f0cb0069850 2024-05-28T14:11:28.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.324+0000 7f0ccd600640 1 -- 172.21.15.42:0/1142838099 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f0cd0163240 con 0x7f0cb0069850 2024-05-28T14:11:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:11:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:11:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:11:28.310 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:11:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:11:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.328+0000 7f0cd8200640 1 -- 172.21.15.42:0/1142838099 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0cb0069850 msgr2=0x7f0cb006bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.328+0000 7f0cd8200640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0cb0069850 0x7f0cb006bd10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc001650 tx=0x7f0cbc009040 comp rx=0 tx=0).stop 2024-05-28T14:11:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.328+0000 7f0cd8200640 1 -- 172.21.15.42:0/1142838099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd00ae240 msgr2=0x7f0cd009d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.328+0000 7f0cd8200640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd00ae240 0x7f0cd009d770 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40b0040 tx=0x7f0cc401ada0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.328+0000 7f0cd8200640 1 -- 172.21.15.42:0/1142838099 shutdown_connections 2024-05-28T14:11:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.328+0000 7f0cd8200640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0cb0069850 0x7f0cb006bd10 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.328+0000 7f0cd8200640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cd015e820 0x7f0cd009edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.328+0000 7f0cd8200640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cd00ae6a0 0x7f0cd009e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.328+0000 7f0cd8200640 1 --2- 172.21.15.42:0/1142838099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd00ae240 0x7f0cd009d770 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.328+0000 7f0cd8200640 1 -- 172.21.15.42:0/1142838099 >> 172.21.15.42:0/1142838099 conn(0x7f0cd00b6780 msgr2=0x7f0cd0070760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.328+0000 7f0cd8200640 1 -- 172.21.15.42:0/1142838099 shutdown_connections 2024-05-28T14:11:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.328+0000 7f0cd8200640 1 -- 172.21.15.42:0/1142838099 wait complete. 2024-05-28T14:11:28.331 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T14:11:28.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc47b200640 1 Processor -- start 2024-05-28T14:11:28.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc47b200640 1 -- start start 2024-05-28T14:11:28.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc47b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc474071de0 0x7fc4740721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc47b200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc4740ce3b0 0x7fc4740ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc47b200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc474072700 con 0x7fc474141d10 2024-05-28T14:11:28.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc47b200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc474160230 con 0x7fc4740ce3b0 2024-05-28T14:11:28.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc47b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc474160510 con 0x7fc474071de0 2024-05-28T14:11:28.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc46be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc4740ce3b0 0x7fc4740ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc46be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc4740ce3b0 0x7fc4740ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45192/0 (socket says 172.21.15.42:45192) 2024-05-28T14:11:28.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc46be00640 1 -- 172.21.15.42:0/3596928985 learned_addr learned my addr 172.21.15.42:0/3596928985 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:28.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc473e00640 1 --2- 172.21.15.42:0/3596928985 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc474071de0 0x7fc4740721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc46be00640 1 -- 172.21.15.42:0/3596928985 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc474071de0 msgr2=0x7fc4740721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:28.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc46be00640 1 --2- 172.21.15.42:0/3596928985 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc474071de0 0x7fc4740721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc46be00640 1 -- 172.21.15.42:0/3596928985 >> v1:172.21.15.63:6789/0 conn(0x7fc474141d10 legacy=0x7fc4741421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:28.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.492+0000 7fc46be00640 1 -- 172.21.15.42:0/3596928985 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4741607f0 con 0x7fc4740ce3b0 2024-05-28T14:11:28.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.496+0000 7fc46be00640 1 --2- 172.21.15.42:0/3596928985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4740ce3b0 0x7fc4740ce810 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009870 tx=0x7fc45c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee3deb228d428d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:28.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.496+0000 7fc473400640 1 -- 172.21.15.42:0/3596928985 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0b0070 con 0x7fc4740ce3b0 2024-05-28T14:11:28.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.496+0000 7fc473400640 1 -- 172.21.15.42:0/3596928985 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45c0ab600 con 0x7fc4740ce3b0 2024-05-28T14:11:28.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.496+0000 7fc473400640 1 -- 172.21.15.42:0/3596928985 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0ab9d0 con 0x7fc4740ce3b0 2024-05-28T14:11:28.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.496+0000 7fc47b200640 1 -- 172.21.15.42:0/3596928985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4740ce3b0 msgr2=0x7fc4740ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.496+0000 7fc47b200640 1 --2- 172.21.15.42:0/3596928985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4740ce3b0 0x7fc4740ce810 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009870 tx=0x7fc45c01c440 comp rx=0 tx=0).stop 2024-05-28T14:11:28.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.496+0000 7fc47b200640 1 -- 172.21.15.42:0/3596928985 shutdown_connections 2024-05-28T14:11:28.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.496+0000 7fc47b200640 1 --2- 172.21.15.42:0/3596928985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4740ce3b0 0x7fc4740ce810 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.496+0000 7fc47b200640 1 --2- 172.21.15.42:0/3596928985 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc474071de0 0x7fc4740721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.496+0000 7fc47b200640 1 -- 172.21.15.42:0/3596928985 >> 172.21.15.42:0/3596928985 conn(0x7fc4740b21f0 msgr2=0x7fc4740b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:28.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.496+0000 7fc47b200640 1 -- 172.21.15.42:0/3596928985 shutdown_connections 2024-05-28T14:11:28.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.496+0000 7fc47b200640 1 -- 172.21.15.42:0/3596928985 wait complete. 2024-05-28T14:11:28.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc47b200640 1 Processor -- start 2024-05-28T14:11:28.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc47b200640 1 -- start start 2024-05-28T14:11:28.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc47b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc474071de0 0x7fc474137bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc47b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4740ce3b0 0x7fc4741f98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc47b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc474141d10 0x7fc474136c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc47b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc474169e30 con 0x7fc4740ce3b0 2024-05-28T14:11:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc47b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc474169cb0 con 0x7fc474141d10 2024-05-28T14:11:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc47b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc474169fb0 con 0x7fc474071de0 2024-05-28T14:11:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc46be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4740ce3b0 0x7fc4741f98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc473e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc474071de0 0x7fc474137bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc46be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4740ce3b0 0x7fc4741f98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45206/0 (socket says 172.21.15.42:45206) 2024-05-28T14:11:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc46be00640 1 -- 172.21.15.42:0/3240450337 learned_addr learned my addr 172.21.15.42:0/3240450337 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc479200640 1 --2- 172.21.15.42:0/3240450337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc474141d10 0x7fc474136c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc46be00640 1 -- 172.21.15.42:0/3240450337 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc474071de0 msgr2=0x7fc474137bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc46be00640 1 --2- 172.21.15.42:0/3240450337 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc474071de0 0x7fc474137bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc46be00640 1 -- 172.21.15.42:0/3240450337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc474141d10 msgr2=0x7fc474136c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc46be00640 1 --2- 172.21.15.42:0/3240450337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc474141d10 0x7fc474136c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc46be00640 1 -- 172.21.15.42:0/3240450337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc474155510 con 0x7fc4740ce3b0 2024-05-28T14:11:28.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc46be00640 1 --2- 172.21.15.42:0/3240450337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4740ce3b0 0x7fc4741f98e0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0ae040 tx=0x7fc45c01ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:28.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc470c00640 1 -- 172.21.15.42:0/3240450337 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0ba020 con 0x7fc4740ce3b0 2024-05-28T14:11:28.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc470c00640 1 -- 172.21.15.42:0/3240450337 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45c0b6620 con 0x7fc4740ce3b0 2024-05-28T14:11:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc470c00640 1 -- 172.21.15.42:0/3240450337 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0b0040 con 0x7fc4740ce3b0 2024-05-28T14:11:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc47b200640 1 -- 172.21.15.42:0/3240450337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4741fa3a0 con 0x7fc4740ce3b0 2024-05-28T14:11:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc47b200640 1 -- 172.21.15.42:0/3240450337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4741557d0 con 0x7fc4740ce3b0 2024-05-28T14:11:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.500+0000 7fc47b200640 1 -- 172.21.15.42:0/3240450337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4740738b0 con 0x7fc4740ce3b0 2024-05-28T14:11:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.504+0000 7fc470c00640 1 -- 172.21.15.42:0/3240450337 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fc45c0bd020 con 0x7fc4740ce3b0 2024-05-28T14:11:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.504+0000 7fc470c00640 1 --2- 172.21.15.42:0/3240450337 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc450069880 0x7fc45006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.504+0000 7fc470c00640 1 -- 172.21.15.42:0/3240450337 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fc45c11f7a0 con 0x7fc4740ce3b0 2024-05-28T14:11:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.504+0000 7fc473e00640 1 --2- 172.21.15.42:0/3240450337 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc450069880 0x7fc45006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.504+0000 7fc473e00640 1 --2- 172.21.15.42:0/3240450337 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc450069880 0x7fc45006bd40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc4640079c0 tx=0x7fc464008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:28.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.508+0000 7fc470c00640 1 -- 172.21.15.42:0/3240450337 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc45c0c4030 con 0x7fc4740ce3b0 2024-05-28T14:11:28.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.672+0000 7fc47b200640 1 -- 172.21.15.42:0/3240450337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc47404fdd0 con 0x7fc4740ce3b0 2024-05-28T14:11:28.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.672+0000 7fc470c00640 1 -- 172.21.15.42:0/3240450337 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7fc45c016150 con 0x7fc4740ce3b0 2024-05-28T14:11:28.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.676+0000 7fc47b200640 1 -- 172.21.15.42:0/3240450337 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc450069880 msgr2=0x7fc45006bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.676+0000 7fc47b200640 1 --2- 172.21.15.42:0/3240450337 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc450069880 0x7fc45006bd40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc4640079c0 tx=0x7fc464008040 comp rx=0 tx=0).stop 2024-05-28T14:11:28.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.676+0000 7fc47b200640 1 -- 172.21.15.42:0/3240450337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4740ce3b0 msgr2=0x7fc4741f98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.676+0000 7fc47b200640 1 --2- 172.21.15.42:0/3240450337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4740ce3b0 0x7fc4741f98e0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0ae040 tx=0x7fc45c01ac80 comp rx=0 tx=0).stop 2024-05-28T14:11:28.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.676+0000 7fc47b200640 1 -- 172.21.15.42:0/3240450337 shutdown_connections 2024-05-28T14:11:28.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.676+0000 7fc47b200640 1 --2- 172.21.15.42:0/3240450337 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc450069880 0x7fc45006bd40 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.676+0000 7fc47b200640 1 --2- 172.21.15.42:0/3240450337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc474141d10 0x7fc474136c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.676+0000 7fc47b200640 1 --2- 172.21.15.42:0/3240450337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4740ce3b0 0x7fc4741f98e0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.676+0000 7fc47b200640 1 --2- 172.21.15.42:0/3240450337 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc474071de0 0x7fc474137bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.676+0000 7fc47b200640 1 -- 172.21.15.42:0/3240450337 >> 172.21.15.42:0/3240450337 conn(0x7fc4740b21f0 msgr2=0x7fc47415ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:28.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.676+0000 7fc47b200640 1 -- 172.21.15.42:0/3240450337 shutdown_connections 2024-05-28T14:11:28.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.676+0000 7fc47b200640 1 -- 172.21.15.42:0/3240450337 wait complete. 2024-05-28T14:11:28.680 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-28T14:11:28.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.840+0000 7fb5dc000640 1 Processor -- start 2024-05-28T14:11:28.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.840+0000 7fb5dc000640 1 -- start start 2024-05-28T14:11:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.840+0000 7fb5dc000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb5d40d0e00 0x7fb5d406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.840+0000 7fb5dc000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d41570a0 con 0x7fb5d406ee80 2024-05-28T14:11:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.840+0000 7fb5dc000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d415cf00 con 0x7fb5d40afa70 2024-05-28T14:11:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.840+0000 7fb5dc000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb5d415d1e0 con 0x7fb5d40d0e00 2024-05-28T14:11:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.844+0000 7fb5da000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb5d40d0e00 0x7fb5d406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.844+0000 7fb5da000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb5d40d0e00 0x7fb5d406dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45212/0 (socket says 172.21.15.42:45212) 2024-05-28T14:11:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.844+0000 7fb5da000640 1 -- 172.21.15.42:0/2857614701 learned_addr learned my addr 172.21.15.42:0/2857614701 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.844+0000 7fb5d9600640 1 --1- 172.21.15.42:0/2857614701 >> v1:172.21.15.42:6789/0 conn(0x7fb5d406ee80 0x7fb5d40af360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:35556/0 (socket says 172.21.15.42:35556) 2024-05-28T14:11:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.844+0000 7fb5da000640 1 -- 172.21.15.42:0/2857614701 >> v1:172.21.15.63:6789/0 conn(0x7fb5d40afa70 legacy=0x7fb5d40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.844+0000 7fb5da000640 1 -- 172.21.15.42:0/2857614701 >> v1:172.21.15.42:6789/0 conn(0x7fb5d406ee80 legacy=0x7fb5d40af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.844+0000 7fb5da000640 1 -- 172.21.15.42:0/2857614701 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d415d4c0 con 0x7fb5d40d0e00 2024-05-28T14:11:28.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.844+0000 7fb5da000640 1 --2- 172.21.15.42:0/2857614701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5d40d0e00 0x7fb5d406dd40 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009e50 tx=0x7fb5cc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb023af572fde95 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:28.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.844+0000 7fb5d3e00640 1 -- 172.21.15.42:0/2857614701 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0a5790 con 0x7fb5d40d0e00 2024-05-28T14:11:28.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.844+0000 7fb5d3e00640 1 -- 172.21.15.42:0/2857614701 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5cc0a5930 con 0x7fb5d40d0e00 2024-05-28T14:11:28.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.844+0000 7fb5d3e00640 1 -- 172.21.15.42:0/2857614701 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0a5d00 con 0x7fb5d40d0e00 2024-05-28T14:11:28.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 -- 172.21.15.42:0/2857614701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5d40d0e00 msgr2=0x7fb5d406dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 --2- 172.21.15.42:0/2857614701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5d40d0e00 0x7fb5d406dd40 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009e50 tx=0x7fb5cc01c970 comp rx=0 tx=0).stop 2024-05-28T14:11:28.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 -- 172.21.15.42:0/2857614701 shutdown_connections 2024-05-28T14:11:28.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 --2- 172.21.15.42:0/2857614701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5d40d0e00 0x7fb5d406dd40 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 -- 172.21.15.42:0/2857614701 >> 172.21.15.42:0/2857614701 conn(0x7fb5d406bdf0 msgr2=0x7fb5d406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:28.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 -- 172.21.15.42:0/2857614701 shutdown_connections 2024-05-28T14:11:28.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 -- 172.21.15.42:0/2857614701 wait complete. 2024-05-28T14:11:28.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 Processor -- start 2024-05-28T14:11:28.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 -- start start 2024-05-28T14:11:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5d406ee80 0x7fb5d414dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5d40afa70 0x7fb5d4157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5d40d0e00 0x7fb5d41577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4160e10 con 0x7fb5d40afa70 2024-05-28T14:11:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4160c90 con 0x7fb5d40d0e00 2024-05-28T14:11:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5dc000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d4160f90 con 0x7fb5d406ee80 2024-05-28T14:11:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5d8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5d40afa70 0x7fb5d4157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5d8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5d40afa70 0x7fb5d4157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45222/0 (socket says 172.21.15.42:45222) 2024-05-28T14:11:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5d8c00640 1 -- 172.21.15.42:0/3248068139 learned_addr learned my addr 172.21.15.42:0/3248068139 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:28.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5d9600640 1 --2- 172.21.15.42:0/3248068139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5d406ee80 0x7fb5d414dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.848+0000 7fb5da000640 1 --2- 172.21.15.42:0/3248068139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5d40d0e00 0x7fb5d41577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5d8c00640 1 -- 172.21.15.42:0/3248068139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5d406ee80 msgr2=0x7fb5d414dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5d8c00640 1 --2- 172.21.15.42:0/3248068139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5d406ee80 0x7fb5d414dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5d8c00640 1 -- 172.21.15.42:0/3248068139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5d40d0e00 msgr2=0x7fb5d41577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:28.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5d8c00640 1 --2- 172.21.15.42:0/3248068139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5d40d0e00 0x7fb5d41577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:28.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5d8c00640 1 -- 172.21.15.42:0/3248068139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d414e4e0 con 0x7fb5d40afa70 2024-05-28T14:11:28.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5d8c00640 1 --2- 172.21.15.42:0/3248068139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5d40afa70 0x7fb5d4157270 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000c460 tx=0x7fb5c000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:28.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5d1600640 1 -- 172.21.15.42:0/3248068139 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0016020 con 0x7fb5d40afa70 2024-05-28T14:11:28.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5d1600640 1 -- 172.21.15.42:0/3248068139 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c00052d0 con 0x7fb5d40afa70 2024-05-28T14:11:28.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5d1600640 1 -- 172.21.15.42:0/3248068139 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0007260 con 0x7fb5d40afa70 2024-05-28T14:11:28.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5dc000640 1 -- 172.21.15.42:0/3248068139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d40723f0 con 0x7fb5d40afa70 2024-05-28T14:11:28.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5dc000640 1 -- 172.21.15.42:0/3248068139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d40729b0 con 0x7fb5d40afa70 2024-05-28T14:11:28.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5dc000640 1 -- 172.21.15.42:0/3248068139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d4068a90 con 0x7fb5d40afa70 2024-05-28T14:11:28.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5d1600640 1 -- 172.21.15.42:0/3248068139 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fb5c0007400 con 0x7fb5d40afa70 2024-05-28T14:11:28.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5d1600640 1 --2- 172.21.15.42:0/3248068139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb5b4069880 0x7fb5b406bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:28.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5d1600640 1 -- 172.21.15.42:0/3248068139 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fb5c0084f10 con 0x7fb5d40afa70 2024-05-28T14:11:28.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.852+0000 7fb5d9600640 1 --2- 172.21.15.42:0/3248068139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb5b4069880 0x7fb5b406bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:28.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.856+0000 7fb5d9600640 1 --2- 172.21.15.42:0/3248068139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb5b4069880 0x7fb5b406bd40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8000f70 tx=0x7fb5c8007e50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:28.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:28.856+0000 7fb5d1600640 1 -- 172.21.15.42:0/3248068139 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb5c0020020 con 0x7fb5d40afa70 2024-05-28T14:11:29.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.048+0000 7fb5dc000640 1 -- 172.21.15.42:0/3248068139 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fb5d40b5160 con 0x7fb5b4069880 2024-05-28T14:11:29.039 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:29.056+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:29.039 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:29.056+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:29.039 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:29.056+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:29.039 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:29.056+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:29.040 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:29.056+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:29.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.088+0000 7fb5d1600640 1 -- 172.21.15.42:0/3248068139 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb5d40b5160 con 0x7fb5b4069880 2024-05-28T14:11:29.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.092+0000 7fb5dc000640 1 -- 172.21.15.42:0/3248068139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb5b4069880 msgr2=0x7fb5b406bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:29.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.092+0000 7fb5dc000640 1 --2- 172.21.15.42:0/3248068139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb5b4069880 0x7fb5b406bd40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8000f70 tx=0x7fb5c8007e50 comp rx=0 tx=0).stop 2024-05-28T14:11:29.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.092+0000 7fb5dc000640 1 -- 172.21.15.42:0/3248068139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5d40afa70 msgr2=0x7fb5d4157270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:29.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.092+0000 7fb5dc000640 1 --2- 172.21.15.42:0/3248068139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5d40afa70 0x7fb5d4157270 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000c460 tx=0x7fb5c000c920 comp rx=0 tx=0).stop 2024-05-28T14:11:29.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.092+0000 7fb5dc000640 1 -- 172.21.15.42:0/3248068139 shutdown_connections 2024-05-28T14:11:29.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.092+0000 7fb5dc000640 1 --2- 172.21.15.42:0/3248068139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb5b4069880 0x7fb5b406bd40 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.092+0000 7fb5dc000640 1 --2- 172.21.15.42:0/3248068139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5d40d0e00 0x7fb5d41577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.092+0000 7fb5dc000640 1 --2- 172.21.15.42:0/3248068139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5d40afa70 0x7fb5d4157270 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.092+0000 7fb5dc000640 1 --2- 172.21.15.42:0/3248068139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5d406ee80 0x7fb5d414dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.092+0000 7fb5dc000640 1 -- 172.21.15.42:0/3248068139 >> 172.21.15.42:0/3248068139 conn(0x7fb5d406bdf0 msgr2=0x7fb5d406f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:29.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.092+0000 7fb5dc000640 1 -- 172.21.15.42:0/3248068139 shutdown_connections 2024-05-28T14:11:29.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.092+0000 7fb5dc000640 1 -- 172.21.15.42:0/3248068139 wait complete. 2024-05-28T14:11:29.094 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:11:29.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab3e00640 1 Processor -- start 2024-05-28T14:11:29.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab3e00640 1 -- start start 2024-05-28T14:11:29.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab3e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbab406c5f0 0x7fbab40cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:29.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbab406b9f0 0x7fbab40d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:29.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab3e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbab40ce340 con 0x7fbab40d0c80 2024-05-28T14:11:29.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab3e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbab40ce100 con 0x7fbab406c5f0 2024-05-28T14:11:29.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab3e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbab40cffd0 con 0x7fbab406b9f0 2024-05-28T14:11:29.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab2a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbab406c5f0 0x7fbab40cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:29.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab2000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbab406b9f0 0x7fbab40d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:29.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab2a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbab406c5f0 0x7fbab40cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:58622/0 (socket says 172.21.15.42:58622) 2024-05-28T14:11:29.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab2a00640 1 -- 172.21.15.42:0/1367502417 learned_addr learned my addr 172.21.15.42:0/1367502417 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:29.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab2a00640 1 -- 172.21.15.42:0/1367502417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbab406b9f0 msgr2=0x7fbab40d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab2a00640 1 --2- 172.21.15.42:0/1367502417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbab406b9f0 0x7fbab40d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab2a00640 1 -- 172.21.15.42:0/1367502417 >> v1:172.21.15.63:6789/0 conn(0x7fbab40d0c80 legacy=0x7fbab415cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab1600640 1 -- 172.21.15.42:0/1367502417 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2525033579 0 0) 0x7fbab40ce340 con 0x7fbab40d0c80 2024-05-28T14:11:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.256+0000 7fbab2a00640 1 -- 172.21.15.42:0/1367502417 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab40ce620 con 0x7fbab406c5f0 2024-05-28T14:11:29.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.260+0000 7fbab2a00640 1 --2- 172.21.15.42:0/1367502417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbab406c5f0 0x7fbab40cdbc0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009870 tx=0x7fbaa401c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdf239cedb9c72dd server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:29.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.260+0000 7fbab1600640 1 -- 172.21.15.42:0/1367502417 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa40b0070 con 0x7fbab406c5f0 2024-05-28T14:11:29.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.260+0000 7fbab1600640 1 -- 172.21.15.42:0/1367502417 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa40ab600 con 0x7fbab406c5f0 2024-05-28T14:11:29.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.260+0000 7fbab1600640 1 -- 172.21.15.42:0/1367502417 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa40ab9d0 con 0x7fbab406c5f0 2024-05-28T14:11:29.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.260+0000 7fbab3e00640 1 -- 172.21.15.42:0/1367502417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbab406c5f0 msgr2=0x7fbab40cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:29.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.260+0000 7fbab3e00640 1 --2- 172.21.15.42:0/1367502417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbab406c5f0 0x7fbab40cdbc0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009870 tx=0x7fbaa401c440 comp rx=0 tx=0).stop 2024-05-28T14:11:29.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.260+0000 7fbab3e00640 1 -- 172.21.15.42:0/1367502417 shutdown_connections 2024-05-28T14:11:29.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.260+0000 7fbab3e00640 1 --2- 172.21.15.42:0/1367502417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbab406b9f0 0x7fbab40d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.260+0000 7fbab3e00640 1 --2- 172.21.15.42:0/1367502417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbab406c5f0 0x7fbab40cdbc0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.260+0000 7fbab3e00640 1 -- 172.21.15.42:0/1367502417 >> 172.21.15.42:0/1367502417 conn(0x7fbab40b2d70 msgr2=0x7fbab40b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:29.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.260+0000 7fbab3e00640 1 -- 172.21.15.42:0/1367502417 shutdown_connections 2024-05-28T14:11:29.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.260+0000 7fbab3e00640 1 -- 172.21.15.42:0/1367502417 wait complete. 2024-05-28T14:11:29.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab3e00640 1 Processor -- start 2024-05-28T14:11:29.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab3e00640 1 -- start start 2024-05-28T14:11:29.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbab406b9f0 0x7fbab4145640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:29.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbab406c5f0 0x7fbab4153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:29.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbab4153850 0x7fbab40b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:29.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab3e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbab41691c0 con 0x7fbab4153850 2024-05-28T14:11:29.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab3e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbab40cd720 con 0x7fbab406b9f0 2024-05-28T14:11:29.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab3e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbab4169340 con 0x7fbab406c5f0 2024-05-28T14:11:29.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbab4153850 0x7fbab40b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:29.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbab406b9f0 0x7fbab4145640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:29.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab2000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbab406c5f0 0x7fbab4153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:29.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbab4153850 0x7fbab40b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45250/0 (socket says 172.21.15.42:45250) 2024-05-28T14:11:29.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab3400640 1 -- 172.21.15.42:0/1763722253 learned_addr learned my addr 172.21.15.42:0/1763722253 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:29.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab2a00640 1 -- 172.21.15.42:0/1763722253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbab406c5f0 msgr2=0x7fbab4153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:29.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab2a00640 1 --2- 172.21.15.42:0/1763722253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbab406c5f0 0x7fbab4153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab2a00640 1 -- 172.21.15.42:0/1763722253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbab4153850 msgr2=0x7fbab40b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:29.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab2a00640 1 --2- 172.21.15.42:0/1763722253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbab4153850 0x7fbab40b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab2a00640 1 -- 172.21.15.42:0/1763722253 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab41568a0 con 0x7fbab406b9f0 2024-05-28T14:11:29.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab2a00640 1 --2- 172.21.15.42:0/1763722253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbab406b9f0 0x7fbab4145640 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40ab000 tx=0x7fbaa401ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:29.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbaaaa00640 1 -- 172.21.15.42:0/1763722253 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa40b0040 con 0x7fbab406b9f0 2024-05-28T14:11:29.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbaaaa00640 1 -- 172.21.15.42:0/1763722253 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa4004030 con 0x7fbab406b9f0 2024-05-28T14:11:29.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbaaaa00640 1 -- 172.21.15.42:0/1763722253 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4004440 con 0x7fbab406b9f0 2024-05-28T14:11:29.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab3e00640 1 -- 172.21.15.42:0/1763722253 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab4156000 con 0x7fbab406b9f0 2024-05-28T14:11:29.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab3e00640 1 -- 172.21.15.42:0/1763722253 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab41565c0 con 0x7fbab406b9f0 2024-05-28T14:11:29.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.264+0000 7fbab3e00640 1 -- 172.21.15.42:0/1763722253 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab404fcf0 con 0x7fbab406b9f0 2024-05-28T14:11:29.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.268+0000 7fbaaaa00640 1 -- 172.21.15.42:0/1763722253 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7fbaa40b3070 con 0x7fbab406b9f0 2024-05-28T14:11:29.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.268+0000 7fbaaaa00640 1 --2- 172.21.15.42:0/1763722253 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba94069850 0x7fba9406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:29.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.268+0000 7fbaaaa00640 1 -- 172.21.15.42:0/1763722253 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fbaa4120090 con 0x7fbab406b9f0 2024-05-28T14:11:29.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.268+0000 7fbab2000640 1 --2- 172.21.15.42:0/1763722253 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba94069850 0x7fba9406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:29.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.268+0000 7fbab2000640 1 --2- 172.21.15.42:0/1763722253 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba94069850 0x7fba9406bd10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbab40b5de0 tx=0x7fbaa0009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:29.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.272+0000 7fbaaaa00640 1 -- 172.21.15.42:0/1763722253 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbab41568a0 con 0x7fbab406b9f0 2024-05-28T14:11:29.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.468+0000 7fbab3e00640 1 -- 172.21.15.42:0/1763722253 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fbab40ae050 con 0x7fba94069850 2024-05-28T14:11:29.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.472+0000 7fbaaaa00640 1 -- 172.21.15.42:0/1763722253 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fbab40ae050 con 0x7fba94069850 2024-05-28T14:11:29.457 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227 2024-05-28T14:11:29.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.476+0000 7fbab3e00640 1 -- 172.21.15.42:0/1763722253 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba94069850 msgr2=0x7fba9406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:29.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.476+0000 7fbab3e00640 1 --2- 172.21.15.42:0/1763722253 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba94069850 0x7fba9406bd10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbab40b5de0 tx=0x7fbaa0009190 comp rx=0 tx=0).stop 2024-05-28T14:11:29.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.476+0000 7fbab3e00640 1 -- 172.21.15.42:0/1763722253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbab406b9f0 msgr2=0x7fbab4145640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:29.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.476+0000 7fbab3e00640 1 --2- 172.21.15.42:0/1763722253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbab406b9f0 0x7fbab4145640 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40ab000 tx=0x7fbaa401ac80 comp rx=0 tx=0).stop 2024-05-28T14:11:29.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.476+0000 7fbab3e00640 1 -- 172.21.15.42:0/1763722253 shutdown_connections 2024-05-28T14:11:29.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.476+0000 7fbab3e00640 1 --2- 172.21.15.42:0/1763722253 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fba94069850 0x7fba9406bd10 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.476+0000 7fbab3e00640 1 --2- 172.21.15.42:0/1763722253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbab4153850 0x7fbab40b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.476+0000 7fbab3e00640 1 --2- 172.21.15.42:0/1763722253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbab406c5f0 0x7fbab4153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.476+0000 7fbab3e00640 1 --2- 172.21.15.42:0/1763722253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbab406b9f0 0x7fbab4145640 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.476+0000 7fbab3e00640 1 -- 172.21.15.42:0/1763722253 >> 172.21.15.42:0/1763722253 conn(0x7fbab40b2d70 msgr2=0x7fbab40aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:29.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.476+0000 7fbab3e00640 1 -- 172.21.15.42:0/1763722253 shutdown_connections 2024-05-28T14:11:29.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.476+0000 7fbab3e00640 1 -- 172.21.15.42:0/1763722253 wait complete. 2024-05-28T14:11:29.479 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:11:29.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e89400640 1 Processor -- start 2024-05-28T14:11:29.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e89400640 1 -- start start 2024-05-28T14:11:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e89400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0e84160e60 0x7f0e840cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e89400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0e840d0ac0 0x7f0e8409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e89400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e840cd170 con 0x7f0e841602c0 2024-05-28T14:11:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e89400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0e840ce400 con 0x7f0e84160e60 2024-05-28T14:11:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e89400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0e840cd3e0 con 0x7f0e840d0ac0 2024-05-28T14:11:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e81e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0e84160e60 0x7f0e840cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e81e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0e84160e60 0x7f0e840cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45256/0 (socket says 172.21.15.42:45256) 2024-05-28T14:11:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e81e00640 1 -- 172.21.15.42:0/3945179362 learned_addr learned my addr 172.21.15.42:0/3945179362 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:29.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e83200640 1 --2- 172.21.15.42:0/3945179362 >> v2:172.21.15.63:3300/0 conn(0x7f0e840d0ac0 0x7f0e8409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:29.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e81e00640 1 -- 172.21.15.42:0/3945179362 >> v2:172.21.15.63:3300/0 conn(0x7f0e840d0ac0 msgr2=0x7f0e8409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:29.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e81e00640 1 --2- 172.21.15.42:0/3945179362 >> v2:172.21.15.63:3300/0 conn(0x7f0e840d0ac0 0x7f0e8409aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e81e00640 1 -- 172.21.15.42:0/3945179362 >> v1:172.21.15.63:6789/0 conn(0x7f0e841602c0 legacy=0x7f0e841606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:29.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.640+0000 7f0e81e00640 1 -- 172.21.15.42:0/3945179362 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e840cd6c0 con 0x7f0e84160e60 2024-05-28T14:11:29.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.644+0000 7f0e81e00640 1 --2- 172.21.15.42:0/3945179362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e84160e60 0x7f0e840cdec0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f0e840733c0 tx=0x7f0e7001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126ddcd00bc098ae server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:29.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.644+0000 7f0e81400640 1 -- 172.21.15.42:0/3945179362 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e700aa5f0 con 0x7f0e84160e60 2024-05-28T14:11:29.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.644+0000 7f0e81400640 1 -- 172.21.15.42:0/3945179362 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e700af070 con 0x7f0e84160e60 2024-05-28T14:11:29.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.644+0000 7f0e81400640 1 -- 172.21.15.42:0/3945179362 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e700aa9b0 con 0x7f0e84160e60 2024-05-28T14:11:29.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.644+0000 7f0e89400640 1 -- 172.21.15.42:0/3945179362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e84160e60 msgr2=0x7f0e840cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:29.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.644+0000 7f0e89400640 1 --2- 172.21.15.42:0/3945179362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e84160e60 0x7f0e840cdec0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f0e840733c0 tx=0x7f0e7001c4f0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.644+0000 7f0e89400640 1 -- 172.21.15.42:0/3945179362 shutdown_connections 2024-05-28T14:11:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.644+0000 7f0e89400640 1 --2- 172.21.15.42:0/3945179362 >> v2:172.21.15.63:3300/0 conn(0x7f0e840d0ac0 0x7f0e8409aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.644+0000 7f0e89400640 1 --2- 172.21.15.42:0/3945179362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e84160e60 0x7f0e840cdec0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.644+0000 7f0e89400640 1 -- 172.21.15.42:0/3945179362 >> 172.21.15.42:0/3945179362 conn(0x7f0e840b3fd0 msgr2=0x7f0e840b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.644+0000 7f0e89400640 1 -- 172.21.15.42:0/3945179362 shutdown_connections 2024-05-28T14:11:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.644+0000 7f0e89400640 1 -- 172.21.15.42:0/3945179362 wait complete. 2024-05-28T14:11:29.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.644+0000 7f0e89400640 1 Processor -- start 2024-05-28T14:11:29.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.644+0000 7f0e89400640 1 -- start start 2024-05-28T14:11:29.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e89400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e840d0ac0 0x7f0e840af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:29.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e89400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e841602c0 0x7f0e841496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:29.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e89400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e84160e60 0x7f0e8415b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e89400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8415dba0 con 0x7f0e840d0ac0 2024-05-28T14:11:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e89400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8415da20 con 0x7f0e84160e60 2024-05-28T14:11:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e89400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e84169640 con 0x7f0e841602c0 2024-05-28T14:11:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e82800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e840d0ac0 0x7f0e840af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e82800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e840d0ac0 0x7f0e840af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45272/0 (socket says 172.21.15.42:45272) 2024-05-28T14:11:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e82800640 1 -- 172.21.15.42:0/1411273322 learned_addr learned my addr 172.21.15.42:0/1411273322 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e81e00640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e841602c0 0x7f0e841496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e83200640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e84160e60 0x7f0e8415b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e82800640 1 -- 172.21.15.42:0/1411273322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e841602c0 msgr2=0x7f0e841496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e82800640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e841602c0 0x7f0e841496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e82800640 1 -- 172.21.15.42:0/1411273322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e84160e60 msgr2=0x7f0e8415b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e82800640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e84160e60 0x7f0e8415b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e82800640 1 -- 172.21.15.42:0/1411273322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e841fc010 con 0x7f0e840d0ac0 2024-05-28T14:11:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e83200640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e84160e60 0x7f0e8415b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:11:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e82800640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e840d0ac0 0x7f0e840af1f0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400c420 tx=0x7f0e7400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e7aa00640 1 -- 172.21.15.42:0/1411273322 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e740052d0 con 0x7f0e840d0ac0 2024-05-28T14:11:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e7aa00640 1 -- 172.21.15.42:0/1411273322 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e74005470 con 0x7f0e840d0ac0 2024-05-28T14:11:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e7aa00640 1 -- 172.21.15.42:0/1411273322 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74007260 con 0x7f0e840d0ac0 2024-05-28T14:11:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e89400640 1 -- 172.21.15.42:0/1411273322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8415e650 con 0x7f0e840d0ac0 2024-05-28T14:11:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e89400640 1 -- 172.21.15.42:0/1411273322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8415ec10 con 0x7f0e840d0ac0 2024-05-28T14:11:29.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.648+0000 7f0e89400640 1 -- 172.21.15.42:0/1411273322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e840a0570 con 0x7f0e840d0ac0 2024-05-28T14:11:29.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.652+0000 7f0e7aa00640 1 -- 172.21.15.42:0/1411273322 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 35) ==== 89874+0+0 (secure 0 0 0) 0x7f0e74007400 con 0x7f0e840d0ac0 2024-05-28T14:11:29.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.652+0000 7f0e7aa00640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e64069850 0x7f0e6406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:29.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.652+0000 7f0e7aa00640 1 -- 172.21.15.42:0/1411273322 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f0e74084ec0 con 0x7f0e840d0ac0 2024-05-28T14:11:29.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.652+0000 7f0e81e00640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e64069850 0x7f0e6406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:29.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.652+0000 7f0e81e00640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e64069850 0x7f0e6406bd10 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0e700ae000 tx=0x7f0e700a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:29.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.656+0000 7f0e7aa00640 1 -- 172.21.15.42:0/1411273322 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0e74056500 con 0x7f0e840d0ac0 2024-05-28T14:11:29.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.844+0000 7f0e89400640 1 -- 172.21.15.42:0/1411273322 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f0e840b1d20 con 0x7f0e64069850 2024-05-28T14:11:29.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.848+0000 7f0e7aa00640 1 -- 172.21.15.42:0/1411273322 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f0e840b1d20 con 0x7f0e64069850 2024-05-28T14:11:29.834 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227 2024-05-28T14:11:29.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.852+0000 7f0e89400640 1 -- 172.21.15.42:0/1411273322 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e64069850 msgr2=0x7f0e6406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:29.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.852+0000 7f0e89400640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e64069850 0x7f0e6406bd10 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0e700ae000 tx=0x7f0e700a7000 comp rx=0 tx=0).stop 2024-05-28T14:11:29.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.852+0000 7f0e89400640 1 -- 172.21.15.42:0/1411273322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e840d0ac0 msgr2=0x7f0e840af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:29.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.852+0000 7f0e89400640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e840d0ac0 0x7f0e840af1f0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400c420 tx=0x7f0e7400c8e0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.852+0000 7f0e89400640 1 -- 172.21.15.42:0/1411273322 shutdown_connections 2024-05-28T14:11:29.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.852+0000 7f0e89400640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0e64069850 0x7f0e6406bd10 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.852+0000 7f0e89400640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e84160e60 0x7f0e8415b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.852+0000 7f0e89400640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e841602c0 0x7f0e841496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.852+0000 7f0e89400640 1 --2- 172.21.15.42:0/1411273322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e840d0ac0 0x7f0e840af1f0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:29.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.852+0000 7f0e89400640 1 -- 172.21.15.42:0/1411273322 >> 172.21.15.42:0/1411273322 conn(0x7f0e840b3fd0 msgr2=0x7f0e8406ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:29.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.852+0000 7f0e89400640 1 -- 172.21.15.42:0/1411273322 shutdown_connections 2024-05-28T14:11:29.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:29.852+0000 7f0e89400640 1 -- 172.21.15.42:0/1411273322 wait complete. 2024-05-28T14:11:29.857 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227 2024-05-28T14:11:29.857 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:11:29.857 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:11:29.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:29.867 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:11:36.980 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:11:36.981 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:11:36.981 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.04332 s, 152 MB/s 2024-05-28T14:11:36.982 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:11:36.982 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:11:36.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:36.991 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:11:43.808 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:11:43.808 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:11:43.808 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.75902 s, 159 MB/s 2024-05-28T14:11:43.810 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:11:43.810 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:11:43.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:43.819 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:11:50.641 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:11:50.645 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:11:50.650 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:11:50.650 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:11:50.650 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.77517 s, 158 MB/s 2024-05-28T14:11:50.654 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-28T14:11:50.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5b0e00640 1 Processor -- start 2024-05-28T14:11:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5b0e00640 1 -- start start 2024-05-28T14:11:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5b0e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ac0d0e00 0x7fa5ac06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5b0e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa5ac072d40 0x7fa5ac0aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5b0e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5ac0ade40 con 0x7fa5ac15b350 2024-05-28T14:11:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5b0e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa5ac0ae9f0 con 0x7fa5ac072d40 2024-05-28T14:11:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5b0e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa5ac0ae0c0 con 0x7fa5ac0d0e00 2024-05-28T14:11:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5a9e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa5ac15b350 0x7fa5ac0d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37470/0 (socket says 172.21.15.42:37470) 2024-05-28T14:11:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5a9e00640 1 -- 172.21.15.42:0/43394581 learned_addr learned my addr 172.21.15.42:0/43394581 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5aa800640 1 --2- 172.21.15.42:0/43394581 >> v2:172.21.15.63:3300/0 conn(0x7fa5ac072d40 0x7fa5ac0aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5a9400640 1 --2- 172.21.15.42:0/43394581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ac0d0e00 0x7fa5ac06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5a8a00640 1 -- 172.21.15.42:0/43394581 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1735448658 0 0) 0x7fa59c0014a0 con 0x7fa5ac15b350 2024-05-28T14:11:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5a8a00640 1 -- 172.21.15.42:0/43394581 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa590005740 con 0x7fa5ac15b350 2024-05-28T14:11:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5aa800640 1 -- 172.21.15.42:0/43394581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ac0d0e00 msgr2=0x7fa5ac06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5aa800640 1 --2- 172.21.15.42:0/43394581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ac0d0e00 0x7fa5ac06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5aa800640 1 -- 172.21.15.42:0/43394581 >> v1:172.21.15.42:6789/0 conn(0x7fa5ac15b350 legacy=0x7fa5ac0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.892+0000 7fa5aa800640 1 -- 172.21.15.42:0/43394581 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac0ae3a0 con 0x7fa5ac072d40 2024-05-28T14:11:50.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.896+0000 7fa5aa800640 1 --2- 172.21.15.42:0/43394581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5ac072d40 0x7fa5ac0aee80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40097c0 tx=0x7fa5a401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=299007dd37a90ad0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:50.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.896+0000 7fa5a8a00640 1 -- 172.21.15.42:0/43394581 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40b0070 con 0x7fa5ac072d40 2024-05-28T14:11:50.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.896+0000 7fa5a8a00640 1 -- 172.21.15.42:0/43394581 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a40ab600 con 0x7fa5ac072d40 2024-05-28T14:11:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.896+0000 7fa5a8a00640 1 -- 172.21.15.42:0/43394581 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40ab9b0 con 0x7fa5ac072d40 2024-05-28T14:11:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.896+0000 7fa5b0e00640 1 -- 172.21.15.42:0/43394581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5ac072d40 msgr2=0x7fa5ac0aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.896+0000 7fa5b0e00640 1 --2- 172.21.15.42:0/43394581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5ac072d40 0x7fa5ac0aee80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40097c0 tx=0x7fa5a401c410 comp rx=0 tx=0).stop 2024-05-28T14:11:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.896+0000 7fa5b0e00640 1 -- 172.21.15.42:0/43394581 shutdown_connections 2024-05-28T14:11:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.896+0000 7fa5b0e00640 1 --2- 172.21.15.42:0/43394581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5ac072d40 0x7fa5ac0aee80 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.896+0000 7fa5b0e00640 1 --2- 172.21.15.42:0/43394581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ac0d0e00 0x7fa5ac06dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.896+0000 7fa5b0e00640 1 -- 172.21.15.42:0/43394581 >> 172.21.15.42:0/43394581 conn(0x7fa5ac06bdf0 msgr2=0x7fa5ac06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.896+0000 7fa5b0e00640 1 -- 172.21.15.42:0/43394581 shutdown_connections 2024-05-28T14:11:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.896+0000 7fa5b0e00640 1 -- 172.21.15.42:0/43394581 wait complete. 2024-05-28T14:11:50.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5b0e00640 1 Processor -- start 2024-05-28T14:11:50.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5b0e00640 1 -- start start 2024-05-28T14:11:50.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5b0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5ac072d40 0x7fa5ac15c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:50.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5b0e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ac0d0e00 0x7fa5ac15e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:50.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5b0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5ac15b350 0x7fa5ac153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:50.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5b0e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac0b3460 con 0x7fa5ac072d40 2024-05-28T14:11:50.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5b0e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac0b32e0 con 0x7fa5ac15b350 2024-05-28T14:11:50.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5b0e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa5ac0b35e0 con 0x7fa5ac0d0e00 2024-05-28T14:11:50.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5a9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5ac072d40 0x7fa5ac15c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:50.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5a9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5ac072d40 0x7fa5ac15c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44488/0 (socket says 172.21.15.42:44488) 2024-05-28T14:11:50.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5a9e00640 1 -- 172.21.15.42:0/3441166765 learned_addr learned my addr 172.21.15.42:0/3441166765 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:50.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5a9e00640 1 -- 172.21.15.42:0/3441166765 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ac0d0e00 msgr2=0x7fa5ac15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:50.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5a9e00640 1 --2- 172.21.15.42:0/3441166765 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ac0d0e00 0x7fa5ac15e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:50.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5a9e00640 1 -- 172.21.15.42:0/3441166765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5ac15b350 msgr2=0x7fa5ac153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:50.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5a9e00640 1 --2- 172.21.15.42:0/3441166765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5ac15b350 0x7fa5ac153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:50.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5a9e00640 1 -- 172.21.15.42:0/3441166765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac0d0bb0 con 0x7fa5ac072d40 2024-05-28T14:11:50.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5a9e00640 1 --2- 172.21.15.42:0/3441166765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5ac072d40 0x7fa5ac15c0d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00cb10 tx=0x7fa59c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:50.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5a2000640 1 -- 172.21.15.42:0/3441166765 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c017070 con 0x7fa5ac072d40 2024-05-28T14:11:50.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5a2000640 1 -- 172.21.15.42:0/3441166765 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59c00ac10 con 0x7fa5ac072d40 2024-05-28T14:11:50.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5b0e00640 1 -- 172.21.15.42:0/3441166765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ac0d0950 con 0x7fa5ac072d40 2024-05-28T14:11:50.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5a2000640 1 -- 172.21.15.42:0/3441166765 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c012400 con 0x7fa5ac072d40 2024-05-28T14:11:50.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5b0e00640 1 -- 172.21.15.42:0/3441166765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ac14a750 con 0x7fa5ac072d40 2024-05-28T14:11:50.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.900+0000 7fa5b0e00640 1 -- 172.21.15.42:0/3441166765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5ac068a90 con 0x7fa5ac072d40 2024-05-28T14:11:50.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.904+0000 7fa5a2000640 1 -- 172.21.15.42:0/3441166765 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 36) ==== 89925+0+0 (secure 0 0 0) 0x7fa59c01a070 con 0x7fa5ac072d40 2024-05-28T14:11:50.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.904+0000 7fa5a2000640 1 --2- 172.21.15.42:0/3441166765 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa58c0698e0 0x7fa58c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:50.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.904+0000 7fa5a2000640 1 -- 172.21.15.42:0/3441166765 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fa59c0858e0 con 0x7fa5ac072d40 2024-05-28T14:11:50.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.904+0000 7fa5a9400640 1 --2- 172.21.15.42:0/3441166765 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa58c0698e0 0x7fa58c06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:50.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.904+0000 7fa5a9400640 1 --2- 172.21.15.42:0/3441166765 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa58c0698e0 0x7fa58c06bda0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa598006aa0 tx=0x7fa598006a30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:50.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:50.908+0000 7fa5a2000640 1 -- 172.21.15.42:0/3441166765 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa5ac0ade40 con 0x7fa5ac072d40 2024-05-28T14:11:51.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.104+0000 7fa5b0e00640 1 -- 172.21.15.42:0/3441166765 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fa5ac072b20 con 0x7fa58c0698e0 2024-05-28T14:11:51.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.116+0000 7fa5a2000640 1 -- 172.21.15.42:0/3441166765 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa5ac072b20 con 0x7fa58c0698e0 2024-05-28T14:11:51.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.120+0000 7fa5b0e00640 1 -- 172.21.15.42:0/3441166765 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa58c0698e0 msgr2=0x7fa58c06bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:51.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.120+0000 7fa5b0e00640 1 --2- 172.21.15.42:0/3441166765 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa58c0698e0 0x7fa58c06bda0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa598006aa0 tx=0x7fa598006a30 comp rx=0 tx=0).stop 2024-05-28T14:11:51.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.120+0000 7fa5b0e00640 1 -- 172.21.15.42:0/3441166765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5ac072d40 msgr2=0x7fa5ac15c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:51.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.120+0000 7fa5b0e00640 1 --2- 172.21.15.42:0/3441166765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5ac072d40 0x7fa5ac15c0d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00cb10 tx=0x7fa59c00a060 comp rx=0 tx=0).stop 2024-05-28T14:11:51.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.120+0000 7fa5b0e00640 1 -- 172.21.15.42:0/3441166765 shutdown_connections 2024-05-28T14:11:51.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.120+0000 7fa5b0e00640 1 --2- 172.21.15.42:0/3441166765 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa58c0698e0 0x7fa58c06bda0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.120+0000 7fa5b0e00640 1 --2- 172.21.15.42:0/3441166765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5ac15b350 0x7fa5ac153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.120+0000 7fa5b0e00640 1 --2- 172.21.15.42:0/3441166765 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ac0d0e00 0x7fa5ac15e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.120+0000 7fa5b0e00640 1 --2- 172.21.15.42:0/3441166765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5ac072d40 0x7fa5ac15c0d0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.120+0000 7fa5b0e00640 1 -- 172.21.15.42:0/3441166765 >> 172.21.15.42:0/3441166765 conn(0x7fa5ac06bdf0 msgr2=0x7fa5ac0cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:51.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.120+0000 7fa5b0e00640 1 -- 172.21.15.42:0/3441166765 shutdown_connections 2024-05-28T14:11:51.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.120+0000 7fa5b0e00640 1 -- 172.21.15.42:0/3441166765 wait complete. 2024-05-28T14:11:51.122 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:11:51.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.284+0000 7f4bb2200640 1 Processor -- start 2024-05-28T14:11:51.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.284+0000 7f4bb2200640 1 -- start start 2024-05-28T14:11:51.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.288+0000 7f4bb2200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4bac0b02e0 0x7f4bac06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.288+0000 7f4bb2200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4bac15f710 0x7f4bac06ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.288+0000 7f4bb2200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bac15fb60 con 0x7f4bac06cf40 2024-05-28T14:11:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.288+0000 7f4bb2200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f4bac06d350 con 0x7f4bac0b02e0 2024-05-28T14:11:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.288+0000 7f4bb2200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4bac06f190 con 0x7f4bac15f710 2024-05-28T14:11:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.288+0000 7f4bab400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4bac0b02e0 0x7f4bac06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.288+0000 7f4babe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f4bac06cf40 0x7f4bac154da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37486/0 (socket says 172.21.15.42:37486) 2024-05-28T14:11:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.288+0000 7f4babe00640 1 -- 172.21.15.42:0/2348991961 learned_addr learned my addr 172.21.15.42:0/2348991961 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.288+0000 7f4ba2a00640 1 --2- 172.21.15.42:0/2348991961 >> v2:172.21.15.63:3300/0 conn(0x7f4bac15f710 0x7f4bac06ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.288+0000 7f4bab400640 1 -- 172.21.15.42:0/2348991961 >> v2:172.21.15.63:3300/0 conn(0x7f4bac15f710 msgr2=0x7f4bac06ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.288+0000 7f4bab400640 1 --2- 172.21.15.42:0/2348991961 >> v2:172.21.15.63:3300/0 conn(0x7f4bac15f710 0x7f4bac06ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.288+0000 7f4bab400640 1 -- 172.21.15.42:0/2348991961 >> v1:172.21.15.42:6789/0 conn(0x7f4bac06cf40 legacy=0x7f4bac154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.288+0000 7f4bab400640 1 -- 172.21.15.42:0/2348991961 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac06f470 con 0x7f4bac0b02e0 2024-05-28T14:11:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.288+0000 7f4baaa00640 1 -- 172.21.15.42:0/2348991961 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1594181839 0 0) 0x7f4ba4001120 con 0x7f4bac06cf40 2024-05-28T14:11:51.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bab400640 1 --2- 172.21.15.42:0/2348991961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bac0b02e0 0x7f4bac06eb00 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f4bac072590 tx=0x7f4b9c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9407b276bceee4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:51.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4baaa00640 1 -- 172.21.15.42:0/2348991961 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c0b2070 con 0x7f4bac0b02e0 2024-05-28T14:11:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4baaa00640 1 -- 172.21.15.42:0/2348991961 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b9c0a5960 con 0x7f4bac0b02e0 2024-05-28T14:11:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4baaa00640 1 -- 172.21.15.42:0/2348991961 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c0a5d30 con 0x7f4bac0b02e0 2024-05-28T14:11:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 -- 172.21.15.42:0/2348991961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bac0b02e0 msgr2=0x7f4bac06eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 --2- 172.21.15.42:0/2348991961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bac0b02e0 0x7f4bac06eb00 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f4bac072590 tx=0x7f4b9c01c8a0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 -- 172.21.15.42:0/2348991961 shutdown_connections 2024-05-28T14:11:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 --2- 172.21.15.42:0/2348991961 >> v2:172.21.15.63:3300/0 conn(0x7f4bac15f710 0x7f4bac06ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 --2- 172.21.15.42:0/2348991961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bac0b02e0 0x7f4bac06eb00 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 -- 172.21.15.42:0/2348991961 >> 172.21.15.42:0/2348991961 conn(0x7f4bac072600 msgr2=0x7f4bac072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 -- 172.21.15.42:0/2348991961 shutdown_connections 2024-05-28T14:11:51.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 -- 172.21.15.42:0/2348991961 wait complete. 2024-05-28T14:11:51.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 Processor -- start 2024-05-28T14:11:51.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 -- start start 2024-05-28T14:11:51.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4bac06cf40 0x7f4bac0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:51.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bac0b02e0 0x7f4bac14d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:51.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bac15f710 0x7f4bac1501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:51.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac0b4020 con 0x7f4bac0b02e0 2024-05-28T14:11:51.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac0b3ea0 con 0x7f4bac15f710 2024-05-28T14:11:51.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4bb2200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4bac0b41a0 con 0x7f4bac06cf40 2024-05-28T14:11:51.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.292+0000 7f4ba2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bac0b02e0 0x7f4bac14d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:51.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4ba2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bac0b02e0 0x7f4bac14d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44514/0 (socket says 172.21.15.42:44514) 2024-05-28T14:11:51.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4ba2a00640 1 -- 172.21.15.42:0/4228169184 learned_addr learned my addr 172.21.15.42:0/4228169184 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:51.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4babe00640 1 --2- 172.21.15.42:0/4228169184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bac15f710 0x7f4bac1501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:51.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4bab400640 1 --2- 172.21.15.42:0/4228169184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4bac06cf40 0x7f4bac0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:51.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4ba2a00640 1 -- 172.21.15.42:0/4228169184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4bac06cf40 msgr2=0x7f4bac0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:51.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4ba2a00640 1 --2- 172.21.15.42:0/4228169184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4bac06cf40 0x7f4bac0cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4ba2a00640 1 -- 172.21.15.42:0/4228169184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bac15f710 msgr2=0x7f4bac1501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:51.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4ba2a00640 1 --2- 172.21.15.42:0/4228169184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bac15f710 0x7f4bac1501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4ba2a00640 1 -- 172.21.15.42:0/4228169184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac0d03b0 con 0x7f4bac0b02e0 2024-05-28T14:11:51.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4ba2a00640 1 --2- 172.21.15.42:0/4228169184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bac0b02e0 0x7f4bac14d790 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400d5f0 tx=0x7f4b9400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:51.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4ba3e00640 1 -- 172.21.15.42:0/4228169184 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b940125f0 con 0x7f4bac0b02e0 2024-05-28T14:11:51.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4ba3e00640 1 -- 172.21.15.42:0/4228169184 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b94019070 con 0x7f4bac0b02e0 2024-05-28T14:11:51.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4ba3e00640 1 -- 172.21.15.42:0/4228169184 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b940129b0 con 0x7f4bac0b02e0 2024-05-28T14:11:51.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4bb2200640 1 -- 172.21.15.42:0/4228169184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bac0d00f0 con 0x7f4bac0b02e0 2024-05-28T14:11:51.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4bb2200640 1 -- 172.21.15.42:0/4228169184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bac1571d0 con 0x7f4bac0b02e0 2024-05-28T14:11:51.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4bb2200640 1 -- 172.21.15.42:0/4228169184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bac073f40 con 0x7f4bac0b02e0 2024-05-28T14:11:51.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4ba3e00640 1 -- 172.21.15.42:0/4228169184 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 36) ==== 89925+0+0 (secure 0 0 0) 0x7f4b94012b50 con 0x7f4bac0b02e0 2024-05-28T14:11:51.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.296+0000 7f4ba3e00640 1 --2- 172.21.15.42:0/4228169184 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b880698e0 0x7f4b8806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:51.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.300+0000 7f4ba3e00640 1 -- 172.21.15.42:0/4228169184 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f4b94085da0 con 0x7f4bac0b02e0 2024-05-28T14:11:51.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.300+0000 7f4bab400640 1 --2- 172.21.15.42:0/4228169184 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b880698e0 0x7f4b8806bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:51.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.300+0000 7f4bab400640 1 --2- 172.21.15.42:0/4228169184 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b880698e0 0x7f4b8806bda0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0af000 tx=0x7f4b9c0a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:51.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.304+0000 7f4ba3e00640 1 -- 172.21.15.42:0/4228169184 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4b94057360 con 0x7f4bac0b02e0 2024-05-28T14:11:51.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.492+0000 7f4bb2200640 1 -- 172.21.15.42:0/4228169184 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f4bac0d0900 con 0x7f4b880698e0 2024-05-28T14:11:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.500+0000 7f4ba3e00640 1 -- 172.21.15.42:0/4228169184 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f4bac0d0900 con 0x7f4b880698e0 2024-05-28T14:11:51.483 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227 2024-05-28T14:11:51.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.500+0000 7f4bb2200640 1 -- 172.21.15.42:0/4228169184 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b880698e0 msgr2=0x7f4b8806bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:51.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.500+0000 7f4bb2200640 1 --2- 172.21.15.42:0/4228169184 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b880698e0 0x7f4b8806bda0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0af000 tx=0x7f4b9c0a7070 comp rx=0 tx=0).stop 2024-05-28T14:11:51.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.500+0000 7f4bb2200640 1 -- 172.21.15.42:0/4228169184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bac0b02e0 msgr2=0x7f4bac14d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:51.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.500+0000 7f4bb2200640 1 --2- 172.21.15.42:0/4228169184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bac0b02e0 0x7f4bac14d790 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400d5f0 tx=0x7f4b9400dab0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.500+0000 7f4bb2200640 1 -- 172.21.15.42:0/4228169184 shutdown_connections 2024-05-28T14:11:51.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.500+0000 7f4bb2200640 1 --2- 172.21.15.42:0/4228169184 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4b880698e0 0x7f4b8806bda0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.500+0000 7f4bb2200640 1 --2- 172.21.15.42:0/4228169184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bac15f710 0x7f4bac1501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.500+0000 7f4bb2200640 1 --2- 172.21.15.42:0/4228169184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bac0b02e0 0x7f4bac14d790 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.500+0000 7f4bb2200640 1 --2- 172.21.15.42:0/4228169184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4bac06cf40 0x7f4bac0cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:51.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.500+0000 7f4bb2200640 1 -- 172.21.15.42:0/4228169184 >> 172.21.15.42:0/4228169184 conn(0x7f4bac072600 msgr2=0x7f4bac162900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:51.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.500+0000 7f4bb2200640 1 -- 172.21.15.42:0/4228169184 shutdown_connections 2024-05-28T14:11:51.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:51.500+0000 7f4bb2200640 1 -- 172.21.15.42:0/4228169184 wait complete. 2024-05-28T14:11:51.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:51.504 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227 2024-05-28T14:11:51.504 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:11:51.518 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227 2024-05-28T14:11:51.519 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3138256896" 2024-05-28T14:11:51.519 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:51.520 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-28T14:11:53.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:53.521 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227 2024-05-28T14:11:53.522 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:11:53.537 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227 2024-05-28T14:11:53.538 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3138256896" 2024-05-28T14:11:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:53.539 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-28T14:11:55.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:55.540 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227 2024-05-28T14:11:55.540 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:11:55.556 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227 2024-05-28T14:11:55.556 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3138256896" 2024-05-28T14:11:55.556 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:55.557 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-28T14:11:57.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:11:57.558 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227 2024-05-28T14:11:57.558 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:11:57.575 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/9d1bb6f5-8933-4444-9d42-272b2b6c2227 2024-05-28T14:11:57.575 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3221225472" 2024-05-28T14:11:57.576 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:11:57.576 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-28T14:11:57.576 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-05-28T14:11:57.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.796+0000 7fa171000640 1 Processor -- start 2024-05-28T14:11:57.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.796+0000 7fa171000640 1 -- start start 2024-05-28T14:11:57.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.796+0000 7fa171000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa16c0b4c10 0x7fa16c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:57.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.796+0000 7fa171000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa16c0b3690 con 0x7fa16c148090 2024-05-28T14:11:57.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.796+0000 7fa171000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa16c0b3970 con 0x7fa16c072320 2024-05-28T14:11:57.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.796+0000 7fa171000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa16c148470 con 0x7fa16c0b4c10 2024-05-28T14:11:57.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.796+0000 7fa169e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa16c148090 0x7fa16c0b5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:33174/0 (socket says 172.21.15.42:33174) 2024-05-28T14:11:57.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.796+0000 7fa169e00640 1 -- 172.21.15.42:0/2789112023 learned_addr learned my addr 172.21.15.42:0/2789112023 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:11:57.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 -- 172.21.15.42:0/2789112023 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2672284613 0 0) 0x7fa16c0b3690 con 0x7fa16c148090 2024-05-28T14:11:57.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa163e00640 1 --2- 172.21.15.42:0/2789112023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa16c0b4c10 0x7fa16c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:57.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 -- 172.21.15.42:0/2789112023 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa150003530 con 0x7fa16c148090 2024-05-28T14:11:57.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 -- 172.21.15.42:0/2789112023 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3162258375 0 0) 0x7fa150003530 con 0x7fa16c148090 2024-05-28T14:11:57.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 -- 172.21.15.42:0/2789112023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa16c0b4c10 msgr2=0x7fa16c071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:57.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 --2- 172.21.15.42:0/2789112023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa16c0b4c10 0x7fa16c071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:57.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 -- 172.21.15.42:0/2789112023 >> v1:172.21.15.63:6789/0 conn(0x7fa16c072320 legacy=0x7fa16c0aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:57.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 -- 172.21.15.42:0/2789112023 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16c0b3c50 con 0x7fa16c148090 2024-05-28T14:11:57.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 -- 172.21.15.42:0/2789112023 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fa15c001680 con 0x7fa16c148090 2024-05-28T14:11:57.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 -- 172.21.15.42:0/2789112023 >> v1:172.21.15.42:6789/0 conn(0x7fa16c148090 legacy=0x7fa16c0b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:57.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 --2- 172.21.15.42:0/2789112023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1500039f0 0x7fa150003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:57.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 --2- 172.21.15.42:0/2789112023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa150004350 0x7fa150004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:57.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 --2- 172.21.15.42:0/2789112023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150005030 0x7fa15000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:57.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 -- 172.21.15.42:0/2789112023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c148470 con 0x7fa150005030 2024-05-28T14:11:57.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 -- 172.21.15.42:0/2789112023 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa15000bf40 con 0x7fa150004350 2024-05-28T14:11:57.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169400640 1 -- 172.21.15.42:0/2789112023 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa15000c140 con 0x7fa1500039f0 2024-05-28T14:11:57.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa16a800640 1 --2- 172.21.15.42:0/2789112023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150005030 0x7fa15000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:57.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa169e00640 1 --2- 172.21.15.42:0/2789112023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1500039f0 0x7fa150003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:57.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa163e00640 1 --2- 172.21.15.42:0/2789112023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa150004350 0x7fa150004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:57.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa16a800640 1 -- 172.21.15.42:0/2789112023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1500039f0 msgr2=0x7fa150003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:57.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa16a800640 1 --2- 172.21.15.42:0/2789112023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1500039f0 0x7fa150003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:57.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa16a800640 1 -- 172.21.15.42:0/2789112023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa150004350 msgr2=0x7fa150004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:57.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa16a800640 1 --2- 172.21.15.42:0/2789112023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa150004350 0x7fa150004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:57.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.800+0000 7fa16a800640 1 -- 172.21.15.42:0/2789112023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa150003530 con 0x7fa150005030 2024-05-28T14:11:57.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa16a800640 1 --2- 172.21.15.42:0/2789112023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150005030 0x7fa15000b9d0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fa164001e70 tx=0x7fa16401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1866e8ec36f0b48c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:57.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa169400640 1 -- 172.21.15.42:0/2789112023 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164001a90 con 0x7fa150005030 2024-05-28T14:11:57.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa169400640 1 -- 172.21.15.42:0/2789112023 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa164003b80 con 0x7fa150005030 2024-05-28T14:11:57.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa169400640 1 -- 172.21.15.42:0/2789112023 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1640050d0 con 0x7fa150005030 2024-05-28T14:11:57.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa171000640 1 -- 172.21.15.42:0/2789112023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150005030 msgr2=0x7fa15000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:57.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa171000640 1 --2- 172.21.15.42:0/2789112023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150005030 0x7fa15000b9d0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fa164001e70 tx=0x7fa16401c780 comp rx=0 tx=0).stop 2024-05-28T14:11:57.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa16a800640 1 -- 172.21.15.42:0/2789112023 reap_dead start 2024-05-28T14:11:57.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa171000640 1 -- 172.21.15.42:0/2789112023 shutdown_connections 2024-05-28T14:11:57.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa171000640 1 -- 172.21.15.42:0/2789112023 >> 172.21.15.42:0/2789112023 conn(0x7fa16c0b21f0 msgr2=0x7fa16c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:57.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa171000640 1 -- 172.21.15.42:0/2789112023 shutdown_connections 2024-05-28T14:11:57.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa171000640 1 -- 172.21.15.42:0/2789112023 wait complete. 2024-05-28T14:11:57.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa171000640 1 Processor -- start 2024-05-28T14:11:57.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa171000640 1 -- start start 2024-05-28T14:11:57.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa171000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa150005030 0x7fa16c158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:57.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa171000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa16c15ff60 0x7fa16c141220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:57.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.804+0000 7fa171000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa16c144880 0x7fa16c144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:57.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa171000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c0b5830 con 0x7fa16c144880 2024-05-28T14:11:57.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa171000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c0b4730 con 0x7fa150005030 2024-05-28T14:11:57.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa171000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa16c0b4210 con 0x7fa16c15ff60 2024-05-28T14:11:57.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa16a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa16c144880 0x7fa16c144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:57.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa16a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa16c144880 0x7fa16c144d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51516/0 (socket says 172.21.15.42:51516) 2024-05-28T14:11:57.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa16a800640 1 -- 172.21.15.42:0/999218530 learned_addr learned my addr 172.21.15.42:0/999218530 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:57.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa169e00640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa150005030 0x7fa16c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:57.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa163e00640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa16c15ff60 0x7fa16c141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:57.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa16a800640 1 -- 172.21.15.42:0/999218530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa16c15ff60 msgr2=0x7fa16c141220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:57.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa16a800640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa16c15ff60 0x7fa16c141220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:57.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa16a800640 1 -- 172.21.15.42:0/999218530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa150005030 msgr2=0x7fa16c158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:57.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa16a800640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa150005030 0x7fa16c158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:57.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa16a800640 1 -- 172.21.15.42:0/999218530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16c141b00 con 0x7fa16c144880 2024-05-28T14:11:57.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa163e00640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa16c15ff60 0x7fa16c141220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:57.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa16a800640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa16c144880 0x7fa16c144d00 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fa1640980b0 tx=0x7fa150005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:57.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa162000640 1 -- 172.21.15.42:0/999218530 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164003980 con 0x7fa16c144880 2024-05-28T14:11:57.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa162000640 1 -- 172.21.15.42:0/999218530 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa164006b60 con 0x7fa16c144880 2024-05-28T14:11:57.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa162000640 1 -- 172.21.15.42:0/999218530 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1640050d0 con 0x7fa16c144880 2024-05-28T14:11:57.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa171000640 1 -- 172.21.15.42:0/999218530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16c141de0 con 0x7fa16c144880 2024-05-28T14:11:57.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa171000640 1 -- 172.21.15.42:0/999218530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa16c1423a0 con 0x7fa16c144880 2024-05-28T14:11:57.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa171000640 1 -- 172.21.15.42:0/999218530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa16c145240 con 0x7fa16c144880 2024-05-28T14:11:57.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.808+0000 7fa162000640 1 -- 172.21.15.42:0/999218530 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 36) ==== 89925+0+0 (secure 0 0 0) 0x7fa1640b8030 con 0x7fa16c144880 2024-05-28T14:11:57.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.812+0000 7fa162000640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa148069910 0x7fa14806bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:57.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.812+0000 7fa162000640 1 -- 172.21.15.42:0/999218530 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fa16411ef40 con 0x7fa16c144880 2024-05-28T14:11:57.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.812+0000 7fa169e00640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa148069910 0x7fa14806bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:57.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.812+0000 7fa169e00640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa148069910 0x7fa14806bdd0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa15c001060 tx=0x7fa15c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:57.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:57.816+0000 7fa162000640 1 -- 172.21.15.42:0/999218530 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa1640ac020 con 0x7fa16c144880 2024-05-28T14:11:57.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:58.012+0000 7fa171000640 1 -- 172.21.15.42:0/999218530 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fa16c0738b0 con 0x7fa148069910 2024-05-28T14:11:58.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:58.064+0000 7fa162000640 1 -- 172.21.15.42:0/999218530 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa16c0738b0 con 0x7fa148069910 2024-05-28T14:11:58.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:58.068+0000 7fa171000640 1 -- 172.21.15.42:0/999218530 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa148069910 msgr2=0x7fa14806bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:58.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:58.068+0000 7fa171000640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa148069910 0x7fa14806bdd0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa15c001060 tx=0x7fa15c009040 comp rx=0 tx=0).stop 2024-05-28T14:11:58.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:58.068+0000 7fa171000640 1 -- 172.21.15.42:0/999218530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa16c144880 msgr2=0x7fa16c144d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:58.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:58.068+0000 7fa171000640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa16c144880 0x7fa16c144d00 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fa1640980b0 tx=0x7fa150005840 comp rx=0 tx=0).stop 2024-05-28T14:11:58.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:58.068+0000 7fa171000640 1 -- 172.21.15.42:0/999218530 shutdown_connections 2024-05-28T14:11:58.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:58.068+0000 7fa171000640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa148069910 0x7fa14806bdd0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:58.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:58.068+0000 7fa171000640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa16c144880 0x7fa16c144d00 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:58.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:58.068+0000 7fa171000640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa16c15ff60 0x7fa16c141220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:58.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:58.068+0000 7fa171000640 1 --2- 172.21.15.42:0/999218530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa150005030 0x7fa16c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:58.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:58.068+0000 7fa171000640 1 -- 172.21.15.42:0/999218530 >> 172.21.15.42:0/999218530 conn(0x7fa16c0b21f0 msgr2=0x7fa16c162cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:58.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:58.068+0000 7fa171000640 1 -- 172.21.15.42:0/999218530 shutdown_connections 2024-05-28T14:11:58.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:58.068+0000 7fa171000640 1 -- 172.21.15.42:0/999218530 wait complete. 2024-05-28T14:11:58.057 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:58.068+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:58.057 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:58.068+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:58.057 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:58.068+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:58.057 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:58.068+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:58.058 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:58.068+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:58.071 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:58.088+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:58.072 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:58.088+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:58.072 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:58.088+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:58.072 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:58.088+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:58.072 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:11:58.088+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:11:59.076 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-05-28T14:11:59.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.248+0000 7f904d800640 1 Processor -- start 2024-05-28T14:11:59.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.248+0000 7f904d800640 1 -- start start 2024-05-28T14:11:59.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904d800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90480cc660 0x7f904815b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:59.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904d800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f90481415e0 0x7f904806db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:59.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904d800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f90480d0320 0x7f90480d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:59.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904d800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f90480d0cd0 con 0x7f90480d0320 2024-05-28T14:11:59.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904d800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f90480ce090 con 0x7f90481415e0 2024-05-28T14:11:59.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904d800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f904815bff0 con 0x7f90480cc660 2024-05-28T14:11:59.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904ce00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f90480d0320 0x7f90480d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:59.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904ce00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f90480d0320 0x7f90480d0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51556/0 (socket says 172.21.15.42:51556) 2024-05-28T14:11:59.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904ce00640 1 -- 172.21.15.42:0/819950366 learned_addr learned my addr 172.21.15.42:0/819950366 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:59.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f9047e00640 1 --2- 172.21.15.42:0/819950366 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90480cc660 0x7f904815b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:59.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f903f400640 1 --2- 172.21.15.42:0/819950366 >> v2:172.21.15.63:3300/0 conn(0x7f90481415e0 0x7f904806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:59.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904ce00640 1 -- 172.21.15.42:0/819950366 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90480cc660 msgr2=0x7f904815b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:59.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904ce00640 1 --2- 172.21.15.42:0/819950366 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90480cc660 0x7f904815b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:59.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904ce00640 1 -- 172.21.15.42:0/819950366 >> v2:172.21.15.63:3300/0 conn(0x7f90481415e0 msgr2=0x7f904806db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:11:59.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904ce00640 1 --2- 172.21.15.42:0/819950366 >> v2:172.21.15.63:3300/0 conn(0x7f90481415e0 0x7f904806db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:59.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904ce00640 1 -- 172.21.15.42:0/819950366 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904815c2d0 con 0x7f90480d0320 2024-05-28T14:11:59.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f903f400640 1 --2- 172.21.15.42:0/819950366 >> v2:172.21.15.63:3300/0 conn(0x7f90481415e0 0x7f904806db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:59.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f9047e00640 1 --2- 172.21.15.42:0/819950366 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90480cc660 0x7f904815b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:59.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.252+0000 7f904ce00640 1 --2- 172.21.15.42:0/819950366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90480d0320 0x7f90480d0790 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f90480cdcd0 tx=0x7f904001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7ca7bb4fded531 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:59.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f9047400640 1 -- 172.21.15.42:0/819950366 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90400b0070 con 0x7f90480d0320 2024-05-28T14:11:59.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f9047400640 1 -- 172.21.15.42:0/819950366 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90400ab600 con 0x7f90480d0320 2024-05-28T14:11:59.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f9047400640 1 -- 172.21.15.42:0/819950366 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90400ab9d0 con 0x7f90480d0320 2024-05-28T14:11:59.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 -- 172.21.15.42:0/819950366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90480d0320 msgr2=0x7f90480d0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:59.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 --2- 172.21.15.42:0/819950366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90480d0320 0x7f90480d0790 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f90480cdcd0 tx=0x7f904001c520 comp rx=0 tx=0).stop 2024-05-28T14:11:59.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 -- 172.21.15.42:0/819950366 shutdown_connections 2024-05-28T14:11:59.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 --2- 172.21.15.42:0/819950366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90480d0320 0x7f90480d0790 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:59.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 --2- 172.21.15.42:0/819950366 >> v2:172.21.15.63:3300/0 conn(0x7f90481415e0 0x7f904806db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:59.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 --2- 172.21.15.42:0/819950366 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90480cc660 0x7f904815b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 -- 172.21.15.42:0/819950366 >> 172.21.15.42:0/819950366 conn(0x7f904806bbc0 msgr2=0x7f904806bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:11:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 -- 172.21.15.42:0/819950366 shutdown_connections 2024-05-28T14:11:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 -- 172.21.15.42:0/819950366 wait complete. 2024-05-28T14:11:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 Processor -- start 2024-05-28T14:11:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 -- start start 2024-05-28T14:11:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90480cc660 0x7f90481436c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90480d0320 0x7f9048141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90481415e0 0x7f904814e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f904806e1f0 con 0x7f90480d0320 2024-05-28T14:11:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f904806e070 con 0x7f90480cc660 2024-05-28T14:11:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904d800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f904815ef90 con 0x7f90481415e0 2024-05-28T14:11:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f903f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90480d0320 0x7f9048141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f903f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90480d0320 0x7f9048141bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51568/0 (socket says 172.21.15.42:51568) 2024-05-28T14:11:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f903f400640 1 -- 172.21.15.42:0/1995682746 learned_addr learned my addr 172.21.15.42:0/1995682746 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:11:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f904ce00640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90481415e0 0x7f904814e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.256+0000 7f9047e00640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90480cc660 0x7f90481436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f903f400640 1 -- 172.21.15.42:0/1995682746 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90481415e0 msgr2=0x7f904814e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f903f400640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90481415e0 0x7f904814e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f903f400640 1 -- 172.21.15.42:0/1995682746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90480cc660 msgr2=0x7f90481436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:11:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f903f400640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90480cc660 0x7f90481436c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:11:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f903f400640 1 -- 172.21.15.42:0/1995682746 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9048142620 con 0x7f90480d0320 2024-05-28T14:11:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f9047e00640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90480cc660 0x7f90481436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:11:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f903f400640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90480d0320 0x7f9048141bd0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f903000c7f0 tx=0x7f903000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f9044c00640 1 -- 172.21.15.42:0/1995682746 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030015070 con 0x7f90480d0320 2024-05-28T14:11:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f9044c00640 1 -- 172.21.15.42:0/1995682746 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9030007820 con 0x7f90480d0320 2024-05-28T14:11:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f904d800640 1 -- 172.21.15.42:0/1995682746 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9048142900 con 0x7f90480d0320 2024-05-28T14:11:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f9044c00640 1 -- 172.21.15.42:0/1995682746 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030007c10 con 0x7f90480d0320 2024-05-28T14:11:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f904d800640 1 -- 172.21.15.42:0/1995682746 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90481431a0 con 0x7f90480d0320 2024-05-28T14:11:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f904d800640 1 -- 172.21.15.42:0/1995682746 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904804fcf0 con 0x7f90480d0320 2024-05-28T14:11:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f9044c00640 1 -- 172.21.15.42:0/1995682746 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 36) ==== 89925+0+0 (secure 0 0 0) 0x7f9030018070 con 0x7f90480d0320 2024-05-28T14:11:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f9044c00640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f90240698e0 0x7f902406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:11:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f9044c00640 1 -- 172.21.15.42:0/1995682746 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f9030085b60 con 0x7f90480d0320 2024-05-28T14:11:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.260+0000 7f9047e00640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f90240698e0 0x7f902406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:11:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.264+0000 7f9047e00640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f90240698e0 0x7f902406bda0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f903800d430 tx=0x7f9038007530 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:11:59.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.264+0000 7f9044c00640 1 -- 172.21.15.42:0/1995682746 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9030057770 con 0x7f90480d0320 2024-05-28T14:11:59.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:11:59.472+0000 7f904d800640 1 -- 172.21.15.42:0/1995682746 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f9048166100 con 0x7f90240698e0 2024-05-28T14:12:00.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:00.020+0000 7f9044c00640 1 -- 172.21.15.42:0/1995682746 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7f90300536e0 con 0x7f90480d0320 2024-05-28T14:12:00.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:00.676+0000 7f9044c00640 1 -- 172.21.15.42:0/1995682746 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9048166100 con 0x7f90240698e0 2024-05-28T14:12:00.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:00.676+0000 7f904d800640 1 -- 172.21.15.42:0/1995682746 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f90240698e0 msgr2=0x7f902406bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:00.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:00.676+0000 7f904d800640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f90240698e0 0x7f902406bda0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f903800d430 tx=0x7f9038007530 comp rx=0 tx=0).stop 2024-05-28T14:12:00.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:00.676+0000 7f904d800640 1 -- 172.21.15.42:0/1995682746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90480d0320 msgr2=0x7f9048141bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:00.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:00.676+0000 7f904d800640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90480d0320 0x7f9048141bd0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f903000c7f0 tx=0x7f903000ccb0 comp rx=0 tx=0).stop 2024-05-28T14:12:00.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:00.676+0000 7f904d800640 1 -- 172.21.15.42:0/1995682746 shutdown_connections 2024-05-28T14:12:00.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:00.676+0000 7f904d800640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f90240698e0 0x7f902406bda0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:00.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:00.676+0000 7f904d800640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90481415e0 0x7f904814e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:00.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:00.676+0000 7f904d800640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90480d0320 0x7f9048141bd0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:00.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:00.676+0000 7f904d800640 1 --2- 172.21.15.42:0/1995682746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90480cc660 0x7f90481436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:00.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:00.676+0000 7f904d800640 1 -- 172.21.15.42:0/1995682746 >> 172.21.15.42:0/1995682746 conn(0x7f904806bbc0 msgr2=0x7f90481538b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:00.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:00.680+0000 7f904d800640 1 -- 172.21.15.42:0/1995682746 shutdown_connections 2024-05-28T14:12:00.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:00.680+0000 7f904d800640 1 -- 172.21.15.42:0/1995682746 wait complete. 2024-05-28T14:12:05.683 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:12:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd960e00640 1 Processor -- start 2024-05-28T14:12:05.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd960e00640 1 -- start start 2024-05-28T14:12:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd960e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd95c15e9d0 0x7fd95c0d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:05.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd960e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd95c09b160 0x7fd95c0d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd960e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd95c0ccce0 con 0x7fd95c0b52d0 2024-05-28T14:12:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd960e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd95c0ccfc0 con 0x7fd95c09b160 2024-05-28T14:12:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd960e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd95c0cd2a0 con 0x7fd95c15e9d0 2024-05-28T14:12:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd95a800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd95c09b160 0x7fd95c0d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd95a800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd95c09b160 0x7fd95c0d1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51584/0 (socket says 172.21.15.42:51584) 2024-05-28T14:12:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd95a800640 1 -- 172.21.15.42:0/1742324934 learned_addr learned my addr 172.21.15.42:0/1742324934 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd959e00640 1 --2- 172.21.15.42:0/1742324934 >> v2:172.21.15.63:3300/0 conn(0x7fd95c15e9d0 0x7fd95c0d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:05.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd95a800640 1 -- 172.21.15.42:0/1742324934 >> v2:172.21.15.63:3300/0 conn(0x7fd95c15e9d0 msgr2=0x7fd95c0d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd95a800640 1 --2- 172.21.15.42:0/1742324934 >> v2:172.21.15.63:3300/0 conn(0x7fd95c15e9d0 0x7fd95c0d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd95a800640 1 -- 172.21.15.42:0/1742324934 >> v1:172.21.15.42:6789/0 conn(0x7fd95c0b52d0 legacy=0x7fd95c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd95a800640 1 -- 172.21.15.42:0/1742324934 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95c0cd580 con 0x7fd95c09b160 2024-05-28T14:12:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.851+0000 7fd959400640 1 -- 172.21.15.42:0/1742324934 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2892320127 0 0) 0x7fd95c0ccce0 con 0x7fd95c0b52d0 2024-05-28T14:12:05.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.855+0000 7fd95a800640 1 --2- 172.21.15.42:0/1742324934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd95c09b160 0x7fd95c0d1800 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd95c06c490 tx=0x7fd95401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74914a7d0b1f3ef2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:05.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.855+0000 7fd959400640 1 -- 172.21.15.42:0/1742324934 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9540a5750 con 0x7fd95c09b160 2024-05-28T14:12:05.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.855+0000 7fd959400640 1 -- 172.21.15.42:0/1742324934 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9540a58f0 con 0x7fd95c09b160 2024-05-28T14:12:05.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.855+0000 7fd959400640 1 -- 172.21.15.42:0/1742324934 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9540a5cc0 con 0x7fd95c09b160 2024-05-28T14:12:05.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.855+0000 7fd960e00640 1 -- 172.21.15.42:0/1742324934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd95c09b160 msgr2=0x7fd95c0d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:05.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.855+0000 7fd960e00640 1 --2- 172.21.15.42:0/1742324934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd95c09b160 0x7fd95c0d1800 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd95c06c490 tx=0x7fd95401c9f0 comp rx=0 tx=0).stop 2024-05-28T14:12:05.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.855+0000 7fd960e00640 1 -- 172.21.15.42:0/1742324934 shutdown_connections 2024-05-28T14:12:05.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.855+0000 7fd960e00640 1 --2- 172.21.15.42:0/1742324934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd95c09b160 0x7fd95c0d1800 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:05.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.855+0000 7fd960e00640 1 --2- 172.21.15.42:0/1742324934 >> v2:172.21.15.63:3300/0 conn(0x7fd95c15e9d0 0x7fd95c0d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:05.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.855+0000 7fd960e00640 1 -- 172.21.15.42:0/1742324934 >> 172.21.15.42:0/1742324934 conn(0x7fd95c0b3f70 msgr2=0x7fd95c0b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:05.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.855+0000 7fd960e00640 1 -- 172.21.15.42:0/1742324934 shutdown_connections 2024-05-28T14:12:05.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.855+0000 7fd960e00640 1 -- 172.21.15.42:0/1742324934 wait complete. 2024-05-28T14:12:05.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd960e00640 1 Processor -- start 2024-05-28T14:12:05.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd960e00640 1 -- start start 2024-05-28T14:12:05.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd960e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd95c09b160 0x7fd95c161650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:05.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd960e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd95c0b52d0 0x7fd95c14c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:05.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd960e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd95c15e9d0 0x7fd95c0b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd960e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c0b3c30 con 0x7fd95c0b52d0 2024-05-28T14:12:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd960e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c0b3ab0 con 0x7fd95c09b160 2024-05-28T14:12:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd960e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd95c0b3db0 con 0x7fd95c15e9d0 2024-05-28T14:12:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd951400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd95c0b52d0 0x7fd95c14c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd951400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd95c0b52d0 0x7fd95c14c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51596/0 (socket says 172.21.15.42:51596) 2024-05-28T14:12:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd951400640 1 -- 172.21.15.42:0/3801070774 learned_addr learned my addr 172.21.15.42:0/3801070774 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd959e00640 1 --2- 172.21.15.42:0/3801070774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd95c09b160 0x7fd95c161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd95a800640 1 --2- 172.21.15.42:0/3801070774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd95c15e9d0 0x7fd95c0b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd951400640 1 -- 172.21.15.42:0/3801070774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd95c15e9d0 msgr2=0x7fd95c0b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd951400640 1 --2- 172.21.15.42:0/3801070774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd95c15e9d0 0x7fd95c0b6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd951400640 1 -- 172.21.15.42:0/3801070774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd95c09b160 msgr2=0x7fd95c161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd951400640 1 --2- 172.21.15.42:0/3801070774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd95c09b160 0x7fd95c161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:05.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd951400640 1 -- 172.21.15.42:0/3801070774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95c0cad00 con 0x7fd95c0b52d0 2024-05-28T14:12:05.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd951400640 1 --2- 172.21.15.42:0/3801070774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd95c0b52d0 0x7fd95c14c780 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd94400c550 tx=0x7fd94400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:05.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd952a00640 1 -- 172.21.15.42:0/3801070774 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944017070 con 0x7fd95c0b52d0 2024-05-28T14:12:05.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd952a00640 1 -- 172.21.15.42:0/3801070774 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd944003600 con 0x7fd95c0b52d0 2024-05-28T14:12:05.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd952a00640 1 -- 172.21.15.42:0/3801070774 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440039f0 con 0x7fd95c0b52d0 2024-05-28T14:12:05.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd960e00640 1 -- 172.21.15.42:0/3801070774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95c0cafe0 con 0x7fd95c0b52d0 2024-05-28T14:12:05.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.859+0000 7fd960e00640 1 -- 172.21.15.42:0/3801070774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd95c0cb880 con 0x7fd95c0b52d0 2024-05-28T14:12:05.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.863+0000 7fd960e00640 1 -- 172.21.15.42:0/3801070774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd95c068a90 con 0x7fd95c0b52d0 2024-05-28T14:12:05.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.863+0000 7fd952a00640 1 -- 172.21.15.42:0/3801070774 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7fd94401a070 con 0x7fd95c0b52d0 2024-05-28T14:12:05.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.863+0000 7fd952a00640 1 --2- 172.21.15.42:0/3801070774 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd938069a00 0x7fd93806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:05.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.863+0000 7fd952a00640 1 -- 172.21.15.42:0/3801070774 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fd944084ab0 con 0x7fd95c0b52d0 2024-05-28T14:12:05.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.863+0000 7fd959e00640 1 --2- 172.21.15.42:0/3801070774 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd938069a00 0x7fd93806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:05.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.863+0000 7fd959e00640 1 --2- 172.21.15.42:0/3801070774 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd938069a00 0x7fd93806bec0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd94c0079c0 tx=0x7fd94c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:05.867+0000 7fd952a00640 1 -- 172.21.15.42:0/3801070774 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd944056480 con 0x7fd95c0b52d0 2024-05-28T14:12:06.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.091+0000 7fd960e00640 1 -- 172.21.15.42:0/3801070774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd95c0c9010 con 0x7fd95c0b52d0 2024-05-28T14:12:06.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.091+0000 7fd952a00640 1 -- 172.21.15.42:0/3801070774 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2113 (secure 0 0 0) 0x7fd94405a780 con 0x7fd95c0b52d0 2024-05-28T14:12:06.076 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:06.076 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:12:06.076 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:12:06.076 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:12:06.076 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:12:06.076 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:12:06.076 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:12:06.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:06.076 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:12:06.077 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:12:06.077 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:12:06.077 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:12:06.077 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:12:06.077 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:06.077 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:12:06.077 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:12:06.077 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:12:06.077 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:12:06.077 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:06.077 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 847, 2024-05-28T14:12:06.077 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:12:06.077 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:12:06.077 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:12:06.078 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:12:06.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:06.078 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:12:06.078 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 105, 2024-05-28T14:12:06.078 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:12:06.078 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:12:06.078 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:12:06.078 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:12:06.078 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:12:06.078 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:12:06.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:06.078 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:12:06.078 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:12:06.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:06.079 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:12:06.079 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:12:06.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:06.079 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:06.079 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:12:06.079 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:12:06.079 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 798, 2024-05-28T14:12:06.079 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 3222129480, 2024-05-28T14:12:06.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 7441719296, 2024-05-28T14:12:06.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 765652393984, 2024-05-28T14:12:06.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:12:06.079 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 597, 2024-05-28T14:12:06.079 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 10147652, 2024-05-28T14:12:06.079 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:12:06.080 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 6 2024-05-28T14:12:06.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:06.080 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:12:06.080 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 65, 2024-05-28T14:12:06.080 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 7, 2024-05-28T14:12:06.080 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:12:06.080 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:12:06.080 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:12:06.080 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:12:06.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:06.080 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 7, 2024-05-28T14:12:06.080 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:12:06.080 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:12:06.080 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:12:06.081 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57056 2024-05-28T14:12:06.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:06.081 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:06.081 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:12:06.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:06.081 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:12:06.081 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:12:06.081 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:12:06.081 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:12:06.081 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:12:06.081 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:12:06.081 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:12:06.081 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:06.081 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:12:06.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:06.082 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:12:06.082 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 16, 2024-05-28T14:12:06.082 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:10:37.128981+0000", 2024-05-28T14:12:06.082 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:12:06.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:06.082 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:12:06.082 INFO:teuthology.orchestra.run.smithi042.stdout: "41758bd8-9a69-4e56-a945-2b876a2d9b46": { 2024-05-28T14:12:06.083 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 27.0 seconds before exiting (4s)\n [============================] ", 2024-05-28T14:12:06.083 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:12:06.083 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:12:06.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:06.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:06.083 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:12:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.091+0000 7fd960e00640 1 -- 172.21.15.42:0/3801070774 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd938069a00 msgr2=0x7fd93806bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.091+0000 7fd960e00640 1 --2- 172.21.15.42:0/3801070774 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd938069a00 0x7fd93806bec0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd94c0079c0 tx=0x7fd94c008040 comp rx=0 tx=0).stop 2024-05-28T14:12:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.091+0000 7fd960e00640 1 -- 172.21.15.42:0/3801070774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd95c0b52d0 msgr2=0x7fd95c14c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.091+0000 7fd960e00640 1 --2- 172.21.15.42:0/3801070774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd95c0b52d0 0x7fd95c14c780 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd94400c550 tx=0x7fd94400ca10 comp rx=0 tx=0).stop 2024-05-28T14:12:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.091+0000 7fd960e00640 1 -- 172.21.15.42:0/3801070774 shutdown_connections 2024-05-28T14:12:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.091+0000 7fd960e00640 1 --2- 172.21.15.42:0/3801070774 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd938069a00 0x7fd93806bec0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.091+0000 7fd960e00640 1 --2- 172.21.15.42:0/3801070774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd95c15e9d0 0x7fd95c0b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.091+0000 7fd960e00640 1 --2- 172.21.15.42:0/3801070774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd95c0b52d0 0x7fd95c14c780 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.091+0000 7fd960e00640 1 --2- 172.21.15.42:0/3801070774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd95c09b160 0x7fd95c161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.091+0000 7fd960e00640 1 -- 172.21.15.42:0/3801070774 >> 172.21.15.42:0/3801070774 conn(0x7fd95c0b3f70 msgr2=0x7fd95c0cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:06.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.091+0000 7fd960e00640 1 -- 172.21.15.42:0/3801070774 shutdown_connections 2024-05-28T14:12:06.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.091+0000 7fd960e00640 1 -- 172.21.15.42:0/3801070774 wait complete. 2024-05-28T14:12:06.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:12:06.097 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/ss1clone1/9d1bb6f5-8933-4444-9d42-272b2b6c2227 2024-05-28T14:12:06.097 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:12:06.111 INFO:teuthology.orchestra.run.smithi042.stderr:ls: cannot access 'volumes/_nogroup/ss1clone1/9d1bb6f5-8933-4444-9d42-272b2b6c2227': No such file or directory 2024-05-28T14:12:06.112 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-28T14:12:06.113 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:12:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f165e800640 1 Processor -- start 2024-05-28T14:12:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f165e800640 1 -- start start 2024-05-28T14:12:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f165e800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1658145920 0x7f165806f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f165e800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f165806db40 0x7f1658071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:06.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f165e800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f165806dfd0 con 0x7f1658072300 2024-05-28T14:12:06.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f165e800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1658072710 con 0x7f1658145920 2024-05-28T14:12:06.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f165e800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f165806e240 con 0x7f165806db40 2024-05-28T14:12:06.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f1657400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1658145920 0x7f165806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:06.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f1657e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1658072300 0x7f1658160100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:33188/0 (socket says 172.21.15.42:33188) 2024-05-28T14:12:06.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f1657e00640 1 -- 172.21.15.42:0/2493568277 learned_addr learned my addr 172.21.15.42:0/2493568277 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:06.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f1657400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1658145920 0x7f165806f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51598/0 (socket says 172.21.15.42:51598) 2024-05-28T14:12:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f164ea00640 1 --2- 172.21.15.42:0/2493568277 >> v2:172.21.15.63:3300/0 conn(0x7f165806db40 0x7f1658071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f1657400640 1 -- 172.21.15.42:0/2493568277 >> v2:172.21.15.63:3300/0 conn(0x7f165806db40 msgr2=0x7f1658071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f1657400640 1 --2- 172.21.15.42:0/2493568277 >> v2:172.21.15.63:3300/0 conn(0x7f165806db40 0x7f1658071dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f1657400640 1 -- 172.21.15.42:0/2493568277 >> v1:172.21.15.42:6789/0 conn(0x7f1658072300 legacy=0x7f1658160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f1656a00640 1 -- 172.21.15.42:0/2493568277 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4118807607 0 0) 0x7f165806dfd0 con 0x7f1658072300 2024-05-28T14:12:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f1657400640 1 -- 172.21.15.42:0/2493568277 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165806e520 con 0x7f1658145920 2024-05-28T14:12:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.315+0000 7f164ea00640 1 --2- 172.21.15.42:0/2493568277 >> v2:172.21.15.63:3300/0 conn(0x7f165806db40 0x7f1658071dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:06.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.319+0000 7f1657400640 1 --2- 172.21.15.42:0/2493568277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1658145920 0x7f165806f030 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f16580b35a0 tx=0x7f164801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af919d0243839a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:06.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.319+0000 7f1656a00640 1 -- 172.21.15.42:0/2493568277 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16480b0070 con 0x7f1658145920 2024-05-28T14:12:06.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.319+0000 7f1656a00640 1 -- 172.21.15.42:0/2493568277 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16480ab600 con 0x7f1658145920 2024-05-28T14:12:06.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.319+0000 7f1656a00640 1 -- 172.21.15.42:0/2493568277 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16480ab9d0 con 0x7f1658145920 2024-05-28T14:12:06.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.319+0000 7f165e800640 1 -- 172.21.15.42:0/2493568277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1658145920 msgr2=0x7f165806f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:06.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.319+0000 7f165e800640 1 --2- 172.21.15.42:0/2493568277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1658145920 0x7f165806f030 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f16580b35a0 tx=0x7f164801c3f0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.319+0000 7f165e800640 1 -- 172.21.15.42:0/2493568277 shutdown_connections 2024-05-28T14:12:06.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.319+0000 7f165e800640 1 --2- 172.21.15.42:0/2493568277 >> v2:172.21.15.63:3300/0 conn(0x7f165806db40 0x7f1658071dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.319+0000 7f165e800640 1 --2- 172.21.15.42:0/2493568277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1658145920 0x7f165806f030 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.319+0000 7f165e800640 1 -- 172.21.15.42:0/2493568277 >> 172.21.15.42:0/2493568277 conn(0x7f165806b9f0 msgr2=0x7f165806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:06.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.319+0000 7f165e800640 1 -- 172.21.15.42:0/2493568277 shutdown_connections 2024-05-28T14:12:06.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.319+0000 7f165e800640 1 -- 172.21.15.42:0/2493568277 wait complete. 2024-05-28T14:12:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f165e800640 1 Processor -- start 2024-05-28T14:12:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f165e800640 1 -- start start 2024-05-28T14:12:06.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f165e800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f165806db40 0x7f16580af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:06.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f165e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1658072300 0x7f1658165880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:06.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f165e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1658145920 0x7f1658165dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:06.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f165e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f16581693d0 con 0x7f1658145920 2024-05-28T14:12:06.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f165e800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1658169250 con 0x7f1658072300 2024-05-28T14:12:06.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f165e800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1658169550 con 0x7f165806db40 2024-05-28T14:12:06.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f1657e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1658145920 0x7f1658165dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:06.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f1657e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1658145920 0x7f1658165dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51604/0 (socket says 172.21.15.42:51604) 2024-05-28T14:12:06.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f1657e00640 1 -- 172.21.15.42:0/1673801056 learned_addr learned my addr 172.21.15.42:0/1673801056 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:06.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f1657400640 1 --2- 172.21.15.42:0/1673801056 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f165806db40 0x7f16580af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:06.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f164ea00640 1 --2- 172.21.15.42:0/1673801056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1658072300 0x7f1658165880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:06.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f1657e00640 1 -- 172.21.15.42:0/1673801056 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f165806db40 msgr2=0x7f16580af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:06.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f1657e00640 1 --2- 172.21.15.42:0/1673801056 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f165806db40 0x7f16580af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f1657e00640 1 -- 172.21.15.42:0/1673801056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1658072300 msgr2=0x7f1658165880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:06.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f1657e00640 1 --2- 172.21.15.42:0/1673801056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1658072300 0x7f1658165880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f1657e00640 1 -- 172.21.15.42:0/1673801056 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16580afae0 con 0x7f1658145920 2024-05-28T14:12:06.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f1657e00640 1 --2- 172.21.15.42:0/1673801056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1658145920 0x7f1658165dc0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f165000c700 tx=0x7f165000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:06.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f164fe00640 1 -- 172.21.15.42:0/1673801056 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650018070 con 0x7f1658145920 2024-05-28T14:12:06.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f164fe00640 1 -- 172.21.15.42:0/1673801056 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165000ce10 con 0x7f1658145920 2024-05-28T14:12:06.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f164fe00640 1 -- 172.21.15.42:0/1673801056 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165000a860 con 0x7f1658145920 2024-05-28T14:12:06.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f165e800640 1 -- 172.21.15.42:0/1673801056 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16581fb620 con 0x7f1658145920 2024-05-28T14:12:06.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f165e800640 1 -- 172.21.15.42:0/1673801056 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16581fbbe0 con 0x7f1658145920 2024-05-28T14:12:06.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.323+0000 7f165e800640 1 -- 172.21.15.42:0/1673801056 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1658073890 con 0x7f1658145920 2024-05-28T14:12:06.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.327+0000 7f164fe00640 1 -- 172.21.15.42:0/1673801056 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7f165000aa00 con 0x7f1658145920 2024-05-28T14:12:06.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.327+0000 7f164fe00640 1 --2- 172.21.15.42:0/1673801056 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1634069a00 0x7f163406bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:06.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.327+0000 7f164fe00640 1 -- 172.21.15.42:0/1673801056 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f1650084ca0 con 0x7f1658145920 2024-05-28T14:12:06.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.327+0000 7f1657400640 1 --2- 172.21.15.42:0/1673801056 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1634069a00 0x7f163406bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:06.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.327+0000 7f1657400640 1 --2- 172.21.15.42:0/1673801056 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1634069a00 0x7f163406bec0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f16480ad000 tx=0x7f16480a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:06.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.331+0000 7f164fe00640 1 -- 172.21.15.42:0/1673801056 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1650056670 con 0x7f1658145920 2024-05-28T14:12:06.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.519+0000 7f165e800640 1 -- 172.21.15.42:0/1673801056 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f165804fdd0 con 0x7f1634069a00 2024-05-28T14:12:06.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.519+0000 7f164fe00640 1 -- 172.21.15.42:0/1673801056 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f165804fdd0 con 0x7f1634069a00 2024-05-28T14:12:06.506 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:12:06.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:06.506 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:12:06.506 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:06.506 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:12:06.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.523+0000 7f165e800640 1 -- 172.21.15.42:0/1673801056 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1634069a00 msgr2=0x7f163406bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:06.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.523+0000 7f165e800640 1 --2- 172.21.15.42:0/1673801056 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1634069a00 0x7f163406bec0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f16480ad000 tx=0x7f16480a6000 comp rx=0 tx=0).stop 2024-05-28T14:12:06.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.523+0000 7f165e800640 1 -- 172.21.15.42:0/1673801056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1658145920 msgr2=0x7f1658165dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:06.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.523+0000 7f165e800640 1 --2- 172.21.15.42:0/1673801056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1658145920 0x7f1658165dc0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f165000c700 tx=0x7f165000cbc0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.523+0000 7f165e800640 1 -- 172.21.15.42:0/1673801056 shutdown_connections 2024-05-28T14:12:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.523+0000 7f165e800640 1 --2- 172.21.15.42:0/1673801056 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1634069a00 0x7f163406bec0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.523+0000 7f165e800640 1 --2- 172.21.15.42:0/1673801056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1658145920 0x7f1658165dc0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.523+0000 7f165e800640 1 --2- 172.21.15.42:0/1673801056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1658072300 0x7f1658165880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.523+0000 7f165e800640 1 --2- 172.21.15.42:0/1673801056 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f165806db40 0x7f16580af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.523+0000 7f165e800640 1 -- 172.21.15.42:0/1673801056 >> 172.21.15.42:0/1673801056 conn(0x7f165806b9f0 msgr2=0x7f16580b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.523+0000 7f165e800640 1 -- 172.21.15.42:0/1673801056 shutdown_connections 2024-05-28T14:12:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.523+0000 7f165e800640 1 -- 172.21.15.42:0/1673801056 wait complete. 2024-05-28T14:12:06.528 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:12:06.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.683+0000 7f299de00640 1 Processor -- start 2024-05-28T14:12:06.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.683+0000 7f299de00640 1 -- start start 2024-05-28T14:12:06.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.687+0000 7f299de00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f299815fd00 0x7f29980d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:06.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.687+0000 7f299de00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f29980d0590 0x7f29980cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:06.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.687+0000 7f299de00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29980d0a30 con 0x7f29980cd5d0 2024-05-28T14:12:06.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.687+0000 7f299de00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f29980b52d0 con 0x7f29980d0590 2024-05-28T14:12:06.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.687+0000 7f299de00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f29980d0d10 con 0x7f299815fd00 2024-05-28T14:12:06.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.687+0000 7f298fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f29980d0590 0x7f29980cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:06.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.687+0000 7f298fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f29980d0590 0x7f29980cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51620/0 (socket says 172.21.15.42:51620) 2024-05-28T14:12:06.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.687+0000 7f298fe00640 1 -- 172.21.15.42:0/3374837 learned_addr learned my addr 172.21.15.42:0/3374837 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:06.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.687+0000 7f2996a00640 1 --2- 172.21.15.42:0/3374837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f299815fd00 0x7f29980d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:06.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.687+0000 7f298fe00640 1 -- 172.21.15.42:0/3374837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f299815fd00 msgr2=0x7f29980d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:06.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.687+0000 7f298fe00640 1 --2- 172.21.15.42:0/3374837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f299815fd00 0x7f29980d0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.687+0000 7f298fe00640 1 -- 172.21.15.42:0/3374837 >> v1:172.21.15.63:6789/0 conn(0x7f29980cd5d0 legacy=0x7f299809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:06.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.687+0000 7f298fe00640 1 -- 172.21.15.42:0/3374837 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29980d0ff0 con 0x7f29980d0590 2024-05-28T14:12:06.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.687+0000 7f298fe00640 1 --2- 172.21.15.42:0/3374837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29980d0590 0x7f29980cd090 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f29980b5450 tx=0x7f298001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824d20c822142303 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:06.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f2996000640 1 -- 172.21.15.42:0/3374837 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29800a95f0 con 0x7f29980d0590 2024-05-28T14:12:06.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f2996000640 1 -- 172.21.15.42:0/3374837 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29800a9790 con 0x7f29980d0590 2024-05-28T14:12:06.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f2996000640 1 -- 172.21.15.42:0/3374837 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29800a9b60 con 0x7f29980d0590 2024-05-28T14:12:06.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 -- 172.21.15.42:0/3374837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29980d0590 msgr2=0x7f29980cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:06.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 --2- 172.21.15.42:0/3374837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29980d0590 0x7f29980cd090 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f29980b5450 tx=0x7f298001c510 comp rx=0 tx=0).stop 2024-05-28T14:12:06.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 -- 172.21.15.42:0/3374837 shutdown_connections 2024-05-28T14:12:06.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 --2- 172.21.15.42:0/3374837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29980d0590 0x7f29980cd090 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 --2- 172.21.15.42:0/3374837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f299815fd00 0x7f29980d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 -- 172.21.15.42:0/3374837 >> 172.21.15.42:0/3374837 conn(0x7f29980b3f70 msgr2=0x7f29980b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:06.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 -- 172.21.15.42:0/3374837 shutdown_connections 2024-05-28T14:12:06.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 -- 172.21.15.42:0/3374837 wait complete. 2024-05-28T14:12:06.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 Processor -- start 2024-05-28T14:12:06.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 -- start start 2024-05-28T14:12:06.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29980cd5d0 0x7f2998154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:06.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29980d0590 0x7f29981533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:06.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f299815fd00 0x7f29981576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:06.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f29980b34d0 con 0x7f299815fd00 2024-05-28T14:12:06.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29981602f0 con 0x7f29980cd5d0 2024-05-28T14:12:06.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f299de00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f29980b3650 con 0x7f29980d0590 2024-05-28T14:12:06.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f2997400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f299815fd00 0x7f29981576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:06.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f2996a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29980cd5d0 0x7f2998154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:06.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f2997400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f299815fd00 0x7f29981576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51626/0 (socket says 172.21.15.42:51626) 2024-05-28T14:12:06.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f2997400640 1 -- 172.21.15.42:0/569316784 learned_addr learned my addr 172.21.15.42:0/569316784 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:06.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.691+0000 7f298fe00640 1 --2- 172.21.15.42:0/569316784 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29980d0590 0x7f29981533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:06.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f2997400640 1 -- 172.21.15.42:0/569316784 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29980d0590 msgr2=0x7f29981533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:06.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f2997400640 1 --2- 172.21.15.42:0/569316784 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29980d0590 0x7f29981533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f2997400640 1 -- 172.21.15.42:0/569316784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29980cd5d0 msgr2=0x7f2998154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f2997400640 1 --2- 172.21.15.42:0/569316784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29980cd5d0 0x7f2998154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f2997400640 1 -- 172.21.15.42:0/569316784 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2998153c20 con 0x7f299815fd00 2024-05-28T14:12:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f298fe00640 1 --2- 172.21.15.42:0/569316784 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29980d0590 0x7f29981533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:12:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f2997400640 1 --2- 172.21.15.42:0/569316784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f299815fd00 0x7f29981576a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f299000c770 tx=0x7f299000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f298f400640 1 -- 172.21.15.42:0/569316784 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990003b80 con 0x7f299815fd00 2024-05-28T14:12:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f298f400640 1 -- 172.21.15.42:0/569316784 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2990003d20 con 0x7f299815fd00 2024-05-28T14:12:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f298f400640 1 -- 172.21.15.42:0/569316784 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299000aa90 con 0x7f299815fd00 2024-05-28T14:12:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f299de00640 1 -- 172.21.15.42:0/569316784 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2998153f00 con 0x7f299815fd00 2024-05-28T14:12:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f299de00640 1 -- 172.21.15.42:0/569316784 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29981544c0 con 0x7f299815fd00 2024-05-28T14:12:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f299de00640 1 -- 172.21.15.42:0/569316784 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2998154700 con 0x7f299815fd00 2024-05-28T14:12:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f298f400640 1 -- 172.21.15.42:0/569316784 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7f299000ac30 con 0x7f299815fd00 2024-05-28T14:12:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f298f400640 1 --2- 172.21.15.42:0/569316784 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2974069a30 0x7f297406bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:06.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f298f400640 1 -- 172.21.15.42:0/569316784 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f2990085510 con 0x7f299815fd00 2024-05-28T14:12:06.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.695+0000 7f2996a00640 1 --2- 172.21.15.42:0/569316784 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2974069a30 0x7f297406bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:06.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.699+0000 7f2996a00640 1 --2- 172.21.15.42:0/569316784 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2974069a30 0x7f297406bef0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f29880079c0 tx=0x7f2988008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:06.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.699+0000 7f298f400640 1 -- 172.21.15.42:0/569316784 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2998154700 con 0x7f299815fd00 2024-05-28T14:12:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.923+0000 7f299de00640 1 -- 172.21.15.42:0/569316784 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f299809bc00 con 0x7f299815fd00 2024-05-28T14:12:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.923+0000 7f298f400640 1 -- 172.21.15.42:0/569316784 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2113 (secure 0 0 0) 0x7f2990056ee0 con 0x7f299815fd00 2024-05-28T14:12:06.908 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:06.908 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:12:06.908 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:12:06.908 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:12:06.908 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:12:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:12:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:12:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:12:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:12:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:12:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:12:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:12:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:12:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:12:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:12:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:12:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:06.910 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 848, 2024-05-28T14:12:06.910 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:12:06.910 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:12:06.910 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:12:06.910 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:12:06.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:06.910 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:12:06.910 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 105, 2024-05-28T14:12:06.910 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:12:06.910 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:12:06.910 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:12:06.910 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:12:06.910 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:12:06.911 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:12:06.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:06.911 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:12:06.911 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:12:06.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:06.911 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:12:06.911 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:12:06.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:06.911 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:06.911 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:12:06.911 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:12:06.911 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 798, 2024-05-28T14:12:06.911 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 3222129480, 2024-05-28T14:12:06.911 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 7441719296, 2024-05-28T14:12:06.912 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 765652393984, 2024-05-28T14:12:06.912 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:12:06.912 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 597, 2024-05-28T14:12:06.912 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 10147652, 2024-05-28T14:12:06.912 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:12:06.912 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 6 2024-05-28T14:12:06.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:06.912 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:12:06.912 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 65, 2024-05-28T14:12:06.912 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 7, 2024-05-28T14:12:06.912 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:12:06.912 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:12:06.913 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:12:06.913 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:12:06.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:06.913 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 7, 2024-05-28T14:12:06.913 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:12:06.913 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:12:06.913 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:12:06.913 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57056 2024-05-28T14:12:06.913 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:06.913 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:06.913 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:12:06.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:06.914 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:12:06.914 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:12:06.914 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:12:06.914 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:12:06.914 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:12:06.914 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:12:06.914 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:12:06.914 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:06.914 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:12:06.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:06.914 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:12:06.914 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 16, 2024-05-28T14:12:06.914 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:10:37.128981+0000", 2024-05-28T14:12:06.915 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:12:06.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:06.915 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:12:06.915 INFO:teuthology.orchestra.run.smithi042.stdout: "41758bd8-9a69-4e56-a945-2b876a2d9b46": { 2024-05-28T14:12:06.915 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 27.0 seconds before exiting (4s)\n [============================] ", 2024-05-28T14:12:06.915 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:12:06.915 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:12:06.915 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:06.915 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:06.916 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:12:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.923+0000 7f299de00640 1 -- 172.21.15.42:0/569316784 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2974069a30 msgr2=0x7f297406bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.923+0000 7f299de00640 1 --2- 172.21.15.42:0/569316784 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2974069a30 0x7f297406bef0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f29880079c0 tx=0x7f2988008040 comp rx=0 tx=0).stop 2024-05-28T14:12:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.923+0000 7f299de00640 1 -- 172.21.15.42:0/569316784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f299815fd00 msgr2=0x7f29981576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.923+0000 7f299de00640 1 --2- 172.21.15.42:0/569316784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f299815fd00 0x7f29981576a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f299000c770 tx=0x7f299000cc30 comp rx=0 tx=0).stop 2024-05-28T14:12:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.923+0000 7f299de00640 1 -- 172.21.15.42:0/569316784 shutdown_connections 2024-05-28T14:12:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.923+0000 7f299de00640 1 --2- 172.21.15.42:0/569316784 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2974069a30 0x7f297406bef0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.923+0000 7f299de00640 1 --2- 172.21.15.42:0/569316784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f299815fd00 0x7f29981576a0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.923+0000 7f299de00640 1 --2- 172.21.15.42:0/569316784 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29980d0590 0x7f29981533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.923+0000 7f299de00640 1 --2- 172.21.15.42:0/569316784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29980cd5d0 0x7f2998154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.923+0000 7f299de00640 1 -- 172.21.15.42:0/569316784 >> 172.21.15.42:0/569316784 conn(0x7f29980b3f70 msgr2=0x7f2998156310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.923+0000 7f299de00640 1 -- 172.21.15.42:0/569316784 shutdown_connections 2024-05-28T14:12:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:06.923+0000 7f299de00640 1 -- 172.21.15.42:0/569316784 wait complete. 2024-05-28T14:12:11.930 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:12:12.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb39200640 1 Processor -- start 2024-05-28T14:12:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb39200640 1 -- start start 2024-05-28T14:12:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb39200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcb3415e860 0x7fcb3415f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb39200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb3415fea0 con 0x7fcb34070070 2024-05-28T14:12:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb39200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb341632f0 con 0x7fcb3406ca00 2024-05-28T14:12:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb39200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcb3406e610 con 0x7fcb3415e860 2024-05-28T14:12:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb32800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fcb34070070 0x7fcb34162be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42116/0 (socket says 172.21.15.42:42116) 2024-05-28T14:12:12.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb32800640 1 -- 172.21.15.42:0/1076836994 learned_addr learned my addr 172.21.15.42:0/1076836994 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:12.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb31e00640 1 --2- 172.21.15.42:0/1076836994 >> v2:172.21.15.63:3300/0 conn(0x7fcb3415e860 0x7fcb3415f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:12.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb31400640 1 -- 172.21.15.42:0/1076836994 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 16459018 0 0) 0x7fcb2c0014a0 con 0x7fcb34070070 2024-05-28T14:12:12.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb31400640 1 -- 172.21.15.42:0/1076836994 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb18003530 con 0x7fcb34070070 2024-05-28T14:12:12.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb31e00640 1 -- 172.21.15.42:0/1076836994 >> v1:172.21.15.63:6789/0 conn(0x7fcb3406ca00 legacy=0x7fcb341624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:12.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb31400640 1 -- 172.21.15.42:0/1076836994 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2203797126 0 0) 0x7fcb341632f0 con 0x7fcb3406ca00 2024-05-28T14:12:12.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb31e00640 1 -- 172.21.15.42:0/1076836994 >> v1:172.21.15.42:6789/0 conn(0x7fcb34070070 legacy=0x7fcb34162be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:12.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.099+0000 7fcb31e00640 1 -- 172.21.15.42:0/1076836994 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb340b4720 con 0x7fcb3415e860 2024-05-28T14:12:12.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.103+0000 7fcb31e00640 1 --2- 172.21.15.42:0/1076836994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb3415e860 0x7fcb3415f960 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcb240097c0 tx=0x7fcb2401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c1e1048830a476 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:12.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.103+0000 7fcb31400640 1 -- 172.21.15.42:0/1076836994 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240b1070 con 0x7fcb3415e860 2024-05-28T14:12:12.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.103+0000 7fcb31400640 1 -- 172.21.15.42:0/1076836994 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb240ac640 con 0x7fcb3415e860 2024-05-28T14:12:12.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.103+0000 7fcb31400640 1 -- 172.21.15.42:0/1076836994 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240aca10 con 0x7fcb3415e860 2024-05-28T14:12:12.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.103+0000 7fcb39200640 1 -- 172.21.15.42:0/1076836994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb3415e860 msgr2=0x7fcb3415f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:12.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.103+0000 7fcb39200640 1 --2- 172.21.15.42:0/1076836994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb3415e860 0x7fcb3415f960 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcb240097c0 tx=0x7fcb2401c390 comp rx=0 tx=0).stop 2024-05-28T14:12:12.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.103+0000 7fcb39200640 1 -- 172.21.15.42:0/1076836994 shutdown_connections 2024-05-28T14:12:12.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.103+0000 7fcb39200640 1 --2- 172.21.15.42:0/1076836994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb3415e860 0x7fcb3415f960 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:12.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.103+0000 7fcb39200640 1 -- 172.21.15.42:0/1076836994 >> 172.21.15.42:0/1076836994 conn(0x7fcb340725b0 msgr2=0x7fcb340729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:12.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.103+0000 7fcb39200640 1 -- 172.21.15.42:0/1076836994 shutdown_connections 2024-05-28T14:12:12.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.103+0000 7fcb39200640 1 -- 172.21.15.42:0/1076836994 wait complete. 2024-05-28T14:12:12.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb39200640 1 Processor -- start 2024-05-28T14:12:12.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb39200640 1 -- start start 2024-05-28T14:12:12.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb39200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb3406ca00 0x7fcb34162be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:12.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb39200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb34070070 0x7fcb3415db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:12.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb39200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb3415e860 0x7fcb340ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:12.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb39200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3406d250 con 0x7fcb3406ca00 2024-05-28T14:12:12.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb39200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3406eb40 con 0x7fcb3415e860 2024-05-28T14:12:12.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb39200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3415ec40 con 0x7fcb34070070 2024-05-28T14:12:12.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb31e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb3406ca00 0x7fcb34162be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:12.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb31e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb3406ca00 0x7fcb34162be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50958/0 (socket says 172.21.15.42:50958) 2024-05-28T14:12:12.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb31e00640 1 -- 172.21.15.42:0/955836558 learned_addr learned my addr 172.21.15.42:0/955836558 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:12.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb32800640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb3415e860 0x7fcb340ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:12.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb29400640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb34070070 0x7fcb3415db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:12.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb31e00640 1 -- 172.21.15.42:0/955836558 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb34070070 msgr2=0x7fcb3415db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:12.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb31e00640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb34070070 0x7fcb3415db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb31e00640 1 -- 172.21.15.42:0/955836558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb3415e860 msgr2=0x7fcb340ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb31e00640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb3415e860 0x7fcb340ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb31e00640 1 -- 172.21.15.42:0/955836558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3415e060 con 0x7fcb3406ca00 2024-05-28T14:12:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb32800640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb3415e860 0x7fcb340ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb31e00640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb3406ca00 0x7fcb34162be0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fcb240097c0 tx=0x7fcb2401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb2aa00640 1 -- 172.21.15.42:0/955836558 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240bc020 con 0x7fcb3406ca00 2024-05-28T14:12:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb2aa00640 1 -- 172.21.15.42:0/955836558 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb240b2040 con 0x7fcb3406ca00 2024-05-28T14:12:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb2aa00640 1 -- 172.21.15.42:0/955836558 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240b1050 con 0x7fcb3406ca00 2024-05-28T14:12:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb39200640 1 -- 172.21.15.42:0/955836558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb34162140 con 0x7fcb3406ca00 2024-05-28T14:12:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.107+0000 7fcb39200640 1 -- 172.21.15.42:0/955836558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb340ceeb0 con 0x7fcb3406ca00 2024-05-28T14:12:12.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.111+0000 7fcb39200640 1 -- 172.21.15.42:0/955836558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb34068a90 con 0x7fcb3406ca00 2024-05-28T14:12:12.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.111+0000 7fcb2aa00640 1 -- 172.21.15.42:0/955836558 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7fcb240bf070 con 0x7fcb3406ca00 2024-05-28T14:12:12.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.111+0000 7fcb2aa00640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcb10069a30 0x7fcb1006bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:12.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.111+0000 7fcb2aa00640 1 -- 172.21.15.42:0/955836558 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fcb2411edf0 con 0x7fcb3406ca00 2024-05-28T14:12:12.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.111+0000 7fcb29400640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcb10069a30 0x7fcb1006bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:12.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.111+0000 7fcb29400640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcb10069a30 0x7fcb1006bef0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fcb34155af0 tx=0x7fcb1c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:12.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.115+0000 7fcb2aa00640 1 -- 172.21.15.42:0/955836558 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcb24015fb0 con 0x7fcb3406ca00 2024-05-28T14:12:12.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.335+0000 7fcb39200640 1 -- 172.21.15.42:0/955836558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcb34161a30 con 0x7fcb3406ca00 2024-05-28T14:12:12.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.339+0000 7fcb2aa00640 1 -- 172.21.15.42:0/955836558 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2111 (secure 0 0 0) 0x7fcb240f0eb0 con 0x7fcb3406ca00 2024-05-28T14:12:12.322 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:12.322 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:12:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:12:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:12:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:12:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:12:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:12:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:12:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:12:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:12:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:12:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:12:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:12.323 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:12:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:12:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:12:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:12:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 853, 2024-05-28T14:12:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:12:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:12:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:12:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:12:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:12:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 105, 2024-05-28T14:12:12.324 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:12:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:12:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:12:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:12:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:12:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:12:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:12:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:12:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:12:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:12:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:12.325 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:12:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:12:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 797, 2024-05-28T14:12:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 3222156092, 2024-05-28T14:12:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 7441768448, 2024-05-28T14:12:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 765652344832, 2024-05-28T14:12:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:12:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 682, 2024-05-28T14:12:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 15867, 2024-05-28T14:12:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:12:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 5 2024-05-28T14:12:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:12:12.326 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 65, 2024-05-28T14:12:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 7, 2024-05-28T14:12:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:12:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:12:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:12:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:12:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 7, 2024-05-28T14:12:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:12:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:12:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:12:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57056 2024-05-28T14:12:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:12.327 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:12:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:12:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:12:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:12:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:12:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:12:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:12:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:12:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:12:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:12:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 16, 2024-05-28T14:12:12.328 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:10:37.128981+0000", 2024-05-28T14:12:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:12:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:12:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "41758bd8-9a69-4e56-a945-2b876a2d9b46": { 2024-05-28T14:12:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 21.0 seconds before exiting (10s)\n [============================] ", 2024-05-28T14:12:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:12:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:12:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:12.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:12.329 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:12:12.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.339+0000 7fcb39200640 1 -- 172.21.15.42:0/955836558 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcb10069a30 msgr2=0x7fcb1006bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:12.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.339+0000 7fcb39200640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcb10069a30 0x7fcb1006bef0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fcb34155af0 tx=0x7fcb1c009040 comp rx=0 tx=0).stop 2024-05-28T14:12:12.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.339+0000 7fcb39200640 1 -- 172.21.15.42:0/955836558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb3406ca00 msgr2=0x7fcb34162be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:12.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.339+0000 7fcb39200640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb3406ca00 0x7fcb34162be0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fcb240097c0 tx=0x7fcb2401abd0 comp rx=0 tx=0).stop 2024-05-28T14:12:12.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.339+0000 7fcb39200640 1 -- 172.21.15.42:0/955836558 shutdown_connections 2024-05-28T14:12:12.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.339+0000 7fcb39200640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcb10069a30 0x7fcb1006bef0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:12.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.339+0000 7fcb39200640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb3415e860 0x7fcb340ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:12.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.339+0000 7fcb39200640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb34070070 0x7fcb3415db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:12.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.339+0000 7fcb39200640 1 --2- 172.21.15.42:0/955836558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb3406ca00 0x7fcb34162be0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:12.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.339+0000 7fcb39200640 1 -- 172.21.15.42:0/955836558 >> 172.21.15.42:0/955836558 conn(0x7fcb340725b0 msgr2=0x7fcb34156460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:12.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.339+0000 7fcb39200640 1 -- 172.21.15.42:0/955836558 shutdown_connections 2024-05-28T14:12:12.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:12.339+0000 7fcb39200640 1 -- 172.21.15.42:0/955836558 wait complete. 2024-05-28T14:12:17.344 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:12:17.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.511+0000 7f292f000640 1 Processor -- start 2024-05-28T14:12:17.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.511+0000 7f292f000640 1 -- start start 2024-05-28T14:12:17.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.515+0000 7f292f000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29281621c0 0x7f29280cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:17.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.515+0000 7f292f000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f29280d03e0 0x7f292815f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:17.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.515+0000 7f292f000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2928163120 con 0x7f29280d0d50 2024-05-28T14:12:17.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.515+0000 7f292f000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f29280d1e00 con 0x7f29280d03e0 2024-05-28T14:12:17.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.515+0000 7f292f000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2928162620 con 0x7f29281621c0 2024-05-28T14:12:17.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.515+0000 7f292e600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f29280d03e0 0x7f292815f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:17.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.515+0000 7f292e600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f29280d03e0 0x7f292815f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44794/0 (socket says 172.21.15.42:44794) 2024-05-28T14:12:17.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.515+0000 7f292e600640 1 -- 172.21.15.42:0/220186922 learned_addr learned my addr 172.21.15.42:0/220186922 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:17.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.515+0000 7f292d200640 1 --2- 172.21.15.42:0/220186922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29281621c0 0x7f29280cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:17.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.515+0000 7f292e600640 1 -- 172.21.15.42:0/220186922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29281621c0 msgr2=0x7f29280cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:17.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.515+0000 7f292e600640 1 --2- 172.21.15.42:0/220186922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29281621c0 0x7f29280cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:17.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.515+0000 7f292e600640 1 -- 172.21.15.42:0/220186922 >> v1:172.21.15.63:6789/0 conn(0x7f29280d0d50 legacy=0x7f29280cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:17.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.515+0000 7f292e600640 1 -- 172.21.15.42:0/220186922 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2928162880 con 0x7f29280d03e0 2024-05-28T14:12:17.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.515+0000 7f292d200640 1 --2- 172.21.15.42:0/220186922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29281621c0 0x7f29280cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:17.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f292e600640 1 --2- 172.21.15.42:0/220186922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29280d03e0 0x7f292815f270 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f2920009e80 tx=0x7f292001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4c39f463b4a4bf server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:17.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f2927e00640 1 -- 172.21.15.42:0/220186922 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29200a5730 con 0x7f29280d03e0 2024-05-28T14:12:17.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f2927e00640 1 -- 172.21.15.42:0/220186922 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29200a58d0 con 0x7f29280d03e0 2024-05-28T14:12:17.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f2927e00640 1 -- 172.21.15.42:0/220186922 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29200a5ca0 con 0x7f29280d03e0 2024-05-28T14:12:17.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f292f000640 1 -- 172.21.15.42:0/220186922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29280d03e0 msgr2=0x7f292815f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:17.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f292f000640 1 --2- 172.21.15.42:0/220186922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29280d03e0 0x7f292815f270 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f2920009e80 tx=0x7f292001c9a0 comp rx=0 tx=0).stop 2024-05-28T14:12:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f292f000640 1 -- 172.21.15.42:0/220186922 shutdown_connections 2024-05-28T14:12:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f292f000640 1 --2- 172.21.15.42:0/220186922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29280d03e0 0x7f292815f270 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f292f000640 1 --2- 172.21.15.42:0/220186922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29281621c0 0x7f29280cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f292f000640 1 -- 172.21.15.42:0/220186922 >> 172.21.15.42:0/220186922 conn(0x7f29280b3ec0 msgr2=0x7f29280b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f292f000640 1 -- 172.21.15.42:0/220186922 shutdown_connections 2024-05-28T14:12:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f292f000640 1 -- 172.21.15.42:0/220186922 wait complete. 2024-05-28T14:12:17.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f292f000640 1 Processor -- start 2024-05-28T14:12:17.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f292f000640 1 -- start start 2024-05-28T14:12:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.519+0000 7f292f000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29280d03e0 0x7f292806e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292f000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29280d0d50 0x7f2928150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292f000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29281621c0 0x7f29280c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292f000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2928154eb0 con 0x7f29280d03e0 2024-05-28T14:12:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292f000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29280d1130 con 0x7f29281621c0 2024-05-28T14:12:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292f000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2928155030 con 0x7f29280d0d50 2024-05-28T14:12:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29280d03e0 0x7f292806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29280d03e0 0x7f292806e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44804/0 (socket says 172.21.15.42:44804) 2024-05-28T14:12:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292dc00640 1 -- 172.21.15.42:0/731206913 learned_addr learned my addr 172.21.15.42:0/731206913 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:17.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292d200640 1 --2- 172.21.15.42:0/731206913 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29280d0d50 0x7f2928150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:17.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292e600640 1 --2- 172.21.15.42:0/731206913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29281621c0 0x7f29280c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:17.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292dc00640 1 -- 172.21.15.42:0/731206913 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29280d0d50 msgr2=0x7f2928150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:17.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292dc00640 1 --2- 172.21.15.42:0/731206913 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29280d0d50 0x7f2928150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:17.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292dc00640 1 -- 172.21.15.42:0/731206913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29281621c0 msgr2=0x7f29280c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:17.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292dc00640 1 --2- 172.21.15.42:0/731206913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29281621c0 0x7f29280c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:17.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292dc00640 1 -- 172.21.15.42:0/731206913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2928150fa0 con 0x7f29280d03e0 2024-05-28T14:12:17.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292dc00640 1 --2- 172.21.15.42:0/731206913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29280d03e0 0x7f292806e520 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f291c00c450 tx=0x7f291c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:17.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f2925600640 1 -- 172.21.15.42:0/731206913 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c012830 con 0x7f29280d03e0 2024-05-28T14:12:17.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f2925600640 1 -- 172.21.15.42:0/731206913 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f291c017070 con 0x7f29280d03e0 2024-05-28T14:12:17.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f2925600640 1 -- 172.21.15.42:0/731206913 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c012c30 con 0x7f29280d03e0 2024-05-28T14:12:17.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292f000640 1 -- 172.21.15.42:0/731206913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29280c76f0 con 0x7f29280d03e0 2024-05-28T14:12:17.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292f000640 1 -- 172.21.15.42:0/731206913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29280c7cb0 con 0x7f29280d03e0 2024-05-28T14:12:17.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f292f000640 1 -- 172.21.15.42:0/731206913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2928068a90 con 0x7f29280d03e0 2024-05-28T14:12:17.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.523+0000 7f2925600640 1 -- 172.21.15.42:0/731206913 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7f291c01c460 con 0x7f29280d03e0 2024-05-28T14:12:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.527+0000 7f2925600640 1 --2- 172.21.15.42:0/731206913 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2908069a30 0x7f290806bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.527+0000 7f2925600640 1 -- 172.21.15.42:0/731206913 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f291c085310 con 0x7f29280d03e0 2024-05-28T14:12:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.527+0000 7f292d200640 1 --2- 172.21.15.42:0/731206913 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2908069a30 0x7f290806bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.527+0000 7f292d200640 1 --2- 172.21.15.42:0/731206913 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2908069a30 0x7f290806bef0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f29280c8920 tx=0x7f2914008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:17.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.531+0000 7f2925600640 1 -- 172.21.15.42:0/731206913 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2928150fa0 con 0x7f29280d03e0 2024-05-28T14:12:17.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.751+0000 7f292f000640 1 -- 172.21.15.42:0/731206913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2928159290 con 0x7f29280d03e0 2024-05-28T14:12:17.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.751+0000 7f2925600640 1 -- 172.21.15.42:0/731206913 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2563 (secure 0 0 0) 0x7f291c056ce0 con 0x7f29280d03e0 2024-05-28T14:12:17.737 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:17.738 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:12:17.738 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:12:17.738 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:12:17.738 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:12:17.738 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:12:17.738 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:12:17.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:17.738 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:12:17.738 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:12:17.738 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:12:17.738 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:12:17.739 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:12:17.739 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:17.739 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:12:17.739 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:12:17.739 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:12:17.739 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:12:17.739 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:17.739 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 859, 2024-05-28T14:12:17.739 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:12:17.739 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:12:17.739 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:12:17.739 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:12:17.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:17.740 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:12:17.740 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 105, 2024-05-28T14:12:17.740 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:12:17.740 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:12:17.740 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:12:17.740 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:12:17.740 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:12:17.740 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:12:17.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:17.740 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:12:17.740 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:12:17.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:17.740 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:12:17.741 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:12:17.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:17.741 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:17.741 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:12:17.741 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:12:17.741 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 797, 2024-05-28T14:12:17.741 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 3222165262, 2024-05-28T14:12:17.741 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 7441784832, 2024-05-28T14:12:17.741 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 765652328448, 2024-05-28T14:12:17.741 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:12:17.741 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 597, 2024-05-28T14:12:17.741 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 8786, 2024-05-28T14:12:17.741 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:12:17.742 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 3 2024-05-28T14:12:17.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:17.742 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:12:17.742 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 65, 2024-05-28T14:12:17.742 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 7, 2024-05-28T14:12:17.742 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:12:17.742 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:12:17.742 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:12:17.742 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:12:17.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:17.743 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 7, 2024-05-28T14:12:17.743 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:12:17.743 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:12:17.743 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:12:17.743 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57056 2024-05-28T14:12:17.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:17.743 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:17.743 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:12:17.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:17.743 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:12:17.743 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:12:17.743 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:12:17.744 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:12:17.744 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:12:17.744 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:12:17.744 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:12:17.744 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:17.744 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:12:17.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:17.744 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:12:17.744 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 17, 2024-05-28T14:12:17.744 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:12:13.152658+0000", 2024-05-28T14:12:17.744 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-05-28T14:12:17.744 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-05-28T14:12:17.745 INFO:teuthology.orchestra.run.smithi042.stdout: "daemons": { 2024-05-28T14:12:17.745 INFO:teuthology.orchestra.run.smithi042.stdout: "summary": "", 2024-05-28T14:12:17.745 INFO:teuthology.orchestra.run.smithi042.stdout: "b": { 2024-05-28T14:12:17.745 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:12:17.745 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:12:17.745 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:12:17.745 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:12:17.745 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:12:17.745 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:12:17.745 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:17.745 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:17.745 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:17.745 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:17.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:17.746 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:12:17.746 INFO:teuthology.orchestra.run.smithi042.stdout: "41758bd8-9a69-4e56-a945-2b876a2d9b46": { 2024-05-28T14:12:17.746 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 17.0 seconds before exiting (14s)\n [============================] ", 2024-05-28T14:12:17.746 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:12:17.746 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:12:17.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:17.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:17.746 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:12:17.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.755+0000 7f292f000640 1 -- 172.21.15.42:0/731206913 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2908069a30 msgr2=0x7f290806bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:17.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.755+0000 7f292f000640 1 --2- 172.21.15.42:0/731206913 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2908069a30 0x7f290806bef0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f29280c8920 tx=0x7f2914008040 comp rx=0 tx=0).stop 2024-05-28T14:12:17.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.755+0000 7f292f000640 1 -- 172.21.15.42:0/731206913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29280d03e0 msgr2=0x7f292806e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:17.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.755+0000 7f292f000640 1 --2- 172.21.15.42:0/731206913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29280d03e0 0x7f292806e520 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f291c00c450 tx=0x7f291c00c910 comp rx=0 tx=0).stop 2024-05-28T14:12:17.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.755+0000 7f292f000640 1 -- 172.21.15.42:0/731206913 shutdown_connections 2024-05-28T14:12:17.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.755+0000 7f292f000640 1 --2- 172.21.15.42:0/731206913 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2908069a30 0x7f290806bef0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:17.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.755+0000 7f292f000640 1 --2- 172.21.15.42:0/731206913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29281621c0 0x7f29280c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:17.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.755+0000 7f292f000640 1 --2- 172.21.15.42:0/731206913 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29280d0d50 0x7f2928150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:17.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.755+0000 7f292f000640 1 --2- 172.21.15.42:0/731206913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29280d03e0 0x7f292806e520 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:17.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.755+0000 7f292f000640 1 -- 172.21.15.42:0/731206913 >> 172.21.15.42:0/731206913 conn(0x7f29280b3ec0 msgr2=0x7f2928151750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:17.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.755+0000 7f292f000640 1 -- 172.21.15.42:0/731206913 shutdown_connections 2024-05-28T14:12:17.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:17.755+0000 7f292f000640 1 -- 172.21.15.42:0/731206913 wait complete. 2024-05-28T14:12:20.723 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:12:20.728 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:12:22.760 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:12:22.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff643400640 1 Processor -- start 2024-05-28T14:12:22.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff643400640 1 -- start start 2024-05-28T14:12:22.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff643400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63c06e800 0x7ff63c0d0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:22.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff643400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff63c06cb00 0x7ff63c09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:22.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff643400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff63c0d1130 con 0x7ff63c0d0c70 2024-05-28T14:12:22.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff643400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff63c0d1410 con 0x7ff63c06cb00 2024-05-28T14:12:22.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff643400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff63c0d16f0 con 0x7ff63c06e800 2024-05-28T14:12:22.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff640a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63c06e800 0x7ff63c0d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:22.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff633e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff63c06cb00 0x7ff63c09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:22.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff640a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63c06e800 0x7ff63c0d0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:39040/0 (socket says 172.21.15.42:39040) 2024-05-28T14:12:22.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff640a00640 1 -- 172.21.15.42:0/3757056580 learned_addr learned my addr 172.21.15.42:0/3757056580 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:22.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff640a00640 1 -- 172.21.15.42:0/3757056580 >> v2:172.21.15.63:3300/0 conn(0x7ff63c06cb00 msgr2=0x7ff63c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:22.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff640a00640 1 --2- 172.21.15.42:0/3757056580 >> v2:172.21.15.63:3300/0 conn(0x7ff63c06cb00 0x7ff63c09aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:22.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff640a00640 1 -- 172.21.15.42:0/3757056580 >> v1:172.21.15.63:6789/0 conn(0x7ff63c0d0c70 legacy=0x7ff63c070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:22.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff640a00640 1 -- 172.21.15.42:0/3757056580 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c0d19d0 con 0x7ff63c06e800 2024-05-28T14:12:22.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff633e00640 1 --2- 172.21.15.42:0/3757056580 >> v2:172.21.15.63:3300/0 conn(0x7ff63c06cb00 0x7ff63c09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:12:22.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.927+0000 7ff63be00640 1 -- 172.21.15.42:0/3757056580 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1518397210 0 0) 0x7ff63c0d1130 con 0x7ff63c0d0c70 2024-05-28T14:12:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.931+0000 7ff640a00640 1 --2- 172.21.15.42:0/3757056580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63c06e800 0x7ff63c0d0730 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009870 tx=0x7ff62c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=100f0cac57d7cb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.931+0000 7ff63be00640 1 -- 172.21.15.42:0/3757056580 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0b1070 con 0x7ff63c06e800 2024-05-28T14:12:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.931+0000 7ff63be00640 1 -- 172.21.15.42:0/3757056580 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62c0a8050 con 0x7ff63c06e800 2024-05-28T14:12:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.931+0000 7ff63be00640 1 -- 172.21.15.42:0/3757056580 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0ac860 con 0x7ff63c06e800 2024-05-28T14:12:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.931+0000 7ff643400640 1 -- 172.21.15.42:0/3757056580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63c06e800 msgr2=0x7ff63c0d0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.931+0000 7ff643400640 1 --2- 172.21.15.42:0/3757056580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63c06e800 0x7ff63c0d0730 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009870 tx=0x7ff62c01c4d0 comp rx=0 tx=0).stop 2024-05-28T14:12:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.931+0000 7ff643400640 1 -- 172.21.15.42:0/3757056580 shutdown_connections 2024-05-28T14:12:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.931+0000 7ff643400640 1 --2- 172.21.15.42:0/3757056580 >> v2:172.21.15.63:3300/0 conn(0x7ff63c06cb00 0x7ff63c09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.931+0000 7ff643400640 1 --2- 172.21.15.42:0/3757056580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63c06e800 0x7ff63c0d0730 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.931+0000 7ff643400640 1 -- 172.21.15.42:0/3757056580 >> 172.21.15.42:0/3757056580 conn(0x7ff63c0b3fe0 msgr2=0x7ff63c0b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.931+0000 7ff643400640 1 -- 172.21.15.42:0/3757056580 shutdown_connections 2024-05-28T14:12:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.931+0000 7ff643400640 1 -- 172.21.15.42:0/3757056580 wait complete. 2024-05-28T14:12:22.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff643400640 1 Processor -- start 2024-05-28T14:12:22.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff643400640 1 -- start start 2024-05-28T14:12:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff643400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff63c06cb00 0x7ff63c0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:22.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff643400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63c06e800 0x7ff63c0c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:22.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff643400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff63c0d0c70 0x7ff63c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:22.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff643400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c15f540 con 0x7ff63c06cb00 2024-05-28T14:12:22.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff643400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c15f3c0 con 0x7ff63c0d0c70 2024-05-28T14:12:22.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff643400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff63c15f6c0 con 0x7ff63c06e800 2024-05-28T14:12:22.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff640a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff63c06cb00 0x7ff63c0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:22.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff640a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff63c06cb00 0x7ff63c0b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44818/0 (socket says 172.21.15.42:44818) 2024-05-28T14:12:22.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff640a00640 1 -- 172.21.15.42:0/2940020360 learned_addr learned my addr 172.21.15.42:0/2940020360 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:22.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff641400640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff63c0d0c70 0x7ff63c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:22.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff633e00640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63c06e800 0x7ff63c0c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:22.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff640a00640 1 -- 172.21.15.42:0/2940020360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63c06e800 msgr2=0x7ff63c0c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:22.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff640a00640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63c06e800 0x7ff63c0c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:22.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff640a00640 1 -- 172.21.15.42:0/2940020360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff63c0d0c70 msgr2=0x7ff63c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:22.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff640a00640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff63c0d0c70 0x7ff63c15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:22.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff640a00640 1 -- 172.21.15.42:0/2940020360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c09b620 con 0x7ff63c06cb00 2024-05-28T14:12:22.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff633e00640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63c06e800 0x7ff63c0c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:22.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff640a00640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff63c06cb00 0x7ff63c0b2ab0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7ff62c01ab80 tx=0x7ff62c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:22.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff639600640 1 -- 172.21.15.42:0/2940020360 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0b1050 con 0x7ff63c06cb00 2024-05-28T14:12:22.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff639600640 1 -- 172.21.15.42:0/2940020360 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62c0bc070 con 0x7ff63c06cb00 2024-05-28T14:12:22.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff639600640 1 -- 172.21.15.42:0/2940020360 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0b6840 con 0x7ff63c06cb00 2024-05-28T14:12:22.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff643400640 1 -- 172.21.15.42:0/2940020360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63c0b3cf0 con 0x7ff63c06cb00 2024-05-28T14:12:22.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.935+0000 7ff643400640 1 -- 172.21.15.42:0/2940020360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63c0b22d0 con 0x7ff63c06cb00 2024-05-28T14:12:22.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.939+0000 7ff643400640 1 -- 172.21.15.42:0/2940020360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63c068a90 con 0x7ff63c06cb00 2024-05-28T14:12:22.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.939+0000 7ff639600640 1 -- 172.21.15.42:0/2940020360 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7ff62c0b69e0 con 0x7ff63c06cb00 2024-05-28T14:12:22.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.939+0000 7ff639600640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff618069a00 0x7ff61806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:22.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.939+0000 7ff639600640 1 -- 172.21.15.42:0/2940020360 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7ff62c11f770 con 0x7ff63c06cb00 2024-05-28T14:12:22.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.939+0000 7ff633e00640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff618069a00 0x7ff61806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:22.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.939+0000 7ff633e00640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff618069a00 0x7ff61806bec0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff63c06cee0 tx=0x7ff624008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:22.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:22.943+0000 7ff639600640 1 -- 172.21.15.42:0/2940020360 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff62c016150 con 0x7ff63c06cb00 2024-05-28T14:12:23.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:23.171+0000 7ff643400640 1 -- 172.21.15.42:0/2940020360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff63c0b1e40 con 0x7ff63c06cb00 2024-05-28T14:12:23.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:23.171+0000 7ff639600640 1 -- 172.21.15.42:0/2940020360 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2562 (secure 0 0 0) 0x7ff62c0b6cd0 con 0x7ff63c06cb00 2024-05-28T14:12:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:23.157 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:12:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:12:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:12:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:12:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:12:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:12:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:12:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:12:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:12:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:12:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:12:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:12:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:12:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:12:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:12:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 864, 2024-05-28T14:12:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:12:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:12:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:12:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:12:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:12:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 105, 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:12:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:12:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 797, 2024-05-28T14:12:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 3222172691, 2024-05-28T14:12:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 7441801216, 2024-05-28T14:12:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 765652312064, 2024-05-28T14:12:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:12:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 85, 2024-05-28T14:12:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 4436, 2024-05-28T14:12:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:12:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 1 2024-05-28T14:12:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:12:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 65, 2024-05-28T14:12:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 7, 2024-05-28T14:12:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:12:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:12:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:12:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:12:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 7, 2024-05-28T14:12:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:12:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:12:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:12:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57056 2024-05-28T14:12:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:12:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:12:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:12:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:12:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:12:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:12:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:12:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:12:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:12:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:12:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 17, 2024-05-28T14:12:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:12:13.152658+0000", 2024-05-28T14:12:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-05-28T14:12:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-05-28T14:12:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "daemons": { 2024-05-28T14:12:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "summary": "", 2024-05-28T14:12:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "b": { 2024-05-28T14:12:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:12:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:12:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:12:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:12:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:12:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:12:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:12:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "41758bd8-9a69-4e56-a945-2b876a2d9b46": { 2024-05-28T14:12:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 11.0 seconds before exiting (20s)\n [============================] ", 2024-05-28T14:12:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:12:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:12:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:23.164 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:12:23.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:23.175+0000 7ff643400640 1 -- 172.21.15.42:0/2940020360 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff618069a00 msgr2=0x7ff61806bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:23.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:23.175+0000 7ff643400640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff618069a00 0x7ff61806bec0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff63c06cee0 tx=0x7ff624008040 comp rx=0 tx=0).stop 2024-05-28T14:12:23.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:23.175+0000 7ff643400640 1 -- 172.21.15.42:0/2940020360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff63c06cb00 msgr2=0x7ff63c0b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:23.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:23.175+0000 7ff643400640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff63c06cb00 0x7ff63c0b2ab0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7ff62c01ab80 tx=0x7ff62c019270 comp rx=0 tx=0).stop 2024-05-28T14:12:23.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:23.175+0000 7ff643400640 1 -- 172.21.15.42:0/2940020360 shutdown_connections 2024-05-28T14:12:23.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:23.175+0000 7ff643400640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff618069a00 0x7ff61806bec0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:23.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:23.175+0000 7ff643400640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff63c0d0c70 0x7ff63c15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:23.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:23.175+0000 7ff643400640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63c06e800 0x7ff63c0c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:23.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:23.175+0000 7ff643400640 1 --2- 172.21.15.42:0/2940020360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff63c06cb00 0x7ff63c0b2ab0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:23.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:23.175+0000 7ff643400640 1 -- 172.21.15.42:0/2940020360 >> 172.21.15.42:0/2940020360 conn(0x7ff63c0b3fe0 msgr2=0x7ff63c0af3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:23.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:23.175+0000 7ff643400640 1 -- 172.21.15.42:0/2940020360 shutdown_connections 2024-05-28T14:12:23.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:23.175+0000 7ff643400640 1 -- 172.21.15.42:0/2940020360 wait complete. 2024-05-28T14:12:28.178 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:12:28.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e4600640 1 Processor -- start 2024-05-28T14:12:28.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e4600640 1 -- start start 2024-05-28T14:12:28.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e4600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f98dc0af150 0x7f98dc15f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e4600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f98dc06ca00 0x7f98dc06ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e4600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98dc067bf0 con 0x7f98dc0af530 2024-05-28T14:12:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e4600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f98dc06d3b0 con 0x7f98dc0af150 2024-05-28T14:12:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e4600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f98dc162f20 con 0x7f98dc06ca00 2024-05-28T14:12:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e1c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f98dc0af150 0x7f98dc15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e1c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f98dc0af150 0x7f98dc15f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41660/0 (socket says 172.21.15.42:41660) 2024-05-28T14:12:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e1c00640 1 -- 172.21.15.42:0/571283746 learned_addr learned my addr 172.21.15.42:0/571283746 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e2600640 1 --2- 172.21.15.42:0/571283746 >> v2:172.21.15.63:3300/0 conn(0x7f98dc06ca00 0x7f98dc06ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e1c00640 1 -- 172.21.15.42:0/571283746 >> v2:172.21.15.63:3300/0 conn(0x7f98dc06ca00 msgr2=0x7f98dc06ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e1c00640 1 --2- 172.21.15.42:0/571283746 >> v2:172.21.15.63:3300/0 conn(0x7f98dc06ca00 0x7f98dc06ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e1c00640 1 -- 172.21.15.42:0/571283746 >> v1:172.21.15.63:6789/0 conn(0x7f98dc0af530 legacy=0x7f98dc1623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.347+0000 7f98e1c00640 1 -- 172.21.15.42:0/571283746 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98dc163200 con 0x7f98dc0af150 2024-05-28T14:12:28.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.351+0000 7f98e1c00640 1 --2- 172.21.15.42:0/571283746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98dc0af150 0x7f98dc15f910 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0b0250 tx=0x7f98d001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d9753f97ddc81b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:28.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.351+0000 7f98dbe00640 1 -- 172.21.15.42:0/571283746 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d00aa650 con 0x7f98dc0af150 2024-05-28T14:12:28.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.351+0000 7f98dbe00640 1 -- 172.21.15.42:0/571283746 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d00af070 con 0x7f98dc0af150 2024-05-28T14:12:28.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.351+0000 7f98dbe00640 1 -- 172.21.15.42:0/571283746 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d00aaa10 con 0x7f98dc0af150 2024-05-28T14:12:28.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.351+0000 7f98e4600640 1 -- 172.21.15.42:0/571283746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98dc0af150 msgr2=0x7f98dc15f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:28.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.351+0000 7f98e4600640 1 --2- 172.21.15.42:0/571283746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98dc0af150 0x7f98dc15f910 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0b0250 tx=0x7f98d001c4f0 comp rx=0 tx=0).stop 2024-05-28T14:12:28.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.351+0000 7f98e4600640 1 -- 172.21.15.42:0/571283746 shutdown_connections 2024-05-28T14:12:28.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.351+0000 7f98e4600640 1 --2- 172.21.15.42:0/571283746 >> v2:172.21.15.63:3300/0 conn(0x7f98dc06ca00 0x7f98dc06ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:28.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.351+0000 7f98e4600640 1 --2- 172.21.15.42:0/571283746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98dc0af150 0x7f98dc15f910 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:28.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.351+0000 7f98e4600640 1 -- 172.21.15.42:0/571283746 >> 172.21.15.42:0/571283746 conn(0x7f98dc0724e0 msgr2=0x7f98dc0728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:28.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.351+0000 7f98e4600640 1 -- 172.21.15.42:0/571283746 shutdown_connections 2024-05-28T14:12:28.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.351+0000 7f98e4600640 1 -- 172.21.15.42:0/571283746 wait complete. 2024-05-28T14:12:28.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e4600640 1 Processor -- start 2024-05-28T14:12:28.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e4600640 1 -- start start 2024-05-28T14:12:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e4600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98dc06ca00 0x7f98dc1405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e4600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98dc0af150 0x7f98dc0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e4600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98dc0af530 0x7f98dc0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e4600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc162c40 con 0x7f98dc0af530 2024-05-28T14:12:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e4600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc162ac0 con 0x7f98dc0af150 2024-05-28T14:12:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e4600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f98dc06ede0 con 0x7f98dc06ca00 2024-05-28T14:12:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e2600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98dc0af530 0x7f98dc0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e2600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98dc0af530 0x7f98dc0b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41674/0 (socket says 172.21.15.42:41674) 2024-05-28T14:12:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e2600640 1 -- 172.21.15.42:0/4227458921 learned_addr learned my addr 172.21.15.42:0/4227458921 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e1c00640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98dc06ca00 0x7f98dc1405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e1200640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98dc0af150 0x7f98dc0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e2600640 1 -- 172.21.15.42:0/4227458921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98dc06ca00 msgr2=0x7f98dc1405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e2600640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98dc06ca00 0x7f98dc1405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e2600640 1 -- 172.21.15.42:0/4227458921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98dc0af150 msgr2=0x7f98dc0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e2600640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98dc0af150 0x7f98dc0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e2600640 1 -- 172.21.15.42:0/4227458921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98dc140ae0 con 0x7f98dc0af530 2024-05-28T14:12:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e1c00640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98dc06ca00 0x7f98dc1405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e2600640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98dc0af530 0x7f98dc0b1af0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f98d4007e60 tx=0x7f98d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98d9600640 1 -- 172.21.15.42:0/4227458921 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4014420 con 0x7f98dc0af530 2024-05-28T14:12:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98d9600640 1 -- 172.21.15.42:0/4227458921 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d4019070 con 0x7f98dc0af530 2024-05-28T14:12:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98d9600640 1 -- 172.21.15.42:0/4227458921 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4014820 con 0x7f98dc0af530 2024-05-28T14:12:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e4600640 1 -- 172.21.15.42:0/4227458921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98dc1519f0 con 0x7f98dc0af530 2024-05-28T14:12:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e4600640 1 -- 172.21.15.42:0/4227458921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98dc151fb0 con 0x7f98dc0af530 2024-05-28T14:12:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.355+0000 7f98e4600640 1 -- 172.21.15.42:0/4227458921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98dc068a90 con 0x7f98dc0af530 2024-05-28T14:12:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.359+0000 7f98d9600640 1 -- 172.21.15.42:0/4227458921 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7f98d40149c0 con 0x7f98dc0af530 2024-05-28T14:12:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.359+0000 7f98d9600640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98bc069a30 0x7f98bc06bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:28.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.359+0000 7f98d9600640 1 -- 172.21.15.42:0/4227458921 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f98d4085fe0 con 0x7f98dc0af530 2024-05-28T14:12:28.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.359+0000 7f98e1c00640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98bc069a30 0x7f98bc06bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:28.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.359+0000 7f98e1c00640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98bc069a30 0x7f98bc06bef0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f98d00ae000 tx=0x7f98d00a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:28.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.363+0000 7f98d9600640 1 -- 172.21.15.42:0/4227458921 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f98d4057f70 con 0x7f98dc0af530 2024-05-28T14:12:28.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.587+0000 7f98e4600640 1 -- 172.21.15.42:0/4227458921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f98dc06f600 con 0x7f98dc0af530 2024-05-28T14:12:28.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.587+0000 7f98d9600640 1 -- 172.21.15.42:0/4227458921 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2077 (secure 0 0 0) 0x7f98d4057870 con 0x7f98dc0af530 2024-05-28T14:12:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:28.573 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:12:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:12:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:12:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:12:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:12:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:12:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:12:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:12:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:12:28.573 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:12:28.574 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:12:28.574 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:28.574 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:12:28.574 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:12:28.574 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:12:28.574 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:12:28.574 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:28.574 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 869, 2024-05-28T14:12:28.574 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:12:28.574 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:12:28.574 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:12:28.574 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:12:28.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:28.575 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:12:28.575 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 105, 2024-05-28T14:12:28.575 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:12:28.575 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:12:28.575 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:12:28.575 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:12:28.575 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:12:28.575 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:12:28.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:28.575 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:12:28.575 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:12:28.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:28.575 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:12:28.575 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:12:28.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:28.576 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:28.576 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:12:28.576 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:12:28.576 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 797, 2024-05-28T14:12:28.576 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 3222172691, 2024-05-28T14:12:28.576 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 7441801216, 2024-05-28T14:12:28.576 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 765652312064, 2024-05-28T14:12:28.576 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:12:28.576 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 767, 2024-05-28T14:12:28.576 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:12:28.576 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:12:28.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:28.576 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 65, 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 7, 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 7, 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57056 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:28.577 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:12:28.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:28.578 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:12:28.578 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:12:28.578 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:12:28.578 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:12:28.578 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:12:28.578 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:12:28.578 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:12:28.578 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:28.578 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:12:28.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:28.578 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:12:28.578 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:12:28.578 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:12:23.155415+0000", 2024-05-28T14:12:28.579 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:12:28.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:28.579 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:12:28.579 INFO:teuthology.orchestra.run.smithi042.stdout: "41758bd8-9a69-4e56-a945-2b876a2d9b46": { 2024-05-28T14:12:28.579 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 5.0 seconds before exiting (26s)\n [============================] ", 2024-05-28T14:12:28.579 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:12:28.579 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:12:28.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:28.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:28.579 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:12:28.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.591+0000 7f98e4600640 1 -- 172.21.15.42:0/4227458921 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98bc069a30 msgr2=0x7f98bc06bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:28.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.591+0000 7f98e4600640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98bc069a30 0x7f98bc06bef0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f98d00ae000 tx=0x7f98d00a7000 comp rx=0 tx=0).stop 2024-05-28T14:12:28.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.591+0000 7f98e4600640 1 -- 172.21.15.42:0/4227458921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98dc0af530 msgr2=0x7f98dc0b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:28.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.591+0000 7f98e4600640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98dc0af530 0x7f98dc0b1af0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f98d4007e60 tx=0x7f98d400e4b0 comp rx=0 tx=0).stop 2024-05-28T14:12:28.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.591+0000 7f98e4600640 1 -- 172.21.15.42:0/4227458921 shutdown_connections 2024-05-28T14:12:28.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.591+0000 7f98e4600640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98bc069a30 0x7f98bc06bef0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:28.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.591+0000 7f98e4600640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98dc0af530 0x7f98dc0b1af0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:28.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.591+0000 7f98e4600640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98dc0af150 0x7f98dc0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:28.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.591+0000 7f98e4600640 1 --2- 172.21.15.42:0/4227458921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98dc06ca00 0x7f98dc1405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:28.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.591+0000 7f98e4600640 1 -- 172.21.15.42:0/4227458921 >> 172.21.15.42:0/4227458921 conn(0x7f98dc0724e0 msgr2=0x7f98dc15ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:28.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.591+0000 7f98e4600640 1 -- 172.21.15.42:0/4227458921 shutdown_connections 2024-05-28T14:12:28.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:28.591+0000 7f98e4600640 1 -- 172.21.15.42:0/4227458921 wait complete. 2024-05-28T14:12:33.596 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:12:33.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.763+0000 7f0936200640 1 Processor -- start 2024-05-28T14:12:33.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.763+0000 7f0936200640 1 -- start start 2024-05-28T14:12:33.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f0936200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f09300b3870 0x7f093015f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:33.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f0936200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09300cce40 con 0x7f093015b6d0 2024-05-28T14:12:33.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f0936200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f093015f820 con 0x7f09300b2760 2024-05-28T14:12:33.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f0936200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f09300ce4b0 con 0x7f09300b3870 2024-05-28T14:12:33.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f0927e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f093015b6d0 0x7f09300cec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:35478/0 (socket says 172.21.15.42:35478) 2024-05-28T14:12:33.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f0927e00640 1 -- 172.21.15.42:0/1809558321 learned_addr learned my addr 172.21.15.42:0/1809558321 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:33.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f092fe00640 1 --2- 172.21.15.42:0/1809558321 >> v2:172.21.15.63:3300/0 conn(0x7f09300b3870 0x7f093015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:33.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f092ea00640 1 -- 172.21.15.42:0/1809558321 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1167652650 0 0) 0x7f09300cce40 con 0x7f093015b6d0 2024-05-28T14:12:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f092ea00640 1 -- 172.21.15.42:0/1809558321 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0914003530 con 0x7f093015b6d0 2024-05-28T14:12:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f092ea00640 1 -- 172.21.15.42:0/1809558321 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 241953866 0 0) 0x7f093015f820 con 0x7f09300b2760 2024-05-28T14:12:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f092ea00640 1 -- 172.21.15.42:0/1809558321 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09300cce40 con 0x7f09300b2760 2024-05-28T14:12:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f092ea00640 1 -- 172.21.15.42:0/1809558321 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3559641348 0 0) 0x7f0914003530 con 0x7f093015b6d0 2024-05-28T14:12:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f092fe00640 1 -- 172.21.15.42:0/1809558321 >> v1:172.21.15.63:6789/0 conn(0x7f09300b2760 legacy=0x7f093015afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f092fe00640 1 -- 172.21.15.42:0/1809558321 >> v1:172.21.15.42:6789/0 conn(0x7f093015b6d0 legacy=0x7f09300cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.767+0000 7f092fe00640 1 -- 172.21.15.42:0/1809558321 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093015fb00 con 0x7f09300b3870 2024-05-28T14:12:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f092fe00640 1 --2- 172.21.15.42:0/1809558321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09300b3870 0x7f093015f120 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0928009d90 tx=0x7f092801c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b792f7e74559e8e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f092ea00640 1 -- 172.21.15.42:0/1809558321 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09280b2070 con 0x7f09300b3870 2024-05-28T14:12:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f092ea00640 1 -- 172.21.15.42:0/1809558321 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09280a5550 con 0x7f09300b3870 2024-05-28T14:12:33.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f092ea00640 1 -- 172.21.15.42:0/1809558321 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918001680 con 0x7f09300b3870 2024-05-28T14:12:33.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 -- 172.21.15.42:0/1809558321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09300b3870 msgr2=0x7f093015f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:33.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 --2- 172.21.15.42:0/1809558321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09300b3870 0x7f093015f120 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0928009d90 tx=0x7f092801c800 comp rx=0 tx=0).stop 2024-05-28T14:12:33.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 -- 172.21.15.42:0/1809558321 shutdown_connections 2024-05-28T14:12:33.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 --2- 172.21.15.42:0/1809558321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09300b3870 0x7f093015f120 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:33.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 -- 172.21.15.42:0/1809558321 >> 172.21.15.42:0/1809558321 conn(0x7f09300d07c0 msgr2=0x7f09300d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:33.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 -- 172.21.15.42:0/1809558321 shutdown_connections 2024-05-28T14:12:33.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 -- 172.21.15.42:0/1809558321 wait complete. 2024-05-28T14:12:33.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 Processor -- start 2024-05-28T14:12:33.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 -- start start 2024-05-28T14:12:33.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09300b2760 0x7f09301510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:33.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09300b3870 0x7f0930136a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:33.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f093015b6d0 0x7f09301591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:33.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f09300cff30 con 0x7f093015b6d0 2024-05-28T14:12:33.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f09300cfdb0 con 0x7f09300b3870 2024-05-28T14:12:33.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.771+0000 7f0936200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f09300ce770 con 0x7f09300b2760 2024-05-28T14:12:33.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f092fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f093015b6d0 0x7f09301591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:33.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f092fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f093015b6d0 0x7f09301591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41688/0 (socket says 172.21.15.42:41688) 2024-05-28T14:12:33.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f092f400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09300b2760 0x7f09301510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:33.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f092fe00640 1 -- 172.21.15.42:0/2765316735 learned_addr learned my addr 172.21.15.42:0/2765316735 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:33.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f0927e00640 1 --2- 172.21.15.42:0/2765316735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09300b3870 0x7f0930136a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:33.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f092fe00640 1 -- 172.21.15.42:0/2765316735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09300b2760 msgr2=0x7f09301510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:33.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f092fe00640 1 --2- 172.21.15.42:0/2765316735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09300b2760 0x7f09301510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:33.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f092fe00640 1 -- 172.21.15.42:0/2765316735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09300b3870 msgr2=0x7f0930136a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:33.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f092fe00640 1 --2- 172.21.15.42:0/2765316735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09300b3870 0x7f0930136a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:33.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f092fe00640 1 -- 172.21.15.42:0/2765316735 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093014a6e0 con 0x7f093015b6d0 2024-05-28T14:12:33.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f092fe00640 1 --2- 172.21.15.42:0/2765316735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f093015b6d0 0x7f09301591b0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f09280b0040 tx=0x7f09280195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:33.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f0927400640 1 -- 172.21.15.42:0/2765316735 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09280b2050 con 0x7f093015b6d0 2024-05-28T14:12:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f0927400640 1 -- 172.21.15.42:0/2765316735 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09280be070 con 0x7f093015b6d0 2024-05-28T14:12:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f0927400640 1 -- 172.21.15.42:0/2765316735 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928003e60 con 0x7f093015b6d0 2024-05-28T14:12:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f0936200640 1 -- 172.21.15.42:0/2765316735 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0930139900 con 0x7f093015b6d0 2024-05-28T14:12:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f0936200640 1 -- 172.21.15.42:0/2765316735 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0930139ec0 con 0x7f093015b6d0 2024-05-28T14:12:33.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f0936200640 1 -- 172.21.15.42:0/2765316735 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0930068a90 con 0x7f093015b6d0 2024-05-28T14:12:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.775+0000 7f0927400640 1 -- 172.21.15.42:0/2765316735 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7f0928004240 con 0x7f093015b6d0 2024-05-28T14:12:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.779+0000 7f0927400640 1 --2- 172.21.15.42:0/2765316735 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f090c069a00 0x7f090c06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.779+0000 7f0927400640 1 -- 172.21.15.42:0/2765316735 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f0928120680 con 0x7f093015b6d0 2024-05-28T14:12:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.779+0000 7f092f400640 1 --2- 172.21.15.42:0/2765316735 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f090c069a00 0x7f090c06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:33.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.779+0000 7f092f400640 1 --2- 172.21.15.42:0/2765316735 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f090c069a00 0x7f090c06bec0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f09200011f0 tx=0x7f0920009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:33.783+0000 7f0927400640 1 -- 172.21.15.42:0/2765316735 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0928016480 con 0x7f093015b6d0 2024-05-28T14:12:33.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:34.007+0000 7f0936200640 1 -- 172.21.15.42:0/2765316735 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0930136fc0 con 0x7f093015b6d0 2024-05-28T14:12:33.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:34.007+0000 7f0927400640 1 -- 172.21.15.42:0/2765316735 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2077 (secure 0 0 0) 0x7f09280f2050 con 0x7f093015b6d0 2024-05-28T14:12:33.991 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:33.991 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:12:33.992 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:12:33.992 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:12:33.992 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:12:33.992 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:12:33.992 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:12:33.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:33.992 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:12:33.992 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:12:33.992 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:12:33.992 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:12:33.992 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:12:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:12:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:12:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:12:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:12:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 875, 2024-05-28T14:12:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:12:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:12:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:12:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:12:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:12:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 105, 2024-05-28T14:12:33.994 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:12:33.994 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:12:33.994 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:12:33.994 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:12:33.994 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:12:33.994 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:12:33.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:33.994 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:12:33.994 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:12:33.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:33.995 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:12:33.995 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:12:33.995 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:33.995 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:33.995 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:12:33.995 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:12:33.995 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 797, 2024-05-28T14:12:33.995 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 3222172691, 2024-05-28T14:12:33.995 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 7441801216, 2024-05-28T14:12:33.995 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 765652312064, 2024-05-28T14:12:33.995 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:12:33.995 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 767, 2024-05-28T14:12:33.996 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:12:33.996 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:12:33.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:33.996 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:12:33.996 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 65, 2024-05-28T14:12:33.996 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 7, 2024-05-28T14:12:33.996 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:12:33.996 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:12:33.996 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:12:33.996 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:12:33.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:33.996 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 7, 2024-05-28T14:12:33.996 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:12:33.996 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57056 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:33.997 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:12:33.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:33.998 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:12:33.998 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:12:33.998 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:12:23.155415+0000", 2024-05-28T14:12:33.998 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:12:33.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:33.998 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:12:33.998 INFO:teuthology.orchestra.run.smithi042.stdout: "41758bd8-9a69-4e56-a945-2b876a2d9b46": { 2024-05-28T14:12:33.998 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 1.0 seconds before exiting (30s)\n [============================] ", 2024-05-28T14:12:33.998 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:12:33.998 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:12:33.998 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:33.998 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:33.998 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:12:33.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:34.007+0000 7f0936200640 1 -- 172.21.15.42:0/2765316735 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f090c069a00 msgr2=0x7f090c06bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:33.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:34.007+0000 7f0936200640 1 --2- 172.21.15.42:0/2765316735 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f090c069a00 0x7f090c06bec0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f09200011f0 tx=0x7f0920009040 comp rx=0 tx=0).stop 2024-05-28T14:12:33.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:34.007+0000 7f0936200640 1 -- 172.21.15.42:0/2765316735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f093015b6d0 msgr2=0x7f09301591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:33.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:34.007+0000 7f0936200640 1 --2- 172.21.15.42:0/2765316735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f093015b6d0 0x7f09301591b0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f09280b0040 tx=0x7f09280195a0 comp rx=0 tx=0).stop 2024-05-28T14:12:34.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:34.007+0000 7f0936200640 1 -- 172.21.15.42:0/2765316735 shutdown_connections 2024-05-28T14:12:34.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:34.007+0000 7f0936200640 1 --2- 172.21.15.42:0/2765316735 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f090c069a00 0x7f090c06bec0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:34.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:34.007+0000 7f0936200640 1 --2- 172.21.15.42:0/2765316735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f093015b6d0 0x7f09301591b0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:34.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:34.007+0000 7f0936200640 1 --2- 172.21.15.42:0/2765316735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09300b3870 0x7f0930136a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:34.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:34.007+0000 7f0936200640 1 --2- 172.21.15.42:0/2765316735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09300b2760 0x7f09301510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:34.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:34.007+0000 7f0936200640 1 -- 172.21.15.42:0/2765316735 >> 172.21.15.42:0/2765316735 conn(0x7f09300d07c0 msgr2=0x7f093015ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:34.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:34.007+0000 7f0936200640 1 -- 172.21.15.42:0/2765316735 shutdown_connections 2024-05-28T14:12:34.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:34.007+0000 7f0936200640 1 -- 172.21.15.42:0/2765316735 wait complete. 2024-05-28T14:12:39.016 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:12:39.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd5c200640 1 Processor -- start 2024-05-28T14:12:39.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd5c200640 1 -- start start 2024-05-28T14:12:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd5c200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd5406b9f0 0x7efd540d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd5c200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efd540cc230 0x7efd5406bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd5c200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd540cd800 con 0x7efd540ce580 2024-05-28T14:12:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd5c200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efd540cdaa0 con 0x7efd540cc230 2024-05-28T14:12:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd5c200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd540cdd80 con 0x7efd5406b9f0 2024-05-28T14:12:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd58e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7efd540ce580 0x7efd540cbb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39828/0 (socket says 172.21.15.42:39828) 2024-05-28T14:12:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd58e00640 1 -- 172.21.15.42:0/1848031346 learned_addr learned my addr 172.21.15.42:0/1848031346 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd5a200640 1 --2- 172.21.15.42:0/1848031346 >> v2:172.21.15.63:3300/0 conn(0x7efd540cc230 0x7efd5406bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd59800640 1 --2- 172.21.15.42:0/1848031346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd5406b9f0 0x7efd540d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd53e00640 1 -- 172.21.15.42:0/1848031346 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 556275950 0 0) 0x7efd540cd800 con 0x7efd540ce580 2024-05-28T14:12:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd53e00640 1 -- 172.21.15.42:0/1848031346 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd3c003530 con 0x7efd540ce580 2024-05-28T14:12:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd5a200640 1 -- 172.21.15.42:0/1848031346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd5406b9f0 msgr2=0x7efd540d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd5a200640 1 --2- 172.21.15.42:0/1848031346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd5406b9f0 0x7efd540d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd5a200640 1 -- 172.21.15.42:0/1848031346 >> v1:172.21.15.42:6789/0 conn(0x7efd540ce580 legacy=0x7efd540cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.183+0000 7efd5a200640 1 -- 172.21.15.42:0/1848031346 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd540ce060 con 0x7efd540cc230 2024-05-28T14:12:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.187+0000 7efd5a200640 1 --2- 172.21.15.42:0/1848031346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd540cc230 0x7efd5406bdd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009d90 tx=0x7efd4c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=469112d9a2960e5a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.187+0000 7efd53e00640 1 -- 172.21.15.42:0/1848031346 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c0b2070 con 0x7efd540cc230 2024-05-28T14:12:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.187+0000 7efd53e00640 1 -- 172.21.15.42:0/1848031346 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd4c0a5940 con 0x7efd540cc230 2024-05-28T14:12:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.187+0000 7efd53e00640 1 -- 172.21.15.42:0/1848031346 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c0a5d10 con 0x7efd540cc230 2024-05-28T14:12:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.187+0000 7efd5c200640 1 -- 172.21.15.42:0/1848031346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd540cc230 msgr2=0x7efd5406bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.187+0000 7efd5c200640 1 --2- 172.21.15.42:0/1848031346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd540cc230 0x7efd5406bdd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009d90 tx=0x7efd4c01c970 comp rx=0 tx=0).stop 2024-05-28T14:12:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.187+0000 7efd5c200640 1 -- 172.21.15.42:0/1848031346 shutdown_connections 2024-05-28T14:12:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.187+0000 7efd5c200640 1 --2- 172.21.15.42:0/1848031346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd540cc230 0x7efd5406bdd0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.187+0000 7efd5c200640 1 --2- 172.21.15.42:0/1848031346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd5406b9f0 0x7efd540d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.187+0000 7efd5c200640 1 -- 172.21.15.42:0/1848031346 >> 172.21.15.42:0/1848031346 conn(0x7efd5409aa50 msgr2=0x7efd540b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.187+0000 7efd5c200640 1 -- 172.21.15.42:0/1848031346 shutdown_connections 2024-05-28T14:12:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.187+0000 7efd5c200640 1 -- 172.21.15.42:0/1848031346 wait complete. 2024-05-28T14:12:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd5c200640 1 Processor -- start 2024-05-28T14:12:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd5c200640 1 -- start start 2024-05-28T14:12:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd5c200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5406b9f0 0x7efd540c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd5c200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd540ce580 0x7efd540b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd5c200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd540b6cd0 0x7efd5415d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd5c200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efd540d0f40 con 0x7efd5406b9f0 2024-05-28T14:12:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd5c200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd540d0dc0 con 0x7efd540b6cd0 2024-05-28T14:12:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd5c200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd5415e960 con 0x7efd540ce580 2024-05-28T14:12:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd59800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5406b9f0 0x7efd540c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd59800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5406b9f0 0x7efd540c8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33248/0 (socket says 172.21.15.42:33248) 2024-05-28T14:12:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd59800640 1 -- 172.21.15.42:0/2286635679 learned_addr learned my addr 172.21.15.42:0/2286635679 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd58e00640 1 --2- 172.21.15.42:0/2286635679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd540ce580 0x7efd540b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd5a200640 1 --2- 172.21.15.42:0/2286635679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd540b6cd0 0x7efd5415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd59800640 1 -- 172.21.15.42:0/2286635679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd540ce580 msgr2=0x7efd540b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd59800640 1 --2- 172.21.15.42:0/2286635679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd540ce580 0x7efd540b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd59800640 1 -- 172.21.15.42:0/2286635679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd540b6cd0 msgr2=0x7efd5415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd59800640 1 --2- 172.21.15.42:0/2286635679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd540b6cd0 0x7efd5415d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd59800640 1 -- 172.21.15.42:0/2286635679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5415dde0 con 0x7efd5406b9f0 2024-05-28T14:12:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd59800640 1 --2- 172.21.15.42:0/2286635679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5406b9f0 0x7efd540c8970 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7efd4800e930 tx=0x7efd4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd51600640 1 -- 172.21.15.42:0/2286635679 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48017070 con 0x7efd5406b9f0 2024-05-28T14:12:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd51600640 1 -- 172.21.15.42:0/2286635679 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd4800ca00 con 0x7efd5406b9f0 2024-05-28T14:12:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd51600640 1 -- 172.21.15.42:0/2286635679 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4800cdb0 con 0x7efd5406b9f0 2024-05-28T14:12:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd5c200640 1 -- 172.21.15.42:0/2286635679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd54158550 con 0x7efd5406b9f0 2024-05-28T14:12:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.191+0000 7efd5c200640 1 -- 172.21.15.42:0/2286635679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5415dad0 con 0x7efd5406b9f0 2024-05-28T14:12:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.195+0000 7efd5c200640 1 -- 172.21.15.42:0/2286635679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd540b70e0 con 0x7efd5406b9f0 2024-05-28T14:12:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.195+0000 7efd51600640 1 -- 172.21.15.42:0/2286635679 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7efd4801a070 con 0x7efd5406b9f0 2024-05-28T14:12:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.195+0000 7efd51600640 1 --2- 172.21.15.42:0/2286635679 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efd34069a00 0x7efd3406bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.195+0000 7efd51600640 1 -- 172.21.15.42:0/2286635679 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7efd48085aa0 con 0x7efd5406b9f0 2024-05-28T14:12:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.195+0000 7efd58e00640 1 --2- 172.21.15.42:0/2286635679 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efd34069a00 0x7efd3406bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.195+0000 7efd58e00640 1 --2- 172.21.15.42:0/2286635679 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efd34069a00 0x7efd3406bec0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7efd40007fd0 tx=0x7efd40009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.199+0000 7efd51600640 1 -- 172.21.15.42:0/2286635679 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efd480574b0 con 0x7efd5406b9f0 2024-05-28T14:12:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.423+0000 7efd5c200640 1 -- 172.21.15.42:0/2286635679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7efd5406f1a0 con 0x7efd5406b9f0 2024-05-28T14:12:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.423+0000 7efd51600640 1 -- 172.21.15.42:0/2286635679 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1730 (secure 0 0 0) 0x7efd4805b7b0 con 0x7efd5406b9f0 2024-05-28T14:12:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:39.469 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:12:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:12:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:12:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:12:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:12:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:12:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:12:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:12:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:12:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:12:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:12:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:12:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:12:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:12:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:12:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 880, 2024-05-28T14:12:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:12:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:12:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:12:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:12:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:12:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 105, 2024-05-28T14:12:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:12:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:12:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:12:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:12:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:12:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:12:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:12:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:12:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 797, 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 3222172691, 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 7441801216, 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 765652312064, 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 65, 2024-05-28T14:12:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 7, 2024-05-28T14:12:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:12:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:12:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:12:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:12:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 7, 2024-05-28T14:12:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:12:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:12:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:12:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57056 2024-05-28T14:12:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:12:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:12:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:12:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:12:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:12:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:12:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:12:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:12:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:12:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:12:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:12:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:12:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:12:23.155415+0000", 2024-05-28T14:12:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:12:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": {} 2024-05-28T14:12:39.475 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:12:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.427+0000 7efd5c200640 1 -- 172.21.15.42:0/2286635679 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efd34069a00 msgr2=0x7efd3406bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.427+0000 7efd5c200640 1 --2- 172.21.15.42:0/2286635679 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efd34069a00 0x7efd3406bec0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7efd40007fd0 tx=0x7efd40009040 comp rx=0 tx=0).stop 2024-05-28T14:12:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.427+0000 7efd5c200640 1 -- 172.21.15.42:0/2286635679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5406b9f0 msgr2=0x7efd540c8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.427+0000 7efd5c200640 1 --2- 172.21.15.42:0/2286635679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5406b9f0 0x7efd540c8970 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7efd4800e930 tx=0x7efd4800edf0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.427+0000 7efd5c200640 1 -- 172.21.15.42:0/2286635679 shutdown_connections 2024-05-28T14:12:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.427+0000 7efd5c200640 1 --2- 172.21.15.42:0/2286635679 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efd34069a00 0x7efd3406bec0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.427+0000 7efd5c200640 1 --2- 172.21.15.42:0/2286635679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd540b6cd0 0x7efd5415d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.427+0000 7efd5c200640 1 --2- 172.21.15.42:0/2286635679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd540ce580 0x7efd540b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.427+0000 7efd5c200640 1 --2- 172.21.15.42:0/2286635679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5406b9f0 0x7efd540c8970 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.427+0000 7efd5c200640 1 -- 172.21.15.42:0/2286635679 >> 172.21.15.42:0/2286635679 conn(0x7efd5409aa50 msgr2=0x7efd540b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.427+0000 7efd5c200640 1 -- 172.21.15.42:0/2286635679 shutdown_connections 2024-05-28T14:12:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.427+0000 7efd5c200640 1 -- 172.21.15.42:0/2286635679 wait complete. 2024-05-28T14:12:39.477 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-28T14:12:39.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.635+0000 7ff031000640 1 Processor -- start 2024-05-28T14:12:39.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.635+0000 7ff031000640 1 -- start start 2024-05-28T14:12:39.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.639+0000 7ff031000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02c15e620 0x7ff02c06ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:39.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.639+0000 7ff031000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff02c06f2e0 con 0x7ff02c0b2c10 2024-05-28T14:12:39.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.639+0000 7ff031000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff02c06f5c0 con 0x7ff02c0b4660 2024-05-28T14:12:39.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.639+0000 7ff031000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff02c0ad8c0 con 0x7ff02c15e620 2024-05-28T14:12:39.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.639+0000 7ff02a800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff02c0b2c10 0x7ff02c0adb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39836/0 (socket says 172.21.15.42:39836) 2024-05-28T14:12:39.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.639+0000 7ff02a800640 1 -- 172.21.15.42:0/686020748 learned_addr learned my addr 172.21.15.42:0/686020748 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:39.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.639+0000 7ff029e00640 1 --2- 172.21.15.42:0/686020748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02c15e620 0x7ff02c06ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:39.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.639+0000 7ff029400640 1 -- 172.21.15.42:0/686020748 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2547305572 0 0) 0x7ff02c06f2e0 con 0x7ff02c0b2c10 2024-05-28T14:12:39.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.639+0000 7ff029400640 1 -- 172.21.15.42:0/686020748 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff010003530 con 0x7ff02c0b2c10 2024-05-28T14:12:39.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.639+0000 7ff029400640 1 -- 172.21.15.42:0/686020748 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2786962505 0 0) 0x7ff02c06f5c0 con 0x7ff02c0b4660 2024-05-28T14:12:39.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.639+0000 7ff029400640 1 -- 172.21.15.42:0/686020748 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff02c06f2e0 con 0x7ff02c0b4660 2024-05-28T14:12:39.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.639+0000 7ff029e00640 1 -- 172.21.15.42:0/686020748 >> v1:172.21.15.63:6789/0 conn(0x7ff02c0b4660 legacy=0x7ff02c0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:39.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.639+0000 7ff029e00640 1 -- 172.21.15.42:0/686020748 >> v1:172.21.15.42:6789/0 conn(0x7ff02c0b2c10 legacy=0x7ff02c0adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:39.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.639+0000 7ff029e00640 1 -- 172.21.15.42:0/686020748 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c06f8a0 con 0x7ff02c15e620 2024-05-28T14:12:39.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff029e00640 1 --2- 172.21.15.42:0/686020748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02c15e620 0x7ff02c06ec00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0ada40 tx=0x7ff01c01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49e8716ecf3da7d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:39.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff029400640 1 -- 172.21.15.42:0/686020748 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c0b1070 con 0x7ff02c15e620 2024-05-28T14:12:39.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff029400640 1 -- 172.21.15.42:0/686020748 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff01c0ac630 con 0x7ff02c15e620 2024-05-28T14:12:39.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff029400640 1 -- 172.21.15.42:0/686020748 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c0a8050 con 0x7ff02c15e620 2024-05-28T14:12:39.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 -- 172.21.15.42:0/686020748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02c15e620 msgr2=0x7ff02c06ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:39.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 --2- 172.21.15.42:0/686020748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02c15e620 0x7ff02c06ec00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0ada40 tx=0x7ff01c01c4c0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 -- 172.21.15.42:0/686020748 shutdown_connections 2024-05-28T14:12:39.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 --2- 172.21.15.42:0/686020748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02c15e620 0x7ff02c06ec00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 -- 172.21.15.42:0/686020748 >> 172.21.15.42:0/686020748 conn(0x7ff02c0d07c0 msgr2=0x7ff02c0d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:39.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 -- 172.21.15.42:0/686020748 shutdown_connections 2024-05-28T14:12:39.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 -- 172.21.15.42:0/686020748 wait complete. 2024-05-28T14:12:39.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 Processor -- start 2024-05-28T14:12:39.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 -- start start 2024-05-28T14:12:39.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff02c0b2c10 0x7ff02c15f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:39.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff02c0b4660 0x7ff02c0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:39.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02c15e620 0x7ff02c0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c0aec10 con 0x7ff02c0b2c10 2024-05-28T14:12:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c0aea90 con 0x7ff02c0b4660 2024-05-28T14:12:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.643+0000 7ff031000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff02c0aed90 con 0x7ff02c15e620 2024-05-28T14:12:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff029e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff02c0b2c10 0x7ff02c15f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff029e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff02c0b2c10 0x7ff02c15f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33254/0 (socket says 172.21.15.42:33254) 2024-05-28T14:12:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff029e00640 1 -- 172.21.15.42:0/1126505073 learned_addr learned my addr 172.21.15.42:0/1126505073 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff021400640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff02c0b4660 0x7ff02c0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff02a800640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02c15e620 0x7ff02c0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff029e00640 1 -- 172.21.15.42:0/1126505073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02c15e620 msgr2=0x7ff02c0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff029e00640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02c15e620 0x7ff02c0ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff029e00640 1 -- 172.21.15.42:0/1126505073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff02c0b4660 msgr2=0x7ff02c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:39.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff029e00640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff02c0b4660 0x7ff02c0ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff029e00640 1 -- 172.21.15.42:0/1126505073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c148990 con 0x7ff02c0b2c10 2024-05-28T14:12:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff021400640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff02c0b4660 0x7ff02c0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff02a800640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02c15e620 0x7ff02c0ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff029e00640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff02c0b2c10 0x7ff02c15f730 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7ff01c01a0f0 tx=0x7ff01c019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff022a00640 1 -- 172.21.15.42:0/1126505073 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c0bc020 con 0x7ff02c0b2c10 2024-05-28T14:12:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff022a00640 1 -- 172.21.15.42:0/1126505073 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff01c0ac7e0 con 0x7ff02c0b2c10 2024-05-28T14:12:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff022a00640 1 -- 172.21.15.42:0/1126505073 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c0b1040 con 0x7ff02c0b2c10 2024-05-28T14:12:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff031000640 1 -- 172.21.15.42:0/1126505073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02c1601f0 con 0x7ff02c0b2c10 2024-05-28T14:12:39.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff031000640 1 -- 172.21.15.42:0/1126505073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02c148c50 con 0x7ff02c0b2c10 2024-05-28T14:12:39.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff031000640 1 -- 172.21.15.42:0/1126505073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff02c073bd0 con 0x7ff02c0b2c10 2024-05-28T14:12:39.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff022a00640 1 -- 172.21.15.42:0/1126505073 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7ff01c0bf020 con 0x7ff02c0b2c10 2024-05-28T14:12:39.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.647+0000 7ff022a00640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff008069a00 0x7ff00806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:39.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.651+0000 7ff022a00640 1 -- 172.21.15.42:0/1126505073 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7ff01c11fc00 con 0x7ff02c0b2c10 2024-05-28T14:12:39.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.651+0000 7ff021400640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff008069a00 0x7ff00806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:39.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.651+0000 7ff021400640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff008069a00 0x7ff00806bec0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff0140016e0 tx=0x7ff014009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:39.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.655+0000 7ff022a00640 1 -- 172.21.15.42:0/1126505073 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff01c0160b0 con 0x7ff02c0b2c10 2024-05-28T14:12:39.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.843+0000 7ff031000640 1 -- 172.21.15.42:0/1126505073 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff02c160710 con 0x7ff008069a00 2024-05-28T14:12:39.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.847+0000 7ff022a00640 1 -- 172.21.15.42:0/1126505073 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7ff014001cc0 con 0x7ff008069a00 2024-05-28T14:12:39.833 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:39.833 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:12:39.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:39.833 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sv1" 2024-05-28T14:12:39.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:39.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:39.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ss1clone1" 2024-05-28T14:12:39.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:39.834 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:12:39.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.851+0000 7ff031000640 1 -- 172.21.15.42:0/1126505073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff008069a00 msgr2=0x7ff00806bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:39.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.851+0000 7ff031000640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff008069a00 0x7ff00806bec0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff0140016e0 tx=0x7ff014009040 comp rx=0 tx=0).stop 2024-05-28T14:12:39.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.851+0000 7ff031000640 1 -- 172.21.15.42:0/1126505073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff02c0b2c10 msgr2=0x7ff02c15f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:39.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.851+0000 7ff031000640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff02c0b2c10 0x7ff02c15f730 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7ff01c01a0f0 tx=0x7ff01c019260 comp rx=0 tx=0).stop 2024-05-28T14:12:39.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.851+0000 7ff031000640 1 -- 172.21.15.42:0/1126505073 shutdown_connections 2024-05-28T14:12:39.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.851+0000 7ff031000640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff008069a00 0x7ff00806bec0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.851+0000 7ff031000640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02c15e620 0x7ff02c0ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.851+0000 7ff031000640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff02c0b4660 0x7ff02c0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.851+0000 7ff031000640 1 --2- 172.21.15.42:0/1126505073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff02c0b2c10 0x7ff02c15f730 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:39.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.851+0000 7ff031000640 1 -- 172.21.15.42:0/1126505073 >> 172.21.15.42:0/1126505073 conn(0x7ff02c0d07c0 msgr2=0x7ff02c0b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:39.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.851+0000 7ff031000640 1 -- 172.21.15.42:0/1126505073 shutdown_connections 2024-05-28T14:12:39.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:39.851+0000 7ff031000640 1 -- 172.21.15.42:0/1126505073 wait complete. 2024-05-28T14:12:39.854 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-28T14:12:40.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa4e200640 1 Processor -- start 2024-05-28T14:12:40.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa4e200640 1 -- start start 2024-05-28T14:12:40.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa4e200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffa4815b350 0x7ffa4806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa4e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa48072d40 0x7ffa480d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa4e200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa480ade40 con 0x7ffa480737c0 2024-05-28T14:12:40.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa4e200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffa480ae9f0 con 0x7ffa4815b350 2024-05-28T14:12:40.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa4e200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa480ae0c0 con 0x7ffa48072d40 2024-05-28T14:12:40.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa47e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ffa480737c0 0x7ffa480aee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39844/0 (socket says 172.21.15.42:39844) 2024-05-28T14:12:40.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa47e00640 1 -- 172.21.15.42:0/215635824 learned_addr learned my addr 172.21.15.42:0/215635824 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:40.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa47400640 1 --2- 172.21.15.42:0/215635824 >> v2:172.21.15.63:3300/0 conn(0x7ffa4815b350 0x7ffa4806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa3ea00640 1 --2- 172.21.15.42:0/215635824 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa48072d40 0x7ffa480d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa46a00640 1 -- 172.21.15.42:0/215635824 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 887942112 0 0) 0x7ffa40001120 con 0x7ffa480737c0 2024-05-28T14:12:40.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa46a00640 1 -- 172.21.15.42:0/215635824 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa2c005740 con 0x7ffa480737c0 2024-05-28T14:12:40.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa47400640 1 -- 172.21.15.42:0/215635824 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa48072d40 msgr2=0x7ffa480d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:40.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa47400640 1 --2- 172.21.15.42:0/215635824 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa48072d40 0x7ffa480d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa47400640 1 -- 172.21.15.42:0/215635824 >> v1:172.21.15.42:6789/0 conn(0x7ffa480737c0 legacy=0x7ffa480aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa46a00640 1 -- 172.21.15.42:0/215635824 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2680094112 0 0) 0x7ffa2c005740 con 0x7ffa480737c0 2024-05-28T14:12:40.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.019+0000 7ffa47400640 1 -- 172.21.15.42:0/215635824 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa480ae3a0 con 0x7ffa4815b350 2024-05-28T14:12:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.023+0000 7ffa47400640 1 --2- 172.21.15.42:0/215635824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa4815b350 0x7ffa4806dd40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009d90 tx=0x7ffa3801c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2fecd8160d414da server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.023+0000 7ffa46a00640 1 -- 172.21.15.42:0/215635824 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa380ad070 con 0x7ffa4815b350 2024-05-28T14:12:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.023+0000 7ffa46a00640 1 -- 172.21.15.42:0/215635824 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa380a9410 con 0x7ffa4815b350 2024-05-28T14:12:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.023+0000 7ffa46a00640 1 -- 172.21.15.42:0/215635824 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40001850 con 0x7ffa4815b350 2024-05-28T14:12:40.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.023+0000 7ffa4e200640 1 -- 172.21.15.42:0/215635824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa4815b350 msgr2=0x7ffa4806dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.023+0000 7ffa4e200640 1 --2- 172.21.15.42:0/215635824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa4815b350 0x7ffa4806dd40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009d90 tx=0x7ffa3801c800 comp rx=0 tx=0).stop 2024-05-28T14:12:40.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.023+0000 7ffa4e200640 1 -- 172.21.15.42:0/215635824 shutdown_connections 2024-05-28T14:12:40.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.023+0000 7ffa4e200640 1 --2- 172.21.15.42:0/215635824 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa48072d40 0x7ffa480d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.023+0000 7ffa4e200640 1 --2- 172.21.15.42:0/215635824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa4815b350 0x7ffa4806dd40 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.023+0000 7ffa4e200640 1 -- 172.21.15.42:0/215635824 >> 172.21.15.42:0/215635824 conn(0x7ffa4806bdf0 msgr2=0x7ffa4806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.023+0000 7ffa4e200640 1 -- 172.21.15.42:0/215635824 shutdown_connections 2024-05-28T14:12:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.023+0000 7ffa4e200640 1 -- 172.21.15.42:0/215635824 wait complete. 2024-05-28T14:12:40.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa4e200640 1 Processor -- start 2024-05-28T14:12:40.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa4e200640 1 -- start start 2024-05-28T14:12:40.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa4e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa48072d40 0x7ffa48156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa4e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa480737c0 0x7ffa4814df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa4e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa4815b350 0x7ffa4814ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa4e200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffa480afb20 con 0x7ffa480737c0 2024-05-28T14:12:40.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa4e200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa480af9a0 con 0x7ffa48072d40 2024-05-28T14:12:40.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa4e200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa480b3af0 con 0x7ffa4815b350 2024-05-28T14:12:40.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa3ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa480737c0 0x7ffa4814df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa47400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa48072d40 0x7ffa48156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa3ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa480737c0 0x7ffa4814df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33260/0 (socket says 172.21.15.42:33260) 2024-05-28T14:12:40.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa3ea00640 1 -- 172.21.15.42:0/4105783362 learned_addr learned my addr 172.21.15.42:0/4105783362 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:40.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa47e00640 1 --2- 172.21.15.42:0/4105783362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa4815b350 0x7ffa4814ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa3ea00640 1 -- 172.21.15.42:0/4105783362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa4815b350 msgr2=0x7ffa4814ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa3ea00640 1 --2- 172.21.15.42:0/4105783362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa4815b350 0x7ffa4814ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa3ea00640 1 -- 172.21.15.42:0/4105783362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa48072d40 msgr2=0x7ffa48156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa3ea00640 1 --2- 172.21.15.42:0/4105783362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa48072d40 0x7ffa48156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa3ea00640 1 -- 172.21.15.42:0/4105783362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4814e4a0 con 0x7ffa480737c0 2024-05-28T14:12:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa3ea00640 1 --2- 172.21.15.42:0/4105783362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa480737c0 0x7ffa4814df60 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000e930 tx=0x7ffa3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa3fe00640 1 -- 172.21.15.42:0/4105783362 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30017070 con 0x7ffa480737c0 2024-05-28T14:12:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa3fe00640 1 -- 172.21.15.42:0/4105783362 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3000ca00 con 0x7ffa480737c0 2024-05-28T14:12:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa3fe00640 1 -- 172.21.15.42:0/4105783362 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000cdf0 con 0x7ffa480737c0 2024-05-28T14:12:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa4e200640 1 -- 172.21.15.42:0/4105783362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4814c360 con 0x7ffa480737c0 2024-05-28T14:12:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.027+0000 7ffa4e200640 1 -- 172.21.15.42:0/4105783362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4815d3b0 con 0x7ffa480737c0 2024-05-28T14:12:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.031+0000 7ffa4e200640 1 -- 172.21.15.42:0/4105783362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa48068a90 con 0x7ffa480737c0 2024-05-28T14:12:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.031+0000 7ffa3fe00640 1 -- 172.21.15.42:0/4105783362 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7ffa3001a070 con 0x7ffa480737c0 2024-05-28T14:12:40.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.031+0000 7ffa3fe00640 1 --2- 172.21.15.42:0/4105783362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa24069a00 0x7ffa2406bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.031+0000 7ffa3fe00640 1 -- 172.21.15.42:0/4105783362 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7ffa30085a50 con 0x7ffa480737c0 2024-05-28T14:12:40.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.031+0000 7ffa47400640 1 --2- 172.21.15.42:0/4105783362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa24069a00 0x7ffa2406bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.031+0000 7ffa47400640 1 --2- 172.21.15.42:0/4105783362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa24069a00 0x7ffa2406bec0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ffa380af000 tx=0x7ffa380b5040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.035+0000 7ffa3fe00640 1 -- 172.21.15.42:0/4105783362 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffa300574e0 con 0x7ffa480737c0 2024-05-28T14:12:40.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.231+0000 7ffa4e200640 1 -- 172.21.15.42:0/4105783362 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa48072b20 con 0x7ffa24069a00 2024-05-28T14:12:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.243+0000 7ffa3fe00640 1 -- 172.21.15.42:0/4105783362 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ffa48072b20 con 0x7ffa24069a00 2024-05-28T14:12:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:40.227 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:12:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ss1" 2024-05-28T14:12:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:40.228 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:12:40.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.243+0000 7ffa4e200640 1 -- 172.21.15.42:0/4105783362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa24069a00 msgr2=0x7ffa2406bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.243+0000 7ffa4e200640 1 --2- 172.21.15.42:0/4105783362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa24069a00 0x7ffa2406bec0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ffa380af000 tx=0x7ffa380b5040 comp rx=0 tx=0).stop 2024-05-28T14:12:40.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.243+0000 7ffa4e200640 1 -- 172.21.15.42:0/4105783362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa480737c0 msgr2=0x7ffa4814df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.243+0000 7ffa4e200640 1 --2- 172.21.15.42:0/4105783362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa480737c0 0x7ffa4814df60 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000e930 tx=0x7ffa3000edf0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.243+0000 7ffa4e200640 1 -- 172.21.15.42:0/4105783362 shutdown_connections 2024-05-28T14:12:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.243+0000 7ffa4e200640 1 --2- 172.21.15.42:0/4105783362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa24069a00 0x7ffa2406bec0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.243+0000 7ffa4e200640 1 --2- 172.21.15.42:0/4105783362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa4815b350 0x7ffa4814ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.243+0000 7ffa4e200640 1 --2- 172.21.15.42:0/4105783362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa480737c0 0x7ffa4814df60 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.243+0000 7ffa4e200640 1 --2- 172.21.15.42:0/4105783362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa48072d40 0x7ffa48156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.243+0000 7ffa4e200640 1 -- 172.21.15.42:0/4105783362 >> 172.21.15.42:0/4105783362 conn(0x7ffa4806bdf0 msgr2=0x7ffa481644b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.243+0000 7ffa4e200640 1 -- 172.21.15.42:0/4105783362 shutdown_connections 2024-05-28T14:12:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.243+0000 7ffa4e200640 1 -- 172.21.15.42:0/4105783362 wait complete. 2024-05-28T14:12:40.248 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:12:40.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1999600640 1 Processor -- start 2024-05-28T14:12:40.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1999600640 1 -- start start 2024-05-28T14:12:40.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1999600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1994144f80 0x7f1994072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1999600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f199406ee20 0x7f199406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1999600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19940afe40 con 0x7f199415b9b0 2024-05-28T14:12:40.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1999600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f199406f7c0 con 0x7f1994144f80 2024-05-28T14:12:40.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1999600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f19940724d0 con 0x7f199406ee20 2024-05-28T14:12:40.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1992a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1994144f80 0x7f1994072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1993400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f199415b9b0 0x7f199415be40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39852/0 (socket says 172.21.15.42:39852) 2024-05-28T14:12:40.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1993400640 1 -- 172.21.15.42:0/1811214232 learned_addr learned my addr 172.21.15.42:0/1811214232 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:40.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1992000640 1 --2- 172.21.15.42:0/1811214232 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f199406ee20 0x7f199406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1992a00640 1 -- 172.21.15.42:0/1811214232 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f199406ee20 msgr2=0x7f199406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:40.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1992a00640 1 --2- 172.21.15.42:0/1811214232 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f199406ee20 0x7f199406f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1992a00640 1 -- 172.21.15.42:0/1811214232 >> v1:172.21.15.42:6789/0 conn(0x7f199415b9b0 legacy=0x7f199415be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1992a00640 1 -- 172.21.15.42:0/1811214232 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199415c600 con 0x7f1994144f80 2024-05-28T14:12:40.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1992000640 1 --2- 172.21.15.42:0/1811214232 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f199406ee20 0x7f199406f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:40.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.411+0000 7f1991600640 1 -- 172.21.15.42:0/1811214232 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1868279132 0 0) 0x7f19940afe40 con 0x7f199415b9b0 2024-05-28T14:12:40.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.415+0000 7f1992a00640 1 --2- 172.21.15.42:0/1811214232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1994144f80 0x7f1994072960 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f19840097c0 tx=0x7f198401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a70ae1c0dab6756 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:40.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.415+0000 7f1991600640 1 -- 172.21.15.42:0/1811214232 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19840b2070 con 0x7f1994144f80 2024-05-28T14:12:40.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.415+0000 7f1991600640 1 -- 172.21.15.42:0/1811214232 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19840ad640 con 0x7f1994144f80 2024-05-28T14:12:40.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.415+0000 7f1991600640 1 -- 172.21.15.42:0/1811214232 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19840ada10 con 0x7f1994144f80 2024-05-28T14:12:40.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.415+0000 7f1999600640 1 -- 172.21.15.42:0/1811214232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1994144f80 msgr2=0x7f1994072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.415+0000 7f1999600640 1 --2- 172.21.15.42:0/1811214232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1994144f80 0x7f1994072960 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f19840097c0 tx=0x7f198401c4a0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.415+0000 7f1999600640 1 -- 172.21.15.42:0/1811214232 shutdown_connections 2024-05-28T14:12:40.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.415+0000 7f1999600640 1 --2- 172.21.15.42:0/1811214232 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f199406ee20 0x7f199406f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.415+0000 7f1999600640 1 --2- 172.21.15.42:0/1811214232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1994144f80 0x7f1994072960 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.415+0000 7f1999600640 1 -- 172.21.15.42:0/1811214232 >> 172.21.15.42:0/1811214232 conn(0x7f19940b21f0 msgr2=0x7f19940b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:40.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.415+0000 7f1999600640 1 -- 172.21.15.42:0/1811214232 shutdown_connections 2024-05-28T14:12:40.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.415+0000 7f1999600640 1 -- 172.21.15.42:0/1811214232 wait complete. 2024-05-28T14:12:40.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.415+0000 7f1999600640 1 Processor -- start 2024-05-28T14:12:40.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1999600640 1 -- start start 2024-05-28T14:12:40.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1999600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f199406ee20 0x7f199413fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1999600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1994144f80 0x7f19940b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1999600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f199415b9b0 0x7f19940b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1999600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1994072ea0 con 0x7f1994144f80 2024-05-28T14:12:40.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1999600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19940b4010 con 0x7f199406ee20 2024-05-28T14:12:40.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1999600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1994142140 con 0x7f199415b9b0 2024-05-28T14:12:40.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1992000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1994144f80 0x7f19940b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1992000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1994144f80 0x7f19940b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33284/0 (socket says 172.21.15.42:33284) 2024-05-28T14:12:40.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1992000640 1 -- 172.21.15.42:0/2148578501 learned_addr learned my addr 172.21.15.42:0/2148578501 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:40.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1992a00640 1 --2- 172.21.15.42:0/2148578501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f199406ee20 0x7f199413fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1993400640 1 --2- 172.21.15.42:0/2148578501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f199415b9b0 0x7f19940b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1992000640 1 -- 172.21.15.42:0/2148578501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f199415b9b0 msgr2=0x7f19940b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1992000640 1 --2- 172.21.15.42:0/2148578501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f199415b9b0 0x7f19940b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1992000640 1 -- 172.21.15.42:0/2148578501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f199406ee20 msgr2=0x7f199413fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1992000640 1 --2- 172.21.15.42:0/2148578501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f199406ee20 0x7f199413fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1992000640 1 -- 172.21.15.42:0/2148578501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1994140f00 con 0x7f1994144f80 2024-05-28T14:12:40.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1992000640 1 --2- 172.21.15.42:0/2148578501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1994144f80 0x7f19940b1330 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f198000e840 tx=0x7f198000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:40.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f198aa00640 1 -- 172.21.15.42:0/2148578501 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980017070 con 0x7f1994144f80 2024-05-28T14:12:40.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f198aa00640 1 -- 172.21.15.42:0/2148578501 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f198000c8d0 con 0x7f1994144f80 2024-05-28T14:12:40.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f198aa00640 1 -- 172.21.15.42:0/2148578501 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198000f040 con 0x7f1994144f80 2024-05-28T14:12:40.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1999600640 1 -- 172.21.15.42:0/2148578501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19941411e0 con 0x7f1994144f80 2024-05-28T14:12:40.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1999600640 1 -- 172.21.15.42:0/2148578501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19941417a0 con 0x7f1994144f80 2024-05-28T14:12:40.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.419+0000 7f1999600640 1 -- 172.21.15.42:0/2148578501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19940738b0 con 0x7f1994144f80 2024-05-28T14:12:40.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.423+0000 7f198aa00640 1 -- 172.21.15.42:0/2148578501 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7f198001a070 con 0x7f1994144f80 2024-05-28T14:12:40.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.423+0000 7f198aa00640 1 --2- 172.21.15.42:0/2148578501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1974069a00 0x7f197406bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.423+0000 7f198aa00640 1 -- 172.21.15.42:0/2148578501 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f19800852e0 con 0x7f1994144f80 2024-05-28T14:12:40.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.423+0000 7f1992a00640 1 --2- 172.21.15.42:0/2148578501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1974069a00 0x7f197406bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.423+0000 7f1992a00640 1 --2- 172.21.15.42:0/2148578501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1974069a00 0x7f197406bec0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f19840af000 tx=0x7f1984008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:40.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.427+0000 7f198aa00640 1 -- 172.21.15.42:0/2148578501 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1980056cb0 con 0x7f1994144f80 2024-05-28T14:12:40.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.623+0000 7f1999600640 1 -- 172.21.15.42:0/2148578501 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1994141b40 con 0x7f1974069a00 2024-05-28T14:12:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.651+0000 7f198aa00640 1 -- 172.21.15.42:0/2148578501 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1994141b40 con 0x7f1974069a00 2024-05-28T14:12:40.637 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:40.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.655+0000 7f1999600640 1 -- 172.21.15.42:0/2148578501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1974069a00 msgr2=0x7f197406bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.655+0000 7f1999600640 1 --2- 172.21.15.42:0/2148578501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1974069a00 0x7f197406bec0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f19840af000 tx=0x7f1984008040 comp rx=0 tx=0).stop 2024-05-28T14:12:40.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.655+0000 7f1999600640 1 -- 172.21.15.42:0/2148578501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1994144f80 msgr2=0x7f19940b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.655+0000 7f1999600640 1 --2- 172.21.15.42:0/2148578501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1994144f80 0x7f19940b1330 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f198000e840 tx=0x7f198000ed00 comp rx=0 tx=0).stop 2024-05-28T14:12:40.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.655+0000 7f1999600640 1 -- 172.21.15.42:0/2148578501 shutdown_connections 2024-05-28T14:12:40.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.655+0000 7f1999600640 1 --2- 172.21.15.42:0/2148578501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1974069a00 0x7f197406bec0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.655+0000 7f1999600640 1 --2- 172.21.15.42:0/2148578501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f199415b9b0 0x7f19940b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.655+0000 7f1999600640 1 --2- 172.21.15.42:0/2148578501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1994144f80 0x7f19940b1330 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.655+0000 7f1999600640 1 --2- 172.21.15.42:0/2148578501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f199406ee20 0x7f199413fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.655+0000 7f1999600640 1 -- 172.21.15.42:0/2148578501 >> 172.21.15.42:0/2148578501 conn(0x7f19940b21f0 msgr2=0x7f199415be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:40.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.655+0000 7f1999600640 1 -- 172.21.15.42:0/2148578501 shutdown_connections 2024-05-28T14:12:40.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.655+0000 7f1999600640 1 -- 172.21.15.42:0/2148578501 wait complete. 2024-05-28T14:12:40.658 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-28T14:12:40.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.815+0000 7fdb33600640 1 Processor -- start 2024-05-28T14:12:40.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.819+0000 7fdb33600640 1 -- start start 2024-05-28T14:12:40.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.819+0000 7fdb33600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdb2c0b2240 0x7fdb2c06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.819+0000 7fdb33600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb2c06bf30 0x7fdb2c0af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.819+0000 7fdb33600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb2c0b4440 con 0x7fdb2c15e8c0 2024-05-28T14:12:40.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.819+0000 7fdb33600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fdb2c0ae420 con 0x7fdb2c0b2240 2024-05-28T14:12:40.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.819+0000 7fdb33600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdb2c0b0f10 con 0x7fdb2c06bf30 2024-05-28T14:12:40.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.819+0000 7fdb23e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdb2c0b2240 0x7fdb2c06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.819+0000 7fdb23e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdb2c0b2240 0x7fdb2c06b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33296/0 (socket says 172.21.15.42:33296) 2024-05-28T14:12:40.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.819+0000 7fdb23e00640 1 -- 172.21.15.42:0/3212076842 learned_addr learned my addr 172.21.15.42:0/3212076842 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:40.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.819+0000 7fdb31600640 1 --2- 172.21.15.42:0/3212076842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb2c06bf30 0x7fdb2c0af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.819+0000 7fdb23e00640 1 -- 172.21.15.42:0/3212076842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb2c06bf30 msgr2=0x7fdb2c0af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:40.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.819+0000 7fdb23e00640 1 --2- 172.21.15.42:0/3212076842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb2c06bf30 0x7fdb2c0af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.819+0000 7fdb23e00640 1 -- 172.21.15.42:0/3212076842 >> v1:172.21.15.63:6789/0 conn(0x7fdb2c15e8c0 legacy=0x7fdb2c0b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.819+0000 7fdb23e00640 1 -- 172.21.15.42:0/3212076842 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2c0ae670 con 0x7fdb2c0b2240 2024-05-28T14:12:40.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.823+0000 7fdb23e00640 1 --2- 172.21.15.42:0/3212076842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb2c0b2240 0x7fdb2c06b9f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009870 tx=0x7fdb1401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb52fcec9d6e3771 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:40.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.823+0000 7fdb2be00640 1 -- 172.21.15.42:0/3212076842 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb140b0070 con 0x7fdb2c0b2240 2024-05-28T14:12:40.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.823+0000 7fdb2be00640 1 -- 172.21.15.42:0/3212076842 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb140ab600 con 0x7fdb2c0b2240 2024-05-28T14:12:40.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.823+0000 7fdb2be00640 1 -- 172.21.15.42:0/3212076842 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb140ab9d0 con 0x7fdb2c0b2240 2024-05-28T14:12:40.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.823+0000 7fdb33600640 1 -- 172.21.15.42:0/3212076842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb2c0b2240 msgr2=0x7fdb2c06b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.823+0000 7fdb33600640 1 --2- 172.21.15.42:0/3212076842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb2c0b2240 0x7fdb2c06b9f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009870 tx=0x7fdb1401c4d0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.823+0000 7fdb33600640 1 -- 172.21.15.42:0/3212076842 shutdown_connections 2024-05-28T14:12:40.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.823+0000 7fdb33600640 1 --2- 172.21.15.42:0/3212076842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb2c06bf30 0x7fdb2c0af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.823+0000 7fdb33600640 1 --2- 172.21.15.42:0/3212076842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb2c0b2240 0x7fdb2c06b9f0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.823+0000 7fdb33600640 1 -- 172.21.15.42:0/3212076842 >> 172.21.15.42:0/3212076842 conn(0x7fdb2c0b6960 msgr2=0x7fdb2c0b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:40.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.823+0000 7fdb33600640 1 -- 172.21.15.42:0/3212076842 shutdown_connections 2024-05-28T14:12:40.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.823+0000 7fdb33600640 1 -- 172.21.15.42:0/3212076842 wait complete. 2024-05-28T14:12:40.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.823+0000 7fdb33600640 1 Processor -- start 2024-05-28T14:12:40.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.823+0000 7fdb33600640 1 -- start start 2024-05-28T14:12:40.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb33600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb2c06bf30 0x7fdb2c14e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb33600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb2c0b2240 0x7fdb2c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb33600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb2c15e8c0 0x7fdb2c154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb33600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c0b0020 con 0x7fdb2c0b2240 2024-05-28T14:12:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb33600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c0afea0 con 0x7fdb2c06bf30 2024-05-28T14:12:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb33600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdb2c0b01a0 con 0x7fdb2c15e8c0 2024-05-28T14:12:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb23e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb2c0b2240 0x7fdb2c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb23e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb2c0b2240 0x7fdb2c15d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33302/0 (socket says 172.21.15.42:33302) 2024-05-28T14:12:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb23e00640 1 -- 172.21.15.42:0/3871819161 learned_addr learned my addr 172.21.15.42:0/3871819161 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb31600640 1 --2- 172.21.15.42:0/3871819161 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb2c15e8c0 0x7fdb2c154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb30c00640 1 --2- 172.21.15.42:0/3871819161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb2c06bf30 0x7fdb2c14e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb23e00640 1 -- 172.21.15.42:0/3871819161 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb2c15e8c0 msgr2=0x7fdb2c154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb23e00640 1 --2- 172.21.15.42:0/3871819161 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb2c15e8c0 0x7fdb2c154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb23e00640 1 -- 172.21.15.42:0/3871819161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb2c06bf30 msgr2=0x7fdb2c14e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb23e00640 1 --2- 172.21.15.42:0/3871819161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb2c06bf30 0x7fdb2c14e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb23e00640 1 -- 172.21.15.42:0/3871819161 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2c155560 con 0x7fdb2c0b2240 2024-05-28T14:12:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb23e00640 1 --2- 172.21.15.42:0/3871819161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb2c0b2240 0x7fdb2c15d900 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fdb140ae040 tx=0x7fdb14019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb29600640 1 -- 172.21.15.42:0/3871819161 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb140b0040 con 0x7fdb2c0b2240 2024-05-28T14:12:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb29600640 1 -- 172.21.15.42:0/3871819161 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb140b9070 con 0x7fdb2c0b2240 2024-05-28T14:12:40.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb29600640 1 -- 172.21.15.42:0/3871819161 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb140b4680 con 0x7fdb2c0b2240 2024-05-28T14:12:40.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb33600640 1 -- 172.21.15.42:0/3871819161 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2c155840 con 0x7fdb2c0b2240 2024-05-28T14:12:40.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb33600640 1 -- 172.21.15.42:0/3871819161 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb2c155e00 con 0x7fdb2c0b2240 2024-05-28T14:12:40.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.827+0000 7fdb33600640 1 -- 172.21.15.42:0/3871819161 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb2c156040 con 0x7fdb2c0b2240 2024-05-28T14:12:40.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.831+0000 7fdb29600640 1 -- 172.21.15.42:0/3871819161 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7fdb140b4820 con 0x7fdb2c0b2240 2024-05-28T14:12:40.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.831+0000 7fdb29600640 1 --2- 172.21.15.42:0/3871819161 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdb08069a00 0x7fdb0806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:40.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.831+0000 7fdb29600640 1 -- 172.21.15.42:0/3871819161 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fdb1411f2a0 con 0x7fdb2c0b2240 2024-05-28T14:12:40.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.831+0000 7fdb30c00640 1 --2- 172.21.15.42:0/3871819161 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdb08069a00 0x7fdb0806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:40.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.831+0000 7fdb30c00640 1 --2- 172.21.15.42:0/3871819161 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdb08069a00 0x7fdb0806bec0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c007fd0 tx=0x7fdb1c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:40.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:40.835+0000 7fdb29600640 1 -- 172.21.15.42:0/3871819161 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdb14016150 con 0x7fdb2c0b2240 2024-05-28T14:12:41.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.023+0000 7fdb33600640 1 -- 172.21.15.42:0/3871819161 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fdb2c09f7a0 con 0x7fdb08069a00 2024-05-28T14:12:41.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.043+0000 7fdb29600640 1 -- 172.21.15.42:0/3871819161 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdb2c09f7a0 con 0x7fdb08069a00 2024-05-28T14:12:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.047+0000 7fdb33600640 1 -- 172.21.15.42:0/3871819161 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdb08069a00 msgr2=0x7fdb0806bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.047+0000 7fdb33600640 1 --2- 172.21.15.42:0/3871819161 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdb08069a00 0x7fdb0806bec0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c007fd0 tx=0x7fdb1c009040 comp rx=0 tx=0).stop 2024-05-28T14:12:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.047+0000 7fdb33600640 1 -- 172.21.15.42:0/3871819161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb2c0b2240 msgr2=0x7fdb2c15d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.047+0000 7fdb33600640 1 --2- 172.21.15.42:0/3871819161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb2c0b2240 0x7fdb2c15d900 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fdb140ae040 tx=0x7fdb14019270 comp rx=0 tx=0).stop 2024-05-28T14:12:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.047+0000 7fdb33600640 1 -- 172.21.15.42:0/3871819161 shutdown_connections 2024-05-28T14:12:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.047+0000 7fdb33600640 1 --2- 172.21.15.42:0/3871819161 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdb08069a00 0x7fdb0806bec0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.047+0000 7fdb33600640 1 --2- 172.21.15.42:0/3871819161 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb2c15e8c0 0x7fdb2c154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.047+0000 7fdb33600640 1 --2- 172.21.15.42:0/3871819161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb2c0b2240 0x7fdb2c15d900 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.047+0000 7fdb33600640 1 --2- 172.21.15.42:0/3871819161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb2c06bf30 0x7fdb2c14e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.047+0000 7fdb33600640 1 -- 172.21.15.42:0/3871819161 >> 172.21.15.42:0/3871819161 conn(0x7fdb2c0b6960 msgr2=0x7fdb2c0b1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:41.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.047+0000 7fdb33600640 1 -- 172.21.15.42:0/3871819161 shutdown_connections 2024-05-28T14:12:41.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.047+0000 7fdb33600640 1 -- 172.21.15.42:0/3871819161 wait complete. 2024-05-28T14:12:41.034 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:12:41.047+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:12:41.034 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:12:41.047+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:12:41.034 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:12:41.047+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:12:41.034 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:12:41.047+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:12:41.034 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:12:41.047+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:12:41.051 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:12:41.067+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:12:41.051 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:12:41.067+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:12:41.051 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:12:41.067+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:12:41.051 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:12:41.067+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:12:41.051 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:12:41.067+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:12:41.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-05-28T14:12:41.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.219+0000 7fd9b5a00640 1 Processor -- start 2024-05-28T14:12:41.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.219+0000 7fd9b5a00640 1 -- start start 2024-05-28T14:12:41.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9b5a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd9b015e840 0x7fd9b00d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:41.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9b5a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9b006cd10 0x7fd9b006d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:41.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9b5a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9b00d1970 con 0x7fd9b006da50 2024-05-28T14:12:41.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9b5a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd9b00b5300 con 0x7fd9b015e840 2024-05-28T14:12:41.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9b5a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b00d1740 con 0x7fd9b006cd10 2024-05-28T14:12:41.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9b006cd10 0x7fd9b006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:41.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9aea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd9b015e840 0x7fd9b00d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:41.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9aea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd9b015e840 0x7fd9b00d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:57694/0 (socket says 172.21.15.42:57694) 2024-05-28T14:12:41.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9b006cd10 0x7fd9b006d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:35314/0 (socket says 172.21.15.42:35314) 2024-05-28T14:12:41.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9af400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fd9b006da50 0x7fd9b009aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:41702/0 (socket says 172.21.15.42:41702) 2024-05-28T14:12:41.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9a7e00640 1 -- 172.21.15.42:0/2292285634 learned_addr learned my addr 172.21.15.42:0/2292285634 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:41.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9a7e00640 1 -- 172.21.15.42:0/2292285634 >> v2:172.21.15.63:3300/0 conn(0x7fd9b015e840 msgr2=0x7fd9b00d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:41.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9a7e00640 1 --2- 172.21.15.42:0/2292285634 >> v2:172.21.15.63:3300/0 conn(0x7fd9b015e840 0x7fd9b00d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9ae000640 1 -- 172.21.15.42:0/2292285634 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3887773148 0 0) 0x7fd9b00d1970 con 0x7fd9b006da50 2024-05-28T14:12:41.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9a7e00640 1 -- 172.21.15.42:0/2292285634 >> v1:172.21.15.63:6789/0 conn(0x7fd9b006da50 legacy=0x7fd9b009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:41.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.223+0000 7fd9a7e00640 1 -- 172.21.15.42:0/2292285634 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b00ccca0 con 0x7fd9b006cd10 2024-05-28T14:12:41.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.227+0000 7fd9a7e00640 1 --2- 172.21.15.42:0/2292285634 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9b006cd10 0x7fd9b006d510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd998009870 tx=0x7fd99801ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fab6ca82827b2487 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:41.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.227+0000 7fd9ae000640 1 -- 172.21.15.42:0/2292285634 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980b5070 con 0x7fd9b006cd10 2024-05-28T14:12:41.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.227+0000 7fd9ae000640 1 -- 172.21.15.42:0/2292285634 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9980a8a50 con 0x7fd9b006cd10 2024-05-28T14:12:41.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.227+0000 7fd9ae000640 1 -- 172.21.15.42:0/2292285634 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980a8e20 con 0x7fd9b006cd10 2024-05-28T14:12:41.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.227+0000 7fd9b5a00640 1 -- 172.21.15.42:0/2292285634 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9b006cd10 msgr2=0x7fd9b006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:41.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.227+0000 7fd9b5a00640 1 --2- 172.21.15.42:0/2292285634 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9b006cd10 0x7fd9b006d510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd998009870 tx=0x7fd99801ed50 comp rx=0 tx=0).stop 2024-05-28T14:12:41.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.227+0000 7fd9b5a00640 1 -- 172.21.15.42:0/2292285634 shutdown_connections 2024-05-28T14:12:41.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.227+0000 7fd9b5a00640 1 --2- 172.21.15.42:0/2292285634 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9b006cd10 0x7fd9b006d510 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.227+0000 7fd9b5a00640 1 --2- 172.21.15.42:0/2292285634 >> v2:172.21.15.63:3300/0 conn(0x7fd9b015e840 0x7fd9b00d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.227+0000 7fd9b5a00640 1 -- 172.21.15.42:0/2292285634 >> 172.21.15.42:0/2292285634 conn(0x7fd9b00b3fa0 msgr2=0x7fd9b00b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:41.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.227+0000 7fd9b5a00640 1 -- 172.21.15.42:0/2292285634 shutdown_connections 2024-05-28T14:12:41.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.227+0000 7fd9b5a00640 1 -- 172.21.15.42:0/2292285634 wait complete. 2024-05-28T14:12:41.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9b5a00640 1 Processor -- start 2024-05-28T14:12:41.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9b5a00640 1 -- start start 2024-05-28T14:12:41.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9b5a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9b006cd10 0x7fd9b01f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:41.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9b5a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9b006da50 0x7fd9b01f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:41.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9b5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9b015e840 0x7fd9b0160f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:41.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9b5a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b00cd260 con 0x7fd9b015e840 2024-05-28T14:12:41.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9b5a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b006dea0 con 0x7fd9b006cd10 2024-05-28T14:12:41.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9b5a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b015ec20 con 0x7fd9b006da50 2024-05-28T14:12:41.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9af400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9b015e840 0x7fd9b0160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:41.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9af400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9b015e840 0x7fd9b0160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33336/0 (socket says 172.21.15.42:33336) 2024-05-28T14:12:41.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9af400640 1 -- 172.21.15.42:0/1567571733 learned_addr learned my addr 172.21.15.42:0/1567571733 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:41.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9aea00640 1 --2- 172.21.15.42:0/1567571733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9b006cd10 0x7fd9b01f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:41.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9a7e00640 1 --2- 172.21.15.42:0/1567571733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9b006da50 0x7fd9b01f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:41.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9af400640 1 -- 172.21.15.42:0/1567571733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9b006da50 msgr2=0x7fd9b01f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:41.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9af400640 1 --2- 172.21.15.42:0/1567571733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9b006da50 0x7fd9b01f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9af400640 1 -- 172.21.15.42:0/1567571733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9b006cd10 msgr2=0x7fd9b01f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:41.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9af400640 1 --2- 172.21.15.42:0/1567571733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9b006cd10 0x7fd9b01f43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9af400640 1 -- 172.21.15.42:0/1567571733 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b00cade0 con 0x7fd9b015e840 2024-05-28T14:12:41.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9af400640 1 --2- 172.21.15.42:0/1567571733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9b015e840 0x7fd9b0160f20 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800c7a0 tx=0x7fd9a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:41.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9a6a00640 1 -- 172.21.15.42:0/1567571733 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8018070 con 0x7fd9b015e840 2024-05-28T14:12:41.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9a6a00640 1 -- 172.21.15.42:0/1567571733 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a800a640 con 0x7fd9b015e840 2024-05-28T14:12:41.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9b5a00640 1 -- 172.21.15.42:0/1567571733 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b00cb0c0 con 0x7fd9b015e840 2024-05-28T14:12:41.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9a6a00640 1 -- 172.21.15.42:0/1567571733 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a800aab0 con 0x7fd9b015e840 2024-05-28T14:12:41.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.231+0000 7fd9b5a00640 1 -- 172.21.15.42:0/1567571733 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b00cb680 con 0x7fd9b015e840 2024-05-28T14:12:41.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.235+0000 7fd9b5a00640 1 -- 172.21.15.42:0/1567571733 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b0068a90 con 0x7fd9b015e840 2024-05-28T14:12:41.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.235+0000 7fd9a6a00640 1 -- 172.21.15.42:0/1567571733 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7fd9a800ac50 con 0x7fd9b015e840 2024-05-28T14:12:41.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.235+0000 7fd9a6a00640 1 --2- 172.21.15.42:0/1567571733 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd98c069a00 0x7fd98c06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:41.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.235+0000 7fd9a6a00640 1 -- 172.21.15.42:0/1567571733 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fd9a8084ce0 con 0x7fd9b015e840 2024-05-28T14:12:41.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.235+0000 7fd9aea00640 1 --2- 172.21.15.42:0/1567571733 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd98c069a00 0x7fd98c06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:41.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.235+0000 7fd9aea00640 1 --2- 172.21.15.42:0/1567571733 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd98c069a00 0x7fd98c06bec0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0070610 tx=0x7fd9a0009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:41.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.239+0000 7fd9a6a00640 1 -- 172.21.15.42:0/1567571733 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd9a80566d0 con 0x7fd9b015e840 2024-05-28T14:12:41.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.435+0000 7fd9b5a00640 1 -- 172.21.15.42:0/1567571733 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fd9b00d1070 con 0x7fd98c069a00 2024-05-28T14:12:41.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.455+0000 7fd9a6a00640 1 -- 172.21.15.42:0/1567571733 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd9b00d1070 con 0x7fd98c069a00 2024-05-28T14:12:41.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.459+0000 7fd9b5a00640 1 -- 172.21.15.42:0/1567571733 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd98c069a00 msgr2=0x7fd98c06bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:41.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.459+0000 7fd9b5a00640 1 --2- 172.21.15.42:0/1567571733 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd98c069a00 0x7fd98c06bec0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0070610 tx=0x7fd9a0009190 comp rx=0 tx=0).stop 2024-05-28T14:12:41.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.459+0000 7fd9b5a00640 1 -- 172.21.15.42:0/1567571733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9b015e840 msgr2=0x7fd9b0160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:41.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.459+0000 7fd9b5a00640 1 --2- 172.21.15.42:0/1567571733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9b015e840 0x7fd9b0160f20 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800c7a0 tx=0x7fd9a800cc60 comp rx=0 tx=0).stop 2024-05-28T14:12:41.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.459+0000 7fd9b5a00640 1 -- 172.21.15.42:0/1567571733 shutdown_connections 2024-05-28T14:12:41.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.459+0000 7fd9b5a00640 1 --2- 172.21.15.42:0/1567571733 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd98c069a00 0x7fd98c06bec0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.459+0000 7fd9b5a00640 1 --2- 172.21.15.42:0/1567571733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9b015e840 0x7fd9b0160f20 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.459+0000 7fd9b5a00640 1 --2- 172.21.15.42:0/1567571733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9b006da50 0x7fd9b01f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.459+0000 7fd9b5a00640 1 --2- 172.21.15.42:0/1567571733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9b006cd10 0x7fd9b01f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.459+0000 7fd9b5a00640 1 -- 172.21.15.42:0/1567571733 >> 172.21.15.42:0/1567571733 conn(0x7fd9b00b3fa0 msgr2=0x7fd9b00d0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:41.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.459+0000 7fd9b5a00640 1 -- 172.21.15.42:0/1567571733 shutdown_connections 2024-05-28T14:12:41.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.459+0000 7fd9b5a00640 1 -- 172.21.15.42:0/1567571733 wait complete. 2024-05-28T14:12:41.464 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-28T14:12:41.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-28T14:12:41.523 INFO:teuthology.orchestra.run.smithi042.stdout:2 2024-05-28T14:12:41.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:12:41.530 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T14:12:41.530 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:12:41.633 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:12:41.743 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T14:12:41.743 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:12:41.743 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:12:41.743 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:12:41.752 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:12:41.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.947+0000 7fdad9c00640 1 Processor -- start 2024-05-28T14:12:41.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.947+0000 7fdad9c00640 1 -- start start 2024-05-28T14:12:41.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad9c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdad406f660 0x7fdad4073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:41.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad9c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdad415ba40 0x7fdad40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:41.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad9c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdad4072dd0 0x7fdad40d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad9c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fdad415f030 con 0x7fdad406f660 2024-05-28T14:12:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad9c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdad415f280 con 0x7fdad415ba40 2024-05-28T14:12:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad9c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdad415f560 con 0x7fdad4072dd0 2024-05-28T14:12:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad2a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdad406f660 0x7fdad4073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad2a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdad406f660 0x7fdad4073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33344/0 (socket says 172.21.15.42:33344) 2024-05-28T14:12:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad2a00640 1 -- 172.21.15.42:0/1669856533 learned_addr learned my addr 172.21.15.42:0/1669856533 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad3400640 1 --2- 172.21.15.42:0/1669856533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdad4072dd0 0x7fdad40d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdaca000640 1 --2- 172.21.15.42:0/1669856533 >> v2:172.21.15.63:3300/0 conn(0x7fdad415ba40 0x7fdad40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad2a00640 1 -- 172.21.15.42:0/1669856533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdad4072dd0 msgr2=0x7fdad40d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad2a00640 1 --2- 172.21.15.42:0/1669856533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdad4072dd0 0x7fdad40d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad2a00640 1 -- 172.21.15.42:0/1669856533 >> v2:172.21.15.63:3300/0 conn(0x7fdad415ba40 msgr2=0x7fdad40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad2a00640 1 --2- 172.21.15.42:0/1669856533 >> v2:172.21.15.63:3300/0 conn(0x7fdad415ba40 0x7fdad40d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.951+0000 7fdad2a00640 1 -- 172.21.15.42:0/1669856533 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad415f840 con 0x7fdad406f660 2024-05-28T14:12:41.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad2a00640 1 --2- 172.21.15.42:0/1669856533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdad406f660 0x7fdad4073980 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fdad415f770 tx=0x7fdac401e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d121edb2d2abf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:41.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad2000640 1 -- 172.21.15.42:0/1669856533 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40b2070 con 0x7fdad406f660 2024-05-28T14:12:41.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad2000640 1 -- 172.21.15.42:0/1669856533 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac40ad600 con 0x7fdad406f660 2024-05-28T14:12:41.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad2000640 1 -- 172.21.15.42:0/1669856533 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40ad9d0 con 0x7fdad406f660 2024-05-28T14:12:41.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad9c00640 1 -- 172.21.15.42:0/1669856533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdad406f660 msgr2=0x7fdad4073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:41.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad9c00640 1 --2- 172.21.15.42:0/1669856533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdad406f660 0x7fdad4073980 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fdad415f770 tx=0x7fdac401e760 comp rx=0 tx=0).stop 2024-05-28T14:12:41.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad9c00640 1 -- 172.21.15.42:0/1669856533 shutdown_connections 2024-05-28T14:12:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad9c00640 1 --2- 172.21.15.42:0/1669856533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdad4072dd0 0x7fdad40d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad9c00640 1 --2- 172.21.15.42:0/1669856533 >> v2:172.21.15.63:3300/0 conn(0x7fdad415ba40 0x7fdad40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad9c00640 1 --2- 172.21.15.42:0/1669856533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdad406f660 0x7fdad4073980 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad9c00640 1 -- 172.21.15.42:0/1669856533 >> 172.21.15.42:0/1669856533 conn(0x7fdad406bdf0 msgr2=0x7fdad406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad9c00640 1 -- 172.21.15.42:0/1669856533 shutdown_connections 2024-05-28T14:12:41.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad9c00640 1 -- 172.21.15.42:0/1669856533 wait complete. 2024-05-28T14:12:41.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad9c00640 1 Processor -- start 2024-05-28T14:12:41.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.955+0000 7fdad9c00640 1 -- start start 2024-05-28T14:12:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad9c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdad406f660 0x7fdad4099900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad9c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdad4072dd0 0x7fdad41ed1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdad41ed720 0x7fdad41f8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad9c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4166f60 con 0x7fdad41ed720 2024-05-28T14:12:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad9c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4166de0 con 0x7fdad406f660 2024-05-28T14:12:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad9c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdad41670e0 con 0x7fdad4072dd0 2024-05-28T14:12:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdad41ed720 0x7fdad41f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdad41ed720 0x7fdad41f8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33352/0 (socket says 172.21.15.42:33352) 2024-05-28T14:12:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad3400640 1 -- 172.21.15.42:0/3226232256 learned_addr learned my addr 172.21.15.42:0/3226232256 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad2a00640 1 --2- 172.21.15.42:0/3226232256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdad406f660 0x7fdad4099900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdaca000640 1 --2- 172.21.15.42:0/3226232256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdad4072dd0 0x7fdad41ed1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:41.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad3400640 1 -- 172.21.15.42:0/3226232256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdad4072dd0 msgr2=0x7fdad41ed1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:41.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad3400640 1 --2- 172.21.15.42:0/3226232256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdad4072dd0 0x7fdad41ed1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad3400640 1 -- 172.21.15.42:0/3226232256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdad406f660 msgr2=0x7fdad4099900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:41.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad3400640 1 --2- 172.21.15.42:0/3226232256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdad406f660 0x7fdad4099900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:41.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad3400640 1 -- 172.21.15.42:0/3226232256 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad41f99b0 con 0x7fdad41ed720 2024-05-28T14:12:41.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad3400640 1 --2- 172.21.15.42:0/3226232256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdad41ed720 0x7fdad41f8660 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00d800 tx=0x7fdacc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:41.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdacb400640 1 -- 172.21.15.42:0/3226232256 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc018020 con 0x7fdad41ed720 2024-05-28T14:12:41.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdacb400640 1 -- 172.21.15.42:0/3226232256 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdacc0096c0 con 0x7fdad41ed720 2024-05-28T14:12:41.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdacb400640 1 -- 172.21.15.42:0/3226232256 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc009ab0 con 0x7fdad41ed720 2024-05-28T14:12:41.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad9c00640 1 -- 172.21.15.42:0/3226232256 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad41f9c90 con 0x7fdad41ed720 2024-05-28T14:12:41.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad9c00640 1 -- 172.21.15.42:0/3226232256 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad41fa250 con 0x7fdad41ed720 2024-05-28T14:12:41.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdad9c00640 1 -- 172.21.15.42:0/3226232256 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad4068a90 con 0x7fdad41ed720 2024-05-28T14:12:41.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.959+0000 7fdacb400640 1 -- 172.21.15.42:0/3226232256 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 90027+0+0 (secure 0 0 0) 0x7fdacc01b020 con 0x7fdad41ed720 2024-05-28T14:12:41.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.963+0000 7fdacb400640 1 --2- 172.21.15.42:0/3226232256 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdab0069a00 0x7fdab006bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:41.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.963+0000 7fdacb400640 1 -- 172.21.15.42:0/3226232256 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fdacc085b30 con 0x7fdad41ed720 2024-05-28T14:12:41.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.963+0000 7fdad2a00640 1 --2- 172.21.15.42:0/3226232256 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdab0069a00 0x7fdab006bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:41.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.963+0000 7fdad2a00640 1 --2- 172.21.15.42:0/3226232256 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdab0069a00 0x7fdab006bec0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdac40af000 tx=0x7fdac40a8000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:41.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:41.967+0000 7fdacb400640 1 -- 172.21.15.42:0/3226232256 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdacc022020 con 0x7fdad41ed720 2024-05-28T14:12:42.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.047+0000 7fdacb400640 1 -- 172.21.15.42:0/3226232256 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fdacc053b70 con 0x7fdad41ed720 2024-05-28T14:12:42.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.139+0000 7fdad9c00640 1 -- 172.21.15.42:0/3226232256 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdad4072bb0 con 0x7fdad41ed720 2024-05-28T14:12:42.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.139+0000 7fdacb400640 1 -- 172.21.15.42:0/3226232256 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16602 (secure 0 0 0) 0x7fdacc074ac0 con 0x7fdad41ed720 2024-05-28T14:12:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:42.124 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":105,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:11:16.396600+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:11:13.194240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-28T14:11:13.743150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:12:42.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.139+0000 7fdad9c00640 1 -- 172.21.15.42:0/3226232256 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdab0069a00 msgr2=0x7fdab006bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.139+0000 7fdad9c00640 1 --2- 172.21.15.42:0/3226232256 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdab0069a00 0x7fdab006bec0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdac40af000 tx=0x7fdac40a8000 comp rx=0 tx=0).stop 2024-05-28T14:12:42.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.139+0000 7fdad9c00640 1 -- 172.21.15.42:0/3226232256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdad41ed720 msgr2=0x7fdad41f8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.139+0000 7fdad9c00640 1 --2- 172.21.15.42:0/3226232256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdad41ed720 0x7fdad41f8660 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00d800 tx=0x7fdacc00dcc0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.139+0000 7fdad9c00640 1 -- 172.21.15.42:0/3226232256 shutdown_connections 2024-05-28T14:12:42.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.139+0000 7fdad9c00640 1 --2- 172.21.15.42:0/3226232256 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdab0069a00 0x7fdab006bec0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.139+0000 7fdad9c00640 1 --2- 172.21.15.42:0/3226232256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdad41ed720 0x7fdad41f8660 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.139+0000 7fdad9c00640 1 --2- 172.21.15.42:0/3226232256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdad4072dd0 0x7fdad41ed1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.139+0000 7fdad9c00640 1 --2- 172.21.15.42:0/3226232256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdad406f660 0x7fdad4099900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.139+0000 7fdad9c00640 1 -- 172.21.15.42:0/3226232256 >> 172.21.15.42:0/3226232256 conn(0x7fdad406bdf0 msgr2=0x7fdad415e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:42.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.139+0000 7fdad9c00640 1 -- 172.21.15.42:0/3226232256 shutdown_connections 2024-05-28T14:12:42.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.139+0000 7fdad9c00640 1 -- 172.21.15.42:0/3226232256 wait complete. 2024-05-28T14:12:42.147 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T14:11:13.194240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T14:11:13.743150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:12:42.147 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:12:42.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.307+0000 7fef9e600640 1 Processor -- start 2024-05-28T14:12:42.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.307+0000 7fef9e600640 1 -- start start 2024-05-28T14:12:42.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.307+0000 7fef9e600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef98144f80 0x7fef98072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:42.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.307+0000 7fef9e600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fef9815c0c0 0x7fef9806f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:42.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.307+0000 7fef9e600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef980afe40 con 0x7fef9806ee20 2024-05-28T14:12:42.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.307+0000 7fef9e600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fef9806f7c0 con 0x7fef9815c0c0 2024-05-28T14:12:42.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.307+0000 7fef9e600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef980724d0 con 0x7fef98144f80 2024-05-28T14:12:42.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.307+0000 7fef8ea00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fef9806ee20 0x7fef9815b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39860/0 (socket says 172.21.15.42:39860) 2024-05-28T14:12:42.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.307+0000 7fef8ea00640 1 -- 172.21.15.42:0/4085664399 learned_addr learned my addr 172.21.15.42:0/4085664399 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:42.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.307+0000 7fef97400640 1 --2- 172.21.15.42:0/4085664399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef98144f80 0x7fef98072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:42.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.307+0000 7fef97e00640 1 --2- 172.21.15.42:0/4085664399 >> v2:172.21.15.63:3300/0 conn(0x7fef9815c0c0 0x7fef9806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:42.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.311+0000 7fef96a00640 1 -- 172.21.15.42:0/4085664399 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3160019517 0 0) 0x7fef980afe40 con 0x7fef9806ee20 2024-05-28T14:12:42.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.311+0000 7fef96a00640 1 -- 172.21.15.42:0/4085664399 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef7c003530 con 0x7fef9806ee20 2024-05-28T14:12:42.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.311+0000 7fef97400640 1 -- 172.21.15.42:0/4085664399 >> v2:172.21.15.63:3300/0 conn(0x7fef9815c0c0 msgr2=0x7fef9806f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:42.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.311+0000 7fef97400640 1 --2- 172.21.15.42:0/4085664399 >> v2:172.21.15.63:3300/0 conn(0x7fef9815c0c0 0x7fef9806f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.311+0000 7fef96a00640 1 -- 172.21.15.42:0/4085664399 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 615943944 0 0) 0x7fef7c003530 con 0x7fef9806ee20 2024-05-28T14:12:42.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.311+0000 7fef97400640 1 -- 172.21.15.42:0/4085664399 >> v1:172.21.15.42:6789/0 conn(0x7fef9806ee20 legacy=0x7fef9815b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.311+0000 7fef97400640 1 -- 172.21.15.42:0/4085664399 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9815c600 con 0x7fef98144f80 2024-05-28T14:12:42.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.311+0000 7fef97400640 1 --2- 172.21.15.42:0/4085664399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef98144f80 0x7fef98072960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fef880097c0 tx=0x7fef8801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaa72c6b601095d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:42.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.311+0000 7fef96a00640 1 -- 172.21.15.42:0/4085664399 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef880b2070 con 0x7fef98144f80 2024-05-28T14:12:42.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.311+0000 7fef96a00640 1 -- 172.21.15.42:0/4085664399 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef880ad640 con 0x7fef98144f80 2024-05-28T14:12:42.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.311+0000 7fef96a00640 1 -- 172.21.15.42:0/4085664399 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef880ada10 con 0x7fef98144f80 2024-05-28T14:12:42.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 -- 172.21.15.42:0/4085664399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef98144f80 msgr2=0x7fef98072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 --2- 172.21.15.42:0/4085664399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef98144f80 0x7fef98072960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fef880097c0 tx=0x7fef8801c4a0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 -- 172.21.15.42:0/4085664399 shutdown_connections 2024-05-28T14:12:42.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 --2- 172.21.15.42:0/4085664399 >> v2:172.21.15.63:3300/0 conn(0x7fef9815c0c0 0x7fef9806f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 --2- 172.21.15.42:0/4085664399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef98144f80 0x7fef98072960 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 -- 172.21.15.42:0/4085664399 >> 172.21.15.42:0/4085664399 conn(0x7fef980b21f0 msgr2=0x7fef980b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:42.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 -- 172.21.15.42:0/4085664399 shutdown_connections 2024-05-28T14:12:42.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 -- 172.21.15.42:0/4085664399 wait complete. 2024-05-28T14:12:42.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 Processor -- start 2024-05-28T14:12:42.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 -- start start 2024-05-28T14:12:42.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9806ee20 0x7fef981f7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:42.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef98144f80 0x7fef980b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:42.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef9815c0c0 0x7fef98130690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:42.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fef98072ea0 con 0x7fef9815c0c0 2024-05-28T14:12:42.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef980b4010 con 0x7fef9806ee20 2024-05-28T14:12:42.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef9e600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef9813eac0 con 0x7fef98144f80 2024-05-28T14:12:42.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef97e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef9815c0c0 0x7fef98130690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:42.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef97e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef9815c0c0 0x7fef98130690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33354/0 (socket says 172.21.15.42:33354) 2024-05-28T14:12:42.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef97e00640 1 -- 172.21.15.42:0/3688113608 learned_addr learned my addr 172.21.15.42:0/3688113608 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef97400640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9806ee20 0x7fef981f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef8ea00640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef98144f80 0x7fef980b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef97e00640 1 -- 172.21.15.42:0/3688113608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef98144f80 msgr2=0x7fef980b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef97e00640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef98144f80 0x7fef980b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef97e00640 1 -- 172.21.15.42:0/3688113608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9806ee20 msgr2=0x7fef981f7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef97e00640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9806ee20 0x7fef981f7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef97e00640 1 -- 172.21.15.42:0/3688113608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9813eda0 con 0x7fef9815c0c0 2024-05-28T14:12:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef97400640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9806ee20 0x7fef981f7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef8ea00640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef98144f80 0x7fef980b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:42.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.315+0000 7fef97e00640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef9815c0c0 0x7fef98130690 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fef90009cb0 tx=0x7fef9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:42.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.319+0000 7fef8fe00640 1 -- 172.21.15.42:0/3688113608 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90018020 con 0x7fef9815c0c0 2024-05-28T14:12:42.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.319+0000 7fef8fe00640 1 -- 172.21.15.42:0/3688113608 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef90014480 con 0x7fef9815c0c0 2024-05-28T14:12:42.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.319+0000 7fef9e600640 1 -- 172.21.15.42:0/3688113608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9813f080 con 0x7fef9815c0c0 2024-05-28T14:12:42.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.319+0000 7fef8fe00640 1 -- 172.21.15.42:0/3688113608 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90014870 con 0x7fef9815c0c0 2024-05-28T14:12:42.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.319+0000 7fef9e600640 1 -- 172.21.15.42:0/3688113608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef980b4b70 con 0x7fef9815c0c0 2024-05-28T14:12:42.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.319+0000 7fef9e600640 1 -- 172.21.15.42:0/3688113608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef980738b0 con 0x7fef9815c0c0 2024-05-28T14:12:42.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.319+0000 7fef8fe00640 1 -- 172.21.15.42:0/3688113608 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fef9001b020 con 0x7fef9815c0c0 2024-05-28T14:12:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.319+0000 7fef8fe00640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef74069b20 0x7fef7406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.319+0000 7fef8fe00640 1 -- 172.21.15.42:0/3688113608 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fef900864e0 con 0x7fef9815c0c0 2024-05-28T14:12:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.319+0000 7fef97400640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef74069b20 0x7fef7406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:42.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.323+0000 7fef97400640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef74069b20 0x7fef7406bfe0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fef880af000 tx=0x7fef88008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:42.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.323+0000 7fef8fe00640 1 -- 172.21.15.42:0/3688113608 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fef90022030 con 0x7fef9815c0c0 2024-05-28T14:12:42.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.491+0000 7fef9e600640 1 -- 172.21.15.42:0/3688113608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef9813f530 con 0x7fef9815c0c0 2024-05-28T14:12:42.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.495+0000 7fef8fe00640 1 -- 172.21.15.42:0/3688113608 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7fef90010040 con 0x7fef9815c0c0 2024-05-28T14:12:42.478 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:12:42.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.495+0000 7fef9e600640 1 -- 172.21.15.42:0/3688113608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef74069b20 msgr2=0x7fef7406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.495+0000 7fef9e600640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef74069b20 0x7fef7406bfe0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fef880af000 tx=0x7fef88008040 comp rx=0 tx=0).stop 2024-05-28T14:12:42.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.495+0000 7fef9e600640 1 -- 172.21.15.42:0/3688113608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef9815c0c0 msgr2=0x7fef98130690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.495+0000 7fef9e600640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef9815c0c0 0x7fef98130690 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fef90009cb0 tx=0x7fef9000c490 comp rx=0 tx=0).stop 2024-05-28T14:12:42.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.495+0000 7fef9e600640 1 -- 172.21.15.42:0/3688113608 shutdown_connections 2024-05-28T14:12:42.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.495+0000 7fef9e600640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef74069b20 0x7fef7406bfe0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.495+0000 7fef9e600640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef9815c0c0 0x7fef98130690 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.495+0000 7fef9e600640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef98144f80 0x7fef980b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.495+0000 7fef9e600640 1 --2- 172.21.15.42:0/3688113608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9806ee20 0x7fef981f7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.495+0000 7fef9e600640 1 -- 172.21.15.42:0/3688113608 >> 172.21.15.42:0/3688113608 conn(0x7fef980b21f0 msgr2=0x7fef9815d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:42.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.495+0000 7fef9e600640 1 -- 172.21.15.42:0/3688113608 shutdown_connections 2024-05-28T14:12:42.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.495+0000 7fef9e600640 1 -- 172.21.15.42:0/3688113608 wait complete. 2024-05-28T14:12:42.500 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:12:42.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bc2e00640 1 Processor -- start 2024-05-28T14:12:42.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bc2e00640 1 -- start start 2024-05-28T14:12:42.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bc2e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc071de0 0x7f1bbc0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:42.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bc2e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1bbc13fd10 0x7f1bbc1401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:42.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bc2e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bbc072700 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bc2e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1bbc160230 con 0x7f1bbc13fd10 2024-05-28T14:12:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bc2e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1bbc160510 con 0x7f1bbc071de0 2024-05-28T14:12:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bc0e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1bbc13fd10 0x7f1bbc1401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bb3400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1bbc0ce3b0 0x7f1bbc0ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39862/0 (socket says 172.21.15.42:39862) 2024-05-28T14:12:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bb3400640 1 -- 172.21.15.42:0/985780298 learned_addr learned my addr 172.21.15.42:0/985780298 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bbbe00640 1 --2- 172.21.15.42:0/985780298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc071de0 0x7f1bbc0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bc0e00640 1 -- 172.21.15.42:0/985780298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc071de0 msgr2=0x7f1bbc0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bc0e00640 1 --2- 172.21.15.42:0/985780298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc071de0 0x7f1bbc0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bc0e00640 1 -- 172.21.15.42:0/985780298 >> v1:172.21.15.42:6789/0 conn(0x7f1bbc0ce3b0 legacy=0x7f1bbc0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bc0e00640 1 -- 172.21.15.42:0/985780298 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc1607f0 con 0x7f1bbc13fd10 2024-05-28T14:12:42.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bbbe00640 1 --2- 172.21.15.42:0/985780298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc071de0 0x7f1bbc0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:42.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.659+0000 7f1bbb400640 1 -- 172.21.15.42:0/985780298 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1494463853 0 0) 0x7f1bbc072700 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.663+0000 7f1bc0e00640 1 --2- 172.21.15.42:0/985780298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bbc13fd10 0x7f1bbc1401a0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009d90 tx=0x7f1bb401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2040edfb0c8cdff0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:42.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.663+0000 7f1bbb400640 1 -- 172.21.15.42:0/985780298 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb40b2070 con 0x7f1bbc13fd10 2024-05-28T14:12:42.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.663+0000 7f1bbb400640 1 -- 172.21.15.42:0/985780298 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb40a5710 con 0x7f1bbc13fd10 2024-05-28T14:12:42.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.663+0000 7f1bbb400640 1 -- 172.21.15.42:0/985780298 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb40a5ae0 con 0x7f1bbc13fd10 2024-05-28T14:12:42.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.663+0000 7f1bc2e00640 1 -- 172.21.15.42:0/985780298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bbc13fd10 msgr2=0x7f1bbc1401a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.663+0000 7f1bc2e00640 1 --2- 172.21.15.42:0/985780298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bbc13fd10 0x7f1bbc1401a0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009d90 tx=0x7f1bb401c850 comp rx=0 tx=0).stop 2024-05-28T14:12:42.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.663+0000 7f1bc2e00640 1 -- 172.21.15.42:0/985780298 shutdown_connections 2024-05-28T14:12:42.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.663+0000 7f1bc2e00640 1 --2- 172.21.15.42:0/985780298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bbc13fd10 0x7f1bbc1401a0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.663+0000 7f1bc2e00640 1 --2- 172.21.15.42:0/985780298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc071de0 0x7f1bbc0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.663+0000 7f1bc2e00640 1 -- 172.21.15.42:0/985780298 >> 172.21.15.42:0/985780298 conn(0x7f1bbc0b21f0 msgr2=0x7f1bbc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:42.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.663+0000 7f1bc2e00640 1 -- 172.21.15.42:0/985780298 shutdown_connections 2024-05-28T14:12:42.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.663+0000 7f1bc2e00640 1 -- 172.21.15.42:0/985780298 wait complete. 2024-05-28T14:12:42.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bc2e00640 1 Processor -- start 2024-05-28T14:12:42.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bc2e00640 1 -- start start 2024-05-28T14:12:42.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bc2e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bbc071de0 0x7f1bbc1554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:42.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bc2e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bbc0ce3b0 0x7f1bbc06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:42.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bc2e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc13fd10 0x7f1bbc15d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:42.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bc2e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc169830 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bc2e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1696b0 con 0x7f1bbc071de0 2024-05-28T14:12:42.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bc2e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1bbc1699b0 con 0x7f1bbc13fd10 2024-05-28T14:12:42.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bb3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bbc0ce3b0 0x7f1bbc06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:42.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bb3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bbc0ce3b0 0x7f1bbc06e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33376/0 (socket says 172.21.15.42:33376) 2024-05-28T14:12:42.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bb3400640 1 -- 172.21.15.42:0/711749661 learned_addr learned my addr 172.21.15.42:0/711749661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:42.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bc0e00640 1 --2- 172.21.15.42:0/711749661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc13fd10 0x7f1bbc15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:42.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bb3400640 1 -- 172.21.15.42:0/711749661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc13fd10 msgr2=0x7f1bbc15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bb3400640 1 --2- 172.21.15.42:0/711749661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc13fd10 0x7f1bbc15d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bb3400640 1 -- 172.21.15.42:0/711749661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bbc071de0 msgr2=0x7f1bbc1554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bb3400640 1 --2- 172.21.15.42:0/711749661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bbc071de0 0x7f1bbc1554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bb3400640 1 -- 172.21.15.42:0/711749661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc0b38e0 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bb3400640 1 --2- 172.21.15.42:0/711749661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bbc0ce3b0 0x7f1bbc06e6e0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40011f0 tx=0x7f1ba400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:42.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bb8c00640 1 -- 172.21.15.42:0/711749661 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4018070 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bb8c00640 1 -- 172.21.15.42:0/711749661 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba40074e0 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bc2e00640 1 -- 172.21.15.42:0/711749661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bbc06efc0 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bb8c00640 1 -- 172.21.15.42:0/711749661 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4004a10 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.667+0000 7f1bc2e00640 1 -- 172.21.15.42:0/711749661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bbc06f580 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.671+0000 7f1bc2e00640 1 -- 172.21.15.42:0/711749661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bbc0738b0 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.671+0000 7f1bb8c00640 1 -- 172.21.15.42:0/711749661 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f1ba401b070 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.671+0000 7f1bb8c00640 1 --2- 172.21.15.42:0/711749661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b98069b20 0x7f1b9806bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:42.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.671+0000 7f1bb8c00640 1 -- 172.21.15.42:0/711749661 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f1ba4084a30 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.671+0000 7f1bbbe00640 1 --2- 172.21.15.42:0/711749661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b98069b20 0x7f1b9806bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:42.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.671+0000 7f1bbbe00640 1 --2- 172.21.15.42:0/711749661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b98069b20 0x7f1b9806bfe0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1bac006f10 tx=0x7f1bac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:42.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.675+0000 7f1bb8c00640 1 -- 172.21.15.42:0/711749661 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ba4056af0 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.839+0000 7f1bc2e00640 1 -- 172.21.15.42:0/711749661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1bbc15b3e0 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.839+0000 7f1bb8c00640 1 -- 172.21.15.42:0/711749661 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7f1ba40563f0 con 0x7f1bbc0ce3b0 2024-05-28T14:12:42.825 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:12:42.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.843+0000 7f1bc2e00640 1 -- 172.21.15.42:0/711749661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b98069b20 msgr2=0x7f1b9806bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.843+0000 7f1bc2e00640 1 --2- 172.21.15.42:0/711749661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b98069b20 0x7f1b9806bfe0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1bac006f10 tx=0x7f1bac008040 comp rx=0 tx=0).stop 2024-05-28T14:12:42.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.843+0000 7f1bc2e00640 1 -- 172.21.15.42:0/711749661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bbc0ce3b0 msgr2=0x7f1bbc06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.843+0000 7f1bc2e00640 1 --2- 172.21.15.42:0/711749661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bbc0ce3b0 0x7f1bbc06e6e0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40011f0 tx=0x7f1ba400c9b0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.843+0000 7f1bc2e00640 1 -- 172.21.15.42:0/711749661 shutdown_connections 2024-05-28T14:12:42.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.843+0000 7f1bc2e00640 1 --2- 172.21.15.42:0/711749661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b98069b20 0x7f1b9806bfe0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.843+0000 7f1bc2e00640 1 --2- 172.21.15.42:0/711749661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc13fd10 0x7f1bbc15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.843+0000 7f1bc2e00640 1 --2- 172.21.15.42:0/711749661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bbc0ce3b0 0x7f1bbc06e6e0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.843+0000 7f1bc2e00640 1 --2- 172.21.15.42:0/711749661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bbc071de0 0x7f1bbc1554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.843+0000 7f1bc2e00640 1 -- 172.21.15.42:0/711749661 >> 172.21.15.42:0/711749661 conn(0x7f1bbc0b21f0 msgr2=0x7f1bbc0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:42.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.843+0000 7f1bc2e00640 1 -- 172.21.15.42:0/711749661 shutdown_connections 2024-05-28T14:12:42.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:42.843+0000 7f1bc2e00640 1 -- 172.21.15.42:0/711749661 wait complete. 2024-05-28T14:12:42.846 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:12:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.003+0000 7fe895800640 1 Processor -- start 2024-05-28T14:12:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe895800640 1 -- start start 2024-05-28T14:12:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe895800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe89015e8c0 0x7fe8900b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe895800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8900b0650 0x7fe89006b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe895800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8900b4440 con 0x7fe8900b0a90 2024-05-28T14:12:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe895800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe8900ae420 con 0x7fe89015e8c0 2024-05-28T14:12:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe895800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe8900b0f10 con 0x7fe8900b0650 2024-05-28T14:12:42.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe886000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8900b0650 0x7fe89006b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:42.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe886000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8900b0650 0x7fe89006b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:35378/0 (socket says 172.21.15.42:35378) 2024-05-28T14:12:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe886000640 1 -- 172.21.15.42:0/2342146810 learned_addr learned my addr 172.21.15.42:0/2342146810 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe88ea00640 1 --2- 172.21.15.42:0/2342146810 >> v2:172.21.15.63:3300/0 conn(0x7fe89015e8c0 0x7fe8900b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe886000640 1 -- 172.21.15.42:0/2342146810 >> v2:172.21.15.63:3300/0 conn(0x7fe89015e8c0 msgr2=0x7fe8900b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe886000640 1 --2- 172.21.15.42:0/2342146810 >> v2:172.21.15.63:3300/0 conn(0x7fe89015e8c0 0x7fe8900b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe886000640 1 -- 172.21.15.42:0/2342146810 >> v1:172.21.15.63:6789/0 conn(0x7fe8900b0a90 legacy=0x7fe8900cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe886000640 1 -- 172.21.15.42:0/2342146810 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8900ae670 con 0x7fe8900b0650 2024-05-28T14:12:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.007+0000 7fe88ea00640 1 --2- 172.21.15.42:0/2342146810 >> v2:172.21.15.63:3300/0 conn(0x7fe89015e8c0 0x7fe8900b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:12:42.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.011+0000 7fe886000640 1 --2- 172.21.15.42:0/2342146810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8900b0650 0x7fe89006b9f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe8780098a0 tx=0x7fe87801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c813c36572b6c98f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:42.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.011+0000 7fe88e000640 1 -- 172.21.15.42:0/2342146810 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780b0070 con 0x7fe8900b0650 2024-05-28T14:12:42.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.011+0000 7fe88e000640 1 -- 172.21.15.42:0/2342146810 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8780ab600 con 0x7fe8900b0650 2024-05-28T14:12:42.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.011+0000 7fe88e000640 1 -- 172.21.15.42:0/2342146810 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780ab9d0 con 0x7fe8900b0650 2024-05-28T14:12:42.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.011+0000 7fe895800640 1 -- 172.21.15.42:0/2342146810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8900b0650 msgr2=0x7fe89006b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:42.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.011+0000 7fe895800640 1 --2- 172.21.15.42:0/2342146810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8900b0650 0x7fe89006b9f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe8780098a0 tx=0x7fe87801c480 comp rx=0 tx=0).stop 2024-05-28T14:12:42.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.011+0000 7fe895800640 1 -- 172.21.15.42:0/2342146810 shutdown_connections 2024-05-28T14:12:42.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.011+0000 7fe895800640 1 --2- 172.21.15.42:0/2342146810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8900b0650 0x7fe89006b9f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.011+0000 7fe895800640 1 --2- 172.21.15.42:0/2342146810 >> v2:172.21.15.63:3300/0 conn(0x7fe89015e8c0 0x7fe8900b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:42.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.011+0000 7fe895800640 1 -- 172.21.15.42:0/2342146810 >> 172.21.15.42:0/2342146810 conn(0x7fe8900b6960 msgr2=0x7fe8900b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:42.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.011+0000 7fe895800640 1 -- 172.21.15.42:0/2342146810 shutdown_connections 2024-05-28T14:12:42.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.011+0000 7fe895800640 1 -- 172.21.15.42:0/2342146810 wait complete. 2024-05-28T14:12:42.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.011+0000 7fe895800640 1 Processor -- start 2024-05-28T14:12:42.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.011+0000 7fe895800640 1 -- start start 2024-05-28T14:12:43.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe895800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8900b0650 0x7fe8900b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:43.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe895800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8900b0a90 0x7fe89014a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:43.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe895800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe89015e8c0 0x7fe8901523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:43.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe895800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe89015ee20 con 0x7fe8900b0650 2024-05-28T14:12:43.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe895800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe89015eca0 con 0x7fe8900b0a90 2024-05-28T14:12:43.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe895800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe89016b520 con 0x7fe89015e8c0 2024-05-28T14:12:43.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe88ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8900b0650 0x7fe8900b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe88ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8900b0650 0x7fe8900b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33392/0 (socket says 172.21.15.42:33392) 2024-05-28T14:12:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe88ea00640 1 -- 172.21.15.42:0/3573497489 learned_addr learned my addr 172.21.15.42:0/3573497489 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe886000640 1 --2- 172.21.15.42:0/3573497489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8900b0a90 0x7fe89014a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe88f400640 1 --2- 172.21.15.42:0/3573497489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe89015e8c0 0x7fe8901523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe88ea00640 1 -- 172.21.15.42:0/3573497489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe89015e8c0 msgr2=0x7fe8901523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe88ea00640 1 --2- 172.21.15.42:0/3573497489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe89015e8c0 0x7fe8901523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe88ea00640 1 -- 172.21.15.42:0/3573497489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8900b0a90 msgr2=0x7fe89014a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe88ea00640 1 --2- 172.21.15.42:0/3573497489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8900b0a90 0x7fe89014a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe88ea00640 1 -- 172.21.15.42:0/3573497489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8900b5e20 con 0x7fe8900b0650 2024-05-28T14:12:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe88ea00640 1 --2- 172.21.15.42:0/3573497489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8900b0650 0x7fe8900b58e0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fe88000e9b0 tx=0x7fe88000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe887400640 1 -- 172.21.15.42:0/3573497489 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880017070 con 0x7fe8900b0650 2024-05-28T14:12:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe887400640 1 -- 172.21.15.42:0/3573497489 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe88000ca00 con 0x7fe8900b0650 2024-05-28T14:12:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe895800640 1 -- 172.21.15.42:0/3573497489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89014b110 con 0x7fe8900b0650 2024-05-28T14:12:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe887400640 1 -- 172.21.15.42:0/3573497489 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88000cdd0 con 0x7fe8900b0650 2024-05-28T14:12:43.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe895800640 1 -- 172.21.15.42:0/3573497489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe890151be0 con 0x7fe8900b0650 2024-05-28T14:12:43.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.015+0000 7fe895800640 1 -- 172.21.15.42:0/3573497489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe89009f7a0 con 0x7fe8900b0650 2024-05-28T14:12:43.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.019+0000 7fe887400640 1 -- 172.21.15.42:0/3573497489 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fe88001a070 con 0x7fe8900b0650 2024-05-28T14:12:43.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.019+0000 7fe887400640 1 --2- 172.21.15.42:0/3573497489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe86c069b50 0x7fe86c06c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:43.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.019+0000 7fe887400640 1 -- 172.21.15.42:0/3573497489 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fe880085640 con 0x7fe8900b0650 2024-05-28T14:12:43.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.019+0000 7fe886000640 1 --2- 172.21.15.42:0/3573497489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe86c069b50 0x7fe86c06c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:43.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.019+0000 7fe886000640 1 --2- 172.21.15.42:0/3573497489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe86c069b50 0x7fe86c06c010 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe8780ad000 tx=0x7fe878008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:43.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.023+0000 7fe887400640 1 -- 172.21.15.42:0/3573497489 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe880056f80 con 0x7fe8900b0650 2024-05-28T14:12:43.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.187+0000 7fe895800640 1 -- 172.21.15.42:0/3573497489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe8901568c0 con 0x7fe8900b0650 2024-05-28T14:12:43.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.187+0000 7fe887400640 1 -- 172.21.15.42:0/3573497489 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7fe88005b280 con 0x7fe8900b0650 2024-05-28T14:12:43.172 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:12:43.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.187+0000 7fe895800640 1 -- 172.21.15.42:0/3573497489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe86c069b50 msgr2=0x7fe86c06c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:43.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.187+0000 7fe895800640 1 --2- 172.21.15.42:0/3573497489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe86c069b50 0x7fe86c06c010 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe8780ad000 tx=0x7fe878008040 comp rx=0 tx=0).stop 2024-05-28T14:12:43.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.187+0000 7fe895800640 1 -- 172.21.15.42:0/3573497489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8900b0650 msgr2=0x7fe8900b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:43.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.187+0000 7fe895800640 1 --2- 172.21.15.42:0/3573497489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8900b0650 0x7fe8900b58e0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fe88000e9b0 tx=0x7fe88000ee70 comp rx=0 tx=0).stop 2024-05-28T14:12:43.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.187+0000 7fe895800640 1 -- 172.21.15.42:0/3573497489 shutdown_connections 2024-05-28T14:12:43.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.187+0000 7fe895800640 1 --2- 172.21.15.42:0/3573497489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe86c069b50 0x7fe86c06c010 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.187+0000 7fe895800640 1 --2- 172.21.15.42:0/3573497489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe89015e8c0 0x7fe8901523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.187+0000 7fe895800640 1 --2- 172.21.15.42:0/3573497489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8900b0a90 0x7fe89014a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.187+0000 7fe895800640 1 --2- 172.21.15.42:0/3573497489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8900b0650 0x7fe8900b58e0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.187+0000 7fe895800640 1 -- 172.21.15.42:0/3573497489 >> 172.21.15.42:0/3573497489 conn(0x7fe8900b6960 msgr2=0x7fe89015d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:43.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.191+0000 7fe895800640 1 -- 172.21.15.42:0/3573497489 shutdown_connections 2024-05-28T14:12:43.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.191+0000 7fe895800640 1 -- 172.21.15.42:0/3573497489 wait complete. 2024-05-28T14:12:43.194 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T14:12:43.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.351+0000 7f0fbc200640 1 Processor -- start 2024-05-28T14:12:43.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.351+0000 7f0fbc200640 1 -- start start 2024-05-28T14:12:43.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fbc200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0fb4155a00 0x7f0fb406ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:43.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fbc200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0fb406cf40 0x7f0fb406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:43.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fbc200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fb406fc20 con 0x7f0fb406f3f0 2024-05-28T14:12:43.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fbc200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0fb406d380 con 0x7f0fb4155a00 2024-05-28T14:12:43.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fbc200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb415ff90 con 0x7f0fb406cf40 2024-05-28T14:12:43.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fb9800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0fb4155a00 0x7f0fb406ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:43.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fb8e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0fb406cf40 0x7f0fb406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:43.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fb9800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0fb4155a00 0x7f0fb406ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:57782/0 (socket says 172.21.15.42:57782) 2024-05-28T14:12:43.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fb9800640 1 -- 172.21.15.42:0/902111995 learned_addr learned my addr 172.21.15.42:0/902111995 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:43.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fba200640 1 --1- 172.21.15.42:0/902111995 >> v1:172.21.15.63:6789/0 conn(0x7f0fb406f3f0 0x7f0fb41623c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:41728/0 (socket says 172.21.15.42:41728) 2024-05-28T14:12:43.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fb8e00640 1 -- 172.21.15.42:0/902111995 >> v2:172.21.15.63:3300/0 conn(0x7f0fb4155a00 msgr2=0x7f0fb406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:43.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fb8e00640 1 --2- 172.21.15.42:0/902111995 >> v2:172.21.15.63:3300/0 conn(0x7f0fb4155a00 0x7f0fb406ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fb8e00640 1 -- 172.21.15.42:0/902111995 >> v1:172.21.15.63:6789/0 conn(0x7f0fb406f3f0 legacy=0x7f0fb41623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fb8e00640 1 -- 172.21.15.42:0/902111995 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb4160270 con 0x7f0fb406cf40 2024-05-28T14:12:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.355+0000 7f0fb9800640 1 --2- 172.21.15.42:0/902111995 >> v2:172.21.15.63:3300/0 conn(0x7f0fb4155a00 0x7f0fb406ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:12:43.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.359+0000 7f0fb8e00640 1 --2- 172.21.15.42:0/902111995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0fb406cf40 0x7f0fb406eeb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00097c0 tx=0x7f0fa001c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edd3f8ba3790d9da server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:43.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.359+0000 7f0fb3e00640 1 -- 172.21.15.42:0/902111995 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa00b0070 con 0x7f0fb406cf40 2024-05-28T14:12:43.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.359+0000 7f0fb3e00640 1 -- 172.21.15.42:0/902111995 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa00ab600 con 0x7f0fb406cf40 2024-05-28T14:12:43.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.359+0000 7f0fb3e00640 1 -- 172.21.15.42:0/902111995 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa00ab9d0 con 0x7f0fb406cf40 2024-05-28T14:12:43.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.359+0000 7f0fbc200640 1 -- 172.21.15.42:0/902111995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0fb406cf40 msgr2=0x7f0fb406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:43.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.359+0000 7f0fbc200640 1 --2- 172.21.15.42:0/902111995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0fb406cf40 0x7f0fb406eeb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00097c0 tx=0x7f0fa001c330 comp rx=0 tx=0).stop 2024-05-28T14:12:43.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.359+0000 7f0fbc200640 1 -- 172.21.15.42:0/902111995 shutdown_connections 2024-05-28T14:12:43.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.359+0000 7f0fbc200640 1 --2- 172.21.15.42:0/902111995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0fb406cf40 0x7f0fb406eeb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.359+0000 7f0fbc200640 1 --2- 172.21.15.42:0/902111995 >> v2:172.21.15.63:3300/0 conn(0x7f0fb4155a00 0x7f0fb406ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.359+0000 7f0fbc200640 1 -- 172.21.15.42:0/902111995 >> 172.21.15.42:0/902111995 conn(0x7f0fb40725e0 msgr2=0x7f0fb40729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:43.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.359+0000 7f0fbc200640 1 -- 172.21.15.42:0/902111995 shutdown_connections 2024-05-28T14:12:43.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.359+0000 7f0fbc200640 1 -- 172.21.15.42:0/902111995 wait complete. 2024-05-28T14:12:43.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.359+0000 7f0fbc200640 1 Processor -- start 2024-05-28T14:12:43.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.359+0000 7f0fbc200640 1 -- start start 2024-05-28T14:12:43.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fbc200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fb406cf40 0x7f0fb4157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:43.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fbc200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fb406f3f0 0x7f0fb41400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:43.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fbc200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0fb4155a00 0x7f0fb4143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:43.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fbc200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb406f880 con 0x7f0fb406cf40 2024-05-28T14:12:43.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fbc200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb4160830 con 0x7f0fb406f3f0 2024-05-28T14:12:43.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fbc200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb406fa00 con 0x7f0fb4155a00 2024-05-28T14:12:43.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fb9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fb406cf40 0x7f0fb4157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:43.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fb9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fb406cf40 0x7f0fb4157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33406/0 (socket says 172.21.15.42:33406) 2024-05-28T14:12:43.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fb9800640 1 -- 172.21.15.42:0/1846734136 learned_addr learned my addr 172.21.15.42:0/1846734136 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:43.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fba200640 1 --2- 172.21.15.42:0/1846734136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0fb4155a00 0x7f0fb4143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:43.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fb8e00640 1 --2- 172.21.15.42:0/1846734136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fb406f3f0 0x7f0fb41400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:43.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fb9800640 1 -- 172.21.15.42:0/1846734136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0fb4155a00 msgr2=0x7f0fb4143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:43.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fb9800640 1 --2- 172.21.15.42:0/1846734136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0fb4155a00 0x7f0fb4143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fb9800640 1 -- 172.21.15.42:0/1846734136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fb406f3f0 msgr2=0x7f0fb41400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:43.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fb9800640 1 --2- 172.21.15.42:0/1846734136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fb406f3f0 0x7f0fb41400d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fb9800640 1 -- 172.21.15.42:0/1846734136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb4143870 con 0x7f0fb406cf40 2024-05-28T14:12:43.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fb9800640 1 --2- 172.21.15.42:0/1846734136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fb406cf40 0x7f0fb4157d60 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800e930 tx=0x7f0fa800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:43.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fb1600640 1 -- 172.21.15.42:0/1846734136 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8017070 con 0x7f0fb406cf40 2024-05-28T14:12:43.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fb1600640 1 -- 172.21.15.42:0/1846734136 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa800ca00 con 0x7f0fb406cf40 2024-05-28T14:12:43.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fb1600640 1 -- 172.21.15.42:0/1846734136 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa800cdd0 con 0x7f0fb406cf40 2024-05-28T14:12:43.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fbc200640 1 -- 172.21.15.42:0/1846734136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb4158460 con 0x7f0fb406cf40 2024-05-28T14:12:43.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fbc200640 1 -- 172.21.15.42:0/1846734136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb4158a20 con 0x7f0fb406cf40 2024-05-28T14:12:43.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fbc200640 1 -- 172.21.15.42:0/1846734136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb4068310 con 0x7f0fb406cf40 2024-05-28T14:12:43.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.363+0000 7f0fb1600640 1 -- 172.21.15.42:0/1846734136 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f0fa801a070 con 0x7f0fb406cf40 2024-05-28T14:12:43.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.367+0000 7f0fb1600640 1 --2- 172.21.15.42:0/1846734136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0f94069b20 0x7f0f9406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:43.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.367+0000 7f0fb1600640 1 -- 172.21.15.42:0/1846734136 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f0fa8085690 con 0x7f0fb406cf40 2024-05-28T14:12:43.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.367+0000 7f0fb8e00640 1 --2- 172.21.15.42:0/1846734136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0f94069b20 0x7f0f9406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:43.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.367+0000 7f0fb8e00640 1 --2- 172.21.15.42:0/1846734136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0f94069b20 0x7f0f9406bfe0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00ad000 tx=0x7f0fa00a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:43.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.371+0000 7f0fb1600640 1 -- 172.21.15.42:0/1846734136 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0fb4143870 con 0x7f0fb406cf40 2024-05-28T14:12:43.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.607+0000 7f0fbc200640 1 -- 172.21.15.42:0/1846734136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0fb4073f20 con 0x7f0fb406cf40 2024-05-28T14:12:43.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.607+0000 7f0fb1600640 1 -- 172.21.15.42:0/1846734136 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7f0fa80576b0 con 0x7f0fb406cf40 2024-05-28T14:12:43.591 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T14:12:43.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.611+0000 7f0fbc200640 1 -- 172.21.15.42:0/1846734136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0f94069b20 msgr2=0x7f0f9406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:43.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.611+0000 7f0fbc200640 1 --2- 172.21.15.42:0/1846734136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0f94069b20 0x7f0f9406bfe0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00ad000 tx=0x7f0fa00a6000 comp rx=0 tx=0).stop 2024-05-28T14:12:43.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.611+0000 7f0fbc200640 1 -- 172.21.15.42:0/1846734136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fb406cf40 msgr2=0x7f0fb4157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:43.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.611+0000 7f0fbc200640 1 --2- 172.21.15.42:0/1846734136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fb406cf40 0x7f0fb4157d60 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800e930 tx=0x7f0fa800edf0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.611+0000 7f0fbc200640 1 -- 172.21.15.42:0/1846734136 shutdown_connections 2024-05-28T14:12:43.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.611+0000 7f0fbc200640 1 --2- 172.21.15.42:0/1846734136 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0f94069b20 0x7f0f9406bfe0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.611+0000 7f0fbc200640 1 --2- 172.21.15.42:0/1846734136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0fb4155a00 0x7f0fb4143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.611+0000 7f0fbc200640 1 --2- 172.21.15.42:0/1846734136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fb406f3f0 0x7f0fb41400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.611+0000 7f0fbc200640 1 --2- 172.21.15.42:0/1846734136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fb406cf40 0x7f0fb4157d60 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:43.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.611+0000 7f0fbc200640 1 -- 172.21.15.42:0/1846734136 >> 172.21.15.42:0/1846734136 conn(0x7f0fb40725e0 msgr2=0x7f0fb40b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:43.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.611+0000 7f0fbc200640 1 -- 172.21.15.42:0/1846734136 shutdown_connections 2024-05-28T14:12:43.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:43.615+0000 7f0fbc200640 1 -- 172.21.15.42:0/1846734136 wait complete. 2024-05-28T14:12:43.632 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-05-28T14:12:43.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:12:43.633 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-28T14:12:43.673 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[50525]: fuse finished with error 0 and tester_r 0 2024-05-28T14:12:43.675 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:12:43.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:12:43.675 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-28T14:12:43.864 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:12:43.865 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:12:43.865 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:12:43.879 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:12:43.879 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:12:43.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:12:43.879 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-28T14:12:43.932 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-28T14:12:43.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:12:43.933 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:12:43.933 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:12:43.933 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:12:43.944 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:12:43.944 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:12:43.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:12:43.944 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-28T14:12:43.993 INFO:teuthology.orchestra.run.smithi042.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-28T14:12:43.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:12:43.994 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:12:43.994 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:12:43.994 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:12:44.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:12:44.003 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:12:44.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:12:44.003 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-28T14:12:44.052 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-28T14:12:44.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:12:44.053 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:12:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.207+0000 7f8bcea00640 1 Processor -- start 2024-05-28T14:12:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.207+0000 7f8bcea00640 1 -- start start 2024-05-28T14:12:44.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.211+0000 7f8bcea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8bc815e5c0 0x7f8bc80b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.211+0000 7f8bcea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8bc80d0e00 0x7f8bc80ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.211+0000 7f8bcea00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bc80b3720 con 0x7f8bc80b51b0 2024-05-28T14:12:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.211+0000 7f8bcea00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8bc80b39f0 con 0x7f8bc80d0e00 2024-05-28T14:12:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.211+0000 7f8bcea00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8bc80b3cd0 con 0x7f8bc815e5c0 2024-05-28T14:12:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.211+0000 7f8bcca00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8bc80d0e00 0x7f8bc80ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.211+0000 7f8bcca00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8bc80d0e00 0x7f8bc80ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33416/0 (socket says 172.21.15.42:33416) 2024-05-28T14:12:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.211+0000 7f8bcca00640 1 -- 172.21.15.42:0/314371779 learned_addr learned my addr 172.21.15.42:0/314371779 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.211+0000 7f8bc7e00640 1 --2- 172.21.15.42:0/314371779 >> v2:172.21.15.63:3300/0 conn(0x7f8bc815e5c0 0x7f8bc80b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.211+0000 7f8bcca00640 1 -- 172.21.15.42:0/314371779 >> v2:172.21.15.63:3300/0 conn(0x7f8bc815e5c0 msgr2=0x7f8bc80b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.211+0000 7f8bcca00640 1 --2- 172.21.15.42:0/314371779 >> v2:172.21.15.63:3300/0 conn(0x7f8bc815e5c0 0x7f8bc80b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.211+0000 7f8bcca00640 1 -- 172.21.15.42:0/314371779 >> v1:172.21.15.63:6789/0 conn(0x7f8bc80b51b0 legacy=0x7f8bc80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:44.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.211+0000 7f8bcca00640 1 -- 172.21.15.42:0/314371779 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc80b3fb0 con 0x7f8bc80d0e00 2024-05-28T14:12:44.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.215+0000 7f8bcca00640 1 --2- 172.21.15.42:0/314371779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bc80d0e00 0x7f8bc80ad860 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80b44a0 tx=0x7f8bc001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95d2d60a32a3a4f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:44.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.215+0000 7f8bc7400640 1 -- 172.21.15.42:0/314371779 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00a5860 con 0x7f8bc80d0e00 2024-05-28T14:12:44.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.215+0000 7f8bc7400640 1 -- 172.21.15.42:0/314371779 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc00a5a00 con 0x7f8bc80d0e00 2024-05-28T14:12:44.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.215+0000 7f8bc7400640 1 -- 172.21.15.42:0/314371779 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00a5dd0 con 0x7f8bc80d0e00 2024-05-28T14:12:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.215+0000 7f8bcea00640 1 -- 172.21.15.42:0/314371779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bc80d0e00 msgr2=0x7f8bc80ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.215+0000 7f8bcea00640 1 --2- 172.21.15.42:0/314371779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bc80d0e00 0x7f8bc80ad860 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80b44a0 tx=0x7f8bc001c980 comp rx=0 tx=0).stop 2024-05-28T14:12:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.215+0000 7f8bcea00640 1 -- 172.21.15.42:0/314371779 shutdown_connections 2024-05-28T14:12:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.215+0000 7f8bcea00640 1 --2- 172.21.15.42:0/314371779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bc80d0e00 0x7f8bc80ad860 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.215+0000 7f8bcea00640 1 --2- 172.21.15.42:0/314371779 >> v2:172.21.15.63:3300/0 conn(0x7f8bc815e5c0 0x7f8bc80b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.215+0000 7f8bcea00640 1 -- 172.21.15.42:0/314371779 >> 172.21.15.42:0/314371779 conn(0x7f8bc806bdf0 msgr2=0x7f8bc806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.215+0000 7f8bcea00640 1 -- 172.21.15.42:0/314371779 shutdown_connections 2024-05-28T14:12:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.215+0000 7f8bcea00640 1 -- 172.21.15.42:0/314371779 wait complete. 2024-05-28T14:12:44.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.215+0000 7f8bcea00640 1 Processor -- start 2024-05-28T14:12:44.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.215+0000 7f8bcea00640 1 -- start start 2024-05-28T14:12:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bc80b51b0 0x7f8bc8073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bc80d0e00 0x7f8bc815ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bc815e5c0 0x7f8bc8159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc80ae940 con 0x7f8bc815e5c0 2024-05-28T14:12:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc80ae7c0 con 0x7f8bc80b51b0 2024-05-28T14:12:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcea00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc80aeac0 con 0x7f8bc80d0e00 2024-05-28T14:12:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bc815e5c0 0x7f8bc8159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bc815e5c0 0x7f8bc8159850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33424/0 (socket says 172.21.15.42:33424) 2024-05-28T14:12:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcca00640 1 -- 172.21.15.42:0/3874635085 learned_addr learned my addr 172.21.15.42:0/3874635085 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:44.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bc7e00640 1 --2- 172.21.15.42:0/3874635085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bc80b51b0 0x7f8bc8073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bbfe00640 1 --2- 172.21.15.42:0/3874635085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bc80d0e00 0x7f8bc815ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcca00640 1 -- 172.21.15.42:0/3874635085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bc80d0e00 msgr2=0x7f8bc815ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:44.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcca00640 1 --2- 172.21.15.42:0/3874635085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bc80d0e00 0x7f8bc815ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcca00640 1 -- 172.21.15.42:0/3874635085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bc80b51b0 msgr2=0x7f8bc8073690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:44.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcca00640 1 --2- 172.21.15.42:0/3874635085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bc80b51b0 0x7f8bc8073690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcca00640 1 -- 172.21.15.42:0/3874635085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc80725f0 con 0x7f8bc815e5c0 2024-05-28T14:12:44.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcca00640 1 --2- 172.21.15.42:0/3874635085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bc815e5c0 0x7f8bc8159850 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00af040 tx=0x7f8bc0019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:44.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bc4c00640 1 -- 172.21.15.42:0/3874635085 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00b8880 con 0x7f8bc815e5c0 2024-05-28T14:12:44.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bc4c00640 1 -- 172.21.15.42:0/3874635085 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc00be070 con 0x7f8bc815e5c0 2024-05-28T14:12:44.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bc4c00640 1 -- 172.21.15.42:0/3874635085 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00b8c80 con 0x7f8bc815e5c0 2024-05-28T14:12:44.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcea00640 1 -- 172.21.15.42:0/3874635085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc80728d0 con 0x7f8bc815e5c0 2024-05-28T14:12:44.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcea00640 1 -- 172.21.15.42:0/3874635085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc8072e90 con 0x7f8bc815e5c0 2024-05-28T14:12:44.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bcea00640 1 -- 172.21.15.42:0/3874635085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc8068a90 con 0x7f8bc815e5c0 2024-05-28T14:12:44.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.219+0000 7f8bc4c00640 1 -- 172.21.15.42:0/3874635085 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f8bc00c3460 con 0x7f8bc815e5c0 2024-05-28T14:12:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.223+0000 7f8bc4c00640 1 --2- 172.21.15.42:0/3874635085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ba4069b20 0x7f8ba406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.223+0000 7f8bc4c00640 1 -- 172.21.15.42:0/3874635085 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f8bc0120ac0 con 0x7f8bc815e5c0 2024-05-28T14:12:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.223+0000 7f8bc7e00640 1 --2- 172.21.15.42:0/3874635085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ba4069b20 0x7f8ba406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.223+0000 7f8bc7e00640 1 --2- 172.21.15.42:0/3874635085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ba4069b20 0x7f8ba406bfe0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800b930 tx=0x7f8bb8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:44.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.227+0000 7f8bc4c00640 1 -- 172.21.15.42:0/3874635085 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8bc00165a0 con 0x7f8bc815e5c0 2024-05-28T14:12:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.427+0000 7f8bcea00640 1 -- 172.21.15.42:0/3874635085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bc80cf810 con 0x7f8bc815e5c0 2024-05-28T14:12:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.427+0000 7f8bc4c00640 1 -- 172.21.15.42:0/3874635085 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5190 (secure 0 0 0) 0x7f8bc00b8e20 con 0x7f8bc815e5c0 2024-05-28T14:12:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:44.414 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8607,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/1661021064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1661021064},{"type":"v1","addr":"172.21.15.42:6836","nonce":1661021064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":57050,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2398440572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":2398440572},{"type":"v1","addr":"172.21.15.42:6837","nonce":2398440572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":57053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/2530315729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":2530315729},{"type":"v1","addr":"172.21.15.63:6834","nonce":2530315729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:11:14.793551+0000","modified":"2024-05-28T14:11:17.812010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57056},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57056":{"gid":57056,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/1184583947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1184583947},{"type":"v1","addr":"172.21.15.63:6835","nonce":1184583947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57056,"qdb_cluster":[57056]},"id":7}]} 2024-05-28T14:12:44.414 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 65 2024-05-28T14:12:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.431+0000 7f8bcea00640 1 -- 172.21.15.42:0/3874635085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ba4069b20 msgr2=0x7f8ba406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.431+0000 7f8bcea00640 1 --2- 172.21.15.42:0/3874635085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ba4069b20 0x7f8ba406bfe0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800b930 tx=0x7f8bb8008040 comp rx=0 tx=0).stop 2024-05-28T14:12:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.431+0000 7f8bcea00640 1 -- 172.21.15.42:0/3874635085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bc815e5c0 msgr2=0x7f8bc8159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.431+0000 7f8bcea00640 1 --2- 172.21.15.42:0/3874635085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bc815e5c0 0x7f8bc8159850 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00af040 tx=0x7f8bc0019720 comp rx=0 tx=0).stop 2024-05-28T14:12:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.431+0000 7f8bcea00640 1 -- 172.21.15.42:0/3874635085 shutdown_connections 2024-05-28T14:12:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.431+0000 7f8bcea00640 1 --2- 172.21.15.42:0/3874635085 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8ba4069b20 0x7f8ba406bfe0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.431+0000 7f8bcea00640 1 --2- 172.21.15.42:0/3874635085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bc815e5c0 0x7f8bc8159850 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.431+0000 7f8bcea00640 1 --2- 172.21.15.42:0/3874635085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bc80d0e00 0x7f8bc815ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.431+0000 7f8bcea00640 1 --2- 172.21.15.42:0/3874635085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bc80b51b0 0x7f8bc8073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.431+0000 7f8bcea00640 1 -- 172.21.15.42:0/3874635085 >> 172.21.15.42:0/3874635085 conn(0x7f8bc806bdf0 msgr2=0x7f8bc81572b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:44.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.431+0000 7f8bcea00640 1 -- 172.21.15.42:0/3874635085 shutdown_connections 2024-05-28T14:12:44.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.431+0000 7f8bcea00640 1 -- 172.21.15.42:0/3874635085 wait complete. 2024-05-28T14:12:44.437 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:12:44.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.591+0000 7f8243200640 1 Processor -- start 2024-05-28T14:12:44.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.591+0000 7f8243200640 1 -- start start 2024-05-28T14:12:44.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f8243200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824415b1e0 0x7f82440cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f8243200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f82440b4670 0x7f824406f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f8243200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82440cff30 con 0x7f824406fb30 2024-05-28T14:12:44.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f8243200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f824406ffa0 con 0x7f82440b4670 2024-05-28T14:12:44.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f8243200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82440b4fa0 con 0x7f824415b1e0 2024-05-28T14:12:44.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f823be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f82440b4670 0x7f824406f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f8242800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f824406fb30 0x7f824415ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39874/0 (socket says 172.21.15.42:39874) 2024-05-28T14:12:44.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f8242800640 1 -- 172.21.15.42:0/3452594330 learned_addr learned my addr 172.21.15.42:0/3452594330 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:44.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f8241e00640 1 --2- 172.21.15.42:0/3452594330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824415b1e0 0x7f82440cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f823be00640 1 -- 172.21.15.42:0/3452594330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824415b1e0 msgr2=0x7f82440cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:44.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f823be00640 1 --2- 172.21.15.42:0/3452594330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824415b1e0 0x7f82440cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f823be00640 1 -- 172.21.15.42:0/3452594330 >> v1:172.21.15.42:6789/0 conn(0x7f824406fb30 legacy=0x7f824415ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:44.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f823be00640 1 -- 172.21.15.42:0/3452594330 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82440d0210 con 0x7f82440b4670 2024-05-28T14:12:44.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f8241400640 1 -- 172.21.15.42:0/3452594330 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3438655067 0 0) 0x7f82440cff30 con 0x7f824406fb30 2024-05-28T14:12:44.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.595+0000 7f8241e00640 1 --2- 172.21.15.42:0/3452594330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824415b1e0 0x7f82440cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:44.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.599+0000 7f823be00640 1 --2- 172.21.15.42:0/3452594330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82440b4670 0x7f824406f5f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f822c0097c0 tx=0x7f822c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef282514ac97fce0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:44.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.599+0000 7f8241400640 1 -- 172.21.15.42:0/3452594330 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c0b0070 con 0x7f82440b4670 2024-05-28T14:12:44.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.599+0000 7f8241400640 1 -- 172.21.15.42:0/3452594330 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822c0ab600 con 0x7f82440b4670 2024-05-28T14:12:44.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.599+0000 7f8241400640 1 -- 172.21.15.42:0/3452594330 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c0ab9d0 con 0x7f82440b4670 2024-05-28T14:12:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.599+0000 7f8243200640 1 -- 172.21.15.42:0/3452594330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82440b4670 msgr2=0x7f824406f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.599+0000 7f8243200640 1 --2- 172.21.15.42:0/3452594330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82440b4670 0x7f824406f5f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f822c0097c0 tx=0x7f822c01c410 comp rx=0 tx=0).stop 2024-05-28T14:12:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.599+0000 7f8243200640 1 -- 172.21.15.42:0/3452594330 shutdown_connections 2024-05-28T14:12:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.599+0000 7f8243200640 1 --2- 172.21.15.42:0/3452594330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82440b4670 0x7f824406f5f0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.599+0000 7f8243200640 1 --2- 172.21.15.42:0/3452594330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824415b1e0 0x7f82440cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.599+0000 7f8243200640 1 -- 172.21.15.42:0/3452594330 >> 172.21.15.42:0/3452594330 conn(0x7f824406b9f0 msgr2=0x7f824406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.599+0000 7f8243200640 1 -- 172.21.15.42:0/3452594330 shutdown_connections 2024-05-28T14:12:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.599+0000 7f8243200640 1 -- 172.21.15.42:0/3452594330 wait complete. 2024-05-28T14:12:44.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.599+0000 7f8243200640 1 Processor -- start 2024-05-28T14:12:44.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.599+0000 7f8243200640 1 -- start start 2024-05-28T14:12:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8243200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824406fb30 0x7f82441580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8243200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82440b4670 0x7f824415dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8243200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f824415b1e0 0x7f82441524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8243200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f824415b740 con 0x7f824415b1e0 2024-05-28T14:12:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8243200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f824415b5c0 con 0x7f82440b4670 2024-05-28T14:12:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8243200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8244167f90 con 0x7f824406fb30 2024-05-28T14:12:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8242800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f824415b1e0 0x7f82441524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8242800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f824415b1e0 0x7f82441524d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33442/0 (socket says 172.21.15.42:33442) 2024-05-28T14:12:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8242800640 1 -- 172.21.15.42:0/2278761489 learned_addr learned my addr 172.21.15.42:0/2278761489 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8242800640 1 -- 172.21.15.42:0/2278761489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824406fb30 msgr2=0x7f82441580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8242800640 1 --2- 172.21.15.42:0/2278761489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824406fb30 0x7f82441580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8242800640 1 -- 172.21.15.42:0/2278761489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82440b4670 msgr2=0x7f824415dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f823be00640 1 --2- 172.21.15.42:0/2278761489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82440b4670 0x7f824415dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8242800640 1 --2- 172.21.15.42:0/2278761489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82440b4670 0x7f824415dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8242800640 1 -- 172.21.15.42:0/2278761489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8244152d40 con 0x7f824415b1e0 2024-05-28T14:12:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f823be00640 1 --2- 172.21.15.42:0/2278761489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82440b4670 0x7f824415dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8242800640 1 --2- 172.21.15.42:0/2278761489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f824415b1e0 0x7f82441524d0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f823c00c7d0 tx=0x7f823c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f823a000640 1 -- 172.21.15.42:0/2278761489 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c018070 con 0x7f824415b1e0 2024-05-28T14:12:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f823a000640 1 -- 172.21.15.42:0/2278761489 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f823c00a640 con 0x7f824415b1e0 2024-05-28T14:12:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8243200640 1 -- 172.21.15.42:0/2278761489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8244153020 con 0x7f824415b1e0 2024-05-28T14:12:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f823a000640 1 -- 172.21.15.42:0/2278761489 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c00aa70 con 0x7f824415b1e0 2024-05-28T14:12:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8243200640 1 -- 172.21.15.42:0/2278761489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8244152a10 con 0x7f824415b1e0 2024-05-28T14:12:44.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f8243200640 1 -- 172.21.15.42:0/2278761489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8244153540 con 0x7f824415b1e0 2024-05-28T14:12:44.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.603+0000 7f823a000640 1 -- 172.21.15.42:0/2278761489 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f823c01b070 con 0x7f824415b1e0 2024-05-28T14:12:44.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.607+0000 7f823a000640 1 --2- 172.21.15.42:0/2278761489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8220069b20 0x7f822006bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.607+0000 7f823a000640 1 -- 172.21.15.42:0/2278761489 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f823c084dc0 con 0x7f824415b1e0 2024-05-28T14:12:44.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.607+0000 7f8241e00640 1 --2- 172.21.15.42:0/2278761489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8220069b20 0x7f822006bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.607+0000 7f8241e00640 1 --2- 172.21.15.42:0/2278761489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8220069b20 0x7f822006bfe0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8234006e90 tx=0x7f8234008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:44.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.611+0000 7f823a000640 1 -- 172.21.15.42:0/2278761489 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8244153540 con 0x7f824415b1e0 2024-05-28T14:12:44.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.775+0000 7f8243200640 1 -- 172.21.15.42:0/2278761489 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f824404fcf0 con 0x7f824415b1e0 2024-05-28T14:12:44.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.775+0000 7f823a000640 1 -- 172.21.15.42:0/2278761489 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16602 (secure 0 0 0) 0x7f823c056de0 con 0x7f824415b1e0 2024-05-28T14:12:44.762 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:44.762 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":105,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:11:16.396600+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:11:13.194240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-28T14:11:13.743150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:12:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.779+0000 7f8243200640 1 -- 172.21.15.42:0/2278761489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8220069b20 msgr2=0x7f822006bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.779+0000 7f8243200640 1 --2- 172.21.15.42:0/2278761489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8220069b20 0x7f822006bfe0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8234006e90 tx=0x7f8234008040 comp rx=0 tx=0).stop 2024-05-28T14:12:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.779+0000 7f8243200640 1 -- 172.21.15.42:0/2278761489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f824415b1e0 msgr2=0x7f82441524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.779+0000 7f8243200640 1 --2- 172.21.15.42:0/2278761489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f824415b1e0 0x7f82441524d0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f823c00c7d0 tx=0x7f823c00cc90 comp rx=0 tx=0).stop 2024-05-28T14:12:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.779+0000 7f8243200640 1 -- 172.21.15.42:0/2278761489 shutdown_connections 2024-05-28T14:12:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.779+0000 7f8243200640 1 --2- 172.21.15.42:0/2278761489 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8220069b20 0x7f822006bfe0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.779+0000 7f8243200640 1 --2- 172.21.15.42:0/2278761489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f824415b1e0 0x7f82441524d0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.779+0000 7f8243200640 1 --2- 172.21.15.42:0/2278761489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82440b4670 0x7f824415dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.779+0000 7f8243200640 1 --2- 172.21.15.42:0/2278761489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824406fb30 0x7f82441580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.779+0000 7f8243200640 1 -- 172.21.15.42:0/2278761489 >> 172.21.15.42:0/2278761489 conn(0x7f824406b9f0 msgr2=0x7f8244155260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.779+0000 7f8243200640 1 -- 172.21.15.42:0/2278761489 shutdown_connections 2024-05-28T14:12:44.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.779+0000 7f8243200640 1 -- 172.21.15.42:0/2278761489 wait complete. 2024-05-28T14:12:44.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T14:11:13.194240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T14:11:13.743150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:12:44.784 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:12:44.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.943+0000 7f1f07200640 1 Processor -- start 2024-05-28T14:12:44.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.943+0000 7f1f07200640 1 -- start start 2024-05-28T14:12:44.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.943+0000 7f1f07200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0815c7a0 0x7f1f080cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.943+0000 7f1f07200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1f080cde90 0x7f1f080d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.943+0000 7f1f07200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1f0809bb20 0x7f1f0809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.943+0000 7f1f07200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1f0815cb80 con 0x7f1f0809bb20 2024-05-28T14:12:44.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.943+0000 7f1f07200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1f080d0110 con 0x7f1f080cde90 2024-05-28T14:12:44.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.943+0000 7f1f07200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1f080d03f0 con 0x7f1f0815c7a0 2024-05-28T14:12:44.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.943+0000 7f1f06800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1f0809bb20 0x7f1f0809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.943+0000 7f1f06800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1f0809bb20 0x7f1f0809aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33444/0 (socket says 172.21.15.42:33444) 2024-05-28T14:12:44.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.943+0000 7f1f06800640 1 -- 172.21.15.42:0/2898721641 learned_addr learned my addr 172.21.15.42:0/2898721641 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:44.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.943+0000 7f1efd400640 1 --2- 172.21.15.42:0/2898721641 >> v2:172.21.15.63:3300/0 conn(0x7f1f080cde90 0x7f1f080d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.943+0000 7f1f05e00640 1 --2- 172.21.15.42:0/2898721641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0815c7a0 0x7f1f080cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f06800640 1 -- 172.21.15.42:0/2898721641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0815c7a0 msgr2=0x7f1f080cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:44.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f06800640 1 --2- 172.21.15.42:0/2898721641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0815c7a0 0x7f1f080cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f06800640 1 -- 172.21.15.42:0/2898721641 >> v2:172.21.15.63:3300/0 conn(0x7f1f080cde90 msgr2=0x7f1f080d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:44.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f06800640 1 --2- 172.21.15.42:0/2898721641 >> v2:172.21.15.63:3300/0 conn(0x7f1f080cde90 0x7f1f080d0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f06800640 1 -- 172.21.15.42:0/2898721641 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f080d06d0 con 0x7f1f0809bb20 2024-05-28T14:12:44.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1efd400640 1 --2- 172.21.15.42:0/2898721641 >> v2:172.21.15.63:3300/0 conn(0x7f1f080cde90 0x7f1f080d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:44.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f06800640 1 --2- 172.21.15.42:0/2898721641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f0809bb20 0x7f1f0809aa50 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1f080753f0 tx=0x7f1f0001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1877f5ba77a7af server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:44.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f05400640 1 -- 172.21.15.42:0/2898721641 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f000a5670 con 0x7f1f0809bb20 2024-05-28T14:12:44.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f05400640 1 -- 172.21.15.42:0/2898721641 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f000a5810 con 0x7f1f0809bb20 2024-05-28T14:12:44.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f05400640 1 -- 172.21.15.42:0/2898721641 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f000a5be0 con 0x7f1f0809bb20 2024-05-28T14:12:44.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f07200640 1 -- 172.21.15.42:0/2898721641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f0809bb20 msgr2=0x7f1f0809aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:44.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f07200640 1 --2- 172.21.15.42:0/2898721641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f0809bb20 0x7f1f0809aa50 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1f080753f0 tx=0x7f1f0001c9f0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f07200640 1 -- 172.21.15.42:0/2898721641 shutdown_connections 2024-05-28T14:12:44.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f07200640 1 --2- 172.21.15.42:0/2898721641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f0809bb20 0x7f1f0809aa50 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f07200640 1 --2- 172.21.15.42:0/2898721641 >> v2:172.21.15.63:3300/0 conn(0x7f1f080cde90 0x7f1f080d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f07200640 1 --2- 172.21.15.42:0/2898721641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0815c7a0 0x7f1f080cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.947+0000 7f1f07200640 1 -- 172.21.15.42:0/2898721641 >> 172.21.15.42:0/2898721641 conn(0x7f1f080b3e90 msgr2=0x7f1f080b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:44.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1f07200640 1 -- 172.21.15.42:0/2898721641 shutdown_connections 2024-05-28T14:12:44.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1f07200640 1 -- 172.21.15.42:0/2898721641 wait complete. 2024-05-28T14:12:44.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1f07200640 1 Processor -- start 2024-05-28T14:12:44.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1f07200640 1 -- start start 2024-05-28T14:12:44.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1f07200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0809bb20 0x7f1f08157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1f07200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f080cde90 0x7f1f080cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1f07200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0815c7a0 0x7f1f080b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1f07200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1f080b2880 con 0x7f1f080cde90 2024-05-28T14:12:44.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1f07200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1f080b2700 con 0x7f1f0809bb20 2024-05-28T14:12:44.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1f07200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1f080b2a00 con 0x7f1f0815c7a0 2024-05-28T14:12:44.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1efd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f080cde90 0x7f1f080cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1f05e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0809bb20 0x7f1f08157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1f06800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0815c7a0 0x7f1f080b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1efd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f080cde90 0x7f1f080cafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33446/0 (socket says 172.21.15.42:33446) 2024-05-28T14:12:44.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1efd400640 1 -- 172.21.15.42:0/2651507666 learned_addr learned my addr 172.21.15.42:0/2651507666 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:44.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1efd400640 1 -- 172.21.15.42:0/2651507666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0815c7a0 msgr2=0x7f1f080b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:44.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1efd400640 1 --2- 172.21.15.42:0/2651507666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0815c7a0 0x7f1f080b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1efd400640 1 -- 172.21.15.42:0/2651507666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0809bb20 msgr2=0x7f1f08157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:44.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1efd400640 1 --2- 172.21.15.42:0/2651507666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0809bb20 0x7f1f08157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:44.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1efd400640 1 -- 172.21.15.42:0/2651507666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f080cb9a0 con 0x7f1f080cde90 2024-05-28T14:12:44.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1f05e00640 1 --2- 172.21.15.42:0/2651507666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0809bb20 0x7f1f08157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:12:44.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.951+0000 7f1efd400640 1 --2- 172.21.15.42:0/2651507666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f080cde90 0x7f1f080cafc0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000e810 tx=0x7f1ef000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:44.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.955+0000 7f1efea00640 1 -- 172.21.15.42:0/2651507666 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef000f040 con 0x7f1f080cde90 2024-05-28T14:12:44.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.955+0000 7f1efea00640 1 -- 172.21.15.42:0/2651507666 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef0004030 con 0x7f1f080cde90 2024-05-28T14:12:44.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.955+0000 7f1f07200640 1 -- 172.21.15.42:0/2651507666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f080cb6c0 con 0x7f1f080cde90 2024-05-28T14:12:44.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.955+0000 7f1efea00640 1 -- 172.21.15.42:0/2651507666 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00043e0 con 0x7f1f080cde90 2024-05-28T14:12:44.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.955+0000 7f1f07200640 1 -- 172.21.15.42:0/2651507666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f080b5600 con 0x7f1f080cde90 2024-05-28T14:12:44.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.955+0000 7f1f07200640 1 -- 172.21.15.42:0/2651507666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f0804fcf0 con 0x7f1f080cde90 2024-05-28T14:12:44.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.955+0000 7f1efea00640 1 -- 172.21.15.42:0/2651507666 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f1ef000c6d0 con 0x7f1f080cde90 2024-05-28T14:12:44.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.955+0000 7f1efea00640 1 --2- 172.21.15.42:0/2651507666 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ee4069b20 0x7f1ee406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:44.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.955+0000 7f1efea00640 1 -- 172.21.15.42:0/2651507666 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f1ef00853d0 con 0x7f1f080cde90 2024-05-28T14:12:44.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.955+0000 7f1f05e00640 1 --2- 172.21.15.42:0/2651507666 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ee4069b20 0x7f1ee406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:44.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.959+0000 7f1f05e00640 1 --2- 172.21.15.42:0/2651507666 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ee4069b20 0x7f1ee406bfe0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8007e50 tx=0x7f1ef8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:44.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:44.959+0000 7f1efea00640 1 -- 172.21.15.42:0/2651507666 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ef0017020 con 0x7f1f080cde90 2024-05-28T14:12:45.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.127+0000 7f1f07200640 1 -- 172.21.15.42:0/2651507666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1f080ce7b0 con 0x7f1f080cde90 2024-05-28T14:12:45.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.127+0000 7f1efea00640 1 -- 172.21.15.42:0/2651507666 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f1ef0056da0 con 0x7f1f080cde90 2024-05-28T14:12:45.114 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:12:45.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.131+0000 7f1f07200640 1 -- 172.21.15.42:0/2651507666 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ee4069b20 msgr2=0x7f1ee406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.131+0000 7f1f07200640 1 --2- 172.21.15.42:0/2651507666 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ee4069b20 0x7f1ee406bfe0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8007e50 tx=0x7f1ef8008040 comp rx=0 tx=0).stop 2024-05-28T14:12:45.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.131+0000 7f1f07200640 1 -- 172.21.15.42:0/2651507666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f080cde90 msgr2=0x7f1f080cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.131+0000 7f1f07200640 1 --2- 172.21.15.42:0/2651507666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f080cde90 0x7f1f080cafc0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000e810 tx=0x7f1ef000ecd0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.131+0000 7f1f07200640 1 -- 172.21.15.42:0/2651507666 shutdown_connections 2024-05-28T14:12:45.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.131+0000 7f1f07200640 1 --2- 172.21.15.42:0/2651507666 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1ee4069b20 0x7f1ee406bfe0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.131+0000 7f1f07200640 1 --2- 172.21.15.42:0/2651507666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0815c7a0 0x7f1f080b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.131+0000 7f1f07200640 1 --2- 172.21.15.42:0/2651507666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f080cde90 0x7f1f080cafc0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.131+0000 7f1f07200640 1 --2- 172.21.15.42:0/2651507666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0809bb20 0x7f1f08157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.131+0000 7f1f07200640 1 -- 172.21.15.42:0/2651507666 >> 172.21.15.42:0/2651507666 conn(0x7f1f080b3e90 msgr2=0x7f1f08151970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:45.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.131+0000 7f1f07200640 1 -- 172.21.15.42:0/2651507666 shutdown_connections 2024-05-28T14:12:45.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.131+0000 7f1f07200640 1 -- 172.21.15.42:0/2651507666 wait complete. 2024-05-28T14:12:45.136 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:12:45.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.295+0000 7f9669200640 1 Processor -- start 2024-05-28T14:12:45.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.295+0000 7f9669200640 1 -- start start 2024-05-28T14:12:45.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.295+0000 7f9669200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9664148090 0x7f96640b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.295+0000 7f9669200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f96640724f0 0x7f96640aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.295+0000 7f9669200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96640b3690 con 0x7f96640b4c10 2024-05-28T14:12:45.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.295+0000 7f9669200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9664148470 con 0x7f96640724f0 2024-05-28T14:12:45.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.295+0000 7f9669200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f96640b4730 con 0x7f9664148090 2024-05-28T14:12:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.299+0000 7f9662800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f96640724f0 0x7f96640aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.299+0000 7f9661e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9664148090 0x7f96640b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.299+0000 7f9662800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f96640724f0 0x7f96640aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:57848/0 (socket says 172.21.15.42:57848) 2024-05-28T14:12:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.299+0000 7f9659400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f96640b4c10 0x7f9664071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:41754/0 (socket says 172.21.15.42:41754) 2024-05-28T14:12:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.299+0000 7f9662800640 1 -- 172.21.15.42:0/3586649103 learned_addr learned my addr 172.21.15.42:0/3586649103 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.299+0000 7f9662800640 1 -- 172.21.15.42:0/3586649103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9664148090 msgr2=0x7f96640b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.299+0000 7f9662800640 1 --2- 172.21.15.42:0/3586649103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9664148090 0x7f96640b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.299+0000 7f9662800640 1 -- 172.21.15.42:0/3586649103 >> v1:172.21.15.63:6789/0 conn(0x7f96640b4c10 legacy=0x7f9664071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.299+0000 7f9662800640 1 -- 172.21.15.42:0/3586649103 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96640b3970 con 0x7f96640724f0 2024-05-28T14:12:45.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.299+0000 7f9661e00640 1 --2- 172.21.15.42:0/3586649103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9664148090 0x7f96640b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:12:45.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.299+0000 7f9661400640 1 -- 172.21.15.42:0/3586649103 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3127366688 0 0) 0x7f96640b3690 con 0x7f96640b4c10 2024-05-28T14:12:45.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.299+0000 7f9662800640 1 --2- 172.21.15.42:0/3586649103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96640724f0 0x7f96640aeb40 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f965c009870 tx=0x7f965c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aa704708b8f33c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:45.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9661400640 1 -- 172.21.15.42:0/3586649103 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c0aa600 con 0x7f96640724f0 2024-05-28T14:12:45.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9661400640 1 -- 172.21.15.42:0/3586649103 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f965c0af070 con 0x7f96640724f0 2024-05-28T14:12:45.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9661400640 1 -- 172.21.15.42:0/3586649103 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c0aa9c0 con 0x7f96640724f0 2024-05-28T14:12:45.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 -- 172.21.15.42:0/3586649103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96640724f0 msgr2=0x7f96640aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 --2- 172.21.15.42:0/3586649103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96640724f0 0x7f96640aeb40 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f965c009870 tx=0x7f965c01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 -- 172.21.15.42:0/3586649103 shutdown_connections 2024-05-28T14:12:45.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 --2- 172.21.15.42:0/3586649103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96640724f0 0x7f96640aeb40 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 --2- 172.21.15.42:0/3586649103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9664148090 0x7f96640b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 -- 172.21.15.42:0/3586649103 >> 172.21.15.42:0/3586649103 conn(0x7f96640b21f0 msgr2=0x7f96640b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:45.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 -- 172.21.15.42:0/3586649103 shutdown_connections 2024-05-28T14:12:45.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 -- 172.21.15.42:0/3586649103 wait complete. 2024-05-28T14:12:45.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 Processor -- start 2024-05-28T14:12:45.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 -- start start 2024-05-28T14:12:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96640724f0 0x7f966413e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9664148090 0x7f966413ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f966413f340 0x7f96641466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96640b57e0 con 0x7f96640724f0 2024-05-28T14:12:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f96640b5660 con 0x7f9664148090 2024-05-28T14:12:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9669200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f96640af890 con 0x7f966413f340 2024-05-28T14:12:45.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9661e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96640724f0 0x7f966413e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9661e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96640724f0 0x7f966413e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33456/0 (socket says 172.21.15.42:33456) 2024-05-28T14:12:45.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9661e00640 1 -- 172.21.15.42:0/2376427533 learned_addr learned my addr 172.21.15.42:0/2376427533 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:45.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9662800640 1 --2- 172.21.15.42:0/2376427533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f966413f340 0x7f96641466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.303+0000 7f9659400640 1 --2- 172.21.15.42:0/2376427533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9664148090 0x7f966413ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f9661e00640 1 -- 172.21.15.42:0/2376427533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f966413f340 msgr2=0x7f96641466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f9661e00640 1 --2- 172.21.15.42:0/2376427533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f966413f340 0x7f96641466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f9661e00640 1 -- 172.21.15.42:0/2376427533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9664148090 msgr2=0x7f966413ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f9661e00640 1 --2- 172.21.15.42:0/2376427533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9664148090 0x7f966413ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f9661e00640 1 -- 172.21.15.42:0/2376427533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96640afd20 con 0x7f96640724f0 2024-05-28T14:12:45.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f9661e00640 1 --2- 172.21.15.42:0/2376427533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96640724f0 0x7f966413e860 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f965400e980 tx=0x7f965400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:45.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f965aa00640 1 -- 172.21.15.42:0/2376427533 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654017070 con 0x7f96640724f0 2024-05-28T14:12:45.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f965aa00640 1 -- 172.21.15.42:0/2376427533 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f965400ca00 con 0x7f96640724f0 2024-05-28T14:12:45.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f9669200640 1 -- 172.21.15.42:0/2376427533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96640b0000 con 0x7f96640724f0 2024-05-28T14:12:45.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f965aa00640 1 -- 172.21.15.42:0/2376427533 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965400cdd0 con 0x7f96640724f0 2024-05-28T14:12:45.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f9669200640 1 -- 172.21.15.42:0/2376427533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96640b05c0 con 0x7f96640724f0 2024-05-28T14:12:45.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f9669200640 1 -- 172.21.15.42:0/2376427533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96640b1860 con 0x7f96640724f0 2024-05-28T14:12:45.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f965aa00640 1 -- 172.21.15.42:0/2376427533 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f965401a070 con 0x7f96640724f0 2024-05-28T14:12:45.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f965aa00640 1 --2- 172.21.15.42:0/2376427533 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9640069b20 0x7f964006bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f965aa00640 1 -- 172.21.15.42:0/2376427533 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f9654085980 con 0x7f96640724f0 2024-05-28T14:12:45.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.307+0000 7f9659400640 1 --2- 172.21.15.42:0/2376427533 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9640069b20 0x7f964006bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.311+0000 7f9659400640 1 --2- 172.21.15.42:0/2376427533 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9640069b20 0x7f964006bfe0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f96640b2150 tx=0x7f964c001770 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:45.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.311+0000 7f965aa00640 1 -- 172.21.15.42:0/2376427533 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9654057350 con 0x7f96640724f0 2024-05-28T14:12:45.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.479+0000 7f9669200640 1 -- 172.21.15.42:0/2376427533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f966404fdd0 con 0x7f96640724f0 2024-05-28T14:12:45.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.479+0000 7f965aa00640 1 -- 172.21.15.42:0/2376427533 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7f965405b650 con 0x7f96640724f0 2024-05-28T14:12:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:12:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.479+0000 7f9669200640 1 -- 172.21.15.42:0/2376427533 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9640069b20 msgr2=0x7f964006bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.479+0000 7f9669200640 1 --2- 172.21.15.42:0/2376427533 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9640069b20 0x7f964006bfe0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f96640b2150 tx=0x7f964c001770 comp rx=0 tx=0).stop 2024-05-28T14:12:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.479+0000 7f9669200640 1 -- 172.21.15.42:0/2376427533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96640724f0 msgr2=0x7f966413e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.479+0000 7f9669200640 1 --2- 172.21.15.42:0/2376427533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96640724f0 0x7f966413e860 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f965400e980 tx=0x7f965400ee40 comp rx=0 tx=0).stop 2024-05-28T14:12:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.479+0000 7f9669200640 1 -- 172.21.15.42:0/2376427533 shutdown_connections 2024-05-28T14:12:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.479+0000 7f9669200640 1 --2- 172.21.15.42:0/2376427533 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9640069b20 0x7f964006bfe0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.479+0000 7f9669200640 1 --2- 172.21.15.42:0/2376427533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f966413f340 0x7f96641466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.479+0000 7f9669200640 1 --2- 172.21.15.42:0/2376427533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9664148090 0x7f966413ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.479+0000 7f9669200640 1 --2- 172.21.15.42:0/2376427533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96640724f0 0x7f966413e860 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.479+0000 7f9669200640 1 -- 172.21.15.42:0/2376427533 >> 172.21.15.42:0/2376427533 conn(0x7f96640b21f0 msgr2=0x7f966406e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:45.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.479+0000 7f9669200640 1 -- 172.21.15.42:0/2376427533 shutdown_connections 2024-05-28T14:12:45.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.479+0000 7f9669200640 1 -- 172.21.15.42:0/2376427533 wait complete. 2024-05-28T14:12:45.486 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:12:45.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.643+0000 7f14cec00640 1 Processor -- start 2024-05-28T14:12:45.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.643+0000 7f14cec00640 1 -- start start 2024-05-28T14:12:45.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.647+0000 7f14cec00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f14c80d0320 0x7f14c80d0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.647+0000 7f14cec00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c80ce090 con 0x7f14c80cc660 2024-05-28T14:12:45.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.647+0000 7f14cec00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c80d0c70 con 0x7f14c815b9a0 2024-05-28T14:12:45.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.647+0000 7f14cec00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f14c806e240 con 0x7f14c80d0320 2024-05-28T14:12:45.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.647+0000 7f14cd800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f14c80cc660 0x7f14c81415e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39890/0 (socket says 172.21.15.42:39890) 2024-05-28T14:12:45.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.647+0000 7f14cd800640 1 -- 172.21.15.42:0/979470666 learned_addr learned my addr 172.21.15.42:0/979470666 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:45.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.647+0000 7f14ce200640 1 --2- 172.21.15.42:0/979470666 >> v2:172.21.15.63:3300/0 conn(0x7f14c80d0320 0x7f14c80d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.647+0000 7f14c7e00640 1 -- 172.21.15.42:0/979470666 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1097851681 0 0) 0x7f14c80ce090 con 0x7f14c80cc660 2024-05-28T14:12:45.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.647+0000 7f14c7e00640 1 -- 172.21.15.42:0/979470666 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14b0005740 con 0x7f14c80cc660 2024-05-28T14:12:45.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.647+0000 7f14ce200640 1 -- 172.21.15.42:0/979470666 >> v1:172.21.15.63:6789/0 conn(0x7f14c815b9a0 legacy=0x7f14c806db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.647+0000 7f14ce200640 1 -- 172.21.15.42:0/979470666 >> v1:172.21.15.42:6789/0 conn(0x7f14c80cc660 legacy=0x7f14c81415e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.647+0000 7f14ce200640 1 -- 172.21.15.42:0/979470666 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c815bff0 con 0x7f14c80d0320 2024-05-28T14:12:45.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.651+0000 7f14ce200640 1 --2- 172.21.15.42:0/979470666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c80d0320 0x7f14c80d0730 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009d90 tx=0x7f14c001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2eb331053219d1c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:45.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.651+0000 7f14c7e00640 1 -- 172.21.15.42:0/979470666 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00b2070 con 0x7f14c80d0320 2024-05-28T14:12:45.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.651+0000 7f14c7e00640 1 -- 172.21.15.42:0/979470666 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c00a5a30 con 0x7f14c80d0320 2024-05-28T14:12:45.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.651+0000 7f14c7e00640 1 -- 172.21.15.42:0/979470666 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00a5de0 con 0x7f14c80d0320 2024-05-28T14:12:45.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.651+0000 7f14cec00640 1 -- 172.21.15.42:0/979470666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c80d0320 msgr2=0x7f14c80d0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.651+0000 7f14cec00640 1 --2- 172.21.15.42:0/979470666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c80d0320 0x7f14c80d0730 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009d90 tx=0x7f14c001c970 comp rx=0 tx=0).stop 2024-05-28T14:12:45.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.651+0000 7f14cec00640 1 -- 172.21.15.42:0/979470666 shutdown_connections 2024-05-28T14:12:45.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.651+0000 7f14cec00640 1 --2- 172.21.15.42:0/979470666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c80d0320 0x7f14c80d0730 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.651+0000 7f14cec00640 1 -- 172.21.15.42:0/979470666 >> 172.21.15.42:0/979470666 conn(0x7f14c806bbc0 msgr2=0x7f14c806bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:45.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.651+0000 7f14cec00640 1 -- 172.21.15.42:0/979470666 shutdown_connections 2024-05-28T14:12:45.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.651+0000 7f14cec00640 1 -- 172.21.15.42:0/979470666 wait complete. 2024-05-28T14:12:45.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.651+0000 7f14cec00640 1 Processor -- start 2024-05-28T14:12:45.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.651+0000 7f14cec00640 1 -- start start 2024-05-28T14:12:45.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.651+0000 7f14cec00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c80cc660 0x7f14c80c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14cec00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c80d0320 0x7f14c80982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14cec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14c815b9a0 0x7f14c8098810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14cec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f14c80ccbc0 con 0x7f14c815b9a0 2024-05-28T14:12:45.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14cec00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f14c80cca40 con 0x7f14c80d0320 2024-05-28T14:12:45.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14cec00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f14c80ccd40 con 0x7f14c80cc660 2024-05-28T14:12:45.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14ce200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14c815b9a0 0x7f14c8098810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14ce200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14c815b9a0 0x7f14c8098810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33466/0 (socket says 172.21.15.42:33466) 2024-05-28T14:12:45.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14ce200640 1 -- 172.21.15.42:0/3079885135 learned_addr learned my addr 172.21.15.42:0/3079885135 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:45.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14cce00640 1 --2- 172.21.15.42:0/3079885135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c80d0320 0x7f14c80982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14cd800640 1 --2- 172.21.15.42:0/3079885135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c80cc660 0x7f14c80c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14ce200640 1 -- 172.21.15.42:0/3079885135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c80cc660 msgr2=0x7f14c80c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14ce200640 1 --2- 172.21.15.42:0/3079885135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c80cc660 0x7f14c80c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14ce200640 1 -- 172.21.15.42:0/3079885135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c80d0320 msgr2=0x7f14c80982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14ce200640 1 --2- 172.21.15.42:0/3079885135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c80d0320 0x7f14c80982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14ce200640 1 -- 172.21.15.42:0/3079885135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c815f7f0 con 0x7f14c815b9a0 2024-05-28T14:12:45.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14ce200640 1 --2- 172.21.15.42:0/3079885135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14c815b9a0 0x7f14c8098810 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009d90 tx=0x7f14c0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:45.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14c5600640 1 -- 172.21.15.42:0/3079885135 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00b2050 con 0x7f14c815b9a0 2024-05-28T14:12:45.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14c5600640 1 -- 172.21.15.42:0/3079885135 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c00b3040 con 0x7f14c815b9a0 2024-05-28T14:12:45.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14c5600640 1 -- 172.21.15.42:0/3079885135 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0003e20 con 0x7f14c815b9a0 2024-05-28T14:12:45.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14cec00640 1 -- 172.21.15.42:0/3079885135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c81453b0 con 0x7f14c815b9a0 2024-05-28T14:12:45.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14cec00640 1 -- 172.21.15.42:0/3079885135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c80c9b80 con 0x7f14c815b9a0 2024-05-28T14:12:45.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14cec00640 1 -- 172.21.15.42:0/3079885135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c804fcf0 con 0x7f14c815b9a0 2024-05-28T14:12:45.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.655+0000 7f14c5600640 1 -- 172.21.15.42:0/3079885135 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f14c00c1060 con 0x7f14c815b9a0 2024-05-28T14:12:45.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.659+0000 7f14c5600640 1 --2- 172.21.15.42:0/3079885135 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f14a8069b20 0x7f14a806bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.659+0000 7f14c5600640 1 -- 172.21.15.42:0/3079885135 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f14c011fc30 con 0x7f14c815b9a0 2024-05-28T14:12:45.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.659+0000 7f14cd800640 1 --2- 172.21.15.42:0/3079885135 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f14a8069b20 0x7f14a806bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.659+0000 7f14cd800640 1 --2- 172.21.15.42:0/3079885135 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f14a8069b20 0x7f14a806bfe0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0011f0 tx=0x7f14bc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:45.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.663+0000 7f14c5600640 1 -- 172.21.15.42:0/3079885135 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f14c0016560 con 0x7f14c815b9a0 2024-05-28T14:12:45.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.823+0000 7f14cec00640 1 -- 172.21.15.42:0/3079885135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f14c8167110 con 0x7f14c815b9a0 2024-05-28T14:12:45.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.827+0000 7f14c5600640 1 -- 172.21.15.42:0/3079885135 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f14c00f1bd0 con 0x7f14c815b9a0 2024-05-28T14:12:45.811 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:12:45.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.827+0000 7f14cec00640 1 -- 172.21.15.42:0/3079885135 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f14a8069b20 msgr2=0x7f14a806bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.827+0000 7f14cec00640 1 --2- 172.21.15.42:0/3079885135 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f14a8069b20 0x7f14a806bfe0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0011f0 tx=0x7f14bc009040 comp rx=0 tx=0).stop 2024-05-28T14:12:45.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.827+0000 7f14cec00640 1 -- 172.21.15.42:0/3079885135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14c815b9a0 msgr2=0x7f14c8098810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.827+0000 7f14cec00640 1 --2- 172.21.15.42:0/3079885135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14c815b9a0 0x7f14c8098810 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009d90 tx=0x7f14c0019710 comp rx=0 tx=0).stop 2024-05-28T14:12:45.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.827+0000 7f14cec00640 1 -- 172.21.15.42:0/3079885135 shutdown_connections 2024-05-28T14:12:45.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.827+0000 7f14cec00640 1 --2- 172.21.15.42:0/3079885135 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f14a8069b20 0x7f14a806bfe0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.827+0000 7f14cec00640 1 --2- 172.21.15.42:0/3079885135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14c815b9a0 0x7f14c8098810 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.827+0000 7f14cec00640 1 --2- 172.21.15.42:0/3079885135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c80d0320 0x7f14c80982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.827+0000 7f14cec00640 1 --2- 172.21.15.42:0/3079885135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c80cc660 0x7f14c80c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.827+0000 7f14cec00640 1 -- 172.21.15.42:0/3079885135 >> 172.21.15.42:0/3079885135 conn(0x7f14c806bbc0 msgr2=0x7f14c8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:45.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.827+0000 7f14cec00640 1 -- 172.21.15.42:0/3079885135 shutdown_connections 2024-05-28T14:12:45.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.827+0000 7f14cec00640 1 -- 172.21.15.42:0/3079885135 wait complete. 2024-05-28T14:12:45.833 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:12:45.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.987+0000 7fdf2a000640 1 Processor -- start 2024-05-28T14:12:45.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.987+0000 7fdf2a000640 1 -- start start 2024-05-28T14:12:45.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.987+0000 7fdf2a000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf2415e970 0x7fdf240d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf2a000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdf240cd210 0x7fdf2409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf2a000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf24070740 con 0x7fdf2406ff60 2024-05-28T14:12:45.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf2a000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdf240d0640 con 0x7fdf240cd210 2024-05-28T14:12:45.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf2a000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdf240d1160 con 0x7fdf2415e970 2024-05-28T14:12:45.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf23e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdf240cd210 0x7fdf2409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf23400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf2415e970 0x7fdf240d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf1aa00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fdf2406ff60 0x7fdf240ccb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:41778/0 (socket says 172.21.15.42:41778) 2024-05-28T14:12:45.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf1aa00640 1 -- 172.21.15.42:0/1113030050 learned_addr learned my addr 172.21.15.42:0/1113030050 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:45.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf22a00640 1 -- 172.21.15.42:0/1113030050 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2347032478 0 0) 0x7fdf24070740 con 0x7fdf2406ff60 2024-05-28T14:12:45.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf22a00640 1 -- 172.21.15.42:0/1113030050 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf08003530 con 0x7fdf2406ff60 2024-05-28T14:12:45.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf23e00640 1 -- 172.21.15.42:0/1113030050 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf2415e970 msgr2=0x7fdf240d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:45.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf23e00640 1 --2- 172.21.15.42:0/1113030050 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf2415e970 0x7fdf240d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf23e00640 1 -- 172.21.15.42:0/1113030050 >> v1:172.21.15.63:6789/0 conn(0x7fdf2406ff60 legacy=0x7fdf240ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf23e00640 1 -- 172.21.15.42:0/1113030050 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf240d0870 con 0x7fdf240cd210 2024-05-28T14:12:45.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.991+0000 7fdf23400640 1 --2- 172.21.15.42:0/1113030050 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf2415e970 0x7fdf240d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:12:45.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.995+0000 7fdf23e00640 1 --2- 172.21.15.42:0/1113030050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf240cd210 0x7fdf2409aa50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009d90 tx=0x7fdf1c01c5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85c543e89cc7ef77 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:45.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.995+0000 7fdf22a00640 1 -- 172.21.15.42:0/1113030050 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0a66c0 con 0x7fdf240cd210 2024-05-28T14:12:45.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.995+0000 7fdf22a00640 1 -- 172.21.15.42:0/1113030050 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1c0a6860 con 0x7fdf240cd210 2024-05-28T14:12:45.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.995+0000 7fdf22a00640 1 -- 172.21.15.42:0/1113030050 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0a6c10 con 0x7fdf240cd210 2024-05-28T14:12:45.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.995+0000 7fdf2a000640 1 -- 172.21.15.42:0/1113030050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf240cd210 msgr2=0x7fdf2409aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.995+0000 7fdf2a000640 1 --2- 172.21.15.42:0/1113030050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf240cd210 0x7fdf2409aa50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009d90 tx=0x7fdf1c01c5f0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.995+0000 7fdf2a000640 1 -- 172.21.15.42:0/1113030050 shutdown_connections 2024-05-28T14:12:45.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.995+0000 7fdf2a000640 1 --2- 172.21.15.42:0/1113030050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf240cd210 0x7fdf2409aa50 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.995+0000 7fdf2a000640 1 --2- 172.21.15.42:0/1113030050 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf2415e970 0x7fdf240d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.995+0000 7fdf2a000640 1 -- 172.21.15.42:0/1113030050 >> 172.21.15.42:0/1113030050 conn(0x7fdf240b3fa0 msgr2=0x7fdf240b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:45.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.995+0000 7fdf2a000640 1 -- 172.21.15.42:0/1113030050 shutdown_connections 2024-05-28T14:12:45.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.995+0000 7fdf2a000640 1 -- 172.21.15.42:0/1113030050 wait complete. 2024-05-28T14:12:45.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.995+0000 7fdf2a000640 1 Processor -- start 2024-05-28T14:12:45.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.995+0000 7fdf2a000640 1 -- start start 2024-05-28T14:12:45.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf2a000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf2406ff60 0x7fdf240cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf2a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf240cd210 0x7fdf2414d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf2a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf2415e970 0x7fdf2414d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf2a000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdf240cd6a0 con 0x7fdf2415e970 2024-05-28T14:12:45.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf2a000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf24070340 con 0x7fdf240cd210 2024-05-28T14:12:45.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf2a000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdf24156140 con 0x7fdf2406ff60 2024-05-28T14:12:45.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf23e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf2415e970 0x7fdf2414d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf23e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf2415e970 0x7fdf2414d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33478/0 (socket says 172.21.15.42:33478) 2024-05-28T14:12:45.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf23e00640 1 -- 172.21.15.42:0/1690925667 learned_addr learned my addr 172.21.15.42:0/1690925667 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:45.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf23400640 1 --2- 172.21.15.42:0/1690925667 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf2406ff60 0x7fdf240cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf1aa00640 1 --2- 172.21.15.42:0/1690925667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf240cd210 0x7fdf2414d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf23e00640 1 -- 172.21.15.42:0/1690925667 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf2406ff60 msgr2=0x7fdf240cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf23e00640 1 --2- 172.21.15.42:0/1690925667 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf2406ff60 0x7fdf240cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf23e00640 1 -- 172.21.15.42:0/1690925667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf240cd210 msgr2=0x7fdf2414d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:45.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf23e00640 1 --2- 172.21.15.42:0/1690925667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf240cd210 0x7fdf2414d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:45.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf23e00640 1 -- 172.21.15.42:0/1690925667 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf24156420 con 0x7fdf2415e970 2024-05-28T14:12:45.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf23e00640 1 --2- 172.21.15.42:0/1690925667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf2415e970 0x7fdf2414d780 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c01a220 tx=0x7fdf1c019390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:45.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf1be00640 1 -- 172.21.15.42:0/1690925667 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c003ba0 con 0x7fdf2415e970 2024-05-28T14:12:45.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf1be00640 1 -- 172.21.15.42:0/1690925667 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1c0bf070 con 0x7fdf2415e970 2024-05-28T14:12:45.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf1be00640 1 -- 172.21.15.42:0/1690925667 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c004040 con 0x7fdf2415e970 2024-05-28T14:12:45.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf2a000640 1 -- 172.21.15.42:0/1690925667 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf24156700 con 0x7fdf2415e970 2024-05-28T14:12:45.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf2a000640 1 -- 172.21.15.42:0/1690925667 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf24156cc0 con 0x7fdf2415e970 2024-05-28T14:12:45.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf2a000640 1 -- 172.21.15.42:0/1690925667 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf24068a90 con 0x7fdf2415e970 2024-05-28T14:12:45.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:45.999+0000 7fdf1be00640 1 -- 172.21.15.42:0/1690925667 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fdf1c0041e0 con 0x7fdf2415e970 2024-05-28T14:12:45.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.003+0000 7fdf1be00640 1 --2- 172.21.15.42:0/1690925667 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf00069b20 0x7fdf0006bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:45.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.003+0000 7fdf1be00640 1 -- 172.21.15.42:0/1690925667 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fdf1c1205e0 con 0x7fdf2415e970 2024-05-28T14:12:45.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.003+0000 7fdf23400640 1 --2- 172.21.15.42:0/1690925667 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf00069b20 0x7fdf0006bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:45.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.003+0000 7fdf23400640 1 --2- 172.21.15.42:0/1690925667 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf00069b20 0x7fdf0006bfe0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009580 tx=0x7fdf14009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.007+0000 7fdf1be00640 1 -- 172.21.15.42:0/1690925667 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdf1c016560 con 0x7fdf2415e970 2024-05-28T14:12:46.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.211+0000 7fdf2a000640 1 -- 172.21.15.42:0/1690925667 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf240d1fd0 con 0x7fdf2415e970 2024-05-28T14:12:46.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.211+0000 7fdf1be00640 1 -- 172.21.15.42:0/1690925667 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5190 (secure 0 0 0) 0x7fdf1c0f2030 con 0x7fdf2415e970 2024-05-28T14:12:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8607,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/1661021064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1661021064},{"type":"v1","addr":"172.21.15.42:6836","nonce":1661021064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":57050,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2398440572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":2398440572},{"type":"v1","addr":"172.21.15.42:6837","nonce":2398440572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":57053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/2530315729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":2530315729},{"type":"v1","addr":"172.21.15.63:6834","nonce":2530315729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:11:14.793551+0000","modified":"2024-05-28T14:11:17.812010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57056},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57056":{"gid":57056,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/1184583947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1184583947},{"type":"v1","addr":"172.21.15.63:6835","nonce":1184583947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57056,"qdb_cluster":[57056]},"id":7}]} 2024-05-28T14:12:46.197 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 65 2024-05-28T14:12:46.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.215+0000 7fdf2a000640 1 -- 172.21.15.42:0/1690925667 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf00069b20 msgr2=0x7fdf0006bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:46.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.215+0000 7fdf2a000640 1 --2- 172.21.15.42:0/1690925667 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf00069b20 0x7fdf0006bfe0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009580 tx=0x7fdf14009190 comp rx=0 tx=0).stop 2024-05-28T14:12:46.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.215+0000 7fdf2a000640 1 -- 172.21.15.42:0/1690925667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf2415e970 msgr2=0x7fdf2414d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:46.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.215+0000 7fdf2a000640 1 --2- 172.21.15.42:0/1690925667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf2415e970 0x7fdf2414d780 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c01a220 tx=0x7fdf1c019390 comp rx=0 tx=0).stop 2024-05-28T14:12:46.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.215+0000 7fdf2a000640 1 -- 172.21.15.42:0/1690925667 shutdown_connections 2024-05-28T14:12:46.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.215+0000 7fdf2a000640 1 --2- 172.21.15.42:0/1690925667 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf00069b20 0x7fdf0006bfe0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.215+0000 7fdf2a000640 1 --2- 172.21.15.42:0/1690925667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf2415e970 0x7fdf2414d780 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.215+0000 7fdf2a000640 1 --2- 172.21.15.42:0/1690925667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf240cd210 0x7fdf2414d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.215+0000 7fdf2a000640 1 --2- 172.21.15.42:0/1690925667 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf2406ff60 0x7fdf240cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.215+0000 7fdf2a000640 1 -- 172.21.15.42:0/1690925667 >> 172.21.15.42:0/1690925667 conn(0x7fdf240b3fa0 msgr2=0x7fdf240d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:46.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.215+0000 7fdf2a000640 1 -- 172.21.15.42:0/1690925667 shutdown_connections 2024-05-28T14:12:46.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.215+0000 7fdf2a000640 1 -- 172.21.15.42:0/1690925667 wait complete. 2024-05-28T14:12:46.219 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:12:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.375+0000 7f118d400640 1 Processor -- start 2024-05-28T14:12:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.375+0000 7f118d400640 1 -- start start 2024-05-28T14:12:46.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.375+0000 7f118d400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f118806f660 0x7f1188073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.375+0000 7f118d400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11880d0e00 0x7f118806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.375+0000 7f118d400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f118815f280 con 0x7f118815ba40 2024-05-28T14:12:46.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.375+0000 7f118d400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f118815f030 con 0x7f118806f660 2024-05-28T14:12:46.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.375+0000 7f118d400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f118815f560 con 0x7f11880d0e00 2024-05-28T14:12:46.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.375+0000 7f117fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f118815ba40 0x7f11880d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39894/0 (socket says 172.21.15.42:39894) 2024-05-28T14:12:46.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.375+0000 7f117fe00640 1 -- 172.21.15.42:0/2394562651 learned_addr learned my addr 172.21.15.42:0/2394562651 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.375+0000 7f1187200640 1 --2- 172.21.15.42:0/2394562651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11880d0e00 0x7f118806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.375+0000 7f1186800640 1 --2- 172.21.15.42:0/2394562651 >> v2:172.21.15.63:3300/0 conn(0x7f118806f660 0x7f1188073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 -- 172.21.15.42:0/2394562651 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3174226252 0 0) 0x7f118815f280 con 0x7f118815ba40 2024-05-28T14:12:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 -- 172.21.15.42:0/2394562651 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f116c003530 con 0x7f118815ba40 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 -- 172.21.15.42:0/2394562651 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 788845576 0 0) 0x7f116c003530 con 0x7f118815ba40 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 -- 172.21.15.42:0/2394562651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11880d0e00 msgr2=0x7f118806dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 --2- 172.21.15.42:0/2394562651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11880d0e00 0x7f118806dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 -- 172.21.15.42:0/2394562651 >> v2:172.21.15.63:3300/0 conn(0x7f118806f660 msgr2=0x7f1188073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 --2- 172.21.15.42:0/2394562651 >> v2:172.21.15.63:3300/0 conn(0x7f118806f660 0x7f1188073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 -- 172.21.15.42:0/2394562651 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118815f840 con 0x7f118815ba40 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 -- 172.21.15.42:0/2394562651 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f1170001680 con 0x7f118815ba40 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 -- 172.21.15.42:0/2394562651 >> v1:172.21.15.42:6789/0 conn(0x7f118815ba40 legacy=0x7f11880d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 --2- 172.21.15.42:0/2394562651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c0039c0 0x7f116c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 --2- 172.21.15.42:0/2394562651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11880d0e00 0x7f116c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 --2- 172.21.15.42:0/2394562651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116c004d50 0x7f116c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 -- 172.21.15.42:0/2394562651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f118815f030 con 0x7f116c004d50 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 -- 172.21.15.42:0/2394562651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f118815f560 con 0x7f116c0039c0 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1185e00640 1 -- 172.21.15.42:0/2394562651 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f116c00bab0 con 0x7f11880d0e00 2024-05-28T14:12:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1187200640 1 --2- 172.21.15.42:0/2394562651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116c004d50 0x7f116c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f117fe00640 1 --2- 172.21.15.42:0/2394562651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11880d0e00 0x7f116c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1186800640 1 --2- 172.21.15.42:0/2394562651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c0039c0 0x7f116c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1187200640 1 -- 172.21.15.42:0/2394562651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11880d0e00 msgr2=0x7f116c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1187200640 1 --2- 172.21.15.42:0/2394562651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11880d0e00 0x7f116c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1187200640 1 -- 172.21.15.42:0/2394562651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c0039c0 msgr2=0x7f116c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1187200640 1 --2- 172.21.15.42:0/2394562651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c0039c0 0x7f116c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f1187200640 1 -- 172.21.15.42:0/2394562651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116c003530 con 0x7f116c004d50 2024-05-28T14:12:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.379+0000 7f117fe00640 1 --2- 172.21.15.42:0/2394562651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11880d0e00 0x7f116c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f1187200640 1 --2- 172.21.15.42:0/2394562651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116c004d50 0x7f116c0051d0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f1180009fc0 tx=0x7f118001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5c3192b408648a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f1185e00640 1 -- 172.21.15.42:0/2394562651 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11800b0070 con 0x7f116c004d50 2024-05-28T14:12:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f1185e00640 1 -- 172.21.15.42:0/2394562651 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11800ab600 con 0x7f116c004d50 2024-05-28T14:12:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f1185e00640 1 -- 172.21.15.42:0/2394562651 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180009c10 con 0x7f116c004d50 2024-05-28T14:12:46.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f118d400640 1 -- 172.21.15.42:0/2394562651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116c004d50 msgr2=0x7f116c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:46.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f118d400640 1 --2- 172.21.15.42:0/2394562651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116c004d50 0x7f116c0051d0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f1180009fc0 tx=0x7f118001c930 comp rx=0 tx=0).stop 2024-05-28T14:12:46.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f1187200640 1 -- 172.21.15.42:0/2394562651 reap_dead start 2024-05-28T14:12:46.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f118d400640 1 -- 172.21.15.42:0/2394562651 shutdown_connections 2024-05-28T14:12:46.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f118d400640 1 -- 172.21.15.42:0/2394562651 >> 172.21.15.42:0/2394562651 conn(0x7f118806bdf0 msgr2=0x7f118806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:46.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f118d400640 1 -- 172.21.15.42:0/2394562651 shutdown_connections 2024-05-28T14:12:46.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f118d400640 1 -- 172.21.15.42:0/2394562651 wait complete. 2024-05-28T14:12:46.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f118d400640 1 Processor -- start 2024-05-28T14:12:46.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f118d400640 1 -- start start 2024-05-28T14:12:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f118d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116c004d50 0x7f118814cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f118d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11880b2090 0x7f118814d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f118d400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f118814d870 0x7f11881527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f118d400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1188166f60 con 0x7f116c004d50 2024-05-28T14:12:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f118d400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1188166de0 con 0x7f11880b2090 2024-05-28T14:12:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f118d400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f11881670e0 con 0x7f118814d870 2024-05-28T14:12:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f1186800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116c004d50 0x7f118814cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f1186800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116c004d50 0x7f118814cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33506/0 (socket says 172.21.15.42:33506) 2024-05-28T14:12:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.383+0000 7f1186800640 1 -- 172.21.15.42:0/1986183980 learned_addr learned my addr 172.21.15.42:0/1986183980 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f1186800640 1 -- 172.21.15.42:0/1986183980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f118814d870 msgr2=0x7f11881527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f1186800640 1 --2- 172.21.15.42:0/1986183980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f118814d870 0x7f11881527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f1186800640 1 -- 172.21.15.42:0/1986183980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11880b2090 msgr2=0x7f118814d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f1186800640 1 --2- 172.21.15.42:0/1986183980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11880b2090 0x7f118814d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f1186800640 1 -- 172.21.15.42:0/1986183980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118814d140 con 0x7f116c004d50 2024-05-28T14:12:46.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f1186800640 1 --2- 172.21.15.42:0/1986183980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116c004d50 0x7f118814cc00 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f1178011960 tx=0x7f1178011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:46.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f117ea00640 1 -- 172.21.15.42:0/1986183980 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117800fa70 con 0x7f116c004d50 2024-05-28T14:12:46.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f117ea00640 1 -- 172.21.15.42:0/1986183980 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f117800fc10 con 0x7f116c004d50 2024-05-28T14:12:46.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f117ea00640 1 -- 172.21.15.42:0/1986183980 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1178014440 con 0x7f116c004d50 2024-05-28T14:12:46.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f118d400640 1 -- 172.21.15.42:0/1986183980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11880b12d0 con 0x7f116c004d50 2024-05-28T14:12:46.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f118d400640 1 -- 172.21.15.42:0/1986183980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11880b1890 con 0x7f116c004d50 2024-05-28T14:12:46.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f118d400640 1 -- 172.21.15.42:0/1986183980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1188068a90 con 0x7f116c004d50 2024-05-28T14:12:46.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f117ea00640 1 -- 172.21.15.42:0/1986183980 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f1178012040 con 0x7f116c004d50 2024-05-28T14:12:46.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f117ea00640 1 --2- 172.21.15.42:0/1986183980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1164069b20 0x7f116406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f117ea00640 1 -- 172.21.15.42:0/1986183980 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5793+0+0 (secure 0 0 0) 0x7f1178085c90 con 0x7f116c004d50 2024-05-28T14:12:46.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.387+0000 7f117fe00640 1 --2- 172.21.15.42:0/1986183980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1164069b20 0x7f116406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:46.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.391+0000 7f117fe00640 1 --2- 172.21.15.42:0/1986183980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1164069b20 0x7f116406bfe0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1170000c10 tx=0x7f1170001c80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:46.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.391+0000 7f117ea00640 1 -- 172.21.15.42:0/1986183980 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1178019030 con 0x7f116c004d50 2024-05-28T14:12:46.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.567+0000 7f118d400640 1 -- 172.21.15.42:0/1986183980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1188072bb0 con 0x7f116c004d50 2024-05-28T14:12:46.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.571+0000 7f117ea00640 1 -- 172.21.15.42:0/1986183980 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16714 (secure 0 0 0) 0x7f1178057d50 con 0x7f116c004d50 2024-05-28T14:12:46.555 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:46.555 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":106,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:12:46.053529+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:11:13.194240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-28T14:11:13.743150+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:12:46.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.571+0000 7f118d400640 1 -- 172.21.15.42:0/1986183980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1164069b20 msgr2=0x7f116406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:46.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.571+0000 7f118d400640 1 --2- 172.21.15.42:0/1986183980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1164069b20 0x7f116406bfe0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1170000c10 tx=0x7f1170001c80 comp rx=0 tx=0).stop 2024-05-28T14:12:46.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.571+0000 7f118d400640 1 -- 172.21.15.42:0/1986183980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116c004d50 msgr2=0x7f118814cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:46.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.571+0000 7f118d400640 1 --2- 172.21.15.42:0/1986183980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116c004d50 0x7f118814cc00 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f1178011960 tx=0x7f1178011e20 comp rx=0 tx=0).stop 2024-05-28T14:12:46.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.571+0000 7f118d400640 1 -- 172.21.15.42:0/1986183980 shutdown_connections 2024-05-28T14:12:46.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.571+0000 7f118d400640 1 --2- 172.21.15.42:0/1986183980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1164069b20 0x7f116406bfe0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.571+0000 7f118d400640 1 --2- 172.21.15.42:0/1986183980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f118814d870 0x7f11881527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.571+0000 7f118d400640 1 --2- 172.21.15.42:0/1986183980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11880b2090 0x7f118814d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.571+0000 7f118d400640 1 --2- 172.21.15.42:0/1986183980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116c004d50 0x7f118814cc00 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.571+0000 7f118d400640 1 -- 172.21.15.42:0/1986183980 >> 172.21.15.42:0/1986183980 conn(0x7f118806bdf0 msgr2=0x7f1188073780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:46.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.571+0000 7f118d400640 1 -- 172.21.15.42:0/1986183980 shutdown_connections 2024-05-28T14:12:46.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.571+0000 7f118d400640 1 -- 172.21.15.42:0/1986183980 wait complete. 2024-05-28T14:12:46.578 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-28T14:12:46.578 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:12:46.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f073b200640 1 Processor -- start 2024-05-28T14:12:46.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f073b200640 1 -- start start 2024-05-28T14:12:46.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f073b200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f073c0b5f60 0x7f073c0d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f073b200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f073c15e7c0 0x7f073c06da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f073b200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f073c06c980 con 0x7f073c0d3550 2024-05-28T14:12:46.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f073b200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f073c0d39a0 con 0x7f073c15e7c0 2024-05-28T14:12:46.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f073b200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f073c06cc60 con 0x7f073c0b5f60 2024-05-28T14:12:46.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f0739400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f073c15e7c0 0x7f073c06da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:46.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f073a800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f073c0d3550 0x7f073c0b6340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39900/0 (socket says 172.21.15.42:39900) 2024-05-28T14:12:46.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f073a800640 1 -- 172.21.15.42:0/2256857674 learned_addr learned my addr 172.21.15.42:0/2256857674 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:46.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f0739e00640 1 --2- 172.21.15.42:0/2256857674 >> v2:172.21.15.63:3300/0 conn(0x7f073c0b5f60 0x7f073c0d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:46.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f0738a00640 1 -- 172.21.15.42:0/2256857674 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 354907306 0 0) 0x7f073c06c980 con 0x7f073c0d3550 2024-05-28T14:12:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f0738a00640 1 -- 172.21.15.42:0/2256857674 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0720003530 con 0x7f073c0d3550 2024-05-28T14:12:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f0739400640 1 -- 172.21.15.42:0/2256857674 >> v2:172.21.15.63:3300/0 conn(0x7f073c0b5f60 msgr2=0x7f073c0d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f0739400640 1 --2- 172.21.15.42:0/2256857674 >> v2:172.21.15.63:3300/0 conn(0x7f073c0b5f60 0x7f073c0d2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f0739400640 1 -- 172.21.15.42:0/2256857674 >> v1:172.21.15.42:6789/0 conn(0x7f073c0d3550 legacy=0x7f073c0b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f0739400640 1 -- 172.21.15.42:0/2256857674 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c06cf40 con 0x7f073c15e7c0 2024-05-28T14:12:46.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.739+0000 7f0739e00640 1 --2- 172.21.15.42:0/2256857674 >> v2:172.21.15.63:3300/0 conn(0x7f073c0b5f60 0x7f073c0d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:46.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.743+0000 7f0739400640 1 --2- 172.21.15.42:0/2256857674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073c15e7c0 0x7f073c06da20 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f073c08cdf0 tx=0x7f072801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef06dd311b2fca7b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.743+0000 7f0738a00640 1 -- 172.21.15.42:0/2256857674 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07280ab430 con 0x7f073c15e7c0 2024-05-28T14:12:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.743+0000 7f0738a00640 1 -- 172.21.15.42:0/2256857674 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07280b0070 con 0x7f073c15e7c0 2024-05-28T14:12:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.743+0000 7f0738a00640 1 -- 172.21.15.42:0/2256857674 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07280ab7f0 con 0x7f073c15e7c0 2024-05-28T14:12:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.743+0000 7f073b200640 1 -- 172.21.15.42:0/2256857674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073c15e7c0 msgr2=0x7f073c06da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.743+0000 7f073b200640 1 --2- 172.21.15.42:0/2256857674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073c15e7c0 0x7f073c06da20 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f073c08cdf0 tx=0x7f072801c430 comp rx=0 tx=0).stop 2024-05-28T14:12:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.743+0000 7f073b200640 1 -- 172.21.15.42:0/2256857674 shutdown_connections 2024-05-28T14:12:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.743+0000 7f073b200640 1 --2- 172.21.15.42:0/2256857674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073c15e7c0 0x7f073c06da20 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.743+0000 7f073b200640 1 --2- 172.21.15.42:0/2256857674 >> v2:172.21.15.63:3300/0 conn(0x7f073c0b5f60 0x7f073c0d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.743+0000 7f073b200640 1 -- 172.21.15.42:0/2256857674 >> 172.21.15.42:0/2256857674 conn(0x7f073c08bfe0 msgr2=0x7f073c0d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.743+0000 7f073b200640 1 -- 172.21.15.42:0/2256857674 shutdown_connections 2024-05-28T14:12:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.743+0000 7f073b200640 1 -- 172.21.15.42:0/2256857674 wait complete. 2024-05-28T14:12:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.743+0000 7f073b200640 1 Processor -- start 2024-05-28T14:12:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.743+0000 7f073b200640 1 -- start start 2024-05-28T14:12:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f073c0b5f60 0x7f073c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c0d3550 0x7f073c1569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073c15e7c0 0x7f073c156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f073c0cf4c0 con 0x7f073c15e7c0 2024-05-28T14:12:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f073c0cf340 con 0x7f073c0b5f60 2024-05-28T14:12:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f073c0cf640 con 0x7f073c0d3550 2024-05-28T14:12:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073c15e7c0 0x7f073c156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073c15e7c0 0x7f073c156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33522/0 (socket says 172.21.15.42:33522) 2024-05-28T14:12:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073a800640 1 -- 172.21.15.42:0/1335447098 learned_addr learned my addr 172.21.15.42:0/1335447098 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f0739400640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c0d3550 0x7f073c1569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f0739e00640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f073c0b5f60 0x7f073c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073a800640 1 -- 172.21.15.42:0/1335447098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c0d3550 msgr2=0x7f073c1569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073a800640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c0d3550 0x7f073c1569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073a800640 1 -- 172.21.15.42:0/1335447098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f073c0b5f60 msgr2=0x7f073c0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073a800640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f073c0b5f60 0x7f073c0cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073a800640 1 -- 172.21.15.42:0/1335447098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c0b8570 con 0x7f073c15e7c0 2024-05-28T14:12:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f0739400640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c0d3550 0x7f073c1569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f0739e00640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f073c0b5f60 0x7f073c0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073a800640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073c15e7c0 0x7f073c156f10 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f073400cb70 tx=0x7f073400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f0732000640 1 -- 172.21.15.42:0/1335447098 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734017070 con 0x7f073c15e7c0 2024-05-28T14:12:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f0732000640 1 -- 172.21.15.42:0/1335447098 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f073400ad40 con 0x7f073c15e7c0 2024-05-28T14:12:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f0732000640 1 -- 172.21.15.42:0/1335447098 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734012620 con 0x7f073c15e7c0 2024-05-28T14:12:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073b200640 1 -- 172.21.15.42:0/1335447098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f073c15fda0 con 0x7f073c15e7c0 2024-05-28T14:12:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073b200640 1 -- 172.21.15.42:0/1335447098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f073c160360 con 0x7f073c15e7c0 2024-05-28T14:12:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f073b200640 1 -- 172.21.15.42:0/1335447098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f073c0b8740 con 0x7f073c15e7c0 2024-05-28T14:12:46.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.747+0000 7f0732000640 1 -- 172.21.15.42:0/1335447098 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f073401a070 con 0x7f073c15e7c0 2024-05-28T14:12:46.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.751+0000 7f0732000640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f071c069b20 0x7f071c06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:46.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.751+0000 7f0732000640 1 -- 172.21.15.42:0/1335447098 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5793+0+0 (secure 0 0 0) 0x7f0734085600 con 0x7f073c15e7c0 2024-05-28T14:12:46.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.751+0000 7f0739e00640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f071c069b20 0x7f071c06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:46.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.751+0000 7f0739e00640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f071c069b20 0x7f071c06bfe0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f072c009970 tx=0x7f072c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:46.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.755+0000 7f0732000640 1 -- 172.21.15.42:0/1335447098 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0734057620 con 0x7f073c15e7c0 2024-05-28T14:12:46.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.955+0000 7f073b200640 1 -- 172.21.15.42:0/1335447098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f073c06e000 con 0x7f073c15e7c0 2024-05-28T14:12:46.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.955+0000 7f0732000640 1 -- 172.21.15.42:0/1335447098 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5190 (secure 0 0 0) 0x7f0734056f20 con 0x7f073c15e7c0 2024-05-28T14:12:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:46.942 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8607,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/1661021064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1661021064},{"type":"v1","addr":"172.21.15.42:6836","nonce":1661021064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":57050,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2398440572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":2398440572},{"type":"v1","addr":"172.21.15.42:6837","nonce":2398440572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":57053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/2530315729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":2530315729},{"type":"v1","addr":"172.21.15.63:6834","nonce":2530315729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:11:14.793551+0000","modified":"2024-05-28T14:11:17.812010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57056},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57056":{"gid":57056,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/1184583947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1184583947},{"type":"v1","addr":"172.21.15.63:6835","nonce":1184583947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57056,"qdb_cluster":[57056]},"id":7}]} 2024-05-28T14:12:46.942 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 65 2024-05-28T14:12:46.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.959+0000 7f073b200640 1 -- 172.21.15.42:0/1335447098 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f071c069b20 msgr2=0x7f071c06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:46.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.959+0000 7f073b200640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f071c069b20 0x7f071c06bfe0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f072c009970 tx=0x7f072c008040 comp rx=0 tx=0).stop 2024-05-28T14:12:46.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.959+0000 7f073b200640 1 -- 172.21.15.42:0/1335447098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073c15e7c0 msgr2=0x7f073c156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:46.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.959+0000 7f073b200640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073c15e7c0 0x7f073c156f10 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f073400cb70 tx=0x7f073400a430 comp rx=0 tx=0).stop 2024-05-28T14:12:46.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.959+0000 7f073b200640 1 -- 172.21.15.42:0/1335447098 shutdown_connections 2024-05-28T14:12:46.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.959+0000 7f073b200640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f071c069b20 0x7f071c06bfe0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.959+0000 7f073b200640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073c15e7c0 0x7f073c156f10 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.959+0000 7f073b200640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c0d3550 0x7f073c1569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.959+0000 7f073b200640 1 --2- 172.21.15.42:0/1335447098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f073c0b5f60 0x7f073c0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:46.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.959+0000 7f073b200640 1 -- 172.21.15.42:0/1335447098 >> 172.21.15.42:0/1335447098 conn(0x7f073c08bfe0 msgr2=0x7f073c168a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:46.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.959+0000 7f073b200640 1 -- 172.21.15.42:0/1335447098 shutdown_connections 2024-05-28T14:12:46.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:46.959+0000 7f073b200640 1 -- 172.21.15.42:0/1335447098 wait complete. 2024-05-28T14:12:46.963 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:12:47.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.119+0000 7fbe7e200640 1 Processor -- start 2024-05-28T14:12:47.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.119+0000 7fbe7e200640 1 -- start start 2024-05-28T14:12:47.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.123+0000 7fbe7e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe7806f660 0x7fbe78073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.123+0000 7fbe7e200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbe7815ba40 0x7fbe780d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.123+0000 7fbe7e200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe7815f280 con 0x7fbe78072dd0 2024-05-28T14:12:47.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.123+0000 7fbe7e200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fbe7815f030 con 0x7fbe7815ba40 2024-05-28T14:12:47.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.123+0000 7fbe7e200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbe7815f560 con 0x7fbe7806f660 2024-05-28T14:12:47.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.123+0000 7fbe77e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fbe78072dd0 0x7fbe780b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39902/0 (socket says 172.21.15.42:39902) 2024-05-28T14:12:47.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.123+0000 7fbe77e00640 1 -- 172.21.15.42:0/2398442268 learned_addr learned my addr 172.21.15.42:0/2398442268 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:47.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.123+0000 7fbe6fe00640 1 --2- 172.21.15.42:0/2398442268 >> v2:172.21.15.42:3300/0 conn(0x7fbe7815ba40 0x7fbe780d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.123+0000 7fbe77400640 1 --2- 172.21.15.42:0/2398442268 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe7806f660 0x7fbe78073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.123+0000 7fbe6fe00640 1 -- 172.21.15.42:0/2398442268 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe7806f660 msgr2=0x7fbe78073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:47.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.123+0000 7fbe6fe00640 1 --2- 172.21.15.42:0/2398442268 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe7806f660 0x7fbe78073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.123+0000 7fbe6fe00640 1 -- 172.21.15.42:0/2398442268 >> v1:172.21.15.42:6789/0 conn(0x7fbe78072dd0 legacy=0x7fbe780b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.123+0000 7fbe6fe00640 1 -- 172.21.15.42:0/2398442268 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7815f840 con 0x7fbe7815ba40 2024-05-28T14:12:47.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe6fe00640 1 --2- 172.21.15.42:0/2398442268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7815ba40 0x7fbe780d06f0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fbe600097c0 tx=0x7fbe6001c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c9fcf59cd60fb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:47.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe76a00640 1 -- 172.21.15.42:0/2398442268 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe600b0070 con 0x7fbe7815ba40 2024-05-28T14:12:47.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe76a00640 1 -- 172.21.15.42:0/2398442268 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe600ab600 con 0x7fbe7815ba40 2024-05-28T14:12:47.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe76a00640 1 -- 172.21.15.42:0/2398442268 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe600ab9d0 con 0x7fbe7815ba40 2024-05-28T14:12:47.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 -- 172.21.15.42:0/2398442268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7815ba40 msgr2=0x7fbe780d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 --2- 172.21.15.42:0/2398442268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7815ba40 0x7fbe780d06f0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fbe600097c0 tx=0x7fbe6001c350 comp rx=0 tx=0).stop 2024-05-28T14:12:47.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 -- 172.21.15.42:0/2398442268 shutdown_connections 2024-05-28T14:12:47.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 --2- 172.21.15.42:0/2398442268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7815ba40 0x7fbe780d06f0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 --2- 172.21.15.42:0/2398442268 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe7806f660 0x7fbe78073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 -- 172.21.15.42:0/2398442268 >> 172.21.15.42:0/2398442268 conn(0x7fbe7806bdf0 msgr2=0x7fbe7806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:47.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 -- 172.21.15.42:0/2398442268 shutdown_connections 2024-05-28T14:12:47.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 -- 172.21.15.42:0/2398442268 wait complete. 2024-05-28T14:12:47.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 Processor -- start 2024-05-28T14:12:47.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 -- start start 2024-05-28T14:12:47.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7806f660 0x7fbe7814aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe78072dd0 0x7fbe78099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe7809a2d0 0x7fbe7814d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbe78166540 con 0x7fbe7806f660 2024-05-28T14:12:47.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbe781663c0 con 0x7fbe78072dd0 2024-05-28T14:12:47.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.127+0000 7fbe7e200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbe781666c0 con 0x7fbe7809a2d0 2024-05-28T14:12:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe77400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7806f660 0x7fbe7814aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe77400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7806f660 0x7fbe7814aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33534/0 (socket says 172.21.15.42:33534) 2024-05-28T14:12:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe77400640 1 -- 172.21.15.42:0/3471699127 learned_addr learned my addr 172.21.15.42:0/3471699127 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe77e00640 1 --2- 172.21.15.42:0/3471699127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe7809a2d0 0x7fbe7814d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe6fe00640 1 --2- 172.21.15.42:0/3471699127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe78072dd0 0x7fbe78099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe77400640 1 -- 172.21.15.42:0/3471699127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe7809a2d0 msgr2=0x7fbe7814d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe77400640 1 --2- 172.21.15.42:0/3471699127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe7809a2d0 0x7fbe7814d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe77400640 1 -- 172.21.15.42:0/3471699127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe78072dd0 msgr2=0x7fbe78099920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe77400640 1 --2- 172.21.15.42:0/3471699127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe78072dd0 0x7fbe78099920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe77400640 1 -- 172.21.15.42:0/3471699127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7814b2b0 con 0x7fbe7806f660 2024-05-28T14:12:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe77400640 1 --2- 172.21.15.42:0/3471699127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7806f660 0x7fbe7814aca0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800d830 tx=0x7fbe6800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe6f400640 1 -- 172.21.15.42:0/3471699127 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6800b6c0 con 0x7fbe7806f660 2024-05-28T14:12:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe6f400640 1 -- 172.21.15.42:0/3471699127 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe68019070 con 0x7fbe7806f660 2024-05-28T14:12:47.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe6f400640 1 -- 172.21.15.42:0/3471699127 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6800ba60 con 0x7fbe7806f660 2024-05-28T14:12:47.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe7e200640 1 -- 172.21.15.42:0/3471699127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7814b590 con 0x7fbe7806f660 2024-05-28T14:12:47.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe7e200640 1 -- 172.21.15.42:0/3471699127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe7814bb50 con 0x7fbe7806f660 2024-05-28T14:12:47.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe7e200640 1 -- 172.21.15.42:0/3471699127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe7809be30 con 0x7fbe7806f660 2024-05-28T14:12:47.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe6f400640 1 -- 172.21.15.42:0/3471699127 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fbe6801e460 con 0x7fbe7806f660 2024-05-28T14:12:47.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.131+0000 7fbe6f400640 1 --2- 172.21.15.42:0/3471699127 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe54069b20 0x7fbe5406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.135+0000 7fbe6f400640 1 -- 172.21.15.42:0/3471699127 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5793+0+0 (secure 0 0 0) 0x7fbe68085e80 con 0x7fbe7806f660 2024-05-28T14:12:47.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.135+0000 7fbe6fe00640 1 --2- 172.21.15.42:0/3471699127 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe54069b20 0x7fbe5406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.135+0000 7fbe6fe00640 1 --2- 172.21.15.42:0/3471699127 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe54069b20 0x7fbe5406bfe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbe600ad000 tx=0x7fbe60008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:47.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.135+0000 7fbe6f400640 1 -- 172.21.15.42:0/3471699127 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbe7809be30 con 0x7fbe7806f660 2024-05-28T14:12:47.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.335+0000 7fbe7e200640 1 -- 172.21.15.42:0/3471699127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe78068a90 con 0x7fbe7806f660 2024-05-28T14:12:47.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.335+0000 7fbe6f400640 1 -- 172.21.15.42:0/3471699127 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5190 (secure 0 0 0) 0x7fbe68057ea0 con 0x7fbe7806f660 2024-05-28T14:12:47.320 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:47.320 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8607,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/1661021064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1661021064},{"type":"v1","addr":"172.21.15.42:6836","nonce":1661021064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":57050,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2398440572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":2398440572},{"type":"v1","addr":"172.21.15.42:6837","nonce":2398440572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":57053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/2530315729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":2530315729},{"type":"v1","addr":"172.21.15.63:6834","nonce":2530315729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:11:14.793551+0000","modified":"2024-05-28T14:11:17.812010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57056},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57056":{"gid":57056,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/1184583947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1184583947},{"type":"v1","addr":"172.21.15.63:6835","nonce":1184583947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57056,"qdb_cluster":[57056]},"id":7}]} 2024-05-28T14:12:47.321 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 65 2024-05-28T14:12:47.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.335+0000 7fbe7e200640 1 -- 172.21.15.42:0/3471699127 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe54069b20 msgr2=0x7fbe5406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.335+0000 7fbe7e200640 1 --2- 172.21.15.42:0/3471699127 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe54069b20 0x7fbe5406bfe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbe600ad000 tx=0x7fbe60008040 comp rx=0 tx=0).stop 2024-05-28T14:12:47.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.335+0000 7fbe7e200640 1 -- 172.21.15.42:0/3471699127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7806f660 msgr2=0x7fbe7814aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.335+0000 7fbe7e200640 1 --2- 172.21.15.42:0/3471699127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7806f660 0x7fbe7814aca0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800d830 tx=0x7fbe6800dcf0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.339+0000 7fbe7e200640 1 -- 172.21.15.42:0/3471699127 shutdown_connections 2024-05-28T14:12:47.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.339+0000 7fbe7e200640 1 --2- 172.21.15.42:0/3471699127 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe54069b20 0x7fbe5406bfe0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.339+0000 7fbe7e200640 1 --2- 172.21.15.42:0/3471699127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe7809a2d0 0x7fbe7814d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.339+0000 7fbe7e200640 1 --2- 172.21.15.42:0/3471699127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe78072dd0 0x7fbe78099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.339+0000 7fbe7e200640 1 --2- 172.21.15.42:0/3471699127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7806f660 0x7fbe7814aca0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.339+0000 7fbe7e200640 1 -- 172.21.15.42:0/3471699127 >> 172.21.15.42:0/3471699127 conn(0x7fbe7806bdf0 msgr2=0x7fbe7806c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:47.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.339+0000 7fbe7e200640 1 -- 172.21.15.42:0/3471699127 shutdown_connections 2024-05-28T14:12:47.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.339+0000 7fbe7e200640 1 -- 172.21.15.42:0/3471699127 wait complete. 2024-05-28T14:12:47.343 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:12:47.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.499+0000 7f946aa00640 1 Processor -- start 2024-05-28T14:12:47.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.499+0000 7f946aa00640 1 -- start start 2024-05-28T14:12:47.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.499+0000 7f946aa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f946406b9f0 0x7f946406be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.499+0000 7f946aa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f94640b5d80 0x7f94640b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.499+0000 7f946aa00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f946415eee0 con 0x7f946415eb00 2024-05-28T14:12:47.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.499+0000 7f946aa00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f94640b66f0 con 0x7f94640b5d80 2024-05-28T14:12:47.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.499+0000 7f946aa00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f94640ceb20 con 0x7f946406b9f0 2024-05-28T14:12:47.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.499+0000 7f9468a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f94640b5d80 0x7f94640b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.499+0000 7f9463e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f946415eb00 0x7f94640cf8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:48562/0 (socket says 172.21.15.42:48562) 2024-05-28T14:12:47.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.499+0000 7f9463e00640 1 -- 172.21.15.42:0/1148118103 learned_addr learned my addr 172.21.15.42:0/1148118103 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:47.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.503+0000 7f945be00640 1 --2- 172.21.15.42:0/1148118103 >> v2:172.21.15.63:3300/0 conn(0x7f946406b9f0 0x7f946406be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.503+0000 7f9468a00640 1 -- 172.21.15.42:0/1148118103 >> v2:172.21.15.63:3300/0 conn(0x7f946406b9f0 msgr2=0x7f946406be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:47.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.503+0000 7f9468a00640 1 --2- 172.21.15.42:0/1148118103 >> v2:172.21.15.63:3300/0 conn(0x7f946406b9f0 0x7f946406be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.503+0000 7f9468a00640 1 -- 172.21.15.42:0/1148118103 >> v1:172.21.15.42:6789/0 conn(0x7f946415eb00 legacy=0x7f94640cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.503+0000 7f9468a00640 1 -- 172.21.15.42:0/1148118103 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94640cee00 con 0x7f94640b5d80 2024-05-28T14:12:47.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.503+0000 7f945be00640 1 --2- 172.21.15.42:0/1148118103 >> v2:172.21.15.63:3300/0 conn(0x7f946406b9f0 0x7f946406be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:47.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.503+0000 7f9468a00640 1 --2- 172.21.15.42:0/1148118103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94640b5d80 0x7f94640b61b0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f94640b7590 tx=0x7f945c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2013c7b76522683 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:47.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.503+0000 7f9463400640 1 -- 172.21.15.42:0/1148118103 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c0b2070 con 0x7f94640b5d80 2024-05-28T14:12:47.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.503+0000 7f9463400640 1 -- 172.21.15.42:0/1148118103 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945c0a57f0 con 0x7f94640b5d80 2024-05-28T14:12:47.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.503+0000 7f9463400640 1 -- 172.21.15.42:0/1148118103 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c0a5bc0 con 0x7f94640b5d80 2024-05-28T14:12:47.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.503+0000 7f946aa00640 1 -- 172.21.15.42:0/1148118103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94640b5d80 msgr2=0x7f94640b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.503+0000 7f946aa00640 1 --2- 172.21.15.42:0/1148118103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94640b5d80 0x7f94640b61b0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f94640b7590 tx=0x7f945c01c930 comp rx=0 tx=0).stop 2024-05-28T14:12:47.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 -- 172.21.15.42:0/1148118103 shutdown_connections 2024-05-28T14:12:47.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 --2- 172.21.15.42:0/1148118103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94640b5d80 0x7f94640b61b0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 --2- 172.21.15.42:0/1148118103 >> v2:172.21.15.63:3300/0 conn(0x7f946406b9f0 0x7f946406be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 -- 172.21.15.42:0/1148118103 >> 172.21.15.42:0/1148118103 conn(0x7f94640b68f0 msgr2=0x7f94640b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:47.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 -- 172.21.15.42:0/1148118103 shutdown_connections 2024-05-28T14:12:47.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 -- 172.21.15.42:0/1148118103 wait complete. 2024-05-28T14:12:47.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 Processor -- start 2024-05-28T14:12:47.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 -- start start 2024-05-28T14:12:47.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946406b9f0 0x7f946414d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94640b5d80 0x7f94640af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f946415eb00 0x7f94640b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f946416b610 con 0x7f94640b5d80 2024-05-28T14:12:47.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f94640ce8d0 con 0x7f946406b9f0 2024-05-28T14:12:47.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f946416b790 con 0x7f946415eb00 2024-05-28T14:12:47.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f945be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94640b5d80 0x7f94640af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f945be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94640b5d80 0x7f94640af0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46578/0 (socket says 172.21.15.42:46578) 2024-05-28T14:12:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f945be00640 1 -- 172.21.15.42:0/3868016447 learned_addr learned my addr 172.21.15.42:0/3868016447 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f9463e00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946406b9f0 0x7f946414d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f9468a00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f946415eb00 0x7f94640b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f945be00640 1 -- 172.21.15.42:0/3868016447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f946415eb00 msgr2=0x7f94640b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f945be00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f946415eb00 0x7f94640b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f945be00640 1 -- 172.21.15.42:0/3868016447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946406b9f0 msgr2=0x7f946414d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f945be00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946406b9f0 0x7f946414d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f945be00640 1 -- 172.21.15.42:0/3868016447 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946415cbb0 con 0x7f94640b5d80 2024-05-28T14:12:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f9468a00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f946415eb00 0x7f94640b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f945be00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94640b5d80 0x7f94640af0f0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f944c00d800 tx=0x7f944c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f9460c00640 1 -- 172.21.15.42:0/3868016447 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c018020 con 0x7f94640b5d80 2024-05-28T14:12:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f9460c00640 1 -- 172.21.15.42:0/3868016447 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944c0096c0 con 0x7f94640b5d80 2024-05-28T14:12:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f9460c00640 1 -- 172.21.15.42:0/3868016447 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c009ab0 con 0x7f94640b5d80 2024-05-28T14:12:47.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.507+0000 7f946aa00640 1 -- 172.21.15.42:0/3868016447 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946415ce90 con 0x7f94640b5d80 2024-05-28T14:12:47.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.511+0000 7f946aa00640 1 -- 172.21.15.42:0/3868016447 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946415d450 con 0x7f94640b5d80 2024-05-28T14:12:47.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.511+0000 7f946aa00640 1 -- 172.21.15.42:0/3868016447 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9464068a90 con 0x7f94640b5d80 2024-05-28T14:12:47.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.511+0000 7f9460c00640 1 -- 172.21.15.42:0/3868016447 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f944c01b020 con 0x7f94640b5d80 2024-05-28T14:12:47.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.511+0000 7f9460c00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9440069b20 0x7f944006bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.511+0000 7f9460c00640 1 -- 172.21.15.42:0/3868016447 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5793+0+0 (secure 0 0 0) 0x7f944c085c70 con 0x7f94640b5d80 2024-05-28T14:12:47.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.511+0000 7f9463e00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9440069b20 0x7f944006bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.515+0000 7f9463e00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9440069b20 0x7f944006bfe0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f94641545e0 tx=0x7f9454009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:47.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.515+0000 7f9460c00640 1 -- 172.21.15.42:0/3868016447 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f944c022020 con 0x7f94640b5d80 2024-05-28T14:12:47.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.683+0000 7f946aa00640 1 -- 172.21.15.42:0/3868016447 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f946415d850 con 0x7f94640b5d80 2024-05-28T14:12:47.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.683+0000 7f9460c00640 1 -- 172.21.15.42:0/3868016447 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16714 (secure 0 0 0) 0x7f944c057d30 con 0x7f94640b5d80 2024-05-28T14:12:47.668 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:47.669 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":106,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:12:46.053529+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:11:13.194240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-28T14:11:13.743150+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:12:47.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.687+0000 7f946aa00640 1 -- 172.21.15.42:0/3868016447 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9440069b20 msgr2=0x7f944006bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.687+0000 7f946aa00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9440069b20 0x7f944006bfe0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f94641545e0 tx=0x7f9454009040 comp rx=0 tx=0).stop 2024-05-28T14:12:47.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.687+0000 7f946aa00640 1 -- 172.21.15.42:0/3868016447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94640b5d80 msgr2=0x7f94640af0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.687+0000 7f946aa00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94640b5d80 0x7f94640af0f0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f944c00d800 tx=0x7f944c00dcc0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.687+0000 7f946aa00640 1 -- 172.21.15.42:0/3868016447 shutdown_connections 2024-05-28T14:12:47.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.687+0000 7f946aa00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9440069b20 0x7f944006bfe0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.687+0000 7f946aa00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f946415eb00 0x7f94640b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.687+0000 7f946aa00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94640b5d80 0x7f94640af0f0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.687+0000 7f946aa00640 1 --2- 172.21.15.42:0/3868016447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946406b9f0 0x7f946414d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.687+0000 7f946aa00640 1 -- 172.21.15.42:0/3868016447 >> 172.21.15.42:0/3868016447 conn(0x7f94640b68f0 msgr2=0x7f94640b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:47.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.687+0000 7f946aa00640 1 -- 172.21.15.42:0/3868016447 shutdown_connections 2024-05-28T14:12:47.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.687+0000 7f946aa00640 1 -- 172.21.15.42:0/3868016447 wait complete. 2024-05-28T14:12:47.689 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-28T14:12:47.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.847+0000 7fbf96a00640 1 Processor -- start 2024-05-28T14:12:47.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.847+0000 7fbf96a00640 1 -- start start 2024-05-28T14:12:47.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.847+0000 7fbf96a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9009aa50 0x7fbf9009ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.847+0000 7fbf96a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf9009b370 con 0x7fbf900d15c0 2024-05-28T14:12:47.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.847+0000 7fbf96a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf901622a0 con 0x7fbf900b4bc0 2024-05-28T14:12:47.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.847+0000 7fbf96a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf900ccee0 con 0x7fbf9009aa50 2024-05-28T14:12:47.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.847+0000 7fbf94a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fbf900d15c0 0x7fbf900cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:48578/0 (socket says 172.21.15.42:48578) 2024-05-28T14:12:47.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.847+0000 7fbf94a00640 1 -- 172.21.15.42:0/3426009644 learned_addr learned my addr 172.21.15.42:0/3426009644 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:47.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.847+0000 7fbf8fe00640 1 --2- 172.21.15.42:0/3426009644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9009aa50 0x7fbf9009ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.847+0000 7fbf8f400640 1 -- 172.21.15.42:0/3426009644 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2232994799 0 0) 0x7fbf9009b370 con 0x7fbf900d15c0 2024-05-28T14:12:47.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.847+0000 7fbf8f400640 1 -- 172.21.15.42:0/3426009644 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf74003530 con 0x7fbf900d15c0 2024-05-28T14:12:47.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.851+0000 7fbf8f400640 1 -- 172.21.15.42:0/3426009644 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3319485466 0 0) 0x7fbf901622a0 con 0x7fbf900b4bc0 2024-05-28T14:12:47.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.851+0000 7fbf8f400640 1 -- 172.21.15.42:0/3426009644 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf9009b370 con 0x7fbf900b4bc0 2024-05-28T14:12:47.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.851+0000 7fbf8fe00640 1 -- 172.21.15.42:0/3426009644 >> v1:172.21.15.63:6789/0 conn(0x7fbf900b4bc0 legacy=0x7fbf900cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.851+0000 7fbf8fe00640 1 -- 172.21.15.42:0/3426009644 >> v1:172.21.15.42:6789/0 conn(0x7fbf900d15c0 legacy=0x7fbf900cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.851+0000 7fbf8fe00640 1 -- 172.21.15.42:0/3426009644 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9009b620 con 0x7fbf9009aa50 2024-05-28T14:12:47.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.851+0000 7fbf8fe00640 1 --2- 172.21.15.42:0/3426009644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9009aa50 0x7fbf9009ae30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbf800097f0 tx=0x7fbf8001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f167475da186a6d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:47.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.851+0000 7fbf8f400640 1 -- 172.21.15.42:0/3426009644 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf800aa430 con 0x7fbf9009aa50 2024-05-28T14:12:47.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.851+0000 7fbf8f400640 1 -- 172.21.15.42:0/3426009644 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf800af070 con 0x7fbf9009aa50 2024-05-28T14:12:47.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.851+0000 7fbf8f400640 1 -- 172.21.15.42:0/3426009644 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf800aa7f0 con 0x7fbf9009aa50 2024-05-28T14:12:47.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.851+0000 7fbf96a00640 1 -- 172.21.15.42:0/3426009644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9009aa50 msgr2=0x7fbf9009ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.851+0000 7fbf96a00640 1 --2- 172.21.15.42:0/3426009644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9009aa50 0x7fbf9009ae30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbf800097f0 tx=0x7fbf8001c4b0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.851+0000 7fbf96a00640 1 -- 172.21.15.42:0/3426009644 shutdown_connections 2024-05-28T14:12:47.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.851+0000 7fbf96a00640 1 --2- 172.21.15.42:0/3426009644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf9009aa50 0x7fbf9009ae30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.851+0000 7fbf96a00640 1 -- 172.21.15.42:0/3426009644 >> 172.21.15.42:0/3426009644 conn(0x7fbf900b4010 msgr2=0x7fbf900b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:47.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf96a00640 1 -- 172.21.15.42:0/3426009644 shutdown_connections 2024-05-28T14:12:47.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf96a00640 1 -- 172.21.15.42:0/3426009644 wait complete. 2024-05-28T14:12:47.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf96a00640 1 Processor -- start 2024-05-28T14:12:47.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf96a00640 1 -- start start 2024-05-28T14:12:47.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf96a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf9009aa50 0x7fbf900c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf96a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf900b4bc0 0x7fbf90159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf96a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf900d15c0 0x7fbf900c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf96a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbf900ce5a0 con 0x7fbf900d15c0 2024-05-28T14:12:47.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf96a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf900ccb70 con 0x7fbf9009aa50 2024-05-28T14:12:47.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf96a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf9016cdc0 con 0x7fbf900b4bc0 2024-05-28T14:12:47.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf94a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf900d15c0 0x7fbf900c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf8fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf9009aa50 0x7fbf900c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf94a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf900d15c0 0x7fbf900c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46588/0 (socket says 172.21.15.42:46588) 2024-05-28T14:12:47.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf94a00640 1 -- 172.21.15.42:0/1258967747 learned_addr learned my addr 172.21.15.42:0/1258967747 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:47.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf87400640 1 --2- 172.21.15.42:0/1258967747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf900b4bc0 0x7fbf90159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf94a00640 1 -- 172.21.15.42:0/1258967747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf900b4bc0 msgr2=0x7fbf90159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf94a00640 1 --2- 172.21.15.42:0/1258967747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf900b4bc0 0x7fbf90159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf94a00640 1 -- 172.21.15.42:0/1258967747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf9009aa50 msgr2=0x7fbf900c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf94a00640 1 --2- 172.21.15.42:0/1258967747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf9009aa50 0x7fbf900c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf94a00640 1 -- 172.21.15.42:0/1258967747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf900c8ad0 con 0x7fbf900d15c0 2024-05-28T14:12:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf94a00640 1 --2- 172.21.15.42:0/1258967747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf900d15c0 0x7fbf900c95f0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800cb40 tx=0x7fbf8800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf8cc00640 1 -- 172.21.15.42:0/1258967747 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88017070 con 0x7fbf900d15c0 2024-05-28T14:12:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf8cc00640 1 -- 172.21.15.42:0/1258967747 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8800acc0 con 0x7fbf900d15c0 2024-05-28T14:12:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf96a00640 1 -- 172.21.15.42:0/1258967747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf90158f50 con 0x7fbf900d15c0 2024-05-28T14:12:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.855+0000 7fbf8cc00640 1 -- 172.21.15.42:0/1258967747 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88012640 con 0x7fbf900d15c0 2024-05-28T14:12:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.859+0000 7fbf96a00640 1 -- 172.21.15.42:0/1258967747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf90159510 con 0x7fbf900d15c0 2024-05-28T14:12:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.859+0000 7fbf96a00640 1 -- 172.21.15.42:0/1258967747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf900a0630 con 0x7fbf900d15c0 2024-05-28T14:12:47.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.859+0000 7fbf8cc00640 1 -- 172.21.15.42:0/1258967747 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fbf8801a070 con 0x7fbf900d15c0 2024-05-28T14:12:47.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.859+0000 7fbf8cc00640 1 --2- 172.21.15.42:0/1258967747 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf6c069b20 0x7fbf6c06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:47.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.859+0000 7fbf8cc00640 1 -- 172.21.15.42:0/1258967747 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5793+0+0 (secure 0 0 0) 0x7fbf88085590 con 0x7fbf900d15c0 2024-05-28T14:12:47.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.859+0000 7fbf8fe00640 1 --2- 172.21.15.42:0/1258967747 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf6c069b20 0x7fbf6c06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:47.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.859+0000 7fbf8fe00640 1 --2- 172.21.15.42:0/1258967747 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf6c069b20 0x7fbf6c06bfe0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf800ae000 tx=0x7fbf800a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:47.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:47.863+0000 7fbf8cc00640 1 -- 172.21.15.42:0/1258967747 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbf88056ee0 con 0x7fbf900d15c0 2024-05-28T14:12:48.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.067+0000 7fbf96a00640 1 -- 172.21.15.42:0/1258967747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbf9006cc00 con 0x7fbf900d15c0 2024-05-28T14:12:48.054 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-05-28T14:12:48.071+0000 7f248b600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-28T14:12:48.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.071+0000 7fbf8cc00640 1 -- 172.21.15.42:0/1258967747 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) ==== 161+0+0 (secure 0 0 0) 0x7fbf8805b1e0 con 0x7fbf900d15c0 2024-05-28T14:12:48.056 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-28T14:12:48.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.075+0000 7fbf96a00640 1 -- 172.21.15.42:0/1258967747 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf6c069b20 msgr2=0x7fbf6c06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:48.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.075+0000 7fbf96a00640 1 --2- 172.21.15.42:0/1258967747 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf6c069b20 0x7fbf6c06bfe0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf800ae000 tx=0x7fbf800a7000 comp rx=0 tx=0).stop 2024-05-28T14:12:48.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.075+0000 7fbf96a00640 1 -- 172.21.15.42:0/1258967747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf900d15c0 msgr2=0x7fbf900c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:48.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.075+0000 7fbf96a00640 1 --2- 172.21.15.42:0/1258967747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf900d15c0 0x7fbf900c95f0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800cb40 tx=0x7fbf8800a060 comp rx=0 tx=0).stop 2024-05-28T14:12:48.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.075+0000 7fbf96a00640 1 -- 172.21.15.42:0/1258967747 shutdown_connections 2024-05-28T14:12:48.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.075+0000 7fbf96a00640 1 --2- 172.21.15.42:0/1258967747 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbf6c069b20 0x7fbf6c06bfe0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:48.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.075+0000 7fbf96a00640 1 --2- 172.21.15.42:0/1258967747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf900d15c0 0x7fbf900c95f0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:48.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.075+0000 7fbf96a00640 1 --2- 172.21.15.42:0/1258967747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf900b4bc0 0x7fbf90159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:48.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.075+0000 7fbf96a00640 1 --2- 172.21.15.42:0/1258967747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf9009aa50 0x7fbf900c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:48.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.075+0000 7fbf96a00640 1 -- 172.21.15.42:0/1258967747 >> 172.21.15.42:0/1258967747 conn(0x7fbf900b4010 msgr2=0x7fbf901577e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:48.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.075+0000 7fbf96a00640 1 -- 172.21.15.42:0/1258967747 shutdown_connections 2024-05-28T14:12:48.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.075+0000 7fbf96a00640 1 -- 172.21.15.42:0/1258967747 wait complete. 2024-05-28T14:12:48.079 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-28T14:12:48.130 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T14:12:48.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f41800640 1 Processor -- start 2024-05-28T14:12:48.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f41800640 1 -- start start 2024-05-28T14:12:48.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f41800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8f3c148090 0x7f8f3c0b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:48.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f41800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8f3c0724f0 0x7f8f3c0aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:48.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f41800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f3c0b3690 con 0x7f8f3c0b4c10 2024-05-28T14:12:48.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f41800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8f3c148470 con 0x7f8f3c148090 2024-05-28T14:12:48.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f41800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8f3c0b4730 con 0x7f8f3c0724f0 2024-05-28T14:12:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f33e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8f3c0b4c10 0x7f8f3c071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:48590/0 (socket says 172.21.15.42:48590) 2024-05-28T14:12:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f33e00640 1 -- 172.21.15.42:0/923817919 learned_addr learned my addr 172.21.15.42:0/923817919 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f3aa00640 1 --2- 172.21.15.42:0/923817919 >> v2:172.21.15.42:3300/0 conn(0x7f8f3c148090 0x7f8f3c0b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f3b400640 1 --2- 172.21.15.42:0/923817919 >> v2:172.21.15.63:3300/0 conn(0x7f8f3c0724f0 0x7f8f3c0aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f3a000640 1 -- 172.21.15.42:0/923817919 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3783585075 0 0) 0x7f8f3c0b3690 con 0x7f8f3c0b4c10 2024-05-28T14:12:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f3a000640 1 -- 172.21.15.42:0/923817919 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f20003530 con 0x7f8f3c0b4c10 2024-05-28T14:12:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f3aa00640 1 -- 172.21.15.42:0/923817919 >> v2:172.21.15.63:3300/0 conn(0x7f8f3c0724f0 msgr2=0x7f8f3c0aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f3aa00640 1 --2- 172.21.15.42:0/923817919 >> v2:172.21.15.63:3300/0 conn(0x7f8f3c0724f0 0x7f8f3c0aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f3aa00640 1 -- 172.21.15.42:0/923817919 >> v1:172.21.15.42:6789/0 conn(0x7f8f3c0b4c10 legacy=0x7f8f3c071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.239+0000 7f8f3aa00640 1 -- 172.21.15.42:0/923817919 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c0b3970 con 0x7f8f3c148090 2024-05-28T14:12:48.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.243+0000 7f8f3aa00640 1 --2- 172.21.15.42:0/923817919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f3c148090 0x7f8f3c0b5120 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0b0110 tx=0x7f8f2c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d80d574d479e8eb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:48.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.243+0000 7f8f3a000640 1 -- 172.21.15.42:0/923817919 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0b1070 con 0x7f8f3c148090 2024-05-28T14:12:48.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.243+0000 7f8f3a000640 1 -- 172.21.15.42:0/923817919 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f2c0ac620 con 0x7f8f3c148090 2024-05-28T14:12:48.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.243+0000 7f8f3a000640 1 -- 172.21.15.42:0/923817919 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0ac9f0 con 0x7f8f3c148090 2024-05-28T14:12:48.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.243+0000 7f8f41800640 1 -- 172.21.15.42:0/923817919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f3c148090 msgr2=0x7f8f3c0b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:48.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.243+0000 7f8f41800640 1 --2- 172.21.15.42:0/923817919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f3c148090 0x7f8f3c0b5120 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0b0110 tx=0x7f8f2c01c3c0 comp rx=0 tx=0).stop 2024-05-28T14:12:48.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.243+0000 7f8f41800640 1 -- 172.21.15.42:0/923817919 shutdown_connections 2024-05-28T14:12:48.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.243+0000 7f8f41800640 1 --2- 172.21.15.42:0/923817919 >> v2:172.21.15.63:3300/0 conn(0x7f8f3c0724f0 0x7f8f3c0aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:48.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.243+0000 7f8f41800640 1 --2- 172.21.15.42:0/923817919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f3c148090 0x7f8f3c0b5120 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:48.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.243+0000 7f8f41800640 1 -- 172.21.15.42:0/923817919 >> 172.21.15.42:0/923817919 conn(0x7f8f3c0b21f0 msgr2=0x7f8f3c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:48.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.243+0000 7f8f41800640 1 -- 172.21.15.42:0/923817919 shutdown_connections 2024-05-28T14:12:48.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.243+0000 7f8f41800640 1 -- 172.21.15.42:0/923817919 wait complete. 2024-05-28T14:12:48.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.243+0000 7f8f41800640 1 Processor -- start 2024-05-28T14:12:48.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.243+0000 7f8f41800640 1 -- start start 2024-05-28T14:12:48.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f41800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f3c0724f0 0x7f8f3c142440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:48.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f41800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f3c148090 0x7f8f3c161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:48.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f41800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f3c1f73e0 0x7f8f3c1f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:48.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f41800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c0b57e0 con 0x7f8f3c148090 2024-05-28T14:12:48.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f41800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c0b5660 con 0x7f8f3c1f73e0 2024-05-28T14:12:48.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f41800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3c0af890 con 0x7f8f3c0724f0 2024-05-28T14:12:48.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f33e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f3c148090 0x7f8f3c161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:48.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f33e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f3c148090 0x7f8f3c161570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46618/0 (socket says 172.21.15.42:46618) 2024-05-28T14:12:48.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f33e00640 1 -- 172.21.15.42:0/1962054723 learned_addr learned my addr 172.21.15.42:0/1962054723 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:48.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f3aa00640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f3c0724f0 0x7f8f3c142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:48.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f3b400640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f3c1f73e0 0x7f8f3c1f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:48.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f33e00640 1 -- 172.21.15.42:0/1962054723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f3c0724f0 msgr2=0x7f8f3c142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:48.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f33e00640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f3c0724f0 0x7f8f3c142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:48.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f33e00640 1 -- 172.21.15.42:0/1962054723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f3c1f73e0 msgr2=0x7f8f3c1f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:48.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f33e00640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f3c1f73e0 0x7f8f3c1f7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:48.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f33e00640 1 -- 172.21.15.42:0/1962054723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c15fd00 con 0x7f8f3c148090 2024-05-28T14:12:48.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f3aa00640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f3c0724f0 0x7f8f3c142440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:48.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f33e00640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f3c148090 0x7f8f3c161570 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400c480 tx=0x7f8f2400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:48.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f32a00640 1 -- 172.21.15.42:0/1962054723 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24017070 con 0x7f8f3c148090 2024-05-28T14:12:48.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f32a00640 1 -- 172.21.15.42:0/1962054723 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f24005cf0 con 0x7f8f3c148090 2024-05-28T14:12:48.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f32a00640 1 -- 172.21.15.42:0/1962054723 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24007290 con 0x7f8f3c148090 2024-05-28T14:12:48.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f41800640 1 -- 172.21.15.42:0/1962054723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3c1f7f60 con 0x7f8f3c148090 2024-05-28T14:12:48.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f41800640 1 -- 172.21.15.42:0/1962054723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3c141c40 con 0x7f8f3c148090 2024-05-28T14:12:48.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.247+0000 7f8f41800640 1 -- 172.21.15.42:0/1962054723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f3c141a30 con 0x7f8f3c148090 2024-05-28T14:12:48.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.251+0000 7f8f32a00640 1 -- 172.21.15.42:0/1962054723 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f8f2401a070 con 0x7f8f3c148090 2024-05-28T14:12:48.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.251+0000 7f8f32a00640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8f18069b20 0x7f8f1806bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:48.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.251+0000 7f8f32a00640 1 -- 172.21.15.42:0/1962054723 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5851+0+0 (secure 0 0 0) 0x7f8f24084b80 con 0x7f8f3c148090 2024-05-28T14:12:48.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.251+0000 7f8f3aa00640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8f18069b20 0x7f8f1806bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:48.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.251+0000 7f8f3aa00640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8f18069b20 0x7f8f1806bfe0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0ae000 tx=0x7f8f2c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:48.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.255+0000 7f8f32a00640 1 -- 172.21.15.42:0/1962054723 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8f24056c40 con 0x7f8f3c148090 2024-05-28T14:12:48.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:48.455+0000 7f8f41800640 1 -- 172.21.15.42:0/1962054723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8f3c073890 con 0x7f8f3c148090 2024-05-28T14:12:49.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.071+0000 7f8f32a00640 1 -- 172.21.15.42:0/1962054723 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) ==== 104+0+0 (secure 0 0 0) 0x7f8f24056540 con 0x7f8f3c148090 2024-05-28T14:12:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.075+0000 7f8f41800640 1 -- 172.21.15.42:0/1962054723 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8f18069b20 msgr2=0x7f8f1806bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.075+0000 7f8f41800640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8f18069b20 0x7f8f1806bfe0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0ae000 tx=0x7f8f2c008040 comp rx=0 tx=0).stop 2024-05-28T14:12:49.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.075+0000 7f8f41800640 1 -- 172.21.15.42:0/1962054723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f3c148090 msgr2=0x7f8f3c161570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.075+0000 7f8f41800640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f3c148090 0x7f8f3c161570 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400c480 tx=0x7f8f2400c940 comp rx=0 tx=0).stop 2024-05-28T14:12:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.075+0000 7f8f41800640 1 -- 172.21.15.42:0/1962054723 shutdown_connections 2024-05-28T14:12:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.075+0000 7f8f41800640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8f18069b20 0x7f8f1806bfe0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.075+0000 7f8f41800640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f3c1f73e0 0x7f8f3c1f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.075+0000 7f8f41800640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f3c148090 0x7f8f3c161570 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.075+0000 7f8f41800640 1 --2- 172.21.15.42:0/1962054723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f3c0724f0 0x7f8f3c142440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.075+0000 7f8f41800640 1 -- 172.21.15.42:0/1962054723 >> 172.21.15.42:0/1962054723 conn(0x7f8f3c0b21f0 msgr2=0x7f8f3c06edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.075+0000 7f8f41800640 1 -- 172.21.15.42:0/1962054723 shutdown_connections 2024-05-28T14:12:49.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.075+0000 7f8f41800640 1 -- 172.21.15.42:0/1962054723 wait complete. 2024-05-28T14:12:49.083 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-28T14:12:49.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069e200640 1 Processor -- start 2024-05-28T14:12:49.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069e200640 1 -- start start 2024-05-28T14:12:49.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f069806f6b0 0x7f069815c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:49.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069e200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f069815ceb0 0x7f0698072a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:49.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069e200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f069815d6f0 con 0x7f06980ccdc0 2024-05-28T14:12:49.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069e200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f069815d9d0 con 0x7f069815ceb0 2024-05-28T14:12:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069e200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f069806ebd0 con 0x7f069806f6b0 2024-05-28T14:12:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069d800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f069815ceb0 0x7f0698072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f068fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f069806f6b0 0x7f069815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069d800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f069815ceb0 0x7f0698072a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:48686/0 (socket says 172.21.15.42:48686) 2024-05-28T14:12:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069d800640 1 -- 172.21.15.42:0/3106483338 learned_addr learned my addr 172.21.15.42:0/3106483338 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069ce00640 1 --1- 172.21.15.42:0/3106483338 >> v1:172.21.15.63:6789/0 conn(0x7f06980ccdc0 0x7f06980cd1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:51956/0 (socket says 172.21.15.42:51956) 2024-05-28T14:12:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069d800640 1 -- 172.21.15.42:0/3106483338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f069806f6b0 msgr2=0x7f069815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069d800640 1 --2- 172.21.15.42:0/3106483338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f069806f6b0 0x7f069815c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069d800640 1 -- 172.21.15.42:0/3106483338 >> v1:172.21.15.63:6789/0 conn(0x7f06980ccdc0 legacy=0x7f06980cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.247+0000 7f069d800640 1 -- 172.21.15.42:0/3106483338 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069806eeb0 con 0x7f069815ceb0 2024-05-28T14:12:49.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.251+0000 7f069d800640 1 --2- 172.21.15.42:0/3106483338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f069815ceb0 0x7f0698072a10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0690009e50 tx=0x7f069001ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b1335aaa80461db server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:49.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.251+0000 7f0697e00640 1 -- 172.21.15.42:0/3106483338 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06900a5700 con 0x7f069815ceb0 2024-05-28T14:12:49.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.251+0000 7f0697e00640 1 -- 172.21.15.42:0/3106483338 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06900a58a0 con 0x7f069815ceb0 2024-05-28T14:12:49.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.251+0000 7f0697e00640 1 -- 172.21.15.42:0/3106483338 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06900a5c70 con 0x7f069815ceb0 2024-05-28T14:12:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.251+0000 7f069e200640 1 -- 172.21.15.42:0/3106483338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f069815ceb0 msgr2=0x7f0698072a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.251+0000 7f069e200640 1 --2- 172.21.15.42:0/3106483338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f069815ceb0 0x7f0698072a10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0690009e50 tx=0x7f069001ca30 comp rx=0 tx=0).stop 2024-05-28T14:12:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.251+0000 7f069e200640 1 -- 172.21.15.42:0/3106483338 shutdown_connections 2024-05-28T14:12:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.251+0000 7f069e200640 1 --2- 172.21.15.42:0/3106483338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f069815ceb0 0x7f0698072a10 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.251+0000 7f069e200640 1 --2- 172.21.15.42:0/3106483338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f069806f6b0 0x7f069815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.251+0000 7f069e200640 1 -- 172.21.15.42:0/3106483338 >> 172.21.15.42:0/3106483338 conn(0x7f069806bbc0 msgr2=0x7f069806bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.251+0000 7f069e200640 1 -- 172.21.15.42:0/3106483338 shutdown_connections 2024-05-28T14:12:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.251+0000 7f069e200640 1 -- 172.21.15.42:0/3106483338 wait complete. 2024-05-28T14:12:49.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f069e200640 1 Processor -- start 2024-05-28T14:12:49.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f069e200640 1 -- start start 2024-05-28T14:12:49.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f069e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f069806f6b0 0x7f0698156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:49.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f069e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06980ccdc0 0x7f06981fa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:49.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f069e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f069815ceb0 0x7f06981536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:49.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f069e200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f069806fa90 con 0x7f06980ccdc0 2024-05-28T14:12:49.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f069e200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0698160ec0 con 0x7f069815ceb0 2024-05-28T14:12:49.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f069e200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0698169350 con 0x7f069806f6b0 2024-05-28T14:12:49.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f068fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06980ccdc0 0x7f06981fa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:49.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f068fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06980ccdc0 0x7f06981fa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46628/0 (socket says 172.21.15.42:46628) 2024-05-28T14:12:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f068fe00640 1 -- 172.21.15.42:0/935719710 learned_addr learned my addr 172.21.15.42:0/935719710 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f069ce00640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f069806f6b0 0x7f0698156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f069d800640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f069815ceb0 0x7f06981536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f068fe00640 1 -- 172.21.15.42:0/935719710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f069806f6b0 msgr2=0x7f0698156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f068fe00640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f069806f6b0 0x7f0698156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f068fe00640 1 -- 172.21.15.42:0/935719710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f069815ceb0 msgr2=0x7f06981536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f068fe00640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f069815ceb0 0x7f06981536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f068fe00640 1 -- 172.21.15.42:0/935719710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06980adcb0 con 0x7f06980ccdc0 2024-05-28T14:12:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f069ce00640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f069806f6b0 0x7f0698156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f068fe00640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06980ccdc0 0x7f06981fa9c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f068000e930 tx=0x7f068000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f0695600640 1 -- 172.21.15.42:0/935719710 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680017070 con 0x7f06980ccdc0 2024-05-28T14:12:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f0695600640 1 -- 172.21.15.42:0/935719710 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f068000ca00 con 0x7f06980ccdc0 2024-05-28T14:12:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f0695600640 1 -- 172.21.15.42:0/935719710 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068000cdb0 con 0x7f06980ccdc0 2024-05-28T14:12:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f069e200640 1 -- 172.21.15.42:0/935719710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0698150b80 con 0x7f06980ccdc0 2024-05-28T14:12:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f069e200640 1 -- 172.21.15.42:0/935719710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06980adf70 con 0x7f06980ccdc0 2024-05-28T14:12:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.255+0000 7f069e200640 1 -- 172.21.15.42:0/935719710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0698068a90 con 0x7f06980ccdc0 2024-05-28T14:12:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.259+0000 7f0695600640 1 -- 172.21.15.42:0/935719710 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f068001a070 con 0x7f06980ccdc0 2024-05-28T14:12:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.259+0000 7f0695600640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0674069b20 0x7f067406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.259+0000 7f0695600640 1 -- 172.21.15.42:0/935719710 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5851+0+0 (secure 0 0 0) 0x7f0680085730 con 0x7f06980ccdc0 2024-05-28T14:12:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.259+0000 7f069ce00640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0674069b20 0x7f067406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.259+0000 7f069ce00640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0674069b20 0x7f067406bfe0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0688000f70 tx=0x7f0688007d70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.263+0000 7f0695600640 1 -- 172.21.15.42:0/935719710 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f06800571c0 con 0x7f06980ccdc0 2024-05-28T14:12:49.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:49.427+0000 7f069e200640 1 -- 172.21.15.42:0/935719710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f06980cd320 con 0x7f06980ccdc0 2024-05-28T14:12:50.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.083+0000 7f0695600640 1 -- 172.21.15.42:0/935719710 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7f068005b4c0 con 0x7f06980ccdc0 2024-05-28T14:12:50.067 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-05-28T14:12:50.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.087+0000 7f069e200640 1 -- 172.21.15.42:0/935719710 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0674069b20 msgr2=0x7f067406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:50.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.087+0000 7f069e200640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0674069b20 0x7f067406bfe0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0688000f70 tx=0x7f0688007d70 comp rx=0 tx=0).stop 2024-05-28T14:12:50.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.087+0000 7f069e200640 1 -- 172.21.15.42:0/935719710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06980ccdc0 msgr2=0x7f06981fa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:50.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.087+0000 7f069e200640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06980ccdc0 0x7f06981fa9c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f068000e930 tx=0x7f068000edf0 comp rx=0 tx=0).stop 2024-05-28T14:12:50.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.087+0000 7f069e200640 1 -- 172.21.15.42:0/935719710 shutdown_connections 2024-05-28T14:12:50.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.087+0000 7f069e200640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0674069b20 0x7f067406bfe0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:50.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.087+0000 7f069e200640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f069815ceb0 0x7f06981536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:50.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.087+0000 7f069e200640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06980ccdc0 0x7f06981fa9c0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:50.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.087+0000 7f069e200640 1 --2- 172.21.15.42:0/935719710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f069806f6b0 0x7f0698156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:50.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.087+0000 7f069e200640 1 -- 172.21.15.42:0/935719710 >> 172.21.15.42:0/935719710 conn(0x7f069806bbc0 msgr2=0x7f06980b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:50.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.087+0000 7f069e200640 1 -- 172.21.15.42:0/935719710 shutdown_connections 2024-05-28T14:12:50.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.087+0000 7f069e200640 1 -- 172.21.15.42:0/935719710 wait complete. 2024-05-28T14:12:50.096 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-28T14:12:50.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.259+0000 7f32be800640 1 Processor -- start 2024-05-28T14:12:50.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.259+0000 7f32be800640 1 -- start start 2024-05-28T14:12:50.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.263+0000 7f32be800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b815b350 0x7f32b806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:50.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.263+0000 7f32be800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f32b80d0e00 0x7f32b80aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:50.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.263+0000 7f32be800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b80ade40 con 0x7f32b8072d40 2024-05-28T14:12:50.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.263+0000 7f32be800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f32b80ae9f0 con 0x7f32b80d0e00 2024-05-28T14:12:50.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.263+0000 7f32be800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f32b80ae0c0 con 0x7f32b815b350 2024-05-28T14:12:50.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.263+0000 7f32b7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f32b80d0e00 0x7f32b80aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:50.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.263+0000 7f32aea00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f32b8072d40 0x7f32b80d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:48602/0 (socket says 172.21.15.42:48602) 2024-05-28T14:12:50.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.263+0000 7f32aea00640 1 -- 172.21.15.42:0/3414789451 learned_addr learned my addr 172.21.15.42:0/3414789451 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:50.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.263+0000 7f32b7400640 1 --2- 172.21.15.42:0/3414789451 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b815b350 0x7f32b806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:50.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.263+0000 7f32b7e00640 1 -- 172.21.15.42:0/3414789451 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b815b350 msgr2=0x7f32b806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:50.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.263+0000 7f32b7e00640 1 --2- 172.21.15.42:0/3414789451 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b815b350 0x7f32b806dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:50.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.263+0000 7f32b7e00640 1 -- 172.21.15.42:0/3414789451 >> v1:172.21.15.42:6789/0 conn(0x7f32b8072d40 legacy=0x7f32b80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:50.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.263+0000 7f32b7e00640 1 -- 172.21.15.42:0/3414789451 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b80ae3a0 con 0x7f32b80d0e00 2024-05-28T14:12:50.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.263+0000 7f32b6a00640 1 -- 172.21.15.42:0/3414789451 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2910717699 0 0) 0x7f32a0001120 con 0x7f32b8072d40 2024-05-28T14:12:50.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32b7e00640 1 --2- 172.21.15.42:0/3414789451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b80d0e00 0x7f32b80aee80 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009d90 tx=0x7f32b001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596e942cefcb3e57 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:50.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32b6a00640 1 -- 172.21.15.42:0/3414789451 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b00b2070 con 0x7f32b80d0e00 2024-05-28T14:12:50.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32b6a00640 1 -- 172.21.15.42:0/3414789451 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b00a5820 con 0x7f32b80d0e00 2024-05-28T14:12:50.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32b6a00640 1 -- 172.21.15.42:0/3414789451 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b00a5bf0 con 0x7f32b80d0e00 2024-05-28T14:12:50.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 -- 172.21.15.42:0/3414789451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b80d0e00 msgr2=0x7f32b80aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:50.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 --2- 172.21.15.42:0/3414789451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b80d0e00 0x7f32b80aee80 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009d90 tx=0x7f32b001c8e0 comp rx=0 tx=0).stop 2024-05-28T14:12:50.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 -- 172.21.15.42:0/3414789451 shutdown_connections 2024-05-28T14:12:50.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 --2- 172.21.15.42:0/3414789451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b80d0e00 0x7f32b80aee80 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:50.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 --2- 172.21.15.42:0/3414789451 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b815b350 0x7f32b806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:50.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 -- 172.21.15.42:0/3414789451 >> 172.21.15.42:0/3414789451 conn(0x7f32b806bdf0 msgr2=0x7f32b806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:50.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 -- 172.21.15.42:0/3414789451 shutdown_connections 2024-05-28T14:12:50.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 -- 172.21.15.42:0/3414789451 wait complete. 2024-05-28T14:12:50.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 Processor -- start 2024-05-28T14:12:50.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 -- start start 2024-05-28T14:12:50.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b8072d40 0x7f32b814e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:50.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b80d0e00 0x7f32b8099970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:50.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b815b350 0x7f32b8154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:50.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8168470 con 0x7f32b815b350 2024-05-28T14:12:50.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81682f0 con 0x7f32b8072d40 2024-05-28T14:12:50.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32be800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f32b81685f0 con 0x7f32b80d0e00 2024-05-28T14:12:50.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b815b350 0x7f32b8154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b815b350 0x7f32b8154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46650/0 (socket says 172.21.15.42:46650) 2024-05-28T14:12:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.267+0000 7f32b7e00640 1 -- 172.21.15.42:0/2143195661 learned_addr learned my addr 172.21.15.42:0/2143195661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32aea00640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b80d0e00 0x7f32b8099970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32b7400640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b8072d40 0x7f32b814e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32b7e00640 1 -- 172.21.15.42:0/2143195661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b80d0e00 msgr2=0x7f32b8099970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32b7e00640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b80d0e00 0x7f32b8099970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32b7e00640 1 -- 172.21.15.42:0/2143195661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b8072d40 msgr2=0x7f32b814e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32b7e00640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b8072d40 0x7f32b814e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32b7e00640 1 -- 172.21.15.42:0/2143195661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b8099eb0 con 0x7f32b815b350 2024-05-28T14:12:50.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32b7400640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b8072d40 0x7f32b814e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32b7e00640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b815b350 0x7f32b8154560 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f32b001af90 tx=0x7f32b0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32afe00640 1 -- 172.21.15.42:0/2143195661 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b00b2050 con 0x7f32b815b350 2024-05-28T14:12:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32afe00640 1 -- 172.21.15.42:0/2143195661 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b00be070 con 0x7f32b815b350 2024-05-28T14:12:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32be800640 1 -- 172.21.15.42:0/2143195661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b815c5b0 con 0x7f32b815b350 2024-05-28T14:12:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32afe00640 1 -- 172.21.15.42:0/2143195661 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b00ada50 con 0x7f32b815b350 2024-05-28T14:12:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32be800640 1 -- 172.21.15.42:0/2143195661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b815cb70 con 0x7f32b815b350 2024-05-28T14:12:50.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32be800640 1 -- 172.21.15.42:0/2143195661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b8068a90 con 0x7f32b815b350 2024-05-28T14:12:50.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32afe00640 1 -- 172.21.15.42:0/2143195661 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f32b00a56a0 con 0x7f32b815b350 2024-05-28T14:12:50.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.271+0000 7f32afe00640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3294069b20 0x7f329406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:50.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.275+0000 7f32afe00640 1 -- 172.21.15.42:0/2143195661 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5413+0+0 (secure 0 0 0) 0x7f32b0120a30 con 0x7f32b815b350 2024-05-28T14:12:50.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.275+0000 7f32b7400640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3294069b20 0x7f329406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:50.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.275+0000 7f32b7400640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3294069b20 0x7f329406bfe0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f32a80097a0 tx=0x7f32a8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:50.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.279+0000 7f32afe00640 1 -- 172.21.15.42:0/2143195661 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f32b0016560 con 0x7f32b815b350 2024-05-28T14:12:50.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:50.443+0000 7f32be800640 1 -- 172.21.15.42:0/2143195661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f32b8156020 con 0x7f32b815b350 2024-05-28T14:12:50.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:12:50.809 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:12:51.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.083+0000 7f32afe00640 1 -- 172.21.15.42:0/2143195661 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7f32b00f2af0 con 0x7f32b815b350 2024-05-28T14:12:51.069 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-05-28T14:12:51.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.087+0000 7f32be800640 1 -- 172.21.15.42:0/2143195661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3294069b20 msgr2=0x7f329406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:51.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.087+0000 7f32be800640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3294069b20 0x7f329406bfe0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f32a80097a0 tx=0x7f32a8008040 comp rx=0 tx=0).stop 2024-05-28T14:12:51.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.087+0000 7f32be800640 1 -- 172.21.15.42:0/2143195661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b815b350 msgr2=0x7f32b8154560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:51.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.087+0000 7f32be800640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b815b350 0x7f32b8154560 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f32b001af90 tx=0x7f32b0019680 comp rx=0 tx=0).stop 2024-05-28T14:12:51.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.087+0000 7f32be800640 1 -- 172.21.15.42:0/2143195661 shutdown_connections 2024-05-28T14:12:51.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.087+0000 7f32be800640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3294069b20 0x7f329406bfe0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.087+0000 7f32be800640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b815b350 0x7f32b8154560 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.087+0000 7f32be800640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32b80d0e00 0x7f32b8099970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.087+0000 7f32be800640 1 --2- 172.21.15.42:0/2143195661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b8072d40 0x7f32b814e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.087+0000 7f32be800640 1 -- 172.21.15.42:0/2143195661 >> 172.21.15.42:0/2143195661 conn(0x7f32b806bdf0 msgr2=0x7f32b8155180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:51.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.087+0000 7f32be800640 1 -- 172.21.15.42:0/2143195661 shutdown_connections 2024-05-28T14:12:51.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.091+0000 7f32be800640 1 -- 172.21.15.42:0/2143195661 wait complete. 2024-05-28T14:12:51.099 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-28T14:12:51.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f9904a00640 1 Processor -- start 2024-05-28T14:12:51.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f9904a00640 1 -- start start 2024-05-28T14:12:51.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f9904a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f990006e4e0 0x7f99000704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:51.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f9904a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f99000b5c90 0x7f990006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:51.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f9904a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99000afaa0 0x7f99000aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:51.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f9904a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f99000af610 con 0x7f99000b5c90 2024-05-28T14:12:51.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f9904a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9900070a10 con 0x7f990006e4e0 2024-05-28T14:12:51.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f9904a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f990006ee00 con 0x7f99000afaa0 2024-05-28T14:12:51.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f98f7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f99000b5c90 0x7f990006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:51.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f98f7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f99000b5c90 0x7f990006e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46652/0 (socket says 172.21.15.42:46652) 2024-05-28T14:12:51.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f98f7e00640 1 -- 172.21.15.42:0/3357162680 learned_addr learned my addr 172.21.15.42:0/3357162680 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:51.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f98ff400640 1 --2- 172.21.15.42:0/3357162680 >> v2:172.21.15.63:3300/0 conn(0x7f990006e4e0 0x7f99000704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:51.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f98ffe00640 1 --2- 172.21.15.42:0/3357162680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99000afaa0 0x7f99000aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:51.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f98f7e00640 1 -- 172.21.15.42:0/3357162680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99000afaa0 msgr2=0x7f99000aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:51.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f98f7e00640 1 --2- 172.21.15.42:0/3357162680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99000afaa0 0x7f99000aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f98f7e00640 1 -- 172.21.15.42:0/3357162680 >> v2:172.21.15.63:3300/0 conn(0x7f990006e4e0 msgr2=0x7f99000704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:51.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f98f7e00640 1 --2- 172.21.15.42:0/3357162680 >> v2:172.21.15.63:3300/0 conn(0x7f990006e4e0 0x7f99000704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.259+0000 7f98f7e00640 1 -- 172.21.15.42:0/3357162680 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99000aece0 con 0x7f99000b5c90 2024-05-28T14:12:51.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.263+0000 7f98f7e00640 1 --2- 172.21.15.42:0/3357162680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99000b5c90 0x7f990006e8c0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f9900072080 tx=0x7f98e801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a271e4773cd7a337 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:51.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.263+0000 7f98fea00640 1 -- 172.21.15.42:0/3357162680 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e80ab620 con 0x7f99000b5c90 2024-05-28T14:12:51.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.263+0000 7f98fea00640 1 -- 172.21.15.42:0/3357162680 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e80b0070 con 0x7f99000b5c90 2024-05-28T14:12:51.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.263+0000 7f98fea00640 1 -- 172.21.15.42:0/3357162680 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e80ab9e0 con 0x7f99000b5c90 2024-05-28T14:12:51.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.263+0000 7f9904a00640 1 -- 172.21.15.42:0/3357162680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99000b5c90 msgr2=0x7f990006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:51.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.263+0000 7f9904a00640 1 --2- 172.21.15.42:0/3357162680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99000b5c90 0x7f990006e8c0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f9900072080 tx=0x7f98e801c460 comp rx=0 tx=0).stop 2024-05-28T14:12:51.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.263+0000 7f9904a00640 1 -- 172.21.15.42:0/3357162680 shutdown_connections 2024-05-28T14:12:51.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.263+0000 7f9904a00640 1 --2- 172.21.15.42:0/3357162680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99000afaa0 0x7f99000aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.263+0000 7f9904a00640 1 --2- 172.21.15.42:0/3357162680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99000b5c90 0x7f990006e8c0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.263+0000 7f9904a00640 1 --2- 172.21.15.42:0/3357162680 >> v2:172.21.15.63:3300/0 conn(0x7f990006e4e0 0x7f99000704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.263+0000 7f9904a00640 1 -- 172.21.15.42:0/3357162680 >> 172.21.15.42:0/3357162680 conn(0x7f9900073230 msgr2=0x7f990006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:51.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.263+0000 7f9904a00640 1 -- 172.21.15.42:0/3357162680 shutdown_connections 2024-05-28T14:12:51.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.263+0000 7f9904a00640 1 -- 172.21.15.42:0/3357162680 wait complete. 2024-05-28T14:12:51.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f9904a00640 1 Processor -- start 2024-05-28T14:12:51.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f9904a00640 1 -- start start 2024-05-28T14:12:51.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f9904a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f990006e4e0 0x7f990014eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:51.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f9904a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99000afaa0 0x7f99000cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:51.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f9904a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99000b5c90 0x7f990014e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f9904a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f990015e970 con 0x7f99000b5c90 2024-05-28T14:12:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f9904a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f990015e7f0 con 0x7f990006e4e0 2024-05-28T14:12:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f9904a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f99001620f0 con 0x7f99000afaa0 2024-05-28T14:12:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f98ffe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99000b5c90 0x7f990014e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f98ffe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99000b5c90 0x7f990014e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46662/0 (socket says 172.21.15.42:46662) 2024-05-28T14:12:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f98ffe00640 1 -- 172.21.15.42:0/2514451431 learned_addr learned my addr 172.21.15.42:0/2514451431 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f98ff400640 1 --2- 172.21.15.42:0/2514451431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f990006e4e0 0x7f990014eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f98f7e00640 1 --2- 172.21.15.42:0/2514451431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99000afaa0 0x7f99000cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f98ffe00640 1 -- 172.21.15.42:0/2514451431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99000afaa0 msgr2=0x7f99000cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f98ffe00640 1 --2- 172.21.15.42:0/2514451431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99000afaa0 0x7f99000cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f98ffe00640 1 -- 172.21.15.42:0/2514451431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f990006e4e0 msgr2=0x7f990014eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:51.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f98ffe00640 1 --2- 172.21.15.42:0/2514451431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f990006e4e0 0x7f990014eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f98ffe00640 1 -- 172.21.15.42:0/2514451431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99000b2020 con 0x7f99000b5c90 2024-05-28T14:12:51.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f98ffe00640 1 --2- 172.21.15.42:0/2514451431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99000b5c90 0x7f990014e2a0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f98f8007e40 tx=0x7f98f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:51.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f98f7400640 1 -- 172.21.15.42:0/2514451431 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8014620 con 0x7f99000b5c90 2024-05-28T14:12:51.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f98f7400640 1 -- 172.21.15.42:0/2514451431 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f8019070 con 0x7f99000b5c90 2024-05-28T14:12:51.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f98f7400640 1 -- 172.21.15.42:0/2514451431 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8014a20 con 0x7f99000b5c90 2024-05-28T14:12:51.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f9904a00640 1 -- 172.21.15.42:0/2514451431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9900153420 con 0x7f99000b5c90 2024-05-28T14:12:51.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.267+0000 7f9904a00640 1 -- 172.21.15.42:0/2514451431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99000b1d40 con 0x7f99000b5c90 2024-05-28T14:12:51.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.271+0000 7f9904a00640 1 -- 172.21.15.42:0/2514451431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990004fcf0 con 0x7f99000b5c90 2024-05-28T14:12:51.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.271+0000 7f98f7400640 1 -- 172.21.15.42:0/2514451431 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f98f8014bc0 con 0x7f99000b5c90 2024-05-28T14:12:51.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.271+0000 7f98f7400640 1 --2- 172.21.15.42:0/2514451431 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98dc069b20 0x7f98dc06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:51.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.271+0000 7f98f7400640 1 -- 172.21.15.42:0/2514451431 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4999+0+0 (secure 0 0 0) 0x7f98f8085f20 con 0x7f99000b5c90 2024-05-28T14:12:51.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.271+0000 7f98ff400640 1 --2- 172.21.15.42:0/2514451431 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98dc069b20 0x7f98dc06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:51.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.271+0000 7f98ff400640 1 --2- 172.21.15.42:0/2514451431 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98dc069b20 0x7f98dc06bfe0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f990014e250 tx=0x7f98f0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:51.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.275+0000 7f98f7400640 1 -- 172.21.15.42:0/2514451431 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f98f80578f0 con 0x7f99000b5c90 2024-05-28T14:12:51.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.443+0000 7f9904a00640 1 -- 172.21.15.42:0/2514451431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f99000b04d0 con 0x7f99000b5c90 2024-05-28T14:12:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.447+0000 7f98f7400640 1 -- 172.21.15.42:0/2514451431 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f98f805bbf0 con 0x7f99000b5c90 2024-05-28T14:12:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.451+0000 7f9904a00640 1 -- 172.21.15.42:0/2514451431 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98dc069b20 msgr2=0x7f98dc06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.451+0000 7f9904a00640 1 --2- 172.21.15.42:0/2514451431 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98dc069b20 0x7f98dc06bfe0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f990014e250 tx=0x7f98f0008040 comp rx=0 tx=0).stop 2024-05-28T14:12:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.451+0000 7f9904a00640 1 -- 172.21.15.42:0/2514451431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99000b5c90 msgr2=0x7f990014e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.451+0000 7f9904a00640 1 --2- 172.21.15.42:0/2514451431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99000b5c90 0x7f990014e2a0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f98f8007e40 tx=0x7f98f800e4b0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.451+0000 7f9904a00640 1 -- 172.21.15.42:0/2514451431 shutdown_connections 2024-05-28T14:12:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.451+0000 7f9904a00640 1 --2- 172.21.15.42:0/2514451431 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98dc069b20 0x7f98dc06bfe0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.451+0000 7f9904a00640 1 --2- 172.21.15.42:0/2514451431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99000b5c90 0x7f990014e2a0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.451+0000 7f9904a00640 1 --2- 172.21.15.42:0/2514451431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99000afaa0 0x7f99000cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.451+0000 7f9904a00640 1 --2- 172.21.15.42:0/2514451431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f990006e4e0 0x7f990014eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.451+0000 7f9904a00640 1 -- 172.21.15.42:0/2514451431 >> 172.21.15.42:0/2514451431 conn(0x7f9900073230 msgr2=0x7f99000b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:51.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.451+0000 7f9904a00640 1 -- 172.21.15.42:0/2514451431 shutdown_connections 2024-05-28T14:12:51.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.451+0000 7f9904a00640 1 -- 172.21.15.42:0/2514451431 wait complete. 2024-05-28T14:12:51.456 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled' 2024-05-28T14:12:51.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.611+0000 7f17a4400640 1 Processor -- start 2024-05-28T14:12:51.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.611+0000 7f17a4400640 1 -- start start 2024-05-28T14:12:51.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f17a4400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f179c06f660 0x7f179c073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:51.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f17a4400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c15ba40 0x7f179c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:51.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f17a4400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f179c15f280 con 0x7f179c072dd0 2024-05-28T14:12:51.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f17a4400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f179c15f030 con 0x7f179c06f660 2024-05-28T14:12:51.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f17a4400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f179c15f560 con 0x7f179c15ba40 2024-05-28T14:12:51.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f17a2400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f179c072dd0 0x7f179c0b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:48618/0 (socket says 172.21.15.42:48618) 2024-05-28T14:12:51.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f17a2400640 1 -- 172.21.15.42:0/3807150405 learned_addr learned my addr 172.21.15.42:0/3807150405 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:51.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f17a1a00640 1 --2- 172.21.15.42:0/3807150405 >> v2:172.21.15.42:3300/0 conn(0x7f179c06f660 0x7f179c073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:51.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f17a1000640 1 --2- 172.21.15.42:0/3807150405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c15ba40 0x7f179c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:51.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f179be00640 1 -- 172.21.15.42:0/3807150405 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2653792241 0 0) 0x7f179c15f280 con 0x7f179c072dd0 2024-05-28T14:12:51.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f179be00640 1 -- 172.21.15.42:0/3807150405 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1784005740 con 0x7f179c072dd0 2024-05-28T14:12:51.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f17a1a00640 1 -- 172.21.15.42:0/3807150405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c15ba40 msgr2=0x7f179c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:51.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f17a1a00640 1 --2- 172.21.15.42:0/3807150405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c15ba40 0x7f179c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f17a1a00640 1 -- 172.21.15.42:0/3807150405 >> v1:172.21.15.42:6789/0 conn(0x7f179c072dd0 legacy=0x7f179c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:51.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f17a1a00640 1 -- 172.21.15.42:0/3807150405 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c15f840 con 0x7f179c06f660 2024-05-28T14:12:51.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.615+0000 7f17a1a00640 1 --2- 172.21.15.42:0/3807150405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f179c06f660 0x7f179c073980 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f17900097c0 tx=0x7f179001c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f703f3486ca63bf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:51.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f179be00640 1 -- 172.21.15.42:0/3807150405 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17900b0070 con 0x7f179c06f660 2024-05-28T14:12:51.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f179be00640 1 -- 172.21.15.42:0/3807150405 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17900ab600 con 0x7f179c06f660 2024-05-28T14:12:51.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f179be00640 1 -- 172.21.15.42:0/3807150405 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17900ab9d0 con 0x7f179c06f660 2024-05-28T14:12:51.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 -- 172.21.15.42:0/3807150405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f179c06f660 msgr2=0x7f179c073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:51.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 --2- 172.21.15.42:0/3807150405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f179c06f660 0x7f179c073980 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f17900097c0 tx=0x7f179001c350 comp rx=0 tx=0).stop 2024-05-28T14:12:51.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 -- 172.21.15.42:0/3807150405 shutdown_connections 2024-05-28T14:12:51.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 --2- 172.21.15.42:0/3807150405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c15ba40 0x7f179c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 --2- 172.21.15.42:0/3807150405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f179c06f660 0x7f179c073980 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 -- 172.21.15.42:0/3807150405 >> 172.21.15.42:0/3807150405 conn(0x7f179c06bdf0 msgr2=0x7f179c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:51.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 -- 172.21.15.42:0/3807150405 shutdown_connections 2024-05-28T14:12:51.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 -- 172.21.15.42:0/3807150405 wait complete. 2024-05-28T14:12:51.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 Processor -- start 2024-05-28T14:12:51.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 -- start start 2024-05-28T14:12:51.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f179c06f660 0x7f179c099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:51.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c072dd0 0x7f179c147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:51.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c142530 0x7f179c1437e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:51.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f179c0b3460 con 0x7f179c06f660 2024-05-28T14:12:51.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f179c0b32e0 con 0x7f179c072dd0 2024-05-28T14:12:51.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a4400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f179c0b35e0 con 0x7f179c142530 2024-05-28T14:12:51.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a1a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f179c06f660 0x7f179c099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:51.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a1a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f179c06f660 0x7f179c099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46674/0 (socket says 172.21.15.42:46674) 2024-05-28T14:12:51.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a1a00640 1 -- 172.21.15.42:0/2942339650 learned_addr learned my addr 172.21.15.42:0/2942339650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:51.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a1000640 1 --2- 172.21.15.42:0/2942339650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c072dd0 0x7f179c147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:51.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a2400640 1 --2- 172.21.15.42:0/2942339650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c142530 0x7f179c1437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:51.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a1a00640 1 -- 172.21.15.42:0/2942339650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c142530 msgr2=0x7f179c1437e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:51.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a1a00640 1 --2- 172.21.15.42:0/2942339650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c142530 0x7f179c1437e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a1a00640 1 -- 172.21.15.42:0/2942339650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c072dd0 msgr2=0x7f179c147390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:51.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a1a00640 1 --2- 172.21.15.42:0/2942339650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c072dd0 0x7f179c147390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:51.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.619+0000 7f17a1a00640 1 -- 172.21.15.42:0/2942339650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c147cc0 con 0x7f179c06f660 2024-05-28T14:12:51.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.623+0000 7f17a1a00640 1 --2- 172.21.15.42:0/2942339650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f179c06f660 0x7f179c099a00 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f17900ae040 tx=0x7f179001ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:51.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.623+0000 7f1799600640 1 -- 172.21.15.42:0/2942339650 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17900b0050 con 0x7f179c06f660 2024-05-28T14:12:51.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.623+0000 7f1799600640 1 -- 172.21.15.42:0/2942339650 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17900bc070 con 0x7f179c06f660 2024-05-28T14:12:51.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.623+0000 7f1799600640 1 -- 172.21.15.42:0/2942339650 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17900b5830 con 0x7f179c06f660 2024-05-28T14:12:51.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.623+0000 7f17a4400640 1 -- 172.21.15.42:0/2942339650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c147fa0 con 0x7f179c06f660 2024-05-28T14:12:51.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.623+0000 7f17a4400640 1 -- 172.21.15.42:0/2942339650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c1478d0 con 0x7f179c06f660 2024-05-28T14:12:51.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.623+0000 7f17a4400640 1 -- 172.21.15.42:0/2942339650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179c068a90 con 0x7f179c06f660 2024-05-28T14:12:51.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.623+0000 7f1799600640 1 -- 172.21.15.42:0/2942339650 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f17900b59d0 con 0x7f179c06f660 2024-05-28T14:12:51.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.623+0000 7f1799600640 1 --2- 172.21.15.42:0/2942339650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f177c069b50 0x7f177c06c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:51.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.623+0000 7f1799600640 1 -- 172.21.15.42:0/2942339650 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4999+0+0 (secure 0 0 0) 0x7f179011ffa0 con 0x7f179c06f660 2024-05-28T14:12:51.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.623+0000 7f17a1000640 1 --2- 172.21.15.42:0/2942339650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f177c069b50 0x7f177c06c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:51.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.627+0000 7f17a1000640 1 --2- 172.21.15.42:0/2942339650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f177c069b50 0x7f177c06c010 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f179c0d0c00 tx=0x7f1788008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:51.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.627+0000 7f1799600640 1 -- 172.21.15.42:0/2942339650 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1790016090 con 0x7f179c06f660 2024-05-28T14:12:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:51.863+0000 7f17a4400640 1 -- 172.21.15.42:0/2942339650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled"]} v 0) -- 0x7f179c15d3d0 con 0x7f179c06f660 2024-05-28T14:12:52.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.091+0000 7f1799600640 1 -- 172.21.15.42:0/2942339650 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled"]}]=0 v662) ==== 161+0+0 (secure 0 0 0) 0x7f17900f1fc0 con 0x7f179c06f660 2024-05-28T14:12:52.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.091+0000 7f17a4400640 1 -- 172.21.15.42:0/2942339650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f177c069b50 msgr2=0x7f177c06c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.091+0000 7f17a4400640 1 --2- 172.21.15.42:0/2942339650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f177c069b50 0x7f177c06c010 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f179c0d0c00 tx=0x7f1788008040 comp rx=0 tx=0).stop 2024-05-28T14:12:52.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.091+0000 7f17a4400640 1 -- 172.21.15.42:0/2942339650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f179c06f660 msgr2=0x7f179c099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.091+0000 7f17a4400640 1 --2- 172.21.15.42:0/2942339650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f179c06f660 0x7f179c099a00 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f17900ae040 tx=0x7f179001ab90 comp rx=0 tx=0).stop 2024-05-28T14:12:52.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.091+0000 7f17a4400640 1 -- 172.21.15.42:0/2942339650 shutdown_connections 2024-05-28T14:12:52.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.091+0000 7f17a4400640 1 --2- 172.21.15.42:0/2942339650 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f177c069b50 0x7f177c06c010 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.091+0000 7f17a4400640 1 --2- 172.21.15.42:0/2942339650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c142530 0x7f179c1437e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.091+0000 7f17a4400640 1 --2- 172.21.15.42:0/2942339650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c072dd0 0x7f179c147390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.091+0000 7f17a4400640 1 --2- 172.21.15.42:0/2942339650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f179c06f660 0x7f179c099a00 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.091+0000 7f17a4400640 1 -- 172.21.15.42:0/2942339650 >> 172.21.15.42:0/2942339650 conn(0x7f179c06bdf0 msgr2=0x7f179c06dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:52.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.091+0000 7f17a4400640 1 -- 172.21.15.42:0/2942339650 shutdown_connections 2024-05-28T14:12:52.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.091+0000 7f17a4400640 1 -- 172.21.15.42:0/2942339650 wait complete. 2024-05-28T14:12:52.098 INFO:tasks.cephfs_test_runner:Test that the progress bar, that is printed for 1 ongoing clone job, is ... ok 2024-05-28T14:12:52.098 INFO:tasks.cephfs_test_runner:Starting test: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress bars, that printed for 3 ongoing clone jobs, are 2024-05-28T14:12:52.098 INFO:tasks.cephfs_test_runner:test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-28T14:12:52.099 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:12:52.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.255+0000 7f0425000640 1 Processor -- start 2024-05-28T14:12:52.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.255+0000 7f0425000640 1 -- start start 2024-05-28T14:12:52.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.255+0000 7f0425000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f042015eab0 0x7f04200d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.255+0000 7f0425000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f042006cda0 0x7f042009b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.255+0000 7f0425000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04200d1aa0 con 0x7f042006d480 2024-05-28T14:12:52.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.255+0000 7f0425000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f042015ee90 con 0x7f042006cda0 2024-05-28T14:12:52.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.255+0000 7f0425000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f042009b1b0 con 0x7f042015eab0 2024-05-28T14:12:52.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.255+0000 7f0415400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f042006cda0 0x7f042009b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.255+0000 7f041e800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f042006d480 0x7f04200ccab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:48628/0 (socket says 172.21.15.42:48628) 2024-05-28T14:12:52.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.255+0000 7f041e800640 1 -- 172.21.15.42:0/2656447673 learned_addr learned my addr 172.21.15.42:0/2656447673 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:52.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f041de00640 1 --2- 172.21.15.42:0/2656447673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f042015eab0 0x7f04200d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f0415400640 1 -- 172.21.15.42:0/2656447673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f042015eab0 msgr2=0x7f04200d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:52.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f0415400640 1 --2- 172.21.15.42:0/2656447673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f042015eab0 0x7f04200d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f0415400640 1 -- 172.21.15.42:0/2656447673 >> v1:172.21.15.42:6789/0 conn(0x7f042006d480 legacy=0x7f04200ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f0415400640 1 -- 172.21.15.42:0/2656447673 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04200d17e0 con 0x7f042006cda0 2024-05-28T14:12:52.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f041d400640 1 -- 172.21.15.42:0/2656447673 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1393890342 0 0) 0x7f04200d1aa0 con 0x7f042006d480 2024-05-28T14:12:52.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f0415400640 1 --2- 172.21.15.42:0/2656447673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042006cda0 0x7f042009b640 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f04080097c0 tx=0x7f040801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88cd0bb8787a6e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:52.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f041d400640 1 -- 172.21.15.42:0/2656447673 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04080b1070 con 0x7f042006cda0 2024-05-28T14:12:52.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f041d400640 1 -- 172.21.15.42:0/2656447673 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04080a8050 con 0x7f042006cda0 2024-05-28T14:12:52.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f041d400640 1 -- 172.21.15.42:0/2656447673 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04080ac860 con 0x7f042006cda0 2024-05-28T14:12:52.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f0425000640 1 -- 172.21.15.42:0/2656447673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042006cda0 msgr2=0x7f042009b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f0425000640 1 --2- 172.21.15.42:0/2656447673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042006cda0 0x7f042009b640 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f04080097c0 tx=0x7f040801c3e0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f0425000640 1 -- 172.21.15.42:0/2656447673 shutdown_connections 2024-05-28T14:12:52.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f0425000640 1 --2- 172.21.15.42:0/2656447673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042006cda0 0x7f042009b640 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f0425000640 1 --2- 172.21.15.42:0/2656447673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f042015eab0 0x7f04200d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.259+0000 7f0425000640 1 -- 172.21.15.42:0/2656447673 >> 172.21.15.42:0/2656447673 conn(0x7f04200b3f50 msgr2=0x7f04200b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:52.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f0425000640 1 -- 172.21.15.42:0/2656447673 shutdown_connections 2024-05-28T14:12:52.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f0425000640 1 -- 172.21.15.42:0/2656447673 wait complete. 2024-05-28T14:12:52.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f0425000640 1 Processor -- start 2024-05-28T14:12:52.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f0425000640 1 -- start start 2024-05-28T14:12:52.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f0425000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042006cda0 0x7f04201481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f0425000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f042006d480 0x7f04200cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f0425000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f042015eab0 0x7f042014a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f0425000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f042009bb80 con 0x7f042006cda0 2024-05-28T14:12:52.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f0425000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f042006dcd0 con 0x7f042015eab0 2024-05-28T14:12:52.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f0425000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f042009bd00 con 0x7f042006d480 2024-05-28T14:12:52.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f041de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042006cda0 0x7f04201481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f041de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042006cda0 0x7f04201481d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46688/0 (socket says 172.21.15.42:46688) 2024-05-28T14:12:52.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f041de00640 1 -- 172.21.15.42:0/1255408852 learned_addr learned my addr 172.21.15.42:0/1255408852 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:52.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f0415400640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f042006d480 0x7f04200cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f041e800640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f042015eab0 0x7f042014a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f041de00640 1 -- 172.21.15.42:0/1255408852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f042006d480 msgr2=0x7f04200cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f041de00640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f042006d480 0x7f04200cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f041de00640 1 -- 172.21.15.42:0/1255408852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f042015eab0 msgr2=0x7f042014a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f041de00640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f042015eab0 0x7f042014a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f041de00640 1 -- 172.21.15.42:0/1255408852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04200b7600 con 0x7f042006cda0 2024-05-28T14:12:52.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f0415400640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f042006d480 0x7f04200cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:52.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f041de00640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042006cda0 0x7f04201481d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f0410007ef0 tx=0x7f041000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:52.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.263+0000 7f0416a00640 1 -- 172.21.15.42:0/1255408852 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410014620 con 0x7f042006cda0 2024-05-28T14:12:52.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.267+0000 7f0416a00640 1 -- 172.21.15.42:0/1255408852 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0410019070 con 0x7f042006cda0 2024-05-28T14:12:52.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.267+0000 7f0416a00640 1 -- 172.21.15.42:0/1255408852 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410014a00 con 0x7f042006cda0 2024-05-28T14:12:52.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.267+0000 7f0425000640 1 -- 172.21.15.42:0/1255408852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04200b78e0 con 0x7f042006cda0 2024-05-28T14:12:52.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.267+0000 7f0425000640 1 -- 172.21.15.42:0/1255408852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04200b7ea0 con 0x7f042006cda0 2024-05-28T14:12:52.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.267+0000 7f0425000640 1 -- 172.21.15.42:0/1255408852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0420068a90 con 0x7f042006cda0 2024-05-28T14:12:52.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.267+0000 7f0416a00640 1 -- 172.21.15.42:0/1255408852 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f0410014ba0 con 0x7f042006cda0 2024-05-28T14:12:52.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.267+0000 7f0416a00640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f03fc069b20 0x7f03fc06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.267+0000 7f0416a00640 1 -- 172.21.15.42:0/1255408852 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4999+0+0 (secure 0 0 0) 0x7f0410086130 con 0x7f042006cda0 2024-05-28T14:12:52.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.267+0000 7f0415400640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f03fc069b20 0x7f03fc06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.267+0000 7f0415400640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f03fc069b20 0x7f03fc06bfe0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f04080ae000 tx=0x7f0408008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:52.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.271+0000 7f0416a00640 1 -- 172.21.15.42:0/1255408852 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0410058150 con 0x7f042006cda0 2024-05-28T14:12:52.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.439+0000 7f0425000640 1 -- 172.21.15.42:0/1255408852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04200691a0 con 0x7f042006cda0 2024-05-28T14:12:52.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.439+0000 7f0416a00640 1 -- 172.21.15.42:0/1255408852 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+13058 (secure 0 0 0) 0x7f0410057a50 con 0x7f042006cda0 2024-05-28T14:12:52.423 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:52.424 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":109,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:12:51.086666+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6835/1184583947":"2024-05-29T14:12:48.074199+0000","172.21.15.63:6833/1184583947":"2024-05-29T14:12:48.074199+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:12:52.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.439+0000 7f0425000640 1 -- 172.21.15.42:0/1255408852 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f03fc069b20 msgr2=0x7f03fc06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.439+0000 7f0425000640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f03fc069b20 0x7f03fc06bfe0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f04080ae000 tx=0x7f0408008040 comp rx=0 tx=0).stop 2024-05-28T14:12:52.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.439+0000 7f0425000640 1 -- 172.21.15.42:0/1255408852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042006cda0 msgr2=0x7f04201481d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.439+0000 7f0425000640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042006cda0 0x7f04201481d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f0410007ef0 tx=0x7f041000e4b0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.439+0000 7f0425000640 1 -- 172.21.15.42:0/1255408852 shutdown_connections 2024-05-28T14:12:52.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.439+0000 7f0425000640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f03fc069b20 0x7f03fc06bfe0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.439+0000 7f0425000640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f042015eab0 0x7f042014a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.439+0000 7f0425000640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f042006d480 0x7f04200cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.439+0000 7f0425000640 1 --2- 172.21.15.42:0/1255408852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042006cda0 0x7f04201481d0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.439+0000 7f0425000640 1 -- 172.21.15.42:0/1255408852 >> 172.21.15.42:0/1255408852 conn(0x7f04200b3f50 msgr2=0x7f04200ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:52.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.439+0000 7f0425000640 1 -- 172.21.15.42:0/1255408852 shutdown_connections 2024-05-28T14:12:52.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.439+0000 7f0425000640 1 -- 172.21.15.42:0/1255408852 wait complete. 2024-05-28T14:12:52.445 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:12:52.445 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:12:52.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.603+0000 7f31c0e00640 1 Processor -- start 2024-05-28T14:12:52.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.603+0000 7f31c0e00640 1 -- start start 2024-05-28T14:12:52.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31c0e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31bc06ee20 0x7f31bc06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31c0e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f31bc15b9b0 0x7f31bc15be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31c0e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31bc0afe40 con 0x7f31bc144f80 2024-05-28T14:12:52.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31c0e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f31bc06f7c0 con 0x7f31bc15b9b0 2024-05-28T14:12:52.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31c0e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f31bc0724d0 con 0x7f31bc06ee20 2024-05-28T14:12:52.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31b3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31bc06ee20 0x7f31bc06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31ba800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f31bc15b9b0 0x7f31bc15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31b3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31bc06ee20 0x7f31bc06f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:41840/0 (socket says 172.21.15.42:41840) 2024-05-28T14:12:52.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31b3e00640 1 -- 172.21.15.42:0/3593829682 learned_addr learned my addr 172.21.15.42:0/3593829682 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:52.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31ba800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f31bc15b9b0 0x7f31bc15be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:48766/0 (socket says 172.21.15.42:48766) 2024-05-28T14:12:52.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31b9e00640 1 --1- 172.21.15.42:0/3593829682 >> v1:172.21.15.63:6789/0 conn(0x7f31bc144f80 0x7f31bc072960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:51964/0 (socket says 172.21.15.42:51964) 2024-05-28T14:12:52.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31b3e00640 1 -- 172.21.15.42:0/3593829682 >> v2:172.21.15.63:3300/0 conn(0x7f31bc15b9b0 msgr2=0x7f31bc15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:52.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31b3e00640 1 --2- 172.21.15.42:0/3593829682 >> v2:172.21.15.63:3300/0 conn(0x7f31bc15b9b0 0x7f31bc15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31b3e00640 1 -- 172.21.15.42:0/3593829682 >> v1:172.21.15.63:6789/0 conn(0x7f31bc144f80 legacy=0x7f31bc072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.607+0000 7f31b3e00640 1 -- 172.21.15.42:0/3593829682 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31bc15c380 con 0x7f31bc06ee20 2024-05-28T14:12:52.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.611+0000 7f31b3e00640 1 --2- 172.21.15.42:0/3593829682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31bc06ee20 0x7f31bc06f280 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009870 tx=0x7f31a401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6caa90aeb193d0d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:52.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.611+0000 7f31b9400640 1 -- 172.21.15.42:0/3593829682 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a40b2070 con 0x7f31bc06ee20 2024-05-28T14:12:52.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.611+0000 7f31b9400640 1 -- 172.21.15.42:0/3593829682 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a40a9070 con 0x7f31bc06ee20 2024-05-28T14:12:52.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.611+0000 7f31b9400640 1 -- 172.21.15.42:0/3593829682 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a40ad880 con 0x7f31bc06ee20 2024-05-28T14:12:52.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.611+0000 7f31c0e00640 1 -- 172.21.15.42:0/3593829682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31bc06ee20 msgr2=0x7f31bc06f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.611+0000 7f31c0e00640 1 --2- 172.21.15.42:0/3593829682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31bc06ee20 0x7f31bc06f280 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009870 tx=0x7f31a401c560 comp rx=0 tx=0).stop 2024-05-28T14:12:52.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.611+0000 7f31c0e00640 1 -- 172.21.15.42:0/3593829682 shutdown_connections 2024-05-28T14:12:52.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.611+0000 7f31c0e00640 1 --2- 172.21.15.42:0/3593829682 >> v2:172.21.15.63:3300/0 conn(0x7f31bc15b9b0 0x7f31bc15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.611+0000 7f31c0e00640 1 --2- 172.21.15.42:0/3593829682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31bc06ee20 0x7f31bc06f280 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.611+0000 7f31c0e00640 1 -- 172.21.15.42:0/3593829682 >> 172.21.15.42:0/3593829682 conn(0x7f31bc0b21f0 msgr2=0x7f31bc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:52.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.611+0000 7f31c0e00640 1 -- 172.21.15.42:0/3593829682 shutdown_connections 2024-05-28T14:12:52.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.611+0000 7f31c0e00640 1 -- 172.21.15.42:0/3593829682 wait complete. 2024-05-28T14:12:52.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.611+0000 7f31c0e00640 1 Processor -- start 2024-05-28T14:12:52.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.611+0000 7f31c0e00640 1 -- start start 2024-05-28T14:12:52.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31c0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31bc06ee20 0x7f31bc15f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31c0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31bc144f80 0x7f31bc0b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31c0e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31bc15b9b0 0x7f31bc0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31c0e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc0b4010 con 0x7f31bc06ee20 2024-05-28T14:12:52.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31c0e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc072160 con 0x7f31bc144f80 2024-05-28T14:12:52.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31c0e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f31bc0b4190 con 0x7f31bc15b9b0 2024-05-28T14:12:52.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31b9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31bc06ee20 0x7f31bc15f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31b9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31bc06ee20 0x7f31bc15f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46692/0 (socket says 172.21.15.42:46692) 2024-05-28T14:12:52.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31b9e00640 1 -- 172.21.15.42:0/532687343 learned_addr learned my addr 172.21.15.42:0/532687343 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:52.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31ba800640 1 --2- 172.21.15.42:0/532687343 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31bc15b9b0 0x7f31bc0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31b3e00640 1 --2- 172.21.15.42:0/532687343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31bc144f80 0x7f31bc0b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31b9e00640 1 -- 172.21.15.42:0/532687343 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31bc15b9b0 msgr2=0x7f31bc0b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31b9e00640 1 --2- 172.21.15.42:0/532687343 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31bc15b9b0 0x7f31bc0b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31b9e00640 1 -- 172.21.15.42:0/532687343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31bc144f80 msgr2=0x7f31bc0b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31b9e00640 1 --2- 172.21.15.42:0/532687343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31bc144f80 0x7f31bc0b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31b9e00640 1 -- 172.21.15.42:0/532687343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31bc0a9db0 con 0x7f31bc06ee20 2024-05-28T14:12:52.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31b9e00640 1 --2- 172.21.15.42:0/532687343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31bc06ee20 0x7f31bc15f880 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00c450 tx=0x7f31ac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:52.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31b2000640 1 -- 172.21.15.42:0/532687343 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac018070 con 0x7f31bc06ee20 2024-05-28T14:12:52.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31b2000640 1 -- 172.21.15.42:0/532687343 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ac007480 con 0x7f31bc06ee20 2024-05-28T14:12:52.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31c0e00640 1 -- 172.21.15.42:0/532687343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31bc0aa090 con 0x7f31bc06ee20 2024-05-28T14:12:52.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31b2000640 1 -- 172.21.15.42:0/532687343 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac0047f0 con 0x7f31bc06ee20 2024-05-28T14:12:52.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31c0e00640 1 -- 172.21.15.42:0/532687343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31bc0aa650 con 0x7f31bc06ee20 2024-05-28T14:12:52.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.615+0000 7f31c0e00640 1 -- 172.21.15.42:0/532687343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31bc0738b0 con 0x7f31bc06ee20 2024-05-28T14:12:52.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.619+0000 7f31b2000640 1 -- 172.21.15.42:0/532687343 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f31ac01b070 con 0x7f31bc06ee20 2024-05-28T14:12:52.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.619+0000 7f31b2000640 1 --2- 172.21.15.42:0/532687343 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3198069b20 0x7f319806bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.619+0000 7f31b2000640 1 -- 172.21.15.42:0/532687343 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4999+0+0 (secure 0 0 0) 0x7f31ac084960 con 0x7f31bc06ee20 2024-05-28T14:12:52.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.619+0000 7f31b3e00640 1 --2- 172.21.15.42:0/532687343 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3198069b20 0x7f319806bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.619+0000 7f31b3e00640 1 --2- 172.21.15.42:0/532687343 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3198069b20 0x7f319806bfe0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f31a40af000 tx=0x7f31a40a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:52.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.623+0000 7f31b2000640 1 -- 172.21.15.42:0/532687343 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f31ac056330 con 0x7f31bc06ee20 2024-05-28T14:12:52.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.787+0000 7f31c0e00640 1 -- 172.21.15.42:0/532687343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f31bc04fdd0 con 0x7f31bc06ee20 2024-05-28T14:12:52.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.787+0000 7f31b2000640 1 -- 172.21.15.42:0/532687343 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7f31ac05a630 con 0x7f31bc06ee20 2024-05-28T14:12:52.774 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:12:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.791+0000 7f31c0e00640 1 -- 172.21.15.42:0/532687343 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3198069b20 msgr2=0x7f319806bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.791+0000 7f31c0e00640 1 --2- 172.21.15.42:0/532687343 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3198069b20 0x7f319806bfe0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f31a40af000 tx=0x7f31a40a7000 comp rx=0 tx=0).stop 2024-05-28T14:12:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.791+0000 7f31c0e00640 1 -- 172.21.15.42:0/532687343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31bc06ee20 msgr2=0x7f31bc15f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.791+0000 7f31c0e00640 1 --2- 172.21.15.42:0/532687343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31bc06ee20 0x7f31bc15f880 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00c450 tx=0x7f31ac00c910 comp rx=0 tx=0).stop 2024-05-28T14:12:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.791+0000 7f31c0e00640 1 -- 172.21.15.42:0/532687343 shutdown_connections 2024-05-28T14:12:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.791+0000 7f31c0e00640 1 --2- 172.21.15.42:0/532687343 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3198069b20 0x7f319806bfe0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.791+0000 7f31c0e00640 1 --2- 172.21.15.42:0/532687343 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31bc15b9b0 0x7f31bc0b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.791+0000 7f31c0e00640 1 --2- 172.21.15.42:0/532687343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31bc144f80 0x7f31bc0b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.791+0000 7f31c0e00640 1 --2- 172.21.15.42:0/532687343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31bc06ee20 0x7f31bc15f880 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.791+0000 7f31c0e00640 1 -- 172.21.15.42:0/532687343 >> 172.21.15.42:0/532687343 conn(0x7f31bc0b21f0 msgr2=0x7f31bc140dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.791+0000 7f31c0e00640 1 -- 172.21.15.42:0/532687343 shutdown_connections 2024-05-28T14:12:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.791+0000 7f31c0e00640 1 -- 172.21.15.42:0/532687343 wait complete. 2024-05-28T14:12:52.795 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled' 2024-05-28T14:12:52.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.951+0000 7f8368800640 1 Processor -- start 2024-05-28T14:12:52.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.951+0000 7f8368800640 1 -- start start 2024-05-28T14:12:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.951+0000 7f8368800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8360156a70 0x7f836006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.951+0000 7f8368800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f83600afa50 0x7f83600d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.951+0000 7f8368800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f836015cb90 con 0x7f8360156e50 2024-05-28T14:12:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.951+0000 7f8368800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f836015ce50 con 0x7f8360156a70 2024-05-28T14:12:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.951+0000 7f8368800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f836015d130 con 0x7f83600afa50 2024-05-28T14:12:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.951+0000 7f8365e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8360156a70 0x7f836006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.951+0000 7f8365e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8360156a70 0x7f836006dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46710/0 (socket says 172.21.15.42:46710) 2024-05-28T14:12:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.951+0000 7f8365e00640 1 -- 172.21.15.42:0/1165411049 learned_addr learned my addr 172.21.15.42:0/1165411049 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.951+0000 7f8366800640 1 --2- 172.21.15.42:0/1165411049 >> v2:172.21.15.63:3300/0 conn(0x7f83600afa50 0x7f83600d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8365e00640 1 -- 172.21.15.42:0/1165411049 >> v2:172.21.15.63:3300/0 conn(0x7f83600afa50 msgr2=0x7f83600d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8365e00640 1 --2- 172.21.15.42:0/1165411049 >> v2:172.21.15.63:3300/0 conn(0x7f83600afa50 0x7f83600d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8365e00640 1 -- 172.21.15.42:0/1165411049 >> v1:172.21.15.63:6789/0 conn(0x7f8360156e50 legacy=0x7f83600af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8365e00640 1 -- 172.21.15.42:0/1165411049 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836015d410 con 0x7f8360156a70 2024-05-28T14:12:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8366800640 1 --2- 172.21.15.42:0/1165411049 >> v2:172.21.15.63:3300/0 conn(0x7f83600afa50 0x7f83600d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:52.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8365e00640 1 --2- 172.21.15.42:0/1165411049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8360156a70 0x7f836006dd60 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f836006fea0 tx=0x7f835401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b06461d45e41e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:52.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8364a00640 1 -- 172.21.15.42:0/1165411049 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540ab430 con 0x7f8360156a70 2024-05-28T14:12:52.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8364a00640 1 -- 172.21.15.42:0/1165411049 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83540b0070 con 0x7f8360156a70 2024-05-28T14:12:52.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8364a00640 1 -- 172.21.15.42:0/1165411049 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540ab7f0 con 0x7f8360156a70 2024-05-28T14:12:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8368800640 1 -- 172.21.15.42:0/1165411049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8360156a70 msgr2=0x7f836006dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8368800640 1 --2- 172.21.15.42:0/1165411049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8360156a70 0x7f836006dd60 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f836006fea0 tx=0x7f835401c520 comp rx=0 tx=0).stop 2024-05-28T14:12:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8368800640 1 -- 172.21.15.42:0/1165411049 shutdown_connections 2024-05-28T14:12:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8368800640 1 --2- 172.21.15.42:0/1165411049 >> v2:172.21.15.63:3300/0 conn(0x7f83600afa50 0x7f83600d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8368800640 1 --2- 172.21.15.42:0/1165411049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8360156a70 0x7f836006dd60 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.955+0000 7f8368800640 1 -- 172.21.15.42:0/1165411049 >> 172.21.15.42:0/1165411049 conn(0x7f836006bdf0 msgr2=0x7f836006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:52.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8368800640 1 -- 172.21.15.42:0/1165411049 shutdown_connections 2024-05-28T14:12:52.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8368800640 1 -- 172.21.15.42:0/1165411049 wait complete. 2024-05-28T14:12:52.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8368800640 1 Processor -- start 2024-05-28T14:12:52.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8368800640 1 -- start start 2024-05-28T14:12:52.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8368800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83600afa50 0x7f8360148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8368800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8360156a70 0x7f8360155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8368800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8360156e50 0x7f8360155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8368800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8360160ed0 con 0x7f8360156e50 2024-05-28T14:12:52.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8368800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8360160d50 con 0x7f8360156a70 2024-05-28T14:12:52.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8368800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8360161050 con 0x7f83600afa50 2024-05-28T14:12:52.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8366800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8360156e50 0x7f8360155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8366800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8360156e50 0x7f8360155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46722/0 (socket says 172.21.15.42:46722) 2024-05-28T14:12:52.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8366800640 1 -- 172.21.15.42:0/3364262608 learned_addr learned my addr 172.21.15.42:0/3364262608 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:52.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8365e00640 1 --2- 172.21.15.42:0/3364262608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83600afa50 0x7f8360148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8365400640 1 --2- 172.21.15.42:0/3364262608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8360156a70 0x7f8360155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8366800640 1 -- 172.21.15.42:0/3364262608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83600afa50 msgr2=0x7f8360148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8366800640 1 --2- 172.21.15.42:0/3364262608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83600afa50 0x7f8360148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8366800640 1 -- 172.21.15.42:0/3364262608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8360156a70 msgr2=0x7f8360155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:52.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8366800640 1 --2- 172.21.15.42:0/3364262608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8360156a70 0x7f8360155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:52.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8366800640 1 -- 172.21.15.42:0/3364262608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83600cd650 con 0x7f8360156e50 2024-05-28T14:12:52.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8366800640 1 --2- 172.21.15.42:0/3364262608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8360156e50 0x7f8360155ed0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f835c007ef0 tx=0x7f835c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:52.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f835a000640 1 -- 172.21.15.42:0/3364262608 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c01a070 con 0x7f8360156e50 2024-05-28T14:12:52.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f835a000640 1 -- 172.21.15.42:0/3364262608 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f835c003ba0 con 0x7f8360156e50 2024-05-28T14:12:52.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f835a000640 1 -- 172.21.15.42:0/3364262608 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c004030 con 0x7f8360156e50 2024-05-28T14:12:52.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.959+0000 7f8368800640 1 -- 172.21.15.42:0/3364262608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8360072e90 con 0x7f8360156e50 2024-05-28T14:12:52.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.963+0000 7f8368800640 1 -- 172.21.15.42:0/3364262608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8360073450 con 0x7f8360156e50 2024-05-28T14:12:52.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.963+0000 7f8368800640 1 -- 172.21.15.42:0/3364262608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8360073650 con 0x7f8360156e50 2024-05-28T14:12:52.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.963+0000 7f835a000640 1 -- 172.21.15.42:0/3364262608 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f835c01d070 con 0x7f8360156e50 2024-05-28T14:12:52.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.963+0000 7f835a000640 1 --2- 172.21.15.42:0/3364262608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8344069b20 0x7f834406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:52.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.963+0000 7f835a000640 1 -- 172.21.15.42:0/3364262608 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4999+0+0 (secure 0 0 0) 0x7f835c085940 con 0x7f8360156e50 2024-05-28T14:12:52.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.963+0000 7f8365e00640 1 --2- 172.21.15.42:0/3364262608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8344069b20 0x7f834406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:52.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.963+0000 7f8365e00640 1 --2- 172.21.15.42:0/3364262608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8344069b20 0x7f834406bfe0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f83540af000 tx=0x7f83540a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:52.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:52.967+0000 7f835a000640 1 -- 172.21.15.42:0/3364262608 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8360073650 con 0x7f8360156e50 2024-05-28T14:12:53.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:53.191+0000 7f8368800640 1 -- 172.21.15.42:0/3364262608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled"]} v 0) -- 0x7f8360068a90 con 0x7f8360156e50 2024-05-28T14:12:54.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.099+0000 7f835a000640 1 -- 172.21.15.42:0/3364262608 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled"]}]=0 v664) ==== 166+0+0 (secure 0 0 0) 0x7f83600cd650 con 0x7f8360156e50 2024-05-28T14:12:54.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.103+0000 7f8368800640 1 -- 172.21.15.42:0/3364262608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8344069b20 msgr2=0x7f834406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:54.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.103+0000 7f8368800640 1 --2- 172.21.15.42:0/3364262608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8344069b20 0x7f834406bfe0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f83540af000 tx=0x7f83540a7000 comp rx=0 tx=0).stop 2024-05-28T14:12:54.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.103+0000 7f8368800640 1 -- 172.21.15.42:0/3364262608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8360156e50 msgr2=0x7f8360155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:54.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.103+0000 7f8368800640 1 --2- 172.21.15.42:0/3364262608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8360156e50 0x7f8360155ed0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f835c007ef0 tx=0x7f835c00d4a0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.103+0000 7f8368800640 1 -- 172.21.15.42:0/3364262608 shutdown_connections 2024-05-28T14:12:54.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.103+0000 7f8368800640 1 --2- 172.21.15.42:0/3364262608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8344069b20 0x7f834406bfe0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.103+0000 7f8368800640 1 --2- 172.21.15.42:0/3364262608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8360156e50 0x7f8360155ed0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.103+0000 7f8368800640 1 --2- 172.21.15.42:0/3364262608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8360156a70 0x7f8360155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.103+0000 7f8368800640 1 --2- 172.21.15.42:0/3364262608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83600afa50 0x7f8360148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.103+0000 7f8368800640 1 -- 172.21.15.42:0/3364262608 >> 172.21.15.42:0/3364262608 conn(0x7f836006bdf0 msgr2=0x7f836015a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:54.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.103+0000 7f8368800640 1 -- 172.21.15.42:0/3364262608 shutdown_connections 2024-05-28T14:12:54.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.103+0000 7f8368800640 1 -- 172.21.15.42:0/3364262608 wait complete. 2024-05-28T14:12:54.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T14:12:54.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6d3200640 1 Processor -- start 2024-05-28T14:12:54.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6d3200640 1 -- start start 2024-05-28T14:12:54.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6d3200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe6cc15e870 0x7fe6cc0d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:54.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6d3200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6cc15ec50 0x7fe6cc0b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:54.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6d3200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe6cc0b7ba0 0x7fe6cc0b7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:54.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6d3200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fe6cc0cd500 con 0x7fe6cc15e870 2024-05-28T14:12:54.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6d3200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe6cc0d1a80 con 0x7fe6cc0b7ba0 2024-05-28T14:12:54.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6d3200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe6cc0ccbd0 con 0x7fe6cc15ec50 2024-05-28T14:12:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6cbe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe6cc15e870 0x7fe6cc0d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6cbe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe6cc15e870 0x7fe6cc0d1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46730/0 (socket says 172.21.15.42:46730) 2024-05-28T14:12:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6cbe00640 1 -- 172.21.15.42:0/2135006353 learned_addr learned my addr 172.21.15.42:0/2135006353 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6d1200640 1 --2- 172.21.15.42:0/2135006353 >> v2:172.21.15.63:3300/0 conn(0x7fe6cc0b7ba0 0x7fe6cc0b7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6c3400640 1 --2- 172.21.15.42:0/2135006353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6cc15ec50 0x7fe6cc0b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6cbe00640 1 -- 172.21.15.42:0/2135006353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6cc15ec50 msgr2=0x7fe6cc0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6cbe00640 1 --2- 172.21.15.42:0/2135006353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6cc15ec50 0x7fe6cc0b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6cbe00640 1 -- 172.21.15.42:0/2135006353 >> v2:172.21.15.63:3300/0 conn(0x7fe6cc0b7ba0 msgr2=0x7fe6cc0b7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:54.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6cbe00640 1 --2- 172.21.15.42:0/2135006353 >> v2:172.21.15.63:3300/0 conn(0x7fe6cc0b7ba0 0x7fe6cc0b7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.275+0000 7fe6cbe00640 1 -- 172.21.15.42:0/2135006353 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc0cceb0 con 0x7fe6cc15e870 2024-05-28T14:12:54.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.279+0000 7fe6cbe00640 1 --2- 172.21.15.42:0/2135006353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6cc15e870 0x7fe6cc0d1230 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc073c50 tx=0x7fe6bc01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee834ad8a14432df server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:54.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.279+0000 7fe6cb400640 1 -- 172.21.15.42:0/2135006353 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0b4070 con 0x7fe6cc15e870 2024-05-28T14:12:54.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.279+0000 7fe6cb400640 1 -- 172.21.15.42:0/2135006353 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6bc0af400 con 0x7fe6cc15e870 2024-05-28T14:12:54.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.279+0000 7fe6cb400640 1 -- 172.21.15.42:0/2135006353 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0af7d0 con 0x7fe6cc15e870 2024-05-28T14:12:54.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.279+0000 7fe6d3200640 1 -- 172.21.15.42:0/2135006353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6cc15e870 msgr2=0x7fe6cc0d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:54.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.279+0000 7fe6d3200640 1 --2- 172.21.15.42:0/2135006353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6cc15e870 0x7fe6cc0d1230 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc073c50 tx=0x7fe6bc01f1e0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.279+0000 7fe6d3200640 1 -- 172.21.15.42:0/2135006353 shutdown_connections 2024-05-28T14:12:54.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.279+0000 7fe6d3200640 1 --2- 172.21.15.42:0/2135006353 >> v2:172.21.15.63:3300/0 conn(0x7fe6cc0b7ba0 0x7fe6cc0b7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.279+0000 7fe6d3200640 1 --2- 172.21.15.42:0/2135006353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6cc15ec50 0x7fe6cc0b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.279+0000 7fe6d3200640 1 --2- 172.21.15.42:0/2135006353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6cc15e870 0x7fe6cc0d1230 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.279+0000 7fe6d3200640 1 -- 172.21.15.42:0/2135006353 >> 172.21.15.42:0/2135006353 conn(0x7fe6cc0b3fd0 msgr2=0x7fe6cc0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:54.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.279+0000 7fe6d3200640 1 -- 172.21.15.42:0/2135006353 shutdown_connections 2024-05-28T14:12:54.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.279+0000 7fe6d3200640 1 -- 172.21.15.42:0/2135006353 wait complete. 2024-05-28T14:12:54.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d3200640 1 Processor -- start 2024-05-28T14:12:54.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d3200640 1 -- start start 2024-05-28T14:12:54.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d3200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6cc0b7ba0 0x7fe6cc0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:54.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6cc15e870 0x7fe6cc1613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:54.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d3200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6cc15ec50 0x7fe6cc15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:54.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d3200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc0b5330 con 0x7fe6cc15ec50 2024-05-28T14:12:54.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d3200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc0d1770 con 0x7fe6cc15e870 2024-05-28T14:12:54.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d3200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe6cc1f8500 con 0x7fe6cc0b7ba0 2024-05-28T14:12:54.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6cc15ec50 0x7fe6cc15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:54.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6cc15ec50 0x7fe6cc15ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46740/0 (socket says 172.21.15.42:46740) 2024-05-28T14:12:54.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d1200640 1 -- 172.21.15.42:0/3177644643 learned_addr learned my addr 172.21.15.42:0/3177644643 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:54.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6cbe00640 1 --2- 172.21.15.42:0/3177644643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6cc0b7ba0 0x7fe6cc0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:54.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6c3400640 1 --2- 172.21.15.42:0/3177644643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6cc15e870 0x7fe6cc1613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:54.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d1200640 1 -- 172.21.15.42:0/3177644643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6cc0b7ba0 msgr2=0x7fe6cc0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:54.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d1200640 1 --2- 172.21.15.42:0/3177644643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6cc0b7ba0 0x7fe6cc0cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d1200640 1 -- 172.21.15.42:0/3177644643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6cc15e870 msgr2=0x7fe6cc1613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:54.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d1200640 1 --2- 172.21.15.42:0/3177644643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6cc15e870 0x7fe6cc1613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d1200640 1 -- 172.21.15.42:0/3177644643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc0cb460 con 0x7fe6cc15ec50 2024-05-28T14:12:54.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d1200640 1 --2- 172.21.15.42:0/3177644643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6cc15ec50 0x7fe6cc15ce90 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400d830 tx=0x7fe6c400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:54.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6c8c00640 1 -- 172.21.15.42:0/3177644643 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400b6c0 con 0x7fe6cc15ec50 2024-05-28T14:12:54.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6c8c00640 1 -- 172.21.15.42:0/3177644643 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c4019070 con 0x7fe6cc15ec50 2024-05-28T14:12:54.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6c8c00640 1 -- 172.21.15.42:0/3177644643 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400baa0 con 0x7fe6cc15ec50 2024-05-28T14:12:54.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d3200640 1 -- 172.21.15.42:0/3177644643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6cc0cb740 con 0x7fe6cc15ec50 2024-05-28T14:12:54.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.283+0000 7fe6d3200640 1 -- 172.21.15.42:0/3177644643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6cc1618f0 con 0x7fe6cc15ec50 2024-05-28T14:12:54.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.287+0000 7fe6d3200640 1 -- 172.21.15.42:0/3177644643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6cc0cb250 con 0x7fe6cc15ec50 2024-05-28T14:12:54.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.287+0000 7fe6c8c00640 1 -- 172.21.15.42:0/3177644643 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fe6c401e460 con 0x7fe6cc15ec50 2024-05-28T14:12:54.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.287+0000 7fe6c8c00640 1 --2- 172.21.15.42:0/3177644643 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6a8069b20 0x7fe6a806bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:54.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.287+0000 7fe6c8c00640 1 -- 172.21.15.42:0/3177644643 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4999+0+0 (secure 0 0 0) 0x7fe6c4085cc0 con 0x7fe6cc15ec50 2024-05-28T14:12:54.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.287+0000 7fe6cbe00640 1 --2- 172.21.15.42:0/3177644643 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6a8069b20 0x7fe6a806bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:54.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.287+0000 7fe6cbe00640 1 --2- 172.21.15.42:0/3177644643 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6a8069b20 0x7fe6a806bfe0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0b1000 tx=0x7fe6bc0aa000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:54.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.291+0000 7fe6c8c00640 1 -- 172.21.15.42:0/3177644643 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe6c4057730 con 0x7fe6cc15ec50 2024-05-28T14:12:54.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.455+0000 7fe6d3200640 1 -- 172.21.15.42:0/3177644643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe6cc04fdd0 con 0x7fe6cc15ec50 2024-05-28T14:12:54.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.459+0000 7fe6c8c00640 1 -- 172.21.15.42:0/3177644643 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fe6c405ba30 con 0x7fe6cc15ec50 2024-05-28T14:12:54.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.463+0000 7fe6d3200640 1 -- 172.21.15.42:0/3177644643 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6a8069b20 msgr2=0x7fe6a806bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:54.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.463+0000 7fe6d3200640 1 --2- 172.21.15.42:0/3177644643 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6a8069b20 0x7fe6a806bfe0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0b1000 tx=0x7fe6bc0aa000 comp rx=0 tx=0).stop 2024-05-28T14:12:54.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.463+0000 7fe6d3200640 1 -- 172.21.15.42:0/3177644643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6cc15ec50 msgr2=0x7fe6cc15ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:54.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.463+0000 7fe6d3200640 1 --2- 172.21.15.42:0/3177644643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6cc15ec50 0x7fe6cc15ce90 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400d830 tx=0x7fe6c400dcf0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.463+0000 7fe6d3200640 1 -- 172.21.15.42:0/3177644643 shutdown_connections 2024-05-28T14:12:54.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.463+0000 7fe6d3200640 1 --2- 172.21.15.42:0/3177644643 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe6a8069b20 0x7fe6a806bfe0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.463+0000 7fe6d3200640 1 --2- 172.21.15.42:0/3177644643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6cc15ec50 0x7fe6cc15ce90 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.463+0000 7fe6d3200640 1 --2- 172.21.15.42:0/3177644643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6cc15e870 0x7fe6cc1613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.463+0000 7fe6d3200640 1 --2- 172.21.15.42:0/3177644643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6cc0b7ba0 0x7fe6cc0cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.463+0000 7fe6d3200640 1 -- 172.21.15.42:0/3177644643 >> 172.21.15.42:0/3177644643 conn(0x7fe6cc0b3fd0 msgr2=0x7fe6cc0b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:54.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.463+0000 7fe6d3200640 1 -- 172.21.15.42:0/3177644643 shutdown_connections 2024-05-28T14:12:54.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.463+0000 7fe6d3200640 1 -- 172.21.15.42:0/3177644643 wait complete. 2024-05-28T14:12:54.467 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T14:12:54.467 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:12:54.580 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:12:54.640 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T14:12:54.641 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:12:54.641 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:12:54.650 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:12:54.651 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:12:54.651 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:12:54.661 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:12:54.661 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:12:54.662 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:12:54.703 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:12:54.703 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:12:54.703 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:12:54.717 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:12:54.717 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-28T14:12:54.717 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:12:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.903+0000 7f06fbc00640 1 Processor -- start 2024-05-28T14:12:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.903+0000 7f06fbc00640 1 -- start start 2024-05-28T14:12:54.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.903+0000 7f06fbc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f06f40afa50 0x7f06f406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:54.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.903+0000 7f06fbc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06f4156e50 0x7f06f40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:54.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.903+0000 7f06fbc00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06f415cb90 con 0x7f06f4156a70 2024-05-28T14:12:54.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.903+0000 7f06fbc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f06f415ce50 con 0x7f06f40afa50 2024-05-28T14:12:54.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.903+0000 7f06fbc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f06f415d130 con 0x7f06f4156e50 2024-05-28T14:12:54.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.903+0000 7f06f9200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f06f4156a70 0x7f06f40af340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:48644/0 (socket says 172.21.15.42:48644) 2024-05-28T14:12:54.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.903+0000 7f06ebe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f06f40afa50 0x7f06f406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:54.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.903+0000 7f06f9200640 1 -- 172.21.15.42:0/425453649 learned_addr learned my addr 172.21.15.42:0/425453649 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:54.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.907+0000 7f06f9c00640 1 --2- 172.21.15.42:0/425453649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06f4156e50 0x7f06f40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:54.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.907+0000 7f06ebe00640 1 -- 172.21.15.42:0/425453649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06f4156e50 msgr2=0x7f06f40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:54.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.907+0000 7f06ebe00640 1 --2- 172.21.15.42:0/425453649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06f4156e50 0x7f06f40d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.907+0000 7f06f3e00640 1 -- 172.21.15.42:0/425453649 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1068412007 0 0) 0x7f06f415cb90 con 0x7f06f4156a70 2024-05-28T14:12:54.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.907+0000 7f06ebe00640 1 -- 172.21.15.42:0/425453649 >> v1:172.21.15.42:6789/0 conn(0x7f06f4156a70 legacy=0x7f06f40af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:54.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.907+0000 7f06ebe00640 1 -- 172.21.15.42:0/425453649 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f415d410 con 0x7f06f40afa50 2024-05-28T14:12:54.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.907+0000 7f06f9c00640 1 --2- 172.21.15.42:0/425453649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06f4156e50 0x7f06f40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.907+0000 7f06ebe00640 1 --2- 172.21.15.42:0/425453649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f40afa50 0x7f06f406dd60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0097c0 tx=0x7f06dc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32728b703277556b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.907+0000 7f06f3e00640 1 -- 172.21.15.42:0/425453649 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0b1070 con 0x7f06f40afa50 2024-05-28T14:12:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.907+0000 7f06f3e00640 1 -- 172.21.15.42:0/425453649 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06dc0ac410 con 0x7f06f40afa50 2024-05-28T14:12:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.907+0000 7f06f3e00640 1 -- 172.21.15.42:0/425453649 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0ac7e0 con 0x7f06f40afa50 2024-05-28T14:12:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 -- 172.21.15.42:0/425453649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f40afa50 msgr2=0x7f06f406dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 --2- 172.21.15.42:0/425453649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f40afa50 0x7f06f406dd60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0097c0 tx=0x7f06dc01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 -- 172.21.15.42:0/425453649 shutdown_connections 2024-05-28T14:12:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 --2- 172.21.15.42:0/425453649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06f4156e50 0x7f06f40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 --2- 172.21.15.42:0/425453649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f40afa50 0x7f06f406dd60 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 -- 172.21.15.42:0/425453649 >> 172.21.15.42:0/425453649 conn(0x7f06f406bdf0 msgr2=0x7f06f406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 -- 172.21.15.42:0/425453649 shutdown_connections 2024-05-28T14:12:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 -- 172.21.15.42:0/425453649 wait complete. 2024-05-28T14:12:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 Processor -- start 2024-05-28T14:12:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 -- start start 2024-05-28T14:12:54.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06f40afa50 0x7f06f40ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f4156a70 0x7f06f40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06f4156e50 0x7f06f40cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f06f4160f50 con 0x7f06f4156a70 2024-05-28T14:12:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f06f4160dd0 con 0x7f06f40afa50 2024-05-28T14:12:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06fbc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f06f41610d0 con 0x7f06f4156e50 2024-05-28T14:12:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06ebe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f4156a70 0x7f06f40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06ebe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f4156a70 0x7f06f40cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46762/0 (socket says 172.21.15.42:46762) 2024-05-28T14:12:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06ebe00640 1 -- 172.21.15.42:0/4242663958 learned_addr learned my addr 172.21.15.42:0/4242663958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06f9200640 1 --2- 172.21.15.42:0/4242663958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06f40afa50 0x7f06f40ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06ebe00640 1 -- 172.21.15.42:0/4242663958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06f4156e50 msgr2=0x7f06f40cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06ebe00640 1 --2- 172.21.15.42:0/4242663958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06f4156e50 0x7f06f40cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06ebe00640 1 -- 172.21.15.42:0/4242663958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06f40afa50 msgr2=0x7f06f40ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06ebe00640 1 --2- 172.21.15.42:0/4242663958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06f40afa50 0x7f06f40ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:54.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06ebe00640 1 -- 172.21.15.42:0/4242663958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f4154910 con 0x7f06f4156a70 2024-05-28T14:12:54.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06f9200640 1 --2- 172.21.15.42:0/4242663958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06f40afa50 0x7f06f40ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:54.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.911+0000 7f06ebe00640 1 --2- 172.21.15.42:0/4242663958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f4156a70 0x7f06f40cd240 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0097c0 tx=0x7f06dc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:54.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.915+0000 7f06f1600640 1 -- 172.21.15.42:0/4242663958 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0bc070 con 0x7f06f4156a70 2024-05-28T14:12:54.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.915+0000 7f06f1600640 1 -- 172.21.15.42:0/4242663958 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06dc004030 con 0x7f06f4156a70 2024-05-28T14:12:54.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.915+0000 7f06f1600640 1 -- 172.21.15.42:0/4242663958 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0b1040 con 0x7f06f4156a70 2024-05-28T14:12:54.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.915+0000 7f06fbc00640 1 -- 172.21.15.42:0/4242663958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f4154bf0 con 0x7f06f4156a70 2024-05-28T14:12:54.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.915+0000 7f06fbc00640 1 -- 172.21.15.42:0/4242663958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f41551b0 con 0x7f06f4156a70 2024-05-28T14:12:54.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.915+0000 7f06fbc00640 1 -- 172.21.15.42:0/4242663958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f4068a90 con 0x7f06f4156a70 2024-05-28T14:12:54.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.915+0000 7f06f1600640 1 -- 172.21.15.42:0/4242663958 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f06dc0bf070 con 0x7f06f4156a70 2024-05-28T14:12:54.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.915+0000 7f06f1600640 1 --2- 172.21.15.42:0/4242663958 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f06d0069b20 0x7f06d006bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:54.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.915+0000 7f06f1600640 1 -- 172.21.15.42:0/4242663958 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4999+0+0 (secure 0 0 0) 0x7f06dc11e6f0 con 0x7f06f4156a70 2024-05-28T14:12:54.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.915+0000 7f06f9200640 1 --2- 172.21.15.42:0/4242663958 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f06d0069b20 0x7f06d006bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:54.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.919+0000 7f06f9200640 1 --2- 172.21.15.42:0/4242663958 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f06d0069b20 0x7f06d006bfe0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f06e4007fd0 tx=0x7f06e4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:54.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:54.919+0000 7f06f1600640 1 -- 172.21.15.42:0/4242663958 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f06dc016090 con 0x7f06f4156a70 2024-05-28T14:12:55.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.119+0000 7f06fbc00640 1 -- 172.21.15.42:0/4242663958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06f40b5160 con 0x7f06f4156a70 2024-05-28T14:12:55.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.123+0000 7f06f1600640 1 -- 172.21.15.42:0/4242663958 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+3871 (secure 0 0 0) 0x7f06dc0f0710 con 0x7f06f4156a70 2024-05-28T14:12:55.107 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:55.107 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8607,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6836/1661021064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1661021064},{"type":"v1","addr":"172.21.15.42:6836","nonce":1661021064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":8614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4220920801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4220920801},{"type":"v1","addr":"172.21.15.63:6835","nonce":4220920801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":57050,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2398440572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6835","nonce":2398440572},{"type":"v1","addr":"172.21.15.42:6837","nonce":2398440572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":57053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/2530315729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":2530315729},{"type":"v1","addr":"172.21.15.63:6834","nonce":2530315729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[]} 2024-05-28T14:12:55.107 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 67 2024-05-28T14:12:55.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.123+0000 7f06fbc00640 1 -- 172.21.15.42:0/4242663958 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f06d0069b20 msgr2=0x7f06d006bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:55.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.123+0000 7f06fbc00640 1 --2- 172.21.15.42:0/4242663958 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f06d0069b20 0x7f06d006bfe0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f06e4007fd0 tx=0x7f06e4009040 comp rx=0 tx=0).stop 2024-05-28T14:12:55.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.123+0000 7f06fbc00640 1 -- 172.21.15.42:0/4242663958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f4156a70 msgr2=0x7f06f40cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:55.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.123+0000 7f06fbc00640 1 --2- 172.21.15.42:0/4242663958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f4156a70 0x7f06f40cd240 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0097c0 tx=0x7f06dc019240 comp rx=0 tx=0).stop 2024-05-28T14:12:55.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.123+0000 7f06fbc00640 1 -- 172.21.15.42:0/4242663958 shutdown_connections 2024-05-28T14:12:55.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.123+0000 7f06fbc00640 1 --2- 172.21.15.42:0/4242663958 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f06d0069b20 0x7f06d006bfe0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:55.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.123+0000 7f06fbc00640 1 --2- 172.21.15.42:0/4242663958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06f4156e50 0x7f06f40cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:55.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.123+0000 7f06fbc00640 1 --2- 172.21.15.42:0/4242663958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f4156a70 0x7f06f40cd240 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:55.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.123+0000 7f06fbc00640 1 --2- 172.21.15.42:0/4242663958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06f40afa50 0x7f06f40ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:55.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.123+0000 7f06fbc00640 1 -- 172.21.15.42:0/4242663958 >> 172.21.15.42:0/4242663958 conn(0x7f06f406bdf0 msgr2=0x7f06f40b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:55.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.123+0000 7f06fbc00640 1 -- 172.21.15.42:0/4242663958 shutdown_connections 2024-05-28T14:12:55.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.123+0000 7f06fbc00640 1 -- 172.21.15.42:0/4242663958 wait complete. 2024-05-28T14:12:55.129 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-28T14:12:55.129 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-28T14:12:55.130 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-28T14:12:55.130 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:12:55.130 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-28T14:12:55.130 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-28T14:12:55.130 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-28T14:12:55.130 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:12:55.130 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-28T14:12:55.131 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-28T14:12:55.131 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-28T14:12:55.131 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:12:55.131 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-28T14:12:55.131 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-28T14:12:55.131 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-28T14:12:55.131 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:12:55.142 INFO:tasks.ceph.mds.a:Stopped 2024-05-28T14:12:55.142 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-28T14:12:55.144 INFO:tasks.ceph.mds.c:Stopped 2024-05-28T14:12:55.144 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-28T14:12:55.146 INFO:tasks.ceph.mds.b:Stopped 2024-05-28T14:12:55.146 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-28T14:12:55.147 INFO:tasks.ceph.mds.d:Stopped 2024-05-28T14:12:55.147 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-28T14:12:55.188 INFO:tasks.ceph.mds.a:Started 2024-05-28T14:12:55.189 DEBUG:teuthology.parallel:result is None 2024-05-28T14:12:55.191 INFO:tasks.ceph.mds.c:Started 2024-05-28T14:12:55.191 DEBUG:teuthology.parallel:result is None 2024-05-28T14:12:55.195 INFO:tasks.ceph.mds.b:Started 2024-05-28T14:12:55.195 DEBUG:teuthology.parallel:result is None 2024-05-28T14:12:55.197 INFO:tasks.ceph.mds.d:Started 2024-05-28T14:12:55.197 DEBUG:teuthology.parallel:result is None 2024-05-28T14:12:55.197 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-28T14:12:55.276 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-05-28T14:12:55.278 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-05-28T14:12:55.289 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T14:12:55.290 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-28T14:12:55.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.359+0000 7fd07a000640 1 Processor -- start 2024-05-28T14:12:55.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.359+0000 7fd07a000640 1 -- start start 2024-05-28T14:12:55.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.363+0000 7fd07a000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd07406f210 0x7fd07406f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:55.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.363+0000 7fd07a000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd074166310 0x7fd074168700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:55.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.363+0000 7fd07a000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd07406fb30 con 0x7fd0740b4e30 2024-05-28T14:12:55.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.363+0000 7fd07a000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd07406dce0 con 0x7fd07406f210 2024-05-28T14:12:55.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.363+0000 7fd07a000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd07406dfc0 con 0x7fd074166310 2024-05-28T14:12:55.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.363+0000 7fd073400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd07406f210 0x7fd07406f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:55.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.363+0000 7fd06be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd0740b4e30 0x7fd0740b52c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:48672/0 (socket says 172.21.15.42:48672) 2024-05-28T14:12:55.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.363+0000 7fd06be00640 1 -- 172.21.15.42:0/3537257168 learned_addr learned my addr 172.21.15.42:0/3537257168 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:55.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.363+0000 7fd073e00640 1 --2- 172.21.15.42:0/3537257168 >> v2:172.21.15.63:3300/0 conn(0x7fd074166310 0x7fd074168700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:55.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.363+0000 7fd073400640 1 -- 172.21.15.42:0/3537257168 >> v2:172.21.15.63:3300/0 conn(0x7fd074166310 msgr2=0x7fd074168700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:55.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.363+0000 7fd073400640 1 --2- 172.21.15.42:0/3537257168 >> v2:172.21.15.63:3300/0 conn(0x7fd074166310 0x7fd074168700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:55.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.363+0000 7fd073400640 1 -- 172.21.15.42:0/3537257168 >> v1:172.21.15.42:6789/0 conn(0x7fd0740b4e30 legacy=0x7fd0740b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:55.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.363+0000 7fd072a00640 1 -- 172.21.15.42:0/3537257168 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1027444182 0 0) 0x7fd07406fb30 con 0x7fd0740b4e30 2024-05-28T14:12:55.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.363+0000 7fd073400640 1 -- 172.21.15.42:0/3537257168 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07406e2a0 con 0x7fd07406f210 2024-05-28T14:12:55.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.367+0000 7fd073400640 1 --2- 172.21.15.42:0/3537257168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07406f210 0x7fd07406f5f0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fd0640097c0 tx=0x7fd06401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70aef8201bb8a86 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:55.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.367+0000 7fd072a00640 1 -- 172.21.15.42:0/3537257168 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0640b0070 con 0x7fd07406f210 2024-05-28T14:12:55.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.367+0000 7fd072a00640 1 -- 172.21.15.42:0/3537257168 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd064004230 con 0x7fd07406f210 2024-05-28T14:12:55.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.367+0000 7fd072a00640 1 -- 172.21.15.42:0/3537257168 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064003620 con 0x7fd07406f210 2024-05-28T14:12:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.367+0000 7fd07a000640 1 -- 172.21.15.42:0/3537257168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07406f210 msgr2=0x7fd07406f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.367+0000 7fd07a000640 1 --2- 172.21.15.42:0/3537257168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07406f210 0x7fd07406f5f0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fd0640097c0 tx=0x7fd06401c460 comp rx=0 tx=0).stop 2024-05-28T14:12:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.367+0000 7fd07a000640 1 -- 172.21.15.42:0/3537257168 shutdown_connections 2024-05-28T14:12:55.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.367+0000 7fd07a000640 1 --2- 172.21.15.42:0/3537257168 >> v2:172.21.15.63:3300/0 conn(0x7fd074166310 0x7fd074168700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:55.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.367+0000 7fd07a000640 1 --2- 172.21.15.42:0/3537257168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07406f210 0x7fd07406f5f0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:55.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.367+0000 7fd07a000640 1 -- 172.21.15.42:0/3537257168 >> 172.21.15.42:0/3537257168 conn(0x7fd07406b9f0 msgr2=0x7fd07406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:55.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.367+0000 7fd07a000640 1 -- 172.21.15.42:0/3537257168 shutdown_connections 2024-05-28T14:12:55.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.367+0000 7fd07a000640 1 -- 172.21.15.42:0/3537257168 wait complete. 2024-05-28T14:12:55.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.367+0000 7fd07a000640 1 Processor -- start 2024-05-28T14:12:55.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.367+0000 7fd07a000640 1 -- start start 2024-05-28T14:12:55.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd07a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07406f210 0x7fd074164c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:55.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd07a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0740b4e30 0x7fd07415ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:55.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd07a000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd074166310 0x7fd07415d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:55.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd07a000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd0740afdd0 con 0x7fd0740b4e30 2024-05-28T14:12:55.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd07a000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0740afc50 con 0x7fd07406f210 2024-05-28T14:12:55.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd07a000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd07416b460 con 0x7fd074166310 2024-05-28T14:12:55.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd06be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0740b4e30 0x7fd07415ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:55.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd06be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0740b4e30 0x7fd07415ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46810/0 (socket says 172.21.15.42:46810) 2024-05-28T14:12:55.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd06be00640 1 -- 172.21.15.42:0/3869061673 learned_addr learned my addr 172.21.15.42:0/3869061673 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:55.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd073400640 1 --2- 172.21.15.42:0/3869061673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07406f210 0x7fd074164c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:55.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd073e00640 1 --2- 172.21.15.42:0/3869061673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd074166310 0x7fd07415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:55.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd06be00640 1 -- 172.21.15.42:0/3869061673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd074166310 msgr2=0x7fd07415d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:55.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd06be00640 1 --2- 172.21.15.42:0/3869061673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd074166310 0x7fd07415d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:55.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd06be00640 1 -- 172.21.15.42:0/3869061673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07406f210 msgr2=0x7fd074164c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:55.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd06be00640 1 --2- 172.21.15.42:0/3869061673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07406f210 0x7fd074164c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:55.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd06be00640 1 -- 172.21.15.42:0/3869061673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0741645f0 con 0x7fd0740b4e30 2024-05-28T14:12:55.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd06be00640 1 --2- 172.21.15.42:0/3869061673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0740b4e30 0x7fd07415ce20 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0011f0 tx=0x7fd05c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:55.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd06b400640 1 -- 172.21.15.42:0/3869061673 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c018070 con 0x7fd0740b4e30 2024-05-28T14:12:55.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd06b400640 1 -- 172.21.15.42:0/3869061673 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05c0074e0 con 0x7fd0740b4e30 2024-05-28T14:12:55.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd06b400640 1 -- 172.21.15.42:0/3869061673 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c004a10 con 0x7fd0740b4e30 2024-05-28T14:12:55.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd07a000640 1 -- 172.21.15.42:0/3869061673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0741648d0 con 0x7fd0740b4e30 2024-05-28T14:12:55.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd07a000640 1 -- 172.21.15.42:0/3869061673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07413ea00 con 0x7fd0740b4e30 2024-05-28T14:12:55.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd07a000640 1 -- 172.21.15.42:0/3869061673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd074073890 con 0x7fd0740b4e30 2024-05-28T14:12:55.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.371+0000 7fd06b400640 1 -- 172.21.15.42:0/3869061673 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fd05c01b070 con 0x7fd0740b4e30 2024-05-28T14:12:55.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.375+0000 7fd06b400640 1 --2- 172.21.15.42:0/3869061673 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd050069b20 0x7fd05006bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:55.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.375+0000 7fd06b400640 1 -- 172.21.15.42:0/3869061673 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4999+0+0 (secure 0 0 0) 0x7fd05c084a60 con 0x7fd0740b4e30 2024-05-28T14:12:55.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.375+0000 7fd073400640 1 --2- 172.21.15.42:0/3869061673 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd050069b20 0x7fd05006bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:55.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.375+0000 7fd073400640 1 --2- 172.21.15.42:0/3869061673 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd050069b20 0x7fd05006bfe0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd0640ad000 tx=0x7fd0640a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:55.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.379+0000 7fd06b400640 1 -- 172.21.15.42:0/3869061673 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd05c08e020 con 0x7fd0740b4e30 2024-05-28T14:12:55.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:55.555+0000 7fd07a000640 1 -- 172.21.15.42:0/3869061673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd07404fdd0 con 0x7fd0740b4e30 2024-05-28T14:12:56.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.111+0000 7fd06b400640 1 -- 172.21.15.42:0/3869061673 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v111) ==== 97+0+0 (secure 0 0 0) 0x7fd05c056430 con 0x7fd0740b4e30 2024-05-28T14:12:56.097 INFO:teuthology.orchestra.run.smithi042.stderr: removed all blocklist entries 2024-05-28T14:12:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.115+0000 7fd07a000640 1 -- 172.21.15.42:0/3869061673 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd050069b20 msgr2=0x7fd05006bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.115+0000 7fd07a000640 1 --2- 172.21.15.42:0/3869061673 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd050069b20 0x7fd05006bfe0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd0640ad000 tx=0x7fd0640a6000 comp rx=0 tx=0).stop 2024-05-28T14:12:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.115+0000 7fd07a000640 1 -- 172.21.15.42:0/3869061673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0740b4e30 msgr2=0x7fd07415ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.115+0000 7fd07a000640 1 --2- 172.21.15.42:0/3869061673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0740b4e30 0x7fd07415ce20 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0011f0 tx=0x7fd05c00c9b0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.115+0000 7fd07a000640 1 -- 172.21.15.42:0/3869061673 shutdown_connections 2024-05-28T14:12:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.115+0000 7fd07a000640 1 --2- 172.21.15.42:0/3869061673 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd050069b20 0x7fd05006bfe0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.115+0000 7fd07a000640 1 --2- 172.21.15.42:0/3869061673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd074166310 0x7fd07415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.115+0000 7fd07a000640 1 --2- 172.21.15.42:0/3869061673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0740b4e30 0x7fd07415ce20 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.115+0000 7fd07a000640 1 --2- 172.21.15.42:0/3869061673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd07406f210 0x7fd074164c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.115+0000 7fd07a000640 1 -- 172.21.15.42:0/3869061673 >> 172.21.15.42:0/3869061673 conn(0x7fd07406b9f0 msgr2=0x7fd0740b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.115+0000 7fd07a000640 1 -- 172.21.15.42:0/3869061673 shutdown_connections 2024-05-28T14:12:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.115+0000 7fd07a000640 1 -- 172.21.15.42:0/3869061673 wait complete. 2024-05-28T14:12:56.129 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-28T14:12:56.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.287+0000 7f5b8bc00640 1 Processor -- start 2024-05-28T14:12:56.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.287+0000 7f5b8bc00640 1 -- start start 2024-05-28T14:12:56.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.287+0000 7f5b8bc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5b84067810 0x7f5b8415ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:56.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.287+0000 7f5b8bc00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8406eaa0 con 0x7f5b840d0420 2024-05-28T14:12:56.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.287+0000 7f5b8bc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8406ed80 con 0x7f5b840b42e0 2024-05-28T14:12:56.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.287+0000 7f5b8bc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8406f060 con 0x7f5b84067810 2024-05-28T14:12:56.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.287+0000 7f5b89200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f5b840d0420 0x7f5b840b3bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:48674/0 (socket says 172.21.15.42:48674) 2024-05-28T14:12:56.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.287+0000 7f5b89200640 1 -- 172.21.15.42:0/2887836024 learned_addr learned my addr 172.21.15.42:0/2887836024 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:56.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.291+0000 7f5b89c00640 1 --2- 172.21.15.42:0/2887836024 >> v2:172.21.15.42:3300/0 conn(0x7f5b84067810 0x7f5b8415ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:56.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.291+0000 7f5b83e00640 1 -- 172.21.15.42:0/2887836024 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1942096814 0 0) 0x7f5b8406eaa0 con 0x7f5b840d0420 2024-05-28T14:12:56.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.291+0000 7f5b83e00640 1 -- 172.21.15.42:0/2887836024 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b68003530 con 0x7f5b840d0420 2024-05-28T14:12:56.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.291+0000 7f5b89c00640 1 -- 172.21.15.42:0/2887836024 >> v1:172.21.15.63:6789/0 conn(0x7f5b840b42e0 legacy=0x7f5b8406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.291+0000 7f5b89c00640 1 -- 172.21.15.42:0/2887836024 >> v1:172.21.15.42:6789/0 conn(0x7f5b840d0420 legacy=0x7f5b840b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.291+0000 7f5b89c00640 1 -- 172.21.15.42:0/2887836024 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8406f340 con 0x7f5b84067810 2024-05-28T14:12:56.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.291+0000 7f5b89c00640 1 --2- 172.21.15.42:0/2887836024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b84067810 0x7f5b8415ed10 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009d90 tx=0x7f5b7c01d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5f898a0301c9c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:56.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.291+0000 7f5b83e00640 1 -- 172.21.15.42:0/2887836024 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0b2070 con 0x7f5b84067810 2024-05-28T14:12:56.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.291+0000 7f5b83e00640 1 -- 172.21.15.42:0/2887836024 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c0a58b0 con 0x7f5b84067810 2024-05-28T14:12:56.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.291+0000 7f5b83e00640 1 -- 172.21.15.42:0/2887836024 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0a5c80 con 0x7f5b84067810 2024-05-28T14:12:56.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 -- 172.21.15.42:0/2887836024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b84067810 msgr2=0x7f5b8415ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 --2- 172.21.15.42:0/2887836024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b84067810 0x7f5b8415ed10 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009d90 tx=0x7f5b7c01d340 comp rx=0 tx=0).stop 2024-05-28T14:12:56.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 -- 172.21.15.42:0/2887836024 shutdown_connections 2024-05-28T14:12:56.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 --2- 172.21.15.42:0/2887836024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b84067810 0x7f5b8415ed10 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 -- 172.21.15.42:0/2887836024 >> 172.21.15.42:0/2887836024 conn(0x7f5b840d0b40 msgr2=0x7f5b840d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:56.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 -- 172.21.15.42:0/2887836024 shutdown_connections 2024-05-28T14:12:56.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 -- 172.21.15.42:0/2887836024 wait complete. 2024-05-28T14:12:56.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 Processor -- start 2024-05-28T14:12:56.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 -- start start 2024-05-28T14:12:56.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b84067810 0x7f5b8415b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:56.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b840b42e0 0x7f5b8414e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:56.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b840d0420 0x7f5b84150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:56.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8406cd30 con 0x7f5b840b42e0 2024-05-28T14:12:56.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8406cbb0 con 0x7f5b84067810 2024-05-28T14:12:56.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b8bc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8406ceb0 con 0x7f5b840d0420 2024-05-28T14:12:56.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b7be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b840b42e0 0x7f5b8414e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:56.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b7be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b840b42e0 0x7f5b8414e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46826/0 (socket says 172.21.15.42:46826) 2024-05-28T14:12:56.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b7be00640 1 -- 172.21.15.42:0/1680339811 learned_addr learned my addr 172.21.15.42:0/1680339811 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:56.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b89c00640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b840d0420 0x7f5b84150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:56.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b89200640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b84067810 0x7f5b8415b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:56.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b7be00640 1 -- 172.21.15.42:0/1680339811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b840d0420 msgr2=0x7f5b84150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b7be00640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b840d0420 0x7f5b84150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b7be00640 1 -- 172.21.15.42:0/1680339811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b84067810 msgr2=0x7f5b8415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b7be00640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b84067810 0x7f5b8415b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b7be00640 1 -- 172.21.15.42:0/1680339811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8414fa90 con 0x7f5b840b42e0 2024-05-28T14:12:56.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.295+0000 7f5b89c00640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b840d0420 0x7f5b84150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:56.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.299+0000 7f5b7be00640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b840b42e0 0x7f5b8414e1f0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c430 tx=0x7f5b6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:56.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.299+0000 7f5b81600640 1 -- 172.21.15.42:0/1680339811 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0052d0 con 0x7f5b840b42e0 2024-05-28T14:12:56.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.299+0000 7f5b81600640 1 -- 172.21.15.42:0/1680339811 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6c005470 con 0x7f5b840b42e0 2024-05-28T14:12:56.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.299+0000 7f5b81600640 1 -- 172.21.15.42:0/1680339811 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c007260 con 0x7f5b840b42e0 2024-05-28T14:12:56.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.299+0000 7f5b8bc00640 1 -- 172.21.15.42:0/1680339811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8414fd70 con 0x7f5b840b42e0 2024-05-28T14:12:56.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.299+0000 7f5b8bc00640 1 -- 172.21.15.42:0/1680339811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b84150330 con 0x7f5b840b42e0 2024-05-28T14:12:56.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.299+0000 7f5b8bc00640 1 -- 172.21.15.42:0/1680339811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b84068a90 con 0x7f5b840b42e0 2024-05-28T14:12:56.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.299+0000 7f5b81600640 1 -- 172.21.15.42:0/1680339811 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f5b6c007400 con 0x7f5b840b42e0 2024-05-28T14:12:56.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.299+0000 7f5b81600640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b60069b50 0x7f5b6006c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:56.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.303+0000 7f5b81600640 1 -- 172.21.15.42:0/1680339811 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4913+0+0 (secure 0 0 0) 0x7f5b6c084c20 con 0x7f5b840b42e0 2024-05-28T14:12:56.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.303+0000 7f5b89200640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b60069b50 0x7f5b6006c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.303+0000 7f5b89200640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b60069b50 0x7f5b6006c010 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5b740011f0 tx=0x7f5b74009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:56.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.307+0000 7f5b81600640 1 -- 172.21.15.42:0/1680339811 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b6c056ce0 con 0x7f5b840b42e0 2024-05-28T14:12:56.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.527+0000 7f5b8bc00640 1 -- 172.21.15.42:0/1680339811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5b84152040 con 0x7f5b840b42e0 2024-05-28T14:12:56.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.527+0000 7f5b81600640 1 -- 172.21.15.42:0/1680339811 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7f5b6c0565e0 con 0x7f5b840b42e0 2024-05-28T14:12:56.514 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:56.514 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:12:56.514 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_dump": [ 2024-05-28T14:12:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.a", 2024-05-28T14:12:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmE+3RLRAADuTKGfKeVafGeLpgr9WKUg==", 2024-05-28T14:12:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:12:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:12:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:12:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.b", 2024-05-28T14:12:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVm1rmVOhAAV46KzDJnRfDZYIZanS0g2g==", 2024-05-28T14:12:56.516 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.516 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:12:56.516 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:12:56.516 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:12:56.516 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.516 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.c", 2024-05-28T14:12:56.516 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVme4aYNRAAVxJTlugnxtFTexUclF/IwA==", 2024-05-28T14:12:56.516 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.516 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:12:56.516 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:12:56.517 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:12:56.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.517 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.d", 2024-05-28T14:12:56.517 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmeAdBBhAAr4IVFQpjQXFVlnFTHnPV9w==", 2024-05-28T14:12:56.517 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.517 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:12:56.517 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:12:56.517 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:12:56.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.518 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.0", 2024-05-28T14:12:56.518 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDB4lVmGMDBJhAA3jIyR7XKLGXEgimIGNsw6A==", 2024-05-28T14:12:56.518 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.518 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:12:56.518 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:12:56.518 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.519 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.1", 2024-05-28T14:12:56.519 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDE4lVmxMisLxAAcIz7GPlkZwvW5ig+DbAVGg==", 2024-05-28T14:12:56.519 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.519 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:12:56.519 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:12:56.519 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.519 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.2", 2024-05-28T14:12:56.519 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDH4lVmSSk4OhAAmLlLysM4Im/Kew4Xbq2/OA==", 2024-05-28T14:12:56.519 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.519 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:12:56.519 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:12:56.520 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.520 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.3", 2024-05-28T14:12:56.520 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDL4lVmvzrYBhAAiAAOQXZBuTsMVYa1vE5rkg==", 2024-05-28T14:12:56.520 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.520 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:12:56.520 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:12:56.520 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.521 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.4", 2024-05-28T14:12:56.521 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDQ4lVmrqnTOhAA2pIfgBT9LJDIhp+GgFy8Gg==", 2024-05-28T14:12:56.521 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.521 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:12:56.521 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:12:56.521 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.521 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.5", 2024-05-28T14:12:56.521 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDU4lVmYpsQCxAAoTK2aMBPrHKvsUnNOTj/Dw==", 2024-05-28T14:12:56.521 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.522 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:12:56.522 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:12:56.522 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.522 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.6", 2024-05-28T14:12:56.522 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDX4lVm6TCAFxAAs3rkDyKcLhWrn/EkIo0bVA==", 2024-05-28T14:12:56.522 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.522 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:12:56.522 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:12:56.522 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.523 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.7", 2024-05-28T14:12:56.523 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDa4lVmr5NwIRAA7Lb70VYV82dr0Frn8CCUXg==", 2024-05-28T14:12:56.523 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.523 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:12:56.523 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:12:56.523 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.523 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.523 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.0", 2024-05-28T14:12:56.523 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw==", 2024-05-28T14:12:56.524 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.524 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:12:56.524 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-05-28T14:12:56.524 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-28T14:12:56.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.524 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.1", 2024-05-28T14:12:56.524 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmcfkLFBAADNyODKTtYAlFWquZ/LeeqQ==", 2024-05-28T14:12:56.524 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.525 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:12:56.525 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:12:56.525 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:12:56.525 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:12:56.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.525 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.2", 2024-05-28T14:12:56.525 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmoxx+EBAA8mLNak5AwRmWQIgkYtMOrg==", 2024-05-28T14:12:56.525 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.525 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:12:56.525 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:12:56.525 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:12:56.526 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:12:56.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.526 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.3", 2024-05-28T14:12:56.526 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmxSUsGhAAbgr1eOh/sMfrEONKKHFo0g==", 2024-05-28T14:12:56.526 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.526 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:12:56.526 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:12:56.526 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:12:56.526 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:12:56.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.527 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.admin", 2024-05-28T14:12:56.527 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmOSBPERAAt4x2DBal4PnqxqeWQutLDQ==", 2024-05-28T14:12:56.527 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.527 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:12:56.527 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:12:56.527 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:12:56.527 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.527 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.527 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.x", 2024-05-28T14:12:56.527 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmNO5tKRAA70hPLwoE83keovlq2rolxQ==", 2024-05-28T14:12:56.527 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.527 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:12:56.528 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T14:12:56.528 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:12:56.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:56.528 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.y", 2024-05-28T14:12:56.528 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmxTVyJRAATMLAAqKne20YvV537uXZ6g==", 2024-05-28T14:12:56.528 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:12:56.528 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:12:56.528 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T14:12:56.528 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:12:56.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:56.528 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-28T14:12:56.529 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:12:56.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.531+0000 7f5b8bc00640 1 -- 172.21.15.42:0/1680339811 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b60069b50 msgr2=0x7f5b6006c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.531+0000 7f5b8bc00640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b60069b50 0x7f5b6006c010 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5b740011f0 tx=0x7f5b74009040 comp rx=0 tx=0).stop 2024-05-28T14:12:56.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.531+0000 7f5b8bc00640 1 -- 172.21.15.42:0/1680339811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b840b42e0 msgr2=0x7f5b8414e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.531+0000 7f5b8bc00640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b840b42e0 0x7f5b8414e1f0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c430 tx=0x7f5b6c00c8f0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.531+0000 7f5b8bc00640 1 -- 172.21.15.42:0/1680339811 shutdown_connections 2024-05-28T14:12:56.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.531+0000 7f5b8bc00640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b60069b50 0x7f5b6006c010 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.531+0000 7f5b8bc00640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b840d0420 0x7f5b84150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.531+0000 7f5b8bc00640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b840b42e0 0x7f5b8414e1f0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.531+0000 7f5b8bc00640 1 --2- 172.21.15.42:0/1680339811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b84067810 0x7f5b8415b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.531+0000 7f5b8bc00640 1 -- 172.21.15.42:0/1680339811 >> 172.21.15.42:0/1680339811 conn(0x7f5b840d0b40 msgr2=0x7f5b8406f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:56.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.531+0000 7f5b8bc00640 1 -- 172.21.15.42:0/1680339811 shutdown_connections 2024-05-28T14:12:56.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.531+0000 7f5b8bc00640 1 -- 172.21.15.42:0/1680339811 wait complete. 2024-05-28T14:12:56.536 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:12:56.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.691+0000 7f0f06600640 1 Processor -- start 2024-05-28T14:12:56.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.691+0000 7f0f06600640 1 -- start start 2024-05-28T14:12:56.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.691+0000 7f0f06600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0f0006f660 0x7f0f00073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:56.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.691+0000 7f0f06600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f0015ba40 0x7f0f000d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:56.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.691+0000 7f0f06600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f0015f280 con 0x7f0f00072dd0 2024-05-28T14:12:56.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.691+0000 7f0f06600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0f0015f030 con 0x7f0f0006f660 2024-05-28T14:12:56.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.691+0000 7f0f06600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0f0015f560 con 0x7f0f0015ba40 2024-05-28T14:12:56.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.695+0000 7f0effe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0f00072dd0 0x7f0f000b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:48684/0 (socket says 172.21.15.42:48684) 2024-05-28T14:12:56.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.695+0000 7f0effe00640 1 -- 172.21.15.42:0/3812760123 learned_addr learned my addr 172.21.15.42:0/3812760123 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:56.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.695+0000 7f0ef6a00640 1 --2- 172.21.15.42:0/3812760123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f0015ba40 0x7f0f000d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:56.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.695+0000 7f0eff400640 1 --2- 172.21.15.42:0/3812760123 >> v2:172.21.15.63:3300/0 conn(0x7f0f0006f660 0x7f0f00073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:56.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.695+0000 7f0efea00640 1 -- 172.21.15.42:0/3812760123 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2930637491 0 0) 0x7f0f0015f280 con 0x7f0f00072dd0 2024-05-28T14:12:56.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.695+0000 7f0efea00640 1 -- 172.21.15.42:0/3812760123 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ee4003530 con 0x7f0f00072dd0 2024-05-28T14:12:56.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.695+0000 7f0eff400640 1 -- 172.21.15.42:0/3812760123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f0015ba40 msgr2=0x7f0f000d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:56.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.695+0000 7f0eff400640 1 --2- 172.21.15.42:0/3812760123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f0015ba40 0x7f0f000d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.695+0000 7f0eff400640 1 -- 172.21.15.42:0/3812760123 >> v1:172.21.15.42:6789/0 conn(0x7f0f00072dd0 legacy=0x7f0f000b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.695+0000 7f0eff400640 1 -- 172.21.15.42:0/3812760123 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0015f840 con 0x7f0f0006f660 2024-05-28T14:12:56.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.695+0000 7f0ef6a00640 1 --2- 172.21.15.42:0/3812760123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f0015ba40 0x7f0f000d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:12:56.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.695+0000 7f0eff400640 1 --2- 172.21.15.42:0/3812760123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f0006f660 0x7f0f00073980 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00097c0 tx=0x7f0ef001c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b599d92238b9caf server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:56.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0efea00640 1 -- 172.21.15.42:0/3812760123 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00b0070 con 0x7f0f0006f660 2024-05-28T14:12:56.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0efea00640 1 -- 172.21.15.42:0/3812760123 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef00ab600 con 0x7f0f0006f660 2024-05-28T14:12:56.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0efea00640 1 -- 172.21.15.42:0/3812760123 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00ab9d0 con 0x7f0f0006f660 2024-05-28T14:12:56.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 -- 172.21.15.42:0/3812760123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f0006f660 msgr2=0x7f0f00073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 --2- 172.21.15.42:0/3812760123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f0006f660 0x7f0f00073980 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00097c0 tx=0x7f0ef001c350 comp rx=0 tx=0).stop 2024-05-28T14:12:56.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 -- 172.21.15.42:0/3812760123 shutdown_connections 2024-05-28T14:12:56.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 --2- 172.21.15.42:0/3812760123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f0015ba40 0x7f0f000d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 --2- 172.21.15.42:0/3812760123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f0006f660 0x7f0f00073980 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 -- 172.21.15.42:0/3812760123 >> 172.21.15.42:0/3812760123 conn(0x7f0f0006bdf0 msgr2=0x7f0f0006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:56.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 -- 172.21.15.42:0/3812760123 shutdown_connections 2024-05-28T14:12:56.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 -- 172.21.15.42:0/3812760123 wait complete. 2024-05-28T14:12:56.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 Processor -- start 2024-05-28T14:12:56.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 -- start start 2024-05-28T14:12:56.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f0006f660 0x7f0f000cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:56.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f00072dd0 0x7f0f0009b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:56.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f00149760 0x7f0f000ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:56.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0f00168550 con 0x7f0f00149760 2024-05-28T14:12:56.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0f001683d0 con 0x7f0f0006f660 2024-05-28T14:12:56.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0f06600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0f001686d0 con 0x7f0f00072dd0 2024-05-28T14:12:56.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0effe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f00149760 0x7f0f000ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:56.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0effe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f00149760 0x7f0f000ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46830/0 (socket says 172.21.15.42:46830) 2024-05-28T14:12:56.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.699+0000 7f0effe00640 1 -- 172.21.15.42:0/2511199151 learned_addr learned my addr 172.21.15.42:0/2511199151 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:56.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0eff400640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f0006f660 0x7f0f000cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:56.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0effe00640 1 -- 172.21.15.42:0/2511199151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f00072dd0 msgr2=0x7f0f0009b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0ef6a00640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f00072dd0 0x7f0f0009b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:56.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0effe00640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f00072dd0 0x7f0f0009b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0effe00640 1 -- 172.21.15.42:0/2511199151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f0006f660 msgr2=0x7f0f000cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0effe00640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f0006f660 0x7f0f000cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0effe00640 1 -- 172.21.15.42:0/2511199151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f00165800 con 0x7f0f00149760 2024-05-28T14:12:56.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0ef6a00640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f00072dd0 0x7f0f0009b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:56.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0eff400640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f0006f660 0x7f0f000cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:56.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0effe00640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f00149760 0x7f0f000ce770 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800cb70 tx=0x7f0ef800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:56.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0ef7e00640 1 -- 172.21.15.42:0/2511199151 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8017070 con 0x7f0f00149760 2024-05-28T14:12:56.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0ef7e00640 1 -- 172.21.15.42:0/2511199151 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef800ad40 con 0x7f0f00149760 2024-05-28T14:12:56.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0ef7e00640 1 -- 172.21.15.42:0/2511199151 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8012620 con 0x7f0f00149760 2024-05-28T14:12:56.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0f06600640 1 -- 172.21.15.42:0/2511199151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f000cd6d0 con 0x7f0f00149760 2024-05-28T14:12:56.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0f06600640 1 -- 172.21.15.42:0/2511199151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f000cdc90 con 0x7f0f00149760 2024-05-28T14:12:56.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0f06600640 1 -- 172.21.15.42:0/2511199151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f00068a90 con 0x7f0f00149760 2024-05-28T14:12:56.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.703+0000 7f0ef7e00640 1 -- 172.21.15.42:0/2511199151 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f0ef801a070 con 0x7f0f00149760 2024-05-28T14:12:56.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.707+0000 7f0ef7e00640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0edc069b20 0x7f0edc06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:56.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.707+0000 7f0ef7e00640 1 -- 172.21.15.42:0/2511199151 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4913+0+0 (secure 0 0 0) 0x7f0ef8085550 con 0x7f0f00149760 2024-05-28T14:12:56.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.707+0000 7f0eff400640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0edc069b20 0x7f0edc06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:56.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.707+0000 7f0eff400640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0edc069b20 0x7f0edc06bfe0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00ad000 tx=0x7f0ef0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:56.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.711+0000 7f0ef7e00640 1 -- 172.21.15.42:0/2511199151 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ef8057610 con 0x7f0f00149760 2024-05-28T14:12:56.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.875+0000 7f0f06600640 1 -- 172.21.15.42:0/2511199151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f00072bb0 con 0x7f0f00149760 2024-05-28T14:12:56.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.875+0000 7f0ef7e00640 1 -- 172.21.15.42:0/2511199151 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+12929 (secure 0 0 0) 0x7f0ef8056f10 con 0x7f0f00149760 2024-05-28T14:12:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:56.860 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":113,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:12:56.123441+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:12:56.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.875+0000 7f0f06600640 1 -- 172.21.15.42:0/2511199151 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0edc069b20 msgr2=0x7f0edc06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.875+0000 7f0f06600640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0edc069b20 0x7f0edc06bfe0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00ad000 tx=0x7f0ef0008040 comp rx=0 tx=0).stop 2024-05-28T14:12:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.875+0000 7f0f06600640 1 -- 172.21.15.42:0/2511199151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f00149760 msgr2=0x7f0f000ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.875+0000 7f0f06600640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f00149760 0x7f0f000ce770 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800cb70 tx=0x7f0ef800a430 comp rx=0 tx=0).stop 2024-05-28T14:12:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.875+0000 7f0f06600640 1 -- 172.21.15.42:0/2511199151 shutdown_connections 2024-05-28T14:12:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.875+0000 7f0f06600640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0edc069b20 0x7f0edc06bfe0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.875+0000 7f0f06600640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f00149760 0x7f0f000ce770 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.875+0000 7f0f06600640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f00072dd0 0x7f0f0009b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.875+0000 7f0f06600640 1 --2- 172.21.15.42:0/2511199151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f0006f660 0x7f0f000cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.875+0000 7f0f06600640 1 -- 172.21.15.42:0/2511199151 >> 172.21.15.42:0/2511199151 conn(0x7f0f0006bdf0 msgr2=0x7f0f0015cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.875+0000 7f0f06600640 1 -- 172.21.15.42:0/2511199151 shutdown_connections 2024-05-28T14:12:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:56.875+0000 7f0f06600640 1 -- 172.21.15.42:0/2511199151 wait complete. 2024-05-28T14:12:56.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:12:56.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:12:57.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.039+0000 7fc8e6400640 1 Processor -- start 2024-05-28T14:12:57.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.039+0000 7fc8e6400640 1 -- start start 2024-05-28T14:12:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.043+0000 7fc8e6400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc8e0144f80 0x7fc8e0072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.043+0000 7fc8e6400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc8e015c0c0 0x7fc8e006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.043+0000 7fc8e6400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e00afe40 con 0x7fc8e006ee20 2024-05-28T14:12:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.043+0000 7fc8e6400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e006f7c0 con 0x7fc8e015c0c0 2024-05-28T14:12:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.043+0000 7fc8e6400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e00724d0 con 0x7fc8e0144f80 2024-05-28T14:12:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.043+0000 7fc8dfe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc8e015c0c0 0x7fc8e006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.043+0000 7fc8dfe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc8e015c0c0 0x7fc8e006f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46840/0 (socket says 172.21.15.42:46840) 2024-05-28T14:12:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.043+0000 7fc8dfe00640 1 -- 172.21.15.42:0/3026152004 learned_addr learned my addr 172.21.15.42:0/3026152004 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.043+0000 7fc8df400640 1 --2- 172.21.15.42:0/3026152004 >> v2:172.21.15.63:3300/0 conn(0x7fc8e0144f80 0x7fc8e0072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.043+0000 7fc8dfe00640 1 -- 172.21.15.42:0/3026152004 >> v2:172.21.15.63:3300/0 conn(0x7fc8e0144f80 msgr2=0x7fc8e0072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:57.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.043+0000 7fc8dfe00640 1 --2- 172.21.15.42:0/3026152004 >> v2:172.21.15.63:3300/0 conn(0x7fc8e0144f80 0x7fc8e0072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.043+0000 7fc8dfe00640 1 -- 172.21.15.42:0/3026152004 >> v1:172.21.15.63:6789/0 conn(0x7fc8e006ee20 legacy=0x7fc8e015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.043+0000 7fc8dfe00640 1 -- 172.21.15.42:0/3026152004 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e015c600 con 0x7fc8e015c0c0 2024-05-28T14:12:57.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8dfe00640 1 --2- 172.21.15.42:0/3026152004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8e015c0c0 0x7fc8e006f280 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0071760 tx=0x7fc8d801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b1e3416cb6df66 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:57.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8dea00640 1 -- 172.21.15.42:0/3026152004 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d80a57a0 con 0x7fc8e015c0c0 2024-05-28T14:12:57.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8dea00640 1 -- 172.21.15.42:0/3026152004 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d80a5940 con 0x7fc8e015c0c0 2024-05-28T14:12:57.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8dea00640 1 -- 172.21.15.42:0/3026152004 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d80a5d10 con 0x7fc8e015c0c0 2024-05-28T14:12:57.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 -- 172.21.15.42:0/3026152004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8e015c0c0 msgr2=0x7fc8e006f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 --2- 172.21.15.42:0/3026152004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8e015c0c0 0x7fc8e006f280 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0071760 tx=0x7fc8d801c9f0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 -- 172.21.15.42:0/3026152004 shutdown_connections 2024-05-28T14:12:57.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 --2- 172.21.15.42:0/3026152004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8e015c0c0 0x7fc8e006f280 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 --2- 172.21.15.42:0/3026152004 >> v2:172.21.15.63:3300/0 conn(0x7fc8e0144f80 0x7fc8e0072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 -- 172.21.15.42:0/3026152004 >> 172.21.15.42:0/3026152004 conn(0x7fc8e00b21f0 msgr2=0x7fc8e00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:57.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 -- 172.21.15.42:0/3026152004 shutdown_connections 2024-05-28T14:12:57.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 -- 172.21.15.42:0/3026152004 wait complete. 2024-05-28T14:12:57.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 Processor -- start 2024-05-28T14:12:57.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 -- start start 2024-05-28T14:12:57.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8e006ee20 0x7fc8e015b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8e0144f80 0x7fc8e01f7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e015c0c0 0x7fc8e01338a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e0072ea0 con 0x7fc8e006ee20 2024-05-28T14:12:57.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e00b4010 con 0x7fc8e015c0c0 2024-05-28T14:12:57.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.047+0000 7fc8e6400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e015b930 con 0x7fc8e0144f80 2024-05-28T14:12:57.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8df400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8e006ee20 0x7fc8e015b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8df400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8e006ee20 0x7fc8e015b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46850/0 (socket says 172.21.15.42:46850) 2024-05-28T14:12:57.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8df400640 1 -- 172.21.15.42:0/1035849134 learned_addr learned my addr 172.21.15.42:0/1035849134 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:57.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8d7e00640 1 --2- 172.21.15.42:0/1035849134 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8e0144f80 0x7fc8e01f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8dfe00640 1 --2- 172.21.15.42:0/1035849134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e015c0c0 0x7fc8e01338a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8df400640 1 -- 172.21.15.42:0/1035849134 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8e0144f80 msgr2=0x7fc8e01f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8df400640 1 --2- 172.21.15.42:0/1035849134 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8e0144f80 0x7fc8e01f7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8df400640 1 -- 172.21.15.42:0/1035849134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e015c0c0 msgr2=0x7fc8e01338a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8df400640 1 --2- 172.21.15.42:0/1035849134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e015c0c0 0x7fc8e01338a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8df400640 1 -- 172.21.15.42:0/1035849134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e01675c0 con 0x7fc8e006ee20 2024-05-28T14:12:57.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8df400640 1 --2- 172.21.15.42:0/1035849134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8e006ee20 0x7fc8e015b3f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000e930 tx=0x7fc8d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:57.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8d7400640 1 -- 172.21.15.42:0/1035849134 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0017070 con 0x7fc8e006ee20 2024-05-28T14:12:57.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8d7400640 1 -- 172.21.15.42:0/1035849134 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d000ca00 con 0x7fc8e006ee20 2024-05-28T14:12:57.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8d7400640 1 -- 172.21.15.42:0/1035849134 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d000cdb0 con 0x7fc8e006ee20 2024-05-28T14:12:57.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8e6400640 1 -- 172.21.15.42:0/1035849134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e01678a0 con 0x7fc8e006ee20 2024-05-28T14:12:57.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8e6400640 1 -- 172.21.15.42:0/1035849134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e0167e60 con 0x7fc8e006ee20 2024-05-28T14:12:57.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8e6400640 1 -- 172.21.15.42:0/1035849134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e00738b0 con 0x7fc8e006ee20 2024-05-28T14:12:57.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8d7400640 1 -- 172.21.15.42:0/1035849134 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fc8d001a070 con 0x7fc8e006ee20 2024-05-28T14:12:57.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8d7400640 1 --2- 172.21.15.42:0/1035849134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc8bc069b20 0x7fc8bc06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.051+0000 7fc8d7400640 1 -- 172.21.15.42:0/1035849134 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4913+0+0 (secure 0 0 0) 0x7fc8d0085ae0 con 0x7fc8e006ee20 2024-05-28T14:12:57.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.055+0000 7fc8d7e00640 1 --2- 172.21.15.42:0/1035849134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc8bc069b20 0x7fc8bc06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.055+0000 7fc8d7e00640 1 --2- 172.21.15.42:0/1035849134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc8bc069b20 0x7fc8bc06bfe0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0132560 tx=0x7fc8c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:57.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.055+0000 7fc8d7400640 1 -- 172.21.15.42:0/1035849134 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc8d0057570 con 0x7fc8e006ee20 2024-05-28T14:12:57.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.223+0000 7fc8e6400640 1 -- 172.21.15.42:0/1035849134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8e004fdd0 con 0x7fc8e006ee20 2024-05-28T14:12:57.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.223+0000 7fc8d7400640 1 -- 172.21.15.42:0/1035849134 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7fc8d005b870 con 0x7fc8e006ee20 2024-05-28T14:12:57.209 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:12:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.227+0000 7fc8e6400640 1 -- 172.21.15.42:0/1035849134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc8bc069b20 msgr2=0x7fc8bc06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.227+0000 7fc8e6400640 1 --2- 172.21.15.42:0/1035849134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc8bc069b20 0x7fc8bc06bfe0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0132560 tx=0x7fc8c8009040 comp rx=0 tx=0).stop 2024-05-28T14:12:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.227+0000 7fc8e6400640 1 -- 172.21.15.42:0/1035849134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8e006ee20 msgr2=0x7fc8e015b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.227+0000 7fc8e6400640 1 --2- 172.21.15.42:0/1035849134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8e006ee20 0x7fc8e015b3f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000e930 tx=0x7fc8d000edf0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.227+0000 7fc8e6400640 1 -- 172.21.15.42:0/1035849134 shutdown_connections 2024-05-28T14:12:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.227+0000 7fc8e6400640 1 --2- 172.21.15.42:0/1035849134 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc8bc069b20 0x7fc8bc06bfe0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.227+0000 7fc8e6400640 1 --2- 172.21.15.42:0/1035849134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e015c0c0 0x7fc8e01338a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.227+0000 7fc8e6400640 1 --2- 172.21.15.42:0/1035849134 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8e0144f80 0x7fc8e01f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.227+0000 7fc8e6400640 1 --2- 172.21.15.42:0/1035849134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8e006ee20 0x7fc8e015b3f0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.227+0000 7fc8e6400640 1 -- 172.21.15.42:0/1035849134 >> 172.21.15.42:0/1035849134 conn(0x7fc8e00b21f0 msgr2=0x7fc8e00b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.227+0000 7fc8e6400640 1 -- 172.21.15.42:0/1035849134 shutdown_connections 2024-05-28T14:12:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.227+0000 7fc8e6400640 1 -- 172.21.15.42:0/1035849134 wait complete. 2024-05-28T14:12:57.230 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-28T14:12:57.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bfca00640 1 Processor -- start 2024-05-28T14:12:57.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bfca00640 1 -- start start 2024-05-28T14:12:57.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bfca00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bf815ba40 0x7f3bf80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bfca00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3bf8072dd0 0x7f3bf80d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bfca00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bf815f280 con 0x7f3bf806f660 2024-05-28T14:12:57.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bfca00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f3bf815f030 con 0x7f3bf8072dd0 2024-05-28T14:12:57.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bfca00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf815f560 con 0x7f3bf815ba40 2024-05-28T14:12:57.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bf6800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3bf8072dd0 0x7f3bf80d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bf5e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f3bf806f660 0x7f3bf8073980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37116/0 (socket says 172.21.15.42:37116) 2024-05-28T14:12:57.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bf5e00640 1 -- 172.21.15.42:0/3880003940 learned_addr learned my addr 172.21.15.42:0/3880003940 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:57.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bed400640 1 --2- 172.21.15.42:0/3880003940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bf815ba40 0x7f3bf80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bf6800640 1 -- 172.21.15.42:0/3880003940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bf815ba40 msgr2=0x7f3bf80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:57.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bf6800640 1 --2- 172.21.15.42:0/3880003940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bf815ba40 0x7f3bf80d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bf6800640 1 -- 172.21.15.42:0/3880003940 >> v1:172.21.15.42:6789/0 conn(0x7f3bf806f660 legacy=0x7f3bf8073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bf5400640 1 -- 172.21.15.42:0/3880003940 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 309822135 0 0) 0x7f3bf815f280 con 0x7f3bf806f660 2024-05-28T14:12:57.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bf6800640 1 -- 172.21.15.42:0/3880003940 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf815f840 con 0x7f3bf8072dd0 2024-05-28T14:12:57.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.387+0000 7f3bed400640 1 --2- 172.21.15.42:0/3880003940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bf815ba40 0x7f3bf80d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:12:57.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.391+0000 7f3bf6800640 1 --2- 172.21.15.42:0/3880003940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bf8072dd0 0x7f3bf80d0c30 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009d90 tx=0x7f3bf001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bff42e8373ccaf server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:57.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.391+0000 7f3bf5400640 1 -- 172.21.15.42:0/3880003940 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf00b1070 con 0x7f3bf8072dd0 2024-05-28T14:12:57.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.391+0000 7f3bf5400640 1 -- 172.21.15.42:0/3880003940 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bf00a4e50 con 0x7f3bf8072dd0 2024-05-28T14:12:57.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.391+0000 7f3bf5400640 1 -- 172.21.15.42:0/3880003940 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf00ac660 con 0x7f3bf8072dd0 2024-05-28T14:12:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.391+0000 7f3bfca00640 1 -- 172.21.15.42:0/3880003940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bf8072dd0 msgr2=0x7f3bf80d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.391+0000 7f3bfca00640 1 --2- 172.21.15.42:0/3880003940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bf8072dd0 0x7f3bf80d0c30 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009d90 tx=0x7f3bf001c500 comp rx=0 tx=0).stop 2024-05-28T14:12:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.391+0000 7f3bfca00640 1 -- 172.21.15.42:0/3880003940 shutdown_connections 2024-05-28T14:12:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.391+0000 7f3bfca00640 1 --2- 172.21.15.42:0/3880003940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bf8072dd0 0x7f3bf80d0c30 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.391+0000 7f3bfca00640 1 --2- 172.21.15.42:0/3880003940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bf815ba40 0x7f3bf80d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.391+0000 7f3bfca00640 1 -- 172.21.15.42:0/3880003940 >> 172.21.15.42:0/3880003940 conn(0x7f3bf806bdf0 msgr2=0x7f3bf806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.391+0000 7f3bfca00640 1 -- 172.21.15.42:0/3880003940 shutdown_connections 2024-05-28T14:12:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.391+0000 7f3bfca00640 1 -- 172.21.15.42:0/3880003940 wait complete. 2024-05-28T14:12:57.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bfca00640 1 Processor -- start 2024-05-28T14:12:57.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bfca00640 1 -- start start 2024-05-28T14:12:57.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bfca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bf806f660 0x7f3bf8099b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bfca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8072dd0 0x7f3bf8151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bfca00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bf8148a20 0x7f3bf80b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bfca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf80b3460 con 0x7f3bf806f660 2024-05-28T14:12:57.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bfca00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf80b32e0 con 0x7f3bf8072dd0 2024-05-28T14:12:57.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bfca00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf80b35e0 con 0x7f3bf8148a20 2024-05-28T14:12:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bf5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bf806f660 0x7f3bf8099b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bf5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bf806f660 0x7f3bf8099b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33784/0 (socket says 172.21.15.42:33784) 2024-05-28T14:12:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bf5e00640 1 -- 172.21.15.42:0/1911345980 learned_addr learned my addr 172.21.15.42:0/1911345980 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bf6800640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bf8148a20 0x7f3bf80b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bed400640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8072dd0 0x7f3bf8151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bf5e00640 1 -- 172.21.15.42:0/1911345980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bf8148a20 msgr2=0x7f3bf80b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bf5e00640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bf8148a20 0x7f3bf80b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bf5e00640 1 -- 172.21.15.42:0/1911345980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8072dd0 msgr2=0x7f3bf8151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bf5e00640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8072dd0 0x7f3bf8151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bf5e00640 1 -- 172.21.15.42:0/1911345980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf8149190 con 0x7f3bf806f660 2024-05-28T14:12:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bed400640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8072dd0 0x7f3bf8151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:57.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bf5e00640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bf806f660 0x7f3bf8099b20 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f3be80011f0 tx=0x7f3be800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:57.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3beea00640 1 -- 172.21.15.42:0/1911345980 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8018070 con 0x7f3bf806f660 2024-05-28T14:12:57.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3beea00640 1 -- 172.21.15.42:0/1911345980 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be80074e0 con 0x7f3bf806f660 2024-05-28T14:12:57.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3beea00640 1 -- 172.21.15.42:0/1911345980 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8004a10 con 0x7f3bf806f660 2024-05-28T14:12:57.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bfca00640 1 -- 172.21.15.42:0/1911345980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf8149470 con 0x7f3bf806f660 2024-05-28T14:12:57.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.395+0000 7f3bfca00640 1 -- 172.21.15.42:0/1911345980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf8149a30 con 0x7f3bf806f660 2024-05-28T14:12:57.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.399+0000 7f3bfca00640 1 -- 172.21.15.42:0/1911345980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf8068a90 con 0x7f3bf806f660 2024-05-28T14:12:57.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.399+0000 7f3beea00640 1 -- 172.21.15.42:0/1911345980 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f3be8004bb0 con 0x7f3bf806f660 2024-05-28T14:12:57.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.399+0000 7f3beea00640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3bd4069b50 0x7f3bd406c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.399+0000 7f3beea00640 1 -- 172.21.15.42:0/1911345980 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4913+0+0 (secure 0 0 0) 0x7f3be8084a00 con 0x7f3bf806f660 2024-05-28T14:12:57.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.399+0000 7f3bed400640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3bd4069b50 0x7f3bd406c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.399+0000 7f3bed400640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3bd4069b50 0x7f3bd406c010 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80cfe00 tx=0x7f3be0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:57.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.403+0000 7f3beea00640 1 -- 172.21.15.42:0/1911345980 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3be8056a20 con 0x7f3bf806f660 2024-05-28T14:12:57.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.571+0000 7f3bfca00640 1 -- 172.21.15.42:0/1911345980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3bf80b4ae0 con 0x7f3bf806f660 2024-05-28T14:12:57.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.571+0000 7f3beea00640 1 -- 172.21.15.42:0/1911345980 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v113) ==== 84+0+65 (secure 0 0 0) 0x7f3be8056320 con 0x7f3bf806f660 2024-05-28T14:12:57.559 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:12:57.559 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:12:57.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:12:57.560 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-05-28T14:12:57.560 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-05-28T14:12:57.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:12:57.560 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:12:57.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.571+0000 7f3bfca00640 1 -- 172.21.15.42:0/1911345980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3bd4069b50 msgr2=0x7f3bd406c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.571+0000 7f3bfca00640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3bd4069b50 0x7f3bd406c010 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80cfe00 tx=0x7f3be0008040 comp rx=0 tx=0).stop 2024-05-28T14:12:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.571+0000 7f3bfca00640 1 -- 172.21.15.42:0/1911345980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bf806f660 msgr2=0x7f3bf8099b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.571+0000 7f3bfca00640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bf806f660 0x7f3bf8099b20 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f3be80011f0 tx=0x7f3be800c9b0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.571+0000 7f3bfca00640 1 -- 172.21.15.42:0/1911345980 shutdown_connections 2024-05-28T14:12:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.571+0000 7f3bfca00640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3bd4069b50 0x7f3bd406c010 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.571+0000 7f3bfca00640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bf8148a20 0x7f3bf80b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.571+0000 7f3bfca00640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8072dd0 0x7f3bf8151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.571+0000 7f3bfca00640 1 --2- 172.21.15.42:0/1911345980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bf806f660 0x7f3bf8099b20 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.571+0000 7f3bfca00640 1 -- 172.21.15.42:0/1911345980 >> 172.21.15.42:0/1911345980 conn(0x7f3bf806bdf0 msgr2=0x7f3bf80b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.571+0000 7f3bfca00640 1 -- 172.21.15.42:0/1911345980 shutdown_connections 2024-05-28T14:12:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.575+0000 7f3bfca00640 1 -- 172.21.15.42:0/1911345980 wait complete. 2024-05-28T14:12:57.577 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-28T14:12:57.577 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-28T14:12:57.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.739+0000 7fac4c400640 1 Processor -- start 2024-05-28T14:12:57.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.739+0000 7fac4c400640 1 -- start start 2024-05-28T14:12:57.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.739+0000 7fac4c400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fac440ccfc0 0x7fac440cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.739+0000 7fac4c400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac440d1250 con 0x7fac440cd960 2024-05-28T14:12:57.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.739+0000 7fac4c400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac440b5510 con 0x7fac4415fcc0 2024-05-28T14:12:57.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.739+0000 7fac4c400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fac440b57f0 con 0x7fac440ccfc0 2024-05-28T14:12:57.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.739+0000 7fac49000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fac440ccfc0 0x7fac440cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.739+0000 7fac4a400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fac440cd960 0x7fac440b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37118/0 (socket says 172.21.15.42:37118) 2024-05-28T14:12:57.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.739+0000 7fac4a400640 1 -- 172.21.15.42:0/3471429707 learned_addr learned my addr 172.21.15.42:0/3471429707 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:57.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.739+0000 7fac49000640 1 -- 172.21.15.42:0/3471429707 >> v1:172.21.15.63:6789/0 conn(0x7fac4415fcc0 legacy=0x7fac440d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.739+0000 7fac49000640 1 -- 172.21.15.42:0/3471429707 >> v1:172.21.15.42:6789/0 conn(0x7fac440cd960 legacy=0x7fac440b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.739+0000 7fac49000640 1 -- 172.21.15.42:0/3471429707 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac440b5ad0 con 0x7fac440ccfc0 2024-05-28T14:12:57.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.739+0000 7fac43e00640 1 -- 172.21.15.42:0/3471429707 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2178412678 0 0) 0x7fac440d1250 con 0x7fac440cd960 2024-05-28T14:12:57.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.743+0000 7fac49000640 1 --2- 172.21.15.42:0/3471429707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac440ccfc0 0x7fac440cd420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fac300097c0 tx=0x7fac3001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9e910f3810e4cc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:57.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.743+0000 7fac43e00640 1 -- 172.21.15.42:0/3471429707 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac300b0070 con 0x7fac440ccfc0 2024-05-28T14:12:57.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.743+0000 7fac43e00640 1 -- 172.21.15.42:0/3471429707 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac300ab600 con 0x7fac440ccfc0 2024-05-28T14:12:57.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.743+0000 7fac43e00640 1 -- 172.21.15.42:0/3471429707 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac300ab9d0 con 0x7fac440ccfc0 2024-05-28T14:12:57.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.743+0000 7fac4c400640 1 -- 172.21.15.42:0/3471429707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac440ccfc0 msgr2=0x7fac440cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.743+0000 7fac4c400640 1 --2- 172.21.15.42:0/3471429707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac440ccfc0 0x7fac440cd420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fac300097c0 tx=0x7fac3001c4a0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.743+0000 7fac4c400640 1 -- 172.21.15.42:0/3471429707 shutdown_connections 2024-05-28T14:12:57.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.743+0000 7fac4c400640 1 --2- 172.21.15.42:0/3471429707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac440ccfc0 0x7fac440cd420 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.743+0000 7fac4c400640 1 -- 172.21.15.42:0/3471429707 >> 172.21.15.42:0/3471429707 conn(0x7fac440b3fa0 msgr2=0x7fac440b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:57.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.743+0000 7fac4c400640 1 -- 172.21.15.42:0/3471429707 shutdown_connections 2024-05-28T14:12:57.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.743+0000 7fac4c400640 1 -- 172.21.15.42:0/3471429707 wait complete. 2024-05-28T14:12:57.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac4c400640 1 Processor -- start 2024-05-28T14:12:57.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac4c400640 1 -- start start 2024-05-28T14:12:57.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac4c400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac440ccfc0 0x7fac440c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac4c400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac440cd960 0x7fac441542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac4c400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac4415fcc0 0x7fac440700d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac4c400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fac440b7320 con 0x7fac440ccfc0 2024-05-28T14:12:57.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac4c400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fac440b71a0 con 0x7fac4415fcc0 2024-05-28T14:12:57.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac4c400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fac4416a280 con 0x7fac440cd960 2024-05-28T14:12:57.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac49a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac440ccfc0 0x7fac440c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac49a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac440ccfc0 0x7fac440c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33786/0 (socket says 172.21.15.42:33786) 2024-05-28T14:12:57.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac49a00640 1 -- 172.21.15.42:0/1935022272 learned_addr learned my addr 172.21.15.42:0/1935022272 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:57.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac49000640 1 --2- 172.21.15.42:0/1935022272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac440cd960 0x7fac441542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac4a400640 1 --2- 172.21.15.42:0/1935022272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac4415fcc0 0x7fac440700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac49a00640 1 -- 172.21.15.42:0/1935022272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac440cd960 msgr2=0x7fac441542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac49a00640 1 --2- 172.21.15.42:0/1935022272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac440cd960 0x7fac441542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac49a00640 1 -- 172.21.15.42:0/1935022272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac4415fcc0 msgr2=0x7fac440700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:57.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac49a00640 1 --2- 172.21.15.42:0/1935022272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac4415fcc0 0x7fac440700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:57.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac49a00640 1 -- 172.21.15.42:0/1935022272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac440c8540 con 0x7fac440ccfc0 2024-05-28T14:12:57.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac49a00640 1 --2- 172.21.15.42:0/1935022272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac440ccfc0 0x7fac440c8d10 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fac3800c440 tx=0x7fac3800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:57.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac41600640 1 -- 172.21.15.42:0/1935022272 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38017070 con 0x7fac440ccfc0 2024-05-28T14:12:57.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac41600640 1 -- 172.21.15.42:0/1935022272 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac380052f0 con 0x7fac440ccfc0 2024-05-28T14:12:57.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac41600640 1 -- 172.21.15.42:0/1935022272 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38003840 con 0x7fac440ccfc0 2024-05-28T14:12:57.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac4c400640 1 -- 172.21.15.42:0/1935022272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac440c74a0 con 0x7fac440ccfc0 2024-05-28T14:12:57.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.747+0000 7fac4c400640 1 -- 172.21.15.42:0/1935022272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac440c7a60 con 0x7fac440ccfc0 2024-05-28T14:12:57.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.751+0000 7fac4c400640 1 -- 172.21.15.42:0/1935022272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac44068a90 con 0x7fac440ccfc0 2024-05-28T14:12:57.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.751+0000 7fac41600640 1 -- 172.21.15.42:0/1935022272 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fac3801a070 con 0x7fac440ccfc0 2024-05-28T14:12:57.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.751+0000 7fac41600640 1 --2- 172.21.15.42:0/1935022272 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac24069b20 0x7fac2406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:57.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.751+0000 7fac41600640 1 -- 172.21.15.42:0/1935022272 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4913+0+0 (secure 0 0 0) 0x7fac380849e0 con 0x7fac440ccfc0 2024-05-28T14:12:57.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.751+0000 7fac49000640 1 --2- 172.21.15.42:0/1935022272 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac24069b20 0x7fac2406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:57.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.751+0000 7fac49000640 1 --2- 172.21.15.42:0/1935022272 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac24069b20 0x7fac2406bfe0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fac300ad000 tx=0x7fac300a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:57.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.755+0000 7fac41600640 1 -- 172.21.15.42:0/1935022272 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fac38056430 con 0x7fac440ccfc0 2024-05-28T14:12:57.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:57.919+0000 7fac4c400640 1 -- 172.21.15.42:0/1935022272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fac440750f0 con 0x7fac440ccfc0 2024-05-28T14:12:58.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.123+0000 7fac41600640 1 -- 172.21.15.42:0/1935022272 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v114) ==== 138+0+0 (secure 0 0 0) 0x7fac3805a730 con 0x7fac440ccfc0 2024-05-28T14:12:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-05-28T14:12:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.123+0000 7fac4c400640 1 -- 172.21.15.42:0/1935022272 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac24069b20 msgr2=0x7fac2406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.123+0000 7fac4c400640 1 --2- 172.21.15.42:0/1935022272 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac24069b20 0x7fac2406bfe0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fac300ad000 tx=0x7fac300a6000 comp rx=0 tx=0).stop 2024-05-28T14:12:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.123+0000 7fac4c400640 1 -- 172.21.15.42:0/1935022272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac440ccfc0 msgr2=0x7fac440c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.123+0000 7fac4c400640 1 --2- 172.21.15.42:0/1935022272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac440ccfc0 0x7fac440c8d10 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fac3800c440 tx=0x7fac3800c900 comp rx=0 tx=0).stop 2024-05-28T14:12:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.123+0000 7fac4c400640 1 -- 172.21.15.42:0/1935022272 shutdown_connections 2024-05-28T14:12:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.123+0000 7fac4c400640 1 --2- 172.21.15.42:0/1935022272 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac24069b20 0x7fac2406bfe0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:58.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.123+0000 7fac4c400640 1 --2- 172.21.15.42:0/1935022272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac4415fcc0 0x7fac440700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.123+0000 7fac4c400640 1 --2- 172.21.15.42:0/1935022272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac440cd960 0x7fac441542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.123+0000 7fac4c400640 1 --2- 172.21.15.42:0/1935022272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac440ccfc0 0x7fac440c8d10 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.123+0000 7fac4c400640 1 -- 172.21.15.42:0/1935022272 >> 172.21.15.42:0/1935022272 conn(0x7fac440b3fa0 msgr2=0x7fac44071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.123+0000 7fac4c400640 1 -- 172.21.15.42:0/1935022272 shutdown_connections 2024-05-28T14:12:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.123+0000 7fac4c400640 1 -- 172.21.15.42:0/1935022272 wait complete. 2024-05-28T14:12:58.134 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-28T14:12:58.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.295+0000 7f6f17a00640 1 Processor -- start 2024-05-28T14:12:58.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.295+0000 7f6f17a00640 1 -- start start 2024-05-28T14:12:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.299+0000 7f6f17a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6f1015c0c0 0x7f6f1006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.299+0000 7f6f17a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f100afe40 con 0x7f6f10144f80 2024-05-28T14:12:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.299+0000 7f6f17a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f1015c600 con 0x7f6f1006ee20 2024-05-28T14:12:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.299+0000 7f6f17a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6f1006f7c0 con 0x7f6f1015c0c0 2024-05-28T14:12:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.299+0000 7f6f15000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f6f10144f80 0x7f6f10072960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37128/0 (socket says 172.21.15.42:37128) 2024-05-28T14:12:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.299+0000 7f6f15000640 1 -- 172.21.15.42:0/1672084347 learned_addr learned my addr 172.21.15.42:0/1672084347 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:12:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.299+0000 7f6f15a00640 1 --2- 172.21.15.42:0/1672084347 >> v2:172.21.15.42:3300/0 conn(0x7f6f1015c0c0 0x7f6f1006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:58.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.299+0000 7f6f0fe00640 1 -- 172.21.15.42:0/1672084347 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982329894 0 0) 0x7f6f100afe40 con 0x7f6f10144f80 2024-05-28T14:12:58.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.299+0000 7f6f0fe00640 1 -- 172.21.15.42:0/1672084347 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ef4003530 con 0x7f6f10144f80 2024-05-28T14:12:58.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.299+0000 7f6f15a00640 1 -- 172.21.15.42:0/1672084347 >> v1:172.21.15.63:6789/0 conn(0x7f6f1006ee20 legacy=0x7f6f1015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:58.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.299+0000 7f6f15a00640 1 -- 172.21.15.42:0/1672084347 >> v1:172.21.15.42:6789/0 conn(0x7f6f10144f80 legacy=0x7f6f10072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:58.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.299+0000 7f6f15a00640 1 -- 172.21.15.42:0/1672084347 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1015c8e0 con 0x7f6f1015c0c0 2024-05-28T14:12:58.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.303+0000 7f6f15a00640 1 --2- 172.21.15.42:0/1672084347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f1015c0c0 0x7f6f1006f280 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009d90 tx=0x7f6f0801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0c9e1b448f3dee server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:58.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.303+0000 7f6f0fe00640 1 -- 172.21.15.42:0/1672084347 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f080b2070 con 0x7f6f1015c0c0 2024-05-28T14:12:58.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.303+0000 7f6f0fe00640 1 -- 172.21.15.42:0/1672084347 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f080a5830 con 0x7f6f1015c0c0 2024-05-28T14:12:58.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.303+0000 7f6f0fe00640 1 -- 172.21.15.42:0/1672084347 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f080a5c00 con 0x7f6f1015c0c0 2024-05-28T14:12:58.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.303+0000 7f6f17a00640 1 -- 172.21.15.42:0/1672084347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f1015c0c0 msgr2=0x7f6f1006f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:58.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.303+0000 7f6f17a00640 1 --2- 172.21.15.42:0/1672084347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f1015c0c0 0x7f6f1006f280 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009d90 tx=0x7f6f0801c970 comp rx=0 tx=0).stop 2024-05-28T14:12:58.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.303+0000 7f6f17a00640 1 -- 172.21.15.42:0/1672084347 shutdown_connections 2024-05-28T14:12:58.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.303+0000 7f6f17a00640 1 --2- 172.21.15.42:0/1672084347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f1015c0c0 0x7f6f1006f280 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:58.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.303+0000 7f6f17a00640 1 -- 172.21.15.42:0/1672084347 >> 172.21.15.42:0/1672084347 conn(0x7f6f100b21f0 msgr2=0x7f6f100b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:58.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.303+0000 7f6f17a00640 1 -- 172.21.15.42:0/1672084347 shutdown_connections 2024-05-28T14:12:58.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.303+0000 7f6f17a00640 1 -- 172.21.15.42:0/1672084347 wait complete. 2024-05-28T14:12:58.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.303+0000 7f6f17a00640 1 Processor -- start 2024-05-28T14:12:58.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.303+0000 7f6f17a00640 1 -- start start 2024-05-28T14:12:58.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f17a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f1006ee20 0x7f6f1015ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:58.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f17a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f10144f80 0x7f6f1006df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:58.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f17a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1015c0c0 0x7f6f100b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:58.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f17a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6f100724d0 con 0x7f6f10144f80 2024-05-28T14:12:58.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f17a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f100b42e0 con 0x7f6f1015c0c0 2024-05-28T14:12:58.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f17a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6f10169140 con 0x7f6f1006ee20 2024-05-28T14:12:58.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f07e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f10144f80 0x7f6f1006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:58.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f07e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f10144f80 0x7f6f1006df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33808/0 (socket says 172.21.15.42:33808) 2024-05-28T14:12:58.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f07e00640 1 -- 172.21.15.42:0/994219411 learned_addr learned my addr 172.21.15.42:0/994219411 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:58.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f15000640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f1006ee20 0x7f6f1015ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:58.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f15a00640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1015c0c0 0x7f6f100b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:58.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f07e00640 1 -- 172.21.15.42:0/994219411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f1006ee20 msgr2=0x7f6f1015ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:58.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f07e00640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f1006ee20 0x7f6f1015ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:58.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f07e00640 1 -- 172.21.15.42:0/994219411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1015c0c0 msgr2=0x7f6f100b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:58.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f07e00640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1015c0c0 0x7f6f100b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:58.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f07e00640 1 -- 172.21.15.42:0/994219411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1006eae0 con 0x7f6f10144f80 2024-05-28T14:12:58.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f15a00640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1015c0c0 0x7f6f100b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:58.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f15000640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f1006ee20 0x7f6f1015ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:58.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f07e00640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f10144f80 0x7f6f1006df20 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800c420 tx=0x7f6ef800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:58.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f0d600640 1 -- 172.21.15.42:0/994219411 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef80052d0 con 0x7f6f10144f80 2024-05-28T14:12:58.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f0d600640 1 -- 172.21.15.42:0/994219411 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef8005470 con 0x7f6f10144f80 2024-05-28T14:12:58.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f0d600640 1 -- 172.21.15.42:0/994219411 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8007260 con 0x7f6f10144f80 2024-05-28T14:12:58.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f17a00640 1 -- 172.21.15.42:0/994219411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1006e800 con 0x7f6f10144f80 2024-05-28T14:12:58.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f17a00640 1 -- 172.21.15.42:0/994219411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f10134150 con 0x7f6f10144f80 2024-05-28T14:12:58.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f17a00640 1 -- 172.21.15.42:0/994219411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f100738b0 con 0x7f6f10144f80 2024-05-28T14:12:58.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.307+0000 7f6f0d600640 1 -- 172.21.15.42:0/994219411 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f6ef8007400 con 0x7f6f10144f80 2024-05-28T14:12:58.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.311+0000 7f6f0d600640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6eec069b20 0x7f6eec06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:58.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.311+0000 7f6f0d600640 1 -- 172.21.15.42:0/994219411 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5283+0+0 (secure 0 0 0) 0x7f6ef80853d0 con 0x7f6f10144f80 2024-05-28T14:12:58.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.311+0000 7f6f15000640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6eec069b20 0x7f6eec06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:58.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.311+0000 7f6f15000640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6eec069b20 0x7f6eec06bfe0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6f000011f0 tx=0x7f6f00009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:58.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.315+0000 7f6f0d600640 1 -- 172.21.15.42:0/994219411 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6ef8057570 con 0x7f6f10144f80 2024-05-28T14:12:58.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:58.479+0000 7f6f17a00640 1 -- 172.21.15.42:0/994219411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6f100cf3a0 con 0x7f6f10144f80 2024-05-28T14:12:59.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.131+0000 7f6f0d600640 1 -- 172.21.15.42:0/994219411 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v115) ==== 170+0+0 (secure 0 0 0) 0x7f6ef8056e70 con 0x7f6f10144f80 2024-05-28T14:12:59.115 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-05-28T14:12:59.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.135+0000 7f6f17a00640 1 -- 172.21.15.42:0/994219411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6eec069b20 msgr2=0x7f6eec06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:59.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.135+0000 7f6f17a00640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6eec069b20 0x7f6eec06bfe0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6f000011f0 tx=0x7f6f00009040 comp rx=0 tx=0).stop 2024-05-28T14:12:59.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.135+0000 7f6f17a00640 1 -- 172.21.15.42:0/994219411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f10144f80 msgr2=0x7f6f1006df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:59.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.135+0000 7f6f17a00640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f10144f80 0x7f6f1006df20 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800c420 tx=0x7f6ef800c8e0 comp rx=0 tx=0).stop 2024-05-28T14:12:59.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.135+0000 7f6f17a00640 1 -- 172.21.15.42:0/994219411 shutdown_connections 2024-05-28T14:12:59.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.135+0000 7f6f17a00640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6eec069b20 0x7f6eec06bfe0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:59.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.135+0000 7f6f17a00640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1015c0c0 0x7f6f100b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:59.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.135+0000 7f6f17a00640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f10144f80 0x7f6f1006df20 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:59.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.135+0000 7f6f17a00640 1 --2- 172.21.15.42:0/994219411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f1006ee20 0x7f6f1015ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:59.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.135+0000 7f6f17a00640 1 -- 172.21.15.42:0/994219411 >> 172.21.15.42:0/994219411 conn(0x7f6f100b21f0 msgr2=0x7f6f1015d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:59.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.135+0000 7f6f17a00640 1 -- 172.21.15.42:0/994219411 shutdown_connections 2024-05-28T14:12:59.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.135+0000 7f6f17a00640 1 -- 172.21.15.42:0/994219411 wait complete. 2024-05-28T14:12:59.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-28T14:12:59.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5e04c00640 1 Processor -- start 2024-05-28T14:12:59.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5e04c00640 1 -- start start 2024-05-28T14:12:59.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5e04c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5e000ce3b0 0x7f5e000ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:59.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5e04c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e0013fd10 0x7f5e00140180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:59.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5e04c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e00160230 con 0x7f5e00071de0 2024-05-28T14:12:59.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5e04c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5e00160510 con 0x7f5e0013fd10 2024-05-28T14:12:59.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5e04c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5e001607f0 con 0x7f5e000ce3b0 2024-05-28T14:12:59.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5dfe800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e0013fd10 0x7f5e00140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:59.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5dfe800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e0013fd10 0x7f5e00140180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33820/0 (socket says 172.21.15.42:33820) 2024-05-28T14:12:59.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5dfe800640 1 -- 172.21.15.42:0/2379544734 learned_addr learned my addr 172.21.15.42:0/2379544734 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:59.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5dfe800640 1 -- 172.21.15.42:0/2379544734 >> v2:172.21.15.63:3300/0 conn(0x7f5e000ce3b0 msgr2=0x7f5e000ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:59.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5df7e00640 1 --2- 172.21.15.42:0/2379544734 >> v2:172.21.15.63:3300/0 conn(0x7f5e000ce3b0 0x7f5e000ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:59.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5dfe800640 1 --2- 172.21.15.42:0/2379544734 >> v2:172.21.15.63:3300/0 conn(0x7f5e000ce3b0 0x7f5e000ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:59.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5dfe800640 1 -- 172.21.15.42:0/2379544734 >> v1:172.21.15.63:6789/0 conn(0x7f5e00071de0 legacy=0x7f5e000721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:12:59.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5dfe800640 1 -- 172.21.15.42:0/2379544734 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e00160ad0 con 0x7f5e0013fd10 2024-05-28T14:12:59.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.323+0000 7f5df7e00640 1 --2- 172.21.15.42:0/2379544734 >> v2:172.21.15.63:3300/0 conn(0x7f5e000ce3b0 0x7f5e000ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:12:59.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.327+0000 7f5dfe800640 1 --2- 172.21.15.42:0/2379544734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0013fd10 0x7f5e00140180 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f5e00073280 tx=0x7f5df801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b615cf53cb6dd34 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:59.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.327+0000 7f5dfd400640 1 -- 172.21.15.42:0/2379544734 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80a5550 con 0x7f5e0013fd10 2024-05-28T14:12:59.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.327+0000 7f5dfd400640 1 -- 172.21.15.42:0/2379544734 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df80a56f0 con 0x7f5e0013fd10 2024-05-28T14:12:59.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.327+0000 7f5dfd400640 1 -- 172.21.15.42:0/2379544734 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80a5ac0 con 0x7f5e0013fd10 2024-05-28T14:12:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.327+0000 7f5e04c00640 1 -- 172.21.15.42:0/2379544734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0013fd10 msgr2=0x7f5e00140180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.327+0000 7f5e04c00640 1 --2- 172.21.15.42:0/2379544734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0013fd10 0x7f5e00140180 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f5e00073280 tx=0x7f5df801c890 comp rx=0 tx=0).stop 2024-05-28T14:12:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.327+0000 7f5e04c00640 1 -- 172.21.15.42:0/2379544734 shutdown_connections 2024-05-28T14:12:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.327+0000 7f5e04c00640 1 --2- 172.21.15.42:0/2379544734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0013fd10 0x7f5e00140180 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.327+0000 7f5e04c00640 1 --2- 172.21.15.42:0/2379544734 >> v2:172.21.15.63:3300/0 conn(0x7f5e000ce3b0 0x7f5e000ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.327+0000 7f5e04c00640 1 -- 172.21.15.42:0/2379544734 >> 172.21.15.42:0/2379544734 conn(0x7f5e000b21f0 msgr2=0x7f5e000b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:12:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.327+0000 7f5e04c00640 1 -- 172.21.15.42:0/2379544734 shutdown_connections 2024-05-28T14:12:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.327+0000 7f5e04c00640 1 -- 172.21.15.42:0/2379544734 wait complete. 2024-05-28T14:12:59.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5e04c00640 1 Processor -- start 2024-05-28T14:12:59.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5e04c00640 1 -- start start 2024-05-28T14:12:59.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5e04c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e00071de0 0x7f5e001561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:59.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5e04c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000ce3b0 0x7f5e000b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:59.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5e04c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0013fd10 0x7f5e001455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:59.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5e04c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00169830 con 0x7f5e0013fd10 2024-05-28T14:12:59.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5e04c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001696b0 con 0x7f5e00071de0 2024-05-28T14:12:59.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5e04c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5e001699b0 con 0x7f5e000ce3b0 2024-05-28T14:12:59.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5dfde00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e00071de0 0x7f5e001561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:59.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5dfe800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0013fd10 0x7f5e001455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:59.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5dfe800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0013fd10 0x7f5e001455f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33832/0 (socket says 172.21.15.42:33832) 2024-05-28T14:12:59.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5dfe800640 1 -- 172.21.15.42:0/2458329052 learned_addr learned my addr 172.21.15.42:0/2458329052 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:12:59.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5df7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000ce3b0 0x7f5e000b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:59.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5dfe800640 1 -- 172.21.15.42:0/2458329052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000ce3b0 msgr2=0x7f5e000b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:59.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5dfe800640 1 --2- 172.21.15.42:0/2458329052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000ce3b0 0x7f5e000b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:59.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5dfe800640 1 -- 172.21.15.42:0/2458329052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e00071de0 msgr2=0x7f5e001561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:12:59.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5dfe800640 1 --2- 172.21.15.42:0/2458329052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e00071de0 0x7f5e001561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:12:59.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5dfe800640 1 -- 172.21.15.42:0/2458329052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e000b4830 con 0x7f5e0013fd10 2024-05-28T14:12:59.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5dfe800640 1 --2- 172.21.15.42:0/2458329052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0013fd10 0x7f5e001455f0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009e20 tx=0x7f5df8019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:59.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5df6000640 1 -- 172.21.15.42:0/2458329052 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80bc020 con 0x7f5e0013fd10 2024-05-28T14:12:59.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5df6000640 1 -- 172.21.15.42:0/2458329052 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df80b8690 con 0x7f5e0013fd10 2024-05-28T14:12:59.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5df6000640 1 -- 172.21.15.42:0/2458329052 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80b8a80 con 0x7f5e0013fd10 2024-05-28T14:12:59.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.331+0000 7f5e04c00640 1 -- 172.21.15.42:0/2458329052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e000b4b10 con 0x7f5e0013fd10 2024-05-28T14:12:59.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.335+0000 7f5e04c00640 1 -- 172.21.15.42:0/2458329052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e000b50d0 con 0x7f5e0013fd10 2024-05-28T14:12:59.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.335+0000 7f5e04c00640 1 -- 172.21.15.42:0/2458329052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e000738b0 con 0x7f5e0013fd10 2024-05-28T14:12:59.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.335+0000 7f5df6000640 1 -- 172.21.15.42:0/2458329052 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f5df80bf020 con 0x7f5e0013fd10 2024-05-28T14:12:59.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.335+0000 7f5df6000640 1 --2- 172.21.15.42:0/2458329052 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ddc069b20 0x7f5ddc06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:12:59.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.335+0000 7f5df6000640 1 -- 172.21.15.42:0/2458329052 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5665+0+0 (secure 0 0 0) 0x7f5df8120980 con 0x7f5e0013fd10 2024-05-28T14:12:59.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.335+0000 7f5dfde00640 1 --2- 172.21.15.42:0/2458329052 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ddc069b20 0x7f5ddc06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:12:59.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.339+0000 7f5dfde00640 1 --2- 172.21.15.42:0/2458329052 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ddc069b20 0x7f5ddc06bfe0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5df00098e0 tx=0x7f5df0009060 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:12:59.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.339+0000 7f5df6000640 1 -- 172.21.15.42:0/2458329052 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5df80c6030 con 0x7f5e0013fd10 2024-05-28T14:12:59.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:12:59.539+0000 7f5e04c00640 1 -- 172.21.15.42:0/2458329052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5e0004fdd0 con 0x7f5e0013fd10 2024-05-28T14:13:00.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.131+0000 7f5df6000640 1 -- 172.21.15.42:0/2458329052 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v70) ==== 174+0+0 (secure 0 0 0) 0x7f5df80165a0 con 0x7f5e0013fd10 2024-05-28T14:13:00.117 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 16 and data pool 17 2024-05-28T14:13:00.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.135+0000 7f5e04c00640 1 -- 172.21.15.42:0/2458329052 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ddc069b20 msgr2=0x7f5ddc06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:00.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.135+0000 7f5e04c00640 1 --2- 172.21.15.42:0/2458329052 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ddc069b20 0x7f5ddc06bfe0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5df00098e0 tx=0x7f5df0009060 comp rx=0 tx=0).stop 2024-05-28T14:13:00.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.135+0000 7f5e04c00640 1 -- 172.21.15.42:0/2458329052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0013fd10 msgr2=0x7f5e001455f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:00.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.135+0000 7f5e04c00640 1 --2- 172.21.15.42:0/2458329052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0013fd10 0x7f5e001455f0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009e20 tx=0x7f5df8019630 comp rx=0 tx=0).stop 2024-05-28T14:13:00.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.139+0000 7f5e04c00640 1 -- 172.21.15.42:0/2458329052 shutdown_connections 2024-05-28T14:13:00.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.139+0000 7f5e04c00640 1 --2- 172.21.15.42:0/2458329052 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5ddc069b20 0x7f5ddc06bfe0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.139+0000 7f5e04c00640 1 --2- 172.21.15.42:0/2458329052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e0013fd10 0x7f5e001455f0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.139+0000 7f5e04c00640 1 --2- 172.21.15.42:0/2458329052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e000ce3b0 0x7f5e000b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.139+0000 7f5e04c00640 1 --2- 172.21.15.42:0/2458329052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e00071de0 0x7f5e001561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.139+0000 7f5e04c00640 1 -- 172.21.15.42:0/2458329052 >> 172.21.15.42:0/2458329052 conn(0x7f5e000b21f0 msgr2=0x7f5e0015ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:00.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.143+0000 7f5e04c00640 1 -- 172.21.15.42:0/2458329052 shutdown_connections 2024-05-28T14:13:00.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.143+0000 7f5e04c00640 1 -- 172.21.15.42:0/2458329052 wait complete. 2024-05-28T14:13:00.174 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:13:00.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55d9800640 1 Processor -- start 2024-05-28T14:13:00.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55d9800640 1 -- start start 2024-05-28T14:13:00.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55d9800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55d406f660 0x7f55d4073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:00.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55d9800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f55d40d0e00 0x7f55d406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:00.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55d9800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55d415f280 con 0x7f55d415ba40 2024-05-28T14:13:00.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55d9800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f55d415f030 con 0x7f55d40d0e00 2024-05-28T14:13:00.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55d9800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f55d415f560 con 0x7f55d406f660 2024-05-28T14:13:00.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55d3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f55d40d0e00 0x7f55d406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:00.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55d2a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55d406f660 0x7f55d4073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:00.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55ca000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f55d415ba40 0x7f55d40d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37142/0 (socket says 172.21.15.42:37142) 2024-05-28T14:13:00.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55ca000640 1 -- 172.21.15.42:0/2040141763 learned_addr learned my addr 172.21.15.42:0/2040141763 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:13:00.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55d3400640 1 -- 172.21.15.42:0/2040141763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55d406f660 msgr2=0x7f55d4073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:00.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55d3400640 1 --2- 172.21.15.42:0/2040141763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55d406f660 0x7f55d4073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55d3400640 1 -- 172.21.15.42:0/2040141763 >> v1:172.21.15.42:6789/0 conn(0x7f55d415ba40 legacy=0x7f55d40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:00.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.331+0000 7f55d3400640 1 -- 172.21.15.42:0/2040141763 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d415f840 con 0x7f55d40d0e00 2024-05-28T14:13:00.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.335+0000 7f55d3400640 1 --2- 172.21.15.42:0/2040141763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d40d0e00 0x7f55d406dd60 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009d90 tx=0x7f55cc01c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2b3c7922751d07 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:00.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.335+0000 7f55d2000640 1 -- 172.21.15.42:0/2040141763 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0b2070 con 0x7f55d40d0e00 2024-05-28T14:13:00.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.335+0000 7f55d2000640 1 -- 172.21.15.42:0/2040141763 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55cc0a58c0 con 0x7f55d40d0e00 2024-05-28T14:13:00.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.335+0000 7f55d2000640 1 -- 172.21.15.42:0/2040141763 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0a5c90 con 0x7f55d40d0e00 2024-05-28T14:13:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.335+0000 7f55d9800640 1 -- 172.21.15.42:0/2040141763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d40d0e00 msgr2=0x7f55d406dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.335+0000 7f55d9800640 1 --2- 172.21.15.42:0/2040141763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d40d0e00 0x7f55d406dd60 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009d90 tx=0x7f55cc01c820 comp rx=0 tx=0).stop 2024-05-28T14:13:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.335+0000 7f55d9800640 1 -- 172.21.15.42:0/2040141763 shutdown_connections 2024-05-28T14:13:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.335+0000 7f55d9800640 1 --2- 172.21.15.42:0/2040141763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d40d0e00 0x7f55d406dd60 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.335+0000 7f55d9800640 1 --2- 172.21.15.42:0/2040141763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55d406f660 0x7f55d4073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.335+0000 7f55d9800640 1 -- 172.21.15.42:0/2040141763 >> 172.21.15.42:0/2040141763 conn(0x7f55d406bdf0 msgr2=0x7f55d406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.335+0000 7f55d9800640 1 -- 172.21.15.42:0/2040141763 shutdown_connections 2024-05-28T14:13:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.335+0000 7f55d9800640 1 -- 172.21.15.42:0/2040141763 wait complete. 2024-05-28T14:13:00.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d9800640 1 Processor -- start 2024-05-28T14:13:00.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d9800640 1 -- start start 2024-05-28T14:13:00.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d406f660 0x7f55d40ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d9800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55d40d0e00 0x7f55d4099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d414fa20 0x7f55d409ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d9800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55d4168550 con 0x7f55d406f660 2024-05-28T14:13:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d9800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f55d41683d0 con 0x7f55d414fa20 2024-05-28T14:13:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d9800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f55d41686d0 con 0x7f55d40d0e00 2024-05-28T14:13:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d406f660 0x7f55d40ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d406f660 0x7f55d40ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33844/0 (socket says 172.21.15.42:33844) 2024-05-28T14:13:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d2a00640 1 -- 172.21.15.42:0/1382866675 learned_addr learned my addr 172.21.15.42:0/1382866675 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d2a00640 1 -- 172.21.15.42:0/1382866675 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55d40d0e00 msgr2=0x7f55d4099920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d2a00640 1 --2- 172.21.15.42:0/1382866675 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55d40d0e00 0x7f55d4099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d2a00640 1 -- 172.21.15.42:0/1382866675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d414fa20 msgr2=0x7f55d409ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d2a00640 1 --2- 172.21.15.42:0/1382866675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d414fa20 0x7f55d409ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d2a00640 1 -- 172.21.15.42:0/1382866675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d40ce5d0 con 0x7f55d406f660 2024-05-28T14:13:00.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d2a00640 1 --2- 172.21.15.42:0/1382866675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d406f660 0x7f55d40ce090 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f55c400e930 tx=0x7f55c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:00.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55cb400640 1 -- 172.21.15.42:0/1382866675 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4017070 con 0x7f55d406f660 2024-05-28T14:13:00.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55cb400640 1 -- 172.21.15.42:0/1382866675 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c400ca00 con 0x7f55d406f660 2024-05-28T14:13:00.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55cb400640 1 -- 172.21.15.42:0/1382866675 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c400cdb0 con 0x7f55d406f660 2024-05-28T14:13:00.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d9800640 1 -- 172.21.15.42:0/1382866675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d40ce8a0 con 0x7f55d406f660 2024-05-28T14:13:00.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.339+0000 7f55d9800640 1 -- 172.21.15.42:0/1382866675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d409a4e0 con 0x7f55d406f660 2024-05-28T14:13:00.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.343+0000 7f55d9800640 1 -- 172.21.15.42:0/1382866675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d406ee00 con 0x7f55d406f660 2024-05-28T14:13:00.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.343+0000 7f55cb400640 1 -- 172.21.15.42:0/1382866675 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f55c401a070 con 0x7f55d406f660 2024-05-28T14:13:00.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.343+0000 7f55cb400640 1 --2- 172.21.15.42:0/1382866675 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55b0069b20 0x7f55b006bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:00.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.343+0000 7f55cb400640 1 -- 172.21.15.42:0/1382866675 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5765+0+0 (secure 0 0 0) 0x7f55c4085750 con 0x7f55d406f660 2024-05-28T14:13:00.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.343+0000 7f55ca000640 1 --2- 172.21.15.42:0/1382866675 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55b0069b20 0x7f55b006bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:00.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.343+0000 7f55ca000640 1 --2- 172.21.15.42:0/1382866675 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55b0069b20 0x7f55b006bfe0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f55bc000f70 tx=0x7f55bc007a90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:00.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.347+0000 7f55cb400640 1 -- 172.21.15.42:0/1382866675 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f55c40578d0 con 0x7f55d406f660 2024-05-28T14:13:00.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.515+0000 7f55d9800640 1 -- 172.21.15.42:0/1382866675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55d4068a90 con 0x7f55d406f660 2024-05-28T14:13:00.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.515+0000 7f55cb400640 1 -- 172.21.15.42:0/1382866675 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16668 (secure 0 0 0) 0x7f55c40571d0 con 0x7f55d406f660 2024-05-28T14:13:00.501 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:00.501 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":116,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:12:59.544590+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:12:57.936312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-28T14:12:58.495752+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:13:00.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.519+0000 7f55d9800640 1 -- 172.21.15.42:0/1382866675 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55b0069b20 msgr2=0x7f55b006bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:00.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.519+0000 7f55d9800640 1 --2- 172.21.15.42:0/1382866675 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55b0069b20 0x7f55b006bfe0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f55bc000f70 tx=0x7f55bc007a90 comp rx=0 tx=0).stop 2024-05-28T14:13:00.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.519+0000 7f55d9800640 1 -- 172.21.15.42:0/1382866675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d406f660 msgr2=0x7f55d40ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:00.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.519+0000 7f55d9800640 1 --2- 172.21.15.42:0/1382866675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d406f660 0x7f55d40ce090 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f55c400e930 tx=0x7f55c400edf0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.519+0000 7f55d9800640 1 -- 172.21.15.42:0/1382866675 shutdown_connections 2024-05-28T14:13:00.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.519+0000 7f55d9800640 1 --2- 172.21.15.42:0/1382866675 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55b0069b20 0x7f55b006bfe0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.519+0000 7f55d9800640 1 --2- 172.21.15.42:0/1382866675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d414fa20 0x7f55d409ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.519+0000 7f55d9800640 1 --2- 172.21.15.42:0/1382866675 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55d40d0e00 0x7f55d4099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.519+0000 7f55d9800640 1 --2- 172.21.15.42:0/1382866675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d406f660 0x7f55d40ce090 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.519+0000 7f55d9800640 1 -- 172.21.15.42:0/1382866675 >> 172.21.15.42:0/1382866675 conn(0x7f55d406bdf0 msgr2=0x7f55d40d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:00.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.519+0000 7f55d9800640 1 -- 172.21.15.42:0/1382866675 shutdown_connections 2024-05-28T14:13:00.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.519+0000 7f55d9800640 1 -- 172.21.15.42:0/1382866675 wait complete. 2024-05-28T14:13:00.523 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:13:00.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.679+0000 7f46cc200640 1 Processor -- start 2024-05-28T14:13:00.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.679+0000 7f46cc200640 1 -- start start 2024-05-28T14:13:00.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46cc200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f46c406f660 0x7f46c4073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:00.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46cc200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46c415ba40 0x7f46c40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:00.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46cc200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f46c4072dd0 0x7f46c40d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:00.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46cc200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f46c415f030 con 0x7f46c4072dd0 2024-05-28T14:13:00.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46cc200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f46c415f280 con 0x7f46c406f660 2024-05-28T14:13:00.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46cc200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46c415f560 con 0x7f46c415ba40 2024-05-28T14:13:00.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46ca200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f46c4072dd0 0x7f46c40d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:00.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46ca200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f46c4072dd0 0x7f46c40d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33846/0 (socket says 172.21.15.42:33846) 2024-05-28T14:13:00.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46ca200640 1 -- 172.21.15.42:0/4195253262 learned_addr learned my addr 172.21.15.42:0/4195253262 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:00.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46c8e00640 1 --2- 172.21.15.42:0/4195253262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46c415ba40 0x7f46c40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:00.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46c9800640 1 --2- 172.21.15.42:0/4195253262 >> v2:172.21.15.63:3300/0 conn(0x7f46c406f660 0x7f46c4073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:00.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46ca200640 1 -- 172.21.15.42:0/4195253262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46c415ba40 msgr2=0x7f46c40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:00.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46ca200640 1 --2- 172.21.15.42:0/4195253262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46c415ba40 0x7f46c40d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46ca200640 1 -- 172.21.15.42:0/4195253262 >> v2:172.21.15.63:3300/0 conn(0x7f46c406f660 msgr2=0x7f46c4073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:00.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46ca200640 1 --2- 172.21.15.42:0/4195253262 >> v2:172.21.15.63:3300/0 conn(0x7f46c406f660 0x7f46c4073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.683+0000 7f46ca200640 1 -- 172.21.15.42:0/4195253262 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c415f840 con 0x7f46c4072dd0 2024-05-28T14:13:00.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46ca200640 1 --2- 172.21.15.42:0/4195253262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c4072dd0 0x7f46c40d0c30 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f46c40d13a0 tx=0x7f46bc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59792ea95fe99bdf server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:00.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46c3e00640 1 -- 172.21.15.42:0/4195253262 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0a5700 con 0x7f46c4072dd0 2024-05-28T14:13:00.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46c3e00640 1 -- 172.21.15.42:0/4195253262 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46bc0a58a0 con 0x7f46c4072dd0 2024-05-28T14:13:00.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46c3e00640 1 -- 172.21.15.42:0/4195253262 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0a5c70 con 0x7f46c4072dd0 2024-05-28T14:13:00.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46cc200640 1 -- 172.21.15.42:0/4195253262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c4072dd0 msgr2=0x7f46c40d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:00.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46cc200640 1 --2- 172.21.15.42:0/4195253262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c4072dd0 0x7f46c40d0c30 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f46c40d13a0 tx=0x7f46bc01c8a0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46cc200640 1 -- 172.21.15.42:0/4195253262 shutdown_connections 2024-05-28T14:13:00.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46cc200640 1 --2- 172.21.15.42:0/4195253262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c4072dd0 0x7f46c40d0c30 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46cc200640 1 --2- 172.21.15.42:0/4195253262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46c415ba40 0x7f46c40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46cc200640 1 --2- 172.21.15.42:0/4195253262 >> v2:172.21.15.63:3300/0 conn(0x7f46c406f660 0x7f46c4073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46cc200640 1 -- 172.21.15.42:0/4195253262 >> 172.21.15.42:0/4195253262 conn(0x7f46c406bdf0 msgr2=0x7f46c406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:00.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46cc200640 1 -- 172.21.15.42:0/4195253262 shutdown_connections 2024-05-28T14:13:00.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46cc200640 1 -- 172.21.15.42:0/4195253262 wait complete. 2024-05-28T14:13:00.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46cc200640 1 Processor -- start 2024-05-28T14:13:00.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.687+0000 7f46cc200640 1 -- start start 2024-05-28T14:13:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46cc200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46c406f660 0x7f46c40ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46cc200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c4072dd0 0x7f46c40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46cc200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46c40ceb10 0x7f46c40cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46cc200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46c40b4d80 con 0x7f46c4072dd0 2024-05-28T14:13:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46cc200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46c40b4c00 con 0x7f46c406f660 2024-05-28T14:13:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46cc200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46c40b4f00 con 0x7f46c40ceb10 2024-05-28T14:13:00.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46c8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c4072dd0 0x7f46c40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46c9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46c406f660 0x7f46c40ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46c9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46c406f660 0x7f46c40ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:50706/0 (socket says 172.21.15.42:50706) 2024-05-28T14:13:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46c9800640 1 -- 172.21.15.42:0/3508105649 learned_addr learned my addr 172.21.15.42:0/3508105649 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46ca200640 1 --2- 172.21.15.42:0/3508105649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46c40ceb10 0x7f46c40cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46c9800640 1 -- 172.21.15.42:0/3508105649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46c40ceb10 msgr2=0x7f46c40cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46c9800640 1 --2- 172.21.15.42:0/3508105649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46c40ceb10 0x7f46c40cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46c9800640 1 -- 172.21.15.42:0/3508105649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c4072dd0 msgr2=0x7f46c40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46c9800640 1 --2- 172.21.15.42:0/3508105649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c4072dd0 0x7f46c40cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46c9800640 1 -- 172.21.15.42:0/3508105649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c409a2d0 con 0x7f46c406f660 2024-05-28T14:13:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46c9800640 1 --2- 172.21.15.42:0/3508105649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46c406f660 0x7f46c40ccd00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f46b800e8e0 tx=0x7f46b800eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46c1600640 1 -- 172.21.15.42:0/3508105649 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8017070 con 0x7f46c406f660 2024-05-28T14:13:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46c1600640 1 -- 172.21.15.42:0/3508105649 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b800ca00 con 0x7f46c406f660 2024-05-28T14:13:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46c1600640 1 -- 172.21.15.42:0/3508105649 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b800cdd0 con 0x7f46c406f660 2024-05-28T14:13:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46cc200640 1 -- 172.21.15.42:0/3508105649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c409a610 con 0x7f46c406f660 2024-05-28T14:13:00.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46cc200640 1 -- 172.21.15.42:0/3508105649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c409ae50 con 0x7f46c406f660 2024-05-28T14:13:00.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46cc200640 1 -- 172.21.15.42:0/3508105649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c4068a90 con 0x7f46c406f660 2024-05-28T14:13:00.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.691+0000 7f46c1600640 1 -- 172.21.15.42:0/3508105649 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f46b801a070 con 0x7f46c406f660 2024-05-28T14:13:00.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.695+0000 7f46c1600640 1 --2- 172.21.15.42:0/3508105649 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f46a4069b20 0x7f46a406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:00.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.695+0000 7f46c1600640 1 -- 172.21.15.42:0/3508105649 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5765+0+0 (secure 0 0 0) 0x7f46b80857f0 con 0x7f46c406f660 2024-05-28T14:13:00.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.695+0000 7f46c8e00640 1 --2- 172.21.15.42:0/3508105649 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f46a4069b20 0x7f46a406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:00.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.695+0000 7f46c8e00640 1 --2- 172.21.15.42:0/3508105649 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f46a4069b20 0x7f46a406bfe0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f46c40ceaa0 tx=0x7f46b0009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:00.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.699+0000 7f46c1600640 1 -- 172.21.15.42:0/3508105649 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f46c409a610 con 0x7f46c406f660 2024-05-28T14:13:00.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.871+0000 7f46cc200640 1 -- 172.21.15.42:0/3508105649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46c4072bb0 con 0x7f46c406f660 2024-05-28T14:13:00.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.871+0000 7f46c1600640 1 -- 172.21.15.42:0/3508105649 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16668 (secure 0 0 0) 0x7f46b80571c0 con 0x7f46c406f660 2024-05-28T14:13:00.858 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":116,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:12:59.544590+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:12:57.936312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-28T14:12:58.495752+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:13:00.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.875+0000 7f46cc200640 1 -- 172.21.15.42:0/3508105649 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f46a4069b20 msgr2=0x7f46a406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:00.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.875+0000 7f46cc200640 1 --2- 172.21.15.42:0/3508105649 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f46a4069b20 0x7f46a406bfe0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f46c40ceaa0 tx=0x7f46b0009190 comp rx=0 tx=0).stop 2024-05-28T14:13:00.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.875+0000 7f46cc200640 1 -- 172.21.15.42:0/3508105649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46c406f660 msgr2=0x7f46c40ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:00.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.875+0000 7f46cc200640 1 --2- 172.21.15.42:0/3508105649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46c406f660 0x7f46c40ccd00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f46b800e8e0 tx=0x7f46b800eda0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.875+0000 7f46cc200640 1 -- 172.21.15.42:0/3508105649 shutdown_connections 2024-05-28T14:13:00.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.875+0000 7f46cc200640 1 --2- 172.21.15.42:0/3508105649 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f46a4069b20 0x7f46a406bfe0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.875+0000 7f46cc200640 1 --2- 172.21.15.42:0/3508105649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46c40ceb10 0x7f46c40cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.875+0000 7f46cc200640 1 --2- 172.21.15.42:0/3508105649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c4072dd0 0x7f46c40cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.875+0000 7f46cc200640 1 --2- 172.21.15.42:0/3508105649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46c406f660 0x7f46c40ccd00 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:00.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.875+0000 7f46cc200640 1 -- 172.21.15.42:0/3508105649 >> 172.21.15.42:0/3508105649 conn(0x7f46c406bdf0 msgr2=0x7f46c415e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:00.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.875+0000 7f46cc200640 1 -- 172.21.15.42:0/3508105649 shutdown_connections 2024-05-28T14:13:00.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:00.875+0000 7f46cc200640 1 -- 172.21.15.42:0/3508105649 wait complete. 2024-05-28T14:13:00.881 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-28T14:13:01.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.039+0000 7f58c4c00640 1 Processor -- start 2024-05-28T14:13:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.039+0000 7f58c4c00640 1 -- start start 2024-05-28T14:13:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.039+0000 7f58c4c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58c015e620 0x7f58c006ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.039+0000 7f58c4c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f58c00b4660 0x7f58c00b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.039+0000 7f58c4c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c006f2e0 con 0x7f58c00b2a40 2024-05-28T14:13:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.039+0000 7f58c4c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f58c00b2e90 con 0x7f58c00b4660 2024-05-28T14:13:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.039+0000 7f58c4c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f58c00ad8c0 con 0x7f58c015e620 2024-05-28T14:13:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.043+0000 7f58b7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f58c00b4660 0x7f58c00b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.043+0000 7f58b7e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f58c00b4660 0x7f58c00b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33854/0 (socket says 172.21.15.42:33854) 2024-05-28T14:13:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.043+0000 7f58b7e00640 1 -- 172.21.15.42:0/2600600653 learned_addr learned my addr 172.21.15.42:0/2600600653 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.043+0000 7f58bde00640 1 --2- 172.21.15.42:0/2600600653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58c015e620 0x7f58c006ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.043+0000 7f58b7e00640 1 -- 172.21.15.42:0/2600600653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58c015e620 msgr2=0x7f58c006ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.043+0000 7f58b7e00640 1 --2- 172.21.15.42:0/2600600653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58c015e620 0x7f58c006ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.043+0000 7f58b7e00640 1 -- 172.21.15.42:0/2600600653 >> v1:172.21.15.63:6789/0 conn(0x7f58c00b2a40 legacy=0x7f58c0072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.043+0000 7f58b7e00640 1 -- 172.21.15.42:0/2600600653 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c006f5c0 con 0x7f58c00b4660 2024-05-28T14:13:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.043+0000 7f58bde00640 1 --2- 172.21.15.42:0/2600600653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58c015e620 0x7f58c006ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.043+0000 7f58b7e00640 1 --2- 172.21.15.42:0/2600600653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c00b4660 0x7f58c00b2500 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009870 tx=0x7f58a801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71273e5e4a441f21 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.043+0000 7f58bd400640 1 -- 172.21.15.42:0/2600600653 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a80ab650 con 0x7f58c00b4660 2024-05-28T14:13:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.043+0000 7f58bd400640 1 -- 172.21.15.42:0/2600600653 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a80b0070 con 0x7f58c00b4660 2024-05-28T14:13:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.043+0000 7f58bd400640 1 -- 172.21.15.42:0/2600600653 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a80aba10 con 0x7f58c00b4660 2024-05-28T14:13:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 -- 172.21.15.42:0/2600600653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c00b4660 msgr2=0x7f58c00b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 --2- 172.21.15.42:0/2600600653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c00b4660 0x7f58c00b2500 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009870 tx=0x7f58a801c560 comp rx=0 tx=0).stop 2024-05-28T14:13:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 -- 172.21.15.42:0/2600600653 shutdown_connections 2024-05-28T14:13:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 --2- 172.21.15.42:0/2600600653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c00b4660 0x7f58c00b2500 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 --2- 172.21.15.42:0/2600600653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58c015e620 0x7f58c006ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 -- 172.21.15.42:0/2600600653 >> 172.21.15.42:0/2600600653 conn(0x7f58c00d07c0 msgr2=0x7f58c00d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 -- 172.21.15.42:0/2600600653 shutdown_connections 2024-05-28T14:13:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 -- 172.21.15.42:0/2600600653 wait complete. 2024-05-28T14:13:01.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 Processor -- start 2024-05-28T14:13:01.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 -- start start 2024-05-28T14:13:01.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58c00b2a40 0x7f58c01f8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c00b4660 0x7f58c0161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58c015e620 0x7f58c00b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f58c015eb80 con 0x7f58c00b4660 2024-05-28T14:13:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f58c015ea00 con 0x7f58c015e620 2024-05-28T14:13:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58c4c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f58c016b360 con 0x7f58c00b2a40 2024-05-28T14:13:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c00b4660 0x7f58c0161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c00b4660 0x7f58c0161570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33858/0 (socket says 172.21.15.42:33858) 2024-05-28T14:13:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58b7e00640 1 -- 172.21.15.42:0/3816179110 learned_addr learned my addr 172.21.15.42:0/3816179110 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58bde00640 1 --2- 172.21.15.42:0/3816179110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58c00b2a40 0x7f58c01f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58be800640 1 --2- 172.21.15.42:0/3816179110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58c015e620 0x7f58c00b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:01.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58b7e00640 1 -- 172.21.15.42:0/3816179110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58c00b2a40 msgr2=0x7f58c01f8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58b7e00640 1 --2- 172.21.15.42:0/3816179110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58c00b2a40 0x7f58c01f8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58b7e00640 1 -- 172.21.15.42:0/3816179110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58c015e620 msgr2=0x7f58c00b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.047+0000 7f58b7e00640 1 --2- 172.21.15.42:0/3816179110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58c015e620 0x7f58c00b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.051+0000 7f58b7e00640 1 -- 172.21.15.42:0/3816179110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c01f8b40 con 0x7f58c00b4660 2024-05-28T14:13:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.051+0000 7f58b7e00640 1 --2- 172.21.15.42:0/3816179110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c00b4660 0x7f58c0161570 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f58a801ac10 tx=0x7f58a801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.051+0000 7f58b6000640 1 -- 172.21.15.42:0/3816179110 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a80be5f0 con 0x7f58c00b4660 2024-05-28T14:13:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.051+0000 7f58b6000640 1 -- 172.21.15.42:0/3816179110 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a80be790 con 0x7f58c00b4660 2024-05-28T14:13:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.051+0000 7f58b6000640 1 -- 172.21.15.42:0/3816179110 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a80beb80 con 0x7f58c00b4660 2024-05-28T14:13:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.051+0000 7f58c4c00640 1 -- 172.21.15.42:0/3816179110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c0152f60 con 0x7f58c00b4660 2024-05-28T14:13:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.051+0000 7f58c4c00640 1 -- 172.21.15.42:0/3816179110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c0153520 con 0x7f58c00b4660 2024-05-28T14:13:01.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.051+0000 7f58c4c00640 1 -- 172.21.15.42:0/3816179110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c0073bd0 con 0x7f58c00b4660 2024-05-28T14:13:01.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.051+0000 7f58b6000640 1 -- 172.21.15.42:0/3816179110 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f58a80b43f0 con 0x7f58c00b4660 2024-05-28T14:13:01.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.055+0000 7f58b6000640 1 --2- 172.21.15.42:0/3816179110 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f589c069b20 0x7f589c06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.055+0000 7f58b6000640 1 -- 172.21.15.42:0/3816179110 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5765+0+0 (secure 0 0 0) 0x7f58a8120040 con 0x7f58c00b4660 2024-05-28T14:13:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.055+0000 7f58bde00640 1 --2- 172.21.15.42:0/3816179110 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f589c069b20 0x7f589c06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.055+0000 7f58bde00640 1 --2- 172.21.15.42:0/3816179110 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f589c069b20 0x7f589c06bfe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f58c00b40d0 tx=0x7f58b0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.055+0000 7f58b6000640 1 -- 172.21.15.42:0/3816179110 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f58c01f8b40 con 0x7f58c00b4660 2024-05-28T14:13:01.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:01.267+0000 7f58c4c00640 1 -- 172.21.15.42:0/3816179110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f58c00d1690 con 0x7f58c00b4660 2024-05-28T14:13:02.106 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-28T14:13:02.124+0000 7f4490c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-28T14:13:02.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.163+0000 7f58b6000640 1 -- 172.21.15.42:0/3816179110 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) ==== 118+0+0 (secure 0 0 0) 0x7f58a8016150 con 0x7f58c00b4660 2024-05-28T14:13:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.167+0000 7f58c4c00640 1 -- 172.21.15.42:0/3816179110 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f589c069b20 msgr2=0x7f589c06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.167+0000 7f58c4c00640 1 --2- 172.21.15.42:0/3816179110 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f589c069b20 0x7f589c06bfe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f58c00b40d0 tx=0x7f58b0008040 comp rx=0 tx=0).stop 2024-05-28T14:13:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.167+0000 7f58c4c00640 1 -- 172.21.15.42:0/3816179110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c00b4660 msgr2=0x7f58c0161570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.167+0000 7f58c4c00640 1 --2- 172.21.15.42:0/3816179110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c00b4660 0x7f58c0161570 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f58a801ac10 tx=0x7f58a801ada0 comp rx=0 tx=0).stop 2024-05-28T14:13:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.167+0000 7f58c4c00640 1 -- 172.21.15.42:0/3816179110 shutdown_connections 2024-05-28T14:13:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.167+0000 7f58c4c00640 1 --2- 172.21.15.42:0/3816179110 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f589c069b20 0x7f589c06bfe0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.167+0000 7f58c4c00640 1 --2- 172.21.15.42:0/3816179110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58c015e620 0x7f58c00b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.167+0000 7f58c4c00640 1 --2- 172.21.15.42:0/3816179110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58c00b4660 0x7f58c0161570 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.167+0000 7f58c4c00640 1 --2- 172.21.15.42:0/3816179110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58c00b2a40 0x7f58c01f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.167+0000 7f58c4c00640 1 -- 172.21.15.42:0/3816179110 >> 172.21.15.42:0/3816179110 conn(0x7f58c00d07c0 msgr2=0x7f58c014b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.167+0000 7f58c4c00640 1 -- 172.21.15.42:0/3816179110 shutdown_connections 2024-05-28T14:13:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.167+0000 7f58c4c00640 1 -- 172.21.15.42:0/3816179110 wait complete. 2024-05-28T14:13:02.175 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-28T14:13:02.176 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-28T14:13:02.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.343+0000 7fd63a400640 1 Processor -- start 2024-05-28T14:13:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.343+0000 7fd63a400640 1 -- start start 2024-05-28T14:13:02.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.343+0000 7fd63a400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd63406d110 0x7fd63406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:02.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.343+0000 7fd63a400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63415fb20 con 0x7fd63415f710 2024-05-28T14:13:02.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.343+0000 7fd63a400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63406f190 con 0x7fd6340b02e0 2024-05-28T14:13:02.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.343+0000 7fd63a400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd63406f470 con 0x7fd63406d110 2024-05-28T14:13:02.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.343+0000 7fd62aa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd63406d110 0x7fd63406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:02.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.343+0000 7fd633e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd63415f710 0x7fd634154da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37158/0 (socket says 172.21.15.42:37158) 2024-05-28T14:13:02.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.343+0000 7fd633e00640 1 -- 172.21.15.42:0/1022383867 learned_addr learned my addr 172.21.15.42:0/1022383867 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:13:02.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.343+0000 7fd62aa00640 1 -- 172.21.15.42:0/1022383867 >> v1:172.21.15.63:6789/0 conn(0x7fd6340b02e0 legacy=0x7fd63406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:02.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.343+0000 7fd62aa00640 1 -- 172.21.15.42:0/1022383867 >> v1:172.21.15.42:6789/0 conn(0x7fd63415f710 legacy=0x7fd634154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:02.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.343+0000 7fd62aa00640 1 -- 172.21.15.42:0/1022383867 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63406f750 con 0x7fd63406d110 2024-05-28T14:13:02.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.347+0000 7fd62aa00640 1 --2- 172.21.15.42:0/1022383867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd63406d110 0x7fd63406eb00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0097f0 tx=0x7fd61c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae98513869b3875 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:02.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.347+0000 7fd632a00640 1 -- 172.21.15.42:0/1022383867 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0b1070 con 0x7fd63406d110 2024-05-28T14:13:02.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.347+0000 7fd632a00640 1 -- 172.21.15.42:0/1022383867 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61c0ac600 con 0x7fd63406d110 2024-05-28T14:13:02.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.347+0000 7fd632a00640 1 -- 172.21.15.42:0/1022383867 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0ac9d0 con 0x7fd63406d110 2024-05-28T14:13:02.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.347+0000 7fd63a400640 1 -- 172.21.15.42:0/1022383867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd63406d110 msgr2=0x7fd63406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:02.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.347+0000 7fd63a400640 1 --2- 172.21.15.42:0/1022383867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd63406d110 0x7fd63406eb00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0097f0 tx=0x7fd61c01c470 comp rx=0 tx=0).stop 2024-05-28T14:13:02.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.347+0000 7fd63a400640 1 -- 172.21.15.42:0/1022383867 shutdown_connections 2024-05-28T14:13:02.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.347+0000 7fd63a400640 1 --2- 172.21.15.42:0/1022383867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd63406d110 0x7fd63406eb00 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:02.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.347+0000 7fd63a400640 1 -- 172.21.15.42:0/1022383867 >> 172.21.15.42:0/1022383867 conn(0x7fd634072600 msgr2=0x7fd634072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:02.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.347+0000 7fd63a400640 1 -- 172.21.15.42:0/1022383867 shutdown_connections 2024-05-28T14:13:02.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.347+0000 7fd63a400640 1 -- 172.21.15.42:0/1022383867 wait complete. 2024-05-28T14:13:02.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.347+0000 7fd63a400640 1 Processor -- start 2024-05-28T14:13:02.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.347+0000 7fd63a400640 1 -- start start 2024-05-28T14:13:02.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd63a400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63406d110 0x7fd63414b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd63a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6340b02e0 0x7fd6340ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd63a400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd63415f710 0x7fd6340cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd63a400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd63416c9c0 con 0x7fd6340b02e0 2024-05-28T14:13:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd63a400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd63416c840 con 0x7fd63406d110 2024-05-28T14:13:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd63a400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd63416cb40 con 0x7fd63415f710 2024-05-28T14:13:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd62aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6340b02e0 0x7fd6340ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd62aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6340b02e0 0x7fd6340ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33866/0 (socket says 172.21.15.42:33866) 2024-05-28T14:13:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd62aa00640 1 -- 172.21.15.42:0/3025095396 learned_addr learned my addr 172.21.15.42:0/3025095396 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd633e00640 1 --2- 172.21.15.42:0/3025095396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd63415f710 0x7fd6340cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:02.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd633400640 1 --2- 172.21.15.42:0/3025095396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63406d110 0x7fd63414b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:02.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd62aa00640 1 -- 172.21.15.42:0/3025095396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd63415f710 msgr2=0x7fd6340cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:02.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd62aa00640 1 --2- 172.21.15.42:0/3025095396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd63415f710 0x7fd6340cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:02.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd62aa00640 1 -- 172.21.15.42:0/3025095396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63406d110 msgr2=0x7fd63414b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:02.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd62aa00640 1 --2- 172.21.15.42:0/3025095396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63406d110 0x7fd63414b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:02.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd62aa00640 1 -- 172.21.15.42:0/3025095396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6340cd6d0 con 0x7fd6340b02e0 2024-05-28T14:13:02.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd62aa00640 1 --2- 172.21.15.42:0/3025095396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6340b02e0 0x7fd6340ce930 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fd61c01a0a0 tx=0x7fd61c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:02.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd62be00640 1 -- 172.21.15.42:0/3025095396 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0b1040 con 0x7fd6340b02e0 2024-05-28T14:13:02.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd62be00640 1 -- 172.21.15.42:0/3025095396 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61c0bc070 con 0x7fd6340b02e0 2024-05-28T14:13:02.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd62be00640 1 -- 172.21.15.42:0/3025095396 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0b7890 con 0x7fd6340b02e0 2024-05-28T14:13:02.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd63a400640 1 -- 172.21.15.42:0/3025095396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6340cd970 con 0x7fd6340b02e0 2024-05-28T14:13:02.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd63a400640 1 -- 172.21.15.42:0/3025095396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6340cdf30 con 0x7fd6340b02e0 2024-05-28T14:13:02.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.351+0000 7fd63a400640 1 -- 172.21.15.42:0/3025095396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd634073f40 con 0x7fd6340b02e0 2024-05-28T14:13:02.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.355+0000 7fd62be00640 1 -- 172.21.15.42:0/3025095396 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fd61c0ac4a0 con 0x7fd6340b02e0 2024-05-28T14:13:02.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.355+0000 7fd62be00640 1 --2- 172.21.15.42:0/3025095396 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd610069ba0 0x7fd61006c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:02.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.355+0000 7fd62be00640 1 -- 172.21.15.42:0/3025095396 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7fd61c11fbe0 con 0x7fd6340b02e0 2024-05-28T14:13:02.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.355+0000 7fd633400640 1 --2- 172.21.15.42:0/3025095396 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd610069ba0 0x7fd61006c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:02.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.355+0000 7fd633400640 1 --2- 172.21.15.42:0/3025095396 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd610069ba0 0x7fd61006c060 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd6240012f0 tx=0x7fd624009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:02.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.359+0000 7fd62be00640 1 -- 172.21.15.42:0/3025095396 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd61c016090 con 0x7fd6340b02e0 2024-05-28T14:13:02.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:02.559+0000 7fd63a400640 1 -- 172.21.15.42:0/3025095396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd6340d0900 con 0x7fd6340b02e0 2024-05-28T14:13:03.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.163+0000 7fd62be00640 1 -- 172.21.15.42:0/3025095396 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) ==== 122+0+0 (secure 0 0 0) 0x7fd61c0f1c80 con 0x7fd6340b02e0 2024-05-28T14:13:03.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.167+0000 7fd63a400640 1 -- 172.21.15.42:0/3025095396 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd610069ba0 msgr2=0x7fd61006c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.167+0000 7fd63a400640 1 --2- 172.21.15.42:0/3025095396 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd610069ba0 0x7fd61006c060 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd6240012f0 tx=0x7fd624009040 comp rx=0 tx=0).stop 2024-05-28T14:13:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.167+0000 7fd63a400640 1 -- 172.21.15.42:0/3025095396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6340b02e0 msgr2=0x7fd6340ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.167+0000 7fd63a400640 1 --2- 172.21.15.42:0/3025095396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6340b02e0 0x7fd6340ce930 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fd61c01a0a0 tx=0x7fd61c01acb0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.167+0000 7fd63a400640 1 -- 172.21.15.42:0/3025095396 shutdown_connections 2024-05-28T14:13:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.167+0000 7fd63a400640 1 --2- 172.21.15.42:0/3025095396 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd610069ba0 0x7fd61006c060 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.167+0000 7fd63a400640 1 --2- 172.21.15.42:0/3025095396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd63415f710 0x7fd6340cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.167+0000 7fd63a400640 1 --2- 172.21.15.42:0/3025095396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6340b02e0 0x7fd6340ce930 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.167+0000 7fd63a400640 1 --2- 172.21.15.42:0/3025095396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63406d110 0x7fd63414b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.167+0000 7fd63a400640 1 -- 172.21.15.42:0/3025095396 >> 172.21.15.42:0/3025095396 conn(0x7fd634072600 msgr2=0x7fd6341615f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.167+0000 7fd63a400640 1 -- 172.21.15.42:0/3025095396 shutdown_connections 2024-05-28T14:13:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.167+0000 7fd63a400640 1 -- 172.21.15.42:0/3025095396 wait complete. 2024-05-28T14:13:03.174 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-28T14:13:03.174 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:13:03.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b5d400640 1 Processor -- start 2024-05-28T14:13:03.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b5d400640 1 -- start start 2024-05-28T14:13:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b5d400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1b5806f660 0x7f1b58073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b5d400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1b580d0e00 0x7f1b5806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b5d400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b5815f280 con 0x7f1b5815ba40 2024-05-28T14:13:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b5d400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1b5815f030 con 0x7f1b5806f660 2024-05-28T14:13:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b5d400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1b5815f560 con 0x7f1b580d0e00 2024-05-28T14:13:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b55e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1b5815ba40 0x7f1b580d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37170/0 (socket says 172.21.15.42:37170) 2024-05-28T14:13:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b55e00640 1 -- 172.21.15.42:0/361690551 learned_addr learned my addr 172.21.15.42:0/361690551 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:13:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b56800640 1 --2- 172.21.15.42:0/361690551 >> v2:172.21.15.42:3300/0 conn(0x7f1b5806f660 0x7f1b58073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b57200640 1 --2- 172.21.15.42:0/361690551 >> v2:172.21.15.63:3300/0 conn(0x7f1b580d0e00 0x7f1b5806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b55400640 1 -- 172.21.15.42:0/361690551 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1090512556 0 0) 0x7f1b5815f280 con 0x7f1b5815ba40 2024-05-28T14:13:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b55400640 1 -- 172.21.15.42:0/361690551 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b3c005740 con 0x7f1b5815ba40 2024-05-28T14:13:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b56800640 1 -- 172.21.15.42:0/361690551 >> v2:172.21.15.63:3300/0 conn(0x7f1b580d0e00 msgr2=0x7f1b5806dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b56800640 1 --2- 172.21.15.42:0/361690551 >> v2:172.21.15.63:3300/0 conn(0x7f1b580d0e00 0x7f1b5806dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b56800640 1 -- 172.21.15.42:0/361690551 >> v1:172.21.15.42:6789/0 conn(0x7f1b5815ba40 legacy=0x7f1b580d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.335+0000 7f1b56800640 1 -- 172.21.15.42:0/361690551 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5815f840 con 0x7f1b5806f660 2024-05-28T14:13:03.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.339+0000 7f1b56800640 1 --2- 172.21.15.42:0/361690551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5806f660 0x7f1b58073980 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f1b5815cb20 tx=0x7f1b4801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ee8682d3c9f904 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:03.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.339+0000 7f1b55400640 1 -- 172.21.15.42:0/361690551 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b480b1070 con 0x7f1b5806f660 2024-05-28T14:13:03.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.339+0000 7f1b55400640 1 -- 172.21.15.42:0/361690551 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b480ac640 con 0x7f1b5806f660 2024-05-28T14:13:03.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.339+0000 7f1b55400640 1 -- 172.21.15.42:0/361690551 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b480aca10 con 0x7f1b5806f660 2024-05-28T14:13:03.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.339+0000 7f1b5d400640 1 -- 172.21.15.42:0/361690551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5806f660 msgr2=0x7f1b58073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.339+0000 7f1b5d400640 1 --2- 172.21.15.42:0/361690551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5806f660 0x7f1b58073980 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f1b5815cb20 tx=0x7f1b4801c330 comp rx=0 tx=0).stop 2024-05-28T14:13:03.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.339+0000 7f1b5d400640 1 -- 172.21.15.42:0/361690551 shutdown_connections 2024-05-28T14:13:03.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.339+0000 7f1b5d400640 1 --2- 172.21.15.42:0/361690551 >> v2:172.21.15.63:3300/0 conn(0x7f1b580d0e00 0x7f1b5806dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.339+0000 7f1b5d400640 1 --2- 172.21.15.42:0/361690551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5806f660 0x7f1b58073980 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.339+0000 7f1b5d400640 1 -- 172.21.15.42:0/361690551 >> 172.21.15.42:0/361690551 conn(0x7f1b5806bdf0 msgr2=0x7f1b5806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:03.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.339+0000 7f1b5d400640 1 -- 172.21.15.42:0/361690551 shutdown_connections 2024-05-28T14:13:03.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.339+0000 7f1b5d400640 1 -- 172.21.15.42:0/361690551 wait complete. 2024-05-28T14:13:03.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b5d400640 1 Processor -- start 2024-05-28T14:13:03.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b5d400640 1 -- start start 2024-05-28T14:13:03.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b5d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5806f660 0x7f1b581fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:03.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b5d400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b580d0e00 0x7f1b580b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:03.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b5d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5815a6a0 0x7f1b581f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:03.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b5d400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1b580b3460 con 0x7f1b5815a6a0 2024-05-28T14:13:03.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b5d400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b580b32e0 con 0x7f1b5806f660 2024-05-28T14:13:03.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b5d400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1b580b35e0 con 0x7f1b580d0e00 2024-05-28T14:13:03.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b57200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5815a6a0 0x7f1b581f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:03.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b57200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5815a6a0 0x7f1b581f7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33890/0 (socket says 172.21.15.42:33890) 2024-05-28T14:13:03.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b57200640 1 -- 172.21.15.42:0/1219016463 learned_addr learned my addr 172.21.15.42:0/1219016463 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:03.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b56800640 1 --2- 172.21.15.42:0/1219016463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5806f660 0x7f1b581fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:03.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b55e00640 1 --2- 172.21.15.42:0/1219016463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b580d0e00 0x7f1b580b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:03.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b57200640 1 -- 172.21.15.42:0/1219016463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b580d0e00 msgr2=0x7f1b580b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b57200640 1 --2- 172.21.15.42:0/1219016463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b580d0e00 0x7f1b580b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b57200640 1 -- 172.21.15.42:0/1219016463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5806f660 msgr2=0x7f1b581fbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b57200640 1 --2- 172.21.15.42:0/1219016463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5806f660 0x7f1b581fbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b57200640 1 -- 172.21.15.42:0/1219016463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b58151fc0 con 0x7f1b5815a6a0 2024-05-28T14:13:03.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b57200640 1 --2- 172.21.15.42:0/1219016463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5815a6a0 0x7f1b581f7180 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000efc0 tx=0x7f1b5000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:03.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b4ea00640 1 -- 172.21.15.42:0/1219016463 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50018020 con 0x7f1b5815a6a0 2024-05-28T14:13:03.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b4ea00640 1 -- 172.21.15.42:0/1219016463 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b50014480 con 0x7f1b5815a6a0 2024-05-28T14:13:03.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b4ea00640 1 -- 172.21.15.42:0/1219016463 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50014870 con 0x7f1b5815a6a0 2024-05-28T14:13:03.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b5d400640 1 -- 172.21.15.42:0/1219016463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b581522a0 con 0x7f1b5815a6a0 2024-05-28T14:13:03.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b5d400640 1 -- 172.21.15.42:0/1219016463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b581fb730 con 0x7f1b5815a6a0 2024-05-28T14:13:03.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.343+0000 7f1b5d400640 1 -- 172.21.15.42:0/1219016463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b58068a90 con 0x7f1b5815a6a0 2024-05-28T14:13:03.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.347+0000 7f1b4ea00640 1 -- 172.21.15.42:0/1219016463 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f1b5001b020 con 0x7f1b5815a6a0 2024-05-28T14:13:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.347+0000 7f1b4ea00640 1 --2- 172.21.15.42:0/1219016463 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b38069b20 0x7f1b3806bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.347+0000 7f1b4ea00640 1 -- 172.21.15.42:0/1219016463 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f1b50086470 con 0x7f1b5815a6a0 2024-05-28T14:13:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.347+0000 7f1b56800640 1 --2- 172.21.15.42:0/1219016463 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b38069b20 0x7f1b3806bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.347+0000 7f1b56800640 1 --2- 172.21.15.42:0/1219016463 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b38069b20 0x7f1b3806bfe0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1b480ae000 tx=0x7f1b480a6070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:03.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.351+0000 7f1b4ea00640 1 -- 172.21.15.42:0/1219016463 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1b50022030 con 0x7f1b5815a6a0 2024-05-28T14:13:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.551+0000 7f1b5d400640 1 -- 172.21.15.42:0/1219016463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b581661a0 con 0x7f1b5815a6a0 2024-05-28T14:13:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.551+0000 7f1b4ea00640 1 -- 172.21.15.42:0/1219016463 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5180 (secure 0 0 0) 0x7f1b50010040 con 0x7f1b5815a6a0 2024-05-28T14:13:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:03.538 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":74,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/56330151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":56330151},{"type":"v1","addr":"172.21.15.42:6835","nonce":56330151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":9183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/192920662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":192920662},{"type":"v1","addr":"172.21.15.63:6835","nonce":192920662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":57437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1225711200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1225711200},{"type":"v1","addr":"172.21.15.42:6837","nonce":1225711200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:12:59.544555+0000","modified":"2024-05-28T14:13:02.565331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57440":{"gid":57440,"name":"b","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/3959626682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3959626682},{"type":"v1","addr":"172.21.15.63:6834","nonce":3959626682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57440,"qdb_cluster":[57440]},"id":8}]} 2024-05-28T14:13:03.538 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 74 2024-05-28T14:13:03.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.555+0000 7f1b5d400640 1 -- 172.21.15.42:0/1219016463 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b38069b20 msgr2=0x7f1b3806bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.555+0000 7f1b5d400640 1 --2- 172.21.15.42:0/1219016463 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b38069b20 0x7f1b3806bfe0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1b480ae000 tx=0x7f1b480a6070 comp rx=0 tx=0).stop 2024-05-28T14:13:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.555+0000 7f1b5d400640 1 -- 172.21.15.42:0/1219016463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5815a6a0 msgr2=0x7f1b581f7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.555+0000 7f1b5d400640 1 --2- 172.21.15.42:0/1219016463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5815a6a0 0x7f1b581f7180 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000efc0 tx=0x7f1b5000c490 comp rx=0 tx=0).stop 2024-05-28T14:13:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.555+0000 7f1b5d400640 1 -- 172.21.15.42:0/1219016463 shutdown_connections 2024-05-28T14:13:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.555+0000 7f1b5d400640 1 --2- 172.21.15.42:0/1219016463 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b38069b20 0x7f1b3806bfe0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.555+0000 7f1b5d400640 1 --2- 172.21.15.42:0/1219016463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5815a6a0 0x7f1b581f7180 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.555+0000 7f1b5d400640 1 --2- 172.21.15.42:0/1219016463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b580d0e00 0x7f1b580b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.555+0000 7f1b5d400640 1 --2- 172.21.15.42:0/1219016463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5806f660 0x7f1b581fbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.555+0000 7f1b5d400640 1 -- 172.21.15.42:0/1219016463 >> 172.21.15.42:0/1219016463 conn(0x7f1b5806bdf0 msgr2=0x7f1b580d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:03.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.555+0000 7f1b5d400640 1 -- 172.21.15.42:0/1219016463 shutdown_connections 2024-05-28T14:13:03.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.555+0000 7f1b5d400640 1 -- 172.21.15.42:0/1219016463 wait complete. 2024-05-28T14:13:03.560 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:13:03.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.715+0000 7fedcee00640 1 Processor -- start 2024-05-28T14:13:03.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.715+0000 7fedcee00640 1 -- start start 2024-05-28T14:13:03.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.719+0000 7fedcee00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fedc80d0e00 0x7fedc81663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:03.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.719+0000 7fedcee00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedc815f280 con 0x7fedc806f660 2024-05-28T14:13:03.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.719+0000 7fedcee00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedc815f560 con 0x7fedc815ba40 2024-05-28T14:13:03.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.719+0000 7fedcee00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fedc815f030 con 0x7fedc80d0e00 2024-05-28T14:13:03.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.719+0000 7fedc7e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fedc806f660 0x7fedc8073980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37186/0 (socket says 172.21.15.42:37186) 2024-05-28T14:13:03.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.719+0000 7fedc7e00640 1 -- 172.21.15.42:0/3776556228 learned_addr learned my addr 172.21.15.42:0/3776556228 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:13:03.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.719+0000 7fedcce00640 1 --2- 172.21.15.42:0/3776556228 >> v2:172.21.15.63:3300/0 conn(0x7fedc80d0e00 0x7fedc81663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:03.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.719+0000 7fedc7400640 1 -- 172.21.15.42:0/3776556228 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1398711703 0 0) 0x7fedc815f280 con 0x7fedc806f660 2024-05-28T14:13:03.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.719+0000 7fedc7400640 1 -- 172.21.15.42:0/3776556228 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedac003530 con 0x7fedc806f660 2024-05-28T14:13:03.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.719+0000 7fedcce00640 1 -- 172.21.15.42:0/3776556228 >> v1:172.21.15.63:6789/0 conn(0x7fedc815ba40 legacy=0x7fedc80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.719+0000 7fedcce00640 1 -- 172.21.15.42:0/3776556228 >> v1:172.21.15.42:6789/0 conn(0x7fedc806f660 legacy=0x7fedc8073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.719+0000 7fedcce00640 1 -- 172.21.15.42:0/3776556228 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc815f840 con 0x7fedc80d0e00 2024-05-28T14:13:03.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.723+0000 7fedcce00640 1 --2- 172.21.15.42:0/3776556228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedc80d0e00 0x7fedc81663c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009d90 tx=0x7fedc001c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=375698584efe0f25 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:03.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.723+0000 7fedc7400640 1 -- 172.21.15.42:0/3776556228 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc00b2070 con 0x7fedc80d0e00 2024-05-28T14:13:03.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.723+0000 7fedc7400640 1 -- 172.21.15.42:0/3776556228 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc00a5900 con 0x7fedc80d0e00 2024-05-28T14:13:03.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.723+0000 7fedc7400640 1 -- 172.21.15.42:0/3776556228 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc00a5cb0 con 0x7fedc80d0e00 2024-05-28T14:13:03.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.723+0000 7fedcee00640 1 -- 172.21.15.42:0/3776556228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedc80d0e00 msgr2=0x7fedc81663c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.723+0000 7fedcee00640 1 --2- 172.21.15.42:0/3776556228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedc80d0e00 0x7fedc81663c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009d90 tx=0x7fedc001c820 comp rx=0 tx=0).stop 2024-05-28T14:13:03.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.723+0000 7fedcee00640 1 -- 172.21.15.42:0/3776556228 shutdown_connections 2024-05-28T14:13:03.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.723+0000 7fedcee00640 1 --2- 172.21.15.42:0/3776556228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedc80d0e00 0x7fedc81663c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.723+0000 7fedcee00640 1 -- 172.21.15.42:0/3776556228 >> 172.21.15.42:0/3776556228 conn(0x7fedc806bdf0 msgr2=0x7fedc806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:03.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.723+0000 7fedcee00640 1 -- 172.21.15.42:0/3776556228 shutdown_connections 2024-05-28T14:13:03.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.723+0000 7fedcee00640 1 -- 172.21.15.42:0/3776556228 wait complete. 2024-05-28T14:13:03.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.723+0000 7fedcee00640 1 Processor -- start 2024-05-28T14:13:03.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.723+0000 7fedcee00640 1 -- start start 2024-05-28T14:13:03.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcee00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fedc806f660 0x7fedc814d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:03.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcee00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedc80d0e00 0x7fedc809a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:03.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcee00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedc809a810 0x7fedc809c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:03.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcee00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fedc81698c0 con 0x7fedc809a810 2024-05-28T14:13:03.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcee00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fedc8169740 con 0x7fedc80d0e00 2024-05-28T14:13:03.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcee00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fedc8169a40 con 0x7fedc806f660 2024-05-28T14:13:03.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedc809a810 0x7fedc809c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:03.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedc809a810 0x7fedc809c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33896/0 (socket says 172.21.15.42:33896) 2024-05-28T14:13:03.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcce00640 1 -- 172.21.15.42:0/287635724 learned_addr learned my addr 172.21.15.42:0/287635724 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:03.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedbf400640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedc80d0e00 0x7fedc809a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:03.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedc7e00640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fedc806f660 0x7fedc814d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:03.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcce00640 1 -- 172.21.15.42:0/287635724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fedc806f660 msgr2=0x7fedc814d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcce00640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fedc806f660 0x7fedc814d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcce00640 1 -- 172.21.15.42:0/287635724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedc80d0e00 msgr2=0x7fedc809a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcce00640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedc80d0e00 0x7fedc809a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcce00640 1 -- 172.21.15.42:0/287635724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc809d4c0 con 0x7fedc809a810 2024-05-28T14:13:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedbf400640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedc80d0e00 0x7fedc809a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcce00640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedc809a810 0x7fedc809c680 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fedc00b0040 tx=0x7fedc00195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedc4c00640 1 -- 172.21.15.42:0/287635724 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc00b2050 con 0x7fedc809a810 2024-05-28T14:13:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedc4c00640 1 -- 172.21.15.42:0/287635724 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc00b3040 con 0x7fedc809a810 2024-05-28T14:13:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcee00640 1 -- 172.21.15.42:0/287635724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc809ae90 con 0x7fedc809a810 2024-05-28T14:13:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedc4c00640 1 -- 172.21.15.42:0/287635724 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc00ad870 con 0x7fedc809a810 2024-05-28T14:13:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcee00640 1 -- 172.21.15.42:0/287635724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc809b450 con 0x7fedc809a810 2024-05-28T14:13:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.727+0000 7fedcee00640 1 -- 172.21.15.42:0/287635724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc8068a90 con 0x7fedc809a810 2024-05-28T14:13:03.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.731+0000 7fedc4c00640 1 -- 172.21.15.42:0/287635724 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fedc00c0020 con 0x7fedc809a810 2024-05-28T14:13:03.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.731+0000 7fedc4c00640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feda4069b20 0x7feda406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:03.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.731+0000 7fedc4c00640 1 -- 172.21.15.42:0/287635724 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7fedc0120100 con 0x7fedc809a810 2024-05-28T14:13:03.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.731+0000 7fedc7e00640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feda4069b20 0x7feda406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:03.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.731+0000 7fedc7e00640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feda4069b20 0x7feda406bfe0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fedb80011f0 tx=0x7fedb8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:03.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.735+0000 7fedc4c00640 1 -- 172.21.15.42:0/287635724 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fedc00c7020 con 0x7fedc809a810 2024-05-28T14:13:03.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.899+0000 7fedcee00640 1 -- 172.21.15.42:0/287635724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedc815e070 con 0x7fedc809a810 2024-05-28T14:13:03.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.903+0000 7fedc4c00640 1 -- 172.21.15.42:0/287635724 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16655 (secure 0 0 0) 0x7fedc0016560 con 0x7fedc809a810 2024-05-28T14:13:03.887 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:03.887 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":117,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:13:01.136209+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:12:57.936312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-28T14:12:58.495752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:13:03.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.903+0000 7fedcee00640 1 -- 172.21.15.42:0/287635724 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feda4069b20 msgr2=0x7feda406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.903+0000 7fedcee00640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feda4069b20 0x7feda406bfe0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fedb80011f0 tx=0x7fedb8009040 comp rx=0 tx=0).stop 2024-05-28T14:13:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.903+0000 7fedcee00640 1 -- 172.21.15.42:0/287635724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedc809a810 msgr2=0x7fedc809c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.903+0000 7fedcee00640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedc809a810 0x7fedc809c680 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fedc00b0040 tx=0x7fedc00195c0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.903+0000 7fedcee00640 1 -- 172.21.15.42:0/287635724 shutdown_connections 2024-05-28T14:13:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.903+0000 7fedcee00640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feda4069b20 0x7feda406bfe0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.903+0000 7fedcee00640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fedc809a810 0x7fedc809c680 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.903+0000 7fedcee00640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedc80d0e00 0x7fedc809a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.903+0000 7fedcee00640 1 --2- 172.21.15.42:0/287635724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fedc806f660 0x7fedc814d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.903+0000 7fedcee00640 1 -- 172.21.15.42:0/287635724 >> 172.21.15.42:0/287635724 conn(0x7fedc806bdf0 msgr2=0x7fedc80d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.903+0000 7fedcee00640 1 -- 172.21.15.42:0/287635724 shutdown_connections 2024-05-28T14:13:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:03.903+0000 7fedcee00640 1 -- 172.21.15.42:0/287635724 wait complete. 2024-05-28T14:13:03.910 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-28T14:13:03.910 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:13:04.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fcd00800640 1 Processor -- start 2024-05-28T14:13:04.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fcd00800640 1 -- start start 2024-05-28T14:13:04.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fcd00800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fccf809b160 0x7fccf8167730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fcd00800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf80d0870 con 0x7fccf809bbd0 2024-05-28T14:13:04.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fcd00800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf80cd300 con 0x7fccf815c990 2024-05-28T14:13:04.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fcd00800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fccf815cd70 con 0x7fccf809b160 2024-05-28T14:13:04.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfd400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fccf809bbd0 0x7fccf809aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37202/0 (socket says 172.21.15.42:37202) 2024-05-28T14:13:04.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfd400640 1 -- 172.21.15.42:0/1056781985 learned_addr learned my addr 172.21.15.42:0/1056781985 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:13:04.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfe800640 1 --2- 172.21.15.42:0/1056781985 >> v2:172.21.15.63:3300/0 conn(0x7fccf809b160 0x7fccf8167730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3694396212 0 0) 0x7fccf80d0870 con 0x7fccf809bbd0 2024-05-28T14:13:04.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcce0003530 con 0x7fccf809bbd0 2024-05-28T14:13:04.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189260943 0 0) 0x7fccf80cd300 con 0x7fccf815c990 2024-05-28T14:13:04.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccf80d0870 con 0x7fccf815c990 2024-05-28T14:13:04.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 682588402 0 0) 0x7fcce0003530 con 0x7fccf809bbd0 2024-05-28T14:13:04.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 >> v2:172.21.15.63:3300/0 conn(0x7fccf809b160 msgr2=0x7fccf8167730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:04.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 --2- 172.21.15.42:0/1056781985 >> v2:172.21.15.63:3300/0 conn(0x7fccf809b160 0x7fccf8167730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 >> v1:172.21.15.63:6789/0 conn(0x7fccf815c990 legacy=0x7fccf80d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf80cd5e0 con 0x7fccf809bbd0 2024-05-28T14:13:04.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fcce8001680 con 0x7fccf809bbd0 2024-05-28T14:13:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 >> v1:172.21.15.42:6789/0 conn(0x7fccf809bbd0 legacy=0x7fccf809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 --2- 172.21.15.42:0/1056781985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcce0003b50 0x7fcce0003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 --2- 172.21.15.42:0/1056781985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcce0004810 0x7fcce0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 --2- 172.21.15.42:0/1056781985 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcce00051d0 0x7fcce000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcce000c020 con 0x7fcce0003b50 2024-05-28T14:13:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcce000c1d0 con 0x7fcce0004810 2024-05-28T14:13:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcce000c3d0 con 0x7fcce00051d0 2024-05-28T14:13:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfde00640 1 --2- 172.21.15.42:0/1056781985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcce0003b50 0x7fcce0003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfe800640 1 --2- 172.21.15.42:0/1056781985 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcce00051d0 0x7fcce000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.067+0000 7fccfd400640 1 --2- 172.21.15.42:0/1056781985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcce0004810 0x7fcce0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fccfde00640 1 -- 172.21.15.42:0/1056781985 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcce00051d0 msgr2=0x7fcce000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fccfde00640 1 --2- 172.21.15.42:0/1056781985 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcce00051d0 0x7fcce000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fccfde00640 1 -- 172.21.15.42:0/1056781985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcce0004810 msgr2=0x7fcce0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fccfde00640 1 --2- 172.21.15.42:0/1056781985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcce0004810 0x7fcce0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fccfde00640 1 -- 172.21.15.42:0/1056781985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce0003530 con 0x7fcce0003b50 2024-05-28T14:13:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fccfe800640 1 -- 172.21.15.42:0/1056781985 reap_dead start 2024-05-28T14:13:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fccfde00640 1 --2- 172.21.15.42:0/1056781985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcce0003b50 0x7fcce0003f50 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fccec0018d0 tx=0x7fccec01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63cda86b16cb616 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec001520 con 0x7fcce0003b50 2024-05-28T14:13:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccec0052f0 con 0x7fcce0003b50 2024-05-28T14:13:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fccfca00640 1 -- 172.21.15.42:0/1056781985 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec003840 con 0x7fcce0003b50 2024-05-28T14:13:04.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fcd00800640 1 -- 172.21.15.42:0/1056781985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcce0003b50 msgr2=0x7fcce0003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fcd00800640 1 --2- 172.21.15.42:0/1056781985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcce0003b50 0x7fcce0003f50 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fccec0018d0 tx=0x7fccec01c510 comp rx=0 tx=0).stop 2024-05-28T14:13:04.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fcd00800640 1 -- 172.21.15.42:0/1056781985 shutdown_connections 2024-05-28T14:13:04.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fcd00800640 1 --2- 172.21.15.42:0/1056781985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcce0003b50 0x7fcce0003f50 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fcd00800640 1 -- 172.21.15.42:0/1056781985 >> 172.21.15.42:0/1056781985 conn(0x7fccf80b4080 msgr2=0x7fccf80b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:04.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.071+0000 7fcd00800640 1 -- 172.21.15.42:0/1056781985 shutdown_connections 2024-05-28T14:13:04.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fcd00800640 1 -- 172.21.15.42:0/1056781985 wait complete. 2024-05-28T14:13:04.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fcd00800640 1 Processor -- start 2024-05-28T14:13:04.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fcd00800640 1 -- start start 2024-05-28T14:13:04.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fcd00800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcce0003b50 0x7fccf81fb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fcd00800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccf8150b00 0x7fccf80b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fcd00800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf80b6ab0 0x7fccf815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fcd00800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fccf809afe0 con 0x7fccf80b6ab0 2024-05-28T14:13:04.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fcd00800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fccf809ae60 con 0x7fccf8150b00 2024-05-28T14:13:04.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fcd00800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fccf80d0570 con 0x7fcce0003b50 2024-05-28T14:13:04.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccfe800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf80b6ab0 0x7fccf815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccfe800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf80b6ab0 0x7fccf815e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33922/0 (socket says 172.21.15.42:33922) 2024-05-28T14:13:04.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccfe800640 1 -- 172.21.15.42:0/2000922064 learned_addr learned my addr 172.21.15.42:0/2000922064 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:04.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccfde00640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcce0003b50 0x7fccf81fb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccfd400640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccf8150b00 0x7fccf80b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccfe800640 1 -- 172.21.15.42:0/2000922064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcce0003b50 msgr2=0x7fccf81fb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccfe800640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcce0003b50 0x7fccf81fb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccfe800640 1 -- 172.21.15.42:0/2000922064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccf8150b00 msgr2=0x7fccf80b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccfe800640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccf8150b00 0x7fccf80b6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccfe800640 1 -- 172.21.15.42:0/2000922064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf80b6f80 con 0x7fccf80b6ab0 2024-05-28T14:13:04.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccfd400640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccf8150b00 0x7fccf80b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:04.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccfe800640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf80b6ab0 0x7fccf815e0e0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fccf8073630 tx=0x7fccf400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:04.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccf2000640 1 -- 172.21.15.42:0/2000922064 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4019070 con 0x7fccf80b6ab0 2024-05-28T14:13:04.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccf2000640 1 -- 172.21.15.42:0/2000922064 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf4011e90 con 0x7fccf80b6ab0 2024-05-28T14:13:04.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fccf2000640 1 -- 172.21.15.42:0/2000922064 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4003be0 con 0x7fccf80b6ab0 2024-05-28T14:13:04.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.075+0000 7fcd00800640 1 -- 172.21.15.42:0/2000922064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf80b7260 con 0x7fccf80b6ab0 2024-05-28T14:13:04.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.079+0000 7fcd00800640 1 -- 172.21.15.42:0/2000922064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf81faec0 con 0x7fccf80b6ab0 2024-05-28T14:13:04.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.079+0000 7fcd00800640 1 -- 172.21.15.42:0/2000922064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf8068a90 con 0x7fccf80b6ab0 2024-05-28T14:13:04.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.079+0000 7fccf2000640 1 -- 172.21.15.42:0/2000922064 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fccf401c070 con 0x7fccf80b6ab0 2024-05-28T14:13:04.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.079+0000 7fccf2000640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fccdc069b20 0x7fccdc06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.079+0000 7fccf2000640 1 -- 172.21.15.42:0/2000922064 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7fccf4086150 con 0x7fccf80b6ab0 2024-05-28T14:13:04.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.079+0000 7fccfde00640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fccdc069b20 0x7fccdc06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.083+0000 7fccfde00640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fccdc069b20 0x7fccdc06bfe0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fccec0ac000 tx=0x7fccec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:04.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.083+0000 7fccf2000640 1 -- 172.21.15.42:0/2000922064 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fccf4010040 con 0x7fccf80b6ab0 2024-05-28T14:13:04.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.247+0000 7fcd00800640 1 -- 172.21.15.42:0/2000922064 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fccf815fe90 con 0x7fccdc069b20 2024-05-28T14:13:04.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.255+0000 7fccf2000640 1 -- 172.21.15.42:0/2000922064 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247294 (secure 0 0 0) 0x7fccec0f8610 con 0x7fccdc069b20 2024-05-28T14:13:04.243 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:04.244 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:13:04.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.259+0000 7fcd00800640 1 -- 172.21.15.42:0/2000922064 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fccdc069b20 msgr2=0x7fccdc06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.259+0000 7fcd00800640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fccdc069b20 0x7fccdc06bfe0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fccec0ac000 tx=0x7fccec009040 comp rx=0 tx=0).stop 2024-05-28T14:13:04.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.259+0000 7fcd00800640 1 -- 172.21.15.42:0/2000922064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf80b6ab0 msgr2=0x7fccf815e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.259+0000 7fcd00800640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf80b6ab0 0x7fccf815e0e0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fccf8073630 tx=0x7fccf400f580 comp rx=0 tx=0).stop 2024-05-28T14:13:04.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.259+0000 7fcd00800640 1 -- 172.21.15.42:0/2000922064 shutdown_connections 2024-05-28T14:13:04.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.259+0000 7fcd00800640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fccdc069b20 0x7fccdc06bfe0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.259+0000 7fcd00800640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf80b6ab0 0x7fccf815e0e0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.259+0000 7fcd00800640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccf8150b00 0x7fccf80b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.259+0000 7fcd00800640 1 --2- 172.21.15.42:0/2000922064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcce0003b50 0x7fccf81fb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.259+0000 7fcd00800640 1 -- 172.21.15.42:0/2000922064 >> 172.21.15.42:0/2000922064 conn(0x7fccf80b4080 msgr2=0x7fccf80ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:04.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.259+0000 7fcd00800640 1 -- 172.21.15.42:0/2000922064 shutdown_connections 2024-05-28T14:13:04.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.259+0000 7fcd00800640 1 -- 172.21.15.42:0/2000922064 wait complete. 2024-05-28T14:13:04.263 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":557,"stamp":"2024-05-28T14:13:03.164561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":972044,"kb_used_data":12316,"kb_used_omap":12,"kb_used_meta":959603,"kb_avail":754002676,"statfs":{"total":773094113280,"available":772098740224,"internally_reserved":0,"allocated":12611584,"data_stored":6162861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":982634034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.037878"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143716+0000","last_change":"2024-05-28T14:12:59.143716+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143716+0000","last_undegraded":"2024-05-28T14:12:59.143716+0000","last_fullsized":"2024-05-28T14:12:59.143716+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145111+0000","last_change":"2024-05-28T14:12:59.145111+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145111+0000","last_undegraded":"2024-05-28T14:12:59.145111+0000","last_fullsized":"2024-05-28T14:12:59.145111+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.150194+0000","last_change":"2024-05-28T14:12:59.150194+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.150194+0000","last_undegraded":"2024-05-28T14:12:59.150194+0000","last_fullsized":"2024-05-28T14:12:59.150194+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143795+0000","last_change":"2024-05-28T14:12:59.143795+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143795+0000","last_undegraded":"2024-05-28T14:12:59.143795+0000","last_fullsized":"2024-05-28T14:12:59.143795+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.151103+0000","last_change":"2024-05-28T14:13:00.146533+0000","last_active":"2024-05-28T14:13:00.151103+0000","last_peered":"2024-05-28T14:13:00.151103+0000","last_clean":"2024-05-28T14:13:00.151103+0000","last_became_active":"2024-05-28T14:13:00.145774+0000","last_became_peered":"2024-05-28T14:13:00.145774+0000","last_unstale":"2024-05-28T14:13:00.151103+0000","last_undegraded":"2024-05-28T14:13:00.151103+0000","last_fullsized":"2024-05-28T14:13:00.151103+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:29:07.224328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141349+0000","last_change":"2024-05-28T14:12:59.141349+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141349+0000","last_undegraded":"2024-05-28T14:12:59.141349+0000","last_fullsized":"2024-05-28T14:12:59.141349+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143106+0000","last_change":"2024-05-28T14:12:59.143106+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143106+0000","last_undegraded":"2024-05-28T14:12:59.143106+0000","last_fullsized":"2024-05-28T14:12:59.143106+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.147478+0000","last_change":"2024-05-28T14:12:59.147478+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.147478+0000","last_undegraded":"2024-05-28T14:12:59.147478+0000","last_fullsized":"2024-05-28T14:12:59.147478+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143317+0000","last_change":"2024-05-28T14:12:59.143317+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143317+0000","last_undegraded":"2024-05-28T14:12:59.143317+0000","last_fullsized":"2024-05-28T14:12:59.143317+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142610+0000","last_change":"2024-05-28T14:12:59.142610+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142610+0000","last_undegraded":"2024-05-28T14:12:59.142610+0000","last_fullsized":"2024-05-28T14:12:59.142610+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144639+0000","last_change":"2024-05-28T14:12:59.144639+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144639+0000","last_undegraded":"2024-05-28T14:12:59.144639+0000","last_fullsized":"2024-05-28T14:12:59.144639+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143340+0000","last_change":"2024-05-28T14:12:59.143340+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143340+0000","last_undegraded":"2024-05-28T14:12:59.143340+0000","last_fullsized":"2024-05-28T14:12:59.143340+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146701+0000","last_change":"2024-05-28T14:12:59.146701+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146701+0000","last_undegraded":"2024-05-28T14:12:59.146701+0000","last_fullsized":"2024-05-28T14:12:59.146701+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.147196+0000","last_change":"2024-05-28T14:12:59.147196+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.147196+0000","last_undegraded":"2024-05-28T14:12:59.147196+0000","last_fullsized":"2024-05-28T14:12:59.147196+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141425+0000","last_change":"2024-05-28T14:12:59.141425+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141425+0000","last_undegraded":"2024-05-28T14:12:59.141425+0000","last_fullsized":"2024-05-28T14:12:59.141425+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144638+0000","last_change":"2024-05-28T14:12:59.144638+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144638+0000","last_undegraded":"2024-05-28T14:12:59.144638+0000","last_fullsized":"2024-05-28T14:12:59.144638+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144519+0000","last_change":"2024-05-28T14:12:59.144519+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144519+0000","last_undegraded":"2024-05-28T14:12:59.144519+0000","last_fullsized":"2024-05-28T14:12:59.144519+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.152420+0000","last_change":"2024-05-28T14:12:59.152420+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.152420+0000","last_undegraded":"2024-05-28T14:12:59.152420+0000","last_fullsized":"2024-05-28T14:12:59.152420+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.149551+0000","last_change":"2024-05-28T14:12:59.149551+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.149551+0000","last_undegraded":"2024-05-28T14:12:59.149551+0000","last_fullsized":"2024-05-28T14:12:59.149551+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143766+0000","last_change":"2024-05-28T14:12:59.143766+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143766+0000","last_undegraded":"2024-05-28T14:12:59.143766+0000","last_fullsized":"2024-05-28T14:12:59.143766+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142806+0000","last_change":"2024-05-28T14:12:59.142806+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142806+0000","last_undegraded":"2024-05-28T14:12:59.142806+0000","last_fullsized":"2024-05-28T14:12:59.142806+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.147836+0000","last_change":"2024-05-28T14:12:59.147836+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.147836+0000","last_undegraded":"2024-05-28T14:12:59.147836+0000","last_fullsized":"2024-05-28T14:12:59.147836+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142797+0000","last_change":"2024-05-28T14:12:59.142797+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142797+0000","last_undegraded":"2024-05-28T14:12:59.142797+0000","last_fullsized":"2024-05-28T14:12:59.142797+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144378+0000","last_change":"2024-05-28T14:12:59.144378+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144378+0000","last_undegraded":"2024-05-28T14:12:59.144378+0000","last_fullsized":"2024-05-28T14:12:59.144378+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143979+0000","last_change":"2024-05-28T14:12:59.143979+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143979+0000","last_undegraded":"2024-05-28T14:12:59.143979+0000","last_fullsized":"2024-05-28T14:12:59.143979+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142104+0000","last_change":"2024-05-28T14:12:59.142104+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142104+0000","last_undegraded":"2024-05-28T14:12:59.142104+0000","last_fullsized":"2024-05-28T14:12:59.142104+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146411+0000","last_change":"2024-05-28T14:12:59.146411+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146411+0000","last_undegraded":"2024-05-28T14:12:59.146411+0000","last_fullsized":"2024-05-28T14:12:59.146411+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145670+0000","last_change":"2024-05-28T14:12:59.145670+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145670+0000","last_undegraded":"2024-05-28T14:12:59.145670+0000","last_fullsized":"2024-05-28T14:12:59.145670+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.148734+0000","last_change":"2024-05-28T14:12:59.148734+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.148734+0000","last_undegraded":"2024-05-28T14:12:59.148734+0000","last_fullsized":"2024-05-28T14:12:59.148734+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143300+0000","last_change":"2024-05-28T14:12:59.143300+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143300+0000","last_undegraded":"2024-05-28T14:12:59.143300+0000","last_fullsized":"2024-05-28T14:12:59.143300+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141526+0000","last_change":"2024-05-28T14:12:59.141526+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141526+0000","last_undegraded":"2024-05-28T14:12:59.141526+0000","last_fullsized":"2024-05-28T14:12:59.141526+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.140649+0000","last_change":"2024-05-28T14:12:59.140649+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.140649+0000","last_undegraded":"2024-05-28T14:12:59.140649+0000","last_fullsized":"2024-05-28T14:12:59.140649+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145011+0000","last_change":"2024-05-28T14:12:59.145011+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145011+0000","last_undegraded":"2024-05-28T14:12:59.145011+0000","last_fullsized":"2024-05-28T14:12:59.145011+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.158758+0000","last_change":"2024-05-28T14:12:59.156662+0000","last_active":"2024-05-28T14:12:59.158758+0000","last_peered":"2024-05-28T14:12:59.158758+0000","last_clean":"2024-05-28T14:12:59.158758+0000","last_became_active":"2024-05-28T14:12:59.156379+0000","last_became_peered":"2024-05-28T14:12:59.156379+0000","last_unstale":"2024-05-28T14:12:59.158758+0000","last_undegraded":"2024-05-28T14:12:59.158758+0000","last_fullsized":"2024-05-28T14:12:59.158758+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:39:27.979374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.151621+0000","last_change":"2024-05-28T14:12:59.151621+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.151621+0000","last_undegraded":"2024-05-28T14:12:59.151621+0000","last_fullsized":"2024-05-28T14:12:59.151621+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.161663+0000","last_change":"2024-05-28T14:12:59.160363+0000","last_active":"2024-05-28T14:12:59.161663+0000","last_peered":"2024-05-28T14:12:59.161663+0000","last_clean":"2024-05-28T14:12:59.161663+0000","last_became_active":"2024-05-28T14:12:59.159749+0000","last_became_peered":"2024-05-28T14:12:59.159749+0000","last_unstale":"2024-05-28T14:12:59.161663+0000","last_undegraded":"2024-05-28T14:12:59.161663+0000","last_fullsized":"2024-05-28T14:12:59.161663+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:15:00.386236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.150359+0000","last_change":"2024-05-28T14:12:59.150359+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.150359+0000","last_undegraded":"2024-05-28T14:12:59.150359+0000","last_fullsized":"2024-05-28T14:12:59.150359+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.171344+0000","last_change":"2024-05-28T14:12:59.155476+0000","last_active":"2024-05-28T14:13:00.171344+0000","last_peered":"2024-05-28T14:13:00.171344+0000","last_clean":"2024-05-28T14:13:00.171344+0000","last_became_active":"2024-05-28T14:12:59.155187+0000","last_became_peered":"2024-05-28T14:12:59.155187+0000","last_unstale":"2024-05-28T14:13:00.171344+0000","last_undegraded":"2024-05-28T14:13:00.171344+0000","last_fullsized":"2024-05-28T14:13:00.171344+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:07:49.480989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.148929+0000","last_change":"2024-05-28T14:12:59.148929+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.148929+0000","last_undegraded":"2024-05-28T14:12:59.148929+0000","last_fullsized":"2024-05-28T14:12:59.148929+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.164533+0000","last_change":"2024-05-28T14:12:59.164490+0000","last_active":"2024-05-28T14:13:00.164533+0000","last_peered":"2024-05-28T14:13:00.164533+0000","last_clean":"2024-05-28T14:13:00.164533+0000","last_became_active":"2024-05-28T14:12:59.163870+0000","last_became_peered":"2024-05-28T14:12:59.163870+0000","last_unstale":"2024-05-28T14:13:00.164533+0000","last_undegraded":"2024-05-28T14:13:00.164533+0000","last_fullsized":"2024-05-28T14:13:00.164533+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:06.681422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141097+0000","last_change":"2024-05-28T14:12:59.141097+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141097+0000","last_undegraded":"2024-05-28T14:12:59.141097+0000","last_fullsized":"2024-05-28T14:12:59.141097+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.159332+0000","last_change":"2024-05-28T14:12:59.157949+0000","last_active":"2024-05-28T14:12:59.159332+0000","last_peered":"2024-05-28T14:12:59.159332+0000","last_clean":"2024-05-28T14:12:59.159332+0000","last_became_active":"2024-05-28T14:12:59.157600+0000","last_became_peered":"2024-05-28T14:12:59.157600+0000","last_unstale":"2024-05-28T14:12:59.159332+0000","last_undegraded":"2024-05-28T14:12:59.159332+0000","last_fullsized":"2024-05-28T14:12:59.159332+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:46:44.183872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143903+0000","last_change":"2024-05-28T14:12:59.143903+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143903+0000","last_undegraded":"2024-05-28T14:12:59.143903+0000","last_fullsized":"2024-05-28T14:12:59.143903+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.161141+0000","last_change":"2024-05-28T14:12:59.158675+0000","last_active":"2024-05-28T14:12:59.161141+0000","last_peered":"2024-05-28T14:12:59.161141+0000","last_clean":"2024-05-28T14:12:59.161141+0000","last_became_active":"2024-05-28T14:12:59.158378+0000","last_became_peered":"2024-05-28T14:12:59.158378+0000","last_unstale":"2024-05-28T14:12:59.161141+0000","last_undegraded":"2024-05-28T14:12:59.161141+0000","last_fullsized":"2024-05-28T14:12:59.161141+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:50:25.519025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144181+0000","last_change":"2024-05-28T14:12:59.144181+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144181+0000","last_undegraded":"2024-05-28T14:12:59.144181+0000","last_fullsized":"2024-05-28T14:12:59.144181+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.163331+0000","last_change":"2024-05-28T14:12:59.162720+0000","last_active":"2024-05-28T14:12:59.163331+0000","last_peered":"2024-05-28T14:12:59.163331+0000","last_clean":"2024-05-28T14:12:59.163331+0000","last_became_active":"2024-05-28T14:12:59.162364+0000","last_became_peered":"2024-05-28T14:12:59.162364+0000","last_unstale":"2024-05-28T14:12:59.163331+0000","last_undegraded":"2024-05-28T14:12:59.163331+0000","last_fullsized":"2024-05-28T14:12:59.163331+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:51:48.221003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143666+0000","last_change":"2024-05-28T14:12:59.143666+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143666+0000","last_undegraded":"2024-05-28T14:12:59.143666+0000","last_fullsized":"2024-05-28T14:12:59.143666+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.165719+0000","last_change":"2024-05-28T14:12:59.163864+0000","last_active":"2024-05-28T14:12:59.165719+0000","last_peered":"2024-05-28T14:12:59.165719+0000","last_clean":"2024-05-28T14:12:59.165719+0000","last_became_active":"2024-05-28T14:12:59.163528+0000","last_became_peered":"2024-05-28T14:12:59.163528+0000","last_unstale":"2024-05-28T14:12:59.165719+0000","last_undegraded":"2024-05-28T14:12:59.165719+0000","last_fullsized":"2024-05-28T14:12:59.165719+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:07:17.818267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145598+0000","last_change":"2024-05-28T14:12:59.145598+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145598+0000","last_undegraded":"2024-05-28T14:12:59.145598+0000","last_fullsized":"2024-05-28T14:12:59.145598+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.144976+0000","last_change":"2024-05-28T14:12:59.165251+0000","last_active":"2024-05-28T14:13:00.144976+0000","last_peered":"2024-05-28T14:13:00.144976+0000","last_clean":"2024-05-28T14:13:00.144976+0000","last_became_active":"2024-05-28T14:12:59.164960+0000","last_became_peered":"2024-05-28T14:12:59.164960+0000","last_unstale":"2024-05-28T14:13:00.144976+0000","last_undegraded":"2024-05-28T14:13:00.144976+0000","last_fullsized":"2024-05-28T14:13:00.144976+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:47:24.425191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.151052+0000","last_change":"2024-05-28T14:12:59.151052+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.151052+0000","last_undegraded":"2024-05-28T14:12:59.151052+0000","last_fullsized":"2024-05-28T14:12:59.151052+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.161182+0000","last_change":"2024-05-28T14:12:59.157811+0000","last_active":"2024-05-28T14:12:59.161182+0000","last_peered":"2024-05-28T14:12:59.161182+0000","last_clean":"2024-05-28T14:12:59.161182+0000","last_became_active":"2024-05-28T14:12:59.157503+0000","last_became_peered":"2024-05-28T14:12:59.157503+0000","last_unstale":"2024-05-28T14:12:59.161182+0000","last_undegraded":"2024-05-28T14:12:59.161182+0000","last_fullsized":"2024-05-28T14:12:59.161182+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:53:50.157477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145229+0000","last_change":"2024-05-28T14:12:59.145229+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145229+0000","last_undegraded":"2024-05-28T14:12:59.145229+0000","last_fullsized":"2024-05-28T14:12:59.145229+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.165554+0000","last_change":"2024-05-28T14:12:59.163447+0000","last_active":"2024-05-28T14:12:59.165554+0000","last_peered":"2024-05-28T14:12:59.165554+0000","last_clean":"2024-05-28T14:12:59.165554+0000","last_became_active":"2024-05-28T14:12:59.163160+0000","last_became_peered":"2024-05-28T14:12:59.163160+0000","last_unstale":"2024-05-28T14:12:59.165554+0000","last_undegraded":"2024-05-28T14:12:59.165554+0000","last_fullsized":"2024-05-28T14:12:59.165554+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:02:28.323335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143506+0000","last_change":"2024-05-28T14:12:59.143506+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143506+0000","last_undegraded":"2024-05-28T14:12:59.143506+0000","last_fullsized":"2024-05-28T14:12:59.143506+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.166405+0000","last_change":"2024-05-28T14:12:59.164229+0000","last_active":"2024-05-28T14:12:59.166405+0000","last_peered":"2024-05-28T14:12:59.166405+0000","last_clean":"2024-05-28T14:12:59.166405+0000","last_became_active":"2024-05-28T14:12:59.163931+0000","last_became_peered":"2024-05-28T14:12:59.163931+0000","last_unstale":"2024-05-28T14:12:59.166405+0000","last_undegraded":"2024-05-28T14:12:59.166405+0000","last_fullsized":"2024-05-28T14:12:59.166405+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:17:03.844751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144233+0000","last_change":"2024-05-28T14:12:59.144233+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144233+0000","last_undegraded":"2024-05-28T14:12:59.144233+0000","last_fullsized":"2024-05-28T14:12:59.144233+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162823+0000","last_change":"2024-05-28T14:12:59.155868+0000","last_active":"2024-05-28T14:12:59.162823+0000","last_peered":"2024-05-28T14:12:59.162823+0000","last_clean":"2024-05-28T14:12:59.162823+0000","last_became_active":"2024-05-28T14:12:59.155584+0000","last_became_peered":"2024-05-28T14:12:59.155584+0000","last_unstale":"2024-05-28T14:12:59.162823+0000","last_undegraded":"2024-05-28T14:12:59.162823+0000","last_fullsized":"2024-05-28T14:12:59.162823+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:03:16.477026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144770+0000","last_change":"2024-05-28T14:12:59.144770+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144770+0000","last_undegraded":"2024-05-28T14:12:59.144770+0000","last_fullsized":"2024-05-28T14:12:59.144770+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162280+0000","last_change":"2024-05-28T14:12:59.161033+0000","last_active":"2024-05-28T14:12:59.162280+0000","last_peered":"2024-05-28T14:12:59.162280+0000","last_clean":"2024-05-28T14:12:59.162280+0000","last_became_active":"2024-05-28T14:12:59.160714+0000","last_became_peered":"2024-05-28T14:12:59.160714+0000","last_unstale":"2024-05-28T14:12:59.162280+0000","last_undegraded":"2024-05-28T14:12:59.162280+0000","last_fullsized":"2024-05-28T14:12:59.162280+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:41:49.719823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144691+0000","last_change":"2024-05-28T14:12:59.144691+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144691+0000","last_undegraded":"2024-05-28T14:12:59.144691+0000","last_fullsized":"2024-05-28T14:12:59.144691+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"116'8","reported_seq":20,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.170210+0000","last_change":"2024-05-28T14:12:59.154364+0000","last_active":"2024-05-28T14:13:00.170210+0000","last_peered":"2024-05-28T14:13:00.170210+0000","last_clean":"2024-05-28T14:13:00.170210+0000","last_became_active":"2024-05-28T14:12:59.152108+0000","last_became_peered":"2024-05-28T14:12:59.152108+0000","last_unstale":"2024-05-28T14:13:00.170210+0000","last_undegraded":"2024-05-28T14:13:00.170210+0000","last_fullsized":"2024-05-28T14:13:00.170210+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:23.615288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.159293+0000","last_change":"2024-05-28T14:13:00.155655+0000","last_active":"2024-05-28T14:13:00.159293+0000","last_peered":"2024-05-28T14:13:00.159293+0000","last_clean":"2024-05-28T14:13:00.159293+0000","last_became_active":"2024-05-28T14:13:00.155230+0000","last_became_peered":"2024-05-28T14:13:00.155230+0000","last_unstale":"2024-05-28T14:13:00.159293+0000","last_undegraded":"2024-05-28T14:13:00.159293+0000","last_fullsized":"2024-05-28T14:13:00.159293+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:23:08.600237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.160459+0000","last_change":"2024-05-28T14:12:59.158148+0000","last_active":"2024-05-28T14:12:59.160459+0000","last_peered":"2024-05-28T14:12:59.160459+0000","last_clean":"2024-05-28T14:12:59.160459+0000","last_became_active":"2024-05-28T14:12:59.156240+0000","last_became_peered":"2024-05-28T14:12:59.156240+0000","last_unstale":"2024-05-28T14:12:59.160459+0000","last_undegraded":"2024-05-28T14:12:59.160459+0000","last_fullsized":"2024-05-28T14:12:59.160459+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:53:50.157477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.150512+0000","last_change":"2024-05-28T14:12:59.150512+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.150512+0000","last_undegraded":"2024-05-28T14:12:59.150512+0000","last_fullsized":"2024-05-28T14:12:59.150512+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.171402+0000","last_change":"2024-05-28T14:12:59.154024+0000","last_active":"2024-05-28T14:13:00.171402+0000","last_peered":"2024-05-28T14:13:00.171402+0000","last_clean":"2024-05-28T14:13:00.171402+0000","last_became_active":"2024-05-28T14:12:59.153637+0000","last_became_peered":"2024-05-28T14:12:59.153637+0000","last_unstale":"2024-05-28T14:13:00.171402+0000","last_undegraded":"2024-05-28T14:13:00.171402+0000","last_fullsized":"2024-05-28T14:13:00.171402+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:24:19.180753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.148185+0000","last_change":"2024-05-28T14:12:59.148185+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.148185+0000","last_undegraded":"2024-05-28T14:12:59.148185+0000","last_fullsized":"2024-05-28T14:12:59.148185+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162705+0000","last_change":"2024-05-28T14:12:59.161950+0000","last_active":"2024-05-28T14:12:59.162705+0000","last_peered":"2024-05-28T14:12:59.162705+0000","last_clean":"2024-05-28T14:12:59.162705+0000","last_became_active":"2024-05-28T14:12:59.161536+0000","last_became_peered":"2024-05-28T14:12:59.161536+0000","last_unstale":"2024-05-28T14:12:59.162705+0000","last_undegraded":"2024-05-28T14:12:59.162705+0000","last_fullsized":"2024-05-28T14:12:59.162705+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:28:45.886716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.147735+0000","last_change":"2024-05-28T14:12:59.147735+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.147735+0000","last_undegraded":"2024-05-28T14:12:59.147735+0000","last_fullsized":"2024-05-28T14:12:59.147735+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.157718+0000","last_change":"2024-05-28T14:12:59.155568+0000","last_active":"2024-05-28T14:12:59.157718+0000","last_peered":"2024-05-28T14:12:59.157718+0000","last_clean":"2024-05-28T14:12:59.157718+0000","last_became_active":"2024-05-28T14:12:59.154202+0000","last_became_peered":"2024-05-28T14:12:59.154202+0000","last_unstale":"2024-05-28T14:12:59.157718+0000","last_undegraded":"2024-05-28T14:12:59.157718+0000","last_fullsized":"2024-05-28T14:12:59.157718+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:23.615288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146108+0000","last_change":"2024-05-28T14:12:59.146108+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146108+0000","last_undegraded":"2024-05-28T14:12:59.146108+0000","last_fullsized":"2024-05-28T14:12:59.146108+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162847+0000","last_change":"2024-05-28T14:12:59.159676+0000","last_active":"2024-05-28T14:12:59.162847+0000","last_peered":"2024-05-28T14:12:59.162847+0000","last_clean":"2024-05-28T14:12:59.162847+0000","last_became_active":"2024-05-28T14:12:59.159391+0000","last_became_peered":"2024-05-28T14:12:59.159391+0000","last_unstale":"2024-05-28T14:12:59.162847+0000","last_undegraded":"2024-05-28T14:12:59.162847+0000","last_fullsized":"2024-05-28T14:12:59.162847+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:25:49.893677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141618+0000","last_change":"2024-05-28T14:12:59.141618+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141618+0000","last_undegraded":"2024-05-28T14:12:59.141618+0000","last_fullsized":"2024-05-28T14:12:59.141618+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.142707+0000","last_change":"2024-05-28T14:12:59.153857+0000","last_active":"2024-05-28T14:13:00.142707+0000","last_peered":"2024-05-28T14:13:00.142707+0000","last_clean":"2024-05-28T14:13:00.142707+0000","last_became_active":"2024-05-28T14:12:59.153528+0000","last_became_peered":"2024-05-28T14:12:59.153528+0000","last_unstale":"2024-05-28T14:13:00.142707+0000","last_undegraded":"2024-05-28T14:13:00.142707+0000","last_fullsized":"2024-05-28T14:13:00.142707+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:46:44.183872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146666+0000","last_change":"2024-05-28T14:12:59.146666+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146666+0000","last_undegraded":"2024-05-28T14:12:59.146666+0000","last_fullsized":"2024-05-28T14:12:59.146666+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.158654+0000","last_change":"2024-05-28T14:12:59.153891+0000","last_active":"2024-05-28T14:12:59.158654+0000","last_peered":"2024-05-28T14:12:59.158654+0000","last_clean":"2024-05-28T14:12:59.158654+0000","last_became_active":"2024-05-28T14:12:59.153629+0000","last_became_peered":"2024-05-28T14:12:59.153629+0000","last_unstale":"2024-05-28T14:12:59.158654+0000","last_undegraded":"2024-05-28T14:12:59.158654+0000","last_fullsized":"2024-05-28T14:12:59.158654+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:56:25.534573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143944+0000","last_change":"2024-05-28T14:12:59.143944+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143944+0000","last_undegraded":"2024-05-28T14:12:59.143944+0000","last_fullsized":"2024-05-28T14:12:59.143944+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162520+0000","last_change":"2024-05-28T14:12:59.159929+0000","last_active":"2024-05-28T14:12:59.162520+0000","last_peered":"2024-05-28T14:12:59.162520+0000","last_clean":"2024-05-28T14:12:59.162520+0000","last_became_active":"2024-05-28T14:12:59.159646+0000","last_became_peered":"2024-05-28T14:12:59.159646+0000","last_unstale":"2024-05-28T14:12:59.162520+0000","last_undegraded":"2024-05-28T14:12:59.162520+0000","last_fullsized":"2024-05-28T14:12:59.162520+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:45:38.347737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.151926+0000","last_change":"2024-05-28T14:12:59.151926+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.151926+0000","last_undegraded":"2024-05-28T14:12:59.151926+0000","last_fullsized":"2024-05-28T14:12:59.151926+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.165506+0000","last_change":"2024-05-28T14:12:59.161594+0000","last_active":"2024-05-28T14:12:59.165506+0000","last_peered":"2024-05-28T14:12:59.165506+0000","last_clean":"2024-05-28T14:12:59.165506+0000","last_became_active":"2024-05-28T14:12:59.161326+0000","last_became_peered":"2024-05-28T14:12:59.161326+0000","last_unstale":"2024-05-28T14:12:59.165506+0000","last_undegraded":"2024-05-28T14:12:59.165506+0000","last_fullsized":"2024-05-28T14:12:59.165506+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:45:38.347737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142671+0000","last_change":"2024-05-28T14:12:59.142671+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142671+0000","last_undegraded":"2024-05-28T14:12:59.142671+0000","last_fullsized":"2024-05-28T14:12:59.142671+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.161105+0000","last_change":"2024-05-28T14:12:59.159230+0000","last_active":"2024-05-28T14:12:59.161105+0000","last_peered":"2024-05-28T14:12:59.161105+0000","last_clean":"2024-05-28T14:12:59.161105+0000","last_became_active":"2024-05-28T14:12:59.158844+0000","last_became_peered":"2024-05-28T14:12:59.158844+0000","last_unstale":"2024-05-28T14:12:59.161105+0000","last_undegraded":"2024-05-28T14:12:59.161105+0000","last_fullsized":"2024-05-28T14:12:59.161105+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:56:46.861095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142149+0000","last_change":"2024-05-28T14:12:59.142149+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142149+0000","last_undegraded":"2024-05-28T14:12:59.142149+0000","last_fullsized":"2024-05-28T14:12:59.142149+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162552+0000","last_change":"2024-05-28T14:12:59.159785+0000","last_active":"2024-05-28T14:12:59.162552+0000","last_peered":"2024-05-28T14:12:59.162552+0000","last_clean":"2024-05-28T14:12:59.162552+0000","last_became_active":"2024-05-28T14:12:59.159495+0000","last_became_peered":"2024-05-28T14:12:59.159495+0000","last_unstale":"2024-05-28T14:12:59.162552+0000","last_undegraded":"2024-05-28T14:12:59.162552+0000","last_fullsized":"2024-05-28T14:12:59.162552+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:15:36.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.153890+0000","last_change":"2024-05-28T14:12:59.153890+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.153890+0000","last_undegraded":"2024-05-28T14:12:59.153890+0000","last_fullsized":"2024-05-28T14:12:59.153890+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162693+0000","last_change":"2024-05-28T14:12:59.160575+0000","last_active":"2024-05-28T14:12:59.162693+0000","last_peered":"2024-05-28T14:12:59.162693+0000","last_clean":"2024-05-28T14:12:59.162693+0000","last_became_active":"2024-05-28T14:12:59.160277+0000","last_became_peered":"2024-05-28T14:12:59.160277+0000","last_unstale":"2024-05-28T14:12:59.162693+0000","last_undegraded":"2024-05-28T14:12:59.162693+0000","last_fullsized":"2024-05-28T14:12:59.162693+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:15:36.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143331+0000","last_change":"2024-05-28T14:12:59.143331+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143331+0000","last_undegraded":"2024-05-28T14:12:59.143331+0000","last_fullsized":"2024-05-28T14:12:59.143331+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.158079+0000","last_change":"2024-05-28T14:12:59.155180+0000","last_active":"2024-05-28T14:12:59.158079+0000","last_peered":"2024-05-28T14:12:59.158079+0000","last_clean":"2024-05-28T14:12:59.158079+0000","last_became_active":"2024-05-28T14:12:59.154912+0000","last_became_peered":"2024-05-28T14:12:59.154912+0000","last_unstale":"2024-05-28T14:12:59.158079+0000","last_undegraded":"2024-05-28T14:12:59.158079+0000","last_fullsized":"2024-05-28T14:12:59.158079+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:04:39.323754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143393+0000","last_change":"2024-05-28T14:12:59.143393+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143393+0000","last_undegraded":"2024-05-28T14:12:59.143393+0000","last_fullsized":"2024-05-28T14:12:59.143393+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.158481+0000","last_change":"2024-05-28T14:12:59.156463+0000","last_active":"2024-05-28T14:12:59.158481+0000","last_peered":"2024-05-28T14:12:59.158481+0000","last_clean":"2024-05-28T14:12:59.158481+0000","last_became_active":"2024-05-28T14:12:59.155528+0000","last_became_peered":"2024-05-28T14:12:59.155528+0000","last_unstale":"2024-05-28T14:12:59.158481+0000","last_undegraded":"2024-05-28T14:12:59.158481+0000","last_fullsized":"2024-05-28T14:12:59.158481+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:16:34.556526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146039+0000","last_change":"2024-05-28T14:12:59.146039+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146039+0000","last_undegraded":"2024-05-28T14:12:59.146039+0000","last_fullsized":"2024-05-28T14:12:59.146039+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.142671+0000","last_change":"2024-05-28T14:12:59.154458+0000","last_active":"2024-05-28T14:13:00.142671+0000","last_peered":"2024-05-28T14:13:00.142671+0000","last_clean":"2024-05-28T14:13:00.142671+0000","last_became_active":"2024-05-28T14:12:59.153652+0000","last_became_peered":"2024-05-28T14:12:59.153652+0000","last_unstale":"2024-05-28T14:13:00.142671+0000","last_undegraded":"2024-05-28T14:13:00.142671+0000","last_fullsized":"2024-05-28T14:13:00.142671+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:39:27.979374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.140964+0000","last_change":"2024-05-28T14:12:59.140964+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.140964+0000","last_undegraded":"2024-05-28T14:12:59.140964+0000","last_fullsized":"2024-05-28T14:12:59.140964+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.160877+0000","last_change":"2024-05-28T14:12:59.158941+0000","last_active":"2024-05-28T14:12:59.160877+0000","last_peered":"2024-05-28T14:12:59.160877+0000","last_clean":"2024-05-28T14:12:59.160877+0000","last_became_active":"2024-05-28T14:12:59.158680+0000","last_became_peered":"2024-05-28T14:12:59.158680+0000","last_unstale":"2024-05-28T14:12:59.160877+0000","last_undegraded":"2024-05-28T14:12:59.160877+0000","last_fullsized":"2024-05-28T14:12:59.160877+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:50:25.519025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143848+0000","last_change":"2024-05-28T14:12:59.143848+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143848+0000","last_undegraded":"2024-05-28T14:12:59.143848+0000","last_fullsized":"2024-05-28T14:12:59.143848+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.157321+0000","last_change":"2024-05-28T14:12:59.154352+0000","last_active":"2024-05-28T14:12:59.157321+0000","last_peered":"2024-05-28T14:12:59.157321+0000","last_clean":"2024-05-28T14:12:59.157321+0000","last_became_active":"2024-05-28T14:12:59.154075+0000","last_became_peered":"2024-05-28T14:12:59.154075+0000","last_unstale":"2024-05-28T14:12:59.157321+0000","last_undegraded":"2024-05-28T14:12:59.157321+0000","last_fullsized":"2024-05-28T14:12:59.157321+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:25:00.922583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":347,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.152540+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:12:59.152540+0000","last_peered":"2024-05-28T14:12:59.152540+0000","last_clean":"2024-05-28T14:12:59.152540+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:12:59.152540+0000","last_undegraded":"2024-05-28T14:12:59.152540+0000","last_fullsized":"2024-05-28T14:12:59.152540+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575028,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131652,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240188,"statfs":{"total":96636764160,"available":96501952512,"internally_reserved":0,"allocated":2023424,"data_stored":1213925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131080,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240760,"statfs":{"total":96636764160,"available":96502538240,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575030,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64436,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307404,"statfs":{"total":96636764160,"available":96570781696,"internally_reserved":0,"allocated":1417216,"data_stored":617226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144964,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226876,"statfs":{"total":96636764160,"available":96488321024,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575028,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122756,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249084,"statfs":{"total":96636764160,"available":96511062016,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":124254664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575029,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136640,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235200,"statfs":{"total":96636764160,"available":96496844800,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149120,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222720,"statfs":{"total":96636764160,"available":96484065280,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575029,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91396,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280444,"statfs":{"total":96636764160,"available":96543174656,"internally_reserved":0,"allocated":2023424,"data_stored":1213925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:13:04.268 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:13:04.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff96e400640 1 Processor -- start 2024-05-28T14:13:04.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff96e400640 1 -- start start 2024-05-28T14:13:04.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff96e400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff9680ce860 0x7ff96806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff96e400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff9680d0900 0x7ff968162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff96e400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff96806dcf0 0x7ff968162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff96e400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff9680d19f0 con 0x7ff9680ce860 2024-05-28T14:13:04.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff96e400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff9680d0de0 con 0x7ff9680d0900 2024-05-28T14:13:04.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff96e400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9680d10c0 con 0x7ff96806dcf0 2024-05-28T14:13:04.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff967400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff9680ce860 0x7ff96806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff967400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff9680ce860 0x7ff96806d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33938/0 (socket says 172.21.15.42:33938) 2024-05-28T14:13:04.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff967400640 1 -- 172.21.15.42:0/1149575718 learned_addr learned my addr 172.21.15.42:0/1149575718 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:04.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff95ea00640 1 --2- 172.21.15.42:0/1149575718 >> v2:172.21.15.63:3300/0 conn(0x7ff9680d0900 0x7ff968162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff967e00640 1 --2- 172.21.15.42:0/1149575718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff96806dcf0 0x7ff968162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff967400640 1 -- 172.21.15.42:0/1149575718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff96806dcf0 msgr2=0x7ff968162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:04.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff967400640 1 --2- 172.21.15.42:0/1149575718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff96806dcf0 0x7ff968162ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff967400640 1 -- 172.21.15.42:0/1149575718 >> v2:172.21.15.63:3300/0 conn(0x7ff9680d0900 msgr2=0x7ff968162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:04.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff967400640 1 --2- 172.21.15.42:0/1149575718 >> v2:172.21.15.63:3300/0 conn(0x7ff9680d0900 0x7ff968162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff967400640 1 -- 172.21.15.42:0/1149575718 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9680d13a0 con 0x7ff9680ce860 2024-05-28T14:13:04.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.463+0000 7ff95ea00640 1 --2- 172.21.15.42:0/1149575718 >> v2:172.21.15.63:3300/0 conn(0x7ff9680d0900 0x7ff968162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:04.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.467+0000 7ff967400640 1 --2- 172.21.15.42:0/1149575718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9680ce860 0x7ff96806d7b0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7ff9680d0670 tx=0x7ff95801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae2476c0ff54672 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:04.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.467+0000 7ff966a00640 1 -- 172.21.15.42:0/1149575718 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9580ab640 con 0x7ff9680ce860 2024-05-28T14:13:04.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.467+0000 7ff966a00640 1 -- 172.21.15.42:0/1149575718 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9580b0070 con 0x7ff9680ce860 2024-05-28T14:13:04.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.467+0000 7ff966a00640 1 -- 172.21.15.42:0/1149575718 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9580aba00 con 0x7ff9680ce860 2024-05-28T14:13:04.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.467+0000 7ff96e400640 1 -- 172.21.15.42:0/1149575718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9680ce860 msgr2=0x7ff96806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.467+0000 7ff96e400640 1 --2- 172.21.15.42:0/1149575718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9680ce860 0x7ff96806d7b0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7ff9680d0670 tx=0x7ff95801c460 comp rx=0 tx=0).stop 2024-05-28T14:13:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.467+0000 7ff96e400640 1 -- 172.21.15.42:0/1149575718 shutdown_connections 2024-05-28T14:13:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.467+0000 7ff96e400640 1 --2- 172.21.15.42:0/1149575718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff96806dcf0 0x7ff968162ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.467+0000 7ff96e400640 1 --2- 172.21.15.42:0/1149575718 >> v2:172.21.15.63:3300/0 conn(0x7ff9680d0900 0x7ff968162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.467+0000 7ff96e400640 1 --2- 172.21.15.42:0/1149575718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9680ce860 0x7ff96806d7b0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.467+0000 7ff96e400640 1 -- 172.21.15.42:0/1149575718 >> 172.21.15.42:0/1149575718 conn(0x7ff9680b4080 msgr2=0x7ff9680b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.467+0000 7ff96e400640 1 -- 172.21.15.42:0/1149575718 shutdown_connections 2024-05-28T14:13:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.467+0000 7ff96e400640 1 -- 172.21.15.42:0/1149575718 wait complete. 2024-05-28T14:13:04.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.467+0000 7ff96e400640 1 Processor -- start 2024-05-28T14:13:04.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff96e400640 1 -- start start 2024-05-28T14:13:04.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff96e400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff96806dcf0 0x7ff9680caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff96e400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9680ce860 0x7ff9680cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff96e400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9680d0900 0x7ff96814f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff96e400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff96815eda0 con 0x7ff9680d0900 2024-05-28T14:13:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff96e400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff96815ec20 con 0x7ff96806dcf0 2024-05-28T14:13:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff96e400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff96809aa50 con 0x7ff9680ce860 2024-05-28T14:13:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff967e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9680d0900 0x7ff96814f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff967400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff96806dcf0 0x7ff9680caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff967e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9680d0900 0x7ff96814f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33940/0 (socket says 172.21.15.42:33940) 2024-05-28T14:13:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff967e00640 1 -- 172.21.15.42:0/67369444 learned_addr learned my addr 172.21.15.42:0/67369444 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff95ea00640 1 --2- 172.21.15.42:0/67369444 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9680ce860 0x7ff9680cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff967e00640 1 -- 172.21.15.42:0/67369444 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9680ce860 msgr2=0x7ff9680cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff967e00640 1 --2- 172.21.15.42:0/67369444 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9680ce860 0x7ff9680cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff967e00640 1 -- 172.21.15.42:0/67369444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff96806dcf0 msgr2=0x7ff9680caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff967e00640 1 --2- 172.21.15.42:0/67369444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff96806dcf0 0x7ff9680caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff967e00640 1 -- 172.21.15.42:0/67369444 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9680cb070 con 0x7ff9680d0900 2024-05-28T14:13:04.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff967e00640 1 --2- 172.21.15.42:0/67369444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9680d0900 0x7ff96814f370 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ff960007e90 tx=0x7ff96000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:04.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff95fe00640 1 -- 172.21.15.42:0/67369444 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960014420 con 0x7ff9680d0900 2024-05-28T14:13:04.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff95fe00640 1 -- 172.21.15.42:0/67369444 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff960019070 con 0x7ff9680d0900 2024-05-28T14:13:04.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff95fe00640 1 -- 172.21.15.42:0/67369444 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960014820 con 0x7ff9680d0900 2024-05-28T14:13:04.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff96e400640 1 -- 172.21.15.42:0/67369444 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9680cb350 con 0x7ff9680d0900 2024-05-28T14:13:04.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff96e400640 1 -- 172.21.15.42:0/67369444 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9680cb910 con 0x7ff9680d0900 2024-05-28T14:13:04.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.471+0000 7ff96e400640 1 -- 172.21.15.42:0/67369444 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff968068a90 con 0x7ff9680d0900 2024-05-28T14:13:04.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.475+0000 7ff95fe00640 1 -- 172.21.15.42:0/67369444 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7ff9600149c0 con 0x7ff9680d0900 2024-05-28T14:13:04.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.475+0000 7ff95fe00640 1 --2- 172.21.15.42:0/67369444 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff944069b20 0x7ff94406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.475+0000 7ff95fe00640 1 -- 172.21.15.42:0/67369444 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7ff960085f00 con 0x7ff9680d0900 2024-05-28T14:13:04.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.475+0000 7ff967400640 1 --2- 172.21.15.42:0/67369444 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff944069b20 0x7ff94406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.475+0000 7ff967400640 1 --2- 172.21.15.42:0/67369444 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff944069b20 0x7ff94406bfe0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff9580af000 tx=0x7ff9580a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:04.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.479+0000 7ff95fe00640 1 -- 172.21.15.42:0/67369444 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff9600578d0 con 0x7ff9680d0900 2024-05-28T14:13:04.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.639+0000 7ff96e400640 1 -- 172.21.15.42:0/67369444 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9680b4ce0 con 0x7ff944069b20 2024-05-28T14:13:04.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.643+0000 7ff95fe00640 1 -- 172.21.15.42:0/67369444 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247294 (secure 0 0 0) 0x7ff9680b4ce0 con 0x7ff944069b20 2024-05-28T14:13:04.632 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:04.633 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:13:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.647+0000 7ff96e400640 1 -- 172.21.15.42:0/67369444 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff944069b20 msgr2=0x7ff94406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.647+0000 7ff96e400640 1 --2- 172.21.15.42:0/67369444 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff944069b20 0x7ff94406bfe0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff9580af000 tx=0x7ff9580a7000 comp rx=0 tx=0).stop 2024-05-28T14:13:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.647+0000 7ff96e400640 1 -- 172.21.15.42:0/67369444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9680d0900 msgr2=0x7ff96814f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.647+0000 7ff96e400640 1 --2- 172.21.15.42:0/67369444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9680d0900 0x7ff96814f370 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ff960007e90 tx=0x7ff96000e4b0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.647+0000 7ff96e400640 1 -- 172.21.15.42:0/67369444 shutdown_connections 2024-05-28T14:13:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.647+0000 7ff96e400640 1 --2- 172.21.15.42:0/67369444 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff944069b20 0x7ff94406bfe0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.647+0000 7ff96e400640 1 --2- 172.21.15.42:0/67369444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9680d0900 0x7ff96814f370 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.647+0000 7ff96e400640 1 --2- 172.21.15.42:0/67369444 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9680ce860 0x7ff9680cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.647+0000 7ff96e400640 1 --2- 172.21.15.42:0/67369444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff96806dcf0 0x7ff9680caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.647+0000 7ff96e400640 1 -- 172.21.15.42:0/67369444 >> 172.21.15.42:0/67369444 conn(0x7ff9680b4080 msgr2=0x7ff968156870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.647+0000 7ff96e400640 1 -- 172.21.15.42:0/67369444 shutdown_connections 2024-05-28T14:13:04.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.647+0000 7ff96e400640 1 -- 172.21.15.42:0/67369444 wait complete. 2024-05-28T14:13:04.653 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":557,"stamp":"2024-05-28T14:13:03.164561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":972044,"kb_used_data":12316,"kb_used_omap":12,"kb_used_meta":959603,"kb_avail":754002676,"statfs":{"total":773094113280,"available":772098740224,"internally_reserved":0,"allocated":12611584,"data_stored":6162861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":982634034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.037878"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143716+0000","last_change":"2024-05-28T14:12:59.143716+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143716+0000","last_undegraded":"2024-05-28T14:12:59.143716+0000","last_fullsized":"2024-05-28T14:12:59.143716+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145111+0000","last_change":"2024-05-28T14:12:59.145111+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145111+0000","last_undegraded":"2024-05-28T14:12:59.145111+0000","last_fullsized":"2024-05-28T14:12:59.145111+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.150194+0000","last_change":"2024-05-28T14:12:59.150194+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.150194+0000","last_undegraded":"2024-05-28T14:12:59.150194+0000","last_fullsized":"2024-05-28T14:12:59.150194+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143795+0000","last_change":"2024-05-28T14:12:59.143795+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143795+0000","last_undegraded":"2024-05-28T14:12:59.143795+0000","last_fullsized":"2024-05-28T14:12:59.143795+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.151103+0000","last_change":"2024-05-28T14:13:00.146533+0000","last_active":"2024-05-28T14:13:00.151103+0000","last_peered":"2024-05-28T14:13:00.151103+0000","last_clean":"2024-05-28T14:13:00.151103+0000","last_became_active":"2024-05-28T14:13:00.145774+0000","last_became_peered":"2024-05-28T14:13:00.145774+0000","last_unstale":"2024-05-28T14:13:00.151103+0000","last_undegraded":"2024-05-28T14:13:00.151103+0000","last_fullsized":"2024-05-28T14:13:00.151103+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:29:07.224328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141349+0000","last_change":"2024-05-28T14:12:59.141349+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141349+0000","last_undegraded":"2024-05-28T14:12:59.141349+0000","last_fullsized":"2024-05-28T14:12:59.141349+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143106+0000","last_change":"2024-05-28T14:12:59.143106+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143106+0000","last_undegraded":"2024-05-28T14:12:59.143106+0000","last_fullsized":"2024-05-28T14:12:59.143106+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.147478+0000","last_change":"2024-05-28T14:12:59.147478+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.147478+0000","last_undegraded":"2024-05-28T14:12:59.147478+0000","last_fullsized":"2024-05-28T14:12:59.147478+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143317+0000","last_change":"2024-05-28T14:12:59.143317+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143317+0000","last_undegraded":"2024-05-28T14:12:59.143317+0000","last_fullsized":"2024-05-28T14:12:59.143317+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142610+0000","last_change":"2024-05-28T14:12:59.142610+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142610+0000","last_undegraded":"2024-05-28T14:12:59.142610+0000","last_fullsized":"2024-05-28T14:12:59.142610+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144639+0000","last_change":"2024-05-28T14:12:59.144639+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144639+0000","last_undegraded":"2024-05-28T14:12:59.144639+0000","last_fullsized":"2024-05-28T14:12:59.144639+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143340+0000","last_change":"2024-05-28T14:12:59.143340+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143340+0000","last_undegraded":"2024-05-28T14:12:59.143340+0000","last_fullsized":"2024-05-28T14:12:59.143340+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146701+0000","last_change":"2024-05-28T14:12:59.146701+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146701+0000","last_undegraded":"2024-05-28T14:12:59.146701+0000","last_fullsized":"2024-05-28T14:12:59.146701+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.147196+0000","last_change":"2024-05-28T14:12:59.147196+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.147196+0000","last_undegraded":"2024-05-28T14:12:59.147196+0000","last_fullsized":"2024-05-28T14:12:59.147196+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141425+0000","last_change":"2024-05-28T14:12:59.141425+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141425+0000","last_undegraded":"2024-05-28T14:12:59.141425+0000","last_fullsized":"2024-05-28T14:12:59.141425+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144638+0000","last_change":"2024-05-28T14:12:59.144638+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144638+0000","last_undegraded":"2024-05-28T14:12:59.144638+0000","last_fullsized":"2024-05-28T14:12:59.144638+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144519+0000","last_change":"2024-05-28T14:12:59.144519+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144519+0000","last_undegraded":"2024-05-28T14:12:59.144519+0000","last_fullsized":"2024-05-28T14:12:59.144519+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.152420+0000","last_change":"2024-05-28T14:12:59.152420+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.152420+0000","last_undegraded":"2024-05-28T14:12:59.152420+0000","last_fullsized":"2024-05-28T14:12:59.152420+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.149551+0000","last_change":"2024-05-28T14:12:59.149551+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.149551+0000","last_undegraded":"2024-05-28T14:12:59.149551+0000","last_fullsized":"2024-05-28T14:12:59.149551+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143766+0000","last_change":"2024-05-28T14:12:59.143766+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143766+0000","last_undegraded":"2024-05-28T14:12:59.143766+0000","last_fullsized":"2024-05-28T14:12:59.143766+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142806+0000","last_change":"2024-05-28T14:12:59.142806+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142806+0000","last_undegraded":"2024-05-28T14:12:59.142806+0000","last_fullsized":"2024-05-28T14:12:59.142806+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.147836+0000","last_change":"2024-05-28T14:12:59.147836+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.147836+0000","last_undegraded":"2024-05-28T14:12:59.147836+0000","last_fullsized":"2024-05-28T14:12:59.147836+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142797+0000","last_change":"2024-05-28T14:12:59.142797+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142797+0000","last_undegraded":"2024-05-28T14:12:59.142797+0000","last_fullsized":"2024-05-28T14:12:59.142797+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144378+0000","last_change":"2024-05-28T14:12:59.144378+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144378+0000","last_undegraded":"2024-05-28T14:12:59.144378+0000","last_fullsized":"2024-05-28T14:12:59.144378+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143979+0000","last_change":"2024-05-28T14:12:59.143979+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143979+0000","last_undegraded":"2024-05-28T14:12:59.143979+0000","last_fullsized":"2024-05-28T14:12:59.143979+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142104+0000","last_change":"2024-05-28T14:12:59.142104+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142104+0000","last_undegraded":"2024-05-28T14:12:59.142104+0000","last_fullsized":"2024-05-28T14:12:59.142104+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146411+0000","last_change":"2024-05-28T14:12:59.146411+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146411+0000","last_undegraded":"2024-05-28T14:12:59.146411+0000","last_fullsized":"2024-05-28T14:12:59.146411+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145670+0000","last_change":"2024-05-28T14:12:59.145670+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145670+0000","last_undegraded":"2024-05-28T14:12:59.145670+0000","last_fullsized":"2024-05-28T14:12:59.145670+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.148734+0000","last_change":"2024-05-28T14:12:59.148734+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.148734+0000","last_undegraded":"2024-05-28T14:12:59.148734+0000","last_fullsized":"2024-05-28T14:12:59.148734+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143300+0000","last_change":"2024-05-28T14:12:59.143300+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143300+0000","last_undegraded":"2024-05-28T14:12:59.143300+0000","last_fullsized":"2024-05-28T14:12:59.143300+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141526+0000","last_change":"2024-05-28T14:12:59.141526+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141526+0000","last_undegraded":"2024-05-28T14:12:59.141526+0000","last_fullsized":"2024-05-28T14:12:59.141526+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.140649+0000","last_change":"2024-05-28T14:12:59.140649+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.140649+0000","last_undegraded":"2024-05-28T14:12:59.140649+0000","last_fullsized":"2024-05-28T14:12:59.140649+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145011+0000","last_change":"2024-05-28T14:12:59.145011+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145011+0000","last_undegraded":"2024-05-28T14:12:59.145011+0000","last_fullsized":"2024-05-28T14:12:59.145011+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.158758+0000","last_change":"2024-05-28T14:12:59.156662+0000","last_active":"2024-05-28T14:12:59.158758+0000","last_peered":"2024-05-28T14:12:59.158758+0000","last_clean":"2024-05-28T14:12:59.158758+0000","last_became_active":"2024-05-28T14:12:59.156379+0000","last_became_peered":"2024-05-28T14:12:59.156379+0000","last_unstale":"2024-05-28T14:12:59.158758+0000","last_undegraded":"2024-05-28T14:12:59.158758+0000","last_fullsized":"2024-05-28T14:12:59.158758+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:39:27.979374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.151621+0000","last_change":"2024-05-28T14:12:59.151621+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.151621+0000","last_undegraded":"2024-05-28T14:12:59.151621+0000","last_fullsized":"2024-05-28T14:12:59.151621+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.161663+0000","last_change":"2024-05-28T14:12:59.160363+0000","last_active":"2024-05-28T14:12:59.161663+0000","last_peered":"2024-05-28T14:12:59.161663+0000","last_clean":"2024-05-28T14:12:59.161663+0000","last_became_active":"2024-05-28T14:12:59.159749+0000","last_became_peered":"2024-05-28T14:12:59.159749+0000","last_unstale":"2024-05-28T14:12:59.161663+0000","last_undegraded":"2024-05-28T14:12:59.161663+0000","last_fullsized":"2024-05-28T14:12:59.161663+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:15:00.386236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.150359+0000","last_change":"2024-05-28T14:12:59.150359+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.150359+0000","last_undegraded":"2024-05-28T14:12:59.150359+0000","last_fullsized":"2024-05-28T14:12:59.150359+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.171344+0000","last_change":"2024-05-28T14:12:59.155476+0000","last_active":"2024-05-28T14:13:00.171344+0000","last_peered":"2024-05-28T14:13:00.171344+0000","last_clean":"2024-05-28T14:13:00.171344+0000","last_became_active":"2024-05-28T14:12:59.155187+0000","last_became_peered":"2024-05-28T14:12:59.155187+0000","last_unstale":"2024-05-28T14:13:00.171344+0000","last_undegraded":"2024-05-28T14:13:00.171344+0000","last_fullsized":"2024-05-28T14:13:00.171344+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:07:49.480989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.148929+0000","last_change":"2024-05-28T14:12:59.148929+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.148929+0000","last_undegraded":"2024-05-28T14:12:59.148929+0000","last_fullsized":"2024-05-28T14:12:59.148929+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.164533+0000","last_change":"2024-05-28T14:12:59.164490+0000","last_active":"2024-05-28T14:13:00.164533+0000","last_peered":"2024-05-28T14:13:00.164533+0000","last_clean":"2024-05-28T14:13:00.164533+0000","last_became_active":"2024-05-28T14:12:59.163870+0000","last_became_peered":"2024-05-28T14:12:59.163870+0000","last_unstale":"2024-05-28T14:13:00.164533+0000","last_undegraded":"2024-05-28T14:13:00.164533+0000","last_fullsized":"2024-05-28T14:13:00.164533+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:06.681422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141097+0000","last_change":"2024-05-28T14:12:59.141097+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141097+0000","last_undegraded":"2024-05-28T14:12:59.141097+0000","last_fullsized":"2024-05-28T14:12:59.141097+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.159332+0000","last_change":"2024-05-28T14:12:59.157949+0000","last_active":"2024-05-28T14:12:59.159332+0000","last_peered":"2024-05-28T14:12:59.159332+0000","last_clean":"2024-05-28T14:12:59.159332+0000","last_became_active":"2024-05-28T14:12:59.157600+0000","last_became_peered":"2024-05-28T14:12:59.157600+0000","last_unstale":"2024-05-28T14:12:59.159332+0000","last_undegraded":"2024-05-28T14:12:59.159332+0000","last_fullsized":"2024-05-28T14:12:59.159332+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:46:44.183872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143903+0000","last_change":"2024-05-28T14:12:59.143903+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143903+0000","last_undegraded":"2024-05-28T14:12:59.143903+0000","last_fullsized":"2024-05-28T14:12:59.143903+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.161141+0000","last_change":"2024-05-28T14:12:59.158675+0000","last_active":"2024-05-28T14:12:59.161141+0000","last_peered":"2024-05-28T14:12:59.161141+0000","last_clean":"2024-05-28T14:12:59.161141+0000","last_became_active":"2024-05-28T14:12:59.158378+0000","last_became_peered":"2024-05-28T14:12:59.158378+0000","last_unstale":"2024-05-28T14:12:59.161141+0000","last_undegraded":"2024-05-28T14:12:59.161141+0000","last_fullsized":"2024-05-28T14:12:59.161141+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:50:25.519025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144181+0000","last_change":"2024-05-28T14:12:59.144181+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144181+0000","last_undegraded":"2024-05-28T14:12:59.144181+0000","last_fullsized":"2024-05-28T14:12:59.144181+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.163331+0000","last_change":"2024-05-28T14:12:59.162720+0000","last_active":"2024-05-28T14:12:59.163331+0000","last_peered":"2024-05-28T14:12:59.163331+0000","last_clean":"2024-05-28T14:12:59.163331+0000","last_became_active":"2024-05-28T14:12:59.162364+0000","last_became_peered":"2024-05-28T14:12:59.162364+0000","last_unstale":"2024-05-28T14:12:59.163331+0000","last_undegraded":"2024-05-28T14:12:59.163331+0000","last_fullsized":"2024-05-28T14:12:59.163331+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:51:48.221003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143666+0000","last_change":"2024-05-28T14:12:59.143666+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143666+0000","last_undegraded":"2024-05-28T14:12:59.143666+0000","last_fullsized":"2024-05-28T14:12:59.143666+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.165719+0000","last_change":"2024-05-28T14:12:59.163864+0000","last_active":"2024-05-28T14:12:59.165719+0000","last_peered":"2024-05-28T14:12:59.165719+0000","last_clean":"2024-05-28T14:12:59.165719+0000","last_became_active":"2024-05-28T14:12:59.163528+0000","last_became_peered":"2024-05-28T14:12:59.163528+0000","last_unstale":"2024-05-28T14:12:59.165719+0000","last_undegraded":"2024-05-28T14:12:59.165719+0000","last_fullsized":"2024-05-28T14:12:59.165719+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:07:17.818267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145598+0000","last_change":"2024-05-28T14:12:59.145598+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145598+0000","last_undegraded":"2024-05-28T14:12:59.145598+0000","last_fullsized":"2024-05-28T14:12:59.145598+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.144976+0000","last_change":"2024-05-28T14:12:59.165251+0000","last_active":"2024-05-28T14:13:00.144976+0000","last_peered":"2024-05-28T14:13:00.144976+0000","last_clean":"2024-05-28T14:13:00.144976+0000","last_became_active":"2024-05-28T14:12:59.164960+0000","last_became_peered":"2024-05-28T14:12:59.164960+0000","last_unstale":"2024-05-28T14:13:00.144976+0000","last_undegraded":"2024-05-28T14:13:00.144976+0000","last_fullsized":"2024-05-28T14:13:00.144976+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:47:24.425191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.151052+0000","last_change":"2024-05-28T14:12:59.151052+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.151052+0000","last_undegraded":"2024-05-28T14:12:59.151052+0000","last_fullsized":"2024-05-28T14:12:59.151052+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.161182+0000","last_change":"2024-05-28T14:12:59.157811+0000","last_active":"2024-05-28T14:12:59.161182+0000","last_peered":"2024-05-28T14:12:59.161182+0000","last_clean":"2024-05-28T14:12:59.161182+0000","last_became_active":"2024-05-28T14:12:59.157503+0000","last_became_peered":"2024-05-28T14:12:59.157503+0000","last_unstale":"2024-05-28T14:12:59.161182+0000","last_undegraded":"2024-05-28T14:12:59.161182+0000","last_fullsized":"2024-05-28T14:12:59.161182+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:53:50.157477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145229+0000","last_change":"2024-05-28T14:12:59.145229+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145229+0000","last_undegraded":"2024-05-28T14:12:59.145229+0000","last_fullsized":"2024-05-28T14:12:59.145229+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.165554+0000","last_change":"2024-05-28T14:12:59.163447+0000","last_active":"2024-05-28T14:12:59.165554+0000","last_peered":"2024-05-28T14:12:59.165554+0000","last_clean":"2024-05-28T14:12:59.165554+0000","last_became_active":"2024-05-28T14:12:59.163160+0000","last_became_peered":"2024-05-28T14:12:59.163160+0000","last_unstale":"2024-05-28T14:12:59.165554+0000","last_undegraded":"2024-05-28T14:12:59.165554+0000","last_fullsized":"2024-05-28T14:12:59.165554+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:02:28.323335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143506+0000","last_change":"2024-05-28T14:12:59.143506+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143506+0000","last_undegraded":"2024-05-28T14:12:59.143506+0000","last_fullsized":"2024-05-28T14:12:59.143506+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.166405+0000","last_change":"2024-05-28T14:12:59.164229+0000","last_active":"2024-05-28T14:12:59.166405+0000","last_peered":"2024-05-28T14:12:59.166405+0000","last_clean":"2024-05-28T14:12:59.166405+0000","last_became_active":"2024-05-28T14:12:59.163931+0000","last_became_peered":"2024-05-28T14:12:59.163931+0000","last_unstale":"2024-05-28T14:12:59.166405+0000","last_undegraded":"2024-05-28T14:12:59.166405+0000","last_fullsized":"2024-05-28T14:12:59.166405+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:17:03.844751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144233+0000","last_change":"2024-05-28T14:12:59.144233+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144233+0000","last_undegraded":"2024-05-28T14:12:59.144233+0000","last_fullsized":"2024-05-28T14:12:59.144233+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162823+0000","last_change":"2024-05-28T14:12:59.155868+0000","last_active":"2024-05-28T14:12:59.162823+0000","last_peered":"2024-05-28T14:12:59.162823+0000","last_clean":"2024-05-28T14:12:59.162823+0000","last_became_active":"2024-05-28T14:12:59.155584+0000","last_became_peered":"2024-05-28T14:12:59.155584+0000","last_unstale":"2024-05-28T14:12:59.162823+0000","last_undegraded":"2024-05-28T14:12:59.162823+0000","last_fullsized":"2024-05-28T14:12:59.162823+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:03:16.477026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144770+0000","last_change":"2024-05-28T14:12:59.144770+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144770+0000","last_undegraded":"2024-05-28T14:12:59.144770+0000","last_fullsized":"2024-05-28T14:12:59.144770+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162280+0000","last_change":"2024-05-28T14:12:59.161033+0000","last_active":"2024-05-28T14:12:59.162280+0000","last_peered":"2024-05-28T14:12:59.162280+0000","last_clean":"2024-05-28T14:12:59.162280+0000","last_became_active":"2024-05-28T14:12:59.160714+0000","last_became_peered":"2024-05-28T14:12:59.160714+0000","last_unstale":"2024-05-28T14:12:59.162280+0000","last_undegraded":"2024-05-28T14:12:59.162280+0000","last_fullsized":"2024-05-28T14:12:59.162280+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:41:49.719823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144691+0000","last_change":"2024-05-28T14:12:59.144691+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144691+0000","last_undegraded":"2024-05-28T14:12:59.144691+0000","last_fullsized":"2024-05-28T14:12:59.144691+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"116'8","reported_seq":20,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.170210+0000","last_change":"2024-05-28T14:12:59.154364+0000","last_active":"2024-05-28T14:13:00.170210+0000","last_peered":"2024-05-28T14:13:00.170210+0000","last_clean":"2024-05-28T14:13:00.170210+0000","last_became_active":"2024-05-28T14:12:59.152108+0000","last_became_peered":"2024-05-28T14:12:59.152108+0000","last_unstale":"2024-05-28T14:13:00.170210+0000","last_undegraded":"2024-05-28T14:13:00.170210+0000","last_fullsized":"2024-05-28T14:13:00.170210+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:23.615288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.159293+0000","last_change":"2024-05-28T14:13:00.155655+0000","last_active":"2024-05-28T14:13:00.159293+0000","last_peered":"2024-05-28T14:13:00.159293+0000","last_clean":"2024-05-28T14:13:00.159293+0000","last_became_active":"2024-05-28T14:13:00.155230+0000","last_became_peered":"2024-05-28T14:13:00.155230+0000","last_unstale":"2024-05-28T14:13:00.159293+0000","last_undegraded":"2024-05-28T14:13:00.159293+0000","last_fullsized":"2024-05-28T14:13:00.159293+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:23:08.600237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.160459+0000","last_change":"2024-05-28T14:12:59.158148+0000","last_active":"2024-05-28T14:12:59.160459+0000","last_peered":"2024-05-28T14:12:59.160459+0000","last_clean":"2024-05-28T14:12:59.160459+0000","last_became_active":"2024-05-28T14:12:59.156240+0000","last_became_peered":"2024-05-28T14:12:59.156240+0000","last_unstale":"2024-05-28T14:12:59.160459+0000","last_undegraded":"2024-05-28T14:12:59.160459+0000","last_fullsized":"2024-05-28T14:12:59.160459+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:53:50.157477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.150512+0000","last_change":"2024-05-28T14:12:59.150512+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.150512+0000","last_undegraded":"2024-05-28T14:12:59.150512+0000","last_fullsized":"2024-05-28T14:12:59.150512+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.171402+0000","last_change":"2024-05-28T14:12:59.154024+0000","last_active":"2024-05-28T14:13:00.171402+0000","last_peered":"2024-05-28T14:13:00.171402+0000","last_clean":"2024-05-28T14:13:00.171402+0000","last_became_active":"2024-05-28T14:12:59.153637+0000","last_became_peered":"2024-05-28T14:12:59.153637+0000","last_unstale":"2024-05-28T14:13:00.171402+0000","last_undegraded":"2024-05-28T14:13:00.171402+0000","last_fullsized":"2024-05-28T14:13:00.171402+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:24:19.180753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.148185+0000","last_change":"2024-05-28T14:12:59.148185+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.148185+0000","last_undegraded":"2024-05-28T14:12:59.148185+0000","last_fullsized":"2024-05-28T14:12:59.148185+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162705+0000","last_change":"2024-05-28T14:12:59.161950+0000","last_active":"2024-05-28T14:12:59.162705+0000","last_peered":"2024-05-28T14:12:59.162705+0000","last_clean":"2024-05-28T14:12:59.162705+0000","last_became_active":"2024-05-28T14:12:59.161536+0000","last_became_peered":"2024-05-28T14:12:59.161536+0000","last_unstale":"2024-05-28T14:12:59.162705+0000","last_undegraded":"2024-05-28T14:12:59.162705+0000","last_fullsized":"2024-05-28T14:12:59.162705+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:28:45.886716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.147735+0000","last_change":"2024-05-28T14:12:59.147735+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.147735+0000","last_undegraded":"2024-05-28T14:12:59.147735+0000","last_fullsized":"2024-05-28T14:12:59.147735+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.157718+0000","last_change":"2024-05-28T14:12:59.155568+0000","last_active":"2024-05-28T14:12:59.157718+0000","last_peered":"2024-05-28T14:12:59.157718+0000","last_clean":"2024-05-28T14:12:59.157718+0000","last_became_active":"2024-05-28T14:12:59.154202+0000","last_became_peered":"2024-05-28T14:12:59.154202+0000","last_unstale":"2024-05-28T14:12:59.157718+0000","last_undegraded":"2024-05-28T14:12:59.157718+0000","last_fullsized":"2024-05-28T14:12:59.157718+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:23.615288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146108+0000","last_change":"2024-05-28T14:12:59.146108+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146108+0000","last_undegraded":"2024-05-28T14:12:59.146108+0000","last_fullsized":"2024-05-28T14:12:59.146108+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162847+0000","last_change":"2024-05-28T14:12:59.159676+0000","last_active":"2024-05-28T14:12:59.162847+0000","last_peered":"2024-05-28T14:12:59.162847+0000","last_clean":"2024-05-28T14:12:59.162847+0000","last_became_active":"2024-05-28T14:12:59.159391+0000","last_became_peered":"2024-05-28T14:12:59.159391+0000","last_unstale":"2024-05-28T14:12:59.162847+0000","last_undegraded":"2024-05-28T14:12:59.162847+0000","last_fullsized":"2024-05-28T14:12:59.162847+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:25:49.893677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141618+0000","last_change":"2024-05-28T14:12:59.141618+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141618+0000","last_undegraded":"2024-05-28T14:12:59.141618+0000","last_fullsized":"2024-05-28T14:12:59.141618+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.142707+0000","last_change":"2024-05-28T14:12:59.153857+0000","last_active":"2024-05-28T14:13:00.142707+0000","last_peered":"2024-05-28T14:13:00.142707+0000","last_clean":"2024-05-28T14:13:00.142707+0000","last_became_active":"2024-05-28T14:12:59.153528+0000","last_became_peered":"2024-05-28T14:12:59.153528+0000","last_unstale":"2024-05-28T14:13:00.142707+0000","last_undegraded":"2024-05-28T14:13:00.142707+0000","last_fullsized":"2024-05-28T14:13:00.142707+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:46:44.183872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146666+0000","last_change":"2024-05-28T14:12:59.146666+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146666+0000","last_undegraded":"2024-05-28T14:12:59.146666+0000","last_fullsized":"2024-05-28T14:12:59.146666+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.158654+0000","last_change":"2024-05-28T14:12:59.153891+0000","last_active":"2024-05-28T14:12:59.158654+0000","last_peered":"2024-05-28T14:12:59.158654+0000","last_clean":"2024-05-28T14:12:59.158654+0000","last_became_active":"2024-05-28T14:12:59.153629+0000","last_became_peered":"2024-05-28T14:12:59.153629+0000","last_unstale":"2024-05-28T14:12:59.158654+0000","last_undegraded":"2024-05-28T14:12:59.158654+0000","last_fullsized":"2024-05-28T14:12:59.158654+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:56:25.534573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143944+0000","last_change":"2024-05-28T14:12:59.143944+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143944+0000","last_undegraded":"2024-05-28T14:12:59.143944+0000","last_fullsized":"2024-05-28T14:12:59.143944+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162520+0000","last_change":"2024-05-28T14:12:59.159929+0000","last_active":"2024-05-28T14:12:59.162520+0000","last_peered":"2024-05-28T14:12:59.162520+0000","last_clean":"2024-05-28T14:12:59.162520+0000","last_became_active":"2024-05-28T14:12:59.159646+0000","last_became_peered":"2024-05-28T14:12:59.159646+0000","last_unstale":"2024-05-28T14:12:59.162520+0000","last_undegraded":"2024-05-28T14:12:59.162520+0000","last_fullsized":"2024-05-28T14:12:59.162520+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:45:38.347737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.151926+0000","last_change":"2024-05-28T14:12:59.151926+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.151926+0000","last_undegraded":"2024-05-28T14:12:59.151926+0000","last_fullsized":"2024-05-28T14:12:59.151926+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.165506+0000","last_change":"2024-05-28T14:12:59.161594+0000","last_active":"2024-05-28T14:12:59.165506+0000","last_peered":"2024-05-28T14:12:59.165506+0000","last_clean":"2024-05-28T14:12:59.165506+0000","last_became_active":"2024-05-28T14:12:59.161326+0000","last_became_peered":"2024-05-28T14:12:59.161326+0000","last_unstale":"2024-05-28T14:12:59.165506+0000","last_undegraded":"2024-05-28T14:12:59.165506+0000","last_fullsized":"2024-05-28T14:12:59.165506+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:45:38.347737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142671+0000","last_change":"2024-05-28T14:12:59.142671+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142671+0000","last_undegraded":"2024-05-28T14:12:59.142671+0000","last_fullsized":"2024-05-28T14:12:59.142671+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.161105+0000","last_change":"2024-05-28T14:12:59.159230+0000","last_active":"2024-05-28T14:12:59.161105+0000","last_peered":"2024-05-28T14:12:59.161105+0000","last_clean":"2024-05-28T14:12:59.161105+0000","last_became_active":"2024-05-28T14:12:59.158844+0000","last_became_peered":"2024-05-28T14:12:59.158844+0000","last_unstale":"2024-05-28T14:12:59.161105+0000","last_undegraded":"2024-05-28T14:12:59.161105+0000","last_fullsized":"2024-05-28T14:12:59.161105+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:56:46.861095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142149+0000","last_change":"2024-05-28T14:12:59.142149+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142149+0000","last_undegraded":"2024-05-28T14:12:59.142149+0000","last_fullsized":"2024-05-28T14:12:59.142149+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162552+0000","last_change":"2024-05-28T14:12:59.159785+0000","last_active":"2024-05-28T14:12:59.162552+0000","last_peered":"2024-05-28T14:12:59.162552+0000","last_clean":"2024-05-28T14:12:59.162552+0000","last_became_active":"2024-05-28T14:12:59.159495+0000","last_became_peered":"2024-05-28T14:12:59.159495+0000","last_unstale":"2024-05-28T14:12:59.162552+0000","last_undegraded":"2024-05-28T14:12:59.162552+0000","last_fullsized":"2024-05-28T14:12:59.162552+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:15:36.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.153890+0000","last_change":"2024-05-28T14:12:59.153890+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.153890+0000","last_undegraded":"2024-05-28T14:12:59.153890+0000","last_fullsized":"2024-05-28T14:12:59.153890+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162693+0000","last_change":"2024-05-28T14:12:59.160575+0000","last_active":"2024-05-28T14:12:59.162693+0000","last_peered":"2024-05-28T14:12:59.162693+0000","last_clean":"2024-05-28T14:12:59.162693+0000","last_became_active":"2024-05-28T14:12:59.160277+0000","last_became_peered":"2024-05-28T14:12:59.160277+0000","last_unstale":"2024-05-28T14:12:59.162693+0000","last_undegraded":"2024-05-28T14:12:59.162693+0000","last_fullsized":"2024-05-28T14:12:59.162693+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:15:36.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143331+0000","last_change":"2024-05-28T14:12:59.143331+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143331+0000","last_undegraded":"2024-05-28T14:12:59.143331+0000","last_fullsized":"2024-05-28T14:12:59.143331+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.158079+0000","last_change":"2024-05-28T14:12:59.155180+0000","last_active":"2024-05-28T14:12:59.158079+0000","last_peered":"2024-05-28T14:12:59.158079+0000","last_clean":"2024-05-28T14:12:59.158079+0000","last_became_active":"2024-05-28T14:12:59.154912+0000","last_became_peered":"2024-05-28T14:12:59.154912+0000","last_unstale":"2024-05-28T14:12:59.158079+0000","last_undegraded":"2024-05-28T14:12:59.158079+0000","last_fullsized":"2024-05-28T14:12:59.158079+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:04:39.323754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143393+0000","last_change":"2024-05-28T14:12:59.143393+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143393+0000","last_undegraded":"2024-05-28T14:12:59.143393+0000","last_fullsized":"2024-05-28T14:12:59.143393+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.158481+0000","last_change":"2024-05-28T14:12:59.156463+0000","last_active":"2024-05-28T14:12:59.158481+0000","last_peered":"2024-05-28T14:12:59.158481+0000","last_clean":"2024-05-28T14:12:59.158481+0000","last_became_active":"2024-05-28T14:12:59.155528+0000","last_became_peered":"2024-05-28T14:12:59.155528+0000","last_unstale":"2024-05-28T14:12:59.158481+0000","last_undegraded":"2024-05-28T14:12:59.158481+0000","last_fullsized":"2024-05-28T14:12:59.158481+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:16:34.556526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146039+0000","last_change":"2024-05-28T14:12:59.146039+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146039+0000","last_undegraded":"2024-05-28T14:12:59.146039+0000","last_fullsized":"2024-05-28T14:12:59.146039+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.142671+0000","last_change":"2024-05-28T14:12:59.154458+0000","last_active":"2024-05-28T14:13:00.142671+0000","last_peered":"2024-05-28T14:13:00.142671+0000","last_clean":"2024-05-28T14:13:00.142671+0000","last_became_active":"2024-05-28T14:12:59.153652+0000","last_became_peered":"2024-05-28T14:12:59.153652+0000","last_unstale":"2024-05-28T14:13:00.142671+0000","last_undegraded":"2024-05-28T14:13:00.142671+0000","last_fullsized":"2024-05-28T14:13:00.142671+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:39:27.979374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.140964+0000","last_change":"2024-05-28T14:12:59.140964+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.140964+0000","last_undegraded":"2024-05-28T14:12:59.140964+0000","last_fullsized":"2024-05-28T14:12:59.140964+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.160877+0000","last_change":"2024-05-28T14:12:59.158941+0000","last_active":"2024-05-28T14:12:59.160877+0000","last_peered":"2024-05-28T14:12:59.160877+0000","last_clean":"2024-05-28T14:12:59.160877+0000","last_became_active":"2024-05-28T14:12:59.158680+0000","last_became_peered":"2024-05-28T14:12:59.158680+0000","last_unstale":"2024-05-28T14:12:59.160877+0000","last_undegraded":"2024-05-28T14:12:59.160877+0000","last_fullsized":"2024-05-28T14:12:59.160877+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:50:25.519025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143848+0000","last_change":"2024-05-28T14:12:59.143848+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143848+0000","last_undegraded":"2024-05-28T14:12:59.143848+0000","last_fullsized":"2024-05-28T14:12:59.143848+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.157321+0000","last_change":"2024-05-28T14:12:59.154352+0000","last_active":"2024-05-28T14:12:59.157321+0000","last_peered":"2024-05-28T14:12:59.157321+0000","last_clean":"2024-05-28T14:12:59.157321+0000","last_became_active":"2024-05-28T14:12:59.154075+0000","last_became_peered":"2024-05-28T14:12:59.154075+0000","last_unstale":"2024-05-28T14:12:59.157321+0000","last_undegraded":"2024-05-28T14:12:59.157321+0000","last_fullsized":"2024-05-28T14:12:59.157321+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:25:00.922583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":347,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.152540+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:12:59.152540+0000","last_peered":"2024-05-28T14:12:59.152540+0000","last_clean":"2024-05-28T14:12:59.152540+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:12:59.152540+0000","last_undegraded":"2024-05-28T14:12:59.152540+0000","last_fullsized":"2024-05-28T14:12:59.152540+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575028,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131652,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240188,"statfs":{"total":96636764160,"available":96501952512,"internally_reserved":0,"allocated":2023424,"data_stored":1213925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131080,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240760,"statfs":{"total":96636764160,"available":96502538240,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575030,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64436,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307404,"statfs":{"total":96636764160,"available":96570781696,"internally_reserved":0,"allocated":1417216,"data_stored":617226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144964,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226876,"statfs":{"total":96636764160,"available":96488321024,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575028,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122756,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249084,"statfs":{"total":96636764160,"available":96511062016,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":124254664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575029,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136640,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235200,"statfs":{"total":96636764160,"available":96496844800,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149120,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222720,"statfs":{"total":96636764160,"available":96484065280,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575029,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91396,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280444,"statfs":{"total":96636764160,"available":96543174656,"internally_reserved":0,"allocated":2023424,"data_stored":1213925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:13:04.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:13:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35dd800640 1 Processor -- start 2024-05-28T14:13:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35dd800640 1 -- start start 2024-05-28T14:13:04.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35dd800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815c990 0x7f35d80d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35dd800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f35d80d06a0 0x7f35d80b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35dd800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d80cd300 con 0x7f35d809bbd0 2024-05-28T14:13:04.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35dd800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f35d815cd70 con 0x7f35d80d06a0 2024-05-28T14:13:04.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35dd800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f35d80cd5e0 con 0x7f35d815c990 2024-05-28T14:13:04.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35d6a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815c990 0x7f35d80d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35d7400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f35d80d06a0 0x7f35d80b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35d6a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815c990 0x7f35d80d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:51794/0 (socket says 172.21.15.42:51794) 2024-05-28T14:13:04.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35d6a00640 1 -- 172.21.15.42:0/934181453 learned_addr learned my addr 172.21.15.42:0/934181453 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:04.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35cfe00640 1 --1- 172.21.15.42:0/934181453 >> v1:172.21.15.63:6789/0 conn(0x7f35d809bbd0 0x7f35d809aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:54110/0 (socket says 172.21.15.42:54110) 2024-05-28T14:13:04.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35d6a00640 1 -- 172.21.15.42:0/934181453 >> v2:172.21.15.63:3300/0 conn(0x7f35d80d06a0 msgr2=0x7f35d80b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:04.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35d6a00640 1 --2- 172.21.15.42:0/934181453 >> v2:172.21.15.63:3300/0 conn(0x7f35d80d06a0 0x7f35d80b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35d6a00640 1 -- 172.21.15.42:0/934181453 >> v1:172.21.15.63:6789/0 conn(0x7f35d809bbd0 legacy=0x7f35d809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35d6a00640 1 -- 172.21.15.42:0/934181453 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d80cd8c0 con 0x7f35d815c990 2024-05-28T14:13:04.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.851+0000 7f35d6000640 1 -- 172.21.15.42:0/934181453 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1941868287 0 0) 0x7f35d80cd300 con 0x7f35d809bbd0 2024-05-28T14:13:04.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.855+0000 7f35d6a00640 1 --2- 172.21.15.42:0/934181453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815c990 0x7f35d80d0160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009870 tx=0x7f35c801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d7dfe525f921807 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:04.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.855+0000 7f35d6000640 1 -- 172.21.15.42:0/934181453 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c80a95f0 con 0x7f35d815c990 2024-05-28T14:13:04.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.855+0000 7f35d6000640 1 -- 172.21.15.42:0/934181453 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c80a9790 con 0x7f35d815c990 2024-05-28T14:13:04.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.855+0000 7f35d6000640 1 -- 172.21.15.42:0/934181453 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c80a9b60 con 0x7f35d815c990 2024-05-28T14:13:04.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.855+0000 7f35dd800640 1 -- 172.21.15.42:0/934181453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815c990 msgr2=0x7f35d80d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.855+0000 7f35dd800640 1 --2- 172.21.15.42:0/934181453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815c990 0x7f35d80d0160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009870 tx=0x7f35c801c530 comp rx=0 tx=0).stop 2024-05-28T14:13:04.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.855+0000 7f35dd800640 1 -- 172.21.15.42:0/934181453 shutdown_connections 2024-05-28T14:13:04.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.855+0000 7f35dd800640 1 --2- 172.21.15.42:0/934181453 >> v2:172.21.15.63:3300/0 conn(0x7f35d80d06a0 0x7f35d80b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.855+0000 7f35dd800640 1 --2- 172.21.15.42:0/934181453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d815c990 0x7f35d80d0160 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.855+0000 7f35dd800640 1 -- 172.21.15.42:0/934181453 >> 172.21.15.42:0/934181453 conn(0x7f35d80b4080 msgr2=0x7f35d80b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:04.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.855+0000 7f35dd800640 1 -- 172.21.15.42:0/934181453 shutdown_connections 2024-05-28T14:13:04.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.855+0000 7f35dd800640 1 -- 172.21.15.42:0/934181453 wait complete. 2024-05-28T14:13:04.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35dd800640 1 Processor -- start 2024-05-28T14:13:04.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35dd800640 1 -- start start 2024-05-28T14:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35dd800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35d80d06a0 0x7f35d80afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35dd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35d815c990 0x7f35d81f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35dd800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d8156030 0x7f35d81f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35dd800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f35d809b2e0 con 0x7f35d815c990 2024-05-28T14:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35dd800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f35d809b160 con 0x7f35d80d06a0 2024-05-28T14:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35dd800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f35d80b6130 con 0x7f35d8156030 2024-05-28T14:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35cfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35d815c990 0x7f35d81f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35cfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35d815c990 0x7f35d81f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33946/0 (socket says 172.21.15.42:33946) 2024-05-28T14:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35cfe00640 1 -- 172.21.15.42:0/582995962 learned_addr learned my addr 172.21.15.42:0/582995962 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35d7400640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d8156030 0x7f35d81f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35d6a00640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35d80d06a0 0x7f35d80afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35cfe00640 1 -- 172.21.15.42:0/582995962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d8156030 msgr2=0x7f35d81f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35cfe00640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d8156030 0x7f35d81f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35cfe00640 1 -- 172.21.15.42:0/582995962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35d80d06a0 msgr2=0x7f35d80afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:04.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35cfe00640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35d80d06a0 0x7f35d80afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:04.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35cfe00640 1 -- 172.21.15.42:0/582995962 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d80b2f20 con 0x7f35d815c990 2024-05-28T14:13:04.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35d6a00640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35d80d06a0 0x7f35d80afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:04.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35cfe00640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35d815c990 0x7f35d81f64d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c520 tx=0x7f35c000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:04.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35cea00640 1 -- 172.21.15.42:0/582995962 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0018070 con 0x7f35d815c990 2024-05-28T14:13:04.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35cea00640 1 -- 172.21.15.42:0/582995962 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c00074e0 con 0x7f35d815c990 2024-05-28T14:13:04.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35dd800640 1 -- 172.21.15.42:0/582995962 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d8153680 con 0x7f35d815c990 2024-05-28T14:13:04.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35cea00640 1 -- 172.21.15.42:0/582995962 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c00049f0 con 0x7f35d815c990 2024-05-28T14:13:04.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35dd800640 1 -- 172.21.15.42:0/582995962 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d81566c0 con 0x7f35d815c990 2024-05-28T14:13:04.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.859+0000 7f35dd800640 1 -- 172.21.15.42:0/582995962 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d8068a90 con 0x7f35d815c990 2024-05-28T14:13:04.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.863+0000 7f35cea00640 1 -- 172.21.15.42:0/582995962 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f35c001b070 con 0x7f35d815c990 2024-05-28T14:13:04.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.863+0000 7f35cea00640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f35b4069b20 0x7f35b406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:04.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.863+0000 7f35cea00640 1 -- 172.21.15.42:0/582995962 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f35c0084bf0 con 0x7f35d815c990 2024-05-28T14:13:04.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.863+0000 7f35d6a00640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f35b4069b20 0x7f35b406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:04.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.863+0000 7f35d6a00640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f35b4069b20 0x7f35b406bfe0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f35c80ae000 tx=0x7f35c80a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:04.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:04.867+0000 7f35cea00640 1 -- 172.21.15.42:0/582995962 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f35c0056c10 con 0x7f35d815c990 2024-05-28T14:13:05.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.031+0000 7f35dd800640 1 -- 172.21.15.42:0/582995962 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35d81534c0 con 0x7f35b4069b20 2024-05-28T14:13:05.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.035+0000 7f35cea00640 1 -- 172.21.15.42:0/582995962 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247294 (secure 0 0 0) 0x7f35c80b25e0 con 0x7f35b4069b20 2024-05-28T14:13:05.024 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:13:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.039+0000 7f35dd800640 1 -- 172.21.15.42:0/582995962 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f35b4069b20 msgr2=0x7f35b406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.039+0000 7f35dd800640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f35b4069b20 0x7f35b406bfe0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f35c80ae000 tx=0x7f35c80a7000 comp rx=0 tx=0).stop 2024-05-28T14:13:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.039+0000 7f35dd800640 1 -- 172.21.15.42:0/582995962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35d815c990 msgr2=0x7f35d81f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.039+0000 7f35dd800640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35d815c990 0x7f35d81f64d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c520 tx=0x7f35c000c9e0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.039+0000 7f35dd800640 1 -- 172.21.15.42:0/582995962 shutdown_connections 2024-05-28T14:13:05.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.039+0000 7f35dd800640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f35b4069b20 0x7f35b406bfe0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.039+0000 7f35dd800640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35d8156030 0x7f35d81f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.039+0000 7f35dd800640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35d815c990 0x7f35d81f64d0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.039+0000 7f35dd800640 1 --2- 172.21.15.42:0/582995962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35d80d06a0 0x7f35d80afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.039+0000 7f35dd800640 1 -- 172.21.15.42:0/582995962 >> 172.21.15.42:0/582995962 conn(0x7f35d80b4080 msgr2=0x7f35d80ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.039+0000 7f35dd800640 1 -- 172.21.15.42:0/582995962 shutdown_connections 2024-05-28T14:13:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.039+0000 7f35dd800640 1 -- 172.21.15.42:0/582995962 wait complete. 2024-05-28T14:13:05.045 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":557,"stamp":"2024-05-28T14:13:03.164561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":972044,"kb_used_data":12316,"kb_used_omap":12,"kb_used_meta":959603,"kb_avail":754002676,"statfs":{"total":773094113280,"available":772098740224,"internally_reserved":0,"allocated":12611584,"data_stored":6162861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":982634034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.037878"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143716+0000","last_change":"2024-05-28T14:12:59.143716+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143716+0000","last_undegraded":"2024-05-28T14:12:59.143716+0000","last_fullsized":"2024-05-28T14:12:59.143716+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145111+0000","last_change":"2024-05-28T14:12:59.145111+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145111+0000","last_undegraded":"2024-05-28T14:12:59.145111+0000","last_fullsized":"2024-05-28T14:12:59.145111+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.150194+0000","last_change":"2024-05-28T14:12:59.150194+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.150194+0000","last_undegraded":"2024-05-28T14:12:59.150194+0000","last_fullsized":"2024-05-28T14:12:59.150194+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143795+0000","last_change":"2024-05-28T14:12:59.143795+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143795+0000","last_undegraded":"2024-05-28T14:12:59.143795+0000","last_fullsized":"2024-05-28T14:12:59.143795+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.151103+0000","last_change":"2024-05-28T14:13:00.146533+0000","last_active":"2024-05-28T14:13:00.151103+0000","last_peered":"2024-05-28T14:13:00.151103+0000","last_clean":"2024-05-28T14:13:00.151103+0000","last_became_active":"2024-05-28T14:13:00.145774+0000","last_became_peered":"2024-05-28T14:13:00.145774+0000","last_unstale":"2024-05-28T14:13:00.151103+0000","last_undegraded":"2024-05-28T14:13:00.151103+0000","last_fullsized":"2024-05-28T14:13:00.151103+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:29:07.224328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141349+0000","last_change":"2024-05-28T14:12:59.141349+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141349+0000","last_undegraded":"2024-05-28T14:12:59.141349+0000","last_fullsized":"2024-05-28T14:12:59.141349+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143106+0000","last_change":"2024-05-28T14:12:59.143106+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143106+0000","last_undegraded":"2024-05-28T14:12:59.143106+0000","last_fullsized":"2024-05-28T14:12:59.143106+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.147478+0000","last_change":"2024-05-28T14:12:59.147478+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.147478+0000","last_undegraded":"2024-05-28T14:12:59.147478+0000","last_fullsized":"2024-05-28T14:12:59.147478+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143317+0000","last_change":"2024-05-28T14:12:59.143317+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143317+0000","last_undegraded":"2024-05-28T14:12:59.143317+0000","last_fullsized":"2024-05-28T14:12:59.143317+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142610+0000","last_change":"2024-05-28T14:12:59.142610+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142610+0000","last_undegraded":"2024-05-28T14:12:59.142610+0000","last_fullsized":"2024-05-28T14:12:59.142610+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144639+0000","last_change":"2024-05-28T14:12:59.144639+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144639+0000","last_undegraded":"2024-05-28T14:12:59.144639+0000","last_fullsized":"2024-05-28T14:12:59.144639+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143340+0000","last_change":"2024-05-28T14:12:59.143340+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143340+0000","last_undegraded":"2024-05-28T14:12:59.143340+0000","last_fullsized":"2024-05-28T14:12:59.143340+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146701+0000","last_change":"2024-05-28T14:12:59.146701+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146701+0000","last_undegraded":"2024-05-28T14:12:59.146701+0000","last_fullsized":"2024-05-28T14:12:59.146701+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.147196+0000","last_change":"2024-05-28T14:12:59.147196+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.147196+0000","last_undegraded":"2024-05-28T14:12:59.147196+0000","last_fullsized":"2024-05-28T14:12:59.147196+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141425+0000","last_change":"2024-05-28T14:12:59.141425+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141425+0000","last_undegraded":"2024-05-28T14:12:59.141425+0000","last_fullsized":"2024-05-28T14:12:59.141425+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144638+0000","last_change":"2024-05-28T14:12:59.144638+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144638+0000","last_undegraded":"2024-05-28T14:12:59.144638+0000","last_fullsized":"2024-05-28T14:12:59.144638+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144519+0000","last_change":"2024-05-28T14:12:59.144519+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144519+0000","last_undegraded":"2024-05-28T14:12:59.144519+0000","last_fullsized":"2024-05-28T14:12:59.144519+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.152420+0000","last_change":"2024-05-28T14:12:59.152420+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.152420+0000","last_undegraded":"2024-05-28T14:12:59.152420+0000","last_fullsized":"2024-05-28T14:12:59.152420+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.149551+0000","last_change":"2024-05-28T14:12:59.149551+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.149551+0000","last_undegraded":"2024-05-28T14:12:59.149551+0000","last_fullsized":"2024-05-28T14:12:59.149551+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143766+0000","last_change":"2024-05-28T14:12:59.143766+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143766+0000","last_undegraded":"2024-05-28T14:12:59.143766+0000","last_fullsized":"2024-05-28T14:12:59.143766+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142806+0000","last_change":"2024-05-28T14:12:59.142806+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142806+0000","last_undegraded":"2024-05-28T14:12:59.142806+0000","last_fullsized":"2024-05-28T14:12:59.142806+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.147836+0000","last_change":"2024-05-28T14:12:59.147836+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.147836+0000","last_undegraded":"2024-05-28T14:12:59.147836+0000","last_fullsized":"2024-05-28T14:12:59.147836+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142797+0000","last_change":"2024-05-28T14:12:59.142797+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142797+0000","last_undegraded":"2024-05-28T14:12:59.142797+0000","last_fullsized":"2024-05-28T14:12:59.142797+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144378+0000","last_change":"2024-05-28T14:12:59.144378+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144378+0000","last_undegraded":"2024-05-28T14:12:59.144378+0000","last_fullsized":"2024-05-28T14:12:59.144378+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143979+0000","last_change":"2024-05-28T14:12:59.143979+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143979+0000","last_undegraded":"2024-05-28T14:12:59.143979+0000","last_fullsized":"2024-05-28T14:12:59.143979+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142104+0000","last_change":"2024-05-28T14:12:59.142104+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142104+0000","last_undegraded":"2024-05-28T14:12:59.142104+0000","last_fullsized":"2024-05-28T14:12:59.142104+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146411+0000","last_change":"2024-05-28T14:12:59.146411+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146411+0000","last_undegraded":"2024-05-28T14:12:59.146411+0000","last_fullsized":"2024-05-28T14:12:59.146411+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145670+0000","last_change":"2024-05-28T14:12:59.145670+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145670+0000","last_undegraded":"2024-05-28T14:12:59.145670+0000","last_fullsized":"2024-05-28T14:12:59.145670+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.148734+0000","last_change":"2024-05-28T14:12:59.148734+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.148734+0000","last_undegraded":"2024-05-28T14:12:59.148734+0000","last_fullsized":"2024-05-28T14:12:59.148734+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143300+0000","last_change":"2024-05-28T14:12:59.143300+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143300+0000","last_undegraded":"2024-05-28T14:12:59.143300+0000","last_fullsized":"2024-05-28T14:12:59.143300+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141526+0000","last_change":"2024-05-28T14:12:59.141526+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141526+0000","last_undegraded":"2024-05-28T14:12:59.141526+0000","last_fullsized":"2024-05-28T14:12:59.141526+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.140649+0000","last_change":"2024-05-28T14:12:59.140649+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.140649+0000","last_undegraded":"2024-05-28T14:12:59.140649+0000","last_fullsized":"2024-05-28T14:12:59.140649+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145011+0000","last_change":"2024-05-28T14:12:59.145011+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145011+0000","last_undegraded":"2024-05-28T14:12:59.145011+0000","last_fullsized":"2024-05-28T14:12:59.145011+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.158758+0000","last_change":"2024-05-28T14:12:59.156662+0000","last_active":"2024-05-28T14:12:59.158758+0000","last_peered":"2024-05-28T14:12:59.158758+0000","last_clean":"2024-05-28T14:12:59.158758+0000","last_became_active":"2024-05-28T14:12:59.156379+0000","last_became_peered":"2024-05-28T14:12:59.156379+0000","last_unstale":"2024-05-28T14:12:59.158758+0000","last_undegraded":"2024-05-28T14:12:59.158758+0000","last_fullsized":"2024-05-28T14:12:59.158758+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:39:27.979374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.151621+0000","last_change":"2024-05-28T14:12:59.151621+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.151621+0000","last_undegraded":"2024-05-28T14:12:59.151621+0000","last_fullsized":"2024-05-28T14:12:59.151621+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.161663+0000","last_change":"2024-05-28T14:12:59.160363+0000","last_active":"2024-05-28T14:12:59.161663+0000","last_peered":"2024-05-28T14:12:59.161663+0000","last_clean":"2024-05-28T14:12:59.161663+0000","last_became_active":"2024-05-28T14:12:59.159749+0000","last_became_peered":"2024-05-28T14:12:59.159749+0000","last_unstale":"2024-05-28T14:12:59.161663+0000","last_undegraded":"2024-05-28T14:12:59.161663+0000","last_fullsized":"2024-05-28T14:12:59.161663+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:15:00.386236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.150359+0000","last_change":"2024-05-28T14:12:59.150359+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.150359+0000","last_undegraded":"2024-05-28T14:12:59.150359+0000","last_fullsized":"2024-05-28T14:12:59.150359+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.171344+0000","last_change":"2024-05-28T14:12:59.155476+0000","last_active":"2024-05-28T14:13:00.171344+0000","last_peered":"2024-05-28T14:13:00.171344+0000","last_clean":"2024-05-28T14:13:00.171344+0000","last_became_active":"2024-05-28T14:12:59.155187+0000","last_became_peered":"2024-05-28T14:12:59.155187+0000","last_unstale":"2024-05-28T14:13:00.171344+0000","last_undegraded":"2024-05-28T14:13:00.171344+0000","last_fullsized":"2024-05-28T14:13:00.171344+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:07:49.480989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.148929+0000","last_change":"2024-05-28T14:12:59.148929+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.148929+0000","last_undegraded":"2024-05-28T14:12:59.148929+0000","last_fullsized":"2024-05-28T14:12:59.148929+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.164533+0000","last_change":"2024-05-28T14:12:59.164490+0000","last_active":"2024-05-28T14:13:00.164533+0000","last_peered":"2024-05-28T14:13:00.164533+0000","last_clean":"2024-05-28T14:13:00.164533+0000","last_became_active":"2024-05-28T14:12:59.163870+0000","last_became_peered":"2024-05-28T14:12:59.163870+0000","last_unstale":"2024-05-28T14:13:00.164533+0000","last_undegraded":"2024-05-28T14:13:00.164533+0000","last_fullsized":"2024-05-28T14:13:00.164533+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:06.681422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141097+0000","last_change":"2024-05-28T14:12:59.141097+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141097+0000","last_undegraded":"2024-05-28T14:12:59.141097+0000","last_fullsized":"2024-05-28T14:12:59.141097+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.159332+0000","last_change":"2024-05-28T14:12:59.157949+0000","last_active":"2024-05-28T14:12:59.159332+0000","last_peered":"2024-05-28T14:12:59.159332+0000","last_clean":"2024-05-28T14:12:59.159332+0000","last_became_active":"2024-05-28T14:12:59.157600+0000","last_became_peered":"2024-05-28T14:12:59.157600+0000","last_unstale":"2024-05-28T14:12:59.159332+0000","last_undegraded":"2024-05-28T14:12:59.159332+0000","last_fullsized":"2024-05-28T14:12:59.159332+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:46:44.183872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143903+0000","last_change":"2024-05-28T14:12:59.143903+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143903+0000","last_undegraded":"2024-05-28T14:12:59.143903+0000","last_fullsized":"2024-05-28T14:12:59.143903+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.161141+0000","last_change":"2024-05-28T14:12:59.158675+0000","last_active":"2024-05-28T14:12:59.161141+0000","last_peered":"2024-05-28T14:12:59.161141+0000","last_clean":"2024-05-28T14:12:59.161141+0000","last_became_active":"2024-05-28T14:12:59.158378+0000","last_became_peered":"2024-05-28T14:12:59.158378+0000","last_unstale":"2024-05-28T14:12:59.161141+0000","last_undegraded":"2024-05-28T14:12:59.161141+0000","last_fullsized":"2024-05-28T14:12:59.161141+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:50:25.519025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144181+0000","last_change":"2024-05-28T14:12:59.144181+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144181+0000","last_undegraded":"2024-05-28T14:12:59.144181+0000","last_fullsized":"2024-05-28T14:12:59.144181+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.163331+0000","last_change":"2024-05-28T14:12:59.162720+0000","last_active":"2024-05-28T14:12:59.163331+0000","last_peered":"2024-05-28T14:12:59.163331+0000","last_clean":"2024-05-28T14:12:59.163331+0000","last_became_active":"2024-05-28T14:12:59.162364+0000","last_became_peered":"2024-05-28T14:12:59.162364+0000","last_unstale":"2024-05-28T14:12:59.163331+0000","last_undegraded":"2024-05-28T14:12:59.163331+0000","last_fullsized":"2024-05-28T14:12:59.163331+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:51:48.221003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143666+0000","last_change":"2024-05-28T14:12:59.143666+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143666+0000","last_undegraded":"2024-05-28T14:12:59.143666+0000","last_fullsized":"2024-05-28T14:12:59.143666+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.165719+0000","last_change":"2024-05-28T14:12:59.163864+0000","last_active":"2024-05-28T14:12:59.165719+0000","last_peered":"2024-05-28T14:12:59.165719+0000","last_clean":"2024-05-28T14:12:59.165719+0000","last_became_active":"2024-05-28T14:12:59.163528+0000","last_became_peered":"2024-05-28T14:12:59.163528+0000","last_unstale":"2024-05-28T14:12:59.165719+0000","last_undegraded":"2024-05-28T14:12:59.165719+0000","last_fullsized":"2024-05-28T14:12:59.165719+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:07:17.818267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145598+0000","last_change":"2024-05-28T14:12:59.145598+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145598+0000","last_undegraded":"2024-05-28T14:12:59.145598+0000","last_fullsized":"2024-05-28T14:12:59.145598+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.144976+0000","last_change":"2024-05-28T14:12:59.165251+0000","last_active":"2024-05-28T14:13:00.144976+0000","last_peered":"2024-05-28T14:13:00.144976+0000","last_clean":"2024-05-28T14:13:00.144976+0000","last_became_active":"2024-05-28T14:12:59.164960+0000","last_became_peered":"2024-05-28T14:12:59.164960+0000","last_unstale":"2024-05-28T14:13:00.144976+0000","last_undegraded":"2024-05-28T14:13:00.144976+0000","last_fullsized":"2024-05-28T14:13:00.144976+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:47:24.425191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.151052+0000","last_change":"2024-05-28T14:12:59.151052+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.151052+0000","last_undegraded":"2024-05-28T14:12:59.151052+0000","last_fullsized":"2024-05-28T14:12:59.151052+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.161182+0000","last_change":"2024-05-28T14:12:59.157811+0000","last_active":"2024-05-28T14:12:59.161182+0000","last_peered":"2024-05-28T14:12:59.161182+0000","last_clean":"2024-05-28T14:12:59.161182+0000","last_became_active":"2024-05-28T14:12:59.157503+0000","last_became_peered":"2024-05-28T14:12:59.157503+0000","last_unstale":"2024-05-28T14:12:59.161182+0000","last_undegraded":"2024-05-28T14:12:59.161182+0000","last_fullsized":"2024-05-28T14:12:59.161182+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:53:50.157477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.145229+0000","last_change":"2024-05-28T14:12:59.145229+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.145229+0000","last_undegraded":"2024-05-28T14:12:59.145229+0000","last_fullsized":"2024-05-28T14:12:59.145229+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.165554+0000","last_change":"2024-05-28T14:12:59.163447+0000","last_active":"2024-05-28T14:12:59.165554+0000","last_peered":"2024-05-28T14:12:59.165554+0000","last_clean":"2024-05-28T14:12:59.165554+0000","last_became_active":"2024-05-28T14:12:59.163160+0000","last_became_peered":"2024-05-28T14:12:59.163160+0000","last_unstale":"2024-05-28T14:12:59.165554+0000","last_undegraded":"2024-05-28T14:12:59.165554+0000","last_fullsized":"2024-05-28T14:12:59.165554+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:02:28.323335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143506+0000","last_change":"2024-05-28T14:12:59.143506+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143506+0000","last_undegraded":"2024-05-28T14:12:59.143506+0000","last_fullsized":"2024-05-28T14:12:59.143506+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.166405+0000","last_change":"2024-05-28T14:12:59.164229+0000","last_active":"2024-05-28T14:12:59.166405+0000","last_peered":"2024-05-28T14:12:59.166405+0000","last_clean":"2024-05-28T14:12:59.166405+0000","last_became_active":"2024-05-28T14:12:59.163931+0000","last_became_peered":"2024-05-28T14:12:59.163931+0000","last_unstale":"2024-05-28T14:12:59.166405+0000","last_undegraded":"2024-05-28T14:12:59.166405+0000","last_fullsized":"2024-05-28T14:12:59.166405+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:17:03.844751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144233+0000","last_change":"2024-05-28T14:12:59.144233+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144233+0000","last_undegraded":"2024-05-28T14:12:59.144233+0000","last_fullsized":"2024-05-28T14:12:59.144233+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162823+0000","last_change":"2024-05-28T14:12:59.155868+0000","last_active":"2024-05-28T14:12:59.162823+0000","last_peered":"2024-05-28T14:12:59.162823+0000","last_clean":"2024-05-28T14:12:59.162823+0000","last_became_active":"2024-05-28T14:12:59.155584+0000","last_became_peered":"2024-05-28T14:12:59.155584+0000","last_unstale":"2024-05-28T14:12:59.162823+0000","last_undegraded":"2024-05-28T14:12:59.162823+0000","last_fullsized":"2024-05-28T14:12:59.162823+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:03:16.477026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144770+0000","last_change":"2024-05-28T14:12:59.144770+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144770+0000","last_undegraded":"2024-05-28T14:12:59.144770+0000","last_fullsized":"2024-05-28T14:12:59.144770+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162280+0000","last_change":"2024-05-28T14:12:59.161033+0000","last_active":"2024-05-28T14:12:59.162280+0000","last_peered":"2024-05-28T14:12:59.162280+0000","last_clean":"2024-05-28T14:12:59.162280+0000","last_became_active":"2024-05-28T14:12:59.160714+0000","last_became_peered":"2024-05-28T14:12:59.160714+0000","last_unstale":"2024-05-28T14:12:59.162280+0000","last_undegraded":"2024-05-28T14:12:59.162280+0000","last_fullsized":"2024-05-28T14:12:59.162280+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:41:49.719823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.144691+0000","last_change":"2024-05-28T14:12:59.144691+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.144691+0000","last_undegraded":"2024-05-28T14:12:59.144691+0000","last_fullsized":"2024-05-28T14:12:59.144691+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"116'8","reported_seq":20,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.170210+0000","last_change":"2024-05-28T14:12:59.154364+0000","last_active":"2024-05-28T14:13:00.170210+0000","last_peered":"2024-05-28T14:13:00.170210+0000","last_clean":"2024-05-28T14:13:00.170210+0000","last_became_active":"2024-05-28T14:12:59.152108+0000","last_became_peered":"2024-05-28T14:12:59.152108+0000","last_unstale":"2024-05-28T14:13:00.170210+0000","last_undegraded":"2024-05-28T14:13:00.170210+0000","last_fullsized":"2024-05-28T14:13:00.170210+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:23.615288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.159293+0000","last_change":"2024-05-28T14:13:00.155655+0000","last_active":"2024-05-28T14:13:00.159293+0000","last_peered":"2024-05-28T14:13:00.159293+0000","last_clean":"2024-05-28T14:13:00.159293+0000","last_became_active":"2024-05-28T14:13:00.155230+0000","last_became_peered":"2024-05-28T14:13:00.155230+0000","last_unstale":"2024-05-28T14:13:00.159293+0000","last_undegraded":"2024-05-28T14:13:00.159293+0000","last_fullsized":"2024-05-28T14:13:00.159293+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:23:08.600237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.160459+0000","last_change":"2024-05-28T14:12:59.158148+0000","last_active":"2024-05-28T14:12:59.160459+0000","last_peered":"2024-05-28T14:12:59.160459+0000","last_clean":"2024-05-28T14:12:59.160459+0000","last_became_active":"2024-05-28T14:12:59.156240+0000","last_became_peered":"2024-05-28T14:12:59.156240+0000","last_unstale":"2024-05-28T14:12:59.160459+0000","last_undegraded":"2024-05-28T14:12:59.160459+0000","last_fullsized":"2024-05-28T14:12:59.160459+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:53:50.157477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.150512+0000","last_change":"2024-05-28T14:12:59.150512+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.150512+0000","last_undegraded":"2024-05-28T14:12:59.150512+0000","last_fullsized":"2024-05-28T14:12:59.150512+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.171402+0000","last_change":"2024-05-28T14:12:59.154024+0000","last_active":"2024-05-28T14:13:00.171402+0000","last_peered":"2024-05-28T14:13:00.171402+0000","last_clean":"2024-05-28T14:13:00.171402+0000","last_became_active":"2024-05-28T14:12:59.153637+0000","last_became_peered":"2024-05-28T14:12:59.153637+0000","last_unstale":"2024-05-28T14:13:00.171402+0000","last_undegraded":"2024-05-28T14:13:00.171402+0000","last_fullsized":"2024-05-28T14:13:00.171402+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:24:19.180753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.148185+0000","last_change":"2024-05-28T14:12:59.148185+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.148185+0000","last_undegraded":"2024-05-28T14:12:59.148185+0000","last_fullsized":"2024-05-28T14:12:59.148185+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162705+0000","last_change":"2024-05-28T14:12:59.161950+0000","last_active":"2024-05-28T14:12:59.162705+0000","last_peered":"2024-05-28T14:12:59.162705+0000","last_clean":"2024-05-28T14:12:59.162705+0000","last_became_active":"2024-05-28T14:12:59.161536+0000","last_became_peered":"2024-05-28T14:12:59.161536+0000","last_unstale":"2024-05-28T14:12:59.162705+0000","last_undegraded":"2024-05-28T14:12:59.162705+0000","last_fullsized":"2024-05-28T14:12:59.162705+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:28:45.886716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.147735+0000","last_change":"2024-05-28T14:12:59.147735+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.147735+0000","last_undegraded":"2024-05-28T14:12:59.147735+0000","last_fullsized":"2024-05-28T14:12:59.147735+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.157718+0000","last_change":"2024-05-28T14:12:59.155568+0000","last_active":"2024-05-28T14:12:59.157718+0000","last_peered":"2024-05-28T14:12:59.157718+0000","last_clean":"2024-05-28T14:12:59.157718+0000","last_became_active":"2024-05-28T14:12:59.154202+0000","last_became_peered":"2024-05-28T14:12:59.154202+0000","last_unstale":"2024-05-28T14:12:59.157718+0000","last_undegraded":"2024-05-28T14:12:59.157718+0000","last_fullsized":"2024-05-28T14:12:59.157718+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:23.615288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146108+0000","last_change":"2024-05-28T14:12:59.146108+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146108+0000","last_undegraded":"2024-05-28T14:12:59.146108+0000","last_fullsized":"2024-05-28T14:12:59.146108+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162847+0000","last_change":"2024-05-28T14:12:59.159676+0000","last_active":"2024-05-28T14:12:59.162847+0000","last_peered":"2024-05-28T14:12:59.162847+0000","last_clean":"2024-05-28T14:12:59.162847+0000","last_became_active":"2024-05-28T14:12:59.159391+0000","last_became_peered":"2024-05-28T14:12:59.159391+0000","last_unstale":"2024-05-28T14:12:59.162847+0000","last_undegraded":"2024-05-28T14:12:59.162847+0000","last_fullsized":"2024-05-28T14:12:59.162847+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:25:49.893677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.141618+0000","last_change":"2024-05-28T14:12:59.141618+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.141618+0000","last_undegraded":"2024-05-28T14:12:59.141618+0000","last_fullsized":"2024-05-28T14:12:59.141618+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.142707+0000","last_change":"2024-05-28T14:12:59.153857+0000","last_active":"2024-05-28T14:13:00.142707+0000","last_peered":"2024-05-28T14:13:00.142707+0000","last_clean":"2024-05-28T14:13:00.142707+0000","last_became_active":"2024-05-28T14:12:59.153528+0000","last_became_peered":"2024-05-28T14:12:59.153528+0000","last_unstale":"2024-05-28T14:13:00.142707+0000","last_undegraded":"2024-05-28T14:13:00.142707+0000","last_fullsized":"2024-05-28T14:13:00.142707+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:46:44.183872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146666+0000","last_change":"2024-05-28T14:12:59.146666+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146666+0000","last_undegraded":"2024-05-28T14:12:59.146666+0000","last_fullsized":"2024-05-28T14:12:59.146666+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.158654+0000","last_change":"2024-05-28T14:12:59.153891+0000","last_active":"2024-05-28T14:12:59.158654+0000","last_peered":"2024-05-28T14:12:59.158654+0000","last_clean":"2024-05-28T14:12:59.158654+0000","last_became_active":"2024-05-28T14:12:59.153629+0000","last_became_peered":"2024-05-28T14:12:59.153629+0000","last_unstale":"2024-05-28T14:12:59.158654+0000","last_undegraded":"2024-05-28T14:12:59.158654+0000","last_fullsized":"2024-05-28T14:12:59.158654+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:56:25.534573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143944+0000","last_change":"2024-05-28T14:12:59.143944+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143944+0000","last_undegraded":"2024-05-28T14:12:59.143944+0000","last_fullsized":"2024-05-28T14:12:59.143944+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162520+0000","last_change":"2024-05-28T14:12:59.159929+0000","last_active":"2024-05-28T14:12:59.162520+0000","last_peered":"2024-05-28T14:12:59.162520+0000","last_clean":"2024-05-28T14:12:59.162520+0000","last_became_active":"2024-05-28T14:12:59.159646+0000","last_became_peered":"2024-05-28T14:12:59.159646+0000","last_unstale":"2024-05-28T14:12:59.162520+0000","last_undegraded":"2024-05-28T14:12:59.162520+0000","last_fullsized":"2024-05-28T14:12:59.162520+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:45:38.347737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.151926+0000","last_change":"2024-05-28T14:12:59.151926+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.151926+0000","last_undegraded":"2024-05-28T14:12:59.151926+0000","last_fullsized":"2024-05-28T14:12:59.151926+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.165506+0000","last_change":"2024-05-28T14:12:59.161594+0000","last_active":"2024-05-28T14:12:59.165506+0000","last_peered":"2024-05-28T14:12:59.165506+0000","last_clean":"2024-05-28T14:12:59.165506+0000","last_became_active":"2024-05-28T14:12:59.161326+0000","last_became_peered":"2024-05-28T14:12:59.161326+0000","last_unstale":"2024-05-28T14:12:59.165506+0000","last_undegraded":"2024-05-28T14:12:59.165506+0000","last_fullsized":"2024-05-28T14:12:59.165506+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:45:38.347737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142671+0000","last_change":"2024-05-28T14:12:59.142671+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142671+0000","last_undegraded":"2024-05-28T14:12:59.142671+0000","last_fullsized":"2024-05-28T14:12:59.142671+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.161105+0000","last_change":"2024-05-28T14:12:59.159230+0000","last_active":"2024-05-28T14:12:59.161105+0000","last_peered":"2024-05-28T14:12:59.161105+0000","last_clean":"2024-05-28T14:12:59.161105+0000","last_became_active":"2024-05-28T14:12:59.158844+0000","last_became_peered":"2024-05-28T14:12:59.158844+0000","last_unstale":"2024-05-28T14:12:59.161105+0000","last_undegraded":"2024-05-28T14:12:59.161105+0000","last_fullsized":"2024-05-28T14:12:59.161105+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:56:46.861095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.142149+0000","last_change":"2024-05-28T14:12:59.142149+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.142149+0000","last_undegraded":"2024-05-28T14:12:59.142149+0000","last_fullsized":"2024-05-28T14:12:59.142149+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162552+0000","last_change":"2024-05-28T14:12:59.159785+0000","last_active":"2024-05-28T14:12:59.162552+0000","last_peered":"2024-05-28T14:12:59.162552+0000","last_clean":"2024-05-28T14:12:59.162552+0000","last_became_active":"2024-05-28T14:12:59.159495+0000","last_became_peered":"2024-05-28T14:12:59.159495+0000","last_unstale":"2024-05-28T14:12:59.162552+0000","last_undegraded":"2024-05-28T14:12:59.162552+0000","last_fullsized":"2024-05-28T14:12:59.162552+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:15:36.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.153890+0000","last_change":"2024-05-28T14:12:59.153890+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.153890+0000","last_undegraded":"2024-05-28T14:12:59.153890+0000","last_fullsized":"2024-05-28T14:12:59.153890+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.162693+0000","last_change":"2024-05-28T14:12:59.160575+0000","last_active":"2024-05-28T14:12:59.162693+0000","last_peered":"2024-05-28T14:12:59.162693+0000","last_clean":"2024-05-28T14:12:59.162693+0000","last_became_active":"2024-05-28T14:12:59.160277+0000","last_became_peered":"2024-05-28T14:12:59.160277+0000","last_unstale":"2024-05-28T14:12:59.162693+0000","last_undegraded":"2024-05-28T14:12:59.162693+0000","last_fullsized":"2024-05-28T14:12:59.162693+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:15:36.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143331+0000","last_change":"2024-05-28T14:12:59.143331+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143331+0000","last_undegraded":"2024-05-28T14:12:59.143331+0000","last_fullsized":"2024-05-28T14:12:59.143331+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.158079+0000","last_change":"2024-05-28T14:12:59.155180+0000","last_active":"2024-05-28T14:12:59.158079+0000","last_peered":"2024-05-28T14:12:59.158079+0000","last_clean":"2024-05-28T14:12:59.158079+0000","last_became_active":"2024-05-28T14:12:59.154912+0000","last_became_peered":"2024-05-28T14:12:59.154912+0000","last_unstale":"2024-05-28T14:12:59.158079+0000","last_undegraded":"2024-05-28T14:12:59.158079+0000","last_fullsized":"2024-05-28T14:12:59.158079+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:04:39.323754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143393+0000","last_change":"2024-05-28T14:12:59.143393+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143393+0000","last_undegraded":"2024-05-28T14:12:59.143393+0000","last_fullsized":"2024-05-28T14:12:59.143393+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.158481+0000","last_change":"2024-05-28T14:12:59.156463+0000","last_active":"2024-05-28T14:12:59.158481+0000","last_peered":"2024-05-28T14:12:59.158481+0000","last_clean":"2024-05-28T14:12:59.158481+0000","last_became_active":"2024-05-28T14:12:59.155528+0000","last_became_peered":"2024-05-28T14:12:59.155528+0000","last_unstale":"2024-05-28T14:12:59.158481+0000","last_undegraded":"2024-05-28T14:12:59.158481+0000","last_fullsized":"2024-05-28T14:12:59.158481+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:16:34.556526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.146039+0000","last_change":"2024-05-28T14:12:59.146039+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.146039+0000","last_undegraded":"2024-05-28T14:12:59.146039+0000","last_fullsized":"2024-05-28T14:12:59.146039+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.142671+0000","last_change":"2024-05-28T14:12:59.154458+0000","last_active":"2024-05-28T14:13:00.142671+0000","last_peered":"2024-05-28T14:13:00.142671+0000","last_clean":"2024-05-28T14:13:00.142671+0000","last_became_active":"2024-05-28T14:12:59.153652+0000","last_became_peered":"2024-05-28T14:12:59.153652+0000","last_unstale":"2024-05-28T14:13:00.142671+0000","last_undegraded":"2024-05-28T14:13:00.142671+0000","last_fullsized":"2024-05-28T14:13:00.142671+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:39:27.979374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.140964+0000","last_change":"2024-05-28T14:12:59.140964+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.140964+0000","last_undegraded":"2024-05-28T14:12:59.140964+0000","last_fullsized":"2024-05-28T14:12:59.140964+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.160877+0000","last_change":"2024-05-28T14:12:59.158941+0000","last_active":"2024-05-28T14:12:59.160877+0000","last_peered":"2024-05-28T14:12:59.160877+0000","last_clean":"2024-05-28T14:12:59.160877+0000","last_became_active":"2024-05-28T14:12:59.158680+0000","last_became_peered":"2024-05-28T14:12:59.158680+0000","last_unstale":"2024-05-28T14:12:59.160877+0000","last_undegraded":"2024-05-28T14:12:59.160877+0000","last_fullsized":"2024-05-28T14:12:59.160877+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:50:25.519025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-28T14:12:59.143848+0000","last_change":"2024-05-28T14:12:59.143848+0000","last_active":"2024-05-28T14:12:59.126694+0000","last_peered":"2024-05-28T14:12:59.126694+0000","last_clean":"2024-05-28T14:12:59.126694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-28T14:12:59.143848+0000","last_undegraded":"2024-05-28T14:12:59.143848+0000","last_fullsized":"2024-05-28T14:12:59.143848+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.157321+0000","last_change":"2024-05-28T14:12:59.154352+0000","last_active":"2024-05-28T14:12:59.157321+0000","last_peered":"2024-05-28T14:12:59.157321+0000","last_clean":"2024-05-28T14:12:59.157321+0000","last_became_active":"2024-05-28T14:12:59.154075+0000","last_became_peered":"2024-05-28T14:12:59.154075+0000","last_unstale":"2024-05-28T14:12:59.157321+0000","last_undegraded":"2024-05-28T14:12:59.157321+0000","last_fullsized":"2024-05-28T14:12:59.157321+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:25:00.922583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":347,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-28T14:12:59.152540+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:12:59.152540+0000","last_peered":"2024-05-28T14:12:59.152540+0000","last_clean":"2024-05-28T14:12:59.152540+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:12:59.152540+0000","last_undegraded":"2024-05-28T14:12:59.152540+0000","last_fullsized":"2024-05-28T14:12:59.152540+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575028,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131652,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240188,"statfs":{"total":96636764160,"available":96501952512,"internally_reserved":0,"allocated":2023424,"data_stored":1213925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131080,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240760,"statfs":{"total":96636764160,"available":96502538240,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575030,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64436,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307404,"statfs":{"total":96636764160,"available":96570781696,"internally_reserved":0,"allocated":1417216,"data_stored":617226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144964,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226876,"statfs":{"total":96636764160,"available":96488321024,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575028,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122756,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249084,"statfs":{"total":96636764160,"available":96511062016,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":124254664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575029,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136640,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235200,"statfs":{"total":96636764160,"available":96496844800,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149120,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222720,"statfs":{"total":96636764160,"available":96484065280,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575029,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91396,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280444,"statfs":{"total":96636764160,"available":96543174656,"internally_reserved":0,"allocated":2023424,"data_stored":1213925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:13:05.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3f is not active+clean 2024-05-28T14:13:05.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143716+0000', 'last_change': '2024-05-28T14:12:59.143716+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143716+0000', 'last_undegraded': '2024-05-28T14:12:59.143716+0000', 'last_fullsized': '2024-05-28T14:12:59.143716+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:13:05.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3e is not active+clean 2024-05-28T14:13:05.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.145111+0000', 'last_change': '2024-05-28T14:12:59.145111+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.145111+0000', 'last_undegraded': '2024-05-28T14:12:59.145111+0000', 'last_fullsized': '2024-05-28T14:12:59.145111+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:13:05.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3d is not active+clean 2024-05-28T14:13:05.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.150194+0000', 'last_change': '2024-05-28T14:12:59.150194+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.150194+0000', 'last_undegraded': '2024-05-28T14:12:59.150194+0000', 'last_fullsized': '2024-05-28T14:12:59.150194+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:13:05.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3c is not active+clean 2024-05-28T14:13:05.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143795+0000', 'last_change': '2024-05-28T14:12:59.143795+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143795+0000', 'last_undegraded': '2024-05-28T14:12:59.143795+0000', 'last_fullsized': '2024-05-28T14:12:59.143795+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:13:05.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3a is not active+clean 2024-05-28T14:13:05.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.141349+0000', 'last_change': '2024-05-28T14:12:59.141349+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.141349+0000', 'last_undegraded': '2024-05-28T14:12:59.141349+0000', 'last_fullsized': '2024-05-28T14:12:59.141349+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:13:05.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.39 is not active+clean 2024-05-28T14:13:05.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143106+0000', 'last_change': '2024-05-28T14:12:59.143106+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143106+0000', 'last_undegraded': '2024-05-28T14:12:59.143106+0000', 'last_fullsized': '2024-05-28T14:12:59.143106+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:13:05.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.38 is not active+clean 2024-05-28T14:13:05.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.147478+0000', 'last_change': '2024-05-28T14:12:59.147478+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.147478+0000', 'last_undegraded': '2024-05-28T14:12:59.147478+0000', 'last_fullsized': '2024-05-28T14:12:59.147478+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:13:05.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.37 is not active+clean 2024-05-28T14:13:05.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143317+0000', 'last_change': '2024-05-28T14:12:59.143317+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143317+0000', 'last_undegraded': '2024-05-28T14:12:59.143317+0000', 'last_fullsized': '2024-05-28T14:12:59.143317+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:13:05.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.36 is not active+clean 2024-05-28T14:13:05.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.142610+0000', 'last_change': '2024-05-28T14:12:59.142610+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.142610+0000', 'last_undegraded': '2024-05-28T14:12:59.142610+0000', 'last_fullsized': '2024-05-28T14:12:59.142610+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:13:05.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.35 is not active+clean 2024-05-28T14:13:05.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.144639+0000', 'last_change': '2024-05-28T14:12:59.144639+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.144639+0000', 'last_undegraded': '2024-05-28T14:12:59.144639+0000', 'last_fullsized': '2024-05-28T14:12:59.144639+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:13:05.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.34 is not active+clean 2024-05-28T14:13:05.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143340+0000', 'last_change': '2024-05-28T14:12:59.143340+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143340+0000', 'last_undegraded': '2024-05-28T14:12:59.143340+0000', 'last_fullsized': '2024-05-28T14:12:59.143340+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:13:05.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.33 is not active+clean 2024-05-28T14:13:05.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.146701+0000', 'last_change': '2024-05-28T14:12:59.146701+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.146701+0000', 'last_undegraded': '2024-05-28T14:12:59.146701+0000', 'last_fullsized': '2024-05-28T14:12:59.146701+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:13:05.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.32 is not active+clean 2024-05-28T14:13:05.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.147196+0000', 'last_change': '2024-05-28T14:12:59.147196+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.147196+0000', 'last_undegraded': '2024-05-28T14:12:59.147196+0000', 'last_fullsized': '2024-05-28T14:12:59.147196+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:13:05.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.31 is not active+clean 2024-05-28T14:13:05.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.141425+0000', 'last_change': '2024-05-28T14:12:59.141425+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.141425+0000', 'last_undegraded': '2024-05-28T14:12:59.141425+0000', 'last_fullsized': '2024-05-28T14:12:59.141425+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:13:05.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.30 is not active+clean 2024-05-28T14:13:05.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.144638+0000', 'last_change': '2024-05-28T14:12:59.144638+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.144638+0000', 'last_undegraded': '2024-05-28T14:12:59.144638+0000', 'last_fullsized': '2024-05-28T14:12:59.144638+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:13:05.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2f is not active+clean 2024-05-28T14:13:05.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.144519+0000', 'last_change': '2024-05-28T14:12:59.144519+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.144519+0000', 'last_undegraded': '2024-05-28T14:12:59.144519+0000', 'last_fullsized': '2024-05-28T14:12:59.144519+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:13:05.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2e is not active+clean 2024-05-28T14:13:05.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.152420+0000', 'last_change': '2024-05-28T14:12:59.152420+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.152420+0000', 'last_undegraded': '2024-05-28T14:12:59.152420+0000', 'last_fullsized': '2024-05-28T14:12:59.152420+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:13:05.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2d is not active+clean 2024-05-28T14:13:05.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.149551+0000', 'last_change': '2024-05-28T14:12:59.149551+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.149551+0000', 'last_undegraded': '2024-05-28T14:12:59.149551+0000', 'last_fullsized': '2024-05-28T14:12:59.149551+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:13:05.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2c is not active+clean 2024-05-28T14:13:05.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143766+0000', 'last_change': '2024-05-28T14:12:59.143766+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143766+0000', 'last_undegraded': '2024-05-28T14:12:59.143766+0000', 'last_fullsized': '2024-05-28T14:12:59.143766+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:13:05.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2b is not active+clean 2024-05-28T14:13:05.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.142806+0000', 'last_change': '2024-05-28T14:12:59.142806+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.142806+0000', 'last_undegraded': '2024-05-28T14:12:59.142806+0000', 'last_fullsized': '2024-05-28T14:12:59.142806+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:13:05.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2a is not active+clean 2024-05-28T14:13:05.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.147836+0000', 'last_change': '2024-05-28T14:12:59.147836+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.147836+0000', 'last_undegraded': '2024-05-28T14:12:59.147836+0000', 'last_fullsized': '2024-05-28T14:12:59.147836+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:13:05.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.29 is not active+clean 2024-05-28T14:13:05.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.142797+0000', 'last_change': '2024-05-28T14:12:59.142797+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.142797+0000', 'last_undegraded': '2024-05-28T14:12:59.142797+0000', 'last_fullsized': '2024-05-28T14:12:59.142797+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:13:05.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.28 is not active+clean 2024-05-28T14:13:05.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.144378+0000', 'last_change': '2024-05-28T14:12:59.144378+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.144378+0000', 'last_undegraded': '2024-05-28T14:12:59.144378+0000', 'last_fullsized': '2024-05-28T14:12:59.144378+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:13:05.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.27 is not active+clean 2024-05-28T14:13:05.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143979+0000', 'last_change': '2024-05-28T14:12:59.143979+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143979+0000', 'last_undegraded': '2024-05-28T14:12:59.143979+0000', 'last_fullsized': '2024-05-28T14:12:59.143979+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:13:05.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.26 is not active+clean 2024-05-28T14:13:05.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.142104+0000', 'last_change': '2024-05-28T14:12:59.142104+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.142104+0000', 'last_undegraded': '2024-05-28T14:12:59.142104+0000', 'last_fullsized': '2024-05-28T14:12:59.142104+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:13:05.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.25 is not active+clean 2024-05-28T14:13:05.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.146411+0000', 'last_change': '2024-05-28T14:12:59.146411+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.146411+0000', 'last_undegraded': '2024-05-28T14:12:59.146411+0000', 'last_fullsized': '2024-05-28T14:12:59.146411+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:13:05.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.24 is not active+clean 2024-05-28T14:13:05.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.145670+0000', 'last_change': '2024-05-28T14:12:59.145670+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.145670+0000', 'last_undegraded': '2024-05-28T14:12:59.145670+0000', 'last_fullsized': '2024-05-28T14:12:59.145670+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:13:05.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.23 is not active+clean 2024-05-28T14:13:05.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.148734+0000', 'last_change': '2024-05-28T14:12:59.148734+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.148734+0000', 'last_undegraded': '2024-05-28T14:12:59.148734+0000', 'last_fullsized': '2024-05-28T14:12:59.148734+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:13:05.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.22 is not active+clean 2024-05-28T14:13:05.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143300+0000', 'last_change': '2024-05-28T14:12:59.143300+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143300+0000', 'last_undegraded': '2024-05-28T14:12:59.143300+0000', 'last_fullsized': '2024-05-28T14:12:59.143300+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:13:05.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.21 is not active+clean 2024-05-28T14:13:05.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.141526+0000', 'last_change': '2024-05-28T14:12:59.141526+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.141526+0000', 'last_undegraded': '2024-05-28T14:12:59.141526+0000', 'last_fullsized': '2024-05-28T14:12:59.141526+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:13:05.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.20 is not active+clean 2024-05-28T14:13:05.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.140649+0000', 'last_change': '2024-05-28T14:12:59.140649+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.140649+0000', 'last_undegraded': '2024-05-28T14:12:59.140649+0000', 'last_fullsized': '2024-05-28T14:12:59.140649+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:13:05.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1e is not active+clean 2024-05-28T14:13:05.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.145011+0000', 'last_change': '2024-05-28T14:12:59.145011+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.145011+0000', 'last_undegraded': '2024-05-28T14:12:59.145011+0000', 'last_fullsized': '2024-05-28T14:12:59.145011+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:13:05.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1f is not active+clean 2024-05-28T14:13:05.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.151621+0000', 'last_change': '2024-05-28T14:12:59.151621+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.151621+0000', 'last_undegraded': '2024-05-28T14:12:59.151621+0000', 'last_fullsized': '2024-05-28T14:12:59.151621+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:13:05.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1c is not active+clean 2024-05-28T14:13:05.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.150359+0000', 'last_change': '2024-05-28T14:12:59.150359+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.150359+0000', 'last_undegraded': '2024-05-28T14:12:59.150359+0000', 'last_fullsized': '2024-05-28T14:12:59.150359+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:13:05.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1d is not active+clean 2024-05-28T14:13:05.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.148929+0000', 'last_change': '2024-05-28T14:12:59.148929+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.148929+0000', 'last_undegraded': '2024-05-28T14:12:59.148929+0000', 'last_fullsized': '2024-05-28T14:12:59.148929+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:13:05.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1a is not active+clean 2024-05-28T14:13:05.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.141097+0000', 'last_change': '2024-05-28T14:12:59.141097+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.141097+0000', 'last_undegraded': '2024-05-28T14:12:59.141097+0000', 'last_fullsized': '2024-05-28T14:12:59.141097+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:13:05.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1b is not active+clean 2024-05-28T14:13:05.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143903+0000', 'last_change': '2024-05-28T14:12:59.143903+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143903+0000', 'last_undegraded': '2024-05-28T14:12:59.143903+0000', 'last_fullsized': '2024-05-28T14:12:59.143903+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:13:05.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.18 is not active+clean 2024-05-28T14:13:05.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.144181+0000', 'last_change': '2024-05-28T14:12:59.144181+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.144181+0000', 'last_undegraded': '2024-05-28T14:12:59.144181+0000', 'last_fullsized': '2024-05-28T14:12:59.144181+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:13:05.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.19 is not active+clean 2024-05-28T14:13:05.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143666+0000', 'last_change': '2024-05-28T14:12:59.143666+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143666+0000', 'last_undegraded': '2024-05-28T14:12:59.143666+0000', 'last_fullsized': '2024-05-28T14:12:59.143666+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:13:05.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.16 is not active+clean 2024-05-28T14:13:05.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.145598+0000', 'last_change': '2024-05-28T14:12:59.145598+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.145598+0000', 'last_undegraded': '2024-05-28T14:12:59.145598+0000', 'last_fullsized': '2024-05-28T14:12:59.145598+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:13:05.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.17 is not active+clean 2024-05-28T14:13:05.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.151052+0000', 'last_change': '2024-05-28T14:12:59.151052+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.151052+0000', 'last_undegraded': '2024-05-28T14:12:59.151052+0000', 'last_fullsized': '2024-05-28T14:12:59.151052+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:13:05.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.14 is not active+clean 2024-05-28T14:13:05.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.145229+0000', 'last_change': '2024-05-28T14:12:59.145229+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.145229+0000', 'last_undegraded': '2024-05-28T14:12:59.145229+0000', 'last_fullsized': '2024-05-28T14:12:59.145229+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:13:05.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.15 is not active+clean 2024-05-28T14:13:05.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143506+0000', 'last_change': '2024-05-28T14:12:59.143506+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143506+0000', 'last_undegraded': '2024-05-28T14:12:59.143506+0000', 'last_fullsized': '2024-05-28T14:12:59.143506+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-28T14:13:05.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.12 is not active+clean 2024-05-28T14:13:05.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.144233+0000', 'last_change': '2024-05-28T14:12:59.144233+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.144233+0000', 'last_undegraded': '2024-05-28T14:12:59.144233+0000', 'last_fullsized': '2024-05-28T14:12:59.144233+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:13:05.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.13 is not active+clean 2024-05-28T14:13:05.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.144770+0000', 'last_change': '2024-05-28T14:12:59.144770+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.144770+0000', 'last_undegraded': '2024-05-28T14:12:59.144770+0000', 'last_fullsized': '2024-05-28T14:12:59.144770+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:13:05.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.11 is not active+clean 2024-05-28T14:13:05.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.144691+0000', 'last_change': '2024-05-28T14:12:59.144691+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.144691+0000', 'last_undegraded': '2024-05-28T14:12:59.144691+0000', 'last_fullsized': '2024-05-28T14:12:59.144691+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:13:05.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.f is not active+clean 2024-05-28T14:13:05.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.150512+0000', 'last_change': '2024-05-28T14:12:59.150512+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.150512+0000', 'last_undegraded': '2024-05-28T14:12:59.150512+0000', 'last_fullsized': '2024-05-28T14:12:59.150512+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-28T14:13:05.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.c is not active+clean 2024-05-28T14:13:05.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.148185+0000', 'last_change': '2024-05-28T14:12:59.148185+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.148185+0000', 'last_undegraded': '2024-05-28T14:12:59.148185+0000', 'last_fullsized': '2024-05-28T14:12:59.148185+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:13:05.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.d is not active+clean 2024-05-28T14:13:05.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.147735+0000', 'last_change': '2024-05-28T14:12:59.147735+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.147735+0000', 'last_undegraded': '2024-05-28T14:12:59.147735+0000', 'last_fullsized': '2024-05-28T14:12:59.147735+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:13:05.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.a is not active+clean 2024-05-28T14:13:05.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.146108+0000', 'last_change': '2024-05-28T14:12:59.146108+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.146108+0000', 'last_undegraded': '2024-05-28T14:12:59.146108+0000', 'last_fullsized': '2024-05-28T14:12:59.146108+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:13:05.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.b is not active+clean 2024-05-28T14:13:05.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.141618+0000', 'last_change': '2024-05-28T14:12:59.141618+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.141618+0000', 'last_undegraded': '2024-05-28T14:12:59.141618+0000', 'last_fullsized': '2024-05-28T14:12:59.141618+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:13:05.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.8 is not active+clean 2024-05-28T14:13:05.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.146666+0000', 'last_change': '2024-05-28T14:12:59.146666+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.146666+0000', 'last_undegraded': '2024-05-28T14:12:59.146666+0000', 'last_fullsized': '2024-05-28T14:12:59.146666+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:13:05.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.9 is not active+clean 2024-05-28T14:13:05.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143944+0000', 'last_change': '2024-05-28T14:12:59.143944+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143944+0000', 'last_undegraded': '2024-05-28T14:12:59.143944+0000', 'last_fullsized': '2024-05-28T14:12:59.143944+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:13:05.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.6 is not active+clean 2024-05-28T14:13:05.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.151926+0000', 'last_change': '2024-05-28T14:12:59.151926+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.151926+0000', 'last_undegraded': '2024-05-28T14:12:59.151926+0000', 'last_fullsized': '2024-05-28T14:12:59.151926+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:13:05.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.7 is not active+clean 2024-05-28T14:13:05.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.142671+0000', 'last_change': '2024-05-28T14:12:59.142671+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.142671+0000', 'last_undegraded': '2024-05-28T14:12:59.142671+0000', 'last_fullsized': '2024-05-28T14:12:59.142671+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:13:05.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.4 is not active+clean 2024-05-28T14:13:05.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.142149+0000', 'last_change': '2024-05-28T14:12:59.142149+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.142149+0000', 'last_undegraded': '2024-05-28T14:12:59.142149+0000', 'last_fullsized': '2024-05-28T14:12:59.142149+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:13:05.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.5 is not active+clean 2024-05-28T14:13:05.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.153890+0000', 'last_change': '2024-05-28T14:12:59.153890+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.153890+0000', 'last_undegraded': '2024-05-28T14:12:59.153890+0000', 'last_fullsized': '2024-05-28T14:12:59.153890+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-28T14:13:05.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2 is not active+clean 2024-05-28T14:13:05.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143331+0000', 'last_change': '2024-05-28T14:12:59.143331+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143331+0000', 'last_undegraded': '2024-05-28T14:12:59.143331+0000', 'last_fullsized': '2024-05-28T14:12:59.143331+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:13:05.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3 is not active+clean 2024-05-28T14:13:05.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143393+0000', 'last_change': '2024-05-28T14:12:59.143393+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143393+0000', 'last_undegraded': '2024-05-28T14:12:59.143393+0000', 'last_fullsized': '2024-05-28T14:12:59.143393+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-28T14:13:05.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.0 is not active+clean 2024-05-28T14:13:05.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.146039+0000', 'last_change': '2024-05-28T14:12:59.146039+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.146039+0000', 'last_undegraded': '2024-05-28T14:12:59.146039+0000', 'last_fullsized': '2024-05-28T14:12:59.146039+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-28T14:13:05.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1 is not active+clean 2024-05-28T14:13:05.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.140964+0000', 'last_change': '2024-05-28T14:12:59.140964+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.140964+0000', 'last_undegraded': '2024-05-28T14:12:59.140964+0000', 'last_fullsized': '2024-05-28T14:12:59.140964+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-28T14:13:05.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.10 is not active+clean 2024-05-28T14:13:05.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-05-28T14:12:59.143848+0000', 'last_change': '2024-05-28T14:12:59.143848+0000', 'last_active': '2024-05-28T14:12:59.126694+0000', 'last_peered': '2024-05-28T14:12:59.126694+0000', 'last_clean': '2024-05-28T14:12:59.126694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-28T14:12:59.143848+0000', 'last_undegraded': '2024-05-28T14:12:59.143848+0000', 'last_fullsized': '2024-05-28T14:12:59.143848+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'last_clean_scrub_stamp': '2024-05-28T14:12:59.126694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-28T14:13:05.062 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-28T14:13:05.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f5356800640 1 Processor -- start 2024-05-28T14:13:05.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f5356800640 1 -- start start 2024-05-28T14:13:05.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f5356800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f535015e9d0 0x7f53500d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:05.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f5356800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53500b52d0 0x7f53500d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:05.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f5356800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f535009aa50 0x7f535009af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f5356800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f53500ccce0 con 0x7f535015e9d0 2024-05-28T14:13:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f5356800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f53500ccfc0 con 0x7f535009aa50 2024-05-28T14:13:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f5356800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f53500cd2a0 con 0x7f53500b52d0 2024-05-28T14:13:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f534f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f535015e9d0 0x7f53500d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f534f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f535015e9d0 0x7f53500d12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33956/0 (socket says 172.21.15.42:33956) 2024-05-28T14:13:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f534f400640 1 -- 172.21.15.42:0/726765886 learned_addr learned my addr 172.21.15.42:0/726765886 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f534ea00640 1 --2- 172.21.15.42:0/726765886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53500b52d0 0x7f53500d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f534fe00640 1 --2- 172.21.15.42:0/726765886 >> v2:172.21.15.63:3300/0 conn(0x7f535009aa50 0x7f535009af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f534f400640 1 -- 172.21.15.42:0/726765886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53500b52d0 msgr2=0x7f53500d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f534f400640 1 --2- 172.21.15.42:0/726765886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53500b52d0 0x7f53500d1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f534f400640 1 -- 172.21.15.42:0/726765886 >> v2:172.21.15.63:3300/0 conn(0x7f535009aa50 msgr2=0x7f535009af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f534f400640 1 --2- 172.21.15.42:0/726765886 >> v2:172.21.15.63:3300/0 conn(0x7f535009aa50 0x7f535009af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.243+0000 7f534f400640 1 -- 172.21.15.42:0/726765886 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53500cd580 con 0x7f535015e9d0 2024-05-28T14:13:05.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f534f400640 1 --2- 172.21.15.42:0/726765886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f535015e9d0 0x7f53500d12c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f53500739e0 tx=0x7f534001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0aced186f5282f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:05.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f534e000640 1 -- 172.21.15.42:0/726765886 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53400b0070 con 0x7f535015e9d0 2024-05-28T14:13:05.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f534e000640 1 -- 172.21.15.42:0/726765886 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53400ab600 con 0x7f535015e9d0 2024-05-28T14:13:05.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f534e000640 1 -- 172.21.15.42:0/726765886 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53400ab9d0 con 0x7f535015e9d0 2024-05-28T14:13:05.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f5356800640 1 -- 172.21.15.42:0/726765886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f535015e9d0 msgr2=0x7f53500d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:05.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f5356800640 1 --2- 172.21.15.42:0/726765886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f535015e9d0 0x7f53500d12c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f53500739e0 tx=0x7f534001c520 comp rx=0 tx=0).stop 2024-05-28T14:13:05.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f5356800640 1 -- 172.21.15.42:0/726765886 shutdown_connections 2024-05-28T14:13:05.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f5356800640 1 --2- 172.21.15.42:0/726765886 >> v2:172.21.15.63:3300/0 conn(0x7f535009aa50 0x7f535009af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f5356800640 1 --2- 172.21.15.42:0/726765886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53500b52d0 0x7f53500d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f5356800640 1 --2- 172.21.15.42:0/726765886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f535015e9d0 0x7f53500d12c0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f5356800640 1 -- 172.21.15.42:0/726765886 >> 172.21.15.42:0/726765886 conn(0x7f53500b3f70 msgr2=0x7f53500b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:05.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f5356800640 1 -- 172.21.15.42:0/726765886 shutdown_connections 2024-05-28T14:13:05.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f5356800640 1 -- 172.21.15.42:0/726765886 wait complete. 2024-05-28T14:13:05.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f5356800640 1 Processor -- start 2024-05-28T14:13:05.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.247+0000 7f5356800640 1 -- start start 2024-05-28T14:13:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f5356800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f535009aa50 0x7f535014e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f5356800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53500b52d0 0x7f53500c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f5356800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f535015e9d0 0x7f5350154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f5356800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f53500b3c30 con 0x7f535015e9d0 2024-05-28T14:13:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f5356800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53500b3ab0 con 0x7f53500b52d0 2024-05-28T14:13:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f5356800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f53500b3db0 con 0x7f535009aa50 2024-05-28T14:13:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f534fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f535015e9d0 0x7f5350154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f534fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f535015e9d0 0x7f5350154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33962/0 (socket says 172.21.15.42:33962) 2024-05-28T14:13:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f534fe00640 1 -- 172.21.15.42:0/2442991067 learned_addr learned my addr 172.21.15.42:0/2442991067 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f534f400640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f535009aa50 0x7f535014e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f534ea00640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53500b52d0 0x7f53500c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f534fe00640 1 -- 172.21.15.42:0/2442991067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f535009aa50 msgr2=0x7f535014e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f534fe00640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f535009aa50 0x7f535014e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f534fe00640 1 -- 172.21.15.42:0/2442991067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53500b52d0 msgr2=0x7f53500c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f534fe00640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53500b52d0 0x7f53500c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f534fe00640 1 -- 172.21.15.42:0/2442991067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5350155540 con 0x7f535015e9d0 2024-05-28T14:13:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f534ea00640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53500b52d0 0x7f53500c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f534fe00640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f535015e9d0 0x7f5350154dd0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f5348007d20 tx=0x7f534800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f5347400640 1 -- 172.21.15.42:0/2442991067 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348014420 con 0x7f535015e9d0 2024-05-28T14:13:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f5347400640 1 -- 172.21.15.42:0/2442991067 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5348019070 con 0x7f535015e9d0 2024-05-28T14:13:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f5347400640 1 -- 172.21.15.42:0/2442991067 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348014820 con 0x7f535015e9d0 2024-05-28T14:13:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f5356800640 1 -- 172.21.15.42:0/2442991067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5350155820 con 0x7f535015e9d0 2024-05-28T14:13:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f5356800640 1 -- 172.21.15.42:0/2442991067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5350155de0 con 0x7f535015e9d0 2024-05-28T14:13:05.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.251+0000 7f5356800640 1 -- 172.21.15.42:0/2442991067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5350068a90 con 0x7f535015e9d0 2024-05-28T14:13:05.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.255+0000 7f5347400640 1 -- 172.21.15.42:0/2442991067 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f53480149c0 con 0x7f535015e9d0 2024-05-28T14:13:05.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.255+0000 7f5347400640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5330069b50 0x7f533006c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:05.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.255+0000 7f5347400640 1 -- 172.21.15.42:0/2442991067 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f5348085d80 con 0x7f535015e9d0 2024-05-28T14:13:05.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.255+0000 7f534f400640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5330069b50 0x7f533006c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:05.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.255+0000 7f534f400640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5330069b50 0x7f533006c010 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f53400ad000 tx=0x7f5340008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:05.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.259+0000 7f5347400640 1 -- 172.21.15.42:0/2442991067 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5348057e40 con 0x7f535015e9d0 2024-05-28T14:13:05.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.479+0000 7f5356800640 1 -- 172.21.15.42:0/2442991067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f53500c92f0 con 0x7f535015e9d0 2024-05-28T14:13:05.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.479+0000 7f5347400640 1 -- 172.21.15.42:0/2442991067 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1016 (secure 0 0 0) 0x7f5348057740 con 0x7f535015e9d0 2024-05-28T14:13:05.465 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:05.466 INFO:teuthology.orchestra.run.smithi042.stdout:{"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":906,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1716904689,"num_in_osds":8,"osd_in_since":1716904680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":995614720,"bytes_avail":772098498560,"bytes_total":773094113280,"write_bytes_sec":2377,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":74,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":57440}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-05-28T14:12:23.155415+0000","services":{}},"progress_events":{}} 2024-05-28T14:13:05.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.483+0000 7f5356800640 1 -- 172.21.15.42:0/2442991067 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5330069b50 msgr2=0x7f533006c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.483+0000 7f5356800640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5330069b50 0x7f533006c010 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f53400ad000 tx=0x7f5340008040 comp rx=0 tx=0).stop 2024-05-28T14:13:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.483+0000 7f5356800640 1 -- 172.21.15.42:0/2442991067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f535015e9d0 msgr2=0x7f5350154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.483+0000 7f5356800640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f535015e9d0 0x7f5350154dd0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f5348007d20 tx=0x7f534800e4b0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.483+0000 7f5356800640 1 -- 172.21.15.42:0/2442991067 shutdown_connections 2024-05-28T14:13:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.483+0000 7f5356800640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5330069b50 0x7f533006c010 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.483+0000 7f5356800640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f535015e9d0 0x7f5350154dd0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.483+0000 7f5356800640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53500b52d0 0x7f53500c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.483+0000 7f5356800640 1 --2- 172.21.15.42:0/2442991067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f535009aa50 0x7f535014e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.483+0000 7f5356800640 1 -- 172.21.15.42:0/2442991067 >> 172.21.15.42:0/2442991067 conn(0x7f53500b3f70 msgr2=0x7f53500cf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.483+0000 7f5356800640 1 -- 172.21.15.42:0/2442991067 shutdown_connections 2024-05-28T14:13:05.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.483+0000 7f5356800640 1 -- 172.21.15.42:0/2442991067 wait complete. 2024-05-28T14:13:05.487 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-28T14:13:05.487 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:13:05.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.643+0000 7fb052200640 1 Processor -- start 2024-05-28T14:13:05.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.643+0000 7fb052200640 1 -- start start 2024-05-28T14:13:05.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.643+0000 7fb052200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb04c0afa50 0x7fb04c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:05.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.643+0000 7fb052200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb04c156e50 0x7fb04c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:05.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.643+0000 7fb052200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb04c15cb90 con 0x7fb04c156a70 2024-05-28T14:13:05.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.643+0000 7fb052200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb04c15ce50 con 0x7fb04c156e50 2024-05-28T14:13:05.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.643+0000 7fb052200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb04c15d130 con 0x7fb04c0afa50 2024-05-28T14:13:05.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.643+0000 7fb04b400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb04c156a70 0x7fb04c0af340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37208/0 (socket says 172.21.15.42:37208) 2024-05-28T14:13:05.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.643+0000 7fb04b400640 1 -- 172.21.15.42:0/1256992875 learned_addr learned my addr 172.21.15.42:0/1256992875 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:13:05.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.643+0000 7fb04be00640 1 --2- 172.21.15.42:0/1256992875 >> v2:172.21.15.42:3300/0 conn(0x7fb04c156e50 0x7fb04c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:05.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb04aa00640 1 --2- 172.21.15.42:0/1256992875 >> v2:172.21.15.63:3300/0 conn(0x7fb04c0afa50 0x7fb04c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:05.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb04a000640 1 -- 172.21.15.42:0/1256992875 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3599862677 0 0) 0x7fb04c15cb90 con 0x7fb04c156a70 2024-05-28T14:13:05.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb04a000640 1 -- 172.21.15.42:0/1256992875 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb030003530 con 0x7fb04c156a70 2024-05-28T14:13:05.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb04be00640 1 -- 172.21.15.42:0/1256992875 >> v2:172.21.15.63:3300/0 conn(0x7fb04c0afa50 msgr2=0x7fb04c06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:05.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb04be00640 1 --2- 172.21.15.42:0/1256992875 >> v2:172.21.15.63:3300/0 conn(0x7fb04c0afa50 0x7fb04c06dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb04be00640 1 -- 172.21.15.42:0/1256992875 >> v1:172.21.15.42:6789/0 conn(0x7fb04c156a70 legacy=0x7fb04c0af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:05.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb04be00640 1 -- 172.21.15.42:0/1256992875 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04c15d410 con 0x7fb04c156e50 2024-05-28T14:13:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb04be00640 1 --2- 172.21.15.42:0/1256992875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb04c156e50 0x7fb04c0d06f0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0701f0 tx=0x7fb04401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f415920b97fbe654 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb04a000640 1 -- 172.21.15.42:0/1256992875 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0440b2070 con 0x7fb04c156e50 2024-05-28T14:13:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb04a000640 1 -- 172.21.15.42:0/1256992875 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0440a5900 con 0x7fb04c156e50 2024-05-28T14:13:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb04a000640 1 -- 172.21.15.42:0/1256992875 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0440a5cd0 con 0x7fb04c156e50 2024-05-28T14:13:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb052200640 1 -- 172.21.15.42:0/1256992875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb04c156e50 msgr2=0x7fb04c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb052200640 1 --2- 172.21.15.42:0/1256992875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb04c156e50 0x7fb04c0d06f0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0701f0 tx=0x7fb04401c930 comp rx=0 tx=0).stop 2024-05-28T14:13:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb052200640 1 -- 172.21.15.42:0/1256992875 shutdown_connections 2024-05-28T14:13:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.647+0000 7fb052200640 1 --2- 172.21.15.42:0/1256992875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb04c156e50 0x7fb04c0d06f0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb052200640 1 --2- 172.21.15.42:0/1256992875 >> v2:172.21.15.63:3300/0 conn(0x7fb04c0afa50 0x7fb04c06dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb052200640 1 -- 172.21.15.42:0/1256992875 >> 172.21.15.42:0/1256992875 conn(0x7fb04c06bdf0 msgr2=0x7fb04c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb052200640 1 -- 172.21.15.42:0/1256992875 shutdown_connections 2024-05-28T14:13:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb052200640 1 -- 172.21.15.42:0/1256992875 wait complete. 2024-05-28T14:13:05.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb052200640 1 Processor -- start 2024-05-28T14:13:05.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb052200640 1 -- start start 2024-05-28T14:13:05.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb052200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb04c0afa50 0x7fb04c0cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:05.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb052200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb04c156a70 0x7fb04c15f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:05.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb052200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb04c156e50 0x7fb04c157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:05.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb052200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c160ed0 con 0x7fb04c156a70 2024-05-28T14:13:05.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb052200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c160d50 con 0x7fb04c156e50 2024-05-28T14:13:05.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb052200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb04c161050 con 0x7fb04c0afa50 2024-05-28T14:13:05.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb04aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb04c156a70 0x7fb04c15f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:05.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb04aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb04c156a70 0x7fb04c15f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33988/0 (socket says 172.21.15.42:33988) 2024-05-28T14:13:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb04aa00640 1 -- 172.21.15.42:0/95391581 learned_addr learned my addr 172.21.15.42:0/95391581 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb04b400640 1 --2- 172.21.15.42:0/95391581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb04c0afa50 0x7fb04c0cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb04be00640 1 --2- 172.21.15.42:0/95391581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb04c156e50 0x7fb04c157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb04aa00640 1 -- 172.21.15.42:0/95391581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb04c0afa50 msgr2=0x7fb04c0cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb04aa00640 1 --2- 172.21.15.42:0/95391581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb04c0afa50 0x7fb04c0cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb04aa00640 1 -- 172.21.15.42:0/95391581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb04c156e50 msgr2=0x7fb04c157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb04aa00640 1 --2- 172.21.15.42:0/95391581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb04c156e50 0x7fb04c157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb04aa00640 1 -- 172.21.15.42:0/95391581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04c0cfb00 con 0x7fb04c156a70 2024-05-28T14:13:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb04aa00640 1 --2- 172.21.15.42:0/95391581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb04c156a70 0x7fb04c15f070 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fb03800d830 tx=0x7fb03800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb043400640 1 -- 172.21.15.42:0/95391581 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03800b6c0 con 0x7fb04c156a70 2024-05-28T14:13:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb043400640 1 -- 172.21.15.42:0/95391581 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb038019070 con 0x7fb04c156a70 2024-05-28T14:13:05.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb043400640 1 -- 172.21.15.42:0/95391581 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03800ba60 con 0x7fb04c156a70 2024-05-28T14:13:05.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.651+0000 7fb052200640 1 -- 172.21.15.42:0/95391581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb04c14d370 con 0x7fb04c156a70 2024-05-28T14:13:05.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.655+0000 7fb052200640 1 -- 172.21.15.42:0/95391581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb04c15e870 con 0x7fb04c156a70 2024-05-28T14:13:05.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.655+0000 7fb052200640 1 -- 172.21.15.42:0/95391581 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb04c068a90 con 0x7fb04c156a70 2024-05-28T14:13:05.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.655+0000 7fb043400640 1 -- 172.21.15.42:0/95391581 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fb03801e460 con 0x7fb04c156a70 2024-05-28T14:13:05.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.655+0000 7fb043400640 1 --2- 172.21.15.42:0/95391581 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb02c069b50 0x7fb02c06c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:05.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.655+0000 7fb043400640 1 -- 172.21.15.42:0/95391581 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7fb038085d30 con 0x7fb04c156a70 2024-05-28T14:13:05.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.655+0000 7fb04b400640 1 --2- 172.21.15.42:0/95391581 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb02c069b50 0x7fb02c06c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:05.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.655+0000 7fb04b400640 1 --2- 172.21.15.42:0/95391581 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb02c069b50 0x7fb02c06c010 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb03c007fd0 tx=0x7fb03c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:05.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.659+0000 7fb043400640 1 -- 172.21.15.42:0/95391581 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb038057700 con 0x7fb04c156a70 2024-05-28T14:13:05.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.819+0000 7fb052200640 1 -- 172.21.15.42:0/95391581 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb04c15fe70 con 0x7fb02c069b50 2024-05-28T14:13:05.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.827+0000 7fb043400640 1 -- 172.21.15.42:0/95391581 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252499 (secure 0 0 0) 0x7fb04c15fe70 con 0x7fb02c069b50 2024-05-28T14:13:05.816 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:05.818 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:13:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.831+0000 7fb052200640 1 -- 172.21.15.42:0/95391581 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb02c069b50 msgr2=0x7fb02c06c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.831+0000 7fb052200640 1 --2- 172.21.15.42:0/95391581 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb02c069b50 0x7fb02c06c010 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb03c007fd0 tx=0x7fb03c009040 comp rx=0 tx=0).stop 2024-05-28T14:13:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.831+0000 7fb052200640 1 -- 172.21.15.42:0/95391581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb04c156a70 msgr2=0x7fb04c15f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.831+0000 7fb052200640 1 --2- 172.21.15.42:0/95391581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb04c156a70 0x7fb04c15f070 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fb03800d830 tx=0x7fb03800dcf0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.831+0000 7fb052200640 1 -- 172.21.15.42:0/95391581 shutdown_connections 2024-05-28T14:13:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.831+0000 7fb052200640 1 --2- 172.21.15.42:0/95391581 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb02c069b50 0x7fb02c06c010 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.831+0000 7fb052200640 1 --2- 172.21.15.42:0/95391581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb04c156e50 0x7fb04c157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.831+0000 7fb052200640 1 --2- 172.21.15.42:0/95391581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb04c156a70 0x7fb04c15f070 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.831+0000 7fb052200640 1 --2- 172.21.15.42:0/95391581 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb04c0afa50 0x7fb04c0cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:05.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.831+0000 7fb052200640 1 -- 172.21.15.42:0/95391581 >> 172.21.15.42:0/95391581 conn(0x7fb04c06bdf0 msgr2=0x7fb04c0b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:05.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.831+0000 7fb052200640 1 -- 172.21.15.42:0/95391581 shutdown_connections 2024-05-28T14:13:05.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:05.831+0000 7fb052200640 1 -- 172.21.15.42:0/95391581 wait complete. 2024-05-28T14:13:05.836 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":558,"stamp":"2024-05-28T14:13:05.164881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":972280,"kb_used_data":12560,"kb_used_omap":12,"kb_used_meta":959603,"kb_avail":754002440,"statfs":{"total":773094113280,"available":772098498560,"internally_reserved":0,"allocated":12861440,"data_stored":6266321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":982634034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.029679"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683531+0000","last_change":"2024-05-28T14:13:00.168249+0000","last_active":"2024-05-28T14:13:01.683531+0000","last_peered":"2024-05-28T14:13:01.683531+0000","last_clean":"2024-05-28T14:13:01.683531+0000","last_became_active":"2024-05-28T14:13:00.168012+0000","last_became_peered":"2024-05-28T14:13:00.168012+0000","last_unstale":"2024-05-28T14:13:01.683531+0000","last_undegraded":"2024-05-28T14:13:01.683531+0000","last_fullsized":"2024-05-28T14:13:01.683531+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:00:01.461588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.142978+0000","last_change":"2024-05-28T14:13:00.164766+0000","last_active":"2024-05-28T14:13:01.142978+0000","last_peered":"2024-05-28T14:13:01.142978+0000","last_clean":"2024-05-28T14:13:01.142978+0000","last_became_active":"2024-05-28T14:13:00.164510+0000","last_became_peered":"2024-05-28T14:13:00.164510+0000","last_unstale":"2024-05-28T14:13:01.142978+0000","last_undegraded":"2024-05-28T14:13:01.142978+0000","last_fullsized":"2024-05-28T14:13:01.142978+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:17:04.853171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683126+0000","last_change":"2024-05-28T14:13:00.166389+0000","last_active":"2024-05-28T14:13:01.683126+0000","last_peered":"2024-05-28T14:13:01.683126+0000","last_clean":"2024-05-28T14:13:01.683126+0000","last_became_active":"2024-05-28T14:13:00.165990+0000","last_became_peered":"2024-05-28T14:13:00.165990+0000","last_unstale":"2024-05-28T14:13:01.683126+0000","last_undegraded":"2024-05-28T14:13:01.683126+0000","last_fullsized":"2024-05-28T14:13:01.683126+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:07:22.275443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682922+0000","last_change":"2024-05-28T14:13:00.174762+0000","last_active":"2024-05-28T14:13:01.682922+0000","last_peered":"2024-05-28T14:13:01.682922+0000","last_clean":"2024-05-28T14:13:01.682922+0000","last_became_active":"2024-05-28T14:13:00.174288+0000","last_became_peered":"2024-05-28T14:13:00.174288+0000","last_unstale":"2024-05-28T14:13:01.682922+0000","last_undegraded":"2024-05-28T14:13:01.682922+0000","last_fullsized":"2024-05-28T14:13:01.682922+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:51:22.209282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.151103+0000","last_change":"2024-05-28T14:13:00.146533+0000","last_active":"2024-05-28T14:13:00.151103+0000","last_peered":"2024-05-28T14:13:00.151103+0000","last_clean":"2024-05-28T14:13:00.151103+0000","last_became_active":"2024-05-28T14:13:00.145774+0000","last_became_peered":"2024-05-28T14:13:00.145774+0000","last_unstale":"2024-05-28T14:13:00.151103+0000","last_undegraded":"2024-05-28T14:13:00.151103+0000","last_fullsized":"2024-05-28T14:13:00.151103+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:29:07.224328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140454+0000","last_change":"2024-05-28T14:13:00.171900+0000","last_active":"2024-05-28T14:13:01.140454+0000","last_peered":"2024-05-28T14:13:01.140454+0000","last_clean":"2024-05-28T14:13:01.140454+0000","last_became_active":"2024-05-28T14:13:00.171595+0000","last_became_peered":"2024-05-28T14:13:00.171595+0000","last_unstale":"2024-05-28T14:13:01.140454+0000","last_undegraded":"2024-05-28T14:13:01.140454+0000","last_fullsized":"2024-05-28T14:13:01.140454+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:47:25.433610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.142830+0000","last_change":"2024-05-28T14:13:00.169682+0000","last_active":"2024-05-28T14:13:01.142830+0000","last_peered":"2024-05-28T14:13:01.142830+0000","last_clean":"2024-05-28T14:13:01.142830+0000","last_became_active":"2024-05-28T14:13:00.169114+0000","last_became_peered":"2024-05-28T14:13:00.169114+0000","last_unstale":"2024-05-28T14:13:01.142830+0000","last_undegraded":"2024-05-28T14:13:01.142830+0000","last_fullsized":"2024-05-28T14:13:01.142830+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:40:57.469457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682647+0000","last_change":"2024-05-28T14:13:00.159023+0000","last_active":"2024-05-28T14:13:01.682647+0000","last_peered":"2024-05-28T14:13:01.682647+0000","last_clean":"2024-05-28T14:13:01.682647+0000","last_became_active":"2024-05-28T14:13:00.158756+0000","last_became_peered":"2024-05-28T14:13:00.158756+0000","last_unstale":"2024-05-28T14:13:01.682647+0000","last_undegraded":"2024-05-28T14:13:01.682647+0000","last_fullsized":"2024-05-28T14:13:01.682647+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:49:44.497244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683577+0000","last_change":"2024-05-28T14:13:00.161469+0000","last_active":"2024-05-28T14:13:01.683577+0000","last_peered":"2024-05-28T14:13:01.683577+0000","last_clean":"2024-05-28T14:13:01.683577+0000","last_became_active":"2024-05-28T14:13:00.161203+0000","last_became_peered":"2024-05-28T14:13:00.161203+0000","last_unstale":"2024-05-28T14:13:01.683577+0000","last_undegraded":"2024-05-28T14:13:01.683577+0000","last_fullsized":"2024-05-28T14:13:01.683577+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:25:27.542376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683102+0000","last_change":"2024-05-28T14:13:00.154915+0000","last_active":"2024-05-28T14:13:01.683102+0000","last_peered":"2024-05-28T14:13:01.683102+0000","last_clean":"2024-05-28T14:13:01.683102+0000","last_became_active":"2024-05-28T14:13:00.152360+0000","last_became_peered":"2024-05-28T14:13:00.152360+0000","last_unstale":"2024-05-28T14:13:01.683102+0000","last_undegraded":"2024-05-28T14:13:01.683102+0000","last_fullsized":"2024-05-28T14:13:01.683102+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:26:30.453204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143023+0000","last_change":"2024-05-28T14:13:00.172238+0000","last_active":"2024-05-28T14:13:01.143023+0000","last_peered":"2024-05-28T14:13:01.143023+0000","last_clean":"2024-05-28T14:13:01.143023+0000","last_became_active":"2024-05-28T14:13:00.171973+0000","last_became_peered":"2024-05-28T14:13:00.171973+0000","last_unstale":"2024-05-28T14:13:01.143023+0000","last_undegraded":"2024-05-28T14:13:01.143023+0000","last_fullsized":"2024-05-28T14:13:01.143023+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:28:59.192834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683790+0000","last_change":"2024-05-28T14:13:00.152818+0000","last_active":"2024-05-28T14:13:01.683790+0000","last_peered":"2024-05-28T14:13:01.683790+0000","last_clean":"2024-05-28T14:13:01.683790+0000","last_became_active":"2024-05-28T14:13:00.152551+0000","last_became_peered":"2024-05-28T14:13:00.152551+0000","last_unstale":"2024-05-28T14:13:01.683790+0000","last_undegraded":"2024-05-28T14:13:01.683790+0000","last_fullsized":"2024-05-28T14:13:01.683790+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:39:28.289636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683028+0000","last_change":"2024-05-28T14:13:00.157994+0000","last_active":"2024-05-28T14:13:01.683028+0000","last_peered":"2024-05-28T14:13:01.683028+0000","last_clean":"2024-05-28T14:13:01.683028+0000","last_became_active":"2024-05-28T14:13:00.157671+0000","last_became_peered":"2024-05-28T14:13:00.157671+0000","last_unstale":"2024-05-28T14:13:01.683028+0000","last_undegraded":"2024-05-28T14:13:01.683028+0000","last_fullsized":"2024-05-28T14:13:01.683028+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:31:55.696017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682579+0000","last_change":"2024-05-28T14:13:00.168829+0000","last_active":"2024-05-28T14:13:01.682579+0000","last_peered":"2024-05-28T14:13:01.682579+0000","last_clean":"2024-05-28T14:13:01.682579+0000","last_became_active":"2024-05-28T14:13:00.168582+0000","last_became_peered":"2024-05-28T14:13:00.168582+0000","last_unstale":"2024-05-28T14:13:01.682579+0000","last_undegraded":"2024-05-28T14:13:01.682579+0000","last_fullsized":"2024-05-28T14:13:01.682579+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:34:57.501530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140255+0000","last_change":"2024-05-28T14:13:00.167892+0000","last_active":"2024-05-28T14:13:01.140255+0000","last_peered":"2024-05-28T14:13:01.140255+0000","last_clean":"2024-05-28T14:13:01.140255+0000","last_became_active":"2024-05-28T14:13:00.167515+0000","last_became_peered":"2024-05-28T14:13:00.167515+0000","last_unstale":"2024-05-28T14:13:01.140255+0000","last_undegraded":"2024-05-28T14:13:01.140255+0000","last_fullsized":"2024-05-28T14:13:01.140255+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:07.689842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.681856+0000","last_change":"2024-05-28T14:13:00.160368+0000","last_active":"2024-05-28T14:13:01.681856+0000","last_peered":"2024-05-28T14:13:01.681856+0000","last_clean":"2024-05-28T14:13:01.681856+0000","last_became_active":"2024-05-28T14:13:00.157771+0000","last_became_peered":"2024-05-28T14:13:00.157771+0000","last_unstale":"2024-05-28T14:13:01.681856+0000","last_undegraded":"2024-05-28T14:13:01.681856+0000","last_fullsized":"2024-05-28T14:13:01.681856+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:24:24.603471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143283+0000","last_change":"2024-05-28T14:13:00.172222+0000","last_active":"2024-05-28T14:13:01.143283+0000","last_peered":"2024-05-28T14:13:01.143283+0000","last_clean":"2024-05-28T14:13:01.143283+0000","last_became_active":"2024-05-28T14:13:00.170660+0000","last_became_peered":"2024-05-28T14:13:00.170660+0000","last_unstale":"2024-05-28T14:13:01.143283+0000","last_undegraded":"2024-05-28T14:13:01.143283+0000","last_fullsized":"2024-05-28T14:13:01.143283+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:48:07.755962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.215031+0000","last_change":"2024-05-28T14:13:00.173278+0000","last_active":"2024-05-28T14:13:01.215031+0000","last_peered":"2024-05-28T14:13:01.215031+0000","last_clean":"2024-05-28T14:13:01.215031+0000","last_became_active":"2024-05-28T14:13:00.173005+0000","last_became_peered":"2024-05-28T14:13:00.173005+0000","last_unstale":"2024-05-28T14:13:01.215031+0000","last_undegraded":"2024-05-28T14:13:01.215031+0000","last_fullsized":"2024-05-28T14:13:01.215031+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:56.215885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683072+0000","last_change":"2024-05-28T14:13:00.159008+0000","last_active":"2024-05-28T14:13:01.683072+0000","last_peered":"2024-05-28T14:13:01.683072+0000","last_clean":"2024-05-28T14:13:01.683072+0000","last_became_active":"2024-05-28T14:13:00.157662+0000","last_became_peered":"2024-05-28T14:13:00.157662+0000","last_unstale":"2024-05-28T14:13:01.683072+0000","last_undegraded":"2024-05-28T14:13:01.683072+0000","last_fullsized":"2024-05-28T14:13:01.683072+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:26:30.453204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143369+0000","last_change":"2024-05-28T14:13:00.162934+0000","last_active":"2024-05-28T14:13:01.143369+0000","last_peered":"2024-05-28T14:13:01.143369+0000","last_clean":"2024-05-28T14:13:01.143369+0000","last_became_active":"2024-05-28T14:13:00.160020+0000","last_became_peered":"2024-05-28T14:13:00.160020+0000","last_unstale":"2024-05-28T14:13:01.143369+0000","last_undegraded":"2024-05-28T14:13:01.143369+0000","last_fullsized":"2024-05-28T14:13:01.143369+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:52:25.954742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143829+0000","last_change":"2024-05-28T14:13:00.152974+0000","last_active":"2024-05-28T14:13:01.143829+0000","last_peered":"2024-05-28T14:13:01.143829+0000","last_clean":"2024-05-28T14:13:01.143829+0000","last_became_active":"2024-05-28T14:13:00.152536+0000","last_became_peered":"2024-05-28T14:13:00.152536+0000","last_unstale":"2024-05-28T14:13:01.143829+0000","last_undegraded":"2024-05-28T14:13:01.143829+0000","last_fullsized":"2024-05-28T14:13:01.143829+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:03:18.352698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683689+0000","last_change":"2024-05-28T14:13:00.170898+0000","last_active":"2024-05-28T14:13:01.683689+0000","last_peered":"2024-05-28T14:13:01.683689+0000","last_clean":"2024-05-28T14:13:01.683689+0000","last_became_active":"2024-05-28T14:13:00.170425+0000","last_became_peered":"2024-05-28T14:13:00.170425+0000","last_unstale":"2024-05-28T14:13:01.683689+0000","last_undegraded":"2024-05-28T14:13:01.683689+0000","last_fullsized":"2024-05-28T14:13:01.683689+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:06.645451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.142671+0000","last_change":"2024-05-28T14:13:00.158631+0000","last_active":"2024-05-28T14:13:01.142671+0000","last_peered":"2024-05-28T14:13:01.142671+0000","last_clean":"2024-05-28T14:13:01.142671+0000","last_became_active":"2024-05-28T14:13:00.158095+0000","last_became_peered":"2024-05-28T14:13:00.158095+0000","last_unstale":"2024-05-28T14:13:01.142671+0000","last_undegraded":"2024-05-28T14:13:01.142671+0000","last_fullsized":"2024-05-28T14:13:01.142671+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:07:18.826687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682838+0000","last_change":"2024-05-28T14:13:00.156579+0000","last_active":"2024-05-28T14:13:01.682838+0000","last_peered":"2024-05-28T14:13:01.682838+0000","last_clean":"2024-05-28T14:13:01.682838+0000","last_became_active":"2024-05-28T14:13:00.156100+0000","last_became_peered":"2024-05-28T14:13:00.156100+0000","last_unstale":"2024-05-28T14:13:01.682838+0000","last_undegraded":"2024-05-28T14:13:01.682838+0000","last_fullsized":"2024-05-28T14:13:01.682838+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:19:52.357432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143442+0000","last_change":"2024-05-28T14:13:00.170638+0000","last_active":"2024-05-28T14:13:01.143442+0000","last_peered":"2024-05-28T14:13:01.143442+0000","last_clean":"2024-05-28T14:13:01.143442+0000","last_became_active":"2024-05-28T14:13:00.170278+0000","last_became_peered":"2024-05-28T14:13:00.170278+0000","last_unstale":"2024-05-28T14:13:01.143442+0000","last_undegraded":"2024-05-28T14:13:01.143442+0000","last_fullsized":"2024-05-28T14:13:01.143442+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:46:22.522782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683376+0000","last_change":"2024-05-28T14:13:00.154163+0000","last_active":"2024-05-28T14:13:01.683376+0000","last_peered":"2024-05-28T14:13:01.683376+0000","last_clean":"2024-05-28T14:13:01.683376+0000","last_became_active":"2024-05-28T14:13:00.153904+0000","last_became_peered":"2024-05-28T14:13:00.153904+0000","last_unstale":"2024-05-28T14:13:01.683376+0000","last_undegraded":"2024-05-28T14:13:01.683376+0000","last_fullsized":"2024-05-28T14:13:01.683376+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:07:22.275443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140930+0000","last_change":"2024-05-28T14:13:00.173166+0000","last_active":"2024-05-28T14:13:01.140930+0000","last_peered":"2024-05-28T14:13:01.140930+0000","last_clean":"2024-05-28T14:13:01.140930+0000","last_became_active":"2024-05-28T14:13:00.171662+0000","last_became_peered":"2024-05-28T14:13:00.171662+0000","last_unstale":"2024-05-28T14:13:01.140930+0000","last_undegraded":"2024-05-28T14:13:01.140930+0000","last_fullsized":"2024-05-28T14:13:01.140930+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:23:08.600237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683761+0000","last_change":"2024-05-28T14:13:00.152220+0000","last_active":"2024-05-28T14:13:01.683761+0000","last_peered":"2024-05-28T14:13:01.683761+0000","last_clean":"2024-05-28T14:13:01.683761+0000","last_became_active":"2024-05-28T14:13:00.151960+0000","last_became_peered":"2024-05-28T14:13:00.151960+0000","last_unstale":"2024-05-28T14:13:01.683761+0000","last_undegraded":"2024-05-28T14:13:01.683761+0000","last_fullsized":"2024-05-28T14:13:01.683761+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:53:04.689621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143174+0000","last_change":"2024-05-28T14:13:00.165826+0000","last_active":"2024-05-28T14:13:01.143174+0000","last_peered":"2024-05-28T14:13:01.143174+0000","last_clean":"2024-05-28T14:13:01.143174+0000","last_became_active":"2024-05-28T14:13:00.162459+0000","last_became_peered":"2024-05-28T14:13:00.162459+0000","last_unstale":"2024-05-28T14:13:01.143174+0000","last_undegraded":"2024-05-28T14:13:01.143174+0000","last_fullsized":"2024-05-28T14:13:01.143174+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:41:04.892264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143445+0000","last_change":"2024-05-28T14:13:00.146761+0000","last_active":"2024-05-28T14:13:01.143445+0000","last_peered":"2024-05-28T14:13:01.143445+0000","last_clean":"2024-05-28T14:13:01.143445+0000","last_became_active":"2024-05-28T14:13:00.146487+0000","last_became_peered":"2024-05-28T14:13:00.146487+0000","last_unstale":"2024-05-28T14:13:01.143445+0000","last_undegraded":"2024-05-28T14:13:01.143445+0000","last_fullsized":"2024-05-28T14:13:01.143445+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:16:35.564945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682331+0000","last_change":"2024-05-28T14:13:00.154658+0000","last_active":"2024-05-28T14:13:01.682331+0000","last_peered":"2024-05-28T14:13:01.682331+0000","last_clean":"2024-05-28T14:13:01.682331+0000","last_became_active":"2024-05-28T14:13:00.154417+0000","last_became_peered":"2024-05-28T14:13:00.154417+0000","last_unstale":"2024-05-28T14:13:01.682331+0000","last_undegraded":"2024-05-28T14:13:01.682331+0000","last_fullsized":"2024-05-28T14:13:01.682331+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:01:52.004242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140316+0000","last_change":"2024-05-28T14:13:00.165310+0000","last_active":"2024-05-28T14:13:01.140316+0000","last_peered":"2024-05-28T14:13:01.140316+0000","last_clean":"2024-05-28T14:13:01.140316+0000","last_became_active":"2024-05-28T14:13:00.164732+0000","last_became_peered":"2024-05-28T14:13:00.164732+0000","last_unstale":"2024-05-28T14:13:01.140316+0000","last_undegraded":"2024-05-28T14:13:01.140316+0000","last_fullsized":"2024-05-28T14:13:01.140316+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:24:20.189172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.141038+0000","last_change":"2024-05-28T14:13:00.163393+0000","last_active":"2024-05-28T14:13:01.141038+0000","last_peered":"2024-05-28T14:13:01.141038+0000","last_clean":"2024-05-28T14:13:01.141038+0000","last_became_active":"2024-05-28T14:13:00.163122+0000","last_became_peered":"2024-05-28T14:13:00.163122+0000","last_unstale":"2024-05-28T14:13:01.141038+0000","last_undegraded":"2024-05-28T14:13:01.141038+0000","last_fullsized":"2024-05-28T14:13:01.141038+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:23:44.667753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"116'4","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.144025+0000","last_change":"2024-05-28T14:12:59.156662+0000","last_active":"2024-05-28T14:13:01.144025+0000","last_peered":"2024-05-28T14:13:01.144025+0000","last_clean":"2024-05-28T14:13:01.144025+0000","last_became_active":"2024-05-28T14:12:59.156379+0000","last_became_peered":"2024-05-28T14:12:59.156379+0000","last_unstale":"2024-05-28T14:13:01.144025+0000","last_undegraded":"2024-05-28T14:13:01.144025+0000","last_fullsized":"2024-05-28T14:13:01.144025+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:29:06.215909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.215278+0000","last_change":"2024-05-28T14:13:00.165334+0000","last_active":"2024-05-28T14:13:01.215278+0000","last_peered":"2024-05-28T14:13:01.215278+0000","last_clean":"2024-05-28T14:13:01.215278+0000","last_became_active":"2024-05-28T14:13:00.165055+0000","last_became_peered":"2024-05-28T14:13:00.165055+0000","last_unstale":"2024-05-28T14:13:01.215278+0000","last_undegraded":"2024-05-28T14:13:01.215278+0000","last_fullsized":"2024-05-28T14:13:01.215278+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:46:22.522782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682807+0000","last_change":"2024-05-28T14:12:59.160363+0000","last_active":"2024-05-28T14:13:01.682807+0000","last_peered":"2024-05-28T14:13:01.682807+0000","last_clean":"2024-05-28T14:13:01.682807+0000","last_became_active":"2024-05-28T14:12:59.159749+0000","last_became_peered":"2024-05-28T14:12:59.159749+0000","last_unstale":"2024-05-28T14:13:01.682807+0000","last_undegraded":"2024-05-28T14:13:01.682807+0000","last_fullsized":"2024-05-28T14:13:01.682807+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:19:51.349013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140744+0000","last_change":"2024-05-28T14:13:00.168895+0000","last_active":"2024-05-28T14:13:01.140744+0000","last_peered":"2024-05-28T14:13:01.140744+0000","last_clean":"2024-05-28T14:13:01.140744+0000","last_became_active":"2024-05-28T14:13:00.168608+0000","last_became_peered":"2024-05-28T14:13:00.168608+0000","last_unstale":"2024-05-28T14:13:01.140744+0000","last_undegraded":"2024-05-28T14:13:01.140744+0000","last_fullsized":"2024-05-28T14:13:01.140744+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:29:07.224328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.171344+0000","last_change":"2024-05-28T14:12:59.155476+0000","last_active":"2024-05-28T14:13:00.171344+0000","last_peered":"2024-05-28T14:13:00.171344+0000","last_clean":"2024-05-28T14:13:00.171344+0000","last_became_active":"2024-05-28T14:12:59.155187+0000","last_became_peered":"2024-05-28T14:12:59.155187+0000","last_unstale":"2024-05-28T14:13:00.171344+0000","last_undegraded":"2024-05-28T14:13:00.171344+0000","last_fullsized":"2024-05-28T14:13:00.171344+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:07:49.480989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683267+0000","last_change":"2024-05-28T14:13:00.162908+0000","last_active":"2024-05-28T14:13:01.683267+0000","last_peered":"2024-05-28T14:13:01.683267+0000","last_clean":"2024-05-28T14:13:01.683267+0000","last_became_active":"2024-05-28T14:13:00.162616+0000","last_became_peered":"2024-05-28T14:13:00.162616+0000","last_unstale":"2024-05-28T14:13:01.683267+0000","last_undegraded":"2024-05-28T14:13:01.683267+0000","last_fullsized":"2024-05-28T14:13:01.683267+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:00:01.461588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.164533+0000","last_change":"2024-05-28T14:12:59.164490+0000","last_active":"2024-05-28T14:13:00.164533+0000","last_peered":"2024-05-28T14:13:00.164533+0000","last_clean":"2024-05-28T14:13:00.164533+0000","last_became_active":"2024-05-28T14:12:59.163870+0000","last_became_peered":"2024-05-28T14:12:59.163870+0000","last_unstale":"2024-05-28T14:13:00.164533+0000","last_undegraded":"2024-05-28T14:13:00.164533+0000","last_fullsized":"2024-05-28T14:13:00.164533+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:06.681422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.681910+0000","last_change":"2024-05-28T14:13:00.166324+0000","last_active":"2024-05-28T14:13:01.681910+0000","last_peered":"2024-05-28T14:13:01.681910+0000","last_clean":"2024-05-28T14:13:01.681910+0000","last_became_active":"2024-05-28T14:13:00.166054+0000","last_became_peered":"2024-05-28T14:13:00.166054+0000","last_unstale":"2024-05-28T14:13:01.681910+0000","last_undegraded":"2024-05-28T14:13:01.681910+0000","last_fullsized":"2024-05-28T14:13:01.681910+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:34:57.501530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143618+0000","last_change":"2024-05-28T14:12:59.157949+0000","last_active":"2024-05-28T14:13:01.143618+0000","last_peered":"2024-05-28T14:13:01.143618+0000","last_clean":"2024-05-28T14:13:01.143618+0000","last_became_active":"2024-05-28T14:12:59.157600+0000","last_became_peered":"2024-05-28T14:12:59.157600+0000","last_unstale":"2024-05-28T14:13:01.143618+0000","last_undegraded":"2024-05-28T14:13:01.143618+0000","last_fullsized":"2024-05-28T14:13:01.143618+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:24:19.180753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143624+0000","last_change":"2024-05-28T14:13:00.167579+0000","last_active":"2024-05-28T14:13:01.143624+0000","last_peered":"2024-05-28T14:13:01.143624+0000","last_clean":"2024-05-28T14:13:01.143624+0000","last_became_active":"2024-05-28T14:13:00.166167+0000","last_became_peered":"2024-05-28T14:13:00.166167+0000","last_unstale":"2024-05-28T14:13:01.143624+0000","last_undegraded":"2024-05-28T14:13:01.143624+0000","last_fullsized":"2024-05-28T14:13:01.143624+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:04:15.853596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683796+0000","last_change":"2024-05-28T14:12:59.158675+0000","last_active":"2024-05-28T14:13:01.683796+0000","last_peered":"2024-05-28T14:13:01.683796+0000","last_clean":"2024-05-28T14:13:01.683796+0000","last_became_active":"2024-05-28T14:12:59.158378+0000","last_became_peered":"2024-05-28T14:12:59.158378+0000","last_unstale":"2024-05-28T14:13:01.683796+0000","last_undegraded":"2024-05-28T14:13:01.683796+0000","last_fullsized":"2024-05-28T14:13:01.683796+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:07:17.818267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682021+0000","last_change":"2024-05-28T14:13:00.160893+0000","last_active":"2024-05-28T14:13:01.682021+0000","last_peered":"2024-05-28T14:13:01.682021+0000","last_clean":"2024-05-28T14:13:01.682021+0000","last_became_active":"2024-05-28T14:13:00.157458+0000","last_became_peered":"2024-05-28T14:13:00.157458+0000","last_unstale":"2024-05-28T14:13:01.682021+0000","last_undegraded":"2024-05-28T14:13:01.682021+0000","last_fullsized":"2024-05-28T14:13:01.682021+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:49:44.497244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.142805+0000","last_change":"2024-05-28T14:12:59.162720+0000","last_active":"2024-05-28T14:13:01.142805+0000","last_peered":"2024-05-28T14:13:01.142805+0000","last_clean":"2024-05-28T14:13:01.142805+0000","last_became_active":"2024-05-28T14:12:59.162364+0000","last_became_peered":"2024-05-28T14:12:59.162364+0000","last_unstale":"2024-05-28T14:13:01.142805+0000","last_undegraded":"2024-05-28T14:13:01.142805+0000","last_fullsized":"2024-05-28T14:13:01.142805+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:58:22.303545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143576+0000","last_change":"2024-05-28T14:13:00.169846+0000","last_active":"2024-05-28T14:13:01.143576+0000","last_peered":"2024-05-28T14:13:01.143576+0000","last_clean":"2024-05-28T14:13:01.143576+0000","last_became_active":"2024-05-28T14:13:00.169447+0000","last_became_peered":"2024-05-28T14:13:00.169447+0000","last_unstale":"2024-05-28T14:13:01.143576+0000","last_undegraded":"2024-05-28T14:13:01.143576+0000","last_fullsized":"2024-05-28T14:13:01.143576+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:51:49.229422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682218+0000","last_change":"2024-05-28T14:12:59.163864+0000","last_active":"2024-05-28T14:13:01.682218+0000","last_peered":"2024-05-28T14:13:01.682218+0000","last_clean":"2024-05-28T14:13:01.682218+0000","last_became_active":"2024-05-28T14:12:59.163528+0000","last_became_peered":"2024-05-28T14:12:59.163528+0000","last_unstale":"2024-05-28T14:13:01.682218+0000","last_undegraded":"2024-05-28T14:13:01.682218+0000","last_fullsized":"2024-05-28T14:13:01.682218+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:55.207466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.144100+0000","last_change":"2024-05-28T14:13:00.150217+0000","last_active":"2024-05-28T14:13:01.144100+0000","last_peered":"2024-05-28T14:13:01.144100+0000","last_clean":"2024-05-28T14:13:01.144100+0000","last_became_active":"2024-05-28T14:13:00.149915+0000","last_became_peered":"2024-05-28T14:13:00.149915+0000","last_unstale":"2024-05-28T14:13:01.144100+0000","last_undegraded":"2024-05-28T14:13:01.144100+0000","last_fullsized":"2024-05-28T14:13:01.144100+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:44:59.426282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.144976+0000","last_change":"2024-05-28T14:12:59.165251+0000","last_active":"2024-05-28T14:13:00.144976+0000","last_peered":"2024-05-28T14:13:00.144976+0000","last_clean":"2024-05-28T14:13:00.144976+0000","last_became_active":"2024-05-28T14:12:59.164960+0000","last_became_peered":"2024-05-28T14:12:59.164960+0000","last_unstale":"2024-05-28T14:13:00.144976+0000","last_undegraded":"2024-05-28T14:13:00.144976+0000","last_fullsized":"2024-05-28T14:13:00.144976+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:47:24.425191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.215277+0000","last_change":"2024-05-28T14:13:00.172027+0000","last_active":"2024-05-28T14:13:01.215277+0000","last_peered":"2024-05-28T14:13:01.215277+0000","last_clean":"2024-05-28T14:13:01.215277+0000","last_became_active":"2024-05-28T14:13:00.171756+0000","last_became_peered":"2024-05-28T14:13:00.171756+0000","last_unstale":"2024-05-28T14:13:01.215277+0000","last_undegraded":"2024-05-28T14:13:01.215277+0000","last_fullsized":"2024-05-28T14:13:01.215277+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:52:25.954742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683751+0000","last_change":"2024-05-28T14:12:59.157811+0000","last_active":"2024-05-28T14:13:01.683751+0000","last_peered":"2024-05-28T14:13:01.683751+0000","last_clean":"2024-05-28T14:13:01.683751+0000","last_became_active":"2024-05-28T14:12:59.157503+0000","last_became_peered":"2024-05-28T14:12:59.157503+0000","last_unstale":"2024-05-28T14:13:01.683751+0000","last_undegraded":"2024-05-28T14:13:01.683751+0000","last_fullsized":"2024-05-28T14:13:01.683751+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:17:03.844751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683905+0000","last_change":"2024-05-28T14:13:00.166804+0000","last_active":"2024-05-28T14:13:01.683905+0000","last_peered":"2024-05-28T14:13:01.683905+0000","last_clean":"2024-05-28T14:13:01.683905+0000","last_became_active":"2024-05-28T14:13:00.166553+0000","last_became_peered":"2024-05-28T14:13:00.166553+0000","last_unstale":"2024-05-28T14:13:01.683905+0000","last_undegraded":"2024-05-28T14:13:01.683905+0000","last_fullsized":"2024-05-28T14:13:01.683905+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:27:44.085288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"116'3","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683231+0000","last_change":"2024-05-28T14:12:59.163447+0000","last_active":"2024-05-28T14:13:01.683231+0000","last_peered":"2024-05-28T14:13:01.683231+0000","last_clean":"2024-05-28T14:13:01.683231+0000","last_became_active":"2024-05-28T14:12:59.163160+0000","last_became_peered":"2024-05-28T14:12:59.163160+0000","last_unstale":"2024-05-28T14:13:01.683231+0000","last_undegraded":"2024-05-28T14:13:01.683231+0000","last_fullsized":"2024-05-28T14:13:01.683231+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:28:58.184414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143906+0000","last_change":"2024-05-28T14:13:00.156046+0000","last_active":"2024-05-28T14:13:01.143906+0000","last_peered":"2024-05-28T14:13:01.143906+0000","last_clean":"2024-05-28T14:13:01.143906+0000","last_became_active":"2024-05-28T14:13:00.155769+0000","last_became_peered":"2024-05-28T14:13:00.155769+0000","last_unstale":"2024-05-28T14:13:01.143906+0000","last_undegraded":"2024-05-28T14:13:01.143906+0000","last_fullsized":"2024-05-28T14:13:01.143906+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:57:34.669207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"116'6","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683184+0000","last_change":"2024-05-28T14:12:59.164229+0000","last_active":"2024-05-28T14:13:01.683184+0000","last_peered":"2024-05-28T14:13:01.683184+0000","last_clean":"2024-05-28T14:13:01.683184+0000","last_became_active":"2024-05-28T14:12:59.163931+0000","last_became_peered":"2024-05-28T14:12:59.163931+0000","last_unstale":"2024-05-28T14:13:01.683184+0000","last_undegraded":"2024-05-28T14:13:01.683184+0000","last_fullsized":"2024-05-28T14:13:01.683184+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:46:21.514362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683673+0000","last_change":"2024-05-28T14:13:00.144970+0000","last_active":"2024-05-28T14:13:01.683673+0000","last_peered":"2024-05-28T14:13:01.683673+0000","last_clean":"2024-05-28T14:13:01.683673+0000","last_became_active":"2024-05-28T14:13:00.144678+0000","last_became_peered":"2024-05-28T14:13:00.144678+0000","last_unstale":"2024-05-28T14:13:01.683673+0000","last_undegraded":"2024-05-28T14:13:01.683673+0000","last_fullsized":"2024-05-28T14:13:01.683673+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:43:55.342468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140582+0000","last_change":"2024-05-28T14:12:59.155868+0000","last_active":"2024-05-28T14:13:01.140582+0000","last_peered":"2024-05-28T14:13:01.140582+0000","last_clean":"2024-05-28T14:13:01.140582+0000","last_became_active":"2024-05-28T14:12:59.155584+0000","last_became_peered":"2024-05-28T14:12:59.155584+0000","last_unstale":"2024-05-28T14:13:01.140582+0000","last_undegraded":"2024-05-28T14:13:01.140582+0000","last_fullsized":"2024-05-28T14:13:01.140582+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:53:45.000843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683407+0000","last_change":"2024-05-28T14:13:00.160238+0000","last_active":"2024-05-28T14:13:01.683407+0000","last_peered":"2024-05-28T14:13:01.683407+0000","last_clean":"2024-05-28T14:13:01.683407+0000","last_became_active":"2024-05-28T14:13:00.159985+0000","last_became_peered":"2024-05-28T14:13:00.159985+0000","last_unstale":"2024-05-28T14:13:01.683407+0000","last_undegraded":"2024-05-28T14:13:01.683407+0000","last_fullsized":"2024-05-28T14:13:01.683407+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:27:35.279931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.214915+0000","last_change":"2024-05-28T14:12:59.161033+0000","last_active":"2024-05-28T14:13:01.214915+0000","last_peered":"2024-05-28T14:13:01.214915+0000","last_clean":"2024-05-28T14:13:01.214915+0000","last_became_active":"2024-05-28T14:12:59.160714+0000","last_became_peered":"2024-05-28T14:12:59.160714+0000","last_unstale":"2024-05-28T14:13:01.214915+0000","last_undegraded":"2024-05-28T14:13:01.214915+0000","last_fullsized":"2024-05-28T14:13:01.214915+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:34:57.372805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683087+0000","last_change":"2024-05-28T14:13:00.165320+0000","last_active":"2024-05-28T14:13:01.683087+0000","last_peered":"2024-05-28T14:13:01.683087+0000","last_clean":"2024-05-28T14:13:01.683087+0000","last_became_active":"2024-05-28T14:13:00.165062+0000","last_became_peered":"2024-05-28T14:13:00.165062+0000","last_unstale":"2024-05-28T14:13:01.683087+0000","last_undegraded":"2024-05-28T14:13:01.683087+0000","last_fullsized":"2024-05-28T14:13:01.683087+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:11:55.042880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"116'8","reported_seq":20,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.170210+0000","last_change":"2024-05-28T14:12:59.154364+0000","last_active":"2024-05-28T14:13:00.170210+0000","last_peered":"2024-05-28T14:13:00.170210+0000","last_clean":"2024-05-28T14:13:00.170210+0000","last_became_active":"2024-05-28T14:12:59.152108+0000","last_became_peered":"2024-05-28T14:12:59.152108+0000","last_unstale":"2024-05-28T14:13:00.170210+0000","last_undegraded":"2024-05-28T14:13:00.170210+0000","last_fullsized":"2024-05-28T14:13:00.170210+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:23.615288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.159293+0000","last_change":"2024-05-28T14:13:00.155655+0000","last_active":"2024-05-28T14:13:00.159293+0000","last_peered":"2024-05-28T14:13:00.159293+0000","last_clean":"2024-05-28T14:13:00.159293+0000","last_became_active":"2024-05-28T14:13:00.155230+0000","last_became_peered":"2024-05-28T14:13:00.155230+0000","last_unstale":"2024-05-28T14:13:00.159293+0000","last_undegraded":"2024-05-28T14:13:00.159293+0000","last_fullsized":"2024-05-28T14:13:00.159293+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:23:08.600237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683686+0000","last_change":"2024-05-28T14:12:59.158148+0000","last_active":"2024-05-28T14:13:01.683686+0000","last_peered":"2024-05-28T14:13:01.683686+0000","last_clean":"2024-05-28T14:13:01.683686+0000","last_became_active":"2024-05-28T14:12:59.156240+0000","last_became_peered":"2024-05-28T14:12:59.156240+0000","last_unstale":"2024-05-28T14:13:01.683686+0000","last_undegraded":"2024-05-28T14:13:01.683686+0000","last_fullsized":"2024-05-28T14:13:01.683686+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:04:14.845177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.141011+0000","last_change":"2024-05-28T14:13:00.167593+0000","last_active":"2024-05-28T14:13:01.141011+0000","last_peered":"2024-05-28T14:13:01.141011+0000","last_clean":"2024-05-28T14:13:01.141011+0000","last_became_active":"2024-05-28T14:13:00.167330+0000","last_became_peered":"2024-05-28T14:13:00.167330+0000","last_unstale":"2024-05-28T14:13:01.141011+0000","last_undegraded":"2024-05-28T14:13:01.141011+0000","last_fullsized":"2024-05-28T14:13:01.141011+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:54.956610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.171402+0000","last_change":"2024-05-28T14:12:59.154024+0000","last_active":"2024-05-28T14:13:00.171402+0000","last_peered":"2024-05-28T14:13:00.171402+0000","last_clean":"2024-05-28T14:13:00.171402+0000","last_became_active":"2024-05-28T14:12:59.153637+0000","last_became_peered":"2024-05-28T14:12:59.153637+0000","last_unstale":"2024-05-28T14:13:00.171402+0000","last_undegraded":"2024-05-28T14:13:00.171402+0000","last_fullsized":"2024-05-28T14:13:00.171402+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:24:19.180753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682885+0000","last_change":"2024-05-28T14:13:00.153780+0000","last_active":"2024-05-28T14:13:01.682885+0000","last_peered":"2024-05-28T14:13:01.682885+0000","last_clean":"2024-05-28T14:13:01.682885+0000","last_became_active":"2024-05-28T14:13:00.153504+0000","last_became_peered":"2024-05-28T14:13:00.153504+0000","last_unstale":"2024-05-28T14:13:01.682885+0000","last_undegraded":"2024-05-28T14:13:01.682885+0000","last_fullsized":"2024-05-28T14:13:01.682885+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:24.726727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.215015+0000","last_change":"2024-05-28T14:12:59.161950+0000","last_active":"2024-05-28T14:13:01.215015+0000","last_peered":"2024-05-28T14:13:01.215015+0000","last_clean":"2024-05-28T14:13:01.215015+0000","last_became_active":"2024-05-28T14:12:59.161536+0000","last_became_peered":"2024-05-28T14:12:59.161536+0000","last_unstale":"2024-05-28T14:13:01.215015+0000","last_undegraded":"2024-05-28T14:13:01.215015+0000","last_fullsized":"2024-05-28T14:13:01.215015+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:09:38.636477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683289+0000","last_change":"2024-05-28T14:13:00.157344+0000","last_active":"2024-05-28T14:13:01.683289+0000","last_peered":"2024-05-28T14:13:01.683289+0000","last_clean":"2024-05-28T14:13:01.683289+0000","last_became_active":"2024-05-28T14:13:00.157069+0000","last_became_peered":"2024-05-28T14:13:00.157069+0000","last_unstale":"2024-05-28T14:13:01.683289+0000","last_undegraded":"2024-05-28T14:13:01.683289+0000","last_fullsized":"2024-05-28T14:13:01.683289+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:25:27.542376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143656+0000","last_change":"2024-05-28T14:12:59.155568+0000","last_active":"2024-05-28T14:13:01.143656+0000","last_peered":"2024-05-28T14:13:01.143656+0000","last_clean":"2024-05-28T14:13:01.143656+0000","last_became_active":"2024-05-28T14:12:59.154202+0000","last_became_peered":"2024-05-28T14:12:59.154202+0000","last_unstale":"2024-05-28T14:13:01.143656+0000","last_undegraded":"2024-05-28T14:13:01.143656+0000","last_fullsized":"2024-05-28T14:13:01.143656+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:06.681422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682756+0000","last_change":"2024-05-28T14:13:00.164974+0000","last_active":"2024-05-28T14:13:01.682756+0000","last_peered":"2024-05-28T14:13:01.682756+0000","last_clean":"2024-05-28T14:13:01.682756+0000","last_became_active":"2024-05-28T14:13:00.164697+0000","last_became_peered":"2024-05-28T14:13:00.164697+0000","last_unstale":"2024-05-28T14:13:01.682756+0000","last_undegraded":"2024-05-28T14:13:01.682756+0000","last_fullsized":"2024-05-28T14:13:01.682756+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:01:52.004242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140683+0000","last_change":"2024-05-28T14:12:59.159676+0000","last_active":"2024-05-28T14:13:01.140683+0000","last_peered":"2024-05-28T14:13:01.140683+0000","last_clean":"2024-05-28T14:13:01.140683+0000","last_became_active":"2024-05-28T14:12:59.159391+0000","last_became_peered":"2024-05-28T14:12:59.159391+0000","last_unstale":"2024-05-28T14:13:01.140683+0000","last_undegraded":"2024-05-28T14:13:01.140683+0000","last_fullsized":"2024-05-28T14:13:01.140683+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:53:37.358277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682709+0000","last_change":"2024-05-28T14:13:00.158035+0000","last_active":"2024-05-28T14:13:01.682709+0000","last_peered":"2024-05-28T14:13:01.682709+0000","last_clean":"2024-05-28T14:13:01.682709+0000","last_became_active":"2024-05-28T14:13:00.157761+0000","last_became_peered":"2024-05-28T14:13:00.157761+0000","last_unstale":"2024-05-28T14:13:01.682709+0000","last_undegraded":"2024-05-28T14:13:01.682709+0000","last_fullsized":"2024-05-28T14:13:01.682709+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:24.726727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.142707+0000","last_change":"2024-05-28T14:12:59.153857+0000","last_active":"2024-05-28T14:13:00.142707+0000","last_peered":"2024-05-28T14:13:00.142707+0000","last_clean":"2024-05-28T14:13:00.142707+0000","last_became_active":"2024-05-28T14:12:59.153528+0000","last_became_peered":"2024-05-28T14:12:59.153528+0000","last_unstale":"2024-05-28T14:13:00.142707+0000","last_undegraded":"2024-05-28T14:13:00.142707+0000","last_fullsized":"2024-05-28T14:13:00.142707+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:46:44.183872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682718+0000","last_change":"2024-05-28T14:13:00.164800+0000","last_active":"2024-05-28T14:13:01.682718+0000","last_peered":"2024-05-28T14:13:01.682718+0000","last_clean":"2024-05-28T14:13:01.682718+0000","last_became_active":"2024-05-28T14:13:00.164540+0000","last_became_peered":"2024-05-28T14:13:00.164540+0000","last_unstale":"2024-05-28T14:13:01.682718+0000","last_undegraded":"2024-05-28T14:13:01.682718+0000","last_fullsized":"2024-05-28T14:13:01.682718+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:37:35.996044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140333+0000","last_change":"2024-05-28T14:12:59.153891+0000","last_active":"2024-05-28T14:13:01.140333+0000","last_peered":"2024-05-28T14:13:01.140333+0000","last_clean":"2024-05-28T14:13:01.140333+0000","last_became_active":"2024-05-28T14:12:59.153629+0000","last_became_peered":"2024-05-28T14:12:59.153629+0000","last_unstale":"2024-05-28T14:13:01.140333+0000","last_undegraded":"2024-05-28T14:13:01.140333+0000","last_fullsized":"2024-05-28T14:13:01.140333+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:38:02.389035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683220+0000","last_change":"2024-05-28T14:13:00.151659+0000","last_active":"2024-05-28T14:13:01.683220+0000","last_peered":"2024-05-28T14:13:01.683220+0000","last_clean":"2024-05-28T14:13:01.683220+0000","last_became_active":"2024-05-28T14:13:00.148987+0000","last_became_peered":"2024-05-28T14:13:00.148987+0000","last_unstale":"2024-05-28T14:13:01.683220+0000","last_undegraded":"2024-05-28T14:13:01.683220+0000","last_fullsized":"2024-05-28T14:13:01.683220+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:58:42.460180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682311+0000","last_change":"2024-05-28T14:12:59.159929+0000","last_active":"2024-05-28T14:13:01.682311+0000","last_peered":"2024-05-28T14:13:01.682311+0000","last_clean":"2024-05-28T14:13:01.682311+0000","last_became_active":"2024-05-28T14:12:59.159646+0000","last_became_peered":"2024-05-28T14:12:59.159646+0000","last_unstale":"2024-05-28T14:13:01.682311+0000","last_undegraded":"2024-05-28T14:13:01.682311+0000","last_fullsized":"2024-05-28T14:13:01.682311+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:48:06.747542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.215106+0000","last_change":"2024-05-28T14:13:00.173025+0000","last_active":"2024-05-28T14:13:01.215106+0000","last_peered":"2024-05-28T14:13:01.215106+0000","last_clean":"2024-05-28T14:13:01.215106+0000","last_became_active":"2024-05-28T14:13:00.172597+0000","last_became_peered":"2024-05-28T14:13:00.172597+0000","last_unstale":"2024-05-28T14:13:01.215106+0000","last_undegraded":"2024-05-28T14:13:01.215106+0000","last_fullsized":"2024-05-28T14:13:01.215106+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:48:07.755962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683951+0000","last_change":"2024-05-28T14:12:59.161594+0000","last_active":"2024-05-28T14:13:01.683951+0000","last_peered":"2024-05-28T14:13:01.683951+0000","last_clean":"2024-05-28T14:13:01.683951+0000","last_became_active":"2024-05-28T14:12:59.161326+0000","last_became_peered":"2024-05-28T14:12:59.161326+0000","last_unstale":"2024-05-28T14:13:01.683951+0000","last_undegraded":"2024-05-28T14:13:01.683951+0000","last_fullsized":"2024-05-28T14:13:01.683951+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:28:58.184414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683682+0000","last_change":"2024-05-28T14:13:00.173835+0000","last_active":"2024-05-28T14:13:01.683682+0000","last_peered":"2024-05-28T14:13:01.683682+0000","last_clean":"2024-05-28T14:13:01.683682+0000","last_became_active":"2024-05-28T14:13:00.173372+0000","last_became_peered":"2024-05-28T14:13:00.173372+0000","last_unstale":"2024-05-28T14:13:01.683682+0000","last_undegraded":"2024-05-28T14:13:01.683682+0000","last_fullsized":"2024-05-28T14:13:01.683682+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:42:30.904711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"117'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.215211+0000","last_change":"2024-05-28T14:12:59.159230+0000","last_active":"2024-05-28T14:13:01.215211+0000","last_peered":"2024-05-28T14:13:01.215211+0000","last_clean":"2024-05-28T14:13:01.215211+0000","last_became_active":"2024-05-28T14:12:59.158844+0000","last_became_peered":"2024-05-28T14:12:59.158844+0000","last_unstale":"2024-05-28T14:13:01.215211+0000","last_undegraded":"2024-05-28T14:13:01.215211+0000","last_fullsized":"2024-05-28T14:13:01.215211+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:50:25.519025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683083+0000","last_change":"2024-05-28T14:13:00.155961+0000","last_active":"2024-05-28T14:13:01.683083+0000","last_peered":"2024-05-28T14:13:01.683083+0000","last_clean":"2024-05-28T14:13:01.683083+0000","last_became_active":"2024-05-28T14:13:00.155684+0000","last_became_peered":"2024-05-28T14:13:00.155684+0000","last_unstale":"2024-05-28T14:13:01.683083+0000","last_undegraded":"2024-05-28T14:13:01.683083+0000","last_fullsized":"2024-05-28T14:13:01.683083+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:31:55.696017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683326+0000","last_change":"2024-05-28T14:12:59.159785+0000","last_active":"2024-05-28T14:13:01.683326+0000","last_peered":"2024-05-28T14:13:01.683326+0000","last_clean":"2024-05-28T14:13:01.683326+0000","last_became_active":"2024-05-28T14:12:59.159495+0000","last_became_peered":"2024-05-28T14:12:59.159495+0000","last_unstale":"2024-05-28T14:13:01.683326+0000","last_undegraded":"2024-05-28T14:13:01.683326+0000","last_fullsized":"2024-05-28T14:13:01.683326+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:41:03.883845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.214942+0000","last_change":"2024-05-28T14:13:00.172970+0000","last_active":"2024-05-28T14:13:01.214942+0000","last_peered":"2024-05-28T14:13:01.214942+0000","last_clean":"2024-05-28T14:13:01.214942+0000","last_became_active":"2024-05-28T14:13:00.172680+0000","last_became_peered":"2024-05-28T14:13:00.172680+0000","last_unstale":"2024-05-28T14:13:01.214942+0000","last_undegraded":"2024-05-28T14:13:01.214942+0000","last_fullsized":"2024-05-28T14:13:01.214942+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:41:04.892264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683864+0000","last_change":"2024-05-28T14:12:59.160575+0000","last_active":"2024-05-28T14:13:01.683864+0000","last_peered":"2024-05-28T14:13:01.683864+0000","last_clean":"2024-05-28T14:13:01.683864+0000","last_became_active":"2024-05-28T14:12:59.160277+0000","last_became_peered":"2024-05-28T14:12:59.160277+0000","last_unstale":"2024-05-28T14:13:01.683864+0000","last_undegraded":"2024-05-28T14:13:01.683864+0000","last_fullsized":"2024-05-28T14:13:01.683864+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:41:03.883845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143241+0000","last_change":"2024-05-28T14:13:00.167453+0000","last_active":"2024-05-28T14:13:01.143241+0000","last_peered":"2024-05-28T14:13:01.143241+0000","last_clean":"2024-05-28T14:13:01.143241+0000","last_became_active":"2024-05-28T14:13:00.167176+0000","last_became_peered":"2024-05-28T14:13:00.167176+0000","last_unstale":"2024-05-28T14:13:01.143241+0000","last_undegraded":"2024-05-28T14:13:01.143241+0000","last_fullsized":"2024-05-28T14:13:01.143241+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:56.215885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143756+0000","last_change":"2024-05-28T14:12:59.155180+0000","last_active":"2024-05-28T14:13:01.143756+0000","last_peered":"2024-05-28T14:13:01.143756+0000","last_clean":"2024-05-28T14:13:01.143756+0000","last_became_active":"2024-05-28T14:12:59.154912+0000","last_became_peered":"2024-05-28T14:12:59.154912+0000","last_unstale":"2024-05-28T14:13:01.143756+0000","last_undegraded":"2024-05-28T14:13:01.143756+0000","last_fullsized":"2024-05-28T14:13:01.143756+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:47:24.425191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143654+0000","last_change":"2024-05-28T14:13:00.155179+0000","last_active":"2024-05-28T14:13:01.143654+0000","last_peered":"2024-05-28T14:13:01.143654+0000","last_clean":"2024-05-28T14:13:01.143654+0000","last_became_active":"2024-05-28T14:13:00.154781+0000","last_became_peered":"2024-05-28T14:13:00.154781+0000","last_unstale":"2024-05-28T14:13:01.143654+0000","last_undegraded":"2024-05-28T14:13:01.143654+0000","last_fullsized":"2024-05-28T14:13:01.143654+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:02:35.428286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143226+0000","last_change":"2024-05-28T14:12:59.156463+0000","last_active":"2024-05-28T14:13:01.143226+0000","last_peered":"2024-05-28T14:13:01.143226+0000","last_clean":"2024-05-28T14:13:01.143226+0000","last_became_active":"2024-05-28T14:12:59.155528+0000","last_became_peered":"2024-05-28T14:12:59.155528+0000","last_unstale":"2024-05-28T14:13:01.143226+0000","last_undegraded":"2024-05-28T14:13:01.143226+0000","last_fullsized":"2024-05-28T14:13:01.143226+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:38:27.527964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682877+0000","last_change":"2024-05-28T14:13:00.173560+0000","last_active":"2024-05-28T14:13:01.682877+0000","last_peered":"2024-05-28T14:13:01.682877+0000","last_clean":"2024-05-28T14:13:01.682877+0000","last_became_active":"2024-05-28T14:13:00.173281+0000","last_became_peered":"2024-05-28T14:13:00.173281+0000","last_unstale":"2024-05-28T14:13:01.682877+0000","last_undegraded":"2024-05-28T14:13:01.682877+0000","last_fullsized":"2024-05-28T14:13:01.682877+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:51:22.209282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-28T14:13:00.142671+0000","last_change":"2024-05-28T14:12:59.154458+0000","last_active":"2024-05-28T14:13:00.142671+0000","last_peered":"2024-05-28T14:13:00.142671+0000","last_clean":"2024-05-28T14:13:00.142671+0000","last_became_active":"2024-05-28T14:12:59.153652+0000","last_became_peered":"2024-05-28T14:12:59.153652+0000","last_unstale":"2024-05-28T14:13:00.142671+0000","last_undegraded":"2024-05-28T14:13:00.142671+0000","last_fullsized":"2024-05-28T14:13:00.142671+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:39:27.979374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682492+0000","last_change":"2024-05-28T14:13:00.164867+0000","last_active":"2024-05-28T14:13:01.682492+0000","last_peered":"2024-05-28T14:13:01.682492+0000","last_clean":"2024-05-28T14:13:01.682492+0000","last_became_active":"2024-05-28T14:13:00.164613+0000","last_became_peered":"2024-05-28T14:13:00.164613+0000","last_unstale":"2024-05-28T14:13:01.682492+0000","last_undegraded":"2024-05-28T14:13:01.682492+0000","last_fullsized":"2024-05-28T14:13:01.682492+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:37:35.996044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682406+0000","last_change":"2024-05-28T14:12:59.158941+0000","last_active":"2024-05-28T14:13:01.682406+0000","last_peered":"2024-05-28T14:13:01.682406+0000","last_clean":"2024-05-28T14:13:01.682406+0000","last_became_active":"2024-05-28T14:12:59.158680+0000","last_became_peered":"2024-05-28T14:12:59.158680+0000","last_unstale":"2024-05-28T14:13:01.682406+0000","last_undegraded":"2024-05-28T14:13:01.682406+0000","last_fullsized":"2024-05-28T14:13:01.682406+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:52:24.946323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683030+0000","last_change":"2024-05-28T14:13:00.157032+0000","last_active":"2024-05-28T14:13:01.683030+0000","last_peered":"2024-05-28T14:13:01.683030+0000","last_clean":"2024-05-28T14:13:01.683030+0000","last_became_active":"2024-05-28T14:13:00.156051+0000","last_became_peered":"2024-05-28T14:13:00.156051+0000","last_unstale":"2024-05-28T14:13:01.683030+0000","last_undegraded":"2024-05-28T14:13:01.683030+0000","last_fullsized":"2024-05-28T14:13:01.683030+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:59.737418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.142947+0000","last_change":"2024-05-28T14:12:59.154352+0000","last_active":"2024-05-28T14:13:01.142947+0000","last_peered":"2024-05-28T14:13:01.142947+0000","last_clean":"2024-05-28T14:13:01.142947+0000","last_became_active":"2024-05-28T14:12:59.154075+0000","last_became_peered":"2024-05-28T14:12:59.154075+0000","last_unstale":"2024-05-28T14:13:01.142947+0000","last_undegraded":"2024-05-28T14:13:01.142947+0000","last_fullsized":"2024-05-28T14:13:01.142947+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:07:49.480989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":351,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683050+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:13:01.683050+0000","last_peered":"2024-05-28T14:13:01.683050+0000","last_clean":"2024-05-28T14:13:01.683050+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:13:01.683050+0000","last_undegraded":"2024-05-28T14:13:01.683050+0000","last_fullsized":"2024-05-28T14:13:01.683050+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575029,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131684,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240156,"statfs":{"total":96636764160,"available":96501919744,"internally_reserved":0,"allocated":2056192,"data_stored":1227728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575029,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131100,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240740,"statfs":{"total":96636764160,"available":96502517760,"internally_reserved":0,"allocated":1458176,"data_stored":636814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575031,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64480,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307360,"statfs":{"total":96636764160,"available":96570736640,"internally_reserved":0,"allocated":1462272,"data_stored":637348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575030,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144984,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226856,"statfs":{"total":96636764160,"available":96488300544,"internally_reserved":0,"allocated":1454080,"data_stored":636744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575029,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122792,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249048,"statfs":{"total":96636764160,"available":96511025152,"internally_reserved":0,"allocated":1470464,"data_stored":638426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":124254664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575029,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136640,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235200,"statfs":{"total":96636764160,"available":96496844800,"internally_reserved":0,"allocated":1429504,"data_stored":623557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575029,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149160,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222680,"statfs":{"total":96636764160,"available":96484024320,"internally_reserved":0,"allocated":1462272,"data_stored":636868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575030,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91440,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280400,"statfs":{"total":96636764160,"available":96543129600,"internally_reserved":0,"allocated":2068480,"data_stored":1228836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:13:08.841 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:13:08.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd18d800640 1 Processor -- start 2024-05-28T14:13:08.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd18d800640 1 -- start start 2024-05-28T14:13:08.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd18d800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd18815c720 0x7fd1880b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:08.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd18d800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd188071df0 0x7fd18809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:08.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd18d800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd18809af90 0x7fd188070220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:08.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd18d800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd1880cd390 con 0x7fd18815c720 2024-05-28T14:13:08.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd18d800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd1880cd670 con 0x7fd188071df0 2024-05-28T14:13:08.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd18d800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd1880cd950 con 0x7fd18809af90 2024-05-28T14:13:08.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd186a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd18815c720 0x7fd1880b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:08.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd186a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd18815c720 0x7fd1880b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39476/0 (socket says 172.21.15.42:39476) 2024-05-28T14:13:08.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd186a00640 1 -- 172.21.15.42:0/1627736378 learned_addr learned my addr 172.21.15.42:0/1627736378 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:08.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd17e000640 1 --2- 172.21.15.42:0/1627736378 >> v2:172.21.15.63:3300/0 conn(0x7fd188071df0 0x7fd18809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:08.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd187400640 1 --2- 172.21.15.42:0/1627736378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd18809af90 0x7fd188070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:08.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd186a00640 1 -- 172.21.15.42:0/1627736378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd18809af90 msgr2=0x7fd188070220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:08.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd186a00640 1 --2- 172.21.15.42:0/1627736378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd18809af90 0x7fd188070220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:08.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd186a00640 1 -- 172.21.15.42:0/1627736378 >> v2:172.21.15.63:3300/0 conn(0x7fd188071df0 msgr2=0x7fd18809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:08.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd186a00640 1 --2- 172.21.15.42:0/1627736378 >> v2:172.21.15.63:3300/0 conn(0x7fd188071df0 0x7fd18809aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:08.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.011+0000 7fd186a00640 1 -- 172.21.15.42:0/1627736378 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1880cdc30 con 0x7fd18815c720 2024-05-28T14:13:09.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.015+0000 7fd186a00640 1 --2- 172.21.15.42:0/1627736378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd18815c720 0x7fd1880b1760 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fd1880cee90 tx=0x7fd17801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ced7920e1dd894 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:09.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.015+0000 7fd186000640 1 -- 172.21.15.42:0/1627736378 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1780b2070 con 0x7fd18815c720 2024-05-28T14:13:09.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.015+0000 7fd186000640 1 -- 172.21.15.42:0/1627736378 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1780ad640 con 0x7fd18815c720 2024-05-28T14:13:09.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.015+0000 7fd186000640 1 -- 172.21.15.42:0/1627736378 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1780ada10 con 0x7fd18815c720 2024-05-28T14:13:09.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.015+0000 7fd18d800640 1 -- 172.21.15.42:0/1627736378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd18815c720 msgr2=0x7fd1880b1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.015+0000 7fd18d800640 1 --2- 172.21.15.42:0/1627736378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd18815c720 0x7fd1880b1760 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fd1880cee90 tx=0x7fd17801c520 comp rx=0 tx=0).stop 2024-05-28T14:13:09.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.015+0000 7fd18d800640 1 -- 172.21.15.42:0/1627736378 shutdown_connections 2024-05-28T14:13:09.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.015+0000 7fd18d800640 1 --2- 172.21.15.42:0/1627736378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd18809af90 0x7fd188070220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.015+0000 7fd18d800640 1 --2- 172.21.15.42:0/1627736378 >> v2:172.21.15.63:3300/0 conn(0x7fd188071df0 0x7fd18809aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.015+0000 7fd18d800640 1 --2- 172.21.15.42:0/1627736378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd18815c720 0x7fd1880b1760 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.015+0000 7fd18d800640 1 -- 172.21.15.42:0/1627736378 >> 172.21.15.42:0/1627736378 conn(0x7fd1880b3fa0 msgr2=0x7fd1880b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:09.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.015+0000 7fd18d800640 1 -- 172.21.15.42:0/1627736378 shutdown_connections 2024-05-28T14:13:09.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.015+0000 7fd18d800640 1 -- 172.21.15.42:0/1627736378 wait complete. 2024-05-28T14:13:09.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd18d800640 1 Processor -- start 2024-05-28T14:13:09.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd18d800640 1 -- start start 2024-05-28T14:13:09.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd18d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd188071df0 0x7fd18814d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd18d800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd18809af90 0x7fd1881513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd18d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd18815c720 0x7fd188151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd18d800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd1880d04e0 con 0x7fd18815c720 2024-05-28T14:13:09.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd18d800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd1880d0360 con 0x7fd188071df0 2024-05-28T14:13:09.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd18d800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd1880d0660 con 0x7fd18809af90 2024-05-28T14:13:09.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd187400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd18815c720 0x7fd188151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd187400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd18815c720 0x7fd188151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39482/0 (socket says 172.21.15.42:39482) 2024-05-28T14:13:09.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd187400640 1 -- 172.21.15.42:0/2478889998 learned_addr learned my addr 172.21.15.42:0/2478889998 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:09.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd186a00640 1 --2- 172.21.15.42:0/2478889998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd188071df0 0x7fd18814d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd17e000640 1 --2- 172.21.15.42:0/2478889998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd18809af90 0x7fd1881513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd187400640 1 -- 172.21.15.42:0/2478889998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd18809af90 msgr2=0x7fd1881513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd187400640 1 --2- 172.21.15.42:0/2478889998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd18809af90 0x7fd1881513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd187400640 1 -- 172.21.15.42:0/2478889998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd188071df0 msgr2=0x7fd18814d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd187400640 1 --2- 172.21.15.42:0/2478889998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd188071df0 0x7fd18814d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd187400640 1 -- 172.21.15.42:0/2478889998 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18815a1c0 con 0x7fd18815c720 2024-05-28T14:13:09.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd187400640 1 --2- 172.21.15.42:0/2478889998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd18815c720 0x7fd188151900 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd180007e40 tx=0x7fd18000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:09.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd17f400640 1 -- 172.21.15.42:0/2478889998 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180014620 con 0x7fd18815c720 2024-05-28T14:13:09.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd17f400640 1 -- 172.21.15.42:0/2478889998 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd180019070 con 0x7fd18815c720 2024-05-28T14:13:09.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd18d800640 1 -- 172.21.15.42:0/2478889998 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1881506a0 con 0x7fd18815c720 2024-05-28T14:13:09.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd17f400640 1 -- 172.21.15.42:0/2478889998 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180014a20 con 0x7fd18815c720 2024-05-28T14:13:09.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.019+0000 7fd18d800640 1 -- 172.21.15.42:0/2478889998 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd188150c60 con 0x7fd18815c720 2024-05-28T14:13:09.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.023+0000 7fd18d800640 1 -- 172.21.15.42:0/2478889998 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd188068a90 con 0x7fd18815c720 2024-05-28T14:13:09.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.023+0000 7fd17f400640 1 -- 172.21.15.42:0/2478889998 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fd180014bc0 con 0x7fd18815c720 2024-05-28T14:13:09.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.023+0000 7fd17f400640 1 --2- 172.21.15.42:0/2478889998 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd164069b20 0x7fd16406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.023+0000 7fd17f400640 1 -- 172.21.15.42:0/2478889998 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7fd180085f20 con 0x7fd18815c720 2024-05-28T14:13:09.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.023+0000 7fd186a00640 1 --2- 172.21.15.42:0/2478889998 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd164069b20 0x7fd16406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.023+0000 7fd186a00640 1 --2- 172.21.15.42:0/2478889998 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd164069b20 0x7fd16406bfe0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd1780af000 tx=0x7fd178008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:09.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.027+0000 7fd17f400640 1 -- 172.21.15.42:0/2478889998 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd18815a1c0 con 0x7fd18815c720 2024-05-28T14:13:09.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.191+0000 7fd18d800640 1 -- 172.21.15.42:0/2478889998 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd188156440 con 0x7fd164069b20 2024-05-28T14:13:09.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.199+0000 7fd17f400640 1 -- 172.21.15.42:0/2478889998 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252501 (secure 0 0 0) 0x7fd188156440 con 0x7fd164069b20 2024-05-28T14:13:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:09.188 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:13:09.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.203+0000 7fd18d800640 1 -- 172.21.15.42:0/2478889998 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd164069b20 msgr2=0x7fd16406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.203+0000 7fd18d800640 1 --2- 172.21.15.42:0/2478889998 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd164069b20 0x7fd16406bfe0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd1780af000 tx=0x7fd178008040 comp rx=0 tx=0).stop 2024-05-28T14:13:09.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.203+0000 7fd18d800640 1 -- 172.21.15.42:0/2478889998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd18815c720 msgr2=0x7fd188151900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.203+0000 7fd18d800640 1 --2- 172.21.15.42:0/2478889998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd18815c720 0x7fd188151900 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd180007e40 tx=0x7fd18000e4b0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.203+0000 7fd18d800640 1 -- 172.21.15.42:0/2478889998 shutdown_connections 2024-05-28T14:13:09.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.203+0000 7fd18d800640 1 --2- 172.21.15.42:0/2478889998 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd164069b20 0x7fd16406bfe0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.203+0000 7fd18d800640 1 --2- 172.21.15.42:0/2478889998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd18815c720 0x7fd188151900 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.203+0000 7fd18d800640 1 --2- 172.21.15.42:0/2478889998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd18809af90 0x7fd1881513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.203+0000 7fd18d800640 1 --2- 172.21.15.42:0/2478889998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd188071df0 0x7fd18814d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.203+0000 7fd18d800640 1 -- 172.21.15.42:0/2478889998 >> 172.21.15.42:0/2478889998 conn(0x7fd1880b3fa0 msgr2=0x7fd188074a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:09.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.203+0000 7fd18d800640 1 -- 172.21.15.42:0/2478889998 shutdown_connections 2024-05-28T14:13:09.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.203+0000 7fd18d800640 1 -- 172.21.15.42:0/2478889998 wait complete. 2024-05-28T14:13:09.207 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":560,"stamp":"2024-05-28T14:13:09.165779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":972312,"kb_used_data":12584,"kb_used_omap":12,"kb_used_meta":959603,"kb_avail":754002408,"statfs":{"total":773094113280,"available":772098465792,"internally_reserved":0,"allocated":12886016,"data_stored":6279508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":982634034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.028604"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683531+0000","last_change":"2024-05-28T14:13:00.168249+0000","last_active":"2024-05-28T14:13:01.683531+0000","last_peered":"2024-05-28T14:13:01.683531+0000","last_clean":"2024-05-28T14:13:01.683531+0000","last_became_active":"2024-05-28T14:13:00.168012+0000","last_became_peered":"2024-05-28T14:13:00.168012+0000","last_unstale":"2024-05-28T14:13:01.683531+0000","last_undegraded":"2024-05-28T14:13:01.683531+0000","last_fullsized":"2024-05-28T14:13:01.683531+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:00:01.461588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.142978+0000","last_change":"2024-05-28T14:13:00.164766+0000","last_active":"2024-05-28T14:13:01.142978+0000","last_peered":"2024-05-28T14:13:01.142978+0000","last_clean":"2024-05-28T14:13:01.142978+0000","last_became_active":"2024-05-28T14:13:00.164510+0000","last_became_peered":"2024-05-28T14:13:00.164510+0000","last_unstale":"2024-05-28T14:13:01.142978+0000","last_undegraded":"2024-05-28T14:13:01.142978+0000","last_fullsized":"2024-05-28T14:13:01.142978+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:17:04.853171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683126+0000","last_change":"2024-05-28T14:13:00.166389+0000","last_active":"2024-05-28T14:13:01.683126+0000","last_peered":"2024-05-28T14:13:01.683126+0000","last_clean":"2024-05-28T14:13:01.683126+0000","last_became_active":"2024-05-28T14:13:00.165990+0000","last_became_peered":"2024-05-28T14:13:00.165990+0000","last_unstale":"2024-05-28T14:13:01.683126+0000","last_undegraded":"2024-05-28T14:13:01.683126+0000","last_fullsized":"2024-05-28T14:13:01.683126+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:07:22.275443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682922+0000","last_change":"2024-05-28T14:13:00.174762+0000","last_active":"2024-05-28T14:13:01.682922+0000","last_peered":"2024-05-28T14:13:01.682922+0000","last_clean":"2024-05-28T14:13:01.682922+0000","last_became_active":"2024-05-28T14:13:00.174288+0000","last_became_peered":"2024-05-28T14:13:00.174288+0000","last_unstale":"2024-05-28T14:13:01.682922+0000","last_undegraded":"2024-05-28T14:13:01.682922+0000","last_fullsized":"2024-05-28T14:13:01.682922+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:51:22.209282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682382+0000","last_change":"2024-05-28T14:13:00.146533+0000","last_active":"2024-05-28T14:13:01.682382+0000","last_peered":"2024-05-28T14:13:01.682382+0000","last_clean":"2024-05-28T14:13:01.682382+0000","last_became_active":"2024-05-28T14:13:00.145774+0000","last_became_peered":"2024-05-28T14:13:00.145774+0000","last_unstale":"2024-05-28T14:13:01.682382+0000","last_undegraded":"2024-05-28T14:13:01.682382+0000","last_fullsized":"2024-05-28T14:13:01.682382+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:23:44.667753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140454+0000","last_change":"2024-05-28T14:13:00.171900+0000","last_active":"2024-05-28T14:13:01.140454+0000","last_peered":"2024-05-28T14:13:01.140454+0000","last_clean":"2024-05-28T14:13:01.140454+0000","last_became_active":"2024-05-28T14:13:00.171595+0000","last_became_peered":"2024-05-28T14:13:00.171595+0000","last_unstale":"2024-05-28T14:13:01.140454+0000","last_undegraded":"2024-05-28T14:13:01.140454+0000","last_fullsized":"2024-05-28T14:13:01.140454+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:47:25.433610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.142830+0000","last_change":"2024-05-28T14:13:00.169682+0000","last_active":"2024-05-28T14:13:01.142830+0000","last_peered":"2024-05-28T14:13:01.142830+0000","last_clean":"2024-05-28T14:13:01.142830+0000","last_became_active":"2024-05-28T14:13:00.169114+0000","last_became_peered":"2024-05-28T14:13:00.169114+0000","last_unstale":"2024-05-28T14:13:01.142830+0000","last_undegraded":"2024-05-28T14:13:01.142830+0000","last_fullsized":"2024-05-28T14:13:01.142830+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:40:57.469457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682647+0000","last_change":"2024-05-28T14:13:00.159023+0000","last_active":"2024-05-28T14:13:01.682647+0000","last_peered":"2024-05-28T14:13:01.682647+0000","last_clean":"2024-05-28T14:13:01.682647+0000","last_became_active":"2024-05-28T14:13:00.158756+0000","last_became_peered":"2024-05-28T14:13:00.158756+0000","last_unstale":"2024-05-28T14:13:01.682647+0000","last_undegraded":"2024-05-28T14:13:01.682647+0000","last_fullsized":"2024-05-28T14:13:01.682647+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:49:44.497244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683577+0000","last_change":"2024-05-28T14:13:00.161469+0000","last_active":"2024-05-28T14:13:01.683577+0000","last_peered":"2024-05-28T14:13:01.683577+0000","last_clean":"2024-05-28T14:13:01.683577+0000","last_became_active":"2024-05-28T14:13:00.161203+0000","last_became_peered":"2024-05-28T14:13:00.161203+0000","last_unstale":"2024-05-28T14:13:01.683577+0000","last_undegraded":"2024-05-28T14:13:01.683577+0000","last_fullsized":"2024-05-28T14:13:01.683577+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:25:27.542376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683102+0000","last_change":"2024-05-28T14:13:00.154915+0000","last_active":"2024-05-28T14:13:01.683102+0000","last_peered":"2024-05-28T14:13:01.683102+0000","last_clean":"2024-05-28T14:13:01.683102+0000","last_became_active":"2024-05-28T14:13:00.152360+0000","last_became_peered":"2024-05-28T14:13:00.152360+0000","last_unstale":"2024-05-28T14:13:01.683102+0000","last_undegraded":"2024-05-28T14:13:01.683102+0000","last_fullsized":"2024-05-28T14:13:01.683102+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:26:30.453204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143023+0000","last_change":"2024-05-28T14:13:00.172238+0000","last_active":"2024-05-28T14:13:01.143023+0000","last_peered":"2024-05-28T14:13:01.143023+0000","last_clean":"2024-05-28T14:13:01.143023+0000","last_became_active":"2024-05-28T14:13:00.171973+0000","last_became_peered":"2024-05-28T14:13:00.171973+0000","last_unstale":"2024-05-28T14:13:01.143023+0000","last_undegraded":"2024-05-28T14:13:01.143023+0000","last_fullsized":"2024-05-28T14:13:01.143023+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:28:59.192834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683790+0000","last_change":"2024-05-28T14:13:00.152818+0000","last_active":"2024-05-28T14:13:01.683790+0000","last_peered":"2024-05-28T14:13:01.683790+0000","last_clean":"2024-05-28T14:13:01.683790+0000","last_became_active":"2024-05-28T14:13:00.152551+0000","last_became_peered":"2024-05-28T14:13:00.152551+0000","last_unstale":"2024-05-28T14:13:01.683790+0000","last_undegraded":"2024-05-28T14:13:01.683790+0000","last_fullsized":"2024-05-28T14:13:01.683790+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:39:28.289636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683028+0000","last_change":"2024-05-28T14:13:00.157994+0000","last_active":"2024-05-28T14:13:01.683028+0000","last_peered":"2024-05-28T14:13:01.683028+0000","last_clean":"2024-05-28T14:13:01.683028+0000","last_became_active":"2024-05-28T14:13:00.157671+0000","last_became_peered":"2024-05-28T14:13:00.157671+0000","last_unstale":"2024-05-28T14:13:01.683028+0000","last_undegraded":"2024-05-28T14:13:01.683028+0000","last_fullsized":"2024-05-28T14:13:01.683028+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:31:55.696017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682579+0000","last_change":"2024-05-28T14:13:00.168829+0000","last_active":"2024-05-28T14:13:01.682579+0000","last_peered":"2024-05-28T14:13:01.682579+0000","last_clean":"2024-05-28T14:13:01.682579+0000","last_became_active":"2024-05-28T14:13:00.168582+0000","last_became_peered":"2024-05-28T14:13:00.168582+0000","last_unstale":"2024-05-28T14:13:01.682579+0000","last_undegraded":"2024-05-28T14:13:01.682579+0000","last_fullsized":"2024-05-28T14:13:01.682579+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:34:57.501530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140255+0000","last_change":"2024-05-28T14:13:00.167892+0000","last_active":"2024-05-28T14:13:01.140255+0000","last_peered":"2024-05-28T14:13:01.140255+0000","last_clean":"2024-05-28T14:13:01.140255+0000","last_became_active":"2024-05-28T14:13:00.167515+0000","last_became_peered":"2024-05-28T14:13:00.167515+0000","last_unstale":"2024-05-28T14:13:01.140255+0000","last_undegraded":"2024-05-28T14:13:01.140255+0000","last_fullsized":"2024-05-28T14:13:01.140255+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:07.689842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.681856+0000","last_change":"2024-05-28T14:13:00.160368+0000","last_active":"2024-05-28T14:13:01.681856+0000","last_peered":"2024-05-28T14:13:01.681856+0000","last_clean":"2024-05-28T14:13:01.681856+0000","last_became_active":"2024-05-28T14:13:00.157771+0000","last_became_peered":"2024-05-28T14:13:00.157771+0000","last_unstale":"2024-05-28T14:13:01.681856+0000","last_undegraded":"2024-05-28T14:13:01.681856+0000","last_fullsized":"2024-05-28T14:13:01.681856+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:24:24.603471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143283+0000","last_change":"2024-05-28T14:13:00.172222+0000","last_active":"2024-05-28T14:13:01.143283+0000","last_peered":"2024-05-28T14:13:01.143283+0000","last_clean":"2024-05-28T14:13:01.143283+0000","last_became_active":"2024-05-28T14:13:00.170660+0000","last_became_peered":"2024-05-28T14:13:00.170660+0000","last_unstale":"2024-05-28T14:13:01.143283+0000","last_undegraded":"2024-05-28T14:13:01.143283+0000","last_fullsized":"2024-05-28T14:13:01.143283+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:48:07.755962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.215031+0000","last_change":"2024-05-28T14:13:00.173278+0000","last_active":"2024-05-28T14:13:01.215031+0000","last_peered":"2024-05-28T14:13:01.215031+0000","last_clean":"2024-05-28T14:13:01.215031+0000","last_became_active":"2024-05-28T14:13:00.173005+0000","last_became_peered":"2024-05-28T14:13:00.173005+0000","last_unstale":"2024-05-28T14:13:01.215031+0000","last_undegraded":"2024-05-28T14:13:01.215031+0000","last_fullsized":"2024-05-28T14:13:01.215031+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:56.215885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683072+0000","last_change":"2024-05-28T14:13:00.159008+0000","last_active":"2024-05-28T14:13:01.683072+0000","last_peered":"2024-05-28T14:13:01.683072+0000","last_clean":"2024-05-28T14:13:01.683072+0000","last_became_active":"2024-05-28T14:13:00.157662+0000","last_became_peered":"2024-05-28T14:13:00.157662+0000","last_unstale":"2024-05-28T14:13:01.683072+0000","last_undegraded":"2024-05-28T14:13:01.683072+0000","last_fullsized":"2024-05-28T14:13:01.683072+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:26:30.453204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143369+0000","last_change":"2024-05-28T14:13:00.162934+0000","last_active":"2024-05-28T14:13:01.143369+0000","last_peered":"2024-05-28T14:13:01.143369+0000","last_clean":"2024-05-28T14:13:01.143369+0000","last_became_active":"2024-05-28T14:13:00.160020+0000","last_became_peered":"2024-05-28T14:13:00.160020+0000","last_unstale":"2024-05-28T14:13:01.143369+0000","last_undegraded":"2024-05-28T14:13:01.143369+0000","last_fullsized":"2024-05-28T14:13:01.143369+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:52:25.954742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143829+0000","last_change":"2024-05-28T14:13:00.152974+0000","last_active":"2024-05-28T14:13:01.143829+0000","last_peered":"2024-05-28T14:13:01.143829+0000","last_clean":"2024-05-28T14:13:01.143829+0000","last_became_active":"2024-05-28T14:13:00.152536+0000","last_became_peered":"2024-05-28T14:13:00.152536+0000","last_unstale":"2024-05-28T14:13:01.143829+0000","last_undegraded":"2024-05-28T14:13:01.143829+0000","last_fullsized":"2024-05-28T14:13:01.143829+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:03:18.352698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683689+0000","last_change":"2024-05-28T14:13:00.170898+0000","last_active":"2024-05-28T14:13:01.683689+0000","last_peered":"2024-05-28T14:13:01.683689+0000","last_clean":"2024-05-28T14:13:01.683689+0000","last_became_active":"2024-05-28T14:13:00.170425+0000","last_became_peered":"2024-05-28T14:13:00.170425+0000","last_unstale":"2024-05-28T14:13:01.683689+0000","last_undegraded":"2024-05-28T14:13:01.683689+0000","last_fullsized":"2024-05-28T14:13:01.683689+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:06.645451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.142671+0000","last_change":"2024-05-28T14:13:00.158631+0000","last_active":"2024-05-28T14:13:01.142671+0000","last_peered":"2024-05-28T14:13:01.142671+0000","last_clean":"2024-05-28T14:13:01.142671+0000","last_became_active":"2024-05-28T14:13:00.158095+0000","last_became_peered":"2024-05-28T14:13:00.158095+0000","last_unstale":"2024-05-28T14:13:01.142671+0000","last_undegraded":"2024-05-28T14:13:01.142671+0000","last_fullsized":"2024-05-28T14:13:01.142671+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:07:18.826687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682838+0000","last_change":"2024-05-28T14:13:00.156579+0000","last_active":"2024-05-28T14:13:01.682838+0000","last_peered":"2024-05-28T14:13:01.682838+0000","last_clean":"2024-05-28T14:13:01.682838+0000","last_became_active":"2024-05-28T14:13:00.156100+0000","last_became_peered":"2024-05-28T14:13:00.156100+0000","last_unstale":"2024-05-28T14:13:01.682838+0000","last_undegraded":"2024-05-28T14:13:01.682838+0000","last_fullsized":"2024-05-28T14:13:01.682838+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:19:52.357432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143442+0000","last_change":"2024-05-28T14:13:00.170638+0000","last_active":"2024-05-28T14:13:01.143442+0000","last_peered":"2024-05-28T14:13:01.143442+0000","last_clean":"2024-05-28T14:13:01.143442+0000","last_became_active":"2024-05-28T14:13:00.170278+0000","last_became_peered":"2024-05-28T14:13:00.170278+0000","last_unstale":"2024-05-28T14:13:01.143442+0000","last_undegraded":"2024-05-28T14:13:01.143442+0000","last_fullsized":"2024-05-28T14:13:01.143442+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:46:22.522782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683376+0000","last_change":"2024-05-28T14:13:00.154163+0000","last_active":"2024-05-28T14:13:01.683376+0000","last_peered":"2024-05-28T14:13:01.683376+0000","last_clean":"2024-05-28T14:13:01.683376+0000","last_became_active":"2024-05-28T14:13:00.153904+0000","last_became_peered":"2024-05-28T14:13:00.153904+0000","last_unstale":"2024-05-28T14:13:01.683376+0000","last_undegraded":"2024-05-28T14:13:01.683376+0000","last_fullsized":"2024-05-28T14:13:01.683376+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:07:22.275443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140930+0000","last_change":"2024-05-28T14:13:00.173166+0000","last_active":"2024-05-28T14:13:01.140930+0000","last_peered":"2024-05-28T14:13:01.140930+0000","last_clean":"2024-05-28T14:13:01.140930+0000","last_became_active":"2024-05-28T14:13:00.171662+0000","last_became_peered":"2024-05-28T14:13:00.171662+0000","last_unstale":"2024-05-28T14:13:01.140930+0000","last_undegraded":"2024-05-28T14:13:01.140930+0000","last_fullsized":"2024-05-28T14:13:01.140930+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:23:08.600237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683761+0000","last_change":"2024-05-28T14:13:00.152220+0000","last_active":"2024-05-28T14:13:01.683761+0000","last_peered":"2024-05-28T14:13:01.683761+0000","last_clean":"2024-05-28T14:13:01.683761+0000","last_became_active":"2024-05-28T14:13:00.151960+0000","last_became_peered":"2024-05-28T14:13:00.151960+0000","last_unstale":"2024-05-28T14:13:01.683761+0000","last_undegraded":"2024-05-28T14:13:01.683761+0000","last_fullsized":"2024-05-28T14:13:01.683761+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:53:04.689621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143174+0000","last_change":"2024-05-28T14:13:00.165826+0000","last_active":"2024-05-28T14:13:01.143174+0000","last_peered":"2024-05-28T14:13:01.143174+0000","last_clean":"2024-05-28T14:13:01.143174+0000","last_became_active":"2024-05-28T14:13:00.162459+0000","last_became_peered":"2024-05-28T14:13:00.162459+0000","last_unstale":"2024-05-28T14:13:01.143174+0000","last_undegraded":"2024-05-28T14:13:01.143174+0000","last_fullsized":"2024-05-28T14:13:01.143174+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:41:04.892264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143445+0000","last_change":"2024-05-28T14:13:00.146761+0000","last_active":"2024-05-28T14:13:01.143445+0000","last_peered":"2024-05-28T14:13:01.143445+0000","last_clean":"2024-05-28T14:13:01.143445+0000","last_became_active":"2024-05-28T14:13:00.146487+0000","last_became_peered":"2024-05-28T14:13:00.146487+0000","last_unstale":"2024-05-28T14:13:01.143445+0000","last_undegraded":"2024-05-28T14:13:01.143445+0000","last_fullsized":"2024-05-28T14:13:01.143445+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:16:35.564945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682331+0000","last_change":"2024-05-28T14:13:00.154658+0000","last_active":"2024-05-28T14:13:01.682331+0000","last_peered":"2024-05-28T14:13:01.682331+0000","last_clean":"2024-05-28T14:13:01.682331+0000","last_became_active":"2024-05-28T14:13:00.154417+0000","last_became_peered":"2024-05-28T14:13:00.154417+0000","last_unstale":"2024-05-28T14:13:01.682331+0000","last_undegraded":"2024-05-28T14:13:01.682331+0000","last_fullsized":"2024-05-28T14:13:01.682331+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:01:52.004242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140316+0000","last_change":"2024-05-28T14:13:00.165310+0000","last_active":"2024-05-28T14:13:01.140316+0000","last_peered":"2024-05-28T14:13:01.140316+0000","last_clean":"2024-05-28T14:13:01.140316+0000","last_became_active":"2024-05-28T14:13:00.164732+0000","last_became_peered":"2024-05-28T14:13:00.164732+0000","last_unstale":"2024-05-28T14:13:01.140316+0000","last_undegraded":"2024-05-28T14:13:01.140316+0000","last_fullsized":"2024-05-28T14:13:01.140316+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:24:20.189172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.141038+0000","last_change":"2024-05-28T14:13:00.163393+0000","last_active":"2024-05-28T14:13:01.141038+0000","last_peered":"2024-05-28T14:13:01.141038+0000","last_clean":"2024-05-28T14:13:01.141038+0000","last_became_active":"2024-05-28T14:13:00.163122+0000","last_became_peered":"2024-05-28T14:13:00.163122+0000","last_unstale":"2024-05-28T14:13:01.141038+0000","last_undegraded":"2024-05-28T14:13:01.141038+0000","last_fullsized":"2024-05-28T14:13:01.141038+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:23:44.667753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"116'4","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.144025+0000","last_change":"2024-05-28T14:12:59.156662+0000","last_active":"2024-05-28T14:13:01.144025+0000","last_peered":"2024-05-28T14:13:01.144025+0000","last_clean":"2024-05-28T14:13:01.144025+0000","last_became_active":"2024-05-28T14:12:59.156379+0000","last_became_peered":"2024-05-28T14:12:59.156379+0000","last_unstale":"2024-05-28T14:13:01.144025+0000","last_undegraded":"2024-05-28T14:13:01.144025+0000","last_fullsized":"2024-05-28T14:13:01.144025+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:29:06.215909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.215278+0000","last_change":"2024-05-28T14:13:00.165334+0000","last_active":"2024-05-28T14:13:01.215278+0000","last_peered":"2024-05-28T14:13:01.215278+0000","last_clean":"2024-05-28T14:13:01.215278+0000","last_became_active":"2024-05-28T14:13:00.165055+0000","last_became_peered":"2024-05-28T14:13:00.165055+0000","last_unstale":"2024-05-28T14:13:01.215278+0000","last_undegraded":"2024-05-28T14:13:01.215278+0000","last_fullsized":"2024-05-28T14:13:01.215278+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:46:22.522782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682807+0000","last_change":"2024-05-28T14:12:59.160363+0000","last_active":"2024-05-28T14:13:01.682807+0000","last_peered":"2024-05-28T14:13:01.682807+0000","last_clean":"2024-05-28T14:13:01.682807+0000","last_became_active":"2024-05-28T14:12:59.159749+0000","last_became_peered":"2024-05-28T14:12:59.159749+0000","last_unstale":"2024-05-28T14:13:01.682807+0000","last_undegraded":"2024-05-28T14:13:01.682807+0000","last_fullsized":"2024-05-28T14:13:01.682807+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:19:51.349013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140744+0000","last_change":"2024-05-28T14:13:00.168895+0000","last_active":"2024-05-28T14:13:01.140744+0000","last_peered":"2024-05-28T14:13:01.140744+0000","last_clean":"2024-05-28T14:13:01.140744+0000","last_became_active":"2024-05-28T14:13:00.168608+0000","last_became_peered":"2024-05-28T14:13:00.168608+0000","last_unstale":"2024-05-28T14:13:01.140744+0000","last_undegraded":"2024-05-28T14:13:01.140744+0000","last_fullsized":"2024-05-28T14:13:01.140744+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:29:07.224328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682004+0000","last_change":"2024-05-28T14:12:59.155476+0000","last_active":"2024-05-28T14:13:01.682004+0000","last_peered":"2024-05-28T14:13:01.682004+0000","last_clean":"2024-05-28T14:13:01.682004+0000","last_became_active":"2024-05-28T14:12:59.155187+0000","last_became_peered":"2024-05-28T14:12:59.155187+0000","last_unstale":"2024-05-28T14:13:01.682004+0000","last_undegraded":"2024-05-28T14:13:01.682004+0000","last_fullsized":"2024-05-28T14:13:01.682004+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:07:49.480989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683267+0000","last_change":"2024-05-28T14:13:00.162908+0000","last_active":"2024-05-28T14:13:01.683267+0000","last_peered":"2024-05-28T14:13:01.683267+0000","last_clean":"2024-05-28T14:13:01.683267+0000","last_became_active":"2024-05-28T14:13:00.162616+0000","last_became_peered":"2024-05-28T14:13:00.162616+0000","last_unstale":"2024-05-28T14:13:01.683267+0000","last_undegraded":"2024-05-28T14:13:01.683267+0000","last_fullsized":"2024-05-28T14:13:01.683267+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:00:01.461588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682005+0000","last_change":"2024-05-28T14:12:59.164490+0000","last_active":"2024-05-28T14:13:01.682005+0000","last_peered":"2024-05-28T14:13:01.682005+0000","last_clean":"2024-05-28T14:13:01.682005+0000","last_became_active":"2024-05-28T14:12:59.163870+0000","last_became_peered":"2024-05-28T14:12:59.163870+0000","last_unstale":"2024-05-28T14:13:01.682005+0000","last_undegraded":"2024-05-28T14:13:01.682005+0000","last_fullsized":"2024-05-28T14:13:01.682005+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:06.681422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.681910+0000","last_change":"2024-05-28T14:13:00.166324+0000","last_active":"2024-05-28T14:13:01.681910+0000","last_peered":"2024-05-28T14:13:01.681910+0000","last_clean":"2024-05-28T14:13:01.681910+0000","last_became_active":"2024-05-28T14:13:00.166054+0000","last_became_peered":"2024-05-28T14:13:00.166054+0000","last_unstale":"2024-05-28T14:13:01.681910+0000","last_undegraded":"2024-05-28T14:13:01.681910+0000","last_fullsized":"2024-05-28T14:13:01.681910+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:34:57.501530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143618+0000","last_change":"2024-05-28T14:12:59.157949+0000","last_active":"2024-05-28T14:13:01.143618+0000","last_peered":"2024-05-28T14:13:01.143618+0000","last_clean":"2024-05-28T14:13:01.143618+0000","last_became_active":"2024-05-28T14:12:59.157600+0000","last_became_peered":"2024-05-28T14:12:59.157600+0000","last_unstale":"2024-05-28T14:13:01.143618+0000","last_undegraded":"2024-05-28T14:13:01.143618+0000","last_fullsized":"2024-05-28T14:13:01.143618+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:24:19.180753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143624+0000","last_change":"2024-05-28T14:13:00.167579+0000","last_active":"2024-05-28T14:13:01.143624+0000","last_peered":"2024-05-28T14:13:01.143624+0000","last_clean":"2024-05-28T14:13:01.143624+0000","last_became_active":"2024-05-28T14:13:00.166167+0000","last_became_peered":"2024-05-28T14:13:00.166167+0000","last_unstale":"2024-05-28T14:13:01.143624+0000","last_undegraded":"2024-05-28T14:13:01.143624+0000","last_fullsized":"2024-05-28T14:13:01.143624+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:04:15.853596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683796+0000","last_change":"2024-05-28T14:12:59.158675+0000","last_active":"2024-05-28T14:13:01.683796+0000","last_peered":"2024-05-28T14:13:01.683796+0000","last_clean":"2024-05-28T14:13:01.683796+0000","last_became_active":"2024-05-28T14:12:59.158378+0000","last_became_peered":"2024-05-28T14:12:59.158378+0000","last_unstale":"2024-05-28T14:13:01.683796+0000","last_undegraded":"2024-05-28T14:13:01.683796+0000","last_fullsized":"2024-05-28T14:13:01.683796+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:07:17.818267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682021+0000","last_change":"2024-05-28T14:13:00.160893+0000","last_active":"2024-05-28T14:13:01.682021+0000","last_peered":"2024-05-28T14:13:01.682021+0000","last_clean":"2024-05-28T14:13:01.682021+0000","last_became_active":"2024-05-28T14:13:00.157458+0000","last_became_peered":"2024-05-28T14:13:00.157458+0000","last_unstale":"2024-05-28T14:13:01.682021+0000","last_undegraded":"2024-05-28T14:13:01.682021+0000","last_fullsized":"2024-05-28T14:13:01.682021+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:49:44.497244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.142805+0000","last_change":"2024-05-28T14:12:59.162720+0000","last_active":"2024-05-28T14:13:01.142805+0000","last_peered":"2024-05-28T14:13:01.142805+0000","last_clean":"2024-05-28T14:13:01.142805+0000","last_became_active":"2024-05-28T14:12:59.162364+0000","last_became_peered":"2024-05-28T14:12:59.162364+0000","last_unstale":"2024-05-28T14:13:01.142805+0000","last_undegraded":"2024-05-28T14:13:01.142805+0000","last_fullsized":"2024-05-28T14:13:01.142805+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:58:22.303545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143576+0000","last_change":"2024-05-28T14:13:00.169846+0000","last_active":"2024-05-28T14:13:01.143576+0000","last_peered":"2024-05-28T14:13:01.143576+0000","last_clean":"2024-05-28T14:13:01.143576+0000","last_became_active":"2024-05-28T14:13:00.169447+0000","last_became_peered":"2024-05-28T14:13:00.169447+0000","last_unstale":"2024-05-28T14:13:01.143576+0000","last_undegraded":"2024-05-28T14:13:01.143576+0000","last_fullsized":"2024-05-28T14:13:01.143576+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:51:49.229422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682218+0000","last_change":"2024-05-28T14:12:59.163864+0000","last_active":"2024-05-28T14:13:01.682218+0000","last_peered":"2024-05-28T14:13:01.682218+0000","last_clean":"2024-05-28T14:13:01.682218+0000","last_became_active":"2024-05-28T14:12:59.163528+0000","last_became_peered":"2024-05-28T14:12:59.163528+0000","last_unstale":"2024-05-28T14:13:01.682218+0000","last_undegraded":"2024-05-28T14:13:01.682218+0000","last_fullsized":"2024-05-28T14:13:01.682218+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:55.207466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.144100+0000","last_change":"2024-05-28T14:13:00.150217+0000","last_active":"2024-05-28T14:13:01.144100+0000","last_peered":"2024-05-28T14:13:01.144100+0000","last_clean":"2024-05-28T14:13:01.144100+0000","last_became_active":"2024-05-28T14:13:00.149915+0000","last_became_peered":"2024-05-28T14:13:00.149915+0000","last_unstale":"2024-05-28T14:13:01.144100+0000","last_undegraded":"2024-05-28T14:13:01.144100+0000","last_fullsized":"2024-05-28T14:13:01.144100+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:44:59.426282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682608+0000","last_change":"2024-05-28T14:12:59.165251+0000","last_active":"2024-05-28T14:13:01.682608+0000","last_peered":"2024-05-28T14:13:01.682608+0000","last_clean":"2024-05-28T14:13:01.682608+0000","last_became_active":"2024-05-28T14:12:59.164960+0000","last_became_peered":"2024-05-28T14:12:59.164960+0000","last_unstale":"2024-05-28T14:13:01.682608+0000","last_undegraded":"2024-05-28T14:13:01.682608+0000","last_fullsized":"2024-05-28T14:13:01.682608+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:47:24.425191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.215277+0000","last_change":"2024-05-28T14:13:00.172027+0000","last_active":"2024-05-28T14:13:01.215277+0000","last_peered":"2024-05-28T14:13:01.215277+0000","last_clean":"2024-05-28T14:13:01.215277+0000","last_became_active":"2024-05-28T14:13:00.171756+0000","last_became_peered":"2024-05-28T14:13:00.171756+0000","last_unstale":"2024-05-28T14:13:01.215277+0000","last_undegraded":"2024-05-28T14:13:01.215277+0000","last_fullsized":"2024-05-28T14:13:01.215277+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:52:25.954742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683751+0000","last_change":"2024-05-28T14:12:59.157811+0000","last_active":"2024-05-28T14:13:01.683751+0000","last_peered":"2024-05-28T14:13:01.683751+0000","last_clean":"2024-05-28T14:13:01.683751+0000","last_became_active":"2024-05-28T14:12:59.157503+0000","last_became_peered":"2024-05-28T14:12:59.157503+0000","last_unstale":"2024-05-28T14:13:01.683751+0000","last_undegraded":"2024-05-28T14:13:01.683751+0000","last_fullsized":"2024-05-28T14:13:01.683751+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:17:03.844751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683905+0000","last_change":"2024-05-28T14:13:00.166804+0000","last_active":"2024-05-28T14:13:01.683905+0000","last_peered":"2024-05-28T14:13:01.683905+0000","last_clean":"2024-05-28T14:13:01.683905+0000","last_became_active":"2024-05-28T14:13:00.166553+0000","last_became_peered":"2024-05-28T14:13:00.166553+0000","last_unstale":"2024-05-28T14:13:01.683905+0000","last_undegraded":"2024-05-28T14:13:01.683905+0000","last_fullsized":"2024-05-28T14:13:01.683905+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:27:44.085288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"116'3","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683231+0000","last_change":"2024-05-28T14:12:59.163447+0000","last_active":"2024-05-28T14:13:01.683231+0000","last_peered":"2024-05-28T14:13:01.683231+0000","last_clean":"2024-05-28T14:13:01.683231+0000","last_became_active":"2024-05-28T14:12:59.163160+0000","last_became_peered":"2024-05-28T14:12:59.163160+0000","last_unstale":"2024-05-28T14:13:01.683231+0000","last_undegraded":"2024-05-28T14:13:01.683231+0000","last_fullsized":"2024-05-28T14:13:01.683231+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:28:58.184414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143906+0000","last_change":"2024-05-28T14:13:00.156046+0000","last_active":"2024-05-28T14:13:01.143906+0000","last_peered":"2024-05-28T14:13:01.143906+0000","last_clean":"2024-05-28T14:13:01.143906+0000","last_became_active":"2024-05-28T14:13:00.155769+0000","last_became_peered":"2024-05-28T14:13:00.155769+0000","last_unstale":"2024-05-28T14:13:01.143906+0000","last_undegraded":"2024-05-28T14:13:01.143906+0000","last_fullsized":"2024-05-28T14:13:01.143906+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:57:34.669207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"116'6","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683184+0000","last_change":"2024-05-28T14:12:59.164229+0000","last_active":"2024-05-28T14:13:01.683184+0000","last_peered":"2024-05-28T14:13:01.683184+0000","last_clean":"2024-05-28T14:13:01.683184+0000","last_became_active":"2024-05-28T14:12:59.163931+0000","last_became_peered":"2024-05-28T14:12:59.163931+0000","last_unstale":"2024-05-28T14:13:01.683184+0000","last_undegraded":"2024-05-28T14:13:01.683184+0000","last_fullsized":"2024-05-28T14:13:01.683184+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:46:21.514362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683673+0000","last_change":"2024-05-28T14:13:00.144970+0000","last_active":"2024-05-28T14:13:01.683673+0000","last_peered":"2024-05-28T14:13:01.683673+0000","last_clean":"2024-05-28T14:13:01.683673+0000","last_became_active":"2024-05-28T14:13:00.144678+0000","last_became_peered":"2024-05-28T14:13:00.144678+0000","last_unstale":"2024-05-28T14:13:01.683673+0000","last_undegraded":"2024-05-28T14:13:01.683673+0000","last_fullsized":"2024-05-28T14:13:01.683673+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:43:55.342468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140582+0000","last_change":"2024-05-28T14:12:59.155868+0000","last_active":"2024-05-28T14:13:01.140582+0000","last_peered":"2024-05-28T14:13:01.140582+0000","last_clean":"2024-05-28T14:13:01.140582+0000","last_became_active":"2024-05-28T14:12:59.155584+0000","last_became_peered":"2024-05-28T14:12:59.155584+0000","last_unstale":"2024-05-28T14:13:01.140582+0000","last_undegraded":"2024-05-28T14:13:01.140582+0000","last_fullsized":"2024-05-28T14:13:01.140582+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:53:45.000843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683407+0000","last_change":"2024-05-28T14:13:00.160238+0000","last_active":"2024-05-28T14:13:01.683407+0000","last_peered":"2024-05-28T14:13:01.683407+0000","last_clean":"2024-05-28T14:13:01.683407+0000","last_became_active":"2024-05-28T14:13:00.159985+0000","last_became_peered":"2024-05-28T14:13:00.159985+0000","last_unstale":"2024-05-28T14:13:01.683407+0000","last_undegraded":"2024-05-28T14:13:01.683407+0000","last_fullsized":"2024-05-28T14:13:01.683407+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:27:35.279931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.214915+0000","last_change":"2024-05-28T14:12:59.161033+0000","last_active":"2024-05-28T14:13:01.214915+0000","last_peered":"2024-05-28T14:13:01.214915+0000","last_clean":"2024-05-28T14:13:01.214915+0000","last_became_active":"2024-05-28T14:12:59.160714+0000","last_became_peered":"2024-05-28T14:12:59.160714+0000","last_unstale":"2024-05-28T14:13:01.214915+0000","last_undegraded":"2024-05-28T14:13:01.214915+0000","last_fullsized":"2024-05-28T14:13:01.214915+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:34:57.372805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683087+0000","last_change":"2024-05-28T14:13:00.165320+0000","last_active":"2024-05-28T14:13:01.683087+0000","last_peered":"2024-05-28T14:13:01.683087+0000","last_clean":"2024-05-28T14:13:01.683087+0000","last_became_active":"2024-05-28T14:13:00.165062+0000","last_became_peered":"2024-05-28T14:13:00.165062+0000","last_unstale":"2024-05-28T14:13:01.683087+0000","last_undegraded":"2024-05-28T14:13:01.683087+0000","last_fullsized":"2024-05-28T14:13:01.683087+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:11:55.042880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"116'8","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.681707+0000","last_change":"2024-05-28T14:12:59.154364+0000","last_active":"2024-05-28T14:13:01.681707+0000","last_peered":"2024-05-28T14:13:01.681707+0000","last_clean":"2024-05-28T14:13:01.681707+0000","last_became_active":"2024-05-28T14:12:59.152108+0000","last_became_peered":"2024-05-28T14:12:59.152108+0000","last_unstale":"2024-05-28T14:13:01.681707+0000","last_undegraded":"2024-05-28T14:13:01.681707+0000","last_fullsized":"2024-05-28T14:13:01.681707+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:23.615288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682319+0000","last_change":"2024-05-28T14:13:00.155655+0000","last_active":"2024-05-28T14:13:01.682319+0000","last_peered":"2024-05-28T14:13:01.682319+0000","last_clean":"2024-05-28T14:13:01.682319+0000","last_became_active":"2024-05-28T14:13:00.155230+0000","last_became_peered":"2024-05-28T14:13:00.155230+0000","last_unstale":"2024-05-28T14:13:01.682319+0000","last_undegraded":"2024-05-28T14:13:01.682319+0000","last_fullsized":"2024-05-28T14:13:01.682319+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:54.956610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683686+0000","last_change":"2024-05-28T14:12:59.158148+0000","last_active":"2024-05-28T14:13:01.683686+0000","last_peered":"2024-05-28T14:13:01.683686+0000","last_clean":"2024-05-28T14:13:01.683686+0000","last_became_active":"2024-05-28T14:12:59.156240+0000","last_became_peered":"2024-05-28T14:12:59.156240+0000","last_unstale":"2024-05-28T14:13:01.683686+0000","last_undegraded":"2024-05-28T14:13:01.683686+0000","last_fullsized":"2024-05-28T14:13:01.683686+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:04:14.845177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.141011+0000","last_change":"2024-05-28T14:13:00.167593+0000","last_active":"2024-05-28T14:13:01.141011+0000","last_peered":"2024-05-28T14:13:01.141011+0000","last_clean":"2024-05-28T14:13:01.141011+0000","last_became_active":"2024-05-28T14:13:00.167330+0000","last_became_peered":"2024-05-28T14:13:00.167330+0000","last_unstale":"2024-05-28T14:13:01.141011+0000","last_undegraded":"2024-05-28T14:13:01.141011+0000","last_fullsized":"2024-05-28T14:13:01.141011+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:54.956610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682285+0000","last_change":"2024-05-28T14:12:59.154024+0000","last_active":"2024-05-28T14:13:01.682285+0000","last_peered":"2024-05-28T14:13:01.682285+0000","last_clean":"2024-05-28T14:13:01.682285+0000","last_became_active":"2024-05-28T14:12:59.153637+0000","last_became_peered":"2024-05-28T14:12:59.153637+0000","last_unstale":"2024-05-28T14:13:01.682285+0000","last_undegraded":"2024-05-28T14:13:01.682285+0000","last_fullsized":"2024-05-28T14:13:01.682285+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:24:19.180753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682885+0000","last_change":"2024-05-28T14:13:00.153780+0000","last_active":"2024-05-28T14:13:01.682885+0000","last_peered":"2024-05-28T14:13:01.682885+0000","last_clean":"2024-05-28T14:13:01.682885+0000","last_became_active":"2024-05-28T14:13:00.153504+0000","last_became_peered":"2024-05-28T14:13:00.153504+0000","last_unstale":"2024-05-28T14:13:01.682885+0000","last_undegraded":"2024-05-28T14:13:01.682885+0000","last_fullsized":"2024-05-28T14:13:01.682885+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:24.726727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.215015+0000","last_change":"2024-05-28T14:12:59.161950+0000","last_active":"2024-05-28T14:13:01.215015+0000","last_peered":"2024-05-28T14:13:01.215015+0000","last_clean":"2024-05-28T14:13:01.215015+0000","last_became_active":"2024-05-28T14:12:59.161536+0000","last_became_peered":"2024-05-28T14:12:59.161536+0000","last_unstale":"2024-05-28T14:13:01.215015+0000","last_undegraded":"2024-05-28T14:13:01.215015+0000","last_fullsized":"2024-05-28T14:13:01.215015+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:09:38.636477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683289+0000","last_change":"2024-05-28T14:13:00.157344+0000","last_active":"2024-05-28T14:13:01.683289+0000","last_peered":"2024-05-28T14:13:01.683289+0000","last_clean":"2024-05-28T14:13:01.683289+0000","last_became_active":"2024-05-28T14:13:00.157069+0000","last_became_peered":"2024-05-28T14:13:00.157069+0000","last_unstale":"2024-05-28T14:13:01.683289+0000","last_undegraded":"2024-05-28T14:13:01.683289+0000","last_fullsized":"2024-05-28T14:13:01.683289+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:25:27.542376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143656+0000","last_change":"2024-05-28T14:12:59.155568+0000","last_active":"2024-05-28T14:13:01.143656+0000","last_peered":"2024-05-28T14:13:01.143656+0000","last_clean":"2024-05-28T14:13:01.143656+0000","last_became_active":"2024-05-28T14:12:59.154202+0000","last_became_peered":"2024-05-28T14:12:59.154202+0000","last_unstale":"2024-05-28T14:13:01.143656+0000","last_undegraded":"2024-05-28T14:13:01.143656+0000","last_fullsized":"2024-05-28T14:13:01.143656+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:06.681422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682756+0000","last_change":"2024-05-28T14:13:00.164974+0000","last_active":"2024-05-28T14:13:01.682756+0000","last_peered":"2024-05-28T14:13:01.682756+0000","last_clean":"2024-05-28T14:13:01.682756+0000","last_became_active":"2024-05-28T14:13:00.164697+0000","last_became_peered":"2024-05-28T14:13:00.164697+0000","last_unstale":"2024-05-28T14:13:01.682756+0000","last_undegraded":"2024-05-28T14:13:01.682756+0000","last_fullsized":"2024-05-28T14:13:01.682756+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:01:52.004242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140683+0000","last_change":"2024-05-28T14:12:59.159676+0000","last_active":"2024-05-28T14:13:01.140683+0000","last_peered":"2024-05-28T14:13:01.140683+0000","last_clean":"2024-05-28T14:13:01.140683+0000","last_became_active":"2024-05-28T14:12:59.159391+0000","last_became_peered":"2024-05-28T14:12:59.159391+0000","last_unstale":"2024-05-28T14:13:01.140683+0000","last_undegraded":"2024-05-28T14:13:01.140683+0000","last_fullsized":"2024-05-28T14:13:01.140683+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:53:37.358277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682709+0000","last_change":"2024-05-28T14:13:00.158035+0000","last_active":"2024-05-28T14:13:01.682709+0000","last_peered":"2024-05-28T14:13:01.682709+0000","last_clean":"2024-05-28T14:13:01.682709+0000","last_became_active":"2024-05-28T14:13:00.157761+0000","last_became_peered":"2024-05-28T14:13:00.157761+0000","last_unstale":"2024-05-28T14:13:01.682709+0000","last_undegraded":"2024-05-28T14:13:01.682709+0000","last_fullsized":"2024-05-28T14:13:01.682709+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:24.726727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.681654+0000","last_change":"2024-05-28T14:12:59.153857+0000","last_active":"2024-05-28T14:13:01.681654+0000","last_peered":"2024-05-28T14:13:01.681654+0000","last_clean":"2024-05-28T14:13:01.681654+0000","last_became_active":"2024-05-28T14:12:59.153528+0000","last_became_peered":"2024-05-28T14:12:59.153528+0000","last_unstale":"2024-05-28T14:13:01.681654+0000","last_undegraded":"2024-05-28T14:13:01.681654+0000","last_fullsized":"2024-05-28T14:13:01.681654+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:46:44.183872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682718+0000","last_change":"2024-05-28T14:13:00.164800+0000","last_active":"2024-05-28T14:13:01.682718+0000","last_peered":"2024-05-28T14:13:01.682718+0000","last_clean":"2024-05-28T14:13:01.682718+0000","last_became_active":"2024-05-28T14:13:00.164540+0000","last_became_peered":"2024-05-28T14:13:00.164540+0000","last_unstale":"2024-05-28T14:13:01.682718+0000","last_undegraded":"2024-05-28T14:13:01.682718+0000","last_fullsized":"2024-05-28T14:13:01.682718+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:37:35.996044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.140333+0000","last_change":"2024-05-28T14:12:59.153891+0000","last_active":"2024-05-28T14:13:01.140333+0000","last_peered":"2024-05-28T14:13:01.140333+0000","last_clean":"2024-05-28T14:13:01.140333+0000","last_became_active":"2024-05-28T14:12:59.153629+0000","last_became_peered":"2024-05-28T14:12:59.153629+0000","last_unstale":"2024-05-28T14:13:01.140333+0000","last_undegraded":"2024-05-28T14:13:01.140333+0000","last_fullsized":"2024-05-28T14:13:01.140333+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:38:02.389035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683220+0000","last_change":"2024-05-28T14:13:00.151659+0000","last_active":"2024-05-28T14:13:01.683220+0000","last_peered":"2024-05-28T14:13:01.683220+0000","last_clean":"2024-05-28T14:13:01.683220+0000","last_became_active":"2024-05-28T14:13:00.148987+0000","last_became_peered":"2024-05-28T14:13:00.148987+0000","last_unstale":"2024-05-28T14:13:01.683220+0000","last_undegraded":"2024-05-28T14:13:01.683220+0000","last_fullsized":"2024-05-28T14:13:01.683220+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:58:42.460180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682311+0000","last_change":"2024-05-28T14:12:59.159929+0000","last_active":"2024-05-28T14:13:01.682311+0000","last_peered":"2024-05-28T14:13:01.682311+0000","last_clean":"2024-05-28T14:13:01.682311+0000","last_became_active":"2024-05-28T14:12:59.159646+0000","last_became_peered":"2024-05-28T14:12:59.159646+0000","last_unstale":"2024-05-28T14:13:01.682311+0000","last_undegraded":"2024-05-28T14:13:01.682311+0000","last_fullsized":"2024-05-28T14:13:01.682311+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:48:06.747542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.215106+0000","last_change":"2024-05-28T14:13:00.173025+0000","last_active":"2024-05-28T14:13:01.215106+0000","last_peered":"2024-05-28T14:13:01.215106+0000","last_clean":"2024-05-28T14:13:01.215106+0000","last_became_active":"2024-05-28T14:13:00.172597+0000","last_became_peered":"2024-05-28T14:13:00.172597+0000","last_unstale":"2024-05-28T14:13:01.215106+0000","last_undegraded":"2024-05-28T14:13:01.215106+0000","last_fullsized":"2024-05-28T14:13:01.215106+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:48:07.755962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683951+0000","last_change":"2024-05-28T14:12:59.161594+0000","last_active":"2024-05-28T14:13:01.683951+0000","last_peered":"2024-05-28T14:13:01.683951+0000","last_clean":"2024-05-28T14:13:01.683951+0000","last_became_active":"2024-05-28T14:12:59.161326+0000","last_became_peered":"2024-05-28T14:12:59.161326+0000","last_unstale":"2024-05-28T14:13:01.683951+0000","last_undegraded":"2024-05-28T14:13:01.683951+0000","last_fullsized":"2024-05-28T14:13:01.683951+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:28:58.184414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683682+0000","last_change":"2024-05-28T14:13:00.173835+0000","last_active":"2024-05-28T14:13:01.683682+0000","last_peered":"2024-05-28T14:13:01.683682+0000","last_clean":"2024-05-28T14:13:01.683682+0000","last_became_active":"2024-05-28T14:13:00.173372+0000","last_became_peered":"2024-05-28T14:13:00.173372+0000","last_unstale":"2024-05-28T14:13:01.683682+0000","last_undegraded":"2024-05-28T14:13:01.683682+0000","last_fullsized":"2024-05-28T14:13:01.683682+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:42:30.904711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"117'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.215211+0000","last_change":"2024-05-28T14:12:59.159230+0000","last_active":"2024-05-28T14:13:01.215211+0000","last_peered":"2024-05-28T14:13:01.215211+0000","last_clean":"2024-05-28T14:13:01.215211+0000","last_became_active":"2024-05-28T14:12:59.158844+0000","last_became_peered":"2024-05-28T14:12:59.158844+0000","last_unstale":"2024-05-28T14:13:01.215211+0000","last_undegraded":"2024-05-28T14:13:01.215211+0000","last_fullsized":"2024-05-28T14:13:01.215211+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:50:25.519025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683083+0000","last_change":"2024-05-28T14:13:00.155961+0000","last_active":"2024-05-28T14:13:01.683083+0000","last_peered":"2024-05-28T14:13:01.683083+0000","last_clean":"2024-05-28T14:13:01.683083+0000","last_became_active":"2024-05-28T14:13:00.155684+0000","last_became_peered":"2024-05-28T14:13:00.155684+0000","last_unstale":"2024-05-28T14:13:01.683083+0000","last_undegraded":"2024-05-28T14:13:01.683083+0000","last_fullsized":"2024-05-28T14:13:01.683083+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:31:55.696017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683326+0000","last_change":"2024-05-28T14:12:59.159785+0000","last_active":"2024-05-28T14:13:01.683326+0000","last_peered":"2024-05-28T14:13:01.683326+0000","last_clean":"2024-05-28T14:13:01.683326+0000","last_became_active":"2024-05-28T14:12:59.159495+0000","last_became_peered":"2024-05-28T14:12:59.159495+0000","last_unstale":"2024-05-28T14:13:01.683326+0000","last_undegraded":"2024-05-28T14:13:01.683326+0000","last_fullsized":"2024-05-28T14:13:01.683326+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:41:03.883845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.214942+0000","last_change":"2024-05-28T14:13:00.172970+0000","last_active":"2024-05-28T14:13:01.214942+0000","last_peered":"2024-05-28T14:13:01.214942+0000","last_clean":"2024-05-28T14:13:01.214942+0000","last_became_active":"2024-05-28T14:13:00.172680+0000","last_became_peered":"2024-05-28T14:13:00.172680+0000","last_unstale":"2024-05-28T14:13:01.214942+0000","last_undegraded":"2024-05-28T14:13:01.214942+0000","last_fullsized":"2024-05-28T14:13:01.214942+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:41:04.892264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683864+0000","last_change":"2024-05-28T14:12:59.160575+0000","last_active":"2024-05-28T14:13:01.683864+0000","last_peered":"2024-05-28T14:13:01.683864+0000","last_clean":"2024-05-28T14:13:01.683864+0000","last_became_active":"2024-05-28T14:12:59.160277+0000","last_became_peered":"2024-05-28T14:12:59.160277+0000","last_unstale":"2024-05-28T14:13:01.683864+0000","last_undegraded":"2024-05-28T14:13:01.683864+0000","last_fullsized":"2024-05-28T14:13:01.683864+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:41:03.883845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143241+0000","last_change":"2024-05-28T14:13:00.167453+0000","last_active":"2024-05-28T14:13:01.143241+0000","last_peered":"2024-05-28T14:13:01.143241+0000","last_clean":"2024-05-28T14:13:01.143241+0000","last_became_active":"2024-05-28T14:13:00.167176+0000","last_became_peered":"2024-05-28T14:13:00.167176+0000","last_unstale":"2024-05-28T14:13:01.143241+0000","last_undegraded":"2024-05-28T14:13:01.143241+0000","last_fullsized":"2024-05-28T14:13:01.143241+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:56.215885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143756+0000","last_change":"2024-05-28T14:12:59.155180+0000","last_active":"2024-05-28T14:13:01.143756+0000","last_peered":"2024-05-28T14:13:01.143756+0000","last_clean":"2024-05-28T14:13:01.143756+0000","last_became_active":"2024-05-28T14:12:59.154912+0000","last_became_peered":"2024-05-28T14:12:59.154912+0000","last_unstale":"2024-05-28T14:13:01.143756+0000","last_undegraded":"2024-05-28T14:13:01.143756+0000","last_fullsized":"2024-05-28T14:13:01.143756+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:47:24.425191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143654+0000","last_change":"2024-05-28T14:13:00.155179+0000","last_active":"2024-05-28T14:13:01.143654+0000","last_peered":"2024-05-28T14:13:01.143654+0000","last_clean":"2024-05-28T14:13:01.143654+0000","last_became_active":"2024-05-28T14:13:00.154781+0000","last_became_peered":"2024-05-28T14:13:00.154781+0000","last_unstale":"2024-05-28T14:13:01.143654+0000","last_undegraded":"2024-05-28T14:13:01.143654+0000","last_fullsized":"2024-05-28T14:13:01.143654+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:02:35.428286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.143226+0000","last_change":"2024-05-28T14:12:59.156463+0000","last_active":"2024-05-28T14:13:01.143226+0000","last_peered":"2024-05-28T14:13:01.143226+0000","last_clean":"2024-05-28T14:13:01.143226+0000","last_became_active":"2024-05-28T14:12:59.155528+0000","last_became_peered":"2024-05-28T14:12:59.155528+0000","last_unstale":"2024-05-28T14:13:01.143226+0000","last_undegraded":"2024-05-28T14:13:01.143226+0000","last_fullsized":"2024-05-28T14:13:01.143226+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:38:27.527964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682877+0000","last_change":"2024-05-28T14:13:00.173560+0000","last_active":"2024-05-28T14:13:01.682877+0000","last_peered":"2024-05-28T14:13:01.682877+0000","last_clean":"2024-05-28T14:13:01.682877+0000","last_became_active":"2024-05-28T14:13:00.173281+0000","last_became_peered":"2024-05-28T14:13:00.173281+0000","last_unstale":"2024-05-28T14:13:01.682877+0000","last_undegraded":"2024-05-28T14:13:01.682877+0000","last_fullsized":"2024-05-28T14:13:01.682877+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:51:22.209282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.681699+0000","last_change":"2024-05-28T14:12:59.154458+0000","last_active":"2024-05-28T14:13:01.681699+0000","last_peered":"2024-05-28T14:13:01.681699+0000","last_clean":"2024-05-28T14:13:01.681699+0000","last_became_active":"2024-05-28T14:12:59.153652+0000","last_became_peered":"2024-05-28T14:12:59.153652+0000","last_unstale":"2024-05-28T14:13:01.681699+0000","last_undegraded":"2024-05-28T14:13:01.681699+0000","last_fullsized":"2024-05-28T14:13:01.681699+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:39:27.979374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682492+0000","last_change":"2024-05-28T14:13:00.164867+0000","last_active":"2024-05-28T14:13:01.682492+0000","last_peered":"2024-05-28T14:13:01.682492+0000","last_clean":"2024-05-28T14:13:01.682492+0000","last_became_active":"2024-05-28T14:13:00.164613+0000","last_became_peered":"2024-05-28T14:13:00.164613+0000","last_unstale":"2024-05-28T14:13:01.682492+0000","last_undegraded":"2024-05-28T14:13:01.682492+0000","last_fullsized":"2024-05-28T14:13:01.682492+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:37:35.996044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.682406+0000","last_change":"2024-05-28T14:12:59.158941+0000","last_active":"2024-05-28T14:13:01.682406+0000","last_peered":"2024-05-28T14:13:01.682406+0000","last_clean":"2024-05-28T14:13:01.682406+0000","last_became_active":"2024-05-28T14:12:59.158680+0000","last_became_peered":"2024-05-28T14:12:59.158680+0000","last_unstale":"2024-05-28T14:13:01.682406+0000","last_undegraded":"2024-05-28T14:13:01.682406+0000","last_fullsized":"2024-05-28T14:13:01.682406+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:52:24.946323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683030+0000","last_change":"2024-05-28T14:13:00.157032+0000","last_active":"2024-05-28T14:13:01.683030+0000","last_peered":"2024-05-28T14:13:01.683030+0000","last_clean":"2024-05-28T14:13:01.683030+0000","last_became_active":"2024-05-28T14:13:00.156051+0000","last_became_peered":"2024-05-28T14:13:00.156051+0000","last_unstale":"2024-05-28T14:13:01.683030+0000","last_undegraded":"2024-05-28T14:13:01.683030+0000","last_fullsized":"2024-05-28T14:13:01.683030+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:59.126694+0000","last_clean_scrub_stamp":"2024-05-28T14:12:59.126694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:42:59.737418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.142947+0000","last_change":"2024-05-28T14:12:59.154352+0000","last_active":"2024-05-28T14:13:01.142947+0000","last_peered":"2024-05-28T14:13:01.142947+0000","last_clean":"2024-05-28T14:13:01.142947+0000","last_became_active":"2024-05-28T14:12:59.154075+0000","last_became_peered":"2024-05-28T14:12:59.154075+0000","last_unstale":"2024-05-28T14:13:01.142947+0000","last_undegraded":"2024-05-28T14:13:01.142947+0000","last_fullsized":"2024-05-28T14:13:01.142947+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:12:58.118274+0000","last_clean_scrub_stamp":"2024-05-28T14:12:58.118274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:07:49.480989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":351,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-28T14:13:01.683050+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:13:01.683050+0000","last_peered":"2024-05-28T14:13:01.683050+0000","last_clean":"2024-05-28T14:13:01.683050+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:13:01.683050+0000","last_undegraded":"2024-05-28T14:13:01.683050+0000","last_fullsized":"2024-05-28T14:13:01.683050+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575029,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131684,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240156,"statfs":{"total":96636764160,"available":96501919744,"internally_reserved":0,"allocated":2056192,"data_stored":1227728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575029,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131100,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240740,"statfs":{"total":96636764160,"available":96502517760,"internally_reserved":0,"allocated":1458176,"data_stored":636814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575031,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64480,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307360,"statfs":{"total":96636764160,"available":96570736640,"internally_reserved":0,"allocated":1462272,"data_stored":637348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575030,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144984,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226856,"statfs":{"total":96636764160,"available":96488300544,"internally_reserved":0,"allocated":1454080,"data_stored":636744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575029,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122792,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249048,"statfs":{"total":96636764160,"available":96511025152,"internally_reserved":0,"allocated":1470464,"data_stored":638426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":124254664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575030,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136672,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235168,"statfs":{"total":96636764160,"available":96496812032,"internally_reserved":0,"allocated":1454080,"data_stored":636744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575029,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149160,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222680,"statfs":{"total":96636764160,"available":96484024320,"internally_reserved":0,"allocated":1462272,"data_stored":636868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575030,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91440,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280400,"statfs":{"total":96636764160,"available":96543129600,"internally_reserved":0,"allocated":2068480,"data_stored":1228836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:13:09.211 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-28T14:13:09.212 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-28T14:13:09.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.411+0000 7f4cb2c00640 1 Processor -- start 2024-05-28T14:13:09.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.411+0000 7f4cb2c00640 1 -- start start 2024-05-28T14:13:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.415+0000 7f4cb2c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4cac071de0 0x7f4cac0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.415+0000 7f4cb2c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4cac0ce3b0 0x7f4cac0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.415+0000 7f4cb2c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cac072700 con 0x7f4cac141d10 2024-05-28T14:13:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.415+0000 7f4cb2c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f4cac160230 con 0x7f4cac071de0 2024-05-28T14:13:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.415+0000 7f4cb2c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4cac160510 con 0x7f4cac0ce3b0 2024-05-28T14:13:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.415+0000 7f4cabe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4cac071de0 0x7f4cac0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.415+0000 7f4cabe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4cac071de0 0x7f4cac0721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39484/0 (socket says 172.21.15.42:39484) 2024-05-28T14:13:09.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.415+0000 7f4cabe00640 1 -- 172.21.15.42:0/833161028 learned_addr learned my addr 172.21.15.42:0/833161028 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:09.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.415+0000 7f4ca3e00640 1 --2- 172.21.15.42:0/833161028 >> v2:172.21.15.63:3300/0 conn(0x7f4cac0ce3b0 0x7f4cac0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.415+0000 7f4cabe00640 1 -- 172.21.15.42:0/833161028 >> v2:172.21.15.63:3300/0 conn(0x7f4cac0ce3b0 msgr2=0x7f4cac0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:09.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.415+0000 7f4cabe00640 1 --2- 172.21.15.42:0/833161028 >> v2:172.21.15.63:3300/0 conn(0x7f4cac0ce3b0 0x7f4cac0ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.415+0000 7f4cabe00640 1 -- 172.21.15.42:0/833161028 >> v1:172.21.15.63:6789/0 conn(0x7f4cac141d10 legacy=0x7f4cac142160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.415+0000 7f4cabe00640 1 -- 172.21.15.42:0/833161028 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac1607f0 con 0x7f4cac071de0 2024-05-28T14:13:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cabe00640 1 --2- 172.21.15.42:0/833161028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cac071de0 0x7f4cac0721c0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f4cac073280 tx=0x7f4c9c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4b0d56eb860fdb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:09.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cab400640 1 -- 172.21.15.42:0/833161028 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c0b0070 con 0x7f4cac071de0 2024-05-28T14:13:09.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cab400640 1 -- 172.21.15.42:0/833161028 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c9c0ab600 con 0x7f4cac071de0 2024-05-28T14:13:09.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cab400640 1 -- 172.21.15.42:0/833161028 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c0ab9d0 con 0x7f4cac071de0 2024-05-28T14:13:09.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cb2c00640 1 -- 172.21.15.42:0/833161028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cac071de0 msgr2=0x7f4cac0721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cb2c00640 1 --2- 172.21.15.42:0/833161028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cac071de0 0x7f4cac0721c0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f4cac073280 tx=0x7f4c9c01c400 comp rx=0 tx=0).stop 2024-05-28T14:13:09.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cb2c00640 1 -- 172.21.15.42:0/833161028 shutdown_connections 2024-05-28T14:13:09.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cb2c00640 1 --2- 172.21.15.42:0/833161028 >> v2:172.21.15.63:3300/0 conn(0x7f4cac0ce3b0 0x7f4cac0ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cb2c00640 1 --2- 172.21.15.42:0/833161028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cac071de0 0x7f4cac0721c0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cb2c00640 1 -- 172.21.15.42:0/833161028 >> 172.21.15.42:0/833161028 conn(0x7f4cac0b21f0 msgr2=0x7f4cac0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:09.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cb2c00640 1 -- 172.21.15.42:0/833161028 shutdown_connections 2024-05-28T14:13:09.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cb2c00640 1 -- 172.21.15.42:0/833161028 wait complete. 2024-05-28T14:13:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cb2c00640 1 Processor -- start 2024-05-28T14:13:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cb2c00640 1 -- start start 2024-05-28T14:13:09.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cb2c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cac071de0 0x7f4cac13b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.419+0000 7f4cb2c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cac0ce3b0 0x7f4cac1fbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cb2c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cac141d10 0x7f4cac144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cb2c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac169e30 con 0x7f4cac071de0 2024-05-28T14:13:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cb2c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac169cb0 con 0x7f4cac141d10 2024-05-28T14:13:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cb2c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4cac169fb0 con 0x7f4cac0ce3b0 2024-05-28T14:13:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cabe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cac071de0 0x7f4cac13b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cabe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cac071de0 0x7f4cac13b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39486/0 (socket says 172.21.15.42:39486) 2024-05-28T14:13:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cabe00640 1 -- 172.21.15.42:0/1228153742 learned_addr learned my addr 172.21.15.42:0/1228153742 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cb0c00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cac141d10 0x7f4cac144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4ca3e00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cac0ce3b0 0x7f4cac1fbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cabe00640 1 -- 172.21.15.42:0/1228153742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cac0ce3b0 msgr2=0x7f4cac1fbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cabe00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cac0ce3b0 0x7f4cac1fbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cabe00640 1 -- 172.21.15.42:0/1228153742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cac141d10 msgr2=0x7f4cac144d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cabe00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cac141d10 0x7f4cac144d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cabe00640 1 -- 172.21.15.42:0/1228153742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac1fc140 con 0x7f4cac071de0 2024-05-28T14:13:09.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cb0c00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cac141d10 0x7f4cac144d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:09.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4ca3e00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cac0ce3b0 0x7f4cac1fbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:09.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cabe00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cac071de0 0x7f4cac13b880 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0ae040 tx=0x7f4c9c01ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:09.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4ca8c00640 1 -- 172.21.15.42:0/1228153742 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c0ba020 con 0x7f4cac071de0 2024-05-28T14:13:09.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4ca8c00640 1 -- 172.21.15.42:0/1228153742 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c9c0b6420 con 0x7f4cac071de0 2024-05-28T14:13:09.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cb2c00640 1 -- 172.21.15.42:0/1228153742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cac15de70 con 0x7f4cac071de0 2024-05-28T14:13:09.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4ca8c00640 1 -- 172.21.15.42:0/1228153742 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c0b0040 con 0x7f4cac071de0 2024-05-28T14:13:09.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cb2c00640 1 -- 172.21.15.42:0/1228153742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cac15e430 con 0x7f4cac071de0 2024-05-28T14:13:09.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4cb2c00640 1 -- 172.21.15.42:0/1228153742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cac0738b0 con 0x7f4cac071de0 2024-05-28T14:13:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.423+0000 7f4ca8c00640 1 -- 172.21.15.42:0/1228153742 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f4c9c0bd020 con 0x7f4cac071de0 2024-05-28T14:13:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.427+0000 7f4ca8c00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4c88069b50 0x7f4c8806c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.427+0000 7f4ca8c00640 1 -- 172.21.15.42:0/1228153742 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f4c9c11f8c0 con 0x7f4cac071de0 2024-05-28T14:13:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.427+0000 7f4ca3e00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4c88069b50 0x7f4c8806c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.427+0000 7f4ca3e00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4c88069b50 0x7f4c8806c010 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4cac156690 tx=0x7f4c94008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:09.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.431+0000 7f4ca8c00640 1 -- 172.21.15.42:0/1228153742 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4c9c0c4030 con 0x7f4cac071de0 2024-05-28T14:13:09.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.655+0000 7f4cb2c00640 1 -- 172.21.15.42:0/1228153742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4cac04fdd0 con 0x7f4cac071de0 2024-05-28T14:13:09.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.659+0000 7f4ca8c00640 1 -- 172.21.15.42:0/1228153742 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7f4cac1fc140 con 0x7f4cac071de0 2024-05-28T14:13:09.645 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-05-28T14:13:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw== 2024-05-28T14:13:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow" 2024-05-28T14:13:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-05-28T14:13:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-28T14:13:09.645 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.0 2024-05-28T14:13:09.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.663+0000 7f4cb2c00640 1 -- 172.21.15.42:0/1228153742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4c88069b50 msgr2=0x7f4c8806c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.663+0000 7f4cb2c00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4c88069b50 0x7f4c8806c010 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4cac156690 tx=0x7f4c94008040 comp rx=0 tx=0).stop 2024-05-28T14:13:09.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.663+0000 7f4cb2c00640 1 -- 172.21.15.42:0/1228153742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cac071de0 msgr2=0x7f4cac13b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.663+0000 7f4cb2c00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cac071de0 0x7f4cac13b880 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0ae040 tx=0x7f4c9c01ac40 comp rx=0 tx=0).stop 2024-05-28T14:13:09.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.663+0000 7f4cb2c00640 1 -- 172.21.15.42:0/1228153742 shutdown_connections 2024-05-28T14:13:09.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.663+0000 7f4cb2c00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4c88069b50 0x7f4c8806c010 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.663+0000 7f4cb2c00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cac141d10 0x7f4cac144d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.663+0000 7f4cb2c00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cac0ce3b0 0x7f4cac1fbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.663+0000 7f4cb2c00640 1 --2- 172.21.15.42:0/1228153742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4cac071de0 0x7f4cac13b880 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.663+0000 7f4cb2c00640 1 -- 172.21.15.42:0/1228153742 >> 172.21.15.42:0/1228153742 conn(0x7f4cac0b21f0 msgr2=0x7f4cac0b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:09.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.663+0000 7f4cb2c00640 1 -- 172.21.15.42:0/1228153742 shutdown_connections 2024-05-28T14:13:09.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.663+0000 7f4cb2c00640 1 -- 172.21.15.42:0/1228153742 wait complete. 2024-05-28T14:13:09.669 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:13:09.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.823+0000 7f92bd800640 1 Processor -- start 2024-05-28T14:13:09.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.823+0000 7f92bd800640 1 -- start start 2024-05-28T14:13:09.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.827+0000 7f92bd800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f92b806f660 0x7f92b8073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.827+0000 7f92bd800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92b80d0e00 0x7f92b806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.827+0000 7f92bd800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b815f280 con 0x7f92b815ba40 2024-05-28T14:13:09.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.827+0000 7f92bd800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f92b815f030 con 0x7f92b80d0e00 2024-05-28T14:13:09.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.827+0000 7f92bd800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f92b815f560 con 0x7f92b806f660 2024-05-28T14:13:09.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.827+0000 7f92b7400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92b80d0e00 0x7f92b806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.827+0000 7f92b7400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92b80d0e00 0x7f92b806dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39502/0 (socket says 172.21.15.42:39502) 2024-05-28T14:13:09.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.827+0000 7f92b7400640 1 -- 172.21.15.42:0/1263936213 learned_addr learned my addr 172.21.15.42:0/1263936213 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:09.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.827+0000 7f92b6a00640 1 --2- 172.21.15.42:0/1263936213 >> v2:172.21.15.63:3300/0 conn(0x7f92b806f660 0x7f92b8073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.827+0000 7f92b7400640 1 -- 172.21.15.42:0/1263936213 >> v2:172.21.15.63:3300/0 conn(0x7f92b806f660 msgr2=0x7f92b8073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:09.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.827+0000 7f92b7400640 1 --2- 172.21.15.42:0/1263936213 >> v2:172.21.15.63:3300/0 conn(0x7f92b806f660 0x7f92b8073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.827+0000 7f92b7400640 1 -- 172.21.15.42:0/1263936213 >> v1:172.21.15.63:6789/0 conn(0x7f92b815ba40 legacy=0x7f92b80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.827+0000 7f92b7400640 1 -- 172.21.15.42:0/1263936213 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b815f840 con 0x7f92b80d0e00 2024-05-28T14:13:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.831+0000 7f92b7400640 1 --2- 172.21.15.42:0/1263936213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92b80d0e00 0x7f92b806dd60 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f92b80d13a0 tx=0x7f92b001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2d41b4c1b5b276 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.831+0000 7f92b6000640 1 -- 172.21.15.42:0/1263936213 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b00a5740 con 0x7f92b80d0e00 2024-05-28T14:13:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.831+0000 7f92b6000640 1 -- 172.21.15.42:0/1263936213 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b00a58e0 con 0x7f92b80d0e00 2024-05-28T14:13:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.831+0000 7f92b6000640 1 -- 172.21.15.42:0/1263936213 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b00a5cb0 con 0x7f92b80d0e00 2024-05-28T14:13:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.831+0000 7f92bd800640 1 -- 172.21.15.42:0/1263936213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92b80d0e00 msgr2=0x7f92b806dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.831+0000 7f92bd800640 1 --2- 172.21.15.42:0/1263936213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92b80d0e00 0x7f92b806dd60 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f92b80d13a0 tx=0x7f92b001c8a0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.831+0000 7f92bd800640 1 -- 172.21.15.42:0/1263936213 shutdown_connections 2024-05-28T14:13:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.831+0000 7f92bd800640 1 --2- 172.21.15.42:0/1263936213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92b80d0e00 0x7f92b806dd60 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.831+0000 7f92bd800640 1 --2- 172.21.15.42:0/1263936213 >> v2:172.21.15.63:3300/0 conn(0x7f92b806f660 0x7f92b8073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.831+0000 7f92bd800640 1 -- 172.21.15.42:0/1263936213 >> 172.21.15.42:0/1263936213 conn(0x7f92b806bdf0 msgr2=0x7f92b806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.831+0000 7f92bd800640 1 -- 172.21.15.42:0/1263936213 shutdown_connections 2024-05-28T14:13:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.831+0000 7f92bd800640 1 -- 172.21.15.42:0/1263936213 wait complete. 2024-05-28T14:13:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.831+0000 7f92bd800640 1 Processor -- start 2024-05-28T14:13:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.831+0000 7f92bd800640 1 -- start start 2024-05-28T14:13:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92bd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92b806f660 0x7f92b8152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92bd800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b80d0e00 0x7f92b809a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92bd800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92b809d450 0x7f92b80b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92bd800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f92b80b3460 con 0x7f92b806f660 2024-05-28T14:13:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92bd800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92b80b32e0 con 0x7f92b80d0e00 2024-05-28T14:13:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92bd800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f92b80b35e0 con 0x7f92b809d450 2024-05-28T14:13:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92b6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92b806f660 0x7f92b8152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92b6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92b806f660 0x7f92b8152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39504/0 (socket says 172.21.15.42:39504) 2024-05-28T14:13:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92b6a00640 1 -- 172.21.15.42:0/265762521 learned_addr learned my addr 172.21.15.42:0/265762521 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92afe00640 1 --2- 172.21.15.42:0/265762521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b80d0e00 0x7f92b809a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92b7400640 1 --2- 172.21.15.42:0/265762521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92b809d450 0x7f92b80b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92b6a00640 1 -- 172.21.15.42:0/265762521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92b809d450 msgr2=0x7f92b80b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92b6a00640 1 --2- 172.21.15.42:0/265762521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92b809d450 0x7f92b80b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92b6a00640 1 -- 172.21.15.42:0/265762521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b80d0e00 msgr2=0x7f92b809a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:09.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92b6a00640 1 --2- 172.21.15.42:0/265762521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b80d0e00 0x7f92b809a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:09.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92b6a00640 1 -- 172.21.15.42:0/265762521 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b81f7750 con 0x7f92b806f660 2024-05-28T14:13:09.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92b6a00640 1 --2- 172.21.15.42:0/265762521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92b806f660 0x7f92b8152350 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f92a800e960 tx=0x7f92a800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:09.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92aea00640 1 -- 172.21.15.42:0/265762521 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8017070 con 0x7f92b806f660 2024-05-28T14:13:09.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92aea00640 1 -- 172.21.15.42:0/265762521 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a800ca00 con 0x7f92b806f660 2024-05-28T14:13:09.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92aea00640 1 -- 172.21.15.42:0/265762521 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a800cdb0 con 0x7f92b806f660 2024-05-28T14:13:09.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92bd800640 1 -- 172.21.15.42:0/265762521 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b809d930 con 0x7f92b806f660 2024-05-28T14:13:09.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92bd800640 1 -- 172.21.15.42:0/265762521 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b809def0 con 0x7f92b806f660 2024-05-28T14:13:09.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92bd800640 1 -- 172.21.15.42:0/265762521 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b8068a90 con 0x7f92b806f660 2024-05-28T14:13:09.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.835+0000 7f92aea00640 1 -- 172.21.15.42:0/265762521 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f92a801a070 con 0x7f92b806f660 2024-05-28T14:13:09.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.839+0000 7f92aea00640 1 --2- 172.21.15.42:0/265762521 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9294069b20 0x7f929406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:09.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.839+0000 7f92aea00640 1 -- 172.21.15.42:0/265762521 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f92a8085bb0 con 0x7f92b806f660 2024-05-28T14:13:09.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.839+0000 7f92afe00640 1 --2- 172.21.15.42:0/265762521 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9294069b20 0x7f929406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:09.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.839+0000 7f92afe00640 1 --2- 172.21.15.42:0/265762521 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9294069b20 0x7f929406bfe0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f92b80b0900 tx=0x7f92a0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:09.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:09.843+0000 7f92aea00640 1 -- 172.21.15.42:0/265762521 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f92a8057bd0 con 0x7f92b806f660 2024-05-28T14:13:10.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.039+0000 7f92bd800640 1 -- 172.21.15.42:0/265762521 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92b8072bb0 con 0x7f92b806f660 2024-05-28T14:13:10.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.039+0000 7f92aea00640 1 -- 172.21.15.42:0/265762521 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5180 (secure 0 0 0) 0x7f92a80574d0 con 0x7f92b806f660 2024-05-28T14:13:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:10.026 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":74,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/56330151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":56330151},{"type":"v1","addr":"172.21.15.42:6835","nonce":56330151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":9183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/192920662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":192920662},{"type":"v1","addr":"172.21.15.63:6835","nonce":192920662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":57437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1225711200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1225711200},{"type":"v1","addr":"172.21.15.42:6837","nonce":1225711200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:12:59.544555+0000","modified":"2024-05-28T14:13:02.565331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57440":{"gid":57440,"name":"b","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/3959626682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3959626682},{"type":"v1","addr":"172.21.15.63:6834","nonce":3959626682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57440,"qdb_cluster":[57440]},"id":8}]} 2024-05-28T14:13:10.026 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 74 2024-05-28T14:13:10.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.043+0000 7f92bd800640 1 -- 172.21.15.42:0/265762521 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9294069b20 msgr2=0x7f929406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.043+0000 7f92bd800640 1 --2- 172.21.15.42:0/265762521 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9294069b20 0x7f929406bfe0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f92b80b0900 tx=0x7f92a0009040 comp rx=0 tx=0).stop 2024-05-28T14:13:10.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.043+0000 7f92bd800640 1 -- 172.21.15.42:0/265762521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92b806f660 msgr2=0x7f92b8152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.043+0000 7f92bd800640 1 --2- 172.21.15.42:0/265762521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92b806f660 0x7f92b8152350 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f92a800e960 tx=0x7f92a800ee20 comp rx=0 tx=0).stop 2024-05-28T14:13:10.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.043+0000 7f92bd800640 1 -- 172.21.15.42:0/265762521 shutdown_connections 2024-05-28T14:13:10.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.043+0000 7f92bd800640 1 --2- 172.21.15.42:0/265762521 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9294069b20 0x7f929406bfe0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.043+0000 7f92bd800640 1 --2- 172.21.15.42:0/265762521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92b809d450 0x7f92b80b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.043+0000 7f92bd800640 1 --2- 172.21.15.42:0/265762521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92b80d0e00 0x7f92b809a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.043+0000 7f92bd800640 1 --2- 172.21.15.42:0/265762521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92b806f660 0x7f92b8152350 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.043+0000 7f92bd800640 1 -- 172.21.15.42:0/265762521 >> 172.21.15.42:0/265762521 conn(0x7f92b806bdf0 msgr2=0x7f92b806c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:10.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.043+0000 7f92bd800640 1 -- 172.21.15.42:0/265762521 shutdown_connections 2024-05-28T14:13:10.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.043+0000 7f92bd800640 1 -- 172.21.15.42:0/265762521 wait complete. 2024-05-28T14:13:10.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-28T14:12:59.544555+0000', 'modified': '2024-05-28T14:13:02.565331+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 57440}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_57440': {'gid': 57440, 'name': 'b', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6834/3959626682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6832', 'nonce': 3959626682}, {'type': 'v1', 'addr': '172.21.15.63:6834', 'nonce': 3959626682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 57440, 'qdb_cluster': [57440]} 2024-05-28T14:13:10.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-28T14:13:10.450 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-28T14:13:10.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8be5400640 1 Processor -- start 2024-05-28T14:13:10.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8be5400640 1 -- start start 2024-05-28T14:13:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8be5400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be00ae240 0x7f8be00ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8be5400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8be00b1640 0x7f8be00b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8be5400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be00b6590 con 0x7f8be015e820 2024-05-28T14:13:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8be5400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8be006c870 con 0x7f8be00b1640 2024-05-28T14:13:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8be5400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8be00b5d30 con 0x7f8be00ae240 2024-05-28T14:13:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8bde800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8be015e820 0x7f8be00b4260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34044/0 (socket says 172.21.15.42:34044) 2024-05-28T14:13:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8bde800640 1 -- 172.21.15.42:0/4064404885 learned_addr learned my addr 172.21.15.42:0/4064404885 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:13:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8bdf200640 1 --2- 172.21.15.42:0/4064404885 >> v2:172.21.15.42:3300/0 conn(0x7f8be00b1640 0x7f8be00b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/4064404885 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be00ae240 0x7f8be00ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8bdf200640 1 -- 172.21.15.42:0/4064404885 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be00ae240 msgr2=0x7f8be00ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8bdf200640 1 --2- 172.21.15.42:0/4064404885 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be00ae240 0x7f8be00ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8bdf200640 1 -- 172.21.15.42:0/4064404885 >> v1:172.21.15.42:6789/0 conn(0x7f8be015e820 legacy=0x7f8be00b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.611+0000 7f8bdf200640 1 -- 172.21.15.42:0/4064404885 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be015ec00 con 0x7f8be00b1640 2024-05-28T14:13:10.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.615+0000 7f8bdf200640 1 --2- 172.21.15.42:0/4064404885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8be00b1640 0x7f8be00b1ad0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009d90 tx=0x7f8bd801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17990f7a252d135 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:10.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.615+0000 7f8bdde00640 1 -- 172.21.15.42:0/4064404885 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd80b2070 con 0x7f8be00b1640 2024-05-28T14:13:10.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.615+0000 7f8bdde00640 1 -- 172.21.15.42:0/4064404885 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd80a5830 con 0x7f8be00b1640 2024-05-28T14:13:10.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.615+0000 7f8bdde00640 1 -- 172.21.15.42:0/4064404885 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd80a5c00 con 0x7f8be00b1640 2024-05-28T14:13:10.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.615+0000 7f8be5400640 1 -- 172.21.15.42:0/4064404885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8be00b1640 msgr2=0x7f8be00b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.615+0000 7f8be5400640 1 --2- 172.21.15.42:0/4064404885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8be00b1640 0x7f8be00b1ad0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009d90 tx=0x7f8bd801c970 comp rx=0 tx=0).stop 2024-05-28T14:13:10.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.615+0000 7f8be5400640 1 -- 172.21.15.42:0/4064404885 shutdown_connections 2024-05-28T14:13:10.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.615+0000 7f8be5400640 1 --2- 172.21.15.42:0/4064404885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8be00b1640 0x7f8be00b1ad0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.615+0000 7f8be5400640 1 --2- 172.21.15.42:0/4064404885 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be00ae240 0x7f8be00ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.615+0000 7f8be5400640 1 -- 172.21.15.42:0/4064404885 >> 172.21.15.42:0/4064404885 conn(0x7f8be00b6780 msgr2=0x7f8be00b6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:10.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.615+0000 7f8be5400640 1 -- 172.21.15.42:0/4064404885 shutdown_connections 2024-05-28T14:13:10.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.615+0000 7f8be5400640 1 -- 172.21.15.42:0/4064404885 wait complete. 2024-05-28T14:13:10.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8be5400640 1 Processor -- start 2024-05-28T14:13:10.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8be5400640 1 -- start start 2024-05-28T14:13:10.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8be5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8be00ae240 0x7f8be014b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8be5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8be00b1640 0x7f8be0153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8be5400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be015e820 0x7f8be0153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8be5400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8be00b2850 con 0x7f8be00b1640 2024-05-28T14:13:10.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8be5400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8be00b26d0 con 0x7f8be00ae240 2024-05-28T14:13:10.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8be5400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8be016b510 con 0x7f8be015e820 2024-05-28T14:13:10.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8be00b1640 0x7f8be0153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8be00b1640 0x7f8be0153440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39520/0 (socket says 172.21.15.42:39520) 2024-05-28T14:13:10.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8bd7e00640 1 -- 172.21.15.42:0/1324244504 learned_addr learned my addr 172.21.15.42:0/1324244504 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:10.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8bde800640 1 --2- 172.21.15.42:0/1324244504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8be00ae240 0x7f8be014b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8bdf200640 1 --2- 172.21.15.42:0/1324244504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be015e820 0x7f8be0153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8bd7e00640 1 -- 172.21.15.42:0/1324244504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be015e820 msgr2=0x7f8be0153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/1324244504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be015e820 0x7f8be0153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8bd7e00640 1 -- 172.21.15.42:0/1324244504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8be00ae240 msgr2=0x7f8be014b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/1324244504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8be00ae240 0x7f8be014b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8bd7e00640 1 -- 172.21.15.42:0/1324244504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be014bef0 con 0x7f8be00b1640 2024-05-28T14:13:10.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/1324244504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8be00b1640 0x7f8be0153440 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800e930 tx=0x7f8bc800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:10.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8bd6a00640 1 -- 172.21.15.42:0/1324244504 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8017070 con 0x7f8be00b1640 2024-05-28T14:13:10.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8bd6a00640 1 -- 172.21.15.42:0/1324244504 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc800ca00 con 0x7f8be00b1640 2024-05-28T14:13:10.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8bd6a00640 1 -- 172.21.15.42:0/1324244504 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800cdb0 con 0x7f8be00b1640 2024-05-28T14:13:10.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8be5400640 1 -- 172.21.15.42:0/1324244504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be01597d0 con 0x7f8be00b1640 2024-05-28T14:13:10.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.619+0000 7f8be5400640 1 -- 172.21.15.42:0/1324244504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be0159d90 con 0x7f8be00b1640 2024-05-28T14:13:10.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8bd6a00640 1 -- 172.21.15.42:0/1324244504 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f8bc801a070 con 0x7f8be00b1640 2024-05-28T14:13:10.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8bd6a00640 1 --2- 172.21.15.42:0/1324244504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8bbc069b20 0x7f8bbc06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8bd6a00640 1 -- 172.21.15.42:0/1324244504 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f8bc8085770 con 0x7f8be00b1640 2024-05-28T14:13:10.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8bde800640 1 --2- 172.21.15.42:0/1324244504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8bbc069b20 0x7f8bbc06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8be87fd000 1 Processor -- start 2024-05-28T14:13:10.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8be87fd000 1 -- start start 2024-05-28T14:13:10.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8bde800640 1 --2- 172.21.15.42:0/1324244504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8bbc069b20 0x7f8bbc06bfe0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8be0153130 tx=0x7f8bd0007e90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:10.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8be87fd000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9056f120 0x55fc9056f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8be87fd000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc9056fa40 0x55fc90522110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8be87fd000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc90522650 0x55fc90522a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8be87fd000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fc90373420 con 0x55fc9056fa40 2024-05-28T14:13:10.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8be87fd000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fc9030dbf0 con 0x55fc90522650 2024-05-28T14:13:10.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8be87fd000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55fc9036f3e0 con 0x55fc9056f120 2024-05-28T14:13:10.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc9056fa40 0x55fc90522110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc9056fa40 0x55fc90522110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39534/0 (socket says 172.21.15.42:39534) 2024-05-28T14:13:10.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8bd7e00640 1 -- 172.21.15.42:0/2961365918 learned_addr learned my addr 172.21.15.42:0/2961365918 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:10.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8bde800640 1 --2- 172.21.15.42:0/2961365918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9056f120 0x55fc9056f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.623+0000 7f8bdf200640 1 --2- 172.21.15.42:0/2961365918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc90522650 0x55fc90522a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bd7e00640 1 -- 172.21.15.42:0/2961365918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9056f120 msgr2=0x55fc9056f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:10.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/2961365918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9056f120 0x55fc9056f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bd7e00640 1 -- 172.21.15.42:0/2961365918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc90522650 msgr2=0x55fc90522a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:10.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/2961365918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc90522650 0x55fc90522a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bd7e00640 1 -- 172.21.15.42:0/2961365918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc9041b450 con 0x55fc9056fa40 2024-05-28T14:13:10.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bde800640 1 --2- 172.21.15.42:0/2961365918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9056f120 0x55fc9056f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:10.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bdf200640 1 --2- 172.21.15.42:0/2961365918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc90522650 0x55fc90522a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:10.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/2961365918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc9056fa40 0x55fc90522110 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8056f00 tx=0x7f8bc805b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146415981bfb5c81 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:10.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bcf400640 1 -- 172.21.15.42:0/2961365918 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8021040 con 0x55fc9056fa40 2024-05-28T14:13:10.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bcf400640 1 -- 172.21.15.42:0/2961365918 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc8055500 con 0x55fc9056fa40 2024-05-28T14:13:10.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bcf400640 1 -- 172.21.15.42:0/2961365918 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8053870 con 0x55fc9056fa40 2024-05-28T14:13:10.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 -- 172.21.15.42:0/2961365918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc9056fa40 msgr2=0x55fc90522110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 --2- 172.21.15.42:0/2961365918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc9056fa40 0x55fc90522110 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8056f00 tx=0x7f8bc805b500 comp rx=0 tx=0).stop 2024-05-28T14:13:10.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 -- 172.21.15.42:0/2961365918 shutdown_connections 2024-05-28T14:13:10.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 --2- 172.21.15.42:0/2961365918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc90522650 0x55fc90522a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 --2- 172.21.15.42:0/2961365918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc9056fa40 0x55fc90522110 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 --2- 172.21.15.42:0/2961365918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9056f120 0x55fc9056f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 -- 172.21.15.42:0/2961365918 >> 172.21.15.42:0/2961365918 conn(0x55fc9054fec0 msgr2=0x55fc905522b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:10.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 -- 172.21.15.42:0/2961365918 shutdown_connections 2024-05-28T14:13:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 -- 172.21.15.42:0/2961365918 wait complete. 2024-05-28T14:13:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 -- start start 2024-05-28T14:13:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 Processor -- start 2024-05-28T14:13:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc90522650 0x55fc90527b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 0x55fc90528080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9056fa40 0x55fc905285c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fc90313ec0 con 0x55fc90522650 2024-05-28T14:13:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fc903f4120 con 0x55fc9056f120 2024-05-28T14:13:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55fc9037e350 con 0x55fc9056fa40 2024-05-28T14:13:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8be87fd000 10 client.0 _ll_register_callbacks cb 0x55fc903504d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T14:13:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bde800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc90522650 0x55fc90527b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bde800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc90522650 0x55fc90527b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39544/0 (socket says 172.21.15.42:39544) 2024-05-28T14:13:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bde800640 1 -- 172.21.15.42:0/2022576433 learned_addr learned my addr 172.21.15.42:0/2022576433 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:10.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bdf200640 1 --2- 172.21.15.42:0/2022576433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9056fa40 0x55fc905285c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/2022576433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 0x55fc90528080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bde800640 1 -- 172.21.15.42:0/2022576433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9056fa40 msgr2=0x55fc905285c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bde800640 1 --2- 172.21.15.42:0/2022576433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9056fa40 0x55fc905285c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bde800640 1 -- 172.21.15.42:0/2022576433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 msgr2=0x55fc90528080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bde800640 1 --2- 172.21.15.42:0/2022576433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 0x55fc90528080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bde800640 1 -- 172.21.15.42:0/2022576433 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc903b8f80 con 0x55fc90522650 2024-05-28T14:13:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.627+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/2022576433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 0x55fc90528080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:13:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8bde800640 1 --2- 172.21.15.42:0/2022576433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc90522650 0x55fc90527b40 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0001e20 tx=0x7f8bd0000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a251fd54e0582b9d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8bc3e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T14:13:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8bc3e00640 1 -- 172.21.15.42:0/2022576433 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0027060 con 0x55fc90522650 2024-05-28T14:13:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8bc3e00640 1 -- 172.21.15.42:0/2022576433 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f8bd0013e00 con 0x55fc90522650 2024-05-28T14:13:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 1 -- 172.21.15.42:0/2022576433 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc8055500 con 0x55fc90522650 2024-05-28T14:13:10.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8bc3e00640 1 -- 172.21.15.42:0/2022576433 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f8bd0012560 con 0x55fc90522650 2024-05-28T14:13:10.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 10 client.9351 fetch_fsmap learned FSMap version 74 2024-05-28T14:13:10.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 1 -- 172.21.15.42:0/2022576433 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55fc9034cf00 con 0x55fc90522650 2024-05-28T14:13:10.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8bc3e00640 1 -- 172.21.15.42:0/2022576433 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc8055500 con 0x55fc90522650 2024-05-28T14:13:10.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8bc3e00640 1 -- 172.21.15.42:0/2022576433 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f8bd0012700 con 0x55fc90522650 2024-05-28T14:13:10.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 10 client.9351 fetch_fsmap finished waiting for FSMap version 74 2024-05-28T14:13:10.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 10 client.9351 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-28T14:13:10.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 20 client.9351 populate_metadata read hostname 'smithi042' 2024-05-28T14:13:10.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 1 --2- 172.21.15.42:0/2022576433 >> [v2:172.21.15.63:6832/3959626682,v1:172.21.15.63:6834/3959626682] conn(0x55fc9057e3c0 0x55fc905807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 4 client.9351 mds_command: new command op to 57440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-28T14:13:10.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 1 -- 172.21.15.42:0/2022576433 --> [v2:172.21.15.63:6832/3959626682,v1:172.21.15.63:6834/3959626682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc9030cbc0 con 0x55fc9057e3c0 2024-05-28T14:13:10.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/2022576433 >> [v2:172.21.15.63:6832/3959626682,v1:172.21.15.63:6834/3959626682] conn(0x55fc9057e3c0 0x55fc905807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/2022576433 >> [v2:172.21.15.63:6832/3959626682,v1:172.21.15.63:6834/3959626682] conn(0x55fc9057e3c0 0x55fc905807b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=955091f163fb2903 server_cookie=f703177bfbed0d5a in_seq=0 out_seq=0 2024-05-28T14:13:10.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8bc3e00640 10 client.9351 ms_handle_connect on v2:172.21.15.63:6832/3959626682 2024-05-28T14:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8bc3e00640 1 -- 172.21.15.42:0/2022576433 <== mds.0 v2:172.21.15.63:6832/3959626682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc9030cbc0 con 0x55fc9057e3c0 2024-05-28T14:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8bc3e00640 10 client.9351 handle_command_reply: tid=0 2024-05-28T14:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 1 client.9351 shutdown 2024-05-28T14:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 1 -- 172.21.15.42:0/2022576433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc90522650 msgr2=0x55fc90527b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 1 --2- 172.21.15.42:0/2022576433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc90522650 0x55fc90527b40 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0001e20 tx=0x7f8bd0000bd0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 1 -- 172.21.15.42:0/2022576433 shutdown_connections 2024-05-28T14:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 1 --2- 172.21.15.42:0/2022576433 >> [v2:172.21.15.63:6832/3959626682,v1:172.21.15.63:6834/3959626682] conn(0x55fc9057e3c0 0x55fc905807b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 1 --2- 172.21.15.42:0/2022576433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9056fa40 0x55fc905285c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 1 --2- 172.21.15.42:0/2022576433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 0x55fc90528080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 1 --2- 172.21.15.42:0/2022576433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc90522650 0x55fc90527b40 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8bc3e00640 0 client.9351 ms_handle_reset on v2:172.21.15.63:6832/3959626682 2024-05-28T14:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 1 -- 172.21.15.42:0/2022576433 >> 172.21.15.42:0/2022576433 conn(0x55fc9054fec0 msgr2=0x55fc905522b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:10.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.631+0000 7f8be87fd000 1 -- 172.21.15.42:0/2022576433 shutdown_connections 2024-05-28T14:13:10.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.635+0000 7f8be87fd000 1 -- 172.21.15.42:0/2022576433 wait complete. 2024-05-28T14:13:10.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.635+0000 7f8be87fd000 20 client.9351 trim_cache size 0 max 16384 2024-05-28T14:13:10.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8be87fd000 1 Processor -- start 2024-05-28T14:13:10.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8be87fd000 1 -- start start 2024-05-28T14:13:10.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8be87fd000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9054fec0 0x55fc90521770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8be87fd000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc90522650 0x55fc90521cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8be87fd000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 0x55fc905807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8be87fd000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0203100 con 0x55fc90522650 2024-05-28T14:13:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8be87fd000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0155840 con 0x55fc9056f120 2024-05-28T14:13:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8be87fd000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8be0155e90 con 0x55fc9054fec0 2024-05-28T14:13:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc90522650 0x55fc90521cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc90522650 0x55fc90521cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39550/0 (socket says 172.21.15.42:39550) 2024-05-28T14:13:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8bd7e00640 1 -- 172.21.15.42:0/68308848 learned_addr learned my addr 172.21.15.42:0/68308848 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8bde800640 1 --2- 172.21.15.42:0/68308848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9054fec0 0x55fc90521770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8bdf200640 1 --2- 172.21.15.42:0/68308848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 0x55fc905807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8bd7e00640 1 -- 172.21.15.42:0/68308848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9054fec0 msgr2=0x55fc90521770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:10.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/68308848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9054fec0 0x55fc90521770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8bd7e00640 1 -- 172.21.15.42:0/68308848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 msgr2=0x55fc905807b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/68308848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 0x55fc905807b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8bd7e00640 1 -- 172.21.15.42:0/68308848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc902f9e80 con 0x55fc90522650 2024-05-28T14:13:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.651+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/68308848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc90522650 0x55fc90521cb0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8053660 tx=0x7f8bc801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b07f69e6382cfe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bcea00640 1 -- 172.21.15.42:0/68308848 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8021040 con 0x55fc90522650 2024-05-28T14:13:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bcea00640 1 -- 172.21.15.42:0/68308848 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc8074070 con 0x55fc90522650 2024-05-28T14:13:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bcea00640 1 -- 172.21.15.42:0/68308848 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8056650 con 0x55fc90522650 2024-05-28T14:13:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 -- 172.21.15.42:0/68308848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc90522650 msgr2=0x55fc90521cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 --2- 172.21.15.42:0/68308848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc90522650 0x55fc90521cb0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8053660 tx=0x7f8bc801e000 comp rx=0 tx=0).stop 2024-05-28T14:13:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 -- 172.21.15.42:0/68308848 shutdown_connections 2024-05-28T14:13:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 --2- 172.21.15.42:0/68308848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 0x55fc905807b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 --2- 172.21.15.42:0/68308848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc90522650 0x55fc90521cb0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 --2- 172.21.15.42:0/68308848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc9054fec0 0x55fc90521770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 -- 172.21.15.42:0/68308848 >> 172.21.15.42:0/68308848 conn(0x55fc9052a540 msgr2=0x55fc90583a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:10.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 -- 172.21.15.42:0/68308848 shutdown_connections 2024-05-28T14:13:10.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 -- 172.21.15.42:0/68308848 wait complete. 2024-05-28T14:13:10.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 -- start start 2024-05-28T14:13:10.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 Processor -- start 2024-05-28T14:13:10.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc90522650 0x55fc90583970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc9054fec0 0x55fc9059a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 0x55fc9059a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fc90372730 con 0x55fc9054fec0 2024-05-28T14:13:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fc9033f9e0 con 0x55fc9056f120 2024-05-28T14:13:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55fc90377a50 con 0x55fc90522650 2024-05-28T14:13:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 10 client.0 _ll_register_callbacks cb 0x7f8bd0004b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T14:13:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc9054fec0 0x55fc9059a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc9054fec0 0x55fc9059a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39556/0 (socket says 172.21.15.42:39556) 2024-05-28T14:13:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bd7e00640 1 -- 172.21.15.42:0/3271341390 learned_addr learned my addr 172.21.15.42:0/3271341390 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bde800640 1 --2- 172.21.15.42:0/3271341390 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc90522650 0x55fc90583970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bdf200640 1 --2- 172.21.15.42:0/3271341390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 0x55fc9059a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bd7e00640 1 -- 172.21.15.42:0/3271341390 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc90522650 msgr2=0x55fc90583970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/3271341390 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc90522650 0x55fc90583970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bd7e00640 1 -- 172.21.15.42:0/3271341390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 msgr2=0x55fc9059a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/3271341390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 0x55fc9059a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bd7e00640 1 -- 172.21.15.42:0/3271341390 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc9033bb30 con 0x55fc9054fec0 2024-05-28T14:13:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bd7e00640 1 --2- 172.21.15.42:0/3271341390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc9054fec0 0x55fc9059a340 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8053660 tx=0x7f8bc800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbb28c6bdf9b9cd server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bc3e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T14:13:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bc3e00640 1 -- 172.21.15.42:0/3271341390 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8056650 con 0x55fc9054fec0 2024-05-28T14:13:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bc3e00640 1 -- 172.21.15.42:0/3271341390 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f8bc80567f0 con 0x55fc9054fec0 2024-05-28T14:13:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 -- 172.21.15.42:0/3271341390 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc8074070 con 0x55fc9054fec0 2024-05-28T14:13:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8bc3e00640 1 -- 172.21.15.42:0/3271341390 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f8bc8052f50 con 0x55fc9054fec0 2024-05-28T14:13:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 10 client.9357 fetch_fsmap learned FSMap version 74 2024-05-28T14:13:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.655+0000 7f8be87fd000 1 -- 172.21.15.42:0/3271341390 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55fc903f45b0 con 0x55fc9054fec0 2024-05-28T14:13:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8bc3e00640 1 -- 172.21.15.42:0/3271341390 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc8074070 con 0x55fc9054fec0 2024-05-28T14:13:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8bc3e00640 1 -- 172.21.15.42:0/3271341390 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f8bc807c060 con 0x55fc9054fec0 2024-05-28T14:13:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 10 client.9357 fetch_fsmap finished waiting for FSMap version 74 2024-05-28T14:13:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 10 client.9357 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-28T14:13:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 20 client.9357 populate_metadata read hostname 'smithi042' 2024-05-28T14:13:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 1 --2- 172.21.15.42:0/3271341390 >> [v2:172.21.15.63:6832/3959626682,v1:172.21.15.63:6834/3959626682] conn(0x55fc9056fa40 0x55fc9059adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 4 client.9357 mds_command: new command op to 57440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-28T14:13:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 1 -- 172.21.15.42:0/3271341390 --> [v2:172.21.15.63:6832/3959626682,v1:172.21.15.63:6834/3959626682] -- command(tid 0: {"prefix": "status"}) -- 0x55fc90380e40 con 0x55fc9056fa40 2024-05-28T14:13:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8bde800640 1 --2- 172.21.15.42:0/3271341390 >> [v2:172.21.15.63:6832/3959626682,v1:172.21.15.63:6834/3959626682] conn(0x55fc9056fa40 0x55fc9059adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8bde800640 1 --2- 172.21.15.42:0/3271341390 >> [v2:172.21.15.63:6832/3959626682,v1:172.21.15.63:6834/3959626682] conn(0x55fc9056fa40 0x55fc9059adc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a368ead3ec80d071 server_cookie=8da1345dee6799fe in_seq=0 out_seq=0 2024-05-28T14:13:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8bc3e00640 10 client.9357 ms_handle_connect on v2:172.21.15.63:6832/3959626682 2024-05-28T14:13:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8bc3e00640 1 -- 172.21.15.42:0/3271341390 <== mds.0 v2:172.21.15.63:6832/3959626682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55fc90380e40 con 0x55fc9056fa40 2024-05-28T14:13:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8bc3e00640 10 client.9357 handle_command_reply: tid=0 2024-05-28T14:13:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 1 client.9357 shutdown 2024-05-28T14:13:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 1 -- 172.21.15.42:0/3271341390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc9054fec0 msgr2=0x55fc9059a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 1 --2- 172.21.15.42:0/3271341390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc9054fec0 0x55fc9059a340 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8053660 tx=0x7f8bc800cf50 comp rx=0 tx=0).stop 2024-05-28T14:13:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 1 -- 172.21.15.42:0/3271341390 shutdown_connections 2024-05-28T14:13:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 1 --2- 172.21.15.42:0/3271341390 >> [v2:172.21.15.63:6832/3959626682,v1:172.21.15.63:6834/3959626682] conn(0x55fc9056fa40 0x55fc9059adc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 1 --2- 172.21.15.42:0/3271341390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc9056f120 0x55fc9059a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 1 --2- 172.21.15.42:0/3271341390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc9054fec0 0x55fc9059a340 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 1 --2- 172.21.15.42:0/3271341390 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fc90522650 0x55fc90583970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8bc3e00640 0 client.9357 ms_handle_reset on v2:172.21.15.63:6832/3959626682 2024-05-28T14:13:10.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 1 -- 172.21.15.42:0/3271341390 >> 172.21.15.42:0/3271341390 conn(0x55fc9052a540 msgr2=0x55fc9056d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:10.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 1 -- 172.21.15.42:0/3271341390 shutdown_connections 2024-05-28T14:13:10.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 1 -- 172.21.15.42:0/3271341390 wait complete. 2024-05-28T14:13:10.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be87fd000 20 client.9357 trim_cache size 0 max 16384 2024-05-28T14:13:10.648 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:13:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:13:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-05-28T14:13:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 57440, 2024-05-28T14:13:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-28T14:13:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-28T14:13:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-28T14:13:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 10.525743987, 2024-05-28T14:13:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 74, 2024-05-28T14:13:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 117, 2024-05-28T14:13:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 117, 2024-05-28T14:13:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 15.351023511999999 2024-05-28T14:13:10.649 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:13:10.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be5400640 1 -- 172.21.15.42:0/1324244504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8bbc069b20 msgr2=0x7f8bbc06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be5400640 1 --2- 172.21.15.42:0/1324244504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8bbc069b20 0x7f8bbc06bfe0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8be0153130 tx=0x7f8bd0007e90 comp rx=0 tx=0).stop 2024-05-28T14:13:10.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be5400640 1 -- 172.21.15.42:0/1324244504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8be00b1640 msgr2=0x7f8be0153440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be5400640 1 --2- 172.21.15.42:0/1324244504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8be00b1640 0x7f8be0153440 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800e930 tx=0x7f8bc800edf0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be5400640 1 -- 172.21.15.42:0/1324244504 shutdown_connections 2024-05-28T14:13:10.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be5400640 1 --2- 172.21.15.42:0/1324244504 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8bbc069b20 0x7f8bbc06bfe0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be5400640 1 --2- 172.21.15.42:0/1324244504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be015e820 0x7f8be0153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be5400640 1 --2- 172.21.15.42:0/1324244504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8be00b1640 0x7f8be0153440 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be5400640 1 --2- 172.21.15.42:0/1324244504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8be00ae240 0x7f8be014b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be5400640 1 -- 172.21.15.42:0/1324244504 >> 172.21.15.42:0/1324244504 conn(0x7f8be00b6780 msgr2=0x7f8be006ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:10.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be5400640 1 -- 172.21.15.42:0/1324244504 shutdown_connections 2024-05-28T14:13:10.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.659+0000 7f8be5400640 1 -- 172.21.15.42:0/1324244504 wait complete. 2024-05-28T14:13:10.664 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-28T14:13:10.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.819+0000 7f5130c00640 1 Processor -- start 2024-05-28T14:13:10.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.819+0000 7f5130c00640 1 -- start start 2024-05-28T14:13:10.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.819+0000 7f5130c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f512c0ad680 0x7f512c06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.823+0000 7f5130c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512c073bb0 con 0x7f512c15dfa0 2024-05-28T14:13:10.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.823+0000 7f5130c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512c073e90 con 0x7f512c06e1a0 2024-05-28T14:13:10.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.823+0000 7f5130c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f512c0b31e0 con 0x7f512c0ad680 2024-05-28T14:13:10.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.823+0000 7f5129e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f512c15dfa0 0x7f512c0732d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34060/0 (socket says 172.21.15.42:34060) 2024-05-28T14:13:10.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.823+0000 7f5129e00640 1 -- 172.21.15.42:0/1044426422 learned_addr learned my addr 172.21.15.42:0/1044426422 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:13:10.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.823+0000 7f5121400640 1 --2- 172.21.15.42:0/1044426422 >> v2:172.21.15.63:3300/0 conn(0x7f512c0ad680 0x7f512c06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.823+0000 7f5129400640 1 -- 172.21.15.42:0/1044426422 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1252963927 0 0) 0x7f512c073bb0 con 0x7f512c15dfa0 2024-05-28T14:13:10.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.823+0000 7f5129400640 1 -- 172.21.15.42:0/1044426422 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5110003530 con 0x7f512c15dfa0 2024-05-28T14:13:10.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.823+0000 7f5121400640 1 -- 172.21.15.42:0/1044426422 >> v1:172.21.15.63:6789/0 conn(0x7f512c06e1a0 legacy=0x7f512c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.823+0000 7f5121400640 1 -- 172.21.15.42:0/1044426422 >> v1:172.21.15.42:6789/0 conn(0x7f512c15dfa0 legacy=0x7f512c0732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.823+0000 7f5121400640 1 -- 172.21.15.42:0/1044426422 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c0b3430 con 0x7f512c0ad680 2024-05-28T14:13:10.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.823+0000 7f5121400640 1 --2- 172.21.15.42:0/1044426422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512c0ad680 0x7f512c06e6b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f511400a960 tx=0x7f511401d7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bf065d8d79d431a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:10.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5129400640 1 -- 172.21.15.42:0/1044426422 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51140b3070 con 0x7f512c0ad680 2024-05-28T14:13:10.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5129400640 1 -- 172.21.15.42:0/1044426422 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51140a6720 con 0x7f512c0ad680 2024-05-28T14:13:10.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5129400640 1 -- 172.21.15.42:0/1044426422 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511400b040 con 0x7f512c0ad680 2024-05-28T14:13:10.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 -- 172.21.15.42:0/1044426422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512c0ad680 msgr2=0x7f512c06e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 --2- 172.21.15.42:0/1044426422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512c0ad680 0x7f512c06e6b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f511400a960 tx=0x7f511401d7e0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 -- 172.21.15.42:0/1044426422 shutdown_connections 2024-05-28T14:13:10.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 --2- 172.21.15.42:0/1044426422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512c0ad680 0x7f512c06e6b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 -- 172.21.15.42:0/1044426422 >> 172.21.15.42:0/1044426422 conn(0x7f512c06bdf0 msgr2=0x7f512c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:10.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 -- 172.21.15.42:0/1044426422 shutdown_connections 2024-05-28T14:13:10.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 -- 172.21.15.42:0/1044426422 wait complete. 2024-05-28T14:13:10.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 Processor -- start 2024-05-28T14:13:10.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 -- start start 2024-05-28T14:13:10.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512c06e1a0 0x7f512c154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f512c0ad680 0x7f512c099860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f512c15dfa0 0x7f512c1564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f512c0b44c0 con 0x7f512c0ad680 2024-05-28T14:13:10.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f512c0b4340 con 0x7f512c06e1a0 2024-05-28T14:13:10.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5130c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f512c0b4640 con 0x7f512c15dfa0 2024-05-28T14:13:10.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5121400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f512c0ad680 0x7f512c099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5121400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f512c0ad680 0x7f512c099860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39572/0 (socket says 172.21.15.42:39572) 2024-05-28T14:13:10.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5121400640 1 -- 172.21.15.42:0/1436187309 learned_addr learned my addr 172.21.15.42:0/1436187309 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:10.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f512a800640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f512c15dfa0 0x7f512c1564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.827+0000 7f5129e00640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512c06e1a0 0x7f512c154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5121400640 1 -- 172.21.15.42:0/1436187309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f512c15dfa0 msgr2=0x7f512c1564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5121400640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f512c15dfa0 0x7f512c1564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5121400640 1 -- 172.21.15.42:0/1436187309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512c06e1a0 msgr2=0x7f512c154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:10.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5121400640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512c06e1a0 0x7f512c154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:10.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5121400640 1 -- 172.21.15.42:0/1436187309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c15cd20 con 0x7f512c0ad680 2024-05-28T14:13:10.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5129e00640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512c06e1a0 0x7f512c154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:10.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5121400640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f512c0ad680 0x7f512c099860 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f511401b410 tx=0x7f511401a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:10.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5122a00640 1 -- 172.21.15.42:0/1436187309 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51140b3070 con 0x7f512c0ad680 2024-05-28T14:13:10.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5122a00640 1 -- 172.21.15.42:0/1436187309 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51140ae650 con 0x7f512c0ad680 2024-05-28T14:13:10.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5122a00640 1 -- 172.21.15.42:0/1436187309 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51140aea40 con 0x7f512c0ad680 2024-05-28T14:13:10.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5130c00640 1 -- 172.21.15.42:0/1436187309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512c163d70 con 0x7f512c0ad680 2024-05-28T14:13:10.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5130c00640 1 -- 172.21.15.42:0/1436187309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512c164330 con 0x7f512c0ad680 2024-05-28T14:13:10.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5130c00640 1 -- 172.21.15.42:0/1436187309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f512c068a90 con 0x7f512c0ad680 2024-05-28T14:13:10.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5122a00640 1 -- 172.21.15.42:0/1436187309 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f51140b7030 con 0x7f512c0ad680 2024-05-28T14:13:10.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5122a00640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5108069b20 0x7f510806bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:10.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5122a00640 1 -- 172.21.15.42:0/1436187309 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f511411f2d0 con 0x7f512c0ad680 2024-05-28T14:13:10.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.831+0000 7f5129e00640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5108069b20 0x7f510806bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:10.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.835+0000 7f5129e00640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5108069b20 0x7f510806bfe0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f511c0011f0 tx=0x7f511c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:10.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:10.835+0000 7f5122a00640 1 -- 172.21.15.42:0/1436187309 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5114017460 con 0x7f512c0ad680 2024-05-28T14:13:11.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11.039+0000 7f5130c00640 1 -- 172.21.15.42:0/1436187309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f512c06dd60 con 0x7f512c0ad680 2024-05-28T14:13:11.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11.039+0000 7f5122a00640 1 -- 172.21.15.42:0/1436187309 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7f51140f0d40 con 0x7f512c0ad680 2024-05-28T14:13:11.024 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-28T14:13:11.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11.039+0000 7f5130c00640 1 -- 172.21.15.42:0/1436187309 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5108069b20 msgr2=0x7f510806bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:11.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11.039+0000 7f5130c00640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5108069b20 0x7f510806bfe0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f511c0011f0 tx=0x7f511c009040 comp rx=0 tx=0).stop 2024-05-28T14:13:11.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11.039+0000 7f5130c00640 1 -- 172.21.15.42:0/1436187309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f512c0ad680 msgr2=0x7f512c099860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:11.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11.039+0000 7f5130c00640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f512c0ad680 0x7f512c099860 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f511401b410 tx=0x7f511401a580 comp rx=0 tx=0).stop 2024-05-28T14:13:11.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11.039+0000 7f5130c00640 1 -- 172.21.15.42:0/1436187309 shutdown_connections 2024-05-28T14:13:11.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11.039+0000 7f5130c00640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5108069b20 0x7f510806bfe0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:11.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11.039+0000 7f5130c00640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f512c15dfa0 0x7f512c1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:11.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11.039+0000 7f5130c00640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f512c0ad680 0x7f512c099860 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:11.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11.039+0000 7f5130c00640 1 --2- 172.21.15.42:0/1436187309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512c06e1a0 0x7f512c154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:11.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11.039+0000 7f5130c00640 1 -- 172.21.15.42:0/1436187309 >> 172.21.15.42:0/1436187309 conn(0x7f512c06bdf0 msgr2=0x7f512c14b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:11.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11.039+0000 7f5130c00640 1 -- 172.21.15.42:0/1436187309 shutdown_connections 2024-05-28T14:13:11.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11.039+0000 7f5130c00640 1 -- 172.21.15.42:0/1436187309 wait complete. 2024-05-28T14:13:11.045 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-28T14:13:11.045 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-28T14:13:11.045 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-05-28T14:13:11.046 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-28T14:13:11.046 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-28T14:13:11.046 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-28T14:13:11.046 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-28T14:13:11.046 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-28T14:13:11.046 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:13:11.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:11.046 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-05-28T14:13:11.056 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T14:13:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T14:13:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T14:13:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:13:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-05-28T14:13:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:13:11.056 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:13:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-05-28T14:13:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-05-28T14:13:11.056 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:13:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-05-28T14:13:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-05-28T14:13:11.057 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-28T14:13:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:13:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-05-28T14:13:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-28T14:13:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 41556sec preferred_lft 41556sec 2024-05-28T14:13:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2024-05-28T14:13:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:13:11.057 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:13:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:13:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-05-28T14:13:11.057 INFO:teuthology.orchestra.run.smithi042.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-28T14:13:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 02:42:65:2b:63:73 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:13:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-28T14:13:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:13:11.058 INFO:teuthology.orchestra.run.smithi042.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-28T14:13:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-28T14:13:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-28T14:13:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:13:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-28T14:13:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:13:11.058 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T14:13:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:13:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-28T14:13:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:13:11.059 INFO:teuthology.orchestra.run.smithi042.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T14:13:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:13:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-28T14:13:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:13:11.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:11.060 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-28T14:13:11.120 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-05-28T14:13:11.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:11.122 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-05-28T14:13:11.175 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-05-28T14:13:11.175 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-28T14:13:11.175 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-05-28T14:13:11.176 INFO:teuthology.orchestra.run.smithi042.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-28T14:13:11.176 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-05-28T14:13:11.176 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-28T14:13:11.176 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T14:13:11.176 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T14:13:11.176 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-28T14:13:11.176 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-28T14:13:11.177 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-28T14:13:11.177 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T14:13:11.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T14:13:11.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:13:11.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:11.319 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-28T14:13:11.328 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-28T14:13:11.328 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-28T14:13:11.329 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:13:11.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:11.329 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-05-28T14:13:11.394 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-28T14:13:11.395 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T14:13:11.395 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-28T14:13:11.446 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:13:11.446 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T14:13:11.447 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-28T14:13:11.498 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:13:11.563 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:13:11.615 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:13:11.615 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:13:11.615 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:13:11.616 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:13:11.616 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:13:11.616 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:13:11.616 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:13:11.616 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:13:11.616 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:13:11.616 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:13:11.616 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:13:11.616 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:13:11.616 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:13:11.616 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:13:11.616 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:13:11.617 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:13:11.617 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:13:11.617 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:13:11.617 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:13:11.617 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:13:11.617 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:13:11.617 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:13:11.617 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:13:11.618 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:13:11.618 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:13:11.618 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:13:11.619 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:13:11.619 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:13:11.619 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:13:11.619 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:13:11.619 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:13:11.619 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:13:11.619 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:13:11.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:11.620 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:13:11.670 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-28T14:13:11.671 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-28T14:13:11.717 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:13:11.737 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:13:11.787 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:13:11.787 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:13:11.788 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:13:11.788 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:13:11.788 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:13:11.788 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:13:11.788 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:13:11.788 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:13:11.788 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:13:11.788 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:13:11.788 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:13:11.788 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:13:11.788 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:13:11.788 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:13:11.789 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:13:11.789 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:13:11.789 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:13:11.789 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:13:11.789 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:13:11.789 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:13:11.789 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:13:11.789 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:13:11.789 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:13:11.789 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:13:11.789 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:13:11.789 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:13:11.790 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:13:11.790 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:13:11.790 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:13:11.790 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:13:11.790 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:13:11.790 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:13:11.790 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:13:11.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:11.791 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:13:11.814 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-05-28T14:13:11.827+0000 7f023594c580 -1 init, newargv = 0x55e49a47c1a0 newargc=15 2024-05-28T14:13:11.814 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[52714]: starting ceph client 2024-05-28T14:13:11.821 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[52714]: starting fuse 2024-05-28T14:13:11.822 INFO:teuthology.orchestra.run.smithi042.stdout:44 2024-05-28T14:13:11.822 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-28T14:13:11.823 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-05-28T14:13:11.823 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-05-28T14:13:11.823 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-05-28T14:13:11.823 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-05-28T14:13:11.823 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-05-28T14:13:11.823 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:13:11.823 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-05-28T14:13:11.823 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-28T14:13:11.823 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-05-28T14:13:11.823 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-28T14:13:11.823 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:13:11.823 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-05-28T14:13:11.823 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-05-28T14:13:11.824 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-05-28T14:13:11.824 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-05-28T14:13:11.824 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:13:11.824 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-05-28T14:13:11.824 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-28T14:13:11.824 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-28T14:13:11.824 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-28T14:13:11.824 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-05-28T14:13:11.824 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-05-28T14:13:11.824 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-05-28T14:13:11.824 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-28T14:13:11.824 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:13:11.824 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-05-28T14:13:11.824 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-05-28T14:13:11.949 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.52714.asok 2024-05-28T14:13:11.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:13:11.961 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52714.asok 2024-05-28T14:13:11.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:11.961 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52714.asok status 2024-05-28T14:13:12.134 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:13:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-05-28T14:13:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-28T14:13:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-28T14:13:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-05-28T14:13:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-05-28T14:13:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-28T14:13:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "52714", 2024-05-28T14:13:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-05-28T14:13:12.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-05-28T14:13:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-05-28T14:13:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 9369, 2024-05-28T14:13:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-05-28T14:13:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-05-28T14:13:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-05-28T14:13:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 9369 2024-05-28T14:13:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T14:13:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T14:13:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T14:13:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4141966056 2024-05-28T14:13:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:13:12.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T14:13:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T14:13:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T14:13:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4141966056 2024-05-28T14:13:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.9369 192.168.0.1:0/4141966056", 2024-05-28T14:13:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/4141966056", 2024-05-28T14:13:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-05-28T14:13:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 74, 2024-05-28T14:13:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 117, 2024-05-28T14:13:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-05-28T14:13:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-05-28T14:13:12.136 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-05-28T14:13:12.136 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:13:12.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:12.149 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-05-28T14:13:12.149 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-28T14:13:12.158 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-05-28T14:13:12.158 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-28T14:13:12.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:12.159 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-28T14:13:12.223 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-28T14:13:12.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.423+0000 7fa198200640 1 Processor -- start 2024-05-28T14:13:12.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.423+0000 7fa198200640 1 -- start start 2024-05-28T14:13:12.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.427+0000 7fa198200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1900afb20 0x7fa19006cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:12.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.427+0000 7fa198200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa1900b0070 0x7fa1900d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:12.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.427+0000 7fa198200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1900732e0 con 0x7fa19006cbd0 2024-05-28T14:13:12.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.427+0000 7fa198200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa1900735c0 con 0x7fa1900b0070 2024-05-28T14:13:12.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.427+0000 7fa198200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1900ce110 con 0x7fa1900afb20 2024-05-28T14:13:12.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.427+0000 7fa196200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa1900b0070 0x7fa1900d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:12.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.427+0000 7fa196200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa1900b0070 0x7fa1900d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39582/0 (socket says 172.21.15.42:39582) 2024-05-28T14:13:12.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.427+0000 7fa196200640 1 -- 172.21.15.42:0/3740583716 learned_addr learned my addr 172.21.15.42:0/3740583716 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:12.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.427+0000 7fa194e00640 1 --2- 172.21.15.42:0/3740583716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1900afb20 0x7fa19006cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:12.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.427+0000 7fa196200640 1 -- 172.21.15.42:0/3740583716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1900afb20 msgr2=0x7fa19006cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:12.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.427+0000 7fa196200640 1 --2- 172.21.15.42:0/3740583716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1900afb20 0x7fa19006cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:12.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.427+0000 7fa196200640 1 -- 172.21.15.42:0/3740583716 >> v1:172.21.15.63:6789/0 conn(0x7fa19006cbd0 legacy=0x7fa19015cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:12.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.427+0000 7fa196200640 1 -- 172.21.15.42:0/3740583716 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1900ce3f0 con 0x7fa1900b0070 2024-05-28T14:13:12.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa196200640 1 --2- 172.21.15.42:0/3740583716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1900b0070 0x7fa1900d0040 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fa188009e50 tx=0x7fa18801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afc69a7f707dfe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:12.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa18fe00640 1 -- 172.21.15.42:0/3740583716 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1880a5810 con 0x7fa1900b0070 2024-05-28T14:13:12.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa18fe00640 1 -- 172.21.15.42:0/3740583716 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1880a59b0 con 0x7fa1900b0070 2024-05-28T14:13:12.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa18fe00640 1 -- 172.21.15.42:0/3740583716 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1880a5d80 con 0x7fa1900b0070 2024-05-28T14:13:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa198200640 1 -- 172.21.15.42:0/3740583716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1900b0070 msgr2=0x7fa1900d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa198200640 1 --2- 172.21.15.42:0/3740583716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1900b0070 0x7fa1900d0040 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fa188009e50 tx=0x7fa18801ca30 comp rx=0 tx=0).stop 2024-05-28T14:13:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa198200640 1 -- 172.21.15.42:0/3740583716 shutdown_connections 2024-05-28T14:13:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa198200640 1 --2- 172.21.15.42:0/3740583716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1900b0070 0x7fa1900d0040 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa198200640 1 --2- 172.21.15.42:0/3740583716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1900afb20 0x7fa19006cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa198200640 1 -- 172.21.15.42:0/3740583716 >> 172.21.15.42:0/3740583716 conn(0x7fa1900725b0 msgr2=0x7fa1900729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa198200640 1 -- 172.21.15.42:0/3740583716 shutdown_connections 2024-05-28T14:13:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa198200640 1 -- 172.21.15.42:0/3740583716 wait complete. 2024-05-28T14:13:12.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa198200640 1 Processor -- start 2024-05-28T14:13:12.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa198200640 1 -- start start 2024-05-28T14:13:12.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa198200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa19006cbd0 0x7fa19015b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:12.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.431+0000 7fa198200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1900afb20 0x7fa190151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:12.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa198200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1900b0070 0x7fa19015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:12.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa198200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901612b0 con 0x7fa1900afb20 2024-05-28T14:13:12.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa198200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa190161130 con 0x7fa1900b0070 2024-05-28T14:13:12.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa198200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa190161430 con 0x7fa19006cbd0 2024-05-28T14:13:12.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa194e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1900afb20 0x7fa190151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:12.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa194e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1900afb20 0x7fa190151f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39594/0 (socket says 172.21.15.42:39594) 2024-05-28T14:13:12.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa194e00640 1 -- 172.21.15.42:0/1891572515 learned_addr learned my addr 172.21.15.42:0/1891572515 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:12.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa196200640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1900b0070 0x7fa19015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:12.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa195800640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa19006cbd0 0x7fa19015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:12.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa194e00640 1 -- 172.21.15.42:0/1891572515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa19006cbd0 msgr2=0x7fa19015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:12.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa194e00640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa19006cbd0 0x7fa19015b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:12.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa194e00640 1 -- 172.21.15.42:0/1891572515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1900b0070 msgr2=0x7fa19015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:12.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa194e00640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1900b0070 0x7fa19015bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:12.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa194e00640 1 -- 172.21.15.42:0/1891572515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa190159fd0 con 0x7fa1900afb20 2024-05-28T14:13:12.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa196200640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1900b0070 0x7fa19015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:12.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa194e00640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1900afb20 0x7fa190151f10 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00d830 tx=0x7fa17c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:12.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa18d600640 1 -- 172.21.15.42:0/1891572515 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00b6c0 con 0x7fa1900afb20 2024-05-28T14:13:12.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa18d600640 1 -- 172.21.15.42:0/1891572515 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa17c019070 con 0x7fa1900afb20 2024-05-28T14:13:12.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa18d600640 1 -- 172.21.15.42:0/1891572515 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00bae0 con 0x7fa1900afb20 2024-05-28T14:13:12.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa198200640 1 -- 172.21.15.42:0/1891572515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19015a290 con 0x7fa1900afb20 2024-05-28T14:13:12.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa198200640 1 -- 172.21.15.42:0/1891572515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa19015a850 con 0x7fa1900afb20 2024-05-28T14:13:12.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa198200640 1 -- 172.21.15.42:0/1891572515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa190068a90 con 0x7fa1900afb20 2024-05-28T14:13:12.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.435+0000 7fa18d600640 1 -- 172.21.15.42:0/1891572515 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fa17c01e460 con 0x7fa1900afb20 2024-05-28T14:13:12.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.439+0000 7fa18d600640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa170069b50 0x7fa17006c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:12.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.439+0000 7fa18d600640 1 -- 172.21.15.42:0/1891572515 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7fa17c085ed0 con 0x7fa1900afb20 2024-05-28T14:13:12.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.439+0000 7fa195800640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa170069b50 0x7fa17006c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:12.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.439+0000 7fa195800640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa170069b50 0x7fa17006c010 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa1901fae60 tx=0x7fa184009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:12.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.443+0000 7fa18d600640 1 -- 172.21.15.42:0/1891572515 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa17c057ef0 con 0x7fa1900afb20 2024-05-28T14:13:12.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:12.643+0000 7fa198200640 1 -- 172.21.15.42:0/1891572515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fa19015cb40 con 0x7fa1900afb20 2024-05-28T14:13:13.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.195+0000 7fa18d600640 1 -- 172.21.15.42:0/1891572515 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v75) ==== 136+0+0 (secure 0 0 0) 0x7fa17c05e0c0 con 0x7fa1900afb20 2024-05-28T14:13:13.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.195+0000 7fa198200640 1 -- 172.21.15.42:0/1891572515 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa170069b50 msgr2=0x7fa17006c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.195+0000 7fa198200640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa170069b50 0x7fa17006c010 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa1901fae60 tx=0x7fa184009040 comp rx=0 tx=0).stop 2024-05-28T14:13:13.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.195+0000 7fa198200640 1 -- 172.21.15.42:0/1891572515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1900afb20 msgr2=0x7fa190151f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.195+0000 7fa198200640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1900afb20 0x7fa190151f10 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00d830 tx=0x7fa17c00dcf0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.195+0000 7fa198200640 1 -- 172.21.15.42:0/1891572515 shutdown_connections 2024-05-28T14:13:13.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.195+0000 7fa198200640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa170069b50 0x7fa17006c010 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.195+0000 7fa198200640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1900b0070 0x7fa19015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.195+0000 7fa198200640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1900afb20 0x7fa190151f10 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.195+0000 7fa198200640 1 --2- 172.21.15.42:0/1891572515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa19006cbd0 0x7fa19015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.195+0000 7fa198200640 1 -- 172.21.15.42:0/1891572515 >> 172.21.15.42:0/1891572515 conn(0x7fa1900725b0 msgr2=0x7fa19015cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:13.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.195+0000 7fa198200640 1 -- 172.21.15.42:0/1891572515 shutdown_connections 2024-05-28T14:13:13.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.195+0000 7fa198200640 1 -- 172.21.15.42:0/1891572515 wait complete. 2024-05-28T14:13:13.203 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:13:13.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18be00640 1 Processor -- start 2024-05-28T14:13:13.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18be00640 1 -- start start 2024-05-28T14:13:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18c06c520 0x7fa18c0d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18be00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa18c160420 con 0x7fa18c15c530 2024-05-28T14:13:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18be00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa18c0b2650 con 0x7fa18c15c910 2024-05-28T14:13:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18be00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa18c0b2930 con 0x7fa18c06c520 2024-05-28T14:13:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18aa00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa18c15c530 0x7fa18c0b1e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:34070/0 (socket says 172.21.15.42:34070) 2024-05-28T14:13:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18aa00640 1 -- 172.21.15.42:0/799411760 learned_addr learned my addr 172.21.15.42:0/799411760 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:13:13.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18b400640 1 --2- 172.21.15.42:0/799411760 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18c06c520 0x7fa18c0d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 -- 172.21.15.42:0/799411760 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2259513355 0 0) 0x7fa18c160420 con 0x7fa18c15c530 2024-05-28T14:13:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 -- 172.21.15.42:0/799411760 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa170003530 con 0x7fa18c15c530 2024-05-28T14:13:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 -- 172.21.15.42:0/799411760 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2303833822 0 0) 0x7fa170003530 con 0x7fa18c15c530 2024-05-28T14:13:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 -- 172.21.15.42:0/799411760 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18c06c520 msgr2=0x7fa18c0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 --2- 172.21.15.42:0/799411760 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18c06c520 0x7fa18c0d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 -- 172.21.15.42:0/799411760 >> v1:172.21.15.63:6789/0 conn(0x7fa18c15c910 legacy=0x7fa18c160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 -- 172.21.15.42:0/799411760 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18c1606e0 con 0x7fa18c15c530 2024-05-28T14:13:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 -- 172.21.15.42:0/799411760 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fa17c0016a0 con 0x7fa18c15c530 2024-05-28T14:13:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 -- 172.21.15.42:0/799411760 >> v1:172.21.15.42:6789/0 conn(0x7fa18c15c530 legacy=0x7fa18c0b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 --2- 172.21.15.42:0/799411760 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18c06c520 0x7fa1700039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:13.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 --2- 172.21.15.42:0/799411760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa170003f70 0x7fa170004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 --2- 172.21.15.42:0/799411760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa170004d50 0x7fa1700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 -- 172.21.15.42:0/799411760 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c0b2930 con 0x7fa170004d50 2024-05-28T14:13:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 -- 172.21.15.42:0/799411760 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa17000bab0 con 0x7fa170003f70 2024-05-28T14:13:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18a000640 1 -- 172.21.15.42:0/799411760 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa17000bcb0 con 0x7fa18c06c520 2024-05-28T14:13:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18b400640 1 --2- 172.21.15.42:0/799411760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa170004d50 0x7fa1700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18aa00640 1 --2- 172.21.15.42:0/799411760 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18c06c520 0x7fa1700039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa183e00640 1 --2- 172.21.15.42:0/799411760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa170003f70 0x7fa170004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18b400640 1 -- 172.21.15.42:0/799411760 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18c06c520 msgr2=0x7fa1700039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18b400640 1 --2- 172.21.15.42:0/799411760 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18c06c520 0x7fa1700039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18b400640 1 -- 172.21.15.42:0/799411760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa170003f70 msgr2=0x7fa170004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:13.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18b400640 1 --2- 172.21.15.42:0/799411760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa170003f70 0x7fa170004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.367+0000 7fa18b400640 1 -- 172.21.15.42:0/799411760 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa170003530 con 0x7fa170004d50 2024-05-28T14:13:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.371+0000 7fa18b400640 1 --2- 172.21.15.42:0/799411760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa170004d50 0x7fa1700051d0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fa184009fc0 tx=0x7fa18401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c97f65946fdc3d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.371+0000 7fa18a000640 1 -- 172.21.15.42:0/799411760 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1840b2070 con 0x7fa170004d50 2024-05-28T14:13:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.371+0000 7fa18a000640 1 -- 172.21.15.42:0/799411760 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1840a5c90 con 0x7fa170004d50 2024-05-28T14:13:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.371+0000 7fa18a000640 1 -- 172.21.15.42:0/799411760 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184009c10 con 0x7fa170004d50 2024-05-28T14:13:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.371+0000 7fa18be00640 1 -- 172.21.15.42:0/799411760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa170004d50 msgr2=0x7fa1700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.371+0000 7fa18be00640 1 --2- 172.21.15.42:0/799411760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa170004d50 0x7fa1700051d0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fa184009fc0 tx=0x7fa18401cc80 comp rx=0 tx=0).stop 2024-05-28T14:13:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.371+0000 7fa18b400640 1 -- 172.21.15.42:0/799411760 reap_dead start 2024-05-28T14:13:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.371+0000 7fa18be00640 1 -- 172.21.15.42:0/799411760 shutdown_connections 2024-05-28T14:13:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.371+0000 7fa18be00640 1 -- 172.21.15.42:0/799411760 >> 172.21.15.42:0/799411760 conn(0x7fa18c09aa50 msgr2=0x7fa18c0b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:13.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.371+0000 7fa18be00640 1 -- 172.21.15.42:0/799411760 shutdown_connections 2024-05-28T14:13:13.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.371+0000 7fa18be00640 1 -- 172.21.15.42:0/799411760 wait complete. 2024-05-28T14:13:13.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa18be00640 1 Processor -- start 2024-05-28T14:13:13.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa18be00640 1 -- start start 2024-05-28T14:13:13.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa18be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa170004d50 0x7fa18c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:13.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa18be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa18c1563b0 0x7fa18c15f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:13.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa18be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18c1f7b20 0x7fa18c15f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:13.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa18be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c0d0810 con 0x7fa18c1563b0 2024-05-28T14:13:13.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa18be00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c0d0690 con 0x7fa170004d50 2024-05-28T14:13:13.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa18be00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa18c0d0990 con 0x7fa18c1f7b20 2024-05-28T14:13:13.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa183e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa18c1563b0 0x7fa18c15f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:13.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa183e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa18c1563b0 0x7fa18c15f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39598/0 (socket says 172.21.15.42:39598) 2024-05-28T14:13:13.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa183e00640 1 -- 172.21.15.42:0/4230731619 learned_addr learned my addr 172.21.15.42:0/4230731619 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:13.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa18aa00640 1 --2- 172.21.15.42:0/4230731619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa170004d50 0x7fa18c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:13.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa18b400640 1 --2- 172.21.15.42:0/4230731619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18c1f7b20 0x7fa18c15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:13.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa183e00640 1 -- 172.21.15.42:0/4230731619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18c1f7b20 msgr2=0x7fa18c15f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa183e00640 1 --2- 172.21.15.42:0/4230731619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18c1f7b20 0x7fa18c15f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa183e00640 1 -- 172.21.15.42:0/4230731619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa170004d50 msgr2=0x7fa18c0b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa183e00640 1 --2- 172.21.15.42:0/4230731619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa170004d50 0x7fa18c0b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa183e00640 1 -- 172.21.15.42:0/4230731619 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18c1520c0 con 0x7fa18c1563b0 2024-05-28T14:13:13.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa183e00640 1 --2- 172.21.15.42:0/4230731619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa18c1563b0 0x7fa18c15f110 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fa17400f820 tx=0x7fa17400fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:13.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa182a00640 1 -- 172.21.15.42:0/4230731619 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1740050d0 con 0x7fa18c1563b0 2024-05-28T14:13:13.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa182a00640 1 -- 172.21.15.42:0/4230731619 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa174018070 con 0x7fa18c1563b0 2024-05-28T14:13:13.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa18be00640 1 -- 172.21.15.42:0/4230731619 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa18c1523a0 con 0x7fa18c1563b0 2024-05-28T14:13:13.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa182a00640 1 -- 172.21.15.42:0/4230731619 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17400d730 con 0x7fa18c1563b0 2024-05-28T14:13:13.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.375+0000 7fa18be00640 1 -- 172.21.15.42:0/4230731619 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa18c152960 con 0x7fa18c1563b0 2024-05-28T14:13:13.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.379+0000 7fa18be00640 1 -- 172.21.15.42:0/4230731619 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa18c04fcf0 con 0x7fa18c1563b0 2024-05-28T14:13:13.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.379+0000 7fa182a00640 1 -- 172.21.15.42:0/4230731619 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fa17400d8d0 con 0x7fa18c1563b0 2024-05-28T14:13:13.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.379+0000 7fa182a00640 1 --2- 172.21.15.42:0/4230731619 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa168069b20 0x7fa16806bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:13.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.379+0000 7fa182a00640 1 -- 172.21.15.42:0/4230731619 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7fa174085450 con 0x7fa18c1563b0 2024-05-28T14:13:13.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.379+0000 7fa18aa00640 1 --2- 172.21.15.42:0/4230731619 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa168069b20 0x7fa16806bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:13.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.379+0000 7fa18aa00640 1 --2- 172.21.15.42:0/4230731619 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa168069b20 0x7fa16806bfe0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0b3db0 tx=0x7fa17c007900 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:13.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.383+0000 7fa182a00640 1 -- 172.21.15.42:0/4230731619 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa1740574f0 con 0x7fa18c1563b0 2024-05-28T14:13:13.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.575+0000 7fa18be00640 1 -- 172.21.15.42:0/4230731619 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa18c151e70 con 0x7fa168069b20 2024-05-28T14:13:13.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.575+0000 7fa182a00640 1 -- 172.21.15.42:0/4230731619 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa18c151e70 con 0x7fa168069b20 2024-05-28T14:13:13.561 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:13:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:13:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:13:13.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:13:13.561 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:13:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.579+0000 7fa18be00640 1 -- 172.21.15.42:0/4230731619 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa168069b20 msgr2=0x7fa16806bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.579+0000 7fa18be00640 1 --2- 172.21.15.42:0/4230731619 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa168069b20 0x7fa16806bfe0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0b3db0 tx=0x7fa17c007900 comp rx=0 tx=0).stop 2024-05-28T14:13:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.579+0000 7fa18be00640 1 -- 172.21.15.42:0/4230731619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa18c1563b0 msgr2=0x7fa18c15f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.579+0000 7fa18be00640 1 --2- 172.21.15.42:0/4230731619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa18c1563b0 0x7fa18c15f110 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fa17400f820 tx=0x7fa17400fc20 comp rx=0 tx=0).stop 2024-05-28T14:13:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.579+0000 7fa18be00640 1 -- 172.21.15.42:0/4230731619 shutdown_connections 2024-05-28T14:13:13.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.579+0000 7fa18be00640 1 --2- 172.21.15.42:0/4230731619 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa168069b20 0x7fa16806bfe0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.579+0000 7fa18be00640 1 --2- 172.21.15.42:0/4230731619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18c1f7b20 0x7fa18c15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.579+0000 7fa18be00640 1 --2- 172.21.15.42:0/4230731619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa18c1563b0 0x7fa18c15f110 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.579+0000 7fa18be00640 1 --2- 172.21.15.42:0/4230731619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa170004d50 0x7fa18c0b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.579+0000 7fa18be00640 1 -- 172.21.15.42:0/4230731619 >> 172.21.15.42:0/4230731619 conn(0x7fa18c09aa50 msgr2=0x7fa18c06fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:13.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.579+0000 7fa18be00640 1 -- 172.21.15.42:0/4230731619 shutdown_connections 2024-05-28T14:13:13.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.579+0000 7fa18be00640 1 -- 172.21.15.42:0/4230731619 wait complete. 2024-05-28T14:13:13.582 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T14:13:13.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.739+0000 7fe53f200640 1 Processor -- start 2024-05-28T14:13:13.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.739+0000 7fe53f200640 1 -- start start 2024-05-28T14:13:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.743+0000 7fe53f200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe54015ac00 0x7fe5400cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.743+0000 7fe53f200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe5400af410 0x7fe54006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.743+0000 7fe53f200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5400cf7b0 con 0x7fe54006f290 2024-05-28T14:13:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.743+0000 7fe53f200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fe54006f700 con 0x7fe5400af410 2024-05-28T14:13:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.743+0000 7fe53f200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe5400cfa20 con 0x7fe54015ac00 2024-05-28T14:13:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.743+0000 7fe537e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe5400af410 0x7fe54006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.743+0000 7fe537e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe5400af410 0x7fe54006ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39606/0 (socket says 172.21.15.42:39606) 2024-05-28T14:13:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.743+0000 7fe537e00640 1 -- 172.21.15.42:0/4133452072 learned_addr learned my addr 172.21.15.42:0/4133452072 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.743+0000 7fe53de00640 1 --2- 172.21.15.42:0/4133452072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe54015ac00 0x7fe5400cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.743+0000 7fe537e00640 1 -- 172.21.15.42:0/4133452072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe54015ac00 msgr2=0x7fe5400cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.743+0000 7fe537e00640 1 --2- 172.21.15.42:0/4133452072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe54015ac00 0x7fe5400cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.743+0000 7fe537e00640 1 -- 172.21.15.42:0/4133452072 >> v1:172.21.15.63:6789/0 conn(0x7fe54006f290 legacy=0x7fe54006db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.743+0000 7fe537e00640 1 -- 172.21.15.42:0/4133452072 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5400cfd00 con 0x7fe5400af410 2024-05-28T14:13:13.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.747+0000 7fe537e00640 1 --2- 172.21.15.42:0/4133452072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5400af410 0x7fe54006ed50 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fe528009870 tx=0x7fe52801e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7de00924079eba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:13.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.747+0000 7fe53d400640 1 -- 172.21.15.42:0/4133452072 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5280a9070 con 0x7fe5400af410 2024-05-28T14:13:13.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.747+0000 7fe53d400640 1 -- 172.21.15.42:0/4133452072 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5280b2070 con 0x7fe5400af410 2024-05-28T14:13:13.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.747+0000 7fe53d400640 1 -- 172.21.15.42:0/4133452072 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5280ad890 con 0x7fe5400af410 2024-05-28T14:13:13.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.747+0000 7fe53f200640 1 -- 172.21.15.42:0/4133452072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5400af410 msgr2=0x7fe54006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.747+0000 7fe53f200640 1 --2- 172.21.15.42:0/4133452072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5400af410 0x7fe54006ed50 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fe528009870 tx=0x7fe52801e6e0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.747+0000 7fe53f200640 1 -- 172.21.15.42:0/4133452072 shutdown_connections 2024-05-28T14:13:13.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.747+0000 7fe53f200640 1 --2- 172.21.15.42:0/4133452072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5400af410 0x7fe54006ed50 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.747+0000 7fe53f200640 1 --2- 172.21.15.42:0/4133452072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe54015ac00 0x7fe5400cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.747+0000 7fe53f200640 1 -- 172.21.15.42:0/4133452072 >> 172.21.15.42:0/4133452072 conn(0x7fe54006bbc0 msgr2=0x7fe54006bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:13.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.747+0000 7fe53f200640 1 -- 172.21.15.42:0/4133452072 shutdown_connections 2024-05-28T14:13:13.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.747+0000 7fe53f200640 1 -- 172.21.15.42:0/4133452072 wait complete. 2024-05-28T14:13:13.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.747+0000 7fe53f200640 1 Processor -- start 2024-05-28T14:13:13.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.747+0000 7fe53f200640 1 -- start start 2024-05-28T14:13:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe54006f290 0x7fe5401478a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53f200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5400af410 0x7fe540154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54015ac00 0x7fe5401531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5400cdff0 con 0x7fe54015ac00 2024-05-28T14:13:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53f200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe5400b3340 con 0x7fe54006f290 2024-05-28T14:13:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53f200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe5400ce170 con 0x7fe5400af410 2024-05-28T14:13:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54015ac00 0x7fe5401531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54015ac00 0x7fe5401531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39616/0 (socket says 172.21.15.42:39616) 2024-05-28T14:13:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53e800640 1 -- 172.21.15.42:0/487647644 learned_addr learned my addr 172.21.15.42:0/487647644 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53e800640 1 -- 172.21.15.42:0/487647644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5400af410 msgr2=0x7fe540154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53e800640 1 --2- 172.21.15.42:0/487647644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5400af410 0x7fe540154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53e800640 1 -- 172.21.15.42:0/487647644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe54006f290 msgr2=0x7fe5401478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53de00640 1 --2- 172.21.15.42:0/487647644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe54006f290 0x7fe5401478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53e800640 1 --2- 172.21.15.42:0/487647644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe54006f290 0x7fe5401478a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53e800640 1 -- 172.21.15.42:0/487647644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe540150d90 con 0x7fe54015ac00 2024-05-28T14:13:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53de00640 1 --2- 172.21.15.42:0/487647644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe54006f290 0x7fe5401478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:13:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53e800640 1 --2- 172.21.15.42:0/487647644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54015ac00 0x7fe5401531b0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c7a0 tx=0x7fe53800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe536000640 1 -- 172.21.15.42:0/487647644 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538003b80 con 0x7fe54015ac00 2024-05-28T14:13:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe536000640 1 -- 172.21.15.42:0/487647644 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe538003d20 con 0x7fe54015ac00 2024-05-28T14:13:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53f200640 1 -- 172.21.15.42:0/487647644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe540151070 con 0x7fe54015ac00 2024-05-28T14:13:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe536000640 1 -- 172.21.15.42:0/487647644 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53800aa90 con 0x7fe54015ac00 2024-05-28T14:13:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53f200640 1 -- 172.21.15.42:0/487647644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe540151630 con 0x7fe54015ac00 2024-05-28T14:13:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.751+0000 7fe53f200640 1 -- 172.21.15.42:0/487647644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe54004fcf0 con 0x7fe54015ac00 2024-05-28T14:13:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.755+0000 7fe536000640 1 -- 172.21.15.42:0/487647644 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fe53800ac30 con 0x7fe54015ac00 2024-05-28T14:13:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.755+0000 7fe536000640 1 --2- 172.21.15.42:0/487647644 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe51c069b20 0x7fe51c06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.755+0000 7fe536000640 1 -- 172.21.15.42:0/487647644 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7fe538085740 con 0x7fe54015ac00 2024-05-28T14:13:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.755+0000 7fe53de00640 1 --2- 172.21.15.42:0/487647644 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe51c069b20 0x7fe51c06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:13.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.755+0000 7fe53de00640 1 --2- 172.21.15.42:0/487647644 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe51c069b20 0x7fe51c06bfe0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe530005fd0 tx=0x7fe530009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.759+0000 7fe536000640 1 -- 172.21.15.42:0/487647644 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe538057760 con 0x7fe54015ac00 2024-05-28T14:13:13.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.927+0000 7fe53f200640 1 -- 172.21.15.42:0/487647644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe5400cdcb0 con 0x7fe54015ac00 2024-05-28T14:13:13.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.927+0000 7fe536000640 1 -- 172.21.15.42:0/487647644 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fe538057060 con 0x7fe54015ac00 2024-05-28T14:13:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.931+0000 7fe53f200640 1 -- 172.21.15.42:0/487647644 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe51c069b20 msgr2=0x7fe51c06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.931+0000 7fe53f200640 1 --2- 172.21.15.42:0/487647644 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe51c069b20 0x7fe51c06bfe0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe530005fd0 tx=0x7fe530009040 comp rx=0 tx=0).stop 2024-05-28T14:13:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.931+0000 7fe53f200640 1 -- 172.21.15.42:0/487647644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54015ac00 msgr2=0x7fe5401531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.931+0000 7fe53f200640 1 --2- 172.21.15.42:0/487647644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54015ac00 0x7fe5401531b0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c7a0 tx=0x7fe53800cc60 comp rx=0 tx=0).stop 2024-05-28T14:13:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.931+0000 7fe53f200640 1 -- 172.21.15.42:0/487647644 shutdown_connections 2024-05-28T14:13:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.931+0000 7fe53f200640 1 --2- 172.21.15.42:0/487647644 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe51c069b20 0x7fe51c06bfe0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.931+0000 7fe53f200640 1 --2- 172.21.15.42:0/487647644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54015ac00 0x7fe5401531b0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.931+0000 7fe53f200640 1 --2- 172.21.15.42:0/487647644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5400af410 0x7fe540154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.931+0000 7fe53f200640 1 --2- 172.21.15.42:0/487647644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe54006f290 0x7fe5401478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.931+0000 7fe53f200640 1 -- 172.21.15.42:0/487647644 >> 172.21.15.42:0/487647644 conn(0x7fe54006bbc0 msgr2=0x7fe5400afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.931+0000 7fe53f200640 1 -- 172.21.15.42:0/487647644 shutdown_connections 2024-05-28T14:13:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:13.931+0000 7fe53f200640 1 -- 172.21.15.42:0/487647644 wait complete. 2024-05-28T14:13:13.935 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-28T14:13:14.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.095+0000 7f5a1aa00640 1 Processor -- start 2024-05-28T14:13:14.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.095+0000 7f5a1aa00640 1 -- start start 2024-05-28T14:13:14.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a1aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a140ad680 0x7f5a1406e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a1aa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5a1406ebd0 0x7f5a140b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a1aa00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a1406dff0 con 0x7f5a1415aab0 2024-05-28T14:13:14.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a1aa00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5a14074050 con 0x7f5a1406ebd0 2024-05-28T14:13:14.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a1aa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1406f060 con 0x7f5a140ad680 2024-05-28T14:13:14.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a18a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5a1406ebd0 0x7f5a140b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a18a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5a1406ebd0 0x7f5a140b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39624/0 (socket says 172.21.15.42:39624) 2024-05-28T14:13:14.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a18a00640 1 -- 172.21.15.42:0/722675749 learned_addr learned my addr 172.21.15.42:0/722675749 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:14.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a0b400640 1 --2- 172.21.15.42:0/722675749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a140ad680 0x7f5a1406e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a18a00640 1 -- 172.21.15.42:0/722675749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a140ad680 msgr2=0x7f5a1406e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:14.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a18a00640 1 --2- 172.21.15.42:0/722675749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a140ad680 0x7f5a1406e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a18a00640 1 -- 172.21.15.42:0/722675749 >> v1:172.21.15.42:6789/0 conn(0x7f5a1415aab0 legacy=0x7f5a14073220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a18a00640 1 -- 172.21.15.42:0/722675749 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a14073d70 con 0x7f5a1406ebd0 2024-05-28T14:13:14.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a18a00640 1 --2- 172.21.15.42:0/722675749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a1406ebd0 0x7f5a140b3df0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00bb40 tx=0x7f5a0c01eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ac255b66e3127 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:14.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.099+0000 7f5a13400640 1 -- 172.21.15.42:0/722675749 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0b4070 con 0x7f5a1406ebd0 2024-05-28T14:13:14.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a13400640 1 -- 172.21.15.42:0/722675749 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0c0a7ac0 con 0x7f5a1406ebd0 2024-05-28T14:13:14.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a13400640 1 -- 172.21.15.42:0/722675749 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0af440 con 0x7f5a1406ebd0 2024-05-28T14:13:14.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 -- 172.21.15.42:0/722675749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a1406ebd0 msgr2=0x7f5a140b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 --2- 172.21.15.42:0/722675749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a1406ebd0 0x7f5a140b3df0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00bb40 tx=0x7f5a0c01eb80 comp rx=0 tx=0).stop 2024-05-28T14:13:14.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 -- 172.21.15.42:0/722675749 shutdown_connections 2024-05-28T14:13:14.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 --2- 172.21.15.42:0/722675749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a1406ebd0 0x7f5a140b3df0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 --2- 172.21.15.42:0/722675749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a140ad680 0x7f5a1406e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 -- 172.21.15.42:0/722675749 >> 172.21.15.42:0/722675749 conn(0x7f5a1406bdf0 msgr2=0x7f5a1406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:14.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 -- 172.21.15.42:0/722675749 shutdown_connections 2024-05-28T14:13:14.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 -- 172.21.15.42:0/722675749 wait complete. 2024-05-28T14:13:14.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 Processor -- start 2024-05-28T14:13:14.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 -- start start 2024-05-28T14:13:14.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a1406ebd0 0x7f5a141f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a140ad680 0x7f5a141f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a1415aab0 0x7f5a14147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5a14073930 con 0x7f5a1406ebd0 2024-05-28T14:13:14.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1415ae90 con 0x7f5a1415aab0 2024-05-28T14:13:14.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a1aa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5a14073ab0 con 0x7f5a140ad680 2024-05-28T14:13:14.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a13e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a1406ebd0 0x7f5a141f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a13e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a1406ebd0 0x7f5a141f6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39638/0 (socket says 172.21.15.42:39638) 2024-05-28T14:13:14.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a13e00640 1 -- 172.21.15.42:0/2979967180 learned_addr learned my addr 172.21.15.42:0/2979967180 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:14.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a0b400640 1 --2- 172.21.15.42:0/2979967180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a140ad680 0x7f5a141f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.103+0000 7f5a18a00640 1 --2- 172.21.15.42:0/2979967180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a1415aab0 0x7f5a14147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a13e00640 1 -- 172.21.15.42:0/2979967180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a140ad680 msgr2=0x7f5a141f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a13e00640 1 --2- 172.21.15.42:0/2979967180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a140ad680 0x7f5a141f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a13e00640 1 -- 172.21.15.42:0/2979967180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a1415aab0 msgr2=0x7f5a14147ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a13e00640 1 --2- 172.21.15.42:0/2979967180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a1415aab0 0x7f5a14147ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a13e00640 1 -- 172.21.15.42:0/2979967180 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a140b1030 con 0x7f5a1406ebd0 2024-05-28T14:13:14.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a13e00640 1 --2- 172.21.15.42:0/2979967180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a1406ebd0 0x7f5a141f6ce0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c760 tx=0x7f5a0400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:14.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a10c00640 1 -- 172.21.15.42:0/2979967180 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04016020 con 0x7f5a1406ebd0 2024-05-28T14:13:14.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a10c00640 1 -- 172.21.15.42:0/2979967180 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a04006d60 con 0x7f5a1406ebd0 2024-05-28T14:13:14.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a10c00640 1 -- 172.21.15.42:0/2979967180 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04003bc0 con 0x7f5a1406ebd0 2024-05-28T14:13:14.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a1aa00640 1 -- 172.21.15.42:0/2979967180 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a140b1310 con 0x7f5a1406ebd0 2024-05-28T14:13:14.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a1aa00640 1 -- 172.21.15.42:0/2979967180 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a140b18d0 con 0x7f5a1406ebd0 2024-05-28T14:13:14.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a1aa00640 1 -- 172.21.15.42:0/2979967180 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a14068a90 con 0x7f5a1406ebd0 2024-05-28T14:13:14.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a10c00640 1 -- 172.21.15.42:0/2979967180 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f5a0400a850 con 0x7f5a1406ebd0 2024-05-28T14:13:14.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a10c00640 1 --2- 172.21.15.42:0/2979967180 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59f0069b50 0x7f59f006c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a10c00640 1 -- 172.21.15.42:0/2979967180 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f5a040854b0 con 0x7f5a1406ebd0 2024-05-28T14:13:14.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.107+0000 7f5a0b400640 1 --2- 172.21.15.42:0/2979967180 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59f0069b50 0x7f59f006c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.111+0000 7f5a0b400640 1 --2- 172.21.15.42:0/2979967180 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59f0069b50 0x7f59f006c010 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5a14146580 tx=0x7f59fc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:14.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.111+0000 7f5a10c00640 1 -- 172.21.15.42:0/2979967180 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5a04020020 con 0x7f5a1406ebd0 2024-05-28T14:13:14.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.307+0000 7f5a1aa00640 1 -- 172.21.15.42:0/2979967180 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f5a14072330 con 0x7f59f0069b50 2024-05-28T14:13:14.299 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:14.315+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:14.299 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:14.315+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:14.300 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:14.315+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:14.300 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:14.315+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:14.300 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:14.315+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:14.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.351+0000 7f5a10c00640 1 -- 172.21.15.42:0/2979967180 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5a14072330 con 0x7f59f0069b50 2024-05-28T14:13:14.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.351+0000 7f5a1aa00640 1 -- 172.21.15.42:0/2979967180 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59f0069b50 msgr2=0x7f59f006c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.351+0000 7f5a1aa00640 1 --2- 172.21.15.42:0/2979967180 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59f0069b50 0x7f59f006c010 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5a14146580 tx=0x7f59fc008040 comp rx=0 tx=0).stop 2024-05-28T14:13:14.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.351+0000 7f5a1aa00640 1 -- 172.21.15.42:0/2979967180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a1406ebd0 msgr2=0x7f5a141f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.351+0000 7f5a1aa00640 1 --2- 172.21.15.42:0/2979967180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a1406ebd0 0x7f5a141f6ce0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c760 tx=0x7f5a0400cc20 comp rx=0 tx=0).stop 2024-05-28T14:13:14.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.351+0000 7f5a1aa00640 1 -- 172.21.15.42:0/2979967180 shutdown_connections 2024-05-28T14:13:14.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.351+0000 7f5a1aa00640 1 --2- 172.21.15.42:0/2979967180 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f59f0069b50 0x7f59f006c010 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.351+0000 7f5a1aa00640 1 --2- 172.21.15.42:0/2979967180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a1415aab0 0x7f5a14147ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.351+0000 7f5a1aa00640 1 --2- 172.21.15.42:0/2979967180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a140ad680 0x7f5a141f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.351+0000 7f5a1aa00640 1 --2- 172.21.15.42:0/2979967180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a1406ebd0 0x7f5a141f6ce0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.351+0000 7f5a1aa00640 1 -- 172.21.15.42:0/2979967180 >> 172.21.15.42:0/2979967180 conn(0x7f5a1406bdf0 msgr2=0x7f5a14159770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:14.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.351+0000 7f5a1aa00640 1 -- 172.21.15.42:0/2979967180 shutdown_connections 2024-05-28T14:13:14.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.351+0000 7f5a1aa00640 1 -- 172.21.15.42:0/2979967180 wait complete. 2024-05-28T14:13:14.358 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:13:14.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.515+0000 7fc419400640 1 Processor -- start 2024-05-28T14:13:14.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.515+0000 7fc419400640 1 -- start start 2024-05-28T14:13:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc419400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc414071de0 0x7fc4140721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc419400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc41413fd10 0x7fc414140160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc419400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc414072700 con 0x7fc4140ce3b0 2024-05-28T14:13:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc419400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc414160230 con 0x7fc414071de0 2024-05-28T14:13:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc419400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc414160510 con 0x7fc41413fd10 2024-05-28T14:13:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc412800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc414071de0 0x7fc4140721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc412800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc414071de0 0x7fc4140721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39678/0 (socket says 172.21.15.42:39678) 2024-05-28T14:13:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc412800640 1 -- 172.21.15.42:0/1285107632 learned_addr learned my addr 172.21.15.42:0/1285107632 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc413200640 1 --2- 172.21.15.42:0/1285107632 >> v2:172.21.15.63:3300/0 conn(0x7fc41413fd10 0x7fc414140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc412800640 1 -- 172.21.15.42:0/1285107632 >> v2:172.21.15.63:3300/0 conn(0x7fc41413fd10 msgr2=0x7fc414140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc412800640 1 --2- 172.21.15.42:0/1285107632 >> v2:172.21.15.63:3300/0 conn(0x7fc41413fd10 0x7fc414140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc412800640 1 -- 172.21.15.42:0/1285107632 >> v1:172.21.15.42:6789/0 conn(0x7fc4140ce3b0 legacy=0x7fc4140ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc412800640 1 -- 172.21.15.42:0/1285107632 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4141607f0 con 0x7fc414071de0 2024-05-28T14:13:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc412800640 1 --2- 172.21.15.42:0/1285107632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc414071de0 0x7fc4140721c0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fc4140b2180 tx=0x7fc40401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a399e8270cd9d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.519+0000 7fc411e00640 1 -- 172.21.15.42:0/1285107632 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4040a5590 con 0x7fc414071de0 2024-05-28T14:13:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc411e00640 1 -- 172.21.15.42:0/1285107632 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4040a5730 con 0x7fc414071de0 2024-05-28T14:13:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc411e00640 1 -- 172.21.15.42:0/1285107632 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4040a5b00 con 0x7fc414071de0 2024-05-28T14:13:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 -- 172.21.15.42:0/1285107632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc414071de0 msgr2=0x7fc4140721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 --2- 172.21.15.42:0/1285107632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc414071de0 0x7fc4140721c0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fc4140b2180 tx=0x7fc40401c8d0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 -- 172.21.15.42:0/1285107632 shutdown_connections 2024-05-28T14:13:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 --2- 172.21.15.42:0/1285107632 >> v2:172.21.15.63:3300/0 conn(0x7fc41413fd10 0x7fc414140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 --2- 172.21.15.42:0/1285107632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc414071de0 0x7fc4140721c0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 -- 172.21.15.42:0/1285107632 >> 172.21.15.42:0/1285107632 conn(0x7fc4140b21f0 msgr2=0x7fc4140b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 -- 172.21.15.42:0/1285107632 shutdown_connections 2024-05-28T14:13:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 -- 172.21.15.42:0/1285107632 wait complete. 2024-05-28T14:13:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 Processor -- start 2024-05-28T14:13:14.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 -- start start 2024-05-28T14:13:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc414071de0 0x7fc414156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4140ce3b0 0x7fc414138460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc41413fd10 0x7fc4141455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc414169830 con 0x7fc4140ce3b0 2024-05-28T14:13:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc4141696b0 con 0x7fc41413fd10 2024-05-28T14:13:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc419400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc4141699b0 con 0x7fc414071de0 2024-05-28T14:13:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc409e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4140ce3b0 0x7fc414138460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc409e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4140ce3b0 0x7fc414138460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39688/0 (socket says 172.21.15.42:39688) 2024-05-28T14:13:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc409e00640 1 -- 172.21.15.42:0/1667742182 learned_addr learned my addr 172.21.15.42:0/1667742182 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc413200640 1 --2- 172.21.15.42:0/1667742182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc41413fd10 0x7fc4141455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.523+0000 7fc412800640 1 --2- 172.21.15.42:0/1667742182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc414071de0 0x7fc414156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc409e00640 1 -- 172.21.15.42:0/1667742182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc414071de0 msgr2=0x7fc414156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc409e00640 1 --2- 172.21.15.42:0/1667742182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc414071de0 0x7fc414156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc409e00640 1 -- 172.21.15.42:0/1667742182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc41413fd10 msgr2=0x7fc4141455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc409e00640 1 --2- 172.21.15.42:0/1667742182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc41413fd10 0x7fc4141455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc409e00640 1 -- 172.21.15.42:0/1667742182 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4141565b0 con 0x7fc4140ce3b0 2024-05-28T14:13:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc409e00640 1 --2- 172.21.15.42:0/1667742182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4140ce3b0 0x7fc414138460 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00c370 tx=0x7fc3fc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc40b400640 1 -- 172.21.15.42:0/1667742182 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0052d0 con 0x7fc4140ce3b0 2024-05-28T14:13:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc40b400640 1 -- 172.21.15.42:0/1667742182 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3fc005470 con 0x7fc4140ce3b0 2024-05-28T14:13:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc40b400640 1 -- 172.21.15.42:0/1667742182 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc007260 con 0x7fc4140ce3b0 2024-05-28T14:13:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc419400640 1 -- 172.21.15.42:0/1667742182 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41413af40 con 0x7fc4140ce3b0 2024-05-28T14:13:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc419400640 1 -- 172.21.15.42:0/1667742182 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc41413cb90 con 0x7fc4140ce3b0 2024-05-28T14:13:14.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc419400640 1 -- 172.21.15.42:0/1667742182 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4140738b0 con 0x7fc4140ce3b0 2024-05-28T14:13:14.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc40b400640 1 -- 172.21.15.42:0/1667742182 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7fc3fc007400 con 0x7fc4140ce3b0 2024-05-28T14:13:14.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc40b400640 1 --2- 172.21.15.42:0/1667742182 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc3f0069b50 0x7fc3f006c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc40b400640 1 -- 172.21.15.42:0/1667742182 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7fc3fc084c20 con 0x7fc4140ce3b0 2024-05-28T14:13:14.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.527+0000 7fc412800640 1 --2- 172.21.15.42:0/1667742182 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc3f0069b50 0x7fc3f006c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.531+0000 7fc412800640 1 --2- 172.21.15.42:0/1667742182 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc3f0069b50 0x7fc3f006c010 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc4040ae000 tx=0x7fc4040a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.531+0000 7fc40b400640 1 -- 172.21.15.42:0/1667742182 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc4141565b0 con 0x7fc4140ce3b0 2024-05-28T14:13:14.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.723+0000 7fc419400640 1 -- 172.21.15.42:0/1667742182 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fc41404fdd0 con 0x7fc3f0069b50 2024-05-28T14:13:14.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.727+0000 7fc40b400640 1 -- 172.21.15.42:0/1667742182 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fc41404fdd0 con 0x7fc3f0069b50 2024-05-28T14:13:14.714 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/c4222c14-e22c-4814-8efb-a3c1d580571a 2024-05-28T14:13:14.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.731+0000 7fc419400640 1 -- 172.21.15.42:0/1667742182 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc3f0069b50 msgr2=0x7fc3f006c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.731+0000 7fc419400640 1 --2- 172.21.15.42:0/1667742182 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc3f0069b50 0x7fc3f006c010 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc4040ae000 tx=0x7fc4040a7070 comp rx=0 tx=0).stop 2024-05-28T14:13:14.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.731+0000 7fc419400640 1 -- 172.21.15.42:0/1667742182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4140ce3b0 msgr2=0x7fc414138460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.731+0000 7fc419400640 1 --2- 172.21.15.42:0/1667742182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4140ce3b0 0x7fc414138460 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00c370 tx=0x7fc3fc00c830 comp rx=0 tx=0).stop 2024-05-28T14:13:14.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.731+0000 7fc419400640 1 -- 172.21.15.42:0/1667742182 shutdown_connections 2024-05-28T14:13:14.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.731+0000 7fc419400640 1 --2- 172.21.15.42:0/1667742182 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc3f0069b50 0x7fc3f006c010 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.731+0000 7fc419400640 1 --2- 172.21.15.42:0/1667742182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc41413fd10 0x7fc4141455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.731+0000 7fc419400640 1 --2- 172.21.15.42:0/1667742182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4140ce3b0 0x7fc414138460 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.731+0000 7fc419400640 1 --2- 172.21.15.42:0/1667742182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc414071de0 0x7fc414156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.731+0000 7fc419400640 1 -- 172.21.15.42:0/1667742182 >> 172.21.15.42:0/1667742182 conn(0x7fc4140b21f0 msgr2=0x7fc4140b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:14.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.731+0000 7fc419400640 1 -- 172.21.15.42:0/1667742182 shutdown_connections 2024-05-28T14:13:14.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.731+0000 7fc419400640 1 -- 172.21.15.42:0/1667742182 wait complete. 2024-05-28T14:13:14.735 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:13:14.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3275e00640 1 Processor -- start 2024-05-28T14:13:14.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3275e00640 1 -- start start 2024-05-28T14:13:14.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3275e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f327015c770 0x7f327009b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3275e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f327015cb50 0x7f32700d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3275e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f32700c8bc0 0x7f32700d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3275e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f32700c6720 con 0x7f327015cb50 2024-05-28T14:13:14.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3275e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f327009ba40 con 0x7f32700c8bc0 2024-05-28T14:13:14.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3275e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f32700c60d0 con 0x7f327015c770 2024-05-28T14:13:14.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3266000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f327015cb50 0x7f32700d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3266000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f327015cb50 0x7f32700d0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39698/0 (socket says 172.21.15.42:39698) 2024-05-28T14:13:14.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3266000640 1 -- 172.21.15.42:0/992667973 learned_addr learned my addr 172.21.15.42:0/992667973 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:14.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f326ea00640 1 --2- 172.21.15.42:0/992667973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f327015c770 0x7f327009b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f326f400640 1 --2- 172.21.15.42:0/992667973 >> v2:172.21.15.63:3300/0 conn(0x7f32700c8bc0 0x7f32700d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3266000640 1 -- 172.21.15.42:0/992667973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f327015c770 msgr2=0x7f327009b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:14.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3266000640 1 --2- 172.21.15.42:0/992667973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f327015c770 0x7f327009b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3266000640 1 -- 172.21.15.42:0/992667973 >> v2:172.21.15.63:3300/0 conn(0x7f32700c8bc0 msgr2=0x7f32700d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:14.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3266000640 1 --2- 172.21.15.42:0/992667973 >> v2:172.21.15.63:3300/0 conn(0x7f32700c8bc0 0x7f32700d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f3266000640 1 -- 172.21.15.42:0/992667973 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32700c63b0 con 0x7f327015cb50 2024-05-28T14:13:14.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.895+0000 7f326ea00640 1 --2- 172.21.15.42:0/992667973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f327015c770 0x7f327009b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:14.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.899+0000 7f3266000640 1 --2- 172.21.15.42:0/992667973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f327015cb50 0x7f32700d0ac0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f3270073c50 tx=0x7f325801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80940ed6eea1db6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:14.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.899+0000 7f326e000640 1 -- 172.21.15.42:0/992667973 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32580b5070 con 0x7f327015cb50 2024-05-28T14:13:14.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.899+0000 7f326e000640 1 -- 172.21.15.42:0/992667973 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32580a89d0 con 0x7f327015cb50 2024-05-28T14:13:14.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.899+0000 7f326e000640 1 -- 172.21.15.42:0/992667973 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32580a8da0 con 0x7f327015cb50 2024-05-28T14:13:14.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.899+0000 7f3275e00640 1 -- 172.21.15.42:0/992667973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f327015cb50 msgr2=0x7f32700d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.899+0000 7f3275e00640 1 --2- 172.21.15.42:0/992667973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f327015cb50 0x7f32700d0ac0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f3270073c50 tx=0x7f325801ed10 comp rx=0 tx=0).stop 2024-05-28T14:13:14.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.899+0000 7f3275e00640 1 -- 172.21.15.42:0/992667973 shutdown_connections 2024-05-28T14:13:14.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.899+0000 7f3275e00640 1 --2- 172.21.15.42:0/992667973 >> v2:172.21.15.63:3300/0 conn(0x7f32700c8bc0 0x7f32700d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.899+0000 7f3275e00640 1 --2- 172.21.15.42:0/992667973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f327015cb50 0x7f32700d0ac0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.899+0000 7f3275e00640 1 --2- 172.21.15.42:0/992667973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f327015c770 0x7f327009b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.899+0000 7f3275e00640 1 -- 172.21.15.42:0/992667973 >> 172.21.15.42:0/992667973 conn(0x7f32700b3fd0 msgr2=0x7f32700b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:14.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.899+0000 7f3275e00640 1 -- 172.21.15.42:0/992667973 shutdown_connections 2024-05-28T14:13:14.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.899+0000 7f3275e00640 1 -- 172.21.15.42:0/992667973 wait complete. 2024-05-28T14:13:14.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f3275e00640 1 Processor -- start 2024-05-28T14:13:14.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f3275e00640 1 -- start start 2024-05-28T14:13:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f3275e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32700c8bc0 0x7f32700c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f3275e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f327015c770 0x7f32700cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f3275e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f327015cb50 0x7f327015ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f3275e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f32700d02c0 con 0x7f32700c8bc0 2024-05-28T14:13:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f3275e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32700ce2c0 con 0x7f327015c770 2024-05-28T14:13:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f3275e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f32700d0440 con 0x7f327015cb50 2024-05-28T14:13:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f326ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32700c8bc0 0x7f32700c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f326ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32700c8bc0 0x7f32700c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39706/0 (socket says 172.21.15.42:39706) 2024-05-28T14:13:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f326ea00640 1 -- 172.21.15.42:0/1239556152 learned_addr learned my addr 172.21.15.42:0/1239556152 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f3266000640 1 --2- 172.21.15.42:0/1239556152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f327015c770 0x7f32700cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f326f400640 1 --2- 172.21.15.42:0/1239556152 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f327015cb50 0x7f327015ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f326ea00640 1 -- 172.21.15.42:0/1239556152 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f327015cb50 msgr2=0x7f327015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f326ea00640 1 --2- 172.21.15.42:0/1239556152 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f327015cb50 0x7f327015ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f326ea00640 1 -- 172.21.15.42:0/1239556152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f327015c770 msgr2=0x7f32700cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:14.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f326ea00640 1 --2- 172.21.15.42:0/1239556152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f327015c770 0x7f32700cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:14.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f326ea00640 1 -- 172.21.15.42:0/1239556152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32700cbb80 con 0x7f32700c8bc0 2024-05-28T14:13:14.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f326ea00640 1 --2- 172.21.15.42:0/1239556152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32700c8bc0 0x7f32700c9f70 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f326000e810 tx=0x7f326000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:14.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f3267e00640 1 -- 172.21.15.42:0/1239556152 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260017070 con 0x7f32700c8bc0 2024-05-28T14:13:14.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f3267e00640 1 -- 172.21.15.42:0/1239556152 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f326000c8d0 con 0x7f32700c8bc0 2024-05-28T14:13:14.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f3267e00640 1 -- 172.21.15.42:0/1239556152 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326000f040 con 0x7f32700c8bc0 2024-05-28T14:13:14.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f3275e00640 1 -- 172.21.15.42:0/1239556152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327015f190 con 0x7f32700c8bc0 2024-05-28T14:13:14.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.903+0000 7f3275e00640 1 -- 172.21.15.42:0/1239556152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32700ca880 con 0x7f32700c8bc0 2024-05-28T14:13:14.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.907+0000 7f3275e00640 1 -- 172.21.15.42:0/1239556152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32700a04d0 con 0x7f32700c8bc0 2024-05-28T14:13:14.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.907+0000 7f3267e00640 1 -- 172.21.15.42:0/1239556152 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 90129+0+0 (secure 0 0 0) 0x7f326001a070 con 0x7f32700c8bc0 2024-05-28T14:13:14.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.907+0000 7f3267e00640 1 --2- 172.21.15.42:0/1239556152 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f324c069b20 0x7f324c06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:14.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.907+0000 7f3267e00640 1 -- 172.21.15.42:0/1239556152 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f32600852e0 con 0x7f32700c8bc0 2024-05-28T14:13:14.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.907+0000 7f3266000640 1 --2- 172.21.15.42:0/1239556152 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f324c069b20 0x7f324c06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:14.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.907+0000 7f3266000640 1 --2- 172.21.15.42:0/1239556152 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f324c069b20 0x7f324c06bfe0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f32580b2000 tx=0x7f3258009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:14.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:14.911+0000 7f3267e00640 1 -- 172.21.15.42:0/1239556152 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3260056c30 con 0x7f32700c8bc0 2024-05-28T14:13:15.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:15.107+0000 7f3275e00640 1 -- 172.21.15.42:0/1239556152 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f32700ced40 con 0x7f324c069b20 2024-05-28T14:13:15.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:15.111+0000 7f3267e00640 1 -- 172.21.15.42:0/1239556152 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f32700ced40 con 0x7f324c069b20 2024-05-28T14:13:15.095 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/c4222c14-e22c-4814-8efb-a3c1d580571a 2024-05-28T14:13:15.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:15.111+0000 7f3275e00640 1 -- 172.21.15.42:0/1239556152 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f324c069b20 msgr2=0x7f324c06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:15.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:15.111+0000 7f3275e00640 1 --2- 172.21.15.42:0/1239556152 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f324c069b20 0x7f324c06bfe0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f32580b2000 tx=0x7f3258009040 comp rx=0 tx=0).stop 2024-05-28T14:13:15.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:15.111+0000 7f3275e00640 1 -- 172.21.15.42:0/1239556152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32700c8bc0 msgr2=0x7f32700c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:15.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:15.111+0000 7f3275e00640 1 --2- 172.21.15.42:0/1239556152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32700c8bc0 0x7f32700c9f70 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f326000e810 tx=0x7f326000ecd0 comp rx=0 tx=0).stop 2024-05-28T14:13:15.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:15.111+0000 7f3275e00640 1 -- 172.21.15.42:0/1239556152 shutdown_connections 2024-05-28T14:13:15.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:15.111+0000 7f3275e00640 1 --2- 172.21.15.42:0/1239556152 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f324c069b20 0x7f324c06bfe0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:15.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:15.111+0000 7f3275e00640 1 --2- 172.21.15.42:0/1239556152 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f327015cb50 0x7f327015ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:15.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:15.111+0000 7f3275e00640 1 --2- 172.21.15.42:0/1239556152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f327015c770 0x7f32700cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:15.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:15.111+0000 7f3275e00640 1 --2- 172.21.15.42:0/1239556152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32700c8bc0 0x7f32700c9f70 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:15.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:15.111+0000 7f3275e00640 1 -- 172.21.15.42:0/1239556152 >> 172.21.15.42:0/1239556152 conn(0x7f32700b3fd0 msgr2=0x7f327006ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:15.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:15.111+0000 7f3275e00640 1 -- 172.21.15.42:0/1239556152 shutdown_connections 2024-05-28T14:13:15.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:15.111+0000 7f3275e00640 1 -- 172.21.15.42:0/1239556152 wait complete. 2024-05-28T14:13:15.116 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/c4222c14-e22c-4814-8efb-a3c1d580571a 2024-05-28T14:13:15.117 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:13:15.117 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:13:15.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:15.126 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c4222c14-e22c-4814-8efb-a3c1d580571a/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:13:20.985 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:13:20.990 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:13:22.346 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:13:22.346 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:13:22.346 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.15331 s, 150 MB/s 2024-05-28T14:13:22.348 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:13:22.348 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:13:22.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:22.357 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c4222c14-e22c-4814-8efb-a3c1d580571a/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:13:29.356 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:13:29.356 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:13:29.356 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.94601 s, 155 MB/s 2024-05-28T14:13:29.357 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:13:29.358 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:13:29.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:29.367 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c4222c14-e22c-4814-8efb-a3c1d580571a/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:13:36.219 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:13:36.219 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:13:36.219 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.79646 s, 158 MB/s 2024-05-28T14:13:36.220 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-28T14:13:36.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efee6000640 1 Processor -- start 2024-05-28T14:13:36.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efee6000640 1 -- start start 2024-05-28T14:13:36.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efee6000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efee00d0e00 0x7efee006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:36.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efee6000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efee0072d40 0x7efee00aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:36.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efee6000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efee00ade40 con 0x7efee015b350 2024-05-28T14:13:36.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efee6000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7efee00ae9f0 con 0x7efee0072d40 2024-05-28T14:13:36.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efee6000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efee00ae0c0 con 0x7efee00d0e00 2024-05-28T14:13:36.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efedfe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efee0072d40 0x7efee00aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:36.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efedfe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efee0072d40 0x7efee00aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59948/0 (socket says 172.21.15.42:59948) 2024-05-28T14:13:36.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efedfe00640 1 -- 172.21.15.42:0/1469538615 learned_addr learned my addr 172.21.15.42:0/1469538615 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:36.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efed7e00640 1 --2- 172.21.15.42:0/1469538615 >> v2:172.21.15.63:3300/0 conn(0x7efee00d0e00 0x7efee006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:36.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efedfe00640 1 -- 172.21.15.42:0/1469538615 >> v2:172.21.15.63:3300/0 conn(0x7efee00d0e00 msgr2=0x7efee006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:36.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efedfe00640 1 --2- 172.21.15.42:0/1469538615 >> v2:172.21.15.63:3300/0 conn(0x7efee00d0e00 0x7efee006dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efedfe00640 1 -- 172.21.15.42:0/1469538615 >> v1:172.21.15.63:6789/0 conn(0x7efee015b350 legacy=0x7efee00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:36.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efedfe00640 1 -- 172.21.15.42:0/1469538615 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee00ae3a0 con 0x7efee0072d40 2024-05-28T14:13:36.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.383+0000 7efed7e00640 1 --2- 172.21.15.42:0/1469538615 >> v2:172.21.15.63:3300/0 conn(0x7efee00d0e00 0x7efee006dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:13:36.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.387+0000 7efedfe00640 1 --2- 172.21.15.42:0/1469538615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efee0072d40 0x7efee00aee80 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7efee0072650 tx=0x7efed801e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51ae9ba7f65bc8d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:36.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.387+0000 7efedea00640 1 -- 172.21.15.42:0/1469538615 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed800b2d0 con 0x7efee0072d40 2024-05-28T14:13:36.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.387+0000 7efedea00640 1 -- 172.21.15.42:0/1469538615 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed800b470 con 0x7efee0072d40 2024-05-28T14:13:36.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.387+0000 7efedea00640 1 -- 172.21.15.42:0/1469538615 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8009820 con 0x7efee0072d40 2024-05-28T14:13:36.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.387+0000 7efee6000640 1 -- 172.21.15.42:0/1469538615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efee0072d40 msgr2=0x7efee00aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:36.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.387+0000 7efee6000640 1 --2- 172.21.15.42:0/1469538615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efee0072d40 0x7efee00aee80 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7efee0072650 tx=0x7efed801e8b0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.387+0000 7efee6000640 1 -- 172.21.15.42:0/1469538615 shutdown_connections 2024-05-28T14:13:36.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.387+0000 7efee6000640 1 --2- 172.21.15.42:0/1469538615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efee0072d40 0x7efee00aee80 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.387+0000 7efee6000640 1 --2- 172.21.15.42:0/1469538615 >> v2:172.21.15.63:3300/0 conn(0x7efee00d0e00 0x7efee006dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.387+0000 7efee6000640 1 -- 172.21.15.42:0/1469538615 >> 172.21.15.42:0/1469538615 conn(0x7efee006bdf0 msgr2=0x7efee006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:36.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.387+0000 7efee6000640 1 -- 172.21.15.42:0/1469538615 shutdown_connections 2024-05-28T14:13:36.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.387+0000 7efee6000640 1 -- 172.21.15.42:0/1469538615 wait complete. 2024-05-28T14:13:36.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efee6000640 1 Processor -- start 2024-05-28T14:13:36.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efee6000640 1 -- start start 2024-05-28T14:13:36.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efee6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efee0072d40 0x7efee0164cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efee6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efee00d0e00 0x7efee015e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efee6000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efee015b350 0x7efee01fafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efee6000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efee00b3460 con 0x7efee00d0e00 2024-05-28T14:13:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efee6000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efee00b32e0 con 0x7efee0072d40 2024-05-28T14:13:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efee6000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efee00b35e0 con 0x7efee015b350 2024-05-28T14:13:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efedf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efee0072d40 0x7efee0164cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efed7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efee00d0e00 0x7efee015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efedf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efee0072d40 0x7efee0164cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:36096/0 (socket says 172.21.15.42:36096) 2024-05-28T14:13:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efedf400640 1 -- 172.21.15.42:0/4122780785 learned_addr learned my addr 172.21.15.42:0/4122780785 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efedfe00640 1 --2- 172.21.15.42:0/4122780785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efee015b350 0x7efee01fafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efedf400640 1 -- 172.21.15.42:0/4122780785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efee015b350 msgr2=0x7efee01fafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efedf400640 1 --2- 172.21.15.42:0/4122780785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efee015b350 0x7efee01fafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efedf400640 1 -- 172.21.15.42:0/4122780785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efee00d0e00 msgr2=0x7efee015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:36.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efedf400640 1 --2- 172.21.15.42:0/4122780785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efee00d0e00 0x7efee015e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efedf400640 1 -- 172.21.15.42:0/4122780785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee01480a0 con 0x7efee0072d40 2024-05-28T14:13:36.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efed7e00640 1 --2- 172.21.15.42:0/4122780785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efee00d0e00 0x7efee015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:13:36.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efedf400640 1 --2- 172.21.15.42:0/4122780785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efee0072d40 0x7efee0164cc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efed000b410 tx=0x7efed000b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:36.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efed7400640 1 -- 172.21.15.42:0/4122780785 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0015030 con 0x7efee0072d40 2024-05-28T14:13:36.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efed7400640 1 -- 172.21.15.42:0/4122780785 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed0004020 con 0x7efee0072d40 2024-05-28T14:13:36.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efed7400640 1 -- 172.21.15.42:0/4122780785 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0004410 con 0x7efee0072d40 2024-05-28T14:13:36.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efee6000640 1 -- 172.21.15.42:0/4122780785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee0148380 con 0x7efee0072d40 2024-05-28T14:13:36.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efee6000640 1 -- 172.21.15.42:0/4122780785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee0155400 con 0x7efee0072d40 2024-05-28T14:13:36.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.391+0000 7efee6000640 1 -- 172.21.15.42:0/4122780785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee0068a90 con 0x7efee0072d40 2024-05-28T14:13:36.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.395+0000 7efed7400640 1 -- 172.21.15.42:0/4122780785 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) ==== 90180+0+0 (secure 0 0 0) 0x7efed0018070 con 0x7efee0072d40 2024-05-28T14:13:36.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.395+0000 7efed7400640 1 --2- 172.21.15.42:0/4122780785 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efebc069bb0 0x7efebc06c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:36.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.395+0000 7efed7400640 1 -- 172.21.15.42:0/4122780785 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7efed0085180 con 0x7efee0072d40 2024-05-28T14:13:36.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.395+0000 7efed7e00640 1 --2- 172.21.15.42:0/4122780785 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efebc069bb0 0x7efebc06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:36.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.395+0000 7efed7e00640 1 --2- 172.21.15.42:0/4122780785 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efebc069bb0 0x7efebc06c070 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7efee0149b50 tx=0x7efec8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:36.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.399+0000 7efed7400640 1 -- 172.21.15.42:0/4122780785 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efed0056bf0 con 0x7efee0072d40 2024-05-28T14:13:36.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.591+0000 7efee6000640 1 -- 172.21.15.42:0/4122780785 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7efee01488a0 con 0x7efebc069bb0 2024-05-28T14:13:36.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.607+0000 7efed7400640 1 -- 172.21.15.42:0/4122780785 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efee01488a0 con 0x7efebc069bb0 2024-05-28T14:13:36.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.607+0000 7efee6000640 1 -- 172.21.15.42:0/4122780785 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efebc069bb0 msgr2=0x7efebc06c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:36.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.607+0000 7efee6000640 1 --2- 172.21.15.42:0/4122780785 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efebc069bb0 0x7efebc06c070 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7efee0149b50 tx=0x7efec8008040 comp rx=0 tx=0).stop 2024-05-28T14:13:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.607+0000 7efee6000640 1 -- 172.21.15.42:0/4122780785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efee0072d40 msgr2=0x7efee0164cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.607+0000 7efee6000640 1 --2- 172.21.15.42:0/4122780785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efee0072d40 0x7efee0164cc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efed000b410 tx=0x7efed000b8d0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.607+0000 7efee6000640 1 -- 172.21.15.42:0/4122780785 shutdown_connections 2024-05-28T14:13:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.607+0000 7efee6000640 1 --2- 172.21.15.42:0/4122780785 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efebc069bb0 0x7efebc06c070 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.607+0000 7efee6000640 1 --2- 172.21.15.42:0/4122780785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efee015b350 0x7efee01fafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.607+0000 7efee6000640 1 --2- 172.21.15.42:0/4122780785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efee00d0e00 0x7efee015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.607+0000 7efee6000640 1 --2- 172.21.15.42:0/4122780785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efee0072d40 0x7efee0164cc0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.607+0000 7efee6000640 1 -- 172.21.15.42:0/4122780785 >> 172.21.15.42:0/4122780785 conn(0x7efee006bdf0 msgr2=0x7efee00b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.607+0000 7efee6000640 1 -- 172.21.15.42:0/4122780785 shutdown_connections 2024-05-28T14:13:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.607+0000 7efee6000640 1 -- 172.21.15.42:0/4122780785 wait complete. 2024-05-28T14:13:36.613 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:13:36.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.771+0000 7ff96e200640 1 Processor -- start 2024-05-28T14:13:36.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.771+0000 7ff96e200640 1 -- start start 2024-05-28T14:13:36.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.771+0000 7ff96e200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff968144f80 0x7ff968072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:36.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.771+0000 7ff96e200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff96806ee20 0x7ff96806f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:36.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.771+0000 7ff96e200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9680afe40 con 0x7ff96815b9b0 2024-05-28T14:13:36.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.771+0000 7ff96e200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff96806f7c0 con 0x7ff96806ee20 2024-05-28T14:13:36.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.771+0000 7ff96e200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff9680724d0 con 0x7ff968144f80 2024-05-28T14:13:36.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.771+0000 7ff95ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff96806ee20 0x7ff96806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:36.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.771+0000 7ff967e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff96815b9b0 0x7ff96815be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:60510/0 (socket says 172.21.15.42:60510) 2024-05-28T14:13:36.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.771+0000 7ff967e00640 1 -- 172.21.15.42:0/1769392859 learned_addr learned my addr 172.21.15.42:0/1769392859 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:13:36.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.771+0000 7ff95ea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff96806ee20 0x7ff96806f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59956/0 (socket says 172.21.15.42:59956) 2024-05-28T14:13:36.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.771+0000 7ff967400640 1 --2- 172.21.15.42:0/1769392859 >> v2:172.21.15.63:3300/0 conn(0x7ff968144f80 0x7ff968072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:36.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.775+0000 7ff95ea00640 1 -- 172.21.15.42:0/1769392859 >> v2:172.21.15.63:3300/0 conn(0x7ff968144f80 msgr2=0x7ff968072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:36.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.775+0000 7ff95ea00640 1 --2- 172.21.15.42:0/1769392859 >> v2:172.21.15.63:3300/0 conn(0x7ff968144f80 0x7ff968072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.775+0000 7ff95ea00640 1 -- 172.21.15.42:0/1769392859 >> v1:172.21.15.42:6789/0 conn(0x7ff96815b9b0 legacy=0x7ff96815be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:36.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.775+0000 7ff95ea00640 1 -- 172.21.15.42:0/1769392859 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96815c600 con 0x7ff96806ee20 2024-05-28T14:13:36.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.775+0000 7ff966a00640 1 -- 172.21.15.42:0/1769392859 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2097633191 0 0) 0x7ff9680afe40 con 0x7ff96815b9b0 2024-05-28T14:13:36.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.775+0000 7ff95ea00640 1 --2- 172.21.15.42:0/1769392859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff96806ee20 0x7ff96806f280 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ff968071760 tx=0x7ff95001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9153b966da15ad63 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:36.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.775+0000 7ff966a00640 1 -- 172.21.15.42:0/1769392859 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9500b0070 con 0x7ff96806ee20 2024-05-28T14:13:36.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.775+0000 7ff966a00640 1 -- 172.21.15.42:0/1769392859 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9500ab600 con 0x7ff96806ee20 2024-05-28T14:13:36.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.775+0000 7ff966a00640 1 -- 172.21.15.42:0/1769392859 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9500ab9d0 con 0x7ff96806ee20 2024-05-28T14:13:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.775+0000 7ff96e200640 1 -- 172.21.15.42:0/1769392859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff96806ee20 msgr2=0x7ff96806f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.775+0000 7ff96e200640 1 --2- 172.21.15.42:0/1769392859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff96806ee20 0x7ff96806f280 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ff968071760 tx=0x7ff95001c480 comp rx=0 tx=0).stop 2024-05-28T14:13:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff96e200640 1 -- 172.21.15.42:0/1769392859 shutdown_connections 2024-05-28T14:13:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff96e200640 1 --2- 172.21.15.42:0/1769392859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff96806ee20 0x7ff96806f280 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff96e200640 1 --2- 172.21.15.42:0/1769392859 >> v2:172.21.15.63:3300/0 conn(0x7ff968144f80 0x7ff968072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff96e200640 1 -- 172.21.15.42:0/1769392859 >> 172.21.15.42:0/1769392859 conn(0x7ff9680b21f0 msgr2=0x7ff9680b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff96e200640 1 -- 172.21.15.42:0/1769392859 shutdown_connections 2024-05-28T14:13:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff96e200640 1 -- 172.21.15.42:0/1769392859 wait complete. 2024-05-28T14:13:36.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff96e200640 1 Processor -- start 2024-05-28T14:13:36.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff96e200640 1 -- start start 2024-05-28T14:13:36.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff96e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff96806ee20 0x7ff968140200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:36.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff96e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff968144f80 0x7ff9681423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:36.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff96e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff96815b9b0 0x7ff968142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:36.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff96e200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff968072ea0 con 0x7ff968144f80 2024-05-28T14:13:36.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff96e200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9680b4010 con 0x7ff96815b9b0 2024-05-28T14:13:36.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff96e200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff968140de0 con 0x7ff96806ee20 2024-05-28T14:13:36.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff95ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff968144f80 0x7ff9681423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:36.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff95ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff968144f80 0x7ff9681423e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59958/0 (socket says 172.21.15.42:59958) 2024-05-28T14:13:36.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff95ea00640 1 -- 172.21.15.42:0/3823925500 learned_addr learned my addr 172.21.15.42:0/3823925500 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff967400640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff96806ee20 0x7ff968140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff967e00640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff96815b9b0 0x7ff968142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff95ea00640 1 -- 172.21.15.42:0/3823925500 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff96806ee20 msgr2=0x7ff968140200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff95ea00640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff96806ee20 0x7ff968140200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff95ea00640 1 -- 172.21.15.42:0/3823925500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff96815b9b0 msgr2=0x7ff968142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff95ea00640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff96815b9b0 0x7ff968142920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff95ea00640 1 -- 172.21.15.42:0/3823925500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9681420d0 con 0x7ff968144f80 2024-05-28T14:13:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff967e00640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff96815b9b0 0x7ff968142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff95ea00640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff968144f80 0x7ff9681423e0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7ff9500ae040 tx=0x7ff950019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:36.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.779+0000 7ff95fe00640 1 -- 172.21.15.42:0/3823925500 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9500b8020 con 0x7ff968144f80 2024-05-28T14:13:36.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.783+0000 7ff95fe00640 1 -- 172.21.15.42:0/3823925500 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9500b4410 con 0x7ff968144f80 2024-05-28T14:13:36.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.783+0000 7ff95fe00640 1 -- 172.21.15.42:0/3823925500 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9500b0040 con 0x7ff968144f80 2024-05-28T14:13:36.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.783+0000 7ff96e200640 1 -- 172.21.15.42:0/3823925500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff968141030 con 0x7ff968144f80 2024-05-28T14:13:36.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.783+0000 7ff96e200640 1 -- 172.21.15.42:0/3823925500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9681415f0 con 0x7ff968144f80 2024-05-28T14:13:36.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.783+0000 7ff96e200640 1 -- 172.21.15.42:0/3823925500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9680738b0 con 0x7ff968144f80 2024-05-28T14:13:36.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.783+0000 7ff95fe00640 1 -- 172.21.15.42:0/3823925500 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 39) ==== 90180+0+0 (secure 0 0 0) 0x7ff9500bb020 con 0x7ff968144f80 2024-05-28T14:13:36.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.783+0000 7ff95fe00640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff944069bb0 0x7ff94406c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:36.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.783+0000 7ff95fe00640 1 -- 172.21.15.42:0/3823925500 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7ff95011f300 con 0x7ff968144f80 2024-05-28T14:13:36.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.783+0000 7ff967400640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff944069bb0 0x7ff94406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:36.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.783+0000 7ff967400640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff944069bb0 0x7ff94406c070 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff9680b0940 tx=0x7ff958009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:36.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.787+0000 7ff95fe00640 1 -- 172.21.15.42:0/3823925500 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff9500c2030 con 0x7ff968144f80 2024-05-28T14:13:36.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.979+0000 7ff96e200640 1 -- 172.21.15.42:0/3823925500 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7ff968075150 con 0x7ff944069bb0 2024-05-28T14:13:36.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.983+0000 7ff95fe00640 1 -- 172.21.15.42:0/3823925500 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7ff968075150 con 0x7ff944069bb0 2024-05-28T14:13:36.968 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/c4222c14-e22c-4814-8efb-a3c1d580571a 2024-05-28T14:13:36.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.983+0000 7ff96e200640 1 -- 172.21.15.42:0/3823925500 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff944069bb0 msgr2=0x7ff94406c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:36.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.983+0000 7ff96e200640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff944069bb0 0x7ff94406c070 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff9680b0940 tx=0x7ff958009190 comp rx=0 tx=0).stop 2024-05-28T14:13:36.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.983+0000 7ff96e200640 1 -- 172.21.15.42:0/3823925500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff968144f80 msgr2=0x7ff9681423e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.983+0000 7ff96e200640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff968144f80 0x7ff9681423e0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7ff9500ae040 tx=0x7ff950019220 comp rx=0 tx=0).stop 2024-05-28T14:13:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.983+0000 7ff96e200640 1 -- 172.21.15.42:0/3823925500 shutdown_connections 2024-05-28T14:13:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.983+0000 7ff96e200640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff944069bb0 0x7ff94406c070 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.983+0000 7ff96e200640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff96815b9b0 0x7ff968142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.983+0000 7ff96e200640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff968144f80 0x7ff9681423e0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.983+0000 7ff96e200640 1 --2- 172.21.15.42:0/3823925500 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff96806ee20 0x7ff968140200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.983+0000 7ff96e200640 1 -- 172.21.15.42:0/3823925500 >> 172.21.15.42:0/3823925500 conn(0x7ff9680b21f0 msgr2=0x7ff9680cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.983+0000 7ff96e200640 1 -- 172.21.15.42:0/3823925500 shutdown_connections 2024-05-28T14:13:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:36.987+0000 7ff96e200640 1 -- 172.21.15.42:0/3823925500 wait complete. 2024-05-28T14:13:36.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:36.989 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c4222c14-e22c-4814-8efb-a3c1d580571a 2024-05-28T14:13:36.989 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:13:37.003 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/c4222c14-e22c-4814-8efb-a3c1d580571a 2024-05-28T14:13:37.003 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3138256896" 2024-05-28T14:13:37.003 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:37.004 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-28T14:13:39.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:39.005 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c4222c14-e22c-4814-8efb-a3c1d580571a 2024-05-28T14:13:39.005 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:13:39.020 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/c4222c14-e22c-4814-8efb-a3c1d580571a 2024-05-28T14:13:39.021 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3138256896" 2024-05-28T14:13:39.021 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:39.022 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-28T14:13:41.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:41.023 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c4222c14-e22c-4814-8efb-a3c1d580571a 2024-05-28T14:13:41.023 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:13:41.039 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/c4222c14-e22c-4814-8efb-a3c1d580571a 2024-05-28T14:13:41.039 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3221225472" 2024-05-28T14:13:41.039 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:41.040 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-28T14:13:41.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone86011 2024-05-28T14:13:41.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.255+0000 7f8e24000640 1 Processor -- start 2024-05-28T14:13:41.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.255+0000 7f8e24000640 1 -- start start 2024-05-28T14:13:41.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.255+0000 7f8e24000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8e1c140d20 0x7f8e1c167490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.255+0000 7f8e24000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e1c160230 con 0x7f8e1c071de0 2024-05-28T14:13:41.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.255+0000 7f8e24000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e1c160510 con 0x7f8e1c0ce3b0 2024-05-28T14:13:41.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.255+0000 7f8e24000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8e1c1607f0 con 0x7f8e1c140d20 2024-05-28T14:13:41.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.255+0000 7f8e21600640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8e1c071de0 0x7f8e1c0721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53472/0 (socket says 172.21.15.42:53472) 2024-05-28T14:13:41.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.255+0000 7f8e21600640 1 -- 172.21.15.42:0/3200755780 learned_addr learned my addr 172.21.15.42:0/3200755780 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:13:41.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.255+0000 7f8e1be00640 1 -- 172.21.15.42:0/3200755780 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2032340400 0 0) 0x7f8e1c160230 con 0x7f8e1c071de0 2024-05-28T14:13:41.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.255+0000 7f8e22000640 1 --2- 172.21.15.42:0/3200755780 >> v2:172.21.15.63:3300/0 conn(0x7f8e1c140d20 0x7f8e1c167490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.255+0000 7f8e1be00640 1 -- 172.21.15.42:0/3200755780 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e04003530 con 0x7f8e1c071de0 2024-05-28T14:13:41.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.255+0000 7f8e1be00640 1 -- 172.21.15.42:0/3200755780 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3108442194 0 0) 0x7f8e04003530 con 0x7f8e1c071de0 2024-05-28T14:13:41.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e1be00640 1 -- 172.21.15.42:0/3200755780 >> v2:172.21.15.63:3300/0 conn(0x7f8e1c140d20 msgr2=0x7f8e1c167490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:41.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e1be00640 1 --2- 172.21.15.42:0/3200755780 >> v2:172.21.15.63:3300/0 conn(0x7f8e1c140d20 0x7f8e1c167490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e1be00640 1 -- 172.21.15.42:0/3200755780 >> v1:172.21.15.63:6789/0 conn(0x7f8e1c0ce3b0 legacy=0x7f8e1c0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:41.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e1be00640 1 -- 172.21.15.42:0/3200755780 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c160ad0 con 0x7f8e1c071de0 2024-05-28T14:13:41.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e1be00640 1 -- 172.21.15.42:0/3200755780 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f8e10001a00 con 0x7f8e1c071de0 2024-05-28T14:13:41.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e1be00640 1 -- 172.21.15.42:0/3200755780 >> v1:172.21.15.42:6789/0 conn(0x7f8e1c071de0 legacy=0x7f8e1c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:41.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e1be00640 1 --2- 172.21.15.42:0/3200755780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e040039c0 0x7f8e04003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e1be00640 1 --2- 172.21.15.42:0/3200755780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e04004810 0x7f8e04004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e1be00640 1 --2- 172.21.15.42:0/3200755780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e040052f0 0x7f8e0400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e1be00640 1 -- 172.21.15.42:0/3200755780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1607f0 con 0x7f8e04004810 2024-05-28T14:13:41.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e1be00640 1 -- 172.21.15.42:0/3200755780 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0400c1c0 con 0x7f8e040039c0 2024-05-28T14:13:41.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e1be00640 1 -- 172.21.15.42:0/3200755780 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0400c3c0 con 0x7f8e040052f0 2024-05-28T14:13:41.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e20c00640 1 --2- 172.21.15.42:0/3200755780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e04004810 0x7f8e04004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e21600640 1 --2- 172.21.15.42:0/3200755780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e040039c0 0x7f8e04003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e22000640 1 --2- 172.21.15.42:0/3200755780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e040052f0 0x7f8e0400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e20c00640 1 -- 172.21.15.42:0/3200755780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e040052f0 msgr2=0x7f8e0400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:41.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e20c00640 1 --2- 172.21.15.42:0/3200755780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e040052f0 0x7f8e0400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e20c00640 1 -- 172.21.15.42:0/3200755780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e040039c0 msgr2=0x7f8e04003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:41.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e20c00640 1 --2- 172.21.15.42:0/3200755780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e040039c0 0x7f8e04003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e20c00640 1 -- 172.21.15.42:0/3200755780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e04003530 con 0x7f8e04004810 2024-05-28T14:13:41.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e22000640 1 --2- 172.21.15.42:0/3200755780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e040052f0 0x7f8e0400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:41.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e22000640 1 -- 172.21.15.42:0/3200755780 reap_dead start 2024-05-28T14:13:41.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e20c00640 1 --2- 172.21.15.42:0/3200755780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e04004810 0x7f8e04004c90 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f8e08001750 tx=0x7f8e0801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fa62b662e4ab06 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:41.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.259+0000 7f8e1be00640 1 -- 172.21.15.42:0/3200755780 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e080013a0 con 0x7f8e04004810 2024-05-28T14:13:41.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e1be00640 1 -- 172.21.15.42:0/3200755780 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e080052f0 con 0x7f8e04004810 2024-05-28T14:13:41.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e1be00640 1 -- 172.21.15.42:0/3200755780 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08003840 con 0x7f8e04004810 2024-05-28T14:13:41.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 -- 172.21.15.42:0/3200755780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e04004810 msgr2=0x7f8e04004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:41.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 --2- 172.21.15.42:0/3200755780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e04004810 0x7f8e04004c90 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f8e08001750 tx=0x7f8e0801c3a0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 -- 172.21.15.42:0/3200755780 shutdown_connections 2024-05-28T14:13:41.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 --2- 172.21.15.42:0/3200755780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e04004810 0x7f8e04004c90 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 -- 172.21.15.42:0/3200755780 >> 172.21.15.42:0/3200755780 conn(0x7f8e1c0b21f0 msgr2=0x7f8e1c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:41.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 -- 172.21.15.42:0/3200755780 shutdown_connections 2024-05-28T14:13:41.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 -- 172.21.15.42:0/3200755780 wait complete. 2024-05-28T14:13:41.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 Processor -- start 2024-05-28T14:13:41.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 -- start start 2024-05-28T14:13:41.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e04004810 0x7f8e1c12eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e1c15ca10 0x7f8e1c15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e1c144750 0x7f8e1c144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1420a0 con 0x7f8e1c15ca10 2024-05-28T14:13:41.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c141f20 con 0x7f8e1c144750 2024-05-28T14:13:41.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e24000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8e1c142220 con 0x7f8e04004810 2024-05-28T14:13:41.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e20c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e1c15ca10 0x7f8e1c15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e20c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e1c15ca10 0x7f8e1c15ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40878/0 (socket says 172.21.15.42:40878) 2024-05-28T14:13:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.263+0000 7f8e20c00640 1 -- 172.21.15.42:0/745715906 learned_addr learned my addr 172.21.15.42:0/745715906 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e21600640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e04004810 0x7f8e1c12eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e22000640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e1c144750 0x7f8e1c144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e20c00640 1 -- 172.21.15.42:0/745715906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e04004810 msgr2=0x7f8e1c12eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e20c00640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e04004810 0x7f8e1c12eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e20c00640 1 -- 172.21.15.42:0/745715906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e1c144750 msgr2=0x7f8e1c144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e20c00640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e1c144750 0x7f8e1c144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e20c00640 1 -- 172.21.15.42:0/745715906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c15d4d0 con 0x7f8e1c15ca10 2024-05-28T14:13:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e22000640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e1c144750 0x7f8e1c144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:13:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e20c00640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e1c15ca10 0x7f8e1c15ce90 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f8e08005040 tx=0x7f8e04005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:41.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e19600640 1 -- 172.21.15.42:0/745715906 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08007a40 con 0x7f8e1c15ca10 2024-05-28T14:13:41.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e19600640 1 -- 172.21.15.42:0/745715906 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e080bb070 con 0x7f8e1c15ca10 2024-05-28T14:13:41.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e19600640 1 -- 172.21.15.42:0/745715906 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e080013a0 con 0x7f8e1c15ca10 2024-05-28T14:13:41.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e24000640 1 -- 172.21.15.42:0/745715906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1c15d7b0 con 0x7f8e1c15ca10 2024-05-28T14:13:41.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e24000640 1 -- 172.21.15.42:0/745715906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e1c12f0b0 con 0x7f8e1c15ca10 2024-05-28T14:13:41.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e24000640 1 -- 172.21.15.42:0/745715906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e1c145110 con 0x7f8e1c15ca10 2024-05-28T14:13:41.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e19600640 1 -- 172.21.15.42:0/745715906 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 39) ==== 90180+0+0 (secure 0 0 0) 0x7f8e080b4040 con 0x7f8e1c15ca10 2024-05-28T14:13:41.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e19600640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8dfc069bb0 0x7f8dfc06c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.267+0000 7f8e19600640 1 -- 172.21.15.42:0/745715906 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f8e0811f7a0 con 0x7f8e1c15ca10 2024-05-28T14:13:41.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.271+0000 7f8e21600640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8dfc069bb0 0x7f8dfc06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.271+0000 7f8e21600640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8dfc069bb0 0x7f8dfc06c070 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c12f420 tx=0x7f8e10006040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:41.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.275+0000 7f8e19600640 1 -- 172.21.15.42:0/745715906 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8e080160b0 con 0x7f8e1c15ca10 2024-05-28T14:13:41.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.471+0000 7f8e24000640 1 -- 172.21.15.42:0/745715906 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone86011", "target": ["mon-mgr", ""]}) -- 0x7f8e1c0738b0 con 0x7f8dfc069bb0 2024-05-28T14:13:41.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.531+0000 7f8e19600640 1 -- 172.21.15.42:0/745715906 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8e1c0738b0 con 0x7f8dfc069bb0 2024-05-28T14:13:41.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f8e24000640 1 -- 172.21.15.42:0/745715906 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8dfc069bb0 msgr2=0x7f8dfc06c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:41.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f8e24000640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8dfc069bb0 0x7f8dfc06c070 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c12f420 tx=0x7f8e10006040 comp rx=0 tx=0).stop 2024-05-28T14:13:41.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f8e24000640 1 -- 172.21.15.42:0/745715906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e1c15ca10 msgr2=0x7f8e1c15ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:41.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f8e24000640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e1c15ca10 0x7f8e1c15ce90 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f8e08005040 tx=0x7f8e04005840 comp rx=0 tx=0).stop 2024-05-28T14:13:41.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f8e24000640 1 -- 172.21.15.42:0/745715906 shutdown_connections 2024-05-28T14:13:41.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f8e24000640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8dfc069bb0 0x7f8dfc06c070 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f8e24000640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e1c144750 0x7f8e1c144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f8e24000640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e1c15ca10 0x7f8e1c15ce90 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f8e24000640 1 --2- 172.21.15.42:0/745715906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e04004810 0x7f8e1c12eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f8e24000640 1 -- 172.21.15.42:0/745715906 >> 172.21.15.42:0/745715906 conn(0x7f8e1c0b21f0 msgr2=0x7f8e1c0b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:41.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f8e24000640 1 -- 172.21.15.42:0/745715906 shutdown_connections 2024-05-28T14:13:41.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f8e24000640 1 -- 172.21.15.42:0/745715906 wait complete. 2024-05-28T14:13:41.521 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:41.521 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:41.521 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:41.522 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:41.522 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:41.535+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:41.539 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:41.555+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:41.540 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:41.555+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:41.540 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:41.555+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:41.540 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:41.555+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:41.540 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:41.555+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:41.544 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone86012 2024-05-28T14:13:41.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.711+0000 7f0ee7200640 1 Processor -- start 2024-05-28T14:13:41.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.711+0000 7f0ee7200640 1 -- start start 2024-05-28T14:13:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.711+0000 7f0ee7200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0ee815c6c0 0x7f0ee80d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.711+0000 7f0ee7200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee809aa50 0x7f0ee80d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.711+0000 7f0ee7200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0ee80d0de0 0x7f0ee80ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.711+0000 7f0ee7200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0ee80d1230 con 0x7f0ee815c6c0 2024-05-28T14:13:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.711+0000 7f0ee7200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0ee806fe90 con 0x7f0ee80d0de0 2024-05-28T14:13:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.711+0000 7f0ee7200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee809ba10 con 0x7f0ee809aa50 2024-05-28T14:13:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.711+0000 7f0ee5e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0ee815c6c0 0x7f0ee80d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.711+0000 7f0ee5e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0ee815c6c0 0x7f0ee80d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40928/0 (socket says 172.21.15.42:40928) 2024-05-28T14:13:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.711+0000 7f0ee5e00640 1 -- 172.21.15.42:0/906566288 learned_addr learned my addr 172.21.15.42:0/906566288 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.711+0000 7f0ee6800640 1 --2- 172.21.15.42:0/906566288 >> v2:172.21.15.63:3300/0 conn(0x7f0ee80d0de0 0x7f0ee80ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.711+0000 7f0edd400640 1 --2- 172.21.15.42:0/906566288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee809aa50 0x7f0ee80d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee5e00640 1 -- 172.21.15.42:0/906566288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee809aa50 msgr2=0x7f0ee80d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee5e00640 1 --2- 172.21.15.42:0/906566288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee809aa50 0x7f0ee80d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee5e00640 1 -- 172.21.15.42:0/906566288 >> v2:172.21.15.63:3300/0 conn(0x7f0ee80d0de0 msgr2=0x7f0ee80ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee5e00640 1 --2- 172.21.15.42:0/906566288 >> v2:172.21.15.63:3300/0 conn(0x7f0ee80d0de0 0x7f0ee80ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee5e00640 1 -- 172.21.15.42:0/906566288 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee8070130 con 0x7f0ee815c6c0 2024-05-28T14:13:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0edd400640 1 --2- 172.21.15.42:0/906566288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee809aa50 0x7f0ee80d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee6800640 1 --2- 172.21.15.42:0/906566288 >> v2:172.21.15.63:3300/0 conn(0x7f0ee80d0de0 0x7f0ee80ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:41.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee5e00640 1 --2- 172.21.15.42:0/906566288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee815c6c0 0x7f0ee80d0360 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80741e0 tx=0x7f0ed801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6266188d3014ecef server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:41.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee5400640 1 -- 172.21.15.42:0/906566288 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80ab5f0 con 0x7f0ee815c6c0 2024-05-28T14:13:41.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee5400640 1 -- 172.21.15.42:0/906566288 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed80b0070 con 0x7f0ee815c6c0 2024-05-28T14:13:41.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee5400640 1 -- 172.21.15.42:0/906566288 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80ab9b0 con 0x7f0ee815c6c0 2024-05-28T14:13:41.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee7200640 1 -- 172.21.15.42:0/906566288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee815c6c0 msgr2=0x7f0ee80d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:41.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee7200640 1 --2- 172.21.15.42:0/906566288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee815c6c0 0x7f0ee80d0360 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80741e0 tx=0x7f0ed801c520 comp rx=0 tx=0).stop 2024-05-28T14:13:41.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee7200640 1 -- 172.21.15.42:0/906566288 shutdown_connections 2024-05-28T14:13:41.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee7200640 1 --2- 172.21.15.42:0/906566288 >> v2:172.21.15.63:3300/0 conn(0x7f0ee80d0de0 0x7f0ee80ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee7200640 1 --2- 172.21.15.42:0/906566288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee809aa50 0x7f0ee80d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee7200640 1 --2- 172.21.15.42:0/906566288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee815c6c0 0x7f0ee80d0360 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee7200640 1 -- 172.21.15.42:0/906566288 >> 172.21.15.42:0/906566288 conn(0x7f0ee80b3dd0 msgr2=0x7f0ee80b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:41.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.715+0000 7f0ee7200640 1 -- 172.21.15.42:0/906566288 shutdown_connections 2024-05-28T14:13:41.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee7200640 1 -- 172.21.15.42:0/906566288 wait complete. 2024-05-28T14:13:41.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee7200640 1 Processor -- start 2024-05-28T14:13:41.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee7200640 1 -- start start 2024-05-28T14:13:41.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee7200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee809aa50 0x7f0ee80c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee80d0de0 0x7f0ee814a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee815c6c0 0x7f0ee80cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee7200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee8070ab0 con 0x7f0ee815c6c0 2024-05-28T14:13:41.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee7200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee8070930 con 0x7f0ee80d0de0 2024-05-28T14:13:41.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee7200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee8070c30 con 0x7f0ee809aa50 2024-05-28T14:13:41.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee6800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee815c6c0 0x7f0ee80cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee6800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee815c6c0 0x7f0ee80cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40944/0 (socket says 172.21.15.42:40944) 2024-05-28T14:13:41.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee6800640 1 -- 172.21.15.42:0/3916444842 learned_addr learned my addr 172.21.15.42:0/3916444842 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:41.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0edd400640 1 --2- 172.21.15.42:0/3916444842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee80d0de0 0x7f0ee814a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee5e00640 1 --2- 172.21.15.42:0/3916444842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee809aa50 0x7f0ee80c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee6800640 1 -- 172.21.15.42:0/3916444842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee809aa50 msgr2=0x7f0ee80c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:41.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee6800640 1 --2- 172.21.15.42:0/3916444842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee809aa50 0x7f0ee80c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee6800640 1 -- 172.21.15.42:0/3916444842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee80d0de0 msgr2=0x7f0ee814a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:41.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee6800640 1 --2- 172.21.15.42:0/3916444842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee80d0de0 0x7f0ee814a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:41.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee6800640 1 -- 172.21.15.42:0/3916444842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee815f520 con 0x7f0ee815c6c0 2024-05-28T14:13:41.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.719+0000 7f0ee6800640 1 --2- 172.21.15.42:0/3916444842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee815c6c0 0x7f0ee80cbd00 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009fd0 tx=0x7f0ee000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:41.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.723+0000 7f0edea00640 1 -- 172.21.15.42:0/3916444842 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee001a070 con 0x7f0ee815c6c0 2024-05-28T14:13:41.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.723+0000 7f0edea00640 1 -- 172.21.15.42:0/3916444842 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ee0003bc0 con 0x7f0ee815c6c0 2024-05-28T14:13:41.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.723+0000 7f0edea00640 1 -- 172.21.15.42:0/3916444842 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0004030 con 0x7f0ee815c6c0 2024-05-28T14:13:41.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.723+0000 7f0ee7200640 1 -- 172.21.15.42:0/3916444842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee815f800 con 0x7f0ee815c6c0 2024-05-28T14:13:41.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.723+0000 7f0ee7200640 1 -- 172.21.15.42:0/3916444842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee815add0 con 0x7f0ee815c6c0 2024-05-28T14:13:41.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.723+0000 7f0ee7200640 1 -- 172.21.15.42:0/3916444842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee804fcf0 con 0x7f0ee815c6c0 2024-05-28T14:13:41.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.723+0000 7f0edea00640 1 -- 172.21.15.42:0/3916444842 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 39) ==== 90180+0+0 (secure 0 0 0) 0x7f0ee001d070 con 0x7f0ee815c6c0 2024-05-28T14:13:41.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.723+0000 7f0edea00640 1 --2- 172.21.15.42:0/3916444842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ec4069be0 0x7f0ec406c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:41.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.723+0000 7f0edea00640 1 -- 172.21.15.42:0/3916444842 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f0ee0085990 con 0x7f0ee815c6c0 2024-05-28T14:13:41.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.723+0000 7f0ee5e00640 1 --2- 172.21.15.42:0/3916444842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ec4069be0 0x7f0ec406c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:41.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.723+0000 7f0ee5e00640 1 --2- 172.21.15.42:0/3916444842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ec4069be0 0x7f0ec406c0a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80af000 tx=0x7f0ed80a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:41.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.727+0000 7f0edea00640 1 -- 172.21.15.42:0/3916444842 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ee0010070 con 0x7f0ee815c6c0 2024-05-28T14:13:41.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:41.927+0000 7f0ee7200640 1 -- 172.21.15.42:0/3916444842 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone86012", "target": ["mon-mgr", ""]}) -- 0x7f0ee80cead0 con 0x7f0ec4069be0 2024-05-28T14:13:42.522 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:42.539+0000 7f075c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:42.523 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:42.539+0000 7f075c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:42.523 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:42.539+0000 7f075c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:42.523 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:42.539+0000 7f075c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:42.523 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:42.539+0000 7f075c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:42.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.627+0000 7f0edea00640 1 -- 172.21.15.42:0/3916444842 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0ee80cead0 con 0x7f0ec4069be0 2024-05-28T14:13:42.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.631+0000 7f0ee7200640 1 -- 172.21.15.42:0/3916444842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ec4069be0 msgr2=0x7f0ec406c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:42.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.631+0000 7f0ee7200640 1 --2- 172.21.15.42:0/3916444842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ec4069be0 0x7f0ec406c0a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80af000 tx=0x7f0ed80a7000 comp rx=0 tx=0).stop 2024-05-28T14:13:42.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.631+0000 7f0ee7200640 1 -- 172.21.15.42:0/3916444842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee815c6c0 msgr2=0x7f0ee80cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:42.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.631+0000 7f0ee7200640 1 --2- 172.21.15.42:0/3916444842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee815c6c0 0x7f0ee80cbd00 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009fd0 tx=0x7f0ee000d560 comp rx=0 tx=0).stop 2024-05-28T14:13:42.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.631+0000 7f0ee7200640 1 -- 172.21.15.42:0/3916444842 shutdown_connections 2024-05-28T14:13:42.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.631+0000 7f0ee7200640 1 --2- 172.21.15.42:0/3916444842 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ec4069be0 0x7f0ec406c0a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:42.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.631+0000 7f0ee7200640 1 --2- 172.21.15.42:0/3916444842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ee815c6c0 0x7f0ee80cbd00 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:42.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.631+0000 7f0ee7200640 1 --2- 172.21.15.42:0/3916444842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ee80d0de0 0x7f0ee814a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:42.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.631+0000 7f0ee7200640 1 --2- 172.21.15.42:0/3916444842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ee809aa50 0x7f0ee80c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:42.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.631+0000 7f0ee7200640 1 -- 172.21.15.42:0/3916444842 >> 172.21.15.42:0/3916444842 conn(0x7f0ee80b3dd0 msgr2=0x7f0ee80d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:42.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.631+0000 7f0ee7200640 1 -- 172.21.15.42:0/3916444842 shutdown_connections 2024-05-28T14:13:42.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.631+0000 7f0ee7200640 1 -- 172.21.15.42:0/3916444842 wait complete. 2024-05-28T14:13:42.630 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:42.643+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:42.630 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:42.643+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:42.630 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:42.643+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:42.630 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:42.643+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:42.630 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:42.643+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:42.667 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone86013 2024-05-28T14:13:42.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.959+0000 7f4de3e00640 1 Processor -- start 2024-05-28T14:13:42.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.959+0000 7f4de3e00640 1 -- start start 2024-05-28T14:13:42.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.959+0000 7f4de3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ddc0cf020 0x7f4ddc06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:42.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.959+0000 7f4de3e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ddc0b25c0 con 0x7f4ddc06bf30 2024-05-28T14:13:42.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.959+0000 7f4de3e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ddc0b28a0 con 0x7f4ddc162fe0 2024-05-28T14:13:42.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.959+0000 7f4de3e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4ddc0b2e60 con 0x7f4ddc0cf020 2024-05-28T14:13:42.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.963+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ddc0cf020 0x7f4ddc06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:42.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.963+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ddc0cf020 0x7f4ddc06b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:35704/0 (socket says 172.21.15.42:35704) 2024-05-28T14:13:42.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.963+0000 7f4de0a00640 1 -- 172.21.15.42:0/237179841 learned_addr learned my addr 172.21.15.42:0/237179841 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:42.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.963+0000 7f4de0a00640 1 -- 172.21.15.42:0/237179841 >> v1:172.21.15.63:6789/0 conn(0x7f4ddc162fe0 legacy=0x7f4ddc0b04a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T14:13:42.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.963+0000 7f4de0a00640 1 -- 172.21.15.42:0/237179841 >> v1:172.21.15.42:6789/0 conn(0x7f4ddc06bf30 legacy=0x7f4ddc0b1520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-28T14:13:42.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.963+0000 7f4de0a00640 1 -- 172.21.15.42:0/237179841 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc0b2b80 con 0x7f4ddc0cf020 2024-05-28T14:13:42.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.975+0000 7f4de0a00640 1 --2- 172.21.15.42:0/237179841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ddc0cf020 0x7f4ddc06b9f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00098a0 tx=0x7f4dd001c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0238f7bfcb65da6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:42.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.975+0000 7f4ddbe00640 1 -- 172.21.15.42:0/237179841 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00b0070 con 0x7f4ddc0cf020 2024-05-28T14:13:42.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.975+0000 7f4ddbe00640 1 -- 172.21.15.42:0/237179841 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd00ab400 con 0x7f4ddc0cf020 2024-05-28T14:13:42.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.975+0000 7f4ddbe00640 1 -- 172.21.15.42:0/237179841 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00ab7d0 con 0x7f4ddc0cf020 2024-05-28T14:13:42.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.991+0000 7f4de3e00640 1 -- 172.21.15.42:0/237179841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ddc0cf020 msgr2=0x7f4ddc06b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:42.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.991+0000 7f4de3e00640 1 --2- 172.21.15.42:0/237179841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ddc0cf020 0x7f4ddc06b9f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00098a0 tx=0x7f4dd001c3b0 comp rx=0 tx=0).stop 2024-05-28T14:13:42.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.991+0000 7f4de3e00640 1 -- 172.21.15.42:0/237179841 shutdown_connections 2024-05-28T14:13:42.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.991+0000 7f4de3e00640 1 --2- 172.21.15.42:0/237179841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ddc0cf020 0x7f4ddc06b9f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:42.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.991+0000 7f4de3e00640 1 -- 172.21.15.42:0/237179841 >> 172.21.15.42:0/237179841 conn(0x7f4ddc09aa50 msgr2=0x7f4ddc0b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:42.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.991+0000 7f4de3e00640 1 -- 172.21.15.42:0/237179841 shutdown_connections 2024-05-28T14:13:42.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:42.999+0000 7f4de3e00640 1 -- 172.21.15.42:0/237179841 wait complete. 2024-05-28T14:13:42.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.003+0000 7f4de3e00640 1 Processor -- start 2024-05-28T14:13:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.003+0000 7f4de3e00640 1 -- start start 2024-05-28T14:13:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.003+0000 7f4de3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ddc06bf30 0x7f4ddc0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.003+0000 7f4de3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ddc0cf020 0x7f4ddc070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.003+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ddc0cf020 0x7f4ddc070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.003+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ddc0cf020 0x7f4ddc070490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41000/0 (socket says 172.21.15.42:41000) 2024-05-28T14:13:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.003+0000 7f4de1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ddc06bf30 0x7f4ddc0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.003+0000 7f4de1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ddc06bf30 0x7f4ddc0b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38452/0 (socket says 172.21.15.42:38452) 2024-05-28T14:13:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.003+0000 7f4de0a00640 1 -- 172.21.15.42:0/1381561742 learned_addr learned my addr 172.21.15.42:0/1381561742 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.003+0000 7f4de3e00640 1 --2- 172.21.15.42:0/1381561742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ddc162fe0 0x7f4ddc0af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.003+0000 7f4de3e00640 1 -- 172.21.15.42:0/1381561742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc170d00 con 0x7f4ddc0cf020 2024-05-28T14:13:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.003+0000 7f4de3e00640 1 -- 172.21.15.42:0/1381561742 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc170b80 con 0x7f4ddc06bf30 2024-05-28T14:13:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.003+0000 7f4de3e00640 1 -- 172.21.15.42:0/1381561742 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4ddc170e80 con 0x7f4ddc162fe0 2024-05-28T14:13:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.007+0000 7f4de0a00640 1 -- 172.21.15.42:0/1381561742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ddc162fe0 msgr2=0x7f4ddc0af320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-28T14:13:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.007+0000 7f4de0a00640 1 --2- 172.21.15.42:0/1381561742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ddc162fe0 0x7f4ddc0af320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.007+0000 7f4de0a00640 1 -- 172.21.15.42:0/1381561742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ddc06bf30 msgr2=0x7f4ddc0b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.007+0000 7f4de0a00640 1 --2- 172.21.15.42:0/1381561742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ddc06bf30 0x7f4ddc0b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.007+0000 7f4de0a00640 1 -- 172.21.15.42:0/1381561742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc156d50 con 0x7f4ddc0cf020 2024-05-28T14:13:42.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.011+0000 7f4de0a00640 1 --2- 172.21.15.42:0/1381561742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ddc0cf020 0x7f4ddc070490 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f4dd001aa60 tx=0x7f4dd0019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:42.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.011+0000 7f4dd9600640 1 -- 172.21.15.42:0/1381561742 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00b9020 con 0x7f4ddc0cf020 2024-05-28T14:13:42.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.011+0000 7f4dd9600640 1 -- 172.21.15.42:0/1381561742 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd00b1040 con 0x7f4ddc0cf020 2024-05-28T14:13:42.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.011+0000 7f4de3e00640 1 -- 172.21.15.42:0/1381561742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ddc157030 con 0x7f4ddc0cf020 2024-05-28T14:13:42.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.015+0000 7f4dd9600640 1 -- 172.21.15.42:0/1381561742 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00b0050 con 0x7f4ddc0cf020 2024-05-28T14:13:42.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.015+0000 7f4de3e00640 1 -- 172.21.15.42:0/1381561742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ddc0c85c0 con 0x7f4ddc0cf020 2024-05-28T14:13:43.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.019+0000 7f4de3e00640 1 -- 172.21.15.42:0/1381561742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ddc09f650 con 0x7f4ddc0cf020 2024-05-28T14:13:43.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.023+0000 7f4dd9600640 1 -- 172.21.15.42:0/1381561742 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 39) ==== 90180+0+0 (secure 0 0 0) 0x7f4dd00bc020 con 0x7f4ddc0cf020 2024-05-28T14:13:43.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.023+0000 7f4dd9600640 1 --2- 172.21.15.42:0/1381561742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4dbc069c00 0x7f4dbc06c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:43.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.027+0000 7f4de1400640 1 --2- 172.21.15.42:0/1381561742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4dbc069c00 0x7f4dbc06c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:43.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.027+0000 7f4dd9600640 1 -- 172.21.15.42:0/1381561742 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f4dd011fbf0 con 0x7f4ddc0cf020 2024-05-28T14:13:43.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.027+0000 7f4de1400640 1 --2- 172.21.15.42:0/1381561742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4dbc069c00 0x7f4dbc06c0c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400b450 tx=0x7f4dd400d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:43.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.031+0000 7f4dd9600640 1 -- 172.21.15.42:0/1381561742 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4dd00160f0 con 0x7f4ddc0cf020 2024-05-28T14:13:43.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.327+0000 7f4dd9600640 1 -- 172.21.15.42:0/1381561742 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 40) ==== 90384+0+0 (secure 0 0 0) 0x7f4dd00edd00 con 0x7f4ddc0cf020 2024-05-28T14:13:43.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.483+0000 7f4de3e00640 1 -- 172.21.15.42:0/1381561742 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone86013", "target": ["mon-mgr", ""]}) -- 0x7f4ddc0b7120 con 0x7f4dbc069c00 2024-05-28T14:13:43.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.911+0000 7f4dd9600640 1 -- 172.21.15.42:0/1381561742 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4ddc0b7120 con 0x7f4dbc069c00 2024-05-28T14:13:43.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.919+0000 7f4de3e00640 1 -- 172.21.15.42:0/1381561742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4dbc069c00 msgr2=0x7f4dbc06c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:43.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.919+0000 7f4de3e00640 1 --2- 172.21.15.42:0/1381561742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4dbc069c00 0x7f4dbc06c0c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400b450 tx=0x7f4dd400d040 comp rx=0 tx=0).stop 2024-05-28T14:13:43.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.919+0000 7f4de3e00640 1 -- 172.21.15.42:0/1381561742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ddc0cf020 msgr2=0x7f4ddc070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:43.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.919+0000 7f4de3e00640 1 --2- 172.21.15.42:0/1381561742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ddc0cf020 0x7f4ddc070490 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f4dd001aa60 tx=0x7f4dd0019150 comp rx=0 tx=0).stop 2024-05-28T14:13:43.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.923+0000 7f4de3e00640 1 -- 172.21.15.42:0/1381561742 shutdown_connections 2024-05-28T14:13:43.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.923+0000 7f4de3e00640 1 --2- 172.21.15.42:0/1381561742 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4dbc069c00 0x7f4dbc06c0c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:43.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.923+0000 7f4de3e00640 1 --2- 172.21.15.42:0/1381561742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4ddc162fe0 0x7f4ddc0af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:43.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.923+0000 7f4de3e00640 1 --2- 172.21.15.42:0/1381561742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ddc0cf020 0x7f4ddc070490 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:43.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.923+0000 7f4de3e00640 1 --2- 172.21.15.42:0/1381561742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ddc06bf30 0x7f4ddc0b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:43.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.923+0000 7f4de3e00640 1 -- 172.21.15.42:0/1381561742 >> 172.21.15.42:0/1381561742 conn(0x7f4ddc09aa50 msgr2=0x7f4ddc1588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:43.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.923+0000 7f4de3e00640 1 -- 172.21.15.42:0/1381561742 shutdown_connections 2024-05-28T14:13:43.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:43.923+0000 7f4de3e00640 1 -- 172.21.15.42:0/1381561742 wait complete. 2024-05-28T14:13:43.966 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:43.979+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:43.966 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:43.979+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:43.966 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:43.979+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:43.966 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:43.979+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:43.966 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:13:43.979+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:13:44.987 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone86011 2024-05-28T14:13:45.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.403+0000 7fd52ec00640 1 Processor -- start 2024-05-28T14:13:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.407+0000 7fd52ec00640 1 -- start start 2024-05-28T14:13:45.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.407+0000 7fd52ec00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd5280d0320 0x7fd52815cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:45.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.407+0000 7fd52ec00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd5280cd4a0 0x7fd5280d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:45.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.407+0000 7fd52ec00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5281601a0 con 0x7fd5280b3c30 2024-05-28T14:13:45.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.407+0000 7fd52ec00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd528073ba0 con 0x7fd5280cd4a0 2024-05-28T14:13:45.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.407+0000 7fd52ec00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd5280d0c40 con 0x7fd5280d0320 2024-05-28T14:13:45.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.411+0000 7fd52ce00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd5280cd4a0 0x7fd5280d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:45.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.411+0000 7fd52ce00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd5280cd4a0 0x7fd5280d0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41026/0 (socket says 172.21.15.42:41026) 2024-05-28T14:13:45.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.411+0000 7fd52ce00640 1 -- 172.21.15.42:0/157573282 learned_addr learned my addr 172.21.15.42:0/157573282 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:45.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.411+0000 7fd52d800640 1 --2- 172.21.15.42:0/157573282 >> v2:172.21.15.63:3300/0 conn(0x7fd5280d0320 0x7fd52815cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:45.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.411+0000 7fd52ce00640 1 -- 172.21.15.42:0/157573282 >> v2:172.21.15.63:3300/0 conn(0x7fd5280d0320 msgr2=0x7fd52815cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:45.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.411+0000 7fd52ce00640 1 --2- 172.21.15.42:0/157573282 >> v2:172.21.15.63:3300/0 conn(0x7fd5280d0320 0x7fd52815cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:45.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.411+0000 7fd52ce00640 1 -- 172.21.15.42:0/157573282 >> v1:172.21.15.42:6789/0 conn(0x7fd5280b3c30 legacy=0x7fd528160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:45.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.415+0000 7fd52ce00640 1 -- 172.21.15.42:0/157573282 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5280cd920 con 0x7fd5280cd4a0 2024-05-28T14:13:45.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.415+0000 7fd52d800640 1 --2- 172.21.15.42:0/157573282 >> v2:172.21.15.63:3300/0 conn(0x7fd5280d0320 0x7fd52815cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:13:45.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.423+0000 7fd52ce00640 1 --2- 172.21.15.42:0/157573282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5280cd4a0 0x7fd5280d0700 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fd52806fc20 tx=0x7fd51401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f011553843b8e55 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:45.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.423+0000 7fd527e00640 1 -- 172.21.15.42:0/157573282 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5140ab610 con 0x7fd5280cd4a0 2024-05-28T14:13:45.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.427+0000 7fd527e00640 1 -- 172.21.15.42:0/157573282 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5140b0070 con 0x7fd5280cd4a0 2024-05-28T14:13:45.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.427+0000 7fd527e00640 1 -- 172.21.15.42:0/157573282 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5140ab9d0 con 0x7fd5280cd4a0 2024-05-28T14:13:45.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.427+0000 7fd52ec00640 1 -- 172.21.15.42:0/157573282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5280cd4a0 msgr2=0x7fd5280d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:45.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.427+0000 7fd52ec00640 1 --2- 172.21.15.42:0/157573282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5280cd4a0 0x7fd5280d0700 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fd52806fc20 tx=0x7fd51401c520 comp rx=0 tx=0).stop 2024-05-28T14:13:45.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.427+0000 7fd52ec00640 1 -- 172.21.15.42:0/157573282 shutdown_connections 2024-05-28T14:13:45.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.427+0000 7fd52ec00640 1 --2- 172.21.15.42:0/157573282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5280cd4a0 0x7fd5280d0700 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:45.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.427+0000 7fd52ec00640 1 --2- 172.21.15.42:0/157573282 >> v2:172.21.15.63:3300/0 conn(0x7fd5280d0320 0x7fd52815cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:45.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.427+0000 7fd52ec00640 1 -- 172.21.15.42:0/157573282 >> 172.21.15.42:0/157573282 conn(0x7fd52806bbc0 msgr2=0x7fd52806bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:45.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.427+0000 7fd52ec00640 1 -- 172.21.15.42:0/157573282 shutdown_connections 2024-05-28T14:13:45.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.431+0000 7fd52ec00640 1 -- 172.21.15.42:0/157573282 wait complete. 2024-05-28T14:13:45.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.435+0000 7fd52ec00640 1 Processor -- start 2024-05-28T14:13:45.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.435+0000 7fd52ec00640 1 -- start start 2024-05-28T14:13:45.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.435+0000 7fd52ec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5280b3c30 0x7fd528152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:45.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.435+0000 7fd52ec00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5280cd4a0 0x7fd5280ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:45.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.435+0000 7fd52ec00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5280d0320 0x7fd52815eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:45.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.435+0000 7fd52ec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd5280b5030 con 0x7fd5280b3c30 2024-05-28T14:13:45.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.435+0000 7fd52ec00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5280b4450 con 0x7fd5280cd4a0 2024-05-28T14:13:45.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.435+0000 7fd52ec00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd52815da00 con 0x7fd5280d0320 2024-05-28T14:13:45.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.439+0000 7fd52d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5280b3c30 0x7fd528152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:45.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.439+0000 7fd52ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5280cd4a0 0x7fd5280ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:45.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.439+0000 7fd52e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5280d0320 0x7fd52815eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:45.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.439+0000 7fd52d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5280b3c30 0x7fd528152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41030/0 (socket says 172.21.15.42:41030) 2024-05-28T14:13:45.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.439+0000 7fd52d800640 1 -- 172.21.15.42:0/1534357602 learned_addr learned my addr 172.21.15.42:0/1534357602 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:45.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.439+0000 7fd52e200640 1 -- 172.21.15.42:0/1534357602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5280cd4a0 msgr2=0x7fd5280ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:45.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.439+0000 7fd52e200640 1 --2- 172.21.15.42:0/1534357602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5280cd4a0 0x7fd5280ad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:45.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.439+0000 7fd52e200640 1 -- 172.21.15.42:0/1534357602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5280b3c30 msgr2=0x7fd528152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:45.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.439+0000 7fd52e200640 1 --2- 172.21.15.42:0/1534357602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5280b3c30 0x7fd528152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:45.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.439+0000 7fd52e200640 1 -- 172.21.15.42:0/1534357602 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd528152b50 con 0x7fd5280d0320 2024-05-28T14:13:45.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.443+0000 7fd52e200640 1 --2- 172.21.15.42:0/1534357602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5280d0320 0x7fd52815eeb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c3c0 tx=0x7fd51c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:45.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.443+0000 7fd525600640 1 -- 172.21.15.42:0/1534357602 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c017070 con 0x7fd5280d0320 2024-05-28T14:13:45.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.443+0000 7fd525600640 1 -- 172.21.15.42:0/1534357602 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51c0052d0 con 0x7fd5280d0320 2024-05-28T14:13:45.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.443+0000 7fd525600640 1 -- 172.21.15.42:0/1534357602 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c011890 con 0x7fd5280d0320 2024-05-28T14:13:45.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.443+0000 7fd52ec00640 1 -- 172.21.15.42:0/1534357602 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5280ada90 con 0x7fd5280d0320 2024-05-28T14:13:45.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.447+0000 7fd52ec00640 1 -- 172.21.15.42:0/1534357602 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5280ae050 con 0x7fd5280d0320 2024-05-28T14:13:45.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.447+0000 7fd52ec00640 1 -- 172.21.15.42:0/1534357602 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd52804fcf0 con 0x7fd5280d0320 2024-05-28T14:13:45.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.451+0000 7fd525600640 1 -- 172.21.15.42:0/1534357602 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7fd51c01a070 con 0x7fd5280d0320 2024-05-28T14:13:45.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.455+0000 7fd525600640 1 --2- 172.21.15.42:0/1534357602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd508069ed0 0x7fd50806c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:45.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.455+0000 7fd525600640 1 -- 172.21.15.42:0/1534357602 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7fd51c084f60 con 0x7fd5280d0320 2024-05-28T14:13:45.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.459+0000 7fd525600640 1 -- 172.21.15.42:0/1534357602 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd51c021020 con 0x7fd5280d0320 2024-05-28T14:13:45.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.463+0000 7fd52d800640 1 --2- 172.21.15.42:0/1534357602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd508069ed0 0x7fd50806c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:45.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.495+0000 7fd52d800640 1 --2- 172.21.15.42:0/1534357602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd508069ed0 0x7fd50806c390 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd52000f5e0 tx=0x7fd520007830 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:45.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:45.983+0000 7fd52ec00640 1 -- 172.21.15.42:0/1534357602 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone86011", "target": ["mon-mgr", ""]}) -- 0x7fd52816a420 con 0x7fd508069ed0 2024-05-28T14:13:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.395+0000 7fd525600640 1 -- 172.21.15.42:0/1534357602 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd52816a420 con 0x7fd508069ed0 2024-05-28T14:13:46.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.407+0000 7fd52ec00640 1 -- 172.21.15.42:0/1534357602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd508069ed0 msgr2=0x7fd50806c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:46.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.407+0000 7fd52ec00640 1 --2- 172.21.15.42:0/1534357602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd508069ed0 0x7fd50806c390 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd52000f5e0 tx=0x7fd520007830 comp rx=0 tx=0).stop 2024-05-28T14:13:46.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.407+0000 7fd52ec00640 1 -- 172.21.15.42:0/1534357602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5280d0320 msgr2=0x7fd52815eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:46.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.407+0000 7fd52ec00640 1 --2- 172.21.15.42:0/1534357602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5280d0320 0x7fd52815eeb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c3c0 tx=0x7fd51c00c880 comp rx=0 tx=0).stop 2024-05-28T14:13:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.407+0000 7fd52ec00640 1 -- 172.21.15.42:0/1534357602 shutdown_connections 2024-05-28T14:13:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.407+0000 7fd52ec00640 1 --2- 172.21.15.42:0/1534357602 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd508069ed0 0x7fd50806c390 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.407+0000 7fd52ec00640 1 --2- 172.21.15.42:0/1534357602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5280d0320 0x7fd52815eeb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.407+0000 7fd52ec00640 1 --2- 172.21.15.42:0/1534357602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5280cd4a0 0x7fd5280ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.407+0000 7fd52ec00640 1 --2- 172.21.15.42:0/1534357602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5280b3c30 0x7fd528152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.407+0000 7fd52ec00640 1 -- 172.21.15.42:0/1534357602 >> 172.21.15.42:0/1534357602 conn(0x7fd52806bbc0 msgr2=0x7fd5280b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.411+0000 7fd52ec00640 1 -- 172.21.15.42:0/1534357602 shutdown_connections 2024-05-28T14:13:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.411+0000 7fd52ec00640 1 -- 172.21.15.42:0/1534357602 wait complete. 2024-05-28T14:13:46.492 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone86012 2024-05-28T14:13:46.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.871+0000 7efd22600640 1 Processor -- start 2024-05-28T14:13:46.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.871+0000 7efd22600640 1 -- start start 2024-05-28T14:13:46.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.871+0000 7efd22600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efd1c15e9d0 0x7efd1c0ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:46.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.871+0000 7efd21200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efd1c15e9d0 0x7efd1c0ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:46.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.875+0000 7efd21200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efd1c15e9d0 0x7efd1c0ae050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41042/0 (socket says 172.21.15.42:41042) 2024-05-28T14:13:46.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.875+0000 7efd21200640 1 -- 172.21.15.42:0/457604068 learned_addr learned my addr 172.21.15.42:0/457604068 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:46.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.875+0000 7efd22600640 1 --2- 172.21.15.42:0/457604068 >> v2:172.21.15.63:3300/0 conn(0x7efd1c06ec30 0x7efd1c06f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:46.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.875+0000 7efd22600640 1 -- 172.21.15.42:0/457604068 --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd1c070000 con 0x7efd1c0b3860 2024-05-28T14:13:46.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.875+0000 7efd22600640 1 -- 172.21.15.42:0/457604068 --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7efd1c0cf7c0 con 0x7efd1c15e9d0 2024-05-28T14:13:46.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.875+0000 7efd22600640 1 -- 172.21.15.42:0/457604068 --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efd1c0b0a80 con 0x7efd1c06ec30 2024-05-28T14:13:46.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.875+0000 7efd21c00640 1 --2- 172.21.15.42:0/457604068 >> v2:172.21.15.63:3300/0 conn(0x7efd1c06ec30 0x7efd1c06f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:46.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.875+0000 7efd21200640 1 -- 172.21.15.42:0/457604068 >> v2:172.21.15.63:3300/0 conn(0x7efd1c06ec30 msgr2=0x7efd1c06f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:46.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.875+0000 7efd21200640 1 --2- 172.21.15.42:0/457604068 >> v2:172.21.15.63:3300/0 conn(0x7efd1c06ec30 0x7efd1c06f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:46.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.875+0000 7efd21200640 1 -- 172.21.15.42:0/457604068 >> v1:172.21.15.42:6789/0 conn(0x7efd1c0b3860 legacy=0x7efd1c0ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:46.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.875+0000 7efd21200640 1 -- 172.21.15.42:0/457604068 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c070270 con 0x7efd1c15e9d0 2024-05-28T14:13:46.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.875+0000 7efd21c00640 1 --2- 172.21.15.42:0/457604068 >> v2:172.21.15.63:3300/0 conn(0x7efd1c06ec30 0x7efd1c06f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:13:46.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.879+0000 7efd21200640 1 --2- 172.21.15.42:0/457604068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd1c15e9d0 0x7efd1c0ae050 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0cf680 tx=0x7efd0801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb2770e32864a9d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:46.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.879+0000 7efd1b400640 1 -- 172.21.15.42:0/457604068 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd080b2070 con 0x7efd1c15e9d0 2024-05-28T14:13:46.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.879+0000 7efd1b400640 1 -- 172.21.15.42:0/457604068 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd080ad680 con 0x7efd1c15e9d0 2024-05-28T14:13:46.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.883+0000 7efd1b400640 1 -- 172.21.15.42:0/457604068 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd080b2070 con 0x7efd1c15e9d0 2024-05-28T14:13:46.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.883+0000 7efd22600640 1 -- 172.21.15.42:0/457604068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd1c15e9d0 msgr2=0x7efd1c0ae050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:46.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.883+0000 7efd22600640 1 --2- 172.21.15.42:0/457604068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd1c15e9d0 0x7efd1c0ae050 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0cf680 tx=0x7efd0801c490 comp rx=0 tx=0).stop 2024-05-28T14:13:46.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.883+0000 7efd22600640 1 -- 172.21.15.42:0/457604068 shutdown_connections 2024-05-28T14:13:46.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.883+0000 7efd22600640 1 --2- 172.21.15.42:0/457604068 >> v2:172.21.15.63:3300/0 conn(0x7efd1c06ec30 0x7efd1c06f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:46.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.883+0000 7efd22600640 1 --2- 172.21.15.42:0/457604068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd1c15e9d0 0x7efd1c0ae050 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:46.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.883+0000 7efd22600640 1 -- 172.21.15.42:0/457604068 >> 172.21.15.42:0/457604068 conn(0x7efd1c0b2d70 msgr2=0x7efd1c0b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:46.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.883+0000 7efd22600640 1 -- 172.21.15.42:0/457604068 shutdown_connections 2024-05-28T14:13:46.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.883+0000 7efd22600640 1 -- 172.21.15.42:0/457604068 wait complete. 2024-05-28T14:13:46.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.891+0000 7efd22600640 1 Processor -- start 2024-05-28T14:13:46.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.891+0000 7efd22600640 1 -- start start 2024-05-28T14:13:46.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.891+0000 7efd22600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd1c06ec30 0x7efd1c1ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:46.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.895+0000 7efd22600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c0b3860 0x7efd1c0b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:46.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.895+0000 7efd22600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd1c15e9d0 0x7efd1c0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:46.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.895+0000 7efd22600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c0aee20 con 0x7efd1c15e9d0 2024-05-28T14:13:46.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.895+0000 7efd22600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c0aeca0 con 0x7efd1c0b3860 2024-05-28T14:13:46.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.895+0000 7efd22600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd1c15edb0 con 0x7efd1c06ec30 2024-05-28T14:13:46.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.895+0000 7efd21200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd1c06ec30 0x7efd1c1ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:46.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.895+0000 7efd1be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c0b3860 0x7efd1c0b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:46.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.895+0000 7efd1be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c0b3860 0x7efd1c0b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38506/0 (socket says 172.21.15.42:38506) 2024-05-28T14:13:46.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.895+0000 7efd1be00640 1 -- 172.21.15.42:0/1562696483 learned_addr learned my addr 172.21.15.42:0/1562696483 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:46.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.895+0000 7efd21c00640 1 --2- 172.21.15.42:0/1562696483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd1c15e9d0 0x7efd1c0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:46.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.895+0000 7efd21200640 1 -- 172.21.15.42:0/1562696483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c0b3860 msgr2=0x7efd1c0b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:46.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.895+0000 7efd21200640 1 --2- 172.21.15.42:0/1562696483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c0b3860 0x7efd1c0b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:46.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.899+0000 7efd21200640 1 -- 172.21.15.42:0/1562696483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd1c15e9d0 msgr2=0x7efd1c0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:46.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.899+0000 7efd21200640 1 --2- 172.21.15.42:0/1562696483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd1c15e9d0 0x7efd1c0b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:46.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.899+0000 7efd21200640 1 -- 172.21.15.42:0/1562696483 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c200070 con 0x7efd1c06ec30 2024-05-28T14:13:46.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.899+0000 7efd21200640 1 --2- 172.21.15.42:0/1562696483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd1c06ec30 0x7efd1c1ffb30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd080ac000 tx=0x7efd08019230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.899+0000 7efd18c00640 1 -- 172.21.15.42:0/1562696483 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd080bd070 con 0x7efd1c06ec30 2024-05-28T14:13:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.899+0000 7efd18c00640 1 -- 172.21.15.42:0/1562696483 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd080ada70 con 0x7efd1c06ec30 2024-05-28T14:13:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.899+0000 7efd18c00640 1 -- 172.21.15.42:0/1562696483 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd080b2070 con 0x7efd1c06ec30 2024-05-28T14:13:46.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.903+0000 7efd22600640 1 -- 172.21.15.42:0/1562696483 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c0c9bc0 con 0x7efd1c06ec30 2024-05-28T14:13:46.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.903+0000 7efd22600640 1 -- 172.21.15.42:0/1562696483 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c0ca180 con 0x7efd1c06ec30 2024-05-28T14:13:46.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.903+0000 7efd22600640 1 -- 172.21.15.42:0/1562696483 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd1c04fcf0 con 0x7efd1c06ec30 2024-05-28T14:13:46.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.907+0000 7efd18c00640 1 -- 172.21.15.42:0/1562696483 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7efd080c0070 con 0x7efd1c06ec30 2024-05-28T14:13:46.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.911+0000 7efd18c00640 1 --2- 172.21.15.42:0/1562696483 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efcfc069f50 0x7efcfc06c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:46.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.911+0000 7efd18c00640 1 -- 172.21.15.42:0/1562696483 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7efd0811f9f0 con 0x7efd1c06ec30 2024-05-28T14:13:46.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.915+0000 7efd1be00640 1 --2- 172.21.15.42:0/1562696483 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efcfc069f50 0x7efcfc06c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:46.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.915+0000 7efd1be00640 1 --2- 172.21.15.42:0/1562696483 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efcfc069f50 0x7efcfc06c410 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7efd1c15be80 tx=0x7efd10009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:46.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:46.919+0000 7efd18c00640 1 -- 172.21.15.42:0/1562696483 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efd08016110 con 0x7efd1c06ec30 2024-05-28T14:13:47.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:47.479+0000 7efd22600640 1 -- 172.21.15.42:0/1562696483 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone86012", "target": ["mon-mgr", ""]}) -- 0x7efd1c1654d0 con 0x7efcfc069f50 2024-05-28T14:13:47.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:47.859+0000 7efd18c00640 1 -- 172.21.15.42:0/1562696483 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efd1c1654d0 con 0x7efcfc069f50 2024-05-28T14:13:47.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:47.867+0000 7efd22600640 1 -- 172.21.15.42:0/1562696483 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efcfc069f50 msgr2=0x7efcfc06c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:47.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:47.867+0000 7efd22600640 1 --2- 172.21.15.42:0/1562696483 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efcfc069f50 0x7efcfc06c410 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7efd1c15be80 tx=0x7efd10009040 comp rx=0 tx=0).stop 2024-05-28T14:13:47.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:47.867+0000 7efd22600640 1 -- 172.21.15.42:0/1562696483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd1c06ec30 msgr2=0x7efd1c1ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:47.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:47.867+0000 7efd22600640 1 --2- 172.21.15.42:0/1562696483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd1c06ec30 0x7efd1c1ffb30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd080ac000 tx=0x7efd08019230 comp rx=0 tx=0).stop 2024-05-28T14:13:47.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:47.867+0000 7efd22600640 1 -- 172.21.15.42:0/1562696483 shutdown_connections 2024-05-28T14:13:47.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:47.867+0000 7efd22600640 1 --2- 172.21.15.42:0/1562696483 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efcfc069f50 0x7efcfc06c410 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:47.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:47.867+0000 7efd22600640 1 --2- 172.21.15.42:0/1562696483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd1c15e9d0 0x7efd1c0b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:47.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:47.867+0000 7efd22600640 1 --2- 172.21.15.42:0/1562696483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd1c0b3860 0x7efd1c0b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:47.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:47.867+0000 7efd22600640 1 --2- 172.21.15.42:0/1562696483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd1c06ec30 0x7efd1c1ffb30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:47.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:47.867+0000 7efd22600640 1 -- 172.21.15.42:0/1562696483 >> 172.21.15.42:0/1562696483 conn(0x7efd1c0b2d70 msgr2=0x7efd1c0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:47.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:47.867+0000 7efd22600640 1 -- 172.21.15.42:0/1562696483 shutdown_connections 2024-05-28T14:13:47.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:47.867+0000 7efd22600640 1 -- 172.21.15.42:0/1562696483 wait complete. 2024-05-28T14:13:47.913 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone86013 2024-05-28T14:13:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.219+0000 7f1579400640 1 Processor -- start 2024-05-28T14:13:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.219+0000 7f1579400640 1 -- start start 2024-05-28T14:13:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.219+0000 7f1579400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15740b7d70 0x7f15740cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.219+0000 7f1579400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15740d1a80 con 0x7f157415ec50 2024-05-28T14:13:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.219+0000 7f1579400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15740ccbd0 con 0x7f157415e870 2024-05-28T14:13:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.219+0000 7f1579400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f15740cd500 con 0x7f15740b7d70 2024-05-28T14:13:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.219+0000 7f1569e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f157415ec50 0x7f15740b7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39732/0 (socket says 172.21.15.42:39732) 2024-05-28T14:13:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.219+0000 7f1569e00640 1 -- 172.21.15.42:0/2049537522 learned_addr learned my addr 172.21.15.42:0/2049537522 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:13:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.219+0000 7f1573200640 1 --2- 172.21.15.42:0/2049537522 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15740b7d70 0x7f15740cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.223+0000 7f1571e00640 1 -- 172.21.15.42:0/2049537522 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2177697410 0 0) 0x7f15740ccbd0 con 0x7f157415e870 2024-05-28T14:13:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.223+0000 7f1571e00640 1 -- 172.21.15.42:0/2049537522 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1558003530 con 0x7f157415e870 2024-05-28T14:13:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.223+0000 7f1571e00640 1 -- 172.21.15.42:0/2049537522 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3082946598 0 0) 0x7f15740d1a80 con 0x7f157415ec50 2024-05-28T14:13:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.223+0000 7f1571e00640 1 -- 172.21.15.42:0/2049537522 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15740ccbd0 con 0x7f157415ec50 2024-05-28T14:13:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.223+0000 7f1573200640 1 -- 172.21.15.42:0/2049537522 >> v1:172.21.15.63:6789/0 conn(0x7f157415e870 legacy=0x7f15740d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:48.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.223+0000 7f1573200640 1 -- 172.21.15.42:0/2049537522 >> v1:172.21.15.42:6789/0 conn(0x7f157415ec50 legacy=0x7f15740b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:48.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.223+0000 7f1573200640 1 -- 172.21.15.42:0/2049537522 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15740cceb0 con 0x7f15740b7d70 2024-05-28T14:13:48.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.227+0000 7f1573200640 1 --2- 172.21.15.42:0/2049537522 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15740b7d70 0x7f15740cd990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f156c0097f0 tx=0x7f156c01ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efdf449496a8ac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:48.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.227+0000 7f1571e00640 1 -- 172.21.15.42:0/2049537522 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c0b5070 con 0x7f15740b7d70 2024-05-28T14:13:48.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.227+0000 7f1571e00640 1 -- 172.21.15.42:0/2049537522 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f156c0a89d0 con 0x7f15740b7d70 2024-05-28T14:13:48.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.227+0000 7f1571e00640 1 -- 172.21.15.42:0/2049537522 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c0a8da0 con 0x7f15740b7d70 2024-05-28T14:13:48.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.227+0000 7f1579400640 1 -- 172.21.15.42:0/2049537522 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15740b7d70 msgr2=0x7f15740cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:48.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.227+0000 7f1579400640 1 --2- 172.21.15.42:0/2049537522 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15740b7d70 0x7f15740cd990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f156c0097f0 tx=0x7f156c01ecd0 comp rx=0 tx=0).stop 2024-05-28T14:13:48.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.227+0000 7f1579400640 1 -- 172.21.15.42:0/2049537522 shutdown_connections 2024-05-28T14:13:48.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.227+0000 7f1579400640 1 --2- 172.21.15.42:0/2049537522 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15740b7d70 0x7f15740cd990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:48.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.227+0000 7f1579400640 1 -- 172.21.15.42:0/2049537522 >> 172.21.15.42:0/2049537522 conn(0x7f15740b3fd0 msgr2=0x7f15740b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:48.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.227+0000 7f1579400640 1 -- 172.21.15.42:0/2049537522 shutdown_connections 2024-05-28T14:13:48.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.227+0000 7f1579400640 1 -- 172.21.15.42:0/2049537522 wait complete. 2024-05-28T14:13:48.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.227+0000 7f1579400640 1 Processor -- start 2024-05-28T14:13:48.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.227+0000 7f1579400640 1 -- start start 2024-05-28T14:13:48.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1579400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15740b7d70 0x7f157414e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:48.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1579400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f157415e870 0x7f157415fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:48.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1579400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f157415ec50 0x7f15740cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:48.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1579400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f15740b5330 con 0x7f157415ec50 2024-05-28T14:13:48.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1579400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f15740b81e0 con 0x7f15740b7d70 2024-05-28T14:13:48.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1579400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f157414eee0 con 0x7f157415e870 2024-05-28T14:13:48.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1569e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f157415e870 0x7f157415fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:48.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1573200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f157415ec50 0x7f15740cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:48.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1572800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15740b7d70 0x7f157414e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:48.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1573200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f157415ec50 0x7f15740cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34264/0 (socket says 172.21.15.42:34264) 2024-05-28T14:13:48.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1573200640 1 -- 172.21.15.42:0/3996836660 learned_addr learned my addr 172.21.15.42:0/3996836660 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:48.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1569e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f157415e870 0x7f157415fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:53686/0 (socket says 172.21.15.42:53686) 2024-05-28T14:13:48.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1572800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15740b7d70 0x7f157414e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:44116/0 (socket says 172.21.15.42:44116) 2024-05-28T14:13:48.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1573200640 1 -- 172.21.15.42:0/3996836660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f157415e870 msgr2=0x7f157415fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:48.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1573200640 1 --2- 172.21.15.42:0/3996836660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f157415e870 0x7f157415fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:48.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1573200640 1 -- 172.21.15.42:0/3996836660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15740b7d70 msgr2=0x7f157414e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1573200640 1 --2- 172.21.15.42:0/3996836660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15740b7d70 0x7f157414e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1573200640 1 -- 172.21.15.42:0/3996836660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15741602c0 con 0x7f157415ec50 2024-05-28T14:13:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1573200640 1 --2- 172.21.15.42:0/3996836660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f157415ec50 0x7f15740cb300 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f156c009740 tx=0x7f156c01ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f156b400640 1 -- 172.21.15.42:0/3996836660 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c0bf070 con 0x7f157415ec50 2024-05-28T14:13:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f156b400640 1 -- 172.21.15.42:0/3996836660 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f156c0b0640 con 0x7f157415ec50 2024-05-28T14:13:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f156b400640 1 -- 172.21.15.42:0/3996836660 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c0b5040 con 0x7f157415ec50 2024-05-28T14:13:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1579400640 1 -- 172.21.15.42:0/3996836660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15741605a0 con 0x7f157415ec50 2024-05-28T14:13:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1579400640 1 -- 172.21.15.42:0/3996836660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1574160b60 con 0x7f157415ec50 2024-05-28T14:13:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.231+0000 7f1579400640 1 -- 172.21.15.42:0/3996836660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f157404fdd0 con 0x7f157415ec50 2024-05-28T14:13:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.235+0000 7f156b400640 1 -- 172.21.15.42:0/3996836660 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7f156c0b0a30 con 0x7f157415ec50 2024-05-28T14:13:48.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.235+0000 7f156b400640 1 --2- 172.21.15.42:0/3996836660 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1550069eb0 0x7f155006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:48.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.235+0000 7f156b400640 1 -- 172.21.15.42:0/3996836660 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f156c122090 con 0x7f157415ec50 2024-05-28T14:13:48.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.235+0000 7f1572800640 1 --2- 172.21.15.42:0/3996836660 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1550069eb0 0x7f155006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:48.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.235+0000 7f1572800640 1 --2- 172.21.15.42:0/3996836660 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1550069eb0 0x7f155006c370 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f15640013a0 tx=0x7f1564009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:48.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.239+0000 7f156b400640 1 -- 172.21.15.42:0/3996836660 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f156c018830 con 0x7f157415ec50 2024-05-28T14:13:48.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.607+0000 7f1579400640 1 -- 172.21.15.42:0/3996836660 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone86013", "target": ["mon-mgr", ""]}) -- 0x7f157415dea0 con 0x7f1550069eb0 2024-05-28T14:13:48.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.755+0000 7f156b400640 1 -- 172.21.15.42:0/3996836660 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f157415dea0 con 0x7f1550069eb0 2024-05-28T14:13:48.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.755+0000 7f1579400640 1 -- 172.21.15.42:0/3996836660 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1550069eb0 msgr2=0x7f155006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:48.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.755+0000 7f1579400640 1 --2- 172.21.15.42:0/3996836660 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1550069eb0 0x7f155006c370 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f15640013a0 tx=0x7f1564009040 comp rx=0 tx=0).stop 2024-05-28T14:13:48.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.755+0000 7f1579400640 1 -- 172.21.15.42:0/3996836660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f157415ec50 msgr2=0x7f15740cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:48.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.755+0000 7f1579400640 1 --2- 172.21.15.42:0/3996836660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f157415ec50 0x7f15740cb300 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f156c009740 tx=0x7f156c01ba70 comp rx=0 tx=0).stop 2024-05-28T14:13:48.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.755+0000 7f1579400640 1 -- 172.21.15.42:0/3996836660 shutdown_connections 2024-05-28T14:13:48.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.755+0000 7f1579400640 1 --2- 172.21.15.42:0/3996836660 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1550069eb0 0x7f155006c370 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:48.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.755+0000 7f1579400640 1 --2- 172.21.15.42:0/3996836660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f157415ec50 0x7f15740cb300 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:48.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.755+0000 7f1579400640 1 --2- 172.21.15.42:0/3996836660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f157415e870 0x7f157415fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:48.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.755+0000 7f1579400640 1 --2- 172.21.15.42:0/3996836660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15740b7d70 0x7f157414e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:48.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.755+0000 7f1579400640 1 -- 172.21.15.42:0/3996836660 >> 172.21.15.42:0/3996836660 conn(0x7f15740b3fd0 msgr2=0x7f15740b6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:48.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.755+0000 7f1579400640 1 -- 172.21.15.42:0/3996836660 shutdown_connections 2024-05-28T14:13:48.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:48.755+0000 7f1579400640 1 -- 172.21.15.42:0/3996836660 wait complete. 2024-05-28T14:13:51.034 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:13:51.043 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:13:53.761 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:13:53.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.927+0000 7f6a55c00640 1 Processor -- start 2024-05-28T14:13:53.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.927+0000 7f6a55c00640 1 -- start start 2024-05-28T14:13:53.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a55c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6a50156a20 0x7f6a5006b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:53.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a55c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6a50156e00 0x7f6a5006bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:53.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a55c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a500afb20 con 0x7f6a500aee40 2024-05-28T14:13:53.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a55c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6a500b4d20 con 0x7f6a50156e00 2024-05-28T14:13:53.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a55c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6a5015cc90 con 0x7f6a50156a20 2024-05-28T14:13:53.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a4f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6a50156e00 0x7f6a5006bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a4f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6a50156e00 0x7f6a5006bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34276/0 (socket says 172.21.15.42:34276) 2024-05-28T14:13:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a55200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f6a500aee40 0x7f6a500af260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39738/0 (socket says 172.21.15.42:39738) 2024-05-28T14:13:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a4f400640 1 -- 172.21.15.42:0/3935246012 learned_addr learned my addr 172.21.15.42:0/3935246012 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a4fe00640 1 --2- 172.21.15.42:0/3935246012 >> v2:172.21.15.63:3300/0 conn(0x7f6a50156a20 0x7f6a5006b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a4f400640 1 -- 172.21.15.42:0/3935246012 >> v2:172.21.15.63:3300/0 conn(0x7f6a50156a20 msgr2=0x7f6a5006b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a4f400640 1 --2- 172.21.15.42:0/3935246012 >> v2:172.21.15.63:3300/0 conn(0x7f6a50156a20 0x7f6a5006b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a4f400640 1 -- 172.21.15.42:0/3935246012 >> v1:172.21.15.42:6789/0 conn(0x7f6a500aee40 legacy=0x7f6a500af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:53.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a4f400640 1 -- 172.21.15.42:0/3935246012 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5015cf70 con 0x7f6a50156e00 2024-05-28T14:13:53.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a4ea00640 1 -- 172.21.15.42:0/3935246012 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 989027357 0 0) 0x7f6a500afb20 con 0x7f6a500aee40 2024-05-28T14:13:53.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a4fe00640 1 --2- 172.21.15.42:0/3935246012 >> v2:172.21.15.63:3300/0 conn(0x7f6a50156a20 0x7f6a5006b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:13:53.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.931+0000 7f6a4f400640 1 --2- 172.21.15.42:0/3935246012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a50156e00 0x7f6a5006bf30 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f6a50074090 tx=0x7f6a3c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa94dccc9c088d6e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:53.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a4ea00640 1 -- 172.21.15.42:0/3935246012 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0b2070 con 0x7f6a50156e00 2024-05-28T14:13:53.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a4ea00640 1 -- 172.21.15.42:0/3935246012 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3c0ad680 con 0x7f6a50156e00 2024-05-28T14:13:53.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a4ea00640 1 -- 172.21.15.42:0/3935246012 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0ada50 con 0x7f6a50156e00 2024-05-28T14:13:53.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 -- 172.21.15.42:0/3935246012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a50156e00 msgr2=0x7f6a5006bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:53.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 --2- 172.21.15.42:0/3935246012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a50156e00 0x7f6a5006bf30 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f6a50074090 tx=0x7f6a3c01c490 comp rx=0 tx=0).stop 2024-05-28T14:13:53.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 -- 172.21.15.42:0/3935246012 shutdown_connections 2024-05-28T14:13:53.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 --2- 172.21.15.42:0/3935246012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a50156e00 0x7f6a5006bf30 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:53.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 --2- 172.21.15.42:0/3935246012 >> v2:172.21.15.63:3300/0 conn(0x7f6a50156a20 0x7f6a5006b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:53.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 -- 172.21.15.42:0/3935246012 >> 172.21.15.42:0/3935246012 conn(0x7f6a500b1fa0 msgr2=0x7f6a500b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:53.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 -- 172.21.15.42:0/3935246012 shutdown_connections 2024-05-28T14:13:53.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 -- 172.21.15.42:0/3935246012 wait complete. 2024-05-28T14:13:53.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 Processor -- start 2024-05-28T14:13:53.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 -- start start 2024-05-28T14:13:53.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a500aee40 0x7f6a501549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:53.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a50156a20 0x7f6a5014c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:53.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a50156e00 0x7f6a50071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:53.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6a500b3a60 con 0x7f6a50156a20 2024-05-28T14:13:53.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6a500b38e0 con 0x7f6a50156e00 2024-05-28T14:13:53.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a55c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6a500b3be0 con 0x7f6a500aee40 2024-05-28T14:13:53.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a4f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a50156a20 0x7f6a5014c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:53.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a4f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a50156a20 0x7f6a5014c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34280/0 (socket says 172.21.15.42:34280) 2024-05-28T14:13:53.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.935+0000 7f6a4f400640 1 -- 172.21.15.42:0/1699858518 learned_addr learned my addr 172.21.15.42:0/1699858518 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:53.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a4fe00640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a500aee40 0x7f6a501549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:53.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a4f400640 1 -- 172.21.15.42:0/1699858518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a500aee40 msgr2=0x7f6a501549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:53.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a4f400640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a500aee40 0x7f6a501549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:53.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a4f400640 1 -- 172.21.15.42:0/1699858518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a50156e00 msgr2=0x7f6a50071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:53.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a55200640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a50156e00 0x7f6a50071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:53.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a4f400640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a50156e00 0x7f6a50071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:53.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a4f400640 1 -- 172.21.15.42:0/1699858518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5015dcf0 con 0x7f6a50156a20 2024-05-28T14:13:53.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a4fe00640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a500aee40 0x7f6a501549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:13:53.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a55200640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a50156e00 0x7f6a50071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:13:53.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a4f400640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a50156a20 0x7f6a5014c110 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0190a0 tx=0x7f6a3c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a47e00640 1 -- 172.21.15.42:0/1699858518 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0b2040 con 0x7f6a50156a20 2024-05-28T14:13:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a47e00640 1 -- 172.21.15.42:0/1699858518 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3c0ada70 con 0x7f6a50156a20 2024-05-28T14:13:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a47e00640 1 -- 172.21.15.42:0/1699858518 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0ade20 con 0x7f6a50156a20 2024-05-28T14:13:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a55c00640 1 -- 172.21.15.42:0/1699858518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5015dfd0 con 0x7f6a50156a20 2024-05-28T14:13:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a55c00640 1 -- 172.21.15.42:0/1699858518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5015e590 con 0x7f6a50156a20 2024-05-28T14:13:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a55c00640 1 -- 172.21.15.42:0/1699858518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a50099670 con 0x7f6a50156a20 2024-05-28T14:13:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.939+0000 7f6a47e00640 1 -- 172.21.15.42:0/1699858518 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7f6a3c0bf030 con 0x7f6a50156a20 2024-05-28T14:13:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.943+0000 7f6a47e00640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a30069e80 0x7f6a3006c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:53.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.943+0000 7f6a47e00640 1 -- 172.21.15.42:0/1699858518 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f6a3c11f350 con 0x7f6a50156a20 2024-05-28T14:13:53.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.943+0000 7f6a4fe00640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a30069e80 0x7f6a3006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:53.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.943+0000 7f6a4fe00640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a30069e80 0x7f6a3006c340 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009980 tx=0x7f6a40008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:53.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:53.947+0000 7f6a47e00640 1 -- 172.21.15.42:0/1699858518 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6a50099670 con 0x7f6a50156a20 2024-05-28T14:13:54.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.171+0000 7f6a55c00640 1 -- 172.21.15.42:0/1699858518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6a50068a90 con 0x7f6a50156a20 2024-05-28T14:13:54.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.171+0000 7f6a47e00640 1 -- 172.21.15.42:0/1699858518 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2124 (secure 0 0 0) 0x7f6a3c0f7070 con 0x7f6a50156a20 2024-05-28T14:13:54.156 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:54.157 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:13:54.157 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:13:54.157 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:13:54.157 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:13:54.157 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:13:54.157 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:13:54.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:54.157 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:13:54.157 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:13:54.157 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:13:54.157 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:13:54.158 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:13:54.158 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:13:54.158 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:13:54.158 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:13:54.158 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:13:54.158 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:13:54.158 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:13:54.158 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 955, 2024-05-28T14:13:54.158 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:13:54.158 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:13:54.158 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:13:54.158 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:13:54.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:54.159 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:13:54.159 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 117, 2024-05-28T14:13:54.159 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:13:54.159 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:13:54.159 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:13:54.159 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:13:54.159 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:13:54.159 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:13:54.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:54.159 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:13:54.159 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:13:54.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:13:54.159 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:13:54.160 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:13:54.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:13:54.160 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:13:54.160 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:13:54.160 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:13:54.160 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1781, 2024-05-28T14:13:54.160 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 7316116059, 2024-05-28T14:13:54.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 15623352320, 2024-05-28T14:13:54.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 757470760960, 2024-05-28T14:13:54.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:13:54.160 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 345034765, 2024-05-28T14:13:54.160 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 340887149, 2024-05-28T14:13:54.160 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 85, 2024-05-28T14:13:54.160 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 95 2024-05-28T14:13:54.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:54.161 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:13:54.161 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 75, 2024-05-28T14:13:54.161 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 8, 2024-05-28T14:13:54.161 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:13:54.161 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:13:54.161 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:13:54.161 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:13:54.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:13:54.161 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 8, 2024-05-28T14:13:54.161 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:13:54.161 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:13:54.161 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:13:54.161 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57440 2024-05-28T14:13:54.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:13:54.162 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:13:54.162 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:13:54.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:54.162 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:13:54.162 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:13:54.162 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:13:54.162 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:13:54.162 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:13:54.162 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:13:54.162 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:13:54.162 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:13:54.163 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:13:54.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:54.163 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:13:54.163 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:13:54.163 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:12:23.155415+0000", 2024-05-28T14:13:54.163 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:13:54.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:54.163 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:13:54.163 INFO:teuthology.orchestra.run.smithi042.stdout: "79821641-56ea-4c0f-b18b-17882adc2609": { 2024-05-28T14:13:54.163 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 27.0 seconds before exiting (9s)\n [============================] ", 2024-05-28T14:13:54.163 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:13:54.163 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:13:54.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:13:54.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:13:54.164 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:13:54.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.171+0000 7f6a55c00640 1 -- 172.21.15.42:0/1699858518 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a30069e80 msgr2=0x7f6a3006c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.171+0000 7f6a55c00640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a30069e80 0x7f6a3006c340 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009980 tx=0x7f6a40008040 comp rx=0 tx=0).stop 2024-05-28T14:13:54.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.171+0000 7f6a55c00640 1 -- 172.21.15.42:0/1699858518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a50156a20 msgr2=0x7f6a5014c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.171+0000 7f6a55c00640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a50156a20 0x7f6a5014c110 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0190a0 tx=0x7f6a3c01acd0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.171+0000 7f6a55c00640 1 -- 172.21.15.42:0/1699858518 shutdown_connections 2024-05-28T14:13:54.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.171+0000 7f6a55c00640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a30069e80 0x7f6a3006c340 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.171+0000 7f6a55c00640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a50156e00 0x7f6a50071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.171+0000 7f6a55c00640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a50156a20 0x7f6a5014c110 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.171+0000 7f6a55c00640 1 --2- 172.21.15.42:0/1699858518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a500aee40 0x7f6a501549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.171+0000 7f6a55c00640 1 -- 172.21.15.42:0/1699858518 >> 172.21.15.42:0/1699858518 conn(0x7f6a500b1fa0 msgr2=0x7f6a5014e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:54.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.171+0000 7f6a55c00640 1 -- 172.21.15.42:0/1699858518 shutdown_connections 2024-05-28T14:13:54.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.171+0000 7f6a55c00640 1 -- 172.21.15.42:0/1699858518 wait complete. 2024-05-28T14:13:54.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:13:54.178 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone86011/c4222c14-e22c-4814-8efb-a3c1d580571a 2024-05-28T14:13:54.178 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:13:54.192 INFO:teuthology.orchestra.run.smithi042.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone86011/c4222c14-e22c-4814-8efb-a3c1d580571a': No such file or directory 2024-05-28T14:13:54.193 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-28T14:13:54.194 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:13:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb651600640 1 Processor -- start 2024-05-28T14:13:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb651600640 1 -- start start 2024-05-28T14:13:54.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb651600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb64c06f030 0x7fb64c06db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:54.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb651600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb64c06f480 con 0x7fb64c0724d0 2024-05-28T14:13:54.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb651600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb64c06e240 con 0x7fb64c145920 2024-05-28T14:13:54.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb651600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb64c06e520 con 0x7fb64c06f030 2024-05-28T14:13:54.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb642000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb64c0724d0 0x7fb64c15d120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:39748/0 (socket says 172.21.15.42:39748) 2024-05-28T14:13:54.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb642000640 1 -- 172.21.15.42:0/3002204985 learned_addr learned my addr 172.21.15.42:0/3002204985 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:13:54.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb64b400640 1 --2- 172.21.15.42:0/3002204985 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb64c06f030 0x7fb64c06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:54.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb64a000640 1 -- 172.21.15.42:0/3002204985 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3646937169 0 0) 0x7fb64c06f480 con 0x7fb64c0724d0 2024-05-28T14:13:54.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb64a000640 1 -- 172.21.15.42:0/3002204985 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb630003530 con 0x7fb64c0724d0 2024-05-28T14:13:54.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb64a000640 1 -- 172.21.15.42:0/3002204985 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1000078387 0 0) 0x7fb64c06e240 con 0x7fb64c145920 2024-05-28T14:13:54.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb64a000640 1 -- 172.21.15.42:0/3002204985 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb64c06f480 con 0x7fb64c145920 2024-05-28T14:13:54.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb64b400640 1 -- 172.21.15.42:0/3002204985 >> v1:172.21.15.63:6789/0 conn(0x7fb64c145920 legacy=0x7fb64c071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb64b400640 1 -- 172.21.15.42:0/3002204985 >> v1:172.21.15.42:6789/0 conn(0x7fb64c0724d0 legacy=0x7fb64c15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.391+0000 7fb64b400640 1 -- 172.21.15.42:0/3002204985 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c06e800 con 0x7fb64c06f030 2024-05-28T14:13:54.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.395+0000 7fb64b400640 1 --2- 172.21.15.42:0/3002204985 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb64c06f030 0x7fb64c06db40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb6440097f0 tx=0x7fb64401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d1aa87cedef076a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:54.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.395+0000 7fb64a000640 1 -- 172.21.15.42:0/3002204985 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6440b2070 con 0x7fb64c06f030 2024-05-28T14:13:54.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.395+0000 7fb64a000640 1 -- 172.21.15.42:0/3002204985 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6440a9070 con 0x7fb64c06f030 2024-05-28T14:13:54.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.395+0000 7fb64a000640 1 -- 172.21.15.42:0/3002204985 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6440ad880 con 0x7fb64c06f030 2024-05-28T14:13:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.395+0000 7fb651600640 1 -- 172.21.15.42:0/3002204985 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb64c06f030 msgr2=0x7fb64c06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.395+0000 7fb651600640 1 --2- 172.21.15.42:0/3002204985 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb64c06f030 0x7fb64c06db40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb6440097f0 tx=0x7fb64401c450 comp rx=0 tx=0).stop 2024-05-28T14:13:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.395+0000 7fb651600640 1 -- 172.21.15.42:0/3002204985 shutdown_connections 2024-05-28T14:13:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.395+0000 7fb651600640 1 --2- 172.21.15.42:0/3002204985 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb64c06f030 0x7fb64c06db40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.395+0000 7fb651600640 1 -- 172.21.15.42:0/3002204985 >> 172.21.15.42:0/3002204985 conn(0x7fb64c06b9f0 msgr2=0x7fb64c06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.395+0000 7fb651600640 1 -- 172.21.15.42:0/3002204985 shutdown_connections 2024-05-28T14:13:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.395+0000 7fb651600640 1 -- 172.21.15.42:0/3002204985 wait complete. 2024-05-28T14:13:54.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb651600640 1 Processor -- start 2024-05-28T14:13:54.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb651600640 1 -- start start 2024-05-28T14:13:54.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb651600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb64c06f030 0x7fb64c158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:54.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb651600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb64c0724d0 0x7fb64c144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:54.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb651600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb64c145920 0x7fb64c0ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:54.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb651600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c15e3d0 con 0x7fb64c145920 2024-05-28T14:13:54.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb651600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c15e250 con 0x7fb64c0724d0 2024-05-28T14:13:54.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb651600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb64c16a260 con 0x7fb64c06f030 2024-05-28T14:13:54.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb64b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb64c145920 0x7fb64c0ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:54.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb64b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb64c145920 0x7fb64c0ae150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34294/0 (socket says 172.21.15.42:34294) 2024-05-28T14:13:54.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb64b400640 1 -- 172.21.15.42:0/1973958458 learned_addr learned my addr 172.21.15.42:0/1973958458 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:54.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb64aa00640 1 --2- 172.21.15.42:0/1973958458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb64c06f030 0x7fb64c158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:54.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb642000640 1 --2- 172.21.15.42:0/1973958458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb64c0724d0 0x7fb64c144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:54.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb64b400640 1 -- 172.21.15.42:0/1973958458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb64c06f030 msgr2=0x7fb64c158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb64b400640 1 --2- 172.21.15.42:0/1973958458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb64c06f030 0x7fb64c158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb64b400640 1 -- 172.21.15.42:0/1973958458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb64c0724d0 msgr2=0x7fb64c144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb64b400640 1 --2- 172.21.15.42:0/1973958458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb64c0724d0 0x7fb64c144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb64b400640 1 -- 172.21.15.42:0/1973958458 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c0b4420 con 0x7fb64c145920 2024-05-28T14:13:54.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb64b400640 1 --2- 172.21.15.42:0/1973958458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb64c145920 0x7fb64c0ae150 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fb6440097f0 tx=0x7fb6440191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:54.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb643400640 1 -- 172.21.15.42:0/1973958458 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6440bc070 con 0x7fb64c145920 2024-05-28T14:13:54.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb643400640 1 -- 172.21.15.42:0/1973958458 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6440adcd0 con 0x7fb64c145920 2024-05-28T14:13:54.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb643400640 1 -- 172.21.15.42:0/1973958458 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6440b2040 con 0x7fb64c145920 2024-05-28T14:13:54.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb651600640 1 -- 172.21.15.42:0/1973958458 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64c0b4700 con 0x7fb64c145920 2024-05-28T14:13:54.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.399+0000 7fb651600640 1 -- 172.21.15.42:0/1973958458 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64c0b4cc0 con 0x7fb64c145920 2024-05-28T14:13:54.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.403+0000 7fb651600640 1 -- 172.21.15.42:0/1973958458 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb64c073890 con 0x7fb64c145920 2024-05-28T14:13:54.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.403+0000 7fb643400640 1 -- 172.21.15.42:0/1973958458 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7fb6440ab070 con 0x7fb64c145920 2024-05-28T14:13:54.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.403+0000 7fb643400640 1 --2- 172.21.15.42:0/1973958458 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb628069e80 0x7fb62806c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:54.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.403+0000 7fb643400640 1 -- 172.21.15.42:0/1973958458 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7fb64411e510 con 0x7fb64c145920 2024-05-28T14:13:54.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.403+0000 7fb64aa00640 1 --2- 172.21.15.42:0/1973958458 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb628069e80 0x7fb62806c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:54.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.403+0000 7fb64aa00640 1 --2- 172.21.15.42:0/1973958458 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb628069e80 0x7fb62806c340 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb63c007fd0 tx=0x7fb63c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:54.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.407+0000 7fb643400640 1 -- 172.21.15.42:0/1973958458 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb6440160d0 con 0x7fb64c145920 2024-05-28T14:13:54.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.595+0000 7fb651600640 1 -- 172.21.15.42:0/1973958458 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fb64c13d480 con 0x7fb628069e80 2024-05-28T14:13:54.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.595+0000 7fb643400640 1 -- 172.21.15.42:0/1973958458 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fb63c003980 con 0x7fb628069e80 2024-05-28T14:13:54.582 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:13:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:13:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:13:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:13:54.582 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:13:54.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.599+0000 7fb651600640 1 -- 172.21.15.42:0/1973958458 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb628069e80 msgr2=0x7fb62806c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.599+0000 7fb651600640 1 --2- 172.21.15.42:0/1973958458 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb628069e80 0x7fb62806c340 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb63c007fd0 tx=0x7fb63c009040 comp rx=0 tx=0).stop 2024-05-28T14:13:54.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.599+0000 7fb651600640 1 -- 172.21.15.42:0/1973958458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb64c145920 msgr2=0x7fb64c0ae150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.599+0000 7fb651600640 1 --2- 172.21.15.42:0/1973958458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb64c145920 0x7fb64c0ae150 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fb6440097f0 tx=0x7fb6440191f0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.599+0000 7fb651600640 1 -- 172.21.15.42:0/1973958458 shutdown_connections 2024-05-28T14:13:54.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.599+0000 7fb651600640 1 --2- 172.21.15.42:0/1973958458 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb628069e80 0x7fb62806c340 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.599+0000 7fb651600640 1 --2- 172.21.15.42:0/1973958458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb64c145920 0x7fb64c0ae150 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.599+0000 7fb651600640 1 --2- 172.21.15.42:0/1973958458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb64c0724d0 0x7fb64c144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.599+0000 7fb651600640 1 --2- 172.21.15.42:0/1973958458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb64c06f030 0x7fb64c158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.599+0000 7fb651600640 1 -- 172.21.15.42:0/1973958458 >> 172.21.15.42:0/1973958458 conn(0x7fb64c06b9f0 msgr2=0x7fb64c140520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:54.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.599+0000 7fb651600640 1 -- 172.21.15.42:0/1973958458 shutdown_connections 2024-05-28T14:13:54.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.599+0000 7fb651600640 1 -- 172.21.15.42:0/1973958458 wait complete. 2024-05-28T14:13:54.604 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:13:54.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.759+0000 7f3e09e00640 1 Processor -- start 2024-05-28T14:13:54.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.759+0000 7f3e09e00640 1 -- start start 2024-05-28T14:13:54.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.759+0000 7f3e09e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e04156c00 0x7f3e0406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:54.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.759+0000 7f3e09e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3e040af6d0 0x7f3e0415dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.759+0000 7f3e09e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e0406f6a0 con 0x7f3e0415d8d0 2024-05-28T14:13:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.759+0000 7f3e09e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f3e0406e600 con 0x7f3e040af6d0 2024-05-28T14:13:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.759+0000 7f3e09e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3e0406e8e0 con 0x7f3e04156c00 2024-05-28T14:13:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.759+0000 7f3e03400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3e040af6d0 0x7f3e0415dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.763+0000 7f3e03400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3e040af6d0 0x7f3e0415dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34306/0 (socket says 172.21.15.42:34306) 2024-05-28T14:13:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.763+0000 7f3e03400640 1 -- 172.21.15.42:0/1554880345 learned_addr learned my addr 172.21.15.42:0/1554880345 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.763+0000 7f3e02a00640 1 --2- 172.21.15.42:0/1554880345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e04156c00 0x7f3e0406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.763+0000 7f3e03400640 1 -- 172.21.15.42:0/1554880345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e04156c00 msgr2=0x7f3e0406dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:13:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.763+0000 7f3e03400640 1 --2- 172.21.15.42:0/1554880345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e04156c00 0x7f3e0406dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.763+0000 7f3e03400640 1 -- 172.21.15.42:0/1554880345 >> v1:172.21.15.63:6789/0 conn(0x7f3e0415d8d0 legacy=0x7f3e040aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.763+0000 7f3e03400640 1 -- 172.21.15.42:0/1554880345 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0406ebc0 con 0x7f3e040af6d0 2024-05-28T14:13:54.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.763+0000 7f3e03400640 1 --2- 172.21.15.42:0/1554880345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040af6d0 0x7f3e0415dd70 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009e50 tx=0x7f3dfc01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7edc54bb6b363c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:54.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.763+0000 7f3e02000640 1 -- 172.21.15.42:0/1554880345 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0a5660 con 0x7f3e040af6d0 2024-05-28T14:13:54.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.763+0000 7f3e02000640 1 -- 172.21.15.42:0/1554880345 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dfc0a5800 con 0x7f3e040af6d0 2024-05-28T14:13:54.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.763+0000 7f3e02000640 1 -- 172.21.15.42:0/1554880345 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0a5bd0 con 0x7f3e040af6d0 2024-05-28T14:13:54.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 -- 172.21.15.42:0/1554880345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040af6d0 msgr2=0x7f3e0415dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 --2- 172.21.15.42:0/1554880345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040af6d0 0x7f3e0415dd70 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009e50 tx=0x7f3dfc01c910 comp rx=0 tx=0).stop 2024-05-28T14:13:54.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 -- 172.21.15.42:0/1554880345 shutdown_connections 2024-05-28T14:13:54.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 --2- 172.21.15.42:0/1554880345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040af6d0 0x7f3e0415dd70 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 --2- 172.21.15.42:0/1554880345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e04156c00 0x7f3e0406dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 -- 172.21.15.42:0/1554880345 >> 172.21.15.42:0/1554880345 conn(0x7f3e0406bff0 msgr2=0x7f3e040b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:54.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 -- 172.21.15.42:0/1554880345 shutdown_connections 2024-05-28T14:13:54.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 -- 172.21.15.42:0/1554880345 wait complete. 2024-05-28T14:13:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 Processor -- start 2024-05-28T14:13:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 -- start start 2024-05-28T14:13:54.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040af6d0 0x7f3e040b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e04156c00 0x7f3e041547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e0415d8d0 0x7f3e04099860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0415ec40 con 0x7f3e040af6d0 2024-05-28T14:13:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0415eac0 con 0x7f3e04156c00 2024-05-28T14:13:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e09e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3e0415edc0 con 0x7f3e0415d8d0 2024-05-28T14:13:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e02a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040af6d0 0x7f3e040b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e02a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040af6d0 0x7f3e040b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34322/0 (socket says 172.21.15.42:34322) 2024-05-28T14:13:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e02a00640 1 -- 172.21.15.42:0/3226371273 learned_addr learned my addr 172.21.15.42:0/3226371273 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:13:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e03400640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e0415d8d0 0x7f3e04099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3dfa000640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e04156c00 0x7f3e041547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e02a00640 1 -- 172.21.15.42:0/3226371273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e0415d8d0 msgr2=0x7f3e04099860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e02a00640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e0415d8d0 0x7f3e04099860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e02a00640 1 -- 172.21.15.42:0/3226371273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e04156c00 msgr2=0x7f3e041547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e02a00640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e04156c00 0x7f3e041547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e02a00640 1 -- 172.21.15.42:0/3226371273 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e04154ce0 con 0x7f3e040af6d0 2024-05-28T14:13:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3dfa000640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e04156c00 0x7f3e041547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:13:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.767+0000 7f3e02a00640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040af6d0 0x7f3e040b0990 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f3df400d5f0 tx=0x7f3df400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.771+0000 7f3dfbe00640 1 -- 172.21.15.42:0/3226371273 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df40125f0 con 0x7f3e040af6d0 2024-05-28T14:13:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.771+0000 7f3dfbe00640 1 -- 172.21.15.42:0/3226371273 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df4019070 con 0x7f3e040af6d0 2024-05-28T14:13:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.771+0000 7f3dfbe00640 1 -- 172.21.15.42:0/3226371273 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df40129b0 con 0x7f3e040af6d0 2024-05-28T14:13:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.771+0000 7f3e09e00640 1 -- 172.21.15.42:0/3226371273 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e04154fb0 con 0x7f3e040af6d0 2024-05-28T14:13:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.771+0000 7f3e09e00640 1 -- 172.21.15.42:0/3226371273 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e04155570 con 0x7f3e040af6d0 2024-05-28T14:13:54.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.771+0000 7f3e09e00640 1 -- 172.21.15.42:0/3226371273 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e04068a90 con 0x7f3e040af6d0 2024-05-28T14:13:54.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.771+0000 7f3dfbe00640 1 -- 172.21.15.42:0/3226371273 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7f3df4012b50 con 0x7f3e040af6d0 2024-05-28T14:13:54.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.771+0000 7f3dfbe00640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de0069ed0 0x7f3de006c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:13:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.771+0000 7f3dfbe00640 1 -- 172.21.15.42:0/3226371273 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f3df4085c90 con 0x7f3e040af6d0 2024-05-28T14:13:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.771+0000 7f3dfa000640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de0069ed0 0x7f3de006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:13:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.771+0000 7f3dfa000640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de0069ed0 0x7f3de006c390 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3e040b1490 tx=0x7f3dec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:13:54.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:54.775+0000 7f3dfbe00640 1 -- 172.21.15.42:0/3226371273 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3df4057d30 con 0x7f3e040af6d0 2024-05-28T14:13:54.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:55.003+0000 7f3e09e00640 1 -- 172.21.15.42:0/3226371273 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3e041671a0 con 0x7f3e040af6d0 2024-05-28T14:13:54.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:55.003+0000 7f3dfbe00640 1 -- 172.21.15.42:0/3226371273 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2124 (secure 0 0 0) 0x7f3df4057630 con 0x7f3e040af6d0 2024-05-28T14:13:54.989 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:13:54.989 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:13:54.989 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:13:54.989 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:13:54.989 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:13:54.989 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:13:54.989 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:13:54.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:54.990 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:13:54.990 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:13:54.990 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:13:54.990 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:13:54.990 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:13:54.990 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:13:54.990 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:13:54.990 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:13:54.990 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:13:54.990 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:13:54.990 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 956, 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 117, 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:13:54.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:54.992 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:13:54.992 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:13:54.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:13:54.992 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:13:54.992 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:13:54.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:13:54.992 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:13:54.992 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:13:54.992 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:13:54.992 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1781, 2024-05-28T14:13:54.992 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 7316116059, 2024-05-28T14:13:54.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 15623352320, 2024-05-28T14:13:54.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 757470760960, 2024-05-28T14:13:54.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:13:54.993 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 345034765, 2024-05-28T14:13:54.993 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 340887149, 2024-05-28T14:13:54.993 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 85, 2024-05-28T14:13:54.993 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 95 2024-05-28T14:13:54.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:54.993 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:13:54.993 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 75, 2024-05-28T14:13:54.993 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 8, 2024-05-28T14:13:54.993 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:13:54.993 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:13:54.993 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:13:54.993 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:13:54.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:13:54.993 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 8, 2024-05-28T14:13:54.994 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:13:54.994 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:13:54.994 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:13:54.994 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57440 2024-05-28T14:13:54.994 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:13:54.994 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:13:54.994 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:13:54.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:54.994 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:13:54.994 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:13:54.994 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:13:54.994 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:13:54.994 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:13:54.995 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:13:54.995 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:13:54.995 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:13:54.995 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:13:54.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:54.995 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:13:54.995 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-05-28T14:13:54.995 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:12:23.155415+0000", 2024-05-28T14:13:54.995 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:13:54.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:13:54.995 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:13:54.995 INFO:teuthology.orchestra.run.smithi042.stdout: "79821641-56ea-4c0f-b18b-17882adc2609": { 2024-05-28T14:13:54.995 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 27.0 seconds before exiting (9s)\n [============================] ", 2024-05-28T14:13:54.995 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:13:54.996 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:13:54.996 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:13:54.996 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:13:54.996 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:13:54.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:55.007+0000 7f3e09e00640 1 -- 172.21.15.42:0/3226371273 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de0069ed0 msgr2=0x7f3de006c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:55.007+0000 7f3e09e00640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de0069ed0 0x7f3de006c390 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3e040b1490 tx=0x7f3dec009040 comp rx=0 tx=0).stop 2024-05-28T14:13:54.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:55.007+0000 7f3e09e00640 1 -- 172.21.15.42:0/3226371273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040af6d0 msgr2=0x7f3e040b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:13:54.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:55.007+0000 7f3e09e00640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040af6d0 0x7f3e040b0990 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f3df400d5f0 tx=0x7f3df400dab0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:55.007+0000 7f3e09e00640 1 -- 172.21.15.42:0/3226371273 shutdown_connections 2024-05-28T14:13:54.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:55.007+0000 7f3e09e00640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3de0069ed0 0x7f3de006c390 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:55.007+0000 7f3e09e00640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e0415d8d0 0x7f3e04099860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:55.007+0000 7f3e09e00640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e04156c00 0x7f3e041547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:55.007+0000 7f3e09e00640 1 --2- 172.21.15.42:0/3226371273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040af6d0 0x7f3e040b0990 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:13:54.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:55.007+0000 7f3e09e00640 1 -- 172.21.15.42:0/3226371273 >> 172.21.15.42:0/3226371273 conn(0x7f3e0406bff0 msgr2=0x7f3e04072960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:13:54.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:55.007+0000 7f3e09e00640 1 -- 172.21.15.42:0/3226371273 shutdown_connections 2024-05-28T14:13:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:13:55.007+0000 7f3e09e00640 1 -- 172.21.15.42:0/3226371273 wait complete. 2024-05-28T14:14:00.011 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:14:00.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97c3200640 1 Processor -- start 2024-05-28T14:14:00.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97c3200640 1 -- start start 2024-05-28T14:14:00.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97c3200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97bc15e9d0 0x7f97bc0d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:00.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97c3200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f97bc09b160 0x7f97bc0d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:00.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97c3200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97bc0ccce0 con 0x7f97bc0b52d0 2024-05-28T14:14:00.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97c3200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f97bc0ccfc0 con 0x7f97bc09b160 2024-05-28T14:14:00.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97c3200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f97bc0cd2a0 con 0x7f97bc15e9d0 2024-05-28T14:14:00.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97c1200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f97bc09b160 0x7f97bc0d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:00.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97bbe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97bc15e9d0 0x7f97bc0d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:00.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97bb400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f97bc0b52d0 0x7f97bc09aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:33038/0 (socket says 172.21.15.42:33038) 2024-05-28T14:14:00.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97bb400640 1 -- 172.21.15.42:0/300496023 learned_addr learned my addr 172.21.15.42:0/300496023 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:00.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97c1200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f97bc09b160 0x7f97bc0d1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:39040/0 (socket says 172.21.15.42:39040) 2024-05-28T14:14:00.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97bbe00640 1 -- 172.21.15.42:0/300496023 >> v2:172.21.15.63:3300/0 conn(0x7f97bc09b160 msgr2=0x7f97bc0d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:00.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97baa00640 1 -- 172.21.15.42:0/300496023 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3978127352 0 0) 0x7f97bc0ccce0 con 0x7f97bc0b52d0 2024-05-28T14:14:00.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97bbe00640 1 --2- 172.21.15.42:0/300496023 >> v2:172.21.15.63:3300/0 conn(0x7f97bc09b160 0x7f97bc0d1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:00.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97bbe00640 1 -- 172.21.15.42:0/300496023 >> v1:172.21.15.63:6789/0 conn(0x7f97bc0b52d0 legacy=0x7f97bc09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:00.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97bbe00640 1 -- 172.21.15.42:0/300496023 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc0cd580 con 0x7f97bc15e9d0 2024-05-28T14:14:00.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.175+0000 7f97c1200640 1 --2- 172.21.15.42:0/300496023 >> v2:172.21.15.63:3300/0 conn(0x7f97bc09b160 0x7f97bc0d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:14:00.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.179+0000 7f97bbe00640 1 --2- 172.21.15.42:0/300496023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97bc15e9d0 0x7f97bc0d12c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0097c0 tx=0x7f97ac01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f76132e63feb5f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:00.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.179+0000 7f97baa00640 1 -- 172.21.15.42:0/300496023 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac0b0070 con 0x7f97bc15e9d0 2024-05-28T14:14:00.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.179+0000 7f97baa00640 1 -- 172.21.15.42:0/300496023 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97ac0ab600 con 0x7f97bc15e9d0 2024-05-28T14:14:00.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.179+0000 7f97baa00640 1 -- 172.21.15.42:0/300496023 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac0ab9d0 con 0x7f97bc15e9d0 2024-05-28T14:14:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.179+0000 7f97c3200640 1 -- 172.21.15.42:0/300496023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97bc15e9d0 msgr2=0x7f97bc0d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.179+0000 7f97c3200640 1 --2- 172.21.15.42:0/300496023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97bc15e9d0 0x7f97bc0d12c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0097c0 tx=0x7f97ac01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:14:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.179+0000 7f97c3200640 1 -- 172.21.15.42:0/300496023 shutdown_connections 2024-05-28T14:14:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.179+0000 7f97c3200640 1 --2- 172.21.15.42:0/300496023 >> v2:172.21.15.63:3300/0 conn(0x7f97bc09b160 0x7f97bc0d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.179+0000 7f97c3200640 1 --2- 172.21.15.42:0/300496023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97bc15e9d0 0x7f97bc0d12c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.179+0000 7f97c3200640 1 -- 172.21.15.42:0/300496023 >> 172.21.15.42:0/300496023 conn(0x7f97bc0b3f70 msgr2=0x7f97bc0b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.179+0000 7f97c3200640 1 -- 172.21.15.42:0/300496023 shutdown_connections 2024-05-28T14:14:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.179+0000 7f97c3200640 1 -- 172.21.15.42:0/300496023 wait complete. 2024-05-28T14:14:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c3200640 1 Processor -- start 2024-05-28T14:14:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c3200640 1 -- start start 2024-05-28T14:14:00.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c3200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97bc09b160 0x7f97bc149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:00.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97bc0b52d0 0x7f97bc154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:00.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c3200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97bc15e9d0 0x7f97bc071110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:00.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c3200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc0b3c30 con 0x7f97bc15e9d0 2024-05-28T14:14:00.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c3200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc0b3ab0 con 0x7f97bc0b52d0 2024-05-28T14:14:00.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c3200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f97bc0b3db0 con 0x7f97bc09b160 2024-05-28T14:14:00.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97bc15e9d0 0x7f97bc071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:00.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97bc15e9d0 0x7f97bc071110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41896/0 (socket says 172.21.15.42:41896) 2024-05-28T14:14:00.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c1200640 1 -- 172.21.15.42:0/975756297 learned_addr learned my addr 172.21.15.42:0/975756297 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:00.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97bbe00640 1 --2- 172.21.15.42:0/975756297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97bc09b160 0x7f97bc149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:00.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97bb400640 1 --2- 172.21.15.42:0/975756297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97bc0b52d0 0x7f97bc154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:00.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c1200640 1 -- 172.21.15.42:0/975756297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97bc09b160 msgr2=0x7f97bc149910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:00.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c1200640 1 --2- 172.21.15.42:0/975756297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97bc09b160 0x7f97bc149910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:00.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c1200640 1 -- 172.21.15.42:0/975756297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97bc0b52d0 msgr2=0x7f97bc154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:00.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c1200640 1 --2- 172.21.15.42:0/975756297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97bc0b52d0 0x7f97bc154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:00.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c1200640 1 -- 172.21.15.42:0/975756297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc14bd20 con 0x7f97bc15e9d0 2024-05-28T14:14:00.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c1200640 1 --2- 172.21.15.42:0/975756297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97bc15e9d0 0x7f97bc071110 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f97b400efc0 tx=0x7f97b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:00.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97b3e00640 1 -- 172.21.15.42:0/975756297 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4018020 con 0x7f97bc15e9d0 2024-05-28T14:14:00.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97b3e00640 1 -- 172.21.15.42:0/975756297 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b4014480 con 0x7f97bc15e9d0 2024-05-28T14:14:00.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97b3e00640 1 -- 172.21.15.42:0/975756297 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4014870 con 0x7f97bc15e9d0 2024-05-28T14:14:00.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c3200640 1 -- 172.21.15.42:0/975756297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97bc14c000 con 0x7f97bc15e9d0 2024-05-28T14:14:00.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c3200640 1 -- 172.21.15.42:0/975756297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97bc14c5c0 con 0x7f97bc15e9d0 2024-05-28T14:14:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.183+0000 7f97c3200640 1 -- 172.21.15.42:0/975756297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97bc068a90 con 0x7f97bc15e9d0 2024-05-28T14:14:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.187+0000 7f97b3e00640 1 -- 172.21.15.42:0/975756297 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7f97b401b020 con 0x7f97bc15e9d0 2024-05-28T14:14:00.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.187+0000 7f97b3e00640 1 --2- 172.21.15.42:0/975756297 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f979c069e80 0x7f979c06c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:00.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.187+0000 7f97b3e00640 1 -- 172.21.15.42:0/975756297 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f97b40865a0 con 0x7f97bc15e9d0 2024-05-28T14:14:00.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.187+0000 7f97bbe00640 1 --2- 172.21.15.42:0/975756297 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f979c069e80 0x7f979c06c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.187+0000 7f97bbe00640 1 --2- 172.21.15.42:0/975756297 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f979c069e80 0x7f979c06c340 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0ad000 tx=0x7f97ac0a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:00.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.191+0000 7f97b3e00640 1 -- 172.21.15.42:0/975756297 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f97b4022030 con 0x7f97bc15e9d0 2024-05-28T14:14:00.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.419+0000 7f97c3200640 1 -- 172.21.15.42:0/975756297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f97bc0b7fd0 con 0x7f97bc15e9d0 2024-05-28T14:14:00.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.419+0000 7f97b3e00640 1 -- 172.21.15.42:0/975756297 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+5881 (secure 0 0 0) 0x7f97b4010040 con 0x7f97bc15e9d0 2024-05-28T14:14:00.404 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:00.404 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:14:00.404 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:14:00.404 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:14:00.404 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:14:00.404 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:14:00.404 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:14:00.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.404 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:14:00.404 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:14:00.404 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:14:00.404 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:14:00.405 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:14:00.405 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:00.405 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:14:00.405 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:14:00.405 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:14:00.405 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:14:00.405 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:00.405 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 961, 2024-05-28T14:14:00.405 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:14:00.405 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:14:00.405 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:14:00.405 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:14:00.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.405 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:14:00.406 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 117, 2024-05-28T14:14:00.406 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:14:00.406 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:14:00.406 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:14:00.406 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:14:00.406 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:14:00.406 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:14:00.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.406 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:14:00.406 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:14:00.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:00.406 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:14:00.406 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:14:00.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:00.407 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:00.407 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:14:00.407 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:14:00.407 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1775, 2024-05-28T14:14:00.407 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 7316153951, 2024-05-28T14:14:00.407 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 15677947904, 2024-05-28T14:14:00.407 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 757416165376, 2024-05-28T14:14:00.407 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:14:00.407 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 230982659, 2024-05-28T14:14:00.407 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 234851335, 2024-05-28T14:14:00.407 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 57, 2024-05-28T14:14:00.407 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 66 2024-05-28T14:14:00.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.408 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:14:00.408 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 75, 2024-05-28T14:14:00.408 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 8, 2024-05-28T14:14:00.408 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:14:00.408 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:14:00.408 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:14:00.408 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:14:00.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:00.408 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 8, 2024-05-28T14:14:00.408 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:14:00.408 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:14:00.408 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:14:00.408 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57440 2024-05-28T14:14:00.408 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:00.409 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:00.409 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:14:00.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.409 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:14:00.409 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:14:00.409 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:14:00.409 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:14:00.409 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:14:00.409 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:14:00.409 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:14:00.409 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:00.409 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:14:00.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.410 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:14:00.410 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 19, 2024-05-28T14:14:00.410 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:13:57.184268+0000", 2024-05-28T14:14:00.410 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-05-28T14:14:00.410 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-05-28T14:14:00.410 INFO:teuthology.orchestra.run.smithi042.stdout: "daemons": { 2024-05-28T14:14:00.410 INFO:teuthology.orchestra.run.smithi042.stdout: "summary": "", 2024-05-28T14:14:00.410 INFO:teuthology.orchestra.run.smithi042.stdout: "b": { 2024-05-28T14:14:00.410 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:00.410 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:00.410 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:00.410 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:00.410 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:00.411 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:00.411 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:00.411 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:00.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.411 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-05-28T14:14:00.411 INFO:teuthology.orchestra.run.smithi042.stdout: "daemons": { 2024-05-28T14:14:00.411 INFO:teuthology.orchestra.run.smithi042.stdout: "summary": "", 2024-05-28T14:14:00.411 INFO:teuthology.orchestra.run.smithi042.stdout: "y": { 2024-05-28T14:14:00.411 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:00.411 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:00.411 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:00.411 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:00.411 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:00.411 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: "daemons": { 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: "summary": "", 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: "a": { 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:00.412 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:00.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.413 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-05-28T14:14:00.413 INFO:teuthology.orchestra.run.smithi042.stdout: "daemons": { 2024-05-28T14:14:00.413 INFO:teuthology.orchestra.run.smithi042.stdout: "summary": "", 2024-05-28T14:14:00.413 INFO:teuthology.orchestra.run.smithi042.stdout: "0": { 2024-05-28T14:14:00.413 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:00.413 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:00.413 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:00.413 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:00.413 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:00.414 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:00.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.414 INFO:teuthology.orchestra.run.smithi042.stdout: "1": { 2024-05-28T14:14:00.414 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:00.414 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:00.414 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:00.414 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:00.414 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:00.415 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:00.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.415 INFO:teuthology.orchestra.run.smithi042.stdout: "2": { 2024-05-28T14:14:00.415 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:00.415 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:00.415 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:00.415 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:00.415 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:00.415 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:00.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.415 INFO:teuthology.orchestra.run.smithi042.stdout: "3": { 2024-05-28T14:14:00.415 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:00.415 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:00.415 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: "5": { 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: "6": { 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:00.416 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:00.417 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:00.417 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:00.417 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:00.417 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:00.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.417 INFO:teuthology.orchestra.run.smithi042.stdout: "7": { 2024-05-28T14:14:00.417 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:00.417 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:00.417 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:00.417 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:00.417 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:00.417 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:00.417 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:00.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:00.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:00.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:00.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:00.418 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:14:00.418 INFO:teuthology.orchestra.run.smithi042.stdout: "79821641-56ea-4c0f-b18b-17882adc2609": { 2024-05-28T14:14:00.418 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 21.0 seconds before exiting (15s)\n [============================] ", 2024-05-28T14:14:00.418 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:14:00.418 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:14:00.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:00.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:00.418 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:14:00.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.419+0000 7f97c3200640 1 -- 172.21.15.42:0/975756297 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f979c069e80 msgr2=0x7f979c06c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:00.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.419+0000 7f97c3200640 1 --2- 172.21.15.42:0/975756297 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f979c069e80 0x7f979c06c340 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0ad000 tx=0x7f97ac0a6000 comp rx=0 tx=0).stop 2024-05-28T14:14:00.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.419+0000 7f97c3200640 1 -- 172.21.15.42:0/975756297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97bc15e9d0 msgr2=0x7f97bc071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:00.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.419+0000 7f97c3200640 1 --2- 172.21.15.42:0/975756297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97bc15e9d0 0x7f97bc071110 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f97b400efc0 tx=0x7f97b400c490 comp rx=0 tx=0).stop 2024-05-28T14:14:00.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.419+0000 7f97c3200640 1 -- 172.21.15.42:0/975756297 shutdown_connections 2024-05-28T14:14:00.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.419+0000 7f97c3200640 1 --2- 172.21.15.42:0/975756297 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f979c069e80 0x7f979c06c340 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:00.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.419+0000 7f97c3200640 1 --2- 172.21.15.42:0/975756297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97bc15e9d0 0x7f97bc071110 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:00.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.419+0000 7f97c3200640 1 --2- 172.21.15.42:0/975756297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97bc0b52d0 0x7f97bc154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:00.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.419+0000 7f97c3200640 1 --2- 172.21.15.42:0/975756297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97bc09b160 0x7f97bc149910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:00.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.419+0000 7f97c3200640 1 -- 172.21.15.42:0/975756297 >> 172.21.15.42:0/975756297 conn(0x7f97bc0b3f70 msgr2=0x7f97bc0ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:00.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.419+0000 7f97c3200640 1 -- 172.21.15.42:0/975756297 shutdown_connections 2024-05-28T14:14:00.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:00.419+0000 7f97c3200640 1 -- 172.21.15.42:0/975756297 wait complete. 2024-05-28T14:14:05.426 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:14:05.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.591+0000 7f50b5e00640 1 Processor -- start 2024-05-28T14:14:05.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.591+0000 7f50b5e00640 1 -- start start 2024-05-28T14:14:05.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50b5e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50b01521a0 0x7f50b015f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:05.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50b5e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f50b006ed20 0x7f50b006f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:05.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50b5e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50b015c6e0 con 0x7f50b015b820 2024-05-28T14:14:05.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50b5e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f50b006f6a0 con 0x7f50b006ed20 2024-05-28T14:14:05.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50b5e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f50b00b4590 con 0x7f50b01521a0 2024-05-28T14:14:05.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50b4a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50b01521a0 0x7f50b015f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:05.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50a7e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f50b006ed20 0x7f50b006f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:05.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50b4a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50b01521a0 0x7f50b015f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:34078/0 (socket says 172.21.15.42:34078) 2024-05-28T14:14:05.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50b4a00640 1 -- 172.21.15.42:0/2733872856 learned_addr learned my addr 172.21.15.42:0/2733872856 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:05.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50b5400640 1 --1- 172.21.15.42:0/2733872856 >> v1:172.21.15.63:6789/0 conn(0x7f50b015b820 0x7f50b015bc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:33040/0 (socket says 172.21.15.42:33040) 2024-05-28T14:14:05.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50b4a00640 1 -- 172.21.15.42:0/2733872856 >> v2:172.21.15.63:3300/0 conn(0x7f50b006ed20 msgr2=0x7f50b006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:05.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50b4a00640 1 --2- 172.21.15.42:0/2733872856 >> v2:172.21.15.63:3300/0 conn(0x7f50b006ed20 0x7f50b006f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:05.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50b4a00640 1 -- 172.21.15.42:0/2733872856 >> v1:172.21.15.63:6789/0 conn(0x7f50b015b820 legacy=0x7f50b015bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:05.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50b4a00640 1 -- 172.21.15.42:0/2733872856 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b015c400 con 0x7f50b01521a0 2024-05-28T14:14:05.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.595+0000 7f50a7e00640 1 --2- 172.21.15.42:0/2733872856 >> v2:172.21.15.63:3300/0 conn(0x7f50b006ed20 0x7f50b006f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:14:05.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.599+0000 7f50b4a00640 1 --2- 172.21.15.42:0/2733872856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50b01521a0 0x7f50b015f990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009870 tx=0x7f50a001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcef787767e16c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:05.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.599+0000 7f50afe00640 1 -- 172.21.15.42:0/2733872856 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a00a95f0 con 0x7f50b01521a0 2024-05-28T14:14:05.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.599+0000 7f50afe00640 1 -- 172.21.15.42:0/2733872856 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50a00a9790 con 0x7f50b01521a0 2024-05-28T14:14:05.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.599+0000 7f50afe00640 1 -- 172.21.15.42:0/2733872856 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a00a9b60 con 0x7f50b01521a0 2024-05-28T14:14:05.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.599+0000 7f50b5e00640 1 -- 172.21.15.42:0/2733872856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50b01521a0 msgr2=0x7f50b015f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:05.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.599+0000 7f50b5e00640 1 --2- 172.21.15.42:0/2733872856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50b01521a0 0x7f50b015f990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009870 tx=0x7f50a001c530 comp rx=0 tx=0).stop 2024-05-28T14:14:05.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.599+0000 7f50b5e00640 1 -- 172.21.15.42:0/2733872856 shutdown_connections 2024-05-28T14:14:05.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.599+0000 7f50b5e00640 1 --2- 172.21.15.42:0/2733872856 >> v2:172.21.15.63:3300/0 conn(0x7f50b006ed20 0x7f50b006f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:05.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.599+0000 7f50b5e00640 1 --2- 172.21.15.42:0/2733872856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50b01521a0 0x7f50b015f990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:05.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.599+0000 7f50b5e00640 1 -- 172.21.15.42:0/2733872856 >> 172.21.15.42:0/2733872856 conn(0x7f50b00b1fa0 msgr2=0x7f50b00b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:05.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.599+0000 7f50b5e00640 1 -- 172.21.15.42:0/2733872856 shutdown_connections 2024-05-28T14:14:05.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.599+0000 7f50b5e00640 1 -- 172.21.15.42:0/2733872856 wait complete. 2024-05-28T14:14:05.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.599+0000 7f50b5e00640 1 Processor -- start 2024-05-28T14:14:05.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.599+0000 7f50b5e00640 1 -- start start 2024-05-28T14:14:05.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50b006ed20 0x7f50b00b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:05.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50b01521a0 0x7f50b0147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:05.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50b015b820 0x7f50b00af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f50b0152700 con 0x7f50b015b820 2024-05-28T14:14:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50b0152580 con 0x7f50b01521a0 2024-05-28T14:14:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f50b00729c0 con 0x7f50b006ed20 2024-05-28T14:14:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50b015b820 0x7f50b00af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b4a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50b006ed20 0x7f50b00b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50b015b820 0x7f50b00af3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41906/0 (socket says 172.21.15.42:41906) 2024-05-28T14:14:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5400640 1 -- 172.21.15.42:0/4051178803 learned_addr learned my addr 172.21.15.42:0/4051178803 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:05.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50a7e00640 1 --2- 172.21.15.42:0/4051178803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50b01521a0 0x7f50b0147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:05.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5400640 1 -- 172.21.15.42:0/4051178803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50b006ed20 msgr2=0x7f50b00b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:05.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5400640 1 --2- 172.21.15.42:0/4051178803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50b006ed20 0x7f50b00b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:05.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5400640 1 -- 172.21.15.42:0/4051178803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50b01521a0 msgr2=0x7f50b0147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:05.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5400640 1 --2- 172.21.15.42:0/4051178803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50b01521a0 0x7f50b0147000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:05.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5400640 1 -- 172.21.15.42:0/4051178803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b014b980 con 0x7f50b015b820 2024-05-28T14:14:05.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5400640 1 --2- 172.21.15.42:0/4051178803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50b015b820 0x7f50b00af3a0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f50a800c730 tx=0x7f50a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:05.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50ad600640 1 -- 172.21.15.42:0/4051178803 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8017070 con 0x7f50b015b820 2024-05-28T14:14:05.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50ad600640 1 -- 172.21.15.42:0/4051178803 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50a800ce40 con 0x7f50b015b820 2024-05-28T14:14:05.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50ad600640 1 -- 172.21.15.42:0/4051178803 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a800a840 con 0x7f50b015b820 2024-05-28T14:14:05.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5e00640 1 -- 172.21.15.42:0/4051178803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b00cf420 con 0x7f50b015b820 2024-05-28T14:14:05.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5e00640 1 -- 172.21.15.42:0/4051178803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b014bbf0 con 0x7f50b015b820 2024-05-28T14:14:05.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.603+0000 7f50b5e00640 1 -- 172.21.15.42:0/4051178803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b0068a90 con 0x7f50b015b820 2024-05-28T14:14:05.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.607+0000 7f50ad600640 1 -- 172.21.15.42:0/4051178803 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7f50a801a070 con 0x7f50b015b820 2024-05-28T14:14:05.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.607+0000 7f50ad600640 1 --2- 172.21.15.42:0/4051178803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f508c069e80 0x7f508c06c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:05.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.607+0000 7f50ad600640 1 -- 172.21.15.42:0/4051178803 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5823+0+0 (secure 0 0 0) 0x7f50a8084e00 con 0x7f50b015b820 2024-05-28T14:14:05.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.607+0000 7f50b4a00640 1 --2- 172.21.15.42:0/4051178803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f508c069e80 0x7f508c06c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:05.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.607+0000 7f50b4a00640 1 --2- 172.21.15.42:0/4051178803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f508c069e80 0x7f508c06c340 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f50a00ae000 tx=0x7f50a00b3040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:05.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.611+0000 7f50ad600640 1 -- 172.21.15.42:0/4051178803 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f50a8056750 con 0x7f50b015b820 2024-05-28T14:14:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.851+0000 7f50b5e00640 1 -- 172.21.15.42:0/4051178803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f50b00b3bb0 con 0x7f50b015b820 2024-05-28T14:14:05.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.851+0000 7f50ad600640 1 -- 172.21.15.42:0/4051178803 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+4947 (secure 0 0 0) 0x7f50a805aa50 con 0x7f50b015b820 2024-05-28T14:14:05.839 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:05.839 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:14:05.839 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:14:05.839 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:14:05.840 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:14:05.840 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:14:05.840 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:14:05.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:05.840 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:14:05.841 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:14:05.841 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:14:05.841 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:14:05.841 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:14:05.841 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:05.842 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:14:05.842 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:14:05.842 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:14:05.842 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:14:05.842 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:05.842 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 967, 2024-05-28T14:14:05.843 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:14:05.843 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:14:05.843 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:14:05.843 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:14:05.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:05.844 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:14:05.844 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 119, 2024-05-28T14:14:05.844 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:14:05.844 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:14:05.844 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:14:05.845 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:14:05.845 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:14:05.845 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:14:05.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:05.846 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:14:05.846 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:14:05.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:05.846 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:14:05.846 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:14:05.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:05.847 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:05.847 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:14:05.847 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:14:05.847 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1775, 2024-05-28T14:14:05.847 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 7316203391, 2024-05-28T14:14:05.848 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 15696527360, 2024-05-28T14:14:05.848 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 757397585920, 2024-05-28T14:14:05.848 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:14:05.848 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 5425, 2024-05-28T14:14:05.848 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:14:05.848 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:14:05.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:05.849 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:14:05.849 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 75, 2024-05-28T14:14:05.849 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 8, 2024-05-28T14:14:05.849 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:14:05.850 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:14:05.850 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:14:05.850 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:14:05.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:05.850 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 8, 2024-05-28T14:14:05.850 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:14:05.851 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:14:05.851 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:14:05.851 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57440 2024-05-28T14:14:05.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:05.851 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:05.852 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:14:05.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:05.852 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:14:05.852 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:14:05.852 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:14:05.852 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:14:05.853 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:14:05.853 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:14:05.853 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:14:05.853 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:05.853 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:14:05.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:05.854 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:14:05.854 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 20, 2024-05-28T14:14:05.854 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:14:05.186488+0000", 2024-05-28T14:14:05.854 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-05-28T14:14:05.855 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-05-28T14:14:05.855 INFO:teuthology.orchestra.run.smithi042.stdout: "daemons": { 2024-05-28T14:14:05.855 INFO:teuthology.orchestra.run.smithi042.stdout: "summary": "", 2024-05-28T14:14:05.855 INFO:teuthology.orchestra.run.smithi042.stdout: "b": { 2024-05-28T14:14:05.855 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:05.856 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:05.856 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:05.856 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:05.856 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:05.856 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:05.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:05.857 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:05.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:05.857 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-05-28T14:14:05.857 INFO:teuthology.orchestra.run.smithi042.stdout: "daemons": { 2024-05-28T14:14:05.857 INFO:teuthology.orchestra.run.smithi042.stdout: "summary": "", 2024-05-28T14:14:05.858 INFO:teuthology.orchestra.run.smithi042.stdout: "0": { 2024-05-28T14:14:05.858 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:05.858 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:05.858 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:05.858 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:05.858 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:05.859 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:05.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:05.859 INFO:teuthology.orchestra.run.smithi042.stdout: "1": { 2024-05-28T14:14:05.859 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:05.859 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:05.860 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:05.860 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:05.860 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:05.860 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:05.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:05.861 INFO:teuthology.orchestra.run.smithi042.stdout: "2": { 2024-05-28T14:14:05.861 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:05.861 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:05.861 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:05.861 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:05.861 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:05.862 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:05.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:05.862 INFO:teuthology.orchestra.run.smithi042.stdout: "3": { 2024-05-28T14:14:05.862 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:05.863 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:05.863 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:05.863 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:05.863 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:05.863 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:05.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:05.863 INFO:teuthology.orchestra.run.smithi042.stdout: "5": { 2024-05-28T14:14:05.863 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:05.863 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:05.863 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:05.863 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: "6": { 2024-05-28T14:14:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: "7": { 2024-05-28T14:14:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:05.864 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:05.865 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:05.865 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:05.865 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:05.865 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:05.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:05.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:05.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:05.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:05.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:05.865 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:14:05.865 INFO:teuthology.orchestra.run.smithi042.stdout: "79821641-56ea-4c0f-b18b-17882adc2609": { 2024-05-28T14:14:05.865 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 15.0 seconds before exiting (21s)\n [============================] ", 2024-05-28T14:14:05.865 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:14:05.865 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:14:05.866 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:05.866 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:05.866 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:14:05.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.855+0000 7f50b5e00640 1 -- 172.21.15.42:0/4051178803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f508c069e80 msgr2=0x7f508c06c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:05.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.855+0000 7f50b5e00640 1 --2- 172.21.15.42:0/4051178803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f508c069e80 0x7f508c06c340 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f50a00ae000 tx=0x7f50a00b3040 comp rx=0 tx=0).stop 2024-05-28T14:14:05.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.855+0000 7f50b5e00640 1 -- 172.21.15.42:0/4051178803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50b015b820 msgr2=0x7f50b00af3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:05.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.855+0000 7f50b5e00640 1 --2- 172.21.15.42:0/4051178803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50b015b820 0x7f50b00af3a0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f50a800c730 tx=0x7f50a800cbf0 comp rx=0 tx=0).stop 2024-05-28T14:14:05.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.855+0000 7f50b5e00640 1 -- 172.21.15.42:0/4051178803 shutdown_connections 2024-05-28T14:14:05.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.855+0000 7f50b5e00640 1 --2- 172.21.15.42:0/4051178803 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f508c069e80 0x7f508c06c340 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:05.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.855+0000 7f50b5e00640 1 --2- 172.21.15.42:0/4051178803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50b015b820 0x7f50b00af3a0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:05.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.855+0000 7f50b5e00640 1 --2- 172.21.15.42:0/4051178803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50b01521a0 0x7f50b0147000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:05.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.855+0000 7f50b5e00640 1 --2- 172.21.15.42:0/4051178803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50b006ed20 0x7f50b00b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:05.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.855+0000 7f50b5e00640 1 -- 172.21.15.42:0/4051178803 >> 172.21.15.42:0/4051178803 conn(0x7f50b00b1fa0 msgr2=0x7f50b00d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:05.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.855+0000 7f50b5e00640 1 -- 172.21.15.42:0/4051178803 shutdown_connections 2024-05-28T14:14:05.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:05.855+0000 7f50b5e00640 1 -- 172.21.15.42:0/4051178803 wait complete. 2024-05-28T14:14:10.869 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:14:11.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.035+0000 7f73e7200640 1 Processor -- start 2024-05-28T14:14:11.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.035+0000 7f73e7200640 1 -- start start 2024-05-28T14:14:11.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e7200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f73e815e8b0 0x7f73e80b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:11.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e7200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f73e80b5d50 0x7f73e806b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:11.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e7200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73e80b46c0 0x7f73e806bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:11.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e7200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f73e809ae10 con 0x7f73e815e8b0 2024-05-28T14:14:11.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e7200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f73e80ca500 con 0x7f73e80b5d50 2024-05-28T14:14:11.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e7200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f73e80b61f0 con 0x7f73e80b46c0 2024-05-28T14:14:11.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e5e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f73e815e8b0 0x7f73e80b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:11.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e5e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f73e815e8b0 0x7f73e80b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35414/0 (socket says 172.21.15.42:35414) 2024-05-28T14:14:11.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e5e00640 1 -- 172.21.15.42:0/2128833914 learned_addr learned my addr 172.21.15.42:0/2128833914 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:11.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e6800640 1 --2- 172.21.15.42:0/2128833914 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73e80b46c0 0x7f73e806bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:11.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e5400640 1 --2- 172.21.15.42:0/2128833914 >> v2:172.21.15.63:3300/0 conn(0x7f73e80b5d50 0x7f73e806b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:11.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e5e00640 1 -- 172.21.15.42:0/2128833914 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73e80b46c0 msgr2=0x7f73e806bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:11.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e5e00640 1 --2- 172.21.15.42:0/2128833914 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73e80b46c0 0x7f73e806bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:11.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e5e00640 1 -- 172.21.15.42:0/2128833914 >> v2:172.21.15.63:3300/0 conn(0x7f73e80b5d50 msgr2=0x7f73e806b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:11.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e5e00640 1 --2- 172.21.15.42:0/2128833914 >> v2:172.21.15.63:3300/0 conn(0x7f73e80b5d50 0x7f73e806b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:11.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.039+0000 7f73e5e00640 1 -- 172.21.15.42:0/2128833914 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e80b64d0 con 0x7f73e815e8b0 2024-05-28T14:14:11.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e5e00640 1 --2- 172.21.15.42:0/2128833914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e815e8b0 0x7f73e80b4180 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f73e8072360 tx=0x7f73d801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f82617773b6b97 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:11.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e4a00640 1 -- 172.21.15.42:0/2128833914 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d80b1070 con 0x7f73e815e8b0 2024-05-28T14:14:11.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e4a00640 1 -- 172.21.15.42:0/2128833914 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73d80ac630 con 0x7f73e815e8b0 2024-05-28T14:14:11.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e4a00640 1 -- 172.21.15.42:0/2128833914 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d80a8050 con 0x7f73e815e8b0 2024-05-28T14:14:11.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 -- 172.21.15.42:0/2128833914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e815e8b0 msgr2=0x7f73e80b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:11.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 --2- 172.21.15.42:0/2128833914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e815e8b0 0x7f73e80b4180 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f73e8072360 tx=0x7f73d801c520 comp rx=0 tx=0).stop 2024-05-28T14:14:11.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 -- 172.21.15.42:0/2128833914 shutdown_connections 2024-05-28T14:14:11.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 --2- 172.21.15.42:0/2128833914 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73e80b46c0 0x7f73e806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:11.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 --2- 172.21.15.42:0/2128833914 >> v2:172.21.15.63:3300/0 conn(0x7f73e80b5d50 0x7f73e806b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:11.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 --2- 172.21.15.42:0/2128833914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e815e8b0 0x7f73e80b4180 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:11.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 -- 172.21.15.42:0/2128833914 >> 172.21.15.42:0/2128833914 conn(0x7f73e8073290 msgr2=0x7f73e80b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:11.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 -- 172.21.15.42:0/2128833914 shutdown_connections 2024-05-28T14:14:11.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 -- 172.21.15.42:0/2128833914 wait complete. 2024-05-28T14:14:11.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 Processor -- start 2024-05-28T14:14:11.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 -- start start 2024-05-28T14:14:11.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e80b46c0 0x7f73e80ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:11.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73e80b5d50 0x7f73e8160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:11.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e815e8b0 0x7f73e814f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:11.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f73e8162ef0 con 0x7f73e80b46c0 2024-05-28T14:14:11.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f73e8162d70 con 0x7f73e815e8b0 2024-05-28T14:14:11.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.043+0000 7f73e7200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f73e8163070 con 0x7f73e80b5d50 2024-05-28T14:14:11.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73e5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e80b46c0 0x7f73e80ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:11.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73e5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e80b46c0 0x7f73e80ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35420/0 (socket says 172.21.15.42:35420) 2024-05-28T14:14:11.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73e5e00640 1 -- 172.21.15.42:0/474309753 learned_addr learned my addr 172.21.15.42:0/474309753 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:11.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73e5e00640 1 -- 172.21.15.42:0/474309753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73e80b5d50 msgr2=0x7f73e8160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:11.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73e5e00640 1 --2- 172.21.15.42:0/474309753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73e80b5d50 0x7f73e8160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:11.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73e5e00640 1 -- 172.21.15.42:0/474309753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e815e8b0 msgr2=0x7f73e814f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:11.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73e5e00640 1 --2- 172.21.15.42:0/474309753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e815e8b0 0x7f73e814f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:11.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73e5e00640 1 -- 172.21.15.42:0/474309753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e80cc660 con 0x7f73e80b46c0 2024-05-28T14:14:11.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73e5e00640 1 --2- 172.21.15.42:0/474309753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e80b46c0 0x7f73e80ce630 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f73d80af040 tx=0x7f73d80192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:11.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73de000640 1 -- 172.21.15.42:0/474309753 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d80bc070 con 0x7f73e80b46c0 2024-05-28T14:14:11.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73de000640 1 -- 172.21.15.42:0/474309753 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73d80b7430 con 0x7f73e80b46c0 2024-05-28T14:14:11.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73de000640 1 -- 172.21.15.42:0/474309753 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d80b1040 con 0x7f73e80b46c0 2024-05-28T14:14:11.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73e7200640 1 -- 172.21.15.42:0/474309753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e80d0c80 con 0x7f73e80b46c0 2024-05-28T14:14:11.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73e7200640 1 -- 172.21.15.42:0/474309753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e80cefa0 con 0x7f73e80b46c0 2024-05-28T14:14:11.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73e7200640 1 -- 172.21.15.42:0/474309753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e80691a0 con 0x7f73e80b46c0 2024-05-28T14:14:11.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.047+0000 7f73de000640 1 -- 172.21.15.42:0/474309753 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7f73d80b63d0 con 0x7f73e80b46c0 2024-05-28T14:14:11.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.051+0000 7f73de000640 1 --2- 172.21.15.42:0/474309753 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73c8069eb0 0x7f73c806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:11.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.051+0000 7f73de000640 1 -- 172.21.15.42:0/474309753 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7f73d811f060 con 0x7f73e80b46c0 2024-05-28T14:14:11.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.051+0000 7f73e5400640 1 --2- 172.21.15.42:0/474309753 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73c8069eb0 0x7f73c806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:11.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.051+0000 7f73e5400640 1 --2- 172.21.15.42:0/474309753 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73c8069eb0 0x7f73c806c370 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f73e81fac50 tx=0x7f73d4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:11.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.055+0000 7f73de000640 1 -- 172.21.15.42:0/474309753 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f73d8016110 con 0x7f73e80b46c0 2024-05-28T14:14:11.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.283+0000 7f73e7200640 1 -- 172.21.15.42:0/474309753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f73e8154340 con 0x7f73e80b46c0 2024-05-28T14:14:11.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.283+0000 7f73de000640 1 -- 172.21.15.42:0/474309753 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+4502 (secure 0 0 0) 0x7f73d80f1080 con 0x7f73e80b46c0 2024-05-28T14:14:11.269 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:11.269 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:14:11.269 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:14:11.269 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:14:11.269 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:14:11.269 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:14:11.269 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:14:11.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:11.270 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:14:11.270 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:14:11.270 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:14:11.270 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:14:11.270 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:14:11.270 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:11.270 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:14:11.270 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:14:11.270 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:14:11.270 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:14:11.270 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:11.270 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 972, 2024-05-28T14:14:11.270 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:14:11.270 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:14:11.271 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:14:11.271 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:14:11.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:11.271 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:14:11.271 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 119, 2024-05-28T14:14:11.271 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:14:11.271 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:14:11.271 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:14:11.271 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:14:11.271 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:14:11.271 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:14:11.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:11.271 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:14:11.271 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:14:11.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:11.272 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:14:11.272 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:14:11.272 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:11.272 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:11.272 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:14:11.272 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:14:11.272 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1775, 2024-05-28T14:14:11.272 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 7316203391, 2024-05-28T14:14:11.272 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 15696527360, 2024-05-28T14:14:11.272 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 757397585920, 2024-05-28T14:14:11.272 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:14:11.272 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 1791, 2024-05-28T14:14:11.272 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:14:11.273 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:14:11.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:11.273 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:14:11.273 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 75, 2024-05-28T14:14:11.273 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 8, 2024-05-28T14:14:11.273 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:14:11.273 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:14:11.273 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:14:11.273 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:14:11.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:11.273 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 8, 2024-05-28T14:14:11.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:14:11.273 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:14:11.273 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:14:11.274 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57440 2024-05-28T14:14:11.274 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:11.274 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:11.274 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:14:11.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:11.274 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:14:11.274 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:14:11.274 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:14:11.274 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:14:11.274 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:14:11.274 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:14:11.274 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:14:11.274 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:11.274 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:14:11.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:11.275 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:14:11.275 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 21, 2024-05-28T14:14:11.275 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:14:07.186935+0000", 2024-05-28T14:14:11.275 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-05-28T14:14:11.275 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-05-28T14:14:11.275 INFO:teuthology.orchestra.run.smithi042.stdout: "daemons": { 2024-05-28T14:14:11.275 INFO:teuthology.orchestra.run.smithi042.stdout: "summary": "", 2024-05-28T14:14:11.275 INFO:teuthology.orchestra.run.smithi042.stdout: "0": { 2024-05-28T14:14:11.275 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:11.275 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:11.275 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:11.275 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:11.275 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: "1": { 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: "2": { 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:11.276 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: "3": { 2024-05-28T14:14:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: "5": { 2024-05-28T14:14:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:11.278 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:11.278 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:11.278 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:11.278 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:11.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:11.278 INFO:teuthology.orchestra.run.smithi042.stdout: "6": { 2024-05-28T14:14:11.278 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:11.278 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:11.278 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:11.278 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:11.278 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:11.278 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:11.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:11.278 INFO:teuthology.orchestra.run.smithi042.stdout: "7": { 2024-05-28T14:14:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:14:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:14:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:14:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:14:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:14:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:14:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:14:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: "79821641-56ea-4c0f-b18b-17882adc2609": { 2024-05-28T14:14:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 11.0 seconds before exiting (25s)\n [============================] ", 2024-05-28T14:14:11.280 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:14:11.280 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:14:11.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:11.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:11.280 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:14:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.283+0000 7f73e7200640 1 -- 172.21.15.42:0/474309753 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73c8069eb0 msgr2=0x7f73c806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.283+0000 7f73e7200640 1 --2- 172.21.15.42:0/474309753 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73c8069eb0 0x7f73c806c370 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f73e81fac50 tx=0x7f73d4008040 comp rx=0 tx=0).stop 2024-05-28T14:14:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.283+0000 7f73e7200640 1 -- 172.21.15.42:0/474309753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e80b46c0 msgr2=0x7f73e80ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.283+0000 7f73e7200640 1 --2- 172.21.15.42:0/474309753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e80b46c0 0x7f73e80ce630 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f73d80af040 tx=0x7f73d80192c0 comp rx=0 tx=0).stop 2024-05-28T14:14:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.283+0000 7f73e7200640 1 -- 172.21.15.42:0/474309753 shutdown_connections 2024-05-28T14:14:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.283+0000 7f73e7200640 1 --2- 172.21.15.42:0/474309753 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73c8069eb0 0x7f73c806c370 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.283+0000 7f73e7200640 1 --2- 172.21.15.42:0/474309753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73e815e8b0 0x7f73e814f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.283+0000 7f73e7200640 1 --2- 172.21.15.42:0/474309753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73e80b5d50 0x7f73e8160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.283+0000 7f73e7200640 1 --2- 172.21.15.42:0/474309753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e80b46c0 0x7f73e80ce630 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.283+0000 7f73e7200640 1 -- 172.21.15.42:0/474309753 >> 172.21.15.42:0/474309753 conn(0x7f73e8073290 msgr2=0x7f73e80cbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.283+0000 7f73e7200640 1 -- 172.21.15.42:0/474309753 shutdown_connections 2024-05-28T14:14:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:11.283+0000 7f73e7200640 1 -- 172.21.15.42:0/474309753 wait complete. 2024-05-28T14:14:16.289 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:14:16.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.455+0000 7f4f4cc00640 1 Processor -- start 2024-05-28T14:14:16.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.455+0000 7f4f4cc00640 1 -- start start 2024-05-28T14:14:16.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.455+0000 7f4f4cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f4806f430 0x7f4f4815f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:16.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.455+0000 7f4f4cc00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f4815c6e0 con 0x7f4f481521a0 2024-05-28T14:14:16.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.455+0000 7f4f4cc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f480b4590 con 0x7f4f4815b820 2024-05-28T14:14:16.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.455+0000 7f4f4cc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4815c400 con 0x7f4f4806f430 2024-05-28T14:14:16.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.459+0000 7f4f47400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f4f481521a0 0x7f4f4806ed20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:58080/0 (socket says 172.21.15.42:58080) 2024-05-28T14:14:16.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.459+0000 7f4f47400640 1 -- 172.21.15.42:0/2637590931 learned_addr learned my addr 172.21.15.42:0/2637590931 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:16.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.459+0000 7f4f3ea00640 1 --2- 172.21.15.42:0/2637590931 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f4806f430 0x7f4f4815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:16.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.459+0000 7f4f46a00640 1 -- 172.21.15.42:0/2637590931 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3107533315 0 0) 0x7f4f4815c6e0 con 0x7f4f481521a0 2024-05-28T14:14:16.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.459+0000 7f4f46a00640 1 -- 172.21.15.42:0/2637590931 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f2c003530 con 0x7f4f481521a0 2024-05-28T14:14:16.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.459+0000 7f4f46a00640 1 -- 172.21.15.42:0/2637590931 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3899462970 0 0) 0x7f4f480b4590 con 0x7f4f4815b820 2024-05-28T14:14:16.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.459+0000 7f4f46a00640 1 -- 172.21.15.42:0/2637590931 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f4815c6e0 con 0x7f4f4815b820 2024-05-28T14:14:16.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.459+0000 7f4f3ea00640 1 -- 172.21.15.42:0/2637590931 >> v1:172.21.15.63:6789/0 conn(0x7f4f4815b820 legacy=0x7f4f480721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:16.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.459+0000 7f4f3ea00640 1 -- 172.21.15.42:0/2637590931 >> v1:172.21.15.42:6789/0 conn(0x7f4f481521a0 legacy=0x7f4f4806ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:16.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.459+0000 7f4f3ea00640 1 -- 172.21.15.42:0/2637590931 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4815eeb0 con 0x7f4f4806f430 2024-05-28T14:14:16.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.459+0000 7f4f3ea00640 1 --2- 172.21.15.42:0/2637590931 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f4806f430 0x7f4f4815f990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009dc0 tx=0x7f4f480730d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=551a1534f266219 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:16.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f46a00640 1 -- 172.21.15.42:0/2637590931 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f300b2070 con 0x7f4f4806f430 2024-05-28T14:14:16.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f46a00640 1 -- 172.21.15.42:0/2637590931 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f300a5a20 con 0x7f4f4806f430 2024-05-28T14:14:16.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f46a00640 1 -- 172.21.15.42:0/2637590931 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f300a5df0 con 0x7f4f4806f430 2024-05-28T14:14:16.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 -- 172.21.15.42:0/2637590931 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f4806f430 msgr2=0x7f4f4815f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:16.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 --2- 172.21.15.42:0/2637590931 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f4806f430 0x7f4f4815f990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009dc0 tx=0x7f4f480730d0 comp rx=0 tx=0).stop 2024-05-28T14:14:16.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 -- 172.21.15.42:0/2637590931 shutdown_connections 2024-05-28T14:14:16.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 --2- 172.21.15.42:0/2637590931 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f4806f430 0x7f4f4815f990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:16.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 -- 172.21.15.42:0/2637590931 >> 172.21.15.42:0/2637590931 conn(0x7f4f480b1fa0 msgr2=0x7f4f480b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:16.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 -- 172.21.15.42:0/2637590931 shutdown_connections 2024-05-28T14:14:16.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 -- 172.21.15.42:0/2637590931 wait complete. 2024-05-28T14:14:16.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 Processor -- start 2024-05-28T14:14:16.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 -- start start 2024-05-28T14:14:16.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f4806f430 0x7f4f480cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:16.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f481521a0 0x7f4f48099990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:16.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f4815b820 0x7f4f48155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:16.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4f48152700 con 0x7f4f4815b820 2024-05-28T14:14:16.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4f48152580 con 0x7f4f481521a0 2024-05-28T14:14:16.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f4cc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4f480aed90 con 0x7f4f4806f430 2024-05-28T14:14:16.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f4815b820 0x7f4f48155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:16.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.463+0000 7f4f47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f4815b820 0x7f4f48155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35432/0 (socket says 172.21.15.42:35432) 2024-05-28T14:14:16.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f47e00640 1 -- 172.21.15.42:0/446645786 learned_addr learned my addr 172.21.15.42:0/446645786 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:16.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f3ea00640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f481521a0 0x7f4f48099990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:16.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f47400640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f4806f430 0x7f4f480cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:16.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f47e00640 1 -- 172.21.15.42:0/446645786 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f4806f430 msgr2=0x7f4f480cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:16.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f47e00640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f4806f430 0x7f4f480cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:16.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f47e00640 1 -- 172.21.15.42:0/446645786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f481521a0 msgr2=0x7f4f48099990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:16.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f47e00640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f481521a0 0x7f4f48099990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:16.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f47e00640 1 -- 172.21.15.42:0/446645786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f480cdfe0 con 0x7f4f4815b820 2024-05-28T14:14:16.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f47400640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f4806f430 0x7f4f480cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:16.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f47e00640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f4815b820 0x7f4f48155480 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000c3f0 tx=0x7f4f4000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:16.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f3fe00640 1 -- 172.21.15.42:0/446645786 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40017070 con 0x7f4f4815b820 2024-05-28T14:14:16.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f3fe00640 1 -- 172.21.15.42:0/446645786 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f400052d0 con 0x7f4f4815b820 2024-05-28T14:14:16.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f3fe00640 1 -- 172.21.15.42:0/446645786 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40003820 con 0x7f4f4815b820 2024-05-28T14:14:16.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f4cc00640 1 -- 172.21.15.42:0/446645786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f48099ed0 con 0x7f4f4815b820 2024-05-28T14:14:16.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f4cc00640 1 -- 172.21.15.42:0/446645786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f480ccce0 con 0x7f4f4815b820 2024-05-28T14:14:16.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f4cc00640 1 -- 172.21.15.42:0/446645786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f48068a90 con 0x7f4f4815b820 2024-05-28T14:14:16.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.467+0000 7f4f3fe00640 1 -- 172.21.15.42:0/446645786 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7f4f4001a070 con 0x7f4f4815b820 2024-05-28T14:14:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.471+0000 7f4f3fe00640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4f24069e80 0x7f4f2406c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.471+0000 7f4f3fe00640 1 -- 172.21.15.42:0/446645786 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7f4f40084b30 con 0x7f4f4815b820 2024-05-28T14:14:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.471+0000 7f4f47400640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4f24069e80 0x7f4f2406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:16.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.471+0000 7f4f47400640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4f24069e80 0x7f4f2406c340 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4f380011f0 tx=0x7f4f38009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:16.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.475+0000 7f4f3fe00640 1 -- 172.21.15.42:0/446645786 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4f480cdfe0 con 0x7f4f4815b820 2024-05-28T14:14:16.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.695+0000 7f4f4cc00640 1 -- 172.21.15.42:0/446645786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4f4815fb30 con 0x7f4f4815b820 2024-05-28T14:14:16.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.695+0000 7f4f3fe00640 1 -- 172.21.15.42:0/446645786 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2316 (secure 0 0 0) 0x7f4f40005480 con 0x7f4f4815b820 2024-05-28T14:14:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:16.682 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:14:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:14:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:14:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:14:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:14:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:14:16.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:14:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:14:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:14:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:14:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:14:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:14:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:14:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:14:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:14:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 978, 2024-05-28T14:14:16.683 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:14:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:14:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:14:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:14:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:14:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 119, 2024-05-28T14:14:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:14:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:14:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:14:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:14:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:14:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:14:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:16.684 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:14:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:14:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:14:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:14:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:14:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:14:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1775, 2024-05-28T14:14:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 7316206333, 2024-05-28T14:14:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 15715241984, 2024-05-28T14:14:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 757378871296, 2024-05-28T14:14:16.685 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:14:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_objects_per_sec": 4, 2024-05-28T14:14:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_bytes_per_sec": 20966434, 2024-05-28T14:14:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_keys_per_sec": 0, 2024-05-28T14:14:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects_recovered": 50, 2024-05-28T14:14:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "num_bytes_recovered": 209715200, 2024-05-28T14:14:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "num_keys_recovered": 0, 2024-05-28T14:14:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 307, 2024-05-28T14:14:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:14:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:14:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:14:16.686 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 75, 2024-05-28T14:14:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 8, 2024-05-28T14:14:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:14:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:14:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:14:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:14:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 8, 2024-05-28T14:14:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:14:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:14:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:14:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57440 2024-05-28T14:14:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:16.687 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:14:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:14:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:14:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:14:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:14:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:14:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:14:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:14:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:14:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:14:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 22, 2024-05-28T14:14:16.688 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:14:11.188092+0000", 2024-05-28T14:14:16.689 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:14:16.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:16.689 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:14:16.689 INFO:teuthology.orchestra.run.smithi042.stdout: "79821641-56ea-4c0f-b18b-17882adc2609": { 2024-05-28T14:14:16.689 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 5.0 seconds before exiting (32s)\n [============================] ", 2024-05-28T14:14:16.689 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:14:16.689 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:14:16.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:16.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:16.689 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:14:16.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.699+0000 7f4f4cc00640 1 -- 172.21.15.42:0/446645786 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4f24069e80 msgr2=0x7f4f2406c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:16.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.699+0000 7f4f4cc00640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4f24069e80 0x7f4f2406c340 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4f380011f0 tx=0x7f4f38009040 comp rx=0 tx=0).stop 2024-05-28T14:14:16.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.699+0000 7f4f4cc00640 1 -- 172.21.15.42:0/446645786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f4815b820 msgr2=0x7f4f48155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:16.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.699+0000 7f4f4cc00640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f4815b820 0x7f4f48155480 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000c3f0 tx=0x7f4f4000c8b0 comp rx=0 tx=0).stop 2024-05-28T14:14:16.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.699+0000 7f4f4cc00640 1 -- 172.21.15.42:0/446645786 shutdown_connections 2024-05-28T14:14:16.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.699+0000 7f4f4cc00640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4f24069e80 0x7f4f2406c340 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:16.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.699+0000 7f4f4cc00640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f4815b820 0x7f4f48155480 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:16.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.699+0000 7f4f4cc00640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f481521a0 0x7f4f48099990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:16.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.699+0000 7f4f4cc00640 1 --2- 172.21.15.42:0/446645786 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f4806f430 0x7f4f480cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:16.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.699+0000 7f4f4cc00640 1 -- 172.21.15.42:0/446645786 >> 172.21.15.42:0/446645786 conn(0x7f4f480b1fa0 msgr2=0x7f4f4815c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:16.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.699+0000 7f4f4cc00640 1 -- 172.21.15.42:0/446645786 shutdown_connections 2024-05-28T14:14:16.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:16.699+0000 7f4f4cc00640 1 -- 172.21.15.42:0/446645786 wait complete. 2024-05-28T14:14:21.099 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:14:21.102 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:14:21.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:14:21.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.871+0000 7f4b18400640 1 Processor -- start 2024-05-28T14:14:21.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.871+0000 7f4b18400640 1 -- start start 2024-05-28T14:14:21.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.871+0000 7f4b18400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4b100b52d0 0x7f4b1009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:21.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.871+0000 7f4b18400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b100ccce0 con 0x7f4b1009af90 2024-05-28T14:14:21.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.871+0000 7f4b18400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b100ccfc0 con 0x7f4b1015e9d0 2024-05-28T14:14:21.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.871+0000 7f4b18400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f4b100cd2a0 con 0x7f4b100b52d0 2024-05-28T14:14:21.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.871+0000 7f4b15000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4b100b52d0 0x7f4b1009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:21.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.871+0000 7f4b16400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f4b1009af90 0x7f4b100b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53086/0 (socket says 172.21.15.42:53086) 2024-05-28T14:14:21.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.871+0000 7f4b16400640 1 -- 172.21.15.42:0/25373304 learned_addr learned my addr 172.21.15.42:0/25373304 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:21.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.871+0000 7f4b15000640 1 -- 172.21.15.42:0/25373304 >> v1:172.21.15.63:6789/0 conn(0x7f4b1015e9d0 legacy=0x7f4b100d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:21.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.871+0000 7f4b15000640 1 -- 172.21.15.42:0/25373304 >> v1:172.21.15.42:6789/0 conn(0x7f4b1009af90 legacy=0x7f4b100b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:21.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.871+0000 7f4b15000640 1 -- 172.21.15.42:0/25373304 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b100cd580 con 0x7f4b100b52d0 2024-05-28T14:14:21.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.871+0000 7f4b0fe00640 1 -- 172.21.15.42:0/25373304 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 809481461 0 0) 0x7f4b100ccce0 con 0x7f4b1009af90 2024-05-28T14:14:21.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.875+0000 7f4b15000640 1 --2- 172.21.15.42:0/25373304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b100b52d0 0x7f4b1009aa50 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0097c0 tx=0x7f4afc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bc8c8bba6e87d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:21.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.875+0000 7f4b0fe00640 1 -- 172.21.15.42:0/25373304 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc0b0070 con 0x7f4b100b52d0 2024-05-28T14:14:21.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.875+0000 7f4b0fe00640 1 -- 172.21.15.42:0/25373304 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4afc0ab600 con 0x7f4b100b52d0 2024-05-28T14:14:21.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.875+0000 7f4b0fe00640 1 -- 172.21.15.42:0/25373304 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc0ab9d0 con 0x7f4b100b52d0 2024-05-28T14:14:21.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.875+0000 7f4b18400640 1 -- 172.21.15.42:0/25373304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b100b52d0 msgr2=0x7f4b1009aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:21.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.875+0000 7f4b18400640 1 --2- 172.21.15.42:0/25373304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b100b52d0 0x7f4b1009aa50 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0097c0 tx=0x7f4afc01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:14:21.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.875+0000 7f4b18400640 1 -- 172.21.15.42:0/25373304 shutdown_connections 2024-05-28T14:14:21.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.875+0000 7f4b18400640 1 --2- 172.21.15.42:0/25373304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b100b52d0 0x7f4b1009aa50 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:21.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.875+0000 7f4b18400640 1 -- 172.21.15.42:0/25373304 >> 172.21.15.42:0/25373304 conn(0x7f4b100b3f70 msgr2=0x7f4b100b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:21.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.875+0000 7f4b18400640 1 -- 172.21.15.42:0/25373304 shutdown_connections 2024-05-28T14:14:21.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.875+0000 7f4b18400640 1 -- 172.21.15.42:0/25373304 wait complete. 2024-05-28T14:14:21.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.875+0000 7f4b18400640 1 Processor -- start 2024-05-28T14:14:21.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b18400640 1 -- start start 2024-05-28T14:14:21.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b18400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b1009af90 0x7f4b1015d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:21.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b18400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b100b52d0 0x7f4b10154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:21.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b18400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b1015e9d0 0x7f4b10155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b18400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b100b3d30 con 0x7f4b100b52d0 2024-05-28T14:14:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b18400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b100b3bb0 con 0x7f4b1015e9d0 2024-05-28T14:14:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b18400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1016cdb0 con 0x7f4b1009af90 2024-05-28T14:14:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b15000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b100b52d0 0x7f4b10154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b15000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b100b52d0 0x7f4b10154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48142/0 (socket says 172.21.15.42:48142) 2024-05-28T14:14:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b15000640 1 -- 172.21.15.42:0/70570362 learned_addr learned my addr 172.21.15.42:0/70570362 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b15a00640 1 --2- 172.21.15.42:0/70570362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b1009af90 0x7f4b1015d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b16400640 1 --2- 172.21.15.42:0/70570362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b1015e9d0 0x7f4b10155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b15000640 1 -- 172.21.15.42:0/70570362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b1009af90 msgr2=0x7f4b1015d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b15000640 1 --2- 172.21.15.42:0/70570362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b1009af90 0x7f4b1015d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b15000640 1 -- 172.21.15.42:0/70570362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b1015e9d0 msgr2=0x7f4b10155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b15000640 1 --2- 172.21.15.42:0/70570362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b1015e9d0 0x7f4b10155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b15000640 1 -- 172.21.15.42:0/70570362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b10151530 con 0x7f4b100b52d0 2024-05-28T14:14:21.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b15000640 1 --2- 172.21.15.42:0/70570362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b100b52d0 0x7f4b10154f40 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0190b0 tx=0x7f4afc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:21.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b0d600640 1 -- 172.21.15.42:0/70570362 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc0b0040 con 0x7f4b100b52d0 2024-05-28T14:14:21.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b0d600640 1 -- 172.21.15.42:0/70570362 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4afc0ba070 con 0x7f4b100b52d0 2024-05-28T14:14:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b0d600640 1 -- 172.21.15.42:0/70570362 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc0b4870 con 0x7f4b100b52d0 2024-05-28T14:14:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b18400640 1 -- 172.21.15.42:0/70570362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b10151810 con 0x7f4b100b52d0 2024-05-28T14:14:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b18400640 1 -- 172.21.15.42:0/70570362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b10151dd0 con 0x7f4b100b52d0 2024-05-28T14:14:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.879+0000 7f4b18400640 1 -- 172.21.15.42:0/70570362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b10068a90 con 0x7f4b100b52d0 2024-05-28T14:14:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.883+0000 7f4b0d600640 1 -- 172.21.15.42:0/70570362 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7f4afc0ab480 con 0x7f4b100b52d0 2024-05-28T14:14:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.883+0000 7f4b0d600640 1 --2- 172.21.15.42:0/70570362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4af0069eb0 0x7f4af006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.883+0000 7f4b0d600640 1 -- 172.21.15.42:0/70570362 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7f4afc11f650 con 0x7f4b100b52d0 2024-05-28T14:14:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.883+0000 7f4b15a00640 1 --2- 172.21.15.42:0/70570362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4af0069eb0 0x7f4af006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:21.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.883+0000 7f4b15a00640 1 --2- 172.21.15.42:0/70570362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4af0069eb0 0x7f4af006c370 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4b100a03a0 tx=0x7f4b04009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:21.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:21.887+0000 7f4b0d600640 1 -- 172.21.15.42:0/70570362 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4afc016090 con 0x7f4b100b52d0 2024-05-28T14:14:22.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.111+0000 7f4b18400640 1 -- 172.21.15.42:0/70570362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4b1009b700 con 0x7f4b100b52d0 2024-05-28T14:14:22.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.111+0000 7f4b0d600640 1 -- 172.21.15.42:0/70570362 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2062 (secure 0 0 0) 0x7f4afc0f1670 con 0x7f4b100b52d0 2024-05-28T14:14:22.096 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:22.096 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:14:22.096 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:14:22.096 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:14:22.097 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:14:22.097 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:14:22.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:14:22.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:22.097 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:14:22.097 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:14:22.097 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:14:22.097 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:14:22.097 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:14:22.097 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:22.097 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:14:22.097 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:14:22.097 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:14:22.097 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:14:22.098 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:22.098 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 983, 2024-05-28T14:14:22.098 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:14:22.098 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:14:22.098 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:14:22.098 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:14:22.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:22.098 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:14:22.098 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 119, 2024-05-28T14:14:22.098 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:14:22.098 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:14:22.098 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:14:22.099 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:14:22.099 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:14:22.099 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:14:22.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:22.099 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:14:22.099 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:14:22.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:22.099 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:14:22.099 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:14:22.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:22.099 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:22.099 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:14:22.099 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:14:22.099 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 1775, 2024-05-28T14:14:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 7316206333, 2024-05-28T14:14:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 15715241984, 2024-05-28T14:14:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 757378871296, 2024-05-28T14:14:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:14:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_objects_per_sec": 4, 2024-05-28T14:14:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_bytes_per_sec": 17472142, 2024-05-28T14:14:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_keys_per_sec": 0, 2024-05-28T14:14:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects_recovered": 50, 2024-05-28T14:14:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: "num_bytes_recovered": 209715200, 2024-05-28T14:14:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: "num_keys_recovered": 0, 2024-05-28T14:14:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 255, 2024-05-28T14:14:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:14:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:14:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:22.101 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:14:22.101 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 75, 2024-05-28T14:14:22.101 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 8, 2024-05-28T14:14:22.101 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:14:22.101 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:14:22.101 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:14:22.101 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:14:22.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:22.101 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 8, 2024-05-28T14:14:22.101 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:14:22.101 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "b", 2024-05-28T14:14:22.101 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:14:22.101 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57440 2024-05-28T14:14:22.101 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:22.102 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:22.102 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:14:22.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:22.102 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:14:22.102 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:14:22.102 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:14:22.102 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:14:22.102 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:14:22.102 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:14:22.102 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:14:22.102 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:14:22.102 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:14:22.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:22.103 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:14:22.103 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 22, 2024-05-28T14:14:22.103 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:14:11.188092+0000", 2024-05-28T14:14:22.103 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:14:22.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:22.103 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": {} 2024-05-28T14:14:22.103 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:14:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.111+0000 7f4b18400640 1 -- 172.21.15.42:0/70570362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4af0069eb0 msgr2=0x7f4af006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.111+0000 7f4b18400640 1 --2- 172.21.15.42:0/70570362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4af0069eb0 0x7f4af006c370 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4b100a03a0 tx=0x7f4b04009040 comp rx=0 tx=0).stop 2024-05-28T14:14:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.111+0000 7f4b18400640 1 -- 172.21.15.42:0/70570362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b100b52d0 msgr2=0x7f4b10154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.111+0000 7f4b18400640 1 --2- 172.21.15.42:0/70570362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b100b52d0 0x7f4b10154f40 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0190b0 tx=0x7f4afc019240 comp rx=0 tx=0).stop 2024-05-28T14:14:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.111+0000 7f4b18400640 1 -- 172.21.15.42:0/70570362 shutdown_connections 2024-05-28T14:14:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.111+0000 7f4b18400640 1 --2- 172.21.15.42:0/70570362 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4af0069eb0 0x7f4af006c370 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.111+0000 7f4b18400640 1 --2- 172.21.15.42:0/70570362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b1015e9d0 0x7f4b10155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.111+0000 7f4b18400640 1 --2- 172.21.15.42:0/70570362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b100b52d0 0x7f4b10154f40 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.111+0000 7f4b18400640 1 --2- 172.21.15.42:0/70570362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b1009af90 0x7f4b1015d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.111+0000 7f4b18400640 1 -- 172.21.15.42:0/70570362 >> 172.21.15.42:0/70570362 conn(0x7f4b100b3f70 msgr2=0x7f4b10070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:22.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.111+0000 7f4b18400640 1 -- 172.21.15.42:0/70570362 shutdown_connections 2024-05-28T14:14:22.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.111+0000 7f4b18400640 1 -- 172.21.15.42:0/70570362 wait complete. 2024-05-28T14:14:22.118 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-28T14:14:22.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.275+0000 7f26bfe00640 1 Processor -- start 2024-05-28T14:14:22.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.275+0000 7f26bfe00640 1 -- start start 2024-05-28T14:14:22.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.275+0000 7f26bfe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f26c0145d50 0x7f26c00cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:22.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.275+0000 7f26bfe00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26c0160170 con 0x7f26c00cd050 2024-05-28T14:14:22.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.275+0000 7f26bfe00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26c0160450 con 0x7f26c00cf540 2024-05-28T14:14:22.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.275+0000 7f26bfe00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f26c0160730 con 0x7f26c0145d50 2024-05-28T14:14:22.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.279+0000 7f26bf400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f26c00cd050 0x7f26c00d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53100/0 (socket says 172.21.15.42:53100) 2024-05-28T14:14:22.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.279+0000 7f26bf400640 1 -- 172.21.15.42:0/1167113522 learned_addr learned my addr 172.21.15.42:0/1167113522 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:22.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.279+0000 7f26b6000640 1 --2- 172.21.15.42:0/1167113522 >> v2:172.21.15.63:3300/0 conn(0x7f26c0145d50 0x7f26c00cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.279+0000 7f26be000640 1 -- 172.21.15.42:0/1167113522 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2735993255 0 0) 0x7f26c0160170 con 0x7f26c00cd050 2024-05-28T14:14:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.279+0000 7f26be000640 1 -- 172.21.15.42:0/1167113522 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26a4003530 con 0x7f26c00cd050 2024-05-28T14:14:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.279+0000 7f26b6000640 1 -- 172.21.15.42:0/1167113522 >> v1:172.21.15.63:6789/0 conn(0x7f26c00cf540 legacy=0x7f26c00cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.279+0000 7f26b6000640 1 -- 172.21.15.42:0/1167113522 >> v1:172.21.15.42:6789/0 conn(0x7f26c00cd050 legacy=0x7f26c00d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.279+0000 7f26be000640 1 -- 172.21.15.42:0/1167113522 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2278674083 0 0) 0x7f26a4003530 con 0x7f26c00cd050 2024-05-28T14:14:22.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.279+0000 7f26b6000640 1 -- 172.21.15.42:0/1167113522 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c00707e0 con 0x7f26c0145d50 2024-05-28T14:14:22.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.279+0000 7f26b6000640 1 --2- 172.21.15.42:0/1167113522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26c0145d50 0x7f26c00cfec0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f26a80097c0 tx=0x7f26a801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b603a6887b6b637 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:22.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.279+0000 7f26be000640 1 -- 172.21.15.42:0/1167113522 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a80b1070 con 0x7f26c0145d50 2024-05-28T14:14:22.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26be000640 1 -- 172.21.15.42:0/1167113522 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a80a4c70 con 0x7f26c0145d50 2024-05-28T14:14:22.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26be000640 1 -- 172.21.15.42:0/1167113522 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8001680 con 0x7f26c0145d50 2024-05-28T14:14:22.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 -- 172.21.15.42:0/1167113522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26c0145d50 msgr2=0x7f26c00cfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 --2- 172.21.15.42:0/1167113522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26c0145d50 0x7f26c00cfec0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f26a80097c0 tx=0x7f26a801c330 comp rx=0 tx=0).stop 2024-05-28T14:14:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 -- 172.21.15.42:0/1167113522 shutdown_connections 2024-05-28T14:14:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 --2- 172.21.15.42:0/1167113522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26c0145d50 0x7f26c00cfec0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 -- 172.21.15.42:0/1167113522 >> 172.21.15.42:0/1167113522 conn(0x7f26c006fcc0 msgr2=0x7f26c00700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 -- 172.21.15.42:0/1167113522 shutdown_connections 2024-05-28T14:14:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 -- 172.21.15.42:0/1167113522 wait complete. 2024-05-28T14:14:22.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 Processor -- start 2024-05-28T14:14:22.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 -- start start 2024-05-28T14:14:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26c00cd050 0x7f26c00ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26c00cf540 0x7f26c00b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f26c0145d50 0x7f26c0157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f26c016ab70 con 0x7f26c00cf540 2024-05-28T14:14:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26c0070a70 con 0x7f26c00cd050 2024-05-28T14:14:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bfe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f26c016acf0 con 0x7f26c0145d50 2024-05-28T14:14:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26b6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26c00cf540 0x7f26c00b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26b6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26c00cf540 0x7f26c00b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48144/0 (socket says 172.21.15.42:48144) 2024-05-28T14:14:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26b6000640 1 -- 172.21.15.42:0/2685461571 learned_addr learned my addr 172.21.15.42:0/2685461571 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bf400640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f26c0145d50 0x7f26c0157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.283+0000 7f26bea00640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26c00cd050 0x7f26c00ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.287+0000 7f26b6000640 1 -- 172.21.15.42:0/2685461571 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f26c0145d50 msgr2=0x7f26c0157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.287+0000 7f26b6000640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f26c0145d50 0x7f26c0157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.287+0000 7f26b6000640 1 -- 172.21.15.42:0/2685461571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26c00cd050 msgr2=0x7f26c00ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.287+0000 7f26b6000640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26c00cd050 0x7f26c00ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.287+0000 7f26b6000640 1 -- 172.21.15.42:0/2685461571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c00cb060 con 0x7f26c00cf540 2024-05-28T14:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.287+0000 7f26bf400640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f26c0145d50 0x7f26c0157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:22.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.287+0000 7f26b6000640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26c00cf540 0x7f26c00b2bd0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f26a8018f40 tx=0x7f26a80190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:22.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.287+0000 7f26b7e00640 1 -- 172.21.15.42:0/2685461571 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a80b1040 con 0x7f26c00cf540 2024-05-28T14:14:22.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.287+0000 7f26b7e00640 1 -- 172.21.15.42:0/2685461571 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a80044c0 con 0x7f26c00cf540 2024-05-28T14:14:22.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.287+0000 7f26b7e00640 1 -- 172.21.15.42:0/2685461571 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8006e20 con 0x7f26c00cf540 2024-05-28T14:14:22.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.287+0000 7f26bfe00640 1 -- 172.21.15.42:0/2685461571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c0156a60 con 0x7f26c00cf540 2024-05-28T14:14:22.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.287+0000 7f26bfe00640 1 -- 172.21.15.42:0/2685461571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c00cb340 con 0x7f26c00cf540 2024-05-28T14:14:22.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.287+0000 7f26bfe00640 1 -- 172.21.15.42:0/2685461571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c004fcf0 con 0x7f26c00cf540 2024-05-28T14:14:22.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.287+0000 7f26b7e00640 1 -- 172.21.15.42:0/2685461571 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7f26a80bc020 con 0x7f26c00cf540 2024-05-28T14:14:22.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.291+0000 7f26b7e00640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f269c069e80 0x7f269c06c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:22.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.291+0000 7f26b7e00640 1 -- 172.21.15.42:0/2685461571 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7f26a811f0b0 con 0x7f26c00cf540 2024-05-28T14:14:22.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.291+0000 7f26bea00640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f269c069e80 0x7f269c06c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:22.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.291+0000 7f26bea00640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f269c069e80 0x7f269c06c340 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f26c015bbe0 tx=0x7f26b0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.295+0000 7f26b7e00640 1 -- 172.21.15.42:0/2685461571 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f26a80a4e20 con 0x7f26c00cf540 2024-05-28T14:14:22.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.487+0000 7f26bfe00640 1 -- 172.21.15.42:0/2685461571 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f26c00b1eb0 con 0x7f269c069e80 2024-05-28T14:14:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.491+0000 7f26b7e00640 1 -- 172.21.15.42:0/2685461571 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+185 (secure 0 0 0) 0x7f26c00b1eb0 con 0x7f269c069e80 2024-05-28T14:14:22.478 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:22.478 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:14:22.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:22.478 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone86011" 2024-05-28T14:14:22.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:22.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:22.479 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sv1" 2024-05-28T14:14:22.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:22.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:22.479 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone86013" 2024-05-28T14:14:22.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:22.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:22.479 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone86012" 2024-05-28T14:14:22.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:22.479 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:14:22.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.495+0000 7f26bfe00640 1 -- 172.21.15.42:0/2685461571 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f269c069e80 msgr2=0x7f269c06c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.495+0000 7f26bfe00640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f269c069e80 0x7f269c06c340 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f26c015bbe0 tx=0x7f26b0009040 comp rx=0 tx=0).stop 2024-05-28T14:14:22.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.495+0000 7f26bfe00640 1 -- 172.21.15.42:0/2685461571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26c00cf540 msgr2=0x7f26c00b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.495+0000 7f26bfe00640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26c00cf540 0x7f26c00b2bd0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f26a8018f40 tx=0x7f26a80190d0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.495+0000 7f26bfe00640 1 -- 172.21.15.42:0/2685461571 shutdown_connections 2024-05-28T14:14:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.495+0000 7f26bfe00640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f269c069e80 0x7f269c06c340 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.495+0000 7f26bfe00640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f26c0145d50 0x7f26c0157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.495+0000 7f26bfe00640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26c00cf540 0x7f26c00b2bd0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.495+0000 7f26bfe00640 1 --2- 172.21.15.42:0/2685461571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26c00cd050 0x7f26c00ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.495+0000 7f26bfe00640 1 -- 172.21.15.42:0/2685461571 >> 172.21.15.42:0/2685461571 conn(0x7f26c006fcc0 msgr2=0x7f26c015cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.495+0000 7f26bfe00640 1 -- 172.21.15.42:0/2685461571 shutdown_connections 2024-05-28T14:14:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.495+0000 7f26bfe00640 1 -- 172.21.15.42:0/2685461571 wait complete. 2024-05-28T14:14:22.501 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone86011 2024-05-28T14:14:22.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.659+0000 7f8960600640 1 Processor -- start 2024-05-28T14:14:22.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.659+0000 7f8960600640 1 -- start start 2024-05-28T14:14:22.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f8960600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f895806e740 0x7f89580cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:22.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f8960600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f895809b160 0x7f8958070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:22.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f8960600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f895806ff10 con 0x7f89580ce110 2024-05-28T14:14:22.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f8960600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f895806eb20 con 0x7f895806e740 2024-05-28T14:14:22.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f8960600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f89580701b0 con 0x7f895809b160 2024-05-28T14:14:22.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f895dc00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f895806e740 0x7f89580cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:22.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f895e600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f895809b160 0x7f8958070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:22.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f895e600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f895809b160 0x7f8958070f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:48120/0 (socket says 172.21.15.42:48120) 2024-05-28T14:14:22.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f895e600640 1 -- 172.21.15.42:0/2938691608 learned_addr learned my addr 172.21.15.42:0/2938691608 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:22.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f895d200640 1 --1- 172.21.15.42:0/2938691608 >> v1:172.21.15.63:6789/0 conn(0x7f89580ce110 0x7f895809aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:46686/0 (socket says 172.21.15.42:46686) 2024-05-28T14:14:22.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f895e600640 1 -- 172.21.15.42:0/2938691608 >> v2:172.21.15.63:3300/0 conn(0x7f895806e740 msgr2=0x7f89580cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:22.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f895e600640 1 --2- 172.21.15.42:0/2938691608 >> v2:172.21.15.63:3300/0 conn(0x7f895806e740 0x7f89580cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f895e600640 1 -- 172.21.15.42:0/2938691608 >> v1:172.21.15.63:6789/0 conn(0x7f89580ce110 legacy=0x7f895809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f895e600640 1 -- 172.21.15.42:0/2938691608 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8958070490 con 0x7f895809b160 2024-05-28T14:14:22.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.663+0000 7f8957e00640 1 -- 172.21.15.42:0/2938691608 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 744683438 0 0) 0x7f895806ff10 con 0x7f89580ce110 2024-05-28T14:14:22.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.667+0000 7f895e600640 1 --2- 172.21.15.42:0/2938691608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f895809b160 0x7f8958070f70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8950009e50 tx=0x7f895001ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e19b7e89c8d2df2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.667+0000 7f8957e00640 1 -- 172.21.15.42:0/2938691608 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89500a58d0 con 0x7f895809b160 2024-05-28T14:14:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.667+0000 7f8957e00640 1 -- 172.21.15.42:0/2938691608 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89500a5a70 con 0x7f895809b160 2024-05-28T14:14:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.667+0000 7f8957e00640 1 -- 172.21.15.42:0/2938691608 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89500a5e40 con 0x7f895809b160 2024-05-28T14:14:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.667+0000 7f8960600640 1 -- 172.21.15.42:0/2938691608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f895809b160 msgr2=0x7f8958070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.667+0000 7f8960600640 1 --2- 172.21.15.42:0/2938691608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f895809b160 0x7f8958070f70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8950009e50 tx=0x7f895001ca00 comp rx=0 tx=0).stop 2024-05-28T14:14:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.667+0000 7f8960600640 1 -- 172.21.15.42:0/2938691608 shutdown_connections 2024-05-28T14:14:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.667+0000 7f8960600640 1 --2- 172.21.15.42:0/2938691608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f895809b160 0x7f8958070f70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.667+0000 7f8960600640 1 --2- 172.21.15.42:0/2938691608 >> v2:172.21.15.63:3300/0 conn(0x7f895806e740 0x7f89580cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.667+0000 7f8960600640 1 -- 172.21.15.42:0/2938691608 >> 172.21.15.42:0/2938691608 conn(0x7f89580b3fa0 msgr2=0x7f89580b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.667+0000 7f8960600640 1 -- 172.21.15.42:0/2938691608 shutdown_connections 2024-05-28T14:14:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.667+0000 7f8960600640 1 -- 172.21.15.42:0/2938691608 wait complete. 2024-05-28T14:14:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.667+0000 7f8960600640 1 Processor -- start 2024-05-28T14:14:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.667+0000 7f8960600640 1 -- start start 2024-05-28T14:14:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f8960600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f895806e740 0x7f8958159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f8960600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f895809b160 0x7f895815a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f8960600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89580ce110 0x7f8958154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f8960600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f895815e2a0 con 0x7f895809b160 2024-05-28T14:14:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f8960600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f895815e120 con 0x7f89580ce110 2024-05-28T14:14:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f8960600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f895815e420 con 0x7f895806e740 2024-05-28T14:14:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f895d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f895809b160 0x7f895815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f895d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f895809b160 0x7f895815a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48160/0 (socket says 172.21.15.42:48160) 2024-05-28T14:14:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f895d200640 1 -- 172.21.15.42:0/3733673191 learned_addr learned my addr 172.21.15.42:0/3733673191 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f895dc00640 1 --2- 172.21.15.42:0/3733673191 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f895806e740 0x7f8958159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f895e600640 1 --2- 172.21.15.42:0/3733673191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89580ce110 0x7f8958154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f895d200640 1 -- 172.21.15.42:0/3733673191 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f895806e740 msgr2=0x7f8958159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f895d200640 1 --2- 172.21.15.42:0/3733673191 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f895806e740 0x7f8958159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f895d200640 1 -- 172.21.15.42:0/3733673191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89580ce110 msgr2=0x7f8958154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f895d200640 1 --2- 172.21.15.42:0/3733673191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89580ce110 0x7f8958154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f895d200640 1 -- 172.21.15.42:0/3733673191 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895815a8c0 con 0x7f895809b160 2024-05-28T14:14:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f895d200640 1 --2- 172.21.15.42:0/3733673191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f895809b160 0x7f895815a380 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f894400c480 tx=0x7f894400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f8955600640 1 -- 172.21.15.42:0/3733673191 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944018070 con 0x7f895809b160 2024-05-28T14:14:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f8955600640 1 -- 172.21.15.42:0/3733673191 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8944007500 con 0x7f895809b160 2024-05-28T14:14:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f8955600640 1 -- 172.21.15.42:0/3733673191 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944004a30 con 0x7f895809b160 2024-05-28T14:14:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f8960600640 1 -- 172.21.15.42:0/3733673191 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89580c97d0 con 0x7f895809b160 2024-05-28T14:14:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f8960600640 1 -- 172.21.15.42:0/3733673191 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89580c9d90 con 0x7f895809b160 2024-05-28T14:14:22.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.671+0000 7f8960600640 1 -- 172.21.15.42:0/3733673191 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8958068a90 con 0x7f895809b160 2024-05-28T14:14:22.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.675+0000 7f8955600640 1 -- 172.21.15.42:0/3733673191 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7f894401b070 con 0x7f895809b160 2024-05-28T14:14:22.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.675+0000 7f8955600640 1 --2- 172.21.15.42:0/3733673191 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8938069eb0 0x7f893806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:22.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.675+0000 7f8955600640 1 -- 172.21.15.42:0/3733673191 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7f8944084b30 con 0x7f895809b160 2024-05-28T14:14:22.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.675+0000 7f895dc00640 1 --2- 172.21.15.42:0/3733673191 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8938069eb0 0x7f893806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:22.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.675+0000 7f895dc00640 1 --2- 172.21.15.42:0/3733673191 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8938069eb0 0x7f893806c370 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f894c009af0 tx=0x7f894c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:22.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.679+0000 7f8955600640 1 -- 172.21.15.42:0/3733673191 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8944056bf0 con 0x7f895809b160 2024-05-28T14:14:22.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.871+0000 7f8960600640 1 -- 172.21.15.42:0/3733673191 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone86011", "target": ["mon-mgr", ""]}) -- 0x7f895806ee30 con 0x7f8938069eb0 2024-05-28T14:14:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.895+0000 7f8955600640 1 -- 172.21.15.42:0/3733673191 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f895806ee30 con 0x7f8938069eb0 2024-05-28T14:14:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.895+0000 7f8960600640 1 -- 172.21.15.42:0/3733673191 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8938069eb0 msgr2=0x7f893806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.895+0000 7f8960600640 1 --2- 172.21.15.42:0/3733673191 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8938069eb0 0x7f893806c370 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f894c009af0 tx=0x7f894c008040 comp rx=0 tx=0).stop 2024-05-28T14:14:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.895+0000 7f8960600640 1 -- 172.21.15.42:0/3733673191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f895809b160 msgr2=0x7f895815a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.895+0000 7f8960600640 1 --2- 172.21.15.42:0/3733673191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f895809b160 0x7f895815a380 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f894400c480 tx=0x7f894400c940 comp rx=0 tx=0).stop 2024-05-28T14:14:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.899+0000 7f8960600640 1 -- 172.21.15.42:0/3733673191 shutdown_connections 2024-05-28T14:14:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.899+0000 7f8960600640 1 --2- 172.21.15.42:0/3733673191 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8938069eb0 0x7f893806c370 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.899+0000 7f8960600640 1 --2- 172.21.15.42:0/3733673191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89580ce110 0x7f8958154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.899+0000 7f8960600640 1 --2- 172.21.15.42:0/3733673191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f895809b160 0x7f895815a380 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.899+0000 7f8960600640 1 --2- 172.21.15.42:0/3733673191 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f895806e740 0x7f8958159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.899+0000 7f8960600640 1 -- 172.21.15.42:0/3733673191 >> 172.21.15.42:0/3733673191 conn(0x7f89580b3fa0 msgr2=0x7f89580afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.899+0000 7f8960600640 1 -- 172.21.15.42:0/3733673191 shutdown_connections 2024-05-28T14:14:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:22.899+0000 7f8960600640 1 -- 172.21.15.42:0/3733673191 wait complete. 2024-05-28T14:14:22.884 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:22.899+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:22.885 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:22.899+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:22.885 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:22.899+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:22.885 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:22.899+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:22.885 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:22.899+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:22.901 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:22.915+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:22.901 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:22.915+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:22.902 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:22.915+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:22.902 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:22.915+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:22.902 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:22.915+0000 7f0776e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:22.907 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-28T14:14:23.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.067+0000 7ff3c8200640 1 Processor -- start 2024-05-28T14:14:23.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.067+0000 7ff3c8200640 1 -- start start 2024-05-28T14:14:23.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.067+0000 7ff3c8200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff3c015b350 0x7ff3c006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.067+0000 7ff3c8200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff3c00d0e00 0x7ff3c00aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3c8200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c00ade40 con 0x7ff3c0072d40 2024-05-28T14:14:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3c8200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff3c00ae9f0 con 0x7ff3c00d0e00 2024-05-28T14:14:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3c8200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff3c00ae0c0 con 0x7ff3c015b350 2024-05-28T14:14:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3c4e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff3c0072d40 0x7ff3c00d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53114/0 (socket says 172.21.15.42:53114) 2024-05-28T14:14:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3c4e00640 1 -- 172.21.15.42:0/596729046 learned_addr learned my addr 172.21.15.42:0/596729046 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3c6200640 1 --2- 172.21.15.42:0/596729046 >> v2:172.21.15.42:3300/0 conn(0x7ff3c00d0e00 0x7ff3c00aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3c5800640 1 --2- 172.21.15.42:0/596729046 >> v2:172.21.15.63:3300/0 conn(0x7ff3c015b350 0x7ff3c006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3bfe00640 1 -- 172.21.15.42:0/596729046 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1691460507 0 0) 0x7ff3ac001120 con 0x7ff3c0072d40 2024-05-28T14:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3c6200640 1 -- 172.21.15.42:0/596729046 >> v2:172.21.15.63:3300/0 conn(0x7ff3c015b350 msgr2=0x7ff3c006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3c6200640 1 --2- 172.21.15.42:0/596729046 >> v2:172.21.15.63:3300/0 conn(0x7ff3c015b350 0x7ff3c006dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3c6200640 1 -- 172.21.15.42:0/596729046 >> v1:172.21.15.42:6789/0 conn(0x7ff3c0072d40 legacy=0x7ff3c00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3c6200640 1 -- 172.21.15.42:0/596729046 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c00ae3a0 con 0x7ff3c00d0e00 2024-05-28T14:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3c6200640 1 --2- 172.21.15.42:0/596729046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c00d0e00 0x7ff3c00aee80 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00d13c0 tx=0x7ff3b801c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8818cc8587d5c63c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3bfe00640 1 -- 172.21.15.42:0/596729046 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80b2070 con 0x7ff3c00d0e00 2024-05-28T14:14:23.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3bfe00640 1 -- 172.21.15.42:0/596729046 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b80a5750 con 0x7ff3c00d0e00 2024-05-28T14:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.071+0000 7ff3bfe00640 1 -- 172.21.15.42:0/596729046 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80a5b20 con 0x7ff3c00d0e00 2024-05-28T14:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 -- 172.21.15.42:0/596729046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c00d0e00 msgr2=0x7ff3c00aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 --2- 172.21.15.42:0/596729046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c00d0e00 0x7ff3c00aee80 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00d13c0 tx=0x7ff3b801c810 comp rx=0 tx=0).stop 2024-05-28T14:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 -- 172.21.15.42:0/596729046 shutdown_connections 2024-05-28T14:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 --2- 172.21.15.42:0/596729046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c00d0e00 0x7ff3c00aee80 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 --2- 172.21.15.42:0/596729046 >> v2:172.21.15.63:3300/0 conn(0x7ff3c015b350 0x7ff3c006dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 -- 172.21.15.42:0/596729046 >> 172.21.15.42:0/596729046 conn(0x7ff3c006bdf0 msgr2=0x7ff3c006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 -- 172.21.15.42:0/596729046 shutdown_connections 2024-05-28T14:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 -- 172.21.15.42:0/596729046 wait complete. 2024-05-28T14:14:23.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 Processor -- start 2024-05-28T14:14:23.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 -- start start 2024-05-28T14:14:23.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c0072d40 0x7ff3c0156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3c00d0e00 0x7ff3c014cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c015b350 0x7ff3c0164790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0168470 con 0x7ff3c0072d40 2024-05-28T14:14:23.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01682f0 con 0x7ff3c00d0e00 2024-05-28T14:14:23.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c8200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c01685f0 con 0x7ff3c015b350 2024-05-28T14:14:23.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c0072d40 0x7ff3c0156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c0072d40 0x7ff3c0156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48220/0 (socket says 172.21.15.42:48220) 2024-05-28T14:14:23.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c5800640 1 -- 172.21.15.42:0/1668009719 learned_addr learned my addr 172.21.15.42:0/1668009719 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:23.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c6200640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c015b350 0x7ff3c0164790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c4e00640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3c00d0e00 0x7ff3c014cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c5800640 1 -- 172.21.15.42:0/1668009719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c015b350 msgr2=0x7ff3c0164790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c5800640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c015b350 0x7ff3c0164790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c5800640 1 -- 172.21.15.42:0/1668009719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3c00d0e00 msgr2=0x7ff3c014cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c5800640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3c00d0e00 0x7ff3c014cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c5800640 1 -- 172.21.15.42:0/1668009719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c014dc10 con 0x7ff3c0072d40 2024-05-28T14:14:23.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.075+0000 7ff3c4e00640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3c00d0e00 0x7ff3c014cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:23.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.079+0000 7ff3c5800640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c0072d40 0x7ff3c0156840 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400e870 tx=0x7ff3b400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:23.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.079+0000 7ff3bd600640 1 -- 172.21.15.42:0/1668009719 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4017070 con 0x7ff3c0072d40 2024-05-28T14:14:23.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.079+0000 7ff3bd600640 1 -- 172.21.15.42:0/1668009719 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b400c940 con 0x7ff3c0072d40 2024-05-28T14:14:23.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.079+0000 7ff3c8200640 1 -- 172.21.15.42:0/1668009719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c0158fd0 con 0x7ff3c0072d40 2024-05-28T14:14:23.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.079+0000 7ff3bd600640 1 -- 172.21.15.42:0/1668009719 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400ccf0 con 0x7ff3c0072d40 2024-05-28T14:14:23.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.079+0000 7ff3c8200640 1 -- 172.21.15.42:0/1668009719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c0159590 con 0x7ff3c0072d40 2024-05-28T14:14:23.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.079+0000 7ff3c8200640 1 -- 172.21.15.42:0/1668009719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c0068a90 con 0x7ff3c0072d40 2024-05-28T14:14:23.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.079+0000 7ff3bd600640 1 -- 172.21.15.42:0/1668009719 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7ff3b401a070 con 0x7ff3c0072d40 2024-05-28T14:14:23.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.079+0000 7ff3bd600640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3a0069e80 0x7ff3a006c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.079+0000 7ff3bd600640 1 -- 172.21.15.42:0/1668009719 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7ff3b40856c0 con 0x7ff3c0072d40 2024-05-28T14:14:23.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.079+0000 7ff3c4e00640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3a0069e80 0x7ff3a006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.083+0000 7ff3c4e00640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3a0069e80 0x7ff3a006c340 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff3c009e4e0 tx=0x7ff3ac007a20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:23.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.087+0000 7ff3bd600640 1 -- 172.21.15.42:0/1668009719 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff3b4057090 con 0x7ff3c0072d40 2024-05-28T14:14:23.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.279+0000 7ff3c8200640 1 -- 172.21.15.42:0/1668009719 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3c0072b20 con 0x7ff3a0069e80 2024-05-28T14:14:23.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.291+0000 7ff3bd600640 1 -- 172.21.15.42:0/1668009719 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff3c0072b20 con 0x7ff3a0069e80 2024-05-28T14:14:23.278 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:23.278 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:14:23.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:23.278 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ss1" 2024-05-28T14:14:23.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:23.278 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:14:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.295+0000 7ff3c8200640 1 -- 172.21.15.42:0/1668009719 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3a0069e80 msgr2=0x7ff3a006c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.295+0000 7ff3c8200640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3a0069e80 0x7ff3a006c340 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff3c009e4e0 tx=0x7ff3ac007a20 comp rx=0 tx=0).stop 2024-05-28T14:14:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.295+0000 7ff3c8200640 1 -- 172.21.15.42:0/1668009719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c0072d40 msgr2=0x7ff3c0156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.295+0000 7ff3c8200640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c0072d40 0x7ff3c0156840 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400e870 tx=0x7ff3b400ec70 comp rx=0 tx=0).stop 2024-05-28T14:14:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.295+0000 7ff3c8200640 1 -- 172.21.15.42:0/1668009719 shutdown_connections 2024-05-28T14:14:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.295+0000 7ff3c8200640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3a0069e80 0x7ff3a006c340 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.295+0000 7ff3c8200640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c015b350 0x7ff3c0164790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.295+0000 7ff3c8200640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3c00d0e00 0x7ff3c014cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.295+0000 7ff3c8200640 1 --2- 172.21.15.42:0/1668009719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c0072d40 0x7ff3c0156840 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.295+0000 7ff3c8200640 1 -- 172.21.15.42:0/1668009719 >> 172.21.15.42:0/1668009719 conn(0x7ff3c006bdf0 msgr2=0x7ff3c006e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:23.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.295+0000 7ff3c8200640 1 -- 172.21.15.42:0/1668009719 shutdown_connections 2024-05-28T14:14:23.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.295+0000 7ff3c8200640 1 -- 172.21.15.42:0/1668009719 wait complete. 2024-05-28T14:14:23.300 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:14:23.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.459+0000 7ff8d4000640 1 Processor -- start 2024-05-28T14:14:23.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.459+0000 7ff8d4000640 1 -- start start 2024-05-28T14:14:23.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.463+0000 7ff8d4000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff8cc071de0 0x7ff8cc0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.463+0000 7ff8d4000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff8cc0ce3b0 0x7ff8cc0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.463+0000 7ff8d4000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8cc072700 con 0x7ff8cc141d10 2024-05-28T14:14:23.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.463+0000 7ff8d4000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ff8cc160230 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.463+0000 7ff8d4000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff8cc160510 con 0x7ff8cc071de0 2024-05-28T14:14:23.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.463+0000 7ff8d0c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff8cc0ce3b0 0x7ff8cc0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.463+0000 7ff8d0c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff8cc0ce3b0 0x7ff8cc0ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48236/0 (socket says 172.21.15.42:48236) 2024-05-28T14:14:23.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.463+0000 7ff8d0c00640 1 -- 172.21.15.42:0/2281592391 learned_addr learned my addr 172.21.15.42:0/2281592391 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:23.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.463+0000 7ff8d2000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff8cc141d10 0x7ff8cc142160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53122/0 (socket says 172.21.15.42:53122) 2024-05-28T14:14:23.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.463+0000 7ff8d1600640 1 --2- 172.21.15.42:0/2281592391 >> v2:172.21.15.63:3300/0 conn(0x7ff8cc071de0 0x7ff8cc0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.463+0000 7ff8d0c00640 1 -- 172.21.15.42:0/2281592391 >> v2:172.21.15.63:3300/0 conn(0x7ff8cc071de0 msgr2=0x7ff8cc0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:23.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.463+0000 7ff8d0c00640 1 --2- 172.21.15.42:0/2281592391 >> v2:172.21.15.63:3300/0 conn(0x7ff8cc071de0 0x7ff8cc0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.463+0000 7ff8d0c00640 1 -- 172.21.15.42:0/2281592391 >> v1:172.21.15.42:6789/0 conn(0x7ff8cc141d10 legacy=0x7ff8cc142160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.463+0000 7ff8d0c00640 1 -- 172.21.15.42:0/2281592391 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc1607f0 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.467+0000 7ff8d0c00640 1 --2- 172.21.15.42:0/2281592391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0ce3b0 0x7ff8cc0ce810 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc06cbf0 tx=0x7ff8b801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601f553e51af08c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:23.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.467+0000 7ff8cbe00640 1 -- 172.21.15.42:0/2281592391 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80b0070 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.467+0000 7ff8cbe00640 1 -- 172.21.15.42:0/2281592391 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b80ab600 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.467+0000 7ff8cbe00640 1 -- 172.21.15.42:0/2281592391 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80ab9d0 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.467+0000 7ff8d4000640 1 -- 172.21.15.42:0/2281592391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0ce3b0 msgr2=0x7ff8cc0ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.467+0000 7ff8d4000640 1 --2- 172.21.15.42:0/2281592391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0ce3b0 0x7ff8cc0ce810 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc06cbf0 tx=0x7ff8b801c400 comp rx=0 tx=0).stop 2024-05-28T14:14:23.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.467+0000 7ff8d4000640 1 -- 172.21.15.42:0/2281592391 shutdown_connections 2024-05-28T14:14:23.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.467+0000 7ff8d4000640 1 --2- 172.21.15.42:0/2281592391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0ce3b0 0x7ff8cc0ce810 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.467+0000 7ff8d4000640 1 --2- 172.21.15.42:0/2281592391 >> v2:172.21.15.63:3300/0 conn(0x7ff8cc071de0 0x7ff8cc0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.467+0000 7ff8d4000640 1 -- 172.21.15.42:0/2281592391 >> 172.21.15.42:0/2281592391 conn(0x7ff8cc0b21f0 msgr2=0x7ff8cc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:23.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.467+0000 7ff8d4000640 1 -- 172.21.15.42:0/2281592391 shutdown_connections 2024-05-28T14:14:23.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.467+0000 7ff8d4000640 1 -- 172.21.15.42:0/2281592391 wait complete. 2024-05-28T14:14:23.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.467+0000 7ff8d4000640 1 Processor -- start 2024-05-28T14:14:23.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.467+0000 7ff8d4000640 1 -- start start 2024-05-28T14:14:23.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d4000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8cc071de0 0x7ff8cc0b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d4000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0ce3b0 0x7ff8cc13a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d4000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc141d10 0x7ff8cc1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d4000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc169e30 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d4000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc169cb0 con 0x7ff8cc071de0 2024-05-28T14:14:23.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d4000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff8cc169fb0 con 0x7ff8cc141d10 2024-05-28T14:14:23.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0ce3b0 0x7ff8cc13a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0ce3b0 0x7ff8cc13a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48242/0 (socket says 172.21.15.42:48242) 2024-05-28T14:14:23.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d0c00640 1 -- 172.21.15.42:0/971683233 learned_addr learned my addr 172.21.15.42:0/971683233 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:23.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d0c00640 1 -- 172.21.15.42:0/971683233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc141d10 msgr2=0x7ff8cc1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d0c00640 1 --2- 172.21.15.42:0/971683233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc141d10 0x7ff8cc1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d0c00640 1 -- 172.21.15.42:0/971683233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8cc071de0 msgr2=0x7ff8cc0b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d0c00640 1 --2- 172.21.15.42:0/971683233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8cc071de0 0x7ff8cc0b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d0c00640 1 -- 172.21.15.42:0/971683233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc139a30 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d0c00640 1 --2- 172.21.15.42:0/971683233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0ce3b0 0x7ff8cc13a160 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8019010 tx=0x7ff8b80191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:23.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8c9600640 1 -- 172.21.15.42:0/971683233 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80b0040 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8c9600640 1 -- 172.21.15.42:0/971683233 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b80b4620 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8c9600640 1 -- 172.21.15.42:0/971683233 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80b4a10 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d4000640 1 -- 172.21.15.42:0/971683233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc139d10 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d4000640 1 -- 172.21.15.42:0/971683233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc0b00f0 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8d4000640 1 -- 172.21.15.42:0/971683233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8cc0738b0 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.471+0000 7ff8c9600640 1 -- 172.21.15.42:0/971683233 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7ff8b80ab4c0 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.475+0000 7ff8c9600640 1 --2- 172.21.15.42:0/971683233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8ac069eb0 0x7ff8ac06c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.475+0000 7ff8c9600640 1 -- 172.21.15.42:0/971683233 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7ff8b811f530 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.475+0000 7ff8d1600640 1 --2- 172.21.15.42:0/971683233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8ac069eb0 0x7ff8ac06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.475+0000 7ff8d1600640 1 --2- 172.21.15.42:0/971683233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8ac069eb0 0x7ff8ac06c370 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000b850 tx=0x7ff8c0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:23.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.479+0000 7ff8c9600640 1 -- 172.21.15.42:0/971683233 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff8b8016110 con 0x7ff8cc0ce3b0 2024-05-28T14:14:23.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.675+0000 7ff8d4000640 1 -- 172.21.15.42:0/971683233 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8cc0b0b50 con 0x7ff8ac069eb0 2024-05-28T14:14:23.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.703+0000 7ff8c9600640 1 -- 172.21.15.42:0/971683233 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff8c000be30 con 0x7ff8ac069eb0 2024-05-28T14:14:23.689 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:23.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.707+0000 7ff8d4000640 1 -- 172.21.15.42:0/971683233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8ac069eb0 msgr2=0x7ff8ac06c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.707+0000 7ff8d4000640 1 --2- 172.21.15.42:0/971683233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8ac069eb0 0x7ff8ac06c370 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000b850 tx=0x7ff8c0008040 comp rx=0 tx=0).stop 2024-05-28T14:14:23.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.707+0000 7ff8d4000640 1 -- 172.21.15.42:0/971683233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0ce3b0 msgr2=0x7ff8cc13a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.707+0000 7ff8d4000640 1 --2- 172.21.15.42:0/971683233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0ce3b0 0x7ff8cc13a160 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8019010 tx=0x7ff8b80191a0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.707+0000 7ff8d4000640 1 -- 172.21.15.42:0/971683233 shutdown_connections 2024-05-28T14:14:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.707+0000 7ff8d4000640 1 --2- 172.21.15.42:0/971683233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8ac069eb0 0x7ff8ac06c370 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.707+0000 7ff8d4000640 1 --2- 172.21.15.42:0/971683233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc141d10 0x7ff8cc1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.707+0000 7ff8d4000640 1 --2- 172.21.15.42:0/971683233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0ce3b0 0x7ff8cc13a160 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.707+0000 7ff8d4000640 1 --2- 172.21.15.42:0/971683233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8cc071de0 0x7ff8cc0b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.707+0000 7ff8d4000640 1 -- 172.21.15.42:0/971683233 >> 172.21.15.42:0/971683233 conn(0x7ff8cc0b21f0 msgr2=0x7ff8cc1411c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.707+0000 7ff8d4000640 1 -- 172.21.15.42:0/971683233 shutdown_connections 2024-05-28T14:14:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.707+0000 7ff8d4000640 1 -- 172.21.15.42:0/971683233 wait complete. 2024-05-28T14:14:23.711 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-28T14:14:23.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0639000640 1 Processor -- start 2024-05-28T14:14:23.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0639000640 1 -- start start 2024-05-28T14:14:23.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0639000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0634071de0 0x7f06340721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0639000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06340ce3b0 0x7f06340ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0639000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0634141d10 0x7f06341421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0639000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0634072700 con 0x7f0634141d10 2024-05-28T14:14:23.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0639000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0634160230 con 0x7f0634071de0 2024-05-28T14:14:23.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0639000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0634160510 con 0x7f06340ce3b0 2024-05-28T14:14:23.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0632800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0634141d10 0x7f06341421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0632800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0634141d10 0x7f06341421a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48252/0 (socket says 172.21.15.42:48252) 2024-05-28T14:14:23.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0632800640 1 -- 172.21.15.42:0/3373054648 learned_addr learned my addr 172.21.15.42:0/3373054648 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:23.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0631400640 1 --2- 172.21.15.42:0/3373054648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06340ce3b0 0x7f06340ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0631e00640 1 --2- 172.21.15.42:0/3373054648 >> v2:172.21.15.63:3300/0 conn(0x7f0634071de0 0x7f06340721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0632800640 1 -- 172.21.15.42:0/3373054648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06340ce3b0 msgr2=0x7f06340ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:23.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0632800640 1 --2- 172.21.15.42:0/3373054648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06340ce3b0 0x7f06340ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0632800640 1 -- 172.21.15.42:0/3373054648 >> v2:172.21.15.63:3300/0 conn(0x7f0634071de0 msgr2=0x7f06340721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:23.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0632800640 1 --2- 172.21.15.42:0/3373054648 >> v2:172.21.15.63:3300/0 conn(0x7f0634071de0 0x7f06340721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.871+0000 7f0632800640 1 -- 172.21.15.42:0/3373054648 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06341607f0 con 0x7f0634141d10 2024-05-28T14:14:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0632800640 1 --2- 172.21.15.42:0/3373054648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0634141d10 0x7f06341421a0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f063406cbf0 tx=0x7f062c01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7936c93816c6d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0630a00640 1 -- 172.21.15.42:0/3373054648 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c0a5550 con 0x7f0634141d10 2024-05-28T14:14:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0630a00640 1 -- 172.21.15.42:0/3373054648 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062c0a56f0 con 0x7f0634141d10 2024-05-28T14:14:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0630a00640 1 -- 172.21.15.42:0/3373054648 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c0a5ac0 con 0x7f0634141d10 2024-05-28T14:14:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0639000640 1 -- 172.21.15.42:0/3373054648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0634141d10 msgr2=0x7f06341421a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0639000640 1 --2- 172.21.15.42:0/3373054648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0634141d10 0x7f06341421a0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f063406cbf0 tx=0x7f062c01c8d0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0639000640 1 -- 172.21.15.42:0/3373054648 shutdown_connections 2024-05-28T14:14:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0639000640 1 --2- 172.21.15.42:0/3373054648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0634141d10 0x7f06341421a0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0639000640 1 --2- 172.21.15.42:0/3373054648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06340ce3b0 0x7f06340ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0639000640 1 --2- 172.21.15.42:0/3373054648 >> v2:172.21.15.63:3300/0 conn(0x7f0634071de0 0x7f06340721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0639000640 1 -- 172.21.15.42:0/3373054648 >> 172.21.15.42:0/3373054648 conn(0x7f06340b21f0 msgr2=0x7f06340b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:23.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0639000640 1 -- 172.21.15.42:0/3373054648 shutdown_connections 2024-05-28T14:14:23.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0639000640 1 -- 172.21.15.42:0/3373054648 wait complete. 2024-05-28T14:14:23.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0639000640 1 Processor -- start 2024-05-28T14:14:23.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.875+0000 7f0639000640 1 -- start start 2024-05-28T14:14:23.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0639000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0634071de0 0x7f0634159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0639000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06340ce3b0 0x7f0634138b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0639000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0634141d10 0x7f063413c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0639000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0634140520 con 0x7f0634071de0 2024-05-28T14:14:23.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0639000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f063413f8c0 con 0x7f06340ce3b0 2024-05-28T14:14:23.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0639000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f06341406a0 con 0x7f0634141d10 2024-05-28T14:14:23.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0631e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0634071de0 0x7f0634159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0631e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0634071de0 0x7f0634159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48262/0 (socket says 172.21.15.42:48262) 2024-05-28T14:14:23.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0631e00640 1 -- 172.21.15.42:0/2182338432 learned_addr learned my addr 172.21.15.42:0/2182338432 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:23.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0631400640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06340ce3b0 0x7f0634138b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0632800640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0634141d10 0x7f063413c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0631e00640 1 -- 172.21.15.42:0/2182338432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0634141d10 msgr2=0x7f063413c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0631e00640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0634141d10 0x7f063413c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0631e00640 1 -- 172.21.15.42:0/2182338432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06340ce3b0 msgr2=0x7f0634138b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0631e00640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06340ce3b0 0x7f0634138b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0631e00640 1 -- 172.21.15.42:0/2182338432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063415a0b0 con 0x7f0634071de0 2024-05-28T14:14:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0632800640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0634141d10 0x7f063413c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0631400640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06340ce3b0 0x7f0634138b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0631e00640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0634071de0 0x7f0634159b70 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f062400d800 tx=0x7f062400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f062a000640 1 -- 172.21.15.42:0/2182338432 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624018020 con 0x7f0634071de0 2024-05-28T14:14:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f062a000640 1 -- 172.21.15.42:0/2182338432 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06240096a0 con 0x7f0634071de0 2024-05-28T14:14:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f062a000640 1 -- 172.21.15.42:0/2182338432 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624009a50 con 0x7f0634071de0 2024-05-28T14:14:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0639000640 1 -- 172.21.15.42:0/2182338432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063413a610 con 0x7f0634071de0 2024-05-28T14:14:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0639000640 1 -- 172.21.15.42:0/2182338432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06340b3690 con 0x7f0634071de0 2024-05-28T14:14:23.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.879+0000 7f0639000640 1 -- 172.21.15.42:0/2182338432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06340738b0 con 0x7f0634071de0 2024-05-28T14:14:23.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.883+0000 7f062a000640 1 -- 172.21.15.42:0/2182338432 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 90435+0+0 (secure 0 0 0) 0x7f062401b020 con 0x7f0634071de0 2024-05-28T14:14:23.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.883+0000 7f062a000640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0614069e80 0x7f061406c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:23.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.883+0000 7f062a000640 1 -- 172.21.15.42:0/2182338432 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7f0624085ea0 con 0x7f0634071de0 2024-05-28T14:14:23.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.883+0000 7f0631400640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0614069e80 0x7f061406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:23.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.883+0000 7f0631400640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0614069e80 0x7f061406c340 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0634138650 tx=0x7f0620008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:23.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:23.887+0000 7f062a000640 1 -- 172.21.15.42:0/2182338432 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0624022020 con 0x7f0634071de0 2024-05-28T14:14:24.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.035+0000 7f062a000640 1 -- 172.21.15.42:0/2182338432 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f0624053e30 con 0x7f0634071de0 2024-05-28T14:14:24.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.087+0000 7f0639000640 1 -- 172.21.15.42:0/2182338432 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f063415a290 con 0x7f0614069e80 2024-05-28T14:14:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.107+0000 7f062a000640 1 -- 172.21.15.42:0/2182338432 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f063415a290 con 0x7f0614069e80 2024-05-28T14:14:24.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.107+0000 7f0639000640 1 -- 172.21.15.42:0/2182338432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0614069e80 msgr2=0x7f061406c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.107+0000 7f0639000640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0614069e80 0x7f061406c340 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0634138650 tx=0x7f0620008040 comp rx=0 tx=0).stop 2024-05-28T14:14:24.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.107+0000 7f0639000640 1 -- 172.21.15.42:0/2182338432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0634071de0 msgr2=0x7f0634159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.107+0000 7f0639000640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0634071de0 0x7f0634159b70 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f062400d800 tx=0x7f062400dcc0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.107+0000 7f0639000640 1 -- 172.21.15.42:0/2182338432 shutdown_connections 2024-05-28T14:14:24.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.107+0000 7f0639000640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0614069e80 0x7f061406c340 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.107+0000 7f0639000640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0634141d10 0x7f063413c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.107+0000 7f0639000640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06340ce3b0 0x7f0634138b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.107+0000 7f0639000640 1 --2- 172.21.15.42:0/2182338432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0634071de0 0x7f0634159b70 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.107+0000 7f0639000640 1 -- 172.21.15.42:0/2182338432 >> 172.21.15.42:0/2182338432 conn(0x7f06340b21f0 msgr2=0x7f06340b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:24.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.107+0000 7f0639000640 1 -- 172.21.15.42:0/2182338432 shutdown_connections 2024-05-28T14:14:24.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.107+0000 7f0639000640 1 -- 172.21.15.42:0/2182338432 wait complete. 2024-05-28T14:14:24.116 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone86013 2024-05-28T14:14:24.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0cf5800640 1 Processor -- start 2024-05-28T14:14:24.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0cf5800640 1 -- start start 2024-05-28T14:14:24.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0cf5800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cf00d0fe0 0x7f0cf01630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:24.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0cf5800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cf0162600 con 0x7f0cf00d1f20 2024-05-28T14:14:24.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0cf5800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cf0070ac0 con 0x7f0cf00d0c00 2024-05-28T14:14:24.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0cf5800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf01628a0 con 0x7f0cf00d0fe0 2024-05-28T14:14:24.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0cef400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0cf00d1f20 0x7f0cf006b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53130/0 (socket says 172.21.15.42:53130) 2024-05-28T14:14:24.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0cef400640 1 -- 172.21.15.42:0/1428745841 learned_addr learned my addr 172.21.15.42:0/1428745841 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:24.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0ce7e00640 1 --2- 172.21.15.42:0/1428745841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cf00d0fe0 0x7f0cf01630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:24.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0cee000640 1 -- 172.21.15.42:0/1428745841 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2830826445 0 0) 0x7f0cf0162600 con 0x7f0cf00d1f20 2024-05-28T14:14:24.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0cee000640 1 -- 172.21.15.42:0/1428745841 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cd4003530 con 0x7f0cf00d1f20 2024-05-28T14:14:24.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0cee000640 1 -- 172.21.15.42:0/1428745841 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569358740 0 0) 0x7f0cf0070ac0 con 0x7f0cf00d0c00 2024-05-28T14:14:24.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0ce7e00640 1 -- 172.21.15.42:0/1428745841 >> v1:172.21.15.63:6789/0 conn(0x7f0cf00d0c00 legacy=0x7f0cf00702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0ce7e00640 1 -- 172.21.15.42:0/1428745841 >> v1:172.21.15.42:6789/0 conn(0x7f0cf00d1f20 legacy=0x7f0cf006b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.279+0000 7f0ce7e00640 1 -- 172.21.15.42:0/1428745841 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf0162b80 con 0x7f0cf00d0fe0 2024-05-28T14:14:24.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.283+0000 7f0ce7e00640 1 --2- 172.21.15.42:0/1428745841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cf00d0fe0 0x7f0cf01630a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80097f0 tx=0x7f0cd801ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c330dab66c0db350 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:24.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.283+0000 7f0cee000640 1 -- 172.21.15.42:0/1428745841 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80b5070 con 0x7f0cf00d0fe0 2024-05-28T14:14:24.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.283+0000 7f0cee000640 1 -- 172.21.15.42:0/1428745841 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd80a89d0 con 0x7f0cf00d0fe0 2024-05-28T14:14:24.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.283+0000 7f0cee000640 1 -- 172.21.15.42:0/1428745841 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80a8da0 con 0x7f0cf00d0fe0 2024-05-28T14:14:24.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.283+0000 7f0cf5800640 1 -- 172.21.15.42:0/1428745841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cf00d0fe0 msgr2=0x7f0cf01630a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.283+0000 7f0cf5800640 1 --2- 172.21.15.42:0/1428745841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cf00d0fe0 0x7f0cf01630a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80097f0 tx=0x7f0cd801ecd0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.283+0000 7f0cf5800640 1 -- 172.21.15.42:0/1428745841 shutdown_connections 2024-05-28T14:14:24.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.283+0000 7f0cf5800640 1 --2- 172.21.15.42:0/1428745841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cf00d0fe0 0x7f0cf01630a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.283+0000 7f0cf5800640 1 -- 172.21.15.42:0/1428745841 >> 172.21.15.42:0/1428745841 conn(0x7f0cf00d1400 msgr2=0x7f0cf00d1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:24.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.283+0000 7f0cf5800640 1 -- 172.21.15.42:0/1428745841 shutdown_connections 2024-05-28T14:14:24.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.283+0000 7f0cf5800640 1 -- 172.21.15.42:0/1428745841 wait complete. 2024-05-28T14:14:24.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0cf5800640 1 Processor -- start 2024-05-28T14:14:24.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0cf5800640 1 -- start start 2024-05-28T14:14:24.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0cf5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cf00d0c00 0x7f0cf0149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:24.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0cf5800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cf00d0fe0 0x7f0cf01e9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:24.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0cf5800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cf00d1f20 0x7f0cf006fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:24.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0cf5800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf006c280 con 0x7f0cf00d0c00 2024-05-28T14:14:24.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0cf5800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf006c100 con 0x7f0cf00d1f20 2024-05-28T14:14:24.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0cf5800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf00cba50 con 0x7f0cf00d0fe0 2024-05-28T14:14:24.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0ceea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cf00d0c00 0x7f0cf0149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:24.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0ceea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cf00d0c00 0x7f0cf0149ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48264/0 (socket says 172.21.15.42:48264) 2024-05-28T14:14:24.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0ceea00640 1 -- 172.21.15.42:0/407031658 learned_addr learned my addr 172.21.15.42:0/407031658 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:24.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0ce7e00640 1 --2- 172.21.15.42:0/407031658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cf00d0fe0 0x7f0cf01e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:24.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0cef400640 1 --2- 172.21.15.42:0/407031658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cf00d1f20 0x7f0cf006fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:24.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0ceea00640 1 -- 172.21.15.42:0/407031658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cf00d0fe0 msgr2=0x7f0cf01e9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0ceea00640 1 --2- 172.21.15.42:0/407031658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cf00d0fe0 0x7f0cf01e9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0ceea00640 1 -- 172.21.15.42:0/407031658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cf00d1f20 msgr2=0x7f0cf006fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0ceea00640 1 --2- 172.21.15.42:0/407031658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cf00d1f20 0x7f0cf006fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0ceea00640 1 -- 172.21.15.42:0/407031658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf014cb50 con 0x7f0cf00d0c00 2024-05-28T14:14:24.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0ceea00640 1 --2- 172.21.15.42:0/407031658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cf00d0c00 0x7f0cf0149ff0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000c4d0 tx=0x7f0ce000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:24.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0ce6a00640 1 -- 172.21.15.42:0/407031658 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0017070 con 0x7f0cf00d0c00 2024-05-28T14:14:24.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0ce6a00640 1 -- 172.21.15.42:0/407031658 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce0005480 con 0x7f0cf00d0c00 2024-05-28T14:14:24.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0ce6a00640 1 -- 172.21.15.42:0/407031658 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0003840 con 0x7f0cf00d0c00 2024-05-28T14:14:24.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0cf5800640 1 -- 172.21.15.42:0/407031658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf014ce30 con 0x7f0cf00d0c00 2024-05-28T14:14:24.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0cf5800640 1 -- 172.21.15.42:0/407031658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf014d3f0 con 0x7f0cf00d0c00 2024-05-28T14:14:24.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.287+0000 7f0cf5800640 1 -- 172.21.15.42:0/407031658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf0068a90 con 0x7f0cf00d0c00 2024-05-28T14:14:24.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.291+0000 7f0ce6a00640 1 -- 172.21.15.42:0/407031658 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f0ce001a070 con 0x7f0cf00d0c00 2024-05-28T14:14:24.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.291+0000 7f0ce6a00640 1 --2- 172.21.15.42:0/407031658 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ccc069fa0 0x7f0ccc06c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:24.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.291+0000 7f0ce6a00640 1 -- 172.21.15.42:0/407031658 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7f0ce0084c90 con 0x7f0cf00d0c00 2024-05-28T14:14:24.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.291+0000 7f0ce7e00640 1 --2- 172.21.15.42:0/407031658 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ccc069fa0 0x7f0ccc06c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:24.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.291+0000 7f0ce7e00640 1 --2- 172.21.15.42:0/407031658 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ccc069fa0 0x7f0ccc06c460 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80b2000 tx=0x7f0cd8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:24.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.295+0000 7f0ce6a00640 1 -- 172.21.15.42:0/407031658 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ce0056720 con 0x7f0cf00d0c00 2024-05-28T14:14:24.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.487+0000 7f0cf5800640 1 -- 172.21.15.42:0/407031658 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone86013", "target": ["mon-mgr", ""]}) -- 0x7f0cf0161230 con 0x7f0ccc069fa0 2024-05-28T14:14:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.507+0000 7f0ce6a00640 1 -- 172.21.15.42:0/407031658 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0cf0161230 con 0x7f0ccc069fa0 2024-05-28T14:14:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.511+0000 7f0cf5800640 1 -- 172.21.15.42:0/407031658 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ccc069fa0 msgr2=0x7f0ccc06c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.511+0000 7f0cf5800640 1 --2- 172.21.15.42:0/407031658 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ccc069fa0 0x7f0ccc06c460 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80b2000 tx=0x7f0cd8008040 comp rx=0 tx=0).stop 2024-05-28T14:14:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.511+0000 7f0cf5800640 1 -- 172.21.15.42:0/407031658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cf00d0c00 msgr2=0x7f0cf0149ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.511+0000 7f0cf5800640 1 --2- 172.21.15.42:0/407031658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cf00d0c00 0x7f0cf0149ff0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000c4d0 tx=0x7f0ce000c990 comp rx=0 tx=0).stop 2024-05-28T14:14:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.511+0000 7f0cf5800640 1 -- 172.21.15.42:0/407031658 shutdown_connections 2024-05-28T14:14:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.511+0000 7f0cf5800640 1 --2- 172.21.15.42:0/407031658 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ccc069fa0 0x7f0ccc06c460 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.511+0000 7f0cf5800640 1 --2- 172.21.15.42:0/407031658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cf00d1f20 0x7f0cf006fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.511+0000 7f0cf5800640 1 --2- 172.21.15.42:0/407031658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cf00d0fe0 0x7f0cf01e9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.511+0000 7f0cf5800640 1 --2- 172.21.15.42:0/407031658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cf00d0c00 0x7f0cf0149ff0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.511+0000 7f0cf5800640 1 -- 172.21.15.42:0/407031658 >> 172.21.15.42:0/407031658 conn(0x7f0cf00d1400 msgr2=0x7f0cf00d17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:24.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.511+0000 7f0cf5800640 1 -- 172.21.15.42:0/407031658 shutdown_connections 2024-05-28T14:14:24.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.511+0000 7f0cf5800640 1 -- 172.21.15.42:0/407031658 wait complete. 2024-05-28T14:14:24.520 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone86012 2024-05-28T14:14:24.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.679+0000 7f58e9400640 1 Processor -- start 2024-05-28T14:14:24.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.679+0000 7f58e9400640 1 -- start start 2024-05-28T14:14:24.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.683+0000 7f58e9400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f58e406ee80 0x7f58e406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:24.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.683+0000 7f58e9400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e41570a0 con 0x7f58e40af360 2024-05-28T14:14:24.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.683+0000 7f58e9400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e415cf00 con 0x7f58e40d0e00 2024-05-28T14:14:24.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.683+0000 7f58e9400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f58e415d1e0 con 0x7f58e406ee80 2024-05-28T14:14:24.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.683+0000 7f58dbe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f58e40af360 0x7f58e40d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53136/0 (socket says 172.21.15.42:53136) 2024-05-28T14:14:24.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.683+0000 7f58dbe00640 1 -- 172.21.15.42:0/1918310412 learned_addr learned my addr 172.21.15.42:0/1918310412 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:24.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.683+0000 7f58e2800640 1 --2- 172.21.15.42:0/1918310412 >> v2:172.21.15.63:3300/0 conn(0x7f58e406ee80 0x7f58e406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:24.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.683+0000 7f58e1e00640 1 -- 172.21.15.42:0/1918310412 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3368717785 0 0) 0x7f58e41570a0 con 0x7f58e40af360 2024-05-28T14:14:24.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.683+0000 7f58e1e00640 1 -- 172.21.15.42:0/1918310412 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58c8003530 con 0x7f58e40af360 2024-05-28T14:14:24.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.683+0000 7f58e1e00640 1 -- 172.21.15.42:0/1918310412 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 859790253 0 0) 0x7f58c8003530 con 0x7f58e40af360 2024-05-28T14:14:24.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.683+0000 7f58e2800640 1 -- 172.21.15.42:0/1918310412 >> v1:172.21.15.63:6789/0 conn(0x7f58e40d0e00 legacy=0x7f58e4160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.683+0000 7f58e2800640 1 -- 172.21.15.42:0/1918310412 >> v1:172.21.15.42:6789/0 conn(0x7f58e40af360 legacy=0x7f58e40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.683+0000 7f58e2800640 1 -- 172.21.15.42:0/1918310412 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e415d4c0 con 0x7f58e406ee80 2024-05-28T14:14:24.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.687+0000 7f58e2800640 1 --2- 172.21.15.42:0/1918310412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58e406ee80 0x7f58e406dd40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f58d40097c0 tx=0x7f58d401c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a44fb08dc6642722 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:24.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.687+0000 7f58e1e00640 1 -- 172.21.15.42:0/1918310412 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d40b0070 con 0x7f58e406ee80 2024-05-28T14:14:24.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.687+0000 7f58e1e00640 1 -- 172.21.15.42:0/1918310412 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d40ab400 con 0x7f58e406ee80 2024-05-28T14:14:24.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.687+0000 7f58e1e00640 1 -- 172.21.15.42:0/1918310412 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc001680 con 0x7f58e406ee80 2024-05-28T14:14:24.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.687+0000 7f58e9400640 1 -- 172.21.15.42:0/1918310412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58e406ee80 msgr2=0x7f58e406dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.687+0000 7f58e9400640 1 --2- 172.21.15.42:0/1918310412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58e406ee80 0x7f58e406dd40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f58d40097c0 tx=0x7f58d401c300 comp rx=0 tx=0).stop 2024-05-28T14:14:24.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.687+0000 7f58e9400640 1 -- 172.21.15.42:0/1918310412 shutdown_connections 2024-05-28T14:14:24.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.687+0000 7f58e9400640 1 --2- 172.21.15.42:0/1918310412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58e406ee80 0x7f58e406dd40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.687+0000 7f58e9400640 1 -- 172.21.15.42:0/1918310412 >> 172.21.15.42:0/1918310412 conn(0x7f58e406bdf0 msgr2=0x7f58e406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:24.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.687+0000 7f58e9400640 1 -- 172.21.15.42:0/1918310412 shutdown_connections 2024-05-28T14:14:24.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.687+0000 7f58e9400640 1 -- 172.21.15.42:0/1918310412 wait complete. 2024-05-28T14:14:24.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.687+0000 7f58e9400640 1 Processor -- start 2024-05-28T14:14:24.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.687+0000 7f58e9400640 1 -- start start 2024-05-28T14:14:24.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58e406ee80 0x7f58e4153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:24.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58e40af360 0x7f58e413d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:24.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e9400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58e40d0e00 0x7f58e4156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:24.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e9400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f58e415cce0 con 0x7f58e406ee80 2024-05-28T14:14:24.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e9400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f58e415cb60 con 0x7f58e40af360 2024-05-28T14:14:24.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e9400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f58e4160fa0 con 0x7f58e40d0e00 2024-05-28T14:14:24.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58e406ee80 0x7f58e4153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:24.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58e406ee80 0x7f58e4153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48274/0 (socket says 172.21.15.42:48274) 2024-05-28T14:14:24.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e2800640 1 -- 172.21.15.42:0/2949125800 learned_addr learned my addr 172.21.15.42:0/2949125800 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:24.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58dbe00640 1 --2- 172.21.15.42:0/2949125800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58e40af360 0x7f58e413d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:24.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e3200640 1 --2- 172.21.15.42:0/2949125800 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58e40d0e00 0x7f58e4156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:24.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e2800640 1 -- 172.21.15.42:0/2949125800 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58e40d0e00 msgr2=0x7f58e4156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e2800640 1 --2- 172.21.15.42:0/2949125800 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58e40d0e00 0x7f58e4156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e2800640 1 -- 172.21.15.42:0/2949125800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58e40af360 msgr2=0x7f58e413d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e2800640 1 --2- 172.21.15.42:0/2949125800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58e40af360 0x7f58e413d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e2800640 1 -- 172.21.15.42:0/2949125800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e413dca0 con 0x7f58e406ee80 2024-05-28T14:14:24.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e2800640 1 --2- 172.21.15.42:0/2949125800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58e406ee80 0x7f58e4153df0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f58d4018f10 tx=0x7f58d40190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:24.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58daa00640 1 -- 172.21.15.42:0/2949125800 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d40b9020 con 0x7f58e406ee80 2024-05-28T14:14:24.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58daa00640 1 -- 172.21.15.42:0/2949125800 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d4003920 con 0x7f58e406ee80 2024-05-28T14:14:24.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58daa00640 1 -- 172.21.15.42:0/2949125800 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d40b0040 con 0x7f58e406ee80 2024-05-28T14:14:24.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e9400640 1 -- 172.21.15.42:0/2949125800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e413df80 con 0x7f58e406ee80 2024-05-28T14:14:24.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e9400640 1 -- 172.21.15.42:0/2949125800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e4155d60 con 0x7f58e406ee80 2024-05-28T14:14:24.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.691+0000 7f58e9400640 1 -- 172.21.15.42:0/2949125800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e4068a90 con 0x7f58e406ee80 2024-05-28T14:14:24.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.695+0000 7f58daa00640 1 -- 172.21.15.42:0/2949125800 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f58d40bc020 con 0x7f58e406ee80 2024-05-28T14:14:24.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.695+0000 7f58daa00640 1 --2- 172.21.15.42:0/2949125800 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f58c0069fa0 0x7f58c006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:24.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.695+0000 7f58daa00640 1 -- 172.21.15.42:0/2949125800 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7f58d411fbc0 con 0x7f58e406ee80 2024-05-28T14:14:24.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.695+0000 7f58dbe00640 1 --2- 172.21.15.42:0/2949125800 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f58c0069fa0 0x7f58c006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:24.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.695+0000 7f58dbe00640 1 --2- 172.21.15.42:0/2949125800 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f58c0069fa0 0x7f58c006c460 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f58e41fb030 tx=0x7f58cc005800 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:24.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.699+0000 7f58daa00640 1 -- 172.21.15.42:0/2949125800 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f58d40c3030 con 0x7f58e406ee80 2024-05-28T14:14:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.891+0000 7f58e9400640 1 -- 172.21.15.42:0/2949125800 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone86012", "target": ["mon-mgr", ""]}) -- 0x7f58e40b5160 con 0x7f58c0069fa0 2024-05-28T14:14:24.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.907+0000 7f58daa00640 1 -- 172.21.15.42:0/2949125800 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f58e40b5160 con 0x7f58c0069fa0 2024-05-28T14:14:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.911+0000 7f58e9400640 1 -- 172.21.15.42:0/2949125800 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f58c0069fa0 msgr2=0x7f58c006c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.911+0000 7f58e9400640 1 --2- 172.21.15.42:0/2949125800 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f58c0069fa0 0x7f58c006c460 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f58e41fb030 tx=0x7f58cc005800 comp rx=0 tx=0).stop 2024-05-28T14:14:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.911+0000 7f58e9400640 1 -- 172.21.15.42:0/2949125800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58e406ee80 msgr2=0x7f58e4153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.911+0000 7f58e9400640 1 --2- 172.21.15.42:0/2949125800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58e406ee80 0x7f58e4153df0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f58d4018f10 tx=0x7f58d40190a0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.911+0000 7f58e9400640 1 -- 172.21.15.42:0/2949125800 shutdown_connections 2024-05-28T14:14:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.911+0000 7f58e9400640 1 --2- 172.21.15.42:0/2949125800 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f58c0069fa0 0x7f58c006c460 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.911+0000 7f58e9400640 1 --2- 172.21.15.42:0/2949125800 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58e40d0e00 0x7f58e4156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.911+0000 7f58e9400640 1 --2- 172.21.15.42:0/2949125800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58e40af360 0x7f58e413d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.911+0000 7f58e9400640 1 --2- 172.21.15.42:0/2949125800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58e406ee80 0x7f58e4153df0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.911+0000 7f58e9400640 1 -- 172.21.15.42:0/2949125800 >> 172.21.15.42:0/2949125800 conn(0x7f58e406bdf0 msgr2=0x7f58e415be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.911+0000 7f58e9400640 1 -- 172.21.15.42:0/2949125800 shutdown_connections 2024-05-28T14:14:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24.911+0000 7f58e9400640 1 -- 172.21.15.42:0/2949125800 wait complete. 2024-05-28T14:14:24.917 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-28T14:14:24.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-28T14:14:24.976 INFO:teuthology.orchestra.run.smithi042.stdout:2 2024-05-28T14:14:24.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:14:24.983 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T14:14:24.984 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:14:25.061 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:14:25.108 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T14:14:25.108 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:14:25.109 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:14:25.109 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:14:25.118 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:14:25.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.363+0000 7f9312c00640 1 Processor -- start 2024-05-28T14:14:25.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.363+0000 7f9312c00640 1 -- start start 2024-05-28T14:14:25.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.363+0000 7f9312c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f930c15c990 0x7f930c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:25.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.363+0000 7f9312c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f930c09bbd0 0x7f930c09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:25.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.363+0000 7f9312c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f930c0cd300 con 0x7f930c0d06a0 2024-05-28T14:14:25.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.363+0000 7f9312c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f930c15cd70 con 0x7f930c15c990 2024-05-28T14:14:25.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.363+0000 7f9312c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f930c0b63f0 con 0x7f930c09bbd0 2024-05-28T14:14:25.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.363+0000 7f930be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f930c15c990 0x7f930c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:25.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.363+0000 7f9310c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f930c0d06a0 0x7f930c160400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53140/0 (socket says 172.21.15.42:53140) 2024-05-28T14:14:25.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.363+0000 7f9310c00640 1 -- 172.21.15.42:0/777821324 learned_addr learned my addr 172.21.15.42:0/777821324 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:25.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.363+0000 7f9303400640 1 --2- 172.21.15.42:0/777821324 >> v2:172.21.15.63:3300/0 conn(0x7f930c09bbd0 0x7f930c09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:25.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.367+0000 7f930be00640 1 -- 172.21.15.42:0/777821324 >> v2:172.21.15.63:3300/0 conn(0x7f930c09bbd0 msgr2=0x7f930c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:25.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.367+0000 7f930be00640 1 --2- 172.21.15.42:0/777821324 >> v2:172.21.15.63:3300/0 conn(0x7f930c09bbd0 0x7f930c09aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.367+0000 7f930be00640 1 -- 172.21.15.42:0/777821324 >> v1:172.21.15.42:6789/0 conn(0x7f930c0d06a0 legacy=0x7f930c160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:25.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.367+0000 7f930be00640 1 -- 172.21.15.42:0/777821324 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c0cd5e0 con 0x7f930c15c990 2024-05-28T14:14:25.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.367+0000 7f930b400640 1 -- 172.21.15.42:0/777821324 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1866687891 0 0) 0x7f930c0cd300 con 0x7f930c0d06a0 2024-05-28T14:14:25.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.367+0000 7f930be00640 1 --2- 172.21.15.42:0/777821324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f930c15c990 0x7f930c0d0160 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f930c072560 tx=0x7f92fc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482f13a1f4a8e378 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:25.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.367+0000 7f930b400640 1 -- 172.21.15.42:0/777821324 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc0b2070 con 0x7f930c15c990 2024-05-28T14:14:25.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.367+0000 7f930b400640 1 -- 172.21.15.42:0/777821324 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92fc0a9070 con 0x7f930c15c990 2024-05-28T14:14:25.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.367+0000 7f930b400640 1 -- 172.21.15.42:0/777821324 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc0ad880 con 0x7f930c15c990 2024-05-28T14:14:25.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.367+0000 7f9312c00640 1 -- 172.21.15.42:0/777821324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f930c15c990 msgr2=0x7f930c0d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:25.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.367+0000 7f9312c00640 1 --2- 172.21.15.42:0/777821324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f930c15c990 0x7f930c0d0160 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f930c072560 tx=0x7f92fc01c450 comp rx=0 tx=0).stop 2024-05-28T14:14:25.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9312c00640 1 -- 172.21.15.42:0/777821324 shutdown_connections 2024-05-28T14:14:25.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9312c00640 1 --2- 172.21.15.42:0/777821324 >> v2:172.21.15.63:3300/0 conn(0x7f930c09bbd0 0x7f930c09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9312c00640 1 --2- 172.21.15.42:0/777821324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f930c15c990 0x7f930c0d0160 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9312c00640 1 -- 172.21.15.42:0/777821324 >> 172.21.15.42:0/777821324 conn(0x7f930c0b4080 msgr2=0x7f930c0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:25.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9312c00640 1 -- 172.21.15.42:0/777821324 shutdown_connections 2024-05-28T14:14:25.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9312c00640 1 -- 172.21.15.42:0/777821324 wait complete. 2024-05-28T14:14:25.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9312c00640 1 Processor -- start 2024-05-28T14:14:25.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9312c00640 1 -- start start 2024-05-28T14:14:25.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9312c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f930c0d06a0 0x7f930c0c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:25.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9312c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f930c15c990 0x7f930c0cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:25.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9312c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f930c154a90 0x7f930c165820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:25.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9312c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f930c0ce3a0 con 0x7f930c0d06a0 2024-05-28T14:14:25.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9312c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f930c169740 con 0x7f930c154a90 2024-05-28T14:14:25.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9312c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f930c0c6a20 con 0x7f930c15c990 2024-05-28T14:14:25.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f930be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f930c0d06a0 0x7f930c0c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:25.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f930be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f930c0d06a0 0x7f930c0c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48284/0 (socket says 172.21.15.42:48284) 2024-05-28T14:14:25.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f930be00640 1 -- 172.21.15.42:0/710257522 learned_addr learned my addr 172.21.15.42:0/710257522 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:25.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9303400640 1 --2- 172.21.15.42:0/710257522 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f930c15c990 0x7f930c0cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:25.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f9310c00640 1 --2- 172.21.15.42:0/710257522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f930c154a90 0x7f930c165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:25.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f930be00640 1 -- 172.21.15.42:0/710257522 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f930c15c990 msgr2=0x7f930c0cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:25.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f930be00640 1 --2- 172.21.15.42:0/710257522 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f930c15c990 0x7f930c0cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f930be00640 1 -- 172.21.15.42:0/710257522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f930c154a90 msgr2=0x7f930c165820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:25.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f930be00640 1 --2- 172.21.15.42:0/710257522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f930c154a90 0x7f930c165820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f930be00640 1 -- 172.21.15.42:0/710257522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c0c6c80 con 0x7f930c0d06a0 2024-05-28T14:14:25.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.371+0000 7f930be00640 1 --2- 172.21.15.42:0/710257522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f930c0d06a0 0x7f930c0c64e0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0b0040 tx=0x7f92fc01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:25.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.375+0000 7f9308c00640 1 -- 172.21.15.42:0/710257522 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc0b2050 con 0x7f930c0d06a0 2024-05-28T14:14:25.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.375+0000 7f9308c00640 1 -- 172.21.15.42:0/710257522 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92fc0be070 con 0x7f930c0d06a0 2024-05-28T14:14:25.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.375+0000 7f9308c00640 1 -- 172.21.15.42:0/710257522 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc0b9690 con 0x7f930c0d06a0 2024-05-28T14:14:25.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.375+0000 7f9312c00640 1 -- 172.21.15.42:0/710257522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930c0c6f60 con 0x7f930c0d06a0 2024-05-28T14:14:25.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.375+0000 7f9312c00640 1 -- 172.21.15.42:0/710257522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f930c0c7520 con 0x7f930c0d06a0 2024-05-28T14:14:25.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.375+0000 7f9312c00640 1 -- 172.21.15.42:0/710257522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f930c068a90 con 0x7f930c0d06a0 2024-05-28T14:14:25.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.375+0000 7f9308c00640 1 -- 172.21.15.42:0/710257522 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f92fc0ab070 con 0x7f930c0d06a0 2024-05-28T14:14:25.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.375+0000 7f9308c00640 1 --2- 172.21.15.42:0/710257522 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92e8069fa0 0x7f92e806c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:25.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.375+0000 7f9308c00640 1 -- 172.21.15.42:0/710257522 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7f92fc11fd00 con 0x7f930c0d06a0 2024-05-28T14:14:25.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.375+0000 7f9303400640 1 --2- 172.21.15.42:0/710257522 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92e8069fa0 0x7f92e806c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:25.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.379+0000 7f9303400640 1 --2- 172.21.15.42:0/710257522 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92e8069fa0 0x7f92e806c460 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f930c06ec10 tx=0x7f92f4009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:25.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.379+0000 7f9308c00640 1 -- 172.21.15.42:0/710257522 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f92fc016070 con 0x7f930c0d06a0 2024-05-28T14:14:25.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.547+0000 7f9312c00640 1 -- 172.21.15.42:0/710257522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f930c0cca30 con 0x7f930c0d06a0 2024-05-28T14:14:25.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.547+0000 7f9308c00640 1 -- 172.21.15.42:0/710257522 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16809 (secure 0 0 0) 0x7f92fc0f17f0 con 0x7f930c0d06a0 2024-05-28T14:14:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:25.535 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":119,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:14:05.680742+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:12:57.936312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-28T14:12:58.495752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.24","mappings":[{"from":0,"to":5}]},{"pgid":"17.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:14:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.551+0000 7f9312c00640 1 -- 172.21.15.42:0/710257522 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92e8069fa0 msgr2=0x7f92e806c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.551+0000 7f9312c00640 1 --2- 172.21.15.42:0/710257522 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92e8069fa0 0x7f92e806c460 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f930c06ec10 tx=0x7f92f4009190 comp rx=0 tx=0).stop 2024-05-28T14:14:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.551+0000 7f9312c00640 1 -- 172.21.15.42:0/710257522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f930c0d06a0 msgr2=0x7f930c0c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:25.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.551+0000 7f9312c00640 1 --2- 172.21.15.42:0/710257522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f930c0d06a0 0x7f930c0c64e0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0b0040 tx=0x7f92fc01ac90 comp rx=0 tx=0).stop 2024-05-28T14:14:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.551+0000 7f9312c00640 1 -- 172.21.15.42:0/710257522 shutdown_connections 2024-05-28T14:14:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.551+0000 7f9312c00640 1 --2- 172.21.15.42:0/710257522 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f92e8069fa0 0x7f92e806c460 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.551+0000 7f9312c00640 1 --2- 172.21.15.42:0/710257522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f930c154a90 0x7f930c165820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.551+0000 7f9312c00640 1 --2- 172.21.15.42:0/710257522 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f930c15c990 0x7f930c0cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.551+0000 7f9312c00640 1 --2- 172.21.15.42:0/710257522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f930c0d06a0 0x7f930c0c64e0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.551+0000 7f9312c00640 1 -- 172.21.15.42:0/710257522 >> 172.21.15.42:0/710257522 conn(0x7f930c0b4080 msgr2=0x7f930c15bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.551+0000 7f9312c00640 1 -- 172.21.15.42:0/710257522 shutdown_connections 2024-05-28T14:14:25.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.551+0000 7f9312c00640 1 -- 172.21.15.42:0/710257522 wait complete. 2024-05-28T14:14:25.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T14:12:57.936312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T14:12:58.495752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:14:25.557 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:14:25.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.715+0000 7fd86a800640 1 Processor -- start 2024-05-28T14:14:25.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.715+0000 7fd86a800640 1 -- start start 2024-05-28T14:14:25.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.715+0000 7fd86a800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd86415c0c0 0x7fd86406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:25.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.715+0000 7fd86a800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8640afe40 con 0x7fd86406ee20 2024-05-28T14:14:25.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.715+0000 7fd86a800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd86415c600 con 0x7fd864144f80 2024-05-28T14:14:25.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.715+0000 7fd86a800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd86406f7c0 con 0x7fd86415c0c0 2024-05-28T14:14:25.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.715+0000 7fd85aa00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fd86406ee20 0x7fd86415b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53156/0 (socket says 172.21.15.42:53156) 2024-05-28T14:14:25.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.715+0000 7fd85aa00640 1 -- 172.21.15.42:0/2431117495 learned_addr learned my addr 172.21.15.42:0/2431117495 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:25.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.719+0000 7fd863e00640 1 --2- 172.21.15.42:0/2431117495 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd86415c0c0 0x7fd86406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:25.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.719+0000 7fd862a00640 1 -- 172.21.15.42:0/2431117495 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2228551326 0 0) 0x7fd8640afe40 con 0x7fd86406ee20 2024-05-28T14:14:25.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.719+0000 7fd862a00640 1 -- 172.21.15.42:0/2431117495 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd848003530 con 0x7fd86406ee20 2024-05-28T14:14:25.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.719+0000 7fd862a00640 1 -- 172.21.15.42:0/2431117495 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1401375319 0 0) 0x7fd86415c600 con 0x7fd864144f80 2024-05-28T14:14:25.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.719+0000 7fd862a00640 1 -- 172.21.15.42:0/2431117495 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8640afe40 con 0x7fd864144f80 2024-05-28T14:14:25.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.719+0000 7fd863e00640 1 -- 172.21.15.42:0/2431117495 >> v1:172.21.15.63:6789/0 conn(0x7fd864144f80 legacy=0x7fd864072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:25.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.719+0000 7fd863e00640 1 -- 172.21.15.42:0/2431117495 >> v1:172.21.15.42:6789/0 conn(0x7fd86406ee20 legacy=0x7fd86415b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:25.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.719+0000 7fd863e00640 1 -- 172.21.15.42:0/2431117495 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86415c8e0 con 0x7fd86415c0c0 2024-05-28T14:14:25.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.719+0000 7fd863e00640 1 --2- 172.21.15.42:0/2431117495 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd86415c0c0 0x7fd86406f280 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0097f0 tx=0x7fd85c01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b118197105fef082 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:25.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.719+0000 7fd862a00640 1 -- 172.21.15.42:0/2431117495 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0b0070 con 0x7fd86415c0c0 2024-05-28T14:14:25.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.719+0000 7fd862a00640 1 -- 172.21.15.42:0/2431117495 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85c0ab600 con 0x7fd86415c0c0 2024-05-28T14:14:25.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.719+0000 7fd862a00640 1 -- 172.21.15.42:0/2431117495 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0ab9d0 con 0x7fd86415c0c0 2024-05-28T14:14:25.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 -- 172.21.15.42:0/2431117495 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd86415c0c0 msgr2=0x7fd86406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:25.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 --2- 172.21.15.42:0/2431117495 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd86415c0c0 0x7fd86406f280 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0097f0 tx=0x7fd85c01c4e0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 -- 172.21.15.42:0/2431117495 shutdown_connections 2024-05-28T14:14:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 --2- 172.21.15.42:0/2431117495 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd86415c0c0 0x7fd86406f280 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 -- 172.21.15.42:0/2431117495 >> 172.21.15.42:0/2431117495 conn(0x7fd8640b21f0 msgr2=0x7fd8640b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 -- 172.21.15.42:0/2431117495 shutdown_connections 2024-05-28T14:14:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 -- 172.21.15.42:0/2431117495 wait complete. 2024-05-28T14:14:25.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 Processor -- start 2024-05-28T14:14:25.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 -- start start 2024-05-28T14:14:25.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd86406ee20 0x7fd864154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:25.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd864144f80 0x7fd86413bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:25.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd86415c0c0 0x7fd864140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:25.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd8640724d0 con 0x7fd864144f80 2024-05-28T14:14:25.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd8640b42e0 con 0x7fd86415c0c0 2024-05-28T14:14:25.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd86a800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd86406dd10 con 0x7fd86406ee20 2024-05-28T14:14:25.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd85aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd864144f80 0x7fd86413bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd85aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd864144f80 0x7fd86413bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48288/0 (socket says 172.21.15.42:48288) 2024-05-28T14:14:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd85aa00640 1 -- 172.21.15.42:0/942615828 learned_addr learned my addr 172.21.15.42:0/942615828 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd863e00640 1 --2- 172.21.15.42:0/942615828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd86415c0c0 0x7fd864140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd863400640 1 --2- 172.21.15.42:0/942615828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd86406ee20 0x7fd864154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd85aa00640 1 -- 172.21.15.42:0/942615828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd86406ee20 msgr2=0x7fd864154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd85aa00640 1 --2- 172.21.15.42:0/942615828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd86406ee20 0x7fd864154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd85aa00640 1 -- 172.21.15.42:0/942615828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd86415c0c0 msgr2=0x7fd864140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd85aa00640 1 --2- 172.21.15.42:0/942615828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd86415c0c0 0x7fd864140290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.723+0000 7fd85aa00640 1 -- 172.21.15.42:0/942615828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8641553a0 con 0x7fd864144f80 2024-05-28T14:14:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.727+0000 7fd85aa00640 1 --2- 172.21.15.42:0/942615828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd864144f80 0x7fd86413bad0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00c500 tx=0x7fd84c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.727+0000 7fd85be00640 1 -- 172.21.15.42:0/942615828 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c017070 con 0x7fd864144f80 2024-05-28T14:14:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.727+0000 7fd85be00640 1 -- 172.21.15.42:0/942615828 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84c005cf0 con 0x7fd864144f80 2024-05-28T14:14:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.727+0000 7fd85be00640 1 -- 172.21.15.42:0/942615828 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c007330 con 0x7fd864144f80 2024-05-28T14:14:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.727+0000 7fd86a800640 1 -- 172.21.15.42:0/942615828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86413f1f0 con 0x7fd864144f80 2024-05-28T14:14:25.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.727+0000 7fd86a800640 1 -- 172.21.15.42:0/942615828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd86413f7b0 con 0x7fd864144f80 2024-05-28T14:14:25.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.727+0000 7fd86a800640 1 -- 172.21.15.42:0/942615828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8640738b0 con 0x7fd864144f80 2024-05-28T14:14:25.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.727+0000 7fd85be00640 1 -- 172.21.15.42:0/942615828 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7fd84c01a070 con 0x7fd864144f80 2024-05-28T14:14:25.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.727+0000 7fd85be00640 1 --2- 172.21.15.42:0/942615828 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd840069fa0 0x7fd84006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:25.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.727+0000 7fd85be00640 1 -- 172.21.15.42:0/942615828 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7fd84c084d40 con 0x7fd864144f80 2024-05-28T14:14:25.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.727+0000 7fd863400640 1 --2- 172.21.15.42:0/942615828 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd840069fa0 0x7fd84006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:25.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.731+0000 7fd863400640 1 --2- 172.21.15.42:0/942615828 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd840069fa0 0x7fd84006c460 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd854001570 tx=0x7fd854009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:25.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.731+0000 7fd85be00640 1 -- 172.21.15.42:0/942615828 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd84c056710 con 0x7fd864144f80 2024-05-28T14:14:25.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.895+0000 7fd86a800640 1 -- 172.21.15.42:0/942615828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd86406cdc0 con 0x7fd864144f80 2024-05-28T14:14:25.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.895+0000 7fd85be00640 1 -- 172.21.15.42:0/942615828 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7fd84c05aa10 con 0x7fd864144f80 2024-05-28T14:14:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:14:25.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.899+0000 7fd86a800640 1 -- 172.21.15.42:0/942615828 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd840069fa0 msgr2=0x7fd84006c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:25.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.899+0000 7fd86a800640 1 --2- 172.21.15.42:0/942615828 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd840069fa0 0x7fd84006c460 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd854001570 tx=0x7fd854009040 comp rx=0 tx=0).stop 2024-05-28T14:14:25.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.899+0000 7fd86a800640 1 -- 172.21.15.42:0/942615828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd864144f80 msgr2=0x7fd86413bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:25.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.899+0000 7fd86a800640 1 --2- 172.21.15.42:0/942615828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd864144f80 0x7fd86413bad0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00c500 tx=0x7fd84c00c9c0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.899+0000 7fd86a800640 1 -- 172.21.15.42:0/942615828 shutdown_connections 2024-05-28T14:14:25.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.899+0000 7fd86a800640 1 --2- 172.21.15.42:0/942615828 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd840069fa0 0x7fd84006c460 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.899+0000 7fd86a800640 1 --2- 172.21.15.42:0/942615828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd86415c0c0 0x7fd864140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.899+0000 7fd86a800640 1 --2- 172.21.15.42:0/942615828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd864144f80 0x7fd86413bad0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.899+0000 7fd86a800640 1 --2- 172.21.15.42:0/942615828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd86406ee20 0x7fd864154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:25.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.899+0000 7fd86a800640 1 -- 172.21.15.42:0/942615828 >> 172.21.15.42:0/942615828 conn(0x7fd8640b21f0 msgr2=0x7fd86415f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:25.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.899+0000 7fd86a800640 1 -- 172.21.15.42:0/942615828 shutdown_connections 2024-05-28T14:14:25.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:25.899+0000 7fd86a800640 1 -- 172.21.15.42:0/942615828 wait complete. 2024-05-28T14:14:25.905 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:14:26.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.067+0000 7f0ed0e00640 1 Processor -- start 2024-05-28T14:14:26.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.067+0000 7f0ed0e00640 1 -- start start 2024-05-28T14:14:26.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.067+0000 7f0ed0e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0ecc144f80 0x7f0ecc072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:26.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.067+0000 7f0ed0e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ecc0afe40 con 0x7f0ecc06ee20 2024-05-28T14:14:26.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.067+0000 7f0ed0e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ecc15c600 con 0x7f0ecc15c0c0 2024-05-28T14:14:26.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.067+0000 7f0ed0e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0ecc0724d0 con 0x7f0ecc144f80 2024-05-28T14:14:26.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.071+0000 7f0ec1400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0ecc06ee20 0x7f0ecc15b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53168/0 (socket says 172.21.15.42:53168) 2024-05-28T14:14:26.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.071+0000 7f0ec1400640 1 -- 172.21.15.42:0/3742936858 learned_addr learned my addr 172.21.15.42:0/3742936858 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:26.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.071+0000 7f0ec9e00640 1 --2- 172.21.15.42:0/3742936858 >> v2:172.21.15.63:3300/0 conn(0x7f0ecc144f80 0x7f0ecc072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:26.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.071+0000 7f0ec9400640 1 -- 172.21.15.42:0/3742936858 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 839606634 0 0) 0x7f0ecc0afe40 con 0x7f0ecc06ee20 2024-05-28T14:14:26.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.071+0000 7f0ec9400640 1 -- 172.21.15.42:0/3742936858 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0eb0003530 con 0x7f0ecc06ee20 2024-05-28T14:14:26.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.071+0000 7f0ec9e00640 1 -- 172.21.15.42:0/3742936858 >> v1:172.21.15.63:6789/0 conn(0x7f0ecc15c0c0 legacy=0x7f0ecc0b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:26.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.071+0000 7f0ec9e00640 1 -- 172.21.15.42:0/3742936858 >> v1:172.21.15.42:6789/0 conn(0x7f0ecc06ee20 legacy=0x7f0ecc15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:26.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.071+0000 7f0ec9e00640 1 -- 172.21.15.42:0/3742936858 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ecc15c8e0 con 0x7f0ecc144f80 2024-05-28T14:14:26.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.071+0000 7f0ec9e00640 1 --2- 172.21.15.42:0/3742936858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc144f80 0x7f0ecc072960 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0097c0 tx=0x7f0ebc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=115d8408df991911 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:26.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.071+0000 7f0ec9400640 1 -- 172.21.15.42:0/3742936858 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0b0070 con 0x7f0ecc144f80 2024-05-28T14:14:26.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.071+0000 7f0ec9400640 1 -- 172.21.15.42:0/3742936858 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ebc0ab600 con 0x7f0ecc144f80 2024-05-28T14:14:26.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.071+0000 7f0ec9400640 1 -- 172.21.15.42:0/3742936858 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0ab9d0 con 0x7f0ecc144f80 2024-05-28T14:14:26.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 -- 172.21.15.42:0/3742936858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc144f80 msgr2=0x7f0ecc072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:26.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 --2- 172.21.15.42:0/3742936858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc144f80 0x7f0ecc072960 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0097c0 tx=0x7f0ebc01c4a0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 -- 172.21.15.42:0/3742936858 shutdown_connections 2024-05-28T14:14:26.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 --2- 172.21.15.42:0/3742936858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc144f80 0x7f0ecc072960 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 -- 172.21.15.42:0/3742936858 >> 172.21.15.42:0/3742936858 conn(0x7f0ecc0b21f0 msgr2=0x7f0ecc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:26.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 -- 172.21.15.42:0/3742936858 shutdown_connections 2024-05-28T14:14:26.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 -- 172.21.15.42:0/3742936858 wait complete. 2024-05-28T14:14:26.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 Processor -- start 2024-05-28T14:14:26.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 -- start start 2024-05-28T14:14:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ecc06ee20 0x7f0ecc13cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc144f80 0x7f0ecc0ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc15c0c0 0x7f0ecc1434d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc06f790 con 0x7f0ecc144f80 2024-05-28T14:14:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc06f610 con 0x7f0ecc15c0c0 2024-05-28T14:14:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ed0e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0ecc169950 con 0x7f0ecc06ee20 2024-05-28T14:14:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ec1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc144f80 0x7f0ecc0ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ec1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc144f80 0x7f0ecc0ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48304/0 (socket says 172.21.15.42:48304) 2024-05-28T14:14:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ec1400640 1 -- 172.21.15.42:0/1121703827 learned_addr learned my addr 172.21.15.42:0/1121703827 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:26.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0ec9e00640 1 --2- 172.21.15.42:0/1121703827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ecc06ee20 0x7f0ecc13cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.075+0000 7f0eca800640 1 --2- 172.21.15.42:0/1121703827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc15c0c0 0x7f0ecc1434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.079+0000 7f0ec1400640 1 -- 172.21.15.42:0/1121703827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ecc06ee20 msgr2=0x7f0ecc13cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.079+0000 7f0ec1400640 1 --2- 172.21.15.42:0/1121703827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ecc06ee20 0x7f0ecc13cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.079+0000 7f0ec1400640 1 -- 172.21.15.42:0/1121703827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc15c0c0 msgr2=0x7f0ecc1434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.079+0000 7f0ec1400640 1 --2- 172.21.15.42:0/1121703827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc15c0c0 0x7f0ecc1434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.079+0000 7f0ec1400640 1 -- 172.21.15.42:0/1121703827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ecc0a9e60 con 0x7f0ecc144f80 2024-05-28T14:14:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.079+0000 7f0ec1400640 1 --2- 172.21.15.42:0/1121703827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc144f80 0x7f0ecc0ca560 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400c4d0 tx=0x7f0eb400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.079+0000 7f0ec2a00640 1 -- 172.21.15.42:0/1121703827 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4017070 con 0x7f0ecc144f80 2024-05-28T14:14:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.079+0000 7f0ec2a00640 1 -- 172.21.15.42:0/1121703827 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eb4005cf0 con 0x7f0ecc144f80 2024-05-28T14:14:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.079+0000 7f0ec2a00640 1 -- 172.21.15.42:0/1121703827 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4007270 con 0x7f0ecc144f80 2024-05-28T14:14:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.079+0000 7f0ed0e00640 1 -- 172.21.15.42:0/1121703827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ecc0cb040 con 0x7f0ecc144f80 2024-05-28T14:14:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.079+0000 7f0ed0e00640 1 -- 172.21.15.42:0/1121703827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ecc0cb8e0 con 0x7f0ecc144f80 2024-05-28T14:14:26.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.079+0000 7f0ed0e00640 1 -- 172.21.15.42:0/1121703827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ecc0738b0 con 0x7f0ecc144f80 2024-05-28T14:14:26.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.079+0000 7f0ec2a00640 1 -- 172.21.15.42:0/1121703827 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f0eb401a070 con 0x7f0ecc144f80 2024-05-28T14:14:26.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.083+0000 7f0ec2a00640 1 --2- 172.21.15.42:0/1121703827 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ea8069fa0 0x7f0ea806c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:26.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.083+0000 7f0ec2a00640 1 -- 172.21.15.42:0/1121703827 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5879+0+0 (secure 0 0 0) 0x7f0eb4084c80 con 0x7f0ecc144f80 2024-05-28T14:14:26.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.083+0000 7f0ec9e00640 1 --2- 172.21.15.42:0/1121703827 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ea8069fa0 0x7f0ea806c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:26.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.083+0000 7f0ec9e00640 1 --2- 172.21.15.42:0/1121703827 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ea8069fa0 0x7f0ea806c460 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0ad000 tx=0x7f0ebc0a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:26.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.087+0000 7f0ec2a00640 1 -- 172.21.15.42:0/1121703827 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ecc0a9e60 con 0x7f0ecc144f80 2024-05-28T14:14:26.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.287+0000 7f0ed0e00640 1 -- 172.21.15.42:0/1121703827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0ecc140900 con 0x7f0ecc144f80 2024-05-28T14:14:26.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.287+0000 7f0ec2a00640 1 -- 172.21.15.42:0/1121703827 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) ==== 104+0+11 (secure 0 0 0) 0x7f0eb4056650 con 0x7f0ecc144f80 2024-05-28T14:14:26.274 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:14:26.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.295+0000 7f0ed0e00640 1 -- 172.21.15.42:0/1121703827 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ea8069fa0 msgr2=0x7f0ea806c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:26.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.295+0000 7f0ed0e00640 1 --2- 172.21.15.42:0/1121703827 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ea8069fa0 0x7f0ea806c460 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0ad000 tx=0x7f0ebc0a6000 comp rx=0 tx=0).stop 2024-05-28T14:14:26.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.295+0000 7f0ed0e00640 1 -- 172.21.15.42:0/1121703827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc144f80 msgr2=0x7f0ecc0ca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:26.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.295+0000 7f0ed0e00640 1 --2- 172.21.15.42:0/1121703827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc144f80 0x7f0ecc0ca560 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400c4d0 tx=0x7f0eb400c990 comp rx=0 tx=0).stop 2024-05-28T14:14:26.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.295+0000 7f0ed0e00640 1 -- 172.21.15.42:0/1121703827 shutdown_connections 2024-05-28T14:14:26.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.295+0000 7f0ed0e00640 1 --2- 172.21.15.42:0/1121703827 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0ea8069fa0 0x7f0ea806c460 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.295+0000 7f0ed0e00640 1 --2- 172.21.15.42:0/1121703827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ecc15c0c0 0x7f0ecc1434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.295+0000 7f0ed0e00640 1 --2- 172.21.15.42:0/1121703827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ecc144f80 0x7f0ecc0ca560 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.295+0000 7f0ed0e00640 1 --2- 172.21.15.42:0/1121703827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ecc06ee20 0x7f0ecc13cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.295+0000 7f0ed0e00640 1 -- 172.21.15.42:0/1121703827 >> 172.21.15.42:0/1121703827 conn(0x7f0ecc0b21f0 msgr2=0x7f0ecc0b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:26.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.295+0000 7f0ed0e00640 1 -- 172.21.15.42:0/1121703827 shutdown_connections 2024-05-28T14:14:26.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.295+0000 7f0ed0e00640 1 -- 172.21.15.42:0/1121703827 wait complete. 2024-05-28T14:14:26.317 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:14:26.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.571+0000 7faffcc00640 1 Processor -- start 2024-05-28T14:14:26.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.571+0000 7faffcc00640 1 -- start start 2024-05-28T14:14:26.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.571+0000 7faffcc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faff81663b0 0x7faff80d51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:26.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.571+0000 7faffcc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff80d1830 0x7faff80d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:26.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.571+0000 7faffcc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff80cf6b0 con 0x7faff80c7ba0 2024-05-28T14:14:26.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.571+0000 7faffcc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7faff80cff50 con 0x7faff81663b0 2024-05-28T14:14:26.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.571+0000 7faffcc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faff80cf990 con 0x7faff80d1830 2024-05-28T14:14:26.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.575+0000 7faff5e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faff81663b0 0x7faff80d51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:26.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.575+0000 7faff6800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff80d1830 0x7faff80d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:26.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.575+0000 7faff5e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7faff81663b0 0x7faff80d51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48306/0 (socket says 172.21.15.42:48306) 2024-05-28T14:14:26.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.575+0000 7faff5e00640 1 -- 172.21.15.42:0/4013191730 learned_addr learned my addr 172.21.15.42:0/4013191730 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:26.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.575+0000 7faff6800640 1 -- 172.21.15.42:0/4013191730 >> v2:172.21.15.42:3300/0 conn(0x7faff81663b0 msgr2=0x7faff80d51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:26.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.575+0000 7faff6800640 1 --2- 172.21.15.42:0/4013191730 >> v2:172.21.15.42:3300/0 conn(0x7faff81663b0 0x7faff80d51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.575+0000 7faff6800640 1 -- 172.21.15.42:0/4013191730 >> v1:172.21.15.63:6789/0 conn(0x7faff80c7ba0 legacy=0x7faff80d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:26.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.575+0000 7faff6800640 1 -- 172.21.15.42:0/4013191730 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff80cfc70 con 0x7faff80d1830 2024-05-28T14:14:26.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.575+0000 7faff4a00640 1 -- 172.21.15.42:0/4013191730 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1837960599 0 0) 0x7faff80cf6b0 con 0x7faff80c7ba0 2024-05-28T14:14:26.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.579+0000 7faff6800640 1 --2- 172.21.15.42:0/4013191730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff80d1830 0x7faff80d3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faff0009d90 tx=0x7faff001f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=316d18861d165ebb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:26.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.579+0000 7faff4a00640 1 -- 172.21.15.42:0/4013191730 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff00b4070 con 0x7faff80d1830 2024-05-28T14:14:26.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.579+0000 7faff4a00640 1 -- 172.21.15.42:0/4013191730 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff00af600 con 0x7faff80d1830 2024-05-28T14:14:26.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.579+0000 7faff4a00640 1 -- 172.21.15.42:0/4013191730 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff00af9d0 con 0x7faff80d1830 2024-05-28T14:14:26.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.579+0000 7faffcc00640 1 -- 172.21.15.42:0/4013191730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff80d1830 msgr2=0x7faff80d3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:26.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.579+0000 7faffcc00640 1 --2- 172.21.15.42:0/4013191730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff80d1830 0x7faff80d3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faff0009d90 tx=0x7faff001f150 comp rx=0 tx=0).stop 2024-05-28T14:14:26.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.579+0000 7faffcc00640 1 -- 172.21.15.42:0/4013191730 shutdown_connections 2024-05-28T14:14:26.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.579+0000 7faffcc00640 1 --2- 172.21.15.42:0/4013191730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff80d1830 0x7faff80d3690 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.579+0000 7faffcc00640 1 --2- 172.21.15.42:0/4013191730 >> v2:172.21.15.42:3300/0 conn(0x7faff81663b0 0x7faff80d51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.579+0000 7faffcc00640 1 -- 172.21.15.42:0/4013191730 >> 172.21.15.42:0/4013191730 conn(0x7faff808bfe0 msgr2=0x7faff80d4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:26.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.579+0000 7faffcc00640 1 -- 172.21.15.42:0/4013191730 shutdown_connections 2024-05-28T14:14:26.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.579+0000 7faffcc00640 1 -- 172.21.15.42:0/4013191730 wait complete. 2024-05-28T14:14:26.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.583+0000 7faffcc00640 1 Processor -- start 2024-05-28T14:14:26.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.583+0000 7faffcc00640 1 -- start start 2024-05-28T14:14:26.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.583+0000 7faffcc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff80c7ba0 0x7faff8157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:26.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faffcc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff80d1830 0x7faff815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:26.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faffcc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff81663b0 0x7faff80b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:26.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faffcc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faff80b1550 con 0x7faff81663b0 2024-05-28T14:14:26.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faffcc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faff8166790 con 0x7faff80d1830 2024-05-28T14:14:26.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faffcc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faff80b16d0 con 0x7faff80c7ba0 2024-05-28T14:14:26.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faff5e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff80c7ba0 0x7faff8157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:26.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faff5e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff80c7ba0 0x7faff8157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:48276/0 (socket says 172.21.15.42:48276) 2024-05-28T14:14:26.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faff5e00640 1 -- 172.21.15.42:0/2290066198 learned_addr learned my addr 172.21.15.42:0/2290066198 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:26.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faff5400640 1 --2- 172.21.15.42:0/2290066198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff80d1830 0x7faff815d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:26.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faff6800640 1 --2- 172.21.15.42:0/2290066198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff81663b0 0x7faff80b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:26.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faff5e00640 1 -- 172.21.15.42:0/2290066198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff80d1830 msgr2=0x7faff815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:26.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faff5e00640 1 --2- 172.21.15.42:0/2290066198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff80d1830 0x7faff815d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faff5e00640 1 -- 172.21.15.42:0/2290066198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff81663b0 msgr2=0x7faff80b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:26.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faff5e00640 1 --2- 172.21.15.42:0/2290066198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff81663b0 0x7faff80b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faff5e00640 1 -- 172.21.15.42:0/2290066198 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff8157ab0 con 0x7faff80c7ba0 2024-05-28T14:14:26.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faff5e00640 1 --2- 172.21.15.42:0/2290066198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff80c7ba0 0x7faff8157570 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fafe000e8c0 tx=0x7fafe000ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:26.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7fafee000640 1 -- 172.21.15.42:0/2290066198 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0017070 con 0x7faff80c7ba0 2024-05-28T14:14:26.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7fafee000640 1 -- 172.21.15.42:0/2290066198 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe000c940 con 0x7faff80c7ba0 2024-05-28T14:14:26.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faffcc00640 1 -- 172.21.15.42:0/2290066198 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff80ca750 con 0x7faff80c7ba0 2024-05-28T14:14:26.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7fafee000640 1 -- 172.21.15.42:0/2290066198 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe000cd30 con 0x7faff80c7ba0 2024-05-28T14:14:26.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.587+0000 7faffcc00640 1 -- 172.21.15.42:0/2290066198 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faff80c9990 con 0x7faff80c7ba0 2024-05-28T14:14:26.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.591+0000 7faffcc00640 1 -- 172.21.15.42:0/2290066198 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff80a2680 con 0x7faff80c7ba0 2024-05-28T14:14:26.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.591+0000 7fafee000640 1 -- 172.21.15.42:0/2290066198 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7fafe001a070 con 0x7faff80c7ba0 2024-05-28T14:14:26.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.595+0000 7fafee000640 1 --2- 172.21.15.42:0/2290066198 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fafd4069fa0 0x7fafd406c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:26.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.595+0000 7fafee000640 1 -- 172.21.15.42:0/2290066198 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(120..120 src has 1..120) ==== 5879+0+0 (secure 0 0 0) 0x7fafe0085780 con 0x7faff80c7ba0 2024-05-28T14:14:26.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.595+0000 7faff5400640 1 --2- 172.21.15.42:0/2290066198 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fafd4069fa0 0x7fafd406c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:26.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.595+0000 7faff5400640 1 --2- 172.21.15.42:0/2290066198 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fafd4069fa0 0x7fafd406c460 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7faff80b39e0 tx=0x7fafe8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:26.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.603+0000 7fafee000640 1 -- 172.21.15.42:0/2290066198 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faff8157ab0 con 0x7faff80c7ba0 2024-05-28T14:14:26.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.847+0000 7faffcc00640 1 -- 172.21.15.42:0/2290066198 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faff80d1510 con 0x7faff80c7ba0 2024-05-28T14:14:26.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.851+0000 7fafee000640 1 -- 172.21.15.42:0/2290066198 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) ==== 100+0+11 (secure 0 0 0) 0x7fafe00571f0 con 0x7faff80c7ba0 2024-05-28T14:14:26.835 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:14:26.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.851+0000 7faffcc00640 1 -- 172.21.15.42:0/2290066198 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fafd4069fa0 msgr2=0x7fafd406c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:26.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.851+0000 7faffcc00640 1 --2- 172.21.15.42:0/2290066198 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fafd4069fa0 0x7fafd406c460 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7faff80b39e0 tx=0x7fafe8009040 comp rx=0 tx=0).stop 2024-05-28T14:14:26.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.851+0000 7faffcc00640 1 -- 172.21.15.42:0/2290066198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff80c7ba0 msgr2=0x7faff8157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:26.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.851+0000 7faffcc00640 1 --2- 172.21.15.42:0/2290066198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff80c7ba0 0x7faff8157570 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fafe000e8c0 tx=0x7fafe000ed80 comp rx=0 tx=0).stop 2024-05-28T14:14:26.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.851+0000 7faffcc00640 1 -- 172.21.15.42:0/2290066198 shutdown_connections 2024-05-28T14:14:26.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.851+0000 7faffcc00640 1 --2- 172.21.15.42:0/2290066198 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fafd4069fa0 0x7fafd406c460 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.851+0000 7faffcc00640 1 --2- 172.21.15.42:0/2290066198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faff81663b0 0x7faff80b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.851+0000 7faffcc00640 1 --2- 172.21.15.42:0/2290066198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faff80d1830 0x7faff815d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.851+0000 7faffcc00640 1 --2- 172.21.15.42:0/2290066198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faff80c7ba0 0x7faff8157570 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:26.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.851+0000 7faffcc00640 1 -- 172.21.15.42:0/2290066198 >> 172.21.15.42:0/2290066198 conn(0x7faff808bfe0 msgr2=0x7faff80b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:26.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.851+0000 7faffcc00640 1 -- 172.21.15.42:0/2290066198 shutdown_connections 2024-05-28T14:14:26.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:26.851+0000 7faffcc00640 1 -- 172.21.15.42:0/2290066198 wait complete. 2024-05-28T14:14:26.858 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T14:14:27.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.015+0000 7f787cc00640 1 Processor -- start 2024-05-28T14:14:27.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.015+0000 7f787cc00640 1 -- start start 2024-05-28T14:14:27.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.015+0000 7f787cc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f78780cf230 0x7f7878072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:27.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.015+0000 7f787cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878072ba0 0x7f787806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:27.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.015+0000 7f787cc00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78780cfc10 con 0x7f787815bf30 2024-05-28T14:14:27.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.015+0000 7f787cc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f78780721d0 con 0x7f78780cf230 2024-05-28T14:14:27.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.015+0000 7f787cc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f78780cfef0 con 0x7f7878072ba0 2024-05-28T14:14:27.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.019+0000 7f786fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f78780cf230 0x7f7878072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:27.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.019+0000 7f786fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f78780cf230 0x7f7878072660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48324/0 (socket says 172.21.15.42:48324) 2024-05-28T14:14:27.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.019+0000 7f786fe00640 1 -- 172.21.15.42:0/1793517335 learned_addr learned my addr 172.21.15.42:0/1793517335 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:27.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.019+0000 7f7876800640 1 --2- 172.21.15.42:0/1793517335 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878072ba0 0x7f787806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:27.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.019+0000 7f786fe00640 1 -- 172.21.15.42:0/1793517335 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878072ba0 msgr2=0x7f787806ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:27.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.019+0000 7f786fe00640 1 --2- 172.21.15.42:0/1793517335 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878072ba0 0x7f787806ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.019+0000 7f786fe00640 1 -- 172.21.15.42:0/1793517335 >> v1:172.21.15.63:6789/0 conn(0x7f787815bf30 legacy=0x7f787815c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:27.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.019+0000 7f786fe00640 1 -- 172.21.15.42:0/1793517335 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78780d01d0 con 0x7f78780cf230 2024-05-28T14:14:27.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.019+0000 7f786fe00640 1 --2- 172.21.15.42:0/1793517335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78780cf230 0x7f7878072660 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f7860009870 tx=0x7f786001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919f1673007bb728 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:27.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.019+0000 7f7875400640 1 -- 172.21.15.42:0/1793517335 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78600a7070 con 0x7f78780cf230 2024-05-28T14:14:27.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.019+0000 7f7875400640 1 -- 172.21.15.42:0/1793517335 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78600b0070 con 0x7f78780cf230 2024-05-28T14:14:27.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.019+0000 7f7875400640 1 -- 172.21.15.42:0/1793517335 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78600ab890 con 0x7f78780cf230 2024-05-28T14:14:27.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 -- 172.21.15.42:0/1793517335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78780cf230 msgr2=0x7f7878072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:27.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 --2- 172.21.15.42:0/1793517335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78780cf230 0x7f7878072660 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f7860009870 tx=0x7f786001c560 comp rx=0 tx=0).stop 2024-05-28T14:14:27.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 -- 172.21.15.42:0/1793517335 shutdown_connections 2024-05-28T14:14:27.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 --2- 172.21.15.42:0/1793517335 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878072ba0 0x7f787806ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 --2- 172.21.15.42:0/1793517335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78780cf230 0x7f7878072660 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 -- 172.21.15.42:0/1793517335 >> 172.21.15.42:0/1793517335 conn(0x7f787806b9f0 msgr2=0x7f787806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:27.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 -- 172.21.15.42:0/1793517335 shutdown_connections 2024-05-28T14:14:27.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 -- 172.21.15.42:0/1793517335 wait complete. 2024-05-28T14:14:27.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 Processor -- start 2024-05-28T14:14:27.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 -- start start 2024-05-28T14:14:27.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7878072ba0 0x7f78780b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:27.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78780cf230 0x7f78781514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:27.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f787815bf30 0x7f78781348b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:27.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7878168540 con 0x7f7878072ba0 2024-05-28T14:14:27.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78781683c0 con 0x7f787815bf30 2024-05-28T14:14:27.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f787cc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f78781686c0 con 0x7f78780cf230 2024-05-28T14:14:27.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f7875e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7878072ba0 0x7f78780b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:27.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f7875e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7878072ba0 0x7f78780b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48328/0 (socket says 172.21.15.42:48328) 2024-05-28T14:14:27.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f7875e00640 1 -- 172.21.15.42:0/2617780694 learned_addr learned my addr 172.21.15.42:0/2617780694 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:27.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f786fe00640 1 --2- 172.21.15.42:0/2617780694 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78780cf230 0x7f78781514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:27.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f7876800640 1 --2- 172.21.15.42:0/2617780694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f787815bf30 0x7f78781348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:27.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f7875e00640 1 -- 172.21.15.42:0/2617780694 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78780cf230 msgr2=0x7f78781514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:27.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f7875e00640 1 --2- 172.21.15.42:0/2617780694 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78780cf230 0x7f78781514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f7875e00640 1 -- 172.21.15.42:0/2617780694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f787815bf30 msgr2=0x7f78781348b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:27.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f7875e00640 1 --2- 172.21.15.42:0/2617780694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f787815bf30 0x7f78781348b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f7875e00640 1 -- 172.21.15.42:0/2617780694 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78780b1c60 con 0x7f7878072ba0 2024-05-28T14:14:27.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.023+0000 7f7875e00640 1 --2- 172.21.15.42:0/2617780694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7878072ba0 0x7f78780b2cc0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f786800c370 tx=0x7f786800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:27.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.027+0000 7f786e000640 1 -- 172.21.15.42:0/2617780694 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78680052d0 con 0x7f7878072ba0 2024-05-28T14:14:27.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.027+0000 7f786e000640 1 -- 172.21.15.42:0/2617780694 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7868005470 con 0x7f7878072ba0 2024-05-28T14:14:27.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.027+0000 7f786e000640 1 -- 172.21.15.42:0/2617780694 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868007260 con 0x7f7878072ba0 2024-05-28T14:14:27.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.027+0000 7f787cc00640 1 -- 172.21.15.42:0/2617780694 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78780b1f00 con 0x7f7878072ba0 2024-05-28T14:14:27.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.027+0000 7f787cc00640 1 -- 172.21.15.42:0/2617780694 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78780b24c0 con 0x7f7878072ba0 2024-05-28T14:14:27.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.027+0000 7f787cc00640 1 -- 172.21.15.42:0/2617780694 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7878073890 con 0x7f7878072ba0 2024-05-28T14:14:27.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.027+0000 7f786e000640 1 -- 172.21.15.42:0/2617780694 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f7868007400 con 0x7f7878072ba0 2024-05-28T14:14:27.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.027+0000 7f786e000640 1 --2- 172.21.15.42:0/2617780694 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7854069fa0 0x7f785406c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:27.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.027+0000 7f786e000640 1 -- 172.21.15.42:0/2617780694 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5879+0+0 (secure 0 0 0) 0x7f7868084ca0 con 0x7f7878072ba0 2024-05-28T14:14:27.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.027+0000 7f786fe00640 1 --2- 172.21.15.42:0/2617780694 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7854069fa0 0x7f785406c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:27.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.031+0000 7f786fe00640 1 --2- 172.21.15.42:0/2617780694 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7854069fa0 0x7f785406c460 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f78600af000 tx=0x7f78600a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:27.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.031+0000 7f786e000640 1 -- 172.21.15.42:0/2617780694 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7868056710 con 0x7f7878072ba0 2024-05-28T14:14:27.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.211+0000 7f787cc00640 1 -- 172.21.15.42:0/2617780694 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f787806ea30 con 0x7f7878072ba0 2024-05-28T14:14:27.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.211+0000 7f786e000640 1 -- 172.21.15.42:0/2617780694 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v120) ==== 80+0+0 (secure 0 0 0) 0x7f786805aa10 con 0x7f7878072ba0 2024-05-28T14:14:27.199 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T14:14:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.215+0000 7f787cc00640 1 -- 172.21.15.42:0/2617780694 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7854069fa0 msgr2=0x7f785406c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.215+0000 7f787cc00640 1 --2- 172.21.15.42:0/2617780694 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7854069fa0 0x7f785406c460 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f78600af000 tx=0x7f78600a7000 comp rx=0 tx=0).stop 2024-05-28T14:14:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.215+0000 7f787cc00640 1 -- 172.21.15.42:0/2617780694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7878072ba0 msgr2=0x7f78780b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.215+0000 7f787cc00640 1 --2- 172.21.15.42:0/2617780694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7878072ba0 0x7f78780b2cc0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f786800c370 tx=0x7f786800c830 comp rx=0 tx=0).stop 2024-05-28T14:14:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.215+0000 7f787cc00640 1 -- 172.21.15.42:0/2617780694 shutdown_connections 2024-05-28T14:14:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.215+0000 7f787cc00640 1 --2- 172.21.15.42:0/2617780694 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7854069fa0 0x7f785406c460 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.215+0000 7f787cc00640 1 --2- 172.21.15.42:0/2617780694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f787815bf30 0x7f78781348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.215+0000 7f787cc00640 1 --2- 172.21.15.42:0/2617780694 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78780cf230 0x7f78781514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.215+0000 7f787cc00640 1 --2- 172.21.15.42:0/2617780694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7878072ba0 0x7f78780b2cc0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.215+0000 7f787cc00640 1 -- 172.21.15.42:0/2617780694 >> 172.21.15.42:0/2617780694 conn(0x7f787806b9f0 msgr2=0x7f787815c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:27.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.215+0000 7f787cc00640 1 -- 172.21.15.42:0/2617780694 shutdown_connections 2024-05-28T14:14:27.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.215+0000 7f787cc00640 1 -- 172.21.15.42:0/2617780694 wait complete. 2024-05-28T14:14:27.232 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-05-28T14:14:27.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:27.233 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-28T14:14:27.274 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:14:27.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:27.275 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-28T14:14:27.281 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[52714]: fuse finished with error 0 and tester_r 0 2024-05-28T14:14:27.419 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:14:27.419 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:14:27.419 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:14:27.433 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:14:27.433 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:14:27.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:27.433 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-28T14:14:27.481 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-28T14:14:27.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:14:27.482 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:14:27.482 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:14:27.483 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:14:27.491 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:14:27.492 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:14:27.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:27.492 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-28T14:14:27.546 INFO:teuthology.orchestra.run.smithi042.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-28T14:14:27.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:14:27.547 INFO:tasks.cephfs.mount:Terminating background process 2024-05-28T14:14:27.547 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:14:27.548 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:14:27.556 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:14:27.556 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:14:27.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:27.556 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-28T14:14:27.604 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-28T14:14:27.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:14:27.606 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:14:27.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.759+0000 7f82e3200640 1 Processor -- start 2024-05-28T14:14:27.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.759+0000 7f82e3200640 1 -- start start 2024-05-28T14:14:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.759+0000 7f82e3200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f82e415b7d0 0x7f82e415bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.759+0000 7f82e3200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f82e40b4e00 0x7f82e40b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.763+0000 7f82e3200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e415c290 con 0x7f82e4155810 2024-05-28T14:14:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.763+0000 7f82e3200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f82e40b57b0 con 0x7f82e415b7d0 2024-05-28T14:14:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.763+0000 7f82e3200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f82e40cd840 con 0x7f82e40b4e00 2024-05-28T14:14:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.763+0000 7f82dbe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f82e415b7d0 0x7f82e415bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.763+0000 7f82e2800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f82e40b4e00 0x7f82e40b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:27.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.763+0000 7f82e1e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f82e4155810 0x7f82e40cdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:38232/0 (socket says 172.21.15.42:38232) 2024-05-28T14:14:27.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.763+0000 7f82e1e00640 1 -- 172.21.15.42:0/3386654826 learned_addr learned my addr 172.21.15.42:0/3386654826 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:27.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.763+0000 7f82e1400640 1 -- 172.21.15.42:0/3386654826 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 170372412 0 0) 0x7f82e415c290 con 0x7f82e4155810 2024-05-28T14:14:27.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.763+0000 7f82e2800640 1 -- 172.21.15.42:0/3386654826 >> v2:172.21.15.42:3300/0 conn(0x7f82e415b7d0 msgr2=0x7f82e415bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:27.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.763+0000 7f82e2800640 1 --2- 172.21.15.42:0/3386654826 >> v2:172.21.15.42:3300/0 conn(0x7f82e415b7d0 0x7f82e415bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.763+0000 7f82e2800640 1 -- 172.21.15.42:0/3386654826 >> v1:172.21.15.63:6789/0 conn(0x7f82e4155810 legacy=0x7f82e40cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:27.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.763+0000 7f82e2800640 1 -- 172.21.15.42:0/3386654826 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e415c570 con 0x7f82e40b4e00 2024-05-28T14:14:27.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.763+0000 7f82dbe00640 1 --2- 172.21.15.42:0/3386654826 >> v2:172.21.15.42:3300/0 conn(0x7f82e415b7d0 0x7f82e415bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:14:27.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.763+0000 7f82e2800640 1 --2- 172.21.15.42:0/3386654826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e40b4e00 0x7f82e40b5270 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f82e415ed00 tx=0x7f82dc01eb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da6ef1d169d38b9b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:27.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e1400640 1 -- 172.21.15.42:0/3386654826 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0b4070 con 0x7f82e40b4e00 2024-05-28T14:14:27.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e1400640 1 -- 172.21.15.42:0/3386654826 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82dc0a7b20 con 0x7f82e40b4e00 2024-05-28T14:14:27.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e1400640 1 -- 172.21.15.42:0/3386654826 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0af440 con 0x7f82e40b4e00 2024-05-28T14:14:27.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 -- 172.21.15.42:0/3386654826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e40b4e00 msgr2=0x7f82e40b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:27.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 --2- 172.21.15.42:0/3386654826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e40b4e00 0x7f82e40b5270 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f82e415ed00 tx=0x7f82dc01eb40 comp rx=0 tx=0).stop 2024-05-28T14:14:27.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 -- 172.21.15.42:0/3386654826 shutdown_connections 2024-05-28T14:14:27.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 --2- 172.21.15.42:0/3386654826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e40b4e00 0x7f82e40b5270 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 --2- 172.21.15.42:0/3386654826 >> v2:172.21.15.42:3300/0 conn(0x7f82e415b7d0 0x7f82e415bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 -- 172.21.15.42:0/3386654826 >> 172.21.15.42:0/3386654826 conn(0x7f82e4073310 msgr2=0x7f82e406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:27.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 -- 172.21.15.42:0/3386654826 shutdown_connections 2024-05-28T14:14:27.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 -- 172.21.15.42:0/3386654826 wait complete. 2024-05-28T14:14:27.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 Processor -- start 2024-05-28T14:14:27.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 -- start start 2024-05-28T14:14:27.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e40b4e00 0x7f82e40aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:27.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82e4155810 0x7f82e415a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82e415b7d0 0x7f82e415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f82e40d0140 con 0x7f82e4155810 2024-05-28T14:14:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82e40cffc0 con 0x7f82e40b4e00 2024-05-28T14:14:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e3200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82e40d02c0 con 0x7f82e415b7d0 2024-05-28T14:14:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82dbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82e4155810 0x7f82e415a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82dbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82e4155810 0x7f82e415a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49704/0 (socket says 172.21.15.42:49704) 2024-05-28T14:14:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82dbe00640 1 -- 172.21.15.42:0/1557144864 learned_addr learned my addr 172.21.15.42:0/1557144864 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e1e00640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e40b4e00 0x7f82e40aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.767+0000 7f82e2800640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82e415b7d0 0x7f82e415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82dbe00640 1 -- 172.21.15.42:0/1557144864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82e415b7d0 msgr2=0x7f82e415a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82dbe00640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82e415b7d0 0x7f82e415a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82dbe00640 1 -- 172.21.15.42:0/1557144864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e40b4e00 msgr2=0x7f82e40aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82dbe00640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e40b4e00 0x7f82e40aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82dbe00640 1 -- 172.21.15.42:0/1557144864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e4153bd0 con 0x7f82e4155810 2024-05-28T14:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82e2800640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82e415b7d0 0x7f82e415a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82e1e00640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e40b4e00 0x7f82e40aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82dbe00640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82e4155810 0x7f82e415a2a0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00ebe0 tx=0x7f82cc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82da000640 1 -- 172.21.15.42:0/1557144864 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc017070 con 0x7f82e4155810 2024-05-28T14:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82da000640 1 -- 172.21.15.42:0/1557144864 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82cc00cc30 con 0x7f82e4155810 2024-05-28T14:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82e3200640 1 -- 172.21.15.42:0/1557144864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e4153eb0 con 0x7f82e4155810 2024-05-28T14:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82da000640 1 -- 172.21.15.42:0/1557144864 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc013470 con 0x7f82e4155810 2024-05-28T14:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82e3200640 1 -- 172.21.15.42:0/1557144864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e4154470 con 0x7f82e4155810 2024-05-28T14:14:27.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82e3200640 1 -- 172.21.15.42:0/1557144864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e404fcf0 con 0x7f82e4155810 2024-05-28T14:14:27.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82da000640 1 -- 172.21.15.42:0/1557144864 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f82cc01a070 con 0x7f82e4155810 2024-05-28T14:14:27.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82da000640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f82c0069fa0 0x7f82c006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:27.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.771+0000 7f82da000640 1 -- 172.21.15.42:0/1557144864 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5879+0+0 (secure 0 0 0) 0x7f82cc085c30 con 0x7f82e4155810 2024-05-28T14:14:27.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.775+0000 7f82e1e00640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f82c0069fa0 0x7f82c006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:27.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.775+0000 7f82e1e00640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f82c0069fa0 0x7f82c006c460 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f82d4000f70 tx=0x7f82d4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:27.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.779+0000 7f82da000640 1 -- 172.21.15.42:0/1557144864 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f82cc057c50 con 0x7f82e4155810 2024-05-28T14:14:27.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.979+0000 7f82e3200640 1 -- 172.21.15.42:0/1557144864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82e40b48d0 con 0x7f82e4155810 2024-05-28T14:14:27.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.979+0000 7f82da000640 1 -- 172.21.15.42:0/1557144864 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5180 (secure 0 0 0) 0x7f82cc057550 con 0x7f82e4155810 2024-05-28T14:14:27.964 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:27.964 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/56330151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":56330151},{"type":"v1","addr":"172.21.15.42:6835","nonce":56330151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":9183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/192920662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":192920662},{"type":"v1","addr":"172.21.15.63:6835","nonce":192920662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":57437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1225711200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1225711200},{"type":"v1","addr":"172.21.15.42:6837","nonce":1225711200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:12:59.544555+0000","modified":"2024-05-28T14:13:02.565331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57440":{"gid":57440,"name":"b","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/3959626682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3959626682},{"type":"v1","addr":"172.21.15.63:6834","nonce":3959626682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57440,"qdb_cluster":[57440]},"id":8}]} 2024-05-28T14:14:27.964 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 75 2024-05-28T14:14:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.979+0000 7f82e3200640 1 -- 172.21.15.42:0/1557144864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f82c0069fa0 msgr2=0x7f82c006c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.979+0000 7f82e3200640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f82c0069fa0 0x7f82c006c460 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f82d4000f70 tx=0x7f82d4009040 comp rx=0 tx=0).stop 2024-05-28T14:14:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.979+0000 7f82e3200640 1 -- 172.21.15.42:0/1557144864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82e4155810 msgr2=0x7f82e415a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.979+0000 7f82e3200640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82e4155810 0x7f82e415a2a0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00ebe0 tx=0x7f82cc00c520 comp rx=0 tx=0).stop 2024-05-28T14:14:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.979+0000 7f82e3200640 1 -- 172.21.15.42:0/1557144864 shutdown_connections 2024-05-28T14:14:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.979+0000 7f82e3200640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f82c0069fa0 0x7f82c006c460 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.979+0000 7f82e3200640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82e415b7d0 0x7f82e415a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.979+0000 7f82e3200640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82e4155810 0x7f82e415a2a0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.979+0000 7f82e3200640 1 --2- 172.21.15.42:0/1557144864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e40b4e00 0x7f82e40aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.979+0000 7f82e3200640 1 -- 172.21.15.42:0/1557144864 >> 172.21.15.42:0/1557144864 conn(0x7f82e4073310 msgr2=0x7f82e40b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:27.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.979+0000 7f82e3200640 1 -- 172.21.15.42:0/1557144864 shutdown_connections 2024-05-28T14:14:27.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:27.979+0000 7f82e3200640 1 -- 172.21.15.42:0/1557144864 wait complete. 2024-05-28T14:14:27.986 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:14:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcfa00640 1 Processor -- start 2024-05-28T14:14:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcfa00640 1 -- start start 2024-05-28T14:14:28.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcfa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcbc815c990 0x7fcbc80d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcfa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbc809bbd0 0x7fcbc809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcfa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcbc80d06a0 0x7fcbc809af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcfa00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fcbc815cd70 con 0x7fcbc815c990 2024-05-28T14:14:28.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcfa00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcbc80cd300 con 0x7fcbc80d06a0 2024-05-28T14:14:28.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcfa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc80cd5e0 con 0x7fcbc809bbd0 2024-05-28T14:14:28.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcd000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcbc815c990 0x7fcbc80d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcd000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcbc815c990 0x7fcbc80d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49720/0 (socket says 172.21.15.42:49720) 2024-05-28T14:14:28.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbc7e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbc809bbd0 0x7fcbc809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcd000640 1 -- 172.21.15.42:0/1321746254 learned_addr learned my addr 172.21.15.42:0/1321746254 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:28.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcda00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcbc80d06a0 0x7fcbc809af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcd000640 1 -- 172.21.15.42:0/1321746254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbc809bbd0 msgr2=0x7fcbc809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:28.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcd000640 1 --2- 172.21.15.42:0/1321746254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbc809bbd0 0x7fcbc809aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcd000640 1 -- 172.21.15.42:0/1321746254 >> v2:172.21.15.63:3300/0 conn(0x7fcbc80d06a0 msgr2=0x7fcbc809af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:28.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcd000640 1 --2- 172.21.15.42:0/1321746254 >> v2:172.21.15.63:3300/0 conn(0x7fcbc80d06a0 0x7fcbc809af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbcd000640 1 -- 172.21.15.42:0/1321746254 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc80cd8c0 con 0x7fcbc815c990 2024-05-28T14:14:28.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.143+0000 7fcbc7e00640 1 --2- 172.21.15.42:0/1321746254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbc809bbd0 0x7fcbc809aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:14:28.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.147+0000 7fcbcd000640 1 --2- 172.21.15.42:0/1321746254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcbc815c990 0x7fcbc80d0160 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8072660 tx=0x7fcbbc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bfe46ba75ed5bc server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:28.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.147+0000 7fcbc7400640 1 -- 172.21.15.42:0/1321746254 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc0aa610 con 0x7fcbc815c990 2024-05-28T14:14:28.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.147+0000 7fcbc7400640 1 -- 172.21.15.42:0/1321746254 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbbc0af070 con 0x7fcbc815c990 2024-05-28T14:14:28.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.147+0000 7fcbc7400640 1 -- 172.21.15.42:0/1321746254 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc0aa9d0 con 0x7fcbc815c990 2024-05-28T14:14:28.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.147+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1321746254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcbc815c990 msgr2=0x7fcbc80d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.147+0000 7fcbcfa00640 1 --2- 172.21.15.42:0/1321746254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcbc815c990 0x7fcbc80d0160 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8072660 tx=0x7fcbbc01c4f0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.147+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1321746254 shutdown_connections 2024-05-28T14:14:28.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.147+0000 7fcbcfa00640 1 --2- 172.21.15.42:0/1321746254 >> v2:172.21.15.63:3300/0 conn(0x7fcbc80d06a0 0x7fcbc809af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.147+0000 7fcbcfa00640 1 --2- 172.21.15.42:0/1321746254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbc809bbd0 0x7fcbc809aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.147+0000 7fcbcfa00640 1 --2- 172.21.15.42:0/1321746254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcbc815c990 0x7fcbc80d0160 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.147+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1321746254 >> 172.21.15.42:0/1321746254 conn(0x7fcbc80b4080 msgr2=0x7fcbc80b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:28.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.147+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1321746254 shutdown_connections 2024-05-28T14:14:28.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.147+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1321746254 wait complete. 2024-05-28T14:14:28.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcfa00640 1 Processor -- start 2024-05-28T14:14:28.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcfa00640 1 -- start start 2024-05-28T14:14:28.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcfa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcbc80d06a0 0x7fcbc806d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcfa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbc815c990 0x7fcbc81542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcfa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbc81547e0 0x7fcbc80b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcfa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8160d90 con 0x7fcbc80d06a0 2024-05-28T14:14:28.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcfa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8160c10 con 0x7fcbc81547e0 2024-05-28T14:14:28.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcfa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc8160f10 con 0x7fcbc815c990 2024-05-28T14:14:28.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcd000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcbc80d06a0 0x7fcbc806d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcd000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcbc80d06a0 0x7fcbc806d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49730/0 (socket says 172.21.15.42:49730) 2024-05-28T14:14:28.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcd000640 1 -- 172.21.15.42:0/1197640910 learned_addr learned my addr 172.21.15.42:0/1197640910 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:28.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbc7e00640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbc815c990 0x7fcbc81542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcda00640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbc81547e0 0x7fcbc80b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcd000640 1 -- 172.21.15.42:0/1197640910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbc815c990 msgr2=0x7fcbc81542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcd000640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbc815c990 0x7fcbc81542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcd000640 1 -- 172.21.15.42:0/1197640910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbc81547e0 msgr2=0x7fcbc80b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcd000640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbc81547e0 0x7fcbc80b66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcd000640 1 -- 172.21.15.42:0/1197640910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc8160070 con 0x7fcbc80d06a0 2024-05-28T14:14:28.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbc7e00640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbc815c990 0x7fcbc81542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:28.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcd000640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcbc80d06a0 0x7fcbc806d860 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc01aba0 tx=0x7fcbbc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:28.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbc4c00640 1 -- 172.21.15.42:0/1197640910 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc0bd3f0 con 0x7fcbc80d06a0 2024-05-28T14:14:28.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbc4c00640 1 -- 172.21.15.42:0/1197640910 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbbc0bd590 con 0x7fcbc80d06a0 2024-05-28T14:14:28.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbc4c00640 1 -- 172.21.15.42:0/1197640910 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc0bd980 con 0x7fcbc80d06a0 2024-05-28T14:14:28.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1197640910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc80b6480 con 0x7fcbc80d06a0 2024-05-28T14:14:28.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.151+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1197640910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc8166140 con 0x7fcbc80d06a0 2024-05-28T14:14:28.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.155+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1197640910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc8068a90 con 0x7fcbc80d06a0 2024-05-28T14:14:28.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.155+0000 7fcbc4c00640 1 -- 172.21.15.42:0/1197640910 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7fcbbc0b33f0 con 0x7fcbc80d06a0 2024-05-28T14:14:28.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.155+0000 7fcbc4c00640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcba8069fa0 0x7fcba806c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.155+0000 7fcbc4c00640 1 -- 172.21.15.42:0/1197640910 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5879+0+0 (secure 0 0 0) 0x7fcbbc11f970 con 0x7fcbc80d06a0 2024-05-28T14:14:28.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.155+0000 7fcbc7e00640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcba8069fa0 0x7fcba806c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.155+0000 7fcbc7e00640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcba8069fa0 0x7fcba806c460 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8154f00 tx=0x7fcbb4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:28.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.159+0000 7fcbc4c00640 1 -- 172.21.15.42:0/1197640910 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcbbc0a7070 con 0x7fcbc80d06a0 2024-05-28T14:14:28.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.327+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1197640910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcbc80b4ce0 con 0x7fcbc80d06a0 2024-05-28T14:14:28.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.331+0000 7fcbc4c00640 1 -- 172.21.15.42:0/1197640910 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+16921 (secure 0 0 0) 0x7fcbbc016110 con 0x7fcbc80d06a0 2024-05-28T14:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":120,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:14:26.034186+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:12:57.936312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-28T14:12:58.495752+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":120,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.24","mappings":[{"from":0,"to":5}]},{"pgid":"17.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:14:28.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.331+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1197640910 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcba8069fa0 msgr2=0x7fcba806c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.331+0000 7fcbcfa00640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcba8069fa0 0x7fcba806c460 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8154f00 tx=0x7fcbb4008040 comp rx=0 tx=0).stop 2024-05-28T14:14:28.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.331+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1197640910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcbc80d06a0 msgr2=0x7fcbc806d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.331+0000 7fcbcfa00640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcbc80d06a0 0x7fcbc806d860 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc01aba0 tx=0x7fcbbc01ad30 comp rx=0 tx=0).stop 2024-05-28T14:14:28.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.331+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1197640910 shutdown_connections 2024-05-28T14:14:28.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.331+0000 7fcbcfa00640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcba8069fa0 0x7fcba806c460 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.331+0000 7fcbcfa00640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbc81547e0 0x7fcbc80b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.331+0000 7fcbcfa00640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbc815c990 0x7fcbc81542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.331+0000 7fcbcfa00640 1 --2- 172.21.15.42:0/1197640910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcbc80d06a0 0x7fcbc806d860 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.331+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1197640910 >> 172.21.15.42:0/1197640910 conn(0x7fcbc80b4080 msgr2=0x7fcbc815ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:28.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.331+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1197640910 shutdown_connections 2024-05-28T14:14:28.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.331+0000 7fcbcfa00640 1 -- 172.21.15.42:0/1197640910 wait complete. 2024-05-28T14:14:28.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T14:12:57.936312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T14:12:58.495752+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 120, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:14:28.339 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:14:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.499+0000 7ffa88800640 1 Processor -- start 2024-05-28T14:14:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.499+0000 7ffa88800640 1 -- start start 2024-05-28T14:14:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.499+0000 7ffa88800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffa800ce3b0 0x7ffa800ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.499+0000 7ffa88800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa80160230 con 0x7ffa80071de0 2024-05-28T14:14:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.499+0000 7ffa88800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa80160510 con 0x7ffa8013fd10 2024-05-28T14:14:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.499+0000 7ffa88800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ffa801607f0 con 0x7ffa800ce3b0 2024-05-28T14:14:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.499+0000 7ffa85400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffa800ce3b0 0x7ffa800ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.499+0000 7ffa85e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ffa80071de0 0x7ffa800721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36726/0 (socket says 172.21.15.42:36726) 2024-05-28T14:14:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.499+0000 7ffa85e00640 1 -- 172.21.15.42:0/2087435127 learned_addr learned my addr 172.21.15.42:0/2087435127 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.499+0000 7ffa85400640 1 -- 172.21.15.42:0/2087435127 >> v1:172.21.15.63:6789/0 conn(0x7ffa8013fd10 legacy=0x7ffa80140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.499+0000 7ffa85400640 1 -- 172.21.15.42:0/2087435127 >> v1:172.21.15.42:6789/0 conn(0x7ffa80071de0 legacy=0x7ffa800721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.499+0000 7ffa84a00640 1 -- 172.21.15.42:0/2087435127 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 867450890 0 0) 0x7ffa80160230 con 0x7ffa80071de0 2024-05-28T14:14:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.499+0000 7ffa85400640 1 -- 172.21.15.42:0/2087435127 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa80160ad0 con 0x7ffa800ce3b0 2024-05-28T14:14:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.503+0000 7ffa85400640 1 --2- 172.21.15.42:0/2087435127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa800ce3b0 0x7ffa800ce810 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7ffa700097c0 tx=0x7ffa7001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21484e2a9ee6fd39 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.503+0000 7ffa84a00640 1 -- 172.21.15.42:0/2087435127 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa700b0070 con 0x7ffa800ce3b0 2024-05-28T14:14:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.503+0000 7ffa84a00640 1 -- 172.21.15.42:0/2087435127 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa700ab600 con 0x7ffa800ce3b0 2024-05-28T14:14:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.503+0000 7ffa84a00640 1 -- 172.21.15.42:0/2087435127 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa700ab9d0 con 0x7ffa800ce3b0 2024-05-28T14:14:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.503+0000 7ffa88800640 1 -- 172.21.15.42:0/2087435127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa800ce3b0 msgr2=0x7ffa800ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.503+0000 7ffa88800640 1 --2- 172.21.15.42:0/2087435127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa800ce3b0 0x7ffa800ce810 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7ffa700097c0 tx=0x7ffa7001c380 comp rx=0 tx=0).stop 2024-05-28T14:14:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.503+0000 7ffa88800640 1 -- 172.21.15.42:0/2087435127 shutdown_connections 2024-05-28T14:14:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.503+0000 7ffa88800640 1 --2- 172.21.15.42:0/2087435127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa800ce3b0 0x7ffa800ce810 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.503+0000 7ffa88800640 1 -- 172.21.15.42:0/2087435127 >> 172.21.15.42:0/2087435127 conn(0x7ffa800b21f0 msgr2=0x7ffa800b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:28.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.503+0000 7ffa88800640 1 -- 172.21.15.42:0/2087435127 shutdown_connections 2024-05-28T14:14:28.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.503+0000 7ffa88800640 1 -- 172.21.15.42:0/2087435127 wait complete. 2024-05-28T14:14:28.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.503+0000 7ffa88800640 1 Processor -- start 2024-05-28T14:14:28.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.503+0000 7ffa88800640 1 -- start start 2024-05-28T14:14:28.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa88800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa80071de0 0x7ffa80155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa88800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa800ce3b0 0x7ffa800b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa88800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa8013fd10 0x7ffa8013bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa88800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8016a840 con 0x7ffa800ce3b0 2024-05-28T14:14:28.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa88800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8016a6c0 con 0x7ffa80071de0 2024-05-28T14:14:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa88800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa8016a9c0 con 0x7ffa8013fd10 2024-05-28T14:14:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa85400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa800ce3b0 0x7ffa800b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa85400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa800ce3b0 0x7ffa800b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49748/0 (socket says 172.21.15.42:49748) 2024-05-28T14:14:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa85400640 1 -- 172.21.15.42:0/2929136079 learned_addr learned my addr 172.21.15.42:0/2929136079 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa85e00640 1 --2- 172.21.15.42:0/2929136079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa80071de0 0x7ffa80155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa86800640 1 --2- 172.21.15.42:0/2929136079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa8013fd10 0x7ffa8013bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa85400640 1 -- 172.21.15.42:0/2929136079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa8013fd10 msgr2=0x7ffa8013bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa85400640 1 --2- 172.21.15.42:0/2929136079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa8013fd10 0x7ffa8013bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa85400640 1 -- 172.21.15.42:0/2929136079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa80071de0 msgr2=0x7ffa80155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa85400640 1 --2- 172.21.15.42:0/2929136079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa80071de0 0x7ffa80155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa85400640 1 -- 172.21.15.42:0/2929136079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8013ac40 con 0x7ffa800ce3b0 2024-05-28T14:14:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa85400640 1 --2- 172.21.15.42:0/2929136079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa800ce3b0 0x7ffa800b3f20 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7ffa70018f90 tx=0x7ffa70019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:28.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa7a000640 1 -- 172.21.15.42:0/2929136079 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa700b8020 con 0x7ffa800ce3b0 2024-05-28T14:14:28.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa7a000640 1 -- 172.21.15.42:0/2929136079 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa700b4610 con 0x7ffa800ce3b0 2024-05-28T14:14:28.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa7a000640 1 -- 172.21.15.42:0/2929136079 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa700b0040 con 0x7ffa800ce3b0 2024-05-28T14:14:28.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa88800640 1 -- 172.21.15.42:0/2929136079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8013af20 con 0x7ffa800ce3b0 2024-05-28T14:14:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa88800640 1 -- 172.21.15.42:0/2929136079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8013b4e0 con 0x7ffa800ce3b0 2024-05-28T14:14:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.507+0000 7ffa88800640 1 -- 172.21.15.42:0/2929136079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa800738b0 con 0x7ffa800ce3b0 2024-05-28T14:14:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.511+0000 7ffa7a000640 1 -- 172.21.15.42:0/2929136079 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7ffa700bb020 con 0x7ffa800ce3b0 2024-05-28T14:14:28.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.511+0000 7ffa7a000640 1 --2- 172.21.15.42:0/2929136079 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa64069fa0 0x7ffa6406c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.511+0000 7ffa7a000640 1 -- 172.21.15.42:0/2929136079 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5879+0+0 (secure 0 0 0) 0x7ffa7011f460 con 0x7ffa800ce3b0 2024-05-28T14:14:28.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.511+0000 7ffa85e00640 1 --2- 172.21.15.42:0/2929136079 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa64069fa0 0x7ffa6406c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.511+0000 7ffa85e00640 1 --2- 172.21.15.42:0/2929136079 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa64069fa0 0x7ffa6406c460 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ffa74007bc0 tx=0x7ffa74007b10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:28.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.515+0000 7ffa7a000640 1 -- 172.21.15.42:0/2929136079 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffa700c2030 con 0x7ffa800ce3b0 2024-05-28T14:14:28.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.683+0000 7ffa88800640 1 -- 172.21.15.42:0/2929136079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffa8004fdd0 con 0x7ffa800ce3b0 2024-05-28T14:14:28.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.683+0000 7ffa7a000640 1 -- 172.21.15.42:0/2929136079 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) ==== 93+0+10 (secure 0 0 0) 0x7ffa70016090 con 0x7ffa800ce3b0 2024-05-28T14:14:28.667 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:14:28.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.683+0000 7ffa88800640 1 -- 172.21.15.42:0/2929136079 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa64069fa0 msgr2=0x7ffa6406c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.683+0000 7ffa88800640 1 --2- 172.21.15.42:0/2929136079 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa64069fa0 0x7ffa6406c460 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ffa74007bc0 tx=0x7ffa74007b10 comp rx=0 tx=0).stop 2024-05-28T14:14:28.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.683+0000 7ffa88800640 1 -- 172.21.15.42:0/2929136079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa800ce3b0 msgr2=0x7ffa800b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.683+0000 7ffa88800640 1 --2- 172.21.15.42:0/2929136079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa800ce3b0 0x7ffa800b3f20 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7ffa70018f90 tx=0x7ffa70019120 comp rx=0 tx=0).stop 2024-05-28T14:14:28.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.683+0000 7ffa88800640 1 -- 172.21.15.42:0/2929136079 shutdown_connections 2024-05-28T14:14:28.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.683+0000 7ffa88800640 1 --2- 172.21.15.42:0/2929136079 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa64069fa0 0x7ffa6406c460 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.683+0000 7ffa88800640 1 --2- 172.21.15.42:0/2929136079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa8013fd10 0x7ffa8013bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.683+0000 7ffa88800640 1 --2- 172.21.15.42:0/2929136079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa800ce3b0 0x7ffa800b3f20 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.683+0000 7ffa88800640 1 --2- 172.21.15.42:0/2929136079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa80071de0 0x7ffa80155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.683+0000 7ffa88800640 1 -- 172.21.15.42:0/2929136079 >> 172.21.15.42:0/2929136079 conn(0x7ffa800b21f0 msgr2=0x7ffa8006f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:28.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.683+0000 7ffa88800640 1 -- 172.21.15.42:0/2929136079 shutdown_connections 2024-05-28T14:14:28.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.683+0000 7ffa88800640 1 -- 172.21.15.42:0/2929136079 wait complete. 2024-05-28T14:14:28.689 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:14:28.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.847+0000 7f7754e00640 1 Processor -- start 2024-05-28T14:14:28.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.847+0000 7f7754e00640 1 -- start start 2024-05-28T14:14:28.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.847+0000 7f7754e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7750144f80 0x7f7750072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.847+0000 7f7754e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77500afe40 con 0x7f775006ee20 2024-05-28T14:14:28.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.847+0000 7f7754e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f775015c600 con 0x7f775015c0c0 2024-05-28T14:14:28.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.847+0000 7f7754e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f77500724d0 con 0x7f7750144f80 2024-05-28T14:14:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.847+0000 7f774de00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7750144f80 0x7f7750072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.847+0000 7f7747e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f775006ee20 0x7f775015b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36734/0 (socket says 172.21.15.42:36734) 2024-05-28T14:14:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.847+0000 7f7747e00640 1 -- 172.21.15.42:0/3755244881 learned_addr learned my addr 172.21.15.42:0/3755244881 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.851+0000 7f774de00640 1 -- 172.21.15.42:0/3755244881 >> v1:172.21.15.63:6789/0 conn(0x7f775015c0c0 legacy=0x7f77500b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.851+0000 7f774de00640 1 -- 172.21.15.42:0/3755244881 >> v1:172.21.15.42:6789/0 conn(0x7f775006ee20 legacy=0x7f775015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.851+0000 7f774de00640 1 -- 172.21.15.42:0/3755244881 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775015c8e0 con 0x7f7750144f80 2024-05-28T14:14:28.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.851+0000 7f774de00640 1 --2- 172.21.15.42:0/3755244881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750144f80 0x7f7750072960 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f77400097c0 tx=0x7f774001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2bf20a1f90cf2b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:28.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.851+0000 7f774d400640 1 -- 172.21.15.42:0/3755244881 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77400b0070 con 0x7f7750144f80 2024-05-28T14:14:28.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.851+0000 7f774d400640 1 -- 172.21.15.42:0/3755244881 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77400ab600 con 0x7f7750144f80 2024-05-28T14:14:28.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.851+0000 7f774d400640 1 -- 172.21.15.42:0/3755244881 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77400ab9d0 con 0x7f7750144f80 2024-05-28T14:14:28.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.851+0000 7f7754e00640 1 -- 172.21.15.42:0/3755244881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750144f80 msgr2=0x7f7750072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.851+0000 7f7754e00640 1 --2- 172.21.15.42:0/3755244881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750144f80 0x7f7750072960 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f77400097c0 tx=0x7f774001c4a0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.851+0000 7f7754e00640 1 -- 172.21.15.42:0/3755244881 shutdown_connections 2024-05-28T14:14:28.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.851+0000 7f7754e00640 1 --2- 172.21.15.42:0/3755244881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750144f80 0x7f7750072960 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.851+0000 7f7754e00640 1 -- 172.21.15.42:0/3755244881 >> 172.21.15.42:0/3755244881 conn(0x7f77500b21f0 msgr2=0x7f77500b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:28.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7754e00640 1 -- 172.21.15.42:0/3755244881 shutdown_connections 2024-05-28T14:14:28.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7754e00640 1 -- 172.21.15.42:0/3755244881 wait complete. 2024-05-28T14:14:28.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7754e00640 1 Processor -- start 2024-05-28T14:14:28.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7754e00640 1 -- start start 2024-05-28T14:14:28.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7754e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f775006ee20 0x7f775015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7754e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750144f80 0x7f775013db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7754e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f775015c0c0 0x7f775013e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7754e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f775006f790 con 0x7f7750144f80 2024-05-28T14:14:28.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7754e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f775006f610 con 0x7f775015c0c0 2024-05-28T14:14:28.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7754e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f775006e520 con 0x7f775006ee20 2024-05-28T14:14:28.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7747e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750144f80 0x7f775013db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7747e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750144f80 0x7f775013db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49768/0 (socket says 172.21.15.42:49768) 2024-05-28T14:14:28.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7747e00640 1 -- 172.21.15.42:0/2066820399 learned_addr learned my addr 172.21.15.42:0/2066820399 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:28.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f774de00640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f775006ee20 0x7f775015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f774e800640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f775015c0c0 0x7f775013e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7747e00640 1 -- 172.21.15.42:0/2066820399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f775006ee20 msgr2=0x7f775015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7747e00640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f775006ee20 0x7f775015ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7747e00640 1 -- 172.21.15.42:0/2066820399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f775015c0c0 msgr2=0x7f775013e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:28.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7747e00640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f775015c0c0 0x7f775013e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:28.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7747e00640 1 -- 172.21.15.42:0/2066820399 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77501406f0 con 0x7f7750144f80 2024-05-28T14:14:28.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f774e800640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f775015c0c0 0x7f775013e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:28.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7747e00640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750144f80 0x7f775013db90 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f773800c500 tx=0x7f773800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:28.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.855+0000 7f7746000640 1 -- 172.21.15.42:0/2066820399 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7738017070 con 0x7f7750144f80 2024-05-28T14:14:28.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.859+0000 7f7746000640 1 -- 172.21.15.42:0/2066820399 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7738005cf0 con 0x7f7750144f80 2024-05-28T14:14:28.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.859+0000 7f7754e00640 1 -- 172.21.15.42:0/2066820399 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77501434f0 con 0x7f7750144f80 2024-05-28T14:14:28.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.859+0000 7f7746000640 1 -- 172.21.15.42:0/2066820399 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77380072f0 con 0x7f7750144f80 2024-05-28T14:14:28.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.859+0000 7f7754e00640 1 -- 172.21.15.42:0/2066820399 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f775013fc10 con 0x7f7750144f80 2024-05-28T14:14:28.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.859+0000 7f7754e00640 1 -- 172.21.15.42:0/2066820399 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77500738b0 con 0x7f7750144f80 2024-05-28T14:14:28.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.859+0000 7f7746000640 1 -- 172.21.15.42:0/2066820399 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f773801a070 con 0x7f7750144f80 2024-05-28T14:14:28.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.859+0000 7f7746000640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f772c069fa0 0x7f772c06c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:28.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.859+0000 7f7746000640 1 -- 172.21.15.42:0/2066820399 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5879+0+0 (secure 0 0 0) 0x7f7738084c70 con 0x7f7750144f80 2024-05-28T14:14:28.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.859+0000 7f774de00640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f772c069fa0 0x7f772c06c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:28.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.859+0000 7f774de00640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f772c069fa0 0x7f772c06c460 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f77400ad000 tx=0x7f77400a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:28.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:28.863+0000 7f7746000640 1 -- 172.21.15.42:0/2066820399 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7738056640 con 0x7f7750144f80 2024-05-28T14:14:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.030+0000 7f7754e00640 1 -- 172.21.15.42:0/2066820399 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f77500b0600 con 0x7f7750144f80 2024-05-28T14:14:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.030+0000 7f7746000640 1 -- 172.21.15.42:0/2066820399 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) ==== 104+0+11 (secure 0 0 0) 0x7f773805a940 con 0x7f7750144f80 2024-05-28T14:14:29.016 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:14:29.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.030+0000 7f7754e00640 1 -- 172.21.15.42:0/2066820399 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f772c069fa0 msgr2=0x7f772c06c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.030+0000 7f7754e00640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f772c069fa0 0x7f772c06c460 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f77400ad000 tx=0x7f77400a6000 comp rx=0 tx=0).stop 2024-05-28T14:14:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.030+0000 7f7754e00640 1 -- 172.21.15.42:0/2066820399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750144f80 msgr2=0x7f775013db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.030+0000 7f7754e00640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750144f80 0x7f775013db90 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f773800c500 tx=0x7f773800c9c0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.030+0000 7f7754e00640 1 -- 172.21.15.42:0/2066820399 shutdown_connections 2024-05-28T14:14:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.030+0000 7f7754e00640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f772c069fa0 0x7f772c06c460 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.030+0000 7f7754e00640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f775015c0c0 0x7f775013e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.030+0000 7f7754e00640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750144f80 0x7f775013db90 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.030+0000 7f7754e00640 1 --2- 172.21.15.42:0/2066820399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f775006ee20 0x7f775015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.030+0000 7f7754e00640 1 -- 172.21.15.42:0/2066820399 >> 172.21.15.42:0/2066820399 conn(0x7f77500b21f0 msgr2=0x7f77500a9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.030+0000 7f7754e00640 1 -- 172.21.15.42:0/2066820399 shutdown_connections 2024-05-28T14:14:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.030+0000 7f7754e00640 1 -- 172.21.15.42:0/2066820399 wait complete. 2024-05-28T14:14:29.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:14:29.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.194+0000 7fc236600640 1 Processor -- start 2024-05-28T14:14:29.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.194+0000 7fc236600640 1 -- start start 2024-05-28T14:14:29.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.194+0000 7fc236600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc23009b8e0 0x7fc23006cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.194+0000 7fc236600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2300d0770 con 0x7fc2300b0e80 2024-05-28T14:14:29.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.194+0000 7fc236600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2300d0a10 con 0x7fc23009b160 2024-05-28T14:14:29.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.194+0000 7fc236600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc2300d0cf0 con 0x7fc23009b8e0 2024-05-28T14:14:29.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.194+0000 7fc22fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc23009b8e0 0x7fc23006cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.194+0000 7fc22f400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc2300b0e80 0x7fc23009aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36748/0 (socket says 172.21.15.42:36748) 2024-05-28T14:14:29.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.194+0000 7fc22f400640 1 -- 172.21.15.42:0/3027700692 learned_addr learned my addr 172.21.15.42:0/3027700692 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:29.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc22fe00640 1 -- 172.21.15.42:0/3027700692 >> v1:172.21.15.63:6789/0 conn(0x7fc23009b160 legacy=0x7fc23015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc22fe00640 1 -- 172.21.15.42:0/3027700692 >> v1:172.21.15.42:6789/0 conn(0x7fc2300b0e80 legacy=0x7fc23009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc22fe00640 1 -- 172.21.15.42:0/3027700692 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2300d0fd0 con 0x7fc23009b8e0 2024-05-28T14:14:29.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc22ea00640 1 -- 172.21.15.42:0/3027700692 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2923722485 0 0) 0x7fc2300d0770 con 0x7fc2300b0e80 2024-05-28T14:14:29.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc22fe00640 1 --2- 172.21.15.42:0/3027700692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23009b8e0 0x7fc23006cb00 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fc2280097c0 tx=0x7fc22801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80c6cb60f3eae1 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:29.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc22ea00640 1 -- 172.21.15.42:0/3027700692 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2280b0070 con 0x7fc23009b8e0 2024-05-28T14:14:29.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc22ea00640 1 -- 172.21.15.42:0/3027700692 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2280ab600 con 0x7fc23009b8e0 2024-05-28T14:14:29.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc22ea00640 1 -- 172.21.15.42:0/3027700692 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2280ab9d0 con 0x7fc23009b8e0 2024-05-28T14:14:29.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc236600640 1 -- 172.21.15.42:0/3027700692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23009b8e0 msgr2=0x7fc23006cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc236600640 1 --2- 172.21.15.42:0/3027700692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23009b8e0 0x7fc23006cb00 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fc2280097c0 tx=0x7fc22801c4a0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc236600640 1 -- 172.21.15.42:0/3027700692 shutdown_connections 2024-05-28T14:14:29.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc236600640 1 --2- 172.21.15.42:0/3027700692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23009b8e0 0x7fc23006cb00 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc236600640 1 -- 172.21.15.42:0/3027700692 >> 172.21.15.42:0/3027700692 conn(0x7fc2300b3f90 msgr2=0x7fc2300b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:29.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc236600640 1 -- 172.21.15.42:0/3027700692 shutdown_connections 2024-05-28T14:14:29.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.198+0000 7fc236600640 1 -- 172.21.15.42:0/3027700692 wait complete. 2024-05-28T14:14:29.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc236600640 1 Processor -- start 2024-05-28T14:14:29.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc236600640 1 -- start start 2024-05-28T14:14:29.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc236600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc23009b160 0x7fc23009c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc236600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23009b8e0 0x7fc2300b6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc236600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2300b0e80 0x7fc23015aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc236600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc23015ec30 con 0x7fc23009b8e0 2024-05-28T14:14:29.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc236600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2300d1a90 con 0x7fc2300b0e80 2024-05-28T14:14:29.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc236600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc23016cf90 con 0x7fc23009b160 2024-05-28T14:14:29.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc226a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23009b8e0 0x7fc2300b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc226a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23009b8e0 0x7fc2300b6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49788/0 (socket says 172.21.15.42:49788) 2024-05-28T14:14:29.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc226a00640 1 -- 172.21.15.42:0/1335668470 learned_addr learned my addr 172.21.15.42:0/1335668470 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc22f400640 1 --2- 172.21.15.42:0/1335668470 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc23009b160 0x7fc23009c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc22fe00640 1 --2- 172.21.15.42:0/1335668470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2300b0e80 0x7fc23015aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc226a00640 1 -- 172.21.15.42:0/1335668470 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc23009b160 msgr2=0x7fc23009c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc226a00640 1 --2- 172.21.15.42:0/1335668470 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc23009b160 0x7fc23009c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc226a00640 1 -- 172.21.15.42:0/1335668470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2300b0e80 msgr2=0x7fc23015aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc226a00640 1 --2- 172.21.15.42:0/1335668470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2300b0e80 0x7fc23015aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc226a00640 1 -- 172.21.15.42:0/1335668470 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc230158f00 con 0x7fc23009b8e0 2024-05-28T14:14:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc226a00640 1 --2- 172.21.15.42:0/1335668470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23009b8e0 0x7fc2300b6470 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fc21800c3c0 tx=0x7fc21800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:29.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc227e00640 1 -- 172.21.15.42:0/1335668470 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218017070 con 0x7fc23009b8e0 2024-05-28T14:14:29.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc227e00640 1 -- 172.21.15.42:0/1335668470 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2180052d0 con 0x7fc23009b8e0 2024-05-28T14:14:29.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc227e00640 1 -- 172.21.15.42:0/1335668470 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218003840 con 0x7fc23009b8e0 2024-05-28T14:14:29.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc236600640 1 -- 172.21.15.42:0/1335668470 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2301591e0 con 0x7fc23009b8e0 2024-05-28T14:14:29.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.202+0000 7fc236600640 1 -- 172.21.15.42:0/1335668470 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc230158b90 con 0x7fc23009b8e0 2024-05-28T14:14:29.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.206+0000 7fc236600640 1 -- 172.21.15.42:0/1335668470 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2300a0550 con 0x7fc23009b8e0 2024-05-28T14:14:29.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.206+0000 7fc227e00640 1 -- 172.21.15.42:0/1335668470 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7fc21801a070 con 0x7fc23009b8e0 2024-05-28T14:14:29.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.206+0000 7fc227e00640 1 --2- 172.21.15.42:0/1335668470 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc20c069fa0 0x7fc20c06c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.206+0000 7fc227e00640 1 -- 172.21.15.42:0/1335668470 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5879+0+0 (secure 0 0 0) 0x7fc218084c40 con 0x7fc23009b8e0 2024-05-28T14:14:29.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.206+0000 7fc22f400640 1 --2- 172.21.15.42:0/1335668470 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc20c069fa0 0x7fc20c06c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.206+0000 7fc22f400640 1 --2- 172.21.15.42:0/1335668470 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc20c069fa0 0x7fc20c06c460 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc2200012f0 tx=0x7fc220004d50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:29.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.210+0000 7fc227e00640 1 -- 172.21.15.42:0/1335668470 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc218005480 con 0x7fc23009b8e0 2024-05-28T14:14:29.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.374+0000 7fc236600640 1 -- 172.21.15.42:0/1335668470 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc2300b16a0 con 0x7fc23009b8e0 2024-05-28T14:14:29.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.374+0000 7fc227e00640 1 -- 172.21.15.42:0/1335668470 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) ==== 100+0+11 (secure 0 0 0) 0x7fc218056610 con 0x7fc23009b8e0 2024-05-28T14:14:29.362 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:14:29.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.378+0000 7fc236600640 1 -- 172.21.15.42:0/1335668470 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc20c069fa0 msgr2=0x7fc20c06c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.378+0000 7fc236600640 1 --2- 172.21.15.42:0/1335668470 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc20c069fa0 0x7fc20c06c460 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc2200012f0 tx=0x7fc220004d50 comp rx=0 tx=0).stop 2024-05-28T14:14:29.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.378+0000 7fc236600640 1 -- 172.21.15.42:0/1335668470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23009b8e0 msgr2=0x7fc2300b6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.378+0000 7fc236600640 1 --2- 172.21.15.42:0/1335668470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23009b8e0 0x7fc2300b6470 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fc21800c3c0 tx=0x7fc21800c7c0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.378+0000 7fc236600640 1 -- 172.21.15.42:0/1335668470 shutdown_connections 2024-05-28T14:14:29.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.378+0000 7fc236600640 1 --2- 172.21.15.42:0/1335668470 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc20c069fa0 0x7fc20c06c460 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.378+0000 7fc236600640 1 --2- 172.21.15.42:0/1335668470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2300b0e80 0x7fc23015aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.378+0000 7fc236600640 1 --2- 172.21.15.42:0/1335668470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc23009b8e0 0x7fc2300b6470 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.378+0000 7fc236600640 1 --2- 172.21.15.42:0/1335668470 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc23009b160 0x7fc23009c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.378+0000 7fc236600640 1 -- 172.21.15.42:0/1335668470 >> 172.21.15.42:0/1335668470 conn(0x7fc2300b3f90 msgr2=0x7fc2300b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:29.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.378+0000 7fc236600640 1 -- 172.21.15.42:0/1335668470 shutdown_connections 2024-05-28T14:14:29.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.378+0000 7fc236600640 1 -- 172.21.15.42:0/1335668470 wait complete. 2024-05-28T14:14:29.384 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:14:29.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc0da00640 1 Processor -- start 2024-05-28T14:14:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc0da00640 1 -- start start 2024-05-28T14:14:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc0da00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcc081623d0 0x7fcc08070120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc0da00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcc0806cde0 0x7fcc0806d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc0da00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc080b1ae0 con 0x7fcc081627b0 2024-05-28T14:14:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc0da00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcc080d0b20 con 0x7fcc081623d0 2024-05-28T14:14:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc0da00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcc080d0d40 con 0x7fcc0806cde0 2024-05-28T14:14:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc07e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcc081623d0 0x7fcc08070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc0d000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcc0806cde0 0x7fcc0806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc07e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcc081623d0 0x7fcc08070120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:41970/0 (socket says 172.21.15.42:41970) 2024-05-28T14:14:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc07e00640 1 -- 172.21.15.42:0/1290100319 learned_addr learned my addr 172.21.15.42:0/1290100319 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc07e00640 1 -- 172.21.15.42:0/1290100319 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcc0806cde0 msgr2=0x7fcc0806d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc07e00640 1 --2- 172.21.15.42:0/1290100319 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcc0806cde0 0x7fcc0806d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc07e00640 1 -- 172.21.15.42:0/1290100319 >> v1:172.21.15.63:6789/0 conn(0x7fcc081627b0 legacy=0x7fcc080d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.538+0000 7fcc07e00640 1 -- 172.21.15.42:0/1290100319 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc080d1020 con 0x7fcc081623d0 2024-05-28T14:14:29.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.542+0000 7fcc07e00640 1 --2- 172.21.15.42:0/1290100319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc081623d0 0x7fcc08070120 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009870 tx=0x7fcbf801c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7640c73554abb782 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.542+0000 7fcc07400640 1 -- 172.21.15.42:0/1290100319 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf80a95f0 con 0x7fcc081623d0 2024-05-28T14:14:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.542+0000 7fcc07400640 1 -- 172.21.15.42:0/1290100319 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf80a9790 con 0x7fcc081623d0 2024-05-28T14:14:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.542+0000 7fcc07400640 1 -- 172.21.15.42:0/1290100319 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf80a9b60 con 0x7fcc081623d0 2024-05-28T14:14:29.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 -- 172.21.15.42:0/1290100319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc081623d0 msgr2=0x7fcc08070120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 --2- 172.21.15.42:0/1290100319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc081623d0 0x7fcc08070120 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009870 tx=0x7fcbf801c530 comp rx=0 tx=0).stop 2024-05-28T14:14:29.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 -- 172.21.15.42:0/1290100319 shutdown_connections 2024-05-28T14:14:29.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 --2- 172.21.15.42:0/1290100319 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcc0806cde0 0x7fcc0806d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 --2- 172.21.15.42:0/1290100319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc081623d0 0x7fcc08070120 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 -- 172.21.15.42:0/1290100319 >> 172.21.15.42:0/1290100319 conn(0x7fcc080b3d70 msgr2=0x7fcc080b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 -- 172.21.15.42:0/1290100319 shutdown_connections 2024-05-28T14:14:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 -- 172.21.15.42:0/1290100319 wait complete. 2024-05-28T14:14:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 Processor -- start 2024-05-28T14:14:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 -- start start 2024-05-28T14:14:29.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc0806cde0 0x7fcc080cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc081623d0 0x7fcc080c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcc081627b0 0x7fcc08152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcc08162d10 con 0x7fcc081623d0 2024-05-28T14:14:29.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcc08162b90 con 0x7fcc0806cde0 2024-05-28T14:14:29.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0da00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0816b710 con 0x7fcc081627b0 2024-05-28T14:14:29.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc081623d0 0x7fcc080c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc081623d0 0x7fcc080c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49804/0 (socket says 172.21.15.42:49804) 2024-05-28T14:14:29.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcbffe00640 1 -- 172.21.15.42:0/3458556501 learned_addr learned my addr 172.21.15.42:0/3458556501 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:29.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc07e00640 1 --2- 172.21.15.42:0/3458556501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc0806cde0 0x7fcc080cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcc0d000640 1 --2- 172.21.15.42:0/3458556501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcc081627b0 0x7fcc08152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcbffe00640 1 -- 172.21.15.42:0/3458556501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcc081627b0 msgr2=0x7fcc08152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcbffe00640 1 --2- 172.21.15.42:0/3458556501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcc081627b0 0x7fcc08152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcbffe00640 1 -- 172.21.15.42:0/3458556501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc0806cde0 msgr2=0x7fcc080cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcbffe00640 1 --2- 172.21.15.42:0/3458556501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc0806cde0 0x7fcc080cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcbffe00640 1 -- 172.21.15.42:0/3458556501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc080cc290 con 0x7fcc081623d0 2024-05-28T14:14:29.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.546+0000 7fcbffe00640 1 --2- 172.21.15.42:0/3458556501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc081623d0 0x7fcc080c9d30 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000c420 tx=0x7fcbf000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:29.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.550+0000 7fcc04c00640 1 -- 172.21.15.42:0/3458556501 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00052d0 con 0x7fcc081623d0 2024-05-28T14:14:29.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.550+0000 7fcc04c00640 1 -- 172.21.15.42:0/3458556501 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf0005470 con 0x7fcc081623d0 2024-05-28T14:14:29.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.550+0000 7fcc04c00640 1 -- 172.21.15.42:0/3458556501 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0007260 con 0x7fcc081623d0 2024-05-28T14:14:29.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.550+0000 7fcc0da00640 1 -- 172.21.15.42:0/3458556501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc080aff60 con 0x7fcc081623d0 2024-05-28T14:14:29.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.550+0000 7fcc0da00640 1 -- 172.21.15.42:0/3458556501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc080b0520 con 0x7fcc081623d0 2024-05-28T14:14:29.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.550+0000 7fcc0da00640 1 -- 172.21.15.42:0/3458556501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc0804fcf0 con 0x7fcc081623d0 2024-05-28T14:14:29.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.550+0000 7fcc04c00640 1 -- 172.21.15.42:0/3458556501 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7fcbf0007400 con 0x7fcc081623d0 2024-05-28T14:14:29.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.550+0000 7fcc04c00640 1 --2- 172.21.15.42:0/3458556501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcbe4069fd0 0x7fcbe406c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.550+0000 7fcc04c00640 1 -- 172.21.15.42:0/3458556501 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5879+0+0 (secure 0 0 0) 0x7fcbf0084d60 con 0x7fcc081623d0 2024-05-28T14:14:29.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.550+0000 7fcc07e00640 1 --2- 172.21.15.42:0/3458556501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcbe4069fd0 0x7fcbe406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.550+0000 7fcc07e00640 1 --2- 172.21.15.42:0/3458556501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcbe4069fd0 0x7fcbe406c490 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80ae000 tx=0x7fcbf80b3040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:29.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.554+0000 7fcc04c00640 1 -- 172.21.15.42:0/3458556501 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcbf0056730 con 0x7fcc081623d0 2024-05-28T14:14:29.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.754+0000 7fcc0da00640 1 -- 172.21.15.42:0/3458556501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc08070670 con 0x7fcc081623d0 2024-05-28T14:14:29.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.754+0000 7fcc04c00640 1 -- 172.21.15.42:0/3458556501 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5180 (secure 0 0 0) 0x7fcbf005aa30 con 0x7fcc081623d0 2024-05-28T14:14:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:29.743 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/56330151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":56330151},{"type":"v1","addr":"172.21.15.42:6835","nonce":56330151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":9183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/192920662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":192920662},{"type":"v1","addr":"172.21.15.63:6835","nonce":192920662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":57437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1225711200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1225711200},{"type":"v1","addr":"172.21.15.42:6837","nonce":1225711200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:12:59.544555+0000","modified":"2024-05-28T14:13:02.565331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57440":{"gid":57440,"name":"b","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/3959626682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3959626682},{"type":"v1","addr":"172.21.15.63:6834","nonce":3959626682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57440,"qdb_cluster":[57440]},"id":8}]} 2024-05-28T14:14:29.743 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 75 2024-05-28T14:14:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.758+0000 7fcc0da00640 1 -- 172.21.15.42:0/3458556501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcbe4069fd0 msgr2=0x7fcbe406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.758+0000 7fcc0da00640 1 --2- 172.21.15.42:0/3458556501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcbe4069fd0 0x7fcbe406c490 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80ae000 tx=0x7fcbf80b3040 comp rx=0 tx=0).stop 2024-05-28T14:14:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.758+0000 7fcc0da00640 1 -- 172.21.15.42:0/3458556501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc081623d0 msgr2=0x7fcc080c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.758+0000 7fcc0da00640 1 --2- 172.21.15.42:0/3458556501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc081623d0 0x7fcc080c9d30 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000c420 tx=0x7fcbf000c8e0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.758+0000 7fcc0da00640 1 -- 172.21.15.42:0/3458556501 shutdown_connections 2024-05-28T14:14:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.758+0000 7fcc0da00640 1 --2- 172.21.15.42:0/3458556501 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcbe4069fd0 0x7fcbe406c490 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.758+0000 7fcc0da00640 1 --2- 172.21.15.42:0/3458556501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcc081627b0 0x7fcc08152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.758+0000 7fcc0da00640 1 --2- 172.21.15.42:0/3458556501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc081623d0 0x7fcc080c9d30 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.758+0000 7fcc0da00640 1 --2- 172.21.15.42:0/3458556501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc0806cde0 0x7fcc080cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.758+0000 7fcc0da00640 1 -- 172.21.15.42:0/3458556501 >> 172.21.15.42:0/3458556501 conn(0x7fcc080b3d70 msgr2=0x7fcc080ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.758+0000 7fcc0da00640 1 -- 172.21.15.42:0/3458556501 shutdown_connections 2024-05-28T14:14:29.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.758+0000 7fcc0da00640 1 -- 172.21.15.42:0/3458556501 wait complete. 2024-05-28T14:14:29.764 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:14:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.918+0000 7faa34000640 1 Processor -- start 2024-05-28T14:14:29.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.918+0000 7faa34000640 1 -- start start 2024-05-28T14:14:29.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.918+0000 7faa34000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faa2c156e50 0x7faa2c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.918+0000 7faa34000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa2c15cb90 con 0x7faa2c156a70 2024-05-28T14:14:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.918+0000 7faa34000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa2c15ce50 con 0x7faa2c0afa50 2024-05-28T14:14:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.918+0000 7faa34000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faa2c15d130 con 0x7faa2c156e50 2024-05-28T14:14:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.918+0000 7faa31600640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7faa2c156a70 0x7faa2c0af340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36756/0 (socket says 172.21.15.42:36756) 2024-05-28T14:14:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.918+0000 7faa31600640 1 -- 172.21.15.42:0/3736218235 learned_addr learned my addr 172.21.15.42:0/3736218235 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.918+0000 7faa32000640 1 --2- 172.21.15.42:0/3736218235 >> v2:172.21.15.63:3300/0 conn(0x7faa2c156e50 0x7faa2c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.922+0000 7faa2be00640 1 -- 172.21.15.42:0/3736218235 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 36712380 0 0) 0x7faa2c15cb90 con 0x7faa2c156a70 2024-05-28T14:14:29.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.922+0000 7faa2be00640 1 -- 172.21.15.42:0/3736218235 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa14003530 con 0x7faa2c156a70 2024-05-28T14:14:29.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.922+0000 7faa32000640 1 -- 172.21.15.42:0/3736218235 >> v1:172.21.15.63:6789/0 conn(0x7faa2c0afa50 legacy=0x7faa2c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.922+0000 7faa32000640 1 -- 172.21.15.42:0/3736218235 >> v1:172.21.15.42:6789/0 conn(0x7faa2c156a70 legacy=0x7faa2c0af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.922+0000 7faa32000640 1 -- 172.21.15.42:0/3736218235 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2c15d410 con 0x7faa2c156e50 2024-05-28T14:14:29.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.922+0000 7faa32000640 1 --2- 172.21.15.42:0/3736218235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2c156e50 0x7faa2c06dd60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7faa24009d90 tx=0x7faa2401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bfbe3bfdc5f7de server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:29.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.922+0000 7faa2be00640 1 -- 172.21.15.42:0/3736218235 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa240b2070 con 0x7faa2c156e50 2024-05-28T14:14:29.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.922+0000 7faa2be00640 1 -- 172.21.15.42:0/3736218235 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa240a5a50 con 0x7faa2c156e50 2024-05-28T14:14:29.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.922+0000 7faa2be00640 1 -- 172.21.15.42:0/3736218235 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa240a5e00 con 0x7faa2c156e50 2024-05-28T14:14:29.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 -- 172.21.15.42:0/3736218235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2c156e50 msgr2=0x7faa2c06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 --2- 172.21.15.42:0/3736218235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2c156e50 0x7faa2c06dd60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7faa24009d90 tx=0x7faa2401c970 comp rx=0 tx=0).stop 2024-05-28T14:14:29.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 -- 172.21.15.42:0/3736218235 shutdown_connections 2024-05-28T14:14:29.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 --2- 172.21.15.42:0/3736218235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2c156e50 0x7faa2c06dd60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 -- 172.21.15.42:0/3736218235 >> 172.21.15.42:0/3736218235 conn(0x7faa2c06bdf0 msgr2=0x7faa2c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:29.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 -- 172.21.15.42:0/3736218235 shutdown_connections 2024-05-28T14:14:29.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 -- 172.21.15.42:0/3736218235 wait complete. 2024-05-28T14:14:29.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 Processor -- start 2024-05-28T14:14:29.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 -- start start 2024-05-28T14:14:29.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa2c0afa50 0x7faa2c06ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2c156a70 0x7faa2c073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa2c156e50 0x7faa2c073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c160f10 con 0x7faa2c156e50 2024-05-28T14:14:29.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c160d90 con 0x7faa2c156a70 2024-05-28T14:14:29.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa34000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faa2c161090 con 0x7faa2c0afa50 2024-05-28T14:14:29.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa32000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa2c156e50 0x7faa2c073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa32000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa2c156e50 0x7faa2c073690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49812/0 (socket says 172.21.15.42:49812) 2024-05-28T14:14:29.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa32000640 1 -- 172.21.15.42:0/2535897963 learned_addr learned my addr 172.21.15.42:0/2535897963 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:29.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa31600640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa2c0afa50 0x7faa2c06ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.926+0000 7faa30c00640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2c156a70 0x7faa2c073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa32000640 1 -- 172.21.15.42:0/2535897963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa2c0afa50 msgr2=0x7faa2c06ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa32000640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa2c0afa50 0x7faa2c06ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa32000640 1 -- 172.21.15.42:0/2535897963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2c156a70 msgr2=0x7faa2c073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:29.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa32000640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2c156a70 0x7faa2c073150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:29.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa32000640 1 -- 172.21.15.42:0/2535897963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2c06f130 con 0x7faa2c156e50 2024-05-28T14:14:29.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa31600640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa2c0afa50 0x7faa2c06ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:14:29.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa30c00640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2c156a70 0x7faa2c073150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:14:29.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa32000640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa2c156e50 0x7faa2c073690 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7faa24009d90 tx=0x7faa24019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:29.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa29600640 1 -- 172.21.15.42:0/2535897963 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa240b2050 con 0x7faa2c156e50 2024-05-28T14:14:29.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa29600640 1 -- 172.21.15.42:0/2535897963 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa240b3040 con 0x7faa2c156e50 2024-05-28T14:14:29.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa34000640 1 -- 172.21.15.42:0/2535897963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa2c14c1d0 con 0x7faa2c156e50 2024-05-28T14:14:29.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa29600640 1 -- 172.21.15.42:0/2535897963 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24003ba0 con 0x7faa2c156e50 2024-05-28T14:14:29.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa34000640 1 -- 172.21.15.42:0/2535897963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa2c14c790 con 0x7faa2c156e50 2024-05-28T14:14:29.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa34000640 1 -- 172.21.15.42:0/2535897963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa2c068a90 con 0x7faa2c156e50 2024-05-28T14:14:29.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa29600640 1 -- 172.21.15.42:0/2535897963 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7faa240c0020 con 0x7faa2c156e50 2024-05-28T14:14:29.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa29600640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa0c069fd0 0x7faa0c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:29.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.930+0000 7faa29600640 1 -- 172.21.15.42:0/2535897963 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5879+0+0 (secure 0 0 0) 0x7faa2411fcd0 con 0x7faa2c156e50 2024-05-28T14:14:29.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.934+0000 7faa31600640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa0c069fd0 0x7faa0c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:29.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.934+0000 7faa31600640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa0c069fd0 0x7faa0c06c490 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7faa200011f0 tx=0x7faa20007d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:29.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:29.938+0000 7faa29600640 1 -- 172.21.15.42:0/2535897963 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faa240c7020 con 0x7faa2c156e50 2024-05-28T14:14:30.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.106+0000 7faa34000640 1 -- 172.21.15.42:0/2535897963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa2c06fd30 con 0x7faa2c156e50 2024-05-28T14:14:30.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.106+0000 7faa29600640 1 -- 172.21.15.42:0/2535897963 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+16921 (secure 0 0 0) 0x7faa24016560 con 0x7faa2c156e50 2024-05-28T14:14:30.093 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:30.093 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":120,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:14:26.034186+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:12:57.936312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-28T14:12:58.495752+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":120,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.24","mappings":[{"from":0,"to":5}]},{"pgid":"17.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:14:30.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.110+0000 7faa34000640 1 -- 172.21.15.42:0/2535897963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa0c069fd0 msgr2=0x7faa0c06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:30.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.110+0000 7faa34000640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa0c069fd0 0x7faa0c06c490 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7faa200011f0 tx=0x7faa20007d40 comp rx=0 tx=0).stop 2024-05-28T14:14:30.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.110+0000 7faa34000640 1 -- 172.21.15.42:0/2535897963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa2c156e50 msgr2=0x7faa2c073690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:30.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.110+0000 7faa34000640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa2c156e50 0x7faa2c073690 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7faa24009d90 tx=0x7faa24019710 comp rx=0 tx=0).stop 2024-05-28T14:14:30.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.110+0000 7faa34000640 1 -- 172.21.15.42:0/2535897963 shutdown_connections 2024-05-28T14:14:30.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.110+0000 7faa34000640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa0c069fd0 0x7faa0c06c490 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.110+0000 7faa34000640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa2c156e50 0x7faa2c073690 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.110+0000 7faa34000640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa2c156a70 0x7faa2c073150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.110+0000 7faa34000640 1 --2- 172.21.15.42:0/2535897963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa2c0afa50 0x7faa2c06ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.110+0000 7faa34000640 1 -- 172.21.15.42:0/2535897963 >> 172.21.15.42:0/2535897963 conn(0x7faa2c06bdf0 msgr2=0x7faa2c15bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:30.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.110+0000 7faa34000640 1 -- 172.21.15.42:0/2535897963 shutdown_connections 2024-05-28T14:14:30.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.110+0000 7faa34000640 1 -- 172.21.15.42:0/2535897963 wait complete. 2024-05-28T14:14:30.116 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-28T14:14:30.116 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:14:30.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f461ac00640 1 Processor -- start 2024-05-28T14:14:30.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f461ac00640 1 -- start start 2024-05-28T14:14:30.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f461ac00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f461406f660 0x7f4614073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f461ac00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f46140d0e00 0x7f461406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f461ac00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f461415f280 con 0x7f461415ba40 2024-05-28T14:14:30.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f461ac00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f461415f030 con 0x7f46140d0e00 2024-05-28T14:14:30.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f461ac00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f461415f560 con 0x7f461406f660 2024-05-28T14:14:30.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f4618c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f46140d0e00 0x7f461406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:30.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f4618c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f46140d0e00 0x7f461406dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49820/0 (socket says 172.21.15.42:49820) 2024-05-28T14:14:30.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f4618c00640 1 -- 172.21.15.42:0/4214772524 learned_addr learned my addr 172.21.15.42:0/4214772524 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:30.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f4613e00640 1 --2- 172.21.15.42:0/4214772524 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f461406f660 0x7f4614073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:30.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f4618c00640 1 -- 172.21.15.42:0/4214772524 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f461406f660 msgr2=0x7f4614073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:30.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f4618c00640 1 --2- 172.21.15.42:0/4214772524 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f461406f660 0x7f4614073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f4618c00640 1 -- 172.21.15.42:0/4214772524 >> v1:172.21.15.63:6789/0 conn(0x7f461415ba40 legacy=0x7f46140d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:30.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f4618c00640 1 -- 172.21.15.42:0/4214772524 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461415f840 con 0x7f46140d0e00 2024-05-28T14:14:30.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.270+0000 7f4613e00640 1 --2- 172.21.15.42:0/4214772524 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f461406f660 0x7f4614073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:30.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.274+0000 7f4618c00640 1 --2- 172.21.15.42:0/4214772524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46140d0e00 0x7f461406dd60 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f460c00d150 tx=0x7f460c01eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00fbee3933eedc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:30.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.274+0000 7f4613400640 1 -- 172.21.15.42:0/4214772524 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c00b2d0 con 0x7f46140d0e00 2024-05-28T14:14:30.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.274+0000 7f4613400640 1 -- 172.21.15.42:0/4214772524 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f460c00b470 con 0x7f46140d0e00 2024-05-28T14:14:30.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.274+0000 7f4613400640 1 -- 172.21.15.42:0/4214772524 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c0a7bb0 con 0x7f46140d0e00 2024-05-28T14:14:30.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.274+0000 7f461ac00640 1 -- 172.21.15.42:0/4214772524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46140d0e00 msgr2=0x7f461406dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:30.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.274+0000 7f461ac00640 1 --2- 172.21.15.42:0/4214772524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46140d0e00 0x7f461406dd60 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f460c00d150 tx=0x7f460c01eae0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.274+0000 7f461ac00640 1 -- 172.21.15.42:0/4214772524 shutdown_connections 2024-05-28T14:14:30.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.274+0000 7f461ac00640 1 --2- 172.21.15.42:0/4214772524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46140d0e00 0x7f461406dd60 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.274+0000 7f461ac00640 1 --2- 172.21.15.42:0/4214772524 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f461406f660 0x7f4614073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.274+0000 7f461ac00640 1 -- 172.21.15.42:0/4214772524 >> 172.21.15.42:0/4214772524 conn(0x7f461406bdf0 msgr2=0x7f461406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:30.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.274+0000 7f461ac00640 1 -- 172.21.15.42:0/4214772524 shutdown_connections 2024-05-28T14:14:30.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.274+0000 7f461ac00640 1 -- 172.21.15.42:0/4214772524 wait complete. 2024-05-28T14:14:30.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f461ac00640 1 Processor -- start 2024-05-28T14:14:30.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f461ac00640 1 -- start start 2024-05-28T14:14:30.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f461ac00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f461406f660 0x7f4614146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f461ac00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46140d0e00 0x7f4614149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f461ac00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4614149ff0 0x7f461414cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f461ac00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46140b3460 con 0x7f461406f660 2024-05-28T14:14:30.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f461ac00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46140b32e0 con 0x7f4614149ff0 2024-05-28T14:14:30.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f461ac00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46140b35e0 con 0x7f46140d0e00 2024-05-28T14:14:30.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f4613e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f461406f660 0x7f4614146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:30.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f4613e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f461406f660 0x7f4614146540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49834/0 (socket says 172.21.15.42:49834) 2024-05-28T14:14:30.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f4613e00640 1 -- 172.21.15.42:0/1522517479 learned_addr learned my addr 172.21.15.42:0/1522517479 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:30.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f460b400640 1 --2- 172.21.15.42:0/1522517479 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46140d0e00 0x7f4614149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:30.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f4618c00640 1 --2- 172.21.15.42:0/1522517479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4614149ff0 0x7f461414cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:30.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f4613e00640 1 -- 172.21.15.42:0/1522517479 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46140d0e00 msgr2=0x7f4614149ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:30.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f4613e00640 1 --2- 172.21.15.42:0/1522517479 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46140d0e00 0x7f4614149ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f4613e00640 1 -- 172.21.15.42:0/1522517479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4614149ff0 msgr2=0x7f461414cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:30.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f4613e00640 1 --2- 172.21.15.42:0/1522517479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4614149ff0 0x7f461414cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f4613e00640 1 -- 172.21.15.42:0/1522517479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4614151ba0 con 0x7f461406f660 2024-05-28T14:14:30.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f4613e00640 1 --2- 172.21.15.42:0/1522517479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f461406f660 0x7f4614146540 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f460400e840 tx=0x7f460400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:30.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f4610c00640 1 -- 172.21.15.42:0/1522517479 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604017070 con 0x7f461406f660 2024-05-28T14:14:30.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f4610c00640 1 -- 172.21.15.42:0/1522517479 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f460400c8d0 con 0x7f461406f660 2024-05-28T14:14:30.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f4610c00640 1 -- 172.21.15.42:0/1522517479 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460400f040 con 0x7f461406f660 2024-05-28T14:14:30.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f461ac00640 1 -- 172.21.15.42:0/1522517479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4614151e80 con 0x7f461406f660 2024-05-28T14:14:30.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.278+0000 7f461ac00640 1 -- 172.21.15.42:0/1522517479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f461414f7c0 con 0x7f461406f660 2024-05-28T14:14:30.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.282+0000 7f461ac00640 1 -- 172.21.15.42:0/1522517479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f461414f5b0 con 0x7f461406f660 2024-05-28T14:14:30.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.282+0000 7f4610c00640 1 -- 172.21.15.42:0/1522517479 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f460401a070 con 0x7f461406f660 2024-05-28T14:14:30.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.282+0000 7f4610c00640 1 --2- 172.21.15.42:0/1522517479 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f45f0069fa0 0x7f45f006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.282+0000 7f4610c00640 1 -- 172.21.15.42:0/1522517479 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5879+0+0 (secure 0 0 0) 0x7f4604085480 con 0x7f461406f660 2024-05-28T14:14:30.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.282+0000 7f460b400640 1 --2- 172.21.15.42:0/1522517479 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f45f0069fa0 0x7f45f006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:30.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.282+0000 7f460b400640 1 --2- 172.21.15.42:0/1522517479 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f45f0069fa0 0x7f45f006c460 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f461415b330 tx=0x7f45fc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:30.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.286+0000 7f4610c00640 1 -- 172.21.15.42:0/1522517479 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4604056ef0 con 0x7f461406f660 2024-05-28T14:14:30.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.490+0000 7f461ac00640 1 -- 172.21.15.42:0/1522517479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4614068a90 con 0x7f461406f660 2024-05-28T14:14:30.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.490+0000 7f4610c00640 1 -- 172.21.15.42:0/1522517479 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5180 (secure 0 0 0) 0x7f460405b1f0 con 0x7f461406f660 2024-05-28T14:14:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:30.479 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/56330151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":56330151},{"type":"v1","addr":"172.21.15.42:6835","nonce":56330151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":9183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/192920662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":192920662},{"type":"v1","addr":"172.21.15.63:6835","nonce":192920662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":57437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1225711200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1225711200},{"type":"v1","addr":"172.21.15.42:6837","nonce":1225711200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:12:59.544555+0000","modified":"2024-05-28T14:13:02.565331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57440":{"gid":57440,"name":"b","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/3959626682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3959626682},{"type":"v1","addr":"172.21.15.63:6834","nonce":3959626682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57440,"qdb_cluster":[57440]},"id":8}]} 2024-05-28T14:14:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 75 2024-05-28T14:14:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.494+0000 7f461ac00640 1 -- 172.21.15.42:0/1522517479 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f45f0069fa0 msgr2=0x7f45f006c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.494+0000 7f461ac00640 1 --2- 172.21.15.42:0/1522517479 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f45f0069fa0 0x7f45f006c460 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f461415b330 tx=0x7f45fc009040 comp rx=0 tx=0).stop 2024-05-28T14:14:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.494+0000 7f461ac00640 1 -- 172.21.15.42:0/1522517479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f461406f660 msgr2=0x7f4614146540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.494+0000 7f461ac00640 1 --2- 172.21.15.42:0/1522517479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f461406f660 0x7f4614146540 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f460400e840 tx=0x7f460400ed00 comp rx=0 tx=0).stop 2024-05-28T14:14:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.494+0000 7f461ac00640 1 -- 172.21.15.42:0/1522517479 shutdown_connections 2024-05-28T14:14:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.494+0000 7f461ac00640 1 --2- 172.21.15.42:0/1522517479 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f45f0069fa0 0x7f45f006c460 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.494+0000 7f461ac00640 1 --2- 172.21.15.42:0/1522517479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4614149ff0 0x7f461414cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.494+0000 7f461ac00640 1 --2- 172.21.15.42:0/1522517479 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46140d0e00 0x7f4614149ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.494+0000 7f461ac00640 1 --2- 172.21.15.42:0/1522517479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f461406f660 0x7f4614146540 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.494+0000 7f461ac00640 1 -- 172.21.15.42:0/1522517479 >> 172.21.15.42:0/1522517479 conn(0x7f461406bdf0 msgr2=0x7f461415cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.494+0000 7f461ac00640 1 -- 172.21.15.42:0/1522517479 shutdown_connections 2024-05-28T14:14:30.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.494+0000 7f461ac00640 1 -- 172.21.15.42:0/1522517479 wait complete. 2024-05-28T14:14:30.501 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:14:30.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.654+0000 7f5526600640 1 Processor -- start 2024-05-28T14:14:30.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.654+0000 7f5526600640 1 -- start start 2024-05-28T14:14:30.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.654+0000 7f5526600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f552006f660 0x7f5520073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.654+0000 7f5526600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f552015ba40 0x7f55200d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.654+0000 7f5526600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f552015f280 con 0x7f5520072dd0 2024-05-28T14:14:30.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.654+0000 7f5526600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f552015f030 con 0x7f552006f660 2024-05-28T14:14:30.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.654+0000 7f5526600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f552015f560 con 0x7f552015ba40 2024-05-28T14:14:30.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f5520072dd0 0x7f55200b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36760/0 (socket says 172.21.15.42:36760) 2024-05-28T14:14:30.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551fe00640 1 -- 172.21.15.42:0/1213335939 learned_addr learned my addr 172.21.15.42:0/1213335939 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:30.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f5517e00640 1 --2- 172.21.15.42:0/1213335939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f552015ba40 0x7f55200d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:30.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551f400640 1 --2- 172.21.15.42:0/1213335939 >> v2:172.21.15.63:3300/0 conn(0x7f552006f660 0x7f5520073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:30.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 -- 172.21.15.42:0/1213335939 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 63230085 0 0) 0x7f552015f280 con 0x7f5520072dd0 2024-05-28T14:14:30.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 -- 172.21.15.42:0/1213335939 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5504005740 con 0x7f5520072dd0 2024-05-28T14:14:30.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 -- 172.21.15.42:0/1213335939 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2734986660 0 0) 0x7f5504005740 con 0x7f5520072dd0 2024-05-28T14:14:30.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 -- 172.21.15.42:0/1213335939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f552015ba40 msgr2=0x7f55200d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:30.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 --2- 172.21.15.42:0/1213335939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f552015ba40 0x7f55200d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 -- 172.21.15.42:0/1213335939 >> v2:172.21.15.63:3300/0 conn(0x7f552006f660 msgr2=0x7f5520073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:30.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 --2- 172.21.15.42:0/1213335939 >> v2:172.21.15.63:3300/0 conn(0x7f552006f660 0x7f5520073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 -- 172.21.15.42:0/1213335939 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552015f840 con 0x7f5520072dd0 2024-05-28T14:14:30.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 -- 172.21.15.42:0/1213335939 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f5518001a00 con 0x7f5520072dd0 2024-05-28T14:14:30.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 -- 172.21.15.42:0/1213335939 >> v1:172.21.15.42:6789/0 conn(0x7f5520072dd0 legacy=0x7f55200b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:30.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 --2- 172.21.15.42:0/1213335939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5504005bd0 0x7f5504005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 --2- 172.21.15.42:0/1213335939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5504006530 0x7f5504006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 --2- 172.21.15.42:0/1213335939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5504007110 0x7f550400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 -- 172.21.15.42:0/1213335939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f552015f030 con 0x7f5504007110 2024-05-28T14:14:30.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 -- 172.21.15.42:0/1213335939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f552015f560 con 0x7f5504006530 2024-05-28T14:14:30.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551ea00640 1 -- 172.21.15.42:0/1213335939 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f550400e120 con 0x7f5504005bd0 2024-05-28T14:14:30.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551fe00640 1 --2- 172.21.15.42:0/1213335939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5504007110 0x7f550400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:30.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551f400640 1 --2- 172.21.15.42:0/1213335939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5504005bd0 0x7f5504005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:30.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f5517e00640 1 --2- 172.21.15.42:0/1213335939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5504006530 0x7f5504006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:30.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551fe00640 1 -- 172.21.15.42:0/1213335939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5504005bd0 msgr2=0x7f5504005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:30.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551fe00640 1 --2- 172.21.15.42:0/1213335939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5504005bd0 0x7f5504005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551fe00640 1 -- 172.21.15.42:0/1213335939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5504006530 msgr2=0x7f5504006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:30.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551fe00640 1 --2- 172.21.15.42:0/1213335939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5504006530 0x7f5504006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.658+0000 7f551fe00640 1 -- 172.21.15.42:0/1213335939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5504005740 con 0x7f5504007110 2024-05-28T14:14:30.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.662+0000 7f551fe00640 1 --2- 172.21.15.42:0/1213335939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5504007110 0x7f550400dbe0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f551800a880 tx=0x7f551801d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5cb4577716b1a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:30.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.662+0000 7f551ea00640 1 -- 172.21.15.42:0/1213335939 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551800a4a0 con 0x7f5504007110 2024-05-28T14:14:30.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.662+0000 7f551ea00640 1 -- 172.21.15.42:0/1213335939 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55180ad650 con 0x7f5504007110 2024-05-28T14:14:30.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.662+0000 7f551ea00640 1 -- 172.21.15.42:0/1213335939 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55180ada80 con 0x7f5504007110 2024-05-28T14:14:30.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.662+0000 7f5526600640 1 -- 172.21.15.42:0/1213335939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5504007110 msgr2=0x7f550400dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:30.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.662+0000 7f5526600640 1 --2- 172.21.15.42:0/1213335939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5504007110 0x7f550400dbe0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f551800a880 tx=0x7f551801d370 comp rx=0 tx=0).stop 2024-05-28T14:14:30.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.662+0000 7f551fe00640 1 -- 172.21.15.42:0/1213335939 reap_dead start 2024-05-28T14:14:30.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.662+0000 7f5526600640 1 -- 172.21.15.42:0/1213335939 shutdown_connections 2024-05-28T14:14:30.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.662+0000 7f5526600640 1 -- 172.21.15.42:0/1213335939 >> 172.21.15.42:0/1213335939 conn(0x7f552006bdf0 msgr2=0x7f552006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:30.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.662+0000 7f5526600640 1 -- 172.21.15.42:0/1213335939 shutdown_connections 2024-05-28T14:14:30.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.662+0000 7f5526600640 1 -- 172.21.15.42:0/1213335939 wait complete. 2024-05-28T14:14:30.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.662+0000 7f5526600640 1 Processor -- start 2024-05-28T14:14:30.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.662+0000 7f5526600640 1 -- start start 2024-05-28T14:14:30.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5526600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5504007110 0x7f552014a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5526600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5520160b50 0x7f5520148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5526600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f552014cac0 0x7f552014cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5526600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55200d0870 con 0x7f5520160b50 2024-05-28T14:14:30.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5526600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f55200d06f0 con 0x7f5504007110 2024-05-28T14:14:30.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5526600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f55200d09f0 con 0x7f552014cac0 2024-05-28T14:14:30.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5517e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5520160b50 0x7f5520148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:30.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5517e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5520160b50 0x7f5520148220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49858/0 (socket says 172.21.15.42:49858) 2024-05-28T14:14:30.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5517e00640 1 -- 172.21.15.42:0/3832889704 learned_addr learned my addr 172.21.15.42:0/3832889704 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:30.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5517e00640 1 -- 172.21.15.42:0/3832889704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f552014cac0 msgr2=0x7f552014cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:30.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5517e00640 1 --2- 172.21.15.42:0/3832889704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f552014cac0 0x7f552014cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5517e00640 1 -- 172.21.15.42:0/3832889704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5504007110 msgr2=0x7f552014a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:30.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5517e00640 1 --2- 172.21.15.42:0/3832889704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5504007110 0x7f552014a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5517e00640 1 -- 172.21.15.42:0/3832889704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55200b4670 con 0x7f5520160b50 2024-05-28T14:14:30.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5517e00640 1 --2- 172.21.15.42:0/3832889704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5520160b50 0x7f5520148220 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f5508011640 tx=0x7f5508011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:30.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5517400640 1 -- 172.21.15.42:0/3832889704 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508015870 con 0x7f5520160b50 2024-05-28T14:14:30.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5517400640 1 -- 172.21.15.42:0/3832889704 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5508015a10 con 0x7f5520160b50 2024-05-28T14:14:30.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5517400640 1 -- 172.21.15.42:0/3832889704 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508015dc0 con 0x7f5520160b50 2024-05-28T14:14:30.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5526600640 1 -- 172.21.15.42:0/3832889704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55201529f0 con 0x7f5520160b50 2024-05-28T14:14:30.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5526600640 1 -- 172.21.15.42:0/3832889704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552014ad70 con 0x7f5520160b50 2024-05-28T14:14:30.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5526600640 1 -- 172.21.15.42:0/3832889704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5520068a90 con 0x7f5520160b50 2024-05-28T14:14:30.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.666+0000 7f5517400640 1 -- 172.21.15.42:0/3832889704 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f550800f430 con 0x7f5520160b50 2024-05-28T14:14:30.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.670+0000 7f5517400640 1 --2- 172.21.15.42:0/3832889704 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f54fc069fa0 0x7f54fc06c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:30.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.670+0000 7f5517400640 1 -- 172.21.15.42:0/3832889704 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5879+0+0 (secure 0 0 0) 0x7f5508086180 con 0x7f5520160b50 2024-05-28T14:14:30.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.670+0000 7f551f400640 1 --2- 172.21.15.42:0/3832889704 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f54fc069fa0 0x7f54fc06c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.670+0000 7f551f400640 1 --2- 172.21.15.42:0/3832889704 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f54fc069fa0 0x7f54fc06c460 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5510006b20 tx=0x7f5510006ab0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:30.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.674+0000 7f5517400640 1 -- 172.21.15.42:0/3832889704 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5508019030 con 0x7f5520160b50 2024-05-28T14:14:30.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.874+0000 7f5526600640 1 -- 172.21.15.42:0/3832889704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f552015ed20 con 0x7f5520160b50 2024-05-28T14:14:30.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.874+0000 7f5517400640 1 -- 172.21.15.42:0/3832889704 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5180 (secure 0 0 0) 0x7f5508058240 con 0x7f5520160b50 2024-05-28T14:14:30.861 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:30.862 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/56330151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":56330151},{"type":"v1","addr":"172.21.15.42:6835","nonce":56330151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":9183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/192920662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":192920662},{"type":"v1","addr":"172.21.15.63:6835","nonce":192920662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":57437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1225711200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1225711200},{"type":"v1","addr":"172.21.15.42:6837","nonce":1225711200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:12:59.544555+0000","modified":"2024-05-28T14:13:02.565331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57440":{"gid":57440,"name":"b","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/3959626682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3959626682},{"type":"v1","addr":"172.21.15.63:6834","nonce":3959626682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57440,"qdb_cluster":[57440]},"id":8}]} 2024-05-28T14:14:30.862 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 75 2024-05-28T14:14:30.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.878+0000 7f5526600640 1 -- 172.21.15.42:0/3832889704 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f54fc069fa0 msgr2=0x7f54fc06c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:30.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.878+0000 7f5526600640 1 --2- 172.21.15.42:0/3832889704 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f54fc069fa0 0x7f54fc06c460 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5510006b20 tx=0x7f5510006ab0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.878+0000 7f5526600640 1 -- 172.21.15.42:0/3832889704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5520160b50 msgr2=0x7f5520148220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:30.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.878+0000 7f5526600640 1 --2- 172.21.15.42:0/3832889704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5520160b50 0x7f5520148220 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f5508011640 tx=0x7f5508011b00 comp rx=0 tx=0).stop 2024-05-28T14:14:30.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.878+0000 7f5526600640 1 -- 172.21.15.42:0/3832889704 shutdown_connections 2024-05-28T14:14:30.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.878+0000 7f5526600640 1 --2- 172.21.15.42:0/3832889704 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f54fc069fa0 0x7f54fc06c460 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.878+0000 7f5526600640 1 --2- 172.21.15.42:0/3832889704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f552014cac0 0x7f552014cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.878+0000 7f5526600640 1 --2- 172.21.15.42:0/3832889704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5520160b50 0x7f5520148220 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.878+0000 7f5526600640 1 --2- 172.21.15.42:0/3832889704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5504007110 0x7f552014a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:30.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.878+0000 7f5526600640 1 -- 172.21.15.42:0/3832889704 >> 172.21.15.42:0/3832889704 conn(0x7f552006bdf0 msgr2=0x7f552014ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:30.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.878+0000 7f5526600640 1 -- 172.21.15.42:0/3832889704 shutdown_connections 2024-05-28T14:14:30.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:30.878+0000 7f5526600640 1 -- 172.21.15.42:0/3832889704 wait complete. 2024-05-28T14:14:30.883 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:14:31.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.038+0000 7fa7a7600640 1 Processor -- start 2024-05-28T14:14:31.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.038+0000 7fa7a7600640 1 -- start start 2024-05-28T14:14:31.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.038+0000 7fa7a7600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa7a0162c80 0x7fa7a015e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.038+0000 7fa7a7600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7a00d0de0 con 0x7fa7a00ce860 2024-05-28T14:14:31.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.038+0000 7fa7a7600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7a00d10c0 con 0x7fa7a00d0900 2024-05-28T14:14:31.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.038+0000 7fa7a7600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa7a00d19f0 con 0x7fa7a0162c80 2024-05-28T14:14:31.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.038+0000 7fa7a4c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa7a00ce860 0x7fa7a006d7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36762/0 (socket says 172.21.15.42:36762) 2024-05-28T14:14:31.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.038+0000 7fa7a4c00640 1 -- 172.21.15.42:0/2794772704 learned_addr learned my addr 172.21.15.42:0/2794772704 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:31.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.038+0000 7fa7a5600640 1 --2- 172.21.15.42:0/2794772704 >> v2:172.21.15.63:3300/0 conn(0x7fa7a0162c80 0x7fa7a015e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.038+0000 7fa79fe00640 1 -- 172.21.15.42:0/2794772704 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 771172813 0 0) 0x7fa7a00d0de0 con 0x7fa7a00ce860 2024-05-28T14:14:31.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.038+0000 7fa79fe00640 1 -- 172.21.15.42:0/2794772704 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa784003530 con 0x7fa7a00ce860 2024-05-28T14:14:31.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.038+0000 7fa7a5600640 1 -- 172.21.15.42:0/2794772704 >> v1:172.21.15.63:6789/0 conn(0x7fa7a00d0900 legacy=0x7fa7a0162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.038+0000 7fa7a5600640 1 -- 172.21.15.42:0/2794772704 >> v1:172.21.15.42:6789/0 conn(0x7fa7a00ce860 legacy=0x7fa7a006d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.038+0000 7fa7a5600640 1 -- 172.21.15.42:0/2794772704 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a00d13a0 con 0x7fa7a0162c80 2024-05-28T14:14:31.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.042+0000 7fa7a5600640 1 --2- 172.21.15.42:0/2794772704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7a0162c80 0x7fa7a015e200 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa798009d90 tx=0x7fa79801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9c194bc6114d36e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:31.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.042+0000 7fa79fe00640 1 -- 172.21.15.42:0/2794772704 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980b2070 con 0x7fa7a0162c80 2024-05-28T14:14:31.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.042+0000 7fa79fe00640 1 -- 172.21.15.42:0/2794772704 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7980a5890 con 0x7fa7a0162c80 2024-05-28T14:14:31.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.042+0000 7fa79fe00640 1 -- 172.21.15.42:0/2794772704 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980a5c40 con 0x7fa7a0162c80 2024-05-28T14:14:31.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.042+0000 7fa7a7600640 1 -- 172.21.15.42:0/2794772704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7a0162c80 msgr2=0x7fa7a015e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.042+0000 7fa7a7600640 1 --2- 172.21.15.42:0/2794772704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7a0162c80 0x7fa7a015e200 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa798009d90 tx=0x7fa79801c8b0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.042+0000 7fa7a7600640 1 -- 172.21.15.42:0/2794772704 shutdown_connections 2024-05-28T14:14:31.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.042+0000 7fa7a7600640 1 --2- 172.21.15.42:0/2794772704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7a0162c80 0x7fa7a015e200 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.042+0000 7fa7a7600640 1 -- 172.21.15.42:0/2794772704 >> 172.21.15.42:0/2794772704 conn(0x7fa7a00b4080 msgr2=0x7fa7a00b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:31.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.042+0000 7fa7a7600640 1 -- 172.21.15.42:0/2794772704 shutdown_connections 2024-05-28T14:14:31.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.042+0000 7fa7a7600640 1 -- 172.21.15.42:0/2794772704 wait complete. 2024-05-28T14:14:31.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a7600640 1 Processor -- start 2024-05-28T14:14:31.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a7600640 1 -- start start 2024-05-28T14:14:31.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a7600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7a00ce860 0x7fa7a0141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a7600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7a00d0900 0x7fa7a00c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a7600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a0162c80 0x7fa7a00707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a7600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a016ca40 con 0x7fa7a0162c80 2024-05-28T14:14:31.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a7600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a016c8c0 con 0x7fa7a00ce860 2024-05-28T14:14:31.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a7600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a016cbc0 con 0x7fa7a00d0900 2024-05-28T14:14:31.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a0162c80 0x7fa7a00707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a0162c80 0x7fa7a00707e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49872/0 (socket says 172.21.15.42:49872) 2024-05-28T14:14:31.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a5600640 1 -- 172.21.15.42:0/287170952 learned_addr learned my addr 172.21.15.42:0/287170952 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:31.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a4c00640 1 --2- 172.21.15.42:0/287170952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7a00ce860 0x7fa7a0141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa797e00640 1 --2- 172.21.15.42:0/287170952 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7a00d0900 0x7fa7a00c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a5600640 1 -- 172.21.15.42:0/287170952 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7a00d0900 msgr2=0x7fa7a00c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a5600640 1 --2- 172.21.15.42:0/287170952 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7a00d0900 0x7fa7a00c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a5600640 1 -- 172.21.15.42:0/287170952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7a00ce860 msgr2=0x7fa7a0141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a5600640 1 --2- 172.21.15.42:0/287170952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7a00ce860 0x7fa7a0141ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a5600640 1 -- 172.21.15.42:0/287170952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a00cb4e0 con 0x7fa7a0162c80 2024-05-28T14:14:31.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a5600640 1 --2- 172.21.15.42:0/287170952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a0162c80 0x7fa7a00707e0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fa7980b0040 tx=0x7fa798019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:31.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa79d600640 1 -- 172.21.15.42:0/287170952 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980b2050 con 0x7fa7a0162c80 2024-05-28T14:14:31.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa79d600640 1 -- 172.21.15.42:0/287170952 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7980b3040 con 0x7fa7a0162c80 2024-05-28T14:14:31.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a7600640 1 -- 172.21.15.42:0/287170952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a00cb780 con 0x7fa7a0162c80 2024-05-28T14:14:31.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa79d600640 1 -- 172.21.15.42:0/287170952 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798003bb0 con 0x7fa7a0162c80 2024-05-28T14:14:31.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.046+0000 7fa7a7600640 1 -- 172.21.15.42:0/287170952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a00cc020 con 0x7fa7a0162c80 2024-05-28T14:14:31.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.050+0000 7fa7a7600640 1 -- 172.21.15.42:0/287170952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a00cbb70 con 0x7fa7a0162c80 2024-05-28T14:14:31.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.050+0000 7fa79d600640 1 -- 172.21.15.42:0/287170952 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7fa7980c0020 con 0x7fa7a0162c80 2024-05-28T14:14:31.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.050+0000 7fa79d600640 1 --2- 172.21.15.42:0/287170952 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa77c069fa0 0x7fa77c06c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.050+0000 7fa79d600640 1 -- 172.21.15.42:0/287170952 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5879+0+0 (secure 0 0 0) 0x7fa79811ffb0 con 0x7fa7a0162c80 2024-05-28T14:14:31.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.050+0000 7fa7a4c00640 1 --2- 172.21.15.42:0/287170952 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa77c069fa0 0x7fa77c06c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.050+0000 7fa7a4c00640 1 --2- 172.21.15.42:0/287170952 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa77c069fa0 0x7fa77c06c460 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa7900011f0 tx=0x7fa790007d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:31.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.054+0000 7fa79d600640 1 -- 172.21.15.42:0/287170952 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa798016560 con 0x7fa7a0162c80 2024-05-28T14:14:31.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.222+0000 7fa7a7600640 1 -- 172.21.15.42:0/287170952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7a0068a90 con 0x7fa7a0162c80 2024-05-28T14:14:31.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.222+0000 7fa79d600640 1 -- 172.21.15.42:0/287170952 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+16921 (secure 0 0 0) 0x7fa7980f1fd0 con 0x7fa7a0162c80 2024-05-28T14:14:31.210 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:31.210 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":120,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:14:26.034186+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:12:57.936312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-28T14:12:58.495752+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":120,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.24","mappings":[{"from":0,"to":5}]},{"pgid":"17.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:14:31.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.226+0000 7fa7a7600640 1 -- 172.21.15.42:0/287170952 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa77c069fa0 msgr2=0x7fa77c06c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.226+0000 7fa7a7600640 1 --2- 172.21.15.42:0/287170952 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa77c069fa0 0x7fa77c06c460 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa7900011f0 tx=0x7fa790007d40 comp rx=0 tx=0).stop 2024-05-28T14:14:31.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.226+0000 7fa7a7600640 1 -- 172.21.15.42:0/287170952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a0162c80 msgr2=0x7fa7a00707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.226+0000 7fa7a7600640 1 --2- 172.21.15.42:0/287170952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a0162c80 0x7fa7a00707e0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fa7980b0040 tx=0x7fa798019650 comp rx=0 tx=0).stop 2024-05-28T14:14:31.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.226+0000 7fa7a7600640 1 -- 172.21.15.42:0/287170952 shutdown_connections 2024-05-28T14:14:31.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.226+0000 7fa7a7600640 1 --2- 172.21.15.42:0/287170952 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa77c069fa0 0x7fa77c06c460 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.226+0000 7fa7a7600640 1 --2- 172.21.15.42:0/287170952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a0162c80 0x7fa7a00707e0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.226+0000 7fa7a7600640 1 --2- 172.21.15.42:0/287170952 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7a00d0900 0x7fa7a00c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.226+0000 7fa7a7600640 1 --2- 172.21.15.42:0/287170952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7a00ce860 0x7fa7a0141ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.226+0000 7fa7a7600640 1 -- 172.21.15.42:0/287170952 >> 172.21.15.42:0/287170952 conn(0x7fa7a00b4080 msgr2=0x7fa7a0159450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:31.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.226+0000 7fa7a7600640 1 -- 172.21.15.42:0/287170952 shutdown_connections 2024-05-28T14:14:31.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.226+0000 7fa7a7600640 1 -- 172.21.15.42:0/287170952 wait complete. 2024-05-28T14:14:31.231 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-28T14:14:31.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.386+0000 7fe970400640 1 Processor -- start 2024-05-28T14:14:31.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.386+0000 7fe970400640 1 -- start start 2024-05-28T14:14:31.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe970400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe968155a00 0x7fe96806ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe970400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe96806cf40 0x7fe96806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe970400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe96806f3f0 0x7fe9681623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe970400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fe96806d380 con 0x7fe96806cf40 2024-05-28T14:14:31.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe970400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe96806fc20 con 0x7fe968155a00 2024-05-28T14:14:31.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe970400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe96815ff90 con 0x7fe96806f3f0 2024-05-28T14:14:31.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe96d000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe96806cf40 0x7fe96806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe96d000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe96806cf40 0x7fe96806eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49886/0 (socket says 172.21.15.42:49886) 2024-05-28T14:14:31.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe96d000640 1 -- 172.21.15.42:0/4108621434 learned_addr learned my addr 172.21.15.42:0/4108621434 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:31.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe96da00640 1 --2- 172.21.15.42:0/4108621434 >> v2:172.21.15.63:3300/0 conn(0x7fe968155a00 0x7fe96806ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe96e400640 1 --2- 172.21.15.42:0/4108621434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe96806f3f0 0x7fe9681623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe96d000640 1 -- 172.21.15.42:0/4108621434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe96806f3f0 msgr2=0x7fe9681623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:31.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe96d000640 1 --2- 172.21.15.42:0/4108621434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe96806f3f0 0x7fe9681623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe96d000640 1 -- 172.21.15.42:0/4108621434 >> v2:172.21.15.63:3300/0 conn(0x7fe968155a00 msgr2=0x7fe96806ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:31.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe96d000640 1 --2- 172.21.15.42:0/4108621434 >> v2:172.21.15.63:3300/0 conn(0x7fe968155a00 0x7fe96806ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe96d000640 1 -- 172.21.15.42:0/4108621434 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe968160270 con 0x7fe96806cf40 2024-05-28T14:14:31.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.390+0000 7fe96d000640 1 --2- 172.21.15.42:0/4108621434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe96806cf40 0x7fe96806eeb0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe968072240 tx=0x7fe95401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d0c9317d50ddec server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:31.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe967e00640 1 -- 172.21.15.42:0/4108621434 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9540b0070 con 0x7fe96806cf40 2024-05-28T14:14:31.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe967e00640 1 -- 172.21.15.42:0/4108621434 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9540ab600 con 0x7fe96806cf40 2024-05-28T14:14:31.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe967e00640 1 -- 172.21.15.42:0/4108621434 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9540ab9d0 con 0x7fe96806cf40 2024-05-28T14:14:31.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 -- 172.21.15.42:0/4108621434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe96806cf40 msgr2=0x7fe96806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 --2- 172.21.15.42:0/4108621434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe96806cf40 0x7fe96806eeb0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe968072240 tx=0x7fe95401c260 comp rx=0 tx=0).stop 2024-05-28T14:14:31.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 -- 172.21.15.42:0/4108621434 shutdown_connections 2024-05-28T14:14:31.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 --2- 172.21.15.42:0/4108621434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe96806f3f0 0x7fe9681623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 --2- 172.21.15.42:0/4108621434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe96806cf40 0x7fe96806eeb0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 --2- 172.21.15.42:0/4108621434 >> v2:172.21.15.63:3300/0 conn(0x7fe968155a00 0x7fe96806ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 -- 172.21.15.42:0/4108621434 >> 172.21.15.42:0/4108621434 conn(0x7fe9680725e0 msgr2=0x7fe9680729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:31.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 -- 172.21.15.42:0/4108621434 shutdown_connections 2024-05-28T14:14:31.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 -- 172.21.15.42:0/4108621434 wait complete. 2024-05-28T14:14:31.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 Processor -- start 2024-05-28T14:14:31.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 -- start start 2024-05-28T14:14:31.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe96806cf40 0x7fe9680af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe96806f3f0 0x7fe9680cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe968155a00 0x7fe9681faac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe968160ed0 con 0x7fe96806cf40 2024-05-28T14:14:31.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe968160d50 con 0x7fe968155a00 2024-05-28T14:14:31.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe970400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe968161050 con 0x7fe96806f3f0 2024-05-28T14:14:31.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe96da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe96806cf40 0x7fe9680af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe96da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe96806cf40 0x7fe9680af0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49902/0 (socket says 172.21.15.42:49902) 2024-05-28T14:14:31.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.394+0000 7fe96da00640 1 -- 172.21.15.42:0/318200132 learned_addr learned my addr 172.21.15.42:0/318200132 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:31.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe96e400640 1 --2- 172.21.15.42:0/318200132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe968155a00 0x7fe9681faac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe96d000640 1 --2- 172.21.15.42:0/318200132 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe96806f3f0 0x7fe9680cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe96da00640 1 -- 172.21.15.42:0/318200132 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe96806f3f0 msgr2=0x7fe9680cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe96da00640 1 --2- 172.21.15.42:0/318200132 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe96806f3f0 0x7fe9680cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe96da00640 1 -- 172.21.15.42:0/318200132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe968155a00 msgr2=0x7fe9681faac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe96da00640 1 --2- 172.21.15.42:0/318200132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe968155a00 0x7fe9681faac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe96da00640 1 -- 172.21.15.42:0/318200132 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe968150380 con 0x7fe96806cf40 2024-05-28T14:14:31.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe96da00640 1 --2- 172.21.15.42:0/318200132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe96806cf40 0x7fe9680af0a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00e870 tx=0x7fe95c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:31.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe965600640 1 -- 172.21.15.42:0/318200132 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c017070 con 0x7fe96806cf40 2024-05-28T14:14:31.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe965600640 1 -- 172.21.15.42:0/318200132 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe95c00c940 con 0x7fe96806cf40 2024-05-28T14:14:31.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe970400640 1 -- 172.21.15.42:0/318200132 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9680cf0f0 con 0x7fe96806cf40 2024-05-28T14:14:31.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe965600640 1 -- 172.21.15.42:0/318200132 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00ccf0 con 0x7fe96806cf40 2024-05-28T14:14:31.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe970400640 1 -- 172.21.15.42:0/318200132 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9681500a0 con 0x7fe96806cf40 2024-05-28T14:14:31.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe970400640 1 -- 172.21.15.42:0/318200132 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe968068310 con 0x7fe96806cf40 2024-05-28T14:14:31.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe965600640 1 -- 172.21.15.42:0/318200132 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7fe95c01a070 con 0x7fe96806cf40 2024-05-28T14:14:31.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe965600640 1 --2- 172.21.15.42:0/318200132 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe948069fa0 0x7fe94806c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.398+0000 7fe965600640 1 -- 172.21.15.42:0/318200132 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5879+0+0 (secure 0 0 0) 0x7fe95c085820 con 0x7fe96806cf40 2024-05-28T14:14:31.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.402+0000 7fe96d000640 1 --2- 172.21.15.42:0/318200132 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe948069fa0 0x7fe94806c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.402+0000 7fe96d000640 1 --2- 172.21.15.42:0/318200132 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe948069fa0 0x7fe94806c460 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe9540ad000 tx=0x7fe954008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:31.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.402+0000 7fe965600640 1 -- 172.21.15.42:0/318200132 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe95c057840 con 0x7fe96806cf40 2024-05-28T14:14:31.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.606+0000 7fe970400640 1 -- 172.21.15.42:0/318200132 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe968073f20 con 0x7fe96806cf40 2024-05-28T14:14:31.593 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-05-28T14:14:31.606+0000 7f248b600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-28T14:14:31.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.606+0000 7fe965600640 1 -- 172.21.15.42:0/318200132 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) ==== 161+0+0 (secure 0 0 0) 0x7fe95c057140 con 0x7fe96806cf40 2024-05-28T14:14:31.595 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-28T14:14:31.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.610+0000 7fe970400640 1 -- 172.21.15.42:0/318200132 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe948069fa0 msgr2=0x7fe94806c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.610+0000 7fe970400640 1 --2- 172.21.15.42:0/318200132 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe948069fa0 0x7fe94806c460 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe9540ad000 tx=0x7fe954008040 comp rx=0 tx=0).stop 2024-05-28T14:14:31.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.610+0000 7fe970400640 1 -- 172.21.15.42:0/318200132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe96806cf40 msgr2=0x7fe9680af0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.610+0000 7fe970400640 1 --2- 172.21.15.42:0/318200132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe96806cf40 0x7fe9680af0a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00e870 tx=0x7fe95c00ec70 comp rx=0 tx=0).stop 2024-05-28T14:14:31.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.610+0000 7fe970400640 1 -- 172.21.15.42:0/318200132 shutdown_connections 2024-05-28T14:14:31.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.610+0000 7fe970400640 1 --2- 172.21.15.42:0/318200132 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe948069fa0 0x7fe94806c460 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.610+0000 7fe970400640 1 --2- 172.21.15.42:0/318200132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe968155a00 0x7fe9681faac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.610+0000 7fe970400640 1 --2- 172.21.15.42:0/318200132 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe96806f3f0 0x7fe9680cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.610+0000 7fe970400640 1 --2- 172.21.15.42:0/318200132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe96806cf40 0x7fe9680af0a0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.610+0000 7fe970400640 1 -- 172.21.15.42:0/318200132 >> 172.21.15.42:0/318200132 conn(0x7fe9680725e0 msgr2=0x7fe96806c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:31.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.610+0000 7fe970400640 1 -- 172.21.15.42:0/318200132 shutdown_connections 2024-05-28T14:14:31.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.610+0000 7fe970400640 1 -- 172.21.15.42:0/318200132 wait complete. 2024-05-28T14:14:31.618 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-28T14:14:31.669 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T14:14:31.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.774+0000 7f0378800640 1 Processor -- start 2024-05-28T14:14:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.774+0000 7f0378800640 1 -- start start 2024-05-28T14:14:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.778+0000 7f0378800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f037006d3e0 0x7f03701603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.778+0000 7f0378800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f03700d0d40 0x7f037009b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.778+0000 7f0378800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03700d11f0 con 0x7f037006d7c0 2024-05-28T14:14:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.778+0000 7f0378800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f037009ac80 con 0x7f03700d0d40 2024-05-28T14:14:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.778+0000 7f0378800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f03700d14d0 con 0x7f037006d3e0 2024-05-28T14:14:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.778+0000 7f0376800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f03700d0d40 0x7f037009b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.778+0000 7f0376800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f03700d0d40 0x7f037009b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49916/0 (socket says 172.21.15.42:49916) 2024-05-28T14:14:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.778+0000 7f0376800640 1 -- 172.21.15.42:0/13636379 learned_addr learned my addr 172.21.15.42:0/13636379 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.778+0000 7f0375e00640 1 --2- 172.21.15.42:0/13636379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f037006d3e0 0x7f03701603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.778+0000 7f0376800640 1 -- 172.21.15.42:0/13636379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f037006d3e0 msgr2=0x7f03701603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:31.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.778+0000 7f0376800640 1 --2- 172.21.15.42:0/13636379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f037006d3e0 0x7f03701603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.778+0000 7f0376800640 1 -- 172.21.15.42:0/13636379 >> v1:172.21.15.63:6789/0 conn(0x7f037006d7c0 legacy=0x7f03700d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.778+0000 7f0376800640 1 -- 172.21.15.42:0/13636379 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03700d17b0 con 0x7f03700d0d40 2024-05-28T14:14:31.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.778+0000 7f0375e00640 1 --2- 172.21.15.42:0/13636379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f037006d3e0 0x7f03701603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:31.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0376800640 1 --2- 172.21.15.42:0/13636379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03700d0d40 0x7f037009b110 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f036c009e50 tx=0x7f036c01d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebfccf0e952eea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:31.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0374a00640 1 -- 172.21.15.42:0/13636379 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037009be60 con 0x7f03700d0d40 2024-05-28T14:14:31.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0374a00640 1 -- 172.21.15.42:0/13636379 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f036c0a5810 con 0x7f03700d0d40 2024-05-28T14:14:31.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0374a00640 1 -- 172.21.15.42:0/13636379 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c0a5be0 con 0x7f03700d0d40 2024-05-28T14:14:31.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0378800640 1 -- 172.21.15.42:0/13636379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03700d0d40 msgr2=0x7f037009b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0378800640 1 --2- 172.21.15.42:0/13636379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03700d0d40 0x7f037009b110 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f036c009e50 tx=0x7f036c01d490 comp rx=0 tx=0).stop 2024-05-28T14:14:31.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0378800640 1 -- 172.21.15.42:0/13636379 shutdown_connections 2024-05-28T14:14:31.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0378800640 1 --2- 172.21.15.42:0/13636379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03700d0d40 0x7f037009b110 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0378800640 1 --2- 172.21.15.42:0/13636379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f037006d3e0 0x7f03701603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0378800640 1 -- 172.21.15.42:0/13636379 >> 172.21.15.42:0/13636379 conn(0x7f03700b3f90 msgr2=0x7f03700b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:31.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0378800640 1 -- 172.21.15.42:0/13636379 shutdown_connections 2024-05-28T14:14:31.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0378800640 1 -- 172.21.15.42:0/13636379 wait complete. 2024-05-28T14:14:31.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0378800640 1 Processor -- start 2024-05-28T14:14:31.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0378800640 1 -- start start 2024-05-28T14:14:31.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.782+0000 7f0378800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f037006d3e0 0x7f0370155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0378800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f037006d7c0 0x7f03700c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0378800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03700d0d40 0x7f0370156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0378800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f037016b9f0 con 0x7f037006d3e0 2024-05-28T14:14:31.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0378800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f037016b870 con 0x7f03700d0d40 2024-05-28T14:14:31.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0378800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f037016bb70 con 0x7f037006d7c0 2024-05-28T14:14:31.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0375e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f037006d3e0 0x7f0370155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0375e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f037006d3e0 0x7f0370155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49932/0 (socket says 172.21.15.42:49932) 2024-05-28T14:14:31.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0375e00640 1 -- 172.21.15.42:0/3848856055 learned_addr learned my addr 172.21.15.42:0/3848856055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:31.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0375400640 1 --2- 172.21.15.42:0/3848856055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f037006d7c0 0x7f03700c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0376800640 1 --2- 172.21.15.42:0/3848856055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03700d0d40 0x7f0370156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0375e00640 1 -- 172.21.15.42:0/3848856055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f037006d7c0 msgr2=0x7f03700c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0375e00640 1 --2- 172.21.15.42:0/3848856055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f037006d7c0 0x7f03700c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0375e00640 1 -- 172.21.15.42:0/3848856055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03700d0d40 msgr2=0x7f0370156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:31.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0375e00640 1 --2- 172.21.15.42:0/3848856055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03700d0d40 0x7f0370156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:31.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0375e00640 1 -- 172.21.15.42:0/3848856055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0370158680 con 0x7f037006d3e0 2024-05-28T14:14:31.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0375e00640 1 --2- 172.21.15.42:0/3848856055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f037006d3e0 0x7f0370155610 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f036400d830 tx=0x7f036400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:31.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f036a000640 1 -- 172.21.15.42:0/3848856055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364018020 con 0x7f037006d3e0 2024-05-28T14:14:31.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f036a000640 1 -- 172.21.15.42:0/3848856055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0364009660 con 0x7f037006d3e0 2024-05-28T14:14:31.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0378800640 1 -- 172.21.15.42:0/3848856055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03700c90b0 con 0x7f037006d3e0 2024-05-28T14:14:31.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f036a000640 1 -- 172.21.15.42:0/3848856055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364009a50 con 0x7f037006d3e0 2024-05-28T14:14:31.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0378800640 1 -- 172.21.15.42:0/3848856055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03701583a0 con 0x7f037006d3e0 2024-05-28T14:14:31.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f0378800640 1 -- 172.21.15.42:0/3848856055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03700a0550 con 0x7f037006d3e0 2024-05-28T14:14:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.786+0000 7f036a000640 1 -- 172.21.15.42:0/3848856055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f036401b020 con 0x7f037006d3e0 2024-05-28T14:14:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.790+0000 7f036a000640 1 --2- 172.21.15.42:0/3848856055 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0354069fd0 0x7f035406c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.790+0000 7f036a000640 1 -- 172.21.15.42:0/3848856055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5937+0+0 (secure 0 0 0) 0x7f0364085dd0 con 0x7f037006d3e0 2024-05-28T14:14:31.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.790+0000 7f0375400640 1 --2- 172.21.15.42:0/3848856055 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0354069fd0 0x7f035406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:31.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.790+0000 7f0375400640 1 --2- 172.21.15.42:0/3848856055 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0354069fd0 0x7f035406c490 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0370154e20 tx=0x7f0360009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:31.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.794+0000 7f036a000640 1 -- 172.21.15.42:0/3848856055 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0364022020 con 0x7f037006d3e0 2024-05-28T14:14:31.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:31.990+0000 7f0378800640 1 -- 172.21.15.42:0/3848856055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f037004fdd0 con 0x7f037006d3e0 2024-05-28T14:14:32.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.610+0000 7f036a000640 1 -- 172.21.15.42:0/3848856055 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) ==== 104+0+0 (secure 0 0 0) 0x7f0364057df0 con 0x7f037006d3e0 2024-05-28T14:14:32.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.610+0000 7f0378800640 1 -- 172.21.15.42:0/3848856055 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0354069fd0 msgr2=0x7f035406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:32.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.610+0000 7f0378800640 1 --2- 172.21.15.42:0/3848856055 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0354069fd0 0x7f035406c490 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0370154e20 tx=0x7f0360009040 comp rx=0 tx=0).stop 2024-05-28T14:14:32.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.610+0000 7f0378800640 1 -- 172.21.15.42:0/3848856055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f037006d3e0 msgr2=0x7f0370155610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:32.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.610+0000 7f0378800640 1 --2- 172.21.15.42:0/3848856055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f037006d3e0 0x7f0370155610 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f036400d830 tx=0x7f036400dcf0 comp rx=0 tx=0).stop 2024-05-28T14:14:32.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.610+0000 7f0378800640 1 -- 172.21.15.42:0/3848856055 shutdown_connections 2024-05-28T14:14:32.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.610+0000 7f0378800640 1 --2- 172.21.15.42:0/3848856055 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0354069fd0 0x7f035406c490 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:32.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.610+0000 7f0378800640 1 --2- 172.21.15.42:0/3848856055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03700d0d40 0x7f0370156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:32.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.610+0000 7f0378800640 1 --2- 172.21.15.42:0/3848856055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f037006d7c0 0x7f03700c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:32.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.610+0000 7f0378800640 1 --2- 172.21.15.42:0/3848856055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f037006d3e0 0x7f0370155610 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:32.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.610+0000 7f0378800640 1 -- 172.21.15.42:0/3848856055 >> 172.21.15.42:0/3848856055 conn(0x7f03700b3f90 msgr2=0x7f0370154310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:32.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.614+0000 7f0378800640 1 -- 172.21.15.42:0/3848856055 shutdown_connections 2024-05-28T14:14:32.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.614+0000 7f0378800640 1 -- 172.21.15.42:0/3848856055 wait complete. 2024-05-28T14:14:32.621 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-28T14:14:32.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f768aa00640 1 Processor -- start 2024-05-28T14:14:32.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f768aa00640 1 -- start start 2024-05-28T14:14:32.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f768aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0e00 0x7f768406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:32.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f768aa00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7684072d40 0x7f76840aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f768aa00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76840ade40 con 0x7f768415b350 2024-05-28T14:14:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f768aa00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f76840ae9f0 con 0x7f7684072d40 2024-05-28T14:14:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f768aa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76840ae0c0 con 0x7f76840d0e00 2024-05-28T14:14:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f7688a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7684072d40 0x7f76840aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f7688a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7684072d40 0x7f76840aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49948/0 (socket says 172.21.15.42:49948) 2024-05-28T14:14:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f7688a00640 1 -- 172.21.15.42:0/947365372 learned_addr learned my addr 172.21.15.42:0/947365372 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f767b400640 1 --2- 172.21.15.42:0/947365372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0e00 0x7f768406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f7688a00640 1 -- 172.21.15.42:0/947365372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0e00 msgr2=0x7f768406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f7688a00640 1 --2- 172.21.15.42:0/947365372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0e00 0x7f768406dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f7688a00640 1 -- 172.21.15.42:0/947365372 >> v1:172.21.15.63:6789/0 conn(0x7f768415b350 legacy=0x7f76840d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.782+0000 7f7688a00640 1 -- 172.21.15.42:0/947365372 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76840ae3a0 con 0x7f7684072d40 2024-05-28T14:14:32.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.786+0000 7f7688a00640 1 --2- 172.21.15.42:0/947365372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7684072d40 0x7f76840aee80 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f767c00d150 tx=0x7f767c01eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5699f3afd05ff54 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:32.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.786+0000 7f7683400640 1 -- 172.21.15.42:0/947365372 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c00b2d0 con 0x7f7684072d40 2024-05-28T14:14:32.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.786+0000 7f7683400640 1 -- 172.21.15.42:0/947365372 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f767c00b470 con 0x7f7684072d40 2024-05-28T14:14:32.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.786+0000 7f7683400640 1 -- 172.21.15.42:0/947365372 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c0a7b10 con 0x7f7684072d40 2024-05-28T14:14:32.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.786+0000 7f768aa00640 1 -- 172.21.15.42:0/947365372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7684072d40 msgr2=0x7f76840aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:32.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.786+0000 7f768aa00640 1 --2- 172.21.15.42:0/947365372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7684072d40 0x7f76840aee80 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f767c00d150 tx=0x7f767c01eba0 comp rx=0 tx=0).stop 2024-05-28T14:14:32.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.786+0000 7f768aa00640 1 -- 172.21.15.42:0/947365372 shutdown_connections 2024-05-28T14:14:32.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.786+0000 7f768aa00640 1 --2- 172.21.15.42:0/947365372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7684072d40 0x7f76840aee80 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:32.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.786+0000 7f768aa00640 1 --2- 172.21.15.42:0/947365372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76840d0e00 0x7f768406dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:32.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.786+0000 7f768aa00640 1 -- 172.21.15.42:0/947365372 >> 172.21.15.42:0/947365372 conn(0x7f768406bdf0 msgr2=0x7f768406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:32.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.786+0000 7f768aa00640 1 -- 172.21.15.42:0/947365372 shutdown_connections 2024-05-28T14:14:32.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.786+0000 7f768aa00640 1 -- 172.21.15.42:0/947365372 wait complete. 2024-05-28T14:14:32.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f768aa00640 1 Processor -- start 2024-05-28T14:14:32.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f768aa00640 1 -- start start 2024-05-28T14:14:32.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f768aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7684072d40 0x7f76840b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:32.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f768aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76840d0e00 0x7f76840b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:32.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f768aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768415b350 0x7f7684157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:32.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f768aa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f76840b3460 con 0x7f76840d0e00 2024-05-28T14:14:32.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f768aa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76840b32e0 con 0x7f7684072d40 2024-05-28T14:14:32.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f768aa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76840b35e0 con 0x7f768415b350 2024-05-28T14:14:32.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f767b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76840d0e00 0x7f76840b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:32.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f767b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76840d0e00 0x7f76840b18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49956/0 (socket says 172.21.15.42:49956) 2024-05-28T14:14:32.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f767b400640 1 -- 172.21.15.42:0/1364225215 learned_addr learned my addr 172.21.15.42:0/1364225215 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:32.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f7683e00640 1 --2- 172.21.15.42:0/1364225215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7684072d40 0x7f76840b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:32.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f7688a00640 1 --2- 172.21.15.42:0/1364225215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768415b350 0x7f7684157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:32.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f767b400640 1 -- 172.21.15.42:0/1364225215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768415b350 msgr2=0x7f7684157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:32.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f767b400640 1 --2- 172.21.15.42:0/1364225215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768415b350 0x7f7684157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:32.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f767b400640 1 -- 172.21.15.42:0/1364225215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7684072d40 msgr2=0x7f76840b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:32.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f767b400640 1 --2- 172.21.15.42:0/1364225215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7684072d40 0x7f76840b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:32.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f767b400640 1 -- 172.21.15.42:0/1364225215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76840b2110 con 0x7f76840d0e00 2024-05-28T14:14:32.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f767b400640 1 --2- 172.21.15.42:0/1364225215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76840d0e00 0x7f76840b18f0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f766c00d800 tx=0x7f766c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:32.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f7680c00640 1 -- 172.21.15.42:0/1364225215 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c00b6c0 con 0x7f76840d0e00 2024-05-28T14:14:32.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f7680c00640 1 -- 172.21.15.42:0/1364225215 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766c019070 con 0x7f76840d0e00 2024-05-28T14:14:32.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f7680c00640 1 -- 172.21.15.42:0/1364225215 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c00ba60 con 0x7f76840d0e00 2024-05-28T14:14:32.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f768aa00640 1 -- 172.21.15.42:0/1364225215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76840b23f0 con 0x7f76840d0e00 2024-05-28T14:14:32.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f768aa00640 1 -- 172.21.15.42:0/1364225215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76840b29b0 con 0x7f76840d0e00 2024-05-28T14:14:32.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.790+0000 7f768aa00640 1 -- 172.21.15.42:0/1364225215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7684068a90 con 0x7f76840d0e00 2024-05-28T14:14:32.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.794+0000 7f7680c00640 1 -- 172.21.15.42:0/1364225215 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f766c00bc00 con 0x7f76840d0e00 2024-05-28T14:14:32.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.794+0000 7f7680c00640 1 --2- 172.21.15.42:0/1364225215 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7660069fa0 0x7f766006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:32.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.794+0000 7f7680c00640 1 -- 172.21.15.42:0/1364225215 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5937+0+0 (secure 0 0 0) 0x7f766c085c80 con 0x7f76840d0e00 2024-05-28T14:14:32.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.794+0000 7f7683e00640 1 --2- 172.21.15.42:0/1364225215 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7660069fa0 0x7f766006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:32.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.794+0000 7f7683e00640 1 --2- 172.21.15.42:0/1364225215 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7660069fa0 0x7f766006c460 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7674007fd0 tx=0x7f7674009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:32.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.798+0000 7f7680c00640 1 -- 172.21.15.42:0/1364225215 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f766c057650 con 0x7f76840d0e00 2024-05-28T14:14:32.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:32.962+0000 7f768aa00640 1 -- 172.21.15.42:0/1364225215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7684157710 con 0x7f76840d0e00 2024-05-28T14:14:33.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.674+0000 7f7680c00640 1 -- 172.21.15.42:0/1364225215 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) ==== 181+0+0 (secure 0 0 0) 0x7f766c05b950 con 0x7f76840d0e00 2024-05-28T14:14:33.661 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-05-28T14:14:33.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.678+0000 7f768aa00640 1 -- 172.21.15.42:0/1364225215 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7660069fa0 msgr2=0x7f766006c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:33.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.678+0000 7f768aa00640 1 --2- 172.21.15.42:0/1364225215 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7660069fa0 0x7f766006c460 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7674007fd0 tx=0x7f7674009040 comp rx=0 tx=0).stop 2024-05-28T14:14:33.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.678+0000 7f768aa00640 1 -- 172.21.15.42:0/1364225215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76840d0e00 msgr2=0x7f76840b18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:33.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.678+0000 7f768aa00640 1 --2- 172.21.15.42:0/1364225215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76840d0e00 0x7f76840b18f0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f766c00d800 tx=0x7f766c00dcc0 comp rx=0 tx=0).stop 2024-05-28T14:14:33.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.678+0000 7f768aa00640 1 -- 172.21.15.42:0/1364225215 shutdown_connections 2024-05-28T14:14:33.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.678+0000 7f768aa00640 1 --2- 172.21.15.42:0/1364225215 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7660069fa0 0x7f766006c460 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:33.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.678+0000 7f768aa00640 1 --2- 172.21.15.42:0/1364225215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768415b350 0x7f7684157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:33.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.678+0000 7f768aa00640 1 --2- 172.21.15.42:0/1364225215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76840d0e00 0x7f76840b18f0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:33.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.678+0000 7f768aa00640 1 --2- 172.21.15.42:0/1364225215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7684072d40 0x7f76840b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:33.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.678+0000 7f768aa00640 1 -- 172.21.15.42:0/1364225215 >> 172.21.15.42:0/1364225215 conn(0x7f768406bdf0 msgr2=0x7f7684153740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:33.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.678+0000 7f768aa00640 1 -- 172.21.15.42:0/1364225215 shutdown_connections 2024-05-28T14:14:33.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.678+0000 7f768aa00640 1 -- 172.21.15.42:0/1364225215 wait complete. 2024-05-28T14:14:33.690 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-28T14:14:33.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b70c00640 1 Processor -- start 2024-05-28T14:14:33.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b70c00640 1 -- start start 2024-05-28T14:14:33.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b70c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5b6c06ee80 0x7f5b6c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:33.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b70c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5b6c0af360 0x7f5b6c0af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:33.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b70c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b6c1570a0 con 0x7f5b6c0d06f0 2024-05-28T14:14:33.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b70c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5b6c15cf00 con 0x7f5b6c0af360 2024-05-28T14:14:33.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b70c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5b6c15d1e0 con 0x7f5b6c06ee80 2024-05-28T14:14:33.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b61400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5b6c0af360 0x7f5b6c0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:33.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b6a800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f5b6c0d06f0 0x7f5b6c160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36770/0 (socket says 172.21.15.42:36770) 2024-05-28T14:14:33.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b6a800640 1 -- 172.21.15.42:0/1942470601 learned_addr learned my addr 172.21.15.42:0/1942470601 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:33.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b69e00640 1 --2- 172.21.15.42:0/1942470601 >> v2:172.21.15.63:3300/0 conn(0x7f5b6c06ee80 0x7f5b6c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:33.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b61400640 1 -- 172.21.15.42:0/1942470601 >> v2:172.21.15.63:3300/0 conn(0x7f5b6c06ee80 msgr2=0x7f5b6c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:33.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b61400640 1 --2- 172.21.15.42:0/1942470601 >> v2:172.21.15.63:3300/0 conn(0x7f5b6c06ee80 0x7f5b6c06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:33.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b61400640 1 -- 172.21.15.42:0/1942470601 >> v1:172.21.15.42:6789/0 conn(0x7f5b6c0d06f0 legacy=0x7f5b6c160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:33.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b61400640 1 -- 172.21.15.42:0/1942470601 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c15d4c0 con 0x7f5b6c0af360 2024-05-28T14:14:33.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.850+0000 7f5b69e00640 1 --2- 172.21.15.42:0/1942470601 >> v2:172.21.15.63:3300/0 conn(0x7f5b6c06ee80 0x7f5b6c06dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:33.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.854+0000 7f5b61400640 1 --2- 172.21.15.42:0/1942470601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b6c0af360 0x7f5b6c0af810 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0d1250 tx=0x7f5b5401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e6d18a65f5cf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:33.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.854+0000 7f5b69400640 1 -- 172.21.15.42:0/1942470601 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b540b0070 con 0x7f5b6c0af360 2024-05-28T14:14:33.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.854+0000 7f5b69400640 1 -- 172.21.15.42:0/1942470601 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b540ab600 con 0x7f5b6c0af360 2024-05-28T14:14:33.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.854+0000 7f5b69400640 1 -- 172.21.15.42:0/1942470601 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b540ab9d0 con 0x7f5b6c0af360 2024-05-28T14:14:33.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.854+0000 7f5b70c00640 1 -- 172.21.15.42:0/1942470601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b6c0af360 msgr2=0x7f5b6c0af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:33.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.854+0000 7f5b70c00640 1 --2- 172.21.15.42:0/1942470601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b6c0af360 0x7f5b6c0af810 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0d1250 tx=0x7f5b5401c3c0 comp rx=0 tx=0).stop 2024-05-28T14:14:33.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.854+0000 7f5b70c00640 1 -- 172.21.15.42:0/1942470601 shutdown_connections 2024-05-28T14:14:33.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.854+0000 7f5b70c00640 1 --2- 172.21.15.42:0/1942470601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b6c0af360 0x7f5b6c0af810 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:33.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.854+0000 7f5b70c00640 1 --2- 172.21.15.42:0/1942470601 >> v2:172.21.15.63:3300/0 conn(0x7f5b6c06ee80 0x7f5b6c06dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:33.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.854+0000 7f5b70c00640 1 -- 172.21.15.42:0/1942470601 >> 172.21.15.42:0/1942470601 conn(0x7f5b6c06bdf0 msgr2=0x7f5b6c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:33.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.854+0000 7f5b70c00640 1 -- 172.21.15.42:0/1942470601 shutdown_connections 2024-05-28T14:14:33.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.854+0000 7f5b70c00640 1 -- 172.21.15.42:0/1942470601 wait complete. 2024-05-28T14:14:33.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b70c00640 1 Processor -- start 2024-05-28T14:14:33.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b70c00640 1 -- start start 2024-05-28T14:14:33.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b70c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b6c06ee80 0x7f5b6c155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:33.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b70c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b6c0af360 0x7f5b6c153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:33.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b70c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b6c0d06f0 0x7f5b6c073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:33.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b70c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c0d1060 con 0x7f5b6c0d06f0 2024-05-28T14:14:33.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b70c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c0d0ee0 con 0x7f5b6c0af360 2024-05-28T14:14:33.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b70c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5b6c160fa0 con 0x7f5b6c06ee80 2024-05-28T14:14:33.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b6a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b6c0d06f0 0x7f5b6c073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:33.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b6a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b6c0d06f0 0x7f5b6c073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49986/0 (socket says 172.21.15.42:49986) 2024-05-28T14:14:33.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b6a800640 1 -- 172.21.15.42:0/24323023 learned_addr learned my addr 172.21.15.42:0/24323023 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:33.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b61400640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b6c0af360 0x7f5b6c153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:33.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b69e00640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b6c06ee80 0x7f5b6c155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:33.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b6a800640 1 -- 172.21.15.42:0/24323023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b6c06ee80 msgr2=0x7f5b6c155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:33.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b6a800640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b6c06ee80 0x7f5b6c155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:33.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b6a800640 1 -- 172.21.15.42:0/24323023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b6c0af360 msgr2=0x7f5b6c153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:33.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b6a800640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b6c0af360 0x7f5b6c153700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:33.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b6a800640 1 -- 172.21.15.42:0/24323023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c071d20 con 0x7f5b6c0d06f0 2024-05-28T14:14:33.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b61400640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b6c0af360 0x7f5b6c153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:33.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b6a800640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b6c0d06f0 0x7f5b6c073750 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400ca70 tx=0x7f5b6400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:33.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b62a00640 1 -- 172.21.15.42:0/24323023 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64017070 con 0x7f5b6c0d06f0 2024-05-28T14:14:33.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b62a00640 1 -- 172.21.15.42:0/24323023 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6400abd0 con 0x7f5b6c0d06f0 2024-05-28T14:14:33.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b70c00640 1 -- 172.21.15.42:0/24323023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c155f70 con 0x7f5b6c0d06f0 2024-05-28T14:14:33.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b62a00640 1 -- 172.21.15.42:0/24323023 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64012400 con 0x7f5b6c0d06f0 2024-05-28T14:14:33.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.858+0000 7f5b70c00640 1 -- 172.21.15.42:0/24323023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c071fe0 con 0x7f5b6c0d06f0 2024-05-28T14:14:33.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.862+0000 7f5b70c00640 1 -- 172.21.15.42:0/24323023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6c068a90 con 0x7f5b6c0d06f0 2024-05-28T14:14:33.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.862+0000 7f5b62a00640 1 -- 172.21.15.42:0/24323023 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f5b6401a070 con 0x7f5b6c0d06f0 2024-05-28T14:14:33.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.862+0000 7f5b62a00640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b48069fa0 0x7f5b4806c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:33.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.862+0000 7f5b62a00640 1 -- 172.21.15.42:0/24323023 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5471+0+0 (secure 0 0 0) 0x7f5b640855c0 con 0x7f5b6c0d06f0 2024-05-28T14:14:33.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.862+0000 7f5b69e00640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b48069fa0 0x7f5b4806c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:33.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.862+0000 7f5b69e00640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b48069fa0 0x7f5b4806c460 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00a9a0 tx=0x7f5b5c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:33.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:33.866+0000 7f5b62a00640 1 -- 172.21.15.42:0/24323023 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b640575e0 con 0x7f5b6c0d06f0 2024-05-28T14:14:34.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.034+0000 7f5b70c00640 1 -- 172.21.15.42:0/24323023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5b6c0b5160 con 0x7f5b6c0d06f0 2024-05-28T14:14:34.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.682+0000 7f5b62a00640 1 -- 172.21.15.42:0/24323023 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) ==== 169+0+0 (secure 0 0 0) 0x7f5b64056ee0 con 0x7f5b6c0d06f0 2024-05-28T14:14:34.667 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-05-28T14:14:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.682+0000 7f5b70c00640 1 -- 172.21.15.42:0/24323023 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b48069fa0 msgr2=0x7f5b4806c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.682+0000 7f5b70c00640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b48069fa0 0x7f5b4806c460 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00a9a0 tx=0x7f5b5c008040 comp rx=0 tx=0).stop 2024-05-28T14:14:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.682+0000 7f5b70c00640 1 -- 172.21.15.42:0/24323023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b6c0d06f0 msgr2=0x7f5b6c073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.682+0000 7f5b70c00640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b6c0d06f0 0x7f5b6c073750 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400ca70 tx=0x7f5b6400cf30 comp rx=0 tx=0).stop 2024-05-28T14:14:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.682+0000 7f5b70c00640 1 -- 172.21.15.42:0/24323023 shutdown_connections 2024-05-28T14:14:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.682+0000 7f5b70c00640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5b48069fa0 0x7f5b4806c460 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.682+0000 7f5b70c00640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b6c0d06f0 0x7f5b6c073750 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:34.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.682+0000 7f5b70c00640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b6c0af360 0x7f5b6c153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:34.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.682+0000 7f5b70c00640 1 --2- 172.21.15.42:0/24323023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b6c06ee80 0x7f5b6c155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:34.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.682+0000 7f5b70c00640 1 -- 172.21.15.42:0/24323023 >> 172.21.15.42:0/24323023 conn(0x7f5b6c06bdf0 msgr2=0x7f5b6c0749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:34.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.682+0000 7f5b70c00640 1 -- 172.21.15.42:0/24323023 shutdown_connections 2024-05-28T14:14:34.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.682+0000 7f5b70c00640 1 -- 172.21.15.42:0/24323023 wait complete. 2024-05-28T14:14:34.703 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-28T14:14:34.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.862+0000 7f0a81400640 1 Processor -- start 2024-05-28T14:14:34.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.862+0000 7f0a81400640 1 -- start start 2024-05-28T14:14:34.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.866+0000 7f0a81400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0a7c06ee20 0x7f0a7c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:34.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.866+0000 7f0a81400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0a7c15b9b0 0x7f0a7c15be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:34.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.866+0000 7f0a81400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a7c0afe40 con 0x7f0a7c144f80 2024-05-28T14:14:34.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.866+0000 7f0a81400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0a7c06f7c0 con 0x7f0a7c06ee20 2024-05-28T14:14:34.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.866+0000 7f0a81400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0a7c0724d0 con 0x7f0a7c15b9b0 2024-05-28T14:14:34.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.866+0000 7f0a73e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0a7c06ee20 0x7f0a7c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:34.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.866+0000 7f0a7a800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0a7c144f80 0x7f0a7c072960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36780/0 (socket says 172.21.15.42:36780) 2024-05-28T14:14:34.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.866+0000 7f0a7a800640 1 -- 172.21.15.42:0/3799384653 learned_addr learned my addr 172.21.15.42:0/3799384653 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:34.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.866+0000 7f0a7b200640 1 --2- 172.21.15.42:0/3799384653 >> v2:172.21.15.63:3300/0 conn(0x7f0a7c15b9b0 0x7f0a7c15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:34.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.866+0000 7f0a73e00640 1 -- 172.21.15.42:0/3799384653 >> v2:172.21.15.63:3300/0 conn(0x7f0a7c15b9b0 msgr2=0x7f0a7c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:34.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.866+0000 7f0a73e00640 1 --2- 172.21.15.42:0/3799384653 >> v2:172.21.15.63:3300/0 conn(0x7f0a7c15b9b0 0x7f0a7c15be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:34.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.866+0000 7f0a73e00640 1 -- 172.21.15.42:0/3799384653 >> v1:172.21.15.42:6789/0 conn(0x7f0a7c144f80 legacy=0x7f0a7c072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:34.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.866+0000 7f0a73e00640 1 -- 172.21.15.42:0/3799384653 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c15c340 con 0x7f0a7c06ee20 2024-05-28T14:14:34.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.866+0000 7f0a79e00640 1 -- 172.21.15.42:0/3799384653 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1515533562 0 0) 0x7f0a7c0afe40 con 0x7f0a7c144f80 2024-05-28T14:14:34.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a73e00640 1 --2- 172.21.15.42:0/3799384653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a7c06ee20 0x7f0a7c06f280 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c160060 tx=0x7f0a6401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2858e7ba2847d673 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:34.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a79e00640 1 -- 172.21.15.42:0/3799384653 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a640b0070 con 0x7f0a7c06ee20 2024-05-28T14:14:34.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a79e00640 1 -- 172.21.15.42:0/3799384653 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a640ab600 con 0x7f0a7c06ee20 2024-05-28T14:14:34.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a79e00640 1 -- 172.21.15.42:0/3799384653 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a640ab9d0 con 0x7f0a7c06ee20 2024-05-28T14:14:34.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 -- 172.21.15.42:0/3799384653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a7c06ee20 msgr2=0x7f0a7c06f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:34.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 --2- 172.21.15.42:0/3799384653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a7c06ee20 0x7f0a7c06f280 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c160060 tx=0x7f0a6401c480 comp rx=0 tx=0).stop 2024-05-28T14:14:34.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 -- 172.21.15.42:0/3799384653 shutdown_connections 2024-05-28T14:14:34.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 --2- 172.21.15.42:0/3799384653 >> v2:172.21.15.63:3300/0 conn(0x7f0a7c15b9b0 0x7f0a7c15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:34.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 --2- 172.21.15.42:0/3799384653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a7c06ee20 0x7f0a7c06f280 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:34.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 -- 172.21.15.42:0/3799384653 >> 172.21.15.42:0/3799384653 conn(0x7f0a7c0b21f0 msgr2=0x7f0a7c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:34.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 -- 172.21.15.42:0/3799384653 shutdown_connections 2024-05-28T14:14:34.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 -- 172.21.15.42:0/3799384653 wait complete. 2024-05-28T14:14:34.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 Processor -- start 2024-05-28T14:14:34.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 -- start start 2024-05-28T14:14:34.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a7c06ee20 0x7f0a7c15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a7c144f80 0x7f0a7c13f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a7c15b9b0 0x7f0a7c13d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c0b4010 con 0x7f0a7c06ee20 2024-05-28T14:14:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c072160 con 0x7f0a7c15b9b0 2024-05-28T14:14:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a81400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7c0b4190 con 0x7f0a7c144f80 2024-05-28T14:14:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a7a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a7c06ee20 0x7f0a7c15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a7a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a7c06ee20 0x7f0a7c15aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50000/0 (socket says 172.21.15.42:50000) 2024-05-28T14:14:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.870+0000 7f0a7a800640 1 -- 172.21.15.42:0/2779820963 learned_addr learned my addr 172.21.15.42:0/2779820963 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a7a800640 1 -- 172.21.15.42:0/2779820963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a7c144f80 msgr2=0x7f0a7c13f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a73e00640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a7c144f80 0x7f0a7c13f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a7b200640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a7c15b9b0 0x7f0a7c13d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a7a800640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a7c144f80 0x7f0a7c13f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a7a800640 1 -- 172.21.15.42:0/2779820963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a7c15b9b0 msgr2=0x7f0a7c13d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a7a800640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a7c15b9b0 0x7f0a7c13d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a7a800640 1 -- 172.21.15.42:0/2779820963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c15daf0 con 0x7f0a7c06ee20 2024-05-28T14:14:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a7b200640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a7c15b9b0 0x7f0a7c13d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a73e00640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a7c144f80 0x7f0a7c13f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a7a800640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a7c06ee20 0x7f0a7c15aa70 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00c520 tx=0x7f0a6c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a72a00640 1 -- 172.21.15.42:0/2779820963 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c018070 con 0x7f0a7c06ee20 2024-05-28T14:14:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a72a00640 1 -- 172.21.15.42:0/2779820963 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a6c0074e0 con 0x7f0a7c06ee20 2024-05-28T14:14:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a81400640 1 -- 172.21.15.42:0/2779820963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7c15ddd0 con 0x7f0a7c06ee20 2024-05-28T14:14:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a72a00640 1 -- 172.21.15.42:0/2779820963 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c0049f0 con 0x7f0a7c06ee20 2024-05-28T14:14:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a81400640 1 -- 172.21.15.42:0/2779820963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7c15e390 con 0x7f0a7c06ee20 2024-05-28T14:14:34.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a81400640 1 -- 172.21.15.42:0/2779820963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a7c0738b0 con 0x7f0a7c06ee20 2024-05-28T14:14:34.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a72a00640 1 -- 172.21.15.42:0/2779820963 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f0a6c01b070 con 0x7f0a7c06ee20 2024-05-28T14:14:34.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a72a00640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0a58069fa0 0x7f0a5806c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:34.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a72a00640 1 -- 172.21.15.42:0/2779820963 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4999+0+0 (secure 0 0 0) 0x7f0a6c084ce0 con 0x7f0a7c06ee20 2024-05-28T14:14:34.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.874+0000 7f0a73e00640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0a58069fa0 0x7f0a5806c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:34.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.878+0000 7f0a73e00640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0a58069fa0 0x7f0a5806c460 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0a640ad000 tx=0x7f0a64008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:34.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:34.878+0000 7f0a72a00640 1 -- 172.21.15.42:0/2779820963 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0a6c056da0 con 0x7f0a7c06ee20 2024-05-28T14:14:35.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.046+0000 7f0a81400640 1 -- 172.21.15.42:0/2779820963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f0a7c140770 con 0x7f0a7c06ee20 2024-05-28T14:14:35.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.050+0000 7f0a72a00640 1 -- 172.21.15.42:0/2779820963 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f0a6c0566a0 con 0x7f0a7c06ee20 2024-05-28T14:14:35.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.050+0000 7f0a81400640 1 -- 172.21.15.42:0/2779820963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0a58069fa0 msgr2=0x7f0a5806c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:35.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.050+0000 7f0a81400640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0a58069fa0 0x7f0a5806c460 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0a640ad000 tx=0x7f0a64008040 comp rx=0 tx=0).stop 2024-05-28T14:14:35.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.050+0000 7f0a81400640 1 -- 172.21.15.42:0/2779820963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a7c06ee20 msgr2=0x7f0a7c15aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:35.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.050+0000 7f0a81400640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a7c06ee20 0x7f0a7c15aa70 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00c520 tx=0x7f0a6c00c9e0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.050+0000 7f0a81400640 1 -- 172.21.15.42:0/2779820963 shutdown_connections 2024-05-28T14:14:35.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.050+0000 7f0a81400640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0a58069fa0 0x7f0a5806c460 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.050+0000 7f0a81400640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a7c15b9b0 0x7f0a7c13d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.050+0000 7f0a81400640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a7c144f80 0x7f0a7c13f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.050+0000 7f0a81400640 1 --2- 172.21.15.42:0/2779820963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a7c06ee20 0x7f0a7c15aa70 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.050+0000 7f0a81400640 1 -- 172.21.15.42:0/2779820963 >> 172.21.15.42:0/2779820963 conn(0x7f0a7c0b21f0 msgr2=0x7f0a7c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:35.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.050+0000 7f0a81400640 1 -- 172.21.15.42:0/2779820963 shutdown_connections 2024-05-28T14:14:35.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.050+0000 7f0a81400640 1 -- 172.21.15.42:0/2779820963 wait complete. 2024-05-28T14:14:35.059 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled' 2024-05-28T14:14:35.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.214+0000 7fb086000640 1 Processor -- start 2024-05-28T14:14:35.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.214+0000 7fb086000640 1 -- start start 2024-05-28T14:14:35.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.214+0000 7fb086000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb08015e970 0x7fb0800d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:35.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.214+0000 7fb086000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb0800cd210 0x7fb08009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:35.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.214+0000 7fb086000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb080070740 con 0x7fb08006ff60 2024-05-28T14:14:35.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.214+0000 7fb086000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb0800d0640 con 0x7fb0800cd210 2024-05-28T14:14:35.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.214+0000 7fb086000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0800d1160 con 0x7fb08015e970 2024-05-28T14:14:35.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.214+0000 7fb076a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb08006ff60 0x7fb0800ccb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36794/0 (socket says 172.21.15.42:36794) 2024-05-28T14:14:35.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.214+0000 7fb07fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb0800cd210 0x7fb08009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:35.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.214+0000 7fb076a00640 1 -- 172.21.15.42:0/865864545 learned_addr learned my addr 172.21.15.42:0/865864545 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:35.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.218+0000 7fb07f400640 1 --2- 172.21.15.42:0/865864545 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb08015e970 0x7fb0800d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:35.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.218+0000 7fb07fe00640 1 -- 172.21.15.42:0/865864545 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb08015e970 msgr2=0x7fb0800d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:35.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.218+0000 7fb07fe00640 1 --2- 172.21.15.42:0/865864545 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb08015e970 0x7fb0800d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.218+0000 7fb07fe00640 1 -- 172.21.15.42:0/865864545 >> v1:172.21.15.42:6789/0 conn(0x7fb08006ff60 legacy=0x7fb0800ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:35.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.218+0000 7fb07fe00640 1 -- 172.21.15.42:0/865864545 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0800d0870 con 0x7fb0800cd210 2024-05-28T14:14:35.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.218+0000 7fb07ea00640 1 -- 172.21.15.42:0/865864545 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 190742839 0 0) 0x7fb080070740 con 0x7fb08006ff60 2024-05-28T14:14:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.218+0000 7fb07fe00640 1 --2- 172.21.15.42:0/865864545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0800cd210 0x7fb08009aa50 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fb078009d90 tx=0x7fb07801c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0416296485cf986 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.218+0000 7fb07ea00640 1 -- 172.21.15.42:0/865864545 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0780a66c0 con 0x7fb0800cd210 2024-05-28T14:14:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.218+0000 7fb07ea00640 1 -- 172.21.15.42:0/865864545 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0780a6860 con 0x7fb0800cd210 2024-05-28T14:14:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.218+0000 7fb07ea00640 1 -- 172.21.15.42:0/865864545 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0780a6c30 con 0x7fb0800cd210 2024-05-28T14:14:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 -- 172.21.15.42:0/865864545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0800cd210 msgr2=0x7fb08009aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 --2- 172.21.15.42:0/865864545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0800cd210 0x7fb08009aa50 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fb078009d90 tx=0x7fb07801c5f0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 -- 172.21.15.42:0/865864545 shutdown_connections 2024-05-28T14:14:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 --2- 172.21.15.42:0/865864545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0800cd210 0x7fb08009aa50 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 --2- 172.21.15.42:0/865864545 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb08015e970 0x7fb0800d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 -- 172.21.15.42:0/865864545 >> 172.21.15.42:0/865864545 conn(0x7fb0800b3fa0 msgr2=0x7fb0800b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:35.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 -- 172.21.15.42:0/865864545 shutdown_connections 2024-05-28T14:14:35.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 -- 172.21.15.42:0/865864545 wait complete. 2024-05-28T14:14:35.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 Processor -- start 2024-05-28T14:14:35.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 -- start start 2024-05-28T14:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb08006ff60 0x7fb0800c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0800cd210 0x7fb080159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb08015e970 0x7fb0800b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:35.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb0800cd6a0 con 0x7fb08015e970 2024-05-28T14:14:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb080070380 con 0x7fb08006ff60 2024-05-28T14:14:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb086000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb080159b50 con 0x7fb0800cd210 2024-05-28T14:14:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb07fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb08015e970 0x7fb0800b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb07fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb08015e970 0x7fb0800b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50018/0 (socket says 172.21.15.42:50018) 2024-05-28T14:14:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb07fe00640 1 -- 172.21.15.42:0/505012736 learned_addr learned my addr 172.21.15.42:0/505012736 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb076a00640 1 --2- 172.21.15.42:0/505012736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0800cd210 0x7fb080159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb07f400640 1 --2- 172.21.15.42:0/505012736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb08006ff60 0x7fb0800c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb07fe00640 1 -- 172.21.15.42:0/505012736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0800cd210 msgr2=0x7fb080159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb07fe00640 1 --2- 172.21.15.42:0/505012736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0800cd210 0x7fb080159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb07fe00640 1 -- 172.21.15.42:0/505012736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb08006ff60 msgr2=0x7fb0800c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb07fe00640 1 --2- 172.21.15.42:0/505012736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb08006ff60 0x7fb0800c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb07fe00640 1 -- 172.21.15.42:0/505012736 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb080159e30 con 0x7fb08015e970 2024-05-28T14:14:35.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.222+0000 7fb07fe00640 1 --2- 172.21.15.42:0/505012736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb08015e970 0x7fb0800b02b0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb0780b0040 tx=0x7fb078019390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:35.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.226+0000 7fb077e00640 1 -- 172.21.15.42:0/505012736 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0780ba670 con 0x7fb08015e970 2024-05-28T14:14:35.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.226+0000 7fb077e00640 1 -- 172.21.15.42:0/505012736 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0780c0070 con 0x7fb08015e970 2024-05-28T14:14:35.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.226+0000 7fb077e00640 1 -- 172.21.15.42:0/505012736 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0780baa70 con 0x7fb08015e970 2024-05-28T14:14:35.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.226+0000 7fb086000640 1 -- 172.21.15.42:0/505012736 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08015a110 con 0x7fb08015e970 2024-05-28T14:14:35.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.226+0000 7fb086000640 1 -- 172.21.15.42:0/505012736 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08014df40 con 0x7fb08015e970 2024-05-28T14:14:35.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.226+0000 7fb086000640 1 -- 172.21.15.42:0/505012736 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb080068a90 con 0x7fb08015e970 2024-05-28T14:14:35.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.226+0000 7fb077e00640 1 -- 172.21.15.42:0/505012736 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7fb0780bac10 con 0x7fb08015e970 2024-05-28T14:14:35.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.226+0000 7fb077e00640 1 --2- 172.21.15.42:0/505012736 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb05c069fd0 0x7fb05c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:35.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.226+0000 7fb077e00640 1 -- 172.21.15.42:0/505012736 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4999+0+0 (secure 0 0 0) 0x7fb078120950 con 0x7fb08015e970 2024-05-28T14:14:35.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.226+0000 7fb07f400640 1 --2- 172.21.15.42:0/505012736 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb05c069fd0 0x7fb05c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:35.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.226+0000 7fb07f400640 1 --2- 172.21.15.42:0/505012736 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb05c069fd0 0x7fb05c06c490 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb070009b60 tx=0x7fb070009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:35.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.230+0000 7fb077e00640 1 -- 172.21.15.42:0/505012736 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb078016560 con 0x7fb08015e970 2024-05-28T14:14:35.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.454+0000 7fb086000640 1 -- 172.21.15.42:0/505012736 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled"]} v 0) -- 0x7fb0800716a0 con 0x7fb08015e970 2024-05-28T14:14:35.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.682+0000 7fb077e00640 1 -- 172.21.15.42:0/505012736 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled"]}]=0 v736) ==== 163+0+0 (secure 0 0 0) 0x7fb0780f2320 con 0x7fb08015e970 2024-05-28T14:14:35.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.682+0000 7fb086000640 1 -- 172.21.15.42:0/505012736 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb05c069fd0 msgr2=0x7fb05c06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:35.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.682+0000 7fb086000640 1 --2- 172.21.15.42:0/505012736 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb05c069fd0 0x7fb05c06c490 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb070009b60 tx=0x7fb070009190 comp rx=0 tx=0).stop 2024-05-28T14:14:35.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.682+0000 7fb086000640 1 -- 172.21.15.42:0/505012736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb08015e970 msgr2=0x7fb0800b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:35.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.682+0000 7fb086000640 1 --2- 172.21.15.42:0/505012736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb08015e970 0x7fb0800b02b0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb0780b0040 tx=0x7fb078019390 comp rx=0 tx=0).stop 2024-05-28T14:14:35.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.682+0000 7fb086000640 1 -- 172.21.15.42:0/505012736 shutdown_connections 2024-05-28T14:14:35.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.682+0000 7fb086000640 1 --2- 172.21.15.42:0/505012736 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb05c069fd0 0x7fb05c06c490 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.682+0000 7fb086000640 1 --2- 172.21.15.42:0/505012736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb08015e970 0x7fb0800b02b0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.682+0000 7fb086000640 1 --2- 172.21.15.42:0/505012736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0800cd210 0x7fb080159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.682+0000 7fb086000640 1 --2- 172.21.15.42:0/505012736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb08006ff60 0x7fb0800c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.682+0000 7fb086000640 1 -- 172.21.15.42:0/505012736 >> 172.21.15.42:0/505012736 conn(0x7fb0800b3fa0 msgr2=0x7fb08006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:35.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.686+0000 7fb086000640 1 -- 172.21.15.42:0/505012736 shutdown_connections 2024-05-28T14:14:35.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.686+0000 7fb086000640 1 -- 172.21.15.42:0/505012736 wait complete. 2024-05-28T14:14:35.691 INFO:tasks.cephfs_test_runner:Test that progress bars, that printed for 3 ongoing clone jobs, are ... ok 2024-05-28T14:14:35.691 INFO:tasks.cephfs_test_runner:Starting test: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that both the progress bars, that are printed for all 7 clone 2024-05-28T14:14:35.691 INFO:tasks.cephfs_test_runner:test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-28T14:14:35.692 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:14:35.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55f8400640 1 Processor -- start 2024-05-28T14:14:35.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55f8400640 1 -- start start 2024-05-28T14:14:35.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55f8400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55f00afa50 0x7f55f006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:35.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55f8400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f015cb90 con 0x7f55f0156e50 2024-05-28T14:14:35.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55f8400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f015ce50 con 0x7f55f0156a70 2024-05-28T14:14:35.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55f8400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f55f015d130 con 0x7f55f00afa50 2024-05-28T14:14:35.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55f6400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f55f0156e50 0x7f55f00d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36798/0 (socket says 172.21.15.42:36798) 2024-05-28T14:14:35.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55f6400640 1 -- 172.21.15.42:0/1360745170 learned_addr learned my addr 172.21.15.42:0/1360745170 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:35.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55f5000640 1 --2- 172.21.15.42:0/1360745170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55f00afa50 0x7f55f006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:35.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2794305795 0 0) 0x7f55f015cb90 con 0x7f55f0156e50 2024-05-28T14:14:35.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55d8003530 con 0x7f55f0156e50 2024-05-28T14:14:35.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1332782183 0 0) 0x7f55f015ce50 con 0x7f55f0156a70 2024-05-28T14:14:35.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55f015cb90 con 0x7f55f0156a70 2024-05-28T14:14:35.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1281856369 0 0) 0x7f55d8003530 con 0x7f55f0156e50 2024-05-28T14:14:35.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55f00afa50 msgr2=0x7f55f006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:35.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55efe00640 1 --2- 172.21.15.42:0/1360745170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55f00afa50 0x7f55f006dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.862+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 >> v1:172.21.15.63:6789/0 conn(0x7f55f0156a70 legacy=0x7f55f00af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:35.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f015d410 con 0x7f55f0156e50 2024-05-28T14:14:35.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f55e8001a00 con 0x7f55f0156e50 2024-05-28T14:14:35.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 >> v1:172.21.15.42:6789/0 conn(0x7f55f0156e50 legacy=0x7f55f00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:35.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55efe00640 1 --2- 172.21.15.42:0/1360745170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d8003b80 0x7f55d8003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:35.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55efe00640 1 --2- 172.21.15.42:0/1360745170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d8004810 0x7f55d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:35.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55efe00640 1 --2- 172.21.15.42:0/1360745170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55f00afa50 0x7f55d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:35.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55f015d130 con 0x7f55d8004810 2024-05-28T14:14:35.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f55d800ba80 con 0x7f55d8003b80 2024-05-28T14:14:35.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f55d800bc50 con 0x7f55f00afa50 2024-05-28T14:14:35.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55f5000640 1 --2- 172.21.15.42:0/1360745170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d8004810 0x7f55d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:35.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55f5a00640 1 --2- 172.21.15.42:0/1360745170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d8003b80 0x7f55d8003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:35.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55f6400640 1 --2- 172.21.15.42:0/1360745170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55f00afa50 0x7f55d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:35.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55f5000640 1 -- 172.21.15.42:0/1360745170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55f00afa50 msgr2=0x7f55d80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:35.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55f5000640 1 --2- 172.21.15.42:0/1360745170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55f00afa50 0x7f55d80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55f5000640 1 -- 172.21.15.42:0/1360745170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d8003b80 msgr2=0x7f55d8003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:35.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55f5000640 1 --2- 172.21.15.42:0/1360745170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55d8003b80 0x7f55d8003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55f5000640 1 -- 172.21.15.42:0/1360745170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d8003530 con 0x7f55d8004810 2024-05-28T14:14:35.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55f6400640 1 --2- 172.21.15.42:0/1360745170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55f00afa50 0x7f55d80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:35.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55f5000640 1 --2- 172.21.15.42:0/1360745170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d8004810 0x7f55d8004c90 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009ae0 tx=0x7f55dc01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6a0b38666791c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:35.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc0b2070 con 0x7f55d8004810 2024-05-28T14:14:35.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55dc0a5600 con 0x7f55d8004810 2024-05-28T14:14:35.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.866+0000 7f55efe00640 1 -- 172.21.15.42:0/1360745170 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc0096b0 con 0x7f55d8004810 2024-05-28T14:14:35.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f8400640 1 -- 172.21.15.42:0/1360745170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d8004810 msgr2=0x7f55d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:35.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f8400640 1 --2- 172.21.15.42:0/1360745170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d8004810 0x7f55d8004c90 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009ae0 tx=0x7f55dc01ca10 comp rx=0 tx=0).stop 2024-05-28T14:14:35.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f6400640 1 -- 172.21.15.42:0/1360745170 reap_dead start 2024-05-28T14:14:35.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f8400640 1 -- 172.21.15.42:0/1360745170 shutdown_connections 2024-05-28T14:14:35.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f8400640 1 -- 172.21.15.42:0/1360745170 >> 172.21.15.42:0/1360745170 conn(0x7f55f006bdf0 msgr2=0x7f55f006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:35.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f8400640 1 -- 172.21.15.42:0/1360745170 shutdown_connections 2024-05-28T14:14:35.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f8400640 1 -- 172.21.15.42:0/1360745170 wait complete. 2024-05-28T14:14:35.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f8400640 1 Processor -- start 2024-05-28T14:14:35.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f8400640 1 -- start start 2024-05-28T14:14:35.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f8400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55f01ee890 0x7f55f01f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:35.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f8400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f00ccd00 0x7f55f015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:35.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f8400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f015f2c0 0x7f55f0071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:35.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f8400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55f00d0f80 con 0x7f55f015f2c0 2024-05-28T14:14:35.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f8400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f55f00d0e00 con 0x7f55f00ccd00 2024-05-28T14:14:35.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f8400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f55f006dd60 con 0x7f55f01ee890 2024-05-28T14:14:35.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f6400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f015f2c0 0x7f55f0071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:35.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f6400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f015f2c0 0x7f55f0071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50038/0 (socket says 172.21.15.42:50038) 2024-05-28T14:14:35.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f6400640 1 -- 172.21.15.42:0/391728839 learned_addr learned my addr 172.21.15.42:0/391728839 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:35.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f5000640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f00ccd00 0x7f55f015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:35.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f5a00640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55f01ee890 0x7f55f01f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:35.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f6400640 1 -- 172.21.15.42:0/391728839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55f01ee890 msgr2=0x7f55f01f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:35.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f6400640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55f01ee890 0x7f55f01f5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f6400640 1 -- 172.21.15.42:0/391728839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f00ccd00 msgr2=0x7f55f015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:35.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f6400640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f00ccd00 0x7f55f015ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:35.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f6400640 1 -- 172.21.15.42:0/391728839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f00cd180 con 0x7f55f015f2c0 2024-05-28T14:14:35.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.870+0000 7f55f5000640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f00ccd00 0x7f55f015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:35.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.874+0000 7f55f6400640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f015f2c0 0x7f55f0071d40 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f55e8000f90 tx=0x7f55e800e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:35.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.874+0000 7f55ed600640 1 -- 172.21.15.42:0/391728839 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8013420 con 0x7f55f015f2c0 2024-05-28T14:14:35.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.874+0000 7f55ed600640 1 -- 172.21.15.42:0/391728839 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e8018070 con 0x7f55f015f2c0 2024-05-28T14:14:35.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.874+0000 7f55ed600640 1 -- 172.21.15.42:0/391728839 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8013820 con 0x7f55f015f2c0 2024-05-28T14:14:35.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.874+0000 7f55f8400640 1 -- 172.21.15.42:0/391728839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f00ce1e0 con 0x7f55f015f2c0 2024-05-28T14:14:35.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.874+0000 7f55f8400640 1 -- 172.21.15.42:0/391728839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f00cd710 con 0x7f55f015f2c0 2024-05-28T14:14:35.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.874+0000 7f55f8400640 1 -- 172.21.15.42:0/391728839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f00ce3b0 con 0x7f55f015f2c0 2024-05-28T14:14:35.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.874+0000 7f55ed600640 1 -- 172.21.15.42:0/391728839 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f55e80139c0 con 0x7f55f015f2c0 2024-05-28T14:14:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.874+0000 7f55ed600640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55d0069fa0 0x7f55d006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.878+0000 7f55ed600640 1 -- 172.21.15.42:0/391728839 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4999+0+0 (secure 0 0 0) 0x7f55e80860d0 con 0x7f55f015f2c0 2024-05-28T14:14:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.878+0000 7f55f5a00640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55d0069fa0 0x7f55d006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:35.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.878+0000 7f55f5a00640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55d0069fa0 0x7f55d006c460 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f55f00b28d0 tx=0x7f55e4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:35.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:35.882+0000 7f55ed600640 1 -- 172.21.15.42:0/391728839 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f55e8057aa0 con 0x7f55f015f2c0 2024-05-28T14:14:36.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.058+0000 7f55f8400640 1 -- 172.21.15.42:0/391728839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55f0068a90 con 0x7f55f015f2c0 2024-05-28T14:14:36.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.058+0000 7f55ed600640 1 -- 172.21.15.42:0/391728839 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+13058 (secure 0 0 0) 0x7f55e805bda0 con 0x7f55f015f2c0 2024-05-28T14:14:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:36.045 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":124,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:14:34.680189+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6834/3959626682":"2024-05-29T14:14:31.611816+0000","172.21.15.63:6832/3959626682":"2024-05-29T14:14:31.611816+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:14:36.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.062+0000 7f55f8400640 1 -- 172.21.15.42:0/391728839 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55d0069fa0 msgr2=0x7f55d006c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.062+0000 7f55f8400640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55d0069fa0 0x7f55d006c460 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f55f00b28d0 tx=0x7f55e4009040 comp rx=0 tx=0).stop 2024-05-28T14:14:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.062+0000 7f55f8400640 1 -- 172.21.15.42:0/391728839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f015f2c0 msgr2=0x7f55f0071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.062+0000 7f55f8400640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f015f2c0 0x7f55f0071d40 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f55e8000f90 tx=0x7f55e800e5f0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.062+0000 7f55f8400640 1 -- 172.21.15.42:0/391728839 shutdown_connections 2024-05-28T14:14:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.062+0000 7f55f8400640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f55d0069fa0 0x7f55d006c460 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.062+0000 7f55f8400640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55f015f2c0 0x7f55f0071d40 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.062+0000 7f55f8400640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55f00ccd00 0x7f55f015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.062+0000 7f55f8400640 1 --2- 172.21.15.42:0/391728839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f55f01ee890 0x7f55f01f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.062+0000 7f55f8400640 1 -- 172.21.15.42:0/391728839 >> 172.21.15.42:0/391728839 conn(0x7f55f006bdf0 msgr2=0x7f55f006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.062+0000 7f55f8400640 1 -- 172.21.15.42:0/391728839 shutdown_connections 2024-05-28T14:14:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.062+0000 7f55f8400640 1 -- 172.21.15.42:0/391728839 wait complete. 2024-05-28T14:14:36.067 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:14:36.067 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:14:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c4ce00640 1 Processor -- start 2024-05-28T14:14:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c4ce00640 1 -- start start 2024-05-28T14:14:36.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c4ce00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2c4815c770 0x7f2c4809b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:36.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c4ce00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2c480d11d0 0x7f2c48166470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:36.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c4ce00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c4809ba40 con 0x7f2c4815cb50 2024-05-28T14:14:36.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c4ce00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f2c480c6720 con 0x7f2c4815c770 2024-05-28T14:14:36.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c4ce00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2c480c60d0 con 0x7f2c480d11d0 2024-05-28T14:14:36.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c45400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f2c4815cb50 0x7f2c480d0ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36814/0 (socket says 172.21.15.42:36814) 2024-05-28T14:14:36.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c45400640 1 -- 172.21.15.42:0/1937274161 learned_addr learned my addr 172.21.15.42:0/1937274161 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:36.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c45e00640 1 --2- 172.21.15.42:0/1937274161 >> v2:172.21.15.42:3300/0 conn(0x7f2c4815c770 0x7f2c4809b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:36.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c46800640 1 --2- 172.21.15.42:0/1937274161 >> v2:172.21.15.63:3300/0 conn(0x7f2c480d11d0 0x7f2c48166470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:36.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c44a00640 1 -- 172.21.15.42:0/1937274161 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1131737668 0 0) 0x7f2c4809ba40 con 0x7f2c4815cb50 2024-05-28T14:14:36.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c44a00640 1 -- 172.21.15.42:0/1937274161 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c2c003530 con 0x7f2c4815cb50 2024-05-28T14:14:36.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c45e00640 1 -- 172.21.15.42:0/1937274161 >> v2:172.21.15.63:3300/0 conn(0x7f2c480d11d0 msgr2=0x7f2c48166470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:36.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c45e00640 1 --2- 172.21.15.42:0/1937274161 >> v2:172.21.15.63:3300/0 conn(0x7f2c480d11d0 0x7f2c48166470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c45e00640 1 -- 172.21.15.42:0/1937274161 >> v1:172.21.15.42:6789/0 conn(0x7f2c4815cb50 legacy=0x7f2c480d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:36.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c45e00640 1 -- 172.21.15.42:0/1937274161 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c480c63b0 con 0x7f2c4815c770 2024-05-28T14:14:36.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.226+0000 7f2c46800640 1 --2- 172.21.15.42:0/1937274161 >> v2:172.21.15.63:3300/0 conn(0x7f2c480d11d0 0x7f2c48166470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:36.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.230+0000 7f2c45e00640 1 --2- 172.21.15.42:0/1937274161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4815c770 0x7f2c4809b460 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f2c48160390 tx=0x7f2c3801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65cfd9a0caa88e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:36.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.230+0000 7f2c44a00640 1 -- 172.21.15.42:0/1937274161 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c380b5070 con 0x7f2c4815c770 2024-05-28T14:14:36.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.230+0000 7f2c44a00640 1 -- 172.21.15.42:0/1937274161 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c380a8970 con 0x7f2c4815c770 2024-05-28T14:14:36.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.230+0000 7f2c44a00640 1 -- 172.21.15.42:0/1937274161 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c380a8d20 con 0x7f2c4815c770 2024-05-28T14:14:36.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.230+0000 7f2c4ce00640 1 -- 172.21.15.42:0/1937274161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4815c770 msgr2=0x7f2c4809b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:36.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.230+0000 7f2c4ce00640 1 --2- 172.21.15.42:0/1937274161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4815c770 0x7f2c4809b460 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f2c48160390 tx=0x7f2c3801ec70 comp rx=0 tx=0).stop 2024-05-28T14:14:36.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.230+0000 7f2c4ce00640 1 -- 172.21.15.42:0/1937274161 shutdown_connections 2024-05-28T14:14:36.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.230+0000 7f2c4ce00640 1 --2- 172.21.15.42:0/1937274161 >> v2:172.21.15.63:3300/0 conn(0x7f2c480d11d0 0x7f2c48166470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.230+0000 7f2c4ce00640 1 --2- 172.21.15.42:0/1937274161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4815c770 0x7f2c4809b460 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.230+0000 7f2c4ce00640 1 -- 172.21.15.42:0/1937274161 >> 172.21.15.42:0/1937274161 conn(0x7f2c480b3fd0 msgr2=0x7f2c480b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:36.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.230+0000 7f2c4ce00640 1 -- 172.21.15.42:0/1937274161 shutdown_connections 2024-05-28T14:14:36.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.230+0000 7f2c4ce00640 1 -- 172.21.15.42:0/1937274161 wait complete. 2024-05-28T14:14:36.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c4ce00640 1 Processor -- start 2024-05-28T14:14:36.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c4ce00640 1 -- start start 2024-05-28T14:14:36.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c4ce00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c480d11d0 0x7f2c481538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:36.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c4ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4815c770 0x7f2c480b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:36.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c4ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c4815cb50 0x7f2c4809cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:36.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c4ce00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2c480c8bc0 con 0x7f2c4815c770 2024-05-28T14:14:36.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c4ce00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c480ce2c0 con 0x7f2c4815cb50 2024-05-28T14:14:36.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c4ce00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2c480c8d40 con 0x7f2c480d11d0 2024-05-28T14:14:36.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c45400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4815c770 0x7f2c480b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:36.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c45400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4815c770 0x7f2c480b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50064/0 (socket says 172.21.15.42:50064) 2024-05-28T14:14:36.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c45400640 1 -- 172.21.15.42:0/576465882 learned_addr learned my addr 172.21.15.42:0/576465882 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:36.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c45e00640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c480d11d0 0x7f2c481538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:36.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c46800640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c4815cb50 0x7f2c4809cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:36.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c45400640 1 -- 172.21.15.42:0/576465882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c480d11d0 msgr2=0x7f2c481538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:36.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c45400640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c480d11d0 0x7f2c481538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c45400640 1 -- 172.21.15.42:0/576465882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c4815cb50 msgr2=0x7f2c4809cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:36.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c45400640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c4815cb50 0x7f2c4809cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c45400640 1 -- 172.21.15.42:0/576465882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c480b7440 con 0x7f2c4815c770 2024-05-28T14:14:36.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c45e00640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c480d11d0 0x7f2c481538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:36.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c46800640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c4815cb50 0x7f2c4809cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:36.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c45400640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4815c770 0x7f2c480b6b00 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400c3c0 tx=0x7f2c3400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:36.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c3e000640 1 -- 172.21.15.42:0/576465882 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34017070 con 0x7f2c4815c770 2024-05-28T14:14:36.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c3e000640 1 -- 172.21.15.42:0/576465882 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c34005cf0 con 0x7f2c4815c770 2024-05-28T14:14:36.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c4ce00640 1 -- 172.21.15.42:0/576465882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c480b7720 con 0x7f2c4815c770 2024-05-28T14:14:36.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c3e000640 1 -- 172.21.15.42:0/576465882 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c340072f0 con 0x7f2c4815c770 2024-05-28T14:14:36.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.234+0000 7f2c4ce00640 1 -- 172.21.15.42:0/576465882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c480ca700 con 0x7f2c4815c770 2024-05-28T14:14:36.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.238+0000 7f2c4ce00640 1 -- 172.21.15.42:0/576465882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c480b78f0 con 0x7f2c4815c770 2024-05-28T14:14:36.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.238+0000 7f2c3e000640 1 -- 172.21.15.42:0/576465882 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f2c3401a070 con 0x7f2c4815c770 2024-05-28T14:14:36.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.238+0000 7f2c3e000640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c28069fa0 0x7f2c2806c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:36.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.238+0000 7f2c3e000640 1 -- 172.21.15.42:0/576465882 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4999+0+0 (secure 0 0 0) 0x7f2c34084ce0 con 0x7f2c4815c770 2024-05-28T14:14:36.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.238+0000 7f2c45e00640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c28069fa0 0x7f2c2806c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:36.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.238+0000 7f2c45e00640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c28069fa0 0x7f2c2806c460 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2c380b2000 tx=0x7f2c38008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:36.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.242+0000 7f2c3e000640 1 -- 172.21.15.42:0/576465882 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2c34056da0 con 0x7f2c4815c770 2024-05-28T14:14:36.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.410+0000 7f2c4ce00640 1 -- 172.21.15.42:0/576465882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2c480a04d0 con 0x7f2c4815c770 2024-05-28T14:14:36.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.410+0000 7f2c3e000640 1 -- 172.21.15.42:0/576465882 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) ==== 93+0+10 (secure 0 0 0) 0x7f2c340566a0 con 0x7f2c4815c770 2024-05-28T14:14:36.398 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:14:36.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.414+0000 7f2c4ce00640 1 -- 172.21.15.42:0/576465882 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c28069fa0 msgr2=0x7f2c2806c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:36.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.414+0000 7f2c4ce00640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c28069fa0 0x7f2c2806c460 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2c380b2000 tx=0x7f2c38008040 comp rx=0 tx=0).stop 2024-05-28T14:14:36.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.414+0000 7f2c4ce00640 1 -- 172.21.15.42:0/576465882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4815c770 msgr2=0x7f2c480b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:36.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.414+0000 7f2c4ce00640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4815c770 0x7f2c480b6b00 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400c3c0 tx=0x7f2c3400c880 comp rx=0 tx=0).stop 2024-05-28T14:14:36.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.414+0000 7f2c4ce00640 1 -- 172.21.15.42:0/576465882 shutdown_connections 2024-05-28T14:14:36.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.414+0000 7f2c4ce00640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2c28069fa0 0x7f2c2806c460 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.414+0000 7f2c4ce00640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c4815cb50 0x7f2c4809cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.414+0000 7f2c4ce00640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4815c770 0x7f2c480b6b00 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.414+0000 7f2c4ce00640 1 --2- 172.21.15.42:0/576465882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c480d11d0 0x7f2c481538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.414+0000 7f2c4ce00640 1 -- 172.21.15.42:0/576465882 >> 172.21.15.42:0/576465882 conn(0x7f2c480b3fd0 msgr2=0x7f2c480c6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:36.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.414+0000 7f2c4ce00640 1 -- 172.21.15.42:0/576465882 shutdown_connections 2024-05-28T14:14:36.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.414+0000 7f2c4ce00640 1 -- 172.21.15.42:0/576465882 wait complete. 2024-05-28T14:14:36.420 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_7_clones_are_cancelled' 2024-05-28T14:14:36.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.574+0000 7fdb89600640 1 Processor -- start 2024-05-28T14:14:36.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.574+0000 7fdb89600640 1 -- start start 2024-05-28T14:14:36.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.574+0000 7fdb89600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdb8415e9e0 0x7fdb840b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:36.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.574+0000 7fdb89600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb840d0e40 0x7fdb840b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:36.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.574+0000 7fdb89600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb840cdbd0 con 0x7fdb840ce990 2024-05-28T14:14:36.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.574+0000 7fdb89600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdb840b2b70 con 0x7fdb8415e9e0 2024-05-28T14:14:36.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.574+0000 7fdb89600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdb8415edc0 con 0x7fdb840d0e40 2024-05-28T14:14:36.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.578+0000 7fdb7a000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fdb840ce990 0x7fdb840d0730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:36828/0 (socket says 172.21.15.42:36828) 2024-05-28T14:14:36.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.578+0000 7fdb7a000640 1 -- 172.21.15.42:0/113548974 learned_addr learned my addr 172.21.15.42:0/113548974 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:36.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.578+0000 7fdb82a00640 1 --2- 172.21.15.42:0/113548974 >> v2:172.21.15.63:3300/0 conn(0x7fdb8415e9e0 0x7fdb840b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:36.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.578+0000 7fdb83400640 1 --2- 172.21.15.42:0/113548974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb840d0e40 0x7fdb840b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:36.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.578+0000 7fdb82000640 1 -- 172.21.15.42:0/113548974 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 657364563 0 0) 0x7fdb840cdbd0 con 0x7fdb840ce990 2024-05-28T14:14:36.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.578+0000 7fdb82000640 1 -- 172.21.15.42:0/113548974 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb68003530 con 0x7fdb840ce990 2024-05-28T14:14:36.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.578+0000 7fdb82a00640 1 -- 172.21.15.42:0/113548974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb840d0e40 msgr2=0x7fdb840b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:36.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.578+0000 7fdb82a00640 1 --2- 172.21.15.42:0/113548974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb840d0e40 0x7fdb840b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.578+0000 7fdb82a00640 1 -- 172.21.15.42:0/113548974 >> v1:172.21.15.42:6789/0 conn(0x7fdb840ce990 legacy=0x7fdb840d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:36.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.578+0000 7fdb82a00640 1 -- 172.21.15.42:0/113548974 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb840cdeb0 con 0x7fdb8415e9e0 2024-05-28T14:14:36.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.578+0000 7fdb82a00640 1 --2- 172.21.15.42:0/113548974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb8415e9e0 0x7fdb840b20f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdb740097c0 tx=0x7fdb7401c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be073cec224771fa server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:36.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb82000640 1 -- 172.21.15.42:0/113548974 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb740b0070 con 0x7fdb8415e9e0 2024-05-28T14:14:36.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb82000640 1 -- 172.21.15.42:0/113548974 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb740ab600 con 0x7fdb8415e9e0 2024-05-28T14:14:36.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb82000640 1 -- 172.21.15.42:0/113548974 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb740ab9b0 con 0x7fdb8415e9e0 2024-05-28T14:14:36.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 -- 172.21.15.42:0/113548974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb8415e9e0 msgr2=0x7fdb840b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:36.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 --2- 172.21.15.42:0/113548974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb8415e9e0 0x7fdb840b20f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdb740097c0 tx=0x7fdb7401c380 comp rx=0 tx=0).stop 2024-05-28T14:14:36.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 -- 172.21.15.42:0/113548974 shutdown_connections 2024-05-28T14:14:36.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 --2- 172.21.15.42:0/113548974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb840d0e40 0x7fdb840b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 --2- 172.21.15.42:0/113548974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb8415e9e0 0x7fdb840b20f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 -- 172.21.15.42:0/113548974 >> 172.21.15.42:0/113548974 conn(0x7fdb8409aa50 msgr2=0x7fdb840b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:36.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 -- 172.21.15.42:0/113548974 shutdown_connections 2024-05-28T14:14:36.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 -- 172.21.15.42:0/113548974 wait complete. 2024-05-28T14:14:36.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 Processor -- start 2024-05-28T14:14:36.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 -- start start 2024-05-28T14:14:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb840ce990 0x7fdb8415c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb840d0e40 0x7fdb841578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8415e9e0 0x7fdb84157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdb840b15b0 con 0x7fdb8415e9e0 2024-05-28T14:14:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdb840b1430 con 0x7fdb840ce990 2024-05-28T14:14:36.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb89600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdb840b1730 con 0x7fdb840d0e40 2024-05-28T14:14:36.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb83400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8415e9e0 0x7fdb84157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:36.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb83400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8415e9e0 0x7fdb84157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50076/0 (socket says 172.21.15.42:50076) 2024-05-28T14:14:36.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.582+0000 7fdb83400640 1 -- 172.21.15.42:0/2708656062 learned_addr learned my addr 172.21.15.42:0/2708656062 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:36.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb82a00640 1 --2- 172.21.15.42:0/2708656062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb840ce990 0x7fdb8415c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:36.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb7a000640 1 --2- 172.21.15.42:0/2708656062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb840d0e40 0x7fdb841578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:36.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb83400640 1 -- 172.21.15.42:0/2708656062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb840d0e40 msgr2=0x7fdb841578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:36.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb83400640 1 --2- 172.21.15.42:0/2708656062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb840d0e40 0x7fdb841578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb83400640 1 -- 172.21.15.42:0/2708656062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb840ce990 msgr2=0x7fdb8415c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:36.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb83400640 1 --2- 172.21.15.42:0/2708656062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb840ce990 0x7fdb8415c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:36.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb83400640 1 -- 172.21.15.42:0/2708656062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8415cd70 con 0x7fdb8415e9e0 2024-05-28T14:14:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb83400640 1 --2- 172.21.15.42:0/2708656062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8415e9e0 0x7fdb84157e10 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009d30 tx=0x7fdb7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb7b400640 1 -- 172.21.15.42:0/2708656062 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c018020 con 0x7fdb8415e9e0 2024-05-28T14:14:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb7b400640 1 -- 172.21.15.42:0/2708656062 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb7c014480 con 0x7fdb8415e9e0 2024-05-28T14:14:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb7b400640 1 -- 172.21.15.42:0/2708656062 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c014870 con 0x7fdb8415e9e0 2024-05-28T14:14:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb89600640 1 -- 172.21.15.42:0/2708656062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8415a340 con 0x7fdb8415e9e0 2024-05-28T14:14:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb89600640 1 -- 172.21.15.42:0/2708656062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8415bd50 con 0x7fdb8415e9e0 2024-05-28T14:14:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb89600640 1 -- 172.21.15.42:0/2708656062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb84068a90 con 0x7fdb8415e9e0 2024-05-28T14:14:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.586+0000 7fdb7b400640 1 -- 172.21.15.42:0/2708656062 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7fdb7c01b020 con 0x7fdb8415e9e0 2024-05-28T14:14:36.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.590+0000 7fdb7b400640 1 --2- 172.21.15.42:0/2708656062 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdb60069fa0 0x7fdb6006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:36.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.590+0000 7fdb7b400640 1 -- 172.21.15.42:0/2708656062 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4999+0+0 (secure 0 0 0) 0x7fdb7c0865a0 con 0x7fdb8415e9e0 2024-05-28T14:14:36.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.590+0000 7fdb82a00640 1 --2- 172.21.15.42:0/2708656062 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdb60069fa0 0x7fdb6006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:36.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.590+0000 7fdb82a00640 1 --2- 172.21.15.42:0/2708656062 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdb60069fa0 0x7fdb6006c460 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdb740ad000 tx=0x7fdb740a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:36.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.594+0000 7fdb7b400640 1 -- 172.21.15.42:0/2708656062 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdb7c022030 con 0x7fdb8415e9e0 2024-05-28T14:14:36.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:36.814+0000 7fdb89600640 1 -- 172.21.15.42:0/2708656062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_7_clones_are_cancelled"]} v 0) -- 0x7fdb840b7390 con 0x7fdb8415e9e0 2024-05-28T14:14:37.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.690+0000 7fdb7b400640 1 -- 172.21.15.42:0/2708656062 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_7_clones_are_cancelled"]}]=0 v738) ==== 166+0+0 (secure 0 0 0) 0x7fdb7c010040 con 0x7fdb8415e9e0 2024-05-28T14:14:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.690+0000 7fdb89600640 1 -- 172.21.15.42:0/2708656062 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdb60069fa0 msgr2=0x7fdb6006c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.690+0000 7fdb89600640 1 --2- 172.21.15.42:0/2708656062 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdb60069fa0 0x7fdb6006c460 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdb740ad000 tx=0x7fdb740a6000 comp rx=0 tx=0).stop 2024-05-28T14:14:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.690+0000 7fdb89600640 1 -- 172.21.15.42:0/2708656062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8415e9e0 msgr2=0x7fdb84157e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.690+0000 7fdb89600640 1 --2- 172.21.15.42:0/2708656062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8415e9e0 0x7fdb84157e10 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009d30 tx=0x7fdb7c00c490 comp rx=0 tx=0).stop 2024-05-28T14:14:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.694+0000 7fdb89600640 1 -- 172.21.15.42:0/2708656062 shutdown_connections 2024-05-28T14:14:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.694+0000 7fdb89600640 1 --2- 172.21.15.42:0/2708656062 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdb60069fa0 0x7fdb6006c460 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.694+0000 7fdb89600640 1 --2- 172.21.15.42:0/2708656062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8415e9e0 0x7fdb84157e10 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.694+0000 7fdb89600640 1 --2- 172.21.15.42:0/2708656062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb840d0e40 0x7fdb841578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.694+0000 7fdb89600640 1 --2- 172.21.15.42:0/2708656062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb840ce990 0x7fdb8415c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.694+0000 7fdb89600640 1 -- 172.21.15.42:0/2708656062 >> 172.21.15.42:0/2708656062 conn(0x7fdb8409aa50 msgr2=0x7fdb840cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.694+0000 7fdb89600640 1 -- 172.21.15.42:0/2708656062 shutdown_connections 2024-05-28T14:14:37.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.694+0000 7fdb89600640 1 -- 172.21.15.42:0/2708656062 wait complete. 2024-05-28T14:14:37.704 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T14:14:37.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe3e200640 1 Processor -- start 2024-05-28T14:14:37.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe3e200640 1 -- start start 2024-05-28T14:14:37.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe3e200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efe3815e870 0x7efe380d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:37.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe3e200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efe3815ec50 0x7efe380b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:37.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe3e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe380b7ba0 0x7efe380b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:37.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe3e200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7efe380cd500 con 0x7efe3815e870 2024-05-28T14:14:37.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe3e200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efe380d1a80 con 0x7efe3815ec50 2024-05-28T14:14:37.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe3e200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efe380ccbd0 con 0x7efe380b7ba0 2024-05-28T14:14:37.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe37400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efe3815e870 0x7efe380d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:37.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe37400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efe3815e870 0x7efe380d1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34110/0 (socket says 172.21.15.42:34110) 2024-05-28T14:14:37.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe37400640 1 -- 172.21.15.42:0/754801708 learned_addr learned my addr 172.21.15.42:0/754801708 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:37.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe2ea00640 1 --2- 172.21.15.42:0/754801708 >> v2:172.21.15.63:3300/0 conn(0x7efe3815ec50 0x7efe380b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:37.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe37e00640 1 --2- 172.21.15.42:0/754801708 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe380b7ba0 0x7efe380b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:37.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe37400640 1 -- 172.21.15.42:0/754801708 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe380b7ba0 msgr2=0x7efe380b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:37.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe37400640 1 --2- 172.21.15.42:0/754801708 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe380b7ba0 0x7efe380b8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:37.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe37400640 1 -- 172.21.15.42:0/754801708 >> v2:172.21.15.63:3300/0 conn(0x7efe3815ec50 msgr2=0x7efe380b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:37.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe37400640 1 --2- 172.21.15.42:0/754801708 >> v2:172.21.15.63:3300/0 conn(0x7efe3815ec50 0x7efe380b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:37.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe37400640 1 -- 172.21.15.42:0/754801708 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe380cceb0 con 0x7efe3815e870 2024-05-28T14:14:37.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe2ea00640 1 --2- 172.21.15.42:0/754801708 >> v2:172.21.15.63:3300/0 conn(0x7efe3815ec50 0x7efe380b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:37.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.866+0000 7efe37e00640 1 --2- 172.21.15.42:0/754801708 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe380b7ba0 0x7efe380b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:37.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.870+0000 7efe37400640 1 --2- 172.21.15.42:0/754801708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3815e870 0x7efe380d1230 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7efe38162500 tx=0x7efe2801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f40d35294e2abd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:37.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.870+0000 7efe36a00640 1 -- 172.21.15.42:0/754801708 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe280b5070 con 0x7efe3815e870 2024-05-28T14:14:37.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.870+0000 7efe36a00640 1 -- 172.21.15.42:0/754801708 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe280a89d0 con 0x7efe3815e870 2024-05-28T14:14:37.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.870+0000 7efe36a00640 1 -- 172.21.15.42:0/754801708 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe280a8da0 con 0x7efe3815e870 2024-05-28T14:14:37.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.870+0000 7efe3e200640 1 -- 172.21.15.42:0/754801708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3815e870 msgr2=0x7efe380d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:37.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.870+0000 7efe3e200640 1 --2- 172.21.15.42:0/754801708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3815e870 0x7efe380d1230 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7efe38162500 tx=0x7efe2801ed10 comp rx=0 tx=0).stop 2024-05-28T14:14:37.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.870+0000 7efe3e200640 1 -- 172.21.15.42:0/754801708 shutdown_connections 2024-05-28T14:14:37.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.870+0000 7efe3e200640 1 --2- 172.21.15.42:0/754801708 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe380b7ba0 0x7efe380b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:37.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.870+0000 7efe3e200640 1 --2- 172.21.15.42:0/754801708 >> v2:172.21.15.63:3300/0 conn(0x7efe3815ec50 0x7efe380b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:37.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.870+0000 7efe3e200640 1 --2- 172.21.15.42:0/754801708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3815e870 0x7efe380d1230 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:37.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.870+0000 7efe3e200640 1 -- 172.21.15.42:0/754801708 >> 172.21.15.42:0/754801708 conn(0x7efe380b3fd0 msgr2=0x7efe380b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:37.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.870+0000 7efe3e200640 1 -- 172.21.15.42:0/754801708 shutdown_connections 2024-05-28T14:14:37.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.870+0000 7efe3e200640 1 -- 172.21.15.42:0/754801708 wait complete. 2024-05-28T14:14:37.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe3e200640 1 Processor -- start 2024-05-28T14:14:37.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe3e200640 1 -- start start 2024-05-28T14:14:37.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe3e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe380b7ba0 0x7efe38158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:37.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe3e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3815e870 0x7efe3814b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:37.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe3e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe3815ec50 0x7efe3814bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:37.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe3e200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efe380b5330 con 0x7efe3815e870 2024-05-28T14:14:37.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe3e200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe380d1770 con 0x7efe380b7ba0 2024-05-28T14:14:37.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe3e200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efe380cf2e0 con 0x7efe3815ec50 2024-05-28T14:14:37.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe2ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3815e870 0x7efe3814b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:37.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe2ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3815e870 0x7efe3814b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34120/0 (socket says 172.21.15.42:34120) 2024-05-28T14:14:37.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe37400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe380b7ba0 0x7efe38158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:37.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe2ea00640 1 -- 172.21.15.42:0/2026864231 learned_addr learned my addr 172.21.15.42:0/2026864231 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:37.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe37e00640 1 --2- 172.21.15.42:0/2026864231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe3815ec50 0x7efe3814bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:37.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe2ea00640 1 -- 172.21.15.42:0/2026864231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe3815ec50 msgr2=0x7efe3814bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:37.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe2ea00640 1 --2- 172.21.15.42:0/2026864231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe3815ec50 0x7efe3814bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:37.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe2ea00640 1 -- 172.21.15.42:0/2026864231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe380b7ba0 msgr2=0x7efe38158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:37.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe2ea00640 1 --2- 172.21.15.42:0/2026864231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe380b7ba0 0x7efe38158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:37.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe2ea00640 1 -- 172.21.15.42:0/2026864231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe380ce760 con 0x7efe3815e870 2024-05-28T14:14:37.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe2ea00640 1 --2- 172.21.15.42:0/2026864231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3815e870 0x7efe3814b9f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7efe2000ce30 tx=0x7efe2000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:37.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe2fe00640 1 -- 172.21.15.42:0/2026864231 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2000f890 con 0x7efe3815e870 2024-05-28T14:14:37.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe2fe00640 1 -- 172.21.15.42:0/2026864231 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2000fa30 con 0x7efe3815e870 2024-05-28T14:14:37.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe2fe00640 1 -- 172.21.15.42:0/2026864231 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2000fe00 con 0x7efe3815e870 2024-05-28T14:14:37.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe3e200640 1 -- 172.21.15.42:0/2026864231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe380cea40 con 0x7efe3815e870 2024-05-28T14:14:37.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.874+0000 7efe3e200640 1 -- 172.21.15.42:0/2026864231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe380cf000 con 0x7efe3815e870 2024-05-28T14:14:37.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.878+0000 7efe3e200640 1 -- 172.21.15.42:0/2026864231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe380b3650 con 0x7efe3815e870 2024-05-28T14:14:37.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.878+0000 7efe2fe00640 1 -- 172.21.15.42:0/2026864231 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7efe20018400 con 0x7efe3815e870 2024-05-28T14:14:37.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.878+0000 7efe2fe00640 1 --2- 172.21.15.42:0/2026864231 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe14069fa0 0x7efe1406c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:37.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.878+0000 7efe2fe00640 1 -- 172.21.15.42:0/2026864231 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4999+0+0 (secure 0 0 0) 0x7efe200864c0 con 0x7efe3815e870 2024-05-28T14:14:37.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.878+0000 7efe37400640 1 --2- 172.21.15.42:0/2026864231 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe14069fa0 0x7efe1406c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:37.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.878+0000 7efe37400640 1 --2- 172.21.15.42:0/2026864231 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe14069fa0 0x7efe1406c460 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7efe280b2000 tx=0x7efe28009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:37.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:37.882+0000 7efe2fe00640 1 -- 172.21.15.42:0/2026864231 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efe20014030 con 0x7efe3815e870 2024-05-28T14:14:38.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.046+0000 7efe3e200640 1 -- 172.21.15.42:0/2026864231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7efe3804fdd0 con 0x7efe3815e870 2024-05-28T14:14:38.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.050+0000 7efe2fe00640 1 -- 172.21.15.42:0/2026864231 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7efe2000ae20 con 0x7efe3815e870 2024-05-28T14:14:38.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.050+0000 7efe3e200640 1 -- 172.21.15.42:0/2026864231 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe14069fa0 msgr2=0x7efe1406c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:38.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.050+0000 7efe3e200640 1 --2- 172.21.15.42:0/2026864231 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe14069fa0 0x7efe1406c460 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7efe280b2000 tx=0x7efe28009040 comp rx=0 tx=0).stop 2024-05-28T14:14:38.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.050+0000 7efe3e200640 1 -- 172.21.15.42:0/2026864231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3815e870 msgr2=0x7efe3814b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:38.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.050+0000 7efe3e200640 1 --2- 172.21.15.42:0/2026864231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3815e870 0x7efe3814b9f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7efe2000ce30 tx=0x7efe2000a650 comp rx=0 tx=0).stop 2024-05-28T14:14:38.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.050+0000 7efe3e200640 1 -- 172.21.15.42:0/2026864231 shutdown_connections 2024-05-28T14:14:38.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.050+0000 7efe3e200640 1 --2- 172.21.15.42:0/2026864231 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe14069fa0 0x7efe1406c460 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.050+0000 7efe3e200640 1 --2- 172.21.15.42:0/2026864231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe3815ec50 0x7efe3814bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.050+0000 7efe3e200640 1 --2- 172.21.15.42:0/2026864231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe3815e870 0x7efe3814b9f0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.050+0000 7efe3e200640 1 --2- 172.21.15.42:0/2026864231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe380b7ba0 0x7efe38158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.050+0000 7efe3e200640 1 -- 172.21.15.42:0/2026864231 >> 172.21.15.42:0/2026864231 conn(0x7efe380b3fd0 msgr2=0x7efe38071520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:38.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.050+0000 7efe3e200640 1 -- 172.21.15.42:0/2026864231 shutdown_connections 2024-05-28T14:14:38.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.050+0000 7efe3e200640 1 -- 172.21.15.42:0/2026864231 wait complete. 2024-05-28T14:14:38.059 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-28T14:14:38.059 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:14:38.145 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-28T14:14:38.229 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-28T14:14:38.229 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:14:38.229 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:14:38.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:14:38.239 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:14:38.239 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:14:38.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:14:38.249 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:14:38.249 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:14:38.294 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:14:38.295 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:14:38.295 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:14:38.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:14:38.304 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-28T14:14:38.304 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:14:38.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10e200640 1 Processor -- start 2024-05-28T14:14:38.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10e200640 1 -- start start 2024-05-28T14:14:38.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10e200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa108154fc0 0x7fa10815e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:38.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa10815eb00 0x7fa10806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:38.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10e200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa10806f260 con 0x7fa10806ea00 2024-05-28T14:14:38.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10e200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa1080ad3d0 con 0x7fa108154fc0 2024-05-28T14:14:38.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10e200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa10806f540 con 0x7fa10815eb00 2024-05-28T14:14:38.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10d800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa10806ea00 0x7fa10815af80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37812/0 (socket says 172.21.15.42:37812) 2024-05-28T14:14:38.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10d800640 1 -- 172.21.15.42:0/1122381977 learned_addr learned my addr 172.21.15.42:0/1122381977 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:38.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10ce00640 1 --2- 172.21.15.42:0/1122381977 >> v2:172.21.15.63:3300/0 conn(0x7fa108154fc0 0x7fa10815e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:38.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa0ffe00640 1 --2- 172.21.15.42:0/1122381977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa10815eb00 0x7fa10806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:38.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 -- 172.21.15.42:0/1122381977 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2471547632 0 0) 0x7fa10806f260 con 0x7fa10806ea00 2024-05-28T14:14:38.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 -- 172.21.15.42:0/1122381977 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0ec003530 con 0x7fa10806ea00 2024-05-28T14:14:38.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 -- 172.21.15.42:0/1122381977 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1362947136 0 0) 0x7fa0ec003530 con 0x7fa10806ea00 2024-05-28T14:14:38.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 -- 172.21.15.42:0/1122381977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa10815eb00 msgr2=0x7fa10806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:38.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 --2- 172.21.15.42:0/1122381977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa10815eb00 0x7fa10806e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 -- 172.21.15.42:0/1122381977 >> v2:172.21.15.63:3300/0 conn(0x7fa108154fc0 msgr2=0x7fa10815e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:38.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 --2- 172.21.15.42:0/1122381977 >> v2:172.21.15.63:3300/0 conn(0x7fa108154fc0 0x7fa10815e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 -- 172.21.15.42:0/1122381977 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10806f820 con 0x7fa10806ea00 2024-05-28T14:14:38.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 -- 172.21.15.42:0/1122381977 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fa100001a00 con 0x7fa10806ea00 2024-05-28T14:14:38.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 -- 172.21.15.42:0/1122381977 >> v1:172.21.15.42:6789/0 conn(0x7fa10806ea00 legacy=0x7fa10815af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:38.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 --2- 172.21.15.42:0/1122381977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0ec0039c0 0x7fa0ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:38.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 --2- 172.21.15.42:0/1122381977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa10815eb00 0x7fa0ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:38.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 --2- 172.21.15.42:0/1122381977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0ec004d50 0x7fa0ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:38.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 -- 172.21.15.42:0/1122381977 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa10806f540 con 0x7fa0ec004d50 2024-05-28T14:14:38.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 -- 172.21.15.42:0/1122381977 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec00b9d0 con 0x7fa0ec0039c0 2024-05-28T14:14:38.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa107e00640 1 -- 172.21.15.42:0/1122381977 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ec00bb50 con 0x7fa10815eb00 2024-05-28T14:14:38.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10d800640 1 --2- 172.21.15.42:0/1122381977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0ec004d50 0x7fa0ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:38.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10ce00640 1 --2- 172.21.15.42:0/1122381977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0ec0039c0 0x7fa0ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:38.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa0ffe00640 1 --2- 172.21.15.42:0/1122381977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa10815eb00 0x7fa0ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:38.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10d800640 1 -- 172.21.15.42:0/1122381977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa10815eb00 msgr2=0x7fa0ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:38.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10d800640 1 --2- 172.21.15.42:0/1122381977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa10815eb00 0x7fa0ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10d800640 1 -- 172.21.15.42:0/1122381977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0ec0039c0 msgr2=0x7fa0ec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:38.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10d800640 1 --2- 172.21.15.42:0/1122381977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0ec0039c0 0x7fa0ec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.498+0000 7fa10d800640 1 -- 172.21.15.42:0/1122381977 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec003530 con 0x7fa0ec004d50 2024-05-28T14:14:38.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.502+0000 7fa10d800640 1 --2- 172.21.15.42:0/1122381977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0ec004d50 0x7fa0ec0051d0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fa10000a880 tx=0x7fa10001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a880d4d0a226efcd server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:38.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.502+0000 7fa107e00640 1 -- 172.21.15.42:0/1122381977 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10000a4a0 con 0x7fa0ec004d50 2024-05-28T14:14:38.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.502+0000 7fa107e00640 1 -- 172.21.15.42:0/1122381977 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1000ad630 con 0x7fa0ec004d50 2024-05-28T14:14:38.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.502+0000 7fa107e00640 1 -- 172.21.15.42:0/1122381977 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1000ada60 con 0x7fa0ec004d50 2024-05-28T14:14:38.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.502+0000 7fa10e200640 1 -- 172.21.15.42:0/1122381977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0ec004d50 msgr2=0x7fa0ec0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:38.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.502+0000 7fa10e200640 1 --2- 172.21.15.42:0/1122381977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0ec004d50 0x7fa0ec0051d0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fa10000a880 tx=0x7fa10001d3a0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.502+0000 7fa10d800640 1 -- 172.21.15.42:0/1122381977 reap_dead start 2024-05-28T14:14:38.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.502+0000 7fa10e200640 1 -- 172.21.15.42:0/1122381977 shutdown_connections 2024-05-28T14:14:38.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.502+0000 7fa10e200640 1 -- 172.21.15.42:0/1122381977 >> 172.21.15.42:0/1122381977 conn(0x7fa10806bbc0 msgr2=0x7fa10806bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:38.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.502+0000 7fa10e200640 1 -- 172.21.15.42:0/1122381977 shutdown_connections 2024-05-28T14:14:38.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.502+0000 7fa10e200640 1 -- 172.21.15.42:0/1122381977 wait complete. 2024-05-28T14:14:38.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10e200640 1 Processor -- start 2024-05-28T14:14:38.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10e200640 1 -- start start 2024-05-28T14:14:38.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10e200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0ec004d50 0x7fa108154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:38.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1081503b0 0x7fa1081f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:38.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1080ae210 0x7fa1080ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:38.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10e200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa10806ee90 con 0x7fa1080ae210 2024-05-28T14:14:38.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10e200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081553a0 con 0x7fa1081503b0 2024-05-28T14:14:38.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10e200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa10806f010 con 0x7fa0ec004d50 2024-05-28T14:14:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1080ae210 0x7fa1080ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1080ae210 0x7fa1080ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34134/0 (socket says 172.21.15.42:34134) 2024-05-28T14:14:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10d800640 1 -- 172.21.15.42:0/2244817986 learned_addr learned my addr 172.21.15.42:0/2244817986 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10ce00640 1 --2- 172.21.15.42:0/2244817986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0ec004d50 0x7fa108154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa0ffe00640 1 --2- 172.21.15.42:0/2244817986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1081503b0 0x7fa1081f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10d800640 1 -- 172.21.15.42:0/2244817986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0ec004d50 msgr2=0x7fa108154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10d800640 1 --2- 172.21.15.42:0/2244817986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0ec004d50 0x7fa108154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10d800640 1 -- 172.21.15.42:0/2244817986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1081503b0 msgr2=0x7fa1081f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10d800640 1 --2- 172.21.15.42:0/2244817986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1081503b0 0x7fa1081f3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10d800640 1 -- 172.21.15.42:0/2244817986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1080cfc30 con 0x7fa1080ae210 2024-05-28T14:14:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10d800640 1 --2- 172.21.15.42:0/2244817986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1080ae210 0x7fa1080ae690 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fa10001dc40 tx=0x7fa0ec005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa105600640 1 -- 172.21.15.42:0/2244817986 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1000bc070 con 0x7fa1080ae210 2024-05-28T14:14:38.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa105600640 1 -- 172.21.15.42:0/2244817986 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1000017c0 con 0x7fa1080ae210 2024-05-28T14:14:38.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa105600640 1 -- 172.21.15.42:0/2244817986 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1000b2040 con 0x7fa1080ae210 2024-05-28T14:14:38.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10e200640 1 -- 172.21.15.42:0/2244817986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1080cf3d0 con 0x7fa1080ae210 2024-05-28T14:14:38.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.506+0000 7fa10e200640 1 -- 172.21.15.42:0/2244817986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1080cf950 con 0x7fa1080ae210 2024-05-28T14:14:38.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.510+0000 7fa10e200640 1 -- 172.21.15.42:0/2244817986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa10804fcf0 con 0x7fa1080ae210 2024-05-28T14:14:38.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.510+0000 7fa105600640 1 -- 172.21.15.42:0/2244817986 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7fa1000ad630 con 0x7fa1080ae210 2024-05-28T14:14:38.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.510+0000 7fa105600640 1 --2- 172.21.15.42:0/2244817986 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa0e4069fa0 0x7fa0e406c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:38.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.510+0000 7fa105600640 1 -- 172.21.15.42:0/2244817986 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4999+0+0 (secure 0 0 0) 0x7fa10011f690 con 0x7fa1080ae210 2024-05-28T14:14:38.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.510+0000 7fa10ce00640 1 --2- 172.21.15.42:0/2244817986 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa0e4069fa0 0x7fa0e406c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:38.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.510+0000 7fa10ce00640 1 --2- 172.21.15.42:0/2244817986 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa0e4069fa0 0x7fa0e406c460 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800a770 tx=0x7fa0f800a2a0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:38.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.514+0000 7fa105600640 1 -- 172.21.15.42:0/2244817986 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa1000170b0 con 0x7fa1080ae210 2024-05-28T14:14:38.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.714+0000 7fa10e200640 1 -- 172.21.15.42:0/2244817986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1080cff10 con 0x7fa1080ae210 2024-05-28T14:14:38.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.714+0000 7fa105600640 1 -- 172.21.15.42:0/2244817986 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+3861 (secure 0 0 0) 0x7fa1000f1060 con 0x7fa1080ae210 2024-05-28T14:14:38.703 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":77,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1176993478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1176993478},{"type":"v1","addr":"172.21.15.63:6834","nonce":1176993478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/56330151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":56330151},{"type":"v1","addr":"172.21.15.42:6835","nonce":56330151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":9183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/192920662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":192920662},{"type":"v1","addr":"172.21.15.63:6835","nonce":192920662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":57437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1225711200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1225711200},{"type":"v1","addr":"172.21.15.42:6837","nonce":1225711200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-05-28T14:14:38.704 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 77 2024-05-28T14:14:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.718+0000 7fa10e200640 1 -- 172.21.15.42:0/2244817986 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa0e4069fa0 msgr2=0x7fa0e406c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.718+0000 7fa10e200640 1 --2- 172.21.15.42:0/2244817986 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa0e4069fa0 0x7fa0e406c460 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800a770 tx=0x7fa0f800a2a0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.718+0000 7fa10e200640 1 -- 172.21.15.42:0/2244817986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1080ae210 msgr2=0x7fa1080ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:38.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.718+0000 7fa10e200640 1 --2- 172.21.15.42:0/2244817986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1080ae210 0x7fa1080ae690 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fa10001dc40 tx=0x7fa0ec005840 comp rx=0 tx=0).stop 2024-05-28T14:14:38.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.718+0000 7fa10e200640 1 -- 172.21.15.42:0/2244817986 shutdown_connections 2024-05-28T14:14:38.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.718+0000 7fa10e200640 1 --2- 172.21.15.42:0/2244817986 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa0e4069fa0 0x7fa0e406c460 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.718+0000 7fa10e200640 1 --2- 172.21.15.42:0/2244817986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1080ae210 0x7fa1080ae690 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.718+0000 7fa10e200640 1 --2- 172.21.15.42:0/2244817986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1081503b0 0x7fa1081f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.718+0000 7fa10e200640 1 --2- 172.21.15.42:0/2244817986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0ec004d50 0x7fa108154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.718+0000 7fa10e200640 1 -- 172.21.15.42:0/2244817986 >> 172.21.15.42:0/2244817986 conn(0x7fa10806bbc0 msgr2=0x7fa1080b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:38.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.718+0000 7fa10e200640 1 -- 172.21.15.42:0/2244817986 shutdown_connections 2024-05-28T14:14:38.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.718+0000 7fa10e200640 1 -- 172.21.15.42:0/2244817986 wait complete. 2024-05-28T14:14:38.725 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-28T14:14:38.725 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-28T14:14:38.725 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-28T14:14:38.725 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:14:38.725 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-28T14:14:38.726 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-28T14:14:38.726 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-28T14:14:38.726 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:14:38.726 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-28T14:14:38.726 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-28T14:14:38.726 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-28T14:14:38.726 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:14:38.727 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-28T14:14:38.727 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-28T14:14:38.727 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-28T14:14:38.727 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:14:38.738 INFO:tasks.ceph.mds.a:Stopped 2024-05-28T14:14:38.739 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-28T14:14:38.740 INFO:tasks.ceph.mds.c:Stopped 2024-05-28T14:14:38.740 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-28T14:14:38.743 INFO:tasks.ceph.mds.b:Stopped 2024-05-28T14:14:38.743 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-28T14:14:38.744 INFO:tasks.ceph.mds.d:Stopped 2024-05-28T14:14:38.744 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-28T14:14:38.788 INFO:tasks.ceph.mds.b:Started 2024-05-28T14:14:38.788 DEBUG:teuthology.parallel:result is None 2024-05-28T14:14:38.789 INFO:tasks.ceph.mds.a:Started 2024-05-28T14:14:38.789 DEBUG:teuthology.parallel:result is None 2024-05-28T14:14:38.790 INFO:tasks.ceph.mds.d:Started 2024-05-28T14:14:38.790 DEBUG:teuthology.parallel:result is None 2024-05-28T14:14:38.791 INFO:tasks.ceph.mds.c:Started 2024-05-28T14:14:38.791 DEBUG:teuthology.parallel:result is None 2024-05-28T14:14:38.792 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-28T14:14:38.878 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-05-28T14:14:38.879 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-05-28T14:14:38.881 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-28T14:14:38.881 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-28T14:14:38.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.954+0000 7f6d5de00640 1 Processor -- start 2024-05-28T14:14:38.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.954+0000 7f6d5de00640 1 -- start start 2024-05-28T14:14:38.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.954+0000 7f6d5de00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d58145920 0x7f6d5806f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:38.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.954+0000 7f6d5de00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5806dfa0 con 0x7f6d580724d0 2024-05-28T14:14:38.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.954+0000 7f6d5de00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5806e240 con 0x7f6d5806db40 2024-05-28T14:14:38.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.954+0000 7f6d5de00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5806e520 con 0x7f6d58145920 2024-05-28T14:14:38.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.954+0000 7f6d57400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f6d580724d0 0x7f6d5815d120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37824/0 (socket says 172.21.15.42:37824) 2024-05-28T14:14:38.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.954+0000 7f6d57400640 1 -- 172.21.15.42:0/4140953879 learned_addr learned my addr 172.21.15.42:0/4140953879 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:38.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.954+0000 7f6d56a00640 1 --2- 172.21.15.42:0/4140953879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d58145920 0x7f6d5806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:38.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.954+0000 7f6d56000640 1 -- 172.21.15.42:0/4140953879 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3862749845 0 0) 0x7f6d5806dfa0 con 0x7f6d580724d0 2024-05-28T14:14:38.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.954+0000 7f6d56000640 1 -- 172.21.15.42:0/4140953879 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d3c003530 con 0x7f6d580724d0 2024-05-28T14:14:38.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d56000640 1 -- 172.21.15.42:0/4140953879 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2577813305 0 0) 0x7f6d5806e240 con 0x7f6d5806db40 2024-05-28T14:14:38.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d56000640 1 -- 172.21.15.42:0/4140953879 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d5806dfa0 con 0x7f6d5806db40 2024-05-28T14:14:38.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d56a00640 1 -- 172.21.15.42:0/4140953879 >> v1:172.21.15.63:6789/0 conn(0x7f6d5806db40 legacy=0x7f6d58071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:38.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d56a00640 1 -- 172.21.15.42:0/4140953879 >> v1:172.21.15.42:6789/0 conn(0x7f6d580724d0 legacy=0x7f6d5815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:38.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d56a00640 1 -- 172.21.15.42:0/4140953879 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5806e800 con 0x7f6d58145920 2024-05-28T14:14:38.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d56a00640 1 --2- 172.21.15.42:0/4140953879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d58145920 0x7f6d5806f030 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6d480097f0 tx=0x7f6d4801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c2cca59307eb956 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:38.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d56000640 1 -- 172.21.15.42:0/4140953879 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d480b0070 con 0x7f6d58145920 2024-05-28T14:14:38.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d56000640 1 -- 172.21.15.42:0/4140953879 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d480ab600 con 0x7f6d58145920 2024-05-28T14:14:38.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d56000640 1 -- 172.21.15.42:0/4140953879 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d480ab9d0 con 0x7f6d58145920 2024-05-28T14:14:38.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d5de00640 1 -- 172.21.15.42:0/4140953879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d58145920 msgr2=0x7f6d5806f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:38.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d5de00640 1 --2- 172.21.15.42:0/4140953879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d58145920 0x7f6d5806f030 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6d480097f0 tx=0x7f6d4801c450 comp rx=0 tx=0).stop 2024-05-28T14:14:38.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d5de00640 1 -- 172.21.15.42:0/4140953879 shutdown_connections 2024-05-28T14:14:38.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d5de00640 1 --2- 172.21.15.42:0/4140953879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d58145920 0x7f6d5806f030 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d5de00640 1 -- 172.21.15.42:0/4140953879 >> 172.21.15.42:0/4140953879 conn(0x7f6d5806b9f0 msgr2=0x7f6d5806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:38.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d5de00640 1 -- 172.21.15.42:0/4140953879 shutdown_connections 2024-05-28T14:14:38.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.958+0000 7f6d5de00640 1 -- 172.21.15.42:0/4140953879 wait complete. 2024-05-28T14:14:38.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d5de00640 1 Processor -- start 2024-05-28T14:14:38.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d5de00640 1 -- start start 2024-05-28T14:14:38.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d5de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d5806db40 0x7f6d5815af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d5de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d580724d0 0x7f6d581442f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d5de00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d58145920 0x7f6d58144830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d5de00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5816a260 con 0x7f6d5806db40 2024-05-28T14:14:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d5de00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5815e310 con 0x7f6d580724d0 2024-05-28T14:14:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d5de00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5816a3e0 con 0x7f6d58145920 2024-05-28T14:14:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d56a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d5806db40 0x7f6d5815af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d56a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d5806db40 0x7f6d5815af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34176/0 (socket says 172.21.15.42:34176) 2024-05-28T14:14:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d56a00640 1 -- 172.21.15.42:0/1438280762 learned_addr learned my addr 172.21.15.42:0/1438280762 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d57400640 1 --2- 172.21.15.42:0/1438280762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d58145920 0x7f6d58144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d4e000640 1 --2- 172.21.15.42:0/1438280762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d580724d0 0x7f6d581442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:38.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d56a00640 1 -- 172.21.15.42:0/1438280762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d58145920 msgr2=0x7f6d58144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:38.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d56a00640 1 --2- 172.21.15.42:0/1438280762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d58145920 0x7f6d58144830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d56a00640 1 -- 172.21.15.42:0/1438280762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d580724d0 msgr2=0x7f6d581442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:38.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d56a00640 1 --2- 172.21.15.42:0/1438280762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d580724d0 0x7f6d581442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:38.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d56a00640 1 -- 172.21.15.42:0/1438280762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d580b0300 con 0x7f6d5806db40 2024-05-28T14:14:38.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d56a00640 1 --2- 172.21.15.42:0/1438280762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d5806db40 0x7f6d5815af00 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f6d480097f0 tx=0x7f6d480191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:38.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d4fe00640 1 -- 172.21.15.42:0/1438280762 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d480bb070 con 0x7f6d5806db40 2024-05-28T14:14:38.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d4fe00640 1 -- 172.21.15.42:0/1438280762 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d480b7460 con 0x7f6d5806db40 2024-05-28T14:14:38.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d4fe00640 1 -- 172.21.15.42:0/1438280762 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d480b0040 con 0x7f6d5806db40 2024-05-28T14:14:38.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d5de00640 1 -- 172.21.15.42:0/1438280762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d580b05e0 con 0x7f6d5806db40 2024-05-28T14:14:38.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.962+0000 7f6d5de00640 1 -- 172.21.15.42:0/1438280762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5815b440 con 0x7f6d5806db40 2024-05-28T14:14:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.966+0000 7f6d5de00640 1 -- 172.21.15.42:0/1438280762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d58073890 con 0x7f6d5806db40 2024-05-28T14:14:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.966+0000 7f6d4fe00640 1 -- 172.21.15.42:0/1438280762 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f6d480be070 con 0x7f6d5806db40 2024-05-28T14:14:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.966+0000 7f6d4fe00640 1 --2- 172.21.15.42:0/1438280762 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d34069fd0 0x7f6d3406c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.966+0000 7f6d4fe00640 1 -- 172.21.15.42:0/1438280762 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4999+0+0 (secure 0 0 0) 0x7f6d4811fb60 con 0x7f6d5806db40 2024-05-28T14:14:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.966+0000 7f6d4e000640 1 --2- 172.21.15.42:0/1438280762 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d34069fd0 0x7f6d3406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.966+0000 7f6d4e000640 1 --2- 172.21.15.42:0/1438280762 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d34069fd0 0x7f6d3406c490 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6d580b4db0 tx=0x7f6d40009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:38.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:38.970+0000 7f6d4fe00640 1 -- 172.21.15.42:0/1438280762 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6d480c5030 con 0x7f6d5806db40 2024-05-28T14:14:39.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.146+0000 7f6d5de00640 1 -- 172.21.15.42:0/1438280762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6d5804fdd0 con 0x7f6d5806db40 2024-05-28T14:14:39.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.702+0000 7f6d4fe00640 1 -- 172.21.15.42:0/1438280762 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v126) ==== 97+0+0 (secure 0 0 0) 0x7f6d480160d0 con 0x7f6d5806db40 2024-05-28T14:14:39.690 INFO:teuthology.orchestra.run.smithi042.stderr: removed all blocklist entries 2024-05-28T14:14:39.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.706+0000 7f6d5de00640 1 -- 172.21.15.42:0/1438280762 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d34069fd0 msgr2=0x7f6d3406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:39.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.706+0000 7f6d5de00640 1 --2- 172.21.15.42:0/1438280762 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d34069fd0 0x7f6d3406c490 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6d580b4db0 tx=0x7f6d40009040 comp rx=0 tx=0).stop 2024-05-28T14:14:39.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.706+0000 7f6d5de00640 1 -- 172.21.15.42:0/1438280762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d5806db40 msgr2=0x7f6d5815af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:39.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.706+0000 7f6d5de00640 1 --2- 172.21.15.42:0/1438280762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d5806db40 0x7f6d5815af00 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f6d480097f0 tx=0x7f6d480191f0 comp rx=0 tx=0).stop 2024-05-28T14:14:39.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.706+0000 7f6d5de00640 1 -- 172.21.15.42:0/1438280762 shutdown_connections 2024-05-28T14:14:39.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.706+0000 7f6d5de00640 1 --2- 172.21.15.42:0/1438280762 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6d34069fd0 0x7f6d3406c490 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:39.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.706+0000 7f6d5de00640 1 --2- 172.21.15.42:0/1438280762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d58145920 0x7f6d58144830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:39.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.706+0000 7f6d5de00640 1 --2- 172.21.15.42:0/1438280762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d580724d0 0x7f6d581442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:39.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.706+0000 7f6d5de00640 1 --2- 172.21.15.42:0/1438280762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d5806db40 0x7f6d5815af00 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:39.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.706+0000 7f6d5de00640 1 -- 172.21.15.42:0/1438280762 >> 172.21.15.42:0/1438280762 conn(0x7f6d5806b9f0 msgr2=0x7f6d580ad950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:39.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.706+0000 7f6d5de00640 1 -- 172.21.15.42:0/1438280762 shutdown_connections 2024-05-28T14:14:39.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.706+0000 7f6d5de00640 1 -- 172.21.15.42:0/1438280762 wait complete. 2024-05-28T14:14:39.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-28T14:14:39.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.878+0000 7fdf72e00640 1 Processor -- start 2024-05-28T14:14:39.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.878+0000 7fdf72e00640 1 -- start start 2024-05-28T14:14:39.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf72e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdf6c06f660 0x7fdf6c073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:39.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf72e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf6c15ba40 0x7fdf6c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:39.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf72e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf6c15f280 con 0x7fdf6c072dd0 2024-05-28T14:14:39.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf72e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdf6c15f030 con 0x7fdf6c06f660 2024-05-28T14:14:39.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf72e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6c15f560 con 0x7fdf6c15ba40 2024-05-28T14:14:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf63e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf6c15ba40 0x7fdf6c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf6be00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdf6c06f660 0x7fdf6c073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf70e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fdf6c072dd0 0x7fdf6c0b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:40846/0 (socket says 172.21.15.42:40846) 2024-05-28T14:14:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf70e00640 1 -- 172.21.15.42:0/4268488960 learned_addr learned my addr 172.21.15.42:0/4268488960 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf6b400640 1 -- 172.21.15.42:0/4268488960 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 838828473 0 0) 0x7fdf6c15f280 con 0x7fdf6c072dd0 2024-05-28T14:14:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf63e00640 1 -- 172.21.15.42:0/4268488960 >> v2:172.21.15.63:3300/0 conn(0x7fdf6c06f660 msgr2=0x7fdf6c073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:39.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf63e00640 1 --2- 172.21.15.42:0/4268488960 >> v2:172.21.15.63:3300/0 conn(0x7fdf6c06f660 0x7fdf6c073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:39.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf63e00640 1 -- 172.21.15.42:0/4268488960 >> v1:172.21.15.63:6789/0 conn(0x7fdf6c072dd0 legacy=0x7fdf6c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:39.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf63e00640 1 -- 172.21.15.42:0/4268488960 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c15f840 con 0x7fdf6c15ba40 2024-05-28T14:14:39.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.882+0000 7fdf6be00640 1 --2- 172.21.15.42:0/4268488960 >> v2:172.21.15.63:3300/0 conn(0x7fdf6c06f660 0x7fdf6c073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:14:39.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.886+0000 7fdf63e00640 1 --2- 172.21.15.42:0/4268488960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf6c15ba40 0x7fdf6c0d06f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdf540097c0 tx=0x7fdf5401c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5758b8b3a62be6e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.886+0000 7fdf6b400640 1 -- 172.21.15.42:0/4268488960 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540b0070 con 0x7fdf6c15ba40 2024-05-28T14:14:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.886+0000 7fdf6b400640 1 -- 172.21.15.42:0/4268488960 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf540ab600 con 0x7fdf6c15ba40 2024-05-28T14:14:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.886+0000 7fdf6b400640 1 -- 172.21.15.42:0/4268488960 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540ab9d0 con 0x7fdf6c15ba40 2024-05-28T14:14:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.886+0000 7fdf72e00640 1 -- 172.21.15.42:0/4268488960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf6c15ba40 msgr2=0x7fdf6c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.886+0000 7fdf72e00640 1 --2- 172.21.15.42:0/4268488960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf6c15ba40 0x7fdf6c0d06f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdf540097c0 tx=0x7fdf5401c350 comp rx=0 tx=0).stop 2024-05-28T14:14:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.886+0000 7fdf72e00640 1 -- 172.21.15.42:0/4268488960 shutdown_connections 2024-05-28T14:14:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.886+0000 7fdf72e00640 1 --2- 172.21.15.42:0/4268488960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf6c15ba40 0x7fdf6c0d06f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.886+0000 7fdf72e00640 1 --2- 172.21.15.42:0/4268488960 >> v2:172.21.15.63:3300/0 conn(0x7fdf6c06f660 0x7fdf6c073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.886+0000 7fdf72e00640 1 -- 172.21.15.42:0/4268488960 >> 172.21.15.42:0/4268488960 conn(0x7fdf6c06bdf0 msgr2=0x7fdf6c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.886+0000 7fdf72e00640 1 -- 172.21.15.42:0/4268488960 shutdown_connections 2024-05-28T14:14:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.886+0000 7fdf72e00640 1 -- 172.21.15.42:0/4268488960 wait complete. 2024-05-28T14:14:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf72e00640 1 Processor -- start 2024-05-28T14:14:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf72e00640 1 -- start start 2024-05-28T14:14:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf72e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c06f660 0x7fdf6c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf72e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf6c072dd0 0x7fdf6c14d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf72e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf6c0cf5f0 0x7fdf6c153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf72e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c0b3460 con 0x7fdf6c06f660 2024-05-28T14:14:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf72e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c0b32e0 con 0x7fdf6c072dd0 2024-05-28T14:14:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf72e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6c0b35e0 con 0x7fdf6c0cf5f0 2024-05-28T14:14:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf6be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c06f660 0x7fdf6c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf6be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c06f660 0x7fdf6c0cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34192/0 (socket says 172.21.15.42:34192) 2024-05-28T14:14:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf6be00640 1 -- 172.21.15.42:0/3377266936 learned_addr learned my addr 172.21.15.42:0/3377266936 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf70e00640 1 --2- 172.21.15.42:0/3377266936 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf6c0cf5f0 0x7fdf6c153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf63e00640 1 --2- 172.21.15.42:0/3377266936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf6c072dd0 0x7fdf6c14d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:39.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf6be00640 1 -- 172.21.15.42:0/3377266936 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf6c0cf5f0 msgr2=0x7fdf6c153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:39.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf6be00640 1 --2- 172.21.15.42:0/3377266936 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf6c0cf5f0 0x7fdf6c153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:39.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf6be00640 1 -- 172.21.15.42:0/3377266936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf6c072dd0 msgr2=0x7fdf6c14d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:39.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf6be00640 1 --2- 172.21.15.42:0/3377266936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf6c072dd0 0x7fdf6c14d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:39.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf6be00640 1 -- 172.21.15.42:0/3377266936 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c0b10a0 con 0x7fdf6c06f660 2024-05-28T14:14:39.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf6be00640 1 --2- 172.21.15.42:0/3377266936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c06f660 0x7fdf6c0cce00 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00ebe0 tx=0x7fdf5c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:39.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf68c00640 1 -- 172.21.15.42:0/3377266936 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c017070 con 0x7fdf6c06f660 2024-05-28T14:14:39.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf68c00640 1 -- 172.21.15.42:0/3377266936 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf5c00cc30 con 0x7fdf6c06f660 2024-05-28T14:14:39.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf72e00640 1 -- 172.21.15.42:0/3377266936 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6c0b1380 con 0x7fdf6c06f660 2024-05-28T14:14:39.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf68c00640 1 -- 172.21.15.42:0/3377266936 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c013470 con 0x7fdf6c06f660 2024-05-28T14:14:39.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf72e00640 1 -- 172.21.15.42:0/3377266936 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf6c14de20 con 0x7fdf6c06f660 2024-05-28T14:14:39.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.890+0000 7fdf72e00640 1 -- 172.21.15.42:0/3377266936 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf6c068a90 con 0x7fdf6c06f660 2024-05-28T14:14:39.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.894+0000 7fdf68c00640 1 -- 172.21.15.42:0/3377266936 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7fdf5c01a070 con 0x7fdf6c06f660 2024-05-28T14:14:39.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.894+0000 7fdf68c00640 1 --2- 172.21.15.42:0/3377266936 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf48069fa0 0x7fdf4806c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:39.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.894+0000 7fdf68c00640 1 -- 172.21.15.42:0/3377266936 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4913+0+0 (secure 0 0 0) 0x7fdf5c085d60 con 0x7fdf6c06f660 2024-05-28T14:14:39.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.894+0000 7fdf63e00640 1 --2- 172.21.15.42:0/3377266936 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf48069fa0 0x7fdf4806c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:39.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.894+0000 7fdf63e00640 1 --2- 172.21.15.42:0/3377266936 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf48069fa0 0x7fdf4806c460 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdf540ad000 tx=0x7fdf54008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:39.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:39.898+0000 7fdf68c00640 1 -- 172.21.15.42:0/3377266936 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdf5c057d00 con 0x7fdf6c06f660 2024-05-28T14:14:40.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.122+0000 7fdf72e00640 1 -- 172.21.15.42:0/3377266936 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fdf6c06c2d0 con 0x7fdf6c06f660 2024-05-28T14:14:40.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.122+0000 7fdf68c00640 1 -- 172.21.15.42:0/3377266936 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) ==== 80+0+5304 (secure 0 0 0) 0x7fdf5c057600 con 0x7fdf6c06f660 2024-05-28T14:14:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:40.108 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:14:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_dump": [ 2024-05-28T14:14:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.a", 2024-05-28T14:14:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmE+3RLRAADuTKGfKeVafGeLpgr9WKUg==", 2024-05-28T14:14:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:14:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:14:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:14:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.b", 2024-05-28T14:14:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVm1rmVOhAAV46KzDJnRfDZYIZanS0g2g==", 2024-05-28T14:14:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:14:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:14:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:14:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.c", 2024-05-28T14:14:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVme4aYNRAAVxJTlugnxtFTexUclF/IwA==", 2024-05-28T14:14:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:14:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:14:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:14:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.d", 2024-05-28T14:14:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmeAdBBhAAr4IVFQpjQXFVlnFTHnPV9w==", 2024-05-28T14:14:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:14:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:14:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:14:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.0", 2024-05-28T14:14:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDB4lVmGMDBJhAA3jIyR7XKLGXEgimIGNsw6A==", 2024-05-28T14:14:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:14:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:14:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.1", 2024-05-28T14:14:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDE4lVmxMisLxAAcIz7GPlkZwvW5ig+DbAVGg==", 2024-05-28T14:14:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:14:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:14:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.2", 2024-05-28T14:14:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDH4lVmSSk4OhAAmLlLysM4Im/Kew4Xbq2/OA==", 2024-05-28T14:14:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:14:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:14:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.3", 2024-05-28T14:14:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDL4lVmvzrYBhAAiAAOQXZBuTsMVYa1vE5rkg==", 2024-05-28T14:14:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:14:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:14:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.4", 2024-05-28T14:14:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDQ4lVmrqnTOhAA2pIfgBT9LJDIhp+GgFy8Gg==", 2024-05-28T14:14:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:14:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:14:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.5", 2024-05-28T14:14:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDU4lVmYpsQCxAAoTK2aMBPrHKvsUnNOTj/Dw==", 2024-05-28T14:14:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:14:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:14:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.6", 2024-05-28T14:14:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDX4lVm6TCAFxAAs3rkDyKcLhWrn/EkIo0bVA==", 2024-05-28T14:14:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:14:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:14:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.7", 2024-05-28T14:14:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDa4lVmr5NwIRAA7Lb70VYV82dr0Frn8CCUXg==", 2024-05-28T14:14:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-05-28T14:14:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-05-28T14:14:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.0", 2024-05-28T14:14:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw==", 2024-05-28T14:14:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:14:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-05-28T14:14:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-28T14:14:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.1", 2024-05-28T14:14:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmcfkLFBAADNyODKTtYAlFWquZ/LeeqQ==", 2024-05-28T14:14:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:14:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:14:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:14:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:14:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.2", 2024-05-28T14:14:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmoxx+EBAA8mLNak5AwRmWQIgkYtMOrg==", 2024-05-28T14:14:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:14:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:14:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:14:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:14:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.3", 2024-05-28T14:14:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQDA4lVmxSUsGhAAbgr1eOh/sMfrEONKKHFo0g==", 2024-05-28T14:14:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-05-28T14:14:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-05-28T14:14:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-05-28T14:14:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-05-28T14:14:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.admin", 2024-05-28T14:14:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmOSBPERAAt4x2DBal4PnqxqeWQutLDQ==", 2024-05-28T14:14:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:14:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-05-28T14:14:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-05-28T14:14:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.x", 2024-05-28T14:14:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmNO5tKRAA70hPLwoE83keovlq2rolxQ==", 2024-05-28T14:14:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:14:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T14:14:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.y", 2024-05-28T14:14:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQC/4lVmxTVyJRAATMLAAqKne20YvV537uXZ6g==", 2024-05-28T14:14:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-05-28T14:14:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-05-28T14:14:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-05-28T14:14:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-05-28T14:14:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-05-28T14:14:40.126 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:14:40.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.122+0000 7fdf72e00640 1 -- 172.21.15.42:0/3377266936 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf48069fa0 msgr2=0x7fdf4806c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.122+0000 7fdf72e00640 1 --2- 172.21.15.42:0/3377266936 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf48069fa0 0x7fdf4806c460 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdf540ad000 tx=0x7fdf54008040 comp rx=0 tx=0).stop 2024-05-28T14:14:40.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.122+0000 7fdf72e00640 1 -- 172.21.15.42:0/3377266936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c06f660 msgr2=0x7fdf6c0cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.122+0000 7fdf72e00640 1 --2- 172.21.15.42:0/3377266936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c06f660 0x7fdf6c0cce00 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00ebe0 tx=0x7fdf5c00c520 comp rx=0 tx=0).stop 2024-05-28T14:14:40.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.122+0000 7fdf72e00640 1 -- 172.21.15.42:0/3377266936 shutdown_connections 2024-05-28T14:14:40.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.122+0000 7fdf72e00640 1 --2- 172.21.15.42:0/3377266936 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdf48069fa0 0x7fdf4806c460 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.122+0000 7fdf72e00640 1 --2- 172.21.15.42:0/3377266936 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf6c0cf5f0 0x7fdf6c153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.122+0000 7fdf72e00640 1 --2- 172.21.15.42:0/3377266936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf6c072dd0 0x7fdf6c14d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.122+0000 7fdf72e00640 1 --2- 172.21.15.42:0/3377266936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf6c06f660 0x7fdf6c0cce00 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.122+0000 7fdf72e00640 1 -- 172.21.15.42:0/3377266936 >> 172.21.15.42:0/3377266936 conn(0x7fdf6c06bdf0 msgr2=0x7fdf6c0aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:40.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.122+0000 7fdf72e00640 1 -- 172.21.15.42:0/3377266936 shutdown_connections 2024-05-28T14:14:40.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.122+0000 7fdf72e00640 1 -- 172.21.15.42:0/3377266936 wait complete. 2024-05-28T14:14:40.130 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:14:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f8c00640 1 Processor -- start 2024-05-28T14:14:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f8c00640 1 -- start start 2024-05-28T14:14:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f8c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f04f415c930 0x7f04f406c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f8c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f40b2e00 con 0x7f04f40ae2b0 2024-05-28T14:14:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f8c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f40b1e60 con 0x7f04f40b20d0 2024-05-28T14:14:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f8c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f04f40b12c0 con 0x7f04f415c930 2024-05-28T14:14:40.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f2800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f04f40ae2b0 0x7f04f40ae700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37828/0 (socket says 172.21.15.42:37828) 2024-05-28T14:14:40.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f2800640 1 -- 172.21.15.42:0/2397198494 learned_addr learned my addr 172.21.15.42:0/2397198494 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:40.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f1e00640 1 --2- 172.21.15.42:0/2397198494 >> v2:172.21.15.63:3300/0 conn(0x7f04f415c930 0x7f04f406c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f1400640 1 -- 172.21.15.42:0/2397198494 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1435493030 0 0) 0x7f04f40b2e00 con 0x7f04f40ae2b0 2024-05-28T14:14:40.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f1400640 1 -- 172.21.15.42:0/2397198494 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04d8003530 con 0x7f04f40ae2b0 2024-05-28T14:14:40.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f1400640 1 -- 172.21.15.42:0/2397198494 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4155523256 0 0) 0x7f04f40b1e60 con 0x7f04f40b20d0 2024-05-28T14:14:40.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f1400640 1 -- 172.21.15.42:0/2397198494 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04f40b2e00 con 0x7f04f40b20d0 2024-05-28T14:14:40.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f1e00640 1 -- 172.21.15.42:0/2397198494 >> v1:172.21.15.63:6789/0 conn(0x7f04f40b20d0 legacy=0x7f04f40b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f1e00640 1 -- 172.21.15.42:0/2397198494 >> v1:172.21.15.42:6789/0 conn(0x7f04f40ae2b0 legacy=0x7f04f40ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.286+0000 7f04f1e00640 1 -- 172.21.15.42:0/2397198494 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f40b2840 con 0x7f04f415c930 2024-05-28T14:14:40.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.290+0000 7f04f1e00640 1 --2- 172.21.15.42:0/2397198494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04f415c930 0x7f04f406c400 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f04e40097c0 tx=0x7f04e401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63e9455a3fecc8be server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:40.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.290+0000 7f04f1400640 1 -- 172.21.15.42:0/2397198494 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e40a7070 con 0x7f04f415c930 2024-05-28T14:14:40.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.290+0000 7f04f1400640 1 -- 172.21.15.42:0/2397198494 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e40b0070 con 0x7f04f415c930 2024-05-28T14:14:40.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.290+0000 7f04f1400640 1 -- 172.21.15.42:0/2397198494 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e40ab870 con 0x7f04f415c930 2024-05-28T14:14:40.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.290+0000 7f04f8c00640 1 -- 172.21.15.42:0/2397198494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04f415c930 msgr2=0x7f04f406c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.290+0000 7f04f8c00640 1 --2- 172.21.15.42:0/2397198494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04f415c930 0x7f04f406c400 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f04e40097c0 tx=0x7f04e401c470 comp rx=0 tx=0).stop 2024-05-28T14:14:40.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.290+0000 7f04f8c00640 1 -- 172.21.15.42:0/2397198494 shutdown_connections 2024-05-28T14:14:40.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.290+0000 7f04f8c00640 1 --2- 172.21.15.42:0/2397198494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04f415c930 0x7f04f406c400 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.290+0000 7f04f8c00640 1 -- 172.21.15.42:0/2397198494 >> 172.21.15.42:0/2397198494 conn(0x7f04f409aa50 msgr2=0x7f04f40b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:40.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.290+0000 7f04f8c00640 1 -- 172.21.15.42:0/2397198494 shutdown_connections 2024-05-28T14:14:40.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.290+0000 7f04f8c00640 1 -- 172.21.15.42:0/2397198494 wait complete. 2024-05-28T14:14:40.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f8c00640 1 Processor -- start 2024-05-28T14:14:40.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f8c00640 1 -- start start 2024-05-28T14:14:40.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04f40ae2b0 0x7f04f4149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f8c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04f40b20d0 0x7f04f40c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04f415c930 0x7f04f40c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f8c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f04f40d0e90 con 0x7f04f415c930 2024-05-28T14:14:40.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f8c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04f40b24b0 con 0x7f04f40ae2b0 2024-05-28T14:14:40.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f8c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f04f40d1010 con 0x7f04f40b20d0 2024-05-28T14:14:40.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04f415c930 0x7f04f40c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04f40ae2b0 0x7f04f4149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04f415c930 0x7f04f40c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34200/0 (socket says 172.21.15.42:34200) 2024-05-28T14:14:40.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f2800640 1 -- 172.21.15.42:0/4108367693 learned_addr learned my addr 172.21.15.42:0/4108367693 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:40.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04ebe00640 1 --2- 172.21.15.42:0/4108367693 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04f40b20d0 0x7f04f40c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f2800640 1 -- 172.21.15.42:0/4108367693 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04f40b20d0 msgr2=0x7f04f40c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f2800640 1 --2- 172.21.15.42:0/4108367693 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04f40b20d0 0x7f04f40c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f2800640 1 -- 172.21.15.42:0/4108367693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04f40ae2b0 msgr2=0x7f04f4149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f2800640 1 --2- 172.21.15.42:0/4108367693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04f40ae2b0 0x7f04f4149e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f2800640 1 -- 172.21.15.42:0/4108367693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f4159bc0 con 0x7f04f415c930 2024-05-28T14:14:40.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f2800640 1 --2- 172.21.15.42:0/4108367693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04f415c930 0x7f04f40c6980 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00caa0 tx=0x7f04ec00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:40.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04ea000640 1 -- 172.21.15.42:0/4108367693 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec017070 con 0x7f04f415c930 2024-05-28T14:14:40.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04ea000640 1 -- 172.21.15.42:0/4108367693 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ec00ac20 con 0x7f04f415c930 2024-05-28T14:14:40.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f8c00640 1 -- 172.21.15.42:0/4108367693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f4159320 con 0x7f04f415c930 2024-05-28T14:14:40.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04ea000640 1 -- 172.21.15.42:0/4108367693 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec012640 con 0x7f04f415c930 2024-05-28T14:14:40.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.294+0000 7f04f8c00640 1 -- 172.21.15.42:0/4108367693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f41598e0 con 0x7f04f415c930 2024-05-28T14:14:40.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.298+0000 7f04f8c00640 1 -- 172.21.15.42:0/4108367693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f4068a90 con 0x7f04f415c930 2024-05-28T14:14:40.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.298+0000 7f04ea000640 1 -- 172.21.15.42:0/4108367693 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f04ec01a070 con 0x7f04f415c930 2024-05-28T14:14:40.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.298+0000 7f04ea000640 1 --2- 172.21.15.42:0/4108367693 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f04d0069fa0 0x7f04d006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.298+0000 7f04ea000640 1 -- 172.21.15.42:0/4108367693 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4913+0+0 (secure 0 0 0) 0x7f04ec085640 con 0x7f04f415c930 2024-05-28T14:14:40.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.298+0000 7f04f1e00640 1 --2- 172.21.15.42:0/4108367693 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f04d0069fa0 0x7f04d006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.298+0000 7f04f1e00640 1 --2- 172.21.15.42:0/4108367693 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f04d0069fa0 0x7f04d006c460 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f04e40af000 tx=0x7f04e40a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:40.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.302+0000 7f04ea000640 1 -- 172.21.15.42:0/4108367693 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f04f4159bc0 con 0x7f04f415c930 2024-05-28T14:14:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.470+0000 7f04f8c00640 1 -- 172.21.15.42:0/4108367693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04f406f3d0 con 0x7f04f415c930 2024-05-28T14:14:40.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.470+0000 7f04ea000640 1 -- 172.21.15.42:0/4108367693 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+12929 (secure 0 0 0) 0x7f04ec057010 con 0x7f04f415c930 2024-05-28T14:14:40.458 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:40.458 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":128,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:14:39.716463+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:14:40.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.474+0000 7f04f8c00640 1 -- 172.21.15.42:0/4108367693 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f04d0069fa0 msgr2=0x7f04d006c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.474+0000 7f04f8c00640 1 --2- 172.21.15.42:0/4108367693 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f04d0069fa0 0x7f04d006c460 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f04e40af000 tx=0x7f04e40a7000 comp rx=0 tx=0).stop 2024-05-28T14:14:40.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.474+0000 7f04f8c00640 1 -- 172.21.15.42:0/4108367693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04f415c930 msgr2=0x7f04f40c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.474+0000 7f04f8c00640 1 --2- 172.21.15.42:0/4108367693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04f415c930 0x7f04f40c6980 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00caa0 tx=0x7f04ec00cf60 comp rx=0 tx=0).stop 2024-05-28T14:14:40.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.474+0000 7f04f8c00640 1 -- 172.21.15.42:0/4108367693 shutdown_connections 2024-05-28T14:14:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.474+0000 7f04f8c00640 1 --2- 172.21.15.42:0/4108367693 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f04d0069fa0 0x7f04d006c460 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.474+0000 7f04f8c00640 1 --2- 172.21.15.42:0/4108367693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04f415c930 0x7f04f40c6980 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.474+0000 7f04f8c00640 1 --2- 172.21.15.42:0/4108367693 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04f40b20d0 0x7f04f40c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.474+0000 7f04f8c00640 1 --2- 172.21.15.42:0/4108367693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04f40ae2b0 0x7f04f4149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.474+0000 7f04f8c00640 1 -- 172.21.15.42:0/4108367693 >> 172.21.15.42:0/4108367693 conn(0x7f04f409aa50 msgr2=0x7f04f4157500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.474+0000 7f04f8c00640 1 -- 172.21.15.42:0/4108367693 shutdown_connections 2024-05-28T14:14:40.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.474+0000 7f04f8c00640 1 -- 172.21.15.42:0/4108367693 wait complete. 2024-05-28T14:14:40.481 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:14:40.481 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:14:40.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f1a600640 1 Processor -- start 2024-05-28T14:14:40.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f1a600640 1 -- start start 2024-05-28T14:14:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f1a600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6f140b02e0 0x7f6f1406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f1a600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6f1406d110 0x7f6f14154da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f1a600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f1415fb60 con 0x7f6f1415f710 2024-05-28T14:14:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f1a600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f6f1406f190 con 0x7f6f140b02e0 2024-05-28T14:14:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f1a600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6f1406f470 con 0x7f6f1406d110 2024-05-28T14:14:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f13400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6f140b02e0 0x7f6f1406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f13400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6f140b02e0 0x7f6f1406eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34212/0 (socket says 172.21.15.42:34212) 2024-05-28T14:14:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f13400640 1 -- 172.21.15.42:0/116417642 learned_addr learned my addr 172.21.15.42:0/116417642 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:40.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f13e00640 1 --2- 172.21.15.42:0/116417642 >> v2:172.21.15.63:3300/0 conn(0x7f6f1406d110 0x7f6f14154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f13400640 1 -- 172.21.15.42:0/116417642 >> v2:172.21.15.63:3300/0 conn(0x7f6f1406d110 msgr2=0x7f6f14154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:40.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f13400640 1 --2- 172.21.15.42:0/116417642 >> v2:172.21.15.63:3300/0 conn(0x7f6f1406d110 0x7f6f14154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f13400640 1 -- 172.21.15.42:0/116417642 >> v1:172.21.15.42:6789/0 conn(0x7f6f1415f710 legacy=0x7f6f1406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.638+0000 7f6f13400640 1 -- 172.21.15.42:0/116417642 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1406f750 con 0x7f6f140b02e0 2024-05-28T14:14:40.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.642+0000 7f6f13400640 1 --2- 172.21.15.42:0/116417642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f140b02e0 0x7f6f1406eb00 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f6f140ad8c0 tx=0x7f6f0401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3911b4be8e9c7867 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:40.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.642+0000 7f6f12a00640 1 -- 172.21.15.42:0/116417642 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f040a5810 con 0x7f6f140b02e0 2024-05-28T14:14:40.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.642+0000 7f6f12a00640 1 -- 172.21.15.42:0/116417642 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f040a59b0 con 0x7f6f140b02e0 2024-05-28T14:14:40.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.642+0000 7f6f12a00640 1 -- 172.21.15.42:0/116417642 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f040a5d80 con 0x7f6f140b02e0 2024-05-28T14:14:40.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.642+0000 7f6f1a600640 1 -- 172.21.15.42:0/116417642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f140b02e0 msgr2=0x7f6f1406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.642+0000 7f6f1a600640 1 --2- 172.21.15.42:0/116417642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f140b02e0 0x7f6f1406eb00 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f6f140ad8c0 tx=0x7f6f0401c9c0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.642+0000 7f6f1a600640 1 -- 172.21.15.42:0/116417642 shutdown_connections 2024-05-28T14:14:40.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.642+0000 7f6f1a600640 1 --2- 172.21.15.42:0/116417642 >> v2:172.21.15.63:3300/0 conn(0x7f6f1406d110 0x7f6f14154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.642+0000 7f6f1a600640 1 --2- 172.21.15.42:0/116417642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f140b02e0 0x7f6f1406eb00 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.642+0000 7f6f1a600640 1 -- 172.21.15.42:0/116417642 >> 172.21.15.42:0/116417642 conn(0x7f6f14072600 msgr2=0x7f6f14072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:40.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.642+0000 7f6f1a600640 1 -- 172.21.15.42:0/116417642 shutdown_connections 2024-05-28T14:14:40.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.642+0000 7f6f1a600640 1 -- 172.21.15.42:0/116417642 wait complete. 2024-05-28T14:14:40.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f1a600640 1 Processor -- start 2024-05-28T14:14:40.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f1a600640 1 -- start start 2024-05-28T14:14:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f1a600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f1406d110 0x7f6f140cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f1a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f140b02e0 0x7f6f140cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f1a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1415f710 0x7f6f141f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f1a600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6f140b4230 con 0x7f6f140b02e0 2024-05-28T14:14:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f1a600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f140b40b0 con 0x7f6f1415f710 2024-05-28T14:14:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f1a600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6f140b43b0 con 0x7f6f1406d110 2024-05-28T14:14:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f0aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f140b02e0 0x7f6f140cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f0aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f140b02e0 0x7f6f140cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34218/0 (socket says 172.21.15.42:34218) 2024-05-28T14:14:40.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f0aa00640 1 -- 172.21.15.42:0/114380061 learned_addr learned my addr 172.21.15.42:0/114380061 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:40.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f13400640 1 --2- 172.21.15.42:0/114380061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f1406d110 0x7f6f140cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f13e00640 1 --2- 172.21.15.42:0/114380061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1415f710 0x7f6f141f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f0aa00640 1 -- 172.21.15.42:0/114380061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f1406d110 msgr2=0x7f6f140cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f0aa00640 1 --2- 172.21.15.42:0/114380061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f1406d110 0x7f6f140cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f0aa00640 1 -- 172.21.15.42:0/114380061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1415f710 msgr2=0x7f6f141f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f0aa00640 1 --2- 172.21.15.42:0/114380061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1415f710 0x7f6f141f76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f0aa00640 1 -- 172.21.15.42:0/114380061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f141f8170 con 0x7f6f140b02e0 2024-05-28T14:14:40.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f0aa00640 1 --2- 172.21.15.42:0/114380061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f140b02e0 0x7f6f140cf040 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00c370 tx=0x7f6efc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:40.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f0be00640 1 -- 172.21.15.42:0/114380061 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0052d0 con 0x7f6f140b02e0 2024-05-28T14:14:40.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f0be00640 1 -- 172.21.15.42:0/114380061 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6efc005470 con 0x7f6f140b02e0 2024-05-28T14:14:40.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f0be00640 1 -- 172.21.15.42:0/114380061 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc007260 con 0x7f6f140b02e0 2024-05-28T14:14:40.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f1a600640 1 -- 172.21.15.42:0/114380061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f141f8450 con 0x7f6f140b02e0 2024-05-28T14:14:40.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.646+0000 7f6f1a600640 1 -- 172.21.15.42:0/114380061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f140ce4c0 con 0x7f6f140b02e0 2024-05-28T14:14:40.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.650+0000 7f6f1a600640 1 -- 172.21.15.42:0/114380061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f14073f40 con 0x7f6f140b02e0 2024-05-28T14:14:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.650+0000 7f6f0be00640 1 -- 172.21.15.42:0/114380061 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f6efc007400 con 0x7f6f140b02e0 2024-05-28T14:14:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.650+0000 7f6f0be00640 1 --2- 172.21.15.42:0/114380061 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ef0069fa0 0x7f6ef006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.650+0000 7f6f0be00640 1 -- 172.21.15.42:0/114380061 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4913+0+0 (secure 0 0 0) 0x7f6efc084df0 con 0x7f6f140b02e0 2024-05-28T14:14:40.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.650+0000 7f6f13400640 1 --2- 172.21.15.42:0/114380061 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ef0069fa0 0x7f6ef006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.650+0000 7f6f13400640 1 --2- 172.21.15.42:0/114380061 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ef0069fa0 0x7f6ef006c460 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6f040ae000 tx=0x7f6f040a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:40.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.654+0000 7f6f0be00640 1 -- 172.21.15.42:0/114380061 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6f1415fb60 con 0x7f6f140b02e0 2024-05-28T14:14:40.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.822+0000 7f6f1a600640 1 -- 172.21.15.42:0/114380061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6f140d0900 con 0x7f6f140b02e0 2024-05-28T14:14:40.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.822+0000 7f6f0be00640 1 -- 172.21.15.42:0/114380061 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) ==== 93+0+10 (secure 0 0 0) 0x7f6efc056e10 con 0x7f6f140b02e0 2024-05-28T14:14:40.810 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:14:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.826+0000 7f6f1a600640 1 -- 172.21.15.42:0/114380061 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ef0069fa0 msgr2=0x7f6ef006c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.826+0000 7f6f1a600640 1 --2- 172.21.15.42:0/114380061 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ef0069fa0 0x7f6ef006c460 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6f040ae000 tx=0x7f6f040a7070 comp rx=0 tx=0).stop 2024-05-28T14:14:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.826+0000 7f6f1a600640 1 -- 172.21.15.42:0/114380061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f140b02e0 msgr2=0x7f6f140cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.826+0000 7f6f1a600640 1 --2- 172.21.15.42:0/114380061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f140b02e0 0x7f6f140cf040 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00c370 tx=0x7f6efc00c830 comp rx=0 tx=0).stop 2024-05-28T14:14:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.826+0000 7f6f1a600640 1 -- 172.21.15.42:0/114380061 shutdown_connections 2024-05-28T14:14:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.826+0000 7f6f1a600640 1 --2- 172.21.15.42:0/114380061 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6ef0069fa0 0x7f6ef006c460 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.826+0000 7f6f1a600640 1 --2- 172.21.15.42:0/114380061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1415f710 0x7f6f141f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.826+0000 7f6f1a600640 1 --2- 172.21.15.42:0/114380061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f140b02e0 0x7f6f140cf040 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.826+0000 7f6f1a600640 1 --2- 172.21.15.42:0/114380061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6f1406d110 0x7f6f140cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.826+0000 7f6f1a600640 1 -- 172.21.15.42:0/114380061 >> 172.21.15.42:0/114380061 conn(0x7f6f14072600 msgr2=0x7f6f141610c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.826+0000 7f6f1a600640 1 -- 172.21.15.42:0/114380061 shutdown_connections 2024-05-28T14:14:40.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.826+0000 7f6f1a600640 1 -- 172.21.15.42:0/114380061 wait complete. 2024-05-28T14:14:40.831 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-28T14:14:40.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.986+0000 7f61c5c00640 1 Processor -- start 2024-05-28T14:14:40.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.986+0000 7f61c5c00640 1 -- start start 2024-05-28T14:14:40.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.986+0000 7f61c5c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f61c00d0420 0x7f61c015ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.986+0000 7f61c5c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61c0067810 0x7f61c00b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.986+0000 7f61c5c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c006eaa0 con 0x7f61c00b4110 2024-05-28T14:14:40.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.986+0000 7f61c5c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f61c006ed80 con 0x7f61c00d0420 2024-05-28T14:14:40.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.986+0000 7f61c5c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f61c006f060 con 0x7f61c0067810 2024-05-28T14:14:40.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.986+0000 7f61bea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f61c00d0420 0x7f61c015ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.986+0000 7f61bea00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f61c00d0420 0x7f61c015ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34226/0 (socket says 172.21.15.42:34226) 2024-05-28T14:14:40.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.986+0000 7f61bea00640 1 -- 172.21.15.42:0/3107702547 learned_addr learned my addr 172.21.15.42:0/3107702547 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:40.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.986+0000 7f61bea00640 1 -- 172.21.15.42:0/3107702547 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61c0067810 msgr2=0x7f61c00b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:40.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.986+0000 7f61bea00640 1 --2- 172.21.15.42:0/3107702547 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61c0067810 0x7f61c00b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.986+0000 7f61bea00640 1 -- 172.21.15.42:0/3107702547 >> v1:172.21.15.63:6789/0 conn(0x7f61c00b4110 legacy=0x7f61c0162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:40.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.986+0000 7f61bea00640 1 -- 172.21.15.42:0/3107702547 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c006f340 con 0x7f61c00d0420 2024-05-28T14:14:40.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.990+0000 7f61bea00640 1 --2- 172.21.15.42:0/3107702547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61c00d0420 0x7f61c015ed10 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009870 tx=0x7f61b001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f91e80249e6ba42 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:40.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.990+0000 7f61be000640 1 -- 172.21.15.42:0/3107702547 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b00b0070 con 0x7f61c00d0420 2024-05-28T14:14:40.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.990+0000 7f61be000640 1 -- 172.21.15.42:0/3107702547 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b00ab600 con 0x7f61c00d0420 2024-05-28T14:14:40.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.990+0000 7f61be000640 1 -- 172.21.15.42:0/3107702547 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b00ab9d0 con 0x7f61c00d0420 2024-05-28T14:14:40.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.990+0000 7f61c5c00640 1 -- 172.21.15.42:0/3107702547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61c00d0420 msgr2=0x7f61c015ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.990+0000 7f61c5c00640 1 --2- 172.21.15.42:0/3107702547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61c00d0420 0x7f61c015ed10 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009870 tx=0x7f61b001c490 comp rx=0 tx=0).stop 2024-05-28T14:14:40.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.990+0000 7f61c5c00640 1 -- 172.21.15.42:0/3107702547 shutdown_connections 2024-05-28T14:14:40.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.990+0000 7f61c5c00640 1 --2- 172.21.15.42:0/3107702547 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61c0067810 0x7f61c00b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.990+0000 7f61c5c00640 1 --2- 172.21.15.42:0/3107702547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61c00d0420 0x7f61c015ed10 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.990+0000 7f61c5c00640 1 -- 172.21.15.42:0/3107702547 >> 172.21.15.42:0/3107702547 conn(0x7f61c00d0b40 msgr2=0x7f61c00d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:40.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.990+0000 7f61c5c00640 1 -- 172.21.15.42:0/3107702547 shutdown_connections 2024-05-28T14:14:40.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.990+0000 7f61c5c00640 1 -- 172.21.15.42:0/3107702547 wait complete. 2024-05-28T14:14:40.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61c5c00640 1 Processor -- start 2024-05-28T14:14:40.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61c5c00640 1 -- start start 2024-05-28T14:14:40.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61c5c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c0067810 0x7f61c014e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61c5c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61c00b4110 0x7f61c0155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61c5c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61c00d0420 0x7f61c014e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61c5c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61c016b640 con 0x7f61c00b4110 2024-05-28T14:14:40.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61c5c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f61c016b4c0 con 0x7f61c0067810 2024-05-28T14:14:40.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61c5c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f61c016b7c0 con 0x7f61c00d0420 2024-05-28T14:14:40.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61c00b4110 0x7f61c0155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61c00b4110 0x7f61c0155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34236/0 (socket says 172.21.15.42:34236) 2024-05-28T14:14:40.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61b7e00640 1 -- 172.21.15.42:0/1292469 learned_addr learned my addr 172.21.15.42:0/1292469 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:40.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61bea00640 1 --2- 172.21.15.42:0/1292469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c0067810 0x7f61c014e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61bf400640 1 --2- 172.21.15.42:0/1292469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61c00d0420 0x7f61c014e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61b7e00640 1 -- 172.21.15.42:0/1292469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61c00d0420 msgr2=0x7f61c014e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61b7e00640 1 --2- 172.21.15.42:0/1292469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61c00d0420 0x7f61c014e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61b7e00640 1 -- 172.21.15.42:0/1292469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c0067810 msgr2=0x7f61c014e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:40.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61b7e00640 1 --2- 172.21.15.42:0/1292469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c0067810 0x7f61c014e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:40.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61b7e00640 1 -- 172.21.15.42:0/1292469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c01566a0 con 0x7f61c00b4110 2024-05-28T14:14:40.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61b7e00640 1 --2- 172.21.15.42:0/1292469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61c00b4110 0x7f61c0155e50 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f61a800b550 tx=0x7f61a800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:40.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61b6a00640 1 -- 172.21.15.42:0/1292469 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8014070 con 0x7f61c00b4110 2024-05-28T14:14:40.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61b6a00640 1 -- 172.21.15.42:0/1292469 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a8010890 con 0x7f61c00b4110 2024-05-28T14:14:40.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61c5c00640 1 -- 172.21.15.42:0/1292469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c0156980 con 0x7f61c00b4110 2024-05-28T14:14:40.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61b6a00640 1 -- 172.21.15.42:0/1292469 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8010c80 con 0x7f61c00b4110 2024-05-28T14:14:40.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.994+0000 7f61c5c00640 1 -- 172.21.15.42:0/1292469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c0156f40 con 0x7f61c00b4110 2024-05-28T14:14:40.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.998+0000 7f61c5c00640 1 -- 172.21.15.42:0/1292469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c015a800 con 0x7f61c00b4110 2024-05-28T14:14:40.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.998+0000 7f61b6a00640 1 -- 172.21.15.42:0/1292469 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f61a8017070 con 0x7f61c00b4110 2024-05-28T14:14:40.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.998+0000 7f61b6a00640 1 --2- 172.21.15.42:0/1292469 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f619c069fa0 0x7f619c06c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:40.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.998+0000 7f61b6a00640 1 -- 172.21.15.42:0/1292469 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4913+0+0 (secure 0 0 0) 0x7f61a80855d0 con 0x7f61c00b4110 2024-05-28T14:14:40.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.998+0000 7f61bea00640 1 --2- 172.21.15.42:0/1292469 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f619c069fa0 0x7f619c06c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:40.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:40.998+0000 7f61bea00640 1 --2- 172.21.15.42:0/1292469 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f619c069fa0 0x7f619c06c460 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f61b00ad000 tx=0x7f61b00a6000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:40.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.002+0000 7f61b6a00640 1 -- 172.21.15.42:0/1292469 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f61c015a800 con 0x7f61c00b4110 2024-05-28T14:14:41.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.170+0000 7f61c5c00640 1 -- 172.21.15.42:0/1292469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f61c015ed00 con 0x7f61c00b4110 2024-05-28T14:14:41.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.170+0000 7f61b6a00640 1 -- 172.21.15.42:0/1292469 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v128) ==== 84+0+65 (secure 0 0 0) 0x7f61a8010e20 con 0x7f61c00b4110 2024-05-28T14:14:41.156 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:41.156 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:14:41.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:41.157 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-05-28T14:14:41.157 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-05-28T14:14:41.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:41.157 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:14:41.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.170+0000 7f61c5c00640 1 -- 172.21.15.42:0/1292469 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f619c069fa0 msgr2=0x7f619c06c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:41.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.170+0000 7f61c5c00640 1 --2- 172.21.15.42:0/1292469 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f619c069fa0 0x7f619c06c460 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f61b00ad000 tx=0x7f61b00a6000 comp rx=0 tx=0).stop 2024-05-28T14:14:41.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.170+0000 7f61c5c00640 1 -- 172.21.15.42:0/1292469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61c00b4110 msgr2=0x7f61c0155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:41.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.170+0000 7f61c5c00640 1 --2- 172.21.15.42:0/1292469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61c00b4110 0x7f61c0155e50 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f61a800b550 tx=0x7f61a800ba10 comp rx=0 tx=0).stop 2024-05-28T14:14:41.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.170+0000 7f61c5c00640 1 -- 172.21.15.42:0/1292469 shutdown_connections 2024-05-28T14:14:41.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.170+0000 7f61c5c00640 1 --2- 172.21.15.42:0/1292469 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f619c069fa0 0x7f619c06c460 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.170+0000 7f61c5c00640 1 --2- 172.21.15.42:0/1292469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61c00d0420 0x7f61c014e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.170+0000 7f61c5c00640 1 --2- 172.21.15.42:0/1292469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61c00b4110 0x7f61c0155e50 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.170+0000 7f61c5c00640 1 --2- 172.21.15.42:0/1292469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c0067810 0x7f61c014e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.170+0000 7f61c5c00640 1 -- 172.21.15.42:0/1292469 >> 172.21.15.42:0/1292469 conn(0x7f61c00d0b40 msgr2=0x7f61c015d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:41.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.170+0000 7f61c5c00640 1 -- 172.21.15.42:0/1292469 shutdown_connections 2024-05-28T14:14:41.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.170+0000 7f61c5c00640 1 -- 172.21.15.42:0/1292469 wait complete. 2024-05-28T14:14:41.178 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-28T14:14:41.179 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-28T14:14:41.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1108600640 1 Processor -- start 2024-05-28T14:14:41.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1108600640 1 -- start start 2024-05-28T14:14:41.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1108600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f110015b350 0x7f110006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:41.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1108600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1100072d40 0x7f11000d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:41.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1108600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11000ade40 con 0x7f11000737c0 2024-05-28T14:14:41.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1108600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f11000ae9f0 con 0x7f1100072d40 2024-05-28T14:14:41.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1108600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f11000ae0c0 con 0x7f110015b350 2024-05-28T14:14:41.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1106600640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f11000737c0 0x7f11000aee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37832/0 (socket says 172.21.15.42:37832) 2024-05-28T14:14:41.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1106600640 1 -- 172.21.15.42:0/2188571613 learned_addr learned my addr 172.21.15.42:0/2188571613 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:41.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1105c00640 1 --2- 172.21.15.42:0/2188571613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f110015b350 0x7f110006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:41.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1105200640 1 --2- 172.21.15.42:0/2188571613 >> v2:172.21.15.63:3300/0 conn(0x7f1100072d40 0x7f11000d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:41.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f10ffe00640 1 -- 172.21.15.42:0/2188571613 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1109083293 0 0) 0x7f10f80014a0 con 0x7f11000737c0 2024-05-28T14:14:41.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f10ffe00640 1 -- 172.21.15.42:0/2188571613 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10e8005740 con 0x7f11000737c0 2024-05-28T14:14:41.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1105c00640 1 -- 172.21.15.42:0/2188571613 >> v2:172.21.15.63:3300/0 conn(0x7f1100072d40 msgr2=0x7f11000d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:41.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1105c00640 1 --2- 172.21.15.42:0/2188571613 >> v2:172.21.15.63:3300/0 conn(0x7f1100072d40 0x7f11000d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1105c00640 1 -- 172.21.15.42:0/2188571613 >> v1:172.21.15.42:6789/0 conn(0x7f11000737c0 legacy=0x7f11000aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:41.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.338+0000 7f1105c00640 1 -- 172.21.15.42:0/2188571613 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11000ae3a0 con 0x7f110015b350 2024-05-28T14:14:41.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.342+0000 7f1105c00640 1 --2- 172.21.15.42:0/2188571613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f110015b350 0x7f110006dd40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f10f40097c0 tx=0x7f10f401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ac371881f99c331 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:41.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.342+0000 7f10ffe00640 1 -- 172.21.15.42:0/2188571613 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f40b0070 con 0x7f110015b350 2024-05-28T14:14:41.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.342+0000 7f10ffe00640 1 -- 172.21.15.42:0/2188571613 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f40ab600 con 0x7f110015b350 2024-05-28T14:14:41.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.342+0000 7f10ffe00640 1 -- 172.21.15.42:0/2188571613 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f40ab9d0 con 0x7f110015b350 2024-05-28T14:14:41.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.342+0000 7f1108600640 1 -- 172.21.15.42:0/2188571613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f110015b350 msgr2=0x7f110006dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:41.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.342+0000 7f1108600640 1 --2- 172.21.15.42:0/2188571613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f110015b350 0x7f110006dd40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f10f40097c0 tx=0x7f10f401c410 comp rx=0 tx=0).stop 2024-05-28T14:14:41.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.342+0000 7f1108600640 1 -- 172.21.15.42:0/2188571613 shutdown_connections 2024-05-28T14:14:41.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.342+0000 7f1108600640 1 --2- 172.21.15.42:0/2188571613 >> v2:172.21.15.63:3300/0 conn(0x7f1100072d40 0x7f11000d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.342+0000 7f1108600640 1 --2- 172.21.15.42:0/2188571613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f110015b350 0x7f110006dd40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.342+0000 7f1108600640 1 -- 172.21.15.42:0/2188571613 >> 172.21.15.42:0/2188571613 conn(0x7f110006bdf0 msgr2=0x7f110006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:41.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.342+0000 7f1108600640 1 -- 172.21.15.42:0/2188571613 shutdown_connections 2024-05-28T14:14:41.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.342+0000 7f1108600640 1 -- 172.21.15.42:0/2188571613 wait complete. 2024-05-28T14:14:41.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1108600640 1 Processor -- start 2024-05-28T14:14:41.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1108600640 1 -- start start 2024-05-28T14:14:41.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1108600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100072d40 0x7f110014ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:41.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1108600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11000737c0 0x7f1100153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:41.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1108600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f110015b350 0x7f11001535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:41.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1108600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f11000afb20 con 0x7f1100072d40 2024-05-28T14:14:41.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1108600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f11000af9a0 con 0x7f110015b350 2024-05-28T14:14:41.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1108600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f11000b3af0 con 0x7f11000737c0 2024-05-28T14:14:41.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1105c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100072d40 0x7f110014ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:41.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1105c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100072d40 0x7f110014ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34244/0 (socket says 172.21.15.42:34244) 2024-05-28T14:14:41.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1105c00640 1 -- 172.21.15.42:0/1022438509 learned_addr learned my addr 172.21.15.42:0/1022438509 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:41.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1105200640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11000737c0 0x7f1100153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:41.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1106600640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f110015b350 0x7f11001535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:41.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1105c00640 1 -- 172.21.15.42:0/1022438509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11000737c0 msgr2=0x7f1100153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:41.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1105c00640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11000737c0 0x7f1100153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1105c00640 1 -- 172.21.15.42:0/1022438509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f110015b350 msgr2=0x7f11001535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:41.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1105c00640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f110015b350 0x7f11001535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1105c00640 1 -- 172.21.15.42:0/1022438509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110014b140 con 0x7f1100072d40 2024-05-28T14:14:41.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1106600640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f110015b350 0x7f11001535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:41.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1105c00640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100072d40 0x7f110014ac00 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f10f40ae040 tx=0x7f10f401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f10fd600640 1 -- 172.21.15.42:0/1022438509 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f40b0050 con 0x7f1100072d40 2024-05-28T14:14:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f10fd600640 1 -- 172.21.15.42:0/1022438509 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f40bc070 con 0x7f1100072d40 2024-05-28T14:14:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1108600640 1 -- 172.21.15.42:0/1022438509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110015c650 con 0x7f1100072d40 2024-05-28T14:14:41.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f10fd600640 1 -- 172.21.15.42:0/1022438509 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4006e20 con 0x7f1100072d40 2024-05-28T14:14:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1108600640 1 -- 172.21.15.42:0/1022438509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f110015cf50 con 0x7f1100072d40 2024-05-28T14:14:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.346+0000 7f1108600640 1 -- 172.21.15.42:0/1022438509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1100068a90 con 0x7f1100072d40 2024-05-28T14:14:41.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.350+0000 7f10fd600640 1 -- 172.21.15.42:0/1022438509 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f10f40b5870 con 0x7f1100072d40 2024-05-28T14:14:41.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.350+0000 7f10fd600640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f10e0069fd0 0x7f10e006c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:41.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.350+0000 7f10fd600640 1 -- 172.21.15.42:0/1022438509 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4913+0+0 (secure 0 0 0) 0x7f10f411fff0 con 0x7f1100072d40 2024-05-28T14:14:41.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.350+0000 7f1105200640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f10e0069fd0 0x7f10e006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:41.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.350+0000 7f1105200640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f10e0069fd0 0x7f10e006c490 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f110009e510 tx=0x7f10ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:41.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.354+0000 7f10fd600640 1 -- 172.21.15.42:0/1022438509 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f10f4016090 con 0x7f1100072d40 2024-05-28T14:14:41.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.522+0000 7f1108600640 1 -- 172.21.15.42:0/1022438509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1100072b20 con 0x7f1100072d40 2024-05-28T14:14:41.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.710+0000 7f10fd600640 1 -- 172.21.15.42:0/1022438509 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v129) ==== 138+0+0 (secure 0 0 0) 0x7f10f40f2010 con 0x7f1100072d40 2024-05-28T14:14:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-05-28T14:14:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.714+0000 7f1108600640 1 -- 172.21.15.42:0/1022438509 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f10e0069fd0 msgr2=0x7f10e006c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.714+0000 7f1108600640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f10e0069fd0 0x7f10e006c490 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f110009e510 tx=0x7f10ec008040 comp rx=0 tx=0).stop 2024-05-28T14:14:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.714+0000 7f1108600640 1 -- 172.21.15.42:0/1022438509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100072d40 msgr2=0x7f110014ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.714+0000 7f1108600640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100072d40 0x7f110014ac00 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f10f40ae040 tx=0x7f10f401ac50 comp rx=0 tx=0).stop 2024-05-28T14:14:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.714+0000 7f1108600640 1 -- 172.21.15.42:0/1022438509 shutdown_connections 2024-05-28T14:14:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.714+0000 7f1108600640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f10e0069fd0 0x7f10e006c490 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.714+0000 7f1108600640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f110015b350 0x7f11001535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.714+0000 7f1108600640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11000737c0 0x7f1100153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.714+0000 7f1108600640 1 --2- 172.21.15.42:0/1022438509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100072d40 0x7f110014ac00 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.714+0000 7f1108600640 1 -- 172.21.15.42:0/1022438509 >> 172.21.15.42:0/1022438509 conn(0x7f110006bdf0 msgr2=0x7f11000cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:41.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.714+0000 7f1108600640 1 -- 172.21.15.42:0/1022438509 shutdown_connections 2024-05-28T14:14:41.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.714+0000 7f1108600640 1 -- 172.21.15.42:0/1022438509 wait complete. 2024-05-28T14:14:41.725 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-28T14:14:41.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.886+0000 7ffa14e00640 1 Processor -- start 2024-05-28T14:14:41.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.886+0000 7ffa14e00640 1 -- start start 2024-05-28T14:14:41.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.890+0000 7ffa14e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffa1006c5f0 0x7ffa100cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:41.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.890+0000 7ffa14e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa1015cbc0 0x7ffa1015d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:41.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.890+0000 7ffa14e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa100d0e50 con 0x7ffa1006b9f0 2024-05-28T14:14:41.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.890+0000 7ffa14e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ffa100ce100 con 0x7ffa1006c5f0 2024-05-28T14:14:41.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.890+0000 7ffa14e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa100cffd0 con 0x7ffa1015cbc0 2024-05-28T14:14:41.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.890+0000 7ffa0f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffa1006c5f0 0x7ffa100cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:41.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.890+0000 7ffa0f400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffa1006c5f0 0x7ffa100cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34260/0 (socket says 172.21.15.42:34260) 2024-05-28T14:14:41.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.890+0000 7ffa0f400640 1 -- 172.21.15.42:0/1186484415 learned_addr learned my addr 172.21.15.42:0/1186484415 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:41.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.890+0000 7ffa0fe00640 1 --2- 172.21.15.42:0/1186484415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa1015cbc0 0x7ffa1015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:41.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.890+0000 7ffa0f400640 1 -- 172.21.15.42:0/1186484415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa1015cbc0 msgr2=0x7ffa1015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:41.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.890+0000 7ffa0f400640 1 --2- 172.21.15.42:0/1186484415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa1015cbc0 0x7ffa1015d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.890+0000 7ffa0f400640 1 -- 172.21.15.42:0/1186484415 >> v1:172.21.15.63:6789/0 conn(0x7ffa1006b9f0 legacy=0x7ffa100d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:41.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.890+0000 7ffa0f400640 1 -- 172.21.15.42:0/1186484415 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1015d570 con 0x7ffa1006c5f0 2024-05-28T14:14:41.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.890+0000 7ffa0f400640 1 --2- 172.21.15.42:0/1186484415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa1006c5f0 0x7ffa100cdbc0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009e50 tx=0x7ffa0001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ac222af740ac0c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:41.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa0ea00640 1 -- 172.21.15.42:0/1186484415 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa000a5550 con 0x7ffa1006c5f0 2024-05-28T14:14:41.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa0ea00640 1 -- 172.21.15.42:0/1186484415 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa000a56f0 con 0x7ffa1006c5f0 2024-05-28T14:14:41.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa0ea00640 1 -- 172.21.15.42:0/1186484415 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa000a5ac0 con 0x7ffa1006c5f0 2024-05-28T14:14:41.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 -- 172.21.15.42:0/1186484415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa1006c5f0 msgr2=0x7ffa100cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:41.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 --2- 172.21.15.42:0/1186484415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa1006c5f0 0x7ffa100cdbc0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009e50 tx=0x7ffa0001c8d0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 -- 172.21.15.42:0/1186484415 shutdown_connections 2024-05-28T14:14:41.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 --2- 172.21.15.42:0/1186484415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa1015cbc0 0x7ffa1015d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 --2- 172.21.15.42:0/1186484415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa1006c5f0 0x7ffa100cdbc0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 -- 172.21.15.42:0/1186484415 >> 172.21.15.42:0/1186484415 conn(0x7ffa100b2d70 msgr2=0x7ffa100b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:41.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 -- 172.21.15.42:0/1186484415 shutdown_connections 2024-05-28T14:14:41.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 -- 172.21.15.42:0/1186484415 wait complete. 2024-05-28T14:14:41.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 Processor -- start 2024-05-28T14:14:41.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 -- start start 2024-05-28T14:14:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa1006b9f0 0x7ffa10151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa1006c5f0 0x7ffa1014f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa1015cbc0 0x7ffa100c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffa100cd510 con 0x7ffa1015cbc0 2024-05-28T14:14:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa100d0220 con 0x7ffa1006c5f0 2024-05-28T14:14:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa14e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa100cd690 con 0x7ffa1006b9f0 2024-05-28T14:14:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa0fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa1015cbc0 0x7ffa100c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa0fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa1015cbc0 0x7ffa100c5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34262/0 (socket says 172.21.15.42:34262) 2024-05-28T14:14:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa0fe00640 1 -- 172.21.15.42:0/1927224646 learned_addr learned my addr 172.21.15.42:0/1927224646 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa06a00640 1 --2- 172.21.15.42:0/1927224646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa1006c5f0 0x7ffa1014f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.894+0000 7ffa0f400640 1 --2- 172.21.15.42:0/1927224646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa1006b9f0 0x7ffa10151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.898+0000 7ffa0fe00640 1 -- 172.21.15.42:0/1927224646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa1006b9f0 msgr2=0x7ffa10151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.898+0000 7ffa0fe00640 1 --2- 172.21.15.42:0/1927224646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa1006b9f0 0x7ffa10151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.898+0000 7ffa0fe00640 1 -- 172.21.15.42:0/1927224646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa1006c5f0 msgr2=0x7ffa1014f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.898+0000 7ffa0fe00640 1 --2- 172.21.15.42:0/1927224646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa1006c5f0 0x7ffa1014f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.898+0000 7ffa0fe00640 1 -- 172.21.15.42:0/1927224646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa100c6660 con 0x7ffa1015cbc0 2024-05-28T14:14:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.898+0000 7ffa0fe00640 1 --2- 172.21.15.42:0/1927224646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa1015cbc0 0x7ffa100c5e10 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800d800 tx=0x7ffa0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.898+0000 7ffa07e00640 1 -- 172.21.15.42:0/1927224646 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0800b6c0 con 0x7ffa1015cbc0 2024-05-28T14:14:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.898+0000 7ffa07e00640 1 -- 172.21.15.42:0/1927224646 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa08019070 con 0x7ffa1015cbc0 2024-05-28T14:14:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.898+0000 7ffa14e00640 1 -- 172.21.15.42:0/1927224646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa10151280 con 0x7ffa1015cbc0 2024-05-28T14:14:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.898+0000 7ffa07e00640 1 -- 172.21.15.42:0/1927224646 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0800baa0 con 0x7ffa1015cbc0 2024-05-28T14:14:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.898+0000 7ffa14e00640 1 -- 172.21.15.42:0/1927224646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa10151840 con 0x7ffa1015cbc0 2024-05-28T14:14:41.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.898+0000 7ffa14e00640 1 -- 172.21.15.42:0/1927224646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa1004fcf0 con 0x7ffa1015cbc0 2024-05-28T14:14:41.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.898+0000 7ffa07e00640 1 -- 172.21.15.42:0/1927224646 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7ffa0800bc40 con 0x7ffa1015cbc0 2024-05-28T14:14:41.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.902+0000 7ffa07e00640 1 --2- 172.21.15.42:0/1927224646 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9ec069fa0 0x7ff9ec06c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:41.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.902+0000 7ffa07e00640 1 -- 172.21.15.42:0/1927224646 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5283+0+0 (secure 0 0 0) 0x7ffa080866e0 con 0x7ffa1015cbc0 2024-05-28T14:14:41.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.902+0000 7ffa0f400640 1 --2- 172.21.15.42:0/1927224646 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9ec069fa0 0x7ff9ec06c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:41.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.902+0000 7ffa0f400640 1 --2- 172.21.15.42:0/1927224646 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9ec069fa0 0x7ff9ec06c460 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ffa000ae000 tx=0x7ffa000a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:41.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:41.902+0000 7ffa07e00640 1 -- 172.21.15.42:0/1927224646 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffa08058130 con 0x7ffa1015cbc0 2024-05-28T14:14:42.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.070+0000 7ffa14e00640 1 -- 172.21.15.42:0/1927224646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ffa100cb7c0 con 0x7ffa1015cbc0 2024-05-28T14:14:42.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.714+0000 7ffa07e00640 1 -- 172.21.15.42:0/1927224646 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v130) ==== 170+0+0 (secure 0 0 0) 0x7ffa0805c430 con 0x7ffa1015cbc0 2024-05-28T14:14:42.701 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-05-28T14:14:42.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.718+0000 7ffa14e00640 1 -- 172.21.15.42:0/1927224646 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9ec069fa0 msgr2=0x7ff9ec06c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:42.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.718+0000 7ffa14e00640 1 --2- 172.21.15.42:0/1927224646 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9ec069fa0 0x7ff9ec06c460 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ffa000ae000 tx=0x7ffa000a7070 comp rx=0 tx=0).stop 2024-05-28T14:14:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.718+0000 7ffa14e00640 1 -- 172.21.15.42:0/1927224646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa1015cbc0 msgr2=0x7ffa100c5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.718+0000 7ffa14e00640 1 --2- 172.21.15.42:0/1927224646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa1015cbc0 0x7ffa100c5e10 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800d800 tx=0x7ffa0800dcc0 comp rx=0 tx=0).stop 2024-05-28T14:14:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.718+0000 7ffa14e00640 1 -- 172.21.15.42:0/1927224646 shutdown_connections 2024-05-28T14:14:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.718+0000 7ffa14e00640 1 --2- 172.21.15.42:0/1927224646 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff9ec069fa0 0x7ff9ec06c460 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.718+0000 7ffa14e00640 1 --2- 172.21.15.42:0/1927224646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa1015cbc0 0x7ffa100c5e10 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.718+0000 7ffa14e00640 1 --2- 172.21.15.42:0/1927224646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa1006c5f0 0x7ffa1014f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.718+0000 7ffa14e00640 1 --2- 172.21.15.42:0/1927224646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa1006b9f0 0x7ffa10151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.718+0000 7ffa14e00640 1 -- 172.21.15.42:0/1927224646 >> 172.21.15.42:0/1927224646 conn(0x7ffa100b2d70 msgr2=0x7ffa10150020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.718+0000 7ffa14e00640 1 -- 172.21.15.42:0/1927224646 shutdown_connections 2024-05-28T14:14:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.718+0000 7ffa14e00640 1 -- 172.21.15.42:0/1927224646 wait complete. 2024-05-28T14:14:42.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-28T14:14:42.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.902+0000 7efeb1c00640 1 Processor -- start 2024-05-28T14:14:42.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.902+0000 7efeb1c00640 1 -- start start 2024-05-28T14:14:42.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.906+0000 7efeb1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efeac071de0 0x7efeac0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:42.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.906+0000 7efeb1c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efeac0ce3b0 0x7efeac0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:42.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.906+0000 7efeb1c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeac072700 con 0x7efeac141d10 2024-05-28T14:14:42.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.906+0000 7efeb1c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7efeac160230 con 0x7efeac0ce3b0 2024-05-28T14:14:42.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.906+0000 7efeb1c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efeac160510 con 0x7efeac071de0 2024-05-28T14:14:42.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.906+0000 7efeaa000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efeac0ce3b0 0x7efeac0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:42.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.906+0000 7efeaa000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efeac0ce3b0 0x7efeac0ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34278/0 (socket says 172.21.15.42:34278) 2024-05-28T14:14:42.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.906+0000 7efeaa000640 1 -- 172.21.15.42:0/2351894494 learned_addr learned my addr 172.21.15.42:0/2351894494 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:42.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.906+0000 7efeaaa00640 1 --2- 172.21.15.42:0/2351894494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efeac071de0 0x7efeac0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:42.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.906+0000 7efeaa000640 1 -- 172.21.15.42:0/2351894494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efeac071de0 msgr2=0x7efeac0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:42.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.906+0000 7efeaa000640 1 --2- 172.21.15.42:0/2351894494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efeac071de0 0x7efeac0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:42.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.906+0000 7efeaa000640 1 -- 172.21.15.42:0/2351894494 >> v1:172.21.15.42:6789/0 conn(0x7efeac141d10 legacy=0x7efeac1421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:42.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.906+0000 7efeaa000640 1 -- 172.21.15.42:0/2351894494 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac1607f0 con 0x7efeac0ce3b0 2024-05-28T14:14:42.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.910+0000 7efeaa000640 1 --2- 172.21.15.42:0/2351894494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeac0ce3b0 0x7efeac0ce810 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7efe98009870 tx=0x7efe9801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ae48693f72028f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:42.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.910+0000 7efea9600640 1 -- 172.21.15.42:0/2351894494 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe980b0070 con 0x7efeac0ce3b0 2024-05-28T14:14:42.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.910+0000 7efea9600640 1 -- 172.21.15.42:0/2351894494 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe980ab600 con 0x7efeac0ce3b0 2024-05-28T14:14:42.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.910+0000 7efea9600640 1 -- 172.21.15.42:0/2351894494 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe980ab9d0 con 0x7efeac0ce3b0 2024-05-28T14:14:42.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.910+0000 7efeb1c00640 1 -- 172.21.15.42:0/2351894494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeac0ce3b0 msgr2=0x7efeac0ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:42.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.910+0000 7efeb1c00640 1 --2- 172.21.15.42:0/2351894494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeac0ce3b0 0x7efeac0ce810 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7efe98009870 tx=0x7efe9801c440 comp rx=0 tx=0).stop 2024-05-28T14:14:42.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.910+0000 7efeb1c00640 1 -- 172.21.15.42:0/2351894494 shutdown_connections 2024-05-28T14:14:42.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.910+0000 7efeb1c00640 1 --2- 172.21.15.42:0/2351894494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeac0ce3b0 0x7efeac0ce810 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:42.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.910+0000 7efeb1c00640 1 --2- 172.21.15.42:0/2351894494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efeac071de0 0x7efeac0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:42.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.910+0000 7efeb1c00640 1 -- 172.21.15.42:0/2351894494 >> 172.21.15.42:0/2351894494 conn(0x7efeac0b21f0 msgr2=0x7efeac0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:42.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.910+0000 7efeb1c00640 1 -- 172.21.15.42:0/2351894494 shutdown_connections 2024-05-28T14:14:42.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.910+0000 7efeb1c00640 1 -- 172.21.15.42:0/2351894494 wait complete. 2024-05-28T14:14:42.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.910+0000 7efeb1c00640 1 Processor -- start 2024-05-28T14:14:42.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.910+0000 7efeb1c00640 1 -- start start 2024-05-28T14:14:42.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeb1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac071de0 0x7efeac137bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:42.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeb1c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeac0ce3b0 0x7efeac1f98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:42.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeb1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efeac141d10 0x7efeac136c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:42.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeb1c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efeac169e30 con 0x7efeac0ce3b0 2024-05-28T14:14:42.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeb1c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efeac169cb0 con 0x7efeac071de0 2024-05-28T14:14:42.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeb1c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efeac169fb0 con 0x7efeac141d10 2024-05-28T14:14:42.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeaa000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeac0ce3b0 0x7efeac1f98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:42.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeaa000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeac0ce3b0 0x7efeac1f98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34284/0 (socket says 172.21.15.42:34284) 2024-05-28T14:14:42.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeaa000640 1 -- 172.21.15.42:0/562131853 learned_addr learned my addr 172.21.15.42:0/562131853 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:42.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeaaa00640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac071de0 0x7efeac137bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:42.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeab400640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efeac141d10 0x7efeac136c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:42.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeaa000640 1 -- 172.21.15.42:0/562131853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efeac141d10 msgr2=0x7efeac136c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:42.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeaa000640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efeac141d10 0x7efeac136c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:42.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeaa000640 1 -- 172.21.15.42:0/562131853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac071de0 msgr2=0x7efeac137bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:42.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeaa000640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac071de0 0x7efeac137bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:42.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeaa000640 1 -- 172.21.15.42:0/562131853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac155510 con 0x7efeac0ce3b0 2024-05-28T14:14:42.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeaaa00640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac071de0 0x7efeac137bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:42.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeab400640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efeac141d10 0x7efeac136c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:42.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeaa000640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeac0ce3b0 0x7efeac1f98e0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7efe980ae040 tx=0x7efe9801ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:42.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efea2a00640 1 -- 172.21.15.42:0/562131853 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe980ba020 con 0x7efeac0ce3b0 2024-05-28T14:14:42.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efea2a00640 1 -- 172.21.15.42:0/562131853 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe980b6420 con 0x7efeac0ce3b0 2024-05-28T14:14:42.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efea2a00640 1 -- 172.21.15.42:0/562131853 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe980b0040 con 0x7efeac0ce3b0 2024-05-28T14:14:42.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeb1c00640 1 -- 172.21.15.42:0/562131853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeac1fa3a0 con 0x7efeac0ce3b0 2024-05-28T14:14:42.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeb1c00640 1 -- 172.21.15.42:0/562131853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeac1557d0 con 0x7efeac0ce3b0 2024-05-28T14:14:42.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.914+0000 7efeb1c00640 1 -- 172.21.15.42:0/562131853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeac0738b0 con 0x7efeac0ce3b0 2024-05-28T14:14:42.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.918+0000 7efea2a00640 1 -- 172.21.15.42:0/562131853 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7efe980bd020 con 0x7efeac0ce3b0 2024-05-28T14:14:42.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.918+0000 7efea2a00640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe8c069fd0 0x7efe8c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:42.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.918+0000 7efea2a00640 1 -- 172.21.15.42:0/562131853 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5665+0+0 (secure 0 0 0) 0x7efe9811fd90 con 0x7efeac0ce3b0 2024-05-28T14:14:42.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.918+0000 7efeaaa00640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe8c069fd0 0x7efe8c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:42.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.918+0000 7efeaaa00640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe8c069fd0 0x7efe8c06c490 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009a40 tx=0x7efe9c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:42.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:42.922+0000 7efea2a00640 1 -- 172.21.15.42:0/562131853 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efe980c4030 con 0x7efeac0ce3b0 2024-05-28T14:14:43.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.122+0000 7efeb1c00640 1 -- 172.21.15.42:0/562131853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efeac04fdd0 con 0x7efeac0ce3b0 2024-05-28T14:14:43.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.742+0000 7efea2a00640 1 -- 172.21.15.42:0/562131853 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v80) ==== 174+0+0 (secure 0 0 0) 0x7efe98016150 con 0x7efeac0ce3b0 2024-05-28T14:14:43.736 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 18 and data pool 19 2024-05-28T14:14:43.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.754+0000 7efeb1c00640 1 -- 172.21.15.42:0/562131853 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe8c069fd0 msgr2=0x7efe8c06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:43.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.754+0000 7efeb1c00640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe8c069fd0 0x7efe8c06c490 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009a40 tx=0x7efe9c008040 comp rx=0 tx=0).stop 2024-05-28T14:14:43.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.754+0000 7efeb1c00640 1 -- 172.21.15.42:0/562131853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeac0ce3b0 msgr2=0x7efeac1f98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:43.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.754+0000 7efeb1c00640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeac0ce3b0 0x7efeac1f98e0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7efe980ae040 tx=0x7efe9801ac80 comp rx=0 tx=0).stop 2024-05-28T14:14:43.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.754+0000 7efeb1c00640 1 -- 172.21.15.42:0/562131853 shutdown_connections 2024-05-28T14:14:43.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.754+0000 7efeb1c00640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7efe8c069fd0 0x7efe8c06c490 secure :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009a40 tx=0x7efe9c008040 comp rx=0 tx=0).stop 2024-05-28T14:14:43.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.754+0000 7efeb1c00640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efeac141d10 0x7efeac136c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:43.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.754+0000 7efeb1c00640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efeac0ce3b0 0x7efeac1f98e0 secure :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7efe980ae040 tx=0x7efe9801ac80 comp rx=0 tx=0).stop 2024-05-28T14:14:43.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.754+0000 7efeb1c00640 1 --2- 172.21.15.42:0/562131853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac071de0 0x7efeac137bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:43.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.754+0000 7efeb1c00640 1 -- 172.21.15.42:0/562131853 >> 172.21.15.42:0/562131853 conn(0x7efeac0b21f0 msgr2=0x7efeac15ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:43.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.758+0000 7efeb1c00640 1 -- 172.21.15.42:0/562131853 shutdown_connections 2024-05-28T14:14:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.758+0000 7efeb1c00640 1 -- 172.21.15.42:0/562131853 wait complete. 2024-05-28T14:14:43.776 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:14:43.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3706a00640 1 Processor -- start 2024-05-28T14:14:43.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3706a00640 1 -- start start 2024-05-28T14:14:43.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3706a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f370009aa50 0x7f370009ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:43.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3706a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f37000cff00 0x7f37000d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:43.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3706a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f370015edb0 con 0x7f37000d1ae0 2024-05-28T14:14:43.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3706a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f3700071560 con 0x7f370009aa50 2024-05-28T14:14:43.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3706a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f370009b3d0 con 0x7f37000cff00 2024-05-28T14:14:43.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3704c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f370009aa50 0x7f370009ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:43.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3704c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f370009aa50 0x7f370009ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34296/0 (socket says 172.21.15.42:34296) 2024-05-28T14:14:43.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3704c00640 1 -- 172.21.15.42:0/1738000135 learned_addr learned my addr 172.21.15.42:0/1738000135 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:43.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3706000640 1 --2- 172.21.15.42:0/1738000135 >> v2:172.21.15.63:3300/0 conn(0x7f37000cff00 0x7f37000d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:43.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3704c00640 1 -- 172.21.15.42:0/1738000135 >> v2:172.21.15.63:3300/0 conn(0x7f37000cff00 msgr2=0x7f37000d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:43.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3704c00640 1 --2- 172.21.15.42:0/1738000135 >> v2:172.21.15.63:3300/0 conn(0x7f37000cff00 0x7f37000d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:43.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3704c00640 1 -- 172.21.15.42:0/1738000135 >> v1:172.21.15.63:6789/0 conn(0x7f37000d1ae0 legacy=0x7f370006d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:43.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.934+0000 7f3704c00640 1 -- 172.21.15.42:0/1738000135 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3700070950 con 0x7f370009aa50 2024-05-28T14:14:43.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.938+0000 7f3704c00640 1 --2- 172.21.15.42:0/1738000135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f370009aa50 0x7f370009ae90 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f3700163560 tx=0x7f36ec01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d162169bb815351 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:43.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.938+0000 7f36ffe00640 1 -- 172.21.15.42:0/1738000135 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec0b5070 con 0x7f370009aa50 2024-05-28T14:14:43.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.938+0000 7f36ffe00640 1 -- 172.21.15.42:0/1738000135 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36ec0a8a10 con 0x7f370009aa50 2024-05-28T14:14:43.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.938+0000 7f36ffe00640 1 -- 172.21.15.42:0/1738000135 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec0a8de0 con 0x7f370009aa50 2024-05-28T14:14:43.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.938+0000 7f3706a00640 1 -- 172.21.15.42:0/1738000135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f370009aa50 msgr2=0x7f370009ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:43.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.938+0000 7f3706a00640 1 --2- 172.21.15.42:0/1738000135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f370009aa50 0x7f370009ae90 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f3700163560 tx=0x7f36ec01ed10 comp rx=0 tx=0).stop 2024-05-28T14:14:43.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.938+0000 7f3706a00640 1 -- 172.21.15.42:0/1738000135 shutdown_connections 2024-05-28T14:14:43.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.938+0000 7f3706a00640 1 --2- 172.21.15.42:0/1738000135 >> v2:172.21.15.63:3300/0 conn(0x7f37000cff00 0x7f37000d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:43.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.938+0000 7f3706a00640 1 --2- 172.21.15.42:0/1738000135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f370009aa50 0x7f370009ae90 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:43.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.938+0000 7f3706a00640 1 -- 172.21.15.42:0/1738000135 >> 172.21.15.42:0/1738000135 conn(0x7f37000d2330 msgr2=0x7f37000d2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:43.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.938+0000 7f3706a00640 1 -- 172.21.15.42:0/1738000135 shutdown_connections 2024-05-28T14:14:43.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.938+0000 7f3706a00640 1 -- 172.21.15.42:0/1738000135 wait complete. 2024-05-28T14:14:43.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.938+0000 7f3706a00640 1 Processor -- start 2024-05-28T14:14:43.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.938+0000 7f3706a00640 1 -- start start 2024-05-28T14:14:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f370009aa50 0x7f37000b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37000cff00 0x7f3700159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37000d1ae0 0x7f370014f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f37000711f0 con 0x7f37000d1ae0 2024-05-28T14:14:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3700070f10 con 0x7f370009aa50 2024-05-28T14:14:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f370009bc00 con 0x7f37000cff00 2024-05-28T14:14:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37000d1ae0 0x7f370014f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37000d1ae0 0x7f370014f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34300/0 (socket says 172.21.15.42:34300) 2024-05-28T14:14:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706000640 1 -- 172.21.15.42:0/1010338661 learned_addr learned my addr 172.21.15.42:0/1010338661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3705600640 1 --2- 172.21.15.42:0/1010338661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f370009aa50 0x7f37000b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3704c00640 1 --2- 172.21.15.42:0/1010338661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37000cff00 0x7f3700159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:43.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706000640 1 -- 172.21.15.42:0/1010338661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37000cff00 msgr2=0x7f3700159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:43.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706000640 1 --2- 172.21.15.42:0/1010338661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37000cff00 0x7f3700159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:43.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706000640 1 -- 172.21.15.42:0/1010338661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f370009aa50 msgr2=0x7f37000b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:43.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706000640 1 --2- 172.21.15.42:0/1010338661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f370009aa50 0x7f37000b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:43.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706000640 1 -- 172.21.15.42:0/1010338661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37000b56e0 con 0x7f37000d1ae0 2024-05-28T14:14:43.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706000640 1 --2- 172.21.15.42:0/1010338661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37000d1ae0 0x7f370014f260 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f36f8007e40 tx=0x7f36f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:43.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f36fd600640 1 -- 172.21.15.42:0/1010338661 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8014620 con 0x7f37000d1ae0 2024-05-28T14:14:43.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f36fd600640 1 -- 172.21.15.42:0/1010338661 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f8019070 con 0x7f37000d1ae0 2024-05-28T14:14:43.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f36fd600640 1 -- 172.21.15.42:0/1010338661 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8014a20 con 0x7f37000d1ae0 2024-05-28T14:14:43.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706a00640 1 -- 172.21.15.42:0/1010338661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37000b59c0 con 0x7f37000d1ae0 2024-05-28T14:14:43.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706a00640 1 -- 172.21.15.42:0/1010338661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37000b5f80 con 0x7f37000d1ae0 2024-05-28T14:14:43.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.942+0000 7f3706a00640 1 -- 172.21.15.42:0/1010338661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f370004fcf0 con 0x7f37000d1ae0 2024-05-28T14:14:43.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.946+0000 7f36fd600640 1 -- 172.21.15.42:0/1010338661 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f36f8014bc0 con 0x7f37000d1ae0 2024-05-28T14:14:43.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.946+0000 7f36fd600640 1 --2- 172.21.15.42:0/1010338661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36e0069fd0 0x7f36e006c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:43.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.946+0000 7f36fd600640 1 -- 172.21.15.42:0/1010338661 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5765+0+0 (secure 0 0 0) 0x7f36f80860b0 con 0x7f37000d1ae0 2024-05-28T14:14:43.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.946+0000 7f3705600640 1 --2- 172.21.15.42:0/1010338661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36e0069fd0 0x7f36e006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:43.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.946+0000 7f3705600640 1 --2- 172.21.15.42:0/1010338661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36e0069fd0 0x7f36e006c490 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f36f4001770 tx=0x7f36f4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:43.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:43.950+0000 7f36fd600640 1 -- 172.21.15.42:0/1010338661 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f36f8057a80 con 0x7f37000d1ae0 2024-05-28T14:14:44.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.114+0000 7f3706a00640 1 -- 172.21.15.42:0/1010338661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37000d2f90 con 0x7f37000d1ae0 2024-05-28T14:14:44.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.114+0000 7f36fd600640 1 -- 172.21.15.42:0/1010338661 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16720 (secure 0 0 0) 0x7f36f8075160 con 0x7f37000d1ae0 2024-05-28T14:14:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:44.103 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":131,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:14:43.128139+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:14:41.538199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-28T14:14:42.086660+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:14:44.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.118+0000 7f3706a00640 1 -- 172.21.15.42:0/1010338661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36e0069fd0 msgr2=0x7f36e006c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:44.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.118+0000 7f3706a00640 1 --2- 172.21.15.42:0/1010338661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36e0069fd0 0x7f36e006c490 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f36f4001770 tx=0x7f36f4009040 comp rx=0 tx=0).stop 2024-05-28T14:14:44.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.118+0000 7f3706a00640 1 -- 172.21.15.42:0/1010338661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37000d1ae0 msgr2=0x7f370014f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:44.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.118+0000 7f3706a00640 1 --2- 172.21.15.42:0/1010338661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37000d1ae0 0x7f370014f260 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f36f8007e40 tx=0x7f36f800e4b0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.118+0000 7f3706a00640 1 -- 172.21.15.42:0/1010338661 shutdown_connections 2024-05-28T14:14:44.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.118+0000 7f3706a00640 1 --2- 172.21.15.42:0/1010338661 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f36e0069fd0 0x7f36e006c490 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.118+0000 7f3706a00640 1 --2- 172.21.15.42:0/1010338661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37000d1ae0 0x7f370014f260 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.118+0000 7f3706a00640 1 --2- 172.21.15.42:0/1010338661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37000cff00 0x7f3700159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.118+0000 7f3706a00640 1 --2- 172.21.15.42:0/1010338661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f370009aa50 0x7f37000b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.118+0000 7f3706a00640 1 -- 172.21.15.42:0/1010338661 >> 172.21.15.42:0/1010338661 conn(0x7f37000d2330 msgr2=0x7f37000c71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:44.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.118+0000 7f3706a00640 1 -- 172.21.15.42:0/1010338661 shutdown_connections 2024-05-28T14:14:44.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.118+0000 7f3706a00640 1 -- 172.21.15.42:0/1010338661 wait complete. 2024-05-28T14:14:44.125 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:14:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.278+0000 7fea79000640 1 Processor -- start 2024-05-28T14:14:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.278+0000 7fea79000640 1 -- start start 2024-05-28T14:14:44.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea79000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea7415ba40 0x7fea740d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea79000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fea74072dd0 0x7fea740d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea79000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea7415f280 con 0x7fea7406f660 2024-05-28T14:14:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea79000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fea7415f030 con 0x7fea74072dd0 2024-05-28T14:14:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea79000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea7415f560 con 0x7fea7415ba40 2024-05-28T14:14:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea72800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fea74072dd0 0x7fea740d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea72800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fea74072dd0 0x7fea740d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34314/0 (socket says 172.21.15.42:34314) 2024-05-28T14:14:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea72800640 1 -- 172.21.15.42:0/1104333874 learned_addr learned my addr 172.21.15.42:0/1104333874 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea69400640 1 --2- 172.21.15.42:0/1104333874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea7415ba40 0x7fea740d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea72800640 1 -- 172.21.15.42:0/1104333874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea7415ba40 msgr2=0x7fea740d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea72800640 1 --2- 172.21.15.42:0/1104333874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea7415ba40 0x7fea740d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea72800640 1 -- 172.21.15.42:0/1104333874 >> v1:172.21.15.42:6789/0 conn(0x7fea7406f660 legacy=0x7fea74073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea72800640 1 -- 172.21.15.42:0/1104333874 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7415f840 con 0x7fea74072dd0 2024-05-28T14:14:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea72800640 1 --2- 172.21.15.42:0/1104333874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea74072dd0 0x7fea740d0c30 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009870 tx=0x7fea6c01c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2525989f79f86602 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea71400640 1 -- 172.21.15.42:0/1104333874 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0b1070 con 0x7fea74072dd0 2024-05-28T14:14:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea71400640 1 -- 172.21.15.42:0/1104333874 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea6c0a4930 con 0x7fea74072dd0 2024-05-28T14:14:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.282+0000 7fea71400640 1 -- 172.21.15.42:0/1104333874 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0a4d00 con 0x7fea74072dd0 2024-05-28T14:14:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 -- 172.21.15.42:0/1104333874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea74072dd0 msgr2=0x7fea740d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 --2- 172.21.15.42:0/1104333874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea74072dd0 0x7fea740d0c30 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009870 tx=0x7fea6c01c0f0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 -- 172.21.15.42:0/1104333874 shutdown_connections 2024-05-28T14:14:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 --2- 172.21.15.42:0/1104333874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea74072dd0 0x7fea740d0c30 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 --2- 172.21.15.42:0/1104333874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea7415ba40 0x7fea740d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 -- 172.21.15.42:0/1104333874 >> 172.21.15.42:0/1104333874 conn(0x7fea7406bdf0 msgr2=0x7fea7406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 -- 172.21.15.42:0/1104333874 shutdown_connections 2024-05-28T14:14:44.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 -- 172.21.15.42:0/1104333874 wait complete. 2024-05-28T14:14:44.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 Processor -- start 2024-05-28T14:14:44.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 -- start start 2024-05-28T14:14:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea7406f660 0x7fea7414c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea74072dd0 0x7fea7414d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea7414c2f0 0x7fea7414db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea74168f70 con 0x7fea7414c2f0 2024-05-28T14:14:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea74168df0 con 0x7fea74072dd0 2024-05-28T14:14:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea79000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea741690f0 con 0x7fea7406f660 2024-05-28T14:14:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea72800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea7414c2f0 0x7fea7414db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea72800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea7414c2f0 0x7fea7414db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34324/0 (socket says 172.21.15.42:34324) 2024-05-28T14:14:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea72800640 1 -- 172.21.15.42:0/779538032 learned_addr learned my addr 172.21.15.42:0/779538032 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea69400640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea74072dd0 0x7fea7414d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea72800640 1 -- 172.21.15.42:0/779538032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea7406f660 msgr2=0x7fea7414c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea71e00640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea7406f660 0x7fea7414c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea72800640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea7406f660 0x7fea7414c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea72800640 1 -- 172.21.15.42:0/779538032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea74072dd0 msgr2=0x7fea7414d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea72800640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea74072dd0 0x7fea7414d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea72800640 1 -- 172.21.15.42:0/779538032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea74154ac0 con 0x7fea7414c2f0 2024-05-28T14:14:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.286+0000 7fea69400640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea74072dd0 0x7fea7414d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.290+0000 7fea71e00640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea7406f660 0x7fea7414c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.290+0000 7fea72800640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea7414c2f0 0x7fea7414db50 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0af040 tx=0x7fea6c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.290+0000 7fea6aa00640 1 -- 172.21.15.42:0/779538032 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0b1070 con 0x7fea7414c2f0 2024-05-28T14:14:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.290+0000 7fea6aa00640 1 -- 172.21.15.42:0/779538032 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea6c0acb80 con 0x7fea7414c2f0 2024-05-28T14:14:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.290+0000 7fea79000640 1 -- 172.21.15.42:0/779538032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea740ce5c0 con 0x7fea7414c2f0 2024-05-28T14:14:44.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.290+0000 7fea6aa00640 1 -- 172.21.15.42:0/779538032 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0b7410 con 0x7fea7414c2f0 2024-05-28T14:14:44.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.290+0000 7fea79000640 1 -- 172.21.15.42:0/779538032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea74155040 con 0x7fea7414c2f0 2024-05-28T14:14:44.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.290+0000 7fea79000640 1 -- 172.21.15.42:0/779538032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea74068a90 con 0x7fea7414c2f0 2024-05-28T14:14:44.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.290+0000 7fea6aa00640 1 -- 172.21.15.42:0/779538032 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7fea6c0be060 con 0x7fea7414c2f0 2024-05-28T14:14:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.290+0000 7fea6aa00640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea50069fa0 0x7fea5006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.294+0000 7fea6aa00640 1 -- 172.21.15.42:0/779538032 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5765+0+0 (secure 0 0 0) 0x7fea6c11ebd0 con 0x7fea7414c2f0 2024-05-28T14:14:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.294+0000 7fea71e00640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea50069fa0 0x7fea5006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:44.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.294+0000 7fea71e00640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea50069fa0 0x7fea5006c460 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fea64007fd0 tx=0x7fea64009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:44.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.298+0000 7fea6aa00640 1 -- 172.21.15.42:0/779538032 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fea6c016150 con 0x7fea7414c2f0 2024-05-28T14:14:44.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.462+0000 7fea79000640 1 -- 172.21.15.42:0/779538032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea74072bb0 con 0x7fea7414c2f0 2024-05-28T14:14:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.462+0000 7fea6aa00640 1 -- 172.21.15.42:0/779538032 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16720 (secure 0 0 0) 0x7fea6c0f0bf0 con 0x7fea7414c2f0 2024-05-28T14:14:44.450 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:44.451 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":131,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:14:43.128139+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:14:41.538199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-28T14:14:42.086660+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:14:44.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.466+0000 7fea79000640 1 -- 172.21.15.42:0/779538032 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea50069fa0 msgr2=0x7fea5006c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:44.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.466+0000 7fea79000640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea50069fa0 0x7fea5006c460 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fea64007fd0 tx=0x7fea64009040 comp rx=0 tx=0).stop 2024-05-28T14:14:44.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.466+0000 7fea79000640 1 -- 172.21.15.42:0/779538032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea7414c2f0 msgr2=0x7fea7414db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:44.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.466+0000 7fea79000640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea7414c2f0 0x7fea7414db50 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0af040 tx=0x7fea6c0191b0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.466+0000 7fea79000640 1 -- 172.21.15.42:0/779538032 shutdown_connections 2024-05-28T14:14:44.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.466+0000 7fea79000640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea50069fa0 0x7fea5006c460 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.466+0000 7fea79000640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea7414c2f0 0x7fea7414db50 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.466+0000 7fea79000640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea74072dd0 0x7fea7414d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.466+0000 7fea79000640 1 --2- 172.21.15.42:0/779538032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea7406f660 0x7fea7414c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.466+0000 7fea79000640 1 -- 172.21.15.42:0/779538032 >> 172.21.15.42:0/779538032 conn(0x7fea7406bdf0 msgr2=0x7fea7415e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:44.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.466+0000 7fea79000640 1 -- 172.21.15.42:0/779538032 shutdown_connections 2024-05-28T14:14:44.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.466+0000 7fea79000640 1 -- 172.21.15.42:0/779538032 wait complete. 2024-05-28T14:14:44.472 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-28T14:14:44.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.630+0000 7f0684000640 1 Processor -- start 2024-05-28T14:14:44.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.630+0000 7f0684000640 1 -- start start 2024-05-28T14:14:44.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.630+0000 7f0684000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f067c0b4eb0 0x7f067c06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:44.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.630+0000 7f0684000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f067c06f190 con 0x7f067c06d110 2024-05-28T14:14:44.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.630+0000 7f0684000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f067c06f470 con 0x7f067c0b02e0 2024-05-28T14:14:44.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.630+0000 7f0684000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f067c06f750 con 0x7f067c0b4eb0 2024-05-28T14:14:44.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.630+0000 7f0680c00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f067c06d110 0x7f067c154da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37858/0 (socket says 172.21.15.42:37858) 2024-05-28T14:14:44.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.630+0000 7f0680c00640 1 -- 172.21.15.42:0/3930656998 learned_addr learned my addr 172.21.15.42:0/3930656998 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:44.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f0682000640 1 --2- 172.21.15.42:0/3930656998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f067c0b4eb0 0x7f067c06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:44.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2296859348 0 0) 0x7f067c06f190 con 0x7f067c06d110 2024-05-28T14:14:44.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0664003530 con 0x7f067c06d110 2024-05-28T14:14:44.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3481889017 0 0) 0x7f067c06f470 con 0x7f067c0b02e0 2024-05-28T14:14:44.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f067c06f190 con 0x7f067c0b02e0 2024-05-28T14:14:44.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 322508657 0 0) 0x7f0664003530 con 0x7f067c06d110 2024-05-28T14:14:44.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f067c0b4eb0 msgr2=0x7f067c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:44.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 --2- 172.21.15.42:0/3930656998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f067c0b4eb0 0x7f067c06eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 >> v1:172.21.15.63:6789/0 conn(0x7f067c0b02e0 legacy=0x7f067c06ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:44.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c06fa30 con 0x7f067c06d110 2024-05-28T14:14:44.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f0668001680 con 0x7f067c06d110 2024-05-28T14:14:44.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 >> v1:172.21.15.42:6789/0 conn(0x7f067c06d110 legacy=0x7f067c154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:44.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 --2- 172.21.15.42:0/3930656998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f067c0b4eb0 0x7f0664003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:44.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 --2- 172.21.15.42:0/3930656998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640040e0 0x7f0664004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:44.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 --2- 172.21.15.42:0/3930656998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0664004d50 0x7f06640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:44.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f067c06f750 con 0x7f0664004d50 2024-05-28T14:14:44.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f066400bab0 con 0x7f06640040e0 2024-05-28T14:14:44.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f066400bcb0 con 0x7f067c0b4eb0 2024-05-28T14:14:44.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f0682000640 1 --2- 172.21.15.42:0/3930656998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0664004d50 0x7f06640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:44.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f0681600640 1 --2- 172.21.15.42:0/3930656998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f067c0b4eb0 0x7f0664003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:44.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f0680c00640 1 --2- 172.21.15.42:0/3930656998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640040e0 0x7f0664004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:44.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f0682000640 1 -- 172.21.15.42:0/3930656998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f067c0b4eb0 msgr2=0x7f0664003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:44.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f0682000640 1 --2- 172.21.15.42:0/3930656998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f067c0b4eb0 0x7f0664003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f0682000640 1 -- 172.21.15.42:0/3930656998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640040e0 msgr2=0x7f0664004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:44.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f0682000640 1 --2- 172.21.15.42:0/3930656998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640040e0 0x7f0664004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f0682000640 1 -- 172.21.15.42:0/3930656998 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0664003530 con 0x7f0664004d50 2024-05-28T14:14:44.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.634+0000 7f0680c00640 1 --2- 172.21.15.42:0/3930656998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06640040e0 0x7f0664004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:44.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0682000640 1 --2- 172.21.15.42:0/3930656998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0664004d50 0x7f06640051d0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f0674009fc0 tx=0x7f067401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94be85007d2417c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:44.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06740b2070 con 0x7f0664004d50 2024-05-28T14:14:44.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06740a5b80 con 0x7f0664004d50 2024-05-28T14:14:44.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f067be00640 1 -- 172.21.15.42:0/3930656998 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674009c10 con 0x7f0664004d50 2024-05-28T14:14:44.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0684000640 1 -- 172.21.15.42:0/3930656998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0664004d50 msgr2=0x7f06640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:44.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0684000640 1 --2- 172.21.15.42:0/3930656998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0664004d50 0x7f06640051d0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f0674009fc0 tx=0x7f067401cc80 comp rx=0 tx=0).stop 2024-05-28T14:14:44.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0682000640 1 -- 172.21.15.42:0/3930656998 reap_dead start 2024-05-28T14:14:44.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0684000640 1 -- 172.21.15.42:0/3930656998 shutdown_connections 2024-05-28T14:14:44.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0684000640 1 -- 172.21.15.42:0/3930656998 >> 172.21.15.42:0/3930656998 conn(0x7f067c072600 msgr2=0x7f067c072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:44.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0684000640 1 -- 172.21.15.42:0/3930656998 shutdown_connections 2024-05-28T14:14:44.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0684000640 1 -- 172.21.15.42:0/3930656998 wait complete. 2024-05-28T14:14:44.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0684000640 1 Processor -- start 2024-05-28T14:14:44.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0684000640 1 -- start start 2024-05-28T14:14:44.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0684000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0664004d50 0x7f067c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:44.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0684000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f067c15b6b0 0x7f067c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:44.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0684000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f067c0ce680 0x7f067c0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:44.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0684000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f067c06ec80 con 0x7f067c0ce680 2024-05-28T14:14:44.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0684000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f067c06eb00 con 0x7f0664004d50 2024-05-28T14:14:44.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.638+0000 7f0684000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f067c06ee00 con 0x7f067c15b6b0 2024-05-28T14:14:44.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0682000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f067c0ce680 0x7f067c0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:44.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0682000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f067c0ce680 0x7f067c0ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34332/0 (socket says 172.21.15.42:34332) 2024-05-28T14:14:44.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0682000640 1 -- 172.21.15.42:0/910969828 learned_addr learned my addr 172.21.15.42:0/910969828 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:44.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0681600640 1 --2- 172.21.15.42:0/910969828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0664004d50 0x7f067c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:44.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0680c00640 1 --2- 172.21.15.42:0/910969828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f067c15b6b0 0x7f067c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:44.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0682000640 1 -- 172.21.15.42:0/910969828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f067c15b6b0 msgr2=0x7f067c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:44.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0682000640 1 --2- 172.21.15.42:0/910969828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f067c15b6b0 0x7f067c15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0682000640 1 -- 172.21.15.42:0/910969828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0664004d50 msgr2=0x7f067c1567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:44.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0682000640 1 --2- 172.21.15.42:0/910969828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0664004d50 0x7f067c1567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:44.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0682000640 1 -- 172.21.15.42:0/910969828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c0cf3f0 con 0x7f067c0ce680 2024-05-28T14:14:44.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0682000640 1 --2- 172.21.15.42:0/910969828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f067c0ce680 0x7f067c0ceb00 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f0674002240 tx=0x7f0674009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:44.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0679600640 1 -- 172.21.15.42:0/910969828 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06740bd070 con 0x7f067c0ce680 2024-05-28T14:14:44.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0679600640 1 -- 172.21.15.42:0/910969828 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06740a5980 con 0x7f067c0ce680 2024-05-28T14:14:44.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0684000640 1 -- 172.21.15.42:0/910969828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067c0cf6d0 con 0x7f067c0ce680 2024-05-28T14:14:44.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0679600640 1 -- 172.21.15.42:0/910969828 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06740b2040 con 0x7f067c0ce680 2024-05-28T14:14:44.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0684000640 1 -- 172.21.15.42:0/910969828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067c0cfc90 con 0x7f067c0ce680 2024-05-28T14:14:44.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0684000640 1 -- 172.21.15.42:0/910969828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f067c073f40 con 0x7f067c0ce680 2024-05-28T14:14:44.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.642+0000 7f0679600640 1 -- 172.21.15.42:0/910969828 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f06740aecf0 con 0x7f067c0ce680 2024-05-28T14:14:44.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.646+0000 7f0679600640 1 --2- 172.21.15.42:0/910969828 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f065c069ff0 0x7f065c06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:44.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.646+0000 7f0679600640 1 -- 172.21.15.42:0/910969828 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5765+0+0 (secure 0 0 0) 0x7f067411f9e0 con 0x7f067c0ce680 2024-05-28T14:14:44.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.646+0000 7f0681600640 1 --2- 172.21.15.42:0/910969828 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f065c069ff0 0x7f065c06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:44.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.646+0000 7f0681600640 1 --2- 172.21.15.42:0/910969828 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f065c069ff0 0x7f065c06c4b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0670000c10 tx=0x7f0670009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:44.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.650+0000 7f0679600640 1 -- 172.21.15.42:0/910969828 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0674016990 con 0x7f067c0ce680 2024-05-28T14:14:44.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:44.854+0000 7f0684000640 1 -- 172.21.15.42:0/910969828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f067c15f470 con 0x7f067c0ce680 2024-05-28T14:14:45.700 INFO:tasks.ceph.mds.d.smithi063.stderr:2024-05-28T14:14:45.715+0000 7f3e58e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-28T14:14:45.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.762+0000 7f0679600640 1 -- 172.21.15.42:0/910969828 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) ==== 118+0+0 (secure 0 0 0) 0x7f06740f1530 con 0x7f067c0ce680 2024-05-28T14:14:45.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.762+0000 7f0684000640 1 -- 172.21.15.42:0/910969828 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f065c069ff0 msgr2=0x7f065c06c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:45.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.762+0000 7f0684000640 1 --2- 172.21.15.42:0/910969828 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f065c069ff0 0x7f065c06c4b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0670000c10 tx=0x7f0670009040 comp rx=0 tx=0).stop 2024-05-28T14:14:45.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.762+0000 7f0684000640 1 -- 172.21.15.42:0/910969828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f067c0ce680 msgr2=0x7f067c0ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:45.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.762+0000 7f0684000640 1 --2- 172.21.15.42:0/910969828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f067c0ce680 0x7f067c0ceb00 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f0674002240 tx=0x7f0674009750 comp rx=0 tx=0).stop 2024-05-28T14:14:45.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.762+0000 7f0684000640 1 -- 172.21.15.42:0/910969828 shutdown_connections 2024-05-28T14:14:45.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.762+0000 7f0684000640 1 --2- 172.21.15.42:0/910969828 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f065c069ff0 0x7f065c06c4b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:45.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.762+0000 7f0684000640 1 --2- 172.21.15.42:0/910969828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f067c0ce680 0x7f067c0ceb00 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:45.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.762+0000 7f0684000640 1 --2- 172.21.15.42:0/910969828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f067c15b6b0 0x7f067c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:45.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.762+0000 7f0684000640 1 --2- 172.21.15.42:0/910969828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0664004d50 0x7f067c1567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:45.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.762+0000 7f0684000640 1 -- 172.21.15.42:0/910969828 >> 172.21.15.42:0/910969828 conn(0x7f067c072600 msgr2=0x7f067c0749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:45.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.762+0000 7f0684000640 1 -- 172.21.15.42:0/910969828 shutdown_connections 2024-05-28T14:14:45.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.762+0000 7f0684000640 1 -- 172.21.15.42:0/910969828 wait complete. 2024-05-28T14:14:45.774 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-28T14:14:45.774 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-28T14:14:45.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.938+0000 7f06cba00640 1 Processor -- start 2024-05-28T14:14:45.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.938+0000 7f06cba00640 1 -- start start 2024-05-28T14:14:45.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.938+0000 7f06cba00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f06c40724f0 0x7f06c4169530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:45.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.938+0000 7f06cba00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06c40b3690 con 0x7f06c40b4c10 2024-05-28T14:14:45.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.938+0000 7f06cba00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06c40b3970 con 0x7f06c4148090 2024-05-28T14:14:45.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.938+0000 7f06cba00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f06c4148470 con 0x7f06c40724f0 2024-05-28T14:14:45.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.938+0000 7f06bbe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f06c40b4c10 0x7f06c4071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37872/0 (socket says 172.21.15.42:37872) 2024-05-28T14:14:45.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.938+0000 7f06bbe00640 1 -- 172.21.15.42:0/1302238610 learned_addr learned my addr 172.21.15.42:0/1302238610 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:45.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.938+0000 7f06c9a00640 1 --2- 172.21.15.42:0/1302238610 >> v2:172.21.15.42:3300/0 conn(0x7f06c40724f0 0x7f06c4169530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:45.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.938+0000 7f06c9a00640 1 -- 172.21.15.42:0/1302238610 >> v1:172.21.15.63:6789/0 conn(0x7f06c4148090 legacy=0x7f06c40b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:45.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.938+0000 7f06c9a00640 1 -- 172.21.15.42:0/1302238610 >> v1:172.21.15.42:6789/0 conn(0x7f06c40b4c10 legacy=0x7f06c4071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:45.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.938+0000 7f06c9a00640 1 -- 172.21.15.42:0/1302238610 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c40b3c50 con 0x7f06c40724f0 2024-05-28T14:14:45.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.942+0000 7f06c9a00640 1 --2- 172.21.15.42:0/1302238610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06c40724f0 0x7f06c4169530 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009d90 tx=0x7f06bc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1c0e37db651f73 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:45.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.942+0000 7f06c3e00640 1 -- 172.21.15.42:0/1302238610 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0b2070 con 0x7f06c40724f0 2024-05-28T14:14:45.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.942+0000 7f06c3e00640 1 -- 172.21.15.42:0/1302238610 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06bc0a57a0 con 0x7f06c40724f0 2024-05-28T14:14:45.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.942+0000 7f06c3e00640 1 -- 172.21.15.42:0/1302238610 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0a5b70 con 0x7f06c40724f0 2024-05-28T14:14:45.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.942+0000 7f06cba00640 1 -- 172.21.15.42:0/1302238610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06c40724f0 msgr2=0x7f06c4169530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:45.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.942+0000 7f06cba00640 1 --2- 172.21.15.42:0/1302238610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06c40724f0 0x7f06c4169530 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009d90 tx=0x7f06bc01c8e0 comp rx=0 tx=0).stop 2024-05-28T14:14:45.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.942+0000 7f06cba00640 1 -- 172.21.15.42:0/1302238610 shutdown_connections 2024-05-28T14:14:45.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.942+0000 7f06cba00640 1 --2- 172.21.15.42:0/1302238610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06c40724f0 0x7f06c4169530 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:45.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.942+0000 7f06cba00640 1 -- 172.21.15.42:0/1302238610 >> 172.21.15.42:0/1302238610 conn(0x7f06c40b21f0 msgr2=0x7f06c40b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:45.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.942+0000 7f06cba00640 1 -- 172.21.15.42:0/1302238610 shutdown_connections 2024-05-28T14:14:45.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.942+0000 7f06cba00640 1 -- 172.21.15.42:0/1302238610 wait complete. 2024-05-28T14:14:45.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06cba00640 1 Processor -- start 2024-05-28T14:14:45.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06cba00640 1 -- start start 2024-05-28T14:14:45.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06cba00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40724f0 0x7f06c4132c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:45.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06cba00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06c4148090 0x7f06c415db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06cba00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06c415e060 0x7f06c4159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06cba00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f06c416ca30 con 0x7f06c415e060 2024-05-28T14:14:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06cba00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f06c416c8b0 con 0x7f06c4148090 2024-05-28T14:14:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06cba00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f06c416cbb0 con 0x7f06c40724f0 2024-05-28T14:14:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06c9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06c415e060 0x7f06c4159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06c9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06c415e060 0x7f06c4159190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34340/0 (socket says 172.21.15.42:34340) 2024-05-28T14:14:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06c9a00640 1 -- 172.21.15.42:0/2668080766 learned_addr learned my addr 172.21.15.42:0/2668080766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06bbe00640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06c4148090 0x7f06c415db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06c9000640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40724f0 0x7f06c4132c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06c9a00640 1 -- 172.21.15.42:0/2668080766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40724f0 msgr2=0x7f06c4132c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:45.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06c9a00640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40724f0 0x7f06c4132c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06c9a00640 1 -- 172.21.15.42:0/2668080766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06c4148090 msgr2=0x7f06c415db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06c9a00640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06c4148090 0x7f06c415db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06c9a00640 1 -- 172.21.15.42:0/2668080766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c4143340 con 0x7f06c415e060 2024-05-28T14:14:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06bbe00640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06c4148090 0x7f06c415db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06c9a00640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06c415e060 0x7f06c4159190 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0194f0 tx=0x7f06bc019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06c1600640 1 -- 172.21.15.42:0/2668080766 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0b2050 con 0x7f06c415e060 2024-05-28T14:14:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06c1600640 1 -- 172.21.15.42:0/2668080766 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06bc0be070 con 0x7f06c415e060 2024-05-28T14:14:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06c1600640 1 -- 172.21.15.42:0/2668080766 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0adab0 con 0x7f06c415e060 2024-05-28T14:14:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06cba00640 1 -- 172.21.15.42:0/2668080766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c4159d70 con 0x7f06c415e060 2024-05-28T14:14:45.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.946+0000 7f06cba00640 1 -- 172.21.15.42:0/2668080766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c415a330 con 0x7f06c415e060 2024-05-28T14:14:45.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.950+0000 7f06cba00640 1 -- 172.21.15.42:0/2668080766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c40738b0 con 0x7f06c415e060 2024-05-28T14:14:45.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.950+0000 7f06c1600640 1 -- 172.21.15.42:0/2668080766 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f06bc0adc50 con 0x7f06c415e060 2024-05-28T14:14:45.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.950+0000 7f06c1600640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f06a0069fa0 0x7f06a006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:45.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.950+0000 7f06c1600640 1 -- 172.21.15.42:0/2668080766 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f06bc120a90 con 0x7f06c415e060 2024-05-28T14:14:45.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.950+0000 7f06c9000640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f06a0069fa0 0x7f06a006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:45.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.950+0000 7f06c9000640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f06a0069fa0 0x7f06a006c460 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f06b40015b0 tx=0x7f06b4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:45.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:45.954+0000 7f06c1600640 1 -- 172.21.15.42:0/2668080766 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f06bc016560 con 0x7f06c415e060 2024-05-28T14:14:46.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.154+0000 7f06cba00640 1 -- 172.21.15.42:0/2668080766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f06c404fdd0 con 0x7f06c415e060 2024-05-28T14:14:46.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.762+0000 7f06c1600640 1 -- 172.21.15.42:0/2668080766 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) ==== 122+0+0 (secure 0 0 0) 0x7f06bc0ab340 con 0x7f06c415e060 2024-05-28T14:14:46.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.766+0000 7f06cba00640 1 -- 172.21.15.42:0/2668080766 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f06a0069fa0 msgr2=0x7f06a006c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:46.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.766+0000 7f06cba00640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f06a0069fa0 0x7f06a006c460 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f06b40015b0 tx=0x7f06b4009040 comp rx=0 tx=0).stop 2024-05-28T14:14:46.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.766+0000 7f06cba00640 1 -- 172.21.15.42:0/2668080766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06c415e060 msgr2=0x7f06c4159190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:46.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.766+0000 7f06cba00640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06c415e060 0x7f06c4159190 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0194f0 tx=0x7f06bc019680 comp rx=0 tx=0).stop 2024-05-28T14:14:46.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.766+0000 7f06cba00640 1 -- 172.21.15.42:0/2668080766 shutdown_connections 2024-05-28T14:14:46.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.766+0000 7f06cba00640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f06a0069fa0 0x7f06a006c460 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:46.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.766+0000 7f06cba00640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06c415e060 0x7f06c4159190 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:46.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.766+0000 7f06cba00640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06c4148090 0x7f06c415db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:46.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.766+0000 7f06cba00640 1 --2- 172.21.15.42:0/2668080766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40724f0 0x7f06c4132c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:46.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.766+0000 7f06cba00640 1 -- 172.21.15.42:0/2668080766 >> 172.21.15.42:0/2668080766 conn(0x7f06c40b21f0 msgr2=0x7f06c4071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:46.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.766+0000 7f06cba00640 1 -- 172.21.15.42:0/2668080766 shutdown_connections 2024-05-28T14:14:46.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.766+0000 7f06cba00640 1 -- 172.21.15.42:0/2668080766 wait complete. 2024-05-28T14:14:46.775 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-28T14:14:46.776 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:14:46.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.934+0000 7f8c56200640 1 Processor -- start 2024-05-28T14:14:46.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.934+0000 7f8c56200640 1 -- start start 2024-05-28T14:14:46.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.938+0000 7f8c56200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8c5006ff10 0x7f8c5015e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:46.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.938+0000 7f8c56200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c500d0660 con 0x7f8c500703a0 2024-05-28T14:14:46.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.938+0000 7f8c56200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c500d0940 con 0x7f8c50070800 2024-05-28T14:14:46.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.938+0000 7f8c56200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8c5009ba60 con 0x7f8c5006ff10 2024-05-28T14:14:46.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.938+0000 7f8c47e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8c5006ff10 0x7f8c5015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:46.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.938+0000 7f8c47e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8c5006ff10 0x7f8c5015e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34348/0 (socket says 172.21.15.42:34348) 2024-05-28T14:14:46.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.938+0000 7f8c47e00640 1 -- 172.21.15.42:0/346593597 learned_addr learned my addr 172.21.15.42:0/346593597 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:46.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.938+0000 7f8c47e00640 1 -- 172.21.15.42:0/346593597 >> v1:172.21.15.63:6789/0 conn(0x7f8c50070800 legacy=0x7f8c50070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:46.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.938+0000 7f8c47e00640 1 -- 172.21.15.42:0/346593597 >> v1:172.21.15.42:6789/0 conn(0x7f8c500703a0 legacy=0x7f8c5015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:46.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.938+0000 7f8c47e00640 1 -- 172.21.15.42:0/346593597 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c500d0c20 con 0x7f8c5006ff10 2024-05-28T14:14:46.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.942+0000 7f8c47e00640 1 --2- 172.21.15.42:0/346593597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c5006ff10 0x7f8c5015e910 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f8c380098a0 tx=0x7f8c3801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca69e1b0b12ea70 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:46.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.942+0000 7f8c4fe00640 1 -- 172.21.15.42:0/346593597 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c380ab5f0 con 0x7f8c5006ff10 2024-05-28T14:14:46.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.942+0000 7f8c4fe00640 1 -- 172.21.15.42:0/346593597 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c380b0070 con 0x7f8c5006ff10 2024-05-28T14:14:46.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.942+0000 7f8c4fe00640 1 -- 172.21.15.42:0/346593597 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c380ab9b0 con 0x7f8c5006ff10 2024-05-28T14:14:46.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.942+0000 7f8c56200640 1 -- 172.21.15.42:0/346593597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c5006ff10 msgr2=0x7f8c5015e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:46.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.942+0000 7f8c56200640 1 --2- 172.21.15.42:0/346593597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c5006ff10 0x7f8c5015e910 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f8c380098a0 tx=0x7f8c3801c500 comp rx=0 tx=0).stop 2024-05-28T14:14:46.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.942+0000 7f8c56200640 1 -- 172.21.15.42:0/346593597 shutdown_connections 2024-05-28T14:14:46.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.942+0000 7f8c56200640 1 --2- 172.21.15.42:0/346593597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c5006ff10 0x7f8c5015e910 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:46.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.942+0000 7f8c56200640 1 -- 172.21.15.42:0/346593597 >> 172.21.15.42:0/346593597 conn(0x7f8c500b3dd0 msgr2=0x7f8c500b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:46.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.942+0000 7f8c56200640 1 -- 172.21.15.42:0/346593597 shutdown_connections 2024-05-28T14:14:46.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.942+0000 7f8c56200640 1 -- 172.21.15.42:0/346593597 wait complete. 2024-05-28T14:14:46.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.942+0000 7f8c56200640 1 Processor -- start 2024-05-28T14:14:46.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.942+0000 7f8c56200640 1 -- start start 2024-05-28T14:14:46.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.942+0000 7f8c56200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c5006ff10 0x7f8c500b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:46.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c56200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c500703a0 0x7f8c500cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:46.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c56200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c50070800 0x7f8c500ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:46.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c56200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8c500b1a10 con 0x7f8c50070800 2024-05-28T14:14:46.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c56200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8c500b1890 con 0x7f8c500703a0 2024-05-28T14:14:46.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c56200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8c5015f970 con 0x7f8c5006ff10 2024-05-28T14:14:46.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c55800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c50070800 0x7f8c500ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:46.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c55800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c50070800 0x7f8c500ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34354/0 (socket says 172.21.15.42:34354) 2024-05-28T14:14:46.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c55800640 1 -- 172.21.15.42:0/1827002492 learned_addr learned my addr 172.21.15.42:0/1827002492 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:46.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c54e00640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c5006ff10 0x7f8c500b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:46.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c47e00640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c500703a0 0x7f8c500cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:46.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c55800640 1 -- 172.21.15.42:0/1827002492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c5006ff10 msgr2=0x7f8c500b7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:46.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c55800640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c5006ff10 0x7f8c500b7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:46.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c55800640 1 -- 172.21.15.42:0/1827002492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c500703a0 msgr2=0x7f8c500cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:46.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c55800640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c500703a0 0x7f8c500cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:46.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c55800640 1 -- 172.21.15.42:0/1827002492 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c500b7f70 con 0x7f8c50070800 2024-05-28T14:14:46.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c47e00640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c500703a0 0x7f8c500cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:46.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c55800640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c50070800 0x7f8c500ccb30 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800c730 tx=0x7f8c4800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:46.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c4d600640 1 -- 172.21.15.42:0/1827002492 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48003b80 con 0x7f8c50070800 2024-05-28T14:14:46.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c4d600640 1 -- 172.21.15.42:0/1827002492 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c48003d20 con 0x7f8c50070800 2024-05-28T14:14:46.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c56200640 1 -- 172.21.15.42:0/1827002492 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5015c9c0 con 0x7f8c50070800 2024-05-28T14:14:46.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c4d600640 1 -- 172.21.15.42:0/1827002492 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4800aa90 con 0x7f8c50070800 2024-05-28T14:14:46.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c56200640 1 -- 172.21.15.42:0/1827002492 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c5015cf80 con 0x7f8c50070800 2024-05-28T14:14:46.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c56200640 1 -- 172.21.15.42:0/1827002492 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c5004fcf0 con 0x7f8c50070800 2024-05-28T14:14:46.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.946+0000 7f8c4d600640 1 -- 172.21.15.42:0/1827002492 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f8c4800ac30 con 0x7f8c50070800 2024-05-28T14:14:46.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.950+0000 7f8c4d600640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c2c069fa0 0x7f8c2c06c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.950+0000 7f8c4d600640 1 -- 172.21.15.42:0/1827002492 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f8c480857d0 con 0x7f8c50070800 2024-05-28T14:14:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.950+0000 7f8c54e00640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c2c069fa0 0x7f8c2c06c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.950+0000 7f8c54e00640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c2c069fa0 0x7f8c2c06c460 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000b810 tx=0x7f8c40008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:46.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:46.954+0000 7f8c4d600640 1 -- 172.21.15.42:0/1827002492 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8c480577f0 con 0x7f8c50070800 2024-05-28T14:14:47.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.154+0000 7f8c56200640 1 -- 172.21.15.42:0/1827002492 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c500d1d50 con 0x7f8c50070800 2024-05-28T14:14:47.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.154+0000 7f8c4d600640 1 -- 172.21.15.42:0/1827002492 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5188 (secure 0 0 0) 0x7f8c480570f0 con 0x7f8c50070800 2024-05-28T14:14:47.140 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:47.140 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":84,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9208,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1217785682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1217785682},{"type":"v1","addr":"172.21.15.63:6834","nonce":1217785682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9813,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1231554897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1231554897},{"type":"v1","addr":"172.21.15.42:6835","nonce":1231554897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9819,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3152913608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3152913608},{"type":"v1","addr":"172.21.15.42:6837","nonce":3152913608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:14:43.128104+0000","modified":"2024-05-28T14:14:46.159994+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57851":{"gid":57851,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3508679013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3508679013},{"type":"v1","addr":"172.21.15.63:6835","nonce":3508679013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57851,"qdb_cluster":[57851]},"id":9}]} 2024-05-28T14:14:47.140 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 84 2024-05-28T14:14:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.154+0000 7f8c56200640 1 -- 172.21.15.42:0/1827002492 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c2c069fa0 msgr2=0x7f8c2c06c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.154+0000 7f8c56200640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c2c069fa0 0x7f8c2c06c460 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000b810 tx=0x7f8c40008040 comp rx=0 tx=0).stop 2024-05-28T14:14:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.154+0000 7f8c56200640 1 -- 172.21.15.42:0/1827002492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c50070800 msgr2=0x7f8c500ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.154+0000 7f8c56200640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c50070800 0x7f8c500ccb30 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800c730 tx=0x7f8c4800cbf0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.154+0000 7f8c56200640 1 -- 172.21.15.42:0/1827002492 shutdown_connections 2024-05-28T14:14:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.154+0000 7f8c56200640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8c2c069fa0 0x7f8c2c06c460 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.154+0000 7f8c56200640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c50070800 0x7f8c500ccb30 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.154+0000 7f8c56200640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c500703a0 0x7f8c500cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.154+0000 7f8c56200640 1 --2- 172.21.15.42:0/1827002492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c5006ff10 0x7f8c500b7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.154+0000 7f8c56200640 1 -- 172.21.15.42:0/1827002492 >> 172.21.15.42:0/1827002492 conn(0x7f8c500b3dd0 msgr2=0x7f8c5006e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.154+0000 7f8c56200640 1 -- 172.21.15.42:0/1827002492 shutdown_connections 2024-05-28T14:14:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.154+0000 7f8c56200640 1 -- 172.21.15.42:0/1827002492 wait complete. 2024-05-28T14:14:47.163 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:14:47.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f37a4a00640 1 Processor -- start 2024-05-28T14:14:47.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f37a4a00640 1 -- start start 2024-05-28T14:14:47.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f37a4a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37a015eb00 0x7f37a00cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:47.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f37a4a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f37a006c100 0x7f37a00b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:47.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f37a4a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37a015eee0 con 0x7f37a00cfe00 2024-05-28T14:14:47.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f37a4a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f37a00ceb20 con 0x7f37a006c100 2024-05-28T14:14:47.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f37a4a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f37a00cee00 con 0x7f37a015eb00 2024-05-28T14:14:47.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f379e800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f37a006c100 0x7f37a00b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:47.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f3797e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f37a00cfe00 0x7f37a006b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37900/0 (socket says 172.21.15.42:37900) 2024-05-28T14:14:47.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f3797e00640 1 -- 172.21.15.42:0/3286584404 learned_addr learned my addr 172.21.15.42:0/3286584404 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:47.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f379de00640 1 --2- 172.21.15.42:0/3286584404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37a015eb00 0x7f37a00cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:47.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f379e800640 1 -- 172.21.15.42:0/3286584404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37a015eb00 msgr2=0x7f37a00cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:47.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f379e800640 1 --2- 172.21.15.42:0/3286584404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37a015eb00 0x7f37a00cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f379e800640 1 -- 172.21.15.42:0/3286584404 >> v1:172.21.15.42:6789/0 conn(0x7f37a00cfe00 legacy=0x7f37a006b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:47.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.318+0000 7f379e800640 1 -- 172.21.15.42:0/3286584404 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a00b5580 con 0x7f37a006c100 2024-05-28T14:14:47.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.322+0000 7f379e800640 1 --2- 172.21.15.42:0/3286584404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a006c100 0x7f37a00b5d80 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f3798009d90 tx=0x7f379801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d01ce658f249d5d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:47.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.322+0000 7f379d400640 1 -- 172.21.15.42:0/3286584404 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37980b2070 con 0x7f37a006c100 2024-05-28T14:14:47.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.322+0000 7f379d400640 1 -- 172.21.15.42:0/3286584404 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37980a5630 con 0x7f37a006c100 2024-05-28T14:14:47.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.322+0000 7f379d400640 1 -- 172.21.15.42:0/3286584404 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37980a5a00 con 0x7f37a006c100 2024-05-28T14:14:47.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.322+0000 7f37a4a00640 1 -- 172.21.15.42:0/3286584404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a006c100 msgr2=0x7f37a00b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:47.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.322+0000 7f37a4a00640 1 --2- 172.21.15.42:0/3286584404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a006c100 0x7f37a00b5d80 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f3798009d90 tx=0x7f379801c970 comp rx=0 tx=0).stop 2024-05-28T14:14:47.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.322+0000 7f37a4a00640 1 -- 172.21.15.42:0/3286584404 shutdown_connections 2024-05-28T14:14:47.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.322+0000 7f37a4a00640 1 --2- 172.21.15.42:0/3286584404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a006c100 0x7f37a00b5d80 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.322+0000 7f37a4a00640 1 --2- 172.21.15.42:0/3286584404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37a015eb00 0x7f37a00cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.322+0000 7f37a4a00640 1 -- 172.21.15.42:0/3286584404 >> 172.21.15.42:0/3286584404 conn(0x7f37a00b68f0 msgr2=0x7f37a00b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:47.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.322+0000 7f37a4a00640 1 -- 172.21.15.42:0/3286584404 shutdown_connections 2024-05-28T14:14:47.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.322+0000 7f37a4a00640 1 -- 172.21.15.42:0/3286584404 wait complete. 2024-05-28T14:14:47.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f37a4a00640 1 Processor -- start 2024-05-28T14:14:47.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f37a4a00640 1 -- start start 2024-05-28T14:14:47.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f37a4a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37a006c100 0x7f37a015c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:47.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f37a4a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a00cfe00 0x7f37a00d0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:47.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f37a4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a015eb00 0x7f37a00d0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:47.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f37a4a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f37a016b610 con 0x7f37a00cfe00 2024-05-28T14:14:47.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f37a4a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f37a00ce850 con 0x7f37a015eb00 2024-05-28T14:14:47.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f37a4a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f37a016b790 con 0x7f37a006c100 2024-05-28T14:14:47.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f3797e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a00cfe00 0x7f37a00d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:47.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f3797e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a00cfe00 0x7f37a00d0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34374/0 (socket says 172.21.15.42:34374) 2024-05-28T14:14:47.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f3797e00640 1 -- 172.21.15.42:0/4186601048 learned_addr learned my addr 172.21.15.42:0/4186601048 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:47.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f379de00640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37a006c100 0x7f37a015c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:47.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f379e800640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a015eb00 0x7f37a00d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:47.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f3797e00640 1 -- 172.21.15.42:0/4186601048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37a006c100 msgr2=0x7f37a015c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:47.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f3797e00640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37a006c100 0x7f37a015c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f3797e00640 1 -- 172.21.15.42:0/4186601048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a015eb00 msgr2=0x7f37a00d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:47.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f3797e00640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a015eb00 0x7f37a00d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f3797e00640 1 -- 172.21.15.42:0/4186601048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a00af5b0 con 0x7f37a00cfe00 2024-05-28T14:14:47.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f379e800640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a015eb00 0x7f37a00d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:47.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f3797e00640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a00cfe00 0x7f37a00d0310 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f378800c420 tx=0x7f378800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f3796000640 1 -- 172.21.15.42:0/4186601048 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788017070 con 0x7f37a00cfe00 2024-05-28T14:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f3796000640 1 -- 172.21.15.42:0/4186601048 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3788005cf0 con 0x7f37a00cfe00 2024-05-28T14:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f3796000640 1 -- 172.21.15.42:0/4186601048 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37880072f0 con 0x7f37a00cfe00 2024-05-28T14:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f37a4a00640 1 -- 172.21.15.42:0/4186601048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a00af890 con 0x7f37a00cfe00 2024-05-28T14:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.326+0000 7f37a4a00640 1 -- 172.21.15.42:0/4186601048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a014cbe0 con 0x7f37a00cfe00 2024-05-28T14:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.330+0000 7f37a4a00640 1 -- 172.21.15.42:0/4186601048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a0068a90 con 0x7f37a00cfe00 2024-05-28T14:14:47.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.330+0000 7f3796000640 1 -- 172.21.15.42:0/4186601048 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f378801a070 con 0x7f37a00cfe00 2024-05-28T14:14:47.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.330+0000 7f3796000640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f377c069fa0 0x7f377c06c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:47.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.330+0000 7f3796000640 1 -- 172.21.15.42:0/4186601048 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f3788084c60 con 0x7f37a00cfe00 2024-05-28T14:14:47.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.330+0000 7f379de00640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f377c069fa0 0x7f377c06c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:47.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.330+0000 7f379de00640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f377c069fa0 0x7f377c06c460 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f37900079c0 tx=0x7f3790008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:47.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.334+0000 7f3796000640 1 -- 172.21.15.42:0/4186601048 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f37a015eee0 con 0x7f37a00cfe00 2024-05-28T14:14:47.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.502+0000 7f37a4a00640 1 -- 172.21.15.42:0/4186601048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37a00cf880 con 0x7f37a00cfe00 2024-05-28T14:14:47.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.502+0000 7f3796000640 1 -- 172.21.15.42:0/4186601048 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+16707 (secure 0 0 0) 0x7f3788056630 con 0x7f37a00cfe00 2024-05-28T14:14:47.489 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:47.489 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":132,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:14:44.748499+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:14:41.538199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-28T14:14:42.086660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:14:47.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.502+0000 7f37a4a00640 1 -- 172.21.15.42:0/4186601048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f377c069fa0 msgr2=0x7f377c06c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.502+0000 7f37a4a00640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f377c069fa0 0x7f377c06c460 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f37900079c0 tx=0x7f3790008040 comp rx=0 tx=0).stop 2024-05-28T14:14:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.502+0000 7f37a4a00640 1 -- 172.21.15.42:0/4186601048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a00cfe00 msgr2=0x7f37a00d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.502+0000 7f37a4a00640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a00cfe00 0x7f37a00d0310 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f378800c420 tx=0x7f378800c8e0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.502+0000 7f37a4a00640 1 -- 172.21.15.42:0/4186601048 shutdown_connections 2024-05-28T14:14:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.506+0000 7f37a4a00640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f377c069fa0 0x7f377c06c460 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.506+0000 7f37a4a00640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37a015eb00 0x7f37a00d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.506+0000 7f37a4a00640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a00cfe00 0x7f37a00d0310 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.506+0000 7f37a4a00640 1 --2- 172.21.15.42:0/4186601048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37a006c100 0x7f37a015c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.506+0000 7f37a4a00640 1 -- 172.21.15.42:0/4186601048 >> 172.21.15.42:0/4186601048 conn(0x7f37a00b68f0 msgr2=0x7f37a014d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.506+0000 7f37a4a00640 1 -- 172.21.15.42:0/4186601048 shutdown_connections 2024-05-28T14:14:47.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.506+0000 7f37a4a00640 1 -- 172.21.15.42:0/4186601048 wait complete. 2024-05-28T14:14:47.512 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-28T14:14:47.512 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:14:47.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd424a00640 1 Processor -- start 2024-05-28T14:14:47.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd424a00640 1 -- start start 2024-05-28T14:14:47.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd424a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd42015c990 0x7fd4200d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:47.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd424a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd4200d06a0 0x7fd4200b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:47.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd424a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4200cd300 con 0x7fd42009bbd0 2024-05-28T14:14:47.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd424a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fd42015cd70 con 0x7fd4200d06a0 2024-05-28T14:14:47.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd424a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd4200cd5e0 con 0x7fd42015c990 2024-05-28T14:14:47.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd41e800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd4200d06a0 0x7fd4200b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:47.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd41e800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd4200d06a0 0x7fd4200b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47660/0 (socket says 172.21.15.42:47660) 2024-05-28T14:14:47.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd41e800640 1 -- 172.21.15.42:0/2020612685 learned_addr learned my addr 172.21.15.42:0/2020612685 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:47.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd417e00640 1 --1- 172.21.15.42:0/2020612685 >> v1:172.21.15.42:6789/0 conn(0x7fd42009bbd0 0x7fd42009aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53506/0 (socket says 172.21.15.42:53506) 2024-05-28T14:14:47.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd41de00640 1 --2- 172.21.15.42:0/2020612685 >> v2:172.21.15.63:3300/0 conn(0x7fd42015c990 0x7fd4200d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:47.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd41e800640 1 -- 172.21.15.42:0/2020612685 >> v2:172.21.15.63:3300/0 conn(0x7fd42015c990 msgr2=0x7fd4200d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:47.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd41e800640 1 --2- 172.21.15.42:0/2020612685 >> v2:172.21.15.63:3300/0 conn(0x7fd42015c990 0x7fd4200d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd41e800640 1 -- 172.21.15.42:0/2020612685 >> v1:172.21.15.42:6789/0 conn(0x7fd42009bbd0 legacy=0x7fd42009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:47.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.666+0000 7fd41e800640 1 -- 172.21.15.42:0/2020612685 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4200cd8c0 con 0x7fd4200d06a0 2024-05-28T14:14:47.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.670+0000 7fd41e800640 1 --2- 172.21.15.42:0/2020612685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4200d06a0 0x7fd4200b53e0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fd420073350 tx=0x7fd41801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc339102ca4be0f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:47.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.670+0000 7fd41d400640 1 -- 172.21.15.42:0/2020612685 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4180a5880 con 0x7fd4200d06a0 2024-05-28T14:14:47.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.670+0000 7fd41d400640 1 -- 172.21.15.42:0/2020612685 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4180a5a20 con 0x7fd4200d06a0 2024-05-28T14:14:47.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.670+0000 7fd41d400640 1 -- 172.21.15.42:0/2020612685 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4180a5df0 con 0x7fd4200d06a0 2024-05-28T14:14:47.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.670+0000 7fd424a00640 1 -- 172.21.15.42:0/2020612685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4200d06a0 msgr2=0x7fd4200b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:47.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.670+0000 7fd424a00640 1 --2- 172.21.15.42:0/2020612685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4200d06a0 0x7fd4200b53e0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fd420073350 tx=0x7fd41801c9c0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.670+0000 7fd424a00640 1 -- 172.21.15.42:0/2020612685 shutdown_connections 2024-05-28T14:14:47.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.670+0000 7fd424a00640 1 --2- 172.21.15.42:0/2020612685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4200d06a0 0x7fd4200b53e0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.670+0000 7fd424a00640 1 --2- 172.21.15.42:0/2020612685 >> v2:172.21.15.63:3300/0 conn(0x7fd42015c990 0x7fd4200d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.670+0000 7fd424a00640 1 -- 172.21.15.42:0/2020612685 >> 172.21.15.42:0/2020612685 conn(0x7fd4200b4080 msgr2=0x7fd4200b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:47.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.670+0000 7fd424a00640 1 -- 172.21.15.42:0/2020612685 shutdown_connections 2024-05-28T14:14:47.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.670+0000 7fd424a00640 1 -- 172.21.15.42:0/2020612685 wait complete. 2024-05-28T14:14:47.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd424a00640 1 Processor -- start 2024-05-28T14:14:47.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd424a00640 1 -- start start 2024-05-28T14:14:47.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd424a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4200d06a0 0x7fd4201518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:47.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd424a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd42015c990 0x7fd4201574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:47.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd424a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd42006e760 0x7fd4200caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:47.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd424a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd42009b2e0 con 0x7fd4200d06a0 2024-05-28T14:14:47.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd424a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd42009b160 con 0x7fd42015c990 2024-05-28T14:14:47.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd424a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd4200b6130 con 0x7fd42006e760 2024-05-28T14:14:47.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd41de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4200d06a0 0x7fd4201518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:47.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd41de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4200d06a0 0x7fd4201518e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47668/0 (socket says 172.21.15.42:47668) 2024-05-28T14:14:47.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd41de00640 1 -- 172.21.15.42:0/745529562 learned_addr learned my addr 172.21.15.42:0/745529562 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:47.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd41e800640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd42006e760 0x7fd4200caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:47.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd417e00640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd42015c990 0x7fd4201574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:47.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd41de00640 1 -- 172.21.15.42:0/745529562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd42006e760 msgr2=0x7fd4200caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:47.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd41de00640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd42006e760 0x7fd4200caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd41de00640 1 -- 172.21.15.42:0/745529562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd42015c990 msgr2=0x7fd4201574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:47.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd41de00640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd42015c990 0x7fd4201574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd41de00640 1 -- 172.21.15.42:0/745529562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd420158e70 con 0x7fd4200d06a0 2024-05-28T14:14:47.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd41e800640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd42006e760 0x7fd4200caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:47.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd41de00640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4200d06a0 0x7fd4201518e0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fd41000e870 tx=0x7fd41000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:47.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd416000640 1 -- 172.21.15.42:0/745529562 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410017070 con 0x7fd4200d06a0 2024-05-28T14:14:47.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd416000640 1 -- 172.21.15.42:0/745529562 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41000c940 con 0x7fd4200d06a0 2024-05-28T14:14:47.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd416000640 1 -- 172.21.15.42:0/745529562 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41000ccf0 con 0x7fd4200d06a0 2024-05-28T14:14:47.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd424a00640 1 -- 172.21.15.42:0/745529562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd420157dd0 con 0x7fd4200d06a0 2024-05-28T14:14:47.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.674+0000 7fd424a00640 1 -- 172.21.15.42:0/745529562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd420158670 con 0x7fd4200d06a0 2024-05-28T14:14:47.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.678+0000 7fd424a00640 1 -- 172.21.15.42:0/745529562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd420068a90 con 0x7fd4200d06a0 2024-05-28T14:14:47.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.678+0000 7fd416000640 1 -- 172.21.15.42:0/745529562 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7fd41001a070 con 0x7fd4200d06a0 2024-05-28T14:14:47.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.678+0000 7fd416000640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3fc069fa0 0x7fd3fc06c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:47.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.678+0000 7fd416000640 1 -- 172.21.15.42:0/745529562 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7fd410085780 con 0x7fd4200d06a0 2024-05-28T14:14:47.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.678+0000 7fd417e00640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3fc069fa0 0x7fd3fc06c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:47.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.678+0000 7fd417e00640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3fc069fa0 0x7fd3fc06c460 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd420152990 tx=0x7fd408007cd0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:47.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.682+0000 7fd416000640 1 -- 172.21.15.42:0/745529562 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd410057840 con 0x7fd4200d06a0 2024-05-28T14:14:47.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.846+0000 7fd424a00640 1 -- 172.21.15.42:0/745529562 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd4200b4ce0 con 0x7fd3fc069fa0 2024-05-28T14:14:47.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.854+0000 7fd416000640 1 -- 172.21.15.42:0/745529562 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252442 (secure 0 0 0) 0x7fd4200b4ce0 con 0x7fd3fc069fa0 2024-05-28T14:14:47.840 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:47.844 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:14:47.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.854+0000 7fd424a00640 1 -- 172.21.15.42:0/745529562 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3fc069fa0 msgr2=0x7fd3fc06c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:47.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.854+0000 7fd424a00640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3fc069fa0 0x7fd3fc06c460 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd420152990 tx=0x7fd408007cd0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.854+0000 7fd424a00640 1 -- 172.21.15.42:0/745529562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4200d06a0 msgr2=0x7fd4201518e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:47.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.854+0000 7fd424a00640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4200d06a0 0x7fd4201518e0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fd41000e870 tx=0x7fd41000ec70 comp rx=0 tx=0).stop 2024-05-28T14:14:47.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.858+0000 7fd424a00640 1 -- 172.21.15.42:0/745529562 shutdown_connections 2024-05-28T14:14:47.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.858+0000 7fd424a00640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd3fc069fa0 0x7fd3fc06c460 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.858+0000 7fd424a00640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd42006e760 0x7fd4200caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.858+0000 7fd424a00640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd42015c990 0x7fd4201574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.858+0000 7fd424a00640 1 --2- 172.21.15.42:0/745529562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4200d06a0 0x7fd4201518e0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:47.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.858+0000 7fd424a00640 1 -- 172.21.15.42:0/745529562 >> 172.21.15.42:0/745529562 conn(0x7fd4200b4080 msgr2=0x7fd42009aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:47.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.858+0000 7fd424a00640 1 -- 172.21.15.42:0/745529562 shutdown_connections 2024-05-28T14:14:47.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:47.858+0000 7fd424a00640 1 -- 172.21.15.42:0/745529562 wait complete. 2024-05-28T14:14:47.863 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":624,"stamp":"2024-05-28T14:14:47.195444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1080556,"kb_used_data":13944,"kb_used_omap":12,"kb_used_meta":1066483,"kb_avail":753894164,"statfs":{"total":773094113280,"available":771987623936,"internally_reserved":0,"allocated":14278656,"data_stored":6922964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1092079154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.477013"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.769185+0000","last_change":"2024-05-28T14:14:43.766434+0000","last_active":"2024-05-28T14:14:43.769185+0000","last_peered":"2024-05-28T14:14:43.769185+0000","last_clean":"2024-05-28T14:14:43.769185+0000","last_became_active":"2024-05-28T14:14:43.766125+0000","last_became_peered":"2024-05-28T14:14:43.766125+0000","last_unstale":"2024-05-28T14:14:43.769185+0000","last_undegraded":"2024-05-28T14:14:43.769185+0000","last_fullsized":"2024-05-28T14:14:43.769185+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:39:19.585570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.771000+0000","last_change":"2024-05-28T14:14:43.769925+0000","last_active":"2024-05-28T14:14:43.771000+0000","last_peered":"2024-05-28T14:14:43.771000+0000","last_clean":"2024-05-28T14:14:43.771000+0000","last_became_active":"2024-05-28T14:14:43.769641+0000","last_became_peered":"2024-05-28T14:14:43.769641+0000","last_unstale":"2024-05-28T14:14:43.771000+0000","last_undegraded":"2024-05-28T14:14:43.771000+0000","last_fullsized":"2024-05-28T14:14:43.771000+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:11.879163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.773239+0000","last_change":"2024-05-28T14:14:43.769199+0000","last_active":"2024-05-28T14:14:43.773239+0000","last_peered":"2024-05-28T14:14:43.773239+0000","last_clean":"2024-05-28T14:14:43.773239+0000","last_became_active":"2024-05-28T14:14:43.764098+0000","last_became_peered":"2024-05-28T14:14:43.764098+0000","last_unstale":"2024-05-28T14:14:43.773239+0000","last_undegraded":"2024-05-28T14:14:43.773239+0000","last_fullsized":"2024-05-28T14:14:43.773239+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:14:36.957242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.765583+0000","last_change":"2024-05-28T14:14:43.759234+0000","last_active":"2024-05-28T14:14:43.765583+0000","last_peered":"2024-05-28T14:14:43.765583+0000","last_clean":"2024-05-28T14:14:43.765583+0000","last_became_active":"2024-05-28T14:14:43.758925+0000","last_became_peered":"2024-05-28T14:14:43.758925+0000","last_unstale":"2024-05-28T14:14:43.765583+0000","last_undegraded":"2024-05-28T14:14:43.765583+0000","last_fullsized":"2024-05-28T14:14:43.765583+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:10:36.047568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.772958+0000","last_change":"2024-05-28T14:14:43.766612+0000","last_active":"2024-05-28T14:14:43.772958+0000","last_peered":"2024-05-28T14:14:43.772958+0000","last_clean":"2024-05-28T14:14:43.772958+0000","last_became_active":"2024-05-28T14:14:43.764703+0000","last_became_peered":"2024-05-28T14:14:43.764703+0000","last_unstale":"2024-05-28T14:14:43.772958+0000","last_undegraded":"2024-05-28T14:14:43.772958+0000","last_fullsized":"2024-05-28T14:14:43.772958+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:16.773436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.755190+0000","last_change":"2024-05-28T14:14:43.762958+0000","last_active":"2024-05-28T14:14:44.755190+0000","last_peered":"2024-05-28T14:14:44.755190+0000","last_clean":"2024-05-28T14:14:44.755190+0000","last_became_active":"2024-05-28T14:14:43.762689+0000","last_became_peered":"2024-05-28T14:14:43.762689+0000","last_unstale":"2024-05-28T14:14:44.755190+0000","last_undegraded":"2024-05-28T14:14:44.755190+0000","last_fullsized":"2024-05-28T14:14:44.755190+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:56:11.671806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.754669+0000","last_change":"2024-05-28T14:14:43.770900+0000","last_active":"2024-05-28T14:14:44.754669+0000","last_peered":"2024-05-28T14:14:44.754669+0000","last_clean":"2024-05-28T14:14:44.754669+0000","last_became_active":"2024-05-28T14:14:43.770616+0000","last_became_peered":"2024-05-28T14:14:43.770616+0000","last_unstale":"2024-05-28T14:14:44.754669+0000","last_undegraded":"2024-05-28T14:14:44.754669+0000","last_fullsized":"2024-05-28T14:14:44.754669+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:07:11.105627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766491+0000","last_change":"2024-05-28T14:14:43.762262+0000","last_active":"2024-05-28T14:14:43.766491+0000","last_peered":"2024-05-28T14:14:43.766491+0000","last_clean":"2024-05-28T14:14:43.766491+0000","last_became_active":"2024-05-28T14:14:43.761977+0000","last_became_peered":"2024-05-28T14:14:43.761977+0000","last_unstale":"2024-05-28T14:14:43.766491+0000","last_undegraded":"2024-05-28T14:14:43.766491+0000","last_fullsized":"2024-05-28T14:14:43.766491+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:55:59.790610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.773781+0000","last_change":"2024-05-28T14:14:43.770664+0000","last_active":"2024-05-28T14:14:43.773781+0000","last_peered":"2024-05-28T14:14:43.773781+0000","last_clean":"2024-05-28T14:14:43.773781+0000","last_became_active":"2024-05-28T14:14:43.770335+0000","last_became_peered":"2024-05-28T14:14:43.770335+0000","last_unstale":"2024-05-28T14:14:43.773781+0000","last_undegraded":"2024-05-28T14:14:43.773781+0000","last_fullsized":"2024-05-28T14:14:43.773781+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:36:35.332059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.771021+0000","last_change":"2024-05-28T14:14:43.769965+0000","last_active":"2024-05-28T14:14:43.771021+0000","last_peered":"2024-05-28T14:14:43.771021+0000","last_clean":"2024-05-28T14:14:43.771021+0000","last_became_active":"2024-05-28T14:14:43.769687+0000","last_became_peered":"2024-05-28T14:14:43.769687+0000","last_unstale":"2024-05-28T14:14:43.771021+0000","last_undegraded":"2024-05-28T14:14:43.771021+0000","last_fullsized":"2024-05-28T14:14:43.771021+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:00:34.527970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.764294+0000","last_change":"2024-05-28T14:14:43.758128+0000","last_active":"2024-05-28T14:14:43.764294+0000","last_peered":"2024-05-28T14:14:43.764294+0000","last_clean":"2024-05-28T14:14:43.764294+0000","last_became_active":"2024-05-28T14:14:43.757685+0000","last_became_peered":"2024-05-28T14:14:43.757685+0000","last_unstale":"2024-05-28T14:14:43.764294+0000","last_undegraded":"2024-05-28T14:14:43.764294+0000","last_fullsized":"2024-05-28T14:14:43.764294+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:27:22.008435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.773718+0000","last_change":"2024-05-28T14:14:43.769235+0000","last_active":"2024-05-28T14:14:43.773718+0000","last_peered":"2024-05-28T14:14:43.773718+0000","last_clean":"2024-05-28T14:14:43.773718+0000","last_became_active":"2024-05-28T14:14:43.768939+0000","last_became_peered":"2024-05-28T14:14:43.768939+0000","last_unstale":"2024-05-28T14:14:43.773718+0000","last_undegraded":"2024-05-28T14:14:43.773718+0000","last_fullsized":"2024-05-28T14:14:43.773718+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:58:09.147319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.768877+0000","last_change":"2024-05-28T14:14:43.767408+0000","last_active":"2024-05-28T14:14:43.768877+0000","last_peered":"2024-05-28T14:14:43.768877+0000","last_clean":"2024-05-28T14:14:43.768877+0000","last_became_active":"2024-05-28T14:14:43.766269+0000","last_became_peered":"2024-05-28T14:14:43.766269+0000","last_unstale":"2024-05-28T14:14:43.768877+0000","last_undegraded":"2024-05-28T14:14:43.768877+0000","last_fullsized":"2024-05-28T14:14:43.768877+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:08.316254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766915+0000","last_change":"2024-05-28T14:14:43.762819+0000","last_active":"2024-05-28T14:14:43.766915+0000","last_peered":"2024-05-28T14:14:43.766915+0000","last_clean":"2024-05-28T14:14:43.766915+0000","last_became_active":"2024-05-28T14:14:43.762494+0000","last_became_peered":"2024-05-28T14:14:43.762494+0000","last_unstale":"2024-05-28T14:14:43.766915+0000","last_undegraded":"2024-05-28T14:14:43.766915+0000","last_fullsized":"2024-05-28T14:14:43.766915+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:39.285543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.765450+0000","last_change":"2024-05-28T14:14:43.761245+0000","last_active":"2024-05-28T14:14:43.765450+0000","last_peered":"2024-05-28T14:14:43.765450+0000","last_clean":"2024-05-28T14:14:43.765450+0000","last_became_active":"2024-05-28T14:14:43.760977+0000","last_became_peered":"2024-05-28T14:14:43.760977+0000","last_unstale":"2024-05-28T14:14:43.765450+0000","last_undegraded":"2024-05-28T14:14:43.765450+0000","last_fullsized":"2024-05-28T14:14:43.765450+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:26:08.192997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.765787+0000","last_change":"2024-05-28T14:14:43.758776+0000","last_active":"2024-05-28T14:14:43.765787+0000","last_peered":"2024-05-28T14:14:43.765787+0000","last_clean":"2024-05-28T14:14:43.765787+0000","last_became_active":"2024-05-28T14:14:43.758483+0000","last_became_peered":"2024-05-28T14:14:43.758483+0000","last_unstale":"2024-05-28T14:14:43.765787+0000","last_undegraded":"2024-05-28T14:14:43.765787+0000","last_fullsized":"2024-05-28T14:14:43.765787+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:44:14.580012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.755687+0000","last_change":"2024-05-28T14:14:43.756230+0000","last_active":"2024-05-28T14:14:44.755687+0000","last_peered":"2024-05-28T14:14:44.755687+0000","last_clean":"2024-05-28T14:14:44.755687+0000","last_became_active":"2024-05-28T14:14:43.755912+0000","last_became_peered":"2024-05-28T14:14:43.755912+0000","last_unstale":"2024-05-28T14:14:44.755687+0000","last_undegraded":"2024-05-28T14:14:44.755687+0000","last_fullsized":"2024-05-28T14:14:44.755687+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:22.709561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.750515+0000","last_change":"2024-05-28T14:14:43.749113+0000","last_active":"2024-05-28T14:14:43.750515+0000","last_peered":"2024-05-28T14:14:43.750515+0000","last_clean":"2024-05-28T14:14:43.750515+0000","last_became_active":"2024-05-28T14:14:43.748801+0000","last_became_peered":"2024-05-28T14:14:43.748801+0000","last_unstale":"2024-05-28T14:14:43.750515+0000","last_undegraded":"2024-05-28T14:14:43.750515+0000","last_fullsized":"2024-05-28T14:14:43.750515+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:01:10.675534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.772062+0000","last_change":"2024-05-28T14:14:43.769214+0000","last_active":"2024-05-28T14:14:43.772062+0000","last_peered":"2024-05-28T14:14:43.772062+0000","last_clean":"2024-05-28T14:14:43.772062+0000","last_became_active":"2024-05-28T14:14:43.768757+0000","last_became_peered":"2024-05-28T14:14:43.768757+0000","last_unstale":"2024-05-28T14:14:43.772062+0000","last_undegraded":"2024-05-28T14:14:43.772062+0000","last_fullsized":"2024-05-28T14:14:43.772062+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:16.773436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.757383+0000","last_change":"2024-05-28T14:14:43.756647+0000","last_active":"2024-05-28T14:14:43.757383+0000","last_peered":"2024-05-28T14:14:43.757383+0000","last_clean":"2024-05-28T14:14:43.757383+0000","last_became_active":"2024-05-28T14:14:43.756244+0000","last_became_peered":"2024-05-28T14:14:43.756244+0000","last_unstale":"2024-05-28T14:14:43.757383+0000","last_undegraded":"2024-05-28T14:14:43.757383+0000","last_fullsized":"2024-05-28T14:14:43.757383+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:51.774252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.760953+0000","last_change":"2024-05-28T14:14:43.757686+0000","last_active":"2024-05-28T14:14:43.760953+0000","last_peered":"2024-05-28T14:14:43.760953+0000","last_clean":"2024-05-28T14:14:43.760953+0000","last_became_active":"2024-05-28T14:14:43.757288+0000","last_became_peered":"2024-05-28T14:14:43.757288+0000","last_unstale":"2024-05-28T14:14:43.760953+0000","last_undegraded":"2024-05-28T14:14:43.760953+0000","last_fullsized":"2024-05-28T14:14:43.760953+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:05:56.557590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.764372+0000","last_change":"2024-05-28T14:14:43.759851+0000","last_active":"2024-05-28T14:14:43.764372+0000","last_peered":"2024-05-28T14:14:43.764372+0000","last_clean":"2024-05-28T14:14:43.764372+0000","last_became_active":"2024-05-28T14:14:43.759580+0000","last_became_peered":"2024-05-28T14:14:43.759580+0000","last_unstale":"2024-05-28T14:14:43.764372+0000","last_undegraded":"2024-05-28T14:14:43.764372+0000","last_fullsized":"2024-05-28T14:14:43.764372+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:57:52.097273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.774137+0000","last_change":"2024-05-28T14:14:43.768741+0000","last_active":"2024-05-28T14:14:43.774137+0000","last_peered":"2024-05-28T14:14:43.774137+0000","last_clean":"2024-05-28T14:14:43.774137+0000","last_became_active":"2024-05-28T14:14:43.767741+0000","last_became_peered":"2024-05-28T14:14:43.767741+0000","last_unstale":"2024-05-28T14:14:43.774137+0000","last_undegraded":"2024-05-28T14:14:43.774137+0000","last_fullsized":"2024-05-28T14:14:43.774137+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:22:10.225982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.763103+0000","last_change":"2024-05-28T14:14:43.755177+0000","last_active":"2024-05-28T14:14:43.763103+0000","last_peered":"2024-05-28T14:14:43.763103+0000","last_clean":"2024-05-28T14:14:43.763103+0000","last_became_active":"2024-05-28T14:14:43.753794+0000","last_became_peered":"2024-05-28T14:14:43.753794+0000","last_unstale":"2024-05-28T14:14:43.763103+0000","last_undegraded":"2024-05-28T14:14:43.763103+0000","last_fullsized":"2024-05-28T14:14:43.763103+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:08.045261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766911+0000","last_change":"2024-05-28T14:14:43.763725+0000","last_active":"2024-05-28T14:14:43.766911+0000","last_peered":"2024-05-28T14:14:43.766911+0000","last_clean":"2024-05-28T14:14:43.766911+0000","last_became_active":"2024-05-28T14:14:43.763332+0000","last_became_peered":"2024-05-28T14:14:43.763332+0000","last_unstale":"2024-05-28T14:14:43.766911+0000","last_undegraded":"2024-05-28T14:14:43.766911+0000","last_fullsized":"2024-05-28T14:14:43.766911+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:14.042730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766239+0000","last_change":"2024-05-28T14:14:43.759381+0000","last_active":"2024-05-28T14:14:43.766239+0000","last_peered":"2024-05-28T14:14:43.766239+0000","last_clean":"2024-05-28T14:14:43.766239+0000","last_became_active":"2024-05-28T14:14:43.758722+0000","last_became_peered":"2024-05-28T14:14:43.758722+0000","last_unstale":"2024-05-28T14:14:43.766239+0000","last_undegraded":"2024-05-28T14:14:43.766239+0000","last_fullsized":"2024-05-28T14:14:43.766239+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:57:19.311096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.769538+0000","last_change":"2024-05-28T14:14:43.768220+0000","last_active":"2024-05-28T14:14:43.769538+0000","last_peered":"2024-05-28T14:14:43.769538+0000","last_clean":"2024-05-28T14:14:43.769538+0000","last_became_active":"2024-05-28T14:14:43.767925+0000","last_became_peered":"2024-05-28T14:14:43.767925+0000","last_unstale":"2024-05-28T14:14:43.769538+0000","last_undegraded":"2024-05-28T14:14:43.769538+0000","last_fullsized":"2024-05-28T14:14:43.769538+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:39.285543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.750353+0000","last_change":"2024-05-28T14:14:43.746536+0000","last_active":"2024-05-28T14:14:43.750353+0000","last_peered":"2024-05-28T14:14:43.750353+0000","last_clean":"2024-05-28T14:14:43.750353+0000","last_became_active":"2024-05-28T14:14:43.746249+0000","last_became_peered":"2024-05-28T14:14:43.746249+0000","last_unstale":"2024-05-28T14:14:43.750353+0000","last_undegraded":"2024-05-28T14:14:43.750353+0000","last_fullsized":"2024-05-28T14:14:43.750353+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:27:11.131902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.767601+0000","last_change":"2024-05-28T14:14:43.761643+0000","last_active":"2024-05-28T14:14:43.767601+0000","last_peered":"2024-05-28T14:14:43.767601+0000","last_clean":"2024-05-28T14:14:43.767601+0000","last_became_active":"2024-05-28T14:14:43.761161+0000","last_became_peered":"2024-05-28T14:14:43.761161+0000","last_unstale":"2024-05-28T14:14:43.767601+0000","last_undegraded":"2024-05-28T14:14:43.767601+0000","last_fullsized":"2024-05-28T14:14:43.767601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:36:41.091057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.767561+0000","last_change":"2024-05-28T14:14:43.764133+0000","last_active":"2024-05-28T14:14:43.767561+0000","last_peered":"2024-05-28T14:14:43.767561+0000","last_clean":"2024-05-28T14:14:43.767561+0000","last_became_active":"2024-05-28T14:14:43.763522+0000","last_became_peered":"2024-05-28T14:14:43.763522+0000","last_unstale":"2024-05-28T14:14:43.767561+0000","last_undegraded":"2024-05-28T14:14:43.767561+0000","last_fullsized":"2024-05-28T14:14:43.767561+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:03:35.593768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.769727+0000","last_change":"2024-05-28T14:14:43.763447+0000","last_active":"2024-05-28T14:14:43.769727+0000","last_peered":"2024-05-28T14:14:43.769727+0000","last_clean":"2024-05-28T14:14:43.769727+0000","last_became_active":"2024-05-28T14:14:43.763142+0000","last_became_peered":"2024-05-28T14:14:43.763142+0000","last_unstale":"2024-05-28T14:14:43.769727+0000","last_undegraded":"2024-05-28T14:14:43.769727+0000","last_fullsized":"2024-05-28T14:14:43.769727+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:27:22.008435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.754670+0000","last_change":"2024-05-28T14:14:43.747637+0000","last_active":"2024-05-28T14:14:44.754670+0000","last_peered":"2024-05-28T14:14:44.754670+0000","last_clean":"2024-05-28T14:14:44.754670+0000","last_became_active":"2024-05-28T14:14:43.747272+0000","last_became_peered":"2024-05-28T14:14:43.747272+0000","last_unstale":"2024-05-28T14:14:44.754670+0000","last_undegraded":"2024-05-28T14:14:44.754670+0000","last_fullsized":"2024-05-28T14:14:44.754670+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:19.860056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766264+0000","last_change":"2024-05-28T14:14:43.759397+0000","last_active":"2024-05-28T14:14:43.766264+0000","last_peered":"2024-05-28T14:14:43.766264+0000","last_clean":"2024-05-28T14:14:43.766264+0000","last_became_active":"2024-05-28T14:14:43.759136+0000","last_became_peered":"2024-05-28T14:14:43.759136+0000","last_unstale":"2024-05-28T14:14:43.766264+0000","last_undegraded":"2024-05-28T14:14:43.766264+0000","last_fullsized":"2024-05-28T14:14:43.766264+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:48.608539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"131'4","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.775255+0000","last_change":"2024-05-28T14:14:42.751872+0000","last_active":"2024-05-28T14:14:43.775255+0000","last_peered":"2024-05-28T14:14:43.775255+0000","last_clean":"2024-05-28T14:14:43.775255+0000","last_became_active":"2024-05-28T14:14:42.751450+0000","last_became_peered":"2024-05-28T14:14:42.751450+0000","last_unstale":"2024-05-28T14:14:43.775255+0000","last_undegraded":"2024-05-28T14:14:43.775255+0000","last_fullsized":"2024-05-28T14:14:43.775255+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:48:49.635601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.772858+0000","last_change":"2024-05-28T14:14:43.766763+0000","last_active":"2024-05-28T14:14:43.772858+0000","last_peered":"2024-05-28T14:14:43.772858+0000","last_clean":"2024-05-28T14:14:43.772858+0000","last_became_active":"2024-05-28T14:14:43.766347+0000","last_became_peered":"2024-05-28T14:14:43.766347+0000","last_unstale":"2024-05-28T14:14:43.772858+0000","last_undegraded":"2024-05-28T14:14:43.772858+0000","last_fullsized":"2024-05-28T14:14:43.772858+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:02:15.815971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.781328+0000","last_change":"2024-05-28T14:14:42.742952+0000","last_active":"2024-05-28T14:14:43.781328+0000","last_peered":"2024-05-28T14:14:43.781328+0000","last_clean":"2024-05-28T14:14:43.781328+0000","last_became_active":"2024-05-28T14:14:42.742668+0000","last_became_peered":"2024-05-28T14:14:42.742668+0000","last_unstale":"2024-05-28T14:14:43.781328+0000","last_undegraded":"2024-05-28T14:14:43.781328+0000","last_fullsized":"2024-05-28T14:14:43.781328+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:53:04.792926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.773426+0000","last_change":"2024-05-28T14:14:43.765567+0000","last_active":"2024-05-28T14:14:43.773426+0000","last_peered":"2024-05-28T14:14:43.773426+0000","last_clean":"2024-05-28T14:14:43.773426+0000","last_became_active":"2024-05-28T14:14:43.765230+0000","last_became_peered":"2024-05-28T14:14:43.765230+0000","last_unstale":"2024-05-28T14:14:43.773426+0000","last_undegraded":"2024-05-28T14:14:43.773426+0000","last_fullsized":"2024-05-28T14:14:43.773426+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:57:52.097273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.773607+0000","last_change":"2024-05-28T14:14:42.733510+0000","last_active":"2024-05-28T14:14:43.773607+0000","last_peered":"2024-05-28T14:14:43.773607+0000","last_clean":"2024-05-28T14:14:43.773607+0000","last_became_active":"2024-05-28T14:14:42.733231+0000","last_became_peered":"2024-05-28T14:14:42.733231+0000","last_unstale":"2024-05-28T14:14:43.773607+0000","last_undegraded":"2024-05-28T14:14:43.773607+0000","last_fullsized":"2024-05-28T14:14:43.773607+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:37.532653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.762651+0000","last_change":"2024-05-28T14:14:43.760573+0000","last_active":"2024-05-28T14:14:43.762651+0000","last_peered":"2024-05-28T14:14:43.762651+0000","last_clean":"2024-05-28T14:14:43.762651+0000","last_became_active":"2024-05-28T14:14:43.759307+0000","last_became_peered":"2024-05-28T14:14:43.759307+0000","last_unstale":"2024-05-28T14:14:43.762651+0000","last_undegraded":"2024-05-28T14:14:43.762651+0000","last_fullsized":"2024-05-28T14:14:43.762651+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:26.042780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.776144+0000","last_change":"2024-05-28T14:14:42.745411+0000","last_active":"2024-05-28T14:14:43.776144+0000","last_peered":"2024-05-28T14:14:43.776144+0000","last_clean":"2024-05-28T14:14:43.776144+0000","last_became_active":"2024-05-28T14:14:42.745097+0000","last_became_peered":"2024-05-28T14:14:42.745097+0000","last_unstale":"2024-05-28T14:14:43.776144+0000","last_undegraded":"2024-05-28T14:14:43.776144+0000","last_fullsized":"2024-05-28T14:14:43.776144+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:39:18.579687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.756326+0000","last_change":"2024-05-28T14:14:43.751125+0000","last_active":"2024-05-28T14:14:43.756326+0000","last_peered":"2024-05-28T14:14:43.756326+0000","last_clean":"2024-05-28T14:14:43.756326+0000","last_became_active":"2024-05-28T14:14:43.747282+0000","last_became_peered":"2024-05-28T14:14:43.747282+0000","last_unstale":"2024-05-28T14:14:43.756326+0000","last_undegraded":"2024-05-28T14:14:43.756326+0000","last_fullsized":"2024-05-28T14:14:43.756326+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:15:26.978033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.775945+0000","last_change":"2024-05-28T14:14:42.741716+0000","last_active":"2024-05-28T14:14:43.775945+0000","last_peered":"2024-05-28T14:14:43.775945+0000","last_clean":"2024-05-28T14:14:43.775945+0000","last_became_active":"2024-05-28T14:14:42.741425+0000","last_became_peered":"2024-05-28T14:14:42.741425+0000","last_unstale":"2024-05-28T14:14:43.775945+0000","last_undegraded":"2024-05-28T14:14:43.775945+0000","last_fullsized":"2024-05-28T14:14:43.775945+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:36:40.085174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.755183+0000","last_change":"2024-05-28T14:14:43.760960+0000","last_active":"2024-05-28T14:14:44.755183+0000","last_peered":"2024-05-28T14:14:44.755183+0000","last_clean":"2024-05-28T14:14:44.755183+0000","last_became_active":"2024-05-28T14:14:43.760691+0000","last_became_peered":"2024-05-28T14:14:43.760691+0000","last_unstale":"2024-05-28T14:14:44.755183+0000","last_undegraded":"2024-05-28T14:14:44.755183+0000","last_fullsized":"2024-05-28T14:14:44.755183+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:33:40.268549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.775716+0000","last_change":"2024-05-28T14:14:42.739561+0000","last_active":"2024-05-28T14:14:43.775716+0000","last_peered":"2024-05-28T14:14:43.775716+0000","last_clean":"2024-05-28T14:14:43.775716+0000","last_became_active":"2024-05-28T14:14:42.739191+0000","last_became_peered":"2024-05-28T14:14:42.739191+0000","last_unstale":"2024-05-28T14:14:43.775716+0000","last_undegraded":"2024-05-28T14:14:43.775716+0000","last_fullsized":"2024-05-28T14:14:43.775716+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:49:44.727202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766518+0000","last_change":"2024-05-28T14:14:43.762652+0000","last_active":"2024-05-28T14:14:43.766518+0000","last_peered":"2024-05-28T14:14:43.766518+0000","last_clean":"2024-05-28T14:14:43.766518+0000","last_became_active":"2024-05-28T14:14:43.761547+0000","last_became_peered":"2024-05-28T14:14:43.761547+0000","last_unstale":"2024-05-28T14:14:43.766518+0000","last_undegraded":"2024-05-28T14:14:43.766518+0000","last_fullsized":"2024-05-28T14:14:43.766518+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:14:37.836936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.754796+0000","last_change":"2024-05-28T14:14:42.745986+0000","last_active":"2024-05-28T14:14:44.754796+0000","last_peered":"2024-05-28T14:14:44.754796+0000","last_clean":"2024-05-28T14:14:44.754796+0000","last_became_active":"2024-05-28T14:14:42.745691+0000","last_became_peered":"2024-05-28T14:14:42.745691+0000","last_unstale":"2024-05-28T14:14:44.754796+0000","last_undegraded":"2024-05-28T14:14:44.754796+0000","last_fullsized":"2024-05-28T14:14:44.754796+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:22:09.220099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.767406+0000","last_change":"2024-05-28T14:14:43.764538+0000","last_active":"2024-05-28T14:14:43.767406+0000","last_peered":"2024-05-28T14:14:43.767406+0000","last_clean":"2024-05-28T14:14:43.767406+0000","last_became_active":"2024-05-28T14:14:43.762358+0000","last_became_peered":"2024-05-28T14:14:43.762358+0000","last_unstale":"2024-05-28T14:14:43.767406+0000","last_undegraded":"2024-05-28T14:14:43.767406+0000","last_fullsized":"2024-05-28T14:14:43.767406+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:27:11.131902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.751056+0000","last_change":"2024-05-28T14:14:42.747379+0000","last_active":"2024-05-28T14:14:43.751056+0000","last_peered":"2024-05-28T14:14:43.751056+0000","last_clean":"2024-05-28T14:14:43.751056+0000","last_became_active":"2024-05-28T14:14:42.747127+0000","last_became_peered":"2024-05-28T14:14:42.747127+0000","last_unstale":"2024-05-28T14:14:43.751056+0000","last_undegraded":"2024-05-28T14:14:43.751056+0000","last_fullsized":"2024-05-28T14:14:43.751056+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:10:35.041686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.765555+0000","last_change":"2024-05-28T14:14:43.757145+0000","last_active":"2024-05-28T14:14:43.765555+0000","last_peered":"2024-05-28T14:14:43.765555+0000","last_clean":"2024-05-28T14:14:43.765555+0000","last_became_active":"2024-05-28T14:14:43.756803+0000","last_became_peered":"2024-05-28T14:14:43.756803+0000","last_unstale":"2024-05-28T14:14:43.765555+0000","last_undegraded":"2024-05-28T14:14:43.765555+0000","last_fullsized":"2024-05-28T14:14:43.765555+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:29:27.674815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.743163+0000","last_change":"2024-05-28T14:14:42.746521+0000","last_active":"2024-05-28T14:14:43.743163+0000","last_peered":"2024-05-28T14:14:43.743163+0000","last_clean":"2024-05-28T14:14:43.743163+0000","last_became_active":"2024-05-28T14:14:42.746243+0000","last_became_peered":"2024-05-28T14:14:42.746243+0000","last_unstale":"2024-05-28T14:14:43.743163+0000","last_undegraded":"2024-05-28T14:14:43.743163+0000","last_fullsized":"2024-05-28T14:14:43.743163+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:44.045231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.754921+0000","last_change":"2024-05-28T14:14:43.751232+0000","last_active":"2024-05-28T14:14:43.754921+0000","last_peered":"2024-05-28T14:14:43.754921+0000","last_clean":"2024-05-28T14:14:43.754921+0000","last_became_active":"2024-05-28T14:14:43.750768+0000","last_became_peered":"2024-05-28T14:14:43.750768+0000","last_unstale":"2024-05-28T14:14:43.754921+0000","last_undegraded":"2024-05-28T14:14:43.754921+0000","last_fullsized":"2024-05-28T14:14:43.754921+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:14.494237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.741873+0000","last_change":"2024-05-28T14:14:42.745263+0000","last_active":"2024-05-28T14:14:43.741873+0000","last_peered":"2024-05-28T14:14:43.741873+0000","last_clean":"2024-05-28T14:14:43.741873+0000","last_became_active":"2024-05-28T14:14:42.744565+0000","last_became_peered":"2024-05-28T14:14:42.744565+0000","last_unstale":"2024-05-28T14:14:43.741873+0000","last_undegraded":"2024-05-28T14:14:43.741873+0000","last_fullsized":"2024-05-28T14:14:43.741873+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:59.384784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.763407+0000","last_change":"2024-05-28T14:14:43.760442+0000","last_active":"2024-05-28T14:14:43.763407+0000","last_peered":"2024-05-28T14:14:43.763407+0000","last_clean":"2024-05-28T14:14:43.763407+0000","last_became_active":"2024-05-28T14:14:43.760106+0000","last_became_peered":"2024-05-28T14:14:43.760106+0000","last_unstale":"2024-05-28T14:14:43.763407+0000","last_undegraded":"2024-05-28T14:14:43.763407+0000","last_fullsized":"2024-05-28T14:14:43.763407+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:38.538536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"131'3","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.781593+0000","last_change":"2024-05-28T14:14:42.735627+0000","last_active":"2024-05-28T14:14:43.781593+0000","last_peered":"2024-05-28T14:14:43.781593+0000","last_clean":"2024-05-28T14:14:43.781593+0000","last_became_active":"2024-05-28T14:14:42.734095+0000","last_became_peered":"2024-05-28T14:14:42.734095+0000","last_unstale":"2024-05-28T14:14:43.781593+0000","last_undegraded":"2024-05-28T14:14:43.781593+0000","last_fullsized":"2024-05-28T14:14:43.781593+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:07.310371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.774577+0000","last_change":"2024-05-28T14:14:43.768144+0000","last_active":"2024-05-28T14:14:43.774577+0000","last_peered":"2024-05-28T14:14:43.774577+0000","last_clean":"2024-05-28T14:14:43.774577+0000","last_became_active":"2024-05-28T14:14:43.766334+0000","last_became_peered":"2024-05-28T14:14:43.766334+0000","last_unstale":"2024-05-28T14:14:43.774577+0000","last_undegraded":"2024-05-28T14:14:43.774577+0000","last_fullsized":"2024-05-28T14:14:43.774577+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:50:20.486680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"131'6","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.783642+0000","last_change":"2024-05-28T14:14:42.744568+0000","last_active":"2024-05-28T14:14:43.783642+0000","last_peered":"2024-05-28T14:14:43.783642+0000","last_clean":"2024-05-28T14:14:43.783642+0000","last_became_active":"2024-05-28T14:14:42.744240+0000","last_became_peered":"2024-05-28T14:14:42.744240+0000","last_unstale":"2024-05-28T14:14:43.783642+0000","last_undegraded":"2024-05-28T14:14:43.783642+0000","last_fullsized":"2024-05-28T14:14:43.783642+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:25:02.568973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.767579+0000","last_change":"2024-05-28T14:14:43.758776+0000","last_active":"2024-05-28T14:14:43.767579+0000","last_peered":"2024-05-28T14:14:43.767579+0000","last_clean":"2024-05-28T14:14:43.767579+0000","last_became_active":"2024-05-28T14:14:43.758474+0000","last_became_peered":"2024-05-28T14:14:43.758474+0000","last_unstale":"2024-05-28T14:14:43.767579+0000","last_undegraded":"2024-05-28T14:14:43.767579+0000","last_fullsized":"2024-05-28T14:14:43.767579+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:14:41.836526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.747855+0000","last_change":"2024-05-28T14:14:42.737773+0000","last_active":"2024-05-28T14:14:43.747855+0000","last_peered":"2024-05-28T14:14:43.747855+0000","last_clean":"2024-05-28T14:14:43.747855+0000","last_became_active":"2024-05-28T14:14:42.737442+0000","last_became_peered":"2024-05-28T14:14:42.737442+0000","last_unstale":"2024-05-28T14:14:43.747855+0000","last_undegraded":"2024-05-28T14:14:43.747855+0000","last_fullsized":"2024-05-28T14:14:43.747855+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:45:49.229094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766947+0000","last_change":"2024-05-28T14:14:43.762668+0000","last_active":"2024-05-28T14:14:43.766947+0000","last_peered":"2024-05-28T14:14:43.766947+0000","last_clean":"2024-05-28T14:14:43.766947+0000","last_became_active":"2024-05-28T14:14:43.762049+0000","last_became_peered":"2024-05-28T14:14:43.762049+0000","last_unstale":"2024-05-28T14:14:43.766947+0000","last_undegraded":"2024-05-28T14:14:43.766947+0000","last_fullsized":"2024-05-28T14:14:43.766947+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:53:05.798809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.751344+0000","last_change":"2024-05-28T14:14:42.746077+0000","last_active":"2024-05-28T14:14:43.751344+0000","last_peered":"2024-05-28T14:14:43.751344+0000","last_clean":"2024-05-28T14:14:43.751344+0000","last_became_active":"2024-05-28T14:14:42.745811+0000","last_became_peered":"2024-05-28T14:14:42.745811+0000","last_unstale":"2024-05-28T14:14:43.751344+0000","last_undegraded":"2024-05-28T14:14:43.751344+0000","last_fullsized":"2024-05-28T14:14:43.751344+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:26:07.187114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.773086+0000","last_change":"2024-05-28T14:14:43.768732+0000","last_active":"2024-05-28T14:14:43.773086+0000","last_peered":"2024-05-28T14:14:43.773086+0000","last_clean":"2024-05-28T14:14:43.773086+0000","last_became_active":"2024-05-28T14:14:43.768334+0000","last_became_peered":"2024-05-28T14:14:43.768334+0000","last_unstale":"2024-05-28T14:14:43.773086+0000","last_undegraded":"2024-05-28T14:14:43.773086+0000","last_fullsized":"2024-05-28T14:14:43.773086+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:10:44.749184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"131'8","reported_seq":20,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.781998+0000","last_change":"2024-05-28T14:14:42.742861+0000","last_active":"2024-05-28T14:14:43.781998+0000","last_peered":"2024-05-28T14:14:43.781998+0000","last_clean":"2024-05-28T14:14:43.781998+0000","last_became_active":"2024-05-28T14:14:42.742518+0000","last_became_peered":"2024-05-28T14:14:42.742518+0000","last_unstale":"2024-05-28T14:14:43.781998+0000","last_undegraded":"2024-05-28T14:14:43.781998+0000","last_fullsized":"2024-05-28T14:14:43.781998+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:43:29.312316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766344+0000","last_change":"2024-05-28T14:14:43.762081+0000","last_active":"2024-05-28T14:14:43.766344+0000","last_peered":"2024-05-28T14:14:43.766344+0000","last_clean":"2024-05-28T14:14:43.766344+0000","last_became_active":"2024-05-28T14:14:43.761775+0000","last_became_peered":"2024-05-28T14:14:43.761775+0000","last_unstale":"2024-05-28T14:14:43.766344+0000","last_undegraded":"2024-05-28T14:14:43.766344+0000","last_fullsized":"2024-05-28T14:14:43.766344+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:10.253064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.782601+0000","last_change":"2024-05-28T14:14:42.746381+0000","last_active":"2024-05-28T14:14:43.782601+0000","last_peered":"2024-05-28T14:14:43.782601+0000","last_clean":"2024-05-28T14:14:43.782601+0000","last_became_active":"2024-05-28T14:14:42.746098+0000","last_became_peered":"2024-05-28T14:14:42.746098+0000","last_unstale":"2024-05-28T14:14:43.782601+0000","last_undegraded":"2024-05-28T14:14:43.782601+0000","last_fullsized":"2024-05-28T14:14:43.782601+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:04.859086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.768737+0000","last_change":"2024-05-28T14:14:43.756453+0000","last_active":"2024-05-28T14:14:43.768737+0000","last_peered":"2024-05-28T14:14:43.768737+0000","last_clean":"2024-05-28T14:14:43.768737+0000","last_became_active":"2024-05-28T14:14:43.756138+0000","last_became_peered":"2024-05-28T14:14:43.756138+0000","last_unstale":"2024-05-28T14:14:43.768737+0000","last_undegraded":"2024-05-28T14:14:43.768737+0000","last_fullsized":"2024-05-28T14:14:43.768737+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:11.879163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.778751+0000","last_change":"2024-05-28T14:14:42.742055+0000","last_active":"2024-05-28T14:14:43.778751+0000","last_peered":"2024-05-28T14:14:43.778751+0000","last_clean":"2024-05-28T14:14:43.778751+0000","last_became_active":"2024-05-28T14:14:42.741754+0000","last_became_peered":"2024-05-28T14:14:42.741754+0000","last_unstale":"2024-05-28T14:14:43.778751+0000","last_undegraded":"2024-05-28T14:14:43.778751+0000","last_fullsized":"2024-05-28T14:14:43.778751+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:25.036897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.765359+0000","last_change":"2024-05-28T14:14:43.758320+0000","last_active":"2024-05-28T14:14:43.765359+0000","last_peered":"2024-05-28T14:14:43.765359+0000","last_clean":"2024-05-28T14:14:43.765359+0000","last_became_active":"2024-05-28T14:14:43.757996+0000","last_became_peered":"2024-05-28T14:14:43.757996+0000","last_unstale":"2024-05-28T14:14:43.765359+0000","last_undegraded":"2024-05-28T14:14:43.765359+0000","last_fullsized":"2024-05-28T14:14:43.765359+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:31.534150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.755297+0000","last_change":"2024-05-28T14:14:42.734922+0000","last_active":"2024-05-28T14:14:44.755297+0000","last_peered":"2024-05-28T14:14:44.755297+0000","last_clean":"2024-05-28T14:14:44.755297+0000","last_became_active":"2024-05-28T14:14:42.734647+0000","last_became_peered":"2024-05-28T14:14:42.734647+0000","last_unstale":"2024-05-28T14:14:44.755297+0000","last_undegraded":"2024-05-28T14:14:44.755297+0000","last_fullsized":"2024-05-28T14:14:44.755297+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:36:34.326176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766994+0000","last_change":"2024-05-28T14:14:43.764186+0000","last_active":"2024-05-28T14:14:43.766994+0000","last_peered":"2024-05-28T14:14:43.766994+0000","last_clean":"2024-05-28T14:14:43.766994+0000","last_became_active":"2024-05-28T14:14:43.762407+0000","last_became_peered":"2024-05-28T14:14:43.762407+0000","last_unstale":"2024-05-28T14:14:43.766994+0000","last_undegraded":"2024-05-28T14:14:43.766994+0000","last_fullsized":"2024-05-28T14:14:43.766994+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:45.051114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.742553+0000","last_change":"2024-05-28T14:14:42.733737+0000","last_active":"2024-05-28T14:14:43.742553+0000","last_peered":"2024-05-28T14:14:43.742553+0000","last_clean":"2024-05-28T14:14:43.742553+0000","last_became_active":"2024-05-28T14:14:42.733393+0000","last_became_peered":"2024-05-28T14:14:42.733393+0000","last_unstale":"2024-05-28T14:14:43.742553+0000","last_undegraded":"2024-05-28T14:14:43.742553+0000","last_fullsized":"2024-05-28T14:14:43.742553+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:13.036848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.768905+0000","last_change":"2024-05-28T14:14:43.767748+0000","last_active":"2024-05-28T14:14:43.768905+0000","last_peered":"2024-05-28T14:14:43.768905+0000","last_clean":"2024-05-28T14:14:43.768905+0000","last_became_active":"2024-05-28T14:14:43.767351+0000","last_became_peered":"2024-05-28T14:14:43.767351+0000","last_unstale":"2024-05-28T14:14:43.768905+0000","last_undegraded":"2024-05-28T14:14:43.768905+0000","last_fullsized":"2024-05-28T14:14:43.768905+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:53:05.798809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.782102+0000","last_change":"2024-05-28T14:14:42.746787+0000","last_active":"2024-05-28T14:14:43.782102+0000","last_peered":"2024-05-28T14:14:43.782102+0000","last_clean":"2024-05-28T14:14:43.782102+0000","last_became_active":"2024-05-28T14:14:42.746273+0000","last_became_peered":"2024-05-28T14:14:42.746273+0000","last_unstale":"2024-05-28T14:14:43.782102+0000","last_undegraded":"2024-05-28T14:14:43.782102+0000","last_fullsized":"2024-05-28T14:14:43.782102+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:38.279660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.755367+0000","last_change":"2024-05-28T14:14:43.771048+0000","last_active":"2024-05-28T14:14:44.755367+0000","last_peered":"2024-05-28T14:14:44.755367+0000","last_clean":"2024-05-28T14:14:44.755367+0000","last_became_active":"2024-05-28T14:14:43.770751+0000","last_became_peered":"2024-05-28T14:14:43.770751+0000","last_unstale":"2024-05-28T14:14:44.755367+0000","last_undegraded":"2024-05-28T14:14:44.755367+0000","last_fullsized":"2024-05-28T14:14:44.755367+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:34:09.617785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.740542+0000","last_change":"2024-05-28T14:14:42.742397+0000","last_active":"2024-05-28T14:14:43.740542+0000","last_peered":"2024-05-28T14:14:43.740542+0000","last_clean":"2024-05-28T14:14:43.740542+0000","last_became_active":"2024-05-28T14:14:42.742104+0000","last_became_peered":"2024-05-28T14:14:42.742104+0000","last_unstale":"2024-05-28T14:14:43.740542+0000","last_undegraded":"2024-05-28T14:14:43.740542+0000","last_fullsized":"2024-05-28T14:14:43.740542+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:21.703678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.758509+0000","last_change":"2024-05-28T14:14:43.756230+0000","last_active":"2024-05-28T14:14:43.758509+0000","last_peered":"2024-05-28T14:14:43.758509+0000","last_clean":"2024-05-28T14:14:43.758509+0000","last_became_active":"2024-05-28T14:14:43.755945+0000","last_became_peered":"2024-05-28T14:14:43.755945+0000","last_unstale":"2024-05-28T14:14:43.758509+0000","last_undegraded":"2024-05-28T14:14:43.758509+0000","last_fullsized":"2024-05-28T14:14:43.758509+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:08.316254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.739981+0000","last_change":"2024-05-28T14:14:42.737496+0000","last_active":"2024-05-28T14:14:43.739981+0000","last_peered":"2024-05-28T14:14:43.739981+0000","last_clean":"2024-05-28T14:14:43.739981+0000","last_became_active":"2024-05-28T14:14:42.737194+0000","last_became_peered":"2024-05-28T14:14:42.737194+0000","last_unstale":"2024-05-28T14:14:43.739981+0000","last_undegraded":"2024-05-28T14:14:43.739981+0000","last_fullsized":"2024-05-28T14:14:43.739981+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:56:10.665924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.760641+0000","last_change":"2024-05-28T14:14:43.758200+0000","last_active":"2024-05-28T14:14:43.760641+0000","last_peered":"2024-05-28T14:14:43.760641+0000","last_clean":"2024-05-28T14:14:43.760641+0000","last_became_active":"2024-05-28T14:14:43.757875+0000","last_became_peered":"2024-05-28T14:14:43.757875+0000","last_unstale":"2024-05-28T14:14:43.760641+0000","last_undegraded":"2024-05-28T14:14:43.760641+0000","last_fullsized":"2024-05-28T14:14:43.760641+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:48:30.643597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.745077+0000","last_change":"2024-05-28T14:14:42.738809+0000","last_active":"2024-05-28T14:14:43.745077+0000","last_peered":"2024-05-28T14:14:43.745077+0000","last_clean":"2024-05-28T14:14:43.745077+0000","last_became_active":"2024-05-28T14:14:42.738494+0000","last_became_peered":"2024-05-28T14:14:42.738494+0000","last_unstale":"2024-05-28T14:14:43.745077+0000","last_undegraded":"2024-05-28T14:14:43.745077+0000","last_fullsized":"2024-05-28T14:14:43.745077+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:17.145363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.755630+0000","last_change":"2024-05-28T14:14:43.770616+0000","last_active":"2024-05-28T14:14:44.755630+0000","last_peered":"2024-05-28T14:14:44.755630+0000","last_clean":"2024-05-28T14:14:44.755630+0000","last_became_active":"2024-05-28T14:14:43.770253+0000","last_became_peered":"2024-05-28T14:14:43.770253+0000","last_unstale":"2024-05-28T14:14:44.755630+0000","last_undegraded":"2024-05-28T14:14:44.755630+0000","last_fullsized":"2024-05-28T14:14:44.755630+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:49:45.733085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.776273+0000","last_change":"2024-05-28T14:14:42.746355+0000","last_active":"2024-05-28T14:14:43.776273+0000","last_peered":"2024-05-28T14:14:43.776273+0000","last_clean":"2024-05-28T14:14:43.776273+0000","last_became_active":"2024-05-28T14:14:42.745926+0000","last_became_peered":"2024-05-28T14:14:42.745926+0000","last_unstale":"2024-05-28T14:14:43.776273+0000","last_undegraded":"2024-05-28T14:14:43.776273+0000","last_fullsized":"2024-05-28T14:14:43.776273+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:54:47.273264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.765619+0000","last_change":"2024-05-28T14:14:43.763195+0000","last_active":"2024-05-28T14:14:43.765619+0000","last_peered":"2024-05-28T14:14:43.765619+0000","last_clean":"2024-05-28T14:14:43.765619+0000","last_became_active":"2024-05-28T14:14:43.761832+0000","last_became_peered":"2024-05-28T14:14:43.761832+0000","last_unstale":"2024-05-28T14:14:43.765619+0000","last_undegraded":"2024-05-28T14:14:43.765619+0000","last_fullsized":"2024-05-28T14:14:43.765619+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:51:28.086771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.752284+0000","last_change":"2024-05-28T14:14:42.749733+0000","last_active":"2024-05-28T14:14:43.752284+0000","last_peered":"2024-05-28T14:14:43.752284+0000","last_clean":"2024-05-28T14:14:43.752284+0000","last_became_active":"2024-05-28T14:14:42.749348+0000","last_became_peered":"2024-05-28T14:14:42.749348+0000","last_unstale":"2024-05-28T14:14:43.752284+0000","last_undegraded":"2024-05-28T14:14:43.752284+0000","last_fullsized":"2024-05-28T14:14:43.752284+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:17.145363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.767216+0000","last_change":"2024-05-28T14:14:43.764008+0000","last_active":"2024-05-28T14:14:43.767216+0000","last_peered":"2024-05-28T14:14:43.767216+0000","last_clean":"2024-05-28T14:14:43.767216+0000","last_became_active":"2024-05-28T14:14:43.762648+0000","last_became_peered":"2024-05-28T14:14:43.762648+0000","last_unstale":"2024-05-28T14:14:43.767216+0000","last_undegraded":"2024-05-28T14:14:43.767216+0000","last_fullsized":"2024-05-28T14:14:43.767216+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:05.864969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.751619+0000","last_change":"2024-05-28T14:14:42.744189+0000","last_active":"2024-05-28T14:14:43.751619+0000","last_peered":"2024-05-28T14:14:43.751619+0000","last_clean":"2024-05-28T14:14:43.751619+0000","last_became_active":"2024-05-28T14:14:42.743858+0000","last_became_peered":"2024-05-28T14:14:42.743858+0000","last_unstale":"2024-05-28T14:14:43.751619+0000","last_undegraded":"2024-05-28T14:14:43.751619+0000","last_fullsized":"2024-05-28T14:14:43.751619+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:03:34.587886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.755152+0000","last_change":"2024-05-28T14:14:43.758779+0000","last_active":"2024-05-28T14:14:44.755152+0000","last_peered":"2024-05-28T14:14:44.755152+0000","last_clean":"2024-05-28T14:14:44.755152+0000","last_became_active":"2024-05-28T14:14:43.758516+0000","last_became_peered":"2024-05-28T14:14:43.758516+0000","last_unstale":"2024-05-28T14:14:44.755152+0000","last_undegraded":"2024-05-28T14:14:44.755152+0000","last_fullsized":"2024-05-28T14:14:44.755152+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:13:21.112865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.745589+0000","last_change":"2024-05-28T14:14:42.748521+0000","last_active":"2024-05-28T14:14:43.745589+0000","last_peered":"2024-05-28T14:14:43.745589+0000","last_clean":"2024-05-28T14:14:43.745589+0000","last_became_active":"2024-05-28T14:14:42.748193+0000","last_became_peered":"2024-05-28T14:14:42.748193+0000","last_unstale":"2024-05-28T14:14:43.745589+0000","last_undegraded":"2024-05-28T14:14:43.745589+0000","last_fullsized":"2024-05-28T14:14:43.745589+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:48:29.637714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.774765+0000","last_change":"2024-05-28T14:14:43.767848+0000","last_active":"2024-05-28T14:14:43.774765+0000","last_peered":"2024-05-28T14:14:43.774765+0000","last_clean":"2024-05-28T14:14:43.774765+0000","last_became_active":"2024-05-28T14:14:43.767552+0000","last_became_peered":"2024-05-28T14:14:43.767552+0000","last_unstale":"2024-05-28T14:14:43.774765+0000","last_undegraded":"2024-05-28T14:14:43.774765+0000","last_fullsized":"2024-05-28T14:14:43.774765+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:56:28.627044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.751566+0000","last_change":"2024-05-28T14:14:42.746605+0000","last_active":"2024-05-28T14:14:43.751566+0000","last_peered":"2024-05-28T14:14:43.751566+0000","last_clean":"2024-05-28T14:14:43.751566+0000","last_became_active":"2024-05-28T14:14:42.746338+0000","last_became_peered":"2024-05-28T14:14:42.746338+0000","last_unstale":"2024-05-28T14:14:43.751566+0000","last_undegraded":"2024-05-28T14:14:43.751566+0000","last_fullsized":"2024-05-28T14:14:43.751566+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:51:27.080888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.768968+0000","last_change":"2024-05-28T14:14:43.760455+0000","last_active":"2024-05-28T14:14:43.768968+0000","last_peered":"2024-05-28T14:14:43.768968+0000","last_clean":"2024-05-28T14:14:43.768968+0000","last_became_active":"2024-05-28T14:14:43.760063+0000","last_became_peered":"2024-05-28T14:14:43.760063+0000","last_unstale":"2024-05-28T14:14:43.768968+0000","last_undegraded":"2024-05-28T14:14:43.768968+0000","last_fullsized":"2024-05-28T14:14:43.768968+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:00:34.527970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.749327+0000","last_change":"2024-05-28T14:14:42.749618+0000","last_active":"2024-05-28T14:14:43.749327+0000","last_peered":"2024-05-28T14:14:43.749327+0000","last_clean":"2024-05-28T14:14:43.749327+0000","last_became_active":"2024-05-28T14:14:42.749259+0000","last_became_peered":"2024-05-28T14:14:42.749259+0000","last_unstale":"2024-05-28T14:14:43.749327+0000","last_undegraded":"2024-05-28T14:14:43.749327+0000","last_fullsized":"2024-05-28T14:14:43.749327+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:44:02.935848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766609+0000","last_change":"2024-05-28T14:14:43.761804+0000","last_active":"2024-05-28T14:14:43.766609+0000","last_peered":"2024-05-28T14:14:43.766609+0000","last_clean":"2024-05-28T14:14:43.766609+0000","last_became_active":"2024-05-28T14:14:43.761382+0000","last_became_peered":"2024-05-28T14:14:43.761382+0000","last_unstale":"2024-05-28T14:14:43.766609+0000","last_undegraded":"2024-05-28T14:14:43.766609+0000","last_fullsized":"2024-05-28T14:14:43.766609+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:44:09.700729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.739994+0000","last_change":"2024-05-28T14:14:42.739854+0000","last_active":"2024-05-28T14:14:43.739994+0000","last_peered":"2024-05-28T14:14:43.739994+0000","last_clean":"2024-05-28T14:14:43.739994+0000","last_became_active":"2024-05-28T14:14:42.738369+0000","last_became_peered":"2024-05-28T14:14:42.738369+0000","last_unstale":"2024-05-28T14:14:43.739994+0000","last_undegraded":"2024-05-28T14:14:43.739994+0000","last_fullsized":"2024-05-28T14:14:43.739994+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:34:08.611902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.767427+0000","last_change":"2024-05-28T14:14:43.765205+0000","last_active":"2024-05-28T14:14:43.767427+0000","last_peered":"2024-05-28T14:14:43.767427+0000","last_clean":"2024-05-28T14:14:43.767427+0000","last_became_active":"2024-05-28T14:14:43.764879+0000","last_became_peered":"2024-05-28T14:14:43.764879+0000","last_unstale":"2024-05-28T14:14:43.767427+0000","last_undegraded":"2024-05-28T14:14:43.767427+0000","last_fullsized":"2024-05-28T14:14:43.767427+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:14.494237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.745266+0000","last_change":"2024-05-28T14:14:42.730611+0000","last_active":"2024-05-28T14:14:43.745266+0000","last_peered":"2024-05-28T14:14:43.745266+0000","last_clean":"2024-05-28T14:14:43.745266+0000","last_became_active":"2024-05-28T14:14:42.730333+0000","last_became_peered":"2024-05-28T14:14:42.730333+0000","last_unstale":"2024-05-28T14:14:43.745266+0000","last_undegraded":"2024-05-28T14:14:43.745266+0000","last_fullsized":"2024-05-28T14:14:43.745266+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:05:55.551707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.764935+0000","last_change":"2024-05-28T14:14:43.761743+0000","last_active":"2024-05-28T14:14:43.764935+0000","last_peered":"2024-05-28T14:14:43.764935+0000","last_clean":"2024-05-28T14:14:43.764935+0000","last_became_active":"2024-05-28T14:14:43.760828+0000","last_became_peered":"2024-05-28T14:14:43.760828+0000","last_unstale":"2024-05-28T14:14:43.764935+0000","last_undegraded":"2024-05-28T14:14:43.764935+0000","last_fullsized":"2024-05-28T14:14:43.764935+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:08.045261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.774995+0000","last_change":"2024-05-28T14:14:42.751868+0000","last_active":"2024-05-28T14:14:43.774995+0000","last_peered":"2024-05-28T14:14:43.774995+0000","last_clean":"2024-05-28T14:14:43.774995+0000","last_became_active":"2024-05-28T14:14:42.751447+0000","last_became_peered":"2024-05-28T14:14:42.751447+0000","last_unstale":"2024-05-28T14:14:43.774995+0000","last_undegraded":"2024-05-28T14:14:43.774995+0000","last_fullsized":"2024-05-28T14:14:43.774995+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:28:17.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":376,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.746410+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:14:43.746410+0000","last_peered":"2024-05-28T14:14:43.746410+0000","last_clean":"2024-05-28T14:14:43.746410+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:14:43.746410+0000","last_undegraded":"2024-05-28T14:14:43.746410+0000","last_fullsized":"2024-05-28T14:14:43.746410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575049,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136004,"kb_used_data":2164,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94235836,"statfs":{"total":96636764160,"available":96497496064,"internally_reserved":0,"allocated":2215936,"data_stored":1302702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":137034186},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575049,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135444,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236396,"statfs":{"total":96636764160,"available":96498069504,"internally_reserved":0,"allocated":1646592,"data_stored":720777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":137034182},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575051,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86864,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284976,"statfs":{"total":96636764160,"available":96547815424,"internally_reserved":0,"allocated":1642496,"data_stored":719277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":87292358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149324,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222516,"statfs":{"total":96636764160,"available":96483856384,"internally_reserved":0,"allocated":1638400,"data_stored":719235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575049,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145172,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226668,"statfs":{"total":96636764160,"available":96488108032,"internally_reserved":0,"allocated":1642496,"data_stored":719325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":146995656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141016,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230824,"statfs":{"total":96636764160,"available":96492363776,"internally_reserved":0,"allocated":1642496,"data_stored":719781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136832,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235008,"statfs":{"total":96636764160,"available":96496648192,"internally_reserved":0,"allocated":1622016,"data_stored":712334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575050,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149900,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221940,"statfs":{"total":96636764160,"available":96483266560,"internally_reserved":0,"allocated":2228224,"data_stored":1309533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:14:47.868 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:14:48.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc458800640 1 Processor -- start 2024-05-28T14:14:48.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc458800640 1 -- start start 2024-05-28T14:14:48.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc458800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc45009b160 0x7fc4500cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc458800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc4500b1a80 0x7fc4500cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc458800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4500d0330 con 0x7fc45015e990 2024-05-28T14:14:48.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc458800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc45006d2f0 con 0x7fc45009b160 2024-05-28T14:14:48.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc458800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc45006cca0 con 0x7fc4500b1a80 2024-05-28T14:14:48.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc455400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc45009b160 0x7fc4500cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc455400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc45009b160 0x7fc4500cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47678/0 (socket says 172.21.15.42:47678) 2024-05-28T14:14:48.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc455400640 1 -- 172.21.15.42:0/226033503 learned_addr learned my addr 172.21.15.42:0/226033503 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:48.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc456800640 1 --2- 172.21.15.42:0/226033503 >> v2:172.21.15.63:3300/0 conn(0x7fc4500b1a80 0x7fc4500cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc455400640 1 -- 172.21.15.42:0/226033503 >> v2:172.21.15.63:3300/0 conn(0x7fc4500b1a80 msgr2=0x7fc4500cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:48.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc455400640 1 --2- 172.21.15.42:0/226033503 >> v2:172.21.15.63:3300/0 conn(0x7fc4500b1a80 0x7fc4500cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc455400640 1 -- 172.21.15.42:0/226033503 >> v1:172.21.15.42:6789/0 conn(0x7fc45015e990 legacy=0x7fc45009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.062+0000 7fc455400640 1 -- 172.21.15.42:0/226033503 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45006cf80 con 0x7fc45009b160 2024-05-28T14:14:48.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.066+0000 7fc455400640 1 --2- 172.21.15.42:0/226033503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc45009b160 0x7fc4500cd2a0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fc450050400 tx=0x7fc44001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71e86b08dc2098b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:48.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.066+0000 7fc454a00640 1 -- 172.21.15.42:0/226033503 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4400b1070 con 0x7fc45009b160 2024-05-28T14:14:48.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.066+0000 7fc454a00640 1 -- 172.21.15.42:0/226033503 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4400ac600 con 0x7fc45009b160 2024-05-28T14:14:48.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.066+0000 7fc454a00640 1 -- 172.21.15.42:0/226033503 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4400ac9d0 con 0x7fc45009b160 2024-05-28T14:14:48.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.066+0000 7fc458800640 1 -- 172.21.15.42:0/226033503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc45009b160 msgr2=0x7fc4500cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.066+0000 7fc458800640 1 --2- 172.21.15.42:0/226033503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc45009b160 0x7fc4500cd2a0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fc450050400 tx=0x7fc44001c490 comp rx=0 tx=0).stop 2024-05-28T14:14:48.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.066+0000 7fc458800640 1 -- 172.21.15.42:0/226033503 shutdown_connections 2024-05-28T14:14:48.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.066+0000 7fc458800640 1 --2- 172.21.15.42:0/226033503 >> v2:172.21.15.63:3300/0 conn(0x7fc4500b1a80 0x7fc4500cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.066+0000 7fc458800640 1 --2- 172.21.15.42:0/226033503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc45009b160 0x7fc4500cd2a0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.066+0000 7fc458800640 1 -- 172.21.15.42:0/226033503 >> 172.21.15.42:0/226033503 conn(0x7fc4500b3fa0 msgr2=0x7fc4500b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:48.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.066+0000 7fc458800640 1 -- 172.21.15.42:0/226033503 shutdown_connections 2024-05-28T14:14:48.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.066+0000 7fc458800640 1 -- 172.21.15.42:0/226033503 wait complete. 2024-05-28T14:14:48.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.066+0000 7fc458800640 1 Processor -- start 2024-05-28T14:14:48.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.066+0000 7fc458800640 1 -- start start 2024-05-28T14:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc458800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc45009b160 0x7fc4500cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc458800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4500b1a80 0x7fc4500c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc458800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc45015e990 0x7fc45014d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc458800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc45016b950 con 0x7fc45009b160 2024-05-28T14:14:48.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc458800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc45016b7d0 con 0x7fc4500b1a80 2024-05-28T14:14:48.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc458800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc45016bad0 con 0x7fc45015e990 2024-05-28T14:14:48.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc455e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc45009b160 0x7fc4500cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc455e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc45009b160 0x7fc4500cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47680/0 (socket says 172.21.15.42:47680) 2024-05-28T14:14:48.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc455e00640 1 -- 172.21.15.42:0/2899052180 learned_addr learned my addr 172.21.15.42:0/2899052180 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:48.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc455400640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4500b1a80 0x7fc4500c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc456800640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc45015e990 0x7fc45014d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc455e00640 1 -- 172.21.15.42:0/2899052180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc45015e990 msgr2=0x7fc45014d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc455e00640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc45015e990 0x7fc45014d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc455e00640 1 -- 172.21.15.42:0/2899052180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4500b1a80 msgr2=0x7fc4500c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc455e00640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4500b1a80 0x7fc4500c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc455e00640 1 -- 172.21.15.42:0/2899052180 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45015c160 con 0x7fc45009b160 2024-05-28T14:14:48.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc456800640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc45015e990 0x7fc45014d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:48.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc455e00640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc45009b160 0x7fc4500cbfe0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fc44400c370 tx=0x7fc44400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:48.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc44a000640 1 -- 172.21.15.42:0/2899052180 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4440052d0 con 0x7fc45009b160 2024-05-28T14:14:48.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc44a000640 1 -- 172.21.15.42:0/2899052180 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc444005470 con 0x7fc45009b160 2024-05-28T14:14:48.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc458800640 1 -- 172.21.15.42:0/2899052180 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45015c440 con 0x7fc45009b160 2024-05-28T14:14:48.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc44a000640 1 -- 172.21.15.42:0/2899052180 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc444007260 con 0x7fc45009b160 2024-05-28T14:14:48.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc458800640 1 -- 172.21.15.42:0/2899052180 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45015ca00 con 0x7fc45009b160 2024-05-28T14:14:48.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.070+0000 7fc458800640 1 -- 172.21.15.42:0/2899052180 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc450068a90 con 0x7fc45009b160 2024-05-28T14:14:48.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.074+0000 7fc44a000640 1 -- 172.21.15.42:0/2899052180 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7fc444007400 con 0x7fc45009b160 2024-05-28T14:14:48.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.074+0000 7fc44a000640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc434069fa0 0x7fc43406c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.074+0000 7fc44a000640 1 -- 172.21.15.42:0/2899052180 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7fc444084d30 con 0x7fc45009b160 2024-05-28T14:14:48.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.074+0000 7fc455400640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc434069fa0 0x7fc43406c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.074+0000 7fc455400640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc434069fa0 0x7fc43406c460 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc4400ae000 tx=0x7fc440008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:48.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.078+0000 7fc44a000640 1 -- 172.21.15.42:0/2899052180 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc444056df0 con 0x7fc45009b160 2024-05-28T14:14:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.238+0000 7fc458800640 1 -- 172.21.15.42:0/2899052180 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4500cb7c0 con 0x7fc434069fa0 2024-05-28T14:14:48.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.246+0000 7fc44a000640 1 -- 172.21.15.42:0/2899052180 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252442 (secure 0 0 0) 0x7fc4400b6460 con 0x7fc434069fa0 2024-05-28T14:14:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:48.236 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:14:48.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.250+0000 7fc458800640 1 -- 172.21.15.42:0/2899052180 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc434069fa0 msgr2=0x7fc43406c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.250+0000 7fc458800640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc434069fa0 0x7fc43406c460 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc4400ae000 tx=0x7fc440008040 comp rx=0 tx=0).stop 2024-05-28T14:14:48.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.250+0000 7fc458800640 1 -- 172.21.15.42:0/2899052180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc45009b160 msgr2=0x7fc4500cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.250+0000 7fc458800640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc45009b160 0x7fc4500cbfe0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fc44400c370 tx=0x7fc44400c830 comp rx=0 tx=0).stop 2024-05-28T14:14:48.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.250+0000 7fc458800640 1 -- 172.21.15.42:0/2899052180 shutdown_connections 2024-05-28T14:14:48.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.250+0000 7fc458800640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc434069fa0 0x7fc43406c460 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.250+0000 7fc458800640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc45015e990 0x7fc45014d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.250+0000 7fc458800640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4500b1a80 0x7fc4500c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.250+0000 7fc458800640 1 --2- 172.21.15.42:0/2899052180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc45009b160 0x7fc4500cbfe0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.250+0000 7fc458800640 1 -- 172.21.15.42:0/2899052180 >> 172.21.15.42:0/2899052180 conn(0x7fc4500b3fa0 msgr2=0x7fc4501576f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:48.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.250+0000 7fc458800640 1 -- 172.21.15.42:0/2899052180 shutdown_connections 2024-05-28T14:14:48.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.250+0000 7fc458800640 1 -- 172.21.15.42:0/2899052180 wait complete. 2024-05-28T14:14:48.255 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":624,"stamp":"2024-05-28T14:14:47.195444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1080556,"kb_used_data":13944,"kb_used_omap":12,"kb_used_meta":1066483,"kb_avail":753894164,"statfs":{"total":773094113280,"available":771987623936,"internally_reserved":0,"allocated":14278656,"data_stored":6922964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1092079154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.477013"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.769185+0000","last_change":"2024-05-28T14:14:43.766434+0000","last_active":"2024-05-28T14:14:43.769185+0000","last_peered":"2024-05-28T14:14:43.769185+0000","last_clean":"2024-05-28T14:14:43.769185+0000","last_became_active":"2024-05-28T14:14:43.766125+0000","last_became_peered":"2024-05-28T14:14:43.766125+0000","last_unstale":"2024-05-28T14:14:43.769185+0000","last_undegraded":"2024-05-28T14:14:43.769185+0000","last_fullsized":"2024-05-28T14:14:43.769185+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:39:19.585570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.771000+0000","last_change":"2024-05-28T14:14:43.769925+0000","last_active":"2024-05-28T14:14:43.771000+0000","last_peered":"2024-05-28T14:14:43.771000+0000","last_clean":"2024-05-28T14:14:43.771000+0000","last_became_active":"2024-05-28T14:14:43.769641+0000","last_became_peered":"2024-05-28T14:14:43.769641+0000","last_unstale":"2024-05-28T14:14:43.771000+0000","last_undegraded":"2024-05-28T14:14:43.771000+0000","last_fullsized":"2024-05-28T14:14:43.771000+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:11.879163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.773239+0000","last_change":"2024-05-28T14:14:43.769199+0000","last_active":"2024-05-28T14:14:43.773239+0000","last_peered":"2024-05-28T14:14:43.773239+0000","last_clean":"2024-05-28T14:14:43.773239+0000","last_became_active":"2024-05-28T14:14:43.764098+0000","last_became_peered":"2024-05-28T14:14:43.764098+0000","last_unstale":"2024-05-28T14:14:43.773239+0000","last_undegraded":"2024-05-28T14:14:43.773239+0000","last_fullsized":"2024-05-28T14:14:43.773239+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:14:36.957242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.765583+0000","last_change":"2024-05-28T14:14:43.759234+0000","last_active":"2024-05-28T14:14:43.765583+0000","last_peered":"2024-05-28T14:14:43.765583+0000","last_clean":"2024-05-28T14:14:43.765583+0000","last_became_active":"2024-05-28T14:14:43.758925+0000","last_became_peered":"2024-05-28T14:14:43.758925+0000","last_unstale":"2024-05-28T14:14:43.765583+0000","last_undegraded":"2024-05-28T14:14:43.765583+0000","last_fullsized":"2024-05-28T14:14:43.765583+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:10:36.047568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.772958+0000","last_change":"2024-05-28T14:14:43.766612+0000","last_active":"2024-05-28T14:14:43.772958+0000","last_peered":"2024-05-28T14:14:43.772958+0000","last_clean":"2024-05-28T14:14:43.772958+0000","last_became_active":"2024-05-28T14:14:43.764703+0000","last_became_peered":"2024-05-28T14:14:43.764703+0000","last_unstale":"2024-05-28T14:14:43.772958+0000","last_undegraded":"2024-05-28T14:14:43.772958+0000","last_fullsized":"2024-05-28T14:14:43.772958+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:16.773436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.755190+0000","last_change":"2024-05-28T14:14:43.762958+0000","last_active":"2024-05-28T14:14:44.755190+0000","last_peered":"2024-05-28T14:14:44.755190+0000","last_clean":"2024-05-28T14:14:44.755190+0000","last_became_active":"2024-05-28T14:14:43.762689+0000","last_became_peered":"2024-05-28T14:14:43.762689+0000","last_unstale":"2024-05-28T14:14:44.755190+0000","last_undegraded":"2024-05-28T14:14:44.755190+0000","last_fullsized":"2024-05-28T14:14:44.755190+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:56:11.671806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.754669+0000","last_change":"2024-05-28T14:14:43.770900+0000","last_active":"2024-05-28T14:14:44.754669+0000","last_peered":"2024-05-28T14:14:44.754669+0000","last_clean":"2024-05-28T14:14:44.754669+0000","last_became_active":"2024-05-28T14:14:43.770616+0000","last_became_peered":"2024-05-28T14:14:43.770616+0000","last_unstale":"2024-05-28T14:14:44.754669+0000","last_undegraded":"2024-05-28T14:14:44.754669+0000","last_fullsized":"2024-05-28T14:14:44.754669+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:07:11.105627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766491+0000","last_change":"2024-05-28T14:14:43.762262+0000","last_active":"2024-05-28T14:14:43.766491+0000","last_peered":"2024-05-28T14:14:43.766491+0000","last_clean":"2024-05-28T14:14:43.766491+0000","last_became_active":"2024-05-28T14:14:43.761977+0000","last_became_peered":"2024-05-28T14:14:43.761977+0000","last_unstale":"2024-05-28T14:14:43.766491+0000","last_undegraded":"2024-05-28T14:14:43.766491+0000","last_fullsized":"2024-05-28T14:14:43.766491+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:55:59.790610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.773781+0000","last_change":"2024-05-28T14:14:43.770664+0000","last_active":"2024-05-28T14:14:43.773781+0000","last_peered":"2024-05-28T14:14:43.773781+0000","last_clean":"2024-05-28T14:14:43.773781+0000","last_became_active":"2024-05-28T14:14:43.770335+0000","last_became_peered":"2024-05-28T14:14:43.770335+0000","last_unstale":"2024-05-28T14:14:43.773781+0000","last_undegraded":"2024-05-28T14:14:43.773781+0000","last_fullsized":"2024-05-28T14:14:43.773781+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:36:35.332059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.771021+0000","last_change":"2024-05-28T14:14:43.769965+0000","last_active":"2024-05-28T14:14:43.771021+0000","last_peered":"2024-05-28T14:14:43.771021+0000","last_clean":"2024-05-28T14:14:43.771021+0000","last_became_active":"2024-05-28T14:14:43.769687+0000","last_became_peered":"2024-05-28T14:14:43.769687+0000","last_unstale":"2024-05-28T14:14:43.771021+0000","last_undegraded":"2024-05-28T14:14:43.771021+0000","last_fullsized":"2024-05-28T14:14:43.771021+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:00:34.527970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.764294+0000","last_change":"2024-05-28T14:14:43.758128+0000","last_active":"2024-05-28T14:14:43.764294+0000","last_peered":"2024-05-28T14:14:43.764294+0000","last_clean":"2024-05-28T14:14:43.764294+0000","last_became_active":"2024-05-28T14:14:43.757685+0000","last_became_peered":"2024-05-28T14:14:43.757685+0000","last_unstale":"2024-05-28T14:14:43.764294+0000","last_undegraded":"2024-05-28T14:14:43.764294+0000","last_fullsized":"2024-05-28T14:14:43.764294+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:27:22.008435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.773718+0000","last_change":"2024-05-28T14:14:43.769235+0000","last_active":"2024-05-28T14:14:43.773718+0000","last_peered":"2024-05-28T14:14:43.773718+0000","last_clean":"2024-05-28T14:14:43.773718+0000","last_became_active":"2024-05-28T14:14:43.768939+0000","last_became_peered":"2024-05-28T14:14:43.768939+0000","last_unstale":"2024-05-28T14:14:43.773718+0000","last_undegraded":"2024-05-28T14:14:43.773718+0000","last_fullsized":"2024-05-28T14:14:43.773718+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:58:09.147319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.768877+0000","last_change":"2024-05-28T14:14:43.767408+0000","last_active":"2024-05-28T14:14:43.768877+0000","last_peered":"2024-05-28T14:14:43.768877+0000","last_clean":"2024-05-28T14:14:43.768877+0000","last_became_active":"2024-05-28T14:14:43.766269+0000","last_became_peered":"2024-05-28T14:14:43.766269+0000","last_unstale":"2024-05-28T14:14:43.768877+0000","last_undegraded":"2024-05-28T14:14:43.768877+0000","last_fullsized":"2024-05-28T14:14:43.768877+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:08.316254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766915+0000","last_change":"2024-05-28T14:14:43.762819+0000","last_active":"2024-05-28T14:14:43.766915+0000","last_peered":"2024-05-28T14:14:43.766915+0000","last_clean":"2024-05-28T14:14:43.766915+0000","last_became_active":"2024-05-28T14:14:43.762494+0000","last_became_peered":"2024-05-28T14:14:43.762494+0000","last_unstale":"2024-05-28T14:14:43.766915+0000","last_undegraded":"2024-05-28T14:14:43.766915+0000","last_fullsized":"2024-05-28T14:14:43.766915+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:39.285543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.765450+0000","last_change":"2024-05-28T14:14:43.761245+0000","last_active":"2024-05-28T14:14:43.765450+0000","last_peered":"2024-05-28T14:14:43.765450+0000","last_clean":"2024-05-28T14:14:43.765450+0000","last_became_active":"2024-05-28T14:14:43.760977+0000","last_became_peered":"2024-05-28T14:14:43.760977+0000","last_unstale":"2024-05-28T14:14:43.765450+0000","last_undegraded":"2024-05-28T14:14:43.765450+0000","last_fullsized":"2024-05-28T14:14:43.765450+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:26:08.192997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.765787+0000","last_change":"2024-05-28T14:14:43.758776+0000","last_active":"2024-05-28T14:14:43.765787+0000","last_peered":"2024-05-28T14:14:43.765787+0000","last_clean":"2024-05-28T14:14:43.765787+0000","last_became_active":"2024-05-28T14:14:43.758483+0000","last_became_peered":"2024-05-28T14:14:43.758483+0000","last_unstale":"2024-05-28T14:14:43.765787+0000","last_undegraded":"2024-05-28T14:14:43.765787+0000","last_fullsized":"2024-05-28T14:14:43.765787+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:44:14.580012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.755687+0000","last_change":"2024-05-28T14:14:43.756230+0000","last_active":"2024-05-28T14:14:44.755687+0000","last_peered":"2024-05-28T14:14:44.755687+0000","last_clean":"2024-05-28T14:14:44.755687+0000","last_became_active":"2024-05-28T14:14:43.755912+0000","last_became_peered":"2024-05-28T14:14:43.755912+0000","last_unstale":"2024-05-28T14:14:44.755687+0000","last_undegraded":"2024-05-28T14:14:44.755687+0000","last_fullsized":"2024-05-28T14:14:44.755687+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:22.709561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.750515+0000","last_change":"2024-05-28T14:14:43.749113+0000","last_active":"2024-05-28T14:14:43.750515+0000","last_peered":"2024-05-28T14:14:43.750515+0000","last_clean":"2024-05-28T14:14:43.750515+0000","last_became_active":"2024-05-28T14:14:43.748801+0000","last_became_peered":"2024-05-28T14:14:43.748801+0000","last_unstale":"2024-05-28T14:14:43.750515+0000","last_undegraded":"2024-05-28T14:14:43.750515+0000","last_fullsized":"2024-05-28T14:14:43.750515+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:01:10.675534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.772062+0000","last_change":"2024-05-28T14:14:43.769214+0000","last_active":"2024-05-28T14:14:43.772062+0000","last_peered":"2024-05-28T14:14:43.772062+0000","last_clean":"2024-05-28T14:14:43.772062+0000","last_became_active":"2024-05-28T14:14:43.768757+0000","last_became_peered":"2024-05-28T14:14:43.768757+0000","last_unstale":"2024-05-28T14:14:43.772062+0000","last_undegraded":"2024-05-28T14:14:43.772062+0000","last_fullsized":"2024-05-28T14:14:43.772062+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:16.773436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.757383+0000","last_change":"2024-05-28T14:14:43.756647+0000","last_active":"2024-05-28T14:14:43.757383+0000","last_peered":"2024-05-28T14:14:43.757383+0000","last_clean":"2024-05-28T14:14:43.757383+0000","last_became_active":"2024-05-28T14:14:43.756244+0000","last_became_peered":"2024-05-28T14:14:43.756244+0000","last_unstale":"2024-05-28T14:14:43.757383+0000","last_undegraded":"2024-05-28T14:14:43.757383+0000","last_fullsized":"2024-05-28T14:14:43.757383+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:51.774252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.760953+0000","last_change":"2024-05-28T14:14:43.757686+0000","last_active":"2024-05-28T14:14:43.760953+0000","last_peered":"2024-05-28T14:14:43.760953+0000","last_clean":"2024-05-28T14:14:43.760953+0000","last_became_active":"2024-05-28T14:14:43.757288+0000","last_became_peered":"2024-05-28T14:14:43.757288+0000","last_unstale":"2024-05-28T14:14:43.760953+0000","last_undegraded":"2024-05-28T14:14:43.760953+0000","last_fullsized":"2024-05-28T14:14:43.760953+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:05:56.557590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.764372+0000","last_change":"2024-05-28T14:14:43.759851+0000","last_active":"2024-05-28T14:14:43.764372+0000","last_peered":"2024-05-28T14:14:43.764372+0000","last_clean":"2024-05-28T14:14:43.764372+0000","last_became_active":"2024-05-28T14:14:43.759580+0000","last_became_peered":"2024-05-28T14:14:43.759580+0000","last_unstale":"2024-05-28T14:14:43.764372+0000","last_undegraded":"2024-05-28T14:14:43.764372+0000","last_fullsized":"2024-05-28T14:14:43.764372+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:57:52.097273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.774137+0000","last_change":"2024-05-28T14:14:43.768741+0000","last_active":"2024-05-28T14:14:43.774137+0000","last_peered":"2024-05-28T14:14:43.774137+0000","last_clean":"2024-05-28T14:14:43.774137+0000","last_became_active":"2024-05-28T14:14:43.767741+0000","last_became_peered":"2024-05-28T14:14:43.767741+0000","last_unstale":"2024-05-28T14:14:43.774137+0000","last_undegraded":"2024-05-28T14:14:43.774137+0000","last_fullsized":"2024-05-28T14:14:43.774137+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:22:10.225982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.763103+0000","last_change":"2024-05-28T14:14:43.755177+0000","last_active":"2024-05-28T14:14:43.763103+0000","last_peered":"2024-05-28T14:14:43.763103+0000","last_clean":"2024-05-28T14:14:43.763103+0000","last_became_active":"2024-05-28T14:14:43.753794+0000","last_became_peered":"2024-05-28T14:14:43.753794+0000","last_unstale":"2024-05-28T14:14:43.763103+0000","last_undegraded":"2024-05-28T14:14:43.763103+0000","last_fullsized":"2024-05-28T14:14:43.763103+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:08.045261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766911+0000","last_change":"2024-05-28T14:14:43.763725+0000","last_active":"2024-05-28T14:14:43.766911+0000","last_peered":"2024-05-28T14:14:43.766911+0000","last_clean":"2024-05-28T14:14:43.766911+0000","last_became_active":"2024-05-28T14:14:43.763332+0000","last_became_peered":"2024-05-28T14:14:43.763332+0000","last_unstale":"2024-05-28T14:14:43.766911+0000","last_undegraded":"2024-05-28T14:14:43.766911+0000","last_fullsized":"2024-05-28T14:14:43.766911+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:14.042730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766239+0000","last_change":"2024-05-28T14:14:43.759381+0000","last_active":"2024-05-28T14:14:43.766239+0000","last_peered":"2024-05-28T14:14:43.766239+0000","last_clean":"2024-05-28T14:14:43.766239+0000","last_became_active":"2024-05-28T14:14:43.758722+0000","last_became_peered":"2024-05-28T14:14:43.758722+0000","last_unstale":"2024-05-28T14:14:43.766239+0000","last_undegraded":"2024-05-28T14:14:43.766239+0000","last_fullsized":"2024-05-28T14:14:43.766239+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:57:19.311096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.769538+0000","last_change":"2024-05-28T14:14:43.768220+0000","last_active":"2024-05-28T14:14:43.769538+0000","last_peered":"2024-05-28T14:14:43.769538+0000","last_clean":"2024-05-28T14:14:43.769538+0000","last_became_active":"2024-05-28T14:14:43.767925+0000","last_became_peered":"2024-05-28T14:14:43.767925+0000","last_unstale":"2024-05-28T14:14:43.769538+0000","last_undegraded":"2024-05-28T14:14:43.769538+0000","last_fullsized":"2024-05-28T14:14:43.769538+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:39.285543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.750353+0000","last_change":"2024-05-28T14:14:43.746536+0000","last_active":"2024-05-28T14:14:43.750353+0000","last_peered":"2024-05-28T14:14:43.750353+0000","last_clean":"2024-05-28T14:14:43.750353+0000","last_became_active":"2024-05-28T14:14:43.746249+0000","last_became_peered":"2024-05-28T14:14:43.746249+0000","last_unstale":"2024-05-28T14:14:43.750353+0000","last_undegraded":"2024-05-28T14:14:43.750353+0000","last_fullsized":"2024-05-28T14:14:43.750353+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:27:11.131902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.767601+0000","last_change":"2024-05-28T14:14:43.761643+0000","last_active":"2024-05-28T14:14:43.767601+0000","last_peered":"2024-05-28T14:14:43.767601+0000","last_clean":"2024-05-28T14:14:43.767601+0000","last_became_active":"2024-05-28T14:14:43.761161+0000","last_became_peered":"2024-05-28T14:14:43.761161+0000","last_unstale":"2024-05-28T14:14:43.767601+0000","last_undegraded":"2024-05-28T14:14:43.767601+0000","last_fullsized":"2024-05-28T14:14:43.767601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:36:41.091057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.767561+0000","last_change":"2024-05-28T14:14:43.764133+0000","last_active":"2024-05-28T14:14:43.767561+0000","last_peered":"2024-05-28T14:14:43.767561+0000","last_clean":"2024-05-28T14:14:43.767561+0000","last_became_active":"2024-05-28T14:14:43.763522+0000","last_became_peered":"2024-05-28T14:14:43.763522+0000","last_unstale":"2024-05-28T14:14:43.767561+0000","last_undegraded":"2024-05-28T14:14:43.767561+0000","last_fullsized":"2024-05-28T14:14:43.767561+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:03:35.593768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.769727+0000","last_change":"2024-05-28T14:14:43.763447+0000","last_active":"2024-05-28T14:14:43.769727+0000","last_peered":"2024-05-28T14:14:43.769727+0000","last_clean":"2024-05-28T14:14:43.769727+0000","last_became_active":"2024-05-28T14:14:43.763142+0000","last_became_peered":"2024-05-28T14:14:43.763142+0000","last_unstale":"2024-05-28T14:14:43.769727+0000","last_undegraded":"2024-05-28T14:14:43.769727+0000","last_fullsized":"2024-05-28T14:14:43.769727+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:27:22.008435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.754670+0000","last_change":"2024-05-28T14:14:43.747637+0000","last_active":"2024-05-28T14:14:44.754670+0000","last_peered":"2024-05-28T14:14:44.754670+0000","last_clean":"2024-05-28T14:14:44.754670+0000","last_became_active":"2024-05-28T14:14:43.747272+0000","last_became_peered":"2024-05-28T14:14:43.747272+0000","last_unstale":"2024-05-28T14:14:44.754670+0000","last_undegraded":"2024-05-28T14:14:44.754670+0000","last_fullsized":"2024-05-28T14:14:44.754670+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:19.860056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766264+0000","last_change":"2024-05-28T14:14:43.759397+0000","last_active":"2024-05-28T14:14:43.766264+0000","last_peered":"2024-05-28T14:14:43.766264+0000","last_clean":"2024-05-28T14:14:43.766264+0000","last_became_active":"2024-05-28T14:14:43.759136+0000","last_became_peered":"2024-05-28T14:14:43.759136+0000","last_unstale":"2024-05-28T14:14:43.766264+0000","last_undegraded":"2024-05-28T14:14:43.766264+0000","last_fullsized":"2024-05-28T14:14:43.766264+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:48.608539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"131'4","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.775255+0000","last_change":"2024-05-28T14:14:42.751872+0000","last_active":"2024-05-28T14:14:43.775255+0000","last_peered":"2024-05-28T14:14:43.775255+0000","last_clean":"2024-05-28T14:14:43.775255+0000","last_became_active":"2024-05-28T14:14:42.751450+0000","last_became_peered":"2024-05-28T14:14:42.751450+0000","last_unstale":"2024-05-28T14:14:43.775255+0000","last_undegraded":"2024-05-28T14:14:43.775255+0000","last_fullsized":"2024-05-28T14:14:43.775255+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:48:49.635601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.772858+0000","last_change":"2024-05-28T14:14:43.766763+0000","last_active":"2024-05-28T14:14:43.772858+0000","last_peered":"2024-05-28T14:14:43.772858+0000","last_clean":"2024-05-28T14:14:43.772858+0000","last_became_active":"2024-05-28T14:14:43.766347+0000","last_became_peered":"2024-05-28T14:14:43.766347+0000","last_unstale":"2024-05-28T14:14:43.772858+0000","last_undegraded":"2024-05-28T14:14:43.772858+0000","last_fullsized":"2024-05-28T14:14:43.772858+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:02:15.815971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.781328+0000","last_change":"2024-05-28T14:14:42.742952+0000","last_active":"2024-05-28T14:14:43.781328+0000","last_peered":"2024-05-28T14:14:43.781328+0000","last_clean":"2024-05-28T14:14:43.781328+0000","last_became_active":"2024-05-28T14:14:42.742668+0000","last_became_peered":"2024-05-28T14:14:42.742668+0000","last_unstale":"2024-05-28T14:14:43.781328+0000","last_undegraded":"2024-05-28T14:14:43.781328+0000","last_fullsized":"2024-05-28T14:14:43.781328+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:53:04.792926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.773426+0000","last_change":"2024-05-28T14:14:43.765567+0000","last_active":"2024-05-28T14:14:43.773426+0000","last_peered":"2024-05-28T14:14:43.773426+0000","last_clean":"2024-05-28T14:14:43.773426+0000","last_became_active":"2024-05-28T14:14:43.765230+0000","last_became_peered":"2024-05-28T14:14:43.765230+0000","last_unstale":"2024-05-28T14:14:43.773426+0000","last_undegraded":"2024-05-28T14:14:43.773426+0000","last_fullsized":"2024-05-28T14:14:43.773426+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:57:52.097273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.773607+0000","last_change":"2024-05-28T14:14:42.733510+0000","last_active":"2024-05-28T14:14:43.773607+0000","last_peered":"2024-05-28T14:14:43.773607+0000","last_clean":"2024-05-28T14:14:43.773607+0000","last_became_active":"2024-05-28T14:14:42.733231+0000","last_became_peered":"2024-05-28T14:14:42.733231+0000","last_unstale":"2024-05-28T14:14:43.773607+0000","last_undegraded":"2024-05-28T14:14:43.773607+0000","last_fullsized":"2024-05-28T14:14:43.773607+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:37.532653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.762651+0000","last_change":"2024-05-28T14:14:43.760573+0000","last_active":"2024-05-28T14:14:43.762651+0000","last_peered":"2024-05-28T14:14:43.762651+0000","last_clean":"2024-05-28T14:14:43.762651+0000","last_became_active":"2024-05-28T14:14:43.759307+0000","last_became_peered":"2024-05-28T14:14:43.759307+0000","last_unstale":"2024-05-28T14:14:43.762651+0000","last_undegraded":"2024-05-28T14:14:43.762651+0000","last_fullsized":"2024-05-28T14:14:43.762651+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:26.042780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.776144+0000","last_change":"2024-05-28T14:14:42.745411+0000","last_active":"2024-05-28T14:14:43.776144+0000","last_peered":"2024-05-28T14:14:43.776144+0000","last_clean":"2024-05-28T14:14:43.776144+0000","last_became_active":"2024-05-28T14:14:42.745097+0000","last_became_peered":"2024-05-28T14:14:42.745097+0000","last_unstale":"2024-05-28T14:14:43.776144+0000","last_undegraded":"2024-05-28T14:14:43.776144+0000","last_fullsized":"2024-05-28T14:14:43.776144+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:39:18.579687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.756326+0000","last_change":"2024-05-28T14:14:43.751125+0000","last_active":"2024-05-28T14:14:43.756326+0000","last_peered":"2024-05-28T14:14:43.756326+0000","last_clean":"2024-05-28T14:14:43.756326+0000","last_became_active":"2024-05-28T14:14:43.747282+0000","last_became_peered":"2024-05-28T14:14:43.747282+0000","last_unstale":"2024-05-28T14:14:43.756326+0000","last_undegraded":"2024-05-28T14:14:43.756326+0000","last_fullsized":"2024-05-28T14:14:43.756326+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:15:26.978033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.775945+0000","last_change":"2024-05-28T14:14:42.741716+0000","last_active":"2024-05-28T14:14:43.775945+0000","last_peered":"2024-05-28T14:14:43.775945+0000","last_clean":"2024-05-28T14:14:43.775945+0000","last_became_active":"2024-05-28T14:14:42.741425+0000","last_became_peered":"2024-05-28T14:14:42.741425+0000","last_unstale":"2024-05-28T14:14:43.775945+0000","last_undegraded":"2024-05-28T14:14:43.775945+0000","last_fullsized":"2024-05-28T14:14:43.775945+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:36:40.085174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.755183+0000","last_change":"2024-05-28T14:14:43.760960+0000","last_active":"2024-05-28T14:14:44.755183+0000","last_peered":"2024-05-28T14:14:44.755183+0000","last_clean":"2024-05-28T14:14:44.755183+0000","last_became_active":"2024-05-28T14:14:43.760691+0000","last_became_peered":"2024-05-28T14:14:43.760691+0000","last_unstale":"2024-05-28T14:14:44.755183+0000","last_undegraded":"2024-05-28T14:14:44.755183+0000","last_fullsized":"2024-05-28T14:14:44.755183+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:33:40.268549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.775716+0000","last_change":"2024-05-28T14:14:42.739561+0000","last_active":"2024-05-28T14:14:43.775716+0000","last_peered":"2024-05-28T14:14:43.775716+0000","last_clean":"2024-05-28T14:14:43.775716+0000","last_became_active":"2024-05-28T14:14:42.739191+0000","last_became_peered":"2024-05-28T14:14:42.739191+0000","last_unstale":"2024-05-28T14:14:43.775716+0000","last_undegraded":"2024-05-28T14:14:43.775716+0000","last_fullsized":"2024-05-28T14:14:43.775716+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:49:44.727202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766518+0000","last_change":"2024-05-28T14:14:43.762652+0000","last_active":"2024-05-28T14:14:43.766518+0000","last_peered":"2024-05-28T14:14:43.766518+0000","last_clean":"2024-05-28T14:14:43.766518+0000","last_became_active":"2024-05-28T14:14:43.761547+0000","last_became_peered":"2024-05-28T14:14:43.761547+0000","last_unstale":"2024-05-28T14:14:43.766518+0000","last_undegraded":"2024-05-28T14:14:43.766518+0000","last_fullsized":"2024-05-28T14:14:43.766518+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:14:37.836936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.754796+0000","last_change":"2024-05-28T14:14:42.745986+0000","last_active":"2024-05-28T14:14:44.754796+0000","last_peered":"2024-05-28T14:14:44.754796+0000","last_clean":"2024-05-28T14:14:44.754796+0000","last_became_active":"2024-05-28T14:14:42.745691+0000","last_became_peered":"2024-05-28T14:14:42.745691+0000","last_unstale":"2024-05-28T14:14:44.754796+0000","last_undegraded":"2024-05-28T14:14:44.754796+0000","last_fullsized":"2024-05-28T14:14:44.754796+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:22:09.220099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.767406+0000","last_change":"2024-05-28T14:14:43.764538+0000","last_active":"2024-05-28T14:14:43.767406+0000","last_peered":"2024-05-28T14:14:43.767406+0000","last_clean":"2024-05-28T14:14:43.767406+0000","last_became_active":"2024-05-28T14:14:43.762358+0000","last_became_peered":"2024-05-28T14:14:43.762358+0000","last_unstale":"2024-05-28T14:14:43.767406+0000","last_undegraded":"2024-05-28T14:14:43.767406+0000","last_fullsized":"2024-05-28T14:14:43.767406+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:27:11.131902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.751056+0000","last_change":"2024-05-28T14:14:42.747379+0000","last_active":"2024-05-28T14:14:43.751056+0000","last_peered":"2024-05-28T14:14:43.751056+0000","last_clean":"2024-05-28T14:14:43.751056+0000","last_became_active":"2024-05-28T14:14:42.747127+0000","last_became_peered":"2024-05-28T14:14:42.747127+0000","last_unstale":"2024-05-28T14:14:43.751056+0000","last_undegraded":"2024-05-28T14:14:43.751056+0000","last_fullsized":"2024-05-28T14:14:43.751056+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:10:35.041686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.765555+0000","last_change":"2024-05-28T14:14:43.757145+0000","last_active":"2024-05-28T14:14:43.765555+0000","last_peered":"2024-05-28T14:14:43.765555+0000","last_clean":"2024-05-28T14:14:43.765555+0000","last_became_active":"2024-05-28T14:14:43.756803+0000","last_became_peered":"2024-05-28T14:14:43.756803+0000","last_unstale":"2024-05-28T14:14:43.765555+0000","last_undegraded":"2024-05-28T14:14:43.765555+0000","last_fullsized":"2024-05-28T14:14:43.765555+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:29:27.674815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.743163+0000","last_change":"2024-05-28T14:14:42.746521+0000","last_active":"2024-05-28T14:14:43.743163+0000","last_peered":"2024-05-28T14:14:43.743163+0000","last_clean":"2024-05-28T14:14:43.743163+0000","last_became_active":"2024-05-28T14:14:42.746243+0000","last_became_peered":"2024-05-28T14:14:42.746243+0000","last_unstale":"2024-05-28T14:14:43.743163+0000","last_undegraded":"2024-05-28T14:14:43.743163+0000","last_fullsized":"2024-05-28T14:14:43.743163+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:44.045231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.754921+0000","last_change":"2024-05-28T14:14:43.751232+0000","last_active":"2024-05-28T14:14:43.754921+0000","last_peered":"2024-05-28T14:14:43.754921+0000","last_clean":"2024-05-28T14:14:43.754921+0000","last_became_active":"2024-05-28T14:14:43.750768+0000","last_became_peered":"2024-05-28T14:14:43.750768+0000","last_unstale":"2024-05-28T14:14:43.754921+0000","last_undegraded":"2024-05-28T14:14:43.754921+0000","last_fullsized":"2024-05-28T14:14:43.754921+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:14.494237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.741873+0000","last_change":"2024-05-28T14:14:42.745263+0000","last_active":"2024-05-28T14:14:43.741873+0000","last_peered":"2024-05-28T14:14:43.741873+0000","last_clean":"2024-05-28T14:14:43.741873+0000","last_became_active":"2024-05-28T14:14:42.744565+0000","last_became_peered":"2024-05-28T14:14:42.744565+0000","last_unstale":"2024-05-28T14:14:43.741873+0000","last_undegraded":"2024-05-28T14:14:43.741873+0000","last_fullsized":"2024-05-28T14:14:43.741873+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:59.384784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.763407+0000","last_change":"2024-05-28T14:14:43.760442+0000","last_active":"2024-05-28T14:14:43.763407+0000","last_peered":"2024-05-28T14:14:43.763407+0000","last_clean":"2024-05-28T14:14:43.763407+0000","last_became_active":"2024-05-28T14:14:43.760106+0000","last_became_peered":"2024-05-28T14:14:43.760106+0000","last_unstale":"2024-05-28T14:14:43.763407+0000","last_undegraded":"2024-05-28T14:14:43.763407+0000","last_fullsized":"2024-05-28T14:14:43.763407+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:38.538536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"131'3","reported_seq":15,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.781593+0000","last_change":"2024-05-28T14:14:42.735627+0000","last_active":"2024-05-28T14:14:43.781593+0000","last_peered":"2024-05-28T14:14:43.781593+0000","last_clean":"2024-05-28T14:14:43.781593+0000","last_became_active":"2024-05-28T14:14:42.734095+0000","last_became_peered":"2024-05-28T14:14:42.734095+0000","last_unstale":"2024-05-28T14:14:43.781593+0000","last_undegraded":"2024-05-28T14:14:43.781593+0000","last_fullsized":"2024-05-28T14:14:43.781593+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:07.310371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.774577+0000","last_change":"2024-05-28T14:14:43.768144+0000","last_active":"2024-05-28T14:14:43.774577+0000","last_peered":"2024-05-28T14:14:43.774577+0000","last_clean":"2024-05-28T14:14:43.774577+0000","last_became_active":"2024-05-28T14:14:43.766334+0000","last_became_peered":"2024-05-28T14:14:43.766334+0000","last_unstale":"2024-05-28T14:14:43.774577+0000","last_undegraded":"2024-05-28T14:14:43.774577+0000","last_fullsized":"2024-05-28T14:14:43.774577+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:50:20.486680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"131'6","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.783642+0000","last_change":"2024-05-28T14:14:42.744568+0000","last_active":"2024-05-28T14:14:43.783642+0000","last_peered":"2024-05-28T14:14:43.783642+0000","last_clean":"2024-05-28T14:14:43.783642+0000","last_became_active":"2024-05-28T14:14:42.744240+0000","last_became_peered":"2024-05-28T14:14:42.744240+0000","last_unstale":"2024-05-28T14:14:43.783642+0000","last_undegraded":"2024-05-28T14:14:43.783642+0000","last_fullsized":"2024-05-28T14:14:43.783642+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:25:02.568973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.767579+0000","last_change":"2024-05-28T14:14:43.758776+0000","last_active":"2024-05-28T14:14:43.767579+0000","last_peered":"2024-05-28T14:14:43.767579+0000","last_clean":"2024-05-28T14:14:43.767579+0000","last_became_active":"2024-05-28T14:14:43.758474+0000","last_became_peered":"2024-05-28T14:14:43.758474+0000","last_unstale":"2024-05-28T14:14:43.767579+0000","last_undegraded":"2024-05-28T14:14:43.767579+0000","last_fullsized":"2024-05-28T14:14:43.767579+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:14:41.836526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.747855+0000","last_change":"2024-05-28T14:14:42.737773+0000","last_active":"2024-05-28T14:14:43.747855+0000","last_peered":"2024-05-28T14:14:43.747855+0000","last_clean":"2024-05-28T14:14:43.747855+0000","last_became_active":"2024-05-28T14:14:42.737442+0000","last_became_peered":"2024-05-28T14:14:42.737442+0000","last_unstale":"2024-05-28T14:14:43.747855+0000","last_undegraded":"2024-05-28T14:14:43.747855+0000","last_fullsized":"2024-05-28T14:14:43.747855+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:45:49.229094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766947+0000","last_change":"2024-05-28T14:14:43.762668+0000","last_active":"2024-05-28T14:14:43.766947+0000","last_peered":"2024-05-28T14:14:43.766947+0000","last_clean":"2024-05-28T14:14:43.766947+0000","last_became_active":"2024-05-28T14:14:43.762049+0000","last_became_peered":"2024-05-28T14:14:43.762049+0000","last_unstale":"2024-05-28T14:14:43.766947+0000","last_undegraded":"2024-05-28T14:14:43.766947+0000","last_fullsized":"2024-05-28T14:14:43.766947+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:53:05.798809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.751344+0000","last_change":"2024-05-28T14:14:42.746077+0000","last_active":"2024-05-28T14:14:43.751344+0000","last_peered":"2024-05-28T14:14:43.751344+0000","last_clean":"2024-05-28T14:14:43.751344+0000","last_became_active":"2024-05-28T14:14:42.745811+0000","last_became_peered":"2024-05-28T14:14:42.745811+0000","last_unstale":"2024-05-28T14:14:43.751344+0000","last_undegraded":"2024-05-28T14:14:43.751344+0000","last_fullsized":"2024-05-28T14:14:43.751344+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:26:07.187114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.773086+0000","last_change":"2024-05-28T14:14:43.768732+0000","last_active":"2024-05-28T14:14:43.773086+0000","last_peered":"2024-05-28T14:14:43.773086+0000","last_clean":"2024-05-28T14:14:43.773086+0000","last_became_active":"2024-05-28T14:14:43.768334+0000","last_became_peered":"2024-05-28T14:14:43.768334+0000","last_unstale":"2024-05-28T14:14:43.773086+0000","last_undegraded":"2024-05-28T14:14:43.773086+0000","last_fullsized":"2024-05-28T14:14:43.773086+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:10:44.749184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"131'8","reported_seq":20,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.781998+0000","last_change":"2024-05-28T14:14:42.742861+0000","last_active":"2024-05-28T14:14:43.781998+0000","last_peered":"2024-05-28T14:14:43.781998+0000","last_clean":"2024-05-28T14:14:43.781998+0000","last_became_active":"2024-05-28T14:14:42.742518+0000","last_became_peered":"2024-05-28T14:14:42.742518+0000","last_unstale":"2024-05-28T14:14:43.781998+0000","last_undegraded":"2024-05-28T14:14:43.781998+0000","last_fullsized":"2024-05-28T14:14:43.781998+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:43:29.312316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766344+0000","last_change":"2024-05-28T14:14:43.762081+0000","last_active":"2024-05-28T14:14:43.766344+0000","last_peered":"2024-05-28T14:14:43.766344+0000","last_clean":"2024-05-28T14:14:43.766344+0000","last_became_active":"2024-05-28T14:14:43.761775+0000","last_became_peered":"2024-05-28T14:14:43.761775+0000","last_unstale":"2024-05-28T14:14:43.766344+0000","last_undegraded":"2024-05-28T14:14:43.766344+0000","last_fullsized":"2024-05-28T14:14:43.766344+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:10.253064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"131'1","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.782601+0000","last_change":"2024-05-28T14:14:42.746381+0000","last_active":"2024-05-28T14:14:43.782601+0000","last_peered":"2024-05-28T14:14:43.782601+0000","last_clean":"2024-05-28T14:14:43.782601+0000","last_became_active":"2024-05-28T14:14:42.746098+0000","last_became_peered":"2024-05-28T14:14:42.746098+0000","last_unstale":"2024-05-28T14:14:43.782601+0000","last_undegraded":"2024-05-28T14:14:43.782601+0000","last_fullsized":"2024-05-28T14:14:43.782601+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:04.859086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.768737+0000","last_change":"2024-05-28T14:14:43.756453+0000","last_active":"2024-05-28T14:14:43.768737+0000","last_peered":"2024-05-28T14:14:43.768737+0000","last_clean":"2024-05-28T14:14:43.768737+0000","last_became_active":"2024-05-28T14:14:43.756138+0000","last_became_peered":"2024-05-28T14:14:43.756138+0000","last_unstale":"2024-05-28T14:14:43.768737+0000","last_undegraded":"2024-05-28T14:14:43.768737+0000","last_fullsized":"2024-05-28T14:14:43.768737+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:11.879163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.778751+0000","last_change":"2024-05-28T14:14:42.742055+0000","last_active":"2024-05-28T14:14:43.778751+0000","last_peered":"2024-05-28T14:14:43.778751+0000","last_clean":"2024-05-28T14:14:43.778751+0000","last_became_active":"2024-05-28T14:14:42.741754+0000","last_became_peered":"2024-05-28T14:14:42.741754+0000","last_unstale":"2024-05-28T14:14:43.778751+0000","last_undegraded":"2024-05-28T14:14:43.778751+0000","last_fullsized":"2024-05-28T14:14:43.778751+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:25.036897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.765359+0000","last_change":"2024-05-28T14:14:43.758320+0000","last_active":"2024-05-28T14:14:43.765359+0000","last_peered":"2024-05-28T14:14:43.765359+0000","last_clean":"2024-05-28T14:14:43.765359+0000","last_became_active":"2024-05-28T14:14:43.757996+0000","last_became_peered":"2024-05-28T14:14:43.757996+0000","last_unstale":"2024-05-28T14:14:43.765359+0000","last_undegraded":"2024-05-28T14:14:43.765359+0000","last_fullsized":"2024-05-28T14:14:43.765359+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:11:31.534150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.755297+0000","last_change":"2024-05-28T14:14:42.734922+0000","last_active":"2024-05-28T14:14:44.755297+0000","last_peered":"2024-05-28T14:14:44.755297+0000","last_clean":"2024-05-28T14:14:44.755297+0000","last_became_active":"2024-05-28T14:14:42.734647+0000","last_became_peered":"2024-05-28T14:14:42.734647+0000","last_unstale":"2024-05-28T14:14:44.755297+0000","last_undegraded":"2024-05-28T14:14:44.755297+0000","last_fullsized":"2024-05-28T14:14:44.755297+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:36:34.326176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766994+0000","last_change":"2024-05-28T14:14:43.764186+0000","last_active":"2024-05-28T14:14:43.766994+0000","last_peered":"2024-05-28T14:14:43.766994+0000","last_clean":"2024-05-28T14:14:43.766994+0000","last_became_active":"2024-05-28T14:14:43.762407+0000","last_became_peered":"2024-05-28T14:14:43.762407+0000","last_unstale":"2024-05-28T14:14:43.766994+0000","last_undegraded":"2024-05-28T14:14:43.766994+0000","last_fullsized":"2024-05-28T14:14:43.766994+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:45.051114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.742553+0000","last_change":"2024-05-28T14:14:42.733737+0000","last_active":"2024-05-28T14:14:43.742553+0000","last_peered":"2024-05-28T14:14:43.742553+0000","last_clean":"2024-05-28T14:14:43.742553+0000","last_became_active":"2024-05-28T14:14:42.733393+0000","last_became_peered":"2024-05-28T14:14:42.733393+0000","last_unstale":"2024-05-28T14:14:43.742553+0000","last_undegraded":"2024-05-28T14:14:43.742553+0000","last_fullsized":"2024-05-28T14:14:43.742553+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:13.036848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.768905+0000","last_change":"2024-05-28T14:14:43.767748+0000","last_active":"2024-05-28T14:14:43.768905+0000","last_peered":"2024-05-28T14:14:43.768905+0000","last_clean":"2024-05-28T14:14:43.768905+0000","last_became_active":"2024-05-28T14:14:43.767351+0000","last_became_peered":"2024-05-28T14:14:43.767351+0000","last_unstale":"2024-05-28T14:14:43.768905+0000","last_undegraded":"2024-05-28T14:14:43.768905+0000","last_fullsized":"2024-05-28T14:14:43.768905+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:53:05.798809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.782102+0000","last_change":"2024-05-28T14:14:42.746787+0000","last_active":"2024-05-28T14:14:43.782102+0000","last_peered":"2024-05-28T14:14:43.782102+0000","last_clean":"2024-05-28T14:14:43.782102+0000","last_became_active":"2024-05-28T14:14:42.746273+0000","last_became_peered":"2024-05-28T14:14:42.746273+0000","last_unstale":"2024-05-28T14:14:43.782102+0000","last_undegraded":"2024-05-28T14:14:43.782102+0000","last_fullsized":"2024-05-28T14:14:43.782102+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:38.279660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.755367+0000","last_change":"2024-05-28T14:14:43.771048+0000","last_active":"2024-05-28T14:14:44.755367+0000","last_peered":"2024-05-28T14:14:44.755367+0000","last_clean":"2024-05-28T14:14:44.755367+0000","last_became_active":"2024-05-28T14:14:43.770751+0000","last_became_peered":"2024-05-28T14:14:43.770751+0000","last_unstale":"2024-05-28T14:14:44.755367+0000","last_undegraded":"2024-05-28T14:14:44.755367+0000","last_fullsized":"2024-05-28T14:14:44.755367+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:34:09.617785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.740542+0000","last_change":"2024-05-28T14:14:42.742397+0000","last_active":"2024-05-28T14:14:43.740542+0000","last_peered":"2024-05-28T14:14:43.740542+0000","last_clean":"2024-05-28T14:14:43.740542+0000","last_became_active":"2024-05-28T14:14:42.742104+0000","last_became_peered":"2024-05-28T14:14:42.742104+0000","last_unstale":"2024-05-28T14:14:43.740542+0000","last_undegraded":"2024-05-28T14:14:43.740542+0000","last_fullsized":"2024-05-28T14:14:43.740542+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:21.703678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.758509+0000","last_change":"2024-05-28T14:14:43.756230+0000","last_active":"2024-05-28T14:14:43.758509+0000","last_peered":"2024-05-28T14:14:43.758509+0000","last_clean":"2024-05-28T14:14:43.758509+0000","last_became_active":"2024-05-28T14:14:43.755945+0000","last_became_peered":"2024-05-28T14:14:43.755945+0000","last_unstale":"2024-05-28T14:14:43.758509+0000","last_undegraded":"2024-05-28T14:14:43.758509+0000","last_fullsized":"2024-05-28T14:14:43.758509+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:08.316254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.739981+0000","last_change":"2024-05-28T14:14:42.737496+0000","last_active":"2024-05-28T14:14:43.739981+0000","last_peered":"2024-05-28T14:14:43.739981+0000","last_clean":"2024-05-28T14:14:43.739981+0000","last_became_active":"2024-05-28T14:14:42.737194+0000","last_became_peered":"2024-05-28T14:14:42.737194+0000","last_unstale":"2024-05-28T14:14:43.739981+0000","last_undegraded":"2024-05-28T14:14:43.739981+0000","last_fullsized":"2024-05-28T14:14:43.739981+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:56:10.665924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.760641+0000","last_change":"2024-05-28T14:14:43.758200+0000","last_active":"2024-05-28T14:14:43.760641+0000","last_peered":"2024-05-28T14:14:43.760641+0000","last_clean":"2024-05-28T14:14:43.760641+0000","last_became_active":"2024-05-28T14:14:43.757875+0000","last_became_peered":"2024-05-28T14:14:43.757875+0000","last_unstale":"2024-05-28T14:14:43.760641+0000","last_undegraded":"2024-05-28T14:14:43.760641+0000","last_fullsized":"2024-05-28T14:14:43.760641+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:48:30.643597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.745077+0000","last_change":"2024-05-28T14:14:42.738809+0000","last_active":"2024-05-28T14:14:43.745077+0000","last_peered":"2024-05-28T14:14:43.745077+0000","last_clean":"2024-05-28T14:14:43.745077+0000","last_became_active":"2024-05-28T14:14:42.738494+0000","last_became_peered":"2024-05-28T14:14:42.738494+0000","last_unstale":"2024-05-28T14:14:43.745077+0000","last_undegraded":"2024-05-28T14:14:43.745077+0000","last_fullsized":"2024-05-28T14:14:43.745077+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:17.145363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.755630+0000","last_change":"2024-05-28T14:14:43.770616+0000","last_active":"2024-05-28T14:14:44.755630+0000","last_peered":"2024-05-28T14:14:44.755630+0000","last_clean":"2024-05-28T14:14:44.755630+0000","last_became_active":"2024-05-28T14:14:43.770253+0000","last_became_peered":"2024-05-28T14:14:43.770253+0000","last_unstale":"2024-05-28T14:14:44.755630+0000","last_undegraded":"2024-05-28T14:14:44.755630+0000","last_fullsized":"2024-05-28T14:14:44.755630+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:49:45.733085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.776273+0000","last_change":"2024-05-28T14:14:42.746355+0000","last_active":"2024-05-28T14:14:43.776273+0000","last_peered":"2024-05-28T14:14:43.776273+0000","last_clean":"2024-05-28T14:14:43.776273+0000","last_became_active":"2024-05-28T14:14:42.745926+0000","last_became_peered":"2024-05-28T14:14:42.745926+0000","last_unstale":"2024-05-28T14:14:43.776273+0000","last_undegraded":"2024-05-28T14:14:43.776273+0000","last_fullsized":"2024-05-28T14:14:43.776273+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:54:47.273264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.765619+0000","last_change":"2024-05-28T14:14:43.763195+0000","last_active":"2024-05-28T14:14:43.765619+0000","last_peered":"2024-05-28T14:14:43.765619+0000","last_clean":"2024-05-28T14:14:43.765619+0000","last_became_active":"2024-05-28T14:14:43.761832+0000","last_became_peered":"2024-05-28T14:14:43.761832+0000","last_unstale":"2024-05-28T14:14:43.765619+0000","last_undegraded":"2024-05-28T14:14:43.765619+0000","last_fullsized":"2024-05-28T14:14:43.765619+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:51:28.086771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.752284+0000","last_change":"2024-05-28T14:14:42.749733+0000","last_active":"2024-05-28T14:14:43.752284+0000","last_peered":"2024-05-28T14:14:43.752284+0000","last_clean":"2024-05-28T14:14:43.752284+0000","last_became_active":"2024-05-28T14:14:42.749348+0000","last_became_peered":"2024-05-28T14:14:42.749348+0000","last_unstale":"2024-05-28T14:14:43.752284+0000","last_undegraded":"2024-05-28T14:14:43.752284+0000","last_fullsized":"2024-05-28T14:14:43.752284+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:17.145363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.767216+0000","last_change":"2024-05-28T14:14:43.764008+0000","last_active":"2024-05-28T14:14:43.767216+0000","last_peered":"2024-05-28T14:14:43.767216+0000","last_clean":"2024-05-28T14:14:43.767216+0000","last_became_active":"2024-05-28T14:14:43.762648+0000","last_became_peered":"2024-05-28T14:14:43.762648+0000","last_unstale":"2024-05-28T14:14:43.767216+0000","last_undegraded":"2024-05-28T14:14:43.767216+0000","last_fullsized":"2024-05-28T14:14:43.767216+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:05.864969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.751619+0000","last_change":"2024-05-28T14:14:42.744189+0000","last_active":"2024-05-28T14:14:43.751619+0000","last_peered":"2024-05-28T14:14:43.751619+0000","last_clean":"2024-05-28T14:14:43.751619+0000","last_became_active":"2024-05-28T14:14:42.743858+0000","last_became_peered":"2024-05-28T14:14:42.743858+0000","last_unstale":"2024-05-28T14:14:43.751619+0000","last_undegraded":"2024-05-28T14:14:43.751619+0000","last_fullsized":"2024-05-28T14:14:43.751619+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:03:34.587886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-28T14:14:44.755152+0000","last_change":"2024-05-28T14:14:43.758779+0000","last_active":"2024-05-28T14:14:44.755152+0000","last_peered":"2024-05-28T14:14:44.755152+0000","last_clean":"2024-05-28T14:14:44.755152+0000","last_became_active":"2024-05-28T14:14:43.758516+0000","last_became_peered":"2024-05-28T14:14:43.758516+0000","last_unstale":"2024-05-28T14:14:44.755152+0000","last_undegraded":"2024-05-28T14:14:44.755152+0000","last_fullsized":"2024-05-28T14:14:44.755152+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:13:21.112865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.745589+0000","last_change":"2024-05-28T14:14:42.748521+0000","last_active":"2024-05-28T14:14:43.745589+0000","last_peered":"2024-05-28T14:14:43.745589+0000","last_clean":"2024-05-28T14:14:43.745589+0000","last_became_active":"2024-05-28T14:14:42.748193+0000","last_became_peered":"2024-05-28T14:14:42.748193+0000","last_unstale":"2024-05-28T14:14:43.745589+0000","last_undegraded":"2024-05-28T14:14:43.745589+0000","last_fullsized":"2024-05-28T14:14:43.745589+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:48:29.637714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.774765+0000","last_change":"2024-05-28T14:14:43.767848+0000","last_active":"2024-05-28T14:14:43.774765+0000","last_peered":"2024-05-28T14:14:43.774765+0000","last_clean":"2024-05-28T14:14:43.774765+0000","last_became_active":"2024-05-28T14:14:43.767552+0000","last_became_peered":"2024-05-28T14:14:43.767552+0000","last_unstale":"2024-05-28T14:14:43.774765+0000","last_undegraded":"2024-05-28T14:14:43.774765+0000","last_fullsized":"2024-05-28T14:14:43.774765+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:56:28.627044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.751566+0000","last_change":"2024-05-28T14:14:42.746605+0000","last_active":"2024-05-28T14:14:43.751566+0000","last_peered":"2024-05-28T14:14:43.751566+0000","last_clean":"2024-05-28T14:14:43.751566+0000","last_became_active":"2024-05-28T14:14:42.746338+0000","last_became_peered":"2024-05-28T14:14:42.746338+0000","last_unstale":"2024-05-28T14:14:43.751566+0000","last_undegraded":"2024-05-28T14:14:43.751566+0000","last_fullsized":"2024-05-28T14:14:43.751566+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:51:27.080888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.768968+0000","last_change":"2024-05-28T14:14:43.760455+0000","last_active":"2024-05-28T14:14:43.768968+0000","last_peered":"2024-05-28T14:14:43.768968+0000","last_clean":"2024-05-28T14:14:43.768968+0000","last_became_active":"2024-05-28T14:14:43.760063+0000","last_became_peered":"2024-05-28T14:14:43.760063+0000","last_unstale":"2024-05-28T14:14:43.768968+0000","last_undegraded":"2024-05-28T14:14:43.768968+0000","last_fullsized":"2024-05-28T14:14:43.768968+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:00:34.527970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.749327+0000","last_change":"2024-05-28T14:14:42.749618+0000","last_active":"2024-05-28T14:14:43.749327+0000","last_peered":"2024-05-28T14:14:43.749327+0000","last_clean":"2024-05-28T14:14:43.749327+0000","last_became_active":"2024-05-28T14:14:42.749259+0000","last_became_peered":"2024-05-28T14:14:42.749259+0000","last_unstale":"2024-05-28T14:14:43.749327+0000","last_undegraded":"2024-05-28T14:14:43.749327+0000","last_fullsized":"2024-05-28T14:14:43.749327+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:44:02.935848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.766609+0000","last_change":"2024-05-28T14:14:43.761804+0000","last_active":"2024-05-28T14:14:43.766609+0000","last_peered":"2024-05-28T14:14:43.766609+0000","last_clean":"2024-05-28T14:14:43.766609+0000","last_became_active":"2024-05-28T14:14:43.761382+0000","last_became_peered":"2024-05-28T14:14:43.761382+0000","last_unstale":"2024-05-28T14:14:43.766609+0000","last_undegraded":"2024-05-28T14:14:43.766609+0000","last_fullsized":"2024-05-28T14:14:43.766609+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:44:09.700729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.739994+0000","last_change":"2024-05-28T14:14:42.739854+0000","last_active":"2024-05-28T14:14:43.739994+0000","last_peered":"2024-05-28T14:14:43.739994+0000","last_clean":"2024-05-28T14:14:43.739994+0000","last_became_active":"2024-05-28T14:14:42.738369+0000","last_became_peered":"2024-05-28T14:14:42.738369+0000","last_unstale":"2024-05-28T14:14:43.739994+0000","last_undegraded":"2024-05-28T14:14:43.739994+0000","last_fullsized":"2024-05-28T14:14:43.739994+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:34:08.611902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.767427+0000","last_change":"2024-05-28T14:14:43.765205+0000","last_active":"2024-05-28T14:14:43.767427+0000","last_peered":"2024-05-28T14:14:43.767427+0000","last_clean":"2024-05-28T14:14:43.767427+0000","last_became_active":"2024-05-28T14:14:43.764879+0000","last_became_peered":"2024-05-28T14:14:43.764879+0000","last_unstale":"2024-05-28T14:14:43.767427+0000","last_undegraded":"2024-05-28T14:14:43.767427+0000","last_fullsized":"2024-05-28T14:14:43.767427+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:14.494237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.745266+0000","last_change":"2024-05-28T14:14:42.730611+0000","last_active":"2024-05-28T14:14:43.745266+0000","last_peered":"2024-05-28T14:14:43.745266+0000","last_clean":"2024-05-28T14:14:43.745266+0000","last_became_active":"2024-05-28T14:14:42.730333+0000","last_became_peered":"2024-05-28T14:14:42.730333+0000","last_unstale":"2024-05-28T14:14:43.745266+0000","last_undegraded":"2024-05-28T14:14:43.745266+0000","last_fullsized":"2024-05-28T14:14:43.745266+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:05:55.551707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.764935+0000","last_change":"2024-05-28T14:14:43.761743+0000","last_active":"2024-05-28T14:14:43.764935+0000","last_peered":"2024-05-28T14:14:43.764935+0000","last_clean":"2024-05-28T14:14:43.764935+0000","last_became_active":"2024-05-28T14:14:43.760828+0000","last_became_peered":"2024-05-28T14:14:43.760828+0000","last_unstale":"2024-05-28T14:14:43.764935+0000","last_undegraded":"2024-05-28T14:14:43.764935+0000","last_fullsized":"2024-05-28T14:14:43.764935+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:18:08.045261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"131'2","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.774995+0000","last_change":"2024-05-28T14:14:42.751868+0000","last_active":"2024-05-28T14:14:43.774995+0000","last_peered":"2024-05-28T14:14:43.774995+0000","last_clean":"2024-05-28T14:14:43.774995+0000","last_became_active":"2024-05-28T14:14:42.751447+0000","last_became_peered":"2024-05-28T14:14:42.751447+0000","last_unstale":"2024-05-28T14:14:43.774995+0000","last_undegraded":"2024-05-28T14:14:43.774995+0000","last_fullsized":"2024-05-28T14:14:43.774995+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:28:17.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":376,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-05-28T14:14:43.746410+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:14:43.746410+0000","last_peered":"2024-05-28T14:14:43.746410+0000","last_clean":"2024-05-28T14:14:43.746410+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:14:43.746410+0000","last_undegraded":"2024-05-28T14:14:43.746410+0000","last_fullsized":"2024-05-28T14:14:43.746410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575049,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136004,"kb_used_data":2164,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94235836,"statfs":{"total":96636764160,"available":96497496064,"internally_reserved":0,"allocated":2215936,"data_stored":1302702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":137034186},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575049,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135444,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236396,"statfs":{"total":96636764160,"available":96498069504,"internally_reserved":0,"allocated":1646592,"data_stored":720777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":137034182},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575051,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86864,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284976,"statfs":{"total":96636764160,"available":96547815424,"internally_reserved":0,"allocated":1642496,"data_stored":719277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":87292358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149324,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222516,"statfs":{"total":96636764160,"available":96483856384,"internally_reserved":0,"allocated":1638400,"data_stored":719235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575049,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145172,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226668,"statfs":{"total":96636764160,"available":96488108032,"internally_reserved":0,"allocated":1642496,"data_stored":719325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":146995656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141016,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230824,"statfs":{"total":96636764160,"available":96492363776,"internally_reserved":0,"allocated":1642496,"data_stored":719781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136832,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235008,"statfs":{"total":96636764160,"available":96496648192,"internally_reserved":0,"allocated":1622016,"data_stored":712334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575050,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149900,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221940,"statfs":{"total":96636764160,"available":96483266560,"internally_reserved":0,"allocated":2228224,"data_stored":1309533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:14:48.259 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-28T14:14:48.259 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-28T14:14:48.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.458+0000 7f72c5000640 1 Processor -- start 2024-05-28T14:14:48.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.458+0000 7f72c5000640 1 -- start start 2024-05-28T14:14:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.462+0000 7f72c5000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f72c00ce3b0 0x7f72c00ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.462+0000 7f72c5000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f72c013fd10 0x7f72c0140180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.462+0000 7f72c5000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72c0160230 con 0x7f72c0071de0 2024-05-28T14:14:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.462+0000 7f72c5000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f72c0160510 con 0x7f72c00ce3b0 2024-05-28T14:14:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.462+0000 7f72c5000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f72c01607f0 con 0x7f72c013fd10 2024-05-28T14:14:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.462+0000 7f72b5400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f72c00ce3b0 0x7f72c00ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.462+0000 7f72b5400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f72c00ce3b0 0x7f72c00ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47686/0 (socket says 172.21.15.42:47686) 2024-05-28T14:14:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.462+0000 7f72b5400640 1 -- 172.21.15.42:0/2682297764 learned_addr learned my addr 172.21.15.42:0/2682297764 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.462+0000 7f72be800640 1 --2- 172.21.15.42:0/2682297764 >> v2:172.21.15.63:3300/0 conn(0x7f72c013fd10 0x7f72c0140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.462+0000 7f72b5400640 1 -- 172.21.15.42:0/2682297764 >> v2:172.21.15.63:3300/0 conn(0x7f72c013fd10 msgr2=0x7f72c0140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.462+0000 7f72b5400640 1 --2- 172.21.15.42:0/2682297764 >> v2:172.21.15.63:3300/0 conn(0x7f72c013fd10 0x7f72c0140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.462+0000 7f72b5400640 1 -- 172.21.15.42:0/2682297764 >> v1:172.21.15.63:6789/0 conn(0x7f72c0071de0 legacy=0x7f72c00721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.462+0000 7f72b5400640 1 -- 172.21.15.42:0/2682297764 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c0160ad0 con 0x7f72c00ce3b0 2024-05-28T14:14:48.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.466+0000 7f72b5400640 1 --2- 172.21.15.42:0/2682297764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c00ce3b0 0x7f72c00ce810 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f72c0071680 tx=0x7f72a801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5e302cdbe2fc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:48.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.466+0000 7f72bd400640 1 -- 172.21.15.42:0/2682297764 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a80b0070 con 0x7f72c00ce3b0 2024-05-28T14:14:48.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.466+0000 7f72bd400640 1 -- 172.21.15.42:0/2682297764 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a80ab600 con 0x7f72c00ce3b0 2024-05-28T14:14:48.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.466+0000 7f72bd400640 1 -- 172.21.15.42:0/2682297764 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a80ab9d0 con 0x7f72c00ce3b0 2024-05-28T14:14:48.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.466+0000 7f72c5000640 1 -- 172.21.15.42:0/2682297764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c00ce3b0 msgr2=0x7f72c00ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.466+0000 7f72c5000640 1 --2- 172.21.15.42:0/2682297764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c00ce3b0 0x7f72c00ce810 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f72c0071680 tx=0x7f72a801c400 comp rx=0 tx=0).stop 2024-05-28T14:14:48.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.466+0000 7f72c5000640 1 -- 172.21.15.42:0/2682297764 shutdown_connections 2024-05-28T14:14:48.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.466+0000 7f72c5000640 1 --2- 172.21.15.42:0/2682297764 >> v2:172.21.15.63:3300/0 conn(0x7f72c013fd10 0x7f72c0140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.466+0000 7f72c5000640 1 --2- 172.21.15.42:0/2682297764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c00ce3b0 0x7f72c00ce810 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.466+0000 7f72c5000640 1 -- 172.21.15.42:0/2682297764 >> 172.21.15.42:0/2682297764 conn(0x7f72c00b21f0 msgr2=0x7f72c00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:48.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.466+0000 7f72c5000640 1 -- 172.21.15.42:0/2682297764 shutdown_connections 2024-05-28T14:14:48.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.466+0000 7f72c5000640 1 -- 172.21.15.42:0/2682297764 wait complete. 2024-05-28T14:14:48.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.466+0000 7f72c5000640 1 Processor -- start 2024-05-28T14:14:48.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.466+0000 7f72c5000640 1 -- start start 2024-05-28T14:14:48.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72c5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c0071de0 0x7f72c01548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72c5000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72c00ce3b0 0x7f72c01371d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72c5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72c013fd10 0x7f72c0159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72c5000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f72c0169830 con 0x7f72c0071de0 2024-05-28T14:14:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72c5000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01696b0 con 0x7f72c013fd10 2024-05-28T14:14:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72c5000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f72c01699b0 con 0x7f72c00ce3b0 2024-05-28T14:14:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72bde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c0071de0 0x7f72c01548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72bde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c0071de0 0x7f72c01548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47698/0 (socket says 172.21.15.42:47698) 2024-05-28T14:14:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72bde00640 1 -- 172.21.15.42:0/1886733295 learned_addr learned my addr 172.21.15.42:0/1886733295 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72be800640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72c013fd10 0x7f72c0159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72b5400640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72c00ce3b0 0x7f72c01371d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72bde00640 1 -- 172.21.15.42:0/1886733295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72c00ce3b0 msgr2=0x7f72c01371d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72bde00640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72c00ce3b0 0x7f72c01371d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72bde00640 1 -- 172.21.15.42:0/1886733295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72c013fd10 msgr2=0x7f72c0159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72bde00640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72c013fd10 0x7f72c0159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72bde00640 1 -- 172.21.15.42:0/1886733295 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c0154f50 con 0x7f72c0071de0 2024-05-28T14:14:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72be800640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72c013fd10 0x7f72c0159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72b5400640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72c00ce3b0 0x7f72c01371d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72bde00640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c0071de0 0x7f72c01548c0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c370 tx=0x7f72b000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72b6a00640 1 -- 172.21.15.42:0/1886733295 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b00052d0 con 0x7f72c0071de0 2024-05-28T14:14:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72b6a00640 1 -- 172.21.15.42:0/1886733295 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72b0005470 con 0x7f72c0071de0 2024-05-28T14:14:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72c5000640 1 -- 172.21.15.42:0/1886733295 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c0155230 con 0x7f72c0071de0 2024-05-28T14:14:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72b6a00640 1 -- 172.21.15.42:0/1886733295 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0007260 con 0x7f72c0071de0 2024-05-28T14:14:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72c5000640 1 -- 172.21.15.42:0/1886733295 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c01557f0 con 0x7f72c0071de0 2024-05-28T14:14:48.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72c5000640 1 -- 172.21.15.42:0/1886733295 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c00738b0 con 0x7f72c0071de0 2024-05-28T14:14:48.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.470+0000 7f72b6a00640 1 -- 172.21.15.42:0/1886733295 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f72b0007400 con 0x7f72c0071de0 2024-05-28T14:14:48.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.474+0000 7f72b6a00640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f729c069fd0 0x7f729c06c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.474+0000 7f72b6a00640 1 -- 172.21.15.42:0/1886733295 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f72b0084ce0 con 0x7f72c0071de0 2024-05-28T14:14:48.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.474+0000 7f72b5400640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f729c069fd0 0x7f729c06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.474+0000 7f72b5400640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f729c069fd0 0x7f729c06c490 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f72a80ad000 tx=0x7f72a8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:48.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.478+0000 7f72b6a00640 1 -- 172.21.15.42:0/1886733295 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f72b0056da0 con 0x7f72c0071de0 2024-05-28T14:14:48.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.702+0000 7f72c5000640 1 -- 172.21.15.42:0/1886733295 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f72c004fdd0 con 0x7f72c0071de0 2024-05-28T14:14:48.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.702+0000 7f72b6a00640 1 -- 172.21.15.42:0/1886733295 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7f72b00566a0 con 0x7f72c0071de0 2024-05-28T14:14:48.691 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-05-28T14:14:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDA4lVmFZ4+ChAAreLMDEuIiazMsVMtVboiCw== 2024-05-28T14:14:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow" 2024-05-28T14:14:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-05-28T14:14:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-28T14:14:48.691 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.0 2024-05-28T14:14:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.706+0000 7f72c5000640 1 -- 172.21.15.42:0/1886733295 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f729c069fd0 msgr2=0x7f729c06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.706+0000 7f72c5000640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f729c069fd0 0x7f729c06c490 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f72a80ad000 tx=0x7f72a8008040 comp rx=0 tx=0).stop 2024-05-28T14:14:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.706+0000 7f72c5000640 1 -- 172.21.15.42:0/1886733295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c0071de0 msgr2=0x7f72c01548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.706+0000 7f72c5000640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c0071de0 0x7f72c01548c0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c370 tx=0x7f72b000c830 comp rx=0 tx=0).stop 2024-05-28T14:14:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.706+0000 7f72c5000640 1 -- 172.21.15.42:0/1886733295 shutdown_connections 2024-05-28T14:14:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.706+0000 7f72c5000640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f729c069fd0 0x7f729c06c490 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.706+0000 7f72c5000640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72c013fd10 0x7f72c0159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.706+0000 7f72c5000640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72c00ce3b0 0x7f72c01371d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.706+0000 7f72c5000640 1 --2- 172.21.15.42:0/1886733295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c0071de0 0x7f72c01548c0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.706+0000 7f72c5000640 1 -- 172.21.15.42:0/1886733295 >> 172.21.15.42:0/1886733295 conn(0x7f72c00b21f0 msgr2=0x7f72c015b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.706+0000 7f72c5000640 1 -- 172.21.15.42:0/1886733295 shutdown_connections 2024-05-28T14:14:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.706+0000 7f72c5000640 1 -- 172.21.15.42:0/1886733295 wait complete. 2024-05-28T14:14:48.713 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:14:48.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8018400640 1 Processor -- start 2024-05-28T14:14:48.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8018400640 1 -- start start 2024-05-28T14:14:48.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8018400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f801009bbd0 0x7f801009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8018400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f801009af90 0x7f80100b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8018400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80100d0870 con 0x7f801015c990 2024-05-28T14:14:48.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8018400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f801009b400 con 0x7f801009af90 2024-05-28T14:14:48.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8018400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f801015cd70 con 0x7f801009bbd0 2024-05-28T14:14:48.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8016400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f801009af90 0x7f80100b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8016400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f801009af90 0x7f80100b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47710/0 (socket says 172.21.15.42:47710) 2024-05-28T14:14:48.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8016400640 1 -- 172.21.15.42:0/1165537667 learned_addr learned my addr 172.21.15.42:0/1165537667 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:48.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8015000640 1 --2- 172.21.15.42:0/1165537667 >> v2:172.21.15.63:3300/0 conn(0x7f801009bbd0 0x7f801009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8016400640 1 -- 172.21.15.42:0/1165537667 >> v2:172.21.15.63:3300/0 conn(0x7f801009bbd0 msgr2=0x7f801009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:48.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8016400640 1 --2- 172.21.15.42:0/1165537667 >> v2:172.21.15.63:3300/0 conn(0x7f801009bbd0 0x7f801009aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8016400640 1 -- 172.21.15.42:0/1165537667 >> v1:172.21.15.63:6789/0 conn(0x7f801015c990 legacy=0x7f80100d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:48.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8016400640 1 -- 172.21.15.42:0/1165537667 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80100cd300 con 0x7f801009af90 2024-05-28T14:14:48.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.870+0000 7f8015000640 1 --2- 172.21.15.42:0/1165537667 >> v2:172.21.15.63:3300/0 conn(0x7f801009bbd0 0x7f801009aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:48.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.874+0000 7f8016400640 1 --2- 172.21.15.42:0/1165537667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801009af90 0x7f80100b53e0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f8010072560 tx=0x7f800801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca787f10301117d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:48.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.874+0000 7f800fe00640 1 -- 172.21.15.42:0/1165537667 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80080a5840 con 0x7f801009af90 2024-05-28T14:14:48.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.874+0000 7f800fe00640 1 -- 172.21.15.42:0/1165537667 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80080a59e0 con 0x7f801009af90 2024-05-28T14:14:48.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.874+0000 7f800fe00640 1 -- 172.21.15.42:0/1165537667 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80080a5db0 con 0x7f801009af90 2024-05-28T14:14:48.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.874+0000 7f8018400640 1 -- 172.21.15.42:0/1165537667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801009af90 msgr2=0x7f80100b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.874+0000 7f8018400640 1 --2- 172.21.15.42:0/1165537667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801009af90 0x7f80100b53e0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f8010072560 tx=0x7f800801c980 comp rx=0 tx=0).stop 2024-05-28T14:14:48.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.874+0000 7f8018400640 1 -- 172.21.15.42:0/1165537667 shutdown_connections 2024-05-28T14:14:48.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.874+0000 7f8018400640 1 --2- 172.21.15.42:0/1165537667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801009af90 0x7f80100b53e0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.874+0000 7f8018400640 1 --2- 172.21.15.42:0/1165537667 >> v2:172.21.15.63:3300/0 conn(0x7f801009bbd0 0x7f801009aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.874+0000 7f8018400640 1 -- 172.21.15.42:0/1165537667 >> 172.21.15.42:0/1165537667 conn(0x7f80100b4080 msgr2=0x7f80100b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:48.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.874+0000 7f8018400640 1 -- 172.21.15.42:0/1165537667 shutdown_connections 2024-05-28T14:14:48.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.874+0000 7f8018400640 1 -- 172.21.15.42:0/1165537667 wait complete. 2024-05-28T14:14:48.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.874+0000 7f8018400640 1 Processor -- start 2024-05-28T14:14:48.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8018400640 1 -- start start 2024-05-28T14:14:48.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8018400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801009af90 0x7f801014fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8018400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801015c990 0x7f801006d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8018400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f801015bca0 0x7f801014f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8018400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f80100ce3a0 con 0x7f801015c990 2024-05-28T14:14:48.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8018400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80100cde80 con 0x7f801009af90 2024-05-28T14:14:48.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8018400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8010158e40 con 0x7f801015bca0 2024-05-28T14:14:48.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8015000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801015c990 0x7f801006d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8015000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801015c990 0x7f801006d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47716/0 (socket says 172.21.15.42:47716) 2024-05-28T14:14:48.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8015000640 1 -- 172.21.15.42:0/3920248233 learned_addr learned my addr 172.21.15.42:0/3920248233 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:48.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8016400640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f801015bca0 0x7f801014f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8015a00640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801009af90 0x7f801014fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8015000640 1 -- 172.21.15.42:0/3920248233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f801015bca0 msgr2=0x7f801014f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8015000640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f801015bca0 0x7f801014f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8015000640 1 -- 172.21.15.42:0/3920248233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801009af90 msgr2=0x7f801014fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:48.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8015000640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801009af90 0x7f801014fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:48.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8015000640 1 -- 172.21.15.42:0/3920248233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801006dc00 con 0x7f801015c990 2024-05-28T14:14:48.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8015a00640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801009af90 0x7f801014fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:48.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8015000640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801015c990 0x7f801006d6c0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00c7c0 tx=0x7f7ffc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:48.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f800d600640 1 -- 172.21.15.42:0/3920248233 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc007a30 con 0x7f801015c990 2024-05-28T14:14:48.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f800d600640 1 -- 172.21.15.42:0/3920248233 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ffc007bd0 con 0x7f801015c990 2024-05-28T14:14:48.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8018400640 1 -- 172.21.15.42:0/3920248233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801015aee0 con 0x7f801015c990 2024-05-28T14:14:48.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f800d600640 1 -- 172.21.15.42:0/3920248233 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0113f0 con 0x7f801015c990 2024-05-28T14:14:48.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8018400640 1 -- 172.21.15.42:0/3920248233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801015b4a0 con 0x7f801015c990 2024-05-28T14:14:48.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.878+0000 7f8018400640 1 -- 172.21.15.42:0/3920248233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8010068a90 con 0x7f801015c990 2024-05-28T14:14:48.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.882+0000 7f800d600640 1 -- 172.21.15.42:0/3920248233 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f7ffc011590 con 0x7f801015c990 2024-05-28T14:14:48.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.882+0000 7f800d600640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7ff0069fa0 0x7f7ff006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:48.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.882+0000 7f800d600640 1 -- 172.21.15.42:0/3920248233 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f7ffc0859d0 con 0x7f801015c990 2024-05-28T14:14:48.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.882+0000 7f8015a00640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7ff0069fa0 0x7f7ff006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:48.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.882+0000 7f8015a00640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7ff0069fa0 0x7f7ff006c460 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f80040098b0 tx=0x7f8004008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:48.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:48.886+0000 7f800d600640 1 -- 172.21.15.42:0/3920248233 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7ffc015020 con 0x7f801015c990 2024-05-28T14:14:49.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.086+0000 7f8018400640 1 -- 172.21.15.42:0/3920248233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80100b4ce0 con 0x7f801015c990 2024-05-28T14:14:49.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.086+0000 7f800d600640 1 -- 172.21.15.42:0/3920248233 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5188 (secure 0 0 0) 0x7f7ffc057440 con 0x7f801015c990 2024-05-28T14:14:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:14:49.073 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":84,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9208,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1217785682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1217785682},{"type":"v1","addr":"172.21.15.63:6834","nonce":1217785682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9813,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1231554897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1231554897},{"type":"v1","addr":"172.21.15.42:6835","nonce":1231554897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9819,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3152913608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3152913608},{"type":"v1","addr":"172.21.15.42:6837","nonce":3152913608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:14:43.128104+0000","modified":"2024-05-28T14:14:46.159994+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57851":{"gid":57851,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3508679013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3508679013},{"type":"v1","addr":"172.21.15.63:6835","nonce":3508679013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57851,"qdb_cluster":[57851]},"id":9}]} 2024-05-28T14:14:49.073 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 84 2024-05-28T14:14:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.086+0000 7f8018400640 1 -- 172.21.15.42:0/3920248233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7ff0069fa0 msgr2=0x7f7ff006c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.086+0000 7f8018400640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7ff0069fa0 0x7f7ff006c460 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f80040098b0 tx=0x7f8004008040 comp rx=0 tx=0).stop 2024-05-28T14:14:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.086+0000 7f8018400640 1 -- 172.21.15.42:0/3920248233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801015c990 msgr2=0x7f801006d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.086+0000 7f8018400640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801015c990 0x7f801006d6c0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00c7c0 tx=0x7f7ffc00cc80 comp rx=0 tx=0).stop 2024-05-28T14:14:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.086+0000 7f8018400640 1 -- 172.21.15.42:0/3920248233 shutdown_connections 2024-05-28T14:14:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.086+0000 7f8018400640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7ff0069fa0 0x7f7ff006c460 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.086+0000 7f8018400640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f801015bca0 0x7f801014f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.086+0000 7f8018400640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801015c990 0x7f801006d6c0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.086+0000 7f8018400640 1 --2- 172.21.15.42:0/3920248233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f801009af90 0x7f801014fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.090+0000 7f8018400640 1 -- 172.21.15.42:0/3920248233 >> 172.21.15.42:0/3920248233 conn(0x7f80100b4080 msgr2=0x7f80100b7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:49.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.090+0000 7f8018400640 1 -- 172.21.15.42:0/3920248233 shutdown_connections 2024-05-28T14:14:49.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.090+0000 7f8018400640 1 -- 172.21.15.42:0/3920248233 wait complete. 2024-05-28T14:14:49.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-28T14:14:43.128104+0000', 'modified': '2024-05-28T14:14:46.159994+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 57851}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_57851': {'gid': 57851, 'name': 'd', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6835/3508679013', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6833', 'nonce': 3508679013}, {'type': 'v1', 'addr': '172.21.15.63:6835', 'nonce': 3508679013}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 57851, 'qdb_cluster': [57851]} 2024-05-28T14:14:49.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-28T14:14:49.499 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-28T14:14:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.658+0000 7ff8d4200640 1 Processor -- start 2024-05-28T14:14:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.658+0000 7ff8d4200640 1 -- start start 2024-05-28T14:14:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8d4200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc15fe80 0x7ff8cc0afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8d4200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff8cc0b1940 0x7ff8cc09b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8d4200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8cc160260 con 0x7ff8cc0b13a0 2024-05-28T14:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8d4200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff8cc0ced80 con 0x7ff8cc0b1940 2024-05-28T14:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8d4200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff8cc0d1210 con 0x7ff8cc15fe80 2024-05-28T14:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8d2200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff8cc0b13a0 0x7ff8cc0d0b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53514/0 (socket says 172.21.15.42:53514) 2024-05-28T14:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8d2200640 1 -- 172.21.15.42:0/3008810632 learned_addr learned my addr 172.21.15.42:0/3008810632 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8d1800640 1 --2- 172.21.15.42:0/3008810632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc15fe80 0x7ff8cc0afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/3008810632 >> v2:172.21.15.63:3300/0 conn(0x7ff8cc0b1940 0x7ff8cc09b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8cbe00640 1 -- 172.21.15.42:0/3008810632 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 184394576 0 0) 0x7ff8cc160260 con 0x7ff8cc0b13a0 2024-05-28T14:14:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8cbe00640 1 -- 172.21.15.42:0/3008810632 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8b4003530 con 0x7ff8cc0b13a0 2024-05-28T14:14:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8d1800640 1 -- 172.21.15.42:0/3008810632 >> v2:172.21.15.63:3300/0 conn(0x7ff8cc0b1940 msgr2=0x7ff8cc09b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8d1800640 1 --2- 172.21.15.42:0/3008810632 >> v2:172.21.15.63:3300/0 conn(0x7ff8cc0b1940 0x7ff8cc09b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8d1800640 1 -- 172.21.15.42:0/3008810632 >> v1:172.21.15.42:6789/0 conn(0x7ff8cc0b13a0 legacy=0x7ff8cc0d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.662+0000 7ff8d1800640 1 -- 172.21.15.42:0/3008810632 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc0d0420 con 0x7ff8cc15fe80 2024-05-28T14:14:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.666+0000 7ff8d1800640 1 --2- 172.21.15.42:0/3008810632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc15fe80 0x7ff8cc0afcc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00097c0 tx=0x7ff8c001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13ab82abf3721f8c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.666+0000 7ff8cbe00640 1 -- 172.21.15.42:0/3008810632 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00b1070 con 0x7ff8cc15fe80 2024-05-28T14:14:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.666+0000 7ff8cbe00640 1 -- 172.21.15.42:0/3008810632 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8c00ac410 con 0x7ff8cc15fe80 2024-05-28T14:14:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.666+0000 7ff8cbe00640 1 -- 172.21.15.42:0/3008810632 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00ac5e0 con 0x7ff8cc15fe80 2024-05-28T14:14:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.666+0000 7ff8d4200640 1 -- 172.21.15.42:0/3008810632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc15fe80 msgr2=0x7ff8cc0afcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.666+0000 7ff8d4200640 1 --2- 172.21.15.42:0/3008810632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc15fe80 0x7ff8cc0afcc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00097c0 tx=0x7ff8c001c4a0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.666+0000 7ff8d4200640 1 -- 172.21.15.42:0/3008810632 shutdown_connections 2024-05-28T14:14:49.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.666+0000 7ff8d4200640 1 --2- 172.21.15.42:0/3008810632 >> v2:172.21.15.63:3300/0 conn(0x7ff8cc0b1940 0x7ff8cc09b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.666+0000 7ff8d4200640 1 --2- 172.21.15.42:0/3008810632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc15fe80 0x7ff8cc0afcc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.666+0000 7ff8d4200640 1 -- 172.21.15.42:0/3008810632 >> 172.21.15.42:0/3008810632 conn(0x7ff8cc0b4010 msgr2=0x7ff8cc0b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:49.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.666+0000 7ff8d4200640 1 -- 172.21.15.42:0/3008810632 shutdown_connections 2024-05-28T14:14:49.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.666+0000 7ff8d4200640 1 -- 172.21.15.42:0/3008810632 wait complete. 2024-05-28T14:14:49.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.666+0000 7ff8d4200640 1 Processor -- start 2024-05-28T14:14:49.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.666+0000 7ff8d4200640 1 -- start start 2024-05-28T14:14:49.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d4200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0b13a0 0x7ff8cc15ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d4200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc0b1940 0x7ff8cc0c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d4200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8cc15fe80 0x7ff8cc0c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d4200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc1674e0 con 0x7ff8cc0b13a0 2024-05-28T14:14:49.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d4200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc0d00f0 con 0x7ff8cc15fe80 2024-05-28T14:14:49.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d4200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff8cc167660 con 0x7ff8cc0b1940 2024-05-28T14:14:49.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0b13a0 0x7ff8cc15ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0b13a0 0x7ff8cc15ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47718/0 (socket says 172.21.15.42:47718) 2024-05-28T14:14:49.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d1800640 1 -- 172.21.15.42:0/1989678813 learned_addr learned my addr 172.21.15.42:0/1989678813 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:49.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc0b1940 0x7ff8cc0c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d2200640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8cc15fe80 0x7ff8cc0c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d1800640 1 -- 172.21.15.42:0/1989678813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc0b1940 msgr2=0x7ff8cc0c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d1800640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc0b1940 0x7ff8cc0c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d1800640 1 -- 172.21.15.42:0/1989678813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8cc15fe80 msgr2=0x7ff8cc0c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d1800640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8cc15fe80 0x7ff8cc0c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d1800640 1 -- 172.21.15.42:0/1989678813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc1530c0 con 0x7ff8cc0b13a0 2024-05-28T14:14:49.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d2200640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8cc15fe80 0x7ff8cc0c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:49.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d1800640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0b13a0 0x7ff8cc15ec30 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00af040 tx=0x7ff8c0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:49.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8c9600640 1 -- 172.21.15.42:0/1989678813 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00b1040 con 0x7ff8cc0b13a0 2024-05-28T14:14:49.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8c9600640 1 -- 172.21.15.42:0/1989678813 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8c00bc070 con 0x7ff8cc0b13a0 2024-05-28T14:14:49.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d4200640 1 -- 172.21.15.42:0/1989678813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc0ca8b0 con 0x7ff8cc0b13a0 2024-05-28T14:14:49.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8c9600640 1 -- 172.21.15.42:0/1989678813 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00b5410 con 0x7ff8cc0b13a0 2024-05-28T14:14:49.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.670+0000 7ff8d4200640 1 -- 172.21.15.42:0/1989678813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc0cae70 con 0x7ff8cc0b13a0 2024-05-28T14:14:49.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8c9600640 1 -- 172.21.15.42:0/1989678813 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7ff8c00b55b0 con 0x7ff8cc0b13a0 2024-05-28T14:14:49.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8c9600640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8ac069fa0 0x7ff8ac06c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8c9600640 1 -- 172.21.15.42:0/1989678813 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7ff8c011f1f0 con 0x7ff8cc0b13a0 2024-05-28T14:14:49.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d778a000 1 Processor -- start 2024-05-28T14:14:49.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8ac069fa0 0x7ff8ac06c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d778a000 1 -- start start 2024-05-28T14:14:49.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8ac069fa0 0x7ff8ac06c460 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0c5c20 tx=0x7ff8b800f040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:49.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d778a000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7ba120 0x5561ab7ba500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d778a000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7baa40 0x5561ab76d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d778a000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab76d650 0x5561ab76da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d778a000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5561ab5be420 con 0x5561ab7baa40 2024-05-28T14:14:49.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d778a000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5561ab558bf0 con 0x5561ab7ba120 2024-05-28T14:14:49.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d778a000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5561ab5ba3e0 con 0x5561ab76d650 2024-05-28T14:14:49.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7baa40 0x5561ab76d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7baa40 0x5561ab76d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47734/0 (socket says 172.21.15.42:47734) 2024-05-28T14:14:49.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d0e00640 1 -- 172.21.15.42:0/3157686003 learned_addr learned my addr 172.21.15.42:0/3157686003 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:49.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d2200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab76d650 0x5561ab76da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d1800640 1 --2- 172.21.15.42:0/3157686003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7ba120 0x5561ab7ba500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d0e00640 1 -- 172.21.15.42:0/3157686003 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab76d650 msgr2=0x5561ab76da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:49.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/3157686003 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab76d650 0x5561ab76da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d0e00640 1 -- 172.21.15.42:0/3157686003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7ba120 msgr2=0x5561ab7ba500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:49.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/3157686003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7ba120 0x5561ab7ba500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d0e00640 1 -- 172.21.15.42:0/3157686003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ab666450 con 0x5561ab7baa40 2024-05-28T14:14:49.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.674+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/3157686003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7baa40 0x5561ab76d110 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7ff8b801b490 tx=0x7ff8b8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebca5f6f37592715 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:49.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8be000640 1 -- 172.21.15.42:0/3157686003 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8029070 con 0x5561ab7baa40 2024-05-28T14:14:49.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8be000640 1 -- 172.21.15.42:0/3157686003 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b801f070 con 0x5561ab7baa40 2024-05-28T14:14:49.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8be000640 1 -- 172.21.15.42:0/3157686003 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8004430 con 0x5561ab7baa40 2024-05-28T14:14:49.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 -- 172.21.15.42:0/3157686003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7baa40 msgr2=0x5561ab76d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 --2- 172.21.15.42:0/3157686003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7baa40 0x5561ab76d110 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7ff8b801b490 tx=0x7ff8b8018000 comp rx=0 tx=0).stop 2024-05-28T14:14:49.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 -- 172.21.15.42:0/3157686003 shutdown_connections 2024-05-28T14:14:49.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 --2- 172.21.15.42:0/3157686003 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab76d650 0x5561ab76da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 --2- 172.21.15.42:0/3157686003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7baa40 0x5561ab76d110 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 --2- 172.21.15.42:0/3157686003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7ba120 0x5561ab7ba500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 -- 172.21.15.42:0/3157686003 >> 172.21.15.42:0/3157686003 conn(0x5561ab79aec0 msgr2=0x5561ab79d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:49.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 -- 172.21.15.42:0/3157686003 shutdown_connections 2024-05-28T14:14:49.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 -- 172.21.15.42:0/3157686003 wait complete. 2024-05-28T14:14:49.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 -- start start 2024-05-28T14:14:49.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 Processor -- start 2024-05-28T14:14:49.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab76d650 0x5561ab772b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7ba120 0x5561ab773080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab7baa40 0x5561ab7735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5561ab55eec0 con 0x5561ab7ba120 2024-05-28T14:14:49.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5561ab63f120 con 0x5561ab76d650 2024-05-28T14:14:49.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5561ab5c9350 con 0x5561ab7baa40 2024-05-28T14:14:49.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d778a000 10 client.0 _ll_register_callbacks cb 0x5561ab59b4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T14:14:49.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7ba120 0x5561ab773080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7ba120 0x5561ab773080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47748/0 (socket says 172.21.15.42:47748) 2024-05-28T14:14:49.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d0e00640 1 -- 172.21.15.42:0/1428378066 learned_addr learned my addr 172.21.15.42:0/1428378066 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d1800640 1 --2- 172.21.15.42:0/1428378066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab76d650 0x5561ab772b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d2200640 1 --2- 172.21.15.42:0/1428378066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab7baa40 0x5561ab7735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d0e00640 1 -- 172.21.15.42:0/1428378066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab7baa40 msgr2=0x5561ab7735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/1428378066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab7baa40 0x5561ab7735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d0e00640 1 -- 172.21.15.42:0/1428378066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab76d650 msgr2=0x5561ab772b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/1428378066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab76d650 0x5561ab772b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d0e00640 1 -- 172.21.15.42:0/1428378066 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ab603f80 con 0x5561ab7ba120 2024-05-28T14:14:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/1428378066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7ba120 0x5561ab773080 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800efd0 tx=0x7ff8b80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bfad61c724100d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.678+0000 7ff8b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T14:14:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8b2a00640 1 -- 172.21.15.42:0/1428378066 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8033070 con 0x5561ab7ba120 2024-05-28T14:14:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 -- 172.21.15.42:0/1428378066 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8b801f070 con 0x5561ab7ba120 2024-05-28T14:14:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8b2a00640 1 -- 172.21.15.42:0/1428378066 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7ff8b8003b70 con 0x5561ab7ba120 2024-05-28T14:14:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8b2a00640 1 -- 172.21.15.42:0/1428378066 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7ff8b803f050 con 0x5561ab7ba120 2024-05-28T14:14:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 10 client.9951 fetch_fsmap learned FSMap version 84 2024-05-28T14:14:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 -- 172.21.15.42:0/1428378066 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x5561ab597f00 con 0x5561ab7ba120 2024-05-28T14:14:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8b2a00640 1 -- 172.21.15.42:0/1428378066 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b801f070 con 0x5561ab7ba120 2024-05-28T14:14:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8b2a00640 1 -- 172.21.15.42:0/1428378066 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7ff8b8042020 con 0x5561ab7ba120 2024-05-28T14:14:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 10 client.9951 fetch_fsmap finished waiting for FSMap version 84 2024-05-28T14:14:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 10 client.9951 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-28T14:14:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 20 client.9951 populate_metadata read hostname 'smithi042' 2024-05-28T14:14:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 --2- 172.21.15.42:0/1428378066 >> [v2:172.21.15.63:6833/3508679013,v1:172.21.15.63:6835/3508679013] conn(0x5561ab7c93c0 0x5561ab7cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 4 client.9951 mds_command: new command op to 57851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-28T14:14:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 -- 172.21.15.42:0/1428378066 --> [v2:172.21.15.63:6833/3508679013,v1:172.21.15.63:6835/3508679013] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561ab557bc0 con 0x5561ab7c93c0 2024-05-28T14:14:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d1800640 1 --2- 172.21.15.42:0/1428378066 >> [v2:172.21.15.63:6833/3508679013,v1:172.21.15.63:6835/3508679013] conn(0x5561ab7c93c0 0x5561ab7cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d1800640 1 --2- 172.21.15.42:0/1428378066 >> [v2:172.21.15.63:6833/3508679013,v1:172.21.15.63:6835/3508679013] conn(0x5561ab7c93c0 0x5561ab7cb7b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a7045d8183c98d server_cookie=585399324f90814c in_seq=0 out_seq=0 2024-05-28T14:14:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8b2a00640 10 client.9951 ms_handle_connect on v2:172.21.15.63:6833/3508679013 2024-05-28T14:14:49.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8b2a00640 1 -- 172.21.15.42:0/1428378066 <== mds.0 v2:172.21.15.63:6833/3508679013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561ab557bc0 con 0x5561ab7c93c0 2024-05-28T14:14:49.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8b2a00640 10 client.9951 handle_command_reply: tid=0 2024-05-28T14:14:49.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 client.9951 shutdown 2024-05-28T14:14:49.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 -- 172.21.15.42:0/1428378066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7ba120 msgr2=0x5561ab773080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 --2- 172.21.15.42:0/1428378066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7ba120 0x5561ab773080 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800efd0 tx=0x7ff8b80046d0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 -- 172.21.15.42:0/1428378066 shutdown_connections 2024-05-28T14:14:49.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 --2- 172.21.15.42:0/1428378066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab7baa40 0x5561ab7735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 --2- 172.21.15.42:0/1428378066 >> [v2:172.21.15.63:6833/3508679013,v1:172.21.15.63:6835/3508679013] conn(0x5561ab7c93c0 0x5561ab7cb7b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 --2- 172.21.15.42:0/1428378066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7ba120 0x5561ab773080 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 --2- 172.21.15.42:0/1428378066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab76d650 0x5561ab772b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8b2a00640 0 client.9951 ms_handle_reset on v2:172.21.15.63:6833/3508679013 2024-05-28T14:14:49.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 -- 172.21.15.42:0/1428378066 >> 172.21.15.42:0/1428378066 conn(0x5561ab79aec0 msgr2=0x5561ab79d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:49.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 -- 172.21.15.42:0/1428378066 shutdown_connections 2024-05-28T14:14:49.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 1 -- 172.21.15.42:0/1428378066 wait complete. 2024-05-28T14:14:49.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.682+0000 7ff8d778a000 20 client.9951 trim_cache size 0 max 16384 2024-05-28T14:14:49.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 Processor -- start 2024-05-28T14:14:49.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 -- start start 2024-05-28T14:14:49.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab76d650 0x5561ab76c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7ba120 0x5561ab76cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7baa40 0x5561ab7cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc1fe9e0 con 0x5561ab7ba120 2024-05-28T14:14:49.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc1fdd70 con 0x5561ab7baa40 2024-05-28T14:14:49.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff8cc1fe390 con 0x5561ab76d650 2024-05-28T14:14:49.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7ba120 0x5561ab76cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7ba120 0x5561ab76cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47750/0 (socket says 172.21.15.42:47750) 2024-05-28T14:14:49.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d0e00640 1 -- 172.21.15.42:0/3586705464 learned_addr learned my addr 172.21.15.42:0/3586705464 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:49.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d2200640 1 --2- 172.21.15.42:0/3586705464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7baa40 0x5561ab7cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d0e00640 1 -- 172.21.15.42:0/3586705464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab76d650 msgr2=0x5561ab76c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:49.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/3586705464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab76d650 0x5561ab76c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d0e00640 1 -- 172.21.15.42:0/3586705464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7baa40 msgr2=0x5561ab7cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:49.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/3586705464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7baa40 0x5561ab7cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d0e00640 1 -- 172.21.15.42:0/3586705464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ab7ba870 con 0x5561ab7ba120 2024-05-28T14:14:49.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d2200640 1 --2- 172.21.15.42:0/3586705464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7baa40 0x5561ab7cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:49.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/3586705464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7ba120 0x5561ab76cd50 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8003b00 tx=0x7ff8b8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b99843a29310f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:49.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8bd600640 1 -- 172.21.15.42:0/3586705464 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8033070 con 0x5561ab7ba120 2024-05-28T14:14:49.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8bd600640 1 -- 172.21.15.42:0/3586705464 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b801f040 con 0x5561ab7ba120 2024-05-28T14:14:49.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8bd600640 1 -- 172.21.15.42:0/3586705464 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8029040 con 0x5561ab7ba120 2024-05-28T14:14:49.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 -- 172.21.15.42:0/3586705464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7ba120 msgr2=0x5561ab76cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 --2- 172.21.15.42:0/3586705464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7ba120 0x5561ab76cd50 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8003b00 tx=0x7ff8b8030000 comp rx=0 tx=0).stop 2024-05-28T14:14:49.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 -- 172.21.15.42:0/3586705464 shutdown_connections 2024-05-28T14:14:49.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 --2- 172.21.15.42:0/3586705464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7baa40 0x5561ab7cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 --2- 172.21.15.42:0/3586705464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab7ba120 0x5561ab76cd50 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 --2- 172.21.15.42:0/3586705464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab76d650 0x5561ab76c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 -- 172.21.15.42:0/3586705464 >> 172.21.15.42:0/3586705464 conn(0x5561ab79aec0 msgr2=0x5561ab76c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:49.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 -- 172.21.15.42:0/3586705464 shutdown_connections 2024-05-28T14:14:49.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.702+0000 7ff8d778a000 1 -- 172.21.15.42:0/3586705464 wait complete. 2024-05-28T14:14:49.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 1 -- start start 2024-05-28T14:14:49.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 1 Processor -- start 2024-05-28T14:14:49.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab76d650 0x5561ab7ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7ba120 0x5561ab7e5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab7baa40 0x5561ab7e5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5561ab5bd730 con 0x5561ab76d650 2024-05-28T14:14:49.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5561ab58a9e0 con 0x5561ab7ba120 2024-05-28T14:14:49.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5561ab5c2a50 con 0x5561ab7baa40 2024-05-28T14:14:49.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 10 client.0 _ll_register_callbacks cb 0x7ff8c0006ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-28T14:14:49.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab76d650 0x5561ab7ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab76d650 0x5561ab7ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47764/0 (socket says 172.21.15.42:47764) 2024-05-28T14:14:49.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d1800640 1 -- 172.21.15.42:0/726239276 learned_addr learned my addr 172.21.15.42:0/726239276 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:49.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/726239276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7ba120 0x5561ab7e5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d2200640 1 --2- 172.21.15.42:0/726239276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab7baa40 0x5561ab7e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d1800640 1 -- 172.21.15.42:0/726239276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab7baa40 msgr2=0x5561ab7e5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:49.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d1800640 1 --2- 172.21.15.42:0/726239276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab7baa40 0x5561ab7e5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d1800640 1 -- 172.21.15.42:0/726239276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7ba120 msgr2=0x5561ab7e5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:49.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d1800640 1 --2- 172.21.15.42:0/726239276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7ba120 0x5561ab7e5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d1800640 1 -- 172.21.15.42:0/726239276 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ab544e80 con 0x5561ab76d650 2024-05-28T14:14:49.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/726239276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7ba120 0x5561ab7e5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:49.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d1800640 1 --2- 172.21.15.42:0/726239276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab76d650 0x5561ab7ce650 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7ff8c001a0d0 tx=0x7ff8c00c3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d89a6caba25e0b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:49.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-05-28T14:14:49.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8b2a00640 1 -- 172.21.15.42:0/726239276 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00f0b10 con 0x5561ab76d650 2024-05-28T14:14:49.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8b2a00640 1 -- 172.21.15.42:0/726239276 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7ff8c00eff60 con 0x5561ab76d650 2024-05-28T14:14:49.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 1 -- 172.21.15.42:0/726239276 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8b801f040 con 0x5561ab76d650 2024-05-28T14:14:49.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8b2a00640 1 -- 172.21.15.42:0/726239276 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7ff8c00f4ec0 con 0x5561ab76d650 2024-05-28T14:14:49.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 10 client.9957 fetch_fsmap learned FSMap version 84 2024-05-28T14:14:49.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 1 -- 172.21.15.42:0/726239276 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x5561ab586b30 con 0x5561ab76d650 2024-05-28T14:14:49.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8b2a00640 1 -- 172.21.15.42:0/726239276 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b801f040 con 0x5561ab76d650 2024-05-28T14:14:49.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8b2a00640 1 -- 172.21.15.42:0/726239276 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7ff8c010a040 con 0x5561ab76d650 2024-05-28T14:14:49.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 10 client.9957 fetch_fsmap finished waiting for FSMap version 84 2024-05-28T14:14:49.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 10 client.9957 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-28T14:14:49.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 20 client.9957 populate_metadata read hostname 'smithi042' 2024-05-28T14:14:49.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 1 --2- 172.21.15.42:0/726239276 >> [v2:172.21.15.63:6833/3508679013,v1:172.21.15.63:6835/3508679013] conn(0x5561ab7e5dc0 0x5561ab828fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 4 client.9957 mds_command: new command op to 57851 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-28T14:14:49.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d778a000 1 -- 172.21.15.42:0/726239276 --> [v2:172.21.15.63:6833/3508679013,v1:172.21.15.63:6835/3508679013] -- command(tid 0: {"prefix": "status"}) -- 0x5561ab5cbe40 con 0x5561ab7e5dc0 2024-05-28T14:14:49.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/726239276 >> [v2:172.21.15.63:6833/3508679013,v1:172.21.15.63:6835/3508679013] conn(0x5561ab7e5dc0 0x5561ab828fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8d0e00640 1 --2- 172.21.15.42:0/726239276 >> [v2:172.21.15.63:6833/3508679013,v1:172.21.15.63:6835/3508679013] conn(0x5561ab7e5dc0 0x5561ab828fb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cee918ef174acda server_cookie=11138c2e7062a41c in_seq=0 out_seq=0 2024-05-28T14:14:49.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.706+0000 7ff8b2a00640 10 client.9957 ms_handle_connect on v2:172.21.15.63:6833/3508679013 2024-05-28T14:14:49.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8b2a00640 1 -- 172.21.15.42:0/726239276 <== mds.0 v2:172.21.15.63:6833/3508679013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5561ab5cbe40 con 0x5561ab7e5dc0 2024-05-28T14:14:49.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8b2a00640 10 client.9957 handle_command_reply: tid=0 2024-05-28T14:14:49.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d778a000 1 client.9957 shutdown 2024-05-28T14:14:49.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d778a000 1 -- 172.21.15.42:0/726239276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab76d650 msgr2=0x5561ab7ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d778a000 1 --2- 172.21.15.42:0/726239276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab76d650 0x5561ab7ce650 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7ff8c001a0d0 tx=0x7ff8c00c3040 comp rx=0 tx=0).stop 2024-05-28T14:14:49.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d778a000 1 -- 172.21.15.42:0/726239276 shutdown_connections 2024-05-28T14:14:49.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d778a000 1 --2- 172.21.15.42:0/726239276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5561ab7baa40 0x5561ab7e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d778a000 1 --2- 172.21.15.42:0/726239276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561ab7ba120 0x5561ab7e5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d778a000 1 --2- 172.21.15.42:0/726239276 >> [v2:172.21.15.63:6833/3508679013,v1:172.21.15.63:6835/3508679013] conn(0x5561ab7e5dc0 0x5561ab828fb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d778a000 1 --2- 172.21.15.42:0/726239276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561ab76d650 0x5561ab7ce650 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8b2a00640 0 client.9957 ms_handle_reset on v2:172.21.15.63:6833/3508679013 2024-05-28T14:14:49.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d778a000 1 -- 172.21.15.42:0/726239276 >> 172.21.15.42:0/726239276 conn(0x5561ab79aec0 msgr2=0x5561ab76c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:49.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d778a000 1 -- 172.21.15.42:0/726239276 shutdown_connections 2024-05-28T14:14:49.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d778a000 1 -- 172.21.15.42:0/726239276 wait complete. 2024-05-28T14:14:49.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d778a000 20 client.9957 trim_cache size 0 max 16384 2024-05-28T14:14:49.700 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:14:49.700 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:14:49.700 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-05-28T14:14:49.700 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 57851, 2024-05-28T14:14:49.700 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-05-28T14:14:49.700 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-05-28T14:14:49.700 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-05-28T14:14:49.701 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 5.9589058890000004, 2024-05-28T14:14:49.701 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 84, 2024-05-28T14:14:49.701 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 132, 2024-05-28T14:14:49.701 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 132, 2024-05-28T14:14:49.701 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 10.809923927 2024-05-28T14:14:49.701 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:14:49.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d4200640 1 -- 172.21.15.42:0/1989678813 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8ac069fa0 msgr2=0x7ff8ac06c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d4200640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8ac069fa0 0x7ff8ac06c460 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0c5c20 tx=0x7ff8b800f040 comp rx=0 tx=0).stop 2024-05-28T14:14:49.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d4200640 1 -- 172.21.15.42:0/1989678813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0b13a0 msgr2=0x7ff8cc15ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d4200640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0b13a0 0x7ff8cc15ec30 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00af040 tx=0x7ff8c0019240 comp rx=0 tx=0).stop 2024-05-28T14:14:49.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d4200640 1 -- 172.21.15.42:0/1989678813 shutdown_connections 2024-05-28T14:14:49.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d4200640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8ac069fa0 0x7ff8ac06c460 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d4200640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8cc15fe80 0x7ff8cc0c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d4200640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8cc0b1940 0x7ff8cc0c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d4200640 1 --2- 172.21.15.42:0/1989678813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8cc0b13a0 0x7ff8cc15ec30 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d4200640 1 -- 172.21.15.42:0/1989678813 >> 172.21.15.42:0/1989678813 conn(0x7ff8cc0b4010 msgr2=0x7ff8cc0b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:49.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d4200640 1 -- 172.21.15.42:0/1989678813 shutdown_connections 2024-05-28T14:14:49.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.710+0000 7ff8d4200640 1 -- 172.21.15.42:0/1989678813 wait complete. 2024-05-28T14:14:49.716 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-28T14:14:49.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.870+0000 7f6e86e00640 1 Processor -- start 2024-05-28T14:14:49.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.870+0000 7f6e86e00640 1 -- start start 2024-05-28T14:14:49.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e86e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6e80067810 0x7f6e8015ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e86e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e8006eaa0 con 0x7f6e800b42e0 2024-05-28T14:14:49.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e86e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e8006ed80 con 0x7f6e800d0420 2024-05-28T14:14:49.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e86e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6e8006f060 con 0x7f6e80067810 2024-05-28T14:14:49.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e77e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f6e800b42e0 0x7f6e8006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53516/0 (socket says 172.21.15.42:53516) 2024-05-28T14:14:49.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e77e00640 1 -- 172.21.15.42:0/2191348842 learned_addr learned my addr 172.21.15.42:0/2191348842 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e84e00640 1 --2- 172.21.15.42:0/2191348842 >> v2:172.21.15.63:3300/0 conn(0x7f6e80067810 0x7f6e8015ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e7f400640 1 -- 172.21.15.42:0/2191348842 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1689355348 0 0) 0x7f6e8006eaa0 con 0x7f6e800b42e0 2024-05-28T14:14:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e7f400640 1 -- 172.21.15.42:0/2191348842 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e64003530 con 0x7f6e800b42e0 2024-05-28T14:14:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e7f400640 1 -- 172.21.15.42:0/2191348842 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1312680334 0 0) 0x7f6e8006ed80 con 0x7f6e800d0420 2024-05-28T14:14:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e7f400640 1 -- 172.21.15.42:0/2191348842 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e8006eaa0 con 0x7f6e800d0420 2024-05-28T14:14:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e84e00640 1 -- 172.21.15.42:0/2191348842 >> v1:172.21.15.63:6789/0 conn(0x7f6e800d0420 legacy=0x7f6e800b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e7f400640 1 -- 172.21.15.42:0/2191348842 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4286757286 0 0) 0x7f6e64003530 con 0x7f6e800b42e0 2024-05-28T14:14:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e84e00640 1 -- 172.21.15.42:0/2191348842 >> v1:172.21.15.42:6789/0 conn(0x7f6e800b42e0 legacy=0x7f6e8006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.874+0000 7f6e84e00640 1 -- 172.21.15.42:0/2191348842 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8006f340 con 0x7f6e80067810 2024-05-28T14:14:49.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e84e00640 1 --2- 172.21.15.42:0/2191348842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e80067810 0x7f6e8015ed10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009d90 tx=0x7f6e7801d260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28c849614772862b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:49.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e7f400640 1 -- 172.21.15.42:0/2191348842 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e780b2070 con 0x7f6e80067810 2024-05-28T14:14:49.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e7f400640 1 -- 172.21.15.42:0/2191348842 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e780a5550 con 0x7f6e80067810 2024-05-28T14:14:49.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e7f400640 1 -- 172.21.15.42:0/2191348842 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68001680 con 0x7f6e80067810 2024-05-28T14:14:49.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e86e00640 1 -- 172.21.15.42:0/2191348842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e80067810 msgr2=0x7f6e8015ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e86e00640 1 --2- 172.21.15.42:0/2191348842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e80067810 0x7f6e8015ed10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009d90 tx=0x7f6e7801d260 comp rx=0 tx=0).stop 2024-05-28T14:14:49.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e86e00640 1 -- 172.21.15.42:0/2191348842 shutdown_connections 2024-05-28T14:14:49.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e86e00640 1 --2- 172.21.15.42:0/2191348842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e80067810 0x7f6e8015ed10 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e86e00640 1 -- 172.21.15.42:0/2191348842 >> 172.21.15.42:0/2191348842 conn(0x7f6e800d0b40 msgr2=0x7f6e800d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:49.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e86e00640 1 -- 172.21.15.42:0/2191348842 shutdown_connections 2024-05-28T14:14:49.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e86e00640 1 -- 172.21.15.42:0/2191348842 wait complete. 2024-05-28T14:14:49.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e86e00640 1 Processor -- start 2024-05-28T14:14:49.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e86e00640 1 -- start start 2024-05-28T14:14:49.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e86e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e80067810 0x7f6e80153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e86e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e800b42e0 0x7f6e80151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.878+0000 7f6e86e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e800d0420 0x7f6e8006f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e86e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8006cd30 con 0x7f6e80067810 2024-05-28T14:14:49.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e86e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8006cbb0 con 0x7f6e800d0420 2024-05-28T14:14:49.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e86e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6e8006ceb0 con 0x7f6e800b42e0 2024-05-28T14:14:49.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e7fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e80067810 0x7f6e80153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e7fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e80067810 0x7f6e80153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47766/0 (socket says 172.21.15.42:47766) 2024-05-28T14:14:49.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e7fe00640 1 -- 172.21.15.42:0/1147415537 learned_addr learned my addr 172.21.15.42:0/1147415537 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:49.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e84e00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e800d0420 0x7f6e8006f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e77e00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e800b42e0 0x7f6e80151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e7fe00640 1 -- 172.21.15.42:0/1147415537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e800b42e0 msgr2=0x7f6e80151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e7fe00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e800b42e0 0x7f6e80151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e7fe00640 1 -- 172.21.15.42:0/1147415537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e800d0420 msgr2=0x7f6e8006f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:49.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e7fe00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e800d0420 0x7f6e8006f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:49.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e7fe00640 1 -- 172.21.15.42:0/1147415537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e80152f00 con 0x7f6e80067810 2024-05-28T14:14:49.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e77e00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e800b42e0 0x7f6e80151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:49.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e7fe00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e80067810 0x7f6e80153fa0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000c490 tx=0x7f6e7000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:49.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e7cc00640 1 -- 172.21.15.42:0/1147415537 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e700052d0 con 0x7f6e80067810 2024-05-28T14:14:49.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e7cc00640 1 -- 172.21.15.42:0/1147415537 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e70005470 con 0x7f6e80067810 2024-05-28T14:14:49.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e7cc00640 1 -- 172.21.15.42:0/1147415537 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70012870 con 0x7f6e80067810 2024-05-28T14:14:49.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e86e00640 1 -- 172.21.15.42:0/1147415537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e801531e0 con 0x7f6e80067810 2024-05-28T14:14:49.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e86e00640 1 -- 172.21.15.42:0/1147415537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e801537a0 con 0x7f6e80067810 2024-05-28T14:14:49.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e86e00640 1 -- 172.21.15.42:0/1147415537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e80068a90 con 0x7f6e80067810 2024-05-28T14:14:49.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.882+0000 7f6e7cc00640 1 -- 172.21.15.42:0/1147415537 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f6e70012a10 con 0x7f6e80067810 2024-05-28T14:14:49.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.886+0000 7f6e7cc00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6e5c069fa0 0x7f6e5c06c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:49.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.886+0000 7f6e7cc00640 1 -- 172.21.15.42:0/1147415537 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f6e70085470 con 0x7f6e80067810 2024-05-28T14:14:49.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.886+0000 7f6e77e00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6e5c069fa0 0x7f6e5c06c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:49.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.886+0000 7f6e77e00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6e5c069fa0 0x7f6e5c06c460 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6e800b20c0 tx=0x7f6e68009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:49.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:49.890+0000 7f6e7cc00640 1 -- 172.21.15.42:0/1147415537 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6e70057530 con 0x7f6e80067810 2024-05-28T14:14:50.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50.094+0000 7f6e86e00640 1 -- 172.21.15.42:0/1147415537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6e80162c10 con 0x7f6e80067810 2024-05-28T14:14:50.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50.094+0000 7f6e7cc00640 1 -- 172.21.15.42:0/1147415537 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) ==== 53+0+73 (secure 0 0 0) 0x7f6e70056e30 con 0x7f6e80067810 2024-05-28T14:14:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-28T14:14:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50.094+0000 7f6e86e00640 1 -- 172.21.15.42:0/1147415537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6e5c069fa0 msgr2=0x7f6e5c06c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50.094+0000 7f6e86e00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6e5c069fa0 0x7f6e5c06c460 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6e800b20c0 tx=0x7f6e68009040 comp rx=0 tx=0).stop 2024-05-28T14:14:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50.094+0000 7f6e86e00640 1 -- 172.21.15.42:0/1147415537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e80067810 msgr2=0x7f6e80153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50.094+0000 7f6e86e00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e80067810 0x7f6e80153fa0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000c490 tx=0x7f6e7000c950 comp rx=0 tx=0).stop 2024-05-28T14:14:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50.094+0000 7f6e86e00640 1 -- 172.21.15.42:0/1147415537 shutdown_connections 2024-05-28T14:14:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50.094+0000 7f6e86e00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6e5c069fa0 0x7f6e5c06c460 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50.094+0000 7f6e86e00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e800d0420 0x7f6e8006f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50.094+0000 7f6e86e00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e800b42e0 0x7f6e80151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50.094+0000 7f6e86e00640 1 --2- 172.21.15.42:0/1147415537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e80067810 0x7f6e80153fa0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50.094+0000 7f6e86e00640 1 -- 172.21.15.42:0/1147415537 >> 172.21.15.42:0/1147415537 conn(0x7f6e800d0b40 msgr2=0x7f6e8015e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50.094+0000 7f6e86e00640 1 -- 172.21.15.42:0/1147415537 shutdown_connections 2024-05-28T14:14:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50.094+0000 7f6e86e00640 1 -- 172.21.15.42:0/1147415537 wait complete. 2024-05-28T14:14:50.102 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-28T14:14:50.102 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-28T14:14:50.102 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-05-28T14:14:50.102 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-28T14:14:50.102 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-28T14:14:50.102 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-28T14:14:50.102 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-28T14:14:50.102 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-28T14:14:50.103 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:14:50.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:50.103 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-05-28T14:14:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-28T14:14:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-28T14:14:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-05-28T14:14:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:14:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-05-28T14:14:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:14:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:14:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-05-28T14:14:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-05-28T14:14:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:14:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-05-28T14:14:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-05-28T14:14:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-28T14:14:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:14:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-05-28T14:14:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-28T14:14:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 41457sec preferred_lft 41457sec 2024-05-28T14:14:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2024-05-28T14:14:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:14:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-28T14:14:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:14:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-05-28T14:14:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-28T14:14:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 02:42:65:2b:63:73 brd ff:ff:ff:ff:ff:ff 2024-05-28T14:14:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-28T14:14:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:14:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-28T14:14:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-28T14:14:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-28T14:14:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:14:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-28T14:14:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:14:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T14:14:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:14:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-28T14:14:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:14:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-28T14:14:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:14:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-28T14:14:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-05-28T14:14:50.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:50.116 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-28T14:14:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-05-28T14:14:50.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:50.184 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-05-28T14:14:50.240 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-05-28T14:14:50.240 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-28T14:14:50.240 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-05-28T14:14:50.240 INFO:teuthology.orchestra.run.smithi042.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-28T14:14:50.240 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-05-28T14:14:50.240 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-28T14:14:50.241 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-28T14:14:50.241 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-05-28T14:14:50.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-28T14:14:50.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-28T14:14:50.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-28T14:14:50.241 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-05-28T14:14:50.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-28T14:14:50.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:14:50.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:50.380 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-28T14:14:50.389 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-28T14:14:50.389 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-28T14:14:50.390 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:14:50.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:50.390 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-05-28T14:14:50.455 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-28T14:14:50.455 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T14:14:50.455 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-28T14:14:50.507 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-28T14:14:50.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-28T14:14:50.507 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-28T14:14:50.558 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:14:50.623 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:14:50.676 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:14:50.676 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:14:50.677 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:14:50.677 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:14:50.677 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:14:50.677 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:14:50.677 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:14:50.677 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:14:50.677 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:14:50.677 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:14:50.677 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:14:50.677 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:14:50.677 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:14:50.677 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:14:50.677 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:14:50.678 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:14:50.678 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:14:50.678 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:14:50.678 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:14:50.678 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:14:50.678 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:14:50.678 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:14:50.678 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:14:50.678 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:14:50.678 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:14:50.678 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:14:50.678 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:14:50.678 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:14:50.679 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:14:50.679 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:14:50.679 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:14:50.679 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:14:50.679 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:14:50.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:50.680 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:14:50.731 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-28T14:14:50.731 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-28T14:14:50.777 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:14:50.798 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:14:50.848 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:14:50.848 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:14:50.848 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:14:50.848 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:14:50.849 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:14:50.849 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:14:50.849 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:14:50.849 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:14:50.849 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:14:50.849 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:14:50.849 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:14:50.849 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:14:50.849 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:14:50.849 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:14:50.849 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:14:50.849 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:14:50.850 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:14:50.850 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:14:50.850 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:14:50.850 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:14:50.850 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:14:50.850 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:14:50.850 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:14:50.850 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:14:50.850 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:14:50.850 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:14:50.850 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:14:50.850 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:14:50.850 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:14:50.851 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:14:50.851 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:14:50.851 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:14:50.851 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:14:50.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:50.852 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:14:50.875 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-05-28T14:14:50.886+0000 7fa06856e580 -1 init, newargv = 0x5629bab34c80 newargc=15 2024-05-28T14:14:50.875 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[54965]: starting ceph client 2024-05-28T14:14:50.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[54965]: starting fuse 2024-05-28T14:14:51.155 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:14:51.158 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:14:51.986 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-05-28T14:14:52.010 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-05-28T14:14:52.065 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-05-28T14:14:52.065 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-05-28T14:14:52.065 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-05-28T14:14:52.065 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-05-28T14:14:52.066 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-05-28T14:14:52.066 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-05-28T14:14:52.066 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-05-28T14:14:52.066 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-05-28T14:14:52.066 INFO:teuthology.orchestra.run.smithi042.stdout:/run/lock 2024-05-28T14:14:52.066 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-05-28T14:14:52.066 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-05-28T14:14:52.066 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-05-28T14:14:52.066 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-05-28T14:14:52.066 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-05-28T14:14:52.066 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-05-28T14:14:52.066 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-05-28T14:14:52.066 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-05-28T14:14:52.067 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/fs/nfsd 2024-05-28T14:14:52.067 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-05-28T14:14:52.067 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-05-28T14:14:52.067 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-05-28T14:14:52.067 INFO:teuthology.orchestra.run.smithi042.stdout:/run/rpc_pipefs 2024-05-28T14:14:52.067 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-05-28T14:14:52.067 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-05-28T14:14:52.067 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-05-28T14:14:52.067 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-05-28T14:14:52.068 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-05-28T14:14:52.068 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-3 2024-05-28T14:14:52.068 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-05-28T14:14:52.068 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:14:52.068 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:14:52.068 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:14:52.068 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:14:52.068 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-28T14:14:52.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:52.069 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-05-28T14:14:52.124 INFO:teuthology.orchestra.run.smithi042.stdout:44 2024-05-28T14:14:52.124 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-28T14:14:52.124 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-05-28T14:14:52.124 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:14:52.125 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-05-28T14:14:52.126 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-28T14:14:52.126 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-28T14:14:52.126 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-28T14:14:52.126 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-05-28T14:14:52.126 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-05-28T14:14:52.126 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-05-28T14:14:52.126 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-28T14:14:52.126 DEBUG:teuthology.orchestra.run.smithi042:> 2024-05-28T14:14:52.126 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-05-28T14:14:52.126 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-05-28T14:14:52.259 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.54965.asok 2024-05-28T14:14:52.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-28T14:14:52.274 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54965.asok 2024-05-28T14:14:52.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:52.274 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54965.asok status 2024-05-28T14:14:52.449 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:14:52.450 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-05-28T14:14:52.450 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-28T14:14:52.450 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-28T14:14:52.450 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-05-28T14:14:52.450 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-05-28T14:14:52.450 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-28T14:14:52.450 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "54965", 2024-05-28T14:14:52.450 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-05-28T14:14:52.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:52.450 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-05-28T14:14:52.450 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-05-28T14:14:52.450 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 9969, 2024-05-28T14:14:52.451 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-05-28T14:14:52.451 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-05-28T14:14:52.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-05-28T14:14:52.451 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 9969 2024-05-28T14:14:52.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:52.451 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T14:14:52.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T14:14:52.451 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T14:14:52.451 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3429907456 2024-05-28T14:14:52.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:52.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:52.451 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-05-28T14:14:52.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-05-28T14:14:52.451 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-05-28T14:14:52.452 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3429907456 2024-05-28T14:14:52.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:14:52.452 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.9969 192.168.0.1:0/3429907456", 2024-05-28T14:14:52.452 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/3429907456", 2024-05-28T14:14:52.452 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-05-28T14:14:52.452 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 84, 2024-05-28T14:14:52.452 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 132, 2024-05-28T14:14:52.452 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-05-28T14:14:52.452 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-05-28T14:14:52.452 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-05-28T14:14:52.452 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:14:52.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:52.465 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-05-28T14:14:52.465 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-28T14:14:52.474 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-05-28T14:14:52.474 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-28T14:14:52.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:52.475 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-28T14:14:52.537 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-28T14:14:52.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a4200640 1 Processor -- start 2024-05-28T14:14:52.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a4200640 1 -- start start 2024-05-28T14:14:52.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a4200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f619c15b350 0x7f619c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:52.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a4200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f619c072d40 0x7f619c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:52.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a4200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619c0737c0 0x7f619c0d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:52.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a4200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f619c0ae9f0 con 0x7f619c072d40 2024-05-28T14:14:52.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a4200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f619c0ade40 con 0x7f619c15b350 2024-05-28T14:14:52.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a4200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f619c0ae0c0 con 0x7f619c0737c0 2024-05-28T14:14:52.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a0e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f619c072d40 0x7f619c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:52.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a0e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f619c072d40 0x7f619c0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47774/0 (socket says 172.21.15.42:47774) 2024-05-28T14:14:52.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a0e00640 1 -- 172.21.15.42:0/1833133891 learned_addr learned my addr 172.21.15.42:0/1833133891 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:52.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a1800640 1 --2- 172.21.15.42:0/1833133891 >> v2:172.21.15.63:3300/0 conn(0x7f619c15b350 0x7f619c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:52.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a2200640 1 --2- 172.21.15.42:0/1833133891 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619c0737c0 0x7f619c0d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:52.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a0e00640 1 -- 172.21.15.42:0/1833133891 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619c0737c0 msgr2=0x7f619c0d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:52.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a0e00640 1 --2- 172.21.15.42:0/1833133891 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619c0737c0 0x7f619c0d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:52.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a0e00640 1 -- 172.21.15.42:0/1833133891 >> v2:172.21.15.63:3300/0 conn(0x7f619c15b350 msgr2=0x7f619c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:52.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a0e00640 1 --2- 172.21.15.42:0/1833133891 >> v2:172.21.15.63:3300/0 conn(0x7f619c15b350 0x7f619c06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:52.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.738+0000 7f61a0e00640 1 -- 172.21.15.42:0/1833133891 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c0ae3a0 con 0x7f619c072d40 2024-05-28T14:14:52.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.742+0000 7f61a0e00640 1 --2- 172.21.15.42:0/1833133891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f619c072d40 0x7f619c0d06f0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f619c0d13c0 tx=0x7f618801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8583a27ef4611d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:52.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.742+0000 7f619be00640 1 -- 172.21.15.42:0/1833133891 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880b0070 con 0x7f619c072d40 2024-05-28T14:14:52.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.742+0000 7f619be00640 1 -- 172.21.15.42:0/1833133891 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61880ab600 con 0x7f619c072d40 2024-05-28T14:14:52.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.742+0000 7f619be00640 1 -- 172.21.15.42:0/1833133891 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880ab9d0 con 0x7f619c072d40 2024-05-28T14:14:52.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.742+0000 7f61a4200640 1 -- 172.21.15.42:0/1833133891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f619c072d40 msgr2=0x7f619c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:52.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.742+0000 7f61a4200640 1 --2- 172.21.15.42:0/1833133891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f619c072d40 0x7f619c0d06f0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f619c0d13c0 tx=0x7f618801c490 comp rx=0 tx=0).stop 2024-05-28T14:14:52.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.742+0000 7f61a4200640 1 -- 172.21.15.42:0/1833133891 shutdown_connections 2024-05-28T14:14:52.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.742+0000 7f61a4200640 1 --2- 172.21.15.42:0/1833133891 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619c0737c0 0x7f619c0d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:52.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.742+0000 7f61a4200640 1 --2- 172.21.15.42:0/1833133891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f619c072d40 0x7f619c0d06f0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:52.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.742+0000 7f61a4200640 1 --2- 172.21.15.42:0/1833133891 >> v2:172.21.15.63:3300/0 conn(0x7f619c15b350 0x7f619c06dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:52.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.742+0000 7f61a4200640 1 -- 172.21.15.42:0/1833133891 >> 172.21.15.42:0/1833133891 conn(0x7f619c06bdf0 msgr2=0x7f619c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:52.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.742+0000 7f61a4200640 1 -- 172.21.15.42:0/1833133891 shutdown_connections 2024-05-28T14:14:52.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.742+0000 7f61a4200640 1 -- 172.21.15.42:0/1833133891 wait complete. 2024-05-28T14:14:52.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a4200640 1 Processor -- start 2024-05-28T14:14:52.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a4200640 1 -- start start 2024-05-28T14:14:52.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a4200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619c072d40 0x7f619c150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:52.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a4200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619c0737c0 0x7f619c1f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:52.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a4200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f619c15b350 0x7f619c14b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:52.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a4200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f619c0af7f0 con 0x7f619c15b350 2024-05-28T14:14:52.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a4200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f619c0af670 con 0x7f619c0737c0 2024-05-28T14:14:52.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a4200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f619c0af970 con 0x7f619c072d40 2024-05-28T14:14:52.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a2200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f619c15b350 0x7f619c14b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:52.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a1800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619c072d40 0x7f619c150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:52.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a2200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f619c15b350 0x7f619c14b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47778/0 (socket says 172.21.15.42:47778) 2024-05-28T14:14:52.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a2200640 1 -- 172.21.15.42:0/99756341 learned_addr learned my addr 172.21.15.42:0/99756341 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:52.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a0e00640 1 --2- 172.21.15.42:0/99756341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619c0737c0 0x7f619c1f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:52.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a2200640 1 -- 172.21.15.42:0/99756341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619c072d40 msgr2=0x7f619c150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:52.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a2200640 1 --2- 172.21.15.42:0/99756341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619c072d40 0x7f619c150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:52.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a2200640 1 -- 172.21.15.42:0/99756341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619c0737c0 msgr2=0x7f619c1f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:52.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a2200640 1 --2- 172.21.15.42:0/99756341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619c0737c0 0x7f619c1f8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:52.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a2200640 1 -- 172.21.15.42:0/99756341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c150ec0 con 0x7f619c15b350 2024-05-28T14:14:52.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a2200640 1 --2- 172.21.15.42:0/99756341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f619c15b350 0x7f619c14b730 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f6194007e60 tx=0x7f619400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:52.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f6199600640 1 -- 172.21.15.42:0/99756341 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6194014420 con 0x7f619c15b350 2024-05-28T14:14:52.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f6199600640 1 -- 172.21.15.42:0/99756341 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6194019070 con 0x7f619c15b350 2024-05-28T14:14:52.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f6199600640 1 -- 172.21.15.42:0/99756341 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6194014800 con 0x7f619c15b350 2024-05-28T14:14:52.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a4200640 1 -- 172.21.15.42:0/99756341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619c1f87a0 con 0x7f619c15b350 2024-05-28T14:14:52.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.746+0000 7f61a4200640 1 -- 172.21.15.42:0/99756341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619c14fbc0 con 0x7f619c15b350 2024-05-28T14:14:52.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.750+0000 7f61a4200640 1 -- 172.21.15.42:0/99756341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f619c068a90 con 0x7f619c15b350 2024-05-28T14:14:52.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.750+0000 7f6199600640 1 -- 172.21.15.42:0/99756341 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f61940149a0 con 0x7f619c15b350 2024-05-28T14:14:52.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.750+0000 7f6199600640 1 --2- 172.21.15.42:0/99756341 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f617c069ff0 0x7f617c06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:52.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.750+0000 7f6199600640 1 -- 172.21.15.42:0/99756341 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f6194085fd0 con 0x7f619c15b350 2024-05-28T14:14:52.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.750+0000 7f61a1800640 1 --2- 172.21.15.42:0/99756341 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f617c069ff0 0x7f617c06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:52.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.750+0000 7f61a1800640 1 --2- 172.21.15.42:0/99756341 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f617c069ff0 0x7f617c06c4b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f619c15a690 tx=0x7f6190008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:52.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.754+0000 7f6199600640 1 -- 172.21.15.42:0/99756341 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6194091080 con 0x7f619c15b350 2024-05-28T14:14:52.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:52.954+0000 7f61a4200640 1 -- 172.21.15.42:0/99756341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f619c0b03e0 con 0x7f619c15b350 2024-05-28T14:14:53.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.786+0000 7f6199600640 1 -- 172.21.15.42:0/99756341 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v85) ==== 136+0+0 (secure 0 0 0) 0x7f6194057a20 con 0x7f619c15b350 2024-05-28T14:14:53.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.790+0000 7f61a4200640 1 -- 172.21.15.42:0/99756341 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f617c069ff0 msgr2=0x7f617c06c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:53.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.790+0000 7f61a4200640 1 --2- 172.21.15.42:0/99756341 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f617c069ff0 0x7f617c06c4b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f619c15a690 tx=0x7f6190008040 comp rx=0 tx=0).stop 2024-05-28T14:14:53.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.790+0000 7f61a4200640 1 -- 172.21.15.42:0/99756341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f619c15b350 msgr2=0x7f619c14b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:53.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.790+0000 7f61a4200640 1 --2- 172.21.15.42:0/99756341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f619c15b350 0x7f619c14b730 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f6194007e60 tx=0x7f619400e4b0 comp rx=0 tx=0).stop 2024-05-28T14:14:53.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.790+0000 7f61a4200640 1 -- 172.21.15.42:0/99756341 shutdown_connections 2024-05-28T14:14:53.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.790+0000 7f61a4200640 1 --2- 172.21.15.42:0/99756341 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f617c069ff0 0x7f617c06c4b0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:53.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.790+0000 7f61a4200640 1 --2- 172.21.15.42:0/99756341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f619c15b350 0x7f619c14b730 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:53.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.790+0000 7f61a4200640 1 --2- 172.21.15.42:0/99756341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619c0737c0 0x7f619c1f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:53.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.790+0000 7f61a4200640 1 --2- 172.21.15.42:0/99756341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619c072d40 0x7f619c150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:53.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.790+0000 7f61a4200640 1 -- 172.21.15.42:0/99756341 >> 172.21.15.42:0/99756341 conn(0x7f619c06bdf0 msgr2=0x7f619c0b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:53.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.790+0000 7f61a4200640 1 -- 172.21.15.42:0/99756341 shutdown_connections 2024-05-28T14:14:53.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.790+0000 7f61a4200640 1 -- 172.21.15.42:0/99756341 wait complete. 2024-05-28T14:14:53.800 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:14:53.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.962+0000 7f23ce200640 1 Processor -- start 2024-05-28T14:14:53.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.962+0000 7f23ce200640 1 -- start start 2024-05-28T14:14:53.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.962+0000 7f23ce200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f23c8099b20 0x7f23c806f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:53.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.962+0000 7f23ce200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c806e8e0 con 0x7f23c80af690 2024-05-28T14:14:53.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.962+0000 7f23ce200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c806ebc0 con 0x7f23c80b38f0 2024-05-28T14:14:53.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.962+0000 7f23ce200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f23c8099f00 con 0x7f23c8099b20 2024-05-28T14:14:53.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.962+0000 7f23c7e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f23c80af690 0x7f23c80b31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53524/0 (socket says 172.21.15.42:53524) 2024-05-28T14:14:53.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.962+0000 7f23c7e00640 1 -- 172.21.15.42:0/1601604198 learned_addr learned my addr 172.21.15.42:0/1601604198 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:53.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.962+0000 7f23cce00640 1 --2- 172.21.15.42:0/1601604198 >> v2:172.21.15.63:3300/0 conn(0x7f23c8099b20 0x7f23c806f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:53.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:53.966+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3756000549 0 0) 0x7f23c806e8e0 con 0x7f23c80af690 2024-05-28T14:14:53.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.002+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23b0003510 con 0x7f23c80af690 2024-05-28T14:14:53.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.002+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 776370747 0 0) 0x7f23c806ebc0 con 0x7f23c80b38f0 2024-05-28T14:14:53.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.002+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23c806e8e0 con 0x7f23c80b38f0 2024-05-28T14:14:53.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 301225892 0 0) 0x7f23b0003510 con 0x7f23c80af690 2024-05-28T14:14:53.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 >> v2:172.21.15.63:3300/0 conn(0x7f23c8099b20 msgr2=0x7f23c806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:53.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23c7400640 1 --2- 172.21.15.42:0/1601604198 >> v2:172.21.15.63:3300/0 conn(0x7f23c8099b20 0x7f23c806f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:53.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 >> v1:172.21.15.63:6789/0 conn(0x7f23c80b38f0 legacy=0x7f23c80ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:53.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c806eea0 con 0x7f23c80af690 2024-05-28T14:14:53.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f23b4001680 con 0x7f23c80af690 2024-05-28T14:14:53.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 >> v1:172.21.15.42:6789/0 conn(0x7f23c80af690 legacy=0x7f23c80b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:53.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23c7400640 1 --2- 172.21.15.42:0/1601604198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23b0003ce0 0x7f23b00040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:53.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23c7400640 1 --2- 172.21.15.42:0/1601604198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23b0004690 0x7f23b000b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:53.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23c7400640 1 --2- 172.21.15.42:0/1601604198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b000b560 0x7f23b000d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:53.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8099f00 con 0x7f23b000b560 2024-05-28T14:14:53.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f23b000e1b0 con 0x7f23b0004690 2024-05-28T14:14:53.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f23b000e3b0 con 0x7f23b0003ce0 2024-05-28T14:14:53.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23cd800640 1 --2- 172.21.15.42:0/1601604198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b000b560 0x7f23b000d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:53.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23cce00640 1 --2- 172.21.15.42:0/1601604198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23b0003ce0 0x7f23b00040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:53.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23c7e00640 1 --2- 172.21.15.42:0/1601604198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23b0004690 0x7f23b000b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:53.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23cd800640 1 -- 172.21.15.42:0/1601604198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23b0003ce0 msgr2=0x7f23b00040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:53.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23cd800640 1 --2- 172.21.15.42:0/1601604198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23b0003ce0 0x7f23b00040e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:53.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23cd800640 1 -- 172.21.15.42:0/1601604198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23b0004690 msgr2=0x7f23b000b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:53.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23cd800640 1 --2- 172.21.15.42:0/1601604198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23b0004690 0x7f23b000b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:53.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23cd800640 1 -- 172.21.15.42:0/1601604198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b0003510 con 0x7f23b000b560 2024-05-28T14:14:54.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.006+0000 7f23cd800640 1 -- 172.21.15.42:0/1601604198 reap_dead start 2024-05-28T14:14:54.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23cd800640 1 --2- 172.21.15.42:0/1601604198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b000b560 0x7f23b000d970 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f23c0001fd0 tx=0x7f23c001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e0838fb6390fb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:54.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c0001c20 con 0x7f23b000b560 2024-05-28T14:14:54.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c0003980 con 0x7f23b000b560 2024-05-28T14:14:54.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23c7400640 1 -- 172.21.15.42:0/1601604198 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c0003d70 con 0x7f23b000b560 2024-05-28T14:14:54.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23ce200640 1 -- 172.21.15.42:0/1601604198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b000b560 msgr2=0x7f23b000d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23ce200640 1 --2- 172.21.15.42:0/1601604198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b000b560 0x7f23b000d970 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f23c0001fd0 tx=0x7f23c001c900 comp rx=0 tx=0).stop 2024-05-28T14:14:54.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23ce200640 1 -- 172.21.15.42:0/1601604198 shutdown_connections 2024-05-28T14:14:54.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23ce200640 1 --2- 172.21.15.42:0/1601604198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b000b560 0x7f23b000d970 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23ce200640 1 -- 172.21.15.42:0/1601604198 >> 172.21.15.42:0/1601604198 conn(0x7f23c80b1ee0 msgr2=0x7f23c80b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:54.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23ce200640 1 -- 172.21.15.42:0/1601604198 shutdown_connections 2024-05-28T14:14:54.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23ce200640 1 -- 172.21.15.42:0/1601604198 wait complete. 2024-05-28T14:14:54.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23ce200640 1 Processor -- start 2024-05-28T14:14:54.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23ce200640 1 -- start start 2024-05-28T14:14:54.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23ce200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b000b560 0x7f23c8092d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23ce200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23c8096180 0x7f23c816ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.010+0000 7f23ce200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23c816e4d0 0x7f23c816e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23ce200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f23c80b2b80 con 0x7f23b000b560 2024-05-28T14:14:54.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23ce200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f23c80b2a00 con 0x7f23c816e4d0 2024-05-28T14:14:54.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23ce200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f23c80b2d00 con 0x7f23c8096180 2024-05-28T14:14:54.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23cce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b000b560 0x7f23c8092d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23cce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b000b560 0x7f23c8092d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47794/0 (socket says 172.21.15.42:47794) 2024-05-28T14:14:54.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23cce00640 1 -- 172.21.15.42:0/3848058395 learned_addr learned my addr 172.21.15.42:0/3848058395 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:54.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23c7e00640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23c8096180 0x7f23c816ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23cd800640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23c816e4d0 0x7f23c816e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23cce00640 1 -- 172.21.15.42:0/3848058395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23c8096180 msgr2=0x7f23c816ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23cce00640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23c8096180 0x7f23c816ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23cce00640 1 -- 172.21.15.42:0/3848058395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23c816e4d0 msgr2=0x7f23c816e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23cce00640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23c816e4d0 0x7f23c816e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23cce00640 1 -- 172.21.15.42:0/3848058395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c8092af0 con 0x7f23b000b560 2024-05-28T14:14:54.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23c7e00640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23c8096180 0x7f23c816ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:54.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23cce00640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b000b560 0x7f23c8092d60 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00e8a0 tx=0x7f23bc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:54.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23c4c00640 1 -- 172.21.15.42:0/3848058395 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc019070 con 0x7f23b000b560 2024-05-28T14:14:54.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23c4c00640 1 -- 172.21.15.42:0/3848058395 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23bc004240 con 0x7f23b000b560 2024-05-28T14:14:54.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23c4c00640 1 -- 172.21.15.42:0/3848058395 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc011810 con 0x7f23b000b560 2024-05-28T14:14:54.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23ce200640 1 -- 172.21.15.42:0/3848058395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c816d430 con 0x7f23b000b560 2024-05-28T14:14:54.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23ce200640 1 -- 172.21.15.42:0/3848058395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c816d9f0 con 0x7f23b000b560 2024-05-28T14:14:54.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23ce200640 1 -- 172.21.15.42:0/3848058395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c804fcf0 con 0x7f23b000b560 2024-05-28T14:14:54.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.014+0000 7f23c4c00640 1 -- 172.21.15.42:0/3848058395 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f23bc01c070 con 0x7f23b000b560 2024-05-28T14:14:54.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.018+0000 7f23c4c00640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23a8069fd0 0x7f23a806c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.018+0000 7f23c4c00640 1 -- 172.21.15.42:0/3848058395 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f23bc085fd0 con 0x7f23b000b560 2024-05-28T14:14:54.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.018+0000 7f23c7e00640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23a8069fd0 0x7f23a806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.018+0000 7f23c7e00640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23a8069fd0 0x7f23a806c490 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f23b4001060 tx=0x7f23b4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:54.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.022+0000 7f23c4c00640 1 -- 172.21.15.42:0/3848058395 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f23bc010040 con 0x7f23b000b560 2024-05-28T14:14:54.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.210+0000 7f23ce200640 1 -- 172.21.15.42:0/3848058395 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f23c80737e0 con 0x7f23a8069fd0 2024-05-28T14:14:54.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.210+0000 7f23c4c00640 1 -- 172.21.15.42:0/3848058395 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f23c80737e0 con 0x7f23a8069fd0 2024-05-28T14:14:54.198 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:14:54.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:14:54.198 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:14:54.198 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:14:54.198 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:14:54.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.214+0000 7f23ce200640 1 -- 172.21.15.42:0/3848058395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23a8069fd0 msgr2=0x7f23a806c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.214+0000 7f23ce200640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23a8069fd0 0x7f23a806c490 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f23b4001060 tx=0x7f23b4009040 comp rx=0 tx=0).stop 2024-05-28T14:14:54.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.214+0000 7f23ce200640 1 -- 172.21.15.42:0/3848058395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b000b560 msgr2=0x7f23c8092d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.214+0000 7f23ce200640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b000b560 0x7f23c8092d60 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00e8a0 tx=0x7f23bc00ed60 comp rx=0 tx=0).stop 2024-05-28T14:14:54.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.214+0000 7f23ce200640 1 -- 172.21.15.42:0/3848058395 shutdown_connections 2024-05-28T14:14:54.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.214+0000 7f23ce200640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f23a8069fd0 0x7f23a806c490 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.214+0000 7f23ce200640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23c816e4d0 0x7f23c816e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.214+0000 7f23ce200640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23c8096180 0x7f23c816ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.214+0000 7f23ce200640 1 --2- 172.21.15.42:0/3848058395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23b000b560 0x7f23c8092d60 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.214+0000 7f23ce200640 1 -- 172.21.15.42:0/3848058395 >> 172.21.15.42:0/3848058395 conn(0x7f23c80b1ee0 msgr2=0x7f23c8098310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:54.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.214+0000 7f23ce200640 1 -- 172.21.15.42:0/3848058395 shutdown_connections 2024-05-28T14:14:54.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.214+0000 7f23ce200640 1 -- 172.21.15.42:0/3848058395 wait complete. 2024-05-28T14:14:54.220 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-28T14:14:54.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1cb3a00640 1 Processor -- start 2024-05-28T14:14:54.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1cb3a00640 1 -- start start 2024-05-28T14:14:54.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1cb3a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1cac071de0 0x7f1cac0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1cb3a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1cac0ce3b0 0x7f1cac0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1cb3a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cac072700 con 0x7f1cac141d10 2024-05-28T14:14:54.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1cb3a00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1cac160230 con 0x7f1cac0ce3b0 2024-05-28T14:14:54.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1cb3a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1cac160510 con 0x7f1cac071de0 2024-05-28T14:14:54.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1ca3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1cac0ce3b0 0x7f1cac0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1ca3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1cac0ce3b0 0x7f1cac0ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47800/0 (socket says 172.21.15.42:47800) 2024-05-28T14:14:54.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1ca3e00640 1 -- 172.21.15.42:0/3309594797 learned_addr learned my addr 172.21.15.42:0/3309594797 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:54.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1cb1000640 1 --2- 172.21.15.42:0/3309594797 >> v2:172.21.15.63:3300/0 conn(0x7f1cac071de0 0x7f1cac0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1ca3e00640 1 -- 172.21.15.42:0/3309594797 >> v2:172.21.15.63:3300/0 conn(0x7f1cac071de0 msgr2=0x7f1cac0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:54.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1ca3e00640 1 --2- 172.21.15.42:0/3309594797 >> v2:172.21.15.63:3300/0 conn(0x7f1cac071de0 0x7f1cac0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1ca3e00640 1 -- 172.21.15.42:0/3309594797 >> v1:172.21.15.63:6789/0 conn(0x7f1cac141d10 legacy=0x7f1cac142160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.378+0000 7f1ca3e00640 1 -- 172.21.15.42:0/3309594797 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cac1607f0 con 0x7f1cac0ce3b0 2024-05-28T14:14:54.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.382+0000 7f1ca3e00640 1 --2- 172.21.15.42:0/3309594797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac0ce3b0 0x7f1cac0ce810 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f1cac06cbf0 tx=0x7f1c9401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a713a9965b797fed server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:54.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.382+0000 7f1cabe00640 1 -- 172.21.15.42:0/3309594797 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c940b0070 con 0x7f1cac0ce3b0 2024-05-28T14:14:54.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.382+0000 7f1cabe00640 1 -- 172.21.15.42:0/3309594797 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c940ab600 con 0x7f1cac0ce3b0 2024-05-28T14:14:54.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.382+0000 7f1cabe00640 1 -- 172.21.15.42:0/3309594797 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c940ab9d0 con 0x7f1cac0ce3b0 2024-05-28T14:14:54.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.382+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3309594797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac0ce3b0 msgr2=0x7f1cac0ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.382+0000 7f1cb3a00640 1 --2- 172.21.15.42:0/3309594797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac0ce3b0 0x7f1cac0ce810 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f1cac06cbf0 tx=0x7f1c9401c400 comp rx=0 tx=0).stop 2024-05-28T14:14:54.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.382+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3309594797 shutdown_connections 2024-05-28T14:14:54.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.382+0000 7f1cb3a00640 1 --2- 172.21.15.42:0/3309594797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac0ce3b0 0x7f1cac0ce810 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.382+0000 7f1cb3a00640 1 --2- 172.21.15.42:0/3309594797 >> v2:172.21.15.63:3300/0 conn(0x7f1cac071de0 0x7f1cac0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.382+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3309594797 >> 172.21.15.42:0/3309594797 conn(0x7f1cac0b21f0 msgr2=0x7f1cac0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:54.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.382+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3309594797 shutdown_connections 2024-05-28T14:14:54.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.382+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3309594797 wait complete. 2024-05-28T14:14:54.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb3a00640 1 Processor -- start 2024-05-28T14:14:54.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb3a00640 1 -- start start 2024-05-28T14:14:54.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb3a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac071de0 0x7f1cac137e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb3a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cac0ce3b0 0x7f1cac1456c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb3a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cac141d10 0x7f1cac154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb3a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac169e30 con 0x7f1cac071de0 2024-05-28T14:14:54.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb3a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac169cb0 con 0x7f1cac0ce3b0 2024-05-28T14:14:54.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb3a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1cac169fb0 con 0x7f1cac141d10 2024-05-28T14:14:54.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb1000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac071de0 0x7f1cac137e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb1000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac071de0 0x7f1cac137e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47812/0 (socket says 172.21.15.42:47812) 2024-05-28T14:14:54.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb1000640 1 -- 172.21.15.42:0/3458591239 learned_addr learned my addr 172.21.15.42:0/3458591239 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:54.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1ca3e00640 1 --2- 172.21.15.42:0/3458591239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cac0ce3b0 0x7f1cac1456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb1a00640 1 --2- 172.21.15.42:0/3458591239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cac141d10 0x7f1cac154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb1000640 1 -- 172.21.15.42:0/3458591239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cac141d10 msgr2=0x7f1cac154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb1000640 1 --2- 172.21.15.42:0/3458591239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cac141d10 0x7f1cac154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb1000640 1 -- 172.21.15.42:0/3458591239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cac0ce3b0 msgr2=0x7f1cac1456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb1000640 1 --2- 172.21.15.42:0/3458591239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cac0ce3b0 0x7f1cac1456c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb1000640 1 -- 172.21.15.42:0/3458591239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cac15f690 con 0x7f1cac071de0 2024-05-28T14:14:54.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb1000640 1 --2- 172.21.15.42:0/3458591239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac071de0 0x7f1cac137e90 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00d620 tx=0x7f1c9c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:54.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1ca9600640 1 -- 172.21.15.42:0/3458591239 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c00b620 con 0x7f1cac071de0 2024-05-28T14:14:54.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1ca9600640 1 -- 172.21.15.42:0/3458591239 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9c019070 con 0x7f1cac071de0 2024-05-28T14:14:54.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1ca9600640 1 -- 172.21.15.42:0/3458591239 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c00b9e0 con 0x7f1cac071de0 2024-05-28T14:14:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3458591239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cac15f970 con 0x7f1cac071de0 2024-05-28T14:14:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3458591239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cac145c00 con 0x7f1cac071de0 2024-05-28T14:14:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.386+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3458591239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cac15fe90 con 0x7f1cac071de0 2024-05-28T14:14:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.390+0000 7f1ca9600640 1 -- 172.21.15.42:0/3458591239 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f1c9c00bb80 con 0x7f1cac071de0 2024-05-28T14:14:54.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.390+0000 7f1ca9600640 1 --2- 172.21.15.42:0/3458591239 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c88069fa0 0x7f1c8806c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.390+0000 7f1ca9600640 1 -- 172.21.15.42:0/3458591239 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f1c9c085cd0 con 0x7f1cac071de0 2024-05-28T14:14:54.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.390+0000 7f1ca3e00640 1 --2- 172.21.15.42:0/3458591239 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c88069fa0 0x7f1c8806c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.390+0000 7f1ca3e00640 1 --2- 172.21.15.42:0/3458591239 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c88069fa0 0x7f1c8806c460 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1c940ad000 tx=0x7f1c94008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:54.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.394+0000 7f1ca9600640 1 -- 172.21.15.42:0/3458591239 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1c9c057cf0 con 0x7f1cac071de0 2024-05-28T14:14:54.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.558+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3458591239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1cac04fdd0 con 0x7f1cac071de0 2024-05-28T14:14:54.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.558+0000 7f1ca9600640 1 -- 172.21.15.42:0/3458591239 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f1c9c0575f0 con 0x7f1cac071de0 2024-05-28T14:14:54.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.562+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3458591239 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c88069fa0 msgr2=0x7f1c8806c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.562+0000 7f1cb3a00640 1 --2- 172.21.15.42:0/3458591239 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c88069fa0 0x7f1c8806c460 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1c940ad000 tx=0x7f1c94008040 comp rx=0 tx=0).stop 2024-05-28T14:14:54.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.562+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3458591239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac071de0 msgr2=0x7f1cac137e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.562+0000 7f1cb3a00640 1 --2- 172.21.15.42:0/3458591239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac071de0 0x7f1cac137e90 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00d620 tx=0x7f1c9c00dae0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.562+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3458591239 shutdown_connections 2024-05-28T14:14:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.562+0000 7f1cb3a00640 1 --2- 172.21.15.42:0/3458591239 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1c88069fa0 0x7f1c8806c460 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.562+0000 7f1cb3a00640 1 --2- 172.21.15.42:0/3458591239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cac141d10 0x7f1cac154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.562+0000 7f1cb3a00640 1 --2- 172.21.15.42:0/3458591239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cac0ce3b0 0x7f1cac1456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.562+0000 7f1cb3a00640 1 --2- 172.21.15.42:0/3458591239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cac071de0 0x7f1cac137e90 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.562+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3458591239 >> 172.21.15.42:0/3458591239 conn(0x7f1cac0b21f0 msgr2=0x7f1cac13c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.562+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3458591239 shutdown_connections 2024-05-28T14:14:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.562+0000 7f1cb3a00640 1 -- 172.21.15.42:0/3458591239 wait complete. 2024-05-28T14:14:54.569 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-05-28T14:14:54.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.726+0000 7ff922c00640 1 Processor -- start 2024-05-28T14:14:54.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.726+0000 7ff922c00640 1 -- start start 2024-05-28T14:14:54.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.726+0000 7ff922c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff91c06d110 0x7ff91c06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.726+0000 7ff922c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff91c15f710 0x7ff91c154da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.726+0000 7ff922c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff91c15fb20 con 0x7ff91c0b02e0 2024-05-28T14:14:54.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.726+0000 7ff922c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff91c06f190 con 0x7ff91c06d110 2024-05-28T14:14:54.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.726+0000 7ff922c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff91c06f470 con 0x7ff91c15f710 2024-05-28T14:14:54.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.730+0000 7ff913400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff91c06d110 0x7ff91c06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.730+0000 7ff91be00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7ff91c0b02e0 0x7ff91c06ca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:48946/0 (socket says 172.21.15.42:48946) 2024-05-28T14:14:54.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.730+0000 7ff920c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff91c15f710 0x7ff91c154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.730+0000 7ff913400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff91c06d110 0x7ff91c06eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:38686/0 (socket says 172.21.15.42:38686) 2024-05-28T14:14:54.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.730+0000 7ff913400640 1 -- 172.21.15.42:0/3237106384 learned_addr learned my addr 172.21.15.42:0/3237106384 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:54.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.730+0000 7ff913400640 1 -- 172.21.15.42:0/3237106384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff91c15f710 msgr2=0x7ff91c154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:54.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.730+0000 7ff913400640 1 --2- 172.21.15.42:0/3237106384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff91c15f710 0x7ff91c154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.730+0000 7ff913400640 1 -- 172.21.15.42:0/3237106384 >> v1:172.21.15.63:6789/0 conn(0x7ff91c0b02e0 legacy=0x7ff91c06ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.730+0000 7ff913400640 1 -- 172.21.15.42:0/3237106384 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c06f750 con 0x7ff91c06d110 2024-05-28T14:14:54.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.730+0000 7ff913400640 1 --2- 172.21.15.42:0/3237106384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c06d110 0x7ff91c06eb00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff904009870 tx=0x7ff90401c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fbcc6f9c665e1a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:54.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff91b400640 1 -- 172.21.15.42:0/3237106384 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9040aa620 con 0x7ff91c06d110 2024-05-28T14:14:54.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff91b400640 1 -- 172.21.15.42:0/3237106384 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9040af070 con 0x7ff91c06d110 2024-05-28T14:14:54.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff91b400640 1 -- 172.21.15.42:0/3237106384 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9040aa9e0 con 0x7ff91c06d110 2024-05-28T14:14:54.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 -- 172.21.15.42:0/3237106384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c06d110 msgr2=0x7ff91c06eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 --2- 172.21.15.42:0/3237106384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c06d110 0x7ff91c06eb00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff904009870 tx=0x7ff90401c530 comp rx=0 tx=0).stop 2024-05-28T14:14:54.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 -- 172.21.15.42:0/3237106384 shutdown_connections 2024-05-28T14:14:54.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 --2- 172.21.15.42:0/3237106384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff91c15f710 0x7ff91c154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 --2- 172.21.15.42:0/3237106384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c06d110 0x7ff91c06eb00 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 -- 172.21.15.42:0/3237106384 >> 172.21.15.42:0/3237106384 conn(0x7ff91c072600 msgr2=0x7ff91c072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:54.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 -- 172.21.15.42:0/3237106384 shutdown_connections 2024-05-28T14:14:54.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 -- 172.21.15.42:0/3237106384 wait complete. 2024-05-28T14:14:54.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 Processor -- start 2024-05-28T14:14:54.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 -- start start 2024-05-28T14:14:54.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff91c06d110 0x7ff91c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c0b02e0 0x7ff91c14f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff91c15f710 0x7ff91c15d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c0b4270 con 0x7ff91c06d110 2024-05-28T14:14:54.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c0b40f0 con 0x7ff91c0b02e0 2024-05-28T14:14:54.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff922c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff91c0b43f0 con 0x7ff91c15f710 2024-05-28T14:14:54.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff91be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff91c06d110 0x7ff91c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.734+0000 7ff91be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff91c06d110 0x7ff91c0ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47818/0 (socket says 172.21.15.42:47818) 2024-05-28T14:14:54.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff91be00640 1 -- 172.21.15.42:0/1563430572 learned_addr learned my addr 172.21.15.42:0/1563430572 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:54.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff913400640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c0b02e0 0x7ff91c14f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff920c00640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff91c15f710 0x7ff91c15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff91be00640 1 -- 172.21.15.42:0/1563430572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff91c15f710 msgr2=0x7ff91c15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff91be00640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff91c15f710 0x7ff91c15d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff91be00640 1 -- 172.21.15.42:0/1563430572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c0b02e0 msgr2=0x7ff91c14f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff91be00640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c0b02e0 0x7ff91c14f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff91be00640 1 -- 172.21.15.42:0/1563430572 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c0ce6a0 con 0x7ff91c06d110 2024-05-28T14:14:54.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff920c00640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff91c15f710 0x7ff91c15d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:54.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff913400640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c0b02e0 0x7ff91c14f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:54.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff91be00640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff91c06d110 0x7ff91c0ce160 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c450 tx=0x7ff90c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:54.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff918c00640 1 -- 172.21.15.42:0/1563430572 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c018070 con 0x7ff91c06d110 2024-05-28T14:14:54.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff918c00640 1 -- 172.21.15.42:0/1563430572 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff90c007480 con 0x7ff91c06d110 2024-05-28T14:14:54.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff918c00640 1 -- 172.21.15.42:0/1563430572 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0047f0 con 0x7ff91c06d110 2024-05-28T14:14:54.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff922c00640 1 -- 172.21.15.42:0/1563430572 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91c1f8970 con 0x7ff91c06d110 2024-05-28T14:14:54.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff922c00640 1 -- 172.21.15.42:0/1563430572 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff91c0ae3c0 con 0x7ff91c06d110 2024-05-28T14:14:54.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff922c00640 1 -- 172.21.15.42:0/1563430572 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff91c073f40 con 0x7ff91c06d110 2024-05-28T14:14:54.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.738+0000 7ff918c00640 1 -- 172.21.15.42:0/1563430572 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7ff90c004990 con 0x7ff91c06d110 2024-05-28T14:14:54.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.742+0000 7ff918c00640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8f8069fa0 0x7ff8f806c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:54.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.742+0000 7ff918c00640 1 -- 172.21.15.42:0/1563430572 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7ff90c084a50 con 0x7ff91c06d110 2024-05-28T14:14:54.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.742+0000 7ff913400640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8f8069fa0 0x7ff8f806c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:54.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.742+0000 7ff913400640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8f8069fa0 0x7ff8f806c460 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff9040ae000 tx=0x7ff9040a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:54.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.746+0000 7ff918c00640 1 -- 172.21.15.42:0/1563430572 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff90c056b10 con 0x7ff91c06d110 2024-05-28T14:14:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.910+0000 7ff922c00640 1 -- 172.21.15.42:0/1563430572 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7ff91c04fdd0 con 0x7ff91c06d110 2024-05-28T14:14:54.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.914+0000 7ff918c00640 1 -- 172.21.15.42:0/1563430572 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v29)=0 v29) ==== 136+0+0 (secure 0 0 0) 0x7ff90c056410 con 0x7ff91c06d110 2024-05-28T14:14:54.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.918+0000 7ff922c00640 1 -- 172.21.15.42:0/1563430572 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8f8069fa0 msgr2=0x7ff8f806c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.918+0000 7ff922c00640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8f8069fa0 0x7ff8f806c460 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff9040ae000 tx=0x7ff9040a7000 comp rx=0 tx=0).stop 2024-05-28T14:14:54.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.918+0000 7ff922c00640 1 -- 172.21.15.42:0/1563430572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff91c06d110 msgr2=0x7ff91c0ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:54.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.918+0000 7ff922c00640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff91c06d110 0x7ff91c0ce160 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c450 tx=0x7ff90c00c910 comp rx=0 tx=0).stop 2024-05-28T14:14:54.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.918+0000 7ff922c00640 1 -- 172.21.15.42:0/1563430572 shutdown_connections 2024-05-28T14:14:54.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.918+0000 7ff922c00640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8f8069fa0 0x7ff8f806c460 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.918+0000 7ff922c00640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff91c15f710 0x7ff91c15d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.918+0000 7ff922c00640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c0b02e0 0x7ff91c14f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.918+0000 7ff922c00640 1 --2- 172.21.15.42:0/1563430572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff91c06d110 0x7ff91c0ce160 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:54.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.918+0000 7ff922c00640 1 -- 172.21.15.42:0/1563430572 >> 172.21.15.42:0/1563430572 conn(0x7ff91c072600 msgr2=0x7ff91c0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:54.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.918+0000 7ff922c00640 1 -- 172.21.15.42:0/1563430572 shutdown_connections 2024-05-28T14:14:54.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:54.918+0000 7ff922c00640 1 -- 172.21.15.42:0/1563430572 wait complete. 2024-05-28T14:14:54.922 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:54.934+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:54.922 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:54.934+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:54.922 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:54.934+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:54.923 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:54.934+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:54.923 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:54.934+0000 7f0776400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:54.926 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-28T14:14:55.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.082+0000 7f446ea00640 1 Processor -- start 2024-05-28T14:14:55.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f446ea00640 1 -- start start 2024-05-28T14:14:55.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f446ea00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f44680cd6f0 0x7f44680cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f446ea00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44680b4f00 con 0x7f4468071e90 2024-05-28T14:14:55.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f446ea00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f446815af00 con 0x7f446815ab20 2024-05-28T14:14:55.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f446ea00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f446806eea0 con 0x7f44680cd6f0 2024-05-28T14:14:55.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f446e000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f4468071e90 0x7f446806f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53540/0 (socket says 172.21.15.42:53540) 2024-05-28T14:14:55.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f446e000640 1 -- 172.21.15.42:0/3946584044 learned_addr learned my addr 172.21.15.42:0/3946584044 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:55.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f446cc00640 1 --2- 172.21.15.42:0/3946584044 >> v2:172.21.15.63:3300/0 conn(0x7f44680cd6f0 0x7f44680cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f4467e00640 1 -- 172.21.15.42:0/3946584044 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3180346735 0 0) 0x7f44600014a0 con 0x7f4468071e90 2024-05-28T14:14:55.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f4467e00640 1 -- 172.21.15.42:0/3946584044 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4450003450 con 0x7f4468071e90 2024-05-28T14:14:55.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f4467e00640 1 -- 172.21.15.42:0/3946584044 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2946093189 0 0) 0x7f446815af00 con 0x7f446815ab20 2024-05-28T14:14:55.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f4467e00640 1 -- 172.21.15.42:0/3946584044 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44600014a0 con 0x7f446815ab20 2024-05-28T14:14:55.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f446cc00640 1 -- 172.21.15.42:0/3946584044 >> v1:172.21.15.63:6789/0 conn(0x7f446815ab20 legacy=0x7f44680b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f446cc00640 1 -- 172.21.15.42:0/3946584044 >> v1:172.21.15.42:6789/0 conn(0x7f4468071e90 legacy=0x7f446806f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.086+0000 7f446cc00640 1 -- 172.21.15.42:0/3946584044 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446806e290 con 0x7f44680cd6f0 2024-05-28T14:14:55.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.090+0000 7f446cc00640 1 --2- 172.21.15.42:0/3946584044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44680cd6f0 0x7f44680cdb50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f44540097c0 tx=0x7f445401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94fc31f01e76a3bd server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:55.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.090+0000 7f4467e00640 1 -- 172.21.15.42:0/3946584044 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44540b0070 con 0x7f44680cd6f0 2024-05-28T14:14:55.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.090+0000 7f4467e00640 1 -- 172.21.15.42:0/3946584044 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44540ab600 con 0x7f44680cd6f0 2024-05-28T14:14:55.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.090+0000 7f4467e00640 1 -- 172.21.15.42:0/3946584044 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44540ab9d0 con 0x7f44680cd6f0 2024-05-28T14:14:55.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.090+0000 7f446ea00640 1 -- 172.21.15.42:0/3946584044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44680cd6f0 msgr2=0x7f44680cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.090+0000 7f446ea00640 1 --2- 172.21.15.42:0/3946584044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44680cd6f0 0x7f44680cdb50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f44540097c0 tx=0x7f445401c410 comp rx=0 tx=0).stop 2024-05-28T14:14:55.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.090+0000 7f446ea00640 1 -- 172.21.15.42:0/3946584044 shutdown_connections 2024-05-28T14:14:55.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.090+0000 7f446ea00640 1 --2- 172.21.15.42:0/3946584044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44680cd6f0 0x7f44680cdb50 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.090+0000 7f446ea00640 1 -- 172.21.15.42:0/3946584044 >> 172.21.15.42:0/3946584044 conn(0x7f4468067b60 msgr2=0x7f446806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:55.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.090+0000 7f446ea00640 1 -- 172.21.15.42:0/3946584044 shutdown_connections 2024-05-28T14:14:55.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.090+0000 7f446ea00640 1 -- 172.21.15.42:0/3946584044 wait complete. 2024-05-28T14:14:55.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.090+0000 7f446ea00640 1 Processor -- start 2024-05-28T14:14:55.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.090+0000 7f446ea00640 1 -- start start 2024-05-28T14:14:55.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4468071e90 0x7f4468141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44680cd6f0 0x7f446814fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f446815ab20 0x7f44680cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446ea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f44680726b0 con 0x7f446815ab20 2024-05-28T14:14:55.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446ea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f446806deb0 con 0x7f44680cd6f0 2024-05-28T14:14:55.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446ea00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4468150350 con 0x7f4468071e90 2024-05-28T14:14:55.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f446815ab20 0x7f44680cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f446815ab20 0x7f44680cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47856/0 (socket says 172.21.15.42:47856) 2024-05-28T14:14:55.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446e000640 1 -- 172.21.15.42:0/441861843 learned_addr learned my addr 172.21.15.42:0/441861843 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:55.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446cc00640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44680cd6f0 0x7f446814fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446d600640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4468071e90 0x7f4468141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446e000640 1 -- 172.21.15.42:0/441861843 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4468071e90 msgr2=0x7f4468141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446e000640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4468071e90 0x7f4468141060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446e000640 1 -- 172.21.15.42:0/441861843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44680cd6f0 msgr2=0x7f446814fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446e000640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44680cd6f0 0x7f446814fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446e000640 1 -- 172.21.15.42:0/441861843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4468150630 con 0x7f446815ab20 2024-05-28T14:14:55.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446d600640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4468071e90 0x7f4468141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:14:55.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446e000640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f446815ab20 0x7f44680cf170 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4460001fa0 tx=0x7f446000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:55.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f4465600640 1 -- 172.21.15.42:0/441861843 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460017070 con 0x7f446815ab20 2024-05-28T14:14:55.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f4465600640 1 -- 172.21.15.42:0/441861843 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4460012400 con 0x7f446815ab20 2024-05-28T14:14:55.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f4465600640 1 -- 172.21.15.42:0/441861843 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44600127b0 con 0x7f446815ab20 2024-05-28T14:14:55.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446ea00640 1 -- 172.21.15.42:0/441861843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4468150910 con 0x7f446815ab20 2024-05-28T14:14:55.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446ea00640 1 -- 172.21.15.42:0/441861843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446815de70 con 0x7f446815ab20 2024-05-28T14:14:55.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.094+0000 7f446ea00640 1 -- 172.21.15.42:0/441861843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4468150ae0 con 0x7f446815ab20 2024-05-28T14:14:55.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.098+0000 7f4465600640 1 -- 172.21.15.42:0/441861843 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f446001a070 con 0x7f446815ab20 2024-05-28T14:14:55.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.098+0000 7f4465600640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4448069fa0 0x7f444806c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.098+0000 7f4465600640 1 -- 172.21.15.42:0/441861843 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f4460085840 con 0x7f446815ab20 2024-05-28T14:14:55.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.098+0000 7f446d600640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4448069fa0 0x7f444806c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.098+0000 7f446d600640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4448069fa0 0x7f444806c460 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f445c0011f0 tx=0x7f445c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:55.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.102+0000 7f4465600640 1 -- 172.21.15.42:0/441861843 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f44680b4f00 con 0x7f446815ab20 2024-05-28T14:14:55.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.290+0000 7f446ea00640 1 -- 172.21.15.42:0/441861843 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f4468068a90 con 0x7f4448069fa0 2024-05-28T14:14:55.285 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:55.298+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:55.286 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:55.298+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:55.286 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:55.298+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:55.286 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:55.298+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:55.286 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:14:55.298+0000 7f0774600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:14:55.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.338+0000 7f4465600640 1 -- 172.21.15.42:0/441861843 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4468068a90 con 0x7f4448069fa0 2024-05-28T14:14:55.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.338+0000 7f446ea00640 1 -- 172.21.15.42:0/441861843 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4448069fa0 msgr2=0x7f444806c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.338+0000 7f446ea00640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4448069fa0 0x7f444806c460 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f445c0011f0 tx=0x7f445c009040 comp rx=0 tx=0).stop 2024-05-28T14:14:55.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.338+0000 7f446ea00640 1 -- 172.21.15.42:0/441861843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f446815ab20 msgr2=0x7f44680cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.338+0000 7f446ea00640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f446815ab20 0x7f44680cf170 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4460001fa0 tx=0x7f446000a430 comp rx=0 tx=0).stop 2024-05-28T14:14:55.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.338+0000 7f446ea00640 1 -- 172.21.15.42:0/441861843 shutdown_connections 2024-05-28T14:14:55.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.338+0000 7f446ea00640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f4448069fa0 0x7f444806c460 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.338+0000 7f446ea00640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f446815ab20 0x7f44680cf170 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.338+0000 7f446ea00640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44680cd6f0 0x7f446814fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.338+0000 7f446ea00640 1 --2- 172.21.15.42:0/441861843 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4468071e90 0x7f4468141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.338+0000 7f446ea00640 1 -- 172.21.15.42:0/441861843 >> 172.21.15.42:0/441861843 conn(0x7f4468067b60 msgr2=0x7f44680ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:55.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.338+0000 7f446ea00640 1 -- 172.21.15.42:0/441861843 shutdown_connections 2024-05-28T14:14:55.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.338+0000 7f446ea00640 1 -- 172.21.15.42:0/441861843 wait complete. 2024-05-28T14:14:55.346 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:14:55.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.502+0000 7f2d60e00640 1 Processor -- start 2024-05-28T14:14:55.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.502+0000 7f2d60e00640 1 -- start start 2024-05-28T14:14:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d60e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2d5c0cfec0 0x7f2d5c0d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d60e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d5c143f90 0x7f2d5c071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d60e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2d5c147a80 0x7f2d5c071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d60e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f2d5c06cb00 con 0x7f2d5c147a80 2024-05-28T14:14:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d60e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2d5c0d07e0 con 0x7f2d5c0cfec0 2024-05-28T14:14:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d60e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2d5c144540 con 0x7f2d5c143f90 2024-05-28T14:14:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d5be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2d5c147a80 0x7f2d5c071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d5be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2d5c147a80 0x7f2d5c071c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47892/0 (socket says 172.21.15.42:47892) 2024-05-28T14:14:55.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d5be00640 1 -- 172.21.15.42:0/3111665600 learned_addr learned my addr 172.21.15.42:0/3111665600 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:55.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d5b400640 1 --2- 172.21.15.42:0/3111665600 >> v2:172.21.15.63:3300/0 conn(0x7f2d5c0cfec0 0x7f2d5c0d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d52a00640 1 --2- 172.21.15.42:0/3111665600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d5c143f90 0x7f2d5c071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d5be00640 1 -- 172.21.15.42:0/3111665600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d5c143f90 msgr2=0x7f2d5c071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:55.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d5be00640 1 --2- 172.21.15.42:0/3111665600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d5c143f90 0x7f2d5c071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d5be00640 1 -- 172.21.15.42:0/3111665600 >> v2:172.21.15.63:3300/0 conn(0x7f2d5c0cfec0 msgr2=0x7f2d5c0d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:55.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d5be00640 1 --2- 172.21.15.42:0/3111665600 >> v2:172.21.15.63:3300/0 conn(0x7f2d5c0cfec0 0x7f2d5c0d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.506+0000 7f2d5be00640 1 -- 172.21.15.42:0/3111665600 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c144820 con 0x7f2d5c147a80 2024-05-28T14:14:55.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d5be00640 1 --2- 172.21.15.42:0/3111665600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d5c147a80 0x7f2d5c071c50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0d2250 tx=0x7f2d5401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721e3f2577102ea4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d5aa00640 1 -- 172.21.15.42:0/3111665600 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d540b0070 con 0x7f2d5c147a80 2024-05-28T14:14:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d5aa00640 1 -- 172.21.15.42:0/3111665600 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d540ab600 con 0x7f2d5c147a80 2024-05-28T14:14:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d5aa00640 1 -- 172.21.15.42:0/3111665600 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d540ab9d0 con 0x7f2d5c147a80 2024-05-28T14:14:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 -- 172.21.15.42:0/3111665600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d5c147a80 msgr2=0x7f2d5c071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 --2- 172.21.15.42:0/3111665600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d5c147a80 0x7f2d5c071c50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0d2250 tx=0x7f2d5401c3d0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 -- 172.21.15.42:0/3111665600 shutdown_connections 2024-05-28T14:14:55.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 --2- 172.21.15.42:0/3111665600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d5c147a80 0x7f2d5c071c50 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 --2- 172.21.15.42:0/3111665600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d5c143f90 0x7f2d5c071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 --2- 172.21.15.42:0/3111665600 >> v2:172.21.15.63:3300/0 conn(0x7f2d5c0cfec0 0x7f2d5c0d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 -- 172.21.15.42:0/3111665600 >> 172.21.15.42:0/3111665600 conn(0x7f2d5c06fcc0 msgr2=0x7f2d5c0700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:55.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 -- 172.21.15.42:0/3111665600 shutdown_connections 2024-05-28T14:14:55.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 -- 172.21.15.42:0/3111665600 wait complete. 2024-05-28T14:14:55.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 Processor -- start 2024-05-28T14:14:55.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 -- start start 2024-05-28T14:14:55.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d5c0cfec0 0x7f2d5c158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d5c143f90 0x7f2d5c0c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d5c147a80 0x7f2d5c0c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c161e60 con 0x7f2d5c0cfec0 2024-05-28T14:14:55.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c0725a0 con 0x7f2d5c143f90 2024-05-28T14:14:55.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d60e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2d5c161fe0 con 0x7f2d5c147a80 2024-05-28T14:14:55.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d5b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d5c0cfec0 0x7f2d5c158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d5b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d5c0cfec0 0x7f2d5c158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47908/0 (socket says 172.21.15.42:47908) 2024-05-28T14:14:55.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d5b400640 1 -- 172.21.15.42:0/346853475 learned_addr learned my addr 172.21.15.42:0/346853475 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:55.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.514+0000 7f2d52a00640 1 --2- 172.21.15.42:0/346853475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d5c143f90 0x7f2d5c0c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.510+0000 7f2d5be00640 1 --2- 172.21.15.42:0/346853475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d5c147a80 0x7f2d5c0c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.514+0000 7f2d5b400640 1 -- 172.21.15.42:0/346853475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d5c147a80 msgr2=0x7f2d5c0c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.514+0000 7f2d5b400640 1 --2- 172.21.15.42:0/346853475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d5c147a80 0x7f2d5c0c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.514+0000 7f2d5b400640 1 -- 172.21.15.42:0/346853475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d5c143f90 msgr2=0x7f2d5c0c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.514+0000 7f2d5b400640 1 --2- 172.21.15.42:0/346853475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d5c143f90 0x7f2d5c0c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.514+0000 7f2d5b400640 1 -- 172.21.15.42:0/346853475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c161430 con 0x7f2d5c0cfec0 2024-05-28T14:14:55.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.514+0000 7f2d5b400640 1 --2- 172.21.15.42:0/346853475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d5c0cfec0 0x7f2d5c158ca0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c007e40 tx=0x7f2d4c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:55.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.514+0000 7f2d53e00640 1 -- 172.21.15.42:0/346853475 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c014620 con 0x7f2d5c0cfec0 2024-05-28T14:14:55.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.514+0000 7f2d53e00640 1 -- 172.21.15.42:0/346853475 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d4c019070 con 0x7f2d5c0cfec0 2024-05-28T14:14:55.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.514+0000 7f2d60e00640 1 -- 172.21.15.42:0/346853475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d5c159280 con 0x7f2d5c0cfec0 2024-05-28T14:14:55.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.514+0000 7f2d53e00640 1 -- 172.21.15.42:0/346853475 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c014a00 con 0x7f2d5c0cfec0 2024-05-28T14:14:55.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.514+0000 7f2d60e00640 1 -- 172.21.15.42:0/346853475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d5c159840 con 0x7f2d5c0cfec0 2024-05-28T14:14:55.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.514+0000 7f2d60e00640 1 -- 172.21.15.42:0/346853475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d5c04fcf0 con 0x7f2d5c0cfec0 2024-05-28T14:14:55.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.514+0000 7f2d53e00640 1 -- 172.21.15.42:0/346853475 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f2d4c014ba0 con 0x7f2d5c0cfec0 2024-05-28T14:14:55.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.518+0000 7f2d53e00640 1 --2- 172.21.15.42:0/346853475 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2d38069fa0 0x7f2d3806c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.518+0000 7f2d53e00640 1 -- 172.21.15.42:0/346853475 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f2d4c0860a0 con 0x7f2d5c0cfec0 2024-05-28T14:14:55.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.518+0000 7f2d52a00640 1 --2- 172.21.15.42:0/346853475 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2d38069fa0 0x7f2d3806c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.518+0000 7f2d52a00640 1 --2- 172.21.15.42:0/346853475 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2d38069fa0 0x7f2d3806c460 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0c5c90 tx=0x7f2d44008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:55.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.522+0000 7f2d53e00640 1 -- 172.21.15.42:0/346853475 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2d4c057a70 con 0x7f2d5c0cfec0 2024-05-28T14:14:55.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.714+0000 7f2d60e00640 1 -- 172.21.15.42:0/346853475 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f2d5c0c4450 con 0x7f2d38069fa0 2024-05-28T14:14:55.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.718+0000 7f2d53e00640 1 -- 172.21.15.42:0/346853475 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f2d5c0c4450 con 0x7f2d38069fa0 2024-05-28T14:14:55.704 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/2eaf73b2-6b4e-4b8f-acd8-93d1b171c401 2024-05-28T14:14:55.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.718+0000 7f2d60e00640 1 -- 172.21.15.42:0/346853475 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2d38069fa0 msgr2=0x7f2d3806c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.718+0000 7f2d60e00640 1 --2- 172.21.15.42:0/346853475 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2d38069fa0 0x7f2d3806c460 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0c5c90 tx=0x7f2d44008040 comp rx=0 tx=0).stop 2024-05-28T14:14:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.718+0000 7f2d60e00640 1 -- 172.21.15.42:0/346853475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d5c0cfec0 msgr2=0x7f2d5c158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.718+0000 7f2d60e00640 1 --2- 172.21.15.42:0/346853475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d5c0cfec0 0x7f2d5c158ca0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c007e40 tx=0x7f2d4c00e4b0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.718+0000 7f2d60e00640 1 -- 172.21.15.42:0/346853475 shutdown_connections 2024-05-28T14:14:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.718+0000 7f2d60e00640 1 --2- 172.21.15.42:0/346853475 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2d38069fa0 0x7f2d3806c460 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.718+0000 7f2d60e00640 1 --2- 172.21.15.42:0/346853475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d5c147a80 0x7f2d5c0c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.718+0000 7f2d60e00640 1 --2- 172.21.15.42:0/346853475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d5c143f90 0x7f2d5c0c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.718+0000 7f2d60e00640 1 --2- 172.21.15.42:0/346853475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d5c0cfec0 0x7f2d5c158ca0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.718+0000 7f2d60e00640 1 -- 172.21.15.42:0/346853475 >> 172.21.15.42:0/346853475 conn(0x7f2d5c06fcc0 msgr2=0x7f2d5c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.718+0000 7f2d60e00640 1 -- 172.21.15.42:0/346853475 shutdown_connections 2024-05-28T14:14:55.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.718+0000 7f2d60e00640 1 -- 172.21.15.42:0/346853475 wait complete. 2024-05-28T14:14:55.726 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:14:55.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.882+0000 7f0e14800640 1 Processor -- start 2024-05-28T14:14:55.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.882+0000 7f0e14800640 1 -- start start 2024-05-28T14:14:55.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e14800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c144f80 0x7f0e0c072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e14800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0e0c15c0c0 0x7f0e0c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e14800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e0c0afe40 con 0x7f0e0c06ee20 2024-05-28T14:14:55.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e14800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0e0c06f7c0 con 0x7f0e0c15c0c0 2024-05-28T14:14:55.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e14800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0c0724d0 con 0x7f0e0c144f80 2024-05-28T14:14:55.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e11400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0e0c06ee20 0x7f0e0c15b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53554/0 (socket says 172.21.15.42:53554) 2024-05-28T14:14:55.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e11400640 1 -- 172.21.15.42:0/217591507 learned_addr learned my addr 172.21.15.42:0/217591507 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:14:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e11e00640 1 --2- 172.21.15.42:0/217591507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c144f80 0x7f0e0c072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e12800640 1 --2- 172.21.15.42:0/217591507 >> v2:172.21.15.63:3300/0 conn(0x7f0e0c15c0c0 0x7f0e0c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 -- 172.21.15.42:0/217591507 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1022184943 0 0) 0x7f0e0c0afe40 con 0x7f0e0c06ee20 2024-05-28T14:14:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 -- 172.21.15.42:0/217591507 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0df4003530 con 0x7f0e0c06ee20 2024-05-28T14:14:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 -- 172.21.15.42:0/217591507 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3384752730 0 0) 0x7f0df4003530 con 0x7f0e0c06ee20 2024-05-28T14:14:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 -- 172.21.15.42:0/217591507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c144f80 msgr2=0x7f0e0c072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 --2- 172.21.15.42:0/217591507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c144f80 0x7f0e0c072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 -- 172.21.15.42:0/217591507 >> v2:172.21.15.63:3300/0 conn(0x7f0e0c15c0c0 msgr2=0x7f0e0c06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 --2- 172.21.15.42:0/217591507 >> v2:172.21.15.63:3300/0 conn(0x7f0e0c15c0c0 0x7f0e0c06f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 -- 172.21.15.42:0/217591507 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c15c600 con 0x7f0e0c06ee20 2024-05-28T14:14:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 -- 172.21.15.42:0/217591507 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f0dfc001680 con 0x7f0e0c06ee20 2024-05-28T14:14:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e11e00640 1 --2- 172.21.15.42:0/217591507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c144f80 0x7f0e0c072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:14:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e12800640 1 --2- 172.21.15.42:0/217591507 >> v2:172.21.15.63:3300/0 conn(0x7f0e0c15c0c0 0x7f0e0c06f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:14:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 -- 172.21.15.42:0/217591507 >> v1:172.21.15.42:6789/0 conn(0x7f0e0c06ee20 legacy=0x7f0e0c15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 --2- 172.21.15.42:0/217591507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0df40039c0 0x7f0df4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 --2- 172.21.15.42:0/217591507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c144f80 0x7f0df4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 --2- 172.21.15.42:0/217591507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df4004d50 0x7f0df40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 -- 172.21.15.42:0/217591507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c0724d0 con 0x7f0df4004d50 2024-05-28T14:14:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 -- 172.21.15.42:0/217591507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0df400dbe0 con 0x7f0df40039c0 2024-05-28T14:14:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e10a00640 1 -- 172.21.15.42:0/217591507 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0df400dd60 con 0x7f0e0c144f80 2024-05-28T14:14:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e12800640 1 --2- 172.21.15.42:0/217591507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df4004d50 0x7f0df40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e11e00640 1 --2- 172.21.15.42:0/217591507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0df40039c0 0x7f0df4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e11400640 1 --2- 172.21.15.42:0/217591507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c144f80 0x7f0df4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e12800640 1 -- 172.21.15.42:0/217591507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c144f80 msgr2=0x7f0df4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e12800640 1 --2- 172.21.15.42:0/217591507 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c144f80 0x7f0df4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e12800640 1 -- 172.21.15.42:0/217591507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0df40039c0 msgr2=0x7f0df4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:14:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e12800640 1 --2- 172.21.15.42:0/217591507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0df40039c0 0x7f0df4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.886+0000 7f0e12800640 1 -- 172.21.15.42:0/217591507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df4003530 con 0x7f0df4004d50 2024-05-28T14:14:55.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.890+0000 7f0e12800640 1 --2- 172.21.15.42:0/217591507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df4004d50 0x7f0df40051d0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800e320 tx=0x7f0e0801ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eccb3b5c723ce82 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:55.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.890+0000 7f0e10a00640 1 -- 172.21.15.42:0/217591507 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e080b2070 con 0x7f0df4004d50 2024-05-28T14:14:55.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.890+0000 7f0e10a00640 1 -- 172.21.15.42:0/217591507 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e080a5e50 con 0x7f0df4004d50 2024-05-28T14:14:55.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.890+0000 7f0e10a00640 1 -- 172.21.15.42:0/217591507 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08009de0 con 0x7f0df4004d50 2024-05-28T14:14:55.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.890+0000 7f0e14800640 1 -- 172.21.15.42:0/217591507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df4004d50 msgr2=0x7f0df40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.890+0000 7f0e14800640 1 --2- 172.21.15.42:0/217591507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df4004d50 0x7f0df40051d0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800e320 tx=0x7f0e0801ce40 comp rx=0 tx=0).stop 2024-05-28T14:14:55.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.890+0000 7f0e12800640 1 -- 172.21.15.42:0/217591507 reap_dead start 2024-05-28T14:14:55.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.890+0000 7f0e14800640 1 -- 172.21.15.42:0/217591507 shutdown_connections 2024-05-28T14:14:55.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.890+0000 7f0e14800640 1 -- 172.21.15.42:0/217591507 >> 172.21.15.42:0/217591507 conn(0x7f0e0c0b21f0 msgr2=0x7f0e0c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:55.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.890+0000 7f0e14800640 1 -- 172.21.15.42:0/217591507 shutdown_connections 2024-05-28T14:14:55.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.890+0000 7f0e14800640 1 -- 172.21.15.42:0/217591507 wait complete. 2024-05-28T14:14:55.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.890+0000 7f0e14800640 1 Processor -- start 2024-05-28T14:14:55.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e14800640 1 -- start start 2024-05-28T14:14:55.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e14800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df4004d50 0x7f0e0c15a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e14800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c140cd0 0x7f0e0c0b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e14800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e0c141130 0x7f0e0c0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e14800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c072160 con 0x7f0df4004d50 2024-05-28T14:14:55.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e14800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c071e80 con 0x7f0e0c141130 2024-05-28T14:14:55.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e14800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0c072960 con 0x7f0e0c140cd0 2024-05-28T14:14:55.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e11e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df4004d50 0x7f0e0c15a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e11e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df4004d50 0x7f0e0c15a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47918/0 (socket says 172.21.15.42:47918) 2024-05-28T14:14:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e11e00640 1 -- 172.21.15.42:0/2095109145 learned_addr learned my addr 172.21.15.42:0/2095109145 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:14:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e11e00640 1 -- 172.21.15.42:0/2095109145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c140cd0 msgr2=0x7f0e0c0b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e11400640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c140cd0 0x7f0e0c0b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e12800640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e0c141130 0x7f0e0c0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e11e00640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c140cd0 0x7f0e0c0b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e11e00640 1 -- 172.21.15.42:0/2095109145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e0c141130 msgr2=0x7f0e0c0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e11e00640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e0c141130 0x7f0e0c0b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e11e00640 1 -- 172.21.15.42:0/2095109145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c0af240 con 0x7f0df4004d50 2024-05-28T14:14:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e11400640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c140cd0 0x7f0e0c0b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e12800640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e0c141130 0x7f0e0c0b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:14:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e11e00640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df4004d50 0x7f0e0c15a9d0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f0e000119d0 tx=0x7f0e00011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e06000640 1 -- 172.21.15.42:0/2095109145 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000fb50 con 0x7f0df4004d50 2024-05-28T14:14:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e06000640 1 -- 172.21.15.42:0/2095109145 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0000fcf0 con 0x7f0df4004d50 2024-05-28T14:14:55.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e14800640 1 -- 172.21.15.42:0/2095109145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0c13e000 con 0x7f0df4004d50 2024-05-28T14:14:55.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e06000640 1 -- 172.21.15.42:0/2095109145 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000fea0 con 0x7f0df4004d50 2024-05-28T14:14:55.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e14800640 1 -- 172.21.15.42:0/2095109145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0c13e5c0 con 0x7f0df4004d50 2024-05-28T14:14:55.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.894+0000 7f0e14800640 1 -- 172.21.15.42:0/2095109145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e0c0738b0 con 0x7f0df4004d50 2024-05-28T14:14:55.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.898+0000 7f0e06000640 1 -- 172.21.15.42:0/2095109145 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 90537+0+0 (secure 0 0 0) 0x7f0e00012040 con 0x7f0df4004d50 2024-05-28T14:14:55.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.898+0000 7f0e06000640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0df0069fa0 0x7f0df006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:14:55.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.898+0000 7f0e06000640 1 -- 172.21.15.42:0/2095109145 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5765+0+0 (secure 0 0 0) 0x7f0e00085cf0 con 0x7f0df4004d50 2024-05-28T14:14:55.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.898+0000 7f0e11400640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0df0069fa0 0x7f0df006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:14:55.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.898+0000 7f0e11400640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0df0069fa0 0x7f0df006c460 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c13b4e0 tx=0x7f0dfc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:14:55.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.902+0000 7f0e06000640 1 -- 172.21.15.42:0/2095109145 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0e00019070 con 0x7f0df4004d50 2024-05-28T14:14:55.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:55.926+0000 7f0e06000640 1 -- 172.21.15.42:0/2095109145 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 43) ==== 90588+0+0 (secure 0 0 0) 0x7f0e00053d20 con 0x7f0df4004d50 2024-05-28T14:14:56.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:56.098+0000 7f0e14800640 1 -- 172.21.15.42:0/2095109145 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f0e0c13f280 con 0x7f0df0069fa0 2024-05-28T14:14:56.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:56.102+0000 7f0e06000640 1 -- 172.21.15.42:0/2095109145 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f0e0c13f280 con 0x7f0df0069fa0 2024-05-28T14:14:56.089 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/2eaf73b2-6b4e-4b8f-acd8-93d1b171c401 2024-05-28T14:14:56.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:56.106+0000 7f0e14800640 1 -- 172.21.15.42:0/2095109145 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0df0069fa0 msgr2=0x7f0df006c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:56.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:56.106+0000 7f0e14800640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0df0069fa0 0x7f0df006c460 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c13b4e0 tx=0x7f0dfc009040 comp rx=0 tx=0).stop 2024-05-28T14:14:56.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:56.106+0000 7f0e14800640 1 -- 172.21.15.42:0/2095109145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df4004d50 msgr2=0x7f0e0c15a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:14:56.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:56.106+0000 7f0e14800640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df4004d50 0x7f0e0c15a9d0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f0e000119d0 tx=0x7f0e00011e90 comp rx=0 tx=0).stop 2024-05-28T14:14:56.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:56.106+0000 7f0e14800640 1 -- 172.21.15.42:0/2095109145 shutdown_connections 2024-05-28T14:14:56.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:56.106+0000 7f0e14800640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0df0069fa0 0x7f0df006c460 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:56.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:56.106+0000 7f0e14800640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e0c141130 0x7f0e0c0b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:56.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:56.106+0000 7f0e14800640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e0c140cd0 0x7f0e0c0b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:56.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:56.106+0000 7f0e14800640 1 --2- 172.21.15.42:0/2095109145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df4004d50 0x7f0e0c15a9d0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:14:56.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:56.106+0000 7f0e14800640 1 -- 172.21.15.42:0/2095109145 >> 172.21.15.42:0/2095109145 conn(0x7f0e0c0b21f0 msgr2=0x7f0e0c0b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:14:56.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:56.106+0000 7f0e14800640 1 -- 172.21.15.42:0/2095109145 shutdown_connections 2024-05-28T14:14:56.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:14:56.106+0000 7f0e14800640 1 -- 172.21.15.42:0/2095109145 wait complete. 2024-05-28T14:14:56.112 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/2eaf73b2-6b4e-4b8f-acd8-93d1b171c401 2024-05-28T14:14:56.112 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:14:56.112 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:14:56.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:14:56.121 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/2eaf73b2-6b4e-4b8f-acd8-93d1b171c401/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:15:03.284 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:15:03.284 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:15:03.284 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.09871 s, 151 MB/s 2024-05-28T14:15:03.285 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:15:03.286 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:15:03.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:15:03.294 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/2eaf73b2-6b4e-4b8f-acd8-93d1b171c401/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:15:10.246 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:15:10.246 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:15:10.246 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.89873 s, 156 MB/s 2024-05-28T14:15:10.248 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:15:10.248 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:15:10.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:15:10.258 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/2eaf73b2-6b4e-4b8f-acd8-93d1b171c401/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-28T14:15:17.263 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records in 2024-05-28T14:15:17.263 INFO:teuthology.orchestra.run.smithi042.stderr:1024+0 records out 2024-05-28T14:15:17.263 INFO:teuthology.orchestra.run.smithi042.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.95059 s, 154 MB/s 2024-05-28T14:15:17.264 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-28T14:15:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f9000640 1 Processor -- start 2024-05-28T14:15:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f9000640 1 -- start start 2024-05-28T14:15:17.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f9000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f75f415c530 0x7f75f40ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:17.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f9000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f75f40b20a0 0x7f75f40b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:17.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f9000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75f406c890 con 0x7f75f406c400 2024-05-28T14:15:17.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f9000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f75f40b29f0 con 0x7f75f415c530 2024-05-28T14:15:17.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f9000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f75f40cd4b0 con 0x7f75f40b20a0 2024-05-28T14:15:17.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f75f415c530 0x7f75f40ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:17.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f75f415c530 0x7f75f40ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55378/0 (socket says 172.21.15.42:55378) 2024-05-28T14:15:17.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f3400640 1 -- 172.21.15.42:0/729028710 learned_addr learned my addr 172.21.15.42:0/729028710 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:17.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f3e00640 1 --2- 172.21.15.42:0/729028710 >> v2:172.21.15.63:3300/0 conn(0x7f75f40b20a0 0x7f75f40b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:17.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f3400640 1 -- 172.21.15.42:0/729028710 >> v2:172.21.15.63:3300/0 conn(0x7f75f40b20a0 msgr2=0x7f75f40b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:17.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f3400640 1 --2- 172.21.15.42:0/729028710 >> v2:172.21.15.63:3300/0 conn(0x7f75f40b20a0 0x7f75f40b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f3400640 1 -- 172.21.15.42:0/729028710 >> v1:172.21.15.42:6789/0 conn(0x7f75f406c400 legacy=0x7f75f4160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:17.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f3400640 1 -- 172.21.15.42:0/729028710 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f40cd790 con 0x7f75f415c530 2024-05-28T14:15:17.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.426+0000 7f75f3e00640 1 --2- 172.21.15.42:0/729028710 >> v2:172.21.15.63:3300/0 conn(0x7f75f40b20a0 0x7f75f40b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:15:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.430+0000 7f75f3400640 1 --2- 172.21.15.42:0/729028710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f415c530 0x7f75f40ccdd0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f75f40c8310 tx=0x7f75e401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea66552c1f593be server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.430+0000 7f75f2000640 1 -- 172.21.15.42:0/729028710 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e40ab610 con 0x7f75f415c530 2024-05-28T14:15:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.430+0000 7f75f2000640 1 -- 172.21.15.42:0/729028710 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75e40b0070 con 0x7f75f415c530 2024-05-28T14:15:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.430+0000 7f75f2000640 1 -- 172.21.15.42:0/729028710 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e40ab9d0 con 0x7f75f415c530 2024-05-28T14:15:17.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.430+0000 7f75f9000640 1 -- 172.21.15.42:0/729028710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f415c530 msgr2=0x7f75f40ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:17.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.430+0000 7f75f9000640 1 --2- 172.21.15.42:0/729028710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f415c530 0x7f75f40ccdd0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f75f40c8310 tx=0x7f75e401c520 comp rx=0 tx=0).stop 2024-05-28T14:15:17.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.430+0000 7f75f9000640 1 -- 172.21.15.42:0/729028710 shutdown_connections 2024-05-28T14:15:17.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.430+0000 7f75f9000640 1 --2- 172.21.15.42:0/729028710 >> v2:172.21.15.63:3300/0 conn(0x7f75f40b20a0 0x7f75f40b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.430+0000 7f75f9000640 1 --2- 172.21.15.42:0/729028710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f415c530 0x7f75f40ccdd0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.430+0000 7f75f9000640 1 -- 172.21.15.42:0/729028710 >> 172.21.15.42:0/729028710 conn(0x7f75f409aa50 msgr2=0x7f75f40b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:17.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.430+0000 7f75f9000640 1 -- 172.21.15.42:0/729028710 shutdown_connections 2024-05-28T14:15:17.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.430+0000 7f75f9000640 1 -- 172.21.15.42:0/729028710 wait complete. 2024-05-28T14:15:17.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f9000640 1 Processor -- start 2024-05-28T14:15:17.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f9000640 1 -- start start 2024-05-28T14:15:17.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f9000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f406c400 0x7f75f4152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:17.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f9000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f40b20a0 0x7f75f415df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:17.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f9000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75f415c530 0x7f75f40c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:17.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f9000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f75f415ca90 con 0x7f75f40b20a0 2024-05-28T14:15:17.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f9000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f75f415c910 con 0x7f75f406c400 2024-05-28T14:15:17.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f9000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f75f40b4910 con 0x7f75f415c530 2024-05-28T14:15:17.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f40b20a0 0x7f75f415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:17.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f40b20a0 0x7f75f415df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55386/0 (socket says 172.21.15.42:55386) 2024-05-28T14:15:17.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f2a00640 1 -- 172.21.15.42:0/1616896756 learned_addr learned my addr 172.21.15.42:0/1616896756 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:17.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f3400640 1 --2- 172.21.15.42:0/1616896756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f406c400 0x7f75f4152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:17.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f3e00640 1 --2- 172.21.15.42:0/1616896756 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75f415c530 0x7f75f40c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:17.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f2a00640 1 -- 172.21.15.42:0/1616896756 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75f415c530 msgr2=0x7f75f40c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:17.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f2a00640 1 --2- 172.21.15.42:0/1616896756 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75f415c530 0x7f75f40c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f2a00640 1 -- 172.21.15.42:0/1616896756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f406c400 msgr2=0x7f75f4152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:17.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f2a00640 1 --2- 172.21.15.42:0/1616896756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f406c400 0x7f75f4152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f2a00640 1 -- 172.21.15.42:0/1616896756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f4153560 con 0x7f75f40b20a0 2024-05-28T14:15:17.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f2a00640 1 --2- 172.21.15.42:0/1616896756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f40b20a0 0x7f75f415df10 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f75e000c420 tx=0x7f75e000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:17.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75eb400640 1 -- 172.21.15.42:0/1616896756 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e00052d0 con 0x7f75f40b20a0 2024-05-28T14:15:17.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75eb400640 1 -- 172.21.15.42:0/1616896756 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75e0005470 con 0x7f75f40b20a0 2024-05-28T14:15:17.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75eb400640 1 -- 172.21.15.42:0/1616896756 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0007260 con 0x7f75f40b20a0 2024-05-28T14:15:17.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f9000640 1 -- 172.21.15.42:0/1616896756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f4153840 con 0x7f75f40b20a0 2024-05-28T14:15:17.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f9000640 1 -- 172.21.15.42:0/1616896756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f4153e00 con 0x7f75f40b20a0 2024-05-28T14:15:17.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.434+0000 7f75f9000640 1 -- 172.21.15.42:0/1616896756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75f4068a90 con 0x7f75f40b20a0 2024-05-28T14:15:17.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.438+0000 7f75eb400640 1 -- 172.21.15.42:0/1616896756 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 44) ==== 90639+0+0 (secure 0 0 0) 0x7f75e0007400 con 0x7f75f40b20a0 2024-05-28T14:15:17.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.438+0000 7f75eb400640 1 --2- 172.21.15.42:0/1616896756 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f75d406a0c0 0x7f75d406c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:17.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.438+0000 7f75eb400640 1 -- 172.21.15.42:0/1616896756 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f75e0084e20 con 0x7f75f40b20a0 2024-05-28T14:15:17.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.438+0000 7f75f3400640 1 --2- 172.21.15.42:0/1616896756 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f75d406a0c0 0x7f75d406c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:17.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.438+0000 7f75f3400640 1 --2- 172.21.15.42:0/1616896756 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f75d406a0c0 0x7f75d406c580 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f75e40af000 tx=0x7f75e40a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:17.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.442+0000 7f75eb400640 1 -- 172.21.15.42:0/1616896756 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f75e00567f0 con 0x7f75f40b20a0 2024-05-28T14:15:17.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.634+0000 7f75f9000640 1 -- 172.21.15.42:0/1616896756 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f75f40b7230 con 0x7f75d406a0c0 2024-05-28T14:15:17.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.646+0000 7f75eb400640 1 -- 172.21.15.42:0/1616896756 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f75f40b7230 con 0x7f75d406a0c0 2024-05-28T14:15:17.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.650+0000 7f75f9000640 1 -- 172.21.15.42:0/1616896756 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f75d406a0c0 msgr2=0x7f75d406c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:17.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.650+0000 7f75f9000640 1 --2- 172.21.15.42:0/1616896756 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f75d406a0c0 0x7f75d406c580 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f75e40af000 tx=0x7f75e40a7000 comp rx=0 tx=0).stop 2024-05-28T14:15:17.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.650+0000 7f75f9000640 1 -- 172.21.15.42:0/1616896756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f40b20a0 msgr2=0x7f75f415df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:17.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.650+0000 7f75f9000640 1 --2- 172.21.15.42:0/1616896756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f40b20a0 0x7f75f415df10 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f75e000c420 tx=0x7f75e000c8e0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.650+0000 7f75f9000640 1 -- 172.21.15.42:0/1616896756 shutdown_connections 2024-05-28T14:15:17.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.650+0000 7f75f9000640 1 --2- 172.21.15.42:0/1616896756 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f75d406a0c0 0x7f75d406c580 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.650+0000 7f75f9000640 1 --2- 172.21.15.42:0/1616896756 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75f415c530 0x7f75f40c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.650+0000 7f75f9000640 1 --2- 172.21.15.42:0/1616896756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f40b20a0 0x7f75f415df10 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.650+0000 7f75f9000640 1 --2- 172.21.15.42:0/1616896756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f406c400 0x7f75f4152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.650+0000 7f75f9000640 1 -- 172.21.15.42:0/1616896756 >> 172.21.15.42:0/1616896756 conn(0x7f75f409aa50 msgr2=0x7f75f4160a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:17.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.650+0000 7f75f9000640 1 -- 172.21.15.42:0/1616896756 shutdown_connections 2024-05-28T14:15:17.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.650+0000 7f75f9000640 1 -- 172.21.15.42:0/1616896756 wait complete. 2024-05-28T14:15:17.656 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-28T14:15:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.810+0000 7faac3e00640 1 Processor -- start 2024-05-28T14:15:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.810+0000 7faac3e00640 1 -- start start 2024-05-28T14:15:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faac3e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faabc0b4c10 0x7faabc071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faac3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc072320 0x7faabc0aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faac3e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faabc0b3690 con 0x7faabc148090 2024-05-28T14:15:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faac3e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faabc148470 con 0x7faabc0b4c10 2024-05-28T14:15:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faac3e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faabc0b4730 con 0x7faabc072320 2024-05-28T14:15:17.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faac1e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc072320 0x7faabc0aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:17.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faac1400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7faabc148090 0x7faabc0b5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:47182/0 (socket says 172.21.15.42:47182) 2024-05-28T14:15:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faac1400640 1 -- 172.21.15.42:0/1826568208 learned_addr learned my addr 172.21.15.42:0/1826568208 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:15:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faac0a00640 1 --2- 172.21.15.42:0/1826568208 >> v2:172.21.15.63:3300/0 conn(0x7faabc0b4c10 0x7faabc071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faabbe00640 1 -- 172.21.15.42:0/1826568208 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3989682883 0 0) 0x7faabc0b3690 con 0x7faabc148090 2024-05-28T14:15:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faac1e00640 1 -- 172.21.15.42:0/1826568208 >> v2:172.21.15.63:3300/0 conn(0x7faabc0b4c10 msgr2=0x7faabc071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faac1e00640 1 --2- 172.21.15.42:0/1826568208 >> v2:172.21.15.63:3300/0 conn(0x7faabc0b4c10 0x7faabc071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faac1e00640 1 -- 172.21.15.42:0/1826568208 >> v1:172.21.15.63:6789/0 conn(0x7faabc148090 legacy=0x7faabc0b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faac1e00640 1 -- 172.21.15.42:0/1826568208 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc0b3970 con 0x7faabc072320 2024-05-28T14:15:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.814+0000 7faac0a00640 1 --2- 172.21.15.42:0/1826568208 >> v2:172.21.15.63:3300/0 conn(0x7faabc0b4c10 0x7faabc071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:15:17.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.818+0000 7faac1e00640 1 --2- 172.21.15.42:0/1826568208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc072320 0x7faabc0aeb40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faab4009d90 tx=0x7faab401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88bd626d701ed3f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:17.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.818+0000 7faabbe00640 1 -- 172.21.15.42:0/1826568208 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab40b2070 con 0x7faabc072320 2024-05-28T14:15:17.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.818+0000 7faabbe00640 1 -- 172.21.15.42:0/1826568208 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab40a57a0 con 0x7faabc072320 2024-05-28T14:15:17.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.818+0000 7faabbe00640 1 -- 172.21.15.42:0/1826568208 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab40a5b70 con 0x7faabc072320 2024-05-28T14:15:17.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.818+0000 7faac3e00640 1 -- 172.21.15.42:0/1826568208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc072320 msgr2=0x7faabc0aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:17.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.818+0000 7faac3e00640 1 --2- 172.21.15.42:0/1826568208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc072320 0x7faabc0aeb40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faab4009d90 tx=0x7faab401c8e0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.818+0000 7faac3e00640 1 -- 172.21.15.42:0/1826568208 shutdown_connections 2024-05-28T14:15:17.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.818+0000 7faac3e00640 1 --2- 172.21.15.42:0/1826568208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc072320 0x7faabc0aeb40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.818+0000 7faac3e00640 1 --2- 172.21.15.42:0/1826568208 >> v2:172.21.15.63:3300/0 conn(0x7faabc0b4c10 0x7faabc071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.818+0000 7faac3e00640 1 -- 172.21.15.42:0/1826568208 >> 172.21.15.42:0/1826568208 conn(0x7faabc0b21f0 msgr2=0x7faabc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:17.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.818+0000 7faac3e00640 1 -- 172.21.15.42:0/1826568208 shutdown_connections 2024-05-28T14:15:17.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.818+0000 7faac3e00640 1 -- 172.21.15.42:0/1826568208 wait complete. 2024-05-28T14:15:17.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.818+0000 7faac3e00640 1 Processor -- start 2024-05-28T14:15:17.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.818+0000 7faac3e00640 1 -- start start 2024-05-28T14:15:17.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faabc072320 0x7faabc13f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:17.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc148090 0x7faabc13d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:17.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc12d9d0 0x7faabc0b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:17.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac3e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faabc0afa10 con 0x7faabc148090 2024-05-28T14:15:17.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac3e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faabc0af890 con 0x7faabc072320 2024-05-28T14:15:17.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac3e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faabc0afb90 con 0x7faabc12d9d0 2024-05-28T14:15:17.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc148090 0x7faabc13d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:17.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc148090 0x7faabc13d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55398/0 (socket says 172.21.15.42:55398) 2024-05-28T14:15:17.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac0a00640 1 -- 172.21.15.42:0/1703779430 learned_addr learned my addr 172.21.15.42:0/1703779430 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:17.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac1e00640 1 --2- 172.21.15.42:0/1703779430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc12d9d0 0x7faabc0b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:17.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac1400640 1 --2- 172.21.15.42:0/1703779430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faabc072320 0x7faabc13f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:17.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac0a00640 1 -- 172.21.15.42:0/1703779430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc12d9d0 msgr2=0x7faabc0b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:17.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac0a00640 1 --2- 172.21.15.42:0/1703779430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc12d9d0 0x7faabc0b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac0a00640 1 -- 172.21.15.42:0/1703779430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faabc072320 msgr2=0x7faabc13f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:17.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac0a00640 1 --2- 172.21.15.42:0/1703779430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faabc072320 0x7faabc13f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:17.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac0a00640 1 -- 172.21.15.42:0/1703779430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc13f820 con 0x7faabc148090 2024-05-28T14:15:17.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac0a00640 1 --2- 172.21.15.42:0/1703779430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc148090 0x7faabc13d630 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7faaa800ebe0 tx=0x7faaa800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:17.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faab9600640 1 -- 172.21.15.42:0/1703779430 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8017070 con 0x7faabc148090 2024-05-28T14:15:17.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faab9600640 1 -- 172.21.15.42:0/1703779430 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa800cc30 con 0x7faabc148090 2024-05-28T14:15:17.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faab9600640 1 -- 172.21.15.42:0/1703779430 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8013470 con 0x7faabc148090 2024-05-28T14:15:17.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac3e00640 1 -- 172.21.15.42:0/1703779430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faabc13f560 con 0x7faabc148090 2024-05-28T14:15:17.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac3e00640 1 -- 172.21.15.42:0/1703779430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faabc15cdd0 con 0x7faabc148090 2024-05-28T14:15:17.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.822+0000 7faac3e00640 1 -- 172.21.15.42:0/1703779430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faabc0738b0 con 0x7faabc148090 2024-05-28T14:15:17.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.826+0000 7faab9600640 1 -- 172.21.15.42:0/1703779430 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 44) ==== 90639+0+0 (secure 0 0 0) 0x7faaa801a070 con 0x7faabc148090 2024-05-28T14:15:17.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.826+0000 7faab9600640 1 --2- 172.21.15.42:0/1703779430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa9c06a0f0 0x7faa9c06c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:17.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.826+0000 7faab9600640 1 -- 172.21.15.42:0/1703779430 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7faaa80858d0 con 0x7faabc148090 2024-05-28T14:15:17.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.826+0000 7faac1400640 1 --2- 172.21.15.42:0/1703779430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa9c06a0f0 0x7faa9c06c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:17.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.826+0000 7faac1400640 1 --2- 172.21.15.42:0/1703779430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa9c06a0f0 0x7faa9c06c5b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7faab0000f70 tx=0x7faab0007c10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:17.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:17.830+0000 7faab9600640 1 -- 172.21.15.42:0/1703779430 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faaa80578f0 con 0x7faabc148090 2024-05-28T14:15:18.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:18.022+0000 7faac3e00640 1 -- 172.21.15.42:0/1703779430 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7faabc04fdd0 con 0x7faa9c06a0f0 2024-05-28T14:15:18.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:18.026+0000 7faab9600640 1 -- 172.21.15.42:0/1703779430 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7faabc04fdd0 con 0x7faa9c06a0f0 2024-05-28T14:15:18.014 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv1/2eaf73b2-6b4e-4b8f-acd8-93d1b171c401 2024-05-28T14:15:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:18.030+0000 7faac3e00640 1 -- 172.21.15.42:0/1703779430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa9c06a0f0 msgr2=0x7faa9c06c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:18.030+0000 7faac3e00640 1 --2- 172.21.15.42:0/1703779430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa9c06a0f0 0x7faa9c06c5b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7faab0000f70 tx=0x7faab0007c10 comp rx=0 tx=0).stop 2024-05-28T14:15:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:18.030+0000 7faac3e00640 1 -- 172.21.15.42:0/1703779430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc148090 msgr2=0x7faabc13d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:18.030+0000 7faac3e00640 1 --2- 172.21.15.42:0/1703779430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc148090 0x7faabc13d630 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7faaa800ebe0 tx=0x7faaa800c520 comp rx=0 tx=0).stop 2024-05-28T14:15:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:18.030+0000 7faac3e00640 1 -- 172.21.15.42:0/1703779430 shutdown_connections 2024-05-28T14:15:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:18.030+0000 7faac3e00640 1 --2- 172.21.15.42:0/1703779430 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faa9c06a0f0 0x7faa9c06c5b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:18.030+0000 7faac3e00640 1 --2- 172.21.15.42:0/1703779430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faabc12d9d0 0x7faabc0b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:18.030+0000 7faac3e00640 1 --2- 172.21.15.42:0/1703779430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faabc148090 0x7faabc13d630 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:18.030+0000 7faac3e00640 1 --2- 172.21.15.42:0/1703779430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faabc072320 0x7faabc13f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:18.030+0000 7faac3e00640 1 -- 172.21.15.42:0/1703779430 >> 172.21.15.42:0/1703779430 conn(0x7faabc0b21f0 msgr2=0x7faabc168da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:18.030+0000 7faac3e00640 1 -- 172.21.15.42:0/1703779430 shutdown_connections 2024-05-28T14:15:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:18.030+0000 7faac3e00640 1 -- 172.21.15.42:0/1703779430 wait complete. 2024-05-28T14:15:18.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:15:18.035 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/2eaf73b2-6b4e-4b8f-acd8-93d1b171c401 2024-05-28T14:15:18.035 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:15:18.049 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/2eaf73b2-6b4e-4b8f-acd8-93d1b171c401 2024-05-28T14:15:18.049 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3138256896" 2024-05-28T14:15:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:15:18.050 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-28T14:15:20.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:15:20.051 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/2eaf73b2-6b4e-4b8f-acd8-93d1b171c401 2024-05-28T14:15:20.051 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:15:20.066 INFO:teuthology.orchestra.run.smithi042.stdout:# file: volumes/_nogroup/sv1/2eaf73b2-6b4e-4b8f-acd8-93d1b171c401 2024-05-28T14:15:20.066 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.rbytes="3221225472" 2024-05-28T14:15:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:15:20.067 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-28T14:15:20.067 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone6271 2024-05-28T14:15:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba9200640 1 Processor -- start 2024-05-28T14:15:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba9200640 1 -- start start 2024-05-28T14:15:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba9200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feba4071de0 0x7feba40721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba9200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feba40ce3b0 0x7feba40ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba9200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7feba4141d10 0x7feba41421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba9200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7feba4072700 con 0x7feba4141d10 2024-05-28T14:15:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba9200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7feba4160230 con 0x7feba4071de0 2024-05-28T14:15:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba9200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feba4160510 con 0x7feba40ce3b0 2024-05-28T14:15:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba2800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7feba4141d10 0x7feba41421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:20.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba2800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7feba4141d10 0x7feba41421a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55414/0 (socket says 172.21.15.42:55414) 2024-05-28T14:15:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba2800640 1 -- 172.21.15.42:0/990607652 learned_addr learned my addr 172.21.15.42:0/990607652 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba1e00640 1 --2- 172.21.15.42:0/990607652 >> v2:172.21.15.63:3300/0 conn(0x7feba4071de0 0x7feba40721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feb99400640 1 --2- 172.21.15.42:0/990607652 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feba40ce3b0 0x7feba40ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba2800640 1 -- 172.21.15.42:0/990607652 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feba40ce3b0 msgr2=0x7feba40ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba2800640 1 --2- 172.21.15.42:0/990607652 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feba40ce3b0 0x7feba40ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba2800640 1 -- 172.21.15.42:0/990607652 >> v2:172.21.15.63:3300/0 conn(0x7feba4071de0 msgr2=0x7feba40721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba2800640 1 --2- 172.21.15.42:0/990607652 >> v2:172.21.15.63:3300/0 conn(0x7feba4071de0 0x7feba40721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.282+0000 7feba2800640 1 -- 172.21.15.42:0/990607652 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba41607f0 con 0x7feba4141d10 2024-05-28T14:15:20.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.286+0000 7feba2800640 1 --2- 172.21.15.42:0/990607652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba4141d10 0x7feba41421a0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7feba406cbf0 tx=0x7feb9c01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321f4ec3025b4414 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:20.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.286+0000 7feba1400640 1 -- 172.21.15.42:0/990607652 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0a5550 con 0x7feba4141d10 2024-05-28T14:15:20.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.286+0000 7feba1400640 1 -- 172.21.15.42:0/990607652 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9c0a56f0 con 0x7feba4141d10 2024-05-28T14:15:20.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.286+0000 7feba1400640 1 -- 172.21.15.42:0/990607652 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0a5ac0 con 0x7feba4141d10 2024-05-28T14:15:20.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.286+0000 7feba9200640 1 -- 172.21.15.42:0/990607652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba4141d10 msgr2=0x7feba41421a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:20.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.286+0000 7feba9200640 1 --2- 172.21.15.42:0/990607652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba4141d10 0x7feba41421a0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7feba406cbf0 tx=0x7feb9c01c8d0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.286+0000 7feba9200640 1 -- 172.21.15.42:0/990607652 shutdown_connections 2024-05-28T14:15:20.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.286+0000 7feba9200640 1 --2- 172.21.15.42:0/990607652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba4141d10 0x7feba41421a0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.286+0000 7feba9200640 1 --2- 172.21.15.42:0/990607652 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feba40ce3b0 0x7feba40ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.286+0000 7feba9200640 1 --2- 172.21.15.42:0/990607652 >> v2:172.21.15.63:3300/0 conn(0x7feba4071de0 0x7feba40721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.286+0000 7feba9200640 1 -- 172.21.15.42:0/990607652 >> 172.21.15.42:0/990607652 conn(0x7feba40b21f0 msgr2=0x7feba40b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:20.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.286+0000 7feba9200640 1 -- 172.21.15.42:0/990607652 shutdown_connections 2024-05-28T14:15:20.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.286+0000 7feba9200640 1 -- 172.21.15.42:0/990607652 wait complete. 2024-05-28T14:15:20.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feba9200640 1 Processor -- start 2024-05-28T14:15:20.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feba9200640 1 -- start start 2024-05-28T14:15:20.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feba9200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feba4071de0 0x7feba40b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:20.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feba9200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba40ce3b0 0x7feba4159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:20.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feba9200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feba4141d10 0x7feba41408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:20.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feba9200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feba4140520 con 0x7feba40ce3b0 2024-05-28T14:15:20.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feba9200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feba413f8c0 con 0x7feba4071de0 2024-05-28T14:15:20.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feba9200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feba41406a0 con 0x7feba4141d10 2024-05-28T14:15:20.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feb99400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba40ce3b0 0x7feba4159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:20.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feb99400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba40ce3b0 0x7feba4159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55416/0 (socket says 172.21.15.42:55416) 2024-05-28T14:15:20.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feb99400640 1 -- 172.21.15.42:0/828982058 learned_addr learned my addr 172.21.15.42:0/828982058 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:20.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feba1e00640 1 --2- 172.21.15.42:0/828982058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feba4071de0 0x7feba40b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:20.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feba2800640 1 --2- 172.21.15.42:0/828982058 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feba4141d10 0x7feba41408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:20.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feb99400640 1 -- 172.21.15.42:0/828982058 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feba4141d10 msgr2=0x7feba41408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:20.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feb99400640 1 --2- 172.21.15.42:0/828982058 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feba4141d10 0x7feba41408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feb99400640 1 -- 172.21.15.42:0/828982058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feba4071de0 msgr2=0x7feba40b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:20.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feb99400640 1 --2- 172.21.15.42:0/828982058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feba4071de0 0x7feba40b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feb99400640 1 -- 172.21.15.42:0/828982058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba4136860 con 0x7feba40ce3b0 2024-05-28T14:15:20.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feb99400640 1 --2- 172.21.15.42:0/828982058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba40ce3b0 0x7feba4159c10 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00d800 tx=0x7feb8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:20.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feb9aa00640 1 -- 172.21.15.42:0/828982058 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c00b6c0 con 0x7feba40ce3b0 2024-05-28T14:15:20.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feb9aa00640 1 -- 172.21.15.42:0/828982058 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb8c019070 con 0x7feba40ce3b0 2024-05-28T14:15:20.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feb9aa00640 1 -- 172.21.15.42:0/828982058 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c00ba60 con 0x7feba40ce3b0 2024-05-28T14:15:20.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.290+0000 7feba9200640 1 -- 172.21.15.42:0/828982058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba4136ba0 con 0x7feba40ce3b0 2024-05-28T14:15:20.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.294+0000 7feba9200640 1 -- 172.21.15.42:0/828982058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba41373e0 con 0x7feba40ce3b0 2024-05-28T14:15:20.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.294+0000 7feba9200640 1 -- 172.21.15.42:0/828982058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba40738b0 con 0x7feba40ce3b0 2024-05-28T14:15:20.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.294+0000 7feb9aa00640 1 -- 172.21.15.42:0/828982058 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 44) ==== 90639+0+0 (secure 0 0 0) 0x7feb8c00bc00 con 0x7feba40ce3b0 2024-05-28T14:15:20.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.294+0000 7feb9aa00640 1 --2- 172.21.15.42:0/828982058 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feb8006a0f0 0x7feb8006c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:20.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.294+0000 7feb9aa00640 1 -- 172.21.15.42:0/828982058 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7feb8c085c40 con 0x7feba40ce3b0 2024-05-28T14:15:20.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.294+0000 7feba1e00640 1 --2- 172.21.15.42:0/828982058 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feb8006a0f0 0x7feb8006c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:20.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.298+0000 7feba1e00640 1 --2- 172.21.15.42:0/828982058 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feb8006a0f0 0x7feb8006c5b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7feb940045f0 tx=0x7feb94008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:20.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.298+0000 7feb9aa00640 1 -- 172.21.15.42:0/828982058 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feba4136ba0 con 0x7feba40ce3b0 2024-05-28T14:15:20.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.494+0000 7feba9200640 1 -- 172.21.15.42:0/828982058 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone6271", "target": ["mon-mgr", ""]}) -- 0x7feba404fdd0 con 0x7feb8006a0f0 2024-05-28T14:15:20.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.562+0000 7feb9aa00640 1 -- 172.21.15.42:0/828982058 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feba404fdd0 con 0x7feb8006a0f0 2024-05-28T14:15:20.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.562+0000 7feba9200640 1 -- 172.21.15.42:0/828982058 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feb8006a0f0 msgr2=0x7feb8006c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:20.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.562+0000 7feba9200640 1 --2- 172.21.15.42:0/828982058 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feb8006a0f0 0x7feb8006c5b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7feb940045f0 tx=0x7feb94008040 comp rx=0 tx=0).stop 2024-05-28T14:15:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.562+0000 7feba9200640 1 -- 172.21.15.42:0/828982058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba40ce3b0 msgr2=0x7feba4159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.562+0000 7feba9200640 1 --2- 172.21.15.42:0/828982058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba40ce3b0 0x7feba4159c10 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00d800 tx=0x7feb8c00dcc0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.562+0000 7feba9200640 1 -- 172.21.15.42:0/828982058 shutdown_connections 2024-05-28T14:15:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.562+0000 7feba9200640 1 --2- 172.21.15.42:0/828982058 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7feb8006a0f0 0x7feb8006c5b0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.562+0000 7feba9200640 1 --2- 172.21.15.42:0/828982058 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feba4141d10 0x7feba41408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.562+0000 7feba9200640 1 --2- 172.21.15.42:0/828982058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feba40ce3b0 0x7feba4159c10 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.562+0000 7feba9200640 1 --2- 172.21.15.42:0/828982058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feba4071de0 0x7feba40b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.562+0000 7feba9200640 1 -- 172.21.15.42:0/828982058 >> 172.21.15.42:0/828982058 conn(0x7feba40b21f0 msgr2=0x7feba41392e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.562+0000 7feba9200640 1 -- 172.21.15.42:0/828982058 shutdown_connections 2024-05-28T14:15:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.562+0000 7feba9200640 1 -- 172.21.15.42:0/828982058 wait complete. 2024-05-28T14:15:20.552 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:20.566+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:20.552 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:20.566+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:20.552 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:20.566+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:20.552 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:20.566+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:20.552 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:20.566+0000 7f077be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:20.570 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:20.582+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:20.570 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:20.582+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:20.570 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:20.582+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:20.570 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:20.582+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:20.570 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:20.582+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:20.573 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone6272 2024-05-28T14:15:20.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f226ee00640 1 Processor -- start 2024-05-28T14:15:20.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f226ee00640 1 -- start start 2024-05-28T14:15:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f226ee00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f22680ce3b0 0x7f22680ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f226ee00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2268160230 con 0x7f226813fd10 2024-05-28T14:15:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f226ee00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2268160510 con 0x7f2268071de0 2024-05-28T14:15:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f226ee00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f22681607f0 con 0x7f22680ce3b0 2024-05-28T14:15:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f226ce00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f226813fd10 0x7f2268140180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37070/0 (socket says 172.21.15.42:37070) 2024-05-28T14:15:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f226ce00640 1 -- 172.21.15.42:0/1598880512 learned_addr learned my addr 172.21.15.42:0/1598880512 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:15:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f2267400640 1 --2- 172.21.15.42:0/1598880512 >> v2:172.21.15.42:3300/0 conn(0x7f22680ce3b0 0x7f22680ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f2266a00640 1 -- 172.21.15.42:0/1598880512 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2017491210 0 0) 0x7f2268160230 con 0x7f226813fd10 2024-05-28T14:15:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f2266a00640 1 -- 172.21.15.42:0/1598880512 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f224c003530 con 0x7f226813fd10 2024-05-28T14:15:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f2266a00640 1 -- 172.21.15.42:0/1598880512 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2109468577 0 0) 0x7f2268160510 con 0x7f2268071de0 2024-05-28T14:15:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f2267400640 1 -- 172.21.15.42:0/1598880512 >> v1:172.21.15.63:6789/0 conn(0x7f2268071de0 legacy=0x7f22680721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:20.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f2267400640 1 -- 172.21.15.42:0/1598880512 >> v1:172.21.15.42:6789/0 conn(0x7f226813fd10 legacy=0x7f2268140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:20.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.738+0000 7f2267400640 1 -- 172.21.15.42:0/1598880512 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2268160ad0 con 0x7f22680ce3b0 2024-05-28T14:15:20.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.742+0000 7f2267400640 1 --2- 172.21.15.42:0/1598880512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22680ce3b0 0x7f22680ce810 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f22540097c0 tx=0x7f225401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd9dba628622d73 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:20.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.742+0000 7f2266a00640 1 -- 172.21.15.42:0/1598880512 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540b0070 con 0x7f22680ce3b0 2024-05-28T14:15:20.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.742+0000 7f2266a00640 1 -- 172.21.15.42:0/1598880512 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22540ab600 con 0x7f22680ce3b0 2024-05-28T14:15:20.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.742+0000 7f2266a00640 1 -- 172.21.15.42:0/1598880512 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540ab9d0 con 0x7f22680ce3b0 2024-05-28T14:15:20.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.742+0000 7f226ee00640 1 -- 172.21.15.42:0/1598880512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22680ce3b0 msgr2=0x7f22680ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:20.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.742+0000 7f226ee00640 1 --2- 172.21.15.42:0/1598880512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22680ce3b0 0x7f22680ce810 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f22540097c0 tx=0x7f225401c380 comp rx=0 tx=0).stop 2024-05-28T14:15:20.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.742+0000 7f226ee00640 1 -- 172.21.15.42:0/1598880512 shutdown_connections 2024-05-28T14:15:20.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.742+0000 7f226ee00640 1 --2- 172.21.15.42:0/1598880512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22680ce3b0 0x7f22680ce810 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.742+0000 7f226ee00640 1 -- 172.21.15.42:0/1598880512 >> 172.21.15.42:0/1598880512 conn(0x7f22680b21f0 msgr2=0x7f22680b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:20.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.742+0000 7f226ee00640 1 -- 172.21.15.42:0/1598880512 shutdown_connections 2024-05-28T14:15:20.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.742+0000 7f226ee00640 1 -- 172.21.15.42:0/1598880512 wait complete. 2024-05-28T14:15:20.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f226ee00640 1 Processor -- start 2024-05-28T14:15:20.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f226ee00640 1 -- start start 2024-05-28T14:15:20.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f226ee00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2268071de0 0x7f2268139d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:20.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f226ee00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22680ce3b0 0x7f226813ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:20.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f226ee00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f226813fd10 0x7f22681455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:20.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f226ee00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f226816a840 con 0x7f22680ce3b0 2024-05-28T14:15:20.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f226ee00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f226816a6c0 con 0x7f2268071de0 2024-05-28T14:15:20.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f226ee00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f226816a9c0 con 0x7f226813fd10 2024-05-28T14:15:20.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f2267400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22680ce3b0 0x7f226813ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:20.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f2267400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22680ce3b0 0x7f226813ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55476/0 (socket says 172.21.15.42:55476) 2024-05-28T14:15:20.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f2267400640 1 -- 172.21.15.42:0/2478848384 learned_addr learned my addr 172.21.15.42:0/2478848384 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:20.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f2267e00640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2268071de0 0x7f2268139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:20.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f226ce00640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f226813fd10 0x7f22681455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:20.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f2267400640 1 -- 172.21.15.42:0/2478848384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f226813fd10 msgr2=0x7f22681455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:20.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f2267400640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f226813fd10 0x7f22681455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f2267400640 1 -- 172.21.15.42:0/2478848384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2268071de0 msgr2=0x7f2268139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:20.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f2267400640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2268071de0 0x7f2268139d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:20.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f2267400640 1 -- 172.21.15.42:0/2478848384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226815cc80 con 0x7f22680ce3b0 2024-05-28T14:15:20.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f2267e00640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2268071de0 0x7f2268139d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:15:20.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f226ce00640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f226813fd10 0x7f22681455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:15:20.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f2267400640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22680ce3b0 0x7f226813ce20 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f22540ae040 tx=0x7f2254019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:20.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f225fe00640 1 -- 172.21.15.42:0/2478848384 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540b8070 con 0x7f22680ce3b0 2024-05-28T14:15:20.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f225fe00640 1 -- 172.21.15.42:0/2478848384 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2254004020 con 0x7f22680ce3b0 2024-05-28T14:15:20.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f226ee00640 1 -- 172.21.15.42:0/2478848384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226815cf60 con 0x7f22680ce3b0 2024-05-28T14:15:20.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f225fe00640 1 -- 172.21.15.42:0/2478848384 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540b0040 con 0x7f22680ce3b0 2024-05-28T14:15:20.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.746+0000 7f226ee00640 1 -- 172.21.15.42:0/2478848384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f226813a260 con 0x7f22680ce3b0 2024-05-28T14:15:20.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.750+0000 7f226ee00640 1 -- 172.21.15.42:0/2478848384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22680738b0 con 0x7f22680ce3b0 2024-05-28T14:15:20.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.750+0000 7f225fe00640 1 -- 172.21.15.42:0/2478848384 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 44) ==== 90639+0+0 (secure 0 0 0) 0x7f2254003620 con 0x7f22680ce3b0 2024-05-28T14:15:20.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.750+0000 7f225fe00640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f224806a0c0 0x7f224806c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:20.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.750+0000 7f225fe00640 1 -- 172.21.15.42:0/2478848384 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f225411f520 con 0x7f22680ce3b0 2024-05-28T14:15:20.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.750+0000 7f2267e00640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f224806a0c0 0x7f224806c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:20.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.750+0000 7f2267e00640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f224806a0c0 0x7f224806c580 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f2258000f70 tx=0x7f2258009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:20.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.754+0000 7f225fe00640 1 -- 172.21.15.42:0/2478848384 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2254016090 con 0x7f22680ce3b0 2024-05-28T14:15:20.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:20.950+0000 7f226ee00640 1 -- 172.21.15.42:0/2478848384 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone6272", "target": ["mon-mgr", ""]}) -- 0x7f226804fdd0 con 0x7f224806a0c0 2024-05-28T14:15:21.220 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:15:21.223 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:15:21.553 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:21.566+0000 7f070fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:21.553 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:21.566+0000 7f070fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:21.553 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:21.566+0000 7f070fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:21.553 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:21.566+0000 7f070fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:21.553 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:21.566+0000 7f070fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:21.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:21.650+0000 7f225fe00640 1 -- 172.21.15.42:0/2478848384 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f226804fdd0 con 0x7f224806a0c0 2024-05-28T14:15:21.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:21.650+0000 7f226ee00640 1 -- 172.21.15.42:0/2478848384 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f224806a0c0 msgr2=0x7f224806c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:21.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:21.650+0000 7f226ee00640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f224806a0c0 0x7f224806c580 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f2258000f70 tx=0x7f2258009040 comp rx=0 tx=0).stop 2024-05-28T14:15:21.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:21.650+0000 7f226ee00640 1 -- 172.21.15.42:0/2478848384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22680ce3b0 msgr2=0x7f226813ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:21.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:21.650+0000 7f226ee00640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22680ce3b0 0x7f226813ce20 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f22540ae040 tx=0x7f2254019120 comp rx=0 tx=0).stop 2024-05-28T14:15:21.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:21.654+0000 7f226ee00640 1 -- 172.21.15.42:0/2478848384 shutdown_connections 2024-05-28T14:15:21.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:21.654+0000 7f226ee00640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f224806a0c0 0x7f224806c580 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:21.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:21.654+0000 7f226ee00640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f226813fd10 0x7f22681455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:21.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:21.654+0000 7f226ee00640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22680ce3b0 0x7f226813ce20 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:21.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:21.654+0000 7f226ee00640 1 --2- 172.21.15.42:0/2478848384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2268071de0 0x7f2268139d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:21.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:21.654+0000 7f226ee00640 1 -- 172.21.15.42:0/2478848384 >> 172.21.15.42:0/2478848384 conn(0x7f22680b21f0 msgr2=0x7f226815b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:21.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:21.654+0000 7f226ee00640 1 -- 172.21.15.42:0/2478848384 shutdown_connections 2024-05-28T14:15:21.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:21.654+0000 7f226ee00640 1 -- 172.21.15.42:0/2478848384 wait complete. 2024-05-28T14:15:21.658 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:21.670+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:21.658 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:21.670+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:21.658 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:21.670+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:21.658 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:21.670+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:21.658 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:21.670+0000 7f077a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:21.688 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone6273 2024-05-28T14:15:22.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.014+0000 7fb8cc200640 1 Processor -- start 2024-05-28T14:15:22.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.014+0000 7fb8cc200640 1 -- start start 2024-05-28T14:15:22.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.018+0000 7fb8cc200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb8c40afce0 0x7fb8c4071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:22.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.018+0000 7fb8cc200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8c40aef20 con 0x7fb8c4144e60 2024-05-28T14:15:22.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.018+0000 7fb8cc200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8c40af200 con 0x7fb8c40b5410 2024-05-28T14:15:22.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.018+0000 7fb8cc200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c40af850 con 0x7fb8c40afce0 2024-05-28T14:15:22.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.018+0000 7fb8c9800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb8c4144e60 0x7fb8c40b4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37072/0 (socket says 172.21.15.42:37072) 2024-05-28T14:15:22.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.018+0000 7fb8c8e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb8c40afce0 0x7fb8c4071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:22.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.018+0000 7fb8c8e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb8c40afce0 0x7fb8c4071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55514/0 (socket says 172.21.15.42:55514) 2024-05-28T14:15:22.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.018+0000 7fb8c8e00640 1 -- 172.21.15.42:0/4021721220 learned_addr learned my addr 172.21.15.42:0/4021721220 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:22.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.022+0000 7fb8c8e00640 1 -- 172.21.15.42:0/4021721220 >> v1:172.21.15.63:6789/0 conn(0x7fb8c40b5410 legacy=0x7fb8c406dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:22.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.022+0000 7fb8c8e00640 1 -- 172.21.15.42:0/4021721220 >> v1:172.21.15.42:6789/0 conn(0x7fb8c4144e60 legacy=0x7fb8c40b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:22.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.022+0000 7fb8c8e00640 1 -- 172.21.15.42:0/4021721220 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c40af4e0 con 0x7fb8c40afce0 2024-05-28T14:15:22.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.026+0000 7fb8c8e00640 1 --2- 172.21.15.42:0/4021721220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c40afce0 0x7fb8c4071de0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009870 tx=0x7fb8b801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e55c9812b0e0d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:22.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.026+0000 7fb8c3e00640 1 -- 172.21.15.42:0/4021721220 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80aa5f0 con 0x7fb8c40afce0 2024-05-28T14:15:22.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.030+0000 7fb8c3e00640 1 -- 172.21.15.42:0/4021721220 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b80af070 con 0x7fb8c40afce0 2024-05-28T14:15:22.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.030+0000 7fb8c3e00640 1 -- 172.21.15.42:0/4021721220 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80aa9f0 con 0x7fb8c40afce0 2024-05-28T14:15:22.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.030+0000 7fb8cc200640 1 -- 172.21.15.42:0/4021721220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c40afce0 msgr2=0x7fb8c4071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:22.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.030+0000 7fb8cc200640 1 --2- 172.21.15.42:0/4021721220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c40afce0 0x7fb8c4071de0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009870 tx=0x7fb8b801c540 comp rx=0 tx=0).stop 2024-05-28T14:15:22.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.030+0000 7fb8cc200640 1 -- 172.21.15.42:0/4021721220 shutdown_connections 2024-05-28T14:15:22.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.030+0000 7fb8cc200640 1 --2- 172.21.15.42:0/4021721220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c40afce0 0x7fb8c4071de0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:22.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.030+0000 7fb8cc200640 1 -- 172.21.15.42:0/4021721220 >> 172.21.15.42:0/4021721220 conn(0x7fb8c40b21f0 msgr2=0x7fb8c40b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:22.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.030+0000 7fb8cc200640 1 -- 172.21.15.42:0/4021721220 shutdown_connections 2024-05-28T14:15:22.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.030+0000 7fb8cc200640 1 -- 172.21.15.42:0/4021721220 wait complete. 2024-05-28T14:15:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.034+0000 7fb8cc200640 1 Processor -- start 2024-05-28T14:15:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.034+0000 7fb8cc200640 1 -- start start 2024-05-28T14:15:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.038+0000 7fb8cc200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8c40afce0 0x7fb8c4143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.038+0000 7fb8cc200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c40b5410 0x7fb8c415f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.038+0000 7fb8cc200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8c4144e60 0x7fb8c415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.038+0000 7fb8cc200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c416c880 con 0x7fb8c40b5410 2024-05-28T14:15:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.038+0000 7fb8cc200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c416c700 con 0x7fb8c4144e60 2024-05-28T14:15:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.038+0000 7fb8cc200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c416ca00 con 0x7fb8c40afce0 2024-05-28T14:15:22.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.038+0000 7fb8c9800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8c40afce0 0x7fb8c4143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:22.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.038+0000 7fb8c9800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8c40afce0 0x7fb8c4143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:56222/0 (socket says 172.21.15.42:56222) 2024-05-28T14:15:22.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.038+0000 7fb8c9800640 1 -- 172.21.15.42:0/4042900557 learned_addr learned my addr 172.21.15.42:0/4042900557 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.038+0000 7fb8ca200640 1 --2- 172.21.15.42:0/4042900557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8c4144e60 0x7fb8c415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.042+0000 7fb8c8e00640 1 --2- 172.21.15.42:0/4042900557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c40b5410 0x7fb8c415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.042+0000 7fb8ca200640 1 -- 172.21.15.42:0/4042900557 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8c40afce0 msgr2=0x7fb8c4143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.042+0000 7fb8ca200640 1 --2- 172.21.15.42:0/4042900557 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8c40afce0 0x7fb8c4143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.042+0000 7fb8ca200640 1 -- 172.21.15.42:0/4042900557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c40b5410 msgr2=0x7fb8c415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.042+0000 7fb8ca200640 1 --2- 172.21.15.42:0/4042900557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c40b5410 0x7fb8c415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:22.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.042+0000 7fb8ca200640 1 -- 172.21.15.42:0/4042900557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c41420e0 con 0x7fb8c4144e60 2024-05-28T14:15:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.042+0000 7fb8ca200640 1 --2- 172.21.15.42:0/4042900557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8c4144e60 0x7fb8c415baa0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000c3f0 tx=0x7fb8b000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:22.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.050+0000 7fb8c1600640 1 -- 172.21.15.42:0/4042900557 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b00052d0 con 0x7fb8c4144e60 2024-05-28T14:15:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.050+0000 7fb8c1600640 1 -- 172.21.15.42:0/4042900557 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b0005470 con 0x7fb8c4144e60 2024-05-28T14:15:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.050+0000 7fb8c1600640 1 -- 172.21.15.42:0/4042900557 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0007220 con 0x7fb8c4144e60 2024-05-28T14:15:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.050+0000 7fb8cc200640 1 -- 172.21.15.42:0/4042900557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c41423c0 con 0x7fb8c4144e60 2024-05-28T14:15:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.054+0000 7fb8cc200640 1 -- 172.21.15.42:0/4042900557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c415ef30 con 0x7fb8c4144e60 2024-05-28T14:15:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.054+0000 7fb8cc200640 1 -- 172.21.15.42:0/4042900557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c40738b0 con 0x7fb8c4144e60 2024-05-28T14:15:22.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.066+0000 7fb8c1600640 1 -- 172.21.15.42:0/4042900557 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 45) ==== 90741+0+0 (secure 0 0 0) 0x7fb8b00073c0 con 0x7fb8c4144e60 2024-05-28T14:15:22.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.066+0000 7fb8c1600640 1 --2- 172.21.15.42:0/4042900557 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb8a406a1e0 0x7fb8a406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:22.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.066+0000 7fb8c1600640 1 -- 172.21.15.42:0/4042900557 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fb8b0085240 con 0x7fb8c4144e60 2024-05-28T14:15:22.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.066+0000 7fb8c1600640 1 -- 172.21.15.42:0/4042900557 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb8b0086670 con 0x7fb8c4144e60 2024-05-28T14:15:22.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.074+0000 7fb8c9800640 1 --2- 172.21.15.42:0/4042900557 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb8a406a1e0 0x7fb8a406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:22.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.082+0000 7fb8c9800640 1 --2- 172.21.15.42:0/4042900557 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb8a406a1e0 0x7fb8a406c6a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc004fd0 tx=0x7fb8bc0016b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:22.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.514+0000 7fb8cc200640 1 -- 172.21.15.42:0/4042900557 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone6273", "target": ["mon-mgr", ""]}) -- 0x7fb8c4141e90 con 0x7fb8a406a1e0 2024-05-28T14:15:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.930+0000 7fb8c1600640 1 -- 172.21.15.42:0/4042900557 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb8c4141e90 con 0x7fb8a406a1e0 2024-05-28T14:15:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.934+0000 7fb8cc200640 1 -- 172.21.15.42:0/4042900557 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb8a406a1e0 msgr2=0x7fb8a406c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.934+0000 7fb8cc200640 1 --2- 172.21.15.42:0/4042900557 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb8a406a1e0 0x7fb8a406c6a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc004fd0 tx=0x7fb8bc0016b0 comp rx=0 tx=0).stop 2024-05-28T14:15:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.934+0000 7fb8cc200640 1 -- 172.21.15.42:0/4042900557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8c4144e60 msgr2=0x7fb8c415baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.934+0000 7fb8cc200640 1 --2- 172.21.15.42:0/4042900557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8c4144e60 0x7fb8c415baa0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000c3f0 tx=0x7fb8b000c8b0 comp rx=0 tx=0).stop 2024-05-28T14:15:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.934+0000 7fb8cc200640 1 -- 172.21.15.42:0/4042900557 shutdown_connections 2024-05-28T14:15:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.934+0000 7fb8cc200640 1 --2- 172.21.15.42:0/4042900557 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb8a406a1e0 0x7fb8a406c6a0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:22.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.934+0000 7fb8cc200640 1 --2- 172.21.15.42:0/4042900557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8c4144e60 0x7fb8c415baa0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:22.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.934+0000 7fb8cc200640 1 --2- 172.21.15.42:0/4042900557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c40b5410 0x7fb8c415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:22.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.934+0000 7fb8cc200640 1 --2- 172.21.15.42:0/4042900557 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8c40afce0 0x7fb8c4143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:22.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.934+0000 7fb8cc200640 1 -- 172.21.15.42:0/4042900557 >> 172.21.15.42:0/4042900557 conn(0x7fb8c40b21f0 msgr2=0x7fb8c406efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:22.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.934+0000 7fb8cc200640 1 -- 172.21.15.42:0/4042900557 shutdown_connections 2024-05-28T14:15:22.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:22.934+0000 7fb8cc200640 1 -- 172.21.15.42:0/4042900557 wait complete. 2024-05-28T14:15:22.960 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:22.970+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:22.960 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:22.970+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:22.960 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:22.970+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:22.960 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:22.970+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:22.960 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:22.970+0000 7f077aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:23.001 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone6274 2024-05-28T14:15:23.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.378+0000 7ffbf6600640 1 Processor -- start 2024-05-28T14:15:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.378+0000 7ffbf6600640 1 -- start start 2024-05-28T14:15:23.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.382+0000 7ffbf6600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffbf0162d10 0x7ffbf006f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:23.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.382+0000 7ffbf6600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbf0166800 con 0x7ffbf00724f0 2024-05-28T14:15:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.382+0000 7ffbf6600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbf006f8c0 con 0x7ffbf00cfb40 2024-05-28T14:15:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.382+0000 7ffbf6600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7ffbf01630f0 con 0x7ffbf0162d10 2024-05-28T14:15:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.382+0000 7ffbef400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffbf0162d10 0x7ffbf006f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.382+0000 7ffbefe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ffbf00724f0 0x7ffbf00b34f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:37078/0 (socket says 172.21.15.42:37078) 2024-05-28T14:15:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.382+0000 7ffbefe00640 1 -- 172.21.15.42:0/3883932869 learned_addr learned my addr 172.21.15.42:0/3883932869 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:15:23.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.382+0000 7ffbef400640 1 -- 172.21.15.42:0/3883932869 >> v1:172.21.15.63:6789/0 conn(0x7ffbf00cfb40 legacy=0x7ffbf0071de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-28T14:15:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.382+0000 7ffbef400640 1 -- 172.21.15.42:0/3883932869 >> v1:172.21.15.42:6789/0 conn(0x7ffbf00724f0 legacy=0x7ffbf00b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.382+0000 7ffbef400640 1 -- 172.21.15.42:0/3883932869 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf0166ac0 con 0x7ffbf0162d10 2024-05-28T14:15:23.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.390+0000 7ffbef400640 1 --2- 172.21.15.42:0/3883932869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbf0162d10 0x7ffbf006f2e0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00097c0 tx=0x7ffbe001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efa934dadd0e6a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:23.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.394+0000 7ffbee000640 1 -- 172.21.15.42:0/3883932869 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00b1070 con 0x7ffbf0162d10 2024-05-28T14:15:23.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.394+0000 7ffbee000640 1 -- 172.21.15.42:0/3883932869 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbe00ac640 con 0x7ffbf0162d10 2024-05-28T14:15:23.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.394+0000 7ffbee000640 1 -- 172.21.15.42:0/3883932869 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00aca10 con 0x7ffbf0162d10 2024-05-28T14:15:23.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.394+0000 7ffbf6600640 1 -- 172.21.15.42:0/3883932869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbf0162d10 msgr2=0x7ffbf006f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:23.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.394+0000 7ffbf6600640 1 --2- 172.21.15.42:0/3883932869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbf0162d10 0x7ffbf006f2e0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00097c0 tx=0x7ffbe001c420 comp rx=0 tx=0).stop 2024-05-28T14:15:23.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.398+0000 7ffbf6600640 1 -- 172.21.15.42:0/3883932869 shutdown_connections 2024-05-28T14:15:23.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.398+0000 7ffbf6600640 1 --2- 172.21.15.42:0/3883932869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbf0162d10 0x7ffbf006f2e0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:23.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.398+0000 7ffbf6600640 1 -- 172.21.15.42:0/3883932869 >> 172.21.15.42:0/3883932869 conn(0x7ffbf00b21f0 msgr2=0x7ffbf00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:23.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.398+0000 7ffbf6600640 1 -- 172.21.15.42:0/3883932869 shutdown_connections 2024-05-28T14:15:23.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.398+0000 7ffbf6600640 1 -- 172.21.15.42:0/3883932869 wait complete. 2024-05-28T14:15:23.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.402+0000 7ffbf6600640 1 Processor -- start 2024-05-28T14:15:23.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.402+0000 7ffbf6600640 1 -- start start 2024-05-28T14:15:23.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.402+0000 7ffbf6600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbf00724f0 0x7ffbf00cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:23.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.402+0000 7ffbf6600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbf00cfb40 0x7ffbf015c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:23.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.402+0000 7ffbf6600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbf0162d10 0x7ffbf006dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:23.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.402+0000 7ffbf6600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf01675a0 con 0x7ffbf00cfb40 2024-05-28T14:15:23.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.402+0000 7ffbef400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbf00724f0 0x7ffbf00cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:23.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.402+0000 7ffbef400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbf00724f0 0x7ffbf00cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:56232/0 (socket says 172.21.15.42:56232) 2024-05-28T14:15:23.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.402+0000 7ffbef400640 1 -- 172.21.15.42:0/1235635305 learned_addr learned my addr 172.21.15.42:0/1235635305 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:23.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.402+0000 7ffbf6600640 1 -- 172.21.15.42:0/1235635305 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf01709a0 con 0x7ffbf0162d10 2024-05-28T14:15:23.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.402+0000 7ffbf6600640 1 -- 172.21.15.42:0/1235635305 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffbf00aa2a0 con 0x7ffbf00724f0 2024-05-28T14:15:23.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.402+0000 7ffbeea00640 1 --2- 172.21.15.42:0/1235635305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbf00cfb40 0x7ffbf015c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:23.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.406+0000 7ffbefe00640 1 --2- 172.21.15.42:0/1235635305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbf0162d10 0x7ffbf006dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:23.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.406+0000 7ffbef400640 1 -- 172.21.15.42:0/1235635305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbf0162d10 msgr2=0x7ffbf006dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:23.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.406+0000 7ffbef400640 1 --2- 172.21.15.42:0/1235635305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbf0162d10 0x7ffbf006dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:23.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.406+0000 7ffbef400640 1 -- 172.21.15.42:0/1235635305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbf00cfb40 msgr2=0x7ffbf015c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:23.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.406+0000 7ffbef400640 1 --2- 172.21.15.42:0/1235635305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbf00cfb40 0x7ffbf015c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:23.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.406+0000 7ffbef400640 1 -- 172.21.15.42:0/1235635305 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf00aa580 con 0x7ffbf00724f0 2024-05-28T14:15:23.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.406+0000 7ffbef400640 1 --2- 172.21.15.42:0/1235635305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbf00724f0 0x7ffbf00cc870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00af040 tx=0x7ffbe001ac60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:23.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.406+0000 7ffbe7400640 1 -- 172.21.15.42:0/1235635305 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00b1050 con 0x7ffbf00724f0 2024-05-28T14:15:23.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.406+0000 7ffbe7400640 1 -- 172.21.15.42:0/1235635305 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbe00bd070 con 0x7ffbf00724f0 2024-05-28T14:15:23.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.406+0000 7ffbe7400640 1 -- 172.21.15.42:0/1235635305 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00b6a70 con 0x7ffbf00724f0 2024-05-28T14:15:23.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.406+0000 7ffbf6600640 1 -- 172.21.15.42:0/1235635305 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbf00aa860 con 0x7ffbf00724f0 2024-05-28T14:15:23.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.406+0000 7ffbf6600640 1 -- 172.21.15.42:0/1235635305 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbf00aae20 con 0x7ffbf00724f0 2024-05-28T14:15:23.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.406+0000 7ffbf6600640 1 -- 172.21.15.42:0/1235635305 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbf00738b0 con 0x7ffbf00724f0 2024-05-28T14:15:23.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.410+0000 7ffbe7400640 1 -- 172.21.15.42:0/1235635305 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 45) ==== 90741+0+0 (secure 0 0 0) 0x7ffbe00b6c10 con 0x7ffbf00724f0 2024-05-28T14:15:23.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.414+0000 7ffbe7400640 1 --2- 172.21.15.42:0/1235635305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffbd006a230 0x7ffbd006c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:23.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.414+0000 7ffbe7400640 1 -- 172.21.15.42:0/1235635305 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7ffbe0120ef0 con 0x7ffbf00724f0 2024-05-28T14:15:23.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.418+0000 7ffbe7400640 1 -- 172.21.15.42:0/1235635305 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffbe0016010 con 0x7ffbf00724f0 2024-05-28T14:15:23.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.442+0000 7ffbeea00640 1 --2- 172.21.15.42:0/1235635305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffbd006a230 0x7ffbd006c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:23.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.506+0000 7ffbeea00640 1 --2- 172.21.15.42:0/1235635305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffbd006a230 0x7ffbd006c6f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ffbf00b0910 tx=0x7ffbdc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:23.990+0000 7ffbf6600640 1 -- 172.21.15.42:0/1235635305 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone6274", "target": ["mon-mgr", ""]}) -- 0x7ffbf0166220 con 0x7ffbd006a230 2024-05-28T14:15:24.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:24.102+0000 7ffbe7400640 1 -- 172.21.15.42:0/1235635305 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mgrmap(e 46) ==== 90894+0+0 (secure 0 0 0) 0x7ffbe00eef00 con 0x7ffbf00724f0 2024-05-28T14:15:24.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:24.546+0000 7ffbe7400640 1 -- 172.21.15.42:0/1235635305 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffbf0166220 con 0x7ffbd006a230 2024-05-28T14:15:24.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:24.558+0000 7ffbf6600640 1 -- 172.21.15.42:0/1235635305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffbd006a230 msgr2=0x7ffbd006c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:24.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:24.558+0000 7ffbf6600640 1 --2- 172.21.15.42:0/1235635305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffbd006a230 0x7ffbd006c6f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ffbf00b0910 tx=0x7ffbdc008040 comp rx=0 tx=0).stop 2024-05-28T14:15:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:24.558+0000 7ffbf6600640 1 -- 172.21.15.42:0/1235635305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbf00724f0 msgr2=0x7ffbf00cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:24.558+0000 7ffbf6600640 1 --2- 172.21.15.42:0/1235635305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbf00724f0 0x7ffbf00cc870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00af040 tx=0x7ffbe001ac60 comp rx=0 tx=0).stop 2024-05-28T14:15:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:24.562+0000 7ffbf6600640 1 -- 172.21.15.42:0/1235635305 shutdown_connections 2024-05-28T14:15:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:24.562+0000 7ffbf6600640 1 --2- 172.21.15.42:0/1235635305 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffbd006a230 0x7ffbd006c6f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:24.562+0000 7ffbf6600640 1 --2- 172.21.15.42:0/1235635305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbf0162d10 0x7ffbf006dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:24.562+0000 7ffbf6600640 1 --2- 172.21.15.42:0/1235635305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbf00cfb40 0x7ffbf015c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:24.562+0000 7ffbf6600640 1 --2- 172.21.15.42:0/1235635305 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbf00724f0 0x7ffbf00cc870 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:24.562+0000 7ffbf6600640 1 -- 172.21.15.42:0/1235635305 >> 172.21.15.42:0/1235635305 conn(0x7ffbf00b21f0 msgr2=0x7ffbf00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:24.562+0000 7ffbf6600640 1 -- 172.21.15.42:0/1235635305 shutdown_connections 2024-05-28T14:15:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:24.562+0000 7ffbf6600640 1 -- 172.21.15.42:0/1235635305 wait complete. 2024-05-28T14:15:24.666 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone6275 2024-05-28T14:15:25.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.158+0000 7fd6b8200640 1 Processor -- start 2024-05-28T14:15:25.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.158+0000 7fd6b8200640 1 -- start start 2024-05-28T14:15:25.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.158+0000 7fd6b8200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6b015c770 0x7fd6b00d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:25.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.158+0000 7fd6b8200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd6b00c6030 0x7fd6b00c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:25.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.158+0000 7fd6b8200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b015cb50 con 0x7fd6b00cdb00 2024-05-28T14:15:25.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.158+0000 7fd6b8200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd6b009b140 con 0x7fd6b00c6030 2024-05-28T14:15:25.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.158+0000 7fd6b8200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b00ce3a0 con 0x7fd6b015c770 2024-05-28T14:15:25.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.162+0000 7fd6b6200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd6b00c6030 0x7fd6b00c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:25.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.162+0000 7fd6b6200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd6b00c6030 0x7fd6b00c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:47442/0 (socket says 172.21.15.42:47442) 2024-05-28T14:15:25.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.162+0000 7fd6b6200640 1 -- 172.21.15.42:0/1685972989 learned_addr learned my addr 172.21.15.42:0/1685972989 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:25.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.162+0000 7fd6b5800640 1 --2- 172.21.15.42:0/1685972989 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6b015c770 0x7fd6b00d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:25.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.162+0000 7fd6b5800640 1 -- 172.21.15.42:0/1685972989 >> v2:172.21.15.63:3300/0 conn(0x7fd6b00c6030 msgr2=0x7fd6b00c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:25.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.162+0000 7fd6b5800640 1 --2- 172.21.15.42:0/1685972989 >> v2:172.21.15.63:3300/0 conn(0x7fd6b00c6030 0x7fd6b00c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:25.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.162+0000 7fd6b5800640 1 -- 172.21.15.42:0/1685972989 >> v1:172.21.15.63:6789/0 conn(0x7fd6b00cdb00 legacy=0x7fd6b009b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:25.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.162+0000 7fd6b5800640 1 -- 172.21.15.42:0/1685972989 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b00ce0c0 con 0x7fd6b015c770 2024-05-28T14:15:25.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.182+0000 7fd6b5800640 1 --2- 172.21.15.42:0/1685972989 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6b015c770 0x7fd6b00d02d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40097c0 tx=0x7fd6a401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e2b2239aafca7e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:25.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.182+0000 7fd6afe00640 1 -- 172.21.15.42:0/1685972989 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a40b2070 con 0x7fd6b015c770 2024-05-28T14:15:25.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.182+0000 7fd6afe00640 1 -- 172.21.15.42:0/1685972989 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a40ad640 con 0x7fd6b015c770 2024-05-28T14:15:25.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.182+0000 7fd6afe00640 1 -- 172.21.15.42:0/1685972989 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a40ada10 con 0x7fd6b015c770 2024-05-28T14:15:25.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.186+0000 7fd6b8200640 1 -- 172.21.15.42:0/1685972989 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6b015c770 msgr2=0x7fd6b00d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:25.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.186+0000 7fd6b8200640 1 --2- 172.21.15.42:0/1685972989 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6b015c770 0x7fd6b00d02d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40097c0 tx=0x7fd6a401c4c0 comp rx=0 tx=0).stop 2024-05-28T14:15:25.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.190+0000 7fd6b8200640 1 -- 172.21.15.42:0/1685972989 shutdown_connections 2024-05-28T14:15:25.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.190+0000 7fd6b8200640 1 --2- 172.21.15.42:0/1685972989 >> v2:172.21.15.63:3300/0 conn(0x7fd6b00c6030 0x7fd6b00c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:25.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.190+0000 7fd6b8200640 1 --2- 172.21.15.42:0/1685972989 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6b015c770 0x7fd6b00d02d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:25.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.190+0000 7fd6b8200640 1 -- 172.21.15.42:0/1685972989 >> 172.21.15.42:0/1685972989 conn(0x7fd6b00b3fd0 msgr2=0x7fd6b00b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:25.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.190+0000 7fd6b8200640 1 -- 172.21.15.42:0/1685972989 shutdown_connections 2024-05-28T14:15:25.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.190+0000 7fd6b8200640 1 -- 172.21.15.42:0/1685972989 wait complete. 2024-05-28T14:15:25.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.194+0000 7fd6b8200640 1 Processor -- start 2024-05-28T14:15:25.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.194+0000 7fd6b8200640 1 -- start start 2024-05-28T14:15:25.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.194+0000 7fd6b8200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6b00c6030 0x7fd6b009d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:25.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.194+0000 7fd6b8200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b00cdb00 0x7fd6b00b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:25.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.194+0000 7fd6b8200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6b015c770 0x7fd6b0154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:25.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b8200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b006cb00 con 0x7fd6b00cdb00 2024-05-28T14:15:25.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b8200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b009bce0 con 0x7fd6b00c6030 2024-05-28T14:15:25.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b8200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b006cc80 con 0x7fd6b015c770 2024-05-28T14:15:25.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b5800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6b00c6030 0x7fd6b009d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:25.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b5800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6b00c6030 0x7fd6b009d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:47446/0 (socket says 172.21.15.42:47446) 2024-05-28T14:15:25.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b5800640 1 -- 172.21.15.42:0/545052111 learned_addr learned my addr 172.21.15.42:0/545052111 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:25.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b4e00640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b00cdb00 0x7fd6b00b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:25.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b6200640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6b015c770 0x7fd6b0154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:25.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b4e00640 1 -- 172.21.15.42:0/545052111 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6b015c770 msgr2=0x7fd6b0154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:25.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b4e00640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6b015c770 0x7fd6b0154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:25.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b4e00640 1 -- 172.21.15.42:0/545052111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6b00c6030 msgr2=0x7fd6b009d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:25.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b4e00640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6b00c6030 0x7fd6b009d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:25.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b4e00640 1 -- 172.21.15.42:0/545052111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b015bda0 con 0x7fd6b00cdb00 2024-05-28T14:15:25.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b6200640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6b015c770 0x7fd6b0154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:15:25.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6b4e00640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b00cdb00 0x7fd6b00b5100 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fd69c000f70 tx=0x7fd69c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:25.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.198+0000 7fd6ad600640 1 -- 172.21.15.42:0/545052111 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c017070 con 0x7fd6b00cdb00 2024-05-28T14:15:25.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.202+0000 7fd6ad600640 1 -- 172.21.15.42:0/545052111 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd69c0052d0 con 0x7fd6b00cdb00 2024-05-28T14:15:25.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.202+0000 7fd6ad600640 1 -- 172.21.15.42:0/545052111 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0054c0 con 0x7fd6b00cdb00 2024-05-28T14:15:25.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.202+0000 7fd6b8200640 1 -- 172.21.15.42:0/545052111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b009db60 con 0x7fd6b00cdb00 2024-05-28T14:15:25.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.202+0000 7fd6b8200640 1 -- 172.21.15.42:0/545052111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b009e120 con 0x7fd6b00cdb00 2024-05-28T14:15:25.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.206+0000 7fd6b8200640 1 -- 172.21.15.42:0/545052111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd68c0052a0 con 0x7fd6b00cdb00 2024-05-28T14:15:25.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.210+0000 7fd6ad600640 1 -- 172.21.15.42:0/545052111 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 46) ==== 90894+0+0 (secure 0 0 0) 0x7fd69c01a070 con 0x7fd6b00cdb00 2024-05-28T14:15:25.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.210+0000 7fd6ad600640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd69006a390 0x7fd69006c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:25.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.210+0000 7fd6ad600640 1 -- 172.21.15.42:0/545052111 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fd69c084dd0 con 0x7fd6b00cdb00 2024-05-28T14:15:25.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.222+0000 7fd6ad600640 1 -- 172.21.15.42:0/545052111 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd6b015bda0 con 0x7fd6b00cdb00 2024-05-28T14:15:25.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.238+0000 7fd6b5800640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd69006a390 0x7fd69006c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:25.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.250+0000 7fd6b5800640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd69006a390 0x7fd69006c850 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4000c10 tx=0x7fd6a40a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:25.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:25.694+0000 7fd6b8200640 1 -- 172.21.15.42:0/545052111 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone6275", "target": ["mon-mgr", ""]}) -- 0x7fd68c000c90 con 0x7fd69006a390 2024-05-28T14:15:26.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:26.558+0000 7fd6ad600640 1 -- 172.21.15.42:0/545052111 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd68c000c90 con 0x7fd69006a390 2024-05-28T14:15:26.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:26.582+0000 7fd6b8200640 1 -- 172.21.15.42:0/545052111 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd69006a390 msgr2=0x7fd69006c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:26.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:26.582+0000 7fd6b8200640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd69006a390 0x7fd69006c850 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4000c10 tx=0x7fd6a40a7070 comp rx=0 tx=0).stop 2024-05-28T14:15:26.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:26.582+0000 7fd6b8200640 1 -- 172.21.15.42:0/545052111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b00cdb00 msgr2=0x7fd6b00b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:26.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:26.582+0000 7fd6b8200640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b00cdb00 0x7fd6b00b5100 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fd69c000f70 tx=0x7fd69c00c890 comp rx=0 tx=0).stop 2024-05-28T14:15:26.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:26.582+0000 7fd6b8200640 1 -- 172.21.15.42:0/545052111 shutdown_connections 2024-05-28T14:15:26.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:26.582+0000 7fd6b8200640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd69006a390 0x7fd69006c850 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:26.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:26.582+0000 7fd6b8200640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6b015c770 0x7fd6b0154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:26.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:26.582+0000 7fd6b8200640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b00cdb00 0x7fd6b00b5100 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:26.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:26.582+0000 7fd6b8200640 1 --2- 172.21.15.42:0/545052111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6b00c6030 0x7fd6b009d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:26.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:26.586+0000 7fd6b8200640 1 -- 172.21.15.42:0/545052111 >> 172.21.15.42:0/545052111 conn(0x7fd6b00b3fd0 msgr2=0x7fd6b00cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:26.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:26.586+0000 7fd6b8200640 1 -- 172.21.15.42:0/545052111 shutdown_connections 2024-05-28T14:15:26.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:26.586+0000 7fd6b8200640 1 -- 172.21.15.42:0/545052111 wait complete. 2024-05-28T14:15:26.673 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone6276 2024-05-28T14:15:27.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.098+0000 7f2692c00640 1 Processor -- start 2024-05-28T14:15:27.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.102+0000 7f2692c00640 1 -- start start 2024-05-28T14:15:27.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.102+0000 7f2692c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f268c162d10 0x7f268c06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:27.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.102+0000 7f2692c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f268c0cfb40 0x7f268c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:27.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.102+0000 7f2692c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f268c166800 con 0x7f268c072320 2024-05-28T14:15:27.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.102+0000 7f2692c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f268c1630f0 con 0x7f268c0cfb40 2024-05-28T14:15:27.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.102+0000 7f2692c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f268c0b4500 con 0x7f268c162d10 2024-05-28T14:15:27.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.106+0000 7f2690c00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f268c072320 0x7f268c06dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:47230/0 (socket says 172.21.15.42:47230) 2024-05-28T14:15:27.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.106+0000 7f2690c00640 1 -- 172.21.15.42:0/1129099118 learned_addr learned my addr 172.21.15.42:0/1129099118 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:15:27.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.102+0000 7f268be00640 1 --2- 172.21.15.42:0/1129099118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f268c162d10 0x7f268c06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:27.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.106+0000 7f268b400640 1 --2- 172.21.15.42:0/1129099118 >> v2:172.21.15.63:3300/0 conn(0x7f268c0cfb40 0x7f268c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:27.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.106+0000 7f268aa00640 1 -- 172.21.15.42:0/1129099118 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 597676177 0 0) 0x7f268c166800 con 0x7f268c072320 2024-05-28T14:15:27.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.106+0000 7f268aa00640 1 -- 172.21.15.42:0/1129099118 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2670003530 con 0x7f268c072320 2024-05-28T14:15:27.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.106+0000 7f268b400640 1 -- 172.21.15.42:0/1129099118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f268c162d10 msgr2=0x7f268c06f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:27.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.106+0000 7f268b400640 1 --2- 172.21.15.42:0/1129099118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f268c162d10 0x7f268c06f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:27.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.106+0000 7f268b400640 1 -- 172.21.15.42:0/1129099118 >> v1:172.21.15.63:6789/0 conn(0x7f268c072320 legacy=0x7f268c06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:27.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.106+0000 7f268b400640 1 -- 172.21.15.42:0/1129099118 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268c06f8c0 con 0x7f268c0cfb40 2024-05-28T14:15:27.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.110+0000 7f268b400640 1 --2- 172.21.15.42:0/1129099118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f268c0cfb40 0x7f268c071de0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2684009d90 tx=0x7f268401c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f85c8b56a6119d0b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:27.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.110+0000 7f268aa00640 1 -- 172.21.15.42:0/1129099118 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26840b2070 con 0x7f268c0cfb40 2024-05-28T14:15:27.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.114+0000 7f268aa00640 1 -- 172.21.15.42:0/1129099118 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26840a5750 con 0x7f268c0cfb40 2024-05-28T14:15:27.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.114+0000 7f268aa00640 1 -- 172.21.15.42:0/1129099118 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26840a5b20 con 0x7f268c0cfb40 2024-05-28T14:15:27.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.114+0000 7f2692c00640 1 -- 172.21.15.42:0/1129099118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f268c0cfb40 msgr2=0x7f268c071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:27.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.114+0000 7f2692c00640 1 --2- 172.21.15.42:0/1129099118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f268c0cfb40 0x7f268c071de0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2684009d90 tx=0x7f268401c9b0 comp rx=0 tx=0).stop 2024-05-28T14:15:27.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.114+0000 7f2692c00640 1 -- 172.21.15.42:0/1129099118 shutdown_connections 2024-05-28T14:15:27.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.114+0000 7f2692c00640 1 --2- 172.21.15.42:0/1129099118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f268c0cfb40 0x7f268c071de0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:27.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.114+0000 7f2692c00640 1 --2- 172.21.15.42:0/1129099118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f268c162d10 0x7f268c06f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:27.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.114+0000 7f2692c00640 1 -- 172.21.15.42:0/1129099118 >> 172.21.15.42:0/1129099118 conn(0x7f268c0b21f0 msgr2=0x7f268c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:27.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.114+0000 7f2692c00640 1 -- 172.21.15.42:0/1129099118 shutdown_connections 2024-05-28T14:15:27.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.114+0000 7f2692c00640 1 -- 172.21.15.42:0/1129099118 wait complete. 2024-05-28T14:15:27.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.118+0000 7f2692c00640 1 Processor -- start 2024-05-28T14:15:27.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.118+0000 7f2692c00640 1 -- start start 2024-05-28T14:15:27.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.122+0000 7f2692c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f268c072320 0x7f268c153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:27.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.122+0000 7f2692c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f268c0cfb40 0x7f268c163c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:27.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.122+0000 7f2692c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f268c162d10 0x7f268c154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:27.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.122+0000 7f268be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f268c072320 0x7f268c153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:27.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.122+0000 7f268be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f268c072320 0x7f268c153bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:47472/0 (socket says 172.21.15.42:47472) 2024-05-28T14:15:27.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.122+0000 7f268be00640 1 -- 172.21.15.42:0/2771251676 learned_addr learned my addr 172.21.15.42:0/2771251676 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:27.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.126+0000 7f2692c00640 1 -- 172.21.15.42:0/2771251676 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1675a0 con 0x7f268c0cfb40 2024-05-28T14:15:27.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.126+0000 7f2692c00640 1 -- 172.21.15.42:0/2771251676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f268c0afec0 con 0x7f268c072320 2024-05-28T14:15:27.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.126+0000 7f2692c00640 1 -- 172.21.15.42:0/2771251676 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f268c164920 con 0x7f268c162d10 2024-05-28T14:15:27.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.126+0000 7f268b400640 1 --2- 172.21.15.42:0/2771251676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f268c0cfb40 0x7f268c163c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:27.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.126+0000 7f2690c00640 1 --2- 172.21.15.42:0/2771251676 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f268c162d10 0x7f268c154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:27.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.130+0000 7f268be00640 1 -- 172.21.15.42:0/2771251676 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f268c162d10 msgr2=0x7f268c154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:27.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.130+0000 7f268be00640 1 --2- 172.21.15.42:0/2771251676 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f268c162d10 0x7f268c154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.130+0000 7f268be00640 1 -- 172.21.15.42:0/2771251676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f268c0cfb40 msgr2=0x7f268c163c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.130+0000 7f268be00640 1 --2- 172.21.15.42:0/2771251676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f268c0cfb40 0x7f268c163c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.130+0000 7f268be00640 1 -- 172.21.15.42:0/2771251676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268c164b70 con 0x7f268c072320 2024-05-28T14:15:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.130+0000 7f268be00640 1 --2- 172.21.15.42:0/2771251676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f268c072320 0x7f268c153bb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f267c00d5f0 tx=0x7f267c00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.130+0000 7f2683e00640 1 -- 172.21.15.42:0/2771251676 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c0125f0 con 0x7f268c072320 2024-05-28T14:15:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.130+0000 7f2683e00640 1 -- 172.21.15.42:0/2771251676 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f267c019070 con 0x7f268c072320 2024-05-28T14:15:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.130+0000 7f2683e00640 1 -- 172.21.15.42:0/2771251676 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c0129d0 con 0x7f268c072320 2024-05-28T14:15:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.130+0000 7f2692c00640 1 -- 172.21.15.42:0/2771251676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268c164e50 con 0x7f268c072320 2024-05-28T14:15:27.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.130+0000 7f2692c00640 1 -- 172.21.15.42:0/2771251676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268c1540f0 con 0x7f268c072320 2024-05-28T14:15:27.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.138+0000 7f2692c00640 1 -- 172.21.15.42:0/2771251676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f268c0738b0 con 0x7f268c072320 2024-05-28T14:15:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.138+0000 7f2683e00640 1 -- 172.21.15.42:0/2771251676 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 46) ==== 90894+0+0 (secure 0 0 0) 0x7f267c012b70 con 0x7f268c072320 2024-05-28T14:15:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.142+0000 7f2683e00640 1 --2- 172.21.15.42:0/2771251676 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f266c06a3e0 0x7f266c06c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:27.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.142+0000 7f2683e00640 1 -- 172.21.15.42:0/2771251676 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f267c085cf0 con 0x7f268c072320 2024-05-28T14:15:27.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.154+0000 7f2683e00640 1 -- 172.21.15.42:0/2771251676 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f267c057e90 con 0x7f268c072320 2024-05-28T14:15:27.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.190+0000 7f268b400640 1 --2- 172.21.15.42:0/2771251676 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f266c06a3e0 0x7f266c06c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:27.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.234+0000 7f268b400640 1 --2- 172.21.15.42:0/2771251676 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f266c06a3e0 0x7f266c06c8a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f26840af000 tx=0x7f26840a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:27.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:27.766+0000 7f2692c00640 1 -- 172.21.15.42:0/2771251676 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone6276", "target": ["mon-mgr", ""]}) -- 0x7f268c04fdd0 con 0x7f266c06a3e0 2024-05-28T14:15:28.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.318+0000 7f2683e00640 1 -- 172.21.15.42:0/2771251676 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f268c04fdd0 con 0x7f266c06a3e0 2024-05-28T14:15:28.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.342+0000 7f2692c00640 1 -- 172.21.15.42:0/2771251676 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f266c06a3e0 msgr2=0x7f266c06c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:28.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.342+0000 7f2692c00640 1 --2- 172.21.15.42:0/2771251676 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f266c06a3e0 0x7f266c06c8a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f26840af000 tx=0x7f26840a7070 comp rx=0 tx=0).stop 2024-05-28T14:15:28.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.342+0000 7f2692c00640 1 -- 172.21.15.42:0/2771251676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f268c072320 msgr2=0x7f268c153bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:28.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.342+0000 7f2692c00640 1 --2- 172.21.15.42:0/2771251676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f268c072320 0x7f268c153bb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f267c00d5f0 tx=0x7f267c00dab0 comp rx=0 tx=0).stop 2024-05-28T14:15:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.346+0000 7f2692c00640 1 -- 172.21.15.42:0/2771251676 shutdown_connections 2024-05-28T14:15:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.346+0000 7f2692c00640 1 --2- 172.21.15.42:0/2771251676 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f266c06a3e0 0x7f266c06c8a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.346+0000 7f2692c00640 1 --2- 172.21.15.42:0/2771251676 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f268c162d10 0x7f268c154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.346+0000 7f2692c00640 1 --2- 172.21.15.42:0/2771251676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f268c0cfb40 0x7f268c163c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.346+0000 7f2692c00640 1 --2- 172.21.15.42:0/2771251676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f268c072320 0x7f268c153bb0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.346+0000 7f2692c00640 1 -- 172.21.15.42:0/2771251676 >> 172.21.15.42:0/2771251676 conn(0x7f268c0b21f0 msgr2=0x7f268c0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.346+0000 7f2692c00640 1 -- 172.21.15.42:0/2771251676 shutdown_connections 2024-05-28T14:15:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.346+0000 7f2692c00640 1 -- 172.21.15.42:0/2771251676 wait complete. 2024-05-28T14:15:28.442 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone6277 2024-05-28T14:15:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.886+0000 7fca1a400640 1 Processor -- start 2024-05-28T14:15:28.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.890+0000 7fca1a400640 1 -- start start 2024-05-28T14:15:28.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.890+0000 7fca1a400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fca140d0460 0x7fca14166800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:28.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.894+0000 7fca1a400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca140b31d0 0x7fca14166d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:28.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.894+0000 7fca1a400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fca140d3a20 0x7fca140d3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:28.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.894+0000 7fca1a400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fca140d0840 con 0x7fca140d0460 2024-05-28T14:15:28.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.894+0000 7fca1a400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fca140b1e90 con 0x7fca140d3a20 2024-05-28T14:15:28.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.894+0000 7fca1a400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fca140b2130 con 0x7fca140b31d0 2024-05-28T14:15:28.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.898+0000 7fca13e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fca140d3a20 0x7fca140d3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.898+0000 7fca13e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fca140d3a20 0x7fca140d3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:46640/0 (socket says 172.21.15.42:46640) 2024-05-28T14:15:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.898+0000 7fca13e00640 1 -- 172.21.15.42:0/759182719 learned_addr learned my addr 172.21.15.42:0/759182719 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.898+0000 7fca13400640 1 --2- 172.21.15.42:0/759182719 >> v2:172.21.15.42:3300/0 conn(0x7fca140d0460 0x7fca14166800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.898+0000 7fca12a00640 1 --2- 172.21.15.42:0/759182719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca140b31d0 0x7fca14166d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.902+0000 7fca13e00640 1 -- 172.21.15.42:0/759182719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca140b31d0 msgr2=0x7fca14166d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.902+0000 7fca13e00640 1 --2- 172.21.15.42:0/759182719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca140b31d0 0x7fca14166d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.902+0000 7fca13e00640 1 -- 172.21.15.42:0/759182719 >> v2:172.21.15.42:3300/0 conn(0x7fca140d0460 msgr2=0x7fca14166800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.902+0000 7fca13e00640 1 --2- 172.21.15.42:0/759182719 >> v2:172.21.15.42:3300/0 conn(0x7fca140d0460 0x7fca14166800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.902+0000 7fca13e00640 1 -- 172.21.15.42:0/759182719 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca140b2410 con 0x7fca140d3a20 2024-05-28T14:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.902+0000 7fca12a00640 1 --2- 172.21.15.42:0/759182719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca140b31d0 0x7fca14166d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:15:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.906+0000 7fca13400640 1 --2- 172.21.15.42:0/759182719 >> v2:172.21.15.42:3300/0 conn(0x7fca140d0460 0x7fca14166800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:15:28.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.910+0000 7fca13e00640 1 --2- 172.21.15.42:0/759182719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca140d3a20 0x7fca140d3e60 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fca140d2230 tx=0x7fca0c01c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9265d504ed738428 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:28.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.910+0000 7fca12000640 1 -- 172.21.15.42:0/759182719 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c0a57c0 con 0x7fca140d3a20 2024-05-28T14:15:28.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.910+0000 7fca12000640 1 -- 172.21.15.42:0/759182719 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0c0a5960 con 0x7fca140d3a20 2024-05-28T14:15:28.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.910+0000 7fca12000640 1 -- 172.21.15.42:0/759182719 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c0a5d30 con 0x7fca140d3a20 2024-05-28T14:15:28.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.910+0000 7fca1a400640 1 -- 172.21.15.42:0/759182719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca140d3a20 msgr2=0x7fca140d3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:28.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.910+0000 7fca1a400640 1 --2- 172.21.15.42:0/759182719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca140d3a20 0x7fca140d3e60 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fca140d2230 tx=0x7fca0c01c9b0 comp rx=0 tx=0).stop 2024-05-28T14:15:28.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.914+0000 7fca1a400640 1 -- 172.21.15.42:0/759182719 shutdown_connections 2024-05-28T14:15:28.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.914+0000 7fca1a400640 1 --2- 172.21.15.42:0/759182719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca140d3a20 0x7fca140d3e60 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:28.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.914+0000 7fca1a400640 1 --2- 172.21.15.42:0/759182719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca140b31d0 0x7fca14166d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:28.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.914+0000 7fca1a400640 1 --2- 172.21.15.42:0/759182719 >> v2:172.21.15.42:3300/0 conn(0x7fca140d0460 0x7fca14166800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:28.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.914+0000 7fca1a400640 1 -- 172.21.15.42:0/759182719 >> 172.21.15.42:0/759182719 conn(0x7fca140d11f0 msgr2=0x7fca140d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:28.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.914+0000 7fca1a400640 1 -- 172.21.15.42:0/759182719 shutdown_connections 2024-05-28T14:15:28.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.914+0000 7fca1a400640 1 -- 172.21.15.42:0/759182719 wait complete. 2024-05-28T14:15:28.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.918+0000 7fca1a400640 1 Processor -- start 2024-05-28T14:15:28.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.918+0000 7fca1a400640 1 -- start start 2024-05-28T14:15:28.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.918+0000 7fca1a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca140b31d0 0x7fca140c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:28.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.918+0000 7fca1a400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca140d0460 0x7fca140c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:28.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.918+0000 7fca1a400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca140d3a20 0x7fca141521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:28.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.918+0000 7fca1a400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fca1409ce40 con 0x7fca140b31d0 2024-05-28T14:15:28.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.918+0000 7fca1a400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fca140d43a0 con 0x7fca140d0460 2024-05-28T14:15:28.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.918+0000 7fca1a400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fca1409cfc0 con 0x7fca140d3a20 2024-05-28T14:15:28.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.922+0000 7fca13400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca140b31d0 0x7fca140c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:28.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.922+0000 7fca13400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca140b31d0 0x7fca140c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36512/0 (socket says 172.21.15.42:36512) 2024-05-28T14:15:28.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.922+0000 7fca13400640 1 -- 172.21.15.42:0/1038707199 learned_addr learned my addr 172.21.15.42:0/1038707199 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:28.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.922+0000 7fca12a00640 1 --2- 172.21.15.42:0/1038707199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca140d0460 0x7fca140c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:28.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.922+0000 7fca13e00640 1 --2- 172.21.15.42:0/1038707199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca140d3a20 0x7fca141521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:28.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.922+0000 7fca13e00640 1 -- 172.21.15.42:0/1038707199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca140d0460 msgr2=0x7fca140c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:28.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.922+0000 7fca13e00640 1 --2- 172.21.15.42:0/1038707199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca140d0460 0x7fca140c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:28.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.922+0000 7fca13e00640 1 -- 172.21.15.42:0/1038707199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca140b31d0 msgr2=0x7fca140c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:28.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.922+0000 7fca13e00640 1 --2- 172.21.15.42:0/1038707199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca140b31d0 0x7fca140c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:28.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.922+0000 7fca13e00640 1 -- 172.21.15.42:0/1038707199 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1415d1d0 con 0x7fca140d3a20 2024-05-28T14:15:28.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.926+0000 7fca13e00640 1 --2- 172.21.15.42:0/1038707199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca140d3a20 0x7fca141521e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0ab000 tx=0x7fca0c019750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:28.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.926+0000 7fca0b400640 1 -- 172.21.15.42:0/1038707199 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c0b8470 con 0x7fca140d3a20 2024-05-28T14:15:28.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.930+0000 7fca0b400640 1 -- 172.21.15.42:0/1038707199 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0c0b8610 con 0x7fca140d3a20 2024-05-28T14:15:28.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.930+0000 7fca0b400640 1 -- 172.21.15.42:0/1038707199 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c0b8a00 con 0x7fca140d3a20 2024-05-28T14:15:28.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.930+0000 7fca1a400640 1 -- 172.21.15.42:0/1038707199 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1415d470 con 0x7fca140d3a20 2024-05-28T14:15:28.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.930+0000 7fca1a400640 1 -- 172.21.15.42:0/1038707199 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1415da30 con 0x7fca140d3a20 2024-05-28T14:15:28.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.934+0000 7fca1a400640 1 -- 172.21.15.42:0/1038707199 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca1415dc00 con 0x7fca140d3a20 2024-05-28T14:15:28.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.938+0000 7fca0b400640 1 -- 172.21.15.42:0/1038707199 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 46) ==== 90894+0+0 (secure 0 0 0) 0x7fca0c0c0020 con 0x7fca140d3a20 2024-05-28T14:15:28.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.938+0000 7fca0b400640 1 --2- 172.21.15.42:0/1038707199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9f406a460 0x7fc9f406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:28.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.942+0000 7fca0b400640 1 -- 172.21.15.42:0/1038707199 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fca0c1200e0 con 0x7fca140d3a20 2024-05-28T14:15:28.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.950+0000 7fca0b400640 1 -- 172.21.15.42:0/1038707199 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fca0c0c7020 con 0x7fca140d3a20 2024-05-28T14:15:28.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.966+0000 7fca13400640 1 --2- 172.21.15.42:0/1038707199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9f406a460 0x7fc9f406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:28.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:28.990+0000 7fca13400640 1 --2- 172.21.15.42:0/1038707199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9f406a460 0x7fc9f406c920 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fca04009a10 tx=0x7fca04008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:29.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:29.550+0000 7fca1a400640 1 -- 172.21.15.42:0/1038707199 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone6277", "target": ["mon-mgr", ""]}) -- 0x7fca1404fdd0 con 0x7fc9f406a460 2024-05-28T14:15:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:30.670+0000 7fca0b400640 1 -- 172.21.15.42:0/1038707199 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fca1404fdd0 con 0x7fc9f406a460 2024-05-28T14:15:30.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:30.678+0000 7fca1a400640 1 -- 172.21.15.42:0/1038707199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9f406a460 msgr2=0x7fc9f406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:30.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:30.678+0000 7fca1a400640 1 --2- 172.21.15.42:0/1038707199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9f406a460 0x7fc9f406c920 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fca04009a10 tx=0x7fca04008040 comp rx=0 tx=0).stop 2024-05-28T14:15:30.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:30.678+0000 7fca1a400640 1 -- 172.21.15.42:0/1038707199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca140d3a20 msgr2=0x7fca141521e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:30.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:30.678+0000 7fca1a400640 1 --2- 172.21.15.42:0/1038707199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca140d3a20 0x7fca141521e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0ab000 tx=0x7fca0c019750 comp rx=0 tx=0).stop 2024-05-28T14:15:30.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:30.678+0000 7fca1a400640 1 -- 172.21.15.42:0/1038707199 shutdown_connections 2024-05-28T14:15:30.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:30.678+0000 7fca1a400640 1 --2- 172.21.15.42:0/1038707199 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9f406a460 0x7fc9f406c920 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:30.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:30.678+0000 7fca1a400640 1 --2- 172.21.15.42:0/1038707199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca140d3a20 0x7fca141521e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:30.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:30.678+0000 7fca1a400640 1 --2- 172.21.15.42:0/1038707199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca140d0460 0x7fca140c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:30.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:30.678+0000 7fca1a400640 1 --2- 172.21.15.42:0/1038707199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca140b31d0 0x7fca140c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:30.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:30.678+0000 7fca1a400640 1 -- 172.21.15.42:0/1038707199 >> 172.21.15.42:0/1038707199 conn(0x7fca140d11f0 msgr2=0x7fca140b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:30.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:30.678+0000 7fca1a400640 1 -- 172.21.15.42:0/1038707199 shutdown_connections 2024-05-28T14:15:30.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:30.682+0000 7fca1a400640 1 -- 172.21.15.42:0/1038707199 wait complete. 2024-05-28T14:15:35.801 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone6271 2024-05-28T14:15:36.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.322+0000 7f1ce1800640 1 Processor -- start 2024-05-28T14:15:36.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.322+0000 7f1ce1800640 1 -- start start 2024-05-28T14:15:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.326+0000 7f1ce1800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1cdc162d10 0x7f1cdc06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.326+0000 7f1ce1800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cdc166800 con 0x7f1cdc0cfb40 2024-05-28T14:15:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.326+0000 7f1ce1800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cdc06f8c0 con 0x7f1cdc0724f0 2024-05-28T14:15:36.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.326+0000 7f1ce1800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1cdc1630f0 con 0x7f1cdc162d10 2024-05-28T14:15:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.330+0000 7f1cdb400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f1cdc0724f0 0x7f1cdc0b34f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:47390/0 (socket says 172.21.15.42:47390) 2024-05-28T14:15:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.330+0000 7f1cdb400640 1 -- 172.21.15.42:0/3524756939 learned_addr learned my addr 172.21.15.42:0/3524756939 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:15:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.330+0000 7f1cdaa00640 1 --2- 172.21.15.42:0/3524756939 >> v2:172.21.15.63:3300/0 conn(0x7f1cdc162d10 0x7f1cdc06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.330+0000 7f1cd9600640 1 -- 172.21.15.42:0/3524756939 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2798771471 0 0) 0x7f1cdc06f8c0 con 0x7f1cdc0724f0 2024-05-28T14:15:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.330+0000 7f1cd9600640 1 -- 172.21.15.42:0/3524756939 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cc0003530 con 0x7f1cdc0724f0 2024-05-28T14:15:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.330+0000 7f1cd9600640 1 -- 172.21.15.42:0/3524756939 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 677222217 0 0) 0x7f1cc0003530 con 0x7f1cdc0724f0 2024-05-28T14:15:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.330+0000 7f1cd9600640 1 -- 172.21.15.42:0/3524756939 >> v2:172.21.15.63:3300/0 conn(0x7f1cdc162d10 msgr2=0x7f1cdc06f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.330+0000 7f1cd9600640 1 --2- 172.21.15.42:0/3524756939 >> v2:172.21.15.63:3300/0 conn(0x7f1cdc162d10 0x7f1cdc06f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.330+0000 7f1cd9600640 1 -- 172.21.15.42:0/3524756939 >> v1:172.21.15.42:6789/0 conn(0x7f1cdc0cfb40 legacy=0x7f1cdc071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.330+0000 7f1cd9600640 1 -- 172.21.15.42:0/3524756939 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc166ac0 con 0x7f1cdc0724f0 2024-05-28T14:15:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.330+0000 7f1cd9600640 1 -- 172.21.15.42:0/3524756939 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f1ccc001680 con 0x7f1cdc0724f0 2024-05-28T14:15:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.330+0000 7f1cd9600640 1 -- 172.21.15.42:0/3524756939 >> v1:172.21.15.63:6789/0 conn(0x7f1cdc0724f0 legacy=0x7f1cdc0b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.334+0000 7f1cd9600640 1 --2- 172.21.15.42:0/3524756939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cc00039c0 0x7f1cc0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.334+0000 7f1cd9600640 1 --2- 172.21.15.42:0/3524756939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cc0004810 0x7f1cc0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.334+0000 7f1cd9600640 1 --2- 172.21.15.42:0/3524756939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc00051d0 0x7f1cc000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.334+0000 7f1cd9600640 1 -- 172.21.15.42:0/3524756939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc000bff0 con 0x7f1cc00039c0 2024-05-28T14:15:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.334+0000 7f1cd9600640 1 -- 172.21.15.42:0/3524756939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc000c1a0 con 0x7f1cc00051d0 2024-05-28T14:15:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.334+0000 7f1cd9600640 1 -- 172.21.15.42:0/3524756939 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc000c3a0 con 0x7f1cc0004810 2024-05-28T14:15:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.334+0000 7f1cdaa00640 1 --2- 172.21.15.42:0/3524756939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cc00039c0 0x7f1cc0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.334+0000 7f1cda000640 1 --2- 172.21.15.42:0/3524756939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cc0004810 0x7f1cc0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.334+0000 7f1cda000640 1 -- 172.21.15.42:0/3524756939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc00051d0 msgr2=0x7f1cc000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.334+0000 7f1cda000640 1 --2- 172.21.15.42:0/3524756939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc00051d0 0x7f1cc000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.334+0000 7f1cda000640 1 -- 172.21.15.42:0/3524756939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cc00039c0 msgr2=0x7f1cc0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.334+0000 7f1cda000640 1 --2- 172.21.15.42:0/3524756939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cc00039c0 0x7f1cc0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.334+0000 7f1cda000640 1 -- 172.21.15.42:0/3524756939 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc166ac0 con 0x7f1cc0004810 2024-05-28T14:15:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.334+0000 7f1cdb400640 1 -- 172.21.15.42:0/3524756939 reap_dead start 2024-05-28T14:15:36.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.342+0000 7f1cda000640 1 --2- 172.21.15.42:0/3524756939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cc0004810 0x7f1cc0004c90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8001780 tx=0x7f1cc801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bd01657c59744af server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:36.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.342+0000 7f1cd9600640 1 -- 172.21.15.42:0/3524756939 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80013a0 con 0x7f1cc0004810 2024-05-28T14:15:36.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.342+0000 7f1cd9600640 1 -- 172.21.15.42:0/3524756939 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc80052f0 con 0x7f1cc0004810 2024-05-28T14:15:36.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.342+0000 7f1cd9600640 1 -- 172.21.15.42:0/3524756939 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8003840 con 0x7f1cc0004810 2024-05-28T14:15:36.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.342+0000 7f1ce1800640 1 -- 172.21.15.42:0/3524756939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cc0004810 msgr2=0x7f1cc0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:36.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.342+0000 7f1ce1800640 1 --2- 172.21.15.42:0/3524756939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cc0004810 0x7f1cc0004c90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8001780 tx=0x7f1cc801c400 comp rx=0 tx=0).stop 2024-05-28T14:15:36.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.346+0000 7f1ce1800640 1 -- 172.21.15.42:0/3524756939 shutdown_connections 2024-05-28T14:15:36.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.346+0000 7f1ce1800640 1 --2- 172.21.15.42:0/3524756939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cc0004810 0x7f1cc0004c90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:36.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.346+0000 7f1ce1800640 1 -- 172.21.15.42:0/3524756939 >> 172.21.15.42:0/3524756939 conn(0x7f1cdc0b21f0 msgr2=0x7f1cdc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:36.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.346+0000 7f1ce1800640 1 -- 172.21.15.42:0/3524756939 shutdown_connections 2024-05-28T14:15:36.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.346+0000 7f1ce1800640 1 -- 172.21.15.42:0/3524756939 wait complete. 2024-05-28T14:15:36.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.350+0000 7f1ce1800640 1 Processor -- start 2024-05-28T14:15:36.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.350+0000 7f1ce1800640 1 -- start start 2024-05-28T14:15:36.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.350+0000 7f1ce1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cc0004810 0x7f1cdc151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:36.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.350+0000 7f1ce1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cdc165720 0x7f1cdc165b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:36.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.350+0000 7f1ce1800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cdc0b32e0 0x7f1cdc0b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:36.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.350+0000 7f1ce1800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc072370 con 0x7f1cc0004810 2024-05-28T14:15:36.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.350+0000 7f1ce1800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc0721f0 con 0x7f1cdc165720 2024-05-28T14:15:36.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.350+0000 7f1ce1800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1cdc1675a0 con 0x7f1cdc0b32e0 2024-05-28T14:15:36.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.354+0000 7f1cda000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cdc165720 0x7f1cdc165b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:36.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.354+0000 7f1cdb400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cdc0b32e0 0x7f1cdc0b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:36.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.354+0000 7f1cdb400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cdc0b32e0 0x7f1cdc0b3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:49892/0 (socket says 172.21.15.42:49892) 2024-05-28T14:15:36.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.354+0000 7f1cdb400640 1 -- 172.21.15.42:0/2882142428 learned_addr learned my addr 172.21.15.42:0/2882142428 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:36.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.354+0000 7f1cdaa00640 1 --2- 172.21.15.42:0/2882142428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cc0004810 0x7f1cdc151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:36.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.354+0000 7f1cdb400640 1 -- 172.21.15.42:0/2882142428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cdc165720 msgr2=0x7f1cdc165b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:36.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.354+0000 7f1cdb400640 1 --2- 172.21.15.42:0/2882142428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cdc165720 0x7f1cdc165b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:36.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.354+0000 7f1cdb400640 1 -- 172.21.15.42:0/2882142428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cc0004810 msgr2=0x7f1cdc151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:36.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.354+0000 7f1cdb400640 1 --2- 172.21.15.42:0/2882142428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cc0004810 0x7f1cdc151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:36.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.354+0000 7f1cdb400640 1 -- 172.21.15.42:0/2882142428 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc1632d0 con 0x7f1cdc0b32e0 2024-05-28T14:15:36.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.358+0000 7f1cdb400640 1 --2- 172.21.15.42:0/2882142428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cdc0b32e0 0x7f1cdc0b3770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0b2f60 tx=0x7f1ccc00ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:36.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.358+0000 7f1cd2a00640 1 -- 172.21.15.42:0/2882142428 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0074e0 con 0x7f1cdc0b32e0 2024-05-28T14:15:36.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.358+0000 7f1cd2a00640 1 -- 172.21.15.42:0/2882142428 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ccc0047f0 con 0x7f1cdc0b32e0 2024-05-28T14:15:36.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.358+0000 7f1cd2a00640 1 -- 172.21.15.42:0/2882142428 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc007680 con 0x7f1cdc0b32e0 2024-05-28T14:15:36.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.358+0000 7f1ce1800640 1 -- 172.21.15.42:0/2882142428 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cdc15eab0 con 0x7f1cdc0b32e0 2024-05-28T14:15:36.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.358+0000 7f1ce1800640 1 -- 172.21.15.42:0/2882142428 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cdc15f070 con 0x7f1cdc0b32e0 2024-05-28T14:15:36.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.366+0000 7f1ce1800640 1 -- 172.21.15.42:0/2882142428 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cdc0738b0 con 0x7f1cdc0b32e0 2024-05-28T14:15:36.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.366+0000 7f1cd2a00640 1 -- 172.21.15.42:0/2882142428 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 46) ==== 90894+0+0 (secure 0 0 0) 0x7f1ccc004c20 con 0x7f1cdc0b32e0 2024-05-28T14:15:36.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.370+0000 7f1cd2a00640 1 --2- 172.21.15.42:0/2882142428 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1cbc06a3e0 0x7f1cbc06c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:36.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.374+0000 7f1cd2a00640 1 -- 172.21.15.42:0/2882142428 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f1ccc084fb0 con 0x7f1cdc0b32e0 2024-05-28T14:15:36.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.382+0000 7f1cd2a00640 1 -- 172.21.15.42:0/2882142428 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ccc057150 con 0x7f1cdc0b32e0 2024-05-28T14:15:36.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.430+0000 7f1cdaa00640 1 --2- 172.21.15.42:0/2882142428 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1cbc06a3e0 0x7f1cbc06c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:36.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.514+0000 7f1cdaa00640 1 --2- 172.21.15.42:0/2882142428 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1cbc06a3e0 0x7f1cbc06c8a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc051740 tx=0x7f1cd400b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:36.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:36.986+0000 7f1ce1800640 1 -- 172.21.15.42:0/2882142428 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone6271", "target": ["mon-mgr", ""]}) -- 0x7f1cdc0b3cb0 con 0x7f1cbc06a3e0 2024-05-28T14:15:37.251 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:37.262+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:37.251 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:37.262+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:37.251 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:37.262+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:37.251 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:37.262+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:37.251 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:37.262+0000 7f077b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:15:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:37.550+0000 7f1cd2a00640 1 -- 172.21.15.42:0/2882142428 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mgrmap(e 47) ==== 90945+0+0 (secure 0 0 0) 0x7f1ccc0530c0 con 0x7f1cdc0b32e0 2024-05-28T14:15:37.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:37.762+0000 7f1cd2a00640 1 -- 172.21.15.42:0/2882142428 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1cd40039a0 con 0x7f1cbc06a3e0 2024-05-28T14:15:37.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:37.766+0000 7f1ce1800640 1 -- 172.21.15.42:0/2882142428 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1cbc06a3e0 msgr2=0x7f1cbc06c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:37.766+0000 7f1ce1800640 1 --2- 172.21.15.42:0/2882142428 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1cbc06a3e0 0x7f1cbc06c8a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc051740 tx=0x7f1cd400b040 comp rx=0 tx=0).stop 2024-05-28T14:15:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:37.766+0000 7f1ce1800640 1 -- 172.21.15.42:0/2882142428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cdc0b32e0 msgr2=0x7f1cdc0b3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:37.766+0000 7f1ce1800640 1 --2- 172.21.15.42:0/2882142428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cdc0b32e0 0x7f1cdc0b3770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0b2f60 tx=0x7f1ccc00ca50 comp rx=0 tx=0).stop 2024-05-28T14:15:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:37.766+0000 7f1ce1800640 1 -- 172.21.15.42:0/2882142428 shutdown_connections 2024-05-28T14:15:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:37.766+0000 7f1ce1800640 1 --2- 172.21.15.42:0/2882142428 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1cbc06a3e0 0x7f1cbc06c8a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:37.766+0000 7f1ce1800640 1 --2- 172.21.15.42:0/2882142428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cdc0b32e0 0x7f1cdc0b3770 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:37.766+0000 7f1ce1800640 1 --2- 172.21.15.42:0/2882142428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cdc165720 0x7f1cdc165b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:37.766+0000 7f1ce1800640 1 --2- 172.21.15.42:0/2882142428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cc0004810 0x7f1cdc151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:37.766+0000 7f1ce1800640 1 -- 172.21.15.42:0/2882142428 >> 172.21.15.42:0/2882142428 conn(0x7f1cdc0b21f0 msgr2=0x7f1cdc0b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:37.766+0000 7f1ce1800640 1 -- 172.21.15.42:0/2882142428 shutdown_connections 2024-05-28T14:15:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:37.766+0000 7f1ce1800640 1 -- 172.21.15.42:0/2882142428 wait complete. 2024-05-28T14:15:37.804 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone6272 2024-05-28T14:15:38.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.118+0000 7fadcf800640 1 Processor -- start 2024-05-28T14:15:38.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.118+0000 7fadcf800640 1 -- start start 2024-05-28T14:15:38.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.122+0000 7fadcf800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fadc80b20f0 0x7fadc80b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:38.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.122+0000 7fadcf800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fadc806c100 0x7fadc80cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:38.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.122+0000 7fadcf800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc80b2a10 con 0x7fadc80b4260 2024-05-28T14:15:38.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.122+0000 7fadcf800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fadc80b3bd0 con 0x7fadc806c100 2024-05-28T14:15:38.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.122+0000 7fadcf800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fadc806c590 con 0x7fadc80b20f0 2024-05-28T14:15:38.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.122+0000 7fadcce00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fadc80b20f0 0x7fadc80b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:38.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.122+0000 7fadc7e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fadc80b4260 0x7fadc806b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:58246/0 (socket says 172.21.15.42:58246) 2024-05-28T14:15:38.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.122+0000 7fadc7e00640 1 -- 172.21.15.42:0/3719374765 learned_addr learned my addr 172.21.15.42:0/3719374765 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:15:38.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.122+0000 7fadcd800640 1 --2- 172.21.15.42:0/3719374765 >> v2:172.21.15.42:3300/0 conn(0x7fadc806c100 0x7fadc80cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:38.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.122+0000 7fadcce00640 1 -- 172.21.15.42:0/3719374765 >> v2:172.21.15.42:3300/0 conn(0x7fadc806c100 msgr2=0x7fadc80cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:38.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.122+0000 7fadcce00640 1 --2- 172.21.15.42:0/3719374765 >> v2:172.21.15.42:3300/0 conn(0x7fadc806c100 0x7fadc80cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:38.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.122+0000 7fadcce00640 1 -- 172.21.15.42:0/3719374765 >> v1:172.21.15.42:6789/0 conn(0x7fadc80b4260 legacy=0x7fadc806b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:38.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.122+0000 7fadcce00640 1 -- 172.21.15.42:0/3719374765 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc80cbba0 con 0x7fadc80b20f0 2024-05-28T14:15:38.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.122+0000 7fadcd800640 1 --2- 172.21.15.42:0/3719374765 >> v2:172.21.15.42:3300/0 conn(0x7fadc806c100 0x7fadc80cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:15:38.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.126+0000 7fadcce00640 1 --2- 172.21.15.42:0/3719374765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc80b20f0 0x7fadc80b24d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0097c0 tx=0x7fadbc01c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d556cd80964393f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:38.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.126+0000 7fadc7400640 1 -- 172.21.15.42:0/3719374765 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0b0070 con 0x7fadc80b20f0 2024-05-28T14:15:38.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.126+0000 7fadc7400640 1 -- 172.21.15.42:0/3719374765 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadbc004030 con 0x7fadc80b20f0 2024-05-28T14:15:38.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.126+0000 7fadc7400640 1 -- 172.21.15.42:0/3719374765 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc004440 con 0x7fadc80b20f0 2024-05-28T14:15:38.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.126+0000 7fadcf800640 1 -- 172.21.15.42:0/3719374765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc80b20f0 msgr2=0x7fadc80b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:38.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.126+0000 7fadcf800640 1 --2- 172.21.15.42:0/3719374765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc80b20f0 0x7fadc80b24d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0097c0 tx=0x7fadbc01c460 comp rx=0 tx=0).stop 2024-05-28T14:15:38.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.126+0000 7fadcf800640 1 -- 172.21.15.42:0/3719374765 shutdown_connections 2024-05-28T14:15:38.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.126+0000 7fadcf800640 1 --2- 172.21.15.42:0/3719374765 >> v2:172.21.15.42:3300/0 conn(0x7fadc806c100 0x7fadc80cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:38.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.126+0000 7fadcf800640 1 --2- 172.21.15.42:0/3719374765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc80b20f0 0x7fadc80b24d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:38.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.126+0000 7fadcf800640 1 -- 172.21.15.42:0/3719374765 >> 172.21.15.42:0/3719374765 conn(0x7fadc809aa50 msgr2=0x7fadc80b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:38.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.126+0000 7fadcf800640 1 -- 172.21.15.42:0/3719374765 shutdown_connections 2024-05-28T14:15:38.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.126+0000 7fadcf800640 1 -- 172.21.15.42:0/3719374765 wait complete. 2024-05-28T14:15:38.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.130+0000 7fadcf800640 1 Processor -- start 2024-05-28T14:15:38.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.130+0000 7fadcf800640 1 -- start start 2024-05-28T14:15:38.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.130+0000 7fadcf800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fadc80b20f0 0x7fadc815b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.130+0000 7fadcf800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadc80b4260 0x7fadc80c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.130+0000 7fadcf800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc80c7640 0x7fadc8155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.130+0000 7fadcf800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fadc816b700 con 0x7fadc80b4260 2024-05-28T14:15:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.130+0000 7fadcf800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fadc816b880 con 0x7fadc80c7640 2024-05-28T14:15:38.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.130+0000 7fadcf800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fadc816ba00 con 0x7fadc80b20f0 2024-05-28T14:15:38.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.130+0000 7fadcce00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fadc80b20f0 0x7fadc815b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:38.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.130+0000 7fadcd800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc80c7640 0x7fadc8155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:38.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.130+0000 7fadcce00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fadc80b20f0 0x7fadc815b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:44524/0 (socket says 172.21.15.42:44524) 2024-05-28T14:15:38.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.130+0000 7fadcce00640 1 -- 172.21.15.42:0/3975602958 learned_addr learned my addr 172.21.15.42:0/3975602958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:38.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.130+0000 7fadc7e00640 1 --2- 172.21.15.42:0/3975602958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadc80b4260 0x7fadc80c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:38.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.134+0000 7fadcce00640 1 -- 172.21.15.42:0/3975602958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc80c7640 msgr2=0x7fadc8155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:38.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.134+0000 7fadcce00640 1 --2- 172.21.15.42:0/3975602958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc80c7640 0x7fadc8155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:38.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.134+0000 7fadcce00640 1 -- 172.21.15.42:0/3975602958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadc80b4260 msgr2=0x7fadc80c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:38.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.134+0000 7fadcce00640 1 --2- 172.21.15.42:0/3975602958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadc80b4260 0x7fadc80c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:38.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.134+0000 7fadcce00640 1 -- 172.21.15.42:0/3975602958 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc80c6880 con 0x7fadc80b20f0 2024-05-28T14:15:38.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.134+0000 7fadc7e00640 1 --2- 172.21.15.42:0/3975602958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadc80b4260 0x7fadc80c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:15:38.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.134+0000 7fadcce00640 1 --2- 172.21.15.42:0/3975602958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fadc80b20f0 0x7fadc815b260 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fadbc019070 tx=0x7fadbc019200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:38.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.134+0000 7fadc4c00640 1 -- 172.21.15.42:0/3975602958 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0b9020 con 0x7fadc80b20f0 2024-05-28T14:15:38.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.134+0000 7fadc4c00640 1 -- 172.21.15.42:0/3975602958 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadbc003bf0 con 0x7fadc80b20f0 2024-05-28T14:15:38.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.134+0000 7fadc4c00640 1 -- 172.21.15.42:0/3975602958 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0b0040 con 0x7fadc80b20f0 2024-05-28T14:15:38.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.134+0000 7fadcf800640 1 -- 172.21.15.42:0/3975602958 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc80c6b60 con 0x7fadc80b20f0 2024-05-28T14:15:38.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.134+0000 7fadcf800640 1 -- 172.21.15.42:0/3975602958 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc80c7120 con 0x7fadc80b20f0 2024-05-28T14:15:38.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.138+0000 7fadcf800640 1 -- 172.21.15.42:0/3975602958 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc809f5c0 con 0x7fadc80b20f0 2024-05-28T14:15:38.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.138+0000 7fadc4c00640 1 -- 172.21.15.42:0/3975602958 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 47) ==== 90945+0+0 (secure 0 0 0) 0x7fadbc0bc020 con 0x7fadc80b20f0 2024-05-28T14:15:38.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.142+0000 7fadc4c00640 1 --2- 172.21.15.42:0/3975602958 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fada806a420 0x7fada806c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:38.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.142+0000 7fadc4c00640 1 -- 172.21.15.42:0/3975602958 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fadbc11fa60 con 0x7fadc80b20f0 2024-05-28T14:15:38.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.146+0000 7fadc7e00640 1 --2- 172.21.15.42:0/3975602958 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fada806a420 0x7fada806c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:38.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.146+0000 7fadc7e00640 1 --2- 172.21.15.42:0/3975602958 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fada806a420 0x7fada806c8e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fadc80b2c90 tx=0x7fadb4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:38.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.150+0000 7fadc4c00640 1 -- 172.21.15.42:0/3975602958 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fadbc0c3030 con 0x7fadc80b20f0 2024-05-28T14:15:38.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.478+0000 7fadcf800640 1 -- 172.21.15.42:0/3975602958 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone6272", "target": ["mon-mgr", ""]}) -- 0x7fadc815b900 con 0x7fada806a420 2024-05-28T14:15:38.564 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:38.578+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:15:38.565 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:15:38.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.578+0000 7fadc4c00640 1 -- 172.21.15.42:0/3975602958 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fadc815b900 con 0x7fada806a420 2024-05-28T14:15:38.590 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:15:38.590 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone6273 2024-05-28T14:15:38.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.942+0000 7f3b08200640 1 Processor -- start 2024-05-28T14:15:38.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.942+0000 7f3b08200640 1 -- start start 2024-05-28T14:15:38.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.946+0000 7f3b08200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b0015c0c0 0x7f3b0006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:38.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.946+0000 7f3b08200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b000afe40 con 0x7f3b0006ee20 2024-05-28T14:15:38.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.946+0000 7f3b08200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b0015c600 con 0x7f3b00144f80 2024-05-28T14:15:38.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.946+0000 7f3b08200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0006f7c0 con 0x7f3b0015c0c0 2024-05-28T14:15:38.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.946+0000 7f3b04e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f3b0006ee20 0x7f3b0015b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:58260/0 (socket says 172.21.15.42:58260) 2024-05-28T14:15:38.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.946+0000 7f3b04e00640 1 -- 172.21.15.42:0/973889738 learned_addr learned my addr 172.21.15.42:0/973889738 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:15:38.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.946+0000 7f3b06200640 1 --2- 172.21.15.42:0/973889738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b0015c0c0 0x7f3b0006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:38.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.946+0000 7f3affe00640 1 -- 172.21.15.42:0/973889738 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2083791805 0 0) 0x7f3b0015c600 con 0x7f3b00144f80 2024-05-28T14:15:38.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.946+0000 7f3affe00640 1 -- 172.21.15.42:0/973889738 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ae8005740 con 0x7f3b00144f80 2024-05-28T14:15:38.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.946+0000 7f3affe00640 1 -- 172.21.15.42:0/973889738 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3640909535 0 0) 0x7f3b000afe40 con 0x7f3b0006ee20 2024-05-28T14:15:38.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.946+0000 7f3affe00640 1 -- 172.21.15.42:0/973889738 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b0015c600 con 0x7f3b0006ee20 2024-05-28T14:15:38.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.946+0000 7f3b06200640 1 -- 172.21.15.42:0/973889738 >> v1:172.21.15.63:6789/0 conn(0x7f3b00144f80 legacy=0x7f3b00072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:38.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.946+0000 7f3b06200640 1 -- 172.21.15.42:0/973889738 >> v1:172.21.15.42:6789/0 conn(0x7f3b0006ee20 legacy=0x7f3b0015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:38.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.946+0000 7f3b06200640 1 -- 172.21.15.42:0/973889738 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0015c8e0 con 0x7f3b0015c0c0 2024-05-28T14:15:38.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.954+0000 7f3b06200640 1 --2- 172.21.15.42:0/973889738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b0015c0c0 0x7f3b0006f280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009dc0 tx=0x7f3af801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=688520905626cd14 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.962+0000 7f3affe00640 1 -- 172.21.15.42:0/973889738 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af80b2070 con 0x7f3b0015c0c0 2024-05-28T14:15:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.962+0000 7f3affe00640 1 -- 172.21.15.42:0/973889738 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af80a5850 con 0x7f3b0015c0c0 2024-05-28T14:15:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.962+0000 7f3affe00640 1 -- 172.21.15.42:0/973889738 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af80a5c20 con 0x7f3b0015c0c0 2024-05-28T14:15:38.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.966+0000 7f3b08200640 1 -- 172.21.15.42:0/973889738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b0015c0c0 msgr2=0x7f3b0006f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.966+0000 7f3b08200640 1 --2- 172.21.15.42:0/973889738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b0015c0c0 0x7f3b0006f280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009dc0 tx=0x7f3af801c990 comp rx=0 tx=0).stop 2024-05-28T14:15:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.966+0000 7f3b08200640 1 -- 172.21.15.42:0/973889738 shutdown_connections 2024-05-28T14:15:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.966+0000 7f3b08200640 1 --2- 172.21.15.42:0/973889738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b0015c0c0 0x7f3b0006f280 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.966+0000 7f3b08200640 1 -- 172.21.15.42:0/973889738 >> 172.21.15.42:0/973889738 conn(0x7f3b000b21f0 msgr2=0x7f3b000b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.966+0000 7f3b08200640 1 -- 172.21.15.42:0/973889738 shutdown_connections 2024-05-28T14:15:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.970+0000 7f3b08200640 1 -- 172.21.15.42:0/973889738 wait complete. 2024-05-28T14:15:38.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.970+0000 7f3b08200640 1 Processor -- start 2024-05-28T14:15:38.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b08200640 1 -- start start 2024-05-28T14:15:38.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b08200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b0006ee20 0x7f3b001fb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:38.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b08200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b00144f80 0x7f3b001fb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:38.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b08200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b0015c0c0 0x7f3b0015d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:38.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b08200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3b000724d0 con 0x7f3b0006ee20 2024-05-28T14:15:38.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b08200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b000b42e0 con 0x7f3b0015c0c0 2024-05-28T14:15:38.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b08200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0006dd10 con 0x7f3b00144f80 2024-05-28T14:15:38.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b05800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b0006ee20 0x7f3b001fb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:38.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b06200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b0015c0c0 0x7f3b0015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:38.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b04e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b00144f80 0x7f3b001fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:38.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b06200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b0015c0c0 0x7f3b0015d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:43244/0 (socket says 172.21.15.42:43244) 2024-05-28T14:15:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b06200640 1 -- 172.21.15.42:0/3715779291 learned_addr learned my addr 172.21.15.42:0/3715779291 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b05800640 1 -- 172.21.15.42:0/3715779291 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b00144f80 msgr2=0x7f3b001fb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b05800640 1 --2- 172.21.15.42:0/3715779291 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b00144f80 0x7f3b001fb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b05800640 1 -- 172.21.15.42:0/3715779291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b0015c0c0 msgr2=0x7f3b0015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b05800640 1 --2- 172.21.15.42:0/3715779291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b0015c0c0 0x7f3b0015d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b05800640 1 -- 172.21.15.42:0/3715779291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b001355e0 con 0x7f3b0006ee20 2024-05-28T14:15:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.974+0000 7f3b05800640 1 --2- 172.21.15.42:0/3715779291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b0006ee20 0x7f3b001fb0d0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f3af400c3e0 tx=0x7f3af400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.978+0000 7f3afd600640 1 -- 172.21.15.42:0/3715779291 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4017070 con 0x7f3b0006ee20 2024-05-28T14:15:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.978+0000 7f3afd600640 1 -- 172.21.15.42:0/3715779291 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af40052d0 con 0x7f3b0006ee20 2024-05-28T14:15:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.978+0000 7f3afd600640 1 -- 172.21.15.42:0/3715779291 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af40054c0 con 0x7f3b0006ee20 2024-05-28T14:15:38.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.978+0000 7f3b08200640 1 -- 172.21.15.42:0/3715779291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b001358c0 con 0x7f3b0006ee20 2024-05-28T14:15:38.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.978+0000 7f3b08200640 1 -- 172.21.15.42:0/3715779291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b00135e80 con 0x7f3b0006ee20 2024-05-28T14:15:38.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.978+0000 7f3b08200640 1 -- 172.21.15.42:0/3715779291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b000738b0 con 0x7f3b0006ee20 2024-05-28T14:15:38.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.982+0000 7f3afd600640 1 -- 172.21.15.42:0/3715779291 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 47) ==== 90945+0+0 (secure 0 0 0) 0x7f3af401a070 con 0x7f3b0006ee20 2024-05-28T14:15:38.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.982+0000 7f3afd600640 1 --2- 172.21.15.42:0/3715779291 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3ae006a470 0x7f3ae006c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:38.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.982+0000 7f3afd600640 1 -- 172.21.15.42:0/3715779291 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f3af4084d40 con 0x7f3b0006ee20 2024-05-28T14:15:38.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.990+0000 7f3b04e00640 1 --2- 172.21.15.42:0/3715779291 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3ae006a470 0x7f3ae006c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:38.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.994+0000 7f3afd600640 1 -- 172.21.15.42:0/3715779291 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3af4056890 con 0x7f3b0006ee20 2024-05-28T14:15:38.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:38.998+0000 7f3b04e00640 1 --2- 172.21.15.42:0/3715779291 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3ae006a470 0x7f3ae006c930 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3aec007fd0 tx=0x7f3aec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:39.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:39.534+0000 7f3b08200640 1 -- 172.21.15.42:0/3715779291 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone6273", "target": ["mon-mgr", ""]}) -- 0x7f3b0015f2b0 con 0x7f3ae006a470 2024-05-28T14:15:39.724 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:15:39.734+0000 7f0774600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-28T14:15:39.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:39.746+0000 7f3afd600640 1 -- 172.21.15.42:0/3715779291 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f3b0015f2b0 con 0x7f3ae006a470 2024-05-28T14:15:39.744 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-28T14:15:39.791 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-28T14:15:39.791 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone6274 2024-05-28T14:15:40.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.314+0000 7fbe2e800640 1 Processor -- start 2024-05-28T14:15:40.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.314+0000 7fbe2e800640 1 -- start start 2024-05-28T14:15:40.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.314+0000 7fbe2e800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe28071de0 0x7fbe28072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:40.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.314+0000 7fbe2e800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe2815b850 con 0x7fbe280cfdc0 2024-05-28T14:15:40.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.318+0000 7fbe2e800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe2815bb30 con 0x7fbe2806fed0 2024-05-28T14:15:40.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.318+0000 7fbe2e800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbe2806df20 con 0x7fbe28071de0 2024-05-28T14:15:40.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.318+0000 7fbe27400640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fbe2806fed0 0x7fbe2806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:58304/0 (socket says 172.21.15.42:58304) 2024-05-28T14:15:40.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.318+0000 7fbe27400640 1 -- 172.21.15.42:0/235205094 learned_addr learned my addr 172.21.15.42:0/235205094 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:15:40.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.318+0000 7fbe27e00640 1 --2- 172.21.15.42:0/235205094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe28071de0 0x7fbe28072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:40.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.318+0000 7fbe26000640 1 -- 172.21.15.42:0/235205094 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1904343581 0 0) 0x7fbe2815bb30 con 0x7fbe2806fed0 2024-05-28T14:15:40.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.318+0000 7fbe26000640 1 -- 172.21.15.42:0/235205094 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe0c007940 con 0x7fbe2806fed0 2024-05-28T14:15:40.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.318+0000 7fbe26000640 1 -- 172.21.15.42:0/235205094 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 284075784 0 0) 0x7fbe0c007940 con 0x7fbe2806fed0 2024-05-28T14:15:40.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.318+0000 7fbe26000640 1 -- 172.21.15.42:0/235205094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe28071de0 msgr2=0x7fbe28072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:40.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.318+0000 7fbe26000640 1 --2- 172.21.15.42:0/235205094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe28071de0 0x7fbe28072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:40.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.318+0000 7fbe26000640 1 -- 172.21.15.42:0/235205094 >> v1:172.21.15.42:6789/0 conn(0x7fbe280cfdc0 legacy=0x7fbe2815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:40.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.322+0000 7fbe26000640 1 -- 172.21.15.42:0/235205094 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2806e200 con 0x7fbe2806fed0 2024-05-28T14:15:40.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.322+0000 7fbe26000640 1 -- 172.21.15.42:0/235205094 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fbe18001a00 con 0x7fbe2806fed0 2024-05-28T14:15:40.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.322+0000 7fbe26000640 1 -- 172.21.15.42:0/235205094 >> v1:172.21.15.63:6789/0 conn(0x7fbe2806fed0 legacy=0x7fbe2806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:40.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.322+0000 7fbe26000640 1 --2- 172.21.15.42:0/235205094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe0c007dd0 0x7fbe0c0081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:40.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.322+0000 7fbe26000640 1 --2- 172.21.15.42:0/235205094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe0c008c20 0x7fbe0c0090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:40.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.322+0000 7fbe26000640 1 --2- 172.21.15.42:0/235205094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe0c0095e0 0x7fbe0c00fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:40.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.322+0000 7fbe26000640 1 -- 172.21.15.42:0/235205094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2806df20 con 0x7fbe0c0095e0 2024-05-28T14:15:40.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.322+0000 7fbe26000640 1 -- 172.21.15.42:0/235205094 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c010400 con 0x7fbe0c008c20 2024-05-28T14:15:40.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.322+0000 7fbe26000640 1 -- 172.21.15.42:0/235205094 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbe0c010600 con 0x7fbe0c007dd0 2024-05-28T14:15:40.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.322+0000 7fbe27e00640 1 --2- 172.21.15.42:0/235205094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe0c0095e0 0x7fbe0c00fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:40.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.322+0000 7fbe26a00640 1 --2- 172.21.15.42:0/235205094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe0c008c20 0x7fbe0c0090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:40.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.326+0000 7fbe27400640 1 --2- 172.21.15.42:0/235205094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe0c007dd0 0x7fbe0c0081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:40.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.326+0000 7fbe27e00640 1 -- 172.21.15.42:0/235205094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe0c007dd0 msgr2=0x7fbe0c0081d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:40.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.326+0000 7fbe27e00640 1 --2- 172.21.15.42:0/235205094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe0c007dd0 0x7fbe0c0081d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:40.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.326+0000 7fbe27e00640 1 -- 172.21.15.42:0/235205094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe0c008c20 msgr2=0x7fbe0c0090a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:40.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.330+0000 7fbe27e00640 1 --2- 172.21.15.42:0/235205094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe0c008c20 0x7fbe0c0090a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:40.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.330+0000 7fbe27e00640 1 -- 172.21.15.42:0/235205094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2806e200 con 0x7fbe0c0095e0 2024-05-28T14:15:40.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.330+0000 7fbe26a00640 1 --2- 172.21.15.42:0/235205094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe0c008c20 0x7fbe0c0090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:15:40.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.338+0000 7fbe27e00640 1 --2- 172.21.15.42:0/235205094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe0c0095e0 0x7fbe0c00fde0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fbe280721c0 tx=0x7fbe2001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a6154f20ea4d16 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:40.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.338+0000 7fbe26000640 1 -- 172.21.15.42:0/235205094 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000b530 con 0x7fbe0c0095e0 2024-05-28T14:15:40.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.338+0000 7fbe26000640 1 -- 172.21.15.42:0/235205094 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe200b1070 con 0x7fbe0c0095e0 2024-05-28T14:15:40.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.338+0000 7fbe26000640 1 -- 172.21.15.42:0/235205094 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe200ac850 con 0x7fbe0c0095e0 2024-05-28T14:15:40.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.338+0000 7fbe2e800640 1 -- 172.21.15.42:0/235205094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe0c0095e0 msgr2=0x7fbe0c00fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:40.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.338+0000 7fbe2e800640 1 --2- 172.21.15.42:0/235205094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe0c0095e0 0x7fbe0c00fde0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fbe280721c0 tx=0x7fbe2001c560 comp rx=0 tx=0).stop 2024-05-28T14:15:40.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.338+0000 7fbe27e00640 1 -- 172.21.15.42:0/235205094 reap_dead start 2024-05-28T14:15:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.342+0000 7fbe2e800640 1 -- 172.21.15.42:0/235205094 shutdown_connections 2024-05-28T14:15:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.342+0000 7fbe2e800640 1 -- 172.21.15.42:0/235205094 >> 172.21.15.42:0/235205094 conn(0x7fbe280b21f0 msgr2=0x7fbe280b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.342+0000 7fbe2e800640 1 -- 172.21.15.42:0/235205094 shutdown_connections 2024-05-28T14:15:40.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.342+0000 7fbe2e800640 1 -- 172.21.15.42:0/235205094 wait complete. 2024-05-28T14:15:40.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.346+0000 7fbe2e800640 1 Processor -- start 2024-05-28T14:15:40.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.346+0000 7fbe2e800640 1 -- start start 2024-05-28T14:15:40.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.346+0000 7fbe2e800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe0c0095e0 0x7fbe280b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:40.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.346+0000 7fbe2e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe28159b10 0x7fbe28159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:40.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.346+0000 7fbe2e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe280c64b0 0x7fbe280c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:40.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.346+0000 7fbe2e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2806fc20 con 0x7fbe28159b10 2024-05-28T14:15:40.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.346+0000 7fbe2e800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2806faa0 con 0x7fbe280c64b0 2024-05-28T14:15:40.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.346+0000 7fbe2e800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbe28072270 con 0x7fbe0c0095e0 2024-05-28T14:15:40.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.346+0000 7fbe26a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe28159b10 0x7fbe28159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:40.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe26a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe28159b10 0x7fbe28159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40810/0 (socket says 172.21.15.42:40810) 2024-05-28T14:15:40.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe26a00640 1 -- 172.21.15.42:0/2267219747 learned_addr learned my addr 172.21.15.42:0/2267219747 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:40.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe26a00640 1 -- 172.21.15.42:0/2267219747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe0c0095e0 msgr2=0x7fbe280b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:40.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe26a00640 1 --2- 172.21.15.42:0/2267219747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe0c0095e0 0x7fbe280b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:40.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe27e00640 1 --2- 172.21.15.42:0/2267219747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe280c64b0 0x7fbe280c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:40.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe26a00640 1 -- 172.21.15.42:0/2267219747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe280c64b0 msgr2=0x7fbe280c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:40.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe26a00640 1 --2- 172.21.15.42:0/2267219747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe280c64b0 0x7fbe280c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:40.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe26a00640 1 -- 172.21.15.42:0/2267219747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe280b1580 con 0x7fbe28159b10 2024-05-28T14:15:40.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe27e00640 1 --2- 172.21.15.42:0/2267219747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe280c64b0 0x7fbe280c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:15:40.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe26a00640 1 --2- 172.21.15.42:0/2267219747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe28159b10 0x7fbe28159f90 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400f8a0 tx=0x7fbe1400fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:40.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe1f400640 1 -- 172.21.15.42:0/2267219747 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe140052e0 con 0x7fbe28159b10 2024-05-28T14:15:40.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe1f400640 1 -- 172.21.15.42:0/2267219747 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe14018070 con 0x7fbe28159b10 2024-05-28T14:15:40.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe1f400640 1 -- 172.21.15.42:0/2267219747 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1400d970 con 0x7fbe28159b10 2024-05-28T14:15:40.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe2e800640 1 -- 172.21.15.42:0/2267219747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe280c6e70 con 0x7fbe28159b10 2024-05-28T14:15:40.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.350+0000 7fbe2e800640 1 -- 172.21.15.42:0/2267219747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe280c7410 con 0x7fbe28159b10 2024-05-28T14:15:40.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.354+0000 7fbe2e800640 1 -- 172.21.15.42:0/2267219747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe280738b0 con 0x7fbe28159b10 2024-05-28T14:15:40.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.354+0000 7fbe1f400640 1 -- 172.21.15.42:0/2267219747 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 47) ==== 90945+0+0 (secure 0 0 0) 0x7fbe1400db10 con 0x7fbe28159b10 2024-05-28T14:15:40.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.358+0000 7fbe1f400640 1 --2- 172.21.15.42:0/2267219747 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe0806a520 0x7fbe0806c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:40.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.358+0000 7fbe1f400640 1 -- 172.21.15.42:0/2267219747 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fbe140859d0 con 0x7fbe28159b10 2024-05-28T14:15:40.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.370+0000 7fbe1f400640 1 -- 172.21.15.42:0/2267219747 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbe1401a310 con 0x7fbe28159b10 2024-05-28T14:15:40.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.374+0000 7fbe27400640 1 --2- 172.21.15.42:0/2267219747 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe0806a520 0x7fbe0806c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:40.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:40.382+0000 7fbe27400640 1 --2- 172.21.15.42:0/2267219747 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe0806a520 0x7fbe0806c9e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fbe18002000 tx=0x7fbe18001f70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:40.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:41.010+0000 7fbe2e800640 1 -- 172.21.15.42:0/2267219747 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone6274", "target": ["mon-mgr", ""]}) -- 0x7fbe2804fdd0 con 0x7fbe0806a520 2024-05-28T14:15:42.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:42.714+0000 7fbe1f400640 1 -- 172.21.15.42:0/2267219747 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbe2804fdd0 con 0x7fbe0806a520 2024-05-28T14:15:42.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:42.726+0000 7fbe2e800640 1 -- 172.21.15.42:0/2267219747 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe0806a520 msgr2=0x7fbe0806c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:42.726+0000 7fbe2e800640 1 --2- 172.21.15.42:0/2267219747 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe0806a520 0x7fbe0806c9e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fbe18002000 tx=0x7fbe18001f70 comp rx=0 tx=0).stop 2024-05-28T14:15:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:42.726+0000 7fbe2e800640 1 -- 172.21.15.42:0/2267219747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe28159b10 msgr2=0x7fbe28159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:42.726+0000 7fbe2e800640 1 --2- 172.21.15.42:0/2267219747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe28159b10 0x7fbe28159f90 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400f8a0 tx=0x7fbe1400fd60 comp rx=0 tx=0).stop 2024-05-28T14:15:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:42.730+0000 7fbe2e800640 1 -- 172.21.15.42:0/2267219747 shutdown_connections 2024-05-28T14:15:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:42.730+0000 7fbe2e800640 1 --2- 172.21.15.42:0/2267219747 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe0806a520 0x7fbe0806c9e0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:42.730+0000 7fbe2e800640 1 --2- 172.21.15.42:0/2267219747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe280c64b0 0x7fbe280c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:42.730+0000 7fbe2e800640 1 --2- 172.21.15.42:0/2267219747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe28159b10 0x7fbe28159f90 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:42.730+0000 7fbe2e800640 1 --2- 172.21.15.42:0/2267219747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe0c0095e0 0x7fbe280b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:42.730+0000 7fbe2e800640 1 -- 172.21.15.42:0/2267219747 >> 172.21.15.42:0/2267219747 conn(0x7fbe280b21f0 msgr2=0x7fbe280b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:42.730+0000 7fbe2e800640 1 -- 172.21.15.42:0/2267219747 shutdown_connections 2024-05-28T14:15:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:42.730+0000 7fbe2e800640 1 -- 172.21.15.42:0/2267219747 wait complete. 2024-05-28T14:15:42.815 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone6275 2024-05-28T14:15:43.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.322+0000 7f5e5ba00640 1 Processor -- start 2024-05-28T14:15:43.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.326+0000 7f5e5ba00640 1 -- start start 2024-05-28T14:15:43.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.330+0000 7f5e5ba00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e5415e870 0x7f5e540d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:43.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.330+0000 7f5e5ba00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e540d1a80 con 0x7f5e5415ec50 2024-05-28T14:15:43.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.330+0000 7f5e5ba00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e540ccbd0 con 0x7f5e540b7d70 2024-05-28T14:15:43.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.330+0000 7f5e5ba00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5e540cd500 con 0x7f5e5415e870 2024-05-28T14:15:43.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.330+0000 7f5e53e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f5e5415ec50 0x7f5e540b7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:58276/0 (socket says 172.21.15.42:58276) 2024-05-28T14:15:43.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.330+0000 7f5e53e00640 1 -- 172.21.15.42:0/3219696150 learned_addr learned my addr 172.21.15.42:0/3219696150 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:15:43.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.334+0000 7f5e59000640 1 --2- 172.21.15.42:0/3219696150 >> v2:172.21.15.42:3300/0 conn(0x7f5e5415e870 0x7f5e540d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:43.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.334+0000 7f5e59000640 1 -- 172.21.15.42:0/3219696150 >> v1:172.21.15.63:6789/0 conn(0x7f5e540b7d70 legacy=0x7f5e5409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:43.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.334+0000 7f5e59000640 1 -- 172.21.15.42:0/3219696150 >> v1:172.21.15.42:6789/0 conn(0x7f5e5415ec50 legacy=0x7f5e540b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:43.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.334+0000 7f5e59000640 1 -- 172.21.15.42:0/3219696150 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e540cceb0 con 0x7f5e5415e870 2024-05-28T14:15:43.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.342+0000 7f5e59000640 1 --2- 172.21.15.42:0/3219696150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e5415e870 0x7f5e540d1230 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f5e480097c0 tx=0x7f5e4801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08e4740379a10f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:43.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.342+0000 7f5e53400640 1 -- 172.21.15.42:0/3219696150 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e480b5070 con 0x7f5e5415e870 2024-05-28T14:15:43.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.342+0000 7f5e53400640 1 -- 172.21.15.42:0/3219696150 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e480a8950 con 0x7f5e5415e870 2024-05-28T14:15:43.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.342+0000 7f5e53400640 1 -- 172.21.15.42:0/3219696150 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e480a8d20 con 0x7f5e5415e870 2024-05-28T14:15:43.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.346+0000 7f5e5ba00640 1 -- 172.21.15.42:0/3219696150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e5415e870 msgr2=0x7f5e540d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:43.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.346+0000 7f5e5ba00640 1 --2- 172.21.15.42:0/3219696150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e5415e870 0x7f5e540d1230 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f5e480097c0 tx=0x7f5e4801ec70 comp rx=0 tx=0).stop 2024-05-28T14:15:43.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.346+0000 7f5e5ba00640 1 -- 172.21.15.42:0/3219696150 shutdown_connections 2024-05-28T14:15:43.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.346+0000 7f5e5ba00640 1 --2- 172.21.15.42:0/3219696150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e5415e870 0x7f5e540d1230 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:43.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.346+0000 7f5e5ba00640 1 -- 172.21.15.42:0/3219696150 >> 172.21.15.42:0/3219696150 conn(0x7f5e540b3fd0 msgr2=0x7f5e540b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:43.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.346+0000 7f5e5ba00640 1 -- 172.21.15.42:0/3219696150 shutdown_connections 2024-05-28T14:15:43.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.346+0000 7f5e5ba00640 1 -- 172.21.15.42:0/3219696150 wait complete. 2024-05-28T14:15:43.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.350+0000 7f5e5ba00640 1 Processor -- start 2024-05-28T14:15:43.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.350+0000 7f5e5ba00640 1 -- start start 2024-05-28T14:15:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.354+0000 7f5e5ba00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e540b7d70 0x7f5e54200d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.354+0000 7f5e5ba00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e5415e870 0x7f5e5415f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.354+0000 7f5e5ba00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e5415ec50 0x7f5e540b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.354+0000 7f5e5ba00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e540d1770 con 0x7f5e540b7d70 2024-05-28T14:15:43.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.354+0000 7f5e5ba00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e540ce9a0 con 0x7f5e5415ec50 2024-05-28T14:15:43.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.354+0000 7f5e5ba00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5e540b5330 con 0x7f5e5415e870 2024-05-28T14:15:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.354+0000 7f5e53e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e5415e870 0x7f5e5415f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.354+0000 7f5e53e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e5415e870 0x7f5e5415f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:44574/0 (socket says 172.21.15.42:44574) 2024-05-28T14:15:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.354+0000 7f5e53e00640 1 -- 172.21.15.42:0/600538192 learned_addr learned my addr 172.21.15.42:0/600538192 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.354+0000 7f5e59a00640 1 --2- 172.21.15.42:0/600538192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e5415ec50 0x7f5e540b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.354+0000 7f5e53e00640 1 -- 172.21.15.42:0/600538192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e5415ec50 msgr2=0x7f5e540b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.358+0000 7f5e53e00640 1 --2- 172.21.15.42:0/600538192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e5415ec50 0x7f5e540b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.358+0000 7f5e53e00640 1 -- 172.21.15.42:0/600538192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e540b7d70 msgr2=0x7f5e54200d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.358+0000 7f5e53e00640 1 --2- 172.21.15.42:0/600538192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e540b7d70 0x7f5e54200d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:43.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.358+0000 7f5e53e00640 1 -- 172.21.15.42:0/600538192 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e54201240 con 0x7f5e5415e870 2024-05-28T14:15:43.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.358+0000 7f5e59a00640 1 --2- 172.21.15.42:0/600538192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e5415ec50 0x7f5e540b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:15:43.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.358+0000 7f5e53e00640 1 --2- 172.21.15.42:0/600538192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e5415e870 0x7f5e5415f9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000c420 tx=0x7f5e4000c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:43.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.358+0000 7f5e50c00640 1 -- 172.21.15.42:0/600538192 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40017070 con 0x7f5e5415e870 2024-05-28T14:15:43.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.358+0000 7f5e50c00640 1 -- 172.21.15.42:0/600538192 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e40005cf0 con 0x7f5e5415e870 2024-05-28T14:15:43.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.358+0000 7f5e50c00640 1 -- 172.21.15.42:0/600538192 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e400072d0 con 0x7f5e5415e870 2024-05-28T14:15:43.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.358+0000 7f5e5ba00640 1 -- 172.21.15.42:0/600538192 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e54160d60 con 0x7f5e5415e870 2024-05-28T14:15:43.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.362+0000 7f5e5ba00640 1 -- 172.21.15.42:0/600538192 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e540c9a30 con 0x7f5e5415e870 2024-05-28T14:15:43.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.362+0000 7f5e5ba00640 1 -- 172.21.15.42:0/600538192 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e5404fdd0 con 0x7f5e5415e870 2024-05-28T14:15:43.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.370+0000 7f5e50c00640 1 -- 172.21.15.42:0/600538192 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 47) ==== 90945+0+0 (secure 0 0 0) 0x7f5e4001a070 con 0x7f5e5415e870 2024-05-28T14:15:43.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.370+0000 7f5e50c00640 1 --2- 172.21.15.42:0/600538192 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e3406a4f0 0x7f5e3406c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:43.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.370+0000 7f5e50c00640 1 -- 172.21.15.42:0/600538192 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f5e40084db0 con 0x7f5e5415e870 2024-05-28T14:15:43.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.374+0000 7f5e50c00640 1 -- 172.21.15.42:0/600538192 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5e40056800 con 0x7f5e5415e870 2024-05-28T14:15:43.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.378+0000 7f5e59000640 1 --2- 172.21.15.42:0/600538192 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e3406a4f0 0x7f5e3406c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:43.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.394+0000 7f5e59000640 1 --2- 172.21.15.42:0/600538192 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e3406a4f0 0x7f5e3406c9b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5e480b2000 tx=0x7f5e48004740 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:43.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:43.910+0000 7f5e5ba00640 1 -- 172.21.15.42:0/600538192 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone6275", "target": ["mon-mgr", ""]}) -- 0x7f5e5406e320 con 0x7f5e3406a4f0 2024-05-28T14:15:44.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:44.714+0000 7f5e50c00640 1 -- 172.21.15.42:0/600538192 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5e5406e320 con 0x7f5e3406a4f0 2024-05-28T14:15:44.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:44.726+0000 7f5e5ba00640 1 -- 172.21.15.42:0/600538192 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e3406a4f0 msgr2=0x7f5e3406c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:44.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:44.726+0000 7f5e5ba00640 1 --2- 172.21.15.42:0/600538192 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e3406a4f0 0x7f5e3406c9b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5e480b2000 tx=0x7f5e48004740 comp rx=0 tx=0).stop 2024-05-28T14:15:44.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:44.726+0000 7f5e5ba00640 1 -- 172.21.15.42:0/600538192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e5415e870 msgr2=0x7f5e5415f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:44.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:44.726+0000 7f5e5ba00640 1 --2- 172.21.15.42:0/600538192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e5415e870 0x7f5e5415f9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000c420 tx=0x7f5e4000c8e0 comp rx=0 tx=0).stop 2024-05-28T14:15:44.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:44.730+0000 7f5e5ba00640 1 -- 172.21.15.42:0/600538192 shutdown_connections 2024-05-28T14:15:44.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:44.730+0000 7f5e5ba00640 1 --2- 172.21.15.42:0/600538192 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e3406a4f0 0x7f5e3406c9b0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:44.730+0000 7f5e5ba00640 1 --2- 172.21.15.42:0/600538192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e5415ec50 0x7f5e540b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:44.730+0000 7f5e5ba00640 1 --2- 172.21.15.42:0/600538192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e5415e870 0x7f5e5415f9f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:44.730+0000 7f5e5ba00640 1 --2- 172.21.15.42:0/600538192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e540b7d70 0x7f5e54200d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:44.730+0000 7f5e5ba00640 1 -- 172.21.15.42:0/600538192 >> 172.21.15.42:0/600538192 conn(0x7f5e540b3fd0 msgr2=0x7f5e540707d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:44.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:44.734+0000 7f5e5ba00640 1 -- 172.21.15.42:0/600538192 shutdown_connections 2024-05-28T14:15:44.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:44.734+0000 7f5e5ba00640 1 -- 172.21.15.42:0/600538192 wait complete. 2024-05-28T14:15:44.839 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone6276 2024-05-28T14:15:45.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.266+0000 7f99d3c00640 1 Processor -- start 2024-05-28T14:15:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.274+0000 7f99d3c00640 1 -- start start 2024-05-28T14:15:45.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.274+0000 7f99d3c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f99cc15ba70 0x7f99cc06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:45.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.274+0000 7f99d3c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99cc072710 con 0x7f99cc154dd0 2024-05-28T14:15:45.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.274+0000 7f99d3c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99cc15e330 con 0x7f99cc06f800 2024-05-28T14:15:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.274+0000 7f99d3c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f99cc0724f0 con 0x7f99cc15ba70 2024-05-28T14:15:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.278+0000 7f99d1200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f99cc154dd0 0x7f99cc071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:58286/0 (socket says 172.21.15.42:58286) 2024-05-28T14:15:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.278+0000 7f99d1200640 1 -- 172.21.15.42:0/3170668134 learned_addr learned my addr 172.21.15.42:0/3170668134 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:15:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.278+0000 7f99cbe00640 1 --2- 172.21.15.42:0/3170668134 >> v2:172.21.15.63:3300/0 conn(0x7f99cc15ba70 0x7f99cc06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.278+0000 7f99cb400640 1 -- 172.21.15.42:0/3170668134 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3798449539 0 0) 0x7f99cc15e330 con 0x7f99cc06f800 2024-05-28T14:15:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.278+0000 7f99cb400640 1 -- 172.21.15.42:0/3170668134 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99b0007940 con 0x7f99cc06f800 2024-05-28T14:15:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.278+0000 7f99cbe00640 1 -- 172.21.15.42:0/3170668134 >> v1:172.21.15.63:6789/0 conn(0x7f99cc06f800 legacy=0x7f99cc0ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.278+0000 7f99cbe00640 1 -- 172.21.15.42:0/3170668134 >> v1:172.21.15.42:6789/0 conn(0x7f99cc154dd0 legacy=0x7f99cc071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.278+0000 7f99cbe00640 1 -- 172.21.15.42:0/3170668134 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99cc15e610 con 0x7f99cc15ba70 2024-05-28T14:15:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.282+0000 7f99cbe00640 1 --2- 172.21.15.42:0/3170668134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc15ba70 0x7f99cc06f2c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0097f0 tx=0x7f99bc01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1395c18b4b54aa91 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.282+0000 7f99cb400640 1 -- 172.21.15.42:0/3170668134 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0a7070 con 0x7f99cc15ba70 2024-05-28T14:15:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.282+0000 7f99cb400640 1 -- 172.21.15.42:0/3170668134 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99bc0b0070 con 0x7f99cc15ba70 2024-05-28T14:15:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.282+0000 7f99cb400640 1 -- 172.21.15.42:0/3170668134 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0ab890 con 0x7f99cc15ba70 2024-05-28T14:15:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.282+0000 7f99d3c00640 1 -- 172.21.15.42:0/3170668134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc15ba70 msgr2=0x7f99cc06f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.282+0000 7f99d3c00640 1 --2- 172.21.15.42:0/3170668134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc15ba70 0x7f99cc06f2c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0097f0 tx=0x7f99bc01c470 comp rx=0 tx=0).stop 2024-05-28T14:15:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.282+0000 7f99d3c00640 1 -- 172.21.15.42:0/3170668134 shutdown_connections 2024-05-28T14:15:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.282+0000 7f99d3c00640 1 --2- 172.21.15.42:0/3170668134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc15ba70 0x7f99cc06f2c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.282+0000 7f99d3c00640 1 -- 172.21.15.42:0/3170668134 >> 172.21.15.42:0/3170668134 conn(0x7f99cc0b21f0 msgr2=0x7f99cc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.282+0000 7f99d3c00640 1 -- 172.21.15.42:0/3170668134 shutdown_connections 2024-05-28T14:15:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.286+0000 7f99d3c00640 1 -- 172.21.15.42:0/3170668134 wait complete. 2024-05-28T14:15:45.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.286+0000 7f99d3c00640 1 Processor -- start 2024-05-28T14:15:45.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.286+0000 7f99d3c00640 1 -- start start 2024-05-28T14:15:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d3c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99cc06f800 0x7f99cc15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d3c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99cc154dd0 0x7f99cc15bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d3c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc15ba70 0x7f99cc146780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d3c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc15f0f0 con 0x7f99cc06f800 2024-05-28T14:15:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d3c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc06fc70 con 0x7f99cc15ba70 2024-05-28T14:15:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d3c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f99cc15c850 con 0x7f99cc154dd0 2024-05-28T14:15:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99cc06f800 0x7f99cc15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99cc06f800 0x7f99cc15d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40832/0 (socket says 172.21.15.42:40832) 2024-05-28T14:15:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d1200640 1 -- 172.21.15.42:0/4122636483 learned_addr learned my addr 172.21.15.42:0/4122636483 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99cbe00640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99cc154dd0 0x7f99cc15bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d1c00640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc15ba70 0x7f99cc146780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d1200640 1 -- 172.21.15.42:0/4122636483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99cc154dd0 msgr2=0x7f99cc15bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d1200640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99cc154dd0 0x7f99cc15bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d1200640 1 -- 172.21.15.42:0/4122636483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc15ba70 msgr2=0x7f99cc146780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d1200640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc15ba70 0x7f99cc146780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d1200640 1 -- 172.21.15.42:0/4122636483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99cc15cb30 con 0x7f99cc06f800 2024-05-28T14:15:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99cbe00640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99cc154dd0 0x7f99cc15bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:15:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d1200640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99cc06f800 0x7f99cc15d8f0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f99c400cb40 tx=0x7f99c400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99c8c00640 1 -- 172.21.15.42:0/4122636483 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4017070 con 0x7f99cc06f800 2024-05-28T14:15:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99c8c00640 1 -- 172.21.15.42:0/4122636483 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c400aca0 con 0x7f99cc06f800 2024-05-28T14:15:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99c8c00640 1 -- 172.21.15.42:0/4122636483 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4012620 con 0x7f99cc06f800 2024-05-28T14:15:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.290+0000 7f99d3c00640 1 -- 172.21.15.42:0/4122636483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99cc15ce10 con 0x7f99cc06f800 2024-05-28T14:15:45.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.294+0000 7f99d3c00640 1 -- 172.21.15.42:0/4122636483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99cc15d3d0 con 0x7f99cc06f800 2024-05-28T14:15:45.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.294+0000 7f99d3c00640 1 -- 172.21.15.42:0/4122636483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99cc0738b0 con 0x7f99cc06f800 2024-05-28T14:15:45.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.298+0000 7f99c8c00640 1 -- 172.21.15.42:0/4122636483 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 47) ==== 90945+0+0 (secure 0 0 0) 0x7f99c401a070 con 0x7f99cc06f800 2024-05-28T14:15:45.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.298+0000 7f99c8c00640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99ac06a420 0x7f99ac06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:45.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.298+0000 7f99c8c00640 1 -- 172.21.15.42:0/4122636483 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f99c4085a90 con 0x7f99cc06f800 2024-05-28T14:15:45.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.310+0000 7f99c8c00640 1 -- 172.21.15.42:0/4122636483 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f99c4021020 con 0x7f99cc06f800 2024-05-28T14:15:45.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.326+0000 7f99cbe00640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99ac06a420 0x7f99ac06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.334+0000 7f99cbe00640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99ac06a420 0x7f99ac06c8e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0af000 tx=0x7f99bc0a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:45.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:45.922+0000 7f99d3c00640 1 -- 172.21.15.42:0/4122636483 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone6276", "target": ["mon-mgr", ""]}) -- 0x7f99cc15d610 con 0x7f99ac06a420 2024-05-28T14:15:46.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.466+0000 7f99c8c00640 1 -- 172.21.15.42:0/4122636483 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f99bc0b0040 con 0x7f99ac06a420 2024-05-28T14:15:46.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.466+0000 7f99d3c00640 1 -- 172.21.15.42:0/4122636483 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99ac06a420 msgr2=0x7f99ac06c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:46.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.466+0000 7f99d3c00640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99ac06a420 0x7f99ac06c8e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0af000 tx=0x7f99bc0a7000 comp rx=0 tx=0).stop 2024-05-28T14:15:46.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.466+0000 7f99d3c00640 1 -- 172.21.15.42:0/4122636483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99cc06f800 msgr2=0x7f99cc15d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:46.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.466+0000 7f99d3c00640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99cc06f800 0x7f99cc15d8f0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f99c400cb40 tx=0x7f99c400a060 comp rx=0 tx=0).stop 2024-05-28T14:15:46.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.466+0000 7f99d3c00640 1 -- 172.21.15.42:0/4122636483 shutdown_connections 2024-05-28T14:15:46.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.466+0000 7f99d3c00640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f99ac06a420 0x7f99ac06c8e0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:46.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.466+0000 7f99d3c00640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc15ba70 0x7f99cc146780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:46.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.466+0000 7f99d3c00640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99cc154dd0 0x7f99cc15bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:46.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.466+0000 7f99d3c00640 1 --2- 172.21.15.42:0/4122636483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99cc06f800 0x7f99cc15d8f0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:46.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.466+0000 7f99d3c00640 1 -- 172.21.15.42:0/4122636483 >> 172.21.15.42:0/4122636483 conn(0x7f99cc0b21f0 msgr2=0x7f99cc15a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:46.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.466+0000 7f99d3c00640 1 -- 172.21.15.42:0/4122636483 shutdown_connections 2024-05-28T14:15:46.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.466+0000 7f99d3c00640 1 -- 172.21.15.42:0/4122636483 wait complete. 2024-05-28T14:15:46.519 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone6277 2024-05-28T14:15:46.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff922a00640 1 Processor -- start 2024-05-28T14:15:46.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff922a00640 1 -- start start 2024-05-28T14:15:46.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff922a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff91c0d11c0 0x7ff91c1675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff922a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff91c0cd600 con 0x7ff91c15ccb0 2024-05-28T14:15:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff922a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff91c0cd8e0 con 0x7ff91c0cd0d0 2024-05-28T14:15:46.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff922a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff91c0ce210 con 0x7ff91c0d11c0 2024-05-28T14:15:46.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91be00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff91c15ccb0 0x7ff91c15d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:58300/0 (socket says 172.21.15.42:58300) 2024-05-28T14:15:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91be00640 1 -- 172.21.15.42:0/2634793243 learned_addr learned my addr 172.21.15.42:0/2634793243 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:15:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff920a00640 1 --2- 172.21.15.42:0/2634793243 >> v2:172.21.15.63:3300/0 conn(0x7ff91c0d11c0 0x7ff91c1675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 -- 172.21.15.42:0/2634793243 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708619427 0 0) 0x7ff91c0cd600 con 0x7ff91c15ccb0 2024-05-28T14:15:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 -- 172.21.15.42:0/2634793243 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff900003530 con 0x7ff91c15ccb0 2024-05-28T14:15:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 -- 172.21.15.42:0/2634793243 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1390391545 0 0) 0x7ff900003530 con 0x7ff91c15ccb0 2024-05-28T14:15:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 -- 172.21.15.42:0/2634793243 >> v2:172.21.15.63:3300/0 conn(0x7ff91c0d11c0 msgr2=0x7ff91c1675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 --2- 172.21.15.42:0/2634793243 >> v2:172.21.15.63:3300/0 conn(0x7ff91c0d11c0 0x7ff91c1675a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 -- 172.21.15.42:0/2634793243 >> v1:172.21.15.63:6789/0 conn(0x7ff91c0cd0d0 legacy=0x7ff91c09b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 -- 172.21.15.42:0/2634793243 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c0cdbc0 con 0x7ff91c15ccb0 2024-05-28T14:15:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 -- 172.21.15.42:0/2634793243 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7ff90c001680 con 0x7ff91c15ccb0 2024-05-28T14:15:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 -- 172.21.15.42:0/2634793243 >> v1:172.21.15.42:6789/0 conn(0x7ff91c15ccb0 legacy=0x7ff91c15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 --2- 172.21.15.42:0/2634793243 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9000039c0 0x7ff900003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:46.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 --2- 172.21.15.42:0/2634793243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff900004810 0x7ff900004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:46.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 --2- 172.21.15.42:0/2634793243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9000051d0 0x7ff90000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:46.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 -- 172.21.15.42:0/2634793243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c0ce210 con 0x7ff900004810 2024-05-28T14:15:46.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 -- 172.21.15.42:0/2634793243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff90000c100 con 0x7ff9000051d0 2024-05-28T14:15:46.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91aa00640 1 -- 172.21.15.42:0/2634793243 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff90000c300 con 0x7ff9000039c0 2024-05-28T14:15:46.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.790+0000 7ff91b400640 1 --2- 172.21.15.42:0/2634793243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff900004810 0x7ff900004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:46.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.794+0000 7ff91be00640 1 --2- 172.21.15.42:0/2634793243 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9000039c0 0x7ff900003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:46.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.794+0000 7ff920a00640 1 --2- 172.21.15.42:0/2634793243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9000051d0 0x7ff90000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:46.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.794+0000 7ff91b400640 1 -- 172.21.15.42:0/2634793243 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9000039c0 msgr2=0x7ff900003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:46.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.794+0000 7ff91b400640 1 --2- 172.21.15.42:0/2634793243 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9000039c0 0x7ff900003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:46.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.794+0000 7ff91b400640 1 -- 172.21.15.42:0/2634793243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9000051d0 msgr2=0x7ff90000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:46.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.794+0000 7ff91b400640 1 --2- 172.21.15.42:0/2634793243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9000051d0 0x7ff90000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:46.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.794+0000 7ff91b400640 1 -- 172.21.15.42:0/2634793243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff900003530 con 0x7ff900004810 2024-05-28T14:15:46.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.794+0000 7ff920a00640 1 -- 172.21.15.42:0/2634793243 reap_dead start 2024-05-28T14:15:46.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.794+0000 7ff91be00640 1 --2- 172.21.15.42:0/2634793243 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9000039c0 0x7ff900003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:15:46.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.798+0000 7ff91b400640 1 --2- 172.21.15.42:0/2634793243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff900004810 0x7ff900004c90 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7ff908007fd0 tx=0x7ff90801c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b156ce6cb29573 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:46.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.798+0000 7ff91aa00640 1 -- 172.21.15.42:0/2634793243 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9080b0070 con 0x7ff900004810 2024-05-28T14:15:46.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.798+0000 7ff91aa00640 1 -- 172.21.15.42:0/2634793243 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9080ab620 con 0x7ff900004810 2024-05-28T14:15:46.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.798+0000 7ff91aa00640 1 -- 172.21.15.42:0/2634793243 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9080039b0 con 0x7ff900004810 2024-05-28T14:15:46.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.802+0000 7ff922a00640 1 -- 172.21.15.42:0/2634793243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff900004810 msgr2=0x7ff900004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:46.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.802+0000 7ff922a00640 1 --2- 172.21.15.42:0/2634793243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff900004810 0x7ff900004c90 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7ff908007fd0 tx=0x7ff90801c2e0 comp rx=0 tx=0).stop 2024-05-28T14:15:46.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.802+0000 7ff922a00640 1 -- 172.21.15.42:0/2634793243 shutdown_connections 2024-05-28T14:15:46.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.802+0000 7ff922a00640 1 --2- 172.21.15.42:0/2634793243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff900004810 0x7ff900004c90 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:46.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.802+0000 7ff922a00640 1 -- 172.21.15.42:0/2634793243 >> 172.21.15.42:0/2634793243 conn(0x7ff91c0b40d0 msgr2=0x7ff91c0b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:46.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.802+0000 7ff922a00640 1 -- 172.21.15.42:0/2634793243 shutdown_connections 2024-05-28T14:15:46.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.802+0000 7ff922a00640 1 -- 172.21.15.42:0/2634793243 wait complete. 2024-05-28T14:15:46.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff922a00640 1 Processor -- start 2024-05-28T14:15:46.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff922a00640 1 -- start start 2024-05-28T14:15:46.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff922a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff900004810 0x7ff91c14eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:46.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff922a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff91c1e4eb0 0x7ff91c0b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:46.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff922a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c15eed0 0x7ff91c15f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:46.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff922a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c15ca60 con 0x7ff91c1e4eb0 2024-05-28T14:15:46.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff922a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c15c8e0 con 0x7ff91c15eed0 2024-05-28T14:15:46.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff922a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff91c15d090 con 0x7ff900004810 2024-05-28T14:15:46.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff91b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff91c1e4eb0 0x7ff91c0b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:46.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff91b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff91c1e4eb0 0x7ff91c0b7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40850/0 (socket says 172.21.15.42:40850) 2024-05-28T14:15:46.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff91b400640 1 -- 172.21.15.42:0/1585397537 learned_addr learned my addr 172.21.15.42:0/1585397537 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:46.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff920a00640 1 --2- 172.21.15.42:0/1585397537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c15eed0 0x7ff91c15f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:46.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff91be00640 1 --2- 172.21.15.42:0/1585397537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff900004810 0x7ff91c14eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:46.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff91b400640 1 -- 172.21.15.42:0/1585397537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff900004810 msgr2=0x7ff91c14eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:46.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff91b400640 1 --2- 172.21.15.42:0/1585397537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff900004810 0x7ff91c14eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:46.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff91b400640 1 -- 172.21.15.42:0/1585397537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c15eed0 msgr2=0x7ff91c15f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:46.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff91b400640 1 --2- 172.21.15.42:0/1585397537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c15eed0 0x7ff91c15f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:46.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff91b400640 1 -- 172.21.15.42:0/1585397537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c159af0 con 0x7ff91c1e4eb0 2024-05-28T14:15:46.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff91b400640 1 --2- 172.21.15.42:0/1585397537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff91c1e4eb0 0x7ff91c0b7d10 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7ff90801c110 tx=0x7ff900005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:46.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff913e00640 1 -- 172.21.15.42:0/1585397537 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9080ba020 con 0x7ff91c1e4eb0 2024-05-28T14:15:46.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff913e00640 1 -- 172.21.15.42:0/1585397537 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9080ab7d0 con 0x7ff91c1e4eb0 2024-05-28T14:15:46.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff922a00640 1 -- 172.21.15.42:0/1585397537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91c159dd0 con 0x7ff91c1e4eb0 2024-05-28T14:15:46.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.806+0000 7ff913e00640 1 -- 172.21.15.42:0/1585397537 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9080b0040 con 0x7ff91c1e4eb0 2024-05-28T14:15:46.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.810+0000 7ff922a00640 1 -- 172.21.15.42:0/1585397537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff91c15a390 con 0x7ff91c1e4eb0 2024-05-28T14:15:46.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.810+0000 7ff922a00640 1 -- 172.21.15.42:0/1585397537 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f40052a0 con 0x7ff91c1e4eb0 2024-05-28T14:15:46.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.810+0000 7ff913e00640 1 -- 172.21.15.42:0/1585397537 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 47) ==== 90945+0+0 (secure 0 0 0) 0x7ff9080bd020 con 0x7ff91c1e4eb0 2024-05-28T14:15:46.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.814+0000 7ff913e00640 1 --2- 172.21.15.42:0/1585397537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8fc06a450 0x7ff8fc06c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:46.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.814+0000 7ff913e00640 1 -- 172.21.15.42:0/1585397537 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7ff90811fbd0 con 0x7ff91c1e4eb0 2024-05-28T14:15:46.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.814+0000 7ff91be00640 1 --2- 172.21.15.42:0/1585397537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8fc06a450 0x7ff8fc06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:46.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.814+0000 7ff91be00640 1 --2- 172.21.15.42:0/1585397537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8fc06a450 0x7ff8fc06c910 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff90c000c10 tx=0x7ff90c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:46.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:46.818+0000 7ff913e00640 1 -- 172.21.15.42:0/1585397537 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff9080c4070 con 0x7ff91c1e4eb0 2024-05-28T14:15:47.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.162+0000 7ff922a00640 1 -- 172.21.15.42:0/1585397537 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone6277", "target": ["mon-mgr", ""]}) -- 0x7ff8f4000bd0 con 0x7ff8fc06a450 2024-05-28T14:15:47.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.370+0000 7ff913e00640 1 -- 172.21.15.42:0/1585397537 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff8f4000bd0 con 0x7ff8fc06a450 2024-05-28T14:15:47.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.370+0000 7ff922a00640 1 -- 172.21.15.42:0/1585397537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8fc06a450 msgr2=0x7ff8fc06c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:47.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.370+0000 7ff922a00640 1 --2- 172.21.15.42:0/1585397537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8fc06a450 0x7ff8fc06c910 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff90c000c10 tx=0x7ff90c009040 comp rx=0 tx=0).stop 2024-05-28T14:15:47.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.370+0000 7ff922a00640 1 -- 172.21.15.42:0/1585397537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff91c1e4eb0 msgr2=0x7ff91c0b7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:47.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.370+0000 7ff922a00640 1 --2- 172.21.15.42:0/1585397537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff91c1e4eb0 0x7ff91c0b7d10 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7ff90801c110 tx=0x7ff900005840 comp rx=0 tx=0).stop 2024-05-28T14:15:47.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.370+0000 7ff922a00640 1 -- 172.21.15.42:0/1585397537 shutdown_connections 2024-05-28T14:15:47.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.370+0000 7ff922a00640 1 --2- 172.21.15.42:0/1585397537 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff8fc06a450 0x7ff8fc06c910 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:47.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.370+0000 7ff922a00640 1 --2- 172.21.15.42:0/1585397537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff91c15eed0 0x7ff91c15f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:47.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.370+0000 7ff922a00640 1 --2- 172.21.15.42:0/1585397537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff91c1e4eb0 0x7ff91c0b7d10 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:47.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.370+0000 7ff922a00640 1 --2- 172.21.15.42:0/1585397537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff900004810 0x7ff91c14eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:47.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.370+0000 7ff922a00640 1 -- 172.21.15.42:0/1585397537 >> 172.21.15.42:0/1585397537 conn(0x7ff91c0b40d0 msgr2=0x7ff91c0cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:47.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.370+0000 7ff922a00640 1 -- 172.21.15.42:0/1585397537 shutdown_connections 2024-05-28T14:15:47.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.370+0000 7ff922a00640 1 -- 172.21.15.42:0/1585397537 wait complete. 2024-05-28T14:15:47.379 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:15:47.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.530+0000 7fd83b800640 1 Processor -- start 2024-05-28T14:15:47.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.530+0000 7fd83b800640 1 -- start start 2024-05-28T14:15:47.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd83b800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8340cd090 0x7fd8340cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:47.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd83b800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd8340b5760 0x7fd8340b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:47.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd83b800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8340d0760 con 0x7fd83415fd00 2024-05-28T14:15:47.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd83b800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd8340b52d0 con 0x7fd8340b5760 2024-05-28T14:15:47.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd83b800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd8340d0a30 con 0x7fd8340cd090 2024-05-28T14:15:47.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd839800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd8340b5760 0x7fd8340b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:47.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd82be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8340cd090 0x7fd8340cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:47.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd838e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fd83415fd00 0x7fd8340d0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:58308/0 (socket says 172.21.15.42:58308) 2024-05-28T14:15:47.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd838e00640 1 -- 172.21.15.42:0/2715995200 learned_addr learned my addr 172.21.15.42:0/2715995200 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:15:47.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd833e00640 1 -- 172.21.15.42:0/2715995200 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2301322399 0 0) 0x7fd8340d0760 con 0x7fd83415fd00 2024-05-28T14:15:47.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd833e00640 1 -- 172.21.15.42:0/2715995200 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd818003530 con 0x7fd83415fd00 2024-05-28T14:15:47.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd839800640 1 -- 172.21.15.42:0/2715995200 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8340cd090 msgr2=0x7fd8340cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:47.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd839800640 1 --2- 172.21.15.42:0/2715995200 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8340cd090 0x7fd8340cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:47.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd839800640 1 -- 172.21.15.42:0/2715995200 >> v1:172.21.15.63:6789/0 conn(0x7fd83415fd00 legacy=0x7fd8340d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:47.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd839800640 1 -- 172.21.15.42:0/2715995200 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8340d0d10 con 0x7fd8340b5760 2024-05-28T14:15:47.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.534+0000 7fd82be00640 1 --2- 172.21.15.42:0/2715995200 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8340cd090 0x7fd8340cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:15:47.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.538+0000 7fd839800640 1 --2- 172.21.15.42:0/2715995200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8340b5760 0x7fd8340b5bf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009d90 tx=0x7fd82c01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f569c1a70eae3d24 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.538+0000 7fd833e00640 1 -- 172.21.15.42:0/2715995200 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0b2070 con 0x7fd8340b5760 2024-05-28T14:15:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.538+0000 7fd833e00640 1 -- 172.21.15.42:0/2715995200 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd82c0a5a00 con 0x7fd8340b5760 2024-05-28T14:15:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.538+0000 7fd833e00640 1 -- 172.21.15.42:0/2715995200 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0a5dd0 con 0x7fd8340b5760 2024-05-28T14:15:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.538+0000 7fd83b800640 1 -- 172.21.15.42:0/2715995200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8340b5760 msgr2=0x7fd8340b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:47.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.538+0000 7fd83b800640 1 --2- 172.21.15.42:0/2715995200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8340b5760 0x7fd8340b5bf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009d90 tx=0x7fd82c01c940 comp rx=0 tx=0).stop 2024-05-28T14:15:47.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.538+0000 7fd83b800640 1 -- 172.21.15.42:0/2715995200 shutdown_connections 2024-05-28T14:15:47.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.538+0000 7fd83b800640 1 --2- 172.21.15.42:0/2715995200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8340b5760 0x7fd8340b5bf0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:47.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.538+0000 7fd83b800640 1 --2- 172.21.15.42:0/2715995200 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8340cd090 0x7fd8340cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:47.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.538+0000 7fd83b800640 1 -- 172.21.15.42:0/2715995200 >> 172.21.15.42:0/2715995200 conn(0x7fd8340b3f70 msgr2=0x7fd8340b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:47.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.538+0000 7fd83b800640 1 -- 172.21.15.42:0/2715995200 shutdown_connections 2024-05-28T14:15:47.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.538+0000 7fd83b800640 1 -- 172.21.15.42:0/2715995200 wait complete. 2024-05-28T14:15:47.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.538+0000 7fd83b800640 1 Processor -- start 2024-05-28T14:15:47.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.538+0000 7fd83b800640 1 -- start start 2024-05-28T14:15:47.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd83b800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8340b5760 0x7fd834151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:47.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd83b800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8340cd090 0x7fd83414ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:47.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd83b800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd83415fd00 0x7fd8340c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:47.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd83b800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd834160260 con 0x7fd8340b5760 2024-05-28T14:15:47.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd83b800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341600e0 con 0x7fd83415fd00 2024-05-28T14:15:47.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd83b800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd8340b34d0 con 0x7fd8340cd090 2024-05-28T14:15:47.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd838e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8340b5760 0x7fd834151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:47.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd838e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8340b5760 0x7fd834151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50950/0 (socket says 172.21.15.42:50950) 2024-05-28T14:15:47.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd838e00640 1 -- 172.21.15.42:0/1153635662 learned_addr learned my addr 172.21.15.42:0/1153635662 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:47.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd839800640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd83415fd00 0x7fd8340c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:47.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd82be00640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8340cd090 0x7fd83414ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:47.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd838e00640 1 -- 172.21.15.42:0/1153635662 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8340cd090 msgr2=0x7fd83414ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:47.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd838e00640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8340cd090 0x7fd83414ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:47.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd838e00640 1 -- 172.21.15.42:0/1153635662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd83415fd00 msgr2=0x7fd8340c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:47.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd838e00640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd83415fd00 0x7fd8340c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:47.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd838e00640 1 -- 172.21.15.42:0/1153635662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd834151c30 con 0x7fd8340b5760 2024-05-28T14:15:47.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd839800640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd83415fd00 0x7fd8340c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:15:47.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd838e00640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8340b5760 0x7fd834151650 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fd82400c3c0 tx=0x7fd82400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:47.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd831600640 1 -- 172.21.15.42:0/1153635662 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824017070 con 0x7fd8340b5760 2024-05-28T14:15:47.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd831600640 1 -- 172.21.15.42:0/1153635662 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd824005cf0 con 0x7fd8340b5760 2024-05-28T14:15:47.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd83b800640 1 -- 172.21.15.42:0/1153635662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd834151f10 con 0x7fd8340b5760 2024-05-28T14:15:47.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd831600640 1 -- 172.21.15.42:0/1153635662 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8240072f0 con 0x7fd8340b5760 2024-05-28T14:15:47.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd83b800640 1 -- 172.21.15.42:0/1153635662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8340c96f0 con 0x7fd8340b5760 2024-05-28T14:15:47.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.542+0000 7fd83b800640 1 -- 172.21.15.42:0/1153635662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd834152430 con 0x7fd8340b5760 2024-05-28T14:15:47.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.546+0000 7fd831600640 1 -- 172.21.15.42:0/1153635662 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 47) ==== 90945+0+0 (secure 0 0 0) 0x7fd82401a070 con 0x7fd8340b5760 2024-05-28T14:15:47.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.546+0000 7fd831600640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd81006a420 0x7fd81006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:47.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.546+0000 7fd831600640 1 -- 172.21.15.42:0/1153635662 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fd824084e80 con 0x7fd8340b5760 2024-05-28T14:15:47.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.546+0000 7fd82be00640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd81006a420 0x7fd81006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:47.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.546+0000 7fd82be00640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd81006a420 0x7fd81006c8e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009580 tx=0x7fd81c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:47.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.550+0000 7fd831600640 1 -- 172.21.15.42:0/1153635662 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd834152430 con 0x7fd8340b5760 2024-05-28T14:15:47.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.782+0000 7fd83b800640 1 -- 172.21.15.42:0/1153635662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd834068a90 con 0x7fd8340b5760 2024-05-28T14:15:47.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.782+0000 7fd831600640 1 -- 172.21.15.42:0/1153635662 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2877 (secure 0 0 0) 0x7fd8240567d0 con 0x7fd8340b5760 2024-05-28T14:15:47.769 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:15:47.769 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:15:47.770 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:15:47.770 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:15:47.770 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:15:47.770 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:15:47.770 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:15:47.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:47.770 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:15:47.770 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:15:47.770 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:15:47.770 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:15:47.770 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 1069, 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 134, 2024-05-28T14:15:47.771 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:15:47.772 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:15:47.772 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:15:47.772 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:15:47.772 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:15:47.772 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:15:47.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:47.772 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:15:47.772 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:15:47.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:15:47.772 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:15:47.772 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:15:47.772 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:47.772 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:47.772 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:15:47.773 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:15:47.773 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4662, 2024-05-28T14:15:47.773 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 19392299125, 2024-05-28T14:15:47.773 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 39214444544, 2024-05-28T14:15:47.773 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 733879668736, 2024-05-28T14:15:47.773 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:15:47.773 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 579278966, 2024-05-28T14:15:47.773 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 578627763, 2024-05-28T14:15:47.773 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 140, 2024-05-28T14:15:47.773 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 152 2024-05-28T14:15:47.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:47.773 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:15:47.773 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 85, 2024-05-28T14:15:47.773 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 9, 2024-05-28T14:15:47.774 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:15:47.774 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:15:47.774 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:15:47.774 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:15:47.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:15:47.774 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 9, 2024-05-28T14:15:47.774 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:15:47.774 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:15:47.774 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:15:47.774 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57851 2024-05-28T14:15:47.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:47.774 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:47.775 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:15:47.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:47.775 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:15:47.775 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:15:47.775 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:15:47.775 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:15:47.775 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:15:47.775 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:15:47.775 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:15:47.775 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:47.775 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:15:47.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:47.775 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:15:47.775 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 23, 2024-05-28T14:15:47.776 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:15:41.227805+0000", 2024-05-28T14:15:47.776 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-05-28T14:15:47.776 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-05-28T14:15:47.776 INFO:teuthology.orchestra.run.smithi042.stdout: "daemons": { 2024-05-28T14:15:47.776 INFO:teuthology.orchestra.run.smithi042.stdout: "summary": "", 2024-05-28T14:15:47.776 INFO:teuthology.orchestra.run.smithi042.stdout: "d": { 2024-05-28T14:15:47.776 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:15:47.776 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:15:47.776 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:15:47.776 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:15:47.776 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:15:47.776 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:15:47.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:47.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:47.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:47.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:47.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:47.777 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:15:47.777 INFO:teuthology.orchestra.run.smithi042.stdout: "4fe61782-f924-464c-a4bf-e422bf3c1add": { 2024-05-28T14:15:47.777 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "2 ongoing clones; Avg progress is 57.552% (24s)\n [================............] (remaining: 18s)", 2024-05-28T14:15:47.777 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.57551997900009155, 2024-05-28T14:15:47.777 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:15:47.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:47.777 INFO:teuthology.orchestra.run.smithi042.stdout: "a3d48d0e-354a-43e1-9d6b-92bf56d654e0": { 2024-05-28T14:15:47.777 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "2 ongoing+pending clones; Avg progress is 57.552% (18s)\n [================............] (remaining: 13s)", 2024-05-28T14:15:47.777 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 0.57551997900009155, 2024-05-28T14:15:47.777 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:15:47.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:47.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:47.778 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:15:47.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.782+0000 7fd83b800640 1 -- 172.21.15.42:0/1153635662 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd81006a420 msgr2=0x7fd81006c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:47.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.782+0000 7fd83b800640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd81006a420 0x7fd81006c8e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009580 tx=0x7fd81c009190 comp rx=0 tx=0).stop 2024-05-28T14:15:47.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.782+0000 7fd83b800640 1 -- 172.21.15.42:0/1153635662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8340b5760 msgr2=0x7fd834151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:47.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.782+0000 7fd83b800640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8340b5760 0x7fd834151650 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fd82400c3c0 tx=0x7fd82400c880 comp rx=0 tx=0).stop 2024-05-28T14:15:47.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.782+0000 7fd83b800640 1 -- 172.21.15.42:0/1153635662 shutdown_connections 2024-05-28T14:15:47.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.782+0000 7fd83b800640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fd81006a420 0x7fd81006c8e0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:47.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.782+0000 7fd83b800640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd83415fd00 0x7fd8340c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:47.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.782+0000 7fd83b800640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8340cd090 0x7fd83414ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:47.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.782+0000 7fd83b800640 1 --2- 172.21.15.42:0/1153635662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8340b5760 0x7fd834151650 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:47.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.782+0000 7fd83b800640 1 -- 172.21.15.42:0/1153635662 >> 172.21.15.42:0/1153635662 conn(0x7fd8340b3f70 msgr2=0x7fd8340b7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:47.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.782+0000 7fd83b800640 1 -- 172.21.15.42:0/1153635662 shutdown_connections 2024-05-28T14:15:47.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:47.782+0000 7fd83b800640 1 -- 172.21.15.42:0/1153635662 wait complete. 2024-05-28T14:15:49.795 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:15:49.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.958+0000 7f61bf800640 1 Processor -- start 2024-05-28T14:15:49.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.958+0000 7f61bf800640 1 -- start start 2024-05-28T14:15:49.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61bf800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61b80b52d0 0x7f61b809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:49.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61bf800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f61b809af90 0x7f61b80b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:49.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61bf800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61b80ccce0 con 0x7f61b815e9d0 2024-05-28T14:15:49.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61bf800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f61b80ccfc0 con 0x7f61b809af90 2024-05-28T14:15:49.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61bf800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f61b80cd2a0 con 0x7f61b80b52d0 2024-05-28T14:15:49.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61bce00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f61b815e9d0 0x7f61b80d12c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:33922/0 (socket says 172.21.15.42:33922) 2024-05-28T14:15:49.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61bce00640 1 -- 172.21.15.42:0/783642599 learned_addr learned my addr 172.21.15.42:0/783642599 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:15:49.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61afe00640 1 --2- 172.21.15.42:0/783642599 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61b80b52d0 0x7f61b809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:49.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61bd800640 1 --2- 172.21.15.42:0/783642599 >> v2:172.21.15.63:3300/0 conn(0x7f61b809af90 0x7f61b80b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:49.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61b7e00640 1 -- 172.21.15.42:0/783642599 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1124974258 0 0) 0x7f61b80ccce0 con 0x7f61b815e9d0 2024-05-28T14:15:49.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61b7e00640 1 -- 172.21.15.42:0/783642599 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f619c003530 con 0x7f61b815e9d0 2024-05-28T14:15:49.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61afe00640 1 -- 172.21.15.42:0/783642599 >> v2:172.21.15.63:3300/0 conn(0x7f61b809af90 msgr2=0x7f61b80b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:49.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61afe00640 1 --2- 172.21.15.42:0/783642599 >> v2:172.21.15.63:3300/0 conn(0x7f61b809af90 0x7f61b80b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:49.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61b7e00640 1 -- 172.21.15.42:0/783642599 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3889106415 0 0) 0x7f619c003530 con 0x7f61b815e9d0 2024-05-28T14:15:49.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61afe00640 1 -- 172.21.15.42:0/783642599 >> v1:172.21.15.42:6789/0 conn(0x7f61b815e9d0 legacy=0x7f61b80d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:49.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61afe00640 1 -- 172.21.15.42:0/783642599 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b80cd580 con 0x7f61b80b52d0 2024-05-28T14:15:49.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.962+0000 7f61bd800640 1 --2- 172.21.15.42:0/783642599 >> v2:172.21.15.63:3300/0 conn(0x7f61b809af90 0x7f61b80b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:15:49.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.966+0000 7f61afe00640 1 --2- 172.21.15.42:0/783642599 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61b80b52d0 0x7f61b809aa50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f61a00097c0 tx=0x7f61a001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d22201e924d23062 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:49.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.966+0000 7f61b7e00640 1 -- 172.21.15.42:0/783642599 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a00b0070 con 0x7f61b80b52d0 2024-05-28T14:15:49.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.966+0000 7f61b7e00640 1 -- 172.21.15.42:0/783642599 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a00ab400 con 0x7f61b80b52d0 2024-05-28T14:15:49.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.966+0000 7f61b7e00640 1 -- 172.21.15.42:0/783642599 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8001680 con 0x7f61b80b52d0 2024-05-28T14:15:49.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.966+0000 7f61bf800640 1 -- 172.21.15.42:0/783642599 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61b80b52d0 msgr2=0x7f61b809aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:49.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.966+0000 7f61bf800640 1 --2- 172.21.15.42:0/783642599 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61b80b52d0 0x7f61b809aa50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f61a00097c0 tx=0x7f61a001c3c0 comp rx=0 tx=0).stop 2024-05-28T14:15:49.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.966+0000 7f61bf800640 1 -- 172.21.15.42:0/783642599 shutdown_connections 2024-05-28T14:15:49.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.966+0000 7f61bf800640 1 --2- 172.21.15.42:0/783642599 >> v2:172.21.15.63:3300/0 conn(0x7f61b809af90 0x7f61b80b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:49.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.966+0000 7f61bf800640 1 --2- 172.21.15.42:0/783642599 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61b80b52d0 0x7f61b809aa50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:49.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.966+0000 7f61bf800640 1 -- 172.21.15.42:0/783642599 >> 172.21.15.42:0/783642599 conn(0x7f61b80b3f70 msgr2=0x7f61b80b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:49.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.966+0000 7f61bf800640 1 -- 172.21.15.42:0/783642599 shutdown_connections 2024-05-28T14:15:49.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.966+0000 7f61bf800640 1 -- 172.21.15.42:0/783642599 wait complete. 2024-05-28T14:15:49.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.966+0000 7f61bf800640 1 Processor -- start 2024-05-28T14:15:49.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bf800640 1 -- start start 2024-05-28T14:15:49.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bf800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61b809af90 0x7f61b81527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:49.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bf800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61b80b52d0 0x7f61b80c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:49.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bf800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b815e9d0 0x7f61b80caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:49.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bf800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61b80cdf00 con 0x7f61b815e9d0 2024-05-28T14:15:49.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bf800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f61b80cdd80 con 0x7f61b80b52d0 2024-05-28T14:15:49.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bf800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f61b80ce080 con 0x7f61b809af90 2024-05-28T14:15:49.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b815e9d0 0x7f61b80caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:49.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b815e9d0 0x7f61b80caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50964/0 (socket says 172.21.15.42:50964) 2024-05-28T14:15:49.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bd800640 1 -- 172.21.15.42:0/4229504033 learned_addr learned my addr 172.21.15.42:0/4229504033 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:49.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bce00640 1 --2- 172.21.15.42:0/4229504033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61b809af90 0x7f61b81527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:49.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61afe00640 1 --2- 172.21.15.42:0/4229504033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61b80b52d0 0x7f61b80c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:49.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bd800640 1 -- 172.21.15.42:0/4229504033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61b809af90 msgr2=0x7f61b81527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:49.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bd800640 1 --2- 172.21.15.42:0/4229504033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61b809af90 0x7f61b81527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:49.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bd800640 1 -- 172.21.15.42:0/4229504033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61b80b52d0 msgr2=0x7f61b80c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:49.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bd800640 1 --2- 172.21.15.42:0/4229504033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61b80b52d0 0x7f61b80c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:49.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bd800640 1 -- 172.21.15.42:0/4229504033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b80c9ea0 con 0x7f61b815e9d0 2024-05-28T14:15:49.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bd800640 1 --2- 172.21.15.42:0/4229504033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b815e9d0 0x7f61b80caf40 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009f00 tx=0x7f61b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:49.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61b5600640 1 -- 172.21.15.42:0/4229504033 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0018030 con 0x7f61b815e9d0 2024-05-28T14:15:49.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61b5600640 1 -- 172.21.15.42:0/4229504033 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b000ce50 con 0x7f61b815e9d0 2024-05-28T14:15:49.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61b5600640 1 -- 172.21.15.42:0/4229504033 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0003bc0 con 0x7f61b815e9d0 2024-05-28T14:15:49.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bf800640 1 -- 172.21.15.42:0/4229504033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b80ca180 con 0x7f61b815e9d0 2024-05-28T14:15:49.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bf800640 1 -- 172.21.15.42:0/4229504033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b80ca740 con 0x7f61b815e9d0 2024-05-28T14:15:49.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.970+0000 7f61bf800640 1 -- 172.21.15.42:0/4229504033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b8068a90 con 0x7f61b815e9d0 2024-05-28T14:15:49.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.974+0000 7f61b5600640 1 -- 172.21.15.42:0/4229504033 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 47) ==== 90945+0+0 (secure 0 0 0) 0x7f61b001b070 con 0x7f61b815e9d0 2024-05-28T14:15:49.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.974+0000 7f61b5600640 1 --2- 172.21.15.42:0/4229504033 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f619406a420 0x7f619406c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:49.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.974+0000 7f61b5600640 1 -- 172.21.15.42:0/4229504033 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f61b00863a0 con 0x7f61b815e9d0 2024-05-28T14:15:49.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.974+0000 7f61bce00640 1 --2- 172.21.15.42:0/4229504033 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f619406a420 0x7f619406c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:49.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.974+0000 7f61bce00640 1 --2- 172.21.15.42:0/4229504033 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f619406a420 0x7f619406c8e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f61a80017f0 tx=0x7f61a8001580 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:49.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:49.978+0000 7f61b5600640 1 -- 172.21.15.42:0/4229504033 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f61b0010040 con 0x7f61b815e9d0 2024-05-28T14:15:50.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.198+0000 7f61bf800640 1 -- 172.21.15.42:0/4229504033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f61b8070350 con 0x7f61b815e9d0 2024-05-28T14:15:50.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.198+0000 7f61b5600640 1 -- 172.21.15.42:0/4229504033 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2835 (secure 0 0 0) 0x7f61b0057d70 con 0x7f61b815e9d0 2024-05-28T14:15:50.188 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:15:50.188 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:15:50.188 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:15:50.188 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:15:50.188 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:15:50.188 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:15:50.188 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:15:50.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:50.188 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:15:50.188 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:15:50.188 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:15:50.188 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:15:50.189 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:15:50.189 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:50.189 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:15:50.189 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:15:50.189 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:15:50.189 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:15:50.189 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:50.189 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 1071, 2024-05-28T14:15:50.189 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:15:50.189 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:15:50.189 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:15:50.189 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:15:50.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:50.190 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:15:50.190 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 134, 2024-05-28T14:15:50.190 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:15:50.190 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:15:50.190 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:15:50.190 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:15:50.190 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:15:50.190 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:15:50.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:50.190 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:15:50.190 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:15:50.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:15:50.190 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:15:50.191 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:15:50.191 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:50.191 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:50.191 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:15:50.191 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:15:50.191 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4662, 2024-05-28T14:15:50.191 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 19392299125, 2024-05-28T14:15:50.191 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 39214444544, 2024-05-28T14:15:50.191 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 733879668736, 2024-05-28T14:15:50.191 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:15:50.191 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 579427627, 2024-05-28T14:15:50.191 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 578776257, 2024-05-28T14:15:50.191 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 140, 2024-05-28T14:15:50.191 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 152 2024-05-28T14:15:50.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:50.192 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:15:50.192 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 85, 2024-05-28T14:15:50.192 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 9, 2024-05-28T14:15:50.192 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:15:50.192 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:15:50.192 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:15:50.192 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:15:50.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:15:50.192 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 9, 2024-05-28T14:15:50.192 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:15:50.192 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:15:50.192 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:15:50.192 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57851 2024-05-28T14:15:50.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:50.193 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:50.193 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:15:50.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:50.193 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:15:50.193 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:15:50.193 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:15:50.193 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:15:50.193 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:15:50.193 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:15:50.193 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:15:50.193 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:50.193 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:15:50.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:50.194 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:15:50.194 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 23, 2024-05-28T14:15:50.194 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:15:41.227805+0000", 2024-05-28T14:15:50.194 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-05-28T14:15:50.194 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-05-28T14:15:50.194 INFO:teuthology.orchestra.run.smithi042.stdout: "daemons": { 2024-05-28T14:15:50.194 INFO:teuthology.orchestra.run.smithi042.stdout: "summary": "", 2024-05-28T14:15:50.194 INFO:teuthology.orchestra.run.smithi042.stdout: "d": { 2024-05-28T14:15:50.194 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:15:50.194 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:15:50.194 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:15:50.194 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:15:50.194 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:15:50.194 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: "4fe61782-f924-464c-a4bf-e422bf3c1add": { 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 29.0 seconds before exiting (26s)\n [============================] ", 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: "a3d48d0e-354a-43e1-9d6b-92bf56d654e0": { 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 29.0 seconds before exiting (20s)\n [============================] ", 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:15:50.195 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:15:50.196 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:50.196 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:50.196 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:15:50.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.202+0000 7f61bf800640 1 -- 172.21.15.42:0/4229504033 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f619406a420 msgr2=0x7f619406c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:50.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.202+0000 7f61bf800640 1 --2- 172.21.15.42:0/4229504033 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f619406a420 0x7f619406c8e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f61a80017f0 tx=0x7f61a8001580 comp rx=0 tx=0).stop 2024-05-28T14:15:50.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.202+0000 7f61bf800640 1 -- 172.21.15.42:0/4229504033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b815e9d0 msgr2=0x7f61b80caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:50.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.202+0000 7f61bf800640 1 --2- 172.21.15.42:0/4229504033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b815e9d0 0x7f61b80caf40 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009f00 tx=0x7f61b000c490 comp rx=0 tx=0).stop 2024-05-28T14:15:50.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.202+0000 7f61bf800640 1 -- 172.21.15.42:0/4229504033 shutdown_connections 2024-05-28T14:15:50.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.202+0000 7f61bf800640 1 --2- 172.21.15.42:0/4229504033 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f619406a420 0x7f619406c8e0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.202+0000 7f61bf800640 1 --2- 172.21.15.42:0/4229504033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61b815e9d0 0x7f61b80caf40 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.202+0000 7f61bf800640 1 --2- 172.21.15.42:0/4229504033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61b80b52d0 0x7f61b80c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.202+0000 7f61bf800640 1 --2- 172.21.15.42:0/4229504033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61b809af90 0x7f61b81527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.202+0000 7f61bf800640 1 -- 172.21.15.42:0/4229504033 >> 172.21.15.42:0/4229504033 conn(0x7f61b80b3f70 msgr2=0x7f61b80b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:50.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.202+0000 7f61bf800640 1 -- 172.21.15.42:0/4229504033 shutdown_connections 2024-05-28T14:15:50.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.202+0000 7f61bf800640 1 -- 172.21.15.42:0/4229504033 wait complete. 2024-05-28T14:15:50.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:15:50.209 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone6271/2eaf73b2-6b4e-4b8f-acd8-93d1b171c401 2024-05-28T14:15:50.210 DEBUG:teuthology.orchestra.run.smithi042:> ) 2024-05-28T14:15:50.224 INFO:teuthology.orchestra.run.smithi042.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone6271/2eaf73b2-6b4e-4b8f-acd8-93d1b171c401': No such file or directory 2024-05-28T14:15:50.225 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-28T14:15:50.225 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-28T14:15:50.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf7ba00640 1 Processor -- start 2024-05-28T14:15:50.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf7ba00640 1 -- start start 2024-05-28T14:15:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf7ba00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faf740cf230 0x7faf74072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf7ba00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf74072ba0 0x7faf7406ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf7ba00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf740cfc10 con 0x7faf7415bf30 2024-05-28T14:15:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf7ba00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faf740721d0 con 0x7faf740cf230 2024-05-28T14:15:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf7ba00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faf740cfef0 con 0x7faf74072ba0 2024-05-28T14:15:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf73e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faf740cf230 0x7faf74072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf79a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf74072ba0 0x7faf7406ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf79a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf74072ba0 0x7faf7406ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:60370/0 (socket says 172.21.15.42:60370) 2024-05-28T14:15:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf79a00640 1 -- 172.21.15.42:0/3040526849 learned_addr learned my addr 172.21.15.42:0/3040526849 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:50.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf73e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faf740cf230 0x7faf74072660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:54398/0 (socket says 172.21.15.42:54398) 2024-05-28T14:15:50.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf79000640 1 --1- 172.21.15.42:0/3040526849 >> v1:172.21.15.63:6789/0 conn(0x7faf7415bf30 0x7faf7415c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:58316/0 (socket says 172.21.15.42:58316) 2024-05-28T14:15:50.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf73400640 1 -- 172.21.15.42:0/3040526849 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1351729160 0 0) 0x7faf740cfc10 con 0x7faf7415bf30 2024-05-28T14:15:50.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf73e00640 1 -- 172.21.15.42:0/3040526849 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf74072ba0 msgr2=0x7faf7406ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:50.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf73e00640 1 --2- 172.21.15.42:0/3040526849 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf74072ba0 0x7faf7406ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf73e00640 1 -- 172.21.15.42:0/3040526849 >> v1:172.21.15.63:6789/0 conn(0x7faf7415bf30 legacy=0x7faf7415c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:50.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf73e00640 1 -- 172.21.15.42:0/3040526849 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf740d01d0 con 0x7faf740cf230 2024-05-28T14:15:50.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.422+0000 7faf79a00640 1 --2- 172.21.15.42:0/3040526849 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf74072ba0 0x7faf7406ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:15:50.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.426+0000 7faf73e00640 1 --2- 172.21.15.42:0/3040526849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf740cf230 0x7faf74072660 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7faf600097c0 tx=0x7faf6001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d95e1ccffdf63394 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:50.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.426+0000 7faf73400640 1 -- 172.21.15.42:0/3040526849 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf600b1070 con 0x7faf740cf230 2024-05-28T14:15:50.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.426+0000 7faf73400640 1 -- 172.21.15.42:0/3040526849 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf600ac410 con 0x7faf740cf230 2024-05-28T14:15:50.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.426+0000 7faf73400640 1 -- 172.21.15.42:0/3040526849 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf600ac7e0 con 0x7faf740cf230 2024-05-28T14:15:50.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.426+0000 7faf7ba00640 1 -- 172.21.15.42:0/3040526849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf740cf230 msgr2=0x7faf74072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:50.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.426+0000 7faf7ba00640 1 --2- 172.21.15.42:0/3040526849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf740cf230 0x7faf74072660 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7faf600097c0 tx=0x7faf6001c4a0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.426+0000 7faf7ba00640 1 -- 172.21.15.42:0/3040526849 shutdown_connections 2024-05-28T14:15:50.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.426+0000 7faf7ba00640 1 --2- 172.21.15.42:0/3040526849 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf74072ba0 0x7faf7406ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.426+0000 7faf7ba00640 1 --2- 172.21.15.42:0/3040526849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf740cf230 0x7faf74072660 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.426+0000 7faf7ba00640 1 -- 172.21.15.42:0/3040526849 >> 172.21.15.42:0/3040526849 conn(0x7faf7406b9f0 msgr2=0x7faf7406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:50.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.426+0000 7faf7ba00640 1 -- 172.21.15.42:0/3040526849 shutdown_connections 2024-05-28T14:15:50.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.426+0000 7faf7ba00640 1 -- 172.21.15.42:0/3040526849 wait complete. 2024-05-28T14:15:50.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf7ba00640 1 Processor -- start 2024-05-28T14:15:50.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf7ba00640 1 -- start start 2024-05-28T14:15:50.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf7ba00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf74072ba0 0x7faf74152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:50.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf7ba00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf740cf230 0x7faf7414bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:50.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf7ba00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf7415bf30 0x7faf74167680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:50.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf7ba00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faf74166570 con 0x7faf7415bf30 2024-05-28T14:15:50.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf7ba00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf741663f0 con 0x7faf740cf230 2024-05-28T14:15:50.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf7ba00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faf741666f0 con 0x7faf74072ba0 2024-05-28T14:15:50.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf79000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf74072ba0 0x7faf74152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:50.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf79a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf7415bf30 0x7faf74167680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:50.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf79000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf74072ba0 0x7faf74152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:60372/0 (socket says 172.21.15.42:60372) 2024-05-28T14:15:50.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf79000640 1 -- 172.21.15.42:0/674374884 learned_addr learned my addr 172.21.15.42:0/674374884 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:50.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf73e00640 1 --2- 172.21.15.42:0/674374884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf740cf230 0x7faf7414bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:50.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf79a00640 1 -- 172.21.15.42:0/674374884 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf74072ba0 msgr2=0x7faf74152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:50.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf79a00640 1 --2- 172.21.15.42:0/674374884 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf74072ba0 0x7faf74152930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf79a00640 1 -- 172.21.15.42:0/674374884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf740cf230 msgr2=0x7faf7414bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:50.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf79a00640 1 --2- 172.21.15.42:0/674374884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf740cf230 0x7faf7414bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf79a00640 1 -- 172.21.15.42:0/674374884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf74155e30 con 0x7faf7415bf30 2024-05-28T14:15:50.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf79a00640 1 --2- 172.21.15.42:0/674374884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf7415bf30 0x7faf74167680 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00efc0 tx=0x7faf6c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:50.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf70c00640 1 -- 172.21.15.42:0/674374884 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c00cb20 con 0x7faf7415bf30 2024-05-28T14:15:50.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf70c00640 1 -- 172.21.15.42:0/674374884 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf6c00cce0 con 0x7faf7415bf30 2024-05-28T14:15:50.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf70c00640 1 -- 172.21.15.42:0/674374884 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c003be0 con 0x7faf7415bf30 2024-05-28T14:15:50.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf7ba00640 1 -- 172.21.15.42:0/674374884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf74152e70 con 0x7faf7415bf30 2024-05-28T14:15:50.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.430+0000 7faf7ba00640 1 -- 172.21.15.42:0/674374884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf74155870 con 0x7faf7415bf30 2024-05-28T14:15:50.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.434+0000 7faf7ba00640 1 -- 172.21.15.42:0/674374884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf74073890 con 0x7faf7415bf30 2024-05-28T14:15:50.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.434+0000 7faf70c00640 1 -- 172.21.15.42:0/674374884 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 47) ==== 90945+0+0 (secure 0 0 0) 0x7faf6c01b030 con 0x7faf7415bf30 2024-05-28T14:15:50.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.434+0000 7faf70c00640 1 --2- 172.21.15.42:0/674374884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf5406a420 0x7faf5406c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:50.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.434+0000 7faf70c00640 1 -- 172.21.15.42:0/674374884 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7faf6c085d50 con 0x7faf7415bf30 2024-05-28T14:15:50.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.434+0000 7faf79000640 1 --2- 172.21.15.42:0/674374884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf5406a420 0x7faf5406c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:50.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.434+0000 7faf79000640 1 --2- 172.21.15.42:0/674374884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf5406a420 0x7faf5406c8e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7faf68000f70 tx=0x7faf68007ee0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:50.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.438+0000 7faf70c00640 1 -- 172.21.15.42:0/674374884 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faf6c022020 con 0x7faf7415bf30 2024-05-28T14:15:50.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.626+0000 7faf7ba00640 1 -- 172.21.15.42:0/674374884 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7faf7404fdd0 con 0x7faf5406a420 2024-05-28T14:15:50.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.630+0000 7faf70c00640 1 -- 172.21.15.42:0/674374884 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7faf7404fdd0 con 0x7faf5406a420 2024-05-28T14:15:50.616 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:15:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:15:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephfs" 2024-05-28T14:15:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:50.616 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:15:50.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.630+0000 7faf7ba00640 1 -- 172.21.15.42:0/674374884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf5406a420 msgr2=0x7faf5406c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:50.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.630+0000 7faf7ba00640 1 --2- 172.21.15.42:0/674374884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf5406a420 0x7faf5406c8e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7faf68000f70 tx=0x7faf68007ee0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.630+0000 7faf7ba00640 1 -- 172.21.15.42:0/674374884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf7415bf30 msgr2=0x7faf74167680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:50.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.630+0000 7faf7ba00640 1 --2- 172.21.15.42:0/674374884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf7415bf30 0x7faf74167680 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00efc0 tx=0x7faf6c00c490 comp rx=0 tx=0).stop 2024-05-28T14:15:50.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.630+0000 7faf7ba00640 1 -- 172.21.15.42:0/674374884 shutdown_connections 2024-05-28T14:15:50.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.630+0000 7faf7ba00640 1 --2- 172.21.15.42:0/674374884 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7faf5406a420 0x7faf5406c8e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.630+0000 7faf7ba00640 1 --2- 172.21.15.42:0/674374884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf7415bf30 0x7faf74167680 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.630+0000 7faf7ba00640 1 --2- 172.21.15.42:0/674374884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf740cf230 0x7faf7414bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.630+0000 7faf7ba00640 1 --2- 172.21.15.42:0/674374884 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf74072ba0 0x7faf74152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.630+0000 7faf7ba00640 1 -- 172.21.15.42:0/674374884 >> 172.21.15.42:0/674374884 conn(0x7faf7406b9f0 msgr2=0x7faf7406bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:50.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.630+0000 7faf7ba00640 1 -- 172.21.15.42:0/674374884 shutdown_connections 2024-05-28T14:15:50.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.630+0000 7faf7ba00640 1 -- 172.21.15.42:0/674374884 wait complete. 2024-05-28T14:15:50.638 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:15:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed9800640 1 Processor -- start 2024-05-28T14:15:50.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed9800640 1 -- start start 2024-05-28T14:15:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed9800640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3ed415e9d0 0x7f3ed40d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed9800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3ed409b160 0x7f3ed40d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed9800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ed40ccce0 con 0x7f3ed40b52d0 2024-05-28T14:15:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed9800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f3ed40ccfc0 con 0x7f3ed409b160 2024-05-28T14:15:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed9800640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3ed40cd2a0 con 0x7f3ed415e9d0 2024-05-28T14:15:50.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3ed409b160 0x7f3ed40d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:50.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3ed409b160 0x7f3ed40d1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50978/0 (socket says 172.21.15.42:50978) 2024-05-28T14:15:50.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed3400640 1 -- 172.21.15.42:0/211341555 learned_addr learned my addr 172.21.15.42:0/211341555 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:50.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed3400640 1 -- 172.21.15.42:0/211341555 >> v2:172.21.15.63:3300/0 conn(0x7f3ed415e9d0 msgr2=0x7f3ed40d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:50.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed2a00640 1 --2- 172.21.15.42:0/211341555 >> v2:172.21.15.63:3300/0 conn(0x7f3ed415e9d0 0x7f3ed40d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:50.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed3400640 1 --2- 172.21.15.42:0/211341555 >> v2:172.21.15.63:3300/0 conn(0x7f3ed415e9d0 0x7f3ed40d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed3400640 1 -- 172.21.15.42:0/211341555 >> v1:172.21.15.42:6789/0 conn(0x7f3ed40b52d0 legacy=0x7f3ed409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:50.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed3400640 1 -- 172.21.15.42:0/211341555 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed40cd580 con 0x7f3ed409b160 2024-05-28T14:15:50.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.794+0000 7f3ed2a00640 1 --2- 172.21.15.42:0/211341555 >> v2:172.21.15.63:3300/0 conn(0x7f3ed415e9d0 0x7f3ed40d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:15:50.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.798+0000 7f3ed3400640 1 --2- 172.21.15.42:0/211341555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ed409b160 0x7f3ed40d1800 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f3ed406c490 tx=0x7f3ecc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b657785177e9d51 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:50.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.798+0000 7f3ed1600640 1 -- 172.21.15.42:0/211341555 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc0a5750 con 0x7f3ed409b160 2024-05-28T14:15:50.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.798+0000 7f3ed1600640 1 -- 172.21.15.42:0/211341555 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ecc0a58f0 con 0x7f3ed409b160 2024-05-28T14:15:50.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.798+0000 7f3ed1600640 1 -- 172.21.15.42:0/211341555 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc0a5cc0 con 0x7f3ed409b160 2024-05-28T14:15:50.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.798+0000 7f3ed9800640 1 -- 172.21.15.42:0/211341555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ed409b160 msgr2=0x7f3ed40d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:50.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.798+0000 7f3ed9800640 1 --2- 172.21.15.42:0/211341555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ed409b160 0x7f3ed40d1800 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f3ed406c490 tx=0x7f3ecc01c9f0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.798+0000 7f3ed9800640 1 -- 172.21.15.42:0/211341555 shutdown_connections 2024-05-28T14:15:50.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.798+0000 7f3ed9800640 1 --2- 172.21.15.42:0/211341555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ed409b160 0x7f3ed40d1800 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.798+0000 7f3ed9800640 1 --2- 172.21.15.42:0/211341555 >> v2:172.21.15.63:3300/0 conn(0x7f3ed415e9d0 0x7f3ed40d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.798+0000 7f3ed9800640 1 -- 172.21.15.42:0/211341555 >> 172.21.15.42:0/211341555 conn(0x7f3ed40b3f70 msgr2=0x7f3ed40b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:50.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.798+0000 7f3ed9800640 1 -- 172.21.15.42:0/211341555 shutdown_connections 2024-05-28T14:15:50.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.798+0000 7f3ed9800640 1 -- 172.21.15.42:0/211341555 wait complete. 2024-05-28T14:15:50.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.798+0000 7f3ed9800640 1 Processor -- start 2024-05-28T14:15:50.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.798+0000 7f3ed9800640 1 -- start start 2024-05-28T14:15:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ed409b160 0x7f3ed4155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ed40b52d0 0x7f3ed40b6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed9800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ed415e9d0 0x7f3ed41535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed9800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed40b3c30 con 0x7f3ed40b52d0 2024-05-28T14:15:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed9800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed40b3ab0 con 0x7f3ed409b160 2024-05-28T14:15:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed9800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed40b3db0 con 0x7f3ed415e9d0 2024-05-28T14:15:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ed40b52d0 0x7f3ed40b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:50.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ed40b52d0 0x7f3ed40b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50992/0 (socket says 172.21.15.42:50992) 2024-05-28T14:15:50.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed2000640 1 -- 172.21.15.42:0/734885106 learned_addr learned my addr 172.21.15.42:0/734885106 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed2a00640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ed409b160 0x7f3ed4155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed3400640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ed415e9d0 0x7f3ed41535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed2000640 1 -- 172.21.15.42:0/734885106 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ed415e9d0 msgr2=0x7f3ed41535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed2000640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ed415e9d0 0x7f3ed41535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed2000640 1 -- 172.21.15.42:0/734885106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ed409b160 msgr2=0x7f3ed4155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed2000640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ed409b160 0x7f3ed4155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed2000640 1 -- 172.21.15.42:0/734885106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed4154380 con 0x7f3ed40b52d0 2024-05-28T14:15:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed3400640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ed415e9d0 0x7f3ed41535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:15:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed2000640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ed40b52d0 0x7f3ed40b6990 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000c430 tx=0x7f3ec000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ecaa00640 1 -- 172.21.15.42:0/734885106 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00052d0 con 0x7f3ed40b52d0 2024-05-28T14:15:50.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ecaa00640 1 -- 172.21.15.42:0/734885106 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec0005470 con 0x7f3ed40b52d0 2024-05-28T14:15:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ecaa00640 1 -- 172.21.15.42:0/734885106 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0007260 con 0x7f3ed40b52d0 2024-05-28T14:15:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed9800640 1 -- 172.21.15.42:0/734885106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed4154620 con 0x7f3ed40b52d0 2024-05-28T14:15:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed9800640 1 -- 172.21.15.42:0/734885106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed4154040 con 0x7f3ed40b52d0 2024-05-28T14:15:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ed9800640 1 -- 172.21.15.42:0/734885106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed4154a10 con 0x7f3ed40b52d0 2024-05-28T14:15:50.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.802+0000 7f3ecaa00640 1 -- 172.21.15.42:0/734885106 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 47) ==== 90945+0+0 (secure 0 0 0) 0x7f3ec0007400 con 0x7f3ed40b52d0 2024-05-28T14:15:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.806+0000 7f3ecaa00640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3eb406a450 0x7f3eb406c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.806+0000 7f3ecaa00640 1 -- 172.21.15.42:0/734885106 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f3ec0084f50 con 0x7f3ed40b52d0 2024-05-28T14:15:50.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.806+0000 7f3ed2a00640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3eb406a450 0x7f3eb406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:50.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.806+0000 7f3ed2a00640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3eb406a450 0x7f3eb406c910 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4006fd0 tx=0x7f3ec4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:50.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:50.810+0000 7f3ecaa00640 1 -- 172.21.15.42:0/734885106 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3ec0057010 con 0x7f3ed40b52d0 2024-05-28T14:15:51.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:51.034+0000 7f3ed9800640 1 -- 172.21.15.42:0/734885106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3ed41547f0 con 0x7f3ed40b52d0 2024-05-28T14:15:51.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:51.034+0000 7f3ecaa00640 1 -- 172.21.15.42:0/734885106 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2835 (secure 0 0 0) 0x7f3ec0056910 con 0x7f3ed40b52d0 2024-05-28T14:15:51.021 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:15:51.021 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:15:51.021 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:15:51.021 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:15:51.021 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:15:51.021 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:15:51.021 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:15:51.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:51.021 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:15:51.021 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:15:51.022 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:15:51.022 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:15:51.022 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:15:51.022 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:51.022 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:15:51.022 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:15:51.022 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:15:51.022 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:15:51.022 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:51.022 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 1072, 2024-05-28T14:15:51.022 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:15:51.022 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:15:51.022 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:15:51.023 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:15:51.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:51.023 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:15:51.023 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 134, 2024-05-28T14:15:51.023 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:15:51.023 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:15:51.023 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:15:51.023 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:15:51.023 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:15:51.023 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:15:51.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:51.023 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:15:51.023 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:15:51.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:15:51.024 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:15:51.024 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:15:51.024 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:51.024 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:51.024 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:15:51.024 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:15:51.024 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 4662, 2024-05-28T14:15:51.024 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 19392299125, 2024-05-28T14:15:51.024 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 39214444544, 2024-05-28T14:15:51.024 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 733879668736, 2024-05-28T14:15:51.024 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:15:51.024 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 579427627, 2024-05-28T14:15:51.024 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 578776257, 2024-05-28T14:15:51.025 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 140, 2024-05-28T14:15:51.025 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 152 2024-05-28T14:15:51.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:51.025 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:15:51.025 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 85, 2024-05-28T14:15:51.025 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 9, 2024-05-28T14:15:51.025 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:15:51.025 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:15:51.025 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:15:51.025 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:15:51.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:15:51.025 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 9, 2024-05-28T14:15:51.026 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:15:51.026 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:15:51.026 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:15:51.026 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57851 2024-05-28T14:15:51.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:51.026 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:51.026 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:15:51.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:51.026 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:15:51.026 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:15:51.026 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:15:51.026 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:15:51.026 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:15:51.027 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:15:51.027 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:15:51.027 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:51.027 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:15:51.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:51.027 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:15:51.027 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 23, 2024-05-28T14:15:51.027 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:15:41.227805+0000", 2024-05-28T14:15:51.027 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-05-28T14:15:51.027 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-05-28T14:15:51.027 INFO:teuthology.orchestra.run.smithi042.stdout: "daemons": { 2024-05-28T14:15:51.027 INFO:teuthology.orchestra.run.smithi042.stdout: "summary": "", 2024-05-28T14:15:51.027 INFO:teuthology.orchestra.run.smithi042.stdout: "d": { 2024-05-28T14:15:51.027 INFO:teuthology.orchestra.run.smithi042.stdout: "start_epoch": 0, 2024-05-28T14:15:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: "start_stamp": "0.000000", 2024-05-28T14:15:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 0, 2024-05-28T14:15:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-28T14:15:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": {}, 2024-05-28T14:15:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: "task_status": {} 2024-05-28T14:15:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:15:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: "4fe61782-f924-464c-a4bf-e422bf3c1add": { 2024-05-28T14:15:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 29.0 seconds before exiting (26s)\n [============================] ", 2024-05-28T14:15:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:15:51.029 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:15:51.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:51.029 INFO:teuthology.orchestra.run.smithi042.stdout: "a3d48d0e-354a-43e1-9d6b-92bf56d654e0": { 2024-05-28T14:15:51.029 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 29.0 seconds before exiting (20s)\n [============================] ", 2024-05-28T14:15:51.029 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:15:51.029 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:15:51.029 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:51.029 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:51.029 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:15:51.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:51.034+0000 7f3ed9800640 1 -- 172.21.15.42:0/734885106 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3eb406a450 msgr2=0x7f3eb406c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:51.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:51.034+0000 7f3ed9800640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3eb406a450 0x7f3eb406c910 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4006fd0 tx=0x7f3ec4008040 comp rx=0 tx=0).stop 2024-05-28T14:15:51.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:51.034+0000 7f3ed9800640 1 -- 172.21.15.42:0/734885106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ed40b52d0 msgr2=0x7f3ed40b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:51.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:51.034+0000 7f3ed9800640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ed40b52d0 0x7f3ed40b6990 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000c430 tx=0x7f3ec000c8f0 comp rx=0 tx=0).stop 2024-05-28T14:15:51.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:51.034+0000 7f3ed9800640 1 -- 172.21.15.42:0/734885106 shutdown_connections 2024-05-28T14:15:51.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:51.034+0000 7f3ed9800640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f3eb406a450 0x7f3eb406c910 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:51.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:51.034+0000 7f3ed9800640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ed415e9d0 0x7f3ed41535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:51.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:51.034+0000 7f3ed9800640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ed40b52d0 0x7f3ed40b6990 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:51.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:51.034+0000 7f3ed9800640 1 --2- 172.21.15.42:0/734885106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ed409b160 0x7f3ed4155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:51.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:51.034+0000 7f3ed9800640 1 -- 172.21.15.42:0/734885106 >> 172.21.15.42:0/734885106 conn(0x7f3ed40b3f70 msgr2=0x7f3ed40ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:51.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:51.034+0000 7f3ed9800640 1 -- 172.21.15.42:0/734885106 shutdown_connections 2024-05-28T14:15:51.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:51.034+0000 7f3ed9800640 1 -- 172.21.15.42:0/734885106 wait complete. 2024-05-28T14:15:51.310 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:15:51.314 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:15:56.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:15:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.210+0000 7f8d4f000640 1 Processor -- start 2024-05-28T14:15:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.210+0000 7f8d4f000640 1 -- start start 2024-05-28T14:15:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.210+0000 7f8d4f000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8d480af530 0x7f8d4815f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.210+0000 7f8d4f000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d48067bf0 con 0x7f8d480af150 2024-05-28T14:15:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.210+0000 7f8d4f000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d48162f20 con 0x7f8d4806ca00 2024-05-28T14:15:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.210+0000 7f8d4f000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f8d48163200 con 0x7f8d480af530 2024-05-28T14:15:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.210+0000 7f8d3fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8d480af530 0x7f8d4815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.210+0000 7f8d47e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8d480af150 0x7f8d481623b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:33946/0 (socket says 172.21.15.42:33946) 2024-05-28T14:15:56.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.210+0000 7f8d47e00640 1 -- 172.21.15.42:0/4039543722 learned_addr learned my addr 172.21.15.42:0/4039543722 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:15:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.210+0000 7f8d3fe00640 1 -- 172.21.15.42:0/4039543722 >> v1:172.21.15.63:6789/0 conn(0x7f8d4806ca00 legacy=0x7f8d4806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:15:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.210+0000 7f8d3fe00640 1 -- 172.21.15.42:0/4039543722 >> v1:172.21.15.42:6789/0 conn(0x7f8d480af150 legacy=0x7f8d481623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.210+0000 7f8d3fe00640 1 -- 172.21.15.42:0/4039543722 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4815f110 con 0x7f8d480af530 2024-05-28T14:15:56.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.214+0000 7f8d3fe00640 1 --2- 172.21.15.42:0/4039543722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480af530 0x7f8d4815f910 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f8d300097c0 tx=0x7f8d3001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b715a84103e259 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:56.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.214+0000 7f8d47400640 1 -- 172.21.15.42:0/4039543722 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d300ab650 con 0x7f8d480af530 2024-05-28T14:15:56.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.214+0000 7f8d47400640 1 -- 172.21.15.42:0/4039543722 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d300b0070 con 0x7f8d480af530 2024-05-28T14:15:56.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.214+0000 7f8d47400640 1 -- 172.21.15.42:0/4039543722 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d300aba10 con 0x7f8d480af530 2024-05-28T14:15:56.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.214+0000 7f8d4f000640 1 -- 172.21.15.42:0/4039543722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480af530 msgr2=0x7f8d4815f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:56.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.214+0000 7f8d4f000640 1 --2- 172.21.15.42:0/4039543722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480af530 0x7f8d4815f910 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f8d300097c0 tx=0x7f8d3001c430 comp rx=0 tx=0).stop 2024-05-28T14:15:56.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.214+0000 7f8d4f000640 1 -- 172.21.15.42:0/4039543722 shutdown_connections 2024-05-28T14:15:56.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.214+0000 7f8d4f000640 1 --2- 172.21.15.42:0/4039543722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480af530 0x7f8d4815f910 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:56.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.214+0000 7f8d4f000640 1 -- 172.21.15.42:0/4039543722 >> 172.21.15.42:0/4039543722 conn(0x7f8d480724e0 msgr2=0x7f8d480728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:56.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.214+0000 7f8d4f000640 1 -- 172.21.15.42:0/4039543722 shutdown_connections 2024-05-28T14:15:56.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.214+0000 7f8d4f000640 1 -- 172.21.15.42:0/4039543722 wait complete. 2024-05-28T14:15:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4f000640 1 Processor -- start 2024-05-28T14:15:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4f000640 1 -- start start 2024-05-28T14:15:56.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4f000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d4806ca00 0x7f8d48155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:56.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4f000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d480af150 0x7f8d481514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:56.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4f000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480af530 0x7f8d48156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:56.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4f000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4806f270 con 0x7f8d480af530 2024-05-28T14:15:56.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4f000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4806f0f0 con 0x7f8d480af150 2024-05-28T14:15:56.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4f000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8d48162ac0 con 0x7f8d4806ca00 2024-05-28T14:15:56.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480af530 0x7f8d48156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:56.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480af530 0x7f8d48156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51010/0 (socket says 172.21.15.42:51010) 2024-05-28T14:15:56.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4d000640 1 -- 172.21.15.42:0/3254446766 learned_addr learned my addr 172.21.15.42:0/3254446766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:15:56.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d47e00640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d4806ca00 0x7f8d48155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:56.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d3fe00640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d480af150 0x7f8d481514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:56.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4d000640 1 -- 172.21.15.42:0/3254446766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d4806ca00 msgr2=0x7f8d48155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4d000640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d4806ca00 0x7f8d48155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4d000640 1 -- 172.21.15.42:0/3254446766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d480af150 msgr2=0x7f8d481514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4d000640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d480af150 0x7f8d481514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4d000640 1 -- 172.21.15.42:0/3254446766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d48151a00 con 0x7f8d480af530 2024-05-28T14:15:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d3fe00640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d480af150 0x7f8d481514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:15:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d47e00640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d4806ca00 0x7f8d48155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:15:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4d000640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480af530 0x7f8d48156470 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000b970 tx=0x7f8d4000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d44c00640 1 -- 172.21.15.42:0/3254446766 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d40019070 con 0x7f8d480af530 2024-05-28T14:15:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d44c00640 1 -- 172.21.15.42:0/3254446766 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d4000cb40 con 0x7f8d480af530 2024-05-28T14:15:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4f000640 1 -- 172.21.15.42:0/3254446766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d48150420 con 0x7f8d480af530 2024-05-28T14:15:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d44c00640 1 -- 172.21.15.42:0/3254446766 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d40014440 con 0x7f8d480af530 2024-05-28T14:15:56.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4f000640 1 -- 172.21.15.42:0/3254446766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d481509e0 con 0x7f8d480af530 2024-05-28T14:15:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.218+0000 7f8d4f000640 1 -- 172.21.15.42:0/3254446766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d48068a90 con 0x7f8d480af530 2024-05-28T14:15:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.222+0000 7f8d44c00640 1 -- 172.21.15.42:0/3254446766 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 47) ==== 90945+0+0 (secure 0 0 0) 0x7f8d4001c070 con 0x7f8d480af530 2024-05-28T14:15:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.222+0000 7f8d44c00640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d2406a450 0x7f8d2406c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:15:56.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.222+0000 7f8d44c00640 1 -- 172.21.15.42:0/3254446766 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f8d400856f0 con 0x7f8d480af530 2024-05-28T14:15:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.222+0000 7f8d47e00640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d2406a450 0x7f8d2406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:15:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.222+0000 7f8d47e00640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d2406a450 0x7f8d2406c910 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8d38000f70 tx=0x7f8d38009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:15:56.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.226+0000 7f8d44c00640 1 -- 172.21.15.42:0/3254446766 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8d40057710 con 0x7f8d480af530 2024-05-28T14:15:56.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.450+0000 7f8d4f000640 1 -- 172.21.15.42:0/3254446766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8d4815e860 con 0x7f8d480af530 2024-05-28T14:15:56.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.450+0000 7f8d44c00640 1 -- 172.21.15.42:0/3254446766 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2382 (secure 0 0 0) 0x7f8d40057010 con 0x7f8d480af530 2024-05-28T14:15:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:15:56.437 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:15:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:15:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:15:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:15:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:15:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:15:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:15:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:15:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:15:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:15:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:15:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:15:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:15:56.438 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:15:56.439 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:15:56.439 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:56.439 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 1077, 2024-05-28T14:15:56.439 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:15:56.439 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:15:56.439 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:15:56.439 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:15:56.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:56.439 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:15:56.439 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 134, 2024-05-28T14:15:56.439 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:15:56.439 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:15:56.440 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:15:56.440 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:15:56.440 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:15:56.440 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:15:56.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:56.440 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:15:56.440 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:15:56.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:15:56.440 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:15:56.440 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:15:56.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:56.440 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:56.440 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:15:56.440 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:15:56.441 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 5083, 2024-05-28T14:15:56.441 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 21175109282, 2024-05-28T14:15:56.441 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 43518431232, 2024-05-28T14:15:56.441 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 729575682048, 2024-05-28T14:15:56.441 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:15:56.441 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 444776207, 2024-05-28T14:15:56.441 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 447962854, 2024-05-28T14:15:56.441 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 108, 2024-05-28T14:15:56.441 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 122 2024-05-28T14:15:56.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:56.441 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:15:56.441 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 85, 2024-05-28T14:15:56.441 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 9, 2024-05-28T14:15:56.442 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:15:56.442 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:15:56.442 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:15:56.442 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:15:56.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:15:56.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 9, 2024-05-28T14:15:56.442 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:15:56.442 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:15:56.442 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:15:56.442 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57851 2024-05-28T14:15:56.442 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:56.442 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:56.442 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:15:56.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:56.443 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:15:56.443 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:15:56.443 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:15:56.443 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:15:56.443 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:15:56.443 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:15:56.443 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:15:56.443 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:15:56.443 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:15:56.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:56.443 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:15:56.443 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 24, 2024-05-28T14:15:56.443 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:15:51.232273+0000", 2024-05-28T14:15:56.444 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:15:56.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:56.444 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:15:56.444 INFO:teuthology.orchestra.run.smithi042.stdout: "4fe61782-f924-464c-a4bf-e422bf3c1add": { 2024-05-28T14:15:56.444 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 23.0 seconds before exiting (32s)\n [============================] ", 2024-05-28T14:15:56.444 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:15:56.444 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:15:56.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:15:56.444 INFO:teuthology.orchestra.run.smithi042.stdout: "a3d48d0e-354a-43e1-9d6b-92bf56d654e0": { 2024-05-28T14:15:56.444 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 23.0 seconds before exiting (26s)\n [============================] ", 2024-05-28T14:15:56.444 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:15:56.444 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:15:56.444 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:56.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:15:56.445 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:15:56.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.450+0000 7f8d4f000640 1 -- 172.21.15.42:0/3254446766 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d2406a450 msgr2=0x7f8d2406c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:56.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.450+0000 7f8d4f000640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d2406a450 0x7f8d2406c910 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8d38000f70 tx=0x7f8d38009040 comp rx=0 tx=0).stop 2024-05-28T14:15:56.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.450+0000 7f8d4f000640 1 -- 172.21.15.42:0/3254446766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480af530 msgr2=0x7f8d48156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:15:56.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.450+0000 7f8d4f000640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480af530 0x7f8d48156470 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000b970 tx=0x7f8d4000be30 comp rx=0 tx=0).stop 2024-05-28T14:15:56.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.450+0000 7f8d4f000640 1 -- 172.21.15.42:0/3254446766 shutdown_connections 2024-05-28T14:15:56.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.450+0000 7f8d4f000640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f8d2406a450 0x7f8d2406c910 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:56.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.450+0000 7f8d4f000640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d480af530 0x7f8d48156470 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:56.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.450+0000 7f8d4f000640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d480af150 0x7f8d481514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:56.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.450+0000 7f8d4f000640 1 --2- 172.21.15.42:0/3254446766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8d4806ca00 0x7f8d48155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:15:56.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.450+0000 7f8d4f000640 1 -- 172.21.15.42:0/3254446766 >> 172.21.15.42:0/3254446766 conn(0x7f8d480724e0 msgr2=0x7f8d480ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:15:56.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.450+0000 7f8d4f000640 1 -- 172.21.15.42:0/3254446766 shutdown_connections 2024-05-28T14:15:56.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:15:56.450+0000 7f8d4f000640 1 -- 172.21.15.42:0/3254446766 wait complete. 2024-05-28T14:16:01.460 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:16:01.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd2c400640 1 Processor -- start 2024-05-28T14:16:01.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd2c400640 1 -- start start 2024-05-28T14:16:01.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd2c400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdd240cd090 0x7fdd240cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:01.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd2c400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd240b5760 0x7fdd240b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:01.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd2c400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd240d0760 con 0x7fdd2415fd00 2024-05-28T14:16:01.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd2c400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdd240b52d0 con 0x7fdd240cd090 2024-05-28T14:16:01.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd2c400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdd240d0a30 con 0x7fdd240b5760 2024-05-28T14:16:01.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd2a400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd240b5760 0x7fdd240b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:01.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd29000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdd240cd090 0x7fdd240cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:01.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd2a400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd240b5760 0x7fdd240b5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:56222/0 (socket says 172.21.15.42:56222) 2024-05-28T14:16:01.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd29000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdd240cd090 0x7fdd240cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:32870/0 (socket says 172.21.15.42:32870) 2024-05-28T14:16:01.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd2a400640 1 -- 172.21.15.42:0/3325742261 learned_addr learned my addr 172.21.15.42:0/3325742261 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:01.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd2a400640 1 -- 172.21.15.42:0/3325742261 >> v2:172.21.15.63:3300/0 conn(0x7fdd240cd090 msgr2=0x7fdd240cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:01.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd2a400640 1 --2- 172.21.15.42:0/3325742261 >> v2:172.21.15.63:3300/0 conn(0x7fdd240cd090 0x7fdd240cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:01.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd2a400640 1 -- 172.21.15.42:0/3325742261 >> v1:172.21.15.63:6789/0 conn(0x7fdd2415fd00 legacy=0x7fdd240d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:01.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd2a400640 1 -- 172.21.15.42:0/3325742261 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd240d0d10 con 0x7fdd240b5760 2024-05-28T14:16:01.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.626+0000 7fdd29000640 1 --2- 172.21.15.42:0/3325742261 >> v2:172.21.15.63:3300/0 conn(0x7fdd240cd090 0x7fdd240cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:16:01.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.630+0000 7fdd2a400640 1 --2- 172.21.15.42:0/3325742261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd240b5760 0x7fdd240b5bf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009e50 tx=0x7fdd1c01ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae54094de5105a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.630+0000 7fdd23e00640 1 -- 172.21.15.42:0/3325742261 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0a58d0 con 0x7fdd240b5760 2024-05-28T14:16:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.630+0000 7fdd23e00640 1 -- 172.21.15.42:0/3325742261 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1c0a5a70 con 0x7fdd240b5760 2024-05-28T14:16:01.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.630+0000 7fdd23e00640 1 -- 172.21.15.42:0/3325742261 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0a5e40 con 0x7fdd240b5760 2024-05-28T14:16:01.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.630+0000 7fdd2c400640 1 -- 172.21.15.42:0/3325742261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd240b5760 msgr2=0x7fdd240b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:01.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.630+0000 7fdd2c400640 1 --2- 172.21.15.42:0/3325742261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd240b5760 0x7fdd240b5bf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009e50 tx=0x7fdd1c01ca00 comp rx=0 tx=0).stop 2024-05-28T14:16:01.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.630+0000 7fdd2c400640 1 -- 172.21.15.42:0/3325742261 shutdown_connections 2024-05-28T14:16:01.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.630+0000 7fdd2c400640 1 --2- 172.21.15.42:0/3325742261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd240b5760 0x7fdd240b5bf0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:01.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.630+0000 7fdd2c400640 1 --2- 172.21.15.42:0/3325742261 >> v2:172.21.15.63:3300/0 conn(0x7fdd240cd090 0x7fdd240cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:01.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.630+0000 7fdd2c400640 1 -- 172.21.15.42:0/3325742261 >> 172.21.15.42:0/3325742261 conn(0x7fdd240b3f70 msgr2=0x7fdd240b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:01.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.630+0000 7fdd2c400640 1 -- 172.21.15.42:0/3325742261 shutdown_connections 2024-05-28T14:16:01.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.630+0000 7fdd2c400640 1 -- 172.21.15.42:0/3325742261 wait complete. 2024-05-28T14:16:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd2c400640 1 Processor -- start 2024-05-28T14:16:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd2c400640 1 -- start start 2024-05-28T14:16:01.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd2c400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd240b5760 0x7fdd241f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd2c400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd240cd090 0x7fdd24164fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd2c400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd2415fd00 0x7fdd2415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd2c400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdd24160260 con 0x7fdd240b5760 2024-05-28T14:16:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd2c400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdd241600e0 con 0x7fdd240cd090 2024-05-28T14:16:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd2c400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdd240b34d0 con 0x7fdd2415fd00 2024-05-28T14:16:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd29a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd240b5760 0x7fdd241f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd29a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd240b5760 0x7fdd241f9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32810/0 (socket says 172.21.15.42:32810) 2024-05-28T14:16:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd29a00640 1 -- 172.21.15.42:0/1674074261 learned_addr learned my addr 172.21.15.42:0/1674074261 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd29000640 1 --2- 172.21.15.42:0/1674074261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd240cd090 0x7fdd24164fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd2a400640 1 --2- 172.21.15.42:0/1674074261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd2415fd00 0x7fdd2415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd29a00640 1 -- 172.21.15.42:0/1674074261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd2415fd00 msgr2=0x7fdd2415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd29a00640 1 --2- 172.21.15.42:0/1674074261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd2415fd00 0x7fdd2415d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd29a00640 1 -- 172.21.15.42:0/1674074261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd240cd090 msgr2=0x7fdd24164fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd29a00640 1 --2- 172.21.15.42:0/1674074261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd240cd090 0x7fdd24164fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd29a00640 1 -- 172.21.15.42:0/1674074261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd24165880 con 0x7fdd240b5760 2024-05-28T14:16:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd29a00640 1 --2- 172.21.15.42:0/1674074261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd240b5760 0x7fdd241f9e70 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800c3a0 tx=0x7fdd1800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd21600640 1 -- 172.21.15.42:0/1674074261 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd180052d0 con 0x7fdd240b5760 2024-05-28T14:16:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd21600640 1 -- 172.21.15.42:0/1674074261 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd18005470 con 0x7fdd240b5760 2024-05-28T14:16:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd21600640 1 -- 172.21.15.42:0/1674074261 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18007260 con 0x7fdd240b5760 2024-05-28T14:16:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd2c400640 1 -- 172.21.15.42:0/1674074261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd24165b60 con 0x7fdd240b5760 2024-05-28T14:16:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.634+0000 7fdd2c400640 1 -- 172.21.15.42:0/1674074261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd24166120 con 0x7fdd240b5760 2024-05-28T14:16:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.638+0000 7fdd2c400640 1 -- 172.21.15.42:0/1674074261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd24068a90 con 0x7fdd240b5760 2024-05-28T14:16:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.638+0000 7fdd21600640 1 -- 172.21.15.42:0/1674074261 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 47) ==== 90945+0+0 (secure 0 0 0) 0x7fdd18007400 con 0x7fdd240b5760 2024-05-28T14:16:01.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.638+0000 7fdd21600640 1 --2- 172.21.15.42:0/1674074261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd0406a420 0x7fdd0406c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:01.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.638+0000 7fdd21600640 1 -- 172.21.15.42:0/1674074261 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fdd18084e50 con 0x7fdd240b5760 2024-05-28T14:16:01.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.638+0000 7fdd29000640 1 --2- 172.21.15.42:0/1674074261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd0406a420 0x7fdd0406c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:01.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.638+0000 7fdd29000640 1 --2- 172.21.15.42:0/1674074261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd0406a420 0x7fdd0406c8e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009580 tx=0x7fdd10009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:01.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.642+0000 7fdd21600640 1 -- 172.21.15.42:0/1674074261 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdd180568c0 con 0x7fdd240b5760 2024-05-28T14:16:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.866+0000 7fdd2c400640 1 -- 172.21.15.42:0/1674074261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdd241fa3b0 con 0x7fdd240b5760 2024-05-28T14:16:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.866+0000 7fdd21600640 1 -- 172.21.15.42:0/1674074261 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2380 (secure 0 0 0) 0x7fdd1805abc0 con 0x7fdd240b5760 2024-05-28T14:16:01.853 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:01.853 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:16:01.854 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:16:01.854 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:16:01.854 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:16:01.854 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:16:01.854 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:16:01.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:01.854 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:16:01.854 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:16:01.854 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:16:01.854 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:16:01.854 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:16:01.854 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:01.854 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:16:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:16:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:16:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:16:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 1083, 2024-05-28T14:16:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:16:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:16:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:16:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:16:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:16:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 134, 2024-05-28T14:16:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:16:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:16:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:16:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:16:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:16:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:16:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:16:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:16:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:16:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:16:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:16:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:16:01.857 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 5083, 2024-05-28T14:16:01.857 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 21175114489, 2024-05-28T14:16:01.857 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 43518476288, 2024-05-28T14:16:01.857 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 729575636992, 2024-05-28T14:16:01.857 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:16:01.857 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 143962576, 2024-05-28T14:16:01.857 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 148526781, 2024-05-28T14:16:01.857 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 36, 2024-05-28T14:16:01.857 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 43 2024-05-28T14:16:01.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:01.858 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:16:01.858 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 85, 2024-05-28T14:16:01.858 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 9, 2024-05-28T14:16:01.858 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:16:01.858 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:16:01.858 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:16:01.858 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:16:01.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:01.858 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 9, 2024-05-28T14:16:01.858 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:16:01.858 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:16:01.858 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:16:01.858 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57851 2024-05-28T14:16:01.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:16:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:16:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:16:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:16:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:16:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:16:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:16:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:16:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:16:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:01.859 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 24, 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:15:51.232273+0000", 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: "4fe61782-f924-464c-a4bf-e422bf3c1add": { 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 17.0 seconds before exiting (39s)\n [============================] ", 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: "a3d48d0e-354a-43e1-9d6b-92bf56d654e0": { 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 17.0 seconds before exiting (32s)\n [============================] ", 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:16:01.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:01.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:01.861 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:16:01.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.866+0000 7fdd2c400640 1 -- 172.21.15.42:0/1674074261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd0406a420 msgr2=0x7fdd0406c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:01.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.866+0000 7fdd2c400640 1 --2- 172.21.15.42:0/1674074261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd0406a420 0x7fdd0406c8e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009580 tx=0x7fdd10009190 comp rx=0 tx=0).stop 2024-05-28T14:16:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.866+0000 7fdd2c400640 1 -- 172.21.15.42:0/1674074261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd240b5760 msgr2=0x7fdd241f9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.866+0000 7fdd2c400640 1 --2- 172.21.15.42:0/1674074261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd240b5760 0x7fdd241f9e70 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800c3a0 tx=0x7fdd1800c860 comp rx=0 tx=0).stop 2024-05-28T14:16:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.866+0000 7fdd2c400640 1 -- 172.21.15.42:0/1674074261 shutdown_connections 2024-05-28T14:16:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.866+0000 7fdd2c400640 1 --2- 172.21.15.42:0/1674074261 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fdd0406a420 0x7fdd0406c8e0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.866+0000 7fdd2c400640 1 --2- 172.21.15.42:0/1674074261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd2415fd00 0x7fdd2415d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.866+0000 7fdd2c400640 1 --2- 172.21.15.42:0/1674074261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd240cd090 0x7fdd24164fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.866+0000 7fdd2c400640 1 --2- 172.21.15.42:0/1674074261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd240b5760 0x7fdd241f9e70 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.866+0000 7fdd2c400640 1 -- 172.21.15.42:0/1674074261 >> 172.21.15.42:0/1674074261 conn(0x7fdd240b3f70 msgr2=0x7fdd2406d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.866+0000 7fdd2c400640 1 -- 172.21.15.42:0/1674074261 shutdown_connections 2024-05-28T14:16:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:01.866+0000 7fdd2c400640 1 -- 172.21.15.42:0/1674074261 wait complete. 2024-05-28T14:16:06.876 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:16:07.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f95800640 1 Processor -- start 2024-05-28T14:16:07.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f95800640 1 -- start start 2024-05-28T14:16:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f95800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f9015e9d0 0x7f7f900d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f95800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f900ccce0 con 0x7f7f9009b160 2024-05-28T14:16:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f95800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f900ccfc0 con 0x7f7f900b52d0 2024-05-28T14:16:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f95800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7f900cd2a0 con 0x7f7f9015e9d0 2024-05-28T14:16:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8f400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f7f9009b160 0x7f7f900b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:45946/0 (socket says 172.21.15.42:45946) 2024-05-28T14:16:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8f400640 1 -- 172.21.15.42:0/397623692 learned_addr learned my addr 172.21.15.42:0/397623692 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:07.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8ea00640 1 --2- 172.21.15.42:0/397623692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f9015e9d0 0x7f7f900d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:07.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 -- 172.21.15.42:0/397623692 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2340916607 0 0) 0x7f7f900ccce0 con 0x7f7f9009b160 2024-05-28T14:16:07.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 -- 172.21.15.42:0/397623692 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f74003530 con 0x7f7f9009b160 2024-05-28T14:16:07.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 -- 172.21.15.42:0/397623692 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1329086580 0 0) 0x7f7f74003530 con 0x7f7f9009b160 2024-05-28T14:16:07.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 -- 172.21.15.42:0/397623692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f9015e9d0 msgr2=0x7f7f900d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:07.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 --2- 172.21.15.42:0/397623692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f9015e9d0 0x7f7f900d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:07.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 -- 172.21.15.42:0/397623692 >> v1:172.21.15.63:6789/0 conn(0x7f7f900b52d0 legacy=0x7f7f9009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 -- 172.21.15.42:0/397623692 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f900cd580 con 0x7f7f9009b160 2024-05-28T14:16:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 -- 172.21.15.42:0/397623692 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f7f88001a00 con 0x7f7f9009b160 2024-05-28T14:16:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 -- 172.21.15.42:0/397623692 >> v1:172.21.15.42:6789/0 conn(0x7f7f9009b160 legacy=0x7f7f900b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 --2- 172.21.15.42:0/397623692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f9015e9d0 0x7f7f740039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 --2- 172.21.15.42:0/397623692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f74003f70 0x7f7f74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 --2- 172.21.15.42:0/397623692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f74004d50 0x7f7f740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 -- 172.21.15.42:0/397623692 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7f900cd2a0 con 0x7f7f74004d50 2024-05-28T14:16:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 -- 172.21.15.42:0/397623692 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7400bab0 con 0x7f7f74003f70 2024-05-28T14:16:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8e000640 1 -- 172.21.15.42:0/397623692 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7400bcb0 con 0x7f7f9015e9d0 2024-05-28T14:16:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8f400640 1 --2- 172.21.15.42:0/397623692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f74004d50 0x7f7f740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f8ea00640 1 --2- 172.21.15.42:0/397623692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f9015e9d0 0x7f7f740039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.042+0000 7f7f86000640 1 --2- 172.21.15.42:0/397623692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f74003f70 0x7f7f74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f8ea00640 1 -- 172.21.15.42:0/397623692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f74003f70 msgr2=0x7f7f74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f8ea00640 1 --2- 172.21.15.42:0/397623692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f74003f70 0x7f7f74004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:07.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f8ea00640 1 -- 172.21.15.42:0/397623692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f74004d50 msgr2=0x7f7f740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:07.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f8ea00640 1 --2- 172.21.15.42:0/397623692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f74004d50 0x7f7f740051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:07.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f8ea00640 1 -- 172.21.15.42:0/397623692 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f74003530 con 0x7f7f9015e9d0 2024-05-28T14:16:07.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f8f400640 1 --2- 172.21.15.42:0/397623692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f74004d50 0x7f7f740051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:16:07.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f8ea00640 1 --2- 172.21.15.42:0/397623692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f9015e9d0 0x7f7f740039f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009ae0 tx=0x7f7f8001cab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc2fa1fe45acdd4a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:07.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f8e000640 1 -- 172.21.15.42:0/397623692 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f800b2070 con 0x7f7f9015e9d0 2024-05-28T14:16:07.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f8e000640 1 -- 172.21.15.42:0/397623692 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f800a56a0 con 0x7f7f9015e9d0 2024-05-28T14:16:07.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f8e000640 1 -- 172.21.15.42:0/397623692 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f800096b0 con 0x7f7f9015e9d0 2024-05-28T14:16:07.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f95800640 1 -- 172.21.15.42:0/397623692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f9015e9d0 msgr2=0x7f7f740039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:07.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f95800640 1 --2- 172.21.15.42:0/397623692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f9015e9d0 0x7f7f740039f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009ae0 tx=0x7f7f8001cab0 comp rx=0 tx=0).stop 2024-05-28T14:16:07.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f8f400640 1 -- 172.21.15.42:0/397623692 reap_dead start 2024-05-28T14:16:07.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f95800640 1 -- 172.21.15.42:0/397623692 shutdown_connections 2024-05-28T14:16:07.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f95800640 1 -- 172.21.15.42:0/397623692 >> 172.21.15.42:0/397623692 conn(0x7f7f900b3f70 msgr2=0x7f7f900b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:07.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.046+0000 7f7f95800640 1 -- 172.21.15.42:0/397623692 shutdown_connections 2024-05-28T14:16:07.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f95800640 1 -- 172.21.15.42:0/397623692 wait complete. 2024-05-28T14:16:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f95800640 1 Processor -- start 2024-05-28T14:16:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f95800640 1 -- start start 2024-05-28T14:16:07.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f95800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f9015e9d0 0x7f7f900ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:07.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f95800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f900c6670 0x7f7f900cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:07.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f95800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f900cb3f0 0x7f7f900cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:07.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f95800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7f900b37a0 con 0x7f7f9015e9d0 2024-05-28T14:16:07.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f95800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9015edb0 con 0x7f7f900cb3f0 2024-05-28T14:16:07.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f95800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7f900b3920 con 0x7f7f900c6670 2024-05-28T14:16:07.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f8ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f9015e9d0 0x7f7f900ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:07.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f8ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f9015e9d0 0x7f7f900ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32826/0 (socket says 172.21.15.42:32826) 2024-05-28T14:16:07.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f8ea00640 1 -- 172.21.15.42:0/226754427 learned_addr learned my addr 172.21.15.42:0/226754427 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:07.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f8f400640 1 --2- 172.21.15.42:0/226754427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f900cb3f0 0x7f7f900cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:07.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f86000640 1 --2- 172.21.15.42:0/226754427 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f900c6670 0x7f7f900cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f8ea00640 1 -- 172.21.15.42:0/226754427 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f900c6670 msgr2=0x7f7f900cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f8ea00640 1 --2- 172.21.15.42:0/226754427 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f900c6670 0x7f7f900cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f8ea00640 1 -- 172.21.15.42:0/226754427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f900cb3f0 msgr2=0x7f7f900cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f8ea00640 1 --2- 172.21.15.42:0/226754427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f900cb3f0 0x7f7f900cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f8ea00640 1 -- 172.21.15.42:0/226754427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f9015ef30 con 0x7f7f9015e9d0 2024-05-28T14:16:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.050+0000 7f7f8ea00640 1 --2- 172.21.15.42:0/226754427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f9015e9d0 0x7f7f900ca710 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009ae0 tx=0x7f7f74005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.054+0000 7f7f87400640 1 -- 172.21.15.42:0/226754427 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f800b2040 con 0x7f7f9015e9d0 2024-05-28T14:16:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.054+0000 7f7f87400640 1 -- 172.21.15.42:0/226754427 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f80004230 con 0x7f7f9015e9d0 2024-05-28T14:16:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.054+0000 7f7f95800640 1 -- 172.21.15.42:0/226754427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f9015f1d0 con 0x7f7f9015e9d0 2024-05-28T14:16:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.054+0000 7f7f87400640 1 -- 172.21.15.42:0/226754427 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80003620 con 0x7f7f9015e9d0 2024-05-28T14:16:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.054+0000 7f7f95800640 1 -- 172.21.15.42:0/226754427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f9015f790 con 0x7f7f9015e9d0 2024-05-28T14:16:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.054+0000 7f7f95800640 1 -- 172.21.15.42:0/226754427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f90068a90 con 0x7f7f9015e9d0 2024-05-28T14:16:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.054+0000 7f7f87400640 1 -- 172.21.15.42:0/226754427 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 90894+0+0 (secure 0 0 0) 0x7f7f800037c0 con 0x7f7f9015e9d0 2024-05-28T14:16:07.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.054+0000 7f7f87400640 1 --2- 172.21.15.42:0/226754427 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7f6c06a390 0x7f7f6c06c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:07.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.054+0000 7f7f87400640 1 -- 172.21.15.42:0/226754427 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f7f801202f0 con 0x7f7f9015e9d0 2024-05-28T14:16:07.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.054+0000 7f7f86000640 1 --2- 172.21.15.42:0/226754427 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7f6c06a390 0x7f7f6c06c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:07.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.058+0000 7f7f86000640 1 --2- 172.21.15.42:0/226754427 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7f6c06a390 0x7f7f6c06c850 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7f900d0c10 tx=0x7f7f78009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.058+0000 7f7f87400640 1 -- 172.21.15.42:0/226754427 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7f800167c0 con 0x7f7f9015e9d0 2024-05-28T14:16:07.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.290+0000 7f7f95800640 1 -- 172.21.15.42:0/226754427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7f90151dc0 con 0x7f7f9015e9d0 2024-05-28T14:16:07.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.290+0000 7f7f87400640 1 -- 172.21.15.42:0/226754427 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2367 (secure 0 0 0) 0x7f7f800f1cc0 con 0x7f7f9015e9d0 2024-05-28T14:16:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:07.276 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:16:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:16:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:16:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:16:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:16:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:16:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:16:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:16:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:16:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:16:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:16:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:16:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:16:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:16:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:16:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 1088, 2024-05-28T14:16:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:16:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:16:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:16:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:16:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:16:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 134, 2024-05-28T14:16:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:16:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:16:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:16:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:16:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:16:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:16:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:16:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:16:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:16:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:16:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:16:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:16:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 5083, 2024-05-28T14:16:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 21175116245, 2024-05-28T14:16:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 43518488576, 2024-05-28T14:16:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 729575624704, 2024-05-28T14:16:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:16:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "read_bytes_sec": 170, 2024-05-28T14:16:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 5288, 2024-05-28T14:16:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:16:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 2 2024-05-28T14:16:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:16:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 85, 2024-05-28T14:16:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 9, 2024-05-28T14:16:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:16:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:16:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:16:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:16:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 9, 2024-05-28T14:16:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:16:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:16:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:16:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57851 2024-05-28T14:16:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:16:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:16:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:16:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:16:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:16:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:16:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:16:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:16:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:16:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:16:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 24, 2024-05-28T14:16:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:15:51.232273+0000", 2024-05-28T14:16:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:16:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:16:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "4fe61782-f924-464c-a4bf-e422bf3c1add": { 2024-05-28T14:16:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 13.0 seconds before exiting (43s)\n [============================] ", 2024-05-28T14:16:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:16:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:16:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "a3d48d0e-354a-43e1-9d6b-92bf56d654e0": { 2024-05-28T14:16:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 13.0 seconds before exiting (36s)\n [============================] ", 2024-05-28T14:16:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:16:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:16:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:07.285 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:16:07.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.290+0000 7f7f95800640 1 -- 172.21.15.42:0/226754427 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7f6c06a390 msgr2=0x7f7f6c06c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:07.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.290+0000 7f7f95800640 1 --2- 172.21.15.42:0/226754427 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7f6c06a390 0x7f7f6c06c850 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7f900d0c10 tx=0x7f7f78009040 comp rx=0 tx=0).stop 2024-05-28T14:16:07.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.290+0000 7f7f95800640 1 -- 172.21.15.42:0/226754427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f9015e9d0 msgr2=0x7f7f900ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:07.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.290+0000 7f7f95800640 1 --2- 172.21.15.42:0/226754427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f9015e9d0 0x7f7f900ca710 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009ae0 tx=0x7f7f74005840 comp rx=0 tx=0).stop 2024-05-28T14:16:07.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.290+0000 7f7f95800640 1 -- 172.21.15.42:0/226754427 shutdown_connections 2024-05-28T14:16:07.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.290+0000 7f7f95800640 1 --2- 172.21.15.42:0/226754427 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7f6c06a390 0x7f7f6c06c850 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:07.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.290+0000 7f7f95800640 1 --2- 172.21.15.42:0/226754427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f900cb3f0 0x7f7f900cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:07.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.290+0000 7f7f95800640 1 --2- 172.21.15.42:0/226754427 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f900c6670 0x7f7f900cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:07.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.290+0000 7f7f95800640 1 --2- 172.21.15.42:0/226754427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f9015e9d0 0x7f7f900ca710 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:07.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.290+0000 7f7f95800640 1 -- 172.21.15.42:0/226754427 >> 172.21.15.42:0/226754427 conn(0x7f7f900b3f70 msgr2=0x7f7f90071520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:07.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.290+0000 7f7f95800640 1 -- 172.21.15.42:0/226754427 shutdown_connections 2024-05-28T14:16:07.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:07.290+0000 7f7f95800640 1 -- 172.21.15.42:0/226754427 wait complete. 2024-05-28T14:16:12.299 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:16:12.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7fdc00640 1 Processor -- start 2024-05-28T14:16:12.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7fdc00640 1 -- start start 2024-05-28T14:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7fdc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc7f8163a10 0x7fc7f8163120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7fdc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7f8163df0 0x7fc7f806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7fdc00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7f8072aa0 con 0x7fc7f8071ce0 2024-05-28T14:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7fdc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc7f806e240 con 0x7fc7f8163a10 2024-05-28T14:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7fdc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f806e470 con 0x7fc7f8163df0 2024-05-28T14:16:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7f6a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc7f8163a10 0x7fc7f8163120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7f7400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc7f8071ce0 0x7fc7f8166710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:59832/0 (socket says 172.21.15.42:59832) 2024-05-28T14:16:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7f7400640 1 -- 172.21.15.42:0/599502273 learned_addr learned my addr 172.21.15.42:0/599502273 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7efe00640 1 --2- 172.21.15.42:0/599502273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7f8163df0 0x7fc7f806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7f6a00640 1 -- 172.21.15.42:0/599502273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7f8163df0 msgr2=0x7fc7f806dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7f6a00640 1 --2- 172.21.15.42:0/599502273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7f8163df0 0x7fc7f806dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7f6a00640 1 -- 172.21.15.42:0/599502273 >> v1:172.21.15.42:6789/0 conn(0x7fc7f8071ce0 legacy=0x7fc7f8166710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7f6000640 1 -- 172.21.15.42:0/599502273 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1890270779 0 0) 0x7fc7f8072aa0 con 0x7fc7f8071ce0 2024-05-28T14:16:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.466+0000 7fc7f6a00640 1 -- 172.21.15.42:0/599502273 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f806e750 con 0x7fc7f8163a10 2024-05-28T14:16:12.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.470+0000 7fc7f6a00640 1 --2- 172.21.15.42:0/599502273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7f8163a10 0x7fc7f8163120 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80097c0 tx=0x7fc7e801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f65d6e5b06938f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:12.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.470+0000 7fc7f6000640 1 -- 172.21.15.42:0/599502273 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80b2070 con 0x7fc7f8163a10 2024-05-28T14:16:12.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.470+0000 7fc7f6000640 1 -- 172.21.15.42:0/599502273 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e80ad640 con 0x7fc7f8163a10 2024-05-28T14:16:12.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.470+0000 7fc7f6000640 1 -- 172.21.15.42:0/599502273 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80ada10 con 0x7fc7f8163a10 2024-05-28T14:16:12.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.470+0000 7fc7fdc00640 1 -- 172.21.15.42:0/599502273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7f8163a10 msgr2=0x7fc7f8163120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:12.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.470+0000 7fc7fdc00640 1 --2- 172.21.15.42:0/599502273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7f8163a10 0x7fc7f8163120 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80097c0 tx=0x7fc7e801c4a0 comp rx=0 tx=0).stop 2024-05-28T14:16:12.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.470+0000 7fc7fdc00640 1 -- 172.21.15.42:0/599502273 shutdown_connections 2024-05-28T14:16:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.470+0000 7fc7fdc00640 1 --2- 172.21.15.42:0/599502273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7f8163df0 0x7fc7f806dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.470+0000 7fc7fdc00640 1 --2- 172.21.15.42:0/599502273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7f8163a10 0x7fc7f8163120 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.470+0000 7fc7fdc00640 1 -- 172.21.15.42:0/599502273 >> 172.21.15.42:0/599502273 conn(0x7fc7f806bff0 msgr2=0x7fc7f80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.470+0000 7fc7fdc00640 1 -- 172.21.15.42:0/599502273 shutdown_connections 2024-05-28T14:16:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.470+0000 7fc7fdc00640 1 -- 172.21.15.42:0/599502273 wait complete. 2024-05-28T14:16:12.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7fdc00640 1 Processor -- start 2024-05-28T14:16:12.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7fdc00640 1 -- start start 2024-05-28T14:16:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7fdc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7f8071ce0 0x7fc7f8159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7fdc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7f8163a10 0x7fc7f8152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7fdc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7f8163df0 0x7fc7f81527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7fdc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f806f510 con 0x7fc7f8163df0 2024-05-28T14:16:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7fdc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f8072560 con 0x7fc7f8071ce0 2024-05-28T14:16:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7fdc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f8165bd0 con 0x7fc7f8163a10 2024-05-28T14:16:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7f7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7f8163df0 0x7fc7f81527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7f7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7f8163df0 0x7fc7f81527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41462/0 (socket says 172.21.15.42:41462) 2024-05-28T14:16:12.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7f7400640 1 -- 172.21.15.42:0/3416145157 learned_addr learned my addr 172.21.15.42:0/3416145157 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7efe00640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7f8163a10 0x7fc7f8152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7f6a00640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7f8071ce0 0x7fc7f8159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7f7400640 1 -- 172.21.15.42:0/3416145157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7f8163a10 msgr2=0x7fc7f8152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7f7400640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7f8163a10 0x7fc7f8152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7f7400640 1 -- 172.21.15.42:0/3416145157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7f8071ce0 msgr2=0x7fc7f8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7f7400640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7f8071ce0 0x7fc7f8159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7f7400640 1 -- 172.21.15.42:0/3416145157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f81586c0 con 0x7fc7f8163df0 2024-05-28T14:16:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7f6a00640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7f8071ce0 0x7fc7f8159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7efe00640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7f8163a10 0x7fc7f8152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:12.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7f7400640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7f8163df0 0x7fc7f81527c0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000c700 tx=0x7fc7f000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:12.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7eea00640 1 -- 172.21.15.42:0/3416145157 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0018070 con 0x7fc7f8163df0 2024-05-28T14:16:12.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7eea00640 1 -- 172.21.15.42:0/3416145157 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f000ce10 con 0x7fc7f8163df0 2024-05-28T14:16:12.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7eea00640 1 -- 172.21.15.42:0/3416145157 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f000a860 con 0x7fc7f8163df0 2024-05-28T14:16:12.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7fdc00640 1 -- 172.21.15.42:0/3416145157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f8157bf0 con 0x7fc7f8163df0 2024-05-28T14:16:12.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.474+0000 7fc7fdc00640 1 -- 172.21.15.42:0/3416145157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f8165860 con 0x7fc7f8163df0 2024-05-28T14:16:12.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.478+0000 7fc7fdc00640 1 -- 172.21.15.42:0/3416145157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f8068a90 con 0x7fc7f8163df0 2024-05-28T14:16:12.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.478+0000 7fc7eea00640 1 -- 172.21.15.42:0/3416145157 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 90894+0+0 (secure 0 0 0) 0x7fc7f001b070 con 0x7fc7f8163df0 2024-05-28T14:16:12.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.478+0000 7fc7eea00640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7d406a390 0x7fc7d406c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:12.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.478+0000 7fc7eea00640 1 -- 172.21.15.42:0/3416145157 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fc7f0084fe0 con 0x7fc7f8163df0 2024-05-28T14:16:12.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.478+0000 7fc7f6a00640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7d406a390 0x7fc7d406c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:12.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.478+0000 7fc7f6a00640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7d406a390 0x7fc7d406c850 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80af000 tx=0x7fc7e8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.482+0000 7fc7eea00640 1 -- 172.21.15.42:0/3416145157 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc7f81586c0 con 0x7fc7f8163df0 2024-05-28T14:16:12.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.710+0000 7fc7fdc00640 1 -- 172.21.15.42:0/3416145157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc7f80b2930 con 0x7fc7f8163df0 2024-05-28T14:16:12.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.710+0000 7fc7eea00640 1 -- 172.21.15.42:0/3416145157 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2333 (secure 0 0 0) 0x7fc7f0057000 con 0x7fc7f8163df0 2024-05-28T14:16:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:12.697 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:16:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:16:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:16:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:16:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:16:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:16:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:16:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:16:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:16:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:16:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:16:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:16:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:16:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:16:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:16:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 1094, 2024-05-28T14:16:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:16:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:16:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:16:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:16:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:16:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 134, 2024-05-28T14:16:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:16:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:16:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:16:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:16:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:16:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:16:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:16:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:16:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:16:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:16:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:16:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:16:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 5083, 2024-05-28T14:16:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 21175116245, 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 43518492672, 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 729575620608, 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280, 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "write_bytes_sec": 853, 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "read_op_per_sec": 0, 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "write_op_per_sec": 0 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 85, 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 9, 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:16:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 9, 2024-05-28T14:16:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:16:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:16:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:16:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57851 2024-05-28T14:16:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:16:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:16:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:16:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:16:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:16:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:16:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:16:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:16:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:16:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:16:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 24, 2024-05-28T14:16:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:15:51.232273+0000", 2024-05-28T14:16:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:16:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:16:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "4fe61782-f924-464c-a4bf-e422bf3c1add": { 2024-05-28T14:16:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 7.0 seconds before exiting (49s)\n [============================] ", 2024-05-28T14:16:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:16:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:16:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "a3d48d0e-354a-43e1-9d6b-92bf56d654e0": { 2024-05-28T14:16:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 7.0 seconds before exiting (42s)\n [============================] ", 2024-05-28T14:16:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:16:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:16:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:12.704 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:16:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.710+0000 7fc7fdc00640 1 -- 172.21.15.42:0/3416145157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7d406a390 msgr2=0x7fc7d406c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.710+0000 7fc7fdc00640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7d406a390 0x7fc7d406c850 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80af000 tx=0x7fc7e8008040 comp rx=0 tx=0).stop 2024-05-28T14:16:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.710+0000 7fc7fdc00640 1 -- 172.21.15.42:0/3416145157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7f8163df0 msgr2=0x7fc7f81527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.710+0000 7fc7fdc00640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7f8163df0 0x7fc7f81527c0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000c700 tx=0x7fc7f000cbc0 comp rx=0 tx=0).stop 2024-05-28T14:16:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.710+0000 7fc7fdc00640 1 -- 172.21.15.42:0/3416145157 shutdown_connections 2024-05-28T14:16:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.710+0000 7fc7fdc00640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc7d406a390 0x7fc7d406c850 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.710+0000 7fc7fdc00640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7f8163df0 0x7fc7f81527c0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.710+0000 7fc7fdc00640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7f8163a10 0x7fc7f8152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.710+0000 7fc7fdc00640 1 --2- 172.21.15.42:0/3416145157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7f8071ce0 0x7fc7f8159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.710+0000 7fc7fdc00640 1 -- 172.21.15.42:0/3416145157 >> 172.21.15.42:0/3416145157 conn(0x7fc7f806bff0 msgr2=0x7fc7f80b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.710+0000 7fc7fdc00640 1 -- 172.21.15.42:0/3416145157 shutdown_connections 2024-05-28T14:16:12.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:12.710+0000 7fc7fdc00640 1 -- 172.21.15.42:0/3416145157 wait complete. 2024-05-28T14:16:17.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:16:17.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.886+0000 7fa634400640 1 Processor -- start 2024-05-28T14:16:17.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.886+0000 7fa634400640 1 -- start start 2024-05-28T14:16:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.886+0000 7fa634400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62c15d8d0 0x7fa62c0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.886+0000 7fa634400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa62c0af500 0x7fa62c15dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.886+0000 7fa634400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa62c06f6a0 con 0x7fa62c156c00 2024-05-28T14:16:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.886+0000 7fa634400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa62c06e600 con 0x7fa62c0af500 2024-05-28T14:16:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.886+0000 7fa634400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa62c06e8e0 con 0x7fa62c15d8d0 2024-05-28T14:16:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.886+0000 7fa631a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa62c156c00 0x7fa62c06dd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50676/0 (socket says 172.21.15.42:50676) 2024-05-28T14:16:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.886+0000 7fa631a00640 1 -- 172.21.15.42:0/1091906445 learned_addr learned my addr 172.21.15.42:0/1091906445 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.886+0000 7fa632400640 1 --2- 172.21.15.42:0/1091906445 >> v2:172.21.15.42:3300/0 conn(0x7fa62c0af500 0x7fa62c15dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.886+0000 7fa631000640 1 --2- 172.21.15.42:0/1091906445 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62c15d8d0 0x7fa62c0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.886+0000 7fa632400640 1 -- 172.21.15.42:0/1091906445 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62c15d8d0 msgr2=0x7fa62c0aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.886+0000 7fa632400640 1 --2- 172.21.15.42:0/1091906445 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62c15d8d0 0x7fa62c0aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.886+0000 7fa62be00640 1 -- 172.21.15.42:0/1091906445 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2067076438 0 0) 0x7fa62c06f6a0 con 0x7fa62c156c00 2024-05-28T14:16:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa632400640 1 -- 172.21.15.42:0/1091906445 >> v1:172.21.15.42:6789/0 conn(0x7fa62c156c00 legacy=0x7fa62c06dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa632400640 1 -- 172.21.15.42:0/1091906445 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c06ebc0 con 0x7fa62c0af500 2024-05-28T14:16:17.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa631000640 1 --2- 172.21.15.42:0/1091906445 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62c15d8d0 0x7fa62c0aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:17.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa632400640 1 --2- 172.21.15.42:0/1091906445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa62c0af500 0x7fa62c15dd70 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fa624009d90 tx=0x7fa62401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c001c3fa67e4f78 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:17.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa62be00640 1 -- 172.21.15.42:0/1091906445 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6240b2070 con 0x7fa62c0af500 2024-05-28T14:16:17.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa62be00640 1 -- 172.21.15.42:0/1091906445 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6240a57a0 con 0x7fa62c0af500 2024-05-28T14:16:17.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa62be00640 1 -- 172.21.15.42:0/1091906445 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6240a5b70 con 0x7fa62c0af500 2024-05-28T14:16:17.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa634400640 1 -- 172.21.15.42:0/1091906445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa62c0af500 msgr2=0x7fa62c15dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:17.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa634400640 1 --2- 172.21.15.42:0/1091906445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa62c0af500 0x7fa62c15dd70 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fa624009d90 tx=0x7fa62401c8e0 comp rx=0 tx=0).stop 2024-05-28T14:16:17.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa634400640 1 -- 172.21.15.42:0/1091906445 shutdown_connections 2024-05-28T14:16:17.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa634400640 1 --2- 172.21.15.42:0/1091906445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa62c0af500 0x7fa62c15dd70 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:17.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa634400640 1 --2- 172.21.15.42:0/1091906445 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62c15d8d0 0x7fa62c0aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:17.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa634400640 1 -- 172.21.15.42:0/1091906445 >> 172.21.15.42:0/1091906445 conn(0x7fa62c06bff0 msgr2=0x7fa62c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:17.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa634400640 1 -- 172.21.15.42:0/1091906445 shutdown_connections 2024-05-28T14:16:17.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.890+0000 7fa634400640 1 -- 172.21.15.42:0/1091906445 wait complete. 2024-05-28T14:16:17.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa634400640 1 Processor -- start 2024-05-28T14:16:17.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa634400640 1 -- start start 2024-05-28T14:16:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa634400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62c0af500 0x7fa62c14e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa634400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa62c156c00 0x7fa62c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa634400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa62c15d8d0 0x7fa62c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa634400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c15ec40 con 0x7fa62c15d8d0 2024-05-28T14:16:17.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa634400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c15eac0 con 0x7fa62c156c00 2024-05-28T14:16:17.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa634400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa62c15edc0 con 0x7fa62c0af500 2024-05-28T14:16:17.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa632400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa62c15d8d0 0x7fa62c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:17.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa632400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa62c15d8d0 0x7fa62c15b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34796/0 (socket says 172.21.15.42:34796) 2024-05-28T14:16:17.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa632400640 1 -- 172.21.15.42:0/3537821246 learned_addr learned my addr 172.21.15.42:0/3537821246 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:17.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa631a00640 1 --2- 172.21.15.42:0/3537821246 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62c0af500 0x7fa62c14e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:17.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa631000640 1 --2- 172.21.15.42:0/3537821246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa62c156c00 0x7fa62c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:17.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa632400640 1 -- 172.21.15.42:0/3537821246 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62c0af500 msgr2=0x7fa62c14e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:17.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa632400640 1 --2- 172.21.15.42:0/3537821246 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62c0af500 0x7fa62c14e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:17.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa632400640 1 -- 172.21.15.42:0/3537821246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa62c156c00 msgr2=0x7fa62c0b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:17.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa632400640 1 --2- 172.21.15.42:0/3537821246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa62c156c00 0x7fa62c0b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:17.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa632400640 1 -- 172.21.15.42:0/3537821246 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c0b5310 con 0x7fa62c15d8d0 2024-05-28T14:16:17.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa632400640 1 --2- 172.21.15.42:0/3537821246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa62c15d8d0 0x7fa62c15b860 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fa6240b0040 tx=0x7fa624019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:17.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.894+0000 7fa629600640 1 -- 172.21.15.42:0/3537821246 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6240b2040 con 0x7fa62c15d8d0 2024-05-28T14:16:17.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.898+0000 7fa629600640 1 -- 172.21.15.42:0/3537821246 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6240bf070 con 0x7fa62c15d8d0 2024-05-28T14:16:17.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.898+0000 7fa629600640 1 -- 172.21.15.42:0/3537821246 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6240b8690 con 0x7fa62c15d8d0 2024-05-28T14:16:17.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.898+0000 7fa634400640 1 -- 172.21.15.42:0/3537821246 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62c14d530 con 0x7fa62c15d8d0 2024-05-28T14:16:17.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.898+0000 7fa634400640 1 -- 172.21.15.42:0/3537821246 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62c14daf0 con 0x7fa62c15d8d0 2024-05-28T14:16:17.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.898+0000 7fa634400640 1 -- 172.21.15.42:0/3537821246 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62c068a90 con 0x7fa62c15d8d0 2024-05-28T14:16:17.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.898+0000 7fa629600640 1 -- 172.21.15.42:0/3537821246 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 90894+0+0 (secure 0 0 0) 0x7fa6240b8830 con 0x7fa62c15d8d0 2024-05-28T14:16:17.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.898+0000 7fa629600640 1 --2- 172.21.15.42:0/3537821246 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa60c06a390 0x7fa60c06c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:17.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.898+0000 7fa629600640 1 -- 172.21.15.42:0/3537821246 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fa6241209b0 con 0x7fa62c15d8d0 2024-05-28T14:16:17.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.898+0000 7fa631a00640 1 --2- 172.21.15.42:0/3537821246 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa60c06a390 0x7fa60c06c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:17.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.898+0000 7fa631a00640 1 --2- 172.21.15.42:0/3537821246 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa60c06a390 0x7fa60c06c850 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa620000f70 tx=0x7fa620007df0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:17.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:17.906+0000 7fa629600640 1 -- 172.21.15.42:0/3537821246 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa62c0b5310 con 0x7fa62c15d8d0 2024-05-28T14:16:18.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:18.130+0000 7fa634400640 1 -- 172.21.15.42:0/3537821246 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa62c0cfce0 con 0x7fa62c15d8d0 2024-05-28T14:16:18.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:18.130+0000 7fa629600640 1 -- 172.21.15.42:0/3537821246 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2240 (secure 0 0 0) 0x7fa624016560 con 0x7fa62c15d8d0 2024-05-28T14:16:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:18.117 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:16:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:16:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:16:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:16:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:16:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:16:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:16:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:16:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:16:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:16:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:16:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:16:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:16:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:16:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:16:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 1099, 2024-05-28T14:16:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:16:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:16:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:16:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:16:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:16:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 134, 2024-05-28T14:16:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:16:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:16:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:16:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:16:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:16:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:16:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:16:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:16:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:16:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:16:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:16:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:16:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 5083, 2024-05-28T14:16:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 21175116245, 2024-05-28T14:16:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 43518492672, 2024-05-28T14:16:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 729575620608, 2024-05-28T14:16:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280 2024-05-28T14:16:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:16:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 85, 2024-05-28T14:16:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 9, 2024-05-28T14:16:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:16:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:16:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:16:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:16:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 9, 2024-05-28T14:16:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:16:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:16:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:16:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57851 2024-05-28T14:16:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:16:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:16:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:16:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:16:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:16:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:16:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:16:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:16:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:16:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:16:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 24, 2024-05-28T14:16:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:15:51.232273+0000", 2024-05-28T14:16:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:16:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": { 2024-05-28T14:16:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "4fe61782-f924-464c-a4bf-e422bf3c1add": { 2024-05-28T14:16:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "All cloning finished, waiting 1.0 seconds before exiting (55s)\n [============================] ", 2024-05-28T14:16:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:16:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:16:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: "a3d48d0e-354a-43e1-9d6b-92bf56d654e0": { 2024-05-28T14:16:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "No pending clones left, waiting 1.0 seconds before exiting (48s)\n [============================] ", 2024-05-28T14:16:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": 1, 2024-05-28T14:16:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: "add_to_ceph_s": true 2024-05-28T14:16:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:18.124 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:16:18.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:18.130+0000 7fa634400640 1 -- 172.21.15.42:0/3537821246 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa60c06a390 msgr2=0x7fa60c06c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:18.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:18.130+0000 7fa634400640 1 --2- 172.21.15.42:0/3537821246 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa60c06a390 0x7fa60c06c850 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa620000f70 tx=0x7fa620007df0 comp rx=0 tx=0).stop 2024-05-28T14:16:18.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:18.130+0000 7fa634400640 1 -- 172.21.15.42:0/3537821246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa62c15d8d0 msgr2=0x7fa62c15b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:18.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:18.130+0000 7fa634400640 1 --2- 172.21.15.42:0/3537821246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa62c15d8d0 0x7fa62c15b860 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fa6240b0040 tx=0x7fa624019680 comp rx=0 tx=0).stop 2024-05-28T14:16:18.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:18.130+0000 7fa634400640 1 -- 172.21.15.42:0/3537821246 shutdown_connections 2024-05-28T14:16:18.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:18.130+0000 7fa634400640 1 --2- 172.21.15.42:0/3537821246 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa60c06a390 0x7fa60c06c850 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:18.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:18.130+0000 7fa634400640 1 --2- 172.21.15.42:0/3537821246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa62c15d8d0 0x7fa62c15b860 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:18.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:18.130+0000 7fa634400640 1 --2- 172.21.15.42:0/3537821246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa62c156c00 0x7fa62c0b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:18.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:18.130+0000 7fa634400640 1 --2- 172.21.15.42:0/3537821246 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62c0af500 0x7fa62c14e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:18.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:18.130+0000 7fa634400640 1 -- 172.21.15.42:0/3537821246 >> 172.21.15.42:0/3537821246 conn(0x7fa62c06bff0 msgr2=0x7fa62c074a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:18.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:18.130+0000 7fa634400640 1 -- 172.21.15.42:0/3537821246 shutdown_connections 2024-05-28T14:16:18.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:18.130+0000 7fa634400640 1 -- 172.21.15.42:0/3537821246 wait complete. 2024-05-28T14:16:21.528 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:16:21.532 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:16:23.141 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-28T14:16:23.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa3c00640 1 Processor -- start 2024-05-28T14:16:23.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa3c00640 1 -- start start 2024-05-28T14:16:23.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa3c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5f9c15ea30 0x7f5f9c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:23.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa3c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5f9c0ccce0 0x7f5f9c06be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:23.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa3c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f9c0ce040 con 0x7f5f9c06b9f0 2024-05-28T14:16:23.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa3c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5f9c0cec50 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa3c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5f9c0ce320 con 0x7f5f9c15ea30 2024-05-28T14:16:23.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa1c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5f9c0ccce0 0x7f5f9c06be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:23.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa1c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5f9c0ccce0 0x7f5f9c06be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34808/0 (socket says 172.21.15.42:34808) 2024-05-28T14:16:23.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa1c00640 1 -- 172.21.15.42:0/4103094266 learned_addr learned my addr 172.21.15.42:0/4103094266 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:23.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa1c00640 1 -- 172.21.15.42:0/4103094266 >> v2:172.21.15.63:3300/0 conn(0x7f5f9c15ea30 msgr2=0x7f5f9c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:23.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa1200640 1 --2- 172.21.15.42:0/4103094266 >> v2:172.21.15.63:3300/0 conn(0x7f5f9c15ea30 0x7f5f9c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:23.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa1c00640 1 --2- 172.21.15.42:0/4103094266 >> v2:172.21.15.63:3300/0 conn(0x7f5f9c15ea30 0x7f5f9c0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa1c00640 1 -- 172.21.15.42:0/4103094266 >> v1:172.21.15.63:6789/0 conn(0x7f5f9c06b9f0 legacy=0x7f5f9c1626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:23.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa1c00640 1 -- 172.21.15.42:0/4103094266 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9c0ce600 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.306+0000 7f5fa1200640 1 --2- 172.21.15.42:0/4103094266 >> v2:172.21.15.63:3300/0 conn(0x7f5f9c15ea30 0x7f5f9c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:23.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.310+0000 7f5fa1c00640 1 --2- 172.21.15.42:0/4103094266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f9c0ccce0 0x7f5f9c06be80 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0d2570 tx=0x7f5f9401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b197b6c77865af86 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:23.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.310+0000 7f5f9be00640 1 -- 172.21.15.42:0/4103094266 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f940a56a0 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.310+0000 7f5f9be00640 1 -- 172.21.15.42:0/4103094266 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f940a5840 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.310+0000 7f5f9be00640 1 -- 172.21.15.42:0/4103094266 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f940a5c10 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.310+0000 7f5fa3c00640 1 -- 172.21.15.42:0/4103094266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f9c0ccce0 msgr2=0x7f5f9c06be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:23.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.310+0000 7f5fa3c00640 1 --2- 172.21.15.42:0/4103094266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f9c0ccce0 0x7f5f9c06be80 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0d2570 tx=0x7f5f9401c9b0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.310+0000 7f5fa3c00640 1 -- 172.21.15.42:0/4103094266 shutdown_connections 2024-05-28T14:16:23.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.310+0000 7f5fa3c00640 1 --2- 172.21.15.42:0/4103094266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f9c0ccce0 0x7f5f9c06be80 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.310+0000 7f5fa3c00640 1 --2- 172.21.15.42:0/4103094266 >> v2:172.21.15.63:3300/0 conn(0x7f5f9c15ea30 0x7f5f9c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.310+0000 7f5fa3c00640 1 -- 172.21.15.42:0/4103094266 >> 172.21.15.42:0/4103094266 conn(0x7f5f9c09aa50 msgr2=0x7f5f9c0b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:23.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.310+0000 7f5fa3c00640 1 -- 172.21.15.42:0/4103094266 shutdown_connections 2024-05-28T14:16:23.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.310+0000 7f5fa3c00640 1 -- 172.21.15.42:0/4103094266 wait complete. 2024-05-28T14:16:23.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5fa3c00640 1 Processor -- start 2024-05-28T14:16:23.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5fa3c00640 1 -- start start 2024-05-28T14:16:23.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5fa3c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f9c06b9f0 0x7f5f9c14b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:23.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5fa3c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f9c0ccce0 0x7f5f9c0ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:23.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5fa3c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f9c15ea30 0x7f5f9c14c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:23.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5fa3c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c0cde20 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5fa3c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c15f020 con 0x7f5f9c06b9f0 2024-05-28T14:16:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5fa3c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9c0b0890 con 0x7f5f9c15ea30 2024-05-28T14:16:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5f93e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f9c0ccce0 0x7f5f9c0ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5f93e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f9c0ccce0 0x7f5f9c0ae2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34820/0 (socket says 172.21.15.42:34820) 2024-05-28T14:16:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5f93e00640 1 -- 172.21.15.42:0/3416018139 learned_addr learned my addr 172.21.15.42:0/3416018139 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5fa1200640 1 --2- 172.21.15.42:0/3416018139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f9c06b9f0 0x7f5f9c14b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5fa1c00640 1 --2- 172.21.15.42:0/3416018139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f9c15ea30 0x7f5f9c14c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5f93e00640 1 -- 172.21.15.42:0/3416018139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f9c15ea30 msgr2=0x7f5f9c14c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5f93e00640 1 --2- 172.21.15.42:0/3416018139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f9c15ea30 0x7f5f9c14c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5f93e00640 1 -- 172.21.15.42:0/3416018139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f9c06b9f0 msgr2=0x7f5f9c14b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5f93e00640 1 --2- 172.21.15.42:0/3416018139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f9c06b9f0 0x7f5f9c14b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5f93e00640 1 -- 172.21.15.42:0/3416018139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9c0aecb0 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5f93e00640 1 --2- 172.21.15.42:0/3416018139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f9c0ccce0 0x7f5f9c0ae2d0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400c450 tx=0x7f5f8400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:23.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5f99600640 1 -- 172.21.15.42:0/3416018139 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84012830 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5f99600640 1 -- 172.21.15.42:0/3416018139 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f84017070 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5f99600640 1 -- 172.21.15.42:0/3416018139 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84012c30 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5fa3c00640 1 -- 172.21.15.42:0/3416018139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9c0aef90 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.314+0000 7f5fa3c00640 1 -- 172.21.15.42:0/3416018139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9c14be40 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.318+0000 7f5fa3c00640 1 -- 172.21.15.42:0/3416018139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f9c068a90 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.318+0000 7f5f99600640 1 -- 172.21.15.42:0/3416018139 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 90894+0+0 (secure 0 0 0) 0x7f5f8401c460 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.318+0000 7f5f99600640 1 --2- 172.21.15.42:0/3416018139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5f7806a390 0x7f5f7806c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.318+0000 7f5f99600640 1 -- 172.21.15.42:0/3416018139 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f5f840856c0 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.318+0000 7f5fa1200640 1 --2- 172.21.15.42:0/3416018139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5f7806a390 0x7f5f7806c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:23.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.318+0000 7f5fa1200640 1 --2- 172.21.15.42:0/3416018139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5f7806a390 0x7f5f7806c850 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c006fd0 tx=0x7f5f8c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:23.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.322+0000 7f5f99600640 1 -- 172.21.15.42:0/3416018139 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5f84057090 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.546+0000 7f5fa3c00640 1 -- 172.21.15.42:0/3416018139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5f9c0739d0 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.546+0000 7f5f99600640 1 -- 172.21.15.42:0/3416018139 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1734 (secure 0 0 0) 0x7f5f8405b390 con 0x7f5f9c0ccce0 2024-05-28T14:16:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:23.534 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:16:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "7380b1c2-773d-4bd7-abac-066c3536cd24", 2024-05-28T14:16:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "health": { 2024-05-28T14:16:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "HEALTH_OK", 2024-05-28T14:16:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "checks": {}, 2024-05-28T14:16:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "mutes": [] 2024-05-28T14:16:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "election_epoch": 8, 2024-05-28T14:16:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum": [ 2024-05-28T14:16:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: 0, 2024-05-28T14:16:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-05-28T14:16:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: 2 2024-05-28T14:16:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_names": [ 2024-05-28T14:16:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "a", 2024-05-28T14:16:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "b", 2024-05-28T14:16:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "c" 2024-05-28T14:16:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "quorum_age": 1104, 2024-05-28T14:16:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": { 2024-05-28T14:16:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 1, 2024-05-28T14:16:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "min_mon_release_name": "squid", 2024-05-28T14:16:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "num_mons": 3 2024-05-28T14:16:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap": { 2024-05-28T14:16:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 134, 2024-05-28T14:16:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "num_osds": 8, 2024-05-28T14:16:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "num_up_osds": 8, 2024-05-28T14:16:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_up_since": 1716904689, 2024-05-28T14:16:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "num_in_osds": 8, 2024-05-28T14:16:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_in_since": 1716904680, 2024-05-28T14:16:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "num_remapped_pgs": 0 2024-05-28T14:16:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap": { 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "pgs_by_state": [ 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "state_name": "active+clean", 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "count": 97 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pgs": 97, 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "num_pools": 3, 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "num_objects": 5083, 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "data_bytes": 21175116245, 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_used": 43518492672, 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_avail": 729575620608, 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "bytes_total": 773094113280 2024-05-28T14:16:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "fsmap": { 2024-05-28T14:16:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 85, 2024-05-28T14:16:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 9, 2024-05-28T14:16:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-05-28T14:16:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-05-28T14:16:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1, 2024-05-28T14:16:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "by_rank": [ 2024-05-28T14:16:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem_id": 9, 2024-05-28T14:16:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rank": 0, 2024-05-28T14:16:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "d", 2024-05-28T14:16:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "up:active", 2024-05-28T14:16:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 57851 2024-05-28T14:16:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "up:standby": 3 2024-05-28T14:16:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "mgrmap": { 2024-05-28T14:16:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-05-28T14:16:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "num_standbys": 1, 2024-05-28T14:16:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-05-28T14:16:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-05-28T14:16:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-05-28T14:16:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-05-28T14:16:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-05-28T14:16:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:16:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "servicemap": { 2024-05-28T14:16:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 24, 2024-05-28T14:16:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-05-28T14:15:51.232273+0000", 2024-05-28T14:16:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "services": {} 2024-05-28T14:16:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "progress_events": {} 2024-05-28T14:16:23.542 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:16:23.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.546+0000 7f5fa3c00640 1 -- 172.21.15.42:0/3416018139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5f7806a390 msgr2=0x7f5f7806c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:23.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.546+0000 7f5fa3c00640 1 --2- 172.21.15.42:0/3416018139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5f7806a390 0x7f5f7806c850 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c006fd0 tx=0x7f5f8c008040 comp rx=0 tx=0).stop 2024-05-28T14:16:23.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.546+0000 7f5fa3c00640 1 -- 172.21.15.42:0/3416018139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f9c0ccce0 msgr2=0x7f5f9c0ae2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:23.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.546+0000 7f5fa3c00640 1 --2- 172.21.15.42:0/3416018139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f9c0ccce0 0x7f5f9c0ae2d0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400c450 tx=0x7f5f8400c910 comp rx=0 tx=0).stop 2024-05-28T14:16:23.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.546+0000 7f5fa3c00640 1 -- 172.21.15.42:0/3416018139 shutdown_connections 2024-05-28T14:16:23.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.546+0000 7f5fa3c00640 1 --2- 172.21.15.42:0/3416018139 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5f7806a390 0x7f5f7806c850 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.546+0000 7f5fa3c00640 1 --2- 172.21.15.42:0/3416018139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f9c15ea30 0x7f5f9c14c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.546+0000 7f5fa3c00640 1 --2- 172.21.15.42:0/3416018139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f9c0ccce0 0x7f5f9c0ae2d0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.546+0000 7f5fa3c00640 1 --2- 172.21.15.42:0/3416018139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f9c06b9f0 0x7f5f9c14b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.546+0000 7f5fa3c00640 1 -- 172.21.15.42:0/3416018139 >> 172.21.15.42:0/3416018139 conn(0x7f5f9c09aa50 msgr2=0x7f5f9c15db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:23.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.550+0000 7f5fa3c00640 1 -- 172.21.15.42:0/3416018139 shutdown_connections 2024-05-28T14:16:23.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.550+0000 7f5fa3c00640 1 -- 172.21.15.42:0/3416018139 wait complete. 2024-05-28T14:16:23.555 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-28T14:16:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.710+0000 7ff3c5a00640 1 Processor -- start 2024-05-28T14:16:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.710+0000 7ff3c5a00640 1 -- start start 2024-05-28T14:16:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.710+0000 7ff3c5a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c0140d20 0x7ff3c0167490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.710+0000 7ff3c5a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c0160230 con 0x7ff3c0071de0 2024-05-28T14:16:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.710+0000 7ff3c5a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c0160510 con 0x7ff3c00ce3b0 2024-05-28T14:16:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.710+0000 7ff3c5a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c01607f0 con 0x7ff3c0140d20 2024-05-28T14:16:23.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.710+0000 7ff3bea00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff3c0071de0 0x7ff3c00721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50686/0 (socket says 172.21.15.42:50686) 2024-05-28T14:16:23.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.710+0000 7ff3bea00640 1 -- 172.21.15.42:0/2523944255 learned_addr learned my addr 172.21.15.42:0/2523944255 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:23.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bf400640 1 --2- 172.21.15.42:0/2523944255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c0140d20 0x7ff3c0167490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:23.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 -- 172.21.15.42:0/2523944255 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3642381817 0 0) 0x7ff3c0160230 con 0x7ff3c0071de0 2024-05-28T14:16:23.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 -- 172.21.15.42:0/2523944255 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3a4003530 con 0x7ff3c0071de0 2024-05-28T14:16:23.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 -- 172.21.15.42:0/2523944255 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3572803860 0 0) 0x7ff3a4003530 con 0x7ff3c0071de0 2024-05-28T14:16:23.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 -- 172.21.15.42:0/2523944255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c0140d20 msgr2=0x7ff3c0167490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:23.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 --2- 172.21.15.42:0/2523944255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c0140d20 0x7ff3c0167490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 -- 172.21.15.42:0/2523944255 >> v1:172.21.15.63:6789/0 conn(0x7ff3c00ce3b0 legacy=0x7ff3c00ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:23.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 -- 172.21.15.42:0/2523944255 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c0160ad0 con 0x7ff3c0071de0 2024-05-28T14:16:23.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 -- 172.21.15.42:0/2523944255 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7ff3b0001680 con 0x7ff3c0071de0 2024-05-28T14:16:23.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 -- 172.21.15.42:0/2523944255 >> v1:172.21.15.42:6789/0 conn(0x7ff3c0071de0 legacy=0x7ff3c00721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:23.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 --2- 172.21.15.42:0/2523944255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a40039f0 0x7ff3a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:23.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 --2- 172.21.15.42:0/2523944255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c0140d20 0x7ff3a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:23.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 --2- 172.21.15.42:0/2523944255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a4004d50 0x7ff3a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:23.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 -- 172.21.15.42:0/2523944255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01607f0 con 0x7ff3a4004d50 2024-05-28T14:16:23.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 -- 172.21.15.42:0/2523944255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a400bb60 con 0x7ff3a40039f0 2024-05-28T14:16:23.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bd600640 1 -- 172.21.15.42:0/2523944255 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a400bd60 con 0x7ff3c0140d20 2024-05-28T14:16:23.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bf400640 1 --2- 172.21.15.42:0/2523944255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a4004d50 0x7ff3a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:23.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bea00640 1 --2- 172.21.15.42:0/2523944255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a40039f0 0x7ff3a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:23.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3be000640 1 --2- 172.21.15.42:0/2523944255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c0140d20 0x7ff3a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:23.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bf400640 1 -- 172.21.15.42:0/2523944255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c0140d20 msgr2=0x7ff3a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:23.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bf400640 1 --2- 172.21.15.42:0/2523944255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c0140d20 0x7ff3a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bf400640 1 -- 172.21.15.42:0/2523944255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a40039f0 msgr2=0x7ff3a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:23.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bf400640 1 --2- 172.21.15.42:0/2523944255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a40039f0 0x7ff3a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.714+0000 7ff3bf400640 1 -- 172.21.15.42:0/2523944255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a4003530 con 0x7ff3a4004d50 2024-05-28T14:16:23.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3bf400640 1 --2- 172.21.15.42:0/2523944255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a4004d50 0x7ff3a40051d0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009f90 tx=0x7ff3b801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7211e4482132a0c server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:23.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3bd600640 1 -- 172.21.15.42:0/2523944255 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80b2070 con 0x7ff3a4004d50 2024-05-28T14:16:23.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3bd600640 1 -- 172.21.15.42:0/2523944255 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b80a5c50 con 0x7ff3a4004d50 2024-05-28T14:16:23.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3bd600640 1 -- 172.21.15.42:0/2523944255 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8009c10 con 0x7ff3a4004d50 2024-05-28T14:16:23.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3c5a00640 1 -- 172.21.15.42:0/2523944255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a4004d50 msgr2=0x7ff3a40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:23.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3c5a00640 1 --2- 172.21.15.42:0/2523944255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3a4004d50 0x7ff3a40051d0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009f90 tx=0x7ff3b801cc40 comp rx=0 tx=0).stop 2024-05-28T14:16:23.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3bf400640 1 -- 172.21.15.42:0/2523944255 reap_dead start 2024-05-28T14:16:23.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3c5a00640 1 -- 172.21.15.42:0/2523944255 shutdown_connections 2024-05-28T14:16:23.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3c5a00640 1 -- 172.21.15.42:0/2523944255 >> 172.21.15.42:0/2523944255 conn(0x7ff3c00b21f0 msgr2=0x7ff3c00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:23.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3c5a00640 1 -- 172.21.15.42:0/2523944255 shutdown_connections 2024-05-28T14:16:23.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3c5a00640 1 -- 172.21.15.42:0/2523944255 wait complete. 2024-05-28T14:16:23.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3c5a00640 1 Processor -- start 2024-05-28T14:16:23.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3c5a00640 1 -- start start 2024-05-28T14:16:23.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3c5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c013fee0 0x7ff3c013a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:23.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3c5a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c0155d80 0x7ff3c0155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:23.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3c5a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3c01402c0 0x7ff3c0155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:23.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3c5a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0165600 con 0x7ff3c013fee0 2024-05-28T14:16:23.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3c5a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0165480 con 0x7ff3c01402c0 2024-05-28T14:16:23.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.718+0000 7ff3c5a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c0165780 con 0x7ff3c0155d80 2024-05-28T14:16:23.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3bea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c013fee0 0x7ff3c013a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:23.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3bea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c013fee0 0x7ff3c013a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34838/0 (socket says 172.21.15.42:34838) 2024-05-28T14:16:23.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3bea00640 1 -- 172.21.15.42:0/1481590978 learned_addr learned my addr 172.21.15.42:0/1481590978 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:23.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3bea00640 1 -- 172.21.15.42:0/1481590978 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c0155d80 msgr2=0x7ff3c0155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:23.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3bea00640 1 --2- 172.21.15.42:0/1481590978 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c0155d80 0x7ff3c0155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3bea00640 1 -- 172.21.15.42:0/1481590978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3c01402c0 msgr2=0x7ff3c0155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:23.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3bea00640 1 --2- 172.21.15.42:0/1481590978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3c01402c0 0x7ff3c0155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3bea00640 1 -- 172.21.15.42:0/1481590978 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c013b1e0 con 0x7ff3c013fee0 2024-05-28T14:16:23.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3bea00640 1 --2- 172.21.15.42:0/1481590978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c013fee0 0x7ff3c013a8e0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00072c0 tx=0x7ff3b0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:23.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3b6a00640 1 -- 172.21.15.42:0/1481590978 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b000f940 con 0x7ff3c013fee0 2024-05-28T14:16:23.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3b6a00640 1 -- 172.21.15.42:0/1481590978 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b0018070 con 0x7ff3c013fee0 2024-05-28T14:16:23.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3b6a00640 1 -- 172.21.15.42:0/1481590978 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b000fd60 con 0x7ff3c013fee0 2024-05-28T14:16:23.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3c5a00640 1 -- 172.21.15.42:0/1481590978 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c013b4c0 con 0x7ff3c013fee0 2024-05-28T14:16:23.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3c5a00640 1 -- 172.21.15.42:0/1481590978 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c013ba80 con 0x7ff3c013fee0 2024-05-28T14:16:23.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3c5a00640 1 -- 172.21.15.42:0/1481590978 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c00738b0 con 0x7ff3c013fee0 2024-05-28T14:16:23.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3b6a00640 1 -- 172.21.15.42:0/1481590978 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 90894+0+0 (secure 0 0 0) 0x7ff3b001d460 con 0x7ff3c013fee0 2024-05-28T14:16:23.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.722+0000 7ff3b6a00640 1 --2- 172.21.15.42:0/1481590978 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3a006a390 0x7ff3a006c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:23.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.726+0000 7ff3b6a00640 1 -- 172.21.15.42:0/1481590978 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7ff3b0085bf0 con 0x7ff3c013fee0 2024-05-28T14:16:23.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.726+0000 7ff3be000640 1 --2- 172.21.15.42:0/1481590978 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3a006a390 0x7ff3a006c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:23.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.726+0000 7ff3be000640 1 --2- 172.21.15.42:0/1481590978 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3a006a390 0x7ff3a006c850 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac007990 tx=0x7ff3ac007920 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:23.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.730+0000 7ff3b6a00640 1 -- 172.21.15.42:0/1481590978 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff3b0057c10 con 0x7ff3c013fee0 2024-05-28T14:16:23.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.922+0000 7ff3c5a00640 1 -- 172.21.15.42:0/1481590978 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3c00cdd40 con 0x7ff3a006a390 2024-05-28T14:16:23.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.930+0000 7ff3b6a00640 1 -- 172.21.15.42:0/1481590978 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+374 (secure 0 0 0) 0x7ff3c00cdd40 con 0x7ff3a006a390 2024-05-28T14:16:23.917 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:23.917 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:16:23.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:23.918 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone6275" 2024-05-28T14:16:23.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:23.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:23.918 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone6276" 2024-05-28T14:16:23.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:23.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:23.918 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone6272" 2024-05-28T14:16:23.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:23.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:23.918 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sv1" 2024-05-28T14:16:23.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:23.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:23.918 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone6271" 2024-05-28T14:16:23.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:23.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:23.919 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone6274" 2024-05-28T14:16:23.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:23.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:23.919 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone6273" 2024-05-28T14:16:23.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-05-28T14:16:23.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:23.919 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subvol_clone6277" 2024-05-28T14:16:23.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:23.919 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:16:23.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.930+0000 7ff3c5a00640 1 -- 172.21.15.42:0/1481590978 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3a006a390 msgr2=0x7ff3a006c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:23.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.930+0000 7ff3c5a00640 1 --2- 172.21.15.42:0/1481590978 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3a006a390 0x7ff3a006c850 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac007990 tx=0x7ff3ac007920 comp rx=0 tx=0).stop 2024-05-28T14:16:23.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.930+0000 7ff3c5a00640 1 -- 172.21.15.42:0/1481590978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c013fee0 msgr2=0x7ff3c013a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:23.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.930+0000 7ff3c5a00640 1 --2- 172.21.15.42:0/1481590978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c013fee0 0x7ff3c013a8e0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00072c0 tx=0x7ff3b0004f90 comp rx=0 tx=0).stop 2024-05-28T14:16:23.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.930+0000 7ff3c5a00640 1 -- 172.21.15.42:0/1481590978 shutdown_connections 2024-05-28T14:16:23.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.930+0000 7ff3c5a00640 1 --2- 172.21.15.42:0/1481590978 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff3a006a390 0x7ff3a006c850 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.930+0000 7ff3c5a00640 1 --2- 172.21.15.42:0/1481590978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3c01402c0 0x7ff3c0155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.930+0000 7ff3c5a00640 1 --2- 172.21.15.42:0/1481590978 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3c0155d80 0x7ff3c0155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.930+0000 7ff3c5a00640 1 --2- 172.21.15.42:0/1481590978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3c013fee0 0x7ff3c013a8e0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:23.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.930+0000 7ff3c5a00640 1 -- 172.21.15.42:0/1481590978 >> 172.21.15.42:0/1481590978 conn(0x7ff3c00b21f0 msgr2=0x7ff3c00b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:23.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.930+0000 7ff3c5a00640 1 -- 172.21.15.42:0/1481590978 shutdown_connections 2024-05-28T14:16:23.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:23.930+0000 7ff3c5a00640 1 -- 172.21.15.42:0/1481590978 wait complete. 2024-05-28T14:16:23.939 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone6275 2024-05-28T14:16:24.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.098+0000 7fea4e600640 1 Processor -- start 2024-05-28T14:16:24.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.098+0000 7fea4e600640 1 -- start start 2024-05-28T14:16:24.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.098+0000 7fea4e600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fea480ccdc0 0x7fea4806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.098+0000 7fea4e600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea4815d080 0x7fea48072a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.098+0000 7fea4e600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea4815d6f0 con 0x7fea480cd1a0 2024-05-28T14:16:24.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.098+0000 7fea4e600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fea4815d9d0 con 0x7fea480ccdc0 2024-05-28T14:16:24.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.098+0000 7fea4e600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea4806ebd0 con 0x7fea4815d080 2024-05-28T14:16:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.098+0000 7fea47e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fea480cd1a0 0x7fea4815c970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50692/0 (socket says 172.21.15.42:50692) 2024-05-28T14:16:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.098+0000 7fea47e00640 1 -- 172.21.15.42:0/2124572161 learned_addr learned my addr 172.21.15.42:0/2124572161 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.098+0000 7fea4d200640 1 --2- 172.21.15.42:0/2124572161 >> v2:172.21.15.63:3300/0 conn(0x7fea480ccdc0 0x7fea4806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.098+0000 7fea4dc00640 1 --2- 172.21.15.42:0/2124572161 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea4815d080 0x7fea48072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.102+0000 7fea47400640 1 -- 172.21.15.42:0/2124572161 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1976572629 0 0) 0x7fea4815d6f0 con 0x7fea480cd1a0 2024-05-28T14:16:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.102+0000 7fea47400640 1 -- 172.21.15.42:0/2124572161 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea30003530 con 0x7fea480cd1a0 2024-05-28T14:16:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.102+0000 7fea4dc00640 1 -- 172.21.15.42:0/2124572161 >> v2:172.21.15.63:3300/0 conn(0x7fea480ccdc0 msgr2=0x7fea4806f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.102+0000 7fea4dc00640 1 --2- 172.21.15.42:0/2124572161 >> v2:172.21.15.63:3300/0 conn(0x7fea480ccdc0 0x7fea4806f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.102+0000 7fea4dc00640 1 -- 172.21.15.42:0/2124572161 >> v1:172.21.15.42:6789/0 conn(0x7fea480cd1a0 legacy=0x7fea4815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.102+0000 7fea4dc00640 1 -- 172.21.15.42:0/2124572161 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4806eeb0 con 0x7fea4815d080 2024-05-28T14:16:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.102+0000 7fea4d200640 1 --2- 172.21.15.42:0/2124572161 >> v2:172.21.15.63:3300/0 conn(0x7fea480ccdc0 0x7fea4806f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:16:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.102+0000 7fea4dc00640 1 --2- 172.21.15.42:0/2124572161 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea4815d080 0x7fea48072a10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea40009d90 tx=0x7fea4001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de4a1ad0ad2827eb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.102+0000 7fea47400640 1 -- 172.21.15.42:0/2124572161 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea400b2070 con 0x7fea4815d080 2024-05-28T14:16:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.102+0000 7fea47400640 1 -- 172.21.15.42:0/2124572161 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea400a5830 con 0x7fea4815d080 2024-05-28T14:16:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.102+0000 7fea47400640 1 -- 172.21.15.42:0/2124572161 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea400a5c00 con 0x7fea4815d080 2024-05-28T14:16:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 -- 172.21.15.42:0/2124572161 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea4815d080 msgr2=0x7fea48072a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 --2- 172.21.15.42:0/2124572161 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea4815d080 0x7fea48072a10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea40009d90 tx=0x7fea4001c970 comp rx=0 tx=0).stop 2024-05-28T14:16:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 -- 172.21.15.42:0/2124572161 shutdown_connections 2024-05-28T14:16:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 --2- 172.21.15.42:0/2124572161 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea4815d080 0x7fea48072a10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 --2- 172.21.15.42:0/2124572161 >> v2:172.21.15.63:3300/0 conn(0x7fea480ccdc0 0x7fea4806f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 -- 172.21.15.42:0/2124572161 >> 172.21.15.42:0/2124572161 conn(0x7fea4806bbc0 msgr2=0x7fea4806bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 -- 172.21.15.42:0/2124572161 shutdown_connections 2024-05-28T14:16:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 -- 172.21.15.42:0/2124572161 wait complete. 2024-05-28T14:16:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 Processor -- start 2024-05-28T14:16:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 -- start start 2024-05-28T14:16:24.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea480ccdc0 0x7fea48099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea480cd1a0 0x7fea4809a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4815d080 0x7fea4809a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea481694d0 con 0x7fea4815d080 2024-05-28T14:16:24.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea48169350 con 0x7fea480ccdc0 2024-05-28T14:16:24.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4e600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea48169650 con 0x7fea480cd1a0 2024-05-28T14:16:24.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4815d080 0x7fea4809a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4815d080 0x7fea4809a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34852/0 (socket says 172.21.15.42:34852) 2024-05-28T14:16:24.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4dc00640 1 -- 172.21.15.42:0/2087755786 learned_addr learned my addr 172.21.15.42:0/2087755786 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:24.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4d200640 1 --2- 172.21.15.42:0/2087755786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea480ccdc0 0x7fea48099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea47e00640 1 --2- 172.21.15.42:0/2087755786 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea480cd1a0 0x7fea4809a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4dc00640 1 -- 172.21.15.42:0/2087755786 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea480cd1a0 msgr2=0x7fea4809a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4dc00640 1 --2- 172.21.15.42:0/2087755786 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea480cd1a0 0x7fea4809a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4dc00640 1 -- 172.21.15.42:0/2087755786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea480ccdc0 msgr2=0x7fea48099920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4dc00640 1 --2- 172.21.15.42:0/2087755786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea480ccdc0 0x7fea48099920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4dc00640 1 -- 172.21.15.42:0/2087755786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4809c280 con 0x7fea4815d080 2024-05-28T14:16:24.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.106+0000 7fea4dc00640 1 --2- 172.21.15.42:0/2087755786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4815d080 0x7fea4809a810 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fea400b0040 tx=0x7fea40019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:24.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.110+0000 7fea44c00640 1 -- 172.21.15.42:0/2087755786 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea400b2050 con 0x7fea4815d080 2024-05-28T14:16:24.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.110+0000 7fea44c00640 1 -- 172.21.15.42:0/2087755786 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea400be070 con 0x7fea4815d080 2024-05-28T14:16:24.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.110+0000 7fea44c00640 1 -- 172.21.15.42:0/2087755786 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea400ad850 con 0x7fea4815d080 2024-05-28T14:16:24.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.110+0000 7fea4e600640 1 -- 172.21.15.42:0/2087755786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea480adff0 con 0x7fea4815d080 2024-05-28T14:16:24.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.110+0000 7fea4e600640 1 -- 172.21.15.42:0/2087755786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea480ae5b0 con 0x7fea4815d080 2024-05-28T14:16:24.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.110+0000 7fea4e600640 1 -- 172.21.15.42:0/2087755786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea48068a90 con 0x7fea4815d080 2024-05-28T14:16:24.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.110+0000 7fea44c00640 1 -- 172.21.15.42:0/2087755786 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 90894+0+0 (secure 0 0 0) 0x7fea400ad9f0 con 0x7fea4815d080 2024-05-28T14:16:24.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.110+0000 7fea44c00640 1 --2- 172.21.15.42:0/2087755786 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea2806a390 0x7fea2806c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.110+0000 7fea44c00640 1 -- 172.21.15.42:0/2087755786 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fea40120bc0 con 0x7fea4815d080 2024-05-28T14:16:24.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.110+0000 7fea4d200640 1 --2- 172.21.15.42:0/2087755786 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea2806a390 0x7fea2806c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.114+0000 7fea4d200640 1 --2- 172.21.15.42:0/2087755786 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea2806a390 0x7fea2806c850 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009a50 tx=0x7fea3c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:24.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.114+0000 7fea44c00640 1 -- 172.21.15.42:0/2087755786 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fea4809c280 con 0x7fea4815d080 2024-05-28T14:16:24.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.310+0000 7fea4e600640 1 -- 172.21.15.42:0/2087755786 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone6275", "target": ["mon-mgr", ""]}) -- 0x7fea4806e300 con 0x7fea2806a390 2024-05-28T14:16:24.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.334+0000 7fea44c00640 1 -- 172.21.15.42:0/2087755786 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fea4806e300 con 0x7fea2806a390 2024-05-28T14:16:24.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.334+0000 7fea4e600640 1 -- 172.21.15.42:0/2087755786 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea2806a390 msgr2=0x7fea2806c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.334+0000 7fea4e600640 1 --2- 172.21.15.42:0/2087755786 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea2806a390 0x7fea2806c850 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009a50 tx=0x7fea3c008040 comp rx=0 tx=0).stop 2024-05-28T14:16:24.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.334+0000 7fea4e600640 1 -- 172.21.15.42:0/2087755786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4815d080 msgr2=0x7fea4809a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.334+0000 7fea4e600640 1 --2- 172.21.15.42:0/2087755786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4815d080 0x7fea4809a810 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fea400b0040 tx=0x7fea40019710 comp rx=0 tx=0).stop 2024-05-28T14:16:24.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.334+0000 7fea4e600640 1 -- 172.21.15.42:0/2087755786 shutdown_connections 2024-05-28T14:16:24.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.334+0000 7fea4e600640 1 --2- 172.21.15.42:0/2087755786 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea2806a390 0x7fea2806c850 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.334+0000 7fea4e600640 1 --2- 172.21.15.42:0/2087755786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea4815d080 0x7fea4809a810 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.334+0000 7fea4e600640 1 --2- 172.21.15.42:0/2087755786 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea480cd1a0 0x7fea4809a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.334+0000 7fea4e600640 1 --2- 172.21.15.42:0/2087755786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea480ccdc0 0x7fea48099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.334+0000 7fea4e600640 1 -- 172.21.15.42:0/2087755786 >> 172.21.15.42:0/2087755786 conn(0x7fea4806bbc0 msgr2=0x7fea4815f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:24.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.334+0000 7fea4e600640 1 -- 172.21.15.42:0/2087755786 shutdown_connections 2024-05-28T14:16:24.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.334+0000 7fea4e600640 1 -- 172.21.15.42:0/2087755786 wait complete. 2024-05-28T14:16:24.326 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:24.338+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:16:24.326 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:24.338+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:16:24.326 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:24.338+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:16:24.327 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:24.338+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:16:24.327 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:24.338+0000 7f0777800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:16:24.341 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:24.354+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:16:24.341 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:24.354+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:16:24.341 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:24.354+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:16:24.341 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:24.354+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:16:24.342 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:24.354+0000 7f0778200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-28T14:16:24.345 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone6276 2024-05-28T14:16:24.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f0485c00640 1 Processor -- start 2024-05-28T14:16:24.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f0485c00640 1 -- start start 2024-05-28T14:16:24.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f0485c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f048006c940 0x7f048009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f0485c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f048009af90 0x7f0480070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f0485c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f048006ff10 con 0x7f048006e740 2024-05-28T14:16:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f0485c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f048006eb20 con 0x7f048009af90 2024-05-28T14:16:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f0485c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f04800701b0 con 0x7f048006c940 2024-05-28T14:16:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047ea00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f048006e740 0x7f04800cdbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50696/0 (socket says 172.21.15.42:50696) 2024-05-28T14:16:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047ea00640 1 -- 172.21.15.42:0/451997292 learned_addr learned my addr 172.21.15.42:0/451997292 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047e000640 1 --2- 172.21.15.42:0/451997292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f048006c940 0x7f048009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047f400640 1 --2- 172.21.15.42:0/451997292 >> v2:172.21.15.63:3300/0 conn(0x7f048009af90 0x7f0480070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 -- 172.21.15.42:0/451997292 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1054187484 0 0) 0x7f048006ff10 con 0x7f048006e740 2024-05-28T14:16:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 -- 172.21.15.42:0/451997292 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0464003530 con 0x7f048006e740 2024-05-28T14:16:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 -- 172.21.15.42:0/451997292 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3648413117 0 0) 0x7f0464003530 con 0x7f048006e740 2024-05-28T14:16:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 -- 172.21.15.42:0/451997292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f048006c940 msgr2=0x7f048009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 --2- 172.21.15.42:0/451997292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f048006c940 0x7f048009aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 -- 172.21.15.42:0/451997292 >> v2:172.21.15.63:3300/0 conn(0x7f048009af90 msgr2=0x7f0480070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 --2- 172.21.15.42:0/451997292 >> v2:172.21.15.63:3300/0 conn(0x7f048009af90 0x7f0480070f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 -- 172.21.15.42:0/451997292 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0480070490 con 0x7f048006e740 2024-05-28T14:16:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 -- 172.21.15.42:0/451997292 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f0470001680 con 0x7f048006e740 2024-05-28T14:16:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047f400640 1 --2- 172.21.15.42:0/451997292 >> v2:172.21.15.63:3300/0 conn(0x7f048009af90 0x7f0480070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:16:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 -- 172.21.15.42:0/451997292 >> v1:172.21.15.42:6789/0 conn(0x7f048006e740 legacy=0x7f04800cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 --2- 172.21.15.42:0/451997292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f048006c940 0x7f04640039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 --2- 172.21.15.42:0/451997292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0464003f40 0x7f0464004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 --2- 172.21.15.42:0/451997292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0464004d50 0x7f04640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 -- 172.21.15.42:0/451997292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f048006eb20 con 0x7f0464004d50 2024-05-28T14:16:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 -- 172.21.15.42:0/451997292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04800701b0 con 0x7f0464003f40 2024-05-28T14:16:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047d600640 1 -- 172.21.15.42:0/451997292 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f046400ba20 con 0x7f048006c940 2024-05-28T14:16:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047f400640 1 --2- 172.21.15.42:0/451997292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0464004d50 0x7f04640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047ea00640 1 --2- 172.21.15.42:0/451997292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f048006c940 0x7f04640039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047e000640 1 --2- 172.21.15.42:0/451997292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0464003f40 0x7f0464004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047f400640 1 -- 172.21.15.42:0/451997292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f048006c940 msgr2=0x7f04640039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:24.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047f400640 1 --2- 172.21.15.42:0/451997292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f048006c940 0x7f04640039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047f400640 1 -- 172.21.15.42:0/451997292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0464003f40 msgr2=0x7f0464004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:24.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047f400640 1 --2- 172.21.15.42:0/451997292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0464003f40 0x7f0464004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.506+0000 7f047f400640 1 -- 172.21.15.42:0/451997292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0464003530 con 0x7f0464004d50 2024-05-28T14:16:24.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.510+0000 7f047f400640 1 --2- 172.21.15.42:0/451997292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0464004d50 0x7f04640051d0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f047800e380 tx=0x7f047801cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7deec70916e12eab server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:24.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.510+0000 7f047d600640 1 -- 172.21.15.42:0/451997292 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04780b0070 con 0x7f0464004d50 2024-05-28T14:16:24.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.510+0000 7f047d600640 1 -- 172.21.15.42:0/451997292 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04780ab600 con 0x7f0464004d50 2024-05-28T14:16:24.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.510+0000 7f047d600640 1 -- 172.21.15.42:0/451997292 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478009de0 con 0x7f0464004d50 2024-05-28T14:16:24.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.510+0000 7f0485c00640 1 -- 172.21.15.42:0/451997292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0464004d50 msgr2=0x7f04640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.510+0000 7f0485c00640 1 --2- 172.21.15.42:0/451997292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0464004d50 0x7f04640051d0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f047800e380 tx=0x7f047801cb40 comp rx=0 tx=0).stop 2024-05-28T14:16:24.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.510+0000 7f047f400640 1 -- 172.21.15.42:0/451997292 reap_dead start 2024-05-28T14:16:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.510+0000 7f0485c00640 1 -- 172.21.15.42:0/451997292 shutdown_connections 2024-05-28T14:16:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.510+0000 7f0485c00640 1 -- 172.21.15.42:0/451997292 >> 172.21.15.42:0/451997292 conn(0x7f04800b3fa0 msgr2=0x7f04800b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.510+0000 7f0485c00640 1 -- 172.21.15.42:0/451997292 shutdown_connections 2024-05-28T14:16:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.510+0000 7f0485c00640 1 -- 172.21.15.42:0/451997292 wait complete. 2024-05-28T14:16:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f0485c00640 1 Processor -- start 2024-05-28T14:16:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f0485c00640 1 -- start start 2024-05-28T14:16:24.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f0485c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04800cbd90 0x7f04801d4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f0485c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04800cc170 0x7f04801d5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f0485c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04801e9760 0x7f04801d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f0485c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f04800718c0 con 0x7f04801e9760 2024-05-28T14:16:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f0485c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f048009baa0 con 0x7f04800cc170 2024-05-28T14:16:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f0485c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0480071a40 con 0x7f04800cbd90 2024-05-28T14:16:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f047f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04801e9760 0x7f04801d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f047ea00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04800cbd90 0x7f04801d4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f047f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04801e9760 0x7f04801d1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34906/0 (socket says 172.21.15.42:34906) 2024-05-28T14:16:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f047f400640 1 -- 172.21.15.42:0/1401221521 learned_addr learned my addr 172.21.15.42:0/1401221521 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f047e000640 1 --2- 172.21.15.42:0/1401221521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04800cc170 0x7f04801d5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f047f400640 1 -- 172.21.15.42:0/1401221521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04800cbd90 msgr2=0x7f04801d4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f047f400640 1 --2- 172.21.15.42:0/1401221521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04800cbd90 0x7f04801d4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f047f400640 1 -- 172.21.15.42:0/1401221521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04800cc170 msgr2=0x7f04801d5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f047f400640 1 --2- 172.21.15.42:0/1401221521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04800cc170 0x7f04801d5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f047f400640 1 -- 172.21.15.42:0/1401221521 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04800b54f0 con 0x7f04801e9760 2024-05-28T14:16:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f047f400640 1 --2- 172.21.15.42:0/1401221521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04801e9760 0x7f04801d1d20 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f04780ae040 tx=0x7f0478009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:24.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f0476a00640 1 -- 172.21.15.42:0/1401221521 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04780bd020 con 0x7f04801e9760 2024-05-28T14:16:24.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f0476a00640 1 -- 172.21.15.42:0/1401221521 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04780abd30 con 0x7f04801e9760 2024-05-28T14:16:24.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f0476a00640 1 -- 172.21.15.42:0/1401221521 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04780b0040 con 0x7f04801e9760 2024-05-28T14:16:24.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f0485c00640 1 -- 172.21.15.42:0/1401221521 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04801e8c80 con 0x7f04801e9760 2024-05-28T14:16:24.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.514+0000 7f0485c00640 1 -- 172.21.15.42:0/1401221521 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04801d9000 con 0x7f04801e9760 2024-05-28T14:16:24.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.518+0000 7f0485c00640 1 -- 172.21.15.42:0/1401221521 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0480068a90 con 0x7f04801e9760 2024-05-28T14:16:24.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.518+0000 7f0476a00640 1 -- 172.21.15.42:0/1401221521 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 90894+0+0 (secure 0 0 0) 0x7f04780b6440 con 0x7f04801e9760 2024-05-28T14:16:24.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.518+0000 7f0476a00640 1 --2- 172.21.15.42:0/1401221521 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f046006a390 0x7f046006c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.518+0000 7f0476a00640 1 -- 172.21.15.42:0/1401221521 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f047811f1f0 con 0x7f04801e9760 2024-05-28T14:16:24.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.518+0000 7f047ea00640 1 --2- 172.21.15.42:0/1401221521 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f046006a390 0x7f046006c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.518+0000 7f047ea00640 1 --2- 172.21.15.42:0/1401221521 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f046006a390 0x7f046006c850 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0470001060 tx=0x7f0470007900 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:24.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.522+0000 7f0476a00640 1 -- 172.21.15.42:0/1401221521 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0478016bb0 con 0x7f04801e9760 2024-05-28T14:16:24.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.714+0000 7f0485c00640 1 -- 172.21.15.42:0/1401221521 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone6276", "target": ["mon-mgr", ""]}) -- 0x7f04800b57d0 con 0x7f046006a390 2024-05-28T14:16:24.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.738+0000 7f0476a00640 1 -- 172.21.15.42:0/1401221521 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f04800b57d0 con 0x7f046006a390 2024-05-28T14:16:24.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.738+0000 7f0485c00640 1 -- 172.21.15.42:0/1401221521 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f046006a390 msgr2=0x7f046006c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.738+0000 7f0485c00640 1 --2- 172.21.15.42:0/1401221521 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f046006a390 0x7f046006c850 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0470001060 tx=0x7f0470007900 comp rx=0 tx=0).stop 2024-05-28T14:16:24.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.738+0000 7f0485c00640 1 -- 172.21.15.42:0/1401221521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04801e9760 msgr2=0x7f04801d1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.738+0000 7f0485c00640 1 --2- 172.21.15.42:0/1401221521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04801e9760 0x7f04801d1d20 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f04780ae040 tx=0x7f0478009750 comp rx=0 tx=0).stop 2024-05-28T14:16:24.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.738+0000 7f0485c00640 1 -- 172.21.15.42:0/1401221521 shutdown_connections 2024-05-28T14:16:24.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.738+0000 7f0485c00640 1 --2- 172.21.15.42:0/1401221521 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f046006a390 0x7f046006c850 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.738+0000 7f0485c00640 1 --2- 172.21.15.42:0/1401221521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04801e9760 0x7f04801d1d20 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.738+0000 7f0485c00640 1 --2- 172.21.15.42:0/1401221521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04800cc170 0x7f04801d5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.738+0000 7f0485c00640 1 --2- 172.21.15.42:0/1401221521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04800cbd90 0x7f04801d4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.738+0000 7f0485c00640 1 -- 172.21.15.42:0/1401221521 >> 172.21.15.42:0/1401221521 conn(0x7f04800b3fa0 msgr2=0x7f04800b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:24.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.738+0000 7f0485c00640 1 -- 172.21.15.42:0/1401221521 shutdown_connections 2024-05-28T14:16:24.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.738+0000 7f0485c00640 1 -- 172.21.15.42:0/1401221521 wait complete. 2024-05-28T14:16:24.748 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone6272 2024-05-28T14:16:24.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.906+0000 7fbeaa600640 1 Processor -- start 2024-05-28T14:16:24.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.906+0000 7fbeaa600640 1 -- start start 2024-05-28T14:16:24.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.906+0000 7fbeaa600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbea406ee80 0x7fbea406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.906+0000 7fbeaa600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbea40af360 0x7fbea40af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.906+0000 7fbeaa600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbea40d06f0 0x7fbea40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.906+0000 7fbeaa600640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fbea40d1070 con 0x7fbea406ee80 2024-05-28T14:16:24.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.906+0000 7fbeaa600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbea41570a0 con 0x7fbea40af360 2024-05-28T14:16:24.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.906+0000 7fbeaa600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbea415cf00 con 0x7fbea40d06f0 2024-05-28T14:16:24.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.906+0000 7fbea3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbea406ee80 0x7fbea406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.906+0000 7fbea3400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbea406ee80 0x7fbea406dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34920/0 (socket says 172.21.15.42:34920) 2024-05-28T14:16:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.906+0000 7fbea3400640 1 -- 172.21.15.42:0/1896544744 learned_addr learned my addr 172.21.15.42:0/1896544744 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.910+0000 7fbea3e00640 1 --2- 172.21.15.42:0/1896544744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbea40d06f0 0x7fbea40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.910+0000 7fbe9aa00640 1 --2- 172.21.15.42:0/1896544744 >> v2:172.21.15.63:3300/0 conn(0x7fbea40af360 0x7fbea40af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.910+0000 7fbea3400640 1 -- 172.21.15.42:0/1896544744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbea40d06f0 msgr2=0x7fbea40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:24.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.910+0000 7fbea3400640 1 --2- 172.21.15.42:0/1896544744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbea40d06f0 0x7fbea40d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.910+0000 7fbea3400640 1 -- 172.21.15.42:0/1896544744 >> v2:172.21.15.63:3300/0 conn(0x7fbea40af360 msgr2=0x7fbea40af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:24.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.910+0000 7fbea3400640 1 --2- 172.21.15.42:0/1896544744 >> v2:172.21.15.63:3300/0 conn(0x7fbea40af360 0x7fbea40af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.910+0000 7fbea3400640 1 -- 172.21.15.42:0/1896544744 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea415d1e0 con 0x7fbea406ee80 2024-05-28T14:16:24.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.910+0000 7fbea3400640 1 --2- 172.21.15.42:0/1896544744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbea406ee80 0x7fbea406dd40 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fbea4073fc0 tx=0x7fbe9401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc559bcaff7b5c2d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:24.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.910+0000 7fbea2a00640 1 -- 172.21.15.42:0/1896544744 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe940a5720 con 0x7fbea406ee80 2024-05-28T14:16:24.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.910+0000 7fbea2a00640 1 -- 172.21.15.42:0/1896544744 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe940a58c0 con 0x7fbea406ee80 2024-05-28T14:16:24.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.910+0000 7fbea2a00640 1 -- 172.21.15.42:0/1896544744 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe940a5c90 con 0x7fbea406ee80 2024-05-28T14:16:24.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 -- 172.21.15.42:0/1896544744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbea406ee80 msgr2=0x7fbea406dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 --2- 172.21.15.42:0/1896544744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbea406ee80 0x7fbea406dd40 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fbea4073fc0 tx=0x7fbe9401c8a0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 -- 172.21.15.42:0/1896544744 shutdown_connections 2024-05-28T14:16:24.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 --2- 172.21.15.42:0/1896544744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbea40d06f0 0x7fbea40d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 --2- 172.21.15.42:0/1896544744 >> v2:172.21.15.63:3300/0 conn(0x7fbea40af360 0x7fbea40af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 --2- 172.21.15.42:0/1896544744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbea406ee80 0x7fbea406dd40 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 -- 172.21.15.42:0/1896544744 >> 172.21.15.42:0/1896544744 conn(0x7fbea406bdf0 msgr2=0x7fbea406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:24.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 -- 172.21.15.42:0/1896544744 shutdown_connections 2024-05-28T14:16:24.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 -- 172.21.15.42:0/1896544744 wait complete. 2024-05-28T14:16:24.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 Processor -- start 2024-05-28T14:16:24.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 -- start start 2024-05-28T14:16:24.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbea406ee80 0x7fbea4153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbea40af360 0x7fbea4071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbea40d06f0 0x7fbea4156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbea415cce0 con 0x7fbea40af360 2024-05-28T14:16:24.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbea415cb60 con 0x7fbea40d06f0 2024-05-28T14:16:24.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbeaa600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbea4160c90 con 0x7fbea406ee80 2024-05-28T14:16:24.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbe9aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbea40af360 0x7fbea4071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbe9aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbea40af360 0x7fbea4071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34932/0 (socket says 172.21.15.42:34932) 2024-05-28T14:16:24.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbe9aa00640 1 -- 172.21.15.42:0/3895073374 learned_addr learned my addr 172.21.15.42:0/3895073374 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:24.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbea3e00640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbea40d06f0 0x7fbea4156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbea3400640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbea406ee80 0x7fbea4153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbe9aa00640 1 -- 172.21.15.42:0/3895073374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbea406ee80 msgr2=0x7fbea4153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbe9aa00640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbea406ee80 0x7fbea4153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbe9aa00640 1 -- 172.21.15.42:0/3895073374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbea40d06f0 msgr2=0x7fbea4156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:24.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbe9aa00640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbea40d06f0 0x7fbea4156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:24.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbe9aa00640 1 -- 172.21.15.42:0/3895073374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea4072590 con 0x7fbea40af360 2024-05-28T14:16:24.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbea3e00640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbea40d06f0 0x7fbea4156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:24.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.914+0000 7fbea3400640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbea406ee80 0x7fbea4153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:24.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.918+0000 7fbe9aa00640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbea40af360 0x7fbea4071e40 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00d800 tx=0x7fbe8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:24.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.918+0000 7fbe9be00640 1 -- 172.21.15.42:0/3895073374 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c018020 con 0x7fbea40af360 2024-05-28T14:16:24.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.918+0000 7fbe9be00640 1 -- 172.21.15.42:0/3895073374 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe8c0096a0 con 0x7fbea40af360 2024-05-28T14:16:24.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.918+0000 7fbeaa600640 1 -- 172.21.15.42:0/3895073374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea4072870 con 0x7fbea40af360 2024-05-28T14:16:24.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.918+0000 7fbe9be00640 1 -- 172.21.15.42:0/3895073374 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c009a50 con 0x7fbea40af360 2024-05-28T14:16:24.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.918+0000 7fbeaa600640 1 -- 172.21.15.42:0/3895073374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea4072e30 con 0x7fbea40af360 2024-05-28T14:16:24.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.918+0000 7fbeaa600640 1 -- 172.21.15.42:0/3895073374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea4068a90 con 0x7fbea40af360 2024-05-28T14:16:24.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.918+0000 7fbe9be00640 1 -- 172.21.15.42:0/3895073374 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 90894+0+0 (secure 0 0 0) 0x7fbe8c01b020 con 0x7fbea40af360 2024-05-28T14:16:24.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.918+0000 7fbe9be00640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe8006a390 0x7fbe8006c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:24.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.918+0000 7fbe9be00640 1 -- 172.21.15.42:0/3895073374 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fbe8c086060 con 0x7fbea40af360 2024-05-28T14:16:24.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.918+0000 7fbea3400640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe8006a390 0x7fbe8006c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:24.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.922+0000 7fbea3400640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe8006a390 0x7fbe8006c850 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fbe940ae000 tx=0x7fbe940a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:24.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:24.922+0000 7fbe9be00640 1 -- 172.21.15.42:0/3895073374 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbe8c022020 con 0x7fbea40af360 2024-05-28T14:16:25.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.118+0000 7fbeaa600640 1 -- 172.21.15.42:0/3895073374 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone6272", "target": ["mon-mgr", ""]}) -- 0x7fbea41610d0 con 0x7fbe8006a390 2024-05-28T14:16:25.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.142+0000 7fbe9be00640 1 -- 172.21.15.42:0/3895073374 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbea41610d0 con 0x7fbe8006a390 2024-05-28T14:16:25.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.146+0000 7fbeaa600640 1 -- 172.21.15.42:0/3895073374 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe8006a390 msgr2=0x7fbe8006c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:25.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.146+0000 7fbeaa600640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe8006a390 0x7fbe8006c850 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fbe940ae000 tx=0x7fbe940a7070 comp rx=0 tx=0).stop 2024-05-28T14:16:25.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.146+0000 7fbeaa600640 1 -- 172.21.15.42:0/3895073374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbea40af360 msgr2=0x7fbea4071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:25.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.146+0000 7fbeaa600640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbea40af360 0x7fbea4071e40 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00d800 tx=0x7fbe8c00dcc0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.146+0000 7fbeaa600640 1 -- 172.21.15.42:0/3895073374 shutdown_connections 2024-05-28T14:16:25.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.146+0000 7fbeaa600640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbe8006a390 0x7fbe8006c850 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.146+0000 7fbeaa600640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbea40d06f0 0x7fbea4156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.146+0000 7fbeaa600640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbea40af360 0x7fbea4071e40 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.146+0000 7fbeaa600640 1 --2- 172.21.15.42:0/3895073374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbea406ee80 0x7fbea4153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.146+0000 7fbeaa600640 1 -- 172.21.15.42:0/3895073374 >> 172.21.15.42:0/3895073374 conn(0x7fbea406bdf0 msgr2=0x7fbea415a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:25.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.146+0000 7fbeaa600640 1 -- 172.21.15.42:0/3895073374 shutdown_connections 2024-05-28T14:16:25.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.146+0000 7fbeaa600640 1 -- 172.21.15.42:0/3895073374 wait complete. 2024-05-28T14:16:25.153 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-28T14:16:25.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.314+0000 7fdeae000640 1 Processor -- start 2024-05-28T14:16:25.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.314+0000 7fdeae000640 1 -- start start 2024-05-28T14:16:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.314+0000 7fdeae000640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdea815b700 0x7fdea80aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.314+0000 7fdeae000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdea80b4ff0 con 0x7fdea815b320 2024-05-28T14:16:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.314+0000 7fdeae000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdea80b52d0 con 0x7fdea80b4af0 2024-05-28T14:16:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.314+0000 7fdeae000640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdea80b5890 con 0x7fdea815b700 2024-05-28T14:16:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.314+0000 7fdeacc00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fdea815b320 0x7fdea80ae7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:50700/0 (socket says 172.21.15.42:50700) 2024-05-28T14:16:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.314+0000 7fdeacc00640 1 -- 172.21.15.42:0/3214200482 learned_addr learned my addr 172.21.15.42:0/3214200482 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.314+0000 7fde9fe00640 1 --2- 172.21.15.42:0/3214200482 >> v2:172.21.15.63:3300/0 conn(0x7fdea815b700 0x7fdea80aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.314+0000 7fdea7e00640 1 -- 172.21.15.42:0/3214200482 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1025291878 0 0) 0x7fdea80b4ff0 con 0x7fdea815b320 2024-05-28T14:16:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.314+0000 7fdea7e00640 1 -- 172.21.15.42:0/3214200482 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde8c003530 con 0x7fdea815b320 2024-05-28T14:16:25.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.314+0000 7fde9fe00640 1 -- 172.21.15.42:0/3214200482 >> v1:172.21.15.63:6789/0 conn(0x7fdea80b4af0 legacy=0x7fdea80b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:25.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.314+0000 7fde9fe00640 1 -- 172.21.15.42:0/3214200482 >> v1:172.21.15.42:6789/0 conn(0x7fdea815b320 legacy=0x7fdea80ae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:25.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.314+0000 7fde9fe00640 1 -- 172.21.15.42:0/3214200482 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea80b55b0 con 0x7fdea815b700 2024-05-28T14:16:25.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.318+0000 7fde9fe00640 1 --2- 172.21.15.42:0/3214200482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdea815b700 0x7fdea80aeeb0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fdea80b5a10 tx=0x7fde9001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdad85191f485c38 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:25.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.318+0000 7fdea7e00640 1 -- 172.21.15.42:0/3214200482 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde900b0070 con 0x7fdea815b700 2024-05-28T14:16:25.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.318+0000 7fdea7e00640 1 -- 172.21.15.42:0/3214200482 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde900ab600 con 0x7fdea815b700 2024-05-28T14:16:25.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.318+0000 7fdea7e00640 1 -- 172.21.15.42:0/3214200482 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde900ab9d0 con 0x7fdea815b700 2024-05-28T14:16:25.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.318+0000 7fdeae000640 1 -- 172.21.15.42:0/3214200482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdea815b700 msgr2=0x7fdea80aeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:25.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.318+0000 7fdeae000640 1 --2- 172.21.15.42:0/3214200482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdea815b700 0x7fdea80aeeb0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fdea80b5a10 tx=0x7fde9001c410 comp rx=0 tx=0).stop 2024-05-28T14:16:25.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.318+0000 7fdeae000640 1 -- 172.21.15.42:0/3214200482 shutdown_connections 2024-05-28T14:16:25.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.318+0000 7fdeae000640 1 --2- 172.21.15.42:0/3214200482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdea815b700 0x7fdea80aeeb0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.318+0000 7fdeae000640 1 -- 172.21.15.42:0/3214200482 >> 172.21.15.42:0/3214200482 conn(0x7fdea80b6600 msgr2=0x7fdea80b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:25.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.318+0000 7fdeae000640 1 -- 172.21.15.42:0/3214200482 shutdown_connections 2024-05-28T14:16:25.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.318+0000 7fdeae000640 1 -- 172.21.15.42:0/3214200482 wait complete. 2024-05-28T14:16:25.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdeae000640 1 Processor -- start 2024-05-28T14:16:25.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdeae000640 1 -- start start 2024-05-28T14:16:25.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdeae000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdea80b4af0 0x7fdea81481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:25.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdeae000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdea815b320 0x7fdea815bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:25.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdeae000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdea815b700 0x7fdea8154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:25.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdeae000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdea80b2460 con 0x7fdea815b320 2024-05-28T14:16:25.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdeae000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdea80b22e0 con 0x7fdea80b4af0 2024-05-28T14:16:25.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdeae000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdea8169480 con 0x7fdea815b700 2024-05-28T14:16:25.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fde9fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdea815b320 0x7fdea815bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:25.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdeacc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdea80b4af0 0x7fdea81481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:25.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fde9fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdea815b320 0x7fdea815bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34944/0 (socket says 172.21.15.42:34944) 2024-05-28T14:16:25.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fde9fe00640 1 -- 172.21.15.42:0/622406167 learned_addr learned my addr 172.21.15.42:0/622406167 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:25.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdead600640 1 --2- 172.21.15.42:0/622406167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdea815b700 0x7fdea8154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:25.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fde9fe00640 1 -- 172.21.15.42:0/622406167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdea815b700 msgr2=0x7fdea8154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:25.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fde9fe00640 1 --2- 172.21.15.42:0/622406167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdea815b700 0x7fdea8154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fde9fe00640 1 -- 172.21.15.42:0/622406167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdea80b4af0 msgr2=0x7fdea81481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:25.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fde9fe00640 1 --2- 172.21.15.42:0/622406167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdea80b4af0 0x7fdea81481f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fde9fe00640 1 -- 172.21.15.42:0/622406167 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea815f540 con 0x7fdea815b320 2024-05-28T14:16:25.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fde9fe00640 1 --2- 172.21.15.42:0/622406167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdea815b320 0x7fdea815bb10 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fde9001aac0 tx=0x7fde900191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:25.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdea5600640 1 -- 172.21.15.42:0/622406167 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde900b0040 con 0x7fdea815b320 2024-05-28T14:16:25.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdea5600640 1 -- 172.21.15.42:0/622406167 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde900b9070 con 0x7fdea815b320 2024-05-28T14:16:25.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdeae000640 1 -- 172.21.15.42:0/622406167 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea815f820 con 0x7fdea815b320 2024-05-28T14:16:25.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdea5600640 1 -- 172.21.15.42:0/622406167 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90006e20 con 0x7fdea815b320 2024-05-28T14:16:25.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.322+0000 7fdeae000640 1 -- 172.21.15.42:0/622406167 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea815e4e0 con 0x7fdea815b320 2024-05-28T14:16:25.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.326+0000 7fdeae000640 1 -- 172.21.15.42:0/622406167 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea8068a90 con 0x7fdea815b320 2024-05-28T14:16:25.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.326+0000 7fdea5600640 1 -- 172.21.15.42:0/622406167 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 90894+0+0 (secure 0 0 0) 0x7fde900b4650 con 0x7fdea815b320 2024-05-28T14:16:25.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.326+0000 7fdea5600640 1 --2- 172.21.15.42:0/622406167 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fde8406a3e0 0x7fde8406c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:25.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.326+0000 7fdea5600640 1 -- 172.21.15.42:0/622406167 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fde9011f490 con 0x7fdea815b320 2024-05-28T14:16:25.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.326+0000 7fdeacc00640 1 --2- 172.21.15.42:0/622406167 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fde8406a3e0 0x7fde8406c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:25.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.326+0000 7fdeacc00640 1 --2- 172.21.15.42:0/622406167 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fde8406a3e0 0x7fde8406c8a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fde980011f0 tx=0x7fde98007d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:25.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.330+0000 7fdea5600640 1 -- 172.21.15.42:0/622406167 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fde90016090 con 0x7fdea815b320 2024-05-28T14:16:25.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.522+0000 7fdeae000640 1 -- 172.21.15.42:0/622406167 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdea8152ad0 con 0x7fde8406a3e0 2024-05-28T14:16:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.542+0000 7fdea5600640 1 -- 172.21.15.42:0/622406167 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdea8152ad0 con 0x7fde8406a3e0 2024-05-28T14:16:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:25.529 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-05-28T14:16:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-05-28T14:16:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ss1" 2024-05-28T14:16:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-05-28T14:16:25.530 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-05-28T14:16:25.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.542+0000 7fdeae000640 1 -- 172.21.15.42:0/622406167 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fde8406a3e0 msgr2=0x7fde8406c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:25.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.542+0000 7fdeae000640 1 --2- 172.21.15.42:0/622406167 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fde8406a3e0 0x7fde8406c8a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fde980011f0 tx=0x7fde98007d40 comp rx=0 tx=0).stop 2024-05-28T14:16:25.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.542+0000 7fdeae000640 1 -- 172.21.15.42:0/622406167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdea815b320 msgr2=0x7fdea815bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:25.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.542+0000 7fdeae000640 1 --2- 172.21.15.42:0/622406167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdea815b320 0x7fdea815bb10 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fde9001aac0 tx=0x7fde900191b0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.542+0000 7fdeae000640 1 -- 172.21.15.42:0/622406167 shutdown_connections 2024-05-28T14:16:25.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.542+0000 7fdeae000640 1 --2- 172.21.15.42:0/622406167 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fde8406a3e0 0x7fde8406c8a0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.542+0000 7fdeae000640 1 --2- 172.21.15.42:0/622406167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdea815b700 0x7fdea8154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.542+0000 7fdeae000640 1 --2- 172.21.15.42:0/622406167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdea815b320 0x7fdea815bb10 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.542+0000 7fdeae000640 1 --2- 172.21.15.42:0/622406167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdea80b4af0 0x7fdea81481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.542+0000 7fdeae000640 1 -- 172.21.15.42:0/622406167 >> 172.21.15.42:0/622406167 conn(0x7fdea80b6600 msgr2=0x7fdea8156600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:25.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.542+0000 7fdeae000640 1 -- 172.21.15.42:0/622406167 shutdown_connections 2024-05-28T14:16:25.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.542+0000 7fdeae000640 1 -- 172.21.15.42:0/622406167 wait complete. 2024-05-28T14:16:25.550 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:16:25.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f3c00640 1 Processor -- start 2024-05-28T14:16:25.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f3c00640 1 -- start start 2024-05-28T14:16:25.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f3c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f85ec0b02e0 0x7f85ec06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:25.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f3c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec15f710 0x7f85ec06ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:25.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f3c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f85ec06cf40 0x7f85ec154da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:25.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f3c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f85ec06d390 con 0x7f85ec06cf40 2024-05-28T14:16:25.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f3c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f85ec15fb30 con 0x7f85ec0b02e0 2024-05-28T14:16:25.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f3c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f85ec06f190 con 0x7f85ec15f710 2024-05-28T14:16:25.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f1c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f85ec06cf40 0x7f85ec154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:25.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85e3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec15f710 0x7f85ec06ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:25.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f1200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f85ec0b02e0 0x7f85ec06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:25.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85e3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec15f710 0x7f85ec06ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:59178/0 (socket says 172.21.15.42:59178) 2024-05-28T14:16:25.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85e3e00640 1 -- 172.21.15.42:0/3840291399 learned_addr learned my addr 172.21.15.42:0/3840291399 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:25.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f1c00640 1 -- 172.21.15.42:0/3840291399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec15f710 msgr2=0x7f85ec06ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:25.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f1c00640 1 --2- 172.21.15.42:0/3840291399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec15f710 0x7f85ec06ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f1c00640 1 -- 172.21.15.42:0/3840291399 >> v2:172.21.15.63:3300/0 conn(0x7f85ec0b02e0 msgr2=0x7f85ec06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:25.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f1c00640 1 --2- 172.21.15.42:0/3840291399 >> v2:172.21.15.63:3300/0 conn(0x7f85ec0b02e0 0x7f85ec06eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85f1c00640 1 -- 172.21.15.42:0/3840291399 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec06f470 con 0x7f85ec06cf40 2024-05-28T14:16:25.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.710+0000 7f85e3e00640 1 --2- 172.21.15.42:0/3840291399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec15f710 0x7f85ec06ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:16:25.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.714+0000 7f85f1c00640 1 --2- 172.21.15.42:0/3840291399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec06cf40 0x7f85ec154da0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f85ec072220 tx=0x7f85e401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93afea99b611446 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:25.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.714+0000 7f85ebe00640 1 -- 172.21.15.42:0/3840291399 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e40b2070 con 0x7f85ec06cf40 2024-05-28T14:16:25.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.714+0000 7f85ebe00640 1 -- 172.21.15.42:0/3840291399 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e40a5980 con 0x7f85ec06cf40 2024-05-28T14:16:25.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.714+0000 7f85ebe00640 1 -- 172.21.15.42:0/3840291399 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e40a5d50 con 0x7f85ec06cf40 2024-05-28T14:16:25.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.714+0000 7f85f3c00640 1 -- 172.21.15.42:0/3840291399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec06cf40 msgr2=0x7f85ec154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:25.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.714+0000 7f85f3c00640 1 --2- 172.21.15.42:0/3840291399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec06cf40 0x7f85ec154da0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f85ec072220 tx=0x7f85e401c900 comp rx=0 tx=0).stop 2024-05-28T14:16:25.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.714+0000 7f85f3c00640 1 -- 172.21.15.42:0/3840291399 shutdown_connections 2024-05-28T14:16:25.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.714+0000 7f85f3c00640 1 --2- 172.21.15.42:0/3840291399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec06cf40 0x7f85ec154da0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.714+0000 7f85f3c00640 1 --2- 172.21.15.42:0/3840291399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec15f710 0x7f85ec06ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.714+0000 7f85f3c00640 1 --2- 172.21.15.42:0/3840291399 >> v2:172.21.15.63:3300/0 conn(0x7f85ec0b02e0 0x7f85ec06eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.714+0000 7f85f3c00640 1 -- 172.21.15.42:0/3840291399 >> 172.21.15.42:0/3840291399 conn(0x7f85ec072600 msgr2=0x7f85ec072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:25.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.714+0000 7f85f3c00640 1 -- 172.21.15.42:0/3840291399 shutdown_connections 2024-05-28T14:16:25.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.714+0000 7f85f3c00640 1 -- 172.21.15.42:0/3840291399 wait complete. 2024-05-28T14:16:25.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85f3c00640 1 Processor -- start 2024-05-28T14:16:25.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85f3c00640 1 -- start start 2024-05-28T14:16:25.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85f3c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85ec06cf40 0x7f85ec13e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:25.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85f3c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec0b02e0 0x7f85ec0b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:25.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85f3c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec15f710 0x7f85ec15d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:25.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85f3c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec162400 con 0x7f85ec0b02e0 2024-05-28T14:16:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85f3c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec162280 con 0x7f85ec06cf40 2024-05-28T14:16:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85f3c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f85ec162580 con 0x7f85ec15f710 2024-05-28T14:16:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85e3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec0b02e0 0x7f85ec0b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85e3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec0b02e0 0x7f85ec0b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34960/0 (socket says 172.21.15.42:34960) 2024-05-28T14:16:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85e3e00640 1 -- 172.21.15.42:0/3285929314 learned_addr learned my addr 172.21.15.42:0/3285929314 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85f1200640 1 --2- 172.21.15.42:0/3285929314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85ec06cf40 0x7f85ec13e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85f1c00640 1 --2- 172.21.15.42:0/3285929314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec15f710 0x7f85ec15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:25.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85e3e00640 1 -- 172.21.15.42:0/3285929314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec15f710 msgr2=0x7f85ec15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:25.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85e3e00640 1 --2- 172.21.15.42:0/3285929314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85ec15f710 0x7f85ec15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85e3e00640 1 -- 172.21.15.42:0/3285929314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85ec06cf40 msgr2=0x7f85ec13e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:25.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85e3e00640 1 --2- 172.21.15.42:0/3285929314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85ec06cf40 0x7f85ec13e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:25.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85e3e00640 1 -- 172.21.15.42:0/3285929314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec0cd780 con 0x7f85ec0b02e0 2024-05-28T14:16:25.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85e3e00640 1 --2- 172.21.15.42:0/3285929314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85ec0b02e0 0x7f85ec0b10d0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f85d400e930 tx=0x7f85d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:25.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85e9600640 1 -- 172.21.15.42:0/3285929314 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4017070 con 0x7f85ec0b02e0 2024-05-28T14:16:25.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85e9600640 1 -- 172.21.15.42:0/3285929314 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d400ca00 con 0x7f85ec0b02e0 2024-05-28T14:16:25.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85f3c00640 1 -- 172.21.15.42:0/3285929314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ec0cda60 con 0x7f85ec0b02e0 2024-05-28T14:16:25.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85e9600640 1 -- 172.21.15.42:0/3285929314 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d400cdd0 con 0x7f85ec0b02e0 2024-05-28T14:16:25.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.718+0000 7f85f3c00640 1 -- 172.21.15.42:0/3285929314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ec0ce020 con 0x7f85ec0b02e0 2024-05-28T14:16:25.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.722+0000 7f85f3c00640 1 -- 172.21.15.42:0/3285929314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85ec073f40 con 0x7f85ec0b02e0 2024-05-28T14:16:25.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.722+0000 7f85e9600640 1 -- 172.21.15.42:0/3285929314 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 90894+0+0 (secure 0 0 0) 0x7f85d401a070 con 0x7f85ec0b02e0 2024-05-28T14:16:25.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.722+0000 7f85e9600640 1 --2- 172.21.15.42:0/3285929314 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85c806a3c0 0x7f85c806c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.722+0000 7f85e9600640 1 -- 172.21.15.42:0/3285929314 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f85d40858f0 con 0x7f85ec0b02e0 2024-05-28T14:16:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.722+0000 7f85f1200640 1 --2- 172.21.15.42:0/3285929314 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85c806a3c0 0x7f85c806c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.722+0000 7f85f1200640 1 --2- 172.21.15.42:0/3285929314 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f85c806a3c0 0x7f85c806c880 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009ae0 tx=0x7f85dc009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:25.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.726+0000 7f85e9600640 1 -- 172.21.15.42:0/3285929314 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f85d40572c0 con 0x7f85ec0b02e0 2024-05-28T14:16:25.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.922+0000 7f85f3c00640 1 -- 172.21.15.42:0/3285929314 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85ec0d0900 con 0x7f85c806a3c0 2024-05-28T14:16:25.915 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:25.926+0000 7f0774600640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-28T14:16:25.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:25.926+0000 7f85e9600640 1 -- 172.21.15.42:0/3285929314 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7f85ec0d0900 con 0x7f85c806a3c0 2024-05-28T14:16:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:25.916 INFO:teuthology.orchestra.run.smithi042.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-28T14:16:25.925 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-28T14:16:27.926 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:16:28.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.086+0000 7ffa7fe00640 1 Processor -- start 2024-05-28T14:16:28.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa7fe00640 1 -- start start 2024-05-28T14:16:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa7fe00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffa7806d110 0x7ffa7806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa7fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7815f710 0x7ffa78154da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa7fe00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7815fb20 con 0x7ffa780b02e0 2024-05-28T14:16:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa7fe00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffa7806f190 con 0x7ffa7806d110 2024-05-28T14:16:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa7fe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7806f470 con 0x7ffa7815f710 2024-05-28T14:16:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa7d400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ffa780b02e0 0x7ffa7806ca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42716/0 (socket says 172.21.15.42:42716) 2024-05-28T14:16:28.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa7d400640 1 -- 172.21.15.42:0/2315591486 learned_addr learned my addr 172.21.15.42:0/2315591486 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:28.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa7ca00640 1 --2- 172.21.15.42:0/2315591486 >> v2:172.21.15.63:3300/0 conn(0x7ffa7806d110 0x7ffa7806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:28.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa7de00640 1 --2- 172.21.15.42:0/2315591486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7815f710 0x7ffa78154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:28.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa77e00640 1 -- 172.21.15.42:0/2315591486 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 296457607 0 0) 0x7ffa7815fb20 con 0x7ffa780b02e0 2024-05-28T14:16:28.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa77e00640 1 -- 172.21.15.42:0/2315591486 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa60003530 con 0x7ffa780b02e0 2024-05-28T14:16:28.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa7de00640 1 -- 172.21.15.42:0/2315591486 >> v2:172.21.15.63:3300/0 conn(0x7ffa7806d110 msgr2=0x7ffa7806eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:28.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa7de00640 1 --2- 172.21.15.42:0/2315591486 >> v2:172.21.15.63:3300/0 conn(0x7ffa7806d110 0x7ffa7806eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:28.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa7de00640 1 -- 172.21.15.42:0/2315591486 >> v1:172.21.15.42:6789/0 conn(0x7ffa780b02e0 legacy=0x7ffa7806ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:28.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.090+0000 7ffa7de00640 1 -- 172.21.15.42:0/2315591486 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7806f750 con 0x7ffa7815f710 2024-05-28T14:16:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.094+0000 7ffa7de00640 1 --2- 172.21.15.42:0/2315591486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7815f710 0x7ffa78154da0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009d90 tx=0x7ffa7001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be4e92b7b7cd6196 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.094+0000 7ffa77e00640 1 -- 172.21.15.42:0/2315591486 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa700b2070 con 0x7ffa7815f710 2024-05-28T14:16:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.094+0000 7ffa77e00640 1 -- 172.21.15.42:0/2315591486 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa700a5990 con 0x7ffa7815f710 2024-05-28T14:16:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.094+0000 7ffa77e00640 1 -- 172.21.15.42:0/2315591486 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa700a5d60 con 0x7ffa7815f710 2024-05-28T14:16:28.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.094+0000 7ffa7fe00640 1 -- 172.21.15.42:0/2315591486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7815f710 msgr2=0x7ffa78154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:28.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.094+0000 7ffa7fe00640 1 --2- 172.21.15.42:0/2315591486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7815f710 0x7ffa78154da0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009d90 tx=0x7ffa7001c940 comp rx=0 tx=0).stop 2024-05-28T14:16:28.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.094+0000 7ffa7fe00640 1 -- 172.21.15.42:0/2315591486 shutdown_connections 2024-05-28T14:16:28.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.094+0000 7ffa7fe00640 1 --2- 172.21.15.42:0/2315591486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7815f710 0x7ffa78154da0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:28.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.094+0000 7ffa7fe00640 1 --2- 172.21.15.42:0/2315591486 >> v2:172.21.15.63:3300/0 conn(0x7ffa7806d110 0x7ffa7806eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:28.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.094+0000 7ffa7fe00640 1 -- 172.21.15.42:0/2315591486 >> 172.21.15.42:0/2315591486 conn(0x7ffa78072600 msgr2=0x7ffa78072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:28.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.094+0000 7ffa7fe00640 1 -- 172.21.15.42:0/2315591486 shutdown_connections 2024-05-28T14:16:28.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.094+0000 7ffa7fe00640 1 -- 172.21.15.42:0/2315591486 wait complete. 2024-05-28T14:16:28.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.094+0000 7ffa7fe00640 1 Processor -- start 2024-05-28T14:16:28.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.094+0000 7ffa7fe00640 1 -- start start 2024-05-28T14:16:28.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa7806d110 0x7ffa780cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:28.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa780b02e0 0x7ffa781572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:28.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7fe00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7815f710 0x7ffa78153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:28.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffa780b4230 con 0x7ffa7806d110 2024-05-28T14:16:28.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa780b40b0 con 0x7ffa780b02e0 2024-05-28T14:16:28.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7fe00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa780b43b0 con 0x7ffa7815f710 2024-05-28T14:16:28.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa7806d110 0x7ffa780cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:28.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa7806d110 0x7ffa780cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59788/0 (socket says 172.21.15.42:59788) 2024-05-28T14:16:28.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7d400640 1 -- 172.21.15.42:0/3354087110 learned_addr learned my addr 172.21.15.42:0/3354087110 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:28.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7de00640 1 --2- 172.21.15.42:0/3354087110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7815f710 0x7ffa78153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:28.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7ca00640 1 --2- 172.21.15.42:0/3354087110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa780b02e0 0x7ffa781572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:28.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7d400640 1 -- 172.21.15.42:0/3354087110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7815f710 msgr2=0x7ffa78153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:28.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7d400640 1 --2- 172.21.15.42:0/3354087110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7815f710 0x7ffa78153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:28.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7d400640 1 -- 172.21.15.42:0/3354087110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa780b02e0 msgr2=0x7ffa781572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:28.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7d400640 1 --2- 172.21.15.42:0/3354087110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa780b02e0 0x7ffa781572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:28.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7d400640 1 -- 172.21.15.42:0/3354087110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa780cf6b0 con 0x7ffa7806d110 2024-05-28T14:16:28.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7d400640 1 --2- 172.21.15.42:0/3354087110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa7806d110 0x7ffa780cf040 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00c450 tx=0x7ffa6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:28.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa75600640 1 -- 172.21.15.42:0/3354087110 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c017070 con 0x7ffa7806d110 2024-05-28T14:16:28.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa75600640 1 -- 172.21.15.42:0/3354087110 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c005cf0 con 0x7ffa7806d110 2024-05-28T14:16:28.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa75600640 1 -- 172.21.15.42:0/3354087110 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c007290 con 0x7ffa7806d110 2024-05-28T14:16:28.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7fe00640 1 -- 172.21.15.42:0/3354087110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa780cf990 con 0x7ffa7806d110 2024-05-28T14:16:28.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7fe00640 1 -- 172.21.15.42:0/3354087110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa780cff50 con 0x7ffa7806d110 2024-05-28T14:16:28.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.098+0000 7ffa7fe00640 1 -- 172.21.15.42:0/3354087110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa78073f40 con 0x7ffa7806d110 2024-05-28T14:16:28.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.102+0000 7ffa75600640 1 -- 172.21.15.42:0/3354087110 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 49) ==== 90996+0+0 (secure 0 0 0) 0x7ffa6c01a070 con 0x7ffa7806d110 2024-05-28T14:16:28.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.102+0000 7ffa75600640 1 --2- 172.21.15.42:0/3354087110 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa5806a4e0 0x7ffa5806c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:28.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.102+0000 7ffa75600640 1 -- 172.21.15.42:0/3354087110 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7ffa6c084d30 con 0x7ffa7806d110 2024-05-28T14:16:28.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.102+0000 7ffa7ca00640 1 --2- 172.21.15.42:0/3354087110 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa5806a4e0 0x7ffa5806c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:28.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.102+0000 7ffa7ca00640 1 --2- 172.21.15.42:0/3354087110 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ffa5806a4e0 0x7ffa5806c9a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ffa780cd0f0 tx=0x7ffa64009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:28.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.106+0000 7ffa75600640 1 -- 172.21.15.42:0/3354087110 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffa6c0567a0 con 0x7ffa7806d110 2024-05-28T14:16:28.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.306+0000 7ffa7fe00640 1 -- 172.21.15.42:0/3354087110 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa78159080 con 0x7ffa5806a4e0 2024-05-28T14:16:28.298 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:28.310+0000 7f0774600640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-28T14:16:28.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:28.310+0000 7ffa75600640 1 -- 172.21.15.42:0/3354087110 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7ffa78159080 con 0x7ffa5806a4e0 2024-05-28T14:16:28.299 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:28.299 INFO:teuthology.orchestra.run.smithi042.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-28T14:16:28.308 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-28T14:16:30.310 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:16:30.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf79200640 1 Processor -- start 2024-05-28T14:16:30.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf79200640 1 -- start start 2024-05-28T14:16:30.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf79200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcf740b02e0 0x7fcf7406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:30.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf79200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcf7406d110 0x7fcf74154da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:30.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf79200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf7415fb60 con 0x7fcf7415f710 2024-05-28T14:16:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf79200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fcf7406f190 con 0x7fcf7406d110 2024-05-28T14:16:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf79200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcf7406f470 con 0x7fcf740b02e0 2024-05-28T14:16:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf72800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcf7406d110 0x7fcf74154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf72800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcf7406d110 0x7fcf74154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59804/0 (socket says 172.21.15.42:59804) 2024-05-28T14:16:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf72800640 1 -- 172.21.15.42:0/1319815491 learned_addr learned my addr 172.21.15.42:0/1319815491 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf71e00640 1 --2- 172.21.15.42:0/1319815491 >> v2:172.21.15.63:3300/0 conn(0x7fcf740b02e0 0x7fcf7406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf72800640 1 -- 172.21.15.42:0/1319815491 >> v2:172.21.15.63:3300/0 conn(0x7fcf740b02e0 msgr2=0x7fcf7406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf72800640 1 --2- 172.21.15.42:0/1319815491 >> v2:172.21.15.63:3300/0 conn(0x7fcf740b02e0 0x7fcf7406eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf72800640 1 -- 172.21.15.42:0/1319815491 >> v1:172.21.15.63:6789/0 conn(0x7fcf7415f710 legacy=0x7fcf7406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:30.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.482+0000 7fcf72800640 1 -- 172.21.15.42:0/1319815491 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7406f750 con 0x7fcf7406d110 2024-05-28T14:16:30.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.486+0000 7fcf72800640 1 --2- 172.21.15.42:0/1319815491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf7406d110 0x7fcf74154da0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fcf74072220 tx=0x7fcf6c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf73f011ab86676 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:30.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.486+0000 7fcf71400640 1 -- 172.21.15.42:0/1319815491 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0a5810 con 0x7fcf7406d110 2024-05-28T14:16:30.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.486+0000 7fcf71400640 1 -- 172.21.15.42:0/1319815491 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf6c0a59b0 con 0x7fcf7406d110 2024-05-28T14:16:30.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.486+0000 7fcf71400640 1 -- 172.21.15.42:0/1319815491 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0a5d80 con 0x7fcf7406d110 2024-05-28T14:16:30.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.486+0000 7fcf79200640 1 -- 172.21.15.42:0/1319815491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf7406d110 msgr2=0x7fcf74154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:30.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.486+0000 7fcf79200640 1 --2- 172.21.15.42:0/1319815491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf7406d110 0x7fcf74154da0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fcf74072220 tx=0x7fcf6c01c9c0 comp rx=0 tx=0).stop 2024-05-28T14:16:30.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.486+0000 7fcf79200640 1 -- 172.21.15.42:0/1319815491 shutdown_connections 2024-05-28T14:16:30.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.486+0000 7fcf79200640 1 --2- 172.21.15.42:0/1319815491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf7406d110 0x7fcf74154da0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:30.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.486+0000 7fcf79200640 1 --2- 172.21.15.42:0/1319815491 >> v2:172.21.15.63:3300/0 conn(0x7fcf740b02e0 0x7fcf7406eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:30.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.486+0000 7fcf79200640 1 -- 172.21.15.42:0/1319815491 >> 172.21.15.42:0/1319815491 conn(0x7fcf74072600 msgr2=0x7fcf74072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:30.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.486+0000 7fcf79200640 1 -- 172.21.15.42:0/1319815491 shutdown_connections 2024-05-28T14:16:30.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.486+0000 7fcf79200640 1 -- 172.21.15.42:0/1319815491 wait complete. 2024-05-28T14:16:30.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf79200640 1 Processor -- start 2024-05-28T14:16:30.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf79200640 1 -- start start 2024-05-28T14:16:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf79200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf7406d110 0x7fcf740cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf79200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf740b02e0 0x7fcf74151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf79200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf7415f710 0x7fcf741f7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf79200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcf740b4230 con 0x7fcf7406d110 2024-05-28T14:16:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf79200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf740b40b0 con 0x7fcf7415f710 2024-05-28T14:16:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf79200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcf740b43b0 con 0x7fcf740b02e0 2024-05-28T14:16:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf71e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf7406d110 0x7fcf740cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf71e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf7406d110 0x7fcf740cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59810/0 (socket says 172.21.15.42:59810) 2024-05-28T14:16:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf71e00640 1 -- 172.21.15.42:0/3873717618 learned_addr learned my addr 172.21.15.42:0/3873717618 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf72800640 1 --2- 172.21.15.42:0/3873717618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf7415f710 0x7fcf741f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf6be00640 1 --2- 172.21.15.42:0/3873717618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf740b02e0 0x7fcf74151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf71e00640 1 -- 172.21.15.42:0/3873717618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf740b02e0 msgr2=0x7fcf74151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf71e00640 1 --2- 172.21.15.42:0/3873717618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf740b02e0 0x7fcf74151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf71e00640 1 -- 172.21.15.42:0/3873717618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf7415f710 msgr2=0x7fcf741f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf71e00640 1 --2- 172.21.15.42:0/3873717618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf7415f710 0x7fcf741f7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf71e00640 1 -- 172.21.15.42:0/3873717618 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf74153290 con 0x7fcf7406d110 2024-05-28T14:16:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf72800640 1 --2- 172.21.15.42:0/3873717618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf7415f710 0x7fcf741f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf71e00640 1 --2- 172.21.15.42:0/3873717618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf7406d110 0x7fcf740cf880 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400d830 tx=0x7fcf6400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:30.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf6a000640 1 -- 172.21.15.42:0/3873717618 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400b6c0 con 0x7fcf7406d110 2024-05-28T14:16:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf6a000640 1 -- 172.21.15.42:0/3873717618 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf64019070 con 0x7fcf7406d110 2024-05-28T14:16:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf6a000640 1 -- 172.21.15.42:0/3873717618 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400ba60 con 0x7fcf7406d110 2024-05-28T14:16:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf79200640 1 -- 172.21.15.42:0/3873717618 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf74153570 con 0x7fcf7406d110 2024-05-28T14:16:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.490+0000 7fcf79200640 1 -- 172.21.15.42:0/3873717618 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf74153b30 con 0x7fcf7406d110 2024-05-28T14:16:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.494+0000 7fcf79200640 1 -- 172.21.15.42:0/3873717618 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf74073f40 con 0x7fcf7406d110 2024-05-28T14:16:30.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.494+0000 7fcf6a000640 1 -- 172.21.15.42:0/3873717618 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 49) ==== 90996+0+0 (secure 0 0 0) 0x7fcf6401e460 con 0x7fcf7406d110 2024-05-28T14:16:30.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.494+0000 7fcf6a000640 1 --2- 172.21.15.42:0/3873717618 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf5006a4b0 0x7fcf5006c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:30.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.494+0000 7fcf6a000640 1 -- 172.21.15.42:0/3873717618 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fcf640861e0 con 0x7fcf7406d110 2024-05-28T14:16:30.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.494+0000 7fcf6be00640 1 --2- 172.21.15.42:0/3873717618 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf5006a4b0 0x7fcf5006c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:30.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.494+0000 7fcf6be00640 1 --2- 172.21.15.42:0/3873717618 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fcf5006a4b0 0x7fcf5006c970 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fcf7415e010 tx=0x7fcf5c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:30.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.498+0000 7fcf6a000640 1 -- 172.21.15.42:0/3873717618 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcf64058200 con 0x7fcf7406d110 2024-05-28T14:16:30.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.694+0000 7fcf79200640 1 -- 172.21.15.42:0/3873717618 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcf7415dab0 con 0x7fcf5006a4b0 2024-05-28T14:16:30.688 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:30.702+0000 7f0774600640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-28T14:16:30.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:30.702+0000 7fcf6a000640 1 -- 172.21.15.42:0/3873717618 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7fcf7415dab0 con 0x7fcf5006a4b0 2024-05-28T14:16:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:30.689 INFO:teuthology.orchestra.run.smithi042.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-28T14:16:30.697 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-28T14:16:32.699 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:16:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1850c00640 1 Processor -- start 2024-05-28T14:16:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1850c00640 1 -- start start 2024-05-28T14:16:32.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1850c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f184c15c0c0 0x7f184c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:32.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1850c00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f184c0afe40 con 0x7f184c06ee20 2024-05-28T14:16:32.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1850c00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f184c15c600 con 0x7f184c144f80 2024-05-28T14:16:32.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1850c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f184c06f7c0 con 0x7f184c15c0c0 2024-05-28T14:16:32.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1843e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f184c06ee20 0x7f184c15b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:42720/0 (socket says 172.21.15.42:42720) 2024-05-28T14:16:32.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1843e00640 1 -- 172.21.15.42:0/89942058 learned_addr learned my addr 172.21.15.42:0/89942058 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:32.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f184a800640 1 --2- 172.21.15.42:0/89942058 >> v2:172.21.15.63:3300/0 conn(0x7f184c15c0c0 0x7f184c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:32.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 -- 172.21.15.42:0/89942058 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2509685975 0 0) 0x7f184c0afe40 con 0x7f184c06ee20 2024-05-28T14:16:32.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 -- 172.21.15.42:0/89942058 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1830003530 con 0x7f184c06ee20 2024-05-28T14:16:32.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 -- 172.21.15.42:0/89942058 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3846578278 0 0) 0x7f1830003530 con 0x7f184c06ee20 2024-05-28T14:16:32.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 -- 172.21.15.42:0/89942058 >> v2:172.21.15.63:3300/0 conn(0x7f184c15c0c0 msgr2=0x7f184c06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:32.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 --2- 172.21.15.42:0/89942058 >> v2:172.21.15.63:3300/0 conn(0x7f184c15c0c0 0x7f184c06f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:32.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 -- 172.21.15.42:0/89942058 >> v1:172.21.15.63:6789/0 conn(0x7f184c144f80 legacy=0x7f184c072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:32.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 -- 172.21.15.42:0/89942058 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c15c8e0 con 0x7f184c06ee20 2024-05-28T14:16:32.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 -- 172.21.15.42:0/89942058 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f1834001680 con 0x7f184c06ee20 2024-05-28T14:16:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 -- 172.21.15.42:0/89942058 >> v1:172.21.15.42:6789/0 conn(0x7f184c06ee20 legacy=0x7f184c15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 --2- 172.21.15.42:0/89942058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18300039c0 0x7f1830003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 --2- 172.21.15.42:0/89942058 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1830004810 0x7f1830004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 --2- 172.21.15.42:0/89942058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18300051d0 0x7f183000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 -- 172.21.15.42:0/89942058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f183000c040 con 0x7f18300039c0 2024-05-28T14:16:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 -- 172.21.15.42:0/89942058 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f183000c1f0 con 0x7f18300051d0 2024-05-28T14:16:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849400640 1 -- 172.21.15.42:0/89942058 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f183000c3f0 con 0x7f1830004810 2024-05-28T14:16:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1849e00640 1 --2- 172.21.15.42:0/89942058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18300039c0 0x7f1830003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f1843e00640 1 --2- 172.21.15.42:0/89942058 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1830004810 0x7f1830004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.870+0000 7f184a800640 1 --2- 172.21.15.42:0/89942058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18300051d0 0x7f183000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1849e00640 1 -- 172.21.15.42:0/89942058 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1830004810 msgr2=0x7f1830004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1849e00640 1 --2- 172.21.15.42:0/89942058 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1830004810 0x7f1830004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1849e00640 1 -- 172.21.15.42:0/89942058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18300051d0 msgr2=0x7f183000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1849e00640 1 --2- 172.21.15.42:0/89942058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18300051d0 0x7f183000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1849e00640 1 -- 172.21.15.42:0/89942058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1830003530 con 0x7f18300039c0 2024-05-28T14:16:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f184a800640 1 -- 172.21.15.42:0/89942058 reap_dead start 2024-05-28T14:16:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1849e00640 1 --2- 172.21.15.42:0/89942058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18300039c0 0x7f1830003de0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f183c001780 tx=0x7f183c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11515b182321f05a server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:32.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1849400640 1 -- 172.21.15.42:0/89942058 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c0013a0 con 0x7f18300039c0 2024-05-28T14:16:32.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1849400640 1 -- 172.21.15.42:0/89942058 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f183c0052f0 con 0x7f18300039c0 2024-05-28T14:16:32.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1849400640 1 -- 172.21.15.42:0/89942058 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c003840 con 0x7f18300039c0 2024-05-28T14:16:32.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1850c00640 1 -- 172.21.15.42:0/89942058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18300039c0 msgr2=0x7f1830003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:32.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1850c00640 1 --2- 172.21.15.42:0/89942058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18300039c0 0x7f1830003de0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f183c001780 tx=0x7f183c01c400 comp rx=0 tx=0).stop 2024-05-28T14:16:32.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1850c00640 1 -- 172.21.15.42:0/89942058 shutdown_connections 2024-05-28T14:16:32.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1850c00640 1 --2- 172.21.15.42:0/89942058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18300039c0 0x7f1830003de0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:32.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1850c00640 1 -- 172.21.15.42:0/89942058 >> 172.21.15.42:0/89942058 conn(0x7f184c0b21f0 msgr2=0x7f184c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:32.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1850c00640 1 -- 172.21.15.42:0/89942058 shutdown_connections 2024-05-28T14:16:32.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.874+0000 7f1850c00640 1 -- 172.21.15.42:0/89942058 wait complete. 2024-05-28T14:16:32.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1850c00640 1 Processor -- start 2024-05-28T14:16:32.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1850c00640 1 -- start start 2024-05-28T14:16:32.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1850c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18300039c0 0x7f184c144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1850c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f184c136ce0 0x7f184c1ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1850c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f184c1ec050 0x7f184c1ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1850c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f184c06f610 con 0x7f18300039c0 2024-05-28T14:16:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1850c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f184c0724d0 con 0x7f184c1ec050 2024-05-28T14:16:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1850c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f184c0b3800 con 0x7f184c136ce0 2024-05-28T14:16:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1849e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18300039c0 0x7f184c144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1849e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18300039c0 0x7f184c144090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59826/0 (socket says 172.21.15.42:59826) 2024-05-28T14:16:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1849e00640 1 -- 172.21.15.42:0/2089329712 learned_addr learned my addr 172.21.15.42:0/2089329712 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:32.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f184a800640 1 --2- 172.21.15.42:0/2089329712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f184c1ec050 0x7f184c1ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:32.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1843e00640 1 --2- 172.21.15.42:0/2089329712 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f184c136ce0 0x7f184c1ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:32.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1849e00640 1 -- 172.21.15.42:0/2089329712 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f184c136ce0 msgr2=0x7f184c1ebb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:32.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1849e00640 1 --2- 172.21.15.42:0/2089329712 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f184c136ce0 0x7f184c1ebb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:32.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1849e00640 1 -- 172.21.15.42:0/2089329712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f184c1ec050 msgr2=0x7f184c1ec4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:32.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1849e00640 1 --2- 172.21.15.42:0/2089329712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f184c1ec050 0x7f184c1ec4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1849e00640 1 -- 172.21.15.42:0/2089329712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c1eb700 con 0x7f18300039c0 2024-05-28T14:16:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f184a800640 1 --2- 172.21.15.42:0/2089329712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f184c1ec050 0x7f184c1ec4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1843e00640 1 --2- 172.21.15.42:0/2089329712 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f184c136ce0 0x7f184c1ebb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.878+0000 7f1849e00640 1 --2- 172.21.15.42:0/2089329712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18300039c0 0x7f184c144090 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f183c01c230 tx=0x7f1830005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.882+0000 7f1842000640 1 -- 172.21.15.42:0/2089329712 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c007a40 con 0x7f18300039c0 2024-05-28T14:16:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.882+0000 7f1842000640 1 -- 172.21.15.42:0/2089329712 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f183c0ba070 con 0x7f18300039c0 2024-05-28T14:16:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.882+0000 7f1850c00640 1 -- 172.21.15.42:0/2089329712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c1eb220 con 0x7f18300039c0 2024-05-28T14:16:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.882+0000 7f1842000640 1 -- 172.21.15.42:0/2089329712 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c0054a0 con 0x7f18300039c0 2024-05-28T14:16:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.882+0000 7f1850c00640 1 -- 172.21.15.42:0/2089329712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c1eccc0 con 0x7f18300039c0 2024-05-28T14:16:32.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.882+0000 7f1850c00640 1 -- 172.21.15.42:0/2089329712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184c0738b0 con 0x7f18300039c0 2024-05-28T14:16:32.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.882+0000 7f1842000640 1 -- 172.21.15.42:0/2089329712 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 49) ==== 90996+0+0 (secure 0 0 0) 0x7f183c007030 con 0x7f18300039c0 2024-05-28T14:16:32.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.882+0000 7f1842000640 1 --2- 172.21.15.42:0/2089329712 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f182806a4b0 0x7f182806c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:32.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.882+0000 7f1842000640 1 -- 172.21.15.42:0/2089329712 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f183c11faa0 con 0x7f18300039c0 2024-05-28T14:16:32.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.882+0000 7f1843e00640 1 --2- 172.21.15.42:0/2089329712 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f182806a4b0 0x7f182806c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:32.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.886+0000 7f1843e00640 1 --2- 172.21.15.42:0/2089329712 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f182806a4b0 0x7f182806c970 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f184c1ea160 tx=0x7f1834009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:32.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:32.886+0000 7f1842000640 1 -- 172.21.15.42:0/2089329712 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f183c016110 con 0x7f18300039c0 2024-05-28T14:16:33.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:33.086+0000 7f1850c00640 1 -- 172.21.15.42:0/2089329712 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f184c1370c0 con 0x7f182806a4b0 2024-05-28T14:16:33.589 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:33.602+0000 7f0774600640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-28T14:16:33.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:33.602+0000 7f1842000640 1 -- 172.21.15.42:0/2089329712 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7f184c1370c0 con 0x7f182806a4b0 2024-05-28T14:16:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:33.590 INFO:teuthology.orchestra.run.smithi042.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-28T14:16:33.600 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-28T14:16:35.601 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:16:35.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f110dc00640 1 Processor -- start 2024-05-28T14:16:35.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f110dc00640 1 -- start start 2024-05-28T14:16:35.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f110dc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11080d0ae0 0x7f11080d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:35.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f110dc00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f11080d00d0 0x7f1108160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:35.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f110dc00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f11080d0530 0x7f11081607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:35.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f110dc00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f110815dae0 con 0x7f11080d0530 2024-05-28T14:16:35.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f110dc00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f110815cef0 con 0x7f11080d00d0 2024-05-28T14:16:35.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f110dc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f110815d1b0 con 0x7f11080d0ae0 2024-05-28T14:16:35.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f1107400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f11080d0530 0x7f11081607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:35.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f1107400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f11080d0530 0x7f11081607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59838/0 (socket says 172.21.15.42:59838) 2024-05-28T14:16:35.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f1107400640 1 -- 172.21.15.42:0/756135098 learned_addr learned my addr 172.21.15.42:0/756135098 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:35.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f10ffe00640 1 --2- 172.21.15.42:0/756135098 >> v2:172.21.15.63:3300/0 conn(0x7f11080d00d0 0x7f1108160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:35.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f1106a00640 1 --2- 172.21.15.42:0/756135098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11080d0ae0 0x7f11080d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:35.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f1107400640 1 -- 172.21.15.42:0/756135098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11080d0ae0 msgr2=0x7f11080d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:35.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f1107400640 1 --2- 172.21.15.42:0/756135098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11080d0ae0 0x7f11080d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:35.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f1107400640 1 -- 172.21.15.42:0/756135098 >> v2:172.21.15.63:3300/0 conn(0x7f11080d00d0 msgr2=0x7f1108160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:35.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f1107400640 1 --2- 172.21.15.42:0/756135098 >> v2:172.21.15.63:3300/0 conn(0x7f11080d00d0 0x7f1108160270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:35.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.774+0000 7f1107400640 1 -- 172.21.15.42:0/756135098 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110815d490 con 0x7f11080d0530 2024-05-28T14:16:35.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.778+0000 7f1107400640 1 --2- 172.21.15.42:0/756135098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11080d0530 0x7f11081607b0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f1108072ec0 tx=0x7f110001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f07c14e905b3df server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:35.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.778+0000 7f1106000640 1 -- 172.21.15.42:0/756135098 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11000a5670 con 0x7f11080d0530 2024-05-28T14:16:35.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.778+0000 7f1106000640 1 -- 172.21.15.42:0/756135098 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11000a5810 con 0x7f11080d0530 2024-05-28T14:16:35.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.778+0000 7f1106000640 1 -- 172.21.15.42:0/756135098 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11000a5be0 con 0x7f11080d0530 2024-05-28T14:16:35.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.778+0000 7f110dc00640 1 -- 172.21.15.42:0/756135098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11080d0530 msgr2=0x7f11081607b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:35.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.778+0000 7f110dc00640 1 --2- 172.21.15.42:0/756135098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11080d0530 0x7f11081607b0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f1108072ec0 tx=0x7f110001c9f0 comp rx=0 tx=0).stop 2024-05-28T14:16:35.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.778+0000 7f110dc00640 1 -- 172.21.15.42:0/756135098 shutdown_connections 2024-05-28T14:16:35.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.778+0000 7f110dc00640 1 --2- 172.21.15.42:0/756135098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11080d0530 0x7f11081607b0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:35.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.778+0000 7f110dc00640 1 --2- 172.21.15.42:0/756135098 >> v2:172.21.15.63:3300/0 conn(0x7f11080d00d0 0x7f1108160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:35.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.778+0000 7f110dc00640 1 --2- 172.21.15.42:0/756135098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11080d0ae0 0x7f11080d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:35.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.778+0000 7f110dc00640 1 -- 172.21.15.42:0/756135098 >> 172.21.15.42:0/756135098 conn(0x7f11080b3ff0 msgr2=0x7f11080b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:35.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.778+0000 7f110dc00640 1 -- 172.21.15.42:0/756135098 shutdown_connections 2024-05-28T14:16:35.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.778+0000 7f110dc00640 1 -- 172.21.15.42:0/756135098 wait complete. 2024-05-28T14:16:35.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f110dc00640 1 Processor -- start 2024-05-28T14:16:35.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f110dc00640 1 -- start start 2024-05-28T14:16:35.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f110dc00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11080d00d0 0x7f11080c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:35.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f110dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11080d0530 0x7f11081583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:35.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f110dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11080d0ae0 0x7f11081588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:35.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f110dc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f110815ccb0 con 0x7f11080d0ae0 2024-05-28T14:16:35.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f110dc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f110815cb30 con 0x7f11080d0530 2024-05-28T14:16:35.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f110dc00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f11080cd9c0 con 0x7f11080d00d0 2024-05-28T14:16:35.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f1107400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11080d0ae0 0x7f11081588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:35.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f1107400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11080d0ae0 0x7f11081588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59854/0 (socket says 172.21.15.42:59854) 2024-05-28T14:16:35.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f1107400640 1 -- 172.21.15.42:0/1174276540 learned_addr learned my addr 172.21.15.42:0/1174276540 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:35.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f10ffe00640 1 --2- 172.21.15.42:0/1174276540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11080d0530 0x7f11081583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:35.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f1106a00640 1 --2- 172.21.15.42:0/1174276540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11080d00d0 0x7f11080c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:35.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f1107400640 1 -- 172.21.15.42:0/1174276540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11080d00d0 msgr2=0x7f11080c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:35.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f1107400640 1 --2- 172.21.15.42:0/1174276540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11080d00d0 0x7f11080c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:35.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f1107400640 1 -- 172.21.15.42:0/1174276540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11080d0530 msgr2=0x7f11081583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:35.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f1107400640 1 --2- 172.21.15.42:0/1174276540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11080d0530 0x7f11081583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:35.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f1107400640 1 -- 172.21.15.42:0/1174276540 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1108165620 con 0x7f11080d0ae0 2024-05-28T14:16:35.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f1106a00640 1 --2- 172.21.15.42:0/1174276540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11080d00d0 0x7f11080c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:35.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f10ffe00640 1 --2- 172.21.15.42:0/1174276540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11080d0530 0x7f11081583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:35.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f1107400640 1 --2- 172.21.15.42:0/1174276540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11080d0ae0 0x7f11081588f0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f1100009e20 tx=0x7f1100019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:35.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f10fea00640 1 -- 172.21.15.42:0/1174276540 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100009510 con 0x7f11080d0ae0 2024-05-28T14:16:35.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f10fea00640 1 -- 172.21.15.42:0/1174276540 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11000be070 con 0x7f11080d0ae0 2024-05-28T14:16:35.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f110dc00640 1 -- 172.21.15.42:0/1174276540 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1108158f00 con 0x7f11080d0ae0 2024-05-28T14:16:35.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f10fea00640 1 -- 172.21.15.42:0/1174276540 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11000b88b0 con 0x7f11080d0ae0 2024-05-28T14:16:35.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.782+0000 7f110dc00640 1 -- 172.21.15.42:0/1174276540 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11081594c0 con 0x7f11080d0ae0 2024-05-28T14:16:35.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.786+0000 7f110dc00640 1 -- 172.21.15.42:0/1174276540 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11080a0650 con 0x7f11080d0ae0 2024-05-28T14:16:35.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.786+0000 7f10fea00640 1 -- 172.21.15.42:0/1174276540 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 49) ==== 90996+0+0 (secure 0 0 0) 0x7f11000b8a50 con 0x7f11080d0ae0 2024-05-28T14:16:35.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.786+0000 7f10fea00640 1 --2- 172.21.15.42:0/1174276540 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f10e406a4b0 0x7f10e406c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:35.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.786+0000 7f10fea00640 1 -- 172.21.15.42:0/1174276540 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f1100120b90 con 0x7f11080d0ae0 2024-05-28T14:16:35.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.786+0000 7f1106a00640 1 --2- 172.21.15.42:0/1174276540 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f10e406a4b0 0x7f10e406c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:35.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.786+0000 7f1106a00640 1 --2- 172.21.15.42:0/1174276540 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f10e406a4b0 0x7f10e406c970 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009720 tx=0x7f10f8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:35.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.790+0000 7f10fea00640 1 -- 172.21.15.42:0/1174276540 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f11000165e0 con 0x7f11080d0ae0 2024-05-28T14:16:35.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.986+0000 7f110dc00640 1 -- 172.21.15.42:0/1174276540 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f11080d11e0 con 0x7f10e406a4b0 2024-05-28T14:16:35.981 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:35.994+0000 7f0774600640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-28T14:16:35.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:35.994+0000 7f10fea00640 1 -- 172.21.15.42:0/1174276540 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7f11080d11e0 con 0x7f10e406a4b0 2024-05-28T14:16:35.982 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:35.982 INFO:teuthology.orchestra.run.smithi042.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-28T14:16:35.990 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-28T14:16:37.992 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:16:38.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.166+0000 7f87a9200640 1 Processor -- start 2024-05-28T14:16:38.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.166+0000 7f87a9200640 1 -- start start 2024-05-28T14:16:38.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.170+0000 7f87a9200640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f87a406ee20 0x7f87a406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:38.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.170+0000 7f87a9200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f87a415b9b0 0x7f87a415be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:38.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.170+0000 7f87a9200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87a40afe40 con 0x7f87a4144f80 2024-05-28T14:16:38.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.170+0000 7f87a9200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f87a406f7c0 con 0x7f87a415b9b0 2024-05-28T14:16:38.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.170+0000 7f87a9200640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f87a40724d0 con 0x7f87a406ee20 2024-05-28T14:16:38.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.170+0000 7f87a2800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f87a415b9b0 0x7f87a415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:38.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.170+0000 7f87a2800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f87a415b9b0 0x7f87a415be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49698/0 (socket says 172.21.15.42:49698) 2024-05-28T14:16:38.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.170+0000 7f87a2800640 1 -- 172.21.15.42:0/1236234731 learned_addr learned my addr 172.21.15.42:0/1236234731 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:38.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.170+0000 7f879be00640 1 --2- 172.21.15.42:0/1236234731 >> v2:172.21.15.63:3300/0 conn(0x7f87a406ee20 0x7f87a406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:38.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.170+0000 7f87a2800640 1 -- 172.21.15.42:0/1236234731 >> v2:172.21.15.63:3300/0 conn(0x7f87a406ee20 msgr2=0x7f87a406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:38.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.170+0000 7f87a2800640 1 --2- 172.21.15.42:0/1236234731 >> v2:172.21.15.63:3300/0 conn(0x7f87a406ee20 0x7f87a406f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:38.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.170+0000 7f87a2800640 1 -- 172.21.15.42:0/1236234731 >> v1:172.21.15.63:6789/0 conn(0x7f87a4144f80 legacy=0x7f87a4072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:38.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.170+0000 7f87a2800640 1 -- 172.21.15.42:0/1236234731 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a415c340 con 0x7f87a415b9b0 2024-05-28T14:16:38.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.174+0000 7f87a2800640 1 --2- 172.21.15.42:0/1236234731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87a415b9b0 0x7f87a415be00 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f87a4075500 tx=0x7f879c01ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29ce8facc97427e server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:38.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.174+0000 7f87a1400640 1 -- 172.21.15.42:0/1236234731 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c00b2d0 con 0x7f87a415b9b0 2024-05-28T14:16:38.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.174+0000 7f87a1400640 1 -- 172.21.15.42:0/1236234731 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f879c00b470 con 0x7f87a415b9b0 2024-05-28T14:16:38.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.174+0000 7f87a1400640 1 -- 172.21.15.42:0/1236234731 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c0a7aa0 con 0x7f87a415b9b0 2024-05-28T14:16:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.174+0000 7f87a9200640 1 -- 172.21.15.42:0/1236234731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87a415b9b0 msgr2=0x7f87a415be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.174+0000 7f87a9200640 1 --2- 172.21.15.42:0/1236234731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87a415b9b0 0x7f87a415be00 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f87a4075500 tx=0x7f879c01ebb0 comp rx=0 tx=0).stop 2024-05-28T14:16:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.174+0000 7f87a9200640 1 -- 172.21.15.42:0/1236234731 shutdown_connections 2024-05-28T14:16:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.174+0000 7f87a9200640 1 --2- 172.21.15.42:0/1236234731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87a415b9b0 0x7f87a415be00 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.174+0000 7f87a9200640 1 --2- 172.21.15.42:0/1236234731 >> v2:172.21.15.63:3300/0 conn(0x7f87a406ee20 0x7f87a406f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.174+0000 7f87a9200640 1 -- 172.21.15.42:0/1236234731 >> 172.21.15.42:0/1236234731 conn(0x7f87a40b21f0 msgr2=0x7f87a40b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.174+0000 7f87a9200640 1 -- 172.21.15.42:0/1236234731 shutdown_connections 2024-05-28T14:16:38.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.174+0000 7f87a9200640 1 -- 172.21.15.42:0/1236234731 wait complete. 2024-05-28T14:16:38.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.174+0000 7f87a9200640 1 Processor -- start 2024-05-28T14:16:38.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.174+0000 7f87a9200640 1 -- start start 2024-05-28T14:16:38.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a9200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87a406ee20 0x7f87a415d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:38.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a9200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87a4144f80 0x7f87a413fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:38.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a9200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87a415b9b0 0x7f87a413c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:38.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a9200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f87a40b4010 con 0x7f87a415b9b0 2024-05-28T14:16:38.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a9200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f87a4072160 con 0x7f87a406ee20 2024-05-28T14:16:38.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a9200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f87a40b4190 con 0x7f87a4144f80 2024-05-28T14:16:38.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87a415b9b0 0x7f87a413c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:38.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87a415b9b0 0x7f87a413c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49704/0 (socket says 172.21.15.42:49704) 2024-05-28T14:16:38.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a2800640 1 -- 172.21.15.42:0/4110899363 learned_addr learned my addr 172.21.15.42:0/4110899363 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:38.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a1e00640 1 --2- 172.21.15.42:0/4110899363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87a406ee20 0x7f87a415d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:38.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f879be00640 1 --2- 172.21.15.42:0/4110899363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87a4144f80 0x7f87a413fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:38.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a2800640 1 -- 172.21.15.42:0/4110899363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87a4144f80 msgr2=0x7f87a413fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:38.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a2800640 1 --2- 172.21.15.42:0/4110899363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f87a4144f80 0x7f87a413fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:38.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a2800640 1 -- 172.21.15.42:0/4110899363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87a406ee20 msgr2=0x7f87a415d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:38.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a2800640 1 --2- 172.21.15.42:0/4110899363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87a406ee20 0x7f87a415d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:38.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a2800640 1 -- 172.21.15.42:0/4110899363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a415df80 con 0x7f87a415b9b0 2024-05-28T14:16:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a2800640 1 --2- 172.21.15.42:0/4110899363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87a415b9b0 0x7f87a413c770 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f879c0b1040 tx=0x7f879c01b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f879a000640 1 -- 172.21.15.42:0/4110899363 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c009800 con 0x7f87a415b9b0 2024-05-28T14:16:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f879a000640 1 -- 172.21.15.42:0/4110899363 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f879c0be070 con 0x7f87a415b9b0 2024-05-28T14:16:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f879a000640 1 -- 172.21.15.42:0/4110899363 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c009be0 con 0x7f87a415b9b0 2024-05-28T14:16:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a9200640 1 -- 172.21.15.42:0/4110899363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a415e260 con 0x7f87a415b9b0 2024-05-28T14:16:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a9200640 1 -- 172.21.15.42:0/4110899363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a415e820 con 0x7f87a415b9b0 2024-05-28T14:16:38.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.178+0000 7f87a9200640 1 -- 172.21.15.42:0/4110899363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a40738b0 con 0x7f87a415b9b0 2024-05-28T14:16:38.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.182+0000 7f879a000640 1 -- 172.21.15.42:0/4110899363 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f879c0b6070 con 0x7f87a415b9b0 2024-05-28T14:16:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.182+0000 7f879a000640 1 --2- 172.21.15.42:0/4110899363 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f878006a420 0x7f878006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.182+0000 7f879a000640 1 -- 172.21.15.42:0/4110899363 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f879c120870 con 0x7f87a415b9b0 2024-05-28T14:16:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.182+0000 7f87a1e00640 1 --2- 172.21.15.42:0/4110899363 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f878006a420 0x7f878006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:38.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.182+0000 7f87a1e00640 1 --2- 172.21.15.42:0/4110899363 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f878006a420 0x7f878006c8e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f8794007fd0 tx=0x7f8794007e40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:38.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.186+0000 7f879a000640 1 -- 172.21.15.42:0/4110899363 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f879c0187a0 con 0x7f87a415b9b0 2024-05-28T14:16:38.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.382+0000 7f87a9200640 1 -- 172.21.15.42:0/4110899363 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f87a404fdd0 con 0x7f878006a420 2024-05-28T14:16:38.372 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:38.390+0000 7f0774600640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-28T14:16:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:38.390+0000 7f879a000640 1 -- 172.21.15.42:0/4110899363 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7f87a404fdd0 con 0x7f878006a420 2024-05-28T14:16:38.373 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:38.373 INFO:teuthology.orchestra.run.smithi042.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-28T14:16:38.382 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-28T14:16:40.383 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:16:40.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.558+0000 7f09ee400640 1 Processor -- start 2024-05-28T14:16:40.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.558+0000 7f09ee400640 1 -- start start 2024-05-28T14:16:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.558+0000 7f09ee400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09e80d0ef0 0x7f09e80af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.558+0000 7f09ee400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e80cda60 con 0x7f09e815fb20 2024-05-28T14:16:40.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.558+0000 7f09ee400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e80cdd40 con 0x7f09e80b1550 2024-05-28T14:16:40.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.558+0000 7f09ee400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f09e80ce300 con 0x7f09e80d0ef0 2024-05-28T14:16:40.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.558+0000 7f09ed000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f09e815fb20 0x7f09e80cff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55956/0 (socket says 172.21.15.42:55956) 2024-05-28T14:16:40.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.558+0000 7f09ed000640 1 -- 172.21.15.42:0/3097674480 learned_addr learned my addr 172.21.15.42:0/3097674480 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:40.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.558+0000 7f09eda00640 1 --2- 172.21.15.42:0/3097674480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09e80d0ef0 0x7f09e80af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:40.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1875236493 0 0) 0x7f09e80cda60 con 0x7f09e815fb20 2024-05-28T14:16:40.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09cc003530 con 0x7f09e815fb20 2024-05-28T14:16:40.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1587779301 0 0) 0x7f09e80cdd40 con 0x7f09e80b1550 2024-05-28T14:16:40.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09e80cda60 con 0x7f09e80b1550 2024-05-28T14:16:40.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1880289959 0 0) 0x7f09cc003530 con 0x7f09e815fb20 2024-05-28T14:16:40.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09e80d0ef0 msgr2=0x7f09e80af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:40.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 --2- 172.21.15.42:0/3097674480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09e80d0ef0 0x7f09e80af0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:40.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 >> v1:172.21.15.63:6789/0 conn(0x7f09e80b1550 legacy=0x7f09e80d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e80ce020 con 0x7f09e815fb20 2024-05-28T14:16:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f09d8001680 con 0x7f09e815fb20 2024-05-28T14:16:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 >> v1:172.21.15.42:6789/0 conn(0x7f09e815fb20 legacy=0x7f09e80cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 --2- 172.21.15.42:0/3097674480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09cc003b80 0x7f09cc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 --2- 172.21.15.42:0/3097674480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09e80d0ef0 0x7f09cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 --2- 172.21.15.42:0/3097674480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09cc004d50 0x7f09cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f09e80ce300 con 0x7f09cc004d50 2024-05-28T14:16:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc00bbc0 con 0x7f09cc003b80 2024-05-28T14:16:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f09cc00bdc0 con 0x7f09e80d0ef0 2024-05-28T14:16:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09eda00640 1 --2- 172.21.15.42:0/3097674480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09cc004d50 0x7f09cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09ed000640 1 --2- 172.21.15.42:0/3097674480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09cc003b80 0x7f09cc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09dfe00640 1 --2- 172.21.15.42:0/3097674480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09e80d0ef0 0x7f09cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09eda00640 1 -- 172.21.15.42:0/3097674480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09e80d0ef0 msgr2=0x7f09cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09eda00640 1 --2- 172.21.15.42:0/3097674480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09e80d0ef0 0x7f09cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:40.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09eda00640 1 -- 172.21.15.42:0/3097674480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09cc003b80 msgr2=0x7f09cc003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:40.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09eda00640 1 --2- 172.21.15.42:0/3097674480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09cc003b80 0x7f09cc003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:40.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09eda00640 1 -- 172.21.15.42:0/3097674480 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09cc003530 con 0x7f09cc004d50 2024-05-28T14:16:40.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.562+0000 7f09dfe00640 1 --2- 172.21.15.42:0/3097674480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09e80d0ef0 0x7f09cc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:40.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.566+0000 7f09eda00640 1 --2- 172.21.15.42:0/3097674480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09cc004d50 0x7f09cc0051d0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f09e8073290 tx=0x7f09e001cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12b98c325c94800 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:40.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.566+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e00b2070 con 0x7f09cc004d50 2024-05-28T14:16:40.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.566+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e00a5bc0 con 0x7f09cc004d50 2024-05-28T14:16:40.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.566+0000 7f09e7e00640 1 -- 172.21.15.42:0/3097674480 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0009bf0 con 0x7f09cc004d50 2024-05-28T14:16:40.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.566+0000 7f09ee400640 1 -- 172.21.15.42:0/3097674480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09cc004d50 msgr2=0x7f09cc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:40.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.566+0000 7f09ee400640 1 --2- 172.21.15.42:0/3097674480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09cc004d50 0x7f09cc0051d0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f09e8073290 tx=0x7f09e001cc00 comp rx=0 tx=0).stop 2024-05-28T14:16:40.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.566+0000 7f09eda00640 1 -- 172.21.15.42:0/3097674480 reap_dead start 2024-05-28T14:16:40.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.566+0000 7f09ee400640 1 -- 172.21.15.42:0/3097674480 shutdown_connections 2024-05-28T14:16:40.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.566+0000 7f09ee400640 1 -- 172.21.15.42:0/3097674480 >> 172.21.15.42:0/3097674480 conn(0x7f09e80b3e90 msgr2=0x7f09e80b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:40.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.566+0000 7f09ee400640 1 -- 172.21.15.42:0/3097674480 shutdown_connections 2024-05-28T14:16:40.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.566+0000 7f09ee400640 1 -- 172.21.15.42:0/3097674480 wait complete. 2024-05-28T14:16:40.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.566+0000 7f09ee400640 1 Processor -- start 2024-05-28T14:16:40.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.566+0000 7f09ee400640 1 -- start start 2024-05-28T14:16:40.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09ee400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09cc004d50 0x7f09e81498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:40.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09ee400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e806d630 0x7f09e806dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:40.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09ee400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09e8070010 0x7f09e80704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:40.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09ee400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f09e809bd30 con 0x7f09e806d630 2024-05-28T14:16:40.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09ee400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f09e809bbb0 con 0x7f09e8070010 2024-05-28T14:16:40.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09ee400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f09e8168340 con 0x7f09cc004d50 2024-05-28T14:16:40.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09dfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e806d630 0x7f09e806dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:40.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09dfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e806d630 0x7f09e806dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49732/0 (socket says 172.21.15.42:49732) 2024-05-28T14:16:40.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09dfe00640 1 -- 172.21.15.42:0/2968696806 learned_addr learned my addr 172.21.15.42:0/2968696806 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:40.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09ed000640 1 --2- 172.21.15.42:0/2968696806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09cc004d50 0x7f09e81498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:40.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09eda00640 1 --2- 172.21.15.42:0/2968696806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09e8070010 0x7f09e80704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:40.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09dfe00640 1 -- 172.21.15.42:0/2968696806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09cc004d50 msgr2=0x7f09e81498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:40.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09dfe00640 1 --2- 172.21.15.42:0/2968696806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09cc004d50 0x7f09e81498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:40.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09dfe00640 1 -- 172.21.15.42:0/2968696806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09e8070010 msgr2=0x7f09e80704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:40.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09dfe00640 1 --2- 172.21.15.42:0/2968696806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09e8070010 0x7f09e80704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:40.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09dfe00640 1 -- 172.21.15.42:0/2968696806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e8164f30 con 0x7f09e806d630 2024-05-28T14:16:40.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09dfe00640 1 --2- 172.21.15.42:0/2968696806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e806d630 0x7f09e806dab0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f09d00079b0 tx=0x7f09d000ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:40.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09e5600640 1 -- 172.21.15.42:0/2968696806 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d00052e0 con 0x7f09e806d630 2024-05-28T14:16:40.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09e5600640 1 -- 172.21.15.42:0/2968696806 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d0005480 con 0x7f09e806d630 2024-05-28T14:16:40.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09e5600640 1 -- 172.21.15.42:0/2968696806 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0007020 con 0x7f09e806d630 2024-05-28T14:16:40.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09ee400640 1 -- 172.21.15.42:0/2968696806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e8165210 con 0x7f09e806d630 2024-05-28T14:16:40.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09ee400640 1 -- 172.21.15.42:0/2968696806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e81657d0 con 0x7f09e806d630 2024-05-28T14:16:40.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09ee400640 1 -- 172.21.15.42:0/2968696806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e804fcf0 con 0x7f09e806d630 2024-05-28T14:16:40.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.570+0000 7f09e5600640 1 -- 172.21.15.42:0/2968696806 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f09d001a020 con 0x7f09e806d630 2024-05-28T14:16:40.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.574+0000 7f09e5600640 1 --2- 172.21.15.42:0/2968696806 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f09c406a420 0x7f09c406c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:40.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.574+0000 7f09e5600640 1 -- 172.21.15.42:0/2968696806 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f09d0085270 con 0x7f09e806d630 2024-05-28T14:16:40.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.574+0000 7f09ed000640 1 --2- 172.21.15.42:0/2968696806 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f09c406a420 0x7f09c406c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:40.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.574+0000 7f09ed000640 1 --2- 172.21.15.42:0/2968696806 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f09c406a420 0x7f09c406c8e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f09d8000c10 tx=0x7f09d8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:40.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.578+0000 7f09e5600640 1 -- 172.21.15.42:0/2968696806 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f09d0057290 con 0x7f09e806d630 2024-05-28T14:16:40.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.774+0000 7f09ee400640 1 -- 172.21.15.42:0/2968696806 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09e80ce580 con 0x7f09c406a420 2024-05-28T14:16:40.762 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:40.778+0000 7f0774600640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-28T14:16:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:40.778+0000 7f09e5600640 1 -- 172.21.15.42:0/2968696806 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7f09e80ce580 con 0x7f09c406a420 2024-05-28T14:16:40.763 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:40.763 INFO:teuthology.orchestra.run.smithi042.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-28T14:16:40.771 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-28T14:16:42.773 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:16:42.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.945+0000 7f9d15e00640 1 Processor -- start 2024-05-28T14:16:42.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.945+0000 7f9d15e00640 1 -- start start 2024-05-28T14:16:42.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d15e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9d100ce3b0 0x7f9d100ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:42.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d15e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d1013fd10 0x7f9d101401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:42.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d15e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d10160230 con 0x7f9d10071de0 2024-05-28T14:16:42.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d15e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9d10160510 con 0x7f9d100ce3b0 2024-05-28T14:16:42.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d15e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9d101607f0 con 0x7f9d1013fd10 2024-05-28T14:16:42.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d07e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9d100ce3b0 0x7f9d100ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:42.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d07e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9d100ce3b0 0x7f9d100ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49746/0 (socket says 172.21.15.42:49746) 2024-05-28T14:16:42.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d07e00640 1 -- 172.21.15.42:0/318980210 learned_addr learned my addr 172.21.15.42:0/318980210 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:42.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d0f400640 1 --2- 172.21.15.42:0/318980210 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d1013fd10 0x7f9d101401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:42.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d07e00640 1 -- 172.21.15.42:0/318980210 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d1013fd10 msgr2=0x7f9d101401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:42.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d07e00640 1 --2- 172.21.15.42:0/318980210 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d1013fd10 0x7f9d101401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:42.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d07e00640 1 -- 172.21.15.42:0/318980210 >> v1:172.21.15.63:6789/0 conn(0x7f9d10071de0 legacy=0x7f9d100721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:42.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d07e00640 1 -- 172.21.15.42:0/318980210 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d10160ad0 con 0x7f9d100ce3b0 2024-05-28T14:16:42.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d0f400640 1 --2- 172.21.15.42:0/318980210 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d1013fd10 0x7f9d101401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:42.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.949+0000 7f9d07e00640 1 --2- 172.21.15.42:0/318980210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d100ce3b0 0x7f9d100ce810 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009870 tx=0x7f9cf801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8426add77e25d9e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:42.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d0e000640 1 -- 172.21.15.42:0/318980210 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf80b0070 con 0x7f9d100ce3b0 2024-05-28T14:16:42.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d0e000640 1 -- 172.21.15.42:0/318980210 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf80a4e50 con 0x7f9d100ce3b0 2024-05-28T14:16:42.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d0e000640 1 -- 172.21.15.42:0/318980210 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf80ab820 con 0x7f9d100ce3b0 2024-05-28T14:16:42.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 -- 172.21.15.42:0/318980210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d100ce3b0 msgr2=0x7f9d100ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:42.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 --2- 172.21.15.42:0/318980210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d100ce3b0 0x7f9d100ce810 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009870 tx=0x7f9cf801c400 comp rx=0 tx=0).stop 2024-05-28T14:16:42.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 -- 172.21.15.42:0/318980210 shutdown_connections 2024-05-28T14:16:42.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 --2- 172.21.15.42:0/318980210 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d1013fd10 0x7f9d101401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:42.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 --2- 172.21.15.42:0/318980210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d100ce3b0 0x7f9d100ce810 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:42.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 -- 172.21.15.42:0/318980210 >> 172.21.15.42:0/318980210 conn(0x7f9d100b21f0 msgr2=0x7f9d100b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:42.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 -- 172.21.15.42:0/318980210 shutdown_connections 2024-05-28T14:16:42.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 -- 172.21.15.42:0/318980210 wait complete. 2024-05-28T14:16:42.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 Processor -- start 2024-05-28T14:16:42.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 -- start start 2024-05-28T14:16:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d10071de0 0x7f9d101375f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d100ce3b0 0x7f9d1013cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d1013fd10 0x7f9d101394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10169830 con 0x7f9d100ce3b0 2024-05-28T14:16:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9d101696b0 con 0x7f9d1013fd10 2024-05-28T14:16:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.953+0000 7f9d15e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9d101699b0 con 0x7f9d10071de0 2024-05-28T14:16:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d07e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d100ce3b0 0x7f9d1013cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:42.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d07e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d100ce3b0 0x7f9d1013cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49748/0 (socket says 172.21.15.42:49748) 2024-05-28T14:16:42.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d07e00640 1 -- 172.21.15.42:0/4172949901 learned_addr learned my addr 172.21.15.42:0/4172949901 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:42.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d0ea00640 1 --2- 172.21.15.42:0/4172949901 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d10071de0 0x7f9d101375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:42.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d0f400640 1 --2- 172.21.15.42:0/4172949901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d1013fd10 0x7f9d101394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:42.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d07e00640 1 -- 172.21.15.42:0/4172949901 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d10071de0 msgr2=0x7f9d101375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:42.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d07e00640 1 --2- 172.21.15.42:0/4172949901 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d10071de0 0x7f9d101375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:42.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d07e00640 1 -- 172.21.15.42:0/4172949901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d1013fd10 msgr2=0x7f9d101394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:42.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d07e00640 1 --2- 172.21.15.42:0/4172949901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d1013fd10 0x7f9d101394f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:42.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d07e00640 1 -- 172.21.15.42:0/4172949901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d10138730 con 0x7f9d100ce3b0 2024-05-28T14:16:42.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d0f400640 1 --2- 172.21.15.42:0/4172949901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d1013fd10 0x7f9d101394f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:42.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d07e00640 1 --2- 172.21.15.42:0/4172949901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d100ce3b0 0x7f9d1013cd80 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80ae040 tx=0x7f9cf801ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:42.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d07400640 1 -- 172.21.15.42:0/4172949901 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf80ba020 con 0x7f9d100ce3b0 2024-05-28T14:16:42.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d07400640 1 -- 172.21.15.42:0/4172949901 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf80b6420 con 0x7f9d100ce3b0 2024-05-28T14:16:42.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d15e00640 1 -- 172.21.15.42:0/4172949901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d10138a10 con 0x7f9d100ce3b0 2024-05-28T14:16:42.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d07400640 1 -- 172.21.15.42:0/4172949901 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf80b0040 con 0x7f9d100ce3b0 2024-05-28T14:16:42.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d15e00640 1 -- 172.21.15.42:0/4172949901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d10138fd0 con 0x7f9d100ce3b0 2024-05-28T14:16:42.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d15e00640 1 -- 172.21.15.42:0/4172949901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d100738b0 con 0x7f9d100ce3b0 2024-05-28T14:16:42.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.957+0000 7f9d07400640 1 -- 172.21.15.42:0/4172949901 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f9cf80bd020 con 0x7f9d100ce3b0 2024-05-28T14:16:42.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.961+0000 7f9d07400640 1 --2- 172.21.15.42:0/4172949901 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9cec06a450 0x7f9cec06c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:42.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.961+0000 7f9d07400640 1 -- 172.21.15.42:0/4172949901 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f9cf811fcb0 con 0x7f9d100ce3b0 2024-05-28T14:16:42.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.961+0000 7f9d0ea00640 1 --2- 172.21.15.42:0/4172949901 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9cec06a450 0x7f9cec06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:42.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.961+0000 7f9d0ea00640 1 --2- 172.21.15.42:0/4172949901 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9cec06a450 0x7f9cec06c910 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000b810 tx=0x7f9d00008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:42.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:42.965+0000 7f9d07400640 1 -- 172.21.15.42:0/4172949901 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9cf812a020 con 0x7f9d100ce3b0 2024-05-28T14:16:43.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:43.165+0000 7f9d15e00640 1 -- 172.21.15.42:0/4172949901 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d1004fdd0 con 0x7f9cec06a450 2024-05-28T14:16:43.155 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:43.169+0000 7f0774600640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-28T14:16:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:43.169+0000 7f9d07400640 1 -- 172.21.15.42:0/4172949901 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7f9d1004fdd0 con 0x7f9cec06a450 2024-05-28T14:16:43.155 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-28T14:16:43.163 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-28T14:16:45.164 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:16:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.337+0000 7f18a7000640 1 Processor -- start 2024-05-28T14:16:45.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.337+0000 7f18a7000640 1 -- start start 2024-05-28T14:16:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.341+0000 7f18a7000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f18a00b4eb0 0x7f18a006eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.341+0000 7f18a7000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a006f190 con 0x7f18a00b02e0 2024-05-28T14:16:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.341+0000 7f18a7000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a006f470 con 0x7f18a006d110 2024-05-28T14:16:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.341+0000 7f18a7000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f18a006f750 con 0x7f18a00b4eb0 2024-05-28T14:16:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.341+0000 7f18a5000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f18a00b4eb0 0x7f18a006eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.341+0000 7f189fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f18a00b02e0 0x7f18a006ca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55968/0 (socket says 172.21.15.42:55968) 2024-05-28T14:16:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.341+0000 7f189fe00640 1 -- 172.21.15.42:0/1057329493 learned_addr learned my addr 172.21.15.42:0/1057329493 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.341+0000 7f18a5000640 1 -- 172.21.15.42:0/1057329493 >> v1:172.21.15.63:6789/0 conn(0x7f18a006d110 legacy=0x7f18a0154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.341+0000 7f18a5000640 1 -- 172.21.15.42:0/1057329493 >> v1:172.21.15.42:6789/0 conn(0x7f18a00b02e0 legacy=0x7f18a006ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.341+0000 7f18a5000640 1 -- 172.21.15.42:0/1057329493 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a006fa30 con 0x7f18a00b4eb0 2024-05-28T14:16:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.341+0000 7f189f400640 1 -- 172.21.15.42:0/1057329493 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3822535127 0 0) 0x7f18a006f190 con 0x7f18a00b02e0 2024-05-28T14:16:45.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.345+0000 7f18a5000640 1 --2- 172.21.15.42:0/1057329493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a00b4eb0 0x7f18a006eb00 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f18980097f0 tx=0x7f189801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f55922438860261 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:45.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.345+0000 7f189f400640 1 -- 172.21.15.42:0/1057329493 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18980b1070 con 0x7f18a00b4eb0 2024-05-28T14:16:45.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.345+0000 7f189f400640 1 -- 172.21.15.42:0/1057329493 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18980ac630 con 0x7f18a00b4eb0 2024-05-28T14:16:45.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.345+0000 7f189f400640 1 -- 172.21.15.42:0/1057329493 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18980a8050 con 0x7f18a00b4eb0 2024-05-28T14:16:45.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.345+0000 7f18a7000640 1 -- 172.21.15.42:0/1057329493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a00b4eb0 msgr2=0x7f18a006eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:45.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.345+0000 7f18a7000640 1 --2- 172.21.15.42:0/1057329493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a00b4eb0 0x7f18a006eb00 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f18980097f0 tx=0x7f189801c410 comp rx=0 tx=0).stop 2024-05-28T14:16:45.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.345+0000 7f18a7000640 1 -- 172.21.15.42:0/1057329493 shutdown_connections 2024-05-28T14:16:45.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.345+0000 7f18a7000640 1 --2- 172.21.15.42:0/1057329493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a00b4eb0 0x7f18a006eb00 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.345+0000 7f18a7000640 1 -- 172.21.15.42:0/1057329493 >> 172.21.15.42:0/1057329493 conn(0x7f18a0072600 msgr2=0x7f18a0072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.345+0000 7f18a7000640 1 -- 172.21.15.42:0/1057329493 shutdown_connections 2024-05-28T14:16:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.345+0000 7f18a7000640 1 -- 172.21.15.42:0/1057329493 wait complete. 2024-05-28T14:16:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.345+0000 7f18a7000640 1 Processor -- start 2024-05-28T14:16:45.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.345+0000 7f18a7000640 1 -- start start 2024-05-28T14:16:45.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f18a7000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a006d110 0x7f18a0143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f18a7000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a00b02e0 0x7f18a00cf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f18a7000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a00b4eb0 0x7f18a0154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f18a7000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f18a016c9c0 con 0x7f18a006d110 2024-05-28T14:16:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f18a7000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18a016c840 con 0x7f18a00b02e0 2024-05-28T14:16:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f18a7000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f18a016cb40 con 0x7f18a00b4eb0 2024-05-28T14:16:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f189fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a006d110 0x7f18a0143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f189fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a006d110 0x7f18a0143110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49766/0 (socket says 172.21.15.42:49766) 2024-05-28T14:16:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f189fe00640 1 -- 172.21.15.42:0/862659100 learned_addr learned my addr 172.21.15.42:0/862659100 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f1897400640 1 --2- 172.21.15.42:0/862659100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a00b02e0 0x7f18a00cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:45.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f18a5000640 1 --2- 172.21.15.42:0/862659100 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a00b4eb0 0x7f18a0154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f189fe00640 1 -- 172.21.15.42:0/862659100 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a00b4eb0 msgr2=0x7f18a0154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f189fe00640 1 --2- 172.21.15.42:0/862659100 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a00b4eb0 0x7f18a0154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f189fe00640 1 -- 172.21.15.42:0/862659100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a00b02e0 msgr2=0x7f18a00cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f189fe00640 1 --2- 172.21.15.42:0/862659100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a00b02e0 0x7f18a00cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f189fe00640 1 -- 172.21.15.42:0/862659100 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a01503a0 con 0x7f18a006d110 2024-05-28T14:16:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f1897400640 1 --2- 172.21.15.42:0/862659100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a00b02e0 0x7f18a00cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f189fe00640 1 --2- 172.21.15.42:0/862659100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a006d110 0x7f18a0143110 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f189000c8c0 tx=0x7f189000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f189cc00640 1 -- 172.21.15.42:0/862659100 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890018070 con 0x7f18a006d110 2024-05-28T14:16:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f189cc00640 1 -- 172.21.15.42:0/862659100 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189000a640 con 0x7f18a006d110 2024-05-28T14:16:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f189cc00640 1 -- 172.21.15.42:0/862659100 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189000aa30 con 0x7f18a006d110 2024-05-28T14:16:45.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f18a7000640 1 -- 172.21.15.42:0/862659100 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a0150680 con 0x7f18a006d110 2024-05-28T14:16:45.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f18a7000640 1 -- 172.21.15.42:0/862659100 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a0150c40 con 0x7f18a006d110 2024-05-28T14:16:45.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f18a7000640 1 -- 172.21.15.42:0/862659100 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a0073f40 con 0x7f18a006d110 2024-05-28T14:16:45.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.349+0000 7f189cc00640 1 -- 172.21.15.42:0/862659100 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f189001b070 con 0x7f18a006d110 2024-05-28T14:16:45.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.353+0000 7f189cc00640 1 --2- 172.21.15.42:0/862659100 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f187c06a420 0x7f187c06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:45.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.353+0000 7f189cc00640 1 -- 172.21.15.42:0/862659100 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f1890085180 con 0x7f18a006d110 2024-05-28T14:16:45.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.353+0000 7f1897400640 1 --2- 172.21.15.42:0/862659100 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f187c06a420 0x7f187c06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:45.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.353+0000 7f1897400640 1 --2- 172.21.15.42:0/862659100 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f187c06a420 0x7f187c06c8e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f18a0150000 tx=0x7f1888009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:45.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.357+0000 7f189cc00640 1 -- 172.21.15.42:0/862659100 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f18900571a0 con 0x7f18a006d110 2024-05-28T14:16:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.553+0000 7f18a7000640 1 -- 172.21.15.42:0/862659100 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18a016a400 con 0x7f187c06a420 2024-05-28T14:16:45.544 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-05-28T14:16:45.561+0000 7f0774600640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-28T14:16:45.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:45.561+0000 7f189cc00640 1 -- 172.21.15.42:0/862659100 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7f18a016a400 con 0x7f187c06a420 2024-05-28T14:16:45.545 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:45.545 INFO:teuthology.orchestra.run.smithi042.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-28T14:16:45.554 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-28T14:16:45.557 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone ... ERROR 2024-05-28T14:16:45.558 INFO:tasks.cephfs_test_runner: 2024-05-28T14:16:45.558 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-28T14:16:45.558 INFO:tasks.cephfs_test_runner:ERROR: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-28T14:16:45.558 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone 2024-05-28T14:16:45.558 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-28T14:16:45.558 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-28T14:16:45.558 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_9ed025fbf4406fb54980a0647506b15895b6d863/qa/tasks/cephfs/test_volumes.py", line 7795, in tearDown 2024-05-28T14:16:45.559 INFO:tasks.cephfs_test_runner: self.loop_cmd_if_eagain('fs subvolume snapshot rm --force ' 2024-05-28T14:16:45.559 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_9ed025fbf4406fb54980a0647506b15895b6d863/qa/tasks/cephfs/test_volumes.py", line 7749, in loop_cmd_if_eagain 2024-05-28T14:16:45.559 INFO:tasks.cephfs_test_runner: raise RuntimeError(f'This cmd failed 10 times: {cmdargs}') 2024-05-28T14:16:45.559 INFO:tasks.cephfs_test_runner:RuntimeError: This cmd failed 10 times: fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:16:45.559 INFO:tasks.cephfs_test_runner: 2024-05-28T14:16:45.559 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-28T14:16:45.559 INFO:tasks.cephfs_test_runner:Ran 8 tests in 1070.390s 2024-05-28T14:16:45.559 INFO:tasks.cephfs_test_runner: 2024-05-28T14:16:45.559 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-05-28T14:16:45.559 INFO:tasks.cephfs_test_runner: 2024-05-28T14:16:45.559 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-28T14:16:45.559 INFO:tasks.cephfs_test_runner:ERROR: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-28T14:16:45.559 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone 2024-05-28T14:16:45.560 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-28T14:16:45.560 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-28T14:16:45.560 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_9ed025fbf4406fb54980a0647506b15895b6d863/qa/tasks/cephfs/test_volumes.py", line 7795, in tearDown 2024-05-28T14:16:45.560 INFO:tasks.cephfs_test_runner: self.loop_cmd_if_eagain('fs subvolume snapshot rm --force ' 2024-05-28T14:16:45.560 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_9ed025fbf4406fb54980a0647506b15895b6d863/qa/tasks/cephfs/test_volumes.py", line 7749, in loop_cmd_if_eagain 2024-05-28T14:16:45.560 INFO:tasks.cephfs_test_runner: raise RuntimeError(f'This cmd failed 10 times: {cmdargs}') 2024-05-28T14:16:45.560 INFO:tasks.cephfs_test_runner:RuntimeError: This cmd failed 10 times: fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-28T14:16:45.560 INFO:tasks.cephfs_test_runner: 2024-05-28T14:16:45.560 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_9ed025fbf4406fb54980a0647506b15895b6d863/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-28T14:16:45.742 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=57155276759542309610d688ad16fc46 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_9ed025fbf4406fb54980a0647506b15895b6d863/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-28T14:16:45.744 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-28T14:16:45.755 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-28T14:16:45.774 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-28T14:16:45.774 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:16:45.774 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:16:45.789 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:16:45.789 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:16:45.840 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:16:46.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f61e00640 1 Processor -- start 2024-05-28T14:16:46.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f61e00640 1 -- start start 2024-05-28T14:16:46.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f61e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f5c0c5560 0x7f2f5c06cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f61e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2f5c0b35d0 0x7f2f5c0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f61e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f5c06d0e0 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f61e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f2f5c0d1340 con 0x7f2f5c0b35d0 2024-05-28T14:16:46.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f61e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2f5c0d0cf0 con 0x7f2f5c0c5560 2024-05-28T14:16:46.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f5b400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f2f5c0ccbe0 0x7f2f5c0d17d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55976/0 (socket says 172.21.15.42:55976) 2024-05-28T14:16:46.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f5b400640 1 -- 172.21.15.42:0/72013399 learned_addr learned my addr 172.21.15.42:0/72013399 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:46.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f5a000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2f5c0b35d0 0x7f2f5c0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f5aa00640 1 --2- 172.21.15.42:0/72013399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f5c0c5560 0x7f2f5c06cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f59600640 1 -- 172.21.15.42:0/72013399 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2897250051 0 0) 0x7f2f5c06d0e0 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f59600640 1 -- 172.21.15.42:0/72013399 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f40003530 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f5a000640 1 -- 172.21.15.42:0/72013399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f5c0c5560 msgr2=0x7f2f5c06cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f5a000640 1 --2- 172.21.15.42:0/72013399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f5c0c5560 0x7f2f5c06cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f5a000640 1 -- 172.21.15.42:0/72013399 >> v1:172.21.15.42:6789/0 conn(0x7f2f5c0ccbe0 legacy=0x7f2f5c0d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f5a000640 1 -- 172.21.15.42:0/72013399 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5c0d0fd0 con 0x7f2f5c0b35d0 2024-05-28T14:16:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.045+0000 7f2f5aa00640 1 --2- 172.21.15.42:0/72013399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f5c0c5560 0x7f2f5c06cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.049+0000 7f2f5a000640 1 --2- 172.21.15.42:0/72013399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c0b35d0 0x7f2f5c0c5940 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f2f480097c0 tx=0x7f2f4801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89863d6bf23e5dd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.049+0000 7f2f59600640 1 -- 172.21.15.42:0/72013399 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480b1070 con 0x7f2f5c0b35d0 2024-05-28T14:16:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.049+0000 7f2f59600640 1 -- 172.21.15.42:0/72013399 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f480ac410 con 0x7f2f5c0b35d0 2024-05-28T14:16:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.049+0000 7f2f59600640 1 -- 172.21.15.42:0/72013399 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480ac7e0 con 0x7f2f5c0b35d0 2024-05-28T14:16:46.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.049+0000 7f2f61e00640 1 -- 172.21.15.42:0/72013399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c0b35d0 msgr2=0x7f2f5c0c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.049+0000 7f2f61e00640 1 --2- 172.21.15.42:0/72013399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c0b35d0 0x7f2f5c0c5940 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f2f480097c0 tx=0x7f2f4801c4a0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.049+0000 7f2f61e00640 1 -- 172.21.15.42:0/72013399 shutdown_connections 2024-05-28T14:16:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.049+0000 7f2f61e00640 1 --2- 172.21.15.42:0/72013399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c0b35d0 0x7f2f5c0c5940 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.049+0000 7f2f61e00640 1 --2- 172.21.15.42:0/72013399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f5c0c5560 0x7f2f5c06cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.049+0000 7f2f61e00640 1 -- 172.21.15.42:0/72013399 >> 172.21.15.42:0/72013399 conn(0x7f2f5c0b4080 msgr2=0x7f2f5c0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.049+0000 7f2f61e00640 1 -- 172.21.15.42:0/72013399 shutdown_connections 2024-05-28T14:16:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.049+0000 7f2f61e00640 1 -- 172.21.15.42:0/72013399 wait complete. 2024-05-28T14:16:46.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.049+0000 7f2f61e00640 1 Processor -- start 2024-05-28T14:16:46.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.049+0000 7f2f61e00640 1 -- start start 2024-05-28T14:16:46.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f61e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f5c0b35d0 0x7f2f5c153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f61e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0c5560 0x7f2f5c152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f61e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c0ccbe0 0x7f2f5c1553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f61e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c0d2060 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f61e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c0d1ee0 con 0x7f2f5c0c5560 2024-05-28T14:16:46.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f61e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2f5c170100 con 0x7f2f5c0b35d0 2024-05-28T14:16:46.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f5b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c0ccbe0 0x7f2f5c1553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f5b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c0ccbe0 0x7f2f5c1553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49776/0 (socket says 172.21.15.42:49776) 2024-05-28T14:16:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f5b400640 1 -- 172.21.15.42:0/2142547151 learned_addr learned my addr 172.21.15.42:0/2142547151 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f5aa00640 1 --2- 172.21.15.42:0/2142547151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f5c0b35d0 0x7f2f5c153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f5a000640 1 --2- 172.21.15.42:0/2142547151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0c5560 0x7f2f5c152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f5b400640 1 -- 172.21.15.42:0/2142547151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f5c0b35d0 msgr2=0x7f2f5c153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f5b400640 1 --2- 172.21.15.42:0/2142547151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f5c0b35d0 0x7f2f5c153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f5b400640 1 -- 172.21.15.42:0/2142547151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0c5560 msgr2=0x7f2f5c152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f5b400640 1 --2- 172.21.15.42:0/2142547151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0c5560 0x7f2f5c152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f5b400640 1 -- 172.21.15.42:0/2142547151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5c155c10 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f5b400640 1 --2- 172.21.15.42:0/2142547151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c0ccbe0 0x7f2f5c1553c0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400cb10 tx=0x7f2f5400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f53400640 1 -- 172.21.15.42:0/2142547151 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54017070 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f53400640 1 -- 172.21.15.42:0/2142547151 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5400ac70 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f61e00640 1 -- 172.21.15.42:0/2142547151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5c155ef0 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f53400640 1 -- 172.21.15.42:0/2142547151 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5400ae60 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f61e00640 1 -- 172.21.15.42:0/2142547151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f5c1564b0 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.053+0000 7f2f61e00640 1 -- 172.21.15.42:0/2142547151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f5c0b5c40 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.057+0000 7f2f53400640 1 -- 172.21.15.42:0/2142547151 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f2f5401a070 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.057+0000 7f2f53400640 1 --2- 172.21.15.42:0/2142547151 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2f3c06a420 0x7f2f3c06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.057+0000 7f2f53400640 1 -- 172.21.15.42:0/2142547151 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f2f54085820 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.057+0000 7f2f5aa00640 1 --2- 172.21.15.42:0/2142547151 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2f3c06a420 0x7f2f3c06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.057+0000 7f2f5aa00640 1 --2- 172.21.15.42:0/2142547151 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2f3c06a420 0x7f2f3c06c8e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009950 tx=0x7f2f4c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:46.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.061+0000 7f2f53400640 1 -- 172.21.15.42:0/2142547151 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2f540571f0 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.229+0000 7f2f61e00640 1 -- 172.21.15.42:0/2142547151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f5c068a90 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.229+0000 7f2f53400640 1 -- 172.21.15.42:0/2142547151 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16747 (secure 0 0 0) 0x7f2f5405b4f0 con 0x7f2f5c0ccbe0 2024-05-28T14:16:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:46.215 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":134,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:15:05.956598+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:14:41.538199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-28T14:14:42.086660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.7","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:16:46.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.233+0000 7f2f61e00640 1 -- 172.21.15.42:0/2142547151 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2f3c06a420 msgr2=0x7f2f3c06c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.233+0000 7f2f61e00640 1 --2- 172.21.15.42:0/2142547151 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2f3c06a420 0x7f2f3c06c8e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009950 tx=0x7f2f4c008040 comp rx=0 tx=0).stop 2024-05-28T14:16:46.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.233+0000 7f2f61e00640 1 -- 172.21.15.42:0/2142547151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c0ccbe0 msgr2=0x7f2f5c1553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.233+0000 7f2f61e00640 1 --2- 172.21.15.42:0/2142547151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c0ccbe0 0x7f2f5c1553c0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400cb10 tx=0x7f2f5400a060 comp rx=0 tx=0).stop 2024-05-28T14:16:46.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.233+0000 7f2f61e00640 1 -- 172.21.15.42:0/2142547151 shutdown_connections 2024-05-28T14:16:46.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.233+0000 7f2f61e00640 1 --2- 172.21.15.42:0/2142547151 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f2f3c06a420 0x7f2f3c06c8e0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.233+0000 7f2f61e00640 1 --2- 172.21.15.42:0/2142547151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f5c0ccbe0 0x7f2f5c1553c0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.233+0000 7f2f61e00640 1 --2- 172.21.15.42:0/2142547151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0c5560 0x7f2f5c152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.233+0000 7f2f61e00640 1 --2- 172.21.15.42:0/2142547151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f5c0b35d0 0x7f2f5c153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.233+0000 7f2f61e00640 1 -- 172.21.15.42:0/2142547151 >> 172.21.15.42:0/2142547151 conn(0x7f2f5c0b4080 msgr2=0x7f2f5c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:46.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.233+0000 7f2f61e00640 1 -- 172.21.15.42:0/2142547151 shutdown_connections 2024-05-28T14:16:46.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.233+0000 7f2f61e00640 1 -- 172.21.15.42:0/2142547151 wait complete. 2024-05-28T14:16:46.237 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T14:14:41.538199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T14:14:42.086660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:16:46.238 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:16:46.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.397+0000 7fb097000640 1 Processor -- start 2024-05-28T14:16:46.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.397+0000 7fb097000640 1 -- start start 2024-05-28T14:16:46.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.397+0000 7fb097000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb090144f80 0x7fb090072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.397+0000 7fb097000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb09015c0c0 0x7fb09006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.397+0000 7fb097000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0900afe40 con 0x7fb09006ee20 2024-05-28T14:16:46.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.397+0000 7fb097000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb09006f7c0 con 0x7fb09015c0c0 2024-05-28T14:16:46.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.397+0000 7fb097000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0900724d0 con 0x7fb090144f80 2024-05-28T14:16:46.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.397+0000 7fb095000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb09015c0c0 0x7fb09006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.397+0000 7fb087400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb09006ee20 0x7fb09015b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55982/0 (socket says 172.21.15.42:55982) 2024-05-28T14:16:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.397+0000 7fb087400640 1 -- 172.21.15.42:0/2919747890 learned_addr learned my addr 172.21.15.42:0/2919747890 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.401+0000 7fb08fe00640 1 --2- 172.21.15.42:0/2919747890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb090144f80 0x7fb090072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.401+0000 7fb095000640 1 -- 172.21.15.42:0/2919747890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb090144f80 msgr2=0x7fb090072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.401+0000 7fb095000640 1 --2- 172.21.15.42:0/2919747890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb090144f80 0x7fb090072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.401+0000 7fb095000640 1 -- 172.21.15.42:0/2919747890 >> v1:172.21.15.42:6789/0 conn(0x7fb09006ee20 legacy=0x7fb09015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.401+0000 7fb095000640 1 -- 172.21.15.42:0/2919747890 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09015c600 con 0x7fb09015c0c0 2024-05-28T14:16:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.401+0000 7fb08f400640 1 -- 172.21.15.42:0/2919747890 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1530759182 0 0) 0x7fb0900afe40 con 0x7fb09006ee20 2024-05-28T14:16:46.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.401+0000 7fb095000640 1 --2- 172.21.15.42:0/2919747890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb09015c0c0 0x7fb09006f280 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fb088009d90 tx=0x7fb08801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221ef5ed34cc03ce server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:46.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.401+0000 7fb08f400640 1 -- 172.21.15.42:0/2919747890 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880b2070 con 0x7fb09015c0c0 2024-05-28T14:16:46.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.401+0000 7fb08f400640 1 -- 172.21.15.42:0/2919747890 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0880a5920 con 0x7fb09015c0c0 2024-05-28T14:16:46.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.401+0000 7fb08f400640 1 -- 172.21.15.42:0/2919747890 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880a5cf0 con 0x7fb09015c0c0 2024-05-28T14:16:46.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.401+0000 7fb097000640 1 -- 172.21.15.42:0/2919747890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb09015c0c0 msgr2=0x7fb09006f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.401+0000 7fb097000640 1 --2- 172.21.15.42:0/2919747890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb09015c0c0 0x7fb09006f280 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fb088009d90 tx=0x7fb08801c970 comp rx=0 tx=0).stop 2024-05-28T14:16:46.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 -- 172.21.15.42:0/2919747890 shutdown_connections 2024-05-28T14:16:46.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 --2- 172.21.15.42:0/2919747890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb09015c0c0 0x7fb09006f280 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 --2- 172.21.15.42:0/2919747890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb090144f80 0x7fb090072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 -- 172.21.15.42:0/2919747890 >> 172.21.15.42:0/2919747890 conn(0x7fb0900b21f0 msgr2=0x7fb0900b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 -- 172.21.15.42:0/2919747890 shutdown_connections 2024-05-28T14:16:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 -- 172.21.15.42:0/2919747890 wait complete. 2024-05-28T14:16:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 Processor -- start 2024-05-28T14:16:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 -- start start 2024-05-28T14:16:46.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb09006ee20 0x7fb090153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb090144f80 0x7fb09013bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb09015c0c0 0x7fb09013a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb090072ea0 con 0x7fb09006ee20 2024-05-28T14:16:46.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0900b4010 con 0x7fb09015c0c0 2024-05-28T14:16:46.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0901539b0 con 0x7fb090144f80 2024-05-28T14:16:46.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb08fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb09006ee20 0x7fb090153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb08fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb09006ee20 0x7fb090153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49790/0 (socket says 172.21.15.42:49790) 2024-05-28T14:16:46.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb08fe00640 1 -- 172.21.15.42:0/3944001512 learned_addr learned my addr 172.21.15.42:0/3944001512 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:46.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb087400640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb090144f80 0x7fb09013bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb095000640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb09015c0c0 0x7fb09013a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb08fe00640 1 -- 172.21.15.42:0/3944001512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb090144f80 msgr2=0x7fb09013bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb08fe00640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb090144f80 0x7fb09013bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb08fe00640 1 -- 172.21.15.42:0/3944001512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb09015c0c0 msgr2=0x7fb09013a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb08fe00640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb09015c0c0 0x7fb09013a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb08fe00640 1 -- 172.21.15.42:0/3944001512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0901544b0 con 0x7fb09006ee20 2024-05-28T14:16:46.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb095000640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb09015c0c0 0x7fb09013a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:46.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb08fe00640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb09006ee20 0x7fb090153470 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fb08000d800 tx=0x7fb08000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:46.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb08cc00640 1 -- 172.21.15.42:0/3944001512 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08000b6c0 con 0x7fb09006ee20 2024-05-28T14:16:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb08cc00640 1 -- 172.21.15.42:0/3944001512 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb080019070 con 0x7fb09006ee20 2024-05-28T14:16:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb097000640 1 -- 172.21.15.42:0/3944001512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb090154790 con 0x7fb09006ee20 2024-05-28T14:16:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.405+0000 7fb08cc00640 1 -- 172.21.15.42:0/3944001512 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08000ba60 con 0x7fb09006ee20 2024-05-28T14:16:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.409+0000 7fb097000640 1 -- 172.21.15.42:0/3944001512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb090154d50 con 0x7fb09006ee20 2024-05-28T14:16:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.409+0000 7fb097000640 1 -- 172.21.15.42:0/3944001512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb090150a20 con 0x7fb09006ee20 2024-05-28T14:16:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.409+0000 7fb08cc00640 1 -- 172.21.15.42:0/3944001512 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7fb08000bc00 con 0x7fb09006ee20 2024-05-28T14:16:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.409+0000 7fb08cc00640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb06c06a420 0x7fb06c06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.409+0000 7fb08cc00640 1 -- 172.21.15.42:0/3944001512 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fb080085fe0 con 0x7fb09006ee20 2024-05-28T14:16:46.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.409+0000 7fb087400640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb06c06a420 0x7fb06c06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.409+0000 7fb087400640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb06c06a420 0x7fb06c06c8e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb090155be0 tx=0x7fb078007b60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:46.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.413+0000 7fb08cc00640 1 -- 172.21.15.42:0/3944001512 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb080058000 con 0x7fb09006ee20 2024-05-28T14:16:46.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.581+0000 7fb097000640 1 -- 172.21.15.42:0/3944001512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb0900738b0 con 0x7fb09006ee20 2024-05-28T14:16:46.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.581+0000 7fb08cc00640 1 -- 172.21.15.42:0/3944001512 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) ==== 93+0+10 (secure 0 0 0) 0x7fb080057900 con 0x7fb09006ee20 2024-05-28T14:16:46.565 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:16:46.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.581+0000 7fb097000640 1 -- 172.21.15.42:0/3944001512 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb06c06a420 msgr2=0x7fb06c06c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.581+0000 7fb097000640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb06c06a420 0x7fb06c06c8e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb090155be0 tx=0x7fb078007b60 comp rx=0 tx=0).stop 2024-05-28T14:16:46.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.581+0000 7fb097000640 1 -- 172.21.15.42:0/3944001512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb09006ee20 msgr2=0x7fb090153470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.581+0000 7fb097000640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb09006ee20 0x7fb090153470 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fb08000d800 tx=0x7fb08000dcc0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.581+0000 7fb097000640 1 -- 172.21.15.42:0/3944001512 shutdown_connections 2024-05-28T14:16:46.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.581+0000 7fb097000640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb06c06a420 0x7fb06c06c8e0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.581+0000 7fb097000640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb09015c0c0 0x7fb09013a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.581+0000 7fb097000640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb090144f80 0x7fb09013bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.581+0000 7fb097000640 1 --2- 172.21.15.42:0/3944001512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb09006ee20 0x7fb090153470 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.581+0000 7fb097000640 1 -- 172.21.15.42:0/3944001512 >> 172.21.15.42:0/3944001512 conn(0x7fb0900b21f0 msgr2=0x7fb0900cd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:46.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.581+0000 7fb097000640 1 -- 172.21.15.42:0/3944001512 shutdown_connections 2024-05-28T14:16:46.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.581+0000 7fb097000640 1 -- 172.21.15.42:0/3944001512 wait complete. 2024-05-28T14:16:46.586 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:16:46.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.745+0000 7f1e01600640 1 Processor -- start 2024-05-28T14:16:46.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.745+0000 7f1e01600640 1 -- start start 2024-05-28T14:16:46.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.745+0000 7f1e01600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1dfc144f80 0x7f1dfc072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.745+0000 7f1e01600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1dfc15c0c0 0x7f1dfc06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.745+0000 7f1e01600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dfc0afe40 con 0x7f1dfc06ee20 2024-05-28T14:16:46.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.745+0000 7f1e01600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1dfc06f7c0 con 0x7f1dfc144f80 2024-05-28T14:16:46.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.745+0000 7f1e01600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1dfc0724d0 con 0x7f1dfc15c0c0 2024-05-28T14:16:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.749+0000 7f1dfb400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1dfc15c0c0 0x7f1dfc06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.749+0000 7f1dfaa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1dfc144f80 0x7f1dfc072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.749+0000 7f1dfaa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1dfc144f80 0x7f1dfc072960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.42:57878/0 (socket says 172.21.15.42:57878) 2024-05-28T14:16:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.749+0000 7f1dfaa00640 1 -- 172.21.15.42:0/1199209360 learned_addr learned my addr 172.21.15.42:0/1199209360 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.749+0000 7f1df2000640 1 --1- 172.21.15.42:0/1199209360 >> v1:172.21.15.63:6789/0 conn(0x7f1dfc06ee20 0x7f1dfc15b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:50410/0 (socket says 172.21.15.42:50410) 2024-05-28T14:16:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.749+0000 7f1dfaa00640 1 -- 172.21.15.42:0/1199209360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1dfc15c0c0 msgr2=0x7f1dfc06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:46.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.749+0000 7f1dfaa00640 1 --2- 172.21.15.42:0/1199209360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1dfc15c0c0 0x7f1dfc06f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.749+0000 7f1dfaa00640 1 -- 172.21.15.42:0/1199209360 >> v1:172.21.15.63:6789/0 conn(0x7f1dfc06ee20 legacy=0x7f1dfc15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.749+0000 7f1dfaa00640 1 -- 172.21.15.42:0/1199209360 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc15c600 con 0x7f1dfc144f80 2024-05-28T14:16:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.749+0000 7f1dfaa00640 1 --2- 172.21.15.42:0/1199209360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1dfc144f80 0x7f1dfc072960 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00bb10 tx=0x7f1dec01ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=833f20003da1922b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:46.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.749+0000 7f1dfa000640 1 -- 172.21.15.42:0/1199209360 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec0b4070 con 0x7f1dfc144f80 2024-05-28T14:16:46.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.749+0000 7f1dfa000640 1 -- 172.21.15.42:0/1199209360 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dec0a7bf0 con 0x7f1dfc144f80 2024-05-28T14:16:46.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.749+0000 7f1dfa000640 1 -- 172.21.15.42:0/1199209360 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec0af440 con 0x7f1dfc144f80 2024-05-28T14:16:46.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 -- 172.21.15.42:0/1199209360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1dfc144f80 msgr2=0x7f1dfc072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 --2- 172.21.15.42:0/1199209360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1dfc144f80 0x7f1dfc072960 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00bb10 tx=0x7f1dec01ec30 comp rx=0 tx=0).stop 2024-05-28T14:16:46.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 -- 172.21.15.42:0/1199209360 shutdown_connections 2024-05-28T14:16:46.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 --2- 172.21.15.42:0/1199209360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1dfc15c0c0 0x7f1dfc06f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 --2- 172.21.15.42:0/1199209360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1dfc144f80 0x7f1dfc072960 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 -- 172.21.15.42:0/1199209360 >> 172.21.15.42:0/1199209360 conn(0x7f1dfc0b21f0 msgr2=0x7f1dfc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:46.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 -- 172.21.15.42:0/1199209360 shutdown_connections 2024-05-28T14:16:46.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 -- 172.21.15.42:0/1199209360 wait complete. 2024-05-28T14:16:46.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 Processor -- start 2024-05-28T14:16:46.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 -- start start 2024-05-28T14:16:46.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dfc06ee20 0x7f1dfc13d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1dfc144f80 0x7f1dfc140730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1dfc15c0c0 0x7f1dfc15a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc072ea0 con 0x7f1dfc06ee20 2024-05-28T14:16:46.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc0b4010 con 0x7f1dfc144f80 2024-05-28T14:16:46.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1e01600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1dfc13e040 con 0x7f1dfc15c0c0 2024-05-28T14:16:46.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1dfaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dfc06ee20 0x7f1dfc13d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1dfaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dfc06ee20 0x7f1dfc13d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49798/0 (socket says 172.21.15.42:49798) 2024-05-28T14:16:46.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1dfaa00640 1 -- 172.21.15.42:0/2761763551 learned_addr learned my addr 172.21.15.42:0/2761763551 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1dfb400640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1dfc15c0c0 0x7f1dfc15a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1df2000640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1dfc144f80 0x7f1dfc140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1dfaa00640 1 -- 172.21.15.42:0/2761763551 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1dfc15c0c0 msgr2=0x7f1dfc15a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1dfaa00640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1dfc15c0c0 0x7f1dfc15a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1dfaa00640 1 -- 172.21.15.42:0/2761763551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1dfc144f80 msgr2=0x7f1dfc140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1dfaa00640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1dfc144f80 0x7f1dfc140730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1dfaa00640 1 -- 172.21.15.42:0/2761763551 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc141550 con 0x7f1dfc06ee20 2024-05-28T14:16:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.753+0000 7f1df2000640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1dfc144f80 0x7f1dfc140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:46.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.757+0000 7f1dfaa00640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dfc06ee20 0x7f1dfc13d940 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f1dec01c860 tx=0x7f1dec01d470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.757+0000 7f1df3400640 1 -- 172.21.15.42:0/2761763551 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec0bf070 con 0x7f1dfc06ee20 2024-05-28T14:16:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.757+0000 7f1df3400640 1 -- 172.21.15.42:0/2761763551 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dec0ba440 con 0x7f1dfc06ee20 2024-05-28T14:16:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.757+0000 7f1df3400640 1 -- 172.21.15.42:0/2761763551 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec0b4070 con 0x7f1dfc06ee20 2024-05-28T14:16:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.757+0000 7f1e01600640 1 -- 172.21.15.42:0/2761763551 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dfc141830 con 0x7f1dfc06ee20 2024-05-28T14:16:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.757+0000 7f1e01600640 1 -- 172.21.15.42:0/2761763551 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dfc15af00 con 0x7f1dfc06ee20 2024-05-28T14:16:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.757+0000 7f1e01600640 1 -- 172.21.15.42:0/2761763551 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dfc0738b0 con 0x7f1dfc06ee20 2024-05-28T14:16:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.757+0000 7f1df3400640 1 -- 172.21.15.42:0/2761763551 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f1dec0c2070 con 0x7f1dfc06ee20 2024-05-28T14:16:46.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.757+0000 7f1df3400640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1dd806a420 0x7f1dd806c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:46.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.757+0000 7f1df3400640 1 -- 172.21.15.42:0/2761763551 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f1dec11fea0 con 0x7f1dfc06ee20 2024-05-28T14:16:46.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.757+0000 7f1df2000640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1dd806a420 0x7f1dd806c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:46.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.761+0000 7f1df2000640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1dd806a420 0x7f1dd806c8e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc13d430 tx=0x7f1de4007d70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:46.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.761+0000 7f1df3400640 1 -- 172.21.15.42:0/2761763551 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1dec018820 con 0x7f1dfc06ee20 2024-05-28T14:16:46.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.925+0000 7f1e01600640 1 -- 172.21.15.42:0/2761763551 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1dfc04fdd0 con 0x7f1dfc06ee20 2024-05-28T14:16:46.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.929+0000 7f1df3400640 1 -- 172.21.15.42:0/2761763551 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) ==== 104+0+11 (secure 0 0 0) 0x7f1dec0f1910 con 0x7f1dfc06ee20 2024-05-28T14:16:46.912 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:16:46.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.929+0000 7f1e01600640 1 -- 172.21.15.42:0/2761763551 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1dd806a420 msgr2=0x7f1dd806c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.929+0000 7f1e01600640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1dd806a420 0x7f1dd806c8e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc13d430 tx=0x7f1de4007d70 comp rx=0 tx=0).stop 2024-05-28T14:16:46.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.929+0000 7f1e01600640 1 -- 172.21.15.42:0/2761763551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dfc06ee20 msgr2=0x7f1dfc13d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:46.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.929+0000 7f1e01600640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dfc06ee20 0x7f1dfc13d940 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f1dec01c860 tx=0x7f1dec01d470 comp rx=0 tx=0).stop 2024-05-28T14:16:46.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.929+0000 7f1e01600640 1 -- 172.21.15.42:0/2761763551 shutdown_connections 2024-05-28T14:16:46.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.929+0000 7f1e01600640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1dd806a420 0x7f1dd806c8e0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.929+0000 7f1e01600640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1dfc15c0c0 0x7f1dfc15a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.929+0000 7f1e01600640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1dfc144f80 0x7f1dfc140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.929+0000 7f1e01600640 1 --2- 172.21.15.42:0/2761763551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dfc06ee20 0x7f1dfc13d940 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:46.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.929+0000 7f1e01600640 1 -- 172.21.15.42:0/2761763551 >> 172.21.15.42:0/2761763551 conn(0x7f1dfc0b21f0 msgr2=0x7f1dfc0b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:46.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.929+0000 7f1e01600640 1 -- 172.21.15.42:0/2761763551 shutdown_connections 2024-05-28T14:16:46.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:46.929+0000 7f1e01600640 1 -- 172.21.15.42:0/2761763551 wait complete. 2024-05-28T14:16:46.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:16:47.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a76600640 1 Processor -- start 2024-05-28T14:16:47.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a76600640 1 -- start start 2024-05-28T14:16:47.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a76600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a70144f80 0x7f7a70072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a76600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7a7015c0c0 0x7f7a7006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a76600640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a700afe40 con 0x7f7a7006ee20 2024-05-28T14:16:47.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a76600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7a7006f7c0 con 0x7f7a7015c0c0 2024-05-28T14:16:47.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a76600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a700724d0 con 0x7f7a70144f80 2024-05-28T14:16:47.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a67e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f7a7006ee20 0x7f7a7015b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:55990/0 (socket says 172.21.15.42:55990) 2024-05-28T14:16:47.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a67e00640 1 -- 172.21.15.42:0/2147187128 learned_addr learned my addr 172.21.15.42:0/2147187128 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:47.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a6f400640 1 --2- 172.21.15.42:0/2147187128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a70144f80 0x7f7a70072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a6fe00640 1 --2- 172.21.15.42:0/2147187128 >> v2:172.21.15.63:3300/0 conn(0x7f7a7015c0c0 0x7f7a7006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a6ea00640 1 -- 172.21.15.42:0/2147187128 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4100471365 0 0) 0x7f7a700afe40 con 0x7f7a7006ee20 2024-05-28T14:16:47.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a6ea00640 1 -- 172.21.15.42:0/2147187128 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a54003530 con 0x7f7a7006ee20 2024-05-28T14:16:47.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a6f400640 1 -- 172.21.15.42:0/2147187128 >> v2:172.21.15.63:3300/0 conn(0x7f7a7015c0c0 msgr2=0x7f7a7006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:47.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a6f400640 1 --2- 172.21.15.42:0/2147187128 >> v2:172.21.15.63:3300/0 conn(0x7f7a7015c0c0 0x7f7a7006f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a6f400640 1 -- 172.21.15.42:0/2147187128 >> v1:172.21.15.42:6789/0 conn(0x7f7a7006ee20 legacy=0x7f7a7015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a6f400640 1 -- 172.21.15.42:0/2147187128 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7015c600 con 0x7f7a70144f80 2024-05-28T14:16:47.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.093+0000 7f7a6fe00640 1 --2- 172.21.15.42:0/2147187128 >> v2:172.21.15.63:3300/0 conn(0x7f7a7015c0c0 0x7f7a7006f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:16:47.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.097+0000 7f7a6f400640 1 --2- 172.21.15.42:0/2147187128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a70144f80 0x7f7a70072960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7a600097c0 tx=0x7f7a6001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd8253e2789cfd25 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:47.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.097+0000 7f7a6ea00640 1 -- 172.21.15.42:0/2147187128 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a600b2070 con 0x7f7a70144f80 2024-05-28T14:16:47.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.097+0000 7f7a6ea00640 1 -- 172.21.15.42:0/2147187128 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a600ad640 con 0x7f7a70144f80 2024-05-28T14:16:47.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.097+0000 7f7a6ea00640 1 -- 172.21.15.42:0/2147187128 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a600ada10 con 0x7f7a70144f80 2024-05-28T14:16:47.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.097+0000 7f7a76600640 1 -- 172.21.15.42:0/2147187128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a70144f80 msgr2=0x7f7a70072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.097+0000 7f7a76600640 1 --2- 172.21.15.42:0/2147187128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a70144f80 0x7f7a70072960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7a600097c0 tx=0x7f7a6001c4a0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.097+0000 7f7a76600640 1 -- 172.21.15.42:0/2147187128 shutdown_connections 2024-05-28T14:16:47.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.097+0000 7f7a76600640 1 --2- 172.21.15.42:0/2147187128 >> v2:172.21.15.63:3300/0 conn(0x7f7a7015c0c0 0x7f7a7006f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.097+0000 7f7a76600640 1 --2- 172.21.15.42:0/2147187128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a70144f80 0x7f7a70072960 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.097+0000 7f7a76600640 1 -- 172.21.15.42:0/2147187128 >> 172.21.15.42:0/2147187128 conn(0x7f7a700b21f0 msgr2=0x7f7a700b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:47.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.097+0000 7f7a76600640 1 -- 172.21.15.42:0/2147187128 shutdown_connections 2024-05-28T14:16:47.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.097+0000 7f7a76600640 1 -- 172.21.15.42:0/2147187128 wait complete. 2024-05-28T14:16:47.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a76600640 1 Processor -- start 2024-05-28T14:16:47.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a76600640 1 -- start start 2024-05-28T14:16:47.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a76600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a7006ee20 0x7f7a70155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a76600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a70144f80 0x7f7a7015f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a76600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a7015c0c0 0x7f7a7015e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a76600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70072ea0 con 0x7f7a70144f80 2024-05-28T14:16:47.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a76600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7a700b4010 con 0x7f7a7006ee20 2024-05-28T14:16:47.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a76600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a7013e9c0 con 0x7f7a7015c0c0 2024-05-28T14:16:47.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a67e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a70144f80 0x7f7a7015f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a67e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a70144f80 0x7f7a7015f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49804/0 (socket says 172.21.15.42:49804) 2024-05-28T14:16:47.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a67e00640 1 -- 172.21.15.42:0/2478748912 learned_addr learned my addr 172.21.15.42:0/2478748912 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:47.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a6f400640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a7006ee20 0x7f7a70155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a6fe00640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a7015c0c0 0x7f7a7015e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a67e00640 1 -- 172.21.15.42:0/2478748912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a7015c0c0 msgr2=0x7f7a7015e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a67e00640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a7015c0c0 0x7f7a7015e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a67e00640 1 -- 172.21.15.42:0/2478748912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a7006ee20 msgr2=0x7f7a70155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a67e00640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a7006ee20 0x7f7a70155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a67e00640 1 -- 172.21.15.42:0/2478748912 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a700b57e0 con 0x7f7a70144f80 2024-05-28T14:16:47.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a6fe00640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a7015c0c0 0x7f7a7015e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:47.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a67e00640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a70144f80 0x7f7a7015f0e0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800c450 tx=0x7f7a5800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:47.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a67400640 1 -- 172.21.15.42:0/2478748912 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58017070 con 0x7f7a70144f80 2024-05-28T14:16:47.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a67400640 1 -- 172.21.15.42:0/2478748912 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a58005cf0 con 0x7f7a70144f80 2024-05-28T14:16:47.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a76600640 1 -- 172.21.15.42:0/2478748912 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7013e6e0 con 0x7f7a70144f80 2024-05-28T14:16:47.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a67400640 1 -- 172.21.15.42:0/2478748912 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58007290 con 0x7f7a70144f80 2024-05-28T14:16:47.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.101+0000 7f7a76600640 1 -- 172.21.15.42:0/2478748912 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7015de00 con 0x7f7a70144f80 2024-05-28T14:16:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.105+0000 7f7a76600640 1 -- 172.21.15.42:0/2478748912 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a700738b0 con 0x7f7a70144f80 2024-05-28T14:16:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.105+0000 7f7a67400640 1 -- 172.21.15.42:0/2478748912 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f7a5801a070 con 0x7f7a70144f80 2024-05-28T14:16:47.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.105+0000 7f7a67400640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a4c06a420 0x7f7a4c06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.105+0000 7f7a67400640 1 -- 172.21.15.42:0/2478748912 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f7a58084ec0 con 0x7f7a70144f80 2024-05-28T14:16:47.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.105+0000 7f7a6f400640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a4c06a420 0x7f7a4c06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.105+0000 7f7a6f400640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a4c06a420 0x7f7a4c06c8e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7a600af000 tx=0x7f7a600a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:47.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.109+0000 7f7a67400640 1 -- 172.21.15.42:0/2478748912 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7a58056ee0 con 0x7f7a70144f80 2024-05-28T14:16:47.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.273+0000 7f7a76600640 1 -- 172.21.15.42:0/2478748912 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7a7015bb60 con 0x7f7a70144f80 2024-05-28T14:16:47.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.277+0000 7f7a67400640 1 -- 172.21.15.42:0/2478748912 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) ==== 100+0+11 (secure 0 0 0) 0x7f7a580567e0 con 0x7f7a70144f80 2024-05-28T14:16:47.260 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:16:47.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.277+0000 7f7a76600640 1 -- 172.21.15.42:0/2478748912 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a4c06a420 msgr2=0x7f7a4c06c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.277+0000 7f7a76600640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a4c06a420 0x7f7a4c06c8e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7a600af000 tx=0x7f7a600a7070 comp rx=0 tx=0).stop 2024-05-28T14:16:47.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.277+0000 7f7a76600640 1 -- 172.21.15.42:0/2478748912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a70144f80 msgr2=0x7f7a7015f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.277+0000 7f7a76600640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a70144f80 0x7f7a7015f0e0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800c450 tx=0x7f7a5800c910 comp rx=0 tx=0).stop 2024-05-28T14:16:47.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.277+0000 7f7a76600640 1 -- 172.21.15.42:0/2478748912 shutdown_connections 2024-05-28T14:16:47.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.277+0000 7f7a76600640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a4c06a420 0x7f7a4c06c8e0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.277+0000 7f7a76600640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a7015c0c0 0x7f7a7015e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.277+0000 7f7a76600640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a70144f80 0x7f7a7015f0e0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.277+0000 7f7a76600640 1 --2- 172.21.15.42:0/2478748912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a7006ee20 0x7f7a70155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.277+0000 7f7a76600640 1 -- 172.21.15.42:0/2478748912 >> 172.21.15.42:0/2478748912 conn(0x7f7a700b21f0 msgr2=0x7f7a700b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:47.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.277+0000 7f7a76600640 1 -- 172.21.15.42:0/2478748912 shutdown_connections 2024-05-28T14:16:47.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.277+0000 7f7a76600640 1 -- 172.21.15.42:0/2478748912 wait complete. 2024-05-28T14:16:47.282 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T14:16:47.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e1b200640 1 Processor -- start 2024-05-28T14:16:47.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e1b200640 1 -- start start 2024-05-28T14:16:47.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e1b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e1c1628b0 0x7f0e1c15fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e1b200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0e1c06ef60 0x7f0e1c06f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e1b200640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1c06fce0 con 0x7f0e1c161d50 2024-05-28T14:16:47.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e1b200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1c0b5430 con 0x7f0e1c06ef60 2024-05-28T14:16:47.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e1b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c0b5710 con 0x7f0e1c1628b0 2024-05-28T14:16:47.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e1a800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0e1c06ef60 0x7f0e1c06f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e1a800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0e1c06ef60 0x7f0e1c06f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37820/0 (socket says 172.21.15.42:37820) 2024-05-28T14:16:47.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e1a800640 1 -- 172.21.15.42:0/642817395 learned_addr learned my addr 172.21.15.42:0/642817395 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:47.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e11400640 1 --2- 172.21.15.42:0/642817395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e1c1628b0 0x7f0e1c15fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e1a800640 1 -- 172.21.15.42:0/642817395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e1c1628b0 msgr2=0x7f0e1c15fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:47.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e1a800640 1 --2- 172.21.15.42:0/642817395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e1c1628b0 0x7f0e1c15fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e1a800640 1 -- 172.21.15.42:0/642817395 >> v1:172.21.15.63:6789/0 conn(0x7f0e1c161d50 legacy=0x7f0e1c1621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e1a800640 1 -- 172.21.15.42:0/642817395 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c0b59f0 con 0x7f0e1c06ef60 2024-05-28T14:16:47.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.441+0000 7f0e11400640 1 --2- 172.21.15.42:0/642817395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e1c1628b0 0x7f0e1c15fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:47.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.445+0000 7f0e1a800640 1 --2- 172.21.15.42:0/642817395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c06ef60 0x7f0e1c06f3b0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009870 tx=0x7f0e1401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4a8bfe3d32305d server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:47.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.445+0000 7f0e19400640 1 -- 172.21.15.42:0/642817395 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e140aa680 con 0x7f0e1c06ef60 2024-05-28T14:16:47.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.445+0000 7f0e19400640 1 -- 172.21.15.42:0/642817395 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e140af070 con 0x7f0e1c06ef60 2024-05-28T14:16:47.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.445+0000 7f0e19400640 1 -- 172.21.15.42:0/642817395 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e140aaa40 con 0x7f0e1c06ef60 2024-05-28T14:16:47.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.445+0000 7f0e1b200640 1 -- 172.21.15.42:0/642817395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c06ef60 msgr2=0x7f0e1c06f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.445+0000 7f0e1b200640 1 --2- 172.21.15.42:0/642817395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c06ef60 0x7f0e1c06f3b0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009870 tx=0x7f0e1401c500 comp rx=0 tx=0).stop 2024-05-28T14:16:47.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.445+0000 7f0e1b200640 1 -- 172.21.15.42:0/642817395 shutdown_connections 2024-05-28T14:16:47.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.445+0000 7f0e1b200640 1 --2- 172.21.15.42:0/642817395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c06ef60 0x7f0e1c06f3b0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.445+0000 7f0e1b200640 1 --2- 172.21.15.42:0/642817395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e1c1628b0 0x7f0e1c15fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.445+0000 7f0e1b200640 1 -- 172.21.15.42:0/642817395 >> 172.21.15.42:0/642817395 conn(0x7f0e1c073370 msgr2=0x7f0e1c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:47.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.445+0000 7f0e1b200640 1 -- 172.21.15.42:0/642817395 shutdown_connections 2024-05-28T14:16:47.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.445+0000 7f0e1b200640 1 -- 172.21.15.42:0/642817395 wait complete. 2024-05-28T14:16:47.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e1b200640 1 Processor -- start 2024-05-28T14:16:47.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e1b200640 1 -- start start 2024-05-28T14:16:47.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e1b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c06ef60 0x7f0e1c1595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e1b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e1c161d50 0x7f0e1c15d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e1b200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e1c1628b0 0x7f0e1c0b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e1b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c0b6ab0 con 0x7f0e1c06ef60 2024-05-28T14:16:47.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e1b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c0b6930 con 0x7f0e1c161d50 2024-05-28T14:16:47.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e1b200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c0b6c30 con 0x7f0e1c1628b0 2024-05-28T14:16:47.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e19e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c06ef60 0x7f0e1c1595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e19e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c06ef60 0x7f0e1c1595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37836/0 (socket says 172.21.15.42:37836) 2024-05-28T14:16:47.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e19e00640 1 -- 172.21.15.42:0/3798175030 learned_addr learned my addr 172.21.15.42:0/3798175030 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:47.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e19e00640 1 -- 172.21.15.42:0/3798175030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e1c1628b0 msgr2=0x7f0e1c0b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e19e00640 1 --2- 172.21.15.42:0/3798175030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e1c1628b0 0x7f0e1c0b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e19e00640 1 -- 172.21.15.42:0/3798175030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e1c161d50 msgr2=0x7f0e1c15d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e19e00640 1 --2- 172.21.15.42:0/3798175030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e1c161d50 0x7f0e1c15d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e19e00640 1 -- 172.21.15.42:0/3798175030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c15df00 con 0x7f0e1c06ef60 2024-05-28T14:16:47.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e19e00640 1 --2- 172.21.15.42:0/3798175030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c06ef60 0x7f0e1c1595c0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00c370 tx=0x7f0e0c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:47.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e12a00640 1 -- 172.21.15.42:0/3798175030 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0052d0 con 0x7f0e1c06ef60 2024-05-28T14:16:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e12a00640 1 -- 172.21.15.42:0/3798175030 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0c005470 con 0x7f0e1c06ef60 2024-05-28T14:16:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e1b200640 1 -- 172.21.15.42:0/3798175030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c13e660 con 0x7f0e1c06ef60 2024-05-28T14:16:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e12a00640 1 -- 172.21.15.42:0/3798175030 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c007260 con 0x7f0e1c06ef60 2024-05-28T14:16:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e1b200640 1 -- 172.21.15.42:0/3798175030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c13ec00 con 0x7f0e1c06ef60 2024-05-28T14:16:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.449+0000 7f0e1b200640 1 -- 172.21.15.42:0/3798175030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e1c04fcf0 con 0x7f0e1c06ef60 2024-05-28T14:16:47.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.453+0000 7f0e12a00640 1 -- 172.21.15.42:0/3798175030 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f0e0c007400 con 0x7f0e1c06ef60 2024-05-28T14:16:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.453+0000 7f0e12a00640 1 --2- 172.21.15.42:0/3798175030 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0df806a450 0x7f0df806c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.453+0000 7f0e12a00640 1 -- 172.21.15.42:0/3798175030 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f0e0c084f40 con 0x7f0e1c06ef60 2024-05-28T14:16:47.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.453+0000 7f0e11400640 1 --2- 172.21.15.42:0/3798175030 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0df806a450 0x7f0df806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.453+0000 7f0e11400640 1 --2- 172.21.15.42:0/3798175030 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0df806a450 0x7f0df806c910 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0709b0 tx=0x7f0e040094a0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:47.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.457+0000 7f0e12a00640 1 -- 172.21.15.42:0/3798175030 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0e0c056f60 con 0x7f0e1c06ef60 2024-05-28T14:16:47.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.641+0000 7f0e1b200640 1 -- 172.21.15.42:0/3798175030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0e1c067a70 con 0x7f0e1c06ef60 2024-05-28T14:16:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.641+0000 7f0e12a00640 1 -- 172.21.15.42:0/3798175030 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v134) ==== 80+0+0 (secure 0 0 0) 0x7f0e0c056860 con 0x7f0e1c06ef60 2024-05-28T14:16:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T14:16:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.641+0000 7f0e1b200640 1 -- 172.21.15.42:0/3798175030 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0df806a450 msgr2=0x7f0df806c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.641+0000 7f0e1b200640 1 --2- 172.21.15.42:0/3798175030 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0df806a450 0x7f0df806c910 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0709b0 tx=0x7f0e040094a0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.641+0000 7f0e1b200640 1 -- 172.21.15.42:0/3798175030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c06ef60 msgr2=0x7f0e1c1595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.641+0000 7f0e1b200640 1 --2- 172.21.15.42:0/3798175030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c06ef60 0x7f0e1c1595c0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00c370 tx=0x7f0e0c00c830 comp rx=0 tx=0).stop 2024-05-28T14:16:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.641+0000 7f0e1b200640 1 -- 172.21.15.42:0/3798175030 shutdown_connections 2024-05-28T14:16:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.641+0000 7f0e1b200640 1 --2- 172.21.15.42:0/3798175030 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f0df806a450 0x7f0df806c910 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.641+0000 7f0e1b200640 1 --2- 172.21.15.42:0/3798175030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e1c1628b0 0x7f0e1c0b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.641+0000 7f0e1b200640 1 --2- 172.21.15.42:0/3798175030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e1c161d50 0x7f0e1c15d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.641+0000 7f0e1b200640 1 --2- 172.21.15.42:0/3798175030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e1c06ef60 0x7f0e1c1595c0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.641+0000 7f0e1b200640 1 -- 172.21.15.42:0/3798175030 >> 172.21.15.42:0/3798175030 conn(0x7f0e1c073370 msgr2=0x7f0e1c0749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.641+0000 7f0e1b200640 1 -- 172.21.15.42:0/3798175030 shutdown_connections 2024-05-28T14:16:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.641+0000 7f0e1b200640 1 -- 172.21.15.42:0/3798175030 wait complete. 2024-05-28T14:16:47.648 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:16:47.648 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:16:47.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:16:47.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.857+0000 7fc9d7000640 1 Processor -- start 2024-05-28T14:16:47.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.857+0000 7fc9d7000640 1 -- start start 2024-05-28T14:16:47.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.857+0000 7fc9d7000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc9d00d0240 0x7fc9d00b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.857+0000 7fc9d7000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9d01631f0 con 0x7fc9d00cf390 2024-05-28T14:16:47.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.857+0000 7fc9d7000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9d0162f10 con 0x7fc9d0161d80 2024-05-28T14:16:47.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.857+0000 7fc9d7000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fc9d00b3770 con 0x7fc9d00d0240 2024-05-28T14:16:47.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.857+0000 7fc9d6600640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc9d00d0240 0x7fc9d00b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.857+0000 7fc9d5200640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fc9d00cf390 0x7fc9d009aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:52976/0 (socket says 172.21.15.42:52976) 2024-05-28T14:16:47.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.857+0000 7fc9d5200640 1 -- 172.21.15.42:0/168074180 learned_addr learned my addr 172.21.15.42:0/168074180 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.857+0000 7fc9cfe00640 1 -- 172.21.15.42:0/168074180 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3380559559 0 0) 0x7fc9d01631f0 con 0x7fc9d00cf390 2024-05-28T14:16:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.857+0000 7fc9d6600640 1 -- 172.21.15.42:0/168074180 >> v1:172.21.15.63:6789/0 conn(0x7fc9d0161d80 legacy=0x7fc9d00cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.857+0000 7fc9d6600640 1 -- 172.21.15.42:0/168074180 >> v1:172.21.15.42:6789/0 conn(0x7fc9d00cf390 legacy=0x7fc9d009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.857+0000 7fc9d6600640 1 -- 172.21.15.42:0/168074180 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d00b3a50 con 0x7fc9d00d0240 2024-05-28T14:16:47.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.861+0000 7fc9d6600640 1 --2- 172.21.15.42:0/168074180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9d00d0240 0x7fc9d00b4a50 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009d90 tx=0x7fc9c801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641ba39e94b9163f server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:47.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.861+0000 7fc9cfe00640 1 -- 172.21.15.42:0/168074180 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80b2070 con 0x7fc9d00d0240 2024-05-28T14:16:47.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.861+0000 7fc9cfe00640 1 -- 172.21.15.42:0/168074180 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c80a58b0 con 0x7fc9d00d0240 2024-05-28T14:16:47.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.861+0000 7fc9cfe00640 1 -- 172.21.15.42:0/168074180 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80a5c80 con 0x7fc9d00d0240 2024-05-28T14:16:47.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.861+0000 7fc9d7000640 1 -- 172.21.15.42:0/168074180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9d00d0240 msgr2=0x7fc9d00b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.861+0000 7fc9d7000640 1 --2- 172.21.15.42:0/168074180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9d00d0240 0x7fc9d00b4a50 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009d90 tx=0x7fc9c801c8e0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.861+0000 7fc9d7000640 1 -- 172.21.15.42:0/168074180 shutdown_connections 2024-05-28T14:16:47.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.861+0000 7fc9d7000640 1 --2- 172.21.15.42:0/168074180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9d00d0240 0x7fc9d00b4a50 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.861+0000 7fc9d7000640 1 -- 172.21.15.42:0/168074180 >> 172.21.15.42:0/168074180 conn(0x7fc9d00b7a30 msgr2=0x7fc9d00b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:47.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.861+0000 7fc9d7000640 1 -- 172.21.15.42:0/168074180 shutdown_connections 2024-05-28T14:16:47.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.861+0000 7fc9d7000640 1 -- 172.21.15.42:0/168074180 wait complete. 2024-05-28T14:16:47.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.861+0000 7fc9d7000640 1 Processor -- start 2024-05-28T14:16:47.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.861+0000 7fc9d7000640 1 -- start start 2024-05-28T14:16:47.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d7000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9d00cf390 0x7fc9d0161160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d7000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9d00d0240 0x7fc9d00ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d7000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9d0161d80 0x7fc9d00afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d7000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d016d240 con 0x7fc9d00d0240 2024-05-28T14:16:47.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d7000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d016d0c0 con 0x7fc9d00cf390 2024-05-28T14:16:47.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d7000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d016d3c0 con 0x7fc9d0161d80 2024-05-28T14:16:47.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9d00d0240 0x7fc9d00ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9d00d0240 0x7fc9d00ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37846/0 (socket says 172.21.15.42:37846) 2024-05-28T14:16:47.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d5200640 1 -- 172.21.15.42:0/2477207010 learned_addr learned my addr 172.21.15.42:0/2477207010 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:47.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d5c00640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9d00cf390 0x7fc9d0161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d6600640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9d0161d80 0x7fc9d00afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d5200640 1 -- 172.21.15.42:0/2477207010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9d0161d80 msgr2=0x7fc9d00afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d5200640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9d0161d80 0x7fc9d00afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d5200640 1 -- 172.21.15.42:0/2477207010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9d00cf390 msgr2=0x7fc9d0161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:47.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d5200640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9d00cf390 0x7fc9d0161160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:47.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d5200640 1 -- 172.21.15.42:0/2477207010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d00ce5f0 con 0x7fc9d00d0240 2024-05-28T14:16:47.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d5c00640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9d00cf390 0x7fc9d0161160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:47.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d6600640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9d0161d80 0x7fc9d00afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:47.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d5200640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9d00d0240 0x7fc9d00ce0b0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00c440 tx=0x7fc9bc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:47.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9cd600640 1 -- 172.21.15.42:0/2477207010 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc0074e0 con 0x7fc9d00d0240 2024-05-28T14:16:47.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9cd600640 1 -- 172.21.15.42:0/2477207010 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9bc0047f0 con 0x7fc9d00d0240 2024-05-28T14:16:47.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9cd600640 1 -- 172.21.15.42:0/2477207010 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc007680 con 0x7fc9d00d0240 2024-05-28T14:16:47.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d7000640 1 -- 172.21.15.42:0/2477207010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d0158cb0 con 0x7fc9d00d0240 2024-05-28T14:16:47.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d7000640 1 -- 172.21.15.42:0/2477207010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d00cd8b0 con 0x7fc9d00d0240 2024-05-28T14:16:47.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9d7000640 1 -- 172.21.15.42:0/2477207010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d004fcf0 con 0x7fc9d00d0240 2024-05-28T14:16:47.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.865+0000 7fc9cd600640 1 -- 172.21.15.42:0/2477207010 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7fc9bc011070 con 0x7fc9d00d0240 2024-05-28T14:16:47.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.869+0000 7fc9cd600640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9b006a420 0x7fc9b006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:47.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.869+0000 7fc9cd600640 1 -- 172.21.15.42:0/2477207010 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fc9bc085620 con 0x7fc9d00d0240 2024-05-28T14:16:47.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.869+0000 7fc9d5c00640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9b006a420 0x7fc9b006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:47.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.869+0000 7fc9d5c00640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9b006a420 0x7fc9b006c8e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00b7960 tx=0x7fc9c4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:47.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:47.873+0000 7fc9cd600640 1 -- 172.21.15.42:0/2477207010 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc9bc057640 con 0x7fc9d00d0240 2024-05-28T14:16:48.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.037+0000 7fc9d7000640 1 -- 172.21.15.42:0/2477207010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9d00cfb30 con 0x7fc9d00d0240 2024-05-28T14:16:48.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.037+0000 7fc9cd600640 1 -- 172.21.15.42:0/2477207010 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16747 (secure 0 0 0) 0x7fc9bc056f40 con 0x7fc9d00d0240 2024-05-28T14:16:48.023 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:48.024 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":134,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:15:05.956598+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:14:41.538199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-28T14:14:42.086660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.7","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:16:48.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.041+0000 7fc9d7000640 1 -- 172.21.15.42:0/2477207010 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9b006a420 msgr2=0x7fc9b006c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.041+0000 7fc9d7000640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9b006a420 0x7fc9b006c8e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00b7960 tx=0x7fc9c4009040 comp rx=0 tx=0).stop 2024-05-28T14:16:48.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.041+0000 7fc9d7000640 1 -- 172.21.15.42:0/2477207010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9d00d0240 msgr2=0x7fc9d00ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.041+0000 7fc9d7000640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9d00d0240 0x7fc9d00ce0b0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00c440 tx=0x7fc9bc00c900 comp rx=0 tx=0).stop 2024-05-28T14:16:48.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.041+0000 7fc9d7000640 1 -- 172.21.15.42:0/2477207010 shutdown_connections 2024-05-28T14:16:48.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.041+0000 7fc9d7000640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fc9b006a420 0x7fc9b006c8e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.041+0000 7fc9d7000640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9d0161d80 0x7fc9d00afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.041+0000 7fc9d7000640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9d00d0240 0x7fc9d00ce0b0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.041+0000 7fc9d7000640 1 --2- 172.21.15.42:0/2477207010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9d00cf390 0x7fc9d0161160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.041+0000 7fc9d7000640 1 -- 172.21.15.42:0/2477207010 >> 172.21.15.42:0/2477207010 conn(0x7fc9d00b7a30 msgr2=0x7fc9d0070620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:48.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.041+0000 7fc9d7000640 1 -- 172.21.15.42:0/2477207010 shutdown_connections 2024-05-28T14:16:48.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.041+0000 7fc9d7000640 1 -- 172.21.15.42:0/2477207010 wait complete. 2024-05-28T14:16:48.045 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-28T13:58:10.881461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-28T14:14:41.538199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-05-28T14:14:42.086660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-28T14:16:48.045 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-28T14:16:48.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72e8400640 1 Processor -- start 2024-05-28T14:16:48.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72e8400640 1 -- start start 2024-05-28T14:16:48.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72e8400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72e015c0c0 0x7f72e006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72e8400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72e00afe40 con 0x7f72e006ee20 2024-05-28T14:16:48.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72e8400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72e015c600 con 0x7f72e0144f80 2024-05-28T14:16:48.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72e8400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f72e006f7c0 con 0x7f72e015c0c0 2024-05-28T14:16:48.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72e5000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f72e006ee20 0x7f72e015b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:52992/0 (socket says 172.21.15.42:52992) 2024-05-28T14:16:48.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72e5000640 1 -- 172.21.15.42:0/2929536768 learned_addr learned my addr 172.21.15.42:0/2929536768 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:48.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72e6400640 1 --2- 172.21.15.42:0/2929536768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72e015c0c0 0x7f72e006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72dfe00640 1 -- 172.21.15.42:0/2929536768 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 236876416 0 0) 0x7f72e00afe40 con 0x7f72e006ee20 2024-05-28T14:16:48.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72dfe00640 1 -- 172.21.15.42:0/2929536768 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72c8003530 con 0x7f72e006ee20 2024-05-28T14:16:48.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72dfe00640 1 -- 172.21.15.42:0/2929536768 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 989891032 0 0) 0x7f72e015c600 con 0x7f72e0144f80 2024-05-28T14:16:48.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72dfe00640 1 -- 172.21.15.42:0/2929536768 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72e00afe40 con 0x7f72e0144f80 2024-05-28T14:16:48.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72e6400640 1 -- 172.21.15.42:0/2929536768 >> v1:172.21.15.63:6789/0 conn(0x7f72e0144f80 legacy=0x7f72e0072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72e6400640 1 -- 172.21.15.42:0/2929536768 >> v1:172.21.15.42:6789/0 conn(0x7f72e006ee20 legacy=0x7f72e015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.209+0000 7f72e6400640 1 -- 172.21.15.42:0/2929536768 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e015c8e0 con 0x7f72e015c0c0 2024-05-28T14:16:48.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.213+0000 7f72e6400640 1 --2- 172.21.15.42:0/2929536768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72e015c0c0 0x7f72e006f280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f72d80097f0 tx=0x7f72d801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5c817ab0ff6b3cb server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:48.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.213+0000 7f72dfe00640 1 -- 172.21.15.42:0/2929536768 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d80b0070 con 0x7f72e015c0c0 2024-05-28T14:16:48.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.213+0000 7f72dfe00640 1 -- 172.21.15.42:0/2929536768 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d80ab600 con 0x7f72e015c0c0 2024-05-28T14:16:48.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.213+0000 7f72dfe00640 1 -- 172.21.15.42:0/2929536768 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d80ab9d0 con 0x7f72e015c0c0 2024-05-28T14:16:48.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.213+0000 7f72e8400640 1 -- 172.21.15.42:0/2929536768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72e015c0c0 msgr2=0x7f72e006f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.213+0000 7f72e8400640 1 --2- 172.21.15.42:0/2929536768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72e015c0c0 0x7f72e006f280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f72d80097f0 tx=0x7f72d801c4e0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.213+0000 7f72e8400640 1 -- 172.21.15.42:0/2929536768 shutdown_connections 2024-05-28T14:16:48.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.213+0000 7f72e8400640 1 --2- 172.21.15.42:0/2929536768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72e015c0c0 0x7f72e006f280 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.213+0000 7f72e8400640 1 -- 172.21.15.42:0/2929536768 >> 172.21.15.42:0/2929536768 conn(0x7f72e00b21f0 msgr2=0x7f72e00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:48.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.213+0000 7f72e8400640 1 -- 172.21.15.42:0/2929536768 shutdown_connections 2024-05-28T14:16:48.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.213+0000 7f72e8400640 1 -- 172.21.15.42:0/2929536768 wait complete. 2024-05-28T14:16:48.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e8400640 1 Processor -- start 2024-05-28T14:16:48.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e8400640 1 -- start start 2024-05-28T14:16:48.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e8400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72e006ee20 0x7f72e00aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e8400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72e0144f80 0x7f72e015cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e8400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e015c0c0 0x7f72e013edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e8400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f72e00724d0 con 0x7f72e015c0c0 2024-05-28T14:16:48.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e8400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72e00b42e0 con 0x7f72e006ee20 2024-05-28T14:16:48.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e8400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f72e006dd10 con 0x7f72e0144f80 2024-05-28T14:16:48.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e6400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e015c0c0 0x7f72e013edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e6400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e015c0c0 0x7f72e013edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37854/0 (socket says 172.21.15.42:37854) 2024-05-28T14:16:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e5a00640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72e006ee20 0x7f72e00aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e6400640 1 -- 172.21.15.42:0/4055226078 learned_addr learned my addr 172.21.15.42:0/4055226078 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e5000640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72e0144f80 0x7f72e015cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e6400640 1 -- 172.21.15.42:0/4055226078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72e0144f80 msgr2=0x7f72e015cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e6400640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72e0144f80 0x7f72e015cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e6400640 1 -- 172.21.15.42:0/4055226078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72e006ee20 msgr2=0x7f72e00aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e6400640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72e006ee20 0x7f72e00aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e6400640 1 -- 172.21.15.42:0/4055226078 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e01583b0 con 0x7f72e015c0c0 2024-05-28T14:16:48.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e5a00640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72e006ee20 0x7f72e00aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:16:48.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e6400640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e015c0c0 0x7f72e013edc0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f72d80097f0 tx=0x7f72d8019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72dd600640 1 -- 172.21.15.42:0/4055226078 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d80b0040 con 0x7f72e015c0c0 2024-05-28T14:16:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72dd600640 1 -- 172.21.15.42:0/4055226078 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d80ba070 con 0x7f72e015c0c0 2024-05-28T14:16:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72dd600640 1 -- 172.21.15.42:0/4055226078 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d80b4820 con 0x7f72e015c0c0 2024-05-28T14:16:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e8400640 1 -- 172.21.15.42:0/4055226078 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e0158690 con 0x7f72e015c0c0 2024-05-28T14:16:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.217+0000 7f72e8400640 1 -- 172.21.15.42:0/4055226078 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e0158c50 con 0x7f72e015c0c0 2024-05-28T14:16:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.221+0000 7f72e8400640 1 -- 172.21.15.42:0/4055226078 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72e00738b0 con 0x7f72e015c0c0 2024-05-28T14:16:48.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.221+0000 7f72dd600640 1 -- 172.21.15.42:0/4055226078 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f72d80b3070 con 0x7f72e015c0c0 2024-05-28T14:16:48.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.221+0000 7f72dd600640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f72c006a420 0x7f72c006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.221+0000 7f72dd600640 1 -- 172.21.15.42:0/4055226078 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f72d811f150 con 0x7f72e015c0c0 2024-05-28T14:16:48.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.221+0000 7f72e5a00640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f72c006a420 0x7f72c006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.221+0000 7f72e5a00640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f72c006a420 0x7f72c006c8e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f72e0072730 tx=0x7f72d4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:48.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.225+0000 7f72dd600640 1 -- 172.21.15.42:0/4055226078 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f72d80160d0 con 0x7f72e015c0c0 2024-05-28T14:16:48.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.389+0000 7f72e8400640 1 -- 172.21.15.42:0/4055226078 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f72e004fdd0 con 0x7f72e015c0c0 2024-05-28T14:16:48.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.389+0000 7f72dd600640 1 -- 172.21.15.42:0/4055226078 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) ==== 93+0+10 (secure 0 0 0) 0x7f72d80f0b20 con 0x7f72e015c0c0 2024-05-28T14:16:48.375 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-05-28T14:16:48.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.393+0000 7f72e8400640 1 -- 172.21.15.42:0/4055226078 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f72c006a420 msgr2=0x7f72c006c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.393+0000 7f72e8400640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f72c006a420 0x7f72c006c8e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f72e0072730 tx=0x7f72d4009040 comp rx=0 tx=0).stop 2024-05-28T14:16:48.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.393+0000 7f72e8400640 1 -- 172.21.15.42:0/4055226078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e015c0c0 msgr2=0x7f72e013edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.393+0000 7f72e8400640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e015c0c0 0x7f72e013edc0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f72d80097f0 tx=0x7f72d8019280 comp rx=0 tx=0).stop 2024-05-28T14:16:48.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.393+0000 7f72e8400640 1 -- 172.21.15.42:0/4055226078 shutdown_connections 2024-05-28T14:16:48.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.393+0000 7f72e8400640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f72c006a420 0x7f72c006c8e0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.393+0000 7f72e8400640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e015c0c0 0x7f72e013edc0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.393+0000 7f72e8400640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72e0144f80 0x7f72e015cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.393+0000 7f72e8400640 1 --2- 172.21.15.42:0/4055226078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72e006ee20 0x7f72e00aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.393+0000 7f72e8400640 1 -- 172.21.15.42:0/4055226078 >> 172.21.15.42:0/4055226078 conn(0x7f72e00b21f0 msgr2=0x7f72e00b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:48.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.393+0000 7f72e8400640 1 -- 172.21.15.42:0/4055226078 shutdown_connections 2024-05-28T14:16:48.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.393+0000 7f72e8400640 1 -- 172.21.15.42:0/4055226078 wait complete. 2024-05-28T14:16:48.395 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-28T14:16:48.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.553+0000 7f699be00640 1 Processor -- start 2024-05-28T14:16:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.553+0000 7f699be00640 1 -- start start 2024-05-28T14:16:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.557+0000 7f699be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6994071de0 0x7f69940721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.557+0000 7f699be00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6994072700 con 0x7f69940ce3b0 2024-05-28T14:16:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.557+0000 7f699be00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6994160230 con 0x7f699413fd10 2024-05-28T14:16:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.557+0000 7f699be00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6994160510 con 0x7f6994071de0 2024-05-28T14:16:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.557+0000 7f6998a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f69940ce3b0 0x7f69940ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:52994/0 (socket says 172.21.15.42:52994) 2024-05-28T14:16:48.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.557+0000 7f6998a00640 1 -- 172.21.15.42:0/2110153016 learned_addr learned my addr 172.21.15.42:0/2110153016 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:48.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.557+0000 7f6999400640 1 --2- 172.21.15.42:0/2110153016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6994071de0 0x7f69940721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.557+0000 7f6993e00640 1 -- 172.21.15.42:0/2110153016 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 158077741 0 0) 0x7f6994072700 con 0x7f69940ce3b0 2024-05-28T14:16:48.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.557+0000 7f6993e00640 1 -- 172.21.15.42:0/2110153016 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f697c003530 con 0x7f69940ce3b0 2024-05-28T14:16:48.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.557+0000 7f6993e00640 1 -- 172.21.15.42:0/2110153016 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 818565340 0 0) 0x7f6994160230 con 0x7f699413fd10 2024-05-28T14:16:48.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.557+0000 7f6993e00640 1 -- 172.21.15.42:0/2110153016 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6994072700 con 0x7f699413fd10 2024-05-28T14:16:48.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.557+0000 7f6999400640 1 -- 172.21.15.42:0/2110153016 >> v1:172.21.15.63:6789/0 conn(0x7f699413fd10 legacy=0x7f69941401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.557+0000 7f6999400640 1 -- 172.21.15.42:0/2110153016 >> v1:172.21.15.42:6789/0 conn(0x7f69940ce3b0 legacy=0x7f69940ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.557+0000 7f6999400640 1 -- 172.21.15.42:0/2110153016 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69941607f0 con 0x7f6994071de0 2024-05-28T14:16:48.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.561+0000 7f6999400640 1 --2- 172.21.15.42:0/2110153016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6994071de0 0x7f69940721c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f69880097f0 tx=0x7f698801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4951930deffcf87 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:48.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.561+0000 7f6993e00640 1 -- 172.21.15.42:0/2110153016 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69880b0070 con 0x7f6994071de0 2024-05-28T14:16:48.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.561+0000 7f6993e00640 1 -- 172.21.15.42:0/2110153016 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69880a4e50 con 0x7f6994071de0 2024-05-28T14:16:48.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.561+0000 7f6993e00640 1 -- 172.21.15.42:0/2110153016 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69880ab820 con 0x7f6994071de0 2024-05-28T14:16:48.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.561+0000 7f699be00640 1 -- 172.21.15.42:0/2110153016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6994071de0 msgr2=0x7f69940721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.561+0000 7f699be00640 1 --2- 172.21.15.42:0/2110153016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6994071de0 0x7f69940721c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f69880097f0 tx=0x7f698801c3c0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.561+0000 7f699be00640 1 -- 172.21.15.42:0/2110153016 shutdown_connections 2024-05-28T14:16:48.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.561+0000 7f699be00640 1 --2- 172.21.15.42:0/2110153016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6994071de0 0x7f69940721c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.561+0000 7f699be00640 1 -- 172.21.15.42:0/2110153016 >> 172.21.15.42:0/2110153016 conn(0x7f69940b21f0 msgr2=0x7f69940b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:48.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.561+0000 7f699be00640 1 -- 172.21.15.42:0/2110153016 shutdown_connections 2024-05-28T14:16:48.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.561+0000 7f699be00640 1 -- 172.21.15.42:0/2110153016 wait complete. 2024-05-28T14:16:48.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.561+0000 7f699be00640 1 Processor -- start 2024-05-28T14:16:48.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.561+0000 7f699be00640 1 -- start start 2024-05-28T14:16:48.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f699be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6994071de0 0x7f69941546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f699be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69940ce3b0 0x7f6994153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f699be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699413fd10 0x7f69941571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f699be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f699416a840 con 0x7f6994071de0 2024-05-28T14:16:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f699be00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f699416a6c0 con 0x7f699413fd10 2024-05-28T14:16:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f699be00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f699416a9c0 con 0x7f69940ce3b0 2024-05-28T14:16:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6999400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6994071de0 0x7f69941546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6999400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6994071de0 0x7f69941546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37868/0 (socket says 172.21.15.42:37868) 2024-05-28T14:16:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6999400640 1 -- 172.21.15.42:0/2736043695 learned_addr learned my addr 172.21.15.42:0/2736043695 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6998a00640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69940ce3b0 0x7f6994153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6999e00640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699413fd10 0x7f69941571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6999400640 1 -- 172.21.15.42:0/2736043695 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69940ce3b0 msgr2=0x7f6994153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6999400640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69940ce3b0 0x7f6994153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6999400640 1 -- 172.21.15.42:0/2736043695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699413fd10 msgr2=0x7f69941571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6999400640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699413fd10 0x7f69941571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6999400640 1 -- 172.21.15.42:0/2736043695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699415f520 con 0x7f6994071de0 2024-05-28T14:16:48.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6998a00640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69940ce3b0 0x7f6994153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6999400640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6994071de0 0x7f69941546a0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f6988018fd0 tx=0x7f6988019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6991600640 1 -- 172.21.15.42:0/2736043695 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69880b9070 con 0x7f6994071de0 2024-05-28T14:16:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6991600640 1 -- 172.21.15.42:0/2736043695 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69880abd80 con 0x7f6994071de0 2024-05-28T14:16:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6991600640 1 -- 172.21.15.42:0/2736043695 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69880b0040 con 0x7f6994071de0 2024-05-28T14:16:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f699be00640 1 -- 172.21.15.42:0/2736043695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699415f7c0 con 0x7f6994071de0 2024-05-28T14:16:48.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f699be00640 1 -- 172.21.15.42:0/2736043695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69941543d0 con 0x7f6994071de0 2024-05-28T14:16:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f699be00640 1 -- 172.21.15.42:0/2736043695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69940738b0 con 0x7f6994071de0 2024-05-28T14:16:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.565+0000 7f6991600640 1 -- 172.21.15.42:0/2736043695 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f69880bc070 con 0x7f6994071de0 2024-05-28T14:16:48.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.569+0000 7f6991600640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f697406a420 0x7f697406c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.569+0000 7f6991600640 1 -- 172.21.15.42:0/2736043695 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f698811f960 con 0x7f6994071de0 2024-05-28T14:16:48.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.569+0000 7f6998a00640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f697406a420 0x7f697406c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.569+0000 7f6998a00640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f697406a420 0x7f697406c8e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f699415c760 tx=0x7f6980009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:48.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.573+0000 7f6991600640 1 -- 172.21.15.42:0/2736043695 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f69880c3030 con 0x7f6994071de0 2024-05-28T14:16:48.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.737+0000 7f699be00640 1 -- 172.21.15.42:0/2736043695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f699415a4a0 con 0x7f6994071de0 2024-05-28T14:16:48.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.737+0000 7f6991600640 1 -- 172.21.15.42:0/2736043695 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) ==== 104+0+11 (secure 0 0 0) 0x7f69880160d0 con 0x7f6994071de0 2024-05-28T14:16:48.722 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-05-28T14:16:48.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.737+0000 7f699be00640 1 -- 172.21.15.42:0/2736043695 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f697406a420 msgr2=0x7f697406c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.737+0000 7f699be00640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f697406a420 0x7f697406c8e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f699415c760 tx=0x7f6980009040 comp rx=0 tx=0).stop 2024-05-28T14:16:48.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.737+0000 7f699be00640 1 -- 172.21.15.42:0/2736043695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6994071de0 msgr2=0x7f69941546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.737+0000 7f699be00640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6994071de0 0x7f69941546a0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f6988018fd0 tx=0x7f6988019160 comp rx=0 tx=0).stop 2024-05-28T14:16:48.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.737+0000 7f699be00640 1 -- 172.21.15.42:0/2736043695 shutdown_connections 2024-05-28T14:16:48.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.737+0000 7f699be00640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f697406a420 0x7f697406c8e0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.737+0000 7f699be00640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699413fd10 0x7f69941571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.737+0000 7f699be00640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69940ce3b0 0x7f6994153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.737+0000 7f699be00640 1 --2- 172.21.15.42:0/2736043695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6994071de0 0x7f69941546a0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.737+0000 7f699be00640 1 -- 172.21.15.42:0/2736043695 >> 172.21.15.42:0/2736043695 conn(0x7f69940b21f0 msgr2=0x7f69940b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:48.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.737+0000 7f699be00640 1 -- 172.21.15.42:0/2736043695 shutdown_connections 2024-05-28T14:16:48.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.737+0000 7f699be00640 1 -- 172.21.15.42:0/2736043695 wait complete. 2024-05-28T14:16:48.743 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-28T14:16:48.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.901+0000 7fe583a00640 1 Processor -- start 2024-05-28T14:16:48.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.901+0000 7fe583a00640 1 -- start start 2024-05-28T14:16:48.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe583a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe57c140d20 0x7fe57c167490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe583a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe57c160230 con 0x7fe57c0ce3b0 2024-05-28T14:16:48.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe583a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe57c160510 con 0x7fe57c071de0 2024-05-28T14:16:48.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe583a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe57c1607f0 con 0x7fe57c140d20 2024-05-28T14:16:48.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe573e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe57c0ce3b0 0x7fe57c0ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53006/0 (socket says 172.21.15.42:53006) 2024-05-28T14:16:48.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe573e00640 1 -- 172.21.15.42:0/1506850906 learned_addr learned my addr 172.21.15.42:0/1506850906 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:48.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe581a00640 1 --2- 172.21.15.42:0/1506850906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe57c140d20 0x7fe57c167490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe57be00640 1 -- 172.21.15.42:0/1506850906 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 473350991 0 0) 0x7fe57c160230 con 0x7fe57c0ce3b0 2024-05-28T14:16:48.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe57be00640 1 -- 172.21.15.42:0/1506850906 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe560003530 con 0x7fe57c0ce3b0 2024-05-28T14:16:48.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe57be00640 1 -- 172.21.15.42:0/1506850906 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 887125211 0 0) 0x7fe57c160510 con 0x7fe57c071de0 2024-05-28T14:16:48.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe57be00640 1 -- 172.21.15.42:0/1506850906 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe57c160230 con 0x7fe57c071de0 2024-05-28T14:16:48.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe57be00640 1 -- 172.21.15.42:0/1506850906 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4115789113 0 0) 0x7fe560003530 con 0x7fe57c0ce3b0 2024-05-28T14:16:48.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe581a00640 1 -- 172.21.15.42:0/1506850906 >> v1:172.21.15.63:6789/0 conn(0x7fe57c071de0 legacy=0x7fe57c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe581a00640 1 -- 172.21.15.42:0/1506850906 >> v1:172.21.15.42:6789/0 conn(0x7fe57c0ce3b0 legacy=0x7fe57c0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.905+0000 7fe581a00640 1 -- 172.21.15.42:0/1506850906 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57c160ad0 con 0x7fe57c140d20 2024-05-28T14:16:48.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.909+0000 7fe581a00640 1 --2- 172.21.15.42:0/1506850906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe57c140d20 0x7fe57c167490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe574009dc0 tx=0x7fe57401c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6bf3e5930329b04 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:48.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.909+0000 7fe57be00640 1 -- 172.21.15.42:0/1506850906 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564001680 con 0x7fe57c140d20 2024-05-28T14:16:48.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.909+0000 7fe57be00640 1 -- 172.21.15.42:0/1506850906 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5740a8440 con 0x7fe57c140d20 2024-05-28T14:16:48.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.909+0000 7fe57be00640 1 -- 172.21.15.42:0/1506850906 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5740a8810 con 0x7fe57c140d20 2024-05-28T14:16:48.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.909+0000 7fe583a00640 1 -- 172.21.15.42:0/1506850906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe57c140d20 msgr2=0x7fe57c167490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.909+0000 7fe583a00640 1 --2- 172.21.15.42:0/1506850906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe57c140d20 0x7fe57c167490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe574009dc0 tx=0x7fe57401c790 comp rx=0 tx=0).stop 2024-05-28T14:16:48.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.909+0000 7fe583a00640 1 -- 172.21.15.42:0/1506850906 shutdown_connections 2024-05-28T14:16:48.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.909+0000 7fe583a00640 1 --2- 172.21.15.42:0/1506850906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe57c140d20 0x7fe57c167490 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.909+0000 7fe583a00640 1 -- 172.21.15.42:0/1506850906 >> 172.21.15.42:0/1506850906 conn(0x7fe57c0b21f0 msgr2=0x7fe57c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:48.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.909+0000 7fe583a00640 1 -- 172.21.15.42:0/1506850906 shutdown_connections 2024-05-28T14:16:48.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.909+0000 7fe583a00640 1 -- 172.21.15.42:0/1506850906 wait complete. 2024-05-28T14:16:48.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.909+0000 7fe583a00640 1 Processor -- start 2024-05-28T14:16:48.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.909+0000 7fe583a00640 1 -- start start 2024-05-28T14:16:48.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe583a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe57c071de0 0x7fe57c0b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe583a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe57c0ce3b0 0x7fe57c155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe583a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe57c140d20 0x7fe57c13b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe583a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c16a990 con 0x7fe57c071de0 2024-05-28T14:16:48.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe583a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c16a810 con 0x7fe57c140d20 2024-05-28T14:16:48.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe583a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe57c16ab10 con 0x7fe57c0ce3b0 2024-05-28T14:16:48.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe581000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe57c071de0 0x7fe57c0b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe581000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe57c071de0 0x7fe57c0b4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37872/0 (socket says 172.21.15.42:37872) 2024-05-28T14:16:48.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe581000640 1 -- 172.21.15.42:0/2230742608 learned_addr learned my addr 172.21.15.42:0/2230742608 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:48.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe573e00640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe57c0ce3b0 0x7fe57c155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe581a00640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe57c140d20 0x7fe57c13b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe581000640 1 -- 172.21.15.42:0/2230742608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe57c0ce3b0 msgr2=0x7fe57c155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe581000640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe57c0ce3b0 0x7fe57c155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe581000640 1 -- 172.21.15.42:0/2230742608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe57c140d20 msgr2=0x7fe57c13b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:48.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe581000640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe57c140d20 0x7fe57c13b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:48.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe581000640 1 -- 172.21.15.42:0/2230742608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57c15cdf0 con 0x7fe57c071de0 2024-05-28T14:16:48.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe573e00640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe57c0ce3b0 0x7fe57c155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:48.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe581a00640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe57c140d20 0x7fe57c13b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:48.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe581000640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe57c071de0 0x7fe57c0b4af0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00c490 tx=0x7fe56c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:48.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe579600640 1 -- 172.21.15.42:0/2230742608 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c0052d0 con 0x7fe57c071de0 2024-05-28T14:16:48.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe579600640 1 -- 172.21.15.42:0/2230742608 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56c005470 con 0x7fe57c071de0 2024-05-28T14:16:48.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe579600640 1 -- 172.21.15.42:0/2230742608 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c012870 con 0x7fe57c071de0 2024-05-28T14:16:48.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe583a00640 1 -- 172.21.15.42:0/2230742608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57c0b51d0 con 0x7fe57c071de0 2024-05-28T14:16:48.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe583a00640 1 -- 172.21.15.42:0/2230742608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57c13aa10 con 0x7fe57c071de0 2024-05-28T14:16:48.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.913+0000 7fe583a00640 1 -- 172.21.15.42:0/2230742608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe57c0738b0 con 0x7fe57c071de0 2024-05-28T14:16:48.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.917+0000 7fe579600640 1 -- 172.21.15.42:0/2230742608 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7fe56c012a10 con 0x7fe57c071de0 2024-05-28T14:16:48.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.917+0000 7fe579600640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe55806a420 0x7fe55806c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:48.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.917+0000 7fe579600640 1 -- 172.21.15.42:0/2230742608 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fe56c085790 con 0x7fe57c071de0 2024-05-28T14:16:48.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.917+0000 7fe573e00640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe55806a420 0x7fe55806c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:48.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.917+0000 7fe573e00640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe55806a420 0x7fe55806c8e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe57c15fb50 tx=0x7fe564009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:48.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:48.921+0000 7fe579600640 1 -- 172.21.15.42:0/2230742608 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe56c0577b0 con 0x7fe57c071de0 2024-05-28T14:16:49.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.089+0000 7fe583a00640 1 -- 172.21.15.42:0/2230742608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe57c15fb80 con 0x7fe57c071de0 2024-05-28T14:16:49.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.089+0000 7fe579600640 1 -- 172.21.15.42:0/2230742608 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) ==== 100+0+11 (secure 0 0 0) 0x7fe56c0570b0 con 0x7fe57c071de0 2024-05-28T14:16:49.073 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-05-28T14:16:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.089+0000 7fe583a00640 1 -- 172.21.15.42:0/2230742608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe55806a420 msgr2=0x7fe55806c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.089+0000 7fe583a00640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe55806a420 0x7fe55806c8e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe57c15fb50 tx=0x7fe564009040 comp rx=0 tx=0).stop 2024-05-28T14:16:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.089+0000 7fe583a00640 1 -- 172.21.15.42:0/2230742608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe57c071de0 msgr2=0x7fe57c0b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.089+0000 7fe583a00640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe57c071de0 0x7fe57c0b4af0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00c490 tx=0x7fe56c00c950 comp rx=0 tx=0).stop 2024-05-28T14:16:49.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.089+0000 7fe583a00640 1 -- 172.21.15.42:0/2230742608 shutdown_connections 2024-05-28T14:16:49.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.089+0000 7fe583a00640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe55806a420 0x7fe55806c8e0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:49.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.089+0000 7fe583a00640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe57c140d20 0x7fe57c13b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:49.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.089+0000 7fe583a00640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe57c0ce3b0 0x7fe57c155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:49.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.089+0000 7fe583a00640 1 --2- 172.21.15.42:0/2230742608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe57c071de0 0x7fe57c0b4af0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:49.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.089+0000 7fe583a00640 1 -- 172.21.15.42:0/2230742608 >> 172.21.15.42:0/2230742608 conn(0x7fe57c0b21f0 msgr2=0x7fe57c145540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:49.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.089+0000 7fe583a00640 1 -- 172.21.15.42:0/2230742608 shutdown_connections 2024-05-28T14:16:49.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.089+0000 7fe583a00640 1 -- 172.21.15.42:0/2230742608 wait complete. 2024-05-28T14:16:49.095 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-28T14:16:49.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.253+0000 7f2353000640 1 Processor -- start 2024-05-28T14:16:49.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.253+0000 7f2353000640 1 -- start start 2024-05-28T14:16:49.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.253+0000 7f2353000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f234c09aa50 0x7f234c09ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:49.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.253+0000 7f2353000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f234c09b370 con 0x7f234c0d15c0 2024-05-28T14:16:49.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.253+0000 7f2353000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f234c1622a0 con 0x7f234c0b4bc0 2024-05-28T14:16:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.253+0000 7f2353000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f234c0ccee0 con 0x7f234c09aa50 2024-05-28T14:16:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.253+0000 7f234be00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f234c09aa50 0x7f234c09ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.253+0000 7f2351000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f234c0d15c0 0x7f234c0cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53020/0 (socket says 172.21.15.42:53020) 2024-05-28T14:16:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.253+0000 7f2351000640 1 -- 172.21.15.42:0/907194445 learned_addr learned my addr 172.21.15.42:0/907194445 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.257+0000 7f234be00640 1 -- 172.21.15.42:0/907194445 >> v1:172.21.15.63:6789/0 conn(0x7f234c0b4bc0 legacy=0x7f234c0cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.257+0000 7f234be00640 1 -- 172.21.15.42:0/907194445 >> v1:172.21.15.42:6789/0 conn(0x7f234c0d15c0 legacy=0x7f234c0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.257+0000 7f234be00640 1 -- 172.21.15.42:0/907194445 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c09b620 con 0x7f234c09aa50 2024-05-28T14:16:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.257+0000 7f234b400640 1 -- 172.21.15.42:0/907194445 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2775792251 0 0) 0x7f234c09b370 con 0x7f234c0d15c0 2024-05-28T14:16:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.257+0000 7f234be00640 1 --2- 172.21.15.42:0/907194445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234c09aa50 0x7f234c09ae30 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f233c0097c0 tx=0x7f233c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443ca95444303adf server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.257+0000 7f234b400640 1 -- 172.21.15.42:0/907194445 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c0aa5f0 con 0x7f234c09aa50 2024-05-28T14:16:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.257+0000 7f234b400640 1 -- 172.21.15.42:0/907194445 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f233c0af070 con 0x7f234c09aa50 2024-05-28T14:16:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.257+0000 7f234b400640 1 -- 172.21.15.42:0/907194445 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c0aa9b0 con 0x7f234c09aa50 2024-05-28T14:16:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.257+0000 7f2353000640 1 -- 172.21.15.42:0/907194445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234c09aa50 msgr2=0x7f234c09ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.257+0000 7f2353000640 1 --2- 172.21.15.42:0/907194445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234c09aa50 0x7f234c09ae30 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f233c0097c0 tx=0x7f233c01c470 comp rx=0 tx=0).stop 2024-05-28T14:16:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2353000640 1 -- 172.21.15.42:0/907194445 shutdown_connections 2024-05-28T14:16:49.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2353000640 1 --2- 172.21.15.42:0/907194445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234c09aa50 0x7f234c09ae30 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:49.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2353000640 1 -- 172.21.15.42:0/907194445 >> 172.21.15.42:0/907194445 conn(0x7f234c0b4010 msgr2=0x7f234c0b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:49.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2353000640 1 -- 172.21.15.42:0/907194445 shutdown_connections 2024-05-28T14:16:49.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2353000640 1 -- 172.21.15.42:0/907194445 wait complete. 2024-05-28T14:16:49.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2353000640 1 Processor -- start 2024-05-28T14:16:49.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2353000640 1 -- start start 2024-05-28T14:16:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2353000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234c09aa50 0x7f234c157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2353000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f234c0b4bc0 0x7f234c152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2353000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f234c0d15c0 0x7f234c1541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2353000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0ce5a0 con 0x7f234c09aa50 2024-05-28T14:16:49.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2353000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0ccb70 con 0x7f234c0d15c0 2024-05-28T14:16:49.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2353000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f234c16cdc0 con 0x7f234c0b4bc0 2024-05-28T14:16:49.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f234be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234c09aa50 0x7f234c157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:49.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f234be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234c09aa50 0x7f234c157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37894/0 (socket says 172.21.15.42:37894) 2024-05-28T14:16:49.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f234be00640 1 -- 172.21.15.42:0/851603350 learned_addr learned my addr 172.21.15.42:0/851603350 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:49.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2343400640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f234c0b4bc0 0x7f234c152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:49.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2351000640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f234c0d15c0 0x7f234c1541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:49.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f234be00640 1 -- 172.21.15.42:0/851603350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f234c0b4bc0 msgr2=0x7f234c152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:49.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f234be00640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f234c0b4bc0 0x7f234c152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:49.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f234be00640 1 -- 172.21.15.42:0/851603350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f234c0d15c0 msgr2=0x7f234c1541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:49.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f234be00640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f234c0d15c0 0x7f234c1541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:49.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f234be00640 1 -- 172.21.15.42:0/851603350 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c0b57c0 con 0x7f234c09aa50 2024-05-28T14:16:49.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f2351000640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f234c0d15c0 0x7f234c1541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:49.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.261+0000 7f234be00640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234c09aa50 0x7f234c157070 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f233c0ac040 tx=0x7f233c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:49.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.265+0000 7f2348c00640 1 -- 172.21.15.42:0/851603350 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c0be610 con 0x7f234c09aa50 2024-05-28T14:16:49.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.265+0000 7f2348c00640 1 -- 172.21.15.42:0/851603350 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f233c0be7d0 con 0x7f234c09aa50 2024-05-28T14:16:49.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.265+0000 7f2348c00640 1 -- 172.21.15.42:0/851603350 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c0bebc0 con 0x7f234c09aa50 2024-05-28T14:16:49.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.265+0000 7f2353000640 1 -- 172.21.15.42:0/851603350 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234c0b5aa0 con 0x7f234c09aa50 2024-05-28T14:16:49.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.265+0000 7f2353000640 1 -- 172.21.15.42:0/851603350 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f234c0b6060 con 0x7f234c09aa50 2024-05-28T14:16:49.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.265+0000 7f2353000640 1 -- 172.21.15.42:0/851603350 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f234c0a0630 con 0x7f234c09aa50 2024-05-28T14:16:49.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.265+0000 7f2348c00640 1 -- 172.21.15.42:0/851603350 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f233c0bc020 con 0x7f234c09aa50 2024-05-28T14:16:49.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.265+0000 7f2348c00640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f232806a450 0x7f232806c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:49.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.265+0000 7f2348c00640 1 -- 172.21.15.42:0/851603350 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f233c11fe20 con 0x7f234c09aa50 2024-05-28T14:16:49.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.265+0000 7f2343400640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f232806a450 0x7f232806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:49.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.265+0000 7f2343400640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f232806a450 0x7f232806c910 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f234c161680 tx=0x7f2334009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:49.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.269+0000 7f2348c00640 1 -- 172.21.15.42:0/851603350 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f233c0c5030 con 0x7f234c09aa50 2024-05-28T14:16:49.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.453+0000 7f2353000640 1 -- 172.21.15.42:0/851603350 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f234c06cc00 con 0x7f234c09aa50 2024-05-28T14:16:49.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.457+0000 7f2348c00640 1 -- 172.21.15.42:0/851603350 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v134) ==== 80+0+0 (secure 0 0 0) 0x7f233c016090 con 0x7f234c09aa50 2024-05-28T14:16:49.440 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-05-28T14:16:49.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.457+0000 7f2353000640 1 -- 172.21.15.42:0/851603350 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f232806a450 msgr2=0x7f232806c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:49.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.457+0000 7f2353000640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f232806a450 0x7f232806c910 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f234c161680 tx=0x7f2334009040 comp rx=0 tx=0).stop 2024-05-28T14:16:49.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.457+0000 7f2353000640 1 -- 172.21.15.42:0/851603350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234c09aa50 msgr2=0x7f234c157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:49.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.457+0000 7f2353000640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234c09aa50 0x7f234c157070 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f233c0ac040 tx=0x7f233c01acb0 comp rx=0 tx=0).stop 2024-05-28T14:16:49.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.457+0000 7f2353000640 1 -- 172.21.15.42:0/851603350 shutdown_connections 2024-05-28T14:16:49.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.457+0000 7f2353000640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f232806a450 0x7f232806c910 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:49.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.457+0000 7f2353000640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f234c0d15c0 0x7f234c1541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:49.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.457+0000 7f2353000640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f234c0b4bc0 0x7f234c152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:49.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.457+0000 7f2353000640 1 --2- 172.21.15.42:0/851603350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f234c09aa50 0x7f234c157070 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:49.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.457+0000 7f2353000640 1 -- 172.21.15.42:0/851603350 >> 172.21.15.42:0/851603350 conn(0x7f234c0b4010 msgr2=0x7f234c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:49.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.457+0000 7f2353000640 1 -- 172.21.15.42:0/851603350 shutdown_connections 2024-05-28T14:16:49.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:49.457+0000 7f2353000640 1 -- 172.21.15.42:0/851603350 wait complete. 2024-05-28T14:16:49.462 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-05-28T14:16:49.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:16:49.463 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-28T14:16:49.480 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[54965]: fuse finished with error 0 and tester_r 0 2024-05-28T14:16:50.053 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:16:50.054 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:16:50.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:16:50.054 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-28T14:16:50.063 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-28T14:16:50.063 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:16:50.115 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-28T14:16:50.115 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:16:50.115 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:16:50.125 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:16:50.125 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-28T14:16:50.125 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-28T14:16:50.175 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350320k,nr_inodes=4087580,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-28T14:16:50.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:16:50.176 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-28T14:16:50.226 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-28T14:16:50.227 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-28T14:16:50.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:16:50.228 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-28T14:16:50.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:16:50.296 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-28T14:16:50.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:16:50.361 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-28T14:16:50.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:16:50.521 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-05-28T14:16:50.531 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-28T14:16:50.531 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-28T14:16:50.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:16:50.531 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-28T14:16:50.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:16:50.596 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-28T14:16:50.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-28T14:16:50.661 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-05-28T14:16:50.807 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-28T14:16:50.816 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-28T14:16:50.816 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:16:50.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f9765400640 1 Processor -- start 2024-05-28T14:16:50.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f9765400640 1 -- start start 2024-05-28T14:16:50.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f9765400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f97600afa50 0x7f976006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:50.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f9765400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9760156e50 0x7f97600d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:50.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f9765400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f976015cb90 con 0x7f9760156a70 2024-05-28T14:16:50.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f9765400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f976015ce50 con 0x7f97600afa50 2024-05-28T14:16:50.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f9765400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f976015d130 con 0x7f9760156e50 2024-05-28T14:16:50.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f975e800640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f9760156a70 0x7f97600af340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53030/0 (socket says 172.21.15.42:53030) 2024-05-28T14:16:50.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f975e800640 1 -- 172.21.15.42:0/701148193 learned_addr learned my addr 172.21.15.42:0/701148193 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:50.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f9757e00640 1 --2- 172.21.15.42:0/701148193 >> v2:172.21.15.42:3300/0 conn(0x7f97600afa50 0x7f976006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:50.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f975f200640 1 --2- 172.21.15.42:0/701148193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9760156e50 0x7f97600d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:50.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f975de00640 1 -- 172.21.15.42:0/701148193 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1045130643 0 0) 0x7f976015cb90 con 0x7f9760156a70 2024-05-28T14:16:50.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f975de00640 1 -- 172.21.15.42:0/701148193 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9744003530 con 0x7f9760156a70 2024-05-28T14:16:50.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f9757e00640 1 -- 172.21.15.42:0/701148193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9760156e50 msgr2=0x7f97600d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:50.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f9757e00640 1 --2- 172.21.15.42:0/701148193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9760156e50 0x7f97600d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:50.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f9757e00640 1 -- 172.21.15.42:0/701148193 >> v1:172.21.15.42:6789/0 conn(0x7f9760156a70 legacy=0x7f97600af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:50.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.985+0000 7f9757e00640 1 -- 172.21.15.42:0/701148193 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976015d410 con 0x7f97600afa50 2024-05-28T14:16:50.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.989+0000 7f9757e00640 1 --2- 172.21.15.42:0/701148193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97600afa50 0x7f976006dd60 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f97480097c0 tx=0x7f974801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db98e58b37ac2e91 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:50.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.989+0000 7f975de00640 1 -- 172.21.15.42:0/701148193 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97480b1070 con 0x7f97600afa50 2024-05-28T14:16:50.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.989+0000 7f975de00640 1 -- 172.21.15.42:0/701148193 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97480ac410 con 0x7f97600afa50 2024-05-28T14:16:50.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.989+0000 7f975de00640 1 -- 172.21.15.42:0/701148193 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97480ac7e0 con 0x7f97600afa50 2024-05-28T14:16:50.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.989+0000 7f9765400640 1 -- 172.21.15.42:0/701148193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97600afa50 msgr2=0x7f976006dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:50.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.989+0000 7f9765400640 1 --2- 172.21.15.42:0/701148193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97600afa50 0x7f976006dd60 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f97480097c0 tx=0x7f974801c4a0 comp rx=0 tx=0).stop 2024-05-28T14:16:50.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.989+0000 7f9765400640 1 -- 172.21.15.42:0/701148193 shutdown_connections 2024-05-28T14:16:50.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.989+0000 7f9765400640 1 --2- 172.21.15.42:0/701148193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9760156e50 0x7f97600d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:50.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.989+0000 7f9765400640 1 --2- 172.21.15.42:0/701148193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97600afa50 0x7f976006dd60 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:50.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.989+0000 7f9765400640 1 -- 172.21.15.42:0/701148193 >> 172.21.15.42:0/701148193 conn(0x7f976006bdf0 msgr2=0x7f976006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:50.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.989+0000 7f9765400640 1 -- 172.21.15.42:0/701148193 shutdown_connections 2024-05-28T14:16:50.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.989+0000 7f9765400640 1 -- 172.21.15.42:0/701148193 wait complete. 2024-05-28T14:16:50.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9765400640 1 Processor -- start 2024-05-28T14:16:50.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9765400640 1 -- start start 2024-05-28T14:16:50.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9765400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97600afa50 0x7f976014fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:50.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9765400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760156a70 0x7f976015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:50.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9765400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9760156e50 0x7f976015baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:50.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9765400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9760160f50 con 0x7f9760156a70 2024-05-28T14:16:50.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9765400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9760160dd0 con 0x7f97600afa50 2024-05-28T14:16:50.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9765400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f97601610d0 con 0x7f9760156e50 2024-05-28T14:16:50.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9757e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760156a70 0x7f976015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:50.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9757e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760156a70 0x7f976015b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37910/0 (socket says 172.21.15.42:37910) 2024-05-28T14:16:50.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9757e00640 1 -- 172.21.15.42:0/4167631902 learned_addr learned my addr 172.21.15.42:0/4167631902 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:50.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f975f200640 1 --2- 172.21.15.42:0/4167631902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9760156e50 0x7f976015baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:50.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f975e800640 1 --2- 172.21.15.42:0/4167631902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97600afa50 0x7f976014fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:50.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9757e00640 1 -- 172.21.15.42:0/4167631902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9760156e50 msgr2=0x7f976015baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9757e00640 1 --2- 172.21.15.42:0/4167631902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9760156e50 0x7f976015baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9757e00640 1 -- 172.21.15.42:0/4167631902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97600afa50 msgr2=0x7f976014fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9757e00640 1 --2- 172.21.15.42:0/4167631902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97600afa50 0x7f976014fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9757e00640 1 -- 172.21.15.42:0/4167631902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976015a830 con 0x7f9760156a70 2024-05-28T14:16:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9757e00640 1 --2- 172.21.15.42:0/4167631902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760156a70 0x7f976015b5b0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f97480af040 tx=0x7f9748019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9756a00640 1 -- 172.21.15.42:0/4167631902 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97480b1040 con 0x7f9760156a70 2024-05-28T14:16:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9756a00640 1 -- 172.21.15.42:0/4167631902 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97480bb070 con 0x7f9760156a70 2024-05-28T14:16:50.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9756a00640 1 -- 172.21.15.42:0/4167631902 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748006e20 con 0x7f9760156a70 2024-05-28T14:16:50.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9765400640 1 -- 172.21.15.42:0/4167631902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976015aad0 con 0x7f9760156a70 2024-05-28T14:16:50.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.993+0000 7f9765400640 1 -- 172.21.15.42:0/4167631902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976015b090 con 0x7f9760156a70 2024-05-28T14:16:50.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.997+0000 7f9765400640 1 -- 172.21.15.42:0/4167631902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9760068a90 con 0x7f9760156a70 2024-05-28T14:16:50.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.997+0000 7f9756a00640 1 -- 172.21.15.42:0/4167631902 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f97480b5670 con 0x7f9760156a70 2024-05-28T14:16:50.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.997+0000 7f9756a00640 1 --2- 172.21.15.42:0/4167631902 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f973c06a420 0x7f973c06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:50.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.997+0000 7f9756a00640 1 -- 172.21.15.42:0/4167631902 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f974811f570 con 0x7f9760156a70 2024-05-28T14:16:50.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.997+0000 7f975e800640 1 --2- 172.21.15.42:0/4167631902 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f973c06a420 0x7f973c06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:50.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:50.997+0000 7f975e800640 1 --2- 172.21.15.42:0/4167631902 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f973c06a420 0x7f973c06c8e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9750005890 tx=0x7f9750009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:50.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.001+0000 7f9756a00640 1 -- 172.21.15.42:0/4167631902 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9748016090 con 0x7f9760156a70 2024-05-28T14:16:51.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.161+0000 7f9765400640 1 -- 172.21.15.42:0/4167631902 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f97600b5160 con 0x7f973c06a420 2024-05-28T14:16:51.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.169+0000 7f9756a00640 1 -- 172.21.15.42:0/4167631902 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254907 (secure 0 0 0) 0x7f97600b5160 con 0x7f973c06a420 2024-05-28T14:16:51.158 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:51.160 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:16:51.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.173+0000 7f9765400640 1 -- 172.21.15.42:0/4167631902 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f973c06a420 msgr2=0x7f973c06c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.173+0000 7f9765400640 1 --2- 172.21.15.42:0/4167631902 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f973c06a420 0x7f973c06c8e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9750005890 tx=0x7f9750009040 comp rx=0 tx=0).stop 2024-05-28T14:16:51.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.173+0000 7f9765400640 1 -- 172.21.15.42:0/4167631902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760156a70 msgr2=0x7f976015b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.173+0000 7f9765400640 1 --2- 172.21.15.42:0/4167631902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760156a70 0x7f976015b5b0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f97480af040 tx=0x7f9748019240 comp rx=0 tx=0).stop 2024-05-28T14:16:51.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.173+0000 7f9765400640 1 -- 172.21.15.42:0/4167631902 shutdown_connections 2024-05-28T14:16:51.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.173+0000 7f9765400640 1 --2- 172.21.15.42:0/4167631902 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f973c06a420 0x7f973c06c8e0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.173+0000 7f9765400640 1 --2- 172.21.15.42:0/4167631902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9760156e50 0x7f976015baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.173+0000 7f9765400640 1 --2- 172.21.15.42:0/4167631902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760156a70 0x7f976015b5b0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.173+0000 7f9765400640 1 --2- 172.21.15.42:0/4167631902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97600afa50 0x7f976014fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.173+0000 7f9765400640 1 -- 172.21.15.42:0/4167631902 >> 172.21.15.42:0/4167631902 conn(0x7f976006bdf0 msgr2=0x7f9760166760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:51.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.173+0000 7f9765400640 1 -- 172.21.15.42:0/4167631902 shutdown_connections 2024-05-28T14:16:51.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.173+0000 7f9765400640 1 -- 172.21.15.42:0/4167631902 wait complete. 2024-05-28T14:16:51.179 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":687,"stamp":"2024-05-28T14:16:49.248705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15085095885,"num_objects":3628,"num_object_clones":1,"num_object_copies":7256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3628,"num_whiteouts":1,"num_read":4511,"num_read_kb":17629376,"num_write":7376,"num_write_kb":20681055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":50331770,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7295,"ondisk_log_size":7295,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":30620568,"kb_used_data":29476248,"kb_used_omap":12,"kb_used_meta":1144243,"kb_avail":724354152,"statfs":{"total":773094113280,"available":741738651648,"internally_reserved":0,"allocated":30183677952,"data_stored":30176092898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1171705394},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002921"},"pg_stats":[{"pgid":"19.3f","version":"134'85","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.443809+0000","last_change":"2024-05-28T14:14:43.766434+0000","last_active":"2024-05-28T14:16:27.443809+0000","last_peered":"2024-05-28T14:16:27.443809+0000","last_clean":"2024-05-28T14:16:27.443809+0000","last_became_active":"2024-05-28T14:14:43.766125+0000","last_became_peered":"2024-05-28T14:14:43.766125+0000","last_unstale":"2024-05-28T14:16:27.443809+0000","last_undegraded":"2024-05-28T14:16:27.443809+0000","last_fullsized":"2024-05-28T14:16:27.443809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:50:53.753974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":77,"num_read_kb":315392,"num_write":85,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"134'106","reported_seq":197,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.496809+0000","last_change":"2024-05-28T14:14:43.769925+0000","last_active":"2024-05-28T14:16:27.496809+0000","last_peered":"2024-05-28T14:16:27.496809+0000","last_clean":"2024-05-28T14:16:27.496809+0000","last_became_active":"2024-05-28T14:14:43.769641+0000","last_became_peered":"2024-05-28T14:14:43.769641+0000","last_unstale":"2024-05-28T14:16:27.496809+0000","last_undegraded":"2024-05-28T14:16:27.496809+0000","last_fullsized":"2024-05-28T14:16:27.496809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:25:03.574855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":106,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"134'108","reported_seq":194,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.569959+0000","last_change":"2024-05-28T14:14:43.769199+0000","last_active":"2024-05-28T14:16:27.569959+0000","last_peered":"2024-05-28T14:16:27.569959+0000","last_clean":"2024-05-28T14:16:27.569959+0000","last_became_active":"2024-05-28T14:14:43.764098+0000","last_became_peered":"2024-05-28T14:14:43.764098+0000","last_unstale":"2024-05-28T14:16:27.569959+0000","last_undegraded":"2024-05-28T14:16:27.569959+0000","last_fullsized":"2024-05-28T14:16:27.569959+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:57:51.444394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":70,"num_read_kb":278530,"num_write":108,"num_write_kb":323587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"134'115","reported_seq":180,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.566221+0000","last_change":"2024-05-28T14:14:43.759234+0000","last_active":"2024-05-28T14:16:27.566221+0000","last_peered":"2024-05-28T14:16:27.566221+0000","last_clean":"2024-05-28T14:16:27.566221+0000","last_became_active":"2024-05-28T14:14:43.758925+0000","last_became_peered":"2024-05-28T14:14:43.758925+0000","last_unstale":"2024-05-28T14:16:27.566221+0000","last_undegraded":"2024-05-28T14:16:27.566221+0000","last_fullsized":"2024-05-28T14:16:27.566221+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:14.042730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":48,"num_read_kb":184323,"num_write":115,"num_write_kb":339969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"134'75","reported_seq":136,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.511181+0000","last_change":"2024-05-28T14:14:43.766612+0000","last_active":"2024-05-28T14:16:27.511181+0000","last_peered":"2024-05-28T14:16:27.511181+0000","last_clean":"2024-05-28T14:16:27.511181+0000","last_became_active":"2024-05-28T14:14:43.764703+0000","last_became_peered":"2024-05-28T14:14:43.764703+0000","last_unstale":"2024-05-28T14:16:27.511181+0000","last_undegraded":"2024-05-28T14:16:27.511181+0000","last_fullsized":"2024-05-28T14:16:27.511181+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:33.925255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":45,"num_read_kb":184320,"num_write":75,"num_write_kb":253952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"134'115","reported_seq":226,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.466012+0000","last_change":"2024-05-28T14:14:43.762958+0000","last_active":"2024-05-28T14:16:27.466012+0000","last_peered":"2024-05-28T14:16:27.466012+0000","last_clean":"2024-05-28T14:16:27.466012+0000","last_became_active":"2024-05-28T14:14:43.762689+0000","last_became_peered":"2024-05-28T14:14:43.762689+0000","last_unstale":"2024-05-28T14:16:27.466012+0000","last_undegraded":"2024-05-28T14:16:27.466012+0000","last_fullsized":"2024-05-28T14:16:27.466012+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:56:11.671806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":95,"num_read_kb":385025,"num_write":115,"num_write_kb":372738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"134'89","reported_seq":155,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.557040+0000","last_change":"2024-05-28T14:14:43.770900+0000","last_active":"2024-05-28T14:16:27.557040+0000","last_peered":"2024-05-28T14:16:27.557040+0000","last_clean":"2024-05-28T14:16:27.557040+0000","last_became_active":"2024-05-28T14:14:43.770616+0000","last_became_peered":"2024-05-28T14:14:43.770616+0000","last_unstale":"2024-05-28T14:16:27.557040+0000","last_undegraded":"2024-05-28T14:16:27.557040+0000","last_fullsized":"2024-05-28T14:16:27.557040+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:07:11.105627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":89,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"134'113","reported_seq":183,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.521158+0000","last_change":"2024-05-28T14:14:43.762262+0000","last_active":"2024-05-28T14:16:27.521158+0000","last_peered":"2024-05-28T14:16:27.521158+0000","last_clean":"2024-05-28T14:16:27.521158+0000","last_became_active":"2024-05-28T14:14:43.761977+0000","last_became_peered":"2024-05-28T14:14:43.761977+0000","last_unstale":"2024-05-28T14:16:27.521158+0000","last_undegraded":"2024-05-28T14:16:27.521158+0000","last_fullsized":"2024-05-28T14:16:27.521158+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:54.517890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":54,"num_read_kb":221184,"num_write":113,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"134'102","reported_seq":175,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.505567+0000","last_change":"2024-05-28T14:14:43.770664+0000","last_active":"2024-05-28T14:16:27.505567+0000","last_peered":"2024-05-28T14:16:27.505567+0000","last_clean":"2024-05-28T14:16:27.505567+0000","last_became_active":"2024-05-28T14:14:43.770335+0000","last_became_peered":"2024-05-28T14:14:43.770335+0000","last_unstale":"2024-05-28T14:16:27.505567+0000","last_undegraded":"2024-05-28T14:16:27.505567+0000","last_fullsized":"2024-05-28T14:16:27.505567+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:20:08.770733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":102,"num_write_kb":311297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"134'93","reported_seq":176,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.559409+0000","last_change":"2024-05-28T14:14:43.769965+0000","last_active":"2024-05-28T14:16:27.559409+0000","last_peered":"2024-05-28T14:16:27.559409+0000","last_clean":"2024-05-28T14:16:27.559409+0000","last_became_active":"2024-05-28T14:14:43.769687+0000","last_became_peered":"2024-05-28T14:14:43.769687+0000","last_unstale":"2024-05-28T14:16:27.559409+0000","last_undegraded":"2024-05-28T14:16:27.559409+0000","last_fullsized":"2024-05-28T14:16:27.559409+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:43:30.318199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":67,"num_read_kb":274432,"num_write":93,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"134'103","reported_seq":227,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.492452+0000","last_change":"2024-05-28T14:14:43.758128+0000","last_active":"2024-05-28T14:16:27.492452+0000","last_peered":"2024-05-28T14:16:27.492452+0000","last_clean":"2024-05-28T14:16:27.492452+0000","last_became_active":"2024-05-28T14:14:43.757685+0000","last_became_peered":"2024-05-28T14:14:43.757685+0000","last_unstale":"2024-05-28T14:16:27.492452+0000","last_undegraded":"2024-05-28T14:16:27.492452+0000","last_fullsized":"2024-05-28T14:16:27.492452+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:53.917272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":108,"num_read_kb":442368,"num_write":103,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"134'98","reported_seq":219,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.546924+0000","last_change":"2024-05-28T14:14:43.769235+0000","last_active":"2024-05-28T14:16:27.546924+0000","last_peered":"2024-05-28T14:16:27.546924+0000","last_clean":"2024-05-28T14:16:27.546924+0000","last_became_active":"2024-05-28T14:14:43.768939+0000","last_became_peered":"2024-05-28T14:14:43.768939+0000","last_unstale":"2024-05-28T14:16:27.546924+0000","last_undegraded":"2024-05-28T14:16:27.546924+0000","last_fullsized":"2024-05-28T14:16:27.546924+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:26.186099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":105,"num_read_kb":413700,"num_write":98,"num_write_kb":315393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"134'105","reported_seq":203,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.563971+0000","last_change":"2024-05-28T14:14:43.767408+0000","last_active":"2024-05-28T14:16:27.563971+0000","last_peered":"2024-05-28T14:16:27.563971+0000","last_clean":"2024-05-28T14:16:27.563971+0000","last_became_active":"2024-05-28T14:14:43.766269+0000","last_became_peered":"2024-05-28T14:14:43.766269+0000","last_unstale":"2024-05-28T14:16:27.563971+0000","last_undegraded":"2024-05-28T14:16:27.563971+0000","last_fullsized":"2024-05-28T14:16:27.563971+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:45.051114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":105,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"134'92","reported_seq":174,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.472754+0000","last_change":"2024-05-28T14:14:43.762819+0000","last_active":"2024-05-28T14:16:27.472754+0000","last_peered":"2024-05-28T14:16:27.472754+0000","last_clean":"2024-05-28T14:16:27.472754+0000","last_became_active":"2024-05-28T14:14:43.762494+0000","last_became_peered":"2024-05-28T14:14:43.762494+0000","last_unstale":"2024-05-28T14:16:27.472754+0000","last_undegraded":"2024-05-28T14:16:27.472754+0000","last_fullsized":"2024-05-28T14:16:27.472754+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:50:53.753974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":66,"num_read_kb":270336,"num_write":92,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"134'74","reported_seq":121,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.422586+0000","last_change":"2024-05-28T14:14:43.761245+0000","last_active":"2024-05-28T14:16:27.422586+0000","last_peered":"2024-05-28T14:16:27.422586+0000","last_clean":"2024-05-28T14:16:27.422586+0000","last_became_active":"2024-05-28T14:14:43.760977+0000","last_became_peered":"2024-05-28T14:14:43.760977+0000","last_unstale":"2024-05-28T14:16:27.422586+0000","last_undegraded":"2024-05-28T14:16:27.422586+0000","last_fullsized":"2024-05-28T14:16:27.422586+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:14.494237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":31,"num_read_kb":126976,"num_write":74,"num_write_kb":249856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"134'124","reported_seq":197,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.527258+0000","last_change":"2024-05-28T14:14:43.758776+0000","last_active":"2024-05-28T14:16:27.527258+0000","last_peered":"2024-05-28T14:16:27.527258+0000","last_clean":"2024-05-28T14:16:27.527258+0000","last_became_active":"2024-05-28T14:14:43.758483+0000","last_became_peered":"2024-05-28T14:14:43.758483+0000","last_unstale":"2024-05-28T14:16:27.527258+0000","last_undegraded":"2024-05-28T14:16:27.527258+0000","last_fullsized":"2024-05-28T14:16:27.527258+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:44:52.410827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":57,"num_read_kb":225282,"num_write":123,"num_write_kb":368641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"134'95","reported_seq":158,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.536820+0000","last_change":"2024-05-28T14:14:43.756230+0000","last_active":"2024-05-28T14:16:27.536820+0000","last_peered":"2024-05-28T14:16:27.536820+0000","last_clean":"2024-05-28T14:16:27.536820+0000","last_became_active":"2024-05-28T14:14:43.755912+0000","last_became_peered":"2024-05-28T14:14:43.755912+0000","last_unstale":"2024-05-28T14:16:27.536820+0000","last_undegraded":"2024-05-28T14:16:27.536820+0000","last_fullsized":"2024-05-28T14:16:27.536820+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:22.709561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":47,"num_read_kb":180227,"num_write":95,"num_write_kb":286721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"134'107","reported_seq":221,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.497520+0000","last_change":"2024-05-28T14:14:43.749113+0000","last_active":"2024-05-28T14:16:27.497520+0000","last_peered":"2024-05-28T14:16:27.497520+0000","last_clean":"2024-05-28T14:16:27.497520+0000","last_became_active":"2024-05-28T14:14:43.748801+0000","last_became_peered":"2024-05-28T14:14:43.748801+0000","last_unstale":"2024-05-28T14:16:27.497520+0000","last_undegraded":"2024-05-28T14:16:27.497520+0000","last_fullsized":"2024-05-28T14:16:27.497520+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:05:56.557590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":98,"num_read_kb":389123,"num_write":107,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"134'116","reported_seq":201,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.509116+0000","last_change":"2024-05-28T14:14:43.769214+0000","last_active":"2024-05-28T14:16:27.509116+0000","last_peered":"2024-05-28T14:16:27.509116+0000","last_clean":"2024-05-28T14:16:27.509116+0000","last_became_active":"2024-05-28T14:14:43.768757+0000","last_became_peered":"2024-05-28T14:14:43.768757+0000","last_unstale":"2024-05-28T14:16:27.509116+0000","last_undegraded":"2024-05-28T14:16:27.509116+0000","last_fullsized":"2024-05-28T14:16:27.509116+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:55:07.023689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":115,"num_write_kb":372736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"134'102","reported_seq":182,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.520712+0000","last_change":"2024-05-28T14:14:43.756647+0000","last_active":"2024-05-28T14:16:27.520712+0000","last_peered":"2024-05-28T14:16:27.520712+0000","last_clean":"2024-05-28T14:16:27.520712+0000","last_became_active":"2024-05-28T14:14:43.756244+0000","last_became_peered":"2024-05-28T14:14:43.756244+0000","last_unstale":"2024-05-28T14:16:27.520712+0000","last_undegraded":"2024-05-28T14:16:27.520712+0000","last_fullsized":"2024-05-28T14:16:27.520712+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:28:18.841947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":101,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"134'108","reported_seq":224,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.509454+0000","last_change":"2024-05-28T14:14:43.757686+0000","last_active":"2024-05-28T14:16:27.509454+0000","last_peered":"2024-05-28T14:16:27.509454+0000","last_clean":"2024-05-28T14:16:27.509454+0000","last_became_active":"2024-05-28T14:14:43.757288+0000","last_became_peered":"2024-05-28T14:14:43.757288+0000","last_unstale":"2024-05-28T14:16:27.509454+0000","last_undegraded":"2024-05-28T14:16:27.509454+0000","last_fullsized":"2024-05-28T14:16:27.509454+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:55:07.023689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":100,"num_read_kb":389125,"num_write":108,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"134'108","reported_seq":185,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.554312+0000","last_change":"2024-05-28T14:14:43.759851+0000","last_active":"2024-05-28T14:16:27.554312+0000","last_peered":"2024-05-28T14:16:27.554312+0000","last_clean":"2024-05-28T14:16:27.554312+0000","last_became_active":"2024-05-28T14:14:43.759580+0000","last_became_peered":"2024-05-28T14:14:43.759580+0000","last_unstale":"2024-05-28T14:16:27.554312+0000","last_undegraded":"2024-05-28T14:16:27.554312+0000","last_fullsized":"2024-05-28T14:16:27.554312+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:50:20.486680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":61,"num_read_kb":241666,"num_write":107,"num_write_kb":356353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"134'101","reported_seq":179,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.533323+0000","last_change":"2024-05-28T14:14:43.768741+0000","last_active":"2024-05-28T14:16:27.533323+0000","last_peered":"2024-05-28T14:16:27.533323+0000","last_clean":"2024-05-28T14:16:27.533323+0000","last_became_active":"2024-05-28T14:14:43.767741+0000","last_became_peered":"2024-05-28T14:14:43.767741+0000","last_unstale":"2024-05-28T14:16:27.533323+0000","last_undegraded":"2024-05-28T14:16:27.533323+0000","last_fullsized":"2024-05-28T14:16:27.533323+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:20.117904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":62,"num_read_kb":253952,"num_write":101,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"134'103","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.546575+0000","last_change":"2024-05-28T14:14:43.755177+0000","last_active":"2024-05-28T14:16:27.546575+0000","last_peered":"2024-05-28T14:16:27.546575+0000","last_clean":"2024-05-28T14:16:27.546575+0000","last_became_active":"2024-05-28T14:14:43.753794+0000","last_became_peered":"2024-05-28T14:14:43.753794+0000","last_unstale":"2024-05-28T14:16:27.546575+0000","last_undegraded":"2024-05-28T14:16:27.546575+0000","last_fullsized":"2024-05-28T14:16:27.546575+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:56:28.627044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":59,"num_read_kb":241664,"num_write":103,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"134'112","reported_seq":196,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.520025+0000","last_change":"2024-05-28T14:14:43.763725+0000","last_active":"2024-05-28T14:16:27.520025+0000","last_peered":"2024-05-28T14:16:27.520025+0000","last_clean":"2024-05-28T14:16:27.520025+0000","last_became_active":"2024-05-28T14:14:43.763332+0000","last_became_peered":"2024-05-28T14:14:43.763332+0000","last_unstale":"2024-05-28T14:16:27.520025+0000","last_undegraded":"2024-05-28T14:16:27.520025+0000","last_fullsized":"2024-05-28T14:16:27.520025+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:10.253064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298372,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":68,"num_read_kb":274433,"num_write":112,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"134'113","reported_seq":199,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.564705+0000","last_change":"2024-05-28T14:14:43.759381+0000","last_active":"2024-05-28T14:16:27.564705+0000","last_peered":"2024-05-28T14:16:27.564705+0000","last_clean":"2024-05-28T14:16:27.564705+0000","last_became_active":"2024-05-28T14:14:43.758722+0000","last_became_peered":"2024-05-28T14:14:43.758722+0000","last_unstale":"2024-05-28T14:16:27.564705+0000","last_undegraded":"2024-05-28T14:16:27.564705+0000","last_fullsized":"2024-05-28T14:16:27.564705+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:48:30.643597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251658240,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":112,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"134'99","reported_seq":190,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.455897+0000","last_change":"2024-05-28T14:14:43.768220+0000","last_active":"2024-05-28T14:16:27.455897+0000","last_peered":"2024-05-28T14:16:27.455897+0000","last_clean":"2024-05-28T14:16:27.455897+0000","last_became_active":"2024-05-28T14:14:43.767925+0000","last_became_peered":"2024-05-28T14:14:43.767925+0000","last_unstale":"2024-05-28T14:16:27.455897+0000","last_undegraded":"2024-05-28T14:16:27.455897+0000","last_fullsized":"2024-05-28T14:16:27.455897+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:35:31.221160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":75,"num_read_kb":294915,"num_write":99,"num_write_kb":290818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"134'94","reported_seq":144,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.459449+0000","last_change":"2024-05-28T14:14:43.746536+0000","last_active":"2024-05-28T14:16:27.459449+0000","last_peered":"2024-05-28T14:16:27.459449+0000","last_clean":"2024-05-28T14:16:27.459449+0000","last_became_active":"2024-05-28T14:14:43.746249+0000","last_became_peered":"2024-05-28T14:14:43.746249+0000","last_unstale":"2024-05-28T14:16:27.459449+0000","last_undegraded":"2024-05-28T14:16:27.459449+0000","last_fullsized":"2024-05-28T14:16:27.459449+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:30:48.180755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909764,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":34,"num_read_kb":131074,"num_write":94,"num_write_kb":290818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"134'104","reported_seq":192,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.538549+0000","last_change":"2024-05-28T14:14:43.761643+0000","last_active":"2024-05-28T14:16:27.538549+0000","last_peered":"2024-05-28T14:16:27.538549+0000","last_clean":"2024-05-28T14:16:27.538549+0000","last_became_active":"2024-05-28T14:14:43.761161+0000","last_became_peered":"2024-05-28T14:14:43.761161+0000","last_unstale":"2024-05-28T14:16:27.538549+0000","last_undegraded":"2024-05-28T14:16:27.538549+0000","last_fullsized":"2024-05-28T14:16:27.538549+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:11.879163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075588,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":72,"num_read_kb":294912,"num_write":104,"num_write_kb":327681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"134'106","reported_seq":191,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.526557+0000","last_change":"2024-05-28T14:14:43.764133+0000","last_active":"2024-05-28T14:16:27.526557+0000","last_peered":"2024-05-28T14:16:27.526557+0000","last_clean":"2024-05-28T14:16:27.526557+0000","last_became_active":"2024-05-28T14:14:43.763522+0000","last_became_peered":"2024-05-28T14:14:43.763522+0000","last_unstale":"2024-05-28T14:16:27.526557+0000","last_undegraded":"2024-05-28T14:16:27.526557+0000","last_fullsized":"2024-05-28T14:16:27.526557+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:05.864969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251658240,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":106,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"134'88","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.550227+0000","last_change":"2024-05-28T14:14:43.763447+0000","last_active":"2024-05-28T14:16:27.550227+0000","last_peered":"2024-05-28T14:16:27.550227+0000","last_clean":"2024-05-28T14:16:27.550227+0000","last_became_active":"2024-05-28T14:14:43.763142+0000","last_became_peered":"2024-05-28T14:14:43.763142+0000","last_unstale":"2024-05-28T14:16:27.550227+0000","last_undegraded":"2024-05-28T14:16:27.550227+0000","last_fullsized":"2024-05-28T14:16:27.550227+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:38:22.872839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":88,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"134'96","reported_seq":187,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.520563+0000","last_change":"2024-05-28T14:14:43.747637+0000","last_active":"2024-05-28T14:16:27.520563+0000","last_peered":"2024-05-28T14:16:27.520563+0000","last_clean":"2024-05-28T14:16:27.520563+0000","last_became_active":"2024-05-28T14:14:43.747272+0000","last_became_peered":"2024-05-28T14:14:43.747272+0000","last_unstale":"2024-05-28T14:16:27.520563+0000","last_undegraded":"2024-05-28T14:16:27.520563+0000","last_fullsized":"2024-05-28T14:16:27.520563+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:19.860056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":96,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"134'103","reported_seq":164,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.531516+0000","last_change":"2024-05-28T14:14:43.759397+0000","last_active":"2024-05-28T14:16:27.531516+0000","last_peered":"2024-05-28T14:16:27.531516+0000","last_clean":"2024-05-28T14:16:27.531516+0000","last_became_active":"2024-05-28T14:14:43.759136+0000","last_became_peered":"2024-05-28T14:14:43.759136+0000","last_unstale":"2024-05-28T14:16:27.531516+0000","last_undegraded":"2024-05-28T14:16:27.531516+0000","last_fullsized":"2024-05-28T14:16:27.531516+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:38.538536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":44,"num_read_kb":180224,"num_write":103,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"131'4","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973161+0000","last_change":"2024-05-28T14:14:42.751872+0000","last_active":"2024-05-28T14:15:05.973161+0000","last_peered":"2024-05-28T14:15:05.973161+0000","last_clean":"2024-05-28T14:15:05.973161+0000","last_became_active":"2024-05-28T14:14:42.751450+0000","last_became_peered":"2024-05-28T14:14:42.751450+0000","last_unstale":"2024-05-28T14:15:05.973161+0000","last_undegraded":"2024-05-28T14:15:05.973161+0000","last_fullsized":"2024-05-28T14:15:05.973161+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:48:49.635601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"134'112","reported_seq":200,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.571099+0000","last_change":"2024-05-28T14:14:43.766763+0000","last_active":"2024-05-28T14:16:27.571099+0000","last_peered":"2024-05-28T14:16:27.571099+0000","last_clean":"2024-05-28T14:16:27.571099+0000","last_became_active":"2024-05-28T14:14:43.766347+0000","last_became_peered":"2024-05-28T14:14:43.766347+0000","last_unstale":"2024-05-28T14:16:27.571099+0000","last_undegraded":"2024-05-28T14:16:27.571099+0000","last_fullsized":"2024-05-28T14:16:27.571099+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:43:49.334848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":72,"num_read_kb":290817,"num_write":112,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013729+0000","last_change":"2024-05-28T14:14:42.742952+0000","last_active":"2024-05-28T14:15:06.013729+0000","last_peered":"2024-05-28T14:15:06.013729+0000","last_clean":"2024-05-28T14:15:06.013729+0000","last_became_active":"2024-05-28T14:14:42.742668+0000","last_became_peered":"2024-05-28T14:14:42.742668+0000","last_unstale":"2024-05-28T14:15:06.013729+0000","last_undegraded":"2024-05-28T14:15:06.013729+0000","last_fullsized":"2024-05-28T14:15:06.013729+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:53:04.792926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"134'87","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.444881+0000","last_change":"2024-05-28T14:14:43.765567+0000","last_active":"2024-05-28T14:16:27.444881+0000","last_peered":"2024-05-28T14:16:27.444881+0000","last_clean":"2024-05-28T14:16:27.444881+0000","last_became_active":"2024-05-28T14:14:43.765230+0000","last_became_peered":"2024-05-28T14:14:43.765230+0000","last_unstale":"2024-05-28T14:16:27.444881+0000","last_undegraded":"2024-05-28T14:16:27.444881+0000","last_fullsized":"2024-05-28T14:16:27.444881+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:56:02.866180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":75,"num_read_kb":303105,"num_write":87,"num_write_kb":270337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"134'3","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971807+0000","last_change":"2024-05-28T14:14:42.733510+0000","last_active":"2024-05-28T14:15:05.971807+0000","last_peered":"2024-05-28T14:15:05.971807+0000","last_clean":"2024-05-28T14:15:05.971807+0000","last_became_active":"2024-05-28T14:14:42.733231+0000","last_became_peered":"2024-05-28T14:14:42.733231+0000","last_unstale":"2024-05-28T14:15:05.971807+0000","last_undegraded":"2024-05-28T14:15:05.971807+0000","last_fullsized":"2024-05-28T14:15:05.971807+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:37.532653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"134'107","reported_seq":210,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.435934+0000","last_change":"2024-05-28T14:14:43.760573+0000","last_active":"2024-05-28T14:16:27.435934+0000","last_peered":"2024-05-28T14:16:27.435934+0000","last_clean":"2024-05-28T14:16:27.435934+0000","last_became_active":"2024-05-28T14:14:43.759307+0000","last_became_peered":"2024-05-28T14:14:43.759307+0000","last_unstale":"2024-05-28T14:16:27.435934+0000","last_undegraded":"2024-05-28T14:16:27.435934+0000","last_fullsized":"2024-05-28T14:16:27.435934+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:30:49.015188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":107,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"131'2","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077844+0000","last_change":"2024-05-28T14:14:42.745411+0000","last_active":"2024-05-28T14:15:06.077844+0000","last_peered":"2024-05-28T14:15:06.077844+0000","last_clean":"2024-05-28T14:15:06.077844+0000","last_became_active":"2024-05-28T14:14:42.745097+0000","last_became_peered":"2024-05-28T14:14:42.745097+0000","last_unstale":"2024-05-28T14:15:06.077844+0000","last_undegraded":"2024-05-28T14:15:06.077844+0000","last_fullsized":"2024-05-28T14:15:06.077844+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:39:18.579687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"134'105","reported_seq":182,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.558340+0000","last_change":"2024-05-28T14:14:43.751125+0000","last_active":"2024-05-28T14:16:27.558340+0000","last_peered":"2024-05-28T14:16:27.558340+0000","last_clean":"2024-05-28T14:16:27.558340+0000","last_became_active":"2024-05-28T14:14:43.747282+0000","last_became_peered":"2024-05-28T14:14:43.747282+0000","last_unstale":"2024-05-28T14:16:27.558340+0000","last_undegraded":"2024-05-28T14:16:27.558340+0000","last_fullsized":"2024-05-28T14:16:27.558340+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:44:03.941731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":105,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077611+0000","last_change":"2024-05-28T14:14:42.741716+0000","last_active":"2024-05-28T14:15:06.077611+0000","last_peered":"2024-05-28T14:15:06.077611+0000","last_clean":"2024-05-28T14:15:06.077611+0000","last_became_active":"2024-05-28T14:14:42.741425+0000","last_became_peered":"2024-05-28T14:14:42.741425+0000","last_unstale":"2024-05-28T14:15:06.077611+0000","last_undegraded":"2024-05-28T14:15:06.077611+0000","last_fullsized":"2024-05-28T14:15:06.077611+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:36:40.085174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"134'102","reported_seq":217,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.510027+0000","last_change":"2024-05-28T14:14:43.760960+0000","last_active":"2024-05-28T14:16:27.510027+0000","last_peered":"2024-05-28T14:16:27.510027+0000","last_clean":"2024-05-28T14:16:27.510027+0000","last_became_active":"2024-05-28T14:14:43.760691+0000","last_became_peered":"2024-05-28T14:14:43.760691+0000","last_unstale":"2024-05-28T14:16:27.510027+0000","last_undegraded":"2024-05-28T14:16:27.510027+0000","last_fullsized":"2024-05-28T14:16:27.510027+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:33:40.268549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":99,"num_read_kb":405504,"num_write":102,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969182+0000","last_change":"2024-05-28T14:14:42.739561+0000","last_active":"2024-05-28T14:15:05.969182+0000","last_peered":"2024-05-28T14:15:05.969182+0000","last_clean":"2024-05-28T14:15:05.969182+0000","last_became_active":"2024-05-28T14:14:42.739191+0000","last_became_peered":"2024-05-28T14:14:42.739191+0000","last_unstale":"2024-05-28T14:15:05.969182+0000","last_undegraded":"2024-05-28T14:15:05.969182+0000","last_fullsized":"2024-05-28T14:15:05.969182+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:49:44.727202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"134'102","reported_seq":197,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.570052+0000","last_change":"2024-05-28T14:14:43.762652+0000","last_active":"2024-05-28T14:16:27.570052+0000","last_peered":"2024-05-28T14:16:27.570052+0000","last_clean":"2024-05-28T14:16:27.570052+0000","last_became_active":"2024-05-28T14:14:43.761547+0000","last_became_peered":"2024-05-28T14:14:43.761547+0000","last_unstale":"2024-05-28T14:16:27.570052+0000","last_undegraded":"2024-05-28T14:16:27.570052+0000","last_fullsized":"2024-05-28T14:16:27.570052+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:11:03.103530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":102,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973841+0000","last_change":"2024-05-28T14:14:42.745986+0000","last_active":"2024-05-28T14:15:05.973841+0000","last_peered":"2024-05-28T14:15:05.973841+0000","last_clean":"2024-05-28T14:15:05.973841+0000","last_became_active":"2024-05-28T14:14:42.745691+0000","last_became_peered":"2024-05-28T14:14:42.745691+0000","last_unstale":"2024-05-28T14:15:05.973841+0000","last_undegraded":"2024-05-28T14:15:05.973841+0000","last_fullsized":"2024-05-28T14:15:05.973841+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:22:09.220099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"134'97","reported_seq":179,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.532792+0000","last_change":"2024-05-28T14:14:43.764538+0000","last_active":"2024-05-28T14:16:27.532792+0000","last_peered":"2024-05-28T14:16:27.532792+0000","last_clean":"2024-05-28T14:16:27.532792+0000","last_became_active":"2024-05-28T14:14:43.762358+0000","last_became_peered":"2024-05-28T14:14:43.762358+0000","last_unstale":"2024-05-28T14:16:27.532792+0000","last_undegraded":"2024-05-28T14:16:27.532792+0000","last_fullsized":"2024-05-28T14:16:27.532792+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:37:33.489101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":66,"num_read_kb":262146,"num_write":97,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077572+0000","last_change":"2024-05-28T14:14:42.747379+0000","last_active":"2024-05-28T14:15:06.077572+0000","last_peered":"2024-05-28T14:15:06.077572+0000","last_clean":"2024-05-28T14:15:06.077572+0000","last_became_active":"2024-05-28T14:14:42.747127+0000","last_became_peered":"2024-05-28T14:14:42.747127+0000","last_unstale":"2024-05-28T14:15:06.077572+0000","last_undegraded":"2024-05-28T14:15:06.077572+0000","last_fullsized":"2024-05-28T14:15:06.077572+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:10:35.041686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"134'85","reported_seq":138,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.536903+0000","last_change":"2024-05-28T14:14:43.757145+0000","last_active":"2024-05-28T14:16:27.536903+0000","last_peered":"2024-05-28T14:16:27.536903+0000","last_clean":"2024-05-28T14:16:27.536903+0000","last_became_active":"2024-05-28T14:14:43.756803+0000","last_became_peered":"2024-05-28T14:14:43.756803+0000","last_unstale":"2024-05-28T14:16:27.536903+0000","last_undegraded":"2024-05-28T14:16:27.536903+0000","last_fullsized":"2024-05-28T14:16:27.536903+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:30:48.180755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188743680,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":85,"num_write_kb":266240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"134'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013914+0000","last_change":"2024-05-28T14:14:42.746521+0000","last_active":"2024-05-28T14:15:06.013914+0000","last_peered":"2024-05-28T14:15:06.013914+0000","last_clean":"2024-05-28T14:15:06.013914+0000","last_became_active":"2024-05-28T14:14:42.746243+0000","last_became_peered":"2024-05-28T14:14:42.746243+0000","last_unstale":"2024-05-28T14:15:06.013914+0000","last_undegraded":"2024-05-28T14:15:06.013914+0000","last_fullsized":"2024-05-28T14:15:06.013914+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:44.045231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"134'89","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.385831+0000","last_change":"2024-05-28T14:14:43.751232+0000","last_active":"2024-05-28T14:16:27.385831+0000","last_peered":"2024-05-28T14:16:27.385831+0000","last_clean":"2024-05-28T14:16:27.385831+0000","last_became_active":"2024-05-28T14:14:43.750768+0000","last_became_peered":"2024-05-28T14:14:43.750768+0000","last_unstale":"2024-05-28T14:16:27.385831+0000","last_undegraded":"2024-05-28T14:16:27.385831+0000","last_fullsized":"2024-05-28T14:16:27.385831+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:37:33.489101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":41,"num_read_kb":167936,"num_write":89,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969503+0000","last_change":"2024-05-28T14:14:42.745263+0000","last_active":"2024-05-28T14:15:05.969503+0000","last_peered":"2024-05-28T14:15:05.969503+0000","last_clean":"2024-05-28T14:15:05.969503+0000","last_became_active":"2024-05-28T14:14:42.744565+0000","last_became_peered":"2024-05-28T14:14:42.744565+0000","last_unstale":"2024-05-28T14:15:05.969503+0000","last_undegraded":"2024-05-28T14:15:05.969503+0000","last_fullsized":"2024-05-28T14:15:05.969503+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:59.384784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"134'81","reported_seq":180,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.491125+0000","last_change":"2024-05-28T14:14:43.760442+0000","last_active":"2024-05-28T14:16:27.491125+0000","last_peered":"2024-05-28T14:16:27.491125+0000","last_clean":"2024-05-28T14:16:27.491125+0000","last_became_active":"2024-05-28T14:14:43.760106+0000","last_became_peered":"2024-05-28T14:14:43.760106+0000","last_unstale":"2024-05-28T14:16:27.491125+0000","last_undegraded":"2024-05-28T14:16:27.491125+0000","last_fullsized":"2024-05-28T14:16:27.491125+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:56:47.127973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":80,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"134'25","reported_seq":43,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.001817+0000","last_change":"2024-05-28T14:14:42.735627+0000","last_active":"2024-05-28T14:16:27.001817+0000","last_peered":"2024-05-28T14:16:27.001817+0000","last_clean":"2024-05-28T14:16:27.001817+0000","last_became_active":"2024-05-28T14:14:42.734095+0000","last_became_peered":"2024-05-28T14:14:42.734095+0000","last_unstale":"2024-05-28T14:16:27.001817+0000","last_undegraded":"2024-05-28T14:16:27.001817+0000","last_fullsized":"2024-05-28T14:16:27.001817+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:07.310371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"134'87","reported_seq":170,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.446414+0000","last_change":"2024-05-28T14:14:43.768144+0000","last_active":"2024-05-28T14:16:27.446414+0000","last_peered":"2024-05-28T14:16:27.446414+0000","last_clean":"2024-05-28T14:16:27.446414+0000","last_became_active":"2024-05-28T14:14:43.766334+0000","last_became_peered":"2024-05-28T14:14:43.766334+0000","last_unstale":"2024-05-28T14:16:27.446414+0000","last_undegraded":"2024-05-28T14:16:27.446414+0000","last_fullsized":"2024-05-28T14:16:27.446414+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:56:51.599701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":86,"num_write_kb":274434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"134'457","reported_seq":475,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:33.919527+0000","last_change":"2024-05-28T14:14:42.744568+0000","last_active":"2024-05-28T14:16:33.919527+0000","last_peered":"2024-05-28T14:16:33.919527+0000","last_clean":"2024-05-28T14:16:33.919527+0000","last_became_active":"2024-05-28T14:14:42.744240+0000","last_became_peered":"2024-05-28T14:14:42.744240+0000","last_unstale":"2024-05-28T14:16:33.919527+0000","last_undegraded":"2024-05-28T14:16:33.919527+0000","last_fullsized":"2024-05-28T14:16:33.919527+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:25:02.568973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1780759,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":455,"num_write_kb":1899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"134'119","reported_seq":189,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.503726+0000","last_change":"2024-05-28T14:14:43.758776+0000","last_active":"2024-05-28T14:16:27.503726+0000","last_peered":"2024-05-28T14:16:27.503726+0000","last_clean":"2024-05-28T14:16:27.503726+0000","last_became_active":"2024-05-28T14:14:43.758474+0000","last_became_peered":"2024-05-28T14:14:43.758474+0000","last_unstale":"2024-05-28T14:16:27.503726+0000","last_undegraded":"2024-05-28T14:16:27.503726+0000","last_fullsized":"2024-05-28T14:16:27.503726+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:00:34.527970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":54,"num_read_kb":212994,"num_write":119,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973982+0000","last_change":"2024-05-28T14:14:42.737773+0000","last_active":"2024-05-28T14:15:05.973982+0000","last_peered":"2024-05-28T14:15:05.973982+0000","last_clean":"2024-05-28T14:15:05.973982+0000","last_became_active":"2024-05-28T14:14:42.737442+0000","last_became_peered":"2024-05-28T14:14:42.737442+0000","last_unstale":"2024-05-28T14:15:05.973982+0000","last_undegraded":"2024-05-28T14:15:05.973982+0000","last_fullsized":"2024-05-28T14:15:05.973982+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:45:49.229094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"134'106","reported_seq":183,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.525309+0000","last_change":"2024-05-28T14:14:43.762668+0000","last_active":"2024-05-28T14:16:27.525309+0000","last_peered":"2024-05-28T14:16:27.525309+0000","last_clean":"2024-05-28T14:16:27.525309+0000","last_became_active":"2024-05-28T14:14:43.762049+0000","last_became_peered":"2024-05-28T14:14:43.762049+0000","last_unstale":"2024-05-28T14:16:27.525309+0000","last_undegraded":"2024-05-28T14:16:27.525309+0000","last_fullsized":"2024-05-28T14:16:27.525309+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:57:46.984712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":106,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"134'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077675+0000","last_change":"2024-05-28T14:14:42.746077+0000","last_active":"2024-05-28T14:15:06.077675+0000","last_peered":"2024-05-28T14:15:06.077675+0000","last_clean":"2024-05-28T14:15:06.077675+0000","last_became_active":"2024-05-28T14:14:42.745811+0000","last_became_peered":"2024-05-28T14:14:42.745811+0000","last_unstale":"2024-05-28T14:15:06.077675+0000","last_undegraded":"2024-05-28T14:15:06.077675+0000","last_fullsized":"2024-05-28T14:15:06.077675+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:26:07.187114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"134'118","reported_seq":206,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.507583+0000","last_change":"2024-05-28T14:14:43.768732+0000","last_active":"2024-05-28T14:16:27.507583+0000","last_peered":"2024-05-28T14:16:27.507583+0000","last_clean":"2024-05-28T14:16:27.507583+0000","last_became_active":"2024-05-28T14:14:43.768334+0000","last_became_peered":"2024-05-28T14:14:43.768334+0000","last_unstale":"2024-05-28T14:16:27.507583+0000","last_undegraded":"2024-05-28T14:16:27.507583+0000","last_fullsized":"2024-05-28T14:16:27.507583+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:01:12.201409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852724,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":72,"num_read_kb":282627,"num_write":118,"num_write_kb":360450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"131'8","reported_seq":26,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.975292+0000","last_change":"2024-05-28T14:14:42.742861+0000","last_active":"2024-05-28T14:15:05.975292+0000","last_peered":"2024-05-28T14:15:05.975292+0000","last_clean":"2024-05-28T14:15:05.975292+0000","last_became_active":"2024-05-28T14:14:42.742518+0000","last_became_peered":"2024-05-28T14:14:42.742518+0000","last_unstale":"2024-05-28T14:15:05.975292+0000","last_undegraded":"2024-05-28T14:15:05.975292+0000","last_fullsized":"2024-05-28T14:15:05.975292+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:43:29.312316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"134'86","reported_seq":127,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.551366+0000","last_change":"2024-05-28T14:14:43.762081+0000","last_active":"2024-05-28T14:16:27.551366+0000","last_peered":"2024-05-28T14:16:27.551366+0000","last_clean":"2024-05-28T14:16:27.551366+0000","last_became_active":"2024-05-28T14:14:43.761775+0000","last_became_peered":"2024-05-28T14:14:43.761775+0000","last_unstale":"2024-05-28T14:16:27.551366+0000","last_undegraded":"2024-05-28T14:16:27.551366+0000","last_fullsized":"2024-05-28T14:16:27.551366+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:07:11.105627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176160768,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":25,"num_read_kb":102400,"num_write":86,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013893+0000","last_change":"2024-05-28T14:14:42.746381+0000","last_active":"2024-05-28T14:15:06.013893+0000","last_peered":"2024-05-28T14:15:06.013893+0000","last_clean":"2024-05-28T14:15:06.013893+0000","last_became_active":"2024-05-28T14:14:42.746098+0000","last_became_peered":"2024-05-28T14:14:42.746098+0000","last_unstale":"2024-05-28T14:15:06.013893+0000","last_undegraded":"2024-05-28T14:15:06.013893+0000","last_fullsized":"2024-05-28T14:15:06.013893+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:04.859086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"134'129","reported_seq":227,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.553615+0000","last_change":"2024-05-28T14:14:43.756453+0000","last_active":"2024-05-28T14:16:27.553615+0000","last_peered":"2024-05-28T14:16:27.553615+0000","last_clean":"2024-05-28T14:16:27.553615+0000","last_became_active":"2024-05-28T14:14:43.756138+0000","last_became_peered":"2024-05-28T14:14:43.756138+0000","last_unstale":"2024-05-28T14:16:27.553615+0000","last_undegraded":"2024-05-28T14:16:27.553615+0000","last_fullsized":"2024-05-28T14:16:27.553615+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:07.658669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":129,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"131'2","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971765+0000","last_change":"2024-05-28T14:14:42.742055+0000","last_active":"2024-05-28T14:15:05.971765+0000","last_peered":"2024-05-28T14:15:05.971765+0000","last_clean":"2024-05-28T14:15:05.971765+0000","last_became_active":"2024-05-28T14:14:42.741754+0000","last_became_peered":"2024-05-28T14:14:42.741754+0000","last_unstale":"2024-05-28T14:15:05.971765+0000","last_undegraded":"2024-05-28T14:15:05.971765+0000","last_fullsized":"2024-05-28T14:15:05.971765+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:25.036897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"134'101","reported_seq":180,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.538137+0000","last_change":"2024-05-28T14:14:43.758320+0000","last_active":"2024-05-28T14:16:27.538137+0000","last_peered":"2024-05-28T14:16:27.538137+0000","last_clean":"2024-05-28T14:16:27.538137+0000","last_became_active":"2024-05-28T14:14:43.757996+0000","last_became_peered":"2024-05-28T14:14:43.757996+0000","last_unstale":"2024-05-28T14:16:27.538137+0000","last_undegraded":"2024-05-28T14:16:27.538137+0000","last_fullsized":"2024-05-28T14:16:27.538137+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:18.151246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":63,"num_read_kb":241668,"num_write":101,"num_write_kb":299010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.974449+0000","last_change":"2024-05-28T14:14:42.734922+0000","last_active":"2024-05-28T14:15:05.974449+0000","last_peered":"2024-05-28T14:15:05.974449+0000","last_clean":"2024-05-28T14:15:05.974449+0000","last_became_active":"2024-05-28T14:14:42.734647+0000","last_became_peered":"2024-05-28T14:14:42.734647+0000","last_unstale":"2024-05-28T14:15:05.974449+0000","last_undegraded":"2024-05-28T14:15:05.974449+0000","last_fullsized":"2024-05-28T14:15:05.974449+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:36:34.326176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"134'107","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.571982+0000","last_change":"2024-05-28T14:14:43.764186+0000","last_active":"2024-05-28T14:16:27.571982+0000","last_peered":"2024-05-28T14:16:27.571982+0000","last_clean":"2024-05-28T14:16:27.571982+0000","last_became_active":"2024-05-28T14:14:43.762407+0000","last_became_peered":"2024-05-28T14:14:43.762407+0000","last_unstale":"2024-05-28T14:16:27.571982+0000","last_undegraded":"2024-05-28T14:16:27.571982+0000","last_fullsized":"2024-05-28T14:16:27.571982+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:07:34.099826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":82,"num_read_kb":327682,"num_write":107,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013781+0000","last_change":"2024-05-28T14:14:42.733737+0000","last_active":"2024-05-28T14:15:06.013781+0000","last_peered":"2024-05-28T14:15:06.013781+0000","last_clean":"2024-05-28T14:15:06.013781+0000","last_became_active":"2024-05-28T14:14:42.733393+0000","last_became_peered":"2024-05-28T14:14:42.733393+0000","last_unstale":"2024-05-28T14:15:06.013781+0000","last_undegraded":"2024-05-28T14:15:06.013781+0000","last_fullsized":"2024-05-28T14:15:06.013781+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:13.036848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"134'103","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.555195+0000","last_change":"2024-05-28T14:14:43.767748+0000","last_active":"2024-05-28T14:16:27.555195+0000","last_peered":"2024-05-28T14:16:27.555195+0000","last_clean":"2024-05-28T14:16:27.555195+0000","last_became_active":"2024-05-28T14:14:43.767351+0000","last_became_peered":"2024-05-28T14:14:43.767351+0000","last_unstale":"2024-05-28T14:16:27.555195+0000","last_undegraded":"2024-05-28T14:16:27.555195+0000","last_fullsized":"2024-05-28T14:16:27.555195+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:27:11.131902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":85,"num_read_kb":348160,"num_write":103,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"134'3","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013626+0000","last_change":"2024-05-28T14:14:42.746787+0000","last_active":"2024-05-28T14:15:06.013626+0000","last_peered":"2024-05-28T14:15:06.013626+0000","last_clean":"2024-05-28T14:15:06.013626+0000","last_became_active":"2024-05-28T14:14:42.746273+0000","last_became_peered":"2024-05-28T14:14:42.746273+0000","last_unstale":"2024-05-28T14:15:06.013626+0000","last_undegraded":"2024-05-28T14:15:06.013626+0000","last_fullsized":"2024-05-28T14:15:06.013626+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:38.279660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"134'111","reported_seq":194,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.532473+0000","last_change":"2024-05-28T14:14:43.771048+0000","last_active":"2024-05-28T14:16:27.532473+0000","last_peered":"2024-05-28T14:16:27.532473+0000","last_clean":"2024-05-28T14:16:27.532473+0000","last_became_active":"2024-05-28T14:14:43.770751+0000","last_became_peered":"2024-05-28T14:14:43.770751+0000","last_unstale":"2024-05-28T14:16:27.532473+0000","last_undegraded":"2024-05-28T14:16:27.532473+0000","last_fullsized":"2024-05-28T14:16:27.532473+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:34:09.617785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272629760,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":111,"num_write_kb":348163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969248+0000","last_change":"2024-05-28T14:14:42.742397+0000","last_active":"2024-05-28T14:15:05.969248+0000","last_peered":"2024-05-28T14:15:05.969248+0000","last_clean":"2024-05-28T14:15:05.969248+0000","last_became_active":"2024-05-28T14:14:42.742104+0000","last_became_peered":"2024-05-28T14:14:42.742104+0000","last_unstale":"2024-05-28T14:15:05.969248+0000","last_undegraded":"2024-05-28T14:15:05.969248+0000","last_fullsized":"2024-05-28T14:15:05.969248+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:21.703678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"134'111","reported_seq":176,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.461921+0000","last_change":"2024-05-28T14:14:43.756230+0000","last_active":"2024-05-28T14:16:27.461921+0000","last_peered":"2024-05-28T14:16:27.461921+0000","last_clean":"2024-05-28T14:16:27.461921+0000","last_became_active":"2024-05-28T14:14:43.755945+0000","last_became_peered":"2024-05-28T14:14:43.755945+0000","last_unstale":"2024-05-28T14:16:27.461921+0000","last_undegraded":"2024-05-28T14:16:27.461921+0000","last_fullsized":"2024-05-28T14:16:27.461921+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:30:48.180755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":49,"num_read_kb":196609,"num_write":111,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969005+0000","last_change":"2024-05-28T14:14:42.737496+0000","last_active":"2024-05-28T14:15:05.969005+0000","last_peered":"2024-05-28T14:15:05.969005+0000","last_clean":"2024-05-28T14:15:05.969005+0000","last_became_active":"2024-05-28T14:14:42.737194+0000","last_became_peered":"2024-05-28T14:14:42.737194+0000","last_unstale":"2024-05-28T14:15:05.969005+0000","last_undegraded":"2024-05-28T14:15:05.969005+0000","last_fullsized":"2024-05-28T14:15:05.969005+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:56:10.665924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"134'110","reported_seq":190,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.515934+0000","last_change":"2024-05-28T14:14:43.758200+0000","last_active":"2024-05-28T14:16:27.515934+0000","last_peered":"2024-05-28T14:16:27.515934+0000","last_clean":"2024-05-28T14:16:27.515934+0000","last_became_active":"2024-05-28T14:14:43.757875+0000","last_became_peered":"2024-05-28T14:14:43.757875+0000","last_unstale":"2024-05-28T14:16:27.515934+0000","last_undegraded":"2024-05-28T14:16:27.515934+0000","last_fullsized":"2024-05-28T14:16:27.515934+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:53.917272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":64,"num_read_kb":253954,"num_write":110,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"134'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971403+0000","last_change":"2024-05-28T14:14:42.738809+0000","last_active":"2024-05-28T14:15:05.971403+0000","last_peered":"2024-05-28T14:15:05.971403+0000","last_clean":"2024-05-28T14:15:05.971403+0000","last_became_active":"2024-05-28T14:14:42.738494+0000","last_became_peered":"2024-05-28T14:14:42.738494+0000","last_unstale":"2024-05-28T14:15:05.971403+0000","last_undegraded":"2024-05-28T14:15:05.971403+0000","last_fullsized":"2024-05-28T14:15:05.971403+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:17.145363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"134'118","reported_seq":174,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.515735+0000","last_change":"2024-05-28T14:14:43.770616+0000","last_active":"2024-05-28T14:16:27.515735+0000","last_peered":"2024-05-28T14:16:27.515735+0000","last_clean":"2024-05-28T14:16:27.515735+0000","last_became_active":"2024-05-28T14:14:43.770253+0000","last_became_peered":"2024-05-28T14:14:43.770253+0000","last_unstale":"2024-05-28T14:16:27.515735+0000","last_undegraded":"2024-05-28T14:16:27.515735+0000","last_fullsized":"2024-05-28T14:16:27.515735+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:49:45.733085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":40,"num_read_kb":163840,"num_write":118,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"131'2","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.974495+0000","last_change":"2024-05-28T14:14:42.746355+0000","last_active":"2024-05-28T14:15:05.974495+0000","last_peered":"2024-05-28T14:15:05.974495+0000","last_clean":"2024-05-28T14:15:05.974495+0000","last_became_active":"2024-05-28T14:14:42.745926+0000","last_became_peered":"2024-05-28T14:14:42.745926+0000","last_unstale":"2024-05-28T14:15:05.974495+0000","last_undegraded":"2024-05-28T14:15:05.974495+0000","last_fullsized":"2024-05-28T14:15:05.974495+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:54:47.273264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"134'107","reported_seq":214,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.531994+0000","last_change":"2024-05-28T14:15:06.178022+0000","last_active":"2024-05-28T14:16:27.531994+0000","last_peered":"2024-05-28T14:16:27.531994+0000","last_clean":"2024-05-28T14:16:27.531994+0000","last_became_active":"2024-05-28T14:15:05.975545+0000","last_became_peered":"2024-05-28T14:15:05.975545+0000","last_unstale":"2024-05-28T14:16:27.531994+0000","last_undegraded":"2024-05-28T14:16:27.531994+0000","last_fullsized":"2024-05-28T14:16:27.531994+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:56:51.599701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":85,"num_read_kb":348160,"num_write":107,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"134'41","reported_seq":59,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:25.782834+0000","last_change":"2024-05-28T14:14:42.749733+0000","last_active":"2024-05-28T14:16:25.782834+0000","last_peered":"2024-05-28T14:16:25.782834+0000","last_clean":"2024-05-28T14:16:25.782834+0000","last_became_active":"2024-05-28T14:14:42.749348+0000","last_became_peered":"2024-05-28T14:14:42.749348+0000","last_unstale":"2024-05-28T14:16:25.782834+0000","last_undegraded":"2024-05-28T14:16:25.782834+0000","last_fullsized":"2024-05-28T14:16:25.782834+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:17.145363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"134'86","reported_seq":202,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.431859+0000","last_change":"2024-05-28T14:14:43.764008+0000","last_active":"2024-05-28T14:16:27.431859+0000","last_peered":"2024-05-28T14:16:27.431859+0000","last_clean":"2024-05-28T14:16:27.431859+0000","last_became_active":"2024-05-28T14:14:43.762648+0000","last_became_peered":"2024-05-28T14:14:43.762648+0000","last_unstale":"2024-05-28T14:16:27.431859+0000","last_undegraded":"2024-05-28T14:16:27.431859+0000","last_fullsized":"2024-05-28T14:16:27.431859+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:35:31.221160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":86,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077854+0000","last_change":"2024-05-28T14:14:42.744189+0000","last_active":"2024-05-28T14:15:06.077854+0000","last_peered":"2024-05-28T14:15:06.077854+0000","last_clean":"2024-05-28T14:15:06.077854+0000","last_became_active":"2024-05-28T14:14:42.743858+0000","last_became_peered":"2024-05-28T14:14:42.743858+0000","last_unstale":"2024-05-28T14:15:06.077854+0000","last_undegraded":"2024-05-28T14:15:06.077854+0000","last_fullsized":"2024-05-28T14:15:06.077854+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:03:34.587886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"134'96","reported_seq":176,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.471051+0000","last_change":"2024-05-28T14:15:06.036082+0000","last_active":"2024-05-28T14:16:27.471051+0000","last_peered":"2024-05-28T14:16:27.471051+0000","last_clean":"2024-05-28T14:16:27.471051+0000","last_became_active":"2024-05-28T14:15:05.976669+0000","last_became_peered":"2024-05-28T14:15:05.976669+0000","last_unstale":"2024-05-28T14:16:27.471051+0000","last_undegraded":"2024-05-28T14:16:27.471051+0000","last_fullsized":"2024-05-28T14:16:27.471051+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:57:19.311096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132541,"num_objects":50,"num_object_clones":1,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":1,"num_read":52,"num_read_kb":208897,"num_write":94,"num_write_kb":282626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8388730,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971591+0000","last_change":"2024-05-28T14:14:42.748521+0000","last_active":"2024-05-28T14:15:05.971591+0000","last_peered":"2024-05-28T14:15:05.971591+0000","last_clean":"2024-05-28T14:15:05.971591+0000","last_became_active":"2024-05-28T14:14:42.748193+0000","last_became_peered":"2024-05-28T14:14:42.748193+0000","last_unstale":"2024-05-28T14:15:05.971591+0000","last_undegraded":"2024-05-28T14:15:05.971591+0000","last_fullsized":"2024-05-28T14:15:05.971591+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:48:29.637714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"134'116","reported_seq":193,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.489037+0000","last_change":"2024-05-28T14:14:43.767848+0000","last_active":"2024-05-28T14:16:27.489037+0000","last_peered":"2024-05-28T14:16:27.489037+0000","last_clean":"2024-05-28T14:16:27.489037+0000","last_became_active":"2024-05-28T14:14:43.767552+0000","last_became_peered":"2024-05-28T14:14:43.767552+0000","last_unstale":"2024-05-28T14:16:27.489037+0000","last_undegraded":"2024-05-28T14:16:27.489037+0000","last_fullsized":"2024-05-28T14:16:27.489037+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:09:18.039029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":285212672,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":116,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077717+0000","last_change":"2024-05-28T14:14:42.746605+0000","last_active":"2024-05-28T14:15:06.077717+0000","last_peered":"2024-05-28T14:15:06.077717+0000","last_clean":"2024-05-28T14:15:06.077717+0000","last_became_active":"2024-05-28T14:14:42.746338+0000","last_became_peered":"2024-05-28T14:14:42.746338+0000","last_unstale":"2024-05-28T14:15:06.077717+0000","last_undegraded":"2024-05-28T14:15:06.077717+0000","last_fullsized":"2024-05-28T14:15:06.077717+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:51:27.080888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"134'129","reported_seq":235,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.540996+0000","last_change":"2024-05-28T14:14:43.760455+0000","last_active":"2024-05-28T14:16:27.540996+0000","last_peered":"2024-05-28T14:16:27.540996+0000","last_clean":"2024-05-28T14:16:27.540996+0000","last_became_active":"2024-05-28T14:14:43.760063+0000","last_became_peered":"2024-05-28T14:14:43.760063+0000","last_unstale":"2024-05-28T14:16:27.540996+0000","last_undegraded":"2024-05-28T14:16:27.540996+0000","last_fullsized":"2024-05-28T14:16:27.540996+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:50:53.753974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272629760,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":90,"num_read_kb":356355,"num_write":129,"num_write_kb":393217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973106+0000","last_change":"2024-05-28T14:14:42.749618+0000","last_active":"2024-05-28T14:15:05.973106+0000","last_peered":"2024-05-28T14:15:05.973106+0000","last_clean":"2024-05-28T14:15:05.973106+0000","last_became_active":"2024-05-28T14:14:42.749259+0000","last_became_peered":"2024-05-28T14:14:42.749259+0000","last_unstale":"2024-05-28T14:15:05.973106+0000","last_undegraded":"2024-05-28T14:15:05.973106+0000","last_fullsized":"2024-05-28T14:15:05.973106+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:44:02.935848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"134'114","reported_seq":210,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.564959+0000","last_change":"2024-05-28T14:14:43.761804+0000","last_active":"2024-05-28T14:16:27.564959+0000","last_peered":"2024-05-28T14:16:27.564959+0000","last_clean":"2024-05-28T14:16:27.564959+0000","last_became_active":"2024-05-28T14:14:43.761382+0000","last_became_peered":"2024-05-28T14:14:43.761382+0000","last_unstale":"2024-05-28T14:16:27.564959+0000","last_undegraded":"2024-05-28T14:16:27.564959+0000","last_fullsized":"2024-05-28T14:16:27.564959+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:48:50.641484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":114,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"134'31","reported_seq":79,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:26.862832+0000","last_change":"2024-05-28T14:14:42.739854+0000","last_active":"2024-05-28T14:16:26.862832+0000","last_peered":"2024-05-28T14:16:26.862832+0000","last_clean":"2024-05-28T14:16:26.862832+0000","last_became_active":"2024-05-28T14:14:42.738369+0000","last_became_peered":"2024-05-28T14:14:42.738369+0000","last_unstale":"2024-05-28T14:16:26.862832+0000","last_undegraded":"2024-05-28T14:16:26.862832+0000","last_fullsized":"2024-05-28T14:16:26.862832+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:34:08.611902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":31,"num_write":30,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"134'116","reported_seq":215,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.484442+0000","last_change":"2024-05-28T14:14:43.765205+0000","last_active":"2024-05-28T14:16:27.484442+0000","last_peered":"2024-05-28T14:16:27.484442+0000","last_clean":"2024-05-28T14:16:27.484442+0000","last_became_active":"2024-05-28T14:14:43.764879+0000","last_became_peered":"2024-05-28T14:14:43.764879+0000","last_unstale":"2024-05-28T14:16:27.484442+0000","last_undegraded":"2024-05-28T14:16:27.484442+0000","last_fullsized":"2024-05-28T14:16:27.484442+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:07.658669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":116,"num_write_kb":372736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971518+0000","last_change":"2024-05-28T14:14:42.730611+0000","last_active":"2024-05-28T14:15:05.971518+0000","last_peered":"2024-05-28T14:15:05.971518+0000","last_clean":"2024-05-28T14:15:05.971518+0000","last_became_active":"2024-05-28T14:14:42.730333+0000","last_became_peered":"2024-05-28T14:14:42.730333+0000","last_unstale":"2024-05-28T14:15:05.971518+0000","last_undegraded":"2024-05-28T14:15:05.971518+0000","last_fullsized":"2024-05-28T14:15:05.971518+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:05:55.551707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"134'101","reported_seq":167,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.469784+0000","last_change":"2024-05-28T14:14:43.761743+0000","last_active":"2024-05-28T14:16:27.469784+0000","last_peered":"2024-05-28T14:16:27.469784+0000","last_clean":"2024-05-28T14:16:27.469784+0000","last_became_active":"2024-05-28T14:14:43.760828+0000","last_became_peered":"2024-05-28T14:14:43.760828+0000","last_unstale":"2024-05-28T14:16:27.469784+0000","last_undegraded":"2024-05-28T14:16:27.469784+0000","last_fullsized":"2024-05-28T14:16:27.469784+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:44:21.055874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":101,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"131'2","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973324+0000","last_change":"2024-05-28T14:14:42.751868+0000","last_active":"2024-05-28T14:15:05.973324+0000","last_peered":"2024-05-28T14:15:05.973324+0000","last_clean":"2024-05-28T14:15:05.973324+0000","last_became_active":"2024-05-28T14:14:42.751447+0000","last_became_peered":"2024-05-28T14:14:42.751447+0000","last_unstale":"2024-05-28T14:15:05.973324+0000","last_undegraded":"2024-05-28T14:15:05.973324+0000","last_fullsized":"2024-05-28T14:15:05.973324+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:28:17.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":383,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.972861+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:15:05.972861+0000","last_peered":"2024-05-28T14:15:05.972861+0000","last_clean":"2024-05-28T14:15:05.972861+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:15:05.972861+0000","last_undegraded":"2024-05-28T14:15:05.972861+0000","last_fullsized":"2024-05-28T14:15:05.972861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":15082718397,"num_objects":3603,"num_object_clones":1,"num_object_copies":7206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":1,"num_read":4365,"num_read_kb":17629245,"num_write":6581,"num_write_kb":20676651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":50331770,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":30165483520,"data_stored":30165436794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6590,"ondisk_log_size":6590,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1787120,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":30,"num_read_kb":31,"num_write":636,"num_write_kb":1988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3637248,"data_stored":3574240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":588,"ondisk_log_size":588,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575073,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3272720,"kb_used_data":3133320,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":91099120,"statfs":{"total":96636764160,"available":93285498880,"internally_reserved":0,"allocated":3208519680,"data_stored":3207557922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":142735818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575073,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4222476,"kb_used_data":4078916,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90149364,"statfs":{"total":96636764160,"available":92312948736,"internally_reserved":0,"allocated":4176809984,"data_stored":4175856883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575075,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3121336,"kb_used_data":2995824,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91250504,"statfs":{"total":96636764160,"available":93440516096,"internally_reserved":0,"allocated":3067723776,"data_stored":3066774417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":128514502},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575074,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4572916,"kb_used_data":4433516,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":89798924,"statfs":{"total":96636764160,"available":91954098176,"internally_reserved":0,"allocated":4539920384,"data_stored":4538974892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575073,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3790716,"kb_used_data":3642996,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":90581124,"statfs":{"total":96636764160,"available":92755070976,"internally_reserved":0,"allocated":3730427904,"data_stored":3729479880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":151255496},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575074,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4906484,"kb_used_data":4740716,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":89465356,"statfs":{"total":96636764160,"available":91612524544,"internally_reserved":0,"allocated":4854493184,"data_stored":4853548245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575073,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3483436,"kb_used_data":3339876,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90888404,"statfs":{"total":96636764160,"available":93069725696,"internally_reserved":0,"allocated":3420033024,"data_stored":3419095661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575074,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3250484,"kb_used_data":3111084,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":91121356,"statfs":{"total":96636764160,"available":93308268544,"internally_reserved":0,"allocated":3185750016,"data_stored":3184804998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1798144,"data_stored":1786322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1802240,"data_stored":1781397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3183484928,"data_stored":3183476989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3418361856,"data_stored":3418358020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4852813824,"data_stored":4852809988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3728740352,"data_stored":3728736516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4538245120,"data_stored":4538237181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3066044416,"data_stored":3066036664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4173340672,"data_stored":4173332920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3204452352,"data_stored":3204448516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:16:51.185 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:16:51.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.381+0000 7f6a5b400640 1 Processor -- start 2024-05-28T14:16:51.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.381+0000 7f6a5b400640 1 -- start start 2024-05-28T14:16:51.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a5b400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a54067810 0x7f6a5415ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:51.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a5b400640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a5406eaa0 con 0x7f6a540d0420 2024-05-28T14:16:51.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a5b400640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a5406ed80 con 0x7f6a540b42e0 2024-05-28T14:16:51.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a5b400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6a5406f060 con 0x7f6a54067810 2024-05-28T14:16:51.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a58a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f6a540d0420 0x7f6a540b3bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53044/0 (socket says 172.21.15.42:53044) 2024-05-28T14:16:51.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a58a00640 1 -- 172.21.15.42:0/2094291959 learned_addr learned my addr 172.21.15.42:0/2094291959 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:51.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a59400640 1 --2- 172.21.15.42:0/2094291959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a54067810 0x7f6a5415ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:51.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 522867476 0 0) 0x7f6a5406eaa0 con 0x7f6a540d0420 2024-05-28T14:16:51.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a3c003530 con 0x7f6a540d0420 2024-05-28T14:16:51.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 473730657 0 0) 0x7f6a5406ed80 con 0x7f6a540b42e0 2024-05-28T14:16:51.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a5406eaa0 con 0x7f6a540b42e0 2024-05-28T14:16:51.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 992614971 0 0) 0x7f6a3c003530 con 0x7f6a540d0420 2024-05-28T14:16:51.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a54067810 msgr2=0x7f6a5415ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:51.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 --2- 172.21.15.42:0/2094291959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a54067810 0x7f6a5415ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 >> v1:172.21.15.63:6789/0 conn(0x7f6a540b42e0 legacy=0x7f6a5406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5406f340 con 0x7f6a540d0420 2024-05-28T14:16:51.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f6a48001680 con 0x7f6a540d0420 2024-05-28T14:16:51.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 >> v1:172.21.15.42:6789/0 conn(0x7f6a540d0420 legacy=0x7f6a540b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 --2- 172.21.15.42:0/2094291959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3c003b80 0x7f6a3c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:51.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 --2- 172.21.15.42:0/2094291959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a3c004810 0x7f6a3c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:51.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 --2- 172.21.15.42:0/2094291959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a3c0051d0 0x7f6a3c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:51.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5406f060 con 0x7f6a3c003b80 2024-05-28T14:16:51.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c00c040 con 0x7f6a3c004810 2024-05-28T14:16:51.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3c00c240 con 0x7f6a3c0051d0 2024-05-28T14:16:51.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a59400640 1 --2- 172.21.15.42:0/2094291959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a54067810 0x7f6a5415ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:16:51.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a58a00640 1 --2- 172.21.15.42:0/2094291959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3c003b80 0x7f6a3c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:51.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53e00640 1 --2- 172.21.15.42:0/2094291959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a3c004810 0x7f6a3c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:51.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a59400640 1 --2- 172.21.15.42:0/2094291959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a3c0051d0 0x7f6a3c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:51.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a58a00640 1 -- 172.21.15.42:0/2094291959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a3c0051d0 msgr2=0x7f6a3c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:51.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a58a00640 1 --2- 172.21.15.42:0/2094291959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a3c0051d0 0x7f6a3c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a58a00640 1 -- 172.21.15.42:0/2094291959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a3c004810 msgr2=0x7f6a3c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:51.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a58a00640 1 --2- 172.21.15.42:0/2094291959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a3c004810 0x7f6a3c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a58a00640 1 -- 172.21.15.42:0/2094291959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3c003530 con 0x7f6a3c003b80 2024-05-28T14:16:51.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.385+0000 7f6a53e00640 1 --2- 172.21.15.42:0/2094291959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a3c004810 0x7f6a3c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:16:51.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.389+0000 7f6a58a00640 1 --2- 172.21.15.42:0/2094291959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3c003b80 0x7f6a3c003f80 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800c160 tx=0x7f6a4801cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f481a3b5067bc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:51.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.389+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48007cb0 con 0x7f6a3c003b80 2024-05-28T14:16:51.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.389+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a480a9020 con 0x7f6a3c003b80 2024-05-28T14:16:51.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.389+0000 7f6a53400640 1 -- 172.21.15.42:0/2094291959 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48005ae0 con 0x7f6a3c003b80 2024-05-28T14:16:51.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.389+0000 7f6a5b400640 1 -- 172.21.15.42:0/2094291959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3c003b80 msgr2=0x7f6a3c003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.389+0000 7f6a5b400640 1 --2- 172.21.15.42:0/2094291959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3c003b80 0x7f6a3c003f80 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800c160 tx=0x7f6a4801cc00 comp rx=0 tx=0).stop 2024-05-28T14:16:51.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.389+0000 7f6a59400640 1 -- 172.21.15.42:0/2094291959 reap_dead start 2024-05-28T14:16:51.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.389+0000 7f6a5b400640 1 -- 172.21.15.42:0/2094291959 shutdown_connections 2024-05-28T14:16:51.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.389+0000 7f6a5b400640 1 -- 172.21.15.42:0/2094291959 >> 172.21.15.42:0/2094291959 conn(0x7f6a540d0b40 msgr2=0x7f6a540d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:51.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.389+0000 7f6a5b400640 1 -- 172.21.15.42:0/2094291959 shutdown_connections 2024-05-28T14:16:51.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.389+0000 7f6a5b400640 1 -- 172.21.15.42:0/2094291959 wait complete. 2024-05-28T14:16:51.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.389+0000 7f6a5b400640 1 Processor -- start 2024-05-28T14:16:51.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.389+0000 7f6a5b400640 1 -- start start 2024-05-28T14:16:51.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a5b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3c003b80 0x7f6a5414e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:51.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a5b400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a5414ea70 0x7f6a54160e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:51.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a5b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a54163420 0x7f6a541638d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:51.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a5b400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5406cd30 con 0x7f6a3c003b80 2024-05-28T14:16:51.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a5b400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5406cbb0 con 0x7f6a54163420 2024-05-28T14:16:51.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a5b400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6a5406ceb0 con 0x7f6a5414ea70 2024-05-28T14:16:51.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a58a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3c003b80 0x7f6a5414e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:51.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a58a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3c003b80 0x7f6a5414e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37934/0 (socket says 172.21.15.42:37934) 2024-05-28T14:16:51.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a58a00640 1 -- 172.21.15.42:0/1921158405 learned_addr learned my addr 172.21.15.42:0/1921158405 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:51.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a53e00640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a5414ea70 0x7f6a54160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:51.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a59400640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a54163420 0x7f6a541638d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:51.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a58a00640 1 -- 172.21.15.42:0/1921158405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a5414ea70 msgr2=0x7f6a54160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a58a00640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a5414ea70 0x7f6a54160e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a58a00640 1 -- 172.21.15.42:0/1921158405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a54163420 msgr2=0x7f6a541638d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a58a00640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a54163420 0x7f6a541638d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a58a00640 1 -- 172.21.15.42:0/1921158405 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a541617d0 con 0x7f6a3c003b80 2024-05-28T14:16:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a59400640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a54163420 0x7f6a541638d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a58a00640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3c003b80 0x7f6a5414e140 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f6a480ab040 tx=0x7f6a3c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a50c00640 1 -- 172.21.15.42:0/1921158405 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a480047f0 con 0x7f6a3c003b80 2024-05-28T14:16:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a50c00640 1 -- 172.21.15.42:0/1921158405 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a48004990 con 0x7f6a3c003b80 2024-05-28T14:16:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a50c00640 1 -- 172.21.15.42:0/1921158405 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48004d60 con 0x7f6a3c003b80 2024-05-28T14:16:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a5b400640 1 -- 172.21.15.42:0/1921158405 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a54161ab0 con 0x7f6a3c003b80 2024-05-28T14:16:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a5b400640 1 -- 172.21.15.42:0/1921158405 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a54162070 con 0x7f6a3c003b80 2024-05-28T14:16:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.393+0000 7f6a5b400640 1 -- 172.21.15.42:0/1921158405 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a540af6f0 con 0x7f6a3c003b80 2024-05-28T14:16:51.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.397+0000 7f6a50c00640 1 -- 172.21.15.42:0/1921158405 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f6a480a56e0 con 0x7f6a3c003b80 2024-05-28T14:16:51.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.397+0000 7f6a50c00640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a3406a450 0x7f6a3406c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:51.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.397+0000 7f6a50c00640 1 -- 172.21.15.42:0/1921158405 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f6a4811f090 con 0x7f6a3c003b80 2024-05-28T14:16:51.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.397+0000 7f6a53e00640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a3406a450 0x7f6a3406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:51.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.397+0000 7f6a53e00640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a3406a450 0x7f6a3406c910 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f6a541facf0 tx=0x7f6a40009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:51.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.401+0000 7f6a50c00640 1 -- 172.21.15.42:0/1921158405 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6a48016910 con 0x7f6a3c003b80 2024-05-28T14:16:51.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.561+0000 7f6a5b400640 1 -- 172.21.15.42:0/1921158405 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a54162270 con 0x7f6a3406a450 2024-05-28T14:16:51.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.569+0000 7f6a50c00640 1 -- 172.21.15.42:0/1921158405 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+255065 (secure 0 0 0) 0x7f6a54162270 con 0x7f6a3406a450 2024-05-28T14:16:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:51.556 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:16:51.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.573+0000 7f6a5b400640 1 -- 172.21.15.42:0/1921158405 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a3406a450 msgr2=0x7f6a3406c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.573+0000 7f6a5b400640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a3406a450 0x7f6a3406c910 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f6a541facf0 tx=0x7f6a40009040 comp rx=0 tx=0).stop 2024-05-28T14:16:51.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.573+0000 7f6a5b400640 1 -- 172.21.15.42:0/1921158405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3c003b80 msgr2=0x7f6a5414e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.573+0000 7f6a5b400640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3c003b80 0x7f6a5414e140 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f6a480ab040 tx=0x7f6a3c005840 comp rx=0 tx=0).stop 2024-05-28T14:16:51.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.573+0000 7f6a5b400640 1 -- 172.21.15.42:0/1921158405 shutdown_connections 2024-05-28T14:16:51.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.573+0000 7f6a5b400640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f6a3406a450 0x7f6a3406c910 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.573+0000 7f6a5b400640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a54163420 0x7f6a541638d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.573+0000 7f6a5b400640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a5414ea70 0x7f6a54160e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.573+0000 7f6a5b400640 1 --2- 172.21.15.42:0/1921158405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3c003b80 0x7f6a5414e140 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.573+0000 7f6a5b400640 1 -- 172.21.15.42:0/1921158405 >> 172.21.15.42:0/1921158405 conn(0x7f6a540d0b40 msgr2=0x7f6a54150290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:51.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.573+0000 7f6a5b400640 1 -- 172.21.15.42:0/1921158405 shutdown_connections 2024-05-28T14:16:51.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.573+0000 7f6a5b400640 1 -- 172.21.15.42:0/1921158405 wait complete. 2024-05-28T14:16:51.575 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":688,"stamp":"2024-05-28T14:16:51.249025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15085096058,"num_objects":3628,"num_object_clones":1,"num_object_copies":7256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3628,"num_whiteouts":1,"num_read":4511,"num_read_kb":17629376,"num_write":7378,"num_write_kb":20681057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":50331770,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7775,"ondisk_log_size":7775,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":30620568,"kb_used_data":29476248,"kb_used_omap":12,"kb_used_meta":1144243,"kb_avail":724354152,"statfs":{"total":773094113280,"available":741738651648,"internally_reserved":0,"allocated":30183677952,"data_stored":30176092898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1171705394},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,9,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":173,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002732"},"pg_stats":[{"pgid":"19.3f","version":"134'94","reported_seq":188,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.570602+0000","last_change":"2024-05-28T14:14:43.766434+0000","last_active":"2024-05-28T14:16:49.570602+0000","last_peered":"2024-05-28T14:16:49.570602+0000","last_clean":"2024-05-28T14:16:49.570602+0000","last_became_active":"2024-05-28T14:14:43.766125+0000","last_became_peered":"2024-05-28T14:14:43.766125+0000","last_unstale":"2024-05-28T14:16:49.570602+0000","last_undegraded":"2024-05-28T14:16:49.570602+0000","last_fullsized":"2024-05-28T14:16:49.570602+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:50:53.753974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":77,"num_read_kb":315392,"num_write":85,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"134'120","reported_seq":212,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.572026+0000","last_change":"2024-05-28T14:14:43.769925+0000","last_active":"2024-05-28T14:16:49.572026+0000","last_peered":"2024-05-28T14:16:49.572026+0000","last_clean":"2024-05-28T14:16:49.572026+0000","last_became_active":"2024-05-28T14:14:43.769641+0000","last_became_peered":"2024-05-28T14:14:43.769641+0000","last_unstale":"2024-05-28T14:16:49.572026+0000","last_undegraded":"2024-05-28T14:16:49.572026+0000","last_fullsized":"2024-05-28T14:16:49.572026+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:25:03.574855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":106,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"134'116","reported_seq":203,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.551667+0000","last_change":"2024-05-28T14:14:43.769199+0000","last_active":"2024-05-28T14:16:49.551667+0000","last_peered":"2024-05-28T14:16:49.551667+0000","last_clean":"2024-05-28T14:16:49.551667+0000","last_became_active":"2024-05-28T14:14:43.764098+0000","last_became_peered":"2024-05-28T14:14:43.764098+0000","last_unstale":"2024-05-28T14:16:49.551667+0000","last_undegraded":"2024-05-28T14:16:49.551667+0000","last_fullsized":"2024-05-28T14:16:49.551667+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:57:51.444394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":70,"num_read_kb":278530,"num_write":108,"num_write_kb":323587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"134'122","reported_seq":187,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.562442+0000","last_change":"2024-05-28T14:14:43.759234+0000","last_active":"2024-05-28T14:16:49.562442+0000","last_peered":"2024-05-28T14:16:49.562442+0000","last_clean":"2024-05-28T14:16:49.562442+0000","last_became_active":"2024-05-28T14:14:43.758925+0000","last_became_peered":"2024-05-28T14:14:43.758925+0000","last_unstale":"2024-05-28T14:16:49.562442+0000","last_undegraded":"2024-05-28T14:16:49.562442+0000","last_fullsized":"2024-05-28T14:16:49.562442+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:14.042730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":48,"num_read_kb":184323,"num_write":115,"num_write_kb":339969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"134'79","reported_seq":140,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.538628+0000","last_change":"2024-05-28T14:14:43.766612+0000","last_active":"2024-05-28T14:16:49.538628+0000","last_peered":"2024-05-28T14:16:49.538628+0000","last_clean":"2024-05-28T14:16:49.538628+0000","last_became_active":"2024-05-28T14:14:43.764703+0000","last_became_peered":"2024-05-28T14:14:43.764703+0000","last_unstale":"2024-05-28T14:16:49.538628+0000","last_undegraded":"2024-05-28T14:16:49.538628+0000","last_fullsized":"2024-05-28T14:16:49.538628+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:33.925255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":45,"num_read_kb":184320,"num_write":75,"num_write_kb":253952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"134'124","reported_seq":235,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.570848+0000","last_change":"2024-05-28T14:14:43.762958+0000","last_active":"2024-05-28T14:16:49.570848+0000","last_peered":"2024-05-28T14:16:49.570848+0000","last_clean":"2024-05-28T14:16:49.570848+0000","last_became_active":"2024-05-28T14:14:43.762689+0000","last_became_peered":"2024-05-28T14:14:43.762689+0000","last_unstale":"2024-05-28T14:16:49.570848+0000","last_undegraded":"2024-05-28T14:16:49.570848+0000","last_fullsized":"2024-05-28T14:16:49.570848+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:56:11.671806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":95,"num_read_kb":385025,"num_write":115,"num_write_kb":372738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"134'103","reported_seq":169,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.572155+0000","last_change":"2024-05-28T14:14:43.770900+0000","last_active":"2024-05-28T14:16:49.572155+0000","last_peered":"2024-05-28T14:16:49.572155+0000","last_clean":"2024-05-28T14:16:49.572155+0000","last_became_active":"2024-05-28T14:14:43.770616+0000","last_became_peered":"2024-05-28T14:14:43.770616+0000","last_unstale":"2024-05-28T14:16:49.572155+0000","last_undegraded":"2024-05-28T14:16:49.572155+0000","last_fullsized":"2024-05-28T14:16:49.572155+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:07:11.105627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":89,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"134'120","reported_seq":190,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.559090+0000","last_change":"2024-05-28T14:14:43.762262+0000","last_active":"2024-05-28T14:16:49.559090+0000","last_peered":"2024-05-28T14:16:49.559090+0000","last_clean":"2024-05-28T14:16:49.559090+0000","last_became_active":"2024-05-28T14:14:43.761977+0000","last_became_peered":"2024-05-28T14:14:43.761977+0000","last_unstale":"2024-05-28T14:16:49.559090+0000","last_undegraded":"2024-05-28T14:16:49.559090+0000","last_fullsized":"2024-05-28T14:16:49.559090+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:54.517890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":54,"num_read_kb":221184,"num_write":113,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"134'110","reported_seq":181,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.551447+0000","last_change":"2024-05-28T14:14:43.770664+0000","last_active":"2024-05-28T14:16:49.551447+0000","last_peered":"2024-05-28T14:16:49.551447+0000","last_clean":"2024-05-28T14:16:49.551447+0000","last_became_active":"2024-05-28T14:14:43.770335+0000","last_became_peered":"2024-05-28T14:14:43.770335+0000","last_unstale":"2024-05-28T14:16:49.551447+0000","last_undegraded":"2024-05-28T14:16:49.551447+0000","last_fullsized":"2024-05-28T14:16:49.551447+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:20:08.770733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":102,"num_write_kb":311297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"134'99","reported_seq":182,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.562911+0000","last_change":"2024-05-28T14:14:43.769965+0000","last_active":"2024-05-28T14:16:49.562911+0000","last_peered":"2024-05-28T14:16:49.562911+0000","last_clean":"2024-05-28T14:16:49.562911+0000","last_became_active":"2024-05-28T14:14:43.769687+0000","last_became_peered":"2024-05-28T14:14:43.769687+0000","last_unstale":"2024-05-28T14:16:49.562911+0000","last_undegraded":"2024-05-28T14:16:49.562911+0000","last_fullsized":"2024-05-28T14:16:49.562911+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:43:30.318199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":67,"num_read_kb":274432,"num_write":93,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"134'112","reported_seq":236,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.559694+0000","last_change":"2024-05-28T14:14:43.758128+0000","last_active":"2024-05-28T14:16:49.559694+0000","last_peered":"2024-05-28T14:16:49.559694+0000","last_clean":"2024-05-28T14:16:49.559694+0000","last_became_active":"2024-05-28T14:14:43.757685+0000","last_became_peered":"2024-05-28T14:14:43.757685+0000","last_unstale":"2024-05-28T14:16:49.559694+0000","last_undegraded":"2024-05-28T14:16:49.559694+0000","last_fullsized":"2024-05-28T14:16:49.559694+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:53.917272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":108,"num_read_kb":442368,"num_write":103,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"134'101","reported_seq":222,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.544613+0000","last_change":"2024-05-28T14:14:43.769235+0000","last_active":"2024-05-28T14:16:49.544613+0000","last_peered":"2024-05-28T14:16:49.544613+0000","last_clean":"2024-05-28T14:16:49.544613+0000","last_became_active":"2024-05-28T14:14:43.768939+0000","last_became_peered":"2024-05-28T14:14:43.768939+0000","last_unstale":"2024-05-28T14:16:49.544613+0000","last_undegraded":"2024-05-28T14:16:49.544613+0000","last_fullsized":"2024-05-28T14:16:49.544613+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:26.186099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":105,"num_read_kb":413700,"num_write":98,"num_write_kb":315393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"134'110","reported_seq":208,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.570753+0000","last_change":"2024-05-28T14:14:43.767408+0000","last_active":"2024-05-28T14:16:49.570753+0000","last_peered":"2024-05-28T14:16:49.570753+0000","last_clean":"2024-05-28T14:16:49.570753+0000","last_became_active":"2024-05-28T14:14:43.766269+0000","last_became_peered":"2024-05-28T14:14:43.766269+0000","last_unstale":"2024-05-28T14:16:49.570753+0000","last_undegraded":"2024-05-28T14:16:49.570753+0000","last_fullsized":"2024-05-28T14:16:49.570753+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:45.051114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":105,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"134'100","reported_seq":182,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.561793+0000","last_change":"2024-05-28T14:14:43.762819+0000","last_active":"2024-05-28T14:16:49.561793+0000","last_peered":"2024-05-28T14:16:49.561793+0000","last_clean":"2024-05-28T14:16:49.561793+0000","last_became_active":"2024-05-28T14:14:43.762494+0000","last_became_peered":"2024-05-28T14:14:43.762494+0000","last_unstale":"2024-05-28T14:16:49.561793+0000","last_undegraded":"2024-05-28T14:16:49.561793+0000","last_fullsized":"2024-05-28T14:16:49.561793+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:50:53.753974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":66,"num_read_kb":270336,"num_write":92,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"134'83","reported_seq":130,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.565126+0000","last_change":"2024-05-28T14:14:43.761245+0000","last_active":"2024-05-28T14:16:49.565126+0000","last_peered":"2024-05-28T14:16:49.565126+0000","last_clean":"2024-05-28T14:16:49.565126+0000","last_became_active":"2024-05-28T14:14:43.760977+0000","last_became_peered":"2024-05-28T14:14:43.760977+0000","last_unstale":"2024-05-28T14:16:49.565126+0000","last_undegraded":"2024-05-28T14:16:49.565126+0000","last_fullsized":"2024-05-28T14:16:49.565126+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:14.494237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":31,"num_read_kb":126976,"num_write":74,"num_write_kb":249856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"134'131","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.556971+0000","last_change":"2024-05-28T14:14:43.758776+0000","last_active":"2024-05-28T14:16:49.556971+0000","last_peered":"2024-05-28T14:16:49.556971+0000","last_clean":"2024-05-28T14:16:49.556971+0000","last_became_active":"2024-05-28T14:14:43.758483+0000","last_became_peered":"2024-05-28T14:14:43.758483+0000","last_unstale":"2024-05-28T14:16:49.556971+0000","last_undegraded":"2024-05-28T14:16:49.556971+0000","last_fullsized":"2024-05-28T14:16:49.556971+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:44:52.410827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":57,"num_read_kb":225282,"num_write":123,"num_write_kb":368641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"134'101","reported_seq":164,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.555651+0000","last_change":"2024-05-28T14:14:43.756230+0000","last_active":"2024-05-28T14:16:49.555651+0000","last_peered":"2024-05-28T14:16:49.555651+0000","last_clean":"2024-05-28T14:16:49.555651+0000","last_became_active":"2024-05-28T14:14:43.755912+0000","last_became_peered":"2024-05-28T14:14:43.755912+0000","last_unstale":"2024-05-28T14:16:49.555651+0000","last_undegraded":"2024-05-28T14:16:49.555651+0000","last_fullsized":"2024-05-28T14:16:49.555651+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:22.709561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":47,"num_read_kb":180227,"num_write":95,"num_write_kb":286721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"134'118","reported_seq":232,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571290+0000","last_change":"2024-05-28T14:14:43.749113+0000","last_active":"2024-05-28T14:16:49.571290+0000","last_peered":"2024-05-28T14:16:49.571290+0000","last_clean":"2024-05-28T14:16:49.571290+0000","last_became_active":"2024-05-28T14:14:43.748801+0000","last_became_peered":"2024-05-28T14:14:43.748801+0000","last_unstale":"2024-05-28T14:16:49.571290+0000","last_undegraded":"2024-05-28T14:16:49.571290+0000","last_fullsized":"2024-05-28T14:16:49.571290+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:05:56.557590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":98,"num_read_kb":389123,"num_write":107,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"134'123","reported_seq":208,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.542886+0000","last_change":"2024-05-28T14:14:43.769214+0000","last_active":"2024-05-28T14:16:49.542886+0000","last_peered":"2024-05-28T14:16:49.542886+0000","last_clean":"2024-05-28T14:16:49.542886+0000","last_became_active":"2024-05-28T14:14:43.768757+0000","last_became_peered":"2024-05-28T14:14:43.768757+0000","last_unstale":"2024-05-28T14:16:49.542886+0000","last_undegraded":"2024-05-28T14:16:49.542886+0000","last_fullsized":"2024-05-28T14:16:49.542886+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:55:07.023689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":115,"num_write_kb":372736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"134'113","reported_seq":193,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.560761+0000","last_change":"2024-05-28T14:14:43.756647+0000","last_active":"2024-05-28T14:16:49.560761+0000","last_peered":"2024-05-28T14:16:49.560761+0000","last_clean":"2024-05-28T14:16:49.560761+0000","last_became_active":"2024-05-28T14:14:43.756244+0000","last_became_peered":"2024-05-28T14:14:43.756244+0000","last_unstale":"2024-05-28T14:16:49.560761+0000","last_undegraded":"2024-05-28T14:16:49.560761+0000","last_fullsized":"2024-05-28T14:16:49.560761+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:28:18.841947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":101,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"134'118","reported_seq":234,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.548472+0000","last_change":"2024-05-28T14:14:43.757686+0000","last_active":"2024-05-28T14:16:49.548472+0000","last_peered":"2024-05-28T14:16:49.548472+0000","last_clean":"2024-05-28T14:16:49.548472+0000","last_became_active":"2024-05-28T14:14:43.757288+0000","last_became_peered":"2024-05-28T14:14:43.757288+0000","last_unstale":"2024-05-28T14:16:49.548472+0000","last_undegraded":"2024-05-28T14:16:49.548472+0000","last_fullsized":"2024-05-28T14:16:49.548472+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:55:07.023689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":100,"num_read_kb":389125,"num_write":108,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"134'114","reported_seq":191,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.560882+0000","last_change":"2024-05-28T14:14:43.759851+0000","last_active":"2024-05-28T14:16:49.560882+0000","last_peered":"2024-05-28T14:16:49.560882+0000","last_clean":"2024-05-28T14:16:49.560882+0000","last_became_active":"2024-05-28T14:14:43.759580+0000","last_became_peered":"2024-05-28T14:14:43.759580+0000","last_unstale":"2024-05-28T14:16:49.560882+0000","last_undegraded":"2024-05-28T14:16:49.560882+0000","last_fullsized":"2024-05-28T14:16:49.560882+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:50:20.486680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":61,"num_read_kb":241666,"num_write":107,"num_write_kb":356353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"134'109","reported_seq":184,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.555983+0000","last_change":"2024-05-28T14:14:43.768741+0000","last_active":"2024-05-28T14:16:49.555983+0000","last_peered":"2024-05-28T14:16:49.555983+0000","last_clean":"2024-05-28T14:16:49.555983+0000","last_became_active":"2024-05-28T14:14:43.767741+0000","last_became_peered":"2024-05-28T14:14:43.767741+0000","last_unstale":"2024-05-28T14:16:49.555983+0000","last_undegraded":"2024-05-28T14:16:49.555983+0000","last_fullsized":"2024-05-28T14:16:49.555983+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:20.117904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":62,"num_read_kb":253952,"num_write":101,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"134'111","reported_seq":186,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.541445+0000","last_change":"2024-05-28T14:14:43.755177+0000","last_active":"2024-05-28T14:16:49.541445+0000","last_peered":"2024-05-28T14:16:49.541445+0000","last_clean":"2024-05-28T14:16:49.541445+0000","last_became_active":"2024-05-28T14:14:43.753794+0000","last_became_peered":"2024-05-28T14:14:43.753794+0000","last_unstale":"2024-05-28T14:16:49.541445+0000","last_undegraded":"2024-05-28T14:16:49.541445+0000","last_fullsized":"2024-05-28T14:16:49.541445+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:56:28.627044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":59,"num_read_kb":241664,"num_write":103,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"134'122","reported_seq":206,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.553873+0000","last_change":"2024-05-28T14:14:43.763725+0000","last_active":"2024-05-28T14:16:49.553873+0000","last_peered":"2024-05-28T14:16:49.553873+0000","last_clean":"2024-05-28T14:16:49.553873+0000","last_became_active":"2024-05-28T14:14:43.763332+0000","last_became_peered":"2024-05-28T14:14:43.763332+0000","last_unstale":"2024-05-28T14:16:49.553873+0000","last_undegraded":"2024-05-28T14:16:49.553873+0000","last_fullsized":"2024-05-28T14:16:49.553873+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:10.253064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298372,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":68,"num_read_kb":274433,"num_write":112,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"134'121","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.550549+0000","last_change":"2024-05-28T14:14:43.759381+0000","last_active":"2024-05-28T14:16:49.550549+0000","last_peered":"2024-05-28T14:16:49.550549+0000","last_clean":"2024-05-28T14:16:49.550549+0000","last_became_active":"2024-05-28T14:14:43.758722+0000","last_became_peered":"2024-05-28T14:14:43.758722+0000","last_unstale":"2024-05-28T14:16:49.550549+0000","last_undegraded":"2024-05-28T14:16:49.550549+0000","last_fullsized":"2024-05-28T14:16:49.550549+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:48:30.643597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251658240,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":112,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"134'104","reported_seq":195,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571297+0000","last_change":"2024-05-28T14:14:43.768220+0000","last_active":"2024-05-28T14:16:49.571297+0000","last_peered":"2024-05-28T14:16:49.571297+0000","last_clean":"2024-05-28T14:16:49.571297+0000","last_became_active":"2024-05-28T14:14:43.767925+0000","last_became_peered":"2024-05-28T14:14:43.767925+0000","last_unstale":"2024-05-28T14:16:49.571297+0000","last_undegraded":"2024-05-28T14:16:49.571297+0000","last_fullsized":"2024-05-28T14:16:49.571297+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:35:31.221160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":75,"num_read_kb":294915,"num_write":99,"num_write_kb":290818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"134'99","reported_seq":149,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.539531+0000","last_change":"2024-05-28T14:14:43.746536+0000","last_active":"2024-05-28T14:16:49.539531+0000","last_peered":"2024-05-28T14:16:49.539531+0000","last_clean":"2024-05-28T14:16:49.539531+0000","last_became_active":"2024-05-28T14:14:43.746249+0000","last_became_peered":"2024-05-28T14:14:43.746249+0000","last_unstale":"2024-05-28T14:16:49.539531+0000","last_undegraded":"2024-05-28T14:16:49.539531+0000","last_fullsized":"2024-05-28T14:16:49.539531+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:30:48.180755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909764,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":34,"num_read_kb":131074,"num_write":94,"num_write_kb":290818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"134'111","reported_seq":200,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.564301+0000","last_change":"2024-05-28T14:14:43.761643+0000","last_active":"2024-05-28T14:16:49.564301+0000","last_peered":"2024-05-28T14:16:49.564301+0000","last_clean":"2024-05-28T14:16:49.564301+0000","last_became_active":"2024-05-28T14:14:43.761161+0000","last_became_peered":"2024-05-28T14:14:43.761161+0000","last_unstale":"2024-05-28T14:16:49.564301+0000","last_undegraded":"2024-05-28T14:16:49.564301+0000","last_fullsized":"2024-05-28T14:16:49.564301+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:11.879163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075588,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":72,"num_read_kb":294912,"num_write":104,"num_write_kb":327681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"134'117","reported_seq":203,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.565084+0000","last_change":"2024-05-28T14:14:43.764133+0000","last_active":"2024-05-28T14:16:49.565084+0000","last_peered":"2024-05-28T14:16:49.565084+0000","last_clean":"2024-05-28T14:16:49.565084+0000","last_became_active":"2024-05-28T14:14:43.763522+0000","last_became_peered":"2024-05-28T14:14:43.763522+0000","last_unstale":"2024-05-28T14:16:49.565084+0000","last_undegraded":"2024-05-28T14:16:49.565084+0000","last_fullsized":"2024-05-28T14:16:49.565084+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:05.864969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251658240,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":106,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"134'99","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.558327+0000","last_change":"2024-05-28T14:14:43.763447+0000","last_active":"2024-05-28T14:16:27.558327+0000","last_peered":"2024-05-28T14:16:27.558327+0000","last_clean":"2024-05-28T14:16:27.558327+0000","last_became_active":"2024-05-28T14:14:43.763142+0000","last_became_peered":"2024-05-28T14:14:43.763142+0000","last_unstale":"2024-05-28T14:16:27.558327+0000","last_undegraded":"2024-05-28T14:16:27.558327+0000","last_fullsized":"2024-05-28T14:16:27.558327+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:38:22.872839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":88,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"134'106","reported_seq":198,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.563409+0000","last_change":"2024-05-28T14:14:43.747637+0000","last_active":"2024-05-28T14:16:49.563409+0000","last_peered":"2024-05-28T14:16:49.563409+0000","last_clean":"2024-05-28T14:16:49.563409+0000","last_became_active":"2024-05-28T14:14:43.747272+0000","last_became_peered":"2024-05-28T14:14:43.747272+0000","last_unstale":"2024-05-28T14:16:49.563409+0000","last_undegraded":"2024-05-28T14:16:49.563409+0000","last_fullsized":"2024-05-28T14:16:49.563409+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:19.860056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":96,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"134'107","reported_seq":168,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.534735+0000","last_change":"2024-05-28T14:14:43.759397+0000","last_active":"2024-05-28T14:16:49.534735+0000","last_peered":"2024-05-28T14:16:49.534735+0000","last_clean":"2024-05-28T14:16:49.534735+0000","last_became_active":"2024-05-28T14:14:43.759136+0000","last_became_peered":"2024-05-28T14:14:43.759136+0000","last_unstale":"2024-05-28T14:16:49.534735+0000","last_undegraded":"2024-05-28T14:16:49.534735+0000","last_fullsized":"2024-05-28T14:16:49.534735+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:38.538536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":44,"num_read_kb":180224,"num_write":103,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"131'4","reported_seq":23,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973209+0000","last_change":"2024-05-28T14:14:42.751872+0000","last_active":"2024-05-28T14:15:05.973209+0000","last_peered":"2024-05-28T14:15:05.973209+0000","last_clean":"2024-05-28T14:15:05.973209+0000","last_became_active":"2024-05-28T14:14:42.751450+0000","last_became_peered":"2024-05-28T14:14:42.751450+0000","last_unstale":"2024-05-28T14:15:05.973209+0000","last_undegraded":"2024-05-28T14:15:05.973209+0000","last_fullsized":"2024-05-28T14:15:05.973209+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:48:49.635601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"134'121","reported_seq":206,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.559319+0000","last_change":"2024-05-28T14:14:43.766763+0000","last_active":"2024-05-28T14:16:49.559319+0000","last_peered":"2024-05-28T14:16:49.559319+0000","last_clean":"2024-05-28T14:16:49.559319+0000","last_became_active":"2024-05-28T14:14:43.766347+0000","last_became_peered":"2024-05-28T14:14:43.766347+0000","last_unstale":"2024-05-28T14:16:49.559319+0000","last_undegraded":"2024-05-28T14:16:49.559319+0000","last_fullsized":"2024-05-28T14:16:49.559319+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:43:49.334848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":72,"num_read_kb":290817,"num_write":112,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013729+0000","last_change":"2024-05-28T14:14:42.742952+0000","last_active":"2024-05-28T14:15:06.013729+0000","last_peered":"2024-05-28T14:15:06.013729+0000","last_clean":"2024-05-28T14:15:06.013729+0000","last_became_active":"2024-05-28T14:14:42.742668+0000","last_became_peered":"2024-05-28T14:14:42.742668+0000","last_unstale":"2024-05-28T14:15:06.013729+0000","last_undegraded":"2024-05-28T14:15:06.013729+0000","last_fullsized":"2024-05-28T14:15:06.013729+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:53:04.792926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"134'87","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.444881+0000","last_change":"2024-05-28T14:14:43.765567+0000","last_active":"2024-05-28T14:16:27.444881+0000","last_peered":"2024-05-28T14:16:27.444881+0000","last_clean":"2024-05-28T14:16:27.444881+0000","last_became_active":"2024-05-28T14:14:43.765230+0000","last_became_peered":"2024-05-28T14:14:43.765230+0000","last_unstale":"2024-05-28T14:16:27.444881+0000","last_undegraded":"2024-05-28T14:16:27.444881+0000","last_fullsized":"2024-05-28T14:16:27.444881+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:56:02.866180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":75,"num_read_kb":303105,"num_write":87,"num_write_kb":270337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"134'3","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971807+0000","last_change":"2024-05-28T14:14:42.733510+0000","last_active":"2024-05-28T14:15:05.971807+0000","last_peered":"2024-05-28T14:15:05.971807+0000","last_clean":"2024-05-28T14:15:05.971807+0000","last_became_active":"2024-05-28T14:14:42.733231+0000","last_became_peered":"2024-05-28T14:14:42.733231+0000","last_unstale":"2024-05-28T14:15:05.971807+0000","last_undegraded":"2024-05-28T14:15:05.971807+0000","last_fullsized":"2024-05-28T14:15:05.971807+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:37.532653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"134'114","reported_seq":215,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.536179+0000","last_change":"2024-05-28T14:14:43.760573+0000","last_active":"2024-05-28T14:16:49.536179+0000","last_peered":"2024-05-28T14:16:49.536179+0000","last_clean":"2024-05-28T14:16:49.536179+0000","last_became_active":"2024-05-28T14:14:43.759307+0000","last_became_peered":"2024-05-28T14:14:43.759307+0000","last_unstale":"2024-05-28T14:16:49.536179+0000","last_undegraded":"2024-05-28T14:16:49.536179+0000","last_fullsized":"2024-05-28T14:16:49.536179+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:30:49.015188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":107,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"131'2","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077844+0000","last_change":"2024-05-28T14:14:42.745411+0000","last_active":"2024-05-28T14:15:06.077844+0000","last_peered":"2024-05-28T14:15:06.077844+0000","last_clean":"2024-05-28T14:15:06.077844+0000","last_became_active":"2024-05-28T14:14:42.745097+0000","last_became_peered":"2024-05-28T14:14:42.745097+0000","last_unstale":"2024-05-28T14:15:06.077844+0000","last_undegraded":"2024-05-28T14:15:06.077844+0000","last_fullsized":"2024-05-28T14:15:06.077844+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:39:18.579687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"134'112","reported_seq":189,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571629+0000","last_change":"2024-05-28T14:14:43.751125+0000","last_active":"2024-05-28T14:16:49.571629+0000","last_peered":"2024-05-28T14:16:49.571629+0000","last_clean":"2024-05-28T14:16:49.571629+0000","last_became_active":"2024-05-28T14:14:43.747282+0000","last_became_peered":"2024-05-28T14:14:43.747282+0000","last_unstale":"2024-05-28T14:16:49.571629+0000","last_undegraded":"2024-05-28T14:16:49.571629+0000","last_fullsized":"2024-05-28T14:16:49.571629+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:44:03.941731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":105,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077611+0000","last_change":"2024-05-28T14:14:42.741716+0000","last_active":"2024-05-28T14:15:06.077611+0000","last_peered":"2024-05-28T14:15:06.077611+0000","last_clean":"2024-05-28T14:15:06.077611+0000","last_became_active":"2024-05-28T14:14:42.741425+0000","last_became_peered":"2024-05-28T14:14:42.741425+0000","last_unstale":"2024-05-28T14:15:06.077611+0000","last_undegraded":"2024-05-28T14:15:06.077611+0000","last_fullsized":"2024-05-28T14:15:06.077611+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:36:40.085174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"134'114","reported_seq":229,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.566591+0000","last_change":"2024-05-28T14:14:43.760960+0000","last_active":"2024-05-28T14:16:49.566591+0000","last_peered":"2024-05-28T14:16:49.566591+0000","last_clean":"2024-05-28T14:16:49.566591+0000","last_became_active":"2024-05-28T14:14:43.760691+0000","last_became_peered":"2024-05-28T14:14:43.760691+0000","last_unstale":"2024-05-28T14:16:49.566591+0000","last_undegraded":"2024-05-28T14:16:49.566591+0000","last_fullsized":"2024-05-28T14:16:49.566591+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:33:40.268549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":99,"num_read_kb":405504,"num_write":102,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969182+0000","last_change":"2024-05-28T14:14:42.739561+0000","last_active":"2024-05-28T14:15:05.969182+0000","last_peered":"2024-05-28T14:15:05.969182+0000","last_clean":"2024-05-28T14:15:05.969182+0000","last_became_active":"2024-05-28T14:14:42.739191+0000","last_became_peered":"2024-05-28T14:14:42.739191+0000","last_unstale":"2024-05-28T14:15:05.969182+0000","last_undegraded":"2024-05-28T14:15:05.969182+0000","last_fullsized":"2024-05-28T14:15:05.969182+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:49:44.727202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"134'109","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.558982+0000","last_change":"2024-05-28T14:14:43.762652+0000","last_active":"2024-05-28T14:16:49.558982+0000","last_peered":"2024-05-28T14:16:49.558982+0000","last_clean":"2024-05-28T14:16:49.558982+0000","last_became_active":"2024-05-28T14:14:43.761547+0000","last_became_peered":"2024-05-28T14:14:43.761547+0000","last_unstale":"2024-05-28T14:16:49.558982+0000","last_undegraded":"2024-05-28T14:16:49.558982+0000","last_fullsized":"2024-05-28T14:16:49.558982+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:11:03.103530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":102,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973841+0000","last_change":"2024-05-28T14:14:42.745986+0000","last_active":"2024-05-28T14:15:05.973841+0000","last_peered":"2024-05-28T14:15:05.973841+0000","last_clean":"2024-05-28T14:15:05.973841+0000","last_became_active":"2024-05-28T14:14:42.745691+0000","last_became_peered":"2024-05-28T14:14:42.745691+0000","last_unstale":"2024-05-28T14:15:05.973841+0000","last_undegraded":"2024-05-28T14:15:05.973841+0000","last_fullsized":"2024-05-28T14:15:05.973841+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:22:09.220099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"134'107","reported_seq":189,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.555005+0000","last_change":"2024-05-28T14:14:43.764538+0000","last_active":"2024-05-28T14:16:49.555005+0000","last_peered":"2024-05-28T14:16:49.555005+0000","last_clean":"2024-05-28T14:16:49.555005+0000","last_became_active":"2024-05-28T14:14:43.762358+0000","last_became_peered":"2024-05-28T14:14:43.762358+0000","last_unstale":"2024-05-28T14:16:49.555005+0000","last_undegraded":"2024-05-28T14:16:49.555005+0000","last_fullsized":"2024-05-28T14:16:49.555005+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:37:33.489101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":66,"num_read_kb":262146,"num_write":97,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077572+0000","last_change":"2024-05-28T14:14:42.747379+0000","last_active":"2024-05-28T14:15:06.077572+0000","last_peered":"2024-05-28T14:15:06.077572+0000","last_clean":"2024-05-28T14:15:06.077572+0000","last_became_active":"2024-05-28T14:14:42.747127+0000","last_became_peered":"2024-05-28T14:14:42.747127+0000","last_unstale":"2024-05-28T14:15:06.077572+0000","last_undegraded":"2024-05-28T14:15:06.077572+0000","last_fullsized":"2024-05-28T14:15:06.077572+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:10:35.041686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"134'96","reported_seq":149,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.561345+0000","last_change":"2024-05-28T14:14:43.757145+0000","last_active":"2024-05-28T14:16:49.561345+0000","last_peered":"2024-05-28T14:16:49.561345+0000","last_clean":"2024-05-28T14:16:49.561345+0000","last_became_active":"2024-05-28T14:14:43.756803+0000","last_became_peered":"2024-05-28T14:14:43.756803+0000","last_unstale":"2024-05-28T14:16:49.561345+0000","last_undegraded":"2024-05-28T14:16:49.561345+0000","last_fullsized":"2024-05-28T14:16:49.561345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:30:48.180755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188743680,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":85,"num_write_kb":266240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"134'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013914+0000","last_change":"2024-05-28T14:14:42.746521+0000","last_active":"2024-05-28T14:15:06.013914+0000","last_peered":"2024-05-28T14:15:06.013914+0000","last_clean":"2024-05-28T14:15:06.013914+0000","last_became_active":"2024-05-28T14:14:42.746243+0000","last_became_peered":"2024-05-28T14:14:42.746243+0000","last_unstale":"2024-05-28T14:15:06.013914+0000","last_undegraded":"2024-05-28T14:15:06.013914+0000","last_fullsized":"2024-05-28T14:15:06.013914+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:44.045231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"134'96","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.543616+0000","last_change":"2024-05-28T14:14:43.751232+0000","last_active":"2024-05-28T14:16:49.543616+0000","last_peered":"2024-05-28T14:16:49.543616+0000","last_clean":"2024-05-28T14:16:49.543616+0000","last_became_active":"2024-05-28T14:14:43.750768+0000","last_became_peered":"2024-05-28T14:14:43.750768+0000","last_unstale":"2024-05-28T14:16:49.543616+0000","last_undegraded":"2024-05-28T14:16:49.543616+0000","last_fullsized":"2024-05-28T14:16:49.543616+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:37:33.489101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":41,"num_read_kb":167936,"num_write":89,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969503+0000","last_change":"2024-05-28T14:14:42.745263+0000","last_active":"2024-05-28T14:15:05.969503+0000","last_peered":"2024-05-28T14:15:05.969503+0000","last_clean":"2024-05-28T14:15:05.969503+0000","last_became_active":"2024-05-28T14:14:42.744565+0000","last_became_peered":"2024-05-28T14:14:42.744565+0000","last_unstale":"2024-05-28T14:15:05.969503+0000","last_undegraded":"2024-05-28T14:15:05.969503+0000","last_fullsized":"2024-05-28T14:15:05.969503+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:59.384784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"134'85","reported_seq":184,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.543513+0000","last_change":"2024-05-28T14:14:43.760442+0000","last_active":"2024-05-28T14:16:49.543513+0000","last_peered":"2024-05-28T14:16:49.543513+0000","last_clean":"2024-05-28T14:16:49.543513+0000","last_became_active":"2024-05-28T14:14:43.760106+0000","last_became_peered":"2024-05-28T14:14:43.760106+0000","last_unstale":"2024-05-28T14:16:49.543513+0000","last_undegraded":"2024-05-28T14:16:49.543513+0000","last_fullsized":"2024-05-28T14:16:49.543513+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:56:47.127973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":80,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"134'25","reported_seq":43,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.001817+0000","last_change":"2024-05-28T14:14:42.735627+0000","last_active":"2024-05-28T14:16:27.001817+0000","last_peered":"2024-05-28T14:16:27.001817+0000","last_clean":"2024-05-28T14:16:27.001817+0000","last_became_active":"2024-05-28T14:14:42.734095+0000","last_became_peered":"2024-05-28T14:14:42.734095+0000","last_unstale":"2024-05-28T14:16:27.001817+0000","last_undegraded":"2024-05-28T14:16:27.001817+0000","last_fullsized":"2024-05-28T14:16:27.001817+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:07.310371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"134'95","reported_seq":175,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.552078+0000","last_change":"2024-05-28T14:14:43.768144+0000","last_active":"2024-05-28T14:16:49.552078+0000","last_peered":"2024-05-28T14:16:49.552078+0000","last_clean":"2024-05-28T14:16:49.552078+0000","last_became_active":"2024-05-28T14:14:43.766334+0000","last_became_peered":"2024-05-28T14:14:43.766334+0000","last_unstale":"2024-05-28T14:16:49.552078+0000","last_undegraded":"2024-05-28T14:16:49.552078+0000","last_fullsized":"2024-05-28T14:16:49.552078+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:56:51.599701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":86,"num_write_kb":274434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"134'459","reported_seq":477,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.508408+0000","last_change":"2024-05-28T14:14:42.744568+0000","last_active":"2024-05-28T14:16:49.508408+0000","last_peered":"2024-05-28T14:16:49.508408+0000","last_clean":"2024-05-28T14:16:49.508408+0000","last_became_active":"2024-05-28T14:14:42.744240+0000","last_became_peered":"2024-05-28T14:14:42.744240+0000","last_unstale":"2024-05-28T14:16:49.508408+0000","last_undegraded":"2024-05-28T14:16:49.508408+0000","last_fullsized":"2024-05-28T14:16:49.508408+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:25:02.568973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1780932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":457,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"134'125","reported_seq":195,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.556970+0000","last_change":"2024-05-28T14:14:43.758776+0000","last_active":"2024-05-28T14:16:49.556970+0000","last_peered":"2024-05-28T14:16:49.556970+0000","last_clean":"2024-05-28T14:16:49.556970+0000","last_became_active":"2024-05-28T14:14:43.758474+0000","last_became_peered":"2024-05-28T14:14:43.758474+0000","last_unstale":"2024-05-28T14:16:49.556970+0000","last_undegraded":"2024-05-28T14:16:49.556970+0000","last_fullsized":"2024-05-28T14:16:49.556970+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:00:34.527970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":54,"num_read_kb":212994,"num_write":119,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973982+0000","last_change":"2024-05-28T14:14:42.737773+0000","last_active":"2024-05-28T14:15:05.973982+0000","last_peered":"2024-05-28T14:15:05.973982+0000","last_clean":"2024-05-28T14:15:05.973982+0000","last_became_active":"2024-05-28T14:14:42.737442+0000","last_became_peered":"2024-05-28T14:14:42.737442+0000","last_unstale":"2024-05-28T14:15:05.973982+0000","last_undegraded":"2024-05-28T14:15:05.973982+0000","last_fullsized":"2024-05-28T14:15:05.973982+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:45:49.229094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"134'110","reported_seq":187,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.560960+0000","last_change":"2024-05-28T14:14:43.762668+0000","last_active":"2024-05-28T14:16:49.560960+0000","last_peered":"2024-05-28T14:16:49.560960+0000","last_clean":"2024-05-28T14:16:49.560960+0000","last_became_active":"2024-05-28T14:14:43.762049+0000","last_became_peered":"2024-05-28T14:14:43.762049+0000","last_unstale":"2024-05-28T14:16:49.560960+0000","last_undegraded":"2024-05-28T14:16:49.560960+0000","last_fullsized":"2024-05-28T14:16:49.560960+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:57:46.984712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":106,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"134'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077675+0000","last_change":"2024-05-28T14:14:42.746077+0000","last_active":"2024-05-28T14:15:06.077675+0000","last_peered":"2024-05-28T14:15:06.077675+0000","last_clean":"2024-05-28T14:15:06.077675+0000","last_became_active":"2024-05-28T14:14:42.745811+0000","last_became_peered":"2024-05-28T14:14:42.745811+0000","last_unstale":"2024-05-28T14:15:06.077675+0000","last_undegraded":"2024-05-28T14:15:06.077675+0000","last_fullsized":"2024-05-28T14:15:06.077675+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:26:07.187114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"134'123","reported_seq":211,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.556596+0000","last_change":"2024-05-28T14:14:43.768732+0000","last_active":"2024-05-28T14:16:49.556596+0000","last_peered":"2024-05-28T14:16:49.556596+0000","last_clean":"2024-05-28T14:16:49.556596+0000","last_became_active":"2024-05-28T14:14:43.768334+0000","last_became_peered":"2024-05-28T14:14:43.768334+0000","last_unstale":"2024-05-28T14:16:49.556596+0000","last_undegraded":"2024-05-28T14:16:49.556596+0000","last_fullsized":"2024-05-28T14:16:49.556596+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:01:12.201409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852724,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":72,"num_read_kb":282627,"num_write":118,"num_write_kb":360450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"131'8","reported_seq":26,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.975292+0000","last_change":"2024-05-28T14:14:42.742861+0000","last_active":"2024-05-28T14:15:05.975292+0000","last_peered":"2024-05-28T14:15:05.975292+0000","last_clean":"2024-05-28T14:15:05.975292+0000","last_became_active":"2024-05-28T14:14:42.742518+0000","last_became_peered":"2024-05-28T14:14:42.742518+0000","last_unstale":"2024-05-28T14:15:05.975292+0000","last_undegraded":"2024-05-28T14:15:05.975292+0000","last_fullsized":"2024-05-28T14:15:05.975292+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:43:29.312316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"134'90","reported_seq":128,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.570119+0000","last_change":"2024-05-28T14:14:43.762081+0000","last_active":"2024-05-28T14:16:27.570119+0000","last_peered":"2024-05-28T14:16:27.570119+0000","last_clean":"2024-05-28T14:16:27.570119+0000","last_became_active":"2024-05-28T14:14:43.761775+0000","last_became_peered":"2024-05-28T14:14:43.761775+0000","last_unstale":"2024-05-28T14:16:27.570119+0000","last_undegraded":"2024-05-28T14:16:27.570119+0000","last_fullsized":"2024-05-28T14:16:27.570119+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:07:11.105627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176160768,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":25,"num_read_kb":102400,"num_write":86,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013893+0000","last_change":"2024-05-28T14:14:42.746381+0000","last_active":"2024-05-28T14:15:06.013893+0000","last_peered":"2024-05-28T14:15:06.013893+0000","last_clean":"2024-05-28T14:15:06.013893+0000","last_became_active":"2024-05-28T14:14:42.746098+0000","last_became_peered":"2024-05-28T14:14:42.746098+0000","last_unstale":"2024-05-28T14:15:06.013893+0000","last_undegraded":"2024-05-28T14:15:06.013893+0000","last_fullsized":"2024-05-28T14:15:06.013893+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:04.859086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"134'135","reported_seq":233,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.546980+0000","last_change":"2024-05-28T14:14:43.756453+0000","last_active":"2024-05-28T14:16:49.546980+0000","last_peered":"2024-05-28T14:16:49.546980+0000","last_clean":"2024-05-28T14:16:49.546980+0000","last_became_active":"2024-05-28T14:14:43.756138+0000","last_became_peered":"2024-05-28T14:14:43.756138+0000","last_unstale":"2024-05-28T14:16:49.546980+0000","last_undegraded":"2024-05-28T14:16:49.546980+0000","last_fullsized":"2024-05-28T14:16:49.546980+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:07.658669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":129,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"131'2","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971765+0000","last_change":"2024-05-28T14:14:42.742055+0000","last_active":"2024-05-28T14:15:05.971765+0000","last_peered":"2024-05-28T14:15:05.971765+0000","last_clean":"2024-05-28T14:15:05.971765+0000","last_became_active":"2024-05-28T14:14:42.741754+0000","last_became_peered":"2024-05-28T14:14:42.741754+0000","last_unstale":"2024-05-28T14:15:05.971765+0000","last_undegraded":"2024-05-28T14:15:05.971765+0000","last_fullsized":"2024-05-28T14:15:05.971765+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:25.036897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"134'111","reported_seq":190,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.560349+0000","last_change":"2024-05-28T14:14:43.758320+0000","last_active":"2024-05-28T14:16:49.560349+0000","last_peered":"2024-05-28T14:16:49.560349+0000","last_clean":"2024-05-28T14:16:49.560349+0000","last_became_active":"2024-05-28T14:14:43.757996+0000","last_became_peered":"2024-05-28T14:14:43.757996+0000","last_unstale":"2024-05-28T14:16:49.560349+0000","last_undegraded":"2024-05-28T14:16:49.560349+0000","last_fullsized":"2024-05-28T14:16:49.560349+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:18.151246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":63,"num_read_kb":241668,"num_write":101,"num_write_kb":299010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.974449+0000","last_change":"2024-05-28T14:14:42.734922+0000","last_active":"2024-05-28T14:15:05.974449+0000","last_peered":"2024-05-28T14:15:05.974449+0000","last_clean":"2024-05-28T14:15:05.974449+0000","last_became_active":"2024-05-28T14:14:42.734647+0000","last_became_peered":"2024-05-28T14:14:42.734647+0000","last_unstale":"2024-05-28T14:15:05.974449+0000","last_undegraded":"2024-05-28T14:15:05.974449+0000","last_fullsized":"2024-05-28T14:15:05.974449+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:36:34.326176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"134'112","reported_seq":210,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.554691+0000","last_change":"2024-05-28T14:14:43.764186+0000","last_active":"2024-05-28T14:16:49.554691+0000","last_peered":"2024-05-28T14:16:49.554691+0000","last_clean":"2024-05-28T14:16:49.554691+0000","last_became_active":"2024-05-28T14:14:43.762407+0000","last_became_peered":"2024-05-28T14:14:43.762407+0000","last_unstale":"2024-05-28T14:16:49.554691+0000","last_undegraded":"2024-05-28T14:16:49.554691+0000","last_fullsized":"2024-05-28T14:16:49.554691+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:07:34.099826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":82,"num_read_kb":327682,"num_write":107,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013781+0000","last_change":"2024-05-28T14:14:42.733737+0000","last_active":"2024-05-28T14:15:06.013781+0000","last_peered":"2024-05-28T14:15:06.013781+0000","last_clean":"2024-05-28T14:15:06.013781+0000","last_became_active":"2024-05-28T14:14:42.733393+0000","last_became_peered":"2024-05-28T14:14:42.733393+0000","last_unstale":"2024-05-28T14:15:06.013781+0000","last_undegraded":"2024-05-28T14:15:06.013781+0000","last_fullsized":"2024-05-28T14:15:06.013781+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:13.036848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"134'107","reported_seq":208,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571484+0000","last_change":"2024-05-28T14:14:43.767748+0000","last_active":"2024-05-28T14:16:49.571484+0000","last_peered":"2024-05-28T14:16:49.571484+0000","last_clean":"2024-05-28T14:16:49.571484+0000","last_became_active":"2024-05-28T14:14:43.767351+0000","last_became_peered":"2024-05-28T14:14:43.767351+0000","last_unstale":"2024-05-28T14:16:49.571484+0000","last_undegraded":"2024-05-28T14:16:49.571484+0000","last_fullsized":"2024-05-28T14:16:49.571484+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:27:11.131902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":85,"num_read_kb":348160,"num_write":103,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"134'3","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013626+0000","last_change":"2024-05-28T14:14:42.746787+0000","last_active":"2024-05-28T14:15:06.013626+0000","last_peered":"2024-05-28T14:15:06.013626+0000","last_clean":"2024-05-28T14:15:06.013626+0000","last_became_active":"2024-05-28T14:14:42.746273+0000","last_became_peered":"2024-05-28T14:14:42.746273+0000","last_unstale":"2024-05-28T14:15:06.013626+0000","last_undegraded":"2024-05-28T14:15:06.013626+0000","last_fullsized":"2024-05-28T14:15:06.013626+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:38.279660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"134'119","reported_seq":202,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571021+0000","last_change":"2024-05-28T14:14:43.771048+0000","last_active":"2024-05-28T14:16:49.571021+0000","last_peered":"2024-05-28T14:16:49.571021+0000","last_clean":"2024-05-28T14:16:49.571021+0000","last_became_active":"2024-05-28T14:14:43.770751+0000","last_became_peered":"2024-05-28T14:14:43.770751+0000","last_unstale":"2024-05-28T14:16:49.571021+0000","last_undegraded":"2024-05-28T14:16:49.571021+0000","last_fullsized":"2024-05-28T14:16:49.571021+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:34:09.617785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272629760,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":111,"num_write_kb":348163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969248+0000","last_change":"2024-05-28T14:14:42.742397+0000","last_active":"2024-05-28T14:15:05.969248+0000","last_peered":"2024-05-28T14:15:05.969248+0000","last_clean":"2024-05-28T14:15:05.969248+0000","last_became_active":"2024-05-28T14:14:42.742104+0000","last_became_peered":"2024-05-28T14:14:42.742104+0000","last_unstale":"2024-05-28T14:15:05.969248+0000","last_undegraded":"2024-05-28T14:15:05.969248+0000","last_fullsized":"2024-05-28T14:15:05.969248+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:21.703678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"134'122","reported_seq":187,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.562957+0000","last_change":"2024-05-28T14:14:43.756230+0000","last_active":"2024-05-28T14:16:49.562957+0000","last_peered":"2024-05-28T14:16:49.562957+0000","last_clean":"2024-05-28T14:16:49.562957+0000","last_became_active":"2024-05-28T14:14:43.755945+0000","last_became_peered":"2024-05-28T14:14:43.755945+0000","last_unstale":"2024-05-28T14:16:49.562957+0000","last_undegraded":"2024-05-28T14:16:49.562957+0000","last_fullsized":"2024-05-28T14:16:49.562957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:30:48.180755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":49,"num_read_kb":196609,"num_write":111,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969005+0000","last_change":"2024-05-28T14:14:42.737496+0000","last_active":"2024-05-28T14:15:05.969005+0000","last_peered":"2024-05-28T14:15:05.969005+0000","last_clean":"2024-05-28T14:15:05.969005+0000","last_became_active":"2024-05-28T14:14:42.737194+0000","last_became_peered":"2024-05-28T14:14:42.737194+0000","last_unstale":"2024-05-28T14:15:05.969005+0000","last_undegraded":"2024-05-28T14:15:05.969005+0000","last_fullsized":"2024-05-28T14:15:05.969005+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:56:10.665924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"134'110","reported_seq":190,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.515934+0000","last_change":"2024-05-28T14:14:43.758200+0000","last_active":"2024-05-28T14:16:27.515934+0000","last_peered":"2024-05-28T14:16:27.515934+0000","last_clean":"2024-05-28T14:16:27.515934+0000","last_became_active":"2024-05-28T14:14:43.757875+0000","last_became_peered":"2024-05-28T14:14:43.757875+0000","last_unstale":"2024-05-28T14:16:27.515934+0000","last_undegraded":"2024-05-28T14:16:27.515934+0000","last_fullsized":"2024-05-28T14:16:27.515934+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:53.917272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":64,"num_read_kb":253954,"num_write":110,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"134'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971403+0000","last_change":"2024-05-28T14:14:42.738809+0000","last_active":"2024-05-28T14:15:05.971403+0000","last_peered":"2024-05-28T14:15:05.971403+0000","last_clean":"2024-05-28T14:15:05.971403+0000","last_became_active":"2024-05-28T14:14:42.738494+0000","last_became_peered":"2024-05-28T14:14:42.738494+0000","last_unstale":"2024-05-28T14:15:05.971403+0000","last_undegraded":"2024-05-28T14:15:05.971403+0000","last_fullsized":"2024-05-28T14:15:05.971403+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:17.145363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"134'127","reported_seq":183,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.568965+0000","last_change":"2024-05-28T14:14:43.770616+0000","last_active":"2024-05-28T14:16:49.568965+0000","last_peered":"2024-05-28T14:16:49.568965+0000","last_clean":"2024-05-28T14:16:49.568965+0000","last_became_active":"2024-05-28T14:14:43.770253+0000","last_became_peered":"2024-05-28T14:14:43.770253+0000","last_unstale":"2024-05-28T14:16:49.568965+0000","last_undegraded":"2024-05-28T14:16:49.568965+0000","last_fullsized":"2024-05-28T14:16:49.568965+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:49:45.733085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":40,"num_read_kb":163840,"num_write":118,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"131'2","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.974495+0000","last_change":"2024-05-28T14:14:42.746355+0000","last_active":"2024-05-28T14:15:05.974495+0000","last_peered":"2024-05-28T14:15:05.974495+0000","last_clean":"2024-05-28T14:15:05.974495+0000","last_became_active":"2024-05-28T14:14:42.745926+0000","last_became_peered":"2024-05-28T14:14:42.745926+0000","last_unstale":"2024-05-28T14:15:05.974495+0000","last_undegraded":"2024-05-28T14:15:05.974495+0000","last_fullsized":"2024-05-28T14:15:05.974495+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:54:47.273264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"134'115","reported_seq":222,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.553138+0000","last_change":"2024-05-28T14:15:06.178022+0000","last_active":"2024-05-28T14:16:49.553138+0000","last_peered":"2024-05-28T14:16:49.553138+0000","last_clean":"2024-05-28T14:16:49.553138+0000","last_became_active":"2024-05-28T14:15:05.975545+0000","last_became_peered":"2024-05-28T14:15:05.975545+0000","last_unstale":"2024-05-28T14:16:49.553138+0000","last_undegraded":"2024-05-28T14:16:49.553138+0000","last_fullsized":"2024-05-28T14:16:49.553138+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:56:51.599701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":85,"num_read_kb":348160,"num_write":107,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"134'41","reported_seq":59,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:25.782834+0000","last_change":"2024-05-28T14:14:42.749733+0000","last_active":"2024-05-28T14:16:25.782834+0000","last_peered":"2024-05-28T14:16:25.782834+0000","last_clean":"2024-05-28T14:16:25.782834+0000","last_became_active":"2024-05-28T14:14:42.749348+0000","last_became_peered":"2024-05-28T14:14:42.749348+0000","last_unstale":"2024-05-28T14:16:25.782834+0000","last_undegraded":"2024-05-28T14:16:25.782834+0000","last_fullsized":"2024-05-28T14:16:25.782834+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:17.145363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"134'93","reported_seq":210,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.554090+0000","last_change":"2024-05-28T14:14:43.764008+0000","last_active":"2024-05-28T14:16:49.554090+0000","last_peered":"2024-05-28T14:16:49.554090+0000","last_clean":"2024-05-28T14:16:49.554090+0000","last_became_active":"2024-05-28T14:14:43.762648+0000","last_became_peered":"2024-05-28T14:14:43.762648+0000","last_unstale":"2024-05-28T14:16:49.554090+0000","last_undegraded":"2024-05-28T14:16:49.554090+0000","last_fullsized":"2024-05-28T14:16:49.554090+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:35:31.221160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":86,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077916+0000","last_change":"2024-05-28T14:14:42.744189+0000","last_active":"2024-05-28T14:15:06.077916+0000","last_peered":"2024-05-28T14:15:06.077916+0000","last_clean":"2024-05-28T14:15:06.077916+0000","last_became_active":"2024-05-28T14:14:42.743858+0000","last_became_peered":"2024-05-28T14:14:42.743858+0000","last_unstale":"2024-05-28T14:15:06.077916+0000","last_undegraded":"2024-05-28T14:15:06.077916+0000","last_fullsized":"2024-05-28T14:15:06.077916+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:03:34.587886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"134'103","reported_seq":184,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571199+0000","last_change":"2024-05-28T14:15:06.036082+0000","last_active":"2024-05-28T14:16:49.571199+0000","last_peered":"2024-05-28T14:16:49.571199+0000","last_clean":"2024-05-28T14:16:49.571199+0000","last_became_active":"2024-05-28T14:15:05.976669+0000","last_became_peered":"2024-05-28T14:15:05.976669+0000","last_unstale":"2024-05-28T14:16:49.571199+0000","last_undegraded":"2024-05-28T14:16:49.571199+0000","last_fullsized":"2024-05-28T14:16:49.571199+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:57:19.311096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132541,"num_objects":50,"num_object_clones":1,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":1,"num_read":52,"num_read_kb":208897,"num_write":94,"num_write_kb":282626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8388730,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971624+0000","last_change":"2024-05-28T14:14:42.748521+0000","last_active":"2024-05-28T14:15:05.971624+0000","last_peered":"2024-05-28T14:15:05.971624+0000","last_clean":"2024-05-28T14:15:05.971624+0000","last_became_active":"2024-05-28T14:14:42.748193+0000","last_became_peered":"2024-05-28T14:14:42.748193+0000","last_unstale":"2024-05-28T14:15:05.971624+0000","last_undegraded":"2024-05-28T14:15:05.971624+0000","last_fullsized":"2024-05-28T14:15:05.971624+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:48:29.637714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"134'128","reported_seq":194,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.556050+0000","last_change":"2024-05-28T14:14:43.767848+0000","last_active":"2024-05-28T14:16:27.556050+0000","last_peered":"2024-05-28T14:16:27.556050+0000","last_clean":"2024-05-28T14:16:27.556050+0000","last_became_active":"2024-05-28T14:14:43.767552+0000","last_became_peered":"2024-05-28T14:14:43.767552+0000","last_unstale":"2024-05-28T14:16:27.556050+0000","last_undegraded":"2024-05-28T14:16:27.556050+0000","last_fullsized":"2024-05-28T14:16:27.556050+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:09:18.039029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":285212672,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":116,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077762+0000","last_change":"2024-05-28T14:14:42.746605+0000","last_active":"2024-05-28T14:15:06.077762+0000","last_peered":"2024-05-28T14:15:06.077762+0000","last_clean":"2024-05-28T14:15:06.077762+0000","last_became_active":"2024-05-28T14:14:42.746338+0000","last_became_peered":"2024-05-28T14:14:42.746338+0000","last_unstale":"2024-05-28T14:15:06.077762+0000","last_undegraded":"2024-05-28T14:15:06.077762+0000","last_fullsized":"2024-05-28T14:15:06.077762+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:51:27.080888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"134'137","reported_seq":238,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.537160+0000","last_change":"2024-05-28T14:14:43.760455+0000","last_active":"2024-05-28T14:16:49.537160+0000","last_peered":"2024-05-28T14:16:49.537160+0000","last_clean":"2024-05-28T14:16:49.537160+0000","last_became_active":"2024-05-28T14:14:43.760063+0000","last_became_peered":"2024-05-28T14:14:43.760063+0000","last_unstale":"2024-05-28T14:16:49.537160+0000","last_undegraded":"2024-05-28T14:16:49.537160+0000","last_fullsized":"2024-05-28T14:16:49.537160+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:50:53.753974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272629760,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":90,"num_read_kb":356355,"num_write":129,"num_write_kb":393217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973173+0000","last_change":"2024-05-28T14:14:42.749618+0000","last_active":"2024-05-28T14:15:05.973173+0000","last_peered":"2024-05-28T14:15:05.973173+0000","last_clean":"2024-05-28T14:15:05.973173+0000","last_became_active":"2024-05-28T14:14:42.749259+0000","last_became_peered":"2024-05-28T14:14:42.749259+0000","last_unstale":"2024-05-28T14:15:05.973173+0000","last_undegraded":"2024-05-28T14:15:05.973173+0000","last_fullsized":"2024-05-28T14:15:05.973173+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:44:02.935848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"134'119","reported_seq":215,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.564680+0000","last_change":"2024-05-28T14:14:43.761804+0000","last_active":"2024-05-28T14:16:49.564680+0000","last_peered":"2024-05-28T14:16:49.564680+0000","last_clean":"2024-05-28T14:16:49.564680+0000","last_became_active":"2024-05-28T14:14:43.761382+0000","last_became_peered":"2024-05-28T14:14:43.761382+0000","last_unstale":"2024-05-28T14:16:49.564680+0000","last_undegraded":"2024-05-28T14:16:49.564680+0000","last_fullsized":"2024-05-28T14:16:49.564680+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:48:50.641484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":114,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"134'31","reported_seq":80,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:26.863095+0000","last_change":"2024-05-28T14:14:42.739854+0000","last_active":"2024-05-28T14:16:26.863095+0000","last_peered":"2024-05-28T14:16:26.863095+0000","last_clean":"2024-05-28T14:16:26.863095+0000","last_became_active":"2024-05-28T14:14:42.738369+0000","last_became_peered":"2024-05-28T14:14:42.738369+0000","last_unstale":"2024-05-28T14:16:26.863095+0000","last_undegraded":"2024-05-28T14:16:26.863095+0000","last_fullsized":"2024-05-28T14:16:26.863095+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:34:08.611902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":31,"num_write":30,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"134'123","reported_seq":223,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.554421+0000","last_change":"2024-05-28T14:14:43.765205+0000","last_active":"2024-05-28T14:16:49.554421+0000","last_peered":"2024-05-28T14:16:49.554421+0000","last_clean":"2024-05-28T14:16:49.554421+0000","last_became_active":"2024-05-28T14:14:43.764879+0000","last_became_peered":"2024-05-28T14:14:43.764879+0000","last_unstale":"2024-05-28T14:16:49.554421+0000","last_undegraded":"2024-05-28T14:16:49.554421+0000","last_fullsized":"2024-05-28T14:16:49.554421+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:07.658669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":116,"num_write_kb":372736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971518+0000","last_change":"2024-05-28T14:14:42.730611+0000","last_active":"2024-05-28T14:15:05.971518+0000","last_peered":"2024-05-28T14:15:05.971518+0000","last_clean":"2024-05-28T14:15:05.971518+0000","last_became_active":"2024-05-28T14:14:42.730333+0000","last_became_peered":"2024-05-28T14:14:42.730333+0000","last_unstale":"2024-05-28T14:15:05.971518+0000","last_undegraded":"2024-05-28T14:15:05.971518+0000","last_fullsized":"2024-05-28T14:15:05.971518+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:05:55.551707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"134'106","reported_seq":169,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.543332+0000","last_change":"2024-05-28T14:14:43.761743+0000","last_active":"2024-05-28T14:16:49.543332+0000","last_peered":"2024-05-28T14:16:49.543332+0000","last_clean":"2024-05-28T14:16:49.543332+0000","last_became_active":"2024-05-28T14:14:43.760828+0000","last_became_peered":"2024-05-28T14:14:43.760828+0000","last_unstale":"2024-05-28T14:16:49.543332+0000","last_undegraded":"2024-05-28T14:16:49.543332+0000","last_fullsized":"2024-05-28T14:16:49.543332+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:44:21.055874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":101,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"131'2","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973324+0000","last_change":"2024-05-28T14:14:42.751868+0000","last_active":"2024-05-28T14:15:05.973324+0000","last_peered":"2024-05-28T14:15:05.973324+0000","last_clean":"2024-05-28T14:15:05.973324+0000","last_became_active":"2024-05-28T14:14:42.751447+0000","last_became_peered":"2024-05-28T14:14:42.751447+0000","last_unstale":"2024-05-28T14:15:05.973324+0000","last_undegraded":"2024-05-28T14:15:05.973324+0000","last_fullsized":"2024-05-28T14:15:05.973324+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:28:17.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":383,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.972861+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:15:05.972861+0000","last_peered":"2024-05-28T14:15:05.972861+0000","last_clean":"2024-05-28T14:15:05.972861+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:15:05.972861+0000","last_undegraded":"2024-05-28T14:15:05.972861+0000","last_fullsized":"2024-05-28T14:15:05.972861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":15082718397,"num_objects":3603,"num_object_clones":1,"num_object_copies":7206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":1,"num_read":4365,"num_read_kb":17629245,"num_write":6581,"num_write_kb":20676651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":50331770,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":30165483520,"data_stored":30165436794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7068,"ondisk_log_size":7068,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1787293,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":30,"num_read_kb":31,"num_write":638,"num_write_kb":1990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3637248,"data_stored":3574586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":590,"ondisk_log_size":590,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575074,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3272720,"kb_used_data":3133320,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":91099120,"statfs":{"total":96636764160,"available":93285498880,"internally_reserved":0,"allocated":3208519680,"data_stored":3207557922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":142735818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575074,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4222476,"kb_used_data":4078916,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90149364,"statfs":{"total":96636764160,"available":92312948736,"internally_reserved":0,"allocated":4176809984,"data_stored":4175856883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575076,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3121336,"kb_used_data":2995824,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91250504,"statfs":{"total":96636764160,"available":93440516096,"internally_reserved":0,"allocated":3067723776,"data_stored":3066774417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":128514502},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575075,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4572916,"kb_used_data":4433516,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":89798924,"statfs":{"total":96636764160,"available":91954098176,"internally_reserved":0,"allocated":4539920384,"data_stored":4538974892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575074,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3790716,"kb_used_data":3642996,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":90581124,"statfs":{"total":96636764160,"available":92755070976,"internally_reserved":0,"allocated":3730427904,"data_stored":3729479880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":151255496},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575075,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4906484,"kb_used_data":4740716,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":89465356,"statfs":{"total":96636764160,"available":91612524544,"internally_reserved":0,"allocated":4854493184,"data_stored":4853548245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575074,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3483436,"kb_used_data":3339876,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90888404,"statfs":{"total":96636764160,"available":93069725696,"internally_reserved":0,"allocated":3420033024,"data_stored":3419095661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575075,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3250484,"kb_used_data":3111084,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":91121356,"statfs":{"total":96636764160,"available":93308268544,"internally_reserved":0,"allocated":3185750016,"data_stored":3184804998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,9,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1798144,"data_stored":1786495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1802240,"data_stored":1781570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3183484928,"data_stored":3183476989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3418361856,"data_stored":3418358020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4852813824,"data_stored":4852809988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3728740352,"data_stored":3728736516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4538245120,"data_stored":4538237181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3066044416,"data_stored":3066036664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4173340672,"data_stored":4173332920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3204452352,"data_stored":3204448516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:16:51.579 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-28T14:16:51.579 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:16:51.582 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:16:51.624 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:16:51.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.777+0000 7f4071e00640 1 Processor -- start 2024-05-28T14:16:51.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.777+0000 7f4071e00640 1 -- start start 2024-05-28T14:16:51.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.777+0000 7f4071e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406c06f660 0x7f406c073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:51.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.777+0000 7f4071e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f406c0d0e00 0x7f406c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:51.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.777+0000 7f4071e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406c15f280 con 0x7f406c15ba40 2024-05-28T14:16:51.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.777+0000 7f4071e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f406c15f030 con 0x7f406c0d0e00 2024-05-28T14:16:51.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.777+0000 7f4071e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f406c15f560 con 0x7f406c06f660 2024-05-28T14:16:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.781+0000 7f406aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406c06f660 0x7f406c073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.781+0000 7f4063e00640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f406c15ba40 0x7f406c0d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.42:43896/0 (socket says 172.21.15.42:43896) 2024-05-28T14:16:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.781+0000 7f4063e00640 1 -- 172.21.15.42:0/3265786449 learned_addr learned my addr 172.21.15.42:0/3265786449 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.781+0000 7f406b400640 1 --2- 172.21.15.42:0/3265786449 >> v2:172.21.15.63:3300/0 conn(0x7f406c0d0e00 0x7f406c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.781+0000 7f406b400640 1 -- 172.21.15.42:0/3265786449 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406c06f660 msgr2=0x7f406c073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.781+0000 7f406b400640 1 --2- 172.21.15.42:0/3265786449 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406c06f660 0x7f406c073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.781+0000 7f406b400640 1 -- 172.21.15.42:0/3265786449 >> v1:172.21.15.63:6789/0 conn(0x7f406c15ba40 legacy=0x7f406c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.781+0000 7f406b400640 1 -- 172.21.15.42:0/3265786449 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c15f840 con 0x7f406c0d0e00 2024-05-28T14:16:51.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.781+0000 7f406b400640 1 --2- 172.21.15.42:0/3265786449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f406c0d0e00 0x7f406c06dd60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4064009d90 tx=0x7f406401ea30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bac14cece64b158 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:51.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.781+0000 7f406a000640 1 -- 172.21.15.42:0/3265786449 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40640b4070 con 0x7f406c0d0e00 2024-05-28T14:16:51.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f406a000640 1 -- 172.21.15.42:0/3265786449 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40640a7ae0 con 0x7f406c0d0e00 2024-05-28T14:16:51.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f406a000640 1 -- 172.21.15.42:0/3265786449 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40640af440 con 0x7f406c0d0e00 2024-05-28T14:16:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 -- 172.21.15.42:0/3265786449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f406c0d0e00 msgr2=0x7f406c06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 --2- 172.21.15.42:0/3265786449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f406c0d0e00 0x7f406c06dd60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4064009d90 tx=0x7f406401ea30 comp rx=0 tx=0).stop 2024-05-28T14:16:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 -- 172.21.15.42:0/3265786449 shutdown_connections 2024-05-28T14:16:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 --2- 172.21.15.42:0/3265786449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f406c0d0e00 0x7f406c06dd60 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 --2- 172.21.15.42:0/3265786449 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406c06f660 0x7f406c073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 -- 172.21.15.42:0/3265786449 >> 172.21.15.42:0/3265786449 conn(0x7f406c06bdf0 msgr2=0x7f406c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 -- 172.21.15.42:0/3265786449 shutdown_connections 2024-05-28T14:16:51.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 -- 172.21.15.42:0/3265786449 wait complete. 2024-05-28T14:16:51.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 Processor -- start 2024-05-28T14:16:51.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 -- start start 2024-05-28T14:16:51.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406c06f660 0x7f406c13b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:51.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f406c0d0e00 0x7f406c099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:51.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f406c138e30 0x7f406c151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:51.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f406c0b3460 con 0x7f406c0d0e00 2024-05-28T14:16:51.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f406c0b32e0 con 0x7f406c138e30 2024-05-28T14:16:51.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4071e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f406c0b35e0 con 0x7f406c06f660 2024-05-28T14:16:51.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4063e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f406c0d0e00 0x7f406c099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:51.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f406aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406c06f660 0x7f406c13b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:51.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4063e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f406c0d0e00 0x7f406c099920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37942/0 (socket says 172.21.15.42:37942) 2024-05-28T14:16:51.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.785+0000 7f4063e00640 1 -- 172.21.15.42:0/2039152940 learned_addr learned my addr 172.21.15.42:0/2039152940 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:51.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f406b400640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f406c138e30 0x7f406c151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:51.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f4063e00640 1 -- 172.21.15.42:0/2039152940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406c06f660 msgr2=0x7f406c13b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f4063e00640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406c06f660 0x7f406c13b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f4063e00640 1 -- 172.21.15.42:0/2039152940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f406c138e30 msgr2=0x7f406c151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f4063e00640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f406c138e30 0x7f406c151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f4063e00640 1 -- 172.21.15.42:0/2039152940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c152390 con 0x7f406c0d0e00 2024-05-28T14:16:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f406aa00640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406c06f660 0x7f406c13b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:16:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f406b400640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f406c138e30 0x7f406c151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:16:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f4063e00640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f406c0d0e00 0x7f406c099920 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f405400c370 tx=0x7f405400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f4063400640 1 -- 172.21.15.42:0/2039152940 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054017070 con 0x7f406c0d0e00 2024-05-28T14:16:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f4063400640 1 -- 172.21.15.42:0/2039152940 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4054005cf0 con 0x7f406c0d0e00 2024-05-28T14:16:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f4063400640 1 -- 172.21.15.42:0/2039152940 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540072f0 con 0x7f406c0d0e00 2024-05-28T14:16:51.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f4071e00640 1 -- 172.21.15.42:0/2039152940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406c14d3e0 con 0x7f406c0d0e00 2024-05-28T14:16:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f4071e00640 1 -- 172.21.15.42:0/2039152940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406c14f3d0 con 0x7f406c0d0e00 2024-05-28T14:16:51.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f4071e00640 1 -- 172.21.15.42:0/2039152940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406c068a90 con 0x7f406c0d0e00 2024-05-28T14:16:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f4063400640 1 -- 172.21.15.42:0/2039152940 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f405401a070 con 0x7f406c0d0e00 2024-05-28T14:16:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.789+0000 7f4063400640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f404806a450 0x7f404806c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.793+0000 7f4063400640 1 -- 172.21.15.42:0/2039152940 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f4054084d90 con 0x7f406c0d0e00 2024-05-28T14:16:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.793+0000 7f406aa00640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f404806a450 0x7f404806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:51.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.793+0000 7f406aa00640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f404806a450 0x7f404806c910 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f405c009f50 tx=0x7f405c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:51.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.793+0000 7f4063400640 1 -- 172.21.15.42:0/2039152940 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4054056760 con 0x7f406c0d0e00 2024-05-28T14:16:51.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.957+0000 7f4071e00640 1 -- 172.21.15.42:0/2039152940 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f406c1661a0 con 0x7f404806a450 2024-05-28T14:16:51.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.961+0000 7f4063400640 1 -- 172.21.15.42:0/2039152940 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+255065 (secure 0 0 0) 0x7f406c1661a0 con 0x7f404806a450 2024-05-28T14:16:51.950 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:51.951 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:16:51.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.965+0000 7f4071e00640 1 -- 172.21.15.42:0/2039152940 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f404806a450 msgr2=0x7f404806c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.965+0000 7f4071e00640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f404806a450 0x7f404806c910 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f405c009f50 tx=0x7f405c009190 comp rx=0 tx=0).stop 2024-05-28T14:16:51.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.965+0000 7f4071e00640 1 -- 172.21.15.42:0/2039152940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f406c0d0e00 msgr2=0x7f406c099920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:51.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.965+0000 7f4071e00640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f406c0d0e00 0x7f406c099920 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f405400c370 tx=0x7f405400c830 comp rx=0 tx=0).stop 2024-05-28T14:16:51.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.965+0000 7f4071e00640 1 -- 172.21.15.42:0/2039152940 shutdown_connections 2024-05-28T14:16:51.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.965+0000 7f4071e00640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f404806a450 0x7f404806c910 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.965+0000 7f4071e00640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f406c138e30 0x7f406c151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.965+0000 7f4071e00640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f406c0d0e00 0x7f406c099920 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.965+0000 7f4071e00640 1 --2- 172.21.15.42:0/2039152940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f406c06f660 0x7f406c13b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:51.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.965+0000 7f4071e00640 1 -- 172.21.15.42:0/2039152940 >> 172.21.15.42:0/2039152940 conn(0x7f406c06bdf0 msgr2=0x7f406c14c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:51.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.965+0000 7f4071e00640 1 -- 172.21.15.42:0/2039152940 shutdown_connections 2024-05-28T14:16:51.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:51.965+0000 7f4071e00640 1 -- 172.21.15.42:0/2039152940 wait complete. 2024-05-28T14:16:51.971 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":688,"stamp":"2024-05-28T14:16:51.249025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15085096058,"num_objects":3628,"num_object_clones":1,"num_object_copies":7256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3628,"num_whiteouts":1,"num_read":4511,"num_read_kb":17629376,"num_write":7378,"num_write_kb":20681057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":50331770,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7775,"ondisk_log_size":7775,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":30620568,"kb_used_data":29476248,"kb_used_omap":12,"kb_used_meta":1144243,"kb_avail":724354152,"statfs":{"total":773094113280,"available":741738651648,"internally_reserved":0,"allocated":30183677952,"data_stored":30176092898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1171705394},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,9,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":173,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002732"},"pg_stats":[{"pgid":"19.3f","version":"134'94","reported_seq":188,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.570602+0000","last_change":"2024-05-28T14:14:43.766434+0000","last_active":"2024-05-28T14:16:49.570602+0000","last_peered":"2024-05-28T14:16:49.570602+0000","last_clean":"2024-05-28T14:16:49.570602+0000","last_became_active":"2024-05-28T14:14:43.766125+0000","last_became_peered":"2024-05-28T14:14:43.766125+0000","last_unstale":"2024-05-28T14:16:49.570602+0000","last_undegraded":"2024-05-28T14:16:49.570602+0000","last_fullsized":"2024-05-28T14:16:49.570602+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:50:53.753974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":77,"num_read_kb":315392,"num_write":85,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"134'120","reported_seq":212,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.572026+0000","last_change":"2024-05-28T14:14:43.769925+0000","last_active":"2024-05-28T14:16:49.572026+0000","last_peered":"2024-05-28T14:16:49.572026+0000","last_clean":"2024-05-28T14:16:49.572026+0000","last_became_active":"2024-05-28T14:14:43.769641+0000","last_became_peered":"2024-05-28T14:14:43.769641+0000","last_unstale":"2024-05-28T14:16:49.572026+0000","last_undegraded":"2024-05-28T14:16:49.572026+0000","last_fullsized":"2024-05-28T14:16:49.572026+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:25:03.574855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":106,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"134'116","reported_seq":203,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.551667+0000","last_change":"2024-05-28T14:14:43.769199+0000","last_active":"2024-05-28T14:16:49.551667+0000","last_peered":"2024-05-28T14:16:49.551667+0000","last_clean":"2024-05-28T14:16:49.551667+0000","last_became_active":"2024-05-28T14:14:43.764098+0000","last_became_peered":"2024-05-28T14:14:43.764098+0000","last_unstale":"2024-05-28T14:16:49.551667+0000","last_undegraded":"2024-05-28T14:16:49.551667+0000","last_fullsized":"2024-05-28T14:16:49.551667+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:57:51.444394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":70,"num_read_kb":278530,"num_write":108,"num_write_kb":323587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"134'122","reported_seq":187,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.562442+0000","last_change":"2024-05-28T14:14:43.759234+0000","last_active":"2024-05-28T14:16:49.562442+0000","last_peered":"2024-05-28T14:16:49.562442+0000","last_clean":"2024-05-28T14:16:49.562442+0000","last_became_active":"2024-05-28T14:14:43.758925+0000","last_became_peered":"2024-05-28T14:14:43.758925+0000","last_unstale":"2024-05-28T14:16:49.562442+0000","last_undegraded":"2024-05-28T14:16:49.562442+0000","last_fullsized":"2024-05-28T14:16:49.562442+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:14.042730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":48,"num_read_kb":184323,"num_write":115,"num_write_kb":339969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"134'79","reported_seq":140,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.538628+0000","last_change":"2024-05-28T14:14:43.766612+0000","last_active":"2024-05-28T14:16:49.538628+0000","last_peered":"2024-05-28T14:16:49.538628+0000","last_clean":"2024-05-28T14:16:49.538628+0000","last_became_active":"2024-05-28T14:14:43.764703+0000","last_became_peered":"2024-05-28T14:14:43.764703+0000","last_unstale":"2024-05-28T14:16:49.538628+0000","last_undegraded":"2024-05-28T14:16:49.538628+0000","last_fullsized":"2024-05-28T14:16:49.538628+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:33.925255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":45,"num_read_kb":184320,"num_write":75,"num_write_kb":253952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"134'124","reported_seq":235,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.570848+0000","last_change":"2024-05-28T14:14:43.762958+0000","last_active":"2024-05-28T14:16:49.570848+0000","last_peered":"2024-05-28T14:16:49.570848+0000","last_clean":"2024-05-28T14:16:49.570848+0000","last_became_active":"2024-05-28T14:14:43.762689+0000","last_became_peered":"2024-05-28T14:14:43.762689+0000","last_unstale":"2024-05-28T14:16:49.570848+0000","last_undegraded":"2024-05-28T14:16:49.570848+0000","last_fullsized":"2024-05-28T14:16:49.570848+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:56:11.671806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":95,"num_read_kb":385025,"num_write":115,"num_write_kb":372738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"134'103","reported_seq":169,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.572155+0000","last_change":"2024-05-28T14:14:43.770900+0000","last_active":"2024-05-28T14:16:49.572155+0000","last_peered":"2024-05-28T14:16:49.572155+0000","last_clean":"2024-05-28T14:16:49.572155+0000","last_became_active":"2024-05-28T14:14:43.770616+0000","last_became_peered":"2024-05-28T14:14:43.770616+0000","last_unstale":"2024-05-28T14:16:49.572155+0000","last_undegraded":"2024-05-28T14:16:49.572155+0000","last_fullsized":"2024-05-28T14:16:49.572155+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:07:11.105627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":89,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"134'120","reported_seq":190,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.559090+0000","last_change":"2024-05-28T14:14:43.762262+0000","last_active":"2024-05-28T14:16:49.559090+0000","last_peered":"2024-05-28T14:16:49.559090+0000","last_clean":"2024-05-28T14:16:49.559090+0000","last_became_active":"2024-05-28T14:14:43.761977+0000","last_became_peered":"2024-05-28T14:14:43.761977+0000","last_unstale":"2024-05-28T14:16:49.559090+0000","last_undegraded":"2024-05-28T14:16:49.559090+0000","last_fullsized":"2024-05-28T14:16:49.559090+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:54.517890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":54,"num_read_kb":221184,"num_write":113,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"134'110","reported_seq":181,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.551447+0000","last_change":"2024-05-28T14:14:43.770664+0000","last_active":"2024-05-28T14:16:49.551447+0000","last_peered":"2024-05-28T14:16:49.551447+0000","last_clean":"2024-05-28T14:16:49.551447+0000","last_became_active":"2024-05-28T14:14:43.770335+0000","last_became_peered":"2024-05-28T14:14:43.770335+0000","last_unstale":"2024-05-28T14:16:49.551447+0000","last_undegraded":"2024-05-28T14:16:49.551447+0000","last_fullsized":"2024-05-28T14:16:49.551447+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:20:08.770733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":102,"num_write_kb":311297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"134'99","reported_seq":182,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.562911+0000","last_change":"2024-05-28T14:14:43.769965+0000","last_active":"2024-05-28T14:16:49.562911+0000","last_peered":"2024-05-28T14:16:49.562911+0000","last_clean":"2024-05-28T14:16:49.562911+0000","last_became_active":"2024-05-28T14:14:43.769687+0000","last_became_peered":"2024-05-28T14:14:43.769687+0000","last_unstale":"2024-05-28T14:16:49.562911+0000","last_undegraded":"2024-05-28T14:16:49.562911+0000","last_fullsized":"2024-05-28T14:16:49.562911+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:43:30.318199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":67,"num_read_kb":274432,"num_write":93,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"134'112","reported_seq":236,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.559694+0000","last_change":"2024-05-28T14:14:43.758128+0000","last_active":"2024-05-28T14:16:49.559694+0000","last_peered":"2024-05-28T14:16:49.559694+0000","last_clean":"2024-05-28T14:16:49.559694+0000","last_became_active":"2024-05-28T14:14:43.757685+0000","last_became_peered":"2024-05-28T14:14:43.757685+0000","last_unstale":"2024-05-28T14:16:49.559694+0000","last_undegraded":"2024-05-28T14:16:49.559694+0000","last_fullsized":"2024-05-28T14:16:49.559694+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:53.917272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":108,"num_read_kb":442368,"num_write":103,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"134'101","reported_seq":222,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.544613+0000","last_change":"2024-05-28T14:14:43.769235+0000","last_active":"2024-05-28T14:16:49.544613+0000","last_peered":"2024-05-28T14:16:49.544613+0000","last_clean":"2024-05-28T14:16:49.544613+0000","last_became_active":"2024-05-28T14:14:43.768939+0000","last_became_peered":"2024-05-28T14:14:43.768939+0000","last_unstale":"2024-05-28T14:16:49.544613+0000","last_undegraded":"2024-05-28T14:16:49.544613+0000","last_fullsized":"2024-05-28T14:16:49.544613+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:26.186099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":105,"num_read_kb":413700,"num_write":98,"num_write_kb":315393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"134'110","reported_seq":208,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.570753+0000","last_change":"2024-05-28T14:14:43.767408+0000","last_active":"2024-05-28T14:16:49.570753+0000","last_peered":"2024-05-28T14:16:49.570753+0000","last_clean":"2024-05-28T14:16:49.570753+0000","last_became_active":"2024-05-28T14:14:43.766269+0000","last_became_peered":"2024-05-28T14:14:43.766269+0000","last_unstale":"2024-05-28T14:16:49.570753+0000","last_undegraded":"2024-05-28T14:16:49.570753+0000","last_fullsized":"2024-05-28T14:16:49.570753+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:45.051114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":105,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"134'100","reported_seq":182,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.561793+0000","last_change":"2024-05-28T14:14:43.762819+0000","last_active":"2024-05-28T14:16:49.561793+0000","last_peered":"2024-05-28T14:16:49.561793+0000","last_clean":"2024-05-28T14:16:49.561793+0000","last_became_active":"2024-05-28T14:14:43.762494+0000","last_became_peered":"2024-05-28T14:14:43.762494+0000","last_unstale":"2024-05-28T14:16:49.561793+0000","last_undegraded":"2024-05-28T14:16:49.561793+0000","last_fullsized":"2024-05-28T14:16:49.561793+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:50:53.753974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":66,"num_read_kb":270336,"num_write":92,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"134'83","reported_seq":130,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.565126+0000","last_change":"2024-05-28T14:14:43.761245+0000","last_active":"2024-05-28T14:16:49.565126+0000","last_peered":"2024-05-28T14:16:49.565126+0000","last_clean":"2024-05-28T14:16:49.565126+0000","last_became_active":"2024-05-28T14:14:43.760977+0000","last_became_peered":"2024-05-28T14:14:43.760977+0000","last_unstale":"2024-05-28T14:16:49.565126+0000","last_undegraded":"2024-05-28T14:16:49.565126+0000","last_fullsized":"2024-05-28T14:16:49.565126+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:14.494237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":31,"num_read_kb":126976,"num_write":74,"num_write_kb":249856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"134'131","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.556971+0000","last_change":"2024-05-28T14:14:43.758776+0000","last_active":"2024-05-28T14:16:49.556971+0000","last_peered":"2024-05-28T14:16:49.556971+0000","last_clean":"2024-05-28T14:16:49.556971+0000","last_became_active":"2024-05-28T14:14:43.758483+0000","last_became_peered":"2024-05-28T14:14:43.758483+0000","last_unstale":"2024-05-28T14:16:49.556971+0000","last_undegraded":"2024-05-28T14:16:49.556971+0000","last_fullsized":"2024-05-28T14:16:49.556971+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:44:52.410827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":57,"num_read_kb":225282,"num_write":123,"num_write_kb":368641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"134'101","reported_seq":164,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.555651+0000","last_change":"2024-05-28T14:14:43.756230+0000","last_active":"2024-05-28T14:16:49.555651+0000","last_peered":"2024-05-28T14:16:49.555651+0000","last_clean":"2024-05-28T14:16:49.555651+0000","last_became_active":"2024-05-28T14:14:43.755912+0000","last_became_peered":"2024-05-28T14:14:43.755912+0000","last_unstale":"2024-05-28T14:16:49.555651+0000","last_undegraded":"2024-05-28T14:16:49.555651+0000","last_fullsized":"2024-05-28T14:16:49.555651+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:22.709561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":47,"num_read_kb":180227,"num_write":95,"num_write_kb":286721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"134'118","reported_seq":232,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571290+0000","last_change":"2024-05-28T14:14:43.749113+0000","last_active":"2024-05-28T14:16:49.571290+0000","last_peered":"2024-05-28T14:16:49.571290+0000","last_clean":"2024-05-28T14:16:49.571290+0000","last_became_active":"2024-05-28T14:14:43.748801+0000","last_became_peered":"2024-05-28T14:14:43.748801+0000","last_unstale":"2024-05-28T14:16:49.571290+0000","last_undegraded":"2024-05-28T14:16:49.571290+0000","last_fullsized":"2024-05-28T14:16:49.571290+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:05:56.557590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":98,"num_read_kb":389123,"num_write":107,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"134'123","reported_seq":208,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.542886+0000","last_change":"2024-05-28T14:14:43.769214+0000","last_active":"2024-05-28T14:16:49.542886+0000","last_peered":"2024-05-28T14:16:49.542886+0000","last_clean":"2024-05-28T14:16:49.542886+0000","last_became_active":"2024-05-28T14:14:43.768757+0000","last_became_peered":"2024-05-28T14:14:43.768757+0000","last_unstale":"2024-05-28T14:16:49.542886+0000","last_undegraded":"2024-05-28T14:16:49.542886+0000","last_fullsized":"2024-05-28T14:16:49.542886+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:55:07.023689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":115,"num_write_kb":372736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"134'113","reported_seq":193,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.560761+0000","last_change":"2024-05-28T14:14:43.756647+0000","last_active":"2024-05-28T14:16:49.560761+0000","last_peered":"2024-05-28T14:16:49.560761+0000","last_clean":"2024-05-28T14:16:49.560761+0000","last_became_active":"2024-05-28T14:14:43.756244+0000","last_became_peered":"2024-05-28T14:14:43.756244+0000","last_unstale":"2024-05-28T14:16:49.560761+0000","last_undegraded":"2024-05-28T14:16:49.560761+0000","last_fullsized":"2024-05-28T14:16:49.560761+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:28:18.841947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":101,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"134'118","reported_seq":234,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.548472+0000","last_change":"2024-05-28T14:14:43.757686+0000","last_active":"2024-05-28T14:16:49.548472+0000","last_peered":"2024-05-28T14:16:49.548472+0000","last_clean":"2024-05-28T14:16:49.548472+0000","last_became_active":"2024-05-28T14:14:43.757288+0000","last_became_peered":"2024-05-28T14:14:43.757288+0000","last_unstale":"2024-05-28T14:16:49.548472+0000","last_undegraded":"2024-05-28T14:16:49.548472+0000","last_fullsized":"2024-05-28T14:16:49.548472+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:55:07.023689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":100,"num_read_kb":389125,"num_write":108,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"134'114","reported_seq":191,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.560882+0000","last_change":"2024-05-28T14:14:43.759851+0000","last_active":"2024-05-28T14:16:49.560882+0000","last_peered":"2024-05-28T14:16:49.560882+0000","last_clean":"2024-05-28T14:16:49.560882+0000","last_became_active":"2024-05-28T14:14:43.759580+0000","last_became_peered":"2024-05-28T14:14:43.759580+0000","last_unstale":"2024-05-28T14:16:49.560882+0000","last_undegraded":"2024-05-28T14:16:49.560882+0000","last_fullsized":"2024-05-28T14:16:49.560882+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:50:20.486680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":61,"num_read_kb":241666,"num_write":107,"num_write_kb":356353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"134'109","reported_seq":184,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.555983+0000","last_change":"2024-05-28T14:14:43.768741+0000","last_active":"2024-05-28T14:16:49.555983+0000","last_peered":"2024-05-28T14:16:49.555983+0000","last_clean":"2024-05-28T14:16:49.555983+0000","last_became_active":"2024-05-28T14:14:43.767741+0000","last_became_peered":"2024-05-28T14:14:43.767741+0000","last_unstale":"2024-05-28T14:16:49.555983+0000","last_undegraded":"2024-05-28T14:16:49.555983+0000","last_fullsized":"2024-05-28T14:16:49.555983+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:20.117904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":62,"num_read_kb":253952,"num_write":101,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"134'111","reported_seq":186,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.541445+0000","last_change":"2024-05-28T14:14:43.755177+0000","last_active":"2024-05-28T14:16:49.541445+0000","last_peered":"2024-05-28T14:16:49.541445+0000","last_clean":"2024-05-28T14:16:49.541445+0000","last_became_active":"2024-05-28T14:14:43.753794+0000","last_became_peered":"2024-05-28T14:14:43.753794+0000","last_unstale":"2024-05-28T14:16:49.541445+0000","last_undegraded":"2024-05-28T14:16:49.541445+0000","last_fullsized":"2024-05-28T14:16:49.541445+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:56:28.627044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":59,"num_read_kb":241664,"num_write":103,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"134'122","reported_seq":206,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.553873+0000","last_change":"2024-05-28T14:14:43.763725+0000","last_active":"2024-05-28T14:16:49.553873+0000","last_peered":"2024-05-28T14:16:49.553873+0000","last_clean":"2024-05-28T14:16:49.553873+0000","last_became_active":"2024-05-28T14:14:43.763332+0000","last_became_peered":"2024-05-28T14:14:43.763332+0000","last_unstale":"2024-05-28T14:16:49.553873+0000","last_undegraded":"2024-05-28T14:16:49.553873+0000","last_fullsized":"2024-05-28T14:16:49.553873+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:22:10.253064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298372,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":68,"num_read_kb":274433,"num_write":112,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"134'121","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.550549+0000","last_change":"2024-05-28T14:14:43.759381+0000","last_active":"2024-05-28T14:16:49.550549+0000","last_peered":"2024-05-28T14:16:49.550549+0000","last_clean":"2024-05-28T14:16:49.550549+0000","last_became_active":"2024-05-28T14:14:43.758722+0000","last_became_peered":"2024-05-28T14:14:43.758722+0000","last_unstale":"2024-05-28T14:16:49.550549+0000","last_undegraded":"2024-05-28T14:16:49.550549+0000","last_fullsized":"2024-05-28T14:16:49.550549+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:48:30.643597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251658240,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":112,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"134'104","reported_seq":195,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571297+0000","last_change":"2024-05-28T14:14:43.768220+0000","last_active":"2024-05-28T14:16:49.571297+0000","last_peered":"2024-05-28T14:16:49.571297+0000","last_clean":"2024-05-28T14:16:49.571297+0000","last_became_active":"2024-05-28T14:14:43.767925+0000","last_became_peered":"2024-05-28T14:14:43.767925+0000","last_unstale":"2024-05-28T14:16:49.571297+0000","last_undegraded":"2024-05-28T14:16:49.571297+0000","last_fullsized":"2024-05-28T14:16:49.571297+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:35:31.221160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":75,"num_read_kb":294915,"num_write":99,"num_write_kb":290818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"134'99","reported_seq":149,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.539531+0000","last_change":"2024-05-28T14:14:43.746536+0000","last_active":"2024-05-28T14:16:49.539531+0000","last_peered":"2024-05-28T14:16:49.539531+0000","last_clean":"2024-05-28T14:16:49.539531+0000","last_became_active":"2024-05-28T14:14:43.746249+0000","last_became_peered":"2024-05-28T14:14:43.746249+0000","last_unstale":"2024-05-28T14:16:49.539531+0000","last_undegraded":"2024-05-28T14:16:49.539531+0000","last_fullsized":"2024-05-28T14:16:49.539531+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:30:48.180755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909764,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":34,"num_read_kb":131074,"num_write":94,"num_write_kb":290818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"134'111","reported_seq":200,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.564301+0000","last_change":"2024-05-28T14:14:43.761643+0000","last_active":"2024-05-28T14:16:49.564301+0000","last_peered":"2024-05-28T14:16:49.564301+0000","last_clean":"2024-05-28T14:16:49.564301+0000","last_became_active":"2024-05-28T14:14:43.761161+0000","last_became_peered":"2024-05-28T14:14:43.761161+0000","last_unstale":"2024-05-28T14:16:49.564301+0000","last_undegraded":"2024-05-28T14:16:49.564301+0000","last_fullsized":"2024-05-28T14:16:49.564301+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:11.879163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075588,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":72,"num_read_kb":294912,"num_write":104,"num_write_kb":327681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"134'117","reported_seq":203,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.565084+0000","last_change":"2024-05-28T14:14:43.764133+0000","last_active":"2024-05-28T14:16:49.565084+0000","last_peered":"2024-05-28T14:16:49.565084+0000","last_clean":"2024-05-28T14:16:49.565084+0000","last_became_active":"2024-05-28T14:14:43.763522+0000","last_became_peered":"2024-05-28T14:14:43.763522+0000","last_unstale":"2024-05-28T14:16:49.565084+0000","last_undegraded":"2024-05-28T14:16:49.565084+0000","last_fullsized":"2024-05-28T14:16:49.565084+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:05.864969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251658240,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":106,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"134'99","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.558327+0000","last_change":"2024-05-28T14:14:43.763447+0000","last_active":"2024-05-28T14:16:27.558327+0000","last_peered":"2024-05-28T14:16:27.558327+0000","last_clean":"2024-05-28T14:16:27.558327+0000","last_became_active":"2024-05-28T14:14:43.763142+0000","last_became_peered":"2024-05-28T14:14:43.763142+0000","last_unstale":"2024-05-28T14:16:27.558327+0000","last_undegraded":"2024-05-28T14:16:27.558327+0000","last_fullsized":"2024-05-28T14:16:27.558327+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:38:22.872839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":88,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"134'106","reported_seq":198,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.563409+0000","last_change":"2024-05-28T14:14:43.747637+0000","last_active":"2024-05-28T14:16:49.563409+0000","last_peered":"2024-05-28T14:16:49.563409+0000","last_clean":"2024-05-28T14:16:49.563409+0000","last_became_active":"2024-05-28T14:14:43.747272+0000","last_became_peered":"2024-05-28T14:14:43.747272+0000","last_unstale":"2024-05-28T14:16:49.563409+0000","last_undegraded":"2024-05-28T14:16:49.563409+0000","last_fullsized":"2024-05-28T14:16:49.563409+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:19.860056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":96,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"134'107","reported_seq":168,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.534735+0000","last_change":"2024-05-28T14:14:43.759397+0000","last_active":"2024-05-28T14:16:49.534735+0000","last_peered":"2024-05-28T14:16:49.534735+0000","last_clean":"2024-05-28T14:16:49.534735+0000","last_became_active":"2024-05-28T14:14:43.759136+0000","last_became_peered":"2024-05-28T14:14:43.759136+0000","last_unstale":"2024-05-28T14:16:49.534735+0000","last_undegraded":"2024-05-28T14:16:49.534735+0000","last_fullsized":"2024-05-28T14:16:49.534735+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:38.538536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":44,"num_read_kb":180224,"num_write":103,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"131'4","reported_seq":23,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973209+0000","last_change":"2024-05-28T14:14:42.751872+0000","last_active":"2024-05-28T14:15:05.973209+0000","last_peered":"2024-05-28T14:15:05.973209+0000","last_clean":"2024-05-28T14:15:05.973209+0000","last_became_active":"2024-05-28T14:14:42.751450+0000","last_became_peered":"2024-05-28T14:14:42.751450+0000","last_unstale":"2024-05-28T14:15:05.973209+0000","last_undegraded":"2024-05-28T14:15:05.973209+0000","last_fullsized":"2024-05-28T14:15:05.973209+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:48:49.635601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"134'121","reported_seq":206,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.559319+0000","last_change":"2024-05-28T14:14:43.766763+0000","last_active":"2024-05-28T14:16:49.559319+0000","last_peered":"2024-05-28T14:16:49.559319+0000","last_clean":"2024-05-28T14:16:49.559319+0000","last_became_active":"2024-05-28T14:14:43.766347+0000","last_became_peered":"2024-05-28T14:14:43.766347+0000","last_unstale":"2024-05-28T14:16:49.559319+0000","last_undegraded":"2024-05-28T14:16:49.559319+0000","last_fullsized":"2024-05-28T14:16:49.559319+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:43:49.334848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":72,"num_read_kb":290817,"num_write":112,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013729+0000","last_change":"2024-05-28T14:14:42.742952+0000","last_active":"2024-05-28T14:15:06.013729+0000","last_peered":"2024-05-28T14:15:06.013729+0000","last_clean":"2024-05-28T14:15:06.013729+0000","last_became_active":"2024-05-28T14:14:42.742668+0000","last_became_peered":"2024-05-28T14:14:42.742668+0000","last_unstale":"2024-05-28T14:15:06.013729+0000","last_undegraded":"2024-05-28T14:15:06.013729+0000","last_fullsized":"2024-05-28T14:15:06.013729+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:53:04.792926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"134'87","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.444881+0000","last_change":"2024-05-28T14:14:43.765567+0000","last_active":"2024-05-28T14:16:27.444881+0000","last_peered":"2024-05-28T14:16:27.444881+0000","last_clean":"2024-05-28T14:16:27.444881+0000","last_became_active":"2024-05-28T14:14:43.765230+0000","last_became_peered":"2024-05-28T14:14:43.765230+0000","last_unstale":"2024-05-28T14:16:27.444881+0000","last_undegraded":"2024-05-28T14:16:27.444881+0000","last_fullsized":"2024-05-28T14:16:27.444881+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:56:02.866180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":75,"num_read_kb":303105,"num_write":87,"num_write_kb":270337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"134'3","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971807+0000","last_change":"2024-05-28T14:14:42.733510+0000","last_active":"2024-05-28T14:15:05.971807+0000","last_peered":"2024-05-28T14:15:05.971807+0000","last_clean":"2024-05-28T14:15:05.971807+0000","last_became_active":"2024-05-28T14:14:42.733231+0000","last_became_peered":"2024-05-28T14:14:42.733231+0000","last_unstale":"2024-05-28T14:15:05.971807+0000","last_undegraded":"2024-05-28T14:15:05.971807+0000","last_fullsized":"2024-05-28T14:15:05.971807+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:37.532653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"134'114","reported_seq":215,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.536179+0000","last_change":"2024-05-28T14:14:43.760573+0000","last_active":"2024-05-28T14:16:49.536179+0000","last_peered":"2024-05-28T14:16:49.536179+0000","last_clean":"2024-05-28T14:16:49.536179+0000","last_became_active":"2024-05-28T14:14:43.759307+0000","last_became_peered":"2024-05-28T14:14:43.759307+0000","last_unstale":"2024-05-28T14:16:49.536179+0000","last_undegraded":"2024-05-28T14:16:49.536179+0000","last_fullsized":"2024-05-28T14:16:49.536179+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:30:49.015188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":107,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"131'2","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077844+0000","last_change":"2024-05-28T14:14:42.745411+0000","last_active":"2024-05-28T14:15:06.077844+0000","last_peered":"2024-05-28T14:15:06.077844+0000","last_clean":"2024-05-28T14:15:06.077844+0000","last_became_active":"2024-05-28T14:14:42.745097+0000","last_became_peered":"2024-05-28T14:14:42.745097+0000","last_unstale":"2024-05-28T14:15:06.077844+0000","last_undegraded":"2024-05-28T14:15:06.077844+0000","last_fullsized":"2024-05-28T14:15:06.077844+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:39:18.579687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"134'112","reported_seq":189,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571629+0000","last_change":"2024-05-28T14:14:43.751125+0000","last_active":"2024-05-28T14:16:49.571629+0000","last_peered":"2024-05-28T14:16:49.571629+0000","last_clean":"2024-05-28T14:16:49.571629+0000","last_became_active":"2024-05-28T14:14:43.747282+0000","last_became_peered":"2024-05-28T14:14:43.747282+0000","last_unstale":"2024-05-28T14:16:49.571629+0000","last_undegraded":"2024-05-28T14:16:49.571629+0000","last_fullsized":"2024-05-28T14:16:49.571629+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:44:03.941731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":105,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077611+0000","last_change":"2024-05-28T14:14:42.741716+0000","last_active":"2024-05-28T14:15:06.077611+0000","last_peered":"2024-05-28T14:15:06.077611+0000","last_clean":"2024-05-28T14:15:06.077611+0000","last_became_active":"2024-05-28T14:14:42.741425+0000","last_became_peered":"2024-05-28T14:14:42.741425+0000","last_unstale":"2024-05-28T14:15:06.077611+0000","last_undegraded":"2024-05-28T14:15:06.077611+0000","last_fullsized":"2024-05-28T14:15:06.077611+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:36:40.085174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"134'114","reported_seq":229,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.566591+0000","last_change":"2024-05-28T14:14:43.760960+0000","last_active":"2024-05-28T14:16:49.566591+0000","last_peered":"2024-05-28T14:16:49.566591+0000","last_clean":"2024-05-28T14:16:49.566591+0000","last_became_active":"2024-05-28T14:14:43.760691+0000","last_became_peered":"2024-05-28T14:14:43.760691+0000","last_unstale":"2024-05-28T14:16:49.566591+0000","last_undegraded":"2024-05-28T14:16:49.566591+0000","last_fullsized":"2024-05-28T14:16:49.566591+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:33:40.268549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":99,"num_read_kb":405504,"num_write":102,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969182+0000","last_change":"2024-05-28T14:14:42.739561+0000","last_active":"2024-05-28T14:15:05.969182+0000","last_peered":"2024-05-28T14:15:05.969182+0000","last_clean":"2024-05-28T14:15:05.969182+0000","last_became_active":"2024-05-28T14:14:42.739191+0000","last_became_peered":"2024-05-28T14:14:42.739191+0000","last_unstale":"2024-05-28T14:15:05.969182+0000","last_undegraded":"2024-05-28T14:15:05.969182+0000","last_fullsized":"2024-05-28T14:15:05.969182+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:49:44.727202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"134'109","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.558982+0000","last_change":"2024-05-28T14:14:43.762652+0000","last_active":"2024-05-28T14:16:49.558982+0000","last_peered":"2024-05-28T14:16:49.558982+0000","last_clean":"2024-05-28T14:16:49.558982+0000","last_became_active":"2024-05-28T14:14:43.761547+0000","last_became_peered":"2024-05-28T14:14:43.761547+0000","last_unstale":"2024-05-28T14:16:49.558982+0000","last_undegraded":"2024-05-28T14:16:49.558982+0000","last_fullsized":"2024-05-28T14:16:49.558982+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:11:03.103530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":102,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973841+0000","last_change":"2024-05-28T14:14:42.745986+0000","last_active":"2024-05-28T14:15:05.973841+0000","last_peered":"2024-05-28T14:15:05.973841+0000","last_clean":"2024-05-28T14:15:05.973841+0000","last_became_active":"2024-05-28T14:14:42.745691+0000","last_became_peered":"2024-05-28T14:14:42.745691+0000","last_unstale":"2024-05-28T14:15:05.973841+0000","last_undegraded":"2024-05-28T14:15:05.973841+0000","last_fullsized":"2024-05-28T14:15:05.973841+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:22:09.220099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"134'107","reported_seq":189,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.555005+0000","last_change":"2024-05-28T14:14:43.764538+0000","last_active":"2024-05-28T14:16:49.555005+0000","last_peered":"2024-05-28T14:16:49.555005+0000","last_clean":"2024-05-28T14:16:49.555005+0000","last_became_active":"2024-05-28T14:14:43.762358+0000","last_became_peered":"2024-05-28T14:14:43.762358+0000","last_unstale":"2024-05-28T14:16:49.555005+0000","last_undegraded":"2024-05-28T14:16:49.555005+0000","last_fullsized":"2024-05-28T14:16:49.555005+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:37:33.489101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":66,"num_read_kb":262146,"num_write":97,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077572+0000","last_change":"2024-05-28T14:14:42.747379+0000","last_active":"2024-05-28T14:15:06.077572+0000","last_peered":"2024-05-28T14:15:06.077572+0000","last_clean":"2024-05-28T14:15:06.077572+0000","last_became_active":"2024-05-28T14:14:42.747127+0000","last_became_peered":"2024-05-28T14:14:42.747127+0000","last_unstale":"2024-05-28T14:15:06.077572+0000","last_undegraded":"2024-05-28T14:15:06.077572+0000","last_fullsized":"2024-05-28T14:15:06.077572+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:10:35.041686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"134'96","reported_seq":149,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.561345+0000","last_change":"2024-05-28T14:14:43.757145+0000","last_active":"2024-05-28T14:16:49.561345+0000","last_peered":"2024-05-28T14:16:49.561345+0000","last_clean":"2024-05-28T14:16:49.561345+0000","last_became_active":"2024-05-28T14:14:43.756803+0000","last_became_peered":"2024-05-28T14:14:43.756803+0000","last_unstale":"2024-05-28T14:16:49.561345+0000","last_undegraded":"2024-05-28T14:16:49.561345+0000","last_fullsized":"2024-05-28T14:16:49.561345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:30:48.180755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188743680,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":85,"num_write_kb":266240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"134'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013914+0000","last_change":"2024-05-28T14:14:42.746521+0000","last_active":"2024-05-28T14:15:06.013914+0000","last_peered":"2024-05-28T14:15:06.013914+0000","last_clean":"2024-05-28T14:15:06.013914+0000","last_became_active":"2024-05-28T14:14:42.746243+0000","last_became_peered":"2024-05-28T14:14:42.746243+0000","last_unstale":"2024-05-28T14:15:06.013914+0000","last_undegraded":"2024-05-28T14:15:06.013914+0000","last_fullsized":"2024-05-28T14:15:06.013914+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:44.045231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"134'96","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.543616+0000","last_change":"2024-05-28T14:14:43.751232+0000","last_active":"2024-05-28T14:16:49.543616+0000","last_peered":"2024-05-28T14:16:49.543616+0000","last_clean":"2024-05-28T14:16:49.543616+0000","last_became_active":"2024-05-28T14:14:43.750768+0000","last_became_peered":"2024-05-28T14:14:43.750768+0000","last_unstale":"2024-05-28T14:16:49.543616+0000","last_undegraded":"2024-05-28T14:16:49.543616+0000","last_fullsized":"2024-05-28T14:16:49.543616+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:37:33.489101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":41,"num_read_kb":167936,"num_write":89,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969503+0000","last_change":"2024-05-28T14:14:42.745263+0000","last_active":"2024-05-28T14:15:05.969503+0000","last_peered":"2024-05-28T14:15:05.969503+0000","last_clean":"2024-05-28T14:15:05.969503+0000","last_became_active":"2024-05-28T14:14:42.744565+0000","last_became_peered":"2024-05-28T14:14:42.744565+0000","last_unstale":"2024-05-28T14:15:05.969503+0000","last_undegraded":"2024-05-28T14:15:05.969503+0000","last_fullsized":"2024-05-28T14:15:05.969503+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:59.384784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"134'85","reported_seq":184,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.543513+0000","last_change":"2024-05-28T14:14:43.760442+0000","last_active":"2024-05-28T14:16:49.543513+0000","last_peered":"2024-05-28T14:16:49.543513+0000","last_clean":"2024-05-28T14:16:49.543513+0000","last_became_active":"2024-05-28T14:14:43.760106+0000","last_became_peered":"2024-05-28T14:14:43.760106+0000","last_unstale":"2024-05-28T14:16:49.543513+0000","last_undegraded":"2024-05-28T14:16:49.543513+0000","last_fullsized":"2024-05-28T14:16:49.543513+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:56:47.127973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":80,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"134'25","reported_seq":43,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.001817+0000","last_change":"2024-05-28T14:14:42.735627+0000","last_active":"2024-05-28T14:16:27.001817+0000","last_peered":"2024-05-28T14:16:27.001817+0000","last_clean":"2024-05-28T14:16:27.001817+0000","last_became_active":"2024-05-28T14:14:42.734095+0000","last_became_peered":"2024-05-28T14:14:42.734095+0000","last_unstale":"2024-05-28T14:16:27.001817+0000","last_undegraded":"2024-05-28T14:16:27.001817+0000","last_fullsized":"2024-05-28T14:16:27.001817+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:07.310371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"134'95","reported_seq":175,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.552078+0000","last_change":"2024-05-28T14:14:43.768144+0000","last_active":"2024-05-28T14:16:49.552078+0000","last_peered":"2024-05-28T14:16:49.552078+0000","last_clean":"2024-05-28T14:16:49.552078+0000","last_became_active":"2024-05-28T14:14:43.766334+0000","last_became_peered":"2024-05-28T14:14:43.766334+0000","last_unstale":"2024-05-28T14:16:49.552078+0000","last_undegraded":"2024-05-28T14:16:49.552078+0000","last_fullsized":"2024-05-28T14:16:49.552078+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:56:51.599701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":86,"num_write_kb":274434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"134'459","reported_seq":477,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.508408+0000","last_change":"2024-05-28T14:14:42.744568+0000","last_active":"2024-05-28T14:16:49.508408+0000","last_peered":"2024-05-28T14:16:49.508408+0000","last_clean":"2024-05-28T14:16:49.508408+0000","last_became_active":"2024-05-28T14:14:42.744240+0000","last_became_peered":"2024-05-28T14:14:42.744240+0000","last_unstale":"2024-05-28T14:16:49.508408+0000","last_undegraded":"2024-05-28T14:16:49.508408+0000","last_fullsized":"2024-05-28T14:16:49.508408+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:25:02.568973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1780932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":457,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"134'125","reported_seq":195,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.556970+0000","last_change":"2024-05-28T14:14:43.758776+0000","last_active":"2024-05-28T14:16:49.556970+0000","last_peered":"2024-05-28T14:16:49.556970+0000","last_clean":"2024-05-28T14:16:49.556970+0000","last_became_active":"2024-05-28T14:14:43.758474+0000","last_became_peered":"2024-05-28T14:14:43.758474+0000","last_unstale":"2024-05-28T14:16:49.556970+0000","last_undegraded":"2024-05-28T14:16:49.556970+0000","last_fullsized":"2024-05-28T14:16:49.556970+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:00:34.527970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":54,"num_read_kb":212994,"num_write":119,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973982+0000","last_change":"2024-05-28T14:14:42.737773+0000","last_active":"2024-05-28T14:15:05.973982+0000","last_peered":"2024-05-28T14:15:05.973982+0000","last_clean":"2024-05-28T14:15:05.973982+0000","last_became_active":"2024-05-28T14:14:42.737442+0000","last_became_peered":"2024-05-28T14:14:42.737442+0000","last_unstale":"2024-05-28T14:15:05.973982+0000","last_undegraded":"2024-05-28T14:15:05.973982+0000","last_fullsized":"2024-05-28T14:15:05.973982+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:45:49.229094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"134'110","reported_seq":187,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.560960+0000","last_change":"2024-05-28T14:14:43.762668+0000","last_active":"2024-05-28T14:16:49.560960+0000","last_peered":"2024-05-28T14:16:49.560960+0000","last_clean":"2024-05-28T14:16:49.560960+0000","last_became_active":"2024-05-28T14:14:43.762049+0000","last_became_peered":"2024-05-28T14:14:43.762049+0000","last_unstale":"2024-05-28T14:16:49.560960+0000","last_undegraded":"2024-05-28T14:16:49.560960+0000","last_fullsized":"2024-05-28T14:16:49.560960+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:57:46.984712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":106,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"134'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077675+0000","last_change":"2024-05-28T14:14:42.746077+0000","last_active":"2024-05-28T14:15:06.077675+0000","last_peered":"2024-05-28T14:15:06.077675+0000","last_clean":"2024-05-28T14:15:06.077675+0000","last_became_active":"2024-05-28T14:14:42.745811+0000","last_became_peered":"2024-05-28T14:14:42.745811+0000","last_unstale":"2024-05-28T14:15:06.077675+0000","last_undegraded":"2024-05-28T14:15:06.077675+0000","last_fullsized":"2024-05-28T14:15:06.077675+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:26:07.187114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"134'123","reported_seq":211,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.556596+0000","last_change":"2024-05-28T14:14:43.768732+0000","last_active":"2024-05-28T14:16:49.556596+0000","last_peered":"2024-05-28T14:16:49.556596+0000","last_clean":"2024-05-28T14:16:49.556596+0000","last_became_active":"2024-05-28T14:14:43.768334+0000","last_became_peered":"2024-05-28T14:14:43.768334+0000","last_unstale":"2024-05-28T14:16:49.556596+0000","last_undegraded":"2024-05-28T14:16:49.556596+0000","last_fullsized":"2024-05-28T14:16:49.556596+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:01:12.201409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852724,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":72,"num_read_kb":282627,"num_write":118,"num_write_kb":360450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"131'8","reported_seq":26,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.975292+0000","last_change":"2024-05-28T14:14:42.742861+0000","last_active":"2024-05-28T14:15:05.975292+0000","last_peered":"2024-05-28T14:15:05.975292+0000","last_clean":"2024-05-28T14:15:05.975292+0000","last_became_active":"2024-05-28T14:14:42.742518+0000","last_became_peered":"2024-05-28T14:14:42.742518+0000","last_unstale":"2024-05-28T14:15:05.975292+0000","last_undegraded":"2024-05-28T14:15:05.975292+0000","last_fullsized":"2024-05-28T14:15:05.975292+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:43:29.312316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"134'90","reported_seq":128,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.570119+0000","last_change":"2024-05-28T14:14:43.762081+0000","last_active":"2024-05-28T14:16:27.570119+0000","last_peered":"2024-05-28T14:16:27.570119+0000","last_clean":"2024-05-28T14:16:27.570119+0000","last_became_active":"2024-05-28T14:14:43.761775+0000","last_became_peered":"2024-05-28T14:14:43.761775+0000","last_unstale":"2024-05-28T14:16:27.570119+0000","last_undegraded":"2024-05-28T14:16:27.570119+0000","last_fullsized":"2024-05-28T14:16:27.570119+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:07:11.105627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176160768,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":25,"num_read_kb":102400,"num_write":86,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013893+0000","last_change":"2024-05-28T14:14:42.746381+0000","last_active":"2024-05-28T14:15:06.013893+0000","last_peered":"2024-05-28T14:15:06.013893+0000","last_clean":"2024-05-28T14:15:06.013893+0000","last_became_active":"2024-05-28T14:14:42.746098+0000","last_became_peered":"2024-05-28T14:14:42.746098+0000","last_unstale":"2024-05-28T14:15:06.013893+0000","last_undegraded":"2024-05-28T14:15:06.013893+0000","last_fullsized":"2024-05-28T14:15:06.013893+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:04.859086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"134'135","reported_seq":233,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.546980+0000","last_change":"2024-05-28T14:14:43.756453+0000","last_active":"2024-05-28T14:16:49.546980+0000","last_peered":"2024-05-28T14:16:49.546980+0000","last_clean":"2024-05-28T14:16:49.546980+0000","last_became_active":"2024-05-28T14:14:43.756138+0000","last_became_peered":"2024-05-28T14:14:43.756138+0000","last_unstale":"2024-05-28T14:16:49.546980+0000","last_undegraded":"2024-05-28T14:16:49.546980+0000","last_fullsized":"2024-05-28T14:16:49.546980+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:07.658669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":129,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"131'2","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971765+0000","last_change":"2024-05-28T14:14:42.742055+0000","last_active":"2024-05-28T14:15:05.971765+0000","last_peered":"2024-05-28T14:15:05.971765+0000","last_clean":"2024-05-28T14:15:05.971765+0000","last_became_active":"2024-05-28T14:14:42.741754+0000","last_became_peered":"2024-05-28T14:14:42.741754+0000","last_unstale":"2024-05-28T14:15:05.971765+0000","last_undegraded":"2024-05-28T14:15:05.971765+0000","last_fullsized":"2024-05-28T14:15:05.971765+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:52:25.036897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"134'111","reported_seq":190,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.560349+0000","last_change":"2024-05-28T14:14:43.758320+0000","last_active":"2024-05-28T14:16:49.560349+0000","last_peered":"2024-05-28T14:16:49.560349+0000","last_clean":"2024-05-28T14:16:49.560349+0000","last_became_active":"2024-05-28T14:14:43.757996+0000","last_became_peered":"2024-05-28T14:14:43.757996+0000","last_unstale":"2024-05-28T14:16:49.560349+0000","last_undegraded":"2024-05-28T14:16:49.560349+0000","last_fullsized":"2024-05-28T14:16:49.560349+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:18.151246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":63,"num_read_kb":241668,"num_write":101,"num_write_kb":299010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.974449+0000","last_change":"2024-05-28T14:14:42.734922+0000","last_active":"2024-05-28T14:15:05.974449+0000","last_peered":"2024-05-28T14:15:05.974449+0000","last_clean":"2024-05-28T14:15:05.974449+0000","last_became_active":"2024-05-28T14:14:42.734647+0000","last_became_peered":"2024-05-28T14:14:42.734647+0000","last_unstale":"2024-05-28T14:15:05.974449+0000","last_undegraded":"2024-05-28T14:15:05.974449+0000","last_fullsized":"2024-05-28T14:15:05.974449+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:36:34.326176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"134'112","reported_seq":210,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.554691+0000","last_change":"2024-05-28T14:14:43.764186+0000","last_active":"2024-05-28T14:16:49.554691+0000","last_peered":"2024-05-28T14:16:49.554691+0000","last_clean":"2024-05-28T14:16:49.554691+0000","last_became_active":"2024-05-28T14:14:43.762407+0000","last_became_peered":"2024-05-28T14:14:43.762407+0000","last_unstale":"2024-05-28T14:16:49.554691+0000","last_undegraded":"2024-05-28T14:16:49.554691+0000","last_fullsized":"2024-05-28T14:16:49.554691+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:07:34.099826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":82,"num_read_kb":327682,"num_write":107,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013781+0000","last_change":"2024-05-28T14:14:42.733737+0000","last_active":"2024-05-28T14:15:06.013781+0000","last_peered":"2024-05-28T14:15:06.013781+0000","last_clean":"2024-05-28T14:15:06.013781+0000","last_became_active":"2024-05-28T14:14:42.733393+0000","last_became_peered":"2024-05-28T14:14:42.733393+0000","last_unstale":"2024-05-28T14:15:06.013781+0000","last_undegraded":"2024-05-28T14:15:06.013781+0000","last_fullsized":"2024-05-28T14:15:06.013781+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:13.036848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"134'107","reported_seq":208,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571484+0000","last_change":"2024-05-28T14:14:43.767748+0000","last_active":"2024-05-28T14:16:49.571484+0000","last_peered":"2024-05-28T14:16:49.571484+0000","last_clean":"2024-05-28T14:16:49.571484+0000","last_became_active":"2024-05-28T14:14:43.767351+0000","last_became_peered":"2024-05-28T14:14:43.767351+0000","last_unstale":"2024-05-28T14:16:49.571484+0000","last_undegraded":"2024-05-28T14:16:49.571484+0000","last_fullsized":"2024-05-28T14:16:49.571484+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:27:11.131902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":85,"num_read_kb":348160,"num_write":103,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"134'3","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013626+0000","last_change":"2024-05-28T14:14:42.746787+0000","last_active":"2024-05-28T14:15:06.013626+0000","last_peered":"2024-05-28T14:15:06.013626+0000","last_clean":"2024-05-28T14:15:06.013626+0000","last_became_active":"2024-05-28T14:14:42.746273+0000","last_became_peered":"2024-05-28T14:14:42.746273+0000","last_unstale":"2024-05-28T14:15:06.013626+0000","last_undegraded":"2024-05-28T14:15:06.013626+0000","last_fullsized":"2024-05-28T14:15:06.013626+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:38.279660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"134'119","reported_seq":202,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571021+0000","last_change":"2024-05-28T14:14:43.771048+0000","last_active":"2024-05-28T14:16:49.571021+0000","last_peered":"2024-05-28T14:16:49.571021+0000","last_clean":"2024-05-28T14:16:49.571021+0000","last_became_active":"2024-05-28T14:14:43.770751+0000","last_became_peered":"2024-05-28T14:14:43.770751+0000","last_unstale":"2024-05-28T14:16:49.571021+0000","last_undegraded":"2024-05-28T14:16:49.571021+0000","last_fullsized":"2024-05-28T14:16:49.571021+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:34:09.617785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272629760,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":111,"num_write_kb":348163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969248+0000","last_change":"2024-05-28T14:14:42.742397+0000","last_active":"2024-05-28T14:15:05.969248+0000","last_peered":"2024-05-28T14:15:05.969248+0000","last_clean":"2024-05-28T14:15:05.969248+0000","last_became_active":"2024-05-28T14:14:42.742104+0000","last_became_peered":"2024-05-28T14:14:42.742104+0000","last_unstale":"2024-05-28T14:15:05.969248+0000","last_undegraded":"2024-05-28T14:15:05.969248+0000","last_fullsized":"2024-05-28T14:15:05.969248+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:21.703678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"134'122","reported_seq":187,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.562957+0000","last_change":"2024-05-28T14:14:43.756230+0000","last_active":"2024-05-28T14:16:49.562957+0000","last_peered":"2024-05-28T14:16:49.562957+0000","last_clean":"2024-05-28T14:16:49.562957+0000","last_became_active":"2024-05-28T14:14:43.755945+0000","last_became_peered":"2024-05-28T14:14:43.755945+0000","last_unstale":"2024-05-28T14:16:49.562957+0000","last_undegraded":"2024-05-28T14:16:49.562957+0000","last_fullsized":"2024-05-28T14:16:49.562957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:30:48.180755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":49,"num_read_kb":196609,"num_write":111,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969005+0000","last_change":"2024-05-28T14:14:42.737496+0000","last_active":"2024-05-28T14:15:05.969005+0000","last_peered":"2024-05-28T14:15:05.969005+0000","last_clean":"2024-05-28T14:15:05.969005+0000","last_became_active":"2024-05-28T14:14:42.737194+0000","last_became_peered":"2024-05-28T14:14:42.737194+0000","last_unstale":"2024-05-28T14:15:05.969005+0000","last_undegraded":"2024-05-28T14:15:05.969005+0000","last_fullsized":"2024-05-28T14:15:05.969005+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:56:10.665924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"134'110","reported_seq":190,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.515934+0000","last_change":"2024-05-28T14:14:43.758200+0000","last_active":"2024-05-28T14:16:27.515934+0000","last_peered":"2024-05-28T14:16:27.515934+0000","last_clean":"2024-05-28T14:16:27.515934+0000","last_became_active":"2024-05-28T14:14:43.757875+0000","last_became_peered":"2024-05-28T14:14:43.757875+0000","last_unstale":"2024-05-28T14:16:27.515934+0000","last_undegraded":"2024-05-28T14:16:27.515934+0000","last_fullsized":"2024-05-28T14:16:27.515934+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:53.917272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":64,"num_read_kb":253954,"num_write":110,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"134'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971403+0000","last_change":"2024-05-28T14:14:42.738809+0000","last_active":"2024-05-28T14:15:05.971403+0000","last_peered":"2024-05-28T14:15:05.971403+0000","last_clean":"2024-05-28T14:15:05.971403+0000","last_became_active":"2024-05-28T14:14:42.738494+0000","last_became_peered":"2024-05-28T14:14:42.738494+0000","last_unstale":"2024-05-28T14:15:05.971403+0000","last_undegraded":"2024-05-28T14:15:05.971403+0000","last_fullsized":"2024-05-28T14:15:05.971403+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:17.145363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"134'127","reported_seq":183,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.568965+0000","last_change":"2024-05-28T14:14:43.770616+0000","last_active":"2024-05-28T14:16:49.568965+0000","last_peered":"2024-05-28T14:16:49.568965+0000","last_clean":"2024-05-28T14:16:49.568965+0000","last_became_active":"2024-05-28T14:14:43.770253+0000","last_became_peered":"2024-05-28T14:14:43.770253+0000","last_unstale":"2024-05-28T14:16:49.568965+0000","last_undegraded":"2024-05-28T14:16:49.568965+0000","last_fullsized":"2024-05-28T14:16:49.568965+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:49:45.733085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":40,"num_read_kb":163840,"num_write":118,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"131'2","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.974495+0000","last_change":"2024-05-28T14:14:42.746355+0000","last_active":"2024-05-28T14:15:05.974495+0000","last_peered":"2024-05-28T14:15:05.974495+0000","last_clean":"2024-05-28T14:15:05.974495+0000","last_became_active":"2024-05-28T14:14:42.745926+0000","last_became_peered":"2024-05-28T14:14:42.745926+0000","last_unstale":"2024-05-28T14:15:05.974495+0000","last_undegraded":"2024-05-28T14:15:05.974495+0000","last_fullsized":"2024-05-28T14:15:05.974495+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:54:47.273264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"134'115","reported_seq":222,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.553138+0000","last_change":"2024-05-28T14:15:06.178022+0000","last_active":"2024-05-28T14:16:49.553138+0000","last_peered":"2024-05-28T14:16:49.553138+0000","last_clean":"2024-05-28T14:16:49.553138+0000","last_became_active":"2024-05-28T14:15:05.975545+0000","last_became_peered":"2024-05-28T14:15:05.975545+0000","last_unstale":"2024-05-28T14:16:49.553138+0000","last_undegraded":"2024-05-28T14:16:49.553138+0000","last_fullsized":"2024-05-28T14:16:49.553138+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:56:51.599701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":85,"num_read_kb":348160,"num_write":107,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"134'41","reported_seq":59,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:25.782834+0000","last_change":"2024-05-28T14:14:42.749733+0000","last_active":"2024-05-28T14:16:25.782834+0000","last_peered":"2024-05-28T14:16:25.782834+0000","last_clean":"2024-05-28T14:16:25.782834+0000","last_became_active":"2024-05-28T14:14:42.749348+0000","last_became_peered":"2024-05-28T14:14:42.749348+0000","last_unstale":"2024-05-28T14:16:25.782834+0000","last_undegraded":"2024-05-28T14:16:25.782834+0000","last_fullsized":"2024-05-28T14:16:25.782834+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:17.145363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"134'93","reported_seq":210,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.554090+0000","last_change":"2024-05-28T14:14:43.764008+0000","last_active":"2024-05-28T14:16:49.554090+0000","last_peered":"2024-05-28T14:16:49.554090+0000","last_clean":"2024-05-28T14:16:49.554090+0000","last_became_active":"2024-05-28T14:14:43.762648+0000","last_became_peered":"2024-05-28T14:14:43.762648+0000","last_unstale":"2024-05-28T14:16:49.554090+0000","last_undegraded":"2024-05-28T14:16:49.554090+0000","last_fullsized":"2024-05-28T14:16:49.554090+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:35:31.221160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":86,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077916+0000","last_change":"2024-05-28T14:14:42.744189+0000","last_active":"2024-05-28T14:15:06.077916+0000","last_peered":"2024-05-28T14:15:06.077916+0000","last_clean":"2024-05-28T14:15:06.077916+0000","last_became_active":"2024-05-28T14:14:42.743858+0000","last_became_peered":"2024-05-28T14:14:42.743858+0000","last_unstale":"2024-05-28T14:15:06.077916+0000","last_undegraded":"2024-05-28T14:15:06.077916+0000","last_fullsized":"2024-05-28T14:15:06.077916+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:03:34.587886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"134'103","reported_seq":184,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571199+0000","last_change":"2024-05-28T14:15:06.036082+0000","last_active":"2024-05-28T14:16:49.571199+0000","last_peered":"2024-05-28T14:16:49.571199+0000","last_clean":"2024-05-28T14:16:49.571199+0000","last_became_active":"2024-05-28T14:15:05.976669+0000","last_became_peered":"2024-05-28T14:15:05.976669+0000","last_unstale":"2024-05-28T14:16:49.571199+0000","last_undegraded":"2024-05-28T14:16:49.571199+0000","last_fullsized":"2024-05-28T14:16:49.571199+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:57:19.311096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132541,"num_objects":50,"num_object_clones":1,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":1,"num_read":52,"num_read_kb":208897,"num_write":94,"num_write_kb":282626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8388730,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971624+0000","last_change":"2024-05-28T14:14:42.748521+0000","last_active":"2024-05-28T14:15:05.971624+0000","last_peered":"2024-05-28T14:15:05.971624+0000","last_clean":"2024-05-28T14:15:05.971624+0000","last_became_active":"2024-05-28T14:14:42.748193+0000","last_became_peered":"2024-05-28T14:14:42.748193+0000","last_unstale":"2024-05-28T14:15:05.971624+0000","last_undegraded":"2024-05-28T14:15:05.971624+0000","last_fullsized":"2024-05-28T14:15:05.971624+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:48:29.637714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"134'128","reported_seq":194,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.556050+0000","last_change":"2024-05-28T14:14:43.767848+0000","last_active":"2024-05-28T14:16:27.556050+0000","last_peered":"2024-05-28T14:16:27.556050+0000","last_clean":"2024-05-28T14:16:27.556050+0000","last_became_active":"2024-05-28T14:14:43.767552+0000","last_became_peered":"2024-05-28T14:14:43.767552+0000","last_unstale":"2024-05-28T14:16:27.556050+0000","last_undegraded":"2024-05-28T14:16:27.556050+0000","last_fullsized":"2024-05-28T14:16:27.556050+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:09:18.039029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":285212672,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":116,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077762+0000","last_change":"2024-05-28T14:14:42.746605+0000","last_active":"2024-05-28T14:15:06.077762+0000","last_peered":"2024-05-28T14:15:06.077762+0000","last_clean":"2024-05-28T14:15:06.077762+0000","last_became_active":"2024-05-28T14:14:42.746338+0000","last_became_peered":"2024-05-28T14:14:42.746338+0000","last_unstale":"2024-05-28T14:15:06.077762+0000","last_undegraded":"2024-05-28T14:15:06.077762+0000","last_fullsized":"2024-05-28T14:15:06.077762+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:51:27.080888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"134'137","reported_seq":238,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.537160+0000","last_change":"2024-05-28T14:14:43.760455+0000","last_active":"2024-05-28T14:16:49.537160+0000","last_peered":"2024-05-28T14:16:49.537160+0000","last_clean":"2024-05-28T14:16:49.537160+0000","last_became_active":"2024-05-28T14:14:43.760063+0000","last_became_peered":"2024-05-28T14:14:43.760063+0000","last_unstale":"2024-05-28T14:16:49.537160+0000","last_undegraded":"2024-05-28T14:16:49.537160+0000","last_fullsized":"2024-05-28T14:16:49.537160+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:50:53.753974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272629760,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":90,"num_read_kb":356355,"num_write":129,"num_write_kb":393217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973173+0000","last_change":"2024-05-28T14:14:42.749618+0000","last_active":"2024-05-28T14:15:05.973173+0000","last_peered":"2024-05-28T14:15:05.973173+0000","last_clean":"2024-05-28T14:15:05.973173+0000","last_became_active":"2024-05-28T14:14:42.749259+0000","last_became_peered":"2024-05-28T14:14:42.749259+0000","last_unstale":"2024-05-28T14:15:05.973173+0000","last_undegraded":"2024-05-28T14:15:05.973173+0000","last_fullsized":"2024-05-28T14:15:05.973173+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:44:02.935848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"134'119","reported_seq":215,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.564680+0000","last_change":"2024-05-28T14:14:43.761804+0000","last_active":"2024-05-28T14:16:49.564680+0000","last_peered":"2024-05-28T14:16:49.564680+0000","last_clean":"2024-05-28T14:16:49.564680+0000","last_became_active":"2024-05-28T14:14:43.761382+0000","last_became_peered":"2024-05-28T14:14:43.761382+0000","last_unstale":"2024-05-28T14:16:49.564680+0000","last_undegraded":"2024-05-28T14:16:49.564680+0000","last_fullsized":"2024-05-28T14:16:49.564680+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:48:50.641484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":114,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"134'31","reported_seq":80,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:26.863095+0000","last_change":"2024-05-28T14:14:42.739854+0000","last_active":"2024-05-28T14:16:26.863095+0000","last_peered":"2024-05-28T14:16:26.863095+0000","last_clean":"2024-05-28T14:16:26.863095+0000","last_became_active":"2024-05-28T14:14:42.738369+0000","last_became_peered":"2024-05-28T14:14:42.738369+0000","last_unstale":"2024-05-28T14:16:26.863095+0000","last_undegraded":"2024-05-28T14:16:26.863095+0000","last_fullsized":"2024-05-28T14:16:26.863095+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:34:08.611902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":31,"num_write":30,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"134'123","reported_seq":223,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.554421+0000","last_change":"2024-05-28T14:14:43.765205+0000","last_active":"2024-05-28T14:16:49.554421+0000","last_peered":"2024-05-28T14:16:49.554421+0000","last_clean":"2024-05-28T14:16:49.554421+0000","last_became_active":"2024-05-28T14:14:43.764879+0000","last_became_peered":"2024-05-28T14:14:43.764879+0000","last_unstale":"2024-05-28T14:16:49.554421+0000","last_undegraded":"2024-05-28T14:16:49.554421+0000","last_fullsized":"2024-05-28T14:16:49.554421+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:07.658669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":116,"num_write_kb":372736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971518+0000","last_change":"2024-05-28T14:14:42.730611+0000","last_active":"2024-05-28T14:15:05.971518+0000","last_peered":"2024-05-28T14:15:05.971518+0000","last_clean":"2024-05-28T14:15:05.971518+0000","last_became_active":"2024-05-28T14:14:42.730333+0000","last_became_peered":"2024-05-28T14:14:42.730333+0000","last_unstale":"2024-05-28T14:15:05.971518+0000","last_undegraded":"2024-05-28T14:15:05.971518+0000","last_fullsized":"2024-05-28T14:15:05.971518+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:05:55.551707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"134'106","reported_seq":169,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.543332+0000","last_change":"2024-05-28T14:14:43.761743+0000","last_active":"2024-05-28T14:16:49.543332+0000","last_peered":"2024-05-28T14:16:49.543332+0000","last_clean":"2024-05-28T14:16:49.543332+0000","last_became_active":"2024-05-28T14:14:43.760828+0000","last_became_peered":"2024-05-28T14:14:43.760828+0000","last_unstale":"2024-05-28T14:16:49.543332+0000","last_undegraded":"2024-05-28T14:16:49.543332+0000","last_fullsized":"2024-05-28T14:16:49.543332+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:44:21.055874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":101,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"131'2","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973324+0000","last_change":"2024-05-28T14:14:42.751868+0000","last_active":"2024-05-28T14:15:05.973324+0000","last_peered":"2024-05-28T14:15:05.973324+0000","last_clean":"2024-05-28T14:15:05.973324+0000","last_became_active":"2024-05-28T14:14:42.751447+0000","last_became_peered":"2024-05-28T14:14:42.751447+0000","last_unstale":"2024-05-28T14:15:05.973324+0000","last_undegraded":"2024-05-28T14:15:05.973324+0000","last_fullsized":"2024-05-28T14:15:05.973324+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:28:17.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":383,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.972861+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:15:05.972861+0000","last_peered":"2024-05-28T14:15:05.972861+0000","last_clean":"2024-05-28T14:15:05.972861+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:15:05.972861+0000","last_undegraded":"2024-05-28T14:15:05.972861+0000","last_fullsized":"2024-05-28T14:15:05.972861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":15082718397,"num_objects":3603,"num_object_clones":1,"num_object_copies":7206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":1,"num_read":4365,"num_read_kb":17629245,"num_write":6581,"num_write_kb":20676651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":50331770,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":30165483520,"data_stored":30165436794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7068,"ondisk_log_size":7068,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1787293,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":30,"num_read_kb":31,"num_write":638,"num_write_kb":1990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3637248,"data_stored":3574586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":590,"ondisk_log_size":590,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575074,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3272720,"kb_used_data":3133320,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":91099120,"statfs":{"total":96636764160,"available":93285498880,"internally_reserved":0,"allocated":3208519680,"data_stored":3207557922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":142735818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575074,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4222476,"kb_used_data":4078916,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90149364,"statfs":{"total":96636764160,"available":92312948736,"internally_reserved":0,"allocated":4176809984,"data_stored":4175856883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575076,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3121336,"kb_used_data":2995824,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91250504,"statfs":{"total":96636764160,"available":93440516096,"internally_reserved":0,"allocated":3067723776,"data_stored":3066774417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":128514502},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575075,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4572916,"kb_used_data":4433516,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":89798924,"statfs":{"total":96636764160,"available":91954098176,"internally_reserved":0,"allocated":4539920384,"data_stored":4538974892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575074,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3790716,"kb_used_data":3642996,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":90581124,"statfs":{"total":96636764160,"available":92755070976,"internally_reserved":0,"allocated":3730427904,"data_stored":3729479880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":151255496},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575075,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4906484,"kb_used_data":4740716,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":89465356,"statfs":{"total":96636764160,"available":91612524544,"internally_reserved":0,"allocated":4854493184,"data_stored":4853548245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575074,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3483436,"kb_used_data":3339876,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90888404,"statfs":{"total":96636764160,"available":93069725696,"internally_reserved":0,"allocated":3420033024,"data_stored":3419095661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575075,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3250484,"kb_used_data":3111084,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":91121356,"statfs":{"total":96636764160,"available":93308268544,"internally_reserved":0,"allocated":3185750016,"data_stored":3184804998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,9,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1798144,"data_stored":1786495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1802240,"data_stored":1781570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3183484928,"data_stored":3183476989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3418361856,"data_stored":3418358020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4852813824,"data_stored":4852809988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3728740352,"data_stored":3728736516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4538245120,"data_stored":4538237181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3066044416,"data_stored":3066036664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4173340672,"data_stored":4173332920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3204452352,"data_stored":3204448516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:16:51.975 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-28T14:16:52.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe369a00640 1 Processor -- start 2024-05-28T14:16:52.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe369a00640 1 -- start start 2024-05-28T14:16:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe369a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe36415ec80 0x7fe3640704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe369a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe3640b3a50 0x7fe3640cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe369a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3640cf080 con 0x7fe36406b9f0 2024-05-28T14:16:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe369a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe364070f20 con 0x7fe3640b3a50 2024-05-28T14:16:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe369a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe364070c40 con 0x7fe36415ec80 2024-05-28T14:16:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe363400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe36406b9f0 0x7fe3640ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53050/0 (socket says 172.21.15.42:53050) 2024-05-28T14:16:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe363400640 1 -- 172.21.15.42:0/3621494116 learned_addr learned my addr 172.21.15.42:0/3621494116 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:52.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362a00640 1 --2- 172.21.15.42:0/3621494116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe36415ec80 0x7fe3640704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:52.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe35a000640 1 --2- 172.21.15.42:0/3621494116 >> v2:172.21.15.63:3300/0 conn(0x7fe3640b3a50 0x7fe3640cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:52.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 -- 172.21.15.42:0/3621494116 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2024575250 0 0) 0x7fe3640cf080 con 0x7fe36406b9f0 2024-05-28T14:16:52.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 -- 172.21.15.42:0/3621494116 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe348003530 con 0x7fe36406b9f0 2024-05-28T14:16:52.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 -- 172.21.15.42:0/3621494116 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4163878211 0 0) 0x7fe348003530 con 0x7fe36406b9f0 2024-05-28T14:16:52.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 -- 172.21.15.42:0/3621494116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe36415ec80 msgr2=0x7fe3640704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:52.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 --2- 172.21.15.42:0/3621494116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe36415ec80 0x7fe3640704f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:52.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 -- 172.21.15.42:0/3621494116 >> v2:172.21.15.63:3300/0 conn(0x7fe3640b3a50 msgr2=0x7fe3640cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:52.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 --2- 172.21.15.42:0/3621494116 >> v2:172.21.15.63:3300/0 conn(0x7fe3640b3a50 0x7fe3640cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:52.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 -- 172.21.15.42:0/3621494116 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3640cd6c0 con 0x7fe36406b9f0 2024-05-28T14:16:52.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 -- 172.21.15.42:0/3621494116 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7fe35c001a00 con 0x7fe36406b9f0 2024-05-28T14:16:52.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 -- 172.21.15.42:0/3621494116 >> v1:172.21.15.42:6789/0 conn(0x7fe36406b9f0 legacy=0x7fe3640ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:52.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 --2- 172.21.15.42:0/3621494116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3480039c0 0x7fe348003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:52.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 --2- 172.21.15.42:0/3621494116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe348004810 0x7fe348004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:52.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 --2- 172.21.15.42:0/3621494116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe36415ec80 0x7fe3480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:52.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 -- 172.21.15.42:0/3621494116 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe364070f20 con 0x7fe3480039c0 2024-05-28T14:16:52.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 -- 172.21.15.42:0/3621494116 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe364070c40 con 0x7fe348004810 2024-05-28T14:16:52.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362000640 1 -- 172.21.15.42:0/3621494116 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe34800b9d0 con 0x7fe36415ec80 2024-05-28T14:16:52.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362a00640 1 --2- 172.21.15.42:0/3621494116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3480039c0 0x7fe348003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:52.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe363400640 1 --2- 172.21.15.42:0/3621494116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe36415ec80 0x7fe3480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:52.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe35a000640 1 --2- 172.21.15.42:0/3621494116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe348004810 0x7fe348004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:52.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362a00640 1 -- 172.21.15.42:0/3621494116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe36415ec80 msgr2=0x7fe3480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:52.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362a00640 1 --2- 172.21.15.42:0/3621494116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe36415ec80 0x7fe3480051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:52.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362a00640 1 -- 172.21.15.42:0/3621494116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe348004810 msgr2=0x7fe348004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:52.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362a00640 1 --2- 172.21.15.42:0/3621494116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe348004810 0x7fe348004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:52.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe362a00640 1 -- 172.21.15.42:0/3621494116 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe348003530 con 0x7fe3480039c0 2024-05-28T14:16:52.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe363400640 1 --2- 172.21.15.42:0/3621494116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe36415ec80 0x7fe3480051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:52.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.169+0000 7fe35a000640 1 --2- 172.21.15.42:0/3621494116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe348004810 0x7fe348004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:52.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.173+0000 7fe362a00640 1 --2- 172.21.15.42:0/3621494116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3480039c0 0x7fe348003de0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fe354009ae0 tx=0x7fe35401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa57470086939a91 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:52.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.173+0000 7fe362000640 1 -- 172.21.15.42:0/3621494116 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3540096b0 con 0x7fe3480039c0 2024-05-28T14:16:52.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.173+0000 7fe362000640 1 -- 172.21.15.42:0/3621494116 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3540a6450 con 0x7fe3480039c0 2024-05-28T14:16:52.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.173+0000 7fe362000640 1 -- 172.21.15.42:0/3621494116 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3540a6840 con 0x7fe3480039c0 2024-05-28T14:16:52.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.173+0000 7fe369a00640 1 -- 172.21.15.42:0/3621494116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3480039c0 msgr2=0x7fe348003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:52.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.173+0000 7fe369a00640 1 --2- 172.21.15.42:0/3621494116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3480039c0 0x7fe348003de0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fe354009ae0 tx=0x7fe35401ca40 comp rx=0 tx=0).stop 2024-05-28T14:16:52.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.173+0000 7fe363400640 1 -- 172.21.15.42:0/3621494116 reap_dead start 2024-05-28T14:16:52.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.173+0000 7fe369a00640 1 -- 172.21.15.42:0/3621494116 shutdown_connections 2024-05-28T14:16:52.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.173+0000 7fe369a00640 1 -- 172.21.15.42:0/3621494116 >> 172.21.15.42:0/3621494116 conn(0x7fe3640b2f60 msgr2=0x7fe3640b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:52.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.173+0000 7fe369a00640 1 -- 172.21.15.42:0/3621494116 shutdown_connections 2024-05-28T14:16:52.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.173+0000 7fe369a00640 1 -- 172.21.15.42:0/3621494116 wait complete. 2024-05-28T14:16:52.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe369a00640 1 Processor -- start 2024-05-28T14:16:52.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe369a00640 1 -- start start 2024-05-28T14:16:52.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe369a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3480039c0 0x7fe3641fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:52.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe369a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3640d15f0 0x7fe3641586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:52.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe369a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3640d1a70 0x7fe364158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:52.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe369a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe3640cbf20 con 0x7fe3480039c0 2024-05-28T14:16:52.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe369a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe36406be00 con 0x7fe3640d1a70 2024-05-28T14:16:52.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe369a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe3641fc630 con 0x7fe3640d15f0 2024-05-28T14:16:52.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe362a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3480039c0 0x7fe3641fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:52.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe362a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3480039c0 0x7fe3641fc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37960/0 (socket says 172.21.15.42:37960) 2024-05-28T14:16:52.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe362a00640 1 -- 172.21.15.42:0/2845793258 learned_addr learned my addr 172.21.15.42:0/2845793258 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:52.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe35a000640 1 --2- 172.21.15.42:0/2845793258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3640d15f0 0x7fe3641586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:52.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe363400640 1 --2- 172.21.15.42:0/2845793258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3640d1a70 0x7fe364158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:52.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe362a00640 1 -- 172.21.15.42:0/2845793258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3640d15f0 msgr2=0x7fe3641586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:52.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe362a00640 1 --2- 172.21.15.42:0/2845793258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3640d15f0 0x7fe3641586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:52.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe362a00640 1 -- 172.21.15.42:0/2845793258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3640d1a70 msgr2=0x7fe364158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:52.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe362a00640 1 --2- 172.21.15.42:0/2845793258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3640d1a70 0x7fe364158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:52.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe362a00640 1 -- 172.21.15.42:0/2845793258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3640b17f0 con 0x7fe3480039c0 2024-05-28T14:16:52.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe362a00640 1 --2- 172.21.15.42:0/2845793258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3480039c0 0x7fe3641fc0f0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fe354098260 tx=0x7fe348005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:52.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe35b400640 1 -- 172.21.15.42:0/2845793258 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3540a6e20 con 0x7fe3480039c0 2024-05-28T14:16:52.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe35b400640 1 -- 172.21.15.42:0/2845793258 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe35400f000 con 0x7fe3480039c0 2024-05-28T14:16:52.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe369a00640 1 -- 172.21.15.42:0/2845793258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe364160630 con 0x7fe3480039c0 2024-05-28T14:16:52.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe35b400640 1 -- 172.21.15.42:0/2845793258 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354004230 con 0x7fe3480039c0 2024-05-28T14:16:52.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.177+0000 7fe369a00640 1 -- 172.21.15.42:0/2845793258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe364160bf0 con 0x7fe3480039c0 2024-05-28T14:16:52.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.181+0000 7fe369a00640 1 -- 172.21.15.42:0/2845793258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe364068a90 con 0x7fe3480039c0 2024-05-28T14:16:52.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.181+0000 7fe35b400640 1 -- 172.21.15.42:0/2845793258 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7fe354003620 con 0x7fe3480039c0 2024-05-28T14:16:52.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.181+0000 7fe35b400640 1 --2- 172.21.15.42:0/2845793258 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe34006a420 0x7fe34006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:52.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.181+0000 7fe35b400640 1 -- 172.21.15.42:0/2845793258 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fe35411f980 con 0x7fe3480039c0 2024-05-28T14:16:52.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.181+0000 7fe35a000640 1 --2- 172.21.15.42:0/2845793258 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe34006a420 0x7fe34006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:52.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.181+0000 7fe35a000640 1 --2- 172.21.15.42:0/2845793258 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe34006a420 0x7fe34006c8e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe3640cbae0 tx=0x7fe34c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:52.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.185+0000 7fe35b400640 1 -- 172.21.15.42:0/2845793258 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe3540ac020 con 0x7fe3480039c0 2024-05-28T14:16:52.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.357+0000 7fe369a00640 1 -- 172.21.15.42:0/2845793258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3640ce1a0 con 0x7fe3480039c0 2024-05-28T14:16:52.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.357+0000 7fe35b400640 1 -- 172.21.15.42:0/2845793258 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16747 (secure 0 0 0) 0x7fe354016780 con 0x7fe3480039c0 2024-05-28T14:16:52.343 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:52.343 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":134,"fsid":"7380b1c2-773d-4bd7-abac-066c3536cd24","created":"2024-05-28T13:57:57.667357+0000","modified":"2024-05-28T14:15:05.956598+0000","last_up_change":"2024-05-28T13:58:09.207514+0000","last_in_change":"2024-05-28T13:58:00.239616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-28T13:58:10.881461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-28T14:14:41.538199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-28T14:14:42.086660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7837a445-7a48-4c2e-afb3-bf233faa91e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6809","nonce":2018240931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6811","nonce":2018240931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6815","nonce":2018240931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2018240931},{"type":"v1","addr":"172.21.15.42:6813","nonce":2018240931}]},"public_addr":"172.21.15.42:6809/2018240931","cluster_addr":"172.21.15.42:6811/2018240931","heartbeat_back_addr":"172.21.15.42:6815/2018240931","heartbeat_front_addr":"172.21.15.42:6813/2018240931","state":["exists","up"]},{"osd":1,"uuid":"1b3b7367-edb9-4b74-80e1-c244b4eaf23a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6801","nonce":437002159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6803","nonce":437002159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6807","nonce":437002159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":437002159},{"type":"v1","addr":"172.21.15.42:6805","nonce":437002159}]},"public_addr":"172.21.15.42:6801/437002159","cluster_addr":"172.21.15.42:6803/437002159","heartbeat_back_addr":"172.21.15.42:6807/437002159","heartbeat_front_addr":"172.21.15.42:6805/437002159","state":["exists","up"]},{"osd":2,"uuid":"9f6f77fe-d1e3-4bd3-bfc1-2d0d15cd53d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6817","nonce":254756381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6819","nonce":254756381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6823","nonce":254756381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":254756381},{"type":"v1","addr":"172.21.15.42:6821","nonce":254756381}]},"public_addr":"172.21.15.42:6817/254756381","cluster_addr":"172.21.15.42:6819/254756381","heartbeat_back_addr":"172.21.15.42:6823/254756381","heartbeat_front_addr":"172.21.15.42:6821/254756381","state":["exists","up"]},{"osd":3,"uuid":"54bda6c4-a5ca-4391-8299-b1b05d859110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6825","nonce":2252246131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6827","nonce":2252246131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6831","nonce":2252246131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2252246131},{"type":"v1","addr":"172.21.15.42:6829","nonce":2252246131}]},"public_addr":"172.21.15.42:6825/2252246131","cluster_addr":"172.21.15.42:6827/2252246131","heartbeat_back_addr":"172.21.15.42:6831/2252246131","heartbeat_front_addr":"172.21.15.42:6829/2252246131","state":["exists","up"]},{"osd":4,"uuid":"b0cee18a-b7fa-4dc2-95d2-1d55f92a4b62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6801","nonce":4268829337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6803","nonce":4268829337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6807","nonce":4268829337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":4268829337},{"type":"v1","addr":"172.21.15.63:6805","nonce":4268829337}]},"public_addr":"172.21.15.63:6801/4268829337","cluster_addr":"172.21.15.63:6803/4268829337","heartbeat_back_addr":"172.21.15.63:6807/4268829337","heartbeat_front_addr":"172.21.15.63:6805/4268829337","state":["exists","up"]},{"osd":5,"uuid":"3e53bcb8-bda8-4013-90db-730b694cf40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6809","nonce":1802936259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6811","nonce":1802936259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6815","nonce":1802936259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1802936259},{"type":"v1","addr":"172.21.15.63:6813","nonce":1802936259}]},"public_addr":"172.21.15.63:6809/1802936259","cluster_addr":"172.21.15.63:6811/1802936259","heartbeat_back_addr":"172.21.15.63:6815/1802936259","heartbeat_front_addr":"172.21.15.63:6813/1802936259","state":["exists","up"]},{"osd":6,"uuid":"9ba8ecb4-2658-4657-b6fb-569565983fb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6825","nonce":1958395718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6827","nonce":1958395718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6831","nonce":1958395718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1958395718},{"type":"v1","addr":"172.21.15.63:6829","nonce":1958395718}]},"public_addr":"172.21.15.63:6825/1958395718","cluster_addr":"172.21.15.63:6827/1958395718","heartbeat_back_addr":"172.21.15.63:6831/1958395718","heartbeat_front_addr":"172.21.15.63:6829/1958395718","state":["exists","up"]},{"osd":7,"uuid":"91586b96-af01-4370-b64c-71ff4d46a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6817","nonce":3211394017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6819","nonce":3211394017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6823","nonce":3211394017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3211394017},{"type":"v1","addr":"172.21.15.63:6821","nonce":3211394017}]},"public_addr":"172.21.15.63:6817/3211394017","cluster_addr":"172.21.15.63:6819/3211394017","heartbeat_back_addr":"172.21.15.63:6823/3211394017","heartbeat_front_addr":"172.21.15.63:6821/3211394017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.002846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:06.125094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.967386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.914550+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.871636+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.665743+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.979903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-28T13:58:05.737294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.7","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-28T14:16:52.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.361+0000 7fe369a00640 1 -- 172.21.15.42:0/2845793258 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe34006a420 msgr2=0x7fe34006c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:52.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.361+0000 7fe369a00640 1 --2- 172.21.15.42:0/2845793258 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe34006a420 0x7fe34006c8e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe3640cbae0 tx=0x7fe34c008040 comp rx=0 tx=0).stop 2024-05-28T14:16:52.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.361+0000 7fe369a00640 1 -- 172.21.15.42:0/2845793258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3480039c0 msgr2=0x7fe3641fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:52.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.361+0000 7fe369a00640 1 --2- 172.21.15.42:0/2845793258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3480039c0 0x7fe3641fc0f0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fe354098260 tx=0x7fe348005840 comp rx=0 tx=0).stop 2024-05-28T14:16:52.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.361+0000 7fe369a00640 1 -- 172.21.15.42:0/2845793258 shutdown_connections 2024-05-28T14:16:52.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.361+0000 7fe369a00640 1 --2- 172.21.15.42:0/2845793258 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe34006a420 0x7fe34006c8e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:52.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.361+0000 7fe369a00640 1 --2- 172.21.15.42:0/2845793258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3640d1a70 0x7fe364158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:52.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.361+0000 7fe369a00640 1 --2- 172.21.15.42:0/2845793258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3640d15f0 0x7fe3641586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:52.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.361+0000 7fe369a00640 1 --2- 172.21.15.42:0/2845793258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3480039c0 0x7fe3641fc0f0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:52.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.361+0000 7fe369a00640 1 -- 172.21.15.42:0/2845793258 >> 172.21.15.42:0/2845793258 conn(0x7fe3640b2f60 msgr2=0x7fe3640b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:52.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.361+0000 7fe369a00640 1 -- 172.21.15.42:0/2845793258 shutdown_connections 2024-05-28T14:16:52.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:52.361+0000 7fe369a00640 1 -- 172.21.15.42:0/2845793258 wait complete. 2024-05-28T14:16:53.365 INFO:tasks.ceph:Scrubbing osd.0 2024-05-28T14:16:53.365 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-28T14:16:53.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.537+0000 7f98f3000640 1 Processor -- start 2024-05-28T14:16:53.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.541+0000 7f98f3000640 1 -- start start 2024-05-28T14:16:53.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.541+0000 7f98f3000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98ec15b350 0x7f98ec06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:53.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.541+0000 7f98f3000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98ec0ade40 con 0x7f98ec0d0e00 2024-05-28T14:16:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.541+0000 7f98f3000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98ec0ae0c0 con 0x7f98ec072d40 2024-05-28T14:16:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.541+0000 7f98f3000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f98ec0ae9f0 con 0x7f98ec15b350 2024-05-28T14:16:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.541+0000 7f98f1000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f98ec0d0e00 0x7f98ec0aee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53056/0 (socket says 172.21.15.42:53056) 2024-05-28T14:16:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.541+0000 7f98f1000640 1 -- 172.21.15.42:0/3538304040 learned_addr learned my addr 172.21.15.42:0/3538304040 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:53.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.541+0000 7f98ebe00640 1 --2- 172.21.15.42:0/3538304040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98ec15b350 0x7f98ec06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:53.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.541+0000 7f98eb400640 1 -- 172.21.15.42:0/3538304040 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3505239357 0 0) 0x7f98e4001120 con 0x7f98ec0d0e00 2024-05-28T14:16:53.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.541+0000 7f98eb400640 1 -- 172.21.15.42:0/3538304040 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98d0005740 con 0x7f98ec0d0e00 2024-05-28T14:16:53.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.541+0000 7f98ebe00640 1 -- 172.21.15.42:0/3538304040 >> v1:172.21.15.63:6789/0 conn(0x7f98ec072d40 legacy=0x7f98ec0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.541+0000 7f98eb400640 1 -- 172.21.15.42:0/3538304040 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1263819375 0 0) 0x7f98ec0ae0c0 con 0x7f98ec072d40 2024-05-28T14:16:53.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.541+0000 7f98ebe00640 1 -- 172.21.15.42:0/3538304040 >> v1:172.21.15.42:6789/0 conn(0x7f98ec0d0e00 legacy=0x7f98ec0aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.541+0000 7f98ebe00640 1 -- 172.21.15.42:0/3538304040 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec0ae3a0 con 0x7f98ec15b350 2024-05-28T14:16:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.545+0000 7f98ebe00640 1 --2- 172.21.15.42:0/3538304040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98ec15b350 0x7f98ec06dd40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0aeb70 tx=0x7f98dc01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf69e263a377c081 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.545+0000 7f98eb400640 1 -- 172.21.15.42:0/3538304040 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0b2070 con 0x7f98ec15b350 2024-05-28T14:16:53.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.545+0000 7f98eb400640 1 -- 172.21.15.42:0/3538304040 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98dc0a5740 con 0x7f98ec15b350 2024-05-28T14:16:53.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.545+0000 7f98eb400640 1 -- 172.21.15.42:0/3538304040 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0a5af0 con 0x7f98ec15b350 2024-05-28T14:16:53.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.545+0000 7f98f3000640 1 -- 172.21.15.42:0/3538304040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98ec15b350 msgr2=0x7f98ec06dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.545+0000 7f98f3000640 1 --2- 172.21.15.42:0/3538304040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98ec15b350 0x7f98ec06dd40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0aeb70 tx=0x7f98dc01c900 comp rx=0 tx=0).stop 2024-05-28T14:16:53.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.545+0000 7f98f3000640 1 -- 172.21.15.42:0/3538304040 shutdown_connections 2024-05-28T14:16:53.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.545+0000 7f98f3000640 1 --2- 172.21.15.42:0/3538304040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98ec15b350 0x7f98ec06dd40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:53.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.545+0000 7f98f3000640 1 -- 172.21.15.42:0/3538304040 >> 172.21.15.42:0/3538304040 conn(0x7f98ec06bdf0 msgr2=0x7f98ec06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:53.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.545+0000 7f98f3000640 1 -- 172.21.15.42:0/3538304040 shutdown_connections 2024-05-28T14:16:53.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.545+0000 7f98f3000640 1 -- 172.21.15.42:0/3538304040 wait complete. 2024-05-28T14:16:53.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.545+0000 7f98f3000640 1 Processor -- start 2024-05-28T14:16:53.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.545+0000 7f98f3000640 1 -- start start 2024-05-28T14:16:53.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98f3000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98ec072d40 0x7f98ec0b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:53.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98f3000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98ec0d0e00 0x7f98ec0995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:53.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98f3000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98ec15b350 0x7f98ec144df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:53.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98f3000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec0afb20 con 0x7f98ec072d40 2024-05-28T14:16:53.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98f3000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec0af9a0 con 0x7f98ec0d0e00 2024-05-28T14:16:53.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98f3000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f98ec0b3af0 con 0x7f98ec15b350 2024-05-28T14:16:53.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98ebe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98ec072d40 0x7f98ec0b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:53.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98ebe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98ec072d40 0x7f98ec0b0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37974/0 (socket says 172.21.15.42:37974) 2024-05-28T14:16:53.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98ebe00640 1 -- 172.21.15.42:0/4035015446 learned_addr learned my addr 172.21.15.42:0/4035015446 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:53.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98f1000640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98ec15b350 0x7f98ec144df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:53.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98e3400640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98ec0d0e00 0x7f98ec0995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98ebe00640 1 -- 172.21.15.42:0/4035015446 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98ec15b350 msgr2=0x7f98ec144df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98ebe00640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98ec15b350 0x7f98ec144df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98ebe00640 1 -- 172.21.15.42:0/4035015446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98ec0d0e00 msgr2=0x7f98ec0995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98ebe00640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98ec0d0e00 0x7f98ec0995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98ebe00640 1 -- 172.21.15.42:0/4035015446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec14b7e0 con 0x7f98ec072d40 2024-05-28T14:16:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98e3400640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98ec0d0e00 0x7f98ec0995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:53.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98f1000640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98ec15b350 0x7f98ec144df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:53.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98ebe00640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98ec072d40 0x7f98ec0b0210 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f98dc000f90 tx=0x7f98dc0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:53.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98e8c00640 1 -- 172.21.15.42:0/4035015446 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0bd020 con 0x7f98ec072d40 2024-05-28T14:16:53.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98f3000640 1 -- 172.21.15.42:0/4035015446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ec14bac0 con 0x7f98ec072d40 2024-05-28T14:16:53.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98e8c00640 1 -- 172.21.15.42:0/4035015446 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98dc0b8470 con 0x7f98ec072d40 2024-05-28T14:16:53.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98e8c00640 1 -- 172.21.15.42:0/4035015446 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0b2040 con 0x7f98ec072d40 2024-05-28T14:16:53.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98f3000640 1 -- 172.21.15.42:0/4035015446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ec14c080 con 0x7f98ec072d40 2024-05-28T14:16:53.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.549+0000 7f98f3000640 1 -- 172.21.15.42:0/4035015446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98ec068a90 con 0x7f98ec072d40 2024-05-28T14:16:53.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.553+0000 7f98e8c00640 1 -- 172.21.15.42:0/4035015446 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f98dc0c0020 con 0x7f98ec072d40 2024-05-28T14:16:53.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.553+0000 7f98e8c00640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98c806a420 0x7f98c806c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:53.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.553+0000 7f98e8c00640 1 -- 172.21.15.42:0/4035015446 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f98dc121310 con 0x7f98ec072d40 2024-05-28T14:16:53.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.553+0000 7f98e3400640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98c806a420 0x7f98c806c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:53.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.553+0000 7f98e8c00640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.42:6808/2018240931,v1:172.21.15.42:6809/2018240931] conn(0x7f98c8072740 0x7f98c8074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:53.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.553+0000 7f98e8c00640 1 -- 172.21.15.42:0/4035015446 --> [v2:172.21.15.42:6808/2018240931,v1:172.21.15.42:6809/2018240931] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98c8075290 con 0x7f98c8072740 2024-05-28T14:16:53.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.553+0000 7f98e8c00640 1 -- 172.21.15.42:0/4035015446 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f98dc122400 con 0x7f98ec072d40 2024-05-28T14:16:53.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.553+0000 7f98f1000640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.42:6808/2018240931,v1:172.21.15.42:6809/2018240931] conn(0x7f98c8072740 0x7f98c8074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:53.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.553+0000 7f98e3400640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98c806a420 0x7f98c806c8e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f98ec09a050 tx=0x7f98d4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:53.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.553+0000 7f98f1000640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.42:6808/2018240931,v1:172.21.15.42:6809/2018240931] conn(0x7f98c8072740 0x7f98c8074bc0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:53.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.553+0000 7f98e8c00640 1 -- 172.21.15.42:0/4035015446 <== osd.0 v2:172.21.15.42:6808/2018240931 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98c8075290 con 0x7f98c8072740 2024-05-28T14:16:53.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.569+0000 7f98f3000640 1 -- 172.21.15.42:0/4035015446 --> [v2:172.21.15.42:6808/2018240931,v1:172.21.15.42:6809/2018240931] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f98ec072b20 con 0x7f98c8072740 2024-05-28T14:16:53.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.569+0000 7f98e8c00640 1 -- 172.21.15.42:0/4035015446 <== osd.0 v2:172.21.15.42:6808/2018240931 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f98ec072b20 con 0x7f98c8072740 2024-05-28T14:16:53.553 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:16:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-28T14:16:53.554 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:16:53.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.569+0000 7f98f3000640 1 -- 172.21.15.42:0/4035015446 >> [v2:172.21.15.42:6808/2018240931,v1:172.21.15.42:6809/2018240931] conn(0x7f98c8072740 msgr2=0x7f98c8074bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.569+0000 7f98f3000640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.42:6808/2018240931,v1:172.21.15.42:6809/2018240931] conn(0x7f98c8072740 0x7f98c8074bc0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.569+0000 7f98f3000640 1 -- 172.21.15.42:0/4035015446 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98c806a420 msgr2=0x7f98c806c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.569+0000 7f98f3000640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f98c806a420 0x7f98c806c8e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f98ec09a050 tx=0x7f98d4009040 comp rx=0 tx=0).stop 2024-05-28T14:16:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.569+0000 7f98f3000640 1 -- 172.21.15.42:0/4035015446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98ec072d40 msgr2=0x7f98ec0b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.569+0000 7f98f3000640 1 --2- 172.21.15.42:0/4035015446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98ec072d40 0x7f98ec0b0210 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f98dc000f90 tx=0x7f98dc0196a0 comp rx=0 tx=0).stop 2024-05-28T14:16:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.569+0000 7f98f1000640 1 -- 172.21.15.42:0/4035015446 reap_dead start 2024-05-28T14:16:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.569+0000 7f98f3000640 1 -- 172.21.15.42:0/4035015446 shutdown_connections 2024-05-28T14:16:53.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.569+0000 7f98f3000640 1 -- 172.21.15.42:0/4035015446 >> 172.21.15.42:0/4035015446 conn(0x7f98ec06bdf0 msgr2=0x7f98ec1521f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:53.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.569+0000 7f98f3000640 1 -- 172.21.15.42:0/4035015446 shutdown_connections 2024-05-28T14:16:53.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.569+0000 7f98f3000640 1 -- 172.21.15.42:0/4035015446 wait complete. 2024-05-28T14:16:53.572 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-28T14:16:53.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb4800640 1 Processor -- start 2024-05-28T14:16:53.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb4800640 1 -- start start 2024-05-28T14:16:53.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb4800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facac06f030 0x7facac06db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:53.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb4800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facac06f480 con 0x7facac145920 2024-05-28T14:16:53.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb4800640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facac06e240 con 0x7facac0724d0 2024-05-28T14:16:53.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb4800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7facac06e520 con 0x7facac06f030 2024-05-28T14:16:53.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb1e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7facac145920 0x7facac071dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53058/0 (socket says 172.21.15.42:53058) 2024-05-28T14:16:53.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb1e00640 1 -- 172.21.15.42:0/2547670897 learned_addr learned my addr 172.21.15.42:0/2547670897 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:53.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb2800640 1 --2- 172.21.15.42:0/2547670897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facac06f030 0x7facac06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:53.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb0a00640 1 -- 172.21.15.42:0/2547670897 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2315180438 0 0) 0x7facac06f480 con 0x7facac145920 2024-05-28T14:16:53.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb0a00640 1 -- 172.21.15.42:0/2547670897 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac94003530 con 0x7facac145920 2024-05-28T14:16:53.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb0a00640 1 -- 172.21.15.42:0/2547670897 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 567793728 0 0) 0x7facac06e240 con 0x7facac0724d0 2024-05-28T14:16:53.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb0a00640 1 -- 172.21.15.42:0/2547670897 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facac06f480 con 0x7facac0724d0 2024-05-28T14:16:53.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb2800640 1 -- 172.21.15.42:0/2547670897 >> v1:172.21.15.63:6789/0 conn(0x7facac0724d0 legacy=0x7facac15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb2800640 1 -- 172.21.15.42:0/2547670897 >> v1:172.21.15.42:6789/0 conn(0x7facac145920 legacy=0x7facac071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.729+0000 7facb2800640 1 -- 172.21.15.42:0/2547670897 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac06e800 con 0x7facac06f030 2024-05-28T14:16:53.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.733+0000 7facb2800640 1 --2- 172.21.15.42:0/2547670897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facac06f030 0x7facac06db40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faca8009dc0 tx=0x7faca801c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34380a85c5965599 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:53.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.733+0000 7facb0a00640 1 -- 172.21.15.42:0/2547670897 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca80b2070 con 0x7facac06f030 2024-05-28T14:16:53.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.733+0000 7facb0a00640 1 -- 172.21.15.42:0/2547670897 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faca80a58d0 con 0x7facac06f030 2024-05-28T14:16:53.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.733+0000 7facb0a00640 1 -- 172.21.15.42:0/2547670897 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca80a5ca0 con 0x7facac06f030 2024-05-28T14:16:53.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.733+0000 7facb4800640 1 -- 172.21.15.42:0/2547670897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facac06f030 msgr2=0x7facac06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.733+0000 7facb4800640 1 --2- 172.21.15.42:0/2547670897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facac06f030 0x7facac06db40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faca8009dc0 tx=0x7faca801c900 comp rx=0 tx=0).stop 2024-05-28T14:16:53.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.733+0000 7facb4800640 1 -- 172.21.15.42:0/2547670897 shutdown_connections 2024-05-28T14:16:53.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.733+0000 7facb4800640 1 --2- 172.21.15.42:0/2547670897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facac06f030 0x7facac06db40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:53.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.733+0000 7facb4800640 1 -- 172.21.15.42:0/2547670897 >> 172.21.15.42:0/2547670897 conn(0x7facac06b9f0 msgr2=0x7facac06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:53.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.733+0000 7facb4800640 1 -- 172.21.15.42:0/2547670897 shutdown_connections 2024-05-28T14:16:53.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.733+0000 7facb4800640 1 -- 172.21.15.42:0/2547670897 wait complete. 2024-05-28T14:16:53.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb4800640 1 Processor -- start 2024-05-28T14:16:53.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb4800640 1 -- start start 2024-05-28T14:16:53.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb4800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facac06f030 0x7facac146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:53.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb4800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facac0724d0 0x7facac0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:53.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb4800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facac145920 0x7facac0ab630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:53.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb4800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7facac15e3d0 con 0x7facac06f030 2024-05-28T14:16:53.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb4800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7facac15e250 con 0x7facac145920 2024-05-28T14:16:53.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb4800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7facac16a260 con 0x7facac0724d0 2024-05-28T14:16:53.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facac06f030 0x7facac146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:53.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facac06f030 0x7facac146690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37988/0 (socket says 172.21.15.42:37988) 2024-05-28T14:16:53.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb1e00640 1 -- 172.21.15.42:0/1652775376 learned_addr learned my addr 172.21.15.42:0/1652775376 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:53.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb1400640 1 --2- 172.21.15.42:0/1652775376 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facac0724d0 0x7facac0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:53.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb2800640 1 --2- 172.21.15.42:0/1652775376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facac145920 0x7facac0ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:53.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb1e00640 1 -- 172.21.15.42:0/1652775376 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facac0724d0 msgr2=0x7facac0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb1e00640 1 --2- 172.21.15.42:0/1652775376 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facac0724d0 0x7facac0ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:53.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb1e00640 1 -- 172.21.15.42:0/1652775376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facac145920 msgr2=0x7facac0ab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb1e00640 1 --2- 172.21.15.42:0/1652775376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facac145920 0x7facac0ab630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:53.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb1e00640 1 -- 172.21.15.42:0/1652775376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac15fb20 con 0x7facac06f030 2024-05-28T14:16:53.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb1e00640 1 --2- 172.21.15.42:0/1652775376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facac06f030 0x7facac146690 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7faca000c500 tx=0x7faca000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:53.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7faca6000640 1 -- 172.21.15.42:0/1652775376 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0017070 con 0x7facac06f030 2024-05-28T14:16:53.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7faca6000640 1 -- 172.21.15.42:0/1652775376 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faca0005cf0 con 0x7facac06f030 2024-05-28T14:16:53.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7faca6000640 1 -- 172.21.15.42:0/1652775376 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0007330 con 0x7facac06f030 2024-05-28T14:16:53.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb4800640 1 -- 172.21.15.42:0/1652775376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facac0abfa0 con 0x7facac06f030 2024-05-28T14:16:53.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.737+0000 7facb4800640 1 -- 172.21.15.42:0/1652775376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facac0ac560 con 0x7facac06f030 2024-05-28T14:16:53.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.741+0000 7facb4800640 1 -- 172.21.15.42:0/1652775376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facac073890 con 0x7facac06f030 2024-05-28T14:16:53.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.741+0000 7faca6000640 1 -- 172.21.15.42:0/1652775376 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7faca001a070 con 0x7facac06f030 2024-05-28T14:16:53.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.741+0000 7faca6000640 1 --2- 172.21.15.42:0/1652775376 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac9006a420 0x7fac9006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:53.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.741+0000 7faca6000640 1 -- 172.21.15.42:0/1652775376 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7faca0084f60 con 0x7facac06f030 2024-05-28T14:16:53.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.741+0000 7facb1400640 1 --2- 172.21.15.42:0/1652775376 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac9006a420 0x7fac9006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:53.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.741+0000 7facb1400640 1 --2- 172.21.15.42:0/1652775376 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac9006a420 0x7fac9006c8e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7facac1497b0 tx=0x7fac9c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:53.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.745+0000 7faca6000640 1 -- 172.21.15.42:0/1652775376 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faca0056f80 con 0x7facac06f030 2024-05-28T14:16:53.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.909+0000 7facb4800640 1 -- 172.21.15.42:0/1652775376 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7facac0cec20 con 0x7fac9006a420 2024-05-28T14:16:53.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.909+0000 7faca6000640 1 -- 172.21.15.42:0/1652775376 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7facac0cec20 con 0x7fac9006a420 2024-05-28T14:16:53.895 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 0 to deep-scrub 2024-05-28T14:16:53.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.913+0000 7facb4800640 1 -- 172.21.15.42:0/1652775376 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac9006a420 msgr2=0x7fac9006c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.913+0000 7facb4800640 1 --2- 172.21.15.42:0/1652775376 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac9006a420 0x7fac9006c8e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7facac1497b0 tx=0x7fac9c009040 comp rx=0 tx=0).stop 2024-05-28T14:16:53.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.913+0000 7facb4800640 1 -- 172.21.15.42:0/1652775376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facac06f030 msgr2=0x7facac146690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:53.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.913+0000 7facb4800640 1 --2- 172.21.15.42:0/1652775376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facac06f030 0x7facac146690 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7faca000c500 tx=0x7faca000c9c0 comp rx=0 tx=0).stop 2024-05-28T14:16:53.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.913+0000 7facb4800640 1 -- 172.21.15.42:0/1652775376 shutdown_connections 2024-05-28T14:16:53.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.913+0000 7facb4800640 1 --2- 172.21.15.42:0/1652775376 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fac9006a420 0x7fac9006c8e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:53.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.913+0000 7facb4800640 1 --2- 172.21.15.42:0/1652775376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facac145920 0x7facac0ab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:53.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.913+0000 7facb4800640 1 --2- 172.21.15.42:0/1652775376 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facac0724d0 0x7facac0ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:53.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.913+0000 7facb4800640 1 --2- 172.21.15.42:0/1652775376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facac06f030 0x7facac146690 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:53.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.913+0000 7facb4800640 1 -- 172.21.15.42:0/1652775376 >> 172.21.15.42:0/1652775376 conn(0x7facac06b9f0 msgr2=0x7facac140470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:53.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.913+0000 7facb4800640 1 -- 172.21.15.42:0/1652775376 shutdown_connections 2024-05-28T14:16:53.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:53.913+0000 7facb4800640 1 -- 172.21.15.42:0/1652775376 wait complete. 2024-05-28T14:16:53.917 INFO:tasks.ceph:Scrubbing osd.1 2024-05-28T14:16:53.917 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-28T14:16:54.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.077+0000 7f6117e00640 1 Processor -- start 2024-05-28T14:16:54.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.077+0000 7f6117e00640 1 -- start start 2024-05-28T14:16:54.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.077+0000 7f6117e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f611006e740 0x7f61100cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.077+0000 7f6117e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f611006ff10 con 0x7f61100ce110 2024-05-28T14:16:54.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.077+0000 7f6117e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61100701b0 con 0x7f611009b160 2024-05-28T14:16:54.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.077+0000 7f6117e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f611006eb20 con 0x7f611006e740 2024-05-28T14:16:54.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f6114a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f61100ce110 0x7f611009aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53074/0 (socket says 172.21.15.42:53074) 2024-05-28T14:16:54.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f6114a00640 1 -- 172.21.15.42:0/1664116350 learned_addr learned my addr 172.21.15.42:0/1664116350 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:54.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f6115400640 1 --2- 172.21.15.42:0/1664116350 >> v2:172.21.15.63:3300/0 conn(0x7f611006e740 0x7f61100cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4087833336 0 0) 0x7f611006ff10 con 0x7f61100ce110 2024-05-28T14:16:54.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60f8003530 con 0x7f61100ce110 2024-05-28T14:16:54.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3060509316 0 0) 0x7f61100701b0 con 0x7f611009b160 2024-05-28T14:16:54.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f611006ff10 con 0x7f611009b160 2024-05-28T14:16:54.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2967758046 0 0) 0x7f60f8003530 con 0x7f61100ce110 2024-05-28T14:16:54.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 >> v2:172.21.15.63:3300/0 conn(0x7f611006e740 msgr2=0x7f61100cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:54.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 --2- 172.21.15.42:0/1664116350 >> v2:172.21.15.63:3300/0 conn(0x7f611006e740 0x7f61100cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 >> v1:172.21.15.63:6789/0 conn(0x7f611009b160 legacy=0x7f6110070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6110070490 con 0x7f61100ce110 2024-05-28T14:16:54.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f60fc001680 con 0x7f61100ce110 2024-05-28T14:16:54.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f6115400640 1 --2- 172.21.15.42:0/1664116350 >> v2:172.21.15.63:3300/0 conn(0x7f611006e740 0x7f61100cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:16:54.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 >> v1:172.21.15.42:6789/0 conn(0x7f61100ce110 legacy=0x7f611009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 --2- 172.21.15.42:0/1664116350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f8003ba0 0x7f60f8003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 --2- 172.21.15.42:0/1664116350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8004810 0x7f60f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 --2- 172.21.15.42:0/1664116350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f80051d0 0x7f60f800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f611006eb20 con 0x7f60f8004810 2024-05-28T14:16:54.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60f800c0a0 con 0x7f60f80051d0 2024-05-28T14:16:54.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60f800c2a0 con 0x7f60f8003ba0 2024-05-28T14:16:54.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f6114a00640 1 --2- 172.21.15.42:0/1664116350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8004810 0x7f60f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f6115400640 1 --2- 172.21.15.42:0/1664116350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f8003ba0 0x7f60f8003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f6115e00640 1 --2- 172.21.15.42:0/1664116350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f80051d0 0x7f60f800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f6114a00640 1 -- 172.21.15.42:0/1664116350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f8003ba0 msgr2=0x7f60f8003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:54.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f6114a00640 1 --2- 172.21.15.42:0/1664116350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60f8003ba0 0x7f60f8003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f6114a00640 1 -- 172.21.15.42:0/1664116350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f80051d0 msgr2=0x7f60f800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:54.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f6114a00640 1 --2- 172.21.15.42:0/1664116350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f80051d0 0x7f60f800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f6114a00640 1 -- 172.21.15.42:0/1664116350 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f8003530 con 0x7f60f8004810 2024-05-28T14:16:54.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.081+0000 7f6115e00640 1 -- 172.21.15.42:0/1664116350 reap_dead start 2024-05-28T14:16:54.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.085+0000 7f6114a00640 1 --2- 172.21.15.42:0/1664116350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8004810 0x7f60f8004c90 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f60fc007fc0 tx=0x7f60fc01cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed89f9a0c4a30faf server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:54.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.085+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc007c10 con 0x7f60f8004810 2024-05-28T14:16:54.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.085+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60fc0a9070 con 0x7f60f8004810 2024-05-28T14:16:54.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.085+0000 7f610fe00640 1 -- 172.21.15.42:0/1664116350 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc005d60 con 0x7f60f8004810 2024-05-28T14:16:54.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.085+0000 7f6117e00640 1 -- 172.21.15.42:0/1664116350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8004810 msgr2=0x7f60f8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.085+0000 7f6117e00640 1 --2- 172.21.15.42:0/1664116350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8004810 0x7f60f8004c90 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f60fc007fc0 tx=0x7f60fc01cb60 comp rx=0 tx=0).stop 2024-05-28T14:16:54.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.085+0000 7f6117e00640 1 -- 172.21.15.42:0/1664116350 shutdown_connections 2024-05-28T14:16:54.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.085+0000 7f6117e00640 1 --2- 172.21.15.42:0/1664116350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8004810 0x7f60f8004c90 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.085+0000 7f6117e00640 1 -- 172.21.15.42:0/1664116350 >> 172.21.15.42:0/1664116350 conn(0x7f61100b3fa0 msgr2=0x7f61100b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:54.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.085+0000 7f6117e00640 1 -- 172.21.15.42:0/1664116350 shutdown_connections 2024-05-28T14:16:54.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.085+0000 7f6117e00640 1 -- 172.21.15.42:0/1664116350 wait complete. 2024-05-28T14:16:54.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.085+0000 7f6117e00640 1 Processor -- start 2024-05-28T14:16:54.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.085+0000 7f6117e00640 1 -- start start 2024-05-28T14:16:54.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6117e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8004810 0x7f61100b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6117e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61100ca8c0 0x7f61100b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6117e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61100cad20 0x7f61100b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6117e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f611009baa0 con 0x7f60f8004810 2024-05-28T14:16:54.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6117e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6110070a50 con 0x7f61100cad20 2024-05-28T14:16:54.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6117e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f611006c940 con 0x7f61100ca8c0 2024-05-28T14:16:54.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6115400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8004810 0x7f61100b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6115400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8004810 0x7f61100b50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37998/0 (socket says 172.21.15.42:37998) 2024-05-28T14:16:54.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6115400640 1 -- 172.21.15.42:0/2564632606 learned_addr learned my addr 172.21.15.42:0/2564632606 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:54.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6114a00640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61100ca8c0 0x7f61100b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6115e00640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61100cad20 0x7f61100b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6115400640 1 -- 172.21.15.42:0/2564632606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61100ca8c0 msgr2=0x7f61100b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6115400640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61100ca8c0 0x7f61100b6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6115400640 1 -- 172.21.15.42:0/2564632606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61100cad20 msgr2=0x7f61100b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6115400640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61100cad20 0x7f61100b7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6115400640 1 -- 172.21.15.42:0/2564632606 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61100b57b0 con 0x7f60f8004810 2024-05-28T14:16:54.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6115e00640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61100cad20 0x7f61100b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:54.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6115400640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8004810 0x7f61100b50d0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f6104011c00 tx=0x7f610400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:54.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f610d600640 1 -- 172.21.15.42:0/2564632606 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104011cf0 con 0x7f60f8004810 2024-05-28T14:16:54.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f610d600640 1 -- 172.21.15.42:0/2564632606 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6104011e90 con 0x7f60f8004810 2024-05-28T14:16:54.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f610d600640 1 -- 172.21.15.42:0/2564632606 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104004270 con 0x7f60f8004810 2024-05-28T14:16:54.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6117e00640 1 -- 172.21.15.42:0/2564632606 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61100b5a90 con 0x7f60f8004810 2024-05-28T14:16:54.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6117e00640 1 -- 172.21.15.42:0/2564632606 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61100b6050 con 0x7f60f8004810 2024-05-28T14:16:54.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.089+0000 7f6117e00640 1 -- 172.21.15.42:0/2564632606 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6110068a90 con 0x7f60f8004810 2024-05-28T14:16:54.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.093+0000 7f610d600640 1 -- 172.21.15.42:0/2564632606 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f6104012040 con 0x7f60f8004810 2024-05-28T14:16:54.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.093+0000 7f610d600640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60f006a470 0x7f60f006c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.093+0000 7f610d600640 1 -- 172.21.15.42:0/2564632606 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f6104086360 con 0x7f60f8004810 2024-05-28T14:16:54.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.093+0000 7f6114a00640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60f006a470 0x7f60f006c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.093+0000 7f610d600640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.42:6800/437002159,v1:172.21.15.42:6801/437002159] conn(0x7f60f0072870 0x7f60f0074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.093+0000 7f610d600640 1 -- 172.21.15.42:0/2564632606 --> [v2:172.21.15.42:6800/437002159,v1:172.21.15.42:6801/437002159] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6104006800 con 0x7f60f0072870 2024-05-28T14:16:54.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.093+0000 7f610d600640 1 -- 172.21.15.42:0/2564632606 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f6104087550 con 0x7f60f8004810 2024-05-28T14:16:54.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.093+0000 7f6115e00640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.42:6800/437002159,v1:172.21.15.42:6801/437002159] conn(0x7f60f0072870 0x7f60f0074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.093+0000 7f6114a00640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60f006a470 0x7f60f006c930 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0ab000 tx=0x7f60fc0a7070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:54.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.093+0000 7f6115e00640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.42:6800/437002159,v1:172.21.15.42:6801/437002159] conn(0x7f60f0072870 0x7f60f0074cf0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:54.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.093+0000 7f610d600640 1 -- 172.21.15.42:0/2564632606 <== osd.1 v2:172.21.15.42:6800/437002159 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6104006800 con 0x7f60f0072870 2024-05-28T14:16:54.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.109+0000 7f6117e00640 1 -- 172.21.15.42:0/2564632606 --> [v2:172.21.15.42:6800/437002159,v1:172.21.15.42:6801/437002159] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f611015fbb0 con 0x7f60f0072870 2024-05-28T14:16:54.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.109+0000 7f610d600640 1 -- 172.21.15.42:0/2564632606 <== osd.1 v2:172.21.15.42:6800/437002159 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f611015fbb0 con 0x7f60f0072870 2024-05-28T14:16:54.093 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:16:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-28T14:16:54.093 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:16:54.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.109+0000 7f6117e00640 1 -- 172.21.15.42:0/2564632606 >> [v2:172.21.15.42:6800/437002159,v1:172.21.15.42:6801/437002159] conn(0x7f60f0072870 msgr2=0x7f60f0074cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.109+0000 7f6117e00640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.42:6800/437002159,v1:172.21.15.42:6801/437002159] conn(0x7f60f0072870 0x7f60f0074cf0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.109+0000 7f6117e00640 1 -- 172.21.15.42:0/2564632606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60f006a470 msgr2=0x7f60f006c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.109+0000 7f6117e00640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f60f006a470 0x7f60f006c930 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0ab000 tx=0x7f60fc0a7070 comp rx=0 tx=0).stop 2024-05-28T14:16:54.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.109+0000 7f6117e00640 1 -- 172.21.15.42:0/2564632606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8004810 msgr2=0x7f61100b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.109+0000 7f6117e00640 1 --2- 172.21.15.42:0/2564632606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60f8004810 0x7f61100b50d0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f6104011c00 tx=0x7f610400f610 comp rx=0 tx=0).stop 2024-05-28T14:16:54.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.109+0000 7f6115e00640 1 -- 172.21.15.42:0/2564632606 reap_dead start 2024-05-28T14:16:54.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.109+0000 7f6117e00640 1 -- 172.21.15.42:0/2564632606 shutdown_connections 2024-05-28T14:16:54.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.109+0000 7f6117e00640 1 -- 172.21.15.42:0/2564632606 >> 172.21.15.42:0/2564632606 conn(0x7f61100b3fa0 msgr2=0x7f61100b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:54.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.109+0000 7f6117e00640 1 -- 172.21.15.42:0/2564632606 shutdown_connections 2024-05-28T14:16:54.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.109+0000 7f6117e00640 1 -- 172.21.15.42:0/2564632606 wait complete. 2024-05-28T14:16:54.111 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-28T14:16:54.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b3e00640 1 Processor -- start 2024-05-28T14:16:54.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b3e00640 1 -- start start 2024-05-28T14:16:54.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b3e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe9b40b5f30 0x7fe9b40b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b3e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe9b40ce140 0x7fe9b40ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b3e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b4156f40 con 0x7fe9b4156b60 2024-05-28T14:16:54.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b3e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fe9b40cd6a0 con 0x7fe9b40ce140 2024-05-28T14:16:54.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b3e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe9b406fd00 con 0x7fe9b40b5f30 2024-05-28T14:16:54.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b2a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fe9b4156b60 0x7fe9b415d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53088/0 (socket says 172.21.15.42:53088) 2024-05-28T14:16:54.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b2a00640 1 -- 172.21.15.42:0/206210190 learned_addr learned my addr 172.21.15.42:0/206210190 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:54.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b3400640 1 --2- 172.21.15.42:0/206210190 >> v2:172.21.15.42:3300/0 conn(0x7fe9b40ce140 0x7fe9b40ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9aa000640 1 --2- 172.21.15.42:0/206210190 >> v2:172.21.15.63:3300/0 conn(0x7fe9b40b5f30 0x7fe9b40b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b2000640 1 -- 172.21.15.42:0/206210190 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3811992865 0 0) 0x7fe9b4156f40 con 0x7fe9b4156b60 2024-05-28T14:16:54.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b2000640 1 -- 172.21.15.42:0/206210190 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe998003530 con 0x7fe9b4156b60 2024-05-28T14:16:54.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b3400640 1 -- 172.21.15.42:0/206210190 >> v2:172.21.15.63:3300/0 conn(0x7fe9b40b5f30 msgr2=0x7fe9b40b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:54.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b3400640 1 --2- 172.21.15.42:0/206210190 >> v2:172.21.15.63:3300/0 conn(0x7fe9b40b5f30 0x7fe9b40b63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b3400640 1 -- 172.21.15.42:0/206210190 >> v1:172.21.15.42:6789/0 conn(0x7fe9b4156b60 legacy=0x7fe9b415d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9b3400640 1 -- 172.21.15.42:0/206210190 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b40cd940 con 0x7fe9b40ce140 2024-05-28T14:16:54.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.269+0000 7fe9aa000640 1 --2- 172.21.15.42:0/206210190 >> v2:172.21.15.63:3300/0 conn(0x7fe9b40b5f30 0x7fe9b40b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:54.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.273+0000 7fe9b3400640 1 --2- 172.21.15.42:0/206210190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b40ce140 0x7fe9b40ce540 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40b0840 tx=0x7fe9ac01d390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43b81092c3e4160 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:54.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.273+0000 7fe9b2000640 1 -- 172.21.15.42:0/206210190 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0b2070 con 0x7fe9b40ce140 2024-05-28T14:16:54.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.273+0000 7fe9b2000640 1 -- 172.21.15.42:0/206210190 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac0a5900 con 0x7fe9b40ce140 2024-05-28T14:16:54.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.273+0000 7fe9b2000640 1 -- 172.21.15.42:0/206210190 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0a5cd0 con 0x7fe9b40ce140 2024-05-28T14:16:54.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.273+0000 7fe9b3e00640 1 -- 172.21.15.42:0/206210190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b40ce140 msgr2=0x7fe9b40ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.273+0000 7fe9b3e00640 1 --2- 172.21.15.42:0/206210190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b40ce140 0x7fe9b40ce540 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40b0840 tx=0x7fe9ac01d390 comp rx=0 tx=0).stop 2024-05-28T14:16:54.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.273+0000 7fe9b3e00640 1 -- 172.21.15.42:0/206210190 shutdown_connections 2024-05-28T14:16:54.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.273+0000 7fe9b3e00640 1 --2- 172.21.15.42:0/206210190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b40ce140 0x7fe9b40ce540 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.273+0000 7fe9b3e00640 1 --2- 172.21.15.42:0/206210190 >> v2:172.21.15.63:3300/0 conn(0x7fe9b40b5f30 0x7fe9b40b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.273+0000 7fe9b3e00640 1 -- 172.21.15.42:0/206210190 >> 172.21.15.42:0/206210190 conn(0x7fe9b4073370 msgr2=0x7fe9b406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:54.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.273+0000 7fe9b3e00640 1 -- 172.21.15.42:0/206210190 shutdown_connections 2024-05-28T14:16:54.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.273+0000 7fe9b3e00640 1 -- 172.21.15.42:0/206210190 wait complete. 2024-05-28T14:16:54.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9b3e00640 1 Processor -- start 2024-05-28T14:16:54.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9b3e00640 1 -- start start 2024-05-28T14:16:54.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9b40b5f30 0x7fe9b41f7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9b3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b40ce140 0x7fe9b413f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9b3e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b4156b60 0x7fe9b414c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9b3e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b40cf290 con 0x7fe9b40ce140 2024-05-28T14:16:54.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9b3e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b406f9a0 con 0x7fe9b40b5f30 2024-05-28T14:16:54.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9b3e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b40cf410 con 0x7fe9b4156b60 2024-05-28T14:16:54.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9aa000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b40ce140 0x7fe9b413f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9aa000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b40ce140 0x7fe9b413f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38006/0 (socket says 172.21.15.42:38006) 2024-05-28T14:16:54.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9aa000640 1 -- 172.21.15.42:0/1023006481 learned_addr learned my addr 172.21.15.42:0/1023006481 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:54.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9b2a00640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9b40b5f30 0x7fe9b41f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9b3400640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b4156b60 0x7fe9b414c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9aa000640 1 -- 172.21.15.42:0/1023006481 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b4156b60 msgr2=0x7fe9b414c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9aa000640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b4156b60 0x7fe9b414c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9aa000640 1 -- 172.21.15.42:0/1023006481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9b40b5f30 msgr2=0x7fe9b41f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9aa000640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9b40b5f30 0x7fe9b41f7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9aa000640 1 -- 172.21.15.42:0/1023006481 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b4154800 con 0x7fe9b40ce140 2024-05-28T14:16:54.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9b3400640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b4156b60 0x7fe9b414c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:54.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9aa000640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b40ce140 0x7fe9b413f3c0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00d800 tx=0x7fe99c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:54.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9ab400640 1 -- 172.21.15.42:0/1023006481 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c018020 con 0x7fe9b40ce140 2024-05-28T14:16:54.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9ab400640 1 -- 172.21.15.42:0/1023006481 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe99c0096c0 con 0x7fe9b40ce140 2024-05-28T14:16:54.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9ab400640 1 -- 172.21.15.42:0/1023006481 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c009ab0 con 0x7fe9b40ce140 2024-05-28T14:16:54.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9b3e00640 1 -- 172.21.15.42:0/1023006481 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b414f5a0 con 0x7fe9b40ce140 2024-05-28T14:16:54.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.277+0000 7fe9b3e00640 1 -- 172.21.15.42:0/1023006481 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b414fe40 con 0x7fe9b40ce140 2024-05-28T14:16:54.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.281+0000 7fe9b3e00640 1 -- 172.21.15.42:0/1023006481 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b414f990 con 0x7fe9b40ce140 2024-05-28T14:16:54.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.281+0000 7fe9ab400640 1 -- 172.21.15.42:0/1023006481 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7fe99c01b020 con 0x7fe9b40ce140 2024-05-28T14:16:54.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.281+0000 7fe9ab400640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe99006a420 0x7fe99006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.281+0000 7fe9ab400640 1 -- 172.21.15.42:0/1023006481 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fe99c086030 con 0x7fe9b40ce140 2024-05-28T14:16:54.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.281+0000 7fe9b2a00640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe99006a420 0x7fe99006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.281+0000 7fe9b2a00640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe99006a420 0x7fe99006c8e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4005890 tx=0x7fe9a4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:54.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.285+0000 7fe9ab400640 1 -- 172.21.15.42:0/1023006481 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe99c022020 con 0x7fe9b40ce140 2024-05-28T14:16:54.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.449+0000 7fe9b3e00640 1 -- 172.21.15.42:0/1023006481 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fe9b404fcf0 con 0x7fe99006a420 2024-05-28T14:16:54.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.449+0000 7fe9ab400640 1 -- 172.21.15.42:0/1023006481 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe9b404fcf0 con 0x7fe99006a420 2024-05-28T14:16:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 1 to deep-scrub 2024-05-28T14:16:54.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.453+0000 7fe9b3e00640 1 -- 172.21.15.42:0/1023006481 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe99006a420 msgr2=0x7fe99006c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.453+0000 7fe9b3e00640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe99006a420 0x7fe99006c8e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4005890 tx=0x7fe9a4009040 comp rx=0 tx=0).stop 2024-05-28T14:16:54.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.453+0000 7fe9b3e00640 1 -- 172.21.15.42:0/1023006481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b40ce140 msgr2=0x7fe9b413f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.453+0000 7fe9b3e00640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b40ce140 0x7fe9b413f3c0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00d800 tx=0x7fe99c00dcc0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.453+0000 7fe9b3e00640 1 -- 172.21.15.42:0/1023006481 shutdown_connections 2024-05-28T14:16:54.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.453+0000 7fe9b3e00640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fe99006a420 0x7fe99006c8e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.453+0000 7fe9b3e00640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b4156b60 0x7fe9b414c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.453+0000 7fe9b3e00640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b40ce140 0x7fe9b413f3c0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.453+0000 7fe9b3e00640 1 --2- 172.21.15.42:0/1023006481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9b40b5f30 0x7fe9b41f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.453+0000 7fe9b3e00640 1 -- 172.21.15.42:0/1023006481 >> 172.21.15.42:0/1023006481 conn(0x7fe9b4073370 msgr2=0x7fe9b406ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:54.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.453+0000 7fe9b3e00640 1 -- 172.21.15.42:0/1023006481 shutdown_connections 2024-05-28T14:16:54.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.453+0000 7fe9b3e00640 1 -- 172.21.15.42:0/1023006481 wait complete. 2024-05-28T14:16:54.457 INFO:tasks.ceph:Scrubbing osd.2 2024-05-28T14:16:54.458 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-28T14:16:54.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.617+0000 7f259ce00640 1 Processor -- start 2024-05-28T14:16:54.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.617+0000 7f259ce00640 1 -- start start 2024-05-28T14:16:54.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f259ce00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f259806c940 0x7f259809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f259ce00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f259809af90 0x7f2598070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f259ce00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f259806ff10 con 0x7f259806e740 2024-05-28T14:16:54.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f259ce00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f259806eb20 con 0x7f259806c940 2024-05-28T14:16:54.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f259ce00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f259809b3e0 con 0x7f259809af90 2024-05-28T14:16:54.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f258fe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f259806c940 0x7f259809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f2595e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f259806e740 0x7f25980cdbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53100/0 (socket says 172.21.15.42:53100) 2024-05-28T14:16:54.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f2595e00640 1 -- 172.21.15.42:0/285882604 learned_addr learned my addr 172.21.15.42:0/285882604 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:54.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f2596800640 1 --2- 172.21.15.42:0/285882604 >> v2:172.21.15.63:3300/0 conn(0x7f259809af90 0x7f2598070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f258fe00640 1 -- 172.21.15.42:0/285882604 >> v2:172.21.15.63:3300/0 conn(0x7f259809af90 msgr2=0x7f2598070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:54.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f258fe00640 1 --2- 172.21.15.42:0/285882604 >> v2:172.21.15.63:3300/0 conn(0x7f259809af90 0x7f2598070f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f258fe00640 1 -- 172.21.15.42:0/285882604 >> v1:172.21.15.42:6789/0 conn(0x7f259806e740 legacy=0x7f25980cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f258fe00640 1 -- 172.21.15.42:0/285882604 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25980701b0 con 0x7f259806c940 2024-05-28T14:16:54.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f2596800640 1 --2- 172.21.15.42:0/285882604 >> v2:172.21.15.63:3300/0 conn(0x7f259809af90 0x7f2598070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:54.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.621+0000 7f2595400640 1 -- 172.21.15.42:0/285882604 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 710836438 0 0) 0x7f259806ff10 con 0x7f259806e740 2024-05-28T14:16:54.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f258fe00640 1 --2- 172.21.15.42:0/285882604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f259806c940 0x7f259809aa50 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f25980d3570 tx=0x7f258001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2678c360eae3c7cd server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:54.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f2595400640 1 -- 172.21.15.42:0/285882604 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800b1070 con 0x7f259806c940 2024-05-28T14:16:54.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f2595400640 1 -- 172.21.15.42:0/285882604 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25800ac620 con 0x7f259806c940 2024-05-28T14:16:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f2595400640 1 -- 172.21.15.42:0/285882604 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800ac9f0 con 0x7f259806c940 2024-05-28T14:16:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 -- 172.21.15.42:0/285882604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f259806c940 msgr2=0x7f259809aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 --2- 172.21.15.42:0/285882604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f259806c940 0x7f259809aa50 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f25980d3570 tx=0x7f258001c3c0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 -- 172.21.15.42:0/285882604 shutdown_connections 2024-05-28T14:16:54.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 --2- 172.21.15.42:0/285882604 >> v2:172.21.15.63:3300/0 conn(0x7f259809af90 0x7f2598070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 --2- 172.21.15.42:0/285882604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f259806c940 0x7f259809aa50 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 -- 172.21.15.42:0/285882604 >> 172.21.15.42:0/285882604 conn(0x7f25980b3fa0 msgr2=0x7f25980b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:54.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 -- 172.21.15.42:0/285882604 shutdown_connections 2024-05-28T14:16:54.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 -- 172.21.15.42:0/285882604 wait complete. 2024-05-28T14:16:54.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 Processor -- start 2024-05-28T14:16:54.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 -- start start 2024-05-28T14:16:54.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f259806c940 0x7f25980c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f259806e740 0x7f2598155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f259809af90 0x7f25981573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f259809baa0 con 0x7f259806e740 2024-05-28T14:16:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f25980ce2e0 con 0x7f259809af90 2024-05-28T14:16:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.625+0000 7f259ce00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f259815e120 con 0x7f259806c940 2024-05-28T14:16:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f258fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f259806e740 0x7f2598155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f258fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f259806e740 0x7f2598155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38014/0 (socket says 172.21.15.42:38014) 2024-05-28T14:16:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f258fe00640 1 -- 172.21.15.42:0/1131510391 learned_addr learned my addr 172.21.15.42:0/1131510391 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f2596800640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f259809af90 0x7f25981573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f2595e00640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f259806c940 0x7f25980c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f258fe00640 1 -- 172.21.15.42:0/1131510391 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f259806c940 msgr2=0x7f25980c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f258fe00640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f259806c940 0x7f25980c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f258fe00640 1 -- 172.21.15.42:0/1131510391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f259809af90 msgr2=0x7f25981573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f258fe00640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f259809af90 0x7f25981573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f258fe00640 1 -- 172.21.15.42:0/1131510391 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25980c7db0 con 0x7f259806e740 2024-05-28T14:16:54.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f2595e00640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f259806c940 0x7f25980c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:54.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f258fe00640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f259806e740 0x7f2598155350 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f25800af040 tx=0x7f2580019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:54.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f258e000640 1 -- 172.21.15.42:0/1131510391 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800b9020 con 0x7f259806e740 2024-05-28T14:16:54.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f258e000640 1 -- 172.21.15.42:0/1131510391 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25800b5630 con 0x7f259806e740 2024-05-28T14:16:54.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f258e000640 1 -- 172.21.15.42:0/1131510391 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800b1040 con 0x7f259806e740 2024-05-28T14:16:54.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f259ce00640 1 -- 172.21.15.42:0/1131510391 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25980c8090 con 0x7f259806e740 2024-05-28T14:16:54.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f259ce00640 1 -- 172.21.15.42:0/1131510391 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25980c8650 con 0x7f259806e740 2024-05-28T14:16:54.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f259ce00640 1 -- 172.21.15.42:0/1131510391 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2598068a90 con 0x7f259806e740 2024-05-28T14:16:54.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.629+0000 7f258e000640 1 -- 172.21.15.42:0/1131510391 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f25800bc020 con 0x7f259806e740 2024-05-28T14:16:54.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.633+0000 7f258e000640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f257406a450 0x7f257406c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.633+0000 7f258e000640 1 -- 172.21.15.42:0/1131510391 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f258011fd80 con 0x7f259806e740 2024-05-28T14:16:54.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.633+0000 7f2595e00640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f257406a450 0x7f257406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.633+0000 7f258e000640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.42:6816/254756381,v1:172.21.15.42:6817/254756381] conn(0x7f2574072770 0x7f2574074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.633+0000 7f258e000640 1 -- 172.21.15.42:0/1131510391 --> [v2:172.21.15.42:6816/254756381,v1:172.21.15.42:6817/254756381] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25800bfe50 con 0x7f2574072770 2024-05-28T14:16:54.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.633+0000 7f258e000640 1 -- 172.21.15.42:0/1131510391 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f2580120ef0 con 0x7f259806e740 2024-05-28T14:16:54.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.633+0000 7f2596800640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.42:6816/254756381,v1:172.21.15.42:6817/254756381] conn(0x7f2574072770 0x7f2574074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.633+0000 7f2595e00640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f257406a450 0x7f257406c910 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2588007fd0 tx=0x7f2588007c20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:54.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.633+0000 7f2596800640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.42:6816/254756381,v1:172.21.15.42:6817/254756381] conn(0x7f2574072770 0x7f2574074bf0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:54.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.633+0000 7f258e000640 1 -- 172.21.15.42:0/1131510391 <== osd.2 v2:172.21.15.42:6816/254756381 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f25800bfe50 con 0x7f2574072770 2024-05-28T14:16:54.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.645+0000 7f259ce00640 1 -- 172.21.15.42:0/1131510391 --> [v2:172.21.15.42:6816/254756381,v1:172.21.15.42:6817/254756381] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f25980d2110 con 0x7f2574072770 2024-05-28T14:16:54.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.645+0000 7f258e000640 1 -- 172.21.15.42:0/1131510391 <== osd.2 v2:172.21.15.42:6816/254756381 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f25980d2110 con 0x7f2574072770 2024-05-28T14:16:54.632 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:16:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-28T14:16:54.632 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:16:54.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.645+0000 7f259ce00640 1 -- 172.21.15.42:0/1131510391 >> [v2:172.21.15.42:6816/254756381,v1:172.21.15.42:6817/254756381] conn(0x7f2574072770 msgr2=0x7f2574074bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.645+0000 7f259ce00640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.42:6816/254756381,v1:172.21.15.42:6817/254756381] conn(0x7f2574072770 0x7f2574074bf0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.649+0000 7f259ce00640 1 -- 172.21.15.42:0/1131510391 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f257406a450 msgr2=0x7f257406c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.649+0000 7f259ce00640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f257406a450 0x7f257406c910 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2588007fd0 tx=0x7f2588007c20 comp rx=0 tx=0).stop 2024-05-28T14:16:54.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.649+0000 7f259ce00640 1 -- 172.21.15.42:0/1131510391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f259806e740 msgr2=0x7f2598155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.649+0000 7f259ce00640 1 --2- 172.21.15.42:0/1131510391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f259806e740 0x7f2598155350 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f25800af040 tx=0x7f2580019160 comp rx=0 tx=0).stop 2024-05-28T14:16:54.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.649+0000 7f2596800640 1 -- 172.21.15.42:0/1131510391 reap_dead start 2024-05-28T14:16:54.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.649+0000 7f259ce00640 1 -- 172.21.15.42:0/1131510391 shutdown_connections 2024-05-28T14:16:54.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.649+0000 7f259ce00640 1 -- 172.21.15.42:0/1131510391 >> 172.21.15.42:0/1131510391 conn(0x7f25980b3fa0 msgr2=0x7f25980cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:54.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.649+0000 7f259ce00640 1 -- 172.21.15.42:0/1131510391 shutdown_connections 2024-05-28T14:16:54.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.649+0000 7f259ce00640 1 -- 172.21.15.42:0/1131510391 wait complete. 2024-05-28T14:16:54.651 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-28T14:16:54.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.809+0000 7fb653e00640 1 Processor -- start 2024-05-28T14:16:54.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.809+0000 7fb653e00640 1 -- start start 2024-05-28T14:16:54.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.809+0000 7fb653e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb6540d0a70 0x7fb6540add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.809+0000 7fb653e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb654073400 con 0x7fb65415c370 2024-05-28T14:16:54.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.809+0000 7fb653e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6540736e0 con 0x7fb654158ea0 2024-05-28T14:16:54.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.809+0000 7fb653e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fb654159280 con 0x7fb6540d0a70 2024-05-28T14:16:54.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.809+0000 7fb64a000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb6540d0a70 0x7fb6540add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.809+0000 7fb64a000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb6540d0a70 0x7fb6540add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38028/0 (socket says 172.21.15.42:38028) 2024-05-28T14:16:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.809+0000 7fb64a000640 1 -- 172.21.15.42:0/4097032876 learned_addr learned my addr 172.21.15.42:0/4097032876 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.809+0000 7fb64a000640 1 -- 172.21.15.42:0/4097032876 >> v1:172.21.15.63:6789/0 conn(0x7fb654158ea0 legacy=0x7fb6540d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.809+0000 7fb64a000640 1 -- 172.21.15.42:0/4097032876 >> v1:172.21.15.42:6789/0 conn(0x7fb65415c370 legacy=0x7fb65406f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.809+0000 7fb64a000640 1 -- 172.21.15.42:0/4097032876 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65406e960 con 0x7fb6540d0a70 2024-05-28T14:16:54.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.813+0000 7fb64a000640 1 --2- 172.21.15.42:0/4097032876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6540d0a70 0x7fb6540add40 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009870 tx=0x7fb63c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9988c3125d5d43e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:54.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.813+0000 7fb652000640 1 -- 172.21.15.42:0/4097032876 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0b2070 con 0x7fb6540d0a70 2024-05-28T14:16:54.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.813+0000 7fb652000640 1 -- 172.21.15.42:0/4097032876 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb63c0a9070 con 0x7fb6540d0a70 2024-05-28T14:16:54.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.813+0000 7fb652000640 1 -- 172.21.15.42:0/4097032876 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0ad880 con 0x7fb6540d0a70 2024-05-28T14:16:54.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.813+0000 7fb653e00640 1 -- 172.21.15.42:0/4097032876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6540d0a70 msgr2=0x7fb6540add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.813+0000 7fb653e00640 1 --2- 172.21.15.42:0/4097032876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6540d0a70 0x7fb6540add40 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009870 tx=0x7fb63c01c560 comp rx=0 tx=0).stop 2024-05-28T14:16:54.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.813+0000 7fb653e00640 1 -- 172.21.15.42:0/4097032876 shutdown_connections 2024-05-28T14:16:54.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.813+0000 7fb653e00640 1 --2- 172.21.15.42:0/4097032876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6540d0a70 0x7fb6540add40 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.813+0000 7fb653e00640 1 -- 172.21.15.42:0/4097032876 >> 172.21.15.42:0/4097032876 conn(0x7fb65406bbc0 msgr2=0x7fb65406bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:54.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.813+0000 7fb653e00640 1 -- 172.21.15.42:0/4097032876 shutdown_connections 2024-05-28T14:16:54.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.813+0000 7fb653e00640 1 -- 172.21.15.42:0/4097032876 wait complete. 2024-05-28T14:16:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653e00640 1 Processor -- start 2024-05-28T14:16:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653e00640 1 -- start start 2024-05-28T14:16:54.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6540d0a70 0x7fb6541f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb654158ea0 0x7fb6540b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb65415c370 0x7fb6540b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb654168fd0 con 0x7fb65415c370 2024-05-28T14:16:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb654168e50 con 0x7fb6540d0a70 2024-05-28T14:16:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb654169150 con 0x7fb654158ea0 2024-05-28T14:16:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb65415c370 0x7fb6540b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb65415c370 0x7fb6540b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38032/0 (socket says 172.21.15.42:38032) 2024-05-28T14:16:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653400640 1 -- 172.21.15.42:0/4219968 learned_addr learned my addr 172.21.15.42:0/4219968 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb652a00640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6540d0a70 0x7fb6541f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb64a000640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb654158ea0 0x7fb6540b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653400640 1 -- 172.21.15.42:0/4219968 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb654158ea0 msgr2=0x7fb6540b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653400640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb654158ea0 0x7fb6540b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653400640 1 -- 172.21.15.42:0/4219968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6540d0a70 msgr2=0x7fb6541f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653400640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6540d0a70 0x7fb6541f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653400640 1 -- 172.21.15.42:0/4219968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65414e6d0 con 0x7fb65415c370 2024-05-28T14:16:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb652a00640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6540d0a70 0x7fb6541f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653400640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb65415c370 0x7fb6540b2570 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00c6c0 tx=0x7fb64c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb64b400640 1 -- 172.21.15.42:0/4219968 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c003b80 con 0x7fb65415c370 2024-05-28T14:16:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb64b400640 1 -- 172.21.15.42:0/4219968 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64c003d20 con 0x7fb65415c370 2024-05-28T14:16:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653e00640 1 -- 172.21.15.42:0/4219968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb654150950 con 0x7fb65415c370 2024-05-28T14:16:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb64b400640 1 -- 172.21.15.42:0/4219968 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0052f0 con 0x7fb65415c370 2024-05-28T14:16:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653e00640 1 -- 172.21.15.42:0/4219968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6540b0ef0 con 0x7fb65415c370 2024-05-28T14:16:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.817+0000 7fb653e00640 1 -- 172.21.15.42:0/4219968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb65404fcf0 con 0x7fb65415c370 2024-05-28T14:16:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.821+0000 7fb64b400640 1 -- 172.21.15.42:0/4219968 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7fb64c005490 con 0x7fb65415c370 2024-05-28T14:16:54.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.821+0000 7fb64b400640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb63006a420 0x7fb63006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:54.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.821+0000 7fb64b400640 1 -- 172.21.15.42:0/4219968 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fb64c0858a0 con 0x7fb65415c370 2024-05-28T14:16:54.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.821+0000 7fb652a00640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb63006a420 0x7fb63006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:54.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.821+0000 7fb652a00640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb63006a420 0x7fb63006c8e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fb644000f70 tx=0x7fb644009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:54.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.825+0000 7fb64b400640 1 -- 172.21.15.42:0/4219968 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb64c0578c0 con 0x7fb65415c370 2024-05-28T14:16:54.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.989+0000 7fb653e00640 1 -- 172.21.15.42:0/4219968 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fb654071da0 con 0x7fb63006a420 2024-05-28T14:16:54.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.989+0000 7fb64b400640 1 -- 172.21.15.42:0/4219968 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb654071da0 con 0x7fb63006a420 2024-05-28T14:16:54.973 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 2 to deep-scrub 2024-05-28T14:16:54.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.989+0000 7fb653e00640 1 -- 172.21.15.42:0/4219968 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb63006a420 msgr2=0x7fb63006c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.989+0000 7fb653e00640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb63006a420 0x7fb63006c8e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fb644000f70 tx=0x7fb644009040 comp rx=0 tx=0).stop 2024-05-28T14:16:54.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.989+0000 7fb653e00640 1 -- 172.21.15.42:0/4219968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb65415c370 msgr2=0x7fb6540b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:54.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.989+0000 7fb653e00640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb65415c370 0x7fb6540b2570 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00c6c0 tx=0x7fb64c00cb80 comp rx=0 tx=0).stop 2024-05-28T14:16:54.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.989+0000 7fb653e00640 1 -- 172.21.15.42:0/4219968 shutdown_connections 2024-05-28T14:16:54.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.989+0000 7fb653e00640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fb63006a420 0x7fb63006c8e0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.989+0000 7fb653e00640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb65415c370 0x7fb6540b2570 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.989+0000 7fb653e00640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb654158ea0 0x7fb6540b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.989+0000 7fb653e00640 1 --2- 172.21.15.42:0/4219968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6540d0a70 0x7fb6541f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:54.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.989+0000 7fb653e00640 1 -- 172.21.15.42:0/4219968 >> 172.21.15.42:0/4219968 conn(0x7fb65406bbc0 msgr2=0x7fb6540b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:54.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.989+0000 7fb653e00640 1 -- 172.21.15.42:0/4219968 shutdown_connections 2024-05-28T14:16:54.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:54.989+0000 7fb653e00640 1 -- 172.21.15.42:0/4219968 wait complete. 2024-05-28T14:16:54.996 INFO:tasks.ceph:Scrubbing osd.3 2024-05-28T14:16:54.996 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-28T14:16:55.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.157+0000 7f1d5c400640 1 Processor -- start 2024-05-28T14:16:55.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.157+0000 7f1d5c400640 1 -- start start 2024-05-28T14:16:55.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.157+0000 7f1d5c400640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1d5415ff90 0x7f1d5409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.157+0000 7f1d5c400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1d5409af90 0x7f1d5409b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.157+0000 7f1d5c400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d540b5330 0x7f1d540b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.157+0000 7f1d5c400640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f1d540b3170 con 0x7f1d5409af90 2024-05-28T14:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.157+0000 7f1d5c400640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1d54160370 con 0x7f1d5415ff90 2024-05-28T14:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.157+0000 7f1d5c400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1d540c5ca0 con 0x7f1d540b5330 2024-05-28T14:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.157+0000 7f1d59000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1d5409af90 0x7f1d5409b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.157+0000 7f1d59a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1d5415ff90 0x7f1d5409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.157+0000 7f1d59000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1d5409af90 0x7f1d5409b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38044/0 (socket says 172.21.15.42:38044) 2024-05-28T14:16:55.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.161+0000 7f1d59000640 1 -- 172.21.15.42:0/242699411 learned_addr learned my addr 172.21.15.42:0/242699411 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:55.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.161+0000 7f1d5a400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d540b5330 0x7f1d540b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.161+0000 7f1d59000640 1 -- 172.21.15.42:0/242699411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d540b5330 msgr2=0x7f1d540b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:55.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.161+0000 7f1d59000640 1 --2- 172.21.15.42:0/242699411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d540b5330 0x7f1d540b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.161+0000 7f1d59000640 1 -- 172.21.15.42:0/242699411 >> v2:172.21.15.63:3300/0 conn(0x7f1d5415ff90 msgr2=0x7f1d5409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:55.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.161+0000 7f1d59000640 1 --2- 172.21.15.42:0/242699411 >> v2:172.21.15.63:3300/0 conn(0x7f1d5415ff90 0x7f1d5409aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.161+0000 7f1d59000640 1 -- 172.21.15.42:0/242699411 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d540b33c0 con 0x7f1d5409af90 2024-05-28T14:16:55.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.161+0000 7f1d59000640 1 --2- 172.21.15.42:0/242699411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5409af90 0x7f1d5409b3f0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f1d54073740 tx=0x7f1d4001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5800a539c4bdd1a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.161+0000 7f1d53e00640 1 -- 172.21.15.42:0/242699411 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d400b2070 con 0x7f1d5409af90 2024-05-28T14:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.161+0000 7f1d53e00640 1 -- 172.21.15.42:0/242699411 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d400ad640 con 0x7f1d5409af90 2024-05-28T14:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.161+0000 7f1d53e00640 1 -- 172.21.15.42:0/242699411 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d400ada10 con 0x7f1d5409af90 2024-05-28T14:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 -- 172.21.15.42:0/242699411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5409af90 msgr2=0x7f1d5409b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 --2- 172.21.15.42:0/242699411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5409af90 0x7f1d5409b3f0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f1d54073740 tx=0x7f1d4001c3d0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 -- 172.21.15.42:0/242699411 shutdown_connections 2024-05-28T14:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 --2- 172.21.15.42:0/242699411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d540b5330 0x7f1d540b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 --2- 172.21.15.42:0/242699411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5409af90 0x7f1d5409b3f0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 --2- 172.21.15.42:0/242699411 >> v2:172.21.15.63:3300/0 conn(0x7f1d5415ff90 0x7f1d5409aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 -- 172.21.15.42:0/242699411 >> 172.21.15.42:0/242699411 conn(0x7f1d540b3fd0 msgr2=0x7f1d540b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:55.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 -- 172.21.15.42:0/242699411 shutdown_connections 2024-05-28T14:16:55.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 -- 172.21.15.42:0/242699411 wait complete. 2024-05-28T14:16:55.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 Processor -- start 2024-05-28T14:16:55.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 -- start start 2024-05-28T14:16:55.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d5409af90 0x7f1d54070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d540b5330 0x7f1d540cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5415ff90 0x7f1d54154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5409bc40 con 0x7f1d5415ff90 2024-05-28T14:16:55.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5409bac0 con 0x7f1d540b5330 2024-05-28T14:16:55.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5c400640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1d540b6110 con 0x7f1d5409af90 2024-05-28T14:16:55.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5415ff90 0x7f1d54154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5415ff90 0x7f1d54154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38048/0 (socket says 172.21.15.42:38048) 2024-05-28T14:16:55.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5a400640 1 -- 172.21.15.42:0/2268160314 learned_addr learned my addr 172.21.15.42:0/2268160314 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:55.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d59a00640 1 --2- 172.21.15.42:0/2268160314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d5409af90 0x7f1d54070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d59000640 1 --2- 172.21.15.42:0/2268160314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d540b5330 0x7f1d540cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5a400640 1 -- 172.21.15.42:0/2268160314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d5409af90 msgr2=0x7f1d54070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5a400640 1 --2- 172.21.15.42:0/2268160314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d5409af90 0x7f1d54070920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5a400640 1 -- 172.21.15.42:0/2268160314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d540b5330 msgr2=0x7f1d540cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5a400640 1 --2- 172.21.15.42:0/2268160314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d540b5330 0x7f1d540cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5a400640 1 -- 172.21.15.42:0/2268160314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d540c6e70 con 0x7f1d5415ff90 2024-05-28T14:16:55.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.165+0000 7f1d5a400640 1 --2- 172.21.15.42:0/2268160314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5415ff90 0x7f1d54154940 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00efc0 tx=0x7f1d4c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:55.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d51600640 1 -- 172.21.15.42:0/2268160314 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c018020 con 0x7f1d5415ff90 2024-05-28T14:16:55.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d51600640 1 -- 172.21.15.42:0/2268160314 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d4c014480 con 0x7f1d5415ff90 2024-05-28T14:16:55.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d51600640 1 -- 172.21.15.42:0/2268160314 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c014870 con 0x7f1d5415ff90 2024-05-28T14:16:55.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d5c400640 1 -- 172.21.15.42:0/2268160314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d540c7150 con 0x7f1d5415ff90 2024-05-28T14:16:55.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d5c400640 1 -- 172.21.15.42:0/2268160314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d540c7710 con 0x7f1d5415ff90 2024-05-28T14:16:55.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d5c400640 1 -- 172.21.15.42:0/2268160314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d54068a90 con 0x7f1d5415ff90 2024-05-28T14:16:55.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d51600640 1 -- 172.21.15.42:0/2268160314 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f1d4c01b020 con 0x7f1d5415ff90 2024-05-28T14:16:55.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d51600640 1 --2- 172.21.15.42:0/2268160314 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1d3406a420 0x7f1d3406c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d51600640 1 -- 172.21.15.42:0/2268160314 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f1d4c086550 con 0x7f1d5415ff90 2024-05-28T14:16:55.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d59a00640 1 --2- 172.21.15.42:0/2268160314 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1d3406a420 0x7f1d3406c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d51600640 1 --2- 172.21.15.42:0/2268160314 >> [v2:172.21.15.42:6824/2252246131,v1:172.21.15.42:6825/2252246131] conn(0x7f1d34072740 0x7f1d34074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d51600640 1 -- 172.21.15.42:0/2268160314 --> [v2:172.21.15.42:6824/2252246131,v1:172.21.15.42:6825/2252246131] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d34075290 con 0x7f1d34072740 2024-05-28T14:16:55.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d51600640 1 -- 172.21.15.42:0/2268160314 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f1d4c092050 con 0x7f1d5415ff90 2024-05-28T14:16:55.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d59000640 1 --2- 172.21.15.42:0/2268160314 >> [v2:172.21.15.42:6824/2252246131,v1:172.21.15.42:6825/2252246131] conn(0x7f1d34072740 0x7f1d34074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.169+0000 7f1d59a00640 1 --2- 172.21.15.42:0/2268160314 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1d3406a420 0x7f1d3406c8e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009a40 tx=0x7f1d48009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:55.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.173+0000 7f1d59000640 1 --2- 172.21.15.42:0/2268160314 >> [v2:172.21.15.42:6824/2252246131,v1:172.21.15.42:6825/2252246131] conn(0x7f1d34072740 0x7f1d34074bc0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:55.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.173+0000 7f1d51600640 1 -- 172.21.15.42:0/2268160314 <== osd.3 v2:172.21.15.42:6824/2252246131 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1d34075290 con 0x7f1d34072740 2024-05-28T14:16:55.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.185+0000 7f1d5c400640 1 -- 172.21.15.42:0/2268160314 --> [v2:172.21.15.42:6824/2252246131,v1:172.21.15.42:6825/2252246131] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1d54159c90 con 0x7f1d34072740 2024-05-28T14:16:55.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.185+0000 7f1d51600640 1 -- 172.21.15.42:0/2268160314 <== osd.3 v2:172.21.15.42:6824/2252246131 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1d54159c90 con 0x7f1d34072740 2024-05-28T14:16:55.171 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:16:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-28T14:16:55.171 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:16:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.185+0000 7f1d5c400640 1 -- 172.21.15.42:0/2268160314 >> [v2:172.21.15.42:6824/2252246131,v1:172.21.15.42:6825/2252246131] conn(0x7f1d34072740 msgr2=0x7f1d34074bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.185+0000 7f1d5c400640 1 --2- 172.21.15.42:0/2268160314 >> [v2:172.21.15.42:6824/2252246131,v1:172.21.15.42:6825/2252246131] conn(0x7f1d34072740 0x7f1d34074bc0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.185+0000 7f1d5c400640 1 -- 172.21.15.42:0/2268160314 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1d3406a420 msgr2=0x7f1d3406c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.185+0000 7f1d5c400640 1 --2- 172.21.15.42:0/2268160314 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1d3406a420 0x7f1d3406c8e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009a40 tx=0x7f1d48009190 comp rx=0 tx=0).stop 2024-05-28T14:16:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.185+0000 7f1d5c400640 1 -- 172.21.15.42:0/2268160314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5415ff90 msgr2=0x7f1d54154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.185+0000 7f1d5c400640 1 --2- 172.21.15.42:0/2268160314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5415ff90 0x7f1d54154940 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00efc0 tx=0x7f1d4c00c490 comp rx=0 tx=0).stop 2024-05-28T14:16:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.185+0000 7f1d5a400640 1 -- 172.21.15.42:0/2268160314 reap_dead start 2024-05-28T14:16:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.185+0000 7f1d5c400640 1 -- 172.21.15.42:0/2268160314 shutdown_connections 2024-05-28T14:16:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.185+0000 7f1d5c400640 1 -- 172.21.15.42:0/2268160314 >> 172.21.15.42:0/2268160314 conn(0x7f1d540b3fd0 msgr2=0x7f1d5415a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.185+0000 7f1d5c400640 1 -- 172.21.15.42:0/2268160314 shutdown_connections 2024-05-28T14:16:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.185+0000 7f1d5c400640 1 -- 172.21.15.42:0/2268160314 wait complete. 2024-05-28T14:16:55.190 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-28T14:16:55.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa759000640 1 Processor -- start 2024-05-28T14:16:55.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa759000640 1 -- start start 2024-05-28T14:16:55.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa759000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa7540d0450 0x7fa75409b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa759000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa75415d680 0x7fa7540b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa759000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7540d0160 con 0x7fa75415c880 2024-05-28T14:16:55.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa759000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fa75409b520 con 0x7fa7540d0450 2024-05-28T14:16:55.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa759000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa75409abf0 con 0x7fa75415d680 2024-05-28T14:16:55.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa753400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa7540d0450 0x7fa75409b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa753400640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa7540d0450 0x7fa75409b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38064/0 (socket says 172.21.15.42:38064) 2024-05-28T14:16:55.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa752a00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa75415c880 0x7fa7540d0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53122/0 (socket says 172.21.15.42:53122) 2024-05-28T14:16:55.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa753400640 1 -- 172.21.15.42:0/2309978113 learned_addr learned my addr 172.21.15.42:0/2309978113 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:55.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa753e00640 1 --2- 172.21.15.42:0/2309978113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa75415d680 0x7fa7540b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa753400640 1 -- 172.21.15.42:0/2309978113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa75415d680 msgr2=0x7fa7540b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa753400640 1 --2- 172.21.15.42:0/2309978113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa75415d680 0x7fa7540b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa753400640 1 -- 172.21.15.42:0/2309978113 >> v1:172.21.15.42:6789/0 conn(0x7fa75415c880 legacy=0x7fa7540d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa752000640 1 -- 172.21.15.42:0/2309978113 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2021828308 0 0) 0x7fa7540d0160 con 0x7fa75415c880 2024-05-28T14:16:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.349+0000 7fa753400640 1 -- 172.21.15.42:0/2309978113 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75409aed0 con 0x7fa7540d0450 2024-05-28T14:16:55.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.353+0000 7fa753400640 1 --2- 172.21.15.42:0/2309978113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7540d0450 0x7fa75409b9b0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fa744009870 tx=0x7fa74401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b5fdc28f784aef server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:55.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.353+0000 7fa752000640 1 -- 172.21.15.42:0/2309978113 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7440ab5f0 con 0x7fa7540d0450 2024-05-28T14:16:55.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.353+0000 7fa752000640 1 -- 172.21.15.42:0/2309978113 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7440b0070 con 0x7fa7540d0450 2024-05-28T14:16:55.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.353+0000 7fa752000640 1 -- 172.21.15.42:0/2309978113 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7440ab9b0 con 0x7fa7540d0450 2024-05-28T14:16:55.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.353+0000 7fa759000640 1 -- 172.21.15.42:0/2309978113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7540d0450 msgr2=0x7fa75409b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.353+0000 7fa759000640 1 --2- 172.21.15.42:0/2309978113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7540d0450 0x7fa75409b9b0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fa744009870 tx=0x7fa74401c560 comp rx=0 tx=0).stop 2024-05-28T14:16:55.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.353+0000 7fa759000640 1 -- 172.21.15.42:0/2309978113 shutdown_connections 2024-05-28T14:16:55.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.353+0000 7fa759000640 1 --2- 172.21.15.42:0/2309978113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa75415d680 0x7fa7540b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.353+0000 7fa759000640 1 --2- 172.21.15.42:0/2309978113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7540d0450 0x7fa75409b9b0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.353+0000 7fa759000640 1 -- 172.21.15.42:0/2309978113 >> 172.21.15.42:0/2309978113 conn(0x7fa7540b3dd0 msgr2=0x7fa7540b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:55.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.353+0000 7fa759000640 1 -- 172.21.15.42:0/2309978113 shutdown_connections 2024-05-28T14:16:55.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.353+0000 7fa759000640 1 -- 172.21.15.42:0/2309978113 wait complete. 2024-05-28T14:16:55.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.353+0000 7fa759000640 1 Processor -- start 2024-05-28T14:16:55.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.353+0000 7fa759000640 1 -- start start 2024-05-28T14:16:55.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa759000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7540d0450 0x7fa7540710a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa759000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa75415c880 0x7fa75415f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa759000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75415d680 0x7fa754151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa759000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa7540cda10 con 0x7fa75415d680 2024-05-28T14:16:55.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa759000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa7540cd890 con 0x7fa75415c880 2024-05-28T14:16:55.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa759000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa7540cdb90 con 0x7fa7540d0450 2024-05-28T14:16:55.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa753e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75415d680 0x7fa754151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa753e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75415d680 0x7fa754151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38066/0 (socket says 172.21.15.42:38066) 2024-05-28T14:16:55.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa753e00640 1 -- 172.21.15.42:0/34821443 learned_addr learned my addr 172.21.15.42:0/34821443 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:55.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa752a00640 1 --2- 172.21.15.42:0/34821443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa75415c880 0x7fa75415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa753400640 1 --2- 172.21.15.42:0/34821443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7540d0450 0x7fa7540710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa753e00640 1 -- 172.21.15.42:0/34821443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7540d0450 msgr2=0x7fa7540710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa753e00640 1 --2- 172.21.15.42:0/34821443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7540d0450 0x7fa7540710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa753e00640 1 -- 172.21.15.42:0/34821443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa75415c880 msgr2=0x7fa75415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa753e00640 1 --2- 172.21.15.42:0/34821443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa75415c880 0x7fa75415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa753e00640 1 -- 172.21.15.42:0/34821443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75415fef0 con 0x7fa75415d680 2024-05-28T14:16:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa753e00640 1 --2- 172.21.15.42:0/34821443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75415d680 0x7fa754151ac0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00efc0 tx=0x7fa74c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa74b400640 1 -- 172.21.15.42:0/34821443 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c018020 con 0x7fa75415d680 2024-05-28T14:16:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa74b400640 1 -- 172.21.15.42:0/34821443 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74c014480 con 0x7fa75415d680 2024-05-28T14:16:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa74b400640 1 -- 172.21.15.42:0/34821443 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c014870 con 0x7fa75415d680 2024-05-28T14:16:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa759000640 1 -- 172.21.15.42:0/34821443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75415ddd0 con 0x7fa75415d680 2024-05-28T14:16:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa759000640 1 -- 172.21.15.42:0/34821443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa75415e610 con 0x7fa75415d680 2024-05-28T14:16:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.357+0000 7fa759000640 1 -- 172.21.15.42:0/34821443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa75415e810 con 0x7fa75415d680 2024-05-28T14:16:55.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.361+0000 7fa74b400640 1 -- 172.21.15.42:0/34821443 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7fa74c01b020 con 0x7fa75415d680 2024-05-28T14:16:55.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.361+0000 7fa74b400640 1 --2- 172.21.15.42:0/34821443 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa73406a420 0x7fa73406c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.361+0000 7fa74b400640 1 -- 172.21.15.42:0/34821443 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fa74c086400 con 0x7fa75415d680 2024-05-28T14:16:55.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.361+0000 7fa753400640 1 --2- 172.21.15.42:0/34821443 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa73406a420 0x7fa73406c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.361+0000 7fa753400640 1 --2- 172.21.15.42:0/34821443 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa73406a420 0x7fa73406c8e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa7440af000 tx=0x7fa7440a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:55.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.365+0000 7fa74b400640 1 -- 172.21.15.42:0/34821443 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa75415e810 con 0x7fa75415d680 2024-05-28T14:16:55.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.529+0000 7fa759000640 1 -- 172.21.15.42:0/34821443 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fa7540ce780 con 0x7fa73406a420 2024-05-28T14:16:55.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.529+0000 7fa74b400640 1 -- 172.21.15.42:0/34821443 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa7540ce780 con 0x7fa73406a420 2024-05-28T14:16:55.515 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 3 to deep-scrub 2024-05-28T14:16:55.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.533+0000 7fa759000640 1 -- 172.21.15.42:0/34821443 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa73406a420 msgr2=0x7fa73406c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.533+0000 7fa759000640 1 --2- 172.21.15.42:0/34821443 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa73406a420 0x7fa73406c8e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa7440af000 tx=0x7fa7440a7000 comp rx=0 tx=0).stop 2024-05-28T14:16:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.533+0000 7fa759000640 1 -- 172.21.15.42:0/34821443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75415d680 msgr2=0x7fa754151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.533+0000 7fa759000640 1 --2- 172.21.15.42:0/34821443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75415d680 0x7fa754151ac0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00efc0 tx=0x7fa74c00c490 comp rx=0 tx=0).stop 2024-05-28T14:16:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.533+0000 7fa759000640 1 -- 172.21.15.42:0/34821443 shutdown_connections 2024-05-28T14:16:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.533+0000 7fa759000640 1 --2- 172.21.15.42:0/34821443 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fa73406a420 0x7fa73406c8e0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.533+0000 7fa759000640 1 --2- 172.21.15.42:0/34821443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75415d680 0x7fa754151ac0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.533+0000 7fa759000640 1 --2- 172.21.15.42:0/34821443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa75415c880 0x7fa75415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.533+0000 7fa759000640 1 --2- 172.21.15.42:0/34821443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7540d0450 0x7fa7540710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.533+0000 7fa759000640 1 -- 172.21.15.42:0/34821443 >> 172.21.15.42:0/34821443 conn(0x7fa7540b3dd0 msgr2=0x7fa7540b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:55.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.533+0000 7fa759000640 1 -- 172.21.15.42:0/34821443 shutdown_connections 2024-05-28T14:16:55.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.533+0000 7fa759000640 1 -- 172.21.15.42:0/34821443 wait complete. 2024-05-28T14:16:55.537 INFO:tasks.ceph:Scrubbing osd.4 2024-05-28T14:16:55.538 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-28T14:16:55.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.697+0000 7fef58e00640 1 Processor -- start 2024-05-28T14:16:55.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.697+0000 7fef58e00640 1 -- start start 2024-05-28T14:16:55.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef58e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef540d0e00 0x7fef5406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef58e00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fef54072d40 0x7fef540aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef58e00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef540ade40 con 0x7fef5415b350 2024-05-28T14:16:55.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef58e00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fef540ae9f0 con 0x7fef54072d40 2024-05-28T14:16:55.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef58e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef540ae0c0 con 0x7fef540d0e00 2024-05-28T14:16:55.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef51e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fef5415b350 0x7fef540d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53124/0 (socket says 172.21.15.42:53124) 2024-05-28T14:16:55.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef51e00640 1 -- 172.21.15.42:0/1517896880 learned_addr learned my addr 172.21.15.42:0/1517896880 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:55.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef52800640 1 --2- 172.21.15.42:0/1517896880 >> v2:172.21.15.63:3300/0 conn(0x7fef54072d40 0x7fef540aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef4be00640 1 --2- 172.21.15.42:0/1517896880 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef540d0e00 0x7fef5406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef51400640 1 -- 172.21.15.42:0/1517896880 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2918394212 0 0) 0x7fef44001120 con 0x7fef5415b350 2024-05-28T14:16:55.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef51400640 1 -- 172.21.15.42:0/1517896880 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef38003530 con 0x7fef5415b350 2024-05-28T14:16:55.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef52800640 1 -- 172.21.15.42:0/1517896880 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef540d0e00 msgr2=0x7fef5406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:55.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef52800640 1 --2- 172.21.15.42:0/1517896880 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef540d0e00 0x7fef5406dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef52800640 1 -- 172.21.15.42:0/1517896880 >> v1:172.21.15.42:6789/0 conn(0x7fef5415b350 legacy=0x7fef540d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.701+0000 7fef52800640 1 -- 172.21.15.42:0/1517896880 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef540ae3a0 con 0x7fef54072d40 2024-05-28T14:16:55.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef52800640 1 --2- 172.21.15.42:0/1517896880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef54072d40 0x7fef540aee80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009d90 tx=0x7fef4c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c406e30431d846 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:55.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef51400640 1 -- 172.21.15.42:0/1517896880 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c0b2070 con 0x7fef54072d40 2024-05-28T14:16:55.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef51400640 1 -- 172.21.15.42:0/1517896880 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef4c0a5820 con 0x7fef54072d40 2024-05-28T14:16:55.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef51400640 1 -- 172.21.15.42:0/1517896880 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c0a5bf0 con 0x7fef54072d40 2024-05-28T14:16:55.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 -- 172.21.15.42:0/1517896880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef54072d40 msgr2=0x7fef540aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 --2- 172.21.15.42:0/1517896880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef54072d40 0x7fef540aee80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009d90 tx=0x7fef4c01c8e0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 -- 172.21.15.42:0/1517896880 shutdown_connections 2024-05-28T14:16:55.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 --2- 172.21.15.42:0/1517896880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef54072d40 0x7fef540aee80 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 --2- 172.21.15.42:0/1517896880 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef540d0e00 0x7fef5406dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 -- 172.21.15.42:0/1517896880 >> 172.21.15.42:0/1517896880 conn(0x7fef5406bdf0 msgr2=0x7fef5406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 -- 172.21.15.42:0/1517896880 shutdown_connections 2024-05-28T14:16:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 -- 172.21.15.42:0/1517896880 wait complete. 2024-05-28T14:16:55.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 Processor -- start 2024-05-28T14:16:55.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 -- start start 2024-05-28T14:16:55.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef54072d40 0x7fef540b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef540d0e00 0x7fef5414f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef5415b350 0x7fef54150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fef54168470 con 0x7fef5415b350 2024-05-28T14:16:55.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef541682f0 con 0x7fef540d0e00 2024-05-28T14:16:55.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.705+0000 7fef58e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef541685f0 con 0x7fef54072d40 2024-05-28T14:16:55.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef52800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef5415b350 0x7fef54150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef52800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef5415b350 0x7fef54150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38070/0 (socket says 172.21.15.42:38070) 2024-05-28T14:16:55.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef52800640 1 -- 172.21.15.42:0/2657613042 learned_addr learned my addr 172.21.15.42:0/2657613042 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:55.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef4be00640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef540d0e00 0x7fef5414f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef51e00640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef54072d40 0x7fef540b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef52800640 1 -- 172.21.15.42:0/2657613042 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef54072d40 msgr2=0x7fef540b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef52800640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef54072d40 0x7fef540b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef52800640 1 -- 172.21.15.42:0/2657613042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef540d0e00 msgr2=0x7fef5414f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef52800640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef540d0e00 0x7fef5414f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef52800640 1 -- 172.21.15.42:0/2657613042 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef540b0830 con 0x7fef5415b350 2024-05-28T14:16:55.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef51e00640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef54072d40 0x7fef540b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:55.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef52800640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef5415b350 0x7fef54150ed0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fef4c01af90 tx=0x7fef4c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:55.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef4a000640 1 -- 172.21.15.42:0/2657613042 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c0b2050 con 0x7fef5415b350 2024-05-28T14:16:55.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef4a000640 1 -- 172.21.15.42:0/2657613042 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef4c0be070 con 0x7fef5415b350 2024-05-28T14:16:55.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef4a000640 1 -- 172.21.15.42:0/2657613042 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c0ad850 con 0x7fef5415b350 2024-05-28T14:16:55.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef58e00640 1 -- 172.21.15.42:0/2657613042 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5415bb70 con 0x7fef5415b350 2024-05-28T14:16:55.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef58e00640 1 -- 172.21.15.42:0/2657613042 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5414fcb0 con 0x7fef5415b350 2024-05-28T14:16:55.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef58e00640 1 -- 172.21.15.42:0/2657613042 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef54068a90 con 0x7fef5415b350 2024-05-28T14:16:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.709+0000 7fef4a000640 1 -- 172.21.15.42:0/2657613042 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7fef4c0ad9f0 con 0x7fef5415b350 2024-05-28T14:16:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.713+0000 7fef4a000640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef3006a420 0x7fef3006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.713+0000 7fef4a000640 1 -- 172.21.15.42:0/2657613042 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fef4c120c50 con 0x7fef5415b350 2024-05-28T14:16:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.713+0000 7fef51e00640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef3006a420 0x7fef3006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.713+0000 7fef4a000640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.63:6800/4268829337,v1:172.21.15.63:6801/4268829337] conn(0x7fef30072740 0x7fef30074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.713+0000 7fef4a000640 1 -- 172.21.15.42:0/2657613042 --> [v2:172.21.15.63:6800/4268829337,v1:172.21.15.63:6801/4268829337] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef30075290 con 0x7fef30072740 2024-05-28T14:16:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.713+0000 7fef4a000640 1 -- 172.21.15.42:0/2657613042 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7fef4c12c050 con 0x7fef5415b350 2024-05-28T14:16:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.713+0000 7fef51e00640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef3006a420 0x7fef3006c8e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fef44005890 tx=0x7fef44009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.713+0000 7fef4be00640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.63:6800/4268829337,v1:172.21.15.63:6801/4268829337] conn(0x7fef30072740 0x7fef30074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.713+0000 7fef4be00640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.63:6800/4268829337,v1:172.21.15.63:6801/4268829337] conn(0x7fef30072740 0x7fef30074bc0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.713+0000 7fef4a000640 1 -- 172.21.15.42:0/2657613042 <== osd.4 v2:172.21.15.63:6800/4268829337 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef30075290 con 0x7fef30072740 2024-05-28T14:16:55.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.729+0000 7fef58e00640 1 -- 172.21.15.42:0/2657613042 --> [v2:172.21.15.63:6800/4268829337,v1:172.21.15.63:6801/4268829337] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fef541660c0 con 0x7fef30072740 2024-05-28T14:16:55.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.729+0000 7fef4a000640 1 -- 172.21.15.42:0/2657613042 <== osd.4 v2:172.21.15.63:6800/4268829337 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fef541660c0 con 0x7fef30072740 2024-05-28T14:16:55.712 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:16:55.713 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-28T14:16:55.713 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:16:55.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.729+0000 7fef58e00640 1 -- 172.21.15.42:0/2657613042 >> [v2:172.21.15.63:6800/4268829337,v1:172.21.15.63:6801/4268829337] conn(0x7fef30072740 msgr2=0x7fef30074bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.729+0000 7fef58e00640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.63:6800/4268829337,v1:172.21.15.63:6801/4268829337] conn(0x7fef30072740 0x7fef30074bc0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.729+0000 7fef58e00640 1 -- 172.21.15.42:0/2657613042 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef3006a420 msgr2=0x7fef3006c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.729+0000 7fef58e00640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fef3006a420 0x7fef3006c8e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fef44005890 tx=0x7fef44009040 comp rx=0 tx=0).stop 2024-05-28T14:16:55.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.729+0000 7fef58e00640 1 -- 172.21.15.42:0/2657613042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef5415b350 msgr2=0x7fef54150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.729+0000 7fef58e00640 1 --2- 172.21.15.42:0/2657613042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef5415b350 0x7fef54150ed0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fef4c01af90 tx=0x7fef4c019680 comp rx=0 tx=0).stop 2024-05-28T14:16:55.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.729+0000 7fef52800640 1 -- 172.21.15.42:0/2657613042 reap_dead start 2024-05-28T14:16:55.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.729+0000 7fef58e00640 1 -- 172.21.15.42:0/2657613042 shutdown_connections 2024-05-28T14:16:55.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.729+0000 7fef58e00640 1 -- 172.21.15.42:0/2657613042 >> 172.21.15.42:0/2657613042 conn(0x7fef5406bdf0 msgr2=0x7fef540b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:55.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.729+0000 7fef58e00640 1 -- 172.21.15.42:0/2657613042 shutdown_connections 2024-05-28T14:16:55.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.729+0000 7fef58e00640 1 -- 172.21.15.42:0/2657613042 wait complete. 2024-05-28T14:16:55.731 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-28T14:16:55.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08b1c00640 1 Processor -- start 2024-05-28T14:16:55.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08b1c00640 1 -- start start 2024-05-28T14:16:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08b1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac15e820 0x7f08ac0b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08b1c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f08ac0ae240 0x7f08ac0ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08b1c00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f08ac0b1640 0x7f08ac0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08b1c00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f08ac06c870 con 0x7f08ac0ae240 2024-05-28T14:16:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08b1c00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f08ac0b6590 con 0x7f08ac0b1640 2024-05-28T14:16:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08b1c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f08ac0b5d30 con 0x7f08ac15e820 2024-05-28T14:16:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08a2000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f08ac0ae240 0x7f08ac0ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08a2000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f08ac0ae240 0x7f08ac0ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38084/0 (socket says 172.21.15.42:38084) 2024-05-28T14:16:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08a2000640 1 -- 172.21.15.42:0/3940991421 learned_addr learned my addr 172.21.15.42:0/3940991421 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08ab400640 1 --2- 172.21.15.42:0/3940991421 >> v2:172.21.15.63:3300/0 conn(0x7f08ac0b1640 0x7f08ac0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08aaa00640 1 --2- 172.21.15.42:0/3940991421 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac15e820 0x7f08ac0b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08a2000640 1 -- 172.21.15.42:0/3940991421 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac15e820 msgr2=0x7f08ac0b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08a2000640 1 --2- 172.21.15.42:0/3940991421 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac15e820 0x7f08ac0b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08a2000640 1 -- 172.21.15.42:0/3940991421 >> v2:172.21.15.63:3300/0 conn(0x7f08ac0b1640 msgr2=0x7f08ac0b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08a2000640 1 --2- 172.21.15.42:0/3940991421 >> v2:172.21.15.63:3300/0 conn(0x7f08ac0b1640 0x7f08ac0b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08a2000640 1 -- 172.21.15.42:0/3940991421 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac15ec00 con 0x7f08ac0ae240 2024-05-28T14:16:55.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.889+0000 7f08aaa00640 1 --2- 172.21.15.42:0/3940991421 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac15e820 0x7f08ac0b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:55.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.893+0000 7f08a2000640 1 --2- 172.21.15.42:0/3940991421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0ae240 0x7f08ac0ae6a0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0d2420 tx=0x7f089401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea61e888da4f1528 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:55.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.893+0000 7f08aa000640 1 -- 172.21.15.42:0/3940991421 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08940b1070 con 0x7f08ac0ae240 2024-05-28T14:16:55.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.893+0000 7f08aa000640 1 -- 172.21.15.42:0/3940991421 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08940ac630 con 0x7f08ac0ae240 2024-05-28T14:16:55.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.893+0000 7f08aa000640 1 -- 172.21.15.42:0/3940991421 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08940a8050 con 0x7f08ac0ae240 2024-05-28T14:16:55.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.893+0000 7f08b1c00640 1 -- 172.21.15.42:0/3940991421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0ae240 msgr2=0x7f08ac0ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.893+0000 7f08b1c00640 1 --2- 172.21.15.42:0/3940991421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0ae240 0x7f08ac0ae6a0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0d2420 tx=0x7f089401c520 comp rx=0 tx=0).stop 2024-05-28T14:16:55.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.893+0000 7f08b1c00640 1 -- 172.21.15.42:0/3940991421 shutdown_connections 2024-05-28T14:16:55.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.893+0000 7f08b1c00640 1 --2- 172.21.15.42:0/3940991421 >> v2:172.21.15.63:3300/0 conn(0x7f08ac0b1640 0x7f08ac0b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.893+0000 7f08b1c00640 1 --2- 172.21.15.42:0/3940991421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0ae240 0x7f08ac0ae6a0 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.893+0000 7f08b1c00640 1 --2- 172.21.15.42:0/3940991421 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac15e820 0x7f08ac0b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.893+0000 7f08b1c00640 1 -- 172.21.15.42:0/3940991421 >> 172.21.15.42:0/3940991421 conn(0x7f08ac0b6780 msgr2=0x7f08ac0b6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:55.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.893+0000 7f08b1c00640 1 -- 172.21.15.42:0/3940991421 shutdown_connections 2024-05-28T14:16:55.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.893+0000 7f08b1c00640 1 -- 172.21.15.42:0/3940991421 wait complete. 2024-05-28T14:16:55.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08b1c00640 1 Processor -- start 2024-05-28T14:16:55.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08b1c00640 1 -- start start 2024-05-28T14:16:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08b1c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0ae240 0x7f08ac14bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08b1c00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac0b1640 0x7f08ac14c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08b1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08ac15e820 0x7f08ac153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08b1c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac0b47a0 con 0x7f08ac0ae240 2024-05-28T14:16:55.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08b1c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac0b26d0 con 0x7f08ac15e820 2024-05-28T14:16:55.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08b1c00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f08ac14d520 con 0x7f08ac0b1640 2024-05-28T14:16:55.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0ae240 0x7f08ac14bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0ae240 0x7f08ac14bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38094/0 (socket says 172.21.15.42:38094) 2024-05-28T14:16:55.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08aaa00640 1 -- 172.21.15.42:0/2503430946 learned_addr learned my addr 172.21.15.42:0/2503430946 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:55.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08a2000640 1 --2- 172.21.15.42:0/2503430946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac0b1640 0x7f08ac14c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08ab400640 1 --2- 172.21.15.42:0/2503430946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08ac15e820 0x7f08ac153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08aaa00640 1 -- 172.21.15.42:0/2503430946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac0b1640 msgr2=0x7f08ac14c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08aaa00640 1 --2- 172.21.15.42:0/2503430946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac0b1640 0x7f08ac14c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08aaa00640 1 -- 172.21.15.42:0/2503430946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08ac15e820 msgr2=0x7f08ac153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:55.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08aaa00640 1 --2- 172.21.15.42:0/2503430946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08ac15e820 0x7f08ac153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:55.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08aaa00640 1 -- 172.21.15.42:0/2503430946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac14cc80 con 0x7f08ac0ae240 2024-05-28T14:16:55.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08aaa00640 1 --2- 172.21.15.42:0/2503430946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0ae240 0x7f08ac14bad0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f089c00c7f0 tx=0x7f089c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:55.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08a3400640 1 -- 172.21.15.42:0/2503430946 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c015070 con 0x7f08ac0ae240 2024-05-28T14:16:55.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08a3400640 1 -- 172.21.15.42:0/2503430946 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f089c007820 con 0x7f08ac0ae240 2024-05-28T14:16:55.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08a3400640 1 -- 172.21.15.42:0/2503430946 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c007c10 con 0x7f08ac0ae240 2024-05-28T14:16:55.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08b1c00640 1 -- 172.21.15.42:0/2503430946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ac14cf60 con 0x7f08ac0ae240 2024-05-28T14:16:55.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08b1c00640 1 -- 172.21.15.42:0/2503430946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ac154960 con 0x7f08ac0ae240 2024-05-28T14:16:55.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.897+0000 7f08b1c00640 1 -- 172.21.15.42:0/2503430946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08ac09f5c0 con 0x7f08ac0ae240 2024-05-28T14:16:55.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.901+0000 7f08a3400640 1 -- 172.21.15.42:0/2503430946 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f089c018070 con 0x7f08ac0ae240 2024-05-28T14:16:55.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.901+0000 7f08a3400640 1 --2- 172.21.15.42:0/2503430946 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f088806a420 0x7f088806c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:55.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.901+0000 7f08a3400640 1 -- 172.21.15.42:0/2503430946 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f089c085c10 con 0x7f08ac0ae240 2024-05-28T14:16:55.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.901+0000 7f08a2000640 1 --2- 172.21.15.42:0/2503430946 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f088806a420 0x7f088806c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:55.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.901+0000 7f08a2000640 1 --2- 172.21.15.42:0/2503430946 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f088806a420 0x7f088806c8e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f08940ae000 tx=0x7f0894009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:55.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:55.905+0000 7f08a3400640 1 -- 172.21.15.42:0/2503430946 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f089c0575e0 con 0x7f08ac0ae240 2024-05-28T14:16:56.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.069+0000 7f08b1c00640 1 -- 172.21.15.42:0/2503430946 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f08ac14c010 con 0x7f088806a420 2024-05-28T14:16:56.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.069+0000 7f08a3400640 1 -- 172.21.15.42:0/2503430946 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f08ac14c010 con 0x7f088806a420 2024-05-28T14:16:56.055 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 4 to deep-scrub 2024-05-28T14:16:56.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.073+0000 7f08b1c00640 1 -- 172.21.15.42:0/2503430946 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f088806a420 msgr2=0x7f088806c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.073+0000 7f08b1c00640 1 --2- 172.21.15.42:0/2503430946 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f088806a420 0x7f088806c8e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f08940ae000 tx=0x7f0894009040 comp rx=0 tx=0).stop 2024-05-28T14:16:56.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.073+0000 7f08b1c00640 1 -- 172.21.15.42:0/2503430946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0ae240 msgr2=0x7f08ac14bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.073+0000 7f08b1c00640 1 --2- 172.21.15.42:0/2503430946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0ae240 0x7f08ac14bad0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f089c00c7f0 tx=0x7f089c00ccb0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.073+0000 7f08b1c00640 1 -- 172.21.15.42:0/2503430946 shutdown_connections 2024-05-28T14:16:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.073+0000 7f08b1c00640 1 --2- 172.21.15.42:0/2503430946 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f088806a420 0x7f088806c8e0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.073+0000 7f08b1c00640 1 --2- 172.21.15.42:0/2503430946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08ac15e820 0x7f08ac153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.073+0000 7f08b1c00640 1 --2- 172.21.15.42:0/2503430946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08ac0b1640 0x7f08ac14c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.073+0000 7f08b1c00640 1 --2- 172.21.15.42:0/2503430946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08ac0ae240 0x7f08ac14bad0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.073+0000 7f08b1c00640 1 -- 172.21.15.42:0/2503430946 >> 172.21.15.42:0/2503430946 conn(0x7f08ac0b6780 msgr2=0x7f08ac15b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:56.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.073+0000 7f08b1c00640 1 -- 172.21.15.42:0/2503430946 shutdown_connections 2024-05-28T14:16:56.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.073+0000 7f08b1c00640 1 -- 172.21.15.42:0/2503430946 wait complete. 2024-05-28T14:16:56.076 INFO:tasks.ceph:Scrubbing osd.5 2024-05-28T14:16:56.076 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-28T14:16:56.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.237+0000 7ff97aa00640 1 Processor -- start 2024-05-28T14:16:56.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.237+0000 7ff97aa00640 1 -- start start 2024-05-28T14:16:56.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.237+0000 7ff97aa00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff97415b350 0x7ff97406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.237+0000 7ff97aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9740d0e00 0x7ff9740aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.237+0000 7ff97aa00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9740ade40 con 0x7ff974072d40 2024-05-28T14:16:56.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.237+0000 7ff97aa00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff9740ae9f0 con 0x7ff97415b350 2024-05-28T14:16:56.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.237+0000 7ff97aa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9740ae0c0 con 0x7ff9740d0e00 2024-05-28T14:16:56.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.237+0000 7ff96b400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff974072d40 0x7ff9740d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53126/0 (socket says 172.21.15.42:53126) 2024-05-28T14:16:56.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.237+0000 7ff96b400640 1 -- 172.21.15.42:0/3791122087 learned_addr learned my addr 172.21.15.42:0/3791122087 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:56.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973e00640 1 --2- 172.21.15.42:0/3791122087 >> v2:172.21.15.63:3300/0 conn(0x7ff97415b350 0x7ff97406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff978a00640 1 --2- 172.21.15.42:0/3791122087 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9740d0e00 0x7ff9740aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 -- 172.21.15.42:0/3791122087 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2387456289 0 0) 0x7ff95c001120 con 0x7ff974072d40 2024-05-28T14:16:56.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 -- 172.21.15.42:0/3791122087 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff958005740 con 0x7ff974072d40 2024-05-28T14:16:56.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 -- 172.21.15.42:0/3791122087 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4259149040 0 0) 0x7ff958005740 con 0x7ff974072d40 2024-05-28T14:16:56.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 -- 172.21.15.42:0/3791122087 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9740d0e00 msgr2=0x7ff9740aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:56.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 --2- 172.21.15.42:0/3791122087 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9740d0e00 0x7ff9740aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 -- 172.21.15.42:0/3791122087 >> v2:172.21.15.63:3300/0 conn(0x7ff97415b350 msgr2=0x7ff97406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:56.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 --2- 172.21.15.42:0/3791122087 >> v2:172.21.15.63:3300/0 conn(0x7ff97415b350 0x7ff97406dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 -- 172.21.15.42:0/3791122087 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9740ae3a0 con 0x7ff974072d40 2024-05-28T14:16:56.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 -- 172.21.15.42:0/3791122087 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7ff95c001850 con 0x7ff974072d40 2024-05-28T14:16:56.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 -- 172.21.15.42:0/3791122087 >> v1:172.21.15.42:6789/0 conn(0x7ff974072d40 legacy=0x7ff9740d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 --2- 172.21.15.42:0/3791122087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff958005bd0 0x7ff958005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 --2- 172.21.15.42:0/3791122087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958006a20 0x7ff958006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 --2- 172.21.15.42:0/3791122087 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9740d0e00 0x7ff9580073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 -- 172.21.15.42:0/3791122087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff9740ae9f0 con 0x7ff958006a20 2024-05-28T14:16:56.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 -- 172.21.15.42:0/3791122087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9740ae0c0 con 0x7ff958005bd0 2024-05-28T14:16:56.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973400640 1 -- 172.21.15.42:0/3791122087 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff95800dc10 con 0x7ff9740d0e00 2024-05-28T14:16:56.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973e00640 1 --2- 172.21.15.42:0/3791122087 >> v2:172.21.15.63:3300/0 conn(0x7ff97415b350 0x7ff97406dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:16:56.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff96b400640 1 --2- 172.21.15.42:0/3791122087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958006a20 0x7ff958006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff978a00640 1 --2- 172.21.15.42:0/3791122087 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9740d0e00 0x7ff9580073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff973e00640 1 --2- 172.21.15.42:0/3791122087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff958005bd0 0x7ff958005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff96b400640 1 -- 172.21.15.42:0/3791122087 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9740d0e00 msgr2=0x7ff9580073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:56.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff96b400640 1 --2- 172.21.15.42:0/3791122087 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9740d0e00 0x7ff9580073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff96b400640 1 -- 172.21.15.42:0/3791122087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff958005bd0 msgr2=0x7ff958005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:56.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff96b400640 1 --2- 172.21.15.42:0/3791122087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff958005bd0 0x7ff958005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.241+0000 7ff96b400640 1 -- 172.21.15.42:0/3791122087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff958005740 con 0x7ff958006a20 2024-05-28T14:16:56.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.245+0000 7ff96b400640 1 --2- 172.21.15.42:0/3791122087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958006a20 0x7ff958006ea0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00c490 tx=0x7ff95c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19284e7ee5d2092b server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:56.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.245+0000 7ff973400640 1 -- 172.21.15.42:0/3791122087 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00c0e0 con 0x7ff958006a20 2024-05-28T14:16:56.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.245+0000 7ff973400640 1 -- 172.21.15.42:0/3791122087 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c0a9020 con 0x7ff958006a20 2024-05-28T14:16:56.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.245+0000 7ff973400640 1 -- 172.21.15.42:0/3791122087 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c005b00 con 0x7ff958006a20 2024-05-28T14:16:56.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.245+0000 7ff97aa00640 1 -- 172.21.15.42:0/3791122087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958006a20 msgr2=0x7ff958006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.245+0000 7ff97aa00640 1 --2- 172.21.15.42:0/3791122087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958006a20 0x7ff958006ea0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00c490 tx=0x7ff95c01cc40 comp rx=0 tx=0).stop 2024-05-28T14:16:56.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.245+0000 7ff978a00640 1 -- 172.21.15.42:0/3791122087 reap_dead start 2024-05-28T14:16:56.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.245+0000 7ff97aa00640 1 -- 172.21.15.42:0/3791122087 shutdown_connections 2024-05-28T14:16:56.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.245+0000 7ff97aa00640 1 -- 172.21.15.42:0/3791122087 >> 172.21.15.42:0/3791122087 conn(0x7ff97406bdf0 msgr2=0x7ff97406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:56.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.245+0000 7ff97aa00640 1 -- 172.21.15.42:0/3791122087 shutdown_connections 2024-05-28T14:16:56.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.245+0000 7ff97aa00640 1 -- 172.21.15.42:0/3791122087 wait complete. 2024-05-28T14:16:56.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.245+0000 7ff97aa00640 1 Processor -- start 2024-05-28T14:16:56.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.245+0000 7ff97aa00640 1 -- start start 2024-05-28T14:16:56.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.245+0000 7ff97aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958006a20 0x7ff974146e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff97aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff974156020 0x7ff97409d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff97aa00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9741575f0 0x7ff974157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff97aa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff9740af7f0 con 0x7ff958006a20 2024-05-28T14:16:56.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff97aa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9740af670 con 0x7ff974156020 2024-05-28T14:16:56.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff97aa00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9740af970 con 0x7ff9741575f0 2024-05-28T14:16:56.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff973e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958006a20 0x7ff974146e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff973e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958006a20 0x7ff974146e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38112/0 (socket says 172.21.15.42:38112) 2024-05-28T14:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff973e00640 1 -- 172.21.15.42:0/1490916200 learned_addr learned my addr 172.21.15.42:0/1490916200 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff978a00640 1 --2- 172.21.15.42:0/1490916200 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9741575f0 0x7ff974157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff96b400640 1 --2- 172.21.15.42:0/1490916200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff974156020 0x7ff97409d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff973e00640 1 -- 172.21.15.42:0/1490916200 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9741575f0 msgr2=0x7ff974157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff973e00640 1 --2- 172.21.15.42:0/1490916200 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9741575f0 0x7ff974157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff973e00640 1 -- 172.21.15.42:0/1490916200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff974156020 msgr2=0x7ff97409d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff973e00640 1 --2- 172.21.15.42:0/1490916200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff974156020 0x7ff97409d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff973e00640 1 -- 172.21.15.42:0/1490916200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff974147350 con 0x7ff958006a20 2024-05-28T14:16:56.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff973e00640 1 --2- 172.21.15.42:0/1490916200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958006a20 0x7ff974146e10 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7ff9640114b0 tx=0x7ff9640118b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:56.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff970c00640 1 -- 172.21.15.42:0/1490916200 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96401a020 con 0x7ff958006a20 2024-05-28T14:16:56.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff970c00640 1 -- 172.21.15.42:0/1490916200 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff964016680 con 0x7ff958006a20 2024-05-28T14:16:56.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff970c00640 1 -- 172.21.15.42:0/1490916200 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964016a30 con 0x7ff958006a20 2024-05-28T14:16:56.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff97aa00640 1 -- 172.21.15.42:0/1490916200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff974156550 con 0x7ff958006a20 2024-05-28T14:16:56.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff97aa00640 1 -- 172.21.15.42:0/1490916200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff974156b10 con 0x7ff958006a20 2024-05-28T14:16:56.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff97aa00640 1 -- 172.21.15.42:0/1490916200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff974068a90 con 0x7ff958006a20 2024-05-28T14:16:56.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.249+0000 7ff970c00640 1 -- 172.21.15.42:0/1490916200 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7ff96401d020 con 0x7ff958006a20 2024-05-28T14:16:56.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.253+0000 7ff970c00640 1 --2- 172.21.15.42:0/1490916200 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff95006a420 0x7ff95006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.253+0000 7ff970c00640 1 -- 172.21.15.42:0/1490916200 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7ff9640869c0 con 0x7ff958006a20 2024-05-28T14:16:56.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.253+0000 7ff96b400640 1 --2- 172.21.15.42:0/1490916200 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff95006a420 0x7ff95006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.253+0000 7ff970c00640 1 --2- 172.21.15.42:0/1490916200 >> [v2:172.21.15.63:6808/1802936259,v1:172.21.15.63:6809/1802936259] conn(0x7ff950072740 0x7ff950074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.253+0000 7ff970c00640 1 -- 172.21.15.42:0/1490916200 --> [v2:172.21.15.63:6808/1802936259,v1:172.21.15.63:6809/1802936259] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff96400be50 con 0x7ff950072740 2024-05-28T14:16:56.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.253+0000 7ff970c00640 1 -- 172.21.15.42:0/1490916200 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7ff964088250 con 0x7ff958006a20 2024-05-28T14:16:56.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.253+0000 7ff978a00640 1 --2- 172.21.15.42:0/1490916200 >> [v2:172.21.15.63:6808/1802936259,v1:172.21.15.63:6809/1802936259] conn(0x7ff950072740 0x7ff950074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.253+0000 7ff96b400640 1 --2- 172.21.15.42:0/1490916200 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff95006a420 0x7ff95006c8e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff974073140 tx=0x7ff95c001c20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:56.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.253+0000 7ff978a00640 1 --2- 172.21.15.42:0/1490916200 >> [v2:172.21.15.63:6808/1802936259,v1:172.21.15.63:6809/1802936259] conn(0x7ff950072740 0x7ff950074bc0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:56.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.253+0000 7ff970c00640 1 -- 172.21.15.42:0/1490916200 <== osd.5 v2:172.21.15.63:6808/1802936259 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff96400be50 con 0x7ff950072740 2024-05-28T14:16:56.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.265+0000 7ff97aa00640 1 -- 172.21.15.42:0/1490916200 --> [v2:172.21.15.63:6808/1802936259,v1:172.21.15.63:6809/1802936259] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff974072b20 con 0x7ff950072740 2024-05-28T14:16:56.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.269+0000 7ff970c00640 1 -- 172.21.15.42:0/1490916200 <== osd.5 v2:172.21.15.63:6808/1802936259 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff974072b20 con 0x7ff950072740 2024-05-28T14:16:56.252 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:16:56.253 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-28T14:16:56.253 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:16:56.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.269+0000 7ff97aa00640 1 -- 172.21.15.42:0/1490916200 >> [v2:172.21.15.63:6808/1802936259,v1:172.21.15.63:6809/1802936259] conn(0x7ff950072740 msgr2=0x7ff950074bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.269+0000 7ff97aa00640 1 --2- 172.21.15.42:0/1490916200 >> [v2:172.21.15.63:6808/1802936259,v1:172.21.15.63:6809/1802936259] conn(0x7ff950072740 0x7ff950074bc0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.269+0000 7ff97aa00640 1 -- 172.21.15.42:0/1490916200 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff95006a420 msgr2=0x7ff95006c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.269+0000 7ff97aa00640 1 --2- 172.21.15.42:0/1490916200 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7ff95006a420 0x7ff95006c8e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff974073140 tx=0x7ff95c001c20 comp rx=0 tx=0).stop 2024-05-28T14:16:56.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.269+0000 7ff97aa00640 1 -- 172.21.15.42:0/1490916200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958006a20 msgr2=0x7ff974146e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.269+0000 7ff97aa00640 1 --2- 172.21.15.42:0/1490916200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958006a20 0x7ff974146e10 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7ff9640114b0 tx=0x7ff9640118b0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.269+0000 7ff978a00640 1 -- 172.21.15.42:0/1490916200 reap_dead start 2024-05-28T14:16:56.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.269+0000 7ff97aa00640 1 -- 172.21.15.42:0/1490916200 shutdown_connections 2024-05-28T14:16:56.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.269+0000 7ff97aa00640 1 -- 172.21.15.42:0/1490916200 >> 172.21.15.42:0/1490916200 conn(0x7ff97406bdf0 msgr2=0x7ff9740b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:56.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.269+0000 7ff97aa00640 1 -- 172.21.15.42:0/1490916200 shutdown_connections 2024-05-28T14:16:56.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.269+0000 7ff97aa00640 1 -- 172.21.15.42:0/1490916200 wait complete. 2024-05-28T14:16:56.271 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-28T14:16:56.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.429+0000 7fa003200640 1 Processor -- start 2024-05-28T14:16:56.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.429+0000 7fa003200640 1 -- start start 2024-05-28T14:16:56.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7fa003200640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9ffc15bf30 0x7f9ffc15c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7fa003200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ffc0cf690 0x7f9ffc06ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7fa003200640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ffc0cfc10 con 0x7f9ffc0cf230 2024-05-28T14:16:56.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7fa003200640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f9ffc0721d0 con 0x7f9ffc15bf30 2024-05-28T14:16:56.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7fa003200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9ffc0cfef0 con 0x7f9ffc0cf690 2024-05-28T14:16:56.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7f9ffbe00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9ffc15bf30 0x7f9ffc15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7f9ff3e00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f9ffc0cf230 0x7f9ffc072660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53140/0 (socket says 172.21.15.42:53140) 2024-05-28T14:16:56.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7f9ff3e00640 1 -- 172.21.15.42:0/2370458251 learned_addr learned my addr 172.21.15.42:0/2370458251 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:56.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7f9ffbe00640 1 -- 172.21.15.42:0/2370458251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ffc0cf690 msgr2=0x7f9ffc06ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:56.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7fa001200640 1 --2- 172.21.15.42:0/2370458251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ffc0cf690 0x7f9ffc06ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7f9ffbe00640 1 --2- 172.21.15.42:0/2370458251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ffc0cf690 0x7f9ffc06ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7f9ffbe00640 1 -- 172.21.15.42:0/2370458251 >> v1:172.21.15.42:6789/0 conn(0x7f9ffc0cf230 legacy=0x7f9ffc072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7f9ffb400640 1 -- 172.21.15.42:0/2370458251 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1997809461 0 0) 0x7f9ffc0cfc10 con 0x7f9ffc0cf230 2024-05-28T14:16:56.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7f9ffbe00640 1 -- 172.21.15.42:0/2370458251 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc0d01d0 con 0x7f9ffc15bf30 2024-05-28T14:16:56.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7fa001200640 1 --2- 172.21.15.42:0/2370458251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ffc0cf690 0x7f9ffc06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:56.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.433+0000 7f9ffbe00640 1 --2- 172.21.15.42:0/2370458251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc15bf30 0x7f9ffc15c310 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0097c0 tx=0x7f9fec01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca23cb073f40a4db server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:56.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7f9ffb400640 1 -- 172.21.15.42:0/2370458251 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0b0070 con 0x7f9ffc15bf30 2024-05-28T14:16:56.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7f9ffb400640 1 -- 172.21.15.42:0/2370458251 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec0ab600 con 0x7f9ffc15bf30 2024-05-28T14:16:56.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7f9ffb400640 1 -- 172.21.15.42:0/2370458251 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0ab9d0 con 0x7f9ffc15bf30 2024-05-28T14:16:56.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 -- 172.21.15.42:0/2370458251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc15bf30 msgr2=0x7f9ffc15c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 --2- 172.21.15.42:0/2370458251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc15bf30 0x7f9ffc15c310 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0097c0 tx=0x7f9fec01c3e0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 -- 172.21.15.42:0/2370458251 shutdown_connections 2024-05-28T14:16:56.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 --2- 172.21.15.42:0/2370458251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ffc0cf690 0x7f9ffc06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 --2- 172.21.15.42:0/2370458251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc15bf30 0x7f9ffc15c310 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 -- 172.21.15.42:0/2370458251 >> 172.21.15.42:0/2370458251 conn(0x7f9ffc06b9f0 msgr2=0x7f9ffc06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 -- 172.21.15.42:0/2370458251 shutdown_connections 2024-05-28T14:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 -- 172.21.15.42:0/2370458251 wait complete. 2024-05-28T14:16:56.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 Processor -- start 2024-05-28T14:16:56.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 -- start start 2024-05-28T14:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc0cf230 0x7f9ffc151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc0cf690 0x7f9ffc1f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ffc15bf30 0x7f9ffc1f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc06dcc0 con 0x7f9ffc0cf690 2024-05-28T14:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc06db40 con 0x7f9ffc0cf230 2024-05-28T14:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa003200640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9ffc06de40 con 0x7f9ffc15bf30 2024-05-28T14:16:56.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7f9ff3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc0cf690 0x7f9ffc1f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7f9ff3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc0cf690 0x7f9ffc1f7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38124/0 (socket says 172.21.15.42:38124) 2024-05-28T14:16:56.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7f9ff3e00640 1 -- 172.21.15.42:0/2593070815 learned_addr learned my addr 172.21.15.42:0/2593070815 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:56.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7fa001200640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ffc15bf30 0x7f9ffc1f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.437+0000 7f9ffbe00640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc0cf230 0x7f9ffc151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7f9ff3e00640 1 -- 172.21.15.42:0/2593070815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ffc15bf30 msgr2=0x7f9ffc1f76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7f9ff3e00640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ffc15bf30 0x7f9ffc1f76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7f9ff3e00640 1 -- 172.21.15.42:0/2593070815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc0cf230 msgr2=0x7f9ffc151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7f9ff3e00640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc0cf230 0x7f9ffc151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7f9ff3e00640 1 -- 172.21.15.42:0/2593070815 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc140850 con 0x7f9ffc0cf690 2024-05-28T14:16:56.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7fa001200640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ffc15bf30 0x7f9ffc1f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:16:56.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7f9ff3e00640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc0cf690 0x7f9ffc1f7190 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400c520 tx=0x7f9fe400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:56.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7f9ff8c00640 1 -- 172.21.15.42:0/2593070815 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4018070 con 0x7f9ffc0cf690 2024-05-28T14:16:56.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7f9ff8c00640 1 -- 172.21.15.42:0/2593070815 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe40074e0 con 0x7f9ffc0cf690 2024-05-28T14:16:56.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7fa003200640 1 -- 172.21.15.42:0/2593070815 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ffc13fa90 con 0x7f9ffc0cf690 2024-05-28T14:16:56.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7f9ff8c00640 1 -- 172.21.15.42:0/2593070815 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40049f0 con 0x7f9ffc0cf690 2024-05-28T14:16:56.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7fa003200640 1 -- 172.21.15.42:0/2593070815 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ffc140050 con 0x7f9ffc0cf690 2024-05-28T14:16:56.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7fa003200640 1 -- 172.21.15.42:0/2593070815 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ffc073890 con 0x7f9ffc0cf690 2024-05-28T14:16:56.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7f9ff8c00640 1 -- 172.21.15.42:0/2593070815 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f9fe401b070 con 0x7f9ffc0cf690 2024-05-28T14:16:56.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7f9ff8c00640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fd806a420 0x7f9fd806c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7f9ff8c00640 1 -- 172.21.15.42:0/2593070815 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f9fe4084e70 con 0x7f9ffc0cf690 2024-05-28T14:16:56.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.441+0000 7f9ffbe00640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fd806a420 0x7f9fd806c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.445+0000 7f9ffbe00640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fd806a420 0x7f9fd806c8e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0ad000 tx=0x7f9fec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:56.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.445+0000 7f9ff8c00640 1 -- 172.21.15.42:0/2593070815 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9fe4056840 con 0x7f9ffc0cf690 2024-05-28T14:16:56.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.613+0000 7fa003200640 1 -- 172.21.15.42:0/2593070815 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f9ffc04fdd0 con 0x7f9fd806a420 2024-05-28T14:16:56.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.613+0000 7f9ff8c00640 1 -- 172.21.15.42:0/2593070815 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9ffc04fdd0 con 0x7f9fd806a420 2024-05-28T14:16:56.599 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 5 to deep-scrub 2024-05-28T14:16:56.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.617+0000 7fa003200640 1 -- 172.21.15.42:0/2593070815 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fd806a420 msgr2=0x7f9fd806c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.617+0000 7fa003200640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fd806a420 0x7f9fd806c8e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0ad000 tx=0x7f9fec008040 comp rx=0 tx=0).stop 2024-05-28T14:16:56.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.617+0000 7fa003200640 1 -- 172.21.15.42:0/2593070815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc0cf690 msgr2=0x7f9ffc1f7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.617+0000 7fa003200640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc0cf690 0x7f9ffc1f7190 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400c520 tx=0x7f9fe400c9e0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.617+0000 7fa003200640 1 -- 172.21.15.42:0/2593070815 shutdown_connections 2024-05-28T14:16:56.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.617+0000 7fa003200640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f9fd806a420 0x7f9fd806c8e0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.617+0000 7fa003200640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ffc15bf30 0x7f9ffc1f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.617+0000 7fa003200640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc0cf690 0x7f9ffc1f7190 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.617+0000 7fa003200640 1 --2- 172.21.15.42:0/2593070815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ffc0cf230 0x7f9ffc151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.617+0000 7fa003200640 1 -- 172.21.15.42:0/2593070815 >> 172.21.15.42:0/2593070815 conn(0x7f9ffc06b9f0 msgr2=0x7f9ffc06ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:56.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.617+0000 7fa003200640 1 -- 172.21.15.42:0/2593070815 shutdown_connections 2024-05-28T14:16:56.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.617+0000 7fa003200640 1 -- 172.21.15.42:0/2593070815 wait complete. 2024-05-28T14:16:56.620 INFO:tasks.ceph:Scrubbing osd.6 2024-05-28T14:16:56.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-28T14:16:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a4f000640 1 Processor -- start 2024-05-28T14:16:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a4f000640 1 -- start start 2024-05-28T14:16:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a4f000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4815eb10 0x7f7a480cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a4f000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7a4806b9f0 0x7f7a4806be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a4f000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a480cf190 con 0x7f7a480b3a50 2024-05-28T14:16:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a4f000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f7a480d05e0 con 0x7f7a4806b9f0 2024-05-28T14:16:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a4f000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a480d0810 con 0x7f7a4815eb10 2024-05-28T14:16:56.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a4d000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7a4806b9f0 0x7f7a4806be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a3f400640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f7a480b3a50 0x7f7a480cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53148/0 (socket says 172.21.15.42:53148) 2024-05-28T14:16:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a3f400640 1 -- 172.21.15.42:0/2080840945 learned_addr learned my addr 172.21.15.42:0/2080840945 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a47e00640 1 --2- 172.21.15.42:0/2080840945 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4815eb10 0x7f7a480cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a4d000640 1 -- 172.21.15.42:0/2080840945 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4815eb10 msgr2=0x7f7a480cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:56.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a4d000640 1 --2- 172.21.15.42:0/2080840945 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4815eb10 0x7f7a480cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a4d000640 1 -- 172.21.15.42:0/2080840945 >> v1:172.21.15.42:6789/0 conn(0x7f7a480b3a50 legacy=0x7f7a480cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a4d000640 1 -- 172.21.15.42:0/2080840945 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a480d0af0 con 0x7f7a4806b9f0 2024-05-28T14:16:56.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.781+0000 7f7a47400640 1 -- 172.21.15.42:0/2080840945 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2285509486 0 0) 0x7f7a480cf190 con 0x7f7a480b3a50 2024-05-28T14:16:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.785+0000 7f7a4d000640 1 --2- 172.21.15.42:0/2080840945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4806b9f0 0x7f7a4806be80 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009d90 tx=0x7f7a4001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d4ab89f941dcf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.785+0000 7f7a47400640 1 -- 172.21.15.42:0/2080840945 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a400b2070 con 0x7f7a4806b9f0 2024-05-28T14:16:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.785+0000 7f7a47400640 1 -- 172.21.15.42:0/2080840945 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a400a57a0 con 0x7f7a4806b9f0 2024-05-28T14:16:56.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.785+0000 7f7a47400640 1 -- 172.21.15.42:0/2080840945 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a400a5b70 con 0x7f7a4806b9f0 2024-05-28T14:16:56.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.785+0000 7f7a4f000640 1 -- 172.21.15.42:0/2080840945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4806b9f0 msgr2=0x7f7a4806be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.785+0000 7f7a4f000640 1 --2- 172.21.15.42:0/2080840945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4806b9f0 0x7f7a4806be80 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009d90 tx=0x7f7a4001c8e0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.785+0000 7f7a4f000640 1 -- 172.21.15.42:0/2080840945 shutdown_connections 2024-05-28T14:16:56.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.785+0000 7f7a4f000640 1 --2- 172.21.15.42:0/2080840945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4806b9f0 0x7f7a4806be80 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.785+0000 7f7a4f000640 1 --2- 172.21.15.42:0/2080840945 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4815eb10 0x7f7a480cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.785+0000 7f7a4f000640 1 -- 172.21.15.42:0/2080840945 >> 172.21.15.42:0/2080840945 conn(0x7f7a480b2f60 msgr2=0x7f7a480b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:56.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.785+0000 7f7a4f000640 1 -- 172.21.15.42:0/2080840945 shutdown_connections 2024-05-28T14:16:56.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.785+0000 7f7a4f000640 1 -- 172.21.15.42:0/2080840945 wait complete. 2024-05-28T14:16:56.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a4f000640 1 Processor -- start 2024-05-28T14:16:56.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a4f000640 1 -- start start 2024-05-28T14:16:56.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a4f000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4806b9f0 0x7f7a4814ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a4f000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a480b3a50 0x7f7a480af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a4f000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4815eb10 0x7f7a48153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a4f000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4816b5b0 con 0x7f7a4806b9f0 2024-05-28T14:16:56.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a4f000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7a480ceb40 con 0x7f7a480b3a50 2024-05-28T14:16:56.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a4f000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4816b730 con 0x7f7a4815eb10 2024-05-28T14:16:56.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4806b9f0 0x7f7a4814ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4806b9f0 0x7f7a4814ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38154/0 (socket says 172.21.15.42:38154) 2024-05-28T14:16:56.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a47e00640 1 -- 172.21.15.42:0/369650171 learned_addr learned my addr 172.21.15.42:0/369650171 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:56.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a4d000640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4815eb10 0x7f7a48153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a3f400640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a480b3a50 0x7f7a480af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a47e00640 1 -- 172.21.15.42:0/369650171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4815eb10 msgr2=0x7f7a48153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a47e00640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a4815eb10 0x7f7a48153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a47e00640 1 -- 172.21.15.42:0/369650171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a480b3a50 msgr2=0x7f7a480af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a47e00640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a480b3a50 0x7f7a480af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a47e00640 1 -- 172.21.15.42:0/369650171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a481509b0 con 0x7f7a4806b9f0 2024-05-28T14:16:56.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a3f400640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a480b3a50 0x7f7a480af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-28T14:16:56.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a47e00640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4806b9f0 0x7f7a4814ff70 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800e870 tx=0x7f7a3800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:56.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a44c00640 1 -- 172.21.15.42:0/369650171 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38017070 con 0x7f7a4806b9f0 2024-05-28T14:16:56.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a44c00640 1 -- 172.21.15.42:0/369650171 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a3800c940 con 0x7f7a4806b9f0 2024-05-28T14:16:56.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a44c00640 1 -- 172.21.15.42:0/369650171 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800ccf0 con 0x7f7a4806b9f0 2024-05-28T14:16:56.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a4f000640 1 -- 172.21.15.42:0/369650171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a48152e70 con 0x7f7a4806b9f0 2024-05-28T14:16:56.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a4f000640 1 -- 172.21.15.42:0/369650171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a48153430 con 0x7f7a4806b9f0 2024-05-28T14:16:56.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.789+0000 7f7a4f000640 1 -- 172.21.15.42:0/369650171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a481504b0 con 0x7f7a4806b9f0 2024-05-28T14:16:56.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.793+0000 7f7a44c00640 1 -- 172.21.15.42:0/369650171 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f7a3801a070 con 0x7f7a4806b9f0 2024-05-28T14:16:56.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.793+0000 7f7a44c00640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a2406a450 0x7f7a2406c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.793+0000 7f7a44c00640 1 -- 172.21.15.42:0/369650171 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f7a380858d0 con 0x7f7a4806b9f0 2024-05-28T14:16:56.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.793+0000 7f7a3f400640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a2406a450 0x7f7a2406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.793+0000 7f7a44c00640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.63:6824/1958395718,v1:172.21.15.63:6825/1958395718] conn(0x7f7a240727d0 0x7f7a24074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.793+0000 7f7a44c00640 1 -- 172.21.15.42:0/369650171 --> [v2:172.21.15.63:6824/1958395718,v1:172.21.15.63:6825/1958395718] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a3800be50 con 0x7f7a240727d0 2024-05-28T14:16:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.793+0000 7f7a44c00640 1 -- 172.21.15.42:0/369650171 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f7a38091050 con 0x7f7a4806b9f0 2024-05-28T14:16:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.793+0000 7f7a4d000640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.63:6824/1958395718,v1:172.21.15.63:6825/1958395718] conn(0x7f7a240727d0 0x7f7a24074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.793+0000 7f7a3f400640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a2406a450 0x7f7a2406c910 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f7a30007fd0 tx=0x7f7a30009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.793+0000 7f7a4d000640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.63:6824/1958395718,v1:172.21.15.63:6825/1958395718] conn(0x7f7a240727d0 0x7f7a24074c50 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.797+0000 7f7a44c00640 1 -- 172.21.15.42:0/369650171 <== osd.6 v2:172.21.15.63:6824/1958395718 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a3800be50 con 0x7f7a240727d0 2024-05-28T14:16:56.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.809+0000 7f7a4f000640 1 -- 172.21.15.42:0/369650171 --> [v2:172.21.15.63:6824/1958395718,v1:172.21.15.63:6825/1958395718] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7a48153b50 con 0x7f7a240727d0 2024-05-28T14:16:56.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.809+0000 7f7a44c00640 1 -- 172.21.15.42:0/369650171 <== osd.6 v2:172.21.15.63:6824/1958395718 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7a48153b50 con 0x7f7a240727d0 2024-05-28T14:16:56.794 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:16:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-28T14:16:56.795 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:16:56.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.809+0000 7f7a4f000640 1 -- 172.21.15.42:0/369650171 >> [v2:172.21.15.63:6824/1958395718,v1:172.21.15.63:6825/1958395718] conn(0x7f7a240727d0 msgr2=0x7f7a24074c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.809+0000 7f7a4f000640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.63:6824/1958395718,v1:172.21.15.63:6825/1958395718] conn(0x7f7a240727d0 0x7f7a24074c50 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.809+0000 7f7a4f000640 1 -- 172.21.15.42:0/369650171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a2406a450 msgr2=0x7f7a2406c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.809+0000 7f7a4f000640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f7a2406a450 0x7f7a2406c910 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f7a30007fd0 tx=0x7f7a30009040 comp rx=0 tx=0).stop 2024-05-28T14:16:56.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.809+0000 7f7a4f000640 1 -- 172.21.15.42:0/369650171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4806b9f0 msgr2=0x7f7a4814ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.809+0000 7f7a4f000640 1 --2- 172.21.15.42:0/369650171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a4806b9f0 0x7f7a4814ff70 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800e870 tx=0x7f7a3800ec70 comp rx=0 tx=0).stop 2024-05-28T14:16:56.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.809+0000 7f7a4d000640 1 -- 172.21.15.42:0/369650171 reap_dead start 2024-05-28T14:16:56.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.809+0000 7f7a4f000640 1 -- 172.21.15.42:0/369650171 shutdown_connections 2024-05-28T14:16:56.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.809+0000 7f7a4f000640 1 -- 172.21.15.42:0/369650171 >> 172.21.15.42:0/369650171 conn(0x7f7a480b2f60 msgr2=0x7f7a480cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:56.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.809+0000 7f7a4f000640 1 -- 172.21.15.42:0/369650171 shutdown_connections 2024-05-28T14:16:56.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.809+0000 7f7a4f000640 1 -- 172.21.15.42:0/369650171 wait complete. 2024-05-28T14:16:56.813 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-28T14:16:56.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb39800640 1 Processor -- start 2024-05-28T14:16:56.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb39800640 1 -- start start 2024-05-28T14:16:56.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb39800640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbb3415e820 0x7fbb340b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb39800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb340ae6a0 0x7fbb3406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb39800640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb340b6590 con 0x7fbb340ae240 2024-05-28T14:16:56.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb39800640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7fbb3406c870 con 0x7fbb3415e820 2024-05-28T14:16:56.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb39800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbb340b5d30 con 0x7fbb340ae6a0 2024-05-28T14:16:56.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb32a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbb3415e820 0x7fbb340b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb32a00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbb3415e820 0x7fbb340b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38168/0 (socket says 172.21.15.42:38168) 2024-05-28T14:16:56.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb32a00640 1 -- 172.21.15.42:0/567636338 learned_addr learned my addr 172.21.15.42:0/567636338 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:56.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb2a000640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fbb340ae240 0x7fbb340b1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:53154/0 (socket says 172.21.15.42:53154) 2024-05-28T14:16:56.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb33400640 1 --2- 172.21.15.42:0/567636338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb340ae6a0 0x7fbb3406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb32a00640 1 -- 172.21.15.42:0/567636338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb340ae6a0 msgr2=0x7fbb3406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:56.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb32a00640 1 --2- 172.21.15.42:0/567636338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb340ae6a0 0x7fbb3406b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb32a00640 1 -- 172.21.15.42:0/567636338 >> v1:172.21.15.42:6789/0 conn(0x7fbb340ae240 legacy=0x7fbb340b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb32a00640 1 -- 172.21.15.42:0/567636338 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3415ec00 con 0x7fbb3415e820 2024-05-28T14:16:56.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.973+0000 7fbb32000640 1 -- 172.21.15.42:0/567636338 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 658168072 0 0) 0x7fbb340b6590 con 0x7fbb340ae240 2024-05-28T14:16:56.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.977+0000 7fbb32a00640 1 --2- 172.21.15.42:0/567636338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb3415e820 0x7fbb340b4260 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009870 tx=0x7fbb2401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce019e26e0691013 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:56.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.977+0000 7fbb32000640 1 -- 172.21.15.42:0/567636338 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb240b2070 con 0x7fbb3415e820 2024-05-28T14:16:56.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.977+0000 7fbb32000640 1 -- 172.21.15.42:0/567636338 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb240a9070 con 0x7fbb3415e820 2024-05-28T14:16:56.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.977+0000 7fbb32000640 1 -- 172.21.15.42:0/567636338 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb240ad880 con 0x7fbb3415e820 2024-05-28T14:16:56.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.977+0000 7fbb39800640 1 -- 172.21.15.42:0/567636338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb3415e820 msgr2=0x7fbb340b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.977+0000 7fbb39800640 1 --2- 172.21.15.42:0/567636338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb3415e820 0x7fbb340b4260 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009870 tx=0x7fbb2401c560 comp rx=0 tx=0).stop 2024-05-28T14:16:56.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.977+0000 7fbb39800640 1 -- 172.21.15.42:0/567636338 shutdown_connections 2024-05-28T14:16:56.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.977+0000 7fbb39800640 1 --2- 172.21.15.42:0/567636338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb340ae6a0 0x7fbb3406b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.977+0000 7fbb39800640 1 --2- 172.21.15.42:0/567636338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb3415e820 0x7fbb340b4260 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.977+0000 7fbb39800640 1 -- 172.21.15.42:0/567636338 >> 172.21.15.42:0/567636338 conn(0x7fbb340b6780 msgr2=0x7fbb340b6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:56.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.977+0000 7fbb39800640 1 -- 172.21.15.42:0/567636338 shutdown_connections 2024-05-28T14:16:56.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.977+0000 7fbb39800640 1 -- 172.21.15.42:0/567636338 wait complete. 2024-05-28T14:16:56.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.977+0000 7fbb39800640 1 Processor -- start 2024-05-28T14:16:56.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb39800640 1 -- start start 2024-05-28T14:16:56.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb39800640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb340ae240 0x7fbb3409c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb39800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb340ae6a0 0x7fbb341527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb39800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb3415e820 0x7fbb34153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb39800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbb340b47a0 con 0x7fbb340ae6a0 2024-05-28T14:16:56.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb39800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3406c340 con 0x7fbb3415e820 2024-05-28T14:16:56.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb39800640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3415efc0 con 0x7fbb340ae240 2024-05-28T14:16:56.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb2a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb340ae6a0 0x7fbb341527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb2a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb340ae6a0 0x7fbb341527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38184/0 (socket says 172.21.15.42:38184) 2024-05-28T14:16:56.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb2a000640 1 -- 172.21.15.42:0/1139168898 learned_addr learned my addr 172.21.15.42:0/1139168898 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:56.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb33400640 1 --2- 172.21.15.42:0/1139168898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb3415e820 0x7fbb34153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb32a00640 1 --2- 172.21.15.42:0/1139168898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb340ae240 0x7fbb3409c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb2a000640 1 -- 172.21.15.42:0/1139168898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb340ae240 msgr2=0x7fbb3409c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb2a000640 1 --2- 172.21.15.42:0/1139168898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb340ae240 0x7fbb3409c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb2a000640 1 -- 172.21.15.42:0/1139168898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb3415e820 msgr2=0x7fbb34153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:56.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb2a000640 1 --2- 172.21.15.42:0/1139168898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb3415e820 0x7fbb34153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:56.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb2a000640 1 -- 172.21.15.42:0/1139168898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3409c760 con 0x7fbb340ae6a0 2024-05-28T14:16:56.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb2a000640 1 --2- 172.21.15.42:0/1139168898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb340ae6a0 0x7fbb341527c0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0011f0 tx=0x7fbb1c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:56.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb2b400640 1 -- 172.21.15.42:0/1139168898 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c018070 con 0x7fbb340ae6a0 2024-05-28T14:16:56.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb2b400640 1 -- 172.21.15.42:0/1139168898 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb1c0074e0 con 0x7fbb340ae6a0 2024-05-28T14:16:56.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb2b400640 1 -- 172.21.15.42:0/1139168898 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c004a10 con 0x7fbb340ae6a0 2024-05-28T14:16:56.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb39800640 1 -- 172.21.15.42:0/1139168898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3409b460 con 0x7fbb340ae6a0 2024-05-28T14:16:56.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb39800640 1 -- 172.21.15.42:0/1139168898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3409ba20 con 0x7fbb340ae6a0 2024-05-28T14:16:56.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.981+0000 7fbb39800640 1 -- 172.21.15.42:0/1139168898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3409f5c0 con 0x7fbb340ae6a0 2024-05-28T14:16:56.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.985+0000 7fbb2b400640 1 -- 172.21.15.42:0/1139168898 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7fbb1c01b070 con 0x7fbb340ae6a0 2024-05-28T14:16:56.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.985+0000 7fbb2b400640 1 --2- 172.21.15.42:0/1139168898 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb1006a450 0x7fbb1006c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:56.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.985+0000 7fbb2b400640 1 -- 172.21.15.42:0/1139168898 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fbb1c084da0 con 0x7fbb340ae6a0 2024-05-28T14:16:56.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.985+0000 7fbb32a00640 1 --2- 172.21.15.42:0/1139168898 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb1006a450 0x7fbb1006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:56.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.985+0000 7fbb32a00640 1 --2- 172.21.15.42:0/1139168898 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb1006a450 0x7fbb1006c910 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbb240af000 tx=0x7fbb240a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:56.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:56.989+0000 7fbb2b400640 1 -- 172.21.15.42:0/1139168898 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbb1c056dc0 con 0x7fbb340ae6a0 2024-05-28T14:16:57.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.189+0000 7fbb39800640 1 -- 172.21.15.42:0/1139168898 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fbb3404fdd0 con 0x7fbb1006a450 2024-05-28T14:16:57.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.189+0000 7fbb2b400640 1 -- 172.21.15.42:0/1139168898 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbb3404fdd0 con 0x7fbb1006a450 2024-05-28T14:16:57.175 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 6 to deep-scrub 2024-05-28T14:16:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.193+0000 7fbb39800640 1 -- 172.21.15.42:0/1139168898 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb1006a450 msgr2=0x7fbb1006c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.193+0000 7fbb39800640 1 --2- 172.21.15.42:0/1139168898 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb1006a450 0x7fbb1006c910 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbb240af000 tx=0x7fbb240a7000 comp rx=0 tx=0).stop 2024-05-28T14:16:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.193+0000 7fbb39800640 1 -- 172.21.15.42:0/1139168898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb340ae6a0 msgr2=0x7fbb341527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.193+0000 7fbb39800640 1 --2- 172.21.15.42:0/1139168898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb340ae6a0 0x7fbb341527c0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0011f0 tx=0x7fbb1c00c9b0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.193+0000 7fbb39800640 1 -- 172.21.15.42:0/1139168898 shutdown_connections 2024-05-28T14:16:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.193+0000 7fbb39800640 1 --2- 172.21.15.42:0/1139168898 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fbb1006a450 0x7fbb1006c910 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.193+0000 7fbb39800640 1 --2- 172.21.15.42:0/1139168898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb3415e820 0x7fbb34153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.193+0000 7fbb39800640 1 --2- 172.21.15.42:0/1139168898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb340ae6a0 0x7fbb341527c0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.193+0000 7fbb39800640 1 --2- 172.21.15.42:0/1139168898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb340ae240 0x7fbb3409c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.193+0000 7fbb39800640 1 -- 172.21.15.42:0/1139168898 >> 172.21.15.42:0/1139168898 conn(0x7fbb340b6780 msgr2=0x7fbb340b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.193+0000 7fbb39800640 1 -- 172.21.15.42:0/1139168898 shutdown_connections 2024-05-28T14:16:57.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.193+0000 7fbb39800640 1 -- 172.21.15.42:0/1139168898 wait complete. 2024-05-28T14:16:57.202 INFO:tasks.ceph:Scrubbing osd.7 2024-05-28T14:16:57.202 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-28T14:16:57.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.369+0000 7f7423600640 1 Processor -- start 2024-05-28T14:16:57.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.369+0000 7f7423600640 1 -- start start 2024-05-28T14:16:57.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.373+0000 7f7423600640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f741c06d150 0x7f741c0d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.373+0000 7f7423600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741c0cd730 0x7f741c0cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.373+0000 7f7423600640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f741c0d1770 con 0x7f741c06cb00 2024-05-28T14:16:57.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.373+0000 7f7423600640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f741c0b1850 con 0x7f741c06d150 2024-05-28T14:16:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.373+0000 7f7423600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f741c0ccc50 con 0x7f741c0cd730 2024-05-28T14:16:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.373+0000 7f7421600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741c0cd730 0x7f741c0cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.373+0000 7f7421600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741c0cd730 0x7f741c0cdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:39098/0 (socket says 172.21.15.42:39098) 2024-05-28T14:16:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.373+0000 7f7421600640 1 -- 172.21.15.42:0/4073619680 learned_addr learned my addr 172.21.15.42:0/4073619680 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.373+0000 7f7420c00640 1 --2- 172.21.15.42:0/4073619680 >> v2:172.21.15.63:3300/0 conn(0x7f741c06d150 0x7f741c0d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.373+0000 7f7421600640 1 -- 172.21.15.42:0/4073619680 >> v2:172.21.15.63:3300/0 conn(0x7f741c06d150 msgr2=0x7f741c0d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.373+0000 7f7421600640 1 --2- 172.21.15.42:0/4073619680 >> v2:172.21.15.63:3300/0 conn(0x7f741c06d150 0x7f741c0d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.373+0000 7f7421600640 1 -- 172.21.15.42:0/4073619680 >> v1:172.21.15.63:6789/0 conn(0x7f741c06cb00 legacy=0x7f741c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.373+0000 7f741b400640 1 -- 172.21.15.42:0/4073619680 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 438864129 0 0) 0x7f741c0d1770 con 0x7f741c06cb00 2024-05-28T14:16:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.373+0000 7f7421600640 1 -- 172.21.15.42:0/4073619680 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741c0ccf30 con 0x7f741c0cd730 2024-05-28T14:16:57.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.377+0000 7f7421600640 1 --2- 172.21.15.42:0/4073619680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741c0cd730 0x7f741c0cdbe0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7414009e80 tx=0x7f741401ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5865e325b0b17c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:57.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.377+0000 7f741b400640 1 -- 172.21.15.42:0/4073619680 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c0ce930 con 0x7f741c0cd730 2024-05-28T14:16:57.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.377+0000 7f741b400640 1 -- 172.21.15.42:0/4073619680 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74140a5760 con 0x7f741c0cd730 2024-05-28T14:16:57.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.377+0000 7f741b400640 1 -- 172.21.15.42:0/4073619680 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74140a5b30 con 0x7f741c0cd730 2024-05-28T14:16:57.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.377+0000 7f7423600640 1 -- 172.21.15.42:0/4073619680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741c0cd730 msgr2=0x7f741c0cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.377+0000 7f7423600640 1 --2- 172.21.15.42:0/4073619680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741c0cd730 0x7f741c0cdbe0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7414009e80 tx=0x7f741401ca10 comp rx=0 tx=0).stop 2024-05-28T14:16:57.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.377+0000 7f7423600640 1 -- 172.21.15.42:0/4073619680 shutdown_connections 2024-05-28T14:16:57.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.377+0000 7f7423600640 1 --2- 172.21.15.42:0/4073619680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741c0cd730 0x7f741c0cdbe0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.377+0000 7f7423600640 1 --2- 172.21.15.42:0/4073619680 >> v2:172.21.15.63:3300/0 conn(0x7f741c06d150 0x7f741c0d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.377+0000 7f7423600640 1 -- 172.21.15.42:0/4073619680 >> 172.21.15.42:0/4073619680 conn(0x7f741c0b3f50 msgr2=0x7f741c0b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:57.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.377+0000 7f7423600640 1 -- 172.21.15.42:0/4073619680 shutdown_connections 2024-05-28T14:16:57.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.377+0000 7f7423600640 1 -- 172.21.15.42:0/4073619680 wait complete. 2024-05-28T14:16:57.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.377+0000 7f7423600640 1 Processor -- start 2024-05-28T14:16:57.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.377+0000 7f7423600640 1 -- start start 2024-05-28T14:16:57.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7423600640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741c06cb00 0x7f741c156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7423600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741c06d150 0x7f741c15b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7423600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f741c0cd730 0x7f741c15b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7423600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f741c16b550 con 0x7f741c06d150 2024-05-28T14:16:57.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7423600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f741c1622d0 con 0x7f741c0cd730 2024-05-28T14:16:57.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7423600640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f741c16b6d0 con 0x7f741c06cb00 2024-05-28T14:16:57.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f741be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741c06d150 0x7f741c15b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f741be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741c06d150 0x7f741c15b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39550/0 (socket says 172.21.15.42:39550) 2024-05-28T14:16:57.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f741be00640 1 -- 172.21.15.42:0/1110658996 learned_addr learned my addr 172.21.15.42:0/1110658996 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:57.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7420c00640 1 --2- 172.21.15.42:0/1110658996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741c06cb00 0x7f741c156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7421600640 1 --2- 172.21.15.42:0/1110658996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f741c0cd730 0x7f741c15b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f741be00640 1 -- 172.21.15.42:0/1110658996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741c06cb00 msgr2=0x7f741c156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f741be00640 1 --2- 172.21.15.42:0/1110658996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f741c06cb00 0x7f741c156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f741be00640 1 -- 172.21.15.42:0/1110658996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f741c0cd730 msgr2=0x7f741c15b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f741be00640 1 --2- 172.21.15.42:0/1110658996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f741c0cd730 0x7f741c15b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f741be00640 1 -- 172.21.15.42:0/1110658996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741c1fb9a0 con 0x7f741c06d150 2024-05-28T14:16:57.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f741be00640 1 --2- 172.21.15.42:0/1110658996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741c06d150 0x7f741c15b4a0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f740800c410 tx=0x7f740800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:57.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7418c00640 1 -- 172.21.15.42:0/1110658996 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408017070 con 0x7f741c06d150 2024-05-28T14:16:57.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7418c00640 1 -- 172.21.15.42:0/1110658996 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7408003600 con 0x7f741c06d150 2024-05-28T14:16:57.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7418c00640 1 -- 172.21.15.42:0/1110658996 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74080039d0 con 0x7f741c06d150 2024-05-28T14:16:57.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7423600640 1 -- 172.21.15.42:0/1110658996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f741c0b0fe0 con 0x7f741c06d150 2024-05-28T14:16:57.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7423600640 1 -- 172.21.15.42:0/1110658996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f741c1faec0 con 0x7f741c06d150 2024-05-28T14:16:57.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7423600640 1 -- 172.21.15.42:0/1110658996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f741c068a90 con 0x7f741c06d150 2024-05-28T14:16:57.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.381+0000 7f7418c00640 1 -- 172.21.15.42:0/1110658996 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f740801a070 con 0x7f741c06d150 2024-05-28T14:16:57.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.385+0000 7f7418c00640 1 --2- 172.21.15.42:0/1110658996 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73fc06a420 0x7f73fc06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.385+0000 7f7418c00640 1 -- 172.21.15.42:0/1110658996 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f7408084de0 con 0x7f741c06d150 2024-05-28T14:16:57.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.385+0000 7f7420c00640 1 --2- 172.21.15.42:0/1110658996 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73fc06a420 0x7f73fc06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.385+0000 7f7418c00640 1 --2- 172.21.15.42:0/1110658996 >> [v2:172.21.15.63:6816/3211394017,v1:172.21.15.63:6817/3211394017] conn(0x7f73fc072740 0x7f73fc074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.385+0000 7f7418c00640 1 -- 172.21.15.42:0/1110658996 --> [v2:172.21.15.63:6816/3211394017,v1:172.21.15.63:6817/3211394017] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73fc075290 con 0x7f73fc072740 2024-05-28T14:16:57.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.385+0000 7f7418c00640 1 -- 172.21.15.42:0/1110658996 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f7408091050 con 0x7f741c06d150 2024-05-28T14:16:57.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.385+0000 7f7420c00640 1 --2- 172.21.15.42:0/1110658996 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73fc06a420 0x7f73fc06c8e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f7410009af0 tx=0x7f7410008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:57.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.385+0000 7f7421600640 1 --2- 172.21.15.42:0/1110658996 >> [v2:172.21.15.63:6816/3211394017,v1:172.21.15.63:6817/3211394017] conn(0x7f73fc072740 0x7f73fc074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.385+0000 7f7421600640 1 --2- 172.21.15.42:0/1110658996 >> [v2:172.21.15.63:6816/3211394017,v1:172.21.15.63:6817/3211394017] conn(0x7f73fc072740 0x7f73fc074bc0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:57.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.385+0000 7f7418c00640 1 -- 172.21.15.42:0/1110658996 <== osd.7 v2:172.21.15.63:6816/3211394017 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73fc075290 con 0x7f73fc072740 2024-05-28T14:16:57.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.401+0000 7f7423600640 1 -- 172.21.15.42:0/1110658996 --> [v2:172.21.15.63:6816/3211394017,v1:172.21.15.63:6817/3211394017] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f741c0b1250 con 0x7f73fc072740 2024-05-28T14:16:57.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.401+0000 7f7418c00640 1 -- 172.21.15.42:0/1110658996 <== osd.7 v2:172.21.15.63:6816/3211394017 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f741c0b1250 con 0x7f73fc072740 2024-05-28T14:16:57.385 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-05-28T14:16:57.385 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-28T14:16:57.385 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-05-28T14:16:57.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.401+0000 7f7423600640 1 -- 172.21.15.42:0/1110658996 >> [v2:172.21.15.63:6816/3211394017,v1:172.21.15.63:6817/3211394017] conn(0x7f73fc072740 msgr2=0x7f73fc074bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.401+0000 7f7423600640 1 --2- 172.21.15.42:0/1110658996 >> [v2:172.21.15.63:6816/3211394017,v1:172.21.15.63:6817/3211394017] conn(0x7f73fc072740 0x7f73fc074bc0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.401+0000 7f7423600640 1 -- 172.21.15.42:0/1110658996 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73fc06a420 msgr2=0x7f73fc06c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.401+0000 7f7423600640 1 --2- 172.21.15.42:0/1110658996 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f73fc06a420 0x7f73fc06c8e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f7410009af0 tx=0x7f7410008040 comp rx=0 tx=0).stop 2024-05-28T14:16:57.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.401+0000 7f7423600640 1 -- 172.21.15.42:0/1110658996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741c06d150 msgr2=0x7f741c15b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.401+0000 7f7423600640 1 --2- 172.21.15.42:0/1110658996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f741c06d150 0x7f741c15b4a0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f740800c410 tx=0x7f740800c8d0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.401+0000 7f7421600640 1 -- 172.21.15.42:0/1110658996 reap_dead start 2024-05-28T14:16:57.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.401+0000 7f7423600640 1 -- 172.21.15.42:0/1110658996 shutdown_connections 2024-05-28T14:16:57.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.401+0000 7f7423600640 1 -- 172.21.15.42:0/1110658996 >> 172.21.15.42:0/1110658996 conn(0x7f741c0b3f50 msgr2=0x7f741c1606d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:57.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.401+0000 7f7423600640 1 -- 172.21.15.42:0/1110658996 shutdown_connections 2024-05-28T14:16:57.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.401+0000 7f7423600640 1 -- 172.21.15.42:0/1110658996 wait complete. 2024-05-28T14:16:57.405 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-28T14:16:57.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.561+0000 7f0555e00640 1 Processor -- start 2024-05-28T14:16:57.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.561+0000 7f0555e00640 1 -- start start 2024-05-28T14:16:57.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.565+0000 7f0555e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0550145920 0x7f055006f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.565+0000 7f0555e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f055006db40 0x7f0550071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.565+0000 7f0555e00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f055006dfa0 con 0x7f0550072300 2024-05-28T14:16:57.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.565+0000 7f0555e00640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f0550072750 con 0x7f055006db40 2024-05-28T14:16:57.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.565+0000 7f0555e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f055006e240 con 0x7f0550145920 2024-05-28T14:16:57.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.565+0000 7f0547e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f055006db40 0x7f0550071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.565+0000 7f0547e00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f055006db40 0x7f0550071dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39562/0 (socket says 172.21.15.42:39562) 2024-05-28T14:16:57.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.565+0000 7f0547e00640 1 -- 172.21.15.42:0/2635995780 learned_addr learned my addr 172.21.15.42:0/2635995780 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:57.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.565+0000 7f054ea00640 1 --2- 172.21.15.42:0/2635995780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0550145920 0x7f055006f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.565+0000 7f0547e00640 1 -- 172.21.15.42:0/2635995780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0550145920 msgr2=0x7f055006f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:16:57.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.565+0000 7f0547e00640 1 --2- 172.21.15.42:0/2635995780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0550145920 0x7f055006f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.565+0000 7f0547e00640 1 -- 172.21.15.42:0/2635995780 >> v1:172.21.15.63:6789/0 conn(0x7f0550072300 legacy=0x7f0550160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.565+0000 7f0547e00640 1 -- 172.21.15.42:0/2635995780 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055006e520 con 0x7f055006db40 2024-05-28T14:16:57.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0547e00640 1 --2- 172.21.15.42:0/2635995780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f055006db40 0x7f0550071dc0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f0538009870 tx=0x7f053801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216894fdac796f58 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:57.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f054e000640 1 -- 172.21.15.42:0/2635995780 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05380b0070 con 0x7f055006db40 2024-05-28T14:16:57.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f054e000640 1 -- 172.21.15.42:0/2635995780 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05380a4e50 con 0x7f055006db40 2024-05-28T14:16:57.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f054e000640 1 -- 172.21.15.42:0/2635995780 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05380ab820 con 0x7f055006db40 2024-05-28T14:16:57.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 -- 172.21.15.42:0/2635995780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f055006db40 msgr2=0x7f0550071dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 --2- 172.21.15.42:0/2635995780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f055006db40 0x7f0550071dc0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f0538009870 tx=0x7f053801c4d0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 -- 172.21.15.42:0/2635995780 shutdown_connections 2024-05-28T14:16:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 --2- 172.21.15.42:0/2635995780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f055006db40 0x7f0550071dc0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 --2- 172.21.15.42:0/2635995780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0550145920 0x7f055006f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 -- 172.21.15.42:0/2635995780 >> 172.21.15.42:0/2635995780 conn(0x7f055006b9f0 msgr2=0x7f055006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 -- 172.21.15.42:0/2635995780 shutdown_connections 2024-05-28T14:16:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 -- 172.21.15.42:0/2635995780 wait complete. 2024-05-28T14:16:57.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 Processor -- start 2024-05-28T14:16:57.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 -- start start 2024-05-28T14:16:57.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f055006db40 0x7f055015bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0550072300 0x7f05500a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0550145920 0x7f05500aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f05501693d0 con 0x7f055006db40 2024-05-28T14:16:57.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0550169250 con 0x7f0550145920 2024-05-28T14:16:57.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.569+0000 7f0555e00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0550169550 con 0x7f0550072300 2024-05-28T14:16:57.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f054ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f055006db40 0x7f055015bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f054ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f055006db40 0x7f055015bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39572/0 (socket says 172.21.15.42:39572) 2024-05-28T14:16:57.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f054ea00640 1 -- 172.21.15.42:0/2400921073 learned_addr learned my addr 172.21.15.42:0/2400921073 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:57.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f0547e00640 1 --2- 172.21.15.42:0/2400921073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0550072300 0x7f05500a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f054f400640 1 --2- 172.21.15.42:0/2400921073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0550145920 0x7f05500aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f054ea00640 1 -- 172.21.15.42:0/2400921073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0550072300 msgr2=0x7f05500a9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f054ea00640 1 --2- 172.21.15.42:0/2400921073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0550072300 0x7f05500a9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f054ea00640 1 -- 172.21.15.42:0/2400921073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0550145920 msgr2=0x7f05500aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f054ea00640 1 --2- 172.21.15.42:0/2400921073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0550145920 0x7f05500aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f054ea00640 1 -- 172.21.15.42:0/2400921073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0550141d10 con 0x7f055006db40 2024-05-28T14:16:57.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f054ea00640 1 --2- 172.21.15.42:0/2400921073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f055006db40 0x7f055015bd60 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f054000d800 tx=0x7f054000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:57.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f0547400640 1 -- 172.21.15.42:0/2400921073 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054000b6c0 con 0x7f055006db40 2024-05-28T14:16:57.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f0547400640 1 -- 172.21.15.42:0/2400921073 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0540019070 con 0x7f055006db40 2024-05-28T14:16:57.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f0555e00640 1 -- 172.21.15.42:0/2400921073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05500aaaf0 con 0x7f055006db40 2024-05-28T14:16:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f0547400640 1 -- 172.21.15.42:0/2400921073 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054000ba60 con 0x7f055006db40 2024-05-28T14:16:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f0555e00640 1 -- 172.21.15.42:0/2400921073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05500ab0b0 con 0x7f055006db40 2024-05-28T14:16:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f0555e00640 1 -- 172.21.15.42:0/2400921073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0550073890 con 0x7f055006db40 2024-05-28T14:16:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.573+0000 7f0547400640 1 -- 172.21.15.42:0/2400921073 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f054000bc00 con 0x7f055006db40 2024-05-28T14:16:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.577+0000 7f0547400640 1 --2- 172.21.15.42:0/2400921073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f052c06a420 0x7f052c06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.577+0000 7f0547400640 1 -- 172.21.15.42:0/2400921073 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f0540085da0 con 0x7f055006db40 2024-05-28T14:16:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.577+0000 7f0547e00640 1 --2- 172.21.15.42:0/2400921073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f052c06a420 0x7f052c06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.577+0000 7f0547e00640 1 --2- 172.21.15.42:0/2400921073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f052c06a420 0x7f052c06c8e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f05380ad000 tx=0x7f0538008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:57.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.581+0000 7f0547400640 1 -- 172.21.15.42:0/2400921073 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0540057810 con 0x7f055006db40 2024-05-28T14:16:57.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.745+0000 7f0555e00640 1 -- 172.21.15.42:0/2400921073 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f05500aff40 con 0x7f052c06a420 2024-05-28T14:16:57.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.745+0000 7f0547400640 1 -- 172.21.15.42:0/2400921073 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f05500aff40 con 0x7f052c06a420 2024-05-28T14:16:57.730 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 7 to deep-scrub 2024-05-28T14:16:57.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.745+0000 7f0555e00640 1 -- 172.21.15.42:0/2400921073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f052c06a420 msgr2=0x7f052c06c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.745+0000 7f0555e00640 1 --2- 172.21.15.42:0/2400921073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f052c06a420 0x7f052c06c8e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f05380ad000 tx=0x7f0538008040 comp rx=0 tx=0).stop 2024-05-28T14:16:57.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.745+0000 7f0555e00640 1 -- 172.21.15.42:0/2400921073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f055006db40 msgr2=0x7f055015bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.745+0000 7f0555e00640 1 --2- 172.21.15.42:0/2400921073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f055006db40 0x7f055015bd60 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f054000d800 tx=0x7f054000dcc0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.745+0000 7f0555e00640 1 -- 172.21.15.42:0/2400921073 shutdown_connections 2024-05-28T14:16:57.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.745+0000 7f0555e00640 1 --2- 172.21.15.42:0/2400921073 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f052c06a420 0x7f052c06c8e0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.745+0000 7f0555e00640 1 --2- 172.21.15.42:0/2400921073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0550145920 0x7f05500aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.745+0000 7f0555e00640 1 --2- 172.21.15.42:0/2400921073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0550072300 0x7f05500a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.745+0000 7f0555e00640 1 --2- 172.21.15.42:0/2400921073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f055006db40 0x7f055015bd60 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.745+0000 7f0555e00640 1 -- 172.21.15.42:0/2400921073 >> 172.21.15.42:0/2400921073 conn(0x7f055006b9f0 msgr2=0x7f0550142d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:57.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.745+0000 7f0555e00640 1 -- 172.21.15.42:0/2400921073 shutdown_connections 2024-05-28T14:16:57.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.745+0000 7f0555e00640 1 -- 172.21.15.42:0/2400921073 wait complete. 2024-05-28T14:16:57.752 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:16:57.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.909+0000 7f00be000640 1 Processor -- start 2024-05-28T14:16:57.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.909+0000 7f00be000640 1 -- start start 2024-05-28T14:16:57.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.913+0000 7f00be000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00b815c990 0x7f00b80d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.913+0000 7f00be000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00b80cd300 con 0x7f00b809bbd0 2024-05-28T14:16:57.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.913+0000 7f00be000640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00b80cd5e0 con 0x7f00b80d06a0 2024-05-28T14:16:57.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.913+0000 7f00be000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f00b815cd70 con 0x7f00b815c990 2024-05-28T14:16:57.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.913+0000 7f00afe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f00b809bbd0 0x7f00b809aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:43368/0 (socket says 172.21.15.42:43368) 2024-05-28T14:16:57.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.913+0000 7f00afe00640 1 -- 172.21.15.42:0/2437849529 learned_addr learned my addr 172.21.15.42:0/2437849529 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:16:57.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.913+0000 7f00b7400640 1 --2- 172.21.15.42:0/2437849529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00b815c990 0x7f00b80d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.913+0000 7f00b6a00640 1 -- 172.21.15.42:0/2437849529 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2667070032 0 0) 0x7f00b80cd300 con 0x7f00b809bbd0 2024-05-28T14:16:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.913+0000 7f00b6a00640 1 -- 172.21.15.42:0/2437849529 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f009c003530 con 0x7f00b809bbd0 2024-05-28T14:16:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.913+0000 7f00b6a00640 1 -- 172.21.15.42:0/2437849529 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1984465699 0 0) 0x7f00b80cd5e0 con 0x7f00b80d06a0 2024-05-28T14:16:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.913+0000 7f00b6a00640 1 -- 172.21.15.42:0/2437849529 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00b80cd300 con 0x7f00b80d06a0 2024-05-28T14:16:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.913+0000 7f00b7400640 1 -- 172.21.15.42:0/2437849529 >> v1:172.21.15.63:6789/0 conn(0x7f00b80d06a0 legacy=0x7f00b80b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.913+0000 7f00b7400640 1 -- 172.21.15.42:0/2437849529 >> v1:172.21.15.42:6789/0 conn(0x7f00b809bbd0 legacy=0x7f00b809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.913+0000 7f00b7400640 1 -- 172.21.15.42:0/2437849529 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b80cd8c0 con 0x7f00b815c990 2024-05-28T14:16:57.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00b7400640 1 --2- 172.21.15.42:0/2437849529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00b815c990 0x7f00b80d0160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f00a80097f0 tx=0x7f00a801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4059b34300aa62f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:57.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00b6a00640 1 -- 172.21.15.42:0/2437849529 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a80aa610 con 0x7f00b815c990 2024-05-28T14:16:57.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00b6a00640 1 -- 172.21.15.42:0/2437849529 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a80af070 con 0x7f00b815c990 2024-05-28T14:16:57.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00b6a00640 1 -- 172.21.15.42:0/2437849529 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a80aa9d0 con 0x7f00b815c990 2024-05-28T14:16:57.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 -- 172.21.15.42:0/2437849529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00b815c990 msgr2=0x7f00b80d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 --2- 172.21.15.42:0/2437849529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00b815c990 0x7f00b80d0160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f00a80097f0 tx=0x7f00a801c4b0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 -- 172.21.15.42:0/2437849529 shutdown_connections 2024-05-28T14:16:57.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 --2- 172.21.15.42:0/2437849529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00b815c990 0x7f00b80d0160 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 -- 172.21.15.42:0/2437849529 >> 172.21.15.42:0/2437849529 conn(0x7f00b80b4080 msgr2=0x7f00b80b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:57.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 -- 172.21.15.42:0/2437849529 shutdown_connections 2024-05-28T14:16:57.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 -- 172.21.15.42:0/2437849529 wait complete. 2024-05-28T14:16:57.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 Processor -- start 2024-05-28T14:16:57.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 -- start start 2024-05-28T14:16:57.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b80d06a0 0x7f00b815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00b815c990 0x7f00b8153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b8150940 0x7f00b81f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f00b80ce3a0 con 0x7f00b80d06a0 2024-05-28T14:16:57.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f00b809b160 con 0x7f00b8150940 2024-05-28T14:16:57.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.917+0000 7f00be000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f00b80c7c10 con 0x7f00b815c990 2024-05-28T14:16:57.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00b7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b80d06a0 0x7f00b815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00b7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b80d06a0 0x7f00b815bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39574/0 (socket says 172.21.15.42:39574) 2024-05-28T14:16:57.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00b7400640 1 -- 172.21.15.42:0/3197549827 learned_addr learned my addr 172.21.15.42:0/3197549827 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:16:57.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00b7e00640 1 --2- 172.21.15.42:0/3197549827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b8150940 0x7f00b81f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00afe00640 1 --2- 172.21.15.42:0/3197549827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00b815c990 0x7f00b8153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00b7400640 1 -- 172.21.15.42:0/3197549827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00b815c990 msgr2=0x7f00b8153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00b7400640 1 --2- 172.21.15.42:0/3197549827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00b815c990 0x7f00b8153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00b7400640 1 -- 172.21.15.42:0/3197549827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b8150940 msgr2=0x7f00b81f8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:57.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00b7400640 1 --2- 172.21.15.42:0/3197549827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b8150940 0x7f00b81f8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:57.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00b7400640 1 -- 172.21.15.42:0/3197549827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b80c7000 con 0x7f00b80d06a0 2024-05-28T14:16:57.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00b7400640 1 --2- 172.21.15.42:0/3197549827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b80d06a0 0x7f00b815bea0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f00a801a0e0 tx=0x7f00a8019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:57.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00af400640 1 -- 172.21.15.42:0/3197549827 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a80aabd0 con 0x7f00b80d06a0 2024-05-28T14:16:57.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00af400640 1 -- 172.21.15.42:0/3197549827 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a80aadb0 con 0x7f00b80d06a0 2024-05-28T14:16:57.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00af400640 1 -- 172.21.15.42:0/3197549827 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a80be690 con 0x7f00b80d06a0 2024-05-28T14:16:57.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00be000640 1 -- 172.21.15.42:0/3197549827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b80c72e0 con 0x7f00b80d06a0 2024-05-28T14:16:57.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00be000640 1 -- 172.21.15.42:0/3197549827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b80c78a0 con 0x7f00b80d06a0 2024-05-28T14:16:57.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00be000640 1 -- 172.21.15.42:0/3197549827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b8068a90 con 0x7f00b80d06a0 2024-05-28T14:16:57.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.921+0000 7f00af400640 1 -- 172.21.15.42:0/3197549827 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 90945+0+0 (secure 0 0 0) 0x7f00a80b33f0 con 0x7f00b80d06a0 2024-05-28T14:16:57.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.925+0000 7f00af400640 1 --2- 172.21.15.42:0/3197549827 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f009406a450 0x7f009406c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:16:57.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.925+0000 7f00af400640 1 -- 172.21.15.42:0/3197549827 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f00a811fc50 con 0x7f00b80d06a0 2024-05-28T14:16:57.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.925+0000 7f00afe00640 1 --2- 172.21.15.42:0/3197549827 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f009406a450 0x7f009406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:16:57.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.925+0000 7f00afe00640 1 --2- 172.21.15.42:0/3197549827 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f009406a450 0x7f009406c910 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f00b80c80c0 tx=0x7f00a0007d80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:16:57.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:57.929+0000 7f00af400640 1 -- 172.21.15.42:0/3197549827 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f00a80a7070 con 0x7f00b80d06a0 2024-05-28T14:16:58.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:58.109+0000 7f00be000640 1 -- 172.21.15.42:0/3197549827 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00b80b4ce0 con 0x7f009406a450 2024-05-28T14:16:58.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:58.117+0000 7f00af400640 1 -- 172.21.15.42:0/3197549827 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254097 (secure 0 0 0) 0x7f00b80b4ce0 con 0x7f009406a450 2024-05-28T14:16:58.104 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:16:58.105 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:16:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:58.121+0000 7f00be000640 1 -- 172.21.15.42:0/3197549827 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f009406a450 msgr2=0x7f009406c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:58.121+0000 7f00be000640 1 --2- 172.21.15.42:0/3197549827 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f009406a450 0x7f009406c910 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f00b80c80c0 tx=0x7f00a0007d80 comp rx=0 tx=0).stop 2024-05-28T14:16:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:58.121+0000 7f00be000640 1 -- 172.21.15.42:0/3197549827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b80d06a0 msgr2=0x7f00b815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:16:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:58.121+0000 7f00be000640 1 --2- 172.21.15.42:0/3197549827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b80d06a0 0x7f00b815bea0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f00a801a0e0 tx=0x7f00a8019250 comp rx=0 tx=0).stop 2024-05-28T14:16:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:58.121+0000 7f00be000640 1 -- 172.21.15.42:0/3197549827 shutdown_connections 2024-05-28T14:16:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:58.121+0000 7f00be000640 1 --2- 172.21.15.42:0/3197549827 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f009406a450 0x7f009406c910 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:58.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:58.121+0000 7f00be000640 1 --2- 172.21.15.42:0/3197549827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00b8150940 0x7f00b81f8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:58.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:58.121+0000 7f00be000640 1 --2- 172.21.15.42:0/3197549827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00b815c990 0x7f00b8153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:58.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:58.121+0000 7f00be000640 1 --2- 172.21.15.42:0/3197549827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b80d06a0 0x7f00b815bea0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:16:58.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:58.121+0000 7f00be000640 1 -- 172.21.15.42:0/3197549827 >> 172.21.15.42:0/3197549827 conn(0x7f00b80b4080 msgr2=0x7f00b8071420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:16:58.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:58.121+0000 7f00be000640 1 -- 172.21.15.42:0/3197549827 shutdown_connections 2024-05-28T14:16:58.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:16:58.121+0000 7f00be000640 1 -- 172.21.15.42:0/3197549827 wait complete. 2024-05-28T14:16:58.131 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":691,"stamp":"2024-05-28T14:16:57.250541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15085096134,"num_objects":3628,"num_object_clones":1,"num_object_copies":7256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3628,"num_whiteouts":1,"num_read":4511,"num_read_kb":17629376,"num_write":7381,"num_write_kb":20681058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":50331770,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7799,"ondisk_log_size":7799,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":30620568,"kb_used_data":29476248,"kb_used_omap":12,"kb_used_meta":1144243,"kb_avail":724354152,"statfs":{"total":773094113280,"available":741738651648,"internally_reserved":0,"allocated":30183677952,"data_stored":30176093396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1171705394},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002766"},"pg_stats":[{"pgid":"19.3f","version":"134'94","reported_seq":188,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.570602+0000","last_change":"2024-05-28T14:14:43.766434+0000","last_active":"2024-05-28T14:16:49.570602+0000","last_peered":"2024-05-28T14:16:49.570602+0000","last_clean":"2024-05-28T14:16:49.570602+0000","last_became_active":"2024-05-28T14:14:43.766125+0000","last_became_peered":"2024-05-28T14:14:43.766125+0000","last_unstale":"2024-05-28T14:16:49.570602+0000","last_undegraded":"2024-05-28T14:16:49.570602+0000","last_fullsized":"2024-05-28T14:16:49.570602+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:50:53.753974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":77,"num_read_kb":315392,"num_write":85,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"134'120","reported_seq":213,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.615579+0000","last_change":"2024-05-28T14:14:43.769925+0000","last_active":"2024-05-28T14:16:49.615579+0000","last_peered":"2024-05-28T14:16:49.615579+0000","last_clean":"2024-05-28T14:16:49.615579+0000","last_became_active":"2024-05-28T14:14:43.769641+0000","last_became_peered":"2024-05-28T14:14:43.769641+0000","last_unstale":"2024-05-28T14:16:49.615579+0000","last_undegraded":"2024-05-28T14:16:49.615579+0000","last_fullsized":"2024-05-28T14:16:49.615579+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":106,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"134'116","reported_seq":203,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.551667+0000","last_change":"2024-05-28T14:14:43.769199+0000","last_active":"2024-05-28T14:16:49.551667+0000","last_peered":"2024-05-28T14:16:49.551667+0000","last_clean":"2024-05-28T14:16:49.551667+0000","last_became_active":"2024-05-28T14:14:43.764098+0000","last_became_peered":"2024-05-28T14:14:43.764098+0000","last_unstale":"2024-05-28T14:16:49.551667+0000","last_undegraded":"2024-05-28T14:16:49.551667+0000","last_fullsized":"2024-05-28T14:16:49.551667+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:57:51.444394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":70,"num_read_kb":278530,"num_write":108,"num_write_kb":323587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"134'122","reported_seq":187,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.562442+0000","last_change":"2024-05-28T14:14:43.759234+0000","last_active":"2024-05-28T14:16:49.562442+0000","last_peered":"2024-05-28T14:16:49.562442+0000","last_clean":"2024-05-28T14:16:49.562442+0000","last_became_active":"2024-05-28T14:14:43.758925+0000","last_became_peered":"2024-05-28T14:14:43.758925+0000","last_unstale":"2024-05-28T14:16:49.562442+0000","last_undegraded":"2024-05-28T14:16:49.562442+0000","last_fullsized":"2024-05-28T14:16:49.562442+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:14.042730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":48,"num_read_kb":184323,"num_write":115,"num_write_kb":339969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"134'79","reported_seq":140,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.538628+0000","last_change":"2024-05-28T14:14:43.766612+0000","last_active":"2024-05-28T14:16:49.538628+0000","last_peered":"2024-05-28T14:16:49.538628+0000","last_clean":"2024-05-28T14:16:49.538628+0000","last_became_active":"2024-05-28T14:14:43.764703+0000","last_became_peered":"2024-05-28T14:14:43.764703+0000","last_unstale":"2024-05-28T14:16:49.538628+0000","last_undegraded":"2024-05-28T14:16:49.538628+0000","last_fullsized":"2024-05-28T14:16:49.538628+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:14:33.925255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":45,"num_read_kb":184320,"num_write":75,"num_write_kb":253952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"134'124","reported_seq":235,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.570848+0000","last_change":"2024-05-28T14:14:43.762958+0000","last_active":"2024-05-28T14:16:49.570848+0000","last_peered":"2024-05-28T14:16:49.570848+0000","last_clean":"2024-05-28T14:16:49.570848+0000","last_became_active":"2024-05-28T14:14:43.762689+0000","last_became_peered":"2024-05-28T14:14:43.762689+0000","last_unstale":"2024-05-28T14:16:49.570848+0000","last_undegraded":"2024-05-28T14:16:49.570848+0000","last_fullsized":"2024-05-28T14:16:49.570848+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:56:11.671806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":95,"num_read_kb":385025,"num_write":115,"num_write_kb":372738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"134'103","reported_seq":169,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.572155+0000","last_change":"2024-05-28T14:14:43.770900+0000","last_active":"2024-05-28T14:16:49.572155+0000","last_peered":"2024-05-28T14:16:49.572155+0000","last_clean":"2024-05-28T14:16:49.572155+0000","last_became_active":"2024-05-28T14:14:43.770616+0000","last_became_peered":"2024-05-28T14:14:43.770616+0000","last_unstale":"2024-05-28T14:16:49.572155+0000","last_undegraded":"2024-05-28T14:16:49.572155+0000","last_fullsized":"2024-05-28T14:16:49.572155+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:07:11.105627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":89,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"134'120","reported_seq":190,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.559090+0000","last_change":"2024-05-28T14:14:43.762262+0000","last_active":"2024-05-28T14:16:49.559090+0000","last_peered":"2024-05-28T14:16:49.559090+0000","last_clean":"2024-05-28T14:16:49.559090+0000","last_became_active":"2024-05-28T14:14:43.761977+0000","last_became_peered":"2024-05-28T14:14:43.761977+0000","last_unstale":"2024-05-28T14:16:49.559090+0000","last_undegraded":"2024-05-28T14:16:49.559090+0000","last_fullsized":"2024-05-28T14:16:49.559090+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:34:54.517890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":54,"num_read_kb":221184,"num_write":113,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"134'110","reported_seq":183,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.564623+0000","last_change":"2024-05-28T14:14:43.770664+0000","last_active":"2024-05-28T14:16:49.564623+0000","last_peered":"2024-05-28T14:16:49.564623+0000","last_clean":"2024-05-28T14:16:49.564623+0000","last_became_active":"2024-05-28T14:14:43.770335+0000","last_became_peered":"2024-05-28T14:14:43.770335+0000","last_unstale":"2024-05-28T14:16:49.564623+0000","last_undegraded":"2024-05-28T14:16:49.564623+0000","last_fullsized":"2024-05-28T14:16:49.564623+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:20:08.770733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":102,"num_write_kb":311297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"134'99","reported_seq":183,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571206+0000","last_change":"2024-05-28T14:14:43.769965+0000","last_active":"2024-05-28T14:16:49.571206+0000","last_peered":"2024-05-28T14:16:49.571206+0000","last_clean":"2024-05-28T14:16:49.571206+0000","last_became_active":"2024-05-28T14:14:43.769687+0000","last_became_peered":"2024-05-28T14:14:43.769687+0000","last_unstale":"2024-05-28T14:16:49.571206+0000","last_undegraded":"2024-05-28T14:16:49.571206+0000","last_fullsized":"2024-05-28T14:16:49.571206+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":67,"num_read_kb":274432,"num_write":93,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"134'112","reported_seq":237,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.559977+0000","last_change":"2024-05-28T14:14:43.758128+0000","last_active":"2024-05-28T14:16:49.559977+0000","last_peered":"2024-05-28T14:16:49.559977+0000","last_clean":"2024-05-28T14:16:49.559977+0000","last_became_active":"2024-05-28T14:14:43.757685+0000","last_became_peered":"2024-05-28T14:14:43.757685+0000","last_unstale":"2024-05-28T14:16:49.559977+0000","last_undegraded":"2024-05-28T14:16:49.559977+0000","last_fullsized":"2024-05-28T14:16:49.559977+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":108,"num_read_kb":442368,"num_write":103,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"134'101","reported_seq":222,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.544613+0000","last_change":"2024-05-28T14:14:43.769235+0000","last_active":"2024-05-28T14:16:49.544613+0000","last_peered":"2024-05-28T14:16:49.544613+0000","last_clean":"2024-05-28T14:16:49.544613+0000","last_became_active":"2024-05-28T14:14:43.768939+0000","last_became_peered":"2024-05-28T14:14:43.768939+0000","last_unstale":"2024-05-28T14:16:49.544613+0000","last_undegraded":"2024-05-28T14:16:49.544613+0000","last_fullsized":"2024-05-28T14:16:49.544613+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:49:26.186099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":105,"num_read_kb":413700,"num_write":98,"num_write_kb":315393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"134'110","reported_seq":208,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.570753+0000","last_change":"2024-05-28T14:14:43.767408+0000","last_active":"2024-05-28T14:16:49.570753+0000","last_peered":"2024-05-28T14:16:49.570753+0000","last_clean":"2024-05-28T14:16:49.570753+0000","last_became_active":"2024-05-28T14:14:43.766269+0000","last_became_peered":"2024-05-28T14:14:43.766269+0000","last_unstale":"2024-05-28T14:16:49.570753+0000","last_undegraded":"2024-05-28T14:16:49.570753+0000","last_fullsized":"2024-05-28T14:16:49.570753+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:45.051114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":105,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"134'100","reported_seq":183,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.561915+0000","last_change":"2024-05-28T14:14:43.762819+0000","last_active":"2024-05-28T14:16:49.561915+0000","last_peered":"2024-05-28T14:16:49.561915+0000","last_clean":"2024-05-28T14:16:49.561915+0000","last_became_active":"2024-05-28T14:14:43.762494+0000","last_became_peered":"2024-05-28T14:14:43.762494+0000","last_unstale":"2024-05-28T14:16:49.561915+0000","last_undegraded":"2024-05-28T14:16:49.561915+0000","last_fullsized":"2024-05-28T14:16:49.561915+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":66,"num_read_kb":270336,"num_write":92,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"134'83","reported_seq":130,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.565126+0000","last_change":"2024-05-28T14:14:43.761245+0000","last_active":"2024-05-28T14:16:49.565126+0000","last_peered":"2024-05-28T14:16:49.565126+0000","last_clean":"2024-05-28T14:16:49.565126+0000","last_became_active":"2024-05-28T14:14:43.760977+0000","last_became_peered":"2024-05-28T14:14:43.760977+0000","last_unstale":"2024-05-28T14:16:49.565126+0000","last_undegraded":"2024-05-28T14:16:49.565126+0000","last_fullsized":"2024-05-28T14:16:49.565126+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:14.494237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":31,"num_read_kb":126976,"num_write":74,"num_write_kb":249856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"134'131","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.556971+0000","last_change":"2024-05-28T14:14:43.758776+0000","last_active":"2024-05-28T14:16:49.556971+0000","last_peered":"2024-05-28T14:16:49.556971+0000","last_clean":"2024-05-28T14:16:49.556971+0000","last_became_active":"2024-05-28T14:14:43.758483+0000","last_became_peered":"2024-05-28T14:14:43.758483+0000","last_unstale":"2024-05-28T14:16:49.556971+0000","last_undegraded":"2024-05-28T14:16:49.556971+0000","last_fullsized":"2024-05-28T14:16:49.556971+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:44:52.410827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":57,"num_read_kb":225282,"num_write":123,"num_write_kb":368641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"134'101","reported_seq":164,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.555651+0000","last_change":"2024-05-28T14:14:43.756230+0000","last_active":"2024-05-28T14:16:49.555651+0000","last_peered":"2024-05-28T14:16:49.555651+0000","last_clean":"2024-05-28T14:16:49.555651+0000","last_became_active":"2024-05-28T14:14:43.755912+0000","last_became_peered":"2024-05-28T14:14:43.755912+0000","last_unstale":"2024-05-28T14:16:49.555651+0000","last_undegraded":"2024-05-28T14:16:49.555651+0000","last_fullsized":"2024-05-28T14:16:49.555651+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:22.709561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":47,"num_read_kb":180227,"num_write":95,"num_write_kb":286721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"134'118","reported_seq":232,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571290+0000","last_change":"2024-05-28T14:14:43.749113+0000","last_active":"2024-05-28T14:16:49.571290+0000","last_peered":"2024-05-28T14:16:49.571290+0000","last_clean":"2024-05-28T14:16:49.571290+0000","last_became_active":"2024-05-28T14:14:43.748801+0000","last_became_peered":"2024-05-28T14:14:43.748801+0000","last_unstale":"2024-05-28T14:16:49.571290+0000","last_undegraded":"2024-05-28T14:16:49.571290+0000","last_fullsized":"2024-05-28T14:16:49.571290+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:05:56.557590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":98,"num_read_kb":389123,"num_write":107,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"134'123","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.559002+0000","last_change":"2024-05-28T14:14:43.769214+0000","last_active":"2024-05-28T14:16:49.559002+0000","last_peered":"2024-05-28T14:16:49.559002+0000","last_clean":"2024-05-28T14:16:49.559002+0000","last_became_active":"2024-05-28T14:14:43.768757+0000","last_became_peered":"2024-05-28T14:14:43.768757+0000","last_unstale":"2024-05-28T14:16:49.559002+0000","last_undegraded":"2024-05-28T14:16:49.559002+0000","last_fullsized":"2024-05-28T14:16:49.559002+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":115,"num_write_kb":372736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"134'113","reported_seq":193,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.560761+0000","last_change":"2024-05-28T14:14:43.756647+0000","last_active":"2024-05-28T14:16:49.560761+0000","last_peered":"2024-05-28T14:16:49.560761+0000","last_clean":"2024-05-28T14:16:49.560761+0000","last_became_active":"2024-05-28T14:14:43.756244+0000","last_became_peered":"2024-05-28T14:14:43.756244+0000","last_unstale":"2024-05-28T14:16:49.560761+0000","last_undegraded":"2024-05-28T14:16:49.560761+0000","last_fullsized":"2024-05-28T14:16:49.560761+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:28:18.841947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":101,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"134'118","reported_seq":234,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.548472+0000","last_change":"2024-05-28T14:14:43.757686+0000","last_active":"2024-05-28T14:16:49.548472+0000","last_peered":"2024-05-28T14:16:49.548472+0000","last_clean":"2024-05-28T14:16:49.548472+0000","last_became_active":"2024-05-28T14:14:43.757288+0000","last_became_peered":"2024-05-28T14:14:43.757288+0000","last_unstale":"2024-05-28T14:16:49.548472+0000","last_undegraded":"2024-05-28T14:16:49.548472+0000","last_fullsized":"2024-05-28T14:16:49.548472+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:55:07.023689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":100,"num_read_kb":389125,"num_write":108,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"134'114","reported_seq":192,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.560999+0000","last_change":"2024-05-28T14:14:43.759851+0000","last_active":"2024-05-28T14:16:49.560999+0000","last_peered":"2024-05-28T14:16:49.560999+0000","last_clean":"2024-05-28T14:16:49.560999+0000","last_became_active":"2024-05-28T14:14:43.759580+0000","last_became_peered":"2024-05-28T14:14:43.759580+0000","last_unstale":"2024-05-28T14:16:49.560999+0000","last_undegraded":"2024-05-28T14:16:49.560999+0000","last_fullsized":"2024-05-28T14:16:49.560999+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":61,"num_read_kb":241666,"num_write":107,"num_write_kb":356353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"134'109","reported_seq":187,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.564532+0000","last_change":"2024-05-28T14:14:43.768741+0000","last_active":"2024-05-28T14:16:49.564532+0000","last_peered":"2024-05-28T14:16:49.564532+0000","last_clean":"2024-05-28T14:16:49.564532+0000","last_became_active":"2024-05-28T14:14:43.767741+0000","last_became_peered":"2024-05-28T14:14:43.767741+0000","last_unstale":"2024-05-28T14:16:49.564532+0000","last_undegraded":"2024-05-28T14:16:49.564532+0000","last_fullsized":"2024-05-28T14:16:49.564532+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:44:20.117904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":62,"num_read_kb":253952,"num_write":101,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"134'111","reported_seq":188,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:53.913222+0000","last_change":"2024-05-28T14:14:43.755177+0000","last_active":"2024-05-28T14:16:53.913222+0000","last_peered":"2024-05-28T14:16:53.913222+0000","last_clean":"2024-05-28T14:16:53.913222+0000","last_became_active":"2024-05-28T14:14:43.753794+0000","last_became_peered":"2024-05-28T14:14:43.753794+0000","last_unstale":"2024-05-28T14:16:53.913222+0000","last_undegraded":"2024-05-28T14:16:53.913222+0000","last_fullsized":"2024-05-28T14:16:53.913222+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":59,"num_read_kb":241664,"num_write":103,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"134'122","reported_seq":208,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:54.453899+0000","last_change":"2024-05-28T14:14:43.763725+0000","last_active":"2024-05-28T14:16:54.453899+0000","last_peered":"2024-05-28T14:16:54.453899+0000","last_clean":"2024-05-28T14:16:54.453899+0000","last_became_active":"2024-05-28T14:14:43.763332+0000","last_became_peered":"2024-05-28T14:14:43.763332+0000","last_unstale":"2024-05-28T14:16:54.453899+0000","last_undegraded":"2024-05-28T14:16:54.453899+0000","last_fullsized":"2024-05-28T14:16:54.453899+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298372,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":68,"num_read_kb":274433,"num_write":112,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"134'121","reported_seq":208,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.550688+0000","last_change":"2024-05-28T14:14:43.759381+0000","last_active":"2024-05-28T14:16:49.550688+0000","last_peered":"2024-05-28T14:16:49.550688+0000","last_clean":"2024-05-28T14:16:49.550688+0000","last_became_active":"2024-05-28T14:14:43.758722+0000","last_became_peered":"2024-05-28T14:14:43.758722+0000","last_unstale":"2024-05-28T14:16:49.550688+0000","last_undegraded":"2024-05-28T14:16:49.550688+0000","last_fullsized":"2024-05-28T14:16:49.550688+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:48:30.643597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251658240,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":112,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"134'104","reported_seq":196,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571413+0000","last_change":"2024-05-28T14:14:43.768220+0000","last_active":"2024-05-28T14:16:49.571413+0000","last_peered":"2024-05-28T14:16:49.571413+0000","last_clean":"2024-05-28T14:16:49.571413+0000","last_became_active":"2024-05-28T14:14:43.767925+0000","last_became_peered":"2024-05-28T14:14:43.767925+0000","last_unstale":"2024-05-28T14:16:49.571413+0000","last_undegraded":"2024-05-28T14:16:49.571413+0000","last_fullsized":"2024-05-28T14:16:49.571413+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:35:31.221160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":75,"num_read_kb":294915,"num_write":99,"num_write_kb":290818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"134'99","reported_seq":150,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.546013+0000","last_change":"2024-05-28T14:14:43.746536+0000","last_active":"2024-05-28T14:16:49.546013+0000","last_peered":"2024-05-28T14:16:49.546013+0000","last_clean":"2024-05-28T14:16:49.546013+0000","last_became_active":"2024-05-28T14:14:43.746249+0000","last_became_peered":"2024-05-28T14:14:43.746249+0000","last_unstale":"2024-05-28T14:16:49.546013+0000","last_undegraded":"2024-05-28T14:16:49.546013+0000","last_fullsized":"2024-05-28T14:16:49.546013+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:30:48.180755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909764,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":34,"num_read_kb":131074,"num_write":94,"num_write_kb":290818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"134'111","reported_seq":200,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.564301+0000","last_change":"2024-05-28T14:14:43.761643+0000","last_active":"2024-05-28T14:16:49.564301+0000","last_peered":"2024-05-28T14:16:49.564301+0000","last_clean":"2024-05-28T14:16:49.564301+0000","last_became_active":"2024-05-28T14:14:43.761161+0000","last_became_peered":"2024-05-28T14:14:43.761161+0000","last_unstale":"2024-05-28T14:16:49.564301+0000","last_undegraded":"2024-05-28T14:16:49.564301+0000","last_fullsized":"2024-05-28T14:16:49.564301+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:41:11.879163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075588,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":72,"num_read_kb":294912,"num_write":104,"num_write_kb":327681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"134'117","reported_seq":203,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.565084+0000","last_change":"2024-05-28T14:14:43.764133+0000","last_active":"2024-05-28T14:16:49.565084+0000","last_peered":"2024-05-28T14:16:49.565084+0000","last_clean":"2024-05-28T14:16:49.565084+0000","last_became_active":"2024-05-28T14:14:43.763522+0000","last_became_peered":"2024-05-28T14:14:43.763522+0000","last_unstale":"2024-05-28T14:16:49.565084+0000","last_undegraded":"2024-05-28T14:16:49.565084+0000","last_fullsized":"2024-05-28T14:16:49.565084+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:05.864969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251658240,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":106,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"134'99","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.562783+0000","last_change":"2024-05-28T14:14:43.763447+0000","last_active":"2024-05-28T14:16:49.562783+0000","last_peered":"2024-05-28T14:16:49.562783+0000","last_clean":"2024-05-28T14:16:49.562783+0000","last_became_active":"2024-05-28T14:14:43.763142+0000","last_became_peered":"2024-05-28T14:14:43.763142+0000","last_unstale":"2024-05-28T14:16:49.562783+0000","last_undegraded":"2024-05-28T14:16:49.562783+0000","last_fullsized":"2024-05-28T14:16:49.562783+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:38:22.872839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":88,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"134'106","reported_seq":198,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.563409+0000","last_change":"2024-05-28T14:14:43.747637+0000","last_active":"2024-05-28T14:16:49.563409+0000","last_peered":"2024-05-28T14:16:49.563409+0000","last_clean":"2024-05-28T14:16:49.563409+0000","last_became_active":"2024-05-28T14:14:43.747272+0000","last_became_peered":"2024-05-28T14:14:43.747272+0000","last_unstale":"2024-05-28T14:16:49.563409+0000","last_undegraded":"2024-05-28T14:16:49.563409+0000","last_fullsized":"2024-05-28T14:16:49.563409+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:10:19.860056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":96,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"134'107","reported_seq":168,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.534735+0000","last_change":"2024-05-28T14:14:43.759397+0000","last_active":"2024-05-28T14:16:49.534735+0000","last_peered":"2024-05-28T14:16:49.534735+0000","last_clean":"2024-05-28T14:16:49.534735+0000","last_became_active":"2024-05-28T14:14:43.759136+0000","last_became_peered":"2024-05-28T14:14:43.759136+0000","last_unstale":"2024-05-28T14:16:49.534735+0000","last_undegraded":"2024-05-28T14:16:49.534735+0000","last_fullsized":"2024-05-28T14:16:49.534735+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:38.538536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":44,"num_read_kb":180224,"num_write":103,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"131'4","reported_seq":23,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973209+0000","last_change":"2024-05-28T14:14:42.751872+0000","last_active":"2024-05-28T14:15:05.973209+0000","last_peered":"2024-05-28T14:15:05.973209+0000","last_clean":"2024-05-28T14:15:05.973209+0000","last_became_active":"2024-05-28T14:14:42.751450+0000","last_became_peered":"2024-05-28T14:14:42.751450+0000","last_unstale":"2024-05-28T14:15:05.973209+0000","last_undegraded":"2024-05-28T14:15:05.973209+0000","last_fullsized":"2024-05-28T14:15:05.973209+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:48:49.635601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"134'121","reported_seq":210,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.564484+0000","last_change":"2024-05-28T14:14:43.766763+0000","last_active":"2024-05-28T14:16:49.564484+0000","last_peered":"2024-05-28T14:16:49.564484+0000","last_clean":"2024-05-28T14:16:49.564484+0000","last_became_active":"2024-05-28T14:14:43.766347+0000","last_became_peered":"2024-05-28T14:14:43.766347+0000","last_unstale":"2024-05-28T14:16:49.564484+0000","last_undegraded":"2024-05-28T14:16:49.564484+0000","last_fullsized":"2024-05-28T14:16:49.564484+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:43:49.334848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":72,"num_read_kb":290817,"num_write":112,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013729+0000","last_change":"2024-05-28T14:14:42.742952+0000","last_active":"2024-05-28T14:15:06.013729+0000","last_peered":"2024-05-28T14:15:06.013729+0000","last_clean":"2024-05-28T14:15:06.013729+0000","last_became_active":"2024-05-28T14:14:42.742668+0000","last_became_peered":"2024-05-28T14:14:42.742668+0000","last_unstale":"2024-05-28T14:15:06.013729+0000","last_undegraded":"2024-05-28T14:15:06.013729+0000","last_fullsized":"2024-05-28T14:15:06.013729+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:53:04.792926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"134'94","reported_seq":185,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.570019+0000","last_change":"2024-05-28T14:14:43.765567+0000","last_active":"2024-05-28T14:16:49.570019+0000","last_peered":"2024-05-28T14:16:49.570019+0000","last_clean":"2024-05-28T14:16:49.570019+0000","last_became_active":"2024-05-28T14:14:43.765230+0000","last_became_peered":"2024-05-28T14:14:43.765230+0000","last_unstale":"2024-05-28T14:16:49.570019+0000","last_undegraded":"2024-05-28T14:16:49.570019+0000","last_fullsized":"2024-05-28T14:16:49.570019+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:56:02.866180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":75,"num_read_kb":303105,"num_write":87,"num_write_kb":270337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"134'3","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:22.801725+0000","last_change":"2024-05-28T14:14:42.733510+0000","last_active":"2024-05-28T14:15:22.801725+0000","last_peered":"2024-05-28T14:15:22.801725+0000","last_clean":"2024-05-28T14:15:22.801725+0000","last_became_active":"2024-05-28T14:14:42.733231+0000","last_became_peered":"2024-05-28T14:14:42.733231+0000","last_unstale":"2024-05-28T14:15:22.801725+0000","last_undegraded":"2024-05-28T14:15:22.801725+0000","last_fullsized":"2024-05-28T14:15:22.801725+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"134'118","reported_seq":221,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.562062+0000","last_change":"2024-05-28T14:14:43.760573+0000","last_active":"2024-05-28T14:16:49.562062+0000","last_peered":"2024-05-28T14:16:49.562062+0000","last_clean":"2024-05-28T14:16:49.562062+0000","last_became_active":"2024-05-28T14:14:43.759307+0000","last_became_peered":"2024-05-28T14:14:43.759307+0000","last_unstale":"2024-05-28T14:16:49.562062+0000","last_undegraded":"2024-05-28T14:16:49.562062+0000","last_fullsized":"2024-05-28T14:16:49.562062+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:30:49.015188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":107,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077896+0000","last_change":"2024-05-28T14:14:42.745411+0000","last_active":"2024-05-28T14:15:06.077896+0000","last_peered":"2024-05-28T14:15:06.077896+0000","last_clean":"2024-05-28T14:15:06.077896+0000","last_became_active":"2024-05-28T14:14:42.745097+0000","last_became_peered":"2024-05-28T14:14:42.745097+0000","last_unstale":"2024-05-28T14:15:06.077896+0000","last_undegraded":"2024-05-28T14:15:06.077896+0000","last_fullsized":"2024-05-28T14:15:06.077896+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"134'112","reported_seq":189,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571629+0000","last_change":"2024-05-28T14:14:43.751125+0000","last_active":"2024-05-28T14:16:49.571629+0000","last_peered":"2024-05-28T14:16:49.571629+0000","last_clean":"2024-05-28T14:16:49.571629+0000","last_became_active":"2024-05-28T14:14:43.747282+0000","last_became_peered":"2024-05-28T14:14:43.747282+0000","last_unstale":"2024-05-28T14:16:49.571629+0000","last_undegraded":"2024-05-28T14:16:49.571629+0000","last_fullsized":"2024-05-28T14:16:49.571629+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:44:03.941731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":105,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:54.568228+0000","last_change":"2024-05-28T14:16:54.568249+0000","last_active":"2024-05-28T14:16:54.568228+0000","last_peered":"2024-05-28T14:16:54.568228+0000","last_clean":"2024-05-28T14:16:54.568228+0000","last_became_active":"2024-05-28T14:14:42.741425+0000","last_became_peered":"2024-05-28T14:14:42.741425+0000","last_unstale":"2024-05-28T14:16:54.568228+0000","last_undegraded":"2024-05-28T14:16:54.568228+0000","last_fullsized":"2024-05-28T14:16:54.568228+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'1","last_scrub_stamp":"2024-05-28T14:16:54.568146+0000","last_deep_scrub":"131'1","last_deep_scrub_stamp":"2024-05-28T14:16:54.568146+0000","last_clean_scrub_stamp":"2024-05-28T14:16:54.568146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:54:37.894706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"134'114","reported_seq":229,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.566591+0000","last_change":"2024-05-28T14:14:43.760960+0000","last_active":"2024-05-28T14:16:49.566591+0000","last_peered":"2024-05-28T14:16:49.566591+0000","last_clean":"2024-05-28T14:16:49.566591+0000","last_became_active":"2024-05-28T14:14:43.760691+0000","last_became_peered":"2024-05-28T14:14:43.760691+0000","last_unstale":"2024-05-28T14:16:49.566591+0000","last_undegraded":"2024-05-28T14:16:49.566591+0000","last_fullsized":"2024-05-28T14:16:49.566591+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:33:40.268549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":99,"num_read_kb":405504,"num_write":102,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969182+0000","last_change":"2024-05-28T14:14:42.739561+0000","last_active":"2024-05-28T14:15:05.969182+0000","last_peered":"2024-05-28T14:15:05.969182+0000","last_clean":"2024-05-28T14:15:05.969182+0000","last_became_active":"2024-05-28T14:14:42.739191+0000","last_became_peered":"2024-05-28T14:14:42.739191+0000","last_unstale":"2024-05-28T14:15:05.969182+0000","last_undegraded":"2024-05-28T14:15:05.969182+0000","last_fullsized":"2024-05-28T14:15:05.969182+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:49:44.727202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"134'109","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.558982+0000","last_change":"2024-05-28T14:14:43.762652+0000","last_active":"2024-05-28T14:16:49.558982+0000","last_peered":"2024-05-28T14:16:49.558982+0000","last_clean":"2024-05-28T14:16:49.558982+0000","last_became_active":"2024-05-28T14:14:43.761547+0000","last_became_peered":"2024-05-28T14:14:43.761547+0000","last_unstale":"2024-05-28T14:16:49.558982+0000","last_undegraded":"2024-05-28T14:16:49.558982+0000","last_fullsized":"2024-05-28T14:16:49.558982+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:11:03.103530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":102,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973841+0000","last_change":"2024-05-28T14:14:42.745986+0000","last_active":"2024-05-28T14:15:05.973841+0000","last_peered":"2024-05-28T14:15:05.973841+0000","last_clean":"2024-05-28T14:15:05.973841+0000","last_became_active":"2024-05-28T14:14:42.745691+0000","last_became_peered":"2024-05-28T14:14:42.745691+0000","last_unstale":"2024-05-28T14:15:05.973841+0000","last_undegraded":"2024-05-28T14:15:05.973841+0000","last_fullsized":"2024-05-28T14:15:05.973841+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:22:09.220099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"134'107","reported_seq":190,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.555140+0000","last_change":"2024-05-28T14:14:43.764538+0000","last_active":"2024-05-28T14:16:49.555140+0000","last_peered":"2024-05-28T14:16:49.555140+0000","last_clean":"2024-05-28T14:16:49.555140+0000","last_became_active":"2024-05-28T14:14:43.762358+0000","last_became_peered":"2024-05-28T14:14:43.762358+0000","last_unstale":"2024-05-28T14:16:49.555140+0000","last_undegraded":"2024-05-28T14:16:49.555140+0000","last_fullsized":"2024-05-28T14:16:49.555140+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":66,"num_read_kb":262146,"num_write":97,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.077620+0000","last_change":"2024-05-28T14:14:42.747379+0000","last_active":"2024-05-28T14:15:06.077620+0000","last_peered":"2024-05-28T14:15:06.077620+0000","last_clean":"2024-05-28T14:15:06.077620+0000","last_became_active":"2024-05-28T14:14:42.747127+0000","last_became_peered":"2024-05-28T14:14:42.747127+0000","last_unstale":"2024-05-28T14:15:06.077620+0000","last_undegraded":"2024-05-28T14:15:06.077620+0000","last_fullsized":"2024-05-28T14:15:06.077620+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"134'96","reported_seq":149,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.561345+0000","last_change":"2024-05-28T14:14:43.757145+0000","last_active":"2024-05-28T14:16:49.561345+0000","last_peered":"2024-05-28T14:16:49.561345+0000","last_clean":"2024-05-28T14:16:49.561345+0000","last_became_active":"2024-05-28T14:14:43.756803+0000","last_became_peered":"2024-05-28T14:14:43.756803+0000","last_unstale":"2024-05-28T14:16:49.561345+0000","last_undegraded":"2024-05-28T14:16:49.561345+0000","last_fullsized":"2024-05-28T14:16:49.561345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:30:48.180755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188743680,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":85,"num_write_kb":266240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"134'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013914+0000","last_change":"2024-05-28T14:14:42.746521+0000","last_active":"2024-05-28T14:15:06.013914+0000","last_peered":"2024-05-28T14:15:06.013914+0000","last_clean":"2024-05-28T14:15:06.013914+0000","last_became_active":"2024-05-28T14:14:42.746243+0000","last_became_peered":"2024-05-28T14:14:42.746243+0000","last_unstale":"2024-05-28T14:15:06.013914+0000","last_undegraded":"2024-05-28T14:15:06.013914+0000","last_fullsized":"2024-05-28T14:15:06.013914+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:01:44.045231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"134'96","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.543616+0000","last_change":"2024-05-28T14:14:43.751232+0000","last_active":"2024-05-28T14:16:49.543616+0000","last_peered":"2024-05-28T14:16:49.543616+0000","last_clean":"2024-05-28T14:16:49.543616+0000","last_became_active":"2024-05-28T14:14:43.750768+0000","last_became_peered":"2024-05-28T14:14:43.750768+0000","last_unstale":"2024-05-28T14:16:49.543616+0000","last_undegraded":"2024-05-28T14:16:49.543616+0000","last_fullsized":"2024-05-28T14:16:49.543616+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:37:33.489101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":41,"num_read_kb":167936,"num_write":89,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969503+0000","last_change":"2024-05-28T14:14:42.745263+0000","last_active":"2024-05-28T14:15:05.969503+0000","last_peered":"2024-05-28T14:15:05.969503+0000","last_clean":"2024-05-28T14:15:05.969503+0000","last_became_active":"2024-05-28T14:14:42.744565+0000","last_became_peered":"2024-05-28T14:14:42.744565+0000","last_unstale":"2024-05-28T14:15:05.969503+0000","last_undegraded":"2024-05-28T14:15:05.969503+0000","last_fullsized":"2024-05-28T14:15:05.969503+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:59.384784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"134'85","reported_seq":184,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.543513+0000","last_change":"2024-05-28T14:14:43.760442+0000","last_active":"2024-05-28T14:16:49.543513+0000","last_peered":"2024-05-28T14:16:49.543513+0000","last_clean":"2024-05-28T14:16:49.543513+0000","last_became_active":"2024-05-28T14:14:43.760106+0000","last_became_peered":"2024-05-28T14:14:43.760106+0000","last_unstale":"2024-05-28T14:16:49.543513+0000","last_undegraded":"2024-05-28T14:16:49.543513+0000","last_fullsized":"2024-05-28T14:16:49.543513+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:56:47.127973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":80,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"134'25","reported_seq":43,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.001817+0000","last_change":"2024-05-28T14:14:42.735627+0000","last_active":"2024-05-28T14:16:27.001817+0000","last_peered":"2024-05-28T14:16:27.001817+0000","last_clean":"2024-05-28T14:16:27.001817+0000","last_became_active":"2024-05-28T14:14:42.734095+0000","last_became_peered":"2024-05-28T14:14:42.734095+0000","last_unstale":"2024-05-28T14:16:27.001817+0000","last_undegraded":"2024-05-28T14:16:27.001817+0000","last_fullsized":"2024-05-28T14:16:27.001817+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:24:07.310371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"134'95","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.555670+0000","last_change":"2024-05-28T14:14:43.768144+0000","last_active":"2024-05-28T14:16:49.555670+0000","last_peered":"2024-05-28T14:16:49.555670+0000","last_clean":"2024-05-28T14:16:49.555670+0000","last_became_active":"2024-05-28T14:14:43.766334+0000","last_became_peered":"2024-05-28T14:14:43.766334+0000","last_unstale":"2024-05-28T14:16:49.555670+0000","last_undegraded":"2024-05-28T14:16:49.555670+0000","last_fullsized":"2024-05-28T14:16:49.555670+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:56:51.599701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":86,"num_write_kb":274434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"134'460","reported_seq":478,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.508860+0000","last_change":"2024-05-28T14:14:42.744568+0000","last_active":"2024-05-28T14:16:49.508860+0000","last_peered":"2024-05-28T14:16:49.508860+0000","last_clean":"2024-05-28T14:16:49.508860+0000","last_became_active":"2024-05-28T14:14:42.744240+0000","last_became_peered":"2024-05-28T14:14:42.744240+0000","last_unstale":"2024-05-28T14:16:49.508860+0000","last_undegraded":"2024-05-28T14:16:49.508860+0000","last_fullsized":"2024-05-28T14:16:49.508860+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:25:02.568973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1781008,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":458,"num_write_kb":1902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"134'125","reported_seq":195,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.556970+0000","last_change":"2024-05-28T14:14:43.758776+0000","last_active":"2024-05-28T14:16:49.556970+0000","last_peered":"2024-05-28T14:16:49.556970+0000","last_clean":"2024-05-28T14:16:49.556970+0000","last_became_active":"2024-05-28T14:14:43.758474+0000","last_became_peered":"2024-05-28T14:14:43.758474+0000","last_unstale":"2024-05-28T14:16:49.556970+0000","last_undegraded":"2024-05-28T14:16:49.556970+0000","last_fullsized":"2024-05-28T14:16:49.556970+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:00:34.527970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":54,"num_read_kb":212994,"num_write":119,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973982+0000","last_change":"2024-05-28T14:14:42.737773+0000","last_active":"2024-05-28T14:15:05.973982+0000","last_peered":"2024-05-28T14:15:05.973982+0000","last_clean":"2024-05-28T14:15:05.973982+0000","last_became_active":"2024-05-28T14:14:42.737442+0000","last_became_peered":"2024-05-28T14:14:42.737442+0000","last_unstale":"2024-05-28T14:15:05.973982+0000","last_undegraded":"2024-05-28T14:15:05.973982+0000","last_fullsized":"2024-05-28T14:15:05.973982+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:45:49.229094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"134'110","reported_seq":188,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.561098+0000","last_change":"2024-05-28T14:14:43.762668+0000","last_active":"2024-05-28T14:16:49.561098+0000","last_peered":"2024-05-28T14:16:49.561098+0000","last_clean":"2024-05-28T14:16:49.561098+0000","last_became_active":"2024-05-28T14:14:43.762049+0000","last_became_peered":"2024-05-28T14:14:43.762049+0000","last_unstale":"2024-05-28T14:16:49.561098+0000","last_undegraded":"2024-05-28T14:16:49.561098+0000","last_fullsized":"2024-05-28T14:16:49.561098+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":106,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"134'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:27.574800+0000","last_change":"2024-05-28T14:14:42.746077+0000","last_active":"2024-05-28T14:16:27.574800+0000","last_peered":"2024-05-28T14:16:27.574800+0000","last_clean":"2024-05-28T14:16:27.574800+0000","last_became_active":"2024-05-28T14:14:42.745811+0000","last_became_peered":"2024-05-28T14:14:42.745811+0000","last_unstale":"2024-05-28T14:16:27.574800+0000","last_undegraded":"2024-05-28T14:16:27.574800+0000","last_fullsized":"2024-05-28T14:16:27.574800+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"134'123","reported_seq":211,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.556596+0000","last_change":"2024-05-28T14:14:43.768732+0000","last_active":"2024-05-28T14:16:49.556596+0000","last_peered":"2024-05-28T14:16:49.556596+0000","last_clean":"2024-05-28T14:16:49.556596+0000","last_became_active":"2024-05-28T14:14:43.768334+0000","last_became_peered":"2024-05-28T14:14:43.768334+0000","last_unstale":"2024-05-28T14:16:49.556596+0000","last_undegraded":"2024-05-28T14:16:49.556596+0000","last_fullsized":"2024-05-28T14:16:49.556596+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:01:12.201409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852724,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":72,"num_read_kb":282627,"num_write":118,"num_write_kb":360450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"131'8","reported_seq":26,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.975292+0000","last_change":"2024-05-28T14:14:42.742861+0000","last_active":"2024-05-28T14:15:05.975292+0000","last_peered":"2024-05-28T14:15:05.975292+0000","last_clean":"2024-05-28T14:15:05.975292+0000","last_became_active":"2024-05-28T14:14:42.742518+0000","last_became_peered":"2024-05-28T14:14:42.742518+0000","last_unstale":"2024-05-28T14:15:05.975292+0000","last_undegraded":"2024-05-28T14:15:05.975292+0000","last_fullsized":"2024-05-28T14:15:05.975292+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:43:29.312316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"134'90","reported_seq":131,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.561380+0000","last_change":"2024-05-28T14:14:43.762081+0000","last_active":"2024-05-28T14:16:49.561380+0000","last_peered":"2024-05-28T14:16:49.561380+0000","last_clean":"2024-05-28T14:16:49.561380+0000","last_became_active":"2024-05-28T14:14:43.761775+0000","last_became_peered":"2024-05-28T14:14:43.761775+0000","last_unstale":"2024-05-28T14:16:49.561380+0000","last_undegraded":"2024-05-28T14:16:49.561380+0000","last_fullsized":"2024-05-28T14:16:49.561380+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:07:11.105627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176160768,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":25,"num_read_kb":102400,"num_write":86,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013893+0000","last_change":"2024-05-28T14:14:42.746381+0000","last_active":"2024-05-28T14:15:06.013893+0000","last_peered":"2024-05-28T14:15:06.013893+0000","last_clean":"2024-05-28T14:15:06.013893+0000","last_became_active":"2024-05-28T14:14:42.746098+0000","last_became_peered":"2024-05-28T14:14:42.746098+0000","last_unstale":"2024-05-28T14:15:06.013893+0000","last_undegraded":"2024-05-28T14:15:06.013893+0000","last_fullsized":"2024-05-28T14:15:06.013893+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:09:04.859086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"134'135","reported_seq":233,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.546980+0000","last_change":"2024-05-28T14:14:43.756453+0000","last_active":"2024-05-28T14:16:49.546980+0000","last_peered":"2024-05-28T14:16:49.546980+0000","last_clean":"2024-05-28T14:16:49.546980+0000","last_became_active":"2024-05-28T14:14:43.756138+0000","last_became_peered":"2024-05-28T14:14:43.756138+0000","last_unstale":"2024-05-28T14:16:49.546980+0000","last_undegraded":"2024-05-28T14:16:49.546980+0000","last_fullsized":"2024-05-28T14:16:49.546980+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:07.658669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":129,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.971819+0000","last_change":"2024-05-28T14:14:42.742055+0000","last_active":"2024-05-28T14:15:05.971819+0000","last_peered":"2024-05-28T14:15:05.971819+0000","last_clean":"2024-05-28T14:15:05.971819+0000","last_became_active":"2024-05-28T14:14:42.741754+0000","last_became_peered":"2024-05-28T14:14:42.741754+0000","last_unstale":"2024-05-28T14:15:05.971819+0000","last_undegraded":"2024-05-28T14:15:05.971819+0000","last_fullsized":"2024-05-28T14:15:05.971819+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"134'111","reported_seq":190,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.560349+0000","last_change":"2024-05-28T14:14:43.758320+0000","last_active":"2024-05-28T14:16:49.560349+0000","last_peered":"2024-05-28T14:16:49.560349+0000","last_clean":"2024-05-28T14:16:49.560349+0000","last_became_active":"2024-05-28T14:14:43.757996+0000","last_became_peered":"2024-05-28T14:14:43.757996+0000","last_unstale":"2024-05-28T14:16:49.560349+0000","last_undegraded":"2024-05-28T14:16:49.560349+0000","last_fullsized":"2024-05-28T14:16:49.560349+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:18.151246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":63,"num_read_kb":241668,"num_write":101,"num_write_kb":299010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"131'1","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.974449+0000","last_change":"2024-05-28T14:14:42.734922+0000","last_active":"2024-05-28T14:15:05.974449+0000","last_peered":"2024-05-28T14:15:05.974449+0000","last_clean":"2024-05-28T14:15:05.974449+0000","last_became_active":"2024-05-28T14:14:42.734647+0000","last_became_peered":"2024-05-28T14:14:42.734647+0000","last_unstale":"2024-05-28T14:15:05.974449+0000","last_undegraded":"2024-05-28T14:15:05.974449+0000","last_fullsized":"2024-05-28T14:15:05.974449+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:36:34.326176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"134'112","reported_seq":211,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.557134+0000","last_change":"2024-05-28T14:14:43.764186+0000","last_active":"2024-05-28T14:16:49.557134+0000","last_peered":"2024-05-28T14:16:49.557134+0000","last_clean":"2024-05-28T14:16:49.557134+0000","last_became_active":"2024-05-28T14:14:43.762407+0000","last_became_peered":"2024-05-28T14:14:43.762407+0000","last_unstale":"2024-05-28T14:16:49.557134+0000","last_undegraded":"2024-05-28T14:16:49.557134+0000","last_fullsized":"2024-05-28T14:16:49.557134+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":82,"num_read_kb":327682,"num_write":107,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013781+0000","last_change":"2024-05-28T14:14:42.733737+0000","last_active":"2024-05-28T14:15:06.013781+0000","last_peered":"2024-05-28T14:15:06.013781+0000","last_clean":"2024-05-28T14:15:06.013781+0000","last_became_active":"2024-05-28T14:14:42.733393+0000","last_became_peered":"2024-05-28T14:14:42.733393+0000","last_unstale":"2024-05-28T14:15:06.013781+0000","last_undegraded":"2024-05-28T14:15:06.013781+0000","last_fullsized":"2024-05-28T14:15:06.013781+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:28:13.036848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"134'107","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571637+0000","last_change":"2024-05-28T14:14:43.767748+0000","last_active":"2024-05-28T14:16:49.571637+0000","last_peered":"2024-05-28T14:16:49.571637+0000","last_clean":"2024-05-28T14:16:49.571637+0000","last_became_active":"2024-05-28T14:14:43.767351+0000","last_became_peered":"2024-05-28T14:14:43.767351+0000","last_unstale":"2024-05-28T14:16:49.571637+0000","last_undegraded":"2024-05-28T14:16:49.571637+0000","last_fullsized":"2024-05-28T14:16:49.571637+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:27:11.131902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":85,"num_read_kb":348160,"num_write":103,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"134'3","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:06.013626+0000","last_change":"2024-05-28T14:14:42.746787+0000","last_active":"2024-05-28T14:15:06.013626+0000","last_peered":"2024-05-28T14:15:06.013626+0000","last_clean":"2024-05-28T14:15:06.013626+0000","last_became_active":"2024-05-28T14:14:42.746273+0000","last_became_peered":"2024-05-28T14:14:42.746273+0000","last_unstale":"2024-05-28T14:15:06.013626+0000","last_undegraded":"2024-05-28T14:15:06.013626+0000","last_fullsized":"2024-05-28T14:15:06.013626+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:33:38.279660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"134'119","reported_seq":202,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571021+0000","last_change":"2024-05-28T14:14:43.771048+0000","last_active":"2024-05-28T14:16:49.571021+0000","last_peered":"2024-05-28T14:16:49.571021+0000","last_clean":"2024-05-28T14:16:49.571021+0000","last_became_active":"2024-05-28T14:14:43.770751+0000","last_became_peered":"2024-05-28T14:14:43.770751+0000","last_unstale":"2024-05-28T14:16:49.571021+0000","last_undegraded":"2024-05-28T14:16:49.571021+0000","last_fullsized":"2024-05-28T14:16:49.571021+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:34:09.617785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272629760,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":111,"num_write_kb":348163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969290+0000","last_change":"2024-05-28T14:14:42.742397+0000","last_active":"2024-05-28T14:15:05.969290+0000","last_peered":"2024-05-28T14:15:05.969290+0000","last_clean":"2024-05-28T14:15:05.969290+0000","last_became_active":"2024-05-28T14:14:42.742104+0000","last_became_peered":"2024-05-28T14:14:42.742104+0000","last_unstale":"2024-05-28T14:15:05.969290+0000","last_undegraded":"2024-05-28T14:15:05.969290+0000","last_fullsized":"2024-05-28T14:15:05.969290+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:32:21.703678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"134'122","reported_seq":189,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:54.453780+0000","last_change":"2024-05-28T14:14:43.756230+0000","last_active":"2024-05-28T14:16:54.453780+0000","last_peered":"2024-05-28T14:16:54.453780+0000","last_clean":"2024-05-28T14:16:54.453780+0000","last_became_active":"2024-05-28T14:14:43.755945+0000","last_became_peered":"2024-05-28T14:14:43.755945+0000","last_unstale":"2024-05-28T14:16:54.453780+0000","last_undegraded":"2024-05-28T14:16:54.453780+0000","last_fullsized":"2024-05-28T14:16:54.453780+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":49,"num_read_kb":196609,"num_write":111,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.969065+0000","last_change":"2024-05-28T14:14:42.737496+0000","last_active":"2024-05-28T14:15:05.969065+0000","last_peered":"2024-05-28T14:15:05.969065+0000","last_clean":"2024-05-28T14:15:05.969065+0000","last_became_active":"2024-05-28T14:14:42.737194+0000","last_became_peered":"2024-05-28T14:14:42.737194+0000","last_unstale":"2024-05-28T14:15:05.969065+0000","last_undegraded":"2024-05-28T14:15:05.969065+0000","last_fullsized":"2024-05-28T14:15:05.969065+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:56:10.665924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"134'121","reported_seq":202,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.562459+0000","last_change":"2024-05-28T14:14:43.758200+0000","last_active":"2024-05-28T14:16:49.562459+0000","last_peered":"2024-05-28T14:16:49.562459+0000","last_clean":"2024-05-28T14:16:49.562459+0000","last_became_active":"2024-05-28T14:14:43.757875+0000","last_became_peered":"2024-05-28T14:14:43.757875+0000","last_unstale":"2024-05-28T14:16:49.562459+0000","last_undegraded":"2024-05-28T14:16:49.562459+0000","last_fullsized":"2024-05-28T14:16:49.562459+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:56:53.917272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":64,"num_read_kb":253954,"num_write":110,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"134'1","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:53.913202+0000","last_change":"2024-05-28T14:14:42.738809+0000","last_active":"2024-05-28T14:16:53.913202+0000","last_peered":"2024-05-28T14:16:53.913202+0000","last_clean":"2024-05-28T14:16:53.913202+0000","last_became_active":"2024-05-28T14:14:42.738494+0000","last_became_peered":"2024-05-28T14:14:42.738494+0000","last_unstale":"2024-05-28T14:16:53.913202+0000","last_undegraded":"2024-05-28T14:16:53.913202+0000","last_fullsized":"2024-05-28T14:16:53.913202+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"134'127","reported_seq":184,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.570990+0000","last_change":"2024-05-28T14:14:43.770616+0000","last_active":"2024-05-28T14:16:49.570990+0000","last_peered":"2024-05-28T14:16:49.570990+0000","last_clean":"2024-05-28T14:16:49.570990+0000","last_became_active":"2024-05-28T14:14:43.770253+0000","last_became_peered":"2024-05-28T14:14:43.770253+0000","last_unstale":"2024-05-28T14:16:49.570990+0000","last_undegraded":"2024-05-28T14:16:49.570990+0000","last_fullsized":"2024-05-28T14:16:49.570990+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:49:45.733085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":40,"num_read_kb":163840,"num_write":118,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.974546+0000","last_change":"2024-05-28T14:14:42.746355+0000","last_active":"2024-05-28T14:15:05.974546+0000","last_peered":"2024-05-28T14:15:05.974546+0000","last_clean":"2024-05-28T14:15:05.974546+0000","last_became_active":"2024-05-28T14:14:42.745926+0000","last_became_peered":"2024-05-28T14:14:42.745926+0000","last_unstale":"2024-05-28T14:15:05.974546+0000","last_undegraded":"2024-05-28T14:15:05.974546+0000","last_fullsized":"2024-05-28T14:15:05.974546+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:54:47.273264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"134'115","reported_seq":223,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.553247+0000","last_change":"2024-05-28T14:15:06.178022+0000","last_active":"2024-05-28T14:16:49.553247+0000","last_peered":"2024-05-28T14:16:49.553247+0000","last_clean":"2024-05-28T14:16:49.553247+0000","last_became_active":"2024-05-28T14:15:05.975545+0000","last_became_peered":"2024-05-28T14:15:05.975545+0000","last_unstale":"2024-05-28T14:16:49.553247+0000","last_undegraded":"2024-05-28T14:16:49.553247+0000","last_fullsized":"2024-05-28T14:16:49.553247+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":85,"num_read_kb":348160,"num_write":107,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"134'42","reported_seq":61,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.786589+0000","last_change":"2024-05-28T14:14:42.749733+0000","last_active":"2024-05-28T14:16:49.786589+0000","last_peered":"2024-05-28T14:16:49.786589+0000","last_clean":"2024-05-28T14:16:49.786589+0000","last_became_active":"2024-05-28T14:14:42.749348+0000","last_became_peered":"2024-05-28T14:14:42.749348+0000","last_unstale":"2024-05-28T14:16:49.786589+0000","last_undegraded":"2024-05-28T14:16:49.786589+0000","last_fullsized":"2024-05-28T14:16:49.786589+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:56:17.145363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":93,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"134'93","reported_seq":211,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.615560+0000","last_change":"2024-05-28T14:14:43.764008+0000","last_active":"2024-05-28T14:16:49.615560+0000","last_peered":"2024-05-28T14:16:49.615560+0000","last_clean":"2024-05-28T14:16:49.615560+0000","last_became_active":"2024-05-28T14:14:43.762648+0000","last_became_peered":"2024-05-28T14:14:43.762648+0000","last_unstale":"2024-05-28T14:16:49.615560+0000","last_undegraded":"2024-05-28T14:16:49.615560+0000","last_fullsized":"2024-05-28T14:16:49.615560+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":86,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.615571+0000","last_change":"2024-05-28T14:14:42.744189+0000","last_active":"2024-05-28T14:16:49.615571+0000","last_peered":"2024-05-28T14:16:49.615571+0000","last_clean":"2024-05-28T14:16:49.615571+0000","last_became_active":"2024-05-28T14:14:42.743858+0000","last_became_peered":"2024-05-28T14:14:42.743858+0000","last_unstale":"2024-05-28T14:16:49.615571+0000","last_undegraded":"2024-05-28T14:16:49.615571+0000","last_fullsized":"2024-05-28T14:16:49.615571+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"134'103","reported_seq":184,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.571199+0000","last_change":"2024-05-28T14:15:06.036082+0000","last_active":"2024-05-28T14:16:49.571199+0000","last_peered":"2024-05-28T14:16:49.571199+0000","last_clean":"2024-05-28T14:16:49.571199+0000","last_became_active":"2024-05-28T14:15:05.976669+0000","last_became_peered":"2024-05-28T14:15:05.976669+0000","last_unstale":"2024-05-28T14:16:49.571199+0000","last_undegraded":"2024-05-28T14:16:49.571199+0000","last_fullsized":"2024-05-28T14:16:49.571199+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:57:19.311096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132541,"num_objects":50,"num_object_clones":1,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":1,"num_read":52,"num_read_kb":208897,"num_write":94,"num_write_kb":282626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8388730,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.580208+0000","last_change":"2024-05-28T14:14:42.748521+0000","last_active":"2024-05-28T14:16:49.580208+0000","last_peered":"2024-05-28T14:16:49.580208+0000","last_clean":"2024-05-28T14:16:49.580208+0000","last_became_active":"2024-05-28T14:14:42.748193+0000","last_became_peered":"2024-05-28T14:14:42.748193+0000","last_unstale":"2024-05-28T14:16:49.580208+0000","last_undegraded":"2024-05-28T14:16:49.580208+0000","last_fullsized":"2024-05-28T14:16:49.580208+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"134'128","reported_seq":206,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.572962+0000","last_change":"2024-05-28T14:14:43.767848+0000","last_active":"2024-05-28T14:16:49.572962+0000","last_peered":"2024-05-28T14:16:49.572962+0000","last_clean":"2024-05-28T14:16:49.572962+0000","last_became_active":"2024-05-28T14:14:43.767552+0000","last_became_peered":"2024-05-28T14:14:43.767552+0000","last_unstale":"2024-05-28T14:16:49.572962+0000","last_undegraded":"2024-05-28T14:16:49.572962+0000","last_fullsized":"2024-05-28T14:16:49.572962+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:09:18.039029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":285212672,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":116,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.615552+0000","last_change":"2024-05-28T14:14:42.746605+0000","last_active":"2024-05-28T14:16:49.615552+0000","last_peered":"2024-05-28T14:16:49.615552+0000","last_clean":"2024-05-28T14:16:49.615552+0000","last_became_active":"2024-05-28T14:14:42.746338+0000","last_became_peered":"2024-05-28T14:14:42.746338+0000","last_unstale":"2024-05-28T14:16:49.615552+0000","last_undegraded":"2024-05-28T14:16:49.615552+0000","last_fullsized":"2024-05-28T14:16:49.615552+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"134'137","reported_seq":244,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.559197+0000","last_change":"2024-05-28T14:14:43.760455+0000","last_active":"2024-05-28T14:16:49.559197+0000","last_peered":"2024-05-28T14:16:49.559197+0000","last_clean":"2024-05-28T14:16:49.559197+0000","last_became_active":"2024-05-28T14:14:43.760063+0000","last_became_peered":"2024-05-28T14:14:43.760063+0000","last_unstale":"2024-05-28T14:16:49.559197+0000","last_undegraded":"2024-05-28T14:16:49.559197+0000","last_fullsized":"2024-05-28T14:16:49.559197+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:50:53.753974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272629760,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":90,"num_read_kb":356355,"num_write":129,"num_write_kb":393217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973173+0000","last_change":"2024-05-28T14:14:42.749618+0000","last_active":"2024-05-28T14:15:05.973173+0000","last_peered":"2024-05-28T14:15:05.973173+0000","last_clean":"2024-05-28T14:15:05.973173+0000","last_became_active":"2024-05-28T14:14:42.749259+0000","last_became_peered":"2024-05-28T14:14:42.749259+0000","last_unstale":"2024-05-28T14:15:05.973173+0000","last_undegraded":"2024-05-28T14:15:05.973173+0000","last_fullsized":"2024-05-28T14:15:05.973173+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:44:02.935848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"134'119","reported_seq":215,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.564680+0000","last_change":"2024-05-28T14:14:43.761804+0000","last_active":"2024-05-28T14:16:49.564680+0000","last_peered":"2024-05-28T14:16:49.564680+0000","last_clean":"2024-05-28T14:16:49.564680+0000","last_became_active":"2024-05-28T14:14:43.761382+0000","last_became_peered":"2024-05-28T14:14:43.761382+0000","last_unstale":"2024-05-28T14:16:49.564680+0000","last_undegraded":"2024-05-28T14:16:49.564680+0000","last_fullsized":"2024-05-28T14:16:49.564680+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:48:50.641484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":114,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"134'31","reported_seq":80,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:26.863095+0000","last_change":"2024-05-28T14:14:42.739854+0000","last_active":"2024-05-28T14:16:26.863095+0000","last_peered":"2024-05-28T14:16:26.863095+0000","last_clean":"2024-05-28T14:16:26.863095+0000","last_became_active":"2024-05-28T14:14:42.738369+0000","last_became_peered":"2024-05-28T14:14:42.738369+0000","last_unstale":"2024-05-28T14:16:26.863095+0000","last_undegraded":"2024-05-28T14:16:26.863095+0000","last_fullsized":"2024-05-28T14:16:26.863095+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:34:08.611902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":31,"num_write":30,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"134'123","reported_seq":224,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.615528+0000","last_change":"2024-05-28T14:14:43.765205+0000","last_active":"2024-05-28T14:16:49.615528+0000","last_peered":"2024-05-28T14:16:49.615528+0000","last_clean":"2024-05-28T14:16:49.615528+0000","last_became_active":"2024-05-28T14:14:43.764879+0000","last_became_peered":"2024-05-28T14:14:43.764879+0000","last_unstale":"2024-05-28T14:16:49.615528+0000","last_undegraded":"2024-05-28T14:16:49.615528+0000","last_fullsized":"2024-05-28T14:16:49.615528+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":116,"num_write_kb":372736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:54.227760+0000","last_change":"2024-05-28T14:16:54.227795+0000","last_active":"2024-05-28T14:16:54.227760+0000","last_peered":"2024-05-28T14:16:54.227760+0000","last_clean":"2024-05-28T14:16:54.227760+0000","last_became_active":"2024-05-28T14:14:42.730333+0000","last_became_peered":"2024-05-28T14:14:42.730333+0000","last_unstale":"2024-05-28T14:16:54.227760+0000","last_undegraded":"2024-05-28T14:16:54.227760+0000","last_fullsized":"2024-05-28T14:16:54.227760+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:16:54.227666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:16:54.227666+0000","last_clean_scrub_stamp":"2024-05-28T14:16:54.227666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:08:18.110965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"134'106","reported_seq":172,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:49.559996+0000","last_change":"2024-05-28T14:14:43.761743+0000","last_active":"2024-05-28T14:16:49.559996+0000","last_peered":"2024-05-28T14:16:49.559996+0000","last_clean":"2024-05-28T14:16:49.559996+0000","last_became_active":"2024-05-28T14:14:43.760828+0000","last_became_peered":"2024-05-28T14:14:43.760828+0000","last_unstale":"2024-05-28T14:16:49.559996+0000","last_undegraded":"2024-05-28T14:16:49.559996+0000","last_fullsized":"2024-05-28T14:16:49.559996+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:42.716220+0000","last_clean_scrub_stamp":"2024-05-28T14:14:42.716220+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:44:21.055874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":101,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"131'2","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.973324+0000","last_change":"2024-05-28T14:14:42.751868+0000","last_active":"2024-05-28T14:15:05.973324+0000","last_peered":"2024-05-28T14:15:05.973324+0000","last_clean":"2024-05-28T14:15:05.973324+0000","last_became_active":"2024-05-28T14:14:42.751447+0000","last_became_peered":"2024-05-28T14:14:42.751447+0000","last_unstale":"2024-05-28T14:15:05.973324+0000","last_undegraded":"2024-05-28T14:15:05.973324+0000","last_fullsized":"2024-05-28T14:15:05.973324+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:14:41.710337+0000","last_clean_scrub_stamp":"2024-05-28T14:14:41.710337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:28:17.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":383,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:15:05.972861+0000","last_change":"2024-05-28T13:58:12.961938+0000","last_active":"2024-05-28T14:15:05.972861+0000","last_peered":"2024-05-28T14:15:05.972861+0000","last_clean":"2024-05-28T14:15:05.972861+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:15:05.972861+0000","last_undegraded":"2024-05-28T14:15:05.972861+0000","last_fullsized":"2024-05-28T14:15:05.972861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T13:58:11.588499+0000","last_clean_scrub_stamp":"2024-05-28T13:58:11.588499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:39:59.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":15082718397,"num_objects":3603,"num_object_clones":1,"num_object_copies":7206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":1,"num_read":4365,"num_read_kb":17629245,"num_write":6581,"num_write_kb":20676651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":50331770,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":30165483520,"data_stored":30165436794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7090,"ondisk_log_size":7090,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1787369,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":30,"num_read_kb":31,"num_write":641,"num_write_kb":1991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3637248,"data_stored":3574738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":592,"ondisk_log_size":592,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575075,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3272720,"kb_used_data":3133320,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":91099120,"statfs":{"total":96636764160,"available":93285498880,"internally_reserved":0,"allocated":3208519680,"data_stored":3207558171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":142735818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575075,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4222476,"kb_used_data":4078916,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90149364,"statfs":{"total":96636764160,"available":92312948736,"internally_reserved":0,"allocated":4176809984,"data_stored":4175857132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575077,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3121336,"kb_used_data":2995824,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91250504,"statfs":{"total":96636764160,"available":93440516096,"internally_reserved":0,"allocated":3067723776,"data_stored":3066774417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":128514502},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575076,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4572916,"kb_used_data":4433516,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":89798924,"statfs":{"total":96636764160,"available":91954098176,"internally_reserved":0,"allocated":4539920384,"data_stored":4538974892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575075,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3790716,"kb_used_data":3642996,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":90581124,"statfs":{"total":96636764160,"available":92755070976,"internally_reserved":0,"allocated":3730427904,"data_stored":3729479880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":151255496},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575076,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4906484,"kb_used_data":4740716,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":89465356,"statfs":{"total":96636764160,"available":91612524544,"internally_reserved":0,"allocated":4854493184,"data_stored":4853548245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575075,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3483436,"kb_used_data":3339876,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90888404,"statfs":{"total":96636764160,"available":93069725696,"internally_reserved":0,"allocated":3420033024,"data_stored":3419095661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575076,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3250484,"kb_used_data":3111084,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":91121356,"statfs":{"total":96636764160,"available":93308268544,"internally_reserved":0,"allocated":3185750016,"data_stored":3184804998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1798144,"data_stored":1786571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1802240,"data_stored":1781646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3183484928,"data_stored":3183476989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3418361856,"data_stored":3418358020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4852813824,"data_stored":4852809988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3728740352,"data_stored":3728736516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4538245120,"data_stored":4538237181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3066044416,"data_stored":3066036664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4173340672,"data_stored":4173332920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3204452352,"data_stored":3204448516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:16:58.137 INFO:tasks.ceph:pgid 19.3f last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.137 INFO:tasks.ceph:pgid 19.3e last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.137 INFO:tasks.ceph:pgid 19.3d last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.137 INFO:tasks.ceph:pgid 19.3c last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.138 INFO:tasks.ceph:pgid 19.3b last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.138 INFO:tasks.ceph:pgid 19.3a last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.138 INFO:tasks.ceph:pgid 19.39 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.138 INFO:tasks.ceph:pgid 19.38 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.138 INFO:tasks.ceph:pgid 19.37 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.139 INFO:tasks.ceph:pgid 19.36 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.139 INFO:tasks.ceph:pgid 19.35 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.139 INFO:tasks.ceph:pgid 19.34 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.139 INFO:tasks.ceph:pgid 19.33 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.139 INFO:tasks.ceph:pgid 19.32 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.140 INFO:tasks.ceph:pgid 19.31 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.140 INFO:tasks.ceph:pgid 19.30 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.140 INFO:tasks.ceph:pgid 19.2f last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.140 INFO:tasks.ceph:pgid 19.2e last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.140 INFO:tasks.ceph:pgid 19.2d last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.140 INFO:tasks.ceph:pgid 19.2c last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.141 INFO:tasks.ceph:pgid 19.2b last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.141 INFO:tasks.ceph:pgid 19.2a last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.141 INFO:tasks.ceph:pgid 19.29 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.141 INFO:tasks.ceph:pgid 19.28 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.141 INFO:tasks.ceph:pgid 19.27 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.141 INFO:tasks.ceph:pgid 19.26 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.142 INFO:tasks.ceph:pgid 19.25 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.142 INFO:tasks.ceph:pgid 19.24 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.142 INFO:tasks.ceph:pgid 19.23 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.142 INFO:tasks.ceph:pgid 19.22 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.142 INFO:tasks.ceph:pgid 19.21 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.142 INFO:tasks.ceph:pgid 19.20 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.142 INFO:tasks.ceph:pgid 19.1e last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.143 INFO:tasks.ceph:pgid 18.1f last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.143 INFO:tasks.ceph:pgid 19.1f last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.143 INFO:tasks.ceph:pgid 18.1e last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.143 INFO:tasks.ceph:pgid 19.1c last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.143 INFO:tasks.ceph:pgid 18.1d last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.143 INFO:tasks.ceph:pgid 19.1d last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.144 INFO:tasks.ceph:pgid 18.1c last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.144 INFO:tasks.ceph:pgid 19.1a last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.144 INFO:tasks.ceph:pgid 19.1b last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.144 INFO:tasks.ceph:pgid 18.1a last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.144 INFO:tasks.ceph:pgid 19.18 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.144 INFO:tasks.ceph:pgid 18.19 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.145 INFO:tasks.ceph:pgid 19.19 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.145 INFO:tasks.ceph:pgid 18.18 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.145 INFO:tasks.ceph:pgid 19.16 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.145 INFO:tasks.ceph:pgid 18.17 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.145 INFO:tasks.ceph:pgid 19.17 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.145 INFO:tasks.ceph:pgid 18.16 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.146 INFO:tasks.ceph:pgid 19.14 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.146 INFO:tasks.ceph:pgid 18.15 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.146 INFO:tasks.ceph:pgid 19.15 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.146 INFO:tasks.ceph:pgid 18.14 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.146 INFO:tasks.ceph:pgid 19.13 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.146 INFO:tasks.ceph:pgid 18.12 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.147 INFO:tasks.ceph:pgid 19.10 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.147 INFO:tasks.ceph:pgid 18.11 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.147 INFO:tasks.ceph:pgid 19.11 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.147 INFO:tasks.ceph:pgid 18.10 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.147 INFO:tasks.ceph:pgid 19.e last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.147 INFO:tasks.ceph:pgid 18.f last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.148 INFO:tasks.ceph:pgid 19.f last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.148 INFO:tasks.ceph:pgid 18.e last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.148 INFO:tasks.ceph:pgid 19.c last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.148 INFO:tasks.ceph:pgid 18.d last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.148 INFO:tasks.ceph:pgid 19.d last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.148 INFO:tasks.ceph:pgid 18.c last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.148 INFO:tasks.ceph:pgid 19.a last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.149 INFO:tasks.ceph:pgid 18.b last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.149 INFO:tasks.ceph:pgid 19.b last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.149 INFO:tasks.ceph:pgid 18.a last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.149 INFO:tasks.ceph:pgid 19.8 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.149 INFO:tasks.ceph:pgid 18.9 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.149 INFO:tasks.ceph:pgid 19.9 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.150 INFO:tasks.ceph:pgid 18.8 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.150 INFO:tasks.ceph:pgid 19.6 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.150 INFO:tasks.ceph:pgid 18.7 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.150 INFO:tasks.ceph:pgid 19.7 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.150 INFO:tasks.ceph:pgid 18.6 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.150 INFO:tasks.ceph:pgid 19.4 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.151 INFO:tasks.ceph:pgid 18.5 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.151 INFO:tasks.ceph:pgid 19.5 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.151 INFO:tasks.ceph:pgid 18.4 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.151 INFO:tasks.ceph:pgid 19.2 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.151 INFO:tasks.ceph:pgid 18.3 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.151 INFO:tasks.ceph:pgid 19.3 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.151 INFO:tasks.ceph:pgid 18.2 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.152 INFO:tasks.ceph:pgid 19.0 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.152 INFO:tasks.ceph:pgid 18.1 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.152 INFO:tasks.ceph:pgid 19.1 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.152 INFO:tasks.ceph:pgid 19.12 last_scrub_stamp 2024-05-28T14:14:42.716220+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=42, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.152 INFO:tasks.ceph:pgid 18.13 last_scrub_stamp 2024-05-28T14:14:41.710337+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=14, tm_sec=41, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.152 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-28T13:58:11.588499+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=13, tm_min=58, tm_sec=11, tm_wday=1, tm_yday=149, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=28, tm_hour=14, tm_min=16, tm_sec=52, tm_wday=1, tm_yday=149, tm_isdst=0) 2024-05-28T14:16:58.153 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-28T14:17:18.154 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-28T14:17:18.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e9a000640 1 Processor -- start 2024-05-28T14:17:18.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e9a000640 1 -- start start 2024-05-28T14:17:18.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e9a000640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e94070800 0x7f5e94070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e9a000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e9415f920 0x7f5e9409b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e9a000640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e940d0660 con 0x7f5e9406ff10 2024-05-28T14:17:18.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e9a000640 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 -- 0x7f5e940d0940 con 0x7f5e94070800 2024-05-28T14:17:18.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e9a000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5e940d0c20 con 0x7f5e9415f920 2024-05-28T14:17:18.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e98c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e94070800 0x7f5e94070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e98c00640 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5e94070800 0x7f5e94070be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45870/0 (socket says 172.21.15.42:45870) 2024-05-28T14:17:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e98c00640 1 -- 172.21.15.42:0/380354755 learned_addr learned my addr 172.21.15.42:0/380354755 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:17:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e93e00640 1 --1- 172.21.15.42:0/380354755 >> v1:172.21.15.42:6789/0 conn(0x7f5e9406ff10 0x7f5e9409aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:40160/0 (socket says 172.21.15.42:40160) 2024-05-28T14:17:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e99600640 1 --2- 172.21.15.42:0/380354755 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e9415f920 0x7f5e9409b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e98c00640 1 -- 172.21.15.42:0/380354755 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e9415f920 msgr2=0x7f5e9409b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:17:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e98c00640 1 --2- 172.21.15.42:0/380354755 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e9415f920 0x7f5e9409b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e98c00640 1 -- 172.21.15.42:0/380354755 >> v1:172.21.15.42:6789/0 conn(0x7f5e9406ff10 legacy=0x7f5e9409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e98c00640 1 -- 172.21.15.42:0/380354755 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e940d0f00 con 0x7f5e94070800 2024-05-28T14:17:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.325+0000 7f5e93400640 1 -- 172.21.15.42:0/380354755 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4169716934 0 0) 0x7f5e940d0660 con 0x7f5e9406ff10 2024-05-28T14:17:18.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.329+0000 7f5e98c00640 1 --2- 172.21.15.42:0/380354755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e94070800 0x7f5e94070be0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f5e880098a0 tx=0x7f5e8801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf63ecb6151af47 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:17:18.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.329+0000 7f5e93400640 1 -- 172.21.15.42:0/380354755 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e880b1070 con 0x7f5e94070800 2024-05-28T14:17:18.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.329+0000 7f5e93400640 1 -- 172.21.15.42:0/380354755 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e880ac620 con 0x7f5e94070800 2024-05-28T14:17:18.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.329+0000 7f5e93400640 1 -- 172.21.15.42:0/380354755 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e880ac9f0 con 0x7f5e94070800 2024-05-28T14:17:18.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.329+0000 7f5e9a000640 1 -- 172.21.15.42:0/380354755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e94070800 msgr2=0x7f5e94070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:18.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.329+0000 7f5e9a000640 1 --2- 172.21.15.42:0/380354755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e94070800 0x7f5e94070be0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f5e880098a0 tx=0x7f5e8801c540 comp rx=0 tx=0).stop 2024-05-28T14:17:18.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.329+0000 7f5e9a000640 1 -- 172.21.15.42:0/380354755 shutdown_connections 2024-05-28T14:17:18.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.329+0000 7f5e9a000640 1 --2- 172.21.15.42:0/380354755 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e9415f920 0x7f5e9409b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.329+0000 7f5e9a000640 1 --2- 172.21.15.42:0/380354755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e94070800 0x7f5e94070be0 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.329+0000 7f5e9a000640 1 -- 172.21.15.42:0/380354755 >> 172.21.15.42:0/380354755 conn(0x7f5e940b3dd0 msgr2=0x7f5e940b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:17:18.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.329+0000 7f5e9a000640 1 -- 172.21.15.42:0/380354755 shutdown_connections 2024-05-28T14:17:18.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.329+0000 7f5e9a000640 1 -- 172.21.15.42:0/380354755 wait complete. 2024-05-28T14:17:18.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.329+0000 7f5e9a000640 1 Processor -- start 2024-05-28T14:17:18.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e9a000640 1 -- start start 2024-05-28T14:17:18.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e9a000640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e9406ff10 0x7f5e9414b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e9a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e94070800 0x7f5e940c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e9a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e9415f920 0x7f5e940b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e9a000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e940b1a10 con 0x7f5e94070800 2024-05-28T14:17:18.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e9a000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e940b1890 con 0x7f5e9415f920 2024-05-28T14:17:18.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e9a000640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9416bde0 con 0x7f5e9406ff10 2024-05-28T14:17:18.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e93e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e94070800 0x7f5e940c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e93e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e94070800 0x7f5e940c93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45876/0 (socket says 172.21.15.42:45876) 2024-05-28T14:17:18.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e93e00640 1 -- 172.21.15.42:0/142644242 learned_addr learned my addr 172.21.15.42:0/142644242 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:17:18.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e98c00640 1 --2- 172.21.15.42:0/142644242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e9406ff10 0x7f5e9414b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e99600640 1 --2- 172.21.15.42:0/142644242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e9415f920 0x7f5e940b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e93e00640 1 -- 172.21.15.42:0/142644242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e9406ff10 msgr2=0x7f5e9414b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:18.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e93e00640 1 --2- 172.21.15.42:0/142644242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e9406ff10 0x7f5e9414b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e93e00640 1 -- 172.21.15.42:0/142644242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e9415f920 msgr2=0x7f5e940b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:18.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e93e00640 1 --2- 172.21.15.42:0/142644242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e9415f920 0x7f5e940b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e93e00640 1 -- 172.21.15.42:0/142644242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e940cb1f0 con 0x7f5e94070800 2024-05-28T14:17:18.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e93e00640 1 --2- 172.21.15.42:0/142644242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e94070800 0x7f5e940c93b0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f5e800011f0 tx=0x7f5e8000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:17:18.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e90c00640 1 -- 172.21.15.42:0/142644242 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80018070 con 0x7f5e94070800 2024-05-28T14:17:18.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e90c00640 1 -- 172.21.15.42:0/142644242 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e800074e0 con 0x7f5e94070800 2024-05-28T14:17:18.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e90c00640 1 -- 172.21.15.42:0/142644242 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80004a10 con 0x7f5e94070800 2024-05-28T14:17:18.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e9a000640 1 -- 172.21.15.42:0/142644242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e940cb4d0 con 0x7f5e94070800 2024-05-28T14:17:18.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e9a000640 1 -- 172.21.15.42:0/142644242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e940cba90 con 0x7f5e94070800 2024-05-28T14:17:18.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.333+0000 7f5e9a000640 1 -- 172.21.15.42:0/142644242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e940cda90 con 0x7f5e94070800 2024-05-28T14:17:18.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.337+0000 7f5e90c00640 1 -- 172.21.15.42:0/142644242 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 51) ==== 90690+0+0 (secure 0 0 0) 0x7f5e80004bb0 con 0x7f5e94070800 2024-05-28T14:17:18.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.337+0000 7f5e90c00640 1 --2- 172.21.15.42:0/142644242 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e7406a150 0x7f5e7406c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.337+0000 7f5e90c00640 1 -- 172.21.15.42:0/142644242 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f5e80084cd0 con 0x7f5e94070800 2024-05-28T14:17:18.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.337+0000 7f5e98c00640 1 --2- 172.21.15.42:0/142644242 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e7406a150 0x7f5e7406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.337+0000 7f5e98c00640 1 --2- 172.21.15.42:0/142644242 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e7406a150 0x7f5e7406c610 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f5e880ae000 tx=0x7f5e880a7000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:17:18.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.341+0000 7f5e90c00640 1 -- 172.21.15.42:0/142644242 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5e800566a0 con 0x7f5e94070800 2024-05-28T14:17:18.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.505+0000 7f5e9a000640 1 -- 172.21.15.42:0/142644242 --> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5e9404fcf0 con 0x7f5e7406a150 2024-05-28T14:17:18.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.509+0000 7f5e90c00640 1 -- 172.21.15.42:0/142644242 <== mgr.4098 v2:172.21.15.42:6832/1976130233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+255815 (secure 0 0 0) 0x7f5e9404fcf0 con 0x7f5e7406a150 2024-05-28T14:17:18.498 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:17:18.500 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-05-28T14:17:18.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.513+0000 7f5e9a000640 1 -- 172.21.15.42:0/142644242 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e7406a150 msgr2=0x7f5e7406c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:18.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.513+0000 7f5e9a000640 1 --2- 172.21.15.42:0/142644242 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e7406a150 0x7f5e7406c610 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f5e880ae000 tx=0x7f5e880a7000 comp rx=0 tx=0).stop 2024-05-28T14:17:18.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.513+0000 7f5e9a000640 1 -- 172.21.15.42:0/142644242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e94070800 msgr2=0x7f5e940c93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:18.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.513+0000 7f5e9a000640 1 --2- 172.21.15.42:0/142644242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e94070800 0x7f5e940c93b0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f5e800011f0 tx=0x7f5e8000c9b0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.513+0000 7f5e9a000640 1 -- 172.21.15.42:0/142644242 shutdown_connections 2024-05-28T14:17:18.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.513+0000 7f5e9a000640 1 --2- 172.21.15.42:0/142644242 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f5e7406a150 0x7f5e7406c610 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.513+0000 7f5e9a000640 1 --2- 172.21.15.42:0/142644242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e9415f920 0x7f5e940b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.513+0000 7f5e9a000640 1 --2- 172.21.15.42:0/142644242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e94070800 0x7f5e940c93b0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.513+0000 7f5e9a000640 1 --2- 172.21.15.42:0/142644242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e9406ff10 0x7f5e9414b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.513+0000 7f5e9a000640 1 -- 172.21.15.42:0/142644242 >> 172.21.15.42:0/142644242 conn(0x7f5e940b3dd0 msgr2=0x7f5e9414e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:17:18.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.513+0000 7f5e9a000640 1 -- 172.21.15.42:0/142644242 shutdown_connections 2024-05-28T14:17:18.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.513+0000 7f5e9a000640 1 -- 172.21.15.42:0/142644242 wait complete. 2024-05-28T14:17:18.519 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":701,"stamp":"2024-05-28T14:17:17.255904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15085097443,"num_objects":3628,"num_object_clones":1,"num_object_copies":7256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3628,"num_whiteouts":1,"num_read":4511,"num_read_kb":17629376,"num_write":7398,"num_write_kb":20681069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":50331770,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5739,"num_omap_keys":23,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13938,"ondisk_log_size":13938,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":30616408,"kb_used_data":29476248,"kb_used_omap":12,"kb_used_meta":1140083,"kb_avail":724358312,"statfs":{"total":773094113280,"available":741742911488,"internally_reserved":0,"allocated":30183677952,"data_stored":30176096014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1167445554},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1044,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003761"},"pg_stats":[{"pgid":"19.3f","version":"134'185","reported_seq":290,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.160786+0000","last_change":"2024-05-28T14:17:03.616729+0000","last_active":"2024-05-28T14:17:05.160786+0000","last_peered":"2024-05-28T14:17:05.160786+0000","last_clean":"2024-05-28T14:17:05.160786+0000","last_became_active":"2024-05-28T14:14:43.766125+0000","last_became_peered":"2024-05-28T14:14:43.766125+0000","last_unstale":"2024-05-28T14:17:05.160786+0000","last_undegraded":"2024-05-28T14:17:05.160786+0000","last_fullsized":"2024-05-28T14:17:05.160786+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'145","last_scrub_stamp":"2024-05-28T14:17:03.616585+0000","last_deep_scrub":"134'145","last_deep_scrub_stamp":"2024-05-28T14:17:03.616585+0000","last_clean_scrub_stamp":"2024-05-28T14:17:03.616585+0000","objects_scrubbed":51,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:46:41.956240+0000","scrub_duration":611,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":77,"num_read_kb":315392,"num_write":85,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"134'222","reported_seq":325,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:08.888776+0000","last_change":"2024-05-28T14:17:08.888820+0000","last_active":"2024-05-28T14:17:08.888776+0000","last_peered":"2024-05-28T14:17:08.888776+0000","last_clean":"2024-05-28T14:17:08.888776+0000","last_became_active":"2024-05-28T14:14:43.769641+0000","last_became_peered":"2024-05-28T14:14:43.769641+0000","last_unstale":"2024-05-28T14:17:08.888776+0000","last_undegraded":"2024-05-28T14:17:08.888776+0000","last_fullsized":"2024-05-28T14:17:08.888776+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'222","last_scrub_stamp":"2024-05-28T14:17:08.888706+0000","last_deep_scrub":"134'222","last_deep_scrub_stamp":"2024-05-28T14:17:08.888706+0000","last_clean_scrub_stamp":"2024-05-28T14:17:08.888706+0000","objects_scrubbed":64,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:14:28.372725+0000","scrub_duration":419,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":106,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"134'204","reported_seq":304,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.860276+0000","last_change":"2024-05-28T14:17:05.860340+0000","last_active":"2024-05-28T14:17:05.860276+0000","last_peered":"2024-05-28T14:17:05.860276+0000","last_clean":"2024-05-28T14:17:05.860276+0000","last_became_active":"2024-05-28T14:14:43.764098+0000","last_became_peered":"2024-05-28T14:14:43.764098+0000","last_unstale":"2024-05-28T14:17:05.860276+0000","last_undegraded":"2024-05-28T14:17:05.860276+0000","last_fullsized":"2024-05-28T14:17:05.860276+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'204","last_scrub_stamp":"2024-05-28T14:17:05.860153+0000","last_deep_scrub":"134'204","last_deep_scrub_stamp":"2024-05-28T14:17:05.860153+0000","last_clean_scrub_stamp":"2024-05-28T14:17:05.860153+0000","objects_scrubbed":56,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:15:14.665162+0000","scrub_duration":1467,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":70,"num_read_kb":278530,"num_write":108,"num_write_kb":323587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"134'223","reported_seq":299,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.174723+0000","last_change":"2024-05-28T14:16:59.340493+0000","last_active":"2024-05-28T14:17:05.174723+0000","last_peered":"2024-05-28T14:17:05.174723+0000","last_clean":"2024-05-28T14:17:05.174723+0000","last_became_active":"2024-05-28T14:14:43.758925+0000","last_became_peered":"2024-05-28T14:14:43.758925+0000","last_unstale":"2024-05-28T14:17:05.174723+0000","last_undegraded":"2024-05-28T14:17:05.174723+0000","last_fullsized":"2024-05-28T14:17:05.174723+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'175","last_scrub_stamp":"2024-05-28T14:16:59.340387+0000","last_deep_scrub":"134'175","last_deep_scrub_stamp":"2024-05-28T14:16:59.340387+0000","last_clean_scrub_stamp":"2024-05-28T14:16:59.340387+0000","objects_scrubbed":53,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:00:08.721439+0000","scrub_duration":319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":48,"num_read_kb":184323,"num_write":115,"num_write_kb":339969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"134'169","reported_seq":241,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:10.300397+0000","last_change":"2024-05-28T14:17:10.300397+0000","last_active":"2024-05-28T14:17:10.300397+0000","last_peered":"2024-05-28T14:17:10.300397+0000","last_clean":"2024-05-28T14:17:10.300397+0000","last_became_active":"2024-05-28T14:14:43.764703+0000","last_became_peered":"2024-05-28T14:14:43.764703+0000","last_unstale":"2024-05-28T14:17:10.300397+0000","last_undegraded":"2024-05-28T14:17:10.300397+0000","last_fullsized":"2024-05-28T14:17:10.300397+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'169","last_scrub_stamp":"2024-05-28T14:17:10.300272+0000","last_deep_scrub":"134'169","last_deep_scrub_stamp":"2024-05-28T14:17:10.300272+0000","last_clean_scrub_stamp":"2024-05-28T14:17:10.300272+0000","objects_scrubbed":49,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:58:07.260356+0000","scrub_duration":335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":45,"num_read_kb":184320,"num_write":75,"num_write_kb":253952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"134'238","reported_seq":361,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.171552+0000","last_change":"2024-05-28T14:17:02.181129+0000","last_active":"2024-05-28T14:17:05.171552+0000","last_peered":"2024-05-28T14:17:05.171552+0000","last_clean":"2024-05-28T14:17:05.171552+0000","last_became_active":"2024-05-28T14:14:43.762689+0000","last_became_peered":"2024-05-28T14:14:43.762689+0000","last_unstale":"2024-05-28T14:17:05.171552+0000","last_undegraded":"2024-05-28T14:17:05.171552+0000","last_fullsized":"2024-05-28T14:17:05.171552+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'195","last_scrub_stamp":"2024-05-28T14:17:02.181023+0000","last_deep_scrub":"134'195","last_deep_scrub_stamp":"2024-05-28T14:17:02.181023+0000","last_clean_scrub_stamp":"2024-05-28T14:17:02.181023+0000","objects_scrubbed":71,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:24:29.851342+0000","scrub_duration":571,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":95,"num_read_kb":385025,"num_write":115,"num_write_kb":372738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"134'197","reported_seq":275,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.165266+0000","last_change":"2024-05-28T14:17:04.631710+0000","last_active":"2024-05-28T14:17:05.165266+0000","last_peered":"2024-05-28T14:17:05.165266+0000","last_clean":"2024-05-28T14:17:05.165266+0000","last_became_active":"2024-05-28T14:14:43.770616+0000","last_became_peered":"2024-05-28T14:14:43.770616+0000","last_unstale":"2024-05-28T14:17:05.165266+0000","last_undegraded":"2024-05-28T14:17:05.165266+0000","last_fullsized":"2024-05-28T14:17:05.165266+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'157","last_scrub_stamp":"2024-05-28T14:17:04.631544+0000","last_deep_scrub":"134'157","last_deep_scrub_stamp":"2024-05-28T14:17:04.631544+0000","last_clean_scrub_stamp":"2024-05-28T14:17:04.631544+0000","objects_scrubbed":47,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:30:23.659515+0000","scrub_duration":1027,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":89,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"134'210","reported_seq":292,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.175302+0000","last_change":"2024-05-28T14:17:00.807021+0000","last_active":"2024-05-28T14:17:05.175302+0000","last_peered":"2024-05-28T14:17:05.175302+0000","last_clean":"2024-05-28T14:17:05.175302+0000","last_became_active":"2024-05-28T14:14:43.761977+0000","last_became_peered":"2024-05-28T14:14:43.761977+0000","last_unstale":"2024-05-28T14:17:05.175302+0000","last_undegraded":"2024-05-28T14:17:05.175302+0000","last_fullsized":"2024-05-28T14:17:05.175302+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'179","last_scrub_stamp":"2024-05-28T14:17:00.806908+0000","last_deep_scrub":"134'179","last_deep_scrub_stamp":"2024-05-28T14:17:00.806908+0000","last_clean_scrub_stamp":"2024-05-28T14:17:00.806908+0000","objects_scrubbed":59,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:23:45.176396+0000","scrub_duration":371,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":54,"num_read_kb":221184,"num_write":113,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"134'206","reported_seq":291,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:07.781491+0000","last_change":"2024-05-28T14:17:07.781491+0000","last_active":"2024-05-28T14:17:07.781491+0000","last_peered":"2024-05-28T14:17:07.781491+0000","last_clean":"2024-05-28T14:17:07.781491+0000","last_became_active":"2024-05-28T14:14:43.770335+0000","last_became_peered":"2024-05-28T14:14:43.770335+0000","last_unstale":"2024-05-28T14:17:07.781491+0000","last_undegraded":"2024-05-28T14:17:07.781491+0000","last_fullsized":"2024-05-28T14:17:07.781491+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'206","last_scrub_stamp":"2024-05-28T14:17:07.781372+0000","last_deep_scrub":"134'206","last_deep_scrub_stamp":"2024-05-28T14:17:07.781372+0000","last_clean_scrub_stamp":"2024-05-28T14:17:07.781372+0000","objects_scrubbed":52,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:36:28.692700+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":102,"num_write_kb":311297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"134'192","reported_seq":287,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:09.755196+0000","last_change":"2024-05-28T14:17:09.755220+0000","last_active":"2024-05-28T14:17:09.755196+0000","last_peered":"2024-05-28T14:17:09.755196+0000","last_clean":"2024-05-28T14:17:09.755196+0000","last_became_active":"2024-05-28T14:14:43.769687+0000","last_became_peered":"2024-05-28T14:14:43.769687+0000","last_unstale":"2024-05-28T14:17:09.755196+0000","last_undegraded":"2024-05-28T14:17:09.755196+0000","last_fullsized":"2024-05-28T14:17:09.755196+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'192","last_scrub_stamp":"2024-05-28T14:17:09.755130+0000","last_deep_scrub":"134'192","last_deep_scrub_stamp":"2024-05-28T14:17:09.755130+0000","last_clean_scrub_stamp":"2024-05-28T14:17:09.755130+0000","objects_scrubbed":51,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:24:37.425449+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":67,"num_read_kb":274432,"num_write":93,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"134'202","reported_seq":338,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.158722+0000","last_change":"2024-05-28T14:17:00.540879+0000","last_active":"2024-05-28T14:17:05.158722+0000","last_peered":"2024-05-28T14:17:05.158722+0000","last_clean":"2024-05-28T14:17:05.158722+0000","last_became_active":"2024-05-28T14:14:43.757685+0000","last_became_peered":"2024-05-28T14:14:43.757685+0000","last_unstale":"2024-05-28T14:17:05.158722+0000","last_undegraded":"2024-05-28T14:17:05.158722+0000","last_fullsized":"2024-05-28T14:17:05.158722+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'171","last_scrub_stamp":"2024-05-28T14:17:00.540780+0000","last_deep_scrub":"134'171","last_deep_scrub_stamp":"2024-05-28T14:17:00.540780+0000","last_clean_scrub_stamp":"2024-05-28T14:17:00.540780+0000","objects_scrubbed":59,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:37:48.132061+0000","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":108,"num_read_kb":442368,"num_write":103,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"134'202","reported_seq":335,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.425549+0000","last_change":"2024-05-28T14:17:06.425549+0000","last_active":"2024-05-28T14:17:06.425549+0000","last_peered":"2024-05-28T14:17:06.425549+0000","last_clean":"2024-05-28T14:17:06.425549+0000","last_became_active":"2024-05-28T14:14:43.768939+0000","last_became_peered":"2024-05-28T14:14:43.768939+0000","last_unstale":"2024-05-28T14:17:06.425549+0000","last_undegraded":"2024-05-28T14:17:06.425549+0000","last_fullsized":"2024-05-28T14:17:06.425549+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'202","last_scrub_stamp":"2024-05-28T14:17:06.425375+0000","last_deep_scrub":"134'202","last_deep_scrub_stamp":"2024-05-28T14:17:06.425375+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.425375+0000","objects_scrubbed":58,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:36:36.127688+0000","scrub_duration":1055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":105,"num_read_kb":413700,"num_write":98,"num_write_kb":315393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"134'201","reported_seq":312,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.782293+0000","last_change":"2024-05-28T14:17:05.782408+0000","last_active":"2024-05-28T14:17:05.782293+0000","last_peered":"2024-05-28T14:17:05.782293+0000","last_clean":"2024-05-28T14:17:05.782293+0000","last_became_active":"2024-05-28T14:14:43.766269+0000","last_became_peered":"2024-05-28T14:14:43.766269+0000","last_unstale":"2024-05-28T14:17:05.782293+0000","last_undegraded":"2024-05-28T14:17:05.782293+0000","last_fullsized":"2024-05-28T14:17:05.782293+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'201","last_scrub_stamp":"2024-05-28T14:17:05.782139+0000","last_deep_scrub":"134'201","last_deep_scrub_stamp":"2024-05-28T14:17:05.782139+0000","last_clean_scrub_stamp":"2024-05-28T14:17:05.782139+0000","objects_scrubbed":57,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:59:10.193892+0000","scrub_duration":1795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":105,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"134'197","reported_seq":291,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.167419+0000","last_change":"2024-05-28T14:17:02.980692+0000","last_active":"2024-05-28T14:17:05.167419+0000","last_peered":"2024-05-28T14:17:05.167419+0000","last_clean":"2024-05-28T14:17:05.167419+0000","last_became_active":"2024-05-28T14:14:43.762494+0000","last_became_peered":"2024-05-28T14:14:43.762494+0000","last_unstale":"2024-05-28T14:17:05.167419+0000","last_undegraded":"2024-05-28T14:17:05.167419+0000","last_fullsized":"2024-05-28T14:17:05.167419+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'164","last_scrub_stamp":"2024-05-28T14:17:02.980576+0000","last_deep_scrub":"134'164","last_deep_scrub_stamp":"2024-05-28T14:17:02.980576+0000","last_clean_scrub_stamp":"2024-05-28T14:17:02.980576+0000","objects_scrubbed":64,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:29:58.097454+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":66,"num_read_kb":270336,"num_write":92,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"134'166","reported_seq":224,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.169650+0000","last_change":"2024-05-28T14:17:00.266078+0000","last_active":"2024-05-28T14:17:05.169650+0000","last_peered":"2024-05-28T14:17:05.169650+0000","last_clean":"2024-05-28T14:17:05.169650+0000","last_became_active":"2024-05-28T14:14:43.760977+0000","last_became_peered":"2024-05-28T14:14:43.760977+0000","last_unstale":"2024-05-28T14:17:05.169650+0000","last_undegraded":"2024-05-28T14:17:05.169650+0000","last_fullsized":"2024-05-28T14:17:05.169650+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'131","last_scrub_stamp":"2024-05-28T14:17:00.265920+0000","last_deep_scrub":"134'131","last_deep_scrub_stamp":"2024-05-28T14:17:00.265920+0000","last_clean_scrub_stamp":"2024-05-28T14:17:00.265920+0000","objects_scrubbed":48,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:38.036380+0000","scrub_duration":279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":31,"num_read_kb":126976,"num_write":74,"num_write_kb":249856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"134'227","reported_seq":312,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.387778+0000","last_change":"2024-05-28T14:17:06.387854+0000","last_active":"2024-05-28T14:17:06.387778+0000","last_peered":"2024-05-28T14:17:06.387778+0000","last_clean":"2024-05-28T14:17:06.387778+0000","last_became_active":"2024-05-28T14:14:43.758483+0000","last_became_peered":"2024-05-28T14:14:43.758483+0000","last_unstale":"2024-05-28T14:17:06.387778+0000","last_undegraded":"2024-05-28T14:17:06.387778+0000","last_fullsized":"2024-05-28T14:17:06.387778+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'227","last_scrub_stamp":"2024-05-28T14:17:06.387662+0000","last_deep_scrub":"134'227","last_deep_scrub_stamp":"2024-05-28T14:17:06.387662+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.387662+0000","objects_scrubbed":59,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:27:55.145263+0000","scrub_duration":859,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":57,"num_read_kb":225282,"num_write":123,"num_write_kb":368641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"134'182","reported_seq":256,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.152348+0000","last_change":"2024-05-28T14:16:59.959628+0000","last_active":"2024-05-28T14:17:05.152348+0000","last_peered":"2024-05-28T14:17:05.152348+0000","last_clean":"2024-05-28T14:17:05.152348+0000","last_became_active":"2024-05-28T14:14:43.755912+0000","last_became_peered":"2024-05-28T14:14:43.755912+0000","last_unstale":"2024-05-28T14:17:05.152348+0000","last_undegraded":"2024-05-28T14:17:05.152348+0000","last_fullsized":"2024-05-28T14:17:05.152348+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'148","last_scrub_stamp":"2024-05-28T14:16:59.959494+0000","last_deep_scrub":"134'148","last_deep_scrub_stamp":"2024-05-28T14:16:59.959494+0000","last_clean_scrub_stamp":"2024-05-28T14:16:59.959494+0000","objects_scrubbed":47,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:56:21.612970+0000","scrub_duration":311,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":47,"num_read_kb":180227,"num_write":95,"num_write_kb":286721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"134'210","reported_seq":336,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.176587+0000","last_change":"2024-05-28T14:17:02.050278+0000","last_active":"2024-05-28T14:17:05.176587+0000","last_peered":"2024-05-28T14:17:05.176587+0000","last_clean":"2024-05-28T14:17:05.176587+0000","last_became_active":"2024-05-28T14:14:43.748801+0000","last_became_peered":"2024-05-28T14:14:43.748801+0000","last_unstale":"2024-05-28T14:17:05.176587+0000","last_undegraded":"2024-05-28T14:17:05.176587+0000","last_fullsized":"2024-05-28T14:17:05.176587+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'177","last_scrub_stamp":"2024-05-28T14:17:02.050126+0000","last_deep_scrub":"134'177","last_deep_scrub_stamp":"2024-05-28T14:17:02.050126+0000","last_clean_scrub_stamp":"2024-05-28T14:17:02.050126+0000","objects_scrubbed":59,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:05:48.553817+0000","scrub_duration":487,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":98,"num_read_kb":389123,"num_write":107,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"134'226","reported_seq":323,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.150900+0000","last_change":"2024-05-28T14:17:02.779588+0000","last_active":"2024-05-28T14:17:05.150900+0000","last_peered":"2024-05-28T14:17:05.150900+0000","last_clean":"2024-05-28T14:17:05.150900+0000","last_became_active":"2024-05-28T14:14:43.768757+0000","last_became_peered":"2024-05-28T14:14:43.768757+0000","last_unstale":"2024-05-28T14:17:05.150900+0000","last_undegraded":"2024-05-28T14:17:05.150900+0000","last_fullsized":"2024-05-28T14:17:05.150900+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'190","last_scrub_stamp":"2024-05-28T14:17:02.779359+0000","last_deep_scrub":"134'190","last_deep_scrub_stamp":"2024-05-28T14:17:02.779359+0000","last_clean_scrub_stamp":"2024-05-28T14:17:02.779359+0000","objects_scrubbed":67,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:19.207279+0000","scrub_duration":547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":115,"num_write_kb":372736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"134'218","reported_seq":310,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.203036+0000","last_change":"2024-05-28T14:17:04.124065+0000","last_active":"2024-05-28T14:17:05.203036+0000","last_peered":"2024-05-28T14:17:05.203036+0000","last_clean":"2024-05-28T14:17:05.203036+0000","last_became_active":"2024-05-28T14:14:43.756244+0000","last_became_peered":"2024-05-28T14:14:43.756244+0000","last_unstale":"2024-05-28T14:17:05.203036+0000","last_undegraded":"2024-05-28T14:17:05.203036+0000","last_fullsized":"2024-05-28T14:17:05.203036+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'184","last_scrub_stamp":"2024-05-28T14:17:04.123768+0000","last_deep_scrub":"134'184","last_deep_scrub_stamp":"2024-05-28T14:17:04.123768+0000","last_clean_scrub_stamp":"2024-05-28T14:17:04.123768+0000","objects_scrubbed":61,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:31:22.320235+0000","scrub_duration":591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":101,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"134'215","reported_seq":343,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.172909+0000","last_change":"2024-05-28T14:17:03.547191+0000","last_active":"2024-05-28T14:17:05.172909+0000","last_peered":"2024-05-28T14:17:05.172909+0000","last_clean":"2024-05-28T14:17:05.172909+0000","last_became_active":"2024-05-28T14:14:43.757288+0000","last_became_peered":"2024-05-28T14:14:43.757288+0000","last_unstale":"2024-05-28T14:17:05.172909+0000","last_undegraded":"2024-05-28T14:17:05.172909+0000","last_fullsized":"2024-05-28T14:17:05.172909+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'176","last_scrub_stamp":"2024-05-28T14:17:03.547018+0000","last_deep_scrub":"134'176","last_deep_scrub_stamp":"2024-05-28T14:17:03.547018+0000","last_clean_scrub_stamp":"2024-05-28T14:17:03.547018+0000","objects_scrubbed":58,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:35:19.974938+0000","scrub_duration":711,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":100,"num_read_kb":389125,"num_write":108,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"134'225","reported_seq":314,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.156335+0000","last_change":"2024-05-28T14:17:01.579292+0000","last_active":"2024-05-28T14:17:05.156335+0000","last_peered":"2024-05-28T14:17:05.156335+0000","last_clean":"2024-05-28T14:17:05.156335+0000","last_became_active":"2024-05-28T14:14:43.759580+0000","last_became_peered":"2024-05-28T14:14:43.759580+0000","last_unstale":"2024-05-28T14:17:05.156335+0000","last_undegraded":"2024-05-28T14:17:05.156335+0000","last_fullsized":"2024-05-28T14:17:05.156335+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'183","last_scrub_stamp":"2024-05-28T14:17:01.579022+0000","last_deep_scrub":"134'183","last_deep_scrub_stamp":"2024-05-28T14:17:01.579022+0000","last_clean_scrub_stamp":"2024-05-28T14:17:01.579022+0000","objects_scrubbed":69,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:13:19.451293+0000","scrub_duration":355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":61,"num_read_kb":241666,"num_write":107,"num_write_kb":356353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"134'204","reported_seq":294,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.866998+0000","last_change":"2024-05-28T14:17:06.867025+0000","last_active":"2024-05-28T14:17:06.866998+0000","last_peered":"2024-05-28T14:17:06.866998+0000","last_clean":"2024-05-28T14:17:06.866998+0000","last_became_active":"2024-05-28T14:14:43.767741+0000","last_became_peered":"2024-05-28T14:14:43.767741+0000","last_unstale":"2024-05-28T14:17:06.866998+0000","last_undegraded":"2024-05-28T14:17:06.866998+0000","last_fullsized":"2024-05-28T14:17:06.866998+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'204","last_scrub_stamp":"2024-05-28T14:17:06.866921+0000","last_deep_scrub":"134'204","last_deep_scrub_stamp":"2024-05-28T14:17:06.866921+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.866921+0000","objects_scrubbed":51,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:09:50.994719+0000","scrub_duration":495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":62,"num_read_kb":253952,"num_write":101,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"134'211","reported_seq":298,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.150121+0000","last_change":"2024-05-28T14:16:59.695598+0000","last_active":"2024-05-28T14:17:05.150121+0000","last_peered":"2024-05-28T14:17:05.150121+0000","last_clean":"2024-05-28T14:17:05.150121+0000","last_became_active":"2024-05-28T14:14:43.753794+0000","last_became_peered":"2024-05-28T14:14:43.753794+0000","last_unstale":"2024-05-28T14:17:05.150121+0000","last_undegraded":"2024-05-28T14:17:05.150121+0000","last_fullsized":"2024-05-28T14:17:05.150121+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'170","last_scrub_stamp":"2024-05-28T14:16:59.695457+0000","last_deep_scrub":"134'170","last_deep_scrub_stamp":"2024-05-28T14:16:59.695457+0000","last_clean_scrub_stamp":"2024-05-28T14:16:59.695457+0000","objects_scrubbed":59,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:06:54.839934+0000","scrub_duration":391,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":59,"num_read_kb":241664,"num_write":103,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"134'215","reported_seq":311,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.153423+0000","last_change":"2024-05-28T14:17:04.581405+0000","last_active":"2024-05-28T14:17:05.153423+0000","last_peered":"2024-05-28T14:17:05.153423+0000","last_clean":"2024-05-28T14:17:05.153423+0000","last_became_active":"2024-05-28T14:14:43.763332+0000","last_became_peered":"2024-05-28T14:14:43.763332+0000","last_unstale":"2024-05-28T14:17:05.153423+0000","last_undegraded":"2024-05-28T14:17:05.153423+0000","last_fullsized":"2024-05-28T14:17:05.153423+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'182","last_scrub_stamp":"2024-05-28T14:17:04.581168+0000","last_deep_scrub":"134'182","last_deep_scrub_stamp":"2024-05-28T14:17:04.581168+0000","last_clean_scrub_stamp":"2024-05-28T14:17:04.581168+0000","objects_scrubbed":54,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:08:05.590366+0000","scrub_duration":1047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298372,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":68,"num_read_kb":274433,"num_write":112,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"134'224","reported_seq":322,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.923794+0000","last_change":"2024-05-28T14:17:06.923842+0000","last_active":"2024-05-28T14:17:06.923794+0000","last_peered":"2024-05-28T14:17:06.923794+0000","last_clean":"2024-05-28T14:17:06.923794+0000","last_became_active":"2024-05-28T14:14:43.758722+0000","last_became_peered":"2024-05-28T14:14:43.758722+0000","last_unstale":"2024-05-28T14:17:06.923794+0000","last_undegraded":"2024-05-28T14:17:06.923794+0000","last_fullsized":"2024-05-28T14:17:06.923794+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'224","last_scrub_stamp":"2024-05-28T14:17:06.923728+0000","last_deep_scrub":"134'224","last_deep_scrub_stamp":"2024-05-28T14:17:06.923728+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.923728+0000","objects_scrubbed":60,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T02:00:25.549557+0000","scrub_duration":411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251658240,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":112,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"134'183","reported_seq":285,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.685866+0000","last_change":"2024-05-28T14:17:06.686215+0000","last_active":"2024-05-28T14:17:06.685866+0000","last_peered":"2024-05-28T14:17:06.685866+0000","last_clean":"2024-05-28T14:17:06.685866+0000","last_became_active":"2024-05-28T14:14:43.767925+0000","last_became_peered":"2024-05-28T14:14:43.767925+0000","last_unstale":"2024-05-28T14:17:06.685866+0000","last_undegraded":"2024-05-28T14:17:06.685866+0000","last_fullsized":"2024-05-28T14:17:06.685866+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'183","last_scrub_stamp":"2024-05-28T14:17:06.685756+0000","last_deep_scrub":"134'183","last_deep_scrub_stamp":"2024-05-28T14:17:06.685756+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.685756+0000","objects_scrubbed":47,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:28:42.961301+0000","scrub_duration":687,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":75,"num_read_kb":294915,"num_write":99,"num_write_kb":290818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"134'189","reported_seq":251,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.171622+0000","last_change":"2024-05-28T14:17:03.372715+0000","last_active":"2024-05-28T14:17:05.171622+0000","last_peered":"2024-05-28T14:17:05.171622+0000","last_clean":"2024-05-28T14:17:05.171622+0000","last_became_active":"2024-05-28T14:14:43.746249+0000","last_became_peered":"2024-05-28T14:14:43.746249+0000","last_unstale":"2024-05-28T14:17:05.171622+0000","last_undegraded":"2024-05-28T14:17:05.171622+0000","last_fullsized":"2024-05-28T14:17:05.171622+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'151","last_scrub_stamp":"2024-05-28T14:17:03.372617+0000","last_deep_scrub":"134'151","last_deep_scrub_stamp":"2024-05-28T14:17:03.372617+0000","last_clean_scrub_stamp":"2024-05-28T14:17:03.372617+0000","objects_scrubbed":52,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:57:27.680087+0000","scrub_duration":599,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909764,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":34,"num_read_kb":131074,"num_write":94,"num_write_kb":290818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"134'205","reported_seq":305,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.172603+0000","last_change":"2024-05-28T14:17:01.360517+0000","last_active":"2024-05-28T14:17:05.172603+0000","last_peered":"2024-05-28T14:17:05.172603+0000","last_clean":"2024-05-28T14:17:05.172603+0000","last_became_active":"2024-05-28T14:14:43.761161+0000","last_became_peered":"2024-05-28T14:14:43.761161+0000","last_unstale":"2024-05-28T14:17:05.172603+0000","last_undegraded":"2024-05-28T14:17:05.172603+0000","last_fullsized":"2024-05-28T14:17:05.172603+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'169","last_scrub_stamp":"2024-05-28T14:17:01.360412+0000","last_deep_scrub":"134'169","last_deep_scrub_stamp":"2024-05-28T14:17:01.360412+0000","last_clean_scrub_stamp":"2024-05-28T14:17:01.360412+0000","objects_scrubbed":58,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:57:25.667881+0000","scrub_duration":355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075588,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":72,"num_read_kb":294912,"num_write":104,"num_write_kb":327681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"134'211","reported_seq":308,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.172690+0000","last_change":"2024-05-28T14:17:02.465813+0000","last_active":"2024-05-28T14:17:05.172690+0000","last_peered":"2024-05-28T14:17:05.172690+0000","last_clean":"2024-05-28T14:17:05.172690+0000","last_became_active":"2024-05-28T14:14:43.763522+0000","last_became_peered":"2024-05-28T14:14:43.763522+0000","last_unstale":"2024-05-28T14:17:05.172690+0000","last_undegraded":"2024-05-28T14:17:05.172690+0000","last_fullsized":"2024-05-28T14:17:05.172690+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'177","last_scrub_stamp":"2024-05-28T14:17:02.465642+0000","last_deep_scrub":"134'177","last_deep_scrub_stamp":"2024-05-28T14:17:02.465642+0000","last_clean_scrub_stamp":"2024-05-28T14:17:02.465642+0000","objects_scrubbed":60,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:40:42.622261+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251658240,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":106,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"134'200","reported_seq":328,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:08.243958+0000","last_change":"2024-05-28T14:17:08.243986+0000","last_active":"2024-05-28T14:17:08.243958+0000","last_peered":"2024-05-28T14:17:08.243958+0000","last_clean":"2024-05-28T14:17:08.243958+0000","last_became_active":"2024-05-28T14:14:43.763142+0000","last_became_peered":"2024-05-28T14:14:43.763142+0000","last_unstale":"2024-05-28T14:17:08.243958+0000","last_undegraded":"2024-05-28T14:17:08.243958+0000","last_fullsized":"2024-05-28T14:17:08.243958+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'200","last_scrub_stamp":"2024-05-28T14:17:08.243887+0000","last_deep_scrub":"134'200","last_deep_scrub_stamp":"2024-05-28T14:17:08.243887+0000","last_clean_scrub_stamp":"2024-05-28T14:17:08.243887+0000","objects_scrubbed":52,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:52:11.036377+0000","scrub_duration":339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":88,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"134'209","reported_seq":312,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.171944+0000","last_change":"2024-05-28T14:16:59.021406+0000","last_active":"2024-05-28T14:17:05.171944+0000","last_peered":"2024-05-28T14:17:05.171944+0000","last_clean":"2024-05-28T14:17:05.171944+0000","last_became_active":"2024-05-28T14:14:43.747272+0000","last_became_peered":"2024-05-28T14:14:43.747272+0000","last_unstale":"2024-05-28T14:17:05.171944+0000","last_undegraded":"2024-05-28T14:17:05.171944+0000","last_fullsized":"2024-05-28T14:17:05.171944+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'164","last_scrub_stamp":"2024-05-28T14:16:59.021234+0000","last_deep_scrub":"134'164","last_deep_scrub_stamp":"2024-05-28T14:16:59.021234+0000","last_clean_scrub_stamp":"2024-05-28T14:16:59.021234+0000","objects_scrubbed":58,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:34:27.305316+0000","scrub_duration":351,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":96,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"134'196","reported_seq":268,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:07.987043+0000","last_change":"2024-05-28T14:17:07.987075+0000","last_active":"2024-05-28T14:17:07.987043+0000","last_peered":"2024-05-28T14:17:07.987043+0000","last_clean":"2024-05-28T14:17:07.987043+0000","last_became_active":"2024-05-28T14:14:43.759136+0000","last_became_peered":"2024-05-28T14:14:43.759136+0000","last_unstale":"2024-05-28T14:17:07.987043+0000","last_undegraded":"2024-05-28T14:17:07.987043+0000","last_fullsized":"2024-05-28T14:17:07.987043+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'196","last_scrub_stamp":"2024-05-28T14:17:07.986979+0000","last_deep_scrub":"134'196","last_deep_scrub_stamp":"2024-05-28T14:17:07.986979+0000","last_clean_scrub_stamp":"2024-05-28T14:17:07.986979+0000","objects_scrubbed":53,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:58:35.417797+0000","scrub_duration":439,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":44,"num_read_kb":180224,"num_write":103,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"134'6","reported_seq":34,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:00.852073+0000","last_change":"2024-05-28T14:17:00.852100+0000","last_active":"2024-05-28T14:17:00.852073+0000","last_peered":"2024-05-28T14:17:00.852073+0000","last_clean":"2024-05-28T14:17:00.852073+0000","last_became_active":"2024-05-28T14:14:42.751450+0000","last_became_peered":"2024-05-28T14:14:42.751450+0000","last_unstale":"2024-05-28T14:17:00.852073+0000","last_undegraded":"2024-05-28T14:17:00.852073+0000","last_fullsized":"2024-05-28T14:17:00.852073+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'6","last_scrub_stamp":"2024-05-28T14:17:00.851980+0000","last_deep_scrub":"134'6","last_deep_scrub_stamp":"2024-05-28T14:17:00.851980+0000","last_clean_scrub_stamp":"2024-05-28T14:17:00.851980+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:15:59.240251+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"134'217","reported_seq":317,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.173637+0000","last_change":"2024-05-28T14:17:02.902914+0000","last_active":"2024-05-28T14:17:05.173637+0000","last_peered":"2024-05-28T14:17:05.173637+0000","last_clean":"2024-05-28T14:17:05.173637+0000","last_became_active":"2024-05-28T14:14:43.766347+0000","last_became_peered":"2024-05-28T14:14:43.766347+0000","last_unstale":"2024-05-28T14:17:05.173637+0000","last_undegraded":"2024-05-28T14:17:05.173637+0000","last_fullsized":"2024-05-28T14:17:05.173637+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'175","last_scrub_stamp":"2024-05-28T14:17:02.902711+0000","last_deep_scrub":"134'175","last_deep_scrub_stamp":"2024-05-28T14:17:02.902711+0000","last_clean_scrub_stamp":"2024-05-28T14:17:02.902711+0000","objects_scrubbed":54,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:56:24.556187+0000","scrub_duration":511,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":72,"num_read_kb":290817,"num_write":112,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":29,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:58.828908+0000","last_change":"2024-05-28T14:16:58.828952+0000","last_active":"2024-05-28T14:16:58.828908+0000","last_peered":"2024-05-28T14:16:58.828908+0000","last_clean":"2024-05-28T14:16:58.828908+0000","last_became_active":"2024-05-28T14:14:42.742668+0000","last_became_peered":"2024-05-28T14:14:42.742668+0000","last_unstale":"2024-05-28T14:16:58.828908+0000","last_undegraded":"2024-05-28T14:16:58.828908+0000","last_fullsized":"2024-05-28T14:16:58.828908+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'1","last_scrub_stamp":"2024-05-28T14:16:58.828820+0000","last_deep_scrub":"131'1","last_deep_scrub_stamp":"2024-05-28T14:16:58.828820+0000","last_clean_scrub_stamp":"2024-05-28T14:16:58.828820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:54:42.155380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"134'189","reported_seq":291,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:09.270649+0000","last_change":"2024-05-28T14:17:09.270649+0000","last_active":"2024-05-28T14:17:09.270649+0000","last_peered":"2024-05-28T14:17:09.270649+0000","last_clean":"2024-05-28T14:17:09.270649+0000","last_became_active":"2024-05-28T14:14:43.765230+0000","last_became_peered":"2024-05-28T14:14:43.765230+0000","last_unstale":"2024-05-28T14:17:09.270649+0000","last_undegraded":"2024-05-28T14:17:09.270649+0000","last_fullsized":"2024-05-28T14:17:09.270649+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'189","last_scrub_stamp":"2024-05-28T14:17:09.270519+0000","last_deep_scrub":"134'189","last_deep_scrub_stamp":"2024-05-28T14:17:09.270519+0000","last_clean_scrub_stamp":"2024-05-28T14:17:09.270519+0000","objects_scrubbed":47,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:51:40.682385+0000","scrub_duration":351,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":75,"num_read_kb":303105,"num_write":87,"num_write_kb":270337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"134'4","reported_seq":32,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:55.208018+0000","last_change":"2024-05-28T14:16:55.208018+0000","last_active":"2024-05-28T14:16:55.208018+0000","last_peered":"2024-05-28T14:16:55.208018+0000","last_clean":"2024-05-28T14:16:55.208018+0000","last_became_active":"2024-05-28T14:14:42.733231+0000","last_became_peered":"2024-05-28T14:14:42.733231+0000","last_unstale":"2024-05-28T14:16:55.208018+0000","last_undegraded":"2024-05-28T14:16:55.208018+0000","last_fullsized":"2024-05-28T14:16:55.208018+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'4","last_scrub_stamp":"2024-05-28T14:16:55.207926+0000","last_deep_scrub":"134'4","last_deep_scrub_stamp":"2024-05-28T14:16:55.207926+0000","last_clean_scrub_stamp":"2024-05-28T14:16:55.207926+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:35:37.480640+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"134'206","reported_seq":321,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.674391+0000","last_change":"2024-05-28T14:17:06.674391+0000","last_active":"2024-05-28T14:17:06.674391+0000","last_peered":"2024-05-28T14:17:06.674391+0000","last_clean":"2024-05-28T14:17:06.674391+0000","last_became_active":"2024-05-28T14:14:43.759307+0000","last_became_peered":"2024-05-28T14:14:43.759307+0000","last_unstale":"2024-05-28T14:17:06.674391+0000","last_undegraded":"2024-05-28T14:17:06.674391+0000","last_fullsized":"2024-05-28T14:17:06.674391+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'206","last_scrub_stamp":"2024-05-28T14:17:06.674266+0000","last_deep_scrub":"134'206","last_deep_scrub_stamp":"2024-05-28T14:17:06.674266+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.674266+0000","objects_scrubbed":53,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:38:21.001528+0000","scrub_duration":795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":107,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"134'3","reported_seq":31,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:56.579400+0000","last_change":"2024-05-28T14:16:56.579400+0000","last_active":"2024-05-28T14:16:56.579400+0000","last_peered":"2024-05-28T14:16:56.579400+0000","last_clean":"2024-05-28T14:16:56.579400+0000","last_became_active":"2024-05-28T14:14:42.745097+0000","last_became_peered":"2024-05-28T14:14:42.745097+0000","last_unstale":"2024-05-28T14:16:56.579400+0000","last_undegraded":"2024-05-28T14:16:56.579400+0000","last_fullsized":"2024-05-28T14:16:56.579400+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'3","last_scrub_stamp":"2024-05-28T14:16:56.579303+0000","last_deep_scrub":"134'3","last_deep_scrub_stamp":"2024-05-28T14:16:56.579303+0000","last_clean_scrub_stamp":"2024-05-28T14:16:56.579303+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:03:30.636519+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"134'208","reported_seq":297,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.163398+0000","last_change":"2024-05-28T14:17:03.158084+0000","last_active":"2024-05-28T14:17:05.163398+0000","last_peered":"2024-05-28T14:17:05.163398+0000","last_clean":"2024-05-28T14:17:05.163398+0000","last_became_active":"2024-05-28T14:14:43.747282+0000","last_became_peered":"2024-05-28T14:14:43.747282+0000","last_unstale":"2024-05-28T14:17:05.163398+0000","last_undegraded":"2024-05-28T14:17:05.163398+0000","last_fullsized":"2024-05-28T14:17:05.163398+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'169","last_scrub_stamp":"2024-05-28T14:17:03.157948+0000","last_deep_scrub":"134'169","last_deep_scrub_stamp":"2024-05-28T14:17:03.157948+0000","last_clean_scrub_stamp":"2024-05-28T14:17:03.157948+0000","objects_scrubbed":57,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:56:50.768797+0000","scrub_duration":615,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":105,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":29,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:54.568249+0000","last_change":"2024-05-28T14:16:54.568249+0000","last_active":"2024-05-28T14:16:54.568249+0000","last_peered":"2024-05-28T14:16:54.568249+0000","last_clean":"2024-05-28T14:16:54.568249+0000","last_became_active":"2024-05-28T14:14:42.741425+0000","last_became_peered":"2024-05-28T14:14:42.741425+0000","last_unstale":"2024-05-28T14:16:54.568249+0000","last_undegraded":"2024-05-28T14:16:54.568249+0000","last_fullsized":"2024-05-28T14:16:54.568249+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'1","last_scrub_stamp":"2024-05-28T14:16:54.568146+0000","last_deep_scrub":"131'1","last_deep_scrub_stamp":"2024-05-28T14:16:54.568146+0000","last_clean_scrub_stamp":"2024-05-28T14:16:54.568146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:54:37.894706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"134'202","reported_seq":329,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.159283+0000","last_change":"2024-05-28T14:17:01.025222+0000","last_active":"2024-05-28T14:17:05.159283+0000","last_peered":"2024-05-28T14:17:05.159283+0000","last_clean":"2024-05-28T14:17:05.159283+0000","last_became_active":"2024-05-28T14:14:43.760691+0000","last_became_peered":"2024-05-28T14:14:43.760691+0000","last_unstale":"2024-05-28T14:17:05.159283+0000","last_undegraded":"2024-05-28T14:17:05.159283+0000","last_fullsized":"2024-05-28T14:17:05.159283+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'172","last_scrub_stamp":"2024-05-28T14:17:01.025128+0000","last_deep_scrub":"134'172","last_deep_scrub_stamp":"2024-05-28T14:17:01.025128+0000","last_clean_scrub_stamp":"2024-05-28T14:17:01.025128+0000","objects_scrubbed":58,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:14:20.509147+0000","scrub_duration":375,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":99,"num_read_kb":405504,"num_write":102,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":29,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:57.485443+0000","last_change":"2024-05-28T14:16:57.485443+0000","last_active":"2024-05-28T14:16:57.485443+0000","last_peered":"2024-05-28T14:16:57.485443+0000","last_clean":"2024-05-28T14:16:57.485443+0000","last_became_active":"2024-05-28T14:14:42.739191+0000","last_became_peered":"2024-05-28T14:14:42.739191+0000","last_unstale":"2024-05-28T14:16:57.485443+0000","last_undegraded":"2024-05-28T14:16:57.485443+0000","last_fullsized":"2024-05-28T14:16:57.485443+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'1","last_scrub_stamp":"2024-05-28T14:16:57.485358+0000","last_deep_scrub":"131'1","last_deep_scrub_stamp":"2024-05-28T14:16:57.485358+0000","last_clean_scrub_stamp":"2024-05-28T14:16:57.485358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:18:27.280167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"134'204","reported_seq":311,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.175291+0000","last_change":"2024-05-28T14:17:01.756558+0000","last_active":"2024-05-28T14:17:05.175291+0000","last_peered":"2024-05-28T14:17:05.175291+0000","last_clean":"2024-05-28T14:17:05.175291+0000","last_became_active":"2024-05-28T14:14:43.761547+0000","last_became_peered":"2024-05-28T14:14:43.761547+0000","last_unstale":"2024-05-28T14:17:05.175291+0000","last_undegraded":"2024-05-28T14:17:05.175291+0000","last_fullsized":"2024-05-28T14:17:05.175291+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'161","last_scrub_stamp":"2024-05-28T14:17:01.756424+0000","last_deep_scrub":"134'161","last_deep_scrub_stamp":"2024-05-28T14:17:01.756424+0000","last_clean_scrub_stamp":"2024-05-28T14:17:01.756424+0000","objects_scrubbed":52,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:34:30.040506+0000","scrub_duration":363,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":102,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:57.672353+0000","last_change":"2024-05-28T14:16:57.672353+0000","last_active":"2024-05-28T14:16:57.672353+0000","last_peered":"2024-05-28T14:16:57.672353+0000","last_clean":"2024-05-28T14:16:57.672353+0000","last_became_active":"2024-05-28T14:14:42.745691+0000","last_became_peered":"2024-05-28T14:14:42.745691+0000","last_unstale":"2024-05-28T14:16:57.672353+0000","last_undegraded":"2024-05-28T14:16:57.672353+0000","last_fullsized":"2024-05-28T14:16:57.672353+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:16:57.672229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:16:57.672229+0000","last_clean_scrub_stamp":"2024-05-28T14:16:57.672229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:00:38.028934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"134'191","reported_seq":284,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.837626+0000","last_change":"2024-05-28T14:17:06.837626+0000","last_active":"2024-05-28T14:17:06.837626+0000","last_peered":"2024-05-28T14:17:06.837626+0000","last_clean":"2024-05-28T14:17:06.837626+0000","last_became_active":"2024-05-28T14:14:43.762358+0000","last_became_peered":"2024-05-28T14:14:43.762358+0000","last_unstale":"2024-05-28T14:17:06.837626+0000","last_undegraded":"2024-05-28T14:17:06.837626+0000","last_fullsized":"2024-05-28T14:17:06.837626+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'191","last_scrub_stamp":"2024-05-28T14:17:06.837451+0000","last_deep_scrub":"134'191","last_deep_scrub_stamp":"2024-05-28T14:17:06.837451+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.837451+0000","objects_scrubbed":47,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:18:45.164387+0000","scrub_duration":343,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":66,"num_read_kb":262146,"num_write":97,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:59.595050+0000","last_change":"2024-05-28T14:16:59.595050+0000","last_active":"2024-05-28T14:16:59.595050+0000","last_peered":"2024-05-28T14:16:59.595050+0000","last_clean":"2024-05-28T14:16:59.595050+0000","last_became_active":"2024-05-28T14:14:42.747127+0000","last_became_peered":"2024-05-28T14:14:42.747127+0000","last_unstale":"2024-05-28T14:16:59.595050+0000","last_undegraded":"2024-05-28T14:16:59.595050+0000","last_fullsized":"2024-05-28T14:16:59.595050+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:16:59.594936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:16:59.594936+0000","last_clean_scrub_stamp":"2024-05-28T14:16:59.594936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:16:50.803971+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"134'182","reported_seq":246,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.156740+0000","last_change":"2024-05-28T14:16:57.253698+0000","last_active":"2024-05-28T14:17:05.156740+0000","last_peered":"2024-05-28T14:17:05.156740+0000","last_clean":"2024-05-28T14:17:05.156740+0000","last_became_active":"2024-05-28T14:14:43.756803+0000","last_became_peered":"2024-05-28T14:14:43.756803+0000","last_unstale":"2024-05-28T14:17:05.156740+0000","last_undegraded":"2024-05-28T14:17:05.156740+0000","last_fullsized":"2024-05-28T14:17:05.156740+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'141","last_scrub_stamp":"2024-05-28T14:16:57.253538+0000","last_deep_scrub":"134'141","last_deep_scrub_stamp":"2024-05-28T14:16:57.253538+0000","last_clean_scrub_stamp":"2024-05-28T14:16:57.253538+0000","objects_scrubbed":45,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:54:53.075854+0000","scrub_duration":223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188743680,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":85,"num_write_kb":266240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"134'1","reported_seq":29,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:00.770643+0000","last_change":"2024-05-28T14:17:00.770643+0000","last_active":"2024-05-28T14:17:00.770643+0000","last_peered":"2024-05-28T14:17:00.770643+0000","last_clean":"2024-05-28T14:17:00.770643+0000","last_became_active":"2024-05-28T14:14:42.746243+0000","last_became_peered":"2024-05-28T14:14:42.746243+0000","last_unstale":"2024-05-28T14:17:00.770643+0000","last_undegraded":"2024-05-28T14:17:00.770643+0000","last_fullsized":"2024-05-28T14:17:00.770643+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'1","last_scrub_stamp":"2024-05-28T14:17:00.770511+0000","last_deep_scrub":"134'1","last_deep_scrub_stamp":"2024-05-28T14:17:00.770511+0000","last_clean_scrub_stamp":"2024-05-28T14:17:00.770511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:03:34.827727+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"134'185","reported_seq":254,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.156257+0000","last_change":"2024-05-28T14:17:04.477410+0000","last_active":"2024-05-28T14:17:05.156257+0000","last_peered":"2024-05-28T14:17:05.156257+0000","last_clean":"2024-05-28T14:17:05.156257+0000","last_became_active":"2024-05-28T14:14:43.750768+0000","last_became_peered":"2024-05-28T14:14:43.750768+0000","last_unstale":"2024-05-28T14:17:05.156257+0000","last_undegraded":"2024-05-28T14:17:05.156257+0000","last_fullsized":"2024-05-28T14:17:05.156257+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'155","last_scrub_stamp":"2024-05-28T14:17:04.477220+0000","last_deep_scrub":"134'155","last_deep_scrub_stamp":"2024-05-28T14:17:04.477220+0000","last_clean_scrub_stamp":"2024-05-28T14:17:04.477220+0000","objects_scrubbed":53,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:16:55.686255+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":41,"num_read_kb":167936,"num_write":89,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:00.580818+0000","last_change":"2024-05-28T14:17:00.580818+0000","last_active":"2024-05-28T14:17:00.580818+0000","last_peered":"2024-05-28T14:17:00.580818+0000","last_clean":"2024-05-28T14:17:00.580818+0000","last_became_active":"2024-05-28T14:14:42.744565+0000","last_became_peered":"2024-05-28T14:14:42.744565+0000","last_unstale":"2024-05-28T14:17:00.580818+0000","last_undegraded":"2024-05-28T14:17:00.580818+0000","last_fullsized":"2024-05-28T14:17:00.580818+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:17:00.580738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:17:00.580738+0000","last_clean_scrub_stamp":"2024-05-28T14:17:00.580738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:59:25.473255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"134'178","reported_seq":289,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.188089+0000","last_change":"2024-05-28T14:17:06.188210+0000","last_active":"2024-05-28T14:17:06.188089+0000","last_peered":"2024-05-28T14:17:06.188089+0000","last_clean":"2024-05-28T14:17:06.188089+0000","last_became_active":"2024-05-28T14:14:43.760106+0000","last_became_peered":"2024-05-28T14:14:43.760106+0000","last_unstale":"2024-05-28T14:17:06.188089+0000","last_undegraded":"2024-05-28T14:17:06.188089+0000","last_fullsized":"2024-05-28T14:17:06.188089+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'178","last_scrub_stamp":"2024-05-28T14:17:06.187974+0000","last_deep_scrub":"134'178","last_deep_scrub_stamp":"2024-05-28T14:17:06.187974+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.187974+0000","objects_scrubbed":54,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:51:17.746590+0000","scrub_duration":1351,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":80,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"134'27","reported_seq":55,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:57.835043+0000","last_change":"2024-05-28T14:16:57.835043+0000","last_active":"2024-05-28T14:16:57.835043+0000","last_peered":"2024-05-28T14:16:57.835043+0000","last_clean":"2024-05-28T14:16:57.835043+0000","last_became_active":"2024-05-28T14:14:42.734095+0000","last_became_peered":"2024-05-28T14:14:42.734095+0000","last_unstale":"2024-05-28T14:16:57.835043+0000","last_undegraded":"2024-05-28T14:16:57.835043+0000","last_fullsized":"2024-05-28T14:16:57.835043+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'27","last_scrub_stamp":"2024-05-28T14:16:57.834933+0000","last_deep_scrub":"134'27","last_deep_scrub_stamp":"2024-05-28T14:16:57.834933+0000","last_clean_scrub_stamp":"2024-05-28T14:16:57.834933+0000","objects_scrubbed":2,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:08:11.676304+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"134'180","reported_seq":275,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:12.337813+0000","last_change":"2024-05-28T14:17:12.337844+0000","last_active":"2024-05-28T14:17:12.337813+0000","last_peered":"2024-05-28T14:17:12.337813+0000","last_clean":"2024-05-28T14:17:12.337813+0000","last_became_active":"2024-05-28T14:14:43.766334+0000","last_became_peered":"2024-05-28T14:14:43.766334+0000","last_unstale":"2024-05-28T14:17:12.337813+0000","last_undegraded":"2024-05-28T14:17:12.337813+0000","last_fullsized":"2024-05-28T14:17:12.337813+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'180","last_scrub_stamp":"2024-05-28T14:17:12.337743+0000","last_deep_scrub":"134'180","last_deep_scrub_stamp":"2024-05-28T14:17:12.337743+0000","last_clean_scrub_stamp":"2024-05-28T14:17:12.337743+0000","objects_scrubbed":52,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:59:12.525974+0000","scrub_duration":371,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":86,"num_write_kb":274434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"134'473","reported_seq":501,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.175642+0000","last_change":"2024-05-28T14:16:58.477958+0000","last_active":"2024-05-28T14:17:05.175642+0000","last_peered":"2024-05-28T14:17:05.175642+0000","last_clean":"2024-05-28T14:17:05.175642+0000","last_became_active":"2024-05-28T14:14:42.744240+0000","last_became_peered":"2024-05-28T14:14:42.744240+0000","last_unstale":"2024-05-28T14:17:05.175642+0000","last_undegraded":"2024-05-28T14:17:05.175642+0000","last_fullsized":"2024-05-28T14:17:05.175642+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'462","last_scrub_stamp":"2024-05-28T14:16:58.477832+0000","last_deep_scrub":"134'462","last_deep_scrub_stamp":"2024-05-28T14:16:58.477832+0000","last_clean_scrub_stamp":"2024-05-28T14:16:58.477832+0000","objects_scrubbed":4,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:07:59.487030+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1782317,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":469,"num_write_kb":1913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"134'226","reported_seq":308,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.163777+0000","last_change":"2024-05-28T14:16:58.319714+0000","last_active":"2024-05-28T14:17:05.163777+0000","last_peered":"2024-05-28T14:17:05.163777+0000","last_clean":"2024-05-28T14:17:05.163777+0000","last_became_active":"2024-05-28T14:14:43.758474+0000","last_became_peered":"2024-05-28T14:14:43.758474+0000","last_unstale":"2024-05-28T14:17:05.163777+0000","last_undegraded":"2024-05-28T14:17:05.163777+0000","last_fullsized":"2024-05-28T14:17:05.163777+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'184","last_scrub_stamp":"2024-05-28T14:16:58.319608+0000","last_deep_scrub":"134'184","last_deep_scrub_stamp":"2024-05-28T14:16:58.319608+0000","last_clean_scrub_stamp":"2024-05-28T14:16:58.319608+0000","objects_scrubbed":59,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:20:23.648649+0000","scrub_duration":279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":54,"num_read_kb":212994,"num_write":119,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:55.004675+0000","last_change":"2024-05-28T14:16:55.004675+0000","last_active":"2024-05-28T14:16:55.004675+0000","last_peered":"2024-05-28T14:16:55.004675+0000","last_clean":"2024-05-28T14:16:55.004675+0000","last_became_active":"2024-05-28T14:14:42.737442+0000","last_became_peered":"2024-05-28T14:14:42.737442+0000","last_unstale":"2024-05-28T14:16:55.004675+0000","last_undegraded":"2024-05-28T14:16:55.004675+0000","last_fullsized":"2024-05-28T14:16:55.004675+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:16:55.004592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:16:55.004592+0000","last_clean_scrub_stamp":"2024-05-28T14:16:55.004592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:46:16.230103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"134'205","reported_seq":294,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.163631+0000","last_change":"2024-05-28T14:17:02.039899+0000","last_active":"2024-05-28T14:17:05.163631+0000","last_peered":"2024-05-28T14:17:05.163631+0000","last_clean":"2024-05-28T14:17:05.163631+0000","last_became_active":"2024-05-28T14:14:43.762049+0000","last_became_peered":"2024-05-28T14:14:43.762049+0000","last_unstale":"2024-05-28T14:17:05.163631+0000","last_undegraded":"2024-05-28T14:17:05.163631+0000","last_fullsized":"2024-05-28T14:17:05.163631+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'162","last_scrub_stamp":"2024-05-28T14:17:02.039741+0000","last_deep_scrub":"134'162","last_deep_scrub_stamp":"2024-05-28T14:17:02.039741+0000","last_clean_scrub_stamp":"2024-05-28T14:17:02.039741+0000","objects_scrubbed":52,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:59:06.451494+0000","scrub_duration":443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":106,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"134'1","reported_seq":29,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:57.627049+0000","last_change":"2024-05-28T14:16:57.627049+0000","last_active":"2024-05-28T14:16:57.627049+0000","last_peered":"2024-05-28T14:16:57.627049+0000","last_clean":"2024-05-28T14:16:57.627049+0000","last_became_active":"2024-05-28T14:14:42.745811+0000","last_became_peered":"2024-05-28T14:14:42.745811+0000","last_unstale":"2024-05-28T14:16:57.627049+0000","last_undegraded":"2024-05-28T14:16:57.627049+0000","last_fullsized":"2024-05-28T14:16:57.627049+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'1","last_scrub_stamp":"2024-05-28T14:16:57.626874+0000","last_deep_scrub":"134'1","last_deep_scrub_stamp":"2024-05-28T14:16:57.626874+0000","last_clean_scrub_stamp":"2024-05-28T14:16:57.626874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:59:08.827926+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"134'233","reported_seq":334,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.174885+0000","last_change":"2024-05-28T14:17:04.559414+0000","last_active":"2024-05-28T14:17:05.174885+0000","last_peered":"2024-05-28T14:17:05.174885+0000","last_clean":"2024-05-28T14:17:05.174885+0000","last_became_active":"2024-05-28T14:14:43.768334+0000","last_became_peered":"2024-05-28T14:14:43.768334+0000","last_unstale":"2024-05-28T14:17:05.174885+0000","last_undegraded":"2024-05-28T14:17:05.174885+0000","last_fullsized":"2024-05-28T14:17:05.174885+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'195","last_scrub_stamp":"2024-05-28T14:17:04.559234+0000","last_deep_scrub":"134'195","last_deep_scrub_stamp":"2024-05-28T14:17:04.559234+0000","last_clean_scrub_stamp":"2024-05-28T14:17:04.559234+0000","objects_scrubbed":62,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:39:35.593783+0000","scrub_duration":1175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852724,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":72,"num_read_kb":282627,"num_write":118,"num_write_kb":360450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"134'12","reported_seq":40,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:57.477241+0000","last_change":"2024-05-28T14:16:57.477241+0000","last_active":"2024-05-28T14:16:57.477241+0000","last_peered":"2024-05-28T14:16:57.477241+0000","last_clean":"2024-05-28T14:16:57.477241+0000","last_became_active":"2024-05-28T14:14:42.742518+0000","last_became_peered":"2024-05-28T14:14:42.742518+0000","last_unstale":"2024-05-28T14:16:57.477241+0000","last_undegraded":"2024-05-28T14:16:57.477241+0000","last_fullsized":"2024-05-28T14:16:57.477241+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'12","last_scrub_stamp":"2024-05-28T14:16:57.477113+0000","last_deep_scrub":"134'12","last_deep_scrub_stamp":"2024-05-28T14:16:57.477113+0000","last_clean_scrub_stamp":"2024-05-28T14:16:57.477113+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T19:28:33.752658+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"134'168","reported_seq":220,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.165915+0000","last_change":"2024-05-28T14:16:59.754485+0000","last_active":"2024-05-28T14:17:05.165915+0000","last_peered":"2024-05-28T14:17:05.165915+0000","last_clean":"2024-05-28T14:17:05.165915+0000","last_became_active":"2024-05-28T14:14:43.761775+0000","last_became_peered":"2024-05-28T14:14:43.761775+0000","last_unstale":"2024-05-28T14:17:05.165915+0000","last_undegraded":"2024-05-28T14:17:05.165915+0000","last_fullsized":"2024-05-28T14:17:05.165915+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'132","last_scrub_stamp":"2024-05-28T14:16:59.754321+0000","last_deep_scrub":"134'132","last_deep_scrub_stamp":"2024-05-28T14:16:59.754321+0000","last_clean_scrub_stamp":"2024-05-28T14:16:59.754321+0000","objects_scrubbed":42,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:42:08.688803+0000","scrub_duration":311,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176160768,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":25,"num_read_kb":102400,"num_write":86,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"134'2","reported_seq":30,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:59.804808+0000","last_change":"2024-05-28T14:16:59.804853+0000","last_active":"2024-05-28T14:16:59.804808+0000","last_peered":"2024-05-28T14:16:59.804808+0000","last_clean":"2024-05-28T14:16:59.804808+0000","last_became_active":"2024-05-28T14:14:42.746098+0000","last_became_peered":"2024-05-28T14:14:42.746098+0000","last_unstale":"2024-05-28T14:16:59.804808+0000","last_undegraded":"2024-05-28T14:16:59.804808+0000","last_fullsized":"2024-05-28T14:16:59.804808+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'2","last_scrub_stamp":"2024-05-28T14:16:59.804722+0000","last_deep_scrub":"134'2","last_deep_scrub_stamp":"2024-05-28T14:16:59.804722+0000","last_clean_scrub_stamp":"2024-05-28T14:16:59.804722+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:29:39.096938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"134'233","reported_seq":344,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.215782+0000","last_change":"2024-05-28T14:17:06.215897+0000","last_active":"2024-05-28T14:17:06.215782+0000","last_peered":"2024-05-28T14:17:06.215782+0000","last_clean":"2024-05-28T14:17:06.215782+0000","last_became_active":"2024-05-28T14:14:43.756138+0000","last_became_peered":"2024-05-28T14:14:43.756138+0000","last_unstale":"2024-05-28T14:17:06.215782+0000","last_undegraded":"2024-05-28T14:17:06.215782+0000","last_fullsized":"2024-05-28T14:17:06.215782+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'233","last_scrub_stamp":"2024-05-28T14:17:06.215662+0000","last_deep_scrub":"134'233","last_deep_scrub_stamp":"2024-05-28T14:17:06.215662+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.215662+0000","objects_scrubbed":63,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:59:15.099144+0000","scrub_duration":1511,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":129,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"134'3","reported_seq":31,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:57.270801+0000","last_change":"2024-05-28T14:16:57.270801+0000","last_active":"2024-05-28T14:16:57.270801+0000","last_peered":"2024-05-28T14:16:57.270801+0000","last_clean":"2024-05-28T14:16:57.270801+0000","last_became_active":"2024-05-28T14:14:42.741754+0000","last_became_peered":"2024-05-28T14:14:42.741754+0000","last_unstale":"2024-05-28T14:16:57.270801+0000","last_undegraded":"2024-05-28T14:16:57.270801+0000","last_fullsized":"2024-05-28T14:16:57.270801+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'3","last_scrub_stamp":"2024-05-28T14:16:57.270717+0000","last_deep_scrub":"134'3","last_deep_scrub_stamp":"2024-05-28T14:16:57.270717+0000","last_clean_scrub_stamp":"2024-05-28T14:16:57.270717+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:24:31.097696+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"134'209","reported_seq":301,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.652289+0000","last_change":"2024-05-28T14:17:05.652362+0000","last_active":"2024-05-28T14:17:05.652289+0000","last_peered":"2024-05-28T14:17:05.652289+0000","last_clean":"2024-05-28T14:17:05.652289+0000","last_became_active":"2024-05-28T14:14:43.757996+0000","last_became_peered":"2024-05-28T14:14:43.757996+0000","last_unstale":"2024-05-28T14:17:05.652289+0000","last_undegraded":"2024-05-28T14:17:05.652289+0000","last_fullsized":"2024-05-28T14:17:05.652289+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'209","last_scrub_stamp":"2024-05-28T14:17:05.652150+0000","last_deep_scrub":"134'209","last_deep_scrub_stamp":"2024-05-28T14:17:05.652150+0000","last_clean_scrub_stamp":"2024-05-28T14:17:05.652150+0000","objects_scrubbed":49,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:37:53.243431+0000","scrub_duration":1103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":63,"num_read_kb":241668,"num_write":101,"num_write_kb":299010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"134'2","reported_seq":30,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:56.711700+0000","last_change":"2024-05-28T14:16:56.711700+0000","last_active":"2024-05-28T14:16:56.711700+0000","last_peered":"2024-05-28T14:16:56.711700+0000","last_clean":"2024-05-28T14:16:56.711700+0000","last_became_active":"2024-05-28T14:14:42.734647+0000","last_became_peered":"2024-05-28T14:14:42.734647+0000","last_unstale":"2024-05-28T14:16:56.711700+0000","last_undegraded":"2024-05-28T14:16:56.711700+0000","last_fullsized":"2024-05-28T14:16:56.711700+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'2","last_scrub_stamp":"2024-05-28T14:16:56.711544+0000","last_deep_scrub":"134'2","last_deep_scrub_stamp":"2024-05-28T14:16:56.711544+0000","last_clean_scrub_stamp":"2024-05-28T14:16:56.711544+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:42:05.646026+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"134'202","reported_seq":312,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.265907+0000","last_change":"2024-05-28T14:17:06.265961+0000","last_active":"2024-05-28T14:17:06.265907+0000","last_peered":"2024-05-28T14:17:06.265907+0000","last_clean":"2024-05-28T14:17:06.265907+0000","last_became_active":"2024-05-28T14:14:43.762407+0000","last_became_peered":"2024-05-28T14:14:43.762407+0000","last_unstale":"2024-05-28T14:17:06.265907+0000","last_undegraded":"2024-05-28T14:17:06.265907+0000","last_fullsized":"2024-05-28T14:17:06.265907+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'202","last_scrub_stamp":"2024-05-28T14:17:06.265765+0000","last_deep_scrub":"134'202","last_deep_scrub_stamp":"2024-05-28T14:17:06.265765+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.265765+0000","objects_scrubbed":51,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:27:30.751330+0000","scrub_duration":795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":82,"num_read_kb":327682,"num_write":107,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:56.869060+0000","last_change":"2024-05-28T14:16:56.869060+0000","last_active":"2024-05-28T14:16:56.869060+0000","last_peered":"2024-05-28T14:16:56.869060+0000","last_clean":"2024-05-28T14:16:56.869060+0000","last_became_active":"2024-05-28T14:14:42.733393+0000","last_became_peered":"2024-05-28T14:14:42.733393+0000","last_unstale":"2024-05-28T14:16:56.869060+0000","last_undegraded":"2024-05-28T14:16:56.869060+0000","last_fullsized":"2024-05-28T14:16:56.869060+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:16:56.868938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:16:56.868938+0000","last_clean_scrub_stamp":"2024-05-28T14:16:56.868938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:30:32.994665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"134'212","reported_seq":326,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.446486+0000","last_change":"2024-05-28T14:17:06.446514+0000","last_active":"2024-05-28T14:17:06.446486+0000","last_peered":"2024-05-28T14:17:06.446486+0000","last_clean":"2024-05-28T14:17:06.446486+0000","last_became_active":"2024-05-28T14:14:43.767351+0000","last_became_peered":"2024-05-28T14:14:43.767351+0000","last_unstale":"2024-05-28T14:17:06.446486+0000","last_undegraded":"2024-05-28T14:17:06.446486+0000","last_fullsized":"2024-05-28T14:17:06.446486+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'212","last_scrub_stamp":"2024-05-28T14:17:06.446416+0000","last_deep_scrub":"134'212","last_deep_scrub_stamp":"2024-05-28T14:17:06.446416+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.446416+0000","objects_scrubbed":63,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:30:01.563294+0000","scrub_duration":1467,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":85,"num_read_kb":348160,"num_write":103,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"134'4","reported_seq":32,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:01.768033+0000","last_change":"2024-05-28T14:17:01.768075+0000","last_active":"2024-05-28T14:17:01.768033+0000","last_peered":"2024-05-28T14:17:01.768033+0000","last_clean":"2024-05-28T14:17:01.768033+0000","last_became_active":"2024-05-28T14:14:42.746273+0000","last_became_peered":"2024-05-28T14:14:42.746273+0000","last_unstale":"2024-05-28T14:17:01.768033+0000","last_undegraded":"2024-05-28T14:17:01.768033+0000","last_fullsized":"2024-05-28T14:17:01.768033+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'4","last_scrub_stamp":"2024-05-28T14:17:01.767957+0000","last_deep_scrub":"134'4","last_deep_scrub_stamp":"2024-05-28T14:17:01.767957+0000","last_clean_scrub_stamp":"2024-05-28T14:17:01.767957+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:52:39.538417+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"134'219","reported_seq":315,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.410569+0000","last_change":"2024-05-28T14:17:06.410658+0000","last_active":"2024-05-28T14:17:06.410569+0000","last_peered":"2024-05-28T14:17:06.410569+0000","last_clean":"2024-05-28T14:17:06.410569+0000","last_became_active":"2024-05-28T14:14:43.770751+0000","last_became_peered":"2024-05-28T14:14:43.770751+0000","last_unstale":"2024-05-28T14:17:06.410569+0000","last_undegraded":"2024-05-28T14:17:06.410569+0000","last_fullsized":"2024-05-28T14:17:06.410569+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'219","last_scrub_stamp":"2024-05-28T14:17:06.410478+0000","last_deep_scrub":"134'219","last_deep_scrub_stamp":"2024-05-28T14:17:06.410478+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.410478+0000","objects_scrubbed":65,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:11:45.588388+0000","scrub_duration":1815,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272629760,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":111,"num_write_kb":348163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:59.558709+0000","last_change":"2024-05-28T14:16:59.558728+0000","last_active":"2024-05-28T14:16:59.558709+0000","last_peered":"2024-05-28T14:16:59.558709+0000","last_clean":"2024-05-28T14:16:59.558709+0000","last_became_active":"2024-05-28T14:14:42.742104+0000","last_became_peered":"2024-05-28T14:14:42.742104+0000","last_unstale":"2024-05-28T14:16:59.558709+0000","last_undegraded":"2024-05-28T14:16:59.558709+0000","last_fullsized":"2024-05-28T14:16:59.558709+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:16:59.558630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:16:59.558630+0000","last_clean_scrub_stamp":"2024-05-28T14:16:59.558630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:05:19.423858+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"134'226","reported_seq":303,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.163849+0000","last_change":"2024-05-28T14:17:00.993854+0000","last_active":"2024-05-28T14:17:05.163849+0000","last_peered":"2024-05-28T14:17:05.163849+0000","last_clean":"2024-05-28T14:17:05.163849+0000","last_became_active":"2024-05-28T14:14:43.755945+0000","last_became_peered":"2024-05-28T14:14:43.755945+0000","last_unstale":"2024-05-28T14:17:05.163849+0000","last_undegraded":"2024-05-28T14:17:05.163849+0000","last_fullsized":"2024-05-28T14:17:05.163849+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'189","last_scrub_stamp":"2024-05-28T14:17:00.993734+0000","last_deep_scrub":"134'189","last_deep_scrub_stamp":"2024-05-28T14:17:00.993734+0000","last_clean_scrub_stamp":"2024-05-28T14:17:00.993734+0000","objects_scrubbed":67,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:59:09.877216+0000","scrub_duration":375,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":49,"num_read_kb":196609,"num_write":111,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:56.499061+0000","last_change":"2024-05-28T14:16:56.499080+0000","last_active":"2024-05-28T14:16:56.499061+0000","last_peered":"2024-05-28T14:16:56.499061+0000","last_clean":"2024-05-28T14:16:56.499061+0000","last_became_active":"2024-05-28T14:14:42.737194+0000","last_became_peered":"2024-05-28T14:14:42.737194+0000","last_unstale":"2024-05-28T14:16:56.499061+0000","last_undegraded":"2024-05-28T14:16:56.499061+0000","last_fullsized":"2024-05-28T14:16:56.499061+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:16:56.498973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:16:56.498973+0000","last_clean_scrub_stamp":"2024-05-28T14:16:56.498973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:39:18.966425+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"134'221","reported_seq":313,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.170052+0000","last_change":"2024-05-28T14:17:04.527035+0000","last_active":"2024-05-28T14:17:05.170052+0000","last_peered":"2024-05-28T14:17:05.170052+0000","last_clean":"2024-05-28T14:17:05.170052+0000","last_became_active":"2024-05-28T14:14:43.757875+0000","last_became_peered":"2024-05-28T14:14:43.757875+0000","last_unstale":"2024-05-28T14:17:05.170052+0000","last_undegraded":"2024-05-28T14:17:05.170052+0000","last_fullsized":"2024-05-28T14:17:05.170052+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'183","last_scrub_stamp":"2024-05-28T14:17:04.526908+0000","last_deep_scrub":"134'183","last_deep_scrub_stamp":"2024-05-28T14:17:04.526908+0000","last_clean_scrub_stamp":"2024-05-28T14:17:04.526908+0000","objects_scrubbed":50,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:58:31.957726+0000","scrub_duration":727,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":64,"num_read_kb":253954,"num_write":110,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"134'1","reported_seq":29,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:56.253257+0000","last_change":"2024-05-28T14:16:56.253301+0000","last_active":"2024-05-28T14:16:56.253257+0000","last_peered":"2024-05-28T14:16:56.253257+0000","last_clean":"2024-05-28T14:16:56.253257+0000","last_became_active":"2024-05-28T14:14:42.738494+0000","last_became_peered":"2024-05-28T14:14:42.738494+0000","last_unstale":"2024-05-28T14:16:56.253257+0000","last_undegraded":"2024-05-28T14:16:56.253257+0000","last_fullsized":"2024-05-28T14:16:56.253257+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'1","last_scrub_stamp":"2024-05-28T14:16:56.253170+0000","last_deep_scrub":"134'1","last_deep_scrub_stamp":"2024-05-28T14:16:56.253170+0000","last_clean_scrub_stamp":"2024-05-28T14:16:56.253170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:46:19.378453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"134'225","reported_seq":293,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.149250+0000","last_change":"2024-05-28T14:17:03.283098+0000","last_active":"2024-05-28T14:17:05.149250+0000","last_peered":"2024-05-28T14:17:05.149250+0000","last_clean":"2024-05-28T14:17:05.149250+0000","last_became_active":"2024-05-28T14:14:43.770253+0000","last_became_peered":"2024-05-28T14:14:43.770253+0000","last_unstale":"2024-05-28T14:17:05.149250+0000","last_undegraded":"2024-05-28T14:17:05.149250+0000","last_fullsized":"2024-05-28T14:17:05.149250+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'185","last_scrub_stamp":"2024-05-28T14:17:03.282916+0000","last_deep_scrub":"134'185","last_deep_scrub_stamp":"2024-05-28T14:17:03.282916+0000","last_clean_scrub_stamp":"2024-05-28T14:17:03.282916+0000","objects_scrubbed":58,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T15:06:42.154268+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":40,"num_read_kb":163840,"num_write":118,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"134'3","reported_seq":31,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:55.997333+0000","last_change":"2024-05-28T14:16:55.997356+0000","last_active":"2024-05-28T14:16:55.997333+0000","last_peered":"2024-05-28T14:16:55.997333+0000","last_clean":"2024-05-28T14:16:55.997333+0000","last_became_active":"2024-05-28T14:14:42.745926+0000","last_became_peered":"2024-05-28T14:14:42.745926+0000","last_unstale":"2024-05-28T14:16:55.997333+0000","last_undegraded":"2024-05-28T14:16:55.997333+0000","last_fullsized":"2024-05-28T14:16:55.997333+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'3","last_scrub_stamp":"2024-05-28T14:16:55.997258+0000","last_deep_scrub":"134'3","last_deep_scrub_stamp":"2024-05-28T14:16:55.997258+0000","last_clean_scrub_stamp":"2024-05-28T14:16:55.997258+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:58:31.432284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"134'218","reported_seq":336,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:05.163135+0000","last_change":"2024-05-28T14:17:04.035776+0000","last_active":"2024-05-28T14:17:05.163135+0000","last_peered":"2024-05-28T14:17:05.163135+0000","last_clean":"2024-05-28T14:17:05.163135+0000","last_became_active":"2024-05-28T14:15:05.975545+0000","last_became_peered":"2024-05-28T14:15:05.975545+0000","last_unstale":"2024-05-28T14:17:05.163135+0000","last_undegraded":"2024-05-28T14:17:05.163135+0000","last_fullsized":"2024-05-28T14:17:05.163135+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'184","last_scrub_stamp":"2024-05-28T14:17:04.035626+0000","last_deep_scrub":"134'184","last_deep_scrub_stamp":"2024-05-28T14:17:04.035626+0000","last_clean_scrub_stamp":"2024-05-28T14:17:04.035626+0000","objects_scrubbed":63,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T01:58:31.466444+0000","scrub_duration":807,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":85,"num_read_kb":348160,"num_write":107,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"134'45","reported_seq":73,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:04.789899+0000","last_change":"2024-05-28T14:16:59.860625+0000","last_active":"2024-05-28T14:17:04.789899+0000","last_peered":"2024-05-28T14:17:04.789899+0000","last_clean":"2024-05-28T14:17:04.789899+0000","last_became_active":"2024-05-28T14:14:42.749348+0000","last_became_peered":"2024-05-28T14:14:42.749348+0000","last_unstale":"2024-05-28T14:17:04.789899+0000","last_undegraded":"2024-05-28T14:17:04.789899+0000","last_fullsized":"2024-05-28T14:17:04.789899+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'42","last_scrub_stamp":"2024-05-28T14:16:59.860473+0000","last_deep_scrub":"134'42","last_deep_scrub_stamp":"2024-05-28T14:16:59.860473+0000","last_clean_scrub_stamp":"2024-05-28T14:16:59.860473+0000","objects_scrubbed":1,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:58:25.798258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":739,"num_omap_keys":13,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"134'195","reported_seq":324,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.234157+0000","last_change":"2024-05-28T14:17:06.234213+0000","last_active":"2024-05-28T14:17:06.234157+0000","last_peered":"2024-05-28T14:17:06.234157+0000","last_clean":"2024-05-28T14:17:06.234157+0000","last_became_active":"2024-05-28T14:14:43.762648+0000","last_became_peered":"2024-05-28T14:14:43.762648+0000","last_unstale":"2024-05-28T14:17:06.234157+0000","last_undegraded":"2024-05-28T14:17:06.234157+0000","last_fullsized":"2024-05-28T14:17:06.234157+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'195","last_scrub_stamp":"2024-05-28T14:17:06.234024+0000","last_deep_scrub":"134'195","last_deep_scrub_stamp":"2024-05-28T14:17:06.234024+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.234024+0000","objects_scrubbed":58,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T21:42:15.168506+0000","scrub_duration":1723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":86,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:55.569259+0000","last_change":"2024-05-28T14:16:55.569277+0000","last_active":"2024-05-28T14:16:55.569259+0000","last_peered":"2024-05-28T14:16:55.569259+0000","last_clean":"2024-05-28T14:16:55.569259+0000","last_became_active":"2024-05-28T14:14:42.743858+0000","last_became_peered":"2024-05-28T14:14:42.743858+0000","last_unstale":"2024-05-28T14:16:55.569259+0000","last_undegraded":"2024-05-28T14:16:55.569259+0000","last_fullsized":"2024-05-28T14:16:55.569259+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:16:55.569178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:16:55.569178+0000","last_clean_scrub_stamp":"2024-05-28T14:16:55.569178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:29:34.861394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"134'189","reported_seq":281,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.475641+0000","last_change":"2024-05-28T14:17:06.475686+0000","last_active":"2024-05-28T14:17:06.475641+0000","last_peered":"2024-05-28T14:17:06.475641+0000","last_clean":"2024-05-28T14:17:06.475641+0000","last_became_active":"2024-05-28T14:15:05.976669+0000","last_became_peered":"2024-05-28T14:15:05.976669+0000","last_unstale":"2024-05-28T14:17:06.475641+0000","last_undegraded":"2024-05-28T14:17:06.475641+0000","last_fullsized":"2024-05-28T14:17:06.475641+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'189","last_scrub_stamp":"2024-05-28T14:17:06.475507+0000","last_deep_scrub":"134'189","last_deep_scrub_stamp":"2024-05-28T14:17:06.475507+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.475507+0000","objects_scrubbed":50,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:18:36.270316+0000","scrub_duration":827,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132541,"num_objects":50,"num_object_clones":1,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":1,"num_read":52,"num_read_kb":208897,"num_write":94,"num_write_kb":282626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8388730,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:58.316411+0000","last_change":"2024-05-28T14:16:58.316448+0000","last_active":"2024-05-28T14:16:58.316411+0000","last_peered":"2024-05-28T14:16:58.316411+0000","last_clean":"2024-05-28T14:16:58.316411+0000","last_became_active":"2024-05-28T14:14:42.748193+0000","last_became_peered":"2024-05-28T14:14:42.748193+0000","last_unstale":"2024-05-28T14:16:58.316411+0000","last_undegraded":"2024-05-28T14:16:58.316411+0000","last_fullsized":"2024-05-28T14:16:58.316411+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:16:58.316288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:16:58.316288+0000","last_clean_scrub_stamp":"2024-05-28T14:16:58.316288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:31:16.512755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"134'226","reported_seq":315,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:11.478344+0000","last_change":"2024-05-28T14:17:11.478395+0000","last_active":"2024-05-28T14:17:11.478344+0000","last_peered":"2024-05-28T14:17:11.478344+0000","last_clean":"2024-05-28T14:17:11.478344+0000","last_became_active":"2024-05-28T14:14:43.767552+0000","last_became_peered":"2024-05-28T14:14:43.767552+0000","last_unstale":"2024-05-28T14:17:11.478344+0000","last_undegraded":"2024-05-28T14:17:11.478344+0000","last_fullsized":"2024-05-28T14:17:11.478344+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'226","last_scrub_stamp":"2024-05-28T14:17:11.478276+0000","last_deep_scrub":"134'226","last_deep_scrub_stamp":"2024-05-28T14:17:11.478276+0000","last_clean_scrub_stamp":"2024-05-28T14:17:11.478276+0000","objects_scrubbed":68,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T16:26:03.046542+0000","scrub_duration":487,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":285212672,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":116,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:58.640658+0000","last_change":"2024-05-28T14:16:58.640677+0000","last_active":"2024-05-28T14:16:58.640658+0000","last_peered":"2024-05-28T14:16:58.640658+0000","last_clean":"2024-05-28T14:16:58.640658+0000","last_became_active":"2024-05-28T14:14:42.746338+0000","last_became_peered":"2024-05-28T14:14:42.746338+0000","last_unstale":"2024-05-28T14:16:58.640658+0000","last_undegraded":"2024-05-28T14:16:58.640658+0000","last_fullsized":"2024-05-28T14:16:58.640658+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:16:58.640583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:16:58.640583+0000","last_clean_scrub_stamp":"2024-05-28T14:16:58.640583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:11:33.963393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"134'248","reported_seq":366,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:06.728922+0000","last_change":"2024-05-28T14:17:06.728985+0000","last_active":"2024-05-28T14:17:06.728922+0000","last_peered":"2024-05-28T14:17:06.728922+0000","last_clean":"2024-05-28T14:17:06.728922+0000","last_became_active":"2024-05-28T14:14:43.760063+0000","last_became_peered":"2024-05-28T14:14:43.760063+0000","last_unstale":"2024-05-28T14:17:06.728922+0000","last_undegraded":"2024-05-28T14:17:06.728922+0000","last_fullsized":"2024-05-28T14:17:06.728922+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'248","last_scrub_stamp":"2024-05-28T14:17:06.728859+0000","last_deep_scrub":"134'248","last_deep_scrub_stamp":"2024-05-28T14:17:06.728859+0000","last_clean_scrub_stamp":"2024-05-28T14:17:06.728859+0000","objects_scrubbed":65,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:33:13.027828+0000","scrub_duration":975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272629760,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":90,"num_read_kb":356355,"num_write":129,"num_write_kb":393217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:58.904937+0000","last_change":"2024-05-28T14:16:58.904956+0000","last_active":"2024-05-28T14:16:58.904937+0000","last_peered":"2024-05-28T14:16:58.904937+0000","last_clean":"2024-05-28T14:16:58.904937+0000","last_became_active":"2024-05-28T14:14:42.749259+0000","last_became_peered":"2024-05-28T14:14:42.749259+0000","last_unstale":"2024-05-28T14:16:58.904937+0000","last_undegraded":"2024-05-28T14:16:58.904937+0000","last_fullsized":"2024-05-28T14:16:58.904937+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:16:58.904862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:16:58.904862+0000","last_clean_scrub_stamp":"2024-05-28T14:16:58.904862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-30T00:49:25.765278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"134'225","reported_seq":332,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:09.057691+0000","last_change":"2024-05-28T14:17:09.057733+0000","last_active":"2024-05-28T14:17:09.057691+0000","last_peered":"2024-05-28T14:17:09.057691+0000","last_clean":"2024-05-28T14:17:09.057691+0000","last_became_active":"2024-05-28T14:14:43.761382+0000","last_became_peered":"2024-05-28T14:14:43.761382+0000","last_unstale":"2024-05-28T14:17:09.057691+0000","last_undegraded":"2024-05-28T14:17:09.057691+0000","last_fullsized":"2024-05-28T14:17:09.057691+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'225","last_scrub_stamp":"2024-05-28T14:17:09.057632+0000","last_deep_scrub":"134'225","last_deep_scrub_stamp":"2024-05-28T14:17:09.057632+0000","last_clean_scrub_stamp":"2024-05-28T14:17:09.057632+0000","objects_scrubbed":66,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T23:38:23.384894+0000","scrub_duration":475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":114,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"134'32","reported_seq":90,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:58.518592+0000","last_change":"2024-05-28T14:16:58.518616+0000","last_active":"2024-05-28T14:16:58.518592+0000","last_peered":"2024-05-28T14:16:58.518592+0000","last_clean":"2024-05-28T14:16:58.518592+0000","last_became_active":"2024-05-28T14:14:42.738369+0000","last_became_peered":"2024-05-28T14:14:42.738369+0000","last_unstale":"2024-05-28T14:16:58.518592+0000","last_undegraded":"2024-05-28T14:16:58.518592+0000","last_fullsized":"2024-05-28T14:16:58.518592+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'32","last_scrub_stamp":"2024-05-28T14:16:58.518528+0000","last_deep_scrub":"134'32","last_deep_scrub_stamp":"2024-05-28T14:16:58.518528+0000","last_clean_scrub_stamp":"2024-05-28T14:16:58.518528+0000","objects_scrubbed":1,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:53:42.108211+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":30,"num_read_kb":31,"num_write":30,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"134'212","reported_seq":323,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:08.038697+0000","last_change":"2024-05-28T14:17:08.038740+0000","last_active":"2024-05-28T14:17:08.038697+0000","last_peered":"2024-05-28T14:17:08.038697+0000","last_clean":"2024-05-28T14:17:08.038697+0000","last_became_active":"2024-05-28T14:14:43.764879+0000","last_became_peered":"2024-05-28T14:14:43.764879+0000","last_unstale":"2024-05-28T14:17:08.038697+0000","last_undegraded":"2024-05-28T14:17:08.038697+0000","last_fullsized":"2024-05-28T14:17:08.038697+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'212","last_scrub_stamp":"2024-05-28T14:17:08.038628+0000","last_deep_scrub":"134'212","last_deep_scrub_stamp":"2024-05-28T14:17:08.038628+0000","last_clean_scrub_stamp":"2024-05-28T14:17:08.038628+0000","objects_scrubbed":66,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T14:56:29.692104+0000","scrub_duration":547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":116,"num_write_kb":372736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:54.227760+0000","last_change":"2024-05-28T14:16:54.227795+0000","last_active":"2024-05-28T14:16:54.227760+0000","last_peered":"2024-05-28T14:16:54.227760+0000","last_clean":"2024-05-28T14:16:54.227760+0000","last_became_active":"2024-05-28T14:14:42.730333+0000","last_became_peered":"2024-05-28T14:14:42.730333+0000","last_unstale":"2024-05-28T14:16:54.227760+0000","last_undegraded":"2024-05-28T14:16:54.227760+0000","last_fullsized":"2024-05-28T14:16:54.227760+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-28T14:16:54.227666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-28T14:16:54.227666+0000","last_clean_scrub_stamp":"2024-05-28T14:16:54.227666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T18:08:18.110965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"134'195","reported_seq":273,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:07.183732+0000","last_change":"2024-05-28T14:17:07.183796+0000","last_active":"2024-05-28T14:17:07.183732+0000","last_peered":"2024-05-28T14:17:07.183732+0000","last_clean":"2024-05-28T14:17:07.183732+0000","last_became_active":"2024-05-28T14:14:43.760828+0000","last_became_peered":"2024-05-28T14:14:43.760828+0000","last_unstale":"2024-05-28T14:17:07.183732+0000","last_undegraded":"2024-05-28T14:17:07.183732+0000","last_fullsized":"2024-05-28T14:17:07.183732+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'195","last_scrub_stamp":"2024-05-28T14:17:07.183648+0000","last_deep_scrub":"134'195","last_deep_scrub_stamp":"2024-05-28T14:17:07.183648+0000","last_clean_scrub_stamp":"2024-05-28T14:17:07.183648+0000","objects_scrubbed":51,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T22:13:41.209716+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":101,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"134'4","reported_seq":32,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:17:01.857137+0000","last_change":"2024-05-28T14:17:01.857137+0000","last_active":"2024-05-28T14:17:01.857137+0000","last_peered":"2024-05-28T14:17:01.857137+0000","last_clean":"2024-05-28T14:17:01.857137+0000","last_became_active":"2024-05-28T14:14:42.751447+0000","last_became_peered":"2024-05-28T14:14:42.751447+0000","last_unstale":"2024-05-28T14:17:01.857137+0000","last_undegraded":"2024-05-28T14:17:01.857137+0000","last_fullsized":"2024-05-28T14:17:01.857137+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'4","last_scrub_stamp":"2024-05-28T14:17:01.856889+0000","last_deep_scrub":"134'4","last_deep_scrub_stamp":"2024-05-28T14:17:01.856889+0000","last_clean_scrub_stamp":"2024-05-28T14:17:01.856889+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T17:27:50.614490+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"134'119","reported_seq":394,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-28T14:16:57.947883+0000","last_change":"2024-05-28T14:16:57.947917+0000","last_active":"2024-05-28T14:16:57.947883+0000","last_peered":"2024-05-28T14:16:57.947883+0000","last_clean":"2024-05-28T14:16:57.947883+0000","last_became_active":"2024-05-28T13:58:12.960698+0000","last_became_peered":"2024-05-28T13:58:12.960698+0000","last_unstale":"2024-05-28T14:16:57.947883+0000","last_undegraded":"2024-05-28T14:16:57.947883+0000","last_fullsized":"2024-05-28T14:16:57.947883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'119","last_scrub_stamp":"2024-05-28T14:16:57.947807+0000","last_deep_scrub":"134'119","last_deep_scrub_stamp":"2024-05-28T14:16:57.947807+0000","last_clean_scrub_stamp":"2024-05-28T14:16:57.947807+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T20:24:31.774786+0000","scrub_duration":79,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":15082718397,"num_objects":3603,"num_object_clones":1,"num_object_copies":7206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":1,"num_read":4365,"num_read_kb":17629245,"num_write":6581,"num_write_kb":20676651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":50331770,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":30165483520,"data_stored":30165436794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13193,"ondisk_log_size":13193,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1788678,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":30,"num_read_kb":31,"num_write":658,"num_write_kb":2002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5739,"num_omap_keys":23,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3637248,"data_stored":3577356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":626,"ondisk_log_size":626,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575079,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3276880,"kb_used_data":3133320,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":91094960,"statfs":{"total":96636764160,"available":93281239040,"internally_reserved":0,"allocated":3208519680,"data_stored":3207559480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":146995658},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575079,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4226636,"kb_used_data":4078916,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":90145204,"statfs":{"total":96636764160,"available":92308688896,"internally_reserved":0,"allocated":4176809984,"data_stored":4175858441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575081,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3125496,"kb_used_data":2995824,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":91246344,"statfs":{"total":96636764160,"available":93436256256,"internally_reserved":0,"allocated":3067723776,"data_stored":3066774417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575080,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4577076,"kb_used_data":4433516,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":89794764,"statfs":{"total":96636764160,"available":91949838336,"internally_reserved":0,"allocated":4539920384,"data_stored":4538974892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575079,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3778236,"kb_used_data":3642996,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":90593604,"statfs":{"total":96636764160,"available":92767850496,"internally_reserved":0,"allocated":3730427904,"data_stored":3729479880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":138475976},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575080,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4894004,"kb_used_data":4740716,"kb_used_omap":1,"kb_used_meta":153278,"kb_avail":89477836,"statfs":{"total":96636764160,"available":91625304064,"internally_reserved":0,"allocated":4854493184,"data_stored":4853548245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":156957125},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575079,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3483436,"kb_used_data":3339876,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90888404,"statfs":{"total":96636764160,"available":93069725696,"internally_reserved":0,"allocated":3420033024,"data_stored":3419095661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575080,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3254644,"kb_used_data":3111084,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":91117196,"statfs":{"total":96636764160,"available":93304008704,"internally_reserved":0,"allocated":3185750016,"data_stored":3184804998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1798144,"data_stored":1787880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1802240,"data_stored":1782955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3183484928,"data_stored":3183476989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3418361856,"data_stored":3418358020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4852813824,"data_stored":4852809988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3728740352,"data_stored":3728736516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4538245120,"data_stored":4538237181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3066044416,"data_stored":3066036664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4173340672,"data_stored":4173332920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3204452352,"data_stored":3204448516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-28T14:17:18.525 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-28T14:17:18.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.717+0000 7f1ba6a00640 1 Processor -- start 2024-05-28T14:17:18.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.717+0000 7f1ba6a00640 1 -- start start 2024-05-28T14:17:18.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1ba6a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1ba00ce3e0 0x7f1ba00b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1ba6a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ba00b2430 0x7f1ba00d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1ba6a00640 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba00cbaa0 con 0x7f1ba015ed70 2024-05-28T14:17:18.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1ba6a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba00cfcd0 con 0x7f1ba00ce3e0 2024-05-28T14:17:18.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1ba6a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba00b1ec0 con 0x7f1ba00b2430 2024-05-28T14:17:18.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9fe00640 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1ba015ed70 0x7f1ba00cdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:40170/0 (socket says 172.21.15.42:40170) 2024-05-28T14:17:18.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9fe00640 1 -- 172.21.15.42:0/1000453122 learned_addr learned my addr 172.21.15.42:0/1000453122 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-28T14:17:18.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b97e00640 1 --2- 172.21.15.42:0/1000453122 >> v2:172.21.15.63:3300/0 conn(0x7f1ba00ce3e0 0x7f1ba00b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 -- 172.21.15.42:0/1000453122 <== mon.0 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1221504786 0 0) 0x7f1b90001120 con 0x7f1ba015ed70 2024-05-28T14:17:18.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1ba4a00640 1 --2- 172.21.15.42:0/1000453122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ba00b2430 0x7f1ba00d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 -- 172.21.15.42:0/1000453122 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b84003530 con 0x7f1ba015ed70 2024-05-28T14:17:18.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 -- 172.21.15.42:0/1000453122 <== mon.0 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2167450459 0 0) 0x7f1b84003530 con 0x7f1ba015ed70 2024-05-28T14:17:18.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 -- 172.21.15.42:0/1000453122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ba00b2430 msgr2=0x7f1ba00d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:17:18.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 --2- 172.21.15.42:0/1000453122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ba00b2430 0x7f1ba00d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 -- 172.21.15.42:0/1000453122 >> v2:172.21.15.63:3300/0 conn(0x7f1ba00ce3e0 msgr2=0x7f1ba00b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:17:18.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 --2- 172.21.15.42:0/1000453122 >> v2:172.21.15.63:3300/0 conn(0x7f1ba00ce3e0 0x7f1ba00b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 -- 172.21.15.42:0/1000453122 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba00b21a0 con 0x7f1ba015ed70 2024-05-28T14:17:18.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 -- 172.21.15.42:0/1000453122 <== mon.0 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1364169496 0 0) 0x7f1b90001850 con 0x7f1ba015ed70 2024-05-28T14:17:18.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 -- 172.21.15.42:0/1000453122 >> v1:172.21.15.42:6789/0 conn(0x7f1ba015ed70 legacy=0x7f1ba00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:18.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 --2- 172.21.15.42:0/1000453122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b840039c0 0x7f1b84003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 --2- 172.21.15.42:0/1000453122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b84004810 0x7f1b84004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 --2- 172.21.15.42:0/1000453122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ba00b2430 0x7f1b840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 -- 172.21.15.42:0/1000453122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba00cfcd0 con 0x7f1b84004810 2024-05-28T14:17:18.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 -- 172.21.15.42:0/1000453122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba00b1ec0 con 0x7f1b840039c0 2024-05-28T14:17:18.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9f400640 1 -- 172.21.15.42:0/1000453122 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8400b9d0 con 0x7f1ba00b2430 2024-05-28T14:17:18.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b97e00640 1 --2- 172.21.15.42:0/1000453122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b84004810 0x7f1b84004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b9fe00640 1 --2- 172.21.15.42:0/1000453122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b840039c0 0x7f1b84003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1ba4a00640 1 --2- 172.21.15.42:0/1000453122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ba00b2430 0x7f1b840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b97e00640 1 -- 172.21.15.42:0/1000453122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ba00b2430 msgr2=0x7f1b840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:17:18.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b97e00640 1 --2- 172.21.15.42:0/1000453122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ba00b2430 0x7f1b840051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b97e00640 1 -- 172.21.15.42:0/1000453122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b840039c0 msgr2=0x7f1b84003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:17:18.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b97e00640 1 --2- 172.21.15.42:0/1000453122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b840039c0 0x7f1b84003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1b97e00640 1 -- 172.21.15.42:0/1000453122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b84003530 con 0x7f1b84004810 2024-05-28T14:17:18.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.721+0000 7f1ba4a00640 1 --2- 172.21.15.42:0/1000453122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ba00b2430 0x7f1b840051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-28T14:17:18.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.725+0000 7f1b97e00640 1 --2- 172.21.15.42:0/1000453122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b84004810 0x7f1b84004c90 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009ae0 tx=0x7f1b8801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9774f772a2af112 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:17:18.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.725+0000 7f1b9f400640 1 -- 172.21.15.42:0/1000453122 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b880096b0 con 0x7f1b84004810 2024-05-28T14:17:18.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.725+0000 7f1b9f400640 1 -- 172.21.15.42:0/1000453122 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b880a64c0 con 0x7f1b84004810 2024-05-28T14:17:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.725+0000 7f1b9f400640 1 -- 172.21.15.42:0/1000453122 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b880a68b0 con 0x7f1b84004810 2024-05-28T14:17:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.725+0000 7f1ba6a00640 1 -- 172.21.15.42:0/1000453122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b84004810 msgr2=0x7f1b84004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.725+0000 7f1ba6a00640 1 --2- 172.21.15.42:0/1000453122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b84004810 0x7f1b84004c90 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009ae0 tx=0x7f1b8801cab0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.725+0000 7f1ba4a00640 1 -- 172.21.15.42:0/1000453122 reap_dead start 2024-05-28T14:17:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.725+0000 7f1ba6a00640 1 -- 172.21.15.42:0/1000453122 shutdown_connections 2024-05-28T14:17:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.725+0000 7f1ba6a00640 1 -- 172.21.15.42:0/1000453122 >> 172.21.15.42:0/1000453122 conn(0x7f1ba009aa50 msgr2=0x7f1ba00b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:17:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.725+0000 7f1ba6a00640 1 -- 172.21.15.42:0/1000453122 shutdown_connections 2024-05-28T14:17:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.725+0000 7f1ba6a00640 1 -- 172.21.15.42:0/1000453122 wait complete. 2024-05-28T14:17:18.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.725+0000 7f1ba6a00640 1 Processor -- start 2024-05-28T14:17:18.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.725+0000 7f1ba6a00640 1 -- start start 2024-05-28T14:17:18.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1ba6a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b84004810 0x7f1ba00b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1ba6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ba00b39a0 0x7f1ba00b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1ba6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ba0155e50 0x7f1ba01562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1ba6a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba00cecf0 con 0x7f1ba00b39a0 2024-05-28T14:17:18.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1ba6a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba00cf960 con 0x7f1ba0155e50 2024-05-28T14:17:18.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1ba6a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba0156810 con 0x7f1b84004810 2024-05-28T14:17:18.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1b97e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ba00b39a0 0x7f1ba00b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1b97e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ba00b39a0 0x7f1ba00b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45886/0 (socket says 172.21.15.42:45886) 2024-05-28T14:17:18.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1b97e00640 1 -- 172.21.15.42:0/2268182864 learned_addr learned my addr 172.21.15.42:0/2268182864 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:17:18.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1b9fe00640 1 --2- 172.21.15.42:0/2268182864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b84004810 0x7f1ba00b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1ba4a00640 1 --2- 172.21.15.42:0/2268182864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ba0155e50 0x7f1ba01562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1b97e00640 1 -- 172.21.15.42:0/2268182864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b84004810 msgr2=0x7f1ba00b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:18.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1b97e00640 1 --2- 172.21.15.42:0/2268182864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b84004810 0x7f1ba00b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1b97e00640 1 -- 172.21.15.42:0/2268182864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ba0155e50 msgr2=0x7f1ba01562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:18.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1b97e00640 1 --2- 172.21.15.42:0/2268182864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ba0155e50 0x7f1ba01562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1b97e00640 1 -- 172.21.15.42:0/2268182864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba00b0e40 con 0x7f1ba00b39a0 2024-05-28T14:17:18.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1b97e00640 1 --2- 172.21.15.42:0/2268182864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ba00b39a0 0x7f1ba00b3e00 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f1b8801c8e0 tx=0x7f1b84005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:17:18.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1b9cc00640 1 -- 172.21.15.42:0/2268182864 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b880be020 con 0x7f1ba00b39a0 2024-05-28T14:17:18.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1b9cc00640 1 -- 172.21.15.42:0/2268182864 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b880a6ac0 con 0x7f1ba00b39a0 2024-05-28T14:17:18.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1b9cc00640 1 -- 172.21.15.42:0/2268182864 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88006040 con 0x7f1ba00b39a0 2024-05-28T14:17:18.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1ba6a00640 1 -- 172.21.15.42:0/2268182864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba00b4510 con 0x7f1ba00b39a0 2024-05-28T14:17:18.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1ba6a00640 1 -- 172.21.15.42:0/2268182864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba00b4ad0 con 0x7f1ba00b39a0 2024-05-28T14:17:18.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.729+0000 7f1ba6a00640 1 -- 172.21.15.42:0/2268182864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba009f650 con 0x7f1ba00b39a0 2024-05-28T14:17:18.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.733+0000 7f1b9cc00640 1 -- 172.21.15.42:0/2268182864 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 51) ==== 90690+0+0 (secure 0 0 0) 0x7f1b880c1020 con 0x7f1ba00b39a0 2024-05-28T14:17:18.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.733+0000 7f1b9cc00640 1 --2- 172.21.15.42:0/2268182864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b7c06a150 0x7f1b7c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:18.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.733+0000 7f1b9cc00640 1 -- 172.21.15.42:0/2268182864 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7f1b88120850 con 0x7f1ba00b39a0 2024-05-28T14:17:18.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.733+0000 7f1b9fe00640 1 --2- 172.21.15.42:0/2268182864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b7c06a150 0x7f1b7c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:18.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.733+0000 7f1b9fe00640 1 --2- 172.21.15.42:0/2268182864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b7c06a150 0x7f1b7c06c610 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1b900078f0 tx=0x7f1b90007ad0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:17:18.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.737+0000 7f1b9cc00640 1 -- 172.21.15.42:0/2268182864 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1b880c8030 con 0x7f1ba00b39a0 2024-05-28T14:17:18.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.905+0000 7f1ba6a00640 1 -- 172.21.15.42:0/2268182864 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f1ba009f400 con 0x7f1ba00b39a0 2024-05-28T14:17:18.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.909+0000 7f1b9cc00640 1 -- 172.21.15.42:0/2268182864 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v30)=0 v30) ==== 123+0+0 (secure 0 0 0) 0x7f1b880167c0 con 0x7f1ba00b39a0 2024-05-28T14:17:18.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.909+0000 7f1b9cc00640 1 -- 172.21.15.42:0/2268182864 <== mon.0 v2:172.21.15.42:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1b880a64c0 con 0x7f1ba00b39a0 2024-05-28T14:17:18.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.909+0000 7f1ba6a00640 1 -- 172.21.15.42:0/2268182864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b7c06a150 msgr2=0x7f1b7c06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:18.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.909+0000 7f1ba6a00640 1 --2- 172.21.15.42:0/2268182864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b7c06a150 0x7f1b7c06c610 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1b900078f0 tx=0x7f1b90007ad0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.909+0000 7f1ba6a00640 1 -- 172.21.15.42:0/2268182864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ba00b39a0 msgr2=0x7f1ba00b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:18.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.909+0000 7f1ba6a00640 1 --2- 172.21.15.42:0/2268182864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ba00b39a0 0x7f1ba00b3e00 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f1b8801c8e0 tx=0x7f1b84005840 comp rx=0 tx=0).stop 2024-05-28T14:17:18.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.909+0000 7f1ba6a00640 1 -- 172.21.15.42:0/2268182864 shutdown_connections 2024-05-28T14:17:18.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.909+0000 7f1ba6a00640 1 --2- 172.21.15.42:0/2268182864 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7f1b7c06a150 0x7f1b7c06c610 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.909+0000 7f1ba6a00640 1 --2- 172.21.15.42:0/2268182864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ba0155e50 0x7f1ba01562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.909+0000 7f1ba6a00640 1 --2- 172.21.15.42:0/2268182864 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ba00b39a0 0x7f1ba00b3e00 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.909+0000 7f1ba6a00640 1 --2- 172.21.15.42:0/2268182864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b84004810 0x7f1ba00b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:18.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.909+0000 7f1ba6a00640 1 -- 172.21.15.42:0/2268182864 >> 172.21.15.42:0/2268182864 conn(0x7f1ba009aa50 msgr2=0x7f1ba00d1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:17:18.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.909+0000 7f1ba6a00640 1 -- 172.21.15.42:0/2268182864 shutdown_connections 2024-05-28T14:17:18.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:18.909+0000 7f1ba6a00640 1 -- 172.21.15.42:0/2268182864 wait complete. 2024-05-28T14:17:18.916 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-28T14:17:18.917 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-28T14:17:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.073+0000 7feaa3a00640 1 Processor -- start 2024-05-28T14:17:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.073+0000 7feaa3a00640 1 -- start start 2024-05-28T14:17:19.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.077+0000 7feaa3a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea9c0d0900 0x7fea9c162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:19.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.077+0000 7feaa3a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fea9c162ab0 0x7fea9c15e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:19.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.077+0000 7feaa3a00640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea9c0d0de0 con 0x7fea9c0ce860 2024-05-28T14:17:19.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.077+0000 7feaa3a00640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fea9c0d19f0 con 0x7fea9c162ab0 2024-05-28T14:17:19.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.077+0000 7feaa3a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea9c0d10c0 con 0x7fea9c0d0900 2024-05-28T14:17:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.077+0000 7fea9be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea9c0d0900 0x7fea9c162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.077+0000 7fea9be00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea9c0d0900 0x7fea9c162570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.42:39800/0 (socket says 172.21.15.42:39800) 2024-05-28T14:17:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.077+0000 7feaa1a00640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fea9c162ab0 0x7fea9c15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.077+0000 7fea9be00640 1 -- 172.21.15.42:0/2219731830 learned_addr learned my addr 172.21.15.42:0/2219731830 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:17:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.077+0000 7feaa1a00640 1 -- 172.21.15.42:0/2219731830 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea9c0d0900 msgr2=0x7fea9c162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-28T14:17:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.077+0000 7feaa1a00640 1 --2- 172.21.15.42:0/2219731830 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea9c0d0900 0x7fea9c162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.077+0000 7feaa1a00640 1 -- 172.21.15.42:0/2219731830 >> v1:172.21.15.63:6789/0 conn(0x7fea9c0ce860 legacy=0x7fea9c06d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:19.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.077+0000 7feaa1a00640 1 -- 172.21.15.42:0/2219731830 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c0d13a0 con 0x7fea9c162ab0 2024-05-28T14:17:19.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.077+0000 7fea9be00640 1 --2- 172.21.15.42:0/2219731830 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea9c0d0900 0x7fea9c162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-28T14:17:19.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.081+0000 7feaa1a00640 1 --2- 172.21.15.42:0/2219731830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea9c162ab0 0x7fea9c15e200 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fea94009d90 tx=0x7fea9401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52ba9bc9e7aac1d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:17:19.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.081+0000 7fea9b400640 1 -- 172.21.15.42:0/2219731830 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea940b2070 con 0x7fea9c162ab0 2024-05-28T14:17:19.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.081+0000 7fea9b400640 1 -- 172.21.15.42:0/2219731830 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fea940a57b0 con 0x7fea9c162ab0 2024-05-28T14:17:19.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.081+0000 7fea9b400640 1 -- 172.21.15.42:0/2219731830 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea940a5ba0 con 0x7fea9c162ab0 2024-05-28T14:17:19.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.081+0000 7feaa3a00640 1 -- 172.21.15.42:0/2219731830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea9c162ab0 msgr2=0x7fea9c15e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:19.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.081+0000 7feaa3a00640 1 --2- 172.21.15.42:0/2219731830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea9c162ab0 0x7fea9c15e200 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fea94009d90 tx=0x7fea9401c8e0 comp rx=0 tx=0).stop 2024-05-28T14:17:19.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.081+0000 7feaa3a00640 1 -- 172.21.15.42:0/2219731830 shutdown_connections 2024-05-28T14:17:19.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.081+0000 7feaa3a00640 1 --2- 172.21.15.42:0/2219731830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea9c162ab0 0x7fea9c15e200 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:19.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.081+0000 7feaa3a00640 1 --2- 172.21.15.42:0/2219731830 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea9c0d0900 0x7fea9c162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:19.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.081+0000 7feaa3a00640 1 -- 172.21.15.42:0/2219731830 >> 172.21.15.42:0/2219731830 conn(0x7fea9c0b4080 msgr2=0x7fea9c0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:17:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.081+0000 7feaa3a00640 1 -- 172.21.15.42:0/2219731830 shutdown_connections 2024-05-28T14:17:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.081+0000 7feaa3a00640 1 -- 172.21.15.42:0/2219731830 wait complete. 2024-05-28T14:17:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.081+0000 7feaa3a00640 1 Processor -- start 2024-05-28T14:17:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.081+0000 7feaa3a00640 1 -- start start 2024-05-28T14:17:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa3a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea9c0ce860 0x7fea9c15fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa3a00640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea9c0d0900 0x7fea9c0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa3a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea9c162ab0 0x7fea9c0cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa3a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c0ccd60 con 0x7fea9c162ab0 2024-05-28T14:17:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa3a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c0ccbe0 con 0x7fea9c0ce860 2024-05-28T14:17:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa3a00640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea9c0ccee0 con 0x7fea9c0d0900 2024-05-28T14:17:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa1a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea9c162ab0 0x7fea9c0cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa1a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea9c162ab0 0x7fea9c0cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45900/0 (socket says 172.21.15.42:45900) 2024-05-28T14:17:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa1a00640 1 -- 172.21.15.42:0/80890354 learned_addr learned my addr 172.21.15.42:0/80890354 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-28T14:17:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7fea9be00640 1 --2- 172.21.15.42:0/80890354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea9c0d0900 0x7fea9c0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa1000640 1 --2- 172.21.15.42:0/80890354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea9c0ce860 0x7fea9c15fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa1a00640 1 -- 172.21.15.42:0/80890354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea9c0d0900 msgr2=0x7fea9c0b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa1a00640 1 --2- 172.21.15.42:0/80890354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea9c0d0900 0x7fea9c0b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa1a00640 1 -- 172.21.15.42:0/80890354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea9c0ce860 msgr2=0x7fea9c15fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa1a00640 1 --2- 172.21.15.42:0/80890354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea9c0ce860 0x7fea9c15fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa1a00640 1 -- 172.21.15.42:0/80890354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c0b8470 con 0x7fea9c162ab0 2024-05-28T14:17:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa1a00640 1 --2- 172.21.15.42:0/80890354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea9c162ab0 0x7fea9c0cbbd0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fea9401af90 tx=0x7fea94019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:17:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7fea98c00640 1 -- 172.21.15.42:0/80890354 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea940b2050 con 0x7fea9c162ab0 2024-05-28T14:17:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7fea98c00640 1 -- 172.21.15.42:0/80890354 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fea940b3040 con 0x7fea9c162ab0 2024-05-28T14:17:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa3a00640 1 -- 172.21.15.42:0/80890354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9c1603a0 con 0x7fea9c162ab0 2024-05-28T14:17:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7fea98c00640 1 -- 172.21.15.42:0/80890354 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea940ad890 con 0x7fea9c162ab0 2024-05-28T14:17:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa3a00640 1 -- 172.21.15.42:0/80890354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9c160960 con 0x7fea9c162ab0 2024-05-28T14:17:19.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.085+0000 7feaa3a00640 1 -- 172.21.15.42:0/80890354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea9c068a90 con 0x7fea9c162ab0 2024-05-28T14:17:19.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.089+0000 7fea98c00640 1 -- 172.21.15.42:0/80890354 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 51) ==== 90690+0+0 (secure 0 0 0) 0x7fea940c0030 con 0x7fea9c162ab0 2024-05-28T14:17:19.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.089+0000 7fea98c00640 1 --2- 172.21.15.42:0/80890354 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea7c06a180 0x7fea7c06c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-28T14:17:19.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.089+0000 7fea98c00640 1 -- 172.21.15.42:0/80890354 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5823+0+0 (secure 0 0 0) 0x7fea9411fdf0 con 0x7fea9c162ab0 2024-05-28T14:17:19.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.089+0000 7feaa1000640 1 --2- 172.21.15.42:0/80890354 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea7c06a180 0x7fea7c06c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-28T14:17:19.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.089+0000 7feaa1000640 1 --2- 172.21.15.42:0/80890354 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea7c06a180 0x7fea7c06c640 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fea90001810 tx=0x7fea90009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-28T14:17:19.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.093+0000 7fea98c00640 1 -- 172.21.15.42:0/80890354 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fea940c7060 con 0x7fea9c162ab0 2024-05-28T14:17:19.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.293+0000 7feaa3a00640 1 -- 172.21.15.42:0/80890354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea9c0cb9b0 con 0x7fea9c162ab0 2024-05-28T14:17:19.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.293+0000 7fea98c00640 1 -- 172.21.15.42:0/80890354 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5188 (secure 0 0 0) 0x7fea9c0b8470 con 0x7fea9c162ab0 2024-05-28T14:17:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-28T14:17:19.279 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":85,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9208,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1217785682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1217785682},{"type":"v1","addr":"172.21.15.63:6834","nonce":1217785682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9813,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1231554897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1231554897},{"type":"v1","addr":"172.21.15.42:6835","nonce":1231554897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9819,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3152913608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3152913608},{"type":"v1","addr":"172.21.15.42:6837","nonce":3152913608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-28T14:14:43.128104+0000","modified":"2024-05-28T14:14:46.159994+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":57851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57851":{"gid":57851,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3508679013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3508679013},{"type":"v1","addr":"172.21.15.63:6835","nonce":3508679013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":57851,"qdb_cluster":[57851]},"id":9}]} 2024-05-28T14:17:19.279 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 85 2024-05-28T14:17:19.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.297+0000 7feaa3a00640 1 -- 172.21.15.42:0/80890354 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea7c06a180 msgr2=0x7fea7c06c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:19.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.297+0000 7feaa3a00640 1 --2- 172.21.15.42:0/80890354 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea7c06a180 0x7fea7c06c640 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fea90001810 tx=0x7fea90009040 comp rx=0 tx=0).stop 2024-05-28T14:17:19.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.297+0000 7feaa3a00640 1 -- 172.21.15.42:0/80890354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea9c162ab0 msgr2=0x7fea9c0cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-28T14:17:19.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.297+0000 7feaa3a00640 1 --2- 172.21.15.42:0/80890354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea9c162ab0 0x7fea9c0cbbd0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fea9401af90 tx=0x7fea94019680 comp rx=0 tx=0).stop 2024-05-28T14:17:19.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.297+0000 7feaa3a00640 1 -- 172.21.15.42:0/80890354 shutdown_connections 2024-05-28T14:17:19.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.297+0000 7feaa3a00640 1 --2- 172.21.15.42:0/80890354 >> [v2:172.21.15.42:6832/1976130233,v1:172.21.15.42:6833/1976130233] conn(0x7fea7c06a180 0x7fea7c06c640 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:19.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.297+0000 7feaa3a00640 1 --2- 172.21.15.42:0/80890354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea9c162ab0 0x7fea9c0cbbd0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:19.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.297+0000 7feaa3a00640 1 --2- 172.21.15.42:0/80890354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea9c0d0900 0x7fea9c0b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:19.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.297+0000 7feaa3a00640 1 --2- 172.21.15.42:0/80890354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea9c0ce860 0x7fea9c15fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-28T14:17:19.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.297+0000 7feaa3a00640 1 -- 172.21.15.42:0/80890354 >> 172.21.15.42:0/80890354 conn(0x7fea9c0b4080 msgr2=0x7fea9c0cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-28T14:17:19.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.297+0000 7feaa3a00640 1 -- 172.21.15.42:0/80890354 shutdown_connections 2024-05-28T14:17:19.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-28T14:17:19.297+0000 7feaa3a00640 1 -- 172.21.15.42:0/80890354 wait complete. 2024-05-28T14:17:19.301 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-28T14:17:19.301 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-28T14:17:19.301 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-28T14:17:19.301 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:17:19.312 INFO:tasks.ceph.mds.a:Stopped 2024-05-28T14:17:19.312 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-28T14:17:19.313 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:17:19.368 INFO:tasks.ceph.mds.c:Stopped 2024-05-28T14:17:19.368 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-28T14:17:19.368 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:17:19.383 INFO:tasks.ceph.mds.b:Stopped 2024-05-28T14:17:19.383 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-28T14:17:19.383 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:17:19.444 INFO:tasks.ceph.mds.d:Stopped 2024-05-28T14:17:19.444 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-28T14:17:19.445 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-28T14:17:19.445 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:17:21.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:17:21.709 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:17:23.921 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-05-28T14:17:25.546 INFO:tasks.ceph.osd.0:Stopped 2024-05-28T14:17:25.547 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-28T14:17:25.547 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:17:30.030 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-05-28T14:17:30.030 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-05-28T14:17:31.648 INFO:tasks.ceph.osd.1:Stopped 2024-05-28T14:17:31.649 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-28T14:17:31.649 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:17:36.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-05-28T14:17:36.040 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-05-28T14:17:37.751 INFO:tasks.ceph.osd.2:Stopped 2024-05-28T14:17:37.751 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-28T14:17:37.752 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:17:41.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-05-28T14:17:41.949 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-05-28T14:17:43.853 INFO:tasks.ceph.osd.3:Stopped 2024-05-28T14:17:43.854 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-28T14:17:43.855 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:17:47.755 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-05-28T14:17:47.756 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-05-28T14:17:49.957 INFO:tasks.ceph.osd.4:Stopped 2024-05-28T14:17:49.958 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-28T14:17:49.958 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:17:50.049 INFO:tasks.ceph.osd.5:Stopped 2024-05-28T14:17:50.049 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-28T14:17:50.049 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:17:51.787 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:17:51.791 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:17:53.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-05-28T14:17:53.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-05-28T14:17:56.151 INFO:tasks.ceph.osd.6:Stopped 2024-05-28T14:17:56.152 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-28T14:17:56.152 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:17:56.253 INFO:tasks.ceph.osd.7:Stopped 2024-05-28T14:17:56.253 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-28T14:17:56.253 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-28T14:17:56.253 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:17:58.767 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-05-28T14:17:58.767 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-05-28T14:18:02.355 INFO:tasks.ceph.mgr.y:Stopped 2024-05-28T14:18:02.356 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-28T14:18:02.356 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:18:02.414 INFO:tasks.ceph.mgr.x:Stopped 2024-05-28T14:18:02.414 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-28T14:18:02.415 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-28T14:18:02.415 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:18:02.438 INFO:tasks.ceph.mon.a:Stopped 2024-05-28T14:18:02.438 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-28T14:18:02.438 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:18:02.456 INFO:tasks.ceph.mon.b:Stopped 2024-05-28T14:18:02.457 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-28T14:18:02.457 INFO:teuthology.orchestra.run:waiting for 300 2024-05-28T14:18:02.517 INFO:tasks.ceph.mon.c:Stopped 2024-05-28T14:18:02.518 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-28T14:18:02.518 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-05-28T14:18:02.560 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:18:02.560 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-28T14:18:02.839 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:18:02.839 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-28T14:18:03.055 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:18:03.055 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-28T14:18:03.266 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:18:03.266 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-28T14:18:03.464 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:18:03.465 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-28T14:18:03.639 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:18:03.639 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-28T14:18:03.768 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-05-28T14:18:03.856 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:18:03.857 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-28T14:18:04.067 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:18:04.068 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-28T14:18:04.384 INFO:tasks.ceph:Archiving mon data... 2024-05-28T14:18:04.385 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074/data/mon.a.tgz 2024-05-28T14:18:04.385 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-05-28T14:18:04.395 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.woGFvLRp0j 2024-05-28T14:18:04.395 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.woGFvLRp0j 2024-05-28T14:18:04.754 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.woGFvLRp0j 2024-05-28T14:18:04.830 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.woGFvLRp0j is 2MB 2024-05-28T14:18:04.912 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.woGFvLRp0j 2024-05-28T14:18:04.921 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074/data/mon.b.tgz 2024-05-28T14:18:04.921 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-05-28T14:18:04.931 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.LTKoAnVCim 2024-05-28T14:18:04.931 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.LTKoAnVCim 2024-05-28T14:18:05.308 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.LTKoAnVCim 2024-05-28T14:18:05.384 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.LTKoAnVCim is 2MB 2024-05-28T14:18:05.474 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.LTKoAnVCim 2024-05-28T14:18:05.483 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074/data/mon.c.tgz 2024-05-28T14:18:05.484 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-05-28T14:18:05.534 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.XIDFtV4ikr 2024-05-28T14:18:05.534 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.XIDFtV4ikr 2024-05-28T14:18:05.903 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.XIDFtV4ikr 2024-05-28T14:18:05.979 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.XIDFtV4ikr is 2MB 2024-05-28T14:18:06.058 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.XIDFtV4ikr 2024-05-28T14:18:06.068 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-28T14:18:06.068 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-28T14:18:06.071 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-28T14:18:06.143 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-28T14:18:06.182 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-28T14:18:06.183 INFO:tasks.ceph:Archiving crash dumps... 2024-05-28T14:18:06.184 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074/remote/smithi042/crash 2024-05-28T14:18:06.185 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-28T14:18:06.207 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074/remote/smithi063/crash 2024-05-28T14:18:06.208 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-28T14:18:06.244 INFO:tasks.ceph:Shutting down logrotate 2024-05-28T14:18:06.245 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:18:06.258 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-28T14:18:06.335 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-28T14:18:06.357 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-28T14:18:06.377 INFO:tasks.ceph:Compressing logs... 2024-05-28T14:18:06.378 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-28T14:18:06.403 DEBUG:teuthology.orchestra.run.smithi063:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-28T14:18:06.440 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41448.log 2024-05-28T14:18:06.440 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42859.log 2024-05-28T14:18:06.441 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.41448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41993.log 2024-05-28T14:18:06.442 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52450.log 2024-05-28T14:18:06.443 INFO:teuthology.orchestra.run.smithi042.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42859.log.gz 2024-05-28T14:18:06.443 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43050.log 2024-05-28T14:18:06.443 INFO:teuthology.orchestra.run.smithi042.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41993.log.gz 2024-05-28T14:18:06.444 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47595.log 2024-05-28T14:18:06.444 INFO:teuthology.orchestra.run.smithi042.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52450.log.gz 2024-05-28T14:18:06.445 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40711.log 2024-05-28T14:18:06.446 INFO:teuthology.orchestra.run.smithi042.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43050.log.gz 2024-05-28T14:18:06.446 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50689.log 2024-05-28T14:18:06.446 INFO:teuthology.orchestra.run.smithi042.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47595.log.gz 2024-05-28T14:18:06.447 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56855.log 2024-05-28T14:18:06.447 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40711.log.gz 2024-05-28T14:18:06.448 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-28T14:18:06.449 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51533.log 2024-05-28T14:18:06.449 INFO:teuthology.orchestra.run.smithi042.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50689.log.gz 2024-05-28T14:18:06.449 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35982.log 2024-05-28T14:18:06.449 INFO:teuthology.orchestra.run.smithi042.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56855.log.gz 2024-05-28T14:18:06.452 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37740.log 2024-05-28T14:18:06.452 INFO:teuthology.orchestra.run.smithi042.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51533.log.gz 2024-05-28T14:18:06.452 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56182.log 2024-05-28T14:18:06.452 INFO:teuthology.orchestra.run.smithi042.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35982.log.gz 2024-05-28T14:18:06.452 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41687.log 2024-05-28T14:18:06.452 INFO:teuthology.orchestra.run.smithi042.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37740.log.gz 2024-05-28T14:18:06.454 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-28T14:18:06.454 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33718.log 2024-05-28T14:18:06.454 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28089.log 2024-05-28T14:18:06.454 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.1.33718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28001.log 2024-05-28T14:18:06.454 INFO:teuthology.orchestra.run.smithi063.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.1.33718.log.gz 2024-05-28T14:18:06.454 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.28089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28041.log 2024-05-28T14:18:06.454 INFO:teuthology.orchestra.run.smithi063.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.28089.log.gz 2024-05-28T14:18:06.455 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.28001.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-28T14:18:06.455 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.28041.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.28001.log.gz 2024-05-28T14:18:06.455 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33839.log 2024-05-28T14:18:06.455 INFO:teuthology.orchestra.run.smithi063.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28041.log.gz 2024-05-28T14:18:06.458 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28137.log 2024-05-28T14:18:06.458 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.3.33839.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-28T14:18:06.458 INFO:teuthology.orchestra.run.smithi063.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.3.33839.log.gz 2024-05-28T14:18:06.458 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.28137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28113.log 2024-05-28T14:18:06.458 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mgr.x.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28137.log.gz 2024-05-28T14:18:06.458 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-28T14:18:06.464 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35299.log 2024-05-28T14:18:06.464 INFO:teuthology.orchestra.run.smithi042.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.56182.log.gz 2024-05-28T14:18:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33891.log 2024-05-28T14:18:06.465 INFO:teuthology.orchestra.run.smithi042.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41687.log.gz 2024-05-28T14:18:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46381.log 2024-05-28T14:18:06.465 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35299.log.gz 2024-05-28T14:18:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.33891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41522.log 2024-05-28T14:18:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46381.log: 90.8% -- replaced with /var/log/ceph/ceph-client.0.33891.log.gz 2024-05-28T14:18:06.465 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50812.log 2024-05-28T14:18:06.466 INFO:teuthology.orchestra.run.smithi042.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46381.log.gz 2024-05-28T14:18:06.466 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37930.log 2024-05-28T14:18:06.466 INFO:teuthology.orchestra.run.smithi042.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41522.log.gz 2024-05-28T14:18:06.466 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57089.log 2024-05-28T14:18:06.466 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50812.log.gz 2024-05-28T14:18:06.466 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52424.log 2024-05-28T14:18:06.466 INFO:teuthology.orchestra.run.smithi042.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37930.log.gz 2024-05-28T14:18:06.466 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36606.log 2024-05-28T14:18:06.467 INFO:teuthology.orchestra.run.smithi042.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57089.log.gz 2024-05-28T14:18:06.467 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37655.log 2024-05-28T14:18:06.467 INFO:teuthology.orchestra.run.smithi042.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52424.log.gz 2024-05-28T14:18:06.467 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38190.log 2024-05-28T14:18:06.467 INFO:teuthology.orchestra.run.smithi042.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36606.log.gz 2024-05-28T14:18:06.467 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40182.log 2024-05-28T14:18:06.467 INFO:teuthology.orchestra.run.smithi042.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37655.log.gz 2024-05-28T14:18:06.467 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47435.log 2024-05-28T14:18:06.468 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38190.log.gz 2024-05-28T14:18:06.468 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.28113.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-28T14:18:06.468 INFO:teuthology.orchestra.run.smithi063.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28113.log.gz 2024-05-28T14:18:06.469 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28185.log 2024-05-28T14:18:06.469 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-28T14:18:06.469 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.28185.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28185.log.gz 2024-05-28T14:18:06.469 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-28T14:18:06.469 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mds.d.log: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-28T14:18:06.469 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28161.log 2024-05-28T14:18:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47278.log 2024-05-28T14:18:06.472 INFO:teuthology.orchestra.run.smithi042.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40182.log.gz 2024-05-28T14:18:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46867.log 2024-05-28T14:18:06.473 INFO:teuthology.orchestra.run.smithi042.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47435.log.gz 2024-05-28T14:18:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42616.log 2024-05-28T14:18:06.473 INFO:teuthology.orchestra.run.smithi042.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47278.log.gz 2024-05-28T14:18:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52528.log 2024-05-28T14:18:06.473 INFO:teuthology.orchestra.run.smithi042.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46867.log.gz 2024-05-28T14:18:06.473 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48674.log 2024-05-28T14:18:06.473 INFO:teuthology.orchestra.run.smithi042.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42616.log.gz 2024-05-28T14:18:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49866.log 2024-05-28T14:18:06.474 INFO:teuthology.orchestra.run.smithi042.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52528.log.gz 2024-05-28T14:18:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50235.log 2024-05-28T14:18:06.474 INFO:teuthology.orchestra.run.smithi042.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48674.log.gz 2024-05-28T14:18:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51102.log 2024-05-28T14:18:06.474 INFO:teuthology.orchestra.run.smithi042.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49866.log.gz 2024-05-28T14:18:06.474 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36658.log 2024-05-28T14:18:06.474 INFO:teuthology.orchestra.run.smithi042.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50235.log.gz 2024-05-28T14:18:06.480 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.28161.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28161.log.gz 2024-05-28T14:18:06.480 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-28T14:18:06.480 INFO:teuthology.orchestra.run.smithi063.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-28T14:18:06.480 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-28T14:18:06.480 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mds.b.log: 94.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-28T14:18:06.480 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-28T14:18:06.480 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28065.log 2024-05-28T14:18:06.481 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55542.log 2024-05-28T14:18:06.482 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51102.log.gz 2024-05-28T14:18:06.482 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49371.log 2024-05-28T14:18:06.482 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36658.log.gz 2024-05-28T14:18:06.482 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37705.log 2024-05-28T14:18:06.482 INFO:teuthology.orchestra.run.smithi042.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55542.log.gz 2024-05-28T14:18:06.482 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53754.log 2024-05-28T14:18:06.482 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49371.log.gz 2024-05-28T14:18:06.483 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37705.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-28T14:18:06.483 INFO:teuthology.orchestra.run.smithi042.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37705.log.gz 2024-05-28T14:18:06.483 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54100.log 2024-05-28T14:18:06.483 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53754.log.gz 2024-05-28T14:18:06.483 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40208.log 2024-05-28T14:18:06.483 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52839.log 2024-05-28T14:18:06.483 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54100.log.gz 2024-05-28T14:18:06.483 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49475.log 2024-05-28T14:18:06.483 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40208.log.gz 2024-05-28T14:18:06.484 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52839.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52839.log.gz 2024-05-28T14:18:06.484 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33526.log 2024-05-28T14:18:06.486 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39157.log 2024-05-28T14:18:06.486 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49475.log.gz 2024-05-28T14:18:06.486 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33526.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33526.log.gz 2024-05-28T14:18:06.486 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50105.log 2024-05-28T14:18:06.487 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46225.log 2024-05-28T14:18:06.487 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39157.log.gz 2024-05-28T14:18:06.487 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43876.log 2024-05-28T14:18:06.487 INFO:teuthology.orchestra.run.smithi042.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50105.log.gz 2024-05-28T14:18:06.487 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47985.log 2024-05-28T14:18:06.488 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.28065.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.28065.log.gz 2024-05-28T14:18:06.489 INFO:teuthology.orchestra.run.smithi042.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46225.log.gz 2024-05-28T14:18:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35247.log 2024-05-28T14:18:06.489 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43876.log.gz 2024-05-28T14:18:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47513.log 2024-05-28T14:18:06.489 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35247.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.47985.log.gz 2024-05-28T14:18:06.490 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54315.log 2024-05-28T14:18:06.490 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35247.log.gz 2024-05-28T14:18:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.47513.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46511.log 2024-05-28T14:18:06.491 INFO:teuthology.orchestra.run.smithi042.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47513.log.gz 2024-05-28T14:18:06.491 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33552.log 2024-05-28T14:18:06.492 INFO:teuthology.orchestra.run.smithi042.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54315.log.gz 2024-05-28T14:18:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46511.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46511.log.gz 2024-05-28T14:18:06.492 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44785.log 2024-05-28T14:18:06.493 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44265.log 2024-05-28T14:18:06.494 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33552.log.gz 2024-05-28T14:18:06.494 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44291.log 2024-05-28T14:18:06.494 INFO:teuthology.orchestra.run.smithi042.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44785.log.gz 2024-05-28T14:18:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33578.log 2024-05-28T14:18:06.496 INFO:teuthology.orchestra.run.smithi042.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44265.log.gz 2024-05-28T14:18:06.496 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35193.log 2024-05-28T14:18:06.496 INFO:teuthology.orchestra.run.smithi042.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44291.log.gz 2024-05-28T14:18:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54909.log 2024-05-28T14:18:06.497 INFO:teuthology.orchestra.run.smithi042.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33578.log.gz 2024-05-28T14:18:06.498 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35193.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35193.log.gz 2024-05-28T14:18:06.498 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52398.log 2024-05-28T14:18:06.499 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47023.log 2024-05-28T14:18:06.499 INFO:teuthology.orchestra.run.smithi042.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54909.log.gz 2024-05-28T14:18:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52398.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52398.log.gz 2024-05-28T14:18:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34348.log 2024-05-28T14:18:06.501 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37603.log 2024-05-28T14:18:06.501 INFO:teuthology.orchestra.run.smithi042.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47023.log.gz 2024-05-28T14:18:06.502 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32069.log 2024-05-28T14:18:06.502 INFO:teuthology.orchestra.run.smithi042.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34348.log.gz 2024-05-28T14:18:06.503 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41056.log 2024-05-28T14:18:06.503 INFO:teuthology.orchestra.run.smithi042.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37603.log.gz 2024-05-28T14:18:06.504 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32920.log 2024-05-28T14:18:06.504 INFO:teuthology.orchestra.run.smithi042.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32069.log.gz 2024-05-28T14:18:06.505 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56959.log 2024-05-28T14:18:06.505 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41056.log.gz 2024-05-28T14:18:06.506 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45941.log 2024-05-28T14:18:06.506 INFO:teuthology.orchestra.run.smithi042.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32920.log.gz 2024-05-28T14:18:06.507 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37525.log 2024-05-28T14:18:06.507 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56959.log.gz 2024-05-28T14:18:06.508 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45941.log: gzip -5 88.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.45941.log.gz -- 2024-05-28T14:18:06.508 INFO:teuthology.orchestra.run.smithi042.stderr: /var/log/ceph/ceph-client.admin.37904.log 2024-05-28T14:18:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55977.log 2024-05-28T14:18:06.510 INFO:teuthology.orchestra.run.smithi042.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37525.log.gz 2024-05-28T14:18:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32616.log 2024-05-28T14:18:06.510 INFO:teuthology.orchestra.run.smithi042.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37904.log.gz 2024-05-28T14:18:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57193.log/var/log/ceph/ceph-client.admin.55977.log: 2024-05-28T14:18:06.512 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55977.log.gz 2024-05-28T14:18:06.512 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44865.log 2024-05-28T14:18:06.512 INFO:teuthology.orchestra.run.smithi042.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32616.log.gz 2024-05-28T14:18:06.513 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48139.log 2024-05-28T14:18:06.513 INFO:teuthology.orchestra.run.smithi042.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57193.log.gz 2024-05-28T14:18:06.514 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45030.log 2024-05-28T14:18:06.515 INFO:teuthology.orchestra.run.smithi042.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44865.log.gz 2024-05-28T14:18:06.515 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48139.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-28T14:18:06.516 INFO:teuthology.orchestra.run.smithi042.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48139.log.gz 2024-05-28T14:18:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38665.log 2024-05-28T14:18:06.516 INFO:teuthology.orchestra.run.smithi042.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45030.log.gz 2024-05-28T14:18:06.517 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43076.log 2024-05-28T14:18:06.518 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45058.log 2024-05-28T14:18:06.518 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38665.log.gz 2024-05-28T14:18:06.519 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43625.log 2024-05-28T14:18:06.520 INFO:teuthology.orchestra.run.smithi042.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43076.log.gz 2024-05-28T14:18:06.521 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55490.log 2024-05-28T14:18:06.521 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45058.log.gz 2024-05-28T14:18:06.521 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41212.log 2024-05-28T14:18:06.522 INFO:teuthology.orchestra.run.smithi042.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43625.log.gz 2024-05-28T14:18:06.523 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47647.log 2024-05-28T14:18:06.523 INFO:teuthology.orchestra.run.smithi042.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55490.log.gz 2024-05-28T14:18:06.524 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46971.log 2024-05-28T14:18:06.524 INFO:teuthology.orchestra.run.smithi042.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41212.log.gz 2024-05-28T14:18:06.525 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32913.log 2024-05-28T14:18:06.525 INFO:teuthology.orchestra.run.smithi042.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47647.log.gz 2024-05-28T14:18:06.526 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51925.log 2024-05-28T14:18:06.526 INFO:teuthology.orchestra.run.smithi042.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46971.log.gz 2024-05-28T14:18:06.527 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41134.log 2024-05-28T14:18:06.527 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32913.log: /var/log/ceph/ceph-client.admin.51925.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32913.log.gz 2024-05-28T14:18:06.528 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32907.log 2024-05-28T14:18:06.528 INFO:teuthology.orchestra.run.smithi042.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51925.log.gz 2024-05-28T14:18:06.529 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41650.log 2024-05-28T14:18:06.529 INFO:teuthology.orchestra.run.smithi042.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41134.log.gz 2024-05-28T14:18:06.530 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46433.log 2024-05-28T14:18:06.530 INFO:teuthology.orchestra.run.smithi042.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32907.log.gz 2024-05-28T14:18:06.531 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45004.log 2024-05-28T14:18:06.531 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41650.log.gz 2024-05-28T14:18:06.532 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41392.log 2024-05-28T14:18:06.533 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46433.log.gz 2024-05-28T14:18:06.533 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45163.log 2024-05-28T14:18:06.534 INFO:teuthology.orchestra.run.smithi042.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45004.log.gz 2024-05-28T14:18:06.534 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41392.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41392.log.gz 2024-05-28T14:18:06.535 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47094.log 2024-05-28T14:18:06.535 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32717.log 2024-05-28T14:18:06.536 INFO:teuthology.orchestra.run.smithi042.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45163.log.gz 2024-05-28T14:18:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34270.log 2024-05-28T14:18:06.537 INFO:teuthology.orchestra.run.smithi042.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47094.log.gz 2024-05-28T14:18:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50209.log 2024-05-28T14:18:06.538 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32717.log.gz 2024-05-28T14:18:06.539 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45262.log 2024-05-28T14:18:06.539 INFO:teuthology.orchestra.run.smithi042.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34270.log.gz 2024-05-28T14:18:06.540 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47487.log 2024-05-28T14:18:06.540 INFO:teuthology.orchestra.run.smithi042.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50209.log.gz 2024-05-28T14:18:06.541 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34608.log 2024-05-28T14:18:06.541 INFO:teuthology.orchestra.run.smithi042.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45262.log.gz 2024-05-28T14:18:06.542 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53409.log 2024-05-28T14:18:06.542 INFO:teuthology.orchestra.run.smithi042.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47487.log.gz 2024-05-28T14:18:06.543 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32929.log 2024-05-28T14:18:06.543 INFO:teuthology.orchestra.run.smithi042.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34608.log.gz 2024-05-28T14:18:06.544 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51821.log 2024-05-28T14:18:06.545 INFO:teuthology.orchestra.run.smithi042.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53409.log.gz 2024-05-28T14:18:06.545 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47539.log 2024-05-28T14:18:06.546 INFO:teuthology.orchestra.run.smithi042.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32929.log.gz 2024-05-28T14:18:06.546 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40974.log 2024-05-28T14:18:06.547 INFO:teuthology.orchestra.run.smithi042.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51821.log.gz 2024-05-28T14:18:06.547 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47539.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47539.log.gz 2024-05-28T14:18:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43768.log 2024-05-28T14:18:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41966.log 2024-05-28T14:18:06.549 INFO:teuthology.orchestra.run.smithi042.stderr: 88.9%/var/log/ceph/ceph-client.admin.43768.log: -- replaced with /var/log/ceph/ceph-client.admin.40974.log.gz 2024-05-28T14:18:06.549 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39285.log 2024-05-28T14:18:06.550 INFO:teuthology.orchestra.run.smithi042.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43768.log.gz 2024-05-28T14:18:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41797.log 2024-05-28T14:18:06.551 INFO:teuthology.orchestra.run.smithi042.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41966.log.gz 2024-05-28T14:18:06.551 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50419.log 2024-05-28T14:18:06.552 INFO:teuthology.orchestra.run.smithi042.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39285.log.gz 2024-05-28T14:18:06.553 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42276.log 87.9% 2024-05-28T14:18:06.553 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41797.log.gz 2024-05-28T14:18:06.554 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41496.log 2024-05-28T14:18:06.554 INFO:teuthology.orchestra.run.smithi042.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50419.log.gz 2024-05-28T14:18:06.555 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51206.log 2024-05-28T14:18:06.555 INFO:teuthology.orchestra.run.smithi042.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42276.log.gz 2024-05-28T14:18:06.556 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54204.log 2024-05-28T14:18:06.556 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41496.log.gz 2024-05-28T14:18:06.557 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55464.log 2024-05-28T14:18:06.558 INFO:teuthology.orchestra.run.smithi042.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51206.log.gz 2024-05-28T14:18:06.558 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32590.log 2024-05-28T14:18:06.559 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54204.log: /var/log/ceph/ceph-client.admin.55464.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54204.log.gz 2024-05-28T14:18:06.559 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55400.log 2024-05-28T14:18:06.560 INFO:teuthology.orchestra.run.smithi042.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55464.log.gz 2024-05-28T14:18:06.560 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39552.log 2024-05-28T14:18:06.561 INFO:teuthology.orchestra.run.smithi042.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32590.log.gz 2024-05-28T14:18:06.561 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51715.log 2024-05-28T14:18:06.562 INFO:teuthology.orchestra.run.smithi042.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55400.log.gz 2024-05-28T14:18:06.562 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32248.log 2024-05-28T14:18:06.563 INFO:teuthology.orchestra.run.smithi042.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39552.log.gz 2024-05-28T14:18:06.563 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38138.log 2024-05-28T14:18:06.564 INFO:teuthology.orchestra.run.smithi042.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51715.log.gz 2024-05-28T14:18:06.565 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35930.log 2024-05-28T14:18:06.565 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38138.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32248.log.gz 2024-05-28T14:18:06.566 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32167.log 2024-05-28T14:18:06.566 INFO:teuthology.orchestra.run.smithi042.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38138.log.gz 2024-05-28T14:18:06.567 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50053.log 2024-05-28T14:18:06.567 INFO:teuthology.orchestra.run.smithi042.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35930.log.gz 2024-05-28T14:18:06.568 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45783.log 2024-05-28T14:18:06.568 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32167.log.gz 2024-05-28T14:18:06.569 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42358.log 2024-05-28T14:18:06.569 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50053.log.gz 2024-05-28T14:18:06.570 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34192.log 2024-05-28T14:18:06.570 INFO:teuthology.orchestra.run.smithi042.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45783.log.gz 2024-05-28T14:18:06.571 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34296.log 2024-05-28T14:18:06.572 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42358.log: /var/log/ceph/ceph-client.admin.34192.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42358.log.gz 2024-05-28T14:18:06.572 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46121.log 2024-05-28T14:18:06.572 INFO:teuthology.orchestra.run.smithi042.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34192.log.gz 2024-05-28T14:18:06.573 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35351.log 2024-05-28T14:18:06.573 INFO:teuthology.orchestra.run.smithi042.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34296.log.gz 2024-05-28T14:18:06.574 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53728.log 2024-05-28T14:18:06.575 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46121.log: /var/log/ceph/ceph-client.admin.35351.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46121.log.gz 2024-05-28T14:18:06.575 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33656.log 2024-05-28T14:18:06.575 INFO:teuthology.orchestra.run.smithi042.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35351.log.gz 2024-05-28T14:18:06.576 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36060.log 2024-05-28T14:18:06.577 INFO:teuthology.orchestra.run.smithi042.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53728.log.gz 2024-05-28T14:18:06.577 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38034.log 2024-05-28T14:18:06.578 INFO:teuthology.orchestra.run.smithi042.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33656.log.gz 2024-05-28T14:18:06.578 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48728.log 2024-05-28T14:18:06.579 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36060.log.gz 2024-05-28T14:18:06.580 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42642.log 2024-05-28T14:18:06.580 INFO:teuthology.orchestra.run.smithi042.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38034.log.gz 2024-05-28T14:18:06.581 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28039.log 2024-05-28T14:18:06.581 INFO:teuthology.orchestra.run.smithi042.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48728.log.gz 2024-05-28T14:18:06.582 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37956.log 2024-05-28T14:18:06.582 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42642.log.gz 2024-05-28T14:18:06.583 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35219.log 2024-05-28T14:18:06.583 INFO:teuthology.orchestra.run.smithi042.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.28039.log.gz 2024-05-28T14:18:06.584 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45835.log 2024-05-28T14:18:06.584 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37956.log: /var/log/ceph/ceph-client.admin.35219.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37956.log.gz 2024-05-28T14:18:06.585 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36994.log 2024-05-28T14:18:06.585 INFO:teuthology.orchestra.run.smithi042.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35219.log.gz 2024-05-28T14:18:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53650.log 2024-05-28T14:18:06.586 INFO:teuthology.orchestra.run.smithi042.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45835.log.gz 2024-05-28T14:18:06.587 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5/var/log/ceph/ceph-client.admin.36994.log: --verbose -- /var/log/ceph/ceph-client.admin.37766.log 2024-05-28T14:18:06.587 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53650.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36994.log.gz 2024-05-28T14:18:06.588 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50261.log 2024-05-28T14:18:06.588 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53650.log.gz 2024-05-28T14:18:06.589 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36034.log 2024-05-28T14:18:06.589 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37766.log.gz 2024-05-28T14:18:06.590 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53275.log 2024-05-28T14:18:06.590 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36034.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50261.log.gz 2024-05-28T14:18:06.591 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42138.log 2024-05-28T14:18:06.591 INFO:teuthology.orchestra.run.smithi042.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36034.log.gz 2024-05-28T14:18:06.592 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51256.log 2024-05-28T14:18:06.592 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53275.log.gz 2024-05-28T14:18:06.593 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38086.log 2024-05-28T14:18:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51256.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42138.log.gz 2024-05-28T14:18:06.594 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47122.log 2024-05-28T14:18:06.595 INFO:teuthology.orchestra.run.smithi042.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51256.log.gz 2024-05-28T14:18:06.595 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42945.log 2024-05-28T14:18:06.596 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38086.log.gz 2024-05-28T14:18:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.47122.log: /var/log/ceph/ceph-client.admin.40796.log 2024-05-28T14:18:06.597 INFO:teuthology.orchestra.run.smithi042.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47122.log.gz 2024-05-28T14:18:06.597 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39520.log 2024-05-28T14:18:06.598 INFO:teuthology.orchestra.run.smithi042.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42945.log.gz 2024-05-28T14:18:06.598 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37818.log 2024-05-28T14:18:06.599 INFO:teuthology.orchestra.run.smithi042.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40796.log.gz 2024-05-28T14:18:06.599 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.39520.log: -- /var/log/ceph/ceph-client.admin.53676.log 2024-05-28T14:18:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37818.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39520.log.gz 2024-05-28T14:18:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49709.log 2024-05-28T14:18:06.601 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37818.log.gz 2024-05-28T14:18:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36554.log 2024-05-28T14:18:06.602 INFO:teuthology.orchestra.run.smithi042.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53676.log.gz 2024-05-28T14:18:06.602 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.34036.log 2024-05-28T14:18:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49709.log: /var/log/ceph/ceph-client.admin.36554.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49709.log.gz 2024-05-28T14:18:06.603 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37020.log 2024-05-28T14:18:06.604 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36554.log.gz 2024-05-28T14:18:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.2.34036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48466.log 2024-05-28T14:18:06.605 INFO:teuthology.orchestra.run.smithi042.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.2.34036.log.gz 2024-05-28T14:18:06.606 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39105.log 2024-05-28T14:18:06.606 INFO:teuthology.orchestra.run.smithi042.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37020.log.gz 2024-05-28T14:18:06.607 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40294.log 2024-05-28T14:18:06.607 INFO:teuthology.orchestra.run.smithi042.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48466.log.gz 2024-05-28T14:18:06.608 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53140.log 2024-05-28T14:18:06.608 INFO:teuthology.orchestra.run.smithi042.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39105.log.gz 2024-05-28T14:18:06.609 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37046.log 2024-05-28T14:18:06.609 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40294.log: /var/log/ceph/ceph-client.admin.53140.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40294.log.gz 2024-05-28T14:18:06.610 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38821.log 2024-05-28T14:18:06.610 INFO:teuthology.orchestra.run.smithi042.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53140.log.gz 2024-05-28T14:18:06.611 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52502.log 2024-05-28T14:18:06.611 INFO:teuthology.orchestra.run.smithi042.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37046.log.gz 2024-05-28T14:18:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42302.log 2024-05-28T14:18:06.613 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38821.log: /var/log/ceph/ceph-client.admin.52502.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38821.log.gz 2024-05-28T14:18:06.613 INFO:teuthology.orchestra.run.smithi042.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52502.log.gzgzip 2024-05-28T14:18:06.613 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43981.log 2024-05-28T14:18:06.614 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37444.log 2024-05-28T14:18:06.615 INFO:teuthology.orchestra.run.smithi042.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42302.log.gz 2024-05-28T14:18:06.615 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42564.log 2024-05-28T14:18:06.616 INFO:teuthology.orchestra.run.smithi042.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43981.log.gz 2024-05-28T14:18:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35141.log 2024-05-28T14:18:06.617 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37444.log.gz 2024-05-28T14:18:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54965.log 2024-05-28T14:18:06.618 INFO:teuthology.orchestra.run.smithi042.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42564.log.gz 2024-05-28T14:18:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48165.log 2024-05-28T14:18:06.619 INFO:teuthology.orchestra.run.smithi042.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35141.log.gz 2024-05-28T14:18:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44483.log 2024-05-28T14:18:06.621 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.54965.log: /var/log/ceph/ceph-client.admin.48165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43206.log 2024-05-28T14:18:06.621 INFO:teuthology.orchestra.run.smithi042.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48165.log.gz 2024-05-28T14:18:06.622 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50996.log 2024-05-28T14:18:06.622 INFO:teuthology.orchestra.run.smithi042.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44483.log.gz 2024-05-28T14:18:06.623 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32720.log 2024-05-28T14:18:06.623 INFO:teuthology.orchestra.run.smithi042.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43206.log.gz 2024-05-28T14:18:06.624 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50996.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-28T14:18:06.624 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50996.log.gz 2024-05-28T14:18:06.625 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52877.log 2024-05-28T14:18:06.625 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32720.log.gz 2024-05-28T14:18:06.626 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50157.log 2024-05-28T14:18:06.628 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40477.log 2024-05-28T14:18:06.628 INFO:teuthology.orchestra.run.smithi042.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52877.log.gz 2024-05-28T14:18:06.628 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49553.log 2024-05-28T14:18:06.629 INFO:teuthology.orchestra.run.smithi042.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50157.log.gz 2024-05-28T14:18:06.630 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42971.log 2024-05-28T14:18:06.630 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40477.log.gz 2024-05-28T14:18:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46589.log 2024-05-28T14:18:06.631 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49553.log.gz 2024-05-28T14:18:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54833.log 2024-05-28T14:18:06.632 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42971.log.gz 2024-05-28T14:18:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46563.log 2024-05-28T14:18:06.634 INFO:teuthology.orchestra.run.smithi042.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46589.log.gz 2024-05-28T14:18:06.634 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56257.log 2024-05-28T14:18:06.635 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54833.log.gz 2024-05-28T14:18:06.636 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49945.log 2024-05-28T14:18:06.636 INFO:teuthology.orchestra.run.smithi042.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46563.log.gz 2024-05-28T14:18:06.637 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47621.log 2024-05-28T14:18:06.637 INFO:teuthology.orchestra.run.smithi042.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56257.log.gz 2024-05-28T14:18:06.638 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49181.log 2024-05-28T14:18:06.638 INFO:teuthology.orchestra.run.smithi042.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.49945.log.gz 2024-05-28T14:18:06.639 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47621.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47621.log.gz 2024-05-28T14:18:06.639 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34928.log 2024-05-28T14:18:06.640 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5/var/log/ceph/ceph-client.admin.49181.log: --verbose -- /var/log/ceph/ceph-client.admin.55896.log 2024-05-28T14:18:06.641 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49181.log.gz 2024-05-28T14:18:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35430.log 2024-05-28T14:18:06.642 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34928.log.gz 2024-05-28T14:18:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32642.log 2024-05-28T14:18:06.643 INFO:teuthology.orchestra.run.smithi042.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55896.log.gz 2024-05-28T14:18:06.644 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54651.log 2024-05-28T14:18:06.645 INFO:teuthology.orchestra.run.smithi042.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35430.log.gz 2024-05-28T14:18:06.645 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49657.log 2024-05-28T14:18:06.646 INFO:teuthology.orchestra.run.smithi042.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32642.log.gz 2024-05-28T14:18:06.646 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36788.log 2024-05-28T14:18:06.647 INFO:teuthology.orchestra.run.smithi042.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54651.log.gz 2024-05-28T14:18:06.648 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43310.log 2024-05-28T14:18:06.648 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49657.log.gz 2024-05-28T14:18:06.649 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57141.log 2024-05-28T14:18:06.649 INFO:teuthology.orchestra.run.smithi042.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36788.log.gz 2024-05-28T14:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48570.log 2024-05-28T14:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43310.log.gz 2024-05-28T14:18:06.651 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46251.log 2024-05-28T14:18:06.651 INFO:teuthology.orchestra.run.smithi042.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57141.log.gz 2024-05-28T14:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48255.log 2024-05-28T14:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48570.log.gz 2024-05-28T14:18:06.654 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54755.log 2024-05-28T14:18:06.654 INFO:teuthology.orchestra.run.smithi042.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46251.log.gz 2024-05-28T14:18:06.655 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37577.log 2024-05-28T14:18:06.655 INFO:teuthology.orchestra.run.smithi042.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48255.log.gz 2024-05-28T14:18:06.656 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53589.log 2024-05-28T14:18:06.657 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54755.log.gz 2024-05-28T14:18:06.657 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41186.log 2024-05-28T14:18:06.658 INFO:teuthology.orchestra.run.smithi042.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37577.log.gz 2024-05-28T14:18:06.658 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47777.log 2024-05-28T14:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53589.log.gz 2024-05-28T14:18:06.660 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43598.log 2024-05-28T14:18:06.660 INFO:teuthology.orchestra.run.smithi042.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41186.log.gz 2024-05-28T14:18:06.661 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32722.log 2024-05-28T14:18:06.661 INFO:teuthology.orchestra.run.smithi042.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47777.log.gz 2024-05-28T14:18:06.662 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34556.log 2024-05-28T14:18:06.662 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43598.log.gz 2024-05-28T14:18:06.663 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39131.log 2024-05-28T14:18:06.664 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32722.log.gz 2024-05-28T14:18:06.666 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34504.log 2024-05-28T14:18:06.666 INFO:teuthology.orchestra.run.smithi042.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34556.log.gz 2024-05-28T14:18:06.666 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52714.log 2024-05-28T14:18:06.667 INFO:teuthology.orchestra.run.smithi042.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39131.log.gz 2024-05-28T14:18:06.667 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54438.log 2024-05-28T14:18:06.668 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34504.log.gz 2024-05-28T14:18:06.668 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.52714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45915.log 2024-05-28T14:18:06.670 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54438.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54438.log.gz 2024-05-28T14:18:06.671 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45915.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45915.log.gz 2024-05-28T14:18:06.671 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46997.log 2024-05-28T14:18:06.672 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39757.log 2024-05-28T14:18:06.674 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46997.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46997.log.gz 2024-05-28T14:18:06.675 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39757.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39757.log.gz 2024-05-28T14:18:06.675 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53301.log 2024-05-28T14:18:06.676 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34634.log 2024-05-28T14:18:06.678 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53301.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53301.log.gz 2024-05-28T14:18:06.679 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34634.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34634.log.gz 2024-05-28T14:18:06.679 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46729.log 2024-05-28T14:18:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37878.log 2024-05-28T14:18:06.682 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46729.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46729.log.gz 2024-05-28T14:18:06.683 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37878.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37878.log.gz 2024-05-28T14:18:06.683 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46893.log 2024-05-28T14:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40268.log 2024-05-28T14:18:06.686 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46893.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46893.log.gz 2024-05-28T14:18:06.687 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40268.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40268.log.gz 2024-05-28T14:18:06.687 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-28T14:18:06.689 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46945.log 2024-05-28T14:18:06.692 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.46945.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46945.log.gz 2024-05-28T14:18:06.692 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38405.log 2024-05-28T14:18:06.695 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38405.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38405.log.gz 2024-05-28T14:18:06.695 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48091.log 2024-05-28T14:18:06.700 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36446.log 2024-05-28T14:18:06.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.48091.log: /var/log/ceph/ceph-client.admin.36446.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36446.log.gz 2024-05-28T14:18:06.700 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40398.log 2024-05-28T14:18:06.705 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55762.log 2024-05-28T14:18:06.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40398.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40398.log.gz 2024-05-28T14:18:06.708 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37418.log 2024-05-28T14:18:06.709 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55762.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55762.log.gz 2024-05-28T14:18:06.717 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56357.log 2024-05-28T14:18:06.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37418.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37418.log.gz 2024-05-28T14:18:06.724 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56357.log: 85.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39323.log 2024-05-28T14:18:06.728 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56357.log.gz 2024-05-28T14:18:06.729 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38951.log 2024-05-28T14:18:06.730 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46763.log 2024-05-28T14:18:06.730 INFO:teuthology.orchestra.run.smithi042.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39323.log.gz 2024-05-28T14:18:06.734 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46763.log: /var/log/ceph/ceph-client.admin.38951.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46763.log.gz 2024-05-28T14:18:06.735 INFO:teuthology.orchestra.run.smithi042.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38951.log.gz 2024-05-28T14:18:06.735 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34582.log 2024-05-28T14:18:06.741 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45189.log 2024-05-28T14:18:06.742 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34582.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34582.log.gz 2024-05-28T14:18:06.743 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45189.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45189.log.gz 2024-05-28T14:18:06.744 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37844.log 2024-05-28T14:18:06.750 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37844.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37844.log.gz 2024-05-28T14:18:06.752 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35644.log 2024-05-28T14:18:06.757 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54491.log 2024-05-28T14:18:06.758 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35644.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35644.log.gz 2024-05-28T14:18:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34218.log 2024-05-28T14:18:06.762 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54491.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54491.log.gz 2024-05-28T14:18:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39731.log 2024-05-28T14:18:06.764 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55630.log 2024-05-28T14:18:06.765 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34218.log.gz 2024-05-28T14:18:06.766 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39731.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39731.log.gz 2024-05-28T14:18:06.769 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43902.log 2024-05-28T14:18:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55630.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55630.log.gz 2024-05-28T14:18:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35592.log 2024-05-28T14:18:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56384.log 2024-05-28T14:18:06.773 INFO:teuthology.orchestra.run.smithi042.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43902.log.gz 2024-05-28T14:18:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35592.log: /var/log/ceph/ceph-client.admin.56384.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56384.log.gz 2024-05-28T14:18:06.780 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52787.log 2024-05-28T14:18:06.781 INFO:teuthology.orchestra.run.smithi042.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35592.log.gz 2024-05-28T14:18:06.785 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32354.log 2024-05-28T14:18:06.786 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52787.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52787.log.gz 2024-05-28T14:18:06.789 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40450.log 2024-05-28T14:18:06.790 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32354.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32354.log.gz 2024-05-28T14:18:06.791 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51429.log 2024-05-28T14:18:06.792 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45388.log 2024-05-28T14:18:06.793 INFO:teuthology.orchestra.run.smithi042.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40450.log.gz 2024-05-28T14:18:06.794 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51429.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51429.log.gz 2024-05-28T14:18:06.797 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39404.log 2024-05-28T14:18:06.798 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45388.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.45388.log.gz 2024-05-28T14:18:06.799 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56697.log 2024-05-28T14:18:06.801 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39404.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39404.log.gz 2024-05-28T14:18:06.802 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56697.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56697.log.gz 2024-05-28T14:18:06.804 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44950.log 2024-05-28T14:18:06.809 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33708.log 2024-05-28T14:18:06.810 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44950.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44950.log.gz 2024-05-28T14:18:06.813 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47304.log 2024-05-28T14:18:06.815 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33708.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.33708.log.gz 2024-05-28T14:18:06.816 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40922.log 2024-05-28T14:18:06.817 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47304.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47304.log.gz 2024-05-28T14:18:06.822 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33129.log 2024-05-28T14:18:06.822 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40922.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40922.log.gz 2024-05-28T14:18:06.825 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38609.log 2024-05-28T14:18:06.826 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33129.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33129.log.gz 2024-05-28T14:18:06.830 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38609.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38609.log.gz 2024-05-28T14:18:06.832 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47829.log 2024-05-28T14:18:06.837 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37211.log 2024-05-28T14:18:06.838 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47829.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47829.log.gz 2024-05-28T14:18:06.841 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52294.log 2024-05-28T14:18:06.843 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37211.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37211.log.gz 2024-05-28T14:18:06.844 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47226.log 2024-05-28T14:18:06.849 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52294.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52294.log.gz 2024-05-28T14:18:06.850 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47226.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47226.log.gz 2024-05-28T14:18:06.852 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43481.log 2024-05-28T14:18:06.857 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37288.log 2024-05-28T14:18:06.859 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43481.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43481.log.gz 2024-05-28T14:18:06.861 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54729.log 2024-05-28T14:18:06.862 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37288.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37288.log.gz 2024-05-28T14:18:06.864 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40130.log 2024-05-28T14:18:06.865 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48806.log 2024-05-28T14:18:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40130.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54729.log.gz 2024-05-28T14:18:06.871 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48806.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48806.log.gz 2024-05-28T14:18:06.873 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36086.log 2024-05-28T14:18:06.873 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40130.log.gz 2024-05-28T14:18:06.877 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43154.log 2024-05-28T14:18:06.878 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36086.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36086.log.gz 2024-05-28T14:18:06.881 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45111.log 2024-05-28T14:18:06.882 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43154.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43154.log.gz 2024-05-28T14:18:06.885 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42694.log 2024-05-28T14:18:06.886 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45111.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45111.log.gz 2024-05-28T14:18:06.889 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48318.log 2024-05-28T14:18:06.890 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42694.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42694.log.gz 2024-05-28T14:18:06.893 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47933.log 2024-05-28T14:18:06.894 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48318.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48318.log.gz 2024-05-28T14:18:06.897 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35800.log 2024-05-28T14:18:06.898 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47933.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47933.log.gz 2024-05-28T14:18:06.901 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36290.log 2024-05-28T14:18:06.902 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35800.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35800.log.gz 2024-05-28T14:18:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36290.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36290.log.gz 2024-05-28T14:18:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49801.log 2024-05-28T14:18:06.913 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48035.log 2024-05-28T14:18:06.914 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49801.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49801.log.gz 2024-05-28T14:18:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35037.log 2024-05-28T14:18:06.918 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48035.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48035.log.gz 2024-05-28T14:18:06.921 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44161.log 2024-05-28T14:18:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35037.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35037.log.gz 2024-05-28T14:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49319.log 2024-05-28T14:18:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44161.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44161.log.gz 2024-05-28T14:18:06.929 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45967.log 2024-05-28T14:18:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49319.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49319.log.gz 2024-05-28T14:18:06.933 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53000.log 2024-05-28T14:18:06.934 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45967.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45967.log.gz 2024-05-28T14:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52761.log 2024-05-28T14:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53000.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53000.log.gz 2024-05-28T14:18:06.942 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52761.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52761.log.gz 2024-05-28T14:18:06.944 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32930.log 2024-05-28T14:18:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36146.log 2024-05-28T14:18:06.950 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32930.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32930.log.gz 2024-05-28T14:18:06.953 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44343.log 2024-05-28T14:18:06.954 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36146.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36146.log.gz 2024-05-28T14:18:06.957 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37366.log 2024-05-28T14:18:06.958 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44343.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44343.log.gz 2024-05-28T14:18:06.961 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49241.log 2024-05-28T14:18:06.962 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37366.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37366.log.gz 2024-05-28T14:18:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53996.log 2024-05-28T14:18:06.966 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49241.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49241.log.gz 2024-05-28T14:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48966.log 2024-05-28T14:18:06.970 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53996.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53996.log.gz 2024-05-28T14:18:06.973 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36762.log 2024-05-28T14:18:06.974 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48966.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48966.log.gz 2024-05-28T14:18:06.976 INFO:teuthology.orchestra.run.smithi063.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-28T14:18:06.978 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36762.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36762.log.gz 2024-05-28T14:18:06.980 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57063.log 2024-05-28T14:18:06.985 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32721.log 2024-05-28T14:18:06.986 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57063.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57063.log.gz 2024-05-28T14:18:06.989 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51958.log 2024-05-28T14:18:06.991 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32721.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32721.log.gz 2024-05-28T14:18:06.993 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51180.log 2024-05-28T14:18:06.994 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51958.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51958.log.gz 2024-05-28T14:18:06.997 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43929.log 2024-05-28T14:18:06.998 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51180.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51180.log.gz 2024-05-28T14:18:07.001 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45084.log 2024-05-28T14:18:07.002 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43929.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43929.log.gz 2024-05-28T14:18:07.005 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36968.log 2024-05-28T14:18:07.006 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45084.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45084.log.gz 2024-05-28T14:18:07.009 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53354.log 2024-05-28T14:18:07.011 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36968.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36968.log.gz 2024-05-28T14:18:07.013 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50889.log 2024-05-28T14:18:07.014 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53354.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53354.log.gz 2024-05-28T14:18:07.018 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50889.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50889.log.gz 2024-05-28T14:18:07.020 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51741.log 2024-05-28T14:18:07.025 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48992.log 2024-05-28T14:18:07.026 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51741.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51741.log.gz 2024-05-28T14:18:07.029 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51507.log 2024-05-28T14:18:07.031 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48992.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48992.log.gz 2024-05-28T14:18:07.033 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49345.log 2024-05-28T14:18:07.035 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51507.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51507.log.gz 2024-05-28T14:18:07.037 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37982.log 2024-05-28T14:18:07.038 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49345.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49345.log.gz 2024-05-28T14:18:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45861.log 2024-05-28T14:18:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37982.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37982.log.gz 2024-05-28T14:18:07.045 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46625.log 2024-05-28T14:18:07.046 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45861.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45861.log.gz 2024-05-28T14:18:07.048 INFO:teuthology.orchestra.run.smithi063.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-28T14:18:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55949.log 2024-05-28T14:18:07.050 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46625.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46625.log.gz 2024-05-28T14:18:07.054 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55949.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55949.log.gz 2024-05-28T14:18:07.056 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32102.log 2024-05-28T14:18:07.061 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48351.log 2024-05-28T14:18:07.062 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32102.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32102.log.gz 2024-05-28T14:18:07.065 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53702.log 2024-05-28T14:18:07.067 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48351.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48351.log.gz 2024-05-28T14:18:07.069 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36500.log 2024-05-28T14:18:07.073 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53702.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53702.log.gz 2024-05-28T14:18:07.074 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41914.log 2024-05-28T14:18:07.074 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36500.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36500.log.gz 2024-05-28T14:18:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37471.log 2024-05-28T14:18:07.078 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41914.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41914.log.gz 2024-05-28T14:18:07.081 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33124.log 2024-05-28T14:18:07.082 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37471.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37471.log.gz 2024-05-28T14:18:07.085 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39491.log 2024-05-28T14:18:07.086 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33124.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33124.log.gz 2024-05-28T14:18:07.089 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44515.log 2024-05-28T14:18:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39491.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39491.log.gz 2024-05-28T14:18:07.094 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44515.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44515.log.gz 2024-05-28T14:18:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53381.log 2024-05-28T14:18:07.101 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39625.log 2024-05-28T14:18:07.102 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53381.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53381.log.gz 2024-05-28T14:18:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40607.log 2024-05-28T14:18:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39625.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39625.log.gz 2024-05-28T14:18:07.109 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35115.log 2024-05-28T14:18:07.110 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40607.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40607.log.gz 2024-05-28T14:18:07.113 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54256.log 2024-05-28T14:18:07.115 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35115.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35115.log.gz 2024-05-28T14:18:07.117 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54625.log 2024-05-28T14:18:07.118 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54256.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54256.log.gz 2024-05-28T14:18:07.121 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47803.log 2024-05-28T14:18:07.122 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54625.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54625.log.gz 2024-05-28T14:18:07.125 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46485.log 2024-05-28T14:18:07.126 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47803.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47803.log.gz 2024-05-28T14:18:07.131 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46485.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46485.log.gz 2024-05-28T14:18:07.132 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33682.log 2024-05-28T14:18:07.137 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43651.log 2024-05-28T14:18:07.138 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33682.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33682.log.gz 2024-05-28T14:18:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39057.log 2024-05-28T14:18:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43651.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43651.log.gz 2024-05-28T14:18:07.145 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49397.log 2024-05-28T14:18:07.149 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.39057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37262.log 2024-05-28T14:18:07.151 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49397.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49397.log.gz 2024-05-28T14:18:07.153 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.37262.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36237.log 2024-05-28T14:18:07.153 INFO:teuthology.orchestra.run.smithi042.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37262.log.gz 2024-05-28T14:18:07.159 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.36237.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36237.log.gz 2024-05-28T14:18:07.160 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40555.log 2024-05-28T14:18:07.165 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32706.log 2024-05-28T14:18:07.166 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40555.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40555.log.gz 2024-05-28T14:18:07.168 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39679.log 2024-05-28T14:18:07.169 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32706.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32706.log.gz 2024-05-28T14:18:07.178 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39679.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39679.log.gz 2024-05-28T14:18:07.180 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32668.log 2024-05-28T14:18:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57037.log 2024-05-28T14:18:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32668.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32668.log.gz 2024-05-28T14:18:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55656.log 2024-05-28T14:18:07.191 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57037.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57037.log.gz 2024-05-28T14:18:07.196 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55656.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38795.log 2024-05-28T14:18:07.197 INFO:teuthology.orchestra.run.smithi042.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.55656.log.gz 2024-05-28T14:18:07.201 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39221.log 2024-05-28T14:18:07.202 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38795.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38795.log.gz 2024-05-28T14:18:07.204 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43742.log 2024-05-28T14:18:07.209 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39221.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39221.log.gz 2024-05-28T14:18:07.210 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43742.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43742.log.gz 2024-05-28T14:18:07.212 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51795.log 2024-05-28T14:18:07.217 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40156.log 2024-05-28T14:18:07.219 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51795.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51795.log.gz 2024-05-28T14:18:07.222 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40156.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40156.log.gz 2024-05-28T14:18:07.224 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35774.log 2024-05-28T14:18:07.225 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53944.log 2024-05-28T14:18:07.226 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35774.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35774.log.gz 2024-05-28T14:18:07.229 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49683.log 2024-05-28T14:18:07.230 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53944.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53944.log.gz 2024-05-28T14:18:07.232 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49683.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-28T14:18:07.237 INFO:teuthology.orchestra.run.smithi042.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49683.log.gz 2024-05-28T14:18:07.237 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45414.log 2024-05-28T14:18:07.243 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.45414.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45414.log.gz 2024-05-28T14:18:07.244 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51899.log 2024-05-28T14:18:07.249 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36580.log 2024-05-28T14:18:07.250 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51899.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51899.log.gz 2024-05-28T14:18:07.253 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38556.log 2024-05-28T14:18:07.254 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36580.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36580.log.gz 2024-05-28T14:18:07.257 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34719.log 2024-05-28T14:18:07.258 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38556.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38556.log.gz 2024-05-28T14:18:07.263 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34719.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34719.log.gz 2024-05-28T14:18:07.263 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34140.log 2024-05-28T14:18:07.265 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57011.log 2024-05-28T14:18:07.266 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34140.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34140.log.gz 2024-05-28T14:18:07.270 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57011.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57011.log.gz 2024-05-28T14:18:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38583.log 2024-05-28T14:18:07.277 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27820.log 2024-05-28T14:18:07.278 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38583.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38583.log.gz 2024-05-28T14:18:07.282 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27820.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.27820.log.gz 2024-05-28T14:18:07.283 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35167.log 2024-05-28T14:18:07.284 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56567.log 2024-05-28T14:18:07.289 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35167.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35167.log.gz 2024-05-28T14:18:07.290 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56567.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56567.log.gz 2024-05-28T14:18:07.291 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34902.log 2024-05-28T14:18:07.293 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46459.log 2024-05-28T14:18:07.294 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34902.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34902.log.gz 2024-05-28T14:18:07.298 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46459.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46459.log.gz 2024-05-28T14:18:07.299 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46095.log 2024-05-28T14:18:07.300 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37499.log 2024-05-28T14:18:07.308 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46095.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46095.log.gz 2024-05-28T14:18:07.308 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56829.log 2024-05-28T14:18:07.308 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37499.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37499.log.gz 2024-05-28T14:18:07.310 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56829.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56829.log.gz 2024-05-28T14:18:07.311 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35063.log 2024-05-28T14:18:07.312 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44317.log 2024-05-28T14:18:07.313 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35063.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35063.log.gz 2024-05-28T14:18:07.318 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44317.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44317.log.gz 2024-05-28T14:18:07.318 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42224.log 2024-05-28T14:18:07.319 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43388.log 2024-05-28T14:18:07.325 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42224.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42224.log.gz 2024-05-28T14:18:07.326 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43388.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43388.log.gz 2024-05-28T14:18:07.326 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46173.log 2024-05-28T14:18:07.329 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56131.log 2024-05-28T14:18:07.330 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46173.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46173.log.gz 2024-05-28T14:18:07.333 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50131.log 2024-05-28T14:18:07.334 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56131.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56131.log.gz 2024-05-28T14:18:07.335 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53167.log 2024-05-28T14:18:07.341 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50131.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50131.log.gz 2024-05-28T14:18:07.342 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53167.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53167.log.gz 2024-05-28T14:18:07.343 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41847.log 2024-05-28T14:18:07.345 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45439.log 2024-05-28T14:18:07.349 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41847.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41847.log.gz 2024-05-28T14:18:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45439.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.45439.log.gz 2024-05-28T14:18:07.351 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53461.log 2024-05-28T14:18:07.353 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56515.log 2024-05-28T14:18:07.359 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53461.log: /var/log/ceph/ceph-client.admin.56515.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48402.log 2024-05-28T14:18:07.359 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56515.log.gz 2024-05-28T14:18:07.361 INFO:teuthology.orchestra.run.smithi042.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53461.log.gz 2024-05-28T14:18:07.361 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40372.log 2024-05-28T14:18:07.362 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48402.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48402.log.gz 2024-05-28T14:18:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35878.log 2024-05-28T14:18:07.366 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40372.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40372.log.gz 2024-05-28T14:18:07.369 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50863.log 2024-05-28T14:18:07.370 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35878.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35878.log.gz 2024-05-28T14:18:07.373 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44733.log 2024-05-28T14:18:07.377 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50863.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50863.log.gz 2024-05-28T14:18:07.378 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44733.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44733.log.gz 2024-05-28T14:18:07.379 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43024.log 2024-05-28T14:18:07.381 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47907.log 2024-05-28T14:18:07.382 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43024.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43024.log.gz 2024-05-28T14:18:07.385 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45494.log 2024-05-28T14:18:07.386 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47907.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47907.log.gz 2024-05-28T14:18:07.389 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42410.log 2024-05-28T14:18:07.390 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45494.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45494.log.gz 2024-05-28T14:18:07.394 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42410.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42410.log.gz 2024-05-28T14:18:07.396 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42833.log 2024-05-28T14:18:07.401 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39811.log 2024-05-28T14:18:07.403 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42833.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42833.log.gz 2024-05-28T14:18:07.405 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40424.log 2024-05-28T14:18:07.407 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39811.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39811.log.gz 2024-05-28T14:18:07.409 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53970.log 2024-05-28T14:18:07.411 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40424.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40424.log.gz 2024-05-28T14:18:07.413 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56488.log 2024-05-28T14:18:07.414 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53970.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53970.log.gz 2024-05-28T14:18:07.417 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43416.log 2024-05-28T14:18:07.418 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56488.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56488.log.gz 2024-05-28T14:18:07.421 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39837.log 2024-05-28T14:18:07.422 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43416.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43416.log.gz 2024-05-28T14:18:07.425 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33448.log 2024-05-28T14:18:07.426 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39837.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39837.log.gz 2024-05-28T14:18:07.430 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33448.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33448.log.gz 2024-05-28T14:18:07.432 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42250.log 2024-05-28T14:18:07.437 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48544.log 2024-05-28T14:18:07.438 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42250.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42250.log.gz 2024-05-28T14:18:07.441 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47049.log 2024-05-28T14:18:07.442 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48544.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48544.log.gz 2024-05-28T14:18:07.445 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27938.log 2024-05-28T14:18:07.447 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47049.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47049.log.gz 2024-05-28T14:18:07.449 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44813.log 2024-05-28T14:18:07.450 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27938.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27938.log.gz 2024-05-28T14:18:07.453 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45731.log 2024-05-28T14:18:07.454 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44813.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44813.log.gz 2024-05-28T14:18:07.456 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42057.log 2024-05-28T14:18:07.457 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45731.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45731.log.gz 2024-05-28T14:18:07.462 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42057.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.42057.log.gz 2024-05-28T14:18:07.464 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54048.log 2024-05-28T14:18:07.469 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55923.log 2024-05-28T14:18:07.472 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54048.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54048.log.gz 2024-05-28T14:18:07.473 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45651.log 2024-05-28T14:18:07.474 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55923.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55923.log.gz 2024-05-28T14:18:07.476 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36632.log 2024-05-28T14:18:07.482 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33110.log 2024-05-28T14:18:07.482 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45651.log.gz 2024-05-28T14:18:07.483 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36632.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36632.log.gz 2024-05-28T14:18:07.486 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33110.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33110.log.gz 2024-05-28T14:18:07.488 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39863.log 2024-05-28T14:18:07.493 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47252.log 2024-05-28T14:18:07.495 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39863.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39863.log.gz 2024-05-28T14:18:07.497 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42488.log 2024-05-28T14:18:07.499 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47252.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47252.log.gz 2024-05-28T14:18:07.501 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46841.log 2024-05-28T14:18:07.502 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42488.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42488.log.gz 2024-05-28T14:18:07.504 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54412.log 2024-05-28T14:18:07.509 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46841.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46841.log.gz 2024-05-28T14:18:07.510 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54412.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54412.log.gz 2024-05-28T14:18:07.512 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39967.log 2024-05-28T14:18:07.517 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42172.log 2024-05-28T14:18:07.518 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39967.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39967.log.gz 2024-05-28T14:18:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32704.log 2024-05-28T14:18:07.522 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42172.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42172.log.gz 2024-05-28T14:18:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38008.log 2024-05-28T14:18:07.529 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32704.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32704.log.gz 2024-05-28T14:18:07.529 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42590.log 2024-05-28T14:18:07.530 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38008.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38008.log.gz 2024-05-28T14:18:07.533 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33763.log 2024-05-28T14:18:07.535 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42590.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42590.log.gz 2024-05-28T14:18:07.537 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34850.log 2024-05-28T14:18:07.538 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33763.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33763.log.gz 2024-05-28T14:18:07.542 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34850.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34850.log.gz 2024-05-28T14:18:07.544 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56725.log 2024-05-28T14:18:07.549 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37392.log 2024-05-28T14:18:07.550 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56725.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56725.log.gz 2024-05-28T14:18:07.553 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43128.log 2024-05-28T14:18:07.554 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37392.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37392.log.gz 2024-05-28T14:18:07.556 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-28T14:18:07.557 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36684.log 2024-05-28T14:18:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.1.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43128.log.gz 2024-05-28T14:18:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48914.log 2024-05-28T14:18:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36684.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36684.log.gz 2024-05-28T14:18:07.567 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48914.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48914.log.gz 2024-05-28T14:18:07.568 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53511.log 2024-05-28T14:18:07.574 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53511.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53511.log.gz 2024-05-28T14:18:07.575 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33396.log 2024-05-28T14:18:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33138.log 2024-05-28T14:18:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33396.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33396.log.gz 2024-05-28T14:18:07.582 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33138.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33138.log.gz 2024-05-28T14:18:07.584 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51024.log 2024-05-28T14:18:07.589 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35273.log 2024-05-28T14:18:07.590 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51024.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51024.log.gz 2024-05-28T14:18:07.595 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35273.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35273.log.gz 2024-05-28T14:18:07.596 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42436.log 2024-05-28T14:18:07.602 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42436.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42436.log.gz 2024-05-28T14:18:07.603 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44033.log 2024-05-28T14:18:07.605 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48440.log 2024-05-28T14:18:07.609 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41108.log 2024-05-28T14:18:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48440.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48440.log.gz 2024-05-28T14:18:07.613 INFO:teuthology.orchestra.run.smithi042.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44033.log.gz 2024-05-28T14:18:07.615 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41108.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41108.log.gz 2024-05-28T14:18:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38242.log 2024-05-28T14:18:07.621 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40234.log 2024-05-28T14:18:07.622 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38242.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38242.log.gz 2024-05-28T14:18:07.625 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56282.log 2024-05-28T14:18:07.627 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40234.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40234.log.gz 2024-05-28T14:18:07.629 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56436.log 2024-05-28T14:18:07.631 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56282.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.56282.log.gz 2024-05-28T14:18:07.635 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.56436.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56436.log.gz 2024-05-28T14:18:07.636 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42328.log 2024-05-28T14:18:07.641 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33630.log 2024-05-28T14:18:07.642 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42328.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42328.log.gz 2024-05-28T14:18:07.645 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35566.log 2024-05-28T14:18:07.647 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33630.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33630.log.gz 2024-05-28T14:18:07.650 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35566.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35566.log.gz 2024-05-28T14:18:07.651 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32460.log 2024-05-28T14:18:07.656 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.32460.log: /var/log/ceph/ceph-client.admin.47337.log 2024-05-28T14:18:07.661 INFO:teuthology.orchestra.run.smithi042.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32460.log.gz 2024-05-28T14:18:07.661 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35089.log 2024-05-28T14:18:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47337.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47337.log.gz 2024-05-28T14:18:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34686.log 2024-05-28T14:18:07.671 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35089.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35089.log.gz 2024-05-28T14:18:07.674 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34686.log: 96.4% -- replaced with /var/log/ceph/ceph-client.0.41448.log.gz 2024-05-28T14:18:07.674 INFO:teuthology.orchestra.run.smithi042.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34686.log.gz 2024-05-28T14:18:07.675 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40043.log 2024-05-28T14:18:07.678 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45313.log 2024-05-28T14:18:07.681 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45313.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44616.log 2024-05-28T14:18:07.681 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40043.log.gz 2024-05-28T14:18:07.682 INFO:teuthology.orchestra.run.smithi042.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45313.log.gz 2024-05-28T14:18:07.684 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36420.log 2024-05-28T14:18:07.687 INFO:teuthology.orchestra.run.smithi042.stderr: 88.4%/var/log/ceph/ceph-client.admin.36420.log: -- replaced with /var/log/ceph/ceph-client.admin.44616.log.gz 2024-05-28T14:18:07.688 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43284.log 2024-05-28T14:18:07.690 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43284.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43284.log.gz 2024-05-28T14:18:07.692 INFO:teuthology.orchestra.run.smithi042.stderr:gzip 88.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.36342.log 2024-05-28T14:18:07.695 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36342.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36342.log.gz 2024-05-28T14:18:07.695 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36420.log.gz 2024-05-28T14:18:07.696 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50715.log 2024-05-28T14:18:07.698 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50715.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50715.log.gz 2024-05-28T14:18:07.698 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35482.log 2024-05-28T14:18:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35482.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35482.log.gz 2024-05-28T14:18:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52081.log 2024-05-28T14:18:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52081.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52081.log.gz 2024-05-28T14:18:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42019.log 2024-05-28T14:18:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42019.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42019.log.gz 2024-05-28T14:18:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33318.log 2024-05-28T14:18:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40737.log 2024-05-28T14:18:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33318.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33318.log.gz 2024-05-28T14:18:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38216.log 2024-05-28T14:18:07.713 INFO:teuthology.orchestra.run.smithi042.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40737.log.gz 2024-05-28T14:18:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38216.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38216.log.gz 2024-05-28T14:18:07.720 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43362.log 2024-05-28T14:18:07.725 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35670.log 2024-05-28T14:18:07.726 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43362.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43362.log.gz 2024-05-28T14:18:07.729 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50970.log 2024-05-28T14:18:07.730 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35670.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35670.log.gz 2024-05-28T14:18:07.733 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43572.log 2024-05-28T14:18:07.734 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50970.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50970.log.gz 2024-05-28T14:18:07.736 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43707.log 2024-05-28T14:18:07.741 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43572.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43572.log.gz 2024-05-28T14:18:07.742 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43707.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43707.log.gz 2024-05-28T14:18:07.744 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38112.log 2024-05-28T14:18:07.749 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32300.log 2024-05-28T14:18:07.750 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38112.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38112.log.gz 2024-05-28T14:18:07.753 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46703.log 2024-05-28T14:18:07.754 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32300.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32300.log.gz 2024-05-28T14:18:07.757 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55438.log 2024-05-28T14:18:07.758 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46703.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46703.log.gz 2024-05-28T14:18:07.760 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57167.log 2024-05-28T14:18:07.765 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55438.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55438.log.gz 2024-05-28T14:18:07.766 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57167.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57167.log.gz 2024-05-28T14:18:07.768 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38637.log 2024-05-28T14:18:07.773 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50599.log 2024-05-28T14:18:07.774 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38637.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38637.log.gz 2024-05-28T14:18:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49999.log 2024-05-28T14:18:07.778 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50599.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50599.log.gz 2024-05-28T14:18:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35325.log 2024-05-28T14:18:07.782 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49999.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49999.log.gz 2024-05-28T14:18:07.785 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39599.log 2024-05-28T14:18:07.786 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35325.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35325.log.gz 2024-05-28T14:18:07.789 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48492.log 2024-05-28T14:18:07.790 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39599.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39599.log.gz 2024-05-28T14:18:07.793 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48648.log 2024-05-28T14:18:07.794 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48492.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48492.log.gz 2024-05-28T14:18:07.797 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56881.log 2024-05-28T14:18:07.798 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48648.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48648.log.gz 2024-05-28T14:18:07.802 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56881.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56881.log.gz 2024-05-28T14:18:07.804 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54347.log 2024-05-28T14:18:07.809 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38925.log 2024-05-28T14:18:07.810 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54347.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54347.log.gz 2024-05-28T14:18:07.813 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37792.log 2024-05-28T14:18:07.814 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38925.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38925.log.gz 2024-05-28T14:18:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36211.log 2024-05-28T14:18:07.818 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37792.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37792.log.gz 2024-05-28T14:18:07.821 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51317.log 2024-05-28T14:18:07.822 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36211.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36211.log.gz 2024-05-28T14:18:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36316.log 2024-05-28T14:18:07.827 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51317.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51317.log.gz 2024-05-28T14:18:07.829 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53327.log 2024-05-28T14:18:07.830 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36316.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36316.log.gz 2024-05-28T14:18:07.833 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55201.log 2024-05-28T14:18:07.834 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53327.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53327.log.gz 2024-05-28T14:18:07.838 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55201.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55201.log.gz 2024-05-28T14:18:07.840 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54178.log 2024-05-28T14:18:07.845 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39783.log 2024-05-28T14:18:07.846 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54178.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54178.log.gz 2024-05-28T14:18:07.849 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35011.log 2024-05-28T14:18:07.850 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39783.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39783.log.gz 2024-05-28T14:18:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51873.log 2024-05-28T14:18:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35011.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35011.log.gz 2024-05-28T14:18:07.857 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55816.log 2024-05-28T14:18:07.858 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51873.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51873.log.gz 2024-05-28T14:18:07.861 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38164.log 2024-05-28T14:18:07.862 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55816.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55816.log.gz 2024-05-28T14:18:07.863 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41238.log 2024-05-28T14:18:07.865 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38164.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38164.log.gz 2024-05-28T14:18:07.870 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41238.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41238.log.gz 2024-05-28T14:18:07.872 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46303.log 2024-05-28T14:18:07.877 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55790.log 2024-05-28T14:18:07.878 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46303.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46303.log.gz 2024-05-28T14:18:07.881 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49103.log 2024-05-28T14:18:07.882 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55790.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55790.log.gz 2024-05-28T14:18:07.885 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46407.log 2024-05-28T14:18:07.886 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49103.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49103.log.gz 2024-05-28T14:18:07.889 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41548.log 2024-05-28T14:18:07.890 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46407.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46407.log.gz 2024-05-28T14:18:07.893 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50367.log 2024-05-28T14:18:07.894 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41548.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41548.log.gz 2024-05-28T14:18:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43850.log 2024-05-28T14:18:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50367.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50367.log.gz 2024-05-28T14:18:07.904 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49155.log 2024-05-28T14:18:07.904 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43850.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43850.log.gz 2024-05-28T14:18:07.905 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35456.log 2024-05-28T14:18:07.906 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49155.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49155.log.gz 2024-05-28T14:18:07.910 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35456.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35456.log.gz 2024-05-28T14:18:07.912 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40069.log 2024-05-28T14:18:07.917 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49501.log 2024-05-28T14:18:07.918 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40069.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40069.log.gz 2024-05-28T14:18:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44898.log 2024-05-28T14:18:07.922 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49501.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49501.log.gz 2024-05-28T14:18:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46147.log 2024-05-28T14:18:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44898.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44898.log.gz 2024-05-28T14:18:07.929 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54464.log 2024-05-28T14:18:07.930 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46147.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46147.log.gz 2024-05-28T14:18:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55073.log 2024-05-28T14:18:07.934 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54464.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54464.log.gz 2024-05-28T14:18:07.937 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55844.log 2024-05-28T14:18:07.938 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55073.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55073.log.gz 2024-05-28T14:18:07.941 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54282.log 2024-05-28T14:18:07.942 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55844.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55844.log.gz 2024-05-28T14:18:07.946 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54282.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54282.log.gz 2024-05-28T14:18:07.948 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36368.log 2024-05-28T14:18:07.953 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41940.log 2024-05-28T14:18:07.954 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36368.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36368.log.gz 2024-05-28T14:18:07.957 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46277.log 2024-05-28T14:18:07.958 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41940.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41940.log.gz 2024-05-28T14:18:07.961 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53814.log 2024-05-28T14:18:07.962 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46277.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46277.log.gz 2024-05-28T14:18:07.965 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35826.log 2024-05-28T14:18:07.966 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53814.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53814.log.gz 2024-05-28T14:18:07.969 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48886.log 2024-05-28T14:18:07.972 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35826.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35826.log.gz 2024-05-28T14:18:07.973 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33137.log 2024-05-28T14:18:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48886.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48886.log.gz 2024-05-28T14:18:07.977 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32916.log 2024-05-28T14:18:07.978 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33137.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33137.log.gz 2024-05-28T14:18:07.982 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32916.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32916.log.gz 2024-05-28T14:18:07.984 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51585.log 2024-05-28T14:18:07.989 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34823.log 2024-05-28T14:18:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51585.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51585.log.gz 2024-05-28T14:18:07.993 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40828.log 2024-05-28T14:18:07.994 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34823.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34823.log.gz 2024-05-28T14:18:07.997 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50393.log 2024-05-28T14:18:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40828.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40828.log.gz 2024-05-28T14:18:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49207.log 2024-05-28T14:18:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50393.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50393.log.gz 2024-05-28T14:18:08.005 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44457.log 2024-05-28T14:18:08.006 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49207.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49207.log.gz 2024-05-28T14:18:08.009 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-28T14:18:08.012 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44457.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44457.log.gz 2024-05-28T14:18:08.013 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52607.log 2024-05-28T14:18:08.019 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.52607.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52607.log.gz 2024-05-28T14:18:08.020 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45464.log 2024-05-28T14:18:08.022 INFO:teuthology.orchestra.run.smithi042.stderr: 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-28T14:18:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42781.log 2024-05-28T14:18:08.026 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45464.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45464.log.gz 2024-05-28T14:18:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45993.log 2024-05-28T14:18:08.030 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42781.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42781.log.gz 2024-05-28T14:18:08.033 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34985.log 2024-05-28T14:18:08.034 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45993.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45993.log.gz 2024-05-28T14:18:08.037 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48858.log 2024-05-28T14:18:08.038 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34985.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34985.log.gz 2024-05-28T14:18:08.041 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41574.log 2024-05-28T14:18:08.042 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48858.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48858.log.gz 2024-05-28T14:18:08.045 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34244.log 2024-05-28T14:18:08.046 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41574.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41574.log.gz 2024-05-28T14:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54022.log 2024-05-28T14:18:08.050 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34244.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34244.log.gz 2024-05-28T14:18:08.054 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54022.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54022.log.gz 2024-05-28T14:18:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40320.log 2024-05-28T14:18:08.061 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41888.log 2024-05-28T14:18:08.062 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40320.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40320.log.gz 2024-05-28T14:18:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34771.log 2024-05-28T14:18:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41888.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41888.log.gz 2024-05-28T14:18:08.069 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44007.log 2024-05-28T14:18:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34771.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34771.log.gz 2024-05-28T14:18:08.073 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56541.log 2024-05-28T14:18:08.074 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44007.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44007.log.gz 2024-05-28T14:18:08.077 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42384.log 2024-05-28T14:18:08.078 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56541.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56541.log.gz 2024-05-28T14:18:08.082 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42384.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42384.log.gz 2024-05-28T14:18:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43449.log 2024-05-28T14:18:08.089 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32538.log 2024-05-28T14:18:08.090 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43449.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43449.log.gz 2024-05-28T14:18:08.093 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53249.log 2024-05-28T14:18:08.094 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32538.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32538.log.gz 2024-05-28T14:18:08.097 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54074.log 2024-05-28T14:18:08.102 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53249.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53249.log.gz 2024-05-28T14:18:08.102 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37340.log 2024-05-28T14:18:08.108 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54074.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54074.log.gz 2024-05-28T14:18:08.108 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42462.log 2024-05-28T14:18:08.108 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37340.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37340.log.gz 2024-05-28T14:18:08.109 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54859.log 2024-05-28T14:18:08.110 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42462.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42462.log.gz 2024-05-28T14:18:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35378.log 2024-05-28T14:18:08.115 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54859.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54859.log.gz 2024-05-28T14:18:08.118 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35378.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35378.log.gz 2024-05-28T14:18:08.120 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48940.log 2024-05-28T14:18:08.125 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44111.log 2024-05-28T14:18:08.126 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48940.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48940.log.gz 2024-05-28T14:18:08.129 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54781.log 2024-05-28T14:18:08.131 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44111.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.44111.log.gz 2024-05-28T14:18:08.133 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32564.log 2024-05-28T14:18:08.134 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54781.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54781.log.gz 2024-05-28T14:18:08.137 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46651.log 2024-05-28T14:18:08.138 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32564.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32564.log.gz 2024-05-28T14:18:08.141 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37179.log 2024-05-28T14:18:08.142 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46651.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46651.log.gz 2024-05-28T14:18:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40017.log 2024-05-28T14:18:08.146 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37179.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37179.log.gz 2024-05-28T14:18:08.150 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40017.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40017.log.gz 2024-05-28T14:18:08.152 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51767.log 2024-05-28T14:18:08.157 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40633.log 2024-05-28T14:18:08.158 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51767.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51767.log.gz 2024-05-28T14:18:08.161 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32714.log 2024-05-28T14:18:08.162 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40633.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40633.log.gz 2024-05-28T14:18:08.165 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54126.log 2024-05-28T14:18:08.166 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32714.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32714.log.gz 2024-05-28T14:18:08.169 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39941.log 2024-05-28T14:18:08.170 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54126.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54126.log.gz 2024-05-28T14:18:08.173 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36472.log 2024-05-28T14:18:08.173 INFO:teuthology.orchestra.run.smithi042.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.54965.log.gz 2024-05-28T14:18:08.174 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39941.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39941.log.gz 2024-05-28T14:18:08.175 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39454.log 2024-05-28T14:18:08.177 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36472.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36472.log.gz 2024-05-28T14:18:08.180 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39454.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39454.log.gz 2024-05-28T14:18:08.180 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42919.log 2024-05-28T14:18:08.181 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45544.log 2024-05-28T14:18:08.183 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42919.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42919.log.gz 2024-05-28T14:18:08.184 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42082.log 2024-05-28T14:18:08.186 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45544.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45544.log.gz 2024-05-28T14:18:08.188 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45887.log 2024-05-28T14:18:08.189 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42082.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42082.log.gz 2024-05-28T14:18:08.192 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33126.log 2024-05-28T14:18:08.192 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45887.log.gz 2024-05-28T14:18:08.194 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40685.log 2024-05-28T14:18:08.195 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33126.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33126.log.gz 2024-05-28T14:18:08.197 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51637.log 2024-05-28T14:18:08.198 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40685.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40685.log.gz 2024-05-28T14:18:08.202 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51637.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51637.log.gz 2024-05-28T14:18:08.208 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50525.log 2024-05-28T14:18:08.209 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47174.log 2024-05-28T14:18:08.213 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.50525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52657.log 2024-05-28T14:18:08.214 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47174.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47174.log.gz 2024-05-28T14:18:08.217 INFO:teuthology.orchestra.run.smithi042.stderr:gzip 96.5% -- replaced with /var/log/ceph/ceph-client.0.48091.log.gz 2024-05-28T14:18:08.218 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44085.log 2024-05-28T14:18:08.218 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52657.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52657.log.gz 2024-05-28T14:18:08.222 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.44085.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44085.log.gz 2024-05-28T14:18:08.224 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56307.log 2024-05-28T14:18:08.229 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40503.log 2024-05-28T14:18:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56307.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56307.log.gz 2024-05-28T14:18:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37629.log 2024-05-28T14:18:08.237 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40503.log: 96.4% -- replaced with /var/log/ceph/ceph-client.0.52714.log.gz 2024-05-28T14:18:08.237 INFO:teuthology.orchestra.run.smithi042.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40503.log.gz 2024-05-28T14:18:08.237 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51559.log 2024-05-28T14:18:08.239 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37629.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37629.log.gz 2024-05-28T14:18:08.241 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54545.log 2024-05-28T14:18:08.242 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51559.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51559.log.gz 2024-05-28T14:18:08.245 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43794.log 2024-05-28T14:18:08.246 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54545.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54545.log.gz 2024-05-28T14:18:08.249 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57273.log 2024-05-28T14:18:08.250 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43794.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43794.log.gz 2024-05-28T14:18:08.253 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45809.log 2024-05-28T14:18:08.255 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57273.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57273.log.gz 2024-05-28T14:18:08.259 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45809.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45809.log.gz 2024-05-28T14:18:08.260 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43232.log 2024-05-28T14:18:08.265 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52346.log 2024-05-28T14:18:08.266 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43232.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43232.log.gz 2024-05-28T14:18:08.269 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41028.log 2024-05-28T14:18:08.270 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52346.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52346.log.gz 2024-05-28T14:18:08.273 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53221.log 2024-05-28T14:18:08.274 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41028.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41028.log.gz 2024-05-28T14:18:08.277 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51611.log 2024-05-28T14:18:08.279 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53221.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53221.log.gz 2024-05-28T14:18:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33344.log 2024-05-28T14:18:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51611.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51611.log.gz 2024-05-28T14:18:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45137.log 2024-05-28T14:18:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33344.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33344.log.gz 2024-05-28T14:18:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34530.log 2024-05-28T14:18:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45137.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45137.log.gz 2024-05-28T14:18:08.293 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42807.log 2024-05-28T14:18:08.294 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34530.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34530.log.gz 2024-05-28T14:18:08.299 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42807.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42807.log.gz 2024-05-28T14:18:08.300 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35722.log 2024-05-28T14:18:08.306 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56907.log 2024-05-28T14:18:08.306 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35722.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35722.log.gz 2024-05-28T14:18:08.309 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56777.log 2024-05-28T14:18:08.310 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56907.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56907.log.gz 2024-05-28T14:18:08.312 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51154.log 2024-05-28T14:18:08.316 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36864.log 2024-05-28T14:18:08.317 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56777.log: /var/log/ceph/ceph-client.admin.51154.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56777.log.gz 2024-05-28T14:18:08.317 INFO:teuthology.orchestra.run.smithi042.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51154.log.gz 2024-05-28T14:18:08.321 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47461.log 2024-05-28T14:18:08.322 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36864.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36864.log.gz 2024-05-28T14:18:08.325 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5/var/log/ceph/ceph-client.admin.47461.log: --verbose -- /var/log/ceph/ceph-client.admin.39889.log 2024-05-28T14:18:08.325 INFO:teuthology.orchestra.run.smithi042.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47461.log.gz 2024-05-28T14:18:08.331 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.39889.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39889.log.gz 2024-05-28T14:18:08.332 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.32434.log 2024-05-28T14:18:08.337 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56410.log 2024-05-28T14:18:08.338 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32434.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32434.log.gz 2024-05-28T14:18:08.341 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36920.log 2024-05-28T14:18:08.343 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56410.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56410.log.gz 2024-05-28T14:18:08.345 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34374.log 2024-05-28T14:18:08.349 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.36920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45757.log 2024-05-28T14:18:08.350 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34374.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34374.log.gz 2024-05-28T14:18:08.353 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48292.log 2024-05-28T14:18:08.354 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45757.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45757.log.gz 2024-05-28T14:18:08.356 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49423.log 2024-05-28T14:18:08.361 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48292.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48292.log.gz 2024-05-28T14:18:08.361 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38437.log 2024-05-28T14:18:08.362 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49423.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49423.log.gz 2024-05-28T14:18:08.366 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38437.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38437.log.gz 2024-05-28T14:18:08.368 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40896.log 2024-05-28T14:18:08.373 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45705.log 2024-05-28T14:18:08.375 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40896.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40896.log.gz 2024-05-28T14:18:08.377 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42668.log 2024-05-28T14:18:08.378 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45705.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45705.log.gz 2024-05-28T14:18:08.381 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44218.log 2024-05-28T14:18:08.383 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42668.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42668.log.gz 2024-05-28T14:18:08.385 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34452.log 2024-05-28T14:18:08.389 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.44218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37314.log 2024-05-28T14:18:08.391 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34452.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34452.log.gz 2024-05-28T14:18:08.393 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43824.log 2024-05-28T14:18:08.394 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37314.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37314.log.gz 2024-05-28T14:18:08.397 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35748.log 2024-05-28T14:18:08.398 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43824.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43824.log.gz 2024-05-28T14:18:08.402 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35748.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35748.log.gz 2024-05-28T14:18:08.404 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50943.log 2024-05-28T14:18:08.409 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33604.log 2024-05-28T14:18:08.410 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50943.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50943.log.gz 2024-05-28T14:18:08.413 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36736.log 2024-05-28T14:18:08.414 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33604.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33604.log.gz 2024-05-28T14:18:08.417 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54599.log 2024-05-28T14:18:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36736.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36736.log.gz 2024-05-28T14:18:08.421 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50469.log 2024-05-28T14:18:08.422 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54599.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54599.log.gz 2024-05-28T14:18:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47148.log 2024-05-28T14:18:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50469.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50469.log.gz 2024-05-28T14:18:08.427 INFO:teuthology.orchestra.run.smithi063.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-28T14:18:08.429 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56933.log 2024-05-28T14:18:08.430 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47148.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47148.log.gz 2024-05-28T14:18:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50778.log 2024-05-28T14:18:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56933.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56933.log.gz 2024-05-28T14:18:08.438 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50778.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50778.log.gz 2024-05-28T14:18:08.440 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55605.log 2024-05-28T14:18:08.445 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57247.log 2024-05-28T14:18:08.446 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55605.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.55605.log.gz 2024-05-28T14:18:08.449 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32713.log 2024-05-28T14:18:08.450 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57247.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57247.log.gz 2024-05-28T14:18:08.453 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52372.log 2024-05-28T14:18:08.455 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32713.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32713.log.gz 2024-05-28T14:18:08.457 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53101.log 2024-05-28T14:18:08.458 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52372.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52372.log.gz 2024-05-28T14:18:08.461 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55734.log 2024-05-28T14:18:08.462 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53101.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53101.log.gz 2024-05-28T14:18:08.465 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52055.log 2024-05-28T14:18:08.466 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55734.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55734.log.gz 2024-05-28T14:18:08.469 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49527.log 2024-05-28T14:18:08.470 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52055.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52055.log.gz 2024-05-28T14:18:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49527.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49527.log.gz 2024-05-28T14:18:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53537.log 2024-05-28T14:18:08.481 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46919.log 2024-05-28T14:18:08.482 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53537.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53537.log.gz 2024-05-28T14:18:08.485 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34106.log 2024-05-28T14:18:08.487 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46919.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46919.log.gz 2024-05-28T14:18:08.489 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36114.log 2024-05-28T14:18:08.491 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34106.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34106.log.gz 2024-05-28T14:18:08.493 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56207.log 2024-05-28T14:18:08.494 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36114.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36114.log.gz 2024-05-28T14:18:08.497 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41160.log 2024-05-28T14:18:08.498 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56207.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.56207.log.gz 2024-05-28T14:18:08.501 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32326.log 2024-05-28T14:18:08.502 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41160.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41160.log.gz 2024-05-28T14:18:08.505 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39183.log 2024-05-28T14:18:08.506 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32326.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32326.log.gz 2024-05-28T14:18:08.509 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53892.log 2024-05-28T14:18:08.511 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39183.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39183.log.gz 2024-05-28T14:18:08.514 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53892.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53892.log.gz 2024-05-28T14:18:08.514 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56671.log 2024-05-28T14:18:08.517 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34797.log 2024-05-28T14:18:08.518 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56671.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56671.log.gz 2024-05-28T14:18:08.521 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53615.log 2024-05-28T14:18:08.522 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34797.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34797.log.gz 2024-05-28T14:18:08.525 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45519.log 2024-05-28T14:18:08.526 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53615.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53615.log.gz 2024-05-28T14:18:08.529 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-28T14:18:08.530 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45519.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.45519.log.gz 2024-05-28T14:18:08.536 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49605.log 2024-05-28T14:18:08.541 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47200.log 2024-05-28T14:18:08.542 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49605.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49605.log.gz 2024-05-28T14:18:08.543 INFO:teuthology.orchestra.run.smithi042.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-28T14:18:08.545 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55047.log 2024-05-28T14:18:08.546 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47200.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47200.log.gz 2024-05-28T14:18:08.549 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35956.log 2024-05-28T14:18:08.551 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55047.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55047.log.gz 2024-05-28T14:18:08.553 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-28T14:18:08.554 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35956.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35956.log.gz 2024-05-28T14:18:08.557 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50625.log 2024-05-28T14:18:08.561 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35513.log 2024-05-28T14:18:08.562 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50625.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50625.log.gz 2024-05-28T14:18:08.568 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5/var/log/ceph/ceph-client.admin.35513.log: 87.9% --verbose -- /var/log/ceph/ceph-client.admin.55348.log 2024-05-28T14:18:08.572 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35513.log.gz 2024-05-28T14:18:08.573 INFO:teuthology.orchestra.run.smithi042.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-28T14:18:08.573 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38717.log 2024-05-28T14:18:08.575 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55348.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55348.log.gz 2024-05-28T14:18:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49892.log 2024-05-28T14:18:08.582 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38717.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38717.log.gz 2024-05-28T14:18:08.585 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52476.log 2024-05-28T14:18:08.587 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49892.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49892.log.gz 2024-05-28T14:18:08.589 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56332.log 2024-05-28T14:18:08.590 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52476.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52476.log.gz 2024-05-28T14:18:08.593 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33474.log 2024-05-28T14:18:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56332.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56332.log.gz 2024-05-28T14:18:08.598 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33474.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33474.log.gz 2024-05-28T14:18:08.600 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46199.log 2024-05-28T14:18:08.605 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-28T14:18:08.606 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46199.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46199.log.gz 2024-05-28T14:18:08.609 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34876.log 2024-05-28T14:18:08.613 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33500.log 2024-05-28T14:18:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34876.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34876.log.gz 2024-05-28T14:18:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.33500.log: -5 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33500.log.gz 2024-05-28T14:18:08.619 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44707.log 2024-05-28T14:18:08.620 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41746.log 2024-05-28T14:18:08.625 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.44707.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49129.log 2024-05-28T14:18:08.626 INFO:teuthology.orchestra.run.smithi042.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44707.log.gz 2024-05-28T14:18:08.626 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41746.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41746.log.gz 2024-05-28T14:18:08.631 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49129.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49129.log.gz 2024-05-28T14:18:08.632 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44566.log 2024-05-28T14:18:08.637 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45572.log 2024-05-28T14:18:08.638 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44566.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44566.log.gz 2024-05-28T14:18:08.641 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45598.log 2024-05-28T14:18:08.642 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45572.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45572.log.gz 2024-05-28T14:18:08.646 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45598.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45598.log.gz 2024-05-28T14:18:08.648 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48217.log 2024-05-28T14:18:08.653 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51455.log 2024-05-28T14:18:08.655 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48217.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48217.log.gz 2024-05-28T14:18:08.658 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53193.log 2024-05-28T14:18:08.658 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51455.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51455.log.gz 2024-05-28T14:18:08.661 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50079.log 2024-05-28T14:18:08.663 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53193.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53193.log.gz 2024-05-28T14:18:08.666 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.50079.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50079.log.gz 2024-05-28T14:18:08.668 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36528.log 2024-05-28T14:18:08.674 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36528.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36528.log.gz 2024-05-28T14:18:08.674 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47881.log 2024-05-28T14:18:08.677 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35852.log 2024-05-28T14:18:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42720.log 2024-05-28T14:18:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35852.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35852.log.gz 2024-05-28T14:18:08.685 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35404.log 2024-05-28T14:18:08.685 INFO:teuthology.orchestra.run.smithi042.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47881.log.gz 2024-05-28T14:18:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42720.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42720.log.gz 2024-05-28T14:18:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35404.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35404.log.gz 2024-05-28T14:18:08.692 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56003.log 2024-05-28T14:18:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57223.log 2024-05-28T14:18:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56003.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56003.log.gz 2024-05-28T14:18:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52320.log 2024-05-28T14:18:08.703 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57223.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57223.log.gz 2024-05-28T14:18:08.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52320.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52320.log.gz 2024-05-28T14:18:08.708 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41082.log 2024-05-28T14:18:08.713 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42198.log 2024-05-28T14:18:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41082.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41082.log.gz 2024-05-28T14:18:08.717 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50573.log 2024-05-28T14:18:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42198.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42198.log.gz 2024-05-28T14:18:08.721 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54677.log 2024-05-28T14:18:08.724 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50573.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50573.log.gz 2024-05-28T14:18:08.726 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54677.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54677.log.gz 2024-05-28T14:18:08.728 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39653.log 2024-05-28T14:18:08.733 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55580.log 2024-05-28T14:18:08.734 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39653.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39653.log.gz 2024-05-28T14:18:08.737 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49769.log 2024-05-28T14:18:08.739 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55580.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.55580.log.gz 2024-05-28T14:18:08.742 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49769.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49769.log.gz 2024-05-28T14:18:08.744 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56079.log 2024-05-28T14:18:08.749 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48191.log 2024-05-28T14:18:08.750 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56079.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56079.log.gz 2024-05-28T14:18:08.753 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48596.log 2024-05-28T14:18:08.755 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48191.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48191.log.gz 2024-05-28T14:18:08.758 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48596.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48596.log.gz 2024-05-28T14:18:08.759 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47699.log 2024-05-28T14:18:08.762 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47699.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47699.log.gz 2024-05-28T14:18:08.763 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49449.log 2024-05-28T14:18:08.766 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49449.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49449.log.gz 2024-05-28T14:18:08.767 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51990.log 2024-05-28T14:18:08.770 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51990.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51990.log.gz 2024-05-28T14:18:08.771 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52554.log 2024-05-28T14:18:08.774 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52554.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52554.log.gz 2024-05-28T14:18:08.775 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40763.log 2024-05-28T14:18:08.778 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40763.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40763.log.gz 2024-05-28T14:18:08.784 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53051.log 2024-05-28T14:18:08.789 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54703.log 2024-05-28T14:18:08.790 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53051.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53051.log.gz 2024-05-28T14:18:08.793 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52160.log 2024-05-28T14:18:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54703.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54703.log.gz 2024-05-28T14:18:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52160.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52134.log 2024-05-28T14:18:08.804 INFO:teuthology.orchestra.run.smithi042.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.52160.log.gz 2024-05-28T14:18:08.807 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52134.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52134.log.gz 2024-05-28T14:18:08.808 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-28T14:18:08.808 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38320.log 2024-05-28T14:18:08.813 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38268.log 2024-05-28T14:18:08.814 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38320.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38320.log.gz 2024-05-28T14:18:08.817 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48754.log 2024-05-28T14:18:08.818 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38268.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38268.log.gz 2024-05-28T14:18:08.821 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56462.log 2024-05-28T14:18:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48754.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48754.log.gz 2024-05-28T14:18:08.825 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49918.log 2024-05-28T14:18:08.826 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56462.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56462.log.gz 2024-05-28T14:18:08.829 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38769.log 2024-05-28T14:18:08.830 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49918.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49918.log.gz 2024-05-28T14:18:08.833 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34660.log 2024-05-28T14:18:08.834 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38769.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38769.log.gz 2024-05-28T14:18:08.837 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52188.log 2024-05-28T14:18:08.838 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34660.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34660.log.gz 2024-05-28T14:18:08.842 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52188.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52188.log.gz 2024-05-28T14:18:08.845 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51481.log 2024-05-28T14:18:08.849 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40948.log 2024-05-28T14:18:08.850 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51481.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51481.log.gz 2024-05-28T14:18:08.853 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47855.log 2024-05-28T14:18:08.855 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40948.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40948.log.gz 2024-05-28T14:18:08.857 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33941.log 2024-05-28T14:18:08.858 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47855.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47855.log.gz 2024-05-28T14:18:08.861 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47959.log 2024-05-28T14:18:08.862 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33941.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33941.log.gz 2024-05-28T14:18:08.865 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55516.log 2024-05-28T14:18:08.866 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47959.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47959.log.gz 2024-05-28T14:18:08.869 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51076.log 2024-05-28T14:18:08.870 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55516.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55516.log.gz 2024-05-28T14:18:08.873 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44978.log 2024-05-28T14:18:08.874 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51076.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51076.log.gz 2024-05-28T14:18:08.878 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44978.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44978.log.gz 2024-05-28T14:18:08.880 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52268.log 2024-05-28T14:18:08.885 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56029.log 2024-05-28T14:18:08.886 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52268.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52268.log.gz 2024-05-28T14:18:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52216.log 2024-05-28T14:18:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56029.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56029.log.gz 2024-05-28T14:18:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32001.log 2024-05-28T14:18:08.894 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52216.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52216.log.gz 2024-05-28T14:18:08.897 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43258.log 2024-05-28T14:18:08.898 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32001.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32001.log.gz 2024-05-28T14:18:08.901 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38743.log 2024-05-28T14:18:08.902 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43258.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43258.log.gz 2024-05-28T14:18:08.904 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51050.log 2024-05-28T14:18:08.909 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38743.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38743.log.gz 2024-05-28T14:18:08.910 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51050.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51050.log.gz 2024-05-28T14:18:08.912 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56593.log 2024-05-28T14:18:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34400.log 2024-05-28T14:18:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56593.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56593.log.gz 2024-05-28T14:18:08.921 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36008.log 2024-05-28T14:18:08.922 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34400.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34400.log.gz 2024-05-28T14:18:08.924 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47725.log 2024-05-28T14:18:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47751.log 2024-05-28T14:18:08.930 INFO:teuthology.orchestra.run.smithi042.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36008.log.gz 2024-05-28T14:18:08.930 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47725.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47725.log.gz 2024-05-28T14:18:08.933 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52942.log 2024-05-28T14:18:08.934 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47751.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47751.log.gz 2024-05-28T14:18:08.938 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52942.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52942.log.gz 2024-05-28T14:18:08.940 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51847.log 2024-05-28T14:18:08.945 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55682.log 2024-05-28T14:18:08.946 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51847.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51847.log.gz 2024-05-28T14:18:08.949 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48518.log 2024-05-28T14:18:08.950 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55682.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55682.log.gz 2024-05-28T14:18:08.953 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52580.log 2024-05-28T14:18:08.954 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48518.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48518.log.gz 2024-05-28T14:18:08.957 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38346.log 2024-05-28T14:18:08.959 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52580.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52580.log.gz 2024-05-28T14:18:08.961 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48832.log 2024-05-28T14:18:08.962 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38346.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38346.log.gz 2024-05-28T14:18:08.965 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53840.log 2024-05-28T14:18:08.966 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48832.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48832.log.gz 2024-05-28T14:18:08.969 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39353.log 2024-05-28T14:18:08.970 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53840.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53840.log.gz 2024-05-28T14:18:08.974 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39353.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39353.log.gz 2024-05-28T14:18:08.976 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44381.log 2024-05-28T14:18:08.981 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40581.log 2024-05-28T14:18:08.982 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44381.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44381.log.gz 2024-05-28T14:18:08.985 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46355.log 2024-05-28T14:18:08.986 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40581.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40581.log.gz 2024-05-28T14:18:08.988 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32382.log 2024-05-28T14:18:08.993 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50287.log 2024-05-28T14:18:08.994 INFO:teuthology.orchestra.run.smithi042.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46355.log.gz 2024-05-28T14:18:08.995 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32382.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32382.log.gz 2024-05-28T14:18:08.997 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52968.log 2024-05-28T14:18:08.998 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50287.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50287.log.gz 2024-05-28T14:18:09.000 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.52968.log: -- /var/log/ceph/ceph-client.admin.35539.log 2024-05-28T14:18:09.001 INFO:teuthology.orchestra.run.smithi042.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52968.log.gz 2024-05-28T14:18:09.007 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36814.log/var/log/ceph/ceph-client.admin.35539.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35539.log.gz 2024-05-28T14:18:09.008 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-05-28T14:18:09.013 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45338.log 2024-05-28T14:18:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36814.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.36814.log.gz 2024-05-28T14:18:09.017 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32408.log 2024-05-28T14:18:09.018 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45338.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.45338.log.gz 2024-05-28T14:18:09.020 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43955.log 2024-05-28T14:18:09.021 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32408.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32408.log.gz 2024-05-28T14:18:09.021 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53435.log 2024-05-28T14:18:09.022 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43955.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43955.log.gz 2024-05-28T14:18:09.025 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47567.log 2024-05-28T14:18:09.026 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53435.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53435.log.gz 2024-05-28T14:18:09.030 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47567.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47567.log.gz 2024-05-28T14:18:09.032 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52242.log 2024-05-28T14:18:09.037 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40659.log 2024-05-28T14:18:09.040 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52242.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52242.log.gz 2024-05-28T14:18:09.041 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32222.log 2024-05-28T14:18:09.043 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40659.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40659.log.gz 2024-05-28T14:18:09.045 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38294.log 2024-05-28T14:18:09.047 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32222.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32222.log.gz 2024-05-28T14:18:09.049 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56232.log 2024-05-28T14:18:09.050 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38294.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38294.log.gz 2024-05-28T14:18:09.053 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55021.log 2024-05-28T14:18:09.054 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56232.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56232.log.gz 2024-05-28T14:18:09.056 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50183.log 2024-05-28T14:18:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55021.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55021.log.gz 2024-05-28T14:18:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45679.log 2024-05-28T14:18:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50183.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50183.log.gz 2024-05-28T14:18:09.060 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45679.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45679.log.gz 2024-05-28T14:18:09.064 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49579.log 2024-05-28T14:18:09.069 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46069.log 2024-05-28T14:18:09.072 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49579.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49579.log.gz 2024-05-28T14:18:09.073 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54230.log 2024-05-28T14:18:09.074 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46069.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46069.log.gz 2024-05-28T14:18:09.077 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37088.log 2024-05-28T14:18:09.079 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54230.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54230.log.gz 2024-05-28T14:18:09.081 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32512.log 2024-05-28T14:18:09.082 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37088.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37088.log.gz 2024-05-28T14:18:09.085 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32486.log 2024-05-28T14:18:09.086 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32512.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32512.log.gz 2024-05-28T14:18:09.089 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34954.log 2024-05-28T14:18:09.091 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32486.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32486.log.gz 2024-05-28T14:18:09.093 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32918.log 2024-05-28T14:18:09.094 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34954.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34954.log.gz 2024-05-28T14:18:09.097 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46815.log 2024-05-28T14:18:09.098 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32918.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32918.log.gz 2024-05-28T14:18:09.099 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36394.log 2024-05-28T14:18:09.105 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40346.log 2024-05-28T14:18:09.105 INFO:teuthology.orchestra.run.smithi042.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46815.log.gz 2024-05-28T14:18:09.106 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36394.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36394.log.gz 2024-05-28T14:18:09.110 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40346.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40346.log.gz 2024-05-28T14:18:09.112 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47673.log 2024-05-28T14:18:09.117 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38847.log 2024-05-28T14:18:09.118 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47673.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47673.log.gz 2024-05-28T14:18:09.122 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54807.log 2024-05-28T14:18:09.122 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38847.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38847.log.gz 2024-05-28T14:18:09.125 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42113.log 2024-05-28T14:18:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54807.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54807.log.gz 2024-05-28T14:18:09.130 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42113.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.42113.log.gz 2024-05-28T14:18:09.132 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32193.log 2024-05-28T14:18:09.137 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51291.log 2024-05-28T14:18:09.138 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32193.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32193.log.gz 2024-05-28T14:18:09.141 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55870.log 2024-05-28T14:18:09.142 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51291.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51291.log.gz 2024-05-28T14:18:09.145 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51343.log 2024-05-28T14:18:09.146 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55870.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55870.log.gz 2024-05-28T14:18:09.149 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38372.log 2024-05-28T14:18:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51343.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51343.log.gz 2024-05-28T14:18:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43679.log 2024-05-28T14:18:09.157 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38372.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38372.log.gz 2024-05-28T14:18:09.158 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43679.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43679.log.gz 2024-05-28T14:18:09.160 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44839.log 2024-05-28T14:18:09.165 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41264.log 2024-05-28T14:18:09.166 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44839.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44839.log.gz 2024-05-28T14:18:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54152.log 2024-05-28T14:18:09.171 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41264.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41264.log.gz 2024-05-28T14:18:09.173 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34166.log 2024-05-28T14:18:09.174 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54152.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54152.log.gz 2024-05-28T14:18:09.177 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42998.log 2024-05-28T14:18:09.178 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34166.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34166.log.gz 2024-05-28T14:18:09.181 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46329.log 2024-05-28T14:18:09.182 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42998.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42998.log.gz 2024-05-28T14:18:09.185 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54517.log 2024-05-28T14:18:09.187 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46329.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46329.log.gz 2024-05-28T14:18:09.190 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54517.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54517.log.gz 2024-05-28T14:18:09.192 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38873.log 2024-05-28T14:18:09.197 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56985.log 2024-05-28T14:18:09.198 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38873.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38873.log.gz 2024-05-28T14:18:09.201 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51369.log 2024-05-28T14:18:09.202 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56985.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56985.log.gz 2024-05-28T14:18:09.204 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44654.log 2024-05-28T14:18:09.209 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54573.log 2024-05-28T14:18:09.209 INFO:teuthology.orchestra.run.smithi042.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51369.log.gz 2024-05-28T14:18:09.210 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44654.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44654.log.gz 2024-05-28T14:18:09.213 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46789.log 2024-05-28T14:18:09.214 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54573.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54573.log.gz 2024-05-28T14:18:09.216 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55175.log 2024-05-28T14:18:09.220 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46789.log: /var/log/ceph/ceph-client.admin.55175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33370.log 2024-05-28T14:18:09.221 INFO:teuthology.orchestra.run.smithi042.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46789.log.gz 2024-05-28T14:18:09.225 INFO:teuthology.orchestra.run.smithi042.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55175.log.gz 2024-05-28T14:18:09.225 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51663.log 2024-05-28T14:18:09.226 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33370.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33370.log.gz 2024-05-28T14:18:09.233 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34080.log 2024-05-28T14:18:09.234 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51663.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51663.log.gz 2024-05-28T14:18:09.237 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45288.log 2024-05-28T14:18:09.238 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34080.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34080.log.gz 2024-05-28T14:18:09.241 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42514.log 2024-05-28T14:18:09.243 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45288.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.45288.log.gz 2024-05-28T14:18:09.244 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44759.log 2024-05-28T14:18:09.246 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53918.log 2024-05-28T14:18:09.249 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44759.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42514.log.gz 2024-05-28T14:18:09.250 INFO:teuthology.orchestra.run.smithi042.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44759.log.gz 2024-05-28T14:18:09.250 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49735.log 2024-05-28T14:18:09.256 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53918.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53918.log.gz 2024-05-28T14:18:09.257 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41290.log 2024-05-28T14:18:09.258 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49735.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49735.log.gz 2024-05-28T14:18:09.261 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51395.log 2024-05-28T14:18:09.262 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41290.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41290.log.gz 2024-05-28T14:18:09.266 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51395.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51395.log.gz 2024-05-28T14:18:09.268 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53788.log 2024-05-28T14:18:09.273 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55267.log 2024-05-28T14:18:09.274 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53788.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53788.log.gz 2024-05-28T14:18:09.277 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50651.log 2024-05-28T14:18:09.278 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55267.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55267.log.gz 2024-05-28T14:18:09.281 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37551.log 2024-05-28T14:18:09.282 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50651.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50651.log.gz 2024-05-28T14:18:09.285 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43102.log 2024-05-28T14:18:09.286 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37551.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37551.log.gz 2024-05-28T14:18:09.289 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50027.log 2024-05-28T14:18:09.290 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43102.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43102.log.gz 2024-05-28T14:18:09.293 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56157.log 2024-05-28T14:18:09.294 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50027.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50027.log.gz 2024-05-28T14:18:09.297 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44681.log 2024-05-28T14:18:09.298 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56157.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56157.log.gz 2024-05-28T14:18:09.302 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44681.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44681.log.gz 2024-05-28T14:18:09.304 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46677.log 2024-05-28T14:18:09.305 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55099.log 2024-05-28T14:18:09.307 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46677.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46677.log.gz 2024-05-28T14:18:09.309 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41000.log 2024-05-28T14:18:09.310 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55099.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55099.log.gz 2024-05-28T14:18:09.313 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33130.log 2024-05-28T14:18:09.314 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41000.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41000.log.gz 2024-05-28T14:18:09.317 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46043.log 2024-05-28T14:18:09.318 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33130.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33130.log.gz 2024-05-28T14:18:09.321 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44419.log 2024-05-28T14:18:09.322 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46043.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46043.log.gz 2024-05-28T14:18:09.326 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45624.log 2024-05-28T14:18:09.326 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44419.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44419.log.gz 2024-05-28T14:18:09.329 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34426.log 2024-05-28T14:18:09.331 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45624.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45624.log.gz 2024-05-28T14:18:09.334 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34426.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34426.log.gz 2024-05-28T14:18:09.336 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55241.log 2024-05-28T14:18:09.341 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39705.log 2024-05-28T14:18:09.342 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55241.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55241.log.gz 2024-05-28T14:18:09.345 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53866.log 2024-05-28T14:18:09.346 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39705.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39705.log.gz 2024-05-28T14:18:09.347 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47369.log 2024-05-28T14:18:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34478.log 2024-05-28T14:18:09.353 INFO:teuthology.orchestra.run.smithi042.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53866.log.gz 2024-05-28T14:18:09.354 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47369.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47369.log.gz 2024-05-28T14:18:09.357 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49631.log 2024-05-28T14:18:09.359 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34478.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34478.log.gz 2024-05-28T14:18:09.363 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49631.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49631.log.gz 2024-05-28T14:18:09.364 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33422.log 2024-05-28T14:18:09.369 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49018.log 2024-05-28T14:18:09.370 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33422.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33422.log.gz 2024-05-28T14:18:09.373 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43546.log 2024-05-28T14:18:09.374 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49018.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49018.log.gz 2024-05-28T14:18:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37153.log 2024-05-28T14:18:09.379 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43546.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43546.log.gz 2024-05-28T14:18:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39001.log 2024-05-28T14:18:09.382 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37153.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37153.log.gz 2024-05-28T14:18:09.385 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50315.log 2024-05-28T14:18:09.386 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39001.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39001.log.gz 2024-05-28T14:18:09.389 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52813.log 2024-05-28T14:18:09.391 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50315.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50315.log.gz 2024-05-28T14:18:09.393 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55137.log 2024-05-28T14:18:09.394 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52813.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52813.log.gz 2024-05-28T14:18:09.398 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55137.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55137.log.gz 2024-05-28T14:18:09.400 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39915.log 2024-05-28T14:18:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46537.log 2024-05-28T14:18:09.406 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39915.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39915.log.gz 2024-05-28T14:18:09.409 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51689.log 2024-05-28T14:18:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46537.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46537.log.gz 2024-05-28T14:18:09.412 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49267.log 2024-05-28T14:18:09.413 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51689.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51689.log.gz 2024-05-28T14:18:09.417 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43180.log 2024-05-28T14:18:09.418 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49267.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49267.log.gz 2024-05-28T14:18:09.421 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38899.log 2024-05-28T14:18:09.422 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43180.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43180.log.gz 2024-05-28T14:18:09.424 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34322.log 2024-05-28T14:18:09.428 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38899.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38899.log.gz 2024-05-28T14:18:09.428 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34322.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34322.log.gz 2024-05-28T14:18:09.428 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32922.log 2024-05-28T14:18:09.433 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52107.log 2024-05-28T14:18:09.434 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32922.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32922.log.gz 2024-05-28T14:18:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33125.log 2024-05-28T14:18:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52107.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52107.log.gz 2024-05-28T14:18:09.440 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45363.log 2024-05-28T14:18:09.445 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33125.log: gzip -5 --verbose 89.5% -- /var/log/ceph/ceph-client.admin.38060.log -- replaced with /var/log/ceph/ceph-client.admin.33125.log.gz 2024-05-28T14:18:09.445 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-05-28T14:18:09.446 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45363.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.45363.log.gz 2024-05-28T14:18:09.450 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38060.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38060.log.gz 2024-05-28T14:18:09.452 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51128.log 2024-05-28T14:18:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41713.log 2024-05-28T14:18:09.458 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51128.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51128.log.gz 2024-05-28T14:18:09.461 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57115.log 2024-05-28T14:18:09.462 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41713.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41713.log.gz 2024-05-28T14:18:09.465 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41612.log 2024-05-28T14:18:09.466 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57115.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57115.log.gz 2024-05-28T14:18:09.469 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55297.log 2024-05-28T14:18:09.470 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41612.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41612.log.gz 2024-05-28T14:18:09.473 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36710.log 2024-05-28T14:18:09.474 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55297.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55297.log.gz 2024-05-28T14:18:09.477 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49068.log 2024-05-28T14:18:09.478 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36710.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36710.log.gz 2024-05-28T14:18:09.481 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48780.log 2024-05-28T14:18:09.482 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49068.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49068.log.gz 2024-05-28T14:18:09.485 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50341.log 2024-05-28T14:18:09.486 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48780.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48780.log.gz 2024-05-28T14:18:09.490 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50341.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50341.log.gz 2024-05-28T14:18:09.492 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55708.log 2024-05-28T14:18:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38691.log 2024-05-28T14:18:09.498 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55708.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55708.log.gz 2024-05-28T14:18:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38502.log 2024-05-28T14:18:09.502 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38691.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38691.log.gz 2024-05-28T14:18:09.504 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42893.log 2024-05-28T14:18:09.509 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38502.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38502.log.gz 2024-05-28T14:18:09.509 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48702.log 2024-05-28T14:18:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42893.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42893.log.gz 2024-05-28T14:18:09.513 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49971.log 2024-05-28T14:18:09.515 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48702.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48702.log.gz 2024-05-28T14:18:09.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49971.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.23010.log 2024-05-28T14:18:09.521 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35696.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49971.log.gz 2024-05-28T14:18:09.521 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-05-28T14:18:09.522 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.tmp-client.admin.23010.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.23010.log.gz 2024-05-28T14:18:09.526 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35696.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35696.log.gz 2024-05-28T14:18:09.528 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44059.log 2024-05-28T14:18:09.533 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35618.log 2024-05-28T14:18:09.534 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44059.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44059.log.gz 2024-05-28T14:18:09.537 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40095.log 2024-05-28T14:18:09.538 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35618.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35618.log.gz 2024-05-28T14:18:09.541 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32274.log 2024-05-28T14:18:09.542 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40095.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40095.log.gz 2024-05-28T14:18:09.545 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50917.log 2024-05-28T14:18:09.546 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32274.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32274.log.gz 2024-05-28T14:18:09.549 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48622.log 2024-05-28T14:18:09.550 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50917.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50917.log.gz 2024-05-28T14:18:09.553 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56803.log 2024-05-28T14:18:09.554 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48622.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48622.log.gz 2024-05-28T14:18:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53563.log 2024-05-28T14:18:09.558 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56803.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56803.log.gz 2024-05-28T14:18:09.562 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53563.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53563.log.gz 2024-05-28T14:18:09.564 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50752.log 2024-05-28T14:18:09.569 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56751.log 2024-05-28T14:18:09.570 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50752.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50752.log.gz 2024-05-28T14:18:09.573 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41316.log 2024-05-28T14:18:09.574 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56751.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56751.log.gz 2024-05-28T14:18:09.576 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41342.log 2024-05-28T14:18:09.577 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41316.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41316.log.gz 2024-05-28T14:18:09.581 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35904.log 2024-05-28T14:18:09.583 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41342.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.41342.log.gz 2024-05-28T14:18:09.585 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42755.log 2024-05-28T14:18:09.586 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35904.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35904.log.gz 2024-05-28T14:18:09.588 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38528.log 2024-05-28T14:18:09.593 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42755.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42755.log.gz 2024-05-28T14:18:09.594 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38528.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38528.log.gz 2024-05-28T14:18:09.596 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56105.log 2024-05-28T14:18:09.601 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44924.log 2024-05-28T14:18:09.603 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56105.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56105.log.gz 2024-05-28T14:18:09.605 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49293.log 2024-05-28T14:18:09.606 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44924.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44924.log.gz 2024-05-28T14:18:09.609 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43336.log 2024-05-28T14:18:09.610 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49293.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49293.log.gz 2024-05-28T14:18:09.613 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45217.log 2024-05-28T14:18:09.615 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43336.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43336.log.gz 2024-05-28T14:18:09.617 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52903.log 2024-05-28T14:18:09.618 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45217.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.45217.log.gz 2024-05-28T14:18:09.621 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36263.log 2024-05-28T14:18:09.622 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52903.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52903.log.gz 2024-05-28T14:18:09.625 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40529.log 2024-05-28T14:18:09.626 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36263.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36263.log.gz 2024-05-28T14:18:09.630 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40529.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40529.log.gz 2024-05-28T14:18:09.632 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34745.log 2024-05-28T14:18:09.637 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56619.log 2024-05-28T14:18:09.638 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34745.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34745.log.gz 2024-05-28T14:18:09.642 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56619.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56619.log.gz 2024-05-28T14:18:09.809 INFO:teuthology.orchestra.run.smithi042.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.50525.log.gz 2024-05-28T14:18:10.151 INFO:teuthology.orchestra.run.smithi042.stderr: 96.5% -- replaced with /var/log/ceph/ceph-client.0.44218.log.gz 2024-05-28T14:18:10.236 INFO:teuthology.orchestra.run.smithi063.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-28T14:18:10.563 INFO:teuthology.orchestra.run.smithi063.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-28T14:18:11.023 INFO:teuthology.orchestra.run.smithi063.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-28T14:18:11.529 INFO:teuthology.orchestra.run.smithi042.stderr: 96.5% -- replaced with /var/log/ceph/ceph-client.0.39057.log.gz 2024-05-28T14:18:11.703 INFO:teuthology.orchestra.run.smithi063.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-28T14:18:12.501 INFO:teuthology.orchestra.run.smithi063.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-28T14:18:12.503 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-05-28T14:18:12.503 INFO:teuthology.orchestra.run.smithi063.stderr:real 0m6.079s 2024-05-28T14:18:12.503 INFO:teuthology.orchestra.run.smithi063.stderr:user 0m24.922s 2024-05-28T14:18:12.503 INFO:teuthology.orchestra.run.smithi063.stderr:sys 0m1.952s 2024-05-28T14:18:12.852 INFO:teuthology.orchestra.run.smithi042.stderr: 96.5% -- replaced with /var/log/ceph/ceph-client.0.36920.log.gz 2024-05-28T14:18:13.349 INFO:teuthology.orchestra.run.smithi042.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-28T14:18:13.728 INFO:teuthology.orchestra.run.smithi042.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-28T14:18:13.969 INFO:teuthology.orchestra.run.smithi042.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-28T14:18:14.007 INFO:teuthology.orchestra.run.smithi042.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-28T14:18:14.013 INFO:teuthology.orchestra.run.smithi042.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-28T14:18:14.095 INFO:teuthology.orchestra.run.smithi042.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-28T14:18:14.097 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-05-28T14:18:14.098 INFO:teuthology.orchestra.run.smithi042.stderr:real 0m7.691s 2024-05-28T14:18:14.098 INFO:teuthology.orchestra.run.smithi042.stderr:user 0m50.663s 2024-05-28T14:18:14.098 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m4.568s 2024-05-28T14:18:14.099 INFO:tasks.ceph:Archiving logs... 2024-05-28T14:18:14.099 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074/remote/smithi042/log 2024-05-28T14:18:14.101 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-28T14:18:18.644 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074/remote/smithi063/log 2024-05-28T14:18:18.645 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-28T14:18:21.146 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-28T14:18:21.157 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-28T14:18:21.157 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-28T14:18:21.161 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-28T14:18:21.239 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-28T14:18:21.248 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-28T14:18:21.249 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-28T14:18:21.252 DEBUG:teuthology.orchestra.run.smithi063:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-28T14:18:21.265 INFO:teuthology.orchestra.run.smithi042.stdout: remote refid st t when poll reach delay offset jitter 2024-05-28T14:18:21.265 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================== 2024-05-28T14:18:21.265 INFO:teuthology.orchestra.run.smithi042.stdout:+hv01.front.sepi 67.205.162.81 3 u 3 64 377 0.099 -0.283 0.068 2024-05-28T14:18:21.265 INFO:teuthology.orchestra.run.smithi042.stdout:*hv02.front.sepi 63.231.80.2 3 u 54 64 377 0.096 +0.955 0.106 2024-05-28T14:18:21.265 INFO:teuthology.orchestra.run.smithi042.stdout:+hv03.front.sepi 74.6.168.72 3 u 62 64 377 0.096 +0.972 0.082 2024-05-28T14:18:21.266 INFO:teuthology.orchestra.run.smithi042.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-05-28T14:18:21.269 INFO:teuthology.orchestra.run.smithi063.stdout: remote refid st t when poll reach delay offset jitter 2024-05-28T14:18:21.269 INFO:teuthology.orchestra.run.smithi063.stdout:============================================================================== 2024-05-28T14:18:21.269 INFO:teuthology.orchestra.run.smithi063.stdout:+hv01.front.sepi 67.205.162.81 3 u 57 64 377 0.062 -1.227 0.034 2024-05-28T14:18:21.269 INFO:teuthology.orchestra.run.smithi063.stdout:*hv02.front.sepi 63.231.80.2 3 u 60 64 377 0.094 -0.103 0.065 2024-05-28T14:18:21.269 INFO:teuthology.orchestra.run.smithi063.stdout:+hv03.front.sepi 74.6.168.72 3 u 60 64 377 0.094 +0.099 0.043 2024-05-28T14:18:21.269 INFO:teuthology.orchestra.run.smithi063.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-05-28T14:18:21.270 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-28T14:18:21.279 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-28T14:18:21.280 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-28T14:18:21.289 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-28T14:18:21.298 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-28T14:18:21.307 INFO:teuthology.task.internal:Duration was 1921.091264 seconds 2024-05-28T14:18:21.308 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-28T14:18:21.316 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-28T14:18:21.317 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-28T14:18:21.320 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-28T14:18:21.443 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-28T14:18:21.443 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-05-28T14:18:21.443 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-28T14:18:21.464 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi063.front.sepia.ceph.com 2024-05-28T14:18:21.464 DEBUG:teuthology.orchestra.run.smithi063:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-28T14:18:21.485 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-28T14:18:21.485 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-28T14:18:21.515 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-28T14:18:21.571 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-28T14:18:21.571 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-28T14:18:21.607 DEBUG:teuthology.orchestra.run.smithi063:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-28T14:18:21.701 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-28T14:18:21.711 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-28T14:18:21.712 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-28T14:18:21.732 DEBUG:teuthology.orchestra.run.smithi063:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-28T14:18:21.753 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-28T14:18:21.763 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-28T14:18:21.778 DEBUG:teuthology.orchestra.run.smithi063:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-28T14:18:21.793 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-05-28T14:18:21.812 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = core 2024-05-28T14:18:21.831 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-28T14:18:21.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:18:21.867 DEBUG:teuthology.orchestra.run.smithi063:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-28T14:18:21.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-28T14:18:21.881 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-28T14:18:21.891 INFO:teuthology.task.internal:Transferring archived files... 2024-05-28T14:18:21.892 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074/remote/smithi042 2024-05-28T14:18:21.893 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-28T14:18:21.934 DEBUG:teuthology.misc:Transferring archived files from smithi063:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-28_13:36:18-fs:volumes-rishabh-clone-stats-testing-default-smithi/7730074/remote/smithi063 2024-05-28T14:18:21.934 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-28T14:18:21.960 INFO:teuthology.task.internal:Removing archive directory... 2024-05-28T14:18:21.960 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-28T14:18:21.974 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-28T14:18:22.007 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-28T14:18:22.018 INFO:teuthology.task.internal:Not uploading archives. 2024-05-28T14:18:22.018 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-28T14:18:22.028 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-28T14:18:22.028 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-28T14:18:22.031 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-28T14:18:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: 658124 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 28 14:18 /home/ubuntu/cephtest 2024-05-28T14:18:22.059 INFO:teuthology.orchestra.run.smithi063.stdout: 655432 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 28 14:18 /home/ubuntu/cephtest 2024-05-28T14:18:22.060 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-28T14:18:22.071 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-28T14:18:22.094 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} duration: 1921.0912644863129 failure_reason: 'Test failure: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter)' flavor: default mon.a-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 mon.b-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=57155276759542309610d688ad16fc46 status: fail success: false 2024-05-28T14:18:22.094 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-28T14:18:22.175 INFO:teuthology.run:FAIL